2024-03-28T12:55:32.632 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-28T12:55:32.633 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} --name pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi --verbose --archive /home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849 --owner scheduled_pdonnell@teuthology -- /home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849/orig.config.yaml 2024-03-28T12:55:32.680 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-28T12:55:32.762 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849 branch: wip-batrick-testing-20240327.230800 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: pdonnell@redhat.com first_in_suite: false job_id: '7627849' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240327.230800 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240327.230800 sha1: e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15 owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7752 sha1: e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15 sleep_before_teardown: 0 subset: 1/512 suite: fs suite_branch: wip-batrick-testing-20240327.230800 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLMxKcBcq8C462aleVvkrsMm3/aiMC3doyAFgs+3Pf1icWW+2G/YktWZrB+nwUbFxs1mIEpQArVQzJHRg+TjPYI= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCzV9qvdqhto5/WsJrJRzziGvwAHvJbbvVaHtiQwq7cC3HReg10YTKULIyQYtmcXB/EpldjucBTx3GNsotvFC1w= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-28_07:14:07 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-28T12:55:32.763 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15/qa; will attempt to use it 2024-03-28T12:55:32.764 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15/qa/tasks 2024-03-28T12:55:32.764 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-28T12:55:32.765 INFO:teuthology.task.internal:Checking packages... 2024-03-28T12:55:32.791 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15' 2024-03-28T12:55:32.792 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-28T12:55:32.792 INFO:teuthology.packaging:ref: None 2024-03-28T12:55:32.792 INFO:teuthology.packaging:tag: None 2024-03-28T12:55:32.792 INFO:teuthology.packaging:branch: wip-batrick-testing-20240327.230800 2024-03-28T12:55:32.792 INFO:teuthology.packaging:sha1: e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15 2024-03-28T12:55:32.792 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240327.230800 2024-03-28T12:55:33.009 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2499.ge0e48ba6 2024-03-28T12:55:33.011 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-28T12:55:33.017 INFO:teuthology.task.internal:no buildpackages task found 2024-03-28T12:55:33.017 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-28T12:55:33.023 INFO:teuthology.task.internal:Saving configuration 2024-03-28T12:55:33.053 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-28T12:55:33.059 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-28T12:55:33.085 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 12:50:39.558001', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLMxKcBcq8C462aleVvkrsMm3/aiMC3doyAFgs+3Pf1icWW+2G/YktWZrB+nwUbFxs1mIEpQArVQzJHRg+TjPYI='} 2024-03-28T12:55:33.109 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 12:50:39.559414', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCzV9qvdqhto5/WsJrJRzziGvwAHvJbbvVaHtiQwq7cC3HReg10YTKULIyQYtmcXB/EpldjucBTx3GNsotvFC1w='} 2024-03-28T12:55:33.109 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-28T12:55:33.116 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-28T12:55:33.116 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-28T12:55:33.116 INFO:teuthology.run_tasks:Running task console_log... 2024-03-28T12:55:33.187 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f24ec3d0160>, signals=[15]) 2024-03-28T12:55:33.187 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-28T12:55:33.194 INFO:teuthology.task.internal:Opening connections... 2024-03-28T12:55:33.194 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T12:55:33.196 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T12:55:33.305 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-03-28T12:55:33.307 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T12:55:33.397 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-28T12:55:33.404 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-03-28T12:55:33.421 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-03-28T12:55:33.422 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-03-28T12:55:33.479 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-03-28T12:55:33.479 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-03-28T12:55:33.480 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-03-28T12:55:33.480 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-03-28T12:55:33.480 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-03-28T12:55:33.480 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-03-28T12:55:33.480 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-28T12:55:33.480 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-03-28T12:55:33.480 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-03-28T12:55:33.481 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-28T12:55:33.481 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-03-28T12:55:33.481 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-28T12:55:33.481 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-28T12:55:33.481 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-28T12:55:33.482 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-03-28T12:55:33.508 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-03-28T12:55:33.526 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-03-28T12:55:33.526 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-03-28T12:55:33.582 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-03-28T12:55:33.583 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-03-28T12:55:33.583 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-03-28T12:55:33.583 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-03-28T12:55:33.583 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-03-28T12:55:33.583 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-03-28T12:55:33.583 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-28T12:55:33.583 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-03-28T12:55:33.583 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-03-28T12:55:33.583 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-28T12:55:33.583 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-03-28T12:55:33.583 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-28T12:55:33.583 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-28T12:55:33.584 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-28T12:55:33.584 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-03-28T12:55:33.608 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-28T12:55:33.617 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-28T12:55:33.622 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-28T12:55:33.622 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-03-28T12:55:33.625 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-03-28T12:55:33.640 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-28T12:55:33.646 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-28T12:55:33.646 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-03-28T12:55:33.682 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-03-28T12:55:33.784 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-28T12:55:33.842 INFO:teuthology.run_tasks:Running task kernel... 2024-03-28T12:55:33.858 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-03-28T12:55:33.858 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-03-28T12:55:33.858 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-03-28T12:55:33.858 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-28T12:55:33.859 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-28T12:55:33.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T12:55:33.875 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-28T12:55:33.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T12:55:33.879 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-03-28T12:55:34.075 DEBUG:teuthology.task.kernel:sha1 for client.1 is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T12:55:34.075 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-28T12:55:34.075 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-03-28T12:55:34.092 INFO:teuthology.orchestra.run.smithi188.stdout:5.14.0-361.el9.x86_64 2024-03-28T12:55:34.092 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T12:55:34.092 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-28T12:55:34.093 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum upgrade -y linux-firmware 2024-03-28T12:55:34.218 DEBUG:teuthology.task.kernel:sha1 for client.0 is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T12:55:34.218 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-28T12:55:34.218 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-03-28T12:55:34.236 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-361.el9.x86_64 2024-03-28T12:55:34.236 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T12:55:34.236 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-28T12:55:34.237 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum upgrade -y linux-firmware 2024-03-28T12:55:36.180 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-28T12:55:36.406 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-28T12:55:37.580 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-28T12:55:37.847 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2024-03-28T12:55:38.412 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.0 MB 00:00 2024-03-28T12:55:39.063 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.0 MB 00:00 2024-03-28T12:55:42.255 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 6.8 MB/s | 19 MB 00:02 2024-03-28T12:55:47.097 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.1 MB 00:01 2024-03-28T12:55:48.430 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 16 kB 00:00 2024-03-28T12:55:49.313 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 2.1 MB/s | 19 MB 00:09 2024-03-28T12:55:50.001 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-28T12:55:54.047 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.1 MB 00:01 2024-03-28T12:55:54.940 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-03-28T12:55:55.598 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 16 kB 00:00 2024-03-28T12:55:57.217 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-03-28T12:55:57.228 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-03-28T12:55:57.228 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-03-28T12:55:57.229 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-03-28T12:55:57.229 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-03-28T12:55:57.229 INFO:teuthology.orchestra.run.smithi188.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-28T12:55:57.229 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T12:55:57.229 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-03-28T12:55:57.229 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-03-28T12:55:57.229 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 1 Package 2024-03-28T12:55:57.229 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T12:55:57.229 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 375 M 2024-03-28T12:55:57.229 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-03-28T12:55:57.551 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-28T12:56:02.574 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-03-28T12:56:04.867 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-28T12:56:04.882 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-28T12:56:04.883 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-03-28T12:56:04.883 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-28T12:56:04.883 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-03-28T12:56:04.883 INFO:teuthology.orchestra.run.smithi019.stdout: linux-firmware noarch 20240219-143.el9 CentOS-BaseOS 375 M 2024-03-28T12:56:04.883 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T12:56:04.883 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-28T12:56:04.883 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-03-28T12:56:04.883 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 1 Package 2024-03-28T12:56:04.883 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T12:56:04.883 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 375 M 2024-03-28T12:56:04.883 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-28T12:56:05.698 INFO:teuthology.orchestra.run.smithi188.stdout:linux-firmware-20240219-143.el9.noarch.rpm 44 MB/s | 375 MB 00:08 2024-03-28T12:56:05.699 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-03-28T12:56:05.699 INFO:teuthology.orchestra.run.smithi188.stdout:Total 44 MB/s | 375 MB 00:08 2024-03-28T12:56:05.700 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-03-28T12:56:05.809 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-03-28T12:56:05.810 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-03-28T12:56:08.295 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-03-28T12:56:08.295 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-03-28T12:56:14.374 INFO:teuthology.orchestra.run.smithi019.stdout:linux-firmware-20240219-143.el9.noarch.rpm 40 MB/s | 375 MB 00:09 2024-03-28T12:56:14.376 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-28T12:56:14.376 INFO:teuthology.orchestra.run.smithi019.stdout:Total 39 MB/s | 375 MB 00:09 2024-03-28T12:56:14.376 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-28T12:56:14.471 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-28T12:56:14.471 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-28T12:56:14.635 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-03-28T12:56:14.890 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T12:56:14.972 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T12:56:16.538 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T12:56:16.538 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T12:56:16.901 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T12:56:16.901 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T12:56:16.901 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-03-28T12:56:16.901 INFO:teuthology.orchestra.run.smithi188.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-28T12:56:16.902 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T12:56:16.902 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-03-28T12:56:16.965 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-28T12:56:16.965 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-28T12:56:17.190 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.1... 2024-03-28T12:56:17.191 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T12:56:17.411 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-28T12:56:17.411 DEBUG:teuthology.orchestra.run.smithi188:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-28T12:56:17.414 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi188.front.sepia.ceph.com to complete... 2024-03-28T12:56:23.444 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-28T12:56:23.696 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T12:56:23.778 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T12:56:25.203 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T12:56:25.204 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : linux-firmware-20240219-143.el9.noarch 1/2 2024-03-28T12:56:25.582 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-03-28T12:56:25.582 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T12:56:25.582 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-03-28T12:56:25.582 INFO:teuthology.orchestra.run.smithi019.stdout: linux-firmware-20240219-143.el9.noarch 2024-03-28T12:56:25.583 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T12:56:25.583 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-28T12:56:25.874 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-03-28T12:56:25.875 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-28T12:56:26.022 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ 2024-03-28T12:56:26.022 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/ --input-file=- 2024-03-28T12:56:26.025 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi019.front.sepia.ceph.com to complete... 2024-03-28T12:56:32.964 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28 12:56:32 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-28T12:56:32.964 INFO:teuthology.orchestra.run.smithi188.stderr:FINISHED --2024-03-28 12:56:32-- 2024-03-28T12:56:32.965 INFO:teuthology.orchestra.run.smithi188.stderr:Total wall clock time: 16s 2024-03-28T12:56:32.965 INFO:teuthology.orchestra.run.smithi188.stderr:Downloaded: 1 files, 321M in 15s (21.1 MB/s) 2024-03-28T12:56:32.968 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.1... 2024-03-28T12:56:32.968 DEBUG:teuthology.orchestra.run.smithi188:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-28T12:56:34.897 INFO:teuthology.orchestra.run.smithi188.stdout:Verifying... ######################################## 2024-03-28T12:56:35.024 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing... ######################################## 2024-03-28T12:56:35.104 INFO:teuthology.orchestra.run.smithi188.stdout:Updating / installing... 2024-03-28T12:56:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28 12:56:41 URL:https://3.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/centos/9/flavors/default/x86_64/kernel-6.8.0_g1df0c7e8a235-1.x86_64.rpm [336317861/336317861] -> "/tmp/kernel.x86_64.rpm" [1] 2024-03-28T12:56:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:FINISHED --2024-03-28 12:56:41-- 2024-03-28T12:56:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:Total wall clock time: 15s 2024-03-28T12:56:41.502 INFO:teuthology.orchestra.run.smithi019.stderr:Downloaded: 1 files, 321M in 15s (21.5 MB/s) 2024-03-28T12:56:41.505 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on client.0... 2024-03-28T12:56:41.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-03-28T12:56:42.615 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-28T12:56:43.539 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying... ######################################## 2024-03-28T12:56:43.599 INFO:teuthology.orchestra.run.smithi019.stdout:Preparing... ######################################## 2024-03-28T12:56:43.684 INFO:teuthology.orchestra.run.smithi019.stdout:Updating / installing... 2024-03-28T12:56:51.508 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-6.8.0_g1df0c7e8a235-1 ######################################## 2024-03-28T12:56:58.882 INFO:teuthology.orchestra.run.smithi188.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-28T12:57:07.420 INFO:teuthology.orchestra.run.smithi019.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-28T12:57:21.440 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi188.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-28T12:57:21.440 DEBUG:teuthology.orchestra.run.smithi188:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-28T12:57:21.510 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-28T12:57:21.510 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-28T12:57:21.510 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-28T12:57:21.510 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-28T12:57:21.510 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-28T12:57:21.510 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-28T12:57:21.510 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-28T12:57:21.510 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-28T12:57:21.511 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-28T12:57:21.511 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-28T12:57:21.511 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-28T12:57:21.511 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-28T12:57:21.511 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-28T12:57:21.511 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-28T12:57:21.511 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-28T12:57:21.511 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-28T12:57:21.511 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-28T12:57:21.511 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-28T12:57:21.511 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-28T12:57:21.511 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-28T12:57:21.511 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-28T12:57:21.511 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-28T12:57:21.511 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-28T12:57:21.512 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-28T12:57:21.512 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-28T12:57:21.512 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-28T12:57:21.512 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-28T12:57:21.512 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-28T12:57:21.512 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-28T12:57:21.512 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-28T12:57:21.512 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-28T12:57:21.512 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-28T12:57:21.512 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-28T12:57:21.512 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-28T12:57:21.512 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-28T12:57:21.512 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-28T12:57:21.512 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-28T12:57:21.513 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-28T12:57:21.513 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-28T12:57:21.513 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-28T12:57:21.513 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-28T12:57:21.513 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-28T12:57:21.513 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-28T12:57:21.513 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-28T12:57:21.513 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-28T12:57:21.513 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-28T12:57:21.513 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-28T12:57:21.513 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-28T12:57:21.513 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-28T12:57:21.513 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-28T12:57:21.513 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-28T12:57:21.513 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-28T12:57:21.514 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-28T12:57:21.514 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-28T12:57:21.514 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-28T12:57:21.514 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-28T12:57:21.514 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-28T12:57:21.514 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-28T12:57:21.514 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-28T12:57:21.514 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-28T12:57:21.514 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-28T12:57:21.514 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-28T12:57:21.514 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-28T12:57:21.514 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-28T12:57:21.514 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-28T12:57:21.514 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-28T12:57:21.514 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-28T12:57:21.515 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-28T12:57:21.516 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-28T12:57:21.516 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-28T12:57:21.516 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-28T12:57:21.516 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-28T12:57:21.516 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-28T12:57:21.516 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-28T12:57:21.516 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-28T12:57:21.516 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-28T12:57:21.516 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-28T12:57:21.516 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-28T12:57:21.516 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-28T12:57:21.516 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-28T12:57:21.516 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-28T12:57:21.516 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-28T12:57:21.516 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-28T12:57:21.517 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-28T12:57:21.518 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-28T12:57:21.518 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-28T12:57:21.518 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-28T12:57:21.518 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-28T12:57:21.518 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-28T12:57:21.518 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-28T12:57:21.518 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-28T12:57:21.518 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-28T12:57:21.518 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-28T12:57:21.518 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-28T12:57:21.518 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-28T12:57:21.518 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-28T12:57:21.518 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-28T12:57:21.518 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-28T12:57:21.519 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-28T12:57:21.520 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-28T12:57:21.521 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-28T12:57:21.522 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-28T12:57:21.522 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-28T12:57:21.522 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-28T12:57:21.522 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-28T12:57:21.522 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-28T12:57:21.522 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-28T12:57:21.522 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-28T12:57:21.522 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-28T12:57:21.522 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-28T12:57:21.522 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-28T12:57:21.522 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-28T12:57:21.522 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-28T12:57:21.522 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-28T12:57:21.522 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-28T12:57:21.522 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-28T12:57:21.523 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-28T12:57:21.524 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-28T12:57:21.525 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-28T12:57:21.526 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-28T12:57:21.526 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-28T12:57:21.526 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-28T12:57:21.526 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-28T12:57:21.526 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-28T12:57:21.526 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-28T12:57:21.526 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-28T12:57:21.526 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-28T12:57:21.526 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-28T12:57:21.526 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-28T12:57:21.526 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-28T12:57:21.526 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-28T12:57:21.526 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-28T12:57:21.526 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-28T12:57:21.526 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-28T12:57:21.527 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-28T12:57:21.528 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-28T12:57:21.529 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-28T12:57:21.530 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-28T12:57:21.530 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-28T12:57:21.530 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-28T12:57:21.530 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-28T12:57:21.530 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-28T12:57:21.530 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-28T12:57:21.530 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-28T12:57:21.530 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-28T12:57:21.530 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-28T12:57:21.530 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-28T12:57:21.530 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-28T12:57:21.530 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-28T12:57:21.530 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-28T12:57:21.530 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-28T12:57:21.530 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-28T12:57:21.531 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-28T12:57:21.531 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-28T12:57:21.531 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-28T12:57:21.531 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-28T12:57:21.531 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-28T12:57:21.531 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-28T12:57:21.531 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-28T12:57:21.531 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-28T12:57:21.531 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-28T12:57:21.531 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-28T12:57:21.531 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-28T12:57:21.531 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-28T12:57:21.531 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-28T12:57:21.531 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-28T12:57:21.531 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-28T12:57:21.532 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-28T12:57:21.533 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-28T12:57:21.534 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-28T12:57:21.534 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-28T12:57:21.534 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-28T12:57:21.534 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-28T12:57:21.534 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-28T12:57:21.534 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-28T12:57:21.534 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-28T12:57:21.534 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-28T12:57:21.534 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-28T12:57:21.534 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-28T12:57:21.534 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-28T12:57:21.534 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-28T12:57:21.534 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-28T12:57:21.534 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-28T12:57:21.534 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-28T12:57:21.535 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-28T12:57:21.536 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-28T12:57:21.537 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-28T12:57:21.538 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-28T12:57:21.538 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-28T12:57:21.538 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-28T12:57:21.538 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-28T12:57:21.538 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-28T12:57:21.538 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-28T12:57:21.538 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-28T12:57:21.538 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-28T12:57:21.538 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-28T12:57:21.538 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-28T12:57:21.538 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-28T12:57:21.538 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-28T12:57:21.538 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-28T12:57:21.538 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-28T12:57:21.538 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-28T12:57:21.539 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-28T12:57:21.540 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-28T12:57:21.541 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-28T12:57:21.542 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-28T12:57:21.542 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-28T12:57:21.542 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-28T12:57:21.542 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-28T12:57:21.542 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-28T12:57:21.542 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-28T12:57:21.542 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-28T12:57:21.542 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-28T12:57:21.542 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-28T12:57:21.542 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-28T12:57:21.542 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-28T12:57:21.542 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-28T12:57:21.542 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-28T12:57:21.542 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-28T12:57:21.543 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-28T12:57:21.544 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-28T12:57:21.544 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-28T12:57:21.544 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-28T12:57:21.544 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-28T12:57:21.544 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-28T12:57:21.544 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-28T12:57:21.544 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-28T12:57:21.544 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-28T12:57:21.544 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-28T12:57:21.544 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-28T12:57:21.544 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-28T12:57:21.544 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-28T12:57:21.544 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-28T12:57:21.544 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-28T12:57:21.544 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-28T12:57:21.545 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-28T12:57:21.546 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-28T12:57:21.546 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-28T12:57:21.546 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-28T12:57:21.546 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-28T12:57:21.546 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-28T12:57:21.546 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-28T12:57:21.546 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-28T12:57:21.546 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-28T12:57:21.546 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-28T12:57:21.546 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-28T12:57:21.546 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-28T12:57:21.546 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-28T12:57:21.546 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-28T12:57:21.546 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-28T12:57:21.546 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-28T12:57:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-28T12:57:21.548 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-28T12:57:21.549 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-28T12:57:21.550 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-28T12:57:21.551 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-28T12:57:21.551 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-28T12:57:21.551 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-28T12:57:21.551 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-28T12:57:21.551 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-28T12:57:21.551 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-28T12:57:21.551 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-28T12:57:21.551 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-28T12:57:21.551 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-28T12:57:21.551 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-28T12:57:21.551 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-28T12:57:21.551 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-28T12:57:21.551 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-28T12:57:21.551 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-28T12:57:21.551 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-28T12:57:21.552 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-28T12:57:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-28T12:57:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-28T12:57:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-28T12:57:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-28T12:57:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-28T12:57:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-28T12:57:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-28T12:57:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-28T12:57:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-28T12:57:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-28T12:57:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-28T12:57:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-28T12:57:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-28T12:57:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-28T12:57:21.553 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-28T12:57:21.554 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-28T12:57:21.555 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-28T12:57:21.556 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-28T12:57:21.557 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-28T12:57:21.558 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-28T12:57:21.559 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-28T12:57:21.560 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-28T12:57:21.561 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-28T12:57:21.562 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-28T12:57:21.562 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-28T12:57:21.562 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-28T12:57:21.562 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-28T12:57:21.562 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-28T12:57:21.562 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-28T12:57:21.562 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-28T12:57:21.562 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-28T12:57:21.562 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-28T12:57:21.562 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-28T12:57:21.562 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-28T12:57:21.562 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-28T12:57:21.562 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-28T12:57:21.562 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-28T12:57:21.562 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-28T12:57:21.563 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-28T12:57:21.564 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-28T12:57:21.565 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-28T12:57:21.566 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-28T12:57:21.566 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-28T12:57:21.566 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-28T12:57:21.566 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-28T12:57:21.566 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-28T12:57:21.566 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-28T12:57:21.566 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-28T12:57:21.566 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-28T12:57:21.566 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-28T12:57:21.566 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-28T12:57:21.566 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-28T12:57:21.566 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-28T12:57:21.566 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-28T12:57:21.566 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-28T12:57:21.566 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-28T12:57:21.567 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-28T12:57:21.568 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-28T12:57:21.568 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-28T12:57:21.568 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-28T12:57:21.568 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-28T12:57:21.568 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-28T12:57:21.568 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-28T12:57:21.568 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-28T12:57:21.568 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-28T12:57:21.568 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-28T12:57:21.568 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-28T12:57:21.568 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-28T12:57:21.568 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-28T12:57:21.568 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-28T12:57:21.568 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-28T12:57:21.568 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-28T12:57:21.569 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-28T12:57:21.570 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-28T12:57:21.570 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-28T12:57:21.570 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-28T12:57:21.570 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-28T12:57:21.570 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-28T12:57:21.570 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-28T12:57:21.570 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-28T12:57:21.570 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-28T12:57:21.570 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-28T12:57:21.570 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-28T12:57:21.570 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-28T12:57:21.570 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-28T12:57:21.570 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-28T12:57:21.570 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-28T12:57:21.570 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-28T12:57:21.571 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-28T12:57:21.572 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-28T12:57:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-28T12:57:21.574 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-28T12:57:21.575 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-28T12:57:21.576 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-28T12:57:21.577 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-28T12:57:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-28T12:57:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-28T12:57:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-28T12:57:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-28T12:57:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-28T12:57:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-28T12:57:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-28T12:57:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-28T12:57:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-28T12:57:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-28T12:57:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-28T12:57:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-28T12:57:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-28T12:57:21.578 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-28T12:57:21.579 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-28T12:57:21.580 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-28T12:57:21.581 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-28T12:57:21.582 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-28T12:57:21.582 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-28T12:57:21.582 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-28T12:57:21.582 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-28T12:57:21.582 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-28T12:57:21.582 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-28T12:57:21.582 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-28T12:57:21.582 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-28T12:57:21.582 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-28T12:57:21.582 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-28T12:57:21.582 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-28T12:57:21.582 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-28T12:57:21.582 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-28T12:57:21.582 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-28T12:57:21.582 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-28T12:57:21.583 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-28T12:57:21.584 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-28T12:57:21.585 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-28T12:57:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-28T12:57:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-28T12:57:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-28T12:57:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-28T12:57:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-28T12:57:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-28T12:57:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-28T12:57:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-28T12:57:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-28T12:57:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-28T12:57:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-28T12:57:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-28T12:57:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-28T12:57:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-28T12:57:21.586 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-28T12:57:21.587 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-28T12:57:21.588 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-28T12:57:21.589 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-28T12:57:21.590 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-28T12:57:21.590 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-28T12:57:21.590 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-28T12:57:21.590 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-28T12:57:21.590 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-28T12:57:21.590 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-28T12:57:21.590 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-28T12:57:21.590 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-28T12:57:21.590 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-28T12:57:21.590 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-28T12:57:21.590 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-28T12:57:21.590 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-28T12:57:21.590 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-28T12:57:21.590 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-28T12:57:21.590 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-28T12:57:21.591 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-28T12:57:21.592 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-28T12:57:21.593 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-28T12:57:21.594 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-28T12:57:21.594 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-28T12:57:21.594 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-28T12:57:21.594 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-28T12:57:21.594 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-28T12:57:21.594 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-28T12:57:21.594 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-28T12:57:21.594 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-28T12:57:21.594 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-28T12:57:21.594 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-28T12:57:21.594 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-28T12:57:21.594 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-28T12:57:21.594 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-28T12:57:21.594 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-28T12:57:21.595 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-28T12:57:21.596 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-28T12:57:21.596 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-28T12:57:21.596 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-28T12:57:21.596 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-28T12:57:21.596 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-28T12:57:21.596 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-28T12:57:21.596 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-28T12:57:21.596 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-28T12:57:21.596 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-28T12:57:21.596 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-28T12:57:21.596 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-28T12:57:21.596 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-28T12:57:21.596 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-28T12:57:21.596 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-28T12:57:21.596 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-28T12:57:21.597 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-28T12:57:21.598 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-28T12:57:21.598 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-28T12:57:21.598 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-28T12:57:21.598 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-28T12:57:21.598 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-28T12:57:21.598 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-28T12:57:21.598 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-28T12:57:21.598 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-28T12:57:21.598 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-28T12:57:21.598 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-28T12:57:21.598 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-28T12:57:21.598 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-28T12:57:21.598 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-28T12:57:21.598 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-28T12:57:21.599 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-28T12:57:21.600 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-28T12:57:21.601 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-28T12:57:21.602 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-28T12:57:21.602 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-28T12:57:21.602 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-28T12:57:21.602 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-28T12:57:21.602 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-28T12:57:21.602 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-28T12:57:21.602 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-28T12:57:21.602 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-28T12:57:21.602 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-28T12:57:21.602 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-28T12:57:21.602 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-28T12:57:21.602 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-28T12:57:21.602 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-28T12:57:21.602 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-28T12:57:21.602 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-28T12:57:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-28T12:57:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-28T12:57:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-28T12:57:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-28T12:57:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-28T12:57:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-28T12:57:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-28T12:57:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-28T12:57:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-28T12:57:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-28T12:57:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-28T12:57:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-28T12:57:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-28T12:57:21.603 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-28T12:57:21.604 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-28T12:57:21.604 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-28T12:57:21.604 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-28T12:57:21.604 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-28T12:57:21.604 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-28T12:57:21.604 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-28T12:57:21.604 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-28T12:57:21.604 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-28T12:57:21.604 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-28T12:57:21.604 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-28T12:57:21.604 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-28T12:57:21.604 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-28T12:57:21.604 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-28T12:57:21.604 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-28T12:57:21.604 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-28T12:57:21.605 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-28T12:57:21.606 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-28T12:57:21.606 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-28T12:57:21.606 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-28T12:57:21.606 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-28T12:57:21.606 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-28T12:57:21.606 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-28T12:57:21.606 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-28T12:57:21.606 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-28T12:57:21.606 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-28T12:57:21.606 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-28T12:57:21.606 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-28T12:57:21.606 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-28T12:57:21.606 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-28T12:57:21.606 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-28T12:57:21.606 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-28T12:57:21.607 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-28T12:57:21.608 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-28T12:57:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-28T12:57:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-28T12:57:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-28T12:57:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-28T12:57:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-28T12:57:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-28T12:57:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-28T12:57:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-28T12:57:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-28T12:57:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-28T12:57:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-28T12:57:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-28T12:57:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-28T12:57:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-28T12:57:21.609 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-28T12:57:21.610 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-28T12:57:21.610 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-28T12:57:21.610 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-28T12:57:21.610 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-28T12:57:21.610 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-28T12:57:21.610 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-28T12:57:21.610 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-28T12:57:21.610 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-28T12:57:21.610 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-28T12:57:21.610 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-28T12:57:21.610 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-28T12:57:21.610 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-28T12:57:21.610 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-28T12:57:21.610 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-28T12:57:21.610 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-28T12:57:21.611 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-28T12:57:21.612 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-28T12:57:21.612 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-28T12:57:21.612 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-28T12:57:21.612 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-28T12:57:21.612 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-28T12:57:21.612 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-28T12:57:21.612 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-28T12:57:21.612 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-28T12:57:21.612 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-28T12:57:21.612 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-28T12:57:21.612 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-28T12:57:21.612 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-28T12:57:21.612 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-28T12:57:21.612 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-28T12:57:21.612 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-28T12:57:21.613 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-28T12:57:21.614 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-28T12:57:21.614 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-28T12:57:21.614 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-28T12:57:21.614 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-28T12:57:21.614 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-28T12:57:21.614 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-28T12:57:21.614 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-28T12:57:21.614 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-28T12:57:21.614 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-28T12:57:21.614 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-28T12:57:21.614 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-28T12:57:21.614 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-28T12:57:21.614 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-28T12:57:21.614 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-28T12:57:21.614 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-28T12:57:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-28T12:57:21.616 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-28T12:57:21.617 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-28T12:57:21.618 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-28T12:57:21.618 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-28T12:57:21.618 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-28T12:57:21.618 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-28T12:57:21.618 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-28T12:57:21.618 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-28T12:57:21.618 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-28T12:57:21.618 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-28T12:57:21.618 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-28T12:57:21.618 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-28T12:57:21.618 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-28T12:57:21.618 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-28T12:57:21.618 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-28T12:57:21.618 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-28T12:57:21.618 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-28T12:57:21.619 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-28T12:57:21.620 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-28T12:57:21.620 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-28T12:57:21.620 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-28T12:57:21.620 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-28T12:57:21.620 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-28T12:57:21.620 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-28T12:57:21.620 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-28T12:57:21.620 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-28T12:57:21.620 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-28T12:57:21.620 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-28T12:57:21.620 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-28T12:57:21.620 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-28T12:57:21.620 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-28T12:57:21.620 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-28T12:57:21.620 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-28T12:57:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-28T12:57:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-28T12:57:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-28T12:57:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-28T12:57:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-28T12:57:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-28T12:57:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-28T12:57:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-28T12:57:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-28T12:57:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-28T12:57:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-28T12:57:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-28T12:57:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-28T12:57:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-28T12:57:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-28T12:57:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-28T12:57:21.623 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-28T12:57:21.624 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-28T12:57:21.624 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-28T12:57:21.624 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-28T12:57:21.624 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-28T12:57:21.624 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-28T12:57:21.624 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-28T12:57:21.624 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-28T12:57:21.624 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-28T12:57:21.624 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-28T12:57:21.624 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-28T12:57:21.624 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-28T12:57:21.624 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-28T12:57:21.624 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-28T12:57:21.624 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-28T12:57:21.625 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-28T12:57:21.626 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-28T12:57:21.626 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-28T12:57:21.626 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-28T12:57:21.626 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-28T12:57:21.626 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-28T12:57:21.626 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-28T12:57:21.626 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-28T12:57:21.626 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-28T12:57:21.626 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-28T12:57:21.626 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-28T12:57:21.626 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-28T12:57:21.626 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-28T12:57:21.626 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-28T12:57:21.626 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-28T12:57:21.626 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-28T12:57:21.627 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-28T12:57:21.628 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-28T12:57:21.629 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-28T12:57:21.630 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-28T12:57:21.630 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-28T12:57:21.630 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-28T12:57:21.630 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-28T12:57:21.630 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-28T12:57:21.630 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-28T12:57:21.630 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-28T12:57:21.630 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-28T12:57:21.630 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-28T12:57:21.630 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-28T12:57:21.630 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-28T12:57:21.630 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-28T12:57:21.630 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-28T12:57:21.630 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-28T12:57:21.631 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-28T12:57:21.631 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-28T12:57:21.631 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-28T12:57:21.631 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-28T12:57:21.631 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-28T12:57:21.631 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-28T12:57:21.631 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-28T12:57:21.631 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-28T12:57:21.631 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-28T12:57:21.631 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-28T12:57:21.631 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-28T12:57:21.631 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-28T12:57:21.631 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-28T12:57:21.631 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-28T12:57:21.631 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-28T12:57:21.632 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-28T12:57:21.632 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-28T12:57:21.632 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-28T12:57:21.632 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-28T12:57:21.632 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-28T12:57:21.632 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T12:57:21.636 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-28T12:57:21.636 DEBUG:teuthology.orchestra.run.smithi188:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-28T12:57:21.658 INFO:teuthology.orchestra.run.smithi188.stdout:postinstall scriptlet (using /bin/sh): 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout:fi 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout:for file in vmlinuz System.map config; do 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout: fi 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout:done 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout:fi 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-28T12:57:21.659 INFO:teuthology.orchestra.run.smithi188.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-28T12:57:21.660 INFO:teuthology.orchestra.run.smithi188.stdout:fi 2024-03-28T12:57:21.660 DEBUG:teuthology.orchestra.run.smithi188:> sudo rpm -qi grub2-tools 2024-03-28T12:57:21.745 INFO:teuthology.orchestra.run.smithi188.stdout:Name : grub2-tools 2024-03-28T12:57:21.745 INFO:teuthology.orchestra.run.smithi188.stdout:Epoch : 1 2024-03-28T12:57:21.745 INFO:teuthology.orchestra.run.smithi188.stdout:Version : 2.06 2024-03-28T12:57:21.745 INFO:teuthology.orchestra.run.smithi188.stdout:Release : 68.el9 2024-03-28T12:57:21.745 INFO:teuthology.orchestra.run.smithi188.stdout:Architecture: x86_64 2024-03-28T12:57:21.745 INFO:teuthology.orchestra.run.smithi188.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-28T12:57:21.745 INFO:teuthology.orchestra.run.smithi188.stdout:Group : Unspecified 2024-03-28T12:57:21.745 INFO:teuthology.orchestra.run.smithi188.stdout:Size : 8264254 2024-03-28T12:57:21.745 INFO:teuthology.orchestra.run.smithi188.stdout:License : GPLv3+ 2024-03-28T12:57:21.745 INFO:teuthology.orchestra.run.smithi188.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-28T12:57:21.745 INFO:teuthology.orchestra.run.smithi188.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-28T12:57:21.745 INFO:teuthology.orchestra.run.smithi188.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-28T12:57:21.745 INFO:teuthology.orchestra.run.smithi188.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-28T12:57:21.745 INFO:teuthology.orchestra.run.smithi188.stdout:Packager : builder@centos.org 2024-03-28T12:57:21.745 INFO:teuthology.orchestra.run.smithi188.stdout:Vendor : CentOS 2024-03-28T12:57:21.746 INFO:teuthology.orchestra.run.smithi188.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-28T12:57:21.746 INFO:teuthology.orchestra.run.smithi188.stdout:Summary : Support tools for GRUB. 2024-03-28T12:57:21.746 INFO:teuthology.orchestra.run.smithi188.stdout:Description : 2024-03-28T12:57:21.746 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T12:57:21.746 INFO:teuthology.orchestra.run.smithi188.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-28T12:57:21.746 INFO:teuthology.orchestra.run.smithi188.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-28T12:57:21.746 INFO:teuthology.orchestra.run.smithi188.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-28T12:57:21.746 INFO:teuthology.orchestra.run.smithi188.stdout:hardware devices. 2024-03-28T12:57:21.746 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T12:57:21.746 INFO:teuthology.orchestra.run.smithi188.stdout:This subpackage provides tools for support of all platforms. 2024-03-28T12:57:21.747 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-28T12:57:21.747 INFO:teuthology.task.kernel:Updating grub on smithi188 to boot 6.8.0-g1df0c7e8a235 2024-03-28T12:57:21.747 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-28T12:57:22.329 INFO:teuthology.orchestra.run.smithi188.stderr:Generating grub configuration file ... 2024-03-28T12:57:23.666 INFO:teuthology.orchestra.run.smithi188.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-28T12:57:23.703 INFO:teuthology.orchestra.run.smithi188.stderr:done 2024-03-28T12:57:23.711 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-03-28T12:57:23.727 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.xk75ykIDlg 2024-03-28T12:57:23.727 DEBUG:teuthology.orchestra.run.smithi188:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xk75ykIDlg 2024-03-28T12:57:23.827 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.xk75ykIDlg 2024-03-28T12:57:23.927 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.xk75ykIDlg is 6KB 2024-03-28T12:57:23.946 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.xk75ykIDlg 2024-03-28T12:57:23.960 DEBUG:teuthology.orchestra.run.smithi188:> sudo /bin/ls /boot/loader/entries || true 2024-03-28T12:57:24.033 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-28T12:57:24.033 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-28T12:57:24.034 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-28T12:57:24.034 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-28T12:57:24.035 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-28T12:57:24.204 DEBUG:teuthology.orchestra.run.smithi188:> sudo shutdown -r now 2024-03-28T12:57:30.543 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi019.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-03-28T12:57:30.543 DEBUG:teuthology.orchestra.run.smithi019:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-03-28T12:57:30.612 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235 2024-03-28T12:57:30.612 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/System.map 2024-03-28T12:57:30.612 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/config 2024-03-28T12:57:30.612 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel 2024-03-28T12:57:30.612 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch 2024-03-28T12:57:30.612 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86 2024-03-28T12:57:30.612 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto 2024-03-28T12:57:30.612 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-28T12:57:30.612 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-28T12:57:30.612 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-28T12:57:30.612 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-28T12:57:30.612 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-28T12:57:30.612 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32c-intel.ko 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel 2024-03-28T12:57:30.613 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu 2024-03-28T12:57:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce 2024-03-28T12:57:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-28T12:57:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm 2024-03-28T12:57:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-28T12:57:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-28T12:57:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-28T12:57:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto 2024-03-28T12:57:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-28T12:57:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-28T12:57:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-28T12:57:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx 2024-03-28T12:57:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-28T12:57:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-28T12:57:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-28T12:57:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-28T12:57:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-28T12:57:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-28T12:57:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-28T12:57:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-28T12:57:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-28T12:57:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-28T12:57:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-28T12:57:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-28T12:57:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-28T12:57:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-28T12:57:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-28T12:57:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-28T12:57:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-28T12:57:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-28T12:57:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-28T12:57:30.616 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/custom_method.ko 2024-03-28T12:57:30.617 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ata_generic.ko 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-28T12:57:30.618 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-28T12:57:30.619 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sis.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-28T12:57:30.620 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-28T12:57:30.621 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base 2024-03-28T12:57:30.622 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap 2024-03-28T12:57:30.622 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-i2c.ko 2024-03-28T12:57:30.622 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/base/regmap/regmap-mmio.ko 2024-03-28T12:57:30.622 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma 2024-03-28T12:57:30.622 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-28T12:57:30.622 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block 2024-03-28T12:57:30.622 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe 2024-03-28T12:57:30.622 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-28T12:57:30.622 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/brd.ko 2024-03-28T12:57:30.622 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd 2024-03-28T12:57:30.622 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-28T12:57:30.622 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-28T12:57:30.622 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/loop.ko 2024-03-28T12:57:30.622 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx 2024-03-28T12:57:30.622 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/virtio_blk.ko 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkfront.ko 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-28T12:57:30.623 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-28T12:57:30.624 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm.ko 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis.ko 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/uv_mmtimer.ko 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/virtio_console.ko 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/powernow-k8.ko 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto 2024-03-28T12:57:30.625 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq 2024-03-28T12:57:30.626 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-28T12:57:30.627 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-28T12:57:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-28T12:57:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-28T12:57:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-28T12:57:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware 2024-03-28T12:57:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/edd.ko 2024-03-28T12:57:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-28T12:57:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio 2024-03-28T12:57:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-28T12:57:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-28T12:57:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid 2024-03-28T12:57:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-28T12:57:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-28T12:57:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-28T12:57:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-28T12:57:30.628 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-28T12:57:30.629 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-28T12:57:30.630 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-28T12:57:30.631 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-28T12:57:30.631 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-28T12:57:30.631 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-28T12:57:30.631 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-28T12:57:30.631 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-28T12:57:30.631 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-28T12:57:30.631 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-28T12:57:30.631 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-28T12:57:30.631 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-28T12:57:30.631 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-28T12:57:30.631 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-28T12:57:30.631 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-28T12:57:30.631 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-28T12:57:30.631 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-28T12:57:30.631 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-28T12:57:30.632 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-28T12:57:30.633 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-28T12:57:30.634 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-28T12:57:30.635 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-28T12:57:30.636 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-28T12:57:30.637 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-28T12:57:30.637 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-28T12:57:30.637 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-28T12:57:30.637 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-28T12:57:30.637 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-28T12:57:30.637 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-28T12:57:30.637 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-28T12:57:30.637 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-28T12:57:30.637 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-28T12:57:30.637 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-28T12:57:30.637 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-28T12:57:30.637 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-28T12:57:30.637 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-28T12:57:30.637 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-28T12:57:30.637 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-28T12:57:30.638 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-28T12:57:30.638 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-28T12:57:30.638 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-28T12:57:30.638 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-28T12:57:30.638 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-28T12:57:30.638 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-28T12:57:30.638 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-28T12:57:30.638 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-28T12:57:30.638 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-28T12:57:30.638 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus 2024-03-28T12:57:30.638 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-28T12:57:30.638 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-28T12:57:30.638 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-28T12:57:30.638 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-28T12:57:30.638 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-28T12:57:30.639 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-28T12:57:30.640 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-28T12:57:30.641 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-28T12:57:30.642 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-dev.ko 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4 2024-03-28T12:57:30.643 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt 2024-03-28T12:57:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-28T12:57:30.645 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger 2024-03-28T12:57:30.646 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-28T12:57:30.647 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-28T12:57:30.648 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mfd-core.ko 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-28T12:57:30.649 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-28T12:57:30.650 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-gru/gru.ko 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xp.ko 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpc.ko 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-28T12:57:30.651 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet 2024-03-28T12:57:30.652 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera 2024-03-28T12:57:30.653 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-28T12:57:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd 2024-03-28T12:57:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-28T12:57:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-28T12:57:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-28T12:57:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros 2024-03-28T12:57:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx 2024-03-28T12:57:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-28T12:57:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c 2024-03-28T12:57:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-28T12:57:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e 2024-03-28T12:57:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-28T12:57:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx 2024-03-28T12:57:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-28T12:57:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-28T12:57:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-03-28T12:57:30.655 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-28T12:57:30.656 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-28T12:57:30.657 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell 2024-03-28T12:57:30.658 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi 2024-03-28T12:57:30.659 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc 2024-03-28T12:57:30.660 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc 2024-03-28T12:57:30.661 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-28T12:57:30.662 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-28T12:57:30.663 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/net_failover.ko 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-28T12:57:30.664 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/dp83640.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-28T12:57:30.665 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_generic.ko 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slhc.ko 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-28T12:57:30.666 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tun.ko 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/virtio_net.ko 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback 2024-03-28T12:57:30.667 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-28T12:57:30.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netfront.ko 2024-03-28T12:57:30.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb 2024-03-28T12:57:30.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-28T12:57:30.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme 2024-03-28T12:57:30.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host 2024-03-28T12:57:30.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-28T12:57:30.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-28T12:57:30.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport 2024-03-28T12:57:30.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-28T12:57:30.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-28T12:57:30.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-28T12:57:30.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-28T12:57:30.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci 2024-03-28T12:57:30.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug 2024-03-28T12:57:30.668 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pcie/aer_inject.ko 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-28T12:57:30.669 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-28T12:57:30.670 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-28T12:57:30.671 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-28T12:57:30.672 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-28T12:57:30.673 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-28T12:57:30.674 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-28T12:57:30.675 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-28T12:57:30.676 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-28T12:57:30.677 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-28T12:57:30.678 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx 2024-03-28T12:57:30.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-28T12:57:30.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-28T12:57:30.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx 2024-03-28T12:57:30.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-28T12:57:30.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-28T12:57:30.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-28T12:57:30.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-28T12:57:30.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-28T12:57:30.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-28T12:57:30.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-28T12:57:30.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-28T12:57:30.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-28T12:57:30.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-28T12:57:30.679 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-28T12:57:30.680 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-28T12:57:30.681 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250 2024-03-28T12:57:30.682 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb 2024-03-28T12:57:30.683 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-28T12:57:30.684 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-28T12:57:30.685 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-28T12:57:30.686 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-28T12:57:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-28T12:57:30.688 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci 2024-03-28T12:57:30.689 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio.ko 2024-03-28T12:57:30.690 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_balloon.ko 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_mmio.ko 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci.ko 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/virtio/virtio_ring.ko 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-28T12:57:30.691 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-28T12:57:30.692 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-28T12:57:30.693 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-acpi-processor.ko 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-28T12:57:30.694 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs 2024-03-28T12:57:30.695 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ecryptfs/ecryptfs.ko 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/fat.ko 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/vfat.ko 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/fuse.ko 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs 2024-03-28T12:57:30.696 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-28T12:57:30.697 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-28T12:57:30.698 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-28T12:57:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-28T12:57:30.700 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-28T12:57:30.701 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs 2024-03-28T12:57:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-28T12:57:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-28T12:57:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-28T12:57:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-28T12:57:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-28T12:57:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs 2024-03-28T12:57:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-28T12:57:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore 2024-03-28T12:57:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-28T12:57:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs 2024-03-28T12:57:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-28T12:57:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs 2024-03-28T12:57:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-28T12:57:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb 2024-03-28T12:57:30.702 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client 2024-03-28T12:57:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-28T12:57:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common 2024-03-28T12:57:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-28T12:57:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-28T12:57:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs 2024-03-28T12:57:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-28T12:57:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv 2024-03-28T12:57:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-28T12:57:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf 2024-03-28T12:57:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-28T12:57:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs 2024-03-28T12:57:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-28T12:57:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs 2024-03-28T12:57:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-28T12:57:30.703 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/rcu/rcutorture.ko 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/asn1_encoder.ko 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-28T12:57:30.704 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/zstd/zstd_compress.ko 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-28T12:57:30.705 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p 2024-03-28T12:57:30.706 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-28T12:57:30.707 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-28T12:57:30.708 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/failover.ko 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dns_resolver/dns_resolver.ko 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-28T12:57:30.709 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-28T12:57:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-28T12:57:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154 2024-03-28T12:57:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan 2024-03-28T12:57:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-28T12:57:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-28T12:57:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-28T12:57:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4 2024-03-28T12:57:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-28T12:57:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-28T12:57:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-28T12:57:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-28T12:57:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-28T12:57:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-28T12:57:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-28T12:57:30.710 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-28T12:57:30.711 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-28T12:57:30.712 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-28T12:57:30.713 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-28T12:57:30.714 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp 2024-03-28T12:57:30.715 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-28T12:57:30.716 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-28T12:57:30.717 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-28T12:57:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-28T12:57:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-28T12:57:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-28T12:57:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-28T12:57:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-28T12:57:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-28T12:57:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-28T12:57:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-28T12:57:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-28T12:57:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-28T12:57:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-28T12:57:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-28T12:57:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-28T12:57:30.718 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-28T12:57:30.719 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-28T12:57:30.720 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_quota.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tunnel.ko 2024-03-28T12:57:30.721 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-28T12:57:30.722 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-28T12:57:30.723 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-28T12:57:30.724 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-28T12:57:30.725 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-28T12:57:30.726 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds 2024-03-28T12:57:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-28T12:57:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-28T12:57:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-28T12:57:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched 2024-03-28T12:57:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-28T12:57:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-28T12:57:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-28T12:57:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-28T12:57:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-28T12:57:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-28T12:57:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-28T12:57:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-28T12:57:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-28T12:57:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-28T12:57:30.727 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-28T12:57:30.728 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-28T12:57:30.729 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock 2024-03-28T12:57:30.730 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/security/keys/trusted-keys/trusted.ko 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-28T12:57:30.731 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-28T12:57:30.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-28T12:57:30.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-28T12:57:30.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-28T12:57:30.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-28T12:57:30.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-28T12:57:30.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-28T12:57:30.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-28T12:57:30.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-28T12:57:30.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-28T12:57:30.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-28T12:57:30.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-28T12:57:30.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-28T12:57:30.732 INFO:teuthology.orchestra.run.smithi019.stdout:/lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T12:57:30.735 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-28T12:57:30.736 DEBUG:teuthology.orchestra.run.smithi019:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-03-28T12:57:30.758 INFO:teuthology.orchestra.run.smithi019.stdout:postinstall scriptlet (using /bin/sh): 2024-03-28T12:57:30.758 INFO:teuthology.orchestra.run.smithi019.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-03-28T12:57:30.758 INFO:teuthology.orchestra.run.smithi019.stdout: /usr/bin/kernel-install add 6.8.0-g1df0c7e8a235 /lib/modules/6.8.0-g1df0c7e8a235/vmlinuz 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout:fi 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout:for file in vmlinuz System.map config; do 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout: if ! cmp --silent "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235"; then 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout: cp "/lib/modules/6.8.0-g1df0c7e8a235/${file}" "/boot/${file}-6.8.0-g1df0c7e8a235" 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout: fi 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout:done 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout:preuninstall scriptlet (using /bin/sh): 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout:new-kernel-pkg --remove 6.8.0-g1df0c7e8a235 --rminitrd --initrdfile=/boot/initramfs-6.8.0-g1df0c7e8a235.img 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-install remove 6.8.0-g1df0c7e8a235 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout:fi 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout:postuninstall scriptlet (using /bin/sh): 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout:if [ -x /sbin/update-bootloader ]; then 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/update-bootloader --remove 6.8.0-g1df0c7e8a235 2024-03-28T12:57:30.759 INFO:teuthology.orchestra.run.smithi019.stdout:fi 2024-03-28T12:57:30.760 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -qi grub2-tools 2024-03-28T12:57:30.852 INFO:teuthology.orchestra.run.smithi019.stdout:Name : grub2-tools 2024-03-28T12:57:30.852 INFO:teuthology.orchestra.run.smithi019.stdout:Epoch : 1 2024-03-28T12:57:30.852 INFO:teuthology.orchestra.run.smithi019.stdout:Version : 2.06 2024-03-28T12:57:30.852 INFO:teuthology.orchestra.run.smithi019.stdout:Release : 68.el9 2024-03-28T12:57:30.852 INFO:teuthology.orchestra.run.smithi019.stdout:Architecture: x86_64 2024-03-28T12:57:30.852 INFO:teuthology.orchestra.run.smithi019.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-28T12:57:30.852 INFO:teuthology.orchestra.run.smithi019.stdout:Group : Unspecified 2024-03-28T12:57:30.852 INFO:teuthology.orchestra.run.smithi019.stdout:Size : 8264254 2024-03-28T12:57:30.852 INFO:teuthology.orchestra.run.smithi019.stdout:License : GPLv3+ 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout:Packager : builder@centos.org 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout:Vendor : CentOS 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout:Summary : Support tools for GRUB. 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout:Description : 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout:hardware devices. 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T12:57:30.853 INFO:teuthology.orchestra.run.smithi019.stdout:This subpackage provides tools for support of all platforms. 2024-03-28T12:57:30.854 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-28T12:57:30.854 INFO:teuthology.task.kernel:Updating grub on smithi019 to boot 6.8.0-g1df0c7e8a235 2024-03-28T12:57:30.854 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-28T12:57:31.427 INFO:teuthology.orchestra.run.smithi019.stderr:Generating grub configuration file ... 2024-03-28T12:57:32.810 INFO:teuthology.orchestra.run.smithi019.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-28T12:57:32.830 INFO:teuthology.orchestra.run.smithi019.stderr:done 2024-03-28T12:57:32.841 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-03-28T12:57:32.856 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.ybhKWr8aB1 2024-03-28T12:57:32.856 DEBUG:teuthology.orchestra.run.smithi019:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ybhKWr8aB1 2024-03-28T12:57:32.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.ybhKWr8aB1 2024-03-28T12:57:33.067 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.ybhKWr8aB1 is 6KB 2024-03-28T12:57:33.117 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.ybhKWr8aB1 2024-03-28T12:57:33.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo /bin/ls /boot/loader/entries || true 2024-03-28T12:57:33.208 INFO:teuthology.orchestra.run.smithi019.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-28T12:57:33.208 INFO:teuthology.orchestra.run.smithi019.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-28T12:57:33.208 INFO:teuthology.orchestra.run.smithi019.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-28T12:57:33.208 INFO:teuthology.orchestra.run.smithi019.stdout:15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235.conf 2024-03-28T12:57:33.210 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.8.0-g1df0c7e8a235 2024-03-28T12:57:33.390 DEBUG:teuthology.orchestra.run.smithi019:> sudo shutdown -r now 2024-03-28T12:57:54.226 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-03-28T12:57:54.227 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-03-28T12:57:54.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T12:58:03.398 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-28T12:58:03.399 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-28T12:58:03.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T12:58:21.855 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-03-28T12:58:28.766 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2024-03-28T12:58:30.858 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-28T12:58:30.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T12:58:37.774 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-03-28T12:58:37.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T12:58:56.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2024-03-28T12:59:08.171 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-03-28T12:59:08.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T12:59:11.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2024-03-28T12:59:26.244 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-03-28T12:59:26.245 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T12:59:26.602 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-03-28T12:59:27.379 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-03-28T12:59:27.379 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.8.0-g1df0c7e8a235"... 2024-03-28T12:59:27.379 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-03-28T12:59:27.415 INFO:teuthology.orchestra.run.smithi188.stdout:6.8.0-g1df0c7e8a235 2024-03-28T12:59:27.415 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-28T12:59:27.415 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-28T12:59:27.415 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-28T12:59:28.417 DEBUG:teuthology.parallel:result is None 2024-03-28T12:59:30.861 DEBUG:teuthology.orchestra.remote:timed out 2024-03-28T12:59:42.862 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-28T12:59:42.863 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T12:59:43.337 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-03-28T12:59:44.029 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-28T12:59:44.029 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.8.0-g1df0c7e8a235"... 2024-03-28T12:59:44.029 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-03-28T12:59:44.081 INFO:teuthology.orchestra.run.smithi019.stdout:6.8.0-g1df0c7e8a235 2024-03-28T12:59:44.081 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-28T12:59:44.081 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-28T12:59:44.081 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-28T12:59:45.082 DEBUG:teuthology.parallel:result is None 2024-03-28T12:59:45.083 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-28T12:59:45.095 INFO:teuthology.task.internal:Creating test directory... 2024-03-28T12:59:45.095 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-28T12:59:45.102 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-28T12:59:45.158 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-28T12:59:45.171 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-28T12:59:45.194 INFO:teuthology.task.internal:Creating archive directory... 2024-03-28T12:59:45.195 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-28T12:59:45.202 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-28T12:59:45.276 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-28T12:59:45.282 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-28T12:59:45.282 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-28T12:59:45.287 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-28T12:59:45.425 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T12:59:45.473 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T12:59:45.495 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T12:59:45.545 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-28T12:59:45.550 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-28T12:59:45.558 INFO:teuthology.task.internal:Configuring sudo... 2024-03-28T12:59:45.558 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-28T12:59:45.565 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-28T12:59:45.664 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-28T12:59:45.674 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-28T12:59:45.674 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-28T12:59:45.681 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-28T12:59:45.738 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T12:59:45.821 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T12:59:45.932 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T12:59:46.014 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T12:59:46.109 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T12:59:46.109 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-28T12:59:46.208 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T12:59:46.294 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-28T12:59:46.398 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T12:59:46.482 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-28T12:59:46.580 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T12:59:46.580 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-28T12:59:46.673 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-03-28T12:59:46.682 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-03-28T12:59:46.805 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T12:59:46.844 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T12:59:47.335 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-28T12:59:47.344 INFO:teuthology.task.internal:Starting timer... 2024-03-28T12:59:47.344 INFO:teuthology.run_tasks:Running task pcp... 2024-03-28T12:59:47.359 INFO:teuthology.run_tasks:Running task selinux... 2024-03-28T12:59:47.368 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-03-28T12:59:47.368 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-03-28T12:59:47.494 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-03-28T12:59:47.502 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2024-03-28T12:59:47.632 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2024-03-28T12:59:47.638 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-28T12:59:47.638 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-03-28T12:59:47.698 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-03-28T12:59:47.699 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2024-03-28T12:59:47.778 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2024-03-28T12:59:47.778 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2024-03-28T12:59:47.779 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T12:59:47.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T12:59:47.857 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T12:59:47.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T12:59:47.942 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-28T12:59:47.942 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-03-28T12:59:48.036 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2024-03-28T12:59:48.114 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-28T12:59:48.123 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-28T12:59:48.129 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-28T12:59:48.263 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-28T12:59:48.278 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-28T12:59:48.280 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-28T13:05:55.795 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-03-28T13:05:55.797 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-28T13:05:55.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T13:05:55.952 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-03-28T13:05:56.107 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-03-28T13:05:56.107 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-03-28T13:05:56.108 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-28T13:05:56.246 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-03-28T13:05:56.397 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-03-28T13:05:56.398 INFO:teuthology.run_tasks:Running task clock... 2024-03-28T13:05:56.409 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-28T13:05:56.410 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-28T13:05:56.410 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T13:05:56.417 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-28T13:05:56.418 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T13:05:56.517 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-28T13:05:56.519 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-28T13:05:56.570 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-28T13:05:56.574 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-28T13:05:56.662 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2024-03-28T13:05:56.676 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-03-28T13:05:56.703 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2024-03-28T13:05:56.721 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-03-28T13:05:56.753 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-28T13:05:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-28T13:05:56.807 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-28T13:05:56.838 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-28T13:05:56.935 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-03-28T13:05:56.942 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T13:05:56.942 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-03-28T13:05:56.942 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T13:05:56.942 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T13:05:56.942 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T13:05:56.942 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T13:05:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-03-28T13:05:56.984 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T13:05:56.984 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-03-28T13:05:56.984 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T13:05:56.984 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T13:05:56.985 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T13:05:56.985 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T13:05:56.986 INFO:teuthology.run_tasks:Running task install... 2024-03-28T13:05:56.995 DEBUG:teuthology.task.install:project ceph 2024-03-28T13:05:56.995 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15'}} 2024-03-28T13:05:56.995 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15'} 2024-03-28T13:05:56.995 INFO:teuthology.task.install:Using flavor: default 2024-03-28T13:05:57.001 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-28T13:05:57.001 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-28T13:05:57.002 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15', 'tag': None, 'wait_for_package': False} 2024-03-28T13:05:57.002 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15 2024-03-28T13:05:57.004 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15', 'tag': None, 'wait_for_package': False} 2024-03-28T13:05:57.005 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15 2024-03-28T13:05:57.146 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240327.230800/e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15/centos/9/flavors/default/ 2024-03-28T13:05:57.146 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2499.ge0e48ba6 2024-03-28T13:05:57.222 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240327.230800/e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15/centos/9/flavors/default/ 2024-03-28T13:05:57.222 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2499.ge0e48ba6 2024-03-28T13:05:57.262 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240327.230800/e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240327.230800/e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240327.230800/e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-28T13:05:57.262 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:05:57.262 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-28T13:05:57.375 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-28T13:05:57.375 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-28T13:05:57.396 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240327.230800/e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240327.230800/e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240327.230800/e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-28T13:05:57.396 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:05:57.396 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-28T13:05:57.490 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-28T13:05:57.490 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-28T13:05:57.521 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-28T13:05:57.629 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-28T13:05:57.659 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-28T13:05:57.755 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-03-28T13:05:57.760 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-03-28T13:05:57.763 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-28T13:05:57.851 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2024-03-28T13:05:57.856 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2024-03-28T13:05:58.136 INFO:teuthology.orchestra.run.smithi019.stdout:74 files removed 2024-03-28T13:05:58.176 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-28T13:05:58.219 INFO:teuthology.orchestra.run.smithi188.stdout:74 files removed 2024-03-28T13:05:58.257 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-28T13:05:59.245 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 169 kB/s | 83 kB 00:00 2024-03-28T13:05:59.324 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 133 kB/s | 83 kB 00:00 2024-03-28T13:05:59.796 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 25 kB/s | 11 kB 00:00 2024-03-28T13:06:00.083 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 17 kB/s | 11 kB 00:00 2024-03-28T13:06:00.246 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-03-28T13:06:00.600 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-03-28T13:06:00.825 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-28T13:06:01.203 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-03-28T13:06:02.517 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.0 MB 00:00 2024-03-28T13:06:02.863 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS-9 - BaseOS 5.6 MB/s | 2.0 MB 00:00 2024-03-28T13:06:03.487 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-03-28T13:06:05.430 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-03-28T13:06:10.014 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.1 MB 00:00 2024-03-28T13:06:11.598 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 16 kB 00:00 2024-03-28T13:06:12.274 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 915 kB/s | 8.0 MB 00:08 2024-03-28T13:06:13.264 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-28T13:06:18.991 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-28T13:06:21.621 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T13:06:21.622 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T13:06:21.642 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-28T13:06:21.644 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-28T13:06:21.645 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-28T13:06:21.646 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-28T13:06:21.649 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-28T13:06:21.652 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-28T13:06:21.653 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-28T13:06:21.743 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-03-28T13:06:21.764 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2024-03-28T13:06:21.764 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-03-28T13:06:21.764 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2024-03-28T13:06:21.765 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-03-28T13:06:21.765 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-28T13:06:21.765 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 6.6 k 2024-03-28T13:06:21.765 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 5.4 M 2024-03-28T13:06:21.765 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 890 k 2024-03-28T13:06:21.765 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 145 k 2024-03-28T13:06:21.765 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 1.5 M 2024-03-28T13:06:21.765 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 141 k 2024-03-28T13:06:21.765 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 3.5 M 2024-03-28T13:06:21.765 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 7.4 M 2024-03-28T13:06:21.765 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 49 k 2024-03-28T13:06:21.765 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 9.8 M 2024-03-28T13:06:21.765 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 48 M 2024-03-28T13:06:21.766 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 264 k 2024-03-28T13:06:21.766 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 762 k 2024-03-28T13:06:21.766 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 221 k 2024-03-28T13:06:21.766 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 27 k 2024-03-28T13:06:21.766 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 18 k 2024-03-28T13:06:21.766 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-28T13:06:21.766 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-28T13:06:21.766 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-28T13:06:21.766 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 32 k 2024-03-28T13:06:21.766 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 746 k 2024-03-28T13:06:21.766 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 127 k 2024-03-28T13:06:21.766 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-28T13:06:21.766 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-28T13:06:21.766 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-28T13:06:21.767 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-28T13:06:21.767 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-28T13:06:21.767 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-28T13:06:21.767 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 162 k 2024-03-28T13:06:21.767 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 320 k 2024-03-28T13:06:21.767 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 298 k 2024-03-28T13:06:21.767 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 99 k 2024-03-28T13:06:21.767 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 85 k 2024-03-28T13:06:21.767 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 3.1 M 2024-03-28T13:06:21.767 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 170 k 2024-03-28T13:06:21.767 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-28T13:06:21.767 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-03-28T13:06:21.767 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-28T13:06:21.768 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-28T13:06:21.768 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-28T13:06:21.768 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-28T13:06:21.768 INFO:teuthology.orchestra.run.smithi188.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-28T13:06:21.768 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-28T13:06:21.768 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-28T13:06:21.768 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-28T13:06:21.768 INFO:teuthology.orchestra.run.smithi188.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-28T13:06:21.768 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-28T13:06:21.768 INFO:teuthology.orchestra.run.smithi188.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-28T13:06:21.768 INFO:teuthology.orchestra.run.smithi188.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-28T13:06:21.768 INFO:teuthology.orchestra.run.smithi188.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-28T13:06:21.768 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 3.4 M 2024-03-28T13:06:21.769 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 3.1 M 2024-03-28T13:06:21.769 INFO:teuthology.orchestra.run.smithi188.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-28T13:06:21.769 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-28T13:06:21.769 INFO:teuthology.orchestra.run.smithi188.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-28T13:06:21.769 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-28T13:06:21.769 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-28T13:06:21.769 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-28T13:06:21.769 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-28T13:06:21.769 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-28T13:06:21.769 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-28T13:06:21.769 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-03-28T13:06:21.769 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-28T13:06:21.769 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-28T13:06:21.770 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-28T13:06:21.770 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 21 M 2024-03-28T13:06:21.770 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 34 k 2024-03-28T13:06:21.770 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 2.4 M 2024-03-28T13:06:21.770 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 249 k 2024-03-28T13:06:21.770 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 4.7 M 2024-03-28T13:06:21.770 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 17 M 2024-03-28T13:06:21.770 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 16 k 2024-03-28T13:06:21.770 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 25 k 2024-03-28T13:06:21.770 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-28T13:06:21.770 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-28T13:06:21.770 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-28T13:06:21.770 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-28T13:06:21.771 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-28T13:06:21.771 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-28T13:06:21.771 INFO:teuthology.orchestra.run.smithi188.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-28T13:06:21.771 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-28T13:06:21.771 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-28T13:06:21.771 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 163 k 2024-03-28T13:06:21.771 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-28T13:06:21.771 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-28T13:06:21.771 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-28T13:06:21.771 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-28T13:06:21.771 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-28T13:06:21.771 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 501 k 2024-03-28T13:06:21.771 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-28T13:06:21.772 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 5.1 M 2024-03-28T13:06:21.772 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-28T13:06:21.772 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-28T13:06:21.772 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-28T13:06:21.772 INFO:teuthology.orchestra.run.smithi188.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-28T13:06:21.772 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-28T13:06:21.772 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-28T13:06:21.772 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-28T13:06:21.772 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-28T13:06:21.772 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-28T13:06:21.772 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-28T13:06:21.772 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-28T13:06:21.773 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-28T13:06:21.773 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-28T13:06:21.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-28T13:06:21.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-28T13:06:21.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-28T13:06:21.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-28T13:06:21.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-28T13:06:21.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 45 k 2024-03-28T13:06:21.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 134 k 2024-03-28T13:06:21.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-28T13:06:21.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-28T13:06:21.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-28T13:06:21.773 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-28T13:06:21.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-28T13:06:21.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-28T13:06:21.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-28T13:06:21.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-28T13:06:21.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-28T13:06:21.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-28T13:06:21.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-28T13:06:21.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-28T13:06:21.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-28T13:06:21.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-28T13:06:21.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-28T13:06:21.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-28T13:06:21.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-28T13:06:21.774 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-28T13:06:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-28T13:06:21.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-28T13:06:21.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-28T13:06:21.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-28T13:06:21.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-28T13:06:21.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-28T13:06:21.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-28T13:06:21.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-28T13:06:21.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-28T13:06:21.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-28T13:06:21.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-28T13:06:21.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-28T13:06:21.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-28T13:06:21.776 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-28T13:06:21.777 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-28T13:06:21.777 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-28T13:06:21.777 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-28T13:06:21.777 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-28T13:06:21.777 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-28T13:06:21.777 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-28T13:06:21.777 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-28T13:06:21.777 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-28T13:06:21.777 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-28T13:06:21.777 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-28T13:06:21.777 INFO:teuthology.orchestra.run.smithi188.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-28T13:06:21.777 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-28T13:06:21.777 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-28T13:06:21.777 INFO:teuthology.orchestra.run.smithi188.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-28T13:06:21.777 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-28T13:06:21.778 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-28T13:06:21.778 INFO:teuthology.orchestra.run.smithi188.stdout:Installing weak dependencies: 2024-03-28T13:06:21.778 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-28T13:06:21.778 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:06:21.778 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-03-28T13:06:21.778 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2024-03-28T13:06:21.778 INFO:teuthology.orchestra.run.smithi188.stdout:Install 150 Packages 2024-03-28T13:06:21.778 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 24 Packages 2024-03-28T13:06:21.778 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:06:21.779 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 227 M 2024-03-28T13:06:21.779 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-03-28T13:06:22.417 INFO:teuthology.orchestra.run.smithi188.stdout:(1/174): ceph-19.0.0-2499.ge0e48ba6.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-03-28T13:06:23.108 INFO:teuthology.orchestra.run.smithi188.stdout:(2/174): ceph-fuse-19.0.0-2499.ge0e48ba6.el9.x8 1.3 MB/s | 890 kB 00:00 2024-03-28T13:06:23.224 INFO:teuthology.orchestra.run.smithi188.stdout:(3/174): ceph-immutable-object-cache-19.0.0-249 1.2 MB/s | 145 kB 00:00 2024-03-28T13:06:23.751 INFO:teuthology.orchestra.run.smithi188.stdout:(4/174): ceph-mds-19.0.0-2499.ge0e48ba6.el9.x86 4.5 MB/s | 2.4 MB 00:00 2024-03-28T13:06:23.984 INFO:teuthology.orchestra.run.smithi188.stdout:(5/174): ceph-mgr-19.0.0-2499.ge0e48ba6.el9.x86 6.4 MB/s | 1.5 MB 00:00 2024-03-28T13:06:24.559 INFO:teuthology.orchestra.run.smithi188.stdout:(6/174): ceph-mon-19.0.0-2499.ge0e48ba6.el9.x86 8.2 MB/s | 4.7 MB 00:00 2024-03-28T13:06:25.018 INFO:teuthology.orchestra.run.smithi188.stdout:(7/174): ceph-common-19.0.0-2499.ge0e48ba6.el9. 7.3 MB/s | 21 MB 00:02 2024-03-28T13:06:25.426 INFO:teuthology.orchestra.run.smithi188.stdout:(8/174): ceph-base-19.0.0-2499.ge0e48ba6.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-03-28T13:06:25.535 INFO:teuthology.orchestra.run.smithi188.stdout:(9/174): ceph-selinux-19.0.0-2499.ge0e48ba6.el9 234 kB/s | 25 kB 00:00 2024-03-28T13:06:25.877 INFO:teuthology.orchestra.run.smithi188.stdout:(10/174): ceph-radosgw-19.0.0-2499.ge0e48ba6.el 11 MB/s | 9.8 MB 00:00 2024-03-28T13:06:25.977 INFO:teuthology.orchestra.run.smithi188.stdout:(11/174): cephfs-mirror-19.0.0-2499.ge0e48ba6.e 2.2 MB/s | 221 kB 00:00 2024-03-28T13:06:26.236 INFO:teuthology.orchestra.run.smithi188.stdout:(12/174): ceph-osd-19.0.0-2499.ge0e48ba6.el9.x8 10 MB/s | 17 MB 00:01 2024-03-28T13:06:26.260 INFO:teuthology.orchestra.run.smithi188.stdout:(13/174): cephfs-shell-19.0.0-2499.ge0e48ba6.el 96 kB/s | 27 kB 00:00 2024-03-28T13:06:26.286 INFO:teuthology.orchestra.run.smithi188.stdout:(14/174): libcephfs-devel-19.0.0-2499.ge0e48ba6 648 kB/s | 32 kB 00:00 2024-03-28T13:06:26.353 INFO:teuthology.orchestra.run.smithi188.stdout:(15/174): libcephsqlite-19.0.0-2499.ge0e48ba6.e 2.4 MB/s | 163 kB 00:00 2024-03-28T13:06:26.420 INFO:teuthology.orchestra.run.smithi188.stdout:(16/174): libcephfs2-19.0.0-2499.ge0e48ba6.el9. 4.6 MB/s | 746 kB 00:00 2024-03-28T13:06:26.453 INFO:teuthology.orchestra.run.smithi188.stdout:(17/174): librados-devel-19.0.0-2499.ge0e48ba6. 1.2 MB/s | 127 kB 00:00 2024-03-28T13:06:26.537 INFO:teuthology.orchestra.run.smithi188.stdout:(18/174): libradosstriper1-19.0.0-2499.ge0e48ba 4.2 MB/s | 501 kB 00:00 2024-03-28T13:06:26.630 INFO:teuthology.orchestra.run.smithi188.stdout:(19/174): python3-ceph-argparse-19.0.0-2499.ge0 497 kB/s | 45 kB 00:00 2024-03-28T13:06:26.704 INFO:teuthology.orchestra.run.smithi188.stdout:(20/174): python3-ceph-common-19.0.0-2499.ge0e4 1.8 MB/s | 134 kB 00:00 2024-03-28T13:06:26.788 INFO:teuthology.orchestra.run.smithi188.stdout:(21/174): python3-cephfs-19.0.0-2499.ge0e48ba6. 1.9 MB/s | 162 kB 00:00 2024-03-28T13:06:26.922 INFO:teuthology.orchestra.run.smithi188.stdout:(22/174): librgw2-19.0.0-2499.ge0e48ba6.el9.x86 11 MB/s | 5.1 MB 00:00 2024-03-28T13:06:26.963 INFO:teuthology.orchestra.run.smithi188.stdout:(23/174): python3-rados-19.0.0-2499.ge0e48ba6.e 1.8 MB/s | 320 kB 00:00 2024-03-28T13:06:26.997 INFO:teuthology.orchestra.run.smithi188.stdout:(24/174): python3-rbd-19.0.0-2499.ge0e48ba6.el9 3.9 MB/s | 298 kB 00:00 2024-03-28T13:06:27.047 INFO:teuthology.orchestra.run.smithi188.stdout:(25/174): python3-rgw-19.0.0-2499.ge0e48ba6.el9 1.2 MB/s | 99 kB 00:00 2024-03-28T13:06:27.073 INFO:teuthology.orchestra.run.smithi188.stdout:(26/174): rbd-fuse-19.0.0-2499.ge0e48ba6.el9.x8 1.1 MB/s | 85 kB 00:00 2024-03-28T13:06:27.131 INFO:teuthology.orchestra.run.smithi188.stdout:(27/174): rbd-nbd-19.0.0-2499.ge0e48ba6.el9.x86 2.9 MB/s | 170 kB 00:00 2024-03-28T13:06:27.198 INFO:teuthology.orchestra.run.smithi188.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2499.g 517 kB/s | 34 kB 00:00 2024-03-28T13:06:27.282 INFO:teuthology.orchestra.run.smithi188.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2499.ge0e48ba 1.7 MB/s | 141 kB 00:00 2024-03-28T13:06:27.740 INFO:teuthology.orchestra.run.smithi188.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2499.ge0e48 7.7 MB/s | 3.5 MB 00:00 2024-03-28T13:06:27.816 INFO:teuthology.orchestra.run.smithi188.stdout:(31/174): rbd-mirror-19.0.0-2499.ge0e48ba6.el9. 4.0 MB/s | 3.1 MB 00:00 2024-03-28T13:06:27.891 INFO:teuthology.orchestra.run.smithi188.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2499.ge0 3.3 MB/s | 249 kB 00:00 2024-03-28T13:06:27.966 INFO:teuthology.orchestra.run.smithi188.stdout:(33/174): ceph-mgr-rook-19.0.0-2499.ge0e48ba6.e 667 kB/s | 49 kB 00:00 2024-03-28T13:06:28.041 INFO:teuthology.orchestra.run.smithi188.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2499.ge 221 kB/s | 16 kB 00:00 2024-03-28T13:06:28.133 INFO:teuthology.orchestra.run.smithi188.stdout:(35/174): ceph-volume-19.0.0-2499.ge0e48ba6.el9 2.8 MB/s | 264 kB 00:00 2024-03-28T13:06:28.409 INFO:teuthology.orchestra.run.smithi188.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-03-28T13:06:28.442 INFO:teuthology.orchestra.run.smithi188.stdout:(37/174): cephadm-19.0.0-2499.ge0e48ba6.el9.noa 2.4 MB/s | 762 kB 00:00 2024-03-28T13:06:28.476 INFO:teuthology.orchestra.run.smithi188.stdout:(38/174): cephfs-top-19.0.0-2499.ge0e48ba6.el9. 277 kB/s | 18 kB 00:00 2024-03-28T13:06:28.709 INFO:teuthology.orchestra.run.smithi188.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2024-03-28T13:06:28.751 INFO:teuthology.orchestra.run.smithi188.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 4.3 MB/s | 920 kB 00:00 2024-03-28T13:06:28.776 INFO:teuthology.orchestra.run.smithi188.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-28T13:06:28.810 INFO:teuthology.orchestra.run.smithi188.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2024-03-28T13:06:28.835 INFO:teuthology.orchestra.run.smithi188.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 566 kB/s | 32 kB 00:00 2024-03-28T13:06:28.936 INFO:teuthology.orchestra.run.smithi188.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-03-28T13:06:28.961 INFO:teuthology.orchestra.run.smithi188.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 148 kB/s | 17 kB 00:00 2024-03-28T13:06:28.994 INFO:teuthology.orchestra.run.smithi188.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 314 kB/s | 10 kB 00:00 2024-03-28T13:06:29.020 INFO:teuthology.orchestra.run.smithi188.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-03-28T13:06:29.045 INFO:teuthology.orchestra.run.smithi188.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 913 kB/s | 44 kB 00:00 2024-03-28T13:06:29.107 INFO:teuthology.orchestra.run.smithi188.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.6 MB/s | 661 kB 00:00 2024-03-28T13:06:29.146 INFO:teuthology.orchestra.run.smithi188.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 6.0 MB/s | 578 kB 00:00 2024-03-28T13:06:29.171 INFO:teuthology.orchestra.run.smithi188.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 792 kB/s | 46 kB 00:00 2024-03-28T13:06:29.205 INFO:teuthology.orchestra.run.smithi188.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2024-03-28T13:06:29.238 INFO:teuthology.orchestra.run.smithi188.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.9 MB/s | 187 kB 00:00 2024-03-28T13:06:29.272 INFO:teuthology.orchestra.run.smithi188.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-03-28T13:06:29.289 INFO:teuthology.orchestra.run.smithi188.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 718 kB/s | 34 kB 00:00 2024-03-28T13:06:29.422 INFO:teuthology.orchestra.run.smithi188.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-03-28T13:06:29.464 INFO:teuthology.orchestra.run.smithi188.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 6.3 MB/s | 1.0 MB 00:00 2024-03-28T13:06:29.548 INFO:teuthology.orchestra.run.smithi188.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 946 kB/s | 78 kB 00:00 2024-03-28T13:06:29.682 INFO:teuthology.orchestra.run.smithi188.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 5.8 MB 00:00 2024-03-28T13:06:29.698 INFO:teuthology.orchestra.run.smithi188.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 179 kB/s | 24 kB 00:00 2024-03-28T13:06:29.723 INFO:teuthology.orchestra.run.smithi188.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 763 kB/s | 18 kB 00:00 2024-03-28T13:06:29.749 INFO:teuthology.orchestra.run.smithi188.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2024-03-28T13:06:29.774 INFO:teuthology.orchestra.run.smithi188.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 657 kB/s | 32 kB 00:00 2024-03-28T13:06:29.899 INFO:teuthology.orchestra.run.smithi188.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2024-03-28T13:06:29.950 INFO:teuthology.orchestra.run.smithi188.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-28T13:06:29.991 INFO:teuthology.orchestra.run.smithi188.stdout:(66/174): python3-oauthlib-3.1.1-5.el9.noarch.r 4.2 MB/s | 174 kB 00:00 2024-03-28T13:06:30.016 INFO:teuthology.orchestra.run.smithi188.stdout:(67/174): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 69 kB 00:00 2024-03-28T13:06:30.050 INFO:teuthology.orchestra.run.smithi188.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 1.3 MB/s | 41 kB 00:00 2024-03-28T13:06:30.084 INFO:teuthology.orchestra.run.smithi188.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 238 kB 00:00 2024-03-28T13:06:30.109 INFO:teuthology.orchestra.run.smithi188.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2024-03-28T13:06:30.143 INFO:teuthology.orchestra.run.smithi188.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2024-03-28T13:06:30.168 INFO:teuthology.orchestra.run.smithi188.stdout:(72/174): python3-pytz-2021.1-5.el9.noarch.rpm 846 kB/s | 47 kB 00:00 2024-03-28T13:06:30.193 INFO:teuthology.orchestra.run.smithi188.stdout:(73/174): python3-requests-oauthlib-1.3.0-12.el 875 kB/s | 43 kB 00:00 2024-03-28T13:06:30.502 INFO:teuthology.orchestra.run.smithi188.stdout:(74/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 47 MB/s | 16 MB 00:00 2024-03-28T13:06:30.527 INFO:teuthology.orchestra.run.smithi188.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 121 kB/s | 39 kB 00:00 2024-03-28T13:06:30.552 INFO:teuthology.orchestra.run.smithi188.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 839 kB/s | 41 kB 00:00 2024-03-28T13:06:30.586 INFO:teuthology.orchestra.run.smithi188.stdout:(77/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 300 kB 00:00 2024-03-28T13:06:30.611 INFO:teuthology.orchestra.run.smithi188.stdout:(78/174): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2024-03-28T13:06:30.637 INFO:teuthology.orchestra.run.smithi188.stdout:(79/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-28T13:06:30.662 INFO:teuthology.orchestra.run.smithi188.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 898 kB/s | 44 kB 00:00 2024-03-28T13:06:30.687 INFO:teuthology.orchestra.run.smithi188.stdout:(81/174): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2024-03-28T13:06:30.721 INFO:teuthology.orchestra.run.smithi188.stdout:(82/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.1 MB/s | 241 kB 00:00 2024-03-28T13:06:30.788 INFO:teuthology.orchestra.run.smithi188.stdout:(83/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-28T13:06:30.813 INFO:teuthology.orchestra.run.smithi188.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-03-28T13:06:30.838 INFO:teuthology.orchestra.run.smithi188.stdout:(85/174): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2024-03-28T13:06:30.872 INFO:teuthology.orchestra.run.smithi188.stdout:(86/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2024-03-28T13:06:31.406 INFO:teuthology.orchestra.run.smithi188.stdout:(87/174): ceph-test-19.0.0-2499.ge0e48ba6.el9.x 8.1 MB/s | 48 MB 00:05 2024-03-28T13:06:31.656 INFO:teuthology.orchestra.run.smithi188.stdout:(88/174): jq-1.6-16.el9.x86_64.rpm 230 kB/s | 187 kB 00:00 2024-03-28T13:06:31.735 INFO:teuthology.orchestra.run.smithi188.stdout:(89/174): libquadmath-11.4.1-3.el9.x86_64.rpm 595 kB/s | 195 kB 00:00 2024-03-28T13:06:31.790 INFO:teuthology.orchestra.run.smithi188.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 941 kB/s | 125 kB 00:00 2024-03-28T13:06:31.857 INFO:teuthology.orchestra.run.smithi188.stdout:(91/174): libgfortran-11.4.1-3.el9.x86_64.rpm 816 kB/s | 803 kB 00:00 2024-03-28T13:06:31.949 INFO:teuthology.orchestra.run.smithi188.stdout:(92/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 161 kB/s | 25 kB 00:00 2024-03-28T13:06:31.974 INFO:teuthology.orchestra.run.smithi188.stdout:(93/174): elfutils-libelf-devel-0.190-2.el9.x86 90 kB/s | 21 kB 00:00 2024-03-28T13:06:32.208 INFO:teuthology.orchestra.run.smithi188.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 3.8 MB/s | 911 kB 00:00 2024-03-28T13:06:32.266 INFO:teuthology.orchestra.run.smithi188.stdout:(95/174): postgresql-13.14-1.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-03-28T13:06:32.300 INFO:teuthology.orchestra.run.smithi188.stdout:(96/174): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-03-28T13:06:32.367 INFO:teuthology.orchestra.run.smithi188.stdout:(97/174): python3-devel-3.9.17-2.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-03-28T13:06:32.475 INFO:teuthology.orchestra.run.smithi188.stdout:(98/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 6.7 MB/s | 4.1 MB 00:00 2024-03-28T13:06:32.509 INFO:teuthology.orchestra.run.smithi188.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-03-28T13:06:32.574 INFO:teuthology.orchestra.run.smithi188.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 2.2 MB/s | 221 kB 00:00 2024-03-28T13:06:32.601 INFO:teuthology.orchestra.run.smithi188.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 394 kB/s | 36 kB 00:00 2024-03-28T13:06:32.634 INFO:teuthology.orchestra.run.smithi188.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 792 kB/s | 47 kB 00:00 2024-03-28T13:06:32.660 INFO:teuthology.orchestra.run.smithi188.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.6 MB/s | 94 kB 00:00 2024-03-28T13:06:32.794 INFO:teuthology.orchestra.run.smithi188.stdout:(104/174): postgresql-server-13.14-1.el9.x86_64 11 MB/s | 5.8 MB 00:00 2024-03-28T13:06:33.002 INFO:teuthology.orchestra.run.smithi188.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 61 kB/s | 22 kB 00:00 2024-03-28T13:06:33.127 INFO:teuthology.orchestra.run.smithi188.stdout:(106/174): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-03-28T13:06:33.195 INFO:teuthology.orchestra.run.smithi188.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-03-28T13:06:33.261 INFO:teuthology.orchestra.run.smithi188.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-03-28T13:06:33.295 INFO:teuthology.orchestra.run.smithi188.stdout:(109/174): grpc-data-1.46.7-10.el9.noarch.rpm 117 kB/s | 19 kB 00:00 2024-03-28T13:06:33.320 INFO:teuthology.orchestra.run.smithi188.stdout:(110/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-03-28T13:06:33.412 INFO:teuthology.orchestra.run.smithi188.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 743 kB/s | 67 kB 00:00 2024-03-28T13:06:33.437 INFO:teuthology.orchestra.run.smithi188.stdout:(112/174): liboath-2.6.7-2.el9.x86_64.rpm 344 kB/s | 49 kB 00:00 2024-03-28T13:06:33.488 INFO:teuthology.orchestra.run.smithi188.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-03-28T13:06:33.663 INFO:teuthology.orchestra.run.smithi188.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 9.5 MB/s | 4.4 MB 00:00 2024-03-28T13:06:33.730 INFO:teuthology.orchestra.run.smithi188.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-03-28T13:06:33.755 INFO:teuthology.orchestra.run.smithi188.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 477 kB/s | 43 kB 00:00 2024-03-28T13:06:33.814 INFO:teuthology.orchestra.run.smithi188.stdout:(117/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.2 MB/s | 838 kB 00:00 2024-03-28T13:06:33.847 INFO:teuthology.orchestra.run.smithi188.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2024-03-28T13:06:33.872 INFO:teuthology.orchestra.run.smithi188.stdout:(119/174): python3-cachetools-4.2.4-1.el9.noarc 227 kB/s | 32 kB 00:00 2024-03-28T13:06:33.906 INFO:teuthology.orchestra.run.smithi188.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-03-28T13:06:33.940 INFO:teuthology.orchestra.run.smithi188.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-03-28T13:06:33.965 INFO:teuthology.orchestra.run.smithi188.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 612 kB/s | 35 kB 00:00 2024-03-28T13:06:33.998 INFO:teuthology.orchestra.run.smithi188.stdout:(123/174): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-03-28T13:06:34.032 INFO:teuthology.orchestra.run.smithi188.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.2 MB/s | 203 kB 00:00 2024-03-28T13:06:34.066 INFO:teuthology.orchestra.run.smithi188.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-03-28T13:06:34.091 INFO:teuthology.orchestra.run.smithi188.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 185 kB/s | 11 kB 00:00 2024-03-28T13:06:34.125 INFO:teuthology.orchestra.run.smithi188.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 306 kB/s | 18 kB 00:00 2024-03-28T13:06:34.150 INFO:teuthology.orchestra.run.smithi188.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 399 kB/s | 23 kB 00:00 2024-03-28T13:06:34.184 INFO:teuthology.orchestra.run.smithi188.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 336 kB/s | 19 kB 00:00 2024-03-28T13:06:34.209 INFO:teuthology.orchestra.run.smithi188.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 340 kB/s | 20 kB 00:00 2024-03-28T13:06:34.284 INFO:teuthology.orchestra.run.smithi188.stdout:(131/174): python3-grpcio-1.46.7-10.el9.x86_64. 6.4 MB/s | 2.0 MB 00:00 2024-03-28T13:06:34.309 INFO:teuthology.orchestra.run.smithi188.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 464 kB/s | 46 kB 00:00 2024-03-28T13:06:34.343 INFO:teuthology.orchestra.run.smithi188.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-28T13:06:34.368 INFO:teuthology.orchestra.run.smithi188.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 996 kB/s | 58 kB 00:00 2024-03-28T13:06:34.418 INFO:teuthology.orchestra.run.smithi188.stdout:(135/174): python3-kubernetes-26.1.0-2.el9.noar 4.4 MB/s | 1.0 MB 00:00 2024-03-28T13:06:34.444 INFO:teuthology.orchestra.run.smithi188.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 220 kB/s | 16 kB 00:00 2024-03-28T13:06:34.477 INFO:teuthology.orchestra.run.smithi188.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-03-28T13:06:34.494 INFO:teuthology.orchestra.run.smithi188.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 569 kB/s | 28 kB 00:00 2024-03-28T13:06:34.528 INFO:teuthology.orchestra.run.smithi188.stdout:(139/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 828 kB/s | 90 kB 00:00 2024-03-28T13:06:34.545 INFO:teuthology.orchestra.run.smithi188.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-03-28T13:06:34.549 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 919 kB/s | 19 MB 00:21 2024-03-28T13:06:34.579 INFO:teuthology.orchestra.run.smithi188.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-03-28T13:06:34.595 INFO:teuthology.orchestra.run.smithi188.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 720 kB/s | 36 kB 00:00 2024-03-28T13:06:34.612 INFO:teuthology.orchestra.run.smithi188.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 713 kB/s | 59 kB 00:00 2024-03-28T13:06:34.661 INFO:teuthology.orchestra.run.smithi188.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 629 kB/s | 51 kB 00:00 2024-03-28T13:06:34.690 INFO:teuthology.orchestra.run.smithi188.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-03-28T13:06:34.722 INFO:teuthology.orchestra.run.smithi188.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-03-28T13:06:34.747 INFO:teuthology.orchestra.run.smithi188.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 346 kB/s | 20 kB 00:00 2024-03-28T13:06:34.780 INFO:teuthology.orchestra.run.smithi188.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-03-28T13:06:34.814 INFO:teuthology.orchestra.run.smithi188.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.1 MB/s | 192 kB 00:00 2024-03-28T13:06:34.981 INFO:teuthology.orchestra.run.smithi188.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-03-28T13:06:35.140 INFO:teuthology.orchestra.run.smithi188.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 2.0 MB/s | 321 kB 00:00 2024-03-28T13:06:35.398 INFO:teuthology.orchestra.run.smithi188.stdout:(152/174): librbd1-19.0.0-2499.ge0e48ba6.el9.x8 5.4 MB/s | 3.1 MB 00:00 2024-03-28T13:06:35.507 INFO:teuthology.orchestra.run.smithi188.stdout:(153/174): NetworkManager-1.47.2-1.el9.x86_64.r 6.3 MB/s | 2.3 MB 00:00 2024-03-28T13:06:35.593 INFO:teuthology.orchestra.run.smithi188.stdout:(154/174): librados2-19.0.0-2499.ge0e48ba6.el9. 4.2 MB/s | 3.4 MB 00:00 2024-03-28T13:06:35.618 INFO:teuthology.orchestra.run.smithi188.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 385 kB/s | 42 kB 00:00 2024-03-28T13:06:35.668 INFO:teuthology.orchestra.run.smithi188.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 71 kB 00:00 2024-03-28T13:06:35.719 INFO:teuthology.orchestra.run.smithi188.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 739 kB/s | 36 kB 00:00 2024-03-28T13:06:35.752 INFO:teuthology.orchestra.run.smithi188.stdout:(158/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.5 MB/s | 248 kB 00:00 2024-03-28T13:06:35.786 INFO:teuthology.orchestra.run.smithi188.stdout:(159/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-03-28T13:06:35.845 INFO:teuthology.orchestra.run.smithi188.stdout:(160/174): NetworkManager-libnm-1.47.2-1.el9.x8 4.1 MB/s | 1.8 MB 00:00 2024-03-28T13:06:35.870 INFO:teuthology.orchestra.run.smithi188.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 2.1 MB/s | 256 kB 00:00 2024-03-28T13:06:35.895 INFO:teuthology.orchestra.run.smithi188.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 213 kB/s | 23 kB 00:00 2024-03-28T13:06:35.946 INFO:teuthology.orchestra.run.smithi188.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.1 MB/s | 108 kB 00:00 2024-03-28T13:06:35.971 INFO:teuthology.orchestra.run.smithi188.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.5 MB/s | 155 kB 00:00 2024-03-28T13:06:35.999 INFO:teuthology.orchestra.run.smithi188.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 136 kB 00:00 2024-03-28T13:06:36.013 INFO:teuthology.orchestra.run.smithi188.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 964 kB/s | 64 kB 00:00 2024-03-28T13:06:36.030 INFO:teuthology.orchestra.run.smithi188.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 495 kB/s | 29 kB 00:00 2024-03-28T13:06:36.122 INFO:teuthology.orchestra.run.smithi188.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 11 MB/s | 1.2 MB 00:00 2024-03-28T13:06:36.189 INFO:teuthology.orchestra.run.smithi188.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 6.9 MB/s | 465 kB 00:00 2024-03-28T13:06:36.256 INFO:teuthology.orchestra.run.smithi188.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 10 MB/s | 2.3 MB 00:00 2024-03-28T13:06:36.272 INFO:teuthology.orchestra.run.smithi188.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 376 kB/s | 31 kB 00:00 2024-03-28T13:06:36.365 INFO:teuthology.orchestra.run.smithi188.stdout:(172/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 6.2 MB/s | 2.2 MB 00:00 2024-03-28T13:06:36.415 INFO:teuthology.orchestra.run.smithi188.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 6.8 MB/s | 1.1 MB 00:00 2024-03-28T13:06:36.515 INFO:teuthology.orchestra.run.smithi188.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 308 kB/s | 74 kB 00:00 2024-03-28T13:06:36.523 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-03-28T13:06:36.524 INFO:teuthology.orchestra.run.smithi188.stdout:Total 15 MB/s | 227 MB 00:14 2024-03-28T13:06:38.160 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-03-28T13:06:38.540 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-03-28T13:06:38.540 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-03-28T13:06:38.967 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.1 MB 00:00 2024-03-28T13:06:40.826 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-03-28T13:06:40.981 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-03-28T13:06:40.982 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-03-28T13:06:43.416 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 8.6 MB/s | 21 MB 00:02 2024-03-28T13:06:48.484 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-03-28T13:06:48.603 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-28T13:06:48.653 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-28T13:06:48.690 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T13:06:48.735 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T13:06:48.830 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-28T13:06:48.936 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-28T13:06:48.991 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-28T13:06:49.031 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-28T13:06:49.274 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-28T13:06:49.427 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-03-28T13:06:49.718 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-28T13:06:49.762 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 10/198 2024-03-28T13:06:49.903 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 10/198 2024-03-28T13:06:49.932 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 11/198 2024-03-28T13:06:50.053 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 11/198 2024-03-28T13:06:50.098 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:19.0.0-2499.ge0e48ba6.el9.x86_64 12/198 2024-03-28T13:06:50.168 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-28T13:06:50.465 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-28T13:06:50.537 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-28T13:06:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:19.0.0-2499.ge0e48ba6.el9.x86_64 16/198 2024-03-28T13:06:50.715 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2499.ge0e48ba6.el9.x86_64 16/198 2024-03-28T13:06:50.747 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:19.0.0-2499.ge0e48ba6.el9.x86 17/198 2024-03-28T13:06:50.820 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2499.ge0e48ba6.el9.x86 17/198 2024-03-28T13:06:50.874 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-28T13:06:50.932 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T13:06:50.996 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T13:06:51.309 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T13:06:51.358 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T13:06:51.407 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-28T13:06:51.732 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-28T13:06:51.940 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T13:06:52.029 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T13:06:52.205 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T13:06:52.207 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-28T13:06:52.229 INFO:teuthology.orchestra.run.smithi019.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-28T13:06:52.230 INFO:teuthology.orchestra.run.smithi019.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-28T13:06:52.232 INFO:teuthology.orchestra.run.smithi019.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-28T13:06:52.233 INFO:teuthology.orchestra.run.smithi019.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-28T13:06:52.236 INFO:teuthology.orchestra.run.smithi019.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-28T13:06:52.236 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T13:06:52.239 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-28T13:06:52.241 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-28T13:06:52.310 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T13:06:52.337 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-03-28T13:06:52.360 INFO:teuthology.orchestra.run.smithi019.stdout:============================================================================================ 2024-03-28T13:06:52.360 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-03-28T13:06:52.360 INFO:teuthology.orchestra.run.smithi019.stdout:============================================================================================ 2024-03-28T13:06:52.360 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-03-28T13:06:52.360 INFO:teuthology.orchestra.run.smithi019.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-28T13:06:52.360 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 6.6 k 2024-03-28T13:06:52.360 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 5.4 M 2024-03-28T13:06:52.360 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 890 k 2024-03-28T13:06:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 145 k 2024-03-28T13:06:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 1.5 M 2024-03-28T13:06:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 141 k 2024-03-28T13:06:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 3.5 M 2024-03-28T13:06:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 7.4 M 2024-03-28T13:06:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 49 k 2024-03-28T13:06:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 9.8 M 2024-03-28T13:06:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 48 M 2024-03-28T13:06:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 264 k 2024-03-28T13:06:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 762 k 2024-03-28T13:06:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 221 k 2024-03-28T13:06:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 27 k 2024-03-28T13:06:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 18 k 2024-03-28T13:06:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-28T13:06:52.361 INFO:teuthology.orchestra.run.smithi019.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-28T13:06:52.362 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-28T13:06:52.362 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 32 k 2024-03-28T13:06:52.362 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 746 k 2024-03-28T13:06:52.362 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 127 k 2024-03-28T13:06:52.362 INFO:teuthology.orchestra.run.smithi019.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-28T13:06:52.362 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-28T13:06:52.362 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-28T13:06:52.362 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-03-28T13:06:52.362 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-03-28T13:06:52.362 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-03-28T13:06:52.362 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 162 k 2024-03-28T13:06:52.362 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 320 k 2024-03-28T13:06:52.362 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 298 k 2024-03-28T13:06:52.363 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 99 k 2024-03-28T13:06:52.363 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 85 k 2024-03-28T13:06:52.363 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 3.1 M 2024-03-28T13:06:52.363 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 170 k 2024-03-28T13:06:52.363 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-28T13:06:52.363 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-03-28T13:06:52.363 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-28T13:06:52.363 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-28T13:06:52.363 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-28T13:06:52.363 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-28T13:06:52.363 INFO:teuthology.orchestra.run.smithi019.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-28T13:06:52.363 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-28T13:06:52.364 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-28T13:06:52.364 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-28T13:06:52.364 INFO:teuthology.orchestra.run.smithi019.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-28T13:06:52.364 INFO:teuthology.orchestra.run.smithi019.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-28T13:06:52.364 INFO:teuthology.orchestra.run.smithi019.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-28T13:06:52.364 INFO:teuthology.orchestra.run.smithi019.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-28T13:06:52.364 INFO:teuthology.orchestra.run.smithi019.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-28T13:06:52.364 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 3.4 M 2024-03-28T13:06:52.364 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 3.1 M 2024-03-28T13:06:52.364 INFO:teuthology.orchestra.run.smithi019.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-28T13:06:52.364 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-28T13:06:52.364 INFO:teuthology.orchestra.run.smithi019.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-28T13:06:52.364 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-28T13:06:52.365 INFO:teuthology.orchestra.run.smithi019.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-28T13:06:52.365 INFO:teuthology.orchestra.run.smithi019.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-28T13:06:52.365 INFO:teuthology.orchestra.run.smithi019.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-28T13:06:52.365 INFO:teuthology.orchestra.run.smithi019.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-28T13:06:52.365 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-28T13:06:52.365 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-03-28T13:06:52.365 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-28T13:06:52.365 INFO:teuthology.orchestra.run.smithi019.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-28T13:06:52.365 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-28T13:06:52.365 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 21 M 2024-03-28T13:06:52.365 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 34 k 2024-03-28T13:06:52.365 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 2.4 M 2024-03-28T13:06:52.365 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 249 k 2024-03-28T13:06:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 4.7 M 2024-03-28T13:06:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 17 M 2024-03-28T13:06:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2499.ge0e48ba6.el9 ceph-noarch 16 k 2024-03-28T13:06:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 25 k 2024-03-28T13:06:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-28T13:06:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-28T13:06:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-28T13:06:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-28T13:06:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-28T13:06:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-28T13:06:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-28T13:06:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-28T13:06:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-28T13:06:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 163 k 2024-03-28T13:06:52.366 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-28T13:06:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-28T13:06:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-28T13:06:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-28T13:06:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-28T13:06:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 501 k 2024-03-28T13:06:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-28T13:06:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 5.1 M 2024-03-28T13:06:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-28T13:06:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-28T13:06:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-28T13:06:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-28T13:06:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-28T13:06:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-28T13:06:52.367 INFO:teuthology.orchestra.run.smithi019.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 45 k 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.0.0-2499.ge0e48ba6.el9 ceph 134 k 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-28T13:06:52.368 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-28T13:06:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-28T13:06:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-28T13:06:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-28T13:06:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-28T13:06:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-28T13:06:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-28T13:06:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-28T13:06:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-28T13:06:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-28T13:06:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-28T13:06:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-28T13:06:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-28T13:06:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-28T13:06:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-28T13:06:52.369 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-28T13:06:52.370 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-28T13:06:52.370 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-28T13:06:52.370 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-28T13:06:52.370 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-28T13:06:52.370 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-28T13:06:52.370 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-28T13:06:52.370 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-28T13:06:52.370 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-28T13:06:52.370 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-28T13:06:52.370 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-28T13:06:52.370 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-28T13:06:52.370 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-28T13:06:52.370 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-28T13:06:52.370 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-28T13:06:52.371 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-03-28T13:06:52.372 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-03-28T13:06:52.373 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:06:52.373 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-03-28T13:06:52.373 INFO:teuthology.orchestra.run.smithi019.stdout:============================================================================================ 2024-03-28T13:06:52.373 INFO:teuthology.orchestra.run.smithi019.stdout:Install 150 Packages 2024-03-28T13:06:52.373 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 24 Packages 2024-03-28T13:06:52.373 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:06:52.373 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 227 M 2024-03-28T13:06:52.373 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-03-28T13:06:52.620 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T13:06:52.801 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-28T13:06:52.926 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-28T13:06:52.968 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-28T13:06:53.027 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-28T13:06:53.090 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-28T13:06:53.209 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-28T13:06:53.281 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-28T13:06:53.324 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-28T13:06:53.423 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-28T13:06:53.471 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-28T13:06:53.504 INFO:teuthology.orchestra.run.smithi019.stdout:(1/174): ceph-19.0.0-2499.ge0e48ba6.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-03-28T13:06:53.541 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-28T13:06:53.584 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-28T13:06:53.616 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-28T13:06:53.730 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-28T13:06:53.769 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:19.0.0-2499.ge0e48ba6.el9. 39/198 2024-03-28T13:06:53.829 INFO:teuthology.orchestra.run.smithi019.stdout:(2/174): ceph-fuse-19.0.0-2499.ge0e48ba6.el9.x8 2.7 MB/s | 890 kB 00:00 2024-03-28T13:06:53.860 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:19.0.0-2499.ge0e48ba6.el 40/198 2024-03-28T13:06:53.894 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:19.0.0-2499.ge0e48ba6.el9.x86_6 41/198 2024-03-28T13:06:53.929 INFO:teuthology.orchestra.run.smithi019.stdout:(3/174): ceph-immutable-object-cache-19.0.0-249 1.4 MB/s | 145 kB 00:00 2024-03-28T13:06:54.015 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-28T13:06:54.081 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-28T13:06:54.163 INFO:teuthology.orchestra.run.smithi019.stdout:(4/174): ceph-base-19.0.0-2499.ge0e48ba6.el9.x8 5.6 MB/s | 5.4 MB 00:00 2024-03-28T13:06:54.181 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-28T13:06:54.381 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-28T13:06:54.430 INFO:teuthology.orchestra.run.smithi019.stdout:(5/174): ceph-mgr-19.0.0-2499.ge0e48ba6.el9.x86 5.6 MB/s | 1.5 MB 00:00 2024-03-28T13:06:54.449 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-28T13:06:54.505 INFO:teuthology.orchestra.run.smithi019.stdout:(6/174): ceph-mds-19.0.0-2499.ge0e48ba6.el9.x86 4.1 MB/s | 2.4 MB 00:00 2024-03-28T13:06:55.132 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-28T13:06:55.229 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-28T13:06:55.230 INFO:teuthology.orchestra.run.smithi019.stdout:(7/174): ceph-common-19.0.0-2499.ge0e48ba6.el9. 10 MB/s | 21 MB 00:02 2024-03-28T13:06:55.405 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-28T13:06:55.448 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T13:06:55.480 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T13:06:55.530 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-28T13:06:55.589 INFO:teuthology.orchestra.run.smithi019.stdout:(8/174): ceph-mon-19.0.0-2499.ge0e48ba6.el9.x86 4.1 MB/s | 4.7 MB 00:01 2024-03-28T13:06:55.597 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-28T13:06:55.647 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:19.0.0-2499.ge0e48ba6.el9.x86_6 53/198 2024-03-28T13:06:55.672 INFO:teuthology.orchestra.run.smithi019.stdout:(9/174): ceph-selinux-19.0.0-2499.ge0e48ba6.el9 306 kB/s | 25 kB 00:00 2024-03-28T13:06:55.688 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-28T13:06:56.023 INFO:teuthology.orchestra.run.smithi019.stdout:(10/174): ceph-radosgw-19.0.0-2499.ge0e48ba6.el 12 MB/s | 9.8 MB 00:00 2024-03-28T13:06:56.093 INFO:teuthology.orchestra.run.smithi019.stdout:(11/174): cephfs-mirror-19.0.0-2499.ge0e48ba6.e 3.1 MB/s | 221 kB 00:00 2024-03-28T13:06:56.097 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-28T13:06:56.173 INFO:teuthology.orchestra.run.smithi019.stdout:(12/174): cephfs-shell-19.0.0-2499.ge0e48ba6.el 341 kB/s | 27 kB 00:00 2024-03-28T13:06:56.249 INFO:teuthology.orchestra.run.smithi019.stdout:(13/174): libcephfs-devel-19.0.0-2499.ge0e48ba6 426 kB/s | 32 kB 00:00 2024-03-28T13:06:56.343 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-28T13:06:56.383 INFO:teuthology.orchestra.run.smithi019.stdout:(14/174): libcephfs2-19.0.0-2499.ge0e48ba6.el9. 5.5 MB/s | 746 kB 00:00 2024-03-28T13:06:56.435 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-28T13:06:56.490 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-28T13:06:56.633 INFO:teuthology.orchestra.run.smithi019.stdout:(15/174): ceph-osd-19.0.0-2499.ge0e48ba6.el9.x8 7.9 MB/s | 17 MB 00:02 2024-03-28T13:06:56.675 INFO:teuthology.orchestra.run.smithi019.stdout:(16/174): libcephsqlite-19.0.0-2499.ge0e48ba6.e 560 kB/s | 163 kB 00:00 2024-03-28T13:06:56.686 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-28T13:06:56.708 INFO:teuthology.orchestra.run.smithi019.stdout:(17/174): librados-devel-19.0.0-2499.ge0e48ba6. 1.7 MB/s | 127 kB 00:00 2024-03-28T13:06:56.765 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-28T13:06:56.792 INFO:teuthology.orchestra.run.smithi019.stdout:(18/174): libradosstriper1-19.0.0-2499.ge0e48ba 4.2 MB/s | 501 kB 00:00 2024-03-28T13:06:56.833 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-28T13:06:56.893 INFO:teuthology.orchestra.run.smithi019.stdout:(19/174): python3-ceph-argparse-19.0.0-2499.ge0 455 kB/s | 45 kB 00:00 2024-03-28T13:06:56.929 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-28T13:06:56.974 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-28T13:06:56.984 INFO:teuthology.orchestra.run.smithi019.stdout:(20/174): python3-ceph-common-19.0.0-2499.ge0e4 1.4 MB/s | 134 kB 00:00 2024-03-28T13:06:57.014 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-28T13:06:57.076 INFO:teuthology.orchestra.run.smithi019.stdout:(21/174): python3-cephfs-19.0.0-2499.ge0e48ba6. 1.7 MB/s | 162 kB 00:00 2024-03-28T13:06:57.082 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-28T13:06:57.130 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-28T13:06:57.177 INFO:teuthology.orchestra.run.smithi019.stdout:(22/174): python3-rados-19.0.0-2499.ge0e48ba6.e 3.1 MB/s | 320 kB 00:00 2024-03-28T13:06:57.209 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-28T13:06:57.271 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-28T13:06:57.276 INFO:teuthology.orchestra.run.smithi019.stdout:(23/174): python3-rbd-19.0.0-2499.ge0e48ba6.el9 3.0 MB/s | 298 kB 00:00 2024-03-28T13:06:57.436 INFO:teuthology.orchestra.run.smithi019.stdout:(24/174): librgw2-19.0.0-2499.ge0e48ba6.el9.x86 7.1 MB/s | 5.1 MB 00:00 2024-03-28T13:06:57.461 INFO:teuthology.orchestra.run.smithi019.stdout:(25/174): python3-rgw-19.0.0-2499.ge0e48ba6.el9 542 kB/s | 99 kB 00:00 2024-03-28T13:06:57.470 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-28T13:06:57.494 INFO:teuthology.orchestra.run.smithi019.stdout:(26/174): rbd-fuse-19.0.0-2499.ge0e48ba6.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-28T13:06:57.528 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-28T13:06:57.578 INFO:teuthology.orchestra.run.smithi019.stdout:(27/174): rbd-nbd-19.0.0-2499.ge0e48ba6.el9.x86 2.0 MB/s | 170 kB 00:00 2024-03-28T13:06:57.600 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-28T13:06:57.661 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-28T13:06:57.695 INFO:teuthology.orchestra.run.smithi019.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2499.g 293 kB/s | 34 kB 00:00 2024-03-28T13:06:57.787 INFO:teuthology.orchestra.run.smithi019.stdout:(29/174): rbd-mirror-19.0.0-2499.ge0e48ba6.el9. 9.5 MB/s | 3.1 MB 00:00 2024-03-28T13:06:57.803 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-28T13:06:57.820 INFO:teuthology.orchestra.run.smithi019.stdout:(30/174): ceph-mgr-cephadm-19.0.0-2499.ge0e48ba 1.1 MB/s | 141 kB 00:00 2024-03-28T13:06:57.848 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-28T13:06:57.895 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-28T13:06:57.931 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-28T13:06:57.982 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-28T13:06:58.154 INFO:teuthology.orchestra.run.smithi019.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2499.ge0e48 9.7 MB/s | 3.5 MB 00:00 2024-03-28T13:06:58.159 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-28T13:06:58.238 INFO:teuthology.orchestra.run.smithi019.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2499.ge0 3.0 MB/s | 249 kB 00:00 2024-03-28T13:06:58.338 INFO:teuthology.orchestra.run.smithi019.stdout:(33/174): ceph-mgr-rook-19.0.0-2499.ge0e48ba6.e 498 kB/s | 49 kB 00:00 2024-03-28T13:06:58.342 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-28T13:06:58.455 INFO:teuthology.orchestra.run.smithi019.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2499.ge 142 kB/s | 16 kB 00:00 2024-03-28T13:06:58.506 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-28T13:06:58.555 INFO:teuthology.orchestra.run.smithi019.stdout:(35/174): ceph-volume-19.0.0-2499.ge0e48ba6.el9 2.6 MB/s | 264 kB 00:00 2024-03-28T13:06:58.574 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-28T13:06:58.638 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-28T13:06:58.680 INFO:teuthology.orchestra.run.smithi019.stdout:(36/174): cephadm-19.0.0-2499.ge0e48ba6.el9.noa 6.0 MB/s | 762 kB 00:00 2024-03-28T13:06:58.712 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-28T13:06:58.806 INFO:teuthology.orchestra.run.smithi019.stdout:(37/174): ceph-mgr-diskprediction-local-19.0.0- 7.5 MB/s | 7.4 MB 00:00 2024-03-28T13:06:58.848 INFO:teuthology.orchestra.run.smithi019.stdout:(38/174): cephfs-top-19.0.0-2499.ge0e48ba6.el9. 110 kB/s | 18 kB 00:00 2024-03-28T13:06:58.959 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-28T13:06:59.020 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-28T13:06:59.065 INFO:teuthology.orchestra.run.smithi019.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 662 kB 00:00 2024-03-28T13:06:59.107 INFO:teuthology.orchestra.run.smithi019.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.5 MB/s | 920 kB 00:00 2024-03-28T13:06:59.132 INFO:teuthology.orchestra.run.smithi019.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-28T13:06:59.166 INFO:teuthology.orchestra.run.smithi019.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 307 kB 00:00 2024-03-28T13:06:59.193 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-28T13:06:59.208 INFO:teuthology.orchestra.run.smithi019.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 436 kB/s | 32 kB 00:00 2024-03-28T13:06:59.261 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-28T13:06:59.308 INFO:teuthology.orchestra.run.smithi019.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-03-28T13:06:59.309 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-28T13:06:59.347 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-28T13:06:59.375 INFO:teuthology.orchestra.run.smithi019.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 114 kB/s | 17 kB 00:00 2024-03-28T13:06:59.390 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-28T13:06:59.425 INFO:teuthology.orchestra.run.smithi019.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 206 kB/s | 10 kB 00:00 2024-03-28T13:06:59.436 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-28T13:06:59.451 INFO:teuthology.orchestra.run.smithi019.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2024-03-28T13:06:59.493 INFO:teuthology.orchestra.run.smithi019.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 683 kB/s | 44 kB 00:00 2024-03-28T13:06:59.504 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-28T13:06:59.535 INFO:teuthology.orchestra.run.smithi019.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.9 MB/s | 661 kB 00:00 2024-03-28T13:06:59.560 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-28T13:06:59.576 INFO:teuthology.orchestra.run.smithi019.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 7.3 MB/s | 578 kB 00:00 2024-03-28T13:06:59.617 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-28T13:06:59.618 INFO:teuthology.orchestra.run.smithi019.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 599 kB/s | 46 kB 00:00 2024-03-28T13:06:59.652 INFO:teuthology.orchestra.run.smithi019.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-28T13:06:59.694 INFO:teuthology.orchestra.run.smithi019.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 187 kB 00:00 2024-03-28T13:06:59.728 INFO:teuthology.orchestra.run.smithi019.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.9 MB/s | 217 kB 00:00 2024-03-28T13:06:59.745 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-28T13:06:59.787 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-28T13:06:59.800 INFO:teuthology.orchestra.run.smithi019.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 333 kB/s | 34 kB 00:00 2024-03-28T13:06:59.827 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-28T13:06:59.888 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-28T13:06:59.938 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-28T13:06:59.945 INFO:teuthology.orchestra.run.smithi019.stdout:(56/174): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-03-28T13:06:59.987 INFO:teuthology.orchestra.run.smithi019.stdout:(57/174): protobuf-3.14.0-13.el9.x86_64.rpm 5.9 MB/s | 1.0 MB 00:00 2024-03-28T13:07:00.037 INFO:teuthology.orchestra.run.smithi019.stdout:(58/174): python3-attrs-20.3.0-7.el9.noarch.rpm 1.6 MB/s | 78 kB 00:00 2024-03-28T13:07:00.085 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-28T13:07:00.171 INFO:teuthology.orchestra.run.smithi019.stdout:(59/174): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 5.8 MB 00:00 2024-03-28T13:07:00.197 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-28T13:07:00.213 INFO:teuthology.orchestra.run.smithi019.stdout:(60/174): python3-jsonpatch-1.21-16.el9.noarch. 153 kB/s | 24 kB 00:00 2024-03-28T13:07:00.251 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-28T13:07:00.287 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-28T13:07:00.288 INFO:teuthology.orchestra.run.smithi019.stdout:(61/174): python3-jsonpointer-2.0-4.el9.noarch. 245 kB/s | 18 kB 00:00 2024-03-28T13:07:00.313 INFO:teuthology.orchestra.run.smithi019.stdout:(62/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 144 kB 00:00 2024-03-28T13:07:00.343 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-28T13:07:00.380 INFO:teuthology.orchestra.run.smithi019.stdout:(63/174): python3-markupsafe-1.1.1-12.el9.x86_6 482 kB/s | 32 kB 00:00 2024-03-28T13:07:00.483 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-28T13:07:00.538 INFO:teuthology.orchestra.run.smithi019.stdout:(64/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2024-03-28T13:07:00.561 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-28T13:07:00.564 INFO:teuthology.orchestra.run.smithi019.stdout:(65/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-28T13:07:00.614 INFO:teuthology.orchestra.run.smithi019.stdout:(66/174): python3-packaging-20.9-5.el9.noarch.r 1.4 MB/s | 69 kB 00:00 2024-03-28T13:07:00.672 INFO:teuthology.orchestra.run.smithi019.stdout:(67/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.6 MB/s | 174 kB 00:00 2024-03-28T13:07:00.731 INFO:teuthology.orchestra.run.smithi019.stdout:(68/174): python3-prettytable-0.7.2-27.el9.noar 359 kB/s | 41 kB 00:00 2024-03-28T13:07:00.756 INFO:teuthology.orchestra.run.smithi019.stdout:(69/174): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-03-28T13:07:00.808 INFO:teuthology.orchestra.run.smithi019.stdout:(70/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 132 kB 00:00 2024-03-28T13:07:00.833 INFO:teuthology.orchestra.run.smithi019.stdout:(71/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2024-03-28T13:07:01.367 INFO:teuthology.orchestra.run.smithi019.stdout:(72/174): ceph-test-19.0.0-2499.ge0e48ba6.el9.x 8.4 MB/s | 48 MB 00:05 2024-03-28T13:07:01.408 INFO:teuthology.orchestra.run.smithi019.stdout:(73/174): python3-pytz-2021.1-5.el9.noarch.rpm 79 kB/s | 47 kB 00:00 2024-03-28T13:07:01.450 INFO:teuthology.orchestra.run.smithi019.stdout:(74/174): python3-requests-oauthlib-1.3.0-12.el 69 kB/s | 43 kB 00:00 2024-03-28T13:07:01.514 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-28T13:07:01.569 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-28T13:07:01.602 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-28T13:07:01.658 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-28T13:07:01.770 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-28T13:07:01.792 INFO:teuthology.orchestra.run.smithi019.stdout:(75/174): python3-toml-0.10.2-6.el9.noarch.rpm 116 kB/s | 39 kB 00:00 2024-03-28T13:07:01.819 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-28T13:07:01.834 INFO:teuthology.orchestra.run.smithi019.stdout:(76/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 109 kB/s | 41 kB 00:00 2024-03-28T13:07:02.050 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-28T13:07:02.093 INFO:teuthology.orchestra.run.smithi019.stdout:(77/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-03-28T13:07:02.099 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-28T13:07:02.118 INFO:teuthology.orchestra.run.smithi019.stdout:(78/174): socat-1.7.4.1-5.el9.x86_64.rpm 924 kB/s | 300 kB 00:00 2024-03-28T13:07:02.168 INFO:teuthology.orchestra.run.smithi019.stdout:(79/174): uuid-1.6.2-55.el9.x86_64.rpm 177 kB/s | 56 kB 00:00 2024-03-28T13:07:02.191 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-28T13:07:02.210 INFO:teuthology.orchestra.run.smithi019.stdout:(80/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 490 kB/s | 44 kB 00:00 2024-03-28T13:07:02.261 INFO:teuthology.orchestra.run.smithi019.stdout:(81/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 446 kB/s | 63 kB 00:00 2024-03-28T13:07:02.286 INFO:teuthology.orchestra.run.smithi019.stdout:(82/174): fuse-2.9.9-15.el9.x86_64.rpm 678 kB/s | 79 kB 00:00 2024-03-28T13:07:02.319 INFO:teuthology.orchestra.run.smithi019.stdout:(83/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.2 MB/s | 241 kB 00:00 2024-03-28T13:07:02.369 INFO:teuthology.orchestra.run.smithi019.stdout:(84/174): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 103 kB 00:00 2024-03-28T13:07:02.412 INFO:teuthology.orchestra.run.smithi019.stdout:(85/174): python3-cryptography-36.0.1-4.el9.x86 9.4 MB/s | 1.1 MB 00:00 2024-03-28T13:07:02.437 INFO:teuthology.orchestra.run.smithi019.stdout:(86/174): python3-pycparser-2.20-6.el9.noarch.r 1.0 MB/s | 124 kB 00:00 2024-03-28T13:07:02.462 INFO:teuthology.orchestra.run.smithi019.stdout:(87/174): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 187 kB 00:00 2024-03-28T13:07:02.871 INFO:teuthology.orchestra.run.smithi019.stdout:(88/174): libquadmath-11.4.1-3.el9.x86_64.rpm 478 kB/s | 195 kB 00:00 2024-03-28T13:07:02.926 INFO:teuthology.orchestra.run.smithi019.stdout:(89/174): jq-1.6-16.el9.x86_64.rpm 384 kB/s | 187 kB 00:00 2024-03-28T13:07:02.979 INFO:teuthology.orchestra.run.smithi019.stdout:(90/174): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-03-28T13:07:02.989 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-28T13:07:03.030 INFO:teuthology.orchestra.run.smithi019.stdout:(91/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 803 kB 00:00 2024-03-28T13:07:03.072 INFO:teuthology.orchestra.run.smithi019.stdout:(92/174): elfutils-libelf-devel-0.190-2.el9.x86 149 kB/s | 21 kB 00:00 2024-03-28T13:07:03.114 INFO:teuthology.orchestra.run.smithi019.stdout:(93/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 191 kB/s | 25 kB 00:00 2024-03-28T13:07:03.128 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-28T13:07:03.514 INFO:teuthology.orchestra.run.smithi019.stdout:(94/174): postgresql-contrib-13.14-1.el9.x86_64 2.2 MB/s | 911 kB 00:00 2024-03-28T13:07:03.739 INFO:teuthology.orchestra.run.smithi019.stdout:(95/174): postgresql-private-libs-13.14-1.el9.x 595 kB/s | 133 kB 00:00 2024-03-28T13:07:03.973 INFO:teuthology.orchestra.run.smithi019.stdout:(96/174): postgresql-13.14-1.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-03-28T13:07:04.281 INFO:teuthology.orchestra.run.smithi019.stdout:(97/174): openssl-devel-3.0.7-27.el9.x86_64.rpm 3.3 MB/s | 4.1 MB 00:01 2024-03-28T13:07:04.315 INFO:teuthology.orchestra.run.smithi019.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 720 kB/s | 245 kB 00:00 2024-03-28T13:07:04.415 INFO:teuthology.orchestra.run.smithi019.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-03-28T13:07:04.516 INFO:teuthology.orchestra.run.smithi019.stdout:(100/174): qatlib-24.02.0-1.el9.x86_64.rpm 1.1 MB/s | 221 kB 00:00 2024-03-28T13:07:04.566 INFO:teuthology.orchestra.run.smithi019.stdout:(101/174): qatlib-service-24.02.0-1.el9.x86_64. 240 kB/s | 36 kB 00:00 2024-03-28T13:07:04.642 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-28T13:07:04.708 INFO:teuthology.orchestra.run.smithi019.stdout:(102/174): qatzip-libs-1.2.0-1.el9.x86_64.rpm 248 kB/s | 47 kB 00:00 2024-03-28T13:07:04.737 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-28T13:07:04.758 INFO:teuthology.orchestra.run.smithi019.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 490 kB/s | 94 kB 00:00 2024-03-28T13:07:04.776 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-28T13:07:04.807 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-28T13:07:04.848 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-28T13:07:04.875 INFO:teuthology.orchestra.run.smithi019.stdout:(104/174): postgresql-server-13.14-1.el9.x86_64 5.1 MB/s | 5.8 MB 00:01 2024-03-28T13:07:05.228 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-28T13:07:05.253 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:19.0.0-2499.ge0e48ba6.el9.x86_64 124/198 2024-03-28T13:07:05.373 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:19.0.0-2499.ge0e48ba6.el9.x86_64 124/198 2024-03-28T13:07:05.401 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 125/198 2024-03-28T13:07:05.417 INFO:teuthology.orchestra.run.smithi019.stdout:(105/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-03-28T13:07:05.454 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-28T13:07:05.509 INFO:teuthology.orchestra.run.smithi019.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-03-28T13:07:05.551 INFO:teuthology.orchestra.run.smithi019.stdout:(107/174): lua-devel-5.4.4-4.el9.x86_64.rpm 26 kB/s | 22 kB 00:00 2024-03-28T13:07:05.593 INFO:teuthology.orchestra.run.smithi019.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 240 kB/s | 19 kB 00:00 2024-03-28T13:07:05.710 INFO:teuthology.orchestra.run.smithi019.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 224 kB/s | 26 kB 00:00 2024-03-28T13:07:05.802 INFO:teuthology.orchestra.run.smithi019.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 549 kB/s | 49 kB 00:00 2024-03-28T13:07:05.913 INFO:teuthology.orchestra.run.smithi019.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 613 kB/s | 67 kB 00:00 2024-03-28T13:07:05.986 INFO:teuthology.orchestra.run.smithi019.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.1 MB/s | 151 kB 00:00 2024-03-28T13:07:06.016 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-28T13:07:06.111 INFO:teuthology.orchestra.run.smithi019.stdout:(113/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.6 MB/s | 838 kB 00:00 2024-03-28T13:07:06.253 INFO:teuthology.orchestra.run.smithi019.stdout:(114/174): libarrow-9.0.0-10.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2024-03-28T13:07:06.286 INFO:teuthology.orchestra.run.smithi019.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 3.1 MB/s | 548 kB 00:00 2024-03-28T13:07:06.298 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 128/198 2024-03-28T13:07:06.370 INFO:teuthology.orchestra.run.smithi019.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 375 kB/s | 43 kB 00:00 2024-03-28T13:07:06.406 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 128/198 2024-03-28T13:07:06.421 INFO:teuthology.orchestra.run.smithi019.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 242 kB/s | 32 kB 00:00 2024-03-28T13:07:06.434 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:19.0.0-2499.ge0e48ba6.el9.x86_64 129/198 2024-03-28T13:07:06.471 INFO:teuthology.orchestra.run.smithi019.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 151 kB/s | 15 kB 00:00 2024-03-28T13:07:06.504 INFO:teuthology.orchestra.run.smithi019.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-03-28T13:07:06.563 INFO:teuthology.orchestra.run.smithi019.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-03-28T13:07:06.597 INFO:teuthology.orchestra.run.smithi019.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-03-28T13:07:06.639 INFO:teuthology.orchestra.run.smithi019.stdout:(122/174): protobuf-compiler-3.14.0-13.el9.x86_ 459 kB/s | 863 kB 00:01 2024-03-28T13:07:06.689 INFO:teuthology.orchestra.run.smithi019.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 283 kB/s | 35 kB 00:00 2024-03-28T13:07:06.806 INFO:teuthology.orchestra.run.smithi019.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 983 kB/s | 203 kB 00:00 2024-03-28T13:07:06.839 INFO:teuthology.orchestra.run.smithi019.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.8 MB/s | 144 kB 00:00 2024-03-28T13:07:06.890 INFO:teuthology.orchestra.run.smithi019.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 134 kB/s | 11 kB 00:00 2024-03-28T13:07:06.932 INFO:teuthology.orchestra.run.smithi019.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 193 kB/s | 18 kB 00:00 2024-03-28T13:07:06.974 INFO:teuthology.orchestra.run.smithi019.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 279 kB/s | 23 kB 00:00 2024-03-28T13:07:07.016 INFO:teuthology.orchestra.run.smithi019.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 235 kB/s | 19 kB 00:00 2024-03-28T13:07:07.058 INFO:teuthology.orchestra.run.smithi019.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 237 kB/s | 20 kB 00:00 2024-03-28T13:07:07.133 INFO:teuthology.orchestra.run.smithi019.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 8.8 MB/s | 1.0 MB 00:00 2024-03-28T13:07:07.175 INFO:teuthology.orchestra.run.smithi019.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 398 kB/s | 46 kB 00:00 2024-03-28T13:07:07.217 INFO:teuthology.orchestra.run.smithi019.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 954 kB/s | 79 kB 00:00 2024-03-28T13:07:07.259 INFO:teuthology.orchestra.run.smithi019.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 697 kB/s | 58 kB 00:00 2024-03-28T13:07:07.301 INFO:teuthology.orchestra.run.smithi019.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-03-28T13:07:07.343 INFO:teuthology.orchestra.run.smithi019.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 198 kB/s | 16 kB 00:00 2024-03-28T13:07:07.377 INFO:teuthology.orchestra.run.smithi019.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-03-28T13:07:07.427 INFO:teuthology.orchestra.run.smithi019.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 340 kB/s | 28 kB 00:00 2024-03-28T13:07:07.469 INFO:teuthology.orchestra.run.smithi019.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 336 kB/s | 31 kB 00:00 2024-03-28T13:07:07.503 INFO:teuthology.orchestra.run.smithi019.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-03-28T13:07:07.570 INFO:teuthology.orchestra.run.smithi019.stdout:(141/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.2 MB/s | 2.0 MB 00:00 2024-03-28T13:07:07.645 INFO:teuthology.orchestra.run.smithi019.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 253 kB/s | 36 kB 00:00 2024-03-28T13:07:07.681 INFO:teuthology.orchestra.run.smithi019.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 284 kB/s | 59 kB 00:00 2024-03-28T13:07:07.704 INFO:teuthology.orchestra.run.smithi019.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 384 kB/s | 51 kB 00:00 2024-03-28T13:07:07.737 INFO:teuthology.orchestra.run.smithi019.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 2.5 MB/s | 230 kB 00:00 2024-03-28T13:07:07.763 INFO:teuthology.orchestra.run.smithi019.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2024-03-28T13:07:07.805 INFO:teuthology.orchestra.run.smithi019.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-28T13:07:07.830 INFO:teuthology.orchestra.run.smithi019.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2024-03-28T13:07:07.855 INFO:teuthology.orchestra.run.smithi019.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-03-28T13:07:07.980 INFO:teuthology.orchestra.run.smithi019.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-03-28T13:07:08.039 INFO:teuthology.orchestra.run.smithi019.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 5.7 MB/s | 321 kB 00:00 2024-03-28T13:07:08.406 INFO:teuthology.orchestra.run.smithi019.stdout:(152/174): librados2-19.0.0-2499.ge0e48ba6.el9. 5.9 MB/s | 3.4 MB 00:00 2024-03-28T13:07:08.515 INFO:teuthology.orchestra.run.smithi019.stdout:(153/174): librbd1-19.0.0-2499.ge0e48ba6.el9.x8 4.8 MB/s | 3.1 MB 00:00 2024-03-28T13:07:08.657 INFO:teuthology.orchestra.run.smithi019.stdout:(154/174): NetworkManager-1.47.2-1.el9.x86_64.r 3.7 MB/s | 2.3 MB 00:00 2024-03-28T13:07:08.790 INFO:teuthology.orchestra.run.smithi019.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 3.2 MB/s | 248 kB 00:00 2024-03-28T13:07:08.816 INFO:teuthology.orchestra.run.smithi019.stdout:(156/174): NetworkManager-team-1.47.2-1.el9.x86 140 kB/s | 42 kB 00:00 2024-03-28T13:07:08.866 INFO:teuthology.orchestra.run.smithi019.stdout:(157/174): acl-2.3.1-4.el9.x86_64.rpm 957 kB/s | 71 kB 00:00 2024-03-28T13:07:08.931 INFO:teuthology.orchestra.run.smithi019.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 319 kB/s | 36 kB 00:00 2024-03-28T13:07:09.000 INFO:teuthology.orchestra.run.smithi019.stdout:(159/174): NetworkManager-libnm-1.47.2-1.el9.x8 3.1 MB/s | 1.8 MB 00:00 2024-03-28T13:07:09.034 INFO:teuthology.orchestra.run.smithi019.stdout:(160/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.1 MB/s | 192 kB 00:00 2024-03-28T13:07:09.067 INFO:teuthology.orchestra.run.smithi019.stdout:(161/174): elfutils-libs-0.190-2.el9.x86_64.rpm 1.9 MB/s | 256 kB 00:00 2024-03-28T13:07:09.076 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2499.ge0e48ba6.el9.x86_64 130/198 2024-03-28T13:07:09.092 INFO:teuthology.orchestra.run.smithi019.stdout:(162/174): libacl-2.3.1-4.el9.x86_64.rpm 252 kB/s | 23 kB 00:00 2024-03-28T13:07:09.114 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:19.0.0-2499.ge0e48ba6.el9.x86_64 130/198 2024-03-28T13:07:09.126 INFO:teuthology.orchestra.run.smithi019.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.2 MB/s | 108 kB 00:00 2024-03-28T13:07:09.159 INFO:teuthology.orchestra.run.smithi019.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.7 MB/s | 155 kB 00:00 2024-03-28T13:07:09.193 INFO:teuthology.orchestra.run.smithi019.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 136 kB 00:00 2024-03-28T13:07:09.218 INFO:teuthology.orchestra.run.smithi019.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 700 kB/s | 64 kB 00:00 2024-03-28T13:07:09.253 INFO:teuthology.orchestra.run.smithi019.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 314 kB/s | 29 kB 00:00 2024-03-28T13:07:09.418 INFO:teuthology.orchestra.run.smithi019.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 5.4 MB/s | 1.2 MB 00:00 2024-03-28T13:07:09.678 INFO:teuthology.orchestra.run.smithi019.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 1.8 MB/s | 465 kB 00:00 2024-03-28T13:07:09.737 INFO:teuthology.orchestra.run.smithi019.stdout:(170/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.2 MB/s | 2.2 MB 00:00 2024-03-28T13:07:09.738 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2499.ge0e48ba6.el9.x86_64 130/198 2024-03-28T13:07:09.784 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:19.0.0-2499.ge0e48ba6.el9.x86_64 131/198 2024-03-28T13:07:09.804 INFO:teuthology.orchestra.run.smithi019.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 250 kB/s | 31 kB 00:00 2024-03-28T13:07:09.887 INFO:teuthology.orchestra.run.smithi019.stdout:(172/174): util-linux-2.37.4-18.el9.x86_64.rpm 3.6 MB/s | 2.3 MB 00:00 2024-03-28T13:07:09.946 INFO:teuthology.orchestra.run.smithi019.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 5.2 MB/s | 1.1 MB 00:00 2024-03-28T13:07:09.949 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:19.0.0-2499.ge0e48ba6.el9.x86_64 131/198 2024-03-28T13:07:09.950 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-28T13:07:09.950 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-28T13:07:09.950 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:07:09.984 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:19.0.0-2499.ge0e48ba6.el9.x86_64 132/198 2024-03-28T13:07:10.004 INFO:teuthology.orchestra.run.smithi019.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 373 kB/s | 74 kB 00:00 2024-03-28T13:07:10.023 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-03-28T13:07:10.024 INFO:teuthology.orchestra.run.smithi019.stdout:Total 13 MB/s | 227 MB 00:17 2024-03-28T13:07:11.713 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-03-28T13:07:12.119 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-03-28T13:07:12.119 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-03-28T13:07:14.663 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-03-28T13:07:14.664 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-03-28T13:07:22.698 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-03-28T13:07:22.820 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-28T13:07:22.860 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-28T13:07:22.898 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T13:07:22.913 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2499.ge0e48ba6.el9.x86_64 132/198 2024-03-28T13:07:22.913 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2024-03-28T13:07:22.913 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2024-03-28T13:07:22.913 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2024-03-28T13:07:22.913 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2024-03-28T13:07:22.913 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2024-03-28T13:07:22.913 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /root 2024-03-28T13:07:22.913 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2024-03-28T13:07:22.914 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:07:22.942 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-28T13:07:23.029 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-28T13:07:23.116 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:19.0.0-2499.ge0e48ba6.el9.noarch 133/198 2024-03-28T13:07:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-28T13:07:23.165 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:19.0.0-2499.ge0e48ba6.el9.noarch 133/198 2024-03-28T13:07:23.165 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:07:23.165 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-28T13:07:23.165 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:07:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-28T13:07:23.248 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-28T13:07:23.346 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-28T13:07:23.772 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-28T13:07:23.823 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 10/198 2024-03-28T13:07:23.988 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 10/198 2024-03-28T13:07:24.024 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 11/198 2024-03-28T13:07:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 11/198 2024-03-28T13:07:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.0.0-2499.ge0e48ba6.el9.x86_64 12/198 2024-03-28T13:07:24.241 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 13/198 2024-03-28T13:07:24.542 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-28T13:07:24.595 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-13.14-1.el9.x86_64 15/198 2024-03-28T13:07:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.0.0-2499.ge0e48ba6.el9.x86_64 16/198 2024-03-28T13:07:24.727 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 134/198 2024-03-28T13:07:24.779 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2499.ge0e48ba6.el9.x86_64 16/198 2024-03-28T13:07:24.805 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.0.0-2499.ge0e48ba6.el9.x86 17/198 2024-03-28T13:07:24.812 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 134/198 2024-03-28T13:07:24.812 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:07:24.812 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-28T13:07:24.812 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T13:07:24.813 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T13:07:24.813 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:07:24.904 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2499.ge0e48ba6.el9.x86 17/198 2024-03-28T13:07:24.949 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-28T13:07:24.982 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T13:07:25.046 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 19/198 2024-03-28T13:07:25.077 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:19.0.0-2499.ge0e48ba6.el9.x86_64 135/198 2024-03-28T13:07:25.145 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:19.0.0-2499.ge0e48ba6.el9.x86_64 135/198 2024-03-28T13:07:25.145 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:07:25.145 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-28T13:07:25.145 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T13:07:25.145 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T13:07:25.145 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:07:25.379 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T13:07:25.424 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 20/198 2024-03-28T13:07:25.482 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-28T13:07:25.619 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:19.0.0-2499.ge0e48ba6.el9.x86_64 136/198 2024-03-28T13:07:25.687 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:19.0.0-2499.ge0e48ba6.el9.x86_64 136/198 2024-03-28T13:07:25.688 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:07:25.688 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-28T13:07:25.688 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T13:07:25.688 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T13:07:25.688 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:07:25.797 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-28T13:07:25.841 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-28T13:07:25.989 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T13:07:26.081 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-28T13:07:26.290 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T13:07:26.342 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T13:07:26.448 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:19.0.0-2499.ge0e48ba6.el9.noarch 138/198 2024-03-28T13:07:26.498 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:19.0.0-2499.ge0e48ba6.el9.noarch 138/198 2024-03-28T13:07:26.550 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2499.ge0e48ba6.e 139/198 2024-03-28T13:07:26.651 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2499.ge0e48ba6. 140/198 2024-03-28T13:07:26.697 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-28T13:07:26.710 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2499.ge0e48ba6.el9.noa 141/198 2024-03-28T13:07:26.883 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-28T13:07:27.010 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-28T13:07:27.048 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-28T13:07:27.084 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-28T13:07:27.141 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-28T13:07:27.261 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-28T13:07:27.331 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-28T13:07:27.408 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-28T13:07:27.515 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-28T13:07:27.571 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-28T13:07:27.640 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-28T13:07:27.693 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-28T13:07:27.733 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-28T13:07:27.857 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-28T13:07:27.902 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.0.0-2499.ge0e48ba6.el9. 39/198 2024-03-28T13:07:27.970 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.0.0-2499.ge0e48ba6.el 40/198 2024-03-28T13:07:28.006 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2499.ge0e48ba6.el9.noa 141/198 2024-03-28T13:07:28.010 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.0.0-2499.ge0e48ba6.el9.x86_6 41/198 2024-03-28T13:07:28.045 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2499.ge0e48ba6.el9.n 142/198 2024-03-28T13:07:28.137 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-28T13:07:28.196 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-28T13:07:28.336 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-28T13:07:28.527 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-28T13:07:28.599 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-28T13:07:29.127 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2499.ge0e48ba6.el9.n 142/198 2024-03-28T13:07:29.173 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2499.ge0e 143/198 2024-03-28T13:07:29.317 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2499.ge0e 143/198 2024-03-28T13:07:29.323 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-28T13:07:29.427 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-28T13:07:29.501 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2499.ge0e48ba6.el 144/198 2024-03-28T13:07:29.531 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:19.0.0-2499.ge0e48ba6.el9.x86_64 145/198 2024-03-28T13:07:29.546 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-28T13:07:29.597 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2499.ge0e48ba6.el9.x86_64 145/198 2024-03-28T13:07:29.598 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:07:29.598 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-28T13:07:29.598 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T13:07:29.598 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T13:07:29.598 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:07:29.613 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T13:07:29.641 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-28T13:07:29.657 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:19.0.0-2499.ge0e48ba6.el9.noarch 146/198 2024-03-28T13:07:29.694 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 51/198 2024-03-28T13:07:29.695 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2499.ge0e48ba6.el9.noarch 146/198 2024-03-28T13:07:29.766 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-28T13:07:29.793 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:19.0.0-2499.ge0e48ba6.el9.x86_64 147/198 2024-03-28T13:07:29.825 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.0.0-2499.ge0e48ba6.el9.x86_6 53/198 2024-03-28T13:07:29.871 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-28T13:07:30.296 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-28T13:07:30.523 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-28T13:07:30.608 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-28T13:07:30.674 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-28T13:07:30.757 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-28T13:07:30.792 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:19.0.0-2499.ge0e48ba6.el9.x86_64 149/198 2024-03-28T13:07:30.860 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2499.ge0e48ba6.el9.x86_64 149/198 2024-03-28T13:07:30.860 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:07:30.860 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-28T13:07:30.860 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T13:07:30.860 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T13:07:30.860 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:07:30.889 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-28T13:07:30.909 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2499.ge0e48 150/198 2024-03-28T13:07:30.964 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-28T13:07:30.981 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2499.ge0e48 150/198 2024-03-28T13:07:30.982 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:07:30.982 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-28T13:07:30.982 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:07:31.038 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-28T13:07:31.060 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 151/198 2024-03-28T13:07:31.133 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 151/198 2024-03-28T13:07:31.133 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:07:31.133 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-28T13:07:31.133 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T13:07:31.134 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T13:07:31.134 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:07:31.135 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-28T13:07:31.171 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-28T13:07:31.219 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-28T13:07:31.290 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-28T13:07:31.337 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-28T13:07:31.402 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-28T13:07:31.468 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-28T13:07:31.494 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 152/198 2024-03-28T13:07:31.572 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 152/198 2024-03-28T13:07:31.572 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:07:31.572 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-28T13:07:31.572 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T13:07:31.572 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T13:07:31.572 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:07:31.672 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-28T13:07:31.751 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-28T13:07:31.790 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-28T13:07:31.850 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-28T13:07:31.996 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-28T13:07:32.037 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-28T13:07:32.084 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-28T13:07:32.139 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-28T13:07:32.179 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-28T13:07:32.363 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-28T13:07:32.540 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-28T13:07:32.681 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-28T13:07:32.738 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-28T13:07:32.801 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-28T13:07:32.886 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-28T13:07:33.136 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-28T13:07:33.193 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-28T13:07:33.371 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-28T13:07:33.439 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-28T13:07:33.482 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-28T13:07:33.527 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-28T13:07:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-28T13:07:33.616 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-28T13:07:33.692 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-28T13:07:33.751 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-28T13:07:33.800 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-28T13:07:33.991 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-28T13:07:34.036 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-28T13:07:34.088 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-28T13:07:34.155 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-28T13:07:34.200 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-28T13:07:34.354 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-28T13:07:34.477 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-28T13:07:34.528 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-28T13:07:34.563 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-28T13:07:34.620 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-28T13:07:34.762 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-28T13:07:34.838 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-28T13:07:35.859 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-28T13:07:35.912 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-28T13:07:35.945 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-28T13:07:36.003 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-28T13:07:36.118 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-28T13:07:36.179 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-28T13:07:36.350 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:19.0.0-2499.ge0e48ba6.el9.x86_64 153/198 2024-03-28T13:07:36.394 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:19.0.0-2499.ge0e48ba6.el9.x86_64 154/198 2024-03-28T13:07:36.429 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-28T13:07:36.431 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 155/198 2024-03-28T13:07:36.477 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-28T13:07:36.495 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-28T13:07:36.619 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-28T13:07:36.743 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-shell-2:19.0.0-2499.ge0e48ba6.el9.x86_64 157/198 2024-03-28T13:07:36.862 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-28T13:07:36.918 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:19.0.0-2499.ge0e48ba6.el9.x86_64 159/198 2024-03-28T13:07:36.973 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-28T13:07:37.034 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-28T13:07:37.105 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-28T13:07:37.147 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:19.0.0-2499.ge0e48ba6.el9.x86_ 163/198 2024-03-28T13:07:37.194 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-28T13:07:37.244 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-28T13:07:37.330 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-28T13:07:37.445 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-28T13:07:37.481 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-28T13:07:37.518 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-28T13:07:37.626 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-28T13:07:38.491 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T13:07:38.533 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T13:07:38.589 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T13:07:39.228 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-28T13:07:39.330 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-28T13:07:39.368 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-28T13:07:39.398 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-28T13:07:39.480 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-28T13:07:39.889 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:19.0.0-2499.ge0e48ba6.el9.noarch 170/198 2024-03-28T13:07:39.905 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-28T13:07:39.937 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.0.0-2499.ge0e48ba6.el9.x86_64 124/198 2024-03-28T13:07:40.006 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-28T13:07:40.059 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.0.0-2499.ge0e48ba6.el9.x86_64 124/198 2024-03-28T13:07:40.070 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-28T13:07:40.093 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 125/198 2024-03-28T13:07:40.149 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-28T13:07:40.202 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-28T13:07:40.241 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-28T13:07:40.274 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-28T13:07:40.333 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-28T13:07:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-28T13:07:40.386 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T13:07:40.546 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T13:07:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-28T13:07:40.607 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T13:07:40.718 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-28T13:07:40.756 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 128/198 2024-03-28T13:07:40.759 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T13:07:40.803 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-28T13:07:40.850 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-28T13:07:40.857 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 128/198 2024-03-28T13:07:40.883 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.0.0-2499.ge0e48ba6.el9.x86_64 129/198 2024-03-28T13:07:40.891 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-28T13:07:40.923 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-28T13:07:40.997 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-28T13:07:41.071 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-28T13:07:41.119 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-28T13:07:41.153 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-28T13:07:41.187 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-28T13:07:41.249 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-28T13:07:41.287 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-28T13:07:41.323 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T13:07:41.323 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T13:07:41.677 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T13:07:41.677 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T13:07:41.810 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T13:07:41.840 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-28T13:07:41.894 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-28T13:07:41.927 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-28T13:07:41.966 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-28T13:07:42.085 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T13:07:42.347 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T13:07:42.347 INFO:teuthology.orchestra.run.smithi188.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-28T13:07:42.348 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:07:43.623 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-2499.ge0e48ba6.el9.x86_64 130/198 2024-03-28T13:07:43.671 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.0.0-2499.ge0e48ba6.el9.x86_64 130/198 2024-03-28T13:07:44.341 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.0.0-2499.ge0e48ba6.el9.x86_64 130/198 2024-03-28T13:07:44.383 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.0.0-2499.ge0e48ba6.el9.x86_64 131/198 2024-03-28T13:07:44.537 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.0.0-2499.ge0e48ba6.el9.x86_64 131/198 2024-03-28T13:07:44.537 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-28T13:07:44.537 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-28T13:07:44.538 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:07:44.609 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.0.0-2499.ge0e48ba6.el9.x86_64 132/198 2024-03-28T13:07:47.861 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:19.0.0-2499.ge0e48ba6.el9.x86_64 1/198 2024-03-28T13:07:47.861 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2/198 2024-03-28T13:07:47.861 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:19.0.0-2499.ge0e48ba6.el9.x86_64 3/198 2024-03-28T13:07:47.861 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:19.0.0-2499.ge0e48ba6.el9.x86_64 4/198 2024-03-28T13:07:47.861 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2499.ge0e48 5/198 2024-03-28T13:07:47.861 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:19.0.0-2499.ge0e48ba6.el9.x86_64 6/198 2024-03-28T13:07:47.861 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:19.0.0-2499.ge0e48ba6.el9.x86_64 7/198 2024-03-28T13:07:47.862 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:19.0.0-2499.ge0e48ba6.el9.x86_64 8/198 2024-03-28T13:07:47.862 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 9/198 2024-03-28T13:07:47.862 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:19.0.0-2499.ge0e48ba6.el9.x86_64 10/198 2024-03-28T13:07:47.862 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:19.0.0-2499.ge0e48ba6.el9.x86_64 11/198 2024-03-28T13:07:47.862 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:19.0.0-2499.ge0e48ba6.el9.x86_64 12/198 2024-03-28T13:07:47.862 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 13/198 2024-03-28T13:07:47.862 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-shell-2:19.0.0-2499.ge0e48ba6.el9.x86_64 14/198 2024-03-28T13:07:47.862 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:19.0.0-2499.ge0e48ba6.el9.x86_ 15/198 2024-03-28T13:07:47.862 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 16/198 2024-03-28T13:07:47.862 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:19.0.0-2499.ge0e48ba6.el9.x86_64 17/198 2024-03-28T13:07:47.862 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:19.0.0-2499.ge0e48ba6.el9.x86_6 18/198 2024-03-28T13:07:47.862 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:19.0.0-2499.ge0e48ba6.el9.x86 19/198 2024-03-28T13:07:47.862 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 20/198 2024-03-28T13:07:47.863 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2499.ge0e48ba6.el 21/198 2024-03-28T13:07:47.863 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:19.0.0-2499.ge0e48ba6.el9. 22/198 2024-03-28T13:07:47.863 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:19.0.0-2499.ge0e48ba6.el9.x86_6 23/198 2024-03-28T13:07:47.863 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:19.0.0-2499.ge0e48ba6.el9.x86_64 24/198 2024-03-28T13:07:47.863 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 25/198 2024-03-28T13:07:47.863 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:19.0.0-2499.ge0e48ba6.el9.x86_64 26/198 2024-03-28T13:07:47.863 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:19.0.0-2499.ge0e48ba6.el9.x86_64 27/198 2024-03-28T13:07:47.879 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 28/198 2024-03-28T13:07:47.879 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 29/198 2024-03-28T13:07:47.879 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2499.ge0e48ba6. 30/198 2024-03-28T13:07:47.879 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2499.ge0e48ba6.el9.noa 31/198 2024-03-28T13:07:47.879 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2499.ge0e48ba6.el9.n 32/198 2024-03-28T13:07:47.879 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2499.ge0e 33/198 2024-03-28T13:07:47.879 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2499.ge0e48ba6.el 34/198 2024-03-28T13:07:47.879 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2499.ge0e48ba6.el9.noarch 35/198 2024-03-28T13:07:47.879 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2499.ge0e48ba6.e 36/198 2024-03-28T13:07:47.879 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:19.0.0-2499.ge0e48ba6.el9.noarch 37/198 2024-03-28T13:07:47.880 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:19.0.0-2499.ge0e48ba6.el9.noarch 38/198 2024-03-28T13:07:47.880 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:19.0.0-2499.ge0e48ba6.el9.noarch 39/198 2024-03-28T13:07:47.880 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-28T13:07:47.880 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-28T13:07:47.880 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-28T13:07:47.880 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-28T13:07:47.880 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-28T13:07:47.880 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-28T13:07:47.880 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-28T13:07:47.880 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-28T13:07:47.880 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-28T13:07:47.880 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-28T13:07:47.880 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-28T13:07:47.880 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-28T13:07:47.880 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-28T13:07:47.881 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-28T13:07:47.881 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-28T13:07:47.881 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-28T13:07:47.881 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-28T13:07:47.881 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-28T13:07:47.881 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-28T13:07:47.881 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-28T13:07:47.881 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-28T13:07:47.881 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-28T13:07:47.881 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-28T13:07:47.881 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-28T13:07:47.881 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-28T13:07:47.881 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-28T13:07:47.882 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-28T13:07:47.882 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-28T13:07:47.882 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-28T13:07:47.882 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-28T13:07:47.882 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-28T13:07:47.882 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-28T13:07:47.882 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-28T13:07:47.882 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-28T13:07:47.882 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-28T13:07:47.882 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-28T13:07:47.882 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-28T13:07:47.882 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-28T13:07:47.882 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-28T13:07:47.883 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-28T13:07:47.883 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-28T13:07:47.883 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-28T13:07:47.883 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-28T13:07:47.883 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-28T13:07:47.883 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-28T13:07:47.883 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-28T13:07:47.883 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-28T13:07:47.883 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-28T13:07:47.883 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-28T13:07:47.883 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-28T13:07:47.883 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-28T13:07:47.883 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-28T13:07:47.884 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-28T13:07:47.884 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-28T13:07:47.884 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-28T13:07:47.884 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-28T13:07:47.884 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-28T13:07:47.884 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-28T13:07:47.884 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-28T13:07:47.884 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-28T13:07:47.884 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-28T13:07:47.885 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-28T13:07:47.885 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-28T13:07:47.885 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-28T13:07:47.885 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-28T13:07:47.885 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-28T13:07:47.885 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-28T13:07:47.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-28T13:07:47.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-28T13:07:47.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-28T13:07:47.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-28T13:07:47.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-28T13:07:47.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-28T13:07:47.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-28T13:07:47.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-28T13:07:47.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-28T13:07:47.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-28T13:07:47.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-28T13:07:47.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-28T13:07:47.886 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-28T13:07:47.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-28T13:07:47.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-28T13:07:47.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-28T13:07:47.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-28T13:07:47.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-28T13:07:47.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-28T13:07:47.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-28T13:07:47.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-28T13:07:47.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-28T13:07:47.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-28T13:07:47.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-28T13:07:47.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-28T13:07:47.887 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-28T13:07:47.888 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-28T13:07:47.888 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-28T13:07:47.888 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-28T13:07:47.888 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-28T13:07:47.888 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-28T13:07:47.888 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-28T13:07:47.888 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-28T13:07:47.888 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-28T13:07:47.888 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-28T13:07:47.888 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-28T13:07:47.888 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-28T13:07:47.888 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-28T13:07:47.888 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-28T13:07:47.888 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-28T13:07:47.889 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-28T13:07:47.889 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-28T13:07:47.889 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-28T13:07:47.889 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-28T13:07:47.889 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 151/198 2024-03-28T13:07:47.889 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-28T13:07:47.889 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:19.0.0-2499.ge0e48ba6.el9.x86_64 153/198 2024-03-28T13:07:47.889 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-28T13:07:47.889 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-28T13:07:47.889 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-28T13:07:47.889 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-28T13:07:47.889 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-28T13:07:47.889 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-28T13:07:47.889 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-28T13:07:47.890 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-28T13:07:47.890 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-28T13:07:47.890 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-28T13:07:47.890 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-28T13:07:47.890 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-28T13:07:47.890 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-28T13:07:47.890 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-28T13:07:47.890 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-28T13:07:47.890 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-28T13:07:47.890 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-28T13:07:47.890 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-28T13:07:47.890 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-28T13:07:47.890 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-28T13:07:47.890 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-28T13:07:47.891 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-28T13:07:47.891 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-28T13:07:47.891 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-28T13:07:47.891 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-28T13:07:47.891 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-28T13:07:47.891 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-28T13:07:47.891 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-28T13:07:47.891 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-28T13:07:47.891 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-28T13:07:47.891 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-28T13:07:47.891 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-28T13:07:47.891 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-28T13:07:47.891 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-28T13:07:47.891 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-28T13:07:47.892 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-28T13:07:47.892 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-28T13:07:47.892 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-28T13:07:47.892 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-28T13:07:47.892 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-28T13:07:47.892 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-28T13:07:47.892 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-28T13:07:47.892 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-28T13:07:47.892 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-28T13:07:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-28T13:07:50.548 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:07:50.549 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-03-28T13:07:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-28T13:07:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-28T13:07:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-28T13:07:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-28T13:07:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: acl-2.3.1-4.el9.x86_64 2024-03-28T13:07:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-28T13:07:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-28T13:07:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-28T13:07:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-28T13:07:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-28T13:07:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-28T13:07:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-28T13:07:50.549 INFO:teuthology.orchestra.run.smithi188.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-28T13:07:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-28T13:07:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-28T13:07:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-28T13:07:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-28T13:07:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-28T13:07:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-28T13:07:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-28T13:07:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-28T13:07:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-28T13:07:50.550 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-03-28T13:07:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-28T13:07:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.2-8.el9.noarch 2024-03-28T13:07:50.550 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.7.4-5.el9.x86_64 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.551 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.4-9.el9.x86_64 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-28T13:07:50.552 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: jq-1.6-16.el9.x86_64 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-28T13:07:50.553 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-28T13:07:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-28T13:07:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-28T13:07:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-28T13:07:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-28T13:07:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: lua-5.4.4-4.el9.x86_64 2024-03-28T13:07:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-28T13:07:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-28T13:07:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-28T13:07:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-28T13:07:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-28T13:07:50.554 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-28T13:07:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-28T13:07:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-13.14-1.el9.x86_64 2024-03-28T13:07:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-28T13:07:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-28T13:07:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-28T13:07:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-28T13:07:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-28T13:07:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-28T13:07:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-28T13:07:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-28T13:07:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-28T13:07:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-28T13:07:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.555 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-28T13:07:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-28T13:07:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-28T13:07:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-28T13:07:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-28T13:07:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-28T13:07:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-28T13:07:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-28T13:07:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-28T13:07:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-28T13:07:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-28T13:07:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-28T13:07:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-28T13:07:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-28T13:07:50.556 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-28T13:07:50.557 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply-3.11-14.el9.noarch 2024-03-28T13:07:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-28T13:07:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-28T13:07:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-28T13:07:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-28T13:07:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-28T13:07:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-28T13:07:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-28T13:07:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-28T13:07:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-28T13:07:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-28T13:07:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-28T13:07:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-28T13:07:50.558 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-28T13:07:50.559 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:07:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: re2-1:20211101-3.el9.x86_64 2024-03-28T13:07:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-28T13:07:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-28T13:07:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-28T13:07:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-28T13:07:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-28T13:07:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-28T13:07:50.560 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:07:50.560 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-03-28T13:07:51.273 DEBUG:teuthology.parallel:result is None 2024-03-28T13:07:58.210 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2499.ge0e48ba6.el9.x86_64 132/198 2024-03-28T13:07:58.210 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-03-28T13:07:58.210 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-03-28T13:07:58.210 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-03-28T13:07:58.210 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-03-28T13:07:58.211 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-03-28T13:07:58.211 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-03-28T13:07:58.211 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-03-28T13:07:58.211 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:07:58.423 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.0.0-2499.ge0e48ba6.el9.noarch 133/198 2024-03-28T13:07:58.473 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.0.0-2499.ge0e48ba6.el9.noarch 133/198 2024-03-28T13:07:58.473 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:07:58.473 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-28T13:07:58.473 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:00.135 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 134/198 2024-03-28T13:08:00.227 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 134/198 2024-03-28T13:08:00.227 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:08:00.228 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-28T13:08:00.228 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T13:08:00.228 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-28T13:08:00.228 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:00.485 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.0.0-2499.ge0e48ba6.el9.x86_64 135/198 2024-03-28T13:08:00.554 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.0.0-2499.ge0e48ba6.el9.x86_64 135/198 2024-03-28T13:08:00.554 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:08:00.554 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-28T13:08:00.554 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T13:08:00.554 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-28T13:08:00.554 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:01.061 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.0.0-2499.ge0e48ba6.el9.x86_64 136/198 2024-03-28T13:08:01.133 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.0.0-2499.ge0e48ba6.el9.x86_64 136/198 2024-03-28T13:08:01.133 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:08:01.133 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-28T13:08:01.133 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T13:08:01.133 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-28T13:08:01.133 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:01.263 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-28T13:08:01.866 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.0.0-2499.ge0e48ba6.el9.noarch 138/198 2024-03-28T13:08:01.897 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.0.0-2499.ge0e48ba6.el9.noarch 138/198 2024-03-28T13:08:01.958 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2499.ge0e48ba6.e 139/198 2024-03-28T13:08:02.073 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2499.ge0e48ba6. 140/198 2024-03-28T13:08:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2499.ge0e48ba6.el9.noa 141/198 2024-03-28T13:08:03.457 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2499.ge0e48ba6.el9.noa 141/198 2024-03-28T13:08:03.503 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2499.ge0e48ba6.el9.n 142/198 2024-03-28T13:08:04.663 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2499.ge0e48ba6.el9.n 142/198 2024-03-28T13:08:04.695 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2499.ge0e 143/198 2024-03-28T13:08:04.843 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2499.ge0e 143/198 2024-03-28T13:08:05.034 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2499.ge0e48ba6.el 144/198 2024-03-28T13:08:05.072 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.0.0-2499.ge0e48ba6.el9.x86_64 145/198 2024-03-28T13:08:05.141 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2499.ge0e48ba6.el9.x86_64 145/198 2024-03-28T13:08:05.141 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:08:05.141 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-28T13:08:05.141 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T13:08:05.142 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-28T13:08:05.142 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:05.206 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.0.0-2499.ge0e48ba6.el9.noarch 146/198 2024-03-28T13:08:05.246 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2499.ge0e48ba6.el9.noarch 146/198 2024-03-28T13:08:05.342 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.0.0-2499.ge0e48ba6.el9.x86_64 147/198 2024-03-28T13:08:06.395 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-28T13:08:06.425 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.0.0-2499.ge0e48ba6.el9.x86_64 149/198 2024-03-28T13:08:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2499.ge0e48ba6.el9.x86_64 149/198 2024-03-28T13:08:06.503 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:08:06.503 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-28T13:08:06.503 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T13:08:06.503 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-28T13:08:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:06.559 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2499.ge0e48 150/198 2024-03-28T13:08:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2499.ge0e48 150/198 2024-03-28T13:08:06.635 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:08:06.635 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-28T13:08:06.635 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:06.718 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 151/198 2024-03-28T13:08:06.823 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 151/198 2024-03-28T13:08:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:08:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-28T13:08:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T13:08:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-28T13:08:06.823 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:07.218 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 152/198 2024-03-28T13:08:07.303 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 152/198 2024-03-28T13:08:07.304 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-28T13:08:07.304 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-28T13:08:07.304 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T13:08:07.304 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-28T13:08:07.304 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:12.450 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.0.0-2499.ge0e48ba6.el9.x86_64 153/198 2024-03-28T13:08:12.505 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.0.0-2499.ge0e48ba6.el9.x86_64 154/198 2024-03-28T13:08:12.540 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 155/198 2024-03-28T13:08:12.578 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-28T13:08:12.872 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-shell-2:19.0.0-2499.ge0e48ba6.el9.x86_64 157/198 2024-03-28T13:08:13.018 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 158/198 2024-03-28T13:08:13.066 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.0.0-2499.ge0e48ba6.el9.x86_64 159/198 2024-03-28T13:08:13.149 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-28T13:08:13.216 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-28T13:08:13.272 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-28T13:08:13.315 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.0.0-2499.ge0e48ba6.el9.x86_ 163/198 2024-03-28T13:08:13.366 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-28T13:08:13.418 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-28T13:08:13.499 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-28T13:08:13.638 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-28T13:08:13.683 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-28T13:08:14.640 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T13:08:14.687 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T13:08:14.757 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 169/198 2024-03-28T13:08:16.206 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-top-2:19.0.0-2499.ge0e48ba6.el9.noarch 170/198 2024-03-28T13:08:16.293 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-28T13:08:16.349 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-28T13:08:16.492 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-28T13:08:16.549 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-28T13:08:16.587 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-28T13:08:16.621 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-28T13:08:16.672 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-28T13:08:16.673 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T13:08:16.809 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-28T13:08:16.858 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-28T13:08:16.859 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T13:08:17.021 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-28T13:08:17.066 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-28T13:08:17.112 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-28T13:08:17.153 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-28T13:08:17.185 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-28T13:08:17.255 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-28T13:08:17.334 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-28T13:08:17.406 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-28T13:08:17.449 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-28T13:08:17.482 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-28T13:08:17.515 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-28T13:08:17.548 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-28T13:08:17.586 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T13:08:17.587 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T13:08:17.933 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-28T13:08:17.933 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T13:08:18.072 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-28T13:08:18.102 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-28T13:08:18.156 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-28T13:08:18.189 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-28T13:08:18.264 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-28T13:08:18.395 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T13:08:19.090 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-28T13:08:19.090 INFO:teuthology.orchestra.run.smithi019.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-03-28T13:08:19.090 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.0.0-2499.ge0e48ba6.el9.x86_64 1/198 2024-03-28T13:08:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2/198 2024-03-28T13:08:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.0.0-2499.ge0e48ba6.el9.x86_64 3/198 2024-03-28T13:08:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.0.0-2499.ge0e48ba6.el9.x86_64 4/198 2024-03-28T13:08:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2499.ge0e48 5/198 2024-03-28T13:08:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.0.0-2499.ge0e48ba6.el9.x86_64 6/198 2024-03-28T13:08:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.0.0-2499.ge0e48ba6.el9.x86_64 7/198 2024-03-28T13:08:24.276 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.0.0-2499.ge0e48ba6.el9.x86_64 8/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 9/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.0.0-2499.ge0e48ba6.el9.x86_64 10/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.0.0-2499.ge0e48ba6.el9.x86_64 11/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.0.0-2499.ge0e48ba6.el9.x86_64 12/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 13/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-shell-2:19.0.0-2499.ge0e48ba6.el9.x86_64 14/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.0.0-2499.ge0e48ba6.el9.x86_ 15/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 16/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.0.0-2499.ge0e48ba6.el9.x86_64 17/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.0.0-2499.ge0e48ba6.el9.x86_6 18/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.0.0-2499.ge0e48ba6.el9.x86 19/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 20/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2499.ge0e48ba6.el 21/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.0.0-2499.ge0e48ba6.el9. 22/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.0.0-2499.ge0e48ba6.el9.x86_6 23/198 2024-03-28T13:08:24.277 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.0.0-2499.ge0e48ba6.el9.x86_64 24/198 2024-03-28T13:08:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 25/198 2024-03-28T13:08:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.0.0-2499.ge0e48ba6.el9.x86_64 26/198 2024-03-28T13:08:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.0.0-2499.ge0e48ba6.el9.x86_64 27/198 2024-03-28T13:08:24.278 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 28/198 2024-03-28T13:08:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 29/198 2024-03-28T13:08:24.292 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2499.ge0e48ba6. 30/198 2024-03-28T13:08:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2499.ge0e48ba6.el9.noa 31/198 2024-03-28T13:08:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2499.ge0e48ba6.el9.n 32/198 2024-03-28T13:08:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2499.ge0e 33/198 2024-03-28T13:08:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2499.ge0e48ba6.el 34/198 2024-03-28T13:08:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2499.ge0e48ba6.el9.noarch 35/198 2024-03-28T13:08:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2499.ge0e48ba6.e 36/198 2024-03-28T13:08:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.0.0-2499.ge0e48ba6.el9.noarch 37/198 2024-03-28T13:08:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.0.0-2499.ge0e48ba6.el9.noarch 38/198 2024-03-28T13:08:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-top-2:19.0.0-2499.ge0e48ba6.el9.noarch 39/198 2024-03-28T13:08:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-28T13:08:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-28T13:08:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-28T13:08:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-28T13:08:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-28T13:08:24.293 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-28T13:08:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-28T13:08:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-28T13:08:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-28T13:08:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-28T13:08:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-28T13:08:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-28T13:08:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-28T13:08:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-28T13:08:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-28T13:08:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-28T13:08:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-28T13:08:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-28T13:08:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 58/198 2024-03-28T13:08:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 59/198 2024-03-28T13:08:24.294 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 60/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 61/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 62/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 63/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 64/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 65/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 66/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 67/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 68/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 70/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 71/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 72/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 73/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 74/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 75/198 2024-03-28T13:08:24.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 76/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 77/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 78/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 79/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 80/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 81/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 82/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 83/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 84/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 85/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 86/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 87/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : jq-1.6-16.el9.x86_64 88/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 89/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 90/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 91/198 2024-03-28T13:08:24.296 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 92/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 93/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 94/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-13.14-1.el9.x86_64 95/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 96/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 97/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 98/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 101/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 102/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 103/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-28T13:08:24.297 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-28T13:08:24.298 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-28T13:08:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-28T13:08:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-28T13:08:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-28T13:08:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-28T13:08:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-28T13:08:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-28T13:08:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-28T13:08:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-28T13:08:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-28T13:08:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-28T13:08:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-28T13:08:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-28T13:08:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-28T13:08:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-28T13:08:24.299 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 151/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.0.0-2499.ge0e48ba6.el9.x86_64 153/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-28T13:08:24.300 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-28T13:08:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-28T13:08:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-28T13:08:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-28T13:08:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-28T13:08:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-28T13:08:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-28T13:08:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-28T13:08:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-28T13:08:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-28T13:08:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-28T13:08:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-28T13:08:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-28T13:08:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-28T13:08:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-28T13:08:24.301 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-28T13:08:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-28T13:08:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-28T13:08:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-28T13:08:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-28T13:08:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-28T13:08:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-28T13:08:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-28T13:08:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-28T13:08:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-28T13:08:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-28T13:08:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-28T13:08:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-28T13:08:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-28T13:08:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-28T13:08:24.302 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-28T13:08:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-28T13:08:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-28T13:08:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-28T13:08:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-28T13:08:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-28T13:08:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-28T13:08:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-28T13:08:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-28T13:08:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-28T13:08:24.303 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-28T13:08:27.120 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-28T13:08:27.121 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:27.121 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-03-28T13:08:27.121 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-28T13:08:27.121 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-28T13:08:27.121 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-28T13:08:27.121 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-28T13:08:27.121 INFO:teuthology.orchestra.run.smithi019.stdout: acl-2.3.1-4.el9.x86_64 2024-03-28T13:08:27.121 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-28T13:08:27.121 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-28T13:08:27.121 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-28T13:08:27.121 INFO:teuthology.orchestra.run.smithi019.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-28T13:08:27.121 INFO:teuthology.orchestra.run.smithi019.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-28T13:08:27.121 INFO:teuthology.orchestra.run.smithi019.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-28T13:08:27.122 INFO:teuthology.orchestra.run.smithi019.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-28T13:08:27.122 INFO:teuthology.orchestra.run.smithi019.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-28T13:08:27.122 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.122 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.122 INFO:teuthology.orchestra.run.smithi019.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-28T13:08:27.122 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-28T13:08:27.122 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-28T13:08:27.122 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-28T13:08:27.122 INFO:teuthology.orchestra.run.smithi019.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-28T13:08:27.122 INFO:teuthology.orchestra.run.smithi019.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-28T13:08:27.122 INFO:teuthology.orchestra.run.smithi019.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-28T13:08:27.122 INFO:teuthology.orchestra.run.smithi019.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-28T13:08:27.122 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-28T13:08:27.123 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-03-28T13:08:27.123 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-28T13:08:27.123 INFO:teuthology.orchestra.run.smithi019.stdout: automake-1.16.2-8.el9.noarch 2024-03-28T13:08:27.123 INFO:teuthology.orchestra.run.smithi019.stdout: bison-3.7.4-5.el9.x86_64 2024-03-28T13:08:27.123 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-28T13:08:27.123 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.123 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.123 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.123 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.123 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:08:27.123 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.123 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.123 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.123 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:08:27.123 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:08:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:08:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:08:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:08:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:08:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:08:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:08:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.124 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top-2:19.0.0-2499.ge0e48ba6.el9.noarch 2024-03-28T13:08:27.125 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-28T13:08:27.125 INFO:teuthology.orchestra.run.smithi019.stdout: flex-2.6.4-9.el9.x86_64 2024-03-28T13:08:27.125 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-28T13:08:27.125 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-28T13:08:27.125 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-28T13:08:27.125 INFO:teuthology.orchestra.run.smithi019.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-28T13:08:27.125 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-28T13:08:27.125 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-28T13:08:27.125 INFO:teuthology.orchestra.run.smithi019.stdout: jq-1.6-16.el9.x86_64 2024-03-28T13:08:27.125 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-28T13:08:27.125 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-28T13:08:27.125 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-28T13:08:27.125 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.125 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.126 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.126 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-28T13:08:27.126 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-28T13:08:27.126 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-28T13:08:27.126 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-28T13:08:27.126 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-28T13:08:27.126 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.126 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.126 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-28T13:08:27.126 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.126 INFO:teuthology.orchestra.run.smithi019.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-28T13:08:27.126 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-28T13:08:27.126 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-28T13:08:27.126 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-28T13:08:27.127 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-28T13:08:27.127 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-03-28T13:08:27.127 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-28T13:08:27.127 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-28T13:08:27.127 INFO:teuthology.orchestra.run.smithi019.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-28T13:08:27.127 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-28T13:08:27.127 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-28T13:08:27.127 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-28T13:08:27.127 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-28T13:08:27.127 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-13.14-1.el9.x86_64 2024-03-28T13:08:27.127 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-03-28T13:08:27.127 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-03-28T13:08:27.127 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server-13.14-1.el9.x86_64 2024-03-28T13:08:27.127 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-28T13:08:27.127 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-28T13:08:27.128 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-28T13:08:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-28T13:08:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-28T13:08:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-28T13:08:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-28T13:08:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-28T13:08:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-28T13:08:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-28T13:08:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-28T13:08:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-28T13:08:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-28T13:08:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-28T13:08:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-28T13:08:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-28T13:08:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-28T13:08:27.129 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-28T13:08:27.130 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-28T13:08:27.130 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-28T13:08:27.130 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-28T13:08:27.130 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-28T13:08:27.130 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-28T13:08:27.130 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-28T13:08:27.130 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-28T13:08:27.130 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-03-28T13:08:27.130 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-28T13:08:27.130 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-28T13:08:27.130 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-28T13:08:27.130 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-28T13:08:27.130 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-28T13:08:27.130 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-28T13:08:27.130 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-28T13:08:27.131 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-24.02.0-1.el9.x86_64 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.0.0-2499.ge0e48ba6.el9.x86_64 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-3.el9.x86_64 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-28T13:08:27.132 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-28T13:08:27.133 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-28T13:08:27.133 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:27.133 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-03-28T13:08:27.828 DEBUG:teuthology.parallel:result is None 2024-03-28T13:08:27.829 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15 2024-03-28T13:08:27.973 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-28T13:08:28.040 INFO:teuthology.orchestra.run.smithi019.stdout:19.0.0-2499.ge0e48ba6.el9 2024-03-28T13:08:28.041 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2499.ge0e48ba6.el9 2024-03-28T13:08:28.041 INFO:teuthology.task.install:The correct ceph version 19.0.0-2499.ge0e48ba6 is installed. 2024-03-28T13:08:28.043 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15 2024-03-28T13:08:28.180 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-28T13:08:28.253 INFO:teuthology.orchestra.run.smithi188.stdout:19.0.0-2499.ge0e48ba6.el9 2024-03-28T13:08:28.253 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2499.ge0e48ba6.el9 2024-03-28T13:08:28.253 INFO:teuthology.task.install:The correct ceph version 19.0.0-2499.ge0e48ba6 is installed. 2024-03-28T13:08:28.255 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-28T13:08:28.255 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:08:28.255 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-28T13:08:28.332 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:08:28.332 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-28T13:08:28.431 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-28T13:08:28.432 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:08:28.432 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-03-28T13:08:28.515 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-28T13:08:28.632 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:08:28.632 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-03-28T13:08:28.735 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-28T13:08:28.818 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-28T13:08:28.818 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:08:28.819 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-28T13:08:28.890 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-28T13:08:29.009 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:08:29.009 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-28T13:08:29.098 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-28T13:08:29.227 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-28T13:08:29.228 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:08:29.228 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-03-28T13:08:29.297 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-28T13:08:29.412 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:08:29.413 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-03-28T13:08:29.495 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-28T13:08:29.611 INFO:teuthology.run_tasks:Running task ceph... 2024-03-28T13:08:29.694 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-28T13:08:29.694 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /var/log/ceph 2024-03-28T13:08:29.699 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /var/log/ceph 2024-03-28T13:08:29.780 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-28T13:08:29.780 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-28T13:08:29.784 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-28T13:08:29.857 INFO:tasks.ceph:Creating extra log directories... 2024-03-28T13:08:29.857 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-28T13:08:29.861 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-28T13:08:29.938 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-28T13:08:29.939 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-28T13:08:29.939 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-28T13:08:29.939 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:08:29.939 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-28T13:08:29.978 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-28T13:08:30.184 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-28T13:08:30.252 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:08:30.252 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-28T13:08:30.296 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-28T13:08:30.564 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-03-28T13:08:30.669 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-28T13:08:30.669 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-28T13:08:30.669 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849', 'branch': 'wip-batrick-testing-20240327.230800', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7627849', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-batrick-testing-20240327.230800'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20240327.230800', 'sha1': 'e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7752, 'sha1': 'e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15', 'sleep_before_teardown': 0, 'subset': '1/512', 'suite': 'fs', 'suite_branch': 'wip-batrick-testing-20240327.230800', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi019.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLMxKcBcq8C462aleVvkrsMm3/aiMC3doyAFgs+3Pf1icWW+2G/YktWZrB+nwUbFxs1mIEpQArVQzJHRg+TjPYI=', 'smithi188.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCzV9qvdqhto5/WsJrJRzziGvwAHvJbbvVaHtiQwq7cC3HReg10YTKULIyQYtmcXB/EpldjucBTx3GNsotvFC1w='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'e0e48ba6ee8c8b4c8f38d3b879d6eafbbb1efc15', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-28_07:14:07', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-28T13:08:30.670 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-28T13:08:30.675 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-28T13:08:30.730 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m0777 -- /var/run/ceph 2024-03-28T13:08:30.759 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/run/ceph 2024-03-28T13:08:30.847 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:08:30.847 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-03-28T13:08:30.887 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-28T13:08:30.887 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-03-28T13:08:30.969 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-28T13:08:30.970 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T13:08:30.970 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-03-28T13:08:30.970 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T13:08:30.970 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T13:08:30.970 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-03-28 13:08:19.561047253 +0000 2024-03-28T13:08:30.970 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-03-28 13:05:23.166107404 +0000 2024-03-28T13:08:30.970 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-03-28 13:05:23.166107404 +0000 2024-03-28T13:08:30.970 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-03-28 13:05:23.166107404 +0000 2024-03-28T13:08:30.970 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-28T13:08:31.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-03-28T13:08:31.075 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-03-28T13:08:31.075 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000324318 s, 1.6 MB/s 2024-03-28T13:08:31.077 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-28T13:08:31.117 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-03-28T13:08:31.201 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-28T13:08:31.201 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T13:08:31.201 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 651 Links: 1 2024-03-28T13:08:31.201 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T13:08:31.201 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T13:08:31.201 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-03-28 13:08:19.562047244 +0000 2024-03-28T13:08:31.201 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-03-28 13:05:23.739101856 +0000 2024-03-28T13:08:31.201 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-03-28 13:05:23.739101856 +0000 2024-03-28T13:08:31.201 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-03-28 13:05:23.739101856 +0000 2024-03-28T13:08:31.202 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-28T13:08:31.339 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-03-28T13:08:31.339 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-03-28T13:08:31.339 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000519798 s, 985 kB/s 2024-03-28T13:08:31.343 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-28T13:08:31.406 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-03-28T13:08:31.457 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-28T13:08:31.458 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T13:08:31.458 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-03-28T13:08:31.458 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T13:08:31.458 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T13:08:31.458 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-03-28 13:08:19.563047234 +0000 2024-03-28T13:08:31.458 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-03-28 13:05:24.355095891 +0000 2024-03-28T13:08:31.458 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-03-28 13:05:24.355095891 +0000 2024-03-28T13:08:31.458 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-03-28 13:05:24.355095891 +0000 2024-03-28T13:08:31.458 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-28T13:08:31.559 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-03-28T13:08:31.559 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-03-28T13:08:31.559 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000366696 s, 1.4 MB/s 2024-03-28T13:08:31.564 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-28T13:08:31.624 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-03-28T13:08:31.676 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-28T13:08:31.677 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T13:08:31.677 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-03-28T13:08:31.677 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T13:08:31.677 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T13:08:31.677 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-03-28 13:08:19.563047234 +0000 2024-03-28T13:08:31.677 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-03-28 13:05:24.948090149 +0000 2024-03-28T13:08:31.677 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-03-28 13:05:24.948090149 +0000 2024-03-28T13:08:31.677 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-03-28 13:05:24.948090149 +0000 2024-03-28T13:08:31.677 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-28T13:08:31.772 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-03-28T13:08:31.772 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-03-28T13:08:31.772 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000391454 s, 1.3 MB/s 2024-03-28T13:08:31.777 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-28T13:08:31.838 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-28T13:08:31.839 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:08:31.839 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-03-28T13:08:31.897 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-28T13:08:31.897 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-03-28T13:08:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-28T13:08:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T13:08:31.953 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 634 Links: 1 2024-03-28T13:08:31.953 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T13:08:31.953 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T13:08:31.953 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-03-28 13:07:42.828181640 +0000 2024-03-28T13:08:31.953 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-03-28 13:05:22.001129712 +0000 2024-03-28T13:08:31.953 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-03-28 13:05:22.001129712 +0000 2024-03-28T13:08:31.953 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-03-28 13:05:22.001129712 +0000 2024-03-28T13:08:31.954 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-28T13:08:32.041 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-03-28T13:08:32.041 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-03-28T13:08:32.041 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000331218 s, 1.5 MB/s 2024-03-28T13:08:32.045 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-28T13:08:32.102 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-03-28T13:08:32.152 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-28T13:08:32.152 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T13:08:32.152 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 652 Links: 1 2024-03-28T13:08:32.153 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T13:08:32.153 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T13:08:32.153 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-03-28 13:07:42.830181602 +0000 2024-03-28T13:08:32.153 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-03-28 13:05:22.591118232 +0000 2024-03-28T13:08:32.153 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-03-28 13:05:22.591118232 +0000 2024-03-28T13:08:32.153 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-03-28 13:05:22.591118232 +0000 2024-03-28T13:08:32.153 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-28T13:08:32.235 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-03-28T13:08:32.235 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-03-28T13:08:32.235 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.00029238 s, 1.8 MB/s 2024-03-28T13:08:32.239 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-28T13:08:32.293 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-03-28T13:08:32.341 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-28T13:08:32.341 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T13:08:32.341 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-03-28T13:08:32.341 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T13:08:32.341 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T13:08:32.341 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-03-28 13:07:42.831181583 +0000 2024-03-28T13:08:32.341 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-03-28 13:05:23.172106928 +0000 2024-03-28T13:08:32.341 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-03-28 13:05:23.172106928 +0000 2024-03-28T13:08:32.341 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-03-28 13:05:23.172106928 +0000 2024-03-28T13:08:32.342 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-28T13:08:32.473 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-03-28T13:08:32.473 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-03-28T13:08:32.473 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000321379 s, 1.6 MB/s 2024-03-28T13:08:32.476 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-28T13:08:32.528 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-03-28T13:08:32.578 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-28T13:08:32.579 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-28T13:08:32.579 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 688 Links: 1 2024-03-28T13:08:32.579 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-28T13:08:32.579 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-03-28T13:08:32.579 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-03-28 13:07:42.832181564 +0000 2024-03-28T13:08:32.579 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-03-28 13:05:23.719096286 +0000 2024-03-28T13:08:32.579 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-03-28 13:05:23.720096266 +0000 2024-03-28T13:08:32.579 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-03-28 13:05:23.719096286 +0000 2024-03-28T13:08:32.579 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-28T13:08:32.665 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-03-28T13:08:32.665 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-03-28T13:08:32.666 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000298069 s, 1.7 MB/s 2024-03-28T13:08:32.670 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-28T13:08:32.727 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-28T13:08:32.727 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi019.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-28T13:08:32.728 INFO:tasks.ceph:Generating config... 2024-03-28T13:08:32.729 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-28T13:08:32.729 INFO:tasks.ceph:[client] debug client = 20 2024-03-28T13:08:32.729 INFO:tasks.ceph:[client] debug ms = 1 2024-03-28T13:08:32.729 INFO:tasks.ceph:[client] pid file = 2024-03-28T13:08:32.729 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-28T13:08:32.729 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-28T13:08:32.729 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-03-28T13:08:32.729 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-28T13:08:32.729 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-28T13:08:32.729 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-28T13:08:32.729 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-28T13:08:32.729 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-28T13:08:32.729 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-28T13:08:32.729 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-28T13:08:32.730 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-28T13:08:32.730 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-28T13:08:32.730 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-28T13:08:32.730 INFO:tasks.ceph:[mgr] debug client = 20 2024-03-28T13:08:32.730 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-28T13:08:32.730 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-28T13:08:32.730 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-28T13:08:32.730 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-28T13:08:32.730 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-28T13:08:32.730 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-03-28T13:08:32.730 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-28T13:08:32.730 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-28T13:08:32.731 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-28T13:08:32.731 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-28T13:08:32.731 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-28T13:08:32.731 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-28T13:08:32.731 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-28T13:08:32.731 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-28T13:08:32.731 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-28T13:08:32.731 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-28T13:08:32.731 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-28T13:08:32.731 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-28T13:08:32.731 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-28T13:08:32.731 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-28T13:08:32.732 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-28T13:08:32.732 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-28T13:08:32.732 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-28T13:08:32.732 INFO:tasks.ceph:Setting up mon.a... 2024-03-28T13:08:32.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-28T13:08:32.881 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/ceph.keyring 2024-03-28T13:08:32.890 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-28T13:08:33.030 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-28T13:08:33.113 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.19'), ('mon.b', '172.21.15.188'), ('mon.c', '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]')] 2024-03-28T13:08:33.113 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.19,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-28T13:08:33.114 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:08:33.114 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-28T13:08:33.172 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.19 --add b 172.21.15.188 --addv c '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-28T13:08:33.331 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-28T13:08:33.331 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: generated fsid e6b569d9-f2ce-49dc-bbe7-c0ab33080203 2024-03-28T13:08:33.331 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = quincy 2024-03-28T13:08:33.331 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 0 2024-03-28T13:08:33.331 INFO:teuthology.orchestra.run.smithi019.stdout:fsid e6b569d9-f2ce-49dc-bbe7-c0ab33080203 2024-03-28T13:08:33.331 INFO:teuthology.orchestra.run.smithi019.stdout:last_changed 2024-03-28T13:08:33.332192+0000 2024-03-28T13:08:33.331 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-03-28T13:08:33.332192+0000 2024-03-28T13:08:33.332 INFO:teuthology.orchestra.run.smithi019.stdout:min_mon_release 17 (quincy) 2024-03-28T13:08:33.332 INFO:teuthology.orchestra.run.smithi019.stdout:election_strategy: 1 2024-03-28T13:08:33.332 INFO:teuthology.orchestra.run.smithi019.stdout:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-03-28T13:08:33.332 INFO:teuthology.orchestra.run.smithi019.stdout:1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.b 2024-03-28T13:08:33.332 INFO:teuthology.orchestra.run.smithi019.stdout:2: [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] mon.c 2024-03-28T13:08:33.332 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-28T13:08:33.334 DEBUG:teuthology.orchestra.run.smithi019:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-28T13:08:33.387 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e6b569d9-f2ce-49dc-bbe7-c0ab33080203... 2024-03-28T13:08:33.388 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T13:08:33.394 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T13:08:33.568 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-03-28T13:08:33.568 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2024-03-28T13:08:33.568 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T13:08:33.568 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2024-03-28T13:08:33.568 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.568 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2024-03-28T13:08:33.568 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.568 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2024-03-28T13:08:33.568 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2024-03-28T13:08:33.569 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2024-03-28T13:08:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2024-03-28T13:08:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2024-03-28T13:08:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2024-03-28T13:08:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2024-03-28T13:08:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T13:08:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T13:08:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: mon_allow_pool_size_one = true 2024-03-28T13:08:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T13:08:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2024-03-28T13:08:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2024-03-28T13:08:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.570 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: mon health detail to clog = false 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = "172.21.15.19,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on skipped message = False 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = e6b569d9-f2ce-49dc-bbe7-c0ab33080203 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock profile = high_recovery_ops 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.571 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug trim objects = true 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2024-03-28T13:08:33.572 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: bdev async discard = True 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: bdev enable discard = True 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore allocator = bitmap 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore block size = 96636764160 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore fsck on mount = True 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluefs = 1/20 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluestore = 1/20 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 20 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: debug rocksdb = 4/10 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd full ratio = 0.9 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd nearfull ratio = 0.8 2024-03-28T13:08:33.573 INFO:teuthology.orchestra.run.smithi019.stdout: osd failsafe full ratio = 0.95 2024-03-28T13:08:33.574 INFO:teuthology.orchestra.run.smithi019.stdout: osd objectstore = bluestore 2024-03-28T13:08:33.574 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-03-28T13:08:33.574 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.574 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2024-03-28T13:08:33.574 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:08:33.574 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2024-03-28T13:08:33.574 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-03-28T13:08:33.574 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-03-28T13:08:33.574 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:08:33.574 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:08:33.574 INFO:teuthology.orchestra.run.smithi019.stdout: mgr/telemetry/nag = false 2024-03-28T13:08:33.574 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-03-28T13:08:33.574 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.574 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.575 INFO:teuthology.orchestra.run.smithi019.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T13:08:33.576 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T13:08:33.576 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T13:08:33.576 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.576 INFO:teuthology.orchestra.run.smithi019.stdout: # 1m isn't quite enough 2024-03-28T13:08:33.576 INFO:teuthology.orchestra.run.smithi019.stdout: mon_down_mkfs_grace = 2m 2024-03-28T13:08:33.576 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.576 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_filestore_osds = false 2024-03-28T13:08:33.576 INFO:teuthology.orchestra.run.smithi019.stdout: mon down mkfs grace = 300 2024-03-28T13:08:33.576 INFO:teuthology.orchestra.run.smithi019.stdout: mon op complaint time = 120 2024-03-28T13:08:33.576 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:08:33.576 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2024-03-28T13:08:33.576 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2024-03-28T13:08:33.576 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2024-03-28T13:08:33.576 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2024-03-28T13:08:33.576 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout: client mount timeout = 600 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = "" 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.b] 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.c] 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a] 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c] 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b] 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d] 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout:[mds] 2024-03-28T13:08:33.577 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds = 20 2024-03-28T13:08:33.578 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds balancer = 20 2024-03-28T13:08:33.578 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:08:33.578 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug frag = True 2024-03-28T13:08:33.578 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug scatterstat = True 2024-03-28T13:08:33.578 INFO:teuthology.orchestra.run.smithi019.stdout: mds op complaint time = 180 2024-03-28T13:08:33.578 INFO:teuthology.orchestra.run.smithi019.stdout: mds verify scatter = True 2024-03-28T13:08:33.578 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-03-28T13:08:33.578 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-03-28T13:08:33.578 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-03-28T13:08:33.579 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-03-28T13:08:33.579 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-03-28T13:08:33.579 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T13:08:33.579 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-03-28T13:08:33.579 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.579 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-03-28T13:08:33.579 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.579 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-03-28T13:08:33.579 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.579 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-03-28T13:08:33.579 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-03-28T13:08:33.579 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.579 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-03-28T13:08:33.579 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-03-28T13:08:33.580 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T13:08:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T13:08:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-03-28T13:08:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T13:08:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-03-28T13:08:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-03-28T13:08:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-03-28T13:08:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-03-28T13:08:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-03-28T13:08:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.19,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-03-28T13:08:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2024-03-28T13:08:33.581 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = e6b569d9-f2ce-49dc-bbe7-c0ab33080203 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-03-28T13:08:33.582 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-03-28T13:08:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-03-28T13:08:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-03-28T13:08:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-03-28T13:08:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-03-28T13:08:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-03-28T13:08:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-03-28T13:08:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-03-28T13:08:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-03-28T13:08:33.583 INFO:teuthology.orchestra.run.smithi188.stdout: bdev async discard = True 2024-03-28T13:08:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: bdev enable discard = True 2024-03-28T13:08:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore allocator = bitmap 2024-03-28T13:08:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-03-28T13:08:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-03-28T13:08:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 1/20 2024-03-28T13:08:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 1/20 2024-03-28T13:08:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:08:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-03-28T13:08:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 4/10 2024-03-28T13:08:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T13:08:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-03-28T13:08:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-03-28T13:08:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2024-03-28T13:08:33.584 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2024-03-28T13:08:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-03-28T13:08:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.585 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-03-28T13:08:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:08:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-03-28T13:08:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-03-28T13:08:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-03-28T13:08:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:08:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:08:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-03-28T13:08:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-03-28T13:08:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.585 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-03-28T13:08:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:08:33.585 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-03-28T13:08:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-03-28T13:08:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-03-28T13:08:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-03-28T13:08:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-03-28T13:08:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:08:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-03-28T13:08:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-03-28T13:08:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:08:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T13:08:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T13:08:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T13:08:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T13:08:33.586 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: mon down mkfs grace = 300 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T13:08:33.587 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:08:33.588 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2024-03-28T13:08:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = True 2024-03-28T13:08:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2024-03-28T13:08:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = True 2024-03-28T13:08:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-03-28T13:08:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-03-28T13:08:33.589 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-03-28T13:08:33.617 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-28T13:08:33.618 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-28T13:08:33.774 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-28T13:08:33.775 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:08:33.775 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-28T13:08:33.827 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:08:33.827 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-28T13:08:33.877 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:08:33.877 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:08:33.877 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-28T13:08:33.877 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-28T13:08:34.025 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:08:34.026 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-28T13:08:34.124 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi188.front.sepia.ceph.com 2024-03-28T13:08:34.124 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:08:34.124 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-28T13:08:34.124 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-28T13:08:34.275 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:08:34.275 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-28T13:08:34.371 INFO:tasks.ceph:Setting up mon nodes... 2024-03-28T13:08:34.371 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-28T13:08:34.371 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-28T13:08:34.567 INFO:teuthology.orchestra.run.smithi019.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-28T13:08:34.577 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-28T13:08:34.791 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-28T13:08:34.803 INFO:tasks.ceph:Setting up mds nodes... 2024-03-28T13:08:34.803 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-28T13:08:35.001 INFO:teuthology.orchestra.run.smithi019.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-28T13:08:35.010 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-28T13:08:35.110 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-28T13:08:35.303 INFO:teuthology.orchestra.run.smithi019.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-28T13:08:35.313 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-28T13:08:35.417 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-28T13:08:35.586 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-28T13:08:35.595 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-28T13:08:35.688 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-28T13:08:35.861 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-28T13:08:35.870 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-28T13:08:35.960 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-28T13:08:35.960 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-28T13:08:36.094 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-28T13:08:36.153 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-28T13:08:36.292 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-28T13:08:36.354 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-28T13:08:36.479 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-28T13:08:36.528 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-28T13:08:36.648 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-28T13:08:36.702 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-28T13:08:36.702 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi019.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-28T13:08:36.702 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-28T13:08:36.807 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-28T13:08:36.807 INFO:tasks.ceph:role: osd.0 2024-03-28T13:08:36.807 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:08:36.808 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-28T13:08:36.912 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T13:08:36.913 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T13:08:36.913 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T13:08:36.913 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T13:08:36.913 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T13:08:36.913 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-03-28T13:08:36.913 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T13:08:36.913 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T13:08:36.913 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T13:08:36.913 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T13:08:37.430 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-03-28T13:08:37.437 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-03-28T13:08:37.437 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-28T13:08:37.598 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-28T13:08:37.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-28T13:08:37.803 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-28T13:08:37.803 INFO:tasks.ceph:role: osd.1 2024-03-28T13:08:37.803 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:08:37.803 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-28T13:08:37.905 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T13:08:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T13:08:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T13:08:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T13:08:37.905 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T13:08:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-03-28T13:08:37.905 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T13:08:37.905 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T13:08:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T13:08:37.905 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T13:08:38.379 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-03-28T13:08:38.386 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-03-28T13:08:38.386 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-28T13:08:38.533 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-28T13:08:38.632 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-28T13:08:38.720 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-28T13:08:38.720 INFO:tasks.ceph:role: osd.2 2024-03-28T13:08:38.720 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:08:38.720 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-28T13:08:38.823 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T13:08:38.823 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T13:08:38.823 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T13:08:38.823 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T13:08:38.823 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T13:08:38.823 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-03-28T13:08:38.823 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T13:08:38.823 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T13:08:38.823 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T13:08:38.824 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T13:08:39.325 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-03-28T13:08:39.332 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-03-28T13:08:39.332 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-28T13:08:39.514 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-28T13:08:39.621 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-28T13:08:39.716 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-28T13:08:39.716 INFO:tasks.ceph:role: osd.3 2024-03-28T13:08:39.716 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:08:39.717 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-28T13:08:39.823 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T13:08:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T13:08:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T13:08:39.823 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T13:08:39.823 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T13:08:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-03-28T13:08:39.824 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T13:08:39.824 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T13:08:39.824 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T13:08:39.824 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T13:08:40.292 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-03-28T13:08:40.297 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-03-28T13:08:40.297 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-28T13:08:40.460 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-28T13:08:40.563 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T13:08:40.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:40.738+0000 7fbf24210640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-28T13:08:40.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:40.740+0000 7fbf24210640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-28T13:08:40.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:40.740+0000 7fbf24210640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-28T13:08:40.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:40.740+0000 7fbf24210640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-28T13:08:40.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:40.740+0000 7fbf24210640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-28T13:08:40.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:40.750+0000 7fbf24210640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-28T13:08:43.687 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-28T13:08:43.783 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T13:08:43.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:43.927+0000 7f61f4017640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-28T13:08:43.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:43.928+0000 7f61f4017640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-28T13:08:43.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:43.928+0000 7f61f4017640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-28T13:08:43.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:43.928+0000 7f61f4017640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-28T13:08:43.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:43.928+0000 7f61f4017640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-28T13:08:43.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:43.931+0000 7f61f4017640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-28T13:08:47.113 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-28T13:08:47.217 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T13:08:47.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:47.364+0000 7f9e59415640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-28T13:08:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:47.364+0000 7f9e59415640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-28T13:08:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:47.364+0000 7f9e59415640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-28T13:08:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:47.364+0000 7f9e59415640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-28T13:08:47.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:47.364+0000 7f9e59415640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-28T13:08:47.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:47.367+0000 7f9e59415640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-28T13:08:50.567 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-28T13:08:50.674 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T13:08:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:50.817+0000 7f7411217640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-28T13:08:50.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:50.818+0000 7f7411217640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-28T13:08:50.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:50.819+0000 7f7411217640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-28T13:08:50.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:50.819+0000 7f7411217640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-28T13:08:50.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:50.819+0000 7f7411217640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-28T13:08:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:08:50.821+0000 7f7411217640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-28T13:08:53.745 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-28T13:08:53.844 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-28T13:08:53.940 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-28T13:08:53.940 INFO:tasks.ceph:role: osd.4 2024-03-28T13:08:53.940 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com 2024-03-28T13:08:53.940 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-28T13:08:54.038 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T13:08:54.038 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T13:08:54.038 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T13:08:54.038 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T13:08:54.038 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T13:08:54.038 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-03-28T13:08:54.039 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T13:08:54.039 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T13:08:54.039 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T13:08:54.039 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T13:08:54.552 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-03-28T13:08:54.558 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-03-28T13:08:54.559 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-28T13:08:54.732 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-28T13:08:54.833 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-28T13:08:54.924 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-28T13:08:54.924 INFO:tasks.ceph:role: osd.5 2024-03-28T13:08:54.924 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com 2024-03-28T13:08:54.924 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-28T13:08:55.021 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T13:08:55.021 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T13:08:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T13:08:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T13:08:55.022 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T13:08:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-03-28T13:08:55.022 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T13:08:55.022 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T13:08:55.022 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T13:08:55.022 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T13:08:55.528 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-03-28T13:08:55.535 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-03-28T13:08:55.535 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-28T13:08:55.680 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-28T13:08:55.772 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-28T13:08:55.860 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-28T13:08:55.861 INFO:tasks.ceph:role: osd.6 2024-03-28T13:08:55.861 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com 2024-03-28T13:08:55.861 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-28T13:08:55.962 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T13:08:55.962 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T13:08:55.962 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T13:08:55.962 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T13:08:55.962 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T13:08:55.962 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-03-28T13:08:55.962 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T13:08:55.962 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T13:08:55.962 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T13:08:55.962 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T13:08:56.474 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-03-28T13:08:56.481 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-03-28T13:08:56.481 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-28T13:08:56.642 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-28T13:08:56.741 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-28T13:08:56.831 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-28T13:08:56.831 INFO:tasks.ceph:role: osd.7 2024-03-28T13:08:56.831 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com 2024-03-28T13:08:56.831 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-28T13:08:56.926 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-28T13:08:56.926 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-28T13:08:56.926 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-28T13:08:56.926 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-03-28T13:08:56.927 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-28T13:08:56.927 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-03-28T13:08:56.927 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-28T13:08:56.927 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-03-28T13:08:56.927 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-28T13:08:56.927 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-28T13:08:57.386 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-03-28T13:08:57.393 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-03-28T13:08:57.393 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-28T13:08:57.535 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-28T13:08:57.620 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T13:08:57.766 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:08:57.766+0000 7f463940f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-28T13:08:57.768 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:08:57.768+0000 7f463940f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-28T13:08:57.768 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:08:57.768+0000 7f463940f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-28T13:08:57.768 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:08:57.768+0000 7f463940f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-28T13:08:57.768 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:08:57.768+0000 7f463940f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-28T13:08:57.774 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:08:57.774+0000 7f463940f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-28T13:09:00.671 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:09:00.679 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:09:00.717 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-28T13:09:00.844 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T13:09:01.027 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:01.027+0000 7f806e020640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-28T13:09:01.028 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:01.028+0000 7f806e020640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-28T13:09:01.028 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:01.028+0000 7f806e020640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-28T13:09:01.028 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:01.028+0000 7f806e020640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-28T13:09:01.028 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:01.028+0000 7f806e020640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-28T13:09:01.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:01.030+0000 7f806e020640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-28T13:09:04.455 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-28T13:09:04.553 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T13:09:04.690 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:04.689+0000 7fb722c17640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-28T13:09:04.690 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:04.690+0000 7fb722c17640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-28T13:09:04.690 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:04.690+0000 7fb722c17640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-28T13:09:04.690 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:04.690+0000 7fb722c17640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-28T13:09:04.691 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:04.690+0000 7fb722c17640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-28T13:09:04.692 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:04.692+0000 7fb722c17640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-28T13:09:07.623 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-28T13:09:07.716 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-28T13:09:07.854 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:07.854+0000 7f79366b5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-28T13:09:07.855 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:07.854+0000 7f79366b5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-28T13:09:07.855 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:07.854+0000 7f79366b5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-28T13:09:07.855 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:07.854+0000 7f79366b5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-28T13:09:07.855 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:07.854+0000 7f79366b5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-28T13:09:07.856 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:07.856+0000 7f79366b5640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-28T13:09:10.779 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-28T13:09:10.870 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-28T13:09:10.870 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:09:10.870 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-28T13:09:10.977 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:09:10.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-28T13:09:11.077 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:09:11.077 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-28T13:09:11.173 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:09:11.173 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-28T13:09:11.273 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:09:11.273 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-28T13:09:11.367 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:09:11.367 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-28T13:09:11.470 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:09:11.470 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-28T13:09:11.570 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:09:11.570 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-28T13:09:11.665 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:09:11.665 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-28T13:09:11.754 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:09:11.754 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-28T13:09:11.847 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:09:11.847 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-28T13:09:11.983 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:09:11.983 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-28T13:09:12.075 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:09:12.075 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-28T13:09:12.171 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:09:12.171 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-28T13:09:12.264 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:09:12.264 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-28T13:09:12.323 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:09:12.323 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-28T13:09:12.380 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:09:12.380 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-28T13:09:12.436 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:09:12.437 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-28T13:09:12.489 INFO:tasks.ceph:Adding keys to all mons... 2024-03-28T13:09:12.489 DEBUG:teuthology.orchestra.run.smithi019:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-28T13:09:12.497 DEBUG:teuthology.orchestra.run.smithi188:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-28T13:09:12.587 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.y] 2024-03-28T13:09:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDSawVmebzfIRAAA0+qIIaprr4SfEZT6qRooA== 2024-03-28T13:09:12.587 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-03-28T13:09:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDTawVmNRIkABAAq/nJioDZjp1zFWAYOLPBYw== 2024-03-28T13:09:12.587 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-03-28T13:09:12.587 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDTawVmmlUoEhAA4FUGXoZ/h/ve5fThJNdE2Q== 2024-03-28T13:09:12.587 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.0] 2024-03-28T13:09:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDYawVmxhwWLBAAm7C56UGConFx2D7Lbq2SSA== 2024-03-28T13:09:12.588 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.1] 2024-03-28T13:09:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDbawVmC2BiNxAALQrAcXU44m2dTCjks5yy/Q== 2024-03-28T13:09:12.588 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.2] 2024-03-28T13:09:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDfawVmozrJFRAA7YV1uMucIW7H5SI4KY8aDw== 2024-03-28T13:09:12.588 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.3] 2024-03-28T13:09:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDiawVmccbTMBAATxk24Bf4LuQSqU92HdwgUg== 2024-03-28T13:09:12.588 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.x] 2024-03-28T13:09:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDSawVm5H1FLxAAtyjOPcCEUQyvTpBdu3Fn3g== 2024-03-28T13:09:12.588 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-03-28T13:09:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDTawVmc10CIxAAQWCSjH0hW3KUk8/qQm91QA== 2024-03-28T13:09:12.588 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-03-28T13:09:12.588 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDTawVmWDRnMxAA34LiLI79N5fVJnbcPacbQA== 2024-03-28T13:09:12.589 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.4] 2024-03-28T13:09:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDpawVmdwnILRAAjUEBwnBEeSltu2CLRGhmDQ== 2024-03-28T13:09:12.589 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.5] 2024-03-28T13:09:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDtawVmkwW5ARAAZJ/Nd/Wtq7LcbUjeqPNUag== 2024-03-28T13:09:12.589 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.6] 2024-03-28T13:09:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDwawVm254wKRAAmCYODqSd2Whn4YKQzywrng== 2024-03-28T13:09:12.589 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.7] 2024-03-28T13:09:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDzawVmV8z+MhAA9JeF55K7ph1mQRWalT7XhA== 2024-03-28T13:09:12.589 INFO:teuthology.orchestra.run.smithi188.stdout:[client.0] 2024-03-28T13:09:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q== 2024-03-28T13:09:12.589 INFO:teuthology.orchestra.run.smithi188.stdout:[client.2] 2024-03-28T13:09:12.589 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDUawVmE7V/ERAAYcdvolIMDd5ff+pCn7hN1w== 2024-03-28T13:09:12.589 INFO:teuthology.orchestra.run.smithi188.stdout:[client.1] 2024-03-28T13:09:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDUawVmJYGjHBAAAx2TtR0Gb3fafnZU26FCMA== 2024-03-28T13:09:12.590 INFO:teuthology.orchestra.run.smithi188.stdout:[client.3] 2024-03-28T13:09:12.590 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQDUawVmk6q9JhAAeC3JHQDnK8IXGVoK6qGEFA== 2024-03-28T13:09:12.594 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr.y] 2024-03-28T13:09:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDSawVmebzfIRAAA0+qIIaprr4SfEZT6qRooA== 2024-03-28T13:09:12.594 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a] 2024-03-28T13:09:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDTawVmNRIkABAAq/nJioDZjp1zFWAYOLPBYw== 2024-03-28T13:09:12.594 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c] 2024-03-28T13:09:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDTawVmmlUoEhAA4FUGXoZ/h/ve5fThJNdE2Q== 2024-03-28T13:09:12.594 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.0] 2024-03-28T13:09:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDYawVmxhwWLBAAm7C56UGConFx2D7Lbq2SSA== 2024-03-28T13:09:12.594 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.1] 2024-03-28T13:09:12.594 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDbawVmC2BiNxAALQrAcXU44m2dTCjks5yy/Q== 2024-03-28T13:09:12.595 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.2] 2024-03-28T13:09:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDfawVmozrJFRAA7YV1uMucIW7H5SI4KY8aDw== 2024-03-28T13:09:12.595 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.3] 2024-03-28T13:09:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDiawVmccbTMBAATxk24Bf4LuQSqU92HdwgUg== 2024-03-28T13:09:12.595 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr.x] 2024-03-28T13:09:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDSawVm5H1FLxAAtyjOPcCEUQyvTpBdu3Fn3g== 2024-03-28T13:09:12.595 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b] 2024-03-28T13:09:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDTawVmc10CIxAAQWCSjH0hW3KUk8/qQm91QA== 2024-03-28T13:09:12.595 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d] 2024-03-28T13:09:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDTawVmWDRnMxAA34LiLI79N5fVJnbcPacbQA== 2024-03-28T13:09:12.595 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.4] 2024-03-28T13:09:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDpawVmdwnILRAAjUEBwnBEeSltu2CLRGhmDQ== 2024-03-28T13:09:12.595 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.5] 2024-03-28T13:09:12.595 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDtawVmkwW5ARAAZJ/Nd/Wtq7LcbUjeqPNUag== 2024-03-28T13:09:12.595 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.6] 2024-03-28T13:09:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDwawVm254wKRAAmCYODqSd2Whn4YKQzywrng== 2024-03-28T13:09:12.596 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.7] 2024-03-28T13:09:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDzawVmV8z+MhAA9JeF55K7ph1mQRWalT7XhA== 2024-03-28T13:09:12.596 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-03-28T13:09:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q== 2024-03-28T13:09:12.596 INFO:teuthology.orchestra.run.smithi019.stdout:[client.2] 2024-03-28T13:09:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDUawVmE7V/ERAAYcdvolIMDd5ff+pCn7hN1w== 2024-03-28T13:09:12.596 INFO:teuthology.orchestra.run.smithi019.stdout:[client.1] 2024-03-28T13:09:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDUawVmJYGjHBAAAx2TtR0Gb3fafnZU26FCMA== 2024-03-28T13:09:12.596 INFO:teuthology.orchestra.run.smithi019.stdout:[client.3] 2024-03-28T13:09:12.596 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDUawVmk6q9JhAAeC3JHQDnK8IXGVoK6qGEFA== 2024-03-28T13:09:12.601 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-28T13:09:12.609 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-28T13:09:12.752 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T13:09:12.759 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T13:09:12.907 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T13:09:12.912 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T13:09:13.060 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:13.066 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:13.213 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:13.220 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:13.361 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:13.368 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:13.508 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:13.516 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:13.654 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-28T13:09:13.662 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-28T13:09:13.808 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T13:09:13.815 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T13:09:13.968 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T13:09:13.975 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-28T13:09:14.123 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:14.130 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:14.273 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:14.279 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:14.425 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:14.433 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:14.577 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:14.582 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-28T13:09:14.726 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T13:09:14.733 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T13:09:14.881 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T13:09:14.887 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T13:09:15.032 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T13:09:15.038 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T13:09:15.187 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T13:09:15.191 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-28T13:09:15.336 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-28T13:09:15.336 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-28T13:09:15.434 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-28T13:09:15.589 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-28T13:09:15.673 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-28T13:09:15.769 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-28T13:09:15.924 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-28T13:09:16.009 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-28T13:09:16.098 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-28T13:09:16.246 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-28T13:09:16.324 DEBUG:teuthology.orchestra.run.smithi019:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-28T13:09:16.332 DEBUG:teuthology.orchestra.run.smithi188:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-28T13:09:16.384 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-28T13:09:16.384 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-28T13:09:16.385 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-28T13:09:16.390 INFO:tasks.ceph.mon.a:Started 2024-03-28T13:09:16.390 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-28T13:09:16.390 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-28T13:09:16.395 INFO:tasks.ceph.mon.b:Started 2024-03-28T13:09:16.395 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-28T13:09:16.395 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-28T13:09:16.400 INFO:tasks.ceph.mon.c:Started 2024-03-28T13:09:16.400 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-28T13:09:16.400 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-28T13:09:16.400 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-28T13:09:16.405 INFO:tasks.ceph.mgr.y:Started 2024-03-28T13:09:16.405 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-28T13:09:16.405 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-28T13:09:16.409 INFO:tasks.ceph.mgr.x:Started 2024-03-28T13:09:16.409 INFO:tasks.ceph:Setting crush tunables to default 2024-03-28T13:09:16.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --cluster ceph osd crush tunables default 2024-03-28T13:09:16.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.597+0000 7f9d4e400640 1 Processor -- start 2024-03-28T13:09:16.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.598+0000 7f9d4e400640 1 -- start start 2024-03-28T13:09:16.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.600+0000 7f9d4e400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9d4815e7b0 0x7f9d480b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:16.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.600+0000 7f9d4e400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d480d01a0 con 0x7f9d480c7810 2024-03-28T13:09:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.600+0000 7f9d4e400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d480d0480 con 0x7f9d480d1d70 2024-03-28T13:09:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.600+0000 7f9d4e400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9d480d1240 con 0x7f9d4815e7b0 2024-03-28T13:09:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.601+0000 7f9d47400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9d4815e7b0 0x7f9d480b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.601+0000 7f9d47400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9d4815e7b0 0x7f9d480b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:52162/0 (socket says 172.21.15.19:52162) 2024-03-28T13:09:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.601+0000 7f9d47400640 1 -- 172.21.15.19:0/3072324893 learned_addr learned my addr 172.21.15.19:0/3072324893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.601+0000 7f9d47400640 1 -- 172.21.15.19:0/3072324893 >> v2:172.21.15.188:3300/0 conn(0x7f9d4815e7b0 msgr2=0x7f9d480b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-28T13:09:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.601+0000 7f9d47400640 1 -- 172.21.15.19:0/3072324893 >> v2:172.21.15.188:3300/0 conn(0x7f9d4815e7b0 msgr2=0x7f9d480b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-28T13:09:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.601+0000 7f9d47400640 1 --2- 172.21.15.19:0/3072324893 >> v2:172.21.15.188:3300/0 conn(0x7f9d4815e7b0 0x7f9d480b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-28T13:09:16.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.601+0000 7f9d47400640 1 --2- 172.21.15.19:0/3072324893 >> v2:172.21.15.188:3300/0 conn(0x7f9d4815e7b0 0x7f9d480b4980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-28T13:09:16.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d46000640 1 -- 172.21.15.19:0/3072324893 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2719431136 0 0) 0x7f9d480d01a0 con 0x7f9d480c7810 2024-03-28T13:09:16.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d46000640 1 -- 172.21.15.19:0/3072324893 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d2c003520 con 0x7f9d480c7810 2024-03-28T13:09:16.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d46000640 1 -- 172.21.15.19:0/3072324893 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2760800505 0 0) 0x7f9d2c003520 con 0x7f9d480c7810 2024-03-28T13:09:16.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d46000640 1 -- 172.21.15.19:0/3072324893 >> v2:172.21.15.188:3300/0 conn(0x7f9d4815e7b0 msgr2=0x7f9d480b4980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T13:09:16.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d46000640 1 --2- 172.21.15.19:0/3072324893 >> v2:172.21.15.188:3300/0 conn(0x7f9d4815e7b0 0x7f9d480b4980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:16.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d46000640 1 -- 172.21.15.19:0/3072324893 >> v1:172.21.15.188:6789/0 conn(0x7f9d480d1d70 legacy=0x7f9d480c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d46000640 1 -- 172.21.15.19:0/3072324893 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d480d0760 con 0x7f9d480c7810 2024-03-28T13:09:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d46000640 1 -- 172.21.15.19:0/3072324893 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f9d400019f0 con 0x7f9d480c7810 2024-03-28T13:09:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d46000640 1 -- 172.21.15.19:0/3072324893 >> v1:172.21.15.19:6789/0 conn(0x7f9d480c7810 legacy=0x7f9d4806ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d46000640 1 --2- 172.21.15.19:0/3072324893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d2c0039b0 0x7f9d2c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d46000640 1 --2- 172.21.15.19:0/3072324893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d2c004800 0x7f9d2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d46000640 1 --2- 172.21.15.19:0/3072324893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d2c0051c0 0x7f9d2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d46000640 1 -- 172.21.15.19:0/3072324893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9d480d1240 con 0x7f9d2c0039b0 2024-03-28T13:09:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d46000640 1 -- 172.21.15.19:0/3072324893 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9d2c00c0f0 con 0x7f9d2c0051c0 2024-03-28T13:09:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d46000640 1 -- 172.21.15.19:0/3072324893 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9d2c00c2f0 con 0x7f9d2c004800 2024-03-28T13:09:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.604+0000 7f9d47400640 1 --2- 172.21.15.19:0/3072324893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d2c0039b0 0x7f9d2c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.605+0000 7f9d47e00640 1 --2- 172.21.15.19:0/3072324893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d2c0051c0 0x7f9d2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.605+0000 7f9d47400640 1 -- 172.21.15.19:0/3072324893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d2c004800 msgr2=0x7f9d2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.605+0000 7f9d47400640 1 --2- 172.21.15.19:0/3072324893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d2c004800 0x7f9d2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.605+0000 7f9d47400640 1 -- 172.21.15.19:0/3072324893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d2c0051c0 msgr2=0x7f9d2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.605+0000 7f9d47400640 1 --2- 172.21.15.19:0/3072324893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d2c0051c0 0x7f9d2c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.605+0000 7f9d47400640 1 -- 172.21.15.19:0/3072324893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2c003520 con 0x7f9d2c0039b0 2024-03-28T13:09:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.605+0000 7f9d47e00640 1 -- 172.21.15.19:0/3072324893 reap_dead start 2024-03-28T13:09:16.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.606+0000 7f9d47400640 1 --2- 172.21.15.19:0/3072324893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d2c0039b0 0x7f9d2c003d90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009890 tx=0x7f9d2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c3db7b2fef218b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:16.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.606+0000 7f9d46000640 1 -- 172.21.15.19:0/3072324893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38034070 con 0x7f9d2c0039b0 2024-03-28T13:09:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.606+0000 7f9d46000640 1 -- 172.21.15.19:0/3072324893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d38030660 con 0x7f9d2c0039b0 2024-03-28T13:09:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.606+0000 7f9d46000640 1 -- 172.21.15.19:0/3072324893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3800f040 con 0x7f9d2c0039b0 2024-03-28T13:09:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.607+0000 7f9d4e400640 1 -- 172.21.15.19:0/3072324893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d2c0039b0 msgr2=0x7f9d2c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.607+0000 7f9d4e400640 1 --2- 172.21.15.19:0/3072324893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d2c0039b0 0x7f9d2c003d90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9d38009890 tx=0x7f9d2c005830 comp rx=0 tx=0).stop 2024-03-28T13:09:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.607+0000 7f9d4e400640 1 -- 172.21.15.19:0/3072324893 shutdown_connections 2024-03-28T13:09:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.607+0000 7f9d4e400640 1 --2- 172.21.15.19:0/3072324893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d2c0039b0 0x7f9d2c003d90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.607+0000 7f9d4e400640 1 -- 172.21.15.19:0/3072324893 >> 172.21.15.19:0/3072324893 conn(0x7f9d480b3e90 msgr2=0x7f9d480b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.607+0000 7f9d4e400640 1 -- 172.21.15.19:0/3072324893 shutdown_connections 2024-03-28T13:09:16.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.607+0000 7f9d4e400640 1 -- 172.21.15.19:0/3072324893 wait complete. 2024-03-28T13:09:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.609+0000 7f9d4e400640 1 Processor -- start 2024-03-28T13:09:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.609+0000 7f9d4e400640 1 -- start start 2024-03-28T13:09:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.610+0000 7f9d4e400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d2c0039b0 0x7f9d481531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.610+0000 7f9d4e400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d4815ffc0 0x7f9d480cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.610+0000 7f9d4e400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d480cbd70 0x7f9d480cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.610+0000 7f9d4e400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9d480cfe30 con 0x7f9d480cbd70 2024-03-28T13:09:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.610+0000 7f9d4e400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9d480d0d20 con 0x7f9d2c0039b0 2024-03-28T13:09:16.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.610+0000 7f9d4e400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9d480b4980 con 0x7f9d4815ffc0 2024-03-28T13:09:16.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.610+0000 7f9d47e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d480cbd70 0x7f9d480cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:16.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.611+0000 7f9d47400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d2c0039b0 0x7f9d481531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.611+0000 7f9d47e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d480cbd70 0x7f9d480cec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42786/0 (socket says 172.21.15.19:42786) 2024-03-28T13:09:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.611+0000 7f9d47e00640 1 -- 172.21.15.19:0/229851110 learned_addr learned my addr 172.21.15.19:0/229851110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.611+0000 7f9d46a00640 1 --2- 172.21.15.19:0/229851110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d4815ffc0 0x7f9d480cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.611+0000 7f9d47400640 1 -- 172.21.15.19:0/229851110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d4815ffc0 msgr2=0x7f9d480cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.611+0000 7f9d47400640 1 --2- 172.21.15.19:0/229851110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d4815ffc0 0x7f9d480cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.611+0000 7f9d47400640 1 -- 172.21.15.19:0/229851110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d480cbd70 msgr2=0x7f9d480cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.611+0000 7f9d47400640 1 --2- 172.21.15.19:0/229851110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d480cbd70 0x7f9d480cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.611+0000 7f9d47400640 1 -- 172.21.15.19:0/229851110 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4815ef20 con 0x7f9d2c0039b0 2024-03-28T13:09:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.611+0000 7f9d47e00640 1 --2- 172.21.15.19:0/229851110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d480cbd70 0x7f9d480cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:09:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.611+0000 7f9d46a00640 1 --2- 172.21.15.19:0/229851110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d4815ffc0 0x7f9d480cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.611+0000 7f9d47400640 1 --2- 172.21.15.19:0/229851110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d2c0039b0 0x7f9d481531b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00c0c0 tx=0x7f9d38004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.612+0000 7f9d3f400640 1 -- 172.21.15.19:0/229851110 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38034040 con 0x7f9d2c0039b0 2024-03-28T13:09:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.612+0000 7f9d3f400640 1 -- 172.21.15.19:0/229851110 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d3800f000 con 0x7f9d2c0039b0 2024-03-28T13:09:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.612+0000 7f9d3f400640 1 -- 172.21.15.19:0/229851110 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3803aa20 con 0x7f9d2c0039b0 2024-03-28T13:09:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.612+0000 7f9d4e400640 1 -- 172.21.15.19:0/229851110 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4815f200 con 0x7f9d2c0039b0 2024-03-28T13:09:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.612+0000 7f9d4e400640 1 -- 172.21.15.19:0/229851110 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d4815f7c0 con 0x7f9d2c0039b0 2024-03-28T13:09:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.612+0000 7f9d3f400640 1 -- 172.21.15.19:0/229851110 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9d38037070 con 0x7f9d2c0039b0 2024-03-28T13:09:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.612+0000 7f9d3f400640 1 -- 172.21.15.19:0/229851110 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9d3804cdd0 con 0x7f9d2c0039b0 2024-03-28T13:09:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.613+0000 7f9d4e400640 1 -- 172.21.15.19:0/229851110 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d48068cd0 con 0x7f9d2c0039b0 2024-03-28T13:09:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.616+0000 7f9d3f400640 1 -- 172.21.15.19:0/229851110 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9d3803abe0 con 0x7f9d2c0039b0 2024-03-28T13:09:16.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.688+0000 7f9d4e400640 1 -- 172.21.15.19:0/229851110 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f9d4806de80 con 0x7f9d2c0039b0 2024-03-28T13:09:16.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.696+0000 7f9d3f400640 1 -- 172.21.15.19:0/229851110 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f9d3803adc0 con 0x7f9d2c0039b0 2024-03-28T13:09:16.696 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2024-03-28T13:09:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.697+0000 7f9d4e400640 1 -- 172.21.15.19:0/229851110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d2c0039b0 msgr2=0x7f9d481531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.697+0000 7f9d4e400640 1 --2- 172.21.15.19:0/229851110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d2c0039b0 0x7f9d481531b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c00c0c0 tx=0x7f9d38004730 comp rx=0 tx=0).stop 2024-03-28T13:09:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.697+0000 7f9d4e400640 1 -- 172.21.15.19:0/229851110 shutdown_connections 2024-03-28T13:09:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.697+0000 7f9d4e400640 1 --2- 172.21.15.19:0/229851110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d480cbd70 0x7f9d480cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.697+0000 7f9d4e400640 1 --2- 172.21.15.19:0/229851110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d4815ffc0 0x7f9d480cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.697+0000 7f9d4e400640 1 --2- 172.21.15.19:0/229851110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d2c0039b0 0x7f9d481531b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.697+0000 7f9d4e400640 1 -- 172.21.15.19:0/229851110 >> 172.21.15.19:0/229851110 conn(0x7f9d480b3e90 msgr2=0x7f9d4806ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.697+0000 7f9d4e400640 1 -- 172.21.15.19:0/229851110 shutdown_connections 2024-03-28T13:09:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:16.697+0000 7f9d4e400640 1 -- 172.21.15.19:0/229851110 wait complete. 2024-03-28T13:09:16.715 INFO:tasks.ceph:check_enable_crimson: False 2024-03-28T13:09:16.715 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-28T13:09:16.715 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:09:16.715 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-28T13:09:16.803 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:09:16.804 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-28T13:09:16.880 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:09:16.880 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-28T13:09:17.015 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:09:17.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-28T13:09:17.092 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:09:17.092 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-28T13:09:17.178 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:09:17.178 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-28T13:09:17.303 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:09:17.303 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-28T13:09:17.387 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:17.388+0000 7fd85a9b8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T13:09:17.409 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:17.409+0000 7fb0e2673040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-28T13:09:17.432 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:09:17.432 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-28T13:09:17.452 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:17.452+0000 7fd85a9b8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T13:09:17.480 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:17.480+0000 7fb0e2673040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-28T13:09:17.534 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new d13c2148-f3e7-4d77-9c12-0f722feeba16 0 2024-03-28T13:09:17.721 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.722+0000 7fe3bd200640 1 Processor -- start 2024-03-28T13:09:17.722 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.722+0000 7fe3bd200640 1 -- start start 2024-03-28T13:09:17.723 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.723+0000 7fe3bd200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe3b80cf630 0x7fe3b80b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:17.723 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.724+0000 7fe3bd200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3b80b8320 con 0x7fe3b815e460 2024-03-28T13:09:17.723 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.724+0000 7fe3bd200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3b80d2da0 con 0x7fe3b80d3e40 2024-03-28T13:09:17.723 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.724+0000 7fe3bd200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe3b80d3080 con 0x7fe3b80cf630 2024-03-28T13:09:17.723 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.724+0000 7fe3b6a00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fe3b80d3e40 0x7fe3b80cef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:53574/0 (socket says 172.21.15.188:53574) 2024-03-28T13:09:17.723 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.724+0000 7fe3b6a00640 1 -- 172.21.15.188:0/1807106406 learned_addr learned my addr 172.21.15.188:0/1807106406 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-03-28T13:09:17.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.724+0000 7fe3b7e00640 1 --2- 172.21.15.188:0/1807106406 >> v2:172.21.15.19:3300/0 conn(0x7fe3b80cf630 0x7fe3b80b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:17.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.724+0000 7fe3b6000640 1 -- 172.21.15.188:0/1807106406 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4285012303 0 0) 0x7fe3b80d2da0 con 0x7fe3b80d3e40 2024-03-28T13:09:17.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.724+0000 7fe3b6000640 1 -- 172.21.15.188:0/1807106406 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe39c003520 con 0x7fe3b80d3e40 2024-03-28T13:09:17.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.724+0000 7fe3b6000640 1 -- 172.21.15.188:0/1807106406 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 525735395 0 0) 0x7fe39c003520 con 0x7fe3b80d3e40 2024-03-28T13:09:17.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.724+0000 7fe3b6000640 1 -- 172.21.15.188:0/1807106406 >> v2:172.21.15.19:3300/0 conn(0x7fe3b80cf630 msgr2=0x7fe3b80b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:17.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.724+0000 7fe3b6000640 1 --2- 172.21.15.188:0/1807106406 >> v2:172.21.15.19:3300/0 conn(0x7fe3b80cf630 0x7fe3b80b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:17.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b6000640 1 -- 172.21.15.188:0/1807106406 >> v1:172.21.15.19:6789/0 conn(0x7fe3b815e460 legacy=0x7fe3b80d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:17.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b6000640 1 -- 172.21.15.188:0/1807106406 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b80d3360 con 0x7fe3b80d3e40 2024-03-28T13:09:17.726 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b6000640 1 -- 172.21.15.188:0/1807106406 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fe3a40017f0 con 0x7fe3b80d3e40 2024-03-28T13:09:17.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b6000640 1 -- 172.21.15.188:0/1807106406 >> v1:172.21.15.188:6789/0 conn(0x7fe3b80d3e40 legacy=0x7fe3b80cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:17.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b6000640 1 --2- 172.21.15.188:0/1807106406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe39c0039b0 0x7fe39c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:17.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b6000640 1 --2- 172.21.15.188:0/1807106406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe39c004800 0x7fe39c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:17.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b6000640 1 --2- 172.21.15.188:0/1807106406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe39c0051c0 0x7fe39c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:17.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b6000640 1 -- 172.21.15.188:0/1807106406 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b80d3080 con 0x7fe39c004800 2024-03-28T13:09:17.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b6000640 1 -- 172.21.15.188:0/1807106406 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c00c110 con 0x7fe39c0039b0 2024-03-28T13:09:17.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b6000640 1 -- 172.21.15.188:0/1807106406 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe39c00c310 con 0x7fe39c0051c0 2024-03-28T13:09:17.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b7400640 1 --2- 172.21.15.188:0/1807106406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe39c0039b0 0x7fe39c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:17.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b7e00640 1 --2- 172.21.15.188:0/1807106406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe39c0051c0 0x7fe39c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:17.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b6a00640 1 --2- 172.21.15.188:0/1807106406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe39c004800 0x7fe39c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:17.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b7400640 1 -- 172.21.15.188:0/1807106406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe39c0051c0 msgr2=0x7fe39c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:17.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b7400640 1 --2- 172.21.15.188:0/1807106406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe39c0051c0 0x7fe39c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:17.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b7400640 1 -- 172.21.15.188:0/1807106406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe39c004800 msgr2=0x7fe39c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:17.727 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b7400640 1 --2- 172.21.15.188:0/1807106406 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe39c004800 0x7fe39c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:17.728 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.725+0000 7fe3b7400640 1 -- 172.21.15.188:0/1807106406 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c003520 con 0x7fe39c0039b0 2024-03-28T13:09:17.728 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.726+0000 7fe3b7e00640 1 -- 172.21.15.188:0/1807106406 reap_dead start 2024-03-28T13:09:17.728 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.726+0000 7fe3b7400640 1 --2- 172.21.15.188:0/1807106406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe39c0039b0 0x7fe39c003dd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8001750 tx=0x7fe39c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3abb747066c5f990 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:17.728 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.727+0000 7fe3b6000640 1 -- 172.21.15.188:0/1807106406 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8001370 con 0x7fe39c0039b0 2024-03-28T13:09:17.728 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.727+0000 7fe3b6000640 1 -- 172.21.15.188:0/1807106406 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a80052d0 con 0x7fe39c0039b0 2024-03-28T13:09:17.728 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.727+0000 7fe3b6000640 1 -- 172.21.15.188:0/1807106406 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8003830 con 0x7fe39c0039b0 2024-03-28T13:09:17.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.727+0000 7fe3bd200640 1 -- 172.21.15.188:0/1807106406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe39c0039b0 msgr2=0x7fe39c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:17.731 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.727+0000 7fe3bd200640 1 --2- 172.21.15.188:0/1807106406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe39c0039b0 0x7fe39c003dd0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8001750 tx=0x7fe39c005830 comp rx=0 tx=0).stop 2024-03-28T13:09:17.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.727+0000 7fe3bd200640 1 -- 172.21.15.188:0/1807106406 shutdown_connections 2024-03-28T13:09:17.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.727+0000 7fe3bd200640 1 --2- 172.21.15.188:0/1807106406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe39c0039b0 0x7fe39c003dd0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:17.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.727+0000 7fe3bd200640 1 -- 172.21.15.188:0/1807106406 >> 172.21.15.188:0/1807106406 conn(0x7fe3b80678b0 msgr2=0x7fe3b80d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:17.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.727+0000 7fe3bd200640 1 -- 172.21.15.188:0/1807106406 shutdown_connections 2024-03-28T13:09:17.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.727+0000 7fe3bd200640 1 -- 172.21.15.188:0/1807106406 wait complete. 2024-03-28T13:09:17.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.729+0000 7fe3bd200640 1 Processor -- start 2024-03-28T13:09:17.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.729+0000 7fe3bd200640 1 -- start start 2024-03-28T13:09:17.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.730+0000 7fe3bd200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe39c0039b0 0x7fe3b81f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:17.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.730+0000 7fe3bd200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3b815f490 0x7fe3b815f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:17.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.730+0000 7fe3bd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b81f41e0 0x7fe3b81f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:17.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.730+0000 7fe3bd200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b80b9e20 con 0x7fe3b81f41e0 2024-03-28T13:09:17.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.730+0000 7fe3bd200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b80d3920 con 0x7fe39c0039b0 2024-03-28T13:09:17.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.730+0000 7fe3bd200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe3b80b9fa0 con 0x7fe3b815f490 2024-03-28T13:09:17.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.730+0000 7fe3b6a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3b815f490 0x7fe3b815f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:17.732 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.730+0000 7fe3b7400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe39c0039b0 0x7fe3b81f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:17.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.730+0000 7fe3b7400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe39c0039b0 0x7fe3b81f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53886/0 (socket says 172.21.15.188:53886) 2024-03-28T13:09:17.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.730+0000 7fe3b7400640 1 -- 172.21.15.188:0/3463261047 learned_addr learned my addr 172.21.15.188:0/3463261047 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-28T13:09:17.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.731+0000 7fe3b7e00640 1 --2- 172.21.15.188:0/3463261047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b81f41e0 0x7fe3b81f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:17.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.731+0000 7fe3b7400640 1 -- 172.21.15.188:0/3463261047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3b815f490 msgr2=0x7fe3b815f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:17.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.731+0000 7fe3b7400640 1 --2- 172.21.15.188:0/3463261047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3b815f490 0x7fe3b815f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:17.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.731+0000 7fe3b7400640 1 -- 172.21.15.188:0/3463261047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b81f41e0 msgr2=0x7fe3b81f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:17.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.731+0000 7fe3b7400640 1 --2- 172.21.15.188:0/3463261047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b81f41e0 0x7fe3b81f4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:17.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.731+0000 7fe3b7400640 1 -- 172.21.15.188:0/3463261047 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b815ff20 con 0x7fe39c0039b0 2024-03-28T13:09:17.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.731+0000 7fe3b7400640 1 --2- 172.21.15.188:0/3463261047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe39c0039b0 0x7fe3b81f6c60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8005040 tx=0x7fe3a8001120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:17.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.731+0000 7fe3af400640 1 -- 172.21.15.188:0/3463261047 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8041020 con 0x7fe39c0039b0 2024-03-28T13:09:17.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.731+0000 7fe3af400640 1 -- 172.21.15.188:0/3463261047 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3a8005480 con 0x7fe39c0039b0 2024-03-28T13:09:17.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.731+0000 7fe3bd200640 1 -- 172.21.15.188:0/3463261047 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b8160200 con 0x7fe39c0039b0 2024-03-28T13:09:17.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.731+0000 7fe3af400640 1 -- 172.21.15.188:0/3463261047 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8001370 con 0x7fe39c0039b0 2024-03-28T13:09:17.733 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.731+0000 7fe3bd200640 1 -- 172.21.15.188:0/3463261047 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b81607c0 con 0x7fe39c0039b0 2024-03-28T13:09:17.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.732+0000 7fe3af400640 1 -- 172.21.15.188:0/3463261047 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe3a8044020 con 0x7fe39c0039b0 2024-03-28T13:09:17.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.732+0000 7fe3af400640 1 -- 172.21.15.188:0/3463261047 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fe3a803d740 con 0x7fe39c0039b0 2024-03-28T13:09:17.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.732+0000 7fe3bd200640 1 -- 172.21.15.188:0/3463261047 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b8075940 con 0x7fe39c0039b0 2024-03-28T13:09:17.734 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.735+0000 7fe3af400640 1 -- 172.21.15.188:0/3463261047 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe3a804b030 con 0x7fe39c0039b0 2024-03-28T13:09:17.813 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.814+0000 7fe3bd200640 1 -- 172.21.15.188:0/3463261047 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d13c2148-f3e7-4d77-9c12-0f722feeba16", "id": 0} v 0) -- 0x7fe3b80cf050 con 0x7fe39c0039b0 2024-03-28T13:09:17.819 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.820+0000 7fe3af400640 1 -- 172.21.15.188:0/3463261047 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d13c2148-f3e7-4d77-9c12-0f722feeba16", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fe3a8003bb0 con 0x7fe39c0039b0 2024-03-28T13:09:17.819 INFO:teuthology.orchestra.run.smithi188.stdout:0 2024-03-28T13:09:17.821 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.821+0000 7fe3bd200640 1 -- 172.21.15.188:0/3463261047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe39c0039b0 msgr2=0x7fe3b81f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:17.821 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.821+0000 7fe3bd200640 1 --2- 172.21.15.188:0/3463261047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe39c0039b0 0x7fe3b81f6c60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8005040 tx=0x7fe3a8001120 comp rx=0 tx=0).stop 2024-03-28T13:09:17.821 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.822+0000 7fe3bd200640 1 -- 172.21.15.188:0/3463261047 shutdown_connections 2024-03-28T13:09:17.821 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.822+0000 7fe3bd200640 1 --2- 172.21.15.188:0/3463261047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3b81f41e0 0x7fe3b81f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:17.821 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.822+0000 7fe3bd200640 1 --2- 172.21.15.188:0/3463261047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3b815f490 0x7fe3b815f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:17.821 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.822+0000 7fe3bd200640 1 --2- 172.21.15.188:0/3463261047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe39c0039b0 0x7fe3b81f6c60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:17.821 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.822+0000 7fe3bd200640 1 -- 172.21.15.188:0/3463261047 >> 172.21.15.188:0/3463261047 conn(0x7fe3b80678b0 msgr2=0x7fe3b80c9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:17.822 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.822+0000 7fe3bd200640 1 -- 172.21.15.188:0/3463261047 shutdown_connections 2024-03-28T13:09:17.822 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:17.822+0000 7fe3bd200640 1 -- 172.21.15.188:0/3463261047 wait complete. 2024-03-28T13:09:17.835 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 9ece52d8-56dc-4c72-882a-46f1ea5685cb 1 2024-03-28T13:09:18.021 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.021+0000 7fb275a00640 1 Processor -- start 2024-03-28T13:09:18.021 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.021+0000 7fb275a00640 1 -- start start 2024-03-28T13:09:18.023 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.022+0000 7fb275a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb27015e860 0x7fb27006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.023 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.022+0000 7fb275a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb27006da30 0x7fb2700d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.023 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.022+0000 7fb275a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb27006dfa0 con 0x7fb2700d4910 2024-03-28T13:09:18.023 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.023+0000 7fb275a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb2700d4d60 con 0x7fb27006da30 2024-03-28T13:09:18.023 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.023+0000 7fb275a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb2700cf220 con 0x7fb27015e860 2024-03-28T13:09:18.023 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.023+0000 7fb267e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb27006da30 0x7fb2700d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.023 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.023+0000 7fb26ea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb27015e860 0x7fb27006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.024 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.023+0000 7fb26f400640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fb2700d4910 0x7fb2700d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:53590/0 (socket says 172.21.15.188:53590) 2024-03-28T13:09:18.024 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.023+0000 7fb267e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb27006da30 0x7fb2700d43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53888/0 (socket says 172.21.15.188:53888) 2024-03-28T13:09:18.024 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.023+0000 7fb26f400640 1 -- 172.21.15.188:0/3988058759 learned_addr learned my addr 172.21.15.188:0/3988058759 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-03-28T13:09:18.024 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.023+0000 7fb26ea00640 1 -- 172.21.15.188:0/3988058759 >> v2:172.21.15.188:3300/0 conn(0x7fb27006da30 msgr2=0x7fb2700d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:18.024 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.023+0000 7fb26ea00640 1 --2- 172.21.15.188:0/3988058759 >> v2:172.21.15.188:3300/0 conn(0x7fb27006da30 0x7fb2700d43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.024 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.023+0000 7fb26ea00640 1 -- 172.21.15.188:0/3988058759 >> v1:172.21.15.188:6789/0 conn(0x7fb2700d4910 legacy=0x7fb2700d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.024 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.023+0000 7fb26ea00640 1 -- 172.21.15.188:0/3988058759 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27006e240 con 0x7fb27015e860 2024-03-28T13:09:18.024 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.023+0000 7fb267e00640 1 --2- 172.21.15.188:0/3988058759 >> v2:172.21.15.188:3300/0 conn(0x7fb27006da30 0x7fb2700d43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:09:18.025 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.024+0000 7fb26ea00640 1 --2- 172.21.15.188:0/3988058759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb27015e860 0x7fb27006d4f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb2600097a0 tx=0x7fb26002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76d0f8a147a4454e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:18.026 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.025+0000 7fb26e000640 1 -- 172.21.15.188:0/3988058759 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260035020 con 0x7fb27015e860 2024-03-28T13:09:18.026 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.025+0000 7fb26e000640 1 -- 172.21.15.188:0/3988058759 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb260031810 con 0x7fb27015e860 2024-03-28T13:09:18.026 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.025+0000 7fb26e000640 1 -- 172.21.15.188:0/3988058759 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260031be0 con 0x7fb27015e860 2024-03-28T13:09:18.026 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.025+0000 7fb275a00640 1 -- 172.21.15.188:0/3988058759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb27015e860 msgr2=0x7fb27006d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.026 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.025+0000 7fb275a00640 1 --2- 172.21.15.188:0/3988058759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb27015e860 0x7fb27006d4f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb2600097a0 tx=0x7fb26002b8b0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.026 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.025+0000 7fb275a00640 1 -- 172.21.15.188:0/3988058759 shutdown_connections 2024-03-28T13:09:18.026 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.026+0000 7fb275a00640 1 --2- 172.21.15.188:0/3988058759 >> v2:172.21.15.188:3300/0 conn(0x7fb27006da30 0x7fb2700d43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.026 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.026+0000 7fb275a00640 1 --2- 172.21.15.188:0/3988058759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb27015e860 0x7fb27006d4f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.026 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.026+0000 7fb275a00640 1 -- 172.21.15.188:0/3988058759 >> 172.21.15.188:0/3988058759 conn(0x7fb27009d7d0 msgr2=0x7fb27008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:18.026 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.026+0000 7fb275a00640 1 -- 172.21.15.188:0/3988058759 shutdown_connections 2024-03-28T13:09:18.027 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.026+0000 7fb275a00640 1 -- 172.21.15.188:0/3988058759 wait complete. 2024-03-28T13:09:18.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.027+0000 7fb275a00640 1 Processor -- start 2024-03-28T13:09:18.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.027+0000 7fb275a00640 1 -- start start 2024-03-28T13:09:18.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.028+0000 7fb275a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27006da30 0x7fb2701f6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.028+0000 7fb275a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2700d4910 0x7fb2700c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.028+0000 7fb275a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb27015e860 0x7fb2700c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.028+0000 7fb275a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2700cedb0 con 0x7fb2700d4910 2024-03-28T13:09:18.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.028+0000 7fb275a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb2700d2c50 con 0x7fb27006da30 2024-03-28T13:09:18.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.028+0000 7fb275a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb2700cef30 con 0x7fb27015e860 2024-03-28T13:09:18.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.028+0000 7fb26ea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27006da30 0x7fb2701f6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.029+0000 7fb26ea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27006da30 0x7fb2701f6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53904/0 (socket says 172.21.15.188:53904) 2024-03-28T13:09:18.030 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.029+0000 7fb26ea00640 1 -- 172.21.15.188:0/3230885033 learned_addr learned my addr 172.21.15.188:0/3230885033 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-28T13:09:18.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.029+0000 7fb26f400640 1 --2- 172.21.15.188:0/3230885033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb27015e860 0x7fb2700c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.029+0000 7fb267e00640 1 --2- 172.21.15.188:0/3230885033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2700d4910 0x7fb2700c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.029+0000 7fb26ea00640 1 -- 172.21.15.188:0/3230885033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb27015e860 msgr2=0x7fb2700c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.029+0000 7fb26ea00640 1 --2- 172.21.15.188:0/3230885033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb27015e860 0x7fb2700c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.029+0000 7fb26ea00640 1 -- 172.21.15.188:0/3230885033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2700d4910 msgr2=0x7fb2700c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.029+0000 7fb26ea00640 1 --2- 172.21.15.188:0/3230885033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2700d4910 0x7fb2700c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.029+0000 7fb26ea00640 1 -- 172.21.15.188:0/3230885033 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2700b7c00 con 0x7fb27006da30 2024-03-28T13:09:18.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.029+0000 7fb267e00640 1 --2- 172.21.15.188:0/3230885033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2700d4910 0x7fb2700c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:18.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.029+0000 7fb26ea00640 1 --2- 172.21.15.188:0/3230885033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27006da30 0x7fb2701f6cd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb260000c00 tx=0x7fb2600046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:18.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.030+0000 7fb266a00640 1 -- 172.21.15.188:0/3230885033 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260040070 con 0x7fb27006da30 2024-03-28T13:09:18.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.030+0000 7fb266a00640 1 -- 172.21.15.188:0/3230885033 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb260039cf0 con 0x7fb27006da30 2024-03-28T13:09:18.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.030+0000 7fb266a00640 1 -- 172.21.15.188:0/3230885033 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260035040 con 0x7fb27006da30 2024-03-28T13:09:18.032 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.030+0000 7fb275a00640 1 -- 172.21.15.188:0/3230885033 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2700b7ee0 con 0x7fb27006da30 2024-03-28T13:09:18.032 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.030+0000 7fb275a00640 1 -- 172.21.15.188:0/3230885033 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2700b84a0 con 0x7fb27006da30 2024-03-28T13:09:18.032 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.030+0000 7fb266a00640 1 -- 172.21.15.188:0/3230885033 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb26004b060 con 0x7fb27006da30 2024-03-28T13:09:18.032 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.030+0000 7fb266a00640 1 -- 172.21.15.188:0/3230885033 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fb26004d800 con 0x7fb27006da30 2024-03-28T13:09:18.032 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.031+0000 7fb275a00640 1 -- 172.21.15.188:0/3230885033 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2700b87d0 con 0x7fb27006da30 2024-03-28T13:09:18.033 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.034+0000 7fb266a00640 1 -- 172.21.15.188:0/3230885033 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb260052020 con 0x7fb27006da30 2024-03-28T13:09:18.105 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:18.106+0000 7fd85a9b8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T13:09:18.113 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:18.112+0000 7fb0e2673040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-28T13:09:18.125 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.125+0000 7fb275a00640 1 -- 172.21.15.188:0/3230885033 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9ece52d8-56dc-4c72-882a-46f1ea5685cb", "id": 1} v 0) -- 0x7fb2700685c0 con 0x7fb27006da30 2024-03-28T13:09:18.132 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.132+0000 7fb266a00640 1 -- 172.21.15.188:0/3230885033 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9ece52d8-56dc-4c72-882a-46f1ea5685cb", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fb26003acd0 con 0x7fb27006da30 2024-03-28T13:09:18.132 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-03-28T13:09:18.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.134+0000 7fb275a00640 1 -- 172.21.15.188:0/3230885033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27006da30 msgr2=0x7fb2701f6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.134+0000 7fb275a00640 1 --2- 172.21.15.188:0/3230885033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27006da30 0x7fb2701f6cd0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb260000c00 tx=0x7fb2600046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.134+0000 7fb275a00640 1 -- 172.21.15.188:0/3230885033 shutdown_connections 2024-03-28T13:09:18.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.134+0000 7fb275a00640 1 --2- 172.21.15.188:0/3230885033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb27015e860 0x7fb2700c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.134+0000 7fb275a00640 1 --2- 172.21.15.188:0/3230885033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2700d4910 0x7fb2700c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.133 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.134+0000 7fb275a00640 1 --2- 172.21.15.188:0/3230885033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27006da30 0x7fb2701f6cd0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.134 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.134+0000 7fb275a00640 1 -- 172.21.15.188:0/3230885033 >> 172.21.15.188:0/3230885033 conn(0x7fb27009d7d0 msgr2=0x7fb2700b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:18.134 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.134+0000 7fb275a00640 1 -- 172.21.15.188:0/3230885033 shutdown_connections 2024-03-28T13:09:18.134 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.134+0000 7fb275a00640 1 -- 172.21.15.188:0/3230885033 wait complete. 2024-03-28T13:09:18.151 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 64162cf7-5b2e-483d-9b2b-1366fb36d134 2 2024-03-28T13:09:18.174 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:18.175+0000 7fd85a9b8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T13:09:18.181 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:18.181+0000 7fb0e2673040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-28T13:09:18.228 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:18.229+0000 7fd85a9b8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T13:09:18.236 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:18.236+0000 7fb0e2673040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-28T13:09:18.293 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:18.294+0000 7fd85a9b8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T13:09:18.303 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:18.303+0000 7fb0e2673040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-28T13:09:18.335 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.336+0000 7f34afe00640 1 Processor -- start 2024-03-28T13:09:18.336 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.336+0000 7f34afe00640 1 -- start start 2024-03-28T13:09:18.338 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.338+0000 7f34afe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34b015b0f0 0x7f34b0067490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.338 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.338+0000 7f34afe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f34b015f380 0x7f34b00b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.338 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.338+0000 7f34afe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34b015fa50 con 0x7f34b006c680 2024-03-28T13:09:18.338 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.338+0000 7f34afe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f34b006cbf0 con 0x7f34b015f380 2024-03-28T13:09:18.339 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.338+0000 7f34afe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f34b015b4d0 con 0x7f34b015b0f0 2024-03-28T13:09:18.339 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.338+0000 7f34aea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34b015b0f0 0x7f34b0067490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.339 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.338+0000 7f34a6000640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f34b006c680 0x7f34b015ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:53598/0 (socket says 172.21.15.188:53598) 2024-03-28T13:09:18.339 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.338+0000 7f34aea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34b015b0f0 0x7f34b0067490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:41108/0 (socket says 172.21.15.188:41108) 2024-03-28T13:09:18.339 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.338+0000 7f34aea00640 1 -- 172.21.15.188:0/2066722447 learned_addr learned my addr 172.21.15.188:0/2066722447 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-28T13:09:18.339 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.338+0000 7f34af400640 1 --2- 172.21.15.188:0/2066722447 >> v2:172.21.15.19:3300/0 conn(0x7f34b015f380 0x7f34b00b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.339 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.339+0000 7f34aea00640 1 -- 172.21.15.188:0/2066722447 >> v2:172.21.15.19:3300/0 conn(0x7f34b015f380 msgr2=0x7f34b00b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:18.339 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.339+0000 7f34aea00640 1 --2- 172.21.15.188:0/2066722447 >> v2:172.21.15.19:3300/0 conn(0x7f34b015f380 0x7f34b00b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.339 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.339+0000 7f34aea00640 1 -- 172.21.15.188:0/2066722447 >> v1:172.21.15.188:6789/0 conn(0x7f34b006c680 legacy=0x7f34b015ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.339 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.339+0000 7f34aea00640 1 -- 172.21.15.188:0/2066722447 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b00b3800 con 0x7f34b015b0f0 2024-03-28T13:09:18.339 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.339+0000 7f34ae000640 1 -- 172.21.15.188:0/2066722447 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1282992650 0 0) 0x7f3498001110 con 0x7f34b006c680 2024-03-28T13:09:18.341 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.340+0000 7f34aea00640 1 --2- 172.21.15.188:0/2066722447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34b015b0f0 0x7f34b0067490 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009e50 tx=0x7f34a002bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7959b7e57d0b5d8f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:18.341 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.340+0000 7f34ae000640 1 -- 172.21.15.188:0/2066722447 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a002caf0 con 0x7f34b015b0f0 2024-03-28T13:09:18.341 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.340+0000 7f34ae000640 1 -- 172.21.15.188:0/2066722447 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a002cc90 con 0x7f34b015b0f0 2024-03-28T13:09:18.341 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.340+0000 7f34ae000640 1 -- 172.21.15.188:0/2066722447 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0034610 con 0x7f34b015b0f0 2024-03-28T13:09:18.341 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.340+0000 7f34afe00640 1 -- 172.21.15.188:0/2066722447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34b015b0f0 msgr2=0x7f34b0067490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.341 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.340+0000 7f34afe00640 1 --2- 172.21.15.188:0/2066722447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34b015b0f0 0x7f34b0067490 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f34a0009e50 tx=0x7f34a002bf50 comp rx=0 tx=0).stop 2024-03-28T13:09:18.341 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.341+0000 7f34afe00640 1 -- 172.21.15.188:0/2066722447 shutdown_connections 2024-03-28T13:09:18.341 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.341+0000 7f34afe00640 1 --2- 172.21.15.188:0/2066722447 >> v2:172.21.15.19:3300/0 conn(0x7f34b015f380 0x7f34b00b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.341 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.341+0000 7f34afe00640 1 --2- 172.21.15.188:0/2066722447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34b015b0f0 0x7f34b0067490 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.341 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.341+0000 7f34afe00640 1 -- 172.21.15.188:0/2066722447 >> 172.21.15.188:0/2066722447 conn(0x7f34b00d0770 msgr2=0x7f34b00d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:18.342 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.341+0000 7f34afe00640 1 -- 172.21.15.188:0/2066722447 shutdown_connections 2024-03-28T13:09:18.342 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.341+0000 7f34afe00640 1 -- 172.21.15.188:0/2066722447 wait complete. 2024-03-28T13:09:18.344 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.342+0000 7f34afe00640 1 Processor -- start 2024-03-28T13:09:18.345 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.342+0000 7f34afe00640 1 -- start start 2024-03-28T13:09:18.345 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.343+0000 7f34afe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34b006c680 0x7f34b015de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.345 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.343+0000 7f34afe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34b015b0f0 0x7f34b0155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.345 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.343+0000 7f34afe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b015f380 0x7f34b0156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.345 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.343+0000 7f34afe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34b00b40a0 con 0x7f34b015f380 2024-03-28T13:09:18.345 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.343+0000 7f34afe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f34b00b3dc0 con 0x7f34b006c680 2024-03-28T13:09:18.345 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.343+0000 7f34afe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f34b00ce100 con 0x7f34b015b0f0 2024-03-28T13:09:18.345 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.343+0000 7f34a6000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34b015b0f0 0x7f34b0155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.345 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.343+0000 7f34aea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34b006c680 0x7f34b015de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.345 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.344+0000 7f34a6000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34b015b0f0 0x7f34b0155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:41112/0 (socket says 172.21.15.188:41112) 2024-03-28T13:09:18.345 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.344+0000 7f34a6000640 1 -- 172.21.15.188:0/2403168746 learned_addr learned my addr 172.21.15.188:0/2403168746 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-28T13:09:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.344+0000 7f34af400640 1 --2- 172.21.15.188:0/2403168746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b015f380 0x7f34b0156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.344+0000 7f34a6000640 1 -- 172.21.15.188:0/2403168746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34b006c680 msgr2=0x7f34b015de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.344+0000 7f34a6000640 1 --2- 172.21.15.188:0/2403168746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34b006c680 0x7f34b015de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.344+0000 7f34a6000640 1 -- 172.21.15.188:0/2403168746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b015f380 msgr2=0x7f34b0156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.344+0000 7f34a6000640 1 --2- 172.21.15.188:0/2403168746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b015f380 0x7f34b0156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.344+0000 7f34a6000640 1 -- 172.21.15.188:0/2403168746 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b0145400 con 0x7f34b015b0f0 2024-03-28T13:09:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.344+0000 7f34a6000640 1 --2- 172.21.15.188:0/2403168746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34b015b0f0 0x7f34b0155ff0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f349800c460 tx=0x7f349800c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.344+0000 7f34a7e00640 1 -- 172.21.15.188:0/2403168746 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498018070 con 0x7f34b015b0f0 2024-03-28T13:09:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.344+0000 7f34a7e00640 1 -- 172.21.15.188:0/2403168746 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3498007500 con 0x7f34b015b0f0 2024-03-28T13:09:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.344+0000 7f34a7e00640 1 -- 172.21.15.188:0/2403168746 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498004a20 con 0x7f34b015b0f0 2024-03-28T13:09:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.344+0000 7f34afe00640 1 -- 172.21.15.188:0/2403168746 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b015b800 con 0x7f34b015b0f0 2024-03-28T13:09:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.345+0000 7f34afe00640 1 -- 172.21.15.188:0/2403168746 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b015bdc0 con 0x7f34b015b0f0 2024-03-28T13:09:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.345+0000 7f34a7e00640 1 -- 172.21.15.188:0/2403168746 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3498004bc0 con 0x7f34b015b0f0 2024-03-28T13:09:18.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.345+0000 7f34a7e00640 1 -- 172.21.15.188:0/2403168746 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f3498012820 con 0x7f34b015b0f0 2024-03-28T13:09:18.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.346+0000 7f34afe00640 1 -- 172.21.15.188:0/2403168746 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b0156e60 con 0x7f34b015b0f0 2024-03-28T13:09:18.347 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:18.348+0000 7fd85a9b8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T13:09:18.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.349+0000 7f34a7e00640 1 -- 172.21.15.188:0/2403168746 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f34b0156e60 con 0x7f34b015b0f0 2024-03-28T13:09:18.358 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:18.358+0000 7fb0e2673040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-28T13:09:18.437 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.438+0000 7f34afe00640 1 -- 172.21.15.188:0/2403168746 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "64162cf7-5b2e-483d-9b2b-1366fb36d134", "id": 2} v 0) -- 0x7f34b0075940 con 0x7f34b015b0f0 2024-03-28T13:09:18.444 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.445+0000 7f34a7e00640 1 -- 172.21.15.188:0/2403168746 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "64162cf7-5b2e-483d-9b2b-1366fb36d134", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f3498021070 con 0x7f34b015b0f0 2024-03-28T13:09:18.445 INFO:teuthology.orchestra.run.smithi188.stdout:2 2024-03-28T13:09:18.446 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.446+0000 7f34afe00640 1 -- 172.21.15.188:0/2403168746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34b015b0f0 msgr2=0x7f34b0155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.446 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.446+0000 7f34afe00640 1 --2- 172.21.15.188:0/2403168746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34b015b0f0 0x7f34b0155ff0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f349800c460 tx=0x7f349800c920 comp rx=0 tx=0).stop 2024-03-28T13:09:18.446 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.447+0000 7f34afe00640 1 -- 172.21.15.188:0/2403168746 shutdown_connections 2024-03-28T13:09:18.447 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.447+0000 7f34afe00640 1 --2- 172.21.15.188:0/2403168746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34b015f380 0x7f34b0156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.447 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.447+0000 7f34afe00640 1 --2- 172.21.15.188:0/2403168746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34b015b0f0 0x7f34b0155ff0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.447 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.447+0000 7f34afe00640 1 --2- 172.21.15.188:0/2403168746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34b006c680 0x7f34b015de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.447 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.447+0000 7f34afe00640 1 -- 172.21.15.188:0/2403168746 >> 172.21.15.188:0/2403168746 conn(0x7f34b00d0770 msgr2=0x7f34b00cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:18.447 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.447+0000 7f34afe00640 1 -- 172.21.15.188:0/2403168746 shutdown_connections 2024-03-28T13:09:18.447 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.447+0000 7f34afe00640 1 -- 172.21.15.188:0/2403168746 wait complete. 2024-03-28T13:09:18.462 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new e8162dd0-8c0f-4a78-809e-03f92586a52d 3 2024-03-28T13:09:18.493 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:18.493+0000 7fd85a9b8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T13:09:18.499 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:18.499+0000 7fb0e2673040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-28T13:09:18.577 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:18.578+0000 7fd85a9b8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T13:09:18.585 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:18.585+0000 7fb0e2673040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-28T13:09:18.635 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:18.636+0000 7fd85a9b8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T13:09:18.644 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:18.644+0000 7fb0e2673040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-28T13:09:18.684 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.684+0000 7fbd35000640 1 Processor -- start 2024-03-28T13:09:18.684 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.684+0000 7fbd35000640 1 -- start start 2024-03-28T13:09:18.686 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.685+0000 7fbd35000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3015e4c0 0x7fbd300b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.686 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd35000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbd300d37b0 0x7fbd300b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.686 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd35000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbd300b67f0 0x7fbd300d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.686 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd35000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbd3015e8a0 con 0x7fbd300b67f0 2024-03-28T13:09:18.686 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd35000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fbd300d4040 con 0x7fbd300d37b0 2024-03-28T13:09:18.686 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd35000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbd300d3320 con 0x7fbd3015e4c0 2024-03-28T13:09:18.687 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd2ea00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbd300d37b0 0x7fbd300b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.687 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd2f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3015e4c0 0x7fbd300b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.687 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd2ea00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbd300d37b0 0x7fbd300b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53924/0 (socket says 172.21.15.188:53924) 2024-03-28T13:09:18.687 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd2ea00640 1 -- 172.21.15.188:0/2799194796 learned_addr learned my addr 172.21.15.188:0/2799194796 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-28T13:09:18.687 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd2fe00640 1 --2- 172.21.15.188:0/2799194796 >> v2:172.21.15.19:3300/0 conn(0x7fbd300b67f0 0x7fbd300d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.687 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd2ea00640 1 -- 172.21.15.188:0/2799194796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3015e4c0 msgr2=0x7fbd300b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:18.687 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd2ea00640 1 --2- 172.21.15.188:0/2799194796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3015e4c0 0x7fbd300b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.687 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd2ea00640 1 -- 172.21.15.188:0/2799194796 >> v2:172.21.15.19:3300/0 conn(0x7fbd300b67f0 msgr2=0x7fbd300d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:18.687 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd2ea00640 1 --2- 172.21.15.188:0/2799194796 >> v2:172.21.15.19:3300/0 conn(0x7fbd300b67f0 0x7fbd300d4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.687 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd2ea00640 1 -- 172.21.15.188:0/2799194796 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd300d2710 con 0x7fbd300d37b0 2024-03-28T13:09:18.687 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.686+0000 7fbd2f400640 1 --2- 172.21.15.188:0/2799194796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3015e4c0 0x7fbd300b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:09:18.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.687+0000 7fbd2ea00640 1 --2- 172.21.15.188:0/2799194796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd300d37b0 0x7fbd300b62b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbd300d5330 tx=0x7fbd1c02b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=769e7309dead20cf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:18.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.688+0000 7fbd2e000640 1 -- 172.21.15.188:0/2799194796 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c035020 con 0x7fbd300d37b0 2024-03-28T13:09:18.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.688+0000 7fbd2e000640 1 -- 172.21.15.188:0/2799194796 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd1c031870 con 0x7fbd300d37b0 2024-03-28T13:09:18.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.688+0000 7fbd2e000640 1 -- 172.21.15.188:0/2799194796 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c031c40 con 0x7fbd300d37b0 2024-03-28T13:09:18.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.688+0000 7fbd35000640 1 -- 172.21.15.188:0/2799194796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd300d37b0 msgr2=0x7fbd300b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.688+0000 7fbd35000640 1 --2- 172.21.15.188:0/2799194796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd300d37b0 0x7fbd300b62b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbd300d5330 tx=0x7fbd1c02b910 comp rx=0 tx=0).stop 2024-03-28T13:09:18.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.689+0000 7fbd35000640 1 -- 172.21.15.188:0/2799194796 shutdown_connections 2024-03-28T13:09:18.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.689+0000 7fbd35000640 1 --2- 172.21.15.188:0/2799194796 >> v2:172.21.15.19:3300/0 conn(0x7fbd300b67f0 0x7fbd300d4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.689+0000 7fbd35000640 1 --2- 172.21.15.188:0/2799194796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd300d37b0 0x7fbd300b62b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.689+0000 7fbd35000640 1 --2- 172.21.15.188:0/2799194796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd3015e4c0 0x7fbd300b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.689+0000 7fbd35000640 1 -- 172.21.15.188:0/2799194796 >> 172.21.15.188:0/2799194796 conn(0x7fbd300676a0 msgr2=0x7fbd30067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:18.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.689+0000 7fbd35000640 1 -- 172.21.15.188:0/2799194796 shutdown_connections 2024-03-28T13:09:18.689 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.689+0000 7fbd35000640 1 -- 172.21.15.188:0/2799194796 wait complete. 2024-03-28T13:09:18.690 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.690+0000 7fbd35000640 1 Processor -- start 2024-03-28T13:09:18.690 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.690+0000 7fbd35000640 1 -- start start 2024-03-28T13:09:18.693 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.691+0000 7fbd35000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd300b67f0 0x7fbd301611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.693 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.691+0000 7fbd35000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd300d37b0 0x7fbd300cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.693 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.691+0000 7fbd35000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3015e4c0 0x7fbd300cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.693 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.691+0000 7fbd35000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd300d2fb0 con 0x7fbd3015e4c0 2024-03-28T13:09:18.693 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.691+0000 7fbd35000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbd300d2cd0 con 0x7fbd300b67f0 2024-03-28T13:09:18.693 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.691+0000 7fbd35000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbd300ba130 con 0x7fbd300d37b0 2024-03-28T13:09:18.693 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.692+0000 7fbd2f400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd300b67f0 0x7fbd301611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.693 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.692+0000 7fbd2f400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd300b67f0 0x7fbd301611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53926/0 (socket says 172.21.15.188:53926) 2024-03-28T13:09:18.693 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.692+0000 7fbd2f400640 1 -- 172.21.15.188:0/2972461989 learned_addr learned my addr 172.21.15.188:0/2972461989 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-28T13:09:18.693 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.692+0000 7fbd2ea00640 1 --2- 172.21.15.188:0/2972461989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd300d37b0 0x7fbd300cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.694 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.692+0000 7fbd2fe00640 1 --2- 172.21.15.188:0/2972461989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3015e4c0 0x7fbd300cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.694 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.692+0000 7fbd2f400640 1 -- 172.21.15.188:0/2972461989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd300d37b0 msgr2=0x7fbd300cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.694 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.692+0000 7fbd2f400640 1 --2- 172.21.15.188:0/2972461989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd300d37b0 0x7fbd300cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.694 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.692+0000 7fbd2f400640 1 -- 172.21.15.188:0/2972461989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3015e4c0 msgr2=0x7fbd300cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.694 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.692+0000 7fbd2f400640 1 --2- 172.21.15.188:0/2972461989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3015e4c0 0x7fbd300cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.694 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.692+0000 7fbd2f400640 1 -- 172.21.15.188:0/2972461989 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3015f960 con 0x7fbd300b67f0 2024-03-28T13:09:18.694 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.692+0000 7fbd2fe00640 1 --2- 172.21.15.188:0/2972461989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3015e4c0 0x7fbd300cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:18.694 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.692+0000 7fbd2f400640 1 --2- 172.21.15.188:0/2972461989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd300b67f0 0x7fbd301611e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e960 tx=0x7fbd2000ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:18.694 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.693+0000 7fbd27400640 1 -- 172.21.15.188:0/2972461989 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20017070 con 0x7fbd300b67f0 2024-03-28T13:09:18.694 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.693+0000 7fbd27400640 1 -- 172.21.15.188:0/2972461989 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd2000ca00 con 0x7fbd300b67f0 2024-03-28T13:09:18.694 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.693+0000 7fbd27400640 1 -- 172.21.15.188:0/2972461989 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2000cdd0 con 0x7fbd300b67f0 2024-03-28T13:09:18.694 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.693+0000 7fbd35000640 1 -- 172.21.15.188:0/2972461989 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3015fc40 con 0x7fbd300b67f0 2024-03-28T13:09:18.694 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.693+0000 7fbd35000640 1 -- 172.21.15.188:0/2972461989 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd30160200 con 0x7fbd300b67f0 2024-03-28T13:09:18.694 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.693+0000 7fbd27400640 1 -- 172.21.15.188:0/2972461989 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbd2001a070 con 0x7fbd300b67f0 2024-03-28T13:09:18.695 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.693+0000 7fbd27400640 1 -- 172.21.15.188:0/2972461989 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fbd20009400 con 0x7fbd300b67f0 2024-03-28T13:09:18.695 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.694+0000 7fbd35000640 1 -- 172.21.15.188:0/2972461989 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd300d1e80 con 0x7fbd300b67f0 2024-03-28T13:09:18.695 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.696+0000 7fbd27400640 1 -- 172.21.15.188:0/2972461989 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbd300d1e80 con 0x7fbd300b67f0 2024-03-28T13:09:18.699 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:18.700+0000 7fd85a9b8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T13:09:18.709 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:18.709+0000 7fb0e2673040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-28T13:09:18.756 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:18.756+0000 7fd85a9b8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T13:09:18.769 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:18.768+0000 7fb0e2673040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-28T13:09:18.774 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.774+0000 7fbd35000640 1 -- 172.21.15.188:0/2972461989 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e8162dd0-8c0f-4a78-809e-03f92586a52d", "id": 3} v 0) -- 0x7fbd301f4190 con 0x7fbd300b67f0 2024-03-28T13:09:18.781 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.782+0000 7fbd27400640 1 -- 172.21.15.188:0/2972461989 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e8162dd0-8c0f-4a78-809e-03f92586a52d", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fbd20013470 con 0x7fbd300b67f0 2024-03-28T13:09:18.782 INFO:teuthology.orchestra.run.smithi188.stdout:3 2024-03-28T13:09:18.783 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.783+0000 7fbd35000640 1 -- 172.21.15.188:0/2972461989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd300b67f0 msgr2=0x7fbd301611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.783 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.783+0000 7fbd35000640 1 --2- 172.21.15.188:0/2972461989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd300b67f0 0x7fbd301611e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e960 tx=0x7fbd2000ee20 comp rx=0 tx=0).stop 2024-03-28T13:09:18.783 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.784+0000 7fbd35000640 1 -- 172.21.15.188:0/2972461989 shutdown_connections 2024-03-28T13:09:18.783 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.784+0000 7fbd35000640 1 --2- 172.21.15.188:0/2972461989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd3015e4c0 0x7fbd300cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.783 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.784+0000 7fbd35000640 1 --2- 172.21.15.188:0/2972461989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd300d37b0 0x7fbd300cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.783 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.784+0000 7fbd35000640 1 --2- 172.21.15.188:0/2972461989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd300b67f0 0x7fbd301611e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.784 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.784+0000 7fbd35000640 1 -- 172.21.15.188:0/2972461989 >> 172.21.15.188:0/2972461989 conn(0x7fbd300676a0 msgr2=0x7fbd300b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:18.784 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.784+0000 7fbd35000640 1 -- 172.21.15.188:0/2972461989 shutdown_connections 2024-03-28T13:09:18.784 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.784+0000 7fbd35000640 1 -- 172.21.15.188:0/2972461989 wait complete. 2024-03-28T13:09:18.800 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 72b8a390-bb07-4819-8f19-aa3890ec8ce4 4 2024-03-28T13:09:18.990 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.990+0000 7f0d1fe00640 1 Processor -- start 2024-03-28T13:09:18.990 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.990+0000 7f0d1fe00640 1 -- start start 2024-03-28T13:09:18.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.992+0000 7f0d1fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2006bc80 0x7f0d200702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.992+0000 7f0d1fe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d200cfe80 con 0x7f0d200707e0 2024-03-28T13:09:18.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.992+0000 7f0d1fe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d200d0160 con 0x7f0d20147c80 2024-03-28T13:09:18.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.992+0000 7f0d1fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0d20070c70 con 0x7f0d2006bc80 2024-03-28T13:09:18.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.992+0000 7f0d17e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2006bc80 0x7f0d200702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.992+0000 7f0d1ea00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f0d20147c80 0x7f0d2006b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:53614/0 (socket says 172.21.15.188:53614) 2024-03-28T13:09:18.993 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.992+0000 7f0d1ea00640 1 -- 172.21.15.188:0/3737650676 learned_addr learned my addr 172.21.15.188:0/3737650676 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-03-28T13:09:18.993 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.993+0000 7f0d17e00640 1 -- 172.21.15.188:0/3737650676 >> v1:172.21.15.188:6789/0 conn(0x7f0d20147c80 legacy=0x7f0d2006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.993 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.993+0000 7f0d17e00640 1 -- 172.21.15.188:0/3737650676 >> v1:172.21.15.19:6789/0 conn(0x7f0d200707e0 legacy=0x7f0d200724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:18.993 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.993+0000 7f0d1e000640 1 -- 172.21.15.188:0/3737650676 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 352273550 0 0) 0x7f0d200d0160 con 0x7f0d20147c80 2024-03-28T13:09:18.993 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.993+0000 7f0d17e00640 1 -- 172.21.15.188:0/3737650676 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d200d0440 con 0x7f0d2006bc80 2024-03-28T13:09:18.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.994+0000 7f0d17e00640 1 --2- 172.21.15.188:0/3737650676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2006bc80 0x7f0d200702a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d080097d0 tx=0x7f0d0802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49aa3787e59ff779 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:18.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.994+0000 7f0d1e000640 1 -- 172.21.15.188:0/3737650676 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08035020 con 0x7f0d2006bc80 2024-03-28T13:09:18.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.994+0000 7f0d1e000640 1 -- 172.21.15.188:0/3737650676 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d08031800 con 0x7f0d2006bc80 2024-03-28T13:09:18.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.994+0000 7f0d1e000640 1 -- 172.21.15.188:0/3737650676 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08031bd0 con 0x7f0d2006bc80 2024-03-28T13:09:18.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.995+0000 7f0d1fe00640 1 -- 172.21.15.188:0/3737650676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2006bc80 msgr2=0x7f0d200702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.995+0000 7f0d1fe00640 1 --2- 172.21.15.188:0/3737650676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2006bc80 0x7f0d200702a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0d080097d0 tx=0x7f0d0802b8a0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.995+0000 7f0d1fe00640 1 -- 172.21.15.188:0/3737650676 shutdown_connections 2024-03-28T13:09:18.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.995+0000 7f0d1fe00640 1 --2- 172.21.15.188:0/3737650676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d2006bc80 0x7f0d200702a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.995+0000 7f0d1fe00640 1 -- 172.21.15.188:0/3737650676 >> 172.21.15.188:0/3737650676 conn(0x7f0d2006e730 msgr2=0x7f0d2006eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:18.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.995+0000 7f0d1fe00640 1 -- 172.21.15.188:0/3737650676 shutdown_connections 2024-03-28T13:09:18.995 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.995+0000 7f0d1fe00640 1 -- 172.21.15.188:0/3737650676 wait complete. 2024-03-28T13:09:18.996 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.997+0000 7f0d1fe00640 1 Processor -- start 2024-03-28T13:09:18.996 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.997+0000 7f0d1fe00640 1 -- start start 2024-03-28T13:09:18.998 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.997+0000 7f0d1fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d2006bc80 0x7f0d201e7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.998 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d1fe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d200707e0 0x7f0d200b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.998 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d1fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d20147c80 0x7f0d201f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:18.998 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d1fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d200d0a90 con 0x7f0d2006bc80 2024-03-28T13:09:18.998 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d1fe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0d200d0f20 con 0x7f0d200707e0 2024-03-28T13:09:18.998 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d1fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0d200cfb50 con 0x7f0d20147c80 2024-03-28T13:09:18.998 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d1f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d20147c80 0x7f0d201f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.998 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d17e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d200707e0 0x7f0d200b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.998 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d17e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d200707e0 0x7f0d200b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53934/0 (socket says 172.21.15.188:53934) 2024-03-28T13:09:18.998 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d1f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d20147c80 0x7f0d201f7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:41152/0 (socket says 172.21.15.188:41152) 2024-03-28T13:09:18.998 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d17e00640 1 -- 172.21.15.188:0/2152265009 learned_addr learned my addr 172.21.15.188:0/2152265009 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-28T13:09:18.998 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d1ea00640 1 --2- 172.21.15.188:0/2152265009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d2006bc80 0x7f0d201e7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:18.999 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d17e00640 1 -- 172.21.15.188:0/2152265009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d20147c80 msgr2=0x7f0d201f7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.999 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d17e00640 1 --2- 172.21.15.188:0/2152265009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d20147c80 0x7f0d201f7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.999 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d17e00640 1 -- 172.21.15.188:0/2152265009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d2006bc80 msgr2=0x7f0d201e7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:18.999 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d17e00640 1 --2- 172.21.15.188:0/2152265009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d2006bc80 0x7f0d201e7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:18.999 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.998+0000 7f0d17e00640 1 -- 172.21.15.188:0/2152265009 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d200adac0 con 0x7f0d200707e0 2024-03-28T13:09:18.999 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.999+0000 7f0d17e00640 1 --2- 172.21.15.188:0/2152265009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d200707e0 0x7f0d200b03d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009720 tx=0x7f0d080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:19.000 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.999+0000 7f0d16a00640 1 -- 172.21.15.188:0/2152265009 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0803ad60 con 0x7f0d200707e0 2024-03-28T13:09:19.000 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.999+0000 7f0d16a00640 1 -- 172.21.15.188:0/2152265009 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d080443f0 con 0x7f0d200707e0 2024-03-28T13:09:19.000 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.999+0000 7f0d16a00640 1 -- 172.21.15.188:0/2152265009 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d080447e0 con 0x7f0d200707e0 2024-03-28T13:09:19.000 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.999+0000 7f0d1fe00640 1 -- 172.21.15.188:0/2152265009 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d200b0910 con 0x7f0d200707e0 2024-03-28T13:09:19.000 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.999+0000 7f0d1fe00640 1 -- 172.21.15.188:0/2152265009 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d200add30 con 0x7f0d200707e0 2024-03-28T13:09:19.001 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.999+0000 7f0d16a00640 1 -- 172.21.15.188:0/2152265009 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0d0804b060 con 0x7f0d200707e0 2024-03-28T13:09:19.001 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:18.999+0000 7f0d16a00640 1 -- 172.21.15.188:0/2152265009 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f0d08044d10 con 0x7f0d200707e0 2024-03-28T13:09:19.001 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.000+0000 7f0d1fe00640 1 -- 172.21.15.188:0/2152265009 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d20075940 con 0x7f0d200707e0 2024-03-28T13:09:19.002 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.002+0000 7f0d16a00640 1 -- 172.21.15.188:0/2152265009 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0d08063dd0 con 0x7f0d200707e0 2024-03-28T13:09:19.080 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.081+0000 7f0d1fe00640 1 -- 172.21.15.188:0/2152265009 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "72b8a390-bb07-4819-8f19-aa3890ec8ce4", "id": 4} v 0) -- 0x7f0d201474a0 con 0x7f0d200707e0 2024-03-28T13:09:19.088 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.088+0000 7f0d16a00640 1 -- 172.21.15.188:0/2152265009 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "72b8a390-bb07-4819-8f19-aa3890ec8ce4", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f0d08052030 con 0x7f0d200707e0 2024-03-28T13:09:19.088 INFO:teuthology.orchestra.run.smithi188.stdout:4 2024-03-28T13:09:19.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.090+0000 7f0d1fe00640 1 -- 172.21.15.188:0/2152265009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d200707e0 msgr2=0x7f0d200b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:19.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.090+0000 7f0d1fe00640 1 --2- 172.21.15.188:0/2152265009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d200707e0 0x7f0d200b03d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009720 tx=0x7f0d080046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.089 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.090+0000 7f0d1fe00640 1 -- 172.21.15.188:0/2152265009 shutdown_connections 2024-03-28T13:09:19.090 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.090+0000 7f0d1fe00640 1 --2- 172.21.15.188:0/2152265009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d20147c80 0x7f0d201f7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.090 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.090+0000 7f0d1fe00640 1 --2- 172.21.15.188:0/2152265009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d200707e0 0x7f0d200b03d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.090 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.090+0000 7f0d1fe00640 1 --2- 172.21.15.188:0/2152265009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d2006bc80 0x7f0d201e7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.090 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.090+0000 7f0d1fe00640 1 -- 172.21.15.188:0/2152265009 >> 172.21.15.188:0/2152265009 conn(0x7f0d2006e730 msgr2=0x7f0d20161000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:19.090 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.090+0000 7f0d1fe00640 1 -- 172.21.15.188:0/2152265009 shutdown_connections 2024-03-28T13:09:19.090 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.090+0000 7f0d1fe00640 1 -- 172.21.15.188:0/2152265009 wait complete. 2024-03-28T13:09:19.105 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new bd4eddad-6788-4059-b203-521c62cdf1be 5 2024-03-28T13:09:19.229 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:19.230+0000 7fd85a9b8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T13:09:19.256 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:19.255+0000 7fb0e2673040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-28T13:09:19.296 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.297+0000 7fb6cca00640 1 Processor -- start 2024-03-28T13:09:19.296 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.297+0000 7fb6cca00640 1 -- start start 2024-03-28T13:09:19.297 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.298+0000 7fb6cca00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80cec60 0x7fb6c8162da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:19.298 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.298+0000 7fb6cca00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb6c80cf040 0x7fb6c8166a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:19.298 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.298+0000 7fb6cca00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6c8163dd0 con 0x7fb6c80d1490 2024-03-28T13:09:19.298 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.298+0000 7fb6cca00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb6c80d4010 con 0x7fb6c80cf040 2024-03-28T13:09:19.298 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.298+0000 7fb6cca00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c8166590 con 0x7fb6c80cec60 2024-03-28T13:09:19.298 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.298+0000 7fb6c7200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb6c80cf040 0x7fb6c8166a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:19.298 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.298+0000 7fb6c6800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80cec60 0x7fb6c8162da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:19.298 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.298+0000 7fb6c7200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb6c80cf040 0x7fb6c8166a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53940/0 (socket says 172.21.15.188:53940) 2024-03-28T13:09:19.298 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.298+0000 7fb6c7200640 1 -- 172.21.15.188:0/514922303 learned_addr learned my addr 172.21.15.188:0/514922303 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-28T13:09:19.299 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.299+0000 7fb6c7200640 1 -- 172.21.15.188:0/514922303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80cec60 msgr2=0x7fb6c8162da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:19.299 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.299+0000 7fb6c7200640 1 --2- 172.21.15.188:0/514922303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80cec60 0x7fb6c8162da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.299 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.299+0000 7fb6c7200640 1 -- 172.21.15.188:0/514922303 >> v1:172.21.15.19:6789/0 conn(0x7fb6c80d1490 legacy=0x7fb6c80d2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:19.299 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.299+0000 7fb6c7200640 1 -- 172.21.15.188:0/514922303 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c8163530 con 0x7fb6c80cf040 2024-03-28T13:09:19.299 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.300+0000 7fb6c7200640 1 --2- 172.21.15.188:0/514922303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6c80cf040 0x7fb6c8166a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009e50 tx=0x7fb6c002be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9c3467f9f88acbd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:19.299 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.300+0000 7fb6c5e00640 1 -- 172.21.15.188:0/514922303 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c002c9e0 con 0x7fb6c80cf040 2024-03-28T13:09:19.299 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.300+0000 7fb6c5e00640 1 -- 172.21.15.188:0/514922303 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6c002cb80 con 0x7fb6c80cf040 2024-03-28T13:09:19.299 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.300+0000 7fb6c5e00640 1 -- 172.21.15.188:0/514922303 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c00343f0 con 0x7fb6c80cf040 2024-03-28T13:09:19.300 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.300+0000 7fb6cca00640 1 -- 172.21.15.188:0/514922303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6c80cf040 msgr2=0x7fb6c8166a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:19.300 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.300+0000 7fb6cca00640 1 --2- 172.21.15.188:0/514922303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6c80cf040 0x7fb6c8166a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009e50 tx=0x7fb6c002be40 comp rx=0 tx=0).stop 2024-03-28T13:09:19.301 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.301+0000 7fb6cca00640 1 -- 172.21.15.188:0/514922303 shutdown_connections 2024-03-28T13:09:19.301 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.301+0000 7fb6cca00640 1 --2- 172.21.15.188:0/514922303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6c80cf040 0x7fb6c8166a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.301 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.301+0000 7fb6cca00640 1 --2- 172.21.15.188:0/514922303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80cec60 0x7fb6c8162da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.301 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.301+0000 7fb6cca00640 1 -- 172.21.15.188:0/514922303 >> 172.21.15.188:0/514922303 conn(0x7fb6c80678b0 msgr2=0x7fb6c80d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:19.301 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.301+0000 7fb6cca00640 1 -- 172.21.15.188:0/514922303 shutdown_connections 2024-03-28T13:09:19.301 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.301+0000 7fb6cca00640 1 -- 172.21.15.188:0/514922303 wait complete. 2024-03-28T13:09:19.301 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.302+0000 7fb6cca00640 1 Processor -- start 2024-03-28T13:09:19.302 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.302+0000 7fb6cca00640 1 -- start start 2024-03-28T13:09:19.304 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.303+0000 7fb6cca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c80cec60 0x7fb6c815e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:19.304 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.303+0000 7fb6cca00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6c80cf040 0x7fb6c81f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:19.304 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.303+0000 7fb6cca00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80d1490 0x7fb6c8158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:19.305 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.303+0000 7fb6cca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c8163af0 con 0x7fb6c80cec60 2024-03-28T13:09:19.305 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.303+0000 7fb6cca00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c81632e0 con 0x7fb6c80cf040 2024-03-28T13:09:19.305 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.303+0000 7fb6cca00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c80d2750 con 0x7fb6c80d1490 2024-03-28T13:09:19.305 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.304+0000 7fb6bfe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6c80cf040 0x7fb6c81f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:19.305 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.304+0000 7fb6c7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80d1490 0x7fb6c8158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:19.305 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.304+0000 7fb6bfe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6c80cf040 0x7fb6c81f8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53946/0 (socket says 172.21.15.188:53946) 2024-03-28T13:09:19.305 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.304+0000 7fb6bfe00640 1 -- 172.21.15.188:0/2426853226 learned_addr learned my addr 172.21.15.188:0/2426853226 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-28T13:09:19.305 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.304+0000 7fb6c7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80d1490 0x7fb6c8158b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:41174/0 (socket says 172.21.15.188:41174) 2024-03-28T13:09:19.305 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.304+0000 7fb6c6800640 1 --2- 172.21.15.188:0/2426853226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c80cec60 0x7fb6c815e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:19.305 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.304+0000 7fb6bfe00640 1 -- 172.21.15.188:0/2426853226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80d1490 msgr2=0x7fb6c8158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.304+0000 7fb6bfe00640 1 --2- 172.21.15.188:0/2426853226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80d1490 0x7fb6c8158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.304+0000 7fb6bfe00640 1 -- 172.21.15.188:0/2426853226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c80cec60 msgr2=0x7fb6c815e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.304+0000 7fb6bfe00640 1 --2- 172.21.15.188:0/2426853226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c80cec60 0x7fb6c815e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.304+0000 7fb6bfe00640 1 -- 172.21.15.188:0/2426853226 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c80cbf30 con 0x7fb6c80cf040 2024-03-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.304+0000 7fb6c6800640 1 --2- 172.21.15.188:0/2426853226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c80cec60 0x7fb6c815e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.304+0000 7fb6bfe00640 1 --2- 172.21.15.188:0/2426853226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6c80cf040 0x7fb6c81f8cd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000c450 tx=0x7fb6b000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.304+0000 7fb6bea00640 1 -- 172.21.15.188:0/2426853226 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0016020 con 0x7fb6c80cf040 2024-03-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.305+0000 7fb6bea00640 1 -- 172.21.15.188:0/2426853226 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6b0012830 con 0x7fb6c80cf040 2024-03-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.305+0000 7fb6bea00640 1 -- 172.21.15.188:0/2426853226 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0012c20 con 0x7fb6c80cf040 2024-03-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.305+0000 7fb6cca00640 1 -- 172.21.15.188:0/2426853226 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c80cc210 con 0x7fb6c80cf040 2024-03-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.305+0000 7fb6cca00640 1 -- 172.21.15.188:0/2426853226 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c81f8180 con 0x7fb6c80cf040 2024-03-28T13:09:19.306 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.305+0000 7fb6bea00640 1 -- 172.21.15.188:0/2426853226 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb6b0019020 con 0x7fb6c80cf040 2024-03-28T13:09:19.307 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.305+0000 7fb6bea00640 1 -- 172.21.15.188:0/2426853226 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fb6b001ad40 con 0x7fb6c80cf040 2024-03-28T13:09:19.307 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.306+0000 7fb6cca00640 1 -- 172.21.15.188:0/2426853226 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c8075940 con 0x7fb6c80cf040 2024-03-28T13:09:19.307 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.308+0000 7fb6bea00640 1 -- 172.21.15.188:0/2426853226 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb6b0012dc0 con 0x7fb6c80cf040 2024-03-28T13:09:19.386 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.387+0000 7fb6cca00640 1 -- 172.21.15.188:0/2426853226 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bd4eddad-6788-4059-b203-521c62cdf1be", "id": 5} v 0) -- 0x7fb6c80d1e20 con 0x7fb6c80cf040 2024-03-28T13:09:19.393 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.394+0000 7fb6bea00640 1 -- 172.21.15.188:0/2426853226 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bd4eddad-6788-4059-b203-521c62cdf1be", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fb6b001a3b0 con 0x7fb6c80cf040 2024-03-28T13:09:19.394 INFO:teuthology.orchestra.run.smithi188.stdout:5 2024-03-28T13:09:19.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.395+0000 7fb6cca00640 1 -- 172.21.15.188:0/2426853226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6c80cf040 msgr2=0x7fb6c81f8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:19.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.395+0000 7fb6cca00640 1 --2- 172.21.15.188:0/2426853226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6c80cf040 0x7fb6c81f8cd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb6b000c450 tx=0x7fb6b000c910 comp rx=0 tx=0).stop 2024-03-28T13:09:19.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.395+0000 7fb6cca00640 1 -- 172.21.15.188:0/2426853226 shutdown_connections 2024-03-28T13:09:19.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.395+0000 7fb6cca00640 1 --2- 172.21.15.188:0/2426853226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6c80d1490 0x7fb6c8158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.395+0000 7fb6cca00640 1 --2- 172.21.15.188:0/2426853226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6c80cf040 0x7fb6c81f8cd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.395+0000 7fb6cca00640 1 --2- 172.21.15.188:0/2426853226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb6c80cec60 0x7fb6c815e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.395 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.395+0000 7fb6cca00640 1 -- 172.21.15.188:0/2426853226 >> 172.21.15.188:0/2426853226 conn(0x7fb6c80678b0 msgr2=0x7fb6c8160a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:19.396 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:19.396+0000 7fd85a9b8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T13:09:19.396 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.396+0000 7fb6cca00640 1 -- 172.21.15.188:0/2426853226 shutdown_connections 2024-03-28T13:09:19.396 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.396+0000 7fb6cca00640 1 -- 172.21.15.188:0/2426853226 wait complete. 2024-03-28T13:09:19.410 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 9b3837e6-8631-4178-94d2-6d76f3e0c50a 6 2024-03-28T13:09:19.428 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:19.428+0000 7fb0e2673040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-28T13:09:19.547 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:19.547+0000 7fd85a9b8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T13:09:19.580 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.581+0000 7f96bac00640 1 Processor -- start 2024-03-28T13:09:19.581 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.581+0000 7f96bac00640 1 -- start start 2024-03-28T13:09:19.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.582+0000 7f96bac00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f96b415e860 0x7f96b406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:19.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.582+0000 7f96bac00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f96b406da30 0x7f96b40d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:19.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.582+0000 7f96bac00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96b406dfa0 con 0x7f96b40d4910 2024-03-28T13:09:19.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.582+0000 7f96bac00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f96b40cf220 con 0x7f96b415e860 2024-03-28T13:09:19.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.582+0000 7f96bac00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f96b40d4cf0 con 0x7f96b406da30 2024-03-28T13:09:19.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.583+0000 7f96b3400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f96b406da30 0x7f96b40d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:19.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.583+0000 7f96b3400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f96b406da30 0x7f96b40d43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53962/0 (socket says 172.21.15.188:53962) 2024-03-28T13:09:19.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.583+0000 7f96b3400640 1 -- 172.21.15.188:0/1356139066 learned_addr learned my addr 172.21.15.188:0/1356139066 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-28T13:09:19.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.583+0000 7f96b3e00640 1 --2- 172.21.15.188:0/1356139066 >> v2:172.21.15.19:3300/0 conn(0x7f96b415e860 0x7f96b406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:19.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.583+0000 7f96b3400640 1 -- 172.21.15.188:0/1356139066 >> v2:172.21.15.19:3300/0 conn(0x7f96b415e860 msgr2=0x7f96b406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:19.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.583+0000 7f96b3400640 1 --2- 172.21.15.188:0/1356139066 >> v2:172.21.15.19:3300/0 conn(0x7f96b415e860 0x7f96b406d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.583 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.583+0000 7f96b3400640 1 -- 172.21.15.188:0/1356139066 >> v1:172.21.15.19:6789/0 conn(0x7f96b40d4910 legacy=0x7f96b40d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:19.584 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.583+0000 7f96b3400640 1 -- 172.21.15.188:0/1356139066 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b406e240 con 0x7f96b406da30 2024-03-28T13:09:19.584 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.583+0000 7f96b3e00640 1 --2- 172.21.15.188:0/1356139066 >> v2:172.21.15.19:3300/0 conn(0x7f96b415e860 0x7f96b406d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:19.584 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:19.584+0000 7fb0e2673040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-28T13:09:19.585 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.584+0000 7f96b3400640 1 --2- 172.21.15.188:0/1356139066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96b406da30 0x7f96b40d43d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f96b4162160 tx=0x7f96a002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b07bb0decf45ffae server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:19.585 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.585+0000 7f96b2a00640 1 -- 172.21.15.188:0/1356139066 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0034070 con 0x7f96b406da30 2024-03-28T13:09:19.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.585+0000 7f96b2a00640 1 -- 172.21.15.188:0/1356139066 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a0030860 con 0x7f96b406da30 2024-03-28T13:09:19.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.585+0000 7f96b2a00640 1 -- 172.21.15.188:0/1356139066 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0030c30 con 0x7f96b406da30 2024-03-28T13:09:19.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.585+0000 7f96bac00640 1 -- 172.21.15.188:0/1356139066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96b406da30 msgr2=0x7f96b40d43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:19.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.585+0000 7f96bac00640 1 --2- 172.21.15.188:0/1356139066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96b406da30 0x7f96b40d43d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f96b4162160 tx=0x7f96a002b9a0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.586+0000 7f96bac00640 1 -- 172.21.15.188:0/1356139066 shutdown_connections 2024-03-28T13:09:19.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.586+0000 7f96bac00640 1 --2- 172.21.15.188:0/1356139066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96b406da30 0x7f96b40d43d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.586+0000 7f96bac00640 1 --2- 172.21.15.188:0/1356139066 >> v2:172.21.15.19:3300/0 conn(0x7f96b415e860 0x7f96b406d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.586+0000 7f96bac00640 1 -- 172.21.15.188:0/1356139066 >> 172.21.15.188:0/1356139066 conn(0x7f96b409d7d0 msgr2=0x7f96b408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:19.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.586+0000 7f96bac00640 1 -- 172.21.15.188:0/1356139066 shutdown_connections 2024-03-28T13:09:19.586 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.586+0000 7f96bac00640 1 -- 172.21.15.188:0/1356139066 wait complete. 2024-03-28T13:09:19.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.587+0000 7f96bac00640 1 Processor -- start 2024-03-28T13:09:19.587 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.587+0000 7f96bac00640 1 -- start start 2024-03-28T13:09:19.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.588+0000 7f96bac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b406da30 0x7f96b4152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:19.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.588+0000 7f96bac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b40d4910 0x7f96b4155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:19.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.588+0000 7f96bac00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96b415e860 0x7f96b4156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:19.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.588+0000 7f96bac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f96b40cedb0 con 0x7f96b406da30 2024-03-28T13:09:19.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.588+0000 7f96bac00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f96b40d2c50 con 0x7f96b415e860 2024-03-28T13:09:19.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.588+0000 7f96bac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f96b40cef30 con 0x7f96b40d4910 2024-03-28T13:09:19.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.589+0000 7f96b3400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b40d4910 0x7f96b4155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:19.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.589+0000 7f96b3400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b40d4910 0x7f96b4155e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:41184/0 (socket says 172.21.15.188:41184) 2024-03-28T13:09:19.590 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.589+0000 7f96b3400640 1 -- 172.21.15.188:0/3520553490 learned_addr learned my addr 172.21.15.188:0/3520553490 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-28T13:09:19.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.589+0000 7f96b8c00640 1 --2- 172.21.15.188:0/3520553490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96b415e860 0x7f96b4156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:19.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.589+0000 7f96b3e00640 1 --2- 172.21.15.188:0/3520553490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b406da30 0x7f96b4152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:19.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.589+0000 7f96b3400640 1 -- 172.21.15.188:0/3520553490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96b415e860 msgr2=0x7f96b4156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:19.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.589+0000 7f96b3400640 1 --2- 172.21.15.188:0/3520553490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96b415e860 0x7f96b4156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.589+0000 7f96b3400640 1 -- 172.21.15.188:0/3520553490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b406da30 msgr2=0x7f96b4152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:19.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.589+0000 7f96b3400640 1 --2- 172.21.15.188:0/3520553490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b406da30 0x7f96b4152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.589+0000 7f96b3400640 1 -- 172.21.15.188:0/3520553490 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b41532c0 con 0x7f96b40d4910 2024-03-28T13:09:19.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.589+0000 7f96b8c00640 1 --2- 172.21.15.188:0/3520553490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96b415e860 0x7f96b4156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:09:19.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.590+0000 7f96b3400640 1 --2- 172.21.15.188:0/3520553490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b40d4910 0x7f96b4155e40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96a0000c00 tx=0x7f96a0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:19.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.590+0000 7f96abe00640 1 -- 172.21.15.188:0/3520553490 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0048070 con 0x7f96b40d4910 2024-03-28T13:09:19.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.590+0000 7f96abe00640 1 -- 172.21.15.188:0/3520553490 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a0030660 con 0x7f96b40d4910 2024-03-28T13:09:19.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.590+0000 7f96abe00640 1 -- 172.21.15.188:0/3520553490 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0034040 con 0x7f96b40d4910 2024-03-28T13:09:19.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.590+0000 7f96bac00640 1 -- 172.21.15.188:0/3520553490 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b40c8010 con 0x7f96b40d4910 2024-03-28T13:09:19.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.590+0000 7f96bac00640 1 -- 172.21.15.188:0/3520553490 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b40c8590 con 0x7f96b40d4910 2024-03-28T13:09:19.591 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.590+0000 7f96abe00640 1 -- 172.21.15.188:0/3520553490 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f96a004b070 con 0x7f96b40d4910 2024-03-28T13:09:19.592 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.590+0000 7f96abe00640 1 -- 172.21.15.188:0/3520553490 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f96a0030810 con 0x7f96b40d4910 2024-03-28T13:09:19.592 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.591+0000 7f96bac00640 1 -- 172.21.15.188:0/3520553490 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b40d2610 con 0x7f96b40d4910 2024-03-28T13:09:19.593 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.594+0000 7f96abe00640 1 -- 172.21.15.188:0/3520553490 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f96b40d2610 con 0x7f96b40d4910 2024-03-28T13:09:19.604 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:19.605+0000 7fd85a9b8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T13:09:19.643 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:19.643+0000 7fb0e2673040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-28T13:09:19.673 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.673+0000 7f96bac00640 1 -- 172.21.15.188:0/3520553490 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9b3837e6-8631-4178-94d2-6d76f3e0c50a", "id": 6} v 0) -- 0x7f96b40685c0 con 0x7f96b40d4910 2024-03-28T13:09:19.682 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.682+0000 7f96abe00640 1 -- 172.21.15.188:0/3520553490 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9b3837e6-8631-4178-94d2-6d76f3e0c50a", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f96a0052020 con 0x7f96b40d4910 2024-03-28T13:09:19.682 INFO:teuthology.orchestra.run.smithi188.stdout:6 2024-03-28T13:09:19.683 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.684+0000 7f96bac00640 1 -- 172.21.15.188:0/3520553490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b40d4910 msgr2=0x7f96b4155e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:19.684 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.684+0000 7f96bac00640 1 --2- 172.21.15.188:0/3520553490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b40d4910 0x7f96b4155e40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f96a0000c00 tx=0x7f96a0004730 comp rx=0 tx=0).stop 2024-03-28T13:09:19.684 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.684+0000 7f96bac00640 1 -- 172.21.15.188:0/3520553490 shutdown_connections 2024-03-28T13:09:19.684 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.684+0000 7f96bac00640 1 --2- 172.21.15.188:0/3520553490 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96b415e860 0x7f96b4156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.684 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.684+0000 7f96bac00640 1 --2- 172.21.15.188:0/3520553490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b40d4910 0x7f96b4155e40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.684 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.684+0000 7f96bac00640 1 --2- 172.21.15.188:0/3520553490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96b406da30 0x7f96b4152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.684 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.684+0000 7f96bac00640 1 -- 172.21.15.188:0/3520553490 >> 172.21.15.188:0/3520553490 conn(0x7f96b409d7d0 msgr2=0x7f96b40d1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:19.684 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.684+0000 7f96bac00640 1 -- 172.21.15.188:0/3520553490 shutdown_connections 2024-03-28T13:09:19.684 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.684+0000 7f96bac00640 1 -- 172.21.15.188:0/3520553490 wait complete. 2024-03-28T13:09:19.700 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 2dbbb7c5-ce51-4d70-a98b-954428334789 7 2024-03-28T13:09:19.739 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:19.739+0000 7fd85a9b8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T13:09:19.781 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:19.780+0000 7fb0e2673040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-28T13:09:19.856 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:19.856+0000 7fd85a9b8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T13:09:19.890 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.891+0000 7f4ee2800640 1 Processor -- start 2024-03-28T13:09:19.890 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.891+0000 7f4ee2800640 1 -- start start 2024-03-28T13:09:19.892 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.892+0000 7f4ee2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4edc15e460 0x7f4edc0b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:19.892 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.892+0000 7f4ee2800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4edc0cef20 0x7f4edc0cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:19.892 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.892+0000 7f4ee2800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4edc0d2da0 con 0x7f4edc0d3e40 2024-03-28T13:09:19.892 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.892+0000 7f4ee2800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4edc0cf8d0 con 0x7f4edc15e460 2024-03-28T13:09:19.892 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.892+0000 7f4ee2800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4edc0b8320 con 0x7f4edc0cef20 2024-03-28T13:09:19.892 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.893+0000 7f4ee1e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4edc0cef20 0x7f4edc0cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:19.892 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.893+0000 7f4ee1e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4edc0cef20 0x7f4edc0cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:41196/0 (socket says 172.21.15.188:41196) 2024-03-28T13:09:19.893 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.893+0000 7f4ee1e00640 1 -- 172.21.15.188:0/1695326883 learned_addr learned my addr 172.21.15.188:0/1695326883 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-28T13:09:19.893 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.893+0000 7f4ee1400640 1 --2- 172.21.15.188:0/1695326883 >> v2:172.21.15.19:3300/0 conn(0x7f4edc15e460 0x7f4edc0b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:19.893 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.893+0000 7f4ee1e00640 1 -- 172.21.15.188:0/1695326883 >> v2:172.21.15.19:3300/0 conn(0x7f4edc15e460 msgr2=0x7f4edc0b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:19.893 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.893+0000 7f4ee1e00640 1 --2- 172.21.15.188:0/1695326883 >> v2:172.21.15.19:3300/0 conn(0x7f4edc15e460 0x7f4edc0b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.893 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.893+0000 7f4ee1e00640 1 -- 172.21.15.188:0/1695326883 >> v1:172.21.15.19:6789/0 conn(0x7f4edc0d3e40 legacy=0x7f4edc0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:19.893 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.893+0000 7f4ee1e00640 1 -- 172.21.15.188:0/1695326883 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4edc0d3080 con 0x7f4edc0cef20 2024-03-28T13:09:19.895 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.894+0000 7f4ee1e00640 1 --2- 172.21.15.188:0/1695326883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4edc0cef20 0x7f4edc0cf390 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009e50 tx=0x7f4ed402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd68c39bfc64c248 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:19.895 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.894+0000 7f4edbe00640 1 -- 172.21.15.188:0/1695326883 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed402cb00 con 0x7f4edc0cef20 2024-03-28T13:09:19.895 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.895+0000 7f4edbe00640 1 -- 172.21.15.188:0/1695326883 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ed402cca0 con 0x7f4edc0cef20 2024-03-28T13:09:19.895 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.895+0000 7f4edbe00640 1 -- 172.21.15.188:0/1695326883 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4034610 con 0x7f4edc0cef20 2024-03-28T13:09:19.895 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.895+0000 7f4ee2800640 1 -- 172.21.15.188:0/1695326883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4edc0cef20 msgr2=0x7f4edc0cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:19.895 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.895+0000 7f4ee2800640 1 --2- 172.21.15.188:0/1695326883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4edc0cef20 0x7f4edc0cf390 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009e50 tx=0x7f4ed402bf60 comp rx=0 tx=0).stop 2024-03-28T13:09:19.895 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.895+0000 7f4ee2800640 1 -- 172.21.15.188:0/1695326883 shutdown_connections 2024-03-28T13:09:19.895 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.895+0000 7f4ee2800640 1 --2- 172.21.15.188:0/1695326883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4edc0cef20 0x7f4edc0cf390 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.895 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.895+0000 7f4ee2800640 1 --2- 172.21.15.188:0/1695326883 >> v2:172.21.15.19:3300/0 conn(0x7f4edc15e460 0x7f4edc0b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.895 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.895+0000 7f4ee2800640 1 -- 172.21.15.188:0/1695326883 >> 172.21.15.188:0/1695326883 conn(0x7f4edc0678b0 msgr2=0x7f4edc0d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:19.895 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.895+0000 7f4ee2800640 1 -- 172.21.15.188:0/1695326883 shutdown_connections 2024-03-28T13:09:19.895 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.895+0000 7f4ee2800640 1 -- 172.21.15.188:0/1695326883 wait complete. 2024-03-28T13:09:19.896 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.897+0000 7f4ee2800640 1 Processor -- start 2024-03-28T13:09:19.896 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.897+0000 7f4ee2800640 1 -- start start 2024-03-28T13:09:19.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.898+0000 7f4ee2800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4edc0cef20 0x7f4edc15a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:19.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.898+0000 7f4ee2800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4edc0d3e40 0x7f4edc0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:19.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.898+0000 7f4ee2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4edc15e460 0x7f4edc0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:19.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.898+0000 7f4ee2800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc0d3920 con 0x7f4edc15e460 2024-03-28T13:09:19.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.898+0000 7f4ee2800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc0d3640 con 0x7f4edc0cef20 2024-03-28T13:09:19.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.898+0000 7f4ee2800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4edc0b7310 con 0x7f4edc0d3e40 2024-03-28T13:09:19.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.898+0000 7f4ee0a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4edc0d3e40 0x7f4edc0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:19.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.898+0000 7f4ee1400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4edc0cef20 0x7f4edc15a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:19.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.898+0000 7f4ee0a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4edc0d3e40 0x7f4edc0cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:41206/0 (socket says 172.21.15.188:41206) 2024-03-28T13:09:19.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.898+0000 7f4ee0a00640 1 -- 172.21.15.188:0/833868839 learned_addr learned my addr 172.21.15.188:0/833868839 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-03-28T13:09:19.898 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.898+0000 7f4ee1e00640 1 --2- 172.21.15.188:0/833868839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4edc15e460 0x7f4edc0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:19.899 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.899+0000 7f4ee0a00640 1 -- 172.21.15.188:0/833868839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4edc0cef20 msgr2=0x7f4edc15a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:19.899 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.899+0000 7f4ee0a00640 1 --2- 172.21.15.188:0/833868839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4edc0cef20 0x7f4edc15a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.899 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.899+0000 7f4ee0a00640 1 -- 172.21.15.188:0/833868839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4edc15e460 msgr2=0x7f4edc0c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:19.899 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.899+0000 7f4ee0a00640 1 --2- 172.21.15.188:0/833868839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4edc15e460 0x7f4edc0c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.899 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.899+0000 7f4ee0a00640 1 -- 172.21.15.188:0/833868839 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4edc0cc2f0 con 0x7f4edc0d3e40 2024-03-28T13:09:19.900 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.899+0000 7f4ee0a00640 1 --2- 172.21.15.188:0/833868839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4edc0d3e40 0x7f4edc0cc790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800c3f0 tx=0x7f4ec800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:19.900 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.899+0000 7f4ed9600640 1 -- 172.21.15.188:0/833868839 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80052d0 con 0x7f4edc0d3e40 2024-03-28T13:09:19.900 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.899+0000 7f4ed9600640 1 -- 172.21.15.188:0/833868839 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec8005490 con 0x7f4edc0d3e40 2024-03-28T13:09:19.900 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.899+0000 7f4ed9600640 1 -- 172.21.15.188:0/833868839 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8006a60 con 0x7f4edc0d3e40 2024-03-28T13:09:19.900 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.899+0000 7f4ee2800640 1 -- 172.21.15.188:0/833868839 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4edc0aaef0 con 0x7f4edc0d3e40 2024-03-28T13:09:19.900 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.899+0000 7f4ee2800640 1 -- 172.21.15.188:0/833868839 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4edc0ab4b0 con 0x7f4edc0d3e40 2024-03-28T13:09:19.900 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.900+0000 7f4ed9600640 1 -- 172.21.15.188:0/833868839 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4ec8003640 con 0x7f4edc0d3e40 2024-03-28T13:09:19.900 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.900+0000 7f4ed9600640 1 -- 172.21.15.188:0/833868839 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f4ec8011590 con 0x7f4edc0d3e40 2024-03-28T13:09:19.901 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.901+0000 7f4ee2800640 1 -- 172.21.15.188:0/833868839 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4edc075940 con 0x7f4edc0d3e40 2024-03-28T13:09:19.901 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:19.900+0000 7fb0e2673040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-28T13:09:19.902 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.903+0000 7f4ed9600640 1 -- 172.21.15.188:0/833868839 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4ec8011bd0 con 0x7f4edc0d3e40 2024-03-28T13:09:19.980 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.981+0000 7f4ee2800640 1 -- 172.21.15.188:0/833868839 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2dbbb7c5-ce51-4d70-a98b-954428334789", "id": 7} v 0) -- 0x7f4edc06b570 con 0x7f4edc0d3e40 2024-03-28T13:09:19.992 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.992+0000 7f4ed9600640 1 -- 172.21.15.188:0/833868839 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2dbbb7c5-ce51-4d70-a98b-954428334789", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f4ec8011db0 con 0x7f4edc0d3e40 2024-03-28T13:09:19.992 INFO:teuthology.orchestra.run.smithi188.stdout:7 2024-03-28T13:09:19.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.994+0000 7f4ee2800640 1 -- 172.21.15.188:0/833868839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4edc0d3e40 msgr2=0x7f4edc0cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:19.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.994+0000 7f4ee2800640 1 --2- 172.21.15.188:0/833868839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4edc0d3e40 0x7f4edc0cc790 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800c3f0 tx=0x7f4ec800c8b0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.994+0000 7f4ee2800640 1 -- 172.21.15.188:0/833868839 shutdown_connections 2024-03-28T13:09:19.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.994+0000 7f4ee2800640 1 --2- 172.21.15.188:0/833868839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4edc15e460 0x7f4edc0c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.994+0000 7f4ee2800640 1 --2- 172.21.15.188:0/833868839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4edc0d3e40 0x7f4edc0cc790 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.994+0000 7f4ee2800640 1 --2- 172.21.15.188:0/833868839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4edc0cef20 0x7f4edc15a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:19.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.994+0000 7f4ee2800640 1 -- 172.21.15.188:0/833868839 >> 172.21.15.188:0/833868839 conn(0x7f4edc0678b0 msgr2=0x7f4edc0b7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:19.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.994+0000 7f4ee2800640 1 -- 172.21.15.188:0/833868839 shutdown_connections 2024-03-28T13:09:19.994 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:09:19.994+0000 7f4ee2800640 1 -- 172.21.15.188:0/833868839 wait complete. 2024-03-28T13:09:20.010 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-28T13:09:20.010 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-28T13:09:20.017 INFO:tasks.ceph.osd.0:Started 2024-03-28T13:09:20.017 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-28T13:09:20.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-28T13:09:20.024 INFO:tasks.ceph.osd.1:Started 2024-03-28T13:09:20.025 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-28T13:09:20.025 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-28T13:09:20.031 INFO:tasks.ceph.osd.2:Started 2024-03-28T13:09:20.031 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-28T13:09:20.031 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-28T13:09:20.036 INFO:tasks.ceph.osd.3:Started 2024-03-28T13:09:20.036 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-28T13:09:20.036 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-28T13:09:20.043 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:20.044+0000 7fd85a9b8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T13:09:20.047 INFO:tasks.ceph.osd.4:Started 2024-03-28T13:09:20.047 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-28T13:09:20.047 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-28T13:09:20.054 INFO:tasks.ceph.osd.5:Started 2024-03-28T13:09:20.054 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-28T13:09:20.054 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-28T13:09:20.060 INFO:tasks.ceph.osd.6:Started 2024-03-28T13:09:20.060 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-28T13:09:20.060 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-28T13:09:20.065 INFO:tasks.ceph.osd.7:Started 2024-03-28T13:09:20.065 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:09:20.104 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:20.104+0000 7fd85a9b8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T13:09:20.113 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:20.113+0000 7fb0e2673040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-28T13:09:20.202 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:20.202+0000 7fb0e2673040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-28T13:09:20.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.274+0000 7fb4c1a00640 1 Processor -- start 2024-03-28T13:09:20.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.274+0000 7fb4c1a00640 1 -- start start 2024-03-28T13:09:20.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.275+0000 7fb4c1a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb4bc0d1950 0x7fb4bc0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:20.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.275+0000 7fb4c1a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb4bc0d2d60 0x7fb4bc0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:20.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.275+0000 7fb4c1a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4bc15ebc0 con 0x7fb4bc15e7e0 2024-03-28T13:09:20.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.275+0000 7fb4c1a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb4bc0d1db0 con 0x7fb4bc0d1950 2024-03-28T13:09:20.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.275+0000 7fb4c1a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb4bc06d1c0 con 0x7fb4bc0d2d60 2024-03-28T13:09:20.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.275+0000 7fb4b3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb4bc0d1950 0x7fb4bc0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:20.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.275+0000 7fb4b3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb4bc0d1950 0x7fb4bc0d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42834/0 (socket says 172.21.15.19:42834) 2024-03-28T13:09:20.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.275+0000 7fb4b3e00640 1 -- 172.21.15.19:0/577284594 learned_addr learned my addr 172.21.15.19:0/577284594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:20.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.275+0000 7fb4baa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb4bc15e7e0 0x7fb4bc06ca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35146/0 (socket says 172.21.15.19:35146) 2024-03-28T13:09:20.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.275+0000 7fb4bb400640 1 --2- 172.21.15.19:0/577284594 >> v2:172.21.15.188:3300/0 conn(0x7fb4bc0d2d60 0x7fb4bc0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:20.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.276+0000 7fb4ba000640 1 -- 172.21.15.19:0/577284594 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3075838267 0 0) 0x7fb4bc15ebc0 con 0x7fb4bc15e7e0 2024-03-28T13:09:20.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.276+0000 7fb4b3e00640 1 -- 172.21.15.19:0/577284594 >> v2:172.21.15.188:3300/0 conn(0x7fb4bc0d2d60 msgr2=0x7fb4bc0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:20.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.276+0000 7fb4b3e00640 1 --2- 172.21.15.19:0/577284594 >> v2:172.21.15.188:3300/0 conn(0x7fb4bc0d2d60 0x7fb4bc0cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:20.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.276+0000 7fb4b3e00640 1 -- 172.21.15.19:0/577284594 >> v1:172.21.15.19:6789/0 conn(0x7fb4bc15e7e0 legacy=0x7fb4bc06ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:20.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.276+0000 7fb4b3e00640 1 -- 172.21.15.19:0/577284594 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc06d4a0 con 0x7fb4bc0d1950 2024-03-28T13:09:20.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.278+0000 7fb4b3e00640 1 --2- 172.21.15.19:0/577284594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4bc0d1950 0x7fb4bc0d2820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0d4fa0 tx=0x7fb4a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2159756e6af3e14d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:20.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.278+0000 7fb4ba000640 1 -- 172.21.15.19:0/577284594 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4035020 con 0x7fb4bc0d1950 2024-03-28T13:09:20.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.278+0000 7fb4ba000640 1 -- 172.21.15.19:0/577284594 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4a4031870 con 0x7fb4bc0d1950 2024-03-28T13:09:20.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.278+0000 7fb4ba000640 1 -- 172.21.15.19:0/577284594 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4031c40 con 0x7fb4bc0d1950 2024-03-28T13:09:20.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.279+0000 7fb4c1a00640 1 -- 172.21.15.19:0/577284594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4bc0d1950 msgr2=0x7fb4bc0d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:20.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.279+0000 7fb4c1a00640 1 --2- 172.21.15.19:0/577284594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4bc0d1950 0x7fb4bc0d2820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc0d4fa0 tx=0x7fb4a402b910 comp rx=0 tx=0).stop 2024-03-28T13:09:20.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.279+0000 7fb4c1a00640 1 -- 172.21.15.19:0/577284594 shutdown_connections 2024-03-28T13:09:20.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.279+0000 7fb4c1a00640 1 --2- 172.21.15.19:0/577284594 >> v2:172.21.15.188:3300/0 conn(0x7fb4bc0d2d60 0x7fb4bc0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:20.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.279+0000 7fb4c1a00640 1 --2- 172.21.15.19:0/577284594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4bc0d1950 0x7fb4bc0d2820 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:20.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.279+0000 7fb4c1a00640 1 -- 172.21.15.19:0/577284594 >> 172.21.15.19:0/577284594 conn(0x7fb4bc09d7c0 msgr2=0x7fb4bc06bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:20.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.279+0000 7fb4c1a00640 1 -- 172.21.15.19:0/577284594 shutdown_connections 2024-03-28T13:09:20.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.279+0000 7fb4c1a00640 1 -- 172.21.15.19:0/577284594 wait complete. 2024-03-28T13:09:20.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.280+0000 7fb4c1a00640 1 Processor -- start 2024-03-28T13:09:20.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.280+0000 7fb4c1a00640 1 -- start start 2024-03-28T13:09:20.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.281+0000 7fb4c1a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4bc0d1950 0x7fb4bc15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:20.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.281+0000 7fb4c1a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4bc0d2d60 0x7fb4bc09e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:20.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.281+0000 7fb4c1a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4bc15e7e0 0x7fb4bc0b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:20.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.281+0000 7fb4c1a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc0cefd0 con 0x7fb4bc15e7e0 2024-03-28T13:09:20.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.281+0000 7fb4c1a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc0cf600 con 0x7fb4bc0d1950 2024-03-28T13:09:20.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.281+0000 7fb4c1a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb4bc0d0060 con 0x7fb4bc0d2d60 2024-03-28T13:09:20.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.282+0000 7fb4bb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4bc15e7e0 0x7fb4bc0b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:20.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.282+0000 7fb4bb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4bc15e7e0 0x7fb4bc0b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42842/0 (socket says 172.21.15.19:42842) 2024-03-28T13:09:20.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.282+0000 7fb4bb400640 1 -- 172.21.15.19:0/2295927247 learned_addr learned my addr 172.21.15.19:0/2295927247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:20.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.282+0000 7fb4baa00640 1 --2- 172.21.15.19:0/2295927247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4bc0d1950 0x7fb4bc15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:20.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.282+0000 7fb4b3e00640 1 --2- 172.21.15.19:0/2295927247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4bc0d2d60 0x7fb4bc09e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:20.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.282+0000 7fb4bb400640 1 -- 172.21.15.19:0/2295927247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4bc0d2d60 msgr2=0x7fb4bc09e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:20.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.282+0000 7fb4bb400640 1 --2- 172.21.15.19:0/2295927247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4bc0d2d60 0x7fb4bc09e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:20.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.282+0000 7fb4bb400640 1 -- 172.21.15.19:0/2295927247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4bc0d1950 msgr2=0x7fb4bc15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:20.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.282+0000 7fb4bb400640 1 --2- 172.21.15.19:0/2295927247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4bc0d1950 0x7fb4bc15aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:20.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.282+0000 7fb4bb400640 1 -- 172.21.15.19:0/2295927247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc161840 con 0x7fb4bc15e7e0 2024-03-28T13:09:20.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.282+0000 7fb4bb400640 1 --2- 172.21.15.19:0/2295927247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4bc15e7e0 0x7fb4bc0b1020 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400efc0 tx=0x7fb4b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:20.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.283+0000 7fb4b2a00640 1 -- 172.21.15.19:0/2295927247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4018020 con 0x7fb4bc15e7e0 2024-03-28T13:09:20.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.283+0000 7fb4b2a00640 1 -- 172.21.15.19:0/2295927247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4b4014480 con 0x7fb4bc15e7e0 2024-03-28T13:09:20.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.283+0000 7fb4b2a00640 1 -- 172.21.15.19:0/2295927247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4014870 con 0x7fb4bc15e7e0 2024-03-28T13:09:20.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.283+0000 7fb4c1a00640 1 -- 172.21.15.19:0/2295927247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4bc0a17d0 con 0x7fb4bc15e7e0 2024-03-28T13:09:20.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.283+0000 7fb4c1a00640 1 -- 172.21.15.19:0/2295927247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4bc0a1d90 con 0x7fb4bc15e7e0 2024-03-28T13:09:20.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.283+0000 7fb4b2a00640 1 -- 172.21.15.19:0/2295927247 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb4b4005d10 con 0x7fb4bc15e7e0 2024-03-28T13:09:20.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.283+0000 7fb4b2a00640 1 -- 172.21.15.19:0/2295927247 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fb4b4014da0 con 0x7fb4bc15e7e0 2024-03-28T13:09:20.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.284+0000 7fb4c1a00640 1 -- 172.21.15.19:0/2295927247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4bc0685c0 con 0x7fb4bc15e7e0 2024-03-28T13:09:20.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.287+0000 7fb4b2a00640 1 -- 172.21.15.19:0/2295927247 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb4b4022030 con 0x7fb4bc15e7e0 2024-03-28T13:09:20.329 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:20.330+0000 7fd85a9b8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T13:09:20.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.353+0000 7fb4c1a00640 1 -- 172.21.15.19:0/2295927247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb4bc0d5300 con 0x7fb4bc15e7e0 2024-03-28T13:09:20.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.353+0000 7fb4b2a00640 1 -- 172.21.15.19:0/2295927247 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fb4b4010040 con 0x7fb4bc15e7e0 2024-03-28T13:09:20.354 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:20.354 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":10,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:19.983398+0000","last_up_change":"0.000000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:09:20.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.355+0000 7fb4c1a00640 1 -- 172.21.15.19:0/2295927247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4bc15e7e0 msgr2=0x7fb4bc0b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:20.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.355+0000 7fb4c1a00640 1 --2- 172.21.15.19:0/2295927247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4bc15e7e0 0x7fb4bc0b1020 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400efc0 tx=0x7fb4b400c490 comp rx=0 tx=0).stop 2024-03-28T13:09:20.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.355+0000 7fb4c1a00640 1 -- 172.21.15.19:0/2295927247 shutdown_connections 2024-03-28T13:09:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.355+0000 7fb4c1a00640 1 --2- 172.21.15.19:0/2295927247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4bc15e7e0 0x7fb4bc0b1020 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.355+0000 7fb4c1a00640 1 --2- 172.21.15.19:0/2295927247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4bc0d2d60 0x7fb4bc09e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.355+0000 7fb4c1a00640 1 --2- 172.21.15.19:0/2295927247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4bc0d1950 0x7fb4bc15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.355+0000 7fb4c1a00640 1 -- 172.21.15.19:0/2295927247 >> 172.21.15.19:0/2295927247 conn(0x7fb4bc09d7c0 msgr2=0x7fb4bc0d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.355+0000 7fb4c1a00640 1 -- 172.21.15.19:0/2295927247 shutdown_connections 2024-03-28T13:09:20.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:20.355+0000 7fb4c1a00640 1 -- 172.21.15.19:0/2295927247 wait complete. 2024-03-28T13:09:20.371 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-28T13:09:20.372 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-28T13:09:20.402 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:20.402+0000 7fb0e2673040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-28T13:09:20.727 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-03-28T13:09:20.727+0000 7ff231e0d640 -1 Falling back to public interface 2024-03-28T13:09:20.727 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-03-28T13:09:20.727+0000 7f9d1ea10640 -1 Falling back to public interface 2024-03-28T13:09:20.728 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-03-28T13:09:20.727+0000 7f5e9fb73640 -1 Falling back to public interface 2024-03-28T13:09:20.728 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-03-28T13:09:20.728+0000 7f9d1ce00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-03-28T13:09:20.732 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-03-28T13:09:20.733+0000 7f7390b24640 -1 Falling back to public interface 2024-03-28T13:09:20.746 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-03-28T13:09:20.747+0000 7f392d01a640 -1 Falling back to public interface 2024-03-28T13:09:20.759 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-03-28T13:09:20.759+0000 7fa837f36640 -1 Falling back to public interface 2024-03-28T13:09:20.766 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-03-28T13:09:20.767+0000 7fc817211640 -1 Falling back to public interface 2024-03-28T13:09:20.772 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-03-28T13:09:20.773+0000 7f767636f640 -1 Falling back to public interface 2024-03-28T13:09:21.178 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-28T13:09:21.343 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.343+0000 7fdaab400640 1 Processor -- start 2024-03-28T13:09:21.343 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.343+0000 7fdaab400640 1 -- start start 2024-03-28T13:09:21.345 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.344+0000 7fdaab400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdaa415e910 0x7fdaa40d2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:21.345 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.344+0000 7fdaab400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdaa40d2a40 0x7fdaa40d2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:21.345 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.344+0000 7fdaab400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa40d2780 con 0x7fdaa41622d0 2024-03-28T13:09:21.345 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.344+0000 7fdaab400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdaa406c630 con 0x7fdaa40d2a40 2024-03-28T13:09:21.345 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.344+0000 7fdaab400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa40d33e0 con 0x7fdaa415e910 2024-03-28T13:09:21.345 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.344+0000 7fdaa9400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdaa41622d0 0x7fdaa4162760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35162/0 (socket says 172.21.15.19:35162) 2024-03-28T13:09:21.345 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.344+0000 7fdaa9400640 1 -- 172.21.15.19:0/2814861584 learned_addr learned my addr 172.21.15.19:0/2814861584 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:21.345 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.345+0000 7fdaa3e00640 1 --2- 172.21.15.19:0/2814861584 >> v2:172.21.15.19:3300/0 conn(0x7fdaa40d2a40 0x7fdaa40d2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:21.345 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.345+0000 7fdaa8a00640 1 --2- 172.21.15.19:0/2814861584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdaa415e910 0x7fdaa40d2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:21.346 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.345+0000 7fdaa3400640 1 -- 172.21.15.19:0/2814861584 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4267308509 0 0) 0x7fdaa40d2780 con 0x7fdaa41622d0 2024-03-28T13:09:21.346 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.345+0000 7fdaa3400640 1 -- 172.21.15.19:0/2814861584 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda8c003520 con 0x7fdaa41622d0 2024-03-28T13:09:21.346 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.345+0000 7fdaa3e00640 1 -- 172.21.15.19:0/2814861584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdaa415e910 msgr2=0x7fdaa40d2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:21.346 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.345+0000 7fdaa3e00640 1 --2- 172.21.15.19:0/2814861584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdaa415e910 0x7fdaa40d2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:21.346 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.345+0000 7fdaa3e00640 1 -- 172.21.15.19:0/2814861584 >> v1:172.21.15.19:6789/0 conn(0x7fdaa41622d0 legacy=0x7fdaa4162760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:21.346 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.345+0000 7fdaa3e00640 1 -- 172.21.15.19:0/2814861584 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa40d0ab0 con 0x7fdaa40d2a40 2024-03-28T13:09:21.347 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.347+0000 7fdaa3e00640 1 --2- 172.21.15.19:0/2814861584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaa40d2a40 0x7fdaa40d2ea0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fda900097d0 tx=0x7fda9002b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83542af388515724 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:21.347 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.347+0000 7fdaa3400640 1 -- 172.21.15.19:0/2814861584 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda90034070 con 0x7fdaa40d2a40 2024-03-28T13:09:21.347 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.347+0000 7fdaa3400640 1 -- 172.21.15.19:0/2814861584 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda90030800 con 0x7fdaa40d2a40 2024-03-28T13:09:21.347 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.347+0000 7fdaa3400640 1 -- 172.21.15.19:0/2814861584 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda90030bb0 con 0x7fdaa40d2a40 2024-03-28T13:09:21.348 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.348+0000 7fdaab400640 1 -- 172.21.15.19:0/2814861584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaa40d2a40 msgr2=0x7fdaa40d2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:21.348 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.348+0000 7fdaab400640 1 --2- 172.21.15.19:0/2814861584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaa40d2a40 0x7fdaa40d2ea0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fda900097d0 tx=0x7fda9002b940 comp rx=0 tx=0).stop 2024-03-28T13:09:21.348 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.348+0000 7fdaab400640 1 -- 172.21.15.19:0/2814861584 shutdown_connections 2024-03-28T13:09:21.349 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.348+0000 7fdaab400640 1 --2- 172.21.15.19:0/2814861584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaa40d2a40 0x7fdaa40d2ea0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:21.349 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.348+0000 7fdaab400640 1 --2- 172.21.15.19:0/2814861584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdaa415e910 0x7fdaa40d2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:21.349 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.348+0000 7fdaab400640 1 -- 172.21.15.19:0/2814861584 >> 172.21.15.19:0/2814861584 conn(0x7fdaa409d770 msgr2=0x7fdaa40d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:21.349 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.348+0000 7fdaab400640 1 -- 172.21.15.19:0/2814861584 shutdown_connections 2024-03-28T13:09:21.349 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.348+0000 7fdaab400640 1 -- 172.21.15.19:0/2814861584 wait complete. 2024-03-28T13:09:21.350 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.350+0000 7fdaab400640 1 Processor -- start 2024-03-28T13:09:21.350 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.350+0000 7fdaab400640 1 -- start start 2024-03-28T13:09:21.351 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.351+0000 7fdaab400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaa40d2a40 0x7fdaa40b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:21.352 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.351+0000 7fdaab400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdaa415e910 0x7fdaa4159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:21.352 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.351+0000 7fdaab400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaa41622d0 0x7fdaa4159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:21.352 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.351+0000 7fdaab400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa40d1350 con 0x7fdaa41622d0 2024-03-28T13:09:21.352 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.351+0000 7fdaab400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa40d1070 con 0x7fdaa40d2a40 2024-03-28T13:09:21.352 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.351+0000 7fdaab400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa40d1630 con 0x7fdaa415e910 2024-03-28T13:09:21.352 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.351+0000 7fdaa9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaa41622d0 0x7fdaa4159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:21.352 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.352+0000 7fdaa3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdaa415e910 0x7fdaa4159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:21.352 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.352+0000 7fdaa8a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaa40d2a40 0x7fdaa40b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:21.352 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.352+0000 7fdaa9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaa41622d0 0x7fdaa4159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42858/0 (socket says 172.21.15.19:42858) 2024-03-28T13:09:21.352 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.352+0000 7fdaa9400640 1 -- 172.21.15.19:0/1672163386 learned_addr learned my addr 172.21.15.19:0/1672163386 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:21.353 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.352+0000 7fdaa9400640 1 -- 172.21.15.19:0/1672163386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdaa415e910 msgr2=0x7fdaa4159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:21.353 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.352+0000 7fdaa9400640 1 --2- 172.21.15.19:0/1672163386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdaa415e910 0x7fdaa4159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:21.354 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.352+0000 7fdaa9400640 1 -- 172.21.15.19:0/1672163386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaa40d2a40 msgr2=0x7fdaa40b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:21.354 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.352+0000 7fdaa9400640 1 --2- 172.21.15.19:0/1672163386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaa40d2a40 0x7fdaa40b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:21.354 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.352+0000 7fdaa9400640 1 -- 172.21.15.19:0/1672163386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa40cd6a0 con 0x7fdaa41622d0 2024-03-28T13:09:21.354 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.352+0000 7fdaa8a00640 1 --2- 172.21.15.19:0/1672163386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaa40d2a40 0x7fdaa40b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:09:21.354 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.353+0000 7fdaa9400640 1 --2- 172.21.15.19:0/1672163386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaa41622d0 0x7fdaa4159c90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00caa0 tx=0x7fda9c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:21.354 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.353+0000 7fdaa0c00640 1 -- 172.21.15.19:0/1672163386 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c017070 con 0x7fdaa41622d0 2024-03-28T13:09:21.354 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.353+0000 7fdaa0c00640 1 -- 172.21.15.19:0/1672163386 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda9c00ac00 con 0x7fdaa41622d0 2024-03-28T13:09:21.354 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.353+0000 7fdaa0c00640 1 -- 172.21.15.19:0/1672163386 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c012400 con 0x7fdaa41622d0 2024-03-28T13:09:21.354 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.353+0000 7fdaab400640 1 -- 172.21.15.19:0/1672163386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa40cce00 con 0x7fdaa41622d0 2024-03-28T13:09:21.354 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.353+0000 7fdaab400640 1 -- 172.21.15.19:0/1672163386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa40cd3c0 con 0x7fdaa41622d0 2024-03-28T13:09:21.354 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.353+0000 7fdaa0c00640 1 -- 172.21.15.19:0/1672163386 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fda9c01a070 con 0x7fdaa41622d0 2024-03-28T13:09:21.354 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.353+0000 7fdaa0c00640 1 -- 172.21.15.19:0/1672163386 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fda9c012c70 con 0x7fdaa41622d0 2024-03-28T13:09:21.355 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.355+0000 7fdaab400640 1 -- 172.21.15.19:0/1672163386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa404f8b0 con 0x7fdaa41622d0 2024-03-28T13:09:21.357 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.357+0000 7fdaa0c00640 1 -- 172.21.15.19:0/1672163386 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fda9c021070 con 0x7fdaa41622d0 2024-03-28T13:09:21.417 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:21.417+0000 7fd85a9b8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T13:09:21.437 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.437+0000 7fdaab400640 1 -- 172.21.15.19:0/1672163386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdaa40b38e0 con 0x7fdaa41622d0 2024-03-28T13:09:21.438 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.438+0000 7fdaa0c00640 1 -- 172.21.15.19:0/1672163386 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fda9c01c440 con 0x7fdaa41622d0 2024-03-28T13:09:21.439 INFO:teuthology.misc.health.smithi019.stdout: 2024-03-28T13:09:21.439 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":10,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:19.983398+0000","last_up_change":"0.000000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:09:21.440 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.439+0000 7fdaab400640 1 -- 172.21.15.19:0/1672163386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaa41622d0 msgr2=0x7fdaa4159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:21.440 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.439+0000 7fdaab400640 1 --2- 172.21.15.19:0/1672163386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaa41622d0 0x7fdaa4159c90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00caa0 tx=0x7fda9c00cf60 comp rx=0 tx=0).stop 2024-03-28T13:09:21.440 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.440+0000 7fdaab400640 1 -- 172.21.15.19:0/1672163386 shutdown_connections 2024-03-28T13:09:21.440 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.440+0000 7fdaab400640 1 --2- 172.21.15.19:0/1672163386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaa41622d0 0x7fdaa4159c90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:21.440 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.440+0000 7fdaab400640 1 --2- 172.21.15.19:0/1672163386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdaa415e910 0x7fdaa4159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:21.440 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.440+0000 7fdaab400640 1 --2- 172.21.15.19:0/1672163386 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaa40d2a40 0x7fdaa40b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:21.440 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.440+0000 7fdaab400640 1 -- 172.21.15.19:0/1672163386 >> 172.21.15.19:0/1672163386 conn(0x7fdaa409d770 msgr2=0x7fdaa40b4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:21.440 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.440+0000 7fdaab400640 1 -- 172.21.15.19:0/1672163386 shutdown_connections 2024-03-28T13:09:21.440 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:21.440+0000 7fdaab400640 1 -- 172.21.15.19:0/1672163386 wait complete. 2024-03-28T13:09:21.452 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-28T13:09:21.560 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:21.559+0000 7fb0e2673040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-28T13:09:21.755 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:21.755+0000 7fd85a9b8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T13:09:21.816 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:21.816+0000 7fd85a9b8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T13:09:21.896 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:21.896+0000 7fb0e2673040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-28T13:09:21.949 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:21.949+0000 7fd85a9b8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T13:09:21.962 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:21.962+0000 7fb0e2673040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-28T13:09:22.105 INFO:tasks.ceph.mgr.y.smithi019.stderr:2024-03-28T13:09:22.105+0000 7fb0e2673040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-28T13:09:22.994 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:22.995+0000 7fd7fba00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-28T13:09:22.995 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-03-28T13:09:22.996+0000 7fd7fba00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-28T13:09:23.829 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-03-28T13:09:23.830+0000 7f767636f640 -1 osd.7 0 log_to_monitors true 2024-03-28T13:09:24.095 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-03-28T13:09:24.095+0000 7f5e9fb73640 -1 osd.0 0 log_to_monitors true 2024-03-28T13:09:24.264 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-03-28T13:09:24.265+0000 7ff231e0d640 -1 osd.5 0 log_to_monitors true 2024-03-28T13:09:24.267 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-03-28T13:09:24.267+0000 7f9d1ea10640 -1 osd.1 0 log_to_monitors true 2024-03-28T13:09:24.311 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-03-28T13:09:24.312+0000 7f392d01a640 -1 osd.6 0 log_to_monitors true 2024-03-28T13:09:24.321 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-03-28T13:09:24.322+0000 7f7390b24640 -1 osd.4 0 log_to_monitors true 2024-03-28T13:09:24.329 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-03-28T13:09:24.329+0000 7fc817211640 -1 osd.3 0 log_to_monitors true 2024-03-28T13:09:24.333 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-03-28T13:09:24.333+0000 7fa837f36640 -1 osd.2 0 log_to_monitors true 2024-03-28T13:09:26.309 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-03-28T13:09:26.309+0000 7f7671200640 -1 osd.7 0 waiting for initial osdmap 2024-03-28T13:09:26.343 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-03-28T13:09:26.344+0000 7f7667200640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T13:09:27.862 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-03-28T13:09:27.863+0000 7f3927800640 -1 osd.6 0 waiting for initial osdmap 2024-03-28T13:09:27.863 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-03-28T13:09:27.864+0000 7ff22c600640 -1 osd.5 0 waiting for initial osdmap 2024-03-28T13:09:27.870 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-03-28T13:09:27.871+0000 7f391d800640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T13:09:27.875 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-03-28T13:09:27.876+0000 7ff222600640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T13:09:27.897 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-03-28T13:09:27.897+0000 7f738ba00640 -1 osd.4 0 waiting for initial osdmap 2024-03-28T13:09:27.904 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-03-28T13:09:27.905+0000 7f7381a00640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T13:09:28.259 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-28T13:09:28.430 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-03-28T13:09:28.430+0000 7f5e9b400640 -1 osd.0 0 waiting for initial osdmap 2024-03-28T13:09:28.432 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-03-28T13:09:28.432+0000 7fc812600640 -1 osd.3 0 waiting for initial osdmap 2024-03-28T13:09:28.438 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-03-28T13:09:28.438+0000 7f5e90a00640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T13:09:28.443 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-03-28T13:09:28.444+0000 7f9d19200640 -1 osd.1 0 waiting for initial osdmap 2024-03-28T13:09:28.451 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-03-28T13:09:28.451+0000 7fc807c00640 -1 osd.3 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T13:09:28.455 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-03-28T13:09:28.455+0000 7f9d0f200640 -1 osd.1 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T13:09:28.466 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.466+0000 7f4da7600640 1 Processor -- start 2024-03-28T13:09:28.466 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.466+0000 7f4da7600640 1 -- start start 2024-03-28T13:09:28.468 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.467+0000 7f4da7600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4da00acee0 0x7f4da00ad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:28.468 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.467+0000 7f4da7600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4da006df70 0x7f4da006e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:28.468 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.467+0000 7f4da7600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4da015d420 con 0x7f4da015f960 2024-03-28T13:09:28.468 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.467+0000 7f4da7600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4da00ad880 con 0x7f4da00acee0 2024-03-28T13:09:28.468 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.467+0000 7f4da7600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4da015d700 con 0x7f4da006df70 2024-03-28T13:09:28.469 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.468+0000 7f4d9fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4da00acee0 0x7f4da00ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:28.469 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.468+0000 7f4d9fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4da00acee0 0x7f4da00ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54160/0 (socket says 172.21.15.19:54160) 2024-03-28T13:09:28.469 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.468+0000 7f4d9fe00640 1 -- 172.21.15.19:0/4102602014 learned_addr learned my addr 172.21.15.19:0/4102602014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:28.469 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.468+0000 7f4da5600640 1 --2- 172.21.15.19:0/4102602014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4da006df70 0x7f4da006e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:28.469 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.468+0000 7f4d9fe00640 1 -- 172.21.15.19:0/4102602014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4da006df70 msgr2=0x7f4da006e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:28.469 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.468+0000 7f4d9fe00640 1 --2- 172.21.15.19:0/4102602014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4da006df70 0x7f4da006e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:28.469 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.468+0000 7f4d9fe00640 1 -- 172.21.15.19:0/4102602014 >> v1:172.21.15.188:6789/0 conn(0x7f4da015f960 legacy=0x7f4da015c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:28.469 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.468+0000 7f4d9fe00640 1 -- 172.21.15.19:0/4102602014 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da006d770 con 0x7f4da00acee0 2024-03-28T13:09:28.469 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.468+0000 7f4da5600640 1 --2- 172.21.15.19:0/4102602014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4da006df70 0x7f4da006e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:28.470 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.469+0000 7f4d9fe00640 1 --2- 172.21.15.19:0/4102602014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da00acee0 0x7f4da00ad340 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009870 tx=0x7f4d8c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb80674367f93820 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:28.470 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.469+0000 7f4d9f400640 1 -- 172.21.15.19:0/4102602014 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c035020 con 0x7f4da00acee0 2024-03-28T13:09:28.471 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.470+0000 7f4d9f400640 1 -- 172.21.15.19:0/4102602014 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d8c031640 con 0x7f4da00acee0 2024-03-28T13:09:28.471 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.470+0000 7f4d9f400640 1 -- 172.21.15.19:0/4102602014 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c02d040 con 0x7f4da00acee0 2024-03-28T13:09:28.471 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.470+0000 7f4da7600640 1 -- 172.21.15.19:0/4102602014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da00acee0 msgr2=0x7f4da00ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:28.471 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.470+0000 7f4da7600640 1 --2- 172.21.15.19:0/4102602014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da00acee0 0x7f4da00ad340 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c009870 tx=0x7f4d8c02b8f0 comp rx=0 tx=0).stop 2024-03-28T13:09:28.471 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.470+0000 7f4da7600640 1 -- 172.21.15.19:0/4102602014 shutdown_connections 2024-03-28T13:09:28.471 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.470+0000 7f4da7600640 1 --2- 172.21.15.19:0/4102602014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da00acee0 0x7f4da00ad340 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:28.471 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.470+0000 7f4da7600640 1 --2- 172.21.15.19:0/4102602014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4da006df70 0x7f4da006e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:28.471 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.470+0000 7f4da7600640 1 -- 172.21.15.19:0/4102602014 >> 172.21.15.19:0/4102602014 conn(0x7f4da006bba0 msgr2=0x7f4da00b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:28.471 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.470+0000 7f4da7600640 1 -- 172.21.15.19:0/4102602014 shutdown_connections 2024-03-28T13:09:28.471 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.470+0000 7f4da7600640 1 -- 172.21.15.19:0/4102602014 wait complete. 2024-03-28T13:09:28.472 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.472+0000 7f4da7600640 1 Processor -- start 2024-03-28T13:09:28.472 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.472+0000 7f4da7600640 1 -- start start 2024-03-28T13:09:28.474 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.473+0000 7f4da7600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da006df70 0x7f4da014bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:28.474 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.473+0000 7f4da7600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4da00acee0 0x7f4da00cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:28.474 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.473+0000 7f4da7600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4da015f960 0x7f4da00ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:28.474 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.473+0000 7f4da7600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4da015d000 con 0x7f4da006df70 2024-03-28T13:09:28.474 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.473+0000 7f4da7600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4da009dee0 con 0x7f4da015f960 2024-03-28T13:09:28.474 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.473+0000 7f4da7600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4da015d180 con 0x7f4da00acee0 2024-03-28T13:09:28.474 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.473+0000 7f4da4c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da006df70 0x7f4da014bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:28.474 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.473+0000 7f4da4c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da006df70 0x7f4da014bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54172/0 (socket says 172.21.15.19:54172) 2024-03-28T13:09:28.474 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.473+0000 7f4da4c00640 1 -- 172.21.15.19:0/1249018678 learned_addr learned my addr 172.21.15.19:0/1249018678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:28.474 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.473+0000 7f4d9fe00640 1 --2- 172.21.15.19:0/1249018678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4da00acee0 0x7f4da00cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:28.474 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.473+0000 7f4da5600640 1 --2- 172.21.15.19:0/1249018678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4da015f960 0x7f4da00ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:28.474 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.474+0000 7f4da4c00640 1 -- 172.21.15.19:0/1249018678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4da00acee0 msgr2=0x7f4da00cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:28.475 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.474+0000 7f4da4c00640 1 --2- 172.21.15.19:0/1249018678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4da00acee0 0x7f4da00cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:28.475 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.474+0000 7f4da4c00640 1 -- 172.21.15.19:0/1249018678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4da015f960 msgr2=0x7f4da00ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:28.475 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.474+0000 7f4da4c00640 1 --2- 172.21.15.19:0/1249018678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4da015f960 0x7f4da00ccf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:28.475 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.474+0000 7f4da4c00640 1 -- 172.21.15.19:0/1249018678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da014c600 con 0x7f4da006df70 2024-03-28T13:09:28.475 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.474+0000 7f4da4c00640 1 --2- 172.21.15.19:0/1249018678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da006df70 0x7f4da014bc30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400b3f0 tx=0x7f4d9400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:28.476 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.474+0000 7f4d9cc00640 1 -- 172.21.15.19:0/1249018678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d94015020 con 0x7f4da006df70 2024-03-28T13:09:28.476 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.474+0000 7f4d9cc00640 1 -- 172.21.15.19:0/1249018678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d94011830 con 0x7f4da006df70 2024-03-28T13:09:28.476 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.474+0000 7f4da7600640 1 -- 172.21.15.19:0/1249018678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da00cd550 con 0x7f4da006df70 2024-03-28T13:09:28.476 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.474+0000 7f4d9cc00640 1 -- 172.21.15.19:0/1249018678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d94011c20 con 0x7f4da006df70 2024-03-28T13:09:28.476 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.474+0000 7f4da7600640 1 -- 172.21.15.19:0/1249018678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da00cdb10 con 0x7f4da006df70 2024-03-28T13:09:28.476 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.476+0000 7f4da7600640 1 -- 172.21.15.19:0/1249018678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da004f8b0 con 0x7f4da006df70 2024-03-28T13:09:28.476 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.477+0000 7f4d9cc00640 1 -- 172.21.15.19:0/1249018678 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 90800+0+0 (secure 0 0 0) 0x7f4d94018020 con 0x7f4da006df70 2024-03-28T13:09:28.477 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.477+0000 7f4d9cc00640 1 --2- 172.21.15.19:0/1249018678 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d8006c4a0 0x7f4d8006e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:28.477 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.477+0000 7f4d9cc00640 1 -- 172.21.15.19:0/1249018678 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 3351+0+0 (secure 0 0 0) 0x7f4d9408a9a0 con 0x7f4da006df70 2024-03-28T13:09:28.478 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.478+0000 7f4d9fe00640 1 --2- 172.21.15.19:0/1249018678 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d8006c4a0 0x7f4d8006e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:28.479 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.479+0000 7f4d9fe00640 1 --2- 172.21.15.19:0/1249018678 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d8006c4a0 0x7f4d8006e960 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c036000 tx=0x7f4d8c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:28.483 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.483+0000 7f4d9cc00640 1 -- 172.21.15.19:0/1249018678 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4d9401f030 con 0x7f4da006df70 2024-03-28T13:09:28.486 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-03-28T13:09:28.487+0000 7fa832e00640 -1 osd.2 0 waiting for initial osdmap 2024-03-28T13:09:28.493 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-03-28T13:09:28.493+0000 7fa828e00640 -1 osd.2 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-28T13:09:28.682 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.682+0000 7f4da7600640 1 -- 172.21.15.19:0/1249018678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4da00b3000 con 0x7f4da006df70 2024-03-28T13:09:28.683 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.683+0000 7f4d9cc00640 1 -- 172.21.15.19:0/1249018678 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+9036 (secure 0 0 0) 0x7f4d94059930 con 0x7f4da006df70 2024-03-28T13:09:28.683 INFO:teuthology.misc.health.smithi019.stdout: 2024-03-28T13:09:28.683 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":15,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:28.428439+0000","last_up_change":"2024-03-28T13:09:28.428439+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:09:28.686 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.685+0000 7f4da7600640 1 -- 172.21.15.19:0/1249018678 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d8006c4a0 msgr2=0x7f4d8006e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:28.686 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.685+0000 7f4da7600640 1 --2- 172.21.15.19:0/1249018678 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d8006c4a0 0x7f4d8006e960 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c036000 tx=0x7f4d8c03c040 comp rx=0 tx=0).stop 2024-03-28T13:09:28.686 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.685+0000 7f4da7600640 1 -- 172.21.15.19:0/1249018678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da006df70 msgr2=0x7f4da014bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:28.686 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.685+0000 7f4da7600640 1 --2- 172.21.15.19:0/1249018678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da006df70 0x7f4da014bc30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400b3f0 tx=0x7f4d9400b8b0 comp rx=0 tx=0).stop 2024-03-28T13:09:28.686 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.685+0000 7f4da7600640 1 -- 172.21.15.19:0/1249018678 shutdown_connections 2024-03-28T13:09:28.686 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.685+0000 7f4da7600640 1 --2- 172.21.15.19:0/1249018678 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d8006c4a0 0x7f4d8006e960 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:28.686 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.685+0000 7f4da7600640 1 --2- 172.21.15.19:0/1249018678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4da015f960 0x7f4da00ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:28.686 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.685+0000 7f4da7600640 1 --2- 172.21.15.19:0/1249018678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4da00acee0 0x7f4da00cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:28.686 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.685+0000 7f4da7600640 1 --2- 172.21.15.19:0/1249018678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4da006df70 0x7f4da014bc30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:28.686 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.685+0000 7f4da7600640 1 -- 172.21.15.19:0/1249018678 >> 172.21.15.19:0/1249018678 conn(0x7f4da006bba0 msgr2=0x7f4da00aee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:28.686 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.686+0000 7f4da7600640 1 -- 172.21.15.19:0/1249018678 shutdown_connections 2024-03-28T13:09:28.686 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:28.686+0000 7f4da7600640 1 -- 172.21.15.19:0/1249018678 wait complete. 2024-03-28T13:09:28.699 DEBUG:teuthology.misc:4 of 8 OSDs are up 2024-03-28T13:09:30.831 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:09:30.839 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:09:35.506 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-28T13:09:35.692 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.691+0000 7fcdada00640 1 Processor -- start 2024-03-28T13:09:35.692 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.692+0000 7fcdada00640 1 -- start start 2024-03-28T13:09:35.693 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.693+0000 7fcdada00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcda815fac0 0x7fcda80ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:35.693 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.693+0000 7fcdada00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcda80cf2c0 0x7fcda80b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:35.693 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.693+0000 7fcdada00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcda80d2650 con 0x7fcda80b5d70 2024-03-28T13:09:35.693 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.693+0000 7fcdada00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fcda80d2930 con 0x7fcda815fac0 2024-03-28T13:09:35.694 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.693+0000 7fcdada00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcda80d2c10 con 0x7fcda80cf2c0 2024-03-28T13:09:35.694 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.694+0000 7fcda7400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcda815fac0 0x7fcda80ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:35.694 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.694+0000 7fcda7e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fcda80b5d70 0x7fcda80b6150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:32866/0 (socket says 172.21.15.19:32866) 2024-03-28T13:09:35.694 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.694+0000 7fcda7e00640 1 -- 172.21.15.19:0/4254510683 learned_addr learned my addr 172.21.15.19:0/4254510683 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:35.694 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.694+0000 7fcdad000640 1 --2- 172.21.15.19:0/4254510683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcda80cf2c0 0x7fcda80b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:35.695 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.695+0000 7fcda7400640 1 -- 172.21.15.19:0/4254510683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcda80cf2c0 msgr2=0x7fcda80b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:35.695 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.695+0000 7fcda7400640 1 --2- 172.21.15.19:0/4254510683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcda80cf2c0 0x7fcda80b2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:35.695 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.695+0000 7fcda7400640 1 -- 172.21.15.19:0/4254510683 >> v1:172.21.15.188:6789/0 conn(0x7fcda80b5d70 legacy=0x7fcda80b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:35.695 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.695+0000 7fcda7400640 1 -- 172.21.15.19:0/4254510683 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda80d2ef0 con 0x7fcda815fac0 2024-03-28T13:09:35.696 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.696+0000 7fcda7400640 1 --2- 172.21.15.19:0/4254510683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcda815fac0 0x7fcda80ced80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009790 tx=0x7fcd9002b5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b885a507ba20d22 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:35.697 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.697+0000 7fcda6a00640 1 -- 172.21.15.19:0/4254510683 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd90037070 con 0x7fcda815fac0 2024-03-28T13:09:35.697 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.697+0000 7fcda6a00640 1 -- 172.21.15.19:0/4254510683 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd90030710 con 0x7fcda815fac0 2024-03-28T13:09:35.697 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.697+0000 7fcda6a00640 1 -- 172.21.15.19:0/4254510683 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd90030ae0 con 0x7fcda815fac0 2024-03-28T13:09:35.697 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.697+0000 7fcdada00640 1 -- 172.21.15.19:0/4254510683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcda815fac0 msgr2=0x7fcda80ced80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:35.698 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.697+0000 7fcdada00640 1 --2- 172.21.15.19:0/4254510683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcda815fac0 0x7fcda80ced80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009790 tx=0x7fcd9002b5c0 comp rx=0 tx=0).stop 2024-03-28T13:09:35.698 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.697+0000 7fcdada00640 1 -- 172.21.15.19:0/4254510683 shutdown_connections 2024-03-28T13:09:35.698 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.697+0000 7fcdada00640 1 --2- 172.21.15.19:0/4254510683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcda815fac0 0x7fcda80ced80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:35.698 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.697+0000 7fcdada00640 1 --2- 172.21.15.19:0/4254510683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcda80cf2c0 0x7fcda80b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:35.698 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.697+0000 7fcdada00640 1 -- 172.21.15.19:0/4254510683 >> 172.21.15.19:0/4254510683 conn(0x7fcda8067830 msgr2=0x7fcda80d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:35.698 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.698+0000 7fcdada00640 1 -- 172.21.15.19:0/4254510683 shutdown_connections 2024-03-28T13:09:35.698 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.698+0000 7fcdada00640 1 -- 172.21.15.19:0/4254510683 wait complete. 2024-03-28T13:09:35.699 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.699+0000 7fcdada00640 1 Processor -- start 2024-03-28T13:09:35.699 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.700+0000 7fcdada00640 1 -- start start 2024-03-28T13:09:35.701 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.700+0000 7fcdada00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda80b5d70 0x7fcda815aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:35.701 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.700+0000 7fcdada00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcda80cf2c0 0x7fcda80ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:35.701 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.700+0000 7fcdada00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcda815fac0 0x7fcda80b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:35.701 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.700+0000 7fcdada00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcda80cf8d0 con 0x7fcda80b5d70 2024-03-28T13:09:35.701 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.700+0000 7fcdada00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcda80cf750 con 0x7fcda80cf2c0 2024-03-28T13:09:35.701 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.700+0000 7fcdada00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcda80b32a0 con 0x7fcda815fac0 2024-03-28T13:09:35.701 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.701+0000 7fcda7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda80b5d70 0x7fcda815aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:35.701 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.701+0000 7fcda7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda80b5d70 0x7fcda815aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33948/0 (socket says 172.21.15.19:33948) 2024-03-28T13:09:35.702 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.701+0000 7fcda7e00640 1 -- 172.21.15.19:0/688785456 learned_addr learned my addr 172.21.15.19:0/688785456 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:35.702 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.701+0000 7fcda7400640 1 --2- 172.21.15.19:0/688785456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcda80cf2c0 0x7fcda80ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:35.702 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.701+0000 7fcdad000640 1 --2- 172.21.15.19:0/688785456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcda815fac0 0x7fcda80b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:35.703 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.701+0000 7fcda7e00640 1 -- 172.21.15.19:0/688785456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcda815fac0 msgr2=0x7fcda80b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:35.703 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.701+0000 7fcda7e00640 1 --2- 172.21.15.19:0/688785456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcda815fac0 0x7fcda80b48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:35.703 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.701+0000 7fcda7e00640 1 -- 172.21.15.19:0/688785456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcda80cf2c0 msgr2=0x7fcda80ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:35.703 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.702+0000 7fcda7e00640 1 --2- 172.21.15.19:0/688785456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcda80cf2c0 0x7fcda80ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:35.703 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.702+0000 7fcda7e00640 1 -- 172.21.15.19:0/688785456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda80cc690 con 0x7fcda80b5d70 2024-03-28T13:09:35.703 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.702+0000 7fcda7400640 1 --2- 172.21.15.19:0/688785456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcda80cf2c0 0x7fcda80ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:35.703 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.702+0000 7fcda7e00640 1 --2- 172.21.15.19:0/688785456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda80b5d70 0x7fcda815aab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800c7b0 tx=0x7fcd9800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:35.703 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.702+0000 7fcd9fe00640 1 -- 172.21.15.19:0/688785456 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98018070 con 0x7fcda80b5d70 2024-03-28T13:09:35.703 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.702+0000 7fcd9fe00640 1 -- 172.21.15.19:0/688785456 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd980072d0 con 0x7fcda80b5d70 2024-03-28T13:09:35.703 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.702+0000 7fcdada00640 1 -- 172.21.15.19:0/688785456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda80cc930 con 0x7fcda80b5d70 2024-03-28T13:09:35.704 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.702+0000 7fcd9fe00640 1 -- 172.21.15.19:0/688785456 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd980047e0 con 0x7fcda80b5d70 2024-03-28T13:09:35.704 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.702+0000 7fcdada00640 1 -- 172.21.15.19:0/688785456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda8150d10 con 0x7fcda80b5d70 2024-03-28T13:09:35.704 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.704+0000 7fcdada00640 1 -- 172.21.15.19:0/688785456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda8075a60 con 0x7fcda80b5d70 2024-03-28T13:09:35.705 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.705+0000 7fcd9fe00640 1 -- 172.21.15.19:0/688785456 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcd9801b070 con 0x7fcda80b5d70 2024-03-28T13:09:35.706 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.706+0000 7fcd9fe00640 1 --2- 172.21.15.19:0/688785456 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd8406c560 0x7fcd8406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:35.706 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.706+0000 7fcd9fe00640 1 -- 172.21.15.19:0/688785456 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fcd9808a8e0 con 0x7fcda80b5d70 2024-03-28T13:09:35.706 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.706+0000 7fcda7400640 1 --2- 172.21.15.19:0/688785456 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd8406c560 0x7fcd8406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:35.707 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.707+0000 7fcda7400640 1 --2- 172.21.15.19:0/688785456 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd8406c560 0x7fcd8406ea20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcd90034000 tx=0x7fcd90008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:35.711 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.711+0000 7fcd9fe00640 1 -- 172.21.15.19:0/688785456 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcd98059120 con 0x7fcda80b5d70 2024-03-28T13:09:35.918 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.918+0000 7fcdada00640 1 -- 172.21.15.19:0/688785456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcda8160b70 con 0x7fcda80b5d70 2024-03-28T13:09:35.920 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.920+0000 7fcd9fe00640 1 -- 172.21.15.19:0/688785456 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12884 (secure 0 0 0) 0x7fcd98058a20 con 0x7fcda80b5d70 2024-03-28T13:09:35.921 INFO:teuthology.misc.health.smithi019.stdout: 2024-03-28T13:09:35.921 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":19,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:32.463538+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:09:35.923 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.923+0000 7fcdada00640 1 -- 172.21.15.19:0/688785456 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd8406c560 msgr2=0x7fcd8406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:35.924 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.923+0000 7fcdada00640 1 --2- 172.21.15.19:0/688785456 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd8406c560 0x7fcd8406ea20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fcd90034000 tx=0x7fcd90008030 comp rx=0 tx=0).stop 2024-03-28T13:09:35.924 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.923+0000 7fcdada00640 1 -- 172.21.15.19:0/688785456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda80b5d70 msgr2=0x7fcda815aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:35.924 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.923+0000 7fcdada00640 1 --2- 172.21.15.19:0/688785456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda80b5d70 0x7fcda815aab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcd9800c7b0 tx=0x7fcd9800cc70 comp rx=0 tx=0).stop 2024-03-28T13:09:35.924 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.923+0000 7fcdada00640 1 -- 172.21.15.19:0/688785456 shutdown_connections 2024-03-28T13:09:35.924 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.923+0000 7fcdada00640 1 --2- 172.21.15.19:0/688785456 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd8406c560 0x7fcd8406ea20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:35.924 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.923+0000 7fcdada00640 1 --2- 172.21.15.19:0/688785456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcda815fac0 0x7fcda80b48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:35.924 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.923+0000 7fcdada00640 1 --2- 172.21.15.19:0/688785456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcda80cf2c0 0x7fcda80ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:35.924 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.923+0000 7fcdada00640 1 --2- 172.21.15.19:0/688785456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcda80b5d70 0x7fcda815aab0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:35.924 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.923+0000 7fcdada00640 1 -- 172.21.15.19:0/688785456 >> 172.21.15.19:0/688785456 conn(0x7fcda8067830 msgr2=0x7fcda815ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:35.924 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.923+0000 7fcdada00640 1 -- 172.21.15.19:0/688785456 shutdown_connections 2024-03-28T13:09:35.924 INFO:teuthology.misc.health.smithi019.stderr:2024-03-28T13:09:35.923+0000 7fcdada00640 1 -- 172.21.15.19:0/688785456 wait complete. 2024-03-28T13:09:35.939 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-28T13:09:35.939 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-28T13:09:35.940 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-28T13:09:35.940 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-28T13:09:35.947 INFO:tasks.ceph.mds.a:Started 2024-03-28T13:09:35.947 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-28T13:09:35.947 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-28T13:09:35.954 INFO:tasks.ceph.mds.c:Started 2024-03-28T13:09:35.955 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-28T13:09:35.955 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-28T13:09:35.963 INFO:tasks.ceph.mds.b:Started 2024-03-28T13:09:35.964 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-28T13:09:35.964 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-28T13:09:35.971 INFO:tasks.ceph.mds.d:Started 2024-03-28T13:09:35.971 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-28T13:09:35.972 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:09:36.100 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-03-28T13:09:36.119 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-03-28T13:09:36.132 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-03-28T13:09:36.132 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-03-28T13:09:36.135 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-03-28T13:09:36.136+0000 7f8c35e00640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2024-03-28T13:09:36.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.161+0000 7fd0b5000640 1 Processor -- start 2024-03-28T13:09:36.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.162+0000 7fd0b5000640 1 -- start start 2024-03-28T13:09:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.163+0000 7fd0b5000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b015c390 0x7fd0b0073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.163+0000 7fd0b5000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0b015fe80 con 0x7fd0b00d0650 2024-03-28T13:09:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.163+0000 7fd0b5000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0b00ad120 con 0x7fd0b00af830 2024-03-28T13:09:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.163+0000 7fd0b5000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd0b015c770 con 0x7fd0b015c390 2024-03-28T13:09:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.163+0000 7fd0ae800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd0b00d0650 0x7fd0b0071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59226/0 (socket says 172.21.15.19:59226) 2024-03-28T13:09:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.163+0000 7fd0ae800640 1 -- 172.21.15.19:0/3369217279 learned_addr learned my addr 172.21.15.19:0/3369217279 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.163+0000 7fd0ade00640 1 --2- 172.21.15.19:0/3369217279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b015c390 0x7fd0b0073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.163+0000 7fd0ad400640 1 -- 172.21.15.19:0/3369217279 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1091508094 0 0) 0x7fd0b015fe80 con 0x7fd0b00d0650 2024-03-28T13:09:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.164+0000 7fd0ad400640 1 -- 172.21.15.19:0/3369217279 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd094003520 con 0x7fd0b00d0650 2024-03-28T13:09:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.164+0000 7fd0ad400640 1 -- 172.21.15.19:0/3369217279 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 594241917 0 0) 0x7fd0b00ad120 con 0x7fd0b00af830 2024-03-28T13:09:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.164+0000 7fd0ad400640 1 -- 172.21.15.19:0/3369217279 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0b015fe80 con 0x7fd0b00af830 2024-03-28T13:09:36.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.164+0000 7fd0ade00640 1 -- 172.21.15.19:0/3369217279 >> v1:172.21.15.188:6789/0 conn(0x7fd0b00af830 legacy=0x7fd0b0160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:36.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.164+0000 7fd0ade00640 1 -- 172.21.15.19:0/3369217279 >> v1:172.21.15.19:6789/0 conn(0x7fd0b00d0650 legacy=0x7fd0b0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:36.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.164+0000 7fd0ade00640 1 -- 172.21.15.19:0/3369217279 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b00ad400 con 0x7fd0b015c390 2024-03-28T13:09:36.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.165+0000 7fd0ade00640 1 --2- 172.21.15.19:0/3369217279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b015c390 0x7fd0b0073790 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00097d0 tx=0x7fd0a002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23171359f3930ba9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:36.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.165+0000 7fd0ad400640 1 -- 172.21.15.19:0/3369217279 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0035020 con 0x7fd0b015c390 2024-03-28T13:09:36.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.165+0000 7fd0ad400640 1 -- 172.21.15.19:0/3369217279 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0a0031890 con 0x7fd0b015c390 2024-03-28T13:09:36.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.165+0000 7fd0ad400640 1 -- 172.21.15.19:0/3369217279 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0031c60 con 0x7fd0b015c390 2024-03-28T13:09:36.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.166+0000 7fd0b5000640 1 -- 172.21.15.19:0/3369217279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b015c390 msgr2=0x7fd0b0073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:36.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.166+0000 7fd0b5000640 1 --2- 172.21.15.19:0/3369217279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b015c390 0x7fd0b0073790 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00097d0 tx=0x7fd0a002b930 comp rx=0 tx=0).stop 2024-03-28T13:09:36.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.166+0000 7fd0b5000640 1 -- 172.21.15.19:0/3369217279 shutdown_connections 2024-03-28T13:09:36.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.166+0000 7fd0b5000640 1 --2- 172.21.15.19:0/3369217279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b015c390 0x7fd0b0073790 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:36.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.166+0000 7fd0b5000640 1 -- 172.21.15.19:0/3369217279 >> 172.21.15.19:0/3369217279 conn(0x7fd0b006b990 msgr2=0x7fd0b006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:36.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.166+0000 7fd0b5000640 1 -- 172.21.15.19:0/3369217279 shutdown_connections 2024-03-28T13:09:36.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.166+0000 7fd0b5000640 1 -- 172.21.15.19:0/3369217279 wait complete. 2024-03-28T13:09:36.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.168+0000 7fd0b5000640 1 Processor -- start 2024-03-28T13:09:36.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.168+0000 7fd0b5000640 1 -- start start 2024-03-28T13:09:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.169+0000 7fd0b5000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0b00af830 0x7fd0b014a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.169+0000 7fd0b5000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b00d0650 0x7fd0b015cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.169+0000 7fd0b5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b015c390 0x7fd0b0145c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.169+0000 7fd0b5000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b0160d50 con 0x7fd0b015c390 2024-03-28T13:09:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.169+0000 7fd0b5000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b0160120 con 0x7fd0b00af830 2024-03-28T13:09:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.169+0000 7fd0b5000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd0b00cf980 con 0x7fd0b00d0650 2024-03-28T13:09:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.169+0000 7fd0ade00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0b00af830 0x7fd0b014a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.169+0000 7fd0ade00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0b00af830 0x7fd0b014a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:34002/0 (socket says 172.21.15.19:34002) 2024-03-28T13:09:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.169+0000 7fd0ade00640 1 -- 172.21.15.19:0/3385731947 learned_addr learned my addr 172.21.15.19:0/3385731947 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.169+0000 7fd0ae800640 1 --2- 172.21.15.19:0/3385731947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b015c390 0x7fd0b0145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.169+0000 7fd0a7e00640 1 --2- 172.21.15.19:0/3385731947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b00d0650 0x7fd0b015cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.170+0000 7fd0ade00640 1 -- 172.21.15.19:0/3385731947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b00d0650 msgr2=0x7fd0b015cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.170+0000 7fd0ade00640 1 --2- 172.21.15.19:0/3385731947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b00d0650 0x7fd0b015cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.170+0000 7fd0ade00640 1 -- 172.21.15.19:0/3385731947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b015c390 msgr2=0x7fd0b0145c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.170+0000 7fd0ade00640 1 --2- 172.21.15.19:0/3385731947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b015c390 0x7fd0b0145c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.170+0000 7fd0ade00640 1 -- 172.21.15.19:0/3385731947 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b015d5d0 con 0x7fd0b00af830 2024-03-28T13:09:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.170+0000 7fd0ade00640 1 --2- 172.21.15.19:0/3385731947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0b00af830 0x7fd0b014a230 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0034040 tx=0x7fd0a00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.170+0000 7fd0a6000640 1 -- 172.21.15.19:0/3385731947 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0006e10 con 0x7fd0b00af830 2024-03-28T13:09:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.170+0000 7fd0a6000640 1 -- 172.21.15.19:0/3385731947 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0a003bd70 con 0x7fd0b00af830 2024-03-28T13:09:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.170+0000 7fd0a6000640 1 -- 172.21.15.19:0/3385731947 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0045650 con 0x7fd0b00af830 2024-03-28T13:09:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.170+0000 7fd0b5000640 1 -- 172.21.15.19:0/3385731947 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b015d8b0 con 0x7fd0b00af830 2024-03-28T13:09:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.170+0000 7fd0b5000640 1 -- 172.21.15.19:0/3385731947 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b0149d10 con 0x7fd0b00af830 2024-03-28T13:09:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.172+0000 7fd0b5000640 1 -- 172.21.15.19:0/3385731947 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b00685c0 con 0x7fd0b00af830 2024-03-28T13:09:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.172+0000 7fd0a6000640 1 -- 172.21.15.19:0/3385731947 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd0a004c060 con 0x7fd0b00af830 2024-03-28T13:09:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.173+0000 7fd0a6000640 1 --2- 172.21.15.19:0/3385731947 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd08c06c530 0x7fd08c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.173+0000 7fd0a6000640 1 -- 172.21.15.19:0/3385731947 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fd0a00ad420 con 0x7fd0b00af830 2024-03-28T13:09:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.174+0000 7fd0a7e00640 1 --2- 172.21.15.19:0/3385731947 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd08c06c530 0x7fd08c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.174+0000 7fd0a7e00640 1 --2- 172.21.15.19:0/3385731947 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd08c06c530 0x7fd08c06e9f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd098007fd0 tx=0x7fd098009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.179+0000 7fd0a6000640 1 -- 172.21.15.19:0/3385731947 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd0a0040070 con 0x7fd0b00af830 2024-03-28T13:09:36.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.384+0000 7fd0b5000640 1 -- 172.21.15.19:0/3385731947 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0b006f1b0 con 0x7fd0b00af830 2024-03-28T13:09:36.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.387+0000 7fd0a6000640 1 -- 172.21.15.19:0/3385731947 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12884 (secure 0 0 0) 0x7fd0a002d040 con 0x7fd0b00af830 2024-03-28T13:09:36.387 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:36.387 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":19,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:32.463538+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:09:36.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.390+0000 7fd0b5000640 1 -- 172.21.15.19:0/3385731947 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd08c06c530 msgr2=0x7fd08c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:36.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.390+0000 7fd0b5000640 1 --2- 172.21.15.19:0/3385731947 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd08c06c530 0x7fd08c06e9f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd098007fd0 tx=0x7fd098009040 comp rx=0 tx=0).stop 2024-03-28T13:09:36.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.390+0000 7fd0b5000640 1 -- 172.21.15.19:0/3385731947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0b00af830 msgr2=0x7fd0b014a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:36.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.390+0000 7fd0b5000640 1 --2- 172.21.15.19:0/3385731947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0b00af830 0x7fd0b014a230 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0034040 tx=0x7fd0a00046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:36.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.390+0000 7fd0b5000640 1 -- 172.21.15.19:0/3385731947 shutdown_connections 2024-03-28T13:09:36.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.390+0000 7fd0b5000640 1 --2- 172.21.15.19:0/3385731947 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd08c06c530 0x7fd08c06e9f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:36.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.390+0000 7fd0b5000640 1 --2- 172.21.15.19:0/3385731947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0b015c390 0x7fd0b0145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:36.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.390+0000 7fd0b5000640 1 --2- 172.21.15.19:0/3385731947 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0b00d0650 0x7fd0b015cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:36.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.390+0000 7fd0b5000640 1 --2- 172.21.15.19:0/3385731947 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0b00af830 0x7fd0b014a230 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:36.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.390+0000 7fd0b5000640 1 -- 172.21.15.19:0/3385731947 >> 172.21.15.19:0/3385731947 conn(0x7fd0b006b990 msgr2=0x7fd0b00aece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:36.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.390+0000 7fd0b5000640 1 -- 172.21.15.19:0/3385731947 shutdown_connections 2024-03-28T13:09:36.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.390+0000 7fd0b5000640 1 -- 172.21.15.19:0/3385731947 wait complete. 2024-03-28T13:09:36.409 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:09:36.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:09:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.635+0000 7f3a77c00640 1 Processor -- start 2024-03-28T13:09:36.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.635+0000 7f3a77c00640 1 -- start start 2024-03-28T13:09:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.636+0000 7f3a77c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3a70141c10 0x7f3a70071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.636+0000 7f3a77c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a7015ec80 con 0x7f3a700d0280 2024-03-28T13:09:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.636+0000 7f3a77c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a7015ef20 con 0x7f3a70071eb0 2024-03-28T13:09:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.636+0000 7f3a77c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3a7015f200 con 0x7f3a70141c10 2024-03-28T13:09:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.637+0000 7f3a75200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3a700d0280 0x7f3a700d0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59238/0 (socket says 172.21.15.19:59238) 2024-03-28T13:09:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.637+0000 7f3a75200640 1 -- 172.21.15.19:0/636778194 learned_addr learned my addr 172.21.15.19:0/636778194 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:36.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.637+0000 7f3a6fe00640 1 --2- 172.21.15.19:0/636778194 >> v2:172.21.15.19:3300/0 conn(0x7f3a70141c10 0x7f3a70071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:36.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.637+0000 7f3a6f400640 1 -- 172.21.15.19:0/636778194 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1929587354 0 0) 0x7f3a7015ec80 con 0x7f3a700d0280 2024-03-28T13:09:36.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.637+0000 7f3a6f400640 1 -- 172.21.15.19:0/636778194 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a54003520 con 0x7f3a700d0280 2024-03-28T13:09:36.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.637+0000 7f3a6fe00640 1 -- 172.21.15.19:0/636778194 >> v1:172.21.15.188:6789/0 conn(0x7f3a70071eb0 legacy=0x7f3a70162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:36.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.637+0000 7f3a6fe00640 1 -- 172.21.15.19:0/636778194 >> v1:172.21.15.19:6789/0 conn(0x7f3a700d0280 legacy=0x7f3a700d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:36.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.637+0000 7f3a6fe00640 1 -- 172.21.15.19:0/636778194 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7015f4e0 con 0x7f3a70141c10 2024-03-28T13:09:36.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.638+0000 7f3a6fe00640 1 --2- 172.21.15.19:0/636778194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a70141c10 0x7f3a70071970 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009d90 tx=0x7f3a6002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ae190ee7265196 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:36.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.639+0000 7f3a6f400640 1 -- 172.21.15.19:0/636778194 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60039070 con 0x7f3a70141c10 2024-03-28T13:09:36.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.639+0000 7f3a6f400640 1 -- 172.21.15.19:0/636778194 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a6002ccf0 con 0x7f3a70141c10 2024-03-28T13:09:36.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.639+0000 7f3a6f400640 1 -- 172.21.15.19:0/636778194 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60034660 con 0x7f3a70141c10 2024-03-28T13:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.639+0000 7f3a77c00640 1 -- 172.21.15.19:0/636778194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a70141c10 msgr2=0x7f3a70071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.639+0000 7f3a77c00640 1 --2- 172.21.15.19:0/636778194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a70141c10 0x7f3a70071970 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009d90 tx=0x7f3a6002bf40 comp rx=0 tx=0).stop 2024-03-28T13:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.639+0000 7f3a77c00640 1 -- 172.21.15.19:0/636778194 shutdown_connections 2024-03-28T13:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.640+0000 7f3a77c00640 1 --2- 172.21.15.19:0/636778194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a70141c10 0x7f3a70071970 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.640+0000 7f3a77c00640 1 -- 172.21.15.19:0/636778194 >> 172.21.15.19:0/636778194 conn(0x7f3a7006b750 msgr2=0x7f3a7006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.640+0000 7f3a77c00640 1 -- 172.21.15.19:0/636778194 shutdown_connections 2024-03-28T13:09:36.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.640+0000 7f3a77c00640 1 -- 172.21.15.19:0/636778194 wait complete. 2024-03-28T13:09:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.641+0000 7f3a77c00640 1 Processor -- start 2024-03-28T13:09:36.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.641+0000 7f3a77c00640 1 -- start start 2024-03-28T13:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.642+0000 7f3a77c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a70071eb0 0x7f3a701f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.642+0000 7f3a77c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a700d0280 0x7f3a7009d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.642+0000 7f3a77c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a70141c10 0x7f3a7009dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.642+0000 7f3a77c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70158e90 con 0x7f3a700d0280 2024-03-28T13:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.642+0000 7f3a77c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7006f340 con 0x7f3a70141c10 2024-03-28T13:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.642+0000 7f3a77c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3a701f42c0 con 0x7f3a70071eb0 2024-03-28T13:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.643+0000 7f3a6fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a700d0280 0x7f3a7009d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.643+0000 7f3a6fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a700d0280 0x7f3a7009d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34012/0 (socket says 172.21.15.19:34012) 2024-03-28T13:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.643+0000 7f3a6fe00640 1 -- 172.21.15.19:0/3529804864 learned_addr learned my addr 172.21.15.19:0/3529804864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:36.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.643+0000 7f3a6fe00640 1 -- 172.21.15.19:0/3529804864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a70071eb0 msgr2=0x7f3a701f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.643+0000 7f3a6fe00640 1 --2- 172.21.15.19:0/3529804864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a70071eb0 0x7f3a701f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.643+0000 7f3a6fe00640 1 -- 172.21.15.19:0/3529804864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a70141c10 msgr2=0x7f3a7009dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.643+0000 7f3a6fe00640 1 --2- 172.21.15.19:0/3529804864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a70141c10 0x7f3a7009dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:36.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.643+0000 7f3a6fe00640 1 -- 172.21.15.19:0/3529804864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7014bf50 con 0x7f3a700d0280 2024-03-28T13:09:36.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.643+0000 7f3a6fe00640 1 --2- 172.21.15.19:0/3529804864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a700d0280 0x7f3a7009d810 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3a60037040 tx=0x7f3a60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:36.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.644+0000 7f3a6cc00640 1 -- 172.21.15.19:0/3529804864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a60039050 con 0x7f3a700d0280 2024-03-28T13:09:36.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.644+0000 7f3a6cc00640 1 -- 172.21.15.19:0/3529804864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a60045070 con 0x7f3a700d0280 2024-03-28T13:09:36.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.644+0000 7f3a6cc00640 1 -- 172.21.15.19:0/3529804864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6003e850 con 0x7f3a700d0280 2024-03-28T13:09:36.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.644+0000 7f3a77c00640 1 -- 172.21.15.19:0/3529804864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7014acf0 con 0x7f3a700d0280 2024-03-28T13:09:36.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.644+0000 7f3a77c00640 1 -- 172.21.15.19:0/3529804864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7014b2b0 con 0x7f3a700d0280 2024-03-28T13:09:36.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.645+0000 7f3a77c00640 1 -- 172.21.15.19:0/3529804864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a700685c0 con 0x7f3a700d0280 2024-03-28T13:09:36.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.646+0000 7f3a6cc00640 1 -- 172.21.15.19:0/3529804864 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3a6003e9f0 con 0x7f3a700d0280 2024-03-28T13:09:36.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.647+0000 7f3a6cc00640 1 --2- 172.21.15.19:0/3529804864 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a4c06c5b0 0x7f3a4c06ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:36.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.647+0000 7f3a6cc00640 1 -- 172.21.15.19:0/3529804864 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f3a600ae3a0 con 0x7f3a700d0280 2024-03-28T13:09:36.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.647+0000 7f3a75200640 1 --2- 172.21.15.19:0/3529804864 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a4c06c5b0 0x7f3a4c06ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:36.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.649+0000 7f3a75200640 1 --2- 172.21.15.19:0/3529804864 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a4c06c5b0 0x7f3a4c06ea70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a58001610 tx=0x7f3a58007930 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:36.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.653+0000 7f3a6cc00640 1 -- 172.21.15.19:0/3529804864 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3a7014bf50 con 0x7f3a700d0280 2024-03-28T13:09:36.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.859+0000 7f3a77c00640 1 -- 172.21.15.19:0/3529804864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3a700cfdb0 con 0x7f3a700d0280 2024-03-28T13:09:36.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.860+0000 7f3a6cc00640 1 -- 172.21.15.19:0/3529804864 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7f3a6007ce10 con 0x7f3a700d0280 2024-03-28T13:09:36.861 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:09:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.863+0000 7f3a77c00640 1 -- 172.21.15.19:0/3529804864 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a4c06c5b0 msgr2=0x7f3a4c06ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.863+0000 7f3a77c00640 1 --2- 172.21.15.19:0/3529804864 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a4c06c5b0 0x7f3a4c06ea70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a58001610 tx=0x7f3a58007930 comp rx=0 tx=0).stop 2024-03-28T13:09:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.863+0000 7f3a77c00640 1 -- 172.21.15.19:0/3529804864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a700d0280 msgr2=0x7f3a7009d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.863+0000 7f3a77c00640 1 --2- 172.21.15.19:0/3529804864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a700d0280 0x7f3a7009d810 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3a60037040 tx=0x7f3a60004a40 comp rx=0 tx=0).stop 2024-03-28T13:09:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.863+0000 7f3a77c00640 1 -- 172.21.15.19:0/3529804864 shutdown_connections 2024-03-28T13:09:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.863+0000 7f3a77c00640 1 --2- 172.21.15.19:0/3529804864 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a4c06c5b0 0x7f3a4c06ea70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.863+0000 7f3a77c00640 1 --2- 172.21.15.19:0/3529804864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a70141c10 0x7f3a7009dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.863+0000 7f3a77c00640 1 --2- 172.21.15.19:0/3529804864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a700d0280 0x7f3a7009d810 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.863+0000 7f3a77c00640 1 --2- 172.21.15.19:0/3529804864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a70071eb0 0x7f3a701f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.863+0000 7f3a77c00640 1 -- 172.21.15.19:0/3529804864 >> 172.21.15.19:0/3529804864 conn(0x7f3a7006b750 msgr2=0x7f3a700afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:36.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.864+0000 7f3a77c00640 1 -- 172.21.15.19:0/3529804864 shutdown_connections 2024-03-28T13:09:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:36.864+0000 7f3a77c00640 1 -- 172.21.15.19:0/3529804864 wait complete. 2024-03-28T13:09:36.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:09:37.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.107+0000 7f7ec6000640 1 Processor -- start 2024-03-28T13:09:37.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.107+0000 7f7ec6000640 1 -- start start 2024-03-28T13:09:37.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.108+0000 7f7ec6000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7ec00af830 0x7f7ec0073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:37.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.109+0000 7f7ec6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ec00d0650 0x7f7ec00d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:37.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.109+0000 7f7ec6000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ec015fe80 con 0x7f7ec015c390 2024-03-28T13:09:37.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.109+0000 7f7ec6000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7ec015c770 con 0x7f7ec00d0650 2024-03-28T13:09:37.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.109+0000 7f7ec6000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7ec00d0fc0 con 0x7f7ec00af830 2024-03-28T13:09:37.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.109+0000 7f7ebf400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7ec015c390 0x7f7ec0160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59254/0 (socket says 172.21.15.19:59254) 2024-03-28T13:09:37.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.109+0000 7f7ebf400640 1 -- 172.21.15.19:0/2484157098 learned_addr learned my addr 172.21.15.19:0/2484157098 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:37.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.109+0000 7f7ebfe00640 1 --2- 172.21.15.19:0/2484157098 >> v2:172.21.15.19:3300/0 conn(0x7f7ec00d0650 0x7f7ec00d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:37.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.109+0000 7f7ebea00640 1 --2- 172.21.15.19:0/2484157098 >> v2:172.21.15.188:3300/0 conn(0x7f7ec00af830 0x7f7ec0073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:37.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.109+0000 7f7ebe000640 1 -- 172.21.15.19:0/2484157098 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2797249148 0 0) 0x7f7ec015fe80 con 0x7f7ec015c390 2024-03-28T13:09:37.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.109+0000 7f7ebe000640 1 -- 172.21.15.19:0/2484157098 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ea4003520 con 0x7f7ec015c390 2024-03-28T13:09:37.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.109+0000 7f7ebfe00640 1 -- 172.21.15.19:0/2484157098 >> v2:172.21.15.188:3300/0 conn(0x7f7ec00af830 msgr2=0x7f7ec0073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:37.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.109+0000 7f7ebfe00640 1 --2- 172.21.15.19:0/2484157098 >> v2:172.21.15.188:3300/0 conn(0x7f7ec00af830 0x7f7ec0073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.109+0000 7f7ebfe00640 1 -- 172.21.15.19:0/2484157098 >> v1:172.21.15.19:6789/0 conn(0x7f7ec015c390 legacy=0x7f7ec0160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:37.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.109+0000 7f7ebfe00640 1 -- 172.21.15.19:0/2484157098 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec00ad120 con 0x7f7ec00d0650 2024-03-28T13:09:37.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.111+0000 7f7ebfe00640 1 --2- 172.21.15.19:0/2484157098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec00d0650 0x7f7ec00d0a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7ec006eaa0 tx=0x7f7eb802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcb7bef113e2678 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:37.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.111+0000 7f7ebe000640 1 -- 172.21.15.19:0/2484157098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8039070 con 0x7f7ec00d0650 2024-03-28T13:09:37.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.111+0000 7f7ebe000640 1 -- 172.21.15.19:0/2484157098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb802cc60 con 0x7f7ec00d0650 2024-03-28T13:09:37.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.111+0000 7f7ebe000640 1 -- 172.21.15.19:0/2484157098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8034660 con 0x7f7ec00d0650 2024-03-28T13:09:37.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.112+0000 7f7ec6000640 1 -- 172.21.15.19:0/2484157098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec00d0650 msgr2=0x7f7ec00d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:37.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.112+0000 7f7ec6000640 1 --2- 172.21.15.19:0/2484157098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec00d0650 0x7f7ec00d0a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7ec006eaa0 tx=0x7f7eb802beb0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.112+0000 7f7ec6000640 1 -- 172.21.15.19:0/2484157098 shutdown_connections 2024-03-28T13:09:37.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.112+0000 7f7ec6000640 1 --2- 172.21.15.19:0/2484157098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec00d0650 0x7f7ec00d0a80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.112+0000 7f7ec6000640 1 --2- 172.21.15.19:0/2484157098 >> v2:172.21.15.188:3300/0 conn(0x7f7ec00af830 0x7f7ec0073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.112+0000 7f7ec6000640 1 -- 172.21.15.19:0/2484157098 >> 172.21.15.19:0/2484157098 conn(0x7f7ec006b990 msgr2=0x7f7ec006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:37.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.112+0000 7f7ec6000640 1 -- 172.21.15.19:0/2484157098 shutdown_connections 2024-03-28T13:09:37.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.112+0000 7f7ec6000640 1 -- 172.21.15.19:0/2484157098 wait complete. 2024-03-28T13:09:37.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.114+0000 7f7ec6000640 1 Processor -- start 2024-03-28T13:09:37.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.114+0000 7f7ec6000640 1 -- start start 2024-03-28T13:09:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ec6000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec00af830 0x7f7ec01576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ec6000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ec00d0650 0x7f7ec0146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ec6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec015c390 0x7f7ec0146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ec6000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec0160120 con 0x7f7ec015c390 2024-03-28T13:09:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ec6000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec00ad6e0 con 0x7f7ec00af830 2024-03-28T13:09:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ec6000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec0160d50 con 0x7f7ec00d0650 2024-03-28T13:09:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ebfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec015c390 0x7f7ec0146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ebfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec015c390 0x7f7ec0146640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34028/0 (socket says 172.21.15.19:34028) 2024-03-28T13:09:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ebfe00640 1 -- 172.21.15.19:0/1720112096 learned_addr learned my addr 172.21.15.19:0/1720112096 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ebf400640 1 --2- 172.21.15.19:0/1720112096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec00af830 0x7f7ec01576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ebea00640 1 --2- 172.21.15.19:0/1720112096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ec00d0650 0x7f7ec0146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ebfe00640 1 -- 172.21.15.19:0/1720112096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ec00d0650 msgr2=0x7f7ec0146100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:37.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ebfe00640 1 --2- 172.21.15.19:0/1720112096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ec00d0650 0x7f7ec0146100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ebfe00640 1 -- 172.21.15.19:0/1720112096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec00af830 msgr2=0x7f7ec01576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:37.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ebfe00640 1 --2- 172.21.15.19:0/1720112096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec00af830 0x7f7ec01576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ebfe00640 1 -- 172.21.15.19:0/1720112096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec00afe20 con 0x7f7ec015c390 2024-03-28T13:09:37.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.116+0000 7f7ebea00640 1 --2- 172.21.15.19:0/1720112096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ec00d0650 0x7f7ec0146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:37.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.117+0000 7f7ebfe00640 1 --2- 172.21.15.19:0/1720112096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec015c390 0x7f7ec0146640 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009d90 tx=0x7f7eb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:37.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.117+0000 7f7eb7400640 1 -- 172.21.15.19:0/1720112096 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8039050 con 0x7f7ec015c390 2024-03-28T13:09:37.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.117+0000 7f7eb7400640 1 -- 172.21.15.19:0/1720112096 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eb8045070 con 0x7f7ec015c390 2024-03-28T13:09:37.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.117+0000 7f7ec6000640 1 -- 172.21.15.19:0/1720112096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec00b0090 con 0x7f7ec015c390 2024-03-28T13:09:37.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.117+0000 7f7eb7400640 1 -- 172.21.15.19:0/1720112096 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8009450 con 0x7f7ec015c390 2024-03-28T13:09:37.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.117+0000 7f7ec6000640 1 -- 172.21.15.19:0/1720112096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec00b0650 con 0x7f7ec015c390 2024-03-28T13:09:37.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.119+0000 7f7ec6000640 1 -- 172.21.15.19:0/1720112096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec0070100 con 0x7f7ec015c390 2024-03-28T13:09:37.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.120+0000 7f7eb7400640 1 -- 172.21.15.19:0/1720112096 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7eb803e850 con 0x7f7ec015c390 2024-03-28T13:09:37.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.121+0000 7f7eb7400640 1 --2- 172.21.15.19:0/1720112096 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ea006c530 0x7f7ea006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:37.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.121+0000 7f7eb7400640 1 -- 172.21.15.19:0/1720112096 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f7eb80ae230 con 0x7f7ec015c390 2024-03-28T13:09:37.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.121+0000 7f7ebf400640 1 --2- 172.21.15.19:0/1720112096 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ea006c530 0x7f7ea006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:37.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.122+0000 7f7ebf400640 1 --2- 172.21.15.19:0/1720112096 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ea006c530 0x7f7ea006e9f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0001200 tx=0x7f7eb0000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:37.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.125+0000 7f7eb7400640 1 -- 172.21.15.19:0/1720112096 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7ec0070100 con 0x7f7ec015c390 2024-03-28T13:09:37.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.364+0000 7f7ec6000640 1 -- 172.21.15.19:0/1720112096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ec00685c0 con 0x7f7ec015c390 2024-03-28T13:09:37.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.365+0000 7f7eb7400640 1 -- 172.21.15.19:0/1720112096 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3861 (secure 0 0 0) 0x7f7eb807ccc0 con 0x7f7ec015c390 2024-03-28T13:09:37.365 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:37.365 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/745791856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":745791856},{"type":"v1","addr":"172.21.15.19:6833","nonce":745791856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/639283827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":639283827},{"type":"v1","addr":"172.21.15.188:6836","nonce":639283827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2168352329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2168352329},{"type":"v1","addr":"172.21.15.188:6837","nonce":2168352329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/731659331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":731659331},{"type":"v1","addr":"172.21.15.19:6835","nonce":731659331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-28T13:09:37.365 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 3 2024-03-28T13:09:37.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.367+0000 7f7ec6000640 1 -- 172.21.15.19:0/1720112096 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ea006c530 msgr2=0x7f7ea006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:37.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.367+0000 7f7ec6000640 1 --2- 172.21.15.19:0/1720112096 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ea006c530 0x7f7ea006e9f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0001200 tx=0x7f7eb0000bc0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.367+0000 7f7ec6000640 1 -- 172.21.15.19:0/1720112096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec015c390 msgr2=0x7f7ec0146640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:37.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.367+0000 7f7ec6000640 1 --2- 172.21.15.19:0/1720112096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec015c390 0x7f7ec0146640 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8009d90 tx=0x7f7eb8004a40 comp rx=0 tx=0).stop 2024-03-28T13:09:37.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.367+0000 7f7ec6000640 1 -- 172.21.15.19:0/1720112096 shutdown_connections 2024-03-28T13:09:37.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.367+0000 7f7ec6000640 1 --2- 172.21.15.19:0/1720112096 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ea006c530 0x7f7ea006e9f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.367+0000 7f7ec6000640 1 --2- 172.21.15.19:0/1720112096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ec015c390 0x7f7ec0146640 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.367+0000 7f7ec6000640 1 --2- 172.21.15.19:0/1720112096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ec00d0650 0x7f7ec0146100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.367+0000 7f7ec6000640 1 --2- 172.21.15.19:0/1720112096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ec00af830 0x7f7ec01576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.368+0000 7f7ec6000640 1 -- 172.21.15.19:0/1720112096 >> 172.21.15.19:0/1720112096 conn(0x7f7ec006b990 msgr2=0x7f7ec0154930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:37.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.368+0000 7f7ec6000640 1 -- 172.21.15.19:0/1720112096 shutdown_connections 2024-03-28T13:09:37.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.368+0000 7f7ec6000640 1 -- 172.21.15.19:0/1720112096 wait complete. 2024-03-28T13:09:37.385 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:09:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.590+0000 7ff5fd000640 1 Processor -- start 2024-03-28T13:09:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.590+0000 7ff5fd000640 1 -- start start 2024-03-28T13:09:37.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.593+0000 7ff5fd000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff5f815e540 0x7ff5f815f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:37.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.593+0000 7ff5fd000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5f80b41e0 con 0x7ff5f80b00a0 2024-03-28T13:09:37.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.593+0000 7ff5fd000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5f80b3c40 con 0x7ff5f80b4560 2024-03-28T13:09:37.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.593+0000 7ff5fd000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff5f80b3f00 con 0x7ff5f815e540 2024-03-28T13:09:37.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.593+0000 7ff5efe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff5f80b00a0 0x7ff5f8162b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59256/0 (socket says 172.21.15.19:59256) 2024-03-28T13:09:37.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.593+0000 7ff5efe00640 1 -- 172.21.15.19:0/1535345070 learned_addr learned my addr 172.21.15.19:0/1535345070 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:37.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.593+0000 7ff5f5e00640 1 --2- 172.21.15.19:0/1535345070 >> v2:172.21.15.19:3300/0 conn(0x7ff5f815e540 0x7ff5f815f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:37.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.594+0000 7ff5f5e00640 1 -- 172.21.15.19:0/1535345070 >> v1:172.21.15.188:6789/0 conn(0x7ff5f80b4560 legacy=0x7ff5f80b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:37.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.594+0000 7ff5f5e00640 1 -- 172.21.15.19:0/1535345070 >> v1:172.21.15.19:6789/0 conn(0x7ff5f80b00a0 legacy=0x7ff5f8162b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:37.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.594+0000 7ff5f5e00640 1 -- 172.21.15.19:0/1535345070 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f8162360 con 0x7ff5f815e540 2024-03-28T13:09:37.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.595+0000 7ff5f5e00640 1 --2- 172.21.15.19:0/1535345070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f815e540 0x7ff5f815f6e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80097a0 tx=0x7ff5e802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5520aa2852b122 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:37.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.595+0000 7ff5f5400640 1 -- 172.21.15.19:0/1535345070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8038070 con 0x7ff5f815e540 2024-03-28T13:09:37.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.596+0000 7ff5f5400640 1 -- 172.21.15.19:0/1535345070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e8031700 con 0x7ff5f815e540 2024-03-28T13:09:37.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.596+0000 7ff5f5400640 1 -- 172.21.15.19:0/1535345070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8031ad0 con 0x7ff5f815e540 2024-03-28T13:09:37.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.596+0000 7ff5fd000640 1 -- 172.21.15.19:0/1535345070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f815e540 msgr2=0x7ff5f815f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.596+0000 7ff5fd000640 1 --2- 172.21.15.19:0/1535345070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f815e540 0x7ff5f815f6e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80097a0 tx=0x7ff5e802b7a0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.596+0000 7ff5fd000640 1 -- 172.21.15.19:0/1535345070 shutdown_connections 2024-03-28T13:09:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.596+0000 7ff5fd000640 1 --2- 172.21.15.19:0/1535345070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f815e540 0x7ff5f815f6e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.596+0000 7ff5fd000640 1 -- 172.21.15.19:0/1535345070 >> 172.21.15.19:0/1535345070 conn(0x7ff5f80ce400 msgr2=0x7ff5f8071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.596+0000 7ff5fd000640 1 -- 172.21.15.19:0/1535345070 shutdown_connections 2024-03-28T13:09:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.596+0000 7ff5fd000640 1 -- 172.21.15.19:0/1535345070 wait complete. 2024-03-28T13:09:37.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.598+0000 7ff5fd000640 1 Processor -- start 2024-03-28T13:09:37.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.598+0000 7ff5fd000640 1 -- start start 2024-03-28T13:09:37.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.599+0000 7ff5fd000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5f80b00a0 0x7ff5f8148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:37.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.599+0000 7ff5fd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80b4560 0x7ff5f80b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:37.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.599+0000 7ff5fd000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5f815e540 0x7ff5f80b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:37.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.599+0000 7ff5fd000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f815eaa0 con 0x7ff5f80b4560 2024-03-28T13:09:37.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.599+0000 7ff5fd000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f815e920 con 0x7ff5f80b00a0 2024-03-28T13:09:37.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.599+0000 7ff5fd000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f806b570 con 0x7ff5f815e540 2024-03-28T13:09:37.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.599+0000 7ff5efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80b4560 0x7ff5f80b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:37.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.599+0000 7ff5efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80b4560 0x7ff5f80b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34050/0 (socket says 172.21.15.19:34050) 2024-03-28T13:09:37.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.599+0000 7ff5efe00640 1 -- 172.21.15.19:0/1593116381 learned_addr learned my addr 172.21.15.19:0/1593116381 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:37.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.600+0000 7ff5f5e00640 1 --2- 172.21.15.19:0/1593116381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5f80b00a0 0x7ff5f8148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:37.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.600+0000 7ff5f6800640 1 --2- 172.21.15.19:0/1593116381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5f815e540 0x7ff5f80b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:37.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.600+0000 7ff5efe00640 1 -- 172.21.15.19:0/1593116381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5f815e540 msgr2=0x7ff5f80b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:37.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.600+0000 7ff5efe00640 1 --2- 172.21.15.19:0/1593116381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5f815e540 0x7ff5f80b1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.600+0000 7ff5efe00640 1 -- 172.21.15.19:0/1593116381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5f80b00a0 msgr2=0x7ff5f8148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:37.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.600+0000 7ff5efe00640 1 --2- 172.21.15.19:0/1593116381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5f80b00a0 0x7ff5f8148500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.600+0000 7ff5efe00640 1 -- 172.21.15.19:0/1593116381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f8158e50 con 0x7ff5f80b4560 2024-03-28T13:09:37.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.600+0000 7ff5f5e00640 1 --2- 172.21.15.19:0/1593116381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5f80b00a0 0x7ff5f8148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:37.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.600+0000 7ff5efe00640 1 --2- 172.21.15.19:0/1593116381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80b4560 0x7ff5f80b0fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000c430 tx=0x7ff5e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:37.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.600+0000 7ff5ee000640 1 -- 172.21.15.19:0/1593116381 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0017070 con 0x7ff5f80b4560 2024-03-28T13:09:37.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.600+0000 7ff5ee000640 1 -- 172.21.15.19:0/1593116381 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e0005ce0 con 0x7ff5f80b4560 2024-03-28T13:09:37.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.600+0000 7ff5fd000640 1 -- 172.21.15.19:0/1593116381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f815f8f0 con 0x7ff5f80b4560 2024-03-28T13:09:37.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.600+0000 7ff5ee000640 1 -- 172.21.15.19:0/1593116381 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00072f0 con 0x7ff5f80b4560 2024-03-28T13:09:37.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.600+0000 7ff5fd000640 1 -- 172.21.15.19:0/1593116381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f815feb0 con 0x7ff5f80b4560 2024-03-28T13:09:37.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.602+0000 7ff5fd000640 1 -- 172.21.15.19:0/1593116381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f8068cd0 con 0x7ff5f80b4560 2024-03-28T13:09:37.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.602+0000 7ff5ee000640 1 -- 172.21.15.19:0/1593116381 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff5e001a070 con 0x7ff5f80b4560 2024-03-28T13:09:37.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.603+0000 7ff5ee000640 1 --2- 172.21.15.19:0/1593116381 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff5d406c530 0x7ff5d406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:37.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.603+0000 7ff5ee000640 1 -- 172.21.15.19:0/1593116381 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7ff5e008a7a0 con 0x7ff5f80b4560 2024-03-28T13:09:37.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.604+0000 7ff5f5e00640 1 --2- 172.21.15.19:0/1593116381 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff5d406c530 0x7ff5d406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:37.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.605+0000 7ff5f5e00640 1 --2- 172.21.15.19:0/1593116381 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff5d406c530 0x7ff5d406e9f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8035000 tx=0x7ff5e8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:37.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.608+0000 7ff5ee000640 1 -- 172.21.15.19:0/1593116381 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff5e0059100 con 0x7ff5f80b4560 2024-03-28T13:09:37.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.810+0000 7ff5fd000640 1 -- 172.21.15.19:0/1593116381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5f8160080 con 0x7ff5f80b4560 2024-03-28T13:09:37.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.812+0000 7ff5ee000640 1 -- 172.21.15.19:0/1593116381 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12884 (secure 0 0 0) 0x7ff5e0058a00 con 0x7ff5f80b4560 2024-03-28T13:09:37.813 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:37.813 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":19,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:32.463538+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:09:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.815+0000 7ff5fd000640 1 -- 172.21.15.19:0/1593116381 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff5d406c530 msgr2=0x7ff5d406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.815+0000 7ff5fd000640 1 --2- 172.21.15.19:0/1593116381 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff5d406c530 0x7ff5d406e9f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8035000 tx=0x7ff5e8008030 comp rx=0 tx=0).stop 2024-03-28T13:09:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.815+0000 7ff5fd000640 1 -- 172.21.15.19:0/1593116381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80b4560 msgr2=0x7ff5f80b0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.815+0000 7ff5fd000640 1 --2- 172.21.15.19:0/1593116381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80b4560 0x7ff5f80b0fc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000c430 tx=0x7ff5e000c8f0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.815+0000 7ff5fd000640 1 -- 172.21.15.19:0/1593116381 shutdown_connections 2024-03-28T13:09:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.815+0000 7ff5fd000640 1 --2- 172.21.15.19:0/1593116381 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff5d406c530 0x7ff5d406e9f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.815+0000 7ff5fd000640 1 --2- 172.21.15.19:0/1593116381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5f815e540 0x7ff5f80b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.815+0000 7ff5fd000640 1 --2- 172.21.15.19:0/1593116381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80b4560 0x7ff5f80b0fc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.815+0000 7ff5fd000640 1 --2- 172.21.15.19:0/1593116381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5f80b00a0 0x7ff5f8148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.815+0000 7ff5fd000640 1 -- 172.21.15.19:0/1593116381 >> 172.21.15.19:0/1593116381 conn(0x7ff5f80ce400 msgr2=0x7ff5f8072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.815+0000 7ff5fd000640 1 -- 172.21.15.19:0/1593116381 shutdown_connections 2024-03-28T13:09:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:37.815+0000 7ff5fd000640 1 -- 172.21.15.19:0/1593116381 wait complete. 2024-03-28T13:09:37.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:09:37.835 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:09:38.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.054+0000 7f460ac00640 1 Processor -- start 2024-03-28T13:09:38.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.055+0000 7f460ac00640 1 -- start start 2024-03-28T13:09:38.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.056+0000 7f460ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4604155550 0x7f4604155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:38.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.056+0000 7f460ac00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f460415f800 0x7f46040d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:38.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.057+0000 7f460ac00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46040b40b0 con 0x7f46040d07c0 2024-03-28T13:09:38.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.057+0000 7f460ac00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f46040d0c30 con 0x7f460415f800 2024-03-28T13:09:38.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.057+0000 7f460ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f46040b4390 con 0x7f4604155550 2024-03-28T13:09:38.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.057+0000 7f4603400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f460415f800 0x7f46040d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:38.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.057+0000 7f4603400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f460415f800 0x7f46040d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34052/0 (socket says 172.21.15.19:34052) 2024-03-28T13:09:38.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.057+0000 7f4603400640 1 -- 172.21.15.19:0/1940900668 learned_addr learned my addr 172.21.15.19:0/1940900668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:38.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.057+0000 7f4603e00640 1 --2- 172.21.15.19:0/1940900668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4604155550 0x7f4604155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:38.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.057+0000 7f4603400640 1 -- 172.21.15.19:0/1940900668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4604155550 msgr2=0x7f4604155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:38.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.057+0000 7f4603400640 1 --2- 172.21.15.19:0/1940900668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4604155550 0x7f4604155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.057+0000 7f4603400640 1 -- 172.21.15.19:0/1940900668 >> v1:172.21.15.188:6789/0 conn(0x7f46040d07c0 legacy=0x7f46040aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:38.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.057+0000 7f4603400640 1 -- 172.21.15.19:0/1940900668 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46040b4670 con 0x7f460415f800 2024-03-28T13:09:38.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.059+0000 7f4603400640 1 --2- 172.21.15.19:0/1940900668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460415f800 0x7f46040d0280 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009e50 tx=0x7f45ec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cc7a0a45901487 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:38.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.059+0000 7f4602a00640 1 -- 172.21.15.19:0/1940900668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec02ca70 con 0x7f460415f800 2024-03-28T13:09:38.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.059+0000 7f4602a00640 1 -- 172.21.15.19:0/1940900668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45ec02cc10 con 0x7f460415f800 2024-03-28T13:09:38.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.059+0000 7f4602a00640 1 -- 172.21.15.19:0/1940900668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0343f0 con 0x7f460415f800 2024-03-28T13:09:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.060+0000 7f460ac00640 1 -- 172.21.15.19:0/1940900668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460415f800 msgr2=0x7f46040d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.060+0000 7f460ac00640 1 --2- 172.21.15.19:0/1940900668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460415f800 0x7f46040d0280 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009e50 tx=0x7f45ec02bed0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.060+0000 7f460ac00640 1 -- 172.21.15.19:0/1940900668 shutdown_connections 2024-03-28T13:09:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.060+0000 7f460ac00640 1 --2- 172.21.15.19:0/1940900668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460415f800 0x7f46040d0280 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.060+0000 7f460ac00640 1 --2- 172.21.15.19:0/1940900668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4604155550 0x7f4604155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.060+0000 7f460ac00640 1 -- 172.21.15.19:0/1940900668 >> 172.21.15.19:0/1940900668 conn(0x7f460406b750 msgr2=0x7f460406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.060+0000 7f460ac00640 1 -- 172.21.15.19:0/1940900668 shutdown_connections 2024-03-28T13:09:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.060+0000 7f460ac00640 1 -- 172.21.15.19:0/1940900668 wait complete. 2024-03-28T13:09:38.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.062+0000 7f460ac00640 1 Processor -- start 2024-03-28T13:09:38.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.062+0000 7f460ac00640 1 -- start start 2024-03-28T13:09:38.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.063+0000 7f460ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46040d07c0 0x7f4604099820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:38.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.063+0000 7f460ac00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4604155550 0x7f46040ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:38.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.063+0000 7f460ac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460415f800 0x7f46040720c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:38.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.063+0000 7f460ac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f460415fc80 con 0x7f460415f800 2024-03-28T13:09:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.063+0000 7f460ac00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f46040b4c30 con 0x7f4604155550 2024-03-28T13:09:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.063+0000 7f460ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f460415ba20 con 0x7f46040d07c0 2024-03-28T13:09:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.064+0000 7f4608c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460415f800 0x7f46040720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.064+0000 7f4608c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460415f800 0x7f46040720c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34062/0 (socket says 172.21.15.19:34062) 2024-03-28T13:09:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.064+0000 7f4608c00640 1 -- 172.21.15.19:0/603521664 learned_addr learned my addr 172.21.15.19:0/603521664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.064+0000 7f4603e00640 1 --2- 172.21.15.19:0/603521664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46040d07c0 0x7f4604099820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.064+0000 7f4603400640 1 --2- 172.21.15.19:0/603521664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4604155550 0x7f46040ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.064+0000 7f4608c00640 1 -- 172.21.15.19:0/603521664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46040d07c0 msgr2=0x7f4604099820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.064+0000 7f4608c00640 1 --2- 172.21.15.19:0/603521664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46040d07c0 0x7f4604099820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.064+0000 7f4608c00640 1 -- 172.21.15.19:0/603521664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4604155550 msgr2=0x7f46040ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.064+0000 7f4608c00640 1 --2- 172.21.15.19:0/603521664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4604155550 0x7f46040ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.064+0000 7f4608c00640 1 -- 172.21.15.19:0/603521664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46040ced40 con 0x7f460415f800 2024-03-28T13:09:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.064+0000 7f4608c00640 1 --2- 172.21.15.19:0/603521664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460415f800 0x7f46040720c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f45f400b410 tx=0x7f45f400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:38.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.065+0000 7f45fbe00640 1 -- 172.21.15.19:0/603521664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4015030 con 0x7f460415f800 2024-03-28T13:09:38.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.065+0000 7f45fbe00640 1 -- 172.21.15.19:0/603521664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f4004020 con 0x7f460415f800 2024-03-28T13:09:38.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.065+0000 7f45fbe00640 1 -- 172.21.15.19:0/603521664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4004410 con 0x7f460415f800 2024-03-28T13:09:38.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.065+0000 7f460ac00640 1 -- 172.21.15.19:0/603521664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46040cf020 con 0x7f460415f800 2024-03-28T13:09:38.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.065+0000 7f460ac00640 1 -- 172.21.15.19:0/603521664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46040cf5e0 con 0x7f460415f800 2024-03-28T13:09:38.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.066+0000 7f460ac00640 1 -- 172.21.15.19:0/603521664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46040685c0 con 0x7f460415f800 2024-03-28T13:09:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.067+0000 7f45fbe00640 1 -- 172.21.15.19:0/603521664 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f45f4018070 con 0x7f460415f800 2024-03-28T13:09:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.068+0000 7f45fbe00640 1 --2- 172.21.15.19:0/603521664 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f45e006c530 0x7f45e006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.068+0000 7f45fbe00640 1 -- 172.21.15.19:0/603521664 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f45f408aec0 con 0x7f460415f800 2024-03-28T13:09:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.069+0000 7f4603e00640 1 --2- 172.21.15.19:0/603521664 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f45e006c530 0x7f45e006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.070+0000 7f4603e00640 1 --2- 172.21.15.19:0/603521664 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f45e006c530 0x7f45e006e9f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f45fc006fd0 tx=0x7f45fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.074+0000 7f45fbe00640 1 -- 172.21.15.19:0/603521664 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f45f4059820 con 0x7f460415f800 2024-03-28T13:09:38.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.280+0000 7f460ac00640 1 -- 172.21.15.19:0/603521664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4604099e70 con 0x7f460415f800 2024-03-28T13:09:38.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.281+0000 7f45fbe00640 1 -- 172.21.15.19:0/603521664 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7f45f4059120 con 0x7f460415f800 2024-03-28T13:09:38.281 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:09:38.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.284+0000 7f460ac00640 1 -- 172.21.15.19:0/603521664 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f45e006c530 msgr2=0x7f45e006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:38.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.284+0000 7f460ac00640 1 --2- 172.21.15.19:0/603521664 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f45e006c530 0x7f45e006e9f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f45fc006fd0 tx=0x7f45fc008040 comp rx=0 tx=0).stop 2024-03-28T13:09:38.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.284+0000 7f460ac00640 1 -- 172.21.15.19:0/603521664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460415f800 msgr2=0x7f46040720c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:38.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.284+0000 7f460ac00640 1 --2- 172.21.15.19:0/603521664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460415f800 0x7f46040720c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f45f400b410 tx=0x7f45f400b8d0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.284+0000 7f460ac00640 1 -- 172.21.15.19:0/603521664 shutdown_connections 2024-03-28T13:09:38.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.284+0000 7f460ac00640 1 --2- 172.21.15.19:0/603521664 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f45e006c530 0x7f45e006e9f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.284+0000 7f460ac00640 1 --2- 172.21.15.19:0/603521664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f460415f800 0x7f46040720c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.284+0000 7f460ac00640 1 --2- 172.21.15.19:0/603521664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4604155550 0x7f46040ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.284+0000 7f460ac00640 1 --2- 172.21.15.19:0/603521664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46040d07c0 0x7f4604099820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.284+0000 7f460ac00640 1 -- 172.21.15.19:0/603521664 >> 172.21.15.19:0/603521664 conn(0x7f460406b750 msgr2=0x7f460406e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:38.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.284+0000 7f460ac00640 1 -- 172.21.15.19:0/603521664 shutdown_connections 2024-03-28T13:09:38.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.284+0000 7f460ac00640 1 -- 172.21.15.19:0/603521664 wait complete. 2024-03-28T13:09:38.302 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-28T13:09:38.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.527+0000 7f162d000640 1 Processor -- start 2024-03-28T13:09:38.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.528+0000 7f162d000640 1 -- start start 2024-03-28T13:09:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.529+0000 7f162d000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f162806b570 0x7f162806fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.529+0000 7f162d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f162815fed0 0x7f162815d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.529+0000 7f162d000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1628145f50 con 0x7f162806b950 2024-03-28T13:09:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.529+0000 7f162d000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f162815d920 con 0x7f162815fed0 2024-03-28T13:09:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.529+0000 7f162d000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f16281603d0 con 0x7f162806b570 2024-03-28T13:09:38.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.529+0000 7f1626a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f162815fed0 0x7f162815d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:38.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.529+0000 7f1627e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f162806b950 0x7f16280d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59258/0 (socket says 172.21.15.19:59258) 2024-03-28T13:09:38.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.529+0000 7f1627e00640 1 -- 172.21.15.19:0/2529481005 learned_addr learned my addr 172.21.15.19:0/2529481005 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:38.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.529+0000 7f1627400640 1 --2- 172.21.15.19:0/2529481005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f162806b570 0x7f162806fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:38.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.530+0000 7f1626a00640 1 -- 172.21.15.19:0/2529481005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f162806b570 msgr2=0x7f162806fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:38.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.530+0000 7f1626a00640 1 --2- 172.21.15.19:0/2529481005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f162806b570 0x7f162806fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.530+0000 7f1626a00640 1 -- 172.21.15.19:0/2529481005 >> v1:172.21.15.19:6789/0 conn(0x7f162806b950 legacy=0x7f16280d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:38.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.530+0000 7f1626000640 1 -- 172.21.15.19:0/2529481005 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3027639728 0 0) 0x7f1628145f50 con 0x7f162806b950 2024-03-28T13:09:38.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.530+0000 7f1626a00640 1 -- 172.21.15.19:0/2529481005 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16281606b0 con 0x7f162815fed0 2024-03-28T13:09:38.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.531+0000 7f1626a00640 1 --2- 172.21.15.19:0/2529481005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162815fed0 0x7f162815d3e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1618009dc0 tx=0x7f161802be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e25d94d35238b852 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:38.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.531+0000 7f1626000640 1 -- 172.21.15.19:0/2529481005 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618039070 con 0x7f162815fed0 2024-03-28T13:09:38.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.531+0000 7f1626000640 1 -- 172.21.15.19:0/2529481005 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161802cbb0 con 0x7f162815fed0 2024-03-28T13:09:38.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.531+0000 7f1626000640 1 -- 172.21.15.19:0/2529481005 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618034440 con 0x7f162815fed0 2024-03-28T13:09:38.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.532+0000 7f162d000640 1 -- 172.21.15.19:0/2529481005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162815fed0 msgr2=0x7f162815d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:38.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.532+0000 7f162d000640 1 --2- 172.21.15.19:0/2529481005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162815fed0 0x7f162815d3e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1618009dc0 tx=0x7f161802be00 comp rx=0 tx=0).stop 2024-03-28T13:09:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.532+0000 7f162d000640 1 -- 172.21.15.19:0/2529481005 shutdown_connections 2024-03-28T13:09:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.532+0000 7f162d000640 1 --2- 172.21.15.19:0/2529481005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162815fed0 0x7f162815d3e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.532+0000 7f162d000640 1 --2- 172.21.15.19:0/2529481005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f162806b570 0x7f162806fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.532+0000 7f162d000640 1 -- 172.21.15.19:0/2529481005 >> 172.21.15.19:0/2529481005 conn(0x7f16280d1090 msgr2=0x7f16280d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.532+0000 7f162d000640 1 -- 172.21.15.19:0/2529481005 shutdown_connections 2024-03-28T13:09:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.532+0000 7f162d000640 1 -- 172.21.15.19:0/2529481005 wait complete. 2024-03-28T13:09:38.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.534+0000 7f162d000640 1 Processor -- start 2024-03-28T13:09:38.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.534+0000 7f162d000640 1 -- start start 2024-03-28T13:09:38.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.535+0000 7f162d000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f162806b570 0x7f16280ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:38.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.535+0000 7f162d000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f162806b950 0x7f16280b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:38.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.535+0000 7f162d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162815fed0 0x7f16280aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:38.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.535+0000 7f162d000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16280cfc20 con 0x7f162815fed0 2024-03-28T13:09:38.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.535+0000 7f162d000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1628160c70 con 0x7f162806b950 2024-03-28T13:09:38.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.535+0000 7f162d000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f162806bde0 con 0x7f162806b570 2024-03-28T13:09:38.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.535+0000 7f1627e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162815fed0 0x7f16280aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:38.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.536+0000 7f1627e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162815fed0 0x7f16280aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34092/0 (socket says 172.21.15.19:34092) 2024-03-28T13:09:38.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.536+0000 7f1627e00640 1 -- 172.21.15.19:0/3178284819 learned_addr learned my addr 172.21.15.19:0/3178284819 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:38.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.536+0000 7f1627400640 1 --2- 172.21.15.19:0/3178284819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f162806b570 0x7f16280ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:38.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.536+0000 7f1627e00640 1 -- 172.21.15.19:0/3178284819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f162806b570 msgr2=0x7f16280ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:38.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.536+0000 7f1627e00640 1 --2- 172.21.15.19:0/3178284819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f162806b570 0x7f16280ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.536+0000 7f1627e00640 1 -- 172.21.15.19:0/3178284819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f162806b950 msgr2=0x7f16280b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:38.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.536+0000 7f1626a00640 1 --2- 172.21.15.19:0/3178284819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f162806b950 0x7f16280b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:38.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.536+0000 7f1627e00640 1 --2- 172.21.15.19:0/3178284819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f162806b950 0x7f16280b5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.536+0000 7f1627e00640 1 -- 172.21.15.19:0/3178284819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16281411d0 con 0x7f162815fed0 2024-03-28T13:09:38.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.536+0000 7f1627e00640 1 --2- 172.21.15.19:0/3178284819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162815fed0 0x7f16280aeea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f162000c480 tx=0x7f162000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:38.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.536+0000 7f161f400640 1 -- 172.21.15.19:0/3178284819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620018070 con 0x7f162815fed0 2024-03-28T13:09:38.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.537+0000 7f161f400640 1 -- 172.21.15.19:0/3178284819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1620007500 con 0x7f162815fed0 2024-03-28T13:09:38.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.537+0000 7f162d000640 1 -- 172.21.15.19:0/3178284819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16280afa80 con 0x7f162815fed0 2024-03-28T13:09:38.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.537+0000 7f161f400640 1 -- 172.21.15.19:0/3178284819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1620004a20 con 0x7f162815fed0 2024-03-28T13:09:38.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.537+0000 7f162d000640 1 -- 172.21.15.19:0/3178284819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16280add10 con 0x7f162815fed0 2024-03-28T13:09:38.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.538+0000 7f162d000640 1 -- 172.21.15.19:0/3178284819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1628075940 con 0x7f162815fed0 2024-03-28T13:09:38.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.539+0000 7f161f400640 1 -- 172.21.15.19:0/3178284819 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f162001b070 con 0x7f162815fed0 2024-03-28T13:09:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.540+0000 7f161f400640 1 --2- 172.21.15.19:0/3178284819 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160406c530 0x7f160406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.540+0000 7f161f400640 1 -- 172.21.15.19:0/3178284819 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f162008a660 con 0x7f162815fed0 2024-03-28T13:09:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.540+0000 7f1627400640 1 --2- 172.21.15.19:0/3178284819 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160406c530 0x7f160406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:38.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.541+0000 7f1627400640 1 --2- 172.21.15.19:0/3178284819 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160406c530 0x7f160406e9f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f162815fa50 tx=0x7f1610008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:38.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.544+0000 7f161f400640 1 -- 172.21.15.19:0/3178284819 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f16281411d0 con 0x7f162815fed0 2024-03-28T13:09:38.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.750+0000 7f162d000640 1 -- 172.21.15.19:0/3178284819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f16280d1e60 con 0x7f162815fed0 2024-03-28T13:09:38.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.751+0000 7f161f400640 1 -- 172.21.15.19:0/3178284819 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7f1620058fc0 con 0x7f162815fed0 2024-03-28T13:09:38.751 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:38.751 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-03-28T13:09:38.751 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:09:38.751 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-03-28T13:09:38.752 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-03-28T13:09:38.752 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:09:38.752 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-03-28T13:09:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.753+0000 7f162d000640 1 -- 172.21.15.19:0/3178284819 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160406c530 msgr2=0x7f160406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.753+0000 7f162d000640 1 --2- 172.21.15.19:0/3178284819 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160406c530 0x7f160406e9f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f162815fa50 tx=0x7f1610008040 comp rx=0 tx=0).stop 2024-03-28T13:09:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.753+0000 7f162d000640 1 -- 172.21.15.19:0/3178284819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162815fed0 msgr2=0x7f16280aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.753+0000 7f162d000640 1 --2- 172.21.15.19:0/3178284819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162815fed0 0x7f16280aeea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f162000c480 tx=0x7f162000c940 comp rx=0 tx=0).stop 2024-03-28T13:09:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.754+0000 7f162d000640 1 -- 172.21.15.19:0/3178284819 shutdown_connections 2024-03-28T13:09:38.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.754+0000 7f162d000640 1 --2- 172.21.15.19:0/3178284819 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160406c530 0x7f160406e9f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.754+0000 7f162d000640 1 --2- 172.21.15.19:0/3178284819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f162815fed0 0x7f16280aeea0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.754+0000 7f162d000640 1 --2- 172.21.15.19:0/3178284819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f162806b950 0x7f16280b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.754+0000 7f162d000640 1 --2- 172.21.15.19:0/3178284819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f162806b570 0x7f16280ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.754+0000 7f162d000640 1 -- 172.21.15.19:0/3178284819 >> 172.21.15.19:0/3178284819 conn(0x7f16280d1090 msgr2=0x7f162806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.754+0000 7f162d000640 1 -- 172.21.15.19:0/3178284819 shutdown_connections 2024-03-28T13:09:38.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:38.754+0000 7f162d000640 1 -- 172.21.15.19:0/3178284819 wait complete. 2024-03-28T13:09:38.772 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-28T13:09:38.772 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-28T13:09:39.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.000+0000 7f41c3e00640 1 Processor -- start 2024-03-28T13:09:39.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.000+0000 7f41c3e00640 1 -- start start 2024-03-28T13:09:39.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.002+0000 7f41c3e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f41c4141d20 0x7f41c406e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:39.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.002+0000 7f41c3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41c40cfbf0 con 0x7f41c4141890 2024-03-28T13:09:39.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.002+0000 7f41c3e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41c40cf350 con 0x7f41c413f190 2024-03-28T13:09:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.002+0000 7f41c3e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f41c413f570 con 0x7f41c4141d20 2024-03-28T13:09:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.002+0000 7f41c2000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f41c4141890 0x7f41c406db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59264/0 (socket says 172.21.15.19:59264) 2024-03-28T13:09:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.002+0000 7f41c2000640 1 -- 172.21.15.19:0/74847851 learned_addr learned my addr 172.21.15.19:0/74847851 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:39.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.003+0000 7f41c3400640 1 --2- 172.21.15.19:0/74847851 >> v2:172.21.15.188:3300/0 conn(0x7f41c4141d20 0x7f41c406e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:39.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.003+0000 7f41c1600640 1 -- 172.21.15.19:0/74847851 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2784220436 0 0) 0x7f41c40cfbf0 con 0x7f41c4141890 2024-03-28T13:09:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.003+0000 7f41c1600640 1 -- 172.21.15.19:0/74847851 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41a8003520 con 0x7f41c4141890 2024-03-28T13:09:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.003+0000 7f41c1600640 1 -- 172.21.15.19:0/74847851 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1708524669 0 0) 0x7f41a8003520 con 0x7f41c4141890 2024-03-28T13:09:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.003+0000 7f41c1600640 1 -- 172.21.15.19:0/74847851 >> v2:172.21.15.188:3300/0 conn(0x7f41c4141d20 msgr2=0x7f41c406e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.003+0000 7f41c1600640 1 --2- 172.21.15.19:0/74847851 >> v2:172.21.15.188:3300/0 conn(0x7f41c4141d20 0x7f41c406e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.003+0000 7f41c1600640 1 -- 172.21.15.19:0/74847851 >> v1:172.21.15.188:6789/0 conn(0x7f41c413f190 legacy=0x7f41c40ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.003+0000 7f41c1600640 1 -- 172.21.15.19:0/74847851 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c40cf630 con 0x7f41c4141890 2024-03-28T13:09:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.003+0000 7f41c1600640 1 -- 172.21.15.19:0/74847851 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f41ac001670 con 0x7f41c4141890 2024-03-28T13:09:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.003+0000 7f41c1600640 1 -- 172.21.15.19:0/74847851 >> v1:172.21.15.19:6789/0 conn(0x7f41c4141890 legacy=0x7f41c406db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.003+0000 7f41c1600640 1 --2- 172.21.15.19:0/74847851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41a80039b0 0x7f41a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.004+0000 7f41c1600640 1 --2- 172.21.15.19:0/74847851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41a8004800 0x7f41a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.004+0000 7f41c1600640 1 --2- 172.21.15.19:0/74847851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a80051c0 0x7f41a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.004+0000 7f41c1600640 1 -- 172.21.15.19:0/74847851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41a800c030 con 0x7f41a80039b0 2024-03-28T13:09:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.004+0000 7f41c1600640 1 -- 172.21.15.19:0/74847851 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41a800c1e0 con 0x7f41a8004800 2024-03-28T13:09:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.004+0000 7f41c1600640 1 -- 172.21.15.19:0/74847851 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f41a800c3e0 con 0x7f41a80051c0 2024-03-28T13:09:39.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.004+0000 7f41c2a00640 1 --2- 172.21.15.19:0/74847851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41a80039b0 0x7f41a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:39.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.004+0000 7f41c2000640 1 --2- 172.21.15.19:0/74847851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41a8004800 0x7f41a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:39.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.004+0000 7f41c3400640 1 --2- 172.21.15.19:0/74847851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a80051c0 0x7f41a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:39.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.004+0000 7f41c2a00640 1 -- 172.21.15.19:0/74847851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a80051c0 msgr2=0x7f41a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:39.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.004+0000 7f41c2a00640 1 --2- 172.21.15.19:0/74847851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a80051c0 0x7f41a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:39.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.004+0000 7f41c2a00640 1 -- 172.21.15.19:0/74847851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41a8004800 msgr2=0x7f41a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:39.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.004+0000 7f41c2a00640 1 --2- 172.21.15.19:0/74847851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41a8004800 0x7f41a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:39.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.004+0000 7f41c2a00640 1 -- 172.21.15.19:0/74847851 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a8003520 con 0x7f41a80039b0 2024-03-28T13:09:39.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.004+0000 7f41c3400640 1 --2- 172.21.15.19:0/74847851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a80051c0 0x7f41a800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:39.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.004+0000 7f41c3400640 1 -- 172.21.15.19:0/74847851 reap_dead start 2024-03-28T13:09:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.006+0000 7f41c2a00640 1 --2- 172.21.15.19:0/74847851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41a80039b0 0x7f41a8003dd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f41b4001750 tx=0x7f41a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec72c745155ec2e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.006+0000 7f41c1600640 1 -- 172.21.15.19:0/74847851 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4001370 con 0x7f41a80039b0 2024-03-28T13:09:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.006+0000 7f41c1600640 1 -- 172.21.15.19:0/74847851 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b40052d0 con 0x7f41a80039b0 2024-03-28T13:09:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.006+0000 7f41c1600640 1 -- 172.21.15.19:0/74847851 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4003830 con 0x7f41a80039b0 2024-03-28T13:09:39.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.007+0000 7f41c3e00640 1 -- 172.21.15.19:0/74847851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41a80039b0 msgr2=0x7f41a8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:39.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.007+0000 7f41c3e00640 1 --2- 172.21.15.19:0/74847851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41a80039b0 0x7f41a8003dd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f41b4001750 tx=0x7f41a8005830 comp rx=0 tx=0).stop 2024-03-28T13:09:39.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.007+0000 7f41c3e00640 1 -- 172.21.15.19:0/74847851 shutdown_connections 2024-03-28T13:09:39.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.007+0000 7f41c3e00640 1 --2- 172.21.15.19:0/74847851 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41a80039b0 0x7f41a8003dd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:39.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.007+0000 7f41c3e00640 1 -- 172.21.15.19:0/74847851 >> 172.21.15.19:0/74847851 conn(0x7f41c40b1d80 msgr2=0x7f41c40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:39.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.007+0000 7f41c3e00640 1 -- 172.21.15.19:0/74847851 shutdown_connections 2024-03-28T13:09:39.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.007+0000 7f41c3e00640 1 -- 172.21.15.19:0/74847851 wait complete. 2024-03-28T13:09:39.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.009+0000 7f41c3e00640 1 Processor -- start 2024-03-28T13:09:39.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.009+0000 7f41c3e00640 1 -- start start 2024-03-28T13:09:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.010+0000 7f41c3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a80039b0 0x7f41c413fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.010+0000 7f41c3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c4073020 0x7f41c413ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.010+0000 7f41c3e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41c413b170 0x7f41c413b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.010+0000 7f41c3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41c4142130 con 0x7f41c4073020 2024-03-28T13:09:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.010+0000 7f41c3e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41c406d670 con 0x7f41c413b170 2024-03-28T13:09:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.010+0000 7f41c3e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f41c4142550 con 0x7f41a80039b0 2024-03-28T13:09:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.010+0000 7f41c2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c4073020 0x7f41c413ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:39.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.010+0000 7f41c2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c4073020 0x7f41c413ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34114/0 (socket says 172.21.15.19:34114) 2024-03-28T13:09:39.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.010+0000 7f41c2000640 1 -- 172.21.15.19:0/2728894301 learned_addr learned my addr 172.21.15.19:0/2728894301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:39.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.011+0000 7f41c2a00640 1 --2- 172.21.15.19:0/2728894301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a80039b0 0x7f41c413fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:39.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.011+0000 7f41c2000640 1 -- 172.21.15.19:0/2728894301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a80039b0 msgr2=0x7f41c413fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:39.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.011+0000 7f41c2000640 1 --2- 172.21.15.19:0/2728894301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a80039b0 0x7f41c413fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:39.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.011+0000 7f41c2000640 1 -- 172.21.15.19:0/2728894301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41c413b170 msgr2=0x7f41c413b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:39.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.011+0000 7f41c2000640 1 --2- 172.21.15.19:0/2728894301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41c413b170 0x7f41c413b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:39.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.011+0000 7f41c2000640 1 -- 172.21.15.19:0/2728894301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c4071cb0 con 0x7f41c4073020 2024-03-28T13:09:39.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.011+0000 7f41c2000640 1 --2- 172.21.15.19:0/2728894301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c4073020 0x7f41c413ac30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00e7e0 tx=0x7f41ac00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.012+0000 7f41baa00640 1 -- 172.21.15.19:0/2728894301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac00c640 con 0x7f41c4073020 2024-03-28T13:09:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.012+0000 7f41baa00640 1 -- 172.21.15.19:0/2728894301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ac018070 con 0x7f41c4073020 2024-03-28T13:09:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.012+0000 7f41baa00640 1 -- 172.21.15.19:0/2728894301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac00ca40 con 0x7f41c4073020 2024-03-28T13:09:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.012+0000 7f41c3e00640 1 -- 172.21.15.19:0/2728894301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c4071f70 con 0x7f41c4073020 2024-03-28T13:09:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.012+0000 7f41c3e00640 1 -- 172.21.15.19:0/2728894301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c4072530 con 0x7f41c4073020 2024-03-28T13:09:39.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.013+0000 7f41c3e00640 1 -- 172.21.15.19:0/2728894301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c4075940 con 0x7f41c4073020 2024-03-28T13:09:39.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.014+0000 7f41baa00640 1 -- 172.21.15.19:0/2728894301 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f41ac00cbe0 con 0x7f41c4073020 2024-03-28T13:09:39.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.015+0000 7f41baa00640 1 --2- 172.21.15.19:0/2728894301 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41a006c530 0x7f41a006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:39.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.016+0000 7f41baa00640 1 -- 172.21.15.19:0/2728894301 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f41ac08b0e0 con 0x7f41c4073020 2024-03-28T13:09:39.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.016+0000 7f41c2a00640 1 --2- 172.21.15.19:0/2728894301 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41a006c530 0x7f41a006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:39.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.017+0000 7f41c2a00640 1 --2- 172.21.15.19:0/2728894301 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41a006c530 0x7f41a006e9f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f41b4033000 tx=0x7f41b4009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:39.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.020+0000 7f41baa00640 1 -- 172.21.15.19:0/2728894301 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f41ac059a40 con 0x7f41c4073020 2024-03-28T13:09:39.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.226+0000 7f41c3e00640 1 -- 172.21.15.19:0/2728894301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f41c40b3710 con 0x7f41c4073020 2024-03-28T13:09:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.606+0000 7f41baa00640 1 -- 172.21.15.19:0/2728894301 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7f41ac059340 con 0x7f41c4073020 2024-03-28T13:09:39.607 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-03-28T13:09:39.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.611+0000 7f41c3e00640 1 -- 172.21.15.19:0/2728894301 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41a006c530 msgr2=0x7f41a006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:39.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.611+0000 7f41c3e00640 1 --2- 172.21.15.19:0/2728894301 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41a006c530 0x7f41a006e9f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f41b4033000 tx=0x7f41b4009030 comp rx=0 tx=0).stop 2024-03-28T13:09:39.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.612+0000 7f41c3e00640 1 -- 172.21.15.19:0/2728894301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c4073020 msgr2=0x7f41c413ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:39.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.612+0000 7f41c3e00640 1 --2- 172.21.15.19:0/2728894301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c4073020 0x7f41c413ac30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f41ac00e7e0 tx=0x7f41ac00eca0 comp rx=0 tx=0).stop 2024-03-28T13:09:39.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.612+0000 7f41c3e00640 1 -- 172.21.15.19:0/2728894301 shutdown_connections 2024-03-28T13:09:39.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.612+0000 7f41c3e00640 1 --2- 172.21.15.19:0/2728894301 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41a006c530 0x7f41a006e9f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:39.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.612+0000 7f41c3e00640 1 --2- 172.21.15.19:0/2728894301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41c413b170 0x7f41c413b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:39.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.612+0000 7f41c3e00640 1 --2- 172.21.15.19:0/2728894301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c4073020 0x7f41c413ac30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:39.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.612+0000 7f41c3e00640 1 --2- 172.21.15.19:0/2728894301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41a80039b0 0x7f41c413fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:39.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.612+0000 7f41c3e00640 1 -- 172.21.15.19:0/2728894301 >> 172.21.15.19:0/2728894301 conn(0x7f41c40b1d80 msgr2=0x7f41c40b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:39.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.612+0000 7f41c3e00640 1 -- 172.21.15.19:0/2728894301 shutdown_connections 2024-03-28T13:09:39.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.613+0000 7f41c3e00640 1 -- 172.21.15.19:0/2728894301 wait complete. 2024-03-28T13:09:39.647 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-28T13:09:39.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.852+0000 7fc6a6400640 1 Processor -- start 2024-03-28T13:09:39.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.852+0000 7fc6a6400640 1 -- start start 2024-03-28T13:09:39.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.853+0000 7fc6a6400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc6a00d0990 0x7fc6a00aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:39.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.854+0000 7fc6a6400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6a00afab0 con 0x7fc6a0153580 2024-03-28T13:09:39.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.854+0000 7fc6a6400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6a015cc80 con 0x7fc6a00af2e0 2024-03-28T13:09:39.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.854+0000 7fc6a6400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc6a015da00 con 0x7fc6a00d0990 2024-03-28T13:09:39.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.854+0000 7fc69ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc6a00d0990 0x7fc6a00aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:39.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.854+0000 7fc69f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc6a0153580 0x7fc6a00d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59280/0 (socket says 172.21.15.19:59280) 2024-03-28T13:09:39.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.854+0000 7fc69f400640 1 -- 172.21.15.19:0/1906853991 learned_addr learned my addr 172.21.15.19:0/1906853991 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:39.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.854+0000 7fc69ea00640 1 -- 172.21.15.19:0/1906853991 >> v1:172.21.15.188:6789/0 conn(0x7fc6a00af2e0 legacy=0x7fc6a01600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:39.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.854+0000 7fc69ea00640 1 -- 172.21.15.19:0/1906853991 >> v1:172.21.15.19:6789/0 conn(0x7fc6a0153580 legacy=0x7fc6a00d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:39.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.854+0000 7fc69ea00640 1 -- 172.21.15.19:0/1906853991 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a015cf20 con 0x7fc6a00d0990 2024-03-28T13:09:39.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.856+0000 7fc69ea00640 1 --2- 172.21.15.19:0/1906853991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a00d0990 0x7fc6a00aeda0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0097a0 tx=0x7fc68c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76edc24b0b284cea server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:39.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.856+0000 7fc69e000640 1 -- 172.21.15.19:0/1906853991 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0315a0 con 0x7fc6a00d0990 2024-03-28T13:09:39.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.856+0000 7fc69e000640 1 -- 172.21.15.19:0/1906853991 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc68c037070 con 0x7fc6a00d0990 2024-03-28T13:09:39.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.856+0000 7fc69e000640 1 -- 172.21.15.19:0/1906853991 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c031960 con 0x7fc6a00d0990 2024-03-28T13:09:39.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.857+0000 7fc6a6400640 1 -- 172.21.15.19:0/1906853991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a00d0990 msgr2=0x7fc6a00aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:39.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.857+0000 7fc6a6400640 1 --2- 172.21.15.19:0/1906853991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a00d0990 0x7fc6a00aeda0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0097a0 tx=0x7fc68c02b840 comp rx=0 tx=0).stop 2024-03-28T13:09:39.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.857+0000 7fc6a6400640 1 -- 172.21.15.19:0/1906853991 shutdown_connections 2024-03-28T13:09:39.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.857+0000 7fc6a6400640 1 --2- 172.21.15.19:0/1906853991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a00d0990 0x7fc6a00aeda0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:39.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.857+0000 7fc6a6400640 1 -- 172.21.15.19:0/1906853991 >> 172.21.15.19:0/1906853991 conn(0x7fc6a006b750 msgr2=0x7fc6a006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:39.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.857+0000 7fc6a6400640 1 -- 172.21.15.19:0/1906853991 shutdown_connections 2024-03-28T13:09:39.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.857+0000 7fc6a6400640 1 -- 172.21.15.19:0/1906853991 wait complete. 2024-03-28T13:09:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.859+0000 7fc6a6400640 1 Processor -- start 2024-03-28T13:09:39.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.859+0000 7fc6a6400640 1 -- start start 2024-03-28T13:09:39.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.861+0000 7fc6a6400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a00af2e0 0x7fc6a0071610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.861+0000 7fc6a6400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a00d0990 0x7fc6a014d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.861+0000 7fc6a6400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6a0153580 0x7fc6a00c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.861+0000 7fc6a6400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a0153960 con 0x7fc6a00d0990 2024-03-28T13:09:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.861+0000 7fc6a6400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a015d4e0 con 0x7fc6a00af2e0 2024-03-28T13:09:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.861+0000 7fc6a6400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc6a00af6c0 con 0x7fc6a0153580 2024-03-28T13:09:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.861+0000 7fc69ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a00d0990 0x7fc6a014d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.861+0000 7fc69ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a00d0990 0x7fc6a014d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34124/0 (socket says 172.21.15.19:34124) 2024-03-28T13:09:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.861+0000 7fc69ea00640 1 -- 172.21.15.19:0/4159206492 learned_addr learned my addr 172.21.15.19:0/4159206492 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.861+0000 7fc69f400640 1 --2- 172.21.15.19:0/4159206492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a00af2e0 0x7fc6a0071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.861+0000 7fc69fe00640 1 --2- 172.21.15.19:0/4159206492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6a0153580 0x7fc6a00c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.862+0000 7fc69ea00640 1 -- 172.21.15.19:0/4159206492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6a0153580 msgr2=0x7fc6a00c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.862+0000 7fc69ea00640 1 --2- 172.21.15.19:0/4159206492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6a0153580 0x7fc6a00c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:39.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.862+0000 7fc69ea00640 1 -- 172.21.15.19:0/4159206492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a00af2e0 msgr2=0x7fc6a0071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:39.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.862+0000 7fc69ea00640 1 --2- 172.21.15.19:0/4159206492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a00af2e0 0x7fc6a0071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:39.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.862+0000 7fc69ea00640 1 -- 172.21.15.19:0/4159206492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a01f7bb0 con 0x7fc6a00d0990 2024-03-28T13:09:39.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.862+0000 7fc69f400640 1 --2- 172.21.15.19:0/4159206492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a00af2e0 0x7fc6a0071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:39.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.862+0000 7fc69ea00640 1 --2- 172.21.15.19:0/4159206492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a00d0990 0x7fc6a014d740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc68c000c00 tx=0x7fc68c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:39.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.862+0000 7fc697400640 1 -- 172.21.15.19:0/4159206492 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c032840 con 0x7fc6a00d0990 2024-03-28T13:09:39.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.862+0000 7fc697400640 1 -- 172.21.15.19:0/4159206492 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc68c0329e0 con 0x7fc6a00d0990 2024-03-28T13:09:39.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.862+0000 7fc697400640 1 -- 172.21.15.19:0/4159206492 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc68c032dd0 con 0x7fc6a00d0990 2024-03-28T13:09:39.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.862+0000 7fc6a6400640 1 -- 172.21.15.19:0/4159206492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a01f7e90 con 0x7fc6a00d0990 2024-03-28T13:09:39.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.863+0000 7fc6a6400640 1 -- 172.21.15.19:0/4159206492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a01f8450 con 0x7fc6a00d0990 2024-03-28T13:09:39.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.864+0000 7fc6a6400640 1 -- 172.21.15.19:0/4159206492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a00685c0 con 0x7fc6a00d0990 2024-03-28T13:09:39.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.865+0000 7fc697400640 1 -- 172.21.15.19:0/4159206492 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc68c04d3f0 con 0x7fc6a00d0990 2024-03-28T13:09:39.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.866+0000 7fc697400640 1 --2- 172.21.15.19:0/4159206492 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc68006c530 0x7fc68006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:39.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.866+0000 7fc697400640 1 -- 172.21.15.19:0/4159206492 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7fc68c02f030 con 0x7fc6a00d0990 2024-03-28T13:09:39.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.866+0000 7fc69f400640 1 --2- 172.21.15.19:0/4159206492 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc68006c530 0x7fc68006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:39.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.867+0000 7fc69f400640 1 --2- 172.21.15.19:0/4159206492 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc68006c530 0x7fc68006e9f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc690000f60 tx=0x7fc690009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:39.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:39.871+0000 7fc697400640 1 -- 172.21.15.19:0/4159206492 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc68c040070 con 0x7fc6a00d0990 2024-03-28T13:09:40.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.078+0000 7fc6a6400640 1 -- 172.21.15.19:0/4159206492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc6a01581f0 con 0x7fc6a00d0990 2024-03-28T13:09:40.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.612+0000 7fc697400640 1 -- 172.21.15.19:0/4159206492 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) ==== 170+0+0 (secure 0 0 0) 0x7fc68c07ba10 con 0x7fc6a00d0990 2024-03-28T13:09:40.613 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-03-28T13:09:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.617+0000 7fc6a6400640 1 -- 172.21.15.19:0/4159206492 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc68006c530 msgr2=0x7fc68006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.617+0000 7fc6a6400640 1 --2- 172.21.15.19:0/4159206492 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc68006c530 0x7fc68006e9f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc690000f60 tx=0x7fc690009040 comp rx=0 tx=0).stop 2024-03-28T13:09:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.617+0000 7fc6a6400640 1 -- 172.21.15.19:0/4159206492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a00d0990 msgr2=0x7fc6a014d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:40.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.617+0000 7fc6a6400640 1 --2- 172.21.15.19:0/4159206492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a00d0990 0x7fc6a014d740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc68c000c00 tx=0x7fc68c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:40.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.617+0000 7fc6a6400640 1 -- 172.21.15.19:0/4159206492 shutdown_connections 2024-03-28T13:09:40.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.617+0000 7fc6a6400640 1 --2- 172.21.15.19:0/4159206492 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc68006c530 0x7fc68006e9f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:40.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.617+0000 7fc6a6400640 1 --2- 172.21.15.19:0/4159206492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6a0153580 0x7fc6a00c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:40.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.617+0000 7fc6a6400640 1 --2- 172.21.15.19:0/4159206492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6a00d0990 0x7fc6a014d740 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc68c000c00 tx=0x7fc68c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:40.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.617+0000 7fc6a6400640 1 --2- 172.21.15.19:0/4159206492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6a00af2e0 0x7fc6a0071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:40.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.617+0000 7fc6a6400640 1 -- 172.21.15.19:0/4159206492 >> 172.21.15.19:0/4159206492 conn(0x7fc6a006b750 msgr2=0x7fc6a015a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:40.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.617+0000 7fc6a6400640 1 -- 172.21.15.19:0/4159206492 shutdown_connections 2024-03-28T13:09:40.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.618+0000 7fc6a6400640 1 -- 172.21.15.19:0/4159206492 wait complete. 2024-03-28T13:09:40.654 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-28T13:09:40.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.841+0000 7fd053e00640 1 Processor -- start 2024-03-28T13:09:40.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.842+0000 7fd053e00640 1 -- start start 2024-03-28T13:09:40.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.843+0000 7fd053e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04c15eca0 0x7fd04c0b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:40.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.843+0000 7fd053e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd04c15ac50 0x7fd04c0b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:40.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.843+0000 7fd053e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd04c0b4990 con 0x7fd04c15a870 2024-03-28T13:09:40.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.843+0000 7fd053e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd04c06d5b0 con 0x7fd04c15ac50 2024-03-28T13:09:40.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.843+0000 7fd053e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd04c06db70 con 0x7fd04c15eca0 2024-03-28T13:09:40.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.844+0000 7fd051400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd04c15a870 0x7fd04c15e590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59284/0 (socket says 172.21.15.19:59284) 2024-03-28T13:09:40.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.844+0000 7fd051e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd04c15ac50 0x7fd04c0b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:40.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.844+0000 7fd051400640 1 -- 172.21.15.19:0/1570080579 learned_addr learned my addr 172.21.15.19:0/1570080579 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:40.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.844+0000 7fd050a00640 1 --2- 172.21.15.19:0/1570080579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04c15eca0 0x7fd04c0b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:40.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.845+0000 7fd051e00640 1 -- 172.21.15.19:0/1570080579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04c15eca0 msgr2=0x7fd04c0b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:40.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.845+0000 7fd051e00640 1 --2- 172.21.15.19:0/1570080579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04c15eca0 0x7fd04c0b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:40.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.845+0000 7fd051e00640 1 -- 172.21.15.19:0/1570080579 >> v1:172.21.15.19:6789/0 conn(0x7fd04c15a870 legacy=0x7fd04c15e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:40.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.845+0000 7fd051e00640 1 -- 172.21.15.19:0/1570080579 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c0b4c00 con 0x7fd04c15ac50 2024-03-28T13:09:40.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.846+0000 7fd051e00640 1 --2- 172.21.15.19:0/1570080579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c15ac50 0x7fd04c0b4450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd044009d90 tx=0x7fd04402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17868585233642e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:40.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.846+0000 7fd04be00640 1 -- 172.21.15.19:0/1570080579 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044039070 con 0x7fd04c15ac50 2024-03-28T13:09:40.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.846+0000 7fd04be00640 1 -- 172.21.15.19:0/1570080579 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd04402cc60 con 0x7fd04c15ac50 2024-03-28T13:09:40.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.846+0000 7fd04be00640 1 -- 172.21.15.19:0/1570080579 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044034660 con 0x7fd04c15ac50 2024-03-28T13:09:40.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.847+0000 7fd053e00640 1 -- 172.21.15.19:0/1570080579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c15ac50 msgr2=0x7fd04c0b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:40.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.847+0000 7fd053e00640 1 --2- 172.21.15.19:0/1570080579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c15ac50 0x7fd04c0b4450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd044009d90 tx=0x7fd04402beb0 comp rx=0 tx=0).stop 2024-03-28T13:09:40.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.847+0000 7fd053e00640 1 -- 172.21.15.19:0/1570080579 shutdown_connections 2024-03-28T13:09:40.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.847+0000 7fd053e00640 1 --2- 172.21.15.19:0/1570080579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c15ac50 0x7fd04c0b4450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:40.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.847+0000 7fd053e00640 1 --2- 172.21.15.19:0/1570080579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04c15eca0 0x7fd04c0b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:40.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.847+0000 7fd053e00640 1 -- 172.21.15.19:0/1570080579 >> 172.21.15.19:0/1570080579 conn(0x7fd04c06b750 msgr2=0x7fd04c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:40.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.847+0000 7fd053e00640 1 -- 172.21.15.19:0/1570080579 shutdown_connections 2024-03-28T13:09:40.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.847+0000 7fd053e00640 1 -- 172.21.15.19:0/1570080579 wait complete. 2024-03-28T13:09:40.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.849+0000 7fd053e00640 1 Processor -- start 2024-03-28T13:09:40.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.849+0000 7fd053e00640 1 -- start start 2024-03-28T13:09:40.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd053e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c15a870 0x7fd04c06e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:40.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd053e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04c15ac50 0x7fd04c0989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:40.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd053e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd04c15eca0 0x7fd04c151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:40.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd053e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c0d0280 con 0x7fd04c15a870 2024-03-28T13:09:40.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd053e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c06d800 con 0x7fd04c15eca0 2024-03-28T13:09:40.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd053e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd04c0d0400 con 0x7fd04c15ac50 2024-03-28T13:09:40.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd051400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c15a870 0x7fd04c06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:40.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd051400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c15a870 0x7fd04c06e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34134/0 (socket says 172.21.15.19:34134) 2024-03-28T13:09:40.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd051400640 1 -- 172.21.15.19:0/2553325441 learned_addr learned my addr 172.21.15.19:0/2553325441 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:40.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd050a00640 1 --2- 172.21.15.19:0/2553325441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04c15ac50 0x7fd04c0989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:40.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd051e00640 1 --2- 172.21.15.19:0/2553325441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd04c15eca0 0x7fd04c151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:40.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd051400640 1 -- 172.21.15.19:0/2553325441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04c15ac50 msgr2=0x7fd04c0989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:40.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd051400640 1 --2- 172.21.15.19:0/2553325441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04c15ac50 0x7fd04c0989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:40.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd051400640 1 -- 172.21.15.19:0/2553325441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd04c15eca0 msgr2=0x7fd04c151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:40.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd051400640 1 --2- 172.21.15.19:0/2553325441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd04c15eca0 0x7fd04c151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:40.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd051400640 1 -- 172.21.15.19:0/2553325441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c1508d0 con 0x7fd04c15a870 2024-03-28T13:09:40.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.850+0000 7fd051e00640 1 --2- 172.21.15.19:0/2553325441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd04c15eca0 0x7fd04c151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:40.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.851+0000 7fd051400640 1 --2- 172.21.15.19:0/2553325441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c15a870 0x7fd04c06e520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd04000c430 tx=0x7fd04000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:40.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.851+0000 7fd049600640 1 -- 172.21.15.19:0/2553325441 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040017070 con 0x7fd04c15a870 2024-03-28T13:09:40.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.851+0000 7fd049600640 1 -- 172.21.15.19:0/2553325441 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd040005ce0 con 0x7fd04c15a870 2024-03-28T13:09:40.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.851+0000 7fd053e00640 1 -- 172.21.15.19:0/2553325441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c150bb0 con 0x7fd04c15a870 2024-03-28T13:09:40.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.851+0000 7fd049600640 1 -- 172.21.15.19:0/2553325441 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0400072f0 con 0x7fd04c15a870 2024-03-28T13:09:40.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.851+0000 7fd053e00640 1 -- 172.21.15.19:0/2553325441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c151170 con 0x7fd04c15a870 2024-03-28T13:09:40.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.852+0000 7fd053e00640 1 -- 172.21.15.19:0/2553325441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04c0685c0 con 0x7fd04c15a870 2024-03-28T13:09:40.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.853+0000 7fd049600640 1 -- 172.21.15.19:0/2553325441 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd04001a070 con 0x7fd04c15a870 2024-03-28T13:09:40.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.854+0000 7fd049600640 1 --2- 172.21.15.19:0/2553325441 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd02c06c560 0x7fd02c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:40.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.854+0000 7fd049600640 1 -- 172.21.15.19:0/2553325441 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7fd04008aa80 con 0x7fd04c15a870 2024-03-28T13:09:40.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.855+0000 7fd050a00640 1 --2- 172.21.15.19:0/2553325441 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd02c06c560 0x7fd02c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:40.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.856+0000 7fd050a00640 1 --2- 172.21.15.19:0/2553325441 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd02c06c560 0x7fd02c06ea20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd04c153730 tx=0x7fd038008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:40.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:40.859+0000 7fd049600640 1 -- 172.21.15.19:0/2553325441 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd040059100 con 0x7fd04c15a870 2024-03-28T13:09:41.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.103+0000 7fd053e00640 1 -- 172.21.15.19:0/2553325441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd04c0aff80 con 0x7fd04c15a870 2024-03-28T13:09:41.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.670+0000 7fd049600640 1 -- 172.21.15.19:0/2553325441 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fd040058a00 con 0x7fd04c15a870 2024-03-28T13:09:41.673 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 2 and data pool 3 2024-03-28T13:09:41.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.685+0000 7fd053e00640 1 -- 172.21.15.19:0/2553325441 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd02c06c560 msgr2=0x7fd02c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:41.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.685+0000 7fd053e00640 1 --2- 172.21.15.19:0/2553325441 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd02c06c560 0x7fd02c06ea20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd04c153730 tx=0x7fd038008040 comp rx=0 tx=0).stop 2024-03-28T13:09:41.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.685+0000 7fd053e00640 1 -- 172.21.15.19:0/2553325441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c15a870 msgr2=0x7fd04c06e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:41.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.685+0000 7fd053e00640 1 --2- 172.21.15.19:0/2553325441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c15a870 0x7fd04c06e520 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd04000c430 tx=0x7fd04000c8f0 comp rx=0 tx=0).stop 2024-03-28T13:09:41.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.686+0000 7fd053e00640 1 -- 172.21.15.19:0/2553325441 shutdown_connections 2024-03-28T13:09:41.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.686+0000 7fd053e00640 1 --2- 172.21.15.19:0/2553325441 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd02c06c560 0x7fd02c06ea20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:41.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.686+0000 7fd053e00640 1 --2- 172.21.15.19:0/2553325441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd04c15eca0 0x7fd04c151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:41.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.686+0000 7fd053e00640 1 --2- 172.21.15.19:0/2553325441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd04c15ac50 0x7fd04c0989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:41.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.686+0000 7fd053e00640 1 --2- 172.21.15.19:0/2553325441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04c15a870 0x7fd04c06e520 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:41.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.686+0000 7fd053e00640 1 -- 172.21.15.19:0/2553325441 >> 172.21.15.19:0/2553325441 conn(0x7fd04c06b750 msgr2=0x7fd04c0b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:41.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.687+0000 7fd053e00640 1 -- 172.21.15.19:0/2553325441 shutdown_connections 2024-03-28T13:09:41.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.687+0000 7fd053e00640 1 -- 172.21.15.19:0/2553325441 wait complete. 2024-03-28T13:09:41.718 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:09:41.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.899+0000 7f1b0cc00640 1 Processor -- start 2024-03-28T13:09:41.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.899+0000 7f1b0cc00640 1 -- start start 2024-03-28T13:09:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.900+0000 7f1b0cc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0815c390 0x7f1b08073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.900+0000 7f1b0cc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1b080af830 0x7f1b08160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.900+0000 7f1b0cc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b0815fe80 con 0x7f1b08160b80 2024-03-28T13:09:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.900+0000 7f1b0cc00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1b0815c770 con 0x7f1b080af830 2024-03-28T13:09:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.900+0000 7f1b0cc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1b080ad120 con 0x7f1b0815c390 2024-03-28T13:09:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.901+0000 7f1b05400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1b080af830 0x7f1b08160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.901+0000 7f1b05400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1b080af830 0x7f1b08160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34138/0 (socket says 172.21.15.19:34138) 2024-03-28T13:09:41.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.901+0000 7f1b05400640 1 -- 172.21.15.19:0/2461630327 learned_addr learned my addr 172.21.15.19:0/2461630327 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.901+0000 7f1b05e00640 1 --2- 172.21.15.19:0/2461630327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0815c390 0x7f1b08073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.901+0000 7f1b05400640 1 -- 172.21.15.19:0/2461630327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0815c390 msgr2=0x7f1b08073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.901+0000 7f1b05400640 1 --2- 172.21.15.19:0/2461630327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0815c390 0x7f1b08073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.901+0000 7f1b05400640 1 -- 172.21.15.19:0/2461630327 >> v1:172.21.15.188:6789/0 conn(0x7f1b08160b80 legacy=0x7f1b080d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:41.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.901+0000 7f1b05400640 1 -- 172.21.15.19:0/2461630327 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b080ad400 con 0x7f1b080af830 2024-03-28T13:09:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.902+0000 7f1b05400640 1 --2- 172.21.15.19:0/2461630327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b080af830 0x7f1b08160640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009870 tx=0x7f1af402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c217c6fa45b4c9d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.903+0000 7f1b04a00640 1 -- 172.21.15.19:0/2461630327 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af4035020 con 0x7f1b080af830 2024-03-28T13:09:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.903+0000 7f1b04a00640 1 -- 172.21.15.19:0/2461630327 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af4031640 con 0x7f1b080af830 2024-03-28T13:09:41.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.903+0000 7f1b04a00640 1 -- 172.21.15.19:0/2461630327 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af402d040 con 0x7f1b080af830 2024-03-28T13:09:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.903+0000 7f1b0cc00640 1 -- 172.21.15.19:0/2461630327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b080af830 msgr2=0x7f1b08160640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.903+0000 7f1b0cc00640 1 --2- 172.21.15.19:0/2461630327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b080af830 0x7f1b08160640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009870 tx=0x7f1af402b8f0 comp rx=0 tx=0).stop 2024-03-28T13:09:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.904+0000 7f1b0cc00640 1 -- 172.21.15.19:0/2461630327 shutdown_connections 2024-03-28T13:09:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.904+0000 7f1b0cc00640 1 --2- 172.21.15.19:0/2461630327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b080af830 0x7f1b08160640 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.904+0000 7f1b0cc00640 1 --2- 172.21.15.19:0/2461630327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0815c390 0x7f1b08073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:41.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.904+0000 7f1b0cc00640 1 -- 172.21.15.19:0/2461630327 >> 172.21.15.19:0/2461630327 conn(0x7f1b0806b990 msgr2=0x7f1b0806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.904+0000 7f1b0cc00640 1 -- 172.21.15.19:0/2461630327 shutdown_connections 2024-03-28T13:09:41.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.904+0000 7f1b0cc00640 1 -- 172.21.15.19:0/2461630327 wait complete. 2024-03-28T13:09:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.906+0000 7f1b0cc00640 1 Processor -- start 2024-03-28T13:09:41.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.906+0000 7f1b0cc00640 1 -- start start 2024-03-28T13:09:41.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.907+0000 7f1b0cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b080af830 0x7f1b0813ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:41.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.907+0000 7f1b0cc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0815c390 0x7f1b0806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:41.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.907+0000 7f1b0cc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b08160b80 0x7f1b0806dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:41.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.907+0000 7f1b0cc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b080cf980 con 0x7f1b080af830 2024-03-28T13:09:41.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.907+0000 7f1b0cc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1b08160120 con 0x7f1b08160b80 2024-03-28T13:09:41.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.907+0000 7f1b0cc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1b080d0d60 con 0x7f1b0815c390 2024-03-28T13:09:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.907+0000 7f1b05e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b080af830 0x7f1b0813ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.908+0000 7f1b05e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b080af830 0x7f1b0813ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34140/0 (socket says 172.21.15.19:34140) 2024-03-28T13:09:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.908+0000 7f1b05e00640 1 -- 172.21.15.19:0/1721952860 learned_addr learned my addr 172.21.15.19:0/1721952860 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.908+0000 7f1b05400640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0815c390 0x7f1b0806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.908+0000 7f1b06800640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b08160b80 0x7f1b0806dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.908+0000 7f1b05e00640 1 -- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0815c390 msgr2=0x7f1b0806d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.908+0000 7f1b05e00640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0815c390 0x7f1b0806d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.908+0000 7f1b05e00640 1 -- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b08160b80 msgr2=0x7f1b0806dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.908+0000 7f1b05e00640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b08160b80 0x7f1b0806dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.908+0000 7f1b05e00640 1 -- 172.21.15.19:0/1721952860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b080b2b00 con 0x7f1b080af830 2024-03-28T13:09:41.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.908+0000 7f1b05400640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0815c390 0x7f1b0806d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:41.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.908+0000 7f1b06800640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b08160b80 0x7f1b0806dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:41.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.908+0000 7f1b05e00640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b080af830 0x7f1b0813ecf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1af800d830 tx=0x7f1af800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:41.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.909+0000 7f1afe000640 1 -- 172.21.15.19:0/1721952860 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af800b660 con 0x7f1b080af830 2024-03-28T13:09:41.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.909+0000 7f1afe000640 1 -- 172.21.15.19:0/1721952860 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af8019070 con 0x7f1b080af830 2024-03-28T13:09:41.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.909+0000 7f1afe000640 1 -- 172.21.15.19:0/1721952860 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af800ba20 con 0x7f1b080af830 2024-03-28T13:09:41.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.909+0000 7f1b0cc00640 1 -- 172.21.15.19:0/1721952860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b0813f750 con 0x7f1b080af830 2024-03-28T13:09:41.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.909+0000 7f1b0cc00640 1 -- 172.21.15.19:0/1721952860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b0813fd10 con 0x7f1b080af830 2024-03-28T13:09:41.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.910+0000 7f1b0cc00640 1 -- 172.21.15.19:0/1721952860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b080685c0 con 0x7f1b080af830 2024-03-28T13:09:41.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.911+0000 7f1afe000640 1 -- 172.21.15.19:0/1721952860 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1af801e460 con 0x7f1b080af830 2024-03-28T13:09:41.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.912+0000 7f1afe000640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1ae806c560 0x7f1ae806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:41.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.912+0000 7f1afe000640 1 -- 172.21.15.19:0/1721952860 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f1af808bd80 con 0x7f1b080af830 2024-03-28T13:09:41.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.913+0000 7f1b05400640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1ae806c560 0x7f1ae806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:41.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.914+0000 7f1b05400640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1ae806c560 0x7f1ae806ea20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1af4036000 tx=0x7f1af4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:41.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:41.917+0000 7f1afe000640 1 -- 172.21.15.19:0/1721952860 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1af805a310 con 0x7f1b080af830 2024-03-28T13:09:42.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.122+0000 7f1b0cc00640 1 -- 172.21.15.19:0/1721952860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b0806f1b0 con 0x7f1b080af830 2024-03-28T13:09:42.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.125+0000 7f1afe000640 1 -- 172.21.15.19:0/1721952860 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16622 (secure 0 0 0) 0x7f1af8059c10 con 0x7f1b080af830 2024-03-28T13:09:42.125 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:42.126 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:41.105061+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:09:42.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.127+0000 7f1b0cc00640 1 -- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1ae806c560 msgr2=0x7f1ae806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:42.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.127+0000 7f1b0cc00640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1ae806c560 0x7f1ae806ea20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1af4036000 tx=0x7f1af4008030 comp rx=0 tx=0).stop 2024-03-28T13:09:42.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.127+0000 7f1b0cc00640 1 -- 172.21.15.19:0/1721952860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b080af830 msgr2=0x7f1b0813ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:42.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.127+0000 7f1b0cc00640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b080af830 0x7f1b0813ecf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1af800d830 tx=0x7f1af800dcf0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.127+0000 7f1b0cc00640 1 -- 172.21.15.19:0/1721952860 shutdown_connections 2024-03-28T13:09:42.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.127+0000 7f1b0cc00640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1ae806c560 0x7f1ae806ea20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.127+0000 7f1b0cc00640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b08160b80 0x7f1b0806dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.127+0000 7f1b0cc00640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b0815c390 0x7f1b0806d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.127+0000 7f1b0cc00640 1 --2- 172.21.15.19:0/1721952860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b080af830 0x7f1b0813ecf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.127+0000 7f1b0cc00640 1 -- 172.21.15.19:0/1721952860 >> 172.21.15.19:0/1721952860 conn(0x7f1b0806b990 msgr2=0x7f1b08158090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:42.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.127+0000 7f1b0cc00640 1 -- 172.21.15.19:0/1721952860 shutdown_connections 2024-03-28T13:09:42.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.128+0000 7f1b0cc00640 1 -- 172.21.15.19:0/1721952860 wait complete. 2024-03-28T13:09:42.146 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:09:42.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.371+0000 7fc1aec00640 1 Processor -- start 2024-03-28T13:09:42.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.371+0000 7fc1aec00640 1 -- start start 2024-03-28T13:09:42.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.373+0000 7fc1aec00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc1a80ad120 0x7fc1a80d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:42.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.373+0000 7fc1aec00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc1a80ad580 0x7fc1a80d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:42.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.373+0000 7fc1aec00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1a8071b60 con 0x7fc1a8161b00 2024-03-28T13:09:42.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.373+0000 7fc1aec00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a8161ee0 con 0x7fc1a80ad120 2024-03-28T13:09:42.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.373+0000 7fc1aec00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc1a80b3450 con 0x7fc1a80ad580 2024-03-28T13:09:42.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.373+0000 7fc1a7400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc1a80ad120 0x7fc1a80d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:42.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.373+0000 7fc1a7400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc1a80ad120 0x7fc1a80d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34152/0 (socket says 172.21.15.19:34152) 2024-03-28T13:09:42.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.373+0000 7fc1a7400640 1 -- 172.21.15.19:0/2352558850 learned_addr learned my addr 172.21.15.19:0/2352558850 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:42.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.374+0000 7fc1acc00640 1 --2- 172.21.15.19:0/2352558850 >> v2:172.21.15.188:3300/0 conn(0x7fc1a80ad580 0x7fc1a80d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:42.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.374+0000 7fc1a7400640 1 -- 172.21.15.19:0/2352558850 >> v2:172.21.15.188:3300/0 conn(0x7fc1a80ad580 msgr2=0x7fc1a80d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:42.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.374+0000 7fc1a7400640 1 --2- 172.21.15.19:0/2352558850 >> v2:172.21.15.188:3300/0 conn(0x7fc1a80ad580 0x7fc1a80d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.374+0000 7fc1a7400640 1 -- 172.21.15.19:0/2352558850 >> v1:172.21.15.188:6789/0 conn(0x7fc1a8161b00 legacy=0x7fc1a8162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:42.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.374+0000 7fc1a7400640 1 -- 172.21.15.19:0/2352558850 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a8071e40 con 0x7fc1a80ad120 2024-03-28T13:09:42.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.375+0000 7fc1a7400640 1 --2- 172.21.15.19:0/2352558850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1a80ad120 0x7fc1a80d0650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc1a80d01e0 tx=0x7fc19002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7161778126554ad5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:42.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.375+0000 7fc1a6a00640 1 -- 172.21.15.19:0/2352558850 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19002e070 con 0x7fc1a80ad120 2024-03-28T13:09:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.375+0000 7fc1a6a00640 1 -- 172.21.15.19:0/2352558850 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc190037070 con 0x7fc1a80ad120 2024-03-28T13:09:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.375+0000 7fc1a6a00640 1 -- 172.21.15.19:0/2352558850 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190031a00 con 0x7fc1a80ad120 2024-03-28T13:09:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.376+0000 7fc1aec00640 1 -- 172.21.15.19:0/2352558850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1a80ad120 msgr2=0x7fc1a80d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.376+0000 7fc1aec00640 1 --2- 172.21.15.19:0/2352558850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1a80ad120 0x7fc1a80d0650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc1a80d01e0 tx=0x7fc19002b880 comp rx=0 tx=0).stop 2024-03-28T13:09:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.376+0000 7fc1aec00640 1 -- 172.21.15.19:0/2352558850 shutdown_connections 2024-03-28T13:09:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.376+0000 7fc1aec00640 1 --2- 172.21.15.19:0/2352558850 >> v2:172.21.15.188:3300/0 conn(0x7fc1a80ad580 0x7fc1a80d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.376+0000 7fc1aec00640 1 --2- 172.21.15.19:0/2352558850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1a80ad120 0x7fc1a80d0650 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.376+0000 7fc1aec00640 1 -- 172.21.15.19:0/2352558850 >> 172.21.15.19:0/2352558850 conn(0x7fc1a806b990 msgr2=0x7fc1a806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.376+0000 7fc1aec00640 1 -- 172.21.15.19:0/2352558850 shutdown_connections 2024-03-28T13:09:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.376+0000 7fc1aec00640 1 -- 172.21.15.19:0/2352558850 wait complete. 2024-03-28T13:09:42.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.378+0000 7fc1aec00640 1 Processor -- start 2024-03-28T13:09:42.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.378+0000 7fc1aec00640 1 -- start start 2024-03-28T13:09:42.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.379+0000 7fc1aec00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1a80ad120 0x7fc1a8156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:42.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.379+0000 7fc1aec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1a80ad580 0x7fc1a815eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:42.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.379+0000 7fc1aec00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1a8161b00 0x7fc1a8159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:42.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.379+0000 7fc1aec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a80717f0 con 0x7fc1a80ad580 2024-03-28T13:09:42.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.379+0000 7fc1aec00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a80b3730 con 0x7fc1a80ad120 2024-03-28T13:09:42.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.379+0000 7fc1aec00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc1a80b5020 con 0x7fc1a8161b00 2024-03-28T13:09:42.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.379+0000 7fc1a7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1a80ad580 0x7fc1a815eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:42.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.379+0000 7fc1a7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1a80ad580 0x7fc1a815eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34168/0 (socket says 172.21.15.19:34168) 2024-03-28T13:09:42.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.379+0000 7fc1a7400640 1 -- 172.21.15.19:0/3752387707 learned_addr learned my addr 172.21.15.19:0/3752387707 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:42.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.379+0000 7fc1acc00640 1 --2- 172.21.15.19:0/3752387707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1a8161b00 0x7fc1a8159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:42.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.379+0000 7fc1a7e00640 1 --2- 172.21.15.19:0/3752387707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1a80ad120 0x7fc1a8156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.380+0000 7fc1a7400640 1 -- 172.21.15.19:0/3752387707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1a8161b00 msgr2=0x7fc1a8159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.380+0000 7fc1a7400640 1 --2- 172.21.15.19:0/3752387707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1a8161b00 0x7fc1a8159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.380+0000 7fc1a7400640 1 -- 172.21.15.19:0/3752387707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1a80ad120 msgr2=0x7fc1a8156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.380+0000 7fc1a7400640 1 --2- 172.21.15.19:0/3752387707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1a80ad120 0x7fc1a8156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.380+0000 7fc1a7400640 1 -- 172.21.15.19:0/3752387707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a8155f20 con 0x7fc1a80ad580 2024-03-28T13:09:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.380+0000 7fc1a7e00640 1 --2- 172.21.15.19:0/3752387707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1a80ad120 0x7fc1a8156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.380+0000 7fc1a7400640 1 --2- 172.21.15.19:0/3752387707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1a80ad580 0x7fc1a815eba0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc190000c00 tx=0x7fc1900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.380+0000 7fc19fe00640 1 -- 172.21.15.19:0/3752387707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190032880 con 0x7fc1a80ad580 2024-03-28T13:09:42.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.380+0000 7fc19fe00640 1 -- 172.21.15.19:0/3752387707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc190032a20 con 0x7fc1a80ad580 2024-03-28T13:09:42.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.380+0000 7fc19fe00640 1 -- 172.21.15.19:0/3752387707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190032e10 con 0x7fc1a80ad580 2024-03-28T13:09:42.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.380+0000 7fc1aec00640 1 -- 172.21.15.19:0/3752387707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a8154dd0 con 0x7fc1a80ad580 2024-03-28T13:09:42.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.381+0000 7fc1aec00640 1 -- 172.21.15.19:0/3752387707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a815f0e0 con 0x7fc1a80ad580 2024-03-28T13:09:42.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.382+0000 7fc1aec00640 1 -- 172.21.15.19:0/3752387707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a8161320 con 0x7fc1a80ad580 2024-03-28T13:09:42.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.383+0000 7fc19fe00640 1 -- 172.21.15.19:0/3752387707 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc19004c060 con 0x7fc1a80ad580 2024-03-28T13:09:42.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.384+0000 7fc19fe00640 1 --2- 172.21.15.19:0/3752387707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc18406c530 0x7fc18406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:42.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.384+0000 7fc19fe00640 1 -- 172.21.15.19:0/3752387707 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc1900acdf0 con 0x7fc1a80ad580 2024-03-28T13:09:42.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.385+0000 7fc1a7e00640 1 --2- 172.21.15.19:0/3752387707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc18406c530 0x7fc18406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:42.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.386+0000 7fc1a7e00640 1 --2- 172.21.15.19:0/3752387707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc18406c530 0x7fc18406e9f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc198003eb0 tx=0x7fc198009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:42.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.390+0000 7fc19fe00640 1 -- 172.21.15.19:0/3752387707 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc1a8161320 con 0x7fc1a80ad580 2024-03-28T13:09:42.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.594+0000 7fc1aec00640 1 -- 172.21.15.19:0/3752387707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1a81552f0 con 0x7fc1a80ad580 2024-03-28T13:09:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.597+0000 7fc19fe00640 1 -- 172.21.15.19:0/3752387707 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16622 (secure 0 0 0) 0x7fc190081030 con 0x7fc1a80ad580 2024-03-28T13:09:42.597 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:42.597 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:41.105061+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:09:42.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.599+0000 7fc1aec00640 1 -- 172.21.15.19:0/3752387707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc18406c530 msgr2=0x7fc18406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:42.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.599+0000 7fc1aec00640 1 --2- 172.21.15.19:0/3752387707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc18406c530 0x7fc18406e9f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc198003eb0 tx=0x7fc198009040 comp rx=0 tx=0).stop 2024-03-28T13:09:42.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.600+0000 7fc1aec00640 1 -- 172.21.15.19:0/3752387707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1a80ad580 msgr2=0x7fc1a815eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:42.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.600+0000 7fc1aec00640 1 --2- 172.21.15.19:0/3752387707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1a80ad580 0x7fc1a815eba0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc190000c00 tx=0x7fc1900046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.600+0000 7fc1aec00640 1 -- 172.21.15.19:0/3752387707 shutdown_connections 2024-03-28T13:09:42.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.600+0000 7fc1aec00640 1 --2- 172.21.15.19:0/3752387707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc18406c530 0x7fc18406e9f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.600+0000 7fc1aec00640 1 --2- 172.21.15.19:0/3752387707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1a8161b00 0x7fc1a8159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.600+0000 7fc1aec00640 1 --2- 172.21.15.19:0/3752387707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1a80ad580 0x7fc1a815eba0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.600+0000 7fc1aec00640 1 --2- 172.21.15.19:0/3752387707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1a80ad120 0x7fc1a8156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.600+0000 7fc1aec00640 1 -- 172.21.15.19:0/3752387707 >> 172.21.15.19:0/3752387707 conn(0x7fc1a806b990 msgr2=0x7fc1a806bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:42.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.600+0000 7fc1aec00640 1 -- 172.21.15.19:0/3752387707 shutdown_connections 2024-03-28T13:09:42.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.600+0000 7fc1aec00640 1 -- 172.21.15.19:0/3752387707 wait complete. 2024-03-28T13:09:42.618 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-28T13:09:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.837+0000 7f535d200640 1 Processor -- start 2024-03-28T13:09:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.837+0000 7f535d200640 1 -- start start 2024-03-28T13:09:42.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.840+0000 7f535d200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5358073610 0x7f53580cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.840+0000 7f535d200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f53580cf210 0x7f535806f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.840+0000 7f535d200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53580d0be0 con 0x7f5358144a80 2024-03-28T13:09:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.840+0000 7f535d200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f53580d1a50 con 0x7f5358073610 2024-03-28T13:09:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.840+0000 7f535d200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f53580d0e80 con 0x7f53580cf210 2024-03-28T13:09:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.840+0000 7f5357400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5358144a80 0x7f5358072f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59294/0 (socket says 172.21.15.19:59294) 2024-03-28T13:09:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.840+0000 7f5357400640 1 -- 172.21.15.19:0/2506855144 learned_addr learned my addr 172.21.15.19:0/2506855144 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:42.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.840+0000 7f5356a00640 1 --2- 172.21.15.19:0/2506855144 >> v2:172.21.15.19:3300/0 conn(0x7f5358073610 0x7f53580cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.841+0000 7f5357e00640 1 --2- 172.21.15.19:0/2506855144 >> v2:172.21.15.188:3300/0 conn(0x7f53580cf210 0x7f535806f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.841+0000 7f5356000640 1 -- 172.21.15.19:0/2506855144 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2194707957 0 0) 0x7f53580d0be0 con 0x7f5358144a80 2024-03-28T13:09:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.841+0000 7f5356a00640 1 -- 172.21.15.19:0/2506855144 >> v2:172.21.15.188:3300/0 conn(0x7f53580cf210 msgr2=0x7f535806f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.841+0000 7f5356a00640 1 --2- 172.21.15.19:0/2506855144 >> v2:172.21.15.188:3300/0 conn(0x7f53580cf210 0x7f535806f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.841+0000 7f5356a00640 1 -- 172.21.15.19:0/2506855144 >> v1:172.21.15.19:6789/0 conn(0x7f5358144a80 legacy=0x7f5358072f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.841+0000 7f5356a00640 1 -- 172.21.15.19:0/2506855144 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53580d1120 con 0x7f5358073610 2024-03-28T13:09:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.842+0000 7f5356a00640 1 --2- 172.21.15.19:0/2506855144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5358073610 0x7f53580cecd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f53580739f0 tx=0x7f534402dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3dd71438f0197c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.842+0000 7f5356000640 1 -- 172.21.15.19:0/2506855144 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534402f8a0 con 0x7f5358073610 2024-03-28T13:09:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.842+0000 7f5356000640 1 -- 172.21.15.19:0/2506855144 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f534402fa40 con 0x7f5358073610 2024-03-28T13:09:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.842+0000 7f5356000640 1 -- 172.21.15.19:0/2506855144 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534402fe10 con 0x7f5358073610 2024-03-28T13:09:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.843+0000 7f535d200640 1 -- 172.21.15.19:0/2506855144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5358073610 msgr2=0x7f53580cecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.843+0000 7f535d200640 1 --2- 172.21.15.19:0/2506855144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5358073610 0x7f53580cecd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f53580739f0 tx=0x7f534402dae0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.843+0000 7f535d200640 1 -- 172.21.15.19:0/2506855144 shutdown_connections 2024-03-28T13:09:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.843+0000 7f535d200640 1 --2- 172.21.15.19:0/2506855144 >> v2:172.21.15.188:3300/0 conn(0x7f53580cf210 0x7f535806f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.843+0000 7f535d200640 1 --2- 172.21.15.19:0/2506855144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5358073610 0x7f53580cecd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.843+0000 7f535d200640 1 -- 172.21.15.19:0/2506855144 >> 172.21.15.19:0/2506855144 conn(0x7f535806e730 msgr2=0x7f535806eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.843+0000 7f535d200640 1 -- 172.21.15.19:0/2506855144 shutdown_connections 2024-03-28T13:09:42.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.843+0000 7f535d200640 1 -- 172.21.15.19:0/2506855144 wait complete. 2024-03-28T13:09:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.845+0000 7f535d200640 1 Processor -- start 2024-03-28T13:09:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.845+0000 7f535d200640 1 -- start start 2024-03-28T13:09:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.846+0000 7f535d200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5358073610 0x7f53581f3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.846+0000 7f535d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580cf210 0x7f535813dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.846+0000 7f535d200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5358144a80 0x7f535813e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.846+0000 7f535d200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f53580cf620 con 0x7f53580cf210 2024-03-28T13:09:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.846+0000 7f535d200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53580d16e0 con 0x7f5358073610 2024-03-28T13:09:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.846+0000 7f535d200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f53580cf7a0 con 0x7f5358144a80 2024-03-28T13:09:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.846+0000 7f5356a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580cf210 0x7f535813dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.846+0000 7f5356a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580cf210 0x7f535813dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34176/0 (socket says 172.21.15.19:34176) 2024-03-28T13:09:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.846+0000 7f5356a00640 1 -- 172.21.15.19:0/954162604 learned_addr learned my addr 172.21.15.19:0/954162604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.846+0000 7f5356a00640 1 -- 172.21.15.19:0/954162604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5358144a80 msgr2=0x7f535813e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.846+0000 7f5356a00640 1 --2- 172.21.15.19:0/954162604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5358144a80 0x7f535813e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.846+0000 7f5357400640 1 --2- 172.21.15.19:0/954162604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5358073610 0x7f53581f3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.846+0000 7f5356a00640 1 -- 172.21.15.19:0/954162604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5358073610 msgr2=0x7f53581f3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:42.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.846+0000 7f5356a00640 1 --2- 172.21.15.19:0/954162604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5358073610 0x7f53581f3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:42.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.847+0000 7f5356a00640 1 -- 172.21.15.19:0/954162604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53580afaf0 con 0x7f53580cf210 2024-03-28T13:09:42.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.847+0000 7f5357400640 1 --2- 172.21.15.19:0/954162604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5358073610 0x7f53581f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:42.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.847+0000 7f5356a00640 1 --2- 172.21.15.19:0/954162604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580cf210 0x7f535813dee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5344009890 tx=0x7f53440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:42.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.847+0000 7f534f400640 1 -- 172.21.15.19:0/954162604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344030b70 con 0x7f53580cf210 2024-03-28T13:09:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.847+0000 7f534f400640 1 -- 172.21.15.19:0/954162604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5344030d30 con 0x7f53580cf210 2024-03-28T13:09:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.847+0000 7f534f400640 1 -- 172.21.15.19:0/954162604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344042650 con 0x7f53580cf210 2024-03-28T13:09:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.847+0000 7f535d200640 1 -- 172.21.15.19:0/954162604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53580afdd0 con 0x7f53580cf210 2024-03-28T13:09:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.847+0000 7f535d200640 1 -- 172.21.15.19:0/954162604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53581f4410 con 0x7f53580cf210 2024-03-28T13:09:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.848+0000 7f535d200640 1 -- 172.21.15.19:0/954162604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53580b02f0 con 0x7f53580cf210 2024-03-28T13:09:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.849+0000 7f534f400640 1 -- 172.21.15.19:0/954162604 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5344049020 con 0x7f53580cf210 2024-03-28T13:09:42.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.850+0000 7f534f400640 1 --2- 172.21.15.19:0/954162604 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f533806c530 0x7f533806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:42.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.850+0000 7f534f400640 1 -- 172.21.15.19:0/954162604 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f53440ad110 con 0x7f53580cf210 2024-03-28T13:09:42.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.851+0000 7f5357400640 1 --2- 172.21.15.19:0/954162604 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f533806c530 0x7f533806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:42.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.852+0000 7f5357400640 1 --2- 172.21.15.19:0/954162604 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f533806c530 0x7f533806e9f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5348000f60 tx=0x7f5348009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:42.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:42.854+0000 7f534f400640 1 -- 172.21.15.19:0/954162604 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5344035030 con 0x7f53580cf210 2024-03-28T13:09:43.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.094+0000 7f535d200640 1 -- 172.21.15.19:0/954162604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5358075940 con 0x7f53580cf210 2024-03-28T13:09:43.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.679+0000 7f534f400640 1 -- 172.21.15.19:0/954162604 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f534407b730 con 0x7f53580cf210 2024-03-28T13:09:43.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.683+0000 7f535d200640 1 -- 172.21.15.19:0/954162604 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f533806c530 msgr2=0x7f533806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:43.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.683+0000 7f535d200640 1 --2- 172.21.15.19:0/954162604 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f533806c530 0x7f533806e9f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5348000f60 tx=0x7f5348009040 comp rx=0 tx=0).stop 2024-03-28T13:09:43.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.683+0000 7f535d200640 1 -- 172.21.15.19:0/954162604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580cf210 msgr2=0x7f535813dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:43.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.683+0000 7f535d200640 1 --2- 172.21.15.19:0/954162604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580cf210 0x7f535813dee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5344009890 tx=0x7f53440046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.683+0000 7f535d200640 1 -- 172.21.15.19:0/954162604 shutdown_connections 2024-03-28T13:09:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.683+0000 7f535d200640 1 --2- 172.21.15.19:0/954162604 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f533806c530 0x7f533806e9f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:43.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.683+0000 7f535d200640 1 --2- 172.21.15.19:0/954162604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5358144a80 0x7f535813e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:43.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.683+0000 7f535d200640 1 --2- 172.21.15.19:0/954162604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53580cf210 0x7f535813dee0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:43.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.683+0000 7f535d200640 1 --2- 172.21.15.19:0/954162604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5358073610 0x7f53581f3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:43.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.683+0000 7f535d200640 1 -- 172.21.15.19:0/954162604 >> 172.21.15.19:0/954162604 conn(0x7f535806e730 msgr2=0x7f5358070220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:43.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.683+0000 7f535d200640 1 -- 172.21.15.19:0/954162604 shutdown_connections 2024-03-28T13:09:43.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.683+0000 7f535d200640 1 -- 172.21.15.19:0/954162604 wait complete. 2024-03-28T13:09:43.709 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-28T13:09:43.710 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-28T13:09:43.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.936+0000 7f3547000640 1 Processor -- start 2024-03-28T13:09:43.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.936+0000 7f3547000640 1 -- start start 2024-03-28T13:09:43.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.937+0000 7f3547000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35400b4af0 0x7f354015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:43.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.937+0000 7f3547000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3540141680 0x7f354015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:43.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.938+0000 7f3547000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f354015d7c0 con 0x7f35401412a0 2024-03-28T13:09:43.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.938+0000 7f3547000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3540141be0 con 0x7f3540141680 2024-03-28T13:09:43.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.938+0000 7f3547000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3540141ec0 con 0x7f35400b4af0 2024-03-28T13:09:43.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.938+0000 7f353fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f35401412a0 0x7f35400b43e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59304/0 (socket says 172.21.15.19:59304) 2024-03-28T13:09:43.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.938+0000 7f353fe00640 1 -- 172.21.15.19:0/2256800109 learned_addr learned my addr 172.21.15.19:0/2256800109 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.938+0000 7f3537400640 1 --2- 172.21.15.19:0/2256800109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35400b4af0 0x7f354015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.938+0000 7f3545000640 1 --2- 172.21.15.19:0/2256800109 >> v2:172.21.15.188:3300/0 conn(0x7f3540141680 0x7f354015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.938+0000 7f353f400640 1 -- 172.21.15.19:0/2256800109 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1181401657 0 0) 0x7f354015d7c0 con 0x7f35401412a0 2024-03-28T13:09:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.938+0000 7f353f400640 1 -- 172.21.15.19:0/2256800109 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3524003520 con 0x7f35401412a0 2024-03-28T13:09:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.939+0000 7f353f400640 1 -- 172.21.15.19:0/2256800109 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3712874911 0 0) 0x7f3524003520 con 0x7f35401412a0 2024-03-28T13:09:43.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.939+0000 7f353f400640 1 -- 172.21.15.19:0/2256800109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35400b4af0 msgr2=0x7f354015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.939+0000 7f353f400640 1 --2- 172.21.15.19:0/2256800109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35400b4af0 0x7f354015ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.939+0000 7f353f400640 1 -- 172.21.15.19:0/2256800109 >> v2:172.21.15.188:3300/0 conn(0x7f3540141680 msgr2=0x7f354015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.939+0000 7f353f400640 1 --2- 172.21.15.19:0/2256800109 >> v2:172.21.15.188:3300/0 conn(0x7f3540141680 0x7f354015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.939+0000 7f353f400640 1 -- 172.21.15.19:0/2256800109 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35401421a0 con 0x7f35401412a0 2024-03-28T13:09:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.939+0000 7f353f400640 1 -- 172.21.15.19:0/2256800109 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f35300019f0 con 0x7f35401412a0 2024-03-28T13:09:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.939+0000 7f353f400640 1 -- 172.21.15.19:0/2256800109 >> v1:172.21.15.19:6789/0 conn(0x7f35401412a0 legacy=0x7f35400b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.939+0000 7f353f400640 1 --2- 172.21.15.19:0/2256800109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35240039b0 0x7f3524003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.939+0000 7f353f400640 1 --2- 172.21.15.19:0/2256800109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3524004800 0x7f3524004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.939+0000 7f353f400640 1 --2- 172.21.15.19:0/2256800109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35400b4af0 0x7f35240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.939+0000 7f353f400640 1 -- 172.21.15.19:0/2256800109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3540141be0 con 0x7f35240039b0 2024-03-28T13:09:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.939+0000 7f353f400640 1 -- 172.21.15.19:0/2256800109 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3540141ec0 con 0x7f3524004800 2024-03-28T13:09:43.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.939+0000 7f353f400640 1 -- 172.21.15.19:0/2256800109 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f352400b9c0 con 0x7f35400b4af0 2024-03-28T13:09:43.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.939+0000 7f353fe00640 1 --2- 172.21.15.19:0/2256800109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35240039b0 0x7f3524003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:43.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.940+0000 7f3537400640 1 --2- 172.21.15.19:0/2256800109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3524004800 0x7f3524004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:43.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.940+0000 7f3545000640 1 --2- 172.21.15.19:0/2256800109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35400b4af0 0x7f35240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:43.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.940+0000 7f353fe00640 1 -- 172.21.15.19:0/2256800109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35400b4af0 msgr2=0x7f35240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:43.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.940+0000 7f353fe00640 1 --2- 172.21.15.19:0/2256800109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35400b4af0 0x7f35240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:43.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.940+0000 7f353fe00640 1 -- 172.21.15.19:0/2256800109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3524004800 msgr2=0x7f3524004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:43.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.940+0000 7f353fe00640 1 --2- 172.21.15.19:0/2256800109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3524004800 0x7f3524004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:43.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.940+0000 7f353fe00640 1 -- 172.21.15.19:0/2256800109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3524003520 con 0x7f35240039b0 2024-03-28T13:09:43.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.940+0000 7f3545000640 1 --2- 172.21.15.19:0/2256800109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35400b4af0 0x7f35240051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:43.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.941+0000 7f353fe00640 1 --2- 172.21.15.19:0/2256800109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35240039b0 0x7f3524003dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f353000a870 tx=0x7f3524005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcdbc9cf100a065 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:43.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.941+0000 7f353f400640 1 -- 172.21.15.19:0/2256800109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353000a4c0 con 0x7f35240039b0 2024-03-28T13:09:43.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.941+0000 7f353f400640 1 -- 172.21.15.19:0/2256800109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3530034630 con 0x7f35240039b0 2024-03-28T13:09:43.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.941+0000 7f353f400640 1 -- 172.21.15.19:0/2256800109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530034a20 con 0x7f35240039b0 2024-03-28T13:09:43.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.942+0000 7f3547000640 1 -- 172.21.15.19:0/2256800109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35240039b0 msgr2=0x7f3524003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:43.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.942+0000 7f3547000640 1 --2- 172.21.15.19:0/2256800109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35240039b0 0x7f3524003dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f353000a870 tx=0x7f3524005830 comp rx=0 tx=0).stop 2024-03-28T13:09:43.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.942+0000 7f3545000640 1 -- 172.21.15.19:0/2256800109 reap_dead start 2024-03-28T13:09:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.942+0000 7f3547000640 1 -- 172.21.15.19:0/2256800109 shutdown_connections 2024-03-28T13:09:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.942+0000 7f3547000640 1 -- 172.21.15.19:0/2256800109 >> 172.21.15.19:0/2256800109 conn(0x7f3540067670 msgr2=0x7f3540067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.942+0000 7f3547000640 1 -- 172.21.15.19:0/2256800109 shutdown_connections 2024-03-28T13:09:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.942+0000 7f3547000640 1 -- 172.21.15.19:0/2256800109 wait complete. 2024-03-28T13:09:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.945+0000 7f3547000640 1 Processor -- start 2024-03-28T13:09:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.945+0000 7f3547000640 1 -- start start 2024-03-28T13:09:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.946+0000 7f3547000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35240039b0 0x7f35401540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.946+0000 7f3547000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35400d1720 0x7f35400b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.946+0000 7f3547000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f354015efd0 0x7f354015f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.946+0000 7f3547000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35401429a0 con 0x7f35400d1720 2024-03-28T13:09:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.946+0000 7f3547000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f354015d4d0 con 0x7f354015efd0 2024-03-28T13:09:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.946+0000 7f3547000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f354015ca50 con 0x7f35240039b0 2024-03-28T13:09:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.947+0000 7f3537400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35400d1720 0x7f35400b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.947+0000 7f3537400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35400d1720 0x7f35400b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34194/0 (socket says 172.21.15.19:34194) 2024-03-28T13:09:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.947+0000 7f3537400640 1 -- 172.21.15.19:0/1138744641 learned_addr learned my addr 172.21.15.19:0/1138744641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.947+0000 7f3545000640 1 --2- 172.21.15.19:0/1138744641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f354015efd0 0x7f354015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.947+0000 7f3537400640 1 -- 172.21.15.19:0/1138744641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35240039b0 msgr2=0x7f35401540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.947+0000 7f353fe00640 1 --2- 172.21.15.19:0/1138744641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35240039b0 0x7f35401540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.947+0000 7f3537400640 1 --2- 172.21.15.19:0/1138744641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35240039b0 0x7f35401540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.947+0000 7f3537400640 1 -- 172.21.15.19:0/1138744641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f354015efd0 msgr2=0x7f354015f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.947+0000 7f3537400640 1 --2- 172.21.15.19:0/1138744641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f354015efd0 0x7f354015f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.947+0000 7f3537400640 1 -- 172.21.15.19:0/1138744641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35400d1c70 con 0x7f35400d1720 2024-03-28T13:09:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.947+0000 7f3545000640 1 --2- 172.21.15.19:0/1138744641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f354015efd0 0x7f354015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:43.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.948+0000 7f3537400640 1 --2- 172.21.15.19:0/1138744641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35400d1720 0x7f35400b5d50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3528011940 tx=0x7f3528011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:43.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.948+0000 7f353cc00640 1 -- 172.21.15.19:0/1138744641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352800fa90 con 0x7f35400d1720 2024-03-28T13:09:43.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.948+0000 7f353cc00640 1 -- 172.21.15.19:0/1138744641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f352800fc30 con 0x7f35400d1720 2024-03-28T13:09:43.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.948+0000 7f353cc00640 1 -- 172.21.15.19:0/1138744641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528014440 con 0x7f35400d1720 2024-03-28T13:09:43.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.948+0000 7f3547000640 1 -- 172.21.15.19:0/1138744641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35400d1f50 con 0x7f35400d1720 2024-03-28T13:09:43.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.948+0000 7f3547000640 1 -- 172.21.15.19:0/1138744641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35400d2510 con 0x7f35400d1720 2024-03-28T13:09:43.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.949+0000 7f3547000640 1 -- 172.21.15.19:0/1138744641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3540074f30 con 0x7f35400d1720 2024-03-28T13:09:43.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.950+0000 7f353cc00640 1 -- 172.21.15.19:0/1138744641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3528012040 con 0x7f35400d1720 2024-03-28T13:09:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.952+0000 7f353cc00640 1 --2- 172.21.15.19:0/1138744641 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f351c06c530 0x7f351c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.952+0000 7f353cc00640 1 -- 172.21.15.19:0/1138744641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f352808bd80 con 0x7f35400d1720 2024-03-28T13:09:43.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.952+0000 7f353fe00640 1 --2- 172.21.15.19:0/1138744641 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f351c06c530 0x7f351c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:43.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.953+0000 7f353fe00640 1 --2- 172.21.15.19:0/1138744641 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f351c06c530 0x7f351c06e9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3530034000 tx=0x7f3530009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:43.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:43.956+0000 7f353cc00640 1 -- 172.21.15.19:0/1138744641 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3528019030 con 0x7f35400d1720 2024-03-28T13:09:44.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.201+0000 7f3547000640 1 -- 172.21.15.19:0/1138744641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f35400685c0 con 0x7f35400d1720 2024-03-28T13:09:44.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.682+0000 7f353cc00640 1 -- 172.21.15.19:0/1138744641 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f352805a300 con 0x7f35400d1720 2024-03-28T13:09:44.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.686+0000 7f3547000640 1 -- 172.21.15.19:0/1138744641 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f351c06c530 msgr2=0x7f351c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:44.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.686+0000 7f3547000640 1 --2- 172.21.15.19:0/1138744641 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f351c06c530 0x7f351c06e9f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3530034000 tx=0x7f3530009030 comp rx=0 tx=0).stop 2024-03-28T13:09:44.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.686+0000 7f3547000640 1 -- 172.21.15.19:0/1138744641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35400d1720 msgr2=0x7f35400b5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:44.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.686+0000 7f3547000640 1 --2- 172.21.15.19:0/1138744641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35400d1720 0x7f35400b5d50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3528011940 tx=0x7f3528011e00 comp rx=0 tx=0).stop 2024-03-28T13:09:44.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.686+0000 7f3547000640 1 -- 172.21.15.19:0/1138744641 shutdown_connections 2024-03-28T13:09:44.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.687+0000 7f3547000640 1 --2- 172.21.15.19:0/1138744641 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f351c06c530 0x7f351c06e9f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:44.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.687+0000 7f3547000640 1 --2- 172.21.15.19:0/1138744641 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f354015efd0 0x7f354015f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:44.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.687+0000 7f3547000640 1 --2- 172.21.15.19:0/1138744641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35400d1720 0x7f35400b5d50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:44.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.687+0000 7f3547000640 1 --2- 172.21.15.19:0/1138744641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35240039b0 0x7f35401540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:44.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.687+0000 7f3547000640 1 -- 172.21.15.19:0/1138744641 >> 172.21.15.19:0/1138744641 conn(0x7f3540067670 msgr2=0x7f3540070190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:44.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.687+0000 7f3547000640 1 -- 172.21.15.19:0/1138744641 shutdown_connections 2024-03-28T13:09:44.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.687+0000 7f3547000640 1 -- 172.21.15.19:0/1138744641 wait complete. 2024-03-28T13:09:44.712 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-28T13:09:44.712 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:09:44.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.934+0000 7f8500a00640 1 Processor -- start 2024-03-28T13:09:44.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.934+0000 7f8500a00640 1 -- start start 2024-03-28T13:09:44.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.936+0000 7f8500a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f84fc15e4a0 0x7f84fc0b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:44.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.936+0000 7f8500a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84fc0d2310 0x7f84fc0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:44.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.936+0000 7f8500a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84fc15e880 con 0x7f84fc0cf530 2024-03-28T13:09:44.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.936+0000 7f8500a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f84fc0d34e0 con 0x7f84fc15e4a0 2024-03-28T13:09:44.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.936+0000 7f8500a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f84fc0d28d0 con 0x7f84fc0d2310 2024-03-28T13:09:44.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.937+0000 7f84f9e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84fc0d2310 0x7f84fc0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:44.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.937+0000 7f84f9e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84fc0d2310 0x7f84fc0ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:49186/0 (socket says 172.21.15.19:49186) 2024-03-28T13:09:44.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.937+0000 7f84f9e00640 1 -- 172.21.15.19:0/3195222220 learned_addr learned my addr 172.21.15.19:0/3195222220 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:44.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.937+0000 7f84fa800640 1 --2- 172.21.15.19:0/3195222220 >> v2:172.21.15.188:3300/0 conn(0x7f84fc15e4a0 0x7f84fc0b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:44.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.937+0000 7f84f9e00640 1 -- 172.21.15.19:0/3195222220 >> v2:172.21.15.188:3300/0 conn(0x7f84fc15e4a0 msgr2=0x7f84fc0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:44.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.937+0000 7f84f9e00640 1 --2- 172.21.15.19:0/3195222220 >> v2:172.21.15.188:3300/0 conn(0x7f84fc15e4a0 0x7f84fc0b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:44.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.937+0000 7f84f9e00640 1 -- 172.21.15.19:0/3195222220 >> v1:172.21.15.188:6789/0 conn(0x7f84fc0cf530 legacy=0x7f84fc0d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:44.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.937+0000 7f84f9e00640 1 -- 172.21.15.19:0/3195222220 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84fc0d2bb0 con 0x7f84fc0d2310 2024-03-28T13:09:44.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.939+0000 7f84f9e00640 1 --2- 172.21.15.19:0/3195222220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84fc0d2310 0x7f84fc0ceff0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009870 tx=0x7f84e802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa0856cf6ad80215 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.939+0000 7f84f9400640 1 -- 172.21.15.19:0/3195222220 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8034070 con 0x7f84fc0d2310 2024-03-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.939+0000 7f84f9400640 1 -- 172.21.15.19:0/3195222220 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e8030840 con 0x7f84fc0d2310 2024-03-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.939+0000 7f84f9400640 1 -- 172.21.15.19:0/3195222220 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8030c10 con 0x7f84fc0d2310 2024-03-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.940+0000 7f8500a00640 1 -- 172.21.15.19:0/3195222220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84fc0d2310 msgr2=0x7f84fc0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:44.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.940+0000 7f8500a00640 1 --2- 172.21.15.19:0/3195222220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84fc0d2310 0x7f84fc0ceff0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009870 tx=0x7f84e802b980 comp rx=0 tx=0).stop 2024-03-28T13:09:44.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.940+0000 7f8500a00640 1 -- 172.21.15.19:0/3195222220 shutdown_connections 2024-03-28T13:09:44.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.940+0000 7f8500a00640 1 --2- 172.21.15.19:0/3195222220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84fc0d2310 0x7f84fc0ceff0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:44.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.940+0000 7f8500a00640 1 --2- 172.21.15.19:0/3195222220 >> v2:172.21.15.188:3300/0 conn(0x7f84fc15e4a0 0x7f84fc0b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:44.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.940+0000 7f8500a00640 1 -- 172.21.15.19:0/3195222220 >> 172.21.15.19:0/3195222220 conn(0x7f84fc0676d0 msgr2=0x7f84fc067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:44.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.940+0000 7f8500a00640 1 -- 172.21.15.19:0/3195222220 shutdown_connections 2024-03-28T13:09:44.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.940+0000 7f8500a00640 1 -- 172.21.15.19:0/3195222220 wait complete. 2024-03-28T13:09:44.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.942+0000 7f8500a00640 1 Processor -- start 2024-03-28T13:09:44.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.943+0000 7f8500a00640 1 -- start start 2024-03-28T13:09:44.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.944+0000 7f8500a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84fc0cf530 0x7f84fc0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:44.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.944+0000 7f8500a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84fc0d2310 0x7f84fc1508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:44.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.944+0000 7f8500a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84fc15e4a0 0x7f84fc0ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:44.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.944+0000 7f8500a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc0d3970 con 0x7f84fc15e4a0 2024-03-28T13:09:44.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.944+0000 7f8500a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f84fc0d3170 con 0x7f84fc0d2310 2024-03-28T13:09:44.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.944+0000 7f8500a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f84fc0ba1a0 con 0x7f84fc0cf530 2024-03-28T13:09:44.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.944+0000 7f84fb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84fc15e4a0 0x7f84fc0ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:44.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.944+0000 7f84fb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84fc15e4a0 0x7f84fc0ca780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34198/0 (socket says 172.21.15.19:34198) 2024-03-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.944+0000 7f84fb200640 1 -- 172.21.15.19:0/4017458665 learned_addr learned my addr 172.21.15.19:0/4017458665 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.944+0000 7f84fa800640 1 --2- 172.21.15.19:0/4017458665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84fc0cf530 0x7f84fc0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.945+0000 7f84f9e00640 1 --2- 172.21.15.19:0/4017458665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84fc0d2310 0x7f84fc1508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.945+0000 7f84fb200640 1 -- 172.21.15.19:0/4017458665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84fc0cf530 msgr2=0x7f84fc0ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.945+0000 7f84fb200640 1 --2- 172.21.15.19:0/4017458665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84fc0cf530 0x7f84fc0ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.945+0000 7f84fb200640 1 -- 172.21.15.19:0/4017458665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84fc0d2310 msgr2=0x7f84fc1508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.945+0000 7f84fb200640 1 --2- 172.21.15.19:0/4017458665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84fc0d2310 0x7f84fc1508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.945+0000 7f84fb200640 1 -- 172.21.15.19:0/4017458665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84fc0c8970 con 0x7f84fc15e4a0 2024-03-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.945+0000 7f84fb200640 1 --2- 172.21.15.19:0/4017458665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84fc15e4a0 0x7f84fc0ca780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f84f400c6b0 tx=0x7f84f400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.945+0000 7f84f2a00640 1 -- 172.21.15.19:0/4017458665 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4003b70 con 0x7f84fc15e4a0 2024-03-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.945+0000 7f84f2a00640 1 -- 172.21.15.19:0/4017458665 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f4003d30 con 0x7f84fc15e4a0 2024-03-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.945+0000 7f8500a00640 1 -- 172.21.15.19:0/4017458665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84fc0c8c10 con 0x7f84fc15e4a0 2024-03-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.946+0000 7f84f2a00640 1 -- 172.21.15.19:0/4017458665 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4005310 con 0x7f84fc15e4a0 2024-03-28T13:09:44.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.946+0000 7f8500a00640 1 -- 172.21.15.19:0/4017458665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84fc0c91d0 con 0x7f84fc15e4a0 2024-03-28T13:09:44.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.947+0000 7f8500a00640 1 -- 172.21.15.19:0/4017458665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84fc075940 con 0x7f84fc15e4a0 2024-03-28T13:09:44.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.948+0000 7f84f2a00640 1 -- 172.21.15.19:0/4017458665 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f84f40054b0 con 0x7f84fc15e4a0 2024-03-28T13:09:44.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.949+0000 7f84f2a00640 1 --2- 172.21.15.19:0/4017458665 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f84dc06c530 0x7f84dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:44.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.949+0000 7f84f2a00640 1 -- 172.21.15.19:0/4017458665 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f84f408b580 con 0x7f84fc15e4a0 2024-03-28T13:09:44.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.949+0000 7f84fa800640 1 --2- 172.21.15.19:0/4017458665 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f84dc06c530 0x7f84dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.950+0000 7f84fa800640 1 --2- 172.21.15.19:0/4017458665 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f84dc06c530 0x7f84dc06e9f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009ad0 tx=0x7f84ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:44.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:44.953+0000 7f84f2a00640 1 -- 172.21.15.19:0/4017458665 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f84f4059ba0 con 0x7f84fc15e4a0 2024-03-28T13:09:45.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.194+0000 7f8500a00640 1 -- 172.21.15.19:0/4017458665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84fc15a7a0 con 0x7f84fc15e4a0 2024-03-28T13:09:45.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.195+0000 7f84f2a00640 1 -- 172.21.15.19:0/4017458665 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f84f40594a0 con 0x7f84fc15e4a0 2024-03-28T13:09:45.196 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:45.196 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/745791856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":745791856},{"type":"v1","addr":"172.21.15.19:6833","nonce":745791856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/639283827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":639283827},{"type":"v1","addr":"172.21.15.188:6836","nonce":639283827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2168352329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2168352329},{"type":"v1","addr":"172.21.15.188:6837","nonce":2168352329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:09:41.105429+0000","modified":"2024-03-28T13:09:44.203526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/731659331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":731659331},{"type":"v1","addr":"172.21.15.19:6835","nonce":731659331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4254,"qdb_cluster":[4254]},"id":1}]} 2024-03-28T13:09:45.196 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-03-28T13:09:45.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.198+0000 7f8500a00640 1 -- 172.21.15.19:0/4017458665 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f84dc06c530 msgr2=0x7f84dc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:45.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.198+0000 7f8500a00640 1 --2- 172.21.15.19:0/4017458665 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f84dc06c530 0x7f84dc06e9f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009ad0 tx=0x7f84ec008040 comp rx=0 tx=0).stop 2024-03-28T13:09:45.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.198+0000 7f8500a00640 1 -- 172.21.15.19:0/4017458665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84fc15e4a0 msgr2=0x7f84fc0ca780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:45.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.198+0000 7f8500a00640 1 --2- 172.21.15.19:0/4017458665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84fc15e4a0 0x7f84fc0ca780 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f84f400c6b0 tx=0x7f84f400cb70 comp rx=0 tx=0).stop 2024-03-28T13:09:45.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.198+0000 7f8500a00640 1 -- 172.21.15.19:0/4017458665 shutdown_connections 2024-03-28T13:09:45.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.198+0000 7f8500a00640 1 --2- 172.21.15.19:0/4017458665 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f84dc06c530 0x7f84dc06e9f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.198+0000 7f8500a00640 1 --2- 172.21.15.19:0/4017458665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84fc15e4a0 0x7f84fc0ca780 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.198+0000 7f8500a00640 1 --2- 172.21.15.19:0/4017458665 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84fc0d2310 0x7f84fc1508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.198+0000 7f8500a00640 1 --2- 172.21.15.19:0/4017458665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84fc0cf530 0x7f84fc0ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.198+0000 7f8500a00640 1 -- 172.21.15.19:0/4017458665 >> 172.21.15.19:0/4017458665 conn(0x7f84fc0676d0 msgr2=0x7f84fc15d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:45.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.198+0000 7f8500a00640 1 -- 172.21.15.19:0/4017458665 shutdown_connections 2024-03-28T13:09:45.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.198+0000 7f8500a00640 1 -- 172.21.15.19:0/4017458665 wait complete. 2024-03-28T13:09:45.218 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:09:45.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.433+0000 7f3239e00640 1 Processor -- start 2024-03-28T13:09:45.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.433+0000 7f3239e00640 1 -- start start 2024-03-28T13:09:45.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.434+0000 7f3239e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f32340d0d60 0x7f3234073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:45.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.434+0000 7f3239e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f323415fe80 con 0x7f323415c390 2024-03-28T13:09:45.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.434+0000 7f3239e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32340ad120 con 0x7f32340af830 2024-03-28T13:09:45.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.434+0000 7f3239e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f323415c770 con 0x7f32340d0d60 2024-03-28T13:09:45.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.434+0000 7f3232a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f323415c390 0x7f3234160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42870/0 (socket says 172.21.15.19:42870) 2024-03-28T13:09:45.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.434+0000 7f3232a00640 1 -- 172.21.15.19:0/2496594111 learned_addr learned my addr 172.21.15.19:0/2496594111 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:45.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3233400640 1 --2- 172.21.15.19:0/2496594111 >> v2:172.21.15.188:3300/0 conn(0x7f32340d0d60 0x7f3234073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:45.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3232000640 1 -- 172.21.15.19:0/2496594111 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2499974374 0 0) 0x7f323415fe80 con 0x7f323415c390 2024-03-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3232000640 1 -- 172.21.15.19:0/2496594111 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3214003520 con 0x7f323415c390 2024-03-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3232000640 1 -- 172.21.15.19:0/2496594111 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 715561175 0 0) 0x7f3214003520 con 0x7f323415c390 2024-03-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3232000640 1 -- 172.21.15.19:0/2496594111 >> v2:172.21.15.188:3300/0 conn(0x7f32340d0d60 msgr2=0x7f3234073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3232000640 1 --2- 172.21.15.19:0/2496594111 >> v2:172.21.15.188:3300/0 conn(0x7f32340d0d60 0x7f3234073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3232000640 1 -- 172.21.15.19:0/2496594111 >> v1:172.21.15.188:6789/0 conn(0x7f32340af830 legacy=0x7f32340d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3232000640 1 -- 172.21.15.19:0/2496594111 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32340ad400 con 0x7f323415c390 2024-03-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3232000640 1 -- 172.21.15.19:0/2496594111 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f3224001690 con 0x7f323415c390 2024-03-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3232000640 1 -- 172.21.15.19:0/2496594111 >> v1:172.21.15.19:6789/0 conn(0x7f323415c390 legacy=0x7f3234160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3232000640 1 --2- 172.21.15.19:0/2496594111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32140039b0 0x7f3214003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3232000640 1 --2- 172.21.15.19:0/2496594111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3214004800 0x7f3214004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3232000640 1 --2- 172.21.15.19:0/2496594111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32140051c0 0x7f321400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3232000640 1 -- 172.21.15.19:0/2496594111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f321400c030 con 0x7f32140039b0 2024-03-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3232000640 1 -- 172.21.15.19:0/2496594111 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f321400c1e0 con 0x7f3214004800 2024-03-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.435+0000 7f3232000640 1 -- 172.21.15.19:0/2496594111 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f321400c3e0 con 0x7f32140051c0 2024-03-28T13:09:45.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.436+0000 7f3232a00640 1 --2- 172.21.15.19:0/2496594111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32140039b0 0x7f3214003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.436+0000 7f3232a00640 1 -- 172.21.15.19:0/2496594111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32140051c0 msgr2=0x7f321400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.436+0000 7f3232a00640 1 --2- 172.21.15.19:0/2496594111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32140051c0 0x7f321400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.436+0000 7f322a000640 1 --2- 172.21.15.19:0/2496594111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3214004800 0x7f3214004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.436+0000 7f3232a00640 1 -- 172.21.15.19:0/2496594111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3214004800 msgr2=0x7f3214004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.436+0000 7f3232a00640 1 --2- 172.21.15.19:0/2496594111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3214004800 0x7f3214004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.436+0000 7f3232a00640 1 -- 172.21.15.19:0/2496594111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3214003520 con 0x7f32140039b0 2024-03-28T13:09:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.436+0000 7f3233400640 1 -- 172.21.15.19:0/2496594111 reap_dead start 2024-03-28T13:09:45.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.436+0000 7f322a000640 1 --2- 172.21.15.19:0/2496594111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3214004800 0x7f3214004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:45.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.437+0000 7f3232a00640 1 --2- 172.21.15.19:0/2496594111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32140039b0 0x7f3214003dd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f322400c210 tx=0x7f3214005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ef696a8628be0b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:45.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.437+0000 7f3232000640 1 -- 172.21.15.19:0/2496594111 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224007d10 con 0x7f32140039b0 2024-03-28T13:09:45.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.437+0000 7f3232000640 1 -- 172.21.15.19:0/2496594111 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3224030070 con 0x7f32140039b0 2024-03-28T13:09:45.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.437+0000 7f3232000640 1 -- 172.21.15.19:0/2496594111 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224005d70 con 0x7f32140039b0 2024-03-28T13:09:45.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.438+0000 7f3239e00640 1 -- 172.21.15.19:0/2496594111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32140039b0 msgr2=0x7f3214003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:45.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.438+0000 7f3239e00640 1 --2- 172.21.15.19:0/2496594111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32140039b0 0x7f3214003dd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f322400c210 tx=0x7f3214005830 comp rx=0 tx=0).stop 2024-03-28T13:09:45.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.438+0000 7f3239e00640 1 -- 172.21.15.19:0/2496594111 shutdown_connections 2024-03-28T13:09:45.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.438+0000 7f3239e00640 1 --2- 172.21.15.19:0/2496594111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32140039b0 0x7f3214003dd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.438+0000 7f3239e00640 1 -- 172.21.15.19:0/2496594111 >> 172.21.15.19:0/2496594111 conn(0x7f323406b990 msgr2=0x7f323406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:45.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.438+0000 7f3239e00640 1 -- 172.21.15.19:0/2496594111 shutdown_connections 2024-03-28T13:09:45.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.438+0000 7f3239e00640 1 -- 172.21.15.19:0/2496594111 wait complete. 2024-03-28T13:09:45.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.440+0000 7f3239e00640 1 Processor -- start 2024-03-28T13:09:45.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.440+0000 7f3239e00640 1 -- start start 2024-03-28T13:09:45.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.441+0000 7f3239e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32140039b0 0x7f32341456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:45.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.441+0000 7f3239e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32341593b0 0x7f323415b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:45.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.441+0000 7f3239e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32341e52c0 0x7f32341e5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:45.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.441+0000 7f3239e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32340cf980 con 0x7f32140039b0 2024-03-28T13:09:45.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.441+0000 7f3239e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3234160120 con 0x7f32341e52c0 2024-03-28T13:09:45.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.441+0000 7f3239e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3234073790 con 0x7f32341593b0 2024-03-28T13:09:45.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.441+0000 7f3232a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32140039b0 0x7f32341456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:45.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.441+0000 7f3232a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32140039b0 0x7f32341456b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50534/0 (socket says 172.21.15.19:50534) 2024-03-28T13:09:45.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.441+0000 7f3232a00640 1 -- 172.21.15.19:0/640589476 learned_addr learned my addr 172.21.15.19:0/640589476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.442+0000 7f322a000640 1 --2- 172.21.15.19:0/640589476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32341593b0 0x7f323415b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.442+0000 7f3233400640 1 --2- 172.21.15.19:0/640589476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32341e52c0 0x7f32341e5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.442+0000 7f3232a00640 1 -- 172.21.15.19:0/640589476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32341593b0 msgr2=0x7f323415b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.442+0000 7f3232a00640 1 --2- 172.21.15.19:0/640589476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32341593b0 0x7f323415b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.442+0000 7f3232a00640 1 -- 172.21.15.19:0/640589476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32341e52c0 msgr2=0x7f32341e5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.442+0000 7f3232a00640 1 --2- 172.21.15.19:0/640589476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32341e52c0 0x7f32341e5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.442+0000 7f3232a00640 1 -- 172.21.15.19:0/640589476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f323415b990 con 0x7f32140039b0 2024-03-28T13:09:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.442+0000 7f3232a00640 1 --2- 172.21.15.19:0/640589476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32140039b0 0x7f32341456b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f322400c330 tx=0x7f32340ad2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.443+0000 7f322be00640 1 -- 172.21.15.19:0/640589476 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224046070 con 0x7f32140039b0 2024-03-28T13:09:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.443+0000 7f322be00640 1 -- 172.21.15.19:0/640589476 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3224000ed0 con 0x7f32140039b0 2024-03-28T13:09:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.443+0000 7f322be00640 1 -- 172.21.15.19:0/640589476 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224001c50 con 0x7f32140039b0 2024-03-28T13:09:45.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.443+0000 7f3239e00640 1 -- 172.21.15.19:0/640589476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3234159840 con 0x7f32140039b0 2024-03-28T13:09:45.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.443+0000 7f3239e00640 1 -- 172.21.15.19:0/640589476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3234159e00 con 0x7f32140039b0 2024-03-28T13:09:45.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.444+0000 7f3239e00640 1 -- 172.21.15.19:0/640589476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32340685c0 con 0x7f32140039b0 2024-03-28T13:09:45.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.445+0000 7f322be00640 1 -- 172.21.15.19:0/640589476 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3224007070 con 0x7f32140039b0 2024-03-28T13:09:45.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.446+0000 7f322be00640 1 --2- 172.21.15.19:0/640589476 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f321006c530 0x7f321006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:45.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.446+0000 7f322be00640 1 -- 172.21.15.19:0/640589476 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f32240ac9f0 con 0x7f32140039b0 2024-03-28T13:09:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.447+0000 7f322a000640 1 --2- 172.21.15.19:0/640589476 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f321006c530 0x7f321006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.448+0000 7f322a000640 1 --2- 172.21.15.19:0/640589476 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f321006c530 0x7f321006e9f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f323415f600 tx=0x7f321c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.450+0000 7f322be00640 1 -- 172.21.15.19:0/640589476 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f322407a920 con 0x7f32140039b0 2024-03-28T13:09:45.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.654+0000 7f3239e00640 1 -- 172.21.15.19:0/640589476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f323406f1b0 con 0x7f32140039b0 2024-03-28T13:09:45.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.657+0000 7f322be00640 1 -- 172.21.15.19:0/640589476 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16609 (secure 0 0 0) 0x7f322407ec20 con 0x7f32140039b0 2024-03-28T13:09:45.658 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:45.658 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":23,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:42.668077+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:09:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.660+0000 7f3239e00640 1 -- 172.21.15.19:0/640589476 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f321006c530 msgr2=0x7f321006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.660+0000 7f3239e00640 1 --2- 172.21.15.19:0/640589476 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f321006c530 0x7f321006e9f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f323415f600 tx=0x7f321c009040 comp rx=0 tx=0).stop 2024-03-28T13:09:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.660+0000 7f3239e00640 1 -- 172.21.15.19:0/640589476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32140039b0 msgr2=0x7f32341456b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.660+0000 7f3239e00640 1 --2- 172.21.15.19:0/640589476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32140039b0 0x7f32341456b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f322400c330 tx=0x7f32340ad2f0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.660+0000 7f3239e00640 1 -- 172.21.15.19:0/640589476 shutdown_connections 2024-03-28T13:09:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.660+0000 7f3239e00640 1 --2- 172.21.15.19:0/640589476 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f321006c530 0x7f321006e9f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.660+0000 7f3239e00640 1 --2- 172.21.15.19:0/640589476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32341e52c0 0x7f32341e5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.660+0000 7f3239e00640 1 --2- 172.21.15.19:0/640589476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32341593b0 0x7f323415b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.660+0000 7f3239e00640 1 --2- 172.21.15.19:0/640589476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32140039b0 0x7f32341456b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.660+0000 7f3239e00640 1 -- 172.21.15.19:0/640589476 >> 172.21.15.19:0/640589476 conn(0x7f323406b990 msgr2=0x7f3234154950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.660+0000 7f3239e00640 1 -- 172.21.15.19:0/640589476 shutdown_connections 2024-03-28T13:09:45.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.661+0000 7f3239e00640 1 -- 172.21.15.19:0/640589476 wait complete. 2024-03-28T13:09:45.681 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-28T13:09:45.681 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:09:45.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.895+0000 7f57ffc00640 1 Processor -- start 2024-03-28T13:09:45.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.896+0000 7f57ffc00640 1 -- start start 2024-03-28T13:09:45.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.898+0000 7f57ffc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f57f80d2bf0 0x7f57f80cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:45.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.899+0000 7f57ffc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f57f80d0c30 0x7f57f80d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:45.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.899+0000 7f57ffc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57f80d3140 con 0x7f57f815fa10 2024-03-28T13:09:45.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.899+0000 7f57ffc00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f57f80d15e0 con 0x7f57f80d2bf0 2024-03-28T13:09:45.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.899+0000 7f57ffc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f57f80d3420 con 0x7f57f80d0c30 2024-03-28T13:09:45.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.899+0000 7f57fd200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f57f815fa10 0x7f57f80b55a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42876/0 (socket says 172.21.15.19:42876) 2024-03-28T13:09:45.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.899+0000 7f57fd200640 1 -- 172.21.15.19:0/635163032 learned_addr learned my addr 172.21.15.19:0/635163032 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:45.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.899+0000 7f57f7e00640 1 --2- 172.21.15.19:0/635163032 >> v2:172.21.15.19:3300/0 conn(0x7f57f80d2bf0 0x7f57f80cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:45.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.899+0000 7f57fdc00640 1 --2- 172.21.15.19:0/635163032 >> v2:172.21.15.188:3300/0 conn(0x7f57f80d0c30 0x7f57f80d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:45.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.899+0000 7f57f7400640 1 -- 172.21.15.19:0/635163032 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1665515668 0 0) 0x7f57f80d3140 con 0x7f57f815fa10 2024-03-28T13:09:45.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.899+0000 7f57f7e00640 1 -- 172.21.15.19:0/635163032 >> v2:172.21.15.188:3300/0 conn(0x7f57f80d0c30 msgr2=0x7f57f80d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:45.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.899+0000 7f57f7e00640 1 --2- 172.21.15.19:0/635163032 >> v2:172.21.15.188:3300/0 conn(0x7f57f80d0c30 0x7f57f80d10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.899+0000 7f57f7e00640 1 -- 172.21.15.19:0/635163032 >> v1:172.21.15.19:6789/0 conn(0x7f57f815fa10 legacy=0x7f57f80b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:45.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.900+0000 7f57f7e00640 1 -- 172.21.15.19:0/635163032 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f80cf2b0 con 0x7f57f80d2bf0 2024-03-28T13:09:45.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.901+0000 7f57f7e00640 1 --2- 172.21.15.19:0/635163032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57f80d2bf0 0x7f57f80cfd90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f57f80cb4d0 tx=0x7f57e402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd1756f79a7d52b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:45.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.901+0000 7f57f7400640 1 -- 172.21.15.19:0/635163032 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e4034070 con 0x7f57f80d2bf0 2024-03-28T13:09:45.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.901+0000 7f57f7400640 1 -- 172.21.15.19:0/635163032 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e4030610 con 0x7f57f80d2bf0 2024-03-28T13:09:45.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.901+0000 7f57f7400640 1 -- 172.21.15.19:0/635163032 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e40309e0 con 0x7f57f80d2bf0 2024-03-28T13:09:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.902+0000 7f57ffc00640 1 -- 172.21.15.19:0/635163032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57f80d2bf0 msgr2=0x7f57f80cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.902+0000 7f57ffc00640 1 --2- 172.21.15.19:0/635163032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57f80d2bf0 0x7f57f80cfd90 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f57f80cb4d0 tx=0x7f57e402b960 comp rx=0 tx=0).stop 2024-03-28T13:09:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.902+0000 7f57ffc00640 1 -- 172.21.15.19:0/635163032 shutdown_connections 2024-03-28T13:09:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.902+0000 7f57ffc00640 1 --2- 172.21.15.19:0/635163032 >> v2:172.21.15.188:3300/0 conn(0x7f57f80d0c30 0x7f57f80d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.902+0000 7f57ffc00640 1 --2- 172.21.15.19:0/635163032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57f80d2bf0 0x7f57f80cfd90 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.902+0000 7f57ffc00640 1 -- 172.21.15.19:0/635163032 >> 172.21.15.19:0/635163032 conn(0x7f57f808bfb0 msgr2=0x7f57f80d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.902+0000 7f57ffc00640 1 -- 172.21.15.19:0/635163032 shutdown_connections 2024-03-28T13:09:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.902+0000 7f57ffc00640 1 -- 172.21.15.19:0/635163032 wait complete. 2024-03-28T13:09:45.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.904+0000 7f57ffc00640 1 Processor -- start 2024-03-28T13:09:45.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.904+0000 7f57ffc00640 1 -- start start 2024-03-28T13:09:45.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.905+0000 7f57ffc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57f80d0c30 0x7f57f80ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:45.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.905+0000 7f57ffc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57f80d2bf0 0x7f57f81496d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:45.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.905+0000 7f57ffc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57f815fa10 0x7f57f8149c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:45.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.905+0000 7f57ffc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f57f80d02d0 con 0x7f57f80d0c30 2024-03-28T13:09:45.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.905+0000 7f57ffc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f57f80cf870 con 0x7f57f815fa10 2024-03-28T13:09:45.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.905+0000 7f57ffc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f57f80d0450 con 0x7f57f80d2bf0 2024-03-28T13:09:45.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.905+0000 7f57fd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57f80d0c30 0x7f57f80ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:45.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.905+0000 7f57fd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57f80d0c30 0x7f57f80ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50558/0 (socket says 172.21.15.19:50558) 2024-03-28T13:09:45.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.905+0000 7f57fd200640 1 -- 172.21.15.19:0/1899837944 learned_addr learned my addr 172.21.15.19:0/1899837944 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:45.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.906+0000 7f57f7e00640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57f80d2bf0 0x7f57f81496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:45.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.906+0000 7f57fdc00640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57f815fa10 0x7f57f8149c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:45.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.906+0000 7f57fd200640 1 -- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57f80d2bf0 msgr2=0x7f57f81496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:45.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.906+0000 7f57fd200640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57f80d2bf0 0x7f57f81496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.906+0000 7f57fd200640 1 -- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57f815fa10 msgr2=0x7f57f8149c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:45.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.906+0000 7f57fd200640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57f815fa10 0x7f57f8149c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:45.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.906+0000 7f57fd200640 1 -- 172.21.15.19:0/1899837944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f80cdc40 con 0x7f57f80d0c30 2024-03-28T13:09:45.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.906+0000 7f57f7e00640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57f80d2bf0 0x7f57f81496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:45.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.906+0000 7f57fdc00640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57f815fa10 0x7f57f8149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:45.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.906+0000 7f57fd200640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57f80d0c30 0x7f57f80ca1e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00c430 tx=0x7f57ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:45.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.906+0000 7f57f4c00640 1 -- 172.21.15.19:0/1899837944 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0074e0 con 0x7f57f80d0c30 2024-03-28T13:09:45.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.906+0000 7f57f4c00640 1 -- 172.21.15.19:0/1899837944 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ec0047e0 con 0x7f57f80d0c30 2024-03-28T13:09:45.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.906+0000 7f57f4c00640 1 -- 172.21.15.19:0/1899837944 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec004bd0 con 0x7f57f80d0c30 2024-03-28T13:09:45.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.906+0000 7f57ffc00640 1 -- 172.21.15.19:0/1899837944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f80cdf20 con 0x7f57f80d0c30 2024-03-28T13:09:45.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.906+0000 7f57ffc00640 1 -- 172.21.15.19:0/1899837944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f80ce4e0 con 0x7f57f80d0c30 2024-03-28T13:09:45.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.908+0000 7f57ffc00640 1 -- 172.21.15.19:0/1899837944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f8068cd0 con 0x7f57f80d0c30 2024-03-28T13:09:45.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.909+0000 7f57f4c00640 1 -- 172.21.15.19:0/1899837944 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f57ec011070 con 0x7f57f80d0c30 2024-03-28T13:09:45.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.910+0000 7f57f4c00640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f57d806c530 0x7f57d806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:45.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.910+0000 7f57f4c00640 1 -- 172.21.15.19:0/1899837944 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f57ec08b430 con 0x7f57f80d0c30 2024-03-28T13:09:45.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.910+0000 7f57f7e00640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f57d806c530 0x7f57d806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:45.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.911+0000 7f57f7e00640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f57d806c530 0x7f57d806e9f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f57e4036000 tx=0x7f57e4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:45.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:45.914+0000 7f57f4c00640 1 -- 172.21.15.19:0/1899837944 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f57ec0599b0 con 0x7f57f80d0c30 2024-03-28T13:09:46.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.113+0000 7f57ffc00640 1 -- 172.21.15.19:0/1899837944 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f57f80ba3d0 con 0x7f57d806c530 2024-03-28T13:09:46.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.136+0000 7f57f4c00640 1 -- 172.21.15.19:0/1899837944 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251941 (secure 0 0 0) 0x7f57f80ba3d0 con 0x7f57d806c530 2024-03-28T13:09:46.140 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:46.141 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:09:46.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.141+0000 7f57ffc00640 1 -- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f57d806c530 msgr2=0x7f57d806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:46.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.141+0000 7f57ffc00640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f57d806c530 0x7f57d806e9f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f57e4036000 tx=0x7f57e4008030 comp rx=0 tx=0).stop 2024-03-28T13:09:46.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.141+0000 7f57ffc00640 1 -- 172.21.15.19:0/1899837944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57f80d0c30 msgr2=0x7f57f80ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:46.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.141+0000 7f57ffc00640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57f80d0c30 0x7f57f80ca1e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00c430 tx=0x7f57ec00c8f0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.141+0000 7f57ffc00640 1 -- 172.21.15.19:0/1899837944 shutdown_connections 2024-03-28T13:09:46.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.141+0000 7f57ffc00640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f57d806c530 0x7f57d806e9f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.141+0000 7f57ffc00640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57f815fa10 0x7f57f8149c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.141+0000 7f57ffc00640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57f80d2bf0 0x7f57f81496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.141+0000 7f57ffc00640 1 --2- 172.21.15.19:0/1899837944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57f80d0c30 0x7f57f80ca1e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.141+0000 7f57ffc00640 1 -- 172.21.15.19:0/1899837944 >> 172.21.15.19:0/1899837944 conn(0x7f57f808bfb0 msgr2=0x7f57f806dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:46.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.142+0000 7f57ffc00640 1 -- 172.21.15.19:0/1899837944 shutdown_connections 2024-03-28T13:09:46.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.142+0000 7f57ffc00640 1 -- 172.21.15.19:0/1899837944 wait complete. 2024-03-28T13:09:46.160 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-03-28T13:09:45.982324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219172,"kb_used_data":3004,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755548,"statfs":{"total":773094113280,"available":772869681152,"internally_reserved":0,"allocated":3076096,"data_stored":1817936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001886"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696286+0000","last_change":"2024-03-28T13:09:41.690249+0000","last_active":"2024-03-28T13:09:42.696286+0000","last_peered":"2024-03-28T13:09:42.696286+0000","last_clean":"2024-03-28T13:09:42.696286+0000","last_became_active":"2024-03-28T13:09:41.689984+0000","last_became_peered":"2024-03-28T13:09:41.689984+0000","last_unstale":"2024-03-28T13:09:42.696286+0000","last_undegraded":"2024-03-28T13:09:42.696286+0000","last_fullsized":"2024-03-28T13:09:42.696286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:53:08.763850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687950+0000","last_change":"2024-03-28T13:09:41.669236+0000","last_active":"2024-03-28T13:09:42.687950+0000","last_peered":"2024-03-28T13:09:42.687950+0000","last_clean":"2024-03-28T13:09:42.687950+0000","last_became_active":"2024-03-28T13:09:41.668886+0000","last_became_peered":"2024-03-28T13:09:41.668886+0000","last_unstale":"2024-03-28T13:09:42.687950+0000","last_undegraded":"2024-03-28T13:09:42.687950+0000","last_fullsized":"2024-03-28T13:09:42.687950+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:53:08.763850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687498+0000","last_change":"2024-03-28T13:09:41.683573+0000","last_active":"2024-03-28T13:09:42.687498+0000","last_peered":"2024-03-28T13:09:42.687498+0000","last_clean":"2024-03-28T13:09:42.687498+0000","last_became_active":"2024-03-28T13:09:41.682846+0000","last_became_peered":"2024-03-28T13:09:41.682846+0000","last_unstale":"2024-03-28T13:09:42.687498+0000","last_undegraded":"2024-03-28T13:09:42.687498+0000","last_fullsized":"2024-03-28T13:09:42.687498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:04.694363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680562+0000","last_change":"2024-03-28T13:09:41.684558+0000","last_active":"2024-03-28T13:09:42.680562+0000","last_peered":"2024-03-28T13:09:42.680562+0000","last_clean":"2024-03-28T13:09:42.680562+0000","last_became_active":"2024-03-28T13:09:41.682564+0000","last_became_peered":"2024-03-28T13:09:41.682564+0000","last_unstale":"2024-03-28T13:09:42.680562+0000","last_undegraded":"2024-03-28T13:09:42.680562+0000","last_fullsized":"2024-03-28T13:09:42.680562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:21:17.844127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695969+0000","last_change":"2024-03-28T13:09:41.687175+0000","last_active":"2024-03-28T13:09:42.695969+0000","last_peered":"2024-03-28T13:09:42.695969+0000","last_clean":"2024-03-28T13:09:42.695969+0000","last_became_active":"2024-03-28T13:09:41.686835+0000","last_became_peered":"2024-03-28T13:09:41.686835+0000","last_unstale":"2024-03-28T13:09:42.695969+0000","last_undegraded":"2024-03-28T13:09:42.695969+0000","last_fullsized":"2024-03-28T13:09:42.695969+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:04.694363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.679923+0000","last_change":"2024-03-28T13:09:41.683515+0000","last_active":"2024-03-28T13:09:42.679923+0000","last_peered":"2024-03-28T13:09:42.679923+0000","last_clean":"2024-03-28T13:09:42.679923+0000","last_became_active":"2024-03-28T13:09:41.682795+0000","last_became_peered":"2024-03-28T13:09:41.682795+0000","last_unstale":"2024-03-28T13:09:42.679923+0000","last_undegraded":"2024-03-28T13:09:42.679923+0000","last_fullsized":"2024-03-28T13:09:42.679923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:29.551203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695673+0000","last_change":"2024-03-28T13:09:41.690393+0000","last_active":"2024-03-28T13:09:42.695673+0000","last_peered":"2024-03-28T13:09:42.695673+0000","last_clean":"2024-03-28T13:09:42.695673+0000","last_became_active":"2024-03-28T13:09:41.688973+0000","last_became_peered":"2024-03-28T13:09:41.688973+0000","last_unstale":"2024-03-28T13:09:42.695673+0000","last_undegraded":"2024-03-28T13:09:42.695673+0000","last_fullsized":"2024-03-28T13:09:42.695673+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:48.181534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686877+0000","last_change":"2024-03-28T13:09:41.691319+0000","last_active":"2024-03-28T13:09:42.686877+0000","last_peered":"2024-03-28T13:09:42.686877+0000","last_clean":"2024-03-28T13:09:42.686877+0000","last_became_active":"2024-03-28T13:09:41.690506+0000","last_became_peered":"2024-03-28T13:09:41.690506+0000","last_unstale":"2024-03-28T13:09:42.686877+0000","last_undegraded":"2024-03-28T13:09:42.686877+0000","last_fullsized":"2024-03-28T13:09:42.686877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:28:13.777057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942467+0000","last_change":"2024-03-28T13:09:41.697845+0000","last_active":"2024-03-28T13:09:42.942467+0000","last_peered":"2024-03-28T13:09:42.942467+0000","last_clean":"2024-03-28T13:09:42.942467+0000","last_became_active":"2024-03-28T13:09:41.696890+0000","last_became_peered":"2024-03-28T13:09:41.696890+0000","last_unstale":"2024-03-28T13:09:42.942467+0000","last_undegraded":"2024-03-28T13:09:42.942467+0000","last_fullsized":"2024-03-28T13:09:42.942467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:37.029859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696276+0000","last_change":"2024-03-28T13:09:41.688074+0000","last_active":"2024-03-28T13:09:42.696276+0000","last_peered":"2024-03-28T13:09:42.696276+0000","last_clean":"2024-03-28T13:09:42.696276+0000","last_became_active":"2024-03-28T13:09:41.687755+0000","last_became_peered":"2024-03-28T13:09:41.687755+0000","last_unstale":"2024-03-28T13:09:42.696276+0000","last_undegraded":"2024-03-28T13:09:42.696276+0000","last_fullsized":"2024-03-28T13:09:42.696276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.673442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695935+0000","last_change":"2024-03-28T13:09:41.693370+0000","last_active":"2024-03-28T13:09:42.695935+0000","last_peered":"2024-03-28T13:09:42.695935+0000","last_clean":"2024-03-28T13:09:42.695935+0000","last_became_active":"2024-03-28T13:09:41.690289+0000","last_became_peered":"2024-03-28T13:09:41.690289+0000","last_unstale":"2024-03-28T13:09:42.695935+0000","last_undegraded":"2024-03-28T13:09:42.695935+0000","last_fullsized":"2024-03-28T13:09:42.695935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:03:17.731528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696467+0000","last_change":"2024-03-28T13:09:41.692663+0000","last_active":"2024-03-28T13:09:42.696467+0000","last_peered":"2024-03-28T13:09:42.696467+0000","last_clean":"2024-03-28T13:09:42.696467+0000","last_became_active":"2024-03-28T13:09:41.692130+0000","last_became_peered":"2024-03-28T13:09:41.692130+0000","last_unstale":"2024-03-28T13:09:42.696467+0000","last_undegraded":"2024-03-28T13:09:42.696467+0000","last_fullsized":"2024-03-28T13:09:42.696467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:38.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945637+0000","last_change":"2024-03-28T13:09:41.692414+0000","last_active":"2024-03-28T13:09:42.945637+0000","last_peered":"2024-03-28T13:09:42.945637+0000","last_clean":"2024-03-28T13:09:42.945637+0000","last_became_active":"2024-03-28T13:09:41.692105+0000","last_became_peered":"2024-03-28T13:09:41.692105+0000","last_unstale":"2024-03-28T13:09:42.945637+0000","last_undegraded":"2024-03-28T13:09:42.945637+0000","last_fullsized":"2024-03-28T13:09:42.945637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:21:17.844127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.946036+0000","last_change":"2024-03-28T13:09:41.675736+0000","last_active":"2024-03-28T13:09:42.946036+0000","last_peered":"2024-03-28T13:09:42.946036+0000","last_clean":"2024-03-28T13:09:42.946036+0000","last_became_active":"2024-03-28T13:09:41.675120+0000","last_became_peered":"2024-03-28T13:09:41.675120+0000","last_unstale":"2024-03-28T13:09:42.946036+0000","last_undegraded":"2024-03-28T13:09:42.946036+0000","last_fullsized":"2024-03-28T13:09:42.946036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:16.926876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686621+0000","last_change":"2024-03-28T13:09:41.690417+0000","last_active":"2024-03-28T13:09:42.686621+0000","last_peered":"2024-03-28T13:09:42.686621+0000","last_clean":"2024-03-28T13:09:42.686621+0000","last_became_active":"2024-03-28T13:09:41.689467+0000","last_became_peered":"2024-03-28T13:09:41.689467+0000","last_unstale":"2024-03-28T13:09:42.686621+0000","last_undegraded":"2024-03-28T13:09:42.686621+0000","last_fullsized":"2024-03-28T13:09:42.686621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:50:46.504501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684397+0000","last_change":"2024-03-28T13:09:41.684876+0000","last_active":"2024-03-28T13:09:42.684397+0000","last_peered":"2024-03-28T13:09:42.684397+0000","last_clean":"2024-03-28T13:09:42.684397+0000","last_became_active":"2024-03-28T13:09:41.683988+0000","last_became_peered":"2024-03-28T13:09:41.683988+0000","last_unstale":"2024-03-28T13:09:42.684397+0000","last_undegraded":"2024-03-28T13:09:42.684397+0000","last_fullsized":"2024-03-28T13:09:42.684397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:38.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685867+0000","last_change":"2024-03-28T13:09:41.702219+0000","last_active":"2024-03-28T13:09:42.685867+0000","last_peered":"2024-03-28T13:09:42.685867+0000","last_clean":"2024-03-28T13:09:42.685867+0000","last_became_active":"2024-03-28T13:09:41.701907+0000","last_became_peered":"2024-03-28T13:09:41.701907+0000","last_unstale":"2024-03-28T13:09:42.685867+0000","last_undegraded":"2024-03-28T13:09:42.685867+0000","last_fullsized":"2024-03-28T13:09:42.685867+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:49:20.779575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944358+0000","last_change":"2024-03-28T13:09:41.683832+0000","last_active":"2024-03-28T13:09:42.944358+0000","last_peered":"2024-03-28T13:09:42.944358+0000","last_clean":"2024-03-28T13:09:42.944358+0000","last_became_active":"2024-03-28T13:09:41.683495+0000","last_became_peered":"2024-03-28T13:09:41.683495+0000","last_unstale":"2024-03-28T13:09:42.944358+0000","last_undegraded":"2024-03-28T13:09:42.944358+0000","last_fullsized":"2024-03-28T13:09:42.944358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:44:52.163706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685573+0000","last_change":"2024-03-28T13:09:41.688514+0000","last_active":"2024-03-28T13:09:42.685573+0000","last_peered":"2024-03-28T13:09:42.685573+0000","last_clean":"2024-03-28T13:09:42.685573+0000","last_became_active":"2024-03-28T13:09:41.688158+0000","last_became_peered":"2024-03-28T13:09:41.688158+0000","last_unstale":"2024-03-28T13:09:42.685573+0000","last_undegraded":"2024-03-28T13:09:42.685573+0000","last_fullsized":"2024-03-28T13:09:42.685573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:39:02.380052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696639+0000","last_change":"2024-03-28T13:09:41.695024+0000","last_active":"2024-03-28T13:09:42.696639+0000","last_peered":"2024-03-28T13:09:42.696639+0000","last_clean":"2024-03-28T13:09:42.696639+0000","last_became_active":"2024-03-28T13:09:41.694698+0000","last_became_peered":"2024-03-28T13:09:41.694698+0000","last_unstale":"2024-03-28T13:09:42.696639+0000","last_undegraded":"2024-03-28T13:09:42.696639+0000","last_fullsized":"2024-03-28T13:09:42.696639+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:38:26.092780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691415+0000","last_change":"2024-03-28T13:09:41.686281+0000","last_active":"2024-03-28T13:09:42.691415+0000","last_peered":"2024-03-28T13:09:42.691415+0000","last_clean":"2024-03-28T13:09:42.691415+0000","last_became_active":"2024-03-28T13:09:41.685221+0000","last_became_peered":"2024-03-28T13:09:41.685221+0000","last_unstale":"2024-03-28T13:09:42.691415+0000","last_undegraded":"2024-03-28T13:09:42.691415+0000","last_fullsized":"2024-03-28T13:09:42.691415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:21:24.753764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.941951+0000","last_change":"2024-03-28T13:09:41.694982+0000","last_active":"2024-03-28T13:09:42.941951+0000","last_peered":"2024-03-28T13:09:42.941951+0000","last_clean":"2024-03-28T13:09:42.941951+0000","last_became_active":"2024-03-28T13:09:41.694746+0000","last_became_peered":"2024-03-28T13:09:41.694746+0000","last_unstale":"2024-03-28T13:09:42.941951+0000","last_undegraded":"2024-03-28T13:09:42.941951+0000","last_fullsized":"2024-03-28T13:09:42.941951+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:29.551203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686937+0000","last_change":"2024-03-28T13:09:41.685949+0000","last_active":"2024-03-28T13:09:42.686937+0000","last_peered":"2024-03-28T13:09:42.686937+0000","last_clean":"2024-03-28T13:09:42.686937+0000","last_became_active":"2024-03-28T13:09:41.683610+0000","last_became_peered":"2024-03-28T13:09:41.683610+0000","last_unstale":"2024-03-28T13:09:42.686937+0000","last_undegraded":"2024-03-28T13:09:42.686937+0000","last_fullsized":"2024-03-28T13:09:42.686937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:21:24.753764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684741+0000","last_change":"2024-03-28T13:09:41.672017+0000","last_active":"2024-03-28T13:09:42.684741+0000","last_peered":"2024-03-28T13:09:42.684741+0000","last_clean":"2024-03-28T13:09:42.684741+0000","last_became_active":"2024-03-28T13:09:41.671594+0000","last_became_peered":"2024-03-28T13:09:41.671594+0000","last_unstale":"2024-03-28T13:09:42.684741+0000","last_undegraded":"2024-03-28T13:09:42.684741+0000","last_fullsized":"2024-03-28T13:09:42.684741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:05:06.674720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687844+0000","last_change":"2024-03-28T13:09:41.689019+0000","last_active":"2024-03-28T13:09:42.687844+0000","last_peered":"2024-03-28T13:09:42.687844+0000","last_clean":"2024-03-28T13:09:42.687844+0000","last_became_active":"2024-03-28T13:09:41.688663+0000","last_became_peered":"2024-03-28T13:09:41.688663+0000","last_unstale":"2024-03-28T13:09:42.687844+0000","last_undegraded":"2024-03-28T13:09:42.687844+0000","last_fullsized":"2024-03-28T13:09:42.687844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.673442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680521+0000","last_change":"2024-03-28T13:09:41.698336+0000","last_active":"2024-03-28T13:09:42.680521+0000","last_peered":"2024-03-28T13:09:42.680521+0000","last_clean":"2024-03-28T13:09:42.680521+0000","last_became_active":"2024-03-28T13:09:41.696491+0000","last_became_peered":"2024-03-28T13:09:41.696491+0000","last_unstale":"2024-03-28T13:09:42.680521+0000","last_undegraded":"2024-03-28T13:09:42.680521+0000","last_fullsized":"2024-03-28T13:09:42.680521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:37.029859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945985+0000","last_change":"2024-03-28T13:09:41.676686+0000","last_active":"2024-03-28T13:09:42.945985+0000","last_peered":"2024-03-28T13:09:42.945985+0000","last_clean":"2024-03-28T13:09:42.945985+0000","last_became_active":"2024-03-28T13:09:41.676403+0000","last_became_peered":"2024-03-28T13:09:41.676403+0000","last_unstale":"2024-03-28T13:09:42.945985+0000","last_undegraded":"2024-03-28T13:09:42.945985+0000","last_fullsized":"2024-03-28T13:09:42.945985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:38:55.330149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945843+0000","last_change":"2024-03-28T13:09:41.688698+0000","last_active":"2024-03-28T13:09:42.945843+0000","last_peered":"2024-03-28T13:09:42.945843+0000","last_clean":"2024-03-28T13:09:42.945843+0000","last_became_active":"2024-03-28T13:09:41.688352+0000","last_became_peered":"2024-03-28T13:09:41.688352+0000","last_unstale":"2024-03-28T13:09:42.945843+0000","last_undegraded":"2024-03-28T13:09:42.945843+0000","last_fullsized":"2024-03-28T13:09:42.945843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:26.076499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686871+0000","last_change":"2024-03-28T13:09:41.675052+0000","last_active":"2024-03-28T13:09:42.686871+0000","last_peered":"2024-03-28T13:09:42.686871+0000","last_clean":"2024-03-28T13:09:42.686871+0000","last_became_active":"2024-03-28T13:09:41.674420+0000","last_became_peered":"2024-03-28T13:09:41.674420+0000","last_unstale":"2024-03-28T13:09:42.686871+0000","last_undegraded":"2024-03-28T13:09:42.686871+0000","last_fullsized":"2024-03-28T13:09:42.686871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:48.181534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684993+0000","last_change":"2024-03-28T13:09:41.675625+0000","last_active":"2024-03-28T13:09:42.684993+0000","last_peered":"2024-03-28T13:09:42.684993+0000","last_clean":"2024-03-28T13:09:42.684993+0000","last_became_active":"2024-03-28T13:09:41.674475+0000","last_became_peered":"2024-03-28T13:09:41.674475+0000","last_unstale":"2024-03-28T13:09:42.684993+0000","last_undegraded":"2024-03-28T13:09:42.684993+0000","last_fullsized":"2024-03-28T13:09:42.684993+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:51:59.688473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944380+0000","last_change":"2024-03-28T13:09:41.689798+0000","last_active":"2024-03-28T13:09:42.944380+0000","last_peered":"2024-03-28T13:09:42.944380+0000","last_clean":"2024-03-28T13:09:42.944380+0000","last_became_active":"2024-03-28T13:09:41.689522+0000","last_became_peered":"2024-03-28T13:09:41.689522+0000","last_unstale":"2024-03-28T13:09:42.944380+0000","last_undegraded":"2024-03-28T13:09:42.944380+0000","last_fullsized":"2024-03-28T13:09:42.944380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:02:28.201366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.679983+0000","last_change":"2024-03-28T13:09:41.693508+0000","last_active":"2024-03-28T13:09:42.679983+0000","last_peered":"2024-03-28T13:09:42.679983+0000","last_clean":"2024-03-28T13:09:42.679983+0000","last_became_active":"2024-03-28T13:09:41.692947+0000","last_became_peered":"2024-03-28T13:09:41.692947+0000","last_unstale":"2024-03-28T13:09:42.679983+0000","last_undegraded":"2024-03-28T13:09:42.679983+0000","last_fullsized":"2024-03-28T13:09:42.679983+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:54.213310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942108+0000","last_change":"2024-03-28T13:09:41.681713+0000","last_active":"2024-03-28T13:09:42.942108+0000","last_peered":"2024-03-28T13:09:42.942108+0000","last_clean":"2024-03-28T13:09:42.942108+0000","last_became_active":"2024-03-28T13:09:41.681410+0000","last_became_peered":"2024-03-28T13:09:41.681410+0000","last_unstale":"2024-03-28T13:09:42.942108+0000","last_undegraded":"2024-03-28T13:09:42.942108+0000","last_fullsized":"2024-03-28T13:09:42.942108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:54.213310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686916+0000","last_change":"2024-03-28T13:09:40.648316+0000","last_active":"2024-03-28T13:09:42.686916+0000","last_peered":"2024-03-28T13:09:42.686916+0000","last_clean":"2024-03-28T13:09:42.686916+0000","last_became_active":"2024-03-28T13:09:40.647964+0000","last_became_peered":"2024-03-28T13:09:40.647964+0000","last_unstale":"2024-03-28T13:09:42.686916+0000","last_undegraded":"2024-03-28T13:09:42.686916+0000","last_fullsized":"2024-03-28T13:09:42.686916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691106+0000","last_change":"2024-03-28T13:09:41.692864+0000","last_active":"2024-03-28T13:09:42.691106+0000","last_peered":"2024-03-28T13:09:42.691106+0000","last_clean":"2024-03-28T13:09:42.691106+0000","last_became_active":"2024-03-28T13:09:41.692536+0000","last_became_peered":"2024-03-28T13:09:41.692536+0000","last_unstale":"2024-03-28T13:09:42.691106+0000","last_undegraded":"2024-03-28T13:09:42.691106+0000","last_fullsized":"2024-03-28T13:09:42.691106+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:21:02.278070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685041+0000","last_change":"2024-03-28T13:09:40.655262+0000","last_active":"2024-03-28T13:09:42.685041+0000","last_peered":"2024-03-28T13:09:42.685041+0000","last_clean":"2024-03-28T13:09:42.685041+0000","last_became_active":"2024-03-28T13:09:40.654238+0000","last_became_peered":"2024-03-28T13:09:40.654238+0000","last_unstale":"2024-03-28T13:09:42.685041+0000","last_undegraded":"2024-03-28T13:09:42.685041+0000","last_fullsized":"2024-03-28T13:09:42.685041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:53.208363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945101+0000","last_change":"2024-03-28T13:09:40.651989+0000","last_active":"2024-03-28T13:09:42.945101+0000","last_peered":"2024-03-28T13:09:42.945101+0000","last_clean":"2024-03-28T13:09:42.945101+0000","last_became_active":"2024-03-28T13:09:40.651663+0000","last_became_peered":"2024-03-28T13:09:40.651663+0000","last_unstale":"2024-03-28T13:09:42.945101+0000","last_undegraded":"2024-03-28T13:09:42.945101+0000","last_fullsized":"2024-03-28T13:09:42.945101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:04.285637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687309+0000","last_change":"2024-03-28T13:09:41.667699+0000","last_active":"2024-03-28T13:09:42.687309+0000","last_peered":"2024-03-28T13:09:42.687309+0000","last_clean":"2024-03-28T13:09:42.687309+0000","last_became_active":"2024-03-28T13:09:41.667238+0000","last_became_peered":"2024-03-28T13:09:41.667238+0000","last_unstale":"2024-03-28T13:09:42.687309+0000","last_undegraded":"2024-03-28T13:09:42.687309+0000","last_fullsized":"2024-03-28T13:09:42.687309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:03:17.731528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684468+0000","last_change":"2024-03-28T13:09:40.644508+0000","last_active":"2024-03-28T13:09:42.684468+0000","last_peered":"2024-03-28T13:09:42.684468+0000","last_clean":"2024-03-28T13:09:42.684468+0000","last_became_active":"2024-03-28T13:09:40.644194+0000","last_became_peered":"2024-03-28T13:09:40.644194+0000","last_unstale":"2024-03-28T13:09:42.684468+0000","last_undegraded":"2024-03-28T13:09:42.684468+0000","last_fullsized":"2024-03-28T13:09:42.684468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:36.024913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691016+0000","last_change":"2024-03-28T13:09:41.689932+0000","last_active":"2024-03-28T13:09:42.691016+0000","last_peered":"2024-03-28T13:09:42.691016+0000","last_clean":"2024-03-28T13:09:42.691016+0000","last_became_active":"2024-03-28T13:09:41.689527+0000","last_became_peered":"2024-03-28T13:09:41.689527+0000","last_unstale":"2024-03-28T13:09:42.691016+0000","last_undegraded":"2024-03-28T13:09:42.691016+0000","last_fullsized":"2024-03-28T13:09:42.691016+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:16.926876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942367+0000","last_change":"2024-03-28T13:09:40.648693+0000","last_active":"2024-03-28T13:09:42.942367+0000","last_peered":"2024-03-28T13:09:42.942367+0000","last_clean":"2024-03-28T13:09:42.942367+0000","last_became_active":"2024-03-28T13:09:40.648368+0000","last_became_peered":"2024-03-28T13:09:40.648368+0000","last_unstale":"2024-03-28T13:09:42.942367+0000","last_undegraded":"2024-03-28T13:09:42.942367+0000","last_fullsized":"2024-03-28T13:09:42.942367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:04.285637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680453+0000","last_change":"2024-03-28T13:09:41.696265+0000","last_active":"2024-03-28T13:09:42.680453+0000","last_peered":"2024-03-28T13:09:42.680453+0000","last_clean":"2024-03-28T13:09:42.680453+0000","last_became_active":"2024-03-28T13:09:41.695813+0000","last_became_peered":"2024-03-28T13:09:41.695813+0000","last_unstale":"2024-03-28T13:09:42.680453+0000","last_undegraded":"2024-03-28T13:09:42.680453+0000","last_fullsized":"2024-03-28T13:09:42.680453+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:55:08.842535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696109+0000","last_change":"2024-03-28T13:09:40.645526+0000","last_active":"2024-03-28T13:09:42.696109+0000","last_peered":"2024-03-28T13:09:42.696109+0000","last_clean":"2024-03-28T13:09:42.696109+0000","last_became_active":"2024-03-28T13:09:40.644528+0000","last_became_peered":"2024-03-28T13:09:40.644528+0000","last_unstale":"2024-03-28T13:09:42.696109+0000","last_undegraded":"2024-03-28T13:09:42.696109+0000","last_fullsized":"2024-03-28T13:09:42.696109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:29:29.680319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688789+0000","last_change":"2024-03-28T13:09:41.681452+0000","last_active":"2024-03-28T13:09:42.688789+0000","last_peered":"2024-03-28T13:09:42.688789+0000","last_clean":"2024-03-28T13:09:42.688789+0000","last_became_active":"2024-03-28T13:09:41.681153+0000","last_became_peered":"2024-03-28T13:09:41.681153+0000","last_unstale":"2024-03-28T13:09:42.688789+0000","last_undegraded":"2024-03-28T13:09:42.688789+0000","last_fullsized":"2024-03-28T13:09:42.688789+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:38:26.092780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944138+0000","last_change":"2024-03-28T13:09:40.655107+0000","last_active":"2024-03-28T13:09:42.944138+0000","last_peered":"2024-03-28T13:09:42.944138+0000","last_clean":"2024-03-28T13:09:42.944138+0000","last_became_active":"2024-03-28T13:09:40.653157+0000","last_became_peered":"2024-03-28T13:09:40.653157+0000","last_unstale":"2024-03-28T13:09:42.944138+0000","last_undegraded":"2024-03-28T13:09:42.944138+0000","last_fullsized":"2024-03-28T13:09:42.944138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:38:32.114605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684836+0000","last_change":"2024-03-28T13:09:41.688505+0000","last_active":"2024-03-28T13:09:42.684836+0000","last_peered":"2024-03-28T13:09:42.684836+0000","last_clean":"2024-03-28T13:09:42.684836+0000","last_became_active":"2024-03-28T13:09:41.683889+0000","last_became_peered":"2024-03-28T13:09:41.683889+0000","last_unstale":"2024-03-28T13:09:42.684836+0000","last_undegraded":"2024-03-28T13:09:42.684836+0000","last_fullsized":"2024-03-28T13:09:42.684836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:58:50.776271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680192+0000","last_change":"2024-03-28T13:09:40.652146+0000","last_active":"2024-03-28T13:09:42.680192+0000","last_peered":"2024-03-28T13:09:42.680192+0000","last_clean":"2024-03-28T13:09:42.680192+0000","last_became_active":"2024-03-28T13:09:40.648949+0000","last_became_peered":"2024-03-28T13:09:40.648949+0000","last_unstale":"2024-03-28T13:09:42.680192+0000","last_undegraded":"2024-03-28T13:09:42.680192+0000","last_fullsized":"2024-03-28T13:09:42.680192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:02:19.389612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695962+0000","last_change":"2024-03-28T13:09:41.690754+0000","last_active":"2024-03-28T13:09:42.695962+0000","last_peered":"2024-03-28T13:09:42.695962+0000","last_clean":"2024-03-28T13:09:42.695962+0000","last_became_active":"2024-03-28T13:09:41.689905+0000","last_became_peered":"2024-03-28T13:09:41.689905+0000","last_unstale":"2024-03-28T13:09:42.695962+0000","last_undegraded":"2024-03-28T13:09:42.695962+0000","last_fullsized":"2024-03-28T13:09:42.695962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:28:22.739597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944021+0000","last_change":"2024-03-28T13:09:40.654520+0000","last_active":"2024-03-28T13:09:42.944021+0000","last_peered":"2024-03-28T13:09:42.944021+0000","last_clean":"2024-03-28T13:09:42.944021+0000","last_became_active":"2024-03-28T13:09:40.654016+0000","last_became_peered":"2024-03-28T13:09:40.654016+0000","last_unstale":"2024-03-28T13:09:42.944021+0000","last_undegraded":"2024-03-28T13:09:42.944021+0000","last_fullsized":"2024-03-28T13:09:42.944021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:41:42.775642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945445+0000","last_change":"2024-03-28T13:09:41.678649+0000","last_active":"2024-03-28T13:09:42.945445+0000","last_peered":"2024-03-28T13:09:42.945445+0000","last_clean":"2024-03-28T13:09:42.945445+0000","last_became_active":"2024-03-28T13:09:41.678312+0000","last_became_peered":"2024-03-28T13:09:41.678312+0000","last_unstale":"2024-03-28T13:09:42.945445+0000","last_undegraded":"2024-03-28T13:09:42.945445+0000","last_fullsized":"2024-03-28T13:09:42.945445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:44:34.513311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685122+0000","last_change":"2024-03-28T13:09:40.658418+0000","last_active":"2024-03-28T13:09:42.685122+0000","last_peered":"2024-03-28T13:09:42.685122+0000","last_clean":"2024-03-28T13:09:42.685122+0000","last_became_active":"2024-03-28T13:09:40.658089+0000","last_became_peered":"2024-03-28T13:09:40.658089+0000","last_unstale":"2024-03-28T13:09:42.685122+0000","last_undegraded":"2024-03-28T13:09:42.685122+0000","last_fullsized":"2024-03-28T13:09:42.685122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:21:16.839181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685214+0000","last_change":"2024-03-28T13:09:41.703207+0000","last_active":"2024-03-28T13:09:42.685214+0000","last_peered":"2024-03-28T13:09:42.685214+0000","last_clean":"2024-03-28T13:09:42.685214+0000","last_became_active":"2024-03-28T13:09:41.702892+0000","last_became_peered":"2024-03-28T13:09:41.702892+0000","last_unstale":"2024-03-28T13:09:42.685214+0000","last_undegraded":"2024-03-28T13:09:42.685214+0000","last_fullsized":"2024-03-28T13:09:42.685214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:55:53.479586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695697+0000","last_change":"2024-03-28T13:09:40.657534+0000","last_active":"2024-03-28T13:09:42.695697+0000","last_peered":"2024-03-28T13:09:42.695697+0000","last_clean":"2024-03-28T13:09:42.695697+0000","last_became_active":"2024-03-28T13:09:40.657186+0000","last_became_peered":"2024-03-28T13:09:40.657186+0000","last_unstale":"2024-03-28T13:09:42.695697+0000","last_undegraded":"2024-03-28T13:09:42.695697+0000","last_fullsized":"2024-03-28T13:09:42.695697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:39.886681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.941943+0000","last_change":"2024-03-28T13:09:41.687172+0000","last_active":"2024-03-28T13:09:42.941943+0000","last_peered":"2024-03-28T13:09:42.941943+0000","last_clean":"2024-03-28T13:09:42.941943+0000","last_became_active":"2024-03-28T13:09:41.686911+0000","last_became_peered":"2024-03-28T13:09:41.686911+0000","last_unstale":"2024-03-28T13:09:42.941943+0000","last_undegraded":"2024-03-28T13:09:42.941943+0000","last_fullsized":"2024-03-28T13:09:42.941943+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:40.891627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.690586+0000","last_change":"2024-03-28T13:09:40.646010+0000","last_active":"2024-03-28T13:09:42.690586+0000","last_peered":"2024-03-28T13:09:42.690586+0000","last_clean":"2024-03-28T13:09:42.690586+0000","last_became_active":"2024-03-28T13:09:40.645705+0000","last_became_peered":"2024-03-28T13:09:40.645705+0000","last_unstale":"2024-03-28T13:09:42.690586+0000","last_undegraded":"2024-03-28T13:09:42.690586+0000","last_fullsized":"2024-03-28T13:09:42.690586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684654+0000","last_change":"2024-03-28T13:09:41.685321+0000","last_active":"2024-03-28T13:09:42.684654+0000","last_peered":"2024-03-28T13:09:42.684654+0000","last_clean":"2024-03-28T13:09:42.684654+0000","last_became_active":"2024-03-28T13:09:41.684633+0000","last_became_peered":"2024-03-28T13:09:41.684633+0000","last_unstale":"2024-03-28T13:09:42.684654+0000","last_undegraded":"2024-03-28T13:09:42.684654+0000","last_fullsized":"2024-03-28T13:09:42.684654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:38:26.092780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696357+0000","last_change":"2024-03-28T13:09:40.649526+0000","last_active":"2024-03-28T13:09:42.696357+0000","last_peered":"2024-03-28T13:09:42.696357+0000","last_clean":"2024-03-28T13:09:42.696357+0000","last_became_active":"2024-03-28T13:09:40.649087+0000","last_became_peered":"2024-03-28T13:09:40.649087+0000","last_unstale":"2024-03-28T13:09:42.696357+0000","last_undegraded":"2024-03-28T13:09:42.696357+0000","last_fullsized":"2024-03-28T13:09:42.696357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:28.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680538+0000","last_change":"2024-03-28T13:09:41.704978+0000","last_active":"2024-03-28T13:09:42.680538+0000","last_peered":"2024-03-28T13:09:42.680538+0000","last_clean":"2024-03-28T13:09:42.680538+0000","last_became_active":"2024-03-28T13:09:41.703242+0000","last_became_peered":"2024-03-28T13:09:41.703242+0000","last_unstale":"2024-03-28T13:09:42.680538+0000","last_undegraded":"2024-03-28T13:09:42.680538+0000","last_fullsized":"2024-03-28T13:09:42.680538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:44:34.513311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691459+0000","last_change":"2024-03-28T13:09:40.655868+0000","last_active":"2024-03-28T13:09:42.691459+0000","last_peered":"2024-03-28T13:09:42.691459+0000","last_clean":"2024-03-28T13:09:42.691459+0000","last_became_active":"2024-03-28T13:09:40.654787+0000","last_became_peered":"2024-03-28T13:09:40.654787+0000","last_unstale":"2024-03-28T13:09:42.691459+0000","last_undegraded":"2024-03-28T13:09:42.691459+0000","last_fullsized":"2024-03-28T13:09:42.691459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:19:50.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696355+0000","last_change":"2024-03-28T13:09:41.694953+0000","last_active":"2024-03-28T13:09:42.696355+0000","last_peered":"2024-03-28T13:09:42.696355+0000","last_clean":"2024-03-28T13:09:42.696355+0000","last_became_active":"2024-03-28T13:09:41.694665+0000","last_became_peered":"2024-03-28T13:09:41.694665+0000","last_unstale":"2024-03-28T13:09:42.696355+0000","last_undegraded":"2024-03-28T13:09:42.696355+0000","last_fullsized":"2024-03-28T13:09:42.696355+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:43:20.588688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687780+0000","last_change":"2024-03-28T13:09:40.647574+0000","last_active":"2024-03-28T13:09:42.687780+0000","last_peered":"2024-03-28T13:09:42.687780+0000","last_clean":"2024-03-28T13:09:42.687780+0000","last_became_active":"2024-03-28T13:09:40.647272+0000","last_became_peered":"2024-03-28T13:09:40.647272+0000","last_unstale":"2024-03-28T13:09:42.687780+0000","last_undegraded":"2024-03-28T13:09:42.687780+0000","last_fullsized":"2024-03-28T13:09:42.687780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:42:11.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696806+0000","last_change":"2024-03-28T13:09:41.688065+0000","last_active":"2024-03-28T13:09:42.696806+0000","last_peered":"2024-03-28T13:09:42.696806+0000","last_clean":"2024-03-28T13:09:42.696806+0000","last_became_active":"2024-03-28T13:09:41.687636+0000","last_became_peered":"2024-03-28T13:09:41.687636+0000","last_unstale":"2024-03-28T13:09:42.696806+0000","last_undegraded":"2024-03-28T13:09:42.696806+0000","last_fullsized":"2024-03-28T13:09:42.696806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:00.376154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.946040+0000","last_change":"2024-03-28T13:09:40.655402+0000","last_active":"2024-03-28T13:09:42.946040+0000","last_peered":"2024-03-28T13:09:42.946040+0000","last_clean":"2024-03-28T13:09:42.946040+0000","last_became_active":"2024-03-28T13:09:40.655074+0000","last_became_peered":"2024-03-28T13:09:40.655074+0000","last_unstale":"2024-03-28T13:09:42.946040+0000","last_undegraded":"2024-03-28T13:09:42.946040+0000","last_fullsized":"2024-03-28T13:09:42.946040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685478+0000","last_change":"2024-03-28T13:09:41.691186+0000","last_active":"2024-03-28T13:09:42.685478+0000","last_peered":"2024-03-28T13:09:42.685478+0000","last_clean":"2024-03-28T13:09:42.685478+0000","last_became_active":"2024-03-28T13:09:41.690223+0000","last_became_peered":"2024-03-28T13:09:41.690223+0000","last_unstale":"2024-03-28T13:09:42.685478+0000","last_undegraded":"2024-03-28T13:09:42.685478+0000","last_fullsized":"2024-03-28T13:09:42.685478+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:15:43.557779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.941839+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:09:42.941839+0000","last_peered":"2024-03-28T13:09:42.941839+0000","last_clean":"2024-03-28T13:09:42.941839+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:09:42.941839+0000","last_undegraded":"2024-03-28T13:09:42.941839+0000","last_fullsized":"2024-03-28T13:09:42.941839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685703+0000","last_change":"2024-03-28T13:09:40.651783+0000","last_active":"2024-03-28T13:09:42.685703+0000","last_peered":"2024-03-28T13:09:42.685703+0000","last_clean":"2024-03-28T13:09:42.685703+0000","last_became_active":"2024-03-28T13:09:40.645785+0000","last_became_peered":"2024-03-28T13:09:40.645785+0000","last_unstale":"2024-03-28T13:09:42.685703+0000","last_undegraded":"2024-03-28T13:09:42.685703+0000","last_fullsized":"2024-03-28T13:09:42.685703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:55:07.837589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691146+0000","last_change":"2024-03-28T13:09:41.691719+0000","last_active":"2024-03-28T13:09:42.691146+0000","last_peered":"2024-03-28T13:09:42.691146+0000","last_clean":"2024-03-28T13:09:42.691146+0000","last_became_active":"2024-03-28T13:09:41.691250+0000","last_became_peered":"2024-03-28T13:09:41.691250+0000","last_unstale":"2024-03-28T13:09:42.691146+0000","last_undegraded":"2024-03-28T13:09:42.691146+0000","last_fullsized":"2024-03-28T13:09:42.691146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:50:46.504501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684592+0000","last_change":"2024-03-28T13:09:40.646854+0000","last_active":"2024-03-28T13:09:42.684592+0000","last_peered":"2024-03-28T13:09:42.684592+0000","last_clean":"2024-03-28T13:09:42.684592+0000","last_became_active":"2024-03-28T13:09:40.644192+0000","last_became_peered":"2024-03-28T13:09:40.644192+0000","last_unstale":"2024-03-28T13:09:42.684592+0000","last_undegraded":"2024-03-28T13:09:42.684592+0000","last_fullsized":"2024-03-28T13:09:42.684592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:25.071553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945249+0000","last_change":"2024-03-28T13:09:41.683008+0000","last_active":"2024-03-28T13:09:42.945249+0000","last_peered":"2024-03-28T13:09:42.945249+0000","last_clean":"2024-03-28T13:09:42.945249+0000","last_became_active":"2024-03-28T13:09:41.682664+0000","last_became_peered":"2024-03-28T13:09:41.682664+0000","last_unstale":"2024-03-28T13:09:42.945249+0000","last_undegraded":"2024-03-28T13:09:42.945249+0000","last_fullsized":"2024-03-28T13:09:42.945249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:37.029859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.943669+0000","last_change":"2024-03-28T13:09:40.654742+0000","last_active":"2024-03-28T13:09:42.943669+0000","last_peered":"2024-03-28T13:09:42.943669+0000","last_clean":"2024-03-28T13:09:42.943669+0000","last_became_active":"2024-03-28T13:09:40.654447+0000","last_became_peered":"2024-03-28T13:09:40.654447+0000","last_unstale":"2024-03-28T13:09:42.943669+0000","last_undegraded":"2024-03-28T13:09:42.943669+0000","last_fullsized":"2024-03-28T13:09:42.943669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:42:12.104485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687304+0000","last_change":"2024-03-28T13:09:41.677617+0000","last_active":"2024-03-28T13:09:42.687304+0000","last_peered":"2024-03-28T13:09:42.687304+0000","last_clean":"2024-03-28T13:09:42.687304+0000","last_became_active":"2024-03-28T13:09:41.677322+0000","last_became_peered":"2024-03-28T13:09:41.677322+0000","last_unstale":"2024-03-28T13:09:42.687304+0000","last_undegraded":"2024-03-28T13:09:42.687304+0000","last_fullsized":"2024-03-28T13:09:42.687304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:28:22.739597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685198+0000","last_change":"2024-03-28T13:09:40.650985+0000","last_active":"2024-03-28T13:09:42.685198+0000","last_peered":"2024-03-28T13:09:42.685198+0000","last_clean":"2024-03-28T13:09:42.685198+0000","last_became_active":"2024-03-28T13:09:40.648577+0000","last_became_peered":"2024-03-28T13:09:40.648577+0000","last_unstale":"2024-03-28T13:09:42.685198+0000","last_undegraded":"2024-03-28T13:09:42.685198+0000","last_fullsized":"2024-03-28T13:09:42.685198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:38:54.325203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695776+0000","last_change":"2024-03-28T13:09:41.691409+0000","last_active":"2024-03-28T13:09:42.695776+0000","last_peered":"2024-03-28T13:09:42.695776+0000","last_clean":"2024-03-28T13:09:42.695776+0000","last_became_active":"2024-03-28T13:09:41.690587+0000","last_became_peered":"2024-03-28T13:09:41.690587+0000","last_unstale":"2024-03-28T13:09:42.695776+0000","last_undegraded":"2024-03-28T13:09:42.695776+0000","last_fullsized":"2024-03-28T13:09:42.695776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:21:24.753764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687316+0000","last_change":"2024-03-28T13:09:40.648955+0000","last_active":"2024-03-28T13:09:42.687316+0000","last_peered":"2024-03-28T13:09:42.687316+0000","last_clean":"2024-03-28T13:09:42.687316+0000","last_became_active":"2024-03-28T13:09:40.648645+0000","last_became_peered":"2024-03-28T13:09:40.648645+0000","last_unstale":"2024-03-28T13:09:42.687316+0000","last_undegraded":"2024-03-28T13:09:42.687316+0000","last_fullsized":"2024-03-28T13:09:42.687316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:25:38.914424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.679924+0000","last_change":"2024-03-28T13:09:41.692962+0000","last_active":"2024-03-28T13:09:42.679924+0000","last_peered":"2024-03-28T13:09:42.679924+0000","last_clean":"2024-03-28T13:09:42.679924+0000","last_became_active":"2024-03-28T13:09:41.692660+0000","last_became_peered":"2024-03-28T13:09:41.692660+0000","last_unstale":"2024-03-28T13:09:42.679924+0000","last_undegraded":"2024-03-28T13:09:42.679924+0000","last_fullsized":"2024-03-28T13:09:42.679924+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:40.891627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685533+0000","last_change":"2024-03-28T13:09:40.654663+0000","last_active":"2024-03-28T13:09:42.685533+0000","last_peered":"2024-03-28T13:09:42.685533+0000","last_clean":"2024-03-28T13:09:42.685533+0000","last_became_active":"2024-03-28T13:09:40.654236+0000","last_became_peered":"2024-03-28T13:09:40.654236+0000","last_unstale":"2024-03-28T13:09:42.685533+0000","last_undegraded":"2024-03-28T13:09:42.685533+0000","last_fullsized":"2024-03-28T13:09:42.685533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:28.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.946112+0000","last_change":"2024-03-28T13:09:41.671654+0000","last_active":"2024-03-28T13:09:42.946112+0000","last_peered":"2024-03-28T13:09:42.946112+0000","last_clean":"2024-03-28T13:09:42.946112+0000","last_became_active":"2024-03-28T13:09:41.671232+0000","last_became_peered":"2024-03-28T13:09:41.671232+0000","last_unstale":"2024-03-28T13:09:42.946112+0000","last_undegraded":"2024-03-28T13:09:42.946112+0000","last_fullsized":"2024-03-28T13:09:42.946112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:21:02.278070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.694649+0000","last_change":"2024-03-28T13:09:40.651132+0000","last_active":"2024-03-28T13:09:42.694649+0000","last_peered":"2024-03-28T13:09:42.694649+0000","last_clean":"2024-03-28T13:09:42.694649+0000","last_became_active":"2024-03-28T13:09:40.650806+0000","last_became_peered":"2024-03-28T13:09:40.650806+0000","last_unstale":"2024-03-28T13:09:42.694649+0000","last_undegraded":"2024-03-28T13:09:42.694649+0000","last_fullsized":"2024-03-28T13:09:42.694649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:42:11.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691153+0000","last_change":"2024-03-28T13:09:41.686114+0000","last_active":"2024-03-28T13:09:42.691153+0000","last_peered":"2024-03-28T13:09:42.691153+0000","last_clean":"2024-03-28T13:09:42.691153+0000","last_became_active":"2024-03-28T13:09:41.685846+0000","last_became_peered":"2024-03-28T13:09:41.685846+0000","last_unstale":"2024-03-28T13:09:42.691153+0000","last_undegraded":"2024-03-28T13:09:42.691153+0000","last_fullsized":"2024-03-28T13:09:42.691153+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:28:13.777057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685901+0000","last_change":"2024-03-28T13:09:40.657051+0000","last_active":"2024-03-28T13:09:42.685901+0000","last_peered":"2024-03-28T13:09:42.685901+0000","last_clean":"2024-03-28T13:09:42.685901+0000","last_became_active":"2024-03-28T13:09:40.656680+0000","last_became_peered":"2024-03-28T13:09:40.656680+0000","last_unstale":"2024-03-28T13:09:42.685901+0000","last_undegraded":"2024-03-28T13:09:42.685901+0000","last_fullsized":"2024-03-28T13:09:42.685901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:44:33.508365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691166+0000","last_change":"2024-03-28T13:09:41.681460+0000","last_active":"2024-03-28T13:09:42.691166+0000","last_peered":"2024-03-28T13:09:42.691166+0000","last_clean":"2024-03-28T13:09:42.691166+0000","last_became_active":"2024-03-28T13:09:41.681136+0000","last_became_peered":"2024-03-28T13:09:41.681136+0000","last_unstale":"2024-03-28T13:09:42.691166+0000","last_undegraded":"2024-03-28T13:09:42.691166+0000","last_fullsized":"2024-03-28T13:09:42.691166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:48.181534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695372+0000","last_change":"2024-03-28T13:09:40.649525+0000","last_active":"2024-03-28T13:09:42.695372+0000","last_peered":"2024-03-28T13:09:42.695372+0000","last_clean":"2024-03-28T13:09:42.695372+0000","last_became_active":"2024-03-28T13:09:40.649204+0000","last_became_peered":"2024-03-28T13:09:40.649204+0000","last_unstale":"2024-03-28T13:09:42.695372+0000","last_undegraded":"2024-03-28T13:09:42.695372+0000","last_fullsized":"2024-03-28T13:09:42.695372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684678+0000","last_change":"2024-03-28T13:09:41.685557+0000","last_active":"2024-03-28T13:09:42.684678+0000","last_peered":"2024-03-28T13:09:42.684678+0000","last_clean":"2024-03-28T13:09:42.684678+0000","last_became_active":"2024-03-28T13:09:41.685000+0000","last_became_peered":"2024-03-28T13:09:41.685000+0000","last_unstale":"2024-03-28T13:09:42.684678+0000","last_undegraded":"2024-03-28T13:09:42.684678+0000","last_fullsized":"2024-03-28T13:09:42.684678+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:00.376154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.690868+0000","last_change":"2024-03-28T13:09:40.653861+0000","last_active":"2024-03-28T13:09:42.690868+0000","last_peered":"2024-03-28T13:09:42.690868+0000","last_clean":"2024-03-28T13:09:42.690868+0000","last_became_active":"2024-03-28T13:09:40.653467+0000","last_became_peered":"2024-03-28T13:09:40.653467+0000","last_unstale":"2024-03-28T13:09:42.690868+0000","last_undegraded":"2024-03-28T13:09:42.690868+0000","last_fullsized":"2024-03-28T13:09:42.690868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:28.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.690684+0000","last_change":"2024-03-28T13:09:41.677815+0000","last_active":"2024-03-28T13:09:42.690684+0000","last_peered":"2024-03-28T13:09:42.690684+0000","last_clean":"2024-03-28T13:09:42.690684+0000","last_became_active":"2024-03-28T13:09:41.677544+0000","last_became_peered":"2024-03-28T13:09:41.677544+0000","last_unstale":"2024-03-28T13:09:42.690684+0000","last_undegraded":"2024-03-28T13:09:42.690684+0000","last_fullsized":"2024-03-28T13:09:42.690684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:38:55.330149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945733+0000","last_change":"2024-03-28T13:09:40.649046+0000","last_active":"2024-03-28T13:09:42.945733+0000","last_peered":"2024-03-28T13:09:42.945733+0000","last_clean":"2024-03-28T13:09:42.945733+0000","last_became_active":"2024-03-28T13:09:40.648758+0000","last_became_peered":"2024-03-28T13:09:40.648758+0000","last_unstale":"2024-03-28T13:09:42.945733+0000","last_undegraded":"2024-03-28T13:09:42.945733+0000","last_fullsized":"2024-03-28T13:09:42.945733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:25:38.914424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944019+0000","last_change":"2024-03-28T13:09:41.685698+0000","last_active":"2024-03-28T13:09:42.944019+0000","last_peered":"2024-03-28T13:09:42.944019+0000","last_clean":"2024-03-28T13:09:42.944019+0000","last_became_active":"2024-03-28T13:09:41.685418+0000","last_became_peered":"2024-03-28T13:09:41.685418+0000","last_unstale":"2024-03-28T13:09:42.944019+0000","last_undegraded":"2024-03-28T13:09:42.944019+0000","last_fullsized":"2024-03-28T13:09:42.944019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:57:43.818964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685389+0000","last_change":"2024-03-28T13:09:40.653049+0000","last_active":"2024-03-28T13:09:42.685389+0000","last_peered":"2024-03-28T13:09:42.685389+0000","last_clean":"2024-03-28T13:09:42.685389+0000","last_became_active":"2024-03-28T13:09:40.652732+0000","last_became_peered":"2024-03-28T13:09:40.652732+0000","last_unstale":"2024-03-28T13:09:42.685389+0000","last_undegraded":"2024-03-28T13:09:42.685389+0000","last_fullsized":"2024-03-28T13:09:42.685389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:39.886681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685373+0000","last_change":"2024-03-28T13:09:41.690392+0000","last_active":"2024-03-28T13:09:42.685373+0000","last_peered":"2024-03-28T13:09:42.685373+0000","last_clean":"2024-03-28T13:09:42.685373+0000","last_became_active":"2024-03-28T13:09:41.689363+0000","last_became_peered":"2024-03-28T13:09:41.689363+0000","last_unstale":"2024-03-28T13:09:42.685373+0000","last_undegraded":"2024-03-28T13:09:42.685373+0000","last_fullsized":"2024-03-28T13:09:42.685373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:09:02.095518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685390+0000","last_change":"2024-03-28T13:09:41.700393+0000","last_active":"2024-03-28T13:09:42.685390+0000","last_peered":"2024-03-28T13:09:42.685390+0000","last_clean":"2024-03-28T13:09:42.685390+0000","last_became_active":"2024-03-28T13:09:41.700088+0000","last_became_peered":"2024-03-28T13:09:41.700088+0000","last_unstale":"2024-03-28T13:09:42.685390+0000","last_undegraded":"2024-03-28T13:09:42.685390+0000","last_fullsized":"2024-03-28T13:09:42.685390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:31.835464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696866+0000","last_change":"2024-03-28T13:09:40.648841+0000","last_active":"2024-03-28T13:09:42.696866+0000","last_peered":"2024-03-28T13:09:42.696866+0000","last_clean":"2024-03-28T13:09:42.696866+0000","last_became_active":"2024-03-28T13:09:40.646454+0000","last_became_peered":"2024-03-28T13:09:40.646454+0000","last_unstale":"2024-03-28T13:09:42.696866+0000","last_undegraded":"2024-03-28T13:09:42.696866+0000","last_fullsized":"2024-03-28T13:09:42.696866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:42:11.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942397+0000","last_change":"2024-03-28T13:09:41.705263+0000","last_active":"2024-03-28T13:09:42.942397+0000","last_peered":"2024-03-28T13:09:42.942397+0000","last_clean":"2024-03-28T13:09:42.942397+0000","last_became_active":"2024-03-28T13:09:41.704074+0000","last_became_peered":"2024-03-28T13:09:41.704074+0000","last_unstale":"2024-03-28T13:09:42.942397+0000","last_undegraded":"2024-03-28T13:09:42.942397+0000","last_fullsized":"2024-03-28T13:09:42.942397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:55:08.842535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695978+0000","last_change":"2024-03-28T13:09:40.649403+0000","last_active":"2024-03-28T13:09:42.695978+0000","last_peered":"2024-03-28T13:09:42.695978+0000","last_clean":"2024-03-28T13:09:42.695978+0000","last_became_active":"2024-03-28T13:09:40.648964+0000","last_became_peered":"2024-03-28T13:09:40.648964+0000","last_unstale":"2024-03-28T13:09:42.695978+0000","last_undegraded":"2024-03-28T13:09:42.695978+0000","last_fullsized":"2024-03-28T13:09:42.695978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:19:50.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688252+0000","last_change":"2024-03-28T13:09:41.688111+0000","last_active":"2024-03-28T13:09:42.688252+0000","last_peered":"2024-03-28T13:09:42.688252+0000","last_clean":"2024-03-28T13:09:42.688252+0000","last_became_active":"2024-03-28T13:09:41.687818+0000","last_became_peered":"2024-03-28T13:09:41.687818+0000","last_unstale":"2024-03-28T13:09:42.688252+0000","last_undegraded":"2024-03-28T13:09:42.688252+0000","last_fullsized":"2024-03-28T13:09:42.688252+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:38.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688212+0000","last_change":"2024-03-28T13:09:40.646771+0000","last_active":"2024-03-28T13:09:42.688212+0000","last_peered":"2024-03-28T13:09:42.688212+0000","last_clean":"2024-03-28T13:09:42.688212+0000","last_became_active":"2024-03-28T13:09:40.646496+0000","last_became_peered":"2024-03-28T13:09:40.646496+0000","last_unstale":"2024-03-28T13:09:42.688212+0000","last_undegraded":"2024-03-28T13:09:42.688212+0000","last_fullsized":"2024-03-28T13:09:42.688212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:19:50.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688194+0000","last_change":"2024-03-28T13:09:41.684932+0000","last_active":"2024-03-28T13:09:42.688194+0000","last_peered":"2024-03-28T13:09:42.688194+0000","last_clean":"2024-03-28T13:09:42.688194+0000","last_became_active":"2024-03-28T13:09:41.684314+0000","last_became_peered":"2024-03-28T13:09:41.684314+0000","last_unstale":"2024-03-28T13:09:42.688194+0000","last_undegraded":"2024-03-28T13:09:42.688194+0000","last_fullsized":"2024-03-28T13:09:42.688194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:43:20.588688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691848+0000","last_change":"2024-03-28T13:09:40.656875+0000","last_active":"2024-03-28T13:09:42.691848+0000","last_peered":"2024-03-28T13:09:42.691848+0000","last_clean":"2024-03-28T13:09:42.691848+0000","last_became_active":"2024-03-28T13:09:40.656569+0000","last_became_peered":"2024-03-28T13:09:40.656569+0000","last_unstale":"2024-03-28T13:09:42.691848+0000","last_undegraded":"2024-03-28T13:09:42.691848+0000","last_fullsized":"2024-03-28T13:09:42.691848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:29:29.680319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476744,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":80417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476742,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476742,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":670273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476742,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":79891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":79835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":831488,"data_stored":670295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":82119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":73659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:09:46.165 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:09:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.426+0000 7f159f600640 1 Processor -- start 2024-03-28T13:09:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.426+0000 7f159f600640 1 -- start start 2024-03-28T13:09:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.428+0000 7f159f600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f159815e7e0 0x7f15980d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.428+0000 7f159f600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15980cfb20 0x7f15980cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.428+0000 7f159f600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f159815ebc0 con 0x7f159806ca00 2024-03-28T13:09:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.428+0000 7f159f600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f159806d1c0 con 0x7f159815e7e0 2024-03-28T13:09:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.428+0000 7f159f600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f159806d4a0 con 0x7f15980cfb20 2024-03-28T13:09:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.428+0000 7f1597e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f159806ca00 0x7f15980d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42886/0 (socket says 172.21.15.19:42886) 2024-03-28T13:09:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.428+0000 7f159cc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f159815e7e0 0x7f15980d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.428+0000 7f1597e00640 1 -- 172.21.15.19:0/3738056674 learned_addr learned my addr 172.21.15.19:0/3738056674 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.428+0000 7f159d600640 1 --2- 172.21.15.19:0/3738056674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15980cfb20 0x7f15980cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.429+0000 7f1597400640 1 -- 172.21.15.19:0/3738056674 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2464329000 0 0) 0x7f159815ebc0 con 0x7f159806ca00 2024-03-28T13:09:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.429+0000 7f1597400640 1 -- 172.21.15.19:0/3738056674 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1580003520 con 0x7f159806ca00 2024-03-28T13:09:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.429+0000 7f159cc00640 1 -- 172.21.15.19:0/3738056674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15980cfb20 msgr2=0x7f15980cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.429+0000 7f159cc00640 1 --2- 172.21.15.19:0/3738056674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15980cfb20 0x7f15980cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.429+0000 7f159cc00640 1 -- 172.21.15.19:0/3738056674 >> v1:172.21.15.19:6789/0 conn(0x7f159806ca00 legacy=0x7f15980d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.429+0000 7f159cc00640 1 -- 172.21.15.19:0/3738056674 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15980cf320 con 0x7f159815e7e0 2024-03-28T13:09:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.430+0000 7f159cc00640 1 --2- 172.21.15.19:0/3738056674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f159815e7e0 0x7f15980d1950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f158c0097a0 tx=0x7f158c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484acc372c207da6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.430+0000 7f1597400640 1 -- 172.21.15.19:0/3738056674 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c031780 con 0x7f159815e7e0 2024-03-28T13:09:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.430+0000 7f1597400640 1 -- 172.21.15.19:0/3738056674 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f158c037070 con 0x7f159815e7e0 2024-03-28T13:09:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.430+0000 7f1597400640 1 -- 172.21.15.19:0/3738056674 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c031b40 con 0x7f159815e7e0 2024-03-28T13:09:46.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.431+0000 7f159f600640 1 -- 172.21.15.19:0/3738056674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f159815e7e0 msgr2=0x7f15980d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.431+0000 7f159f600640 1 --2- 172.21.15.19:0/3738056674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f159815e7e0 0x7f15980d1950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f158c0097a0 tx=0x7f158c02b820 comp rx=0 tx=0).stop 2024-03-28T13:09:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.431+0000 7f159f600640 1 -- 172.21.15.19:0/3738056674 shutdown_connections 2024-03-28T13:09:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.431+0000 7f159f600640 1 --2- 172.21.15.19:0/3738056674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15980cfb20 0x7f15980cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.431+0000 7f159f600640 1 --2- 172.21.15.19:0/3738056674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f159815e7e0 0x7f15980d1950 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.431+0000 7f159f600640 1 -- 172.21.15.19:0/3738056674 >> 172.21.15.19:0/3738056674 conn(0x7f159809d7c0 msgr2=0x7f159806bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.431+0000 7f159f600640 1 -- 172.21.15.19:0/3738056674 shutdown_connections 2024-03-28T13:09:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.432+0000 7f159f600640 1 -- 172.21.15.19:0/3738056674 wait complete. 2024-03-28T13:09:46.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.433+0000 7f159f600640 1 Processor -- start 2024-03-28T13:09:46.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.433+0000 7f159f600640 1 -- start start 2024-03-28T13:09:46.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.434+0000 7f159f600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f159806ca00 0x7f1598152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:46.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.434+0000 7f159f600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15980cfb20 0x7f15980c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:46.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.434+0000 7f159f600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f159815e7e0 0x7f15980c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:46.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.434+0000 7f159f600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f159806ce60 con 0x7f15980cfb20 2024-03-28T13:09:46.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.434+0000 7f159f600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f15980cefd0 con 0x7f159806ca00 2024-03-28T13:09:46.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.434+0000 7f159f600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f15980d2f30 con 0x7f159815e7e0 2024-03-28T13:09:46.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.434+0000 7f1597e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15980cfb20 0x7f15980c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:46.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.435+0000 7f1597e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15980cfb20 0x7f15980c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50578/0 (socket says 172.21.15.19:50578) 2024-03-28T13:09:46.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.435+0000 7f1597e00640 1 -- 172.21.15.19:0/2640128738 learned_addr learned my addr 172.21.15.19:0/2640128738 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.435+0000 7f159cc00640 1 --2- 172.21.15.19:0/2640128738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f159806ca00 0x7f1598152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.435+0000 7f159d600640 1 --2- 172.21.15.19:0/2640128738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f159815e7e0 0x7f15980c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.435+0000 7f1597e00640 1 -- 172.21.15.19:0/2640128738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f159815e7e0 msgr2=0x7f15980c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.435+0000 7f1597e00640 1 --2- 172.21.15.19:0/2640128738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f159815e7e0 0x7f15980c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.435+0000 7f1597e00640 1 -- 172.21.15.19:0/2640128738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f159806ca00 msgr2=0x7f1598152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.435+0000 7f1597e00640 1 --2- 172.21.15.19:0/2640128738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f159806ca00 0x7f1598152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.435+0000 7f1597e00640 1 -- 172.21.15.19:0/2640128738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1598152f90 con 0x7f15980cfb20 2024-03-28T13:09:46.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.435+0000 7f1597e00640 1 --2- 172.21.15.19:0/2640128738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15980cfb20 0x7f15980c9150 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f158400c440 tx=0x7f158400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:46.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.436+0000 7f1594c00640 1 -- 172.21.15.19:0/2640128738 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584017070 con 0x7f15980cfb20 2024-03-28T13:09:46.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.436+0000 7f1594c00640 1 -- 172.21.15.19:0/2640128738 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1584005ce0 con 0x7f15980cfb20 2024-03-28T13:09:46.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.436+0000 7f159f600640 1 -- 172.21.15.19:0/2640128738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15980c80b0 con 0x7f15980cfb20 2024-03-28T13:09:46.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.436+0000 7f1594c00640 1 -- 172.21.15.19:0/2640128738 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15840072b0 con 0x7f15980cfb20 2024-03-28T13:09:46.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.436+0000 7f159f600640 1 -- 172.21.15.19:0/2640128738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15980c8670 con 0x7f15980cfb20 2024-03-28T13:09:46.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.437+0000 7f159f600640 1 -- 172.21.15.19:0/2640128738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15980685c0 con 0x7f15980cfb20 2024-03-28T13:09:46.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.438+0000 7f1594c00640 1 -- 172.21.15.19:0/2640128738 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f158401a070 con 0x7f15980cfb20 2024-03-28T13:09:46.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.439+0000 7f1594c00640 1 --2- 172.21.15.19:0/2640128738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f157806c530 0x7f157806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:46.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.439+0000 7f1594c00640 1 -- 172.21.15.19:0/2640128738 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f158408aa30 con 0x7f15980cfb20 2024-03-28T13:09:46.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.440+0000 7f159cc00640 1 --2- 172.21.15.19:0/2640128738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f157806c530 0x7f157806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:46.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.441+0000 7f159cc00640 1 --2- 172.21.15.19:0/2640128738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f157806c530 0x7f157806e9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f158c036000 tx=0x7f158c03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:46.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.449+0000 7f1594c00640 1 -- 172.21.15.19:0/2640128738 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1584059100 con 0x7f15980cfb20 2024-03-28T13:09:46.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.652+0000 7f159f600640 1 -- 172.21.15.19:0/2640128738 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15980d5300 con 0x7f157806c530 2024-03-28T13:09:46.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.672+0000 7f1594c00640 1 -- 172.21.15.19:0/2640128738 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251941 (secure 0 0 0) 0x7f15980d5300 con 0x7f157806c530 2024-03-28T13:09:46.674 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:46.678 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:09:46.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.679+0000 7f159f600640 1 -- 172.21.15.19:0/2640128738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f157806c530 msgr2=0x7f157806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:46.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.679+0000 7f159f600640 1 --2- 172.21.15.19:0/2640128738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f157806c530 0x7f157806e9f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f158c036000 tx=0x7f158c03d040 comp rx=0 tx=0).stop 2024-03-28T13:09:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.679+0000 7f159f600640 1 -- 172.21.15.19:0/2640128738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15980cfb20 msgr2=0x7f15980c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.679+0000 7f159f600640 1 --2- 172.21.15.19:0/2640128738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15980cfb20 0x7f15980c9150 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f158400c440 tx=0x7f158400c900 comp rx=0 tx=0).stop 2024-03-28T13:09:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.679+0000 7f159f600640 1 -- 172.21.15.19:0/2640128738 shutdown_connections 2024-03-28T13:09:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.679+0000 7f159f600640 1 --2- 172.21.15.19:0/2640128738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f157806c530 0x7f157806e9f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.679+0000 7f159f600640 1 --2- 172.21.15.19:0/2640128738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f159815e7e0 0x7f15980c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.679+0000 7f159f600640 1 --2- 172.21.15.19:0/2640128738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15980cfb20 0x7f15980c9150 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.679+0000 7f159f600640 1 --2- 172.21.15.19:0/2640128738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f159806ca00 0x7f1598152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.679+0000 7f159f600640 1 -- 172.21.15.19:0/2640128738 >> 172.21.15.19:0/2640128738 conn(0x7f159809d7c0 msgr2=0x7f15980b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.679+0000 7f159f600640 1 -- 172.21.15.19:0/2640128738 shutdown_connections 2024-03-28T13:09:46.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.679+0000 7f159f600640 1 -- 172.21.15.19:0/2640128738 wait complete. 2024-03-28T13:09:46.702 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-03-28T13:09:45.982324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219172,"kb_used_data":3004,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755548,"statfs":{"total":773094113280,"available":772869681152,"internally_reserved":0,"allocated":3076096,"data_stored":1817936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001886"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696286+0000","last_change":"2024-03-28T13:09:41.690249+0000","last_active":"2024-03-28T13:09:42.696286+0000","last_peered":"2024-03-28T13:09:42.696286+0000","last_clean":"2024-03-28T13:09:42.696286+0000","last_became_active":"2024-03-28T13:09:41.689984+0000","last_became_peered":"2024-03-28T13:09:41.689984+0000","last_unstale":"2024-03-28T13:09:42.696286+0000","last_undegraded":"2024-03-28T13:09:42.696286+0000","last_fullsized":"2024-03-28T13:09:42.696286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:53:08.763850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687950+0000","last_change":"2024-03-28T13:09:41.669236+0000","last_active":"2024-03-28T13:09:42.687950+0000","last_peered":"2024-03-28T13:09:42.687950+0000","last_clean":"2024-03-28T13:09:42.687950+0000","last_became_active":"2024-03-28T13:09:41.668886+0000","last_became_peered":"2024-03-28T13:09:41.668886+0000","last_unstale":"2024-03-28T13:09:42.687950+0000","last_undegraded":"2024-03-28T13:09:42.687950+0000","last_fullsized":"2024-03-28T13:09:42.687950+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:53:08.763850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687498+0000","last_change":"2024-03-28T13:09:41.683573+0000","last_active":"2024-03-28T13:09:42.687498+0000","last_peered":"2024-03-28T13:09:42.687498+0000","last_clean":"2024-03-28T13:09:42.687498+0000","last_became_active":"2024-03-28T13:09:41.682846+0000","last_became_peered":"2024-03-28T13:09:41.682846+0000","last_unstale":"2024-03-28T13:09:42.687498+0000","last_undegraded":"2024-03-28T13:09:42.687498+0000","last_fullsized":"2024-03-28T13:09:42.687498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:04.694363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680562+0000","last_change":"2024-03-28T13:09:41.684558+0000","last_active":"2024-03-28T13:09:42.680562+0000","last_peered":"2024-03-28T13:09:42.680562+0000","last_clean":"2024-03-28T13:09:42.680562+0000","last_became_active":"2024-03-28T13:09:41.682564+0000","last_became_peered":"2024-03-28T13:09:41.682564+0000","last_unstale":"2024-03-28T13:09:42.680562+0000","last_undegraded":"2024-03-28T13:09:42.680562+0000","last_fullsized":"2024-03-28T13:09:42.680562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:21:17.844127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695969+0000","last_change":"2024-03-28T13:09:41.687175+0000","last_active":"2024-03-28T13:09:42.695969+0000","last_peered":"2024-03-28T13:09:42.695969+0000","last_clean":"2024-03-28T13:09:42.695969+0000","last_became_active":"2024-03-28T13:09:41.686835+0000","last_became_peered":"2024-03-28T13:09:41.686835+0000","last_unstale":"2024-03-28T13:09:42.695969+0000","last_undegraded":"2024-03-28T13:09:42.695969+0000","last_fullsized":"2024-03-28T13:09:42.695969+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:04.694363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.679923+0000","last_change":"2024-03-28T13:09:41.683515+0000","last_active":"2024-03-28T13:09:42.679923+0000","last_peered":"2024-03-28T13:09:42.679923+0000","last_clean":"2024-03-28T13:09:42.679923+0000","last_became_active":"2024-03-28T13:09:41.682795+0000","last_became_peered":"2024-03-28T13:09:41.682795+0000","last_unstale":"2024-03-28T13:09:42.679923+0000","last_undegraded":"2024-03-28T13:09:42.679923+0000","last_fullsized":"2024-03-28T13:09:42.679923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:29.551203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695673+0000","last_change":"2024-03-28T13:09:41.690393+0000","last_active":"2024-03-28T13:09:42.695673+0000","last_peered":"2024-03-28T13:09:42.695673+0000","last_clean":"2024-03-28T13:09:42.695673+0000","last_became_active":"2024-03-28T13:09:41.688973+0000","last_became_peered":"2024-03-28T13:09:41.688973+0000","last_unstale":"2024-03-28T13:09:42.695673+0000","last_undegraded":"2024-03-28T13:09:42.695673+0000","last_fullsized":"2024-03-28T13:09:42.695673+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:48.181534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686877+0000","last_change":"2024-03-28T13:09:41.691319+0000","last_active":"2024-03-28T13:09:42.686877+0000","last_peered":"2024-03-28T13:09:42.686877+0000","last_clean":"2024-03-28T13:09:42.686877+0000","last_became_active":"2024-03-28T13:09:41.690506+0000","last_became_peered":"2024-03-28T13:09:41.690506+0000","last_unstale":"2024-03-28T13:09:42.686877+0000","last_undegraded":"2024-03-28T13:09:42.686877+0000","last_fullsized":"2024-03-28T13:09:42.686877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:28:13.777057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942467+0000","last_change":"2024-03-28T13:09:41.697845+0000","last_active":"2024-03-28T13:09:42.942467+0000","last_peered":"2024-03-28T13:09:42.942467+0000","last_clean":"2024-03-28T13:09:42.942467+0000","last_became_active":"2024-03-28T13:09:41.696890+0000","last_became_peered":"2024-03-28T13:09:41.696890+0000","last_unstale":"2024-03-28T13:09:42.942467+0000","last_undegraded":"2024-03-28T13:09:42.942467+0000","last_fullsized":"2024-03-28T13:09:42.942467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:37.029859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696276+0000","last_change":"2024-03-28T13:09:41.688074+0000","last_active":"2024-03-28T13:09:42.696276+0000","last_peered":"2024-03-28T13:09:42.696276+0000","last_clean":"2024-03-28T13:09:42.696276+0000","last_became_active":"2024-03-28T13:09:41.687755+0000","last_became_peered":"2024-03-28T13:09:41.687755+0000","last_unstale":"2024-03-28T13:09:42.696276+0000","last_undegraded":"2024-03-28T13:09:42.696276+0000","last_fullsized":"2024-03-28T13:09:42.696276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.673442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695935+0000","last_change":"2024-03-28T13:09:41.693370+0000","last_active":"2024-03-28T13:09:42.695935+0000","last_peered":"2024-03-28T13:09:42.695935+0000","last_clean":"2024-03-28T13:09:42.695935+0000","last_became_active":"2024-03-28T13:09:41.690289+0000","last_became_peered":"2024-03-28T13:09:41.690289+0000","last_unstale":"2024-03-28T13:09:42.695935+0000","last_undegraded":"2024-03-28T13:09:42.695935+0000","last_fullsized":"2024-03-28T13:09:42.695935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:03:17.731528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696467+0000","last_change":"2024-03-28T13:09:41.692663+0000","last_active":"2024-03-28T13:09:42.696467+0000","last_peered":"2024-03-28T13:09:42.696467+0000","last_clean":"2024-03-28T13:09:42.696467+0000","last_became_active":"2024-03-28T13:09:41.692130+0000","last_became_peered":"2024-03-28T13:09:41.692130+0000","last_unstale":"2024-03-28T13:09:42.696467+0000","last_undegraded":"2024-03-28T13:09:42.696467+0000","last_fullsized":"2024-03-28T13:09:42.696467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:38.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945637+0000","last_change":"2024-03-28T13:09:41.692414+0000","last_active":"2024-03-28T13:09:42.945637+0000","last_peered":"2024-03-28T13:09:42.945637+0000","last_clean":"2024-03-28T13:09:42.945637+0000","last_became_active":"2024-03-28T13:09:41.692105+0000","last_became_peered":"2024-03-28T13:09:41.692105+0000","last_unstale":"2024-03-28T13:09:42.945637+0000","last_undegraded":"2024-03-28T13:09:42.945637+0000","last_fullsized":"2024-03-28T13:09:42.945637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:21:17.844127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.946036+0000","last_change":"2024-03-28T13:09:41.675736+0000","last_active":"2024-03-28T13:09:42.946036+0000","last_peered":"2024-03-28T13:09:42.946036+0000","last_clean":"2024-03-28T13:09:42.946036+0000","last_became_active":"2024-03-28T13:09:41.675120+0000","last_became_peered":"2024-03-28T13:09:41.675120+0000","last_unstale":"2024-03-28T13:09:42.946036+0000","last_undegraded":"2024-03-28T13:09:42.946036+0000","last_fullsized":"2024-03-28T13:09:42.946036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:16.926876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686621+0000","last_change":"2024-03-28T13:09:41.690417+0000","last_active":"2024-03-28T13:09:42.686621+0000","last_peered":"2024-03-28T13:09:42.686621+0000","last_clean":"2024-03-28T13:09:42.686621+0000","last_became_active":"2024-03-28T13:09:41.689467+0000","last_became_peered":"2024-03-28T13:09:41.689467+0000","last_unstale":"2024-03-28T13:09:42.686621+0000","last_undegraded":"2024-03-28T13:09:42.686621+0000","last_fullsized":"2024-03-28T13:09:42.686621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:50:46.504501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684397+0000","last_change":"2024-03-28T13:09:41.684876+0000","last_active":"2024-03-28T13:09:42.684397+0000","last_peered":"2024-03-28T13:09:42.684397+0000","last_clean":"2024-03-28T13:09:42.684397+0000","last_became_active":"2024-03-28T13:09:41.683988+0000","last_became_peered":"2024-03-28T13:09:41.683988+0000","last_unstale":"2024-03-28T13:09:42.684397+0000","last_undegraded":"2024-03-28T13:09:42.684397+0000","last_fullsized":"2024-03-28T13:09:42.684397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:38.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685867+0000","last_change":"2024-03-28T13:09:41.702219+0000","last_active":"2024-03-28T13:09:42.685867+0000","last_peered":"2024-03-28T13:09:42.685867+0000","last_clean":"2024-03-28T13:09:42.685867+0000","last_became_active":"2024-03-28T13:09:41.701907+0000","last_became_peered":"2024-03-28T13:09:41.701907+0000","last_unstale":"2024-03-28T13:09:42.685867+0000","last_undegraded":"2024-03-28T13:09:42.685867+0000","last_fullsized":"2024-03-28T13:09:42.685867+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:49:20.779575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944358+0000","last_change":"2024-03-28T13:09:41.683832+0000","last_active":"2024-03-28T13:09:42.944358+0000","last_peered":"2024-03-28T13:09:42.944358+0000","last_clean":"2024-03-28T13:09:42.944358+0000","last_became_active":"2024-03-28T13:09:41.683495+0000","last_became_peered":"2024-03-28T13:09:41.683495+0000","last_unstale":"2024-03-28T13:09:42.944358+0000","last_undegraded":"2024-03-28T13:09:42.944358+0000","last_fullsized":"2024-03-28T13:09:42.944358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:44:52.163706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685573+0000","last_change":"2024-03-28T13:09:41.688514+0000","last_active":"2024-03-28T13:09:42.685573+0000","last_peered":"2024-03-28T13:09:42.685573+0000","last_clean":"2024-03-28T13:09:42.685573+0000","last_became_active":"2024-03-28T13:09:41.688158+0000","last_became_peered":"2024-03-28T13:09:41.688158+0000","last_unstale":"2024-03-28T13:09:42.685573+0000","last_undegraded":"2024-03-28T13:09:42.685573+0000","last_fullsized":"2024-03-28T13:09:42.685573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:39:02.380052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696639+0000","last_change":"2024-03-28T13:09:41.695024+0000","last_active":"2024-03-28T13:09:42.696639+0000","last_peered":"2024-03-28T13:09:42.696639+0000","last_clean":"2024-03-28T13:09:42.696639+0000","last_became_active":"2024-03-28T13:09:41.694698+0000","last_became_peered":"2024-03-28T13:09:41.694698+0000","last_unstale":"2024-03-28T13:09:42.696639+0000","last_undegraded":"2024-03-28T13:09:42.696639+0000","last_fullsized":"2024-03-28T13:09:42.696639+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:38:26.092780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691415+0000","last_change":"2024-03-28T13:09:41.686281+0000","last_active":"2024-03-28T13:09:42.691415+0000","last_peered":"2024-03-28T13:09:42.691415+0000","last_clean":"2024-03-28T13:09:42.691415+0000","last_became_active":"2024-03-28T13:09:41.685221+0000","last_became_peered":"2024-03-28T13:09:41.685221+0000","last_unstale":"2024-03-28T13:09:42.691415+0000","last_undegraded":"2024-03-28T13:09:42.691415+0000","last_fullsized":"2024-03-28T13:09:42.691415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:21:24.753764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.941951+0000","last_change":"2024-03-28T13:09:41.694982+0000","last_active":"2024-03-28T13:09:42.941951+0000","last_peered":"2024-03-28T13:09:42.941951+0000","last_clean":"2024-03-28T13:09:42.941951+0000","last_became_active":"2024-03-28T13:09:41.694746+0000","last_became_peered":"2024-03-28T13:09:41.694746+0000","last_unstale":"2024-03-28T13:09:42.941951+0000","last_undegraded":"2024-03-28T13:09:42.941951+0000","last_fullsized":"2024-03-28T13:09:42.941951+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:29.551203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686937+0000","last_change":"2024-03-28T13:09:41.685949+0000","last_active":"2024-03-28T13:09:42.686937+0000","last_peered":"2024-03-28T13:09:42.686937+0000","last_clean":"2024-03-28T13:09:42.686937+0000","last_became_active":"2024-03-28T13:09:41.683610+0000","last_became_peered":"2024-03-28T13:09:41.683610+0000","last_unstale":"2024-03-28T13:09:42.686937+0000","last_undegraded":"2024-03-28T13:09:42.686937+0000","last_fullsized":"2024-03-28T13:09:42.686937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:21:24.753764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684741+0000","last_change":"2024-03-28T13:09:41.672017+0000","last_active":"2024-03-28T13:09:42.684741+0000","last_peered":"2024-03-28T13:09:42.684741+0000","last_clean":"2024-03-28T13:09:42.684741+0000","last_became_active":"2024-03-28T13:09:41.671594+0000","last_became_peered":"2024-03-28T13:09:41.671594+0000","last_unstale":"2024-03-28T13:09:42.684741+0000","last_undegraded":"2024-03-28T13:09:42.684741+0000","last_fullsized":"2024-03-28T13:09:42.684741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:05:06.674720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687844+0000","last_change":"2024-03-28T13:09:41.689019+0000","last_active":"2024-03-28T13:09:42.687844+0000","last_peered":"2024-03-28T13:09:42.687844+0000","last_clean":"2024-03-28T13:09:42.687844+0000","last_became_active":"2024-03-28T13:09:41.688663+0000","last_became_peered":"2024-03-28T13:09:41.688663+0000","last_unstale":"2024-03-28T13:09:42.687844+0000","last_undegraded":"2024-03-28T13:09:42.687844+0000","last_fullsized":"2024-03-28T13:09:42.687844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.673442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680521+0000","last_change":"2024-03-28T13:09:41.698336+0000","last_active":"2024-03-28T13:09:42.680521+0000","last_peered":"2024-03-28T13:09:42.680521+0000","last_clean":"2024-03-28T13:09:42.680521+0000","last_became_active":"2024-03-28T13:09:41.696491+0000","last_became_peered":"2024-03-28T13:09:41.696491+0000","last_unstale":"2024-03-28T13:09:42.680521+0000","last_undegraded":"2024-03-28T13:09:42.680521+0000","last_fullsized":"2024-03-28T13:09:42.680521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:37.029859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945985+0000","last_change":"2024-03-28T13:09:41.676686+0000","last_active":"2024-03-28T13:09:42.945985+0000","last_peered":"2024-03-28T13:09:42.945985+0000","last_clean":"2024-03-28T13:09:42.945985+0000","last_became_active":"2024-03-28T13:09:41.676403+0000","last_became_peered":"2024-03-28T13:09:41.676403+0000","last_unstale":"2024-03-28T13:09:42.945985+0000","last_undegraded":"2024-03-28T13:09:42.945985+0000","last_fullsized":"2024-03-28T13:09:42.945985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:38:55.330149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945843+0000","last_change":"2024-03-28T13:09:41.688698+0000","last_active":"2024-03-28T13:09:42.945843+0000","last_peered":"2024-03-28T13:09:42.945843+0000","last_clean":"2024-03-28T13:09:42.945843+0000","last_became_active":"2024-03-28T13:09:41.688352+0000","last_became_peered":"2024-03-28T13:09:41.688352+0000","last_unstale":"2024-03-28T13:09:42.945843+0000","last_undegraded":"2024-03-28T13:09:42.945843+0000","last_fullsized":"2024-03-28T13:09:42.945843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:26.076499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686871+0000","last_change":"2024-03-28T13:09:41.675052+0000","last_active":"2024-03-28T13:09:42.686871+0000","last_peered":"2024-03-28T13:09:42.686871+0000","last_clean":"2024-03-28T13:09:42.686871+0000","last_became_active":"2024-03-28T13:09:41.674420+0000","last_became_peered":"2024-03-28T13:09:41.674420+0000","last_unstale":"2024-03-28T13:09:42.686871+0000","last_undegraded":"2024-03-28T13:09:42.686871+0000","last_fullsized":"2024-03-28T13:09:42.686871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:48.181534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684993+0000","last_change":"2024-03-28T13:09:41.675625+0000","last_active":"2024-03-28T13:09:42.684993+0000","last_peered":"2024-03-28T13:09:42.684993+0000","last_clean":"2024-03-28T13:09:42.684993+0000","last_became_active":"2024-03-28T13:09:41.674475+0000","last_became_peered":"2024-03-28T13:09:41.674475+0000","last_unstale":"2024-03-28T13:09:42.684993+0000","last_undegraded":"2024-03-28T13:09:42.684993+0000","last_fullsized":"2024-03-28T13:09:42.684993+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:51:59.688473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944380+0000","last_change":"2024-03-28T13:09:41.689798+0000","last_active":"2024-03-28T13:09:42.944380+0000","last_peered":"2024-03-28T13:09:42.944380+0000","last_clean":"2024-03-28T13:09:42.944380+0000","last_became_active":"2024-03-28T13:09:41.689522+0000","last_became_peered":"2024-03-28T13:09:41.689522+0000","last_unstale":"2024-03-28T13:09:42.944380+0000","last_undegraded":"2024-03-28T13:09:42.944380+0000","last_fullsized":"2024-03-28T13:09:42.944380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:02:28.201366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.679983+0000","last_change":"2024-03-28T13:09:41.693508+0000","last_active":"2024-03-28T13:09:42.679983+0000","last_peered":"2024-03-28T13:09:42.679983+0000","last_clean":"2024-03-28T13:09:42.679983+0000","last_became_active":"2024-03-28T13:09:41.692947+0000","last_became_peered":"2024-03-28T13:09:41.692947+0000","last_unstale":"2024-03-28T13:09:42.679983+0000","last_undegraded":"2024-03-28T13:09:42.679983+0000","last_fullsized":"2024-03-28T13:09:42.679983+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:54.213310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942108+0000","last_change":"2024-03-28T13:09:41.681713+0000","last_active":"2024-03-28T13:09:42.942108+0000","last_peered":"2024-03-28T13:09:42.942108+0000","last_clean":"2024-03-28T13:09:42.942108+0000","last_became_active":"2024-03-28T13:09:41.681410+0000","last_became_peered":"2024-03-28T13:09:41.681410+0000","last_unstale":"2024-03-28T13:09:42.942108+0000","last_undegraded":"2024-03-28T13:09:42.942108+0000","last_fullsized":"2024-03-28T13:09:42.942108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:54.213310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686916+0000","last_change":"2024-03-28T13:09:40.648316+0000","last_active":"2024-03-28T13:09:42.686916+0000","last_peered":"2024-03-28T13:09:42.686916+0000","last_clean":"2024-03-28T13:09:42.686916+0000","last_became_active":"2024-03-28T13:09:40.647964+0000","last_became_peered":"2024-03-28T13:09:40.647964+0000","last_unstale":"2024-03-28T13:09:42.686916+0000","last_undegraded":"2024-03-28T13:09:42.686916+0000","last_fullsized":"2024-03-28T13:09:42.686916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691106+0000","last_change":"2024-03-28T13:09:41.692864+0000","last_active":"2024-03-28T13:09:42.691106+0000","last_peered":"2024-03-28T13:09:42.691106+0000","last_clean":"2024-03-28T13:09:42.691106+0000","last_became_active":"2024-03-28T13:09:41.692536+0000","last_became_peered":"2024-03-28T13:09:41.692536+0000","last_unstale":"2024-03-28T13:09:42.691106+0000","last_undegraded":"2024-03-28T13:09:42.691106+0000","last_fullsized":"2024-03-28T13:09:42.691106+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:21:02.278070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685041+0000","last_change":"2024-03-28T13:09:40.655262+0000","last_active":"2024-03-28T13:09:42.685041+0000","last_peered":"2024-03-28T13:09:42.685041+0000","last_clean":"2024-03-28T13:09:42.685041+0000","last_became_active":"2024-03-28T13:09:40.654238+0000","last_became_peered":"2024-03-28T13:09:40.654238+0000","last_unstale":"2024-03-28T13:09:42.685041+0000","last_undegraded":"2024-03-28T13:09:42.685041+0000","last_fullsized":"2024-03-28T13:09:42.685041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:53.208363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945101+0000","last_change":"2024-03-28T13:09:40.651989+0000","last_active":"2024-03-28T13:09:42.945101+0000","last_peered":"2024-03-28T13:09:42.945101+0000","last_clean":"2024-03-28T13:09:42.945101+0000","last_became_active":"2024-03-28T13:09:40.651663+0000","last_became_peered":"2024-03-28T13:09:40.651663+0000","last_unstale":"2024-03-28T13:09:42.945101+0000","last_undegraded":"2024-03-28T13:09:42.945101+0000","last_fullsized":"2024-03-28T13:09:42.945101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:04.285637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687309+0000","last_change":"2024-03-28T13:09:41.667699+0000","last_active":"2024-03-28T13:09:42.687309+0000","last_peered":"2024-03-28T13:09:42.687309+0000","last_clean":"2024-03-28T13:09:42.687309+0000","last_became_active":"2024-03-28T13:09:41.667238+0000","last_became_peered":"2024-03-28T13:09:41.667238+0000","last_unstale":"2024-03-28T13:09:42.687309+0000","last_undegraded":"2024-03-28T13:09:42.687309+0000","last_fullsized":"2024-03-28T13:09:42.687309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:03:17.731528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684468+0000","last_change":"2024-03-28T13:09:40.644508+0000","last_active":"2024-03-28T13:09:42.684468+0000","last_peered":"2024-03-28T13:09:42.684468+0000","last_clean":"2024-03-28T13:09:42.684468+0000","last_became_active":"2024-03-28T13:09:40.644194+0000","last_became_peered":"2024-03-28T13:09:40.644194+0000","last_unstale":"2024-03-28T13:09:42.684468+0000","last_undegraded":"2024-03-28T13:09:42.684468+0000","last_fullsized":"2024-03-28T13:09:42.684468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:36.024913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691016+0000","last_change":"2024-03-28T13:09:41.689932+0000","last_active":"2024-03-28T13:09:42.691016+0000","last_peered":"2024-03-28T13:09:42.691016+0000","last_clean":"2024-03-28T13:09:42.691016+0000","last_became_active":"2024-03-28T13:09:41.689527+0000","last_became_peered":"2024-03-28T13:09:41.689527+0000","last_unstale":"2024-03-28T13:09:42.691016+0000","last_undegraded":"2024-03-28T13:09:42.691016+0000","last_fullsized":"2024-03-28T13:09:42.691016+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:16.926876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942367+0000","last_change":"2024-03-28T13:09:40.648693+0000","last_active":"2024-03-28T13:09:42.942367+0000","last_peered":"2024-03-28T13:09:42.942367+0000","last_clean":"2024-03-28T13:09:42.942367+0000","last_became_active":"2024-03-28T13:09:40.648368+0000","last_became_peered":"2024-03-28T13:09:40.648368+0000","last_unstale":"2024-03-28T13:09:42.942367+0000","last_undegraded":"2024-03-28T13:09:42.942367+0000","last_fullsized":"2024-03-28T13:09:42.942367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:04.285637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680453+0000","last_change":"2024-03-28T13:09:41.696265+0000","last_active":"2024-03-28T13:09:42.680453+0000","last_peered":"2024-03-28T13:09:42.680453+0000","last_clean":"2024-03-28T13:09:42.680453+0000","last_became_active":"2024-03-28T13:09:41.695813+0000","last_became_peered":"2024-03-28T13:09:41.695813+0000","last_unstale":"2024-03-28T13:09:42.680453+0000","last_undegraded":"2024-03-28T13:09:42.680453+0000","last_fullsized":"2024-03-28T13:09:42.680453+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:55:08.842535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696109+0000","last_change":"2024-03-28T13:09:40.645526+0000","last_active":"2024-03-28T13:09:42.696109+0000","last_peered":"2024-03-28T13:09:42.696109+0000","last_clean":"2024-03-28T13:09:42.696109+0000","last_became_active":"2024-03-28T13:09:40.644528+0000","last_became_peered":"2024-03-28T13:09:40.644528+0000","last_unstale":"2024-03-28T13:09:42.696109+0000","last_undegraded":"2024-03-28T13:09:42.696109+0000","last_fullsized":"2024-03-28T13:09:42.696109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:29:29.680319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688789+0000","last_change":"2024-03-28T13:09:41.681452+0000","last_active":"2024-03-28T13:09:42.688789+0000","last_peered":"2024-03-28T13:09:42.688789+0000","last_clean":"2024-03-28T13:09:42.688789+0000","last_became_active":"2024-03-28T13:09:41.681153+0000","last_became_peered":"2024-03-28T13:09:41.681153+0000","last_unstale":"2024-03-28T13:09:42.688789+0000","last_undegraded":"2024-03-28T13:09:42.688789+0000","last_fullsized":"2024-03-28T13:09:42.688789+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:38:26.092780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944138+0000","last_change":"2024-03-28T13:09:40.655107+0000","last_active":"2024-03-28T13:09:42.944138+0000","last_peered":"2024-03-28T13:09:42.944138+0000","last_clean":"2024-03-28T13:09:42.944138+0000","last_became_active":"2024-03-28T13:09:40.653157+0000","last_became_peered":"2024-03-28T13:09:40.653157+0000","last_unstale":"2024-03-28T13:09:42.944138+0000","last_undegraded":"2024-03-28T13:09:42.944138+0000","last_fullsized":"2024-03-28T13:09:42.944138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:38:32.114605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684836+0000","last_change":"2024-03-28T13:09:41.688505+0000","last_active":"2024-03-28T13:09:42.684836+0000","last_peered":"2024-03-28T13:09:42.684836+0000","last_clean":"2024-03-28T13:09:42.684836+0000","last_became_active":"2024-03-28T13:09:41.683889+0000","last_became_peered":"2024-03-28T13:09:41.683889+0000","last_unstale":"2024-03-28T13:09:42.684836+0000","last_undegraded":"2024-03-28T13:09:42.684836+0000","last_fullsized":"2024-03-28T13:09:42.684836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:58:50.776271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680192+0000","last_change":"2024-03-28T13:09:40.652146+0000","last_active":"2024-03-28T13:09:42.680192+0000","last_peered":"2024-03-28T13:09:42.680192+0000","last_clean":"2024-03-28T13:09:42.680192+0000","last_became_active":"2024-03-28T13:09:40.648949+0000","last_became_peered":"2024-03-28T13:09:40.648949+0000","last_unstale":"2024-03-28T13:09:42.680192+0000","last_undegraded":"2024-03-28T13:09:42.680192+0000","last_fullsized":"2024-03-28T13:09:42.680192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:02:19.389612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695962+0000","last_change":"2024-03-28T13:09:41.690754+0000","last_active":"2024-03-28T13:09:42.695962+0000","last_peered":"2024-03-28T13:09:42.695962+0000","last_clean":"2024-03-28T13:09:42.695962+0000","last_became_active":"2024-03-28T13:09:41.689905+0000","last_became_peered":"2024-03-28T13:09:41.689905+0000","last_unstale":"2024-03-28T13:09:42.695962+0000","last_undegraded":"2024-03-28T13:09:42.695962+0000","last_fullsized":"2024-03-28T13:09:42.695962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:28:22.739597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944021+0000","last_change":"2024-03-28T13:09:40.654520+0000","last_active":"2024-03-28T13:09:42.944021+0000","last_peered":"2024-03-28T13:09:42.944021+0000","last_clean":"2024-03-28T13:09:42.944021+0000","last_became_active":"2024-03-28T13:09:40.654016+0000","last_became_peered":"2024-03-28T13:09:40.654016+0000","last_unstale":"2024-03-28T13:09:42.944021+0000","last_undegraded":"2024-03-28T13:09:42.944021+0000","last_fullsized":"2024-03-28T13:09:42.944021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:41:42.775642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945445+0000","last_change":"2024-03-28T13:09:41.678649+0000","last_active":"2024-03-28T13:09:42.945445+0000","last_peered":"2024-03-28T13:09:42.945445+0000","last_clean":"2024-03-28T13:09:42.945445+0000","last_became_active":"2024-03-28T13:09:41.678312+0000","last_became_peered":"2024-03-28T13:09:41.678312+0000","last_unstale":"2024-03-28T13:09:42.945445+0000","last_undegraded":"2024-03-28T13:09:42.945445+0000","last_fullsized":"2024-03-28T13:09:42.945445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:44:34.513311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685122+0000","last_change":"2024-03-28T13:09:40.658418+0000","last_active":"2024-03-28T13:09:42.685122+0000","last_peered":"2024-03-28T13:09:42.685122+0000","last_clean":"2024-03-28T13:09:42.685122+0000","last_became_active":"2024-03-28T13:09:40.658089+0000","last_became_peered":"2024-03-28T13:09:40.658089+0000","last_unstale":"2024-03-28T13:09:42.685122+0000","last_undegraded":"2024-03-28T13:09:42.685122+0000","last_fullsized":"2024-03-28T13:09:42.685122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:21:16.839181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685214+0000","last_change":"2024-03-28T13:09:41.703207+0000","last_active":"2024-03-28T13:09:42.685214+0000","last_peered":"2024-03-28T13:09:42.685214+0000","last_clean":"2024-03-28T13:09:42.685214+0000","last_became_active":"2024-03-28T13:09:41.702892+0000","last_became_peered":"2024-03-28T13:09:41.702892+0000","last_unstale":"2024-03-28T13:09:42.685214+0000","last_undegraded":"2024-03-28T13:09:42.685214+0000","last_fullsized":"2024-03-28T13:09:42.685214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:55:53.479586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695697+0000","last_change":"2024-03-28T13:09:40.657534+0000","last_active":"2024-03-28T13:09:42.695697+0000","last_peered":"2024-03-28T13:09:42.695697+0000","last_clean":"2024-03-28T13:09:42.695697+0000","last_became_active":"2024-03-28T13:09:40.657186+0000","last_became_peered":"2024-03-28T13:09:40.657186+0000","last_unstale":"2024-03-28T13:09:42.695697+0000","last_undegraded":"2024-03-28T13:09:42.695697+0000","last_fullsized":"2024-03-28T13:09:42.695697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:39.886681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.941943+0000","last_change":"2024-03-28T13:09:41.687172+0000","last_active":"2024-03-28T13:09:42.941943+0000","last_peered":"2024-03-28T13:09:42.941943+0000","last_clean":"2024-03-28T13:09:42.941943+0000","last_became_active":"2024-03-28T13:09:41.686911+0000","last_became_peered":"2024-03-28T13:09:41.686911+0000","last_unstale":"2024-03-28T13:09:42.941943+0000","last_undegraded":"2024-03-28T13:09:42.941943+0000","last_fullsized":"2024-03-28T13:09:42.941943+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:40.891627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.690586+0000","last_change":"2024-03-28T13:09:40.646010+0000","last_active":"2024-03-28T13:09:42.690586+0000","last_peered":"2024-03-28T13:09:42.690586+0000","last_clean":"2024-03-28T13:09:42.690586+0000","last_became_active":"2024-03-28T13:09:40.645705+0000","last_became_peered":"2024-03-28T13:09:40.645705+0000","last_unstale":"2024-03-28T13:09:42.690586+0000","last_undegraded":"2024-03-28T13:09:42.690586+0000","last_fullsized":"2024-03-28T13:09:42.690586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684654+0000","last_change":"2024-03-28T13:09:41.685321+0000","last_active":"2024-03-28T13:09:42.684654+0000","last_peered":"2024-03-28T13:09:42.684654+0000","last_clean":"2024-03-28T13:09:42.684654+0000","last_became_active":"2024-03-28T13:09:41.684633+0000","last_became_peered":"2024-03-28T13:09:41.684633+0000","last_unstale":"2024-03-28T13:09:42.684654+0000","last_undegraded":"2024-03-28T13:09:42.684654+0000","last_fullsized":"2024-03-28T13:09:42.684654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:38:26.092780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696357+0000","last_change":"2024-03-28T13:09:40.649526+0000","last_active":"2024-03-28T13:09:42.696357+0000","last_peered":"2024-03-28T13:09:42.696357+0000","last_clean":"2024-03-28T13:09:42.696357+0000","last_became_active":"2024-03-28T13:09:40.649087+0000","last_became_peered":"2024-03-28T13:09:40.649087+0000","last_unstale":"2024-03-28T13:09:42.696357+0000","last_undegraded":"2024-03-28T13:09:42.696357+0000","last_fullsized":"2024-03-28T13:09:42.696357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:28.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680538+0000","last_change":"2024-03-28T13:09:41.704978+0000","last_active":"2024-03-28T13:09:42.680538+0000","last_peered":"2024-03-28T13:09:42.680538+0000","last_clean":"2024-03-28T13:09:42.680538+0000","last_became_active":"2024-03-28T13:09:41.703242+0000","last_became_peered":"2024-03-28T13:09:41.703242+0000","last_unstale":"2024-03-28T13:09:42.680538+0000","last_undegraded":"2024-03-28T13:09:42.680538+0000","last_fullsized":"2024-03-28T13:09:42.680538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:44:34.513311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691459+0000","last_change":"2024-03-28T13:09:40.655868+0000","last_active":"2024-03-28T13:09:42.691459+0000","last_peered":"2024-03-28T13:09:42.691459+0000","last_clean":"2024-03-28T13:09:42.691459+0000","last_became_active":"2024-03-28T13:09:40.654787+0000","last_became_peered":"2024-03-28T13:09:40.654787+0000","last_unstale":"2024-03-28T13:09:42.691459+0000","last_undegraded":"2024-03-28T13:09:42.691459+0000","last_fullsized":"2024-03-28T13:09:42.691459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:19:50.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696355+0000","last_change":"2024-03-28T13:09:41.694953+0000","last_active":"2024-03-28T13:09:42.696355+0000","last_peered":"2024-03-28T13:09:42.696355+0000","last_clean":"2024-03-28T13:09:42.696355+0000","last_became_active":"2024-03-28T13:09:41.694665+0000","last_became_peered":"2024-03-28T13:09:41.694665+0000","last_unstale":"2024-03-28T13:09:42.696355+0000","last_undegraded":"2024-03-28T13:09:42.696355+0000","last_fullsized":"2024-03-28T13:09:42.696355+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:43:20.588688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687780+0000","last_change":"2024-03-28T13:09:40.647574+0000","last_active":"2024-03-28T13:09:42.687780+0000","last_peered":"2024-03-28T13:09:42.687780+0000","last_clean":"2024-03-28T13:09:42.687780+0000","last_became_active":"2024-03-28T13:09:40.647272+0000","last_became_peered":"2024-03-28T13:09:40.647272+0000","last_unstale":"2024-03-28T13:09:42.687780+0000","last_undegraded":"2024-03-28T13:09:42.687780+0000","last_fullsized":"2024-03-28T13:09:42.687780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:42:11.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696806+0000","last_change":"2024-03-28T13:09:41.688065+0000","last_active":"2024-03-28T13:09:42.696806+0000","last_peered":"2024-03-28T13:09:42.696806+0000","last_clean":"2024-03-28T13:09:42.696806+0000","last_became_active":"2024-03-28T13:09:41.687636+0000","last_became_peered":"2024-03-28T13:09:41.687636+0000","last_unstale":"2024-03-28T13:09:42.696806+0000","last_undegraded":"2024-03-28T13:09:42.696806+0000","last_fullsized":"2024-03-28T13:09:42.696806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:00.376154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.946040+0000","last_change":"2024-03-28T13:09:40.655402+0000","last_active":"2024-03-28T13:09:42.946040+0000","last_peered":"2024-03-28T13:09:42.946040+0000","last_clean":"2024-03-28T13:09:42.946040+0000","last_became_active":"2024-03-28T13:09:40.655074+0000","last_became_peered":"2024-03-28T13:09:40.655074+0000","last_unstale":"2024-03-28T13:09:42.946040+0000","last_undegraded":"2024-03-28T13:09:42.946040+0000","last_fullsized":"2024-03-28T13:09:42.946040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685478+0000","last_change":"2024-03-28T13:09:41.691186+0000","last_active":"2024-03-28T13:09:42.685478+0000","last_peered":"2024-03-28T13:09:42.685478+0000","last_clean":"2024-03-28T13:09:42.685478+0000","last_became_active":"2024-03-28T13:09:41.690223+0000","last_became_peered":"2024-03-28T13:09:41.690223+0000","last_unstale":"2024-03-28T13:09:42.685478+0000","last_undegraded":"2024-03-28T13:09:42.685478+0000","last_fullsized":"2024-03-28T13:09:42.685478+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:15:43.557779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.941839+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:09:42.941839+0000","last_peered":"2024-03-28T13:09:42.941839+0000","last_clean":"2024-03-28T13:09:42.941839+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:09:42.941839+0000","last_undegraded":"2024-03-28T13:09:42.941839+0000","last_fullsized":"2024-03-28T13:09:42.941839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685703+0000","last_change":"2024-03-28T13:09:40.651783+0000","last_active":"2024-03-28T13:09:42.685703+0000","last_peered":"2024-03-28T13:09:42.685703+0000","last_clean":"2024-03-28T13:09:42.685703+0000","last_became_active":"2024-03-28T13:09:40.645785+0000","last_became_peered":"2024-03-28T13:09:40.645785+0000","last_unstale":"2024-03-28T13:09:42.685703+0000","last_undegraded":"2024-03-28T13:09:42.685703+0000","last_fullsized":"2024-03-28T13:09:42.685703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:55:07.837589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691146+0000","last_change":"2024-03-28T13:09:41.691719+0000","last_active":"2024-03-28T13:09:42.691146+0000","last_peered":"2024-03-28T13:09:42.691146+0000","last_clean":"2024-03-28T13:09:42.691146+0000","last_became_active":"2024-03-28T13:09:41.691250+0000","last_became_peered":"2024-03-28T13:09:41.691250+0000","last_unstale":"2024-03-28T13:09:42.691146+0000","last_undegraded":"2024-03-28T13:09:42.691146+0000","last_fullsized":"2024-03-28T13:09:42.691146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:50:46.504501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684592+0000","last_change":"2024-03-28T13:09:40.646854+0000","last_active":"2024-03-28T13:09:42.684592+0000","last_peered":"2024-03-28T13:09:42.684592+0000","last_clean":"2024-03-28T13:09:42.684592+0000","last_became_active":"2024-03-28T13:09:40.644192+0000","last_became_peered":"2024-03-28T13:09:40.644192+0000","last_unstale":"2024-03-28T13:09:42.684592+0000","last_undegraded":"2024-03-28T13:09:42.684592+0000","last_fullsized":"2024-03-28T13:09:42.684592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:25.071553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945249+0000","last_change":"2024-03-28T13:09:41.683008+0000","last_active":"2024-03-28T13:09:42.945249+0000","last_peered":"2024-03-28T13:09:42.945249+0000","last_clean":"2024-03-28T13:09:42.945249+0000","last_became_active":"2024-03-28T13:09:41.682664+0000","last_became_peered":"2024-03-28T13:09:41.682664+0000","last_unstale":"2024-03-28T13:09:42.945249+0000","last_undegraded":"2024-03-28T13:09:42.945249+0000","last_fullsized":"2024-03-28T13:09:42.945249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:37.029859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.943669+0000","last_change":"2024-03-28T13:09:40.654742+0000","last_active":"2024-03-28T13:09:42.943669+0000","last_peered":"2024-03-28T13:09:42.943669+0000","last_clean":"2024-03-28T13:09:42.943669+0000","last_became_active":"2024-03-28T13:09:40.654447+0000","last_became_peered":"2024-03-28T13:09:40.654447+0000","last_unstale":"2024-03-28T13:09:42.943669+0000","last_undegraded":"2024-03-28T13:09:42.943669+0000","last_fullsized":"2024-03-28T13:09:42.943669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:42:12.104485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687304+0000","last_change":"2024-03-28T13:09:41.677617+0000","last_active":"2024-03-28T13:09:42.687304+0000","last_peered":"2024-03-28T13:09:42.687304+0000","last_clean":"2024-03-28T13:09:42.687304+0000","last_became_active":"2024-03-28T13:09:41.677322+0000","last_became_peered":"2024-03-28T13:09:41.677322+0000","last_unstale":"2024-03-28T13:09:42.687304+0000","last_undegraded":"2024-03-28T13:09:42.687304+0000","last_fullsized":"2024-03-28T13:09:42.687304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:28:22.739597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685198+0000","last_change":"2024-03-28T13:09:40.650985+0000","last_active":"2024-03-28T13:09:42.685198+0000","last_peered":"2024-03-28T13:09:42.685198+0000","last_clean":"2024-03-28T13:09:42.685198+0000","last_became_active":"2024-03-28T13:09:40.648577+0000","last_became_peered":"2024-03-28T13:09:40.648577+0000","last_unstale":"2024-03-28T13:09:42.685198+0000","last_undegraded":"2024-03-28T13:09:42.685198+0000","last_fullsized":"2024-03-28T13:09:42.685198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:38:54.325203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695776+0000","last_change":"2024-03-28T13:09:41.691409+0000","last_active":"2024-03-28T13:09:42.695776+0000","last_peered":"2024-03-28T13:09:42.695776+0000","last_clean":"2024-03-28T13:09:42.695776+0000","last_became_active":"2024-03-28T13:09:41.690587+0000","last_became_peered":"2024-03-28T13:09:41.690587+0000","last_unstale":"2024-03-28T13:09:42.695776+0000","last_undegraded":"2024-03-28T13:09:42.695776+0000","last_fullsized":"2024-03-28T13:09:42.695776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:21:24.753764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687316+0000","last_change":"2024-03-28T13:09:40.648955+0000","last_active":"2024-03-28T13:09:42.687316+0000","last_peered":"2024-03-28T13:09:42.687316+0000","last_clean":"2024-03-28T13:09:42.687316+0000","last_became_active":"2024-03-28T13:09:40.648645+0000","last_became_peered":"2024-03-28T13:09:40.648645+0000","last_unstale":"2024-03-28T13:09:42.687316+0000","last_undegraded":"2024-03-28T13:09:42.687316+0000","last_fullsized":"2024-03-28T13:09:42.687316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:25:38.914424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.679924+0000","last_change":"2024-03-28T13:09:41.692962+0000","last_active":"2024-03-28T13:09:42.679924+0000","last_peered":"2024-03-28T13:09:42.679924+0000","last_clean":"2024-03-28T13:09:42.679924+0000","last_became_active":"2024-03-28T13:09:41.692660+0000","last_became_peered":"2024-03-28T13:09:41.692660+0000","last_unstale":"2024-03-28T13:09:42.679924+0000","last_undegraded":"2024-03-28T13:09:42.679924+0000","last_fullsized":"2024-03-28T13:09:42.679924+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:40.891627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685533+0000","last_change":"2024-03-28T13:09:40.654663+0000","last_active":"2024-03-28T13:09:42.685533+0000","last_peered":"2024-03-28T13:09:42.685533+0000","last_clean":"2024-03-28T13:09:42.685533+0000","last_became_active":"2024-03-28T13:09:40.654236+0000","last_became_peered":"2024-03-28T13:09:40.654236+0000","last_unstale":"2024-03-28T13:09:42.685533+0000","last_undegraded":"2024-03-28T13:09:42.685533+0000","last_fullsized":"2024-03-28T13:09:42.685533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:28.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.946112+0000","last_change":"2024-03-28T13:09:41.671654+0000","last_active":"2024-03-28T13:09:42.946112+0000","last_peered":"2024-03-28T13:09:42.946112+0000","last_clean":"2024-03-28T13:09:42.946112+0000","last_became_active":"2024-03-28T13:09:41.671232+0000","last_became_peered":"2024-03-28T13:09:41.671232+0000","last_unstale":"2024-03-28T13:09:42.946112+0000","last_undegraded":"2024-03-28T13:09:42.946112+0000","last_fullsized":"2024-03-28T13:09:42.946112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:21:02.278070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.694649+0000","last_change":"2024-03-28T13:09:40.651132+0000","last_active":"2024-03-28T13:09:42.694649+0000","last_peered":"2024-03-28T13:09:42.694649+0000","last_clean":"2024-03-28T13:09:42.694649+0000","last_became_active":"2024-03-28T13:09:40.650806+0000","last_became_peered":"2024-03-28T13:09:40.650806+0000","last_unstale":"2024-03-28T13:09:42.694649+0000","last_undegraded":"2024-03-28T13:09:42.694649+0000","last_fullsized":"2024-03-28T13:09:42.694649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:42:11.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691153+0000","last_change":"2024-03-28T13:09:41.686114+0000","last_active":"2024-03-28T13:09:42.691153+0000","last_peered":"2024-03-28T13:09:42.691153+0000","last_clean":"2024-03-28T13:09:42.691153+0000","last_became_active":"2024-03-28T13:09:41.685846+0000","last_became_peered":"2024-03-28T13:09:41.685846+0000","last_unstale":"2024-03-28T13:09:42.691153+0000","last_undegraded":"2024-03-28T13:09:42.691153+0000","last_fullsized":"2024-03-28T13:09:42.691153+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:28:13.777057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685901+0000","last_change":"2024-03-28T13:09:40.657051+0000","last_active":"2024-03-28T13:09:42.685901+0000","last_peered":"2024-03-28T13:09:42.685901+0000","last_clean":"2024-03-28T13:09:42.685901+0000","last_became_active":"2024-03-28T13:09:40.656680+0000","last_became_peered":"2024-03-28T13:09:40.656680+0000","last_unstale":"2024-03-28T13:09:42.685901+0000","last_undegraded":"2024-03-28T13:09:42.685901+0000","last_fullsized":"2024-03-28T13:09:42.685901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:44:33.508365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691166+0000","last_change":"2024-03-28T13:09:41.681460+0000","last_active":"2024-03-28T13:09:42.691166+0000","last_peered":"2024-03-28T13:09:42.691166+0000","last_clean":"2024-03-28T13:09:42.691166+0000","last_became_active":"2024-03-28T13:09:41.681136+0000","last_became_peered":"2024-03-28T13:09:41.681136+0000","last_unstale":"2024-03-28T13:09:42.691166+0000","last_undegraded":"2024-03-28T13:09:42.691166+0000","last_fullsized":"2024-03-28T13:09:42.691166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:48.181534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695372+0000","last_change":"2024-03-28T13:09:40.649525+0000","last_active":"2024-03-28T13:09:42.695372+0000","last_peered":"2024-03-28T13:09:42.695372+0000","last_clean":"2024-03-28T13:09:42.695372+0000","last_became_active":"2024-03-28T13:09:40.649204+0000","last_became_peered":"2024-03-28T13:09:40.649204+0000","last_unstale":"2024-03-28T13:09:42.695372+0000","last_undegraded":"2024-03-28T13:09:42.695372+0000","last_fullsized":"2024-03-28T13:09:42.695372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684678+0000","last_change":"2024-03-28T13:09:41.685557+0000","last_active":"2024-03-28T13:09:42.684678+0000","last_peered":"2024-03-28T13:09:42.684678+0000","last_clean":"2024-03-28T13:09:42.684678+0000","last_became_active":"2024-03-28T13:09:41.685000+0000","last_became_peered":"2024-03-28T13:09:41.685000+0000","last_unstale":"2024-03-28T13:09:42.684678+0000","last_undegraded":"2024-03-28T13:09:42.684678+0000","last_fullsized":"2024-03-28T13:09:42.684678+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:00.376154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.690868+0000","last_change":"2024-03-28T13:09:40.653861+0000","last_active":"2024-03-28T13:09:42.690868+0000","last_peered":"2024-03-28T13:09:42.690868+0000","last_clean":"2024-03-28T13:09:42.690868+0000","last_became_active":"2024-03-28T13:09:40.653467+0000","last_became_peered":"2024-03-28T13:09:40.653467+0000","last_unstale":"2024-03-28T13:09:42.690868+0000","last_undegraded":"2024-03-28T13:09:42.690868+0000","last_fullsized":"2024-03-28T13:09:42.690868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:28.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.690684+0000","last_change":"2024-03-28T13:09:41.677815+0000","last_active":"2024-03-28T13:09:42.690684+0000","last_peered":"2024-03-28T13:09:42.690684+0000","last_clean":"2024-03-28T13:09:42.690684+0000","last_became_active":"2024-03-28T13:09:41.677544+0000","last_became_peered":"2024-03-28T13:09:41.677544+0000","last_unstale":"2024-03-28T13:09:42.690684+0000","last_undegraded":"2024-03-28T13:09:42.690684+0000","last_fullsized":"2024-03-28T13:09:42.690684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:38:55.330149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945733+0000","last_change":"2024-03-28T13:09:40.649046+0000","last_active":"2024-03-28T13:09:42.945733+0000","last_peered":"2024-03-28T13:09:42.945733+0000","last_clean":"2024-03-28T13:09:42.945733+0000","last_became_active":"2024-03-28T13:09:40.648758+0000","last_became_peered":"2024-03-28T13:09:40.648758+0000","last_unstale":"2024-03-28T13:09:42.945733+0000","last_undegraded":"2024-03-28T13:09:42.945733+0000","last_fullsized":"2024-03-28T13:09:42.945733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:25:38.914424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944019+0000","last_change":"2024-03-28T13:09:41.685698+0000","last_active":"2024-03-28T13:09:42.944019+0000","last_peered":"2024-03-28T13:09:42.944019+0000","last_clean":"2024-03-28T13:09:42.944019+0000","last_became_active":"2024-03-28T13:09:41.685418+0000","last_became_peered":"2024-03-28T13:09:41.685418+0000","last_unstale":"2024-03-28T13:09:42.944019+0000","last_undegraded":"2024-03-28T13:09:42.944019+0000","last_fullsized":"2024-03-28T13:09:42.944019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:57:43.818964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685389+0000","last_change":"2024-03-28T13:09:40.653049+0000","last_active":"2024-03-28T13:09:42.685389+0000","last_peered":"2024-03-28T13:09:42.685389+0000","last_clean":"2024-03-28T13:09:42.685389+0000","last_became_active":"2024-03-28T13:09:40.652732+0000","last_became_peered":"2024-03-28T13:09:40.652732+0000","last_unstale":"2024-03-28T13:09:42.685389+0000","last_undegraded":"2024-03-28T13:09:42.685389+0000","last_fullsized":"2024-03-28T13:09:42.685389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:39.886681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685373+0000","last_change":"2024-03-28T13:09:41.690392+0000","last_active":"2024-03-28T13:09:42.685373+0000","last_peered":"2024-03-28T13:09:42.685373+0000","last_clean":"2024-03-28T13:09:42.685373+0000","last_became_active":"2024-03-28T13:09:41.689363+0000","last_became_peered":"2024-03-28T13:09:41.689363+0000","last_unstale":"2024-03-28T13:09:42.685373+0000","last_undegraded":"2024-03-28T13:09:42.685373+0000","last_fullsized":"2024-03-28T13:09:42.685373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:09:02.095518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685390+0000","last_change":"2024-03-28T13:09:41.700393+0000","last_active":"2024-03-28T13:09:42.685390+0000","last_peered":"2024-03-28T13:09:42.685390+0000","last_clean":"2024-03-28T13:09:42.685390+0000","last_became_active":"2024-03-28T13:09:41.700088+0000","last_became_peered":"2024-03-28T13:09:41.700088+0000","last_unstale":"2024-03-28T13:09:42.685390+0000","last_undegraded":"2024-03-28T13:09:42.685390+0000","last_fullsized":"2024-03-28T13:09:42.685390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:31.835464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696866+0000","last_change":"2024-03-28T13:09:40.648841+0000","last_active":"2024-03-28T13:09:42.696866+0000","last_peered":"2024-03-28T13:09:42.696866+0000","last_clean":"2024-03-28T13:09:42.696866+0000","last_became_active":"2024-03-28T13:09:40.646454+0000","last_became_peered":"2024-03-28T13:09:40.646454+0000","last_unstale":"2024-03-28T13:09:42.696866+0000","last_undegraded":"2024-03-28T13:09:42.696866+0000","last_fullsized":"2024-03-28T13:09:42.696866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:42:11.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942397+0000","last_change":"2024-03-28T13:09:41.705263+0000","last_active":"2024-03-28T13:09:42.942397+0000","last_peered":"2024-03-28T13:09:42.942397+0000","last_clean":"2024-03-28T13:09:42.942397+0000","last_became_active":"2024-03-28T13:09:41.704074+0000","last_became_peered":"2024-03-28T13:09:41.704074+0000","last_unstale":"2024-03-28T13:09:42.942397+0000","last_undegraded":"2024-03-28T13:09:42.942397+0000","last_fullsized":"2024-03-28T13:09:42.942397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:55:08.842535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695978+0000","last_change":"2024-03-28T13:09:40.649403+0000","last_active":"2024-03-28T13:09:42.695978+0000","last_peered":"2024-03-28T13:09:42.695978+0000","last_clean":"2024-03-28T13:09:42.695978+0000","last_became_active":"2024-03-28T13:09:40.648964+0000","last_became_peered":"2024-03-28T13:09:40.648964+0000","last_unstale":"2024-03-28T13:09:42.695978+0000","last_undegraded":"2024-03-28T13:09:42.695978+0000","last_fullsized":"2024-03-28T13:09:42.695978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:19:50.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688252+0000","last_change":"2024-03-28T13:09:41.688111+0000","last_active":"2024-03-28T13:09:42.688252+0000","last_peered":"2024-03-28T13:09:42.688252+0000","last_clean":"2024-03-28T13:09:42.688252+0000","last_became_active":"2024-03-28T13:09:41.687818+0000","last_became_peered":"2024-03-28T13:09:41.687818+0000","last_unstale":"2024-03-28T13:09:42.688252+0000","last_undegraded":"2024-03-28T13:09:42.688252+0000","last_fullsized":"2024-03-28T13:09:42.688252+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:38.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688212+0000","last_change":"2024-03-28T13:09:40.646771+0000","last_active":"2024-03-28T13:09:42.688212+0000","last_peered":"2024-03-28T13:09:42.688212+0000","last_clean":"2024-03-28T13:09:42.688212+0000","last_became_active":"2024-03-28T13:09:40.646496+0000","last_became_peered":"2024-03-28T13:09:40.646496+0000","last_unstale":"2024-03-28T13:09:42.688212+0000","last_undegraded":"2024-03-28T13:09:42.688212+0000","last_fullsized":"2024-03-28T13:09:42.688212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:19:50.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688194+0000","last_change":"2024-03-28T13:09:41.684932+0000","last_active":"2024-03-28T13:09:42.688194+0000","last_peered":"2024-03-28T13:09:42.688194+0000","last_clean":"2024-03-28T13:09:42.688194+0000","last_became_active":"2024-03-28T13:09:41.684314+0000","last_became_peered":"2024-03-28T13:09:41.684314+0000","last_unstale":"2024-03-28T13:09:42.688194+0000","last_undegraded":"2024-03-28T13:09:42.688194+0000","last_fullsized":"2024-03-28T13:09:42.688194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:43:20.588688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691848+0000","last_change":"2024-03-28T13:09:40.656875+0000","last_active":"2024-03-28T13:09:42.691848+0000","last_peered":"2024-03-28T13:09:42.691848+0000","last_clean":"2024-03-28T13:09:42.691848+0000","last_became_active":"2024-03-28T13:09:40.656569+0000","last_became_peered":"2024-03-28T13:09:40.656569+0000","last_unstale":"2024-03-28T13:09:42.691848+0000","last_undegraded":"2024-03-28T13:09:42.691848+0000","last_fullsized":"2024-03-28T13:09:42.691848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:29:29.680319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476744,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":80417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476742,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476742,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":670273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476742,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":79891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":79835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":831488,"data_stored":670295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":82119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":73659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:09:46.706 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-28T13:09:46.707 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-28T13:09:46.707 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-28T13:09:46.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-28T13:09:46.707 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-28T13:09:46.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.968+0000 7fc6bea00640 1 Processor -- start 2024-03-28T13:09:46.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.969+0000 7fc6bea00640 1 -- start start 2024-03-28T13:09:46.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.970+0000 7fc6bea00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc6b815e4a0 0x7fc6b80b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.970+0000 7fc6bea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc6b80d2310 0x7fc6b80ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.970+0000 7fc6bea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b80cf530 0x7fc6b80d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.970+0000 7fc6bea00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc6b80d34e0 con 0x7fc6b80d2310 2024-03-28T13:09:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.970+0000 7fc6bea00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc6b815e880 con 0x7fc6b815e4a0 2024-03-28T13:09:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.970+0000 7fc6bea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b80d28d0 con 0x7fc6b80cf530 2024-03-28T13:09:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.970+0000 7fc6bcc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc6b80d2310 0x7fc6b80ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.971+0000 7fc6bcc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc6b80d2310 0x7fc6b80ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50586/0 (socket says 172.21.15.19:50586) 2024-03-28T13:09:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.971+0000 7fc6bcc00640 1 -- 172.21.15.19:0/3198730214 learned_addr learned my addr 172.21.15.19:0/3198730214 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.971+0000 7fc6bd600640 1 --2- 172.21.15.19:0/3198730214 >> v2:172.21.15.188:3300/0 conn(0x7fc6b815e4a0 0x7fc6b80b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:46.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.971+0000 7fc6be000640 1 --2- 172.21.15.19:0/3198730214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b80cf530 0x7fc6b80d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.971+0000 7fc6bcc00640 1 -- 172.21.15.19:0/3198730214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b80cf530 msgr2=0x7fc6b80d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.971+0000 7fc6bcc00640 1 --2- 172.21.15.19:0/3198730214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b80cf530 0x7fc6b80d3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.971+0000 7fc6bcc00640 1 -- 172.21.15.19:0/3198730214 >> v2:172.21.15.188:3300/0 conn(0x7fc6b815e4a0 msgr2=0x7fc6b80b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.971+0000 7fc6bcc00640 1 --2- 172.21.15.19:0/3198730214 >> v2:172.21.15.188:3300/0 conn(0x7fc6b815e4a0 0x7fc6b80b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.971+0000 7fc6bcc00640 1 -- 172.21.15.19:0/3198730214 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b80d2bb0 con 0x7fc6b80d2310 2024-03-28T13:09:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.974+0000 7fc6bcc00640 1 --2- 172.21.15.19:0/3198730214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6b80d2310 0x7fc6b80ceff0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80cfb20 tx=0x7fc6a402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44152c2603b7816b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.974+0000 7fc6b7e00640 1 -- 172.21.15.19:0/3198730214 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a402e070 con 0x7fc6b80d2310 2024-03-28T13:09:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.974+0000 7fc6b7e00640 1 -- 172.21.15.19:0/3198730214 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6a4037070 con 0x7fc6b80d2310 2024-03-28T13:09:46.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.974+0000 7fc6b7e00640 1 -- 172.21.15.19:0/3198730214 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4031a00 con 0x7fc6b80d2310 2024-03-28T13:09:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.975+0000 7fc6bea00640 1 -- 172.21.15.19:0/3198730214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6b80d2310 msgr2=0x7fc6b80ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.975+0000 7fc6bea00640 1 --2- 172.21.15.19:0/3198730214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6b80d2310 0x7fc6b80ceff0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80cfb20 tx=0x7fc6a402b880 comp rx=0 tx=0).stop 2024-03-28T13:09:46.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.975+0000 7fc6bea00640 1 -- 172.21.15.19:0/3198730214 shutdown_connections 2024-03-28T13:09:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.975+0000 7fc6bea00640 1 --2- 172.21.15.19:0/3198730214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b80cf530 0x7fc6b80d3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.975+0000 7fc6bea00640 1 --2- 172.21.15.19:0/3198730214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6b80d2310 0x7fc6b80ceff0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.975+0000 7fc6bea00640 1 --2- 172.21.15.19:0/3198730214 >> v2:172.21.15.188:3300/0 conn(0x7fc6b815e4a0 0x7fc6b80b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.975+0000 7fc6bea00640 1 -- 172.21.15.19:0/3198730214 >> 172.21.15.19:0/3198730214 conn(0x7fc6b80676d0 msgr2=0x7fc6b8067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.975+0000 7fc6bea00640 1 -- 172.21.15.19:0/3198730214 shutdown_connections 2024-03-28T13:09:46.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.975+0000 7fc6bea00640 1 -- 172.21.15.19:0/3198730214 wait complete. 2024-03-28T13:09:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.977+0000 7fc6bea00640 1 Processor -- start 2024-03-28T13:09:46.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.977+0000 7fc6bea00640 1 -- start start 2024-03-28T13:09:46.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.978+0000 7fc6bea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6b80cf530 0x7fc6b81559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:46.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.978+0000 7fc6bea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6b80d2310 0x7fc6b8160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:46.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.978+0000 7fc6bea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b815e4a0 0x7fc6b80ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:46.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.978+0000 7fc6bea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b80d3970 con 0x7fc6b80d2310 2024-03-28T13:09:46.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.978+0000 7fc6bea00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b80d3170 con 0x7fc6b80cf530 2024-03-28T13:09:46.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.978+0000 7fc6bea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc6b80ba1a0 con 0x7fc6b815e4a0 2024-03-28T13:09:46.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.979+0000 7fc6bcc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6b80d2310 0x7fc6b8160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:46.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.979+0000 7fc6bcc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6b80d2310 0x7fc6b8160fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50598/0 (socket says 172.21.15.19:50598) 2024-03-28T13:09:46.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.979+0000 7fc6bcc00640 1 -- 172.21.15.19:0/3501209575 learned_addr learned my addr 172.21.15.19:0/3501209575 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:46.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.979+0000 7fc6bd600640 1 --2- 172.21.15.19:0/3501209575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6b80cf530 0x7fc6b81559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.979+0000 7fc6bcc00640 1 -- 172.21.15.19:0/3501209575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b815e4a0 msgr2=0x7fc6b80ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:46.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.979+0000 7fc6bcc00640 1 --2- 172.21.15.19:0/3501209575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b815e4a0 0x7fc6b80ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.979+0000 7fc6bcc00640 1 -- 172.21.15.19:0/3501209575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6b80cf530 msgr2=0x7fc6b81559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.979+0000 7fc6bcc00640 1 --2- 172.21.15.19:0/3501209575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6b80cf530 0x7fc6b81559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.979+0000 7fc6bcc00640 1 -- 172.21.15.19:0/3501209575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b81562b0 con 0x7fc6b80d2310 2024-03-28T13:09:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.980+0000 7fc6bcc00640 1 --2- 172.21.15.19:0/3501209575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6b80d2310 0x7fc6b8160fb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc6a402b8c0 tx=0x7fc6a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.980+0000 7fc6b5600640 1 -- 172.21.15.19:0/3501209575 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4032670 con 0x7fc6b80d2310 2024-03-28T13:09:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.980+0000 7fc6b5600640 1 -- 172.21.15.19:0/3501209575 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6a4032830 con 0x7fc6b80d2310 2024-03-28T13:09:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.980+0000 7fc6b5600640 1 -- 172.21.15.19:0/3501209575 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4032c20 con 0x7fc6b80d2310 2024-03-28T13:09:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.980+0000 7fc6bea00640 1 -- 172.21.15.19:0/3501209575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b8156590 con 0x7fc6b80d2310 2024-03-28T13:09:46.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.980+0000 7fc6bea00640 1 -- 172.21.15.19:0/3501209575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b8156b50 con 0x7fc6b80d2310 2024-03-28T13:09:46.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.982+0000 7fc6bea00640 1 -- 172.21.15.19:0/3501209575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b80cf320 con 0x7fc6b80d2310 2024-03-28T13:09:46.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.982+0000 7fc6b5600640 1 -- 172.21.15.19:0/3501209575 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc6a404c060 con 0x7fc6b80d2310 2024-03-28T13:09:46.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.984+0000 7fc6b5600640 1 --2- 172.21.15.19:0/3501209575 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc69806c5b0 0x7fc69806ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:46.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.984+0000 7fc6b5600640 1 -- 172.21.15.19:0/3501209575 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc6a40acfc0 con 0x7fc6b80d2310 2024-03-28T13:09:46.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.984+0000 7fc6bd600640 1 --2- 172.21.15.19:0/3501209575 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc69806c5b0 0x7fc69806ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:46.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.985+0000 7fc6bd600640 1 --2- 172.21.15.19:0/3501209575 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc69806c5b0 0x7fc69806ea70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009a50 tx=0x7fc6ac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:46.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:46.989+0000 7fc6b5600640 1 -- 172.21.15.19:0/3501209575 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc6b80cf320 con 0x7fc6b80d2310 2024-03-28T13:09:47.121 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-03-28T13:09:47.121+0000 7fe3b8200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:09:47.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.244+0000 7fc6bea00640 1 -- 172.21.15.19:0/3501209575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc6b8075940 con 0x7fc6b80d2310 2024-03-28T13:09:47.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.248+0000 7fc6b5600640 1 -- 172.21.15.19:0/3501209575 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+173739 (secure 0 0 0) 0x7fc6a4040070 con 0x7fc6b80d2310 2024-03-28T13:09:47.250 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.251+0000 7fc6bea00640 1 -- 172.21.15.19:0/3501209575 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc69806c5b0 msgr2=0x7fc69806ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.251+0000 7fc6bea00640 1 --2- 172.21.15.19:0/3501209575 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc69806c5b0 0x7fc69806ea70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac009a50 tx=0x7fc6ac008040 comp rx=0 tx=0).stop 2024-03-28T13:09:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.251+0000 7fc6bea00640 1 -- 172.21.15.19:0/3501209575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6b80d2310 msgr2=0x7fc6b8160fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.251+0000 7fc6bea00640 1 --2- 172.21.15.19:0/3501209575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6b80d2310 0x7fc6b8160fb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc6a402b8c0 tx=0x7fc6a40046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.251+0000 7fc6bea00640 1 -- 172.21.15.19:0/3501209575 shutdown_connections 2024-03-28T13:09:47.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.251+0000 7fc6bea00640 1 --2- 172.21.15.19:0/3501209575 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc69806c5b0 0x7fc69806ea70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.251+0000 7fc6bea00640 1 --2- 172.21.15.19:0/3501209575 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc6b815e4a0 0x7fc6b80ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.251+0000 7fc6bea00640 1 --2- 172.21.15.19:0/3501209575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc6b80d2310 0x7fc6b8160fb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.251+0000 7fc6bea00640 1 --2- 172.21.15.19:0/3501209575 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc6b80cf530 0x7fc6b81559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.251+0000 7fc6bea00640 1 -- 172.21.15.19:0/3501209575 >> 172.21.15.19:0/3501209575 conn(0x7fc6b80676d0 msgr2=0x7fc6b80b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:47.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.252+0000 7fc6bea00640 1 -- 172.21.15.19:0/3501209575 shutdown_connections 2024-03-28T13:09:47.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.252+0000 7fc6bea00640 1 -- 172.21.15.19:0/3501209575 wait complete. 2024-03-28T13:09:47.270 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3382915391},{"type":"v1","addr":"172.21.15.188:6833","nonce":3382915391}]},"active_addr":"172.21.15.188:6833/3382915391","active_change":"2024-03-28T13:09:21.954039+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":80738434}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":249920926}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":3372162463}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.188:0","nonce":3822996104}]}]} 2024-03-28T13:09:47.273 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-28T13:09:47.274 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-28T13:09:47.274 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:09:47.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.488+0000 7f10dd200640 1 Processor -- start 2024-03-28T13:09:47.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.489+0000 7f10dd200640 1 -- start start 2024-03-28T13:09:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.490+0000 7f10dd200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f10d815fa10 0x7f10d80b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.490+0000 7f10dd200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f10d80d2bf0 0x7f10d80cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.490+0000 7f10dd200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10d80d3140 con 0x7f10d80d0c30 2024-03-28T13:09:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.491+0000 7f10dd200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f10d80d3420 con 0x7f10d815fa10 2024-03-28T13:09:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.491+0000 7f10dd200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f10d80cf2b0 con 0x7f10d80d2bf0 2024-03-28T13:09:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.491+0000 7f10d6800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f10d80d0c30 0x7f10d81603b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42890/0 (socket says 172.21.15.19:42890) 2024-03-28T13:09:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.491+0000 7f10d5e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f10d815fa10 0x7f10d80b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.491+0000 7f10d6800640 1 -- 172.21.15.19:0/1137811374 learned_addr learned my addr 172.21.15.19:0/1137811374 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.491+0000 7f10cfe00640 1 --2- 172.21.15.19:0/1137811374 >> v2:172.21.15.188:3300/0 conn(0x7f10d80d2bf0 0x7f10d80cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.491+0000 7f10d5e00640 1 -- 172.21.15.19:0/1137811374 >> v2:172.21.15.188:3300/0 conn(0x7f10d80d2bf0 msgr2=0x7f10d80cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.492+0000 7f10d5e00640 1 --2- 172.21.15.19:0/1137811374 >> v2:172.21.15.188:3300/0 conn(0x7f10d80d2bf0 0x7f10d80cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.492+0000 7f10d5e00640 1 -- 172.21.15.19:0/1137811374 >> v1:172.21.15.19:6789/0 conn(0x7f10d80d0c30 legacy=0x7f10d81603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.492+0000 7f10d5e00640 1 -- 172.21.15.19:0/1137811374 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d80cf590 con 0x7f10d815fa10 2024-03-28T13:09:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.493+0000 7f10d5e00640 1 --2- 172.21.15.19:0/1137811374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d815fa10 0x7f10d80b55a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f10d808c670 tx=0x7f10c802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ee27da493ebc0c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:47.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.493+0000 7f10d5400640 1 -- 172.21.15.19:0/1137811374 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8034070 con 0x7f10d815fa10 2024-03-28T13:09:47.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.493+0000 7f10d5400640 1 -- 172.21.15.19:0/1137811374 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c8030610 con 0x7f10d815fa10 2024-03-28T13:09:47.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.493+0000 7f10d5400640 1 -- 172.21.15.19:0/1137811374 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c80309e0 con 0x7f10d815fa10 2024-03-28T13:09:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.494+0000 7f10dd200640 1 -- 172.21.15.19:0/1137811374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d815fa10 msgr2=0x7f10d80b55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.494+0000 7f10dd200640 1 --2- 172.21.15.19:0/1137811374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d815fa10 0x7f10d80b55a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f10d808c670 tx=0x7f10c802b960 comp rx=0 tx=0).stop 2024-03-28T13:09:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.494+0000 7f10dd200640 1 -- 172.21.15.19:0/1137811374 shutdown_connections 2024-03-28T13:09:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.494+0000 7f10dd200640 1 --2- 172.21.15.19:0/1137811374 >> v2:172.21.15.188:3300/0 conn(0x7f10d80d2bf0 0x7f10d80cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.494+0000 7f10dd200640 1 --2- 172.21.15.19:0/1137811374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d815fa10 0x7f10d80b55a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.494+0000 7f10dd200640 1 -- 172.21.15.19:0/1137811374 >> 172.21.15.19:0/1137811374 conn(0x7f10d808bfb0 msgr2=0x7f10d80d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:47.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.494+0000 7f10dd200640 1 -- 172.21.15.19:0/1137811374 shutdown_connections 2024-03-28T13:09:47.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.494+0000 7f10dd200640 1 -- 172.21.15.19:0/1137811374 wait complete. 2024-03-28T13:09:47.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.495+0000 7f10dd200640 1 Processor -- start 2024-03-28T13:09:47.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.496+0000 7f10dd200640 1 -- start start 2024-03-28T13:09:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.496+0000 7f10dd200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10d80d0c30 0x7f10d80b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.497+0000 7f10dd200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10d80d2bf0 0x7f10d81568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.497+0000 7f10dd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d815fa10 0x7f10d80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.497+0000 7f10dd200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10d80b5c60 con 0x7f10d815fa10 2024-03-28T13:09:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.497+0000 7f10dd200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f10d80b5ae0 con 0x7f10d80d2bf0 2024-03-28T13:09:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.497+0000 7f10dd200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f10d80d02d0 con 0x7f10d80d0c30 2024-03-28T13:09:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.497+0000 7f10d6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d815fa10 0x7f10d80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.497+0000 7f10d6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d815fa10 0x7f10d80b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50614/0 (socket says 172.21.15.19:50614) 2024-03-28T13:09:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.497+0000 7f10d6800640 1 -- 172.21.15.19:0/627903247 learned_addr learned my addr 172.21.15.19:0/627903247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.497+0000 7f10cfe00640 1 --2- 172.21.15.19:0/627903247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10d80d2bf0 0x7f10d81568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.497+0000 7f10d5e00640 1 --2- 172.21.15.19:0/627903247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10d80d0c30 0x7f10d80b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.498+0000 7f10d6800640 1 -- 172.21.15.19:0/627903247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10d80d0c30 msgr2=0x7f10d80b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.498+0000 7f10d6800640 1 --2- 172.21.15.19:0/627903247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10d80d0c30 0x7f10d80b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.498+0000 7f10d6800640 1 -- 172.21.15.19:0/627903247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10d80d2bf0 msgr2=0x7f10d81568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.498+0000 7f10d6800640 1 --2- 172.21.15.19:0/627903247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10d80d2bf0 0x7f10d81568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.498+0000 7f10d6800640 1 -- 172.21.15.19:0/627903247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10d80b4ec0 con 0x7f10d815fa10 2024-03-28T13:09:47.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.498+0000 7f10d6800640 1 --2- 172.21.15.19:0/627903247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d815fa10 0x7f10d80b1b40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f10d000cbf0 tx=0x7f10d000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:47.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.498+0000 7f10ce000640 1 -- 172.21.15.19:0/627903247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0017070 con 0x7f10d815fa10 2024-03-28T13:09:47.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.498+0000 7f10dd200640 1 -- 172.21.15.19:0/627903247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10d80cb990 con 0x7f10d815fa10 2024-03-28T13:09:47.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.498+0000 7f10ce000640 1 -- 172.21.15.19:0/627903247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d000ad40 con 0x7f10d815fa10 2024-03-28T13:09:47.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.498+0000 7f10ce000640 1 -- 172.21.15.19:0/627903247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d0012600 con 0x7f10d815fa10 2024-03-28T13:09:47.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.498+0000 7f10dd200640 1 -- 172.21.15.19:0/627903247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10d80cbf50 con 0x7f10d815fa10 2024-03-28T13:09:47.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.500+0000 7f10dd200640 1 -- 172.21.15.19:0/627903247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10d8068cd0 con 0x7f10d815fa10 2024-03-28T13:09:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.501+0000 7f10ce000640 1 -- 172.21.15.19:0/627903247 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f10d001a070 con 0x7f10d815fa10 2024-03-28T13:09:47.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.502+0000 7f10ce000640 1 --2- 172.21.15.19:0/627903247 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f10b406c530 0x7f10b406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:47.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.502+0000 7f10ce000640 1 -- 172.21.15.19:0/627903247 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f10d008b4a0 con 0x7f10d815fa10 2024-03-28T13:09:47.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.503+0000 7f10d5e00640 1 --2- 172.21.15.19:0/627903247 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f10b406c530 0x7f10b406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:47.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.504+0000 7f10d5e00640 1 --2- 172.21.15.19:0/627903247 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f10b406c530 0x7f10b406e9f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f10c8036000 tx=0x7f10c803b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:47.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.507+0000 7f10ce000640 1 -- 172.21.15.19:0/627903247 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f10d0059350 con 0x7f10d815fa10 2024-03-28T13:09:47.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.713+0000 7f10dd200640 1 -- 172.21.15.19:0/627903247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10d80ba3d0 con 0x7f10d815fa10 2024-03-28T13:09:47.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.715+0000 7f10ce000640 1 -- 172.21.15.19:0/627903247 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16609 (secure 0 0 0) 0x7f10d005d650 con 0x7f10d815fa10 2024-03-28T13:09:47.715 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:47.716 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":23,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:42.668077+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:09:47.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.718+0000 7f10dd200640 1 -- 172.21.15.19:0/627903247 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f10b406c530 msgr2=0x7f10b406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:47.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.718+0000 7f10dd200640 1 --2- 172.21.15.19:0/627903247 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f10b406c530 0x7f10b406e9f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f10c8036000 tx=0x7f10c803b040 comp rx=0 tx=0).stop 2024-03-28T13:09:47.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.718+0000 7f10dd200640 1 -- 172.21.15.19:0/627903247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d815fa10 msgr2=0x7f10d80b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:47.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.718+0000 7f10dd200640 1 --2- 172.21.15.19:0/627903247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d815fa10 0x7f10d80b1b40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f10d000cbf0 tx=0x7f10d000a430 comp rx=0 tx=0).stop 2024-03-28T13:09:47.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.718+0000 7f10dd200640 1 -- 172.21.15.19:0/627903247 shutdown_connections 2024-03-28T13:09:47.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.718+0000 7f10dd200640 1 --2- 172.21.15.19:0/627903247 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f10b406c530 0x7f10b406e9f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.718+0000 7f10dd200640 1 --2- 172.21.15.19:0/627903247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10d815fa10 0x7f10d80b1b40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.718+0000 7f10dd200640 1 --2- 172.21.15.19:0/627903247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f10d80d2bf0 0x7f10d81568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.718+0000 7f10dd200640 1 --2- 172.21.15.19:0/627903247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10d80d0c30 0x7f10d80b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.718+0000 7f10dd200640 1 -- 172.21.15.19:0/627903247 >> 172.21.15.19:0/627903247 conn(0x7f10d808bfb0 msgr2=0x7f10d815ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:47.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.718+0000 7f10dd200640 1 -- 172.21.15.19:0/627903247 shutdown_connections 2024-03-28T13:09:47.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.718+0000 7f10dd200640 1 -- 172.21.15.19:0/627903247 wait complete. 2024-03-28T13:09:47.736 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-28T13:09:47.736 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:09:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.933+0000 7ff616c00640 1 Processor -- start 2024-03-28T13:09:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.933+0000 7ff616c00640 1 -- start start 2024-03-28T13:09:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.935+0000 7ff616c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff61014ba20 0x7ff6100b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.935+0000 7ff616c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6100b33e0 con 0x7ff6100cfda0 2024-03-28T13:09:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.935+0000 7ff616c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6100cde20 con 0x7ff61014c1b0 2024-03-28T13:09:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.935+0000 7ff616c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff610166ae0 con 0x7ff61014ba20 2024-03-28T13:09:47.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.935+0000 7ff616200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff6100cfda0 0x7ff6100d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42894/0 (socket says 172.21.15.19:42894) 2024-03-28T13:09:47.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.935+0000 7ff616200640 1 -- 172.21.15.19:0/3409646008 learned_addr learned my addr 172.21.15.19:0/3409646008 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:47.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.936+0000 7ff615800640 1 --2- 172.21.15.19:0/3409646008 >> v2:172.21.15.188:3300/0 conn(0x7ff61014ba20 0x7ff6100b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:47.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.936+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3447129379 0 0) 0x7ff6100b33e0 con 0x7ff6100cfda0 2024-03-28T13:09:47.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.936+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5f8003520 con 0x7ff6100cfda0 2024-03-28T13:09:47.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.936+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3935131084 0 0) 0x7ff6100cde20 con 0x7ff61014c1b0 2024-03-28T13:09:47.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.936+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff6100b33e0 con 0x7ff61014c1b0 2024-03-28T13:09:47.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.936+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4003450700 0 0) 0x7ff5f8003520 con 0x7ff6100cfda0 2024-03-28T13:09:47.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.936+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 >> v2:172.21.15.188:3300/0 conn(0x7ff61014ba20 msgr2=0x7ff6100b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:47.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.936+0000 7ff60fe00640 1 --2- 172.21.15.19:0/3409646008 >> v2:172.21.15.188:3300/0 conn(0x7ff61014ba20 0x7ff6100b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.936+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 >> v1:172.21.15.188:6789/0 conn(0x7ff61014c1b0 legacy=0x7ff610166290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:47.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.936+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff610166dc0 con 0x7ff6100cfda0 2024-03-28T13:09:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.936+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7ff6080019f0 con 0x7ff6100cfda0 2024-03-28T13:09:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.936+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 >> v1:172.21.15.19:6789/0 conn(0x7ff6100cfda0 legacy=0x7ff6100d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.937+0000 7ff60fe00640 1 --2- 172.21.15.19:0/3409646008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5f8003b40 0x7ff5f8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.937+0000 7ff60fe00640 1 --2- 172.21.15.19:0/3409646008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5f8004800 0x7ff5f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.937+0000 7ff60fe00640 1 --2- 172.21.15.19:0/3409646008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80051c0 0x7ff5f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.937+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff610166ae0 con 0x7ff5f80051c0 2024-03-28T13:09:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.937+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f800c040 con 0x7ff5f8003b40 2024-03-28T13:09:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.937+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f800c240 con 0x7ff5f8004800 2024-03-28T13:09:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.937+0000 7ff616200640 1 --2- 172.21.15.19:0/3409646008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80051c0 0x7ff5f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.937+0000 7ff615800640 1 --2- 172.21.15.19:0/3409646008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5f8003b40 0x7ff5f8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.937+0000 7ff614e00640 1 --2- 172.21.15.19:0/3409646008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5f8004800 0x7ff5f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.937+0000 7ff616200640 1 -- 172.21.15.19:0/3409646008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5f8004800 msgr2=0x7ff5f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.937+0000 7ff616200640 1 --2- 172.21.15.19:0/3409646008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5f8004800 0x7ff5f8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.937+0000 7ff616200640 1 -- 172.21.15.19:0/3409646008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5f8003b40 msgr2=0x7ff5f8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:47.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.937+0000 7ff616200640 1 --2- 172.21.15.19:0/3409646008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5f8003b40 0x7ff5f8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.937+0000 7ff616200640 1 -- 172.21.15.19:0/3409646008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f8003520 con 0x7ff5f80051c0 2024-03-28T13:09:47.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.937+0000 7ff615800640 1 --2- 172.21.15.19:0/3409646008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5f8003b40 0x7ff5f8003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.938+0000 7ff616200640 1 -- 172.21.15.19:0/3409646008 reap_dead start 2024-03-28T13:09:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.938+0000 7ff616200640 1 --2- 172.21.15.19:0/3409646008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80051c0 0x7ff5f800b9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff60800a800 tx=0x7ff5f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc86a6e866590ddf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.939+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60800a450 con 0x7ff5f80051c0 2024-03-28T13:09:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.939+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff608034650 con 0x7ff5f80051c0 2024-03-28T13:09:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.939+0000 7ff60fe00640 1 -- 172.21.15.19:0/3409646008 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60802ce60 con 0x7ff5f80051c0 2024-03-28T13:09:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.939+0000 7ff616c00640 1 -- 172.21.15.19:0/3409646008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80051c0 msgr2=0x7ff5f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.939+0000 7ff616c00640 1 --2- 172.21.15.19:0/3409646008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80051c0 0x7ff5f800b9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff60800a800 tx=0x7ff5f8005830 comp rx=0 tx=0).stop 2024-03-28T13:09:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.939+0000 7ff616c00640 1 -- 172.21.15.19:0/3409646008 shutdown_connections 2024-03-28T13:09:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.939+0000 7ff616c00640 1 --2- 172.21.15.19:0/3409646008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80051c0 0x7ff5f800b9c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.939+0000 7ff616c00640 1 -- 172.21.15.19:0/3409646008 >> 172.21.15.19:0/3409646008 conn(0x7ff6100ccad0 msgr2=0x7ff6100cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.940+0000 7ff616c00640 1 -- 172.21.15.19:0/3409646008 shutdown_connections 2024-03-28T13:09:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.940+0000 7ff616c00640 1 -- 172.21.15.19:0/3409646008 wait complete. 2024-03-28T13:09:47.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.941+0000 7ff616c00640 1 Processor -- start 2024-03-28T13:09:47.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.941+0000 7ff616c00640 1 -- start start 2024-03-28T13:09:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.942+0000 7ff616c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80051c0 0x7ff610139ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.942+0000 7ff616c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6101ee850 0x7ff6100b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.942+0000 7ff616c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6101eecd0 0x7ff6100b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.942+0000 7ff616c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff61014be00 con 0x7ff5f80051c0 2024-03-28T13:09:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.942+0000 7ff616c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6100ce0c0 con 0x7ff6101eecd0 2024-03-28T13:09:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.942+0000 7ff616c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff61014bf80 con 0x7ff6101ee850 2024-03-28T13:09:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.943+0000 7ff615800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80051c0 0x7ff610139ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.943+0000 7ff615800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80051c0 0x7ff610139ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50644/0 (socket says 172.21.15.19:50644) 2024-03-28T13:09:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.943+0000 7ff615800640 1 -- 172.21.15.19:0/4154603900 learned_addr learned my addr 172.21.15.19:0/4154603900 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.943+0000 7ff615800640 1 -- 172.21.15.19:0/4154603900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6101ee850 msgr2=0x7ff6100b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.943+0000 7ff615800640 1 --2- 172.21.15.19:0/4154603900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6101ee850 0x7ff6100b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.943+0000 7ff615800640 1 -- 172.21.15.19:0/4154603900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6101eecd0 msgr2=0x7ff6100b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.943+0000 7ff615800640 1 --2- 172.21.15.19:0/4154603900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6101eecd0 0x7ff6100b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.943+0000 7ff615800640 1 -- 172.21.15.19:0/4154603900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff610162d10 con 0x7ff5f80051c0 2024-03-28T13:09:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.943+0000 7ff615800640 1 --2- 172.21.15.19:0/4154603900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80051c0 0x7ff610139ef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff604011570 tx=0x7ff604011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.944+0000 7ff60d600640 1 -- 172.21.15.19:0/4154603900 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff604015870 con 0x7ff5f80051c0 2024-03-28T13:09:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.944+0000 7ff60d600640 1 -- 172.21.15.19:0/4154603900 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff604015a10 con 0x7ff5f80051c0 2024-03-28T13:09:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.944+0000 7ff60d600640 1 -- 172.21.15.19:0/4154603900 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff604015dc0 con 0x7ff5f80051c0 2024-03-28T13:09:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.944+0000 7ff616c00640 1 -- 172.21.15.19:0/4154603900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff610162ff0 con 0x7ff5f80051c0 2024-03-28T13:09:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.944+0000 7ff616c00640 1 -- 172.21.15.19:0/4154603900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6101635b0 con 0x7ff5f80051c0 2024-03-28T13:09:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.945+0000 7ff616c00640 1 -- 172.21.15.19:0/4154603900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff610075940 con 0x7ff5f80051c0 2024-03-28T13:09:47.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.946+0000 7ff60d600640 1 -- 172.21.15.19:0/4154603900 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff60401d400 con 0x7ff5f80051c0 2024-03-28T13:09:47.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.947+0000 7ff60d600640 1 --2- 172.21.15.19:0/4154603900 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff5f006c530 0x7ff5f006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:47.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.947+0000 7ff60d600640 1 -- 172.21.15.19:0/4154603900 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff60408bf60 con 0x7ff5f80051c0 2024-03-28T13:09:47.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.948+0000 7ff614e00640 1 --2- 172.21.15.19:0/4154603900 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff5f006c530 0x7ff5f006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:47.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.949+0000 7ff614e00640 1 --2- 172.21.15.19:0/4154603900 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff5f006c530 0x7ff5f006e9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0011e0 tx=0x7ff5fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:47.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:47.952+0000 7ff60d600640 1 -- 172.21.15.19:0/4154603900 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff604019030 con 0x7ff5f80051c0 2024-03-28T13:09:48.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.158+0000 7ff616c00640 1 -- 172.21.15.19:0/4154603900 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff61006f430 con 0x7ff5f80051c0 2024-03-28T13:09:48.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.160+0000 7ff60d600640 1 -- 172.21.15.19:0/4154603900 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16609 (secure 0 0 0) 0x7ff604059f20 con 0x7ff5f80051c0 2024-03-28T13:09:48.160 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:48.160 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":23,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:42.668077+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:09:48.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.163+0000 7ff616c00640 1 -- 172.21.15.19:0/4154603900 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff5f006c530 msgr2=0x7ff5f006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.163+0000 7ff616c00640 1 --2- 172.21.15.19:0/4154603900 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff5f006c530 0x7ff5f006e9f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0011e0 tx=0x7ff5fc009040 comp rx=0 tx=0).stop 2024-03-28T13:09:48.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.163+0000 7ff616c00640 1 -- 172.21.15.19:0/4154603900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80051c0 msgr2=0x7ff610139ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.163+0000 7ff616c00640 1 --2- 172.21.15.19:0/4154603900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80051c0 0x7ff610139ef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff604011570 tx=0x7ff604011a30 comp rx=0 tx=0).stop 2024-03-28T13:09:48.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.163+0000 7ff616c00640 1 -- 172.21.15.19:0/4154603900 shutdown_connections 2024-03-28T13:09:48.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.163+0000 7ff616c00640 1 --2- 172.21.15.19:0/4154603900 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff5f006c530 0x7ff5f006e9f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.163+0000 7ff616c00640 1 --2- 172.21.15.19:0/4154603900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6101eecd0 0x7ff6100b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.163+0000 7ff616c00640 1 --2- 172.21.15.19:0/4154603900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6101ee850 0x7ff6100b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.163+0000 7ff616c00640 1 --2- 172.21.15.19:0/4154603900 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5f80051c0 0x7ff610139ef0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.163+0000 7ff616c00640 1 -- 172.21.15.19:0/4154603900 >> 172.21.15.19:0/4154603900 conn(0x7ff6100ccad0 msgr2=0x7ff6100ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.164+0000 7ff616c00640 1 -- 172.21.15.19:0/4154603900 shutdown_connections 2024-03-28T13:09:48.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.164+0000 7ff616c00640 1 -- 172.21.15.19:0/4154603900 wait complete. 2024-03-28T13:09:48.182 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-28T13:09:48.186 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-28T13:09:48.186 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-28T13:09:48.187 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-28T13:09:48.187 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-28T13:09:48.188 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-28T13:09:48.188 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-28T13:09:48.189 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-28T13:09:48.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.502+0000 7f791ba00640 1 Processor -- start 2024-03-28T13:09:48.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.504+0000 7f791ba00640 1 -- start start 2024-03-28T13:09:48.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.505+0000 7feff4a00640 1 Processor -- start 2024-03-28T13:09:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.507+0000 7f791ba00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79140d03b0 0x7f79140d0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.505+0000 7feff4a00640 1 -- start start 2024-03-28T13:09:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.507+0000 7f791ba00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f79140b3cf0 0x7f7914071940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.507+0000 7f791ba00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7914071e80 0x7f79140ad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.508+0000 7f791ba00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f79140726a0 con 0x7f7914071e80 2024-03-28T13:09:48.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.508+0000 7f7913e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f79140b3cf0 0x7f7914071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.508+0000 7f791ba00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f79140d0cd0 con 0x7f79140b3cf0 2024-03-28T13:09:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.508+0000 7f7913e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f79140b3cf0 0x7f7914071940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:43968/0 (socket says 172.21.15.19:43968) 2024-03-28T13:09:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.508+0000 7f791ba00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f79140723c0 con 0x7f79140d03b0 2024-03-28T13:09:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.508+0000 7f7913e00640 1 -- 172.21.15.19:0/3655669929 learned_addr learned my addr 172.21.15.19:0/3655669929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.508+0000 7f7919a00640 1 --2- 172.21.15.19:0/3655669929 >> v2:172.21.15.19:3300/0 conn(0x7f7914071e80 0x7f79140ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.508+0000 7feff4a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefe80ebb10 0x7fefe80552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.509+0000 7feff4a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe80387a0 con 0x7fefe80eb020 2024-03-28T13:09:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.508+0000 7fc085600640 1 Processor -- start 2024-03-28T13:09:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.509+0000 7fc085600640 1 -- start start 2024-03-28T13:09:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.509+0000 7feff4a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefe8056d50 con 0x7fefe8006160 2024-03-28T13:09:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.509+0000 7feff4a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fefe8057df0 con 0x7fefe80ebb10 2024-03-28T13:09:48.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.509+0000 7fefee800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fefe80eb020 0x7fefe80eb400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42908/0 (socket says 172.21.15.19:42908) 2024-03-28T13:09:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.509+0000 7fefee800640 1 -- 172.21.15.19:0/505335381 learned_addr learned my addr 172.21.15.19:0/505335381 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.510+0000 7f7919a00640 1 -- 172.21.15.19:0/3655669929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79140d03b0 msgr2=0x7f79140d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.510+0000 7fefede00640 1 --2- 172.21.15.19:0/505335381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefe80ebb10 0x7fefe80552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.509+0000 7f7919000640 1 --2- 172.21.15.19:0/3655669929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79140d03b0 0x7f79140d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.511+0000 7ffa29e00640 1 Processor -- start 2024-03-28T13:09:48.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.511+0000 7ffa29e00640 1 -- start start 2024-03-28T13:09:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.511+0000 7fefed400640 1 -- 172.21.15.19:0/505335381 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 458538240 0 0) 0x7feff0069760 con 0x7fefe80eb020 2024-03-28T13:09:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.511+0000 7fefed400640 1 -- 172.21.15.19:0/505335381 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fefd4003520 con 0x7fefe80eb020 2024-03-28T13:09:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.512+0000 7fefed400640 1 -- 172.21.15.19:0/505335381 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2179580241 0 0) 0x7fefe8056d50 con 0x7fefe8006160 2024-03-28T13:09:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.512+0000 7fefede00640 1 -- 172.21.15.19:0/505335381 >> v1:172.21.15.188:6789/0 conn(0x7fefe8006160 legacy=0x7fefe80065f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.512+0000 7fefede00640 1 -- 172.21.15.19:0/505335381 >> v1:172.21.15.19:6789/0 conn(0x7fefe80eb020 legacy=0x7fefe80eb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.511+0000 7f7919a00640 1 --2- 172.21.15.19:0/3655669929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79140d03b0 0x7f79140d0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.511+0000 7f7919a00640 1 -- 172.21.15.19:0/3655669929 >> v2:172.21.15.188:3300/0 conn(0x7f79140b3cf0 msgr2=0x7f7914071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.511+0000 7f7919a00640 1 --2- 172.21.15.19:0/3655669929 >> v2:172.21.15.188:3300/0 conn(0x7f79140b3cf0 0x7f7914071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.511+0000 7fc085600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0781051d0 0x7fc07804f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.511+0000 7fc085600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc078038320 0x7fc078054450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.511+0000 7fc085600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc078052fc0 con 0x7fc078054000 2024-03-28T13:09:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.511+0000 7fc085600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc07804f7c0 con 0x7fc0781051d0 2024-03-28T13:09:48.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.511+0000 7fc085600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc078053260 con 0x7fc078038320 2024-03-28T13:09:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.511+0000 7f7919a00640 1 -- 172.21.15.19:0/3655669929 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79140b3210 con 0x7f7914071e80 2024-03-28T13:09:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.512+0000 7f7913e00640 1 --2- 172.21.15.19:0/3655669929 >> v2:172.21.15.188:3300/0 conn(0x7f79140b3cf0 0x7f7914071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:09:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.513+0000 7f7919000640 1 --2- 172.21.15.19:0/3655669929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79140d03b0 0x7f79140d0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:48.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.513+0000 7fefede00640 1 -- 172.21.15.19:0/505335381 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe8057030 con 0x7fefe80ebb10 2024-03-28T13:09:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.513+0000 7fc07f400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc078038320 0x7fc078054450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.513+0000 7fc07ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0781051d0 0x7fc07804f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.513+0000 7fc07ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0781051d0 0x7fc07804f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50648/0 (socket says 172.21.15.19:50648) 2024-03-28T13:09:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.513+0000 7fc07ea00640 1 -- 172.21.15.19:0/772566859 learned_addr learned my addr 172.21.15.19:0/772566859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.513+0000 7f7919a00640 1 --2- 172.21.15.19:0/3655669929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7914071e80 0x7f79140ad010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f79140b6230 tx=0x7f790c02e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a7c4f6b04ab4e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.513+0000 7fc07ea00640 1 -- 172.21.15.19:0/772566859 >> v2:172.21.15.188:3300/0 conn(0x7fc078038320 msgr2=0x7fc078054450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.513+0000 7fc07ea00640 1 --2- 172.21.15.19:0/772566859 >> v2:172.21.15.188:3300/0 conn(0x7fc078038320 0x7fc078054450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.513+0000 7fc07ea00640 1 -- 172.21.15.19:0/772566859 >> v1:172.21.15.188:6789/0 conn(0x7fc078054000 legacy=0x7fc078037c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.513+0000 7ffa29e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffa240d3e20 0x7ffa240b9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.513+0000 7ffa29e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffa240cef00 0x7ffa240cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.514+0000 7fc07ea00640 1 -- 172.21.15.19:0/772566859 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc078053520 con 0x7fc0781051d0 2024-03-28T13:09:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.513+0000 7ffa29e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa240d2d80 con 0x7ffa2415e440 2024-03-28T13:09:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.513+0000 7ffa29e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ffa240cf8b0 con 0x7ffa240cef00 2024-03-28T13:09:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.513+0000 7ffa29e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ffa240b8300 con 0x7ffa240d3e20 2024-03-28T13:09:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.514+0000 7ffa1be00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffa240d3e20 0x7ffa240b9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.515+0000 7ffa1be00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffa240d3e20 0x7ffa240b9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:43986/0 (socket says 172.21.15.19:43986) 2024-03-28T13:09:48.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.515+0000 7ffa1be00640 1 -- 172.21.15.19:0/2922141718 learned_addr learned my addr 172.21.15.19:0/2922141718 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.516+0000 7ffa1be00640 1 -- 172.21.15.19:0/2922141718 >> v2:172.21.15.19:3300/0 conn(0x7ffa240cef00 msgr2=0x7ffa240cf370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T13:09:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.516+0000 7ffa1be00640 1 --2- 172.21.15.19:0/2922141718 >> v2:172.21.15.19:3300/0 conn(0x7ffa240cef00 0x7ffa240cf370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.516+0000 7ffa1be00640 1 -- 172.21.15.19:0/2922141718 >> v1:172.21.15.19:6789/0 conn(0x7ffa2415e440 legacy=0x7ffa240d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.517+0000 7ffa1be00640 1 -- 172.21.15.19:0/2922141718 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa240d3060 con 0x7ffa240d3e20 2024-03-28T13:09:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.515+0000 7fefede00640 1 --2- 172.21.15.19:0/505335381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefe80ebb10 0x7fefe80552c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fefe00097d0 tx=0x7fefe002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30cee132f90ae71b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.515+0000 7fefed400640 1 -- 172.21.15.19:0/505335381 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0035020 con 0x7fefe80ebb10 2024-03-28T13:09:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.516+0000 7fefed400640 1 -- 172.21.15.19:0/505335381 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe0031800 con 0x7fefe80ebb10 2024-03-28T13:09:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.516+0000 7fefed400640 1 -- 172.21.15.19:0/505335381 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe0031bd0 con 0x7fefe80ebb10 2024-03-28T13:09:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.517+0000 7feff4a00640 1 -- 172.21.15.19:0/505335381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefe80ebb10 msgr2=0x7fefe80552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.517+0000 7feff4a00640 1 --2- 172.21.15.19:0/505335381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefe80ebb10 0x7fefe80552c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fefe00097d0 tx=0x7fefe002b8a0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.517+0000 7feff4a00640 1 -- 172.21.15.19:0/505335381 shutdown_connections 2024-03-28T13:09:48.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.517+0000 7feff4a00640 1 --2- 172.21.15.19:0/505335381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefe80ebb10 0x7fefe80552c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.517+0000 7feff4a00640 1 -- 172.21.15.19:0/505335381 >> 172.21.15.19:0/505335381 conn(0x7fefe80559a0 msgr2=0x7fefe8055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.517+0000 7feff4a00640 1 -- 172.21.15.19:0/505335381 shutdown_connections 2024-03-28T13:09:48.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.517+0000 7feff4a00640 1 -- 172.21.15.19:0/505335381 wait complete. 2024-03-28T13:09:48.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.516+0000 7fc07ea00640 1 --2- 172.21.15.19:0/772566859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0781051d0 0x7fc07804f280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc07800b090 tx=0x7fc07002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986cb502b00f43c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.516+0000 7fc07d600640 1 -- 172.21.15.19:0/772566859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07002f850 con 0x7fc0781051d0 2024-03-28T13:09:48.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.516+0000 7fc07d600640 1 -- 172.21.15.19:0/772566859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc07002f9f0 con 0x7fc0781051d0 2024-03-28T13:09:48.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.516+0000 7fc07d600640 1 -- 172.21.15.19:0/772566859 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07002f850 con 0x7fc0781051d0 2024-03-28T13:09:48.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.518+0000 7fc085600640 1 -- 172.21.15.19:0/772566859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0781051d0 msgr2=0x7fc07804f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.514+0000 7f7913400640 1 -- 172.21.15.19:0/3655669929 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c03b070 con 0x7f7914071e80 2024-03-28T13:09:48.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.514+0000 7f7913400640 1 -- 172.21.15.19:0/3655669929 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f790c036640 con 0x7f7914071e80 2024-03-28T13:09:48.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.514+0000 7f7913400640 1 -- 172.21.15.19:0/3655669929 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c03b070 con 0x7f7914071e80 2024-03-28T13:09:48.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.518+0000 7f791ba00640 1 -- 172.21.15.19:0/3655669929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7914071e80 msgr2=0x7f79140ad010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.518+0000 7f791ba00640 1 --2- 172.21.15.19:0/3655669929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7914071e80 0x7f79140ad010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f79140b6230 tx=0x7f790c02e2f0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.519+0000 7f791ba00640 1 -- 172.21.15.19:0/3655669929 shutdown_connections 2024-03-28T13:09:48.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.519+0000 7f791ba00640 1 --2- 172.21.15.19:0/3655669929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7914071e80 0x7f79140ad010 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.518+0000 7fc085600640 1 --2- 172.21.15.19:0/772566859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0781051d0 0x7fc07804f280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc07800b090 tx=0x7fc07002ba30 comp rx=0 tx=0).stop 2024-03-28T13:09:48.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.519+0000 7f791ba00640 1 --2- 172.21.15.19:0/3655669929 >> v2:172.21.15.188:3300/0 conn(0x7f79140b3cf0 0x7f7914071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.519+0000 7f791ba00640 1 --2- 172.21.15.19:0/3655669929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79140d03b0 0x7f79140d0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.519+0000 7f791ba00640 1 -- 172.21.15.19:0/3655669929 >> 172.21.15.19:0/3655669929 conn(0x7f791406b750 msgr2=0x7f791406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.519+0000 7f791ba00640 1 -- 172.21.15.19:0/3655669929 shutdown_connections 2024-03-28T13:09:48.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.519+0000 7f791ba00640 1 -- 172.21.15.19:0/3655669929 wait complete. 2024-03-28T13:09:48.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.518+0000 7fc085600640 1 -- 172.21.15.19:0/772566859 shutdown_connections 2024-03-28T13:09:48.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.518+0000 7fc085600640 1 --2- 172.21.15.19:0/772566859 >> v2:172.21.15.188:3300/0 conn(0x7fc078038320 0x7fc078054450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.518+0000 7fc085600640 1 --2- 172.21.15.19:0/772566859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0781051d0 0x7fc07804f280 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc07800b090 tx=0x7fc07002ba30 comp rx=0 tx=0).stop 2024-03-28T13:09:48.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.518+0000 7fc085600640 1 -- 172.21.15.19:0/772566859 >> 172.21.15.19:0/772566859 conn(0x7fc078009fa0 msgr2=0x7fc07800a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.520+0000 7ffa1be00640 1 --2- 172.21.15.19:0/2922141718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa240d3e20 0x7ffa240b9e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa2406d630 tx=0x7ffa1402bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5760e15bd58d038 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.520+0000 7eff61c00640 1 Processor -- start 2024-03-28T13:09:48.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.520+0000 7eff61c00640 1 -- start start 2024-03-28T13:09:48.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.520+0000 7fc085600640 1 -- 172.21.15.19:0/772566859 shutdown_connections 2024-03-28T13:09:48.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.520+0000 7feff4a00640 1 Processor -- start 2024-03-28T13:09:48.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.522+0000 7f791ba00640 1 Processor -- start 2024-03-28T13:09:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.522+0000 7eff61c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7eff5c15e6d0 0x7eff5c0d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.521+0000 7feff4a00640 1 -- start start 2024-03-28T13:09:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.522+0000 7feff4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefe8006160 0x7fefe8185910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.522+0000 7feff4a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe80eb020 0x7fefe800cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.522+0000 7feff4a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefe80ebb10 0x7fefe800d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.522+0000 7feff4a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fefe8057960 con 0x7fefe8006160 2024-03-28T13:09:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.522+0000 7feff4a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fefe80575f0 con 0x7fefe80eb020 2024-03-28T13:09:48.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.522+0000 7feff4a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fefe80e1dd0 con 0x7fefe80ebb10 2024-03-28T13:09:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.520+0000 7fc085600640 1 -- 172.21.15.19:0/772566859 wait complete. 2024-03-28T13:09:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.522+0000 7eff61c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5c0ceef0 con 0x7eff5c0d4760 2024-03-28T13:09:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.523+0000 7eff61c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff5c15eab0 con 0x7eff5c0d0cb0 2024-03-28T13:09:48.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.523+0000 7eff61c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7eff5c0d1220 con 0x7eff5c15e6d0 2024-03-28T13:09:48.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.523+0000 7fefee800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefe8006160 0x7fefe8185910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.523+0000 7eff5b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7eff5c0d4760 0x7eff5c0cf3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42920/0 (socket says 172.21.15.19:42920) 2024-03-28T13:09:48.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.523+0000 7eff5b400640 1 -- 172.21.15.19:0/1827970140 learned_addr learned my addr 172.21.15.19:0/1827970140 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:48.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.523+0000 7eff5a000640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7eff5c0d0cb0 0x7eff5c0d4050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:42558/0 (socket says 172.21.15.19:42558) 2024-03-28T13:09:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.523+0000 7fefee800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefe8006160 0x7fefe8185910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50656/0 (socket says 172.21.15.19:50656) 2024-03-28T13:09:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.523+0000 7fefede00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe80eb020 0x7fefe800cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.523+0000 7f791ba00640 1 -- start start 2024-03-28T13:09:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.523+0000 7fefede00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe80eb020 0x7fefe800cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:43998/0 (socket says 172.21.15.19:43998) 2024-03-28T13:09:48.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.523+0000 7fefede00640 1 -- 172.21.15.19:0/1763087624 learned_addr learned my addr 172.21.15.19:0/1763087624 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.524+0000 7fc085600640 1 Processor -- start 2024-03-28T13:09:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.524+0000 7fefef200640 1 --2- 172.21.15.19:0/1763087624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefe80ebb10 0x7fefe800d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.524+0000 7eff5aa00640 1 --2- 172.21.15.19:0/1827970140 >> v2:172.21.15.19:3300/0 conn(0x7eff5c15e6d0 0x7eff5c0d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.524+0000 7eff59600640 1 -- 172.21.15.19:0/1827970140 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3506653500 0 0) 0x7eff5c15eab0 con 0x7eff5c0d0cb0 2024-03-28T13:09:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.524+0000 7eff59600640 1 -- 172.21.15.19:0/1827970140 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff40003520 con 0x7eff5c0d0cb0 2024-03-28T13:09:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.524+0000 7fc085600640 1 -- start start 2024-03-28T13:09:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.526+0000 7fc085600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc078038320 0x7fc0780fab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.526+0000 7fc085600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc078054000 0x7fc0780321e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.525+0000 7eff59600640 1 -- 172.21.15.19:0/1827970140 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2850573350 0 0) 0x7eff40003520 con 0x7eff5c0d0cb0 2024-03-28T13:09:48.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.525+0000 7eff59600640 1 -- 172.21.15.19:0/1827970140 >> v2:172.21.15.19:3300/0 conn(0x7eff5c15e6d0 msgr2=0x7eff5c0d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.525+0000 7eff59600640 1 --2- 172.21.15.19:0/1827970140 >> v2:172.21.15.19:3300/0 conn(0x7eff5c15e6d0 0x7eff5c0d21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.525+0000 7eff59600640 1 -- 172.21.15.19:0/1827970140 >> v1:172.21.15.19:6789/0 conn(0x7eff5c0d4760 legacy=0x7eff5c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.525+0000 7eff59600640 1 -- 172.21.15.19:0/1827970140 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c0d1500 con 0x7eff5c0d0cb0 2024-03-28T13:09:48.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.526+0000 7eff59600640 1 -- 172.21.15.19:0/1827970140 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7eff48004fd0 con 0x7eff5c0d0cb0 2024-03-28T13:09:48.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.526+0000 7eff59600640 1 -- 172.21.15.19:0/1827970140 >> v1:172.21.15.188:6789/0 conn(0x7eff5c0d0cb0 legacy=0x7eff5c0d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.525+0000 7fefee800640 1 -- 172.21.15.19:0/1763087624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefe80ebb10 msgr2=0x7fefe800d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.525+0000 7fefee800640 1 --2- 172.21.15.19:0/1763087624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefe80ebb10 0x7fefe800d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.525+0000 7fefee800640 1 -- 172.21.15.19:0/1763087624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe80eb020 msgr2=0x7fefe800cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.525+0000 7fefee800640 1 --2- 172.21.15.19:0/1763087624 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefe80eb020 0x7fefe800cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.526+0000 7eff59600640 1 --2- 172.21.15.19:0/1827970140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff400039b0 0x7eff40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.526+0000 7eff59600640 1 --2- 172.21.15.19:0/1827970140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff40004800 0x7eff40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.526+0000 7eff59600640 1 --2- 172.21.15.19:0/1827970140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff400051c0 0x7eff4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.526+0000 7eff59600640 1 -- 172.21.15.19:0/1827970140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c0d1220 con 0x7eff400051c0 2024-03-28T13:09:48.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.526+0000 7eff59600640 1 -- 172.21.15.19:0/1827970140 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7eff4000c060 con 0x7eff400039b0 2024-03-28T13:09:48.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.526+0000 7eff59600640 1 -- 172.21.15.19:0/1827970140 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7eff4000c260 con 0x7eff40004800 2024-03-28T13:09:48.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.526+0000 7fc085600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc078033090 0x7fc078107e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.526+0000 7fefee800640 1 -- 172.21.15.19:0/1763087624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefe804e5f0 con 0x7fefe8006160 2024-03-28T13:09:48.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.522+0000 7ffa23e00640 1 -- 172.21.15.19:0/2922141718 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1402cb30 con 0x7ffa240d3e20 2024-03-28T13:09:48.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.522+0000 7ffa23e00640 1 -- 172.21.15.19:0/2922141718 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1402ccd0 con 0x7ffa240d3e20 2024-03-28T13:09:48.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.522+0000 7ffa23e00640 1 -- 172.21.15.19:0/2922141718 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14034610 con 0x7ffa240d3e20 2024-03-28T13:09:48.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.523+0000 7ffa29e00640 1 -- 172.21.15.19:0/2922141718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa240d3e20 msgr2=0x7ffa240b9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.527+0000 7fc085600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc078033f50 con 0x7fc078038320 2024-03-28T13:09:48.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.527+0000 7fc085600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc078053ae0 con 0x7fc078033090 2024-03-28T13:09:48.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.523+0000 7ffa29e00640 1 --2- 172.21.15.19:0/2922141718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa240d3e20 0x7ffa240b9e00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa2406d630 tx=0x7ffa1402bf90 comp rx=0 tx=0).stop 2024-03-28T13:09:48.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.524+0000 7ffa29e00640 1 -- 172.21.15.19:0/2922141718 shutdown_connections 2024-03-28T13:09:48.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.524+0000 7ffa29e00640 1 --2- 172.21.15.19:0/2922141718 >> v2:172.21.15.19:3300/0 conn(0x7ffa240cef00 0x7ffa240cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.524+0000 7ffa29e00640 1 --2- 172.21.15.19:0/2922141718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa240d3e20 0x7ffa240b9e00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.524+0000 7ffa29e00640 1 -- 172.21.15.19:0/2922141718 >> 172.21.15.19:0/2922141718 conn(0x7ffa240678b0 msgr2=0x7ffa240d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.524+0000 7ffa29e00640 1 -- 172.21.15.19:0/2922141718 shutdown_connections 2024-03-28T13:09:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.524+0000 7ffa29e00640 1 -- 172.21.15.19:0/2922141718 wait complete. 2024-03-28T13:09:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.527+0000 7fc085600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc0781055b0 con 0x7fc078054000 2024-03-28T13:09:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.527+0000 7ffa29e00640 1 Processor -- start 2024-03-28T13:09:48.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.527+0000 7fc07f400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc078033090 0x7fc078107e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.527+0000 7fefee800640 1 --2- 172.21.15.19:0/1763087624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefe8006160 0x7fefe8185910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feff0000b60 tx=0x7feff006c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.527+0000 7fc07f400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc078033090 0x7fc078107e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44010/0 (socket says 172.21.15.19:44010) 2024-03-28T13:09:48.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.527+0000 7ffa29e00640 1 -- start start 2024-03-28T13:09:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.527+0000 7fc07f400640 1 -- 172.21.15.19:0/3708458472 learned_addr learned my addr 172.21.15.19:0/3708458472 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.527+0000 7fc07e000640 1 --2- 172.21.15.19:0/3708458472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc078054000 0x7fc0780321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.527+0000 7fefe6a00640 1 -- 172.21.15.19:0/1763087624 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff00733c0 con 0x7fefe8006160 2024-03-28T13:09:48.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.528+0000 7fc07ea00640 1 --2- 172.21.15.19:0/3708458472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc078038320 0x7fc0780fab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.528+0000 7ffa29e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa240cef00 0x7ffa240adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.528+0000 7fefe6a00640 1 -- 172.21.15.19:0/1763087624 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff0073560 con 0x7fefe8006160 2024-03-28T13:09:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.528+0000 7fefe6a00640 1 -- 172.21.15.19:0/1763087624 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0073970 con 0x7fefe8006160 2024-03-28T13:09:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.528+0000 7feff4a00640 1 -- 172.21.15.19:0/1763087624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefe804e8d0 con 0x7fefe8006160 2024-03-28T13:09:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.529+0000 7feff4a00640 1 -- 172.21.15.19:0/1763087624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefe804ee90 con 0x7fefe8006160 2024-03-28T13:09:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.528+0000 7fc07f400640 1 -- 172.21.15.19:0/3708458472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc078054000 msgr2=0x7fc0780321e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.528+0000 7fc07f400640 1 --2- 172.21.15.19:0/3708458472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc078054000 0x7fc0780321e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.528+0000 7fc07f400640 1 -- 172.21.15.19:0/3708458472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc078038320 msgr2=0x7fc0780fab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.528+0000 7f791ba00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7914071e80 0x7f7914148b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.528+0000 7f791ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79140b3cf0 0x7f791415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.528+0000 7eff5b400640 1 --2- 172.21.15.19:0/1827970140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff400051c0 0x7eff4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.528+0000 7eff5a000640 1 --2- 172.21.15.19:0/1827970140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff40004800 0x7eff40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.528+0000 7eff5aa00640 1 --2- 172.21.15.19:0/1827970140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff400039b0 0x7eff40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.528+0000 7ffa29e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa240d3e20 0x7ffa240b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.529+0000 7ffa29e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa2415e440 0x7ffa24208690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.529+0000 7ffa29e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa240d3900 con 0x7ffa240cef00 2024-03-28T13:09:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.529+0000 7ffa29e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffa240d3620 con 0x7ffa240d3e20 2024-03-28T13:09:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.529+0000 7ffa29e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffa240b72f0 con 0x7ffa2415e440 2024-03-28T13:09:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.529+0000 7fc07f400640 1 --2- 172.21.15.19:0/3708458472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc078038320 0x7fc0780fab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.529+0000 7fc07f400640 1 -- 172.21.15.19:0/3708458472 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc078031140 con 0x7fc078033090 2024-03-28T13:09:48.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.529+0000 7f791ba00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79140d03b0 0x7f7914137a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7ffa28a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa240cef00 0x7ffa240adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7f7913e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79140b3cf0 0x7f791415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7ffa29400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa2415e440 0x7ffa24208690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7fc07ea00640 1 --2- 172.21.15.19:0/3708458472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc078038320 0x7fc0780fab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:48.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.529+0000 7f7919000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7914071e80 0x7f7914148b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7f7919000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7914071e80 0x7f7914148b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:54728/0 (socket says 172.21.15.19:54728) 2024-03-28T13:09:48.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7ffa29400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa2415e440 0x7ffa24208690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:54736/0 (socket says 172.21.15.19:54736) 2024-03-28T13:09:48.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7ffa29400640 1 -- 172.21.15.19:0/658239471 learned_addr learned my addr 172.21.15.19:0/658239471 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7ffa1be00640 1 --2- 172.21.15.19:0/658239471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa240d3e20 0x7ffa240b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7f7913e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79140b3cf0 0x7f791415d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50668/0 (socket says 172.21.15.19:50668) 2024-03-28T13:09:48.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7eff5a000640 1 -- 172.21.15.19:0/1827970140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff400039b0 msgr2=0x7eff40003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7eff5a000640 1 --2- 172.21.15.19:0/1827970140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff400039b0 0x7eff40003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7eff5a000640 1 -- 172.21.15.19:0/1827970140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff400051c0 msgr2=0x7eff4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7eff5a000640 1 --2- 172.21.15.19:0/1827970140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff400051c0 0x7eff4000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7eff5a000640 1 -- 172.21.15.19:0/1827970140 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c0d1500 con 0x7eff40004800 2024-03-28T13:09:48.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7f791ba00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f791415a800 con 0x7f79140b3cf0 2024-03-28T13:09:48.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7f791ba00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f791415ef90 con 0x7f79140d03b0 2024-03-28T13:09:48.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.530+0000 7f791ba00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f791415a980 con 0x7f7914071e80 2024-03-28T13:09:48.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.531+0000 7ffa29400640 1 -- 172.21.15.19:0/658239471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa240d3e20 msgr2=0x7ffa240b7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.531+0000 7ffa29400640 1 --2- 172.21.15.19:0/658239471 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa240d3e20 0x7ffa240b7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.531+0000 7ffa29400640 1 -- 172.21.15.19:0/658239471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa240cef00 msgr2=0x7ffa240adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.531+0000 7f7919000640 1 -- 172.21.15.19:0/4262080779 learned_addr learned my addr 172.21.15.19:0/4262080779 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.531+0000 7ffa29400640 1 --2- 172.21.15.19:0/658239471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa240cef00 0x7ffa240adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.531+0000 7f7919a00640 1 --2- 172.21.15.19:0/4262080779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79140d03b0 0x7f7914137a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.531+0000 7eff5b400640 1 -- 172.21.15.19:0/1827970140 reap_dead start 2024-03-28T13:09:48.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.531+0000 7ffa29400640 1 -- 172.21.15.19:0/658239471 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa242060b0 con 0x7ffa2415e440 2024-03-28T13:09:48.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.531+0000 7feff4a00640 1 -- 172.21.15.19:0/1763087624 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fefe80057a0 con 0x7fefe8006160 2024-03-28T13:09:48.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.531+0000 7ffa29400640 1 --2- 172.21.15.19:0/658239471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa2415e440 0x7ffa24208690 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00b3f0 tx=0x7ffa1c00b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.531+0000 7f7919000640 1 -- 172.21.15.19:0/4262080779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79140d03b0 msgr2=0x7f7914137a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.531+0000 7f7919000640 1 --2- 172.21.15.19:0/4262080779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79140d03b0 0x7f7914137a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.531+0000 7f7919000640 1 -- 172.21.15.19:0/4262080779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79140b3cf0 msgr2=0x7f791415d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.531+0000 7f7919000640 1 --2- 172.21.15.19:0/4262080779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f79140b3cf0 0x7f791415d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.531+0000 7f7919000640 1 -- 172.21.15.19:0/4262080779 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7914151740 con 0x7f7914071e80 2024-03-28T13:09:48.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.532+0000 7ffa21600640 1 -- 172.21.15.19:0/658239471 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c015020 con 0x7ffa2415e440 2024-03-28T13:09:48.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.532+0000 7f7919000640 1 --2- 172.21.15.19:0/4262080779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7914071e80 0x7f7914148b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f790800c820 tx=0x7f790800cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.532+0000 7ffa29e00640 1 -- 172.21.15.19:0/658239471 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa24205810 con 0x7ffa2415e440 2024-03-28T13:09:48.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.532+0000 7ffa29e00640 1 -- 172.21.15.19:0/658239471 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa24206350 con 0x7ffa2415e440 2024-03-28T13:09:48.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.532+0000 7eff5a000640 1 --2- 172.21.15.19:0/1827970140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff40004800 0x7eff40004c80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7eff4800c160 tx=0x7eff40005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd0d333c670b528 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.532+0000 7ffa21600640 1 -- 172.21.15.19:0/658239471 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa1c011830 con 0x7ffa2415e440 2024-03-28T13:09:48.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.532+0000 7ffa21600640 1 -- 172.21.15.19:0/658239471 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c015020 con 0x7ffa2415e440 2024-03-28T13:09:48.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.533+0000 7f7910c00640 1 -- 172.21.15.19:0/4262080779 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908015070 con 0x7f7914071e80 2024-03-28T13:09:48.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.533+0000 7f7910c00640 1 -- 172.21.15.19:0/4262080779 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7908007a30 con 0x7f7914071e80 2024-03-28T13:09:48.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.533+0000 7f7910c00640 1 -- 172.21.15.19:0/4262080779 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908007e00 con 0x7f7914071e80 2024-03-28T13:09:48.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.533+0000 7f791ba00640 1 -- 172.21.15.19:0/4262080779 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791415db20 con 0x7f7914071e80 2024-03-28T13:09:48.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.533+0000 7f791ba00640 1 -- 172.21.15.19:0/4262080779 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7914150c60 con 0x7f7914071e80 2024-03-28T13:09:48.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.533+0000 7eff59600640 1 -- 172.21.15.19:0/1827970140 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48007c30 con 0x7eff40004800 2024-03-28T13:09:48.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.533+0000 7eff59600640 1 -- 172.21.15.19:0/1827970140 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff48030070 con 0x7eff40004800 2024-03-28T13:09:48.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.533+0000 7eff59600640 1 -- 172.21.15.19:0/1827970140 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff480074d0 con 0x7eff40004800 2024-03-28T13:09:48.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.534+0000 7eff61c00640 1 -- 172.21.15.19:0/1827970140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff40004800 msgr2=0x7eff40004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.534+0000 7eff61c00640 1 --2- 172.21.15.19:0/1827970140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff40004800 0x7eff40004c80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7eff4800c160 tx=0x7eff40005830 comp rx=0 tx=0).stop 2024-03-28T13:09:48.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.534+0000 7eff61c00640 1 -- 172.21.15.19:0/1827970140 shutdown_connections 2024-03-28T13:09:48.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.534+0000 7eff61c00640 1 --2- 172.21.15.19:0/1827970140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff40004800 0x7eff40004c80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.534+0000 7eff61c00640 1 -- 172.21.15.19:0/1827970140 >> 172.21.15.19:0/1827970140 conn(0x7eff5c09d7b0 msgr2=0x7eff5c08bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.535+0000 7eff61c00640 1 -- 172.21.15.19:0/1827970140 shutdown_connections 2024-03-28T13:09:48.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.535+0000 7eff61c00640 1 -- 172.21.15.19:0/1827970140 wait complete. 2024-03-28T13:09:48.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.536+0000 7ffa1a000640 1 -- 172.21.15.19:0/658239471 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa24075940 con 0x7ffa2415e440 2024-03-28T13:09:48.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.534+0000 7fefe6a00640 1 -- 172.21.15.19:0/1763087624 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7feff0093400 con 0x7fefe8006160 2024-03-28T13:09:48.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.536+0000 7fefe6a00640 1 --2- 172.21.15.19:0/1763087624 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fefd006c580 0x7fefd006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.536+0000 7fefe6a00640 1 -- 172.21.15.19:0/1763087624 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7feff0106b30 con 0x7fefe8006160 2024-03-28T13:09:48.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.537+0000 7fefe6a00640 1 --2- 172.21.15.19:0/1763087624 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x7fefd00747a0 0x7fefd0076bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.537+0000 7fefe6a00640 1 -- 172.21.15.19:0/1763087624 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fefd0077270 con 0x7fefd00747a0 2024-03-28T13:09:48.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.537+0000 7fefe6a00640 1 -- 172.21.15.19:0/1763087624 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7feff0107010 con 0x7fefe8006160 2024-03-28T13:09:48.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.536+0000 7fc07f400640 1 --2- 172.21.15.19:0/3708458472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc078033090 0x7fc078107e30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc08002d900 tx=0x7fc08004ffb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.536+0000 7fc076a00640 1 -- 172.21.15.19:0/3708458472 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080088070 con 0x7fc078033090 2024-03-28T13:09:48.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.536+0000 7fc076a00640 1 -- 172.21.15.19:0/3708458472 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0800733c0 con 0x7fc078033090 2024-03-28T13:09:48.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.537+0000 7fc076a00640 1 -- 172.21.15.19:0/3708458472 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0800737b0 con 0x7fc078033090 2024-03-28T13:09:48.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.537+0000 7fc085600640 1 -- 172.21.15.19:0/3708458472 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc078031420 con 0x7fc078033090 2024-03-28T13:09:48.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.537+0000 7fc085600640 1 -- 172.21.15.19:0/3708458472 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0780319e0 con 0x7fc078033090 2024-03-28T13:09:48.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.538+0000 7eff61c00640 1 Processor -- start 2024-03-28T13:09:48.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.538+0000 7f52ee000640 1 Processor -- start 2024-03-28T13:09:48.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.538+0000 7eff61c00640 1 -- start start 2024-03-28T13:09:48.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.539+0000 7fefef200640 1 --2- 172.21.15.19:0/1763087624 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x7fefd00747a0 0x7fefd0076bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.539+0000 7fefede00640 1 --2- 172.21.15.19:0/1763087624 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fefd006c580 0x7fefd006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.538+0000 7fc085600640 1 -- 172.21.15.19:0/3708458472 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc078005380 con 0x7fc078033090 2024-03-28T13:09:48.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.538+0000 7ffa21600640 1 -- 172.21.15.19:0/658239471 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ffa1c018020 con 0x7ffa2415e440 2024-03-28T13:09:48.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.539+0000 7ffa21600640 1 --2- 172.21.15.19:0/658239471 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffa0406c730 0x7ffa0406ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7ffa21600640 1 -- 172.21.15.19:0/658239471 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ffa1c056eb0 con 0x7ffa2415e440 2024-03-28T13:09:48.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7ffa21600640 1 --2- 172.21.15.19:0/658239471 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x7ffa040748d0 0x7ffa04076cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7ffa21600640 1 -- 172.21.15.19:0/658239471 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa1c01be50 con 0x7ffa040748d0 2024-03-28T13:09:48.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7ffa21600640 1 -- 172.21.15.19:0/658239471 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ffa1c058be0 con 0x7ffa2415e440 2024-03-28T13:09:48.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.541+0000 7ffa1be00640 1 --2- 172.21.15.19:0/658239471 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x7ffa040748d0 0x7ffa04076cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7eff61c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff40004800 0x7eff5c200a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7eff61c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff5c0b69d0 0x7eff5c06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7eff61c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff5c161000 0x7eff5c06d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7eff61c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c06c890 con 0x7eff5c0b69d0 2024-03-28T13:09:48.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7eff61c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7eff5c0d1ac0 con 0x7eff5c161000 2024-03-28T13:09:48.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7eff61c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7eff5c06ca10 con 0x7eff40004800 2024-03-28T13:09:48.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7eff5a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff5c0b69d0 0x7eff5c06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7eff5a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff5c0b69d0 0x7eff5c06cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50682/0 (socket says 172.21.15.19:50682) 2024-03-28T13:09:48.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7eff5a000640 1 -- 172.21.15.19:0/185519724 learned_addr learned my addr 172.21.15.19:0/185519724 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7eff5aa00640 1 --2- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff40004800 0x7eff5c200a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.541+0000 7eff5b400640 1 --2- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff5c161000 0x7eff5c06d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.541+0000 7eff5a000640 1 -- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff40004800 msgr2=0x7eff5c200a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.541+0000 7eff5a000640 1 --2- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff40004800 0x7eff5c200a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.541+0000 7eff5a000640 1 -- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff5c161000 msgr2=0x7eff5c06d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.541+0000 7eff5a000640 1 --2- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff5c161000 0x7eff5c06d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.541+0000 7eff5a000640 1 -- 172.21.15.19:0/185519724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff5c0cdb90 con 0x7eff5c0b69d0 2024-03-28T13:09:48.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.541+0000 7fefef200640 1 --2- 172.21.15.19:0/1763087624 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x7fefd00747a0 0x7fefd0076bc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.542+0000 7eff5a000640 1 --2- 172.21.15.19:0/185519724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff5c0b69d0 0x7eff5c06cbf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff4802c4a0 tx=0x7eff4802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.542+0000 7fefede00640 1 --2- 172.21.15.19:0/1763087624 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fefd006c580 0x7fefd006ea40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fefe0000c00 tx=0x7fefe003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.542+0000 7eff52a00640 1 -- 172.21.15.19:0/185519724 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48007c30 con 0x7eff5c0b69d0 2024-03-28T13:09:48.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.542+0000 7eff52a00640 1 -- 172.21.15.19:0/185519724 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff4802cc60 con 0x7eff5c0b69d0 2024-03-28T13:09:48.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.542+0000 7eff52a00640 1 -- 172.21.15.19:0/185519724 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff48005a30 con 0x7eff5c0b69d0 2024-03-28T13:09:48.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.542+0000 7eff61c00640 1 -- 172.21.15.19:0/185519724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff5c0cde70 con 0x7eff5c0b69d0 2024-03-28T13:09:48.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.542+0000 7eff61c00640 1 -- 172.21.15.19:0/185519724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff5c161710 con 0x7eff5c0b69d0 2024-03-28T13:09:48.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.542+0000 7ffa28a00640 1 --2- 172.21.15.19:0/658239471 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffa0406c730 0x7ffa0406ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.544+0000 7fefe6a00640 1 -- 172.21.15.19:0/1763087624 <== osd.1 v2:172.21.15.19:6801/1998544260 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fefd0077270 con 0x7fefd00747a0 2024-03-28T13:09:48.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.544+0000 7eff61c00640 1 -- 172.21.15.19:0/185519724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7eff34001030 con 0x7eff5c0b69d0 2024-03-28T13:09:48.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.538+0000 7f791ba00640 1 -- 172.21.15.19:0/4262080779 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f79140685c0 con 0x7f7914071e80 2024-03-28T13:09:48.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.544+0000 7f7910c00640 1 -- 172.21.15.19:0/4262080779 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7908018070 con 0x7f7914071e80 2024-03-28T13:09:48.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.538+0000 7f52ee000640 1 -- start start 2024-03-28T13:09:48.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7f52ee000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f52e815de50 0x7f52e80d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7f52ee000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e80b3b70 0x7f52e815c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7f52ee000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e815b240 con 0x7f52e80b1530 2024-03-28T13:09:48.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7f52ee000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f52e815e230 con 0x7f52e815de50 2024-03-28T13:09:48.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.540+0000 7f52ee000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f52e815b520 con 0x7f52e80b3b70 2024-03-28T13:09:48.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.541+0000 7f52e7400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f52e815de50 0x7f52e80d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.541+0000 7f52e7400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f52e815de50 0x7f52e80d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44058/0 (socket says 172.21.15.19:44058) 2024-03-28T13:09:48.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.541+0000 7f52e7400640 1 -- 172.21.15.19:0/1515798796 learned_addr learned my addr 172.21.15.19:0/1515798796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.542+0000 7f52e7e00640 1 --2- 172.21.15.19:0/1515798796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e80b3b70 0x7f52e815c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.542+0000 7f52e7400640 1 -- 172.21.15.19:0/1515798796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e80b3b70 msgr2=0x7f52e815c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.542+0000 7f52e7400640 1 --2- 172.21.15.19:0/1515798796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e80b3b70 0x7f52e815c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.542+0000 7f52e7400640 1 -- 172.21.15.19:0/1515798796 >> v1:172.21.15.188:6789/0 conn(0x7f52e80b1530 legacy=0x7f52e80b3460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-28T13:09:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.542+0000 7f52e7400640 1 -- 172.21.15.19:0/1515798796 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e815b800 con 0x7f52e815de50 2024-03-28T13:09:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.544+0000 7ffa1be00640 1 --2- 172.21.15.19:0/658239471 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x7ffa040748d0 0x7ffa04076cf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.545+0000 7ffa28a00640 1 --2- 172.21.15.19:0/658239471 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffa0406c730 0x7ffa0406ebf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffa240d2910 tx=0x7ffa10009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.544+0000 7fc076a00640 1 -- 172.21.15.19:0/3708458472 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc08008b070 con 0x7fc078033090 2024-03-28T13:09:48.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.545+0000 7fc076a00640 1 --2- 172.21.15.19:0/3708458472 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc06006c600 0x7fc06006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.546+0000 7ffa21600640 1 -- 172.21.15.19:0/658239471 <== osd.0 v2:172.21.15.19:6800/4106483280 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa1c01be50 con 0x7ffa040748d0 2024-03-28T13:09:48.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.545+0000 7fc076a00640 1 -- 172.21.15.19:0/3708458472 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc0800f54c0 con 0x7fc078033090 2024-03-28T13:09:48.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.546+0000 7fc07ea00640 1 --2- 172.21.15.19:0/3708458472 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc06006c600 0x7fc06006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.546+0000 7f7910c00640 1 --2- 172.21.15.19:0/4262080779 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f78f406c600 0x7f78f406eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.547+0000 7f52e7400640 1 --2- 172.21.15.19:0/1515798796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52e815de50 0x7f52e80d0990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009870 tx=0x7f52d402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0bb321d60c45780 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.547+0000 7eff52a00640 1 -- 172.21.15.19:0/185519724 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7eff48048020 con 0x7eff5c0b69d0 2024-03-28T13:09:48.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.547+0000 7f52e6000640 1 -- 172.21.15.19:0/1515798796 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d4035020 con 0x7f52e815de50 2024-03-28T13:09:48.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.547+0000 7f52e6000640 1 -- 172.21.15.19:0/1515798796 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d4031640 con 0x7f52e815de50 2024-03-28T13:09:48.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.548+0000 7f52e6000640 1 -- 172.21.15.19:0/1515798796 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d402d040 con 0x7f52e815de50 2024-03-28T13:09:48.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.548+0000 7f7910c00640 1 -- 172.21.15.19:0/4262080779 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f790808b670 con 0x7f7914071e80 2024-03-28T13:09:48.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.548+0000 7eff52a00640 1 --2- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7eff3c06c630 0x7eff3c06eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.548+0000 7f7910c00640 1 --2- 172.21.15.19:0/4262080779 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x7f78f4074820 0x7f78f4076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.549+0000 7f7913e00640 1 --2- 172.21.15.19:0/4262080779 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f78f406c600 0x7f78f406eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.549+0000 7f52ee000640 1 -- 172.21.15.19:0/1515798796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52e815de50 msgr2=0x7f52e80d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.549+0000 7eff52a00640 1 -- 172.21.15.19:0/185519724 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7eff480acbc0 con 0x7eff5c0b69d0 2024-03-28T13:09:48.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.549+0000 7f52ee000640 1 --2- 172.21.15.19:0/1515798796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52e815de50 0x7f52e80d0990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f52d4009870 tx=0x7f52d402b8f0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.549+0000 7fc076a00640 1 --2- 172.21.15.19:0/3708458472 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x7fc060074880 0x7fc060076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.549+0000 7fc076a00640 1 -- 172.21.15.19:0/3708458472 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc0800753e0 con 0x7fc060074880 2024-03-28T13:09:48.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.549+0000 7fc076a00640 1 -- 172.21.15.19:0/3708458472 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc0800f58b0 con 0x7fc078033090 2024-03-28T13:09:48.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.549+0000 7eff52a00640 1 --2- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x7eff3c074850 0x7eff3c076c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.549+0000 7fc07ea00640 1 --2- 172.21.15.19:0/3708458472 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc06006c600 0x7fc06006eac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc070036000 tx=0x7fc070008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.549+0000 7eff52a00640 1 -- 172.21.15.19:0/185519724 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff3c077320 con 0x7eff3c074850 2024-03-28T13:09:48.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.549+0000 7f52ee000640 1 -- 172.21.15.19:0/1515798796 shutdown_connections 2024-03-28T13:09:48.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.549+0000 7f7910c00640 1 -- 172.21.15.19:0/4262080779 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7908006810 con 0x7f78f4074820 2024-03-28T13:09:48.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.550+0000 7f52ee000640 1 --2- 172.21.15.19:0/1515798796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e80b3b70 0x7f52e815c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.550+0000 7f52ee000640 1 --2- 172.21.15.19:0/1515798796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52e815de50 0x7f52e80d0990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.550+0000 7f7910c00640 1 -- 172.21.15.19:0/4262080779 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f7908092050 con 0x7f7914071e80 2024-03-28T13:09:48.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.549+0000 7eff5aa00640 1 --2- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7eff3c06c630 0x7eff3c06eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.550+0000 7f52ee000640 1 -- 172.21.15.19:0/1515798796 >> 172.21.15.19:0/1515798796 conn(0x7f52e8072a00 msgr2=0x7f52e8072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.550+0000 7f7913e00640 1 --2- 172.21.15.19:0/4262080779 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f78f406c600 0x7f78f406eac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7900009630 tx=0x7f7900008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.550+0000 7eff5b400640 1 --2- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x7eff3c074850 0x7eff3c076c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.550+0000 7f52ee000640 1 -- 172.21.15.19:0/1515798796 shutdown_connections 2024-03-28T13:09:48.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.550+0000 7f7919a00640 1 --2- 172.21.15.19:0/4262080779 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x7f78f4074820 0x7f78f4076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.550+0000 7eff52a00640 1 -- 172.21.15.19:0/185519724 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7eff480b3050 con 0x7eff5c0b69d0 2024-03-28T13:09:48.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.551+0000 7f7919a00640 1 --2- 172.21.15.19:0/4262080779 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x7f78f4074820 0x7f78f4076c40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.551+0000 7eff5aa00640 1 --2- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7eff3c06c630 0x7eff3c06eaf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7eff5c06a480 tx=0x7eff4c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.552+0000 7fc07e000640 1 --2- 172.21.15.19:0/3708458472 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x7fc060074880 0x7fc060076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.552+0000 7eff5b400640 1 --2- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x7eff3c074850 0x7eff3c076c70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.553+0000 7f52ee000640 1 -- 172.21.15.19:0/1515798796 wait complete. 2024-03-28T13:09:48.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.553+0000 7fc07e000640 1 --2- 172.21.15.19:0/3708458472 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x7fc060074880 0x7fc060076ca0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.554+0000 7f7910c00640 1 -- 172.21.15.19:0/4262080779 <== osd.3 v2:172.21.15.19:6819/3265515865 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7908006810 con 0x7f78f4074820 2024-03-28T13:09:48.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.555+0000 7eff52a00640 1 -- 172.21.15.19:0/185519724 <== osd.5 v2:172.21.15.188:6800/3324247755 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7eff3c077320 con 0x7eff3c074850 2024-03-28T13:09:48.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.555+0000 7f52ee000640 1 Processor -- start 2024-03-28T13:09:48.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.555+0000 7fc076a00640 1 -- 172.21.15.19:0/3708458472 <== osd.2 v2:172.21.15.19:6816/1693606238 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc0800753e0 con 0x7fc060074880 2024-03-28T13:09:48.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.558+0000 7f52ee000640 1 -- start start 2024-03-28T13:09:48.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.559+0000 7f52ee000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e80b1530 0x7f52e8151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.559+0000 7f52ee000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52e80b3b70 0x7f52e80adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.559+0000 7f52ee000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52e815de50 0x7f52e809cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.560+0000 7f52ee000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52e815ae30 con 0x7f52e80b3b70 2024-03-28T13:09:48.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.560+0000 7f52ee000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80b4150 con 0x7f52e815de50 2024-03-28T13:09:48.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.560+0000 7f52ee000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f52e815afb0 con 0x7f52e80b1530 2024-03-28T13:09:48.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.560+0000 7f52e7400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e80b1530 0x7f52e8151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.560+0000 7f52e7400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e80b1530 0x7f52e8151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:54762/0 (socket says 172.21.15.19:54762) 2024-03-28T13:09:48.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.560+0000 7f52e7400640 1 -- 172.21.15.19:0/2363931768 learned_addr learned my addr 172.21.15.19:0/2363931768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.561+0000 7f52e7e00640 1 --2- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52e815de50 0x7f52e809cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.561+0000 7f52e7400640 1 -- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52e815de50 msgr2=0x7f52e809cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.561+0000 7f52e7400640 1 --2- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52e815de50 0x7f52e809cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.561+0000 7f52e7400640 1 -- 172.21.15.19:0/2363931768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52e80b3b70 msgr2=0x7f52e80adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.561+0000 7f52e7400640 1 --2- 172.21.15.19:0/2363931768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52e80b3b70 0x7f52e80adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.562+0000 7f52e7400640 1 -- 172.21.15.19:0/2363931768 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e80ae710 con 0x7f52e80b1530 2024-03-28T13:09:48.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.566+0000 7f52e7400640 1 --2- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e80b1530 0x7f52e8151480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f52d402b930 tx=0x7f52d40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.566+0000 7f52df400640 1 -- 172.21.15.19:0/2363931768 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d403aa60 con 0x7f52e80b1530 2024-03-28T13:09:48.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.566+0000 7f52df400640 1 -- 172.21.15.19:0/2363931768 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d403ac00 con 0x7f52e80b1530 2024-03-28T13:09:48.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.566+0000 7f52df400640 1 -- 172.21.15.19:0/2363931768 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d40443f0 con 0x7f52e80b1530 2024-03-28T13:09:48.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.567+0000 7f52ee000640 1 -- 172.21.15.19:0/2363931768 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e80ae9f0 con 0x7f52e80b1530 2024-03-28T13:09:48.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.567+0000 7f52ee000640 1 -- 172.21.15.19:0/2363931768 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e809a730 con 0x7f52e80b1530 2024-03-28T13:09:48.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.568+0000 7f52dcc00640 1 -- 172.21.15.19:0/2363931768 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f52bc001030 con 0x7f52e80b1530 2024-03-28T13:09:48.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.571+0000 7f52df400640 1 -- 172.21.15.19:0/2363931768 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f52d404b060 con 0x7f52e80b1530 2024-03-28T13:09:48.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.572+0000 7f52df400640 1 --2- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52c806c600 0x7f52c806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.572+0000 7f52df400640 1 -- 172.21.15.19:0/2363931768 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f52d40ad940 con 0x7f52e80b1530 2024-03-28T13:09:48.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.573+0000 7f52df400640 1 --2- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x7f52c8074820 0x7f52c8076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.573+0000 7f52df400640 1 -- 172.21.15.19:0/2363931768 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52d4032e50 con 0x7f52c8074820 2024-03-28T13:09:48.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.573+0000 7f52df400640 1 -- 172.21.15.19:0/2363931768 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f52d40add30 con 0x7f52e80b1530 2024-03-28T13:09:48.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.573+0000 7f52e6a00640 1 --2- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52c806c600 0x7f52c806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.574+0000 7f52e7e00640 1 --2- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x7f52c8074820 0x7f52c8076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.576+0000 7f52e7e00640 1 --2- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x7f52c8074820 0x7f52c8076c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.576+0000 7f52e6a00640 1 --2- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52c806c600 0x7f52c806eac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f52e81fbed0 tx=0x7f52d8006c40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.578+0000 7feff4a00640 1 -- 172.21.15.19:0/1763087624 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fefe804c7a0 con 0x7fefd00747a0 2024-03-28T13:09:48.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.578+0000 7ffa1a000640 1 -- 172.21.15.19:0/658239471 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa24206550 con 0x7ffa040748d0 2024-03-28T13:09:48.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.579+0000 7fefe6a00640 1 -- 172.21.15.19:0/1763087624 <== osd.1 v2:172.21.15.19:6801/1998544260 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fefe804c7a0 con 0x7fefd00747a0 2024-03-28T13:09:48.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.579+0000 7ffa21600640 1 -- 172.21.15.19:0/658239471 <== osd.0 v2:172.21.15.19:6800/4106483280 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffa24206550 con 0x7ffa040748d0 2024-03-28T13:09:48.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.579+0000 7fefdbe00640 1 -- 172.21.15.19:0/1763087624 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x7fefd00747a0 msgr2=0x7fefd0076bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.579+0000 7fefdbe00640 1 --2- 172.21.15.19:0/1763087624 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x7fefd00747a0 0x7fefd0076bc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.579+0000 7f52df400640 1 -- 172.21.15.19:0/2363931768 <== osd.7 v2:172.21.15.188:6824/861796513 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52d4032e50 con 0x7f52c8074820 2024-03-28T13:09:48.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.580+0000 7ffa1a000640 1 -- 172.21.15.19:0/658239471 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x7ffa040748d0 msgr2=0x7ffa04076cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.580+0000 7ffa1a000640 1 --2- 172.21.15.19:0/658239471 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x7ffa040748d0 0x7ffa04076cf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.580+0000 7fefdbe00640 1 -- 172.21.15.19:0/1763087624 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fefd006c580 msgr2=0x7fefd006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.580+0000 7fefdbe00640 1 --2- 172.21.15.19:0/1763087624 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fefd006c580 0x7fefd006ea40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fefe0000c00 tx=0x7fefe003c040 comp rx=0 tx=0).stop 2024-03-28T13:09:48.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.580+0000 7fefdbe00640 1 -- 172.21.15.19:0/1763087624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefe8006160 msgr2=0x7fefe8185910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.580+0000 7fefdbe00640 1 --2- 172.21.15.19:0/1763087624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefe8006160 0x7fefe8185910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feff0000b60 tx=0x7feff006c950 comp rx=0 tx=0).stop 2024-03-28T13:09:48.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.580+0000 7fefef200640 1 -- 172.21.15.19:0/1763087624 reap_dead start 2024-03-28T13:09:48.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.580+0000 7ffa1a000640 1 -- 172.21.15.19:0/658239471 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffa0406c730 msgr2=0x7ffa0406ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.580+0000 7faaa2800640 1 Processor -- start 2024-03-28T13:09:48.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.580+0000 7ffa1a000640 1 --2- 172.21.15.19:0/658239471 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffa0406c730 0x7ffa0406ebf0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffa240d2910 tx=0x7ffa10009040 comp rx=0 tx=0).stop 2024-03-28T13:09:48.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.581+0000 7ffa1a000640 1 -- 172.21.15.19:0/658239471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa2415e440 msgr2=0x7ffa24208690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.581+0000 7faaa2800640 1 -- start start 2024-03-28T13:09:48.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.581+0000 7ffa1a000640 1 --2- 172.21.15.19:0/658239471 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa2415e440 0x7ffa24208690 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c00b3f0 tx=0x7ffa1c00b8b0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.581+0000 7ffa29400640 1 -- 172.21.15.19:0/658239471 reap_dead start 2024-03-28T13:09:48.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.581+0000 7ffa1a000640 1 -- 172.21.15.19:0/658239471 shutdown_connections 2024-03-28T13:09:48.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.581+0000 7ffa1a000640 1 -- 172.21.15.19:0/658239471 >> 172.21.15.19:0/658239471 conn(0x7ffa240678b0 msgr2=0x7ffa240d4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.581+0000 7ffa1a000640 1 -- 172.21.15.19:0/658239471 shutdown_connections 2024-03-28T13:09:48.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.581+0000 7ffa1a000640 1 -- 172.21.15.19:0/658239471 wait complete. 2024-03-28T13:09:48.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.582+0000 7fefdbe00640 1 -- 172.21.15.19:0/1763087624 shutdown_connections 2024-03-28T13:09:48.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.582+0000 7fefdbe00640 1 -- 172.21.15.19:0/1763087624 >> 172.21.15.19:0/1763087624 conn(0x7fefe80559a0 msgr2=0x7fefe8051f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.582+0000 7faaa2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faa9c15e860 0x7faa9c06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.583+0000 7faaa2800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faa9c0d4ae0 0x7faa9c0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.583+0000 7faaa2800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa9c06dfa0 con 0x7faa9c06da30 2024-03-28T13:09:48.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.583+0000 7faaa2800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7faa9c0cf220 con 0x7faa9c15e860 2024-03-28T13:09:48.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.583+0000 7faaa2800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7faa9c06e240 con 0x7faa9c0d4ae0 2024-03-28T13:09:48.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.583+0000 7fefdbe00640 1 -- 172.21.15.19:0/1763087624 shutdown_connections 2024-03-28T13:09:48.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.583+0000 7fefdbe00640 1 -- 172.21.15.19:0/1763087624 wait complete. 2024-03-28T13:09:48.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.583+0000 7f53dc400640 1 Processor -- start 2024-03-28T13:09:48.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.584+0000 7faa9aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7faa9c06da30 0x7faa9c0d43d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42928/0 (socket says 172.21.15.19:42928) 2024-03-28T13:09:48.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.584+0000 7f53dc400640 1 -- start start 2024-03-28T13:09:48.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.584+0000 7faa9aa00640 1 -- 172.21.15.19:0/1994327971 learned_addr learned my addr 172.21.15.19:0/1994327971 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:48.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.584+0000 7faa9b400640 1 --2- 172.21.15.19:0/1994327971 >> v2:172.21.15.19:3300/0 conn(0x7faa9c15e860 0x7faa9c06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.584+0000 7faa9be00640 1 --2- 172.21.15.19:0/1994327971 >> v2:172.21.15.188:3300/0 conn(0x7faa9c0d4ae0 0x7faa9c0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.584+0000 7faa9b400640 1 -- 172.21.15.19:0/1994327971 >> v2:172.21.15.188:3300/0 conn(0x7faa9c0d4ae0 msgr2=0x7faa9c0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.584+0000 7faa9b400640 1 --2- 172.21.15.19:0/1994327971 >> v2:172.21.15.188:3300/0 conn(0x7faa9c0d4ae0 0x7faa9c0cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.585+0000 7faa9b400640 1 -- 172.21.15.19:0/1994327971 >> v1:172.21.15.19:6789/0 conn(0x7faa9c06da30 legacy=0x7faa9c0d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.585+0000 7faa9b400640 1 -- 172.21.15.19:0/1994327971 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9c06e520 con 0x7faa9c15e860 2024-03-28T13:09:48.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.586+0000 7f791ba00640 1 -- 172.21.15.19:0/4262080779 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f79140af140 con 0x7f78f4074820 2024-03-28T13:09:48.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.586+0000 7f53dc400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f53d4162920 0x7f53d41670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.586+0000 7f7910c00640 1 -- 172.21.15.19:0/4262080779 <== osd.3 v2:172.21.15.19:6819/3265515865 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f79140af140 con 0x7f78f4074820 2024-03-28T13:09:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.586+0000 7f53dc400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f53d4163ac0 0x7f53d4071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.586+0000 7f53dc400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53d4071d60 con 0x7f53d40d0280 2024-03-28T13:09:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.587+0000 7f53da400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f53d40d0280 0x7f53d40ad010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42936/0 (socket says 172.21.15.19:42936) 2024-03-28T13:09:48.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.587+0000 7f791ba00640 1 -- 172.21.15.19:0/4262080779 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x7f78f4074820 msgr2=0x7f78f4076c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.587+0000 7f791ba00640 1 --2- 172.21.15.19:0/4262080779 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x7f78f4074820 0x7f78f4076c40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.587+0000 7f53da400640 1 -- 172.21.15.19:0/3637278845 learned_addr learned my addr 172.21.15.19:0/3637278845 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:48.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.587+0000 7f53dc400640 1 -- 172.21.15.19:0/3637278845 --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f53d4163f20 con 0x7f53d4162920 2024-03-28T13:09:48.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.587+0000 7f53dc400640 1 -- 172.21.15.19:0/3637278845 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f53d41665d0 con 0x7f53d4163ac0 2024-03-28T13:09:48.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.587+0000 7f53d9a00640 1 --2- 172.21.15.19:0/3637278845 >> v2:172.21.15.19:3300/0 conn(0x7f53d4162920 0x7f53d41670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.587+0000 7f791ba00640 1 -- 172.21.15.19:0/4262080779 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f78f406c600 msgr2=0x7f78f406eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.587+0000 7f791ba00640 1 --2- 172.21.15.19:0/4262080779 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f78f406c600 0x7f78f406eac0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7900009630 tx=0x7f7900008040 comp rx=0 tx=0).stop 2024-03-28T13:09:48.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.587+0000 7f791ba00640 1 -- 172.21.15.19:0/4262080779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7914071e80 msgr2=0x7f7914148b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.587+0000 7f791ba00640 1 --2- 172.21.15.19:0/4262080779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7914071e80 0x7f7914148b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f790800c820 tx=0x7f790800cc20 comp rx=0 tx=0).stop 2024-03-28T13:09:48.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.587+0000 7f53d3e00640 1 -- 172.21.15.19:0/3637278845 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3070554483 0 0) 0x7f53d4071d60 con 0x7f53d40d0280 2024-03-28T13:09:48.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.588+0000 7f53d3e00640 1 -- 172.21.15.19:0/3637278845 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53bc003520 con 0x7f53d40d0280 2024-03-28T13:09:48.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.588+0000 7f7919a00640 1 -- 172.21.15.19:0/4262080779 reap_dead start 2024-03-28T13:09:48.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.588+0000 7f791ba00640 1 -- 172.21.15.19:0/4262080779 shutdown_connections 2024-03-28T13:09:48.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.588+0000 7f791ba00640 1 -- 172.21.15.19:0/4262080779 >> 172.21.15.19:0/4262080779 conn(0x7f791406b750 msgr2=0x7f79140ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.588+0000 7f53d9000640 1 --2- 172.21.15.19:0/3637278845 >> v2:172.21.15.188:3300/0 conn(0x7f53d4163ac0 0x7f53d4071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.588+0000 7f791ba00640 1 -- 172.21.15.19:0/4262080779 shutdown_connections 2024-03-28T13:09:48.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.588+0000 7f791ba00640 1 -- 172.21.15.19:0/4262080779 wait complete. 2024-03-28T13:09:48.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.589+0000 7fc085600640 1 -- 172.21.15.19:0/3708458472 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc07804ee30 con 0x7fc060074880 2024-03-28T13:09:48.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.588+0000 7f53d9a00640 1 -- 172.21.15.19:0/3637278845 >> v2:172.21.15.188:3300/0 conn(0x7f53d4163ac0 msgr2=0x7f53d4071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.588+0000 7f53d9a00640 1 --2- 172.21.15.19:0/3637278845 >> v2:172.21.15.188:3300/0 conn(0x7f53d4163ac0 0x7f53d4071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.588+0000 7f53d9a00640 1 -- 172.21.15.19:0/3637278845 >> v1:172.21.15.19:6789/0 conn(0x7f53d40d0280 legacy=0x7f53d40ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.588+0000 7f53d9a00640 1 -- 172.21.15.19:0/3637278845 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d41668b0 con 0x7f53d4162920 2024-03-28T13:09:48.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.589+0000 7faa9b400640 1 --2- 172.21.15.19:0/1994327971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa9c15e860 0x7faa9c06d4f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0cf3a0 tx=0x7faa8c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d5f962ae79d2cc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.590+0000 7f53d9a00640 1 --2- 172.21.15.19:0/3637278845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4162920 0x7f53d41670b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f53d40d1050 tx=0x7f53c802b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f2a8ced88a894e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.591+0000 7fc076a00640 1 -- 172.21.15.19:0/3708458472 <== osd.2 v2:172.21.15.19:6816/1693606238 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc07804ee30 con 0x7fc060074880 2024-03-28T13:09:48.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.591+0000 7f53d3e00640 1 -- 172.21.15.19:0/3637278845 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8037070 con 0x7f53d4162920 2024-03-28T13:09:48.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.591+0000 7f53d3e00640 1 -- 172.21.15.19:0/3637278845 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c8030510 con 0x7f53d4162920 2024-03-28T13:09:48.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.591+0000 7f53d3e00640 1 -- 172.21.15.19:0/3637278845 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8037070 con 0x7f53d4162920 2024-03-28T13:09:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.590+0000 7faa9a000640 1 -- 172.21.15.19:0/1994327971 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c035020 con 0x7faa9c15e860 2024-03-28T13:09:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.590+0000 7faa9a000640 1 -- 172.21.15.19:0/1994327971 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8c031830 con 0x7faa9c15e860 2024-03-28T13:09:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.590+0000 7faa9a000640 1 -- 172.21.15.19:0/1994327971 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c035020 con 0x7faa9c15e860 2024-03-28T13:09:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.591+0000 7faaa2800640 1 -- 172.21.15.19:0/1994327971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa9c15e860 msgr2=0x7faa9c06d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.591+0000 7faaa2800640 1 --2- 172.21.15.19:0/1994327971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa9c15e860 0x7faa9c06d4f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0cf3a0 tx=0x7faa8c02b8d0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.592+0000 7faaa2800640 1 -- 172.21.15.19:0/1994327971 shutdown_connections 2024-03-28T13:09:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.592+0000 7faaa2800640 1 --2- 172.21.15.19:0/1994327971 >> v2:172.21.15.188:3300/0 conn(0x7faa9c0d4ae0 0x7faa9c0cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.592+0000 7faaa2800640 1 --2- 172.21.15.19:0/1994327971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa9c15e860 0x7faa9c06d4f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.592+0000 7faaa2800640 1 -- 172.21.15.19:0/1994327971 >> 172.21.15.19:0/1994327971 conn(0x7faa9c09d7d0 msgr2=0x7faa9c08bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.593+0000 7f53dc400640 1 -- 172.21.15.19:0/3637278845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4162920 msgr2=0x7f53d41670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.593+0000 7f53dc400640 1 --2- 172.21.15.19:0/3637278845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4162920 0x7f53d41670b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f53d40d1050 tx=0x7f53c802b630 comp rx=0 tx=0).stop 2024-03-28T13:09:48.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.593+0000 7faaa2800640 1 -- 172.21.15.19:0/1994327971 shutdown_connections 2024-03-28T13:09:48.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.593+0000 7faaa2800640 1 -- 172.21.15.19:0/1994327971 wait complete. 2024-03-28T13:09:48.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.593+0000 7f53dc400640 1 -- 172.21.15.19:0/3637278845 shutdown_connections 2024-03-28T13:09:48.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.593+0000 7f53dc400640 1 --2- 172.21.15.19:0/3637278845 >> v2:172.21.15.188:3300/0 conn(0x7f53d4163ac0 0x7f53d4071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.593+0000 7f53dc400640 1 --2- 172.21.15.19:0/3637278845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d4162920 0x7f53d41670b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.593+0000 7f53dc400640 1 -- 172.21.15.19:0/3637278845 >> 172.21.15.19:0/3637278845 conn(0x7f53d406b750 msgr2=0x7f53d406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.594+0000 7f53dc400640 1 -- 172.21.15.19:0/3637278845 shutdown_connections 2024-03-28T13:09:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.593+0000 7fc085600640 1 -- 172.21.15.19:0/3708458472 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x7fc060074880 msgr2=0x7fc060076ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.593+0000 7fc085600640 1 --2- 172.21.15.19:0/3708458472 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x7fc060074880 0x7fc060076ca0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.594+0000 7fc085600640 1 -- 172.21.15.19:0/3708458472 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc06006c600 msgr2=0x7fc06006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.594+0000 7fc085600640 1 --2- 172.21.15.19:0/3708458472 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc06006c600 0x7fc06006eac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc070036000 tx=0x7fc070008030 comp rx=0 tx=0).stop 2024-03-28T13:09:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.594+0000 7fc085600640 1 -- 172.21.15.19:0/3708458472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc078033090 msgr2=0x7fc078107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.594+0000 7fc085600640 1 --2- 172.21.15.19:0/3708458472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc078033090 0x7fc078107e30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc08002d900 tx=0x7fc08004ffb0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.595+0000 7fc07f400640 1 -- 172.21.15.19:0/3708458472 reap_dead start 2024-03-28T13:09:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.595+0000 7fc085600640 1 -- 172.21.15.19:0/3708458472 shutdown_connections 2024-03-28T13:09:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.595+0000 7fc085600640 1 -- 172.21.15.19:0/3708458472 >> 172.21.15.19:0/3708458472 conn(0x7fc078009fa0 msgr2=0x7fc07804d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.595+0000 7faaa2800640 1 Processor -- start 2024-03-28T13:09:48.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.596+0000 7fc085600640 1 -- 172.21.15.19:0/3708458472 shutdown_connections 2024-03-28T13:09:48.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.596+0000 7fc085600640 1 -- 172.21.15.19:0/3708458472 wait complete. 2024-03-28T13:09:48.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.595+0000 7f53dc400640 1 -- 172.21.15.19:0/3637278845 wait complete. 2024-03-28T13:09:48.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.596+0000 7faaa2800640 1 -- start start 2024-03-28T13:09:48.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.599+0000 7f53dc400640 1 Processor -- start 2024-03-28T13:09:48.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.599+0000 7f53dc400640 1 -- start start 2024-03-28T13:09:48.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.601+0000 7f53dc400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d40d0280 0x7f53d414e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.601+0000 7f53dc400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d4163ac0 0x7f53d414bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.601+0000 7eff61c00640 1 -- 172.21.15.19:0/185519724 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7eff34002dd0 con 0x7eff3c074850 2024-03-28T13:09:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.602+0000 7f53dc400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53d41f7c00 0x7f53d414d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.602+0000 7f53dc400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4072040 con 0x7f53d40d0280 2024-03-28T13:09:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.602+0000 7f53dc400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53d4162d00 con 0x7f53d4163ac0 2024-03-28T13:09:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.602+0000 7f53dc400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f53d40721c0 con 0x7f53d41f7c00 2024-03-28T13:09:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.602+0000 7f53d9000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d4163ac0 0x7f53d414bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.602+0000 7f53d9000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d4163ac0 0x7f53d414bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44086/0 (socket says 172.21.15.19:44086) 2024-03-28T13:09:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.602+0000 7f53d9000640 1 -- 172.21.15.19:0/71618799 learned_addr learned my addr 172.21.15.19:0/71618799 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.602+0000 7f53da400640 1 --2- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53d41f7c00 0x7f53d414d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.603+0000 7eff52a00640 1 -- 172.21.15.19:0/185519724 <== osd.5 v2:172.21.15.188:6800/3324247755 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7eff34002dd0 con 0x7eff3c074850 2024-03-28T13:09:48.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.603+0000 7f53d9a00640 1 --2- 172.21.15.19:0/71618799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d40d0280 0x7f53d414e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.603+0000 7faaa2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa9c06da30 0x7faa9c160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.603+0000 7f53da400640 1 -- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d4163ac0 msgr2=0x7f53d414bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.603+0000 7f53da400640 1 --2- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53d4163ac0 0x7f53d414bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.603+0000 7f53da400640 1 -- 172.21.15.19:0/71618799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d40d0280 msgr2=0x7f53d414e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.603+0000 7f53da400640 1 --2- 172.21.15.19:0/71618799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53d40d0280 0x7f53d414e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.603+0000 7f53da400640 1 -- 172.21.15.19:0/71618799 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d414c1f0 con 0x7f53d41f7c00 2024-03-28T13:09:48.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.603+0000 7faaa2800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa9c0d4ae0 0x7faa9c0b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.604+0000 7f53da400640 1 --2- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53d41f7c00 0x7f53d414d530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f53cc002080 tx=0x7f53cc00a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.603+0000 7faaa2800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa9c15e860 0x7faa9c15ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.604+0000 7f53d1600640 1 -- 172.21.15.19:0/71618799 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc018070 con 0x7f53d41f7c00 2024-03-28T13:09:48.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.603+0000 7faaa2800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c0cef30 con 0x7faa9c06da30 2024-03-28T13:09:48.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.604+0000 7f53d1600640 1 -- 172.21.15.19:0/71618799 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53cc004d60 con 0x7f53d41f7c00 2024-03-28T13:09:48.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.604+0000 7f53d1600640 1 -- 172.21.15.19:0/71618799 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53cc007310 con 0x7f53d41f7c00 2024-03-28T13:09:48.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.603+0000 7faaa2800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faa9c0cedb0 con 0x7faa9c0d4ae0 2024-03-28T13:09:48.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.603+0000 7faaa2800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7faa9c0cfbf0 con 0x7faa9c15e860 2024-03-28T13:09:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.604+0000 7eff61c00640 1 -- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x7eff3c074850 msgr2=0x7eff3c076c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.605+0000 7eff61c00640 1 --2- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x7eff3c074850 0x7eff3c076c70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.605+0000 7eff61c00640 1 -- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7eff3c06c630 msgr2=0x7eff3c06eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.605+0000 7eff61c00640 1 --2- 172.21.15.19:0/185519724 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7eff3c06c630 0x7eff3c06eaf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7eff5c06a480 tx=0x7eff4c009190 comp rx=0 tx=0).stop 2024-03-28T13:09:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.605+0000 7eff61c00640 1 -- 172.21.15.19:0/185519724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff5c0b69d0 msgr2=0x7eff5c06cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.605+0000 7eff61c00640 1 --2- 172.21.15.19:0/185519724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff5c0b69d0 0x7eff5c06cbf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7eff4802c4a0 tx=0x7eff4802e070 comp rx=0 tx=0).stop 2024-03-28T13:09:48.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.606+0000 7faa9be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa9c15e860 0x7faa9c15ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.606+0000 7faa9aa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa9c0d4ae0 0x7faa9c0b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.606+0000 7faa9b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa9c06da30 0x7faa9c160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.606+0000 7faa9be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa9c15e860 0x7faa9c15ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:54784/0 (socket says 172.21.15.19:54784) 2024-03-28T13:09:48.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.607+0000 7faa9be00640 1 -- 172.21.15.19:0/3302727325 learned_addr learned my addr 172.21.15.19:0/3302727325 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.613 INFO:teuthology.orchestra.run.smithi019.stdout:68719476743 2024-03-28T13:09:48.613 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-28T13:09:48.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.606+0000 7eff5b400640 1 -- 172.21.15.19:0/185519724 reap_dead start 2024-03-28T13:09:48.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.606+0000 7eff61c00640 1 -- 172.21.15.19:0/185519724 shutdown_connections 2024-03-28T13:09:48.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.606+0000 7eff61c00640 1 -- 172.21.15.19:0/185519724 >> 172.21.15.19:0/185519724 conn(0x7eff5c09d7b0 msgr2=0x7eff5c15d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.608+0000 7eff61c00640 1 -- 172.21.15.19:0/185519724 shutdown_connections 2024-03-28T13:09:48.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.605+0000 7f53dc400640 1 -- 172.21.15.19:0/71618799 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d414c490 con 0x7f53d41f7c00 2024-03-28T13:09:48.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.605+0000 7f53dc400640 1 -- 172.21.15.19:0/71618799 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d414ca50 con 0x7f53d41f7c00 2024-03-28T13:09:48.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.607+0000 7faa9aa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa9c0d4ae0 0x7faa9c0b6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44088/0 (socket says 172.21.15.19:44088) 2024-03-28T13:09:48.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.608+0000 7eff61c00640 1 -- 172.21.15.19:0/185519724 wait complete. 2024-03-28T13:09:48.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.608+0000 7faa9b400640 1 -- 172.21.15.19:0/3302727325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa9c15e860 msgr2=0x7faa9c15ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.609+0000 7f53dc400640 1 -- 172.21.15.19:0/71618799 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f53d40685c0 con 0x7f53d41f7c00 2024-03-28T13:09:48.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.608+0000 7faa9b400640 1 --2- 172.21.15.19:0/3302727325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa9c15e860 0x7faa9c15ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.608+0000 7faa9b400640 1 -- 172.21.15.19:0/3302727325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa9c0d4ae0 msgr2=0x7faa9c0b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.608+0000 7faa9b400640 1 --2- 172.21.15.19:0/3302727325 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa9c0d4ae0 0x7faa9c0b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.608+0000 7faa9b400640 1 -- 172.21.15.19:0/3302727325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9c161630 con 0x7faa9c06da30 2024-03-28T13:09:48.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.610+0000 7faa9b400640 1 --2- 172.21.15.19:0/3302727325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa9c06da30 0x7faa9c160160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faa8c033000 tx=0x7faa8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.611+0000 7faa93400640 1 -- 172.21.15.19:0/3302727325 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c035050 con 0x7faa9c06da30 2024-03-28T13:09:48.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.611+0000 7faa93400640 1 -- 172.21.15.19:0/3302727325 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8c003610 con 0x7faa9c06da30 2024-03-28T13:09:48.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.611+0000 7faa93400640 1 -- 172.21.15.19:0/3302727325 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c035050 con 0x7faa9c06da30 2024-03-28T13:09:48.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.611+0000 7faaa2800640 1 -- 172.21.15.19:0/3302727325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9c1606a0 con 0x7faa9c06da30 2024-03-28T13:09:48.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.611+0000 7faaa2800640 1 -- 172.21.15.19:0/3302727325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa9c0b7200 con 0x7faa9c06da30 2024-03-28T13:09:48.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.610+0000 7f53d1600640 1 -- 172.21.15.19:0/71618799 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f53cc01b070 con 0x7f53d41f7c00 2024-03-28T13:09:48.619 INFO:teuthology.orchestra.run.smithi019.stdout:68719476743 2024-03-28T13:09:48.619 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-28T13:09:48.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.615+0000 7faa93400640 1 -- 172.21.15.19:0/3302727325 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7faa8c038020 con 0x7faa9c06da30 2024-03-28T13:09:48.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.616+0000 7f53d1600640 1 --2- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f53b406c600 0x7f53b406eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.616+0000 7faaa2800640 1 -- 172.21.15.19:0/3302727325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faa9c0685c0 con 0x7faa9c06da30 2024-03-28T13:09:48.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.616+0000 7f53d1600640 1 -- 172.21.15.19:0/71618799 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f53cc08b4b0 con 0x7f53d41f7c00 2024-03-28T13:09:48.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.616+0000 7faa93400640 1 --2- 172.21.15.19:0/3302727325 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faa7c06c730 0x7faa7c06ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.616+0000 7f53d1600640 1 --2- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x7f53b4074880 0x7f53b4076ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.616+0000 7f53d9a00640 1 --2- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f53b406c600 0x7f53b406eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.617+0000 7f53d1600640 1 -- 172.21.15.19:0/71618799 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53cc017ec0 con 0x7f53b4074880 2024-03-28T13:09:48.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.617+0000 7faa93400640 1 -- 172.21.15.19:0/3302727325 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7faa8c0acec0 con 0x7faa9c06da30 2024-03-28T13:09:48.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.617+0000 7faa93400640 1 --2- 172.21.15.19:0/3302727325 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x7faa7c0748d0 0x7faa7c076cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.617+0000 7faa93400640 1 -- 172.21.15.19:0/3302727325 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faa8c04dec0 con 0x7faa7c0748d0 2024-03-28T13:09:48.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.617+0000 7f53d1600640 1 -- 172.21.15.19:0/71618799 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f53cc092050 con 0x7f53d41f7c00 2024-03-28T13:09:48.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.617+0000 7faa9aa00640 1 --2- 172.21.15.19:0/3302727325 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faa7c06c730 0x7faa7c06ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.617+0000 7faa93400640 1 -- 172.21.15.19:0/3302727325 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7faa8c076f50 con 0x7faa9c06da30 2024-03-28T13:09:48.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.617+0000 7f53d9000640 1 --2- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x7f53b4074880 0x7f53b4076ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.618+0000 7faa9be00640 1 --2- 172.21.15.19:0/3302727325 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x7faa7c0748d0 0x7faa7c076cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.624 INFO:teuthology.orchestra.run.smithi019.stdout:68719476745 2024-03-28T13:09:48.624 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-28T13:09:48.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.618+0000 7f53d9a00640 1 --2- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f53b406c600 0x7f53b406eac0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f53c8034000 tx=0x7f53c8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.619+0000 7faa9aa00640 1 --2- 172.21.15.19:0/3302727325 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faa7c06c730 0x7faa7c06ebf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0a2350 tx=0x7faa94009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.619+0000 7f53d9000640 1 --2- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x7f53b4074880 0x7f53b4076ca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.621+0000 7f53d1600640 1 -- 172.21.15.19:0/71618799 <== osd.4 v2:172.21.15.188:6808/1845695132 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f53cc017ec0 con 0x7f53b4074880 2024-03-28T13:09:48.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.619+0000 7faa9be00640 1 --2- 172.21.15.19:0/3302727325 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x7faa7c0748d0 0x7faa7c076cf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.619+0000 7f52dcc00640 1 -- 172.21.15.19:0/2363931768 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f52bc002dd0 con 0x7f52c8074820 2024-03-28T13:09:48.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.621+0000 7f52df400640 1 -- 172.21.15.19:0/2363931768 <== osd.7 v2:172.21.15.188:6824/861796513 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f52bc002dd0 con 0x7f52c8074820 2024-03-28T13:09:48.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.621+0000 7f52dcc00640 1 -- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x7f52c8074820 msgr2=0x7f52c8076c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.621+0000 7f52dcc00640 1 --2- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x7f52c8074820 0x7f52c8076c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.621+0000 7f52dcc00640 1 -- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52c806c600 msgr2=0x7f52c806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.621+0000 7f52dcc00640 1 --2- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52c806c600 0x7f52c806eac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f52e81fbed0 tx=0x7f52d8006c40 comp rx=0 tx=0).stop 2024-03-28T13:09:48.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.621+0000 7f52dcc00640 1 -- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e80b1530 msgr2=0x7f52e8151480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.621+0000 7f52dcc00640 1 --2- 172.21.15.19:0/2363931768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52e80b1530 0x7f52e8151480 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f52d402b930 tx=0x7f52d40046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.621+0000 7f52e7e00640 1 -- 172.21.15.19:0/2363931768 reap_dead start 2024-03-28T13:09:48.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.622+0000 7f52dcc00640 1 -- 172.21.15.19:0/2363931768 shutdown_connections 2024-03-28T13:09:48.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.622+0000 7f52dcc00640 1 -- 172.21.15.19:0/2363931768 >> 172.21.15.19:0/2363931768 conn(0x7f52e8072a00 msgr2=0x7f52e8072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.622+0000 7f52dcc00640 1 -- 172.21.15.19:0/2363931768 shutdown_connections 2024-03-28T13:09:48.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.622+0000 7f52dcc00640 1 -- 172.21.15.19:0/2363931768 wait complete. 2024-03-28T13:09:48.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.623+0000 7faa93400640 1 -- 172.21.15.19:0/3302727325 <== osd.6 v2:172.21.15.188:6816/3597358155 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faa8c04dec0 con 0x7faa7c0748d0 2024-03-28T13:09:48.629 INFO:teuthology.orchestra.run.smithi019.stdout:68719476743 2024-03-28T13:09:48.629 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-28T13:09:48.632 INFO:teuthology.orchestra.run.smithi019.stdout:64424509447 2024-03-28T13:09:48.633 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-28T13:09:48.647 INFO:teuthology.orchestra.run.smithi019.stdout:60129542152 2024-03-28T13:09:48.648 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-28T13:09:48.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.657+0000 7f53dc400640 1 -- 172.21.15.19:0/71618799 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f53d406ef60 con 0x7f53b4074880 2024-03-28T13:09:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.659+0000 7f53d1600640 1 -- 172.21.15.19:0/71618799 <== osd.4 v2:172.21.15.188:6808/1845695132 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f53d406ef60 con 0x7f53b4074880 2024-03-28T13:09:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.659+0000 7f53c6a00640 1 -- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x7f53b4074880 msgr2=0x7f53b4076ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.659+0000 7f53c6a00640 1 --2- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x7f53b4074880 0x7f53b4076ca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.660+0000 7f53c6a00640 1 -- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f53b406c600 msgr2=0x7f53b406eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.660+0000 7f53c6a00640 1 --2- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f53b406c600 0x7f53b406eac0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f53c8034000 tx=0x7f53c8008030 comp rx=0 tx=0).stop 2024-03-28T13:09:48.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.660+0000 7f53c6a00640 1 -- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53d41f7c00 msgr2=0x7f53d414d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.660+0000 7f53c6a00640 1 --2- 172.21.15.19:0/71618799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53d41f7c00 0x7f53d414d530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f53cc002080 tx=0x7f53cc00a430 comp rx=0 tx=0).stop 2024-03-28T13:09:48.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.660+0000 7f53da400640 1 -- 172.21.15.19:0/71618799 reap_dead start 2024-03-28T13:09:48.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.660+0000 7f53c6a00640 1 -- 172.21.15.19:0/71618799 shutdown_connections 2024-03-28T13:09:48.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.660+0000 7f53c6a00640 1 -- 172.21.15.19:0/71618799 >> 172.21.15.19:0/71618799 conn(0x7f53d406b750 msgr2=0x7f53d40ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.660+0000 7f53c6a00640 1 -- 172.21.15.19:0/71618799 shutdown_connections 2024-03-28T13:09:48.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.660+0000 7f53c6a00640 1 -- 172.21.15.19:0/71618799 wait complete. 2024-03-28T13:09:48.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.658+0000 7faaa2800640 1 -- 172.21.15.19:0/3302727325 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faa9c0cb190 con 0x7faa7c0748d0 2024-03-28T13:09:48.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.660+0000 7faa93400640 1 -- 172.21.15.19:0/3302727325 <== osd.6 v2:172.21.15.188:6816/3597358155 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faa9c0cb190 con 0x7faa7c0748d0 2024-03-28T13:09:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.661+0000 7faaa2800640 1 -- 172.21.15.19:0/3302727325 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x7faa7c0748d0 msgr2=0x7faa7c076cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.661+0000 7faaa2800640 1 --2- 172.21.15.19:0/3302727325 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x7faa7c0748d0 0x7faa7c076cf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.662+0000 7faaa2800640 1 -- 172.21.15.19:0/3302727325 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faa7c06c730 msgr2=0x7faa7c06ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.662+0000 7faaa2800640 1 --2- 172.21.15.19:0/3302727325 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faa7c06c730 0x7faa7c06ebf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0a2350 tx=0x7faa94009040 comp rx=0 tx=0).stop 2024-03-28T13:09:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.662+0000 7faaa2800640 1 -- 172.21.15.19:0/3302727325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa9c06da30 msgr2=0x7faa9c160160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.662+0000 7faaa2800640 1 --2- 172.21.15.19:0/3302727325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa9c06da30 0x7faa9c160160 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faa8c033000 tx=0x7faa8c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.663+0000 7faa9be00640 1 -- 172.21.15.19:0/3302727325 reap_dead start 2024-03-28T13:09:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.663+0000 7faaa2800640 1 -- 172.21.15.19:0/3302727325 shutdown_connections 2024-03-28T13:09:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.663+0000 7faaa2800640 1 -- 172.21.15.19:0/3302727325 >> 172.21.15.19:0/3302727325 conn(0x7faa9c09d7d0 msgr2=0x7faa9c0b5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.663+0000 7faaa2800640 1 -- 172.21.15.19:0/3302727325 shutdown_connections 2024-03-28T13:09:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.663+0000 7faaa2800640 1 -- 172.21.15.19:0/3302727325 wait complete. 2024-03-28T13:09:48.688 INFO:teuthology.orchestra.run.smithi019.stdout:64424509447 2024-03-28T13:09:48.689 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-28T13:09:48.695 INFO:teuthology.orchestra.run.smithi019.stdout:64424509446 2024-03-28T13:09:48.695 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-28T13:09:48.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.908+0000 7ff9eac00640 1 Processor -- start 2024-03-28T13:09:48.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.908+0000 7ff9eac00640 1 -- start start 2024-03-28T13:09:48.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.913+0000 7ff9eac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9dc0e30b0 0x7ff9dc057d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.913+0000 7ff9eac00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff9dc03e830 0x7ff9dc053be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.914+0000 7ff9eac00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9dc0582c0 con 0x7ff9dc054120 2024-03-28T13:09:48.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.914+0000 7ff9eac00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff9dc058560 con 0x7ff9dc03e830 2024-03-28T13:09:48.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.914+0000 7ff9eac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc058840 con 0x7ff9dc0e30b0 2024-03-28T13:09:48.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.914+0000 7ff9e3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9dc0e30b0 0x7ff9dc057d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.914+0000 7ff9e3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9dc0e30b0 0x7ff9dc057d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:54790/0 (socket says 172.21.15.19:54790) 2024-03-28T13:09:48.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.915+0000 7ff9e3e00640 1 -- 172.21.15.19:0/548019998 learned_addr learned my addr 172.21.15.19:0/548019998 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.916+0000 7ff9e3400640 1 --2- 172.21.15.19:0/548019998 >> v2:172.21.15.19:3300/0 conn(0x7ff9dc03e830 0x7ff9dc053be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.918+0000 7ff9e2a00640 1 -- 172.21.15.19:0/548019998 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2577749089 0 0) 0x7ff9dc0582c0 con 0x7ff9dc054120 2024-03-28T13:09:48.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.918+0000 7ff9e2a00640 1 -- 172.21.15.19:0/548019998 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9cc003520 con 0x7ff9dc054120 2024-03-28T13:09:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.920+0000 7ff9e3400640 1 -- 172.21.15.19:0/548019998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9dc0e30b0 msgr2=0x7ff9dc057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.920+0000 7ff9e3400640 1 --2- 172.21.15.19:0/548019998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9dc0e30b0 0x7ff9dc057d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.920+0000 7ff9e3400640 1 -- 172.21.15.19:0/548019998 >> v1:172.21.15.19:6789/0 conn(0x7ff9dc054120 legacy=0x7ff9dc0e6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.920+0000 7ff9e3400640 1 -- 172.21.15.19:0/548019998 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc058b20 con 0x7ff9dc03e830 2024-03-28T13:09:48.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.925+0000 7ff9e3400640 1 --2- 172.21.15.19:0/548019998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9dc03e830 0x7ff9dc053be0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4000b60 tx=0x7ff9e40a70e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8f4c8e759b686a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.927+0000 7ff9e2a00640 1 -- 172.21.15.19:0/548019998 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e40b4070 con 0x7ff9dc03e830 2024-03-28T13:09:48.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.927+0000 7ff9e2a00640 1 -- 172.21.15.19:0/548019998 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9e40af440 con 0x7ff9dc03e830 2024-03-28T13:09:48.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.927+0000 7ff9e2a00640 1 -- 172.21.15.19:0/548019998 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e40af8b0 con 0x7ff9dc03e830 2024-03-28T13:09:48.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.928+0000 7ff9eac00640 1 -- 172.21.15.19:0/548019998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9dc03e830 msgr2=0x7ff9dc053be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.928+0000 7ff9eac00640 1 --2- 172.21.15.19:0/548019998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9dc03e830 0x7ff9dc053be0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4000b60 tx=0x7ff9e40a70e0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.928+0000 7ff9eac00640 1 -- 172.21.15.19:0/548019998 shutdown_connections 2024-03-28T13:09:48.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.928+0000 7ff9eac00640 1 --2- 172.21.15.19:0/548019998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9dc03e830 0x7ff9dc053be0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.928+0000 7ff9eac00640 1 --2- 172.21.15.19:0/548019998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9dc0e30b0 0x7ff9dc057d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.928+0000 7ff9eac00640 1 -- 172.21.15.19:0/548019998 >> 172.21.15.19:0/548019998 conn(0x7ff9dc0559d0 msgr2=0x7ff9dc055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.931+0000 7ff9eac00640 1 -- 172.21.15.19:0/548019998 shutdown_connections 2024-03-28T13:09:48.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.931+0000 7ff9eac00640 1 -- 172.21.15.19:0/548019998 wait complete. 2024-03-28T13:09:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.934+0000 7ff9eac00640 1 Processor -- start 2024-03-28T13:09:48.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.934+0000 7ff9eac00640 1 -- start start 2024-03-28T13:09:48.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.936+0000 7ff9eac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9dc03e830 0x7ff9dc039e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.937+0000 7ff9eac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9dc054120 0x7ff9dc04e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.937+0000 7ff9eac00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9dc0e30b0 0x7ff9dc04ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.938+0000 7ff9eac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc0e3490 con 0x7ff9dc03e830 2024-03-28T13:09:48.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.938+0000 7ff9eac00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc03bfb0 con 0x7ff9dc0e30b0 2024-03-28T13:09:48.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.938+0000 7ff9e3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9dc03e830 0x7ff9dc039e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.938+0000 7ff9e8c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9dc0e30b0 0x7ff9dc04ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.938+0000 7ff9e8c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9dc0e30b0 0x7ff9dc04ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44102/0 (socket says 172.21.15.19:44102) 2024-03-28T13:09:48.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.938+0000 7ff9e8c00640 1 -- 172.21.15.19:0/1389208935 learned_addr learned my addr 172.21.15.19:0/1389208935 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.938+0000 7ff9e3400640 1 --2- 172.21.15.19:0/1389208935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9dc054120 0x7ff9dc04e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.939+0000 7ff9eac00640 1 -- 172.21.15.19:0/1389208935 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc0078d0 con 0x7ff9dc054120 2024-03-28T13:09:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.940+0000 7ff9e3e00640 1 -- 172.21.15.19:0/1389208935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9dc054120 msgr2=0x7ff9dc04e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.940+0000 7ff9e3e00640 1 --2- 172.21.15.19:0/1389208935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9dc054120 0x7ff9dc04e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.940+0000 7ff9e3e00640 1 -- 172.21.15.19:0/1389208935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9dc0e30b0 msgr2=0x7ff9dc04ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.940+0000 7ff9e3e00640 1 --2- 172.21.15.19:0/1389208935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9dc0e30b0 0x7ff9dc04ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.940+0000 7ff9e3e00640 1 -- 172.21.15.19:0/1389208935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc052f30 con 0x7ff9dc03e830 2024-03-28T13:09:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.941+0000 7ff9e3e00640 1 --2- 172.21.15.19:0/1389208935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9dc03e830 0x7ff9dc039e70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800ebe0 tx=0x7ff9d800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.942+0000 7ff9d7e00640 1 -- 172.21.15.19:0/1389208935 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8017070 con 0x7ff9dc03e830 2024-03-28T13:09:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.942+0000 7ff9eac00640 1 -- 172.21.15.19:0/1389208935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc179100 con 0x7ff9dc03e830 2024-03-28T13:09:48.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.943+0000 7ff9eac00640 1 -- 172.21.15.19:0/1389208935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc1796c0 con 0x7ff9dc03e830 2024-03-28T13:09:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.944+0000 7fbfac800640 1 Processor -- start 2024-03-28T13:09:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.944+0000 7fbfac800640 1 -- start start 2024-03-28T13:09:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.943+0000 7ff9d7e00640 1 -- 172.21.15.19:0/1389208935 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d800ecd0 con 0x7ff9dc03e830 2024-03-28T13:09:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.943+0000 7ff9d7e00640 1 -- 172.21.15.19:0/1389208935 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800ce30 con 0x7ff9dc03e830 2024-03-28T13:09:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.947+0000 7fbfac800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbfa40b1530 0x7fbfa40d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.947+0000 7fbfac800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfa415b240 con 0x7fbfa40b3b70 2024-03-28T13:09:48.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.947+0000 7fbfac800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfa415b520 con 0x7fbfa415de50 2024-03-28T13:09:48.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.947+0000 7fbfaa800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbfa40b1530 0x7fbfa40d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.948+0000 7fbfaa800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbfa40b1530 0x7fbfa40d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44114/0 (socket says 172.21.15.19:44114) 2024-03-28T13:09:48.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.948+0000 7fbfaa800640 1 -- 172.21.15.19:0/3312602793 learned_addr learned my addr 172.21.15.19:0/3312602793 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.948+0000 7fbfac800640 1 -- 172.21.15.19:0/3312602793 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fbfa415e230 con 0x7fbfa40b1530 2024-03-28T13:09:48.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.949+0000 7fbfa8a00640 1 -- 172.21.15.19:0/3312602793 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3779950097 0 0) 0x7fbfa415b520 con 0x7fbfa415de50 2024-03-28T13:09:48.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.950+0000 7fbfaa800640 1 -- 172.21.15.19:0/3312602793 >> v1:172.21.15.188:6789/0 conn(0x7fbfa415de50 legacy=0x7fbfa40b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.950+0000 7fbfaa800640 1 -- 172.21.15.19:0/3312602793 >> v1:172.21.15.19:6789/0 conn(0x7fbfa40b3b70 legacy=0x7fbfa415c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.950+0000 7fbfaa800640 1 -- 172.21.15.19:0/3312602793 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa415b800 con 0x7fbfa40b1530 2024-03-28T13:09:48.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.950+0000 7ff9d7e00640 1 -- 172.21.15.19:0/1389208935 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff9d801a070 con 0x7ff9dc03e830 2024-03-28T13:09:48.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.953+0000 7fbfaa800640 1 --2- 172.21.15.19:0/3312602793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40b1530 0x7fbfa40d0990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009870 tx=0x7fbf9402ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd99ef61b4017a11 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.953+0000 7ff37aa00640 1 Processor -- start 2024-03-28T13:09:48.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.953+0000 7fbfa8a00640 1 -- 172.21.15.19:0/3312602793 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9402f870 con 0x7fbfa40b1530 2024-03-28T13:09:48.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.953+0000 7fbfa8a00640 1 -- 172.21.15.19:0/3312602793 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf9402fa10 con 0x7fbfa40b1530 2024-03-28T13:09:48.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.954+0000 7fbfac800640 1 -- 172.21.15.19:0/3312602793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40b1530 msgr2=0x7fbfa40d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.954+0000 7ff37aa00640 1 -- start start 2024-03-28T13:09:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.953+0000 7ff9d7e00640 1 --2- 172.21.15.19:0/1389208935 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff9c406c600 0x7ff9c406eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.953+0000 7ff9eac00640 1 -- 172.21.15.19:0/1389208935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c0005290 con 0x7ff9dc03e830 2024-03-28T13:09:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.953+0000 7ff9d7e00640 1 -- 172.21.15.19:0/1389208935 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff9d808b990 con 0x7ff9dc03e830 2024-03-28T13:09:48.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.954+0000 7ff9e3400640 1 --2- 172.21.15.19:0/1389208935 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff9c406c600 0x7ff9c406eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.954+0000 7fbfac800640 1 --2- 172.21.15.19:0/3312602793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40b1530 0x7fbfa40d0990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009870 tx=0x7fbf9402ba50 comp rx=0 tx=0).stop 2024-03-28T13:09:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.955+0000 7fbfac800640 1 -- 172.21.15.19:0/3312602793 shutdown_connections 2024-03-28T13:09:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.955+0000 7fbfac800640 1 --2- 172.21.15.19:0/3312602793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40b1530 0x7fbfa40d0990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.955+0000 7fbfac800640 1 -- 172.21.15.19:0/3312602793 >> 172.21.15.19:0/3312602793 conn(0x7fbfa4072a00 msgr2=0x7fbfa4072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.955+0000 7fbfac800640 1 -- 172.21.15.19:0/3312602793 shutdown_connections 2024-03-28T13:09:48.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.955+0000 7fbfac800640 1 -- 172.21.15.19:0/3312602793 wait complete. 2024-03-28T13:09:48.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.956+0000 7ff9e3400640 1 --2- 172.21.15.19:0/1389208935 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff9c406c600 0x7ff9c406eac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc008620 tx=0x7ff9e40ab000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.956+0000 7ff37aa00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff374073520 0x7ff3740d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.956+0000 7ff37aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3740b1530 0x7ff37415dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.956+0000 7ff37aa00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff374161770 con 0x7ff3740b3460 2024-03-28T13:09:48.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.956+0000 7ff37aa00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff37415b520 con 0x7ff374073520 2024-03-28T13:09:48.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.956+0000 7ff37aa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff37415b800 con 0x7ff3740b1530 2024-03-28T13:09:48.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.957+0000 7ff373400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3740b1530 0x7ff37415dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.957+0000 7ff373e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff374073520 0x7ff3740d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.958+0000 7f0711000640 1 Processor -- start 2024-03-28T13:09:48.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.958+0000 7ff373e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff374073520 0x7ff3740d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44126/0 (socket says 172.21.15.19:44126) 2024-03-28T13:09:48.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.958+0000 7ff373e00640 1 -- 172.21.15.19:0/2130848515 learned_addr learned my addr 172.21.15.19:0/2130848515 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.958+0000 7f0711000640 1 -- start start 2024-03-28T13:09:48.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.959+0000 7ff373e00640 1 -- 172.21.15.19:0/2130848515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3740b1530 msgr2=0x7ff37415dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.959+0000 7ff373e00640 1 --2- 172.21.15.19:0/2130848515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3740b1530 0x7ff37415dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.959+0000 7fbfac800640 1 Processor -- start 2024-03-28T13:09:48.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.959+0000 7fbfac800640 1 -- start start 2024-03-28T13:09:48.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.959+0000 7ff373e00640 1 -- 172.21.15.19:0/2130848515 >> v1:172.21.15.19:6789/0 conn(0x7ff3740b3460 legacy=0x7ff37415e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.959+0000 7ff373e00640 1 -- 172.21.15.19:0/2130848515 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37415bae0 con 0x7ff374073520 2024-03-28T13:09:48.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.961+0000 7fbfac800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40b1530 0x7fbfa40998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.961+0000 7fbfac800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa40b3b70 0x7fbfa409d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.961+0000 7f0711000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f070c0b3b70 0x7f070c0d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.961+0000 7fbfac800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfa415de50 0x7fbfa40aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.962+0000 7fbfac800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa415ae30 con 0x7fbfa415de50 2024-03-28T13:09:48.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.962+0000 7f0711000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f070c15b240 con 0x7f070c15de50 2024-03-28T13:09:48.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.962+0000 7fbfac800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa40b4150 con 0x7fbfa40b1530 2024-03-28T13:09:48.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.962+0000 7fbfac800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa415afb0 con 0x7fbfa40b3b70 2024-03-28T13:09:48.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.962+0000 7fbfa9e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40b1530 0x7fbfa40998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.962+0000 7f0711000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f070c15b520 con 0x7f070c0b1530 2024-03-28T13:09:48.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.962+0000 7fbfa9e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40b1530 0x7fbfa40998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44136/0 (socket says 172.21.15.19:44136) 2024-03-28T13:09:48.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.963+0000 7f070a800640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f070c0b1530 0x7f070c15c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:42580/0 (socket says 172.21.15.19:42580) 2024-03-28T13:09:48.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.963+0000 7f070a800640 1 -- 172.21.15.19:0/1610756512 learned_addr learned my addr 172.21.15.19:0/1610756512 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:48.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.963+0000 7f0711000640 1 -- 172.21.15.19:0/1610756512 --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f070c15e230 con 0x7f070c0b3b70 2024-03-28T13:09:48.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.962+0000 7fbfa9e00640 1 -- 172.21.15.19:0/4168427232 learned_addr learned my addr 172.21.15.19:0/4168427232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.963+0000 7fbfa9400640 1 --2- 172.21.15.19:0/4168427232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa40b3b70 0x7fbfa409d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.963+0000 7ff373e00640 1 --2- 172.21.15.19:0/2130848515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff374073520 0x7ff3740d0990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff36c009e50 tx=0x7ff36c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de4e4e8cd4d85047 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.963+0000 7f0708a00640 1 -- 172.21.15.19:0/1610756512 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4204969648 0 0) 0x7f070c15b520 con 0x7f070c0b1530 2024-03-28T13:09:48.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.963+0000 7f0708a00640 1 -- 172.21.15.19:0/1610756512 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06f0003520 con 0x7f070c0b1530 2024-03-28T13:09:48.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.964+0000 7fbfa9e00640 1 -- 172.21.15.19:0/4168427232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa40b3b70 msgr2=0x7fbfa409d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.964+0000 7fbfa9e00640 1 --2- 172.21.15.19:0/4168427232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa40b3b70 0x7fbfa409d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.964+0000 7fbfa9e00640 1 -- 172.21.15.19:0/4168427232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfa415de50 msgr2=0x7fbfa40aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.964+0000 7f0709400640 1 --2- 172.21.15.19:0/1610756512 >> v2:172.21.15.19:3300/0 conn(0x7f070c0b3b70 0x7f070c0d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.964+0000 7f0708a00640 1 -- 172.21.15.19:0/1610756512 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 195871274 0 0) 0x7f06f0003520 con 0x7f070c0b1530 2024-03-28T13:09:48.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.964+0000 7f0708a00640 1 -- 172.21.15.19:0/1610756512 >> v2:172.21.15.19:3300/0 conn(0x7f070c0b3b70 msgr2=0x7f070c0d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.964+0000 7f0708a00640 1 --2- 172.21.15.19:0/1610756512 >> v2:172.21.15.19:3300/0 conn(0x7f070c0b3b70 0x7f070c0d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.964+0000 7f0708a00640 1 -- 172.21.15.19:0/1610756512 >> v1:172.21.15.19:6789/0 conn(0x7f070c15de50 legacy=0x7f070c0b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.965+0000 7f0708a00640 1 -- 172.21.15.19:0/1610756512 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c15b800 con 0x7f070c0b1530 2024-03-28T13:09:48.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.965+0000 7f0708a00640 1 -- 172.21.15.19:0/1610756512 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f07040019f0 con 0x7f070c0b1530 2024-03-28T13:09:48.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.965+0000 7f0708a00640 1 -- 172.21.15.19:0/1610756512 >> v1:172.21.15.188:6789/0 conn(0x7f070c0b1530 legacy=0x7f070c15c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.965+0000 7f0708a00640 1 --2- 172.21.15.19:0/1610756512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f0004240 0x7f06f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.965+0000 7f0708a00640 1 --2- 172.21.15.19:0/1610756512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06f0004e60 0x7f06f00099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.965+0000 7f0708a00640 1 --2- 172.21.15.19:0/1610756512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06f0009ef0 0x7f06f000c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.965+0000 7f0708a00640 1 -- 172.21.15.19:0/1610756512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f070c15e230 con 0x7f06f0004240 2024-03-28T13:09:48.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.965+0000 7f0708a00640 1 -- 172.21.15.19:0/1610756512 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0003b90 con 0x7f06f0004e60 2024-03-28T13:09:48.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.965+0000 7f0708a00640 1 -- 172.21.15.19:0/1610756512 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f06f0003d90 con 0x7f06f0009ef0 2024-03-28T13:09:48.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.964+0000 7fbfa9e00640 1 --2- 172.21.15.19:0/4168427232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfa415de50 0x7fbfa40aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.964+0000 7fbfa9e00640 1 -- 172.21.15.19:0/4168427232 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa414daa0 con 0x7fbfa40b1530 2024-03-28T13:09:48.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.965+0000 7ff372a00640 1 -- 172.21.15.19:0/2130848515 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c02cb00 con 0x7ff374073520 2024-03-28T13:09:48.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.966+0000 7f070a800640 1 --2- 172.21.15.19:0/1610756512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06f0009ef0 0x7f06f000c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.966+0000 7ff372a00640 1 -- 172.21.15.19:0/2130848515 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff36c02cca0 con 0x7ff374073520 2024-03-28T13:09:48.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.966+0000 7ff372a00640 1 -- 172.21.15.19:0/2130848515 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c034610 con 0x7ff374073520 2024-03-28T13:09:48.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.966+0000 7f0709400640 1 --2- 172.21.15.19:0/1610756512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06f0004e60 0x7f06f00099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7ff37aa00640 1 -- 172.21.15.19:0/2130848515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff374073520 msgr2=0x7ff3740d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7ff37aa00640 1 --2- 172.21.15.19:0/2130848515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff374073520 0x7ff3740d0990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff36c009e50 tx=0x7ff36c02bf60 comp rx=0 tx=0).stop 2024-03-28T13:09:48.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7f4c08400640 1 Processor -- start 2024-03-28T13:09:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7f0709e00640 1 --2- 172.21.15.19:0/1610756512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f0004240 0x7f06f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7fbfa9e00640 1 --2- 172.21.15.19:0/4168427232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40b1530 0x7fbfa40998e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000c7e0 tx=0x7fbfa000cca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7ff37aa00640 1 -- 172.21.15.19:0/2130848515 shutdown_connections 2024-03-28T13:09:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7ff37aa00640 1 --2- 172.21.15.19:0/2130848515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3740b1530 0x7ff37415dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7ff37aa00640 1 --2- 172.21.15.19:0/2130848515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff374073520 0x7ff3740d0990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7ff37aa00640 1 -- 172.21.15.19:0/2130848515 >> 172.21.15.19:0/2130848515 conn(0x7ff374072a00 msgr2=0x7ff374072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7f070a800640 1 -- 172.21.15.19:0/1610756512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06f0004e60 msgr2=0x7f06f00099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7f070a800640 1 --2- 172.21.15.19:0/1610756512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06f0004e60 0x7f06f00099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7f070a800640 1 -- 172.21.15.19:0/1610756512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f0004240 msgr2=0x7f06f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7f4c08400640 1 -- start start 2024-03-28T13:09:48.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7ff9d7e00640 1 -- 172.21.15.19:0/1389208935 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff9d80599d0 con 0x7ff9dc03e830 2024-03-28T13:09:48.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.968+0000 7ff37aa00640 1 -- 172.21.15.19:0/2130848515 shutdown_connections 2024-03-28T13:09:48.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7fbf9e000640 1 -- 172.21.15.19:0/4168427232 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0003da0 con 0x7fbfa40b1530 2024-03-28T13:09:48.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.967+0000 7fbfac800640 1 -- 172.21.15.19:0/4168427232 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa414dd80 con 0x7fbfa40b1530 2024-03-28T13:09:48.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.968+0000 7fbf9e000640 1 -- 172.21.15.19:0/4168427232 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa0005070 con 0x7fbfa40b1530 2024-03-28T13:09:48.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.968+0000 7fbf9e000640 1 -- 172.21.15.19:0/4168427232 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0005420 con 0x7fbfa40b1530 2024-03-28T13:09:48.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.968+0000 7fbfac800640 1 -- 172.21.15.19:0/4168427232 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa414e340 con 0x7fbfa40b1530 2024-03-28T13:09:48.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.968+0000 7ff37aa00640 1 -- 172.21.15.19:0/2130848515 wait complete. 2024-03-28T13:09:48.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.970+0000 7f070a800640 1 --2- 172.21.15.19:0/1610756512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f0004240 0x7f06f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.970+0000 7f070a800640 1 -- 172.21.15.19:0/1610756512 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c15b800 con 0x7f06f0009ef0 2024-03-28T13:09:48.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.970+0000 7f0709e00640 1 --2- 172.21.15.19:0/1610756512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f06f0004240 0x7f06f0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:48.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.970+0000 7fbfac800640 1 -- 172.21.15.19:0/4168427232 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa414e540 con 0x7fbfa40b1530 2024-03-28T13:09:48.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.971+0000 7f070a800640 1 -- 172.21.15.19:0/1610756512 reap_dead start 2024-03-28T13:09:48.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.972+0000 7f070a800640 1 --2- 172.21.15.19:0/1610756512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06f0009ef0 0x7f06f000c2e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f07040013d0 tx=0x7f06f0005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb1dae8bc6e9ec30 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.972+0000 7f4c08400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c0015de50 0x7f4c000d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.973+0000 7f0708a00640 1 -- 172.21.15.19:0/1610756512 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070400a480 con 0x7f06f0009ef0 2024-03-28T13:09:48.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.972+0000 7ff37aa00640 1 Processor -- start 2024-03-28T13:09:48.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.973+0000 7f4c08400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4c000b3b70 0x7f4c0015c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.973+0000 7f0708a00640 1 -- 172.21.15.19:0/1610756512 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0704034630 con 0x7f06f0009ef0 2024-03-28T13:09:48.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.973+0000 7f0708a00640 1 -- 172.21.15.19:0/1610756512 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0704034a20 con 0x7f06f0009ef0 2024-03-28T13:09:48.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.973+0000 7f4c05a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c0015de50 0x7f4c000d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.973+0000 7f4c05a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c0015de50 0x7f4c000d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:54854/0 (socket says 172.21.15.19:54854) 2024-03-28T13:09:48.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.973+0000 7f4c05a00640 1 -- 172.21.15.19:0/3623510075 learned_addr learned my addr 172.21.15.19:0/3623510075 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.973+0000 7f4c08400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c0015b240 con 0x7f4c000b1530 2024-03-28T13:09:48.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.973+0000 7f4c08400640 1 -- 172.21.15.19:0/3623510075 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f4c0015e230 con 0x7f4c000b3b70 2024-03-28T13:09:48.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.972+0000 7fbf9e000640 1 -- 172.21.15.19:0/4168427232 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbfa00055c0 con 0x7fbfa40b1530 2024-03-28T13:09:48.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.973+0000 7fbf9e000640 1 --2- 172.21.15.19:0/4168427232 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbf8806c580 0x7fbf8806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.973+0000 7f4c08400640 1 -- 172.21.15.19:0/3623510075 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4c0015b520 con 0x7f4c0015de50 2024-03-28T13:09:48.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.974+0000 7fbf9e000640 1 -- 172.21.15.19:0/4168427232 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbfa008b690 con 0x7fbfa40b1530 2024-03-28T13:09:48.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.974+0000 7f4c06400640 1 --2- 172.21.15.19:0/3623510075 >> v2:172.21.15.188:3300/0 conn(0x7f4c000b3b70 0x7f4c0015c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.974+0000 7ff37aa00640 1 -- start start 2024-03-28T13:09:48.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.975+0000 7fbfa9400640 1 --2- 172.21.15.19:0/4168427232 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbf8806c580 0x7fbf8806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.975+0000 7ff37aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff374073520 0x7ff37415fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.975+0000 7ff37aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3740b1530 0x7ff37414d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.975+0000 7ff37aa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3740b3460 0x7ff37415e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.975+0000 7ff37aa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff37415ae30 con 0x7ff3740b1530 2024-03-28T13:09:48.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.975+0000 7ff37aa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff37415e120 con 0x7ff3740b3460 2024-03-28T13:09:48.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.975+0000 7ff37aa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff37415afb0 con 0x7ff374073520 2024-03-28T13:09:48.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.975+0000 7f4c06400640 1 -- 172.21.15.19:0/3623510075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c0015de50 msgr2=0x7f4c000d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.975+0000 7f4c06400640 1 --2- 172.21.15.19:0/3623510075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c0015de50 0x7f4c000d0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.975+0000 7f4c06400640 1 -- 172.21.15.19:0/3623510075 >> v1:172.21.15.188:6789/0 conn(0x7f4c000b1530 legacy=0x7f4c000b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.975+0000 7f4c06400640 1 -- 172.21.15.19:0/3623510075 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0015b800 con 0x7f4c000b3b70 2024-03-28T13:09:48.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.976+0000 7f0711000640 1 -- 172.21.15.19:0/1610756512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06f0009ef0 msgr2=0x7f06f000c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.976+0000 7f0711000640 1 --2- 172.21.15.19:0/1610756512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06f0009ef0 0x7f06f000c2e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f07040013d0 tx=0x7f06f0005830 comp rx=0 tx=0).stop 2024-03-28T13:09:48.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.976+0000 7ff373e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff374073520 0x7ff37415fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.976+0000 7ff373e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff374073520 0x7ff37415fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:54860/0 (socket says 172.21.15.19:54860) 2024-03-28T13:09:48.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.976+0000 7ff373e00640 1 -- 172.21.15.19:0/2511116790 learned_addr learned my addr 172.21.15.19:0/2511116790 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.976+0000 7ff378a00640 1 --2- 172.21.15.19:0/2511116790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3740b3460 0x7ff37415e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.976+0000 7f0711000640 1 -- 172.21.15.19:0/1610756512 shutdown_connections 2024-03-28T13:09:48.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.976+0000 7f0711000640 1 --2- 172.21.15.19:0/1610756512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06f0009ef0 0x7f06f000c2e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.976+0000 7f0711000640 1 -- 172.21.15.19:0/1610756512 >> 172.21.15.19:0/1610756512 conn(0x7f070c072a00 msgr2=0x7f070c072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.976+0000 7ff373400640 1 --2- 172.21.15.19:0/2511116790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3740b1530 0x7ff37414d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.977+0000 7f0711000640 1 -- 172.21.15.19:0/1610756512 shutdown_connections 2024-03-28T13:09:48.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.977+0000 7ff373e00640 1 -- 172.21.15.19:0/2511116790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3740b3460 msgr2=0x7ff37415e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.977+0000 7ff373e00640 1 --2- 172.21.15.19:0/2511116790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3740b3460 0x7ff37415e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.977+0000 7ff373e00640 1 -- 172.21.15.19:0/2511116790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3740b1530 msgr2=0x7ff37414d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.977+0000 7ff373e00640 1 --2- 172.21.15.19:0/2511116790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3740b1530 0x7ff37414d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.977+0000 7ff373e00640 1 -- 172.21.15.19:0/2511116790 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37415ecf0 con 0x7ff374073520 2024-03-28T13:09:48.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.977+0000 7f4c06400640 1 --2- 172.21.15.19:0/3623510075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c000b3b70 0x7f4c0015c000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800be30 tx=0x7f4bf80092f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5677e0dbd6b8f2e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.977+0000 7f0711000640 1 -- 172.21.15.19:0/1610756512 wait complete. 2024-03-28T13:09:48.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.978+0000 7ff373e00640 1 --2- 172.21.15.19:0/2511116790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff374073520 0x7ff37415fd90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff36c036040 tx=0x7ff36c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.978+0000 7ff36be00640 1 -- 172.21.15.19:0/2511116790 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c043020 con 0x7ff374073520 2024-03-28T13:09:48.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.978+0000 7fbfa9400640 1 --2- 172.21.15.19:0/4168427232 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbf8806c580 0x7fbf8806ea40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4152a30 tx=0x7fbf98009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.978+0000 7ff36be00640 1 -- 172.21.15.19:0/2511116790 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff36c02ce50 con 0x7ff374073520 2024-03-28T13:09:48.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.978+0000 7ff36be00640 1 -- 172.21.15.19:0/2511116790 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c009510 con 0x7ff374073520 2024-03-28T13:09:48.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.980+0000 7f0711000640 1 Processor -- start 2024-03-28T13:09:48.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.980+0000 7ff37aa00640 1 -- 172.21.15.19:0/2511116790 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37415efd0 con 0x7ff374073520 2024-03-28T13:09:48.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.980+0000 7ff37aa00640 1 -- 172.21.15.19:0/2511116790 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37415f870 con 0x7ff374073520 2024-03-28T13:09:48.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.980+0000 7f0711000640 1 -- start start 2024-03-28T13:09:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.982+0000 7f0711000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06f0009ef0 0x7f070c1f9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.982+0000 7f9dd6000640 1 Processor -- start 2024-03-28T13:09:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.982+0000 7f0711000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f070c1533d0 0x7f070c14f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.982+0000 7ff37aa00640 1 -- 172.21.15.19:0/2511116790 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37415f1a0 con 0x7ff374073520 2024-03-28T13:09:48.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.982+0000 7f9dd6000640 1 -- start start 2024-03-28T13:09:48.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.983+0000 7f0711000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f070c15c830 0x7f070c15ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.983+0000 7f0711000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f070c15ae30 con 0x7f070c1533d0 2024-03-28T13:09:48.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.983+0000 7f0711000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f070c0b4150 con 0x7f06f0009ef0 2024-03-28T13:09:48.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.983+0000 7f0709e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06f0009ef0 0x7f070c1f9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.983+0000 7f0711000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f070c15afb0 con 0x7f070c15c830 2024-03-28T13:09:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.983+0000 7f0709400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f070c1533d0 0x7f070c14f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.983+0000 7f0709e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06f0009ef0 0x7f070c1f9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44186/0 (socket says 172.21.15.19:44186) 2024-03-28T13:09:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.983+0000 7f0709e00640 1 -- 172.21.15.19:0/917104379 learned_addr learned my addr 172.21.15.19:0/917104379 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.983+0000 7f0709400640 1 -- 172.21.15.19:0/917104379 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f070c15c830 msgr2=0x7f070c15ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.983+0000 7f0709400640 1 --2- 172.21.15.19:0/917104379 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f070c15c830 0x7f070c15ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.983+0000 7f0709400640 1 -- 172.21.15.19:0/917104379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06f0009ef0 msgr2=0x7f070c1f9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.983+0000 7f0709400640 1 --2- 172.21.15.19:0/917104379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06f0009ef0 0x7f070c1f9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.984+0000 7f0709400640 1 -- 172.21.15.19:0/917104379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c1f9bd0 con 0x7f070c1533d0 2024-03-28T13:09:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.984+0000 7f0709400640 1 --2- 172.21.15.19:0/917104379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f070c1533d0 0x7f070c14f3a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f06f800b540 tx=0x7f06f800ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.985+0000 7f9dd6000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dd015e4c0 0x7f9dd00b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.984+0000 7ff36be00640 1 -- 172.21.15.19:0/2511116790 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff36c046020 con 0x7ff374073520 2024-03-28T13:09:48.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.985+0000 7f9dd6000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9dd00b69c0 0x7f9dd00b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.985+0000 7f9dd6000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dd00d2710 con 0x7f9dd00d37b0 2024-03-28T13:09:48.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.985+0000 7f9dd6000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd015e8a0 con 0x7f9dd00b69c0 2024-03-28T13:09:48.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.985+0000 7f9dd6000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd00d4040 con 0x7f9dd015e4c0 2024-03-28T13:09:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.985+0000 7f0702000640 1 -- 172.21.15.19:0/917104379 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8018070 con 0x7f070c1533d0 2024-03-28T13:09:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.985+0000 7f0702000640 1 -- 172.21.15.19:0/917104379 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f80107a0 con 0x7f070c1533d0 2024-03-28T13:09:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.985+0000 7f0702000640 1 -- 172.21.15.19:0/917104379 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8010b70 con 0x7f070c1533d0 2024-03-28T13:09:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.985+0000 7f0711000640 1 -- 172.21.15.19:0/917104379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070c156980 con 0x7f070c1533d0 2024-03-28T13:09:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.985+0000 7f0711000640 1 -- 172.21.15.19:0/917104379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f070c1f9f00 con 0x7f070c1533d0 2024-03-28T13:09:48.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.985+0000 7f9dd5600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9dd00b69c0 0x7f9dd00b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.986+0000 7f9dd5600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9dd00b69c0 0x7f9dd00b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44194/0 (socket says 172.21.15.19:44194) 2024-03-28T13:09:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.986+0000 7f9dd5600640 1 -- 172.21.15.19:0/2824996276 learned_addr learned my addr 172.21.15.19:0/2824996276 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.986+0000 7f9dd4c00640 1 --2- 172.21.15.19:0/2824996276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dd015e4c0 0x7f9dd00b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.986+0000 7ff36be00640 1 --2- 172.21.15.19:0/2511116790 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff35006c560 0x7ff35006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.986+0000 7ff36be00640 1 -- 172.21.15.19:0/2511116790 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff36c0add80 con 0x7ff374073520 2024-03-28T13:09:48.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.987+0000 7ff373400640 1 --2- 172.21.15.19:0/2511116790 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff35006c560 0x7ff35006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.986+0000 7f9dd4c00640 1 -- 172.21.15.19:0/2824996276 >> v2:172.21.15.188:3300/0 conn(0x7f9dd00b69c0 msgr2=0x7f9dd00b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.986+0000 7f9dd4c00640 1 --2- 172.21.15.19:0/2824996276 >> v2:172.21.15.188:3300/0 conn(0x7f9dd00b69c0 0x7f9dd00b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.986+0000 7f9dd4c00640 1 -- 172.21.15.19:0/2824996276 >> v1:172.21.15.188:6789/0 conn(0x7f9dd00d37b0 legacy=0x7f9dd00b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.986+0000 7f9dd4c00640 1 -- 172.21.15.19:0/2824996276 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd00d29f0 con 0x7f9dd015e4c0 2024-03-28T13:09:48.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.988+0000 7f0711000640 1 -- 172.21.15.19:0/917104379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f070c0685c0 con 0x7f070c1533d0 2024-03-28T13:09:48.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.989+0000 7f9dd4c00640 1 --2- 172.21.15.19:0/2824996276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dd015e4c0 0x7f9dd00b9bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0097a0 tx=0x7f9dbc02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46b007ee82f2a58c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.989+0000 7f0702000640 1 -- 172.21.15.19:0/917104379 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f06f801b070 con 0x7f070c1533d0 2024-03-28T13:09:48.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.989+0000 7ff373400640 1 --2- 172.21.15.19:0/2511116790 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff35006c560 0x7ff35006ea20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff37414e350 tx=0x7ff364008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.990+0000 7f0702000640 1 --2- 172.21.15.19:0/917104379 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f06ec06c600 0x7f06ec06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:48.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.990+0000 7f0702000640 1 -- 172.21.15.19:0/917104379 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f06f808bbb0 con 0x7f070c1533d0 2024-03-28T13:09:48.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.992+0000 7f4bffe00640 1 -- 172.21.15.19:0/3623510075 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf802c830 con 0x7f4c000b3b70 2024-03-28T13:09:48.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.993+0000 7f4bffe00640 1 -- 172.21.15.19:0/3623510075 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bf802c9d0 con 0x7f4c000b3b70 2024-03-28T13:09:48.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.993+0000 7f4bffe00640 1 -- 172.21.15.19:0/3623510075 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf802cdc0 con 0x7f4c000b3b70 2024-03-28T13:09:48.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.993+0000 7f0709e00640 1 --2- 172.21.15.19:0/917104379 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f06ec06c600 0x7f06ec06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:48.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.993+0000 7fbf9e000640 1 -- 172.21.15.19:0/4168427232 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbfa414e540 con 0x7fbfa40b1530 2024-03-28T13:09:48.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.993+0000 7f4c08400640 1 -- 172.21.15.19:0/3623510075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c000b3b70 msgr2=0x7f4c0015c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:48.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.993+0000 7f4c08400640 1 --2- 172.21.15.19:0/3623510075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c000b3b70 0x7f4c0015c000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800be30 tx=0x7f4bf80092f0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.995+0000 7f0709e00640 1 --2- 172.21.15.19:0/917104379 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f06ec06c600 0x7f06ec06eac0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06fc007fd0 tx=0x7f06fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:48.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.994+0000 7f4c08400640 1 -- 172.21.15.19:0/3623510075 shutdown_connections 2024-03-28T13:09:48.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.994+0000 7f4c08400640 1 --2- 172.21.15.19:0/3623510075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c000b3b70 0x7f4c0015c000 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.994+0000 7f4c08400640 1 --2- 172.21.15.19:0/3623510075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c0015de50 0x7f4c000d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:48.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.994+0000 7f4c08400640 1 -- 172.21.15.19:0/3623510075 >> 172.21.15.19:0/3623510075 conn(0x7f4c00072a00 msgr2=0x7f4c00072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:48.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.995+0000 7f4c08400640 1 -- 172.21.15.19:0/3623510075 shutdown_connections 2024-03-28T13:09:48.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.995+0000 7f4c08400640 1 -- 172.21.15.19:0/3623510075 wait complete. 2024-03-28T13:09:48.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.998+0000 7f4c08400640 1 Processor -- start 2024-03-28T13:09:49.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:48.998+0000 7f4c08400640 1 -- start start 2024-03-28T13:09:49.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.000+0000 7f4c08400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c000b1530 0x7f4c0009a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.000+0000 7f4c08400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000b3b70 0x7f4c0009dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.000+0000 7f4c08400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c0015de50 0x7f4c001ecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.000+0000 7f0702000640 1 -- 172.21.15.19:0/917104379 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f070c1f9bd0 con 0x7f070c1533d0 2024-03-28T13:09:49.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.001+0000 7f9dcf400640 1 -- 172.21.15.19:0/2824996276 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc035020 con 0x7f9dd015e4c0 2024-03-28T13:09:49.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.001+0000 7f9dcf400640 1 -- 172.21.15.19:0/2824996276 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dbc031830 con 0x7f9dd015e4c0 2024-03-28T13:09:49.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.001+0000 7f9dcf400640 1 -- 172.21.15.19:0/2824996276 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc031c40 con 0x7f9dd015e4c0 2024-03-28T13:09:49.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.000+0000 7f4c08400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0015ae30 con 0x7f4c000b3b70 2024-03-28T13:09:49.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.000+0000 7f4c08400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4c000b4150 con 0x7f4c000b1530 2024-03-28T13:09:49.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.002+0000 7f9dd6000640 1 -- 172.21.15.19:0/2824996276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dd015e4c0 msgr2=0x7f9dd00b9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.002+0000 7f9dd6000640 1 --2- 172.21.15.19:0/2824996276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dd015e4c0 0x7f9dd00b9bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0097a0 tx=0x7f9dbc02b8d0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.000+0000 7f4c08400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4c0015afb0 con 0x7f4c0015de50 2024-03-28T13:09:49.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.001+0000 7f4c05000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000b3b70 0x7f4c0009dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.001+0000 7f4c05000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000b3b70 0x7f4c0009dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50790/0 (socket says 172.21.15.19:50790) 2024-03-28T13:09:49.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.001+0000 7f4c05000640 1 -- 172.21.15.19:0/280835463 learned_addr learned my addr 172.21.15.19:0/280835463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:49.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.001+0000 7f4c06400640 1 --2- 172.21.15.19:0/280835463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c0015de50 0x7f4c001ecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.002+0000 7f4c05000640 1 -- 172.21.15.19:0/280835463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c0015de50 msgr2=0x7f4c001ecbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.002+0000 7f4c05000640 1 --2- 172.21.15.19:0/280835463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c0015de50 0x7f4c001ecbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.002+0000 7f4c05000640 1 -- 172.21.15.19:0/280835463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c000b1530 msgr2=0x7f4c0009a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.002+0000 7f4c05000640 1 --2- 172.21.15.19:0/280835463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c000b1530 0x7f4c0009a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.002+0000 7f4c05000640 1 -- 172.21.15.19:0/280835463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0009e1a0 con 0x7f4c000b3b70 2024-03-28T13:09:49.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.002+0000 7f9dd6000640 1 -- 172.21.15.19:0/2824996276 shutdown_connections 2024-03-28T13:09:49.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.002+0000 7f9dd6000640 1 --2- 172.21.15.19:0/2824996276 >> v2:172.21.15.188:3300/0 conn(0x7f9dd00b69c0 0x7f9dd00b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.002+0000 7f9dd6000640 1 --2- 172.21.15.19:0/2824996276 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dd015e4c0 0x7f9dd00b9bf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.002+0000 7f9dd6000640 1 -- 172.21.15.19:0/2824996276 >> 172.21.15.19:0/2824996276 conn(0x7f9dd00676a0 msgr2=0x7f9dd0067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:49.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.002+0000 7f9dd6000640 1 -- 172.21.15.19:0/2824996276 shutdown_connections 2024-03-28T13:09:49.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.003+0000 7f9dd6000640 1 -- 172.21.15.19:0/2824996276 wait complete. 2024-03-28T13:09:49.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.003+0000 7f4c05000640 1 --2- 172.21.15.19:0/280835463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000b3b70 0x7f4c0009dc60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00c3c0 tx=0x7f4bec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:49.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.003+0000 7f4bfd600640 1 -- 172.21.15.19:0/280835463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec0052d0 con 0x7f4c000b3b70 2024-03-28T13:09:49.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.003+0000 7f4bfd600640 1 -- 172.21.15.19:0/280835463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bec005470 con 0x7f4c000b3b70 2024-03-28T13:09:49.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.003+0000 7f4bfd600640 1 -- 172.21.15.19:0/280835463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec0052d0 con 0x7f4c000b3b70 2024-03-28T13:09:49.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.004+0000 7ff36be00640 1 -- 172.21.15.19:0/2511116790 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff36c04d030 con 0x7ff374073520 2024-03-28T13:09:49.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.005+0000 7f4c08400640 1 -- 172.21.15.19:0/280835463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c001ed110 con 0x7f4c000b3b70 2024-03-28T13:09:49.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.005+0000 7f9dd6000640 1 Processor -- start 2024-03-28T13:09:49.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.006+0000 7f9dd6000640 1 -- start start 2024-03-28T13:09:49.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.007+0000 7f9dd6000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dd00b69c0 0x7f9dd01fb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.007+0000 7f9dd6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd00d37b0 0x7f9dd01f9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.007+0000 7f9dd6000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dd015e4c0 0x7f9dd00b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.007+0000 7f9dd6000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd00d2fb0 con 0x7f9dd00d37b0 2024-03-28T13:09:49.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.007+0000 7f9dd6000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd00d3320 con 0x7f9dd015e4c0 2024-03-28T13:09:49.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.007+0000 7f9dd6000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd00ba130 con 0x7f9dd00b69c0 2024-03-28T13:09:49.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.008+0000 7f9dd4c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dd00b69c0 0x7f9dd01fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.008+0000 7f9dd4c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dd00b69c0 0x7f9dd01fb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:54906/0 (socket says 172.21.15.19:54906) 2024-03-28T13:09:49.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.008+0000 7f9dd4c00640 1 -- 172.21.15.19:0/557240121 learned_addr learned my addr 172.21.15.19:0/557240121 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:49.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.008+0000 7f4c08400640 1 -- 172.21.15.19:0/280835463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0009aac0 con 0x7f4c000b3b70 2024-03-28T13:09:49.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.008+0000 7f9dcfe00640 1 --2- 172.21.15.19:0/557240121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd00d37b0 0x7f9dd01f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.009+0000 7f9dd5600640 1 --2- 172.21.15.19:0/557240121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dd015e4c0 0x7f9dd00b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.011+0000 7f4c08400640 1 -- 172.21.15.19:0/280835463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c000b0f70 con 0x7f4c000b3b70 2024-03-28T13:09:49.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.009+0000 7f9dd4c00640 1 -- 172.21.15.19:0/557240121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dd015e4c0 msgr2=0x7f9dd00b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.009+0000 7f9dd4c00640 1 --2- 172.21.15.19:0/557240121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dd015e4c0 0x7f9dd00b7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.010+0000 7f9dd4c00640 1 -- 172.21.15.19:0/557240121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd00d37b0 msgr2=0x7f9dd01f9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.010+0000 7f9dd4c00640 1 --2- 172.21.15.19:0/557240121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd00d37b0 0x7f9dd01f9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.010+0000 7f9dd4c00640 1 -- 172.21.15.19:0/557240121 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd01f8e20 con 0x7f9dd00b69c0 2024-03-28T13:09:49.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.010+0000 7f9dd5600640 1 --2- 172.21.15.19:0/557240121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dd015e4c0 0x7f9dd00b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:49.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.011+0000 7f9dd4c00640 1 --2- 172.21.15.19:0/557240121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dd00b69c0 0x7f9dd01fb090 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0098c0 tx=0x7f9dbc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:49.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.012+0000 7f9dccc00640 1 -- 172.21.15.19:0/557240121 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc040070 con 0x7f9dd00b69c0 2024-03-28T13:09:49.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.012+0000 7f9dccc00640 1 -- 172.21.15.19:0/557240121 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dbc03a910 con 0x7f9dd00b69c0 2024-03-28T13:09:49.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.012+0000 7f9dccc00640 1 -- 172.21.15.19:0/557240121 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc035040 con 0x7f9dd00b69c0 2024-03-28T13:09:49.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.012+0000 7f9dd6000640 1 -- 172.21.15.19:0/557240121 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd00ccf60 con 0x7f9dd00b69c0 2024-03-28T13:09:49.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.011+0000 7f4bfd600640 1 -- 172.21.15.19:0/280835463 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4bec007020 con 0x7f4c000b3b70 2024-03-28T13:09:49.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.012+0000 7f9dd6000640 1 -- 172.21.15.19:0/557240121 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd00cd520 con 0x7f9dd00b69c0 2024-03-28T13:09:49.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.017+0000 7f9dd6000640 1 -- 172.21.15.19:0/557240121 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd0075940 con 0x7f9dd00b69c0 2024-03-28T13:09:49.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.017+0000 7f4bfd600640 1 --2- 172.21.15.19:0/280835463 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4be006c730 0x7f4be006ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.017+0000 7f4bfd600640 1 -- 172.21.15.19:0/280835463 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4bec08f060 con 0x7f4c000b3b70 2024-03-28T13:09:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.019+0000 7f4c05a00640 1 --2- 172.21.15.19:0/280835463 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4be006c730 0x7f4be006ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.020+0000 7f9dccc00640 1 -- 172.21.15.19:0/557240121 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9dbc038020 con 0x7f9dd00b69c0 2024-03-28T13:09:49.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.021+0000 7f9dccc00640 1 --2- 172.21.15.19:0/557240121 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9db006c530 0x7f9db006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.022+0000 7f4c05a00640 1 --2- 172.21.15.19:0/280835463 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4be006c730 0x7f4be006ebf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400de80 tx=0x7f4bf400f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:49.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.022+0000 7f9dccc00640 1 -- 172.21.15.19:0/557240121 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9dbc078eb0 con 0x7f9dd00b69c0 2024-03-28T13:09:49.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.022+0000 7f9dcfe00640 1 --2- 172.21.15.19:0/557240121 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9db006c530 0x7f9db006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.024+0000 7f9dcfe00640 1 --2- 172.21.15.19:0/557240121 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9db006c530 0x7f9db006e9f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0072c70 tx=0x7f9dc4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:49.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.027+0000 7f4bfd600640 1 -- 172.21.15.19:0/280835463 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4c000b0f70 con 0x7f4c000b3b70 2024-03-28T13:09:49.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.042+0000 7f9dccc00640 1 -- 172.21.15.19:0/557240121 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9dbc031800 con 0x7f9dd00b69c0 2024-03-28T13:09:49.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.061+0000 7f3474800640 1 Processor -- start 2024-03-28T13:09:49.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.061+0000 7f3474800640 1 -- start start 2024-03-28T13:09:49.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.063+0000 7f3474800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f346c15e860 0x7f346c06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.063+0000 7f3474800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f346c0d4ae0 0x7f346c0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.063+0000 7f3474800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f346c06dfa0 con 0x7f346c06da30 2024-03-28T13:09:49.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.063+0000 7f3474800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f346c0cf220 con 0x7f346c0d4ae0 2024-03-28T13:09:49.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.063+0000 7f3474800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f346c06e240 con 0x7f346c15e860 2024-03-28T13:09:49.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.066+0000 7f3471e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f346c15e860 0x7f346c06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.066+0000 7f3471e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f346c15e860 0x7f346c06d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44224/0 (socket says 172.21.15.19:44224) 2024-03-28T13:09:49.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.066+0000 7f3471e00640 1 -- 172.21.15.19:0/3645088069 learned_addr learned my addr 172.21.15.19:0/3645088069 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:49.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.067+0000 7f3471e00640 1 -- 172.21.15.19:0/3645088069 >> v2:172.21.15.19:3300/0 conn(0x7f346c0d4ae0 msgr2=0x7f346c0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:49.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.067+0000 7f3471e00640 1 --2- 172.21.15.19:0/3645088069 >> v2:172.21.15.19:3300/0 conn(0x7f346c0d4ae0 0x7f346c0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.067+0000 7f3471e00640 1 -- 172.21.15.19:0/3645088069 >> v1:172.21.15.19:6789/0 conn(0x7f346c06da30 legacy=0x7f346c0d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:49.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.067+0000 7f3471e00640 1 -- 172.21.15.19:0/3645088069 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346c06e520 con 0x7f346c15e860 2024-03-28T13:09:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.075+0000 7f3471e00640 1 --2- 172.21.15.19:0/3645088069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c15e860 0x7f346c06d4f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f346c0cf3a0 tx=0x7f346002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa5b77ed8cd1ef16 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.075+0000 7f3470a00640 1 -- 172.21.15.19:0/3645088069 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460035020 con 0x7f346c15e860 2024-03-28T13:09:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.076+0000 7f3470a00640 1 -- 172.21.15.19:0/3645088069 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3460031870 con 0x7f346c15e860 2024-03-28T13:09:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.076+0000 7f3470a00640 1 -- 172.21.15.19:0/3645088069 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3460031c40 con 0x7f346c15e860 2024-03-28T13:09:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.076+0000 7f3474800640 1 -- 172.21.15.19:0/3645088069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c15e860 msgr2=0x7f346c06d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.076+0000 7f3474800640 1 --2- 172.21.15.19:0/3645088069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c15e860 0x7f346c06d4f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f346c0cf3a0 tx=0x7f346002b910 comp rx=0 tx=0).stop 2024-03-28T13:09:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.077+0000 7f3474800640 1 -- 172.21.15.19:0/3645088069 shutdown_connections 2024-03-28T13:09:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.077+0000 7f3474800640 1 --2- 172.21.15.19:0/3645088069 >> v2:172.21.15.19:3300/0 conn(0x7f346c0d4ae0 0x7f346c0cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.077+0000 7f3474800640 1 --2- 172.21.15.19:0/3645088069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c15e860 0x7f346c06d4f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.077+0000 7f3474800640 1 -- 172.21.15.19:0/3645088069 >> 172.21.15.19:0/3645088069 conn(0x7f346c09d7d0 msgr2=0x7f346c08bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.077+0000 7f3474800640 1 -- 172.21.15.19:0/3645088069 shutdown_connections 2024-03-28T13:09:49.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.077+0000 7f3474800640 1 -- 172.21.15.19:0/3645088069 wait complete. 2024-03-28T13:09:49.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.080+0000 7f3474800640 1 Processor -- start 2024-03-28T13:09:49.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.080+0000 7f3474800640 1 -- start start 2024-03-28T13:09:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.083+0000 7f3474800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f346c06da30 0x7f346c1f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.083+0000 7f3474800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c0d4ae0 0x7f346c0ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.083+0000 7f3474800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c15e860 0x7f346c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.084+0000 7f3474800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f346c0cef30 con 0x7f346c15e860 2024-03-28T13:09:49.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.084+0000 7f3474800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f346c0cedb0 con 0x7f346c0d4ae0 2024-03-28T13:09:49.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.084+0000 7f3474800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f346c0cfbf0 con 0x7f346c06da30 2024-03-28T13:09:49.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.084+0000 7f3472800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c15e860 0x7f346c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.084+0000 7f3471e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f346c06da30 0x7f346c1f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.084+0000 7f3471e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f346c06da30 0x7f346c1f94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:54908/0 (socket says 172.21.15.19:54908) 2024-03-28T13:09:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.084+0000 7f3471e00640 1 -- 172.21.15.19:0/1634608569 learned_addr learned my addr 172.21.15.19:0/1634608569 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.084+0000 7f3471400640 1 --2- 172.21.15.19:0/1634608569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c0d4ae0 0x7f346c0ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.085+0000 7f3472800640 1 -- 172.21.15.19:0/1634608569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f346c06da30 msgr2=0x7f346c1f94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.085+0000 7f3472800640 1 --2- 172.21.15.19:0/1634608569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f346c06da30 0x7f346c1f94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.085+0000 7f3472800640 1 -- 172.21.15.19:0/1634608569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c0d4ae0 msgr2=0x7f346c0ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.085+0000 7f3472800640 1 --2- 172.21.15.19:0/1634608569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c0d4ae0 0x7f346c0ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.085+0000 7f3472800640 1 -- 172.21.15.19:0/1634608569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346c0cb7c0 con 0x7f346c15e860 2024-03-28T13:09:49.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.090+0000 7f3472800640 1 --2- 172.21.15.19:0/1634608569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c15e860 0x7f346c0cd840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3468009f00 tx=0x7f346800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.091+0000 7f3466000640 1 -- 172.21.15.19:0/1634608569 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346802a070 con 0x7f346c15e860 2024-03-28T13:09:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.091+0000 7f3466000640 1 -- 172.21.15.19:0/1634608569 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3468025440 con 0x7f346c15e860 2024-03-28T13:09:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.091+0000 7f3466000640 1 -- 172.21.15.19:0/1634608569 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468025830 con 0x7f346c15e860 2024-03-28T13:09:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.091+0000 7f3474800640 1 -- 172.21.15.19:0/1634608569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346c0cd020 con 0x7f346c15e860 2024-03-28T13:09:49.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.091+0000 7f3474800640 1 -- 172.21.15.19:0/1634608569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346c0cbd20 con 0x7f346c15e860 2024-03-28T13:09:49.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.093+0000 7f3474800640 1 -- 172.21.15.19:0/1634608569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f346c15e0d0 con 0x7f346c15e860 2024-03-28T13:09:49.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.094+0000 7f3466000640 1 -- 172.21.15.19:0/1634608569 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f346802d070 con 0x7f346c15e860 2024-03-28T13:09:49.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.095+0000 7f3466000640 1 --2- 172.21.15.19:0/1634608569 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f345006c580 0x7f345006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.095+0000 7f3466000640 1 -- 172.21.15.19:0/1634608569 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f346809c670 con 0x7f346c15e860 2024-03-28T13:09:49.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.096+0000 7f3471e00640 1 --2- 172.21.15.19:0/1634608569 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f345006c580 0x7f345006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.098+0000 7f3471e00640 1 --2- 172.21.15.19:0/1634608569 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f345006c580 0x7f345006ea40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f34600090b0 tx=0x7f346003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:49.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.110+0000 7f3466000640 1 -- 172.21.15.19:0/1634608569 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f346c0cb7c0 con 0x7f346c15e860 2024-03-28T13:09:49.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.114+0000 7fbe48000640 1 Processor -- start 2024-03-28T13:09:49.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.114+0000 7fbe48000640 1 -- start start 2024-03-28T13:09:49.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.117+0000 7fbe48000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbe400d0990 0x7fbe400aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.118+0000 7fbe48000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe400b40b0 con 0x7fbe40155550 2024-03-28T13:09:49.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.118+0000 7fbe48000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe400b4390 con 0x7fbe40155930 2024-03-28T13:09:49.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.118+0000 7fbe48000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fbe400b4670 con 0x7fbe400d0990 2024-03-28T13:09:49.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.118+0000 7fbe45600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbe40155550 0x7fbe4015f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43012/0 (socket says 172.21.15.19:43012) 2024-03-28T13:09:49.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.118+0000 7fbe45600640 1 -- 172.21.15.19:0/1519212391 learned_addr learned my addr 172.21.15.19:0/1519212391 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:49.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.118+0000 7fbe46000640 1 --2- 172.21.15.19:0/1519212391 >> v2:172.21.15.188:3300/0 conn(0x7fbe400d0990 0x7fbe400aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.120+0000 7fbe3fe00640 1 -- 172.21.15.19:0/1519212391 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3365864830 0 0) 0x7fbe400b40b0 con 0x7fbe40155550 2024-03-28T13:09:49.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.120+0000 7fbe3fe00640 1 -- 172.21.15.19:0/1519212391 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe28005730 con 0x7fbe40155550 2024-03-28T13:09:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.120+0000 7fbe3fe00640 1 -- 172.21.15.19:0/1519212391 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1861921031 0 0) 0x7fbe400b4390 con 0x7fbe40155930 2024-03-28T13:09:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.120+0000 7fbe3fe00640 1 -- 172.21.15.19:0/1519212391 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe28005a90 con 0x7fbe40155930 2024-03-28T13:09:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.120+0000 7fbe3fe00640 1 -- 172.21.15.19:0/1519212391 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 486839374 0 0) 0x7fbe28005730 con 0x7fbe40155550 2024-03-28T13:09:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.120+0000 7fbe3fe00640 1 -- 172.21.15.19:0/1519212391 >> v2:172.21.15.188:3300/0 conn(0x7fbe400d0990 msgr2=0x7fbe400aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.120+0000 7fbe3fe00640 1 --2- 172.21.15.19:0/1519212391 >> v2:172.21.15.188:3300/0 conn(0x7fbe400d0990 0x7fbe400aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.120+0000 7fbe3fe00640 1 -- 172.21.15.19:0/1519212391 >> v1:172.21.15.188:6789/0 conn(0x7fbe40155930 legacy=0x7fbe400d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.120+0000 7fbe3fe00640 1 -- 172.21.15.19:0/1519212391 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe400b4950 con 0x7fbe40155550 2024-03-28T13:09:49.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.121+0000 7fbe3fe00640 1 -- 172.21.15.19:0/1519212391 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fbe2c0015d0 con 0x7fbe40155550 2024-03-28T13:09:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.121+0000 7fbe3fe00640 1 -- 172.21.15.19:0/1519212391 >> v1:172.21.15.19:6789/0 conn(0x7fbe40155550 legacy=0x7fbe4015f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.121+0000 7fbe3fe00640 1 --2- 172.21.15.19:0/1519212391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe28005f40 0x7fbe28006340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.121+0000 7fbe3fe00640 1 --2- 172.21.15.19:0/1519212391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe28006a10 0x7fbe28006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.121+0000 7fbe3fe00640 1 --2- 172.21.15.19:0/1519212391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe28007540 0x7fbe2800ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.121+0000 7fbe3fe00640 1 -- 172.21.15.19:0/1519212391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe400b4670 con 0x7fbe28007540 2024-03-28T13:09:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.121+0000 7fbe3fe00640 1 -- 172.21.15.19:0/1519212391 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe28007c00 con 0x7fbe28005f40 2024-03-28T13:09:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.121+0000 7fbe3fe00640 1 -- 172.21.15.19:0/1519212391 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbe28007e00 con 0x7fbe28006a10 2024-03-28T13:09:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.122+0000 7fbe46000640 1 --2- 172.21.15.19:0/1519212391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe28007540 0x7fbe2800ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.123+0000 7fbe45600640 1 --2- 172.21.15.19:0/1519212391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe28005f40 0x7fbe28006340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.123+0000 7fbe46000640 1 -- 172.21.15.19:0/1519212391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe28006a10 msgr2=0x7fbe28006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.123+0000 7fbe46000640 1 --2- 172.21.15.19:0/1519212391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe28006a10 0x7fbe28006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.123+0000 7fbe46000640 1 -- 172.21.15.19:0/1519212391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe28005f40 msgr2=0x7fbe28006340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.123+0000 7fbe46000640 1 --2- 172.21.15.19:0/1519212391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe28005f40 0x7fbe28006340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.123+0000 7fbe46000640 1 -- 172.21.15.19:0/1519212391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe28005730 con 0x7fbe28007540 2024-03-28T13:09:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.125+0000 7fbe46000640 1 --2- 172.21.15.19:0/1519212391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe28007540 0x7fbe2800ffd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbe38007610 tx=0x7fbe28007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ff62fe697d418c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.125+0000 7fbe46000640 1 -- 172.21.15.19:0/1519212391 reap_dead start 2024-03-28T13:09:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.125+0000 7fbe3fe00640 1 -- 172.21.15.19:0/1519212391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38038070 con 0x7fbe28007540 2024-03-28T13:09:49.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.126+0000 7fbe3fe00640 1 -- 172.21.15.19:0/1519212391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe38033620 con 0x7fbe28007540 2024-03-28T13:09:49.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.127+0000 7fbe48000640 1 -- 172.21.15.19:0/1519212391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe28007540 msgr2=0x7fbe2800ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.127+0000 7fbe48000640 1 --2- 172.21.15.19:0/1519212391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe28007540 0x7fbe2800ffd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbe38007610 tx=0x7fbe28007a40 comp rx=0 tx=0).stop 2024-03-28T13:09:49.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.128+0000 7fbe48000640 1 -- 172.21.15.19:0/1519212391 shutdown_connections 2024-03-28T13:09:49.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.128+0000 7fbe48000640 1 --2- 172.21.15.19:0/1519212391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe28007540 0x7fbe2800ffd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.128+0000 7fbe48000640 1 -- 172.21.15.19:0/1519212391 >> 172.21.15.19:0/1519212391 conn(0x7fbe4006b750 msgr2=0x7fbe4006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:49.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.128+0000 7fbe48000640 1 -- 172.21.15.19:0/1519212391 shutdown_connections 2024-03-28T13:09:49.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.128+0000 7fbe48000640 1 -- 172.21.15.19:0/1519212391 wait complete. 2024-03-28T13:09:49.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.131+0000 7fbe48000640 1 Processor -- start 2024-03-28T13:09:49.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.132+0000 7fbe48000640 1 -- start start 2024-03-28T13:09:49.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.134+0000 7fbe48000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe28007540 0x7fbe401f0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.134+0000 7fbe48000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe401539f0 0x7fbe4009aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.134+0000 7fbe48000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe4009b0e0 0x7fbe4009c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.135+0000 7fbe48000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4015bba0 con 0x7fbe401539f0 2024-03-28T13:09:49.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.135+0000 7fbe48000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4015ba20 con 0x7fbe4009b0e0 2024-03-28T13:09:49.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.135+0000 7fbe48000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbe400b3cf0 con 0x7fbe28007540 2024-03-28T13:09:49.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.135+0000 7fbe45600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe28007540 0x7fbe401f0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.135+0000 7fbe46000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe4009b0e0 0x7fbe4009c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.135+0000 7fbe46000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe4009b0e0 0x7fbe4009c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44264/0 (socket says 172.21.15.19:44264) 2024-03-28T13:09:49.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.135+0000 7fbe46000640 1 -- 172.21.15.19:0/4021341922 learned_addr learned my addr 172.21.15.19:0/4021341922 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:49.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.136+0000 7fbe44c00640 1 --2- 172.21.15.19:0/4021341922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe401539f0 0x7fbe4009aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.136+0000 7fbe46000640 1 -- 172.21.15.19:0/4021341922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe28007540 msgr2=0x7fbe401f0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.136+0000 7fbe46000640 1 --2- 172.21.15.19:0/4021341922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe28007540 0x7fbe401f0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.136+0000 7fbe46000640 1 -- 172.21.15.19:0/4021341922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe401539f0 msgr2=0x7fbe4009aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.136+0000 7fbe46000640 1 --2- 172.21.15.19:0/4021341922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe401539f0 0x7fbe4009aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.136+0000 7fbe46000640 1 -- 172.21.15.19:0/4021341922 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4009b580 con 0x7fbe4009b0e0 2024-03-28T13:09:49.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.137+0000 7fbe46000640 1 --2- 172.21.15.19:0/4021341922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe4009b0e0 0x7fbe4009c620 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbe38007700 tx=0x7fbe38004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:49.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.137+0000 7fbe45600640 1 --2- 172.21.15.19:0/4021341922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe28007540 0x7fbe401f0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:49.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.138+0000 7fbe3d600640 1 -- 172.21.15.19:0/4021341922 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38038040 con 0x7fbe4009b0e0 2024-03-28T13:09:49.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.138+0000 7fbe3d600640 1 -- 172.21.15.19:0/4021341922 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe3803e650 con 0x7fbe4009b0e0 2024-03-28T13:09:49.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.138+0000 7fbe3d600640 1 -- 172.21.15.19:0/4021341922 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3803ea20 con 0x7fbe4009b0e0 2024-03-28T13:09:49.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.138+0000 7fbe48000640 1 -- 172.21.15.19:0/4021341922 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe4009b860 con 0x7fbe4009b0e0 2024-03-28T13:09:49.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.138+0000 7fbe48000640 1 -- 172.21.15.19:0/4021341922 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe4009be20 con 0x7fbe4009b0e0 2024-03-28T13:09:49.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.140+0000 7fbe48000640 1 -- 172.21.15.19:0/4021341922 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe40072ce0 con 0x7fbe4009b0e0 2024-03-28T13:09:49.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.142+0000 7fbe3d600640 1 -- 172.21.15.19:0/4021341922 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbe38005d10 con 0x7fbe4009b0e0 2024-03-28T13:09:49.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.144+0000 7fbe3d600640 1 --2- 172.21.15.19:0/4021341922 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbe2006c5b0 0x7fbe2006ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:49.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.144+0000 7fbe3d600640 1 -- 172.21.15.19:0/4021341922 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbe3807a2c0 con 0x7fbe4009b0e0 2024-03-28T13:09:49.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.145+0000 7fbe45600640 1 --2- 172.21.15.19:0/4021341922 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbe2006c5b0 0x7fbe2006ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:49.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.148+0000 7fbe45600640 1 --2- 172.21.15.19:0/4021341922 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbe2006c5b0 0x7fbe2006ea70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c000ff0 tx=0x7fbe2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:49.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.155+0000 7fbe3d600640 1 -- 172.21.15.19:0/4021341922 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbe40072ce0 con 0x7fbe4009b0e0 2024-03-28T13:09:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.423+0000 7ff9eac00640 1 -- 172.21.15.19:0/1389208935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff9c0005580 con 0x7ff9dc03e830 2024-03-28T13:09:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.430+0000 7ff9d7e00640 1 -- 172.21.15.19:0/1389208935 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff9d805dcd0 con 0x7ff9dc03e830 2024-03-28T13:09:49.434 INFO:teuthology.orchestra.run.smithi019.stdout:68719476742 2024-03-28T13:09:49.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.439+0000 7fbfac800640 1 -- 172.21.15.19:0/4168427232 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fbfa40685c0 con 0x7fbfa40b1530 2024-03-28T13:09:49.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.438+0000 7ff9eac00640 1 -- 172.21.15.19:0/1389208935 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff9c406c600 msgr2=0x7ff9c406eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.438+0000 7ff9eac00640 1 --2- 172.21.15.19:0/1389208935 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff9c406c600 0x7ff9c406eac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc008620 tx=0x7ff9e40ab000 comp rx=0 tx=0).stop 2024-03-28T13:09:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.438+0000 7ff9eac00640 1 -- 172.21.15.19:0/1389208935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9dc03e830 msgr2=0x7ff9dc039e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.438+0000 7ff9eac00640 1 --2- 172.21.15.19:0/1389208935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9dc03e830 0x7ff9dc039e70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800ebe0 tx=0x7ff9d800c520 comp rx=0 tx=0).stop 2024-03-28T13:09:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.440+0000 7ff9eac00640 1 -- 172.21.15.19:0/1389208935 shutdown_connections 2024-03-28T13:09:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.440+0000 7ff9eac00640 1 --2- 172.21.15.19:0/1389208935 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff9c406c600 0x7ff9c406eac0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.440+0000 7ff9eac00640 1 --2- 172.21.15.19:0/1389208935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9dc0e30b0 0x7ff9dc04ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.440+0000 7ff9eac00640 1 --2- 172.21.15.19:0/1389208935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9dc054120 0x7ff9dc04e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.440+0000 7ff9eac00640 1 --2- 172.21.15.19:0/1389208935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9dc03e830 0x7ff9dc039e70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.440+0000 7ff9eac00640 1 -- 172.21.15.19:0/1389208935 >> 172.21.15.19:0/1389208935 conn(0x7ff9dc0559d0 msgr2=0x7ff9dc0e0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.441+0000 7ff9eac00640 1 -- 172.21.15.19:0/1389208935 shutdown_connections 2024-03-28T13:09:49.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.441+0000 7ff9eac00640 1 -- 172.21.15.19:0/1389208935 wait complete. 2024-03-28T13:09:49.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.442+0000 7fbf9e000640 1 -- 172.21.15.19:0/4168427232 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbfa0059c90 con 0x7fbfa40b1530 2024-03-28T13:09:49.444 INFO:teuthology.orchestra.run.smithi019.stdout:68719476744 2024-03-28T13:09:49.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.447+0000 7fbfac800640 1 -- 172.21.15.19:0/4168427232 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbf8806c580 msgr2=0x7fbf8806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.447+0000 7fbfac800640 1 --2- 172.21.15.19:0/4168427232 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbf8806c580 0x7fbf8806ea40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4152a30 tx=0x7fbf98009040 comp rx=0 tx=0).stop 2024-03-28T13:09:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.447+0000 7fbfac800640 1 -- 172.21.15.19:0/4168427232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40b1530 msgr2=0x7fbfa40998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.447+0000 7fbfac800640 1 --2- 172.21.15.19:0/4168427232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40b1530 0x7fbfa40998e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000c7e0 tx=0x7fbfa000cca0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.447+0000 7fbfac800640 1 -- 172.21.15.19:0/4168427232 shutdown_connections 2024-03-28T13:09:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.447+0000 7fbfac800640 1 --2- 172.21.15.19:0/4168427232 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbf8806c580 0x7fbf8806ea40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.447+0000 7fbfac800640 1 --2- 172.21.15.19:0/4168427232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfa415de50 0x7fbfa40aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.447+0000 7fbfac800640 1 --2- 172.21.15.19:0/4168427232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfa40b3b70 0x7fbfa409d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.447+0000 7fbfac800640 1 --2- 172.21.15.19:0/4168427232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfa40b1530 0x7fbfa40998e0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbfa000c7e0 tx=0x7fbfa000cca0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.447+0000 7fbfac800640 1 -- 172.21.15.19:0/4168427232 >> 172.21.15.19:0/4168427232 conn(0x7fbfa4072a00 msgr2=0x7fbfa4073180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:49.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.451+0000 7fbfac800640 1 -- 172.21.15.19:0/4168427232 shutdown_connections 2024-03-28T13:09:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.451+0000 7f4c08400640 1 -- 172.21.15.19:0/280835463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f4c000685c0 con 0x7f4c000b3b70 2024-03-28T13:09:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.451+0000 7fbfac800640 1 -- 172.21.15.19:0/4168427232 wait complete. 2024-03-28T13:09:49.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.452+0000 7f4bfd600640 1 -- 172.21.15.19:0/280835463 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4bec058c70 con 0x7f4c000b3b70 2024-03-28T13:09:49.452 INFO:teuthology.orchestra.run.smithi019.stdout:60129542150 2024-03-28T13:09:49.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.455+0000 7f4c08400640 1 -- 172.21.15.19:0/280835463 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4be006c730 msgr2=0x7f4be006ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.455+0000 7f4c08400640 1 --2- 172.21.15.19:0/280835463 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4be006c730 0x7f4be006ebf0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4bf400de80 tx=0x7f4bf400f040 comp rx=0 tx=0).stop 2024-03-28T13:09:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.455+0000 7f4c08400640 1 -- 172.21.15.19:0/280835463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000b3b70 msgr2=0x7f4c0009dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.455+0000 7f4c08400640 1 --2- 172.21.15.19:0/280835463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000b3b70 0x7f4c0009dc60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00c3c0 tx=0x7f4bec00c880 comp rx=0 tx=0).stop 2024-03-28T13:09:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.456+0000 7f4c08400640 1 -- 172.21.15.19:0/280835463 shutdown_connections 2024-03-28T13:09:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.456+0000 7f4c08400640 1 --2- 172.21.15.19:0/280835463 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4be006c730 0x7f4be006ebf0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.456+0000 7f4c08400640 1 --2- 172.21.15.19:0/280835463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c0015de50 0x7f4c001ecbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.456+0000 7f4c08400640 1 --2- 172.21.15.19:0/280835463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4c000b3b70 0x7f4c0009dc60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.456+0000 7f4c08400640 1 --2- 172.21.15.19:0/280835463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c000b1530 0x7f4c0009a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.456+0000 7f4c08400640 1 -- 172.21.15.19:0/280835463 >> 172.21.15.19:0/280835463 conn(0x7f4c00072a00 msgr2=0x7f4c000cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:49.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.459+0000 7f4c08400640 1 -- 172.21.15.19:0/280835463 shutdown_connections 2024-03-28T13:09:49.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.459+0000 7f4c08400640 1 -- 172.21.15.19:0/280835463 wait complete. 2024-03-28T13:09:49.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.461+0000 7f0711000640 1 -- 172.21.15.19:0/917104379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f070c06cd20 con 0x7f070c1533d0 2024-03-28T13:09:49.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.462+0000 7f0702000640 1 -- 172.21.15.19:0/917104379 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f06f8059bf0 con 0x7f070c1533d0 2024-03-28T13:09:49.464 INFO:teuthology.orchestra.run.smithi019.stdout:68719476742 2024-03-28T13:09:49.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.466+0000 7ff37aa00640 1 -- 172.21.15.19:0/2511116790 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff37415fa70 con 0x7ff374073520 2024-03-28T13:09:49.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.467+0000 7ff36be00640 1 -- 172.21.15.19:0/2511116790 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff37415fa70 con 0x7ff374073520 2024-03-28T13:09:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.469+0000 7f0711000640 1 -- 172.21.15.19:0/917104379 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f06ec06c600 msgr2=0x7f06ec06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.469+0000 7f0711000640 1 --2- 172.21.15.19:0/917104379 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f06ec06c600 0x7f06ec06eac0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06fc007fd0 tx=0x7f06fc009040 comp rx=0 tx=0).stop 2024-03-28T13:09:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.469+0000 7f0711000640 1 -- 172.21.15.19:0/917104379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f070c1533d0 msgr2=0x7f070c14f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.469+0000 7f0711000640 1 --2- 172.21.15.19:0/917104379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f070c1533d0 0x7f070c14f3a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f06f800b540 tx=0x7f06f800ba00 comp rx=0 tx=0).stop 2024-03-28T13:09:49.471 INFO:teuthology.orchestra.run.smithi019.stdout:64424509446 2024-03-28T13:09:49.472 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476742 for osd.0 2024-03-28T13:09:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.471+0000 7f0711000640 1 -- 172.21.15.19:0/917104379 shutdown_connections 2024-03-28T13:09:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.471+0000 7f0711000640 1 --2- 172.21.15.19:0/917104379 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f06ec06c600 0x7f06ec06eac0 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f06fc007fd0 tx=0x7f06fc009040 comp rx=0 tx=0).stop 2024-03-28T13:09:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.471+0000 7f0711000640 1 --2- 172.21.15.19:0/917104379 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f070c15c830 0x7f070c15ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.471+0000 7f0711000640 1 --2- 172.21.15.19:0/917104379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f070c1533d0 0x7f070c14f3a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.471+0000 7f0711000640 1 --2- 172.21.15.19:0/917104379 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06f0009ef0 0x7f070c1f9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.471+0000 7f0711000640 1 -- 172.21.15.19:0/917104379 >> 172.21.15.19:0/917104379 conn(0x7f070c072a00 msgr2=0x7f070c0cf690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.471+0000 7f0711000640 1 -- 172.21.15.19:0/917104379 shutdown_connections 2024-03-28T13:09:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.471+0000 7f0711000640 1 -- 172.21.15.19:0/917104379 wait complete. 2024-03-28T13:09:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.472+0000 7ff369600640 1 -- 172.21.15.19:0/2511116790 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff35006c560 msgr2=0x7ff35006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.472+0000 7ff369600640 1 --2- 172.21.15.19:0/2511116790 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff35006c560 0x7ff35006ea20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff37414e350 tx=0x7ff364008040 comp rx=0 tx=0).stop 2024-03-28T13:09:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.472+0000 7ff369600640 1 -- 172.21.15.19:0/2511116790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff374073520 msgr2=0x7ff37415fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.472+0000 7ff369600640 1 --2- 172.21.15.19:0/2511116790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff374073520 0x7ff37415fd90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff36c036040 tx=0x7ff36c004a40 comp rx=0 tx=0).stop 2024-03-28T13:09:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.472+0000 7ff369600640 1 -- 172.21.15.19:0/2511116790 shutdown_connections 2024-03-28T13:09:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.472+0000 7ff369600640 1 --2- 172.21.15.19:0/2511116790 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff35006c560 0x7ff35006ea20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.472+0000 7ff369600640 1 --2- 172.21.15.19:0/2511116790 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3740b3460 0x7ff37415e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.472+0000 7ff369600640 1 --2- 172.21.15.19:0/2511116790 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3740b1530 0x7ff37414d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.472+0000 7ff369600640 1 --2- 172.21.15.19:0/2511116790 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff374073520 0x7ff37415fd90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.472+0000 7ff369600640 1 -- 172.21.15.19:0/2511116790 >> 172.21.15.19:0/2511116790 conn(0x7ff374072a00 msgr2=0x7ff3740b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.472+0000 7ff369600640 1 -- 172.21.15.19:0/2511116790 shutdown_connections 2024-03-28T13:09:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.472+0000 7ff369600640 1 -- 172.21.15.19:0/2511116790 wait complete. 2024-03-28T13:09:49.487 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.7 2024-03-28T13:09:49.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476744 for osd.3 2024-03-28T13:09:49.496 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476742 for osd.1 2024-03-28T13:09:49.502 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.5 2024-03-28T13:09:49.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.537+0000 7f9dd6000640 1 -- 172.21.15.19:0/557240121 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9dd00b5750 con 0x7f9dd00b69c0 2024-03-28T13:09:49.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.538+0000 7f9dccc00640 1 -- 172.21.15.19:0/557240121 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9dbc07b510 con 0x7f9dd00b69c0 2024-03-28T13:09:49.538 INFO:teuthology.orchestra.run.smithi019.stdout:68719476742 2024-03-28T13:09:49.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.540+0000 7f9dd6000640 1 -- 172.21.15.19:0/557240121 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9db006c530 msgr2=0x7f9db006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.540+0000 7f9dd6000640 1 --2- 172.21.15.19:0/557240121 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9db006c530 0x7f9db006e9f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0072c70 tx=0x7f9dc4009040 comp rx=0 tx=0).stop 2024-03-28T13:09:49.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.540+0000 7f9dd6000640 1 -- 172.21.15.19:0/557240121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dd00b69c0 msgr2=0x7f9dd01fb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.540+0000 7f9dd6000640 1 --2- 172.21.15.19:0/557240121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dd00b69c0 0x7f9dd01fb090 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc0098c0 tx=0x7f9dbc0046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.540+0000 7f9dd6000640 1 -- 172.21.15.19:0/557240121 shutdown_connections 2024-03-28T13:09:49.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.540+0000 7f9dd6000640 1 --2- 172.21.15.19:0/557240121 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9db006c530 0x7f9db006e9f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.540+0000 7f9dd6000640 1 --2- 172.21.15.19:0/557240121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dd015e4c0 0x7f9dd00b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.540+0000 7f9dd6000640 1 --2- 172.21.15.19:0/557240121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dd00d37b0 0x7f9dd01f9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.540+0000 7f9dd6000640 1 --2- 172.21.15.19:0/557240121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dd00b69c0 0x7f9dd01fb090 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.540+0000 7f9dd6000640 1 -- 172.21.15.19:0/557240121 >> 172.21.15.19:0/557240121 conn(0x7f9dd00676a0 msgr2=0x7f9dd015ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:49.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.540+0000 7f9dd6000640 1 -- 172.21.15.19:0/557240121 shutdown_connections 2024-03-28T13:09:49.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.540+0000 7f9dd6000640 1 -- 172.21.15.19:0/557240121 wait complete. 2024-03-28T13:09:49.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476742 for osd.2 2024-03-28T13:09:49.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.572+0000 7f3474800640 1 -- 172.21.15.19:0/1634608569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f346c0d3180 con 0x7f346c15e860 2024-03-28T13:09:49.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.573+0000 7f3466000640 1 -- 172.21.15.19:0/1634608569 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f346806a620 con 0x7f346c15e860 2024-03-28T13:09:49.573 INFO:teuthology.orchestra.run.smithi019.stdout:64424509446 2024-03-28T13:09:49.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.575+0000 7f3474800640 1 -- 172.21.15.19:0/1634608569 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f345006c580 msgr2=0x7f345006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.575+0000 7f3474800640 1 --2- 172.21.15.19:0/1634608569 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f345006c580 0x7f345006ea40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f34600090b0 tx=0x7f346003c040 comp rx=0 tx=0).stop 2024-03-28T13:09:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.575+0000 7f3474800640 1 -- 172.21.15.19:0/1634608569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c15e860 msgr2=0x7f346c0cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.575+0000 7f3474800640 1 --2- 172.21.15.19:0/1634608569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c15e860 0x7f346c0cd840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3468009f00 tx=0x7f346800bdf0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.575+0000 7f3474800640 1 -- 172.21.15.19:0/1634608569 shutdown_connections 2024-03-28T13:09:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.575+0000 7f3474800640 1 --2- 172.21.15.19:0/1634608569 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f345006c580 0x7f345006ea40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.575+0000 7f3474800640 1 --2- 172.21.15.19:0/1634608569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c15e860 0x7f346c0cd840 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.575+0000 7f3474800640 1 --2- 172.21.15.19:0/1634608569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f346c0d4ae0 0x7f346c0ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.575+0000 7f3474800640 1 --2- 172.21.15.19:0/1634608569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f346c06da30 0x7f346c1f94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.575+0000 7f3474800640 1 -- 172.21.15.19:0/1634608569 >> 172.21.15.19:0/1634608569 conn(0x7f346c09d7d0 msgr2=0x7f346c0d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.575+0000 7f3474800640 1 -- 172.21.15.19:0/1634608569 shutdown_connections 2024-03-28T13:09:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.576+0000 7f3474800640 1 -- 172.21.15.19:0/1634608569 wait complete. 2024-03-28T13:09:49.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2024-03-28T13:09:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.713+0000 7fbe48000640 1 -- 172.21.15.19:0/4021341922 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fbe400685c0 con 0x7fbe4009b0e0 2024-03-28T13:09:49.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.720+0000 7fbe3d600640 1 -- 172.21.15.19:0/4021341922 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbe3804f020 con 0x7fbe4009b0e0 2024-03-28T13:09:49.720 INFO:teuthology.orchestra.run.smithi019.stdout:64424509445 2024-03-28T13:09:49.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.723+0000 7fbe48000640 1 -- 172.21.15.19:0/4021341922 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbe2006c5b0 msgr2=0x7fbe2006ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.723+0000 7fbe48000640 1 --2- 172.21.15.19:0/4021341922 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbe2006c5b0 0x7fbe2006ea70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c000ff0 tx=0x7fbe2c009040 comp rx=0 tx=0).stop 2024-03-28T13:09:49.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.723+0000 7fbe48000640 1 -- 172.21.15.19:0/4021341922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe4009b0e0 msgr2=0x7fbe4009c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:49.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.723+0000 7fbe48000640 1 --2- 172.21.15.19:0/4021341922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe4009b0e0 0x7fbe4009c620 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbe38007700 tx=0x7fbe38004ab0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.723+0000 7fbe48000640 1 -- 172.21.15.19:0/4021341922 shutdown_connections 2024-03-28T13:09:49.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.723+0000 7fbe48000640 1 --2- 172.21.15.19:0/4021341922 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbe2006c5b0 0x7fbe2006ea70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.723+0000 7fbe48000640 1 --2- 172.21.15.19:0/4021341922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe4009b0e0 0x7fbe4009c620 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.723+0000 7fbe48000640 1 --2- 172.21.15.19:0/4021341922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe401539f0 0x7fbe4009aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.723+0000 7fbe48000640 1 --2- 172.21.15.19:0/4021341922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe28007540 0x7fbe401f0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:49.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.723+0000 7fbe48000640 1 -- 172.21.15.19:0/4021341922 >> 172.21.15.19:0/4021341922 conn(0x7fbe4006b750 msgr2=0x7fbe4006ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:49.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.723+0000 7fbe48000640 1 -- 172.21.15.19:0/4021341922 shutdown_connections 2024-03-28T13:09:49.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:49.723+0000 7fbe48000640 1 -- 172.21.15.19:0/4021341922 wait complete. 2024-03-28T13:09:49.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.6 2024-03-28T13:09:50.473 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-28T13:09:50.488 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-28T13:09:50.491 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-28T13:09:50.497 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-28T13:09:50.504 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-28T13:09:50.565 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-28T13:09:50.593 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-28T13:09:50.743 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-28T13:09:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.807+0000 7f97a6200640 1 Processor -- start 2024-03-28T13:09:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.807+0000 7f97a6200640 1 -- start start 2024-03-28T13:09:50.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.809+0000 7f97a6200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f97a00d0990 0x7f97a01670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.809+0000 7f97a6200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a0163ac0 0x7f97a0071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.809+0000 7f97a6200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97a0071d60 con 0x7f97a0162920 2024-03-28T13:09:50.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.809+0000 7f97a6200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f97a0163f30 con 0x7f97a00d0990 2024-03-28T13:09:50.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.809+0000 7f97a6200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f97a01665d0 con 0x7f97a0163ac0 2024-03-28T13:09:50.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.811+0000 7fe305200640 1 Processor -- start 2024-03-28T13:09:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.811+0000 7f979ea00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f97a00d0990 0x7f97a01670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.811+0000 7fe305200640 1 -- start start 2024-03-28T13:09:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.811+0000 7f979fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a0163ac0 0x7f97a0071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.811+0000 7f979fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a0163ac0 0x7f97a0071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:54926/0 (socket says 172.21.15.19:54926) 2024-03-28T13:09:50.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.811+0000 7f979fe00640 1 -- 172.21.15.19:0/3512941675 learned_addr learned my addr 172.21.15.19:0/3512941675 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:50.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.812+0000 7f979e000640 1 -- 172.21.15.19:0/3512941675 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3159087440 0 0) 0x7f97a0071d60 con 0x7f97a0162920 2024-03-28T13:09:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.812+0000 7f979e000640 1 -- 172.21.15.19:0/3512941675 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9784003520 con 0x7f97a0162920 2024-03-28T13:09:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.812+0000 7f979ea00640 1 -- 172.21.15.19:0/3512941675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a0163ac0 msgr2=0x7f97a0071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.812+0000 7f979ea00640 1 --2- 172.21.15.19:0/3512941675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a0163ac0 0x7f97a0071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.812+0000 7f979ea00640 1 -- 172.21.15.19:0/3512941675 >> v1:172.21.15.188:6789/0 conn(0x7f97a0162920 legacy=0x7f97a00d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.813+0000 7f979ea00640 1 -- 172.21.15.19:0/3512941675 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a01668b0 con 0x7f97a00d0990 2024-03-28T13:09:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.813+0000 7fe305200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3000b3740 0x7fe3000af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.813+0000 7fe305200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe300073520 0x7fe3000afde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.814+0000 7fe2fd400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3000b3740 0x7fe3000af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.814+0000 7f3a30000640 1 Processor -- start 2024-03-28T13:09:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.814+0000 7fe2fd400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3000b3740 0x7fe3000af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:54930/0 (socket says 172.21.15.19:54930) 2024-03-28T13:09:50.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.814+0000 7f3a30000640 1 -- start start 2024-03-28T13:09:50.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.815+0000 7f979ea00640 1 --2- 172.21.15.19:0/3512941675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a00d0990 0x7f97a01670b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9790009d90 tx=0x7f979002be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31e923f8bacd8779 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.815+0000 7f0c92000640 1 Processor -- start 2024-03-28T13:09:50.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.816+0000 7fecd7400640 1 Processor -- start 2024-03-28T13:09:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.816+0000 7fecd7400640 1 -- start start 2024-03-28T13:09:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.816+0000 7fe2fde00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe30015e950 0x7fe3000b3030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43018/0 (socket says 172.21.15.19:43018) 2024-03-28T13:09:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.816+0000 7fe2fe800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe300073520 0x7fe3000afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.816+0000 7fe2fe800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe300073520 0x7fe3000afde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44280/0 (socket says 172.21.15.19:44280) 2024-03-28T13:09:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.816+0000 7fe2fe800640 1 -- 172.21.15.19:0/3736611712 learned_addr learned my addr 172.21.15.19:0/3736611712 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.816+0000 7fe305200640 1 -- 172.21.15.19:0/3736611712 --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3000af160 con 0x7fe30015e950 2024-03-28T13:09:50.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.816+0000 7fe305200640 1 -- 172.21.15.19:0/3736611712 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fe30015ed30 con 0x7fe300073520 2024-03-28T13:09:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.816+0000 7fe305200640 1 -- 172.21.15.19:0/3736611712 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe3000ad0b0 con 0x7fe3000b3740 2024-03-28T13:09:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.817+0000 7f979e000640 1 -- 172.21.15.19:0/3512941675 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790039070 con 0x7f97a00d0990 2024-03-28T13:09:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.817+0000 7f3a30000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3a280d43d0 0x7f3a280d47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.817+0000 7f979e000640 1 -- 172.21.15.19:0/3512941675 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f979002cbe0 con 0x7f97a00d0990 2024-03-28T13:09:50.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.817+0000 7f979e000640 1 -- 172.21.15.19:0/3512941675 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790034440 con 0x7f97a00d0990 2024-03-28T13:09:50.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.815+0000 7f0c92000640 1 -- start start 2024-03-28T13:09:50.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.817+0000 7f0c92000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0d0e70 0x7f0c8c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.817+0000 7f0c92000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c8c0cf6c0 con 0x7f0c8c0d2110 2024-03-28T13:09:50.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.817+0000 7f0c92000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c8c0cf9a0 con 0x7f0c8c15f6e0 2024-03-28T13:09:50.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.817+0000 7f0c92000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0c8c0cfc80 con 0x7f0c8c0d0e70 2024-03-28T13:09:50.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.818+0000 7fecd7400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fecd00d0e70 0x7fecd00cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.818+0000 7f97a6200640 1 -- 172.21.15.19:0/3512941675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a00d0990 msgr2=0x7f97a01670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.818+0000 7f97a6200640 1 --2- 172.21.15.19:0/3512941675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a00d0990 0x7f97a01670b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9790009d90 tx=0x7f979002be30 comp rx=0 tx=0).stop 2024-03-28T13:09:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.818+0000 7f97a6200640 1 -- 172.21.15.19:0/3512941675 shutdown_connections 2024-03-28T13:09:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.818+0000 7f97a6200640 1 --2- 172.21.15.19:0/3512941675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a00d0990 0x7f97a01670b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.818+0000 7f97a6200640 1 --2- 172.21.15.19:0/3512941675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a0163ac0 0x7f97a0071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.818+0000 7f97a6200640 1 -- 172.21.15.19:0/3512941675 >> 172.21.15.19:0/3512941675 conn(0x7f97a006b750 msgr2=0x7f97a006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.819+0000 7f97a6200640 1 -- 172.21.15.19:0/3512941675 shutdown_connections 2024-03-28T13:09:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.819+0000 7f97a6200640 1 -- 172.21.15.19:0/3512941675 wait complete. 2024-03-28T13:09:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.817+0000 7fe2fe800640 1 -- 172.21.15.19:0/3736611712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3000b3740 msgr2=0x7fe3000af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.817+0000 7fe2fe800640 1 --2- 172.21.15.19:0/3736611712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3000b3740 0x7fe3000af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.817+0000 7fe2fe800640 1 -- 172.21.15.19:0/3736611712 >> v1:172.21.15.19:6789/0 conn(0x7fe30015e950 legacy=0x7fe3000b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.817+0000 7fe2fe800640 1 -- 172.21.15.19:0/3736611712 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3000ad390 con 0x7fe300073520 2024-03-28T13:09:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.817+0000 7fe2fde00640 1 --1- 172.21.15.19:0/3736611712 >> v1:172.21.15.19:6789/0 conn(0x7fe30015e950 0x7fe3000b3030 :-1 s=CLOSED pgs=0 cs=0 l=0).handle_server_banner_and_identify state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-28T13:09:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.818+0000 7fe2fd400640 1 --2- 172.21.15.19:0/3736611712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3000b3740 0x7fe3000af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-03-28T13:09:50.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.818+0000 7fecd7400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fecd015f6e0 0x7fecd00d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.819+0000 7f0c8be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0c8c0d2110 0x7f0c8c0d25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43020/0 (socket says 172.21.15.19:43020) 2024-03-28T13:09:50.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.819+0000 7f0c8be00640 1 -- 172.21.15.19:0/498178263 learned_addr learned my addr 172.21.15.19:0/498178263 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.819+0000 7fecd4a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fecd00d0e70 0x7fecd00cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.819+0000 7fecd4a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fecd00d0e70 0x7fecd00cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44282/0 (socket says 172.21.15.19:44282) 2024-03-28T13:09:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.819+0000 7fecd4a00640 1 -- 172.21.15.19:0/3313776156 learned_addr learned my addr 172.21.15.19:0/3313776156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.819+0000 7f3a30000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3a280cf6b0 0x7f3a280cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.819+0000 7f3a30000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a280d4d20 con 0x7f3a2815e860 2024-03-28T13:09:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.819+0000 7f3a30000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3a280cf220 con 0x7f3a280cf6b0 2024-03-28T13:09:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.819+0000 7f3a30000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3a2806dfa0 con 0x7f3a280d43d0 2024-03-28T13:09:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.820+0000 7f3a2d600640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f3a2815e860 0x7f3a2806d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:42632/0 (socket says 172.21.15.19:42632) 2024-03-28T13:09:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.820+0000 7f3a2d600640 1 -- 172.21.15.19:0/3485082479 learned_addr learned my addr 172.21.15.19:0/3485082479 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.820+0000 7f0c8b400640 1 --2- 172.21.15.19:0/498178263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0d0e70 0x7f0c8c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.819+0000 7fe2fe800640 1 --2- 172.21.15.19:0/3736611712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe300073520 0x7fe3000afde0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0013db0 tx=0x7fe2f0034100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d8f2c326f015021 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.820+0000 7fecd7400640 1 -- 172.21.15.19:0/3313776156 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecd00d1790 con 0x7fecd01621d0 2024-03-28T13:09:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.820+0000 7fe2fca00640 1 -- 172.21.15.19:0/3736611712 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0041070 con 0x7fe300073520 2024-03-28T13:09:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.820+0000 7fecd7400640 1 -- 172.21.15.19:0/3313776156 --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fecd00cf6c0 con 0x7fecd015f6e0 2024-03-28T13:09:50.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.820+0000 7fecd7400640 1 -- 172.21.15.19:0/3313776156 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fecd00cf9a0 con 0x7fecd00d0e70 2024-03-28T13:09:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.821+0000 7feccfe00640 1 --2- 172.21.15.19:0/3313776156 >> v2:172.21.15.19:3300/0 conn(0x7fecd015f6e0 0x7fecd00d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.821+0000 7f0c8a000640 1 -- 172.21.15.19:0/498178263 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 988808558 0 0) 0x7f0c8c0cf6c0 con 0x7f0c8c0d2110 2024-03-28T13:09:50.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.821+0000 7f0c8a000640 1 -- 172.21.15.19:0/498178263 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c70003520 con 0x7f0c8c0d2110 2024-03-28T13:09:50.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.820+0000 7fe2fca00640 1 -- 172.21.15.19:0/3736611712 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f003c440 con 0x7fe300073520 2024-03-28T13:09:50.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.821+0000 7fe305200640 1 -- 172.21.15.19:0/3736611712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe300073520 msgr2=0x7fe3000afde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.822+0000 7fe2fca00640 1 -- 172.21.15.19:0/3736611712 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0041070 con 0x7fe300073520 2024-03-28T13:09:50.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.822+0000 7fe305200640 1 --2- 172.21.15.19:0/3736611712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe300073520 0x7fe3000afde0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0013db0 tx=0x7fe2f0034100 comp rx=0 tx=0).stop 2024-03-28T13:09:50.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.822+0000 7fecd4a00640 1 -- 172.21.15.19:0/3313776156 >> v2:172.21.15.19:3300/0 conn(0x7fecd015f6e0 msgr2=0x7fecd00d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:50.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.822+0000 7f97a6200640 1 Processor -- start 2024-03-28T13:09:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.820+0000 7f3a2e000640 1 --2- 172.21.15.19:0/3485082479 >> v2:172.21.15.19:3300/0 conn(0x7f3a280cf6b0 0x7f3a280cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.821+0000 7f3a2cc00640 1 --2- 172.21.15.19:0/3485082479 >> v2:172.21.15.188:3300/0 conn(0x7f3a280d43d0 0x7f3a280d47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.822+0000 7f0c8a000640 1 -- 172.21.15.19:0/498178263 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 908771624 0 0) 0x7f0c8c0cf9a0 con 0x7f0c8c15f6e0 2024-03-28T13:09:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.822+0000 7fe305200640 1 -- 172.21.15.19:0/3736611712 shutdown_connections 2024-03-28T13:09:50.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.822+0000 7f0c8a000640 1 -- 172.21.15.19:0/498178263 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c8c0cf6c0 con 0x7f0c8c15f6e0 2024-03-28T13:09:50.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.822+0000 7f0c8a000640 1 -- 172.21.15.19:0/498178263 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4094917896 0 0) 0x7f0c70003520 con 0x7f0c8c0d2110 2024-03-28T13:09:50.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7f0c8a000640 1 -- 172.21.15.19:0/498178263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0d0e70 msgr2=0x7f0c8c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:50.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7fe305200640 1 --2- 172.21.15.19:0/3736611712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe300073520 0x7fe3000afde0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7fe305200640 1 --2- 172.21.15.19:0/3736611712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3000b3740 0x7fe3000af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7f3a27e00640 1 -- 172.21.15.19:0/3485082479 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2371114379 0 0) 0x7f3a18001110 con 0x7f3a2815e860 2024-03-28T13:09:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7fe305200640 1 -- 172.21.15.19:0/3736611712 >> 172.21.15.19:0/3736611712 conn(0x7fe300072a00 msgr2=0x7fe300072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7f0c8a000640 1 --2- 172.21.15.19:0/498178263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0d0e70 0x7f0c8c0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7f0c8a000640 1 -- 172.21.15.19:0/498178263 >> v1:172.21.15.188:6789/0 conn(0x7f0c8c15f6e0 legacy=0x7f0c8c0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7f3a27e00640 1 -- 172.21.15.19:0/3485082479 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a10003520 con 0x7f3a2815e860 2024-03-28T13:09:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.822+0000 7fecd4a00640 1 --2- 172.21.15.19:0/3313776156 >> v2:172.21.15.19:3300/0 conn(0x7fecd015f6e0 0x7fecd00d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.822+0000 7fecd4a00640 1 -- 172.21.15.19:0/3313776156 >> v1:172.21.15.188:6789/0 conn(0x7fecd01621d0 legacy=0x7fecd0162660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.822+0000 7fecd4a00640 1 -- 172.21.15.19:0/3313776156 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd00cfc80 con 0x7fecd00d0e70 2024-03-28T13:09:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7feccfe00640 1 --2- 172.21.15.19:0/3313776156 >> v2:172.21.15.19:3300/0 conn(0x7fecd015f6e0 0x7fecd00d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7f0c8a000640 1 -- 172.21.15.19:0/498178263 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8c0cff60 con 0x7f0c8c0d2110 2024-03-28T13:09:50.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7fe305200640 1 -- 172.21.15.19:0/3736611712 shutdown_connections 2024-03-28T13:09:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7fe305200640 1 -- 172.21.15.19:0/3736611712 wait complete. 2024-03-28T13:09:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.824+0000 7f97a6200640 1 -- start start 2024-03-28T13:09:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7f0c8a000640 1 -- 172.21.15.19:0/498178263 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f0c840019f0 con 0x7f0c8c0d2110 2024-03-28T13:09:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7f0c8a000640 1 -- 172.21.15.19:0/498178263 >> v1:172.21.15.19:6789/0 conn(0x7f0c8c0d2110 legacy=0x7f0c8c0d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7f0c8a000640 1 --2- 172.21.15.19:0/498178263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c70003b70 0x7f0c70003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7f0c8a000640 1 --2- 172.21.15.19:0/498178263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c70004800 0x7f0c70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.824+0000 7f0c8a000640 1 --2- 172.21.15.19:0/498178263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c700051c0 0x7f0c7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.824+0000 7f0c8a000640 1 -- 172.21.15.19:0/498178263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c0cfc80 con 0x7f0c70004800 2024-03-28T13:09:50.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.824+0000 7f0c8a000640 1 -- 172.21.15.19:0/498178263 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7000c090 con 0x7f0c700051c0 2024-03-28T13:09:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.824+0000 7f0c8a000640 1 -- 172.21.15.19:0/498178263 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0c7000c290 con 0x7f0c70003b70 2024-03-28T13:09:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.824+0000 7fecd4a00640 1 --2- 172.21.15.19:0/3313776156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecd00d0e70 0x7fecd00cf050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fecd008cf80 tx=0x7fecc002b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec8d2aef9dcabc82 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.824+0000 7f0c8b400640 1 --2- 172.21.15.19:0/498178263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8c0d0e70 0x7f0c8c0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:09:50.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.825+0000 7f0c8be00640 1 --2- 172.21.15.19:0/498178263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c700051c0 0x7f0c7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.823+0000 7f3a2cc00640 1 -- 172.21.15.19:0/3485082479 >> v2:172.21.15.19:3300/0 conn(0x7f3a280cf6b0 msgr2=0x7f3a280cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:50.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.825+0000 7feccf400640 1 -- 172.21.15.19:0/3313776156 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0035020 con 0x7fecd00d0e70 2024-03-28T13:09:50.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.825+0000 7f0c8b400640 1 --2- 172.21.15.19:0/498178263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c70003b70 0x7f0c70003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.824+0000 7f3a2cc00640 1 --2- 172.21.15.19:0/3485082479 >> v2:172.21.15.19:3300/0 conn(0x7f3a280cf6b0 0x7f3a280cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.825+0000 7feccf400640 1 -- 172.21.15.19:0/3313776156 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc0031880 con 0x7fecd00d0e70 2024-03-28T13:09:50.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.825+0000 7f97a6200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a00d0990 0x7f97a009c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.825+0000 7f97a6200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0162920 0x7f97a009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.825+0000 7f97a6200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a0163ac0 0x7f97a0144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.826+0000 7f97a6200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97a0072040 con 0x7f97a0162920 2024-03-28T13:09:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.826+0000 7f97a6200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97a0162d00 con 0x7f97a00d0990 2024-03-28T13:09:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.826+0000 7f97a6200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f97a00721c0 con 0x7f97a0163ac0 2024-03-28T13:09:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.824+0000 7f3a2cc00640 1 -- 172.21.15.19:0/3485082479 >> v1:172.21.15.188:6789/0 conn(0x7f3a2815e860 legacy=0x7f3a2806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.824+0000 7f3a2cc00640 1 -- 172.21.15.19:0/3485082479 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2806e240 con 0x7f3a280d43d0 2024-03-28T13:09:50.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.826+0000 7fe305200640 1 Processor -- start 2024-03-28T13:09:50.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.826+0000 7f3a2e000640 1 --2- 172.21.15.19:0/3485082479 >> v2:172.21.15.19:3300/0 conn(0x7f3a280cf6b0 0x7f3a280cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:09:50.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.826+0000 7f0c8aa00640 1 --2- 172.21.15.19:0/498178263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c70004800 0x7f0c70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.827+0000 7fecd7400640 1 -- 172.21.15.19:0/3313776156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecd00d0e70 msgr2=0x7fecd00cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.827+0000 7fecd7400640 1 --2- 172.21.15.19:0/3313776156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecd00d0e70 0x7fecd00cf050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fecd008cf80 tx=0x7fecc002b920 comp rx=0 tx=0).stop 2024-03-28T13:09:50.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.826+0000 7f979f400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a00d0990 0x7f97a009c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.827+0000 7fecd7400640 1 -- 172.21.15.19:0/3313776156 shutdown_connections 2024-03-28T13:09:50.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.827+0000 7fecd7400640 1 --2- 172.21.15.19:0/3313776156 >> v2:172.21.15.19:3300/0 conn(0x7fecd015f6e0 0x7fecd00d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.827+0000 7fecd7400640 1 --2- 172.21.15.19:0/3313776156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecd00d0e70 0x7fecd00cf050 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.827+0000 7fecd7400640 1 -- 172.21.15.19:0/3313776156 >> 172.21.15.19:0/3313776156 conn(0x7fecd008bfb0 msgr2=0x7fecd00d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:50.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.827+0000 7f3a2cc00640 1 --2- 172.21.15.19:0/3485082479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a280d43d0 0x7f3a280d47e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a28162160 tx=0x7f3a1c02b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e02d4b77fbc1d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.827+0000 7f0c8be00640 1 -- 172.21.15.19:0/498178263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c70003b70 msgr2=0x7f0c70003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:50.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.827+0000 7f979f400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a00d0990 0x7f97a009c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44308/0 (socket says 172.21.15.19:44308) 2024-03-28T13:09:50.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.827+0000 7f979f400640 1 -- 172.21.15.19:0/4035285331 learned_addr learned my addr 172.21.15.19:0/4035285331 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:50.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.826+0000 7fe305200640 1 -- start start 2024-03-28T13:09:50.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.828+0000 7f979ea00640 1 --2- 172.21.15.19:0/4035285331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0162920 0x7f97a009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.828+0000 7fe305200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe300073520 0x7fe30015fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.828+0000 7fe305200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3000b3740 0x7fe3000992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.828+0000 7fe305200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe30015e950 0x7fe300201ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.828+0000 7f3a27e00640 1 -- 172.21.15.19:0/3485082479 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c0317c0 con 0x7f3a280d43d0 2024-03-28T13:09:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.828+0000 7fe2fde00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe300073520 0x7fe30015fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.828+0000 7fe2fde00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe300073520 0x7fe30015fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44312/0 (socket says 172.21.15.19:44312) 2024-03-28T13:09:50.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.828+0000 7f3a27e00640 1 -- 172.21.15.19:0/3485082479 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a1c037070 con 0x7f3a280d43d0 2024-03-28T13:09:50.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.829+0000 7fe2fde00640 1 -- 172.21.15.19:0/2640148347 learned_addr learned my addr 172.21.15.19:0/2640148347 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:50.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.829+0000 7f3a27e00640 1 -- 172.21.15.19:0/3485082479 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c031bc0 con 0x7f3a280d43d0 2024-03-28T13:09:50.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.829+0000 7fe305200640 1 -- 172.21.15.19:0/2640148347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3000adc30 con 0x7fe3000b3740 2024-03-28T13:09:50.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.829+0000 7fe2fe800640 1 --2- 172.21.15.19:0/2640148347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe30015e950 0x7fe300201ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.829+0000 7fe2fd400640 1 --2- 172.21.15.19:0/2640148347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3000b3740 0x7fe3000992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.827+0000 7f0c8be00640 1 --2- 172.21.15.19:0/498178263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c70003b70 0x7f0c70003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.827+0000 7f0c8be00640 1 -- 172.21.15.19:0/498178263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c70004800 msgr2=0x7f0c70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.827+0000 7f0c8be00640 1 --2- 172.21.15.19:0/498178263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c70004800 0x7f0c70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.827+0000 7f0c8be00640 1 -- 172.21.15.19:0/498178263 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c70003520 con 0x7f0c700051c0 2024-03-28T13:09:50.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.828+0000 7f0c8aa00640 1 --2- 172.21.15.19:0/498178263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c70004800 0x7f0c70004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:09:50.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.829+0000 7fe305200640 1 -- 172.21.15.19:0/2640148347 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe3000ad950 con 0x7fe300073520 2024-03-28T13:09:50.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.829+0000 7fe305200640 1 -- 172.21.15.19:0/2640148347 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe3000b0320 con 0x7fe30015e950 2024-03-28T13:09:50.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.829+0000 7f979ea00640 1 -- 172.21.15.19:0/4035285331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a0163ac0 msgr2=0x7f97a0144b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-28T13:09:50.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.829+0000 7f979ea00640 1 --2- 172.21.15.19:0/4035285331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a0163ac0 0x7f97a0144b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.829+0000 7f979ea00640 1 -- 172.21.15.19:0/4035285331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a00d0990 msgr2=0x7f97a009c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.829+0000 7f979ea00640 1 --2- 172.21.15.19:0/4035285331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a00d0990 0x7f97a009c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.829+0000 7f979ea00640 1 -- 172.21.15.19:0/4035285331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a009e000 con 0x7f97a0162920 2024-03-28T13:09:50.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.829+0000 7f979f400640 1 --2- 172.21.15.19:0/4035285331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a00d0990 0x7f97a009c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:09:50.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.830+0000 7f3a30000640 1 -- 172.21.15.19:0/3485082479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a280d43d0 msgr2=0x7f3a280d47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.830+0000 7f3a30000640 1 --2- 172.21.15.19:0/3485082479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a280d43d0 0x7f3a280d47e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a28162160 tx=0x7f3a1c02b860 comp rx=0 tx=0).stop 2024-03-28T13:09:50.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.830+0000 7f3a30000640 1 -- 172.21.15.19:0/3485082479 shutdown_connections 2024-03-28T13:09:50.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.830+0000 7f3a30000640 1 --2- 172.21.15.19:0/3485082479 >> v2:172.21.15.19:3300/0 conn(0x7f3a280cf6b0 0x7f3a280cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.830+0000 7f3a30000640 1 --2- 172.21.15.19:0/3485082479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a280d43d0 0x7f3a280d47e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.830+0000 7f3a30000640 1 -- 172.21.15.19:0/3485082479 >> 172.21.15.19:0/3485082479 conn(0x7f3a2809d7d0 msgr2=0x7f3a2808bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:50.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.830+0000 7f0c8b400640 1 --2- 172.21.15.19:0/498178263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c70003b70 0x7f0c70003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:09:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.830+0000 7f979ea00640 1 --2- 172.21.15.19:0/4035285331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0162920 0x7f97a009dac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9790033000 tx=0x7f9790004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.831+0000 7f0c8be00640 1 --2- 172.21.15.19:0/498178263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c700051c0 0x7f0c7000b9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400a760 tx=0x7f0c70005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f78b4c45564f6ca5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.831+0000 7fecd7400640 1 -- 172.21.15.19:0/3313776156 shutdown_connections 2024-03-28T13:09:50.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.831+0000 7fecd7400640 1 -- 172.21.15.19:0/3313776156 wait complete. 2024-03-28T13:09:50.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.830+0000 7fe2fd400640 1 -- 172.21.15.19:0/2640148347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe30015e950 msgr2=0x7fe300201ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.830+0000 7fe2fd400640 1 --2- 172.21.15.19:0/2640148347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe30015e950 0x7fe300201ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.830+0000 7fe2fd400640 1 -- 172.21.15.19:0/2640148347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe300073520 msgr2=0x7fe30015fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.830+0000 7fe2fd400640 1 --2- 172.21.15.19:0/2640148347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe300073520 0x7fe30015fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.830+0000 7fe2fd400640 1 -- 172.21.15.19:0/2640148347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe300160570 con 0x7fe3000b3740 2024-03-28T13:09:50.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.831+0000 7fe2fd400640 1 --2- 172.21.15.19:0/2640148347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3000b3740 0x7fe3000992b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800cea0 tx=0x7fe2f800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.831+0000 7fe2fde00640 1 --2- 172.21.15.19:0/2640148347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe300073520 0x7fe30015fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:50.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.831+0000 7f9797400640 1 -- 172.21.15.19:0/4035285331 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790044030 con 0x7f97a0162920 2024-03-28T13:09:50.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.831+0000 7f9797400640 1 -- 172.21.15.19:0/4035285331 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f979002c9e0 con 0x7f97a0162920 2024-03-28T13:09:50.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.831+0000 7f9797400640 1 -- 172.21.15.19:0/4035285331 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9790039040 con 0x7f97a0162920 2024-03-28T13:09:50.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.831+0000 7fe2f6000640 1 -- 172.21.15.19:0/2640148347 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f801a070 con 0x7fe3000b3740 2024-03-28T13:09:50.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.831+0000 7f0c8a000640 1 -- 172.21.15.19:0/498178263 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8400a350 con 0x7f0c700051c0 2024-03-28T13:09:50.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.831+0000 7f3a30000640 1 -- 172.21.15.19:0/3485082479 shutdown_connections 2024-03-28T13:09:50.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.832+0000 7f3a30000640 1 -- 172.21.15.19:0/3485082479 wait complete. 2024-03-28T13:09:50.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.831+0000 7f97a6200640 1 -- 172.21.15.19:0/4035285331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a009ca20 con 0x7f97a0162920 2024-03-28T13:09:50.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.832+0000 7fe2f6000640 1 -- 172.21.15.19:0/2640148347 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f800d840 con 0x7fe3000b3740 2024-03-28T13:09:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.832+0000 7f0c8a000640 1 -- 172.21.15.19:0/498178263 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c84034650 con 0x7f0c700051c0 2024-03-28T13:09:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.833+0000 7f0c92000640 1 -- 172.21.15.19:0/498178263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c700051c0 msgr2=0x7f0c7000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.833+0000 7f0c92000640 1 --2- 172.21.15.19:0/498178263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c700051c0 0x7f0c7000b9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0c8400a760 tx=0x7f0c70005830 comp rx=0 tx=0).stop 2024-03-28T13:09:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.833+0000 7f0c8be00640 1 -- 172.21.15.19:0/498178263 reap_dead start 2024-03-28T13:09:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.833+0000 7f0c92000640 1 -- 172.21.15.19:0/498178263 shutdown_connections 2024-03-28T13:09:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.833+0000 7f0c92000640 1 -- 172.21.15.19:0/498178263 >> 172.21.15.19:0/498178263 conn(0x7f0c8c08bfb0 msgr2=0x7f0c8c0d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.834+0000 7f0c92000640 1 -- 172.21.15.19:0/498178263 shutdown_connections 2024-03-28T13:09:50.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.834+0000 7f0c92000640 1 -- 172.21.15.19:0/498178263 wait complete. 2024-03-28T13:09:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.835+0000 7fecd7400640 1 Processor -- start 2024-03-28T13:09:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.832+0000 7fe2f6000640 1 -- 172.21.15.19:0/2640148347 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f800dbf0 con 0x7fe3000b3740 2024-03-28T13:09:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.832+0000 7fe305200640 1 -- 172.21.15.19:0/2640148347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe300160850 con 0x7fe3000b3740 2024-03-28T13:09:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.833+0000 7fe305200640 1 -- 172.21.15.19:0/2640148347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe300160e10 con 0x7fe3000b3740 2024-03-28T13:09:50.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.835+0000 7fecd7400640 1 -- start start 2024-03-28T13:09:50.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.832+0000 7f97a6200640 1 -- 172.21.15.19:0/4035285331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a009cfe0 con 0x7f97a0162920 2024-03-28T13:09:50.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.835+0000 7f3a30000640 1 Processor -- start 2024-03-28T13:09:50.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.837+0000 7fecd7400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecd00d0e70 0x7fecd00cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.837+0000 7fecd7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecd015f6e0 0x7fecd014d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.837+0000 7fecd7400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecd01621d0 0x7fecd00ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.837+0000 7fecd7400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fecd015fb40 con 0x7fecd015f6e0 2024-03-28T13:09:50.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.837+0000 7fecd7400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fecd00d0240 con 0x7fecd00d0e70 2024-03-28T13:09:50.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.837+0000 7fecd7400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fecd015e910 con 0x7fecd01621d0 2024-03-28T13:09:50.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.837+0000 7fecd4a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecd00d0e70 0x7fecd00cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.837+0000 7fecd4a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecd00d0e70 0x7fecd00cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44316/0 (socket says 172.21.15.19:44316) 2024-03-28T13:09:50.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.837+0000 7fecd4a00640 1 -- 172.21.15.19:0/3752430121 learned_addr learned my addr 172.21.15.19:0/3752430121 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:50.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.838+0000 7fecd5400640 1 --2- 172.21.15.19:0/3752430121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecd01621d0 0x7fecd00ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.838+0000 7feccfe00640 1 --2- 172.21.15.19:0/3752430121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecd015f6e0 0x7fecd014d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.838+0000 7f0c92000640 1 Processor -- start 2024-03-28T13:09:50.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.838+0000 7f0c92000640 1 -- start start 2024-03-28T13:09:50.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.838+0000 7fecd4a00640 1 -- 172.21.15.19:0/3752430121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecd01621d0 msgr2=0x7fecd00ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.838+0000 7fecd4a00640 1 --2- 172.21.15.19:0/3752430121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecd01621d0 0x7fecd00ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.838+0000 7fecd4a00640 1 -- 172.21.15.19:0/3752430121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecd015f6e0 msgr2=0x7fecd014d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.838+0000 7fecd4a00640 1 --2- 172.21.15.19:0/3752430121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecd015f6e0 0x7fecd014d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.838+0000 7fecd4a00640 1 -- 172.21.15.19:0/3752430121 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd00cbbd0 con 0x7fecd00d0e70 2024-03-28T13:09:50.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.839+0000 7f9797400640 1 -- 172.21.15.19:0/4035285331 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9790047070 con 0x7f97a0162920 2024-03-28T13:09:50.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.839+0000 7fecd4a00640 1 --2- 172.21.15.19:0/3752430121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecd00d0e70 0x7fecd00cb530 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fecc0000c00 tx=0x7fecc00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.839+0000 7fecccc00640 1 -- 172.21.15.19:0/3752430121 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0040070 con 0x7fecd00d0e70 2024-03-28T13:09:50.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.839+0000 7fecccc00640 1 -- 172.21.15.19:0/3752430121 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc0030e40 con 0x7fecd00d0e70 2024-03-28T13:09:50.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.839+0000 7fecccc00640 1 -- 172.21.15.19:0/3752430121 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0035040 con 0x7fecd00d0e70 2024-03-28T13:09:50.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.839+0000 7f97a6200640 1 -- 172.21.15.19:0/4035285331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a00685c0 con 0x7f97a0162920 2024-03-28T13:09:50.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.839+0000 7fecd7400640 1 -- 172.21.15.19:0/3752430121 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd014ddb0 con 0x7fecd00d0e70 2024-03-28T13:09:50.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.840+0000 7fecd7400640 1 -- 172.21.15.19:0/3752430121 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd00cc150 con 0x7fecd00d0e70 2024-03-28T13:09:50.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.841+0000 7f0c92000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c700051c0 0x7f0c8c0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.839+0000 7fe305200640 1 -- 172.21.15.19:0/2640148347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d8005290 con 0x7fe3000b3740 2024-03-28T13:09:50.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.842+0000 7f0c92000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c0ca230 0x7f0c8c1f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.842+0000 7f0c92000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8c0ccf40 0x7f0c8c0cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.842+0000 7f0c8b400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c700051c0 0x7f0c8c0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.842+0000 7f0c8b400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c700051c0 0x7f0c8c0ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:54956/0 (socket says 172.21.15.19:54956) 2024-03-28T13:09:50.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.840+0000 7f9797400640 1 --2- 172.21.15.19:0/4035285331 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f977c06c600 0x7f977c06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.840+0000 7f9797400640 1 -- 172.21.15.19:0/4035285331 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f97900adda0 con 0x7f97a0162920 2024-03-28T13:09:50.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.842+0000 7f979f400640 1 --2- 172.21.15.19:0/4035285331 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f977c06c600 0x7f977c06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.843+0000 7fe2f6000640 1 -- 172.21.15.19:0/2640148347 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe2f801d360 con 0x7fe3000b3740 2024-03-28T13:09:50.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.843+0000 7f0c8b400640 1 -- 172.21.15.19:0/2890947834 learned_addr learned my addr 172.21.15.19:0/2890947834 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:50.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.843+0000 7f0c92000640 1 -- 172.21.15.19:0/2890947834 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c15e910 con 0x7f0c8c0ca230 2024-03-28T13:09:50.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.844+0000 7fecccc00640 1 -- 172.21.15.19:0/3752430121 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fecc004b060 con 0x7fecd00d0e70 2024-03-28T13:09:50.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.843+0000 7f0c92000640 1 -- 172.21.15.19:0/2890947834 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8c15fb40 con 0x7f0c8c0ccf40 2024-03-28T13:09:50.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.843+0000 7f0c92000640 1 -- 172.21.15.19:0/2890947834 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0c8c15ea90 con 0x7f0c700051c0 2024-03-28T13:09:50.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.844+0000 7f0c8be00640 1 --2- 172.21.15.19:0/2890947834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8c0ccf40 0x7f0c8c0cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.845+0000 7fe2f6000640 1 --2- 172.21.15.19:0/2640148347 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2e006c600 0x7fe2e006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.845+0000 7fecd7400640 1 -- 172.21.15.19:0/3752430121 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd0068cd0 con 0x7fecd00d0e70 2024-03-28T13:09:50.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.845+0000 7f0c8aa00640 1 --2- 172.21.15.19:0/2890947834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c0ca230 0x7f0c8c1f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.841+0000 7f3a30000640 1 -- start start 2024-03-28T13:09:50.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.845+0000 7fe2f6000640 1 -- 172.21.15.19:0/2640148347 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe2f808be10 con 0x7fe3000b3740 2024-03-28T13:09:50.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.846+0000 7f0c8b400640 1 -- 172.21.15.19:0/2890947834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8c0ccf40 msgr2=0x7f0c8c0cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.846+0000 7fecccc00640 1 --2- 172.21.15.19:0/3752430121 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fecb006c600 0x7fecb006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.846+0000 7f0c8b400640 1 --2- 172.21.15.19:0/2890947834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8c0ccf40 0x7f0c8c0cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.846+0000 7f0c8b400640 1 -- 172.21.15.19:0/2890947834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c0ca230 msgr2=0x7f0c8c1f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.846+0000 7fecccc00640 1 -- 172.21.15.19:0/3752430121 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fecc00ad630 con 0x7fecd00d0e70 2024-03-28T13:09:50.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.846+0000 7feccfe00640 1 --2- 172.21.15.19:0/3752430121 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fecb006c600 0x7fecb006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.846+0000 7f3a30000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a280cf6b0 0x7f3a28155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.846+0000 7f3a30000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a2815e860 0x7f3a280b7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.847+0000 7f3a30000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a280b7560 0x7f3a280b88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.847+0000 7f3a30000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2806dc00 con 0x7f3a2815e860 2024-03-28T13:09:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.846+0000 7f0c8b400640 1 --2- 172.21.15.19:0/2890947834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c0ca230 0x7f0c8c1f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.846+0000 7f0c8b400640 1 -- 172.21.15.19:0/2890947834 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c8c0c8010 con 0x7f0c700051c0 2024-03-28T13:09:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.847+0000 7f0c8aa00640 1 --2- 172.21.15.19:0/2890947834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c0ca230 0x7f0c8c1f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:50.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.847+0000 7feccfe00640 1 --2- 172.21.15.19:0/3752430121 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fecb006c600 0x7fecb006eac0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fecd0161d40 tx=0x7fecbc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.847+0000 7f3a30000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3a280d2c50 con 0x7f3a280cf6b0 2024-03-28T13:09:50.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.847+0000 7f3a2e000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a280b7560 0x7f3a280b88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.848+0000 7f3a2e000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a280b7560 0x7f3a280b88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:54962/0 (socket says 172.21.15.19:54962) 2024-03-28T13:09:50.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.848+0000 7f3a2e000640 1 -- 172.21.15.19:0/759771677 learned_addr learned my addr 172.21.15.19:0/759771677 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:50.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.848+0000 7f3a30000640 1 -- 172.21.15.19:0/759771677 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3a2806dd80 con 0x7f3a280b7560 2024-03-28T13:09:50.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.848+0000 7f3a2d600640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a280cf6b0 0x7f3a28155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.847+0000 7fe2fde00640 1 --2- 172.21.15.19:0/2640148347 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2e006c600 0x7fe2e006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.848+0000 7f0c8b400640 1 --2- 172.21.15.19:0/2890947834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c700051c0 0x7f0c8c0ca7b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c011c00 tx=0x7f0c7c00f610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.848+0000 7f3a2cc00640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a2815e860 0x7f3a280b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.848+0000 7f0c83400640 1 -- 172.21.15.19:0/2890947834 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c019070 con 0x7f0c700051c0 2024-03-28T13:09:50.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.849+0000 7f3a2e000640 1 -- 172.21.15.19:0/759771677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a280cf6b0 msgr2=0x7f3a28155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.849+0000 7f3a2e000640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a280cf6b0 0x7f3a28155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.849+0000 7f3a2e000640 1 -- 172.21.15.19:0/759771677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a2815e860 msgr2=0x7f3a280b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.849+0000 7f3a2e000640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a2815e860 0x7f3a280b7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.849+0000 7f3a2e000640 1 -- 172.21.15.19:0/759771677 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a280b7ac0 con 0x7f3a280b7560 2024-03-28T13:09:50.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.849+0000 7f3a2d600640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a280cf6b0 0x7f3a28155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:50.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.849+0000 7f0c83400640 1 -- 172.21.15.19:0/2890947834 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c7c00fe40 con 0x7f0c700051c0 2024-03-28T13:09:50.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.849+0000 7f0c83400640 1 -- 172.21.15.19:0/2890947834 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c004230 con 0x7f0c700051c0 2024-03-28T13:09:50.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.849+0000 7f3a2cc00640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a2815e860 0x7f3a280b7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:50.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.850+0000 7f3a2e000640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a280b7560 0x7f3a280b88d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009f00 tx=0x7f3a2000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.849+0000 7f0c92000640 1 -- 172.21.15.19:0/2890947834 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c8c0c82b0 con 0x7f0c700051c0 2024-03-28T13:09:50.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.849+0000 7f0c92000640 1 -- 172.21.15.19:0/2890947834 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c8c0c8870 con 0x7f0c700051c0 2024-03-28T13:09:50.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.850+0000 7f979f400640 1 --2- 172.21.15.19:0/4035285331 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f977c06c600 0x7f977c06eac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9788000f60 tx=0x7f9788009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.850+0000 7f3a25600640 1 -- 172.21.15.19:0/759771677 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a20018030 con 0x7f3a280b7560 2024-03-28T13:09:50.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.850+0000 7f3a25600640 1 -- 172.21.15.19:0/759771677 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a2000ce50 con 0x7f3a280b7560 2024-03-28T13:09:50.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.850+0000 7f3a25600640 1 -- 172.21.15.19:0/759771677 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a20003bb0 con 0x7f3a280b7560 2024-03-28T13:09:50.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.851+0000 7f3a30000640 1 -- 172.21.15.19:0/759771677 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a280b7da0 con 0x7f3a280b7560 2024-03-28T13:09:50.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.851+0000 7f3a30000640 1 -- 172.21.15.19:0/759771677 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a280b8360 con 0x7f3a280b7560 2024-03-28T13:09:50.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.853+0000 7fe2fde00640 1 --2- 172.21.15.19:0/2640148347 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2e006c600 0x7fe2e006eac0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe30014d980 tx=0x7fe2ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.852+0000 7f9797400640 1 -- 172.21.15.19:0/4035285331 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f979007c430 con 0x7f97a0162920 2024-03-28T13:09:50.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.853+0000 7f0c92000640 1 -- 172.21.15.19:0/2890947834 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c8c068cd0 con 0x7f0c700051c0 2024-03-28T13:09:50.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.854+0000 7f0c83400640 1 -- 172.21.15.19:0/2890947834 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0c7c01c070 con 0x7f0c700051c0 2024-03-28T13:09:50.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.855+0000 7f3a30000640 1 -- 172.21.15.19:0/759771677 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a280685c0 con 0x7f3a280b7560 2024-03-28T13:09:50.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.856+0000 7f3a25600640 1 -- 172.21.15.19:0/759771677 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3a2001b070 con 0x7f3a280b7560 2024-03-28T13:09:50.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.855+0000 7f0c83400640 1 --2- 172.21.15.19:0/2890947834 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c6c06c560 0x7f0c6c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.855+0000 7f0c83400640 1 -- 172.21.15.19:0/2890947834 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0c7c08bda0 con 0x7f0c700051c0 2024-03-28T13:09:50.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.856+0000 7f0c8aa00640 1 --2- 172.21.15.19:0/2890947834 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c6c06c560 0x7f0c6c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.858+0000 7f0c8aa00640 1 --2- 172.21.15.19:0/2890947834 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c6c06c560 0x7f0c6c06ea20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0d5500 tx=0x7f0c78009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.858+0000 7f3a25600640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a0c06c530 0x7f3a0c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.858+0000 7f3a25600640 1 -- 172.21.15.19:0/759771677 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3a2008c210 con 0x7f3a280b7560 2024-03-28T13:09:50.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.859+0000 7f3a2d600640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a0c06c530 0x7f3a0c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.860+0000 7f3a2d600640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a0c06c530 0x7f3a0c06e9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3a18000f60 tx=0x7f3a18007b90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.862+0000 7fe2f6000640 1 -- 172.21.15.19:0/2640148347 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe2f8059e50 con 0x7fe3000b3740 2024-03-28T13:09:50.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.862+0000 7fecccc00640 1 -- 172.21.15.19:0/3752430121 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fecc007bba0 con 0x7fecd00d0e70 2024-03-28T13:09:50.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.872+0000 7f0c83400640 1 -- 172.21.15.19:0/2890947834 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0c7c059e00 con 0x7f0c700051c0 2024-03-28T13:09:50.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.876+0000 7f004c600640 1 Processor -- start 2024-03-28T13:09:50.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.876+0000 7f004c600640 1 -- start start 2024-03-28T13:09:50.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.878+0000 7f004c600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f00441670b0 0x7f0044163ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.878+0000 7f004c600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0044071d60 con 0x7f00440d0990 2024-03-28T13:09:50.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.878+0000 7f004c600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00441665d0 con 0x7f0044162920 2024-03-28T13:09:50.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.878+0000 7f004c600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f00441674d0 con 0x7f00441670b0 2024-03-28T13:09:50.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.879+0000 7f0049c00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f0044162920 0x7f00440d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:42644/0 (socket says 172.21.15.19:42644) 2024-03-28T13:09:50.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.879+0000 7f0049c00640 1 -- 172.21.15.19:0/2788289677 learned_addr learned my addr 172.21.15.19:0/2788289677 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:50.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.880+0000 7f004a600640 1 --2- 172.21.15.19:0/2788289677 >> v2:172.21.15.19:3300/0 conn(0x7f00441670b0 0x7f0044163ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.881+0000 7f0043e00640 1 -- 172.21.15.19:0/2788289677 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2535701733 0 0) 0x7f00441665d0 con 0x7f0044162920 2024-03-28T13:09:50.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.881+0000 7f0043e00640 1 -- 172.21.15.19:0/2788289677 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f002c003520 con 0x7f0044162920 2024-03-28T13:09:50.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.881+0000 7f0043e00640 1 -- 172.21.15.19:0/2788289677 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2575833590 0 0) 0x7f002c003520 con 0x7f0044162920 2024-03-28T13:09:50.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.881+0000 7f0043e00640 1 -- 172.21.15.19:0/2788289677 >> v2:172.21.15.19:3300/0 conn(0x7f00441670b0 msgr2=0x7f0044163ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.884+0000 7f0043e00640 1 --2- 172.21.15.19:0/2788289677 >> v2:172.21.15.19:3300/0 conn(0x7f00441670b0 0x7f0044163ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.884+0000 7f0043e00640 1 -- 172.21.15.19:0/2788289677 >> v1:172.21.15.19:6789/0 conn(0x7f00440d0990 legacy=0x7f00440ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.884+0000 7f0043e00640 1 -- 172.21.15.19:0/2788289677 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00441668b0 con 0x7f0044162920 2024-03-28T13:09:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.884+0000 7f0043e00640 1 -- 172.21.15.19:0/2788289677 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f0038001670 con 0x7f0044162920 2024-03-28T13:09:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.884+0000 7f0043e00640 1 -- 172.21.15.19:0/2788289677 >> v1:172.21.15.188:6789/0 conn(0x7f0044162920 legacy=0x7f00440d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.884+0000 7f0043e00640 1 --2- 172.21.15.19:0/2788289677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f002c0039b0 0x7f002c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.884+0000 7f0043e00640 1 --2- 172.21.15.19:0/2788289677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f002c004800 0x7f002c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.884+0000 7f0043e00640 1 --2- 172.21.15.19:0/2788289677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f002c0051c0 0x7f002c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.884+0000 7f0043e00640 1 -- 172.21.15.19:0/2788289677 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00441674d0 con 0x7f002c0039b0 2024-03-28T13:09:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.884+0000 7f0043e00640 1 -- 172.21.15.19:0/2788289677 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f002c00c040 con 0x7f002c004800 2024-03-28T13:09:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.884+0000 7f0043e00640 1 -- 172.21.15.19:0/2788289677 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f002c00c240 con 0x7f002c0051c0 2024-03-28T13:09:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.885+0000 7f004a600640 1 --2- 172.21.15.19:0/2788289677 >> v2:172.21.15.19:3300/0 conn(0x7f00441670b0 0x7f0044163ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:09:50.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.885+0000 7f0049c00640 1 --2- 172.21.15.19:0/2788289677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f002c0039b0 0x7f002c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.886+0000 7f0049c00640 1 -- 172.21.15.19:0/2788289677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f002c0051c0 msgr2=0x7f002c00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-28T13:09:50.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.886+0000 7f0049c00640 1 --2- 172.21.15.19:0/2788289677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f002c0051c0 0x7f002c00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.886+0000 7f0049c00640 1 -- 172.21.15.19:0/2788289677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f002c004800 msgr2=0x7f002c004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-28T13:09:50.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.886+0000 7f0049c00640 1 --2- 172.21.15.19:0/2788289677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f002c004800 0x7f002c004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.886+0000 7f0049c00640 1 -- 172.21.15.19:0/2788289677 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00441668b0 con 0x7f002c0039b0 2024-03-28T13:09:50.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.888+0000 7f0049c00640 1 --2- 172.21.15.19:0/2788289677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f002c0039b0 0x7f002c003db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f003800c290 tx=0x7f002c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48fc1c141b6ccd6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.888+0000 7f004a600640 1 -- 172.21.15.19:0/2788289677 reap_dead start 2024-03-28T13:09:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.889+0000 7f0043e00640 1 -- 172.21.15.19:0/2788289677 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038007d10 con 0x7f002c0039b0 2024-03-28T13:09:50.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.889+0000 7f0043e00640 1 -- 172.21.15.19:0/2788289677 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0038030070 con 0x7f002c0039b0 2024-03-28T13:09:50.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.890+0000 7f3a25600640 1 -- 172.21.15.19:0/759771677 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3a20010040 con 0x7f3a280b7560 2024-03-28T13:09:50.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.890+0000 7f0043e00640 1 -- 172.21.15.19:0/2788289677 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003802cd80 con 0x7f002c0039b0 2024-03-28T13:09:50.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.891+0000 7f004c600640 1 -- 172.21.15.19:0/2788289677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f002c0039b0 msgr2=0x7f002c003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.891+0000 7f004c600640 1 --2- 172.21.15.19:0/2788289677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f002c0039b0 0x7f002c003db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f003800c290 tx=0x7f002c005830 comp rx=0 tx=0).stop 2024-03-28T13:09:50.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.891+0000 7f004c600640 1 -- 172.21.15.19:0/2788289677 shutdown_connections 2024-03-28T13:09:50.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.891+0000 7f004c600640 1 --2- 172.21.15.19:0/2788289677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f002c0039b0 0x7f002c003db0 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f003800c290 tx=0x7f002c005830 comp rx=0 tx=0).stop 2024-03-28T13:09:50.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.891+0000 7f004c600640 1 -- 172.21.15.19:0/2788289677 >> 172.21.15.19:0/2788289677 conn(0x7f004406b750 msgr2=0x7f004406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:50.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.892+0000 7f004c600640 1 -- 172.21.15.19:0/2788289677 shutdown_connections 2024-03-28T13:09:50.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.892+0000 7f004c600640 1 -- 172.21.15.19:0/2788289677 wait complete. 2024-03-28T13:09:50.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.894+0000 7f004c600640 1 Processor -- start 2024-03-28T13:09:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.895+0000 7f004c600640 1 -- start start 2024-03-28T13:09:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.896+0000 7f004c600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00441565d0 0x7f00440b32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.896+0000 7f004c600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f004415ad80 0x7f004415d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.896+0000 7f004c600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00441569b0 0x7f004415d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.897+0000 7f0049c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00441565d0 0x7f00440b32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.897+0000 7f0049c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00441565d0 0x7f00440b32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50940/0 (socket says 172.21.15.19:50940) 2024-03-28T13:09:50.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.897+0000 7f0049c00640 1 -- 172.21.15.19:0/3212322358 learned_addr learned my addr 172.21.15.19:0/3212322358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:50.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.897+0000 7f004c600640 1 -- 172.21.15.19:0/3212322358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0044072040 con 0x7f00441565d0 2024-03-28T13:09:50.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.897+0000 7f0049200640 1 --2- 172.21.15.19:0/3212322358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f004415ad80 0x7f004415d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.897+0000 7f004a600640 1 --2- 172.21.15.19:0/3212322358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00441569b0 0x7f004415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.899+0000 7f004c600640 1 -- 172.21.15.19:0/3212322358 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0044162d00 con 0x7f00441569b0 2024-03-28T13:09:50.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.899+0000 7f004c600640 1 -- 172.21.15.19:0/3212322358 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f00440721c0 con 0x7f004415ad80 2024-03-28T13:09:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.900+0000 7f0049c00640 1 -- 172.21.15.19:0/3212322358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f004415ad80 msgr2=0x7f004415d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.900+0000 7f0049c00640 1 --2- 172.21.15.19:0/3212322358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f004415ad80 0x7f004415d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.900+0000 7f0049c00640 1 -- 172.21.15.19:0/3212322358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00441569b0 msgr2=0x7f004415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.900+0000 7f0049c00640 1 --2- 172.21.15.19:0/3212322358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00441569b0 0x7f004415d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.900+0000 7f0049c00640 1 -- 172.21.15.19:0/3212322358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004415b210 con 0x7f00441565d0 2024-03-28T13:09:50.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.900+0000 7f0049200640 1 --2- 172.21.15.19:0/3212322358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f004415ad80 0x7f004415d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:09:50.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.904+0000 7f0049c00640 1 --2- 172.21.15.19:0/3212322358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00441565d0 0x7f00440b32b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f003800c290 tx=0x7f0038001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.904+0000 7f0041600640 1 -- 172.21.15.19:0/3212322358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038045070 con 0x7f00441565d0 2024-03-28T13:09:50.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.904+0000 7f004c600640 1 -- 172.21.15.19:0/3212322358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004415b4f0 con 0x7f00441565d0 2024-03-28T13:09:50.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.904+0000 7f004c600640 1 -- 172.21.15.19:0/3212322358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f004415bab0 con 0x7f00441565d0 2024-03-28T13:09:50.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.905+0000 7f0041600640 1 -- 172.21.15.19:0/3212322358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0038007de0 con 0x7f00441565d0 2024-03-28T13:09:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.905+0000 7f0041600640 1 -- 172.21.15.19:0/3212322358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0038045070 con 0x7f00441565d0 2024-03-28T13:09:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.906+0000 7f004c600640 1 -- 172.21.15.19:0/3212322358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00440685c0 con 0x7f00441565d0 2024-03-28T13:09:50.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.915+0000 7f0041600640 1 -- 172.21.15.19:0/3212322358 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0038004bf0 con 0x7f00441565d0 2024-03-28T13:09:50.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.920+0000 7f0041600640 1 --2- 172.21.15.19:0/3212322358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f002406c730 0x7f002406ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.920+0000 7f0041600640 1 -- 172.21.15.19:0/3212322358 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f00380acc80 con 0x7f00441565d0 2024-03-28T13:09:50.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.920+0000 7f0049200640 1 --2- 172.21.15.19:0/3212322358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f002406c730 0x7f002406ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.923+0000 7f0049200640 1 --2- 172.21.15.19:0/3212322358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f002406c730 0x7f002406ebf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f00440b5250 tx=0x7f0030009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.928+0000 7f0041600640 1 -- 172.21.15.19:0/3212322358 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f00380b3080 con 0x7f00441565d0 2024-03-28T13:09:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.937+0000 7f42c3e00640 1 Processor -- start 2024-03-28T13:09:50.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.937+0000 7f42c3e00640 1 -- start start 2024-03-28T13:09:50.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.939+0000 7f42c3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f42bc0d0600 0x7f42bc0d09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.940+0000 7f42c3e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f42bc06ea90 0x7f42bc06ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.940+0000 7f42c3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42bc0b4570 con 0x7f42bc0ccea0 2024-03-28T13:09:50.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.940+0000 7f42c3e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f42bc0cd9c0 con 0x7f42bc0d0600 2024-03-28T13:09:50.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.940+0000 7f42c3e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f42bc0cff20 con 0x7f42bc06ea90 2024-03-28T13:09:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.940+0000 7f42c1e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f42bc0ccea0 0x7f42bc0cd2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43034/0 (socket says 172.21.15.19:43034) 2024-03-28T13:09:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.940+0000 7f42c1e00640 1 -- 172.21.15.19:0/2310616594 learned_addr learned my addr 172.21.15.19:0/2310616594 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.941+0000 7f42c0a00640 1 --2- 172.21.15.19:0/2310616594 >> v2:172.21.15.188:3300/0 conn(0x7f42bc06ea90 0x7f42bc06ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.941+0000 7f42c1400640 1 --2- 172.21.15.19:0/2310616594 >> v2:172.21.15.19:3300/0 conn(0x7f42bc0d0600 0x7f42bc0d09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.941+0000 7f42c0a00640 1 -- 172.21.15.19:0/2310616594 >> v2:172.21.15.19:3300/0 conn(0x7f42bc0d0600 msgr2=0x7f42bc0d09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:50.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.941+0000 7f42c0a00640 1 --2- 172.21.15.19:0/2310616594 >> v2:172.21.15.19:3300/0 conn(0x7f42bc0d0600 0x7f42bc0d09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.941+0000 7f42c0a00640 1 -- 172.21.15.19:0/2310616594 >> v1:172.21.15.19:6789/0 conn(0x7f42bc0ccea0 legacy=0x7f42bc0cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.941+0000 7f42bbe00640 1 -- 172.21.15.19:0/2310616594 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 784124944 0 0) 0x7f42bc0b4570 con 0x7f42bc0ccea0 2024-03-28T13:09:50.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.942+0000 7f42c0a00640 1 -- 172.21.15.19:0/2310616594 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc0d0f20 con 0x7f42bc06ea90 2024-03-28T13:09:50.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.942+0000 7f42c1400640 1 --2- 172.21.15.19:0/2310616594 >> v2:172.21.15.19:3300/0 conn(0x7f42bc0d0600 0x7f42bc0d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:50.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.943+0000 7f42c0a00640 1 --2- 172.21.15.19:0/2310616594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42bc06ea90 0x7f42bc06ef40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0ce940 tx=0x7f42b002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de4c48cf4b52aefe server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.944+0000 7f42bbe00640 1 -- 172.21.15.19:0/2310616594 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0035020 con 0x7f42bc06ea90 2024-03-28T13:09:50.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.944+0000 7f42bbe00640 1 -- 172.21.15.19:0/2310616594 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b0031830 con 0x7f42bc06ea90 2024-03-28T13:09:50.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.944+0000 7f42bbe00640 1 -- 172.21.15.19:0/2310616594 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0031c00 con 0x7f42bc06ea90 2024-03-28T13:09:50.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.945+0000 7f42c3e00640 1 -- 172.21.15.19:0/2310616594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42bc06ea90 msgr2=0x7f42bc06ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.945+0000 7f42c3e00640 1 --2- 172.21.15.19:0/2310616594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42bc06ea90 0x7f42bc06ef40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0ce940 tx=0x7f42b002b8d0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.946+0000 7f42c3e00640 1 -- 172.21.15.19:0/2310616594 shutdown_connections 2024-03-28T13:09:50.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.946+0000 7f42c3e00640 1 --2- 172.21.15.19:0/2310616594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42bc06ea90 0x7f42bc06ef40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.946+0000 7f42c3e00640 1 --2- 172.21.15.19:0/2310616594 >> v2:172.21.15.19:3300/0 conn(0x7f42bc0d0600 0x7f42bc0d09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.948+0000 7f42c3e00640 1 -- 172.21.15.19:0/2310616594 >> 172.21.15.19:0/2310616594 conn(0x7f42bc06b570 msgr2=0x7f42bc06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.948+0000 7f42c3e00640 1 -- 172.21.15.19:0/2310616594 shutdown_connections 2024-03-28T13:09:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.948+0000 7f42c3e00640 1 -- 172.21.15.19:0/2310616594 wait complete. 2024-03-28T13:09:50.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.952+0000 7f42c3e00640 1 Processor -- start 2024-03-28T13:09:50.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.952+0000 7f42c3e00640 1 -- start start 2024-03-28T13:09:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.954+0000 7f42c3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42bc06ea90 0x7f42bc09aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.954+0000 7f42c3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc0ccea0 0x7f42bc1e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.958+0000 7f42c3e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42bc0d0600 0x7f42bc1557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.958+0000 7f42c3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc0b4af0 con 0x7f42bc0ccea0 2024-03-28T13:09:50.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.958+0000 7f42c3e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc0b4810 con 0x7f42bc0d0600 2024-03-28T13:09:50.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.958+0000 7f42c1400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42bc06ea90 0x7f42bc09aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.958+0000 7f42c1400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42bc06ea90 0x7f42bc09aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:54974/0 (socket says 172.21.15.19:54974) 2024-03-28T13:09:50.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.958+0000 7f42c1400640 1 -- 172.21.15.19:0/1384011226 learned_addr learned my addr 172.21.15.19:0/1384011226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:50.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.959+0000 7f42c0a00640 1 --2- 172.21.15.19:0/1384011226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc0ccea0 0x7f42bc1e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.959+0000 7f42c1e00640 1 --2- 172.21.15.19:0/1384011226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42bc0d0600 0x7f42bc1557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.959+0000 7f42c3e00640 1 -- 172.21.15.19:0/1384011226 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f42bc0b4dd0 con 0x7f42bc06ea90 2024-03-28T13:09:50.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.960+0000 7f42c0a00640 1 -- 172.21.15.19:0/1384011226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42bc06ea90 msgr2=0x7f42bc09aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.960+0000 7f42c0a00640 1 --2- 172.21.15.19:0/1384011226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42bc06ea90 0x7f42bc09aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.960+0000 7f42c0a00640 1 -- 172.21.15.19:0/1384011226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42bc0d0600 msgr2=0x7f42bc1557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:50.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.960+0000 7f42c0a00640 1 --2- 172.21.15.19:0/1384011226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42bc0d0600 0x7f42bc1557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:50.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.960+0000 7f42c0a00640 1 -- 172.21.15.19:0/1384011226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc09b120 con 0x7f42bc0ccea0 2024-03-28T13:09:50.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.961+0000 7f42c0a00640 1 --2- 172.21.15.19:0/1384011226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc0ccea0 0x7f42bc1e40b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f42b0033000 tx=0x7f42b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.962+0000 7f42b9600640 1 -- 172.21.15.19:0/1384011226 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0035050 con 0x7f42bc0ccea0 2024-03-28T13:09:50.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.962+0000 7f42b9600640 1 -- 172.21.15.19:0/1384011226 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b003ba70 con 0x7f42bc0ccea0 2024-03-28T13:09:50.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.962+0000 7f42b9600640 1 -- 172.21.15.19:0/1384011226 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b003be60 con 0x7f42bc0ccea0 2024-03-28T13:09:50.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.962+0000 7f42c3e00640 1 -- 172.21.15.19:0/1384011226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42bc09b400 con 0x7f42bc0ccea0 2024-03-28T13:09:50.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.962+0000 7f42c3e00640 1 -- 172.21.15.19:0/1384011226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42bc09b9c0 con 0x7f42bc0ccea0 2024-03-28T13:09:50.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.964+0000 7f42aea00640 1 -- 172.21.15.19:0/1384011226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4290005290 con 0x7f42bc0ccea0 2024-03-28T13:09:50.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.970+0000 7f42b9600640 1 -- 172.21.15.19:0/1384011226 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f42b0038020 con 0x7f42bc0ccea0 2024-03-28T13:09:50.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.972+0000 7f42b9600640 1 --2- 172.21.15.19:0/1384011226 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f429c06c600 0x7f429c06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:50.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.972+0000 7f42b9600640 1 -- 172.21.15.19:0/1384011226 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f42b00adce0 con 0x7f42bc0ccea0 2024-03-28T13:09:50.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.972+0000 7f42c1400640 1 --2- 172.21.15.19:0/1384011226 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f429c06c600 0x7f429c06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:50.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.973+0000 7f42c1400640 1 --2- 172.21.15.19:0/1384011226 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f429c06c600 0x7f429c06eac0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0b19b0 tx=0x7f42b400c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:50.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:50.984+0000 7f42b9600640 1 -- 172.21.15.19:0/1384011226 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f42b0040070 con 0x7f42bc0ccea0 2024-03-28T13:09:51.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.090+0000 7f5114400640 1 Processor -- start 2024-03-28T13:09:51.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.090+0000 7f5114400640 1 -- start start 2024-03-28T13:09:51.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.092+0000 7f5114400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f510c15e250 0x7f510c071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:51.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.092+0000 7f5114400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f510c0d0940 0x7f510c0b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:51.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.093+0000 7f5111000640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f510c161f80 0x7f510c0d0230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:42646/0 (socket says 172.21.15.19:42646) 2024-03-28T13:09:51.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.097+0000 7f5111a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f510c15e250 0x7f510c071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:51.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.097+0000 7f5111a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f510c15e250 0x7f510c071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44356/0 (socket says 172.21.15.19:44356) 2024-03-28T13:09:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.100+0000 7f5111000640 1 -- 172.21.15.19:0/435447381 learned_addr learned my addr 172.21.15.19:0/435447381 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.100+0000 7f5114400640 1 -- 172.21.15.19:0/435447381 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f510c071f20 con 0x7f510c161f80 2024-03-28T13:09:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.100+0000 7f5114400640 1 -- 172.21.15.19:0/435447381 --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f510c0adb60 con 0x7f510c0d0940 2024-03-28T13:09:51.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.100+0000 7f5114400640 1 -- 172.21.15.19:0/435447381 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f510c072200 con 0x7f510c15e250 2024-03-28T13:09:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.100+0000 7f5112400640 1 --2- 172.21.15.19:0/435447381 >> v2:172.21.15.19:3300/0 conn(0x7f510c0d0940 0x7f510c0b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:51.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.101+0000 7f510be00640 1 -- 172.21.15.19:0/435447381 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2759084237 0 0) 0x7f510c071f20 con 0x7f510c161f80 2024-03-28T13:09:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.101+0000 7f510be00640 1 -- 172.21.15.19:0/435447381 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50f4003520 con 0x7f510c161f80 2024-03-28T13:09:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.101+0000 7f5111a00640 1 -- 172.21.15.19:0/435447381 >> v2:172.21.15.19:3300/0 conn(0x7f510c0d0940 msgr2=0x7f510c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.101+0000 7f5111a00640 1 --2- 172.21.15.19:0/435447381 >> v2:172.21.15.19:3300/0 conn(0x7f510c0d0940 0x7f510c0b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.101+0000 7f5111a00640 1 -- 172.21.15.19:0/435447381 >> v1:172.21.15.188:6789/0 conn(0x7f510c161f80 legacy=0x7f510c0d0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.101+0000 7f5111a00640 1 -- 172.21.15.19:0/435447381 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c0724e0 con 0x7f510c15e250 2024-03-28T13:09:51.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.101+0000 7f5112400640 1 --2- 172.21.15.19:0/435447381 >> v2:172.21.15.19:3300/0 conn(0x7f510c0d0940 0x7f510c0b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:09:51.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.106+0000 7f5111a00640 1 --2- 172.21.15.19:0/435447381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f510c15e250 0x7f510c071660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f510c0adb30 tx=0x7f510002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61bcf8d3f3f5d422 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:51.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.110+0000 7f510be00640 1 -- 172.21.15.19:0/435447381 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100035020 con 0x7f510c15e250 2024-03-28T13:09:51.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.110+0000 7f510be00640 1 -- 172.21.15.19:0/435447381 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5100031640 con 0x7f510c15e250 2024-03-28T13:09:51.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.110+0000 7f510be00640 1 -- 172.21.15.19:0/435447381 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510002d040 con 0x7f510c15e250 2024-03-28T13:09:51.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.111+0000 7f5114400640 1 -- 172.21.15.19:0/435447381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f510c15e250 msgr2=0x7f510c071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.111+0000 7f5114400640 1 --2- 172.21.15.19:0/435447381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f510c15e250 0x7f510c071660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f510c0adb30 tx=0x7f510002b8f0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.117+0000 7f5114400640 1 -- 172.21.15.19:0/435447381 shutdown_connections 2024-03-28T13:09:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.117+0000 7f5114400640 1 --2- 172.21.15.19:0/435447381 >> v2:172.21.15.19:3300/0 conn(0x7f510c0d0940 0x7f510c0b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.117+0000 7f5114400640 1 --2- 172.21.15.19:0/435447381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f510c15e250 0x7f510c071660 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.117+0000 7f5114400640 1 -- 172.21.15.19:0/435447381 >> 172.21.15.19:0/435447381 conn(0x7f510c06b570 msgr2=0x7f510c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.118+0000 7f5114400640 1 -- 172.21.15.19:0/435447381 shutdown_connections 2024-03-28T13:09:51.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.118+0000 7f5114400640 1 -- 172.21.15.19:0/435447381 wait complete. 2024-03-28T13:09:51.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.121+0000 7f5114400640 1 Processor -- start 2024-03-28T13:09:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.136+0000 7f5114400640 1 -- start start 2024-03-28T13:09:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.138+0000 7f5114400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f510c0d0940 0x7f510c14fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.146+0000 7f5114400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f510c15e250 0x7f510c0ce650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.146+0000 7f5114400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f510c161f80 0x7f510c0b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.146+0000 7f5114400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f510c15e630 con 0x7f510c0d0940 2024-03-28T13:09:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.147+0000 7f5114400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f510c071ba0 con 0x7f510c161f80 2024-03-28T13:09:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.147+0000 7f5114400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f510c15e7b0 con 0x7f510c15e250 2024-03-28T13:09:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.150+0000 7f5111000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f510c15e250 0x7f510c0ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:51.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.150+0000 7f5111000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f510c15e250 0x7f510c0ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:54978/0 (socket says 172.21.15.19:54978) 2024-03-28T13:09:51.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.150+0000 7f5111000640 1 -- 172.21.15.19:0/2283709644 learned_addr learned my addr 172.21.15.19:0/2283709644 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:51.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.151+0000 7f5111000640 1 -- 172.21.15.19:0/2283709644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f510c161f80 msgr2=0x7f510c0b2ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-28T13:09:51.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.151+0000 7f5111000640 1 --2- 172.21.15.19:0/2283709644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f510c161f80 0x7f510c0b2ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.151+0000 7f5111000640 1 -- 172.21.15.19:0/2283709644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f510c0d0940 msgr2=0x7f510c14fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.151+0000 7f5111000640 1 --2- 172.21.15.19:0/2283709644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f510c0d0940 0x7f510c14fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.151+0000 7f5111000640 1 -- 172.21.15.19:0/2283709644 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510c0ceb90 con 0x7f510c15e250 2024-03-28T13:09:51.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.154+0000 7f5111000640 1 --2- 172.21.15.19:0/2283709644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f510c15e250 0x7f510c0ce650 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50f800c430 tx=0x7f50f800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:51.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.155+0000 7f5109600640 1 -- 172.21.15.19:0/2283709644 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8017070 con 0x7f510c15e250 2024-03-28T13:09:51.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.155+0000 7f5109600640 1 -- 172.21.15.19:0/2283709644 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f8005ce0 con 0x7f510c15e250 2024-03-28T13:09:51.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.155+0000 7f5109600640 1 -- 172.21.15.19:0/2283709644 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8007310 con 0x7f510c15e250 2024-03-28T13:09:51.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.156+0000 7f5114400640 1 -- 172.21.15.19:0/2283709644 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510c0cd890 con 0x7f510c15e250 2024-03-28T13:09:51.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.157+0000 7f5114400640 1 -- 172.21.15.19:0/2283709644 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f510c0cde50 con 0x7f510c15e250 2024-03-28T13:09:51.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.161+0000 7f5114400640 1 -- 172.21.15.19:0/2283709644 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510c075940 con 0x7f510c15e250 2024-03-28T13:09:51.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.185+0000 7f5109600640 1 -- 172.21.15.19:0/2283709644 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f50f801a070 con 0x7f510c15e250 2024-03-28T13:09:51.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.186+0000 7f5109600640 1 --2- 172.21.15.19:0/2283709644 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50ec06c560 0x7f50ec06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:51.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.186+0000 7f5109600640 1 -- 172.21.15.19:0/2283709644 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f50f808aa90 con 0x7f510c15e250 2024-03-28T13:09:51.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.187+0000 7f5109600640 1 -- 172.21.15.19:0/2283709644 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f50f80b8980 con 0x7f510c15e250 2024-03-28T13:09:51.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.195+0000 7f5111a00640 1 --2- 172.21.15.19:0/2283709644 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50ec06c560 0x7f50ec06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:51.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.196+0000 7f5111a00640 1 --2- 172.21.15.19:0/2283709644 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50ec06c560 0x7f50ec06ea20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5100036000 tx=0x7f5100030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:51.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.306+0000 7f3a30000640 1 -- 172.21.15.19:0/759771677 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3a280ce640 con 0x7f3a280b7560 2024-03-28T13:09:51.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.307+0000 7f3a25600640 1 -- 172.21.15.19:0/759771677 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3a20012340 con 0x7f3a280b7560 2024-03-28T13:09:51.307 INFO:teuthology.orchestra.run.smithi019.stdout:68719476745 2024-03-28T13:09:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.317+0000 7f3a30000640 1 -- 172.21.15.19:0/759771677 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a0c06c530 msgr2=0x7f3a0c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.317+0000 7fe305200640 1 -- 172.21.15.19:0/2640148347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe2d8005580 con 0x7fe3000b3740 2024-03-28T13:09:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.317+0000 7f3a30000640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a0c06c530 0x7f3a0c06e9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3a18000f60 tx=0x7f3a18007b90 comp rx=0 tx=0).stop 2024-03-28T13:09:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.317+0000 7f3a30000640 1 -- 172.21.15.19:0/759771677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a280b7560 msgr2=0x7f3a280b88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.317+0000 7f3a30000640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a280b7560 0x7f3a280b88d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009f00 tx=0x7f3a2000c490 comp rx=0 tx=0).stop 2024-03-28T13:09:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.318+0000 7f97a6200640 1 -- 172.21.15.19:0/4035285331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f97a006c630 con 0x7f97a0162920 2024-03-28T13:09:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.318+0000 7f3a30000640 1 -- 172.21.15.19:0/759771677 shutdown_connections 2024-03-28T13:09:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.318+0000 7f3a30000640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a0c06c530 0x7f3a0c06e9f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.318+0000 7f3a30000640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a280b7560 0x7f3a280b88d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.318+0000 7f3a30000640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a2815e860 0x7f3a280b7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.318+0000 7f3a30000640 1 --2- 172.21.15.19:0/759771677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a280cf6b0 0x7f3a28155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.318+0000 7f3a30000640 1 -- 172.21.15.19:0/759771677 >> 172.21.15.19:0/759771677 conn(0x7f3a2809d7d0 msgr2=0x7f3a280d1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.318+0000 7f3a30000640 1 -- 172.21.15.19:0/759771677 shutdown_connections 2024-03-28T13:09:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.319+0000 7f3a30000640 1 -- 172.21.15.19:0/759771677 wait complete. 2024-03-28T13:09:51.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.321+0000 7f9797400640 1 -- 172.21.15.19:0/4035285331 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f979007bd30 con 0x7f97a0162920 2024-03-28T13:09:51.321 INFO:teuthology.orchestra.run.smithi019.stdout:68719476743 2024-03-28T13:09:51.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.318+0000 7fe2f6000640 1 -- 172.21.15.19:0/2640148347 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe2f805e150 con 0x7fe3000b3740 2024-03-28T13:09:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.325+0000 7f97a6200640 1 -- 172.21.15.19:0/4035285331 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f977c06c600 msgr2=0x7f977c06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.325+0000 7f97a6200640 1 --2- 172.21.15.19:0/4035285331 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f977c06c600 0x7f977c06eac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9788000f60 tx=0x7f9788009040 comp rx=0 tx=0).stop 2024-03-28T13:09:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.325+0000 7f97a6200640 1 -- 172.21.15.19:0/4035285331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0162920 msgr2=0x7f97a009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.325+0000 7f97a6200640 1 --2- 172.21.15.19:0/4035285331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0162920 0x7f97a009dac0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9790033000 tx=0x7f9790004a40 comp rx=0 tx=0).stop 2024-03-28T13:09:51.325 INFO:teuthology.orchestra.run.smithi019.stdout:68719476743 2024-03-28T13:09:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.326+0000 7f97a6200640 1 -- 172.21.15.19:0/4035285331 shutdown_connections 2024-03-28T13:09:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.326+0000 7f97a6200640 1 --2- 172.21.15.19:0/4035285331 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f977c06c600 0x7f977c06eac0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.326+0000 7f97a6200640 1 --2- 172.21.15.19:0/4035285331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a0163ac0 0x7f97a0144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.326+0000 7f97a6200640 1 --2- 172.21.15.19:0/4035285331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0162920 0x7f97a009dac0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.326+0000 7f97a6200640 1 --2- 172.21.15.19:0/4035285331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a00d0990 0x7f97a009c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.326+0000 7f97a6200640 1 -- 172.21.15.19:0/4035285331 >> 172.21.15.19:0/4035285331 conn(0x7f97a006b750 msgr2=0x7f97a00d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.328+0000 7fe305200640 1 -- 172.21.15.19:0/2640148347 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2e006c600 msgr2=0x7fe2e006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.328+0000 7fe305200640 1 --2- 172.21.15.19:0/2640148347 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2e006c600 0x7fe2e006eac0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe30014d980 tx=0x7fe2ec008040 comp rx=0 tx=0).stop 2024-03-28T13:09:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.329+0000 7fe305200640 1 -- 172.21.15.19:0/2640148347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3000b3740 msgr2=0x7fe3000992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.329+0000 7fe305200640 1 --2- 172.21.15.19:0/2640148347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3000b3740 0x7fe3000992b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800cea0 tx=0x7fe2f800cf90 comp rx=0 tx=0).stop 2024-03-28T13:09:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.329+0000 7f97a6200640 1 -- 172.21.15.19:0/4035285331 shutdown_connections 2024-03-28T13:09:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.329+0000 7f97a6200640 1 -- 172.21.15.19:0/4035285331 wait complete. 2024-03-28T13:09:51.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.329+0000 7fe305200640 1 -- 172.21.15.19:0/2640148347 shutdown_connections 2024-03-28T13:09:51.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.329+0000 7fe305200640 1 --2- 172.21.15.19:0/2640148347 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2e006c600 0x7fe2e006eac0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.329+0000 7fe305200640 1 --2- 172.21.15.19:0/2640148347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe30015e950 0x7fe300201ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.329+0000 7fe305200640 1 --2- 172.21.15.19:0/2640148347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3000b3740 0x7fe3000992b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.329+0000 7fe305200640 1 --2- 172.21.15.19:0/2640148347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe300073520 0x7fe30015fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.329+0000 7fe305200640 1 -- 172.21.15.19:0/2640148347 >> 172.21.15.19:0/2640148347 conn(0x7fe300072a00 msgr2=0x7fe30006ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:51.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.330+0000 7fe305200640 1 -- 172.21.15.19:0/2640148347 shutdown_connections 2024-03-28T13:09:51.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.330+0000 7fe305200640 1 -- 172.21.15.19:0/2640148347 wait complete. 2024-03-28T13:09:51.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.340+0000 7f0c92000640 1 -- 172.21.15.19:0/2890947834 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0c8c0d1b80 con 0x7f0c700051c0 2024-03-28T13:09:51.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.341+0000 7f0c83400640 1 -- 172.21.15.19:0/2890947834 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0c7c05e100 con 0x7f0c700051c0 2024-03-28T13:09:51.342 INFO:teuthology.orchestra.run.smithi019.stdout:64424509448 2024-03-28T13:09:51.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.344+0000 7fecd7400640 1 -- 172.21.15.19:0/3752430121 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fecd00d1b80 con 0x7fecd00d0e70 2024-03-28T13:09:51.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.345+0000 7f0c92000640 1 -- 172.21.15.19:0/2890947834 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c6c06c560 msgr2=0x7f0c6c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.345+0000 7f0c92000640 1 --2- 172.21.15.19:0/2890947834 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c6c06c560 0x7f0c6c06ea20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c0d5500 tx=0x7f0c78009040 comp rx=0 tx=0).stop 2024-03-28T13:09:51.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.345+0000 7f0c92000640 1 -- 172.21.15.19:0/2890947834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c700051c0 msgr2=0x7f0c8c0ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476745 for osd.3 2024-03-28T13:09:51.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.345+0000 7f0c92000640 1 --2- 172.21.15.19:0/2890947834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c700051c0 0x7f0c8c0ca7b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c011c00 tx=0x7f0c7c00f610 comp rx=0 tx=0).stop 2024-03-28T13:09:51.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.345+0000 7fecccc00640 1 -- 172.21.15.19:0/3752430121 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fecc007b4a0 con 0x7fecd00d0e70 2024-03-28T13:09:51.347 DEBUG:teuthology.parallel:result is None 2024-03-28T13:09:51.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.345+0000 7f0c92000640 1 -- 172.21.15.19:0/2890947834 shutdown_connections 2024-03-28T13:09:51.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.345+0000 7f0c92000640 1 --2- 172.21.15.19:0/2890947834 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c6c06c560 0x7f0c6c06ea20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.345+0000 7f0c92000640 1 --2- 172.21.15.19:0/2890947834 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8c0ccf40 0x7f0c8c0cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.345+0000 7f0c92000640 1 --2- 172.21.15.19:0/2890947834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8c0ca230 0x7f0c8c1f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.345+0000 7f0c92000640 1 --2- 172.21.15.19:0/2890947834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c700051c0 0x7f0c8c0ca7b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.345+0000 7f0c92000640 1 -- 172.21.15.19:0/2890947834 >> 172.21.15.19:0/2890947834 conn(0x7f0c8c08bfb0 msgr2=0x7f0c8c15d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:51.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.346+0000 7f0c92000640 1 -- 172.21.15.19:0/2890947834 shutdown_connections 2024-03-28T13:09:51.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.346+0000 7f0c92000640 1 -- 172.21.15.19:0/2890947834 wait complete. 2024-03-28T13:09:51.348 INFO:teuthology.orchestra.run.smithi019.stdout:60129542152 2024-03-28T13:09:51.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.352+0000 7fecd7400640 1 -- 172.21.15.19:0/3752430121 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fecb006c600 msgr2=0x7fecb006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.352+0000 7fecd7400640 1 --2- 172.21.15.19:0/3752430121 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fecb006c600 0x7fecb006eac0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fecd0161d40 tx=0x7fecbc008040 comp rx=0 tx=0).stop 2024-03-28T13:09:51.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.352+0000 7fecd7400640 1 -- 172.21.15.19:0/3752430121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecd00d0e70 msgr2=0x7fecd00cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.352+0000 7fecd7400640 1 --2- 172.21.15.19:0/3752430121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecd00d0e70 0x7fecd00cb530 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fecc0000c00 tx=0x7fecc00046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.357+0000 7fecd7400640 1 -- 172.21.15.19:0/3752430121 shutdown_connections 2024-03-28T13:09:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.357+0000 7fecd7400640 1 --2- 172.21.15.19:0/3752430121 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fecb006c600 0x7fecb006eac0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.357+0000 7fecd7400640 1 --2- 172.21.15.19:0/3752430121 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecd01621d0 0x7fecd00ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.357+0000 7fecd7400640 1 --2- 172.21.15.19:0/3752430121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fecd015f6e0 0x7fecd014d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.357+0000 7fecd7400640 1 --2- 172.21.15.19:0/3752430121 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecd00d0e70 0x7fecd00cb530 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.357+0000 7fecd7400640 1 -- 172.21.15.19:0/3752430121 >> 172.21.15.19:0/3752430121 conn(0x7fecd008bfb0 msgr2=0x7fecd006d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.357+0000 7fecd7400640 1 -- 172.21.15.19:0/3752430121 shutdown_connections 2024-03-28T13:09:51.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.357+0000 7fecd7400640 1 -- 172.21.15.19:0/3752430121 wait complete. 2024-03-28T13:09:51.360 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476743 for osd.1 2024-03-28T13:09:51.360 DEBUG:teuthology.parallel:result is None 2024-03-28T13:09:51.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476743 for osd.0 2024-03-28T13:09:51.364 DEBUG:teuthology.parallel:result is None 2024-03-28T13:09:51.367 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.5 2024-03-28T13:09:51.367 DEBUG:teuthology.parallel:result is None 2024-03-28T13:09:51.381 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.7 2024-03-28T13:09:51.381 DEBUG:teuthology.parallel:result is None 2024-03-28T13:09:51.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.441+0000 7f42aea00640 1 -- 172.21.15.19:0/1384011226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4290005850 con 0x7f42bc0ccea0 2024-03-28T13:09:51.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.442+0000 7f42b9600640 1 -- 172.21.15.19:0/1384011226 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f42b0082030 con 0x7f42bc0ccea0 2024-03-28T13:09:51.442 INFO:teuthology.orchestra.run.smithi019.stdout:64424509448 2024-03-28T13:09:51.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.444+0000 7f42aea00640 1 -- 172.21.15.19:0/1384011226 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f429c06c600 msgr2=0x7f429c06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.444+0000 7f42aea00640 1 --2- 172.21.15.19:0/1384011226 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f429c06c600 0x7f429c06eac0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f42bc0b19b0 tx=0x7f42b400c040 comp rx=0 tx=0).stop 2024-03-28T13:09:51.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.444+0000 7f42aea00640 1 -- 172.21.15.19:0/1384011226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc0ccea0 msgr2=0x7f42bc1e40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.444+0000 7f42aea00640 1 --2- 172.21.15.19:0/1384011226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc0ccea0 0x7f42bc1e40b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f42b0033000 tx=0x7f42b00046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.444+0000 7f42aea00640 1 -- 172.21.15.19:0/1384011226 shutdown_connections 2024-03-28T13:09:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.444+0000 7f42aea00640 1 --2- 172.21.15.19:0/1384011226 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f429c06c600 0x7f429c06eac0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.444+0000 7f42aea00640 1 --2- 172.21.15.19:0/1384011226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42bc0d0600 0x7f42bc1557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.444+0000 7f42aea00640 1 --2- 172.21.15.19:0/1384011226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42bc0ccea0 0x7f42bc1e40b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.444+0000 7f42aea00640 1 --2- 172.21.15.19:0/1384011226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42bc06ea90 0x7f42bc09aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.444+0000 7f42aea00640 1 -- 172.21.15.19:0/1384011226 >> 172.21.15.19:0/1384011226 conn(0x7f42bc06b570 msgr2=0x7f42bc159ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.444+0000 7f42aea00640 1 -- 172.21.15.19:0/1384011226 shutdown_connections 2024-03-28T13:09:51.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.444+0000 7f42aea00640 1 -- 172.21.15.19:0/1384011226 wait complete. 2024-03-28T13:09:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.445+0000 7f004c600640 1 -- 172.21.15.19:0/3212322358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f004406ef60 con 0x7f00441565d0 2024-03-28T13:09:51.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.445+0000 7f0041600640 1 -- 172.21.15.19:0/3212322358 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f003807ac70 con 0x7f00441565d0 2024-03-28T13:09:51.446 INFO:teuthology.orchestra.run.smithi019.stdout:68719476743 2024-03-28T13:09:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.448+0000 7f004c600640 1 -- 172.21.15.19:0/3212322358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f002406c730 msgr2=0x7f002406ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.448+0000 7f004c600640 1 --2- 172.21.15.19:0/3212322358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f002406c730 0x7f002406ebf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f00440b5250 tx=0x7f0030009040 comp rx=0 tx=0).stop 2024-03-28T13:09:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.448+0000 7f004c600640 1 -- 172.21.15.19:0/3212322358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00441565d0 msgr2=0x7f00440b32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.448+0000 7f004c600640 1 --2- 172.21.15.19:0/3212322358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00441565d0 0x7f00440b32b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f003800c290 tx=0x7f0038001970 comp rx=0 tx=0).stop 2024-03-28T13:09:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.448+0000 7f004c600640 1 -- 172.21.15.19:0/3212322358 shutdown_connections 2024-03-28T13:09:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.448+0000 7f004c600640 1 --2- 172.21.15.19:0/3212322358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f002406c730 0x7f002406ebf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.448+0000 7f004c600640 1 --2- 172.21.15.19:0/3212322358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00441569b0 0x7f004415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.448+0000 7f004c600640 1 --2- 172.21.15.19:0/3212322358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f004415ad80 0x7f004415d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.448+0000 7f004c600640 1 --2- 172.21.15.19:0/3212322358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00441565d0 0x7f00440b32b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.448+0000 7f004c600640 1 -- 172.21.15.19:0/3212322358 >> 172.21.15.19:0/3212322358 conn(0x7f004406b750 msgr2=0x7f00440d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.449+0000 7f004c600640 1 -- 172.21.15.19:0/3212322358 shutdown_connections 2024-03-28T13:09:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.449+0000 7f004c600640 1 -- 172.21.15.19:0/3212322358 wait complete. 2024-03-28T13:09:51.465 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476743 for osd.2 2024-03-28T13:09:51.466 DEBUG:teuthology.parallel:result is None 2024-03-28T13:09:51.468 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.4 2024-03-28T13:09:51.468 DEBUG:teuthology.parallel:result is None 2024-03-28T13:09:51.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.527+0000 7f5114400640 1 -- 172.21.15.19:0/2283709644 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f510c0b2740 con 0x7f510c15e250 2024-03-28T13:09:51.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.528+0000 7f5109600640 1 -- 172.21.15.19:0/2283709644 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f50f80591e0 con 0x7f510c15e250 2024-03-28T13:09:51.528 INFO:teuthology.orchestra.run.smithi019.stdout:64424509447 2024-03-28T13:09:51.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.530+0000 7f5114400640 1 -- 172.21.15.19:0/2283709644 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50ec06c560 msgr2=0x7f50ec06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.530+0000 7f5114400640 1 --2- 172.21.15.19:0/2283709644 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50ec06c560 0x7f50ec06ea20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5100036000 tx=0x7f5100030000 comp rx=0 tx=0).stop 2024-03-28T13:09:51.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.530+0000 7f5114400640 1 -- 172.21.15.19:0/2283709644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f510c15e250 msgr2=0x7f510c0ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.530+0000 7f5114400640 1 --2- 172.21.15.19:0/2283709644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f510c15e250 0x7f510c0ce650 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f50f800c430 tx=0x7f50f800c8f0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.530+0000 7f5114400640 1 -- 172.21.15.19:0/2283709644 shutdown_connections 2024-03-28T13:09:51.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.530+0000 7f5114400640 1 --2- 172.21.15.19:0/2283709644 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50ec06c560 0x7f50ec06ea20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.530+0000 7f5114400640 1 --2- 172.21.15.19:0/2283709644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f510c161f80 0x7f510c0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.530+0000 7f5114400640 1 --2- 172.21.15.19:0/2283709644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f510c15e250 0x7f510c0ce650 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.530+0000 7f5114400640 1 --2- 172.21.15.19:0/2283709644 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f510c0d0940 0x7f510c14fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.530+0000 7f5114400640 1 -- 172.21.15.19:0/2283709644 >> 172.21.15.19:0/2283709644 conn(0x7f510c06b570 msgr2=0x7f510c0af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:51.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.531+0000 7f5114400640 1 -- 172.21.15.19:0/2283709644 shutdown_connections 2024-03-28T13:09:51.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.531+0000 7f5114400640 1 -- 172.21.15.19:0/2283709644 wait complete. 2024-03-28T13:09:51.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.6 2024-03-28T13:09:51.548 DEBUG:teuthology.parallel:result is None 2024-03-28T13:09:51.548 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-28T13:09:51.548 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:09:51.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.747+0000 7f7ed8a00640 1 Processor -- start 2024-03-28T13:09:51.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.747+0000 7f7ed8a00640 1 -- start start 2024-03-28T13:09:51.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.750+0000 7f7ed8a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed415e4a0 0x7f7ed40b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:51.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.750+0000 7f7ed8a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ed40d2310 0x7f7ed40ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:51.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.750+0000 7f7ed8a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed415e880 con 0x7f7ed40cf530 2024-03-28T13:09:51.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.750+0000 7f7ed8a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed40d34e0 con 0x7f7ed40d2310 2024-03-28T13:09:51.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.750+0000 7f7ed8a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed40d28d0 con 0x7f7ed415e4a0 2024-03-28T13:09:51.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.750+0000 7f7ed1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ed40d2310 0x7f7ed40ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:51.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.750+0000 7f7ed1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ed40d2310 0x7f7ed40ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50980/0 (socket says 172.21.15.19:50980) 2024-03-28T13:09:51.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.750+0000 7f7ed1e00640 1 -- 172.21.15.19:0/382183864 learned_addr learned my addr 172.21.15.19:0/382183864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:51.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.750+0000 7f7ed1e00640 1 -- 172.21.15.19:0/382183864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed415e4a0 msgr2=0x7f7ed40b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:51.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.750+0000 7f7ed2800640 1 --2- 172.21.15.19:0/382183864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed415e4a0 0x7f7ed40b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:51.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.751+0000 7f7ed1e00640 1 --2- 172.21.15.19:0/382183864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed415e4a0 0x7f7ed40b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.751+0000 7f7ed1e00640 1 -- 172.21.15.19:0/382183864 >> v1:172.21.15.188:6789/0 conn(0x7f7ed40cf530 legacy=0x7f7ed40d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:51.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.751+0000 7f7ed1e00640 1 -- 172.21.15.19:0/382183864 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed40d2bb0 con 0x7f7ed40d2310 2024-03-28T13:09:51.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.751+0000 7f7ed2800640 1 --2- 172.21.15.19:0/382183864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed415e4a0 0x7f7ed40b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:51.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.752+0000 7f7ed1e00640 1 --2- 172.21.15.19:0/382183864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ed40d2310 0x7f7ed40ceff0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009870 tx=0x7f7ec002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5caaefb342138a0a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:51.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.752+0000 7f7ed1400640 1 -- 172.21.15.19:0/382183864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0035020 con 0x7f7ed40d2310 2024-03-28T13:09:51.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.752+0000 7f7ed1400640 1 -- 172.21.15.19:0/382183864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec0031640 con 0x7f7ed40d2310 2024-03-28T13:09:51.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.752+0000 7f7ed1400640 1 -- 172.21.15.19:0/382183864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec002d040 con 0x7f7ed40d2310 2024-03-28T13:09:51.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.753+0000 7f7ed8a00640 1 -- 172.21.15.19:0/382183864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ed40d2310 msgr2=0x7f7ed40ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.753+0000 7f7ed8a00640 1 --2- 172.21.15.19:0/382183864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ed40d2310 0x7f7ed40ceff0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009870 tx=0x7f7ec002b8f0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.753+0000 7f7ed8a00640 1 -- 172.21.15.19:0/382183864 shutdown_connections 2024-03-28T13:09:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.753+0000 7f7ed8a00640 1 --2- 172.21.15.19:0/382183864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ed40d2310 0x7f7ed40ceff0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.753+0000 7f7ed8a00640 1 --2- 172.21.15.19:0/382183864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed415e4a0 0x7f7ed40b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.753+0000 7f7ed8a00640 1 -- 172.21.15.19:0/382183864 >> 172.21.15.19:0/382183864 conn(0x7f7ed40676d0 msgr2=0x7f7ed4067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.753+0000 7f7ed8a00640 1 -- 172.21.15.19:0/382183864 shutdown_connections 2024-03-28T13:09:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.753+0000 7f7ed8a00640 1 -- 172.21.15.19:0/382183864 wait complete. 2024-03-28T13:09:51.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.756+0000 7f7ed8a00640 1 Processor -- start 2024-03-28T13:09:51.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.757+0000 7f7ed8a00640 1 -- start start 2024-03-28T13:09:51.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ed40cf530 0x7f7ed40c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:51.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed8a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed40d2310 0x7f7ed40b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:51.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed8a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ed415e4a0 0x7f7ed40b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:51.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed8a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed40d3970 con 0x7f7ed40cf530 2024-03-28T13:09:51.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed8a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed40d3170 con 0x7f7ed415e4a0 2024-03-28T13:09:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed8a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed40ba1a0 con 0x7f7ed40d2310 2024-03-28T13:09:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ed40cf530 0x7f7ed40c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ed40cf530 0x7f7ed40c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50994/0 (socket says 172.21.15.19:50994) 2024-03-28T13:09:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed2800640 1 -- 172.21.15.19:0/251230362 learned_addr learned my addr 172.21.15.19:0/251230362 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed1e00640 1 --2- 172.21.15.19:0/251230362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed40d2310 0x7f7ed40b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed2800640 1 -- 172.21.15.19:0/251230362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed40d2310 msgr2=0x7f7ed40b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed2800640 1 --2- 172.21.15.19:0/251230362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed40d2310 0x7f7ed40b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed2800640 1 -- 172.21.15.19:0/251230362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ed415e4a0 msgr2=0x7f7ed40b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed2800640 1 --2- 172.21.15.19:0/251230362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ed415e4a0 0x7f7ed40b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed2800640 1 -- 172.21.15.19:0/251230362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed40c86c0 con 0x7f7ed40cf530 2024-03-28T13:09:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed2800640 1 --2- 172.21.15.19:0/251230362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ed40cf530 0x7f7ed40c7dc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c7f0 tx=0x7f7ec400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ecaa00640 1 -- 172.21.15.19:0/251230362 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4015070 con 0x7f7ed40cf530 2024-03-28T13:09:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ecaa00640 1 -- 172.21.15.19:0/251230362 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec4007820 con 0x7f7ed40cf530 2024-03-28T13:09:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ecaa00640 1 -- 172.21.15.19:0/251230362 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4007bd0 con 0x7f7ed40cf530 2024-03-28T13:09:51.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed8a00640 1 -- 172.21.15.19:0/251230362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed40c89a0 con 0x7f7ed40cf530 2024-03-28T13:09:51.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.759+0000 7f7ed8a00640 1 -- 172.21.15.19:0/251230362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed415ea80 con 0x7f7ed40cf530 2024-03-28T13:09:51.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.761+0000 7f7ed8a00640 1 -- 172.21.15.19:0/251230362 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed4075940 con 0x7f7ed40cf530 2024-03-28T13:09:51.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.761+0000 7f7ecaa00640 1 -- 172.21.15.19:0/251230362 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7ec4018070 con 0x7f7ed40cf530 2024-03-28T13:09:51.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.762+0000 7f7ecaa00640 1 --2- 172.21.15.19:0/251230362 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7eb406c530 0x7f7eb406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:51.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.762+0000 7f7ecaa00640 1 -- 172.21.15.19:0/251230362 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7ec408b960 con 0x7f7ed40cf530 2024-03-28T13:09:51.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.763+0000 7f7ed1e00640 1 --2- 172.21.15.19:0/251230362 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7eb406c530 0x7f7eb406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:51.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.764+0000 7f7ed1e00640 1 --2- 172.21.15.19:0/251230362 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7eb406c530 0x7f7eb406e9f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0036000 tx=0x7f7ec0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:51.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.767+0000 7f7ecaa00640 1 -- 172.21.15.19:0/251230362 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7ec4059e50 con 0x7f7ed40cf530 2024-03-28T13:09:51.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.961+0000 7f7ed8a00640 1 -- 172.21.15.19:0/251230362 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7ed4155140 con 0x7f7eb406c530 2024-03-28T13:09:51.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.980+0000 7f7ecaa00640 1 -- 172.21.15.19:0/251230362 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251832 (secure 0 0 0) 0x7f7ed4155140 con 0x7f7eb406c530 2024-03-28T13:09:51.984 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:51.985 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:09:51.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.986+0000 7f7ed8a00640 1 -- 172.21.15.19:0/251230362 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7eb406c530 msgr2=0x7f7eb406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.986+0000 7f7ed8a00640 1 --2- 172.21.15.19:0/251230362 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7eb406c530 0x7f7eb406e9f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0036000 tx=0x7f7ec0008030 comp rx=0 tx=0).stop 2024-03-28T13:09:51.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.986+0000 7f7ed8a00640 1 -- 172.21.15.19:0/251230362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ed40cf530 msgr2=0x7f7ed40c7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:51.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.986+0000 7f7ed8a00640 1 --2- 172.21.15.19:0/251230362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ed40cf530 0x7f7ed40c7dc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c7f0 tx=0x7f7ec400ccb0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.986+0000 7f7ed8a00640 1 -- 172.21.15.19:0/251230362 shutdown_connections 2024-03-28T13:09:51.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.986+0000 7f7ed8a00640 1 --2- 172.21.15.19:0/251230362 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7eb406c530 0x7f7eb406e9f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.986+0000 7f7ed8a00640 1 --2- 172.21.15.19:0/251230362 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ed415e4a0 0x7f7ed40b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.986+0000 7f7ed8a00640 1 --2- 172.21.15.19:0/251230362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ed40d2310 0x7f7ed40b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.986+0000 7f7ed8a00640 1 --2- 172.21.15.19:0/251230362 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ed40cf530 0x7f7ed40c7dc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:51.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.986+0000 7f7ed8a00640 1 -- 172.21.15.19:0/251230362 >> 172.21.15.19:0/251230362 conn(0x7f7ed40676d0 msgr2=0x7f7ed415c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:51.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.986+0000 7f7ed8a00640 1 -- 172.21.15.19:0/251230362 shutdown_connections 2024-03-28T13:09:51.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:51.986+0000 7f7ed8a00640 1 -- 172.21.15.19:0/251230362 wait complete. 2024-03-28T13:09:52.004 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-03-28T13:09:49.983883+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219188,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755532,"statfs":{"total":773094113280,"available":772869664768,"internally_reserved":0,"allocated":3088384,"data_stored":1824236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.334722"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696286+0000","last_change":"2024-03-28T13:09:41.690249+0000","last_active":"2024-03-28T13:09:42.696286+0000","last_peered":"2024-03-28T13:09:42.696286+0000","last_clean":"2024-03-28T13:09:42.696286+0000","last_became_active":"2024-03-28T13:09:41.689984+0000","last_became_peered":"2024-03-28T13:09:41.689984+0000","last_unstale":"2024-03-28T13:09:42.696286+0000","last_undegraded":"2024-03-28T13:09:42.696286+0000","last_fullsized":"2024-03-28T13:09:42.696286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:53:08.763850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687950+0000","last_change":"2024-03-28T13:09:41.669236+0000","last_active":"2024-03-28T13:09:42.687950+0000","last_peered":"2024-03-28T13:09:42.687950+0000","last_clean":"2024-03-28T13:09:42.687950+0000","last_became_active":"2024-03-28T13:09:41.668886+0000","last_became_peered":"2024-03-28T13:09:41.668886+0000","last_unstale":"2024-03-28T13:09:42.687950+0000","last_undegraded":"2024-03-28T13:09:42.687950+0000","last_fullsized":"2024-03-28T13:09:42.687950+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:53:08.763850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687498+0000","last_change":"2024-03-28T13:09:41.683573+0000","last_active":"2024-03-28T13:09:42.687498+0000","last_peered":"2024-03-28T13:09:42.687498+0000","last_clean":"2024-03-28T13:09:42.687498+0000","last_became_active":"2024-03-28T13:09:41.682846+0000","last_became_peered":"2024-03-28T13:09:41.682846+0000","last_unstale":"2024-03-28T13:09:42.687498+0000","last_undegraded":"2024-03-28T13:09:42.687498+0000","last_fullsized":"2024-03-28T13:09:42.687498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:04.694363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680562+0000","last_change":"2024-03-28T13:09:41.684558+0000","last_active":"2024-03-28T13:09:42.680562+0000","last_peered":"2024-03-28T13:09:42.680562+0000","last_clean":"2024-03-28T13:09:42.680562+0000","last_became_active":"2024-03-28T13:09:41.682564+0000","last_became_peered":"2024-03-28T13:09:41.682564+0000","last_unstale":"2024-03-28T13:09:42.680562+0000","last_undegraded":"2024-03-28T13:09:42.680562+0000","last_fullsized":"2024-03-28T13:09:42.680562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:21:17.844127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695969+0000","last_change":"2024-03-28T13:09:41.687175+0000","last_active":"2024-03-28T13:09:42.695969+0000","last_peered":"2024-03-28T13:09:42.695969+0000","last_clean":"2024-03-28T13:09:42.695969+0000","last_became_active":"2024-03-28T13:09:41.686835+0000","last_became_peered":"2024-03-28T13:09:41.686835+0000","last_unstale":"2024-03-28T13:09:42.695969+0000","last_undegraded":"2024-03-28T13:09:42.695969+0000","last_fullsized":"2024-03-28T13:09:42.695969+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:04.694363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.679923+0000","last_change":"2024-03-28T13:09:41.683515+0000","last_active":"2024-03-28T13:09:42.679923+0000","last_peered":"2024-03-28T13:09:42.679923+0000","last_clean":"2024-03-28T13:09:42.679923+0000","last_became_active":"2024-03-28T13:09:41.682795+0000","last_became_peered":"2024-03-28T13:09:41.682795+0000","last_unstale":"2024-03-28T13:09:42.679923+0000","last_undegraded":"2024-03-28T13:09:42.679923+0000","last_fullsized":"2024-03-28T13:09:42.679923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:29.551203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695673+0000","last_change":"2024-03-28T13:09:41.690393+0000","last_active":"2024-03-28T13:09:42.695673+0000","last_peered":"2024-03-28T13:09:42.695673+0000","last_clean":"2024-03-28T13:09:42.695673+0000","last_became_active":"2024-03-28T13:09:41.688973+0000","last_became_peered":"2024-03-28T13:09:41.688973+0000","last_unstale":"2024-03-28T13:09:42.695673+0000","last_undegraded":"2024-03-28T13:09:42.695673+0000","last_fullsized":"2024-03-28T13:09:42.695673+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:48.181534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686877+0000","last_change":"2024-03-28T13:09:41.691319+0000","last_active":"2024-03-28T13:09:42.686877+0000","last_peered":"2024-03-28T13:09:42.686877+0000","last_clean":"2024-03-28T13:09:42.686877+0000","last_became_active":"2024-03-28T13:09:41.690506+0000","last_became_peered":"2024-03-28T13:09:41.690506+0000","last_unstale":"2024-03-28T13:09:42.686877+0000","last_undegraded":"2024-03-28T13:09:42.686877+0000","last_fullsized":"2024-03-28T13:09:42.686877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:28:13.777057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942467+0000","last_change":"2024-03-28T13:09:41.697845+0000","last_active":"2024-03-28T13:09:42.942467+0000","last_peered":"2024-03-28T13:09:42.942467+0000","last_clean":"2024-03-28T13:09:42.942467+0000","last_became_active":"2024-03-28T13:09:41.696890+0000","last_became_peered":"2024-03-28T13:09:41.696890+0000","last_unstale":"2024-03-28T13:09:42.942467+0000","last_undegraded":"2024-03-28T13:09:42.942467+0000","last_fullsized":"2024-03-28T13:09:42.942467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:37.029859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696276+0000","last_change":"2024-03-28T13:09:41.688074+0000","last_active":"2024-03-28T13:09:42.696276+0000","last_peered":"2024-03-28T13:09:42.696276+0000","last_clean":"2024-03-28T13:09:42.696276+0000","last_became_active":"2024-03-28T13:09:41.687755+0000","last_became_peered":"2024-03-28T13:09:41.687755+0000","last_unstale":"2024-03-28T13:09:42.696276+0000","last_undegraded":"2024-03-28T13:09:42.696276+0000","last_fullsized":"2024-03-28T13:09:42.696276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.673442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695935+0000","last_change":"2024-03-28T13:09:41.693370+0000","last_active":"2024-03-28T13:09:42.695935+0000","last_peered":"2024-03-28T13:09:42.695935+0000","last_clean":"2024-03-28T13:09:42.695935+0000","last_became_active":"2024-03-28T13:09:41.690289+0000","last_became_peered":"2024-03-28T13:09:41.690289+0000","last_unstale":"2024-03-28T13:09:42.695935+0000","last_undegraded":"2024-03-28T13:09:42.695935+0000","last_fullsized":"2024-03-28T13:09:42.695935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:03:17.731528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696467+0000","last_change":"2024-03-28T13:09:41.692663+0000","last_active":"2024-03-28T13:09:42.696467+0000","last_peered":"2024-03-28T13:09:42.696467+0000","last_clean":"2024-03-28T13:09:42.696467+0000","last_became_active":"2024-03-28T13:09:41.692130+0000","last_became_peered":"2024-03-28T13:09:41.692130+0000","last_unstale":"2024-03-28T13:09:42.696467+0000","last_undegraded":"2024-03-28T13:09:42.696467+0000","last_fullsized":"2024-03-28T13:09:42.696467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:38.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945637+0000","last_change":"2024-03-28T13:09:41.692414+0000","last_active":"2024-03-28T13:09:42.945637+0000","last_peered":"2024-03-28T13:09:42.945637+0000","last_clean":"2024-03-28T13:09:42.945637+0000","last_became_active":"2024-03-28T13:09:41.692105+0000","last_became_peered":"2024-03-28T13:09:41.692105+0000","last_unstale":"2024-03-28T13:09:42.945637+0000","last_undegraded":"2024-03-28T13:09:42.945637+0000","last_fullsized":"2024-03-28T13:09:42.945637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:21:17.844127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.946036+0000","last_change":"2024-03-28T13:09:41.675736+0000","last_active":"2024-03-28T13:09:42.946036+0000","last_peered":"2024-03-28T13:09:42.946036+0000","last_clean":"2024-03-28T13:09:42.946036+0000","last_became_active":"2024-03-28T13:09:41.675120+0000","last_became_peered":"2024-03-28T13:09:41.675120+0000","last_unstale":"2024-03-28T13:09:42.946036+0000","last_undegraded":"2024-03-28T13:09:42.946036+0000","last_fullsized":"2024-03-28T13:09:42.946036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:16.926876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686621+0000","last_change":"2024-03-28T13:09:41.690417+0000","last_active":"2024-03-28T13:09:42.686621+0000","last_peered":"2024-03-28T13:09:42.686621+0000","last_clean":"2024-03-28T13:09:42.686621+0000","last_became_active":"2024-03-28T13:09:41.689467+0000","last_became_peered":"2024-03-28T13:09:41.689467+0000","last_unstale":"2024-03-28T13:09:42.686621+0000","last_undegraded":"2024-03-28T13:09:42.686621+0000","last_fullsized":"2024-03-28T13:09:42.686621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:50:46.504501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684397+0000","last_change":"2024-03-28T13:09:41.684876+0000","last_active":"2024-03-28T13:09:42.684397+0000","last_peered":"2024-03-28T13:09:42.684397+0000","last_clean":"2024-03-28T13:09:42.684397+0000","last_became_active":"2024-03-28T13:09:41.683988+0000","last_became_peered":"2024-03-28T13:09:41.683988+0000","last_unstale":"2024-03-28T13:09:42.684397+0000","last_undegraded":"2024-03-28T13:09:42.684397+0000","last_fullsized":"2024-03-28T13:09:42.684397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:38.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685867+0000","last_change":"2024-03-28T13:09:41.702219+0000","last_active":"2024-03-28T13:09:42.685867+0000","last_peered":"2024-03-28T13:09:42.685867+0000","last_clean":"2024-03-28T13:09:42.685867+0000","last_became_active":"2024-03-28T13:09:41.701907+0000","last_became_peered":"2024-03-28T13:09:41.701907+0000","last_unstale":"2024-03-28T13:09:42.685867+0000","last_undegraded":"2024-03-28T13:09:42.685867+0000","last_fullsized":"2024-03-28T13:09:42.685867+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:49:20.779575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944358+0000","last_change":"2024-03-28T13:09:41.683832+0000","last_active":"2024-03-28T13:09:42.944358+0000","last_peered":"2024-03-28T13:09:42.944358+0000","last_clean":"2024-03-28T13:09:42.944358+0000","last_became_active":"2024-03-28T13:09:41.683495+0000","last_became_peered":"2024-03-28T13:09:41.683495+0000","last_unstale":"2024-03-28T13:09:42.944358+0000","last_undegraded":"2024-03-28T13:09:42.944358+0000","last_fullsized":"2024-03-28T13:09:42.944358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:44:52.163706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685573+0000","last_change":"2024-03-28T13:09:41.688514+0000","last_active":"2024-03-28T13:09:42.685573+0000","last_peered":"2024-03-28T13:09:42.685573+0000","last_clean":"2024-03-28T13:09:42.685573+0000","last_became_active":"2024-03-28T13:09:41.688158+0000","last_became_peered":"2024-03-28T13:09:41.688158+0000","last_unstale":"2024-03-28T13:09:42.685573+0000","last_undegraded":"2024-03-28T13:09:42.685573+0000","last_fullsized":"2024-03-28T13:09:42.685573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:39:02.380052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696639+0000","last_change":"2024-03-28T13:09:41.695024+0000","last_active":"2024-03-28T13:09:42.696639+0000","last_peered":"2024-03-28T13:09:42.696639+0000","last_clean":"2024-03-28T13:09:42.696639+0000","last_became_active":"2024-03-28T13:09:41.694698+0000","last_became_peered":"2024-03-28T13:09:41.694698+0000","last_unstale":"2024-03-28T13:09:42.696639+0000","last_undegraded":"2024-03-28T13:09:42.696639+0000","last_fullsized":"2024-03-28T13:09:42.696639+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:38:26.092780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691415+0000","last_change":"2024-03-28T13:09:41.686281+0000","last_active":"2024-03-28T13:09:42.691415+0000","last_peered":"2024-03-28T13:09:42.691415+0000","last_clean":"2024-03-28T13:09:42.691415+0000","last_became_active":"2024-03-28T13:09:41.685221+0000","last_became_peered":"2024-03-28T13:09:41.685221+0000","last_unstale":"2024-03-28T13:09:42.691415+0000","last_undegraded":"2024-03-28T13:09:42.691415+0000","last_fullsized":"2024-03-28T13:09:42.691415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:21:24.753764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.941951+0000","last_change":"2024-03-28T13:09:41.694982+0000","last_active":"2024-03-28T13:09:42.941951+0000","last_peered":"2024-03-28T13:09:42.941951+0000","last_clean":"2024-03-28T13:09:42.941951+0000","last_became_active":"2024-03-28T13:09:41.694746+0000","last_became_peered":"2024-03-28T13:09:41.694746+0000","last_unstale":"2024-03-28T13:09:42.941951+0000","last_undegraded":"2024-03-28T13:09:42.941951+0000","last_fullsized":"2024-03-28T13:09:42.941951+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:29.551203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686937+0000","last_change":"2024-03-28T13:09:41.685949+0000","last_active":"2024-03-28T13:09:42.686937+0000","last_peered":"2024-03-28T13:09:42.686937+0000","last_clean":"2024-03-28T13:09:42.686937+0000","last_became_active":"2024-03-28T13:09:41.683610+0000","last_became_peered":"2024-03-28T13:09:41.683610+0000","last_unstale":"2024-03-28T13:09:42.686937+0000","last_undegraded":"2024-03-28T13:09:42.686937+0000","last_fullsized":"2024-03-28T13:09:42.686937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:21:24.753764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684741+0000","last_change":"2024-03-28T13:09:41.672017+0000","last_active":"2024-03-28T13:09:42.684741+0000","last_peered":"2024-03-28T13:09:42.684741+0000","last_clean":"2024-03-28T13:09:42.684741+0000","last_became_active":"2024-03-28T13:09:41.671594+0000","last_became_peered":"2024-03-28T13:09:41.671594+0000","last_unstale":"2024-03-28T13:09:42.684741+0000","last_undegraded":"2024-03-28T13:09:42.684741+0000","last_fullsized":"2024-03-28T13:09:42.684741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:05:06.674720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687844+0000","last_change":"2024-03-28T13:09:41.689019+0000","last_active":"2024-03-28T13:09:42.687844+0000","last_peered":"2024-03-28T13:09:42.687844+0000","last_clean":"2024-03-28T13:09:42.687844+0000","last_became_active":"2024-03-28T13:09:41.688663+0000","last_became_peered":"2024-03-28T13:09:41.688663+0000","last_unstale":"2024-03-28T13:09:42.687844+0000","last_undegraded":"2024-03-28T13:09:42.687844+0000","last_fullsized":"2024-03-28T13:09:42.687844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.673442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680521+0000","last_change":"2024-03-28T13:09:41.698336+0000","last_active":"2024-03-28T13:09:42.680521+0000","last_peered":"2024-03-28T13:09:42.680521+0000","last_clean":"2024-03-28T13:09:42.680521+0000","last_became_active":"2024-03-28T13:09:41.696491+0000","last_became_peered":"2024-03-28T13:09:41.696491+0000","last_unstale":"2024-03-28T13:09:42.680521+0000","last_undegraded":"2024-03-28T13:09:42.680521+0000","last_fullsized":"2024-03-28T13:09:42.680521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:37.029859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945985+0000","last_change":"2024-03-28T13:09:41.676686+0000","last_active":"2024-03-28T13:09:42.945985+0000","last_peered":"2024-03-28T13:09:42.945985+0000","last_clean":"2024-03-28T13:09:42.945985+0000","last_became_active":"2024-03-28T13:09:41.676403+0000","last_became_peered":"2024-03-28T13:09:41.676403+0000","last_unstale":"2024-03-28T13:09:42.945985+0000","last_undegraded":"2024-03-28T13:09:42.945985+0000","last_fullsized":"2024-03-28T13:09:42.945985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:38:55.330149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945843+0000","last_change":"2024-03-28T13:09:41.688698+0000","last_active":"2024-03-28T13:09:42.945843+0000","last_peered":"2024-03-28T13:09:42.945843+0000","last_clean":"2024-03-28T13:09:42.945843+0000","last_became_active":"2024-03-28T13:09:41.688352+0000","last_became_peered":"2024-03-28T13:09:41.688352+0000","last_unstale":"2024-03-28T13:09:42.945843+0000","last_undegraded":"2024-03-28T13:09:42.945843+0000","last_fullsized":"2024-03-28T13:09:42.945843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:26.076499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686871+0000","last_change":"2024-03-28T13:09:41.675052+0000","last_active":"2024-03-28T13:09:42.686871+0000","last_peered":"2024-03-28T13:09:42.686871+0000","last_clean":"2024-03-28T13:09:42.686871+0000","last_became_active":"2024-03-28T13:09:41.674420+0000","last_became_peered":"2024-03-28T13:09:41.674420+0000","last_unstale":"2024-03-28T13:09:42.686871+0000","last_undegraded":"2024-03-28T13:09:42.686871+0000","last_fullsized":"2024-03-28T13:09:42.686871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:48.181534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684993+0000","last_change":"2024-03-28T13:09:41.675625+0000","last_active":"2024-03-28T13:09:42.684993+0000","last_peered":"2024-03-28T13:09:42.684993+0000","last_clean":"2024-03-28T13:09:42.684993+0000","last_became_active":"2024-03-28T13:09:41.674475+0000","last_became_peered":"2024-03-28T13:09:41.674475+0000","last_unstale":"2024-03-28T13:09:42.684993+0000","last_undegraded":"2024-03-28T13:09:42.684993+0000","last_fullsized":"2024-03-28T13:09:42.684993+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:51:59.688473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944380+0000","last_change":"2024-03-28T13:09:41.689798+0000","last_active":"2024-03-28T13:09:42.944380+0000","last_peered":"2024-03-28T13:09:42.944380+0000","last_clean":"2024-03-28T13:09:42.944380+0000","last_became_active":"2024-03-28T13:09:41.689522+0000","last_became_peered":"2024-03-28T13:09:41.689522+0000","last_unstale":"2024-03-28T13:09:42.944380+0000","last_undegraded":"2024-03-28T13:09:42.944380+0000","last_fullsized":"2024-03-28T13:09:42.944380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:02:28.201366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.679983+0000","last_change":"2024-03-28T13:09:41.693508+0000","last_active":"2024-03-28T13:09:42.679983+0000","last_peered":"2024-03-28T13:09:42.679983+0000","last_clean":"2024-03-28T13:09:42.679983+0000","last_became_active":"2024-03-28T13:09:41.692947+0000","last_became_peered":"2024-03-28T13:09:41.692947+0000","last_unstale":"2024-03-28T13:09:42.679983+0000","last_undegraded":"2024-03-28T13:09:42.679983+0000","last_fullsized":"2024-03-28T13:09:42.679983+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:54.213310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942108+0000","last_change":"2024-03-28T13:09:41.681713+0000","last_active":"2024-03-28T13:09:42.942108+0000","last_peered":"2024-03-28T13:09:42.942108+0000","last_clean":"2024-03-28T13:09:42.942108+0000","last_became_active":"2024-03-28T13:09:41.681410+0000","last_became_peered":"2024-03-28T13:09:41.681410+0000","last_unstale":"2024-03-28T13:09:42.942108+0000","last_undegraded":"2024-03-28T13:09:42.942108+0000","last_fullsized":"2024-03-28T13:09:42.942108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:54.213310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686916+0000","last_change":"2024-03-28T13:09:40.648316+0000","last_active":"2024-03-28T13:09:42.686916+0000","last_peered":"2024-03-28T13:09:42.686916+0000","last_clean":"2024-03-28T13:09:42.686916+0000","last_became_active":"2024-03-28T13:09:40.647964+0000","last_became_peered":"2024-03-28T13:09:40.647964+0000","last_unstale":"2024-03-28T13:09:42.686916+0000","last_undegraded":"2024-03-28T13:09:42.686916+0000","last_fullsized":"2024-03-28T13:09:42.686916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691106+0000","last_change":"2024-03-28T13:09:41.692864+0000","last_active":"2024-03-28T13:09:42.691106+0000","last_peered":"2024-03-28T13:09:42.691106+0000","last_clean":"2024-03-28T13:09:42.691106+0000","last_became_active":"2024-03-28T13:09:41.692536+0000","last_became_peered":"2024-03-28T13:09:41.692536+0000","last_unstale":"2024-03-28T13:09:42.691106+0000","last_undegraded":"2024-03-28T13:09:42.691106+0000","last_fullsized":"2024-03-28T13:09:42.691106+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:21:02.278070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685041+0000","last_change":"2024-03-28T13:09:40.655262+0000","last_active":"2024-03-28T13:09:42.685041+0000","last_peered":"2024-03-28T13:09:42.685041+0000","last_clean":"2024-03-28T13:09:42.685041+0000","last_became_active":"2024-03-28T13:09:40.654238+0000","last_became_peered":"2024-03-28T13:09:40.654238+0000","last_unstale":"2024-03-28T13:09:42.685041+0000","last_undegraded":"2024-03-28T13:09:42.685041+0000","last_fullsized":"2024-03-28T13:09:42.685041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:53.208363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945101+0000","last_change":"2024-03-28T13:09:40.651989+0000","last_active":"2024-03-28T13:09:42.945101+0000","last_peered":"2024-03-28T13:09:42.945101+0000","last_clean":"2024-03-28T13:09:42.945101+0000","last_became_active":"2024-03-28T13:09:40.651663+0000","last_became_peered":"2024-03-28T13:09:40.651663+0000","last_unstale":"2024-03-28T13:09:42.945101+0000","last_undegraded":"2024-03-28T13:09:42.945101+0000","last_fullsized":"2024-03-28T13:09:42.945101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:04.285637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687309+0000","last_change":"2024-03-28T13:09:41.667699+0000","last_active":"2024-03-28T13:09:42.687309+0000","last_peered":"2024-03-28T13:09:42.687309+0000","last_clean":"2024-03-28T13:09:42.687309+0000","last_became_active":"2024-03-28T13:09:41.667238+0000","last_became_peered":"2024-03-28T13:09:41.667238+0000","last_unstale":"2024-03-28T13:09:42.687309+0000","last_undegraded":"2024-03-28T13:09:42.687309+0000","last_fullsized":"2024-03-28T13:09:42.687309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:03:17.731528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684468+0000","last_change":"2024-03-28T13:09:40.644508+0000","last_active":"2024-03-28T13:09:42.684468+0000","last_peered":"2024-03-28T13:09:42.684468+0000","last_clean":"2024-03-28T13:09:42.684468+0000","last_became_active":"2024-03-28T13:09:40.644194+0000","last_became_peered":"2024-03-28T13:09:40.644194+0000","last_unstale":"2024-03-28T13:09:42.684468+0000","last_undegraded":"2024-03-28T13:09:42.684468+0000","last_fullsized":"2024-03-28T13:09:42.684468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:36.024913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691016+0000","last_change":"2024-03-28T13:09:41.689932+0000","last_active":"2024-03-28T13:09:42.691016+0000","last_peered":"2024-03-28T13:09:42.691016+0000","last_clean":"2024-03-28T13:09:42.691016+0000","last_became_active":"2024-03-28T13:09:41.689527+0000","last_became_peered":"2024-03-28T13:09:41.689527+0000","last_unstale":"2024-03-28T13:09:42.691016+0000","last_undegraded":"2024-03-28T13:09:42.691016+0000","last_fullsized":"2024-03-28T13:09:42.691016+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:16.926876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942367+0000","last_change":"2024-03-28T13:09:40.648693+0000","last_active":"2024-03-28T13:09:42.942367+0000","last_peered":"2024-03-28T13:09:42.942367+0000","last_clean":"2024-03-28T13:09:42.942367+0000","last_became_active":"2024-03-28T13:09:40.648368+0000","last_became_peered":"2024-03-28T13:09:40.648368+0000","last_unstale":"2024-03-28T13:09:42.942367+0000","last_undegraded":"2024-03-28T13:09:42.942367+0000","last_fullsized":"2024-03-28T13:09:42.942367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:04.285637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680453+0000","last_change":"2024-03-28T13:09:41.696265+0000","last_active":"2024-03-28T13:09:42.680453+0000","last_peered":"2024-03-28T13:09:42.680453+0000","last_clean":"2024-03-28T13:09:42.680453+0000","last_became_active":"2024-03-28T13:09:41.695813+0000","last_became_peered":"2024-03-28T13:09:41.695813+0000","last_unstale":"2024-03-28T13:09:42.680453+0000","last_undegraded":"2024-03-28T13:09:42.680453+0000","last_fullsized":"2024-03-28T13:09:42.680453+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:55:08.842535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696109+0000","last_change":"2024-03-28T13:09:40.645526+0000","last_active":"2024-03-28T13:09:42.696109+0000","last_peered":"2024-03-28T13:09:42.696109+0000","last_clean":"2024-03-28T13:09:42.696109+0000","last_became_active":"2024-03-28T13:09:40.644528+0000","last_became_peered":"2024-03-28T13:09:40.644528+0000","last_unstale":"2024-03-28T13:09:42.696109+0000","last_undegraded":"2024-03-28T13:09:42.696109+0000","last_fullsized":"2024-03-28T13:09:42.696109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:29:29.680319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688789+0000","last_change":"2024-03-28T13:09:41.681452+0000","last_active":"2024-03-28T13:09:42.688789+0000","last_peered":"2024-03-28T13:09:42.688789+0000","last_clean":"2024-03-28T13:09:42.688789+0000","last_became_active":"2024-03-28T13:09:41.681153+0000","last_became_peered":"2024-03-28T13:09:41.681153+0000","last_unstale":"2024-03-28T13:09:42.688789+0000","last_undegraded":"2024-03-28T13:09:42.688789+0000","last_fullsized":"2024-03-28T13:09:42.688789+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:38:26.092780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944138+0000","last_change":"2024-03-28T13:09:40.655107+0000","last_active":"2024-03-28T13:09:42.944138+0000","last_peered":"2024-03-28T13:09:42.944138+0000","last_clean":"2024-03-28T13:09:42.944138+0000","last_became_active":"2024-03-28T13:09:40.653157+0000","last_became_peered":"2024-03-28T13:09:40.653157+0000","last_unstale":"2024-03-28T13:09:42.944138+0000","last_undegraded":"2024-03-28T13:09:42.944138+0000","last_fullsized":"2024-03-28T13:09:42.944138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:38:32.114605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684836+0000","last_change":"2024-03-28T13:09:41.688505+0000","last_active":"2024-03-28T13:09:42.684836+0000","last_peered":"2024-03-28T13:09:42.684836+0000","last_clean":"2024-03-28T13:09:42.684836+0000","last_became_active":"2024-03-28T13:09:41.683889+0000","last_became_peered":"2024-03-28T13:09:41.683889+0000","last_unstale":"2024-03-28T13:09:42.684836+0000","last_undegraded":"2024-03-28T13:09:42.684836+0000","last_fullsized":"2024-03-28T13:09:42.684836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:58:50.776271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680192+0000","last_change":"2024-03-28T13:09:40.652146+0000","last_active":"2024-03-28T13:09:42.680192+0000","last_peered":"2024-03-28T13:09:42.680192+0000","last_clean":"2024-03-28T13:09:42.680192+0000","last_became_active":"2024-03-28T13:09:40.648949+0000","last_became_peered":"2024-03-28T13:09:40.648949+0000","last_unstale":"2024-03-28T13:09:42.680192+0000","last_undegraded":"2024-03-28T13:09:42.680192+0000","last_fullsized":"2024-03-28T13:09:42.680192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:02:19.389612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695962+0000","last_change":"2024-03-28T13:09:41.690754+0000","last_active":"2024-03-28T13:09:42.695962+0000","last_peered":"2024-03-28T13:09:42.695962+0000","last_clean":"2024-03-28T13:09:42.695962+0000","last_became_active":"2024-03-28T13:09:41.689905+0000","last_became_peered":"2024-03-28T13:09:41.689905+0000","last_unstale":"2024-03-28T13:09:42.695962+0000","last_undegraded":"2024-03-28T13:09:42.695962+0000","last_fullsized":"2024-03-28T13:09:42.695962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:28:22.739597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944021+0000","last_change":"2024-03-28T13:09:40.654520+0000","last_active":"2024-03-28T13:09:42.944021+0000","last_peered":"2024-03-28T13:09:42.944021+0000","last_clean":"2024-03-28T13:09:42.944021+0000","last_became_active":"2024-03-28T13:09:40.654016+0000","last_became_peered":"2024-03-28T13:09:40.654016+0000","last_unstale":"2024-03-28T13:09:42.944021+0000","last_undegraded":"2024-03-28T13:09:42.944021+0000","last_fullsized":"2024-03-28T13:09:42.944021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:41:42.775642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945445+0000","last_change":"2024-03-28T13:09:41.678649+0000","last_active":"2024-03-28T13:09:42.945445+0000","last_peered":"2024-03-28T13:09:42.945445+0000","last_clean":"2024-03-28T13:09:42.945445+0000","last_became_active":"2024-03-28T13:09:41.678312+0000","last_became_peered":"2024-03-28T13:09:41.678312+0000","last_unstale":"2024-03-28T13:09:42.945445+0000","last_undegraded":"2024-03-28T13:09:42.945445+0000","last_fullsized":"2024-03-28T13:09:42.945445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:44:34.513311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685122+0000","last_change":"2024-03-28T13:09:40.658418+0000","last_active":"2024-03-28T13:09:42.685122+0000","last_peered":"2024-03-28T13:09:42.685122+0000","last_clean":"2024-03-28T13:09:42.685122+0000","last_became_active":"2024-03-28T13:09:40.658089+0000","last_became_peered":"2024-03-28T13:09:40.658089+0000","last_unstale":"2024-03-28T13:09:42.685122+0000","last_undegraded":"2024-03-28T13:09:42.685122+0000","last_fullsized":"2024-03-28T13:09:42.685122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:21:16.839181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685214+0000","last_change":"2024-03-28T13:09:41.703207+0000","last_active":"2024-03-28T13:09:42.685214+0000","last_peered":"2024-03-28T13:09:42.685214+0000","last_clean":"2024-03-28T13:09:42.685214+0000","last_became_active":"2024-03-28T13:09:41.702892+0000","last_became_peered":"2024-03-28T13:09:41.702892+0000","last_unstale":"2024-03-28T13:09:42.685214+0000","last_undegraded":"2024-03-28T13:09:42.685214+0000","last_fullsized":"2024-03-28T13:09:42.685214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:55:53.479586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695697+0000","last_change":"2024-03-28T13:09:40.657534+0000","last_active":"2024-03-28T13:09:42.695697+0000","last_peered":"2024-03-28T13:09:42.695697+0000","last_clean":"2024-03-28T13:09:42.695697+0000","last_became_active":"2024-03-28T13:09:40.657186+0000","last_became_peered":"2024-03-28T13:09:40.657186+0000","last_unstale":"2024-03-28T13:09:42.695697+0000","last_undegraded":"2024-03-28T13:09:42.695697+0000","last_fullsized":"2024-03-28T13:09:42.695697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:39.886681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.941943+0000","last_change":"2024-03-28T13:09:41.687172+0000","last_active":"2024-03-28T13:09:42.941943+0000","last_peered":"2024-03-28T13:09:42.941943+0000","last_clean":"2024-03-28T13:09:42.941943+0000","last_became_active":"2024-03-28T13:09:41.686911+0000","last_became_peered":"2024-03-28T13:09:41.686911+0000","last_unstale":"2024-03-28T13:09:42.941943+0000","last_undegraded":"2024-03-28T13:09:42.941943+0000","last_fullsized":"2024-03-28T13:09:42.941943+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:40.891627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.690586+0000","last_change":"2024-03-28T13:09:40.646010+0000","last_active":"2024-03-28T13:09:42.690586+0000","last_peered":"2024-03-28T13:09:42.690586+0000","last_clean":"2024-03-28T13:09:42.690586+0000","last_became_active":"2024-03-28T13:09:40.645705+0000","last_became_peered":"2024-03-28T13:09:40.645705+0000","last_unstale":"2024-03-28T13:09:42.690586+0000","last_undegraded":"2024-03-28T13:09:42.690586+0000","last_fullsized":"2024-03-28T13:09:42.690586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684654+0000","last_change":"2024-03-28T13:09:41.685321+0000","last_active":"2024-03-28T13:09:42.684654+0000","last_peered":"2024-03-28T13:09:42.684654+0000","last_clean":"2024-03-28T13:09:42.684654+0000","last_became_active":"2024-03-28T13:09:41.684633+0000","last_became_peered":"2024-03-28T13:09:41.684633+0000","last_unstale":"2024-03-28T13:09:42.684654+0000","last_undegraded":"2024-03-28T13:09:42.684654+0000","last_fullsized":"2024-03-28T13:09:42.684654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:38:26.092780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696357+0000","last_change":"2024-03-28T13:09:40.649526+0000","last_active":"2024-03-28T13:09:42.696357+0000","last_peered":"2024-03-28T13:09:42.696357+0000","last_clean":"2024-03-28T13:09:42.696357+0000","last_became_active":"2024-03-28T13:09:40.649087+0000","last_became_peered":"2024-03-28T13:09:40.649087+0000","last_unstale":"2024-03-28T13:09:42.696357+0000","last_undegraded":"2024-03-28T13:09:42.696357+0000","last_fullsized":"2024-03-28T13:09:42.696357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:28.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680538+0000","last_change":"2024-03-28T13:09:41.704978+0000","last_active":"2024-03-28T13:09:42.680538+0000","last_peered":"2024-03-28T13:09:42.680538+0000","last_clean":"2024-03-28T13:09:42.680538+0000","last_became_active":"2024-03-28T13:09:41.703242+0000","last_became_peered":"2024-03-28T13:09:41.703242+0000","last_unstale":"2024-03-28T13:09:42.680538+0000","last_undegraded":"2024-03-28T13:09:42.680538+0000","last_fullsized":"2024-03-28T13:09:42.680538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:44:34.513311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691459+0000","last_change":"2024-03-28T13:09:40.655868+0000","last_active":"2024-03-28T13:09:42.691459+0000","last_peered":"2024-03-28T13:09:42.691459+0000","last_clean":"2024-03-28T13:09:42.691459+0000","last_became_active":"2024-03-28T13:09:40.654787+0000","last_became_peered":"2024-03-28T13:09:40.654787+0000","last_unstale":"2024-03-28T13:09:42.691459+0000","last_undegraded":"2024-03-28T13:09:42.691459+0000","last_fullsized":"2024-03-28T13:09:42.691459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:19:50.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696355+0000","last_change":"2024-03-28T13:09:41.694953+0000","last_active":"2024-03-28T13:09:42.696355+0000","last_peered":"2024-03-28T13:09:42.696355+0000","last_clean":"2024-03-28T13:09:42.696355+0000","last_became_active":"2024-03-28T13:09:41.694665+0000","last_became_peered":"2024-03-28T13:09:41.694665+0000","last_unstale":"2024-03-28T13:09:42.696355+0000","last_undegraded":"2024-03-28T13:09:42.696355+0000","last_fullsized":"2024-03-28T13:09:42.696355+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:43:20.588688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687780+0000","last_change":"2024-03-28T13:09:40.647574+0000","last_active":"2024-03-28T13:09:42.687780+0000","last_peered":"2024-03-28T13:09:42.687780+0000","last_clean":"2024-03-28T13:09:42.687780+0000","last_became_active":"2024-03-28T13:09:40.647272+0000","last_became_peered":"2024-03-28T13:09:40.647272+0000","last_unstale":"2024-03-28T13:09:42.687780+0000","last_undegraded":"2024-03-28T13:09:42.687780+0000","last_fullsized":"2024-03-28T13:09:42.687780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:42:11.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696806+0000","last_change":"2024-03-28T13:09:41.688065+0000","last_active":"2024-03-28T13:09:42.696806+0000","last_peered":"2024-03-28T13:09:42.696806+0000","last_clean":"2024-03-28T13:09:42.696806+0000","last_became_active":"2024-03-28T13:09:41.687636+0000","last_became_peered":"2024-03-28T13:09:41.687636+0000","last_unstale":"2024-03-28T13:09:42.696806+0000","last_undegraded":"2024-03-28T13:09:42.696806+0000","last_fullsized":"2024-03-28T13:09:42.696806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:00.376154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.946040+0000","last_change":"2024-03-28T13:09:40.655402+0000","last_active":"2024-03-28T13:09:42.946040+0000","last_peered":"2024-03-28T13:09:42.946040+0000","last_clean":"2024-03-28T13:09:42.946040+0000","last_became_active":"2024-03-28T13:09:40.655074+0000","last_became_peered":"2024-03-28T13:09:40.655074+0000","last_unstale":"2024-03-28T13:09:42.946040+0000","last_undegraded":"2024-03-28T13:09:42.946040+0000","last_fullsized":"2024-03-28T13:09:42.946040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685478+0000","last_change":"2024-03-28T13:09:41.691186+0000","last_active":"2024-03-28T13:09:42.685478+0000","last_peered":"2024-03-28T13:09:42.685478+0000","last_clean":"2024-03-28T13:09:42.685478+0000","last_became_active":"2024-03-28T13:09:41.690223+0000","last_became_peered":"2024-03-28T13:09:41.690223+0000","last_unstale":"2024-03-28T13:09:42.685478+0000","last_undegraded":"2024-03-28T13:09:42.685478+0000","last_fullsized":"2024-03-28T13:09:42.685478+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:15:43.557779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.941839+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:09:42.941839+0000","last_peered":"2024-03-28T13:09:42.941839+0000","last_clean":"2024-03-28T13:09:42.941839+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:09:42.941839+0000","last_undegraded":"2024-03-28T13:09:42.941839+0000","last_fullsized":"2024-03-28T13:09:42.941839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685703+0000","last_change":"2024-03-28T13:09:40.651783+0000","last_active":"2024-03-28T13:09:42.685703+0000","last_peered":"2024-03-28T13:09:42.685703+0000","last_clean":"2024-03-28T13:09:42.685703+0000","last_became_active":"2024-03-28T13:09:40.645785+0000","last_became_peered":"2024-03-28T13:09:40.645785+0000","last_unstale":"2024-03-28T13:09:42.685703+0000","last_undegraded":"2024-03-28T13:09:42.685703+0000","last_fullsized":"2024-03-28T13:09:42.685703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:55:07.837589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691146+0000","last_change":"2024-03-28T13:09:41.691719+0000","last_active":"2024-03-28T13:09:42.691146+0000","last_peered":"2024-03-28T13:09:42.691146+0000","last_clean":"2024-03-28T13:09:42.691146+0000","last_became_active":"2024-03-28T13:09:41.691250+0000","last_became_peered":"2024-03-28T13:09:41.691250+0000","last_unstale":"2024-03-28T13:09:42.691146+0000","last_undegraded":"2024-03-28T13:09:42.691146+0000","last_fullsized":"2024-03-28T13:09:42.691146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:50:46.504501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684592+0000","last_change":"2024-03-28T13:09:40.646854+0000","last_active":"2024-03-28T13:09:42.684592+0000","last_peered":"2024-03-28T13:09:42.684592+0000","last_clean":"2024-03-28T13:09:42.684592+0000","last_became_active":"2024-03-28T13:09:40.644192+0000","last_became_peered":"2024-03-28T13:09:40.644192+0000","last_unstale":"2024-03-28T13:09:42.684592+0000","last_undegraded":"2024-03-28T13:09:42.684592+0000","last_fullsized":"2024-03-28T13:09:42.684592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:25.071553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945249+0000","last_change":"2024-03-28T13:09:41.683008+0000","last_active":"2024-03-28T13:09:42.945249+0000","last_peered":"2024-03-28T13:09:42.945249+0000","last_clean":"2024-03-28T13:09:42.945249+0000","last_became_active":"2024-03-28T13:09:41.682664+0000","last_became_peered":"2024-03-28T13:09:41.682664+0000","last_unstale":"2024-03-28T13:09:42.945249+0000","last_undegraded":"2024-03-28T13:09:42.945249+0000","last_fullsized":"2024-03-28T13:09:42.945249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:37.029859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.943669+0000","last_change":"2024-03-28T13:09:40.654742+0000","last_active":"2024-03-28T13:09:42.943669+0000","last_peered":"2024-03-28T13:09:42.943669+0000","last_clean":"2024-03-28T13:09:42.943669+0000","last_became_active":"2024-03-28T13:09:40.654447+0000","last_became_peered":"2024-03-28T13:09:40.654447+0000","last_unstale":"2024-03-28T13:09:42.943669+0000","last_undegraded":"2024-03-28T13:09:42.943669+0000","last_fullsized":"2024-03-28T13:09:42.943669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:42:12.104485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687304+0000","last_change":"2024-03-28T13:09:41.677617+0000","last_active":"2024-03-28T13:09:42.687304+0000","last_peered":"2024-03-28T13:09:42.687304+0000","last_clean":"2024-03-28T13:09:42.687304+0000","last_became_active":"2024-03-28T13:09:41.677322+0000","last_became_peered":"2024-03-28T13:09:41.677322+0000","last_unstale":"2024-03-28T13:09:42.687304+0000","last_undegraded":"2024-03-28T13:09:42.687304+0000","last_fullsized":"2024-03-28T13:09:42.687304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:28:22.739597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685198+0000","last_change":"2024-03-28T13:09:40.650985+0000","last_active":"2024-03-28T13:09:42.685198+0000","last_peered":"2024-03-28T13:09:42.685198+0000","last_clean":"2024-03-28T13:09:42.685198+0000","last_became_active":"2024-03-28T13:09:40.648577+0000","last_became_peered":"2024-03-28T13:09:40.648577+0000","last_unstale":"2024-03-28T13:09:42.685198+0000","last_undegraded":"2024-03-28T13:09:42.685198+0000","last_fullsized":"2024-03-28T13:09:42.685198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:38:54.325203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695776+0000","last_change":"2024-03-28T13:09:41.691409+0000","last_active":"2024-03-28T13:09:42.695776+0000","last_peered":"2024-03-28T13:09:42.695776+0000","last_clean":"2024-03-28T13:09:42.695776+0000","last_became_active":"2024-03-28T13:09:41.690587+0000","last_became_peered":"2024-03-28T13:09:41.690587+0000","last_unstale":"2024-03-28T13:09:42.695776+0000","last_undegraded":"2024-03-28T13:09:42.695776+0000","last_fullsized":"2024-03-28T13:09:42.695776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:21:24.753764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687316+0000","last_change":"2024-03-28T13:09:40.648955+0000","last_active":"2024-03-28T13:09:42.687316+0000","last_peered":"2024-03-28T13:09:42.687316+0000","last_clean":"2024-03-28T13:09:42.687316+0000","last_became_active":"2024-03-28T13:09:40.648645+0000","last_became_peered":"2024-03-28T13:09:40.648645+0000","last_unstale":"2024-03-28T13:09:42.687316+0000","last_undegraded":"2024-03-28T13:09:42.687316+0000","last_fullsized":"2024-03-28T13:09:42.687316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:25:38.914424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.679924+0000","last_change":"2024-03-28T13:09:41.692962+0000","last_active":"2024-03-28T13:09:42.679924+0000","last_peered":"2024-03-28T13:09:42.679924+0000","last_clean":"2024-03-28T13:09:42.679924+0000","last_became_active":"2024-03-28T13:09:41.692660+0000","last_became_peered":"2024-03-28T13:09:41.692660+0000","last_unstale":"2024-03-28T13:09:42.679924+0000","last_undegraded":"2024-03-28T13:09:42.679924+0000","last_fullsized":"2024-03-28T13:09:42.679924+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:40.891627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685533+0000","last_change":"2024-03-28T13:09:40.654663+0000","last_active":"2024-03-28T13:09:42.685533+0000","last_peered":"2024-03-28T13:09:42.685533+0000","last_clean":"2024-03-28T13:09:42.685533+0000","last_became_active":"2024-03-28T13:09:40.654236+0000","last_became_peered":"2024-03-28T13:09:40.654236+0000","last_unstale":"2024-03-28T13:09:42.685533+0000","last_undegraded":"2024-03-28T13:09:42.685533+0000","last_fullsized":"2024-03-28T13:09:42.685533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:28.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.946112+0000","last_change":"2024-03-28T13:09:41.671654+0000","last_active":"2024-03-28T13:09:42.946112+0000","last_peered":"2024-03-28T13:09:42.946112+0000","last_clean":"2024-03-28T13:09:42.946112+0000","last_became_active":"2024-03-28T13:09:41.671232+0000","last_became_peered":"2024-03-28T13:09:41.671232+0000","last_unstale":"2024-03-28T13:09:42.946112+0000","last_undegraded":"2024-03-28T13:09:42.946112+0000","last_fullsized":"2024-03-28T13:09:42.946112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:21:02.278070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.694649+0000","last_change":"2024-03-28T13:09:40.651132+0000","last_active":"2024-03-28T13:09:42.694649+0000","last_peered":"2024-03-28T13:09:42.694649+0000","last_clean":"2024-03-28T13:09:42.694649+0000","last_became_active":"2024-03-28T13:09:40.650806+0000","last_became_peered":"2024-03-28T13:09:40.650806+0000","last_unstale":"2024-03-28T13:09:42.694649+0000","last_undegraded":"2024-03-28T13:09:42.694649+0000","last_fullsized":"2024-03-28T13:09:42.694649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:42:11.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691153+0000","last_change":"2024-03-28T13:09:41.686114+0000","last_active":"2024-03-28T13:09:42.691153+0000","last_peered":"2024-03-28T13:09:42.691153+0000","last_clean":"2024-03-28T13:09:42.691153+0000","last_became_active":"2024-03-28T13:09:41.685846+0000","last_became_peered":"2024-03-28T13:09:41.685846+0000","last_unstale":"2024-03-28T13:09:42.691153+0000","last_undegraded":"2024-03-28T13:09:42.691153+0000","last_fullsized":"2024-03-28T13:09:42.691153+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:28:13.777057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685901+0000","last_change":"2024-03-28T13:09:40.657051+0000","last_active":"2024-03-28T13:09:42.685901+0000","last_peered":"2024-03-28T13:09:42.685901+0000","last_clean":"2024-03-28T13:09:42.685901+0000","last_became_active":"2024-03-28T13:09:40.656680+0000","last_became_peered":"2024-03-28T13:09:40.656680+0000","last_unstale":"2024-03-28T13:09:42.685901+0000","last_undegraded":"2024-03-28T13:09:42.685901+0000","last_fullsized":"2024-03-28T13:09:42.685901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:44:33.508365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691166+0000","last_change":"2024-03-28T13:09:41.681460+0000","last_active":"2024-03-28T13:09:42.691166+0000","last_peered":"2024-03-28T13:09:42.691166+0000","last_clean":"2024-03-28T13:09:42.691166+0000","last_became_active":"2024-03-28T13:09:41.681136+0000","last_became_peered":"2024-03-28T13:09:41.681136+0000","last_unstale":"2024-03-28T13:09:42.691166+0000","last_undegraded":"2024-03-28T13:09:42.691166+0000","last_fullsized":"2024-03-28T13:09:42.691166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:48.181534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695372+0000","last_change":"2024-03-28T13:09:40.649525+0000","last_active":"2024-03-28T13:09:42.695372+0000","last_peered":"2024-03-28T13:09:42.695372+0000","last_clean":"2024-03-28T13:09:42.695372+0000","last_became_active":"2024-03-28T13:09:40.649204+0000","last_became_peered":"2024-03-28T13:09:40.649204+0000","last_unstale":"2024-03-28T13:09:42.695372+0000","last_undegraded":"2024-03-28T13:09:42.695372+0000","last_fullsized":"2024-03-28T13:09:42.695372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684678+0000","last_change":"2024-03-28T13:09:41.685557+0000","last_active":"2024-03-28T13:09:42.684678+0000","last_peered":"2024-03-28T13:09:42.684678+0000","last_clean":"2024-03-28T13:09:42.684678+0000","last_became_active":"2024-03-28T13:09:41.685000+0000","last_became_peered":"2024-03-28T13:09:41.685000+0000","last_unstale":"2024-03-28T13:09:42.684678+0000","last_undegraded":"2024-03-28T13:09:42.684678+0000","last_fullsized":"2024-03-28T13:09:42.684678+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:00.376154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.690868+0000","last_change":"2024-03-28T13:09:40.653861+0000","last_active":"2024-03-28T13:09:42.690868+0000","last_peered":"2024-03-28T13:09:42.690868+0000","last_clean":"2024-03-28T13:09:42.690868+0000","last_became_active":"2024-03-28T13:09:40.653467+0000","last_became_peered":"2024-03-28T13:09:40.653467+0000","last_unstale":"2024-03-28T13:09:42.690868+0000","last_undegraded":"2024-03-28T13:09:42.690868+0000","last_fullsized":"2024-03-28T13:09:42.690868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:28.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.690684+0000","last_change":"2024-03-28T13:09:41.677815+0000","last_active":"2024-03-28T13:09:42.690684+0000","last_peered":"2024-03-28T13:09:42.690684+0000","last_clean":"2024-03-28T13:09:42.690684+0000","last_became_active":"2024-03-28T13:09:41.677544+0000","last_became_peered":"2024-03-28T13:09:41.677544+0000","last_unstale":"2024-03-28T13:09:42.690684+0000","last_undegraded":"2024-03-28T13:09:42.690684+0000","last_fullsized":"2024-03-28T13:09:42.690684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:38:55.330149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945733+0000","last_change":"2024-03-28T13:09:40.649046+0000","last_active":"2024-03-28T13:09:42.945733+0000","last_peered":"2024-03-28T13:09:42.945733+0000","last_clean":"2024-03-28T13:09:42.945733+0000","last_became_active":"2024-03-28T13:09:40.648758+0000","last_became_peered":"2024-03-28T13:09:40.648758+0000","last_unstale":"2024-03-28T13:09:42.945733+0000","last_undegraded":"2024-03-28T13:09:42.945733+0000","last_fullsized":"2024-03-28T13:09:42.945733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:25:38.914424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944019+0000","last_change":"2024-03-28T13:09:41.685698+0000","last_active":"2024-03-28T13:09:42.944019+0000","last_peered":"2024-03-28T13:09:42.944019+0000","last_clean":"2024-03-28T13:09:42.944019+0000","last_became_active":"2024-03-28T13:09:41.685418+0000","last_became_peered":"2024-03-28T13:09:41.685418+0000","last_unstale":"2024-03-28T13:09:42.944019+0000","last_undegraded":"2024-03-28T13:09:42.944019+0000","last_fullsized":"2024-03-28T13:09:42.944019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:57:43.818964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685389+0000","last_change":"2024-03-28T13:09:40.653049+0000","last_active":"2024-03-28T13:09:42.685389+0000","last_peered":"2024-03-28T13:09:42.685389+0000","last_clean":"2024-03-28T13:09:42.685389+0000","last_became_active":"2024-03-28T13:09:40.652732+0000","last_became_peered":"2024-03-28T13:09:40.652732+0000","last_unstale":"2024-03-28T13:09:42.685389+0000","last_undegraded":"2024-03-28T13:09:42.685389+0000","last_fullsized":"2024-03-28T13:09:42.685389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:39.886681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685373+0000","last_change":"2024-03-28T13:09:41.690392+0000","last_active":"2024-03-28T13:09:42.685373+0000","last_peered":"2024-03-28T13:09:42.685373+0000","last_clean":"2024-03-28T13:09:42.685373+0000","last_became_active":"2024-03-28T13:09:41.689363+0000","last_became_peered":"2024-03-28T13:09:41.689363+0000","last_unstale":"2024-03-28T13:09:42.685373+0000","last_undegraded":"2024-03-28T13:09:42.685373+0000","last_fullsized":"2024-03-28T13:09:42.685373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:09:02.095518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685390+0000","last_change":"2024-03-28T13:09:41.700393+0000","last_active":"2024-03-28T13:09:42.685390+0000","last_peered":"2024-03-28T13:09:42.685390+0000","last_clean":"2024-03-28T13:09:42.685390+0000","last_became_active":"2024-03-28T13:09:41.700088+0000","last_became_peered":"2024-03-28T13:09:41.700088+0000","last_unstale":"2024-03-28T13:09:42.685390+0000","last_undegraded":"2024-03-28T13:09:42.685390+0000","last_fullsized":"2024-03-28T13:09:42.685390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:31.835464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696866+0000","last_change":"2024-03-28T13:09:40.648841+0000","last_active":"2024-03-28T13:09:42.696866+0000","last_peered":"2024-03-28T13:09:42.696866+0000","last_clean":"2024-03-28T13:09:42.696866+0000","last_became_active":"2024-03-28T13:09:40.646454+0000","last_became_peered":"2024-03-28T13:09:40.646454+0000","last_unstale":"2024-03-28T13:09:42.696866+0000","last_undegraded":"2024-03-28T13:09:42.696866+0000","last_fullsized":"2024-03-28T13:09:42.696866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:42:11.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942397+0000","last_change":"2024-03-28T13:09:41.705263+0000","last_active":"2024-03-28T13:09:42.942397+0000","last_peered":"2024-03-28T13:09:42.942397+0000","last_clean":"2024-03-28T13:09:42.942397+0000","last_became_active":"2024-03-28T13:09:41.704074+0000","last_became_peered":"2024-03-28T13:09:41.704074+0000","last_unstale":"2024-03-28T13:09:42.942397+0000","last_undegraded":"2024-03-28T13:09:42.942397+0000","last_fullsized":"2024-03-28T13:09:42.942397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:55:08.842535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695978+0000","last_change":"2024-03-28T13:09:40.649403+0000","last_active":"2024-03-28T13:09:42.695978+0000","last_peered":"2024-03-28T13:09:42.695978+0000","last_clean":"2024-03-28T13:09:42.695978+0000","last_became_active":"2024-03-28T13:09:40.648964+0000","last_became_peered":"2024-03-28T13:09:40.648964+0000","last_unstale":"2024-03-28T13:09:42.695978+0000","last_undegraded":"2024-03-28T13:09:42.695978+0000","last_fullsized":"2024-03-28T13:09:42.695978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:19:50.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688252+0000","last_change":"2024-03-28T13:09:41.688111+0000","last_active":"2024-03-28T13:09:42.688252+0000","last_peered":"2024-03-28T13:09:42.688252+0000","last_clean":"2024-03-28T13:09:42.688252+0000","last_became_active":"2024-03-28T13:09:41.687818+0000","last_became_peered":"2024-03-28T13:09:41.687818+0000","last_unstale":"2024-03-28T13:09:42.688252+0000","last_undegraded":"2024-03-28T13:09:42.688252+0000","last_fullsized":"2024-03-28T13:09:42.688252+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:38.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688212+0000","last_change":"2024-03-28T13:09:40.646771+0000","last_active":"2024-03-28T13:09:42.688212+0000","last_peered":"2024-03-28T13:09:42.688212+0000","last_clean":"2024-03-28T13:09:42.688212+0000","last_became_active":"2024-03-28T13:09:40.646496+0000","last_became_peered":"2024-03-28T13:09:40.646496+0000","last_unstale":"2024-03-28T13:09:42.688212+0000","last_undegraded":"2024-03-28T13:09:42.688212+0000","last_fullsized":"2024-03-28T13:09:42.688212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:19:50.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688194+0000","last_change":"2024-03-28T13:09:41.684932+0000","last_active":"2024-03-28T13:09:42.688194+0000","last_peered":"2024-03-28T13:09:42.688194+0000","last_clean":"2024-03-28T13:09:42.688194+0000","last_became_active":"2024-03-28T13:09:41.684314+0000","last_became_peered":"2024-03-28T13:09:41.684314+0000","last_unstale":"2024-03-28T13:09:42.688194+0000","last_undegraded":"2024-03-28T13:09:42.688194+0000","last_fullsized":"2024-03-28T13:09:42.688194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:43:20.588688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691848+0000","last_change":"2024-03-28T13:09:40.656875+0000","last_active":"2024-03-28T13:09:42.691848+0000","last_peered":"2024-03-28T13:09:42.691848+0000","last_clean":"2024-03-28T13:09:42.691848+0000","last_became_active":"2024-03-28T13:09:40.656569+0000","last_became_peered":"2024-03-28T13:09:40.656569+0000","last_unstale":"2024-03-28T13:09:42.691848+0000","last_undegraded":"2024-03-28T13:09:42.691848+0000","last_fullsized":"2024-03-28T13:09:42.691848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:29:29.680319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476745,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":80417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476743,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476743,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":670273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476743,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":79891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":79835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":831488,"data_stored":670295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":82119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":79959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:09:52.009 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:09:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.264+0000 7fdc3ac00640 1 Processor -- start 2024-03-28T13:09:52.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.264+0000 7fdc3ac00640 1 -- start start 2024-03-28T13:09:52.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.265+0000 7fdc3ac00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdc340af830 0x7fdc34073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:52.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.265+0000 7fdc3ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340d0650 0x7fdc340d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:52.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.265+0000 7fdc3ac00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc3415fe80 con 0x7fdc3415c390 2024-03-28T13:09:52.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.265+0000 7fdc3ac00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdc340d1000 con 0x7fdc340af830 2024-03-28T13:09:52.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.265+0000 7fdc3ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdc3415c770 con 0x7fdc340d0650 2024-03-28T13:09:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.266+0000 7fdc33400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdc340af830 0x7fdc34073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.266+0000 7fdc33400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdc340af830 0x7fdc34073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51004/0 (socket says 172.21.15.19:51004) 2024-03-28T13:09:52.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.266+0000 7fdc33400640 1 -- 172.21.15.19:0/1530611671 learned_addr learned my addr 172.21.15.19:0/1530611671 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:52.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.266+0000 7fdc38c00640 1 --2- 172.21.15.19:0/1530611671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340d0650 0x7fdc340d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:52.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.266+0000 7fdc33400640 1 -- 172.21.15.19:0/1530611671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340d0650 msgr2=0x7fdc340d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:52.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.266+0000 7fdc33400640 1 --2- 172.21.15.19:0/1530611671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340d0650 0x7fdc340d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:52.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.266+0000 7fdc33400640 1 -- 172.21.15.19:0/1530611671 >> v1:172.21.15.188:6789/0 conn(0x7fdc3415c390 legacy=0x7fdc34160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:52.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.266+0000 7fdc33400640 1 -- 172.21.15.19:0/1530611671 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc340ad120 con 0x7fdc340af830 2024-03-28T13:09:52.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.266+0000 7fdc38c00640 1 --2- 172.21.15.19:0/1530611671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340d0650 0x7fdc340d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:52.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.268+0000 7fdc33400640 1 --2- 172.21.15.19:0/1530611671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc340af830 0x7fdc34073790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009870 tx=0x7fdc2002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69cac5ed581ed80 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:52.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.268+0000 7fdc32a00640 1 -- 172.21.15.19:0/1530611671 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20034070 con 0x7fdc340af830 2024-03-28T13:09:52.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.268+0000 7fdc32a00640 1 -- 172.21.15.19:0/1530611671 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc20030840 con 0x7fdc340af830 2024-03-28T13:09:52.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.268+0000 7fdc32a00640 1 -- 172.21.15.19:0/1530611671 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20030c10 con 0x7fdc340af830 2024-03-28T13:09:52.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.269+0000 7fdc3ac00640 1 -- 172.21.15.19:0/1530611671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc340af830 msgr2=0x7fdc34073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:52.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.269+0000 7fdc3ac00640 1 --2- 172.21.15.19:0/1530611671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc340af830 0x7fdc34073790 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009870 tx=0x7fdc2002b980 comp rx=0 tx=0).stop 2024-03-28T13:09:52.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.269+0000 7fdc3ac00640 1 -- 172.21.15.19:0/1530611671 shutdown_connections 2024-03-28T13:09:52.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.269+0000 7fdc3ac00640 1 --2- 172.21.15.19:0/1530611671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc340af830 0x7fdc34073790 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:52.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.269+0000 7fdc3ac00640 1 --2- 172.21.15.19:0/1530611671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340d0650 0x7fdc340d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:52.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.269+0000 7fdc3ac00640 1 -- 172.21.15.19:0/1530611671 >> 172.21.15.19:0/1530611671 conn(0x7fdc3406b990 msgr2=0x7fdc3406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:52.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.269+0000 7fdc3ac00640 1 -- 172.21.15.19:0/1530611671 shutdown_connections 2024-03-28T13:09:52.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.269+0000 7fdc3ac00640 1 -- 172.21.15.19:0/1530611671 wait complete. 2024-03-28T13:09:52.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.272+0000 7fdc3ac00640 1 Processor -- start 2024-03-28T13:09:52.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.272+0000 7fdc3ac00640 1 -- start start 2024-03-28T13:09:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc3ac00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc340af830 0x7fdc3414c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc3ac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc340d0650 0x7fdc340ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc3ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc3415c390 0x7fdc341536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc3ac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc34160120 con 0x7fdc340d0650 2024-03-28T13:09:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc3ac00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdc340ad6e0 con 0x7fdc340af830 2024-03-28T13:09:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc3ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdc34160d50 con 0x7fdc3415c390 2024-03-28T13:09:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc33400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc340d0650 0x7fdc340ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc33400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc340d0650 0x7fdc340ce870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51014/0 (socket says 172.21.15.19:51014) 2024-03-28T13:09:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc33400640 1 -- 172.21.15.19:0/666197865 learned_addr learned my addr 172.21.15.19:0/666197865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc38c00640 1 --2- 172.21.15.19:0/666197865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc3415c390 0x7fdc341536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc33e00640 1 --2- 172.21.15.19:0/666197865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc340af830 0x7fdc3414c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:52.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc33400640 1 -- 172.21.15.19:0/666197865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc3415c390 msgr2=0x7fdc341536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:52.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc33400640 1 --2- 172.21.15.19:0/666197865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc3415c390 0x7fdc341536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:52.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc33400640 1 -- 172.21.15.19:0/666197865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc340af830 msgr2=0x7fdc3414c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:52.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc33400640 1 --2- 172.21.15.19:0/666197865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc340af830 0x7fdc3414c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:52.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.273+0000 7fdc33400640 1 -- 172.21.15.19:0/666197865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc3414cc40 con 0x7fdc340d0650 2024-03-28T13:09:52.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.274+0000 7fdc33400640 1 --2- 172.21.15.19:0/666197865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc340d0650 0x7fdc340ce870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdc2002b9c0 tx=0x7fdc20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.274+0000 7fdc2be00640 1 -- 172.21.15.19:0/666197865 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20048070 con 0x7fdc340d0650 2024-03-28T13:09:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.274+0000 7fdc2be00640 1 -- 172.21.15.19:0/666197865 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc20030640 con 0x7fdc340d0650 2024-03-28T13:09:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.274+0000 7fdc2be00640 1 -- 172.21.15.19:0/666197865 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc20034040 con 0x7fdc340d0650 2024-03-28T13:09:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.274+0000 7fdc3ac00640 1 -- 172.21.15.19:0/666197865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc340b3600 con 0x7fdc340d0650 2024-03-28T13:09:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.274+0000 7fdc3ac00640 1 -- 172.21.15.19:0/666197865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc340b3ea0 con 0x7fdc340d0650 2024-03-28T13:09:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.276+0000 7fdc3ac00640 1 -- 172.21.15.19:0/666197865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc340685c0 con 0x7fdc340d0650 2024-03-28T13:09:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.276+0000 7fdc2be00640 1 -- 172.21.15.19:0/666197865 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdc20037070 con 0x7fdc340d0650 2024-03-28T13:09:52.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.277+0000 7fdc2be00640 1 --2- 172.21.15.19:0/666197865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc1406c530 0x7fdc1406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:52.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.277+0000 7fdc2be00640 1 -- 172.21.15.19:0/666197865 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdc200ad3a0 con 0x7fdc340d0650 2024-03-28T13:09:52.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.278+0000 7fdc33e00640 1 --2- 172.21.15.19:0/666197865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc1406c530 0x7fdc1406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:52.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.280+0000 7fdc33e00640 1 --2- 172.21.15.19:0/666197865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc1406c530 0x7fdc1406e9f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdc24007fd0 tx=0x7fdc24009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:52.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.281+0000 7fdc2be00640 1 -- 172.21.15.19:0/666197865 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdc3414cc40 con 0x7fdc340d0650 2024-03-28T13:09:52.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.477+0000 7fdc3ac00640 1 -- 172.21.15.19:0/666197865 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdc3415bb40 con 0x7fdc1406c530 2024-03-28T13:09:52.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.495+0000 7fdc2be00640 1 -- 172.21.15.19:0/666197865 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251833 (secure 0 0 0) 0x7fdc3415bb40 con 0x7fdc1406c530 2024-03-28T13:09:52.498 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:52.500 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:09:52.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.501+0000 7fdc3ac00640 1 -- 172.21.15.19:0/666197865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc1406c530 msgr2=0x7fdc1406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:52.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.501+0000 7fdc3ac00640 1 --2- 172.21.15.19:0/666197865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc1406c530 0x7fdc1406e9f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdc24007fd0 tx=0x7fdc24009040 comp rx=0 tx=0).stop 2024-03-28T13:09:52.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.501+0000 7fdc3ac00640 1 -- 172.21.15.19:0/666197865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc340d0650 msgr2=0x7fdc340ce870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:52.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.501+0000 7fdc3ac00640 1 --2- 172.21.15.19:0/666197865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc340d0650 0x7fdc340ce870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdc2002b9c0 tx=0x7fdc20004730 comp rx=0 tx=0).stop 2024-03-28T13:09:52.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.501+0000 7fdc3ac00640 1 -- 172.21.15.19:0/666197865 shutdown_connections 2024-03-28T13:09:52.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.501+0000 7fdc3ac00640 1 --2- 172.21.15.19:0/666197865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc1406c530 0x7fdc1406e9f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:52.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.501+0000 7fdc3ac00640 1 --2- 172.21.15.19:0/666197865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc3415c390 0x7fdc341536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:52.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.501+0000 7fdc3ac00640 1 --2- 172.21.15.19:0/666197865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc340d0650 0x7fdc340ce870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:52.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.501+0000 7fdc3ac00640 1 --2- 172.21.15.19:0/666197865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc340af830 0x7fdc3414c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:52.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.501+0000 7fdc3ac00640 1 -- 172.21.15.19:0/666197865 >> 172.21.15.19:0/666197865 conn(0x7fdc3406b990 msgr2=0x7fdc3414e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:52.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.501+0000 7fdc3ac00640 1 -- 172.21.15.19:0/666197865 shutdown_connections 2024-03-28T13:09:52.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.501+0000 7fdc3ac00640 1 -- 172.21.15.19:0/666197865 wait complete. 2024-03-28T13:09:52.521 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-03-28T13:09:51.984879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219188,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755532,"statfs":{"total":773094113280,"available":772869664768,"internally_reserved":0,"allocated":3088384,"data_stored":1824236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003835"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696286+0000","last_change":"2024-03-28T13:09:41.690249+0000","last_active":"2024-03-28T13:09:42.696286+0000","last_peered":"2024-03-28T13:09:42.696286+0000","last_clean":"2024-03-28T13:09:42.696286+0000","last_became_active":"2024-03-28T13:09:41.689984+0000","last_became_peered":"2024-03-28T13:09:41.689984+0000","last_unstale":"2024-03-28T13:09:42.696286+0000","last_undegraded":"2024-03-28T13:09:42.696286+0000","last_fullsized":"2024-03-28T13:09:42.696286+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:53:08.763850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687950+0000","last_change":"2024-03-28T13:09:41.669236+0000","last_active":"2024-03-28T13:09:42.687950+0000","last_peered":"2024-03-28T13:09:42.687950+0000","last_clean":"2024-03-28T13:09:42.687950+0000","last_became_active":"2024-03-28T13:09:41.668886+0000","last_became_peered":"2024-03-28T13:09:41.668886+0000","last_unstale":"2024-03-28T13:09:42.687950+0000","last_undegraded":"2024-03-28T13:09:42.687950+0000","last_fullsized":"2024-03-28T13:09:42.687950+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:53:08.763850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687498+0000","last_change":"2024-03-28T13:09:41.683573+0000","last_active":"2024-03-28T13:09:42.687498+0000","last_peered":"2024-03-28T13:09:42.687498+0000","last_clean":"2024-03-28T13:09:42.687498+0000","last_became_active":"2024-03-28T13:09:41.682846+0000","last_became_peered":"2024-03-28T13:09:41.682846+0000","last_unstale":"2024-03-28T13:09:42.687498+0000","last_undegraded":"2024-03-28T13:09:42.687498+0000","last_fullsized":"2024-03-28T13:09:42.687498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:04.694363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680562+0000","last_change":"2024-03-28T13:09:41.684558+0000","last_active":"2024-03-28T13:09:42.680562+0000","last_peered":"2024-03-28T13:09:42.680562+0000","last_clean":"2024-03-28T13:09:42.680562+0000","last_became_active":"2024-03-28T13:09:41.682564+0000","last_became_peered":"2024-03-28T13:09:41.682564+0000","last_unstale":"2024-03-28T13:09:42.680562+0000","last_undegraded":"2024-03-28T13:09:42.680562+0000","last_fullsized":"2024-03-28T13:09:42.680562+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:21:17.844127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695969+0000","last_change":"2024-03-28T13:09:41.687175+0000","last_active":"2024-03-28T13:09:42.695969+0000","last_peered":"2024-03-28T13:09:42.695969+0000","last_clean":"2024-03-28T13:09:42.695969+0000","last_became_active":"2024-03-28T13:09:41.686835+0000","last_became_peered":"2024-03-28T13:09:41.686835+0000","last_unstale":"2024-03-28T13:09:42.695969+0000","last_undegraded":"2024-03-28T13:09:42.695969+0000","last_fullsized":"2024-03-28T13:09:42.695969+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:04.694363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.679923+0000","last_change":"2024-03-28T13:09:41.683515+0000","last_active":"2024-03-28T13:09:42.679923+0000","last_peered":"2024-03-28T13:09:42.679923+0000","last_clean":"2024-03-28T13:09:42.679923+0000","last_became_active":"2024-03-28T13:09:41.682795+0000","last_became_peered":"2024-03-28T13:09:41.682795+0000","last_unstale":"2024-03-28T13:09:42.679923+0000","last_undegraded":"2024-03-28T13:09:42.679923+0000","last_fullsized":"2024-03-28T13:09:42.679923+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:29.551203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695673+0000","last_change":"2024-03-28T13:09:41.690393+0000","last_active":"2024-03-28T13:09:42.695673+0000","last_peered":"2024-03-28T13:09:42.695673+0000","last_clean":"2024-03-28T13:09:42.695673+0000","last_became_active":"2024-03-28T13:09:41.688973+0000","last_became_peered":"2024-03-28T13:09:41.688973+0000","last_unstale":"2024-03-28T13:09:42.695673+0000","last_undegraded":"2024-03-28T13:09:42.695673+0000","last_fullsized":"2024-03-28T13:09:42.695673+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:48.181534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686877+0000","last_change":"2024-03-28T13:09:41.691319+0000","last_active":"2024-03-28T13:09:42.686877+0000","last_peered":"2024-03-28T13:09:42.686877+0000","last_clean":"2024-03-28T13:09:42.686877+0000","last_became_active":"2024-03-28T13:09:41.690506+0000","last_became_peered":"2024-03-28T13:09:41.690506+0000","last_unstale":"2024-03-28T13:09:42.686877+0000","last_undegraded":"2024-03-28T13:09:42.686877+0000","last_fullsized":"2024-03-28T13:09:42.686877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:28:13.777057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942467+0000","last_change":"2024-03-28T13:09:41.697845+0000","last_active":"2024-03-28T13:09:42.942467+0000","last_peered":"2024-03-28T13:09:42.942467+0000","last_clean":"2024-03-28T13:09:42.942467+0000","last_became_active":"2024-03-28T13:09:41.696890+0000","last_became_peered":"2024-03-28T13:09:41.696890+0000","last_unstale":"2024-03-28T13:09:42.942467+0000","last_undegraded":"2024-03-28T13:09:42.942467+0000","last_fullsized":"2024-03-28T13:09:42.942467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:37.029859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696276+0000","last_change":"2024-03-28T13:09:41.688074+0000","last_active":"2024-03-28T13:09:42.696276+0000","last_peered":"2024-03-28T13:09:42.696276+0000","last_clean":"2024-03-28T13:09:42.696276+0000","last_became_active":"2024-03-28T13:09:41.687755+0000","last_became_peered":"2024-03-28T13:09:41.687755+0000","last_unstale":"2024-03-28T13:09:42.696276+0000","last_undegraded":"2024-03-28T13:09:42.696276+0000","last_fullsized":"2024-03-28T13:09:42.696276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.673442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695935+0000","last_change":"2024-03-28T13:09:41.693370+0000","last_active":"2024-03-28T13:09:42.695935+0000","last_peered":"2024-03-28T13:09:42.695935+0000","last_clean":"2024-03-28T13:09:42.695935+0000","last_became_active":"2024-03-28T13:09:41.690289+0000","last_became_peered":"2024-03-28T13:09:41.690289+0000","last_unstale":"2024-03-28T13:09:42.695935+0000","last_undegraded":"2024-03-28T13:09:42.695935+0000","last_fullsized":"2024-03-28T13:09:42.695935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:03:17.731528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696467+0000","last_change":"2024-03-28T13:09:41.692663+0000","last_active":"2024-03-28T13:09:42.696467+0000","last_peered":"2024-03-28T13:09:42.696467+0000","last_clean":"2024-03-28T13:09:42.696467+0000","last_became_active":"2024-03-28T13:09:41.692130+0000","last_became_peered":"2024-03-28T13:09:41.692130+0000","last_unstale":"2024-03-28T13:09:42.696467+0000","last_undegraded":"2024-03-28T13:09:42.696467+0000","last_fullsized":"2024-03-28T13:09:42.696467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:38.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945637+0000","last_change":"2024-03-28T13:09:41.692414+0000","last_active":"2024-03-28T13:09:42.945637+0000","last_peered":"2024-03-28T13:09:42.945637+0000","last_clean":"2024-03-28T13:09:42.945637+0000","last_became_active":"2024-03-28T13:09:41.692105+0000","last_became_peered":"2024-03-28T13:09:41.692105+0000","last_unstale":"2024-03-28T13:09:42.945637+0000","last_undegraded":"2024-03-28T13:09:42.945637+0000","last_fullsized":"2024-03-28T13:09:42.945637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:21:17.844127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.946036+0000","last_change":"2024-03-28T13:09:41.675736+0000","last_active":"2024-03-28T13:09:42.946036+0000","last_peered":"2024-03-28T13:09:42.946036+0000","last_clean":"2024-03-28T13:09:42.946036+0000","last_became_active":"2024-03-28T13:09:41.675120+0000","last_became_peered":"2024-03-28T13:09:41.675120+0000","last_unstale":"2024-03-28T13:09:42.946036+0000","last_undegraded":"2024-03-28T13:09:42.946036+0000","last_fullsized":"2024-03-28T13:09:42.946036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:16.926876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686621+0000","last_change":"2024-03-28T13:09:41.690417+0000","last_active":"2024-03-28T13:09:42.686621+0000","last_peered":"2024-03-28T13:09:42.686621+0000","last_clean":"2024-03-28T13:09:42.686621+0000","last_became_active":"2024-03-28T13:09:41.689467+0000","last_became_peered":"2024-03-28T13:09:41.689467+0000","last_unstale":"2024-03-28T13:09:42.686621+0000","last_undegraded":"2024-03-28T13:09:42.686621+0000","last_fullsized":"2024-03-28T13:09:42.686621+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:50:46.504501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684397+0000","last_change":"2024-03-28T13:09:41.684876+0000","last_active":"2024-03-28T13:09:42.684397+0000","last_peered":"2024-03-28T13:09:42.684397+0000","last_clean":"2024-03-28T13:09:42.684397+0000","last_became_active":"2024-03-28T13:09:41.683988+0000","last_became_peered":"2024-03-28T13:09:41.683988+0000","last_unstale":"2024-03-28T13:09:42.684397+0000","last_undegraded":"2024-03-28T13:09:42.684397+0000","last_fullsized":"2024-03-28T13:09:42.684397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:38.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685867+0000","last_change":"2024-03-28T13:09:41.702219+0000","last_active":"2024-03-28T13:09:42.685867+0000","last_peered":"2024-03-28T13:09:42.685867+0000","last_clean":"2024-03-28T13:09:42.685867+0000","last_became_active":"2024-03-28T13:09:41.701907+0000","last_became_peered":"2024-03-28T13:09:41.701907+0000","last_unstale":"2024-03-28T13:09:42.685867+0000","last_undegraded":"2024-03-28T13:09:42.685867+0000","last_fullsized":"2024-03-28T13:09:42.685867+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:49:20.779575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944358+0000","last_change":"2024-03-28T13:09:41.683832+0000","last_active":"2024-03-28T13:09:42.944358+0000","last_peered":"2024-03-28T13:09:42.944358+0000","last_clean":"2024-03-28T13:09:42.944358+0000","last_became_active":"2024-03-28T13:09:41.683495+0000","last_became_peered":"2024-03-28T13:09:41.683495+0000","last_unstale":"2024-03-28T13:09:42.944358+0000","last_undegraded":"2024-03-28T13:09:42.944358+0000","last_fullsized":"2024-03-28T13:09:42.944358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:44:52.163706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685573+0000","last_change":"2024-03-28T13:09:41.688514+0000","last_active":"2024-03-28T13:09:42.685573+0000","last_peered":"2024-03-28T13:09:42.685573+0000","last_clean":"2024-03-28T13:09:42.685573+0000","last_became_active":"2024-03-28T13:09:41.688158+0000","last_became_peered":"2024-03-28T13:09:41.688158+0000","last_unstale":"2024-03-28T13:09:42.685573+0000","last_undegraded":"2024-03-28T13:09:42.685573+0000","last_fullsized":"2024-03-28T13:09:42.685573+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:39:02.380052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696639+0000","last_change":"2024-03-28T13:09:41.695024+0000","last_active":"2024-03-28T13:09:42.696639+0000","last_peered":"2024-03-28T13:09:42.696639+0000","last_clean":"2024-03-28T13:09:42.696639+0000","last_became_active":"2024-03-28T13:09:41.694698+0000","last_became_peered":"2024-03-28T13:09:41.694698+0000","last_unstale":"2024-03-28T13:09:42.696639+0000","last_undegraded":"2024-03-28T13:09:42.696639+0000","last_fullsized":"2024-03-28T13:09:42.696639+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:38:26.092780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691415+0000","last_change":"2024-03-28T13:09:41.686281+0000","last_active":"2024-03-28T13:09:42.691415+0000","last_peered":"2024-03-28T13:09:42.691415+0000","last_clean":"2024-03-28T13:09:42.691415+0000","last_became_active":"2024-03-28T13:09:41.685221+0000","last_became_peered":"2024-03-28T13:09:41.685221+0000","last_unstale":"2024-03-28T13:09:42.691415+0000","last_undegraded":"2024-03-28T13:09:42.691415+0000","last_fullsized":"2024-03-28T13:09:42.691415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:21:24.753764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.941951+0000","last_change":"2024-03-28T13:09:41.694982+0000","last_active":"2024-03-28T13:09:42.941951+0000","last_peered":"2024-03-28T13:09:42.941951+0000","last_clean":"2024-03-28T13:09:42.941951+0000","last_became_active":"2024-03-28T13:09:41.694746+0000","last_became_peered":"2024-03-28T13:09:41.694746+0000","last_unstale":"2024-03-28T13:09:42.941951+0000","last_undegraded":"2024-03-28T13:09:42.941951+0000","last_fullsized":"2024-03-28T13:09:42.941951+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:29.551203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686937+0000","last_change":"2024-03-28T13:09:41.685949+0000","last_active":"2024-03-28T13:09:42.686937+0000","last_peered":"2024-03-28T13:09:42.686937+0000","last_clean":"2024-03-28T13:09:42.686937+0000","last_became_active":"2024-03-28T13:09:41.683610+0000","last_became_peered":"2024-03-28T13:09:41.683610+0000","last_unstale":"2024-03-28T13:09:42.686937+0000","last_undegraded":"2024-03-28T13:09:42.686937+0000","last_fullsized":"2024-03-28T13:09:42.686937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:21:24.753764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684741+0000","last_change":"2024-03-28T13:09:41.672017+0000","last_active":"2024-03-28T13:09:42.684741+0000","last_peered":"2024-03-28T13:09:42.684741+0000","last_clean":"2024-03-28T13:09:42.684741+0000","last_became_active":"2024-03-28T13:09:41.671594+0000","last_became_peered":"2024-03-28T13:09:41.671594+0000","last_unstale":"2024-03-28T13:09:42.684741+0000","last_undegraded":"2024-03-28T13:09:42.684741+0000","last_fullsized":"2024-03-28T13:09:42.684741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:05:06.674720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687844+0000","last_change":"2024-03-28T13:09:41.689019+0000","last_active":"2024-03-28T13:09:42.687844+0000","last_peered":"2024-03-28T13:09:42.687844+0000","last_clean":"2024-03-28T13:09:42.687844+0000","last_became_active":"2024-03-28T13:09:41.688663+0000","last_became_peered":"2024-03-28T13:09:41.688663+0000","last_unstale":"2024-03-28T13:09:42.687844+0000","last_undegraded":"2024-03-28T13:09:42.687844+0000","last_fullsized":"2024-03-28T13:09:42.687844+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.673442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680521+0000","last_change":"2024-03-28T13:09:41.698336+0000","last_active":"2024-03-28T13:09:42.680521+0000","last_peered":"2024-03-28T13:09:42.680521+0000","last_clean":"2024-03-28T13:09:42.680521+0000","last_became_active":"2024-03-28T13:09:41.696491+0000","last_became_peered":"2024-03-28T13:09:41.696491+0000","last_unstale":"2024-03-28T13:09:42.680521+0000","last_undegraded":"2024-03-28T13:09:42.680521+0000","last_fullsized":"2024-03-28T13:09:42.680521+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:37.029859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945985+0000","last_change":"2024-03-28T13:09:41.676686+0000","last_active":"2024-03-28T13:09:42.945985+0000","last_peered":"2024-03-28T13:09:42.945985+0000","last_clean":"2024-03-28T13:09:42.945985+0000","last_became_active":"2024-03-28T13:09:41.676403+0000","last_became_peered":"2024-03-28T13:09:41.676403+0000","last_unstale":"2024-03-28T13:09:42.945985+0000","last_undegraded":"2024-03-28T13:09:42.945985+0000","last_fullsized":"2024-03-28T13:09:42.945985+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:38:55.330149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945843+0000","last_change":"2024-03-28T13:09:41.688698+0000","last_active":"2024-03-28T13:09:42.945843+0000","last_peered":"2024-03-28T13:09:42.945843+0000","last_clean":"2024-03-28T13:09:42.945843+0000","last_became_active":"2024-03-28T13:09:41.688352+0000","last_became_peered":"2024-03-28T13:09:41.688352+0000","last_unstale":"2024-03-28T13:09:42.945843+0000","last_undegraded":"2024-03-28T13:09:42.945843+0000","last_fullsized":"2024-03-28T13:09:42.945843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:26.076499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686871+0000","last_change":"2024-03-28T13:09:41.675052+0000","last_active":"2024-03-28T13:09:42.686871+0000","last_peered":"2024-03-28T13:09:42.686871+0000","last_clean":"2024-03-28T13:09:42.686871+0000","last_became_active":"2024-03-28T13:09:41.674420+0000","last_became_peered":"2024-03-28T13:09:41.674420+0000","last_unstale":"2024-03-28T13:09:42.686871+0000","last_undegraded":"2024-03-28T13:09:42.686871+0000","last_fullsized":"2024-03-28T13:09:42.686871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:48.181534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684993+0000","last_change":"2024-03-28T13:09:41.675625+0000","last_active":"2024-03-28T13:09:42.684993+0000","last_peered":"2024-03-28T13:09:42.684993+0000","last_clean":"2024-03-28T13:09:42.684993+0000","last_became_active":"2024-03-28T13:09:41.674475+0000","last_became_peered":"2024-03-28T13:09:41.674475+0000","last_unstale":"2024-03-28T13:09:42.684993+0000","last_undegraded":"2024-03-28T13:09:42.684993+0000","last_fullsized":"2024-03-28T13:09:42.684993+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:51:59.688473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944380+0000","last_change":"2024-03-28T13:09:41.689798+0000","last_active":"2024-03-28T13:09:42.944380+0000","last_peered":"2024-03-28T13:09:42.944380+0000","last_clean":"2024-03-28T13:09:42.944380+0000","last_became_active":"2024-03-28T13:09:41.689522+0000","last_became_peered":"2024-03-28T13:09:41.689522+0000","last_unstale":"2024-03-28T13:09:42.944380+0000","last_undegraded":"2024-03-28T13:09:42.944380+0000","last_fullsized":"2024-03-28T13:09:42.944380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:02:28.201366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.679983+0000","last_change":"2024-03-28T13:09:41.693508+0000","last_active":"2024-03-28T13:09:42.679983+0000","last_peered":"2024-03-28T13:09:42.679983+0000","last_clean":"2024-03-28T13:09:42.679983+0000","last_became_active":"2024-03-28T13:09:41.692947+0000","last_became_peered":"2024-03-28T13:09:41.692947+0000","last_unstale":"2024-03-28T13:09:42.679983+0000","last_undegraded":"2024-03-28T13:09:42.679983+0000","last_fullsized":"2024-03-28T13:09:42.679983+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:54.213310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942108+0000","last_change":"2024-03-28T13:09:41.681713+0000","last_active":"2024-03-28T13:09:42.942108+0000","last_peered":"2024-03-28T13:09:42.942108+0000","last_clean":"2024-03-28T13:09:42.942108+0000","last_became_active":"2024-03-28T13:09:41.681410+0000","last_became_peered":"2024-03-28T13:09:41.681410+0000","last_unstale":"2024-03-28T13:09:42.942108+0000","last_undegraded":"2024-03-28T13:09:42.942108+0000","last_fullsized":"2024-03-28T13:09:42.942108+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:54.213310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.686916+0000","last_change":"2024-03-28T13:09:40.648316+0000","last_active":"2024-03-28T13:09:42.686916+0000","last_peered":"2024-03-28T13:09:42.686916+0000","last_clean":"2024-03-28T13:09:42.686916+0000","last_became_active":"2024-03-28T13:09:40.647964+0000","last_became_peered":"2024-03-28T13:09:40.647964+0000","last_unstale":"2024-03-28T13:09:42.686916+0000","last_undegraded":"2024-03-28T13:09:42.686916+0000","last_fullsized":"2024-03-28T13:09:42.686916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691106+0000","last_change":"2024-03-28T13:09:41.692864+0000","last_active":"2024-03-28T13:09:42.691106+0000","last_peered":"2024-03-28T13:09:42.691106+0000","last_clean":"2024-03-28T13:09:42.691106+0000","last_became_active":"2024-03-28T13:09:41.692536+0000","last_became_peered":"2024-03-28T13:09:41.692536+0000","last_unstale":"2024-03-28T13:09:42.691106+0000","last_undegraded":"2024-03-28T13:09:42.691106+0000","last_fullsized":"2024-03-28T13:09:42.691106+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:21:02.278070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685041+0000","last_change":"2024-03-28T13:09:40.655262+0000","last_active":"2024-03-28T13:09:42.685041+0000","last_peered":"2024-03-28T13:09:42.685041+0000","last_clean":"2024-03-28T13:09:42.685041+0000","last_became_active":"2024-03-28T13:09:40.654238+0000","last_became_peered":"2024-03-28T13:09:40.654238+0000","last_unstale":"2024-03-28T13:09:42.685041+0000","last_undegraded":"2024-03-28T13:09:42.685041+0000","last_fullsized":"2024-03-28T13:09:42.685041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:45:53.208363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945101+0000","last_change":"2024-03-28T13:09:40.651989+0000","last_active":"2024-03-28T13:09:42.945101+0000","last_peered":"2024-03-28T13:09:42.945101+0000","last_clean":"2024-03-28T13:09:42.945101+0000","last_became_active":"2024-03-28T13:09:40.651663+0000","last_became_peered":"2024-03-28T13:09:40.651663+0000","last_unstale":"2024-03-28T13:09:42.945101+0000","last_undegraded":"2024-03-28T13:09:42.945101+0000","last_fullsized":"2024-03-28T13:09:42.945101+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:04.285637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687309+0000","last_change":"2024-03-28T13:09:41.667699+0000","last_active":"2024-03-28T13:09:42.687309+0000","last_peered":"2024-03-28T13:09:42.687309+0000","last_clean":"2024-03-28T13:09:42.687309+0000","last_became_active":"2024-03-28T13:09:41.667238+0000","last_became_peered":"2024-03-28T13:09:41.667238+0000","last_unstale":"2024-03-28T13:09:42.687309+0000","last_undegraded":"2024-03-28T13:09:42.687309+0000","last_fullsized":"2024-03-28T13:09:42.687309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:03:17.731528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684468+0000","last_change":"2024-03-28T13:09:40.644508+0000","last_active":"2024-03-28T13:09:42.684468+0000","last_peered":"2024-03-28T13:09:42.684468+0000","last_clean":"2024-03-28T13:09:42.684468+0000","last_became_active":"2024-03-28T13:09:40.644194+0000","last_became_peered":"2024-03-28T13:09:40.644194+0000","last_unstale":"2024-03-28T13:09:42.684468+0000","last_undegraded":"2024-03-28T13:09:42.684468+0000","last_fullsized":"2024-03-28T13:09:42.684468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:36.024913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691016+0000","last_change":"2024-03-28T13:09:41.689932+0000","last_active":"2024-03-28T13:09:42.691016+0000","last_peered":"2024-03-28T13:09:42.691016+0000","last_clean":"2024-03-28T13:09:42.691016+0000","last_became_active":"2024-03-28T13:09:41.689527+0000","last_became_peered":"2024-03-28T13:09:41.689527+0000","last_unstale":"2024-03-28T13:09:42.691016+0000","last_undegraded":"2024-03-28T13:09:42.691016+0000","last_fullsized":"2024-03-28T13:09:42.691016+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:16.926876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942367+0000","last_change":"2024-03-28T13:09:40.648693+0000","last_active":"2024-03-28T13:09:42.942367+0000","last_peered":"2024-03-28T13:09:42.942367+0000","last_clean":"2024-03-28T13:09:42.942367+0000","last_became_active":"2024-03-28T13:09:40.648368+0000","last_became_peered":"2024-03-28T13:09:40.648368+0000","last_unstale":"2024-03-28T13:09:42.942367+0000","last_undegraded":"2024-03-28T13:09:42.942367+0000","last_fullsized":"2024-03-28T13:09:42.942367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:04.285637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680453+0000","last_change":"2024-03-28T13:09:41.696265+0000","last_active":"2024-03-28T13:09:42.680453+0000","last_peered":"2024-03-28T13:09:42.680453+0000","last_clean":"2024-03-28T13:09:42.680453+0000","last_became_active":"2024-03-28T13:09:41.695813+0000","last_became_peered":"2024-03-28T13:09:41.695813+0000","last_unstale":"2024-03-28T13:09:42.680453+0000","last_undegraded":"2024-03-28T13:09:42.680453+0000","last_fullsized":"2024-03-28T13:09:42.680453+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:55:08.842535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696109+0000","last_change":"2024-03-28T13:09:40.645526+0000","last_active":"2024-03-28T13:09:42.696109+0000","last_peered":"2024-03-28T13:09:42.696109+0000","last_clean":"2024-03-28T13:09:42.696109+0000","last_became_active":"2024-03-28T13:09:40.644528+0000","last_became_peered":"2024-03-28T13:09:40.644528+0000","last_unstale":"2024-03-28T13:09:42.696109+0000","last_undegraded":"2024-03-28T13:09:42.696109+0000","last_fullsized":"2024-03-28T13:09:42.696109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:29:29.680319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688789+0000","last_change":"2024-03-28T13:09:41.681452+0000","last_active":"2024-03-28T13:09:42.688789+0000","last_peered":"2024-03-28T13:09:42.688789+0000","last_clean":"2024-03-28T13:09:42.688789+0000","last_became_active":"2024-03-28T13:09:41.681153+0000","last_became_peered":"2024-03-28T13:09:41.681153+0000","last_unstale":"2024-03-28T13:09:42.688789+0000","last_undegraded":"2024-03-28T13:09:42.688789+0000","last_fullsized":"2024-03-28T13:09:42.688789+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:38:26.092780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944138+0000","last_change":"2024-03-28T13:09:40.655107+0000","last_active":"2024-03-28T13:09:42.944138+0000","last_peered":"2024-03-28T13:09:42.944138+0000","last_clean":"2024-03-28T13:09:42.944138+0000","last_became_active":"2024-03-28T13:09:40.653157+0000","last_became_peered":"2024-03-28T13:09:40.653157+0000","last_unstale":"2024-03-28T13:09:42.944138+0000","last_undegraded":"2024-03-28T13:09:42.944138+0000","last_fullsized":"2024-03-28T13:09:42.944138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:38:32.114605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684836+0000","last_change":"2024-03-28T13:09:41.688505+0000","last_active":"2024-03-28T13:09:42.684836+0000","last_peered":"2024-03-28T13:09:42.684836+0000","last_clean":"2024-03-28T13:09:42.684836+0000","last_became_active":"2024-03-28T13:09:41.683889+0000","last_became_peered":"2024-03-28T13:09:41.683889+0000","last_unstale":"2024-03-28T13:09:42.684836+0000","last_undegraded":"2024-03-28T13:09:42.684836+0000","last_fullsized":"2024-03-28T13:09:42.684836+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:58:50.776271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680192+0000","last_change":"2024-03-28T13:09:40.652146+0000","last_active":"2024-03-28T13:09:42.680192+0000","last_peered":"2024-03-28T13:09:42.680192+0000","last_clean":"2024-03-28T13:09:42.680192+0000","last_became_active":"2024-03-28T13:09:40.648949+0000","last_became_peered":"2024-03-28T13:09:40.648949+0000","last_unstale":"2024-03-28T13:09:42.680192+0000","last_undegraded":"2024-03-28T13:09:42.680192+0000","last_fullsized":"2024-03-28T13:09:42.680192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:02:19.389612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695962+0000","last_change":"2024-03-28T13:09:41.690754+0000","last_active":"2024-03-28T13:09:42.695962+0000","last_peered":"2024-03-28T13:09:42.695962+0000","last_clean":"2024-03-28T13:09:42.695962+0000","last_became_active":"2024-03-28T13:09:41.689905+0000","last_became_peered":"2024-03-28T13:09:41.689905+0000","last_unstale":"2024-03-28T13:09:42.695962+0000","last_undegraded":"2024-03-28T13:09:42.695962+0000","last_fullsized":"2024-03-28T13:09:42.695962+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:28:22.739597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944021+0000","last_change":"2024-03-28T13:09:40.654520+0000","last_active":"2024-03-28T13:09:42.944021+0000","last_peered":"2024-03-28T13:09:42.944021+0000","last_clean":"2024-03-28T13:09:42.944021+0000","last_became_active":"2024-03-28T13:09:40.654016+0000","last_became_peered":"2024-03-28T13:09:40.654016+0000","last_unstale":"2024-03-28T13:09:42.944021+0000","last_undegraded":"2024-03-28T13:09:42.944021+0000","last_fullsized":"2024-03-28T13:09:42.944021+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:41:42.775642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945445+0000","last_change":"2024-03-28T13:09:41.678649+0000","last_active":"2024-03-28T13:09:42.945445+0000","last_peered":"2024-03-28T13:09:42.945445+0000","last_clean":"2024-03-28T13:09:42.945445+0000","last_became_active":"2024-03-28T13:09:41.678312+0000","last_became_peered":"2024-03-28T13:09:41.678312+0000","last_unstale":"2024-03-28T13:09:42.945445+0000","last_undegraded":"2024-03-28T13:09:42.945445+0000","last_fullsized":"2024-03-28T13:09:42.945445+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:44:34.513311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685122+0000","last_change":"2024-03-28T13:09:40.658418+0000","last_active":"2024-03-28T13:09:42.685122+0000","last_peered":"2024-03-28T13:09:42.685122+0000","last_clean":"2024-03-28T13:09:42.685122+0000","last_became_active":"2024-03-28T13:09:40.658089+0000","last_became_peered":"2024-03-28T13:09:40.658089+0000","last_unstale":"2024-03-28T13:09:42.685122+0000","last_undegraded":"2024-03-28T13:09:42.685122+0000","last_fullsized":"2024-03-28T13:09:42.685122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:21:16.839181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685214+0000","last_change":"2024-03-28T13:09:41.703207+0000","last_active":"2024-03-28T13:09:42.685214+0000","last_peered":"2024-03-28T13:09:42.685214+0000","last_clean":"2024-03-28T13:09:42.685214+0000","last_became_active":"2024-03-28T13:09:41.702892+0000","last_became_peered":"2024-03-28T13:09:41.702892+0000","last_unstale":"2024-03-28T13:09:42.685214+0000","last_undegraded":"2024-03-28T13:09:42.685214+0000","last_fullsized":"2024-03-28T13:09:42.685214+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:55:53.479586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695697+0000","last_change":"2024-03-28T13:09:40.657534+0000","last_active":"2024-03-28T13:09:42.695697+0000","last_peered":"2024-03-28T13:09:42.695697+0000","last_clean":"2024-03-28T13:09:42.695697+0000","last_became_active":"2024-03-28T13:09:40.657186+0000","last_became_peered":"2024-03-28T13:09:40.657186+0000","last_unstale":"2024-03-28T13:09:42.695697+0000","last_undegraded":"2024-03-28T13:09:42.695697+0000","last_fullsized":"2024-03-28T13:09:42.695697+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:39.886681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.941943+0000","last_change":"2024-03-28T13:09:41.687172+0000","last_active":"2024-03-28T13:09:42.941943+0000","last_peered":"2024-03-28T13:09:42.941943+0000","last_clean":"2024-03-28T13:09:42.941943+0000","last_became_active":"2024-03-28T13:09:41.686911+0000","last_became_peered":"2024-03-28T13:09:41.686911+0000","last_unstale":"2024-03-28T13:09:42.941943+0000","last_undegraded":"2024-03-28T13:09:42.941943+0000","last_fullsized":"2024-03-28T13:09:42.941943+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:40.891627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.690586+0000","last_change":"2024-03-28T13:09:40.646010+0000","last_active":"2024-03-28T13:09:42.690586+0000","last_peered":"2024-03-28T13:09:42.690586+0000","last_clean":"2024-03-28T13:09:42.690586+0000","last_became_active":"2024-03-28T13:09:40.645705+0000","last_became_peered":"2024-03-28T13:09:40.645705+0000","last_unstale":"2024-03-28T13:09:42.690586+0000","last_undegraded":"2024-03-28T13:09:42.690586+0000","last_fullsized":"2024-03-28T13:09:42.690586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684654+0000","last_change":"2024-03-28T13:09:41.685321+0000","last_active":"2024-03-28T13:09:42.684654+0000","last_peered":"2024-03-28T13:09:42.684654+0000","last_clean":"2024-03-28T13:09:42.684654+0000","last_became_active":"2024-03-28T13:09:41.684633+0000","last_became_peered":"2024-03-28T13:09:41.684633+0000","last_unstale":"2024-03-28T13:09:42.684654+0000","last_undegraded":"2024-03-28T13:09:42.684654+0000","last_fullsized":"2024-03-28T13:09:42.684654+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:38:26.092780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696357+0000","last_change":"2024-03-28T13:09:40.649526+0000","last_active":"2024-03-28T13:09:42.696357+0000","last_peered":"2024-03-28T13:09:42.696357+0000","last_clean":"2024-03-28T13:09:42.696357+0000","last_became_active":"2024-03-28T13:09:40.649087+0000","last_became_peered":"2024-03-28T13:09:40.649087+0000","last_unstale":"2024-03-28T13:09:42.696357+0000","last_undegraded":"2024-03-28T13:09:42.696357+0000","last_fullsized":"2024-03-28T13:09:42.696357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:28.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.680538+0000","last_change":"2024-03-28T13:09:41.704978+0000","last_active":"2024-03-28T13:09:42.680538+0000","last_peered":"2024-03-28T13:09:42.680538+0000","last_clean":"2024-03-28T13:09:42.680538+0000","last_became_active":"2024-03-28T13:09:41.703242+0000","last_became_peered":"2024-03-28T13:09:41.703242+0000","last_unstale":"2024-03-28T13:09:42.680538+0000","last_undegraded":"2024-03-28T13:09:42.680538+0000","last_fullsized":"2024-03-28T13:09:42.680538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:44:34.513311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691459+0000","last_change":"2024-03-28T13:09:40.655868+0000","last_active":"2024-03-28T13:09:42.691459+0000","last_peered":"2024-03-28T13:09:42.691459+0000","last_clean":"2024-03-28T13:09:42.691459+0000","last_became_active":"2024-03-28T13:09:40.654787+0000","last_became_peered":"2024-03-28T13:09:40.654787+0000","last_unstale":"2024-03-28T13:09:42.691459+0000","last_undegraded":"2024-03-28T13:09:42.691459+0000","last_fullsized":"2024-03-28T13:09:42.691459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:19:50.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696355+0000","last_change":"2024-03-28T13:09:41.694953+0000","last_active":"2024-03-28T13:09:42.696355+0000","last_peered":"2024-03-28T13:09:42.696355+0000","last_clean":"2024-03-28T13:09:42.696355+0000","last_became_active":"2024-03-28T13:09:41.694665+0000","last_became_peered":"2024-03-28T13:09:41.694665+0000","last_unstale":"2024-03-28T13:09:42.696355+0000","last_undegraded":"2024-03-28T13:09:42.696355+0000","last_fullsized":"2024-03-28T13:09:42.696355+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:43:20.588688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687780+0000","last_change":"2024-03-28T13:09:40.647574+0000","last_active":"2024-03-28T13:09:42.687780+0000","last_peered":"2024-03-28T13:09:42.687780+0000","last_clean":"2024-03-28T13:09:42.687780+0000","last_became_active":"2024-03-28T13:09:40.647272+0000","last_became_peered":"2024-03-28T13:09:40.647272+0000","last_unstale":"2024-03-28T13:09:42.687780+0000","last_undegraded":"2024-03-28T13:09:42.687780+0000","last_fullsized":"2024-03-28T13:09:42.687780+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:42:11.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696806+0000","last_change":"2024-03-28T13:09:41.688065+0000","last_active":"2024-03-28T13:09:42.696806+0000","last_peered":"2024-03-28T13:09:42.696806+0000","last_clean":"2024-03-28T13:09:42.696806+0000","last_became_active":"2024-03-28T13:09:41.687636+0000","last_became_peered":"2024-03-28T13:09:41.687636+0000","last_unstale":"2024-03-28T13:09:42.696806+0000","last_undegraded":"2024-03-28T13:09:42.696806+0000","last_fullsized":"2024-03-28T13:09:42.696806+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:00.376154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.946040+0000","last_change":"2024-03-28T13:09:40.655402+0000","last_active":"2024-03-28T13:09:42.946040+0000","last_peered":"2024-03-28T13:09:42.946040+0000","last_clean":"2024-03-28T13:09:42.946040+0000","last_became_active":"2024-03-28T13:09:40.655074+0000","last_became_peered":"2024-03-28T13:09:40.655074+0000","last_unstale":"2024-03-28T13:09:42.946040+0000","last_undegraded":"2024-03-28T13:09:42.946040+0000","last_fullsized":"2024-03-28T13:09:42.946040+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685478+0000","last_change":"2024-03-28T13:09:41.691186+0000","last_active":"2024-03-28T13:09:42.685478+0000","last_peered":"2024-03-28T13:09:42.685478+0000","last_clean":"2024-03-28T13:09:42.685478+0000","last_became_active":"2024-03-28T13:09:41.690223+0000","last_became_peered":"2024-03-28T13:09:41.690223+0000","last_unstale":"2024-03-28T13:09:42.685478+0000","last_undegraded":"2024-03-28T13:09:42.685478+0000","last_fullsized":"2024-03-28T13:09:42.685478+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:15:43.557779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":170,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.941839+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:09:42.941839+0000","last_peered":"2024-03-28T13:09:42.941839+0000","last_clean":"2024-03-28T13:09:42.941839+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:09:42.941839+0000","last_undegraded":"2024-03-28T13:09:42.941839+0000","last_fullsized":"2024-03-28T13:09:42.941839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685703+0000","last_change":"2024-03-28T13:09:40.651783+0000","last_active":"2024-03-28T13:09:42.685703+0000","last_peered":"2024-03-28T13:09:42.685703+0000","last_clean":"2024-03-28T13:09:42.685703+0000","last_became_active":"2024-03-28T13:09:40.645785+0000","last_became_peered":"2024-03-28T13:09:40.645785+0000","last_unstale":"2024-03-28T13:09:42.685703+0000","last_undegraded":"2024-03-28T13:09:42.685703+0000","last_fullsized":"2024-03-28T13:09:42.685703+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:55:07.837589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691146+0000","last_change":"2024-03-28T13:09:41.691719+0000","last_active":"2024-03-28T13:09:42.691146+0000","last_peered":"2024-03-28T13:09:42.691146+0000","last_clean":"2024-03-28T13:09:42.691146+0000","last_became_active":"2024-03-28T13:09:41.691250+0000","last_became_peered":"2024-03-28T13:09:41.691250+0000","last_unstale":"2024-03-28T13:09:42.691146+0000","last_undegraded":"2024-03-28T13:09:42.691146+0000","last_fullsized":"2024-03-28T13:09:42.691146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:50:46.504501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684592+0000","last_change":"2024-03-28T13:09:40.646854+0000","last_active":"2024-03-28T13:09:42.684592+0000","last_peered":"2024-03-28T13:09:42.684592+0000","last_clean":"2024-03-28T13:09:42.684592+0000","last_became_active":"2024-03-28T13:09:40.644192+0000","last_became_peered":"2024-03-28T13:09:40.644192+0000","last_unstale":"2024-03-28T13:09:42.684592+0000","last_undegraded":"2024-03-28T13:09:42.684592+0000","last_fullsized":"2024-03-28T13:09:42.684592+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:25.071553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945249+0000","last_change":"2024-03-28T13:09:41.683008+0000","last_active":"2024-03-28T13:09:42.945249+0000","last_peered":"2024-03-28T13:09:42.945249+0000","last_clean":"2024-03-28T13:09:42.945249+0000","last_became_active":"2024-03-28T13:09:41.682664+0000","last_became_peered":"2024-03-28T13:09:41.682664+0000","last_unstale":"2024-03-28T13:09:42.945249+0000","last_undegraded":"2024-03-28T13:09:42.945249+0000","last_fullsized":"2024-03-28T13:09:42.945249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:37.029859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.943669+0000","last_change":"2024-03-28T13:09:40.654742+0000","last_active":"2024-03-28T13:09:42.943669+0000","last_peered":"2024-03-28T13:09:42.943669+0000","last_clean":"2024-03-28T13:09:42.943669+0000","last_became_active":"2024-03-28T13:09:40.654447+0000","last_became_peered":"2024-03-28T13:09:40.654447+0000","last_unstale":"2024-03-28T13:09:42.943669+0000","last_undegraded":"2024-03-28T13:09:42.943669+0000","last_fullsized":"2024-03-28T13:09:42.943669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:42:12.104485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687304+0000","last_change":"2024-03-28T13:09:41.677617+0000","last_active":"2024-03-28T13:09:42.687304+0000","last_peered":"2024-03-28T13:09:42.687304+0000","last_clean":"2024-03-28T13:09:42.687304+0000","last_became_active":"2024-03-28T13:09:41.677322+0000","last_became_peered":"2024-03-28T13:09:41.677322+0000","last_unstale":"2024-03-28T13:09:42.687304+0000","last_undegraded":"2024-03-28T13:09:42.687304+0000","last_fullsized":"2024-03-28T13:09:42.687304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:28:22.739597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685198+0000","last_change":"2024-03-28T13:09:40.650985+0000","last_active":"2024-03-28T13:09:42.685198+0000","last_peered":"2024-03-28T13:09:42.685198+0000","last_clean":"2024-03-28T13:09:42.685198+0000","last_became_active":"2024-03-28T13:09:40.648577+0000","last_became_peered":"2024-03-28T13:09:40.648577+0000","last_unstale":"2024-03-28T13:09:42.685198+0000","last_undegraded":"2024-03-28T13:09:42.685198+0000","last_fullsized":"2024-03-28T13:09:42.685198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:38:54.325203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695776+0000","last_change":"2024-03-28T13:09:41.691409+0000","last_active":"2024-03-28T13:09:42.695776+0000","last_peered":"2024-03-28T13:09:42.695776+0000","last_clean":"2024-03-28T13:09:42.695776+0000","last_became_active":"2024-03-28T13:09:41.690587+0000","last_became_peered":"2024-03-28T13:09:41.690587+0000","last_unstale":"2024-03-28T13:09:42.695776+0000","last_undegraded":"2024-03-28T13:09:42.695776+0000","last_fullsized":"2024-03-28T13:09:42.695776+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:21:24.753764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.687316+0000","last_change":"2024-03-28T13:09:40.648955+0000","last_active":"2024-03-28T13:09:42.687316+0000","last_peered":"2024-03-28T13:09:42.687316+0000","last_clean":"2024-03-28T13:09:42.687316+0000","last_became_active":"2024-03-28T13:09:40.648645+0000","last_became_peered":"2024-03-28T13:09:40.648645+0000","last_unstale":"2024-03-28T13:09:42.687316+0000","last_undegraded":"2024-03-28T13:09:42.687316+0000","last_fullsized":"2024-03-28T13:09:42.687316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:25:38.914424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.679924+0000","last_change":"2024-03-28T13:09:41.692962+0000","last_active":"2024-03-28T13:09:42.679924+0000","last_peered":"2024-03-28T13:09:42.679924+0000","last_clean":"2024-03-28T13:09:42.679924+0000","last_became_active":"2024-03-28T13:09:41.692660+0000","last_became_peered":"2024-03-28T13:09:41.692660+0000","last_unstale":"2024-03-28T13:09:42.679924+0000","last_undegraded":"2024-03-28T13:09:42.679924+0000","last_fullsized":"2024-03-28T13:09:42.679924+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:40.891627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685533+0000","last_change":"2024-03-28T13:09:40.654663+0000","last_active":"2024-03-28T13:09:42.685533+0000","last_peered":"2024-03-28T13:09:42.685533+0000","last_clean":"2024-03-28T13:09:42.685533+0000","last_became_active":"2024-03-28T13:09:40.654236+0000","last_became_peered":"2024-03-28T13:09:40.654236+0000","last_unstale":"2024-03-28T13:09:42.685533+0000","last_undegraded":"2024-03-28T13:09:42.685533+0000","last_fullsized":"2024-03-28T13:09:42.685533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:28.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.946112+0000","last_change":"2024-03-28T13:09:41.671654+0000","last_active":"2024-03-28T13:09:42.946112+0000","last_peered":"2024-03-28T13:09:42.946112+0000","last_clean":"2024-03-28T13:09:42.946112+0000","last_became_active":"2024-03-28T13:09:41.671232+0000","last_became_peered":"2024-03-28T13:09:41.671232+0000","last_unstale":"2024-03-28T13:09:42.946112+0000","last_undegraded":"2024-03-28T13:09:42.946112+0000","last_fullsized":"2024-03-28T13:09:42.946112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:21:02.278070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.694649+0000","last_change":"2024-03-28T13:09:40.651132+0000","last_active":"2024-03-28T13:09:42.694649+0000","last_peered":"2024-03-28T13:09:42.694649+0000","last_clean":"2024-03-28T13:09:42.694649+0000","last_became_active":"2024-03-28T13:09:40.650806+0000","last_became_peered":"2024-03-28T13:09:40.650806+0000","last_unstale":"2024-03-28T13:09:42.694649+0000","last_undegraded":"2024-03-28T13:09:42.694649+0000","last_fullsized":"2024-03-28T13:09:42.694649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:42:11.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691153+0000","last_change":"2024-03-28T13:09:41.686114+0000","last_active":"2024-03-28T13:09:42.691153+0000","last_peered":"2024-03-28T13:09:42.691153+0000","last_clean":"2024-03-28T13:09:42.691153+0000","last_became_active":"2024-03-28T13:09:41.685846+0000","last_became_peered":"2024-03-28T13:09:41.685846+0000","last_unstale":"2024-03-28T13:09:42.691153+0000","last_undegraded":"2024-03-28T13:09:42.691153+0000","last_fullsized":"2024-03-28T13:09:42.691153+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:28:13.777057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685901+0000","last_change":"2024-03-28T13:09:40.657051+0000","last_active":"2024-03-28T13:09:42.685901+0000","last_peered":"2024-03-28T13:09:42.685901+0000","last_clean":"2024-03-28T13:09:42.685901+0000","last_became_active":"2024-03-28T13:09:40.656680+0000","last_became_peered":"2024-03-28T13:09:40.656680+0000","last_unstale":"2024-03-28T13:09:42.685901+0000","last_undegraded":"2024-03-28T13:09:42.685901+0000","last_fullsized":"2024-03-28T13:09:42.685901+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:44:33.508365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691166+0000","last_change":"2024-03-28T13:09:41.681460+0000","last_active":"2024-03-28T13:09:42.691166+0000","last_peered":"2024-03-28T13:09:42.691166+0000","last_clean":"2024-03-28T13:09:42.691166+0000","last_became_active":"2024-03-28T13:09:41.681136+0000","last_became_peered":"2024-03-28T13:09:41.681136+0000","last_unstale":"2024-03-28T13:09:42.691166+0000","last_undegraded":"2024-03-28T13:09:42.691166+0000","last_fullsized":"2024-03-28T13:09:42.691166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:15:48.181534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695372+0000","last_change":"2024-03-28T13:09:40.649525+0000","last_active":"2024-03-28T13:09:42.695372+0000","last_peered":"2024-03-28T13:09:42.695372+0000","last_clean":"2024-03-28T13:09:42.695372+0000","last_became_active":"2024-03-28T13:09:40.649204+0000","last_became_peered":"2024-03-28T13:09:40.649204+0000","last_unstale":"2024-03-28T13:09:42.695372+0000","last_undegraded":"2024-03-28T13:09:42.695372+0000","last_fullsized":"2024-03-28T13:09:42.695372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:26.406992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.684678+0000","last_change":"2024-03-28T13:09:41.685557+0000","last_active":"2024-03-28T13:09:42.684678+0000","last_peered":"2024-03-28T13:09:42.684678+0000","last_clean":"2024-03-28T13:09:42.684678+0000","last_became_active":"2024-03-28T13:09:41.685000+0000","last_became_peered":"2024-03-28T13:09:41.685000+0000","last_unstale":"2024-03-28T13:09:42.684678+0000","last_undegraded":"2024-03-28T13:09:42.684678+0000","last_fullsized":"2024-03-28T13:09:42.684678+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:00.376154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.690868+0000","last_change":"2024-03-28T13:09:40.653861+0000","last_active":"2024-03-28T13:09:42.690868+0000","last_peered":"2024-03-28T13:09:42.690868+0000","last_clean":"2024-03-28T13:09:42.690868+0000","last_became_active":"2024-03-28T13:09:40.653467+0000","last_became_peered":"2024-03-28T13:09:40.653467+0000","last_unstale":"2024-03-28T13:09:42.690868+0000","last_undegraded":"2024-03-28T13:09:42.690868+0000","last_fullsized":"2024-03-28T13:09:42.690868+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:34:28.546257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.690684+0000","last_change":"2024-03-28T13:09:41.677815+0000","last_active":"2024-03-28T13:09:42.690684+0000","last_peered":"2024-03-28T13:09:42.690684+0000","last_clean":"2024-03-28T13:09:42.690684+0000","last_became_active":"2024-03-28T13:09:41.677544+0000","last_became_peered":"2024-03-28T13:09:41.677544+0000","last_unstale":"2024-03-28T13:09:42.690684+0000","last_undegraded":"2024-03-28T13:09:42.690684+0000","last_fullsized":"2024-03-28T13:09:42.690684+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:38:55.330149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.945733+0000","last_change":"2024-03-28T13:09:40.649046+0000","last_active":"2024-03-28T13:09:42.945733+0000","last_peered":"2024-03-28T13:09:42.945733+0000","last_clean":"2024-03-28T13:09:42.945733+0000","last_became_active":"2024-03-28T13:09:40.648758+0000","last_became_peered":"2024-03-28T13:09:40.648758+0000","last_unstale":"2024-03-28T13:09:42.945733+0000","last_undegraded":"2024-03-28T13:09:42.945733+0000","last_fullsized":"2024-03-28T13:09:42.945733+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:25:38.914424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.944019+0000","last_change":"2024-03-28T13:09:41.685698+0000","last_active":"2024-03-28T13:09:42.944019+0000","last_peered":"2024-03-28T13:09:42.944019+0000","last_clean":"2024-03-28T13:09:42.944019+0000","last_became_active":"2024-03-28T13:09:41.685418+0000","last_became_peered":"2024-03-28T13:09:41.685418+0000","last_unstale":"2024-03-28T13:09:42.944019+0000","last_undegraded":"2024-03-28T13:09:42.944019+0000","last_fullsized":"2024-03-28T13:09:42.944019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:57:43.818964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685389+0000","last_change":"2024-03-28T13:09:40.653049+0000","last_active":"2024-03-28T13:09:42.685389+0000","last_peered":"2024-03-28T13:09:42.685389+0000","last_clean":"2024-03-28T13:09:42.685389+0000","last_became_active":"2024-03-28T13:09:40.652732+0000","last_became_peered":"2024-03-28T13:09:40.652732+0000","last_unstale":"2024-03-28T13:09:42.685389+0000","last_undegraded":"2024-03-28T13:09:42.685389+0000","last_fullsized":"2024-03-28T13:09:42.685389+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:39:39.886681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685373+0000","last_change":"2024-03-28T13:09:41.690392+0000","last_active":"2024-03-28T13:09:42.685373+0000","last_peered":"2024-03-28T13:09:42.685373+0000","last_clean":"2024-03-28T13:09:42.685373+0000","last_became_active":"2024-03-28T13:09:41.689363+0000","last_became_peered":"2024-03-28T13:09:41.689363+0000","last_unstale":"2024-03-28T13:09:42.685373+0000","last_undegraded":"2024-03-28T13:09:42.685373+0000","last_fullsized":"2024-03-28T13:09:42.685373+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:09:02.095518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.685390+0000","last_change":"2024-03-28T13:09:41.700393+0000","last_active":"2024-03-28T13:09:42.685390+0000","last_peered":"2024-03-28T13:09:42.685390+0000","last_clean":"2024-03-28T13:09:42.685390+0000","last_became_active":"2024-03-28T13:09:41.700088+0000","last_became_peered":"2024-03-28T13:09:41.700088+0000","last_unstale":"2024-03-28T13:09:42.685390+0000","last_undegraded":"2024-03-28T13:09:42.685390+0000","last_fullsized":"2024-03-28T13:09:42.685390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:24:31.835464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.696866+0000","last_change":"2024-03-28T13:09:40.648841+0000","last_active":"2024-03-28T13:09:42.696866+0000","last_peered":"2024-03-28T13:09:42.696866+0000","last_clean":"2024-03-28T13:09:42.696866+0000","last_became_active":"2024-03-28T13:09:40.646454+0000","last_became_peered":"2024-03-28T13:09:40.646454+0000","last_unstale":"2024-03-28T13:09:42.696866+0000","last_undegraded":"2024-03-28T13:09:42.696866+0000","last_fullsized":"2024-03-28T13:09:42.696866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:42:11.749117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.942397+0000","last_change":"2024-03-28T13:09:41.705263+0000","last_active":"2024-03-28T13:09:42.942397+0000","last_peered":"2024-03-28T13:09:42.942397+0000","last_clean":"2024-03-28T13:09:42.942397+0000","last_became_active":"2024-03-28T13:09:41.704074+0000","last_became_peered":"2024-03-28T13:09:41.704074+0000","last_unstale":"2024-03-28T13:09:42.942397+0000","last_undegraded":"2024-03-28T13:09:42.942397+0000","last_fullsized":"2024-03-28T13:09:42.942397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:55:08.842535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.695978+0000","last_change":"2024-03-28T13:09:40.649403+0000","last_active":"2024-03-28T13:09:42.695978+0000","last_peered":"2024-03-28T13:09:42.695978+0000","last_clean":"2024-03-28T13:09:42.695978+0000","last_became_active":"2024-03-28T13:09:40.648964+0000","last_became_peered":"2024-03-28T13:09:40.648964+0000","last_unstale":"2024-03-28T13:09:42.695978+0000","last_undegraded":"2024-03-28T13:09:42.695978+0000","last_fullsized":"2024-03-28T13:09:42.695978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:19:50.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688252+0000","last_change":"2024-03-28T13:09:41.688111+0000","last_active":"2024-03-28T13:09:42.688252+0000","last_peered":"2024-03-28T13:09:42.688252+0000","last_clean":"2024-03-28T13:09:42.688252+0000","last_became_active":"2024-03-28T13:09:41.687818+0000","last_became_peered":"2024-03-28T13:09:41.687818+0000","last_unstale":"2024-03-28T13:09:42.688252+0000","last_undegraded":"2024-03-28T13:09:42.688252+0000","last_fullsized":"2024-03-28T13:09:42.688252+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:38.960999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688212+0000","last_change":"2024-03-28T13:09:40.646771+0000","last_active":"2024-03-28T13:09:42.688212+0000","last_peered":"2024-03-28T13:09:42.688212+0000","last_clean":"2024-03-28T13:09:42.688212+0000","last_became_active":"2024-03-28T13:09:40.646496+0000","last_became_peered":"2024-03-28T13:09:40.646496+0000","last_unstale":"2024-03-28T13:09:42.688212+0000","last_undegraded":"2024-03-28T13:09:42.688212+0000","last_fullsized":"2024-03-28T13:09:42.688212+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:19:50.921373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.688194+0000","last_change":"2024-03-28T13:09:41.684932+0000","last_active":"2024-03-28T13:09:42.688194+0000","last_peered":"2024-03-28T13:09:42.688194+0000","last_clean":"2024-03-28T13:09:42.688194+0000","last_became_active":"2024-03-28T13:09:41.684314+0000","last_became_peered":"2024-03-28T13:09:41.684314+0000","last_unstale":"2024-03-28T13:09:42.688194+0000","last_undegraded":"2024-03-28T13:09:42.688194+0000","last_fullsized":"2024-03-28T13:09:42.688194+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:40.607072+0000","last_clean_scrub_stamp":"2024-03-28T13:09:40.607072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:43:20.588688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-03-28T13:09:42.691848+0000","last_change":"2024-03-28T13:09:40.656875+0000","last_active":"2024-03-28T13:09:42.691848+0000","last_peered":"2024-03-28T13:09:42.691848+0000","last_clean":"2024-03-28T13:09:42.691848+0000","last_became_active":"2024-03-28T13:09:40.656569+0000","last_became_peered":"2024-03-28T13:09:40.656569+0000","last_unstale":"2024-03-28T13:09:42.691848+0000","last_undegraded":"2024-03-28T13:09:42.691848+0000","last_fullsized":"2024-03-28T13:09:42.691848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:39.602126+0000","last_clean_scrub_stamp":"2024-03-28T13:09:39.602126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:29:29.680319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476746,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":80417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476744,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476744,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":670273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476744,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":79891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":79835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":831488,"data_stored":670295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":82119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":79959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:09:52.526 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-28T13:09:52.526 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-28T13:09:52.526 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-28T13:09:52.526 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-28T13:09:52.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.768+0000 7ff84cc00640 1 Processor -- start 2024-03-28T13:09:52.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.768+0000 7ff84cc00640 1 -- start start 2024-03-28T13:09:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.770+0000 7ff84cc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff84813f300 0x7ff84806e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.770+0000 7ff84cc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff848141e40 0x7ff8480cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.770+0000 7ff84cc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff848142880 con 0x7ff8481419e0 2024-03-28T13:09:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.770+0000 7ff84cc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff8480cee90 con 0x7ff84813f300 2024-03-28T13:09:52.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.770+0000 7ff84cc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff8481422c0 con 0x7ff848141e40 2024-03-28T13:09:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.770+0000 7ff847e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff848141e40 0x7ff8480cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.770+0000 7ff847e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff848141e40 0x7ff8480cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:55014/0 (socket says 172.21.15.19:55014) 2024-03-28T13:09:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.770+0000 7ff847e00640 1 -- 172.21.15.19:0/3604911052 learned_addr learned my addr 172.21.15.19:0/3604911052 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.770+0000 7ff83fe00640 1 --1- 172.21.15.19:0/3604911052 >> v1:172.21.15.188:6789/0 conn(0x7ff8481419e0 0x7ff84815fd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:42676/0 (socket says 172.21.15.19:42676) 2024-03-28T13:09:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.770+0000 7ff847400640 1 --2- 172.21.15.19:0/3604911052 >> v2:172.21.15.188:3300/0 conn(0x7ff84813f300 0x7ff84806e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:52.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.771+0000 7ff847e00640 1 -- 172.21.15.19:0/3604911052 >> v2:172.21.15.188:3300/0 conn(0x7ff84813f300 msgr2=0x7ff84806e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.771+0000 7ff847e00640 1 --2- 172.21.15.19:0/3604911052 >> v2:172.21.15.188:3300/0 conn(0x7ff84813f300 0x7ff84806e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.771+0000 7ff847e00640 1 -- 172.21.15.19:0/3604911052 >> v1:172.21.15.188:6789/0 conn(0x7ff8481419e0 legacy=0x7ff84815fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.771+0000 7ff847e00640 1 -- 172.21.15.19:0/3604911052 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8481425a0 con 0x7ff848141e40 2024-03-28T13:09:52.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.772+0000 7ff847e00640 1 --2- 172.21.15.19:0/3604911052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff848141e40 0x7ff8480cf320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff8480cf010 tx=0x7ff84002bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c7bebf2c591031e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.773+0000 7ff846a00640 1 -- 172.21.15.19:0/3604911052 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84002caf0 con 0x7ff848141e40 2024-03-28T13:09:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.773+0000 7ff846a00640 1 -- 172.21.15.19:0/3604911052 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff84002cc90 con 0x7ff848141e40 2024-03-28T13:09:52.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.773+0000 7ff846a00640 1 -- 172.21.15.19:0/3604911052 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840034610 con 0x7ff848141e40 2024-03-28T13:09:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.773+0000 7ff84cc00640 1 -- 172.21.15.19:0/3604911052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff848141e40 msgr2=0x7ff8480cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.773+0000 7ff84cc00640 1 --2- 172.21.15.19:0/3604911052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff848141e40 0x7ff8480cf320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff8480cf010 tx=0x7ff84002bf50 comp rx=0 tx=0).stop 2024-03-28T13:09:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.773+0000 7ff84cc00640 1 -- 172.21.15.19:0/3604911052 shutdown_connections 2024-03-28T13:09:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.773+0000 7ff84cc00640 1 --2- 172.21.15.19:0/3604911052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff848141e40 0x7ff8480cf320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.773+0000 7ff84cc00640 1 --2- 172.21.15.19:0/3604911052 >> v2:172.21.15.188:3300/0 conn(0x7ff84813f300 0x7ff84806e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.773+0000 7ff84cc00640 1 -- 172.21.15.19:0/3604911052 >> 172.21.15.19:0/3604911052 conn(0x7ff84806b570 msgr2=0x7ff84806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.774+0000 7ff84cc00640 1 -- 172.21.15.19:0/3604911052 shutdown_connections 2024-03-28T13:09:52.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.774+0000 7ff84cc00640 1 -- 172.21.15.19:0/3604911052 wait complete. 2024-03-28T13:09:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.775+0000 7ff84cc00640 1 Processor -- start 2024-03-28T13:09:52.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.775+0000 7ff84cc00640 1 -- start start 2024-03-28T13:09:52.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.776+0000 7ff84cc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff84813f300 0x7ff8480b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:52.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.776+0000 7ff84cc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8481419e0 0x7ff848154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.776+0000 7ff84cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff848141e40 0x7ff848156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.776+0000 7ff84cc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff848160620 con 0x7ff848141e40 2024-03-28T13:09:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.776+0000 7ff84cc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481604a0 con 0x7ff8481419e0 2024-03-28T13:09:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.776+0000 7ff84cc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff8480cf860 con 0x7ff84813f300 2024-03-28T13:09:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.776+0000 7ff847e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff848141e40 0x7ff848156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.777+0000 7ff847e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff848141e40 0x7ff848156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51018/0 (socket says 172.21.15.19:51018) 2024-03-28T13:09:52.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.777+0000 7ff847e00640 1 -- 172.21.15.19:0/3573585053 learned_addr learned my addr 172.21.15.19:0/3573585053 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:52.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.777+0000 7ff847400640 1 --2- 172.21.15.19:0/3573585053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff84813f300 0x7ff8480b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:52.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.777+0000 7ff83fe00640 1 --2- 172.21.15.19:0/3573585053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8481419e0 0x7ff848154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:52.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.777+0000 7ff847e00640 1 -- 172.21.15.19:0/3573585053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff84813f300 msgr2=0x7ff8480b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.777+0000 7ff847e00640 1 --2- 172.21.15.19:0/3573585053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff84813f300 0x7ff8480b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.777+0000 7ff847e00640 1 -- 172.21.15.19:0/3573585053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8481419e0 msgr2=0x7ff848154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.777+0000 7ff847e00640 1 --2- 172.21.15.19:0/3573585053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8481419e0 0x7ff848154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.777+0000 7ff847e00640 1 -- 172.21.15.19:0/3573585053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8480b3390 con 0x7ff848141e40 2024-03-28T13:09:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.777+0000 7ff83fe00640 1 --2- 172.21.15.19:0/3573585053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8481419e0 0x7ff848154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.777+0000 7ff847e00640 1 --2- 172.21.15.19:0/3573585053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff848141e40 0x7ff848156650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff840009f40 tx=0x7ff840004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.778+0000 7ff83f400640 1 -- 172.21.15.19:0/3573585053 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840044020 con 0x7ff848141e40 2024-03-28T13:09:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.778+0000 7ff83f400640 1 -- 172.21.15.19:0/3573585053 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8400406b0 con 0x7ff848141e40 2024-03-28T13:09:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.778+0000 7ff84cc00640 1 -- 172.21.15.19:0/3573585053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8480afd60 con 0x7ff848141e40 2024-03-28T13:09:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.778+0000 7ff83f400640 1 -- 172.21.15.19:0/3573585053 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840009510 con 0x7ff848141e40 2024-03-28T13:09:52.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.778+0000 7ff84cc00640 1 -- 172.21.15.19:0/3573585053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8480b0320 con 0x7ff848141e40 2024-03-28T13:09:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.779+0000 7ff84cc00640 1 -- 172.21.15.19:0/3573585053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff848075940 con 0x7ff848141e40 2024-03-28T13:09:52.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.780+0000 7ff83f400640 1 -- 172.21.15.19:0/3573585053 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff840047020 con 0x7ff848141e40 2024-03-28T13:09:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.781+0000 7ff83f400640 1 --2- 172.21.15.19:0/3573585053 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff82406c560 0x7ff82406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.781+0000 7ff83f400640 1 -- 172.21.15.19:0/3573585053 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff8400ae0b0 con 0x7ff848141e40 2024-03-28T13:09:52.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.781+0000 7ff847400640 1 --2- 172.21.15.19:0/3573585053 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff82406c560 0x7ff82406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:52.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.782+0000 7ff847400640 1 --2- 172.21.15.19:0/3573585053 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff82406c560 0x7ff82406ea20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff838009ad0 tx=0x7ff838008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:52.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:52.785+0000 7ff83f400640 1 -- 172.21.15.19:0/3573585053 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff84004e030 con 0x7ff848141e40 2024-03-28T13:09:53.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.030+0000 7ff84cc00640 1 -- 172.21.15.19:0/3573585053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff8480cd740 con 0x7ff848141e40 2024-03-28T13:09:53.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.031+0000 7ff83f400640 1 -- 172.21.15.19:0/3573585053 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff84007c760 con 0x7ff848141e40 2024-03-28T13:09:53.030 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:53.031 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-28T13:09:53.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.033+0000 7ff84cc00640 1 -- 172.21.15.19:0/3573585053 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff82406c560 msgr2=0x7ff82406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.033+0000 7ff84cc00640 1 --2- 172.21.15.19:0/3573585053 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff82406c560 0x7ff82406ea20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff838009ad0 tx=0x7ff838008040 comp rx=0 tx=0).stop 2024-03-28T13:09:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.033+0000 7ff84cc00640 1 -- 172.21.15.19:0/3573585053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff848141e40 msgr2=0x7ff848156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.033+0000 7ff84cc00640 1 --2- 172.21.15.19:0/3573585053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff848141e40 0x7ff848156650 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff840009f40 tx=0x7ff840004a40 comp rx=0 tx=0).stop 2024-03-28T13:09:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.033+0000 7ff84cc00640 1 -- 172.21.15.19:0/3573585053 shutdown_connections 2024-03-28T13:09:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.033+0000 7ff84cc00640 1 --2- 172.21.15.19:0/3573585053 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff82406c560 0x7ff82406ea20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.033+0000 7ff84cc00640 1 --2- 172.21.15.19:0/3573585053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff848141e40 0x7ff848156650 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.033+0000 7ff84cc00640 1 --2- 172.21.15.19:0/3573585053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8481419e0 0x7ff848154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.033+0000 7ff84cc00640 1 --2- 172.21.15.19:0/3573585053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff84813f300 0x7ff8480b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.033+0000 7ff84cc00640 1 -- 172.21.15.19:0/3573585053 >> 172.21.15.19:0/3573585053 conn(0x7ff84806b570 msgr2=0x7ff84815fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.033+0000 7ff84cc00640 1 -- 172.21.15.19:0/3573585053 shutdown_connections 2024-03-28T13:09:53.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.033+0000 7ff84cc00640 1 -- 172.21.15.19:0/3573585053 wait complete. 2024-03-28T13:09:53.050 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-28T13:09:53.051 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:09:53.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.267+0000 7fe567e00640 1 Processor -- start 2024-03-28T13:09:53.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.268+0000 7fe567e00640 1 -- start start 2024-03-28T13:09:53.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.270+0000 7fe567e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe56815e4a0 0x7fe5680b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:53.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.270+0000 7fe567e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe5680cf700 0x7fe5680b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:53.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.270+0000 7fe567e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe56815e880 con 0x7fe5680d2310 2024-03-28T13:09:53.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.270+0000 7fe567e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe5680d34e0 con 0x7fe56815e4a0 2024-03-28T13:09:53.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.270+0000 7fe567e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fe5680d28d0 con 0x7fe5680cf700 2024-03-28T13:09:53.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.270+0000 7fe566a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe56815e4a0 0x7fe5680b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:53.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.270+0000 7fe566000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe5680d2310 0x7fe5680ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43048/0 (socket says 172.21.15.19:43048) 2024-03-28T13:09:53.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.270+0000 7fe566000640 1 -- 172.21.15.19:0/889961 learned_addr learned my addr 172.21.15.19:0/889961 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:53.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.270+0000 7fe567400640 1 --2- 172.21.15.19:0/889961 >> v2:172.21.15.188:3300/0 conn(0x7fe5680cf700 0x7fe5680b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:53.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.271+0000 7fe566a00640 1 -- 172.21.15.19:0/889961 >> v2:172.21.15.188:3300/0 conn(0x7fe5680cf700 msgr2=0x7fe5680b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:53.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.271+0000 7fe566a00640 1 --2- 172.21.15.19:0/889961 >> v2:172.21.15.188:3300/0 conn(0x7fe5680cf700 0x7fe5680b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.271+0000 7fe566a00640 1 -- 172.21.15.19:0/889961 >> v1:172.21.15.19:6789/0 conn(0x7fe5680d2310 legacy=0x7fe5680ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:53.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.271+0000 7fe565600640 1 -- 172.21.15.19:0/889961 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 733106598 0 0) 0x7fe56815e880 con 0x7fe5680d2310 2024-03-28T13:09:53.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.271+0000 7fe566a00640 1 -- 172.21.15.19:0/889961 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5680d2bb0 con 0x7fe56815e4a0 2024-03-28T13:09:53.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.271+0000 7fe567400640 1 --2- 172.21.15.19:0/889961 >> v2:172.21.15.188:3300/0 conn(0x7fe5680cf700 0x7fe5680b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:53.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.272+0000 7fe566a00640 1 --2- 172.21.15.19:0/889961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe56815e4a0 0x7fe5680b9c60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe5680d20b0 tx=0x7fe55802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b283bfbd0612f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:53.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.272+0000 7fe565600640 1 -- 172.21.15.19:0/889961 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5580315a0 con 0x7fe56815e4a0 2024-03-28T13:09:53.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.272+0000 7fe565600640 1 -- 172.21.15.19:0/889961 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe558037070 con 0x7fe56815e4a0 2024-03-28T13:09:53.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.272+0000 7fe565600640 1 -- 172.21.15.19:0/889961 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558031960 con 0x7fe56815e4a0 2024-03-28T13:09:53.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.273+0000 7fe567e00640 1 -- 172.21.15.19:0/889961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe56815e4a0 msgr2=0x7fe5680b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:53.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.273+0000 7fe567e00640 1 --2- 172.21.15.19:0/889961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe56815e4a0 0x7fe5680b9c60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe5680d20b0 tx=0x7fe55802b840 comp rx=0 tx=0).stop 2024-03-28T13:09:53.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.273+0000 7fe567e00640 1 -- 172.21.15.19:0/889961 shutdown_connections 2024-03-28T13:09:53.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.273+0000 7fe567e00640 1 --2- 172.21.15.19:0/889961 >> v2:172.21.15.188:3300/0 conn(0x7fe5680cf700 0x7fe5680b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.273+0000 7fe567e00640 1 --2- 172.21.15.19:0/889961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe56815e4a0 0x7fe5680b9c60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.273+0000 7fe567e00640 1 -- 172.21.15.19:0/889961 >> 172.21.15.19:0/889961 conn(0x7fe5680676d0 msgr2=0x7fe568067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:53.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.273+0000 7fe567e00640 1 -- 172.21.15.19:0/889961 shutdown_connections 2024-03-28T13:09:53.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.273+0000 7fe567e00640 1 -- 172.21.15.19:0/889961 wait complete. 2024-03-28T13:09:53.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.275+0000 7fe567e00640 1 Processor -- start 2024-03-28T13:09:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.275+0000 7fe567e00640 1 -- start start 2024-03-28T13:09:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe567e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5680cf700 0x7fe568150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe567e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5680d2310 0x7fe568151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe567e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe56815e4a0 0x7fe5680c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe567e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5680d3970 con 0x7fe5680cf700 2024-03-28T13:09:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe567e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe5680d3170 con 0x7fe56815e4a0 2024-03-28T13:09:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe567e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe5680ba1a0 con 0x7fe5680d2310 2024-03-28T13:09:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe566a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5680cf700 0x7fe568150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe566a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5680cf700 0x7fe568150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51044/0 (socket says 172.21.15.19:51044) 2024-03-28T13:09:53.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe566a00640 1 -- 172.21.15.19:0/1004374054 learned_addr learned my addr 172.21.15.19:0/1004374054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe566000640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5680d2310 0x7fe568151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe567400640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe56815e4a0 0x7fe5680c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe566a00640 1 -- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5680d2310 msgr2=0x7fe568151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe566a00640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5680d2310 0x7fe568151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe566a00640 1 -- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe56815e4a0 msgr2=0x7fe5680c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe566a00640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe56815e4a0 0x7fe5680c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe566a00640 1 -- 172.21.15.19:0/1004374054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe568151330 con 0x7fe5680cf700 2024-03-28T13:09:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe566000640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5680d2310 0x7fe568151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.276+0000 7fe567400640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe56815e4a0 0x7fe5680c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.277+0000 7fe566a00640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5680cf700 0x7fe568150df0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe558000c00 tx=0x7fe5580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.277+0000 7fe55ea00640 1 -- 172.21.15.19:0/1004374054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558004230 con 0x7fe5680cf700 2024-03-28T13:09:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.277+0000 7fe55ea00640 1 -- 172.21.15.19:0/1004374054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5580043f0 con 0x7fe5680cf700 2024-03-28T13:09:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.277+0000 7fe55ea00640 1 -- 172.21.15.19:0/1004374054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558003850 con 0x7fe5680cf700 2024-03-28T13:09:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.277+0000 7fe567e00640 1 -- 172.21.15.19:0/1004374054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5681536c0 con 0x7fe5680cf700 2024-03-28T13:09:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.277+0000 7fe567e00640 1 -- 172.21.15.19:0/1004374054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe568152900 con 0x7fe5680cf700 2024-03-28T13:09:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.278+0000 7fe567e00640 1 -- 172.21.15.19:0/1004374054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe568075940 con 0x7fe5680cf700 2024-03-28T13:09:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.279+0000 7fe55ea00640 1 -- 172.21.15.19:0/1004374054 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe5580039f0 con 0x7fe5680cf700 2024-03-28T13:09:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.281+0000 7fe55ea00640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe54806c530 0x7fe54806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.281+0000 7fe55ea00640 1 -- 172.21.15.19:0/1004374054 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe55802f030 con 0x7fe5680cf700 2024-03-28T13:09:53.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.281+0000 7fe566000640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe54806c530 0x7fe54806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:53.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.282+0000 7fe566000640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe54806c530 0x7fe54806e9f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe56815f440 tx=0x7fe554009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:53.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.285+0000 7fe55ea00640 1 -- 172.21.15.19:0/1004374054 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe568151330 con 0x7fe5680cf700 2024-03-28T13:09:53.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.629+0000 7fe567e00640 1 -- 172.21.15.19:0/1004374054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5680d3e70 con 0x7fe5680cf700 2024-03-28T13:09:53.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.631+0000 7fe55ea00640 1 -- 172.21.15.19:0/1004374054 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16609 (secure 0 0 0) 0x7fe558041070 con 0x7fe5680cf700 2024-03-28T13:09:53.630 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:53.631 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":23,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:42.668077+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:09:53.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.634+0000 7fe567e00640 1 -- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe54806c530 msgr2=0x7fe54806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:53.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.634+0000 7fe567e00640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe54806c530 0x7fe54806e9f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe56815f440 tx=0x7fe554009040 comp rx=0 tx=0).stop 2024-03-28T13:09:53.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.634+0000 7fe567e00640 1 -- 172.21.15.19:0/1004374054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5680cf700 msgr2=0x7fe568150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:53.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.634+0000 7fe567e00640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5680cf700 0x7fe568150df0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe558000c00 tx=0x7fe5580046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.634+0000 7fe567e00640 1 -- 172.21.15.19:0/1004374054 shutdown_connections 2024-03-28T13:09:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.634+0000 7fe567e00640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe54806c530 0x7fe54806e9f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.634+0000 7fe567e00640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe56815e4a0 0x7fe5680c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.634+0000 7fe567e00640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5680d2310 0x7fe568151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.634+0000 7fe567e00640 1 --2- 172.21.15.19:0/1004374054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5680cf700 0x7fe568150df0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.634+0000 7fe567e00640 1 -- 172.21.15.19:0/1004374054 >> 172.21.15.19:0/1004374054 conn(0x7fe5680676d0 msgr2=0x7fe5680b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.634+0000 7fe567e00640 1 -- 172.21.15.19:0/1004374054 shutdown_connections 2024-03-28T13:09:53.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.634+0000 7fe567e00640 1 -- 172.21.15.19:0/1004374054 wait complete. 2024-03-28T13:09:53.652 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:09:39.243813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:09:40.092899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:09:53.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:09:53.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.872+0000 7f8009e00640 1 Processor -- start 2024-03-28T13:09:53.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.872+0000 7f8009e00640 1 -- start start 2024-03-28T13:09:53.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.874+0000 7f8009e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f80040ccc40 0x7f800406c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:53.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.874+0000 7f8009e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80040b4710 con 0x7f80040afd80 2024-03-28T13:09:53.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.874+0000 7f8009e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80040b4980 con 0x7f800406cac0 2024-03-28T13:09:53.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.874+0000 7f8009e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f80040b5740 con 0x7f80040ccc40 2024-03-28T13:09:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.874+0000 7f8002a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f80040afd80 0x7f800415d4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43052/0 (socket says 172.21.15.19:43052) 2024-03-28T13:09:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.874+0000 7f8002a00640 1 -- 172.21.15.19:0/4116137946 learned_addr learned my addr 172.21.15.19:0/4116137946 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.874+0000 7f8002000640 1 --2- 172.21.15.19:0/4116137946 >> v2:172.21.15.188:3300/0 conn(0x7f80040ccc40 0x7f800406c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.874+0000 7f8001600640 1 -- 172.21.15.19:0/4116137946 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 720778756 0 0) 0x7f80040b4710 con 0x7f80040afd80 2024-03-28T13:09:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.875+0000 7f8001600640 1 -- 172.21.15.19:0/4116137946 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fe8003520 con 0x7f80040afd80 2024-03-28T13:09:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.875+0000 7f8001600640 1 -- 172.21.15.19:0/4116137946 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4139010586 0 0) 0x7f7fe8003520 con 0x7f80040afd80 2024-03-28T13:09:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.875+0000 7f8001600640 1 -- 172.21.15.19:0/4116137946 >> v2:172.21.15.188:3300/0 conn(0x7f80040ccc40 msgr2=0x7f800406c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.875+0000 7f8001600640 1 --2- 172.21.15.19:0/4116137946 >> v2:172.21.15.188:3300/0 conn(0x7f80040ccc40 0x7f800406c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.875+0000 7f8001600640 1 -- 172.21.15.19:0/4116137946 >> v1:172.21.15.188:6789/0 conn(0x7f800406cac0 legacy=0x7f80040ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:53.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.875+0000 7f8001600640 1 -- 172.21.15.19:0/4116137946 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80040b4c60 con 0x7f80040afd80 2024-03-28T13:09:53.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.875+0000 7f8001600640 1 -- 172.21.15.19:0/4116137946 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f7ff4001670 con 0x7f80040afd80 2024-03-28T13:09:53.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.875+0000 7f8001600640 1 -- 172.21.15.19:0/4116137946 >> v1:172.21.15.19:6789/0 conn(0x7f80040afd80 legacy=0x7f800415d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:53.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.875+0000 7f8001600640 1 --2- 172.21.15.19:0/4116137946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fe8003a00 0x7f7fe8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:53.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.875+0000 7f8001600640 1 --2- 172.21.15.19:0/4116137946 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe8004800 0x7f7fe8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:53.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.875+0000 7f8001600640 1 --2- 172.21.15.19:0/4116137946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fe80051c0 0x7f7fe800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:53.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.875+0000 7f8001600640 1 -- 172.21.15.19:0/4116137946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80040b5740 con 0x7f7fe80051c0 2024-03-28T13:09:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.875+0000 7f8001600640 1 -- 172.21.15.19:0/4116137946 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe800c0c0 con 0x7f7fe8003a00 2024-03-28T13:09:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.875+0000 7f8001600640 1 -- 172.21.15.19:0/4116137946 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe800c2c0 con 0x7f7fe8004800 2024-03-28T13:09:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.876+0000 7f8003400640 1 --2- 172.21.15.19:0/4116137946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fe80051c0 0x7f7fe800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.876+0000 7f8002a00640 1 --2- 172.21.15.19:0/4116137946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fe8003a00 0x7f7fe8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.876+0000 7f8002000640 1 --2- 172.21.15.19:0/4116137946 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe8004800 0x7f7fe8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.876+0000 7f8003400640 1 -- 172.21.15.19:0/4116137946 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe8004800 msgr2=0x7f7fe8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.876+0000 7f8003400640 1 --2- 172.21.15.19:0/4116137946 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe8004800 0x7f7fe8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.876+0000 7f8003400640 1 -- 172.21.15.19:0/4116137946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fe8003a00 msgr2=0x7f7fe8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.876+0000 7f8003400640 1 --2- 172.21.15.19:0/4116137946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fe8003a00 0x7f7fe8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.876+0000 7f8003400640 1 -- 172.21.15.19:0/4116137946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe8003520 con 0x7f7fe80051c0 2024-03-28T13:09:53.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.877+0000 7f8003400640 1 -- 172.21.15.19:0/4116137946 reap_dead start 2024-03-28T13:09:53.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.877+0000 7f8003400640 1 --2- 172.21.15.19:0/4116137946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fe80051c0 0x7f7fe800b9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc001e50 tx=0x7f7fe8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588ed3baafa3467e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:53.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.877+0000 7f8001600640 1 -- 172.21.15.19:0/4116137946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc001a70 con 0x7f7fe80051c0 2024-03-28T13:09:53.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.877+0000 7f8001600640 1 -- 172.21.15.19:0/4116137946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ffc003bb0 con 0x7f7fe80051c0 2024-03-28T13:09:53.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.877+0000 7f8001600640 1 -- 172.21.15.19:0/4116137946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0050d0 con 0x7f7fe80051c0 2024-03-28T13:09:53.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.878+0000 7f8009e00640 1 -- 172.21.15.19:0/4116137946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fe80051c0 msgr2=0x7f7fe800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:53.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.878+0000 7f8009e00640 1 --2- 172.21.15.19:0/4116137946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fe80051c0 0x7f7fe800b9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc001e50 tx=0x7f7fe8005830 comp rx=0 tx=0).stop 2024-03-28T13:09:53.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.878+0000 7f8009e00640 1 -- 172.21.15.19:0/4116137946 shutdown_connections 2024-03-28T13:09:53.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.878+0000 7f8009e00640 1 --2- 172.21.15.19:0/4116137946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fe80051c0 0x7f7fe800b9c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.878+0000 7f8009e00640 1 -- 172.21.15.19:0/4116137946 >> 172.21.15.19:0/4116137946 conn(0x7f8004071e20 msgr2=0x7f8004072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:53.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.878+0000 7f8009e00640 1 -- 172.21.15.19:0/4116137946 shutdown_connections 2024-03-28T13:09:53.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.878+0000 7f8009e00640 1 -- 172.21.15.19:0/4116137946 wait complete. 2024-03-28T13:09:53.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.880+0000 7f8009e00640 1 Processor -- start 2024-03-28T13:09:53.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.880+0000 7f8009e00640 1 -- start start 2024-03-28T13:09:53.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.881+0000 7f8009e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe80051c0 0x7f8004159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:53.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.881+0000 7f8009e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8004141270 0x7f80041586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:53.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.881+0000 7f8009e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8004158c30 0x7f800414ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:53.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.881+0000 7f8009e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80040b5220 con 0x7f8004158c30 2024-03-28T13:09:53.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.881+0000 7f8009e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f800406cef0 con 0x7f8004141270 2024-03-28T13:09:53.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.881+0000 7f8009e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f80040b0160 con 0x7f7fe80051c0 2024-03-28T13:09:53.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.881+0000 7f8003400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8004158c30 0x7f800414ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:53.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.881+0000 7f8003400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8004158c30 0x7f800414ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51054/0 (socket says 172.21.15.19:51054) 2024-03-28T13:09:53.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.881+0000 7f8003400640 1 -- 172.21.15.19:0/657800046 learned_addr learned my addr 172.21.15.19:0/657800046 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:53.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.882+0000 7f8002a00640 1 --2- 172.21.15.19:0/657800046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe80051c0 0x7f8004159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:53.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.882+0000 7f8002000640 1 --2- 172.21.15.19:0/657800046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8004141270 0x7f80041586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:53.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.882+0000 7f8003400640 1 -- 172.21.15.19:0/657800046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe80051c0 msgr2=0x7f8004159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:53.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.882+0000 7f8003400640 1 --2- 172.21.15.19:0/657800046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe80051c0 0x7f8004159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.882+0000 7f8003400640 1 -- 172.21.15.19:0/657800046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8004141270 msgr2=0x7f80041586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:53.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.882+0000 7f8003400640 1 --2- 172.21.15.19:0/657800046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8004141270 0x7f80041586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:53.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.882+0000 7f8003400640 1 -- 172.21.15.19:0/657800046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800414c090 con 0x7f8004158c30 2024-03-28T13:09:53.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.882+0000 7f8003400640 1 --2- 172.21.15.19:0/657800046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8004158c30 0x7f800414ce50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc001f70 tx=0x7f7ffc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:53.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.883+0000 7f7ffb400640 1 -- 172.21.15.19:0/657800046 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc039480 con 0x7f8004158c30 2024-03-28T13:09:53.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.883+0000 7f7ffb400640 1 -- 172.21.15.19:0/657800046 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ffc039620 con 0x7f8004158c30 2024-03-28T13:09:53.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.883+0000 7f7ffb400640 1 -- 172.21.15.19:0/657800046 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0399f0 con 0x7f8004158c30 2024-03-28T13:09:53.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.883+0000 7f8009e00640 1 -- 172.21.15.19:0/657800046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800414c370 con 0x7f8004158c30 2024-03-28T13:09:53.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.883+0000 7f8009e00640 1 -- 172.21.15.19:0/657800046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800414c930 con 0x7f8004158c30 2024-03-28T13:09:53.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.884+0000 7f8009e00640 1 -- 172.21.15.19:0/657800046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8004068cd0 con 0x7f8004158c30 2024-03-28T13:09:53.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.885+0000 7f7ffb400640 1 -- 172.21.15.19:0/657800046 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7ffc039b90 con 0x7f8004158c30 2024-03-28T13:09:53.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.886+0000 7f7ffb400640 1 --2- 172.21.15.19:0/657800046 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fe406c530 0x7f7fe406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:53.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.886+0000 7f7ffb400640 1 -- 172.21.15.19:0/657800046 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7ffc0ad8d0 con 0x7f8004158c30 2024-03-28T13:09:53.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.887+0000 7f8002a00640 1 --2- 172.21.15.19:0/657800046 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fe406c530 0x7f7fe406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:53.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.888+0000 7f8002a00640 1 --2- 172.21.15.19:0/657800046 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fe406c530 0x7f7fe406e9f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4000c00 tx=0x7f7ff4001df0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:53.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:53.895+0000 7f7ffb400640 1 -- 172.21.15.19:0/657800046 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7ffc03e070 con 0x7f8004158c30 2024-03-28T13:09:54.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.226+0000 7f8009e00640 1 -- 172.21.15.19:0/657800046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f80040b20a0 con 0x7f8004158c30 2024-03-28T13:09:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.227+0000 7f7ffb400640 1 -- 172.21.15.19:0/657800046 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f7ffc033030 con 0x7f8004158c30 2024-03-28T13:09:54.226 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:09:54.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.229+0000 7f8009e00640 1 -- 172.21.15.19:0/657800046 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fe406c530 msgr2=0x7f7fe406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:54.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.229+0000 7f8009e00640 1 --2- 172.21.15.19:0/657800046 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fe406c530 0x7f7fe406e9f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4000c00 tx=0x7f7ff4001df0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.229+0000 7f8009e00640 1 -- 172.21.15.19:0/657800046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8004158c30 msgr2=0x7f800414ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.229+0000 7f8009e00640 1 --2- 172.21.15.19:0/657800046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8004158c30 0x7f800414ce50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc001f70 tx=0x7f7ffc0014a0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.229+0000 7f8009e00640 1 -- 172.21.15.19:0/657800046 shutdown_connections 2024-03-28T13:09:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.229+0000 7f8009e00640 1 --2- 172.21.15.19:0/657800046 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fe406c530 0x7f7fe406e9f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.229+0000 7f8009e00640 1 --2- 172.21.15.19:0/657800046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8004158c30 0x7f800414ce50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.229+0000 7f8009e00640 1 --2- 172.21.15.19:0/657800046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8004141270 0x7f80041586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.229+0000 7f8009e00640 1 --2- 172.21.15.19:0/657800046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe80051c0 0x7f8004159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.229+0000 7f8009e00640 1 -- 172.21.15.19:0/657800046 >> 172.21.15.19:0/657800046 conn(0x7f8004071e20 msgr2=0x7f8004154e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.229+0000 7f8009e00640 1 -- 172.21.15.19:0/657800046 shutdown_connections 2024-03-28T13:09:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.230+0000 7f8009e00640 1 -- 172.21.15.19:0/657800046 wait complete. 2024-03-28T13:09:54.246 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:09:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.464+0000 7fdc7da00640 1 Processor -- start 2024-03-28T13:09:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.465+0000 7fdc7da00640 1 -- start start 2024-03-28T13:09:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.466+0000 7fdc7da00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdc78161f60 0x7fdc780d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.466+0000 7fdc7da00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdc780b3170 0x7fdc780d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.466+0000 7fdc7da00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc780b5ce0 con 0x7fdc780b3da0 2024-03-28T13:09:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.466+0000 7fdc7da00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdc780d27f0 con 0x7fdc78161f60 2024-03-28T13:09:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.466+0000 7fdc7da00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fdc780d2a90 con 0x7fdc780b3170 2024-03-28T13:09:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.467+0000 7fdc77400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdc780b3da0 0x7fdc780b4230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43064/0 (socket says 172.21.15.19:43064) 2024-03-28T13:09:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.467+0000 7fdc77400640 1 -- 172.21.15.19:0/3039081163 learned_addr learned my addr 172.21.15.19:0/3039081163 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.467+0000 7fdc76a00640 1 --2- 172.21.15.19:0/3039081163 >> v2:172.21.15.19:3300/0 conn(0x7fdc78161f60 0x7fdc780d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.467+0000 7fdc76000640 1 --2- 172.21.15.19:0/3039081163 >> v2:172.21.15.188:3300/0 conn(0x7fdc780b3170 0x7fdc780d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.467+0000 7fdc75600640 1 -- 172.21.15.19:0/3039081163 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2376106298 0 0) 0x7fdc70001490 con 0x7fdc780b3da0 2024-03-28T13:09:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.467+0000 7fdc75600640 1 -- 172.21.15.19:0/3039081163 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc5c003520 con 0x7fdc780b3da0 2024-03-28T13:09:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.467+0000 7fdc76a00640 1 -- 172.21.15.19:0/3039081163 >> v2:172.21.15.188:3300/0 conn(0x7fdc780b3170 msgr2=0x7fdc780d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.467+0000 7fdc76a00640 1 --2- 172.21.15.19:0/3039081163 >> v2:172.21.15.188:3300/0 conn(0x7fdc780b3170 0x7fdc780d13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.467+0000 7fdc76a00640 1 -- 172.21.15.19:0/3039081163 >> v1:172.21.15.19:6789/0 conn(0x7fdc780b3da0 legacy=0x7fdc780b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.467+0000 7fdc76a00640 1 -- 172.21.15.19:0/3039081163 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc78162340 con 0x7fdc78161f60 2024-03-28T13:09:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.469+0000 7fdc76a00640 1 --2- 172.21.15.19:0/3039081163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc78161f60 0x7fdc780d0e70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdc7808c880 tx=0x7fdc6802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c49c8698ee29bb9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.469+0000 7fdc75600640 1 -- 172.21.15.19:0/3039081163 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68035020 con 0x7fdc78161f60 2024-03-28T13:09:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.469+0000 7fdc75600640 1 -- 172.21.15.19:0/3039081163 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc68031830 con 0x7fdc78161f60 2024-03-28T13:09:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.469+0000 7fdc75600640 1 -- 172.21.15.19:0/3039081163 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68031be0 con 0x7fdc78161f60 2024-03-28T13:09:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.469+0000 7fdc7da00640 1 -- 172.21.15.19:0/3039081163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc78161f60 msgr2=0x7fdc780d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.469+0000 7fdc7da00640 1 --2- 172.21.15.19:0/3039081163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc78161f60 0x7fdc780d0e70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdc7808c880 tx=0x7fdc6802b8d0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.470+0000 7fdc7da00640 1 -- 172.21.15.19:0/3039081163 shutdown_connections 2024-03-28T13:09:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.470+0000 7fdc7da00640 1 --2- 172.21.15.19:0/3039081163 >> v2:172.21.15.188:3300/0 conn(0x7fdc780b3170 0x7fdc780d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.470+0000 7fdc7da00640 1 --2- 172.21.15.19:0/3039081163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc78161f60 0x7fdc780d0e70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.470+0000 7fdc7da00640 1 -- 172.21.15.19:0/3039081163 >> 172.21.15.19:0/3039081163 conn(0x7fdc7808bfb0 msgr2=0x7fdc780d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.470+0000 7fdc7da00640 1 -- 172.21.15.19:0/3039081163 shutdown_connections 2024-03-28T13:09:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.470+0000 7fdc7da00640 1 -- 172.21.15.19:0/3039081163 wait complete. 2024-03-28T13:09:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.472+0000 7fdc7da00640 1 Processor -- start 2024-03-28T13:09:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.473+0000 7fdc7da00640 1 -- start start 2024-03-28T13:09:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.474+0000 7fdc7da00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc780b3170 0x7fdc7814e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.474+0000 7fdc7da00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc780b3da0 0x7fdc780b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.474+0000 7fdc7da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc78161f60 0x7fdc780b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.474+0000 7fdc7da00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc780d3310 con 0x7fdc78161f60 2024-03-28T13:09:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.474+0000 7fdc7da00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdc780d3030 con 0x7fdc780b3da0 2024-03-28T13:09:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.474+0000 7fdc7da00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdc780d00e0 con 0x7fdc780b3170 2024-03-28T13:09:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.475+0000 7fdc77400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc78161f60 0x7fdc780b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.475+0000 7fdc77400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc78161f60 0x7fdc780b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51076/0 (socket says 172.21.15.19:51076) 2024-03-28T13:09:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.475+0000 7fdc77400640 1 -- 172.21.15.19:0/2171946636 learned_addr learned my addr 172.21.15.19:0/2171946636 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.475+0000 7fdc76a00640 1 --2- 172.21.15.19:0/2171946636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc780b3170 0x7fdc7814e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.475+0000 7fdc76000640 1 --2- 172.21.15.19:0/2171946636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc780b3da0 0x7fdc780b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.475+0000 7fdc77400640 1 -- 172.21.15.19:0/2171946636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc780b3170 msgr2=0x7fdc7814e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.475+0000 7fdc77400640 1 --2- 172.21.15.19:0/2171946636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc780b3170 0x7fdc7814e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.475+0000 7fdc77400640 1 -- 172.21.15.19:0/2171946636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc780b3da0 msgr2=0x7fdc780b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.475+0000 7fdc77400640 1 --2- 172.21.15.19:0/2171946636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc780b3da0 0x7fdc780b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.475+0000 7fdc77400640 1 -- 172.21.15.19:0/2171946636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc781f8680 con 0x7fdc78161f60 2024-03-28T13:09:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.476+0000 7fdc77400640 1 --2- 172.21.15.19:0/2171946636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc78161f60 0x7fdc780b1590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000cb50 tx=0x7fdc7000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.476+0000 7fdc6ea00640 1 -- 172.21.15.19:0/2171946636 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70017070 con 0x7fdc78161f60 2024-03-28T13:09:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.476+0000 7fdc6ea00640 1 -- 172.21.15.19:0/2171946636 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc7000ad40 con 0x7fdc78161f60 2024-03-28T13:09:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.476+0000 7fdc6ea00640 1 -- 172.21.15.19:0/2171946636 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70012620 con 0x7fdc78161f60 2024-03-28T13:09:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.476+0000 7fdc7da00640 1 -- 172.21.15.19:0/2171946636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc78150ae0 con 0x7fdc78161f60 2024-03-28T13:09:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.476+0000 7fdc7da00640 1 -- 172.21.15.19:0/2171946636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc781510a0 con 0x7fdc78161f60 2024-03-28T13:09:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.477+0000 7fdc7da00640 1 -- 172.21.15.19:0/2171946636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc780685c0 con 0x7fdc78161f60 2024-03-28T13:09:54.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.479+0000 7fdc6ea00640 1 -- 172.21.15.19:0/2171946636 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdc7001a070 con 0x7fdc78161f60 2024-03-28T13:09:54.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.480+0000 7fdc6ea00640 1 --2- 172.21.15.19:0/2171946636 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc5806c560 0x7fdc5806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:54.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.480+0000 7fdc6ea00640 1 -- 172.21.15.19:0/2171946636 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdc7008b500 con 0x7fdc78161f60 2024-03-28T13:09:54.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.481+0000 7fdc76a00640 1 --2- 172.21.15.19:0/2171946636 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc5806c560 0x7fdc5806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:54.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.482+0000 7fdc76a00640 1 --2- 172.21.15.19:0/2171946636 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc5806c560 0x7fdc5806ea20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdc68036000 tx=0x7fdc6803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:54.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.484+0000 7fdc6ea00640 1 -- 172.21.15.19:0/2171946636 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdc780b5ce0 con 0x7fdc78161f60 2024-03-28T13:09:54.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.686+0000 7fdc7da00640 1 -- 172.21.15.19:0/2171946636 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdc780b2700 con 0x7fdc78161f60 2024-03-28T13:09:54.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.687+0000 7fdc6ea00640 1 -- 172.21.15.19:0/2171946636 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fdc70059430 con 0x7fdc78161f60 2024-03-28T13:09:54.686 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:09:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.689+0000 7fdc7da00640 1 -- 172.21.15.19:0/2171946636 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc5806c560 msgr2=0x7fdc5806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.689+0000 7fdc7da00640 1 --2- 172.21.15.19:0/2171946636 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc5806c560 0x7fdc5806ea20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdc68036000 tx=0x7fdc6803c040 comp rx=0 tx=0).stop 2024-03-28T13:09:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.689+0000 7fdc7da00640 1 -- 172.21.15.19:0/2171946636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc78161f60 msgr2=0x7fdc780b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.689+0000 7fdc7da00640 1 --2- 172.21.15.19:0/2171946636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc78161f60 0x7fdc780b1590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000cb50 tx=0x7fdc7000a430 comp rx=0 tx=0).stop 2024-03-28T13:09:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.689+0000 7fdc7da00640 1 -- 172.21.15.19:0/2171946636 shutdown_connections 2024-03-28T13:09:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.689+0000 7fdc7da00640 1 --2- 172.21.15.19:0/2171946636 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc5806c560 0x7fdc5806ea20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.689+0000 7fdc7da00640 1 --2- 172.21.15.19:0/2171946636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc78161f60 0x7fdc780b1590 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.689+0000 7fdc7da00640 1 --2- 172.21.15.19:0/2171946636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc780b3da0 0x7fdc780b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.689+0000 7fdc7da00640 1 --2- 172.21.15.19:0/2171946636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc780b3170 0x7fdc7814e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.689+0000 7fdc7da00640 1 -- 172.21.15.19:0/2171946636 >> 172.21.15.19:0/2171946636 conn(0x7fdc7808bfb0 msgr2=0x7fdc7806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.690+0000 7fdc7da00640 1 -- 172.21.15.19:0/2171946636 shutdown_connections 2024-03-28T13:09:54.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.690+0000 7fdc7da00640 1 -- 172.21.15.19:0/2171946636 wait complete. 2024-03-28T13:09:54.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:09:54.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.951+0000 7f41f6c00640 1 Processor -- start 2024-03-28T13:09:54.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.951+0000 7f41f6c00640 1 -- start start 2024-03-28T13:09:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.954+0000 7f41f6c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f41f00d37b0 0x7f41f00b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.954+0000 7f41f6c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41f00d2710 con 0x7f41f00b67f0 2024-03-28T13:09:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.954+0000 7f41f6c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41f00d29f0 con 0x7f41f015e4c0 2024-03-28T13:09:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.954+0000 7f41f6c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f41f015e8a0 con 0x7f41f00d37b0 2024-03-28T13:09:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.954+0000 7f41f4e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f41f00d37b0 0x7f41f00b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.954+0000 7f41f6200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f41f00b67f0 0x7f41f00b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43072/0 (socket says 172.21.15.19:43072) 2024-03-28T13:09:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.954+0000 7f41f6200640 1 -- 172.21.15.19:0/3300896245 learned_addr learned my addr 172.21.15.19:0/3300896245 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:54.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.954+0000 7f41efe00640 1 -- 172.21.15.19:0/3300896245 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2881879955 0 0) 0x7f41f00d2710 con 0x7f41f00b67f0 2024-03-28T13:09:54.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.955+0000 7f41efe00640 1 -- 172.21.15.19:0/3300896245 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41d8003520 con 0x7f41f00b67f0 2024-03-28T13:09:54.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.955+0000 7f41f4e00640 1 -- 172.21.15.19:0/3300896245 >> v1:172.21.15.188:6789/0 conn(0x7f41f015e4c0 legacy=0x7f41f00b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:54.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.955+0000 7f41f4e00640 1 -- 172.21.15.19:0/3300896245 >> v1:172.21.15.19:6789/0 conn(0x7f41f00b67f0 legacy=0x7f41f00b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:54.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.955+0000 7f41f4e00640 1 -- 172.21.15.19:0/3300896245 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f00d2cd0 con 0x7f41f00d37b0 2024-03-28T13:09:54.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.956+0000 7f41f4e00640 1 --2- 172.21.15.19:0/3300896245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41f00d37b0 0x7f41f00b62b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0097a0 tx=0x7f41dc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c04d129d0a288a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:54.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.957+0000 7f41efe00640 1 -- 172.21.15.19:0/3300896245 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc034070 con 0x7f41f00d37b0 2024-03-28T13:09:54.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.957+0000 7f41efe00640 1 -- 172.21.15.19:0/3300896245 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41dc030610 con 0x7f41f00d37b0 2024-03-28T13:09:54.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.957+0000 7f41efe00640 1 -- 172.21.15.19:0/3300896245 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0309e0 con 0x7f41f00d37b0 2024-03-28T13:09:54.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.957+0000 7f41f6c00640 1 -- 172.21.15.19:0/3300896245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41f00d37b0 msgr2=0x7f41f00b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:54.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.957+0000 7f41f6c00640 1 --2- 172.21.15.19:0/3300896245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41f00d37b0 0x7f41f00b62b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0097a0 tx=0x7f41dc02b960 comp rx=0 tx=0).stop 2024-03-28T13:09:54.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.957+0000 7f41f6c00640 1 -- 172.21.15.19:0/3300896245 shutdown_connections 2024-03-28T13:09:54.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.957+0000 7f41f6c00640 1 --2- 172.21.15.19:0/3300896245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41f00d37b0 0x7f41f00b62b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.957+0000 7f41f6c00640 1 -- 172.21.15.19:0/3300896245 >> 172.21.15.19:0/3300896245 conn(0x7f41f00676a0 msgr2=0x7f41f0067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:54.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.958+0000 7f41f6c00640 1 -- 172.21.15.19:0/3300896245 shutdown_connections 2024-03-28T13:09:54.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.958+0000 7f41f6c00640 1 -- 172.21.15.19:0/3300896245 wait complete. 2024-03-28T13:09:54.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.959+0000 7f41f6c00640 1 Processor -- start 2024-03-28T13:09:54.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.959+0000 7f41f6c00640 1 -- start start 2024-03-28T13:09:54.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.960+0000 7f41f6c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41f00b67f0 0x7f41f01520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:54.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.960+0000 7f41f6c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41f00d37b0 0x7f41f015cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:54.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.960+0000 7f41f6c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41f015e4c0 0x7f41f015d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:54.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.960+0000 7f41f6c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41f00d3320 con 0x7f41f015e4c0 2024-03-28T13:09:54.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.960+0000 7f41f6c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41f00d4040 con 0x7f41f00b67f0 2024-03-28T13:09:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.960+0000 7f41f6c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f41f00b6c20 con 0x7f41f00d37b0 2024-03-28T13:09:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.961+0000 7f41f6200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41f015e4c0 0x7f41f015d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.961+0000 7f41f6200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41f015e4c0 0x7f41f015d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51086/0 (socket says 172.21.15.19:51086) 2024-03-28T13:09:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.961+0000 7f41f6200640 1 -- 172.21.15.19:0/3106041078 learned_addr learned my addr 172.21.15.19:0/3106041078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.961+0000 7f41f5800640 1 --2- 172.21.15.19:0/3106041078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41f00b67f0 0x7f41f01520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.961+0000 7f41f4e00640 1 --2- 172.21.15.19:0/3106041078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41f00d37b0 0x7f41f015cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.961+0000 7f41f6200640 1 -- 172.21.15.19:0/3106041078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41f00d37b0 msgr2=0x7f41f015cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.961+0000 7f41f6200640 1 --2- 172.21.15.19:0/3106041078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41f00d37b0 0x7f41f015cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.961+0000 7f41f6200640 1 -- 172.21.15.19:0/3106041078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41f00b67f0 msgr2=0x7f41f01520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.961+0000 7f41f6200640 1 --2- 172.21.15.19:0/3106041078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41f00b67f0 0x7f41f01520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.961+0000 7f41f6200640 1 -- 172.21.15.19:0/3106041078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f01f3a80 con 0x7f41f015e4c0 2024-03-28T13:09:54.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.961+0000 7f41f6200640 1 --2- 172.21.15.19:0/3106041078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41f015e4c0 0x7f41f015d080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f41e800cb40 tx=0x7f41e800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:54.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.962+0000 7f41ed600640 1 -- 172.21.15.19:0/3106041078 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8017070 con 0x7f41f015e4c0 2024-03-28T13:09:54.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.962+0000 7f41ed600640 1 -- 172.21.15.19:0/3106041078 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e800aca0 con 0x7f41f015e4c0 2024-03-28T13:09:54.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.962+0000 7f41ed600640 1 -- 172.21.15.19:0/3106041078 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8012620 con 0x7f41f015e4c0 2024-03-28T13:09:54.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.962+0000 7f41f6c00640 1 -- 172.21.15.19:0/3106041078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f0156880 con 0x7f41f015e4c0 2024-03-28T13:09:54.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.962+0000 7f41f6c00640 1 -- 172.21.15.19:0/3106041078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f0156e40 con 0x7f41f015e4c0 2024-03-28T13:09:54.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.964+0000 7f41f6c00640 1 -- 172.21.15.19:0/3106041078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f0075940 con 0x7f41f015e4c0 2024-03-28T13:09:54.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.964+0000 7f41ed600640 1 -- 172.21.15.19:0/3106041078 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f41e801a070 con 0x7f41f015e4c0 2024-03-28T13:09:54.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.965+0000 7f41ed600640 1 --2- 172.21.15.19:0/3106041078 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41d006c530 0x7f41d006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:54.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.965+0000 7f41ed600640 1 -- 172.21.15.19:0/3106041078 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f41e808b520 con 0x7f41f015e4c0 2024-03-28T13:09:54.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.966+0000 7f41f5800640 1 --2- 172.21.15.19:0/3106041078 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41d006c530 0x7f41d006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:54.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.967+0000 7f41f5800640 1 --2- 172.21.15.19:0/3106041078 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41d006c530 0x7f41d006e9f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f41e4000f60 tx=0x7f41e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:54.970+0000 7f41ed600640 1 -- 172.21.15.19:0/3106041078 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f41e8059b30 con 0x7f41f015e4c0 2024-03-28T13:09:55.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.254+0000 7f41f6c00640 1 -- 172.21.15.19:0/3106041078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f41f00cff50 con 0x7f41f015e4c0 2024-03-28T13:09:55.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.255+0000 7f41ed600640 1 -- 172.21.15.19:0/3106041078 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f41e8059430 con 0x7f41f015e4c0 2024-03-28T13:09:55.254 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:09:55.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.258+0000 7f41f6c00640 1 -- 172.21.15.19:0/3106041078 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41d006c530 msgr2=0x7f41d006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:55.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.258+0000 7f41f6c00640 1 --2- 172.21.15.19:0/3106041078 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41d006c530 0x7f41d006e9f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f41e4000f60 tx=0x7f41e4009040 comp rx=0 tx=0).stop 2024-03-28T13:09:55.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.258+0000 7f41f6c00640 1 -- 172.21.15.19:0/3106041078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41f015e4c0 msgr2=0x7f41f015d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:55.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.258+0000 7f41f6c00640 1 --2- 172.21.15.19:0/3106041078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41f015e4c0 0x7f41f015d080 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f41e800cb40 tx=0x7f41e800a060 comp rx=0 tx=0).stop 2024-03-28T13:09:55.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.258+0000 7f41f6c00640 1 -- 172.21.15.19:0/3106041078 shutdown_connections 2024-03-28T13:09:55.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.258+0000 7f41f6c00640 1 --2- 172.21.15.19:0/3106041078 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41d006c530 0x7f41d006e9f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.258+0000 7f41f6c00640 1 --2- 172.21.15.19:0/3106041078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41f015e4c0 0x7f41f015d080 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.258+0000 7f41f6c00640 1 --2- 172.21.15.19:0/3106041078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41f00d37b0 0x7f41f015cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.258+0000 7f41f6c00640 1 --2- 172.21.15.19:0/3106041078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41f00b67f0 0x7f41f01520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.258+0000 7f41f6c00640 1 -- 172.21.15.19:0/3106041078 >> 172.21.15.19:0/3106041078 conn(0x7f41f00676a0 msgr2=0x7f41f00b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:55.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.259+0000 7f41f6c00640 1 -- 172.21.15.19:0/3106041078 shutdown_connections 2024-03-28T13:09:55.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.259+0000 7f41f6c00640 1 -- 172.21.15.19:0/3106041078 wait complete. 2024-03-28T13:09:55.276 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:09:55.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.476+0000 7f344ac00640 1 Processor -- start 2024-03-28T13:09:55.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.476+0000 7f344ac00640 1 -- start start 2024-03-28T13:09:55.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.478+0000 7f344ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34440727f0 0x7f34440b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.478+0000 7f344ac00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f344415b6f0 con 0x7f344415b310 2024-03-28T13:09:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.478+0000 7f344ac00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3444072e80 con 0x7f34440d0780 2024-03-28T13:09:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.478+0000 7f344ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3444072c50 con 0x7f34440727f0 2024-03-28T13:09:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.478+0000 7f3443e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f344415b310 0x7f344415f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49088/0 (socket says 172.21.15.19:49088) 2024-03-28T13:09:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.478+0000 7f3443e00640 1 -- 172.21.15.19:0/931127483 learned_addr learned my addr 172.21.15.19:0/931127483 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.479+0000 7f343be00640 1 --2- 172.21.15.19:0/931127483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34440727f0 0x7f34440b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.479+0000 7f3443400640 1 -- 172.21.15.19:0/931127483 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1851078176 0 0) 0x7f344415b6f0 con 0x7f344415b310 2024-03-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.479+0000 7f3443400640 1 -- 172.21.15.19:0/931127483 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3428005730 con 0x7f344415b310 2024-03-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.479+0000 7f3443400640 1 -- 172.21.15.19:0/931127483 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2706114935 0 0) 0x7f3428005730 con 0x7f344415b310 2024-03-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.479+0000 7f3443400640 1 -- 172.21.15.19:0/931127483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34440727f0 msgr2=0x7f34440b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.479+0000 7f3443400640 1 --2- 172.21.15.19:0/931127483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34440727f0 0x7f34440b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.479+0000 7f3443400640 1 -- 172.21.15.19:0/931127483 >> v1:172.21.15.188:6789/0 conn(0x7f34440d0780 legacy=0x7f34440adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.479+0000 7f3443400640 1 -- 172.21.15.19:0/931127483 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3444073160 con 0x7f344415b310 2024-03-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.479+0000 7f3443400640 1 -- 172.21.15.19:0/931127483 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f3434001670 con 0x7f344415b310 2024-03-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.479+0000 7f3443400640 1 -- 172.21.15.19:0/931127483 >> v1:172.21.15.19:6789/0 conn(0x7f344415b310 legacy=0x7f344415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.479+0000 7f3443400640 1 --2- 172.21.15.19:0/931127483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3428005bf0 0x7f3428006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.479+0000 7f3443400640 1 --2- 172.21.15.19:0/931127483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34440727f0 0x7f3428006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.480+0000 7f3443400640 1 --2- 172.21.15.19:0/931127483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3428006f50 0x7f34280073b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.480+0000 7f3443400640 1 -- 172.21.15.19:0/931127483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f342800ddb0 con 0x7f3428006f50 2024-03-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.480+0000 7f3443400640 1 -- 172.21.15.19:0/931127483 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f342800df60 con 0x7f3428005bf0 2024-03-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.480+0000 7f3443400640 1 -- 172.21.15.19:0/931127483 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f342800e160 con 0x7f34440727f0 2024-03-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.480+0000 7f3448c00640 1 --2- 172.21.15.19:0/931127483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3428006f50 0x7f34280073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.480+0000 7f3443e00640 1 --2- 172.21.15.19:0/931127483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3428005bf0 0x7f3428006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.480+0000 7f343be00640 1 --2- 172.21.15.19:0/931127483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34440727f0 0x7f3428006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.480+0000 7f3448c00640 1 -- 172.21.15.19:0/931127483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34440727f0 msgr2=0x7f3428006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.480+0000 7f3448c00640 1 --2- 172.21.15.19:0/931127483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34440727f0 0x7f3428006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.480+0000 7f3448c00640 1 -- 172.21.15.19:0/931127483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3428005bf0 msgr2=0x7f3428006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.480+0000 7f3448c00640 1 --2- 172.21.15.19:0/931127483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3428005bf0 0x7f3428006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.480+0000 7f3448c00640 1 -- 172.21.15.19:0/931127483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3428005730 con 0x7f3428006f50 2024-03-28T13:09:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.482+0000 7f3448c00640 1 --2- 172.21.15.19:0/931127483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3428006f50 0x7f34280073b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f343c001e50 tx=0x7f3428007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9b6ad64c9c1848 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.482+0000 7f3443400640 1 -- 172.21.15.19:0/931127483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c001a70 con 0x7f3428006f50 2024-03-28T13:09:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.482+0000 7f3443400640 1 -- 172.21.15.19:0/931127483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f343c003b70 con 0x7f3428006f50 2024-03-28T13:09:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.482+0000 7f3443400640 1 -- 172.21.15.19:0/931127483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c0050d0 con 0x7f3428006f50 2024-03-28T13:09:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.483+0000 7f344ac00640 1 -- 172.21.15.19:0/931127483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3428006f50 msgr2=0x7f34280073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.483+0000 7f344ac00640 1 --2- 172.21.15.19:0/931127483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3428006f50 0x7f34280073b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f343c001e50 tx=0x7f3428007a40 comp rx=0 tx=0).stop 2024-03-28T13:09:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.483+0000 7f3448c00640 1 -- 172.21.15.19:0/931127483 reap_dead start 2024-03-28T13:09:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.483+0000 7f344ac00640 1 -- 172.21.15.19:0/931127483 shutdown_connections 2024-03-28T13:09:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.483+0000 7f344ac00640 1 -- 172.21.15.19:0/931127483 >> 172.21.15.19:0/931127483 conn(0x7f344406b990 msgr2=0x7f344406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.483+0000 7f344ac00640 1 -- 172.21.15.19:0/931127483 shutdown_connections 2024-03-28T13:09:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.483+0000 7f344ac00640 1 -- 172.21.15.19:0/931127483 wait complete. 2024-03-28T13:09:55.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.485+0000 7f344ac00640 1 Processor -- start 2024-03-28T13:09:55.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.485+0000 7f344ac00640 1 -- start start 2024-03-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.486+0000 7f344ac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3428006f50 0x7f344409dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.486+0000 7f344ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344415d680 0x7f34441f0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.486+0000 7f344ac00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34440b13f0 0x7f34440b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.486+0000 7f344ac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3444073440 con 0x7f3428006f50 2024-03-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.486+0000 7f344ac00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f344415f2c0 con 0x7f34440b13f0 2024-03-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.486+0000 7f344ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f34440ae390 con 0x7f344415d680 2024-03-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.486+0000 7f3443e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3428006f50 0x7f344409dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.486+0000 7f3443e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3428006f50 0x7f344409dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41800/0 (socket says 172.21.15.19:41800) 2024-03-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.486+0000 7f3443e00640 1 -- 172.21.15.19:0/4287273775 learned_addr learned my addr 172.21.15.19:0/4287273775 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.486+0000 7f3448c00640 1 --2- 172.21.15.19:0/4287273775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34440b13f0 0x7f34440b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.486+0000 7f343be00640 1 --2- 172.21.15.19:0/4287273775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344415d680 0x7f34441f0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:55.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.486+0000 7f3443e00640 1 -- 172.21.15.19:0/4287273775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344415d680 msgr2=0x7f34441f0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.486+0000 7f3443e00640 1 --2- 172.21.15.19:0/4287273775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344415d680 0x7f34441f0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.486+0000 7f3443e00640 1 -- 172.21.15.19:0/4287273775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34440b13f0 msgr2=0x7f34440b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.486+0000 7f3443e00640 1 --2- 172.21.15.19:0/4287273775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34440b13f0 0x7f34440b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.487+0000 7f3443e00640 1 -- 172.21.15.19:0/4287273775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34440afee0 con 0x7f3428006f50 2024-03-28T13:09:55.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.487+0000 7f3443e00640 1 --2- 172.21.15.19:0/4287273775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3428006f50 0x7f344409dd80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3434001200 tx=0x7f343400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.487+0000 7f3440c00640 1 -- 172.21.15.19:0/4287273775 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343400c660 con 0x7f3428006f50 2024-03-28T13:09:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.487+0000 7f3440c00640 1 -- 172.21.15.19:0/4287273775 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3434018070 con 0x7f3428006f50 2024-03-28T13:09:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.487+0000 7f3440c00640 1 -- 172.21.15.19:0/4287273775 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343400ca40 con 0x7f3428006f50 2024-03-28T13:09:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.487+0000 7f344ac00640 1 -- 172.21.15.19:0/4287273775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34440b01c0 con 0x7f3428006f50 2024-03-28T13:09:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.487+0000 7f344ac00640 1 -- 172.21.15.19:0/4287273775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34440b0780 con 0x7f3428006f50 2024-03-28T13:09:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.489+0000 7f344ac00640 1 -- 172.21.15.19:0/4287273775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34440685c0 con 0x7f3428006f50 2024-03-28T13:09:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.489+0000 7f3440c00640 1 -- 172.21.15.19:0/4287273775 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f343400cbe0 con 0x7f3428006f50 2024-03-28T13:09:55.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.490+0000 7f3440c00640 1 --2- 172.21.15.19:0/4287273775 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f342006c530 0x7f342006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:55.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.490+0000 7f3440c00640 1 -- 172.21.15.19:0/4287273775 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f343408b400 con 0x7f3428006f50 2024-03-28T13:09:55.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.491+0000 7f343be00640 1 --2- 172.21.15.19:0/4287273775 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f342006c530 0x7f342006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:55.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.492+0000 7f343be00640 1 --2- 172.21.15.19:0/4287273775 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f342006c530 0x7f342006e9f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f342c009950 tx=0x7f342c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:55.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.495+0000 7f3440c00640 1 -- 172.21.15.19:0/4287273775 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3434059ab0 con 0x7f3428006f50 2024-03-28T13:09:55.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.729+0000 7f344ac00640 1 -- 172.21.15.19:0/4287273775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f344415ec20 con 0x7f3428006f50 2024-03-28T13:09:55.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.730+0000 7f3440c00640 1 -- 172.21.15.19:0/4287273775 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f34340593b0 con 0x7f3428006f50 2024-03-28T13:09:55.729 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:55.730 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/745791856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":745791856},{"type":"v1","addr":"172.21.15.19:6833","nonce":745791856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/639283827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":639283827},{"type":"v1","addr":"172.21.15.188:6836","nonce":639283827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2168352329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2168352329},{"type":"v1","addr":"172.21.15.188:6837","nonce":2168352329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:09:41.105429+0000","modified":"2024-03-28T13:09:44.203526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/731659331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":731659331},{"type":"v1","addr":"172.21.15.19:6835","nonce":731659331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4254,"qdb_cluster":[4254]},"id":1}]} 2024-03-28T13:09:55.730 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-03-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.732+0000 7f344ac00640 1 -- 172.21.15.19:0/4287273775 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f342006c530 msgr2=0x7f342006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.732+0000 7f344ac00640 1 --2- 172.21.15.19:0/4287273775 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f342006c530 0x7f342006e9f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f342c009950 tx=0x7f342c008040 comp rx=0 tx=0).stop 2024-03-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.732+0000 7f344ac00640 1 -- 172.21.15.19:0/4287273775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3428006f50 msgr2=0x7f344409dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.732+0000 7f344ac00640 1 --2- 172.21.15.19:0/4287273775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3428006f50 0x7f344409dd80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3434001200 tx=0x7f343400eb50 comp rx=0 tx=0).stop 2024-03-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.732+0000 7f344ac00640 1 -- 172.21.15.19:0/4287273775 shutdown_connections 2024-03-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.732+0000 7f344ac00640 1 --2- 172.21.15.19:0/4287273775 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f342006c530 0x7f342006e9f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.732+0000 7f344ac00640 1 --2- 172.21.15.19:0/4287273775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34440b13f0 0x7f34440b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.732+0000 7f344ac00640 1 --2- 172.21.15.19:0/4287273775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f344415d680 0x7f34441f0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.732+0000 7f344ac00640 1 --2- 172.21.15.19:0/4287273775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3428006f50 0x7f344409dd80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.732+0000 7f344ac00640 1 -- 172.21.15.19:0/4287273775 >> 172.21.15.19:0/4287273775 conn(0x7f344406b990 msgr2=0x7f344406d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.732+0000 7f344ac00640 1 -- 172.21.15.19:0/4287273775 shutdown_connections 2024-03-28T13:09:55.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.733+0000 7f344ac00640 1 -- 172.21.15.19:0/4287273775 wait complete. 2024-03-28T13:09:55.756 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:09:55.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.991+0000 7f60fec00640 1 Processor -- start 2024-03-28T13:09:55.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.992+0000 7f60fec00640 1 -- start start 2024-03-28T13:09:55.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.994+0000 7f60fec00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f60f815e700 0x7f60f80b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:55.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.994+0000 7f60fec00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f80d1610 0x7f60f80b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:55.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.994+0000 7f60fec00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60f815eae0 con 0x7f60f80d1cf0 2024-03-28T13:09:55.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.994+0000 7f60fec00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f60f80cfeb0 con 0x7f60f815e700 2024-03-28T13:09:55.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.994+0000 7f60fec00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f60f80d0190 con 0x7f60f80d1610 2024-03-28T13:09:55.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.994+0000 7f60fcc00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f60f80d1cf0 0x7f60f8070fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49104/0 (socket says 172.21.15.19:49104) 2024-03-28T13:09:55.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.994+0000 7f60fcc00640 1 -- 172.21.15.19:0/28246113 learned_addr learned my addr 172.21.15.19:0/28246113 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f7400640 1 --2- 172.21.15.19:0/28246113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f80d1610 0x7f60f80b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f7e00640 1 --2- 172.21.15.19:0/28246113 >> v2:172.21.15.188:3300/0 conn(0x7f60f815e700 0x7f60f80b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 -- 172.21.15.19:0/28246113 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4235131888 0 0) 0x7f60f815eae0 con 0x7f60f80d1cf0 2024-03-28T13:09:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 -- 172.21.15.19:0/28246113 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60dc003520 con 0x7f60f80d1cf0 2024-03-28T13:09:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 -- 172.21.15.19:0/28246113 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2828546653 0 0) 0x7f60dc003520 con 0x7f60f80d1cf0 2024-03-28T13:09:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 -- 172.21.15.19:0/28246113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f80d1610 msgr2=0x7f60f80b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 --2- 172.21.15.19:0/28246113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f80d1610 0x7f60f80b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 -- 172.21.15.19:0/28246113 >> v2:172.21.15.188:3300/0 conn(0x7f60f815e700 msgr2=0x7f60f80b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 --2- 172.21.15.19:0/28246113 >> v2:172.21.15.188:3300/0 conn(0x7f60f815e700 0x7f60f80b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 -- 172.21.15.19:0/28246113 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f80d0470 con 0x7f60f80d1cf0 2024-03-28T13:09:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 -- 172.21.15.19:0/28246113 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f60f0001670 con 0x7f60f80d1cf0 2024-03-28T13:09:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 -- 172.21.15.19:0/28246113 >> v1:172.21.15.19:6789/0 conn(0x7f60f80d1cf0 legacy=0x7f60f8070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 --2- 172.21.15.19:0/28246113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60dc0039b0 0x7f60dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:55.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 --2- 172.21.15.19:0/28246113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60dc0043a0 0x7f60dc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 --2- 172.21.15.19:0/28246113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60dc004f90 0x7f60dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 -- 172.21.15.19:0/28246113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60f80cfeb0 con 0x7f60dc004f90 2024-03-28T13:09:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 -- 172.21.15.19:0/28246113 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f60f80d0190 con 0x7f60dc0043a0 2024-03-28T13:09:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.995+0000 7f60f6a00640 1 -- 172.21.15.19:0/28246113 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f60dc00bf30 con 0x7f60dc0039b0 2024-03-28T13:09:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.996+0000 7f60fcc00640 1 --2- 172.21.15.19:0/28246113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60dc004f90 0x7f60dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.996+0000 7f60f7e00640 1 --2- 172.21.15.19:0/28246113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60dc0039b0 0x7f60dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.996+0000 7f60f7400640 1 --2- 172.21.15.19:0/28246113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60dc0043a0 0x7f60dc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.996+0000 7f60fcc00640 1 -- 172.21.15.19:0/28246113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60dc0039b0 msgr2=0x7f60dc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.996+0000 7f60fcc00640 1 --2- 172.21.15.19:0/28246113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60dc0039b0 0x7f60dc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.996+0000 7f60fcc00640 1 -- 172.21.15.19:0/28246113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60dc0043a0 msgr2=0x7f60dc004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.996+0000 7f60fcc00640 1 --2- 172.21.15.19:0/28246113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60dc0043a0 0x7f60dc004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:55.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.996+0000 7f60fcc00640 1 -- 172.21.15.19:0/28246113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc003520 con 0x7f60dc004f90 2024-03-28T13:09:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.997+0000 7f60fcc00640 1 --2- 172.21.15.19:0/28246113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60dc004f90 0x7f60dc00b9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60f000c210 tx=0x7f60dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34022bee2553164 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.997+0000 7f60f6a00640 1 -- 172.21.15.19:0/28246113 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f0007d50 con 0x7f60dc004f90 2024-03-28T13:09:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.997+0000 7f60f6a00640 1 -- 172.21.15.19:0/28246113 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f0030070 con 0x7f60dc004f90 2024-03-28T13:09:55.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.997+0000 7f60f6a00640 1 -- 172.21.15.19:0/28246113 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f002cdb0 con 0x7f60dc004f90 2024-03-28T13:09:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.998+0000 7f60fec00640 1 -- 172.21.15.19:0/28246113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60dc004f90 msgr2=0x7f60dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.998+0000 7f60fec00640 1 --2- 172.21.15.19:0/28246113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60dc004f90 0x7f60dc00b9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f60f000c210 tx=0x7f60dc005830 comp rx=0 tx=0).stop 2024-03-28T13:09:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.998+0000 7f60fcc00640 1 -- 172.21.15.19:0/28246113 reap_dead start 2024-03-28T13:09:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.998+0000 7f60fec00640 1 -- 172.21.15.19:0/28246113 shutdown_connections 2024-03-28T13:09:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.998+0000 7f60fec00640 1 -- 172.21.15.19:0/28246113 >> 172.21.15.19:0/28246113 conn(0x7f60f80d2240 msgr2=0x7f60f80d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.998+0000 7f60fec00640 1 -- 172.21.15.19:0/28246113 shutdown_connections 2024-03-28T13:09:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:55.998+0000 7f60fec00640 1 -- 172.21.15.19:0/28246113 wait complete. 2024-03-28T13:09:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.000+0000 7f60fec00640 1 Processor -- start 2024-03-28T13:09:55.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.000+0000 7f60fec00640 1 -- start start 2024-03-28T13:09:56.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.001+0000 7f60fec00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f8142c70 0x7f60f8154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.001+0000 7f60fec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8153400 0x7f60f8157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.001+0000 7f60fec00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f8153860 0x7f60f8157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.001+0000 7f60fec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60f806d1b0 con 0x7f60f8153400 2024-03-28T13:09:56.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.001+0000 7f60fec00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f60f80d0a30 con 0x7f60f8142c70 2024-03-28T13:09:56.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.001+0000 7f60fec00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f60f806d330 con 0x7f60f8153860 2024-03-28T13:09:56.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.001+0000 7f60f7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8153400 0x7f60f8157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.001+0000 7f60f7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8153400 0x7f60f8157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41822/0 (socket says 172.21.15.19:41822) 2024-03-28T13:09:56.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.001+0000 7f60f7400640 1 -- 172.21.15.19:0/1708831817 learned_addr learned my addr 172.21.15.19:0/1708831817 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:56.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.002+0000 7f60f7e00640 1 --2- 172.21.15.19:0/1708831817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f8142c70 0x7f60f8154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.002+0000 7f60fcc00640 1 --2- 172.21.15.19:0/1708831817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f8153860 0x7f60f8157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.002+0000 7f60f7400640 1 -- 172.21.15.19:0/1708831817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f8153860 msgr2=0x7f60f8157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:56.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.002+0000 7f60f7400640 1 --2- 172.21.15.19:0/1708831817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f8153860 0x7f60f8157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.002+0000 7f60f7400640 1 -- 172.21.15.19:0/1708831817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f8142c70 msgr2=0x7f60f8154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:56.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.002+0000 7f60f7400640 1 --2- 172.21.15.19:0/1708831817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f8142c70 0x7f60f8154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.002+0000 7f60f7400640 1 -- 172.21.15.19:0/1708831817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f8152360 con 0x7f60f8153400 2024-03-28T13:09:56.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.002+0000 7f60f7400640 1 --2- 172.21.15.19:0/1708831817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8153400 0x7f60f8157610 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f60e4011600 tx=0x7f60e4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:56.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.002+0000 7f60efe00640 1 -- 172.21.15.19:0/1708831817 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e400f8c0 con 0x7f60f8153400 2024-03-28T13:09:56.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.002+0000 7f60efe00640 1 -- 172.21.15.19:0/1708831817 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e400fa60 con 0x7f60f8153400 2024-03-28T13:09:56.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.002+0000 7f60fec00640 1 -- 172.21.15.19:0/1708831817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f8152640 con 0x7f60f8153400 2024-03-28T13:09:56.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.002+0000 7f60efe00640 1 -- 172.21.15.19:0/1708831817 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e400fe50 con 0x7f60f8153400 2024-03-28T13:09:56.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.003+0000 7f60fec00640 1 -- 172.21.15.19:0/1708831817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f8152c00 con 0x7f60f8153400 2024-03-28T13:09:56.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.004+0000 7f60fec00640 1 -- 172.21.15.19:0/1708831817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f80685c0 con 0x7f60f8153400 2024-03-28T13:09:56.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.005+0000 7f60efe00640 1 -- 172.21.15.19:0/1708831817 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f60e4015460 con 0x7f60f8153400 2024-03-28T13:09:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.006+0000 7f60efe00640 1 --2- 172.21.15.19:0/1708831817 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f60d806c530 0x7f60d806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.006+0000 7f60efe00640 1 -- 172.21.15.19:0/1708831817 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f60e408c130 con 0x7f60f8153400 2024-03-28T13:09:56.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.006+0000 7f60f7e00640 1 --2- 172.21.15.19:0/1708831817 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f60d806c530 0x7f60d806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.008+0000 7f60f7e00640 1 --2- 172.21.15.19:0/1708831817 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f60d806c530 0x7f60d806e9f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009fd0 tx=0x7f60e800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:56.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.015+0000 7f60efe00640 1 -- 172.21.15.19:0/1708831817 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f60e4019030 con 0x7f60f8153400 2024-03-28T13:09:56.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.219+0000 7f60fec00640 1 -- 172.21.15.19:0/1708831817 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60f80c7510 con 0x7f60f8153400 2024-03-28T13:09:56.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.222+0000 7f60efe00640 1 -- 172.21.15.19:0/1708831817 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16609 (secure 0 0 0) 0x7f60e405a620 con 0x7f60f8153400 2024-03-28T13:09:56.221 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:56.221 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":23,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:42.668077+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:09:56.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.224+0000 7f60fec00640 1 -- 172.21.15.19:0/1708831817 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f60d806c530 msgr2=0x7f60d806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:56.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.224+0000 7f60fec00640 1 --2- 172.21.15.19:0/1708831817 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f60d806c530 0x7f60d806e9f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009fd0 tx=0x7f60e800b040 comp rx=0 tx=0).stop 2024-03-28T13:09:56.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.224+0000 7f60fec00640 1 -- 172.21.15.19:0/1708831817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8153400 msgr2=0x7f60f8157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:56.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.224+0000 7f60fec00640 1 --2- 172.21.15.19:0/1708831817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8153400 0x7f60f8157610 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f60e4011600 tx=0x7f60e4011ac0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.225+0000 7f60fec00640 1 -- 172.21.15.19:0/1708831817 shutdown_connections 2024-03-28T13:09:56.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.225+0000 7f60fec00640 1 --2- 172.21.15.19:0/1708831817 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f60d806c530 0x7f60d806e9f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.225+0000 7f60fec00640 1 --2- 172.21.15.19:0/1708831817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f8153860 0x7f60f8157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.225+0000 7f60fec00640 1 --2- 172.21.15.19:0/1708831817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60f8153400 0x7f60f8157610 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.225+0000 7f60fec00640 1 --2- 172.21.15.19:0/1708831817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f8142c70 0x7f60f8154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.225+0000 7f60fec00640 1 -- 172.21.15.19:0/1708831817 >> 172.21.15.19:0/1708831817 conn(0x7f60f80d2240 msgr2=0x7f60f815da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:56.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.225+0000 7f60fec00640 1 -- 172.21.15.19:0/1708831817 shutdown_connections 2024-03-28T13:09:56.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.225+0000 7f60fec00640 1 -- 172.21.15.19:0/1708831817 wait complete. 2024-03-28T13:09:56.243 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:09:39.243813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:09:40.092899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:09:56.244 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:09:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.451+0000 7f4f68a00640 1 Processor -- start 2024-03-28T13:09:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.452+0000 7f4f68a00640 1 -- start start 2024-03-28T13:09:56.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.453+0000 7f4f68a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4f6413f190 0x7f4f640cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.453+0000 7f4f68a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4f6406dd30 0x7f4f640cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.453+0000 7f4f68a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f640ce890 con 0x7f4f640cc880 2024-03-28T13:09:56.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.453+0000 7f4f68a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4f640cd230 con 0x7f4f6413f190 2024-03-28T13:09:56.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.453+0000 7f4f68a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f4f640ceb50 con 0x7f4f6406dd30 2024-03-28T13:09:56.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.454+0000 7f4f62800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4f6413f190 0x7f4f640cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.454+0000 7f4f62800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4f6413f190 0x7f4f640cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41832/0 (socket says 172.21.15.19:41832) 2024-03-28T13:09:56.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.454+0000 7f4f62800640 1 -- 172.21.15.19:0/4094715191 learned_addr learned my addr 172.21.15.19:0/4094715191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:56.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.454+0000 7f4f62800640 1 -- 172.21.15.19:0/4094715191 >> v2:172.21.15.188:3300/0 conn(0x7f4f6406dd30 msgr2=0x7f4f640cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:56.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.454+0000 7f4f63200640 1 --2- 172.21.15.19:0/4094715191 >> v2:172.21.15.188:3300/0 conn(0x7f4f6406dd30 0x7f4f640cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.454+0000 7f4f62800640 1 --2- 172.21.15.19:0/4094715191 >> v2:172.21.15.188:3300/0 conn(0x7f4f6406dd30 0x7f4f640cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.454+0000 7f4f62800640 1 -- 172.21.15.19:0/4094715191 >> v1:172.21.15.188:6789/0 conn(0x7f4f640cc880 legacy=0x7f4f6406d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.454+0000 7f4f62800640 1 -- 172.21.15.19:0/4094715191 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f640cee30 con 0x7f4f6413f190 2024-03-28T13:09:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.454+0000 7f4f63200640 1 --2- 172.21.15.19:0/4094715191 >> v2:172.21.15.188:3300/0 conn(0x7f4f6406dd30 0x7f4f640cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.455+0000 7f4f62800640 1 --2- 172.21.15.19:0/4094715191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f6413f190 0x7f4f640cf630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4f640b54a0 tx=0x7f4f5402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd05baa93257d3d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.456+0000 7f4f61400640 1 -- 172.21.15.19:0/4094715191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54035020 con 0x7f4f6413f190 2024-03-28T13:09:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.456+0000 7f4f61400640 1 -- 172.21.15.19:0/4094715191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f54031870 con 0x7f4f6413f190 2024-03-28T13:09:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.456+0000 7f4f61400640 1 -- 172.21.15.19:0/4094715191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f54031c40 con 0x7f4f6413f190 2024-03-28T13:09:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.456+0000 7f4f68a00640 1 -- 172.21.15.19:0/4094715191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f6413f190 msgr2=0x7f4f640cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.456+0000 7f4f68a00640 1 --2- 172.21.15.19:0/4094715191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f6413f190 0x7f4f640cf630 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4f640b54a0 tx=0x7f4f5402b910 comp rx=0 tx=0).stop 2024-03-28T13:09:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.457+0000 7f4f68a00640 1 -- 172.21.15.19:0/4094715191 shutdown_connections 2024-03-28T13:09:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.457+0000 7f4f68a00640 1 --2- 172.21.15.19:0/4094715191 >> v2:172.21.15.188:3300/0 conn(0x7f4f6406dd30 0x7f4f640cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.457+0000 7f4f68a00640 1 --2- 172.21.15.19:0/4094715191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f6413f190 0x7f4f640cf630 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.457+0000 7f4f68a00640 1 -- 172.21.15.19:0/4094715191 >> 172.21.15.19:0/4094715191 conn(0x7f4f640b1ba0 msgr2=0x7f4f640b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.457+0000 7f4f68a00640 1 -- 172.21.15.19:0/4094715191 shutdown_connections 2024-03-28T13:09:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.457+0000 7f4f68a00640 1 -- 172.21.15.19:0/4094715191 wait complete. 2024-03-28T13:09:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.459+0000 7f4f68a00640 1 Processor -- start 2024-03-28T13:09:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.460+0000 7f4f68a00640 1 -- start start 2024-03-28T13:09:56.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.461+0000 7f4f68a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f6406dd30 0x7f4f6413c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.461+0000 7f4f68a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f640cc880 0x7f4f641e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.461+0000 7f4f68a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f6413f190 0x7f4f6415e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.461+0000 7f4f68a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6413f570 con 0x7f4f640cc880 2024-03-28T13:09:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.461+0000 7f4f68a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f64160c70 con 0x7f4f6406dd30 2024-03-28T13:09:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.461+0000 7f4f68a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4f640cfb70 con 0x7f4f6413f190 2024-03-28T13:09:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.462+0000 7f4f61e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f640cc880 0x7f4f641e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.462+0000 7f4f61e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f640cc880 0x7f4f641e5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41844/0 (socket says 172.21.15.19:41844) 2024-03-28T13:09:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.462+0000 7f4f61e00640 1 -- 172.21.15.19:0/2368173824 learned_addr learned my addr 172.21.15.19:0/2368173824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.462+0000 7f4f63200640 1 --2- 172.21.15.19:0/2368173824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f6413f190 0x7f4f6415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.462+0000 7f4f62800640 1 --2- 172.21.15.19:0/2368173824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f6406dd30 0x7f4f6413c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.462+0000 7f4f61e00640 1 -- 172.21.15.19:0/2368173824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f6413f190 msgr2=0x7f4f6415e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.462+0000 7f4f61e00640 1 --2- 172.21.15.19:0/2368173824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f6413f190 0x7f4f6415e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.462+0000 7f4f61e00640 1 -- 172.21.15.19:0/2368173824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f6406dd30 msgr2=0x7f4f6413c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.462+0000 7f4f61e00640 1 --2- 172.21.15.19:0/2368173824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f6406dd30 0x7f4f6413c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.462+0000 7f4f61e00640 1 -- 172.21.15.19:0/2368173824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f6413cda0 con 0x7f4f640cc880 2024-03-28T13:09:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.462+0000 7f4f62800640 1 --2- 172.21.15.19:0/2368173824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f6406dd30 0x7f4f6413c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.463+0000 7f4f61e00640 1 --2- 172.21.15.19:0/2368173824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f640cc880 0x7f4f641e5c50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000b420 tx=0x7f4f5000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.463+0000 7f4f5aa00640 1 -- 172.21.15.19:0/2368173824 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50011830 con 0x7f4f640cc880 2024-03-28T13:09:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.463+0000 7f4f5aa00640 1 -- 172.21.15.19:0/2368173824 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f50016070 con 0x7f4f640cc880 2024-03-28T13:09:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.463+0000 7f4f68a00640 1 -- 172.21.15.19:0/2368173824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f6413d080 con 0x7f4f640cc880 2024-03-28T13:09:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.463+0000 7f4f5aa00640 1 -- 172.21.15.19:0/2368173824 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50011c10 con 0x7f4f640cc880 2024-03-28T13:09:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.463+0000 7f4f68a00640 1 -- 172.21.15.19:0/2368173824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f6413d640 con 0x7f4f640cc880 2024-03-28T13:09:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.464+0000 7f4f68a00640 1 -- 172.21.15.19:0/2368173824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f64075940 con 0x7f4f640cc880 2024-03-28T13:09:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.465+0000 7f4f5aa00640 1 -- 172.21.15.19:0/2368173824 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4f5001b460 con 0x7f4f640cc880 2024-03-28T13:09:56.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.466+0000 7f4f5aa00640 1 --2- 172.21.15.19:0/2368173824 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4f4406c530 0x7f4f4406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.466+0000 7f4f5aa00640 1 -- 172.21.15.19:0/2368173824 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4f5008b2b0 con 0x7f4f640cc880 2024-03-28T13:09:56.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.467+0000 7f4f62800640 1 --2- 172.21.15.19:0/2368173824 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4f4406c530 0x7f4f4406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.468+0000 7f4f62800640 1 --2- 172.21.15.19:0/2368173824 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4f4406c530 0x7f4f4406e9f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f54036000 tx=0x7f4f54008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.471+0000 7f4f5aa00640 1 -- 172.21.15.19:0/2368173824 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4f50020020 con 0x7f4f640cc880 2024-03-28T13:09:56.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.677+0000 7f4f68a00640 1 -- 172.21.15.19:0/2368173824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4f640cccf0 con 0x7f4f640cc880 2024-03-28T13:09:56.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.677+0000 7f4f5aa00640 1 -- 172.21.15.19:0/2368173824 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f4f50059830 con 0x7f4f640cc880 2024-03-28T13:09:56.677 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:09:56.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.680+0000 7f4f68a00640 1 -- 172.21.15.19:0/2368173824 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4f4406c530 msgr2=0x7f4f4406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:56.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.680+0000 7f4f68a00640 1 --2- 172.21.15.19:0/2368173824 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4f4406c530 0x7f4f4406e9f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4f54036000 tx=0x7f4f54008030 comp rx=0 tx=0).stop 2024-03-28T13:09:56.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.680+0000 7f4f68a00640 1 -- 172.21.15.19:0/2368173824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f640cc880 msgr2=0x7f4f641e5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:56.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.680+0000 7f4f68a00640 1 --2- 172.21.15.19:0/2368173824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f640cc880 0x7f4f641e5c50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000b420 tx=0x7f4f5000b8e0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.680+0000 7f4f68a00640 1 -- 172.21.15.19:0/2368173824 shutdown_connections 2024-03-28T13:09:56.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.680+0000 7f4f68a00640 1 --2- 172.21.15.19:0/2368173824 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4f4406c530 0x7f4f4406e9f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.680+0000 7f4f68a00640 1 --2- 172.21.15.19:0/2368173824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f6413f190 0x7f4f6415e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.680+0000 7f4f68a00640 1 --2- 172.21.15.19:0/2368173824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f640cc880 0x7f4f641e5c50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.680+0000 7f4f68a00640 1 --2- 172.21.15.19:0/2368173824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f6406dd30 0x7f4f6413c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.680+0000 7f4f68a00640 1 -- 172.21.15.19:0/2368173824 >> 172.21.15.19:0/2368173824 conn(0x7f4f640b1ba0 msgr2=0x7f4f6406d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:56.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.680+0000 7f4f68a00640 1 -- 172.21.15.19:0/2368173824 shutdown_connections 2024-03-28T13:09:56.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.681+0000 7f4f68a00640 1 -- 172.21.15.19:0/2368173824 wait complete. 2024-03-28T13:09:56.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:09:56.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.917+0000 7f4650c00640 1 Processor -- start 2024-03-28T13:09:56.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.918+0000 7f4650c00640 1 -- start start 2024-03-28T13:09:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.919+0000 7f4650c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f464c141c10 0x7f464c071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.919+0000 7f4650c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f464c071eb0 0x7f464c162160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.919+0000 7f4650c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f464c15ec80 con 0x7f464c0d0280 2024-03-28T13:09:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.919+0000 7f4650c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f464c15ef20 con 0x7f464c141c10 2024-03-28T13:09:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.919+0000 7f4650c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f464c15f200 con 0x7f464c071eb0 2024-03-28T13:09:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.920+0000 7f4649400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f464c141c10 0x7f464c071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.920+0000 7f4649400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f464c141c10 0x7f464c071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41858/0 (socket says 172.21.15.19:41858) 2024-03-28T13:09:56.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.920+0000 7f4649400640 1 -- 172.21.15.19:0/448170248 learned_addr learned my addr 172.21.15.19:0/448170248 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:56.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.920+0000 7f464a800640 1 --2- 172.21.15.19:0/448170248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f464c071eb0 0x7f464c162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.920+0000 7f4649400640 1 -- 172.21.15.19:0/448170248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f464c071eb0 msgr2=0x7f464c162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:56.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.920+0000 7f4649400640 1 --2- 172.21.15.19:0/448170248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f464c071eb0 0x7f464c162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.920+0000 7f4649400640 1 -- 172.21.15.19:0/448170248 >> v1:172.21.15.188:6789/0 conn(0x7f464c0d0280 legacy=0x7f464c0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:56.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.920+0000 7f4649400640 1 -- 172.21.15.19:0/448170248 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464c15f4e0 con 0x7f464c141c10 2024-03-28T13:09:56.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.921+0000 7f4649400640 1 --2- 172.21.15.19:0/448170248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f464c141c10 0x7f464c071970 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4638009870 tx=0x7f463802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f66b21780d3dfee server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:56.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.922+0000 7f4648a00640 1 -- 172.21.15.19:0/448170248 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638034070 con 0x7f464c141c10 2024-03-28T13:09:56.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.922+0000 7f4648a00640 1 -- 172.21.15.19:0/448170248 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4638030840 con 0x7f464c141c10 2024-03-28T13:09:56.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.922+0000 7f4648a00640 1 -- 172.21.15.19:0/448170248 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638030c10 con 0x7f464c141c10 2024-03-28T13:09:56.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.922+0000 7f4650c00640 1 -- 172.21.15.19:0/448170248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f464c141c10 msgr2=0x7f464c071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:56.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.922+0000 7f4650c00640 1 --2- 172.21.15.19:0/448170248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f464c141c10 0x7f464c071970 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4638009870 tx=0x7f463802b980 comp rx=0 tx=0).stop 2024-03-28T13:09:56.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.922+0000 7f4650c00640 1 -- 172.21.15.19:0/448170248 shutdown_connections 2024-03-28T13:09:56.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.922+0000 7f4650c00640 1 --2- 172.21.15.19:0/448170248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f464c141c10 0x7f464c071970 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.922+0000 7f4650c00640 1 --2- 172.21.15.19:0/448170248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f464c071eb0 0x7f464c162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.922+0000 7f4650c00640 1 -- 172.21.15.19:0/448170248 >> 172.21.15.19:0/448170248 conn(0x7f464c06b750 msgr2=0x7f464c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:56.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.923+0000 7f4650c00640 1 -- 172.21.15.19:0/448170248 shutdown_connections 2024-03-28T13:09:56.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.923+0000 7f4650c00640 1 -- 172.21.15.19:0/448170248 wait complete. 2024-03-28T13:09:56.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.924+0000 7f4650c00640 1 Processor -- start 2024-03-28T13:09:56.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.924+0000 7f4650c00640 1 -- start start 2024-03-28T13:09:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.925+0000 7f4650c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f464c071eb0 0x7f464c09d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.925+0000 7f4650c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f464c0d0280 0x7f464c09a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.925+0000 7f4650c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f464c141c10 0x7f464c1f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.925+0000 7f4650c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f464c158e90 con 0x7f464c0d0280 2024-03-28T13:09:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.925+0000 7f4650c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f464c06f340 con 0x7f464c141c10 2024-03-28T13:09:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.925+0000 7f4650c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f464c09ced0 con 0x7f464c071eb0 2024-03-28T13:09:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.926+0000 7f4649400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f464c0d0280 0x7f464c09a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.926+0000 7f4649400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f464c0d0280 0x7f464c09a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41874/0 (socket says 172.21.15.19:41874) 2024-03-28T13:09:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.926+0000 7f4649400640 1 -- 172.21.15.19:0/176554781 learned_addr learned my addr 172.21.15.19:0/176554781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.926+0000 7f4649e00640 1 --2- 172.21.15.19:0/176554781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f464c071eb0 0x7f464c09d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.926+0000 7f464a800640 1 --2- 172.21.15.19:0/176554781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f464c141c10 0x7f464c1f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.926+0000 7f4649400640 1 -- 172.21.15.19:0/176554781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f464c071eb0 msgr2=0x7f464c09d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.926+0000 7f4649400640 1 --2- 172.21.15.19:0/176554781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f464c071eb0 0x7f464c09d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.926+0000 7f4649400640 1 -- 172.21.15.19:0/176554781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f464c141c10 msgr2=0x7f464c1f4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.926+0000 7f4649400640 1 --2- 172.21.15.19:0/176554781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f464c141c10 0x7f464c1f4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.926+0000 7f4649400640 1 -- 172.21.15.19:0/176554781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464c14ab20 con 0x7f464c0d0280 2024-03-28T13:09:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.927+0000 7f4649400640 1 --2- 172.21.15.19:0/176554781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f464c0d0280 0x7f464c09a190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4638000c00 tx=0x7f46380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.927+0000 7f4642000640 1 -- 172.21.15.19:0/176554781 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638034040 con 0x7f464c0d0280 2024-03-28T13:09:56.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.927+0000 7f4642000640 1 -- 172.21.15.19:0/176554781 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4638039d90 con 0x7f464c0d0280 2024-03-28T13:09:56.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.927+0000 7f4650c00640 1 -- 172.21.15.19:0/176554781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464c09c2c0 con 0x7f464c0d0280 2024-03-28T13:09:56.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.927+0000 7f4642000640 1 -- 172.21.15.19:0/176554781 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638043610 con 0x7f464c0d0280 2024-03-28T13:09:56.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.927+0000 7f4650c00640 1 -- 172.21.15.19:0/176554781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f464c09cb60 con 0x7f464c0d0280 2024-03-28T13:09:56.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.929+0000 7f4650c00640 1 -- 172.21.15.19:0/176554781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f464c09c6b0 con 0x7f464c0d0280 2024-03-28T13:09:56.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.929+0000 7f4642000640 1 -- 172.21.15.19:0/176554781 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f463804a060 con 0x7f464c0d0280 2024-03-28T13:09:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.931+0000 7f4642000640 1 --2- 172.21.15.19:0/176554781 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f462c06c560 0x7f462c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.931+0000 7f4642000640 1 -- 172.21.15.19:0/176554781 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f46380ad640 con 0x7f464c0d0280 2024-03-28T13:09:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.932+0000 7f4649e00640 1 --2- 172.21.15.19:0/176554781 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f462c06c560 0x7f462c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:56.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.933+0000 7f4649e00640 1 --2- 172.21.15.19:0/176554781 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f462c06c560 0x7f462c06ea20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f464c1e8430 tx=0x7f463c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:56.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:56.935+0000 7f4642000640 1 -- 172.21.15.19:0/176554781 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f463803e070 con 0x7f464c0d0280 2024-03-28T13:09:57.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.138+0000 7f4650c00640 1 -- 172.21.15.19:0/176554781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f464c0685c0 con 0x7f464c0d0280 2024-03-28T13:09:57.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.139+0000 7f4642000640 1 -- 172.21.15.19:0/176554781 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f463807bc50 con 0x7f464c0d0280 2024-03-28T13:09:57.138 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:09:57.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.141+0000 7f4650c00640 1 -- 172.21.15.19:0/176554781 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f462c06c560 msgr2=0x7f462c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:57.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.141+0000 7f4650c00640 1 --2- 172.21.15.19:0/176554781 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f462c06c560 0x7f462c06ea20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f464c1e8430 tx=0x7f463c009040 comp rx=0 tx=0).stop 2024-03-28T13:09:57.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.141+0000 7f4650c00640 1 -- 172.21.15.19:0/176554781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f464c0d0280 msgr2=0x7f464c09a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:57.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.141+0000 7f4650c00640 1 --2- 172.21.15.19:0/176554781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f464c0d0280 0x7f464c09a190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4638000c00 tx=0x7f46380046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.141+0000 7f4650c00640 1 -- 172.21.15.19:0/176554781 shutdown_connections 2024-03-28T13:09:57.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.141+0000 7f4650c00640 1 --2- 172.21.15.19:0/176554781 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f462c06c560 0x7f462c06ea20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.141+0000 7f4650c00640 1 --2- 172.21.15.19:0/176554781 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f464c141c10 0x7f464c1f4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.141+0000 7f4650c00640 1 --2- 172.21.15.19:0/176554781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f464c0d0280 0x7f464c09a190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.141+0000 7f4650c00640 1 --2- 172.21.15.19:0/176554781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f464c071eb0 0x7f464c09d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.141+0000 7f4650c00640 1 -- 172.21.15.19:0/176554781 >> 172.21.15.19:0/176554781 conn(0x7f464c06b750 msgr2=0x7f464c06d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:57.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.142+0000 7f4650c00640 1 -- 172.21.15.19:0/176554781 shutdown_connections 2024-03-28T13:09:57.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.142+0000 7f4650c00640 1 -- 172.21.15.19:0/176554781 wait complete. 2024-03-28T13:09:57.159 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:09:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.379+0000 7fcaabe00640 1 Processor -- start 2024-03-28T13:09:57.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.379+0000 7fcaabe00640 1 -- start start 2024-03-28T13:09:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.381+0000 7fcaabe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcaac0cf630 0x7fcaac0b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.381+0000 7fcaabe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaac0b8320 con 0x7fcaac0d3e40 2024-03-28T13:09:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.381+0000 7fcaabe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaac0d2da0 con 0x7fcaac15e460 2024-03-28T13:09:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.381+0000 7fcaabe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fcaac0d3080 con 0x7fcaac0cf630 2024-03-28T13:09:57.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.381+0000 7fcaa3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcaac0d3e40 0x7fcaac0cef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49108/0 (socket says 172.21.15.19:49108) 2024-03-28T13:09:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.381+0000 7fcaa3e00640 1 -- 172.21.15.19:0/1157809157 learned_addr learned my addr 172.21.15.19:0/1157809157 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.381+0000 7fcaab400640 1 --2- 172.21.15.19:0/1157809157 >> v2:172.21.15.188:3300/0 conn(0x7fcaac0cf630 0x7fcaac0b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.382+0000 7fcaaa000640 1 -- 172.21.15.19:0/1157809157 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1494798258 0 0) 0x7fcaac0b8320 con 0x7fcaac0d3e40 2024-03-28T13:09:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.382+0000 7fcaaa000640 1 -- 172.21.15.19:0/1157809157 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca90003520 con 0x7fcaac0d3e40 2024-03-28T13:09:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.382+0000 7fcaab400640 1 -- 172.21.15.19:0/1157809157 >> v1:172.21.15.188:6789/0 conn(0x7fcaac15e460 legacy=0x7fcaac0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:57.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.382+0000 7fcaaa000640 1 -- 172.21.15.19:0/1157809157 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3378817179 0 0) 0x7fca90003520 con 0x7fcaac0d3e40 2024-03-28T13:09:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.382+0000 7fcaab400640 1 -- 172.21.15.19:0/1157809157 >> v1:172.21.15.19:6789/0 conn(0x7fcaac0d3e40 legacy=0x7fcaac0cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.382+0000 7fcaab400640 1 -- 172.21.15.19:0/1157809157 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaac0d3360 con 0x7fcaac0cf630 2024-03-28T13:09:57.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.383+0000 7fcaab400640 1 --2- 172.21.15.19:0/1157809157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaac0cf630 0x7fcaac0b9e20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009d90 tx=0x7fcaa402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ae25e92dbd75c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.384+0000 7fcaaa000640 1 -- 172.21.15.19:0/1157809157 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4039070 con 0x7fcaac0cf630 2024-03-28T13:09:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.384+0000 7fcaaa000640 1 -- 172.21.15.19:0/1157809157 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaa402c940 con 0x7fcaac0cf630 2024-03-28T13:09:57.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.384+0000 7fcaaa000640 1 -- 172.21.15.19:0/1157809157 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94005090 con 0x7fcaac0cf630 2024-03-28T13:09:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.385+0000 7fcaabe00640 1 -- 172.21.15.19:0/1157809157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaac0cf630 msgr2=0x7fcaac0b9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.385+0000 7fcaabe00640 1 --2- 172.21.15.19:0/1157809157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaac0cf630 0x7fcaac0b9e20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009d90 tx=0x7fcaa402be10 comp rx=0 tx=0).stop 2024-03-28T13:09:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.385+0000 7fcaabe00640 1 -- 172.21.15.19:0/1157809157 shutdown_connections 2024-03-28T13:09:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.385+0000 7fcaabe00640 1 --2- 172.21.15.19:0/1157809157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaac0cf630 0x7fcaac0b9e20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.385+0000 7fcaabe00640 1 -- 172.21.15.19:0/1157809157 >> 172.21.15.19:0/1157809157 conn(0x7fcaac0678b0 msgr2=0x7fcaac0d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.385+0000 7fcaabe00640 1 -- 172.21.15.19:0/1157809157 shutdown_connections 2024-03-28T13:09:57.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.385+0000 7fcaabe00640 1 -- 172.21.15.19:0/1157809157 wait complete. 2024-03-28T13:09:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.388+0000 7fcaabe00640 1 Processor -- start 2024-03-28T13:09:57.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.388+0000 7fcaabe00640 1 -- start start 2024-03-28T13:09:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.390+0000 7fcaabe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaac0cf630 0x7fcaac1428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:57.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.390+0000 7fcaabe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaac0d3e40 0x7fcaac0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.390+0000 7fcaabe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaac15e460 0x7fcaac153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.390+0000 7fcaabe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac168ef0 con 0x7fcaac0d3e40 2024-03-28T13:09:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.390+0000 7fcaabe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcaac0d3920 con 0x7fcaac0cf630 2024-03-28T13:09:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.390+0000 7fcaabe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcaac169070 con 0x7fcaac15e460 2024-03-28T13:09:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.390+0000 7fcaa3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaac0d3e40 0x7fcaac0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.390+0000 7fcaa3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaac0d3e40 0x7fcaac0ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41882/0 (socket says 172.21.15.19:41882) 2024-03-28T13:09:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.390+0000 7fcaa3e00640 1 -- 172.21.15.19:0/3300403843 learned_addr learned my addr 172.21.15.19:0/3300403843 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.390+0000 7fcaaaa00640 1 --2- 172.21.15.19:0/3300403843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaac0cf630 0x7fcaac1428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.390+0000 7fcaab400640 1 --2- 172.21.15.19:0/3300403843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaac15e460 0x7fcaac153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.391+0000 7fcaa3e00640 1 -- 172.21.15.19:0/3300403843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaac15e460 msgr2=0x7fcaac153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.391+0000 7fcaa3e00640 1 --2- 172.21.15.19:0/3300403843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaac15e460 0x7fcaac153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.391+0000 7fcaa3e00640 1 -- 172.21.15.19:0/3300403843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaac0cf630 msgr2=0x7fcaac1428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.391+0000 7fcaa3e00640 1 --2- 172.21.15.19:0/3300403843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaac0cf630 0x7fcaac1428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.391+0000 7fcaa3e00640 1 -- 172.21.15.19:0/3300403843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaac0ce950 con 0x7fcaac0d3e40 2024-03-28T13:09:57.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.391+0000 7fcaab400640 1 --2- 172.21.15.19:0/3300403843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaac15e460 0x7fcaac153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:57.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.391+0000 7fcaa3e00640 1 --2- 172.21.15.19:0/3300403843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaac0d3e40 0x7fcaac0ce410 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca94005200 tx=0x7fca9400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:57.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.391+0000 7fcaa2a00640 1 -- 172.21.15.19:0/3300403843 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94017070 con 0x7fcaac0d3e40 2024-03-28T13:09:57.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.391+0000 7fcaa2a00640 1 -- 172.21.15.19:0/3300403843 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca94005af0 con 0x7fcaac0d3e40 2024-03-28T13:09:57.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.391+0000 7fcaa2a00640 1 -- 172.21.15.19:0/3300403843 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca94007070 con 0x7fcaac0d3e40 2024-03-28T13:09:57.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.391+0000 7fcaabe00640 1 -- 172.21.15.19:0/3300403843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaac15a5e0 con 0x7fcaac0d3e40 2024-03-28T13:09:57.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.391+0000 7fcaabe00640 1 -- 172.21.15.19:0/3300403843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaac1f5600 con 0x7fcaac0d3e40 2024-03-28T13:09:57.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.393+0000 7fcaabe00640 1 -- 172.21.15.19:0/3300403843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaac075940 con 0x7fcaac0d3e40 2024-03-28T13:09:57.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.395+0000 7fcaa2a00640 1 -- 172.21.15.19:0/3300403843 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fca9401a070 con 0x7fcaac0d3e40 2024-03-28T13:09:57.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.396+0000 7fcaa2a00640 1 --2- 172.21.15.19:0/3300403843 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca8806c530 0x7fca8806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:57.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.396+0000 7fcaa2a00640 1 -- 172.21.15.19:0/3300403843 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fca9408ac60 con 0x7fcaac0d3e40 2024-03-28T13:09:57.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.396+0000 7fcaaaa00640 1 --2- 172.21.15.19:0/3300403843 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca8806c530 0x7fca8806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:57.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.397+0000 7fcaaaa00640 1 --2- 172.21.15.19:0/3300403843 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca8806c530 0x7fca8806e9f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fca9c000f60 tx=0x7fca9c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:57.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.401+0000 7fcaa2a00640 1 -- 172.21.15.19:0/3300403843 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fca940591e0 con 0x7fcaac0d3e40 2024-03-28T13:09:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.604+0000 7fcaabe00640 1 -- 172.21.15.19:0/3300403843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcaac15a8c0 con 0x7fcaac0d3e40 2024-03-28T13:09:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.605+0000 7fcaa2a00640 1 -- 172.21.15.19:0/3300403843 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fca94058ae0 con 0x7fcaac0d3e40 2024-03-28T13:09:57.604 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:09:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.608+0000 7fcaabe00640 1 -- 172.21.15.19:0/3300403843 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca8806c530 msgr2=0x7fca8806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.608+0000 7fcaabe00640 1 --2- 172.21.15.19:0/3300403843 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca8806c530 0x7fca8806e9f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fca9c000f60 tx=0x7fca9c009040 comp rx=0 tx=0).stop 2024-03-28T13:09:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.608+0000 7fcaabe00640 1 -- 172.21.15.19:0/3300403843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaac0d3e40 msgr2=0x7fcaac0ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.608+0000 7fcaabe00640 1 --2- 172.21.15.19:0/3300403843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaac0d3e40 0x7fcaac0ce410 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fca94005200 tx=0x7fca9400cbe0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.608+0000 7fcaabe00640 1 -- 172.21.15.19:0/3300403843 shutdown_connections 2024-03-28T13:09:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.608+0000 7fcaabe00640 1 --2- 172.21.15.19:0/3300403843 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca8806c530 0x7fca8806e9f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.608+0000 7fcaabe00640 1 --2- 172.21.15.19:0/3300403843 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcaac15e460 0x7fcaac153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.608+0000 7fcaabe00640 1 --2- 172.21.15.19:0/3300403843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaac0d3e40 0x7fcaac0ce410 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.608+0000 7fcaabe00640 1 --2- 172.21.15.19:0/3300403843 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcaac0cf630 0x7fcaac1428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.608+0000 7fcaabe00640 1 -- 172.21.15.19:0/3300403843 >> 172.21.15.19:0/3300403843 conn(0x7fcaac0678b0 msgr2=0x7fcaac0b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.608+0000 7fcaabe00640 1 -- 172.21.15.19:0/3300403843 shutdown_connections 2024-03-28T13:09:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.608+0000 7fcaabe00640 1 -- 172.21.15.19:0/3300403843 wait complete. 2024-03-28T13:09:57.626 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:09:57.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.841+0000 7f7b3ca00640 1 Processor -- start 2024-03-28T13:09:57.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.842+0000 7f7b3ca00640 1 -- start start 2024-03-28T13:09:57.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.843+0000 7f7b3ca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7b3815da60 0x7f7b380cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:57.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.843+0000 7f7b3ca00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7b3815fe90 0x7f7b380d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:57.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.843+0000 7f7b3ca00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b3815d060 con 0x7f7b3815f900 2024-03-28T13:09:57.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.843+0000 7f7b3ca00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7b3815d340 con 0x7f7b3815da60 2024-03-28T13:09:57.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.843+0000 7f7b3ca00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7b3815d620 con 0x7f7b3815fe90 2024-03-28T13:09:57.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.844+0000 7f7b2d400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7b3815da60 0x7f7b380cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:57.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.844+0000 7f7b2d400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7b3815da60 0x7f7b380cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41886/0 (socket says 172.21.15.19:41886) 2024-03-28T13:09:57.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.844+0000 7f7b2d400640 1 -- 172.21.15.19:0/1875238560 learned_addr learned my addr 172.21.15.19:0/1875238560 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:57.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.844+0000 7f7b36800640 1 --2- 172.21.15.19:0/1875238560 >> v2:172.21.15.188:3300/0 conn(0x7f7b3815fe90 0x7f7b380d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:57.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.844+0000 7f7b2d400640 1 -- 172.21.15.19:0/1875238560 >> v2:172.21.15.188:3300/0 conn(0x7f7b3815fe90 msgr2=0x7f7b380d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:57.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.844+0000 7f7b2d400640 1 --2- 172.21.15.19:0/1875238560 >> v2:172.21.15.188:3300/0 conn(0x7f7b3815fe90 0x7f7b380d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.844+0000 7f7b2d400640 1 -- 172.21.15.19:0/1875238560 >> v1:172.21.15.188:6789/0 conn(0x7f7b3815f900 legacy=0x7f7b3815c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:57.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.844+0000 7f7b2d400640 1 -- 172.21.15.19:0/1875238560 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b380c74b0 con 0x7f7b3815da60 2024-03-28T13:09:57.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.846+0000 7f7b2d400640 1 --2- 172.21.15.19:0/1875238560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3815da60 0x7f7b380cfe30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b380b1d90 tx=0x7f7b2002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276df277a8dbe8c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:57.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.847+0000 7f7b35400640 1 -- 172.21.15.19:0/1875238560 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20034070 con 0x7f7b3815da60 2024-03-28T13:09:57.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.847+0000 7f7b35400640 1 -- 172.21.15.19:0/1875238560 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b20030860 con 0x7f7b3815da60 2024-03-28T13:09:57.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.847+0000 7f7b35400640 1 -- 172.21.15.19:0/1875238560 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20030c30 con 0x7f7b3815da60 2024-03-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.848+0000 7f7b3ca00640 1 -- 172.21.15.19:0/1875238560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3815da60 msgr2=0x7f7b380cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.848+0000 7f7b3ca00640 1 --2- 172.21.15.19:0/1875238560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3815da60 0x7f7b380cfe30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b380b1d90 tx=0x7f7b2002b9a0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.848+0000 7f7b3ca00640 1 -- 172.21.15.19:0/1875238560 shutdown_connections 2024-03-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.848+0000 7f7b3ca00640 1 --2- 172.21.15.19:0/1875238560 >> v2:172.21.15.188:3300/0 conn(0x7f7b3815fe90 0x7f7b380d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.848+0000 7f7b3ca00640 1 --2- 172.21.15.19:0/1875238560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3815da60 0x7f7b380cfe30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.848+0000 7f7b3ca00640 1 -- 172.21.15.19:0/1875238560 >> 172.21.15.19:0/1875238560 conn(0x7f7b380b3e90 msgr2=0x7f7b380b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.848+0000 7f7b3ca00640 1 -- 172.21.15.19:0/1875238560 shutdown_connections 2024-03-28T13:09:57.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.848+0000 7f7b3ca00640 1 -- 172.21.15.19:0/1875238560 wait complete. 2024-03-28T13:09:57.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.850+0000 7f7b3ca00640 1 Processor -- start 2024-03-28T13:09:57.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.850+0000 7f7b3ca00640 1 -- start start 2024-03-28T13:09:57.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b3ca00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b3815da60 0x7f7b380ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:57.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b3ca00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b3815f900 0x7f7b3806d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:57.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b3ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3815fe90 0x7f7b380b7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:57.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b3ca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b380cece0 con 0x7f7b3815fe90 2024-03-28T13:09:57.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b3ca00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b380ceb60 con 0x7f7b3815f900 2024-03-28T13:09:57.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b3ca00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7b380cee60 con 0x7f7b3815da60 2024-03-28T13:09:57.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b36800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3815fe90 0x7f7b380b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:57.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b36800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3815fe90 0x7f7b380b7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41896/0 (socket says 172.21.15.19:41896) 2024-03-28T13:09:57.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b36800640 1 -- 172.21.15.19:0/2730990144 learned_addr learned my addr 172.21.15.19:0/2730990144 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:57.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b35e00640 1 --2- 172.21.15.19:0/2730990144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b3815da60 0x7f7b380ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b36800640 1 -- 172.21.15.19:0/2730990144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b3815da60 msgr2=0x7f7b380ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b36800640 1 --2- 172.21.15.19:0/2730990144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b3815da60 0x7f7b380ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b2d400640 1 --2- 172.21.15.19:0/2730990144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b3815f900 0x7f7b3806d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b36800640 1 -- 172.21.15.19:0/2730990144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b3815f900 msgr2=0x7f7b3806d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b36800640 1 --2- 172.21.15.19:0/2730990144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b3815f900 0x7f7b3806d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.851+0000 7f7b36800640 1 -- 172.21.15.19:0/2730990144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b38146c90 con 0x7f7b3815fe90 2024-03-28T13:09:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.852+0000 7f7b2d400640 1 --2- 172.21.15.19:0/2730990144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b3815f900 0x7f7b3806d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.852+0000 7f7b36800640 1 --2- 172.21.15.19:0/2730990144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3815fe90 0x7f7b380b7560 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b30007e60 tx=0x7f7b3000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:57.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.852+0000 7f7b2ea00640 1 -- 172.21.15.19:0/2730990144 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30014420 con 0x7f7b3815fe90 2024-03-28T13:09:57.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.852+0000 7f7b2ea00640 1 -- 172.21.15.19:0/2730990144 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b30019070 con 0x7f7b3815fe90 2024-03-28T13:09:57.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.852+0000 7f7b2ea00640 1 -- 172.21.15.19:0/2730990144 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30014820 con 0x7f7b3815fe90 2024-03-28T13:09:57.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.852+0000 7f7b3ca00640 1 -- 172.21.15.19:0/2730990144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b38146f70 con 0x7f7b3815fe90 2024-03-28T13:09:57.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.852+0000 7f7b3ca00640 1 -- 172.21.15.19:0/2730990144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b38147530 con 0x7f7b3815fe90 2024-03-28T13:09:57.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.854+0000 7f7b3ca00640 1 -- 172.21.15.19:0/2730990144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b38068cd0 con 0x7f7b3815fe90 2024-03-28T13:09:57.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.854+0000 7f7b2ea00640 1 -- 172.21.15.19:0/2730990144 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7b300149c0 con 0x7f7b3815fe90 2024-03-28T13:09:57.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.855+0000 7f7b2ea00640 1 --2- 172.21.15.19:0/2730990144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b1406c530 0x7f7b1406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:57.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.855+0000 7f7b2ea00640 1 -- 172.21.15.19:0/2730990144 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7b3008bf80 con 0x7f7b3815fe90 2024-03-28T13:09:57.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.856+0000 7f7b35e00640 1 --2- 172.21.15.19:0/2730990144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b1406c530 0x7f7b1406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:57.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.857+0000 7f7b35e00640 1 --2- 172.21.15.19:0/2730990144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b1406c530 0x7f7b1406e9f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7b28001640 tx=0x7f7b28009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:57.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:57.860+0000 7f7b2ea00640 1 -- 172.21.15.19:0/2730990144 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7b3005a5a0 con 0x7f7b3815fe90 2024-03-28T13:09:58.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.108+0000 7f7b3ca00640 1 -- 172.21.15.19:0/2730990144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b3815e9a0 con 0x7f7b3815fe90 2024-03-28T13:09:58.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.109+0000 7f7b2ea00640 1 -- 172.21.15.19:0/2730990144 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f7b30059ea0 con 0x7f7b3815fe90 2024-03-28T13:09:58.108 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:58.108 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/745791856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":745791856},{"type":"v1","addr":"172.21.15.19:6833","nonce":745791856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/639283827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":639283827},{"type":"v1","addr":"172.21.15.188:6836","nonce":639283827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2168352329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2168352329},{"type":"v1","addr":"172.21.15.188:6837","nonce":2168352329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:09:41.105429+0000","modified":"2024-03-28T13:09:44.203526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/731659331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":731659331},{"type":"v1","addr":"172.21.15.19:6835","nonce":731659331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4254,"qdb_cluster":[4254]},"id":1}]} 2024-03-28T13:09:58.109 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-03-28T13:09:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.112+0000 7f7b3ca00640 1 -- 172.21.15.19:0/2730990144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b1406c530 msgr2=0x7f7b1406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.112+0000 7f7b3ca00640 1 --2- 172.21.15.19:0/2730990144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b1406c530 0x7f7b1406e9f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7b28001640 tx=0x7f7b28009040 comp rx=0 tx=0).stop 2024-03-28T13:09:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.112+0000 7f7b3ca00640 1 -- 172.21.15.19:0/2730990144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3815fe90 msgr2=0x7f7b380b7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.112+0000 7f7b3ca00640 1 --2- 172.21.15.19:0/2730990144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3815fe90 0x7f7b380b7560 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b30007e60 tx=0x7f7b3000e4b0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.112+0000 7f7b3ca00640 1 -- 172.21.15.19:0/2730990144 shutdown_connections 2024-03-28T13:09:58.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.112+0000 7f7b3ca00640 1 --2- 172.21.15.19:0/2730990144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b1406c530 0x7f7b1406e9f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.112+0000 7f7b3ca00640 1 --2- 172.21.15.19:0/2730990144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b3815fe90 0x7f7b380b7560 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.112+0000 7f7b3ca00640 1 --2- 172.21.15.19:0/2730990144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b3815f900 0x7f7b3806d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.112+0000 7f7b3ca00640 1 --2- 172.21.15.19:0/2730990144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b3815da60 0x7f7b380ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.112+0000 7f7b3ca00640 1 -- 172.21.15.19:0/2730990144 >> 172.21.15.19:0/2730990144 conn(0x7f7b380b3e90 msgr2=0x7f7b3815de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.112+0000 7f7b3ca00640 1 -- 172.21.15.19:0/2730990144 shutdown_connections 2024-03-28T13:09:58.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.112+0000 7f7b3ca00640 1 -- 172.21.15.19:0/2730990144 wait complete. 2024-03-28T13:09:58.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:09:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.340+0000 7f0dc5800640 1 Processor -- start 2024-03-28T13:09:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.340+0000 7f0dc5800640 1 -- start start 2024-03-28T13:09:58.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.342+0000 7f0dc5800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0dc01442c0 0x7f0dc00ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:58.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.342+0000 7f0dc5800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0dc00cefe0 0x7f0dc00cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:58.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.342+0000 7f0dc5800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dc00d00d0 0x7f0dc006e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:58.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.342+0000 7f0dc5800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0dc0071d20 con 0x7f0dc00cefe0 2024-03-28T13:09:58.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.342+0000 7f0dc5800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f0dc00cf420 con 0x7f0dc01442c0 2024-03-28T13:09:58.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.342+0000 7f0dc5800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc0071460 con 0x7f0dc00d00d0 2024-03-28T13:09:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.342+0000 7f0dbf400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0dc00cefe0 0x7f0dc00cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.342+0000 7f0dbf400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0dc00cefe0 0x7f0dc00cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41906/0 (socket says 172.21.15.19:41906) 2024-03-28T13:09:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.342+0000 7f0dbf400640 1 -- 172.21.15.19:0/579828957 learned_addr learned my addr 172.21.15.19:0/579828957 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.343+0000 7f0dbfe00640 1 --2- 172.21.15.19:0/579828957 >> v2:172.21.15.188:3300/0 conn(0x7f0dc01442c0 0x7f0dc00ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.343+0000 7f0dc4e00640 1 --2- 172.21.15.19:0/579828957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dc00d00d0 0x7f0dc006e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.343+0000 7f0dbf400640 1 -- 172.21.15.19:0/579828957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dc00d00d0 msgr2=0x7f0dc006e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.343+0000 7f0dbf400640 1 --2- 172.21.15.19:0/579828957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dc00d00d0 0x7f0dc006e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.343+0000 7f0dbf400640 1 -- 172.21.15.19:0/579828957 >> v2:172.21.15.188:3300/0 conn(0x7f0dc01442c0 msgr2=0x7f0dc00ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.343+0000 7f0dbf400640 1 --2- 172.21.15.19:0/579828957 >> v2:172.21.15.188:3300/0 conn(0x7f0dc01442c0 0x7f0dc00ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.343+0000 7f0dbf400640 1 -- 172.21.15.19:0/579828957 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc00721b0 con 0x7f0dc00cefe0 2024-03-28T13:09:58.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.344+0000 7f0dbf400640 1 --2- 172.21.15.19:0/579828957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00cefe0 0x7f0dc00cfb90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00d0ae0 tx=0x7f0da802dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f6f76f6802feca server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.344+0000 7f0dbea00640 1 -- 172.21.15.19:0/579828957 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da803b070 con 0x7f0dc00cefe0 2024-03-28T13:09:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.345+0000 7f0dbea00640 1 -- 172.21.15.19:0/579828957 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0da802e950 con 0x7f0dc00cefe0 2024-03-28T13:09:58.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.345+0000 7f0dbea00640 1 -- 172.21.15.19:0/579828957 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da802ed20 con 0x7f0dc00cefe0 2024-03-28T13:09:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.345+0000 7f0dc5800640 1 -- 172.21.15.19:0/579828957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00cefe0 msgr2=0x7f0dc00cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.345+0000 7f0dc5800640 1 --2- 172.21.15.19:0/579828957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00cefe0 0x7f0dc00cfb90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00d0ae0 tx=0x7f0da802dba0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.345+0000 7f0dc5800640 1 -- 172.21.15.19:0/579828957 shutdown_connections 2024-03-28T13:09:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.345+0000 7f0dc5800640 1 --2- 172.21.15.19:0/579828957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dc00d00d0 0x7f0dc006e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.345+0000 7f0dc5800640 1 --2- 172.21.15.19:0/579828957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00cefe0 0x7f0dc00cfb90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.345+0000 7f0dc5800640 1 --2- 172.21.15.19:0/579828957 >> v2:172.21.15.188:3300/0 conn(0x7f0dc01442c0 0x7f0dc00ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.345+0000 7f0dc5800640 1 -- 172.21.15.19:0/579828957 >> 172.21.15.19:0/579828957 conn(0x7f0dc00ccad0 msgr2=0x7f0dc00cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.345+0000 7f0dc5800640 1 -- 172.21.15.19:0/579828957 shutdown_connections 2024-03-28T13:09:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.346+0000 7f0dc5800640 1 -- 172.21.15.19:0/579828957 wait complete. 2024-03-28T13:09:58.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.348+0000 7f0dc5800640 1 Processor -- start 2024-03-28T13:09:58.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.348+0000 7f0dc5800640 1 -- start start 2024-03-28T13:09:58.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.349+0000 7f0dc5800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00cefe0 0x7f0dc0159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:58.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.349+0000 7f0dc5800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dc00d00d0 0x7f0dc00b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:58.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.349+0000 7f0dc5800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dc01442c0 0x7f0dc013c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:58.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.349+0000 7f0dc5800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc00d0770 con 0x7f0dc00cefe0 2024-03-28T13:09:58.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.349+0000 7f0dc5800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc00719b0 con 0x7f0dc00d00d0 2024-03-28T13:09:58.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.349+0000 7f0dc5800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc01446a0 con 0x7f0dc01442c0 2024-03-28T13:09:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.349+0000 7f0dbfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00cefe0 0x7f0dc0159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.350+0000 7f0dbfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00cefe0 0x7f0dc0159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41916/0 (socket says 172.21.15.19:41916) 2024-03-28T13:09:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.350+0000 7f0dbfe00640 1 -- 172.21.15.19:0/2252980681 learned_addr learned my addr 172.21.15.19:0/2252980681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.350+0000 7f0dc4e00640 1 --2- 172.21.15.19:0/2252980681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dc01442c0 0x7f0dc013c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.350+0000 7f0dbf400640 1 --2- 172.21.15.19:0/2252980681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dc00d00d0 0x7f0dc00b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.350+0000 7f0dbfe00640 1 -- 172.21.15.19:0/2252980681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dc01442c0 msgr2=0x7f0dc013c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.350+0000 7f0dbfe00640 1 --2- 172.21.15.19:0/2252980681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dc01442c0 0x7f0dc013c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.350+0000 7f0dbfe00640 1 -- 172.21.15.19:0/2252980681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dc00d00d0 msgr2=0x7f0dc00b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.350+0000 7f0dbfe00640 1 --2- 172.21.15.19:0/2252980681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dc00d00d0 0x7f0dc00b4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.350+0000 7f0dbfe00640 1 -- 172.21.15.19:0/2252980681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc015e1e0 con 0x7f0dc00cefe0 2024-03-28T13:09:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.350+0000 7f0dc4e00640 1 --2- 172.21.15.19:0/2252980681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dc01442c0 0x7f0dc013c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:58.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.350+0000 7f0dbfe00640 1 --2- 172.21.15.19:0/2252980681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00cefe0 0x7f0dc0159070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0db0007e40 tx=0x7f0db000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.351+0000 7f0db7e00640 1 -- 172.21.15.19:0/2252980681 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0014620 con 0x7f0dc00cefe0 2024-03-28T13:09:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.351+0000 7f0db7e00640 1 -- 172.21.15.19:0/2252980681 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db0019070 con 0x7f0dc00cefe0 2024-03-28T13:09:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.351+0000 7f0db7e00640 1 -- 172.21.15.19:0/2252980681 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db0014a00 con 0x7f0dc00cefe0 2024-03-28T13:09:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.351+0000 7f0dc5800640 1 -- 172.21.15.19:0/2252980681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc00ae2a0 con 0x7f0dc00cefe0 2024-03-28T13:09:58.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.351+0000 7f0dc5800640 1 -- 172.21.15.19:0/2252980681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc00ae860 con 0x7f0dc00cefe0 2024-03-28T13:09:58.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.352+0000 7f0dc5800640 1 -- 172.21.15.19:0/2252980681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc0075940 con 0x7f0dc00cefe0 2024-03-28T13:09:58.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.353+0000 7f0db7e00640 1 -- 172.21.15.19:0/2252980681 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0db0014ba0 con 0x7f0dc00cefe0 2024-03-28T13:09:58.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.354+0000 7f0db7e00640 1 --2- 172.21.15.19:0/2252980681 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0d9c06c530 0x7f0d9c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:58.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.354+0000 7f0db7e00640 1 -- 172.21.15.19:0/2252980681 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0db008bf30 con 0x7f0dc00cefe0 2024-03-28T13:09:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.355+0000 7f0dbf400640 1 --2- 172.21.15.19:0/2252980681 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0d9c06c530 0x7f0d9c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:58.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.356+0000 7f0dbf400640 1 --2- 172.21.15.19:0/2252980681 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0d9c06c530 0x7f0d9c06e9f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0da8038000 tx=0x7f0da8032000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:58.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.359+0000 7f0db7e00640 1 -- 172.21.15.19:0/2252980681 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0db005a540 con 0x7f0dc00cefe0 2024-03-28T13:09:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.562+0000 7f0dc5800640 1 -- 172.21.15.19:0/2252980681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0dc00cd190 con 0x7f0dc00cefe0 2024-03-28T13:09:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.564+0000 7f0db7e00640 1 -- 172.21.15.19:0/2252980681 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16609 (secure 0 0 0) 0x7f0db0059e40 con 0x7f0dc00cefe0 2024-03-28T13:09:58.564 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:58.564 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":23,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:42.668077+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:09:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.567+0000 7f0dc5800640 1 -- 172.21.15.19:0/2252980681 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0d9c06c530 msgr2=0x7f0d9c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.567+0000 7f0dc5800640 1 --2- 172.21.15.19:0/2252980681 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0d9c06c530 0x7f0d9c06e9f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0da8038000 tx=0x7f0da8032000 comp rx=0 tx=0).stop 2024-03-28T13:09:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.567+0000 7f0dc5800640 1 -- 172.21.15.19:0/2252980681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00cefe0 msgr2=0x7f0dc0159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:58.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.567+0000 7f0dc5800640 1 --2- 172.21.15.19:0/2252980681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00cefe0 0x7f0dc0159070 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0db0007e40 tx=0x7f0db000e4b0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.567+0000 7f0dc5800640 1 -- 172.21.15.19:0/2252980681 shutdown_connections 2024-03-28T13:09:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.567+0000 7f0dc5800640 1 --2- 172.21.15.19:0/2252980681 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0d9c06c530 0x7f0d9c06e9f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.567+0000 7f0dc5800640 1 --2- 172.21.15.19:0/2252980681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dc01442c0 0x7f0dc013c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.567+0000 7f0dc5800640 1 --2- 172.21.15.19:0/2252980681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dc00d00d0 0x7f0dc00b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.567+0000 7f0dc5800640 1 --2- 172.21.15.19:0/2252980681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0dc00cefe0 0x7f0dc0159070 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.567+0000 7f0dc5800640 1 -- 172.21.15.19:0/2252980681 >> 172.21.15.19:0/2252980681 conn(0x7f0dc00ccad0 msgr2=0x7f0dc006ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.567+0000 7f0dc5800640 1 -- 172.21.15.19:0/2252980681 shutdown_connections 2024-03-28T13:09:58.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.567+0000 7f0dc5800640 1 -- 172.21.15.19:0/2252980681 wait complete. 2024-03-28T13:09:58.585 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:09:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.800+0000 7fcd22600640 1 Processor -- start 2024-03-28T13:09:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.801+0000 7fcd22600640 1 -- start start 2024-03-28T13:09:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.802+0000 7fcd22600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd1c0d2bf0 0x7fcd1c0cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.802+0000 7fcd22600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcd1c0d0c30 0x7fcd1c0d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.802+0000 7fcd22600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd1c0d3140 con 0x7fcd1c15fa10 2024-03-28T13:09:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.802+0000 7fcd22600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fcd1c0d15c0 con 0x7fcd1c0d0c30 2024-03-28T13:09:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.802+0000 7fcd22600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1c0d3420 con 0x7fcd1c0d2bf0 2024-03-28T13:09:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.802+0000 7fcd13e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd1c0d2bf0 0x7fcd1c0cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.802+0000 7fcd13e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd1c0d2bf0 0x7fcd1c0cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:51628/0 (socket says 172.21.15.19:51628) 2024-03-28T13:09:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.802+0000 7fcd13e00640 1 -- 172.21.15.19:0/2769825588 learned_addr learned my addr 172.21.15.19:0/2769825588 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.803+0000 7fcd1be00640 1 --2- 172.21.15.19:0/2769825588 >> v2:172.21.15.188:3300/0 conn(0x7fcd1c0d0c30 0x7fcd1c0d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.803+0000 7fcd1be00640 1 -- 172.21.15.19:0/2769825588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd1c0d2bf0 msgr2=0x7fcd1c0cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.803+0000 7fcd1be00640 1 --2- 172.21.15.19:0/2769825588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd1c0d2bf0 0x7fcd1c0cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.803+0000 7fcd1be00640 1 -- 172.21.15.19:0/2769825588 >> v1:172.21.15.188:6789/0 conn(0x7fcd1c15fa10 legacy=0x7fcd1c0b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.803+0000 7fcd1be00640 1 -- 172.21.15.19:0/2769825588 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c0cf2b0 con 0x7fcd1c0d0c30 2024-03-28T13:09:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.803+0000 7fcd13e00640 1 --2- 172.21.15.19:0/2769825588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd1c0d2bf0 0x7fcd1c0cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:09:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.804+0000 7fcd1be00640 1 --2- 172.21.15.19:0/2769825588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd1c0d0c30 0x7fcd1c0d1080 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009d90 tx=0x7fcd1402bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41bc55fa6071124a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.805+0000 7fcd1aa00640 1 -- 172.21.15.19:0/2769825588 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14039070 con 0x7fcd1c0d0c30 2024-03-28T13:09:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.805+0000 7fcd1aa00640 1 -- 172.21.15.19:0/2769825588 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd1402cce0 con 0x7fcd1c0d0c30 2024-03-28T13:09:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.805+0000 7fcd1aa00640 1 -- 172.21.15.19:0/2769825588 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14034660 con 0x7fcd1c0d0c30 2024-03-28T13:09:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.805+0000 7fcd22600640 1 -- 172.21.15.19:0/2769825588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd1c0d0c30 msgr2=0x7fcd1c0d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.805+0000 7fcd22600640 1 --2- 172.21.15.19:0/2769825588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd1c0d0c30 0x7fcd1c0d1080 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009d90 tx=0x7fcd1402bf30 comp rx=0 tx=0).stop 2024-03-28T13:09:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.806+0000 7fcd22600640 1 -- 172.21.15.19:0/2769825588 shutdown_connections 2024-03-28T13:09:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.806+0000 7fcd22600640 1 --2- 172.21.15.19:0/2769825588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd1c0d0c30 0x7fcd1c0d1080 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.806+0000 7fcd22600640 1 --2- 172.21.15.19:0/2769825588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd1c0d2bf0 0x7fcd1c0cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.806+0000 7fcd22600640 1 -- 172.21.15.19:0/2769825588 >> 172.21.15.19:0/2769825588 conn(0x7fcd1c08bfb0 msgr2=0x7fcd1c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.806+0000 7fcd22600640 1 -- 172.21.15.19:0/2769825588 shutdown_connections 2024-03-28T13:09:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.806+0000 7fcd22600640 1 -- 172.21.15.19:0/2769825588 wait complete. 2024-03-28T13:09:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.807+0000 7fcd22600640 1 Processor -- start 2024-03-28T13:09:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.808+0000 7fcd22600640 1 -- start start 2024-03-28T13:09:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.808+0000 7fcd22600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd1c0d0c30 0x7fcd1c15e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.808+0000 7fcd22600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd1c0d2bf0 0x7fcd1c152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.808+0000 7fcd22600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd1c15fa10 0x7fcd1c15ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.808+0000 7fcd22600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c0d02d0 con 0x7fcd1c0d0c30 2024-03-28T13:09:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.808+0000 7fcd22600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1c0cf870 con 0x7fcd1c0d2bf0 2024-03-28T13:09:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.808+0000 7fcd22600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1c0d0450 con 0x7fcd1c15fa10 2024-03-28T13:09:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.809+0000 7fcd1b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd1c0d0c30 0x7fcd1c15e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.809+0000 7fcd1b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd1c0d0c30 0x7fcd1c15e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41930/0 (socket says 172.21.15.19:41930) 2024-03-28T13:09:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.809+0000 7fcd1b400640 1 -- 172.21.15.19:0/3769053298 learned_addr learned my addr 172.21.15.19:0/3769053298 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.809+0000 7fcd1be00640 1 --2- 172.21.15.19:0/3769053298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd1c15fa10 0x7fcd1c15ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.809+0000 7fcd13e00640 1 --2- 172.21.15.19:0/3769053298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd1c0d2bf0 0x7fcd1c152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.809+0000 7fcd1b400640 1 -- 172.21.15.19:0/3769053298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd1c15fa10 msgr2=0x7fcd1c15ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.809+0000 7fcd1b400640 1 --2- 172.21.15.19:0/3769053298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd1c15fa10 0x7fcd1c15ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.809+0000 7fcd1b400640 1 -- 172.21.15.19:0/3769053298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd1c0d2bf0 msgr2=0x7fcd1c152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.809+0000 7fcd1b400640 1 --2- 172.21.15.19:0/3769053298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd1c0d2bf0 0x7fcd1c152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.809+0000 7fcd1b400640 1 -- 172.21.15.19:0/3769053298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1c15a150 con 0x7fcd1c0d0c30 2024-03-28T13:09:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.809+0000 7fcd13e00640 1 --2- 172.21.15.19:0/3769053298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd1c0d2bf0 0x7fcd1c152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.810+0000 7fcd1b400640 1 --2- 172.21.15.19:0/3769053298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd1c0d0c30 0x7fcd1c15e080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00c3c0 tx=0x7fcd0c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.810+0000 7fcd13400640 1 -- 172.21.15.19:0/3769053298 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c0052d0 con 0x7fcd1c0d0c30 2024-03-28T13:09:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.810+0000 7fcd13400640 1 -- 172.21.15.19:0/3769053298 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd0c005490 con 0x7fcd1c0d0c30 2024-03-28T13:09:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.810+0000 7fcd22600640 1 -- 172.21.15.19:0/3769053298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1c153ad0 con 0x7fcd1c0d0c30 2024-03-28T13:09:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.810+0000 7fcd13400640 1 -- 172.21.15.19:0/3769053298 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c007260 con 0x7fcd1c0d0c30 2024-03-28T13:09:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.810+0000 7fcd22600640 1 -- 172.21.15.19:0/3769053298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd1c15a6d0 con 0x7fcd1c0d0c30 2024-03-28T13:09:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.811+0000 7fcd22600640 1 -- 172.21.15.19:0/3769053298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd1c068cd0 con 0x7fcd1c0d0c30 2024-03-28T13:09:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.812+0000 7fcd13400640 1 -- 172.21.15.19:0/3769053298 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcd0c007400 con 0x7fcd1c0d0c30 2024-03-28T13:09:58.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.813+0000 7fcd13400640 1 --2- 172.21.15.19:0/3769053298 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fccf806c560 0x7fccf806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:58.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.813+0000 7fcd13400640 1 -- 172.21.15.19:0/3769053298 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcd0c08ab80 con 0x7fcd1c0d0c30 2024-03-28T13:09:58.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.814+0000 7fcd13e00640 1 --2- 172.21.15.19:0/3769053298 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fccf806c560 0x7fccf806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:58.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.815+0000 7fcd13e00640 1 --2- 172.21.15.19:0/3769053298 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fccf806c560 0x7fccf806ea20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c158210 tx=0x7fcd04008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:58.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:58.818+0000 7fcd13400640 1 -- 172.21.15.19:0/3769053298 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcd0c059190 con 0x7fcd1c0d0c30 2024-03-28T13:09:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.061+0000 7fcd22600640 1 -- 172.21.15.19:0/3769053298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd1c0ba3d0 con 0x7fcd1c0d0c30 2024-03-28T13:09:59.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.061+0000 7fcd13400640 1 -- 172.21.15.19:0/3769053298 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7fcd0c058a90 con 0x7fcd1c0d0c30 2024-03-28T13:09:59.061 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:09:59.061 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/745791856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":745791856},{"type":"v1","addr":"172.21.15.19:6833","nonce":745791856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/639283827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":639283827},{"type":"v1","addr":"172.21.15.188:6836","nonce":639283827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2168352329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2168352329},{"type":"v1","addr":"172.21.15.188:6837","nonce":2168352329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:09:41.105429+0000","modified":"2024-03-28T13:09:44.203526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/731659331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":731659331},{"type":"v1","addr":"172.21.15.19:6835","nonce":731659331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4254,"qdb_cluster":[4254]},"id":1}]} 2024-03-28T13:09:59.061 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-03-28T13:09:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.064+0000 7fcd22600640 1 -- 172.21.15.19:0/3769053298 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fccf806c560 msgr2=0x7fccf806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.064+0000 7fcd22600640 1 --2- 172.21.15.19:0/3769053298 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fccf806c560 0x7fccf806ea20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c158210 tx=0x7fcd04008040 comp rx=0 tx=0).stop 2024-03-28T13:09:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.064+0000 7fcd22600640 1 -- 172.21.15.19:0/3769053298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd1c0d0c30 msgr2=0x7fcd1c15e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.064+0000 7fcd22600640 1 --2- 172.21.15.19:0/3769053298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd1c0d0c30 0x7fcd1c15e080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c00c3c0 tx=0x7fcd0c00c880 comp rx=0 tx=0).stop 2024-03-28T13:09:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.064+0000 7fcd22600640 1 -- 172.21.15.19:0/3769053298 shutdown_connections 2024-03-28T13:09:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.064+0000 7fcd22600640 1 --2- 172.21.15.19:0/3769053298 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fccf806c560 0x7fccf806ea20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.064+0000 7fcd22600640 1 --2- 172.21.15.19:0/3769053298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd1c15fa10 0x7fcd1c15ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.064+0000 7fcd22600640 1 --2- 172.21.15.19:0/3769053298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd1c0d2bf0 0x7fcd1c152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.064+0000 7fcd22600640 1 --2- 172.21.15.19:0/3769053298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd1c0d0c30 0x7fcd1c15e080 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.064+0000 7fcd22600640 1 -- 172.21.15.19:0/3769053298 >> 172.21.15.19:0/3769053298 conn(0x7fcd1c08bfb0 msgr2=0x7fcd1c0b9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.065+0000 7fcd22600640 1 -- 172.21.15.19:0/3769053298 shutdown_connections 2024-03-28T13:09:59.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.065+0000 7fcd22600640 1 -- 172.21.15.19:0/3769053298 wait complete. 2024-03-28T13:09:59.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:09:41.105429+0000', 'modified': '2024-03-28T13:09:44.203526+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4254}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4254': {'gid': 4254, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6835/731659331', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6834', 'nonce': 731659331}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 731659331}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4254, 'qdb_cluster': [4254]} 2024-03-28T13:09:59.485 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T13:09:59.486 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-28T13:09:59.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.714+0000 7fcf3ac00640 1 Processor -- start 2024-03-28T13:09:59.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.714+0000 7fcf3ac00640 1 -- start start 2024-03-28T13:09:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.715+0000 7fcf3ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf340d0cb0 0x7fcf340d4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.715+0000 7fcf3ac00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf340d4590 0x7fcf340cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.715+0000 7fcf3ac00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf340ceef0 con 0x7fcf3415e6d0 2024-03-28T13:09:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.715+0000 7fcf3ac00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcf3415eab0 con 0x7fcf340d4590 2024-03-28T13:09:59.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.715+0000 7fcf3ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcf340d1220 con 0x7fcf340d0cb0 2024-03-28T13:09:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.716+0000 7fcf33e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fcf3415e6d0 0x7fcf340d21e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:48978/0 (socket says 172.21.15.19:48978) 2024-03-28T13:09:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.716+0000 7fcf33e00640 1 -- 172.21.15.19:0/2055352155 learned_addr learned my addr 172.21.15.19:0/2055352155 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:09:59.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.716+0000 7fcf33400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf340d0cb0 0x7fcf340d4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.716+0000 7fcf38c00640 1 --2- 172.21.15.19:0/2055352155 >> v2:172.21.15.19:3300/0 conn(0x7fcf340d4590 0x7fcf340cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.716+0000 7fcf38c00640 1 -- 172.21.15.19:0/2055352155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf340d0cb0 msgr2=0x7fcf340d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:59.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.716+0000 7fcf38c00640 1 --2- 172.21.15.19:0/2055352155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf340d0cb0 0x7fcf340d4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.716+0000 7fcf38c00640 1 -- 172.21.15.19:0/2055352155 >> v1:172.21.15.188:6789/0 conn(0x7fcf3415e6d0 legacy=0x7fcf340d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:59.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.716+0000 7fcf32a00640 1 -- 172.21.15.19:0/2055352155 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1972146949 0 0) 0x7fcf340ceef0 con 0x7fcf3415e6d0 2024-03-28T13:09:59.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.716+0000 7fcf38c00640 1 -- 172.21.15.19:0/2055352155 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf340d1500 con 0x7fcf340d4590 2024-03-28T13:09:59.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.719+0000 7fcf38c00640 1 --2- 172.21.15.19:0/2055352155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340d4590 0x7fcf340cf3d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0097a0 tx=0x7fcf2c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f222f99de7794a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:59.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.720+0000 7fcf32a00640 1 -- 172.21.15.19:0/2055352155 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c035020 con 0x7fcf340d4590 2024-03-28T13:09:59.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.720+0000 7fcf32a00640 1 -- 172.21.15.19:0/2055352155 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2c031830 con 0x7fcf340d4590 2024-03-28T13:09:59.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.720+0000 7fcf32a00640 1 -- 172.21.15.19:0/2055352155 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c031c00 con 0x7fcf340d4590 2024-03-28T13:09:59.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.720+0000 7fcf3ac00640 1 -- 172.21.15.19:0/2055352155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340d4590 msgr2=0x7fcf340cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:59.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.720+0000 7fcf3ac00640 1 --2- 172.21.15.19:0/2055352155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340d4590 0x7fcf340cf3d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0097a0 tx=0x7fcf2c02b8d0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.720+0000 7fcf3ac00640 1 -- 172.21.15.19:0/2055352155 shutdown_connections 2024-03-28T13:09:59.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.720+0000 7fcf3ac00640 1 --2- 172.21.15.19:0/2055352155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340d4590 0x7fcf340cf3d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.720+0000 7fcf3ac00640 1 --2- 172.21.15.19:0/2055352155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf340d0cb0 0x7fcf340d4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.720+0000 7fcf3ac00640 1 -- 172.21.15.19:0/2055352155 >> 172.21.15.19:0/2055352155 conn(0x7fcf3409d7b0 msgr2=0x7fcf3408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:59.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.721+0000 7fcf3ac00640 1 -- 172.21.15.19:0/2055352155 shutdown_connections 2024-03-28T13:09:59.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.721+0000 7fcf3ac00640 1 -- 172.21.15.19:0/2055352155 wait complete. 2024-03-28T13:09:59.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.722+0000 7fcf3ac00640 1 Processor -- start 2024-03-28T13:09:59.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.722+0000 7fcf3ac00640 1 -- start start 2024-03-28T13:09:59.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.723+0000 7fcf3ac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340d0cb0 0x7fcf3414f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.723+0000 7fcf3ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf340d4590 0x7fcf3409e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.723+0000 7fcf3ac00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3415e6d0 0x7fcf340c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.723+0000 7fcf3ac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcf340d4a20 con 0x7fcf340d0cb0 2024-03-28T13:09:59.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.723+0000 7fcf3ac00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf340d1ac0 con 0x7fcf3415e6d0 2024-03-28T13:09:59.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.723+0000 7fcf3ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3406c890 con 0x7fcf340d4590 2024-03-28T13:09:59.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.724+0000 7fcf33e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340d0cb0 0x7fcf3414f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.724+0000 7fcf33e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340d0cb0 0x7fcf3414f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41944/0 (socket says 172.21.15.19:41944) 2024-03-28T13:09:59.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.724+0000 7fcf33e00640 1 -- 172.21.15.19:0/2177105227 learned_addr learned my addr 172.21.15.19:0/2177105227 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:59.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.724+0000 7fcf33400640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf340d4590 0x7fcf3409e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.724+0000 7fcf38c00640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3415e6d0 0x7fcf340c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.724+0000 7fcf33e00640 1 -- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf340d4590 msgr2=0x7fcf3409e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:59.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.724+0000 7fcf33e00640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf340d4590 0x7fcf3409e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.724+0000 7fcf33e00640 1 -- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3415e6d0 msgr2=0x7fcf340c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:59.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.724+0000 7fcf33e00640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3415e6d0 0x7fcf340c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.724+0000 7fcf33e00640 1 -- 172.21.15.19:0/2177105227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf340a14d0 con 0x7fcf340d0cb0 2024-03-28T13:09:59.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.724+0000 7fcf33400640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf340d4590 0x7fcf3409e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:59.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.724+0000 7fcf38c00640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3415e6d0 0x7fcf340c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:59.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.725+0000 7fcf33e00640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340d0cb0 0x7fcf3414f810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400c7f0 tx=0x7fcf2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:59.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.725+0000 7fcf2be00640 1 -- 172.21.15.19:0/2177105227 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24018070 con 0x7fcf340d0cb0 2024-03-28T13:09:59.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.725+0000 7fcf2be00640 1 -- 172.21.15.19:0/2177105227 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2400a5e0 con 0x7fcf340d0cb0 2024-03-28T13:09:59.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.725+0000 7fcf2be00640 1 -- 172.21.15.19:0/2177105227 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2400a9d0 con 0x7fcf340d0cb0 2024-03-28T13:09:59.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.725+0000 7fcf3ac00640 1 -- 172.21.15.19:0/2177105227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf340a17b0 con 0x7fcf340d0cb0 2024-03-28T13:09:59.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.725+0000 7fcf3ac00640 1 -- 172.21.15.19:0/2177105227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf34150240 con 0x7fcf340d0cb0 2024-03-28T13:09:59.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.728+0000 7fcf2be00640 1 -- 172.21.15.19:0/2177105227 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcf2401b070 con 0x7fcf340d0cb0 2024-03-28T13:09:59.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.728+0000 7fcf2be00640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcf1006c530 0x7fcf1006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.729+0000 7fcf2be00640 1 -- 172.21.15.19:0/2177105227 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcf2408ad70 con 0x7fcf340d0cb0 2024-03-28T13:09:59.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.729+0000 7fcf3d995740 1 Processor -- start 2024-03-28T13:09:59.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.729+0000 7fcf33400640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcf1006c530 0x7fcf1006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.729+0000 7fcf3d995740 1 -- start start 2024-03-28T13:09:59.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.730+0000 7fcf3d995740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987c4fd0 0x559c98797f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.730+0000 7fcf3d995740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c53b0 0x559c987c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.730+0000 7fcf3d995740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987984b0 0x559c987c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.730+0000 7fcf33400640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcf1006c530 0x7fcf1006e9f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcf340cafa0 tx=0x7fcf1c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:59.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.730+0000 7fcf3d995740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559c9875f920 con 0x559c987984b0 2024-03-28T13:09:59.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.730+0000 7fcf3d995740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559c986affa0 con 0x559c987c53b0 2024-03-28T13:09:59.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.730+0000 7fcf3d995740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x559c98701f90 con 0x559c987c4fd0 2024-03-28T13:09:59.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.730+0000 7fcf38c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987984b0 0x559c987c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.730+0000 7fcf38c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987984b0 0x559c987c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41948/0 (socket says 172.21.15.19:41948) 2024-03-28T13:09:59.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.730+0000 7fcf38c00640 1 -- 172.21.15.19:0/1323341612 learned_addr learned my addr 172.21.15.19:0/1323341612 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:59.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.731+0000 7fcf33400640 1 --2- 172.21.15.19:0/1323341612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c53b0 0x559c987c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.731+0000 7fcf33e00640 1 --2- 172.21.15.19:0/1323341612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987c4fd0 0x559c98797f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.731+0000 7fcf38c00640 1 -- 172.21.15.19:0/1323341612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987c4fd0 msgr2=0x559c98797f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.731+0000 7fcf38c00640 1 --2- 172.21.15.19:0/1323341612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987c4fd0 0x559c98797f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.731+0000 7fcf38c00640 1 -- 172.21.15.19:0/1323341612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c53b0 msgr2=0x559c987c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.731+0000 7fcf38c00640 1 --2- 172.21.15.19:0/1323341612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c53b0 0x559c987c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.731+0000 7fcf38c00640 1 -- 172.21.15.19:0/1323341612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c984162d0 con 0x559c987984b0 2024-03-28T13:09:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.731+0000 7fcf33e00640 1 --2- 172.21.15.19:0/1323341612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987c4fd0 0x559c98797f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:09:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.731+0000 7fcf38c00640 1 --2- 172.21.15.19:0/1323341612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987984b0 0x559c987c82a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c02bee0 tx=0x7fcf2c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947313044c01309e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.732+0000 7fcf23e00640 1 -- 172.21.15.19:0/1323341612 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c049070 con 0x559c987984b0 2024-03-28T13:09:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.732+0000 7fcf23e00640 1 -- 172.21.15.19:0/1323341612 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2c031970 con 0x559c987984b0 2024-03-28T13:09:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.732+0000 7fcf23e00640 1 -- 172.21.15.19:0/1323341612 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c035040 con 0x559c987984b0 2024-03-28T13:09:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.733+0000 7fcf3d995740 1 -- 172.21.15.19:0/1323341612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987984b0 msgr2=0x559c987c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.733+0000 7fcf3d995740 1 --2- 172.21.15.19:0/1323341612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987984b0 0x559c987c82a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c02bee0 tx=0x7fcf2c030000 comp rx=0 tx=0).stop 2024-03-28T13:09:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.733+0000 7fcf3d995740 1 -- 172.21.15.19:0/1323341612 shutdown_connections 2024-03-28T13:09:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.733+0000 7fcf3d995740 1 --2- 172.21.15.19:0/1323341612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987984b0 0x559c987c82a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.733+0000 7fcf3d995740 1 --2- 172.21.15.19:0/1323341612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c53b0 0x559c987c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.733+0000 7fcf3d995740 1 --2- 172.21.15.19:0/1323341612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987c4fd0 0x559c98797f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.733+0000 7fcf3d995740 1 -- 172.21.15.19:0/1323341612 >> 172.21.15.19:0/1323341612 conn(0x559c9879a1e0 msgr2=0x559c987e2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:59.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.733+0000 7fcf3d995740 1 -- 172.21.15.19:0/1323341612 shutdown_connections 2024-03-28T13:09:59.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.733+0000 7fcf3d995740 1 -- 172.21.15.19:0/1323341612 wait complete. 2024-03-28T13:09:59.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.733+0000 7fcf3d995740 1 -- start start 2024-03-28T13:09:59.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.734+0000 7fcf3d995740 1 Processor -- start 2024-03-28T13:09:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.735+0000 7fcf3d995740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c4fd0 0x559c987b5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.735+0000 7fcf3d995740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c53b0 0x559c987d9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.735+0000 7fcf3d995740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987f2a20 0x559c987f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.735+0000 7fcf3d995740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559c986fdf40 con 0x559c987c4fd0 2024-03-28T13:09:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.735+0000 7fcf3d995740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559c986aa650 con 0x559c987c53b0 2024-03-28T13:09:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.735+0000 7fcf3d995740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x559c986bda10 con 0x559c987f2a20 2024-03-28T13:09:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.735+0000 7fcf3d995740 10 client.0 _ll_register_callbacks cb 0x559c986fcff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:09:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.735+0000 7fcf33e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c4fd0 0x559c987b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.735+0000 7fcf33e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c4fd0 0x559c987b5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41960/0 (socket says 172.21.15.19:41960) 2024-03-28T13:09:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.735+0000 7fcf33e00640 1 -- 172.21.15.19:0/1829296672 learned_addr learned my addr 172.21.15.19:0/1829296672 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.735+0000 7fcf33400640 1 --2- 172.21.15.19:0/1829296672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c53b0 0x559c987d9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.735+0000 7fcf38c00640 1 --2- 172.21.15.19:0/1829296672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987f2a20 0x559c987f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.735+0000 7fcf33e00640 1 -- 172.21.15.19:0/1829296672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987f2a20 msgr2=0x559c987f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.736+0000 7fcf33e00640 1 --2- 172.21.15.19:0/1829296672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987f2a20 0x559c987f2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.736+0000 7fcf33e00640 1 -- 172.21.15.19:0/1829296672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c53b0 msgr2=0x559c987d9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.736+0000 7fcf33e00640 1 --2- 172.21.15.19:0/1829296672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c53b0 0x559c987d9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.736+0000 7fcf33e00640 1 -- 172.21.15.19:0/1829296672 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c984e06b0 con 0x559c987c4fd0 2024-03-28T13:09:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.736+0000 7fcf38c00640 1 --2- 172.21.15.19:0/1829296672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987f2a20 0x559c987f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:09:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.736+0000 7fcf33e00640 1 --2- 172.21.15.19:0/1829296672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c4fd0 0x559c987b5480 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcf240588f0 tx=0x7fcf24058bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2e1203568aca36 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.736+0000 7fcf20c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:09:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.736+0000 7fcf20c00640 1 -- 172.21.15.19:0/1829296672 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2400ad70 con 0x559c987c4fd0 2024-03-28T13:09:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.736+0000 7fcf20c00640 1 -- 172.21.15.19:0/1829296672 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcf24058040 con 0x559c987c4fd0 2024-03-28T13:09:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.736+0000 7fcf3d995740 1 -- 172.21.15.19:0/1829296672 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf2c031970 con 0x559c987c4fd0 2024-03-28T13:09:59.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.736+0000 7fcf20c00640 1 -- 172.21.15.19:0/1829296672 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fcf24058290 con 0x559c987c4fd0 2024-03-28T13:09:59.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.736+0000 7fcf3d995740 10 client.4632 fetch_fsmap learned FSMap version 8 2024-03-28T13:09:59.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.736+0000 7fcf3d995740 1 -- 172.21.15.19:0/1829296672 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x559c987c2fe0 con 0x559c987c4fd0 2024-03-28T13:09:59.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.737+0000 7fcf20c00640 1 -- 172.21.15.19:0/1829296672 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2c031970 con 0x559c987c4fd0 2024-03-28T13:09:59.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.737+0000 7fcf20c00640 1 -- 172.21.15.19:0/1829296672 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fcf24082b60 con 0x559c987c4fd0 2024-03-28T13:09:59.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.737+0000 7fcf3d995740 10 client.4632 fetch_fsmap finished waiting for FSMap version 8 2024-03-28T13:09:59.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.737+0000 7fcf3d995740 10 client.4632 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T13:09:59.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.737+0000 7fcf3d995740 20 client.4632 populate_metadata read hostname 'smithi019' 2024-03-28T13:09:59.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.737+0000 7fcf3d995740 1 --2- 172.21.15.19:0/1829296672 >> [v2:172.21.15.19:6834/731659331,v1:172.21.15.19:6835/731659331] conn(0x559c987febd0 0x559c98800fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.737+0000 7fcf3d995740 4 client.4632 mds_command: new command op to 4254 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:09:59.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.737+0000 7fcf3d995740 1 -- 172.21.15.19:0/1829296672 --> [v2:172.21.15.19:6834/731659331,v1:172.21.15.19:6835/731659331] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c9870f4d0 con 0x559c987febd0 2024-03-28T13:09:59.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.737+0000 7fcf33400640 1 --2- 172.21.15.19:0/1829296672 >> [v2:172.21.15.19:6834/731659331,v1:172.21.15.19:6835/731659331] conn(0x559c987febd0 0x559c98800fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.738+0000 7fcf33400640 1 --2- 172.21.15.19:0/1829296672 >> [v2:172.21.15.19:6834/731659331,v1:172.21.15.19:6835/731659331] conn(0x559c987febd0 0x559c98800fc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c3d8967471b240b server_cookie=79066c90456b17a8 in_seq=0 out_seq=0 2024-03-28T13:09:59.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.738+0000 7fcf20c00640 10 client.4632 ms_handle_connect on v2:172.21.15.19:6834/731659331 2024-03-28T13:09:59.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.739+0000 7fcf20c00640 1 -- 172.21.15.19:0/1829296672 <== mds.0 v2:172.21.15.19:6834/731659331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c9870f4d0 con 0x559c987febd0 2024-03-28T13:09:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.739+0000 7fcf20c00640 10 client.4632 handle_command_reply: tid=0 2024-03-28T13:09:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.739+0000 7fcf3d995740 1 client.4632 shutdown 2024-03-28T13:09:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.739+0000 7fcf3d995740 1 -- 172.21.15.19:0/1829296672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c4fd0 msgr2=0x559c987b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.739+0000 7fcf3d995740 1 --2- 172.21.15.19:0/1829296672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c4fd0 0x559c987b5480 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcf240588f0 tx=0x7fcf24058bf0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.739+0000 7fcf3d995740 1 -- 172.21.15.19:0/1829296672 shutdown_connections 2024-03-28T13:09:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.739+0000 7fcf3d995740 1 --2- 172.21.15.19:0/1829296672 >> [v2:172.21.15.19:6834/731659331,v1:172.21.15.19:6835/731659331] conn(0x559c987febd0 0x559c98800fc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.739+0000 7fcf3d995740 1 --2- 172.21.15.19:0/1829296672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987f2a20 0x559c987f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.739+0000 7fcf3d995740 1 --2- 172.21.15.19:0/1829296672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c53b0 0x559c987d9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.739+0000 7fcf3d995740 1 --2- 172.21.15.19:0/1829296672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c4fd0 0x559c987b5480 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.739+0000 7fcf3d995740 1 -- 172.21.15.19:0/1829296672 >> 172.21.15.19:0/1829296672 conn(0x559c987984b0 msgr2=0x559c98797690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.739+0000 7fcf20c00640 0 client.4632 ms_handle_reset on v2:172.21.15.19:6834/731659331 2024-03-28T13:09:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.739+0000 7fcf3d995740 1 -- 172.21.15.19:0/1829296672 shutdown_connections 2024-03-28T13:09:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.739+0000 7fcf3d995740 1 -- 172.21.15.19:0/1829296672 wait complete. 2024-03-28T13:09:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.739+0000 7fcf3d995740 20 client.4632 trim_cache size 0 max 16384 2024-03-28T13:09:59.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.759+0000 7fcf3d995740 1 Processor -- start 2024-03-28T13:09:59.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.759+0000 7fcf3d995740 1 -- start start 2024-03-28T13:09:59.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.760+0000 7fcf3d995740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987984b0 0x559c987c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.760+0000 7fcf3d995740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c4fd0 0x559c987bf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.760+0000 7fcf3d995740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c53b0 0x559c987c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.760+0000 7fcf3d995740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcf341ffc10 con 0x559c987c4fd0 2024-03-28T13:09:59.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.760+0000 7fcf3d995740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf340a07f0 con 0x559c987c53b0 2024-03-28T13:09:59.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.760+0000 7fcf3d995740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcf340a0e40 con 0x559c987984b0 2024-03-28T13:09:59.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.760+0000 7fcf33400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c4fd0 0x559c987bf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.760+0000 7fcf33400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c4fd0 0x559c987bf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41968/0 (socket says 172.21.15.19:41968) 2024-03-28T13:09:59.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.760+0000 7fcf33400640 1 -- 172.21.15.19:0/1986859335 learned_addr learned my addr 172.21.15.19:0/1986859335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:59.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.761+0000 7fcf33400640 1 -- 172.21.15.19:0/1986859335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987984b0 msgr2=0x559c987c10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:59.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.761+0000 7fcf33400640 1 --2- 172.21.15.19:0/1986859335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987984b0 0x559c987c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.761+0000 7fcf33400640 1 -- 172.21.15.19:0/1986859335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c53b0 msgr2=0x559c987c34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:59.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.761+0000 7fcf33400640 1 --2- 172.21.15.19:0/1986859335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c53b0 0x559c987c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.761+0000 7fcf33400640 1 -- 172.21.15.19:0/1986859335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c98799ec0 con 0x559c987c4fd0 2024-03-28T13:09:59.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.761+0000 7fcf33400640 1 --2- 172.21.15.19:0/1986859335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c4fd0 0x559c987bf4d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcf341f7920 tx=0x7fcf1c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1143735f018949d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:59.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.761+0000 7fcf23400640 1 -- 172.21.15.19:0/1986859335 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c01c040 con 0x559c987c4fd0 2024-03-28T13:09:59.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.761+0000 7fcf23400640 1 -- 172.21.15.19:0/1986859335 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf1c01d040 con 0x559c987c4fd0 2024-03-28T13:09:59.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.761+0000 7fcf23400640 1 -- 172.21.15.19:0/1986859335 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c00f040 con 0x559c987c4fd0 2024-03-28T13:09:59.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.762+0000 7fcf3d995740 1 -- 172.21.15.19:0/1986859335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c4fd0 msgr2=0x559c987bf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:59.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.762+0000 7fcf3d995740 1 --2- 172.21.15.19:0/1986859335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c4fd0 0x559c987bf4d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcf341f7920 tx=0x7fcf1c013000 comp rx=0 tx=0).stop 2024-03-28T13:09:59.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.762+0000 7fcf3d995740 1 -- 172.21.15.19:0/1986859335 shutdown_connections 2024-03-28T13:09:59.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.762+0000 7fcf3d995740 1 --2- 172.21.15.19:0/1986859335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c53b0 0x559c987c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.762+0000 7fcf3d995740 1 --2- 172.21.15.19:0/1986859335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c4fd0 0x559c987bf4d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.762+0000 7fcf3d995740 1 --2- 172.21.15.19:0/1986859335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987984b0 0x559c987c10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.762+0000 7fcf3d995740 1 -- 172.21.15.19:0/1986859335 >> 172.21.15.19:0/1986859335 conn(0x559c9879e440 msgr2=0x559c987cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:59.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.762+0000 7fcf3d995740 1 -- 172.21.15.19:0/1986859335 shutdown_connections 2024-03-28T13:09:59.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.762+0000 7fcf3d995740 1 -- 172.21.15.19:0/1986859335 wait complete. 2024-03-28T13:09:59.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.763+0000 7fcf3d995740 1 -- start start 2024-03-28T13:09:59.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.763+0000 7fcf3d995740 1 Processor -- start 2024-03-28T13:09:59.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.764+0000 7fcf3d995740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987984b0 0x559c98851510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.764+0000 7fcf3d995740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c4fd0 0x559c987c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.764+0000 7fcf3d995740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c53b0 0x559c987c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.764+0000 7fcf3d995740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559c987d9dc0 con 0x559c987c53b0 2024-03-28T13:09:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.764+0000 7fcf3d995740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559c987c1ef0 con 0x559c987c4fd0 2024-03-28T13:09:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.764+0000 7fcf3d995740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x559c987cb680 con 0x559c987984b0 2024-03-28T13:09:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.764+0000 7fcf3d995740 10 client.0 _ll_register_callbacks cb 0x7fcf240018b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:09:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.764+0000 7fcf38c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c53b0 0x559c987c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.764+0000 7fcf38c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c53b0 0x559c987c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41984/0 (socket says 172.21.15.19:41984) 2024-03-28T13:09:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.764+0000 7fcf38c00640 1 -- 172.21.15.19:0/3946916548 learned_addr learned my addr 172.21.15.19:0/3946916548 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:09:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.765+0000 7fcf33e00640 1 --2- 172.21.15.19:0/3946916548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987984b0 0x559c98851510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.765+0000 7fcf33400640 1 --2- 172.21.15.19:0/3946916548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c4fd0 0x559c987c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.765+0000 7fcf38c00640 1 -- 172.21.15.19:0/3946916548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987984b0 msgr2=0x559c98851510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.765+0000 7fcf38c00640 1 --2- 172.21.15.19:0/3946916548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987984b0 0x559c98851510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.765+0000 7fcf38c00640 1 -- 172.21.15.19:0/3946916548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c4fd0 msgr2=0x559c987c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:09:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.765+0000 7fcf38c00640 1 --2- 172.21.15.19:0/3946916548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c4fd0 0x559c987c8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.765+0000 7fcf38c00640 1 -- 172.21.15.19:0/3946916548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c98851f90 con 0x559c987c53b0 2024-03-28T13:09:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.765+0000 7fcf38c00640 1 --2- 172.21.15.19:0/3946916548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c53b0 0x559c987c9060 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c043000 tx=0x7fcf2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c71c297cfb299c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:09:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.765+0000 7fcf20c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:09:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.765+0000 7fcf20c00640 1 -- 172.21.15.19:0/3946916548 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c049070 con 0x559c987c53b0 2024-03-28T13:09:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.765+0000 7fcf20c00640 1 -- 172.21.15.19:0/3946916548 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcf2c031d70 con 0x559c987c53b0 2024-03-28T13:09:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.765+0000 7fcf3d995740 1 -- 172.21.15.19:0/3946916548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf1c01d040 con 0x559c987c53b0 2024-03-28T13:09:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.766+0000 7fcf20c00640 1 -- 172.21.15.19:0/3946916548 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fcf2c035040 con 0x559c987c53b0 2024-03-28T13:09:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.766+0000 7fcf3d995740 10 client.4638 fetch_fsmap learned FSMap version 8 2024-03-28T13:09:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.766+0000 7fcf3d995740 1 -- 172.21.15.19:0/3946916548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x559c988555a0 con 0x559c987c53b0 2024-03-28T13:09:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.766+0000 7fcf20c00640 1 -- 172.21.15.19:0/3946916548 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf1c01d040 con 0x559c987c53b0 2024-03-28T13:09:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.766+0000 7fcf20c00640 1 -- 172.21.15.19:0/3946916548 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fcf2c043030 con 0x559c987c53b0 2024-03-28T13:09:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.766+0000 7fcf3d995740 10 client.4638 fetch_fsmap finished waiting for FSMap version 8 2024-03-28T13:09:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.766+0000 7fcf3d995740 10 client.4638 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T13:09:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.766+0000 7fcf3d995740 20 client.4638 populate_metadata read hostname 'smithi019' 2024-03-28T13:09:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.766+0000 7fcf3d995740 1 --2- 172.21.15.19:0/3946916548 >> [v2:172.21.15.19:6834/731659331,v1:172.21.15.19:6835/731659331] conn(0x559c987f2a20 0x559c98854f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:09:59.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.766+0000 7fcf3d995740 4 client.4638 mds_command: new command op to 4254 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T13:09:59.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.766+0000 7fcf3d995740 1 -- 172.21.15.19:0/3946916548 --> [v2:172.21.15.19:6834/731659331,v1:172.21.15.19:6835/731659331] -- command(tid 0: {"prefix": "status"}) -- 0x559c98709820 con 0x559c987f2a20 2024-03-28T13:09:59.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.766+0000 7fcf33e00640 1 --2- 172.21.15.19:0/3946916548 >> [v2:172.21.15.19:6834/731659331,v1:172.21.15.19:6835/731659331] conn(0x559c987f2a20 0x559c98854f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:09:59.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.767+0000 7fcf33e00640 1 --2- 172.21.15.19:0/3946916548 >> [v2:172.21.15.19:6834/731659331,v1:172.21.15.19:6835/731659331] conn(0x559c987f2a20 0x559c98854f40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48c5c3023c28ff9 server_cookie=ffec489f637a7541 in_seq=0 out_seq=0 2024-03-28T13:09:59.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.767+0000 7fcf20c00640 10 client.4638 ms_handle_connect on v2:172.21.15.19:6834/731659331 2024-03-28T13:09:59.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.767+0000 7fcf20c00640 1 -- 172.21.15.19:0/3946916548 <== mds.0 v2:172.21.15.19:6834/731659331 1 ==== command_reply(tid 0: 0 ) ==== 8+0+318 (crc 0 0 0) 0x559c98709820 con 0x559c987f2a20 2024-03-28T13:09:59.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.767+0000 7fcf20c00640 10 client.4638 handle_command_reply: tid=0 2024-03-28T13:09:59.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.767+0000 7fcf3d995740 1 client.4638 shutdown 2024-03-28T13:09:59.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.767+0000 7fcf3d995740 1 -- 172.21.15.19:0/3946916548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c53b0 msgr2=0x559c987c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:59.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.767+0000 7fcf3d995740 1 --2- 172.21.15.19:0/3946916548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c53b0 0x559c987c9060 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c043000 tx=0x7fcf2c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.767+0000 7fcf3d995740 1 -- 172.21.15.19:0/3946916548 shutdown_connections 2024-03-28T13:09:59.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.767+0000 7fcf3d995740 1 --2- 172.21.15.19:0/3946916548 >> [v2:172.21.15.19:6834/731659331,v1:172.21.15.19:6835/731659331] conn(0x559c987f2a20 0x559c98854f40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.767+0000 7fcf3d995740 1 --2- 172.21.15.19:0/3946916548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c987c53b0 0x559c987c9060 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.767+0000 7fcf3d995740 1 --2- 172.21.15.19:0/3946916548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c987c4fd0 0x559c987c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.767+0000 7fcf3d995740 1 --2- 172.21.15.19:0/3946916548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c987984b0 0x559c98851510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.767+0000 7fcf20c00640 0 client.4638 ms_handle_reset on v2:172.21.15.19:6834/731659331 2024-03-28T13:09:59.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.767+0000 7fcf3d995740 1 -- 172.21.15.19:0/3946916548 >> 172.21.15.19:0/3946916548 conn(0x559c9879e440 msgr2=0x559c987cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:59.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.768+0000 7fcf3d995740 1 -- 172.21.15.19:0/3946916548 shutdown_connections 2024-03-28T13:09:59.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.768+0000 7fcf3d995740 1 -- 172.21.15.19:0/3946916548 wait complete. 2024-03-28T13:09:59.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.768+0000 7fcf3d995740 20 client.4638 trim_cache size 0 max 16384 2024-03-28T13:09:59.770 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:09:59.770 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "e6b569d9-f2ce-49dc-bbe7-c0ab33080203", 2024-03-28T13:09:59.770 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-03-28T13:09:59.770 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 4254, 2024-03-28T13:09:59.770 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-03-28T13:09:59.770 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-03-28T13:09:59.770 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-03-28T13:09:59.771 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 18.089626359, 2024-03-28T13:09:59.771 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 8, 2024-03-28T13:09:59.771 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 23, 2024-03-28T13:09:59.771 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 23, 2024-03-28T13:09:59.771 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 23.642838284 2024-03-28T13:09:59.771 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:09:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.769+0000 7fcf3ac00640 1 -- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcf1006c530 msgr2=0x7fcf1006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.769+0000 7fcf3ac00640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcf1006c530 0x7fcf1006e9f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcf340cafa0 tx=0x7fcf1c008040 comp rx=0 tx=0).stop 2024-03-28T13:09:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.769+0000 7fcf3ac00640 1 -- 172.21.15.19:0/2177105227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340d0cb0 msgr2=0x7fcf3414f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:09:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.769+0000 7fcf3ac00640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340d0cb0 0x7fcf3414f810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400c7f0 tx=0x7fcf2400ccb0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.769+0000 7fcf3ac00640 1 -- 172.21.15.19:0/2177105227 shutdown_connections 2024-03-28T13:09:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.769+0000 7fcf3ac00640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcf1006c530 0x7fcf1006e9f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.769+0000 7fcf3ac00640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3415e6d0 0x7fcf340c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.769+0000 7fcf3ac00640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf340d4590 0x7fcf3409e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.769+0000 7fcf3ac00640 1 --2- 172.21.15.19:0/2177105227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf340d0cb0 0x7fcf3414f810 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:09:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.769+0000 7fcf3ac00640 1 -- 172.21.15.19:0/2177105227 >> 172.21.15.19:0/2177105227 conn(0x7fcf3409d7b0 msgr2=0x7fcf3406d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:09:59.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.769+0000 7fcf3ac00640 1 -- 172.21.15.19:0/2177105227 shutdown_connections 2024-03-28T13:09:59.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:09:59.769+0000 7fcf3ac00640 1 -- 172.21.15.19:0/2177105227 wait complete. 2024-03-28T13:09:59.785 INFO:teuthology.run_tasks:Running task sequential... 2024-03-28T13:09:59.796 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-28T13:09:59.797 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-28T13:09:59.797 INFO:teuthology.task.print:Enabling mgr modules 2024-03-28T13:09:59.797 INFO:teuthology.run_tasks:Running task kclient... 2024-03-28T13:09:59.823 INFO:tasks.kclient:Mounting kernel clients... 2024-03-28T13:09:59.823 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-28T13:09:59.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:09:59.824 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:09:59.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:09:59.880 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-03-28T13:09:59.988 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-03-28T13:09:59.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T13:09:59.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:09:59.993 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:10:00.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:00.045 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-03-28T13:10:00.140 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-03-28T13:10:00.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T13:10:00.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:00.146 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-03-28T13:10:00.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:00.202 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-03-28T13:10:00.299 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2024-03-28T13:10:00.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T13:10:00.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:00.303 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-03-28T13:10:00.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:00.351 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-03-28T13:10:00.431 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2024-03-28T13:10:00.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T13:10:00.434 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-03-28T13:10:00.434 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-28T13:10:00.434 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:10:00.434 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:10:00.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.647+0000 7f11f5000640 1 Processor -- start 2024-03-28T13:10:00.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.647+0000 7f11f5000640 1 -- start start 2024-03-28T13:10:00.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.649+0000 7f11f5000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11f015e420 0x7f11f00b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:00.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.649+0000 7f11f5000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f11f00d1a90 0x7f11f00d3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:00.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.649+0000 7f11f5000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11f00d2fd0 con 0x7f11f00b3460 2024-03-28T13:10:00.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.649+0000 7f11f5000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f11f00cfa70 con 0x7f11f00d1a90 2024-03-28T13:10:00.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.649+0000 7f11f5000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f11f00cf790 con 0x7f11f015e420 2024-03-28T13:10:00.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.649+0000 7f11efe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f11f00d1a90 0x7f11f00d3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:00.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.649+0000 7f11eea00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f11f00b3460 0x7f11f00d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49116/0 (socket says 172.21.15.19:49116) 2024-03-28T13:10:00.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.649+0000 7f11eea00640 1 -- 172.21.15.19:0/553237757 learned_addr learned my addr 172.21.15.19:0/553237757 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:00.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.649+0000 7f11ef400640 1 --2- 172.21.15.19:0/553237757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11f015e420 0x7f11f00b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:00.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.650+0000 7f11efe00640 1 -- 172.21.15.19:0/553237757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11f015e420 msgr2=0x7f11f00b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:00.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.650+0000 7f11efe00640 1 --2- 172.21.15.19:0/553237757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11f015e420 0x7f11f00b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:00.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.650+0000 7f11efe00640 1 -- 172.21.15.19:0/553237757 >> v1:172.21.15.19:6789/0 conn(0x7f11f00b3460 legacy=0x7f11f00d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:00.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.650+0000 7f11efe00640 1 -- 172.21.15.19:0/553237757 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f00d3270 con 0x7f11f00d1a90 2024-03-28T13:10:00.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.651+0000 7f11efe00640 1 --2- 172.21.15.19:0/553237757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11f00d1a90 0x7f11f00d3a70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f11e80097a0 tx=0x7f11e802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7528d919c4e8cff0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:00.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.651+0000 7f11ee000640 1 -- 172.21.15.19:0/553237757 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8035020 con 0x7f11f00d1a90 2024-03-28T13:10:00.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.651+0000 7f11ee000640 1 -- 172.21.15.19:0/553237757 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11e8031810 con 0x7f11f00d1a90 2024-03-28T13:10:00.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.651+0000 7f11ee000640 1 -- 172.21.15.19:0/553237757 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8031be0 con 0x7f11f00d1a90 2024-03-28T13:10:00.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.652+0000 7f11f5000640 1 -- 172.21.15.19:0/553237757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11f00d1a90 msgr2=0x7f11f00d3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:00.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.652+0000 7f11f5000640 1 --2- 172.21.15.19:0/553237757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11f00d1a90 0x7f11f00d3a70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f11e80097a0 tx=0x7f11e802b8b0 comp rx=0 tx=0).stop 2024-03-28T13:10:00.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.652+0000 7f11f5000640 1 -- 172.21.15.19:0/553237757 shutdown_connections 2024-03-28T13:10:00.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.652+0000 7f11f5000640 1 --2- 172.21.15.19:0/553237757 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11f00d1a90 0x7f11f00d3a70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:00.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.652+0000 7f11f5000640 1 --2- 172.21.15.19:0/553237757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11f015e420 0x7f11f00b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:00.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.652+0000 7f11f5000640 1 -- 172.21.15.19:0/553237757 >> 172.21.15.19:0/553237757 conn(0x7f11f0067670 msgr2=0x7f11f0067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:00.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.652+0000 7f11f5000640 1 -- 172.21.15.19:0/553237757 shutdown_connections 2024-03-28T13:10:00.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.652+0000 7f11f5000640 1 -- 172.21.15.19:0/553237757 wait complete. 2024-03-28T13:10:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.654+0000 7f11f5000640 1 Processor -- start 2024-03-28T13:10:00.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.654+0000 7f11f5000640 1 -- start start 2024-03-28T13:10:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.655+0000 7f11f5000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11f00b3460 0x7f11f0158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.655+0000 7f11f5000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11f00d1a90 0x7f11f00c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.655+0000 7f11f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11f015e420 0x7f11f00cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.655+0000 7f11f5000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11f00cf420 con 0x7f11f015e420 2024-03-28T13:10:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.655+0000 7f11f5000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f11f015e800 con 0x7f11f00b3460 2024-03-28T13:10:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.655+0000 7f11f5000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f11f00d3fb0 con 0x7f11f00d1a90 2024-03-28T13:10:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.655+0000 7f11efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11f015e420 0x7f11f00cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.655+0000 7f11efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11f015e420 0x7f11f00cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42010/0 (socket says 172.21.15.19:42010) 2024-03-28T13:10:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.655+0000 7f11ef400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11f00b3460 0x7f11f0158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.655+0000 7f11efe00640 1 -- 172.21.15.19:0/2254789100 learned_addr learned my addr 172.21.15.19:0/2254789100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:00.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.655+0000 7f11eea00640 1 --2- 172.21.15.19:0/2254789100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11f00d1a90 0x7f11f00c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.656+0000 7f11efe00640 1 -- 172.21.15.19:0/2254789100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11f00d1a90 msgr2=0x7f11f00c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.656+0000 7f11efe00640 1 --2- 172.21.15.19:0/2254789100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11f00d1a90 0x7f11f00c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.656+0000 7f11efe00640 1 -- 172.21.15.19:0/2254789100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11f00b3460 msgr2=0x7f11f0158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.656+0000 7f11efe00640 1 --2- 172.21.15.19:0/2254789100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11f00b3460 0x7f11f0158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.656+0000 7f11efe00640 1 -- 172.21.15.19:0/2254789100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f015ff60 con 0x7f11f015e420 2024-03-28T13:10:00.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.656+0000 7f11efe00640 1 --2- 172.21.15.19:0/2254789100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11f015e420 0x7f11f00cb4b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f11e802bec0 tx=0x7f11e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.656+0000 7f11e7400640 1 -- 172.21.15.19:0/2254789100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8035040 con 0x7f11f015e420 2024-03-28T13:10:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.656+0000 7f11e7400640 1 -- 172.21.15.19:0/2254789100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11e803ac80 con 0x7f11f015e420 2024-03-28T13:10:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.656+0000 7f11e7400640 1 -- 172.21.15.19:0/2254789100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8046480 con 0x7f11f015e420 2024-03-28T13:10:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.656+0000 7f11f5000640 1 -- 172.21.15.19:0/2254789100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f0160240 con 0x7f11f015e420 2024-03-28T13:10:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.656+0000 7f11f5000640 1 -- 172.21.15.19:0/2254789100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f0160800 con 0x7f11f015e420 2024-03-28T13:10:00.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.657+0000 7f11f5000640 1 -- 172.21.15.19:0/2254789100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11f0075940 con 0x7f11f015e420 2024-03-28T13:10:00.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.659+0000 7f11e7400640 1 -- 172.21.15.19:0/2254789100 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f11e8038020 con 0x7f11f015e420 2024-03-28T13:10:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.660+0000 7f11e7400640 1 --2- 172.21.15.19:0/2254789100 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f11cc06c530 0x7f11cc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:00.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.660+0000 7f11e7400640 1 -- 172.21.15.19:0/2254789100 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f11e80ad6f0 con 0x7f11f015e420 2024-03-28T13:10:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.661+0000 7f11ef400640 1 --2- 172.21.15.19:0/2254789100 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f11cc06c530 0x7f11cc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:00.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.661+0000 7f11ef400640 1 --2- 172.21.15.19:0/2254789100 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f11cc06c530 0x7f11cc06e9f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009aa0 tx=0x7f11e0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:00.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.665+0000 7f11e7400640 1 -- 172.21.15.19:0/2254789100 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f11e807b6b0 con 0x7f11f015e420 2024-03-28T13:10:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.912+0000 7f11f5000640 1 -- 172.21.15.19:0/2254789100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f11f00d2340 con 0x7f11f015e420 2024-03-28T13:10:00.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.912+0000 7f11e7400640 1 -- 172.21.15.19:0/2254789100 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f11e807f9b0 con 0x7f11f015e420 2024-03-28T13:10:00.912 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:10:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.915+0000 7f11f5000640 1 -- 172.21.15.19:0/2254789100 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f11cc06c530 msgr2=0x7f11cc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.915+0000 7f11f5000640 1 --2- 172.21.15.19:0/2254789100 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f11cc06c530 0x7f11cc06e9f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f11e0009aa0 tx=0x7f11e0009190 comp rx=0 tx=0).stop 2024-03-28T13:10:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.915+0000 7f11f5000640 1 -- 172.21.15.19:0/2254789100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11f015e420 msgr2=0x7f11f00cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.915+0000 7f11f5000640 1 --2- 172.21.15.19:0/2254789100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11f015e420 0x7f11f00cb4b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f11e802bec0 tx=0x7f11e80046c0 comp rx=0 tx=0).stop 2024-03-28T13:10:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.915+0000 7f11f5000640 1 -- 172.21.15.19:0/2254789100 shutdown_connections 2024-03-28T13:10:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.915+0000 7f11f5000640 1 --2- 172.21.15.19:0/2254789100 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f11cc06c530 0x7f11cc06e9f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.915+0000 7f11f5000640 1 --2- 172.21.15.19:0/2254789100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11f015e420 0x7f11f00cb4b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.915+0000 7f11f5000640 1 --2- 172.21.15.19:0/2254789100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11f00d1a90 0x7f11f00c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.915+0000 7f11f5000640 1 --2- 172.21.15.19:0/2254789100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11f00b3460 0x7f11f0158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.915+0000 7f11f5000640 1 -- 172.21.15.19:0/2254789100 >> 172.21.15.19:0/2254789100 conn(0x7f11f0067670 msgr2=0x7f11f01624d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.915+0000 7f11f5000640 1 -- 172.21.15.19:0/2254789100 shutdown_connections 2024-03-28T13:10:00.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:00.916+0000 7f11f5000640 1 -- 172.21.15.19:0/2254789100 wait complete. 2024-03-28T13:10:00.935 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-28T13:10:00.935 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:10:00.935 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:10:00.935 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:10:00.935 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:10:00.935 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-28T13:10:00.935 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-28T13:10:00.935 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:10:00.936 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:10:00.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:00.936 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:10:00.984 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:10:00.984 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:10:00.984 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:10:00.984 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:00.984 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:10:00.984 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:00.984 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:10:00.984 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:00.984 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:10:00.985 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:10:00.985 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:00.985 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:10:00.985 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:10:00.985 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:00.985 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:10:00.985 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:10:00.985 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42554sec preferred_lft 42554sec 2024-03-28T13:10:00.985 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:10:00.985 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:00.985 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:10:00.985 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:00.985 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:10:00.987 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-28T13:10:00.987 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:00.987 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:10:00.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-03-28T13:10:00.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-03-28T13:10:00.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-03-28T13:10:00.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-28T13:10:00.987 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:10:00.999 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:10:01.038 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:10:01.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:01.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:01.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:01.518 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:10:01.598 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:10:01.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:01.603 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:10:01.652 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:10:01.652 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:10:01.652 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:10:01.652 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:10:01.652 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:10:01.653 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:01.653 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:10:01.653 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:10:01.654 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:10:01.654 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:10:01.654 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:10:01.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:02.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:02.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:02.420 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:10:02.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:02.468 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-03-28T13:10:02.559 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:02.559 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:10:02.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:10:02.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-28T13:10:02.560 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:10:02.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:02.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:02.805 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-28T13:10:02.805 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:02.805 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:10:02.805 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-28T13:10:02.805 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-28T13:10:02.805 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-28T13:10:02.805 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-28T13:10:02.805 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-28T13:10:02.805 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:10:02.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:03.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:03.131 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:03.131 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:10:03.131 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:10:03.131 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-03-28T13:10:03.131 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:10:03.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:03.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:03.339 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:10:03.339 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:10:03.392 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:10:03.392 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:10:03.393 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:10:03.442 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:10:03.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:03.442 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:10:03.617 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:10:03.617 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:10:03.617 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:10:04.265 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:10:04.265 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:10:04.265 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:10:04.265 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:10:04.265 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:10:04.298 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:10:04.298 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:10:04.298 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:10:04.299 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:10:04.299 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:10:04.299 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:10:04.299 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:10:04.299 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:10:04.301 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:10:04.301 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:10:04.302 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:10:04.302 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:10:04.302 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:10:04.306 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:10:04.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:04.307 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:10:04.404 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:10:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:10:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:04.536 INFO:teuthology.orchestra.run.smithi019.stdout:client4463 2024-03-28T13:10:04.537 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:10:04.538 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:04.595 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:10:04.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:10:04.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:04.771 INFO:teuthology.orchestra.run.smithi019.stdout:client4463 2024-03-28T13:10:04.771 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:10:04.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:10:04.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:04.891 INFO:teuthology.orchestra.run.smithi019.stdout:client4463 2024-03-28T13:10:04.891 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e6b569d9-f2ce-49dc-bbe7-c0ab33080203.client4463/status') 2024-03-28T13:10:04.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:05.041 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.4463 (3)172.21.15.19:0/2698708306 2024-03-28T13:10:05.041 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-03-28T13:10:05.041 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-03-28T13:10:05.041 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-03-28T13:10:05.042 INFO:teuthology.orchestra.run.smithi019.stderr:70 bytes copied, 8.0731e-05 s, 867 kB/s 2024-03-28T13:10:05.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:05.057 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-03-28T13:10:05.057 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-28T13:10:05.057 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-28T13:10:05.057 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:10:05.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.279+0000 7f379e400640 1 Processor -- start 2024-03-28T13:10:05.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.280+0000 7f379e400640 1 -- start start 2024-03-28T13:10:05.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.281+0000 7f379e400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f379806fa90 0x7f379815d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:05.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.281+0000 7f379e400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3798145f50 con 0x7f379815fed0 2024-03-28T13:10:05.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.281+0000 7f379e400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37981603d0 con 0x7f379806b570 2024-03-28T13:10:05.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.281+0000 7f379e400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f379815d920 con 0x7f379806fa90 2024-03-28T13:10:05.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.282+0000 7f378fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f379815fed0 0x7f37980d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49118/0 (socket says 172.21.15.19:49118) 2024-03-28T13:10:05.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.282+0000 7f378fe00640 1 -- 172.21.15.19:0/297248819 learned_addr learned my addr 172.21.15.19:0/297248819 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:05.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.282+0000 7f3797e00640 1 -- 172.21.15.19:0/297248819 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1637108484 0 0) 0x7f3798145f50 con 0x7f379815fed0 2024-03-28T13:10:05.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.282+0000 7f3797e00640 1 -- 172.21.15.19:0/297248819 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f377c003520 con 0x7f379815fed0 2024-03-28T13:10:05.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.282+0000 7f379da00640 1 --2- 172.21.15.19:0/297248819 >> v2:172.21.15.188:3300/0 conn(0x7f379806fa90 0x7f379815d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.283+0000 7f3797e00640 1 -- 172.21.15.19:0/297248819 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 527231042 0 0) 0x7f377c003520 con 0x7f379815fed0 2024-03-28T13:10:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.283+0000 7f3797e00640 1 -- 172.21.15.19:0/297248819 >> v2:172.21.15.188:3300/0 conn(0x7f379806fa90 msgr2=0x7f379815d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.283+0000 7f3797e00640 1 --2- 172.21.15.19:0/297248819 >> v2:172.21.15.188:3300/0 conn(0x7f379806fa90 0x7f379815d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.283+0000 7f3797e00640 1 -- 172.21.15.19:0/297248819 >> v1:172.21.15.188:6789/0 conn(0x7f379806b570 legacy=0x7f379806b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.283+0000 7f3797e00640 1 -- 172.21.15.19:0/297248819 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37981606b0 con 0x7f379815fed0 2024-03-28T13:10:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.283+0000 7f3797e00640 1 -- 172.21.15.19:0/297248819 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f3780001670 con 0x7f379815fed0 2024-03-28T13:10:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.283+0000 7f3797e00640 1 -- 172.21.15.19:0/297248819 >> v1:172.21.15.19:6789/0 conn(0x7f379815fed0 legacy=0x7f37980d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.283+0000 7f3797e00640 1 --2- 172.21.15.19:0/297248819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003a00 0x7f377c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.283+0000 7f3797e00640 1 --2- 172.21.15.19:0/297248819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c004800 0x7f377c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.283+0000 7f3797e00640 1 --2- 172.21.15.19:0/297248819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f377c0052e0 0x7f377c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.283+0000 7f3797e00640 1 -- 172.21.15.19:0/297248819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f379815d920 con 0x7f377c003a00 2024-03-28T13:10:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.283+0000 7f3797e00640 1 -- 172.21.15.19:0/297248819 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f377c00c200 con 0x7f377c0052e0 2024-03-28T13:10:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.283+0000 7f3797e00640 1 -- 172.21.15.19:0/297248819 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f377c00c400 con 0x7f377c004800 2024-03-28T13:10:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.283+0000 7f379d000640 1 --2- 172.21.15.19:0/297248819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003a00 0x7f377c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.284+0000 7f378fe00640 1 --2- 172.21.15.19:0/297248819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c004800 0x7f377c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.284+0000 7f379da00640 1 --2- 172.21.15.19:0/297248819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f377c0052e0 0x7f377c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.284+0000 7f379d000640 1 -- 172.21.15.19:0/297248819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c004800 msgr2=0x7f377c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.284+0000 7f379d000640 1 --2- 172.21.15.19:0/297248819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c004800 0x7f377c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.284+0000 7f379d000640 1 -- 172.21.15.19:0/297248819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f377c0052e0 msgr2=0x7f377c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.284+0000 7f379d000640 1 --2- 172.21.15.19:0/297248819 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f377c0052e0 0x7f377c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.284+0000 7f379d000640 1 -- 172.21.15.19:0/297248819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c003520 con 0x7f377c003a00 2024-03-28T13:10:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.284+0000 7f379da00640 1 -- 172.21.15.19:0/297248819 reap_dead start 2024-03-28T13:10:05.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.285+0000 7f379d000640 1 --2- 172.21.15.19:0/297248819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003a00 0x7f377c003e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3788001750 tx=0x7f377c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a836eb3fba443897 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:05.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.285+0000 7f3797e00640 1 -- 172.21.15.19:0/297248819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788001370 con 0x7f377c003a00 2024-03-28T13:10:05.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.285+0000 7f3797e00640 1 -- 172.21.15.19:0/297248819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37880052d0 con 0x7f377c003a00 2024-03-28T13:10:05.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.285+0000 7f3797e00640 1 -- 172.21.15.19:0/297248819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788003830 con 0x7f377c003a00 2024-03-28T13:10:05.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.286+0000 7f379e400640 1 -- 172.21.15.19:0/297248819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003a00 msgr2=0x7f377c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:05.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.286+0000 7f379e400640 1 --2- 172.21.15.19:0/297248819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003a00 0x7f377c003e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3788001750 tx=0x7f377c005830 comp rx=0 tx=0).stop 2024-03-28T13:10:05.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.286+0000 7f379e400640 1 -- 172.21.15.19:0/297248819 shutdown_connections 2024-03-28T13:10:05.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.286+0000 7f379e400640 1 --2- 172.21.15.19:0/297248819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003a00 0x7f377c003e20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:05.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.286+0000 7f379e400640 1 -- 172.21.15.19:0/297248819 >> 172.21.15.19:0/297248819 conn(0x7f37980d1090 msgr2=0x7f37980d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:05.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.286+0000 7f379e400640 1 -- 172.21.15.19:0/297248819 shutdown_connections 2024-03-28T13:10:05.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.286+0000 7f379e400640 1 -- 172.21.15.19:0/297248819 wait complete. 2024-03-28T13:10:05.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.288+0000 7f379e400640 1 Processor -- start 2024-03-28T13:10:05.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.288+0000 7f379e400640 1 -- start start 2024-03-28T13:10:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.289+0000 7f379e400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003a00 0x7f37980cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.289+0000 7f379e400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37980ca480 0x7f37980ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.289+0000 7f379e400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37980b4b60 0x7f37980b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.289+0000 7f379e400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37980cfc20 con 0x7f377c003a00 2024-03-28T13:10:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.289+0000 7f379e400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3798160c70 con 0x7f37980b4b60 2024-03-28T13:10:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.289+0000 7f379e400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3798145340 con 0x7f37980ca480 2024-03-28T13:10:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.290+0000 7f379d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003a00 0x7f37980cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.290+0000 7f379d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003a00 0x7f37980cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42020/0 (socket says 172.21.15.19:42020) 2024-03-28T13:10:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.290+0000 7f379d000640 1 -- 172.21.15.19:0/255456469 learned_addr learned my addr 172.21.15.19:0/255456469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.290+0000 7f378fe00640 1 --2- 172.21.15.19:0/255456469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37980ca480 0x7f37980ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.290+0000 7f379d000640 1 -- 172.21.15.19:0/255456469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37980ca480 msgr2=0x7f37980ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.290+0000 7f379d000640 1 --2- 172.21.15.19:0/255456469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37980ca480 0x7f37980ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.290+0000 7f379d000640 1 -- 172.21.15.19:0/255456469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37980b4b60 msgr2=0x7f37980b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.290+0000 7f379d000640 1 --2- 172.21.15.19:0/255456469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37980b4b60 0x7f37980b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.290+0000 7f379d000640 1 -- 172.21.15.19:0/255456469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3798130940 con 0x7f377c003a00 2024-03-28T13:10:05.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.291+0000 7f379d000640 1 --2- 172.21.15.19:0/255456469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003a00 0x7f37980cba50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3788005040 tx=0x7f3788001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:05.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.291+0000 7f3795600640 1 -- 172.21.15.19:0/255456469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788041070 con 0x7f377c003a00 2024-03-28T13:10:05.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.291+0000 7f3795600640 1 -- 172.21.15.19:0/255456469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3788005480 con 0x7f377c003a00 2024-03-28T13:10:05.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.291+0000 7f3795600640 1 -- 172.21.15.19:0/255456469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788007a30 con 0x7f377c003a00 2024-03-28T13:10:05.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.291+0000 7f379e400640 1 -- 172.21.15.19:0/255456469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37980caff0 con 0x7f377c003a00 2024-03-28T13:10:05.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.291+0000 7f379e400640 1 -- 172.21.15.19:0/255456469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37980b4360 con 0x7f377c003a00 2024-03-28T13:10:05.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.292+0000 7f379e400640 1 -- 172.21.15.19:0/255456469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3798075940 con 0x7f377c003a00 2024-03-28T13:10:05.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.293+0000 7f3795600640 1 -- 172.21.15.19:0/255456469 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3788044070 con 0x7f377c003a00 2024-03-28T13:10:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.294+0000 7f3795600640 1 --2- 172.21.15.19:0/255456469 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f377406c530 0x7f377406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:05.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.294+0000 7f3795600640 1 -- 172.21.15.19:0/255456469 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f37880ad5c0 con 0x7f377c003a00 2024-03-28T13:10:05.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.295+0000 7f378fe00640 1 --2- 172.21.15.19:0/255456469 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f377406c530 0x7f377406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:05.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.296+0000 7f378fe00640 1 --2- 172.21.15.19:0/255456469 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f377406c530 0x7f377406e9f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3780001050 tx=0x7f3780009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:05.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.299+0000 7f3795600640 1 -- 172.21.15.19:0/255456469 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f378807bbd0 con 0x7f377c003a00 2024-03-28T13:10:05.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.542+0000 7f379e400640 1 -- 172.21.15.19:0/255456469 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f37980b1360 con 0x7f377c003a00 2024-03-28T13:10:05.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.543+0000 7f3795600640 1 -- 172.21.15.19:0/255456469 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f378807b4d0 con 0x7f377c003a00 2024-03-28T13:10:05.542 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:10:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.546+0000 7f379e400640 1 -- 172.21.15.19:0/255456469 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f377406c530 msgr2=0x7f377406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.546+0000 7f379e400640 1 --2- 172.21.15.19:0/255456469 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f377406c530 0x7f377406e9f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3780001050 tx=0x7f3780009040 comp rx=0 tx=0).stop 2024-03-28T13:10:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.546+0000 7f379e400640 1 -- 172.21.15.19:0/255456469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003a00 msgr2=0x7f37980cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:05.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.546+0000 7f379e400640 1 --2- 172.21.15.19:0/255456469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003a00 0x7f37980cba50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3788005040 tx=0x7f3788001120 comp rx=0 tx=0).stop 2024-03-28T13:10:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.546+0000 7f379e400640 1 -- 172.21.15.19:0/255456469 shutdown_connections 2024-03-28T13:10:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.546+0000 7f379e400640 1 --2- 172.21.15.19:0/255456469 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f377406c530 0x7f377406e9f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.546+0000 7f379e400640 1 --2- 172.21.15.19:0/255456469 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37980b4b60 0x7f37980b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.546+0000 7f379e400640 1 --2- 172.21.15.19:0/255456469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37980ca480 0x7f37980ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.546+0000 7f379e400640 1 --2- 172.21.15.19:0/255456469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003a00 0x7f37980cba50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.546+0000 7f379e400640 1 -- 172.21.15.19:0/255456469 >> 172.21.15.19:0/255456469 conn(0x7f37980d1090 msgr2=0x7f37981459e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.546+0000 7f379e400640 1 -- 172.21.15.19:0/255456469 shutdown_connections 2024-03-28T13:10:05.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05.546+0000 7f379e400640 1 -- 172.21.15.19:0/255456469 wait complete. 2024-03-28T13:10:05.564 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-28T13:10:05.564 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:10:05.564 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:10:05.565 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-28T13:10:05.565 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-28T13:10:05.565 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-28T13:10:05.565 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-28T13:10:05.565 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:10:05.565 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-28T13:10:05.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:05.565 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:10:05.618 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:10:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:10:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:10:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:10:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:05.619 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:10:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:10:05.619 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:10:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:10:05.619 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:10:05.619 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:10:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:10:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42549sec preferred_lft 42549sec 2024-03-28T13:10:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:10:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:05.620 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:10:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:10:05.620 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:10:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:10:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:10:05.620 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:05.620 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:10:05.621 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:10:05.621 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:10:05.621 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:05.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:05.621 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:10:05.707 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:10:05.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:05.711 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:10:05.756 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:10:05.756 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:10:05.756 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:10:05.756 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:10:05.756 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:10:05.756 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:05.756 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:10:05.757 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:10:05.757 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:10:05.757 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:10:05.757 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:10:05.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:06.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:06.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:06.058 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:10:06.107 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:10:06.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:06.108 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-03-28T13:10:06.157 INFO:teuthology.orchestra.run.smithi019.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-28T13:10:06.158 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:06.158 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:10:06.158 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:10:06.158 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-28T13:10:06.158 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:10:06.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:06.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:06.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:06.386 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-28T13:10:06.481 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:10:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:10:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:10:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:10:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:06.481 INFO:teuthology.orchestra.run.smithi019.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:10:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether d2:31:94:d7:27:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-28T13:10:06.481 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-28T13:10:06.482 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:06.482 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::d031:94ff:fed7:27af/64 scope link 2024-03-28T13:10:06.482 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:06.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:06.489 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-28T13:10:06.580 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:10:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:10:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:10:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:10:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:06.581 INFO:teuthology.orchestra.run.smithi019.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:10:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether d2:31:94:d7:27:af brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-28T13:10:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-28T13:10:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::d031:94ff:fed7:27af/64 scope link 2024-03-28T13:10:06.581 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:06.584 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-28T13:10:06.585 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:06.585 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:10:06.585 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-28T13:10:06.585 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-28T13:10:06.585 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-28T13:10:06.585 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-28T13:10:06.585 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-28T13:10:06.585 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:10:06.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:06.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:06.937 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:06.938 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:10:06.938 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.1 up 2024-03-28T13:10:06.938 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.1 master ceph-brx 2024-03-28T13:10:06.938 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:10:07.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:07.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:07.184 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:10:07.184 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-28T13:10:07.234 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-28T13:10:07.235 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:10:07.235 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-28T13:10:07.281 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:10:07.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:07.281 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-28T13:10:07.461 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-28T13:10:07.461 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-03-28T13:10:07.462 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:10:07.483 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-28T13:10:07.483 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-03-28T13:10:07.483 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:10:07.483 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:10:07.483 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:10:07.512 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-03-28T13:10:07.512 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-03-28T13:10:07.512 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:10:07.512 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:10:07.512 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:10:07.512 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:10:07.512 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:10:07.512 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:10:07.516 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-03-28T13:10:07.516 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:10:07.516 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:10:07.516 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:10:07.516 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:10:07.520 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:10:07.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:07.521 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-28T13:10:07.612 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:10:07.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:10:07.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:07.732 INFO:teuthology.orchestra.run.smithi019.stdout:client4469 2024-03-28T13:10:07.733 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:10:07.733 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:07.784 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:10:07.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:10:07.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:07.903 INFO:teuthology.orchestra.run.smithi019.stdout:client4469 2024-03-28T13:10:07.903 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:10:08.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:10:08.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:08.026 INFO:teuthology.orchestra.run.smithi019.stdout:client4469 2024-03-28T13:10:08.027 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e6b569d9-f2ce-49dc-bbe7-c0ab33080203.client4469/status') 2024-03-28T13:10:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:08.179 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.4469 (3)172.21.15.19:0/1050419032 2024-03-28T13:10:08.179 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-03-28T13:10:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-03-28T13:10:08.180 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-03-28T13:10:08.180 INFO:teuthology.orchestra.run.smithi019.stderr:70 bytes copied, 0.000102277 s, 684 kB/s 2024-03-28T13:10:08.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:08.192 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-03-28T13:10:08.192 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-28T13:10:08.192 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-28T13:10:08.193 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:10:08.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.396+0000 7f7212a00640 1 Processor -- start 2024-03-28T13:10:08.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.397+0000 7f7212a00640 1 -- start start 2024-03-28T13:10:08.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.398+0000 7f7212a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f720c15e7b0 0x7f720c0b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:08.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.398+0000 7f7212a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f720c0d1d70 0x7f720c0c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:08.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.398+0000 7f7212a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f720c0d01a0 con 0x7f720c0c7640 2024-03-28T13:10:08.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.398+0000 7f7212a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f720c0d1240 con 0x7f720c15e7b0 2024-03-28T13:10:08.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.398+0000 7f7212a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f720c0d0480 con 0x7f720c0d1d70 2024-03-28T13:10:08.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.399+0000 7f720be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f720c15e7b0 0x7f720c0b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:08.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.399+0000 7f720be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f720c15e7b0 0x7f720c0b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53880/0 (socket says 172.21.15.19:53880) 2024-03-28T13:10:08.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.399+0000 7f720be00640 1 -- 172.21.15.19:0/1571092312 learned_addr learned my addr 172.21.15.19:0/1571092312 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:08.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.399+0000 7f720b400640 1 --2- 172.21.15.19:0/1571092312 >> v2:172.21.15.188:3300/0 conn(0x7f720c0d1d70 0x7f720c0c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:08.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.399+0000 7f720be00640 1 -- 172.21.15.19:0/1571092312 >> v2:172.21.15.188:3300/0 conn(0x7f720c0d1d70 msgr2=0x7f720c0c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:08.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.399+0000 7f720be00640 1 --2- 172.21.15.19:0/1571092312 >> v2:172.21.15.188:3300/0 conn(0x7f720c0d1d70 0x7f720c0c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:08.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.399+0000 7f720be00640 1 -- 172.21.15.19:0/1571092312 >> v1:172.21.15.188:6789/0 conn(0x7f720c0c7640 legacy=0x7f720c071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:08.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.399+0000 7f720be00640 1 -- 172.21.15.19:0/1571092312 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c0d0760 con 0x7f720c15e7b0 2024-03-28T13:10:08.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.400+0000 7f720be00640 1 --2- 172.21.15.19:0/1571092312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c15e7b0 0x7f720c0b4980 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f720c162150 tx=0x7f71fc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1274c3bb19df4e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:08.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.401+0000 7f720aa00640 1 -- 172.21.15.19:0/1571092312 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc035020 con 0x7f720c15e7b0 2024-03-28T13:10:08.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.401+0000 7f720aa00640 1 -- 172.21.15.19:0/1571092312 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71fc031870 con 0x7f720c15e7b0 2024-03-28T13:10:08.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.401+0000 7f720aa00640 1 -- 172.21.15.19:0/1571092312 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc031c40 con 0x7f720c15e7b0 2024-03-28T13:10:08.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.401+0000 7f7212a00640 1 -- 172.21.15.19:0/1571092312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c15e7b0 msgr2=0x7f720c0b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:08.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.401+0000 7f7212a00640 1 --2- 172.21.15.19:0/1571092312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c15e7b0 0x7f720c0b4980 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f720c162150 tx=0x7f71fc02b910 comp rx=0 tx=0).stop 2024-03-28T13:10:08.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.402+0000 7f7212a00640 1 -- 172.21.15.19:0/1571092312 shutdown_connections 2024-03-28T13:10:08.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.402+0000 7f7212a00640 1 --2- 172.21.15.19:0/1571092312 >> v2:172.21.15.188:3300/0 conn(0x7f720c0d1d70 0x7f720c0c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:08.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.402+0000 7f7212a00640 1 --2- 172.21.15.19:0/1571092312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c15e7b0 0x7f720c0b4980 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:08.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.402+0000 7f7212a00640 1 -- 172.21.15.19:0/1571092312 >> 172.21.15.19:0/1571092312 conn(0x7f720c0b3e90 msgr2=0x7f720c0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:08.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.402+0000 7f7212a00640 1 -- 172.21.15.19:0/1571092312 shutdown_connections 2024-03-28T13:10:08.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.402+0000 7f7212a00640 1 -- 172.21.15.19:0/1571092312 wait complete. 2024-03-28T13:10:08.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.404+0000 7f7212a00640 1 Processor -- start 2024-03-28T13:10:08.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.404+0000 7f7212a00640 1 -- start start 2024-03-28T13:10:08.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.405+0000 7f7212a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f720c0c7640 0x7f720c150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:08.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.405+0000 7f7212a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c0d1d70 0x7f720c0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:08.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.405+0000 7f7212a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f720c15e7b0 0x7f720c0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:08.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.405+0000 7f7212a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f720c0cfe30 con 0x7f720c0d1d70 2024-03-28T13:10:08.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.405+0000 7f7212a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f720c0d0d20 con 0x7f720c15e7b0 2024-03-28T13:10:08.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.405+0000 7f7212a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f720c06da40 con 0x7f720c0c7640 2024-03-28T13:10:08.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.405+0000 7f720b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c0d1d70 0x7f720c0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:08.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.405+0000 7f720b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c0d1d70 0x7f720c0cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53896/0 (socket says 172.21.15.19:53896) 2024-03-28T13:10:08.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.405+0000 7f720b400640 1 -- 172.21.15.19:0/3927474993 learned_addr learned my addr 172.21.15.19:0/3927474993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:08.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.405+0000 7f720be00640 1 --2- 172.21.15.19:0/3927474993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f720c0c7640 0x7f720c150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:08.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.405+0000 7f7210a00640 1 --2- 172.21.15.19:0/3927474993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f720c15e7b0 0x7f720c0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:08.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.406+0000 7f720b400640 1 -- 172.21.15.19:0/3927474993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f720c0c7640 msgr2=0x7f720c150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:08.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.406+0000 7f720b400640 1 --2- 172.21.15.19:0/3927474993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f720c0c7640 0x7f720c150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:08.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.406+0000 7f720b400640 1 -- 172.21.15.19:0/3927474993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f720c15e7b0 msgr2=0x7f720c0cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:08.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.406+0000 7f720b400640 1 --2- 172.21.15.19:0/3927474993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f720c15e7b0 0x7f720c0cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:08.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.406+0000 7f720b400640 1 -- 172.21.15.19:0/3927474993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c157310 con 0x7f720c0d1d70 2024-03-28T13:10:08.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.406+0000 7f720b400640 1 --2- 172.21.15.19:0/3927474993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c0d1d70 0x7f720c0cac70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71f800d5c0 tx=0x7f71f800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.406+0000 7f7203e00640 1 -- 172.21.15.19:0/3927474993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f80125f0 con 0x7f720c0d1d70 2024-03-28T13:10:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.406+0000 7f7203e00640 1 -- 172.21.15.19:0/3927474993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f8019070 con 0x7f720c0d1d70 2024-03-28T13:10:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.406+0000 7f7203e00640 1 -- 172.21.15.19:0/3927474993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f80129d0 con 0x7f720c0d1d70 2024-03-28T13:10:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.406+0000 7f7212a00640 1 -- 172.21.15.19:0/3927474993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720c1575f0 con 0x7f720c0d1d70 2024-03-28T13:10:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.407+0000 7f7212a00640 1 -- 172.21.15.19:0/3927474993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f720c157bb0 con 0x7f720c0d1d70 2024-03-28T13:10:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.408+0000 7f7212a00640 1 -- 172.21.15.19:0/3927474993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f720c068cd0 con 0x7f720c0d1d70 2024-03-28T13:10:08.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.409+0000 7f7203e00640 1 -- 172.21.15.19:0/3927474993 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f71f8012b70 con 0x7f720c0d1d70 2024-03-28T13:10:08.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.410+0000 7f7203e00640 1 --2- 172.21.15.19:0/3927474993 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f71ec06c530 0x7f71ec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:08.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.410+0000 7f7203e00640 1 -- 172.21.15.19:0/3927474993 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f71f808ba70 con 0x7f720c0d1d70 2024-03-28T13:10:08.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.411+0000 7f720be00640 1 --2- 172.21.15.19:0/3927474993 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f71ec06c530 0x7f71ec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.412+0000 7f720be00640 1 --2- 172.21.15.19:0/3927474993 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f71ec06c530 0x7f71ec06e9f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71fc036000 tx=0x7f71fc03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:08.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.415+0000 7f7203e00640 1 -- 172.21.15.19:0/3927474993 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f71f805a080 con 0x7f720c0d1d70 2024-03-28T13:10:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.659+0000 7f7212a00640 1 -- 172.21.15.19:0/3927474993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f720c0ceee0 con 0x7f720c0d1d70 2024-03-28T13:10:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.660+0000 7f7203e00640 1 -- 172.21.15.19:0/3927474993 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f71f8059980 con 0x7f720c0d1d70 2024-03-28T13:10:08.661 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:10:08.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.662+0000 7f7212a00640 1 -- 172.21.15.19:0/3927474993 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f71ec06c530 msgr2=0x7f71ec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:08.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.662+0000 7f7212a00640 1 --2- 172.21.15.19:0/3927474993 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f71ec06c530 0x7f71ec06e9f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71fc036000 tx=0x7f71fc03c040 comp rx=0 tx=0).stop 2024-03-28T13:10:08.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.662+0000 7f7212a00640 1 -- 172.21.15.19:0/3927474993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c0d1d70 msgr2=0x7f720c0cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:08.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.662+0000 7f7212a00640 1 --2- 172.21.15.19:0/3927474993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c0d1d70 0x7f720c0cac70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71f800d5c0 tx=0x7f71f800da80 comp rx=0 tx=0).stop 2024-03-28T13:10:08.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.662+0000 7f7212a00640 1 -- 172.21.15.19:0/3927474993 shutdown_connections 2024-03-28T13:10:08.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.662+0000 7f7212a00640 1 --2- 172.21.15.19:0/3927474993 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f71ec06c530 0x7f71ec06e9f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:08.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.662+0000 7f7212a00640 1 --2- 172.21.15.19:0/3927474993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f720c15e7b0 0x7f720c0cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:08.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.662+0000 7f7212a00640 1 --2- 172.21.15.19:0/3927474993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c0d1d70 0x7f720c0cac70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:08.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.662+0000 7f7212a00640 1 --2- 172.21.15.19:0/3927474993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f720c0c7640 0x7f720c150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:08.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.662+0000 7f7212a00640 1 -- 172.21.15.19:0/3927474993 >> 172.21.15.19:0/3927474993 conn(0x7f720c0b3e90 msgr2=0x7f720c0ccb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:08.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.662+0000 7f7212a00640 1 -- 172.21.15.19:0/3927474993 shutdown_connections 2024-03-28T13:10:08.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:08.662+0000 7f7212a00640 1 -- 172.21.15.19:0/3927474993 wait complete. 2024-03-28T13:10:08.680 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-28T13:10:08.680 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:10:08.680 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-03-28T13:10:08.681 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-28T13:10:08.681 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-28T13:10:08.681 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-28T13:10:08.681 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-28T13:10:08.681 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:10:08.681 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-28T13:10:08.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:08.681 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-03-28T13:10:08.739 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:10:08.739 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:10:08.739 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:10:08.740 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:08.740 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-03-28T13:10:08.740 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:08.740 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:10:08.740 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:08.740 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-03-28T13:10:08.741 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:10:08.741 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:08.741 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-03-28T13:10:08.741 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:10:08.741 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:08.742 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:10:08.742 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:08.742 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-28T13:10:08.742 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42544sec preferred_lft 42544sec 2024-03-28T13:10:08.742 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2024-03-28T13:10:08.742 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:08.744 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-28T13:10:08.744 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:08.744 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-03-28T13:10:08.744 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add name ceph-brx type bridge 2024-03-28T13:10:08.744 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr flush dev ceph-brx 2024-03-28T13:10:08.744 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set ceph-brx up 2024-03-28T13:10:08.744 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-28T13:10:08.744 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-03-28T13:10:08.852 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:09.253 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:09.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:09.301 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:10:09.382 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-03-28T13:10:09.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:09.385 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-03-28T13:10:09.448 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-03-28T13:10:09.448 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:10:09.448 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-28T13:10:09.449 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-28T13:10:09.449 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:10:09.449 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:09.449 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-03-28T13:10:09.449 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-28T13:10:09.449 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-28T13:10:09.449 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-28T13:10:09.449 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-03-28T13:10:09.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:10.223 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:10.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:10.231 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-03-28T13:10:10.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:10.275 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2024-03-28T13:10:10.363 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:10.364 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-03-28T13:10:10.364 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-28T13:10:10.364 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-28T13:10:10.364 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-03-28T13:10:10.456 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:10.542 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:10.550 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-28T13:10:10.550 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:10.550 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-03-28T13:10:10.550 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-28T13:10:10.550 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-28T13:10:10.551 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-28T13:10:10.551 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-28T13:10:10.551 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-28T13:10:10.551 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-03-28T13:10:10.646 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:10.867 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:10.876 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:10.876 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-03-28T13:10:10.876 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-03-28T13:10:10.876 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.0 master ceph-brx 2024-03-28T13:10:10.876 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-03-28T13:10:10.966 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:11.081 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:11.090 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:10:11.090 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-28T13:10:11.138 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-28T13:10:11.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:10:11.139 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-28T13:10:11.183 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:10:11.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:11.184 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-28T13:10:11.350 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-28T13:10:11.350 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-28T13:10:11.350 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-03-28T13:10:11.992 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-28T13:10:11.992 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-28T13:10:11.992 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-03-28T13:10:11.992 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:10:11.992 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:10:12.025 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-03-28T13:10:12.025 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-03-28T13:10:12.025 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-03-28T13:10:12.026 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:10:12.026 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:10:12.026 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:10:12.026 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:10:12.026 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:10:12.028 INFO:teuthology.orchestra.run.smithi188.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-03-28T13:10:12.028 INFO:teuthology.orchestra.run.smithi188.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:10:12.028 INFO:teuthology.orchestra.run.smithi188.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:10:12.028 INFO:teuthology.orchestra.run.smithi188.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:10:12.028 INFO:teuthology.orchestra.run.smithi188.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:10:12.032 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:10:12.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:12.033 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-28T13:10:12.122 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:10:12.226 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:10:12.231 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:12.241 INFO:teuthology.orchestra.run.smithi188.stdout:client4475 2024-03-28T13:10:12.241 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:10:12.241 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:12.295 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:10:12.400 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:10:12.405 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:12.414 INFO:teuthology.orchestra.run.smithi188.stdout:client4475 2024-03-28T13:10:12.415 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:10:12.511 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:10:12.517 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:12.525 INFO:teuthology.orchestra.run.smithi188.stdout:client4475 2024-03-28T13:10:12.525 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e6b569d9-f2ce-49dc-bbe7-c0ab33080203.client4475/status') 2024-03-28T13:10:12.620 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:12.660 INFO:teuthology.orchestra.run.smithi188.stdout:instance: client.4475 (3)192.168.0.1:0/2480678075 2024-03-28T13:10:12.660 INFO:teuthology.orchestra.run.smithi188.stdout:blocklisted: false 2024-03-28T13:10:12.661 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records in 2024-03-28T13:10:12.661 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records out 2024-03-28T13:10:12.661 INFO:teuthology.orchestra.run.smithi188.stderr:69 bytes copied, 6.7674e-05 s, 1.0 MB/s 2024-03-28T13:10:12.663 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:12.673 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-03-28T13:10:12.673 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-28T13:10:12.673 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-28T13:10:12.673 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:10:12.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.906+0000 7f9b50e00640 1 Processor -- start 2024-03-28T13:10:12.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.906+0000 7f9b50e00640 1 -- start start 2024-03-28T13:10:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.908+0000 7f9b50e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b4c1472d0 0x7f9b4c06b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.908+0000 7f9b50e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b4c0733f0 con 0x7f9b4c147730 2024-03-28T13:10:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.908+0000 7f9b50e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b4c0b4cb0 con 0x7f9b4c070c50 2024-03-28T13:10:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.908+0000 7f9b50e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9b4c162fd0 con 0x7f9b4c1472d0 2024-03-28T13:10:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.908+0000 7f9b4aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9b4c147730 0x7f9b4c0725d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:55560/0 (socket says 172.21.15.19:55560) 2024-03-28T13:10:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.908+0000 7f9b4aa00640 1 -- 172.21.15.19:0/886376856 learned_addr learned my addr 172.21.15.19:0/886376856 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:12.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.909+0000 7f9b4b400640 1 --2- 172.21.15.19:0/886376856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b4c1472d0 0x7f9b4c06b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:12.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.909+0000 7f9b4a000640 1 -- 172.21.15.19:0/886376856 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1688256767 0 0) 0x7f9b4c0733f0 con 0x7f9b4c147730 2024-03-28T13:10:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.909+0000 7f9b4a000640 1 -- 172.21.15.19:0/886376856 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b30003520 con 0x7f9b4c147730 2024-03-28T13:10:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.909+0000 7f9b4a000640 1 -- 172.21.15.19:0/886376856 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3630987088 0 0) 0x7f9b30003520 con 0x7f9b4c147730 2024-03-28T13:10:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.909+0000 7f9b4a000640 1 -- 172.21.15.19:0/886376856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b4c1472d0 msgr2=0x7f9b4c06b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.909+0000 7f9b4a000640 1 --2- 172.21.15.19:0/886376856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b4c1472d0 0x7f9b4c06b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.909+0000 7f9b4a000640 1 -- 172.21.15.19:0/886376856 >> v1:172.21.15.188:6789/0 conn(0x7f9b4c070c50 legacy=0x7f9b4c072ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.909+0000 7f9b4a000640 1 -- 172.21.15.19:0/886376856 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4c162a10 con 0x7f9b4c147730 2024-03-28T13:10:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.909+0000 7f9b4a000640 1 -- 172.21.15.19:0/886376856 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f9b38001670 con 0x7f9b4c147730 2024-03-28T13:10:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.910+0000 7f9b4a000640 1 -- 172.21.15.19:0/886376856 >> v1:172.21.15.19:6789/0 conn(0x7f9b4c147730 legacy=0x7f9b4c0725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.910+0000 7f9b4a000640 1 --2- 172.21.15.19:0/886376856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b300039e0 0x7f9b30003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.910+0000 7f9b4a000640 1 --2- 172.21.15.19:0/886376856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b30004800 0x7f9b30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.910+0000 7f9b4a000640 1 --2- 172.21.15.19:0/886376856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b4c1472d0 0x7f9b300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.910+0000 7f9b4a000640 1 -- 172.21.15.19:0/886376856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c162fd0 con 0x7f9b30004800 2024-03-28T13:10:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.910+0000 7f9b4a000640 1 -- 172.21.15.19:0/886376856 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3000baa0 con 0x7f9b300039e0 2024-03-28T13:10:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.910+0000 7f9b4a000640 1 -- 172.21.15.19:0/886376856 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9b3000bca0 con 0x7f9b4c1472d0 2024-03-28T13:10:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.910+0000 7f9b4aa00640 1 --2- 172.21.15.19:0/886376856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b30004800 0x7f9b30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.910+0000 7f9b4aa00640 1 -- 172.21.15.19:0/886376856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b4c1472d0 msgr2=0x7f9b300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.910+0000 7f9b4aa00640 1 --2- 172.21.15.19:0/886376856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b4c1472d0 0x7f9b300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.910+0000 7f9b4b400640 1 --2- 172.21.15.19:0/886376856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b300039e0 0x7f9b30003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.910+0000 7f9b4aa00640 1 -- 172.21.15.19:0/886376856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b300039e0 msgr2=0x7f9b30003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.910+0000 7f9b4aa00640 1 --2- 172.21.15.19:0/886376856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b300039e0 0x7f9b30003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.910+0000 7f9b4aa00640 1 -- 172.21.15.19:0/886376856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b30003520 con 0x7f9b30004800 2024-03-28T13:10:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.910+0000 7f9b4b400640 1 --2- 172.21.15.19:0/886376856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b300039e0 0x7f9b30003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:12.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.911+0000 7f9b4aa00640 1 --2- 172.21.15.19:0/886376856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b30004800 0x7f9b30004c80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800c160 tx=0x7f9b30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f931a9487594e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:12.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.912+0000 7f9b4a000640 1 -- 172.21.15.19:0/886376856 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38007c60 con 0x7f9b30004800 2024-03-28T13:10:12.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.912+0000 7f9b4a000640 1 -- 172.21.15.19:0/886376856 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b38030070 con 0x7f9b30004800 2024-03-28T13:10:12.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.912+0000 7f9b4a000640 1 -- 172.21.15.19:0/886376856 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b38005d50 con 0x7f9b30004800 2024-03-28T13:10:12.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.912+0000 7f9b50e00640 1 -- 172.21.15.19:0/886376856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b30004800 msgr2=0x7f9b30004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:12.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.912+0000 7f9b50e00640 1 --2- 172.21.15.19:0/886376856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b30004800 0x7f9b30004c80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800c160 tx=0x7f9b30005830 comp rx=0 tx=0).stop 2024-03-28T13:10:12.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.912+0000 7f9b4be00640 1 -- 172.21.15.19:0/886376856 reap_dead start 2024-03-28T13:10:12.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.912+0000 7f9b50e00640 1 -- 172.21.15.19:0/886376856 shutdown_connections 2024-03-28T13:10:12.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.912+0000 7f9b50e00640 1 -- 172.21.15.19:0/886376856 >> 172.21.15.19:0/886376856 conn(0x7f9b4c0d1090 msgr2=0x7f9b4c0d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:12.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.913+0000 7f9b50e00640 1 -- 172.21.15.19:0/886376856 shutdown_connections 2024-03-28T13:10:12.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.913+0000 7f9b50e00640 1 -- 172.21.15.19:0/886376856 wait complete. 2024-03-28T13:10:12.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.914+0000 7f9b50e00640 1 Processor -- start 2024-03-28T13:10:12.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.914+0000 7f9b50e00640 1 -- start start 2024-03-28T13:10:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.915+0000 7f9b50e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b30004800 0x7f9b4c138050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.915+0000 7f9b50e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b4c13c1d0 0x7f9b4c13c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.915+0000 7f9b50e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b4c13a450 0x7f9b4c13a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.915+0000 7f9b50e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c147e80 con 0x7f9b30004800 2024-03-28T13:10:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.915+0000 7f9b50e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c0b4fe0 con 0x7f9b4c13a450 2024-03-28T13:10:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.915+0000 7f9b50e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9b4c073760 con 0x7f9b4c13c1d0 2024-03-28T13:10:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.916+0000 7f9b4b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b30004800 0x7f9b4c138050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.916+0000 7f9b4b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b30004800 0x7f9b4c138050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53916/0 (socket says 172.21.15.19:53916) 2024-03-28T13:10:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.916+0000 7f9b4b400640 1 -- 172.21.15.19:0/3430142579 learned_addr learned my addr 172.21.15.19:0/3430142579 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:12.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.916+0000 7f9b4b400640 1 -- 172.21.15.19:0/3430142579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b4c13c1d0 msgr2=0x7f9b4c13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.916+0000 7f9b4b400640 1 --2- 172.21.15.19:0/3430142579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b4c13c1d0 0x7f9b4c13c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.916+0000 7f9b4b400640 1 -- 172.21.15.19:0/3430142579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b4c13a450 msgr2=0x7f9b4c13a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.916+0000 7f9b4b400640 1 --2- 172.21.15.19:0/3430142579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b4c13a450 0x7f9b4c13a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.916+0000 7f9b4b400640 1 -- 172.21.15.19:0/3430142579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4c13dcc0 con 0x7f9b30004800 2024-03-28T13:10:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.916+0000 7f9b4b400640 1 --2- 172.21.15.19:0/3430142579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b30004800 0x7f9b4c138050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c0108c0 tx=0x7f9b3c010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.917+0000 7f9b43400640 1 -- 172.21.15.19:0/3430142579 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c00f040 con 0x7f9b30004800 2024-03-28T13:10:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.917+0000 7f9b43400640 1 -- 172.21.15.19:0/3430142579 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b3c00eb00 con 0x7f9b30004800 2024-03-28T13:10:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.917+0000 7f9b43400640 1 -- 172.21.15.19:0/3430142579 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c01c400 con 0x7f9b30004800 2024-03-28T13:10:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.917+0000 7f9b50e00640 1 -- 172.21.15.19:0/3430142579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4c13cc20 con 0x7f9b30004800 2024-03-28T13:10:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.917+0000 7f9b50e00640 1 -- 172.21.15.19:0/3430142579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b4c13d1e0 con 0x7f9b30004800 2024-03-28T13:10:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.918+0000 7f9b50e00640 1 -- 172.21.15.19:0/3430142579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b4c075940 con 0x7f9b30004800 2024-03-28T13:10:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.919+0000 7f9b43400640 1 -- 172.21.15.19:0/3430142579 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9b3c01c5a0 con 0x7f9b30004800 2024-03-28T13:10:12.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.920+0000 7f9b43400640 1 --2- 172.21.15.19:0/3430142579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9b2c06c560 0x7f9b2c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:12.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.920+0000 7f9b43400640 1 -- 172.21.15.19:0/3430142579 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9b3c08bc50 con 0x7f9b30004800 2024-03-28T13:10:12.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.921+0000 7f9b4aa00640 1 --2- 172.21.15.19:0/3430142579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9b2c06c560 0x7f9b2c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:12.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.922+0000 7f9b4aa00640 1 --2- 172.21.15.19:0/3430142579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9b2c06c560 0x7f9b2c06ea20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b300057e0 tx=0x7f9b38007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:12.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:12.930+0000 7f9b43400640 1 -- 172.21.15.19:0/3430142579 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9b3c091020 con 0x7f9b30004800 2024-03-28T13:10:13.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:13.238+0000 7f9b50e00640 1 -- 172.21.15.19:0/3430142579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9b4c0d1d60 con 0x7f9b30004800 2024-03-28T13:10:13.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:13.239+0000 7f9b43400640 1 -- 172.21.15.19:0/3430142579 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f9b3c018030 con 0x7f9b30004800 2024-03-28T13:10:13.239 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:10:13.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:13.241+0000 7f9b50e00640 1 -- 172.21.15.19:0/3430142579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9b2c06c560 msgr2=0x7f9b2c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:13.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:13.241+0000 7f9b50e00640 1 --2- 172.21.15.19:0/3430142579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9b2c06c560 0x7f9b2c06ea20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b300057e0 tx=0x7f9b38007000 comp rx=0 tx=0).stop 2024-03-28T13:10:13.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:13.241+0000 7f9b50e00640 1 -- 172.21.15.19:0/3430142579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b30004800 msgr2=0x7f9b4c138050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:13.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:13.241+0000 7f9b50e00640 1 --2- 172.21.15.19:0/3430142579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b30004800 0x7f9b4c138050 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c0108c0 tx=0x7f9b3c010d80 comp rx=0 tx=0).stop 2024-03-28T13:10:13.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:13.241+0000 7f9b50e00640 1 -- 172.21.15.19:0/3430142579 shutdown_connections 2024-03-28T13:10:13.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:13.241+0000 7f9b50e00640 1 --2- 172.21.15.19:0/3430142579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9b2c06c560 0x7f9b2c06ea20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:13.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:13.241+0000 7f9b50e00640 1 --2- 172.21.15.19:0/3430142579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b4c13a450 0x7f9b4c13a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:13.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:13.241+0000 7f9b50e00640 1 --2- 172.21.15.19:0/3430142579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9b4c13c1d0 0x7f9b4c13c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:13.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:13.241+0000 7f9b50e00640 1 --2- 172.21.15.19:0/3430142579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b30004800 0x7f9b4c138050 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:13.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:13.241+0000 7f9b50e00640 1 -- 172.21.15.19:0/3430142579 >> 172.21.15.19:0/3430142579 conn(0x7f9b4c0d1090 msgr2=0x7f9b4c0b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:13.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:13.242+0000 7f9b50e00640 1 -- 172.21.15.19:0/3430142579 shutdown_connections 2024-03-28T13:10:13.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:13.242+0000 7f9b50e00640 1 -- 172.21.15.19:0/3430142579 wait complete. 2024-03-28T13:10:13.271 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-28T13:10:13.271 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:10:13.271 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-03-28T13:10:13.271 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-28T13:10:13.271 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-28T13:10:13.271 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-28T13:10:13.272 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-28T13:10:13.272 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:10:13.272 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-28T13:10:13.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:13.272 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-03-28T13:10:13.329 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:10:13.329 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:10:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:10:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-03-28T13:10:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:13.330 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:10:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-03-28T13:10:13.330 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:10:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-03-28T13:10:13.330 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:10:13.330 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:13.330 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:10:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-28T13:10:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42540sec preferred_lft 42540sec 2024-03-28T13:10:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2024-03-28T13:10:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:13.331 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:10:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 76:65:e3:2d:40:32 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:10:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:10:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::944e:57ff:fe82:9c5d/64 scope link 2024-03-28T13:10:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:13.331 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:10:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 76:65:e3:2d:40:32 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-28T13:10:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::7465:e3ff:fe2d:4032/64 scope link 2024-03-28T13:10:13.331 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:13.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:13.333 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:10:13.419 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-03-28T13:10:13.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:13.423 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-03-28T13:10:13.476 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-03-28T13:10:13.476 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:10:13.477 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-28T13:10:13.477 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-28T13:10:13.477 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:10:13.478 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:13.478 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-03-28T13:10:13.478 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-28T13:10:13.478 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-28T13:10:13.478 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-28T13:10:13.478 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-03-28T13:10:13.582 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:13.716 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:13.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:13.726 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-03-28T13:10:13.774 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-28T13:10:13.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:13.775 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2024-03-28T13:10:13.819 INFO:teuthology.orchestra.run.smithi188.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-28T13:10:13.820 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:13.820 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-03-28T13:10:13.820 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-28T13:10:13.820 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-28T13:10:13.820 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-03-28T13:10:13.971 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:14.057 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:14.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:14.067 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-28T13:10:14.151 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:10:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:10:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:10:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-03-28T13:10:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:14.151 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:10:14.151 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 56:92:2f:fc:ef:92 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-28T13:10:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-28T13:10:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::5492:2fff:fefc:ef92/64 scope link 2024-03-28T13:10:14.152 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:14.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:14.156 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-28T13:10:14.248 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:10:14.248 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:10:14.248 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:10:14.248 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:14.249 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-03-28T13:10:14.249 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:14.249 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:10:14.249 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 56:92:2f:fc:ef:92 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-28T13:10:14.249 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-28T13:10:14.250 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:14.250 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::5492:2fff:fefc:ef92/64 scope link 2024-03-28T13:10:14.250 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:10:14.253 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-28T13:10:14.253 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:14.253 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-03-28T13:10:14.253 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-28T13:10:14.254 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-28T13:10:14.254 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-28T13:10:14.254 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-28T13:10:14.254 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-28T13:10:14.254 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-03-28T13:10:14.362 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:14.567 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:14.577 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:10:14.577 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-03-28T13:10:14.577 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.1 up 2024-03-28T13:10:14.577 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.1 master ceph-brx 2024-03-28T13:10:14.577 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-03-28T13:10:14.684 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:14.801 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:14.811 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:10:14.811 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-28T13:10:14.856 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-28T13:10:14.857 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:10:14.857 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-28T13:10:14.953 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:10:14.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:14.953 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-28T13:10:15.090 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-28T13:10:15.090 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-03-28T13:10:15.091 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-03-28T13:10:15.115 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-28T13:10:15.115 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-03-28T13:10:15.115 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-03-28T13:10:15.115 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:10:15.115 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:10:15.141 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-03-28T13:10:15.142 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-03-28T13:10:15.142 INFO:teuthology.orchestra.run.smithi188.stdout:invalid new device string format 2024-03-28T13:10:15.142 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:10:15.142 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:10:15.142 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:10:15.142 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:10:15.142 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:10:15.144 INFO:teuthology.orchestra.run.smithi188.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-03-28T13:10:15.144 INFO:teuthology.orchestra.run.smithi188.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:10:15.144 INFO:teuthology.orchestra.run.smithi188.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:10:15.144 INFO:teuthology.orchestra.run.smithi188.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:10:15.145 INFO:teuthology.orchestra.run.smithi188.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:10:15.148 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:10:15.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:15.148 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-28T13:10:15.232 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:10:15.337 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:10:15.342 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:15.351 INFO:teuthology.orchestra.run.smithi188.stdout:client4481 2024-03-28T13:10:15.351 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:10:15.352 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:15.402 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:10:15.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:10:15.558 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:15.567 INFO:teuthology.orchestra.run.smithi188.stdout:client4481 2024-03-28T13:10:15.567 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:10:15.670 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:10:15.675 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:15.684 INFO:teuthology.orchestra.run.smithi188.stdout:client4481 2024-03-28T13:10:15.685 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/e6b569d9-f2ce-49dc-bbe7-c0ab33080203.client4481/status') 2024-03-28T13:10:15.788 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:10:15.828 INFO:teuthology.orchestra.run.smithi188.stdout:instance: client.4481 (3)192.168.0.2:0/2005065766 2024-03-28T13:10:15.828 INFO:teuthology.orchestra.run.smithi188.stdout:blocklisted: false 2024-03-28T13:10:15.828 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records in 2024-03-28T13:10:15.828 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records out 2024-03-28T13:10:15.828 INFO:teuthology.orchestra.run.smithi188.stderr:69 bytes copied, 6.851e-05 s, 1.0 MB/s 2024-03-28T13:10:15.831 INFO:teuthology.orchestra.run.smithi188.stderr:2024-03-28T13:10:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:10:15.841 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-28T13:10:15.864 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:10:16.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.089+0000 7f3ece000640 1 Processor -- start 2024-03-28T13:10:16.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.089+0000 7f3ece000640 1 -- start start 2024-03-28T13:10:16.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.091+0000 7f3ece000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ec80cc0b0 0x7f3ec806dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:16.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.091+0000 7f3ece000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3ec80d04d0 0x7f3ec815ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:16.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.091+0000 7f3ece000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ec806e4f0 con 0x7f3ec815fc50 2024-03-28T13:10:16.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.091+0000 7f3ece000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3ec815c610 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.091+0000 7f3ece000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3ec80b3150 con 0x7f3ec80d04d0 2024-03-28T13:10:16.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.091+0000 7f3eccc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ec80cc0b0 0x7f3ec806dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:16.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.092+0000 7f3eccc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ec80cc0b0 0x7f3ec806dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53740/0 (socket says 172.21.15.19:53740) 2024-03-28T13:10:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.092+0000 7f3eccc00640 1 -- 172.21.15.19:0/111571734 learned_addr learned my addr 172.21.15.19:0/111571734 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.092+0000 7f3ecd600640 1 --2- 172.21.15.19:0/111571734 >> v2:172.21.15.188:3300/0 conn(0x7f3ec80d04d0 0x7f3ec815ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.092+0000 7f3eccc00640 1 -- 172.21.15.19:0/111571734 >> v2:172.21.15.188:3300/0 conn(0x7f3ec80d04d0 msgr2=0x7f3ec815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.092+0000 7f3eccc00640 1 --2- 172.21.15.19:0/111571734 >> v2:172.21.15.188:3300/0 conn(0x7f3ec80d04d0 0x7f3ec815ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.092+0000 7f3eccc00640 1 -- 172.21.15.19:0/111571734 >> v1:172.21.15.188:6789/0 conn(0x7f3ec815fc50 legacy=0x7f3ec80cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:16.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.092+0000 7f3eccc00640 1 -- 172.21.15.19:0/111571734 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec81600e0 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.095+0000 7f3eccc00640 1 --2- 172.21.15.19:0/111571734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec80cc0b0 0x7f3ec806dfb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80b3dd0 tx=0x7f3ebc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d055fe2a3a046a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:16.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.096+0000 7f3ec7400640 1 -- 172.21.15.19:0/111571734 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc035020 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.096+0000 7f3ec7400640 1 -- 172.21.15.19:0/111571734 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ebc031870 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.096+0000 7f3ec7400640 1 -- 172.21.15.19:0/111571734 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc031c40 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.097+0000 7f3ece000640 1 -- 172.21.15.19:0/111571734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec80cc0b0 msgr2=0x7f3ec806dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:16.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.097+0000 7f3ece000640 1 --2- 172.21.15.19:0/111571734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec80cc0b0 0x7f3ec806dfb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3ec80b3dd0 tx=0x7f3ebc02b910 comp rx=0 tx=0).stop 2024-03-28T13:10:16.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.097+0000 7f3ece000640 1 -- 172.21.15.19:0/111571734 shutdown_connections 2024-03-28T13:10:16.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.097+0000 7f3ece000640 1 --2- 172.21.15.19:0/111571734 >> v2:172.21.15.188:3300/0 conn(0x7f3ec80d04d0 0x7f3ec815ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.097+0000 7f3ece000640 1 --2- 172.21.15.19:0/111571734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec80cc0b0 0x7f3ec806dfb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.097+0000 7f3ece000640 1 -- 172.21.15.19:0/111571734 >> 172.21.15.19:0/111571734 conn(0x7f3ec80ccad0 msgr2=0x7f3ec80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:16.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.097+0000 7f3ece000640 1 -- 172.21.15.19:0/111571734 shutdown_connections 2024-03-28T13:10:16.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.097+0000 7f3ece000640 1 -- 172.21.15.19:0/111571734 wait complete. 2024-03-28T13:10:16.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.100+0000 7f3ece000640 1 Processor -- start 2024-03-28T13:10:16.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.100+0000 7f3ece000640 1 -- start start 2024-03-28T13:10:16.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.101+0000 7f3ece000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec80cc0b0 0x7f3ec81378d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.101+0000 7f3ece000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ec80d04d0 0x7f3ec81380a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.101+0000 7f3ece000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ec8139b90 0x7f3ec813a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.101+0000 7f3ece000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec8160980 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.101+0000 7f3ece000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec81606a0 con 0x7f3ec80d04d0 2024-03-28T13:10:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.101+0000 7f3ece000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec80cc490 con 0x7f3ec8139b90 2024-03-28T13:10:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.102+0000 7f3eccc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec80cc0b0 0x7f3ec81378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.102+0000 7f3eccc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec80cc0b0 0x7f3ec81378d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53742/0 (socket says 172.21.15.19:53742) 2024-03-28T13:10:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.102+0000 7f3eccc00640 1 -- 172.21.15.19:0/1037871569 learned_addr learned my addr 172.21.15.19:0/1037871569 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.102+0000 7f3ec7e00640 1 --2- 172.21.15.19:0/1037871569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ec80d04d0 0x7f3ec81380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.102+0000 7f3eccc00640 1 -- 172.21.15.19:0/1037871569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ec8139b90 msgr2=0x7f3ec813a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.102+0000 7f3ecd600640 1 --2- 172.21.15.19:0/1037871569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ec8139b90 0x7f3ec813a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.102+0000 7f3eccc00640 1 --2- 172.21.15.19:0/1037871569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ec8139b90 0x7f3ec813a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.102+0000 7f3eccc00640 1 -- 172.21.15.19:0/1037871569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ec80d04d0 msgr2=0x7f3ec81380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:16.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.102+0000 7f3eccc00640 1 --2- 172.21.15.19:0/1037871569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ec80d04d0 0x7f3ec81380a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.102+0000 7f3eccc00640 1 -- 172.21.15.19:0/1037871569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec813ab60 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.103+0000 7f3eccc00640 1 --2- 172.21.15.19:0/1037871569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec80cc0b0 0x7f3ec81378d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc02b950 tx=0x7f3ebc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:16.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.103+0000 7f3ec4c00640 1 -- 172.21.15.19:0/1037871569 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc040070 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.103+0000 7f3ece000640 1 -- 172.21.15.19:0/1037871569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec813ae40 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.103+0000 7f3ec4c00640 1 -- 172.21.15.19:0/1037871569 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ebc030d00 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.103+0000 7f3ec4c00640 1 -- 172.21.15.19:0/1037871569 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc035040 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.103+0000 7f3ece000640 1 -- 172.21.15.19:0/1037871569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec8155d50 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.104+0000 7f3ece000640 1 -- 172.21.15.19:0/1037871569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec806dfb0 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.105+0000 7f3ec4c00640 1 -- 172.21.15.19:0/1037871569 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3ebc038020 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.106+0000 7f3ec4c00640 1 --2- 172.21.15.19:0/1037871569 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ea806c530 0x7f3ea806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:16.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.107+0000 7f3ec4c00640 1 -- 172.21.15.19:0/1037871569 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3ebc0ad5f0 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.107+0000 7f3ec7e00640 1 --2- 172.21.15.19:0/1037871569 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ea806c530 0x7f3ea806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:16.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.108+0000 7f3ec7e00640 1 --2- 172.21.15.19:0/1037871569 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ea806c530 0x7f3ea806e9f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4007fd0 tx=0x7f3eb4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:16.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.111+0000 7f3ec4c00640 1 -- 172.21.15.19:0/1037871569 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3ec806dfb0 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.317+0000 7f3ece000640 1 -- 172.21.15.19:0/1037871569 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ec8075940 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.318+0000 7f3ec4c00640 1 -- 172.21.15.19:0/1037871569 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16609 (secure 0 0 0) 0x7f3ebc07bb80 con 0x7f3ec80cc0b0 2024-03-28T13:10:16.319 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:16.319 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":23,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:42.668077+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:10:16.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.321+0000 7f3ece000640 1 -- 172.21.15.19:0/1037871569 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ea806c530 msgr2=0x7f3ea806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.321+0000 7f3ece000640 1 --2- 172.21.15.19:0/1037871569 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ea806c530 0x7f3ea806e9f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4007fd0 tx=0x7f3eb4009040 comp rx=0 tx=0).stop 2024-03-28T13:10:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.321+0000 7f3ece000640 1 -- 172.21.15.19:0/1037871569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec80cc0b0 msgr2=0x7f3ec81378d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.321+0000 7f3ece000640 1 --2- 172.21.15.19:0/1037871569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec80cc0b0 0x7f3ec81378d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc02b950 tx=0x7f3ebc0046c0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.321+0000 7f3ece000640 1 -- 172.21.15.19:0/1037871569 shutdown_connections 2024-03-28T13:10:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.321+0000 7f3ece000640 1 --2- 172.21.15.19:0/1037871569 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ea806c530 0x7f3ea806e9f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.321+0000 7f3ece000640 1 --2- 172.21.15.19:0/1037871569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ec8139b90 0x7f3ec813a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.321+0000 7f3ece000640 1 --2- 172.21.15.19:0/1037871569 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ec80d04d0 0x7f3ec81380a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.321+0000 7f3ece000640 1 --2- 172.21.15.19:0/1037871569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec80cc0b0 0x7f3ec81378d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.321+0000 7f3ece000640 1 -- 172.21.15.19:0/1037871569 >> 172.21.15.19:0/1037871569 conn(0x7f3ec80ccad0 msgr2=0x7f3ec815ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.321+0000 7f3ece000640 1 -- 172.21.15.19:0/1037871569 shutdown_connections 2024-03-28T13:10:16.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.322+0000 7f3ece000640 1 -- 172.21.15.19:0/1037871569 wait complete. 2024-03-28T13:10:16.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:09:39.243813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:09:40.092899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:10:16.342 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:10:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.565+0000 7f13e6000640 1 Processor -- start 2024-03-28T13:10:16.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.566+0000 7f13e6000640 1 -- start start 2024-03-28T13:10:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.567+0000 7f13e6000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f13e015e4c0 0x7f13e00b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.567+0000 7f13e6000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13e00d37b0 0x7f13e00b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.567+0000 7f13e6000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13e00d2710 con 0x7f13e00b67f0 2024-03-28T13:10:16.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.567+0000 7f13e6000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f13e015e8a0 con 0x7f13e015e4c0 2024-03-28T13:10:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.567+0000 7f13e6000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f13e00d4040 con 0x7f13e00d37b0 2024-03-28T13:10:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.568+0000 7f13d7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13e00d37b0 0x7f13e00b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.568+0000 7f13e4c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f13e015e4c0 0x7f13e00b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.568+0000 7f13d7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13e00d37b0 0x7f13e00b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:46592/0 (socket says 172.21.15.19:46592) 2024-03-28T13:10:16.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.568+0000 7f13d7e00640 1 -- 172.21.15.19:0/1810554147 learned_addr learned my addr 172.21.15.19:0/1810554147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:16.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.568+0000 7f13e5600640 1 --1- 172.21.15.19:0/1810554147 >> v1:172.21.15.188:6789/0 conn(0x7f13e00b67f0 0x7f13e00d4260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:50662/0 (socket says 172.21.15.19:50662) 2024-03-28T13:10:16.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.569+0000 7f13d7e00640 1 -- 172.21.15.19:0/1810554147 >> v2:172.21.15.188:3300/0 conn(0x7f13e015e4c0 msgr2=0x7f13e00b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:16.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.569+0000 7f13d7e00640 1 --2- 172.21.15.19:0/1810554147 >> v2:172.21.15.188:3300/0 conn(0x7f13e015e4c0 0x7f13e00b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.569+0000 7f13d7e00640 1 -- 172.21.15.19:0/1810554147 >> v1:172.21.15.188:6789/0 conn(0x7f13e00b67f0 legacy=0x7f13e00d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:16.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.569+0000 7f13d7e00640 1 -- 172.21.15.19:0/1810554147 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e00d29f0 con 0x7f13e00d37b0 2024-03-28T13:10:16.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.569+0000 7f13e4c00640 1 --2- 172.21.15.19:0/1810554147 >> v2:172.21.15.188:3300/0 conn(0x7f13e015e4c0 0x7f13e00b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:10:16.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.571+0000 7f13d7e00640 1 --2- 172.21.15.19:0/1810554147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13e00d37b0 0x7f13e00b62b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009870 tx=0x7f13c802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86b85a81c79cc3aa server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:16.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.571+0000 7f13dfe00640 1 -- 172.21.15.19:0/1810554147 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8034070 con 0x7f13e00d37b0 2024-03-28T13:10:16.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.571+0000 7f13dfe00640 1 -- 172.21.15.19:0/1810554147 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c8030840 con 0x7f13e00d37b0 2024-03-28T13:10:16.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.571+0000 7f13dfe00640 1 -- 172.21.15.19:0/1810554147 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8030c10 con 0x7f13e00d37b0 2024-03-28T13:10:16.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.572+0000 7f13e6000640 1 -- 172.21.15.19:0/1810554147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13e00d37b0 msgr2=0x7f13e00b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:16.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.572+0000 7f13e6000640 1 --2- 172.21.15.19:0/1810554147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13e00d37b0 0x7f13e00b62b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009870 tx=0x7f13c802b980 comp rx=0 tx=0).stop 2024-03-28T13:10:16.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.572+0000 7f13e6000640 1 -- 172.21.15.19:0/1810554147 shutdown_connections 2024-03-28T13:10:16.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.572+0000 7f13e6000640 1 --2- 172.21.15.19:0/1810554147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13e00d37b0 0x7f13e00b62b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.572+0000 7f13e6000640 1 --2- 172.21.15.19:0/1810554147 >> v2:172.21.15.188:3300/0 conn(0x7f13e015e4c0 0x7f13e00b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.572+0000 7f13e6000640 1 -- 172.21.15.19:0/1810554147 >> 172.21.15.19:0/1810554147 conn(0x7f13e00676a0 msgr2=0x7f13e0067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:16.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.572+0000 7f13e6000640 1 -- 172.21.15.19:0/1810554147 shutdown_connections 2024-03-28T13:10:16.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.572+0000 7f13e6000640 1 -- 172.21.15.19:0/1810554147 wait complete. 2024-03-28T13:10:16.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.574+0000 7f13e6000640 1 Processor -- start 2024-03-28T13:10:16.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.575+0000 7f13e6000640 1 -- start start 2024-03-28T13:10:16.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.576+0000 7f13e6000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13e00b67f0 0x7f13e0161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:16.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.576+0000 7f13e6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e00d37b0 0x7f13e01540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:16.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.576+0000 7f13e6000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13e015e4c0 0x7f13e00cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:16.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.576+0000 7f13e6000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13e00d2fb0 con 0x7f13e00d37b0 2024-03-28T13:10:16.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.576+0000 7f13e6000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13e00d3320 con 0x7f13e00b67f0 2024-03-28T13:10:16.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.576+0000 7f13e6000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f13e00ba130 con 0x7f13e015e4c0 2024-03-28T13:10:16.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.576+0000 7f13d7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e00d37b0 0x7f13e01540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:16.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.576+0000 7f13d7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e00d37b0 0x7f13e01540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53758/0 (socket says 172.21.15.19:53758) 2024-03-28T13:10:16.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.576+0000 7f13d7e00640 1 -- 172.21.15.19:0/2708854609 learned_addr learned my addr 172.21.15.19:0/2708854609 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:16.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.576+0000 7f13e5600640 1 --2- 172.21.15.19:0/2708854609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13e015e4c0 0x7f13e00cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:16.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.576+0000 7f13e4c00640 1 --2- 172.21.15.19:0/2708854609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13e00b67f0 0x7f13e0161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:16.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.577+0000 7f13d7e00640 1 -- 172.21.15.19:0/2708854609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13e015e4c0 msgr2=0x7f13e00cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:16.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.577+0000 7f13d7e00640 1 --2- 172.21.15.19:0/2708854609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13e015e4c0 0x7f13e00cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.577+0000 7f13d7e00640 1 -- 172.21.15.19:0/2708854609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13e00b67f0 msgr2=0x7f13e0161140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:16.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.577+0000 7f13d7e00640 1 --2- 172.21.15.19:0/2708854609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13e00b67f0 0x7f13e0161140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.577+0000 7f13d7e00640 1 -- 172.21.15.19:0/2708854609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e0154630 con 0x7f13e00d37b0 2024-03-28T13:10:16.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.577+0000 7f13d7e00640 1 --2- 172.21.15.19:0/2708854609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e00d37b0 0x7f13e01540f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f13c802b9c0 tx=0x7f13c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:16.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.577+0000 7f13dd600640 1 -- 172.21.15.19:0/2708854609 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8048070 con 0x7f13e00d37b0 2024-03-28T13:10:16.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.578+0000 7f13dd600640 1 -- 172.21.15.19:0/2708854609 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c8030640 con 0x7f13e00d37b0 2024-03-28T13:10:16.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.578+0000 7f13dd600640 1 -- 172.21.15.19:0/2708854609 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8034040 con 0x7f13e00d37b0 2024-03-28T13:10:16.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.578+0000 7f13e6000640 1 -- 172.21.15.19:0/2708854609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e00cde40 con 0x7f13e00d37b0 2024-03-28T13:10:16.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.578+0000 7f13e6000640 1 -- 172.21.15.19:0/2708854609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e0160940 con 0x7f13e00d37b0 2024-03-28T13:10:16.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.579+0000 7f13e6000640 1 -- 172.21.15.19:0/2708854609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e0073830 con 0x7f13e00d37b0 2024-03-28T13:10:16.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.580+0000 7f13dd600640 1 -- 172.21.15.19:0/2708854609 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f13c8037070 con 0x7f13e00d37b0 2024-03-28T13:10:16.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.581+0000 7f13dd600640 1 --2- 172.21.15.19:0/2708854609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13bc06c560 0x7f13bc06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:16.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.581+0000 7f13dd600640 1 -- 172.21.15.19:0/2708854609 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f13c80ad3b0 con 0x7f13e00d37b0 2024-03-28T13:10:16.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.581+0000 7f13e4c00640 1 --2- 172.21.15.19:0/2708854609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13bc06c560 0x7f13bc06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:16.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.582+0000 7f13e4c00640 1 --2- 172.21.15.19:0/2708854609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13bc06c560 0x7f13bc06ea20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009a30 tx=0x7f13d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:16.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.585+0000 7f13dd600640 1 -- 172.21.15.19:0/2708854609 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f13e0073830 con 0x7f13e00d37b0 2024-03-28T13:10:16.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.788+0000 7f13e6000640 1 -- 172.21.15.19:0/2708854609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f13e0075940 con 0x7f13e00d37b0 2024-03-28T13:10:16.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.789+0000 7f13dd600640 1 -- 172.21.15.19:0/2708854609 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f13c807bb80 con 0x7f13e00d37b0 2024-03-28T13:10:16.789 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:10:16.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.792+0000 7f13e6000640 1 -- 172.21.15.19:0/2708854609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13bc06c560 msgr2=0x7f13bc06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:16.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.792+0000 7f13e6000640 1 --2- 172.21.15.19:0/2708854609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13bc06c560 0x7f13bc06ea20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009a30 tx=0x7f13d0008040 comp rx=0 tx=0).stop 2024-03-28T13:10:16.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.792+0000 7f13e6000640 1 -- 172.21.15.19:0/2708854609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e00d37b0 msgr2=0x7f13e01540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:16.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.792+0000 7f13e6000640 1 --2- 172.21.15.19:0/2708854609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e00d37b0 0x7f13e01540f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f13c802b9c0 tx=0x7f13c8004730 comp rx=0 tx=0).stop 2024-03-28T13:10:16.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.792+0000 7f13e6000640 1 -- 172.21.15.19:0/2708854609 shutdown_connections 2024-03-28T13:10:16.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.792+0000 7f13e6000640 1 --2- 172.21.15.19:0/2708854609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13bc06c560 0x7f13bc06ea20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.792+0000 7f13e6000640 1 --2- 172.21.15.19:0/2708854609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13e015e4c0 0x7f13e00cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.792+0000 7f13e6000640 1 --2- 172.21.15.19:0/2708854609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13e00d37b0 0x7f13e01540f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.792+0000 7f13e6000640 1 --2- 172.21.15.19:0/2708854609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13e00b67f0 0x7f13e0161140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:16.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.792+0000 7f13e6000640 1 -- 172.21.15.19:0/2708854609 >> 172.21.15.19:0/2708854609 conn(0x7f13e00676a0 msgr2=0x7f13e015c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:16.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.792+0000 7f13e6000640 1 -- 172.21.15.19:0/2708854609 shutdown_connections 2024-03-28T13:10:16.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:16.792+0000 7f13e6000640 1 -- 172.21.15.19:0/2708854609 wait complete. 2024-03-28T13:10:16.818 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:10:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.041+0000 7f3f6d200640 1 Processor -- start 2024-03-28T13:10:17.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.041+0000 7f3f6d200640 1 -- start start 2024-03-28T13:10:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.042+0000 7f3f6d200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6815e460 0x7f3f680b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.042+0000 7f3f6d200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3f680d3e40 0x7f3f680d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.043+0000 7f3f6d200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f680b8320 con 0x7f3f680cef20 2024-03-28T13:10:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.043+0000 7f3f6d200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3f680d2da0 con 0x7f3f680d3e40 2024-03-28T13:10:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.043+0000 7f3f6d200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3f680d3080 con 0x7f3f6815e460 2024-03-28T13:10:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.043+0000 7f3f66a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3f680d3e40 0x7f3f680d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.043+0000 7f3f66a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3f680d3e40 0x7f3f680d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53772/0 (socket says 172.21.15.19:53772) 2024-03-28T13:10:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.043+0000 7f3f66a00640 1 -- 172.21.15.19:0/816676105 learned_addr learned my addr 172.21.15.19:0/816676105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.043+0000 7f3f67400640 1 --2- 172.21.15.19:0/816676105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6815e460 0x7f3f680b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.043+0000 7f3f66a00640 1 -- 172.21.15.19:0/816676105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6815e460 msgr2=0x7f3f680b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.043+0000 7f3f66a00640 1 --2- 172.21.15.19:0/816676105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6815e460 0x7f3f680b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.043+0000 7f3f66a00640 1 -- 172.21.15.19:0/816676105 >> v1:172.21.15.188:6789/0 conn(0x7f3f680cef20 legacy=0x7f3f6806c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.043+0000 7f3f66a00640 1 -- 172.21.15.19:0/816676105 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f680d3360 con 0x7f3f680d3e40 2024-03-28T13:10:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.043+0000 7f3f67400640 1 --2- 172.21.15.19:0/816676105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6815e460 0x7f3f680b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.045+0000 7f3f66a00640 1 --2- 172.21.15.19:0/816676105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f680d3e40 0x7f3f680d2550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009850 tx=0x7f3f5002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1951b2c22042210 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.045+0000 7f3f66000640 1 -- 172.21.15.19:0/816676105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50004230 con 0x7f3f680d3e40 2024-03-28T13:10:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.045+0000 7f3f66000640 1 -- 172.21.15.19:0/816676105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f500043d0 con 0x7f3f680d3e40 2024-03-28T13:10:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.045+0000 7f3f66000640 1 -- 172.21.15.19:0/816676105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50031830 con 0x7f3f680d3e40 2024-03-28T13:10:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.046+0000 7f3f6d200640 1 -- 172.21.15.19:0/816676105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f680d3e40 msgr2=0x7f3f680d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.046+0000 7f3f6d200640 1 --2- 172.21.15.19:0/816676105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f680d3e40 0x7f3f680d2550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009850 tx=0x7f3f5002b8d0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.046+0000 7f3f6d200640 1 -- 172.21.15.19:0/816676105 shutdown_connections 2024-03-28T13:10:17.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.046+0000 7f3f6d200640 1 --2- 172.21.15.19:0/816676105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f680d3e40 0x7f3f680d2550 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.046+0000 7f3f6d200640 1 --2- 172.21.15.19:0/816676105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f6815e460 0x7f3f680b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.046+0000 7f3f6d200640 1 -- 172.21.15.19:0/816676105 >> 172.21.15.19:0/816676105 conn(0x7f3f680678b0 msgr2=0x7f3f680d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:17.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.047+0000 7f3f6d200640 1 -- 172.21.15.19:0/816676105 shutdown_connections 2024-03-28T13:10:17.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.047+0000 7f3f6d200640 1 -- 172.21.15.19:0/816676105 wait complete. 2024-03-28T13:10:17.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.050+0000 7f3f6d200640 1 Processor -- start 2024-03-28T13:10:17.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.050+0000 7f3f6d200640 1 -- start start 2024-03-28T13:10:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f6d200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f680cef20 0x7f3f681e8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f6d200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f680d3e40 0x7f3f6815b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f6d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6815e460 0x7f3f6815ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f6d200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f680d2a90 con 0x7f3f6815e460 2024-03-28T13:10:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f6d200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3f680d3920 con 0x7f3f680d3e40 2024-03-28T13:10:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f6d200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3f680cf3b0 con 0x7f3f680cef20 2024-03-28T13:10:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f67e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6815e460 0x7f3f6815ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f67e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6815e460 0x7f3f6815ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53784/0 (socket says 172.21.15.19:53784) 2024-03-28T13:10:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f67e00640 1 -- 172.21.15.19:0/2535520672 learned_addr learned my addr 172.21.15.19:0/2535520672 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f67400640 1 --2- 172.21.15.19:0/2535520672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f680cef20 0x7f3f681e8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f66a00640 1 --2- 172.21.15.19:0/2535520672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f680d3e40 0x7f3f6815b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f67e00640 1 -- 172.21.15.19:0/2535520672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f680cef20 msgr2=0x7f3f681e8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:17.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f67e00640 1 --2- 172.21.15.19:0/2535520672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f680cef20 0x7f3f681e8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f67e00640 1 -- 172.21.15.19:0/2535520672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f680d3e40 msgr2=0x7f3f6815b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f67e00640 1 --2- 172.21.15.19:0/2535520672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f680d3e40 0x7f3f6815b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.053+0000 7f3f67e00640 1 -- 172.21.15.19:0/2535520672 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f680cc400 con 0x7f3f6815e460 2024-03-28T13:10:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.054+0000 7f3f67e00640 1 --2- 172.21.15.19:0/2535520672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6815e460 0x7f3f6815ba60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000c410 tx=0x7f3f6000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:17.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.054+0000 7f3f5f400640 1 -- 172.21.15.19:0/2535520672 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60011790 con 0x7f3f6815e460 2024-03-28T13:10:17.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.054+0000 7f3f5f400640 1 -- 172.21.15.19:0/2535520672 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f60017070 con 0x7f3f6815e460 2024-03-28T13:10:17.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.054+0000 7f3f5f400640 1 -- 172.21.15.19:0/2535520672 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f60011bb0 con 0x7f3f6815e460 2024-03-28T13:10:17.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.054+0000 7f3f6d200640 1 -- 172.21.15.19:0/2535520672 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f680cc6e0 con 0x7f3f6815e460 2024-03-28T13:10:17.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.054+0000 7f3f6d200640 1 -- 172.21.15.19:0/2535520672 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f680cc0f0 con 0x7f3f6815e460 2024-03-28T13:10:17.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.055+0000 7f3f6d200640 1 -- 172.21.15.19:0/2535520672 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f68075940 con 0x7f3f6815e460 2024-03-28T13:10:17.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.056+0000 7f3f5f400640 1 -- 172.21.15.19:0/2535520672 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3f6001c460 con 0x7f3f6815e460 2024-03-28T13:10:17.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.057+0000 7f3f5f400640 1 --2- 172.21.15.19:0/2535520672 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3f4406c530 0x7f3f4406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:17.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.057+0000 7f3f5f400640 1 -- 172.21.15.19:0/2535520672 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3f6008b230 con 0x7f3f6815e460 2024-03-28T13:10:17.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.058+0000 7f3f67400640 1 --2- 172.21.15.19:0/2535520672 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3f4406c530 0x7f3f4406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:17.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.059+0000 7f3f67400640 1 --2- 172.21.15.19:0/2535520672 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3f4406c530 0x7f3f4406e9f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3f58007510 tx=0x7f3f580074a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:17.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.062+0000 7f3f5f400640 1 -- 172.21.15.19:0/2535520672 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3f60021020 con 0x7f3f6815e460 2024-03-28T13:10:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.431+0000 7f3f6d200640 1 -- 172.21.15.19:0/2535520672 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3f6815dd80 con 0x7f3f6815e460 2024-03-28T13:10:17.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.432+0000 7f3f5f400640 1 -- 172.21.15.19:0/2535520672 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f3f600598e0 con 0x7f3f6815e460 2024-03-28T13:10:17.432 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:10:17.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.435+0000 7f3f6d200640 1 -- 172.21.15.19:0/2535520672 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3f4406c530 msgr2=0x7f3f4406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:17.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.435+0000 7f3f6d200640 1 --2- 172.21.15.19:0/2535520672 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3f4406c530 0x7f3f4406e9f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3f58007510 tx=0x7f3f580074a0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.435+0000 7f3f6d200640 1 -- 172.21.15.19:0/2535520672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6815e460 msgr2=0x7f3f6815ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:17.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.435+0000 7f3f6d200640 1 --2- 172.21.15.19:0/2535520672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6815e460 0x7f3f6815ba60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f6000c410 tx=0x7f3f6000c8d0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.435+0000 7f3f6d200640 1 -- 172.21.15.19:0/2535520672 shutdown_connections 2024-03-28T13:10:17.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.435+0000 7f3f6d200640 1 --2- 172.21.15.19:0/2535520672 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3f4406c530 0x7f3f4406e9f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.435+0000 7f3f6d200640 1 --2- 172.21.15.19:0/2535520672 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f6815e460 0x7f3f6815ba60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.435+0000 7f3f6d200640 1 --2- 172.21.15.19:0/2535520672 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f680d3e40 0x7f3f6815b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.435+0000 7f3f6d200640 1 --2- 172.21.15.19:0/2535520672 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f680cef20 0x7f3f681e8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.435+0000 7f3f6d200640 1 -- 172.21.15.19:0/2535520672 >> 172.21.15.19:0/2535520672 conn(0x7f3f680678b0 msgr2=0x7f3f680b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:17.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.435+0000 7f3f6d200640 1 -- 172.21.15.19:0/2535520672 shutdown_connections 2024-03-28T13:10:17.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.435+0000 7f3f6d200640 1 -- 172.21.15.19:0/2535520672 wait complete. 2024-03-28T13:10:17.452 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:10:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.676+0000 7fad7da00640 1 Processor -- start 2024-03-28T13:10:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.676+0000 7fad7da00640 1 -- start start 2024-03-28T13:10:17.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.679+0000 7fad7da00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fad78072c80 0x7fad780b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.679+0000 7fad7da00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad780b4c70 con 0x7fad7815f6a0 2024-03-28T13:10:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.679+0000 7fad7da00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad7815fb30 con 0x7fad780b4890 2024-03-28T13:10:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.680+0000 7fad7da00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fad7815b7b0 con 0x7fad78072c80 2024-03-28T13:10:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.680+0000 7fad6f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fad7815f6a0 0x7fad780b2ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48020/0 (socket says 172.21.15.19:48020) 2024-03-28T13:10:17.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.680+0000 7fad6f400640 1 -- 172.21.15.19:0/2357167495 learned_addr learned my addr 172.21.15.19:0/2357167495 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.680+0000 7fad7d000640 1 --2- 172.21.15.19:0/2357167495 >> v2:172.21.15.188:3300/0 conn(0x7fad78072c80 0x7fad780b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.680+0000 7fad77400640 1 -- 172.21.15.19:0/2357167495 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4004824369 0 0) 0x7fad780b4c70 con 0x7fad7815f6a0 2024-03-28T13:10:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.680+0000 7fad77400640 1 -- 172.21.15.19:0/2357167495 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad5c005730 con 0x7fad7815f6a0 2024-03-28T13:10:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.681+0000 7fad77400640 1 -- 172.21.15.19:0/2357167495 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2424522917 0 0) 0x7fad5c005730 con 0x7fad7815f6a0 2024-03-28T13:10:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.681+0000 7fad77400640 1 -- 172.21.15.19:0/2357167495 >> v2:172.21.15.188:3300/0 conn(0x7fad78072c80 msgr2=0x7fad780b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.681+0000 7fad77400640 1 --2- 172.21.15.19:0/2357167495 >> v2:172.21.15.188:3300/0 conn(0x7fad78072c80 0x7fad780b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.681+0000 7fad77400640 1 -- 172.21.15.19:0/2357167495 >> v1:172.21.15.188:6789/0 conn(0x7fad780b4890 legacy=0x7fad78072570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:17.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.681+0000 7fad77400640 1 -- 172.21.15.19:0/2357167495 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7815ba90 con 0x7fad7815f6a0 2024-03-28T13:10:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.681+0000 7fad77400640 1 -- 172.21.15.19:0/2357167495 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fad60001670 con 0x7fad7815f6a0 2024-03-28T13:10:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.681+0000 7fad77400640 1 -- 172.21.15.19:0/2357167495 >> v1:172.21.15.19:6789/0 conn(0x7fad7815f6a0 legacy=0x7fad780b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.681+0000 7fad77400640 1 --2- 172.21.15.19:0/2357167495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c005c10 0x7fad5c006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.681+0000 7fad77400640 1 --2- 172.21.15.19:0/2357167495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad5c006a10 0x7fad5c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.681+0000 7fad77400640 1 --2- 172.21.15.19:0/2357167495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad5c0073d0 0x7fad5c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.681+0000 7fad77400640 1 -- 172.21.15.19:0/2357167495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad7815b7b0 con 0x7fad5c005c10 2024-03-28T13:10:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.681+0000 7fad77400640 1 -- 172.21.15.19:0/2357167495 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c00e2d0 con 0x7fad5c006a10 2024-03-28T13:10:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.681+0000 7fad77400640 1 -- 172.21.15.19:0/2357167495 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fad5c00e4d0 con 0x7fad5c0073d0 2024-03-28T13:10:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.681+0000 7fad77e00640 1 --2- 172.21.15.19:0/2357167495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c005c10 0x7fad5c006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.682+0000 7fad6f400640 1 --2- 172.21.15.19:0/2357167495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad5c006a10 0x7fad5c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.682+0000 7fad77e00640 1 -- 172.21.15.19:0/2357167495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad5c0073d0 msgr2=0x7fad5c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.682+0000 7fad77e00640 1 --2- 172.21.15.19:0/2357167495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad5c0073d0 0x7fad5c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.682+0000 7fad77e00640 1 -- 172.21.15.19:0/2357167495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad5c006a10 msgr2=0x7fad5c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.682+0000 7fad77e00640 1 --2- 172.21.15.19:0/2357167495 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad5c006a10 0x7fad5c006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.682+0000 7fad77e00640 1 -- 172.21.15.19:0/2357167495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c005730 con 0x7fad5c005c10 2024-03-28T13:10:17.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.682+0000 7fad7d000640 1 -- 172.21.15.19:0/2357167495 reap_dead start 2024-03-28T13:10:17.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.683+0000 7fad77e00640 1 --2- 172.21.15.19:0/2357167495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c005c10 0x7fad5c006030 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fad68001750 tx=0x7fad5c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3034b46eea6afa server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:17.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.683+0000 7fad77400640 1 -- 172.21.15.19:0/2357167495 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68001370 con 0x7fad5c005c10 2024-03-28T13:10:17.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.683+0000 7fad77400640 1 -- 172.21.15.19:0/2357167495 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad680052d0 con 0x7fad5c005c10 2024-03-28T13:10:17.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.683+0000 7fad77400640 1 -- 172.21.15.19:0/2357167495 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68003830 con 0x7fad5c005c10 2024-03-28T13:10:17.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.684+0000 7fad7da00640 1 -- 172.21.15.19:0/2357167495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c005c10 msgr2=0x7fad5c006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:17.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.684+0000 7fad7da00640 1 --2- 172.21.15.19:0/2357167495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c005c10 0x7fad5c006030 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fad68001750 tx=0x7fad5c007a40 comp rx=0 tx=0).stop 2024-03-28T13:10:17.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.684+0000 7fad7da00640 1 -- 172.21.15.19:0/2357167495 shutdown_connections 2024-03-28T13:10:17.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.684+0000 7fad7da00640 1 --2- 172.21.15.19:0/2357167495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad5c005c10 0x7fad5c006030 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.684+0000 7fad7da00640 1 -- 172.21.15.19:0/2357167495 >> 172.21.15.19:0/2357167495 conn(0x7fad780b1ba0 msgr2=0x7fad780b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:17.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.684+0000 7fad7da00640 1 -- 172.21.15.19:0/2357167495 shutdown_connections 2024-03-28T13:10:17.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.684+0000 7fad7da00640 1 -- 172.21.15.19:0/2357167495 wait complete. 2024-03-28T13:10:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.686+0000 7fad7da00640 1 Processor -- start 2024-03-28T13:10:17.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.686+0000 7fad7da00640 1 -- start start 2024-03-28T13:10:17.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.687+0000 7fad7da00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad5c005c10 0x7fad7814dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.687+0000 7fad7da00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad780cfe70 0x7fad780d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.687+0000 7fad7da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78157a00 0x7fad78157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.687+0000 7fad7da00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad7815c570 con 0x7fad78157a00 2024-03-28T13:10:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.687+0000 7fad7da00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fad7815c050 con 0x7fad780cfe70 2024-03-28T13:10:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.687+0000 7fad7da00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fad780b2900 con 0x7fad5c005c10 2024-03-28T13:10:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.687+0000 7fad7d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78157a00 0x7fad78157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.687+0000 7fad7d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78157a00 0x7fad78157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53798/0 (socket says 172.21.15.19:53798) 2024-03-28T13:10:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.687+0000 7fad7d000640 1 -- 172.21.15.19:0/4232864548 learned_addr learned my addr 172.21.15.19:0/4232864548 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.688+0000 7fad77e00640 1 --2- 172.21.15.19:0/4232864548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad5c005c10 0x7fad7814dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.688+0000 7fad6f400640 1 --2- 172.21.15.19:0/4232864548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad780cfe70 0x7fad780d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.688+0000 7fad7d000640 1 -- 172.21.15.19:0/4232864548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad5c005c10 msgr2=0x7fad7814dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.688+0000 7fad7d000640 1 --2- 172.21.15.19:0/4232864548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad5c005c10 0x7fad7814dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.688+0000 7fad7d000640 1 -- 172.21.15.19:0/4232864548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad780cfe70 msgr2=0x7fad780d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:17.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.688+0000 7fad7d000640 1 --2- 172.21.15.19:0/4232864548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad780cfe70 0x7fad780d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.688+0000 7fad7d000640 1 -- 172.21.15.19:0/4232864548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7814e110 con 0x7fad78157a00 2024-03-28T13:10:17.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.688+0000 7fad6f400640 1 --2- 172.21.15.19:0/4232864548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad780cfe70 0x7fad780d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:17.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.688+0000 7fad7d000640 1 --2- 172.21.15.19:0/4232864548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78157a00 0x7fad78157e80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fad78155d10 tx=0x7fad7000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:17.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.688+0000 7fad74c00640 1 -- 172.21.15.19:0/4232864548 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7000bcb0 con 0x7fad78157a00 2024-03-28T13:10:17.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.688+0000 7fad74c00640 1 -- 172.21.15.19:0/4232864548 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad7001a070 con 0x7fad78157a00 2024-03-28T13:10:17.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.688+0000 7fad74c00640 1 -- 172.21.15.19:0/4232864548 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70014640 con 0x7fad78157a00 2024-03-28T13:10:17.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.688+0000 7fad7da00640 1 -- 172.21.15.19:0/4232864548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad78155e70 con 0x7fad78157a00 2024-03-28T13:10:17.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.689+0000 7fad7da00640 1 -- 172.21.15.19:0/4232864548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad78156430 con 0x7fad78157a00 2024-03-28T13:10:17.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.690+0000 7fad7da00640 1 -- 172.21.15.19:0/4232864548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad78074d90 con 0x7fad78157a00 2024-03-28T13:10:17.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.691+0000 7fad74c00640 1 -- 172.21.15.19:0/4232864548 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fad70013070 con 0x7fad78157a00 2024-03-28T13:10:17.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.692+0000 7fad74c00640 1 --2- 172.21.15.19:0/4232864548 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fad5406c530 0x7fad5406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.692+0000 7fad74c00640 1 -- 172.21.15.19:0/4232864548 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fad7008b960 con 0x7fad78157a00 2024-03-28T13:10:17.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.692+0000 7fad77e00640 1 --2- 172.21.15.19:0/4232864548 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fad5406c530 0x7fad5406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:17.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.694+0000 7fad77e00640 1 --2- 172.21.15.19:0/4232864548 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fad5406c530 0x7fad5406e9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fad68033000 tx=0x7fad68009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:17.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.697+0000 7fad74c00640 1 -- 172.21.15.19:0/4232864548 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fad70059ee0 con 0x7fad78157a00 2024-03-28T13:10:17.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.897+0000 7fad7da00640 1 -- 172.21.15.19:0/4232864548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fad78073380 con 0x7fad78157a00 2024-03-28T13:10:17.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.898+0000 7fad74c00640 1 -- 172.21.15.19:0/4232864548 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fad700597e0 con 0x7fad78157a00 2024-03-28T13:10:17.898 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:10:17.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.900+0000 7fad7da00640 1 -- 172.21.15.19:0/4232864548 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fad5406c530 msgr2=0x7fad5406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:17.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.900+0000 7fad7da00640 1 --2- 172.21.15.19:0/4232864548 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fad5406c530 0x7fad5406e9f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fad68033000 tx=0x7fad68009030 comp rx=0 tx=0).stop 2024-03-28T13:10:17.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.900+0000 7fad7da00640 1 -- 172.21.15.19:0/4232864548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78157a00 msgr2=0x7fad78157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:17.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.900+0000 7fad7da00640 1 --2- 172.21.15.19:0/4232864548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78157a00 0x7fad78157e80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fad78155d10 tx=0x7fad7000b480 comp rx=0 tx=0).stop 2024-03-28T13:10:17.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.900+0000 7fad7da00640 1 -- 172.21.15.19:0/4232864548 shutdown_connections 2024-03-28T13:10:17.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.900+0000 7fad7da00640 1 --2- 172.21.15.19:0/4232864548 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fad5406c530 0x7fad5406e9f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.900+0000 7fad7da00640 1 --2- 172.21.15.19:0/4232864548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78157a00 0x7fad78157e80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.900+0000 7fad7da00640 1 --2- 172.21.15.19:0/4232864548 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad780cfe70 0x7fad780d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.900+0000 7fad7da00640 1 --2- 172.21.15.19:0/4232864548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad5c005c10 0x7fad7814dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:17.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.900+0000 7fad7da00640 1 -- 172.21.15.19:0/4232864548 >> 172.21.15.19:0/4232864548 conn(0x7fad780b1ba0 msgr2=0x7fad7815a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:17.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.901+0000 7fad7da00640 1 -- 172.21.15.19:0/4232864548 shutdown_connections 2024-03-28T13:10:17.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:17.901+0000 7fad7da00640 1 -- 172.21.15.19:0/4232864548 wait complete. 2024-03-28T13:10:17.918 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:10:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.122+0000 7f3a8b000640 1 Processor -- start 2024-03-28T13:10:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.123+0000 7f3a8b000640 1 -- start start 2024-03-28T13:10:18.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.124+0000 7f3a8b000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3a8415e4a0 0x7f3a840b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.124+0000 7f3a8b000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a840d2310 0x7f3a840ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.124+0000 7f3a8b000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a8415e880 con 0x7f3a840cf530 2024-03-28T13:10:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.124+0000 7f3a8b000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3a840d34e0 con 0x7f3a8415e4a0 2024-03-28T13:10:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.124+0000 7f3a8b000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3a840d28d0 con 0x7f3a840d2310 2024-03-28T13:10:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.124+0000 7f3a8a600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3a840cf530 0x7f3a840d3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48022/0 (socket says 172.21.15.19:48022) 2024-03-28T13:10:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.124+0000 7f3a8a600640 1 -- 172.21.15.19:0/2924827253 learned_addr learned my addr 172.21.15.19:0/2924827253 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.125+0000 7f3a89c00640 1 --2- 172.21.15.19:0/2924827253 >> v2:172.21.15.188:3300/0 conn(0x7f3a8415e4a0 0x7f3a840b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:18.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.125+0000 7f3a89200640 1 --2- 172.21.15.19:0/2924827253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a840d2310 0x7f3a840ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.125+0000 7f3a83e00640 1 -- 172.21.15.19:0/2924827253 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 764804249 0 0) 0x7f3a8415e880 con 0x7f3a840cf530 2024-03-28T13:10:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.125+0000 7f3a83e00640 1 -- 172.21.15.19:0/2924827253 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a6c003520 con 0x7f3a840cf530 2024-03-28T13:10:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.125+0000 7f3a83e00640 1 -- 172.21.15.19:0/2924827253 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2064939923 0 0) 0x7f3a6c003520 con 0x7f3a840cf530 2024-03-28T13:10:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.125+0000 7f3a83e00640 1 -- 172.21.15.19:0/2924827253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a840d2310 msgr2=0x7f3a840ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.125+0000 7f3a83e00640 1 --2- 172.21.15.19:0/2924827253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a840d2310 0x7f3a840ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.125+0000 7f3a83e00640 1 -- 172.21.15.19:0/2924827253 >> v2:172.21.15.188:3300/0 conn(0x7f3a8415e4a0 msgr2=0x7f3a840b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.125+0000 7f3a83e00640 1 --2- 172.21.15.19:0/2924827253 >> v2:172.21.15.188:3300/0 conn(0x7f3a8415e4a0 0x7f3a840b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.125+0000 7f3a83e00640 1 -- 172.21.15.19:0/2924827253 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a840d2bb0 con 0x7f3a840cf530 2024-03-28T13:10:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.125+0000 7f3a83e00640 1 -- 172.21.15.19:0/2924827253 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f3a7c0019f0 con 0x7f3a840cf530 2024-03-28T13:10:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.125+0000 7f3a83e00640 1 -- 172.21.15.19:0/2924827253 >> v1:172.21.15.19:6789/0 conn(0x7f3a840cf530 legacy=0x7f3a840d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.125+0000 7f3a83e00640 1 --2- 172.21.15.19:0/2924827253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a840d2310 0x7f3a6c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.125+0000 7f3a83e00640 1 --2- 172.21.15.19:0/2924827253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a6c003f80 0x7f3a6c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.126+0000 7f3a83e00640 1 --2- 172.21.15.19:0/2924827253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a6c004d40 0x7f3a6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.126+0000 7f3a83e00640 1 -- 172.21.15.19:0/2924827253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a840d34e0 con 0x7f3a6c003f80 2024-03-28T13:10:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.126+0000 7f3a83e00640 1 -- 172.21.15.19:0/2924827253 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3a840d28d0 con 0x7f3a6c004d40 2024-03-28T13:10:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.126+0000 7f3a83e00640 1 -- 172.21.15.19:0/2924827253 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6c00b9c0 con 0x7f3a840d2310 2024-03-28T13:10:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.126+0000 7f3a89200640 1 --2- 172.21.15.19:0/2924827253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a6c003f80 0x7f3a6c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.126+0000 7f3a8a600640 1 --2- 172.21.15.19:0/2924827253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a6c004d40 0x7f3a6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.126+0000 7f3a89c00640 1 --2- 172.21.15.19:0/2924827253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a840d2310 0x7f3a6c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:18.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.126+0000 7f3a89200640 1 -- 172.21.15.19:0/2924827253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a840d2310 msgr2=0x7f3a6c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.126+0000 7f3a89200640 1 --2- 172.21.15.19:0/2924827253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a840d2310 0x7f3a6c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.126+0000 7f3a89200640 1 -- 172.21.15.19:0/2924827253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a6c004d40 msgr2=0x7f3a6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.126+0000 7f3a89200640 1 --2- 172.21.15.19:0/2924827253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a6c004d40 0x7f3a6c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.126+0000 7f3a89200640 1 -- 172.21.15.19:0/2924827253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6c003520 con 0x7f3a6c003f80 2024-03-28T13:10:18.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.126+0000 7f3a8a600640 1 --2- 172.21.15.19:0/2924827253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a6c004d40 0x7f3a6c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.127+0000 7f3a89200640 1 --2- 172.21.15.19:0/2924827253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a6c003f80 0x7f3a6c004800 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009af0 tx=0x7f3a6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e860ca20b384671 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.128+0000 7f3a83e00640 1 -- 172.21.15.19:0/2924827253 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70039070 con 0x7f3a6c003f80 2024-03-28T13:10:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.128+0000 7f3a83e00640 1 -- 172.21.15.19:0/2924827253 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a7002c6b0 con 0x7f3a6c003f80 2024-03-28T13:10:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.128+0000 7f3a83e00640 1 -- 172.21.15.19:0/2924827253 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70009690 con 0x7f3a6c003f80 2024-03-28T13:10:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.128+0000 7f3a8b000640 1 -- 172.21.15.19:0/2924827253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a6c003f80 msgr2=0x7f3a6c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.128+0000 7f3a8b000640 1 --2- 172.21.15.19:0/2924827253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a6c003f80 0x7f3a6c004800 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a70009af0 tx=0x7f3a6c005830 comp rx=0 tx=0).stop 2024-03-28T13:10:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.128+0000 7f3a8a600640 1 -- 172.21.15.19:0/2924827253 reap_dead start 2024-03-28T13:10:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.128+0000 7f3a8b000640 1 -- 172.21.15.19:0/2924827253 shutdown_connections 2024-03-28T13:10:18.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.128+0000 7f3a8b000640 1 -- 172.21.15.19:0/2924827253 >> 172.21.15.19:0/2924827253 conn(0x7f3a840676d0 msgr2=0x7f3a84067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.128+0000 7f3a8b000640 1 -- 172.21.15.19:0/2924827253 shutdown_connections 2024-03-28T13:10:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.129+0000 7f3a8b000640 1 -- 172.21.15.19:0/2924827253 wait complete. 2024-03-28T13:10:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.131+0000 7f3a8b000640 1 Processor -- start 2024-03-28T13:10:18.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.132+0000 7f3a8b000640 1 -- start start 2024-03-28T13:10:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.133+0000 7f3a8b000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a6c003f80 0x7f3a84153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.133+0000 7f3a8b000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a84157890 0x7f3a84157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.133+0000 7f3a8b000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a8415e880 0x7f3a8415ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.133+0000 7f3a8b000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a840d3970 con 0x7f3a84157890 2024-03-28T13:10:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.133+0000 7f3a8b000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3a840d3170 con 0x7f3a8415e880 2024-03-28T13:10:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.133+0000 7f3a8b000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3a840ba1a0 con 0x7f3a6c003f80 2024-03-28T13:10:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.133+0000 7f3a89200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a84157890 0x7f3a84157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.134+0000 7f3a89200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a84157890 0x7f3a84157d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53814/0 (socket says 172.21.15.19:53814) 2024-03-28T13:10:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.134+0000 7f3a89200640 1 -- 172.21.15.19:0/3390107265 learned_addr learned my addr 172.21.15.19:0/3390107265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:18.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.134+0000 7f3a89c00640 1 --2- 172.21.15.19:0/3390107265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a6c003f80 0x7f3a84153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:18.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.134+0000 7f3a8a600640 1 --2- 172.21.15.19:0/3390107265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a8415e880 0x7f3a8415ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:18.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.134+0000 7f3a89200640 1 -- 172.21.15.19:0/3390107265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a6c003f80 msgr2=0x7f3a84153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:18.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.134+0000 7f3a89200640 1 --2- 172.21.15.19:0/3390107265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a6c003f80 0x7f3a84153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.134+0000 7f3a89200640 1 -- 172.21.15.19:0/3390107265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a8415e880 msgr2=0x7f3a8415ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:18.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.134+0000 7f3a89200640 1 --2- 172.21.15.19:0/3390107265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a8415e880 0x7f3a8415ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.134+0000 7f3a89200640 1 -- 172.21.15.19:0/3390107265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a84156830 con 0x7f3a84157890 2024-03-28T13:10:18.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.134+0000 7f3a89200640 1 --2- 172.21.15.19:0/3390107265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a84157890 0x7f3a84157d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a70037040 tx=0x7f3a70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:18.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.135+0000 7f3a81600640 1 -- 172.21.15.19:0/3390107265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a70039050 con 0x7f3a84157890 2024-03-28T13:10:18.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.135+0000 7f3a81600640 1 -- 172.21.15.19:0/3390107265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a70045070 con 0x7f3a84157890 2024-03-28T13:10:18.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.135+0000 7f3a8b000640 1 -- 172.21.15.19:0/3390107265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a84156ad0 con 0x7f3a84157890 2024-03-28T13:10:18.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.135+0000 7f3a81600640 1 -- 172.21.15.19:0/3390107265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a700044e0 con 0x7f3a84157890 2024-03-28T13:10:18.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.135+0000 7f3a8b000640 1 -- 172.21.15.19:0/3390107265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a84157090 con 0x7f3a84157890 2024-03-28T13:10:18.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.136+0000 7f3a8b000640 1 -- 172.21.15.19:0/3390107265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a84075940 con 0x7f3a84157890 2024-03-28T13:10:18.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.137+0000 7f3a81600640 1 -- 172.21.15.19:0/3390107265 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3a70003850 con 0x7f3a84157890 2024-03-28T13:10:18.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.138+0000 7f3a81600640 1 --2- 172.21.15.19:0/3390107265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a6406c530 0x7f3a6406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.138+0000 7f3a81600640 1 -- 172.21.15.19:0/3390107265 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3a700ad240 con 0x7f3a84157890 2024-03-28T13:10:18.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.139+0000 7f3a89c00640 1 --2- 172.21.15.19:0/3390107265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a6406c530 0x7f3a6406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:18.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.140+0000 7f3a89c00640 1 --2- 172.21.15.19:0/3390107265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a6406c530 0x7f3a6406e9f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009a90 tx=0x7f3a78008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:18.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.142+0000 7f3a81600640 1 -- 172.21.15.19:0/3390107265 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3a7004e020 con 0x7f3a84157890 2024-03-28T13:10:18.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.381+0000 7f3a8b000640 1 -- 172.21.15.19:0/3390107265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a840b57f0 con 0x7f3a84157890 2024-03-28T13:10:18.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.384+0000 7f3a81600640 1 -- 172.21.15.19:0/3390107265 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16609 (secure 0 0 0) 0x7f3a7007b8e0 con 0x7f3a84157890 2024-03-28T13:10:18.384 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:18.384 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":23,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:42.668077+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:10:18.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.386+0000 7f3a8b000640 1 -- 172.21.15.19:0/3390107265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a6406c530 msgr2=0x7f3a6406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:18.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.386+0000 7f3a8b000640 1 --2- 172.21.15.19:0/3390107265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a6406c530 0x7f3a6406e9f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3a78009a90 tx=0x7f3a78008040 comp rx=0 tx=0).stop 2024-03-28T13:10:18.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.386+0000 7f3a8b000640 1 -- 172.21.15.19:0/3390107265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a84157890 msgr2=0x7f3a84157d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:18.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.386+0000 7f3a8b000640 1 --2- 172.21.15.19:0/3390107265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a84157890 0x7f3a84157d10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3a70037040 tx=0x7f3a70004730 comp rx=0 tx=0).stop 2024-03-28T13:10:18.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.386+0000 7f3a8b000640 1 -- 172.21.15.19:0/3390107265 shutdown_connections 2024-03-28T13:10:18.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.386+0000 7f3a8b000640 1 --2- 172.21.15.19:0/3390107265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3a6406c530 0x7f3a6406e9f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.386+0000 7f3a8b000640 1 --2- 172.21.15.19:0/3390107265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a8415e880 0x7f3a8415ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.386+0000 7f3a8b000640 1 --2- 172.21.15.19:0/3390107265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a84157890 0x7f3a84157d10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.386+0000 7f3a8b000640 1 --2- 172.21.15.19:0/3390107265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a6c003f80 0x7f3a84153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.386+0000 7f3a8b000640 1 -- 172.21.15.19:0/3390107265 >> 172.21.15.19:0/3390107265 conn(0x7f3a840676d0 msgr2=0x7f3a8415b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:18.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.387+0000 7f3a8b000640 1 -- 172.21.15.19:0/3390107265 shutdown_connections 2024-03-28T13:10:18.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.387+0000 7f3a8b000640 1 -- 172.21.15.19:0/3390107265 wait complete. 2024-03-28T13:10:18.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:09:39.243813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:09:40.092899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:10:18.407 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:10:18.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.623+0000 7febd8e00640 1 Processor -- start 2024-03-28T13:10:18.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.623+0000 7febd8e00640 1 -- start start 2024-03-28T13:10:18.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.625+0000 7febd8e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7febd40d1560 0x7febd406c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.625+0000 7febd8e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febd406cfd0 con 0x7febd415e590 2024-03-28T13:10:18.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.625+0000 7febd8e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febd4071060 con 0x7febd406cbc0 2024-03-28T13:10:18.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.625+0000 7febd8e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7febd40d19d0 con 0x7febd40d1560 2024-03-28T13:10:18.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.625+0000 7febd1e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7febd415e590 0x7febd40b47d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48034/0 (socket says 172.21.15.19:48034) 2024-03-28T13:10:18.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.625+0000 7febd1e00640 1 -- 172.21.15.19:0/2347468149 learned_addr learned my addr 172.21.15.19:0/2347468149 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:18.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.625+0000 7febd0a00640 1 -- 172.21.15.19:0/2347468149 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2477291935 0 0) 0x7febc4001110 con 0x7febd415e590 2024-03-28T13:10:18.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd0a00640 1 -- 172.21.15.19:0/2347468149 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febb8003520 con 0x7febd415e590 2024-03-28T13:10:18.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd1400640 1 --2- 172.21.15.19:0/2347468149 >> v2:172.21.15.188:3300/0 conn(0x7febd40d1560 0x7febd406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:18.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd0a00640 1 -- 172.21.15.19:0/2347468149 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2043758117 0 0) 0x7febb8003520 con 0x7febd415e590 2024-03-28T13:10:18.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd0a00640 1 -- 172.21.15.19:0/2347468149 >> v2:172.21.15.188:3300/0 conn(0x7febd40d1560 msgr2=0x7febd406c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:18.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd0a00640 1 --2- 172.21.15.19:0/2347468149 >> v2:172.21.15.188:3300/0 conn(0x7febd40d1560 0x7febd406c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd0a00640 1 -- 172.21.15.19:0/2347468149 >> v1:172.21.15.188:6789/0 conn(0x7febd406cbc0 legacy=0x7febd40cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:18.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd0a00640 1 -- 172.21.15.19:0/2347468149 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd4071340 con 0x7febd415e590 2024-03-28T13:10:18.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd0a00640 1 -- 172.21.15.19:0/2347468149 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7febc4001840 con 0x7febd415e590 2024-03-28T13:10:18.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd0a00640 1 -- 172.21.15.19:0/2347468149 >> v1:172.21.15.19:6789/0 conn(0x7febd415e590 legacy=0x7febd40b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:18.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd0a00640 1 --2- 172.21.15.19:0/2347468149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80039b0 0x7febb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd0a00640 1 --2- 172.21.15.19:0/2347468149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb8004800 0x7febb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd0a00640 1 --2- 172.21.15.19:0/2347468149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb80051c0 0x7febb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd0a00640 1 -- 172.21.15.19:0/2347468149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7febb800c0e0 con 0x7febb80039b0 2024-03-28T13:10:18.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd0a00640 1 -- 172.21.15.19:0/2347468149 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7febb800c290 con 0x7febb80051c0 2024-03-28T13:10:18.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd0a00640 1 -- 172.21.15.19:0/2347468149 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7febb800c490 con 0x7febb8004800 2024-03-28T13:10:18.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.626+0000 7febd1e00640 1 --2- 172.21.15.19:0/2347468149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80039b0 0x7febb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:18.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.627+0000 7febd2800640 1 --2- 172.21.15.19:0/2347468149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb80051c0 0x7febb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:18.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.627+0000 7febd1400640 1 --2- 172.21.15.19:0/2347468149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb8004800 0x7febb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:18.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.627+0000 7febd1e00640 1 -- 172.21.15.19:0/2347468149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb8004800 msgr2=0x7febb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:18.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.627+0000 7febd1e00640 1 --2- 172.21.15.19:0/2347468149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febb8004800 0x7febb8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.627+0000 7febd1e00640 1 -- 172.21.15.19:0/2347468149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb80051c0 msgr2=0x7febb800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:18.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.627+0000 7febd1e00640 1 --2- 172.21.15.19:0/2347468149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febb80051c0 0x7febb800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.627+0000 7febd1e00640 1 -- 172.21.15.19:0/2347468149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb8003520 con 0x7febb80039b0 2024-03-28T13:10:18.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.627+0000 7febd2800640 1 -- 172.21.15.19:0/2347468149 reap_dead start 2024-03-28T13:10:18.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.628+0000 7febd1e00640 1 --2- 172.21.15.19:0/2347468149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80039b0 0x7febb8003dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7febc400c4c0 tx=0x7febb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b2624b7bdf1a3b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:18.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.628+0000 7febd0a00640 1 -- 172.21.15.19:0/2347468149 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc400c0e0 con 0x7febb80039b0 2024-03-28T13:10:18.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.628+0000 7febd0a00640 1 -- 172.21.15.19:0/2347468149 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc4030070 con 0x7febb80039b0 2024-03-28T13:10:18.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.628+0000 7febd0a00640 1 -- 172.21.15.19:0/2347468149 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc40058d0 con 0x7febb80039b0 2024-03-28T13:10:18.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.629+0000 7febd8e00640 1 -- 172.21.15.19:0/2347468149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80039b0 msgr2=0x7febb8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:18.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.629+0000 7febd8e00640 1 --2- 172.21.15.19:0/2347468149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80039b0 0x7febb8003dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7febc400c4c0 tx=0x7febb8005830 comp rx=0 tx=0).stop 2024-03-28T13:10:18.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.629+0000 7febd8e00640 1 -- 172.21.15.19:0/2347468149 shutdown_connections 2024-03-28T13:10:18.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.629+0000 7febd8e00640 1 --2- 172.21.15.19:0/2347468149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80039b0 0x7febb8003dd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.629+0000 7febd8e00640 1 -- 172.21.15.19:0/2347468149 >> 172.21.15.19:0/2347468149 conn(0x7febd40d2240 msgr2=0x7febd40d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:18.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.629+0000 7febd8e00640 1 -- 172.21.15.19:0/2347468149 shutdown_connections 2024-03-28T13:10:18.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.629+0000 7febd8e00640 1 -- 172.21.15.19:0/2347468149 wait complete. 2024-03-28T13:10:18.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.632+0000 7febd8e00640 1 Processor -- start 2024-03-28T13:10:18.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.632+0000 7febd8e00640 1 -- start start 2024-03-28T13:10:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.634+0000 7febd8e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80039b0 0x7febd40cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.634+0000 7febd8e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febd41f5220 0x7febd4156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.634+0000 7febd8e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febd4157510 0x7febd415ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.634+0000 7febd8e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7febd4071be0 con 0x7febb80039b0 2024-03-28T13:10:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.634+0000 7febd8e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7febd4071900 con 0x7febd41f5220 2024-03-28T13:10:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.634+0000 7febd8e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7febd415e970 con 0x7febd4157510 2024-03-28T13:10:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.634+0000 7febd1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80039b0 0x7febd40cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.634+0000 7febd1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80039b0 0x7febd40cac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53826/0 (socket says 172.21.15.19:53826) 2024-03-28T13:10:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.634+0000 7febd1e00640 1 -- 172.21.15.19:0/1098803067 learned_addr learned my addr 172.21.15.19:0/1098803067 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.634+0000 7febd1e00640 1 -- 172.21.15.19:0/1098803067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febd4157510 msgr2=0x7febd415ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.634+0000 7febd1e00640 1 --2- 172.21.15.19:0/1098803067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febd4157510 0x7febd415ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.634+0000 7febd1e00640 1 -- 172.21.15.19:0/1098803067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febd41f5220 msgr2=0x7febd4156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:18.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.634+0000 7febd1e00640 1 --2- 172.21.15.19:0/1098803067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febd41f5220 0x7febd4156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.634+0000 7febd1e00640 1 -- 172.21.15.19:0/1098803067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd41f8cd0 con 0x7febb80039b0 2024-03-28T13:10:18.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.635+0000 7febd1e00640 1 --2- 172.21.15.19:0/1098803067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80039b0 0x7febd40cac80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7febc4001b40 tx=0x7febc402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:18.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.635+0000 7febca000640 1 -- 172.21.15.19:0/1098803067 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4004bf0 con 0x7febb80039b0 2024-03-28T13:10:18.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.635+0000 7febca000640 1 -- 172.21.15.19:0/1098803067 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febc4004d90 con 0x7febb80039b0 2024-03-28T13:10:18.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.635+0000 7febca000640 1 -- 172.21.15.19:0/1098803067 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc4007270 con 0x7febb80039b0 2024-03-28T13:10:18.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.635+0000 7febd8e00640 1 -- 172.21.15.19:0/1098803067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd41f80c0 con 0x7febb80039b0 2024-03-28T13:10:18.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.635+0000 7febd8e00640 1 -- 172.21.15.19:0/1098803067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd41f8680 con 0x7febb80039b0 2024-03-28T13:10:18.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.636+0000 7febd8e00640 1 -- 172.21.15.19:0/1098803067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd4157930 con 0x7febb80039b0 2024-03-28T13:10:18.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.637+0000 7febca000640 1 -- 172.21.15.19:0/1098803067 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7febc4007410 con 0x7febb80039b0 2024-03-28T13:10:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.638+0000 7febca000640 1 --2- 172.21.15.19:0/1098803067 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7febb406c530 0x7febb406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.638+0000 7febca000640 1 -- 172.21.15.19:0/1098803067 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7febc40ad000 con 0x7febb80039b0 2024-03-28T13:10:18.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.639+0000 7febd1400640 1 --2- 172.21.15.19:0/1098803067 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7febb406c530 0x7febb406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:18.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.640+0000 7febd1400640 1 --2- 172.21.15.19:0/1098803067 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7febb406c530 0x7febb406e9f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7febc0006d40 tx=0x7febc0006bd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:18.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.643+0000 7febca000640 1 -- 172.21.15.19:0/1098803067 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7febd406cfd0 con 0x7febb80039b0 2024-03-28T13:10:18.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.846+0000 7febd8e00640 1 -- 172.21.15.19:0/1098803067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7febd40685c0 con 0x7febb80039b0 2024-03-28T13:10:18.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.847+0000 7febca000640 1 -- 172.21.15.19:0/1098803067 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7febc4033370 con 0x7febb80039b0 2024-03-28T13:10:18.847 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:10:18.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.849+0000 7febd8e00640 1 -- 172.21.15.19:0/1098803067 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7febb406c530 msgr2=0x7febb406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:18.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.849+0000 7febd8e00640 1 --2- 172.21.15.19:0/1098803067 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7febb406c530 0x7febb406e9f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7febc0006d40 tx=0x7febc0006bd0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.849+0000 7febd8e00640 1 -- 172.21.15.19:0/1098803067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80039b0 msgr2=0x7febd40cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:18.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.849+0000 7febd8e00640 1 --2- 172.21.15.19:0/1098803067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80039b0 0x7febd40cac80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7febc4001b40 tx=0x7febc402e070 comp rx=0 tx=0).stop 2024-03-28T13:10:18.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.849+0000 7febd8e00640 1 -- 172.21.15.19:0/1098803067 shutdown_connections 2024-03-28T13:10:18.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.850+0000 7febd8e00640 1 --2- 172.21.15.19:0/1098803067 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7febb406c530 0x7febb406e9f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.850+0000 7febd8e00640 1 --2- 172.21.15.19:0/1098803067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febd4157510 0x7febd415ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.850+0000 7febd8e00640 1 --2- 172.21.15.19:0/1098803067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febd41f5220 0x7febd4156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.850+0000 7febd8e00640 1 --2- 172.21.15.19:0/1098803067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febb80039b0 0x7febd40cac80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:18.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.850+0000 7febd8e00640 1 -- 172.21.15.19:0/1098803067 >> 172.21.15.19:0/1098803067 conn(0x7febd40d2240 msgr2=0x7febd40cd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:18.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.850+0000 7febd8e00640 1 -- 172.21.15.19:0/1098803067 shutdown_connections 2024-03-28T13:10:18.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:18.850+0000 7febd8e00640 1 -- 172.21.15.19:0/1098803067 wait complete. 2024-03-28T13:10:18.868 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:10:19.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.081+0000 7fa0e3e00640 1 Processor -- start 2024-03-28T13:10:19.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.081+0000 7fa0e3e00640 1 -- start start 2024-03-28T13:10:19.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.082+0000 7fa0e3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e415e460 0x7fa0e40b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:19.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.082+0000 7fa0e3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0e40d3e40 0x7fa0e40d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:19.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.082+0000 7fa0e3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0e40b8320 con 0x7fa0e40cef20 2024-03-28T13:10:19.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.082+0000 7fa0e3e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e40d2da0 con 0x7fa0e40d3e40 2024-03-28T13:10:19.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.082+0000 7fa0e3e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e40d3080 con 0x7fa0e415e460 2024-03-28T13:10:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.083+0000 7fa0dbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0e40d3e40 0x7fa0e40d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.083+0000 7fa0e3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa0e40cef20 0x7fa0e406c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48050/0 (socket says 172.21.15.19:48050) 2024-03-28T13:10:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.083+0000 7fa0e3400640 1 -- 172.21.15.19:0/2196404254 learned_addr learned my addr 172.21.15.19:0/2196404254 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.083+0000 7fa0e2a00640 1 --2- 172.21.15.19:0/2196404254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e415e460 0x7fa0e40b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.083+0000 7fa0dbe00640 1 -- 172.21.15.19:0/2196404254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e415e460 msgr2=0x7fa0e40b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.084+0000 7fa0dbe00640 1 --2- 172.21.15.19:0/2196404254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e415e460 0x7fa0e40b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.084+0000 7fa0dbe00640 1 -- 172.21.15.19:0/2196404254 >> v1:172.21.15.19:6789/0 conn(0x7fa0e40cef20 legacy=0x7fa0e406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.084+0000 7fa0dbe00640 1 -- 172.21.15.19:0/2196404254 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e40d3360 con 0x7fa0e40d3e40 2024-03-28T13:10:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.084+0000 7fa0e2000640 1 -- 172.21.15.19:0/2196404254 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 612768956 0 0) 0x7fa0e40b8320 con 0x7fa0e40cef20 2024-03-28T13:10:19.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.084+0000 7fa0e2a00640 1 --2- 172.21.15.19:0/2196404254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e415e460 0x7fa0e40b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:10:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.085+0000 7fa0dbe00640 1 --2- 172.21.15.19:0/2196404254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e40d3e40 0x7fa0e40d2550 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009790 tx=0x7fa0cc02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7995a268b7992bd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.085+0000 7fa0e2000640 1 -- 172.21.15.19:0/2196404254 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc035020 con 0x7fa0e40d3e40 2024-03-28T13:10:19.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.085+0000 7fa0e2000640 1 -- 172.21.15.19:0/2196404254 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0cc031800 con 0x7fa0e40d3e40 2024-03-28T13:10:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.085+0000 7fa0e2000640 1 -- 172.21.15.19:0/2196404254 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc031bd0 con 0x7fa0e40d3e40 2024-03-28T13:10:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.086+0000 7fa0e3e00640 1 -- 172.21.15.19:0/2196404254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e40d3e40 msgr2=0x7fa0e40d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.086+0000 7fa0e3e00640 1 --2- 172.21.15.19:0/2196404254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e40d3e40 0x7fa0e40d2550 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc009790 tx=0x7fa0cc02b8a0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.086+0000 7fa0e3e00640 1 -- 172.21.15.19:0/2196404254 shutdown_connections 2024-03-28T13:10:19.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.086+0000 7fa0e3e00640 1 --2- 172.21.15.19:0/2196404254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e40d3e40 0x7fa0e40d2550 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.086+0000 7fa0e3e00640 1 --2- 172.21.15.19:0/2196404254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e415e460 0x7fa0e40b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.086+0000 7fa0e3e00640 1 -- 172.21.15.19:0/2196404254 >> 172.21.15.19:0/2196404254 conn(0x7fa0e40678b0 msgr2=0x7fa0e40d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.086+0000 7fa0e3e00640 1 -- 172.21.15.19:0/2196404254 shutdown_connections 2024-03-28T13:10:19.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.086+0000 7fa0e3e00640 1 -- 172.21.15.19:0/2196404254 wait complete. 2024-03-28T13:10:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.088+0000 7fa0e3e00640 1 Processor -- start 2024-03-28T13:10:19.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.088+0000 7fa0e3e00640 1 -- start start 2024-03-28T13:10:19.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.089+0000 7fa0e3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e40cef20 0x7fa0e414e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:19.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.089+0000 7fa0e3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e40d3e40 0x7fa0e40cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:19.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.089+0000 7fa0e3e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e415e460 0x7fa0e41f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:19.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.089+0000 7fa0e3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e40d2a90 con 0x7fa0e40cef20 2024-03-28T13:10:19.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.089+0000 7fa0e3e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e40d3920 con 0x7fa0e415e460 2024-03-28T13:10:19.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.089+0000 7fa0e3e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e40cf3b0 con 0x7fa0e40d3e40 2024-03-28T13:10:19.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.089+0000 7fa0e2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e40cef20 0x7fa0e414e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:19.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.090+0000 7fa0e2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e40cef20 0x7fa0e414e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53840/0 (socket says 172.21.15.19:53840) 2024-03-28T13:10:19.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.090+0000 7fa0e2a00640 1 -- 172.21.15.19:0/1080186336 learned_addr learned my addr 172.21.15.19:0/1080186336 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.090+0000 7fa0e2a00640 1 -- 172.21.15.19:0/1080186336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e40d3e40 msgr2=0x7fa0e40cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.090+0000 7fa0dbe00640 1 --2- 172.21.15.19:0/1080186336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e40d3e40 0x7fa0e40cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.090+0000 7fa0e2a00640 1 --2- 172.21.15.19:0/1080186336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e40d3e40 0x7fa0e40cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.090+0000 7fa0e2a00640 1 -- 172.21.15.19:0/1080186336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e415e460 msgr2=0x7fa0e41f40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.090+0000 7fa0e2a00640 1 --2- 172.21.15.19:0/1080186336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e415e460 0x7fa0e41f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.090+0000 7fa0e2a00640 1 -- 172.21.15.19:0/1080186336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e414ed00 con 0x7fa0e40cef20 2024-03-28T13:10:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.090+0000 7fa0e2a00640 1 --2- 172.21.15.19:0/1080186336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e40cef20 0x7fa0e414e420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400d800 tx=0x7fa0d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.091+0000 7fa0daa00640 1 -- 172.21.15.19:0/1080186336 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4004030 con 0x7fa0e40cef20 2024-03-28T13:10:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.091+0000 7fa0daa00640 1 -- 172.21.15.19:0/1080186336 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d40041d0 con 0x7fa0e40cef20 2024-03-28T13:10:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.091+0000 7fa0daa00640 1 -- 172.21.15.19:0/1080186336 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d400b6c0 con 0x7fa0e40cef20 2024-03-28T13:10:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.091+0000 7fa0e3e00640 1 -- 172.21.15.19:0/1080186336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e414efe0 con 0x7fa0e40cef20 2024-03-28T13:10:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.091+0000 7fa0e3e00640 1 -- 172.21.15.19:0/1080186336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e414f5a0 con 0x7fa0e40cef20 2024-03-28T13:10:19.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.092+0000 7fa0e3e00640 1 -- 172.21.15.19:0/1080186336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e4075940 con 0x7fa0e40cef20 2024-03-28T13:10:19.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.093+0000 7fa0daa00640 1 -- 172.21.15.19:0/1080186336 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa0d400b860 con 0x7fa0e40cef20 2024-03-28T13:10:19.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.094+0000 7fa0daa00640 1 --2- 172.21.15.19:0/1080186336 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c530 0x7fa0c006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:19.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.094+0000 7fa0daa00640 1 -- 172.21.15.19:0/1080186336 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa0d408ba00 con 0x7fa0e40cef20 2024-03-28T13:10:19.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.095+0000 7fa0dbe00640 1 --2- 172.21.15.19:0/1080186336 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c530 0x7fa0c006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:19.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.096+0000 7fa0dbe00640 1 --2- 172.21.15.19:0/1080186336 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c530 0x7fa0c006e9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc036000 tx=0x7fa0cc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:19.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.099+0000 7fa0daa00640 1 -- 172.21.15.19:0/1080186336 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa0d405a010 con 0x7fa0e40cef20 2024-03-28T13:10:19.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.359+0000 7fa0e3e00640 1 -- 172.21.15.19:0/1080186336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa0e414f9a0 con 0x7fa0e40cef20 2024-03-28T13:10:19.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.359+0000 7fa0daa00640 1 -- 172.21.15.19:0/1080186336 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fa0d4059910 con 0x7fa0e40cef20 2024-03-28T13:10:19.360 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:10:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.362+0000 7fa0e3e00640 1 -- 172.21.15.19:0/1080186336 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c530 msgr2=0x7fa0c006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.362+0000 7fa0e3e00640 1 --2- 172.21.15.19:0/1080186336 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c530 0x7fa0c006e9f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc036000 tx=0x7fa0cc008030 comp rx=0 tx=0).stop 2024-03-28T13:10:19.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.362+0000 7fa0e3e00640 1 -- 172.21.15.19:0/1080186336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e40cef20 msgr2=0x7fa0e414e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.362+0000 7fa0e3e00640 1 --2- 172.21.15.19:0/1080186336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e40cef20 0x7fa0e414e420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400d800 tx=0x7fa0d400dcc0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.362+0000 7fa0e3e00640 1 -- 172.21.15.19:0/1080186336 shutdown_connections 2024-03-28T13:10:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.362+0000 7fa0e3e00640 1 --2- 172.21.15.19:0/1080186336 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c530 0x7fa0c006e9f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.362+0000 7fa0e3e00640 1 --2- 172.21.15.19:0/1080186336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e415e460 0x7fa0e41f40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.362+0000 7fa0e3e00640 1 --2- 172.21.15.19:0/1080186336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e40d3e40 0x7fa0e40cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.362+0000 7fa0e3e00640 1 --2- 172.21.15.19:0/1080186336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e40cef20 0x7fa0e414e420 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.362+0000 7fa0e3e00640 1 -- 172.21.15.19:0/1080186336 >> 172.21.15.19:0/1080186336 conn(0x7fa0e40678b0 msgr2=0x7fa0e40b7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.362+0000 7fa0e3e00640 1 -- 172.21.15.19:0/1080186336 shutdown_connections 2024-03-28T13:10:19.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.362+0000 7fa0e3e00640 1 -- 172.21.15.19:0/1080186336 wait complete. 2024-03-28T13:10:19.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:10:19.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.596+0000 7f0c52400640 1 Processor -- start 2024-03-28T13:10:19.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.596+0000 7f0c52400640 1 -- start start 2024-03-28T13:10:19.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.597+0000 7f0c52400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c4c0cf630 0x7f0c4c0b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:19.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.598+0000 7f0c52400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c4c0b8320 con 0x7f0c4c15e460 2024-03-28T13:10:19.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.598+0000 7f0c52400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c4c0d2da0 con 0x7f0c4c0d3e40 2024-03-28T13:10:19.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.598+0000 7f0c52400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0c4c0d3080 con 0x7f0c4c0cf630 2024-03-28T13:10:19.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.598+0000 7f0c51000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0c4c15e460 0x7f0c4c0d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48058/0 (socket says 172.21.15.19:48058) 2024-03-28T13:10:19.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.598+0000 7f0c51000640 1 -- 172.21.15.19:0/719132424 learned_addr learned my addr 172.21.15.19:0/719132424 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:19.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.598+0000 7f0c51a00640 1 --2- 172.21.15.19:0/719132424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c4c0cf630 0x7f0c4c0b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:19.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 291590386 0 0) 0x7f0c4c0b8320 con 0x7f0c4c15e460 2024-03-28T13:10:19.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c34003520 con 0x7f0c4c15e460 2024-03-28T13:10:19.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3167250499 0 0) 0x7f0c4c0d2da0 con 0x7f0c4c0d3e40 2024-03-28T13:10:19.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c4c0b8320 con 0x7f0c4c0d3e40 2024-03-28T13:10:19.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1378916889 0 0) 0x7f0c34003520 con 0x7f0c4c15e460 2024-03-28T13:10:19.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c4c0cf630 msgr2=0x7f0c4c0b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:19.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 --2- 172.21.15.19:0/719132424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c4c0cf630 0x7f0c4c0b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 >> v1:172.21.15.188:6789/0 conn(0x7f0c4c0d3e40 legacy=0x7f0c4c0cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:19.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4c0d3360 con 0x7f0c4c15e460 2024-03-28T13:10:19.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f0c40001670 con 0x7f0c4c15e460 2024-03-28T13:10:19.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 >> v1:172.21.15.19:6789/0 conn(0x7f0c4c15e460 legacy=0x7f0c4c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:19.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 --2- 172.21.15.19:0/719132424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c34003b70 0x7f0c34003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:19.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 --2- 172.21.15.19:0/719132424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c4c0cf630 0x7f0c34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:19.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 --2- 172.21.15.19:0/719132424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c34004d40 0x7f0c340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:19.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c0d3080 con 0x7f0c34003b70 2024-03-28T13:10:19.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3400baa0 con 0x7f0c34004d40 2024-03-28T13:10:19.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.599+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3400bca0 con 0x7f0c4c0cf630 2024-03-28T13:10:19.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.600+0000 7f0c51000640 1 --2- 172.21.15.19:0/719132424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c34003b70 0x7f0c34003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:19.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.600+0000 7f0c4be00640 1 --2- 172.21.15.19:0/719132424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c4c0cf630 0x7f0c34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:19.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.600+0000 7f0c51a00640 1 --2- 172.21.15.19:0/719132424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c34004d40 0x7f0c340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:19.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.600+0000 7f0c51000640 1 -- 172.21.15.19:0/719132424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c4c0cf630 msgr2=0x7f0c34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:19.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.600+0000 7f0c51000640 1 --2- 172.21.15.19:0/719132424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c4c0cf630 0x7f0c34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.600+0000 7f0c51000640 1 -- 172.21.15.19:0/719132424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c34004d40 msgr2=0x7f0c340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:19.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.600+0000 7f0c51000640 1 --2- 172.21.15.19:0/719132424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c34004d40 0x7f0c340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.600+0000 7f0c51000640 1 -- 172.21.15.19:0/719132424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c34003520 con 0x7f0c34003b70 2024-03-28T13:10:19.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.600+0000 7f0c51a00640 1 --2- 172.21.15.19:0/719132424 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c34004d40 0x7f0c340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.602+0000 7f0c51000640 1 --2- 172.21.15.19:0/719132424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c34003b70 0x7f0c34003f70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000c0e0 tx=0x7f0c34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13b6fbaa55d7e65 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.602+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c40007c00 con 0x7f0c34003b70 2024-03-28T13:10:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.602+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c40030070 con 0x7f0c34003b70 2024-03-28T13:10:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.602+0000 7f0c4b400640 1 -- 172.21.15.19:0/719132424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c40005d10 con 0x7f0c34003b70 2024-03-28T13:10:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.603+0000 7f0c52400640 1 -- 172.21.15.19:0/719132424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c34003b70 msgr2=0x7f0c34003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.603+0000 7f0c52400640 1 --2- 172.21.15.19:0/719132424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c34003b70 0x7f0c34003f70 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0c4000c0e0 tx=0x7f0c34005830 comp rx=0 tx=0).stop 2024-03-28T13:10:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.603+0000 7f0c51a00640 1 -- 172.21.15.19:0/719132424 reap_dead start 2024-03-28T13:10:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.603+0000 7f0c52400640 1 -- 172.21.15.19:0/719132424 shutdown_connections 2024-03-28T13:10:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.603+0000 7f0c52400640 1 -- 172.21.15.19:0/719132424 >> 172.21.15.19:0/719132424 conn(0x7f0c4c0678b0 msgr2=0x7f0c4c0d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.603+0000 7f0c52400640 1 -- 172.21.15.19:0/719132424 shutdown_connections 2024-03-28T13:10:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.603+0000 7f0c52400640 1 -- 172.21.15.19:0/719132424 wait complete. 2024-03-28T13:10:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.605+0000 7f0c52400640 1 Processor -- start 2024-03-28T13:10:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.605+0000 7f0c52400640 1 -- start start 2024-03-28T13:10:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.606+0000 7f0c52400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c34003b70 0x7f0c4c0cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.606+0000 7f0c52400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c4c157280 0x7f0c4c153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.606+0000 7f0c52400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c4c0c9f50 0x7f0c4c0ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.607+0000 7f0c52400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c0b9e20 con 0x7f0c4c157280 2024-03-28T13:10:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.607+0000 7f0c52400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c0d3920 con 0x7f0c4c0c9f50 2024-03-28T13:10:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.607+0000 7f0c52400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0c4c0b9fa0 con 0x7f0c34003b70 2024-03-28T13:10:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.607+0000 7f0c4be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c4c157280 0x7f0c4c153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.607+0000 7f0c4be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c4c157280 0x7f0c4c153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53860/0 (socket says 172.21.15.19:53860) 2024-03-28T13:10:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.607+0000 7f0c4be00640 1 -- 172.21.15.19:0/818968716 learned_addr learned my addr 172.21.15.19:0/818968716 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.607+0000 7f0c51000640 1 --2- 172.21.15.19:0/818968716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c34003b70 0x7f0c4c0cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.607+0000 7f0c4be00640 1 -- 172.21.15.19:0/818968716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c34003b70 msgr2=0x7f0c4c0cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.607+0000 7f0c4be00640 1 --2- 172.21.15.19:0/818968716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c34003b70 0x7f0c4c0cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.607+0000 7f0c4be00640 1 -- 172.21.15.19:0/818968716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c4c0c9f50 msgr2=0x7f0c4c0ca3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.607+0000 7f0c51a00640 1 --2- 172.21.15.19:0/818968716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c4c0c9f50 0x7f0c4c0ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.608+0000 7f0c4be00640 1 --2- 172.21.15.19:0/818968716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c4c0c9f50 0x7f0c4c0ca3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.608+0000 7f0c4be00640 1 -- 172.21.15.19:0/818968716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4c157700 con 0x7f0c4c157280 2024-03-28T13:10:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.608+0000 7f0c4be00640 1 --2- 172.21.15.19:0/818968716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c4c157280 0x7f0c4c153270 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c3800e5b0 tx=0x7f0c3800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.608+0000 7f0c48c00640 1 -- 172.21.15.19:0/818968716 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c380052e0 con 0x7f0c4c157280 2024-03-28T13:10:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.608+0000 7f0c48c00640 1 -- 172.21.15.19:0/818968716 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c38018070 con 0x7f0c4c157280 2024-03-28T13:10:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.608+0000 7f0c52400640 1 -- 172.21.15.19:0/818968716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4c0cdb60 con 0x7f0c4c157280 2024-03-28T13:10:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.608+0000 7f0c48c00640 1 -- 172.21.15.19:0/818968716 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38003880 con 0x7f0c4c157280 2024-03-28T13:10:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.608+0000 7f0c52400640 1 -- 172.21.15.19:0/818968716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4c0ce120 con 0x7f0c4c157280 2024-03-28T13:10:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.610+0000 7f0c52400640 1 -- 172.21.15.19:0/818968716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4c075940 con 0x7f0c4c157280 2024-03-28T13:10:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.611+0000 7f0c48c00640 1 -- 172.21.15.19:0/818968716 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0c38003a20 con 0x7f0c4c157280 2024-03-28T13:10:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.612+0000 7f0c48c00640 1 --2- 172.21.15.19:0/818968716 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c2c06c530 0x7f0c2c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.612+0000 7f0c48c00640 1 -- 172.21.15.19:0/818968716 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0c3808b500 con 0x7f0c4c157280 2024-03-28T13:10:19.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.613+0000 7f0c51000640 1 --2- 172.21.15.19:0/818968716 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c2c06c530 0x7f0c2c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:19.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.614+0000 7f0c51000640 1 --2- 172.21.15.19:0/818968716 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c2c06c530 0x7f0c2c06e9f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0c40034000 tx=0x7f0c40009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:19.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.616+0000 7f0c48c00640 1 -- 172.21.15.19:0/818968716 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0c38059bd0 con 0x7f0c4c157280 2024-03-28T13:10:19.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.938+0000 7f0c52400640 1 -- 172.21.15.19:0/818968716 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0c4c0b3810 con 0x7f0c4c157280 2024-03-28T13:10:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.939+0000 7f0c48c00640 1 -- 172.21.15.19:0/818968716 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f0c380594d0 con 0x7f0c4c157280 2024-03-28T13:10:19.939 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:10:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.941+0000 7f0c52400640 1 -- 172.21.15.19:0/818968716 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c2c06c530 msgr2=0x7f0c2c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.941+0000 7f0c52400640 1 --2- 172.21.15.19:0/818968716 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c2c06c530 0x7f0c2c06e9f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0c40034000 tx=0x7f0c40009030 comp rx=0 tx=0).stop 2024-03-28T13:10:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.941+0000 7f0c52400640 1 -- 172.21.15.19:0/818968716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c4c157280 msgr2=0x7f0c4c153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.941+0000 7f0c52400640 1 --2- 172.21.15.19:0/818968716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c4c157280 0x7f0c4c153270 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c3800e5b0 tx=0x7f0c3800ea70 comp rx=0 tx=0).stop 2024-03-28T13:10:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.941+0000 7f0c52400640 1 -- 172.21.15.19:0/818968716 shutdown_connections 2024-03-28T13:10:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.941+0000 7f0c52400640 1 --2- 172.21.15.19:0/818968716 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c2c06c530 0x7f0c2c06e9f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.942+0000 7f0c52400640 1 --2- 172.21.15.19:0/818968716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c4c0c9f50 0x7f0c4c0ca3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.942+0000 7f0c52400640 1 --2- 172.21.15.19:0/818968716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c4c157280 0x7f0c4c153270 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.942+0000 7f0c52400640 1 --2- 172.21.15.19:0/818968716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c34003b70 0x7f0c4c0cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.942+0000 7f0c52400640 1 -- 172.21.15.19:0/818968716 >> 172.21.15.19:0/818968716 conn(0x7f0c4c0678b0 msgr2=0x7f0c4c0d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.942+0000 7f0c52400640 1 -- 172.21.15.19:0/818968716 shutdown_connections 2024-03-28T13:10:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:19.942+0000 7f0c52400640 1 -- 172.21.15.19:0/818968716 wait complete. 2024-03-28T13:10:19.960 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:10:20.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.187+0000 7f6fae600640 1 Processor -- start 2024-03-28T13:10:20.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.187+0000 7f6fae600640 1 -- start start 2024-03-28T13:10:20.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.188+0000 7f6fae600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa815e6e0 0x7f6fa80d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:20.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.189+0000 7f6fae600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fa80d08c0 con 0x7f6fa806b570 2024-03-28T13:10:20.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.189+0000 7f6fae600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fa80d0ba0 con 0x7f6fa80b0580 2024-03-28T13:10:20.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.189+0000 7f6fae600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa8162f90 con 0x7f6fa815e6e0 2024-03-28T13:10:20.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.189+0000 7f6fadc00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6fa806b570 0x7f6fa815eac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48066/0 (socket says 172.21.15.19:48066) 2024-03-28T13:10:20.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.189+0000 7f6fadc00640 1 -- 172.21.15.19:0/1795127334 learned_addr learned my addr 172.21.15.19:0/1795127334 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:20.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.189+0000 7f6fad200640 1 --2- 172.21.15.19:0/1795127334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa815e6e0 0x7f6fa80d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:20.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.190+0000 7f6fa7400640 1 -- 172.21.15.19:0/1795127334 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 158353905 0 0) 0x7f6fa80d08c0 con 0x7f6fa806b570 2024-03-28T13:10:20.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.190+0000 7f6fa7400640 1 -- 172.21.15.19:0/1795127334 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f90003520 con 0x7f6fa806b570 2024-03-28T13:10:20.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.190+0000 7f6fa7400640 1 -- 172.21.15.19:0/1795127334 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1841302797 0 0) 0x7f6f90003520 con 0x7f6fa806b570 2024-03-28T13:10:20.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.190+0000 7f6fa7400640 1 -- 172.21.15.19:0/1795127334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa815e6e0 msgr2=0x7f6fa80d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.190+0000 7f6fa7400640 1 --2- 172.21.15.19:0/1795127334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa815e6e0 0x7f6fa80d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.190+0000 7f6fa7400640 1 -- 172.21.15.19:0/1795127334 >> v1:172.21.15.188:6789/0 conn(0x7f6fa80b0580 legacy=0x7f6fa8162880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.190+0000 7f6fa7400640 1 -- 172.21.15.19:0/1795127334 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa80d0e80 con 0x7f6fa806b570 2024-03-28T13:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.190+0000 7f6fa7400640 1 -- 172.21.15.19:0/1795127334 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f6fa00019f0 con 0x7f6fa806b570 2024-03-28T13:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.190+0000 7f6fa7400640 1 -- 172.21.15.19:0/1795127334 >> v1:172.21.15.19:6789/0 conn(0x7f6fa806b570 legacy=0x7f6fa815eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.190+0000 7f6fa7400640 1 --2- 172.21.15.19:0/1795127334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f900039e0 0x7f6f90003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.190+0000 7f6fa7400640 1 --2- 172.21.15.19:0/1795127334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa815e6e0 0x7f6f90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.190+0000 7f6fa7400640 1 --2- 172.21.15.19:0/1795127334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f90004d40 0x7f6f900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.190+0000 7f6fa7400640 1 -- 172.21.15.19:0/1795127334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa8162f90 con 0x7f6f90004d40 2024-03-28T13:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.190+0000 7f6fa7400640 1 -- 172.21.15.19:0/1795127334 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9000ba70 con 0x7f6f900039e0 2024-03-28T13:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.190+0000 7f6fa7400640 1 -- 172.21.15.19:0/1795127334 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6f9000bc20 con 0x7f6fa815e6e0 2024-03-28T13:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.191+0000 7f6fadc00640 1 --2- 172.21.15.19:0/1795127334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f90004d40 0x7f6f900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.191+0000 7f6fad200640 1 --2- 172.21.15.19:0/1795127334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f900039e0 0x7f6f90003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:20.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.191+0000 7f6fa7e00640 1 --2- 172.21.15.19:0/1795127334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa815e6e0 0x7f6f90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.191+0000 7f6fadc00640 1 -- 172.21.15.19:0/1795127334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa815e6e0 msgr2=0x7f6f90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.191+0000 7f6fadc00640 1 --2- 172.21.15.19:0/1795127334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa815e6e0 0x7f6f90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.191+0000 7f6fadc00640 1 -- 172.21.15.19:0/1795127334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f900039e0 msgr2=0x7f6f90003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.191+0000 7f6fadc00640 1 --2- 172.21.15.19:0/1795127334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f900039e0 0x7f6f90003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.191+0000 7f6fadc00640 1 -- 172.21.15.19:0/1795127334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f90003520 con 0x7f6f90004d40 2024-03-28T13:10:20.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.191+0000 7f6fa7e00640 1 --2- 172.21.15.19:0/1795127334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa815e6e0 0x7f6f90004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.192+0000 7f6fadc00640 1 --2- 172.21.15.19:0/1795127334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f90004d40 0x7f6f900051c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000a7a0 tx=0x7f6f90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6878da920ac9979c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.192+0000 7f6fa7400640 1 -- 172.21.15.19:0/1795127334 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa000a3f0 con 0x7f6f90004d40 2024-03-28T13:10:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.192+0000 7f6fa7400640 1 -- 172.21.15.19:0/1795127334 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa0034650 con 0x7f6f90004d40 2024-03-28T13:10:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.193+0000 7f6fa7400640 1 -- 172.21.15.19:0/1795127334 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0034a40 con 0x7f6f90004d40 2024-03-28T13:10:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.193+0000 7f6fae600640 1 -- 172.21.15.19:0/1795127334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f90004d40 msgr2=0x7f6f900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.193+0000 7f6fae600640 1 --2- 172.21.15.19:0/1795127334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f90004d40 0x7f6f900051c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000a7a0 tx=0x7f6f90005830 comp rx=0 tx=0).stop 2024-03-28T13:10:20.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.193+0000 7f6fadc00640 1 -- 172.21.15.19:0/1795127334 reap_dead start 2024-03-28T13:10:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.193+0000 7f6fae600640 1 -- 172.21.15.19:0/1795127334 shutdown_connections 2024-03-28T13:10:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.193+0000 7f6fae600640 1 -- 172.21.15.19:0/1795127334 >> 172.21.15.19:0/1795127334 conn(0x7f6fa809aa20 msgr2=0x7f6fa80b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.193+0000 7f6fae600640 1 -- 172.21.15.19:0/1795127334 shutdown_connections 2024-03-28T13:10:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.193+0000 7f6fae600640 1 -- 172.21.15.19:0/1795127334 wait complete. 2024-03-28T13:10:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.196+0000 7f6fae600640 1 Processor -- start 2024-03-28T13:10:20.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.196+0000 7f6fae600640 1 -- start start 2024-03-28T13:10:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.198+0000 7f6fae600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f90004d40 0x7f6fa80b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.198+0000 7f6fae600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa80b41a0 0x7f6fa8161100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.198+0000 7f6fae600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa8161640 0x7f6fa814b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.198+0000 7f6fae600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa815f1d0 con 0x7f6fa8161640 2024-03-28T13:10:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.198+0000 7f6fae600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa80b0a00 con 0x7f6fa80b41a0 2024-03-28T13:10:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.198+0000 7f6fae600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa815f350 con 0x7f6f90004d40 2024-03-28T13:10:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.198+0000 7f6fadc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa8161640 0x7f6fa814b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:20.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.198+0000 7f6fadc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa8161640 0x7f6fa814b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53874/0 (socket says 172.21.15.19:53874) 2024-03-28T13:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.198+0000 7f6fadc00640 1 -- 172.21.15.19:0/2264626932 learned_addr learned my addr 172.21.15.19:0/2264626932 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.199+0000 7f6fa7e00640 1 --2- 172.21.15.19:0/2264626932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa80b41a0 0x7f6fa8161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.199+0000 7f6fad200640 1 --2- 172.21.15.19:0/2264626932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f90004d40 0x7f6fa80b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.199+0000 7f6fadc00640 1 -- 172.21.15.19:0/2264626932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f90004d40 msgr2=0x7f6fa80b6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.199+0000 7f6fadc00640 1 --2- 172.21.15.19:0/2264626932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f90004d40 0x7f6fa80b6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.199+0000 7f6fadc00640 1 -- 172.21.15.19:0/2264626932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa80b41a0 msgr2=0x7f6fa8161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.199+0000 7f6fadc00640 1 --2- 172.21.15.19:0/2264626932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa80b41a0 0x7f6fa8161100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.199+0000 7f6fadc00640 1 -- 172.21.15.19:0/2264626932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa814c240 con 0x7f6fa8161640 2024-03-28T13:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.199+0000 7f6fa7e00640 1 --2- 172.21.15.19:0/2264626932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa80b41a0 0x7f6fa8161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:10:20.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.199+0000 7f6fadc00640 1 --2- 172.21.15.19:0/2264626932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa8161640 0x7f6fa814b4f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000a7a0 tx=0x7f6fa0001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.199+0000 7f6fa4c00640 1 -- 172.21.15.19:0/2264626932 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0034800 con 0x7f6fa8161640 2024-03-28T13:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.200+0000 7f6fa4c00640 1 -- 172.21.15.19:0/2264626932 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa0044070 con 0x7f6fa8161640 2024-03-28T13:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.200+0000 7f6fa4c00640 1 -- 172.21.15.19:0/2264626932 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0034ba0 con 0x7f6fa8161640 2024-03-28T13:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.200+0000 7f6fae600640 1 -- 172.21.15.19:0/2264626932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa80b5ff0 con 0x7f6fa8161640 2024-03-28T13:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.200+0000 7f6fae600640 1 -- 172.21.15.19:0/2264626932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa80b65b0 con 0x7f6fa8161640 2024-03-28T13:10:20.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.201+0000 7f6fae600640 1 -- 172.21.15.19:0/2264626932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa80b67b0 con 0x7f6fa8161640 2024-03-28T13:10:20.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.202+0000 7f6fa4c00640 1 -- 172.21.15.19:0/2264626932 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6fa0049460 con 0x7f6fa8161640 2024-03-28T13:10:20.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.203+0000 7f6fa4c00640 1 --2- 172.21.15.19:0/2264626932 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6f8806c580 0x7f6f8806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:20.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.203+0000 7f6fa4c00640 1 -- 172.21.15.19:0/2264626932 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6fa00adc30 con 0x7f6fa8161640 2024-03-28T13:10:20.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.204+0000 7f6fad200640 1 --2- 172.21.15.19:0/2264626932 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6f8806c580 0x7f6f8806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:20.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.205+0000 7f6fad200640 1 --2- 172.21.15.19:0/2264626932 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6f8806c580 0x7f6f8806ea40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009950 tx=0x7f6f9c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:20.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.207+0000 7f6fa4c00640 1 -- 172.21.15.19:0/2264626932 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6fa80b67b0 con 0x7f6fa8161640 2024-03-28T13:10:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.417+0000 7f6fae600640 1 -- 172.21.15.19:0/2264626932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fa8075940 con 0x7f6fa8161640 2024-03-28T13:10:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.420+0000 7f6fa4c00640 1 -- 172.21.15.19:0/2264626932 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16609 (secure 0 0 0) 0x7f6fa0032380 con 0x7f6fa8161640 2024-03-28T13:10:20.421 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:20.421 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":23,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:09:42.668077+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:10:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.422+0000 7f6fae600640 1 -- 172.21.15.19:0/2264626932 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6f8806c580 msgr2=0x7f6f8806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:20.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.422+0000 7f6fae600640 1 --2- 172.21.15.19:0/2264626932 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6f8806c580 0x7f6f8806ea40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009950 tx=0x7f6f9c008040 comp rx=0 tx=0).stop 2024-03-28T13:10:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.422+0000 7f6fae600640 1 -- 172.21.15.19:0/2264626932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa8161640 msgr2=0x7f6fa814b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.422+0000 7f6fae600640 1 --2- 172.21.15.19:0/2264626932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa8161640 0x7f6fa814b4f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000a7a0 tx=0x7f6fa0001cf0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.422+0000 7f6fae600640 1 -- 172.21.15.19:0/2264626932 shutdown_connections 2024-03-28T13:10:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.422+0000 7f6fae600640 1 --2- 172.21.15.19:0/2264626932 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6f8806c580 0x7f6f8806ea40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.422+0000 7f6fae600640 1 --2- 172.21.15.19:0/2264626932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa8161640 0x7f6fa814b4f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.422+0000 7f6fae600640 1 --2- 172.21.15.19:0/2264626932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa80b41a0 0x7f6fa8161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.422+0000 7f6fae600640 1 --2- 172.21.15.19:0/2264626932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f90004d40 0x7f6fa80b6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.422+0000 7f6fae600640 1 -- 172.21.15.19:0/2264626932 >> 172.21.15.19:0/2264626932 conn(0x7f6fa809aa20 msgr2=0x7f6fa8162780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.423+0000 7f6fae600640 1 -- 172.21.15.19:0/2264626932 shutdown_connections 2024-03-28T13:10:20.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.423+0000 7f6fae600640 1 -- 172.21.15.19:0/2264626932 wait complete. 2024-03-28T13:10:20.442 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:09:39.243813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:09:40.092899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:10:20.443 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:10:20.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.687+0000 7f54fb400640 1 Processor -- start 2024-03-28T13:10:20.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.687+0000 7f54fb400640 1 -- start start 2024-03-28T13:10:20.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.689+0000 7f54fb400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f54f415e860 0x7f54f406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:20.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.689+0000 7f54fb400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54f406da30 0x7f54f40d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:20.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.689+0000 7f54fb400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54f406dfa0 con 0x7f54f40d4910 2024-03-28T13:10:20.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.689+0000 7f54fb400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f54f40d4d60 con 0x7f54f415e860 2024-03-28T13:10:20.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.689+0000 7f54fb400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f54f40cf220 con 0x7f54f406da30 2024-03-28T13:10:20.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.689+0000 7f54f8a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f54f415e860 0x7f54f406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:20.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.689+0000 7f54f9400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f54f40d4910 0x7f54f40d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48080/0 (socket says 172.21.15.19:48080) 2024-03-28T13:10:20.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.689+0000 7f54f9400640 1 -- 172.21.15.19:0/2108062082 learned_addr learned my addr 172.21.15.19:0/2108062082 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.690+0000 7f54f3e00640 1 --2- 172.21.15.19:0/2108062082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54f406da30 0x7f54f40d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.690+0000 7f54f8a00640 1 -- 172.21.15.19:0/2108062082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54f406da30 msgr2=0x7f54f40d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.690+0000 7f54f8a00640 1 --2- 172.21.15.19:0/2108062082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54f406da30 0x7f54f40d43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.690+0000 7f54f8a00640 1 -- 172.21.15.19:0/2108062082 >> v1:172.21.15.19:6789/0 conn(0x7f54f40d4910 legacy=0x7f54f40d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.690+0000 7f54f8a00640 1 -- 172.21.15.19:0/2108062082 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f406e240 con 0x7f54f415e860 2024-03-28T13:10:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.690+0000 7f54f3e00640 1 --2- 172.21.15.19:0/2108062082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54f406da30 0x7f54f40d43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.690+0000 7f54f3400640 1 -- 172.21.15.19:0/2108062082 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2655970188 0 0) 0x7f54f406dfa0 con 0x7f54f40d4910 2024-03-28T13:10:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.691+0000 7f54f8a00640 1 --2- 172.21.15.19:0/2108062082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f415e860 0x7f54f406d4f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f54e80097a0 tx=0x7f54e802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9646fecac4e9ed5f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:20.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.691+0000 7f54f3400640 1 -- 172.21.15.19:0/2108062082 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8035020 con 0x7f54f415e860 2024-03-28T13:10:20.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.691+0000 7f54f3400640 1 -- 172.21.15.19:0/2108062082 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e8031810 con 0x7f54f415e860 2024-03-28T13:10:20.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.691+0000 7f54f3400640 1 -- 172.21.15.19:0/2108062082 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8031be0 con 0x7f54f415e860 2024-03-28T13:10:20.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.692+0000 7f54fb400640 1 -- 172.21.15.19:0/2108062082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f415e860 msgr2=0x7f54f406d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:20.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.692+0000 7f54fb400640 1 --2- 172.21.15.19:0/2108062082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f415e860 0x7f54f406d4f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f54e80097a0 tx=0x7f54e802b8b0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.692+0000 7f54fb400640 1 -- 172.21.15.19:0/2108062082 shutdown_connections 2024-03-28T13:10:20.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.692+0000 7f54fb400640 1 --2- 172.21.15.19:0/2108062082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54f406da30 0x7f54f40d43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.692+0000 7f54fb400640 1 --2- 172.21.15.19:0/2108062082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f415e860 0x7f54f406d4f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.692+0000 7f54fb400640 1 -- 172.21.15.19:0/2108062082 >> 172.21.15.19:0/2108062082 conn(0x7f54f409d7d0 msgr2=0x7f54f408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:20.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.692+0000 7f54fb400640 1 -- 172.21.15.19:0/2108062082 shutdown_connections 2024-03-28T13:10:20.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.692+0000 7f54fb400640 1 -- 172.21.15.19:0/2108062082 wait complete. 2024-03-28T13:10:20.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.694+0000 7f54fb400640 1 Processor -- start 2024-03-28T13:10:20.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.694+0000 7f54fb400640 1 -- start start 2024-03-28T13:10:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.695+0000 7f54fb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f406da30 0x7f54f4157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.695+0000 7f54fb400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54f40d4910 0x7f54f41610b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.695+0000 7f54fb400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f415e860 0x7f54f40b9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.695+0000 7f54fb400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f54f40cedb0 con 0x7f54f406da30 2024-03-28T13:10:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.695+0000 7f54fb400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f54f40d2c50 con 0x7f54f415e860 2024-03-28T13:10:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.695+0000 7f54fb400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f54f40cef30 con 0x7f54f40d4910 2024-03-28T13:10:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.695+0000 7f54f8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f406da30 0x7f54f4157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.695+0000 7f54f8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f406da30 0x7f54f4157ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53902/0 (socket says 172.21.15.19:53902) 2024-03-28T13:10:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.695+0000 7f54f8a00640 1 -- 172.21.15.19:0/1406497144 learned_addr learned my addr 172.21.15.19:0/1406497144 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:20.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.696+0000 7f54f3e00640 1 --2- 172.21.15.19:0/1406497144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54f40d4910 0x7f54f41610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.696+0000 7f54f8a00640 1 -- 172.21.15.19:0/1406497144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54f40d4910 msgr2=0x7f54f41610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.696+0000 7f54f9400640 1 --2- 172.21.15.19:0/1406497144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f415e860 0x7f54f40b9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.696+0000 7f54f8a00640 1 --2- 172.21.15.19:0/1406497144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54f40d4910 0x7f54f41610b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.696+0000 7f54f8a00640 1 -- 172.21.15.19:0/1406497144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f415e860 msgr2=0x7f54f40b9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.696+0000 7f54f8a00640 1 --2- 172.21.15.19:0/1406497144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f415e860 0x7f54f40b9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.696+0000 7f54f8a00640 1 -- 172.21.15.19:0/1406497144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f4161a00 con 0x7f54f406da30 2024-03-28T13:10:20.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.696+0000 7f54f8a00640 1 --2- 172.21.15.19:0/1406497144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f406da30 0x7f54f4157ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f54e80097a0 tx=0x7f54e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:20.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.697+0000 7f54f0c00640 1 -- 172.21.15.19:0/1406497144 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8041070 con 0x7f54f406da30 2024-03-28T13:10:20.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.697+0000 7f54f0c00640 1 -- 172.21.15.19:0/1406497144 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54e803ad00 con 0x7f54f406da30 2024-03-28T13:10:20.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.697+0000 7f54fb400640 1 -- 172.21.15.19:0/1406497144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f4161ce0 con 0x7f54f406da30 2024-03-28T13:10:20.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.697+0000 7f54f0c00640 1 -- 172.21.15.19:0/1406497144 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8035040 con 0x7f54f406da30 2024-03-28T13:10:20.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.697+0000 7f54fb400640 1 -- 172.21.15.19:0/1406497144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f4157ff0 con 0x7f54f406da30 2024-03-28T13:10:20.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.698+0000 7f54fb400640 1 -- 172.21.15.19:0/1406497144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f40685c0 con 0x7f54f406da30 2024-03-28T13:10:20.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.699+0000 7f54f0c00640 1 -- 172.21.15.19:0/1406497144 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f54e8038020 con 0x7f54f406da30 2024-03-28T13:10:20.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.700+0000 7f54f0c00640 1 --2- 172.21.15.19:0/1406497144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f54d406c530 0x7f54d406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:20.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.700+0000 7f54f0c00640 1 -- 172.21.15.19:0/1406497144 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f54e80ad4a0 con 0x7f54f406da30 2024-03-28T13:10:20.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.701+0000 7f54f3e00640 1 --2- 172.21.15.19:0/1406497144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f54d406c530 0x7f54d406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:20.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.702+0000 7f54f3e00640 1 --2- 172.21.15.19:0/1406497144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f54d406c530 0x7f54d406e9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f54f40b4860 tx=0x7f54e0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:20.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.704+0000 7f54f0c00640 1 -- 172.21.15.19:0/1406497144 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f54e807b460 con 0x7f54f406da30 2024-03-28T13:10:20.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.907+0000 7f54fb400640 1 -- 172.21.15.19:0/1406497144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f54f40d1270 con 0x7f54f406da30 2024-03-28T13:10:20.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.908+0000 7f54f0c00640 1 -- 172.21.15.19:0/1406497144 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f54e807f760 con 0x7f54f406da30 2024-03-28T13:10:20.908 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:10:20.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.911+0000 7f54fb400640 1 -- 172.21.15.19:0/1406497144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f54d406c530 msgr2=0x7f54d406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:20.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.911+0000 7f54fb400640 1 --2- 172.21.15.19:0/1406497144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f54d406c530 0x7f54d406e9f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f54f40b4860 tx=0x7f54e0008040 comp rx=0 tx=0).stop 2024-03-28T13:10:20.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.911+0000 7f54fb400640 1 -- 172.21.15.19:0/1406497144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f406da30 msgr2=0x7f54f4157ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:20.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.911+0000 7f54fb400640 1 --2- 172.21.15.19:0/1406497144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f406da30 0x7f54f4157ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f54e80097a0 tx=0x7f54e80046c0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.911+0000 7f54fb400640 1 -- 172.21.15.19:0/1406497144 shutdown_connections 2024-03-28T13:10:20.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.911+0000 7f54fb400640 1 --2- 172.21.15.19:0/1406497144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f54d406c530 0x7f54d406e9f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.911+0000 7f54fb400640 1 --2- 172.21.15.19:0/1406497144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54f415e860 0x7f54f40b9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.911+0000 7f54fb400640 1 --2- 172.21.15.19:0/1406497144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54f40d4910 0x7f54f41610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.911+0000 7f54fb400640 1 --2- 172.21.15.19:0/1406497144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54f406da30 0x7f54f4157ab0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:20.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.911+0000 7f54fb400640 1 -- 172.21.15.19:0/1406497144 >> 172.21.15.19:0/1406497144 conn(0x7f54f409d7d0 msgr2=0x7f54f40b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:20.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.911+0000 7f54fb400640 1 -- 172.21.15.19:0/1406497144 shutdown_connections 2024-03-28T13:10:20.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:20.911+0000 7f54fb400640 1 -- 172.21.15.19:0/1406497144 wait complete. 2024-03-28T13:10:20.931 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:10:21.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.153+0000 7ff02f800640 1 Processor -- start 2024-03-28T13:10:21.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.153+0000 7ff02f800640 1 -- start start 2024-03-28T13:10:21.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.154+0000 7ff02f800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff0280d0280 0x7ff0280d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.154+0000 7ff02f800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff028141c10 0x7ff028071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.154+0000 7ff02f800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff02815ec80 con 0x7ff028071eb0 2024-03-28T13:10:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.154+0000 7ff02f800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff0280d0ba0 con 0x7ff0280d0280 2024-03-28T13:10:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.154+0000 7ff02f800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff02815ef20 con 0x7ff028141c10 2024-03-28T13:10:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.155+0000 7ff027e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff028141c10 0x7ff028071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:21.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.155+0000 7ff027e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff028141c10 0x7ff028071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:46810/0 (socket says 172.21.15.19:46810) 2024-03-28T13:10:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.155+0000 7ff027e00640 1 -- 172.21.15.19:0/645534732 learned_addr learned my addr 172.21.15.19:0/645534732 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.155+0000 7ff02ce00640 1 --2- 172.21.15.19:0/645534732 >> v2:172.21.15.188:3300/0 conn(0x7ff0280d0280 0x7ff0280d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.156+0000 7ff027e00640 1 -- 172.21.15.19:0/645534732 >> v2:172.21.15.188:3300/0 conn(0x7ff0280d0280 msgr2=0x7ff0280d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.156+0000 7ff027e00640 1 --2- 172.21.15.19:0/645534732 >> v2:172.21.15.188:3300/0 conn(0x7ff0280d0280 0x7ff0280d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.156+0000 7ff027e00640 1 -- 172.21.15.19:0/645534732 >> v1:172.21.15.188:6789/0 conn(0x7ff028071eb0 legacy=0x7ff02806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:21.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.156+0000 7ff027e00640 1 -- 172.21.15.19:0/645534732 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02815f200 con 0x7ff028141c10 2024-03-28T13:10:21.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.157+0000 7ff027e00640 1 --2- 172.21.15.19:0/645534732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff028141c10 0x7ff028071970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff014009870 tx=0x7ff01402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed98d95de56db506 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.158+0000 7ff027400640 1 -- 172.21.15.19:0/645534732 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014034070 con 0x7ff028141c10 2024-03-28T13:10:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.158+0000 7ff027400640 1 -- 172.21.15.19:0/645534732 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff014030840 con 0x7ff028141c10 2024-03-28T13:10:21.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.158+0000 7ff027400640 1 -- 172.21.15.19:0/645534732 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff014030c10 con 0x7ff028141c10 2024-03-28T13:10:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.158+0000 7ff02f800640 1 -- 172.21.15.19:0/645534732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff028141c10 msgr2=0x7ff028071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.158+0000 7ff02f800640 1 --2- 172.21.15.19:0/645534732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff028141c10 0x7ff028071970 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff014009870 tx=0x7ff01402b980 comp rx=0 tx=0).stop 2024-03-28T13:10:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.159+0000 7ff02f800640 1 -- 172.21.15.19:0/645534732 shutdown_connections 2024-03-28T13:10:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.159+0000 7ff02f800640 1 --2- 172.21.15.19:0/645534732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff028141c10 0x7ff028071970 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.159+0000 7ff02f800640 1 --2- 172.21.15.19:0/645534732 >> v2:172.21.15.188:3300/0 conn(0x7ff0280d0280 0x7ff0280d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.159+0000 7ff02f800640 1 -- 172.21.15.19:0/645534732 >> 172.21.15.19:0/645534732 conn(0x7ff02806b750 msgr2=0x7ff02806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.159+0000 7ff02f800640 1 -- 172.21.15.19:0/645534732 shutdown_connections 2024-03-28T13:10:21.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.159+0000 7ff02f800640 1 -- 172.21.15.19:0/645534732 wait complete. 2024-03-28T13:10:21.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.160+0000 7ff02f800640 1 Processor -- start 2024-03-28T13:10:21.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.161+0000 7ff02f800640 1 -- start start 2024-03-28T13:10:21.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.162+0000 7ff02f800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff028071eb0 0x7ff028157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:21.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.162+0000 7ff02f800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0280d0280 0x7ff0281e8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:21.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.162+0000 7ff02f800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff028141c10 0x7ff0281f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:21.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.162+0000 7ff02f800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff028158e90 con 0x7ff028141c10 2024-03-28T13:10:21.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.162+0000 7ff02f800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff02815f7c0 con 0x7ff0280d0280 2024-03-28T13:10:21.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.162+0000 7ff02f800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff028157780 con 0x7ff028071eb0 2024-03-28T13:10:21.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.162+0000 7ff02d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff028141c10 0x7ff0281f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:21.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.162+0000 7ff02d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff028141c10 0x7ff0281f3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53914/0 (socket says 172.21.15.19:53914) 2024-03-28T13:10:21.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.162+0000 7ff02d800640 1 -- 172.21.15.19:0/726011046 learned_addr learned my addr 172.21.15.19:0/726011046 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:21.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.162+0000 7ff02ce00640 1 --2- 172.21.15.19:0/726011046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff028071eb0 0x7ff028157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:21.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.162+0000 7ff027e00640 1 --2- 172.21.15.19:0/726011046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0280d0280 0x7ff0281e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:21.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.163+0000 7ff02d800640 1 -- 172.21.15.19:0/726011046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff028071eb0 msgr2=0x7ff028157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:21.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.163+0000 7ff02d800640 1 --2- 172.21.15.19:0/726011046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff028071eb0 0x7ff028157980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.163+0000 7ff02d800640 1 -- 172.21.15.19:0/726011046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0280d0280 msgr2=0x7ff0281e8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:21.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.163+0000 7ff02d800640 1 --2- 172.21.15.19:0/726011046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0280d0280 0x7ff0281e8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.163+0000 7ff02d800640 1 -- 172.21.15.19:0/726011046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0281e8fd0 con 0x7ff028141c10 2024-03-28T13:10:21.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.163+0000 7ff02d800640 1 --2- 172.21.15.19:0/726011046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff028141c10 0x7ff0281f3db0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff02000c7a0 tx=0x7ff02000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:21.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.163+0000 7ff024c00640 1 -- 172.21.15.19:0/726011046 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff020003b70 con 0x7ff028141c10 2024-03-28T13:10:21.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.164+0000 7ff024c00640 1 -- 172.21.15.19:0/726011046 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff020003d30 con 0x7ff028141c10 2024-03-28T13:10:21.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.164+0000 7ff02f800640 1 -- 172.21.15.19:0/726011046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0281e92b0 con 0x7ff028141c10 2024-03-28T13:10:21.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.164+0000 7ff024c00640 1 -- 172.21.15.19:0/726011046 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02000aa30 con 0x7ff028141c10 2024-03-28T13:10:21.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.164+0000 7ff02f800640 1 -- 172.21.15.19:0/726011046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0281e9870 con 0x7ff028141c10 2024-03-28T13:10:21.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.165+0000 7ff02f800640 1 -- 172.21.15.19:0/726011046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0280685c0 con 0x7ff028141c10 2024-03-28T13:10:21.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.166+0000 7ff024c00640 1 -- 172.21.15.19:0/726011046 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff02000abd0 con 0x7ff028141c10 2024-03-28T13:10:21.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.167+0000 7ff024c00640 1 --2- 172.21.15.19:0/726011046 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff00806c530 0x7ff00806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:21.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.167+0000 7ff024c00640 1 -- 172.21.15.19:0/726011046 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff02008b4c0 con 0x7ff028141c10 2024-03-28T13:10:21.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.167+0000 7ff02ce00640 1 --2- 172.21.15.19:0/726011046 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff00806c530 0x7ff00806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:21.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.169+0000 7ff02ce00640 1 --2- 172.21.15.19:0/726011046 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff00806c530 0x7ff00806e9f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009b40 tx=0x7ff01c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:21.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.171+0000 7ff024c00640 1 -- 172.21.15.19:0/726011046 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff0200593f0 con 0x7ff028141c10 2024-03-28T13:10:21.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.381+0000 7ff02f800640 1 -- 172.21.15.19:0/726011046 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff0280cfdb0 con 0x7ff028141c10 2024-03-28T13:10:21.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.382+0000 7ff024c00640 1 -- 172.21.15.19:0/726011046 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7ff02005d6f0 con 0x7ff028141c10 2024-03-28T13:10:21.382 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:10:21.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.386+0000 7ff02f800640 1 -- 172.21.15.19:0/726011046 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff00806c530 msgr2=0x7ff00806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:21.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.386+0000 7ff02f800640 1 --2- 172.21.15.19:0/726011046 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff00806c530 0x7ff00806e9f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff01c009b40 tx=0x7ff01c009190 comp rx=0 tx=0).stop 2024-03-28T13:10:21.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.386+0000 7ff02f800640 1 -- 172.21.15.19:0/726011046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff028141c10 msgr2=0x7ff0281f3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:21.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.386+0000 7ff02f800640 1 --2- 172.21.15.19:0/726011046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff028141c10 0x7ff0281f3db0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff02000c7a0 tx=0x7ff02000cc60 comp rx=0 tx=0).stop 2024-03-28T13:10:21.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.386+0000 7ff02f800640 1 -- 172.21.15.19:0/726011046 shutdown_connections 2024-03-28T13:10:21.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.386+0000 7ff02f800640 1 --2- 172.21.15.19:0/726011046 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff00806c530 0x7ff00806e9f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.386+0000 7ff02f800640 1 --2- 172.21.15.19:0/726011046 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff028141c10 0x7ff0281f3db0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.386+0000 7ff02f800640 1 --2- 172.21.15.19:0/726011046 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0280d0280 0x7ff0281e8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.386+0000 7ff02f800640 1 --2- 172.21.15.19:0/726011046 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff028071eb0 0x7ff028157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.386+0000 7ff02f800640 1 -- 172.21.15.19:0/726011046 >> 172.21.15.19:0/726011046 conn(0x7ff02806b750 msgr2=0x7ff0280b14b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:21.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.386+0000 7ff02f800640 1 -- 172.21.15.19:0/726011046 shutdown_connections 2024-03-28T13:10:21.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.386+0000 7ff02f800640 1 -- 172.21.15.19:0/726011046 wait complete. 2024-03-28T13:10:21.404 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:10:21.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.615+0000 7f908c400640 1 Processor -- start 2024-03-28T13:10:21.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.615+0000 7f908c400640 1 -- start start 2024-03-28T13:10:21.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.616+0000 7f908c400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90840d0e70 0x7f90840cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:21.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.616+0000 7f908c400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f908415f6e0 0x7f90840d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:21.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.616+0000 7f908c400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f90841621d0 0x7f9084162660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:21.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.616+0000 7f908c400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f90840d1790 con 0x7f90841621d0 2024-03-28T13:10:21.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.616+0000 7f908c400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f90840cf6c0 con 0x7f908415f6e0 2024-03-28T13:10:21.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.616+0000 7f908c400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f90840cf9a0 con 0x7f90840d0e70 2024-03-28T13:10:21.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.617+0000 7f908a400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f90841621d0 0x7f9084162660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:21.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.617+0000 7f908a400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f90841621d0 0x7f9084162660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53924/0 (socket says 172.21.15.19:53924) 2024-03-28T13:10:21.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.617+0000 7f908a400640 1 -- 172.21.15.19:0/3428906083 learned_addr learned my addr 172.21.15.19:0/3428906083 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:21.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.617+0000 7f908a400640 1 -- 172.21.15.19:0/3428906083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90840d0e70 msgr2=0x7f90840cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:21.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.617+0000 7f9089a00640 1 --2- 172.21.15.19:0/3428906083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90840d0e70 0x7f90840cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:21.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.617+0000 7f9089000640 1 --2- 172.21.15.19:0/3428906083 >> v2:172.21.15.188:3300/0 conn(0x7f908415f6e0 0x7f90840d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:21.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.617+0000 7f908a400640 1 --2- 172.21.15.19:0/3428906083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90840d0e70 0x7f90840cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.617+0000 7f908a400640 1 -- 172.21.15.19:0/3428906083 >> v2:172.21.15.188:3300/0 conn(0x7f908415f6e0 msgr2=0x7f90840d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:21.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.617+0000 7f908a400640 1 --2- 172.21.15.19:0/3428906083 >> v2:172.21.15.188:3300/0 conn(0x7f908415f6e0 0x7f90840d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.617+0000 7f908a400640 1 -- 172.21.15.19:0/3428906083 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90840cfc80 con 0x7f90841621d0 2024-03-28T13:10:21.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.617+0000 7f9089a00640 1 --2- 172.21.15.19:0/3428906083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90840d0e70 0x7f90840cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:21.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.617+0000 7f9089000640 1 --2- 172.21.15.19:0/3428906083 >> v2:172.21.15.188:3300/0 conn(0x7f908415f6e0 0x7f90840d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:21.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.618+0000 7f908a400640 1 --2- 172.21.15.19:0/3428906083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90841621d0 0x7f9084162660 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f908408ce40 tx=0x7f907c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b462d09e35542f8a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:21.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.619+0000 7f9083e00640 1 -- 172.21.15.19:0/3428906083 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c02e070 con 0x7f90841621d0 2024-03-28T13:10:21.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.619+0000 7f9083e00640 1 -- 172.21.15.19:0/3428906083 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f907c037070 con 0x7f90841621d0 2024-03-28T13:10:21.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.619+0000 7f9083e00640 1 -- 172.21.15.19:0/3428906083 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c031a00 con 0x7f90841621d0 2024-03-28T13:10:21.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.619+0000 7f908c400640 1 -- 172.21.15.19:0/3428906083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90841621d0 msgr2=0x7f9084162660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:21.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.619+0000 7f908c400640 1 --2- 172.21.15.19:0/3428906083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90841621d0 0x7f9084162660 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f908408ce40 tx=0x7f907c02b880 comp rx=0 tx=0).stop 2024-03-28T13:10:21.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.620+0000 7f908c400640 1 -- 172.21.15.19:0/3428906083 shutdown_connections 2024-03-28T13:10:21.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.620+0000 7f908c400640 1 --2- 172.21.15.19:0/3428906083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90841621d0 0x7f9084162660 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.620+0000 7f908c400640 1 --2- 172.21.15.19:0/3428906083 >> v2:172.21.15.188:3300/0 conn(0x7f908415f6e0 0x7f90840d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.620+0000 7f908c400640 1 --2- 172.21.15.19:0/3428906083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f90840d0e70 0x7f90840cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.620+0000 7f908c400640 1 -- 172.21.15.19:0/3428906083 >> 172.21.15.19:0/3428906083 conn(0x7f908408bfb0 msgr2=0x7f90840d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:21.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.620+0000 7f908c400640 1 -- 172.21.15.19:0/3428906083 shutdown_connections 2024-03-28T13:10:21.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.620+0000 7f908c400640 1 -- 172.21.15.19:0/3428906083 wait complete. 2024-03-28T13:10:21.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.622+0000 7f908c400640 1 Processor -- start 2024-03-28T13:10:21.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.622+0000 7f908c400640 1 -- start start 2024-03-28T13:10:21.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.623+0000 7f908c400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90840d0e70 0x7f90840cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:21.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.623+0000 7f908c400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f908415f6e0 0x7f90840c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:21.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.623+0000 7f908c400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90841621d0 0x7f90840c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:21.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.623+0000 7f908c400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f908415fb40 con 0x7f90841621d0 2024-03-28T13:10:21.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.623+0000 7f908c400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f90840d0240 con 0x7f90840d0e70 2024-03-28T13:10:21.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.623+0000 7f908c400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f908415e910 con 0x7f908415f6e0 2024-03-28T13:10:21.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.623+0000 7f908a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90841621d0 0x7f90840c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:21.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.623+0000 7f908a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90841621d0 0x7f90840c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53938/0 (socket says 172.21.15.19:53938) 2024-03-28T13:10:21.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.623+0000 7f908a400640 1 -- 172.21.15.19:0/92878313 learned_addr learned my addr 172.21.15.19:0/92878313 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:21.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.624+0000 7f9089a00640 1 --2- 172.21.15.19:0/92878313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90840d0e70 0x7f90840cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:21.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.624+0000 7f9089000640 1 --2- 172.21.15.19:0/92878313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f908415f6e0 0x7f90840c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:21.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.624+0000 7f908a400640 1 -- 172.21.15.19:0/92878313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f908415f6e0 msgr2=0x7f90840c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:21.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.624+0000 7f908a400640 1 --2- 172.21.15.19:0/92878313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f908415f6e0 0x7f90840c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.624+0000 7f908a400640 1 -- 172.21.15.19:0/92878313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90840d0e70 msgr2=0x7f90840cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:21.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.624+0000 7f908a400640 1 --2- 172.21.15.19:0/92878313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90840d0e70 0x7f90840cd370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.624+0000 7f908a400640 1 -- 172.21.15.19:0/92878313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90840cb230 con 0x7f90841621d0 2024-03-28T13:10:21.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.624+0000 7f908a400640 1 --2- 172.21.15.19:0/92878313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90841621d0 0x7f90840c7f70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f907c02b8c0 tx=0x7f907c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:21.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.624+0000 7f9081600640 1 -- 172.21.15.19:0/92878313 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c032880 con 0x7f90841621d0 2024-03-28T13:10:21.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.625+0000 7f9081600640 1 -- 172.21.15.19:0/92878313 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f907c032a40 con 0x7f90841621d0 2024-03-28T13:10:21.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.625+0000 7f908c400640 1 -- 172.21.15.19:0/92878313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90840ca990 con 0x7f90841621d0 2024-03-28T13:10:21.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.625+0000 7f9081600640 1 -- 172.21.15.19:0/92878313 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f907c032e30 con 0x7f90841621d0 2024-03-28T13:10:21.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.625+0000 7f908c400640 1 -- 172.21.15.19:0/92878313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90840caf50 con 0x7f90841621d0 2024-03-28T13:10:21.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.626+0000 7f908c400640 1 -- 172.21.15.19:0/92878313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9084068cd0 con 0x7f90841621d0 2024-03-28T13:10:21.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.627+0000 7f9081600640 1 -- 172.21.15.19:0/92878313 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f907c0453f0 con 0x7f90841621d0 2024-03-28T13:10:21.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.628+0000 7f9081600640 1 --2- 172.21.15.19:0/92878313 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f906406c530 0x7f906406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:21.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.628+0000 7f9081600640 1 -- 172.21.15.19:0/92878313 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f907c0ad8a0 con 0x7f90841621d0 2024-03-28T13:10:21.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.629+0000 7f9089a00640 1 --2- 172.21.15.19:0/92878313 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f906406c530 0x7f906406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:21.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.630+0000 7f9089a00640 1 --2- 172.21.15.19:0/92878313 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f906406c530 0x7f906406e9f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90780075b0 tx=0x7f9078007540 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:21.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.634+0000 7f9081600640 1 -- 172.21.15.19:0/92878313 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f907c040070 con 0x7f90841621d0 2024-03-28T13:10:21.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.844+0000 7f908c400640 1 -- 172.21.15.19:0/92878313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f90840d1b80 con 0x7f90841621d0 2024-03-28T13:10:21.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.845+0000 7f9081600640 1 -- 172.21.15.19:0/92878313 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f907c07bfd0 con 0x7f90841621d0 2024-03-28T13:10:21.845 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:10:21.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.848+0000 7f908c400640 1 -- 172.21.15.19:0/92878313 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f906406c530 msgr2=0x7f906406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:21.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.848+0000 7f908c400640 1 --2- 172.21.15.19:0/92878313 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f906406c530 0x7f906406e9f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f90780075b0 tx=0x7f9078007540 comp rx=0 tx=0).stop 2024-03-28T13:10:21.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.848+0000 7f908c400640 1 -- 172.21.15.19:0/92878313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90841621d0 msgr2=0x7f90840c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:21.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.848+0000 7f908c400640 1 --2- 172.21.15.19:0/92878313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90841621d0 0x7f90840c7f70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f907c02b8c0 tx=0x7f907c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.848+0000 7f908c400640 1 -- 172.21.15.19:0/92878313 shutdown_connections 2024-03-28T13:10:21.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.848+0000 7f908c400640 1 --2- 172.21.15.19:0/92878313 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f906406c530 0x7f906406e9f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.848+0000 7f908c400640 1 --2- 172.21.15.19:0/92878313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f90841621d0 0x7f90840c7f70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.848+0000 7f908c400640 1 --2- 172.21.15.19:0/92878313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f908415f6e0 0x7f90840c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.848+0000 7f908c400640 1 --2- 172.21.15.19:0/92878313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f90840d0e70 0x7f90840cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:21.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.848+0000 7f908c400640 1 -- 172.21.15.19:0/92878313 >> 172.21.15.19:0/92878313 conn(0x7f908408bfb0 msgr2=0x7f9084160e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:21.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.848+0000 7f908c400640 1 -- 172.21.15.19:0/92878313 shutdown_connections 2024-03-28T13:10:21.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:21.848+0000 7f908c400640 1 -- 172.21.15.19:0/92878313 wait complete. 2024-03-28T13:10:21.875 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:10:22.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.135+0000 7fc2eec00640 1 Processor -- start 2024-03-28T13:10:22.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.135+0000 7fc2eec00640 1 -- start start 2024-03-28T13:10:22.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.137+0000 7fc2eec00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e80d3d90 0x7fc2e80b5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:22.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.137+0000 7fc2eec00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2e80d2cf0 con 0x7fc2e80b61f0 2024-03-28T13:10:22.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.137+0000 7fc2eec00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2e80d2fd0 con 0x7fc2e815e3b0 2024-03-28T13:10:22.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.137+0000 7fc2eec00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e80d32b0 con 0x7fc2e80d3d90 2024-03-28T13:10:22.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.138+0000 7fc2ee200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc2e80b61f0 0x7fc2e80b7360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48086/0 (socket says 172.21.15.19:48086) 2024-03-28T13:10:22.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.138+0000 7fc2ee200640 1 -- 172.21.15.19:0/207823172 learned_addr learned my addr 172.21.15.19:0/207823172 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.138+0000 7fc2ece00640 1 --2- 172.21.15.19:0/207823172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e80d3d90 0x7fc2e80b5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.138+0000 7fc2e7e00640 1 -- 172.21.15.19:0/207823172 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 771903150 0 0) 0x7fc2e80d2cf0 con 0x7fc2e80b61f0 2024-03-28T13:10:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.138+0000 7fc2e7e00640 1 -- 172.21.15.19:0/207823172 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc2d0003520 con 0x7fc2e80b61f0 2024-03-28T13:10:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2e7e00640 1 -- 172.21.15.19:0/207823172 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2065614658 0 0) 0x7fc2d0003520 con 0x7fc2e80b61f0 2024-03-28T13:10:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2e7e00640 1 -- 172.21.15.19:0/207823172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e80d3d90 msgr2=0x7fc2e80b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2e7e00640 1 --2- 172.21.15.19:0/207823172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e80d3d90 0x7fc2e80b5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2e7e00640 1 -- 172.21.15.19:0/207823172 >> v1:172.21.15.188:6789/0 conn(0x7fc2e815e3b0 legacy=0x7fc2e80d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2e7e00640 1 -- 172.21.15.19:0/207823172 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e80d3590 con 0x7fc2e80b61f0 2024-03-28T13:10:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2e7e00640 1 -- 172.21.15.19:0/207823172 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fc2e00019f0 con 0x7fc2e80b61f0 2024-03-28T13:10:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2e7e00640 1 -- 172.21.15.19:0/207823172 >> v1:172.21.15.19:6789/0 conn(0x7fc2e80b61f0 legacy=0x7fc2e80b7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:22.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2e7e00640 1 --2- 172.21.15.19:0/207823172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d00039e0 0x7fc2d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2e7e00640 1 --2- 172.21.15.19:0/207823172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e80d3d90 0x7fc2d0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2e7e00640 1 --2- 172.21.15.19:0/207823172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2d0004d40 0x7fc2d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2e7e00640 1 -- 172.21.15.19:0/207823172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e80d32b0 con 0x7fc2d00039e0 2024-03-28T13:10:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2e7e00640 1 -- 172.21.15.19:0/207823172 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d000bb30 con 0x7fc2d0004d40 2024-03-28T13:10:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2e7e00640 1 -- 172.21.15.19:0/207823172 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d000bd30 con 0x7fc2e80d3d90 2024-03-28T13:10:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2ed800640 1 --2- 172.21.15.19:0/207823172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d00039e0 0x7fc2d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2ee200640 1 --2- 172.21.15.19:0/207823172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2d0004d40 0x7fc2d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.139+0000 7fc2ece00640 1 --2- 172.21.15.19:0/207823172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e80d3d90 0x7fc2d0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.140+0000 7fc2ed800640 1 -- 172.21.15.19:0/207823172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e80d3d90 msgr2=0x7fc2d0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.140+0000 7fc2ed800640 1 --2- 172.21.15.19:0/207823172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e80d3d90 0x7fc2d0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.140+0000 7fc2ed800640 1 -- 172.21.15.19:0/207823172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2d0004d40 msgr2=0x7fc2d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.140+0000 7fc2ed800640 1 --2- 172.21.15.19:0/207823172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2d0004d40 0x7fc2d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.140+0000 7fc2ed800640 1 -- 172.21.15.19:0/207823172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d0003520 con 0x7fc2d00039e0 2024-03-28T13:10:22.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.141+0000 7fc2ed800640 1 --2- 172.21.15.19:0/207823172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d00039e0 0x7fc2d0003e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc001720 tx=0x7fc2d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5290aa5764fd5f15 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:22.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.143+0000 7fc2e7e00640 1 -- 172.21.15.19:0/207823172 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc001370 con 0x7fc2d00039e0 2024-03-28T13:10:22.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.143+0000 7fc2e7e00640 1 -- 172.21.15.19:0/207823172 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2dc0052d0 con 0x7fc2d00039e0 2024-03-28T13:10:22.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.143+0000 7fc2e7e00640 1 -- 172.21.15.19:0/207823172 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc003830 con 0x7fc2d00039e0 2024-03-28T13:10:22.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.144+0000 7fc2eec00640 1 -- 172.21.15.19:0/207823172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d00039e0 msgr2=0x7fc2d0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:22.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.144+0000 7fc2eec00640 1 --2- 172.21.15.19:0/207823172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2d00039e0 0x7fc2d0003e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc001720 tx=0x7fc2d0005830 comp rx=0 tx=0).stop 2024-03-28T13:10:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.144+0000 7fc2ee200640 1 -- 172.21.15.19:0/207823172 reap_dead start 2024-03-28T13:10:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.144+0000 7fc2eec00640 1 -- 172.21.15.19:0/207823172 shutdown_connections 2024-03-28T13:10:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.144+0000 7fc2eec00640 1 -- 172.21.15.19:0/207823172 >> 172.21.15.19:0/207823172 conn(0x7fc2e80678b0 msgr2=0x7fc2e80d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.144+0000 7fc2eec00640 1 -- 172.21.15.19:0/207823172 shutdown_connections 2024-03-28T13:10:22.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.144+0000 7fc2eec00640 1 -- 172.21.15.19:0/207823172 wait complete. 2024-03-28T13:10:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.147+0000 7fc2eec00640 1 Processor -- start 2024-03-28T13:10:22.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.147+0000 7fc2eec00640 1 -- start start 2024-03-28T13:10:22.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.148+0000 7fc2eec00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2d00039e0 0x7fc2e81f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:22.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.148+0000 7fc2eec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2e8150140 0x7fc2e814cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:22.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.148+0000 7fc2eec00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e80cebd0 0x7fc2e80cf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:22.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.148+0000 7fc2eec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e815e790 con 0x7fc2e8150140 2024-03-28T13:10:22.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.148+0000 7fc2eec00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e80d29f0 con 0x7fc2d00039e0 2024-03-28T13:10:22.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.148+0000 7fc2eec00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc2e80ba030 con 0x7fc2e80cebd0 2024-03-28T13:10:22.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.148+0000 7fc2ece00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2e8150140 0x7fc2e814cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:22.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.148+0000 7fc2ece00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2e8150140 0x7fc2e814cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53970/0 (socket says 172.21.15.19:53970) 2024-03-28T13:10:22.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.148+0000 7fc2ece00640 1 -- 172.21.15.19:0/2850662501 learned_addr learned my addr 172.21.15.19:0/2850662501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.149+0000 7fc2ed800640 1 --2- 172.21.15.19:0/2850662501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2d00039e0 0x7fc2e81f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.149+0000 7fc2ece00640 1 -- 172.21.15.19:0/2850662501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e80cebd0 msgr2=0x7fc2e80cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.149+0000 7fc2ece00640 1 --2- 172.21.15.19:0/2850662501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e80cebd0 0x7fc2e80cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.149+0000 7fc2ece00640 1 -- 172.21.15.19:0/2850662501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2d00039e0 msgr2=0x7fc2e81f7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.149+0000 7fc2ece00640 1 --2- 172.21.15.19:0/2850662501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2d00039e0 0x7fc2e81f7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.149+0000 7fc2ece00640 1 -- 172.21.15.19:0/2850662501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e8161100 con 0x7fc2e8150140 2024-03-28T13:10:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.149+0000 7fc2ece00640 1 --2- 172.21.15.19:0/2850662501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2e8150140 0x7fc2e814cd70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40108c0 tx=0x7fc2d4010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:22.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.149+0000 7fc2e5600640 1 -- 172.21.15.19:0/2850662501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d400f040 con 0x7fc2e8150140 2024-03-28T13:10:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.149+0000 7fc2e5600640 1 -- 172.21.15.19:0/2850662501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2d4005b90 con 0x7fc2e8150140 2024-03-28T13:10:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.149+0000 7fc2e5600640 1 -- 172.21.15.19:0/2850662501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d400e900 con 0x7fc2e8150140 2024-03-28T13:10:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.149+0000 7fc2eec00640 1 -- 172.21.15.19:0/2850662501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e81613d0 con 0x7fc2e8150140 2024-03-28T13:10:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.150+0000 7fc2eec00640 1 -- 172.21.15.19:0/2850662501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e81f77b0 con 0x7fc2e8150140 2024-03-28T13:10:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.151+0000 7fc2eec00640 1 -- 172.21.15.19:0/2850662501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e8075940 con 0x7fc2e8150140 2024-03-28T13:10:22.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.152+0000 7fc2e5600640 1 -- 172.21.15.19:0/2850662501 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc2d400eaa0 con 0x7fc2e8150140 2024-03-28T13:10:22.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.152+0000 7fc2e5600640 1 --2- 172.21.15.19:0/2850662501 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc2c806c530 0x7fc2c806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:22.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.153+0000 7fc2e5600640 1 -- 172.21.15.19:0/2850662501 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5767+0+0 (secure 0 0 0) 0x7fc2d408b970 con 0x7fc2e8150140 2024-03-28T13:10:22.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.153+0000 7fc2ed800640 1 --2- 172.21.15.19:0/2850662501 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc2c806c530 0x7fc2c806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:22.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.155+0000 7fc2ed800640 1 --2- 172.21.15.19:0/2850662501 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc2c806c530 0x7fc2c806e9f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc033000 tx=0x7fc2dc009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:22.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.157+0000 7fc2e5600640 1 -- 172.21.15.19:0/2850662501 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc2d4018030 con 0x7fc2e8150140 2024-03-28T13:10:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.360+0000 7fc2eec00640 1 -- 172.21.15.19:0/2850662501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2e81f7580 con 0x7fc2e8150140 2024-03-28T13:10:22.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.363+0000 7fc2e5600640 1 -- 172.21.15.19:0/2850662501 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+16704 (secure 0 0 0) 0x7fc2d4059f40 con 0x7fc2e8150140 2024-03-28T13:10:22.363 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:22.364 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":24,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:10:22.131516+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":0,"to":4}]},{"pgid":"3.1b","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:10:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.366+0000 7fc2eec00640 1 -- 172.21.15.19:0/2850662501 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc2c806c530 msgr2=0x7fc2c806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.366+0000 7fc2eec00640 1 --2- 172.21.15.19:0/2850662501 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc2c806c530 0x7fc2c806e9f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc033000 tx=0x7fc2dc009030 comp rx=0 tx=0).stop 2024-03-28T13:10:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.366+0000 7fc2eec00640 1 -- 172.21.15.19:0/2850662501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2e8150140 msgr2=0x7fc2e814cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.366+0000 7fc2eec00640 1 --2- 172.21.15.19:0/2850662501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2e8150140 0x7fc2e814cd70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc2d40108c0 tx=0x7fc2d4010d80 comp rx=0 tx=0).stop 2024-03-28T13:10:22.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.366+0000 7fc2eec00640 1 -- 172.21.15.19:0/2850662501 shutdown_connections 2024-03-28T13:10:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.366+0000 7fc2eec00640 1 --2- 172.21.15.19:0/2850662501 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc2c806c530 0x7fc2c806e9f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.366+0000 7fc2eec00640 1 --2- 172.21.15.19:0/2850662501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2e80cebd0 0x7fc2e80cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.366+0000 7fc2eec00640 1 --2- 172.21.15.19:0/2850662501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2e8150140 0x7fc2e814cd70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.366+0000 7fc2eec00640 1 --2- 172.21.15.19:0/2850662501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2d00039e0 0x7fc2e81f7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.366+0000 7fc2eec00640 1 -- 172.21.15.19:0/2850662501 >> 172.21.15.19:0/2850662501 conn(0x7fc2e80678b0 msgr2=0x7fc2e80b8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.366+0000 7fc2eec00640 1 -- 172.21.15.19:0/2850662501 shutdown_connections 2024-03-28T13:10:22.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.366+0000 7fc2eec00640 1 -- 172.21.15.19:0/2850662501 wait complete. 2024-03-28T13:10:22.386 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:09:39.243813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:09:40.092899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:10:22.386 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:10:22.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.594+0000 7f9381000640 1 Processor -- start 2024-03-28T13:10:22.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.594+0000 7f9381000640 1 -- start start 2024-03-28T13:10:22.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.596+0000 7f9381000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f937c144450 0x7f937c06ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:22.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.596+0000 7f9381000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f937c15be20 0x7f937c06c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:22.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.596+0000 7f9381000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f937c15e8c0 con 0x7f937c06cac0 2024-03-28T13:10:22.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.596+0000 7f9381000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f937c144830 con 0x7f937c15be20 2024-03-28T13:10:22.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.596+0000 7f9381000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f937c15e670 con 0x7f937c144450 2024-03-28T13:10:22.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.596+0000 7f9379400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f937c15be20 0x7f937c06c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:22.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.597+0000 7f9379400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f937c15be20 0x7f937c06c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53974/0 (socket says 172.21.15.19:53974) 2024-03-28T13:10:22.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.597+0000 7f9379400640 1 -- 172.21.15.19:0/1828624172 learned_addr learned my addr 172.21.15.19:0/1828624172 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:22.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.597+0000 7f9379e00640 1 --2- 172.21.15.19:0/1828624172 >> v2:172.21.15.188:3300/0 conn(0x7f937c144450 0x7f937c06ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:22.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.597+0000 7f9379400640 1 -- 172.21.15.19:0/1828624172 >> v2:172.21.15.188:3300/0 conn(0x7f937c144450 msgr2=0x7f937c06ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:22.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.597+0000 7f9379400640 1 --2- 172.21.15.19:0/1828624172 >> v2:172.21.15.188:3300/0 conn(0x7f937c144450 0x7f937c06ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.597+0000 7f9379400640 1 -- 172.21.15.19:0/1828624172 >> v1:172.21.15.188:6789/0 conn(0x7f937c06cac0 legacy=0x7f937c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:22.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.597+0000 7f9379400640 1 -- 172.21.15.19:0/1828624172 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c15eba0 con 0x7f937c15be20 2024-03-28T13:10:22.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.597+0000 7f9379e00640 1 --2- 172.21.15.19:0/1828624172 >> v2:172.21.15.188:3300/0 conn(0x7f937c144450 0x7f937c06ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:22.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.599+0000 7f9379400640 1 --2- 172.21.15.19:0/1828624172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c15be20 0x7f937c06c580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f937c0729f0 tx=0x7f936802de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13be56509b307aac server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:22.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.599+0000 7f9378a00640 1 -- 172.21.15.19:0/1828624172 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9368034070 con 0x7f937c15be20 2024-03-28T13:10:22.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.600+0000 7f9378a00640 1 -- 172.21.15.19:0/1828624172 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f936802ebd0 con 0x7f937c15be20 2024-03-28T13:10:22.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.600+0000 7f9378a00640 1 -- 172.21.15.19:0/1828624172 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9368038400 con 0x7f937c15be20 2024-03-28T13:10:22.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.600+0000 7f9381000640 1 -- 172.21.15.19:0/1828624172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c15be20 msgr2=0x7f937c06c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:22.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.600+0000 7f9381000640 1 --2- 172.21.15.19:0/1828624172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c15be20 0x7f937c06c580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f937c0729f0 tx=0x7f936802de20 comp rx=0 tx=0).stop 2024-03-28T13:10:22.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.600+0000 7f9381000640 1 -- 172.21.15.19:0/1828624172 shutdown_connections 2024-03-28T13:10:22.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.600+0000 7f9381000640 1 --2- 172.21.15.19:0/1828624172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c15be20 0x7f937c06c580 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.600+0000 7f9381000640 1 --2- 172.21.15.19:0/1828624172 >> v2:172.21.15.188:3300/0 conn(0x7f937c144450 0x7f937c06ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.600+0000 7f9381000640 1 -- 172.21.15.19:0/1828624172 >> 172.21.15.19:0/1828624172 conn(0x7f937c071e20 msgr2=0x7f937c072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:22.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.600+0000 7f9381000640 1 -- 172.21.15.19:0/1828624172 shutdown_connections 2024-03-28T13:10:22.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.600+0000 7f9381000640 1 -- 172.21.15.19:0/1828624172 wait complete. 2024-03-28T13:10:22.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.603+0000 7f9381000640 1 Processor -- start 2024-03-28T13:10:22.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.603+0000 7f9381000640 1 -- start start 2024-03-28T13:10:22.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.605+0000 7f9381000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f937c06cac0 0x7f937c159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:22.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.605+0000 7f9381000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f937c144450 0x7f937c13d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:22.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.605+0000 7f9381000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c15be20 0x7f937c13d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:22.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.605+0000 7f9381000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f937c06f160 con 0x7f937c15be20 2024-03-28T13:10:22.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.605+0000 7f9381000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f937c15f160 con 0x7f937c06cac0 2024-03-28T13:10:22.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.605+0000 7f9381000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f937c06f2e0 con 0x7f937c144450 2024-03-28T13:10:22.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.605+0000 7f937a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c15be20 0x7f937c13d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:22.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.605+0000 7f937a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c15be20 0x7f937c13d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53982/0 (socket says 172.21.15.19:53982) 2024-03-28T13:10:22.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.605+0000 7f937a800640 1 -- 172.21.15.19:0/2365738073 learned_addr learned my addr 172.21.15.19:0/2365738073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:22.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.606+0000 7f9379e00640 1 --2- 172.21.15.19:0/2365738073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f937c06cac0 0x7f937c159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:22.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.606+0000 7f9379400640 1 --2- 172.21.15.19:0/2365738073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f937c144450 0x7f937c13d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:22.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.606+0000 7f937a800640 1 -- 172.21.15.19:0/2365738073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f937c144450 msgr2=0x7f937c13d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:22.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.606+0000 7f937a800640 1 --2- 172.21.15.19:0/2365738073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f937c144450 0x7f937c13d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.606+0000 7f937a800640 1 -- 172.21.15.19:0/2365738073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f937c06cac0 msgr2=0x7f937c159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:22.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.606+0000 7f937a800640 1 --2- 172.21.15.19:0/2365738073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f937c06cac0 0x7f937c159f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.606+0000 7f937a800640 1 -- 172.21.15.19:0/2365738073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c0cb070 con 0x7f937c15be20 2024-03-28T13:10:22.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.606+0000 7f937a800640 1 --2- 172.21.15.19:0/2365738073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c15be20 0x7f937c13d720 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f937400b9a0 tx=0x7f937400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:22.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.606+0000 7f9372000640 1 -- 172.21.15.19:0/2365738073 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374019070 con 0x7f937c15be20 2024-03-28T13:10:22.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.606+0000 7f9372000640 1 -- 172.21.15.19:0/2365738073 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f937400cb40 con 0x7f937c15be20 2024-03-28T13:10:22.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.606+0000 7f9381000640 1 -- 172.21.15.19:0/2365738073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937c0cb310 con 0x7f937c15be20 2024-03-28T13:10:22.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.607+0000 7f9372000640 1 -- 172.21.15.19:0/2365738073 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374014440 con 0x7f937c15be20 2024-03-28T13:10:22.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.607+0000 7f9381000640 1 -- 172.21.15.19:0/2365738073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937c13dd60 con 0x7f937c15be20 2024-03-28T13:10:22.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.608+0000 7f9381000640 1 -- 172.21.15.19:0/2365738073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f937c068cd0 con 0x7f937c15be20 2024-03-28T13:10:22.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.609+0000 7f9372000640 1 -- 172.21.15.19:0/2365738073 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f937401c070 con 0x7f937c15be20 2024-03-28T13:10:22.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.610+0000 7f9372000640 1 --2- 172.21.15.19:0/2365738073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f935c06c530 0x7f935c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:22.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.610+0000 7f9372000640 1 -- 172.21.15.19:0/2365738073 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5767+0+0 (secure 0 0 0) 0x7f937408b300 con 0x7f937c15be20 2024-03-28T13:10:22.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.610+0000 7f9379e00640 1 --2- 172.21.15.19:0/2365738073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f935c06c530 0x7f935c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:22.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.612+0000 7f9379e00640 1 --2- 172.21.15.19:0/2365738073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f935c06c530 0x7f935c06e9f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f937c15d710 tx=0x7f936c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:22.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.614+0000 7f9372000640 1 -- 172.21.15.19:0/2365738073 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f93740598e0 con 0x7f937c15be20 2024-03-28T13:10:22.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.841+0000 7f9381000640 1 -- 172.21.15.19:0/2365738073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f937c0d0020 con 0x7f937c15be20 2024-03-28T13:10:22.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.842+0000 7f9372000640 1 -- 172.21.15.19:0/2365738073 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) ==== 93+0+10 (secure 0 0 0) 0x7f93740591e0 con 0x7f937c15be20 2024-03-28T13:10:22.842 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:10:22.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.845+0000 7f9381000640 1 -- 172.21.15.19:0/2365738073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f935c06c530 msgr2=0x7f935c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:22.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.845+0000 7f9381000640 1 --2- 172.21.15.19:0/2365738073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f935c06c530 0x7f935c06e9f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f937c15d710 tx=0x7f936c008040 comp rx=0 tx=0).stop 2024-03-28T13:10:22.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.845+0000 7f9381000640 1 -- 172.21.15.19:0/2365738073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c15be20 msgr2=0x7f937c13d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:22.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.845+0000 7f9381000640 1 --2- 172.21.15.19:0/2365738073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c15be20 0x7f937c13d720 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f937400b9a0 tx=0x7f937400be60 comp rx=0 tx=0).stop 2024-03-28T13:10:22.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.845+0000 7f9381000640 1 -- 172.21.15.19:0/2365738073 shutdown_connections 2024-03-28T13:10:22.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.845+0000 7f9381000640 1 --2- 172.21.15.19:0/2365738073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f935c06c530 0x7f935c06e9f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.845+0000 7f9381000640 1 --2- 172.21.15.19:0/2365738073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f937c15be20 0x7f937c13d720 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.845+0000 7f9381000640 1 --2- 172.21.15.19:0/2365738073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f937c144450 0x7f937c13d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.845+0000 7f9381000640 1 --2- 172.21.15.19:0/2365738073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f937c06cac0 0x7f937c159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:22.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.845+0000 7f9381000640 1 -- 172.21.15.19:0/2365738073 >> 172.21.15.19:0/2365738073 conn(0x7f937c071e20 msgr2=0x7f937c06e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:22.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.845+0000 7f9381000640 1 -- 172.21.15.19:0/2365738073 shutdown_connections 2024-03-28T13:10:22.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:22.846+0000 7f9381000640 1 -- 172.21.15.19:0/2365738073 wait complete. 2024-03-28T13:10:22.866 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:10:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.077+0000 7f6371e00640 1 Processor -- start 2024-03-28T13:10:23.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.078+0000 7f6371e00640 1 -- start start 2024-03-28T13:10:23.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.079+0000 7f6371e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f636c162920 0x7f636c1670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:23.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.079+0000 7f6371e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f636c071d60 con 0x7f636c0d0990 2024-03-28T13:10:23.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.079+0000 7f6371e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f636c1665d0 con 0x7f636c163ac0 2024-03-28T13:10:23.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.079+0000 7f6371e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f636c163f20 con 0x7f636c162920 2024-03-28T13:10:23.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.080+0000 7f636b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f636c0d0990 0x7f636c0ad010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48094/0 (socket says 172.21.15.19:48094) 2024-03-28T13:10:23.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.080+0000 7f636aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f636c162920 0x7f636c1670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:23.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.080+0000 7f636b400640 1 -- 172.21.15.19:0/647800751 learned_addr learned my addr 172.21.15.19:0/647800751 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.081+0000 7f636a000640 1 -- 172.21.15.19:0/647800751 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3705875768 0 0) 0x7f636c071d60 con 0x7f636c0d0990 2024-03-28T13:10:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.081+0000 7f636a000640 1 -- 172.21.15.19:0/647800751 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6350003520 con 0x7f636c0d0990 2024-03-28T13:10:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.081+0000 7f636aa00640 1 -- 172.21.15.19:0/647800751 >> v1:172.21.15.188:6789/0 conn(0x7f636c163ac0 legacy=0x7f636c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.081+0000 7f636a000640 1 -- 172.21.15.19:0/647800751 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 729151805 0 0) 0x7f636c1665d0 con 0x7f636c163ac0 2024-03-28T13:10:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.081+0000 7f636aa00640 1 -- 172.21.15.19:0/647800751 >> v1:172.21.15.19:6789/0 conn(0x7f636c0d0990 legacy=0x7f636c0ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:23.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.081+0000 7f636aa00640 1 -- 172.21.15.19:0/647800751 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c1668b0 con 0x7f636c162920 2024-03-28T13:10:23.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.083+0000 7f636aa00640 1 --2- 172.21.15.19:0/647800751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f636c162920 0x7f636c1670b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f635c0097a0 tx=0x7f635c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8589357ad009796d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:23.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.084+0000 7f636a000640 1 -- 172.21.15.19:0/647800751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c035020 con 0x7f636c162920 2024-03-28T13:10:23.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.084+0000 7f636a000640 1 -- 172.21.15.19:0/647800751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f635c031830 con 0x7f636c162920 2024-03-28T13:10:23.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.084+0000 7f636a000640 1 -- 172.21.15.19:0/647800751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c031c00 con 0x7f636c162920 2024-03-28T13:10:23.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.084+0000 7f6371e00640 1 -- 172.21.15.19:0/647800751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f636c162920 msgr2=0x7f636c1670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:23.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.084+0000 7f6371e00640 1 --2- 172.21.15.19:0/647800751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f636c162920 0x7f636c1670b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f635c0097a0 tx=0x7f635c02b8d0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.084+0000 7f6371e00640 1 -- 172.21.15.19:0/647800751 shutdown_connections 2024-03-28T13:10:23.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.084+0000 7f6371e00640 1 --2- 172.21.15.19:0/647800751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f636c162920 0x7f636c1670b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.084+0000 7f6371e00640 1 -- 172.21.15.19:0/647800751 >> 172.21.15.19:0/647800751 conn(0x7f636c06b750 msgr2=0x7f636c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:23.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.084+0000 7f6371e00640 1 -- 172.21.15.19:0/647800751 shutdown_connections 2024-03-28T13:10:23.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.084+0000 7f6371e00640 1 -- 172.21.15.19:0/647800751 wait complete. 2024-03-28T13:10:23.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.086+0000 7f6371e00640 1 Processor -- start 2024-03-28T13:10:23.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.086+0000 7f6371e00640 1 -- start start 2024-03-28T13:10:23.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.087+0000 7f6371e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f636c0d0990 0x7f636c161a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:23.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.087+0000 7f6371e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c162920 0x7f636c1f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:23.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.087+0000 7f6371e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c163ac0 0x7f636c0ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:23.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.087+0000 7f6371e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f636c072040 con 0x7f636c0d0990 2024-03-28T13:10:23.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.087+0000 7f6371e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f636c162d00 con 0x7f636c163ac0 2024-03-28T13:10:23.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.087+0000 7f6371e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f636c0721c0 con 0x7f636c162920 2024-03-28T13:10:23.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.087+0000 7f636aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f636c0d0990 0x7f636c161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:23.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.088+0000 7f636aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f636c0d0990 0x7f636c161a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53998/0 (socket says 172.21.15.19:53998) 2024-03-28T13:10:23.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.088+0000 7f636aa00640 1 -- 172.21.15.19:0/3088648526 learned_addr learned my addr 172.21.15.19:0/3088648526 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.088+0000 7f6363e00640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c162920 0x7f636c1f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.088+0000 7f636b400640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c163ac0 0x7f636c0ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.088+0000 7f636aa00640 1 -- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c162920 msgr2=0x7f636c1f8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:23.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.088+0000 7f636aa00640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c162920 0x7f636c1f8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.088+0000 7f636aa00640 1 -- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c163ac0 msgr2=0x7f636c0ad0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.088+0000 7f636aa00640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c163ac0 0x7f636c0ad0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.088+0000 7f636aa00640 1 -- 172.21.15.19:0/3088648526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c1f90a0 con 0x7f636c0d0990 2024-03-28T13:10:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.088+0000 7f6363e00640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c162920 0x7f636c1f8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.088+0000 7f636b400640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c163ac0 0x7f636c0ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.089+0000 7f636aa00640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f636c0d0990 0x7f636c161a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f635c0098c0 tx=0x7f635c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.089+0000 7f6363400640 1 -- 172.21.15.19:0/3088648526 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c035040 con 0x7f636c0d0990 2024-03-28T13:10:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.089+0000 7f6363400640 1 -- 172.21.15.19:0/3088648526 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f635c03a840 con 0x7f636c0d0990 2024-03-28T13:10:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.089+0000 7f6363400640 1 -- 172.21.15.19:0/3088648526 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f635c03ac10 con 0x7f636c0d0990 2024-03-28T13:10:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.089+0000 7f6371e00640 1 -- 172.21.15.19:0/3088648526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636c15d750 con 0x7f636c0d0990 2024-03-28T13:10:23.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.089+0000 7f6371e00640 1 -- 172.21.15.19:0/3088648526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636c15dd10 con 0x7f636c0d0990 2024-03-28T13:10:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.090+0000 7f6371e00640 1 -- 172.21.15.19:0/3088648526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f636c0685c0 con 0x7f636c0d0990 2024-03-28T13:10:23.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.091+0000 7f6363400640 1 -- 172.21.15.19:0/3088648526 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f635c038020 con 0x7f636c0d0990 2024-03-28T13:10:23.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.092+0000 7f6363400640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f634806c530 0x7f634806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:23.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.092+0000 7f6363400640 1 -- 172.21.15.19:0/3088648526 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5767+0+0 (secure 0 0 0) 0x7f635c0ad960 con 0x7f636c0d0990 2024-03-28T13:10:23.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.093+0000 7f6363e00640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f634806c530 0x7f634806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:23.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.094+0000 7f6363e00640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f634806c530 0x7f634806e9f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f636c15f2d0 tx=0x7f6354009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:23.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.097+0000 7f6363400640 1 -- 172.21.15.19:0/3088648526 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f636c1f90a0 con 0x7f636c0d0990 2024-03-28T13:10:23.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.312+0000 7f6371e00640 1 -- 172.21.15.19:0/3088648526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f636c06ef60 con 0x7f636c0d0990 2024-03-28T13:10:23.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.313+0000 7f6363400640 1 -- 172.21.15.19:0/3088648526 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f635c044380 con 0x7f636c0d0990 2024-03-28T13:10:23.313 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:10:23.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.315+0000 7f6371e00640 1 -- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f634806c530 msgr2=0x7f634806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:23.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.315+0000 7f6371e00640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f634806c530 0x7f634806e9f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f636c15f2d0 tx=0x7f6354009040 comp rx=0 tx=0).stop 2024-03-28T13:10:23.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.315+0000 7f6371e00640 1 -- 172.21.15.19:0/3088648526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f636c0d0990 msgr2=0x7f636c161a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:23.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.315+0000 7f6371e00640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f636c0d0990 0x7f636c161a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f635c0098c0 tx=0x7f635c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.315+0000 7f6371e00640 1 -- 172.21.15.19:0/3088648526 shutdown_connections 2024-03-28T13:10:23.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.315+0000 7f6371e00640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f634806c530 0x7f634806e9f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.315+0000 7f6371e00640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f636c163ac0 0x7f636c0ad0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.316+0000 7f6371e00640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f636c162920 0x7f636c1f8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.316+0000 7f6371e00640 1 --2- 172.21.15.19:0/3088648526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f636c0d0990 0x7f636c161a20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.316+0000 7f6371e00640 1 -- 172.21.15.19:0/3088648526 >> 172.21.15.19:0/3088648526 conn(0x7f636c06b750 msgr2=0x7f636c0b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:23.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.316+0000 7f6371e00640 1 -- 172.21.15.19:0/3088648526 shutdown_connections 2024-03-28T13:10:23.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.316+0000 7f6371e00640 1 -- 172.21.15.19:0/3088648526 wait complete. 2024-03-28T13:10:23.334 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:10:23.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.541+0000 7f7ea9800640 1 Processor -- start 2024-03-28T13:10:23.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.541+0000 7f7ea9800640 1 -- start start 2024-03-28T13:10:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.542+0000 7f7ea9800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ea40d37b0 0x7f7ea40b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.542+0000 7f7ea9800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7ea40b67f0 0x7f7ea40b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.542+0000 7f7ea9800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ea40d2710 con 0x7f7ea415e4c0 2024-03-28T13:10:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.542+0000 7f7ea9800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7ea415e8a0 con 0x7f7ea40d37b0 2024-03-28T13:10:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.542+0000 7f7ea9800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7ea40d4040 con 0x7f7ea40b67f0 2024-03-28T13:10:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.543+0000 7f7ea3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7ea415e4c0 0x7f7ea40b9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48110/0 (socket says 172.21.15.19:48110) 2024-03-28T13:10:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.543+0000 7f7ea3e00640 1 -- 172.21.15.19:0/60746811 learned_addr learned my addr 172.21.15.19:0/60746811 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:23.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.543+0000 7f7ea3400640 1 --2- 172.21.15.19:0/60746811 >> v2:172.21.15.19:3300/0 conn(0x7f7ea40d37b0 0x7f7ea40b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:23.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.543+0000 7f7ea8e00640 1 --2- 172.21.15.19:0/60746811 >> v2:172.21.15.188:3300/0 conn(0x7f7ea40b67f0 0x7f7ea40b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:23.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.543+0000 7f7ea3400640 1 -- 172.21.15.19:0/60746811 >> v2:172.21.15.188:3300/0 conn(0x7f7ea40b67f0 msgr2=0x7f7ea40b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:23.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.543+0000 7f7ea3400640 1 --2- 172.21.15.19:0/60746811 >> v2:172.21.15.188:3300/0 conn(0x7f7ea40b67f0 0x7f7ea40b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.543+0000 7f7ea3400640 1 -- 172.21.15.19:0/60746811 >> v1:172.21.15.19:6789/0 conn(0x7f7ea415e4c0 legacy=0x7f7ea40b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:23.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.543+0000 7f7ea3400640 1 -- 172.21.15.19:0/60746811 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea40d29f0 con 0x7f7ea40d37b0 2024-03-28T13:10:23.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.543+0000 7f7ea8e00640 1 --2- 172.21.15.19:0/60746811 >> v2:172.21.15.188:3300/0 conn(0x7f7ea40b67f0 0x7f7ea40b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:23.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.543+0000 7f7ea2a00640 1 -- 172.21.15.19:0/60746811 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3374784791 0 0) 0x7f7ea40d2710 con 0x7f7ea415e4c0 2024-03-28T13:10:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.545+0000 7f7ea3400640 1 --2- 172.21.15.19:0/60746811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea40d37b0 0x7f7ea40b62b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40d13c0 tx=0x7f7e9002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a47a85e03e08ad8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.545+0000 7f7ea2a00640 1 -- 172.21.15.19:0/60746811 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90035020 con 0x7f7ea40d37b0 2024-03-28T13:10:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.545+0000 7f7ea2a00640 1 -- 172.21.15.19:0/60746811 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e90031830 con 0x7f7ea40d37b0 2024-03-28T13:10:23.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.545+0000 7f7ea2a00640 1 -- 172.21.15.19:0/60746811 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90031c00 con 0x7f7ea40d37b0 2024-03-28T13:10:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.545+0000 7f7ea9800640 1 -- 172.21.15.19:0/60746811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea40d37b0 msgr2=0x7f7ea40b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.545+0000 7f7ea9800640 1 --2- 172.21.15.19:0/60746811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea40d37b0 0x7f7ea40b62b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7ea40d13c0 tx=0x7f7e9002b8d0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.546+0000 7f7ea9800640 1 -- 172.21.15.19:0/60746811 shutdown_connections 2024-03-28T13:10:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.546+0000 7f7ea9800640 1 --2- 172.21.15.19:0/60746811 >> v2:172.21.15.188:3300/0 conn(0x7f7ea40b67f0 0x7f7ea40b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.546+0000 7f7ea9800640 1 --2- 172.21.15.19:0/60746811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea40d37b0 0x7f7ea40b62b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.546+0000 7f7ea9800640 1 -- 172.21.15.19:0/60746811 >> 172.21.15.19:0/60746811 conn(0x7f7ea40676a0 msgr2=0x7f7ea4067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.546+0000 7f7ea9800640 1 -- 172.21.15.19:0/60746811 shutdown_connections 2024-03-28T13:10:23.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.546+0000 7f7ea9800640 1 -- 172.21.15.19:0/60746811 wait complete. 2024-03-28T13:10:23.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.547+0000 7f7ea9800640 1 Processor -- start 2024-03-28T13:10:23.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.548+0000 7f7ea9800640 1 -- start start 2024-03-28T13:10:23.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.548+0000 7f7ea9800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea40b67f0 0x7f7ea40b9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:23.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.548+0000 7f7ea9800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ea40d37b0 0x7f7ea4161040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:23.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.548+0000 7f7ea9800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea415e4c0 0x7f7ea414d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:23.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.548+0000 7f7ea9800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea40b6c20 con 0x7f7ea40b67f0 2024-03-28T13:10:23.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.548+0000 7f7ea9800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea40d3320 con 0x7f7ea40d37b0 2024-03-28T13:10:23.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.548+0000 7f7ea9800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea40d2fb0 con 0x7f7ea415e4c0 2024-03-28T13:10:23.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.549+0000 7f7ea3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea40b67f0 0x7f7ea40b9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:23.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.549+0000 7f7ea3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea40b67f0 0x7f7ea40b9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54022/0 (socket says 172.21.15.19:54022) 2024-03-28T13:10:23.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.549+0000 7f7ea3e00640 1 -- 172.21.15.19:0/2500409392 learned_addr learned my addr 172.21.15.19:0/2500409392 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:23.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.549+0000 7f7ea3400640 1 --2- 172.21.15.19:0/2500409392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ea40d37b0 0x7f7ea4161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:23.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.549+0000 7f7ea3e00640 1 -- 172.21.15.19:0/2500409392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea415e4c0 msgr2=0x7f7ea414d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:23.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.549+0000 7f7ea3e00640 1 --2- 172.21.15.19:0/2500409392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea415e4c0 0x7f7ea414d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.549+0000 7f7ea3e00640 1 -- 172.21.15.19:0/2500409392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ea40d37b0 msgr2=0x7f7ea4161040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:23.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.549+0000 7f7ea3e00640 1 --2- 172.21.15.19:0/2500409392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ea40d37b0 0x7f7ea4161040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.549+0000 7f7ea3e00640 1 -- 172.21.15.19:0/2500409392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea41f7380 con 0x7f7ea40b67f0 2024-03-28T13:10:23.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.549+0000 7f7ea3400640 1 --2- 172.21.15.19:0/2500409392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ea40d37b0 0x7f7ea4161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:23.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.550+0000 7f7ea3e00640 1 --2- 172.21.15.19:0/2500409392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea40b67f0 0x7f7ea40b9640 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400c4d0 tx=0x7f7e9400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:23.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.550+0000 7f7e9be00640 1 -- 172.21.15.19:0/2500409392 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94018070 con 0x7f7ea40b67f0 2024-03-28T13:10:23.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.550+0000 7f7e9be00640 1 -- 172.21.15.19:0/2500409392 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e94007500 con 0x7f7ea40b67f0 2024-03-28T13:10:23.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.550+0000 7f7ea9800640 1 -- 172.21.15.19:0/2500409392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea41f7660 con 0x7f7ea40b67f0 2024-03-28T13:10:23.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.550+0000 7f7e9be00640 1 -- 172.21.15.19:0/2500409392 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e94004a20 con 0x7f7ea40b67f0 2024-03-28T13:10:23.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.550+0000 7f7ea9800640 1 -- 172.21.15.19:0/2500409392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea41f7c20 con 0x7f7ea40b67f0 2024-03-28T13:10:23.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.551+0000 7f7ea9800640 1 -- 172.21.15.19:0/2500409392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea4075940 con 0x7f7ea40b67f0 2024-03-28T13:10:23.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.552+0000 7f7e9be00640 1 -- 172.21.15.19:0/2500409392 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7e9401b070 con 0x7f7ea40b67f0 2024-03-28T13:10:23.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.554+0000 7f7e9be00640 1 --2- 172.21.15.19:0/2500409392 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e8406c530 0x7f7e8406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:23.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.554+0000 7f7e9be00640 1 -- 172.21.15.19:0/2500409392 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f7e9408aaf0 con 0x7f7ea40b67f0 2024-03-28T13:10:23.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.555+0000 7f7ea3400640 1 --2- 172.21.15.19:0/2500409392 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e8406c530 0x7f7e8406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:23.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.556+0000 7f7ea3400640 1 --2- 172.21.15.19:0/2500409392 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e8406c530 0x7f7e8406e9f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7e90036000 tx=0x7f7e90008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:23.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.558+0000 7f7e9be00640 1 -- 172.21.15.19:0/2500409392 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7e94059030 con 0x7f7ea40b67f0 2024-03-28T13:10:23.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.771+0000 7f7ea9800640 1 -- 172.21.15.19:0/2500409392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7ea40d4e30 con 0x7f7ea40b67f0 2024-03-28T13:10:23.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.772+0000 7f7e9be00640 1 -- 172.21.15.19:0/2500409392 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f7e94058930 con 0x7f7ea40b67f0 2024-03-28T13:10:23.772 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:10:23.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.776+0000 7f7ea9800640 1 -- 172.21.15.19:0/2500409392 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e8406c530 msgr2=0x7f7e8406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.776+0000 7f7ea9800640 1 --2- 172.21.15.19:0/2500409392 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e8406c530 0x7f7e8406e9f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7e90036000 tx=0x7f7e90008030 comp rx=0 tx=0).stop 2024-03-28T13:10:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.776+0000 7f7ea9800640 1 -- 172.21.15.19:0/2500409392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea40b67f0 msgr2=0x7f7ea40b9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.776+0000 7f7ea9800640 1 --2- 172.21.15.19:0/2500409392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea40b67f0 0x7f7ea40b9640 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400c4d0 tx=0x7f7e9400c990 comp rx=0 tx=0).stop 2024-03-28T13:10:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.776+0000 7f7ea9800640 1 -- 172.21.15.19:0/2500409392 shutdown_connections 2024-03-28T13:10:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.776+0000 7f7ea9800640 1 --2- 172.21.15.19:0/2500409392 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e8406c530 0x7f7e8406e9f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.776+0000 7f7ea9800640 1 --2- 172.21.15.19:0/2500409392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea415e4c0 0x7f7ea414d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.776+0000 7f7ea9800640 1 --2- 172.21.15.19:0/2500409392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ea40d37b0 0x7f7ea4161040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.776+0000 7f7ea9800640 1 --2- 172.21.15.19:0/2500409392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ea40b67f0 0x7f7ea40b9640 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.776+0000 7f7ea9800640 1 -- 172.21.15.19:0/2500409392 >> 172.21.15.19:0/2500409392 conn(0x7f7ea40676a0 msgr2=0x7f7ea415b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.776+0000 7f7ea9800640 1 -- 172.21.15.19:0/2500409392 shutdown_connections 2024-03-28T13:10:23.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:23.776+0000 7f7ea9800640 1 -- 172.21.15.19:0/2500409392 wait complete. 2024-03-28T13:10:23.804 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-28T13:10:23.804 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:10:24.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.014+0000 7f727a200640 1 Processor -- start 2024-03-28T13:10:24.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.015+0000 7f727a200640 1 -- start start 2024-03-28T13:10:24.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.016+0000 7f727a200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f727415c390 0x7f7274073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.016+0000 7f727a200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f727415fe80 con 0x7f72740d0650 2024-03-28T13:10:24.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.016+0000 7f727a200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72740ad120 con 0x7f72740af830 2024-03-28T13:10:24.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.016+0000 7f727a200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f727415c770 con 0x7f727415c390 2024-03-28T13:10:24.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.017+0000 7f7273e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f72740d0650 0x7f7274071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48116/0 (socket says 172.21.15.19:48116) 2024-03-28T13:10:24.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.017+0000 7f7273e00640 1 -- 172.21.15.19:0/533253030 learned_addr learned my addr 172.21.15.19:0/533253030 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:24.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.017+0000 7f7273400640 1 --2- 172.21.15.19:0/533253030 >> v2:172.21.15.188:3300/0 conn(0x7f727415c390 0x7f7274073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.017+0000 7f7272a00640 1 -- 172.21.15.19:0/533253030 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1276027042 0 0) 0x7f727415fe80 con 0x7f72740d0650 2024-03-28T13:10:24.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.017+0000 7f7272a00640 1 -- 172.21.15.19:0/533253030 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7258003520 con 0x7f72740d0650 2024-03-28T13:10:24.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.018+0000 7f7273400640 1 -- 172.21.15.19:0/533253030 >> v1:172.21.15.188:6789/0 conn(0x7f72740af830 legacy=0x7f7274160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.018+0000 7f7273400640 1 -- 172.21.15.19:0/533253030 >> v1:172.21.15.19:6789/0 conn(0x7f72740d0650 legacy=0x7f7274071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.018+0000 7f7273400640 1 -- 172.21.15.19:0/533253030 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72740ad400 con 0x7f727415c390 2024-03-28T13:10:24.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.019+0000 7f7273400640 1 --2- 172.21.15.19:0/533253030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f727415c390 0x7f7274073790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f72640097a0 tx=0x7f726402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecff7ca03ea3ab7c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:24.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.020+0000 7f7272a00640 1 -- 172.21.15.19:0/533253030 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264034070 con 0x7f727415c390 2024-03-28T13:10:24.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.020+0000 7f7272a00640 1 -- 172.21.15.19:0/533253030 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7264030610 con 0x7f727415c390 2024-03-28T13:10:24.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.020+0000 7f7272a00640 1 -- 172.21.15.19:0/533253030 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72640309e0 con 0x7f727415c390 2024-03-28T13:10:24.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.020+0000 7f727a200640 1 -- 172.21.15.19:0/533253030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f727415c390 msgr2=0x7f7274073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.020+0000 7f727a200640 1 --2- 172.21.15.19:0/533253030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f727415c390 0x7f7274073790 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f72640097a0 tx=0x7f726402b960 comp rx=0 tx=0).stop 2024-03-28T13:10:24.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.020+0000 7f727a200640 1 -- 172.21.15.19:0/533253030 shutdown_connections 2024-03-28T13:10:24.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.020+0000 7f727a200640 1 --2- 172.21.15.19:0/533253030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f727415c390 0x7f7274073790 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.020+0000 7f727a200640 1 -- 172.21.15.19:0/533253030 >> 172.21.15.19:0/533253030 conn(0x7f727406b990 msgr2=0x7f727406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:24.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.020+0000 7f727a200640 1 -- 172.21.15.19:0/533253030 shutdown_connections 2024-03-28T13:10:24.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.020+0000 7f727a200640 1 -- 172.21.15.19:0/533253030 wait complete. 2024-03-28T13:10:24.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.022+0000 7f727a200640 1 Processor -- start 2024-03-28T13:10:24.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.022+0000 7f727a200640 1 -- start start 2024-03-28T13:10:24.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.023+0000 7f727a200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72740af830 0x7f7274150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.023+0000 7f727a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72740d0650 0x7f727415d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.023+0000 7f727a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f727415c390 0x7f727415df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.023+0000 7f727a200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7274160d50 con 0x7f72740d0650 2024-03-28T13:10:24.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.023+0000 7f727a200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7274160120 con 0x7f72740af830 2024-03-28T13:10:24.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.023+0000 7f727a200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f72740cf980 con 0x7f727415c390 2024-03-28T13:10:24.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.024+0000 7f726be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72740d0650 0x7f727415d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.024+0000 7f726be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72740d0650 0x7f727415d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54024/0 (socket says 172.21.15.19:54024) 2024-03-28T13:10:24.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.024+0000 7f726be00640 1 -- 172.21.15.19:0/1646263346 learned_addr learned my addr 172.21.15.19:0/1646263346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:24.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.024+0000 7f7273400640 1 --2- 172.21.15.19:0/1646263346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72740af830 0x7f7274150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.024+0000 7f7273e00640 1 --2- 172.21.15.19:0/1646263346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f727415c390 0x7f727415df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.024+0000 7f726be00640 1 -- 172.21.15.19:0/1646263346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f727415c390 msgr2=0x7f727415df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.024+0000 7f726be00640 1 --2- 172.21.15.19:0/1646263346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f727415c390 0x7f727415df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.024+0000 7f726be00640 1 -- 172.21.15.19:0/1646263346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72740af830 msgr2=0x7f7274150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.024+0000 7f726be00640 1 --2- 172.21.15.19:0/1646263346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72740af830 0x7f7274150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.024+0000 7f726be00640 1 -- 172.21.15.19:0/1646263346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72741f4750 con 0x7f72740d0650 2024-03-28T13:10:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.024+0000 7f7273400640 1 --2- 172.21.15.19:0/1646263346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72740af830 0x7f7274150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.024+0000 7f726be00640 1 --2- 172.21.15.19:0/1646263346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72740d0650 0x7f727415d9c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f725c00c450 tx=0x7f725c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.025+0000 7f726b400640 1 -- 172.21.15.19:0/1646263346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c0052d0 con 0x7f72740d0650 2024-03-28T13:10:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.025+0000 7f726b400640 1 -- 172.21.15.19:0/1646263346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f725c005470 con 0x7f72740d0650 2024-03-28T13:10:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.025+0000 7f726b400640 1 -- 172.21.15.19:0/1646263346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c012870 con 0x7f72740d0650 2024-03-28T13:10:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.025+0000 7f727a200640 1 -- 172.21.15.19:0/1646263346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72741f4a30 con 0x7f72740d0650 2024-03-28T13:10:24.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.025+0000 7f727a200640 1 -- 172.21.15.19:0/1646263346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72741f4ff0 con 0x7f72740d0650 2024-03-28T13:10:24.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.026+0000 7f727a200640 1 -- 172.21.15.19:0/1646263346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72740685c0 con 0x7f72740d0650 2024-03-28T13:10:24.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.028+0000 7f726b400640 1 -- 172.21.15.19:0/1646263346 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f725c012a10 con 0x7f72740d0650 2024-03-28T13:10:24.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.029+0000 7f726b400640 1 --2- 172.21.15.19:0/1646263346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f725006c580 0x7f725006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.030+0000 7f726b400640 1 -- 172.21.15.19:0/1646263346 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f725c08b390 con 0x7f72740d0650 2024-03-28T13:10:24.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.030+0000 7f7273400640 1 --2- 172.21.15.19:0/1646263346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f725006c580 0x7f725006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.032+0000 7f7273400640 1 --2- 172.21.15.19:0/1646263346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f725006c580 0x7f725006ea40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7264036000 tx=0x7f7264008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:24.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.035+0000 7f726b400640 1 -- 172.21.15.19:0/1646263346 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f725c059950 con 0x7f72740d0650 2024-03-28T13:10:24.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.242+0000 7f727a200640 1 -- 172.21.15.19:0/1646263346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f727406f1b0 con 0x7f72740d0650 2024-03-28T13:10:24.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.243+0000 7f726b400640 1 -- 172.21.15.19:0/1646263346 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16704 (secure 0 0 0) 0x7f725c059250 con 0x7f72740d0650 2024-03-28T13:10:24.244 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:24.244 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:10:23.142768+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":0,"to":4}]},{"pgid":"3.1b","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:10:24.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.247+0000 7f727a200640 1 -- 172.21.15.19:0/1646263346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f725006c580 msgr2=0x7f725006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.247+0000 7f727a200640 1 --2- 172.21.15.19:0/1646263346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f725006c580 0x7f725006ea40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7264036000 tx=0x7f7264008030 comp rx=0 tx=0).stop 2024-03-28T13:10:24.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.247+0000 7f727a200640 1 -- 172.21.15.19:0/1646263346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72740d0650 msgr2=0x7f727415d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.247+0000 7f727a200640 1 --2- 172.21.15.19:0/1646263346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72740d0650 0x7f727415d9c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f725c00c450 tx=0x7f725c00c910 comp rx=0 tx=0).stop 2024-03-28T13:10:24.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.247+0000 7f727a200640 1 -- 172.21.15.19:0/1646263346 shutdown_connections 2024-03-28T13:10:24.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.247+0000 7f727a200640 1 --2- 172.21.15.19:0/1646263346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f725006c580 0x7f725006ea40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.247+0000 7f727a200640 1 --2- 172.21.15.19:0/1646263346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f727415c390 0x7f727415df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.247+0000 7f727a200640 1 --2- 172.21.15.19:0/1646263346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72740d0650 0x7f727415d9c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.247+0000 7f727a200640 1 --2- 172.21.15.19:0/1646263346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72740af830 0x7f7274150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.247+0000 7f727a200640 1 -- 172.21.15.19:0/1646263346 >> 172.21.15.19:0/1646263346 conn(0x7f727406b990 msgr2=0x7f72740afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:24.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.247+0000 7f727a200640 1 -- 172.21.15.19:0/1646263346 shutdown_connections 2024-03-28T13:10:24.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.247+0000 7f727a200640 1 -- 172.21.15.19:0/1646263346 wait complete. 2024-03-28T13:10:24.266 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:09:39.243813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:09:40.092899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:10:24.266 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:10:24.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.484+0000 7f9651e00640 1 Processor -- start 2024-03-28T13:10:24.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.484+0000 7f9651e00640 1 -- start start 2024-03-28T13:10:24.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.487+0000 7f9651e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f964c15e460 0x7f964c0b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.487+0000 7f9651e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f964c0d3e40 0x7f964c0d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.487+0000 7f9651e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f964c0b8320 con 0x7f964c0cef20 2024-03-28T13:10:24.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.487+0000 7f9651e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f964c0d2da0 con 0x7f964c15e460 2024-03-28T13:10:24.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.487+0000 7f9651e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f964c0d3080 con 0x7f964c0d3e40 2024-03-28T13:10:24.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.487+0000 7f9651400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f964c0cef20 0x7f964c06c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48122/0 (socket says 172.21.15.19:48122) 2024-03-28T13:10:24.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.487+0000 7f9651400640 1 -- 172.21.15.19:0/1647268438 learned_addr learned my addr 172.21.15.19:0/1647268438 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:24.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.487+0000 7f9650a00640 1 --2- 172.21.15.19:0/1647268438 >> v2:172.21.15.188:3300/0 conn(0x7f964c15e460 0x7f964c0b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964be00640 1 --2- 172.21.15.19:0/1647268438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f964c0d3e40 0x7f964c0d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 -- 172.21.15.19:0/1647268438 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3006151281 0 0) 0x7f964c0b8320 con 0x7f964c0cef20 2024-03-28T13:10:24.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 -- 172.21.15.19:0/1647268438 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9634003520 con 0x7f964c0cef20 2024-03-28T13:10:24.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 -- 172.21.15.19:0/1647268438 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2183841163 0 0) 0x7f9634003520 con 0x7f964c0cef20 2024-03-28T13:10:24.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 -- 172.21.15.19:0/1647268438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f964c0d3e40 msgr2=0x7f964c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:24.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 --2- 172.21.15.19:0/1647268438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f964c0d3e40 0x7f964c0d2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 -- 172.21.15.19:0/1647268438 >> v2:172.21.15.188:3300/0 conn(0x7f964c15e460 msgr2=0x7f964c0b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:24.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 --2- 172.21.15.19:0/1647268438 >> v2:172.21.15.188:3300/0 conn(0x7f964c15e460 0x7f964c0b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 -- 172.21.15.19:0/1647268438 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964c0d3360 con 0x7f964c0cef20 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 -- 172.21.15.19:0/1647268438 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f96440019f0 con 0x7f964c0cef20 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 -- 172.21.15.19:0/1647268438 >> v1:172.21.15.19:6789/0 conn(0x7f964c0cef20 legacy=0x7f964c06c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 --2- 172.21.15.19:0/1647268438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f964c0d3e40 0x7f96340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 --2- 172.21.15.19:0/1647268438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9634003f80 0x7f9634004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 --2- 172.21.15.19:0/1647268438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9634004d40 0x7f96340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 -- 172.21.15.19:0/1647268438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f964c0d2da0 con 0x7f9634003f80 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 -- 172.21.15.19:0/1647268438 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f964c0d3080 con 0x7f9634004d40 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.488+0000 7f964b400640 1 -- 172.21.15.19:0/1647268438 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f963400b9c0 con 0x7f964c0d3e40 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.489+0000 7f964be00640 1 --2- 172.21.15.19:0/1647268438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9634003f80 0x7f9634004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.489+0000 7f9650a00640 1 --2- 172.21.15.19:0/1647268438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f964c0d3e40 0x7f96340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.489+0000 7f9651400640 1 --2- 172.21.15.19:0/1647268438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9634004d40 0x7f96340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.489+0000 7f964be00640 1 -- 172.21.15.19:0/1647268438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f964c0d3e40 msgr2=0x7f96340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.489+0000 7f964be00640 1 --2- 172.21.15.19:0/1647268438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f964c0d3e40 0x7f96340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.489+0000 7f964be00640 1 -- 172.21.15.19:0/1647268438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9634004d40 msgr2=0x7f96340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.489+0000 7f964be00640 1 --2- 172.21.15.19:0/1647268438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9634004d40 0x7f96340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.489+0000 7f964be00640 1 -- 172.21.15.19:0/1647268438 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9634003520 con 0x7f9634003f80 2024-03-28T13:10:24.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.489+0000 7f9651400640 1 --2- 172.21.15.19:0/1647268438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9634004d40 0x7f96340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:24.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.491+0000 7f964be00640 1 --2- 172.21.15.19:0/1647268438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9634003f80 0x7f9634004800 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9638009a80 tx=0x7f9634005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acae9e15456bf4a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:24.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.491+0000 7f964b400640 1 -- 172.21.15.19:0/1647268438 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638009680 con 0x7f9634003f80 2024-03-28T13:10:24.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.492+0000 7f964b400640 1 -- 172.21.15.19:0/1647268438 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9638005990 con 0x7f9634003f80 2024-03-28T13:10:24.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.492+0000 7f964b400640 1 -- 172.21.15.19:0/1647268438 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638005d80 con 0x7f9634003f80 2024-03-28T13:10:24.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.492+0000 7f9651e00640 1 -- 172.21.15.19:0/1647268438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9634003f80 msgr2=0x7f9634004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.492+0000 7f9651e00640 1 --2- 172.21.15.19:0/1647268438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9634003f80 0x7f9634004800 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9638009a80 tx=0x7f9634005830 comp rx=0 tx=0).stop 2024-03-28T13:10:24.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.492+0000 7f9651400640 1 -- 172.21.15.19:0/1647268438 reap_dead start 2024-03-28T13:10:24.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.492+0000 7f9651e00640 1 -- 172.21.15.19:0/1647268438 shutdown_connections 2024-03-28T13:10:24.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.492+0000 7f9651e00640 1 -- 172.21.15.19:0/1647268438 >> 172.21.15.19:0/1647268438 conn(0x7f964c0678b0 msgr2=0x7f964c0d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:24.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.492+0000 7f9651e00640 1 -- 172.21.15.19:0/1647268438 shutdown_connections 2024-03-28T13:10:24.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.492+0000 7f9651e00640 1 -- 172.21.15.19:0/1647268438 wait complete. 2024-03-28T13:10:24.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.494+0000 7f9651e00640 1 Processor -- start 2024-03-28T13:10:24.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.494+0000 7f9651e00640 1 -- start start 2024-03-28T13:10:24.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.495+0000 7f9651e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9634003f80 0x7f964c161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.495+0000 7f9651e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f964c1f67d0 0x7f964c1f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.495+0000 7f9651e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f964c1f5290 0x7f964c0b7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.495+0000 7f9651e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f964c0cf370 con 0x7f964c1f5290 2024-03-28T13:10:24.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.495+0000 7f9651e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f964c0d3920 con 0x7f964c1f67d0 2024-03-28T13:10:24.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.495+0000 7f9651e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f964c0cf4f0 con 0x7f9634003f80 2024-03-28T13:10:24.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.496+0000 7f9651400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f964c1f5290 0x7f964c0b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.496+0000 7f9651400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f964c1f5290 0x7f964c0b7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54036/0 (socket says 172.21.15.19:54036) 2024-03-28T13:10:24.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.496+0000 7f9651400640 1 -- 172.21.15.19:0/1079366696 learned_addr learned my addr 172.21.15.19:0/1079366696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:24.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.496+0000 7f9650a00640 1 --2- 172.21.15.19:0/1079366696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9634003f80 0x7f964c161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.496+0000 7f964be00640 1 --2- 172.21.15.19:0/1079366696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f964c1f67d0 0x7f964c1f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.496+0000 7f9651400640 1 -- 172.21.15.19:0/1079366696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9634003f80 msgr2=0x7f964c161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.496+0000 7f9651400640 1 --2- 172.21.15.19:0/1079366696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9634003f80 0x7f964c161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.496+0000 7f9651400640 1 -- 172.21.15.19:0/1079366696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f964c1f67d0 msgr2=0x7f964c1f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.496+0000 7f9651400640 1 --2- 172.21.15.19:0/1079366696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f964c1f67d0 0x7f964c1f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.496+0000 7f9651400640 1 -- 172.21.15.19:0/1079366696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964c161b60 con 0x7f964c1f5290 2024-03-28T13:10:24.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.496+0000 7f9650a00640 1 --2- 172.21.15.19:0/1079366696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9634003f80 0x7f964c161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:10:24.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.497+0000 7f9651400640 1 --2- 172.21.15.19:0/1079366696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f964c1f5290 0x7f964c0b7310 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f964c0d1fb0 tx=0x7f964400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:24.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.497+0000 7f9648c00640 1 -- 172.21.15.19:0/1079366696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644004d60 con 0x7f964c1f5290 2024-03-28T13:10:24.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.497+0000 7f9648c00640 1 -- 172.21.15.19:0/1079366696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f964400bb80 con 0x7f964c1f5290 2024-03-28T13:10:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.497+0000 7f9648c00640 1 -- 172.21.15.19:0/1079366696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644007070 con 0x7f964c1f5290 2024-03-28T13:10:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.497+0000 7f9651e00640 1 -- 172.21.15.19:0/1079366696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f964c1f5730 con 0x7f964c1f5290 2024-03-28T13:10:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.497+0000 7f9651e00640 1 -- 172.21.15.19:0/1079366696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f964c1f5cf0 con 0x7f964c1f5290 2024-03-28T13:10:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.498+0000 7f9651e00640 1 -- 172.21.15.19:0/1079366696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f964c075940 con 0x7f964c1f5290 2024-03-28T13:10:24.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.499+0000 7f9648c00640 1 -- 172.21.15.19:0/1079366696 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f964401b020 con 0x7f964c1f5290 2024-03-28T13:10:24.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.500+0000 7f9648c00640 1 --2- 172.21.15.19:0/1079366696 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f962c06c530 0x7f962c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.500+0000 7f9648c00640 1 -- 172.21.15.19:0/1079366696 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f964408ba00 con 0x7f964c1f5290 2024-03-28T13:10:24.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.501+0000 7f9650a00640 1 --2- 172.21.15.19:0/1079366696 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f962c06c530 0x7f962c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.502+0000 7f9650a00640 1 --2- 172.21.15.19:0/1079366696 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f962c06c530 0x7f962c06e9f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9640009b50 tx=0x7f9640008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:24.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.505+0000 7f9648c00640 1 -- 172.21.15.19:0/1079366696 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9644059f80 con 0x7f964c1f5290 2024-03-28T13:10:24.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.708+0000 7f9651e00640 1 -- 172.21.15.19:0/1079366696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f964c06b570 con 0x7f964c1f5290 2024-03-28T13:10:24.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.709+0000 7f9648c00640 1 -- 172.21.15.19:0/1079366696 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f9644059880 con 0x7f964c1f5290 2024-03-28T13:10:24.709 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:10:24.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.711+0000 7f9651e00640 1 -- 172.21.15.19:0/1079366696 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f962c06c530 msgr2=0x7f962c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.711+0000 7f9651e00640 1 --2- 172.21.15.19:0/1079366696 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f962c06c530 0x7f962c06e9f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9640009b50 tx=0x7f9640008040 comp rx=0 tx=0).stop 2024-03-28T13:10:24.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.711+0000 7f9651e00640 1 -- 172.21.15.19:0/1079366696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f964c1f5290 msgr2=0x7f964c0b7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.711+0000 7f9651e00640 1 --2- 172.21.15.19:0/1079366696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f964c1f5290 0x7f964c0b7310 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f964c0d1fb0 tx=0x7f964400b4a0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.711+0000 7f9651e00640 1 -- 172.21.15.19:0/1079366696 shutdown_connections 2024-03-28T13:10:24.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.711+0000 7f9651e00640 1 --2- 172.21.15.19:0/1079366696 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f962c06c530 0x7f962c06e9f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.711+0000 7f9651e00640 1 --2- 172.21.15.19:0/1079366696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f964c1f5290 0x7f964c0b7310 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.711+0000 7f9651e00640 1 --2- 172.21.15.19:0/1079366696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f964c1f67d0 0x7f964c1f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.711+0000 7f9651e00640 1 --2- 172.21.15.19:0/1079366696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9634003f80 0x7f964c161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.711+0000 7f9651e00640 1 -- 172.21.15.19:0/1079366696 >> 172.21.15.19:0/1079366696 conn(0x7f964c0678b0 msgr2=0x7f964c0b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:24.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.711+0000 7f9651e00640 1 -- 172.21.15.19:0/1079366696 shutdown_connections 2024-03-28T13:10:24.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.711+0000 7f9651e00640 1 -- 172.21.15.19:0/1079366696 wait complete. 2024-03-28T13:10:24.731 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:10:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.951+0000 7f2fce800640 1 Processor -- start 2024-03-28T13:10:24.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.951+0000 7f2fce800640 1 -- start start 2024-03-28T13:10:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.952+0000 7f2fce800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc813f190 0x7f2fc80cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.953+0000 7f2fce800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc80ce890 con 0x7f2fc806dd30 2024-03-28T13:10:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.953+0000 7f2fce800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc80ceb50 con 0x7f2fc80cc880 2024-03-28T13:10:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.953+0000 7f2fce800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc80cee30 con 0x7f2fc813f190 2024-03-28T13:10:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.953+0000 7f2fcde00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2fc806dd30 0x7f2fc815ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48124/0 (socket says 172.21.15.19:48124) 2024-03-28T13:10:24.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.953+0000 7f2fcde00640 1 -- 172.21.15.19:0/3860343383 learned_addr learned my addr 172.21.15.19:0/3860343383 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.953+0000 7f2fcd400640 1 --2- 172.21.15.19:0/3860343383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc813f190 0x7f2fc80cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.954+0000 7f2fc7e00640 1 -- 172.21.15.19:0/3860343383 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4006192465 0 0) 0x7f2fc80ce890 con 0x7f2fc806dd30 2024-03-28T13:10:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.954+0000 7f2fc7e00640 1 -- 172.21.15.19:0/3860343383 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fb0003520 con 0x7f2fc806dd30 2024-03-28T13:10:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.954+0000 7f2fc7e00640 1 -- 172.21.15.19:0/3860343383 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 383452355 0 0) 0x7f2fb0003520 con 0x7f2fc806dd30 2024-03-28T13:10:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.954+0000 7f2fc7e00640 1 -- 172.21.15.19:0/3860343383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc813f190 msgr2=0x7f2fc80cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:24.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.954+0000 7f2fc7e00640 1 --2- 172.21.15.19:0/3860343383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc813f190 0x7f2fc80cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.954+0000 7f2fc7e00640 1 -- 172.21.15.19:0/3860343383 >> v1:172.21.15.188:6789/0 conn(0x7f2fc80cc880 legacy=0x7f2fc806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.954+0000 7f2fc7e00640 1 -- 172.21.15.19:0/3860343383 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc80cf110 con 0x7f2fc806dd30 2024-03-28T13:10:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.954+0000 7f2fc7e00640 1 -- 172.21.15.19:0/3860343383 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f2fc00019f0 con 0x7f2fc806dd30 2024-03-28T13:10:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.954+0000 7f2fc7e00640 1 -- 172.21.15.19:0/3860343383 >> v1:172.21.15.19:6789/0 conn(0x7f2fc806dd30 legacy=0x7f2fc815ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.954+0000 7f2fc7e00640 1 --2- 172.21.15.19:0/3860343383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc813f190 0x7f2fb00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.954+0000 7f2fc7e00640 1 --2- 172.21.15.19:0/3860343383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fb0003f60 0x7f2fb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.954+0000 7f2fc7e00640 1 --2- 172.21.15.19:0/3860343383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb0004d40 0x7f2fb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.954+0000 7f2fc7e00640 1 -- 172.21.15.19:0/3860343383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc80cee30 con 0x7f2fb0004d40 2024-03-28T13:10:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.954+0000 7f2fc7e00640 1 -- 172.21.15.19:0/3860343383 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb000baa0 con 0x7f2fb0003f60 2024-03-28T13:10:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.954+0000 7f2fc7e00640 1 -- 172.21.15.19:0/3860343383 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb000bca0 con 0x7f2fc813f190 2024-03-28T13:10:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.955+0000 7f2fcde00640 1 --2- 172.21.15.19:0/3860343383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb0004d40 0x7f2fb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.955+0000 7f2fcd400640 1 --2- 172.21.15.19:0/3860343383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc813f190 0x7f2fb00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.955+0000 7f2fcca00640 1 --2- 172.21.15.19:0/3860343383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fb0003f60 0x7f2fb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.955+0000 7f2fcde00640 1 -- 172.21.15.19:0/3860343383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc813f190 msgr2=0x7f2fb00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.955+0000 7f2fcde00640 1 --2- 172.21.15.19:0/3860343383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc813f190 0x7f2fb00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.955+0000 7f2fcde00640 1 -- 172.21.15.19:0/3860343383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fb0003f60 msgr2=0x7f2fb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.955+0000 7f2fcde00640 1 --2- 172.21.15.19:0/3860343383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fb0003f60 0x7f2fb0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.955+0000 7f2fcde00640 1 -- 172.21.15.19:0/3860343383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb0003520 con 0x7f2fb0004d40 2024-03-28T13:10:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.956+0000 7f2fcde00640 1 --2- 172.21.15.19:0/3860343383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb0004d40 0x7f2fb00051c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000a850 tx=0x7f2fb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b21b0589504a0e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.957+0000 7f2fc7e00640 1 -- 172.21.15.19:0/3860343383 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc000a4a0 con 0x7f2fb0004d40 2024-03-28T13:10:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.957+0000 7f2fc7e00640 1 -- 172.21.15.19:0/3860343383 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fc0034650 con 0x7f2fb0004d40 2024-03-28T13:10:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.957+0000 7f2fc7e00640 1 -- 172.21.15.19:0/3860343383 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fc0034a80 con 0x7f2fb0004d40 2024-03-28T13:10:24.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.957+0000 7f2fce800640 1 -- 172.21.15.19:0/3860343383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb0004d40 msgr2=0x7f2fb00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.957+0000 7f2fce800640 1 --2- 172.21.15.19:0/3860343383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb0004d40 0x7f2fb00051c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000a850 tx=0x7f2fb0005830 comp rx=0 tx=0).stop 2024-03-28T13:10:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.957+0000 7f2fcde00640 1 -- 172.21.15.19:0/3860343383 reap_dead start 2024-03-28T13:10:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.957+0000 7f2fce800640 1 -- 172.21.15.19:0/3860343383 shutdown_connections 2024-03-28T13:10:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.957+0000 7f2fce800640 1 -- 172.21.15.19:0/3860343383 >> 172.21.15.19:0/3860343383 conn(0x7f2fc80b1ba0 msgr2=0x7f2fc80b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.957+0000 7f2fce800640 1 -- 172.21.15.19:0/3860343383 shutdown_connections 2024-03-28T13:10:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.958+0000 7f2fce800640 1 -- 172.21.15.19:0/3860343383 wait complete. 2024-03-28T13:10:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.959+0000 7f2fce800640 1 Processor -- start 2024-03-28T13:10:24.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.959+0000 7f2fce800640 1 -- start start 2024-03-28T13:10:24.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.960+0000 7f2fce800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb0004d40 0x7f2fc813d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.960+0000 7f2fce800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fc80af080 0x7f2fc80aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.960+0000 7f2fce800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc80ac750 0x7f2fc80ae370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.960+0000 7f2fce800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc8160620 con 0x7f2fb0004d40 2024-03-28T13:10:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.960+0000 7f2fce800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc813f570 con 0x7f2fc80af080 2024-03-28T13:10:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.960+0000 7f2fce800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc81607a0 con 0x7f2fc80ac750 2024-03-28T13:10:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.961+0000 7f2fcd400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb0004d40 0x7f2fc813d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.961+0000 7f2fcd400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb0004d40 0x7f2fc813d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54062/0 (socket says 172.21.15.19:54062) 2024-03-28T13:10:24.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.961+0000 7f2fcd400640 1 -- 172.21.15.19:0/2201075796 learned_addr learned my addr 172.21.15.19:0/2201075796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:24.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.961+0000 7f2fcd400640 1 -- 172.21.15.19:0/2201075796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc80ac750 msgr2=0x7f2fc80ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.961+0000 7f2fcd400640 1 --2- 172.21.15.19:0/2201075796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc80ac750 0x7f2fc80ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.961+0000 7f2fcd400640 1 -- 172.21.15.19:0/2201075796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fc80af080 msgr2=0x7f2fc80aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:24.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.961+0000 7f2fcd400640 1 --2- 172.21.15.19:0/2201075796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fc80af080 0x7f2fc80aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:24.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.961+0000 7f2fcd400640 1 -- 172.21.15.19:0/2201075796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc813df10 con 0x7f2fb0004d40 2024-03-28T13:10:24.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.961+0000 7f2fcd400640 1 --2- 172.21.15.19:0/2201075796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb0004d40 0x7f2fc813d9d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc011630 tx=0x7f2fbc011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:24.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.962+0000 7f2fc5600640 1 -- 172.21.15.19:0/2201075796 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc015870 con 0x7f2fb0004d40 2024-03-28T13:10:24.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.962+0000 7f2fc5600640 1 -- 172.21.15.19:0/2201075796 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fbc015a10 con 0x7f2fb0004d40 2024-03-28T13:10:24.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.962+0000 7f2fce800640 1 -- 172.21.15.19:0/2201075796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc80a9b10 con 0x7f2fb0004d40 2024-03-28T13:10:24.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.962+0000 7f2fc5600640 1 -- 172.21.15.19:0/2201075796 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc015dc0 con 0x7f2fb0004d40 2024-03-28T13:10:24.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.962+0000 7f2fce800640 1 -- 172.21.15.19:0/2201075796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc80aa0d0 con 0x7f2fb0004d40 2024-03-28T13:10:24.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.963+0000 7f2fce800640 1 -- 172.21.15.19:0/2201075796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc8075940 con 0x7f2fb0004d40 2024-03-28T13:10:24.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.964+0000 7f2fc5600640 1 -- 172.21.15.19:0/2201075796 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2fbc00f490 con 0x7f2fb0004d40 2024-03-28T13:10:24.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.965+0000 7f2fc5600640 1 --2- 172.21.15.19:0/2201075796 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2fa806c560 0x7f2fa806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:24.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.965+0000 7f2fc5600640 1 -- 172.21.15.19:0/2201075796 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f2fbc08bfd0 con 0x7f2fb0004d40 2024-03-28T13:10:24.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.966+0000 7f2fcca00640 1 --2- 172.21.15.19:0/2201075796 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2fa806c560 0x7f2fa806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:24.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.967+0000 7f2fcca00640 1 --2- 172.21.15.19:0/2201075796 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2fa806c560 0x7f2fa806ea20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4007fd0 tx=0x7f2fb4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:24.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:24.973+0000 7f2fc5600640 1 -- 172.21.15.19:0/2201075796 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2fbc019030 con 0x7f2fb0004d40 2024-03-28T13:10:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.177+0000 7f2fce800640 1 -- 172.21.15.19:0/2201075796 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2fc80d0200 con 0x7f2fb0004d40 2024-03-28T13:10:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.177+0000 7f2fc5600640 1 -- 172.21.15.19:0/2201075796 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f2fbc05a5b0 con 0x7f2fb0004d40 2024-03-28T13:10:25.178 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:10:25.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.180+0000 7f2fce800640 1 -- 172.21.15.19:0/2201075796 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2fa806c560 msgr2=0x7f2fa806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:25.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.180+0000 7f2fce800640 1 --2- 172.21.15.19:0/2201075796 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2fa806c560 0x7f2fa806ea20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4007fd0 tx=0x7f2fb4009040 comp rx=0 tx=0).stop 2024-03-28T13:10:25.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.180+0000 7f2fce800640 1 -- 172.21.15.19:0/2201075796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb0004d40 msgr2=0x7f2fc813d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:25.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.180+0000 7f2fce800640 1 --2- 172.21.15.19:0/2201075796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb0004d40 0x7f2fc813d9d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc011630 tx=0x7f2fbc011af0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.180+0000 7f2fce800640 1 -- 172.21.15.19:0/2201075796 shutdown_connections 2024-03-28T13:10:25.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.180+0000 7f2fce800640 1 --2- 172.21.15.19:0/2201075796 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2fa806c560 0x7f2fa806ea20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.180+0000 7f2fce800640 1 --2- 172.21.15.19:0/2201075796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fc80ac750 0x7f2fc80ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.180+0000 7f2fce800640 1 --2- 172.21.15.19:0/2201075796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fc80af080 0x7f2fc80aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.180+0000 7f2fce800640 1 --2- 172.21.15.19:0/2201075796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2fb0004d40 0x7f2fc813d9d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.180+0000 7f2fce800640 1 -- 172.21.15.19:0/2201075796 >> 172.21.15.19:0/2201075796 conn(0x7f2fc80b1ba0 msgr2=0x7f2fc815f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:25.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.180+0000 7f2fce800640 1 -- 172.21.15.19:0/2201075796 shutdown_connections 2024-03-28T13:10:25.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.180+0000 7f2fce800640 1 -- 172.21.15.19:0/2201075796 wait complete. 2024-03-28T13:10:25.199 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:10:25.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.412+0000 7f3513200640 1 Processor -- start 2024-03-28T13:10:25.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.412+0000 7f3513200640 1 -- start start 2024-03-28T13:10:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.414+0000 7f3513200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f350c0d0990 0x7f350c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.414+0000 7f3513200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f350c0afab0 con 0x7f350c153580 2024-03-28T13:10:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.414+0000 7f3513200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f350c15cc80 con 0x7f350c0af2e0 2024-03-28T13:10:25.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.414+0000 7f3513200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f350c15da00 con 0x7f350c0d0990 2024-03-28T13:10:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.414+0000 7f350be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f350c153580 0x7f350c0d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48136/0 (socket says 172.21.15.19:48136) 2024-03-28T13:10:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.414+0000 7f350be00640 1 -- 172.21.15.19:0/3871090426 learned_addr learned my addr 172.21.15.19:0/3871090426 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:25.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.415+0000 7f3503e00640 1 --2- 172.21.15.19:0/3871090426 >> v2:172.21.15.188:3300/0 conn(0x7f350c0d0990 0x7f350c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.415+0000 7f350b400640 1 -- 172.21.15.19:0/3871090426 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3919065703 0 0) 0x7f350c0afab0 con 0x7f350c153580 2024-03-28T13:10:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.415+0000 7f350b400640 1 -- 172.21.15.19:0/3871090426 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34f0003520 con 0x7f350c153580 2024-03-28T13:10:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.415+0000 7f350b400640 1 -- 172.21.15.19:0/3871090426 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1057880706 0 0) 0x7f350c15cc80 con 0x7f350c0af2e0 2024-03-28T13:10:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.415+0000 7f350b400640 1 -- 172.21.15.19:0/3871090426 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f350c0afab0 con 0x7f350c0af2e0 2024-03-28T13:10:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.416+0000 7f3503e00640 1 -- 172.21.15.19:0/3871090426 >> v1:172.21.15.188:6789/0 conn(0x7f350c0af2e0 legacy=0x7f350c1600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.416+0000 7f3503e00640 1 -- 172.21.15.19:0/3871090426 >> v1:172.21.15.19:6789/0 conn(0x7f350c153580 legacy=0x7f350c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:25.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.416+0000 7f3503e00640 1 -- 172.21.15.19:0/3871090426 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350c15cf20 con 0x7f350c0d0990 2024-03-28T13:10:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.417+0000 7f3503e00640 1 --2- 172.21.15.19:0/3871090426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0d0990 0x7f350c0aeda0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f34f40097a0 tx=0x7f34f402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=229174866cace77d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:25.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.417+0000 7f350b400640 1 -- 172.21.15.19:0/3871090426 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4035020 con 0x7f350c0d0990 2024-03-28T13:10:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.417+0000 7f350b400640 1 -- 172.21.15.19:0/3871090426 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f4031830 con 0x7f350c0d0990 2024-03-28T13:10:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.417+0000 7f350b400640 1 -- 172.21.15.19:0/3871090426 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4031c00 con 0x7f350c0d0990 2024-03-28T13:10:25.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.418+0000 7f3513200640 1 -- 172.21.15.19:0/3871090426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0d0990 msgr2=0x7f350c0aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.418+0000 7f3513200640 1 --2- 172.21.15.19:0/3871090426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0d0990 0x7f350c0aeda0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f34f40097a0 tx=0x7f34f402b8d0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.418+0000 7f3513200640 1 -- 172.21.15.19:0/3871090426 shutdown_connections 2024-03-28T13:10:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.418+0000 7f3513200640 1 --2- 172.21.15.19:0/3871090426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0d0990 0x7f350c0aeda0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.418+0000 7f3513200640 1 -- 172.21.15.19:0/3871090426 >> 172.21.15.19:0/3871090426 conn(0x7f350c06b750 msgr2=0x7f350c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.418+0000 7f3513200640 1 -- 172.21.15.19:0/3871090426 shutdown_connections 2024-03-28T13:10:25.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.418+0000 7f3513200640 1 -- 172.21.15.19:0/3871090426 wait complete. 2024-03-28T13:10:25.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.420+0000 7f3513200640 1 Processor -- start 2024-03-28T13:10:25.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.421+0000 7f3513200640 1 -- start start 2024-03-28T13:10:25.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.422+0000 7f3513200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0af2e0 0x7f350c151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:25.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.422+0000 7f3513200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f350c0d0990 0x7f350c0cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:25.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.422+0000 7f3513200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c153580 0x7f350c0cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:25.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.422+0000 7f3513200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f350c153960 con 0x7f350c0d0990 2024-03-28T13:10:25.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.422+0000 7f3513200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f350c15d4e0 con 0x7f350c0af2e0 2024-03-28T13:10:25.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.422+0000 7f3513200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f350c0af6c0 con 0x7f350c153580 2024-03-28T13:10:25.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.422+0000 7f3503e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f350c0d0990 0x7f350c0cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:25.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.422+0000 7f3503e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f350c0d0990 0x7f350c0cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40174/0 (socket says 172.21.15.19:40174) 2024-03-28T13:10:25.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.422+0000 7f3503e00640 1 -- 172.21.15.19:0/155966985 learned_addr learned my addr 172.21.15.19:0/155966985 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:25.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.422+0000 7f350be00640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0af2e0 0x7f350c151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:25.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.422+0000 7f3511200640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c153580 0x7f350c0cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:25.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.422+0000 7f3503e00640 1 -- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c153580 msgr2=0x7f350c0cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:25.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.422+0000 7f3503e00640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c153580 0x7f350c0cb300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.423+0000 7f3503e00640 1 -- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0af2e0 msgr2=0x7f350c151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:25.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.423+0000 7f3503e00640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0af2e0 0x7f350c151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.423+0000 7f3503e00640 1 -- 172.21.15.19:0/155966985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350c0cd380 con 0x7f350c0d0990 2024-03-28T13:10:25.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.423+0000 7f3503e00640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f350c0d0990 0x7f350c0cca50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009890 tx=0x7f34f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:25.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.423+0000 7f350be00640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0af2e0 0x7f350c151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:25.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.423+0000 7f3511200640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c153580 0x7f350c0cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:25.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.423+0000 7f3508c00640 1 -- 172.21.15.19:0/155966985 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4035040 con 0x7f350c0d0990 2024-03-28T13:10:25.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.423+0000 7f3508c00640 1 -- 172.21.15.19:0/155966985 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f403b920 con 0x7f350c0d0990 2024-03-28T13:10:25.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.423+0000 7f3508c00640 1 -- 172.21.15.19:0/155966985 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f403bd10 con 0x7f350c0d0990 2024-03-28T13:10:25.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.423+0000 7f3513200640 1 -- 172.21.15.19:0/155966985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350c0cd660 con 0x7f350c0d0990 2024-03-28T13:10:25.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.424+0000 7f3513200640 1 -- 172.21.15.19:0/155966985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f350c0cdc20 con 0x7f350c0d0990 2024-03-28T13:10:25.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.425+0000 7f3513200640 1 -- 172.21.15.19:0/155966985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f350c0685c0 con 0x7f350c0d0990 2024-03-28T13:10:25.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.426+0000 7f3508c00640 1 -- 172.21.15.19:0/155966985 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f34f4038020 con 0x7f350c0d0990 2024-03-28T13:10:25.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.427+0000 7f3508c00640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f34e806c530 0x7f34e806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:25.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.427+0000 7f3508c00640 1 -- 172.21.15.19:0/155966985 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f34f40ad790 con 0x7f350c0d0990 2024-03-28T13:10:25.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.427+0000 7f350be00640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f34e806c530 0x7f34e806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:25.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.429+0000 7f350be00640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f34e806c530 0x7f34e806e9f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f34fc000f60 tx=0x7f34fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:25.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.432+0000 7f3508c00640 1 -- 172.21.15.19:0/155966985 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f34f407bd70 con 0x7f350c0d0990 2024-03-28T13:10:25.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.666+0000 7f3513200640 1 -- 172.21.15.19:0/155966985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f350c06eca0 con 0x7f350c0d0990 2024-03-28T13:10:25.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.667+0000 7f3508c00640 1 -- 172.21.15.19:0/155966985 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f34f407b670 con 0x7f350c0d0990 2024-03-28T13:10:25.667 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:10:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.669+0000 7f3513200640 1 -- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f34e806c530 msgr2=0x7f34e806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.669+0000 7f3513200640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f34e806c530 0x7f34e806e9f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f34fc000f60 tx=0x7f34fc009040 comp rx=0 tx=0).stop 2024-03-28T13:10:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.669+0000 7f3513200640 1 -- 172.21.15.19:0/155966985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f350c0d0990 msgr2=0x7f350c0cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.669+0000 7f3513200640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f350c0d0990 0x7f350c0cca50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009890 tx=0x7f34f40046c0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.669+0000 7f3513200640 1 -- 172.21.15.19:0/155966985 shutdown_connections 2024-03-28T13:10:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.669+0000 7f3513200640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f34e806c530 0x7f34e806e9f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.669+0000 7f3513200640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f350c153580 0x7f350c0cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.669+0000 7f3513200640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f350c0d0990 0x7f350c0cca50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.669+0000 7f3513200640 1 --2- 172.21.15.19:0/155966985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f350c0af2e0 0x7f350c151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.669+0000 7f3513200640 1 -- 172.21.15.19:0/155966985 >> 172.21.15.19:0/155966985 conn(0x7f350c06b750 msgr2=0x7f350c06bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.670+0000 7f3513200640 1 -- 172.21.15.19:0/155966985 shutdown_connections 2024-03-28T13:10:25.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.670+0000 7f3513200640 1 -- 172.21.15.19:0/155966985 wait complete. 2024-03-28T13:10:25.691 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-03-28T13:10:25.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.898+0000 7f7635e00640 1 Processor -- start 2024-03-28T13:10:25.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.898+0000 7f7635e00640 1 -- start start 2024-03-28T13:10:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.900+0000 7f7635e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f763006b570 0x7f763006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.900+0000 7f7635e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f763015d3e0 0x7f76300d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.900+0000 7f7635e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7630145f50 con 0x7f763015fed0 2024-03-28T13:10:25.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.900+0000 7f7635e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f76301603d0 con 0x7f763015d3e0 2024-03-28T13:10:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.900+0000 7f7635e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f76301606b0 con 0x7f763006b570 2024-03-28T13:10:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.901+0000 7f7634a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f763006b570 0x7f763006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.901+0000 7f762fe00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f763015fed0 0x7f763006b950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:56260/0 (socket says 172.21.15.19:56260) 2024-03-28T13:10:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.901+0000 7f762fe00640 1 -- 172.21.15.19:0/3113731422 learned_addr learned my addr 172.21.15.19:0/3113731422 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:25.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.901+0000 7f7635400640 1 --2- 172.21.15.19:0/3113731422 >> v2:172.21.15.188:3300/0 conn(0x7f763015d3e0 0x7f76300d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:25.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.902+0000 7f7634a00640 1 -- 172.21.15.19:0/3113731422 >> v2:172.21.15.188:3300/0 conn(0x7f763015d3e0 msgr2=0x7f76300d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:25.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.902+0000 7f7634a00640 1 --2- 172.21.15.19:0/3113731422 >> v2:172.21.15.188:3300/0 conn(0x7f763015d3e0 0x7f76300d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.902+0000 7f7634a00640 1 -- 172.21.15.19:0/3113731422 >> v1:172.21.15.188:6789/0 conn(0x7f763015fed0 legacy=0x7f763006b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:25.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.902+0000 7f7634a00640 1 -- 172.21.15.19:0/3113731422 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7630160990 con 0x7f763006b570 2024-03-28T13:10:25.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.903+0000 7f7634a00640 1 --2- 172.21.15.19:0/3113731422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f763006b570 0x7f763006fa90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f76240097d0 tx=0x7f762402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=945ee0dc708038f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:25.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.904+0000 7f762f400640 1 -- 172.21.15.19:0/3113731422 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624031780 con 0x7f763006b570 2024-03-28T13:10:25.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.904+0000 7f762f400640 1 -- 172.21.15.19:0/3113731422 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7624037070 con 0x7f763006b570 2024-03-28T13:10:25.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.904+0000 7f762f400640 1 -- 172.21.15.19:0/3113731422 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624031b40 con 0x7f763006b570 2024-03-28T13:10:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.904+0000 7f7635e00640 1 -- 172.21.15.19:0/3113731422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f763006b570 msgr2=0x7f763006fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.904+0000 7f7635e00640 1 --2- 172.21.15.19:0/3113731422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f763006b570 0x7f763006fa90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f76240097d0 tx=0x7f762402b820 comp rx=0 tx=0).stop 2024-03-28T13:10:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.904+0000 7f7635e00640 1 -- 172.21.15.19:0/3113731422 shutdown_connections 2024-03-28T13:10:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.904+0000 7f7635e00640 1 --2- 172.21.15.19:0/3113731422 >> v2:172.21.15.188:3300/0 conn(0x7f763015d3e0 0x7f76300d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.904+0000 7f7635e00640 1 --2- 172.21.15.19:0/3113731422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f763006b570 0x7f763006fa90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.904+0000 7f7635e00640 1 -- 172.21.15.19:0/3113731422 >> 172.21.15.19:0/3113731422 conn(0x7f76300d1090 msgr2=0x7f76300d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.904+0000 7f7635e00640 1 -- 172.21.15.19:0/3113731422 shutdown_connections 2024-03-28T13:10:25.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.904+0000 7f7635e00640 1 -- 172.21.15.19:0/3113731422 wait complete. 2024-03-28T13:10:25.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.906+0000 7f7635e00640 1 Processor -- start 2024-03-28T13:10:25.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.906+0000 7f7635e00640 1 -- start start 2024-03-28T13:10:25.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.907+0000 7f7635e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f763006b570 0x7f76301402f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:25.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.907+0000 7f7635e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f763015d3e0 0x7f763015bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:25.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.907+0000 7f7635e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f763015fed0 0x7f76300b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:25.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.907+0000 7f7635e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76300cfc20 con 0x7f763006b570 2024-03-28T13:10:25.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.907+0000 7f7635e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f763015d870 con 0x7f763015d3e0 2024-03-28T13:10:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.907+0000 7f7635e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7630145340 con 0x7f763015fed0 2024-03-28T13:10:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.907+0000 7f7634a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f763006b570 0x7f76301402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.907+0000 7f7634a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f763006b570 0x7f76301402f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40182/0 (socket says 172.21.15.19:40182) 2024-03-28T13:10:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.907+0000 7f7634a00640 1 -- 172.21.15.19:0/1703074265 learned_addr learned my addr 172.21.15.19:0/1703074265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.908+0000 7f762fe00640 1 --2- 172.21.15.19:0/1703074265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f763015d3e0 0x7f763015bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.908+0000 7f7635400640 1 --2- 172.21.15.19:0/1703074265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f763015fed0 0x7f76300b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.908+0000 7f7634a00640 1 -- 172.21.15.19:0/1703074265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f763015fed0 msgr2=0x7f76300b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.908+0000 7f7634a00640 1 --2- 172.21.15.19:0/1703074265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f763015fed0 0x7f76300b51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.908+0000 7f7634a00640 1 -- 172.21.15.19:0/1703074265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f763015d3e0 msgr2=0x7f763015bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.908+0000 7f7634a00640 1 --2- 172.21.15.19:0/1703074265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f763015d3e0 0x7f763015bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.908+0000 7f7634a00640 1 -- 172.21.15.19:0/1703074265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763015c240 con 0x7f763006b570 2024-03-28T13:10:25.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.908+0000 7f7634a00640 1 --2- 172.21.15.19:0/1703074265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f763006b570 0x7f76301402f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f76240098f0 tx=0x7f76240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:25.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.909+0000 7f762cc00640 1 -- 172.21.15.19:0/1703074265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624032610 con 0x7f763006b570 2024-03-28T13:10:25.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.909+0000 7f762cc00640 1 -- 172.21.15.19:0/1703074265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76240327b0 con 0x7f763006b570 2024-03-28T13:10:25.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.909+0000 7f7635e00640 1 -- 172.21.15.19:0/1703074265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76301f7a20 con 0x7f763006b570 2024-03-28T13:10:25.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.909+0000 7f762cc00640 1 -- 172.21.15.19:0/1703074265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624032ba0 con 0x7f763006b570 2024-03-28T13:10:25.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.909+0000 7f7635e00640 1 -- 172.21.15.19:0/1703074265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76301f7fe0 con 0x7f763006b570 2024-03-28T13:10:25.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.910+0000 7f7635e00640 1 -- 172.21.15.19:0/1703074265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7630075940 con 0x7f763006b570 2024-03-28T13:10:25.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.911+0000 7f762cc00640 1 -- 172.21.15.19:0/1703074265 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f762404c060 con 0x7f763006b570 2024-03-28T13:10:25.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.912+0000 7f762cc00640 1 --2- 172.21.15.19:0/1703074265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f761006c530 0x7f761006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:25.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.913+0000 7f762cc00640 1 -- 172.21.15.19:0/1703074265 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f76240acd80 con 0x7f763006b570 2024-03-28T13:10:25.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.913+0000 7f762fe00640 1 --2- 172.21.15.19:0/1703074265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f761006c530 0x7f761006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:25.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.914+0000 7f762fe00640 1 --2- 172.21.15.19:0/1703074265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f761006c530 0x7f761006e9f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f76301f8600 tx=0x7f761c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:25.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:25.917+0000 7f762cc00640 1 -- 172.21.15.19:0/1703074265 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7624040070 con 0x7f763006b570 2024-03-28T13:10:26.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:26.195+0000 7f7635e00640 1 -- 172.21.15.19:0/1703074265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f763006c420 con 0x7f763006b570 2024-03-28T13:10:27.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.183+0000 7f762cc00640 1 -- 172.21.15.19:0/1703074265 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v64) ==== 145+0+0 (secure 0 0 0) 0x7f762402f050 con 0x7f763006b570 2024-03-28T13:10:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.187+0000 7f7635e00640 1 -- 172.21.15.19:0/1703074265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f761006c530 msgr2=0x7f761006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.187+0000 7f7635e00640 1 --2- 172.21.15.19:0/1703074265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f761006c530 0x7f761006e9f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f76301f8600 tx=0x7f761c009040 comp rx=0 tx=0).stop 2024-03-28T13:10:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.188+0000 7f7635e00640 1 -- 172.21.15.19:0/1703074265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f763006b570 msgr2=0x7f76301402f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.188+0000 7f7635e00640 1 --2- 172.21.15.19:0/1703074265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f763006b570 0x7f76301402f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f76240098f0 tx=0x7f76240046c0 comp rx=0 tx=0).stop 2024-03-28T13:10:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.188+0000 7f7635e00640 1 -- 172.21.15.19:0/1703074265 shutdown_connections 2024-03-28T13:10:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.188+0000 7f7635e00640 1 --2- 172.21.15.19:0/1703074265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f761006c530 0x7f761006e9f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.188+0000 7f7635e00640 1 --2- 172.21.15.19:0/1703074265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f763015fed0 0x7f76300b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.188+0000 7f7635e00640 1 --2- 172.21.15.19:0/1703074265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f763015d3e0 0x7f763015bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.188+0000 7f7635e00640 1 --2- 172.21.15.19:0/1703074265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f763006b570 0x7f76301402f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.188+0000 7f7635e00640 1 -- 172.21.15.19:0/1703074265 >> 172.21.15.19:0/1703074265 conn(0x7f76300d1090 msgr2=0x7f763006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.188+0000 7f7635e00640 1 -- 172.21.15.19:0/1703074265 shutdown_connections 2024-03-28T13:10:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.188+0000 7f7635e00640 1 -- 172.21.15.19:0/1703074265 wait complete. 2024-03-28T13:10:27.213 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-28T13:10:27.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.434+0000 7f5ccee00640 1 Processor -- start 2024-03-28T13:10:27.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.434+0000 7f5ccee00640 1 -- start start 2024-03-28T13:10:27.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.436+0000 7f5ccee00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5cc80b4a30 0x7f5cc80b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:27.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.436+0000 7f5ccee00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5cc8158580 0x7f5cc81589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:27.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.436+0000 7f5ccee00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cc815ede0 con 0x7f5cc80b4650 2024-03-28T13:10:27.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.436+0000 7f5ccee00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5cc8141b30 con 0x7f5cc8158580 2024-03-28T13:10:27.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.436+0000 7f5ccee00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5cc8141d80 con 0x7f5cc80b4a30 2024-03-28T13:10:27.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.436+0000 7f5cc7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5cc80b4650 0x7f5cc806c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54424/0 (socket says 172.21.15.19:54424) 2024-03-28T13:10:27.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.436+0000 7f5cc7e00640 1 -- 172.21.15.19:0/3522608481 learned_addr learned my addr 172.21.15.19:0/3522608481 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:27.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.436+0000 7f5ccce00640 1 --2- 172.21.15.19:0/3522608481 >> v2:172.21.15.19:3300/0 conn(0x7f5cc8158580 0x7f5cc81589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:27.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.437+0000 7f5cbfe00640 1 --2- 172.21.15.19:0/3522608481 >> v2:172.21.15.188:3300/0 conn(0x7f5cc80b4a30 0x7f5cc80b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:27.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.437+0000 7f5ccce00640 1 -- 172.21.15.19:0/3522608481 >> v2:172.21.15.188:3300/0 conn(0x7f5cc80b4a30 msgr2=0x7f5cc80b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:27.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.437+0000 7f5ccce00640 1 --2- 172.21.15.19:0/3522608481 >> v2:172.21.15.188:3300/0 conn(0x7f5cc80b4a30 0x7f5cc80b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:27.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.437+0000 7f5ccce00640 1 -- 172.21.15.19:0/3522608481 >> v1:172.21.15.19:6789/0 conn(0x7f5cc80b4650 legacy=0x7f5cc806c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:27.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.437+0000 7f5cc7400640 1 -- 172.21.15.19:0/3522608481 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2344770663 0 0) 0x7f5cc815ede0 con 0x7f5cc80b4650 2024-03-28T13:10:27.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.437+0000 7f5ccce00640 1 -- 172.21.15.19:0/3522608481 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc8142060 con 0x7f5cc8158580 2024-03-28T13:10:27.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.438+0000 7f5ccce00640 1 --2- 172.21.15.19:0/3522608481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc8158580 0x7f5cc81589a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5cc815c6b0 tx=0x7f5cc002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76657abc274d2f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:27.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.438+0000 7f5cc7400640 1 -- 172.21.15.19:0/3522608481 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0039070 con 0x7f5cc8158580 2024-03-28T13:10:27.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.438+0000 7f5cc7400640 1 -- 172.21.15.19:0/3522608481 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc002cc80 con 0x7f5cc8158580 2024-03-28T13:10:27.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.439+0000 7f5cc7400640 1 -- 172.21.15.19:0/3522608481 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0034660 con 0x7f5cc8158580 2024-03-28T13:10:27.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.439+0000 7f5ccee00640 1 -- 172.21.15.19:0/3522608481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc8158580 msgr2=0x7f5cc81589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:27.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.439+0000 7f5ccee00640 1 --2- 172.21.15.19:0/3522608481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc8158580 0x7f5cc81589a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5cc815c6b0 tx=0x7f5cc002bed0 comp rx=0 tx=0).stop 2024-03-28T13:10:27.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.439+0000 7f5ccee00640 1 -- 172.21.15.19:0/3522608481 shutdown_connections 2024-03-28T13:10:27.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.439+0000 7f5ccee00640 1 --2- 172.21.15.19:0/3522608481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc8158580 0x7f5cc81589a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:27.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.439+0000 7f5ccee00640 1 --2- 172.21.15.19:0/3522608481 >> v2:172.21.15.188:3300/0 conn(0x7f5cc80b4a30 0x7f5cc80b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:27.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.439+0000 7f5ccee00640 1 -- 172.21.15.19:0/3522608481 >> 172.21.15.19:0/3522608481 conn(0x7f5cc8067670 msgr2=0x7f5cc8067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:27.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.439+0000 7f5ccee00640 1 -- 172.21.15.19:0/3522608481 shutdown_connections 2024-03-28T13:10:27.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.439+0000 7f5ccee00640 1 -- 172.21.15.19:0/3522608481 wait complete. 2024-03-28T13:10:27.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.442+0000 7f5ccee00640 1 Processor -- start 2024-03-28T13:10:27.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.442+0000 7f5ccee00640 1 -- start start 2024-03-28T13:10:27.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.444+0000 7f5ccee00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc80b4650 0x7f5cc815b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:27.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.444+0000 7f5ccee00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc80b4a30 0x7f5cc81f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:27.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.444+0000 7f5ccee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc8158580 0x7f5cc815d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:27.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.444+0000 7f5ccee00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8142900 con 0x7f5cc8158580 2024-03-28T13:10:27.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.444+0000 7f5ccee00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc8142620 con 0x7f5cc80b4a30 2024-03-28T13:10:27.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.444+0000 7f5ccee00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5cc8158ee0 con 0x7f5cc80b4650 2024-03-28T13:10:27.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.444+0000 7f5ccce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc8158580 0x7f5cc815d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:27.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.444+0000 7f5ccce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc8158580 0x7f5cc815d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40200/0 (socket says 172.21.15.19:40200) 2024-03-28T13:10:27.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.444+0000 7f5ccce00640 1 -- 172.21.15.19:0/2247573956 learned_addr learned my addr 172.21.15.19:0/2247573956 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.444+0000 7f5cc7e00640 1 --2- 172.21.15.19:0/2247573956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc80b4650 0x7f5cc815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.445+0000 7f5cbfe00640 1 --2- 172.21.15.19:0/2247573956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc80b4a30 0x7f5cc81f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.445+0000 7f5ccce00640 1 -- 172.21.15.19:0/2247573956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc80b4650 msgr2=0x7f5cc815b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.445+0000 7f5ccce00640 1 --2- 172.21.15.19:0/2247573956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc80b4650 0x7f5cc815b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.445+0000 7f5ccce00640 1 -- 172.21.15.19:0/2247573956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc80b4a30 msgr2=0x7f5cc81f33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.445+0000 7f5ccce00640 1 --2- 172.21.15.19:0/2247573956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc80b4a30 0x7f5cc81f33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.445+0000 7f5ccce00640 1 -- 172.21.15.19:0/2247573956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc80cd290 con 0x7f5cc8158580 2024-03-28T13:10:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.445+0000 7f5ccce00640 1 --2- 172.21.15.19:0/2247573956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc8158580 0x7f5cc815d710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0037040 tx=0x7f5cc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:27.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.445+0000 7f5cc4c00640 1 -- 172.21.15.19:0/2247573956 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0044020 con 0x7f5cc8158580 2024-03-28T13:10:27.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.445+0000 7f5cc4c00640 1 -- 172.21.15.19:0/2247573956 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cc003a040 con 0x7f5cc8158580 2024-03-28T13:10:27.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.445+0000 7f5cc4c00640 1 -- 172.21.15.19:0/2247573956 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc0039050 con 0x7f5cc8158580 2024-03-28T13:10:27.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.445+0000 7f5ccee00640 1 -- 172.21.15.19:0/2247573956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc815a850 con 0x7f5cc8158580 2024-03-28T13:10:27.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.446+0000 7f5ccee00640 1 -- 172.21.15.19:0/2247573956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc81f38f0 con 0x7f5cc8158580 2024-03-28T13:10:27.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.447+0000 7f5ccee00640 1 -- 172.21.15.19:0/2247573956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc815ae60 con 0x7f5cc8158580 2024-03-28T13:10:27.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.448+0000 7f5cc4c00640 1 -- 172.21.15.19:0/2247573956 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5cc0003b90 con 0x7f5cc8158580 2024-03-28T13:10:27.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.449+0000 7f5cc4c00640 1 --2- 172.21.15.19:0/2247573956 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ca406c530 0x7f5ca406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:27.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.449+0000 7f5cc4c00640 1 -- 172.21.15.19:0/2247573956 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f5cc00adf00 con 0x7f5cc8158580 2024-03-28T13:10:27.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.449+0000 7f5cc7e00640 1 --2- 172.21.15.19:0/2247573956 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ca406c530 0x7f5ca406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:27.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.451+0000 7f5cc7e00640 1 --2- 172.21.15.19:0/2247573956 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ca406c530 0x7f5ca406e9f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8000f60 tx=0x7f5cb8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:27.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.454+0000 7f5cc4c00640 1 -- 172.21.15.19:0/2247573956 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5cc004e020 con 0x7f5cc8158580 2024-03-28T13:10:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.662+0000 7f5ccee00640 1 -- 172.21.15.19:0/2247573956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5cc8074f30 con 0x7f5cc8158580 2024-03-28T13:10:27.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.667+0000 7f5cc4c00640 1 -- 172.21.15.19:0/2247573956 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f5cc007c4d0 con 0x7f5cc8158580 2024-03-28T13:10:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.671+0000 7f5ccee00640 1 -- 172.21.15.19:0/2247573956 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ca406c530 msgr2=0x7f5ca406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.671+0000 7f5ccee00640 1 --2- 172.21.15.19:0/2247573956 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ca406c530 0x7f5ca406e9f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8000f60 tx=0x7f5cb8009040 comp rx=0 tx=0).stop 2024-03-28T13:10:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.671+0000 7f5ccee00640 1 -- 172.21.15.19:0/2247573956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc8158580 msgr2=0x7f5cc815d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.671+0000 7f5ccee00640 1 --2- 172.21.15.19:0/2247573956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc8158580 0x7f5cc815d710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5cc0037040 tx=0x7f5cc0004a40 comp rx=0 tx=0).stop 2024-03-28T13:10:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.671+0000 7f5ccee00640 1 -- 172.21.15.19:0/2247573956 shutdown_connections 2024-03-28T13:10:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.671+0000 7f5ccee00640 1 --2- 172.21.15.19:0/2247573956 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ca406c530 0x7f5ca406e9f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.671+0000 7f5ccee00640 1 --2- 172.21.15.19:0/2247573956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5cc8158580 0x7f5cc815d710 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.671+0000 7f5ccee00640 1 --2- 172.21.15.19:0/2247573956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5cc80b4a30 0x7f5cc81f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.671+0000 7f5ccee00640 1 --2- 172.21.15.19:0/2247573956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5cc80b4650 0x7f5cc815b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.671+0000 7f5ccee00640 1 -- 172.21.15.19:0/2247573956 >> 172.21.15.19:0/2247573956 conn(0x7f5cc8067670 msgr2=0x7f5cc806e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.672+0000 7f5ccee00640 1 -- 172.21.15.19:0/2247573956 shutdown_connections 2024-03-28T13:10:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:27.672+0000 7f5ccee00640 1 -- 172.21.15.19:0/2247573956 wait complete. 2024-03-28T13:10:27.699 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-28T13:10:27.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:10:27.812 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:10:27.929 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-28T13:10:27.930 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:10:27.930 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:27.985 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:10:27.985 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:28.037 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:10:28.037 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:28.085 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:10:28.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.337+0000 7ff633200640 1 Processor -- start 2024-03-28T13:10:28.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.337+0000 7ff633200640 1 -- start start 2024-03-28T13:10:28.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.339+0000 7ff633200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff63415e4a0 0x7ff6340b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.339+0000 7ff633200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff6340cf700 0x7ff6340b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.339+0000 7ff633200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff63415e880 con 0x7ff6340d2310 2024-03-28T13:10:28.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.339+0000 7ff633200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff6340d34e0 con 0x7ff6340cf700 2024-03-28T13:10:28.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.339+0000 7ff633200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff6340d28d0 con 0x7ff63415e4a0 2024-03-28T13:10:28.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.339+0000 7ff629400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff6340d2310 0x7ff6340ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54428/0 (socket says 172.21.15.19:54428) 2024-03-28T13:10:28.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.339+0000 7ff629400640 1 -- 172.21.15.19:0/2289811761 learned_addr learned my addr 172.21.15.19:0/2289811761 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:28.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.340+0000 7ff631e00640 1 --2- 172.21.15.19:0/2289811761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff63415e4a0 0x7ff6340b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.340+0000 7ff632800640 1 --2- 172.21.15.19:0/2289811761 >> v2:172.21.15.188:3300/0 conn(0x7ff6340cf700 0x7ff6340b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.340+0000 7ff631400640 1 -- 172.21.15.19:0/2289811761 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2028380165 0 0) 0x7ff63415e880 con 0x7ff6340d2310 2024-03-28T13:10:28.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.340+0000 7ff631400640 1 -- 172.21.15.19:0/2289811761 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff618003520 con 0x7ff6340d2310 2024-03-28T13:10:28.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.340+0000 7ff631400640 1 -- 172.21.15.19:0/2289811761 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2052466033 0 0) 0x7ff618003520 con 0x7ff6340d2310 2024-03-28T13:10:28.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.340+0000 7ff631400640 1 -- 172.21.15.19:0/2289811761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff63415e4a0 msgr2=0x7ff6340b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:28.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.340+0000 7ff631400640 1 --2- 172.21.15.19:0/2289811761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff63415e4a0 0x7ff6340b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.340+0000 7ff631400640 1 -- 172.21.15.19:0/2289811761 >> v2:172.21.15.188:3300/0 conn(0x7ff6340cf700 msgr2=0x7ff6340b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:28.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.340+0000 7ff631400640 1 --2- 172.21.15.19:0/2289811761 >> v2:172.21.15.188:3300/0 conn(0x7ff6340cf700 0x7ff6340b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.340+0000 7ff631400640 1 -- 172.21.15.19:0/2289811761 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6340d2bb0 con 0x7ff6340d2310 2024-03-28T13:10:28.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.340+0000 7ff631400640 1 -- 172.21.15.19:0/2289811761 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7ff61c001670 con 0x7ff6340d2310 2024-03-28T13:10:28.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.340+0000 7ff631400640 1 -- 172.21.15.19:0/2289811761 >> v1:172.21.15.19:6789/0 conn(0x7ff6340d2310 legacy=0x7ff6340ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:28.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.340+0000 7ff631400640 1 --2- 172.21.15.19:0/2289811761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff63415e4a0 0x7ff6180039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.340+0000 7ff631400640 1 --2- 172.21.15.19:0/2289811761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff618003f50 0x7ff618004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.340+0000 7ff631400640 1 --2- 172.21.15.19:0/2289811761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff618004d40 0x7ff6180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.341+0000 7ff631400640 1 -- 172.21.15.19:0/2289811761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6340d34e0 con 0x7ff618003f50 2024-03-28T13:10:28.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.341+0000 7ff631400640 1 -- 172.21.15.19:0/2289811761 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6340d28d0 con 0x7ff618004d40 2024-03-28T13:10:28.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.341+0000 7ff631400640 1 -- 172.21.15.19:0/2289811761 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff61800b9c0 con 0x7ff63415e4a0 2024-03-28T13:10:28.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.341+0000 7ff629400640 1 --2- 172.21.15.19:0/2289811761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff618003f50 0x7ff618004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.341+0000 7ff631e00640 1 --2- 172.21.15.19:0/2289811761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff63415e4a0 0x7ff6180039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.341+0000 7ff632800640 1 --2- 172.21.15.19:0/2289811761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff618004d40 0x7ff6180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.341+0000 7ff629400640 1 -- 172.21.15.19:0/2289811761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff63415e4a0 msgr2=0x7ff6180039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:28.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.341+0000 7ff629400640 1 --2- 172.21.15.19:0/2289811761 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff63415e4a0 0x7ff6180039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.341+0000 7ff629400640 1 -- 172.21.15.19:0/2289811761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff618004d40 msgr2=0x7ff6180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:28.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.341+0000 7ff629400640 1 --2- 172.21.15.19:0/2289811761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff618004d40 0x7ff6180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.341+0000 7ff629400640 1 -- 172.21.15.19:0/2289811761 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff618003520 con 0x7ff618003f50 2024-03-28T13:10:28.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.341+0000 7ff632800640 1 --2- 172.21.15.19:0/2289811761 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff618004d40 0x7ff6180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:28.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.343+0000 7ff629400640 1 --2- 172.21.15.19:0/2289811761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff618003f50 0x7ff618004800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00c270 tx=0x7ff618005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fd8c0e7bf69f4b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:28.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.343+0000 7ff631400640 1 -- 172.21.15.19:0/2289811761 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c007cf0 con 0x7ff618003f50 2024-03-28T13:10:28.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.343+0000 7ff631400640 1 -- 172.21.15.19:0/2289811761 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff61c030070 con 0x7ff618003f50 2024-03-28T13:10:28.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.343+0000 7ff631400640 1 -- 172.21.15.19:0/2289811761 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c005d70 con 0x7ff618003f50 2024-03-28T13:10:28.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.344+0000 7ff633200640 1 -- 172.21.15.19:0/2289811761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff618003f50 msgr2=0x7ff618004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:28.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.344+0000 7ff633200640 1 --2- 172.21.15.19:0/2289811761 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff618003f50 0x7ff618004800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00c270 tx=0x7ff618005830 comp rx=0 tx=0).stop 2024-03-28T13:10:28.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.344+0000 7ff632800640 1 -- 172.21.15.19:0/2289811761 reap_dead start 2024-03-28T13:10:28.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.344+0000 7ff633200640 1 -- 172.21.15.19:0/2289811761 shutdown_connections 2024-03-28T13:10:28.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.344+0000 7ff633200640 1 -- 172.21.15.19:0/2289811761 >> 172.21.15.19:0/2289811761 conn(0x7ff6340676d0 msgr2=0x7ff634067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:28.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.344+0000 7ff633200640 1 -- 172.21.15.19:0/2289811761 shutdown_connections 2024-03-28T13:10:28.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.344+0000 7ff633200640 1 -- 172.21.15.19:0/2289811761 wait complete. 2024-03-28T13:10:28.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.346+0000 7ff633200640 1 Processor -- start 2024-03-28T13:10:28.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.346+0000 7ff633200640 1 -- start start 2024-03-28T13:10:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.347+0000 7ff633200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff618003f50 0x7ff6341f7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.347+0000 7ff633200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff63414e2c0 0x7ff63414e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.347+0000 7ff633200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff63415f340 0x7ff63415f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.347+0000 7ff633200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6340d3970 con 0x7ff63414e2c0 2024-03-28T13:10:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.347+0000 7ff633200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6340d3170 con 0x7ff618003f50 2024-03-28T13:10:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.347+0000 7ff633200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff6340b9c60 con 0x7ff63415f340 2024-03-28T13:10:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.347+0000 7ff629400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff63414e2c0 0x7ff63414e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.347+0000 7ff629400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff63414e2c0 0x7ff63414e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40214/0 (socket says 172.21.15.19:40214) 2024-03-28T13:10:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.347+0000 7ff629400640 1 -- 172.21.15.19:0/219690990 learned_addr learned my addr 172.21.15.19:0/219690990 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.348+0000 7ff631e00640 1 --2- 172.21.15.19:0/219690990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff618003f50 0x7ff6341f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.348+0000 7ff632800640 1 --2- 172.21.15.19:0/219690990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff63415f340 0x7ff63415f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.348+0000 7ff629400640 1 -- 172.21.15.19:0/219690990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff63415f340 msgr2=0x7ff63415f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:28.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.348+0000 7ff629400640 1 --2- 172.21.15.19:0/219690990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff63415f340 0x7ff63415f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.348+0000 7ff629400640 1 -- 172.21.15.19:0/219690990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff618003f50 msgr2=0x7ff6341f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:28.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.348+0000 7ff629400640 1 --2- 172.21.15.19:0/219690990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff618003f50 0x7ff6341f7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.348+0000 7ff629400640 1 -- 172.21.15.19:0/219690990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6341600e0 con 0x7ff63414e2c0 2024-03-28T13:10:28.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.348+0000 7ff632800640 1 --2- 172.21.15.19:0/219690990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff63415f340 0x7ff63415f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:28.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.348+0000 7ff629400640 1 --2- 172.21.15.19:0/219690990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff63414e2c0 0x7ff63414e740 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00c270 tx=0x7ff6340d2d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:28.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.348+0000 7ff62aa00640 1 -- 172.21.15.19:0/219690990 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c001c30 con 0x7ff63414e2c0 2024-03-28T13:10:28.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.348+0000 7ff62aa00640 1 -- 172.21.15.19:0/219690990 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff61c004bf0 con 0x7ff63414e2c0 2024-03-28T13:10:28.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.348+0000 7ff62aa00640 1 -- 172.21.15.19:0/219690990 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61c007070 con 0x7ff63414e2c0 2024-03-28T13:10:28.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.348+0000 7ff633200640 1 -- 172.21.15.19:0/219690990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6340cb520 con 0x7ff63414e2c0 2024-03-28T13:10:28.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.349+0000 7ff633200640 1 -- 172.21.15.19:0/219690990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6340cbae0 con 0x7ff63414e2c0 2024-03-28T13:10:28.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.350+0000 7ff633200640 1 -- 172.21.15.19:0/219690990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff634075940 con 0x7ff63414e2c0 2024-03-28T13:10:28.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.351+0000 7ff62aa00640 1 -- 172.21.15.19:0/219690990 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff61c007210 con 0x7ff63414e2c0 2024-03-28T13:10:28.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.352+0000 7ff62aa00640 1 --2- 172.21.15.19:0/219690990 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff61006c530 0x7ff61006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.352+0000 7ff62aa00640 1 -- 172.21.15.19:0/219690990 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7ff61c0ac780 con 0x7ff63414e2c0 2024-03-28T13:10:28.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.353+0000 7ff631e00640 1 --2- 172.21.15.19:0/219690990 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff61006c530 0x7ff61006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.354+0000 7ff631e00640 1 --2- 172.21.15.19:0/219690990 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff61006c530 0x7ff61006e9f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff624007440 tx=0x7ff62400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:28.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.357+0000 7ff62aa00640 1 -- 172.21.15.19:0/219690990 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff61c07ad50 con 0x7ff63414e2c0 2024-03-28T13:10:28.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.559+0000 7ff633200640 1 -- 172.21.15.19:0/219690990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff63415bad0 con 0x7ff63414e2c0 2024-03-28T13:10:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.561+0000 7ff62aa00640 1 -- 172.21.15.19:0/219690990 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16704 (secure 0 0 0) 0x7ff61c07a650 con 0x7ff63414e2c0 2024-03-28T13:10:28.562 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:28.562 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:10:23.142768+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":0,"to":4}]},{"pgid":"3.1b","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:10:28.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.564+0000 7ff633200640 1 -- 172.21.15.19:0/219690990 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff61006c530 msgr2=0x7ff61006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:28.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.564+0000 7ff633200640 1 --2- 172.21.15.19:0/219690990 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff61006c530 0x7ff61006e9f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff624007440 tx=0x7ff62400b040 comp rx=0 tx=0).stop 2024-03-28T13:10:28.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.564+0000 7ff633200640 1 -- 172.21.15.19:0/219690990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff63414e2c0 msgr2=0x7ff63414e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:28.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.564+0000 7ff633200640 1 --2- 172.21.15.19:0/219690990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff63414e2c0 0x7ff63414e740 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00c270 tx=0x7ff6340d2d80 comp rx=0 tx=0).stop 2024-03-28T13:10:28.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.564+0000 7ff633200640 1 -- 172.21.15.19:0/219690990 shutdown_connections 2024-03-28T13:10:28.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.564+0000 7ff633200640 1 --2- 172.21.15.19:0/219690990 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff61006c530 0x7ff61006e9f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.564+0000 7ff633200640 1 --2- 172.21.15.19:0/219690990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff63415f340 0x7ff63415f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.564+0000 7ff633200640 1 --2- 172.21.15.19:0/219690990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff63414e2c0 0x7ff63414e740 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.564+0000 7ff633200640 1 --2- 172.21.15.19:0/219690990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff618003f50 0x7ff6341f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.564+0000 7ff633200640 1 -- 172.21.15.19:0/219690990 >> 172.21.15.19:0/219690990 conn(0x7ff6340676d0 msgr2=0x7ff6340ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:28.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.564+0000 7ff633200640 1 -- 172.21.15.19:0/219690990 shutdown_connections 2024-03-28T13:10:28.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.564+0000 7ff633200640 1 -- 172.21.15.19:0/219690990 wait complete. 2024-03-28T13:10:28.584 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:09:39.243813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:09:40.092899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:10:28.585 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:10:28.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.800+0000 7f9afda00640 1 Processor -- start 2024-03-28T13:10:28.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.801+0000 7f9afda00640 1 -- start start 2024-03-28T13:10:28.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.802+0000 7f9afda00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9af80d0990 0x7f9af80aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.802+0000 7f9afda00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9af80afab0 con 0x7f9af8153580 2024-03-28T13:10:28.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.802+0000 7f9afda00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9af815cc80 con 0x7f9af80af2e0 2024-03-28T13:10:28.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.802+0000 7f9afda00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9af815da00 con 0x7f9af80d0990 2024-03-28T13:10:28.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.802+0000 7f9af6a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9af8153580 0x7f9af80d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54442/0 (socket says 172.21.15.19:54442) 2024-03-28T13:10:28.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.802+0000 7f9af6a00640 1 -- 172.21.15.19:0/2204025763 learned_addr learned my addr 172.21.15.19:0/2204025763 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:28.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.803+0000 7f9af6000640 1 -- 172.21.15.19:0/2204025763 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 801463166 0 0) 0x7f9af80afab0 con 0x7f9af8153580 2024-03-28T13:10:28.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.803+0000 7f9af6000640 1 -- 172.21.15.19:0/2204025763 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9adc003520 con 0x7f9af8153580 2024-03-28T13:10:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.803+0000 7f9aee000640 1 --2- 172.21.15.19:0/2204025763 >> v2:172.21.15.188:3300/0 conn(0x7f9af80d0990 0x7f9af80aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.803+0000 7f9af6000640 1 -- 172.21.15.19:0/2204025763 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 476590503 0 0) 0x7f9adc003520 con 0x7f9af8153580 2024-03-28T13:10:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.803+0000 7f9af6000640 1 -- 172.21.15.19:0/2204025763 >> v2:172.21.15.188:3300/0 conn(0x7f9af80d0990 msgr2=0x7f9af80aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.803+0000 7f9af6000640 1 --2- 172.21.15.19:0/2204025763 >> v2:172.21.15.188:3300/0 conn(0x7f9af80d0990 0x7f9af80aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.803+0000 7f9af6000640 1 -- 172.21.15.19:0/2204025763 >> v1:172.21.15.188:6789/0 conn(0x7f9af80af2e0 legacy=0x7f9af81600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.803+0000 7f9af6000640 1 -- 172.21.15.19:0/2204025763 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af815cf20 con 0x7f9af8153580 2024-03-28T13:10:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.803+0000 7f9af6000640 1 -- 172.21.15.19:0/2204025763 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f9ae80019f0 con 0x7f9af8153580 2024-03-28T13:10:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.803+0000 7f9af6000640 1 -- 172.21.15.19:0/2204025763 >> v1:172.21.15.19:6789/0 conn(0x7f9af8153580 legacy=0x7f9af80d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.804+0000 7f9af6000640 1 --2- 172.21.15.19:0/2204025763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9adc0039b0 0x7f9adc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.804+0000 7f9af6000640 1 --2- 172.21.15.19:0/2204025763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc004800 0x7f9adc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.804+0000 7f9af6000640 1 --2- 172.21.15.19:0/2204025763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9adc0052e0 0x7f9adc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.804+0000 7f9af6000640 1 -- 172.21.15.19:0/2204025763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9af815da00 con 0x7f9adc004800 2024-03-28T13:10:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.804+0000 7f9af6000640 1 -- 172.21.15.19:0/2204025763 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc00c1b0 con 0x7f9adc0052e0 2024-03-28T13:10:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.804+0000 7f9af6000640 1 -- 172.21.15.19:0/2204025763 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9adc00c3b0 con 0x7f9adc0039b0 2024-03-28T13:10:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.804+0000 7f9aee000640 1 --2- 172.21.15.19:0/2204025763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc004800 0x7f9adc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.804+0000 7f9af6a00640 1 --2- 172.21.15.19:0/2204025763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9adc0039b0 0x7f9adc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.804+0000 7f9af7400640 1 --2- 172.21.15.19:0/2204025763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9adc0052e0 0x7f9adc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.805+0000 7f9aee000640 1 -- 172.21.15.19:0/2204025763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9adc0039b0 msgr2=0x7f9adc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.805+0000 7f9aee000640 1 --2- 172.21.15.19:0/2204025763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9adc0039b0 0x7f9adc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.805+0000 7f9aee000640 1 -- 172.21.15.19:0/2204025763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9adc0052e0 msgr2=0x7f9adc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.805+0000 7f9aee000640 1 --2- 172.21.15.19:0/2204025763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9adc0052e0 0x7f9adc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.805+0000 7f9aee000640 1 -- 172.21.15.19:0/2204025763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9adc003520 con 0x7f9adc004800 2024-03-28T13:10:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.805+0000 7f9af7400640 1 -- 172.21.15.19:0/2204025763 reap_dead start 2024-03-28T13:10:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.807+0000 7f9aee000640 1 --2- 172.21.15.19:0/2204025763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc004800 0x7f9adc004c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9af80713f0 tx=0x7f9adc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2993fb773e754358 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.807+0000 7f9af6000640 1 -- 172.21.15.19:0/2204025763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000b610 con 0x7f9adc004800 2024-03-28T13:10:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.807+0000 7f9af6000640 1 -- 172.21.15.19:0/2204025763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae0038070 con 0x7f9adc004800 2024-03-28T13:10:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.807+0000 7f9af6000640 1 -- 172.21.15.19:0/2204025763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae00338a0 con 0x7f9adc004800 2024-03-28T13:10:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.808+0000 7f9afda00640 1 -- 172.21.15.19:0/2204025763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc004800 msgr2=0x7f9adc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.808+0000 7f9afda00640 1 --2- 172.21.15.19:0/2204025763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc004800 0x7f9adc004c80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9af80713f0 tx=0x7f9adc005830 comp rx=0 tx=0).stop 2024-03-28T13:10:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.808+0000 7f9afda00640 1 -- 172.21.15.19:0/2204025763 shutdown_connections 2024-03-28T13:10:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.808+0000 7f9afda00640 1 --2- 172.21.15.19:0/2204025763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc004800 0x7f9adc004c80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.808+0000 7f9afda00640 1 -- 172.21.15.19:0/2204025763 >> 172.21.15.19:0/2204025763 conn(0x7f9af806b750 msgr2=0x7f9af806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.808+0000 7f9afda00640 1 -- 172.21.15.19:0/2204025763 shutdown_connections 2024-03-28T13:10:28.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.808+0000 7f9afda00640 1 -- 172.21.15.19:0/2204025763 wait complete. 2024-03-28T13:10:28.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.810+0000 7f9afda00640 1 Processor -- start 2024-03-28T13:10:28.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.810+0000 7f9afda00640 1 -- start start 2024-03-28T13:10:28.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.811+0000 7f9afda00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc004800 0x7f9af80cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.811+0000 7f9afda00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9af814c4a0 0x7f9af8150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.811+0000 7f9afda00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af80ccbe0 0x7f9af80cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.811+0000 7f9afda00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9af8153960 con 0x7f9adc004800 2024-03-28T13:10:28.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.811+0000 7f9afda00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9af815d4e0 con 0x7f9af80ccbe0 2024-03-28T13:10:28.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.811+0000 7f9afda00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9af80af6c0 con 0x7f9af814c4a0 2024-03-28T13:10:28.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.811+0000 7f9af6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc004800 0x7f9af80cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.811+0000 7f9af6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc004800 0x7f9af80cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40222/0 (socket says 172.21.15.19:40222) 2024-03-28T13:10:28.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.811+0000 7f9af6a00640 1 -- 172.21.15.19:0/731125208 learned_addr learned my addr 172.21.15.19:0/731125208 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:28.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.812+0000 7f9aee000640 1 --2- 172.21.15.19:0/731125208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9af814c4a0 0x7f9af8150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.812+0000 7f9af6a00640 1 -- 172.21.15.19:0/731125208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9af814c4a0 msgr2=0x7f9af8150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:28.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.812+0000 7f9af6a00640 1 --2- 172.21.15.19:0/731125208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9af814c4a0 0x7f9af8150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.812+0000 7f9af6a00640 1 -- 172.21.15.19:0/731125208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af80ccbe0 msgr2=0x7f9af80cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:28.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.812+0000 7f9af6a00640 1 --2- 172.21.15.19:0/731125208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af80ccbe0 0x7f9af80cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:28.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.812+0000 7f9af6a00640 1 -- 172.21.15.19:0/731125208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af8150960 con 0x7f9adc004800 2024-03-28T13:10:28.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.812+0000 7f9af6a00640 1 --2- 172.21.15.19:0/731125208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc004800 0x7f9af80cf6a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8007970 tx=0x7f9ae800a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:28.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.812+0000 7f9aef400640 1 -- 172.21.15.19:0/731125208 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8017020 con 0x7f9adc004800 2024-03-28T13:10:28.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.812+0000 7f9aef400640 1 -- 172.21.15.19:0/731125208 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae80133f0 con 0x7f9adc004800 2024-03-28T13:10:28.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.812+0000 7f9afda00640 1 -- 172.21.15.19:0/731125208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af8150c40 con 0x7f9adc004800 2024-03-28T13:10:28.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.812+0000 7f9aef400640 1 -- 172.21.15.19:0/731125208 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae80137c0 con 0x7f9adc004800 2024-03-28T13:10:28.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.813+0000 7f9afda00640 1 -- 172.21.15.19:0/731125208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af8151200 con 0x7f9adc004800 2024-03-28T13:10:28.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.814+0000 7f9afda00640 1 -- 172.21.15.19:0/731125208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af814c880 con 0x7f9adc004800 2024-03-28T13:10:28.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.815+0000 7f9aef400640 1 -- 172.21.15.19:0/731125208 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9ae801a020 con 0x7f9adc004800 2024-03-28T13:10:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.816+0000 7f9aef400640 1 --2- 172.21.15.19:0/731125208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ad406c560 0x7f9ad406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.816+0000 7f9aef400640 1 -- 172.21.15.19:0/731125208 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f9ae808c4c0 con 0x7f9adc004800 2024-03-28T13:10:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.816+0000 7f9aee000640 1 --2- 172.21.15.19:0/731125208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ad406c560 0x7f9ad406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.817+0000 7f9aee000640 1 --2- 172.21.15.19:0/731125208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ad406c560 0x7f9ad406ea20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9adc004780 tx=0x7f9ae0030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:28.820+0000 7f9aef400640 1 -- 172.21.15.19:0/731125208 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9ae8021020 con 0x7f9adc004800 2024-03-28T13:10:29.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.060+0000 7f9afda00640 1 -- 172.21.15.19:0/731125208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9af80685c0 con 0x7f9adc004800 2024-03-28T13:10:29.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.061+0000 7f9aef400640 1 -- 172.21.15.19:0/731125208 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f9ae805aa90 con 0x7f9adc004800 2024-03-28T13:10:29.061 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:10:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.063+0000 7f9afda00640 1 -- 172.21.15.19:0/731125208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ad406c560 msgr2=0x7f9ad406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.063+0000 7f9afda00640 1 --2- 172.21.15.19:0/731125208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ad406c560 0x7f9ad406ea20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9adc004780 tx=0x7f9ae0030000 comp rx=0 tx=0).stop 2024-03-28T13:10:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.063+0000 7f9afda00640 1 -- 172.21.15.19:0/731125208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc004800 msgr2=0x7f9af80cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.063+0000 7f9afda00640 1 --2- 172.21.15.19:0/731125208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc004800 0x7f9af80cf6a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8007970 tx=0x7f9ae800a730 comp rx=0 tx=0).stop 2024-03-28T13:10:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.063+0000 7f9afda00640 1 -- 172.21.15.19:0/731125208 shutdown_connections 2024-03-28T13:10:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.063+0000 7f9afda00640 1 --2- 172.21.15.19:0/731125208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ad406c560 0x7f9ad406ea20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.063+0000 7f9afda00640 1 --2- 172.21.15.19:0/731125208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9af80ccbe0 0x7f9af80cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.063+0000 7f9afda00640 1 --2- 172.21.15.19:0/731125208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9af814c4a0 0x7f9af8150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.063+0000 7f9afda00640 1 --2- 172.21.15.19:0/731125208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9adc004800 0x7f9af80cf6a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.063+0000 7f9afda00640 1 -- 172.21.15.19:0/731125208 >> 172.21.15.19:0/731125208 conn(0x7f9af806b750 msgr2=0x7f9af80b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.064+0000 7f9afda00640 1 -- 172.21.15.19:0/731125208 shutdown_connections 2024-03-28T13:10:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.064+0000 7f9afda00640 1 -- 172.21.15.19:0/731125208 wait complete. 2024-03-28T13:10:29.084 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:10:29.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.295+0000 7fc2d1800640 1 Processor -- start 2024-03-28T13:10:29.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.295+0000 7fc2d1800640 1 -- start start 2024-03-28T13:10:29.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.297+0000 7fc2d1800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc146f70 0x7fc2cc070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:29.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.297+0000 7fc2d1800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc2cc147350 0x7fc2cc0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:29.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.297+0000 7fc2d1800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2cc06cbf0 con 0x7fc2cc06c680 2024-03-28T13:10:29.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.297+0000 7fc2d1800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc2cc0cffe0 con 0x7fc2cc147350 2024-03-28T13:10:29.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.297+0000 7fc2d1800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc2cc0d0230 con 0x7fc2cc146f70 2024-03-28T13:10:29.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.298+0000 7fc2cbe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc146f70 0x7fc2cc070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:29.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.298+0000 7fc2cbe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc146f70 0x7fc2cc070430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:51372/0 (socket says 172.21.15.19:51372) 2024-03-28T13:10:29.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.298+0000 7fc2cbe00640 1 -- 172.21.15.19:0/3137690860 learned_addr learned my addr 172.21.15.19:0/3137690860 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:29.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.298+0000 7fc2cb400640 1 --2- 172.21.15.19:0/3137690860 >> v2:172.21.15.188:3300/0 conn(0x7fc2cc147350 0x7fc2cc0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:29.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.299+0000 7fc2cbe00640 1 -- 172.21.15.19:0/3137690860 >> v2:172.21.15.188:3300/0 conn(0x7fc2cc147350 msgr2=0x7fc2cc0d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:29.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.299+0000 7fc2cbe00640 1 --2- 172.21.15.19:0/3137690860 >> v2:172.21.15.188:3300/0 conn(0x7fc2cc147350 0x7fc2cc0d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.299+0000 7fc2cbe00640 1 -- 172.21.15.19:0/3137690860 >> v1:172.21.15.188:6789/0 conn(0x7fc2cc06c680 legacy=0x7fc2cc1620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:29.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.299+0000 7fc2cbe00640 1 -- 172.21.15.19:0/3137690860 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc0d0510 con 0x7fc2cc146f70 2024-03-28T13:10:29.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.300+0000 7fc2cbe00640 1 --2- 172.21.15.19:0/3137690860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc146f70 0x7fc2cc070430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009870 tx=0x7fc2bc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e4bce2a6c6bb54a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:29.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.300+0000 7fc2caa00640 1 -- 172.21.15.19:0/3137690860 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc035020 con 0x7fc2cc146f70 2024-03-28T13:10:29.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.301+0000 7fc2caa00640 1 -- 172.21.15.19:0/3137690860 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2bc031640 con 0x7fc2cc146f70 2024-03-28T13:10:29.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.301+0000 7fc2caa00640 1 -- 172.21.15.19:0/3137690860 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc02d040 con 0x7fc2cc146f70 2024-03-28T13:10:29.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.301+0000 7fc2d1800640 1 -- 172.21.15.19:0/3137690860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc146f70 msgr2=0x7fc2cc070430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:29.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.301+0000 7fc2d1800640 1 --2- 172.21.15.19:0/3137690860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc146f70 0x7fc2cc070430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009870 tx=0x7fc2bc02b8f0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.301+0000 7fc2d1800640 1 -- 172.21.15.19:0/3137690860 shutdown_connections 2024-03-28T13:10:29.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.301+0000 7fc2d1800640 1 --2- 172.21.15.19:0/3137690860 >> v2:172.21.15.188:3300/0 conn(0x7fc2cc147350 0x7fc2cc0d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.301+0000 7fc2d1800640 1 --2- 172.21.15.19:0/3137690860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc146f70 0x7fc2cc070430 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.301+0000 7fc2d1800640 1 -- 172.21.15.19:0/3137690860 >> 172.21.15.19:0/3137690860 conn(0x7fc2cc067280 msgr2=0x7fc2cc067660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:29.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.301+0000 7fc2d1800640 1 -- 172.21.15.19:0/3137690860 shutdown_connections 2024-03-28T13:10:29.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.301+0000 7fc2d1800640 1 -- 172.21.15.19:0/3137690860 wait complete. 2024-03-28T13:10:29.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.303+0000 7fc2d1800640 1 Processor -- start 2024-03-28T13:10:29.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.303+0000 7fc2d1800640 1 -- start start 2024-03-28T13:10:29.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.304+0000 7fc2d1800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2cc06c680 0x7fc2cc143820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:29.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.304+0000 7fc2d1800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc146f70 0x7fc2cc143d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:29.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.304+0000 7fc2d1800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2cc147350 0x7fc2cc145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:29.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.304+0000 7fc2d1800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc0d1250 con 0x7fc2cc06c680 2024-03-28T13:10:29.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.304+0000 7fc2d1800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc2cc070970 con 0x7fc2cc147350 2024-03-28T13:10:29.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.304+0000 7fc2d1800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc2cc1f4cb0 con 0x7fc2cc146f70 2024-03-28T13:10:29.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.305+0000 7fc2cbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2cc06c680 0x7fc2cc143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:29.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.305+0000 7fc2cbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2cc06c680 0x7fc2cc143820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40230/0 (socket says 172.21.15.19:40230) 2024-03-28T13:10:29.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.305+0000 7fc2cbe00640 1 -- 172.21.15.19:0/1374897177 learned_addr learned my addr 172.21.15.19:0/1374897177 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:29.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.305+0000 7fc2cb400640 1 --2- 172.21.15.19:0/1374897177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc146f70 0x7fc2cc143d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.305+0000 7fc2d0e00640 1 --2- 172.21.15.19:0/1374897177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2cc147350 0x7fc2cc145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.305+0000 7fc2cbe00640 1 -- 172.21.15.19:0/1374897177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc146f70 msgr2=0x7fc2cc143d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.305+0000 7fc2cbe00640 1 --2- 172.21.15.19:0/1374897177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc146f70 0x7fc2cc143d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.305+0000 7fc2cbe00640 1 -- 172.21.15.19:0/1374897177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2cc147350 msgr2=0x7fc2cc145ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.305+0000 7fc2cbe00640 1 --2- 172.21.15.19:0/1374897177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2cc147350 0x7fc2cc145ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.305+0000 7fc2cbe00640 1 -- 172.21.15.19:0/1374897177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2cc1f4f90 con 0x7fc2cc06c680 2024-03-28T13:10:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.305+0000 7fc2d0e00640 1 --2- 172.21.15.19:0/1374897177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2cc147350 0x7fc2cc145ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.306+0000 7fc2cbe00640 1 --2- 172.21.15.19:0/1374897177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2cc06c680 0x7fc2cc143820 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc000c00 tx=0x7fc2bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.306+0000 7fc2c3e00640 1 -- 172.21.15.19:0/1374897177 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc040070 con 0x7fc2cc06c680 2024-03-28T13:10:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.306+0000 7fc2c3e00640 1 -- 172.21.15.19:0/1374897177 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2bc0399f0 con 0x7fc2cc06c680 2024-03-28T13:10:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.306+0000 7fc2c3e00640 1 -- 172.21.15.19:0/1374897177 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc035040 con 0x7fc2cc06c680 2024-03-28T13:10:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.306+0000 7fc2d1800640 1 -- 172.21.15.19:0/1374897177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2cc1f5270 con 0x7fc2cc06c680 2024-03-28T13:10:29.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.306+0000 7fc2d1800640 1 -- 172.21.15.19:0/1374897177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2cc1f5830 con 0x7fc2cc06c680 2024-03-28T13:10:29.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.308+0000 7fc2d1800640 1 -- 172.21.15.19:0/1374897177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2cc075940 con 0x7fc2cc06c680 2024-03-28T13:10:29.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.309+0000 7fc2c3e00640 1 -- 172.21.15.19:0/1374897177 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc2bc04b060 con 0x7fc2cc06c680 2024-03-28T13:10:29.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.310+0000 7fc2c3e00640 1 --2- 172.21.15.19:0/1374897177 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc2ac06c530 0x7fc2ac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:29.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.310+0000 7fc2c3e00640 1 -- 172.21.15.19:0/1374897177 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7fc2bc0ad110 con 0x7fc2cc06c680 2024-03-28T13:10:29.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.310+0000 7fc2cb400640 1 --2- 172.21.15.19:0/1374897177 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc2ac06c530 0x7fc2ac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:29.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.312+0000 7fc2cb400640 1 --2- 172.21.15.19:0/1374897177 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc2ac06c530 0x7fc2ac06e9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc146740 tx=0x7fc2b8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:29.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.315+0000 7fc2c3e00640 1 -- 172.21.15.19:0/1374897177 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc2bc07b5c0 con 0x7fc2cc06c680 2024-03-28T13:10:29.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.516+0000 7fc2d1800640 1 -- 172.21.15.19:0/1374897177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc2cc146790 con 0x7fc2cc06c680 2024-03-28T13:10:29.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.517+0000 7fc2c3e00640 1 -- 172.21.15.19:0/1374897177 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fc2bc07aec0 con 0x7fc2cc06c680 2024-03-28T13:10:29.518 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:10:29.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.520+0000 7fc2d1800640 1 -- 172.21.15.19:0/1374897177 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc2ac06c530 msgr2=0x7fc2ac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:29.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.520+0000 7fc2d1800640 1 --2- 172.21.15.19:0/1374897177 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc2ac06c530 0x7fc2ac06e9f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc146740 tx=0x7fc2b8008040 comp rx=0 tx=0).stop 2024-03-28T13:10:29.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.520+0000 7fc2d1800640 1 -- 172.21.15.19:0/1374897177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2cc06c680 msgr2=0x7fc2cc143820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:29.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.520+0000 7fc2d1800640 1 --2- 172.21.15.19:0/1374897177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2cc06c680 0x7fc2cc143820 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc000c00 tx=0x7fc2bc0046c0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.520+0000 7fc2d1800640 1 -- 172.21.15.19:0/1374897177 shutdown_connections 2024-03-28T13:10:29.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.520+0000 7fc2d1800640 1 --2- 172.21.15.19:0/1374897177 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc2ac06c530 0x7fc2ac06e9f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.520+0000 7fc2d1800640 1 --2- 172.21.15.19:0/1374897177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc2cc147350 0x7fc2cc145ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.520+0000 7fc2d1800640 1 --2- 172.21.15.19:0/1374897177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc2cc146f70 0x7fc2cc143d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.520+0000 7fc2d1800640 1 --2- 172.21.15.19:0/1374897177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2cc06c680 0x7fc2cc143820 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.520+0000 7fc2d1800640 1 -- 172.21.15.19:0/1374897177 >> 172.21.15.19:0/1374897177 conn(0x7fc2cc067280 msgr2=0x7fc2cc160940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.520+0000 7fc2d1800640 1 -- 172.21.15.19:0/1374897177 shutdown_connections 2024-03-28T13:10:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.520+0000 7fc2d1800640 1 -- 172.21.15.19:0/1374897177 wait complete. 2024-03-28T13:10:29.538 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:10:29.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.751+0000 7f3ca7200640 1 Processor -- start 2024-03-28T13:10:29.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.751+0000 7f3ca7200640 1 -- start start 2024-03-28T13:10:29.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.754+0000 7f3ca7200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3ca015e860 0x7f3ca006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:29.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.754+0000 7f3ca7200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ca006da30 0x7f3ca00d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:29.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.754+0000 7f3ca7200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ca006dfa0 con 0x7f3ca00d4910 2024-03-28T13:10:29.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.754+0000 7f3ca7200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3ca00cf220 con 0x7f3ca006da30 2024-03-28T13:10:29.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.754+0000 7f3ca7200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3ca00d4cf0 con 0x7f3ca015e860 2024-03-28T13:10:29.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.754+0000 7f3c97e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ca006da30 0x7f3ca00d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:29.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.754+0000 7f3c97e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3ca006da30 0x7f3ca00d43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40242/0 (socket says 172.21.15.19:40242) 2024-03-28T13:10:29.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.754+0000 7f3c97e00640 1 -- 172.21.15.19:0/57408041 learned_addr learned my addr 172.21.15.19:0/57408041 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:29.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.755+0000 7f3c9fe00640 1 --2- 172.21.15.19:0/57408041 >> v2:172.21.15.188:3300/0 conn(0x7f3ca015e860 0x7f3ca006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:29.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.755+0000 7f3c97e00640 1 -- 172.21.15.19:0/57408041 >> v2:172.21.15.188:3300/0 conn(0x7f3ca015e860 msgr2=0x7f3ca006d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:29.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.755+0000 7f3c97e00640 1 --2- 172.21.15.19:0/57408041 >> v2:172.21.15.188:3300/0 conn(0x7f3ca015e860 0x7f3ca006d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.755+0000 7f3c97e00640 1 -- 172.21.15.19:0/57408041 >> v1:172.21.15.188:6789/0 conn(0x7f3ca00d4910 legacy=0x7f3ca00d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:29.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.755+0000 7f3c97e00640 1 -- 172.21.15.19:0/57408041 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca006e240 con 0x7f3ca006da30 2024-03-28T13:10:29.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.756+0000 7f3c97e00640 1 --2- 172.21.15.19:0/57408041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca006da30 0x7f3ca00d43d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0162160 tx=0x7f3c8802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ad9df31c066ce9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:29.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.756+0000 7f3c9f400640 1 -- 172.21.15.19:0/57408041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c88035020 con 0x7f3ca006da30 2024-03-28T13:10:29.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.756+0000 7f3c9f400640 1 -- 172.21.15.19:0/57408041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c88031640 con 0x7f3ca006da30 2024-03-28T13:10:29.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.756+0000 7f3c9f400640 1 -- 172.21.15.19:0/57408041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8802d040 con 0x7f3ca006da30 2024-03-28T13:10:29.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.757+0000 7f3ca7200640 1 -- 172.21.15.19:0/57408041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca006da30 msgr2=0x7f3ca00d43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:29.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.757+0000 7f3ca7200640 1 --2- 172.21.15.19:0/57408041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca006da30 0x7f3ca00d43d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0162160 tx=0x7f3c8802b8f0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.757+0000 7f3ca7200640 1 -- 172.21.15.19:0/57408041 shutdown_connections 2024-03-28T13:10:29.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.757+0000 7f3ca7200640 1 --2- 172.21.15.19:0/57408041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca006da30 0x7f3ca00d43d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.757+0000 7f3ca7200640 1 --2- 172.21.15.19:0/57408041 >> v2:172.21.15.188:3300/0 conn(0x7f3ca015e860 0x7f3ca006d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.757+0000 7f3ca7200640 1 -- 172.21.15.19:0/57408041 >> 172.21.15.19:0/57408041 conn(0x7f3ca009d7d0 msgr2=0x7f3ca008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:29.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.757+0000 7f3ca7200640 1 -- 172.21.15.19:0/57408041 shutdown_connections 2024-03-28T13:10:29.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.757+0000 7f3ca7200640 1 -- 172.21.15.19:0/57408041 wait complete. 2024-03-28T13:10:29.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.761+0000 7f3ca7200640 1 Processor -- start 2024-03-28T13:10:29.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.761+0000 7f3ca7200640 1 -- start start 2024-03-28T13:10:29.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.763+0000 7f3ca7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca006da30 0x7f3ca014faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:29.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.763+0000 7f3ca7200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca00d4910 0x7f3ca00cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:29.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.763+0000 7f3ca7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ca015e860 0x7f3ca00cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:29.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.763+0000 7f3ca7200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca00cedb0 con 0x7f3ca006da30 2024-03-28T13:10:29.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.763+0000 7f3ca7200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca00d2c50 con 0x7f3ca00d4910 2024-03-28T13:10:29.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.763+0000 7f3ca7200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca00cef30 con 0x7f3ca015e860 2024-03-28T13:10:29.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.763+0000 7f3c9fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca006da30 0x7f3ca014faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:29.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.763+0000 7f3c9fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca006da30 0x7f3ca014faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40244/0 (socket says 172.21.15.19:40244) 2024-03-28T13:10:29.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.764+0000 7f3c9fe00640 1 -- 172.21.15.19:0/1107978681 learned_addr learned my addr 172.21.15.19:0/1107978681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:29.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.764+0000 7f3c9fe00640 1 -- 172.21.15.19:0/1107978681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ca015e860 msgr2=0x7f3ca00cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:29.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.764+0000 7f3c9fe00640 1 --2- 172.21.15.19:0/1107978681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ca015e860 0x7f3ca00cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.764+0000 7f3c9fe00640 1 -- 172.21.15.19:0/1107978681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca00d4910 msgr2=0x7f3ca00cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:29.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.764+0000 7f3c97e00640 1 --2- 172.21.15.19:0/1107978681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca00d4910 0x7f3ca00cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:29.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.764+0000 7f3c9fe00640 1 --2- 172.21.15.19:0/1107978681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca00d4910 0x7f3ca00cd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.764+0000 7f3c9fe00640 1 -- 172.21.15.19:0/1107978681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca00cde00 con 0x7f3ca006da30 2024-03-28T13:10:29.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.764+0000 7f3c9fe00640 1 --2- 172.21.15.19:0/1107978681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca006da30 0x7f3ca014faa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000d5f0 tx=0x7f3c9000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:29.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.765+0000 7f3c9cc00640 1 -- 172.21.15.19:0/1107978681 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c900125f0 con 0x7f3ca006da30 2024-03-28T13:10:29.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.765+0000 7f3c9cc00640 1 -- 172.21.15.19:0/1107978681 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c90019070 con 0x7f3ca006da30 2024-03-28T13:10:29.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.765+0000 7f3ca7200640 1 -- 172.21.15.19:0/1107978681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca00cc820 con 0x7f3ca006da30 2024-03-28T13:10:29.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.765+0000 7f3c9cc00640 1 -- 172.21.15.19:0/1107978681 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c900129d0 con 0x7f3ca006da30 2024-03-28T13:10:29.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.765+0000 7f3ca7200640 1 -- 172.21.15.19:0/1107978681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca00ccde0 con 0x7f3ca006da30 2024-03-28T13:10:29.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.766+0000 7f3ca7200640 1 -- 172.21.15.19:0/1107978681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca00685c0 con 0x7f3ca006da30 2024-03-28T13:10:29.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.767+0000 7f3c9cc00640 1 -- 172.21.15.19:0/1107978681 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3c90012b70 con 0x7f3ca006da30 2024-03-28T13:10:29.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.768+0000 7f3c9cc00640 1 --2- 172.21.15.19:0/1107978681 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c7c06c560 0x7f3c7c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:29.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.768+0000 7f3c9cc00640 1 -- 172.21.15.19:0/1107978681 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f3c9008bb70 con 0x7f3ca006da30 2024-03-28T13:10:29.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.769+0000 7f3c97e00640 1 --2- 172.21.15.19:0/1107978681 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c7c06c560 0x7f3c7c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:29.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.770+0000 7f3c97e00640 1 --2- 172.21.15.19:0/1107978681 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c7c06c560 0x7f3c7c06ea20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3c88036000 tx=0x7f3c88008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:29.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.773+0000 7f3c9cc00640 1 -- 172.21.15.19:0/1107978681 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3c9005a0b0 con 0x7f3ca006da30 2024-03-28T13:10:29.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.979+0000 7f3ca7200640 1 -- 172.21.15.19:0/1107978681 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3ca00b6ac0 con 0x7f3ca006da30 2024-03-28T13:10:29.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.980+0000 7f3c9cc00640 1 -- 172.21.15.19:0/1107978681 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f3c900599b0 con 0x7f3ca006da30 2024-03-28T13:10:29.980 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:10:29.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.982+0000 7f3ca7200640 1 -- 172.21.15.19:0/1107978681 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c7c06c560 msgr2=0x7f3c7c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:29.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.983+0000 7f3ca7200640 1 --2- 172.21.15.19:0/1107978681 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c7c06c560 0x7f3c7c06ea20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3c88036000 tx=0x7f3c88008030 comp rx=0 tx=0).stop 2024-03-28T13:10:29.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.983+0000 7f3ca7200640 1 -- 172.21.15.19:0/1107978681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca006da30 msgr2=0x7f3ca014faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:29.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.983+0000 7f3ca7200640 1 --2- 172.21.15.19:0/1107978681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca006da30 0x7f3ca014faa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000d5f0 tx=0x7f3c9000dab0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.983+0000 7f3ca7200640 1 -- 172.21.15.19:0/1107978681 shutdown_connections 2024-03-28T13:10:29.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.983+0000 7f3ca7200640 1 --2- 172.21.15.19:0/1107978681 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c7c06c560 0x7f3c7c06ea20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.983+0000 7f3ca7200640 1 --2- 172.21.15.19:0/1107978681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ca015e860 0x7f3ca00cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.983+0000 7f3ca7200640 1 --2- 172.21.15.19:0/1107978681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca00d4910 0x7f3ca00cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.983+0000 7f3ca7200640 1 --2- 172.21.15.19:0/1107978681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca006da30 0x7f3ca014faa0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:29.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.983+0000 7f3ca7200640 1 -- 172.21.15.19:0/1107978681 >> 172.21.15.19:0/1107978681 conn(0x7f3ca009d7d0 msgr2=0x7f3ca00b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:29.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.983+0000 7f3ca7200640 1 -- 172.21.15.19:0/1107978681 shutdown_connections 2024-03-28T13:10:29.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:29.983+0000 7f3ca7200640 1 -- 172.21.15.19:0/1107978681 wait complete. 2024-03-28T13:10:30.002 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:10:30.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.216+0000 7faf5be00640 1 Processor -- start 2024-03-28T13:10:30.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.216+0000 7faf5be00640 1 -- start start 2024-03-28T13:10:30.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.217+0000 7faf5be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faf540d0990 0x7faf540aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:30.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.217+0000 7faf5be00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf540afab0 con 0x7faf54153580 2024-03-28T13:10:30.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.217+0000 7faf5be00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf5415cc80 con 0x7faf540af2e0 2024-03-28T13:10:30.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.217+0000 7faf5be00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7faf5415da00 con 0x7faf540d0990 2024-03-28T13:10:30.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.218+0000 7faf59400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7faf54153580 0x7faf540d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54458/0 (socket says 172.21.15.19:54458) 2024-03-28T13:10:30.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.218+0000 7faf59400640 1 -- 172.21.15.19:0/3546281326 learned_addr learned my addr 172.21.15.19:0/3546281326 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:30.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.218+0000 7faf58a00640 1 --2- 172.21.15.19:0/3546281326 >> v2:172.21.15.19:3300/0 conn(0x7faf540d0990 0x7faf540aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.218+0000 7faf53e00640 1 -- 172.21.15.19:0/3546281326 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1803820750 0 0) 0x7faf540afab0 con 0x7faf54153580 2024-03-28T13:10:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.218+0000 7faf53e00640 1 -- 172.21.15.19:0/3546281326 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf3c003520 con 0x7faf54153580 2024-03-28T13:10:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.218+0000 7faf58a00640 1 -- 172.21.15.19:0/3546281326 >> v1:172.21.15.188:6789/0 conn(0x7faf540af2e0 legacy=0x7faf541600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.218+0000 7faf58a00640 1 -- 172.21.15.19:0/3546281326 >> v1:172.21.15.19:6789/0 conn(0x7faf54153580 legacy=0x7faf540d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.218+0000 7faf58a00640 1 -- 172.21.15.19:0/3546281326 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5415cf20 con 0x7faf540d0990 2024-03-28T13:10:30.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.220+0000 7faf58a00640 1 --2- 172.21.15.19:0/3546281326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf540d0990 0x7faf540aeda0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faf400097a0 tx=0x7faf4002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d071f47d1110096f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:30.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.220+0000 7faf53e00640 1 -- 172.21.15.19:0/3546281326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40035020 con 0x7faf540d0990 2024-03-28T13:10:30.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.220+0000 7faf53e00640 1 -- 172.21.15.19:0/3546281326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf40031830 con 0x7faf540d0990 2024-03-28T13:10:30.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.220+0000 7faf53e00640 1 -- 172.21.15.19:0/3546281326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40031be0 con 0x7faf540d0990 2024-03-28T13:10:30.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.220+0000 7faf5be00640 1 -- 172.21.15.19:0/3546281326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf540d0990 msgr2=0x7faf540aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:30.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.221+0000 7faf5be00640 1 --2- 172.21.15.19:0/3546281326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf540d0990 0x7faf540aeda0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faf400097a0 tx=0x7faf4002b8d0 comp rx=0 tx=0).stop 2024-03-28T13:10:30.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.221+0000 7faf5be00640 1 -- 172.21.15.19:0/3546281326 shutdown_connections 2024-03-28T13:10:30.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.221+0000 7faf5be00640 1 --2- 172.21.15.19:0/3546281326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf540d0990 0x7faf540aeda0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:30.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.221+0000 7faf5be00640 1 -- 172.21.15.19:0/3546281326 >> 172.21.15.19:0/3546281326 conn(0x7faf5406b750 msgr2=0x7faf5406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:30.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.221+0000 7faf5be00640 1 -- 172.21.15.19:0/3546281326 shutdown_connections 2024-03-28T13:10:30.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.221+0000 7faf5be00640 1 -- 172.21.15.19:0/3546281326 wait complete. 2024-03-28T13:10:30.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.222+0000 7faf5be00640 1 Processor -- start 2024-03-28T13:10:30.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.223+0000 7faf5be00640 1 -- start start 2024-03-28T13:10:30.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.224+0000 7faf5be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf540af2e0 0x7faf541483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:30.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.224+0000 7faf5be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf540d0990 0x7faf54158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:30.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.224+0000 7faf5be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf54153580 0x7faf540cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:30.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.224+0000 7faf5be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf54153960 con 0x7faf540af2e0 2024-03-28T13:10:30.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.224+0000 7faf5be00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faf5415d4e0 con 0x7faf54153580 2024-03-28T13:10:30.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.224+0000 7faf5be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7faf540af6c0 con 0x7faf540d0990 2024-03-28T13:10:30.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.224+0000 7faf59400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf540af2e0 0x7faf541483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:30.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.224+0000 7faf59400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf540af2e0 0x7faf541483f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40258/0 (socket says 172.21.15.19:40258) 2024-03-28T13:10:30.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.224+0000 7faf59400640 1 -- 172.21.15.19:0/1320275357 learned_addr learned my addr 172.21.15.19:0/1320275357 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:30.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.224+0000 7faf59400640 1 -- 172.21.15.19:0/1320275357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf540d0990 msgr2=0x7faf54158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:30.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.224+0000 7faf59400640 1 --2- 172.21.15.19:0/1320275357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf540d0990 0x7faf54158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:30.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.224+0000 7faf59400640 1 -- 172.21.15.19:0/1320275357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf54153580 msgr2=0x7faf540cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:30.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.224+0000 7faf59400640 1 --2- 172.21.15.19:0/1320275357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf54153580 0x7faf540cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:30.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.224+0000 7faf59400640 1 -- 172.21.15.19:0/1320275357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf540ceb10 con 0x7faf540af2e0 2024-03-28T13:10:30.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.225+0000 7faf59400640 1 --2- 172.21.15.19:0/1320275357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf540af2e0 0x7faf541483f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faf4800c410 tx=0x7faf4800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:30.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.225+0000 7faf51600640 1 -- 172.21.15.19:0/1320275357 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf48017070 con 0x7faf540af2e0 2024-03-28T13:10:30.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.225+0000 7faf51600640 1 -- 172.21.15.19:0/1320275357 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf48005ce0 con 0x7faf540af2e0 2024-03-28T13:10:30.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.225+0000 7faf5be00640 1 -- 172.21.15.19:0/1320275357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf540cedf0 con 0x7faf540af2e0 2024-03-28T13:10:30.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.225+0000 7faf51600640 1 -- 172.21.15.19:0/1320275357 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf480072f0 con 0x7faf540af2e0 2024-03-28T13:10:30.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.225+0000 7faf5be00640 1 -- 172.21.15.19:0/1320275357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf540cf3b0 con 0x7faf540af2e0 2024-03-28T13:10:30.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.226+0000 7faf5be00640 1 -- 172.21.15.19:0/1320275357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf540685c0 con 0x7faf540af2e0 2024-03-28T13:10:30.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.227+0000 7faf51600640 1 -- 172.21.15.19:0/1320275357 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7faf4801a070 con 0x7faf540af2e0 2024-03-28T13:10:30.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.228+0000 7faf51600640 1 --2- 172.21.15.19:0/1320275357 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faf3406c530 0x7faf3406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:30.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.229+0000 7faf51600640 1 -- 172.21.15.19:0/1320275357 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7faf4808ab50 con 0x7faf540af2e0 2024-03-28T13:10:30.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.229+0000 7faf58a00640 1 --2- 172.21.15.19:0/1320275357 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faf3406c530 0x7faf3406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:30.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.230+0000 7faf58a00640 1 --2- 172.21.15.19:0/1320275357 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faf3406c530 0x7faf3406e9f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faf40036000 tx=0x7faf40030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:30.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.233+0000 7faf51600640 1 -- 172.21.15.19:0/1320275357 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7faf480591f0 con 0x7faf540af2e0 2024-03-28T13:10:30.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.446+0000 7faf5be00640 1 -- 172.21.15.19:0/1320275357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faf5406eca0 con 0x7faf540af2e0 2024-03-28T13:10:30.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.446+0000 7faf51600640 1 -- 172.21.15.19:0/1320275357 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7faf48058af0 con 0x7faf540af2e0 2024-03-28T13:10:30.447 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-03-28T13:10:30.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.449+0000 7faf5be00640 1 -- 172.21.15.19:0/1320275357 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faf3406c530 msgr2=0x7faf3406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:30.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.449+0000 7faf5be00640 1 --2- 172.21.15.19:0/1320275357 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faf3406c530 0x7faf3406e9f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faf40036000 tx=0x7faf40030000 comp rx=0 tx=0).stop 2024-03-28T13:10:30.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.449+0000 7faf5be00640 1 -- 172.21.15.19:0/1320275357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf540af2e0 msgr2=0x7faf541483f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:30.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.449+0000 7faf5be00640 1 --2- 172.21.15.19:0/1320275357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf540af2e0 0x7faf541483f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faf4800c410 tx=0x7faf4800c8d0 comp rx=0 tx=0).stop 2024-03-28T13:10:30.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.449+0000 7faf5be00640 1 -- 172.21.15.19:0/1320275357 shutdown_connections 2024-03-28T13:10:30.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.449+0000 7faf5be00640 1 --2- 172.21.15.19:0/1320275357 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faf3406c530 0x7faf3406e9f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:30.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.449+0000 7faf5be00640 1 --2- 172.21.15.19:0/1320275357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf54153580 0x7faf540cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:30.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.449+0000 7faf5be00640 1 --2- 172.21.15.19:0/1320275357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf540d0990 0x7faf54158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:30.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.449+0000 7faf5be00640 1 --2- 172.21.15.19:0/1320275357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf540af2e0 0x7faf541483f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:30.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.449+0000 7faf5be00640 1 -- 172.21.15.19:0/1320275357 >> 172.21.15.19:0/1320275357 conn(0x7faf5406b750 msgr2=0x7faf5415b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:30.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.449+0000 7faf5be00640 1 -- 172.21.15.19:0/1320275357 shutdown_connections 2024-03-28T13:10:30.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:30.449+0000 7faf5be00640 1 -- 172.21.15.19:0/1320275357 wait complete. 2024-03-28T13:10:30.470 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:10:30.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:30.471 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-03-28T13:10:30.571 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:10:30.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:30.571 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:10:30.620 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:10:30.620 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:30.678 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:10:30.678 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:30.733 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:10:30.733 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:30.781 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:10:31.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.003+0000 7f2e38600640 1 Processor -- start 2024-03-28T13:10:31.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.003+0000 7f2e38600640 1 -- start start 2024-03-28T13:10:31.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.005+0000 7f2e38600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2e30161b00 0x7f2e30162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:31.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.005+0000 7f2e38600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e300ad120 0x7f2e300d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:31.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.005+0000 7f2e38600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e30071b60 con 0x7f2e300ad580 2024-03-28T13:10:31.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.005+0000 7f2e38600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2e30161ee0 con 0x7f2e30161b00 2024-03-28T13:10:31.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.005+0000 7f2e38600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2e300b3450 con 0x7f2e300ad120 2024-03-28T13:10:31.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.005+0000 7f2e35c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2e30161b00 0x7f2e30162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.005+0000 7f2e35c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2e30161b00 0x7f2e30162ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40270/0 (socket says 172.21.15.19:40270) 2024-03-28T13:10:31.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.005+0000 7f2e35c00640 1 -- 172.21.15.19:0/917945129 learned_addr learned my addr 172.21.15.19:0/917945129 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.006+0000 7f2e35200640 1 --2- 172.21.15.19:0/917945129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e300ad120 0x7f2e300d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.006+0000 7f2e35c00640 1 -- 172.21.15.19:0/917945129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e300ad120 msgr2=0x7f2e300d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.006+0000 7f2e35c00640 1 --2- 172.21.15.19:0/917945129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e300ad120 0x7f2e300d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.006+0000 7f2e35c00640 1 -- 172.21.15.19:0/917945129 >> v1:172.21.15.188:6789/0 conn(0x7f2e300ad580 legacy=0x7f2e300b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.006+0000 7f2e35c00640 1 -- 172.21.15.19:0/917945129 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e30071e40 con 0x7f2e30161b00 2024-03-28T13:10:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.006+0000 7f2e35200640 1 --2- 172.21.15.19:0/917945129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e300ad120 0x7f2e300d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:31.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.007+0000 7f2e35c00640 1 --2- 172.21.15.19:0/917945129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e30161b00 0x7f2e30162ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009870 tx=0x7f2e2402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990be79954f6e767 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:31.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.007+0000 7f2e2fe00640 1 -- 172.21.15.19:0/917945129 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24035020 con 0x7f2e30161b00 2024-03-28T13:10:31.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.007+0000 7f2e2fe00640 1 -- 172.21.15.19:0/917945129 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e24031640 con 0x7f2e30161b00 2024-03-28T13:10:31.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.007+0000 7f2e2fe00640 1 -- 172.21.15.19:0/917945129 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e2402d040 con 0x7f2e30161b00 2024-03-28T13:10:31.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.008+0000 7f2e38600640 1 -- 172.21.15.19:0/917945129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e30161b00 msgr2=0x7f2e30162ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.008+0000 7f2e38600640 1 --2- 172.21.15.19:0/917945129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e30161b00 0x7f2e30162ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2e24009870 tx=0x7f2e2402b8f0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.008+0000 7f2e38600640 1 -- 172.21.15.19:0/917945129 shutdown_connections 2024-03-28T13:10:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.008+0000 7f2e38600640 1 --2- 172.21.15.19:0/917945129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e300ad120 0x7f2e300d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.008+0000 7f2e38600640 1 --2- 172.21.15.19:0/917945129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e30161b00 0x7f2e30162ab0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.008+0000 7f2e38600640 1 -- 172.21.15.19:0/917945129 >> 172.21.15.19:0/917945129 conn(0x7f2e3006b990 msgr2=0x7f2e3006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.008+0000 7f2e38600640 1 -- 172.21.15.19:0/917945129 shutdown_connections 2024-03-28T13:10:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.008+0000 7f2e38600640 1 -- 172.21.15.19:0/917945129 wait complete. 2024-03-28T13:10:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.010+0000 7f2e38600640 1 Processor -- start 2024-03-28T13:10:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.010+0000 7f2e38600640 1 -- start start 2024-03-28T13:10:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.011+0000 7f2e38600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e300ad120 0x7f2e301565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.011+0000 7f2e38600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e300ad580 0x7f2e30154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.011+0000 7f2e38600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e30161b00 0x7f2e301486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.011+0000 7f2e38600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e30162ff0 con 0x7f2e300ad580 2024-03-28T13:10:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.011+0000 7f2e38600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2e300b3730 con 0x7f2e300ad120 2024-03-28T13:10:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.011+0000 7f2e38600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2e300717f0 con 0x7f2e30161b00 2024-03-28T13:10:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.011+0000 7f2e35200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e300ad580 0x7f2e30154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.011+0000 7f2e35200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e300ad580 0x7f2e30154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40274/0 (socket says 172.21.15.19:40274) 2024-03-28T13:10:31.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.011+0000 7f2e35200640 1 -- 172.21.15.19:0/857604302 learned_addr learned my addr 172.21.15.19:0/857604302 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.012+0000 7f2e35c00640 1 --2- 172.21.15.19:0/857604302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e300ad120 0x7f2e301565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.012+0000 7f2e36600640 1 --2- 172.21.15.19:0/857604302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e30161b00 0x7f2e301486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.012+0000 7f2e35200640 1 -- 172.21.15.19:0/857604302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e30161b00 msgr2=0x7f2e301486e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.012+0000 7f2e35200640 1 --2- 172.21.15.19:0/857604302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e30161b00 0x7f2e301486e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.012+0000 7f2e35200640 1 -- 172.21.15.19:0/857604302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e300ad120 msgr2=0x7f2e301565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.012+0000 7f2e35200640 1 --2- 172.21.15.19:0/857604302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e300ad120 0x7f2e301565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.012+0000 7f2e35200640 1 -- 172.21.15.19:0/857604302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e301545d0 con 0x7f2e300ad580 2024-03-28T13:10:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.012+0000 7f2e35200640 1 --2- 172.21.15.19:0/857604302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e300ad580 0x7f2e30154090 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00c7f0 tx=0x7f2e1c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.013+0000 7f2e2d600640 1 -- 172.21.15.19:0/857604302 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c015070 con 0x7f2e300ad580 2024-03-28T13:10:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.013+0000 7f2e2d600640 1 -- 172.21.15.19:0/857604302 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e1c007a30 con 0x7f2e300ad580 2024-03-28T13:10:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.013+0000 7f2e2d600640 1 -- 172.21.15.19:0/857604302 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c007e00 con 0x7f2e300ad580 2024-03-28T13:10:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.013+0000 7f2e38600640 1 -- 172.21.15.19:0/857604302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e30156d80 con 0x7f2e300ad580 2024-03-28T13:10:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.013+0000 7f2e38600640 1 -- 172.21.15.19:0/857604302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e30154850 con 0x7f2e300ad580 2024-03-28T13:10:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.014+0000 7f2e38600640 1 -- 172.21.15.19:0/857604302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e30156f50 con 0x7f2e300ad580 2024-03-28T13:10:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.015+0000 7f2e2d600640 1 -- 172.21.15.19:0/857604302 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2e1c018070 con 0x7f2e300ad580 2024-03-28T13:10:31.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.016+0000 7f2e2d600640 1 --2- 172.21.15.19:0/857604302 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e1006c560 0x7f2e1006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:31.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.016+0000 7f2e2d600640 1 -- 172.21.15.19:0/857604302 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f2e1c08b610 con 0x7f2e300ad580 2024-03-28T13:10:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.017+0000 7f2e35c00640 1 --2- 172.21.15.19:0/857604302 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e1006c560 0x7f2e1006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.018+0000 7f2e35c00640 1 --2- 172.21.15.19:0/857604302 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e1006c560 0x7f2e1006ea20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2e24036000 tx=0x7f2e2403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.025+0000 7f2e2d600640 1 -- 172.21.15.19:0/857604302 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2e301545d0 con 0x7f2e300ad580 2024-03-28T13:10:31.186 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:10:31.193 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:10:31.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.234+0000 7f2e38600640 1 -- 172.21.15.19:0/857604302 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e300685c0 con 0x7f2e300ad580 2024-03-28T13:10:31.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.236+0000 7f2e2d600640 1 -- 172.21.15.19:0/857604302 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16704 (secure 0 0 0) 0x7f2e1c0595b0 con 0x7f2e300ad580 2024-03-28T13:10:31.236 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:31.236 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:10:23.142768+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":0,"to":4}]},{"pgid":"3.1b","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:10:31.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.239+0000 7f2e38600640 1 -- 172.21.15.19:0/857604302 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e1006c560 msgr2=0x7f2e1006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:31.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.239+0000 7f2e38600640 1 --2- 172.21.15.19:0/857604302 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e1006c560 0x7f2e1006ea20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2e24036000 tx=0x7f2e2403c040 comp rx=0 tx=0).stop 2024-03-28T13:10:31.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.239+0000 7f2e38600640 1 -- 172.21.15.19:0/857604302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e300ad580 msgr2=0x7f2e30154090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:31.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.239+0000 7f2e38600640 1 --2- 172.21.15.19:0/857604302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e300ad580 0x7f2e30154090 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00c7f0 tx=0x7f2e1c00cbf0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.240+0000 7f2e38600640 1 -- 172.21.15.19:0/857604302 shutdown_connections 2024-03-28T13:10:31.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.240+0000 7f2e38600640 1 --2- 172.21.15.19:0/857604302 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e1006c560 0x7f2e1006ea20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.240+0000 7f2e38600640 1 --2- 172.21.15.19:0/857604302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e30161b00 0x7f2e301486e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.240+0000 7f2e38600640 1 --2- 172.21.15.19:0/857604302 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e300ad580 0x7f2e30154090 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.240+0000 7f2e38600640 1 --2- 172.21.15.19:0/857604302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e300ad120 0x7f2e301565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.240+0000 7f2e38600640 1 -- 172.21.15.19:0/857604302 >> 172.21.15.19:0/857604302 conn(0x7f2e3006b990 msgr2=0x7f2e300af1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:31.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.240+0000 7f2e38600640 1 -- 172.21.15.19:0/857604302 shutdown_connections 2024-03-28T13:10:31.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.240+0000 7f2e38600640 1 -- 172.21.15.19:0/857604302 wait complete. 2024-03-28T13:10:31.264 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:09:39.243813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:09:40.092899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:10:31.264 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:10:31.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.558+0000 7f9761e00640 1 Processor -- start 2024-03-28T13:10:31.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.558+0000 7f9761e00640 1 -- start start 2024-03-28T13:10:31.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.560+0000 7f9761e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f975c0b4650 0x7f975c06c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:31.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.560+0000 7f9761e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f975c0b4a30 0x7f975c0b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:31.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.560+0000 7f9761e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f975c15ede0 con 0x7f975c158580 2024-03-28T13:10:31.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.560+0000 7f9761e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f975c141b30 con 0x7f975c0b4650 2024-03-28T13:10:31.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.560+0000 7f9761e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f975c141d80 con 0x7f975c0b4a30 2024-03-28T13:10:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.561+0000 7f975aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f975c0b4650 0x7f975c06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.561+0000 7f975aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f975c0b4650 0x7f975c06c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40284/0 (socket says 172.21.15.19:40284) 2024-03-28T13:10:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.561+0000 7f975aa00640 1 -- 172.21.15.19:0/3256825332 learned_addr learned my addr 172.21.15.19:0/3256825332 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.561+0000 7f9753e00640 1 --2- 172.21.15.19:0/3256825332 >> v2:172.21.15.188:3300/0 conn(0x7f975c0b4a30 0x7f975c0b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.562+0000 7f975aa00640 1 -- 172.21.15.19:0/3256825332 >> v2:172.21.15.188:3300/0 conn(0x7f975c0b4a30 msgr2=0x7f975c0b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.562+0000 7f975aa00640 1 --2- 172.21.15.19:0/3256825332 >> v2:172.21.15.188:3300/0 conn(0x7f975c0b4a30 0x7f975c0b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.562+0000 7f975aa00640 1 -- 172.21.15.19:0/3256825332 >> v1:172.21.15.19:6789/0 conn(0x7f975c158580 legacy=0x7f975c1589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.562+0000 7f975aa00640 1 -- 172.21.15.19:0/3256825332 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c142060 con 0x7f975c0b4650 2024-03-28T13:10:31.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.562+0000 7f9753e00640 1 --2- 172.21.15.19:0/3256825332 >> v2:172.21.15.188:3300/0 conn(0x7f975c0b4a30 0x7f975c0b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:10:31.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.563+0000 7f975aa00640 1 --2- 172.21.15.19:0/3256825332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975c0b4650 0x7f975c06c680 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f975c15c6b0 tx=0x7f974c02b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbcd83adf3cce820 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:31.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.563+0000 7f975a000640 1 -- 172.21.15.19:0/3256825332 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c034070 con 0x7f975c0b4650 2024-03-28T13:10:31.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.563+0000 7f975a000640 1 -- 172.21.15.19:0/3256825332 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f974c030880 con 0x7f975c0b4650 2024-03-28T13:10:31.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.563+0000 7f975a000640 1 -- 172.21.15.19:0/3256825332 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c030c50 con 0x7f975c0b4650 2024-03-28T13:10:31.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.564+0000 7f9761e00640 1 -- 172.21.15.19:0/3256825332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975c0b4650 msgr2=0x7f975c06c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:31.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.564+0000 7f9761e00640 1 --2- 172.21.15.19:0/3256825332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975c0b4650 0x7f975c06c680 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f975c15c6b0 tx=0x7f974c02b9c0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.564+0000 7f9761e00640 1 -- 172.21.15.19:0/3256825332 shutdown_connections 2024-03-28T13:10:31.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.564+0000 7f9761e00640 1 --2- 172.21.15.19:0/3256825332 >> v2:172.21.15.188:3300/0 conn(0x7f975c0b4a30 0x7f975c0b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.564+0000 7f9761e00640 1 --2- 172.21.15.19:0/3256825332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975c0b4650 0x7f975c06c680 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.564+0000 7f9761e00640 1 -- 172.21.15.19:0/3256825332 >> 172.21.15.19:0/3256825332 conn(0x7f975c067670 msgr2=0x7f975c067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:31.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.565+0000 7f9761e00640 1 -- 172.21.15.19:0/3256825332 shutdown_connections 2024-03-28T13:10:31.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.565+0000 7f9761e00640 1 -- 172.21.15.19:0/3256825332 wait complete. 2024-03-28T13:10:31.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.567+0000 7f9761e00640 1 Processor -- start 2024-03-28T13:10:31.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.568+0000 7f9761e00640 1 -- start start 2024-03-28T13:10:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.569+0000 7f9761e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f975c0b4650 0x7f975c1f7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.569+0000 7f9761e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975c0b4a30 0x7f975c0cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.569+0000 7f9761e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f975c158580 0x7f975c15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.569+0000 7f9761e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f975c142900 con 0x7f975c0b4a30 2024-03-28T13:10:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.569+0000 7f9761e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f975c142620 con 0x7f975c158580 2024-03-28T13:10:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.569+0000 7f9761e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f975c06cbc0 con 0x7f975c0b4650 2024-03-28T13:10:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.570+0000 7f9753e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975c0b4a30 0x7f975c0cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.570+0000 7f9753e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975c0b4a30 0x7f975c0cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40296/0 (socket says 172.21.15.19:40296) 2024-03-28T13:10:31.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.570+0000 7f9753e00640 1 -- 172.21.15.19:0/1637999251 learned_addr learned my addr 172.21.15.19:0/1637999251 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:31.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.570+0000 7f975aa00640 1 --2- 172.21.15.19:0/1637999251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f975c0b4650 0x7f975c1f7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:31.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.570+0000 7f975b400640 1 --2- 172.21.15.19:0/1637999251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f975c158580 0x7f975c15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:31.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.570+0000 7f9753e00640 1 -- 172.21.15.19:0/1637999251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f975c0b4650 msgr2=0x7f975c1f7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:31.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.570+0000 7f9753e00640 1 --2- 172.21.15.19:0/1637999251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f975c0b4650 0x7f975c1f7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.570+0000 7f9753e00640 1 -- 172.21.15.19:0/1637999251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f975c158580 msgr2=0x7f975c15a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:31.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.570+0000 7f9753e00640 1 --2- 172.21.15.19:0/1637999251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f975c158580 0x7f975c15a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.570+0000 7f9753e00640 1 -- 172.21.15.19:0/1637999251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c0cdde0 con 0x7f975c0b4a30 2024-03-28T13:10:31.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.571+0000 7f9753e00640 1 --2- 172.21.15.19:0/1637999251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975c0b4a30 0x7f975c0cd6b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f974400e7e0 tx=0x7f974400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:31.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.571+0000 7f9753400640 1 -- 172.21.15.19:0/1637999251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744017070 con 0x7f975c0b4a30 2024-03-28T13:10:31.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.571+0000 7f9753400640 1 -- 172.21.15.19:0/1637999251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f974400c8d0 con 0x7f975c0b4a30 2024-03-28T13:10:31.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.571+0000 7f9753400640 1 -- 172.21.15.19:0/1637999251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974400f040 con 0x7f975c0b4a30 2024-03-28T13:10:31.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.571+0000 7f9761e00640 1 -- 172.21.15.19:0/1637999251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975c158a20 con 0x7f975c0b4a30 2024-03-28T13:10:31.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.571+0000 7f9761e00640 1 -- 172.21.15.19:0/1637999251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975c158fe0 con 0x7f975c0b4a30 2024-03-28T13:10:31.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.573+0000 7f9761e00640 1 -- 172.21.15.19:0/1637999251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f975c0685c0 con 0x7f975c0b4a30 2024-03-28T13:10:31.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.573+0000 7f9753400640 1 -- 172.21.15.19:0/1637999251 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f974401a070 con 0x7f975c0b4a30 2024-03-28T13:10:31.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.574+0000 7f9753400640 1 --2- 172.21.15.19:0/1637999251 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f973806c530 0x7f973806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:31.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.575+0000 7f9753400640 1 -- 172.21.15.19:0/1637999251 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f974408b280 con 0x7f975c0b4a30 2024-03-28T13:10:31.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.575+0000 7f975aa00640 1 --2- 172.21.15.19:0/1637999251 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f973806c530 0x7f973806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:31.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.577+0000 7f975aa00640 1 --2- 172.21.15.19:0/1637999251 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f973806c530 0x7f973806e9f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f974c036000 tx=0x7f974c03b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:31.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.583+0000 7f9753400640 1 -- 172.21.15.19:0/1637999251 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f97440598f0 con 0x7f975c0b4a30 2024-03-28T13:10:31.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.790+0000 7f9761e00640 1 -- 172.21.15.19:0/1637999251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f975c144750 con 0x7f975c0b4a30 2024-03-28T13:10:31.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.791+0000 7f9753400640 1 -- 172.21.15.19:0/1637999251 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f97440591f0 con 0x7f975c0b4a30 2024-03-28T13:10:31.791 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:10:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.793+0000 7f9761e00640 1 -- 172.21.15.19:0/1637999251 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f973806c530 msgr2=0x7f973806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.793+0000 7f9761e00640 1 --2- 172.21.15.19:0/1637999251 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f973806c530 0x7f973806e9f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f974c036000 tx=0x7f974c03b040 comp rx=0 tx=0).stop 2024-03-28T13:10:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.793+0000 7f9761e00640 1 -- 172.21.15.19:0/1637999251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975c0b4a30 msgr2=0x7f975c0cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.793+0000 7f9761e00640 1 --2- 172.21.15.19:0/1637999251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975c0b4a30 0x7f975c0cd6b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f974400e7e0 tx=0x7f974400eca0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.794+0000 7f9761e00640 1 -- 172.21.15.19:0/1637999251 shutdown_connections 2024-03-28T13:10:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.794+0000 7f9761e00640 1 --2- 172.21.15.19:0/1637999251 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f973806c530 0x7f973806e9f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.794+0000 7f9761e00640 1 --2- 172.21.15.19:0/1637999251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f975c158580 0x7f975c15a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.794+0000 7f9761e00640 1 --2- 172.21.15.19:0/1637999251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f975c0b4a30 0x7f975c0cd6b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.794+0000 7f9761e00640 1 --2- 172.21.15.19:0/1637999251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f975c0b4650 0x7f975c1f7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.794+0000 7f9761e00640 1 -- 172.21.15.19:0/1637999251 >> 172.21.15.19:0/1637999251 conn(0x7f975c067670 msgr2=0x7f975c1611a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.794+0000 7f9761e00640 1 -- 172.21.15.19:0/1637999251 shutdown_connections 2024-03-28T13:10:31.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:31.794+0000 7f9761e00640 1 -- 172.21.15.19:0/1637999251 wait complete. 2024-03-28T13:10:31.813 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:10:32.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.011+0000 7fd446000640 1 Processor -- start 2024-03-28T13:10:32.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.011+0000 7fd446000640 1 -- start start 2024-03-28T13:10:32.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.014+0000 7fd446000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd44006bc80 0x7fd4400702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.014+0000 7fd446000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd440070c10 con 0x7fd4400707e0 2024-03-28T13:10:32.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.014+0000 7fd446000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4400cfe80 con 0x7fd440147c80 2024-03-28T13:10:32.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.014+0000 7fd446000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd4400d0f20 con 0x7fd44006bc80 2024-03-28T13:10:32.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.014+0000 7fd445600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd4400707e0 0x7fd4400724f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54474/0 (socket says 172.21.15.19:54474) 2024-03-28T13:10:32.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.014+0000 7fd445600640 1 -- 172.21.15.19:0/3149295266 learned_addr learned my addr 172.21.15.19:0/3149295266 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:32.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.014+0000 7fd43fe00640 1 --2- 172.21.15.19:0/3149295266 >> v2:172.21.15.19:3300/0 conn(0x7fd44006bc80 0x7fd4400702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.015+0000 7fd43f400640 1 -- 172.21.15.19:0/3149295266 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3778236217 0 0) 0x7fd440070c10 con 0x7fd4400707e0 2024-03-28T13:10:32.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.015+0000 7fd43f400640 1 -- 172.21.15.19:0/3149295266 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd428003520 con 0x7fd4400707e0 2024-03-28T13:10:32.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.015+0000 7fd43fe00640 1 -- 172.21.15.19:0/3149295266 >> v1:172.21.15.188:6789/0 conn(0x7fd440147c80 legacy=0x7fd44006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.015+0000 7fd43fe00640 1 -- 172.21.15.19:0/3149295266 >> v1:172.21.15.19:6789/0 conn(0x7fd4400707e0 legacy=0x7fd4400724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.015+0000 7fd43fe00640 1 -- 172.21.15.19:0/3149295266 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4400d0160 con 0x7fd44006bc80 2024-03-28T13:10:32.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.016+0000 7fd43fe00640 1 --2- 172.21.15.19:0/3149295266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd44006bc80 0x7fd4400702a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0097a0 tx=0x7fd42c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cf979fb49a734a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:32.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.016+0000 7fd43f400640 1 -- 172.21.15.19:0/3149295266 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c034070 con 0x7fd44006bc80 2024-03-28T13:10:32.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.017+0000 7fd43f400640 1 -- 172.21.15.19:0/3149295266 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd42c030610 con 0x7fd44006bc80 2024-03-28T13:10:32.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.017+0000 7fd43f400640 1 -- 172.21.15.19:0/3149295266 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0309c0 con 0x7fd44006bc80 2024-03-28T13:10:32.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.017+0000 7fd446000640 1 -- 172.21.15.19:0/3149295266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd44006bc80 msgr2=0x7fd4400702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.017+0000 7fd446000640 1 --2- 172.21.15.19:0/3149295266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd44006bc80 0x7fd4400702a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0097a0 tx=0x7fd42c02b960 comp rx=0 tx=0).stop 2024-03-28T13:10:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.017+0000 7fd446000640 1 -- 172.21.15.19:0/3149295266 shutdown_connections 2024-03-28T13:10:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.017+0000 7fd446000640 1 --2- 172.21.15.19:0/3149295266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd44006bc80 0x7fd4400702a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.017+0000 7fd446000640 1 -- 172.21.15.19:0/3149295266 >> 172.21.15.19:0/3149295266 conn(0x7fd44006e730 msgr2=0x7fd44006eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.017+0000 7fd446000640 1 -- 172.21.15.19:0/3149295266 shutdown_connections 2024-03-28T13:10:32.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.017+0000 7fd446000640 1 -- 172.21.15.19:0/3149295266 wait complete. 2024-03-28T13:10:32.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.019+0000 7fd446000640 1 Processor -- start 2024-03-28T13:10:32.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.019+0000 7fd446000640 1 -- start start 2024-03-28T13:10:32.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.020+0000 7fd446000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd44006bc80 0x7fd44013e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.020+0000 7fd446000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400707e0 0x7fd4400b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.020+0000 7fd446000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd440147c80 0x7fd4400afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.020+0000 7fd446000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4400d0720 con 0x7fd44006bc80 2024-03-28T13:10:32.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.020+0000 7fd446000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd4400d0a90 con 0x7fd440147c80 2024-03-28T13:10:32.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.020+0000 7fd446000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd4400cfb50 con 0x7fd4400707e0 2024-03-28T13:10:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.020+0000 7fd444c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd44006bc80 0x7fd44013e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.021+0000 7fd444c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd44006bc80 0x7fd44013e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40318/0 (socket says 172.21.15.19:40318) 2024-03-28T13:10:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.021+0000 7fd444c00640 1 -- 172.21.15.19:0/3954729280 learned_addr learned my addr 172.21.15.19:0/3954729280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.021+0000 7fd445600640 1 --2- 172.21.15.19:0/3954729280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd440147c80 0x7fd4400afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.021+0000 7fd43fe00640 1 --2- 172.21.15.19:0/3954729280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400707e0 0x7fd4400b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.021+0000 7fd444c00640 1 -- 172.21.15.19:0/3954729280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400707e0 msgr2=0x7fd4400b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.021+0000 7fd444c00640 1 --2- 172.21.15.19:0/3954729280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400707e0 0x7fd4400b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.021+0000 7fd444c00640 1 -- 172.21.15.19:0/3954729280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd440147c80 msgr2=0x7fd4400afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.021+0000 7fd444c00640 1 --2- 172.21.15.19:0/3954729280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd440147c80 0x7fd4400afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.021+0000 7fd444c00640 1 -- 172.21.15.19:0/3954729280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd440142790 con 0x7fd44006bc80 2024-03-28T13:10:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.021+0000 7fd445600640 1 --2- 172.21.15.19:0/3954729280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd440147c80 0x7fd4400afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:10:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.021+0000 7fd444c00640 1 --2- 172.21.15.19:0/3954729280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd44006bc80 0x7fd44013e780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd43400c400 tx=0x7fd43400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:32.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.022+0000 7fd43cc00640 1 -- 172.21.15.19:0/3954729280 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434017070 con 0x7fd44006bc80 2024-03-28T13:10:32.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.022+0000 7fd43cc00640 1 -- 172.21.15.19:0/3954729280 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4340052d0 con 0x7fd44006bc80 2024-03-28T13:10:32.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.022+0000 7fd43cc00640 1 -- 172.21.15.19:0/3954729280 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd434003830 con 0x7fd44006bc80 2024-03-28T13:10:32.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.022+0000 7fd446000640 1 -- 172.21.15.19:0/3954729280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44013f0c0 con 0x7fd44006bc80 2024-03-28T13:10:32.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.022+0000 7fd446000640 1 -- 172.21.15.19:0/3954729280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4401424b0 con 0x7fd44006bc80 2024-03-28T13:10:32.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.024+0000 7fd446000640 1 -- 172.21.15.19:0/3954729280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd440075940 con 0x7fd44006bc80 2024-03-28T13:10:32.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.024+0000 7fd43cc00640 1 -- 172.21.15.19:0/3954729280 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd43401a070 con 0x7fd44006bc80 2024-03-28T13:10:32.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.025+0000 7fd43cc00640 1 --2- 172.21.15.19:0/3954729280 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd42006c530 0x7fd42006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.025+0000 7fd43cc00640 1 -- 172.21.15.19:0/3954729280 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7fd43408a9a0 con 0x7fd44006bc80 2024-03-28T13:10:32.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.026+0000 7fd43fe00640 1 --2- 172.21.15.19:0/3954729280 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd42006c530 0x7fd42006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.027+0000 7fd43fe00640 1 --2- 172.21.15.19:0/3954729280 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd42006c530 0x7fd42006e9f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd42c036000 tx=0x7fd42c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:32.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.035+0000 7fd43cc00640 1 -- 172.21.15.19:0/3954729280 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd434059010 con 0x7fd44006bc80 2024-03-28T13:10:32.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.242+0000 7fd446000640 1 -- 172.21.15.19:0/3954729280 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd4400d14e0 con 0x7fd44006bc80 2024-03-28T13:10:32.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.243+0000 7fd43cc00640 1 -- 172.21.15.19:0/3954729280 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fd434058910 con 0x7fd44006bc80 2024-03-28T13:10:32.243 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:10:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.246+0000 7fd446000640 1 -- 172.21.15.19:0/3954729280 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd42006c530 msgr2=0x7fd42006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.246+0000 7fd446000640 1 --2- 172.21.15.19:0/3954729280 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd42006c530 0x7fd42006e9f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd42c036000 tx=0x7fd42c008030 comp rx=0 tx=0).stop 2024-03-28T13:10:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.246+0000 7fd446000640 1 -- 172.21.15.19:0/3954729280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd44006bc80 msgr2=0x7fd44013e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.246+0000 7fd446000640 1 --2- 172.21.15.19:0/3954729280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd44006bc80 0x7fd44013e780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd43400c400 tx=0x7fd43400c8c0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.246+0000 7fd446000640 1 -- 172.21.15.19:0/3954729280 shutdown_connections 2024-03-28T13:10:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.246+0000 7fd446000640 1 --2- 172.21.15.19:0/3954729280 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd42006c530 0x7fd42006e9f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.246+0000 7fd446000640 1 --2- 172.21.15.19:0/3954729280 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd440147c80 0x7fd4400afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.246+0000 7fd446000640 1 --2- 172.21.15.19:0/3954729280 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4400707e0 0x7fd4400b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.246+0000 7fd446000640 1 --2- 172.21.15.19:0/3954729280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd44006bc80 0x7fd44013e780 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.246+0000 7fd446000640 1 -- 172.21.15.19:0/3954729280 >> 172.21.15.19:0/3954729280 conn(0x7fd44006e730 msgr2=0x7fd44015fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:32.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.247+0000 7fd446000640 1 -- 172.21.15.19:0/3954729280 shutdown_connections 2024-03-28T13:10:32.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.247+0000 7fd446000640 1 -- 172.21.15.19:0/3954729280 wait complete. 2024-03-28T13:10:32.265 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:10:32.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.475+0000 7f7fd5200640 1 Processor -- start 2024-03-28T13:10:32.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.476+0000 7f7fd5200640 1 -- start start 2024-03-28T13:10:32.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.477+0000 7f7fd5200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7fd015e860 0x7f7fd006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.477+0000 7f7fd5200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fd006da30 0x7f7fd00d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.477+0000 7f7fd5200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7fd00d4910 0x7f7fd00d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.477+0000 7f7fd5200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7fd00d4d60 con 0x7f7fd00d4910 2024-03-28T13:10:32.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.477+0000 7f7fd5200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7fd00cf220 con 0x7f7fd015e860 2024-03-28T13:10:32.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.477+0000 7f7fd5200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd006dfa0 con 0x7f7fd006da30 2024-03-28T13:10:32.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.478+0000 7f7fce800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7fd00d4910 0x7f7fd00d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.478+0000 7f7fce800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7fd00d4910 0x7f7fd00d0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40324/0 (socket says 172.21.15.19:40324) 2024-03-28T13:10:32.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.478+0000 7f7fce800640 1 -- 172.21.15.19:0/1298235151 learned_addr learned my addr 172.21.15.19:0/1298235151 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:32.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.478+0000 7f7fcde00640 1 --2- 172.21.15.19:0/1298235151 >> v2:172.21.15.188:3300/0 conn(0x7f7fd015e860 0x7f7fd006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.478+0000 7f7fcd400640 1 --2- 172.21.15.19:0/1298235151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fd006da30 0x7f7fd00d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.478+0000 7f7fce800640 1 -- 172.21.15.19:0/1298235151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fd006da30 msgr2=0x7f7fd00d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:32.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.478+0000 7f7fce800640 1 --2- 172.21.15.19:0/1298235151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fd006da30 0x7f7fd00d43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.478+0000 7f7fce800640 1 -- 172.21.15.19:0/1298235151 >> v2:172.21.15.188:3300/0 conn(0x7f7fd015e860 msgr2=0x7f7fd006d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:32.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.478+0000 7f7fce800640 1 --2- 172.21.15.19:0/1298235151 >> v2:172.21.15.188:3300/0 conn(0x7f7fd015e860 0x7f7fd006d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.478+0000 7f7fce800640 1 -- 172.21.15.19:0/1298235151 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd006e240 con 0x7f7fd00d4910 2024-03-28T13:10:32.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.478+0000 7f7fcde00640 1 --2- 172.21.15.19:0/1298235151 >> v2:172.21.15.188:3300/0 conn(0x7f7fd015e860 0x7f7fd006d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:10:32.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.480+0000 7f7fce800640 1 --2- 172.21.15.19:0/1298235151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd00d4910 0x7f7fd00d0e30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00d1fa0 tx=0x7f7fc802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937b276826c5fee2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:32.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.480+0000 7f7fcca00640 1 -- 172.21.15.19:0/1298235151 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc802ca70 con 0x7f7fd00d4910 2024-03-28T13:10:32.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.480+0000 7f7fcca00640 1 -- 172.21.15.19:0/1298235151 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc802cc10 con 0x7f7fd00d4910 2024-03-28T13:10:32.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.480+0000 7f7fcca00640 1 -- 172.21.15.19:0/1298235151 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80343f0 con 0x7f7fd00d4910 2024-03-28T13:10:32.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.481+0000 7f7fd5200640 1 -- 172.21.15.19:0/1298235151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd00d4910 msgr2=0x7f7fd00d0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.481+0000 7f7fd5200640 1 --2- 172.21.15.19:0/1298235151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd00d4910 0x7f7fd00d0e30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00d1fa0 tx=0x7f7fc802bed0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.481+0000 7f7fd5200640 1 -- 172.21.15.19:0/1298235151 shutdown_connections 2024-03-28T13:10:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.481+0000 7f7fd5200640 1 --2- 172.21.15.19:0/1298235151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd00d4910 0x7f7fd00d0e30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.481+0000 7f7fd5200640 1 --2- 172.21.15.19:0/1298235151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fd006da30 0x7f7fd00d43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.481+0000 7f7fd5200640 1 --2- 172.21.15.19:0/1298235151 >> v2:172.21.15.188:3300/0 conn(0x7f7fd015e860 0x7f7fd006d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.481+0000 7f7fd5200640 1 -- 172.21.15.19:0/1298235151 >> 172.21.15.19:0/1298235151 conn(0x7f7fd009d7d0 msgr2=0x7f7fd008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.481+0000 7f7fd5200640 1 -- 172.21.15.19:0/1298235151 shutdown_connections 2024-03-28T13:10:32.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.481+0000 7f7fd5200640 1 -- 172.21.15.19:0/1298235151 wait complete. 2024-03-28T13:10:32.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.483+0000 7f7fd5200640 1 Processor -- start 2024-03-28T13:10:32.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.483+0000 7f7fd5200640 1 -- start start 2024-03-28T13:10:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.484+0000 7f7fd5200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fd006da30 0x7f7fd0153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.484+0000 7f7fd5200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fd00d4910 0x7f7fd015fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.484+0000 7f7fd5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd015e860 0x7f7fd00c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.484+0000 7f7fd5200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd00cedb0 con 0x7f7fd015e860 2024-03-28T13:10:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.484+0000 7f7fd5200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd00d2c50 con 0x7f7fd00d4910 2024-03-28T13:10:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.484+0000 7f7fd5200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd00cef30 con 0x7f7fd006da30 2024-03-28T13:10:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.485+0000 7f7fce800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd015e860 0x7f7fd00c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.485+0000 7f7fce800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd015e860 0x7f7fd00c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40338/0 (socket says 172.21.15.19:40338) 2024-03-28T13:10:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.485+0000 7f7fce800640 1 -- 172.21.15.19:0/48620492 learned_addr learned my addr 172.21.15.19:0/48620492 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.485+0000 7f7fcd400640 1 --2- 172.21.15.19:0/48620492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fd00d4910 0x7f7fd015fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.485+0000 7f7fcde00640 1 --2- 172.21.15.19:0/48620492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fd006da30 0x7f7fd0153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.485+0000 7f7fce800640 1 -- 172.21.15.19:0/48620492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fd006da30 msgr2=0x7f7fd0153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.485+0000 7f7fce800640 1 --2- 172.21.15.19:0/48620492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fd006da30 0x7f7fd0153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.485+0000 7f7fce800640 1 -- 172.21.15.19:0/48620492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fd00d4910 msgr2=0x7f7fd015fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.485+0000 7f7fce800640 1 --2- 172.21.15.19:0/48620492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fd00d4910 0x7f7fd015fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.485+0000 7f7fce800640 1 -- 172.21.15.19:0/48620492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd00ce170 con 0x7f7fd015e860 2024-03-28T13:10:32.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.485+0000 7f7fce800640 1 --2- 172.21.15.19:0/48620492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd015e860 0x7f7fd00c8f80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8036040 tx=0x7f7fc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.486+0000 7f7fc6000640 1 -- 172.21.15.19:0/48620492 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8043020 con 0x7f7fd015e860 2024-03-28T13:10:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.486+0000 7f7fc6000640 1 -- 172.21.15.19:0/48620492 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc803e450 con 0x7f7fd015e860 2024-03-28T13:10:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.486+0000 7f7fc6000640 1 -- 172.21.15.19:0/48620492 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8009510 con 0x7f7fd015e860 2024-03-28T13:10:32.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.486+0000 7f7fd5200640 1 -- 172.21.15.19:0/48620492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd00caa00 con 0x7f7fd015e860 2024-03-28T13:10:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.486+0000 7f7fd5200640 1 -- 172.21.15.19:0/48620492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd00c8a60 con 0x7f7fd015e860 2024-03-28T13:10:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.487+0000 7f7fd5200640 1 -- 172.21.15.19:0/48620492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd00c8c60 con 0x7f7fd015e860 2024-03-28T13:10:32.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.488+0000 7f7fc6000640 1 -- 172.21.15.19:0/48620492 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7fc8046020 con 0x7f7fd015e860 2024-03-28T13:10:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.489+0000 7f7fc6000640 1 --2- 172.21.15.19:0/48620492 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fb006c530 0x7f7fb006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.489+0000 7f7fc6000640 1 -- 172.21.15.19:0/48620492 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f7fc80adfe0 con 0x7f7fd015e860 2024-03-28T13:10:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.490+0000 7f7fcde00640 1 --2- 172.21.15.19:0/48620492 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fb006c530 0x7f7fb006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.491+0000 7f7fcde00640 1 --2- 172.21.15.19:0/48620492 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fb006c530 0x7f7fb006e9f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0006fd0 tx=0x7f7fc0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.498+0000 7f7fc6000640 1 -- 172.21.15.19:0/48620492 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7fd00c8c60 con 0x7f7fd015e860 2024-03-28T13:10:32.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.730+0000 7f7fd5200640 1 -- 172.21.15.19:0/48620492 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7fd00685c0 con 0x7f7fd015e860 2024-03-28T13:10:32.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.731+0000 7f7fc6000640 1 -- 172.21.15.19:0/48620492 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f7fc804d030 con 0x7f7fd015e860 2024-03-28T13:10:32.731 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:10:32.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.733+0000 7f7fd5200640 1 -- 172.21.15.19:0/48620492 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fb006c530 msgr2=0x7f7fb006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.733+0000 7f7fd5200640 1 --2- 172.21.15.19:0/48620492 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fb006c530 0x7f7fb006e9f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0006fd0 tx=0x7f7fc0008040 comp rx=0 tx=0).stop 2024-03-28T13:10:32.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.733+0000 7f7fd5200640 1 -- 172.21.15.19:0/48620492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd015e860 msgr2=0x7f7fd00c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.733+0000 7f7fd5200640 1 --2- 172.21.15.19:0/48620492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd015e860 0x7f7fd00c8f80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8036040 tx=0x7f7fc8004a40 comp rx=0 tx=0).stop 2024-03-28T13:10:32.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.733+0000 7f7fd5200640 1 -- 172.21.15.19:0/48620492 shutdown_connections 2024-03-28T13:10:32.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.733+0000 7f7fd5200640 1 --2- 172.21.15.19:0/48620492 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fb006c530 0x7f7fb006e9f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.733+0000 7f7fd5200640 1 --2- 172.21.15.19:0/48620492 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd015e860 0x7f7fd00c8f80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.733+0000 7f7fd5200640 1 --2- 172.21.15.19:0/48620492 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fd00d4910 0x7f7fd015fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.733+0000 7f7fd5200640 1 --2- 172.21.15.19:0/48620492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fd006da30 0x7f7fd0153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.733+0000 7f7fd5200640 1 -- 172.21.15.19:0/48620492 >> 172.21.15.19:0/48620492 conn(0x7f7fd009d7d0 msgr2=0x7f7fd00cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:32.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.733+0000 7f7fd5200640 1 -- 172.21.15.19:0/48620492 shutdown_connections 2024-03-28T13:10:32.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.733+0000 7f7fd5200640 1 -- 172.21.15.19:0/48620492 wait complete. 2024-03-28T13:10:32.751 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:10:32.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.951+0000 7fc3f9200640 1 Processor -- start 2024-03-28T13:10:32.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.951+0000 7fc3f9200640 1 -- start start 2024-03-28T13:10:32.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.952+0000 7fc3f9200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc3f4155550 0x7fc3f4155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.952+0000 7fc3f9200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3f40b40b0 con 0x7fc3f415f800 2024-03-28T13:10:32.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.952+0000 7fc3f9200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3f40b4390 con 0x7fc3f40d0990 2024-03-28T13:10:32.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.952+0000 7fc3f9200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc3f40b4670 con 0x7fc3f4155550 2024-03-28T13:10:32.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.952+0000 7fc3f1400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc3f415f800 0x7fc3f40d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54478/0 (socket says 172.21.15.19:54478) 2024-03-28T13:10:32.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.952+0000 7fc3f1400640 1 -- 172.21.15.19:0/1036854418 learned_addr learned my addr 172.21.15.19:0/1036854418 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:32.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.952+0000 7fc3f1e00640 1 --2- 172.21.15.19:0/1036854418 >> v2:172.21.15.19:3300/0 conn(0x7fc3f4155550 0x7fc3f4155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.953+0000 7fc3f1e00640 1 -- 172.21.15.19:0/1036854418 >> v1:172.21.15.188:6789/0 conn(0x7fc3f40d0990 legacy=0x7fc3f40aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:32.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.953+0000 7fc3f1e00640 1 -- 172.21.15.19:0/1036854418 >> v1:172.21.15.19:6789/0 conn(0x7fc3f415f800 legacy=0x7fc3f40d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.953+0000 7fc3f1e00640 1 -- 172.21.15.19:0/1036854418 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f40b4950 con 0x7fc3f4155550 2024-03-28T13:10:32.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.954+0000 7fc3f1e00640 1 --2- 172.21.15.19:0/1036854418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f4155550 0x7fc3f4155930 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40097a0 tx=0x7fc3e402da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded9fea0f853729e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:32.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.954+0000 7fc3f0a00640 1 -- 172.21.15.19:0/1036854418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4031870 con 0x7fc3f4155550 2024-03-28T13:10:32.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.954+0000 7fc3f0a00640 1 -- 172.21.15.19:0/1036854418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3e4031a10 con 0x7fc3f4155550 2024-03-28T13:10:32.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.954+0000 7fc3f0a00640 1 -- 172.21.15.19:0/1036854418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4031de0 con 0x7fc3f4155550 2024-03-28T13:10:32.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.955+0000 7fc3f9200640 1 -- 172.21.15.19:0/1036854418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f4155550 msgr2=0x7fc3f4155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.955+0000 7fc3f9200640 1 --2- 172.21.15.19:0/1036854418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f4155550 0x7fc3f4155930 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40097a0 tx=0x7fc3e402da50 comp rx=0 tx=0).stop 2024-03-28T13:10:32.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.955+0000 7fc3f9200640 1 -- 172.21.15.19:0/1036854418 shutdown_connections 2024-03-28T13:10:32.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.955+0000 7fc3f9200640 1 --2- 172.21.15.19:0/1036854418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f4155550 0x7fc3f4155930 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.955+0000 7fc3f9200640 1 -- 172.21.15.19:0/1036854418 >> 172.21.15.19:0/1036854418 conn(0x7fc3f406b750 msgr2=0x7fc3f406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:32.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.955+0000 7fc3f9200640 1 -- 172.21.15.19:0/1036854418 shutdown_connections 2024-03-28T13:10:32.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.955+0000 7fc3f9200640 1 -- 172.21.15.19:0/1036854418 wait complete. 2024-03-28T13:10:32.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.958+0000 7fc3f9200640 1 Processor -- start 2024-03-28T13:10:32.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.958+0000 7fc3f9200640 1 -- start start 2024-03-28T13:10:32.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f9200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3f40d0990 0x7fc3f41f0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f9200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f4155550 0x7fc3f41486e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f9200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f415f800 0x7fc3f41544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f9200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f415ba20 con 0x7fc3f4155550 2024-03-28T13:10:32.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f9200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f415fc80 con 0x7fc3f415f800 2024-03-28T13:10:32.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f9200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f415bba0 con 0x7fc3f40d0990 2024-03-28T13:10:32.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f4155550 0x7fc3f41486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f4155550 0x7fc3f41486e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40354/0 (socket says 172.21.15.19:40354) 2024-03-28T13:10:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f1400640 1 -- 172.21.15.19:0/2155556364 learned_addr learned my addr 172.21.15.19:0/2155556364 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f1e00640 1 --2- 172.21.15.19:0/2155556364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3f40d0990 0x7fc3f41f0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f2800640 1 --2- 172.21.15.19:0/2155556364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f415f800 0x7fc3f41544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f1400640 1 -- 172.21.15.19:0/2155556364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3f40d0990 msgr2=0x7fc3f41f0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f1400640 1 --2- 172.21.15.19:0/2155556364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3f40d0990 0x7fc3f41f0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f1400640 1 -- 172.21.15.19:0/2155556364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f415f800 msgr2=0x7fc3f41544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f1400640 1 --2- 172.21.15.19:0/2155556364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f415f800 0x7fc3f41544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f1400640 1 -- 172.21.15.19:0/2155556364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f409a190 con 0x7fc3f4155550 2024-03-28T13:10:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.960+0000 7fc3f2800640 1 --2- 172.21.15.19:0/2155556364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f415f800 0x7fc3f41544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.961+0000 7fc3f1400640 1 --2- 172.21.15.19:0/2155556364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f4155550 0x7fc3f41486e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000c430 tx=0x7fc3e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.961+0000 7fc3ea000640 1 -- 172.21.15.19:0/2155556364 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0017070 con 0x7fc3f4155550 2024-03-28T13:10:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.961+0000 7fc3ea000640 1 -- 172.21.15.19:0/2155556364 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3e0005ce0 con 0x7fc3f4155550 2024-03-28T13:10:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.961+0000 7fc3ea000640 1 -- 172.21.15.19:0/2155556364 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0007310 con 0x7fc3f4155550 2024-03-28T13:10:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.961+0000 7fc3f9200640 1 -- 172.21.15.19:0/2155556364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f409a470 con 0x7fc3f4155550 2024-03-28T13:10:32.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.961+0000 7fc3f9200640 1 -- 172.21.15.19:0/2155556364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f409aa30 con 0x7fc3f4155550 2024-03-28T13:10:32.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.962+0000 7fc3f9200640 1 -- 172.21.15.19:0/2155556364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f415f590 con 0x7fc3f4155550 2024-03-28T13:10:32.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.964+0000 7fc3ea000640 1 -- 172.21.15.19:0/2155556364 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc3e001a070 con 0x7fc3f4155550 2024-03-28T13:10:32.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.965+0000 7fc3ea000640 1 --2- 172.21.15.19:0/2155556364 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc3d406c530 0x7fc3d406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:32.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.965+0000 7fc3ea000640 1 -- 172.21.15.19:0/2155556364 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7fc3e008aba0 con 0x7fc3f4155550 2024-03-28T13:10:32.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.966+0000 7fc3f1e00640 1 --2- 172.21.15.19:0/2155556364 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc3d406c530 0x7fc3d406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:32.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.966+0000 7fc3f1e00640 1 --2- 172.21.15.19:0/2155556364 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc3d406c530 0x7fc3d406e9f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4038000 tx=0x7fc3e4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:32.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:32.970+0000 7fc3ea000640 1 -- 172.21.15.19:0/2155556364 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc3f415f590 con 0x7fc3f4155550 2024-03-28T13:10:33.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.185+0000 7fc3f9200640 1 -- 172.21.15.19:0/2155556364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc3f40685c0 con 0x7fc3f4155550 2024-03-28T13:10:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.186+0000 7fc3ea000640 1 -- 172.21.15.19:0/2155556364 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fc3e00590e0 con 0x7fc3f4155550 2024-03-28T13:10:33.186 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-03-28T13:10:33.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.189+0000 7fc3f9200640 1 -- 172.21.15.19:0/2155556364 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc3d406c530 msgr2=0x7fc3d406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.189+0000 7fc3f9200640 1 --2- 172.21.15.19:0/2155556364 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc3d406c530 0x7fc3d406e9f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4038000 tx=0x7fc3e4008030 comp rx=0 tx=0).stop 2024-03-28T13:10:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.189+0000 7fc3f9200640 1 -- 172.21.15.19:0/2155556364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f4155550 msgr2=0x7fc3f41486e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.189+0000 7fc3f9200640 1 --2- 172.21.15.19:0/2155556364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f4155550 0x7fc3f41486e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000c430 tx=0x7fc3e000c8f0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.189+0000 7fc3f9200640 1 -- 172.21.15.19:0/2155556364 shutdown_connections 2024-03-28T13:10:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.189+0000 7fc3f9200640 1 --2- 172.21.15.19:0/2155556364 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc3d406c530 0x7fc3d406e9f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.189+0000 7fc3f9200640 1 --2- 172.21.15.19:0/2155556364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3f415f800 0x7fc3f41544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.189+0000 7fc3f9200640 1 --2- 172.21.15.19:0/2155556364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3f4155550 0x7fc3f41486e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.189+0000 7fc3f9200640 1 --2- 172.21.15.19:0/2155556364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3f40d0990 0x7fc3f41f0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.189+0000 7fc3f9200640 1 -- 172.21.15.19:0/2155556364 >> 172.21.15.19:0/2155556364 conn(0x7fc3f406b750 msgr2=0x7fc3f406e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.189+0000 7fc3f9200640 1 -- 172.21.15.19:0/2155556364 shutdown_connections 2024-03-28T13:10:33.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.189+0000 7fc3f9200640 1 -- 172.21.15.19:0/2155556364 wait complete. 2024-03-28T13:10:33.207 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-03-28T13:10:33.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:33.208 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-03-28T13:10:33.296 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-03-28T13:10:33.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:33.296 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-28T13:10:33.345 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:10:33.346 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:33.401 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:10:33.401 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:33.452 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:10:33.452 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:33.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:10:33.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.727+0000 7f1b08a00640 1 Processor -- start 2024-03-28T13:10:33.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.728+0000 7f1b08a00640 1 -- start start 2024-03-28T13:10:33.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.729+0000 7f1b08a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b04161b00 0x7f1b04162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:33.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.729+0000 7f1b08a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1b040ad120 0x7f1b040d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:33.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.729+0000 7f1b08a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b04071b60 con 0x7f1b040ad580 2024-03-28T13:10:33.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.729+0000 7f1b08a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1b04161ee0 con 0x7f1b040ad120 2024-03-28T13:10:33.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.729+0000 7f1b08a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1b040b3450 con 0x7f1b04161b00 2024-03-28T13:10:33.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.730+0000 7f1b01e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b04161b00 0x7f1b04162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:33.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.730+0000 7f1b02800640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f1b040ad580 0x7f1b040b3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:56342/0 (socket says 172.21.15.19:56342) 2024-03-28T13:10:33.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.730+0000 7f1b02800640 1 -- 172.21.15.19:0/4184497971 learned_addr learned my addr 172.21.15.19:0/4184497971 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:33.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.730+0000 7f1b01e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b04161b00 0x7f1b04162ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:51494/0 (socket says 172.21.15.19:51494) 2024-03-28T13:10:33.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.730+0000 7f1af9400640 1 --2- 172.21.15.19:0/4184497971 >> v2:172.21.15.188:3300/0 conn(0x7f1b040ad120 0x7f1b040d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:33.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.731+0000 7f1b01e00640 1 -- 172.21.15.19:0/4184497971 >> v2:172.21.15.188:3300/0 conn(0x7f1b040ad120 msgr2=0x7f1b040d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:33.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.731+0000 7f1b01e00640 1 --2- 172.21.15.19:0/4184497971 >> v2:172.21.15.188:3300/0 conn(0x7f1b040ad120 0x7f1b040d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.731+0000 7f1b01e00640 1 -- 172.21.15.19:0/4184497971 >> v1:172.21.15.188:6789/0 conn(0x7f1b040ad580 legacy=0x7f1b040b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:33.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.731+0000 7f1b01e00640 1 -- 172.21.15.19:0/4184497971 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b04071e40 con 0x7f1b04161b00 2024-03-28T13:10:33.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.732+0000 7f1b01e00640 1 --2- 172.21.15.19:0/4184497971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b04161b00 0x7f1b04162ab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1af40097a0 tx=0x7f1af402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f3c06bd7ecdaf70 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:33.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.732+0000 7f1b01400640 1 -- 172.21.15.19:0/4184497971 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af4035020 con 0x7f1b04161b00 2024-03-28T13:10:33.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.732+0000 7f1b01400640 1 -- 172.21.15.19:0/4184497971 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af4031830 con 0x7f1b04161b00 2024-03-28T13:10:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.732+0000 7f1b01400640 1 -- 172.21.15.19:0/4184497971 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af4031c00 con 0x7f1b04161b00 2024-03-28T13:10:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.733+0000 7f1b08a00640 1 -- 172.21.15.19:0/4184497971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b04161b00 msgr2=0x7f1b04162ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.733+0000 7f1b08a00640 1 --2- 172.21.15.19:0/4184497971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b04161b00 0x7f1b04162ab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1af40097a0 tx=0x7f1af402b8d0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.733+0000 7f1b08a00640 1 -- 172.21.15.19:0/4184497971 shutdown_connections 2024-03-28T13:10:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.733+0000 7f1b08a00640 1 --2- 172.21.15.19:0/4184497971 >> v2:172.21.15.188:3300/0 conn(0x7f1b040ad120 0x7f1b040d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.733+0000 7f1b08a00640 1 --2- 172.21.15.19:0/4184497971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b04161b00 0x7f1b04162ab0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.733+0000 7f1b08a00640 1 -- 172.21.15.19:0/4184497971 >> 172.21.15.19:0/4184497971 conn(0x7f1b0406b990 msgr2=0x7f1b0406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:33.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.733+0000 7f1b08a00640 1 -- 172.21.15.19:0/4184497971 shutdown_connections 2024-03-28T13:10:33.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.733+0000 7f1b08a00640 1 -- 172.21.15.19:0/4184497971 wait complete. 2024-03-28T13:10:33.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.735+0000 7f1b08a00640 1 Processor -- start 2024-03-28T13:10:33.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.735+0000 7f1b08a00640 1 -- start start 2024-03-28T13:10:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.736+0000 7f1b08a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b040ad120 0x7f1b040aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.736+0000 7f1b08a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b040ad580 0x7f1b041568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.736+0000 7f1b08a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b04161b00 0x7f1b04154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.736+0000 7f1b08a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b04162ff0 con 0x7f1b040ad120 2024-03-28T13:10:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.736+0000 7f1b08a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1b040b3730 con 0x7f1b040ad580 2024-03-28T13:10:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.736+0000 7f1b08a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1b040717f0 con 0x7f1b04161b00 2024-03-28T13:10:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.736+0000 7f1b01e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b040ad120 0x7f1b040aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.737+0000 7f1b01e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b040ad120 0x7f1b040aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40362/0 (socket says 172.21.15.19:40362) 2024-03-28T13:10:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.737+0000 7f1b01e00640 1 -- 172.21.15.19:0/3391942955 learned_addr learned my addr 172.21.15.19:0/3391942955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.737+0000 7f1af9400640 1 --2- 172.21.15.19:0/3391942955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b040ad580 0x7f1b041568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.737+0000 7f1b02800640 1 --2- 172.21.15.19:0/3391942955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b04161b00 0x7f1b04154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.737+0000 7f1b01e00640 1 -- 172.21.15.19:0/3391942955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b04161b00 msgr2=0x7f1b04154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.737+0000 7f1b01e00640 1 --2- 172.21.15.19:0/3391942955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b04161b00 0x7f1b04154390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.737+0000 7f1b01e00640 1 -- 172.21.15.19:0/3391942955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b040ad580 msgr2=0x7f1b041568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.737+0000 7f1b01e00640 1 --2- 172.21.15.19:0/3391942955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b040ad580 0x7f1b041568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.737+0000 7f1b01e00640 1 -- 172.21.15.19:0/3391942955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b041572a0 con 0x7f1b040ad120 2024-03-28T13:10:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.737+0000 7f1b01e00640 1 --2- 172.21.15.19:0/3391942955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b040ad120 0x7f1b040aefe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1af40097a0 tx=0x7f1af40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.738+0000 7f1afaa00640 1 -- 172.21.15.19:0/3391942955 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af403a970 con 0x7f1b040ad120 2024-03-28T13:10:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.738+0000 7f1afaa00640 1 -- 172.21.15.19:0/3391942955 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af403ab10 con 0x7f1b040ad120 2024-03-28T13:10:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.738+0000 7f1afaa00640 1 -- 172.21.15.19:0/3391942955 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af40443f0 con 0x7f1b040ad120 2024-03-28T13:10:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.738+0000 7f1b08a00640 1 -- 172.21.15.19:0/3391942955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b04157580 con 0x7f1b040ad120 2024-03-28T13:10:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.738+0000 7f1b08a00640 1 -- 172.21.15.19:0/3391942955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b04157b40 con 0x7f1b040ad120 2024-03-28T13:10:33.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.739+0000 7f1b08a00640 1 -- 172.21.15.19:0/3391942955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b040685c0 con 0x7f1b040ad120 2024-03-28T13:10:33.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.740+0000 7f1afaa00640 1 -- 172.21.15.19:0/3391942955 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1af404b060 con 0x7f1b040ad120 2024-03-28T13:10:33.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.741+0000 7f1afaa00640 1 --2- 172.21.15.19:0/3391942955 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1ae006c530 0x7f1ae006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:33.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.741+0000 7f1afaa00640 1 -- 172.21.15.19:0/3391942955 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f1af40ad240 con 0x7f1b040ad120 2024-03-28T13:10:33.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.742+0000 7f1af9400640 1 --2- 172.21.15.19:0/3391942955 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1ae006c530 0x7f1ae006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.743+0000 7f1af9400640 1 --2- 172.21.15.19:0/3391942955 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1ae006c530 0x7f1ae006e9f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1b040b0ed0 tx=0x7f1aec00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.745+0000 7f1afaa00640 1 -- 172.21.15.19:0/3391942955 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1af403f070 con 0x7f1b040ad120 2024-03-28T13:10:33.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.951+0000 7f1b08a00640 1 -- 172.21.15.19:0/3391942955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b0406ea70 con 0x7f1b040ad120 2024-03-28T13:10:33.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.952+0000 7f1afaa00640 1 -- 172.21.15.19:0/3391942955 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16704 (secure 0 0 0) 0x7f1af407b8b0 con 0x7f1b040ad120 2024-03-28T13:10:33.953 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:33.953 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:10:23.142768+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":0,"to":4}]},{"pgid":"3.1b","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:10:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.955+0000 7f1b08a00640 1 -- 172.21.15.19:0/3391942955 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1ae006c530 msgr2=0x7f1ae006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.955+0000 7f1b08a00640 1 --2- 172.21.15.19:0/3391942955 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1ae006c530 0x7f1ae006e9f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1b040b0ed0 tx=0x7f1aec00b040 comp rx=0 tx=0).stop 2024-03-28T13:10:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.955+0000 7f1b08a00640 1 -- 172.21.15.19:0/3391942955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b040ad120 msgr2=0x7f1b040aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.955+0000 7f1b08a00640 1 --2- 172.21.15.19:0/3391942955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b040ad120 0x7f1b040aefe0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1af40097a0 tx=0x7f1af40046c0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.955+0000 7f1b08a00640 1 -- 172.21.15.19:0/3391942955 shutdown_connections 2024-03-28T13:10:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.955+0000 7f1b08a00640 1 --2- 172.21.15.19:0/3391942955 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1ae006c530 0x7f1ae006e9f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.955+0000 7f1b08a00640 1 --2- 172.21.15.19:0/3391942955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b04161b00 0x7f1b04154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.955+0000 7f1b08a00640 1 --2- 172.21.15.19:0/3391942955 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b040ad580 0x7f1b041568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.955+0000 7f1b08a00640 1 --2- 172.21.15.19:0/3391942955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b040ad120 0x7f1b040aefe0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.955+0000 7f1b08a00640 1 -- 172.21.15.19:0/3391942955 >> 172.21.15.19:0/3391942955 conn(0x7f1b0406b990 msgr2=0x7f1b040b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.956+0000 7f1b08a00640 1 -- 172.21.15.19:0/3391942955 shutdown_connections 2024-03-28T13:10:33.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:33.956+0000 7f1b08a00640 1 -- 172.21.15.19:0/3391942955 wait complete. 2024-03-28T13:10:33.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:09:39.243813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:09:40.092899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:10:33.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:10:34.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.188+0000 7f4e1ac00640 1 Processor -- start 2024-03-28T13:10:34.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.188+0000 7f4e1ac00640 1 -- start start 2024-03-28T13:10:34.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.189+0000 7f4e1ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140d43d0 0x7f4e140d47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:34.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.189+0000 7f4e1ac00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4e140cf6b0 0x7f4e140cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:34.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.189+0000 7f4e1ac00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e1406dfa0 con 0x7f4e1415e860 2024-03-28T13:10:34.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.190+0000 7f4e1ac00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4e140cf220 con 0x7f4e140cf6b0 2024-03-28T13:10:34.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.190+0000 7f4e1ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4e140d4cf0 con 0x7f4e140d43d0 2024-03-28T13:10:34.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.190+0000 7f4e18c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4e140cf6b0 0x7f4e140cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:34.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.190+0000 7f4e18c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4e140cf6b0 0x7f4e140cfb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40374/0 (socket says 172.21.15.19:40374) 2024-03-28T13:10:34.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.190+0000 7f4e18c00640 1 -- 172.21.15.19:0/345557977 learned_addr learned my addr 172.21.15.19:0/345557977 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:34.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.190+0000 7f4e13400640 1 --2- 172.21.15.19:0/345557977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140d43d0 0x7f4e140d47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:34.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.191+0000 7f4e18c00640 1 -- 172.21.15.19:0/345557977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140d43d0 msgr2=0x7f4e140d47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:34.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.191+0000 7f4e18c00640 1 --2- 172.21.15.19:0/345557977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140d43d0 0x7f4e140d47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.191+0000 7f4e18c00640 1 -- 172.21.15.19:0/345557977 >> v1:172.21.15.188:6789/0 conn(0x7f4e1415e860 legacy=0x7f4e1406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:34.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.191+0000 7f4e18c00640 1 -- 172.21.15.19:0/345557977 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1406e240 con 0x7f4e140cf6b0 2024-03-28T13:10:34.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.193+0000 7f4e18c00640 1 --2- 172.21.15.19:0/345557977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e140cf6b0 0x7f4e140cfb00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4e1406e170 tx=0x7f4e0c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c264e18fd1d9c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:34.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.194+0000 7f4e12a00640 1 -- 172.21.15.19:0/345557977 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c034070 con 0x7f4e140cf6b0 2024-03-28T13:10:34.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.194+0000 7f4e12a00640 1 -- 172.21.15.19:0/345557977 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e0c030840 con 0x7f4e140cf6b0 2024-03-28T13:10:34.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.194+0000 7f4e12a00640 1 -- 172.21.15.19:0/345557977 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c030c10 con 0x7f4e140cf6b0 2024-03-28T13:10:34.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.194+0000 7f4e1ac00640 1 -- 172.21.15.19:0/345557977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e140cf6b0 msgr2=0x7f4e140cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:34.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.194+0000 7f4e1ac00640 1 --2- 172.21.15.19:0/345557977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e140cf6b0 0x7f4e140cfb00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4e1406e170 tx=0x7f4e0c02b980 comp rx=0 tx=0).stop 2024-03-28T13:10:34.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.194+0000 7f4e1ac00640 1 -- 172.21.15.19:0/345557977 shutdown_connections 2024-03-28T13:10:34.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.194+0000 7f4e1ac00640 1 --2- 172.21.15.19:0/345557977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e140cf6b0 0x7f4e140cfb00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.194+0000 7f4e1ac00640 1 --2- 172.21.15.19:0/345557977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140d43d0 0x7f4e140d47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.194+0000 7f4e1ac00640 1 -- 172.21.15.19:0/345557977 >> 172.21.15.19:0/345557977 conn(0x7f4e1409d7d0 msgr2=0x7f4e1408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:34.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.194+0000 7f4e1ac00640 1 -- 172.21.15.19:0/345557977 shutdown_connections 2024-03-28T13:10:34.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.194+0000 7f4e1ac00640 1 -- 172.21.15.19:0/345557977 wait complete. 2024-03-28T13:10:34.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.197+0000 7f4e1ac00640 1 Processor -- start 2024-03-28T13:10:34.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.197+0000 7f4e1ac00640 1 -- start start 2024-03-28T13:10:34.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.198+0000 7f4e1ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140cf6b0 0x7f4e140cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.198+0000 7f4e1ac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e140d43d0 0x7f4e1415a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.198+0000 7f4e1ac00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e1415e860 0x7f4e14160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.199+0000 7f4e1ac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1406dc00 con 0x7f4e140d43d0 2024-03-28T13:10:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.199+0000 7f4e1ac00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4e140d2c50 con 0x7f4e1415e860 2024-03-28T13:10:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.199+0000 7f4e1ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4e1406dd80 con 0x7f4e140cf6b0 2024-03-28T13:10:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.199+0000 7f4e13400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e140d43d0 0x7f4e1415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.199+0000 7f4e13400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e140d43d0 0x7f4e1415a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40380/0 (socket says 172.21.15.19:40380) 2024-03-28T13:10:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.199+0000 7f4e13400640 1 -- 172.21.15.19:0/1557526282 learned_addr learned my addr 172.21.15.19:0/1557526282 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.199+0000 7f4e13e00640 1 --2- 172.21.15.19:0/1557526282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140cf6b0 0x7f4e140cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.199+0000 7f4e18c00640 1 --2- 172.21.15.19:0/1557526282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e1415e860 0x7f4e14160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.199+0000 7f4e13400640 1 -- 172.21.15.19:0/1557526282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140cf6b0 msgr2=0x7f4e140cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.199+0000 7f4e13400640 1 --2- 172.21.15.19:0/1557526282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140cf6b0 0x7f4e140cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.199+0000 7f4e13400640 1 -- 172.21.15.19:0/1557526282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e1415e860 msgr2=0x7f4e14160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.199+0000 7f4e13400640 1 --2- 172.21.15.19:0/1557526282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e1415e860 0x7f4e14160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.199+0000 7f4e13400640 1 -- 172.21.15.19:0/1557526282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e140cbd40 con 0x7f4e140d43d0 2024-03-28T13:10:34.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.200+0000 7f4e13400640 1 --2- 172.21.15.19:0/1557526282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e140d43d0 0x7f4e1415a490 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00e9e0 tx=0x7f4dfc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:34.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.200+0000 7f4e0be00640 1 -- 172.21.15.19:0/1557526282 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc017070 con 0x7f4e140d43d0 2024-03-28T13:10:34.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.200+0000 7f4e0be00640 1 -- 172.21.15.19:0/1557526282 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dfc00c9d0 con 0x7f4e140d43d0 2024-03-28T13:10:34.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.200+0000 7f4e1ac00640 1 -- 172.21.15.19:0/1557526282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e141f4b00 con 0x7f4e140d43d0 2024-03-28T13:10:34.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.200+0000 7f4e0be00640 1 -- 172.21.15.19:0/1557526282 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc00cdc0 con 0x7f4e140d43d0 2024-03-28T13:10:34.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.200+0000 7f4e1ac00640 1 -- 172.21.15.19:0/1557526282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e1415fd10 con 0x7f4e140d43d0 2024-03-28T13:10:34.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.201+0000 7f4e1ac00640 1 -- 172.21.15.19:0/1557526282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e140685c0 con 0x7f4e140d43d0 2024-03-28T13:10:34.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.203+0000 7f4e0be00640 1 -- 172.21.15.19:0/1557526282 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4dfc01a070 con 0x7f4e140d43d0 2024-03-28T13:10:34.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.203+0000 7f4e0be00640 1 --2- 172.21.15.19:0/1557526282 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4df006c530 0x7f4df006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.204+0000 7f4e0be00640 1 -- 172.21.15.19:0/1557526282 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f4dfc08b7d0 con 0x7f4e140d43d0 2024-03-28T13:10:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.204+0000 7f4e13e00640 1 --2- 172.21.15.19:0/1557526282 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4df006c530 0x7f4df006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.205+0000 7f4e13e00640 1 --2- 172.21.15.19:0/1557526282 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4df006c530 0x7f4df006e9f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e1415b0f0 tx=0x7f4e04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:34.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.213+0000 7f4e0be00640 1 -- 172.21.15.19:0/1557526282 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4dfc059d10 con 0x7f4e140d43d0 2024-03-28T13:10:34.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.414+0000 7f4e1ac00640 1 -- 172.21.15.19:0/1557526282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4e140cba50 con 0x7f4e140d43d0 2024-03-28T13:10:34.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.415+0000 7f4e0be00640 1 -- 172.21.15.19:0/1557526282 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f4dfc059610 con 0x7f4e140d43d0 2024-03-28T13:10:34.415 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:10:34.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.417+0000 7f4e1ac00640 1 -- 172.21.15.19:0/1557526282 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4df006c530 msgr2=0x7f4df006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:34.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.417+0000 7f4e1ac00640 1 --2- 172.21.15.19:0/1557526282 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4df006c530 0x7f4df006e9f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e1415b0f0 tx=0x7f4e04009040 comp rx=0 tx=0).stop 2024-03-28T13:10:34.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.417+0000 7f4e1ac00640 1 -- 172.21.15.19:0/1557526282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e140d43d0 msgr2=0x7f4e1415a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:34.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.417+0000 7f4e1ac00640 1 --2- 172.21.15.19:0/1557526282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e140d43d0 0x7f4e1415a490 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00e9e0 tx=0x7f4dfc00eea0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.418+0000 7f4e1ac00640 1 -- 172.21.15.19:0/1557526282 shutdown_connections 2024-03-28T13:10:34.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.418+0000 7f4e1ac00640 1 --2- 172.21.15.19:0/1557526282 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4df006c530 0x7f4df006e9f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.418+0000 7f4e1ac00640 1 --2- 172.21.15.19:0/1557526282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e1415e860 0x7f4e14160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.418+0000 7f4e1ac00640 1 --2- 172.21.15.19:0/1557526282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e140d43d0 0x7f4e1415a490 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.418+0000 7f4e1ac00640 1 --2- 172.21.15.19:0/1557526282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e140cf6b0 0x7f4e140cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.418+0000 7f4e1ac00640 1 -- 172.21.15.19:0/1557526282 >> 172.21.15.19:0/1557526282 conn(0x7f4e1409d7d0 msgr2=0x7f4e140b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:34.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.418+0000 7f4e1ac00640 1 -- 172.21.15.19:0/1557526282 shutdown_connections 2024-03-28T13:10:34.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.418+0000 7f4e1ac00640 1 -- 172.21.15.19:0/1557526282 wait complete. 2024-03-28T13:10:34.436 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:10:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.655+0000 7ff595600640 1 Processor -- start 2024-03-28T13:10:34.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.655+0000 7ff595600640 1 -- start start 2024-03-28T13:10:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.656+0000 7ff595600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff590155550 0x7ff590155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.656+0000 7ff595600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff59015f800 0x7ff5900d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.656+0000 7ff595600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5900b40b0 con 0x7ff5900d07c0 2024-03-28T13:10:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.656+0000 7ff595600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff5900d0c30 con 0x7ff59015f800 2024-03-28T13:10:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.656+0000 7ff595600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff5900b4390 con 0x7ff590155550 2024-03-28T13:10:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.657+0000 7ff586000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff59015f800 0x7ff5900d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.657+0000 7ff58f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff5900d07c0 0x7ff5900aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54480/0 (socket says 172.21.15.19:54480) 2024-03-28T13:10:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.657+0000 7ff58f400640 1 -- 172.21.15.19:0/598208934 learned_addr learned my addr 172.21.15.19:0/598208934 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.657+0000 7ff58ea00640 1 --2- 172.21.15.19:0/598208934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff590155550 0x7ff590155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.657+0000 7ff586000640 1 -- 172.21.15.19:0/598208934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff590155550 msgr2=0x7ff590155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.657+0000 7ff586000640 1 --2- 172.21.15.19:0/598208934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff590155550 0x7ff590155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.657+0000 7ff586000640 1 -- 172.21.15.19:0/598208934 >> v1:172.21.15.19:6789/0 conn(0x7ff5900d07c0 legacy=0x7ff5900aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:34.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.657+0000 7ff58e000640 1 -- 172.21.15.19:0/598208934 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 301694840 0 0) 0x7ff5900b40b0 con 0x7ff5900d07c0 2024-03-28T13:10:34.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.657+0000 7ff586000640 1 -- 172.21.15.19:0/598208934 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5900b4670 con 0x7ff59015f800 2024-03-28T13:10:34.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.657+0000 7ff58ea00640 1 --2- 172.21.15.19:0/598208934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff590155550 0x7ff590155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:34.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.658+0000 7ff586000640 1 --2- 172.21.15.19:0/598208934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff59015f800 0x7ff5900d0280 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff5780097a0 tx=0x7ff57802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e66a9d892ab646 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:34.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.659+0000 7ff58e000640 1 -- 172.21.15.19:0/598208934 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578035020 con 0x7ff59015f800 2024-03-28T13:10:34.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.659+0000 7ff58e000640 1 -- 172.21.15.19:0/598208934 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff578031810 con 0x7ff59015f800 2024-03-28T13:10:34.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.659+0000 7ff58e000640 1 -- 172.21.15.19:0/598208934 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578031be0 con 0x7ff59015f800 2024-03-28T13:10:34.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.659+0000 7ff595600640 1 -- 172.21.15.19:0/598208934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff59015f800 msgr2=0x7ff5900d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:34.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.659+0000 7ff595600640 1 --2- 172.21.15.19:0/598208934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff59015f800 0x7ff5900d0280 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff5780097a0 tx=0x7ff57802b8b0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.659+0000 7ff595600640 1 -- 172.21.15.19:0/598208934 shutdown_connections 2024-03-28T13:10:34.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.659+0000 7ff595600640 1 --2- 172.21.15.19:0/598208934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff59015f800 0x7ff5900d0280 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.659+0000 7ff595600640 1 --2- 172.21.15.19:0/598208934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff590155550 0x7ff590155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.659+0000 7ff595600640 1 -- 172.21.15.19:0/598208934 >> 172.21.15.19:0/598208934 conn(0x7ff59006b750 msgr2=0x7ff59006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.659+0000 7ff595600640 1 -- 172.21.15.19:0/598208934 shutdown_connections 2024-03-28T13:10:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.659+0000 7ff595600640 1 -- 172.21.15.19:0/598208934 wait complete. 2024-03-28T13:10:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.661+0000 7ff595600640 1 Processor -- start 2024-03-28T13:10:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.661+0000 7ff595600640 1 -- start start 2024-03-28T13:10:34.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.662+0000 7ff595600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5900d07c0 0x7ff5900b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:34.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.662+0000 7ff595600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff590155550 0x7ff59009d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.662+0000 7ff595600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff59015f800 0x7ff5901f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.662+0000 7ff595600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff59015fc80 con 0x7ff590155550 2024-03-28T13:10:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.662+0000 7ff595600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff5900b4c30 con 0x7ff59015f800 2024-03-28T13:10:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.662+0000 7ff595600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff59015ba20 con 0x7ff5900d07c0 2024-03-28T13:10:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.663+0000 7ff586000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff590155550 0x7ff59009d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.663+0000 7ff586000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff590155550 0x7ff59009d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40402/0 (socket says 172.21.15.19:40402) 2024-03-28T13:10:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.663+0000 7ff586000640 1 -- 172.21.15.19:0/4221642003 learned_addr learned my addr 172.21.15.19:0/4221642003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.663+0000 7ff58ea00640 1 --2- 172.21.15.19:0/4221642003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5900d07c0 0x7ff5900b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.663+0000 7ff586000640 1 -- 172.21.15.19:0/4221642003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5900d07c0 msgr2=0x7ff5900b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.663+0000 7ff58f400640 1 --2- 172.21.15.19:0/4221642003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff59015f800 0x7ff5901f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.663+0000 7ff586000640 1 --2- 172.21.15.19:0/4221642003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5900d07c0 0x7ff5900b0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.663+0000 7ff586000640 1 -- 172.21.15.19:0/4221642003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff59015f800 msgr2=0x7ff5901f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:34.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.663+0000 7ff586000640 1 --2- 172.21.15.19:0/4221642003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff59015f800 0x7ff5901f3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.663+0000 7ff586000640 1 -- 172.21.15.19:0/4221642003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59009a2c0 con 0x7ff590155550 2024-03-28T13:10:34.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.663+0000 7ff58f400640 1 --2- 172.21.15.19:0/4221642003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff59015f800 0x7ff5901f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:34.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.664+0000 7ff586000640 1 --2- 172.21.15.19:0/4221642003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff590155550 0x7ff59009d8d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff5780097a0 tx=0x7ff5780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.664+0000 7ff587400640 1 -- 172.21.15.19:0/4221642003 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578035050 con 0x7ff590155550 2024-03-28T13:10:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.664+0000 7ff587400640 1 -- 172.21.15.19:0/4221642003 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57803bd70 con 0x7ff590155550 2024-03-28T13:10:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.664+0000 7ff587400640 1 -- 172.21.15.19:0/4221642003 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578004250 con 0x7ff590155550 2024-03-28T13:10:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.664+0000 7ff595600640 1 -- 172.21.15.19:0/4221642003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff59009a5a0 con 0x7ff590155550 2024-03-28T13:10:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.664+0000 7ff595600640 1 -- 172.21.15.19:0/4221642003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff59009de10 con 0x7ff590155550 2024-03-28T13:10:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.665+0000 7ff595600640 1 -- 172.21.15.19:0/4221642003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5900685c0 con 0x7ff590155550 2024-03-28T13:10:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.666+0000 7ff587400640 1 -- 172.21.15.19:0/4221642003 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff578038020 con 0x7ff590155550 2024-03-28T13:10:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.667+0000 7ff587400640 1 --2- 172.21.15.19:0/4221642003 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff56c06c530 0x7ff56c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.667+0000 7ff587400640 1 -- 172.21.15.19:0/4221642003 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7ff5780ad790 con 0x7ff590155550 2024-03-28T13:10:34.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.668+0000 7ff58ea00640 1 --2- 172.21.15.19:0/4221642003 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff56c06c530 0x7ff56c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.669+0000 7ff58ea00640 1 --2- 172.21.15.19:0/4221642003 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff56c06c530 0x7ff56c06e9f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff580008850 tx=0x7ff580008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:34.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.672+0000 7ff587400640 1 -- 172.21.15.19:0/4221642003 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff578040070 con 0x7ff590155550 2024-03-28T13:10:34.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.875+0000 7ff595600640 1 -- 172.21.15.19:0/4221642003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff5900b3ad0 con 0x7ff590155550 2024-03-28T13:10:34.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.876+0000 7ff587400640 1 -- 172.21.15.19:0/4221642003 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7ff57807be00 con 0x7ff590155550 2024-03-28T13:10:34.876 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:10:34.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.878+0000 7ff595600640 1 -- 172.21.15.19:0/4221642003 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff56c06c530 msgr2=0x7ff56c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:34.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.878+0000 7ff595600640 1 --2- 172.21.15.19:0/4221642003 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff56c06c530 0x7ff56c06e9f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff580008850 tx=0x7ff580008040 comp rx=0 tx=0).stop 2024-03-28T13:10:34.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.878+0000 7ff595600640 1 -- 172.21.15.19:0/4221642003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff590155550 msgr2=0x7ff59009d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:34.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.878+0000 7ff595600640 1 --2- 172.21.15.19:0/4221642003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff590155550 0x7ff59009d8d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff5780097a0 tx=0x7ff5780046c0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.878+0000 7ff595600640 1 -- 172.21.15.19:0/4221642003 shutdown_connections 2024-03-28T13:10:34.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.878+0000 7ff595600640 1 --2- 172.21.15.19:0/4221642003 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff56c06c530 0x7ff56c06e9f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.878+0000 7ff595600640 1 --2- 172.21.15.19:0/4221642003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff59015f800 0x7ff5901f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.878+0000 7ff595600640 1 --2- 172.21.15.19:0/4221642003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff590155550 0x7ff59009d8d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.878+0000 7ff595600640 1 --2- 172.21.15.19:0/4221642003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5900d07c0 0x7ff5900b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:34.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.878+0000 7ff595600640 1 -- 172.21.15.19:0/4221642003 >> 172.21.15.19:0/4221642003 conn(0x7ff59006b750 msgr2=0x7ff590072750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:34.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.879+0000 7ff595600640 1 -- 172.21.15.19:0/4221642003 shutdown_connections 2024-03-28T13:10:34.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:34.879+0000 7ff595600640 1 -- 172.21.15.19:0/4221642003 wait complete. 2024-03-28T13:10:34.898 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:10:35.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.110+0000 7f1abd600640 1 Processor -- start 2024-03-28T13:10:35.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.110+0000 7f1abd600640 1 -- start start 2024-03-28T13:10:35.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.113+0000 7f1abd600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1ab815e4c0 0x7f1ab80b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:35.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.113+0000 7f1abd600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab80d2710 con 0x7f1ab80b69c0 2024-03-28T13:10:35.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.113+0000 7f1abd600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ab80d29f0 con 0x7f1ab80d37b0 2024-03-28T13:10:35.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.113+0000 7f1abd600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1ab815e8a0 con 0x7f1ab815e4c0 2024-03-28T13:10:35.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.113+0000 7f1abcc00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1ab80b69c0 0x7f1ab80b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:54482/0 (socket says 172.21.15.19:54482) 2024-03-28T13:10:35.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.113+0000 7f1ab7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1ab815e4c0 0x7f1ab80b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:35.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.113+0000 7f1abcc00640 1 -- 172.21.15.19:0/2192120942 learned_addr learned my addr 172.21.15.19:0/2192120942 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:35.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.114+0000 7f1ab7e00640 1 -- 172.21.15.19:0/2192120942 >> v1:172.21.15.188:6789/0 conn(0x7f1ab80d37b0 legacy=0x7f1ab80b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:35.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.114+0000 7f1ab7e00640 1 -- 172.21.15.19:0/2192120942 >> v1:172.21.15.19:6789/0 conn(0x7f1ab80b69c0 legacy=0x7f1ab80b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:35.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.114+0000 7f1ab6a00640 1 -- 172.21.15.19:0/2192120942 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3653872821 0 0) 0x7f1ab80d2710 con 0x7f1ab80b69c0 2024-03-28T13:10:35.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.114+0000 7f1ab7e00640 1 -- 172.21.15.19:0/2192120942 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab80d2cd0 con 0x7f1ab815e4c0 2024-03-28T13:10:35.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.115+0000 7f1ab7e00640 1 --2- 172.21.15.19:0/2192120942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab815e4c0 0x7f1ab80b9bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009d90 tx=0x7f1aa802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e4de2ee7de13ec server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:35.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.115+0000 7f1ab6a00640 1 -- 172.21.15.19:0/2192120942 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8039070 con 0x7f1ab815e4c0 2024-03-28T13:10:35.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.115+0000 7f1ab6a00640 1 -- 172.21.15.19:0/2192120942 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa802cc60 con 0x7f1ab815e4c0 2024-03-28T13:10:35.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.115+0000 7f1ab6a00640 1 -- 172.21.15.19:0/2192120942 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8034660 con 0x7f1ab815e4c0 2024-03-28T13:10:35.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.116+0000 7f1abd600640 1 -- 172.21.15.19:0/2192120942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab815e4c0 msgr2=0x7f1ab80b9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:35.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.116+0000 7f1abd600640 1 --2- 172.21.15.19:0/2192120942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab815e4c0 0x7f1ab80b9bf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009d90 tx=0x7f1aa802beb0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.116+0000 7f1abd600640 1 -- 172.21.15.19:0/2192120942 shutdown_connections 2024-03-28T13:10:35.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.116+0000 7f1abd600640 1 --2- 172.21.15.19:0/2192120942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab815e4c0 0x7f1ab80b9bf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.116+0000 7f1abd600640 1 -- 172.21.15.19:0/2192120942 >> 172.21.15.19:0/2192120942 conn(0x7f1ab80676a0 msgr2=0x7f1ab8067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:35.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.116+0000 7f1abd600640 1 -- 172.21.15.19:0/2192120942 shutdown_connections 2024-03-28T13:10:35.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.116+0000 7f1abd600640 1 -- 172.21.15.19:0/2192120942 wait complete. 2024-03-28T13:10:35.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.118+0000 7f1abd600640 1 Processor -- start 2024-03-28T13:10:35.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.118+0000 7f1abd600640 1 -- start start 2024-03-28T13:10:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.119+0000 7f1abd600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab80b69c0 0x7f1ab80cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.119+0000 7f1abd600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab80d37b0 0x7f1ab815f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.119+0000 7f1abd600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ab815e4c0 0x7f1ab814d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.119+0000 7f1abd600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab80d3320 con 0x7f1ab80b69c0 2024-03-28T13:10:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.119+0000 7f1abd600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab80d4040 con 0x7f1ab815e4c0 2024-03-28T13:10:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.119+0000 7f1abd600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab80ba130 con 0x7f1ab80d37b0 2024-03-28T13:10:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.119+0000 7f1ab7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab80b69c0 0x7f1ab80cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.119+0000 7f1ab7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab80b69c0 0x7f1ab80cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40424/0 (socket says 172.21.15.19:40424) 2024-03-28T13:10:35.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.119+0000 7f1ab7e00640 1 -- 172.21.15.19:0/313744380 learned_addr learned my addr 172.21.15.19:0/313744380 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:35.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.120+0000 7f1ab7400640 1 --2- 172.21.15.19:0/313744380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab80d37b0 0x7f1ab815f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:35.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.120+0000 7f1abcc00640 1 --2- 172.21.15.19:0/313744380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ab815e4c0 0x7f1ab814d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:35.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.120+0000 7f1ab7e00640 1 -- 172.21.15.19:0/313744380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab80d37b0 msgr2=0x7f1ab815f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:35.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.120+0000 7f1ab7e00640 1 --2- 172.21.15.19:0/313744380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab80d37b0 0x7f1ab815f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.120+0000 7f1ab7e00640 1 -- 172.21.15.19:0/313744380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ab815e4c0 msgr2=0x7f1ab814d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:35.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.120+0000 7f1ab7e00640 1 --2- 172.21.15.19:0/313744380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ab815e4c0 0x7f1ab814d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.120+0000 7f1ab7e00640 1 -- 172.21.15.19:0/313744380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab815ff60 con 0x7f1ab80b69c0 2024-03-28T13:10:35.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.120+0000 7f1ab7400640 1 --2- 172.21.15.19:0/313744380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab80d37b0 0x7f1ab815f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.120+0000 7f1ab7e00640 1 --2- 172.21.15.19:0/313744380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab80b69c0 0x7f1ab80cc0a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009d90 tx=0x7f1aa8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.121+0000 7f1aafe00640 1 -- 172.21.15.19:0/313744380 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8044020 con 0x7f1ab80b69c0 2024-03-28T13:10:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.121+0000 7f1aafe00640 1 -- 172.21.15.19:0/313744380 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa803f670 con 0x7f1ab80b69c0 2024-03-28T13:10:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.121+0000 7f1abd600640 1 -- 172.21.15.19:0/313744380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab8160240 con 0x7f1ab80b69c0 2024-03-28T13:10:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.121+0000 7f1aafe00640 1 -- 172.21.15.19:0/313744380 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8039040 con 0x7f1ab80b69c0 2024-03-28T13:10:35.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.121+0000 7f1abd600640 1 -- 172.21.15.19:0/313744380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab8160800 con 0x7f1ab80b69c0 2024-03-28T13:10:35.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.122+0000 7f1abd600640 1 -- 172.21.15.19:0/313744380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab8075940 con 0x7f1ab80b69c0 2024-03-28T13:10:35.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.123+0000 7f1aafe00640 1 -- 172.21.15.19:0/313744380 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1aa8047020 con 0x7f1ab80b69c0 2024-03-28T13:10:35.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.125+0000 7f1aafe00640 1 --2- 172.21.15.19:0/313744380 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a9406c530 0x7f1a9406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:35.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.125+0000 7f1aafe00640 1 -- 172.21.15.19:0/313744380 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f1aa80ae360 con 0x7f1ab80b69c0 2024-03-28T13:10:35.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.125+0000 7f1ab7400640 1 --2- 172.21.15.19:0/313744380 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a9406c530 0x7f1a9406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:35.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.126+0000 7f1ab7400640 1 --2- 172.21.15.19:0/313744380 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a9406c530 0x7f1a9406e9f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80ce900 tx=0x7f1aa0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:35.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.129+0000 7f1aafe00640 1 -- 172.21.15.19:0/313744380 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1aa804e030 con 0x7f1ab80b69c0 2024-03-28T13:10:35.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.419+0000 7f1abd600640 1 -- 172.21.15.19:0/313744380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1ab814da60 con 0x7f1ab80b69c0 2024-03-28T13:10:35.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.420+0000 7f1aafe00640 1 -- 172.21.15.19:0/313744380 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f1aa807c940 con 0x7f1ab80b69c0 2024-03-28T13:10:35.420 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:10:35.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.422+0000 7f1abd600640 1 -- 172.21.15.19:0/313744380 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a9406c530 msgr2=0x7f1a9406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:35.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.422+0000 7f1abd600640 1 --2- 172.21.15.19:0/313744380 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a9406c530 0x7f1a9406e9f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80ce900 tx=0x7f1aa0009040 comp rx=0 tx=0).stop 2024-03-28T13:10:35.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.422+0000 7f1abd600640 1 -- 172.21.15.19:0/313744380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab80b69c0 msgr2=0x7f1ab80cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:35.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.422+0000 7f1abd600640 1 --2- 172.21.15.19:0/313744380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab80b69c0 0x7f1ab80cc0a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009d90 tx=0x7f1aa8004a40 comp rx=0 tx=0).stop 2024-03-28T13:10:35.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.423+0000 7f1abd600640 1 -- 172.21.15.19:0/313744380 shutdown_connections 2024-03-28T13:10:35.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.423+0000 7f1abd600640 1 --2- 172.21.15.19:0/313744380 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a9406c530 0x7f1a9406e9f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.423+0000 7f1abd600640 1 --2- 172.21.15.19:0/313744380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ab815e4c0 0x7f1ab814d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.423+0000 7f1abd600640 1 --2- 172.21.15.19:0/313744380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ab80d37b0 0x7f1ab815f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.423+0000 7f1abd600640 1 --2- 172.21.15.19:0/313744380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ab80b69c0 0x7f1ab80cc0a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.423+0000 7f1abd600640 1 -- 172.21.15.19:0/313744380 >> 172.21.15.19:0/313744380 conn(0x7f1ab80676a0 msgr2=0x7f1ab80c7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.423+0000 7f1abd600640 1 -- 172.21.15.19:0/313744380 shutdown_connections 2024-03-28T13:10:35.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.423+0000 7f1abd600640 1 -- 172.21.15.19:0/313744380 wait complete. 2024-03-28T13:10:35.444 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:10:35.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.652+0000 7f3413e00640 1 Processor -- start 2024-03-28T13:10:35.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.653+0000 7f3413e00640 1 -- start start 2024-03-28T13:10:35.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.654+0000 7f3413e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f340c0afd80 0x7f340c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:35.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.654+0000 7f3413e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c06cc90 0x7f340c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:35.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.655+0000 7f3413e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f340c0b4710 con 0x7f340c0ccc40 2024-03-28T13:10:35.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.655+0000 7f3413e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f340c0b5740 con 0x7f340c0afd80 2024-03-28T13:10:35.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.655+0000 7f3413e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f340c15da00 con 0x7f340c06cc90 2024-03-28T13:10:35.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.655+0000 7f3411400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f340c0afd80 0x7f340c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:35.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.655+0000 7f3411400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f340c0afd80 0x7f340c15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44044/0 (socket says 172.21.15.19:44044) 2024-03-28T13:10:35.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.655+0000 7f3411400640 1 -- 172.21.15.19:0/1768400799 learned_addr learned my addr 172.21.15.19:0/1768400799 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:35.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.655+0000 7f3411e00640 1 --2- 172.21.15.19:0/1768400799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c06cc90 0x7f340c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:35.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.656+0000 7f3411400640 1 -- 172.21.15.19:0/1768400799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c06cc90 msgr2=0x7f340c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:35.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.656+0000 7f3411400640 1 --2- 172.21.15.19:0/1768400799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c06cc90 0x7f340c0ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.656+0000 7f3411400640 1 -- 172.21.15.19:0/1768400799 >> v1:172.21.15.188:6789/0 conn(0x7f340c0ccc40 legacy=0x7f340c06c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:35.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.656+0000 7f3411400640 1 -- 172.21.15.19:0/1768400799 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340c0b4980 con 0x7f340c0afd80 2024-03-28T13:10:35.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.657+0000 7f3411400640 1 --2- 172.21.15.19:0/1768400799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c0afd80 0x7f340c15d4c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009e50 tx=0x7f33fc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67c01bd14beef98 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:35.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.657+0000 7f340be00640 1 -- 172.21.15.19:0/1768400799 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc02ca70 con 0x7f340c0afd80 2024-03-28T13:10:35.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.657+0000 7f340be00640 1 -- 172.21.15.19:0/1768400799 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33fc02cc10 con 0x7f340c0afd80 2024-03-28T13:10:35.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.657+0000 7f340be00640 1 -- 172.21.15.19:0/1768400799 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33fc0343f0 con 0x7f340c0afd80 2024-03-28T13:10:35.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.658+0000 7f3413e00640 1 -- 172.21.15.19:0/1768400799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c0afd80 msgr2=0x7f340c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:35.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.658+0000 7f3413e00640 1 --2- 172.21.15.19:0/1768400799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c0afd80 0x7f340c15d4c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009e50 tx=0x7f33fc02bed0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.658+0000 7f3413e00640 1 -- 172.21.15.19:0/1768400799 shutdown_connections 2024-03-28T13:10:35.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.658+0000 7f3413e00640 1 --2- 172.21.15.19:0/1768400799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c06cc90 0x7f340c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.658+0000 7f3413e00640 1 --2- 172.21.15.19:0/1768400799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c0afd80 0x7f340c15d4c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.658+0000 7f3413e00640 1 -- 172.21.15.19:0/1768400799 >> 172.21.15.19:0/1768400799 conn(0x7f340c071e20 msgr2=0x7f340c072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:35.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.658+0000 7f3413e00640 1 -- 172.21.15.19:0/1768400799 shutdown_connections 2024-03-28T13:10:35.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.658+0000 7f3413e00640 1 -- 172.21.15.19:0/1768400799 wait complete. 2024-03-28T13:10:35.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.661+0000 7f3413e00640 1 Processor -- start 2024-03-28T13:10:35.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.661+0000 7f3413e00640 1 -- start start 2024-03-28T13:10:35.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3413e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c06cc90 0x7f340c1593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:35.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3413e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340c0afd80 0x7f340c15bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:35.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3413e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c0ccc40 0x7f340c15eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3413e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f340c0b5220 con 0x7f340c0ccc40 2024-03-28T13:10:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3413e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f340c0b4f40 con 0x7f340c0afd80 2024-03-28T13:10:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3413e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f340c0b0160 con 0x7f340c06cc90 2024-03-28T13:10:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3411e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c0ccc40 0x7f340c15eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3411e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c0ccc40 0x7f340c15eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44048/0 (socket says 172.21.15.19:44048) 2024-03-28T13:10:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3411e00640 1 -- 172.21.15.19:0/2023565191 learned_addr learned my addr 172.21.15.19:0/2023565191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3411400640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c06cc90 0x7f340c1593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3411e00640 1 -- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c06cc90 msgr2=0x7f340c1593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3411e00640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c06cc90 0x7f340c1593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3410a00640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340c0afd80 0x7f340c15bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3411e00640 1 -- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340c0afd80 msgr2=0x7f340c15bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3411e00640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340c0afd80 0x7f340c15bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.663+0000 7f3411e00640 1 -- 172.21.15.19:0/2023565191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340c1580a0 con 0x7f340c0ccc40 2024-03-28T13:10:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.664+0000 7f3411400640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c06cc90 0x7f340c1593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:35.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.664+0000 7f3410a00640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340c0afd80 0x7f340c15bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:35.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.664+0000 7f3411e00640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c0ccc40 0x7f340c15eea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f340400d4e0 tx=0x7f340400d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:35.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.664+0000 7f3409600640 1 -- 172.21.15.19:0/2023565191 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34040125f0 con 0x7f340c0ccc40 2024-03-28T13:10:35.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.664+0000 7f3409600640 1 -- 172.21.15.19:0/2023565191 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3404019070 con 0x7f340c0ccc40 2024-03-28T13:10:35.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.664+0000 7f3413e00640 1 -- 172.21.15.19:0/2023565191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340c158320 con 0x7f340c0ccc40 2024-03-28T13:10:35.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.664+0000 7f3409600640 1 -- 172.21.15.19:0/2023565191 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34040129f0 con 0x7f340c0ccc40 2024-03-28T13:10:35.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.664+0000 7f3413e00640 1 -- 172.21.15.19:0/2023565191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340c1588e0 con 0x7f340c0ccc40 2024-03-28T13:10:35.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.665+0000 7f3413e00640 1 -- 172.21.15.19:0/2023565191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f340c0b1d80 con 0x7f340c0ccc40 2024-03-28T13:10:35.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.667+0000 7f3409600640 1 -- 172.21.15.19:0/2023565191 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3404012b90 con 0x7f340c0ccc40 2024-03-28T13:10:35.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.668+0000 7f3409600640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33ec06c530 0x7f33ec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:35.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.668+0000 7f3409600640 1 -- 172.21.15.19:0/2023565191 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f340408bb40 con 0x7f340c0ccc40 2024-03-28T13:10:35.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.669+0000 7f3411400640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33ec06c530 0x7f33ec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:35.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.670+0000 7f3411400640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33ec06c530 0x7f33ec06e9f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f33fc035000 tx=0x7f33fc030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.672+0000 7f3409600640 1 -- 172.21.15.19:0/2023565191 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f340c1580a0 con 0x7f340c0ccc40 2024-03-28T13:10:35.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.893+0000 7f3413e00640 1 -- 172.21.15.19:0/2023565191 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f340c06d0b0 con 0x7f340c0ccc40 2024-03-28T13:10:35.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.894+0000 7f3409600640 1 -- 172.21.15.19:0/2023565191 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f340405a080 con 0x7f340c0ccc40 2024-03-28T13:10:35.894 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-03-28T13:10:35.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.897+0000 7f3413e00640 1 -- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33ec06c530 msgr2=0x7f33ec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:35.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.897+0000 7f3413e00640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33ec06c530 0x7f33ec06e9f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f33fc035000 tx=0x7f33fc030000 comp rx=0 tx=0).stop 2024-03-28T13:10:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.897+0000 7f3413e00640 1 -- 172.21.15.19:0/2023565191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c0ccc40 msgr2=0x7f340c15eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.897+0000 7f3413e00640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c0ccc40 0x7f340c15eea0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f340400d4e0 tx=0x7f340400d9a0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.897+0000 7f3413e00640 1 -- 172.21.15.19:0/2023565191 shutdown_connections 2024-03-28T13:10:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.897+0000 7f3413e00640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33ec06c530 0x7f33ec06e9f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.897+0000 7f3413e00640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f340c0ccc40 0x7f340c15eea0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.897+0000 7f3413e00640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340c0afd80 0x7f340c15bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.897+0000 7f3413e00640 1 --2- 172.21.15.19:0/2023565191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c06cc90 0x7f340c1593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.897+0000 7f3413e00640 1 -- 172.21.15.19:0/2023565191 >> 172.21.15.19:0/2023565191 conn(0x7f340c071e20 msgr2=0x7f340c06c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.897+0000 7f3413e00640 1 -- 172.21.15.19:0/2023565191 shutdown_connections 2024-03-28T13:10:35.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:35.898+0000 7f3413e00640 1 -- 172.21.15.19:0/2023565191 wait complete. 2024-03-28T13:10:35.916 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-03-28T13:10:35.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:35.917 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-03-28T13:10:36.023 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:10:36.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:36.023 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-28T13:10:36.075 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:10:36.076 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:36.133 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:10:36.133 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:36.187 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:10:36.187 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:10:36.241 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:10:36.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.457+0000 7fb29dc00640 1 Processor -- start 2024-03-28T13:10:36.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.458+0000 7fb29dc00640 1 -- start start 2024-03-28T13:10:36.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.459+0000 7fb29dc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980cf700 0x7fb298168fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:36.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.459+0000 7fb29dc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb29815e880 con 0x7fb2980d2310 2024-03-28T13:10:36.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.459+0000 7fb29dc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2980d28d0 con 0x7fb29815e4a0 2024-03-28T13:10:36.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.459+0000 7fb29dc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb2980d34e0 con 0x7fb2980cf700 2024-03-28T13:10:36.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.459+0000 7fb28f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb2980d2310 0x7fb2980ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37020/0 (socket says 172.21.15.19:37020) 2024-03-28T13:10:36.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.459+0000 7fb28f400640 1 -- 172.21.15.19:0/471767248 learned_addr learned my addr 172.21.15.19:0/471767248 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:36.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb29d200640 1 --2- 172.21.15.19:0/471767248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980cf700 0x7fb298168fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:36.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb297400640 1 -- 172.21.15.19:0/471767248 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1947851912 0 0) 0x7fb29815e880 con 0x7fb2980d2310 2024-03-28T13:10:36.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb297400640 1 -- 172.21.15.19:0/471767248 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb27c003520 con 0x7fb2980d2310 2024-03-28T13:10:36.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb297400640 1 -- 172.21.15.19:0/471767248 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2656077228 0 0) 0x7fb27c003520 con 0x7fb2980d2310 2024-03-28T13:10:36.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb297400640 1 -- 172.21.15.19:0/471767248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980cf700 msgr2=0x7fb298168fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:36.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb297400640 1 --2- 172.21.15.19:0/471767248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980cf700 0x7fb298168fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:36.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb297400640 1 -- 172.21.15.19:0/471767248 >> v1:172.21.15.188:6789/0 conn(0x7fb29815e4a0 legacy=0x7fb2980b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:36.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb297400640 1 -- 172.21.15.19:0/471767248 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2980d2bb0 con 0x7fb2980d2310 2024-03-28T13:10:36.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb297400640 1 -- 172.21.15.19:0/471767248 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fb280001670 con 0x7fb2980d2310 2024-03-28T13:10:36.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb297400640 1 -- 172.21.15.19:0/471767248 >> v1:172.21.15.19:6789/0 conn(0x7fb2980d2310 legacy=0x7fb2980ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:36.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb297400640 1 --2- 172.21.15.19:0/471767248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb27c0039e0 0x7fb27c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:36.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb297400640 1 --2- 172.21.15.19:0/471767248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980cf700 0x7fb27c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:36.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb297400640 1 --2- 172.21.15.19:0/471767248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27c004d40 0x7fb27c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:36.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb297400640 1 -- 172.21.15.19:0/471767248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2980d34e0 con 0x7fb27c0039e0 2024-03-28T13:10:36.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb297400640 1 -- 172.21.15.19:0/471767248 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c00bb80 con 0x7fb27c004d40 2024-03-28T13:10:36.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.460+0000 7fb297400640 1 -- 172.21.15.19:0/471767248 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb27c00bd80 con 0x7fb2980cf700 2024-03-28T13:10:36.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.461+0000 7fb297e00640 1 --2- 172.21.15.19:0/471767248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb27c0039e0 0x7fb27c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:36.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.461+0000 7fb29d200640 1 --2- 172.21.15.19:0/471767248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27c004d40 0x7fb27c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:36.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.461+0000 7fb28f400640 1 --2- 172.21.15.19:0/471767248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980cf700 0x7fb27c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:36.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.461+0000 7fb297e00640 1 -- 172.21.15.19:0/471767248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980cf700 msgr2=0x7fb27c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:36.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.461+0000 7fb297e00640 1 --2- 172.21.15.19:0/471767248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2980cf700 0x7fb27c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:36.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.461+0000 7fb297e00640 1 -- 172.21.15.19:0/471767248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27c004d40 msgr2=0x7fb27c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:36.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.461+0000 7fb297e00640 1 --2- 172.21.15.19:0/471767248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27c004d40 0x7fb27c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:36.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.461+0000 7fb297e00640 1 -- 172.21.15.19:0/471767248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27c003520 con 0x7fb27c0039e0 2024-03-28T13:10:36.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.463+0000 7fb297e00640 1 --2- 172.21.15.19:0/471767248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb27c0039e0 0x7fb27c003e00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb288001750 tx=0x7fb27c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3725a274346fb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:36.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.463+0000 7fb297400640 1 -- 172.21.15.19:0/471767248 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288001370 con 0x7fb27c0039e0 2024-03-28T13:10:36.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.463+0000 7fb297400640 1 -- 172.21.15.19:0/471767248 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2880052d0 con 0x7fb27c0039e0 2024-03-28T13:10:36.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.463+0000 7fb297400640 1 -- 172.21.15.19:0/471767248 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb288003830 con 0x7fb27c0039e0 2024-03-28T13:10:36.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.464+0000 7fb29dc00640 1 -- 172.21.15.19:0/471767248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb27c0039e0 msgr2=0x7fb27c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:36.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.464+0000 7fb29dc00640 1 --2- 172.21.15.19:0/471767248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb27c0039e0 0x7fb27c003e00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb288001750 tx=0x7fb27c005830 comp rx=0 tx=0).stop 2024-03-28T13:10:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.464+0000 7fb29d200640 1 -- 172.21.15.19:0/471767248 reap_dead start 2024-03-28T13:10:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.464+0000 7fb29dc00640 1 -- 172.21.15.19:0/471767248 shutdown_connections 2024-03-28T13:10:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.464+0000 7fb29dc00640 1 -- 172.21.15.19:0/471767248 >> 172.21.15.19:0/471767248 conn(0x7fb2980676d0 msgr2=0x7fb298067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.464+0000 7fb29dc00640 1 -- 172.21.15.19:0/471767248 shutdown_connections 2024-03-28T13:10:36.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.464+0000 7fb29dc00640 1 -- 172.21.15.19:0/471767248 wait complete. 2024-03-28T13:10:36.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.466+0000 7fb29dc00640 1 Processor -- start 2024-03-28T13:10:36.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.466+0000 7fb29dc00640 1 -- start start 2024-03-28T13:10:36.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.467+0000 7fb29dc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27c0039e0 0x7fb2980ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:36.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.467+0000 7fb29dc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb29814f030 0x7fb29814e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:36.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.467+0000 7fb29dc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb298160a50 0x7fb298160ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:36.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.467+0000 7fb29dc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2980d3970 con 0x7fb298160a50 2024-03-28T13:10:36.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.467+0000 7fb29dc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb2980d3170 con 0x7fb27c0039e0 2024-03-28T13:10:36.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.467+0000 7fb29dc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb2980b83f0 con 0x7fb29814f030 2024-03-28T13:10:36.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.467+0000 7fb29d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb298160a50 0x7fb298160ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:36.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.467+0000 7fb29d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb298160a50 0x7fb298160ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44068/0 (socket says 172.21.15.19:44068) 2024-03-28T13:10:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.467+0000 7fb29d200640 1 -- 172.21.15.19:0/2213271950 learned_addr learned my addr 172.21.15.19:0/2213271950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.468+0000 7fb28f400640 1 --2- 172.21.15.19:0/2213271950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb29814f030 0x7fb29814e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.468+0000 7fb297e00640 1 --2- 172.21.15.19:0/2213271950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27c0039e0 0x7fb2980ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.468+0000 7fb29d200640 1 -- 172.21.15.19:0/2213271950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb29814f030 msgr2=0x7fb29814e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.468+0000 7fb29d200640 1 --2- 172.21.15.19:0/2213271950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb29814f030 0x7fb29814e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.468+0000 7fb29d200640 1 -- 172.21.15.19:0/2213271950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27c0039e0 msgr2=0x7fb2980ca670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.468+0000 7fb29d200640 1 --2- 172.21.15.19:0/2213271950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27c0039e0 0x7fb2980ca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.468+0000 7fb29d200640 1 -- 172.21.15.19:0/2213271950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29815f4e0 con 0x7fb298160a50 2024-03-28T13:10:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.468+0000 7fb29d200640 1 --2- 172.21.15.19:0/2213271950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb298160a50 0x7fb298160ed0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb29806d5c0 tx=0x7fb2900115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.468+0000 7fb294c00640 1 -- 172.21.15.19:0/2213271950 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290011e10 con 0x7fb298160a50 2024-03-28T13:10:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.469+0000 7fb294c00640 1 -- 172.21.15.19:0/2213271950 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb29001b070 con 0x7fb298160a50 2024-03-28T13:10:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.469+0000 7fb294c00640 1 -- 172.21.15.19:0/2213271950 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290003990 con 0x7fb298160a50 2024-03-28T13:10:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.469+0000 7fb29dc00640 1 -- 172.21.15.19:0/2213271950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29815f7c0 con 0x7fb298160a50 2024-03-28T13:10:36.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.469+0000 7fb29dc00640 1 -- 172.21.15.19:0/2213271950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb29815fd80 con 0x7fb298160a50 2024-03-28T13:10:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.470+0000 7fb29dc00640 1 -- 172.21.15.19:0/2213271950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb29815f270 con 0x7fb298160a50 2024-03-28T13:10:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.471+0000 7fb294c00640 1 -- 172.21.15.19:0/2213271950 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb290005d10 con 0x7fb298160a50 2024-03-28T13:10:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.472+0000 7fb294c00640 1 --2- 172.21.15.19:0/2213271950 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb27406c560 0x7fb27406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.472+0000 7fb294c00640 1 -- 172.21.15.19:0/2213271950 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7fb29008bc30 con 0x7fb298160a50 2024-03-28T13:10:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.472+0000 7fb297e00640 1 --2- 172.21.15.19:0/2213271950 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb27406c560 0x7fb27406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:36.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.473+0000 7fb297e00640 1 --2- 172.21.15.19:0/2213271950 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb27406c560 0x7fb27406ea20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb288033000 tx=0x7fb288009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:36.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.476+0000 7fb294c00640 1 -- 172.21.15.19:0/2213271950 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb290025020 con 0x7fb298160a50 2024-03-28T13:10:36.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.751+0000 7fb29dc00640 1 -- 172.21.15.19:0/2213271950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb29815dd00 con 0x7fb298160a50 2024-03-28T13:10:36.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.753+0000 7fb294c00640 1 -- 172.21.15.19:0/2213271950 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16704 (secure 0 0 0) 0x7fb29815dd00 con 0x7fb298160a50 2024-03-28T13:10:36.754 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:36.754 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:10:23.142768+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":0,"to":4}]},{"pgid":"3.1b","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:10:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.756+0000 7fb29dc00640 1 -- 172.21.15.19:0/2213271950 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb27406c560 msgr2=0x7fb27406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.756+0000 7fb29dc00640 1 --2- 172.21.15.19:0/2213271950 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb27406c560 0x7fb27406ea20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb288033000 tx=0x7fb288009030 comp rx=0 tx=0).stop 2024-03-28T13:10:36.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.756+0000 7fb29dc00640 1 -- 172.21.15.19:0/2213271950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb298160a50 msgr2=0x7fb298160ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.756+0000 7fb29dc00640 1 --2- 172.21.15.19:0/2213271950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb298160a50 0x7fb298160ed0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb29806d5c0 tx=0x7fb2900115a0 comp rx=0 tx=0).stop 2024-03-28T13:10:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.756+0000 7fb29dc00640 1 -- 172.21.15.19:0/2213271950 shutdown_connections 2024-03-28T13:10:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.756+0000 7fb29dc00640 1 --2- 172.21.15.19:0/2213271950 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb27406c560 0x7fb27406ea20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.756+0000 7fb29dc00640 1 --2- 172.21.15.19:0/2213271950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb298160a50 0x7fb298160ed0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.756+0000 7fb29dc00640 1 --2- 172.21.15.19:0/2213271950 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb29814f030 0x7fb29814e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.756+0000 7fb29dc00640 1 --2- 172.21.15.19:0/2213271950 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb27c0039e0 0x7fb2980ca670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.756+0000 7fb29dc00640 1 -- 172.21.15.19:0/2213271950 >> 172.21.15.19:0/2213271950 conn(0x7fb2980676d0 msgr2=0x7fb298067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.756+0000 7fb29dc00640 1 -- 172.21.15.19:0/2213271950 shutdown_connections 2024-03-28T13:10:36.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:36.756+0000 7fb29dc00640 1 -- 172.21.15.19:0/2213271950 wait complete. 2024-03-28T13:10:36.777 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:09:39.243813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:09:40.092899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:10:36.778 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:10:37.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.010+0000 7fa359200640 1 Processor -- start 2024-03-28T13:10:37.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.010+0000 7fa359200640 1 -- start start 2024-03-28T13:10:37.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.011+0000 7fa359200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35415e4c0 0x7fa3540b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.011+0000 7fa359200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa3540b69c0 0x7fa3540b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.011+0000 7fa359200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3540d2710 con 0x7fa3540d37b0 2024-03-28T13:10:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.011+0000 7fa359200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa35415e8a0 con 0x7fa3540b69c0 2024-03-28T13:10:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.011+0000 7fa359200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa3540d4040 con 0x7fa35415e4c0 2024-03-28T13:10:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.012+0000 7fa352a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa3540d37b0 0x7fa3540b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37026/0 (socket says 172.21.15.19:37026) 2024-03-28T13:10:37.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.012+0000 7fa352a00640 1 -- 172.21.15.19:0/263044317 learned_addr learned my addr 172.21.15.19:0/263044317 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:37.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.012+0000 7fa353400640 1 --2- 172.21.15.19:0/263044317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35415e4c0 0x7fa3540b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.012+0000 7fa352000640 1 -- 172.21.15.19:0/263044317 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1207743191 0 0) 0x7fa3540d2710 con 0x7fa3540d37b0 2024-03-28T13:10:37.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.012+0000 7fa353e00640 1 --2- 172.21.15.19:0/263044317 >> v2:172.21.15.188:3300/0 conn(0x7fa3540b69c0 0x7fa3540b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.012+0000 7fa352000640 1 -- 172.21.15.19:0/263044317 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa338003520 con 0x7fa3540d37b0 2024-03-28T13:10:37.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa352000640 1 -- 172.21.15.19:0/263044317 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4136284714 0 0) 0x7fa338003520 con 0x7fa3540d37b0 2024-03-28T13:10:37.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa352000640 1 -- 172.21.15.19:0/263044317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35415e4c0 msgr2=0x7fa3540b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa352000640 1 --2- 172.21.15.19:0/263044317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35415e4c0 0x7fa3540b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa352000640 1 -- 172.21.15.19:0/263044317 >> v2:172.21.15.188:3300/0 conn(0x7fa3540b69c0 msgr2=0x7fa3540b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa352000640 1 --2- 172.21.15.19:0/263044317 >> v2:172.21.15.188:3300/0 conn(0x7fa3540b69c0 0x7fa3540b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa352000640 1 -- 172.21.15.19:0/263044317 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3540d29f0 con 0x7fa3540d37b0 2024-03-28T13:10:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa352000640 1 -- 172.21.15.19:0/263044317 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fa33c001670 con 0x7fa3540d37b0 2024-03-28T13:10:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa353e00640 1 --2- 172.21.15.19:0/263044317 >> v2:172.21.15.188:3300/0 conn(0x7fa3540b69c0 0x7fa3540b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:10:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa352000640 1 -- 172.21.15.19:0/263044317 >> v1:172.21.15.19:6789/0 conn(0x7fa3540d37b0 legacy=0x7fa3540b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa352000640 1 --2- 172.21.15.19:0/263044317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3380039b0 0x7fa338003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa352000640 1 --2- 172.21.15.19:0/263044317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa338004800 0x7fa338004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa352000640 1 --2- 172.21.15.19:0/263044317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35415e4c0 0x7fa3380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa352000640 1 -- 172.21.15.19:0/263044317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3540d4040 con 0x7fa338004800 2024-03-28T13:10:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa352000640 1 -- 172.21.15.19:0/263044317 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa33800b9f0 con 0x7fa3380039b0 2024-03-28T13:10:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa352000640 1 -- 172.21.15.19:0/263044317 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa33800bbf0 con 0x7fa35415e4c0 2024-03-28T13:10:37.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa352a00640 1 --2- 172.21.15.19:0/263044317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa338004800 0x7fa338004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa353400640 1 --2- 172.21.15.19:0/263044317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3380039b0 0x7fa338003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.013+0000 7fa353e00640 1 --2- 172.21.15.19:0/263044317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35415e4c0 0x7fa3380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.014+0000 7fa352a00640 1 -- 172.21.15.19:0/263044317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35415e4c0 msgr2=0x7fa3380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.014+0000 7fa352a00640 1 --2- 172.21.15.19:0/263044317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35415e4c0 0x7fa3380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.014+0000 7fa352a00640 1 -- 172.21.15.19:0/263044317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3380039b0 msgr2=0x7fa338003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.014+0000 7fa352a00640 1 --2- 172.21.15.19:0/263044317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3380039b0 0x7fa338003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.014+0000 7fa352a00640 1 -- 172.21.15.19:0/263044317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa338003520 con 0x7fa338004800 2024-03-28T13:10:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.014+0000 7fa353e00640 1 --2- 172.21.15.19:0/263044317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa35415e4c0 0x7fa3380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:37.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.014+0000 7fa353400640 1 --2- 172.21.15.19:0/263044317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3380039b0 0x7fa338003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:37.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.015+0000 7fa352a00640 1 --2- 172.21.15.19:0/263044317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa338004800 0x7fa338004c80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00c210 tx=0x7fa338005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4793d551570ef78 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:37.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.015+0000 7fa352000640 1 -- 172.21.15.19:0/263044317 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c007d30 con 0x7fa338004800 2024-03-28T13:10:37.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.015+0000 7fa352000640 1 -- 172.21.15.19:0/263044317 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa33c030070 con 0x7fa338004800 2024-03-28T13:10:37.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.015+0000 7fa352000640 1 -- 172.21.15.19:0/263044317 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c005d30 con 0x7fa338004800 2024-03-28T13:10:37.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.016+0000 7fa359200640 1 -- 172.21.15.19:0/263044317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa338004800 msgr2=0x7fa338004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.016+0000 7fa359200640 1 --2- 172.21.15.19:0/263044317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa338004800 0x7fa338004c80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00c210 tx=0x7fa338005830 comp rx=0 tx=0).stop 2024-03-28T13:10:37.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.016+0000 7fa353e00640 1 -- 172.21.15.19:0/263044317 reap_dead start 2024-03-28T13:10:37.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.016+0000 7fa359200640 1 -- 172.21.15.19:0/263044317 shutdown_connections 2024-03-28T13:10:37.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.016+0000 7fa359200640 1 -- 172.21.15.19:0/263044317 >> 172.21.15.19:0/263044317 conn(0x7fa3540676a0 msgr2=0x7fa354067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:37.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.016+0000 7fa359200640 1 -- 172.21.15.19:0/263044317 shutdown_connections 2024-03-28T13:10:37.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.016+0000 7fa359200640 1 -- 172.21.15.19:0/263044317 wait complete. 2024-03-28T13:10:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.018+0000 7fa359200640 1 Processor -- start 2024-03-28T13:10:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.018+0000 7fa359200640 1 -- start start 2024-03-28T13:10:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.019+0000 7fa359200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa338004800 0x7fa3541f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.019+0000 7fa359200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3541433e0 0x7fa3541f2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.019+0000 7fa359200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3541f34d0 0x7fa35415fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.019+0000 7fa359200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3540d2fb0 con 0x7fa3541433e0 2024-03-28T13:10:37.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.019+0000 7fa359200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa3540d3320 con 0x7fa3541f34d0 2024-03-28T13:10:37.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.019+0000 7fa359200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa3540d2350 con 0x7fa338004800 2024-03-28T13:10:37.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.019+0000 7fa352a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3541433e0 0x7fa3541f2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.019+0000 7fa352a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3541433e0 0x7fa3541f2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44094/0 (socket says 172.21.15.19:44094) 2024-03-28T13:10:37.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.019+0000 7fa352a00640 1 -- 172.21.15.19:0/2454041526 learned_addr learned my addr 172.21.15.19:0/2454041526 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:37.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.019+0000 7fa353400640 1 --2- 172.21.15.19:0/2454041526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa338004800 0x7fa3541f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.019+0000 7fa353e00640 1 --2- 172.21.15.19:0/2454041526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3541f34d0 0x7fa35415fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.020+0000 7fa352a00640 1 -- 172.21.15.19:0/2454041526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa338004800 msgr2=0x7fa3541f72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.020+0000 7fa352a00640 1 --2- 172.21.15.19:0/2454041526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa338004800 0x7fa3541f72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.020+0000 7fa352a00640 1 -- 172.21.15.19:0/2454041526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3541f34d0 msgr2=0x7fa35415fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.020+0000 7fa352a00640 1 --2- 172.21.15.19:0/2454041526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3541f34d0 0x7fa35415fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.020+0000 7fa352a00640 1 -- 172.21.15.19:0/2454041526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3541f3930 con 0x7fa3541433e0 2024-03-28T13:10:37.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.020+0000 7fa352a00640 1 --2- 172.21.15.19:0/2454041526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3541433e0 0x7fa3541f2f90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00c210 tx=0x7fa3540d28e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:37.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.020+0000 7fa34b400640 1 -- 172.21.15.19:0/2454041526 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c004bf0 con 0x7fa3541433e0 2024-03-28T13:10:37.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.020+0000 7fa34b400640 1 -- 172.21.15.19:0/2454041526 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa33c004d90 con 0x7fa3541433e0 2024-03-28T13:10:37.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.020+0000 7fa359200640 1 -- 172.21.15.19:0/2454041526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3540ccd20 con 0x7fa3541433e0 2024-03-28T13:10:37.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.020+0000 7fa34b400640 1 -- 172.21.15.19:0/2454041526 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c007270 con 0x7fa3541433e0 2024-03-28T13:10:37.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.020+0000 7fa359200640 1 -- 172.21.15.19:0/2454041526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3540cd2e0 con 0x7fa3541433e0 2024-03-28T13:10:37.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.022+0000 7fa359200640 1 -- 172.21.15.19:0/2454041526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa354075940 con 0x7fa3541433e0 2024-03-28T13:10:37.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.023+0000 7fa34b400640 1 -- 172.21.15.19:0/2454041526 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa33c047020 con 0x7fa3541433e0 2024-03-28T13:10:37.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.023+0000 7fa34b400640 1 --2- 172.21.15.19:0/2454041526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa33006c530 0x7fa33006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.024+0000 7fa34b400640 1 -- 172.21.15.19:0/2454041526 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7fa33c0ac980 con 0x7fa3541433e0 2024-03-28T13:10:37.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.024+0000 7fa353400640 1 --2- 172.21.15.19:0/2454041526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa33006c530 0x7fa33006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.025+0000 7fa353400640 1 --2- 172.21.15.19:0/2454041526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa33006c530 0x7fa33006e9f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa344006fd0 tx=0x7fa344008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:37.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.028+0000 7fa34b400640 1 -- 172.21.15.19:0/2454041526 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa33c07af60 con 0x7fa3541433e0 2024-03-28T13:10:37.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.230+0000 7fa359200640 1 -- 172.21.15.19:0/2454041526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa3540008d0 con 0x7fa3541433e0 2024-03-28T13:10:37.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.231+0000 7fa34b400640 1 -- 172.21.15.19:0/2454041526 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fa33c07a860 con 0x7fa3541433e0 2024-03-28T13:10:37.231 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:10:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.234+0000 7fa359200640 1 -- 172.21.15.19:0/2454041526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa33006c530 msgr2=0x7fa33006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.234+0000 7fa359200640 1 --2- 172.21.15.19:0/2454041526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa33006c530 0x7fa33006e9f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa344006fd0 tx=0x7fa344008040 comp rx=0 tx=0).stop 2024-03-28T13:10:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.234+0000 7fa359200640 1 -- 172.21.15.19:0/2454041526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3541433e0 msgr2=0x7fa3541f2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.234+0000 7fa359200640 1 --2- 172.21.15.19:0/2454041526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3541433e0 0x7fa3541f2f90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00c210 tx=0x7fa3540d28e0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.234+0000 7fa359200640 1 -- 172.21.15.19:0/2454041526 shutdown_connections 2024-03-28T13:10:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.234+0000 7fa359200640 1 --2- 172.21.15.19:0/2454041526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa33006c530 0x7fa33006e9f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.234+0000 7fa359200640 1 --2- 172.21.15.19:0/2454041526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3541f34d0 0x7fa35415fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.234+0000 7fa359200640 1 --2- 172.21.15.19:0/2454041526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3541433e0 0x7fa3541f2f90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.234+0000 7fa359200640 1 --2- 172.21.15.19:0/2454041526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa338004800 0x7fa3541f72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.234+0000 7fa359200640 1 -- 172.21.15.19:0/2454041526 >> 172.21.15.19:0/2454041526 conn(0x7fa3540676a0 msgr2=0x7fa35415c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.234+0000 7fa359200640 1 -- 172.21.15.19:0/2454041526 shutdown_connections 2024-03-28T13:10:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.234+0000 7fa359200640 1 -- 172.21.15.19:0/2454041526 wait complete. 2024-03-28T13:10:37.255 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:10:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.467+0000 7f389d800640 1 Processor -- start 2024-03-28T13:10:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.467+0000 7f389d800640 1 -- start start 2024-03-28T13:10:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.469+0000 7f389d800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3898161f60 0x7f38980d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.469+0000 7f389d800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f38980d13b0 0x7f38980d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.469+0000 7f389d800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38980b5ce0 con 0x7f38980b3170 2024-03-28T13:10:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.469+0000 7f389d800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f38980d0620 con 0x7f3898161f60 2024-03-28T13:10:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.469+0000 7f389d800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f38980d27f0 con 0x7f38980d13b0 2024-03-28T13:10:37.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.469+0000 7f3896a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3898161f60 0x7f38980d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.469+0000 7f3896a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3898161f60 0x7f38980d0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44106/0 (socket says 172.21.15.19:44106) 2024-03-28T13:10:37.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.469+0000 7f3896a00640 1 -- 172.21.15.19:0/224136480 learned_addr learned my addr 172.21.15.19:0/224136480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:37.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.470+0000 7f3897400640 1 --2- 172.21.15.19:0/224136480 >> v2:172.21.15.188:3300/0 conn(0x7f38980d13b0 0x7f38980d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.470+0000 7f3896a00640 1 -- 172.21.15.19:0/224136480 >> v2:172.21.15.188:3300/0 conn(0x7f38980d13b0 msgr2=0x7f38980d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:37.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.470+0000 7f3896a00640 1 --2- 172.21.15.19:0/224136480 >> v2:172.21.15.188:3300/0 conn(0x7f38980d13b0 0x7f38980d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.470+0000 7f3896a00640 1 -- 172.21.15.19:0/224136480 >> v1:172.21.15.188:6789/0 conn(0x7f38980b3170 legacy=0x7f38980b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.470+0000 7f3896a00640 1 -- 172.21.15.19:0/224136480 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38980d2a90 con 0x7f3898161f60 2024-03-28T13:10:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.473+0000 7f3896a00640 1 --2- 172.21.15.19:0/224136480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3898161f60 0x7f38980d0e70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f389808cd50 tx=0x7f388802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a17aee369650ce server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.473+0000 7f3896000640 1 -- 172.21.15.19:0/224136480 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888034070 con 0x7f3898161f60 2024-03-28T13:10:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.473+0000 7f3896000640 1 -- 172.21.15.19:0/224136480 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3888030860 con 0x7f3898161f60 2024-03-28T13:10:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.473+0000 7f3896000640 1 -- 172.21.15.19:0/224136480 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888030c30 con 0x7f3898161f60 2024-03-28T13:10:37.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.474+0000 7f389d800640 1 -- 172.21.15.19:0/224136480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3898161f60 msgr2=0x7f38980d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.474+0000 7f389d800640 1 --2- 172.21.15.19:0/224136480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3898161f60 0x7f38980d0e70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f389808cd50 tx=0x7f388802b9a0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.474+0000 7f389d800640 1 -- 172.21.15.19:0/224136480 shutdown_connections 2024-03-28T13:10:37.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.474+0000 7f389d800640 1 --2- 172.21.15.19:0/224136480 >> v2:172.21.15.188:3300/0 conn(0x7f38980d13b0 0x7f38980d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.474+0000 7f389d800640 1 --2- 172.21.15.19:0/224136480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3898161f60 0x7f38980d0e70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.474+0000 7f389d800640 1 -- 172.21.15.19:0/224136480 >> 172.21.15.19:0/224136480 conn(0x7f389808bfb0 msgr2=0x7f38980d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:37.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.474+0000 7f389d800640 1 -- 172.21.15.19:0/224136480 shutdown_connections 2024-03-28T13:10:37.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.475+0000 7f389d800640 1 -- 172.21.15.19:0/224136480 wait complete. 2024-03-28T13:10:37.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.477+0000 7f389d800640 1 Processor -- start 2024-03-28T13:10:37.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.477+0000 7f389d800640 1 -- start start 2024-03-28T13:10:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.478+0000 7f389d800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38980b3170 0x7f3898151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.478+0000 7f389d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38980d13b0 0x7f38980ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.478+0000 7f389d800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3898161f60 0x7f38980c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.478+0000 7f389d800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f38980d3030 con 0x7f38980d13b0 2024-03-28T13:10:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.478+0000 7f389d800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f38980d2d50 con 0x7f3898161f60 2024-03-28T13:10:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.478+0000 7f389d800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f38980d3310 con 0x7f38980b3170 2024-03-28T13:10:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.478+0000 7f388fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38980d13b0 0x7f38980ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.478+0000 7f388fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38980d13b0 0x7f38980ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44110/0 (socket says 172.21.15.19:44110) 2024-03-28T13:10:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.478+0000 7f388fe00640 1 -- 172.21.15.19:0/1401079944 learned_addr learned my addr 172.21.15.19:0/1401079944 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.478+0000 7f3897400640 1 --2- 172.21.15.19:0/1401079944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3898161f60 0x7f38980c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.478+0000 7f3896a00640 1 --2- 172.21.15.19:0/1401079944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38980b3170 0x7f3898151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.479+0000 7f388fe00640 1 -- 172.21.15.19:0/1401079944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38980b3170 msgr2=0x7f3898151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.479+0000 7f388fe00640 1 --2- 172.21.15.19:0/1401079944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38980b3170 0x7f3898151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.479+0000 7f388fe00640 1 -- 172.21.15.19:0/1401079944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3898161f60 msgr2=0x7f38980c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.479+0000 7f388fe00640 1 --2- 172.21.15.19:0/1401079944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3898161f60 0x7f38980c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.479+0000 7f388fe00640 1 -- 172.21.15.19:0/1401079944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38980b7bc0 con 0x7f38980d13b0 2024-03-28T13:10:37.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.479+0000 7f3897400640 1 --2- 172.21.15.19:0/1401079944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3898161f60 0x7f38980c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:10:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.479+0000 7f388fe00640 1 --2- 172.21.15.19:0/1401079944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38980d13b0 0x7f38980ce060 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f388000c3c0 tx=0x7f388000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.480+0000 7f388ea00640 1 -- 172.21.15.19:0/1401079944 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38800052d0 con 0x7f38980d13b0 2024-03-28T13:10:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.480+0000 7f388ea00640 1 -- 172.21.15.19:0/1401079944 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3880005490 con 0x7f38980d13b0 2024-03-28T13:10:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.480+0000 7f388ea00640 1 -- 172.21.15.19:0/1401079944 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3880007260 con 0x7f38980d13b0 2024-03-28T13:10:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.480+0000 7f389d800640 1 -- 172.21.15.19:0/1401079944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38980cb7a0 con 0x7f38980d13b0 2024-03-28T13:10:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.480+0000 7f389d800640 1 -- 172.21.15.19:0/1401079944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38980b7800 con 0x7f38980d13b0 2024-03-28T13:10:37.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.481+0000 7f389d800640 1 -- 172.21.15.19:0/1401079944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38980685c0 con 0x7f38980d13b0 2024-03-28T13:10:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.482+0000 7f388ea00640 1 -- 172.21.15.19:0/1401079944 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3880007400 con 0x7f38980d13b0 2024-03-28T13:10:37.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.483+0000 7f388ea00640 1 --2- 172.21.15.19:0/1401079944 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f387406c560 0x7f387406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.483+0000 7f388ea00640 1 -- 172.21.15.19:0/1401079944 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f388008abb0 con 0x7f38980d13b0 2024-03-28T13:10:37.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.484+0000 7f3896a00640 1 --2- 172.21.15.19:0/1401079944 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f387406c560 0x7f387406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.485+0000 7f3896a00640 1 --2- 172.21.15.19:0/1401079944 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f387406c560 0x7f387406ea20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3888036000 tx=0x7f388803b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:37.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.490+0000 7f388ea00640 1 -- 172.21.15.19:0/1401079944 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f38980b5ce0 con 0x7f38980d13b0 2024-03-28T13:10:37.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.697+0000 7f389d800640 1 -- 172.21.15.19:0/1401079944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f38980cb570 con 0x7f38980d13b0 2024-03-28T13:10:37.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.698+0000 7f388ea00640 1 -- 172.21.15.19:0/1401079944 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f3880059190 con 0x7f38980d13b0 2024-03-28T13:10:37.698 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:10:37.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.701+0000 7f389d800640 1 -- 172.21.15.19:0/1401079944 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f387406c560 msgr2=0x7f387406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.701+0000 7f389d800640 1 --2- 172.21.15.19:0/1401079944 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f387406c560 0x7f387406ea20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3888036000 tx=0x7f388803b040 comp rx=0 tx=0).stop 2024-03-28T13:10:37.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.701+0000 7f389d800640 1 -- 172.21.15.19:0/1401079944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38980d13b0 msgr2=0x7f38980ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.701+0000 7f389d800640 1 --2- 172.21.15.19:0/1401079944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38980d13b0 0x7f38980ce060 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f388000c3c0 tx=0x7f388000c880 comp rx=0 tx=0).stop 2024-03-28T13:10:37.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.701+0000 7f389d800640 1 -- 172.21.15.19:0/1401079944 shutdown_connections 2024-03-28T13:10:37.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.701+0000 7f389d800640 1 --2- 172.21.15.19:0/1401079944 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f387406c560 0x7f387406ea20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.701+0000 7f389d800640 1 --2- 172.21.15.19:0/1401079944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3898161f60 0x7f38980c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.701+0000 7f389d800640 1 --2- 172.21.15.19:0/1401079944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38980d13b0 0x7f38980ce060 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.701+0000 7f389d800640 1 --2- 172.21.15.19:0/1401079944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f38980b3170 0x7f3898151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.701+0000 7f389d800640 1 -- 172.21.15.19:0/1401079944 >> 172.21.15.19:0/1401079944 conn(0x7f389808bfb0 msgr2=0x7f389806d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:37.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.701+0000 7f389d800640 1 -- 172.21.15.19:0/1401079944 shutdown_connections 2024-03-28T13:10:37.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.701+0000 7f389d800640 1 -- 172.21.15.19:0/1401079944 wait complete. 2024-03-28T13:10:37.720 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:10:37.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.942+0000 7f049f200640 1 Processor -- start 2024-03-28T13:10:37.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.942+0000 7f049f200640 1 -- start start 2024-03-28T13:10:37.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.943+0000 7f049f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f04a006dd30 0x7f04a00cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.944+0000 7f049f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04a00ce890 con 0x7f04a00cc880 2024-03-28T13:10:37.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.944+0000 7f049f200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04a00ceb50 con 0x7f04a013f190 2024-03-28T13:10:37.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.944+0000 7f049f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f04a00cd230 con 0x7f04a006dd30 2024-03-28T13:10:37.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.944+0000 7f049e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f04a006dd30 0x7f04a00cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.944+0000 7f049d400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f04a00cc880 0x7f04a006d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37036/0 (socket says 172.21.15.19:37036) 2024-03-28T13:10:37.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.944+0000 7f049d400640 1 -- 172.21.15.19:0/3780592459 learned_addr learned my addr 172.21.15.19:0/3780592459 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:37.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.944+0000 7f049e800640 1 -- 172.21.15.19:0/3780592459 >> v1:172.21.15.188:6789/0 conn(0x7f04a013f190 legacy=0x7f04a00cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.944+0000 7f049e800640 1 -- 172.21.15.19:0/3780592459 >> v1:172.21.15.19:6789/0 conn(0x7f04a00cc880 legacy=0x7f04a006d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.945+0000 7f049ca00640 1 -- 172.21.15.19:0/3780592459 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1996590394 0 0) 0x7f04a00ce890 con 0x7f04a00cc880 2024-03-28T13:10:37.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.945+0000 7f049e800640 1 -- 172.21.15.19:0/3780592459 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a00cee30 con 0x7f04a006dd30 2024-03-28T13:10:37.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.946+0000 7f049e800640 1 --2- 172.21.15.19:0/3780592459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a006dd30 0x7f04a00cccf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0498009d90 tx=0x7f049802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d9036207787631 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:37.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.946+0000 7f049ca00640 1 -- 172.21.15.19:0/3780592459 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498039070 con 0x7f04a006dd30 2024-03-28T13:10:37.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.946+0000 7f049ca00640 1 -- 172.21.15.19:0/3780592459 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f049802cd00 con 0x7f04a006dd30 2024-03-28T13:10:37.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.946+0000 7f049ca00640 1 -- 172.21.15.19:0/3780592459 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498034660 con 0x7f04a006dd30 2024-03-28T13:10:37.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.947+0000 7f049f200640 1 -- 172.21.15.19:0/3780592459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a006dd30 msgr2=0x7f04a00cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.947+0000 7f049f200640 1 --2- 172.21.15.19:0/3780592459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a006dd30 0x7f04a00cccf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0498009d90 tx=0x7f049802bf50 comp rx=0 tx=0).stop 2024-03-28T13:10:37.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.947+0000 7f049f200640 1 -- 172.21.15.19:0/3780592459 shutdown_connections 2024-03-28T13:10:37.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.947+0000 7f049f200640 1 --2- 172.21.15.19:0/3780592459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a006dd30 0x7f04a00cccf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.947+0000 7f049f200640 1 -- 172.21.15.19:0/3780592459 >> 172.21.15.19:0/3780592459 conn(0x7f04a00b1ba0 msgr2=0x7f04a00b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:37.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.947+0000 7f049f200640 1 -- 172.21.15.19:0/3780592459 shutdown_connections 2024-03-28T13:10:37.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.947+0000 7f049f200640 1 -- 172.21.15.19:0/3780592459 wait complete. 2024-03-28T13:10:37.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.951+0000 7f049f200640 1 Processor -- start 2024-03-28T13:10:37.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.952+0000 7f049f200640 1 -- start start 2024-03-28T13:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.954+0000 7f049f200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04a006dd30 0x7f04a015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.954+0000 7f049f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a00cc880 0x7f04a013b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.954+0000 7f049f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04a013f190 0x7f04a00aa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.954+0000 7f049f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04a013f570 con 0x7f04a00cc880 2024-03-28T13:10:37.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.954+0000 7f049f200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04a0160c70 con 0x7f04a006dd30 2024-03-28T13:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.954+0000 7f049f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f04a015ff10 con 0x7f04a013f190 2024-03-28T13:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.954+0000 7f049d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a00cc880 0x7f04a013b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.954+0000 7f049d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a00cc880 0x7f04a013b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44134/0 (socket says 172.21.15.19:44134) 2024-03-28T13:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.954+0000 7f049d400640 1 -- 172.21.15.19:0/2106945163 learned_addr learned my addr 172.21.15.19:0/2106945163 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.955+0000 7f049de00640 1 --2- 172.21.15.19:0/2106945163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04a006dd30 0x7f04a015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.955+0000 7f049e800640 1 --2- 172.21.15.19:0/2106945163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04a013f190 0x7f04a00aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.955+0000 7f049d400640 1 -- 172.21.15.19:0/2106945163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04a013f190 msgr2=0x7f04a00aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.955+0000 7f049d400640 1 --2- 172.21.15.19:0/2106945163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04a013f190 0x7f04a00aa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.955+0000 7f049d400640 1 -- 172.21.15.19:0/2106945163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04a006dd30 msgr2=0x7f04a015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.955+0000 7f049d400640 1 --2- 172.21.15.19:0/2106945163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04a006dd30 0x7f04a015baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.955+0000 7f049d400640 1 -- 172.21.15.19:0/2106945163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a013bb30 con 0x7f04a00cc880 2024-03-28T13:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.955+0000 7f049e800640 1 --2- 172.21.15.19:0/2106945163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04a013f190 0x7f04a00aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:37.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.955+0000 7f049d400640 1 --2- 172.21.15.19:0/2106945163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a00cc880 0x7f04a013b5f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f048c00c4a0 tx=0x7f048c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:37.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.955+0000 7f0496000640 1 -- 172.21.15.19:0/2106945163 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c018070 con 0x7f04a00cc880 2024-03-28T13:10:37.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.956+0000 7f0496000640 1 -- 172.21.15.19:0/2106945163 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048c0074e0 con 0x7f04a00cc880 2024-03-28T13:10:37.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.956+0000 7f0496000640 1 -- 172.21.15.19:0/2106945163 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c004a20 con 0x7f04a00cc880 2024-03-28T13:10:37.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.956+0000 7f049f200640 1 -- 172.21.15.19:0/2106945163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a00a9480 con 0x7f04a00cc880 2024-03-28T13:10:37.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.956+0000 7f049f200640 1 -- 172.21.15.19:0/2106945163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a00a9a40 con 0x7f04a00cc880 2024-03-28T13:10:37.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.957+0000 7f049f200640 1 -- 172.21.15.19:0/2106945163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a0075940 con 0x7f04a00cc880 2024-03-28T13:10:37.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.958+0000 7f0496000640 1 -- 172.21.15.19:0/2106945163 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f048c01b070 con 0x7f04a00cc880 2024-03-28T13:10:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.959+0000 7f0496000640 1 --2- 172.21.15.19:0/2106945163 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f048006c560 0x7f048006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:37.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.960+0000 7f0496000640 1 -- 172.21.15.19:0/2106945163 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f048c08aad0 con 0x7f04a00cc880 2024-03-28T13:10:37.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.960+0000 7f049de00640 1 --2- 172.21.15.19:0/2106945163 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f048006c560 0x7f048006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:37.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.961+0000 7f049de00640 1 --2- 172.21.15.19:0/2106945163 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f048006c560 0x7f048006ea20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0490000f60 tx=0x7f0490009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:37.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:37.964+0000 7f0496000640 1 -- 172.21.15.19:0/2106945163 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f048c059010 con 0x7f04a00cc880 2024-03-28T13:10:38.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.165+0000 7f049f200640 1 -- 172.21.15.19:0/2106945163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f04a0073850 con 0x7f04a00cc880 2024-03-28T13:10:38.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.166+0000 7f0496000640 1 -- 172.21.15.19:0/2106945163 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f048c058910 con 0x7f04a00cc880 2024-03-28T13:10:38.166 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:10:38.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.169+0000 7f049f200640 1 -- 172.21.15.19:0/2106945163 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f048006c560 msgr2=0x7f048006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.169+0000 7f049f200640 1 --2- 172.21.15.19:0/2106945163 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f048006c560 0x7f048006ea20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0490000f60 tx=0x7f0490009040 comp rx=0 tx=0).stop 2024-03-28T13:10:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.169+0000 7f049f200640 1 -- 172.21.15.19:0/2106945163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a00cc880 msgr2=0x7f04a013b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.169+0000 7f049f200640 1 --2- 172.21.15.19:0/2106945163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a00cc880 0x7f04a013b5f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f048c00c4a0 tx=0x7f048c00c960 comp rx=0 tx=0).stop 2024-03-28T13:10:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.169+0000 7f049f200640 1 -- 172.21.15.19:0/2106945163 shutdown_connections 2024-03-28T13:10:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.169+0000 7f049f200640 1 --2- 172.21.15.19:0/2106945163 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f048006c560 0x7f048006ea20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.169+0000 7f049f200640 1 --2- 172.21.15.19:0/2106945163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04a013f190 0x7f04a00aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.169+0000 7f049f200640 1 --2- 172.21.15.19:0/2106945163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04a00cc880 0x7f04a013b5f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.169+0000 7f049f200640 1 --2- 172.21.15.19:0/2106945163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04a006dd30 0x7f04a015baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.169+0000 7f049f200640 1 -- 172.21.15.19:0/2106945163 >> 172.21.15.19:0/2106945163 conn(0x7f04a00b1ba0 msgr2=0x7f04a006d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.169+0000 7f049f200640 1 -- 172.21.15.19:0/2106945163 shutdown_connections 2024-03-28T13:10:38.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.169+0000 7f049f200640 1 -- 172.21.15.19:0/2106945163 wait complete. 2024-03-28T13:10:38.190 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:10:38.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.402+0000 7f5333e00640 1 Processor -- start 2024-03-28T13:10:38.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.402+0000 7f5333e00640 1 -- start start 2024-03-28T13:10:38.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.404+0000 7f5333e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f533415e440 0x7f53340b9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:38.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.404+0000 7f5333e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f53340d3e20 0x7f53340d2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:38.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.404+0000 7f5333e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53340b8300 con 0x7f53340cef00 2024-03-28T13:10:38.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.404+0000 7f5333e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f53340d2d80 con 0x7f53340d3e20 2024-03-28T13:10:38.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.404+0000 7f5333e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f53340d3060 con 0x7f533415e440 2024-03-28T13:10:38.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.404+0000 7f532a000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f53340d3e20 0x7f53340d2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:38.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.404+0000 7f532a000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f53340d3e20 0x7f53340d2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44142/0 (socket says 172.21.15.19:44142) 2024-03-28T13:10:38.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.404+0000 7f532a000640 1 -- 172.21.15.19:0/3824139596 learned_addr learned my addr 172.21.15.19:0/3824139596 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:38.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.405+0000 7f5332a00640 1 --2- 172.21.15.19:0/3824139596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f533415e440 0x7f53340b9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:38.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.405+0000 7f532a000640 1 -- 172.21.15.19:0/3824139596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f533415e440 msgr2=0x7f53340b9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:38.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.405+0000 7f532a000640 1 --2- 172.21.15.19:0/3824139596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f533415e440 0x7f53340b9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:38.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.405+0000 7f532a000640 1 -- 172.21.15.19:0/3824139596 >> v1:172.21.15.188:6789/0 conn(0x7f53340cef00 legacy=0x7f533406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:38.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.405+0000 7f532a000640 1 -- 172.21.15.19:0/3824139596 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53340d3340 con 0x7f53340d3e20 2024-03-28T13:10:38.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.406+0000 7f532a000640 1 --2- 172.21.15.19:0/3824139596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53340d3e20 0x7f53340d2530 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f531c009870 tx=0x7f531c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ef3419ccd1c452 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:38.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.407+0000 7f5332000640 1 -- 172.21.15.19:0/3824139596 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c034070 con 0x7f53340d3e20 2024-03-28T13:10:38.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.407+0000 7f5332000640 1 -- 172.21.15.19:0/3824139596 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f531c030840 con 0x7f53340d3e20 2024-03-28T13:10:38.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.407+0000 7f5332000640 1 -- 172.21.15.19:0/3824139596 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c030c10 con 0x7f53340d3e20 2024-03-28T13:10:38.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.407+0000 7f5333e00640 1 -- 172.21.15.19:0/3824139596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53340d3e20 msgr2=0x7f53340d2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:38.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.407+0000 7f5333e00640 1 --2- 172.21.15.19:0/3824139596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53340d3e20 0x7f53340d2530 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f531c009870 tx=0x7f531c02b980 comp rx=0 tx=0).stop 2024-03-28T13:10:38.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.407+0000 7f5333e00640 1 -- 172.21.15.19:0/3824139596 shutdown_connections 2024-03-28T13:10:38.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.407+0000 7f5333e00640 1 --2- 172.21.15.19:0/3824139596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53340d3e20 0x7f53340d2530 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:38.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.407+0000 7f5333e00640 1 --2- 172.21.15.19:0/3824139596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f533415e440 0x7f53340b9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:38.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.407+0000 7f5333e00640 1 -- 172.21.15.19:0/3824139596 >> 172.21.15.19:0/3824139596 conn(0x7f53340678b0 msgr2=0x7f53340d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:38.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.408+0000 7f5333e00640 1 -- 172.21.15.19:0/3824139596 shutdown_connections 2024-03-28T13:10:38.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.408+0000 7f5333e00640 1 -- 172.21.15.19:0/3824139596 wait complete. 2024-03-28T13:10:38.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.410+0000 7f5333e00640 1 Processor -- start 2024-03-28T13:10:38.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.410+0000 7f5333e00640 1 -- start start 2024-03-28T13:10:38.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.412+0000 7f5333e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53340cef00 0x7f53340cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:38.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.412+0000 7f5333e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53340d3e20 0x7f5334150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:38.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.412+0000 7f5333e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f533415e440 0x7f53340cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:38.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.412+0000 7f5333e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f53340d2a70 con 0x7f53340cef00 2024-03-28T13:10:38.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.412+0000 7f5333e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53340d3900 con 0x7f533415e440 2024-03-28T13:10:38.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.412+0000 7f5333e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f53340cf390 con 0x7f53340d3e20 2024-03-28T13:10:38.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.412+0000 7f5332a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53340cef00 0x7f53340cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:38.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.412+0000 7f5332a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53340cef00 0x7f53340cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44144/0 (socket says 172.21.15.19:44144) 2024-03-28T13:10:38.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.412+0000 7f5332a00640 1 -- 172.21.15.19:0/1162215171 learned_addr learned my addr 172.21.15.19:0/1162215171 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:38.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.413+0000 7f532a000640 1 --2- 172.21.15.19:0/1162215171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53340d3e20 0x7f5334150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:38.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.413+0000 7f5332a00640 1 -- 172.21.15.19:0/1162215171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53340d3e20 msgr2=0x7f5334150920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:38.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.413+0000 7f5332a00640 1 --2- 172.21.15.19:0/1162215171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53340d3e20 0x7f5334150920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:38.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.413+0000 7f5332a00640 1 -- 172.21.15.19:0/1162215171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f533415e440 msgr2=0x7f53340cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:38.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.413+0000 7f5333400640 1 --2- 172.21.15.19:0/1162215171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f533415e440 0x7f53340cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:38.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.413+0000 7f5332a00640 1 --2- 172.21.15.19:0/1162215171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f533415e440 0x7f53340cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:38.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.413+0000 7f5332a00640 1 -- 172.21.15.19:0/1162215171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53340cbb50 con 0x7f53340cef00 2024-03-28T13:10:38.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.413+0000 7f5333400640 1 --2- 172.21.15.19:0/1162215171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f533415e440 0x7f53340cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:38.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.413+0000 7f5332a00640 1 --2- 172.21.15.19:0/1162215171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53340cef00 0x7f53340cb300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5324007e60 tx=0x7f532400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:38.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.413+0000 7f532be00640 1 -- 172.21.15.19:0/1162215171 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5324014620 con 0x7f53340cef00 2024-03-28T13:10:38.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.413+0000 7f532be00640 1 -- 172.21.15.19:0/1162215171 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5324019070 con 0x7f53340cef00 2024-03-28T13:10:38.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.413+0000 7f532be00640 1 -- 172.21.15.19:0/1162215171 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5324014a20 con 0x7f53340cef00 2024-03-28T13:10:38.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.413+0000 7f5333e00640 1 -- 172.21.15.19:0/1162215171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53340cbe30 con 0x7f53340cef00 2024-03-28T13:10:38.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.414+0000 7f5333e00640 1 -- 172.21.15.19:0/1162215171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53340cc3f0 con 0x7f53340cef00 2024-03-28T13:10:38.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.415+0000 7f5333e00640 1 -- 172.21.15.19:0/1162215171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5334075940 con 0x7f53340cef00 2024-03-28T13:10:38.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.416+0000 7f532be00640 1 -- 172.21.15.19:0/1162215171 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5324014bc0 con 0x7f53340cef00 2024-03-28T13:10:38.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.417+0000 7f532be00640 1 --2- 172.21.15.19:0/1162215171 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f531006c560 0x7f531006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:38.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.417+0000 7f532be00640 1 -- 172.21.15.19:0/1162215171 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f532408bfc0 con 0x7f53340cef00 2024-03-28T13:10:38.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.417+0000 7f532a000640 1 --2- 172.21.15.19:0/1162215171 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f531006c560 0x7f531006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:38.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.418+0000 7f532a000640 1 --2- 172.21.15.19:0/1162215171 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f531006c560 0x7f531006ea20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f531c036000 tx=0x7f531c03b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:38.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.421+0000 7f532be00640 1 -- 172.21.15.19:0/1162215171 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f532405a590 con 0x7f53340cef00 2024-03-28T13:10:38.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.738+0000 7f5333e00640 1 -- 172.21.15.19:0/1162215171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f53340cb840 con 0x7f53340cef00 2024-03-28T13:10:38.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.739+0000 7f532be00640 1 -- 172.21.15.19:0/1162215171 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f5324059e90 con 0x7f53340cef00 2024-03-28T13:10:38.739 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-03-28T13:10:38.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.741+0000 7f5333e00640 1 -- 172.21.15.19:0/1162215171 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f531006c560 msgr2=0x7f531006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:38.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.741+0000 7f5333e00640 1 --2- 172.21.15.19:0/1162215171 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f531006c560 0x7f531006ea20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f531c036000 tx=0x7f531c03b040 comp rx=0 tx=0).stop 2024-03-28T13:10:38.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.741+0000 7f5333e00640 1 -- 172.21.15.19:0/1162215171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53340cef00 msgr2=0x7f53340cb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:38.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.741+0000 7f5333e00640 1 --2- 172.21.15.19:0/1162215171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53340cef00 0x7f53340cb300 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5324007e60 tx=0x7f532400e4b0 comp rx=0 tx=0).stop 2024-03-28T13:10:38.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.741+0000 7f5333e00640 1 -- 172.21.15.19:0/1162215171 shutdown_connections 2024-03-28T13:10:38.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.741+0000 7f5333e00640 1 --2- 172.21.15.19:0/1162215171 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f531006c560 0x7f531006ea20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.741+0000 7f5333e00640 1 --2- 172.21.15.19:0/1162215171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f533415e440 0x7f53340cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.741+0000 7f5333e00640 1 --2- 172.21.15.19:0/1162215171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53340d3e20 0x7f5334150920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.741+0000 7f5333e00640 1 --2- 172.21.15.19:0/1162215171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53340cef00 0x7f53340cb300 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.741+0000 7f5333e00640 1 -- 172.21.15.19:0/1162215171 >> 172.21.15.19:0/1162215171 conn(0x7f53340678b0 msgr2=0x7f53340ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.742+0000 7f5333e00640 1 -- 172.21.15.19:0/1162215171 shutdown_connections 2024-03-28T13:10:38.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:38.742+0000 7f5333e00640 1 -- 172.21.15.19:0/1162215171 wait complete. 2024-03-28T13:10:38.759 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-03-28T13:10:38.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:38.759 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-03-28T13:10:38.863 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-03-28T13:10:38.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:10:38.864 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-28T13:10:38.914 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-28T13:10:38.915 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:10:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.132+0000 7f4ed4a00640 1 Processor -- start 2024-03-28T13:10:39.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.133+0000 7f4ed4a00640 1 -- start start 2024-03-28T13:10:39.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.134+0000 7f4ed4a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4ed013f100 0x7f4ed006fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:39.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.134+0000 7f4ed4a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ed00704a0 con 0x7f4ed0070080 2024-03-28T13:10:39.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.134+0000 7f4ed4a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ed00d2ed0 con 0x7f4ed00d28b0 2024-03-28T13:10:39.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.134+0000 7f4ed4a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f4ed0142270 con 0x7f4ed013f100 2024-03-28T13:10:39.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.134+0000 7f4ec5e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4ed0070080 0x7f4ed00d21a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37052/0 (socket says 172.21.15.19:37052) 2024-03-28T13:10:39.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.134+0000 7f4ec5e00640 1 -- 172.21.15.19:0/3384216134 learned_addr learned my addr 172.21.15.19:0/3384216134 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ece800640 1 --2- 172.21.15.19:0/3384216134 >> v2:172.21.15.188:3300/0 conn(0x7f4ed013f100 0x7f4ed006fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ecde00640 1 -- 172.21.15.19:0/3384216134 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3939329150 0 0) 0x7f4ed00704a0 con 0x7f4ed0070080 2024-03-28T13:10:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ecde00640 1 -- 172.21.15.19:0/3384216134 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4eb4003520 con 0x7f4ed0070080 2024-03-28T13:10:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ecde00640 1 -- 172.21.15.19:0/3384216134 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3391878220 0 0) 0x7f4eb4003520 con 0x7f4ed0070080 2024-03-28T13:10:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ecde00640 1 -- 172.21.15.19:0/3384216134 >> v2:172.21.15.188:3300/0 conn(0x7f4ed013f100 msgr2=0x7f4ed006fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ecde00640 1 --2- 172.21.15.19:0/3384216134 >> v2:172.21.15.188:3300/0 conn(0x7f4ed013f100 0x7f4ed006fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ecde00640 1 -- 172.21.15.19:0/3384216134 >> v1:172.21.15.188:6789/0 conn(0x7f4ed00d28b0 legacy=0x7f4ed00d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ecde00640 1 -- 172.21.15.19:0/3384216134 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed013f4e0 con 0x7f4ed0070080 2024-03-28T13:10:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ecde00640 1 -- 172.21.15.19:0/3384216134 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f4eb8001670 con 0x7f4ed0070080 2024-03-28T13:10:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ecde00640 1 -- 172.21.15.19:0/3384216134 >> v1:172.21.15.19:6789/0 conn(0x7f4ed0070080 legacy=0x7f4ed00d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ecde00640 1 --2- 172.21.15.19:0/3384216134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4003a00 0x7f4eb4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ecde00640 1 --2- 172.21.15.19:0/3384216134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb4004800 0x7f4eb4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ecde00640 1 --2- 172.21.15.19:0/3384216134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eb4005210 0x7f4eb400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ecde00640 1 -- 172.21.15.19:0/3384216134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed0142270 con 0x7f4eb4003a00 2024-03-28T13:10:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ecde00640 1 -- 172.21.15.19:0/3384216134 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb400c160 con 0x7f4eb4004800 2024-03-28T13:10:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ecde00640 1 -- 172.21.15.19:0/3384216134 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb400c360 con 0x7f4eb4005210 2024-03-28T13:10:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.135+0000 7f4ece800640 1 --2- 172.21.15.19:0/3384216134 >> v2:172.21.15.188:3300/0 conn(0x7f4ed013f100 0x7f4ed006fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:10:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.136+0000 7f4ece800640 1 --2- 172.21.15.19:0/3384216134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4003a00 0x7f4eb4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.136+0000 7f4ec5e00640 1 --2- 172.21.15.19:0/3384216134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb4004800 0x7f4eb4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.136+0000 7f4ece800640 1 -- 172.21.15.19:0/3384216134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eb4005210 msgr2=0x7f4eb400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.136+0000 7f4ece800640 1 --2- 172.21.15.19:0/3384216134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4eb4005210 0x7f4eb400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.136+0000 7f4ece800640 1 -- 172.21.15.19:0/3384216134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb4004800 msgr2=0x7f4eb4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.136+0000 7f4ece800640 1 --2- 172.21.15.19:0/3384216134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb4004800 0x7f4eb4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.136+0000 7f4ece800640 1 -- 172.21.15.19:0/3384216134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb4003520 con 0x7f4eb4003a00 2024-03-28T13:10:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.136+0000 7f4ecf200640 1 -- 172.21.15.19:0/3384216134 reap_dead start 2024-03-28T13:10:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.137+0000 7f4ece800640 1 --2- 172.21.15.19:0/3384216134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4003a00 0x7f4eb4003e20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00098a0 tx=0x7f4eb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68fc3afd6b56fec server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.137+0000 7f4ecde00640 1 -- 172.21.15.19:0/3384216134 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0034070 con 0x7f4eb4003a00 2024-03-28T13:10:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.137+0000 7f4ecde00640 1 -- 172.21.15.19:0/3384216134 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ec0030660 con 0x7f4eb4003a00 2024-03-28T13:10:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.137+0000 7f4ecde00640 1 -- 172.21.15.19:0/3384216134 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec000f040 con 0x7f4eb4003a00 2024-03-28T13:10:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.138+0000 7f4ed4a00640 1 -- 172.21.15.19:0/3384216134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4003a00 msgr2=0x7f4eb4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.138+0000 7f4ed4a00640 1 --2- 172.21.15.19:0/3384216134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4003a00 0x7f4eb4003e20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00098a0 tx=0x7f4eb4005830 comp rx=0 tx=0).stop 2024-03-28T13:10:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.138+0000 7f4ed4a00640 1 -- 172.21.15.19:0/3384216134 shutdown_connections 2024-03-28T13:10:39.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.138+0000 7f4ed4a00640 1 --2- 172.21.15.19:0/3384216134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4eb4003a00 0x7f4eb4003e20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.138+0000 7f4ed4a00640 1 -- 172.21.15.19:0/3384216134 >> 172.21.15.19:0/3384216134 conn(0x7f4ed0067460 msgr2=0x7f4ed0067840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:39.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.138+0000 7f4ed4a00640 1 -- 172.21.15.19:0/3384216134 shutdown_connections 2024-03-28T13:10:39.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.138+0000 7f4ed4a00640 1 -- 172.21.15.19:0/3384216134 wait complete. 2024-03-28T13:10:39.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.140+0000 7f4ed4a00640 1 Processor -- start 2024-03-28T13:10:39.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.140+0000 7f4ed4a00640 1 -- start start 2024-03-28T13:10:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.141+0000 7f4ed4a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb4003a00 0x7f4ed006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.141+0000 7f4ed4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ed0155ec0 0x7f4ed0154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.141+0000 7f4ed4a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ed006c680 0x7f4ed015b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.141+0000 7f4ed4a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed0141f00 con 0x7f4ed0155ec0 2024-03-28T13:10:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.141+0000 7f4ed4a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed0141c20 con 0x7f4eb4003a00 2024-03-28T13:10:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.141+0000 7f4ed4a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4ed0142700 con 0x7f4ed006c680 2024-03-28T13:10:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.141+0000 7f4ec5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ed0155ec0 0x7f4ed0154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.141+0000 7f4ec5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ed0155ec0 0x7f4ed0154820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44168/0 (socket says 172.21.15.19:44168) 2024-03-28T13:10:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.141+0000 7f4ec5e00640 1 -- 172.21.15.19:0/41795373 learned_addr learned my addr 172.21.15.19:0/41795373 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.142+0000 7f4ece800640 1 --2- 172.21.15.19:0/41795373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb4003a00 0x7f4ed006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.142+0000 7f4ec5e00640 1 -- 172.21.15.19:0/41795373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ed006c680 msgr2=0x7f4ed015b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.142+0000 7f4ec5e00640 1 --2- 172.21.15.19:0/41795373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ed006c680 0x7f4ed015b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.142+0000 7f4ec5e00640 1 -- 172.21.15.19:0/41795373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb4003a00 msgr2=0x7f4ed006d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.142+0000 7f4ec5e00640 1 --2- 172.21.15.19:0/41795373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb4003a00 0x7f4ed006d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.142+0000 7f4ec5e00640 1 -- 172.21.15.19:0/41795373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed0155070 con 0x7f4ed0155ec0 2024-03-28T13:10:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.142+0000 7f4ece800640 1 --2- 172.21.15.19:0/41795373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb4003a00 0x7f4ed006d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.142+0000 7f4ec5e00640 1 --2- 172.21.15.19:0/41795373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ed0155ec0 0x7f4ed0154820 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800e7b0 tx=0x7f4eb800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.142+0000 7f4ec7400640 1 -- 172.21.15.19:0/41795373 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb800ee60 con 0x7f4ed0155ec0 2024-03-28T13:10:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.142+0000 7f4ec7400640 1 -- 172.21.15.19:0/41795373 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb8018070 con 0x7f4ed0155ec0 2024-03-28T13:10:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.142+0000 7f4ed4a00640 1 -- 172.21.15.19:0/41795373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed0155350 con 0x7f4ed0155ec0 2024-03-28T13:10:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.142+0000 7f4ec7400640 1 -- 172.21.15.19:0/41795373 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb800cb10 con 0x7f4ed0155ec0 2024-03-28T13:10:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.142+0000 7f4ed4a00640 1 -- 172.21.15.19:0/41795373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed013d730 con 0x7f4ed0155ec0 2024-03-28T13:10:39.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.144+0000 7f4ed4a00640 1 -- 172.21.15.19:0/41795373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed013d930 con 0x7f4ed0155ec0 2024-03-28T13:10:39.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.145+0000 7f4ec7400640 1 -- 172.21.15.19:0/41795373 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4eb800ccb0 con 0x7f4ed0155ec0 2024-03-28T13:10:39.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.146+0000 7f4ec7400640 1 --2- 172.21.15.19:0/41795373 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4eac06c530 0x7f4eac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:39.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.146+0000 7f4ec7400640 1 -- 172.21.15.19:0/41795373 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f4eb808b650 con 0x7f4ed0155ec0 2024-03-28T13:10:39.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.146+0000 7f4ece800640 1 --2- 172.21.15.19:0/41795373 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4eac06c530 0x7f4eac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:39.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.147+0000 7f4ece800640 1 --2- 172.21.15.19:0/41795373 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4eac06c530 0x7f4eac06e9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0036000 tx=0x7f4ec0031000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:39.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.150+0000 7f4ec7400640 1 -- 172.21.15.19:0/41795373 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4ed013d930 con 0x7f4ed0155ec0 2024-03-28T13:10:39.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.391+0000 7f4ed4a00640 1 -- 172.21.15.19:0/41795373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ed0075940 con 0x7f4ed0155ec0 2024-03-28T13:10:39.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.392+0000 7f4ec7400640 1 -- 172.21.15.19:0/41795373 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f4eb8059c30 con 0x7f4ed0155ec0 2024-03-28T13:10:39.393 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:39.393 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/745791856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":745791856},{"type":"v1","addr":"172.21.15.19:6833","nonce":745791856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/639283827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":639283827},{"type":"v1","addr":"172.21.15.188:6836","nonce":639283827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2168352329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2168352329},{"type":"v1","addr":"172.21.15.188:6837","nonce":2168352329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:09:41.105429+0000","modified":"2024-03-28T13:09:44.203526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/731659331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":731659331},{"type":"v1","addr":"172.21.15.19:6835","nonce":731659331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4254,"qdb_cluster":[4254]},"id":1}]} 2024-03-28T13:10:39.393 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-03-28T13:10:39.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.396+0000 7f4ed4a00640 1 -- 172.21.15.19:0/41795373 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4eac06c530 msgr2=0x7f4eac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:39.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.396+0000 7f4ed4a00640 1 --2- 172.21.15.19:0/41795373 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4eac06c530 0x7f4eac06e9f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0036000 tx=0x7f4ec0031000 comp rx=0 tx=0).stop 2024-03-28T13:10:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.396+0000 7f4ed4a00640 1 -- 172.21.15.19:0/41795373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ed0155ec0 msgr2=0x7f4ed0154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.396+0000 7f4ed4a00640 1 --2- 172.21.15.19:0/41795373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ed0155ec0 0x7f4ed0154820 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800e7b0 tx=0x7f4eb800ec70 comp rx=0 tx=0).stop 2024-03-28T13:10:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.396+0000 7f4ed4a00640 1 -- 172.21.15.19:0/41795373 shutdown_connections 2024-03-28T13:10:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.396+0000 7f4ed4a00640 1 --2- 172.21.15.19:0/41795373 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4eac06c530 0x7f4eac06e9f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.396+0000 7f4ed4a00640 1 --2- 172.21.15.19:0/41795373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ed006c680 0x7f4ed015b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.396+0000 7f4ed4a00640 1 --2- 172.21.15.19:0/41795373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ed0155ec0 0x7f4ed0154820 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.396+0000 7f4ed4a00640 1 --2- 172.21.15.19:0/41795373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4eb4003a00 0x7f4ed006d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.396+0000 7f4ed4a00640 1 -- 172.21.15.19:0/41795373 >> 172.21.15.19:0/41795373 conn(0x7f4ed0067460 msgr2=0x7f4ed0067840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:39.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.397+0000 7f4ed4a00640 1 -- 172.21.15.19:0/41795373 shutdown_connections 2024-03-28T13:10:39.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.397+0000 7f4ed4a00640 1 -- 172.21.15.19:0/41795373 wait complete. 2024-03-28T13:10:39.424 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:10:39.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.635+0000 7effb3000640 1 Processor -- start 2024-03-28T13:10:39.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.635+0000 7effb3000640 1 -- start start 2024-03-28T13:10:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.637+0000 7effb3000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7effac15e980 0x7effac0cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.637+0000 7effb3000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7effac0d4f10 0x7effac0d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.637+0000 7effb3000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effac0d2f60 0x7effac162390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.637+0000 7effb3000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7effac0d2530 con 0x7effac15e980 2024-03-28T13:10:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.637+0000 7effb3000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7effac0d5370 con 0x7effac0d4f10 2024-03-28T13:10:39.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.637+0000 7effb3000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7effac0cf630 con 0x7effac0d2f60 2024-03-28T13:10:39.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.637+0000 7effabe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7effac15e980 0x7effac0cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:39.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.637+0000 7effabe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7effac15e980 0x7effac0cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44176/0 (socket says 172.21.15.19:44176) 2024-03-28T13:10:39.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.637+0000 7effabe00640 1 -- 172.21.15.19:0/2131875486 learned_addr learned my addr 172.21.15.19:0/2131875486 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:39.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.638+0000 7effb1000640 1 --2- 172.21.15.19:0/2131875486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effac0d2f60 0x7effac162390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:39.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.638+0000 7effa3e00640 1 --2- 172.21.15.19:0/2131875486 >> v2:172.21.15.188:3300/0 conn(0x7effac0d4f10 0x7effac0d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:39.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.638+0000 7effabe00640 1 -- 172.21.15.19:0/2131875486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effac0d2f60 msgr2=0x7effac162390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:39.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.638+0000 7effabe00640 1 --2- 172.21.15.19:0/2131875486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effac0d2f60 0x7effac162390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.638+0000 7effabe00640 1 -- 172.21.15.19:0/2131875486 >> v2:172.21.15.188:3300/0 conn(0x7effac0d4f10 msgr2=0x7effac0d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:39.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.638+0000 7effabe00640 1 --2- 172.21.15.19:0/2131875486 >> v2:172.21.15.188:3300/0 conn(0x7effac0d4f10 0x7effac0d2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.638+0000 7effabe00640 1 -- 172.21.15.19:0/2131875486 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effac0cf910 con 0x7effac15e980 2024-03-28T13:10:39.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.640+0000 7effabe00640 1 --2- 172.21.15.19:0/2131875486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effac15e980 0x7effac0cefe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7effac0b7030 tx=0x7eff9c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d694992814499c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:39.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.640+0000 7effab400640 1 -- 172.21.15.19:0/2131875486 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c035020 con 0x7effac15e980 2024-03-28T13:10:39.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.640+0000 7effab400640 1 -- 172.21.15.19:0/2131875486 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff9c02d040 con 0x7effac15e980 2024-03-28T13:10:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.640+0000 7effab400640 1 -- 172.21.15.19:0/2131875486 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c031a80 con 0x7effac15e980 2024-03-28T13:10:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.641+0000 7effb3000640 1 -- 172.21.15.19:0/2131875486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effac15e980 msgr2=0x7effac0cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.641+0000 7effb3000640 1 --2- 172.21.15.19:0/2131875486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effac15e980 0x7effac0cefe0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7effac0b7030 tx=0x7eff9c02b900 comp rx=0 tx=0).stop 2024-03-28T13:10:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.641+0000 7effb3000640 1 -- 172.21.15.19:0/2131875486 shutdown_connections 2024-03-28T13:10:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.641+0000 7effb3000640 1 --2- 172.21.15.19:0/2131875486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effac0d2f60 0x7effac162390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.641+0000 7effb3000640 1 --2- 172.21.15.19:0/2131875486 >> v2:172.21.15.188:3300/0 conn(0x7effac0d4f10 0x7effac0d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.641+0000 7effb3000640 1 --2- 172.21.15.19:0/2131875486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effac15e980 0x7effac0cefe0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.641+0000 7effb3000640 1 -- 172.21.15.19:0/2131875486 >> 172.21.15.19:0/2131875486 conn(0x7effac08bfb0 msgr2=0x7effac0d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:39.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.641+0000 7effb3000640 1 -- 172.21.15.19:0/2131875486 shutdown_connections 2024-03-28T13:10:39.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.641+0000 7effb3000640 1 -- 172.21.15.19:0/2131875486 wait complete. 2024-03-28T13:10:39.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.643+0000 7effb3000640 1 Processor -- start 2024-03-28T13:10:39.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.643+0000 7effb3000640 1 -- start start 2024-03-28T13:10:39.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.644+0000 7effb3000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effac0d2f60 0x7effac1f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:39.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.644+0000 7effb3000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effac0d4f10 0x7effac0cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:39.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.645+0000 7effb3000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effac15e980 0x7effac0ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:39.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.645+0000 7effb3000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7effac0d01b0 con 0x7effac15e980 2024-03-28T13:10:39.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.645+0000 7effb3000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7effac0cfed0 con 0x7effac0d4f10 2024-03-28T13:10:39.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.645+0000 7effb3000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7effac1628d0 con 0x7effac0d2f60 2024-03-28T13:10:39.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.645+0000 7effb1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effac15e980 0x7effac0ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:39.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.645+0000 7effb1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effac15e980 0x7effac0ccda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44178/0 (socket says 172.21.15.19:44178) 2024-03-28T13:10:39.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.645+0000 7effb1000640 1 -- 172.21.15.19:0/406917650 learned_addr learned my addr 172.21.15.19:0/406917650 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:39.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.645+0000 7effabe00640 1 --2- 172.21.15.19:0/406917650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effac0d2f60 0x7effac1f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:39.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.645+0000 7effa3e00640 1 --2- 172.21.15.19:0/406917650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effac0d4f10 0x7effac0cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:39.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.645+0000 7effb1000640 1 -- 172.21.15.19:0/406917650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effac0d2f60 msgr2=0x7effac1f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:39.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.645+0000 7effb1000640 1 --2- 172.21.15.19:0/406917650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effac0d2f60 0x7effac1f5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.645+0000 7effb1000640 1 -- 172.21.15.19:0/406917650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effac0d4f10 msgr2=0x7effac0cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:39.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.645+0000 7effb1000640 1 --2- 172.21.15.19:0/406917650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effac0d4f10 0x7effac0cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.645+0000 7effb1000640 1 -- 172.21.15.19:0/406917650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effac0b39f0 con 0x7effac15e980 2024-03-28T13:10:39.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.646+0000 7effb1000640 1 --2- 172.21.15.19:0/406917650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effac15e980 0x7effac0ccda0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7effa4007e90 tx=0x7effa400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:39.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.646+0000 7effa8c00640 1 -- 172.21.15.19:0/406917650 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4014420 con 0x7effac15e980 2024-03-28T13:10:39.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.646+0000 7effa8c00640 1 -- 172.21.15.19:0/406917650 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effa4019070 con 0x7effac15e980 2024-03-28T13:10:39.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.646+0000 7effa8c00640 1 -- 172.21.15.19:0/406917650 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4014820 con 0x7effac15e980 2024-03-28T13:10:39.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.646+0000 7effb3000640 1 -- 172.21.15.19:0/406917650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effac0b3cd0 con 0x7effac15e980 2024-03-28T13:10:39.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.646+0000 7effb3000640 1 -- 172.21.15.19:0/406917650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effac0b4290 con 0x7effac15e980 2024-03-28T13:10:39.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.648+0000 7effb3000640 1 -- 172.21.15.19:0/406917650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effac0685c0 con 0x7effac15e980 2024-03-28T13:10:39.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.649+0000 7effa8c00640 1 -- 172.21.15.19:0/406917650 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7effa40149c0 con 0x7effac15e980 2024-03-28T13:10:39.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.649+0000 7effa8c00640 1 --2- 172.21.15.19:0/406917650 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7eff8806c530 0x7eff8806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:39.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.650+0000 7effa8c00640 1 -- 172.21.15.19:0/406917650 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7effa408bfb0 con 0x7effac15e980 2024-03-28T13:10:39.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.650+0000 7effabe00640 1 --2- 172.21.15.19:0/406917650 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7eff8806c530 0x7eff8806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:39.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.651+0000 7effabe00640 1 --2- 172.21.15.19:0/406917650 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7eff8806c530 0x7eff8806e9f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7eff9c036000 tx=0x7eff9c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:39.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.654+0000 7effa8c00640 1 -- 172.21.15.19:0/406917650 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7effa405a580 con 0x7effac15e980 2024-03-28T13:10:39.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.867+0000 7effb3000640 1 -- 172.21.15.19:0/406917650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7effac0d19f0 con 0x7effac15e980 2024-03-28T13:10:39.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.868+0000 7effa8c00640 1 -- 172.21.15.19:0/406917650 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16704 (secure 0 0 0) 0x7effa4059e80 con 0x7effac15e980 2024-03-28T13:10:39.869 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:39.870 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:10:23.142768+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":0,"to":4}]},{"pgid":"3.1b","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:10:39.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.871+0000 7effb3000640 1 -- 172.21.15.19:0/406917650 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7eff8806c530 msgr2=0x7eff8806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:39.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.871+0000 7effb3000640 1 --2- 172.21.15.19:0/406917650 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7eff8806c530 0x7eff8806e9f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7eff9c036000 tx=0x7eff9c03c040 comp rx=0 tx=0).stop 2024-03-28T13:10:39.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.871+0000 7effb3000640 1 -- 172.21.15.19:0/406917650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effac15e980 msgr2=0x7effac0ccda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:39.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.871+0000 7effb3000640 1 --2- 172.21.15.19:0/406917650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effac15e980 0x7effac0ccda0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7effa4007e90 tx=0x7effa400e4b0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.871+0000 7effb3000640 1 -- 172.21.15.19:0/406917650 shutdown_connections 2024-03-28T13:10:39.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.871+0000 7effb3000640 1 --2- 172.21.15.19:0/406917650 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7eff8806c530 0x7eff8806e9f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.871+0000 7effb3000640 1 --2- 172.21.15.19:0/406917650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effac15e980 0x7effac0ccda0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.871+0000 7effb3000640 1 --2- 172.21.15.19:0/406917650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7effac0d4f10 0x7effac0cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.871+0000 7effb3000640 1 --2- 172.21.15.19:0/406917650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7effac0d2f60 0x7effac1f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:39.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.871+0000 7effb3000640 1 -- 172.21.15.19:0/406917650 >> 172.21.15.19:0/406917650 conn(0x7effac08bfb0 msgr2=0x7effac0d33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:39.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.871+0000 7effb3000640 1 -- 172.21.15.19:0/406917650 shutdown_connections 2024-03-28T13:10:39.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:39.871+0000 7effb3000640 1 -- 172.21.15.19:0/406917650 wait complete. 2024-03-28T13:10:39.895 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:09:39.243813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:09:40.092899+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:10:39.896 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:10:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.106+0000 7fa077200640 1 Processor -- start 2024-03-28T13:10:40.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.106+0000 7fa077200640 1 -- start start 2024-03-28T13:10:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.108+0000 7fa077200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa07815e460 0x7fa0780b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.108+0000 7fa077200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0780d3e40 0x7fa0780d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.108+0000 7fa077200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0780b8320 con 0x7fa0780cef20 2024-03-28T13:10:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.108+0000 7fa077200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa0780d2da0 con 0x7fa0780d3e40 2024-03-28T13:10:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.108+0000 7fa077200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa0780d3080 con 0x7fa07815e460 2024-03-28T13:10:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.108+0000 7fa06fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0780d3e40 0x7fa0780d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:40.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.108+0000 7fa076800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa0780cef20 0x7fa07806c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37060/0 (socket says 172.21.15.19:37060) 2024-03-28T13:10:40.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.108+0000 7fa076800640 1 -- 172.21.15.19:0/3843944671 learned_addr learned my addr 172.21.15.19:0/3843944671 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:40.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.108+0000 7fa075e00640 1 --2- 172.21.15.19:0/3843944671 >> v2:172.21.15.188:3300/0 conn(0x7fa07815e460 0x7fa0780b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:40.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.109+0000 7fa06fe00640 1 -- 172.21.15.19:0/3843944671 >> v2:172.21.15.188:3300/0 conn(0x7fa07815e460 msgr2=0x7fa0780b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:40.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.109+0000 7fa06fe00640 1 --2- 172.21.15.19:0/3843944671 >> v2:172.21.15.188:3300/0 conn(0x7fa07815e460 0x7fa0780b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.109+0000 7fa06fe00640 1 -- 172.21.15.19:0/3843944671 >> v1:172.21.15.19:6789/0 conn(0x7fa0780cef20 legacy=0x7fa07806c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:40.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.109+0000 7fa075400640 1 -- 172.21.15.19:0/3843944671 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2275550851 0 0) 0x7fa0780b8320 con 0x7fa0780cef20 2024-03-28T13:10:40.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.109+0000 7fa06fe00640 1 -- 172.21.15.19:0/3843944671 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0780d3360 con 0x7fa0780d3e40 2024-03-28T13:10:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.110+0000 7fa06fe00640 1 --2- 172.21.15.19:0/3843944671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0780d3e40 0x7fa0780d2550 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa07806d6a0 tx=0x7fa06002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12dad94870733d73 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.110+0000 7fa075400640 1 -- 172.21.15.19:0/3843944671 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060035020 con 0x7fa0780d3e40 2024-03-28T13:10:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.110+0000 7fa075400640 1 -- 172.21.15.19:0/3843944671 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa060031830 con 0x7fa0780d3e40 2024-03-28T13:10:40.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.110+0000 7fa075400640 1 -- 172.21.15.19:0/3843944671 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa060031c00 con 0x7fa0780d3e40 2024-03-28T13:10:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.111+0000 7fa077200640 1 -- 172.21.15.19:0/3843944671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0780d3e40 msgr2=0x7fa0780d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.111+0000 7fa077200640 1 --2- 172.21.15.19:0/3843944671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0780d3e40 0x7fa0780d2550 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa07806d6a0 tx=0x7fa06002b8d0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.111+0000 7fa077200640 1 -- 172.21.15.19:0/3843944671 shutdown_connections 2024-03-28T13:10:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.111+0000 7fa077200640 1 --2- 172.21.15.19:0/3843944671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0780d3e40 0x7fa0780d2550 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.111+0000 7fa077200640 1 --2- 172.21.15.19:0/3843944671 >> v2:172.21.15.188:3300/0 conn(0x7fa07815e460 0x7fa0780b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.111+0000 7fa077200640 1 -- 172.21.15.19:0/3843944671 >> 172.21.15.19:0/3843944671 conn(0x7fa0780678b0 msgr2=0x7fa0780d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.111+0000 7fa077200640 1 -- 172.21.15.19:0/3843944671 shutdown_connections 2024-03-28T13:10:40.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.111+0000 7fa077200640 1 -- 172.21.15.19:0/3843944671 wait complete. 2024-03-28T13:10:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.113+0000 7fa077200640 1 Processor -- start 2024-03-28T13:10:40.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.113+0000 7fa077200640 1 -- start start 2024-03-28T13:10:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.114+0000 7fa077200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0780cef20 0x7fa07815c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:40.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.114+0000 7fa077200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0780d3e40 0x7fa0780cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.114+0000 7fa077200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07815e460 0x7fa0780ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.114+0000 7fa077200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0780d2a90 con 0x7fa07815e460 2024-03-28T13:10:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.114+0000 7fa077200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0780d3920 con 0x7fa0780d3e40 2024-03-28T13:10:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.114+0000 7fa077200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa0780cf390 con 0x7fa0780cef20 2024-03-28T13:10:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.114+0000 7fa076800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07815e460 0x7fa0780ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.115+0000 7fa076800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07815e460 0x7fa0780ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44204/0 (socket says 172.21.15.19:44204) 2024-03-28T13:10:40.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.115+0000 7fa076800640 1 -- 172.21.15.19:0/637279018 learned_addr learned my addr 172.21.15.19:0/637279018 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.115+0000 7fa075e00640 1 --2- 172.21.15.19:0/637279018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0780cef20 0x7fa07815c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:40.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.115+0000 7fa06fe00640 1 --2- 172.21.15.19:0/637279018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0780d3e40 0x7fa0780cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.115+0000 7fa076800640 1 -- 172.21.15.19:0/637279018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0780cef20 msgr2=0x7fa07815c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.115+0000 7fa076800640 1 --2- 172.21.15.19:0/637279018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0780cef20 0x7fa07815c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.115+0000 7fa076800640 1 -- 172.21.15.19:0/637279018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0780d3e40 msgr2=0x7fa0780cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.115+0000 7fa076800640 1 --2- 172.21.15.19:0/637279018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0780d3e40 0x7fa0780cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.115+0000 7fa076800640 1 -- 172.21.15.19:0/637279018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa078160470 con 0x7fa07815e460 2024-03-28T13:10:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.115+0000 7fa076800640 1 --2- 172.21.15.19:0/637279018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07815e460 0x7fa0780ce0d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c7a0 tx=0x7fa07000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.116+0000 7fa06e000640 1 -- 172.21.15.19:0/637279018 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070018070 con 0x7fa07815e460 2024-03-28T13:10:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.116+0000 7fa06e000640 1 -- 172.21.15.19:0/637279018 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa07000a640 con 0x7fa07815e460 2024-03-28T13:10:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.116+0000 7fa06e000640 1 -- 172.21.15.19:0/637279018 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07000aab0 con 0x7fa07815e460 2024-03-28T13:10:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.116+0000 7fa077200640 1 -- 172.21.15.19:0/637279018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa078160750 con 0x7fa07815e460 2024-03-28T13:10:40.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.116+0000 7fa077200640 1 -- 172.21.15.19:0/637279018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa078160d10 con 0x7fa07815e460 2024-03-28T13:10:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.117+0000 7fa077200640 1 -- 172.21.15.19:0/637279018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa078075940 con 0x7fa07815e460 2024-03-28T13:10:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.118+0000 7fa06e000640 1 -- 172.21.15.19:0/637279018 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa07000ac50 con 0x7fa07815e460 2024-03-28T13:10:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.120+0000 7fa06e000640 1 --2- 172.21.15.19:0/637279018 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa05406c560 0x7fa05406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.120+0000 7fa06e000640 1 -- 172.21.15.19:0/637279018 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7fa07008ad10 con 0x7fa07815e460 2024-03-28T13:10:40.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.120+0000 7fa075e00640 1 --2- 172.21.15.19:0/637279018 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa05406c560 0x7fa05406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:40.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.122+0000 7fa075e00640 1 --2- 172.21.15.19:0/637279018 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa05406c560 0x7fa05406ea20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa068006fd0 tx=0x7fa068008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:40.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.124+0000 7fa06e000640 1 -- 172.21.15.19:0/637279018 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa070090020 con 0x7fa07815e460 2024-03-28T13:10:40.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.330+0000 7fa077200640 1 -- 172.21.15.19:0/637279018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa0780d21d0 con 0x7fa07815e460 2024-03-28T13:10:40.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.331+0000 7fa06e000640 1 -- 172.21.15.19:0/637279018 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fa070059250 con 0x7fa07815e460 2024-03-28T13:10:40.331 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:10:40.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.333+0000 7fa077200640 1 -- 172.21.15.19:0/637279018 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa05406c560 msgr2=0x7fa05406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:40.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.333+0000 7fa077200640 1 --2- 172.21.15.19:0/637279018 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa05406c560 0x7fa05406ea20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa068006fd0 tx=0x7fa068008040 comp rx=0 tx=0).stop 2024-03-28T13:10:40.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.333+0000 7fa077200640 1 -- 172.21.15.19:0/637279018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07815e460 msgr2=0x7fa0780ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:40.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.333+0000 7fa077200640 1 --2- 172.21.15.19:0/637279018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07815e460 0x7fa0780ce0d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c7a0 tx=0x7fa07000cc60 comp rx=0 tx=0).stop 2024-03-28T13:10:40.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.334+0000 7fa077200640 1 -- 172.21.15.19:0/637279018 shutdown_connections 2024-03-28T13:10:40.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.334+0000 7fa077200640 1 --2- 172.21.15.19:0/637279018 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa05406c560 0x7fa05406ea20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.334+0000 7fa077200640 1 --2- 172.21.15.19:0/637279018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa07815e460 0x7fa0780ce0d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.334+0000 7fa077200640 1 --2- 172.21.15.19:0/637279018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0780d3e40 0x7fa0780cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.334+0000 7fa077200640 1 --2- 172.21.15.19:0/637279018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0780cef20 0x7fa07815c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.334+0000 7fa077200640 1 -- 172.21.15.19:0/637279018 >> 172.21.15.19:0/637279018 conn(0x7fa0780678b0 msgr2=0x7fa0780b4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:40.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.334+0000 7fa077200640 1 -- 172.21.15.19:0/637279018 shutdown_connections 2024-03-28T13:10:40.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.334+0000 7fa077200640 1 -- 172.21.15.19:0/637279018 wait complete. 2024-03-28T13:10:40.352 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:10:40.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.561+0000 7f952b200640 1 Processor -- start 2024-03-28T13:10:40.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.562+0000 7f952b200640 1 -- start start 2024-03-28T13:10:40.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.563+0000 7f952b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f952415e4c0 0x7f95240b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:40.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.563+0000 7f952b200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f95240d37b0 0x7f95240b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:40.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.563+0000 7f952b200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95240d2710 con 0x7f95240b67f0 2024-03-28T13:10:40.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.563+0000 7f952b200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f952415e8a0 con 0x7f95240d37b0 2024-03-28T13:10:40.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.563+0000 7f952b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f95240d4040 con 0x7f952415e4c0 2024-03-28T13:10:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.564+0000 7f9529e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f952415e4c0 0x7f95240b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.564+0000 7f952a800640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f95240b67f0 0x7f95240d4260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:51518/0 (socket says 172.21.15.19:51518) 2024-03-28T13:10:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.564+0000 7f952a800640 1 -- 172.21.15.19:0/209665252 learned_addr learned my addr 172.21.15.19:0/209665252 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.564+0000 7f9529e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f952415e4c0 0x7f95240b9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:51916/0 (socket says 172.21.15.19:51916) 2024-03-28T13:10:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.564+0000 7f9529400640 1 --2- 172.21.15.19:0/209665252 >> v2:172.21.15.188:3300/0 conn(0x7f95240d37b0 0x7f95240b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.565+0000 7f9529e00640 1 -- 172.21.15.19:0/209665252 >> v2:172.21.15.188:3300/0 conn(0x7f95240d37b0 msgr2=0x7f95240b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.565+0000 7f9529e00640 1 --2- 172.21.15.19:0/209665252 >> v2:172.21.15.188:3300/0 conn(0x7f95240d37b0 0x7f95240b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.565+0000 7f9528a00640 1 -- 172.21.15.19:0/209665252 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3540055341 0 0) 0x7f95240d2710 con 0x7f95240b67f0 2024-03-28T13:10:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.565+0000 7f9529e00640 1 -- 172.21.15.19:0/209665252 >> v1:172.21.15.188:6789/0 conn(0x7f95240b67f0 legacy=0x7f95240d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.565+0000 7f9529e00640 1 -- 172.21.15.19:0/209665252 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95240d29f0 con 0x7f952415e4c0 2024-03-28T13:10:40.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.566+0000 7f9529e00640 1 --2- 172.21.15.19:0/209665252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f952415e4c0 0x7f95240b9bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f951c009d90 tx=0x7f951c02beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4343da052a5b7ee server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:40.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.566+0000 7f9528a00640 1 -- 172.21.15.19:0/209665252 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c039070 con 0x7f952415e4c0 2024-03-28T13:10:40.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.566+0000 7f9528a00640 1 -- 172.21.15.19:0/209665252 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951c02cc60 con 0x7f952415e4c0 2024-03-28T13:10:40.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.566+0000 7f9528a00640 1 -- 172.21.15.19:0/209665252 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c034660 con 0x7f952415e4c0 2024-03-28T13:10:40.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.567+0000 7f952b200640 1 -- 172.21.15.19:0/209665252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f952415e4c0 msgr2=0x7f95240b9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:40.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.567+0000 7f952b200640 1 --2- 172.21.15.19:0/209665252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f952415e4c0 0x7f95240b9bf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f951c009d90 tx=0x7f951c02beb0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.567+0000 7f952b200640 1 -- 172.21.15.19:0/209665252 shutdown_connections 2024-03-28T13:10:40.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.567+0000 7f952b200640 1 --2- 172.21.15.19:0/209665252 >> v2:172.21.15.188:3300/0 conn(0x7f95240d37b0 0x7f95240b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.567+0000 7f952b200640 1 --2- 172.21.15.19:0/209665252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f952415e4c0 0x7f95240b9bf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.567+0000 7f952b200640 1 -- 172.21.15.19:0/209665252 >> 172.21.15.19:0/209665252 conn(0x7f95240676a0 msgr2=0x7f9524067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:40.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.567+0000 7f952b200640 1 -- 172.21.15.19:0/209665252 shutdown_connections 2024-03-28T13:10:40.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.567+0000 7f952b200640 1 -- 172.21.15.19:0/209665252 wait complete. 2024-03-28T13:10:40.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.569+0000 7f952b200640 1 Processor -- start 2024-03-28T13:10:40.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.569+0000 7f952b200640 1 -- start start 2024-03-28T13:10:40.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.570+0000 7f952b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95240b67f0 0x7f9524161020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:40.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.570+0000 7f952b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95240d37b0 0x7f95241f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:40.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.570+0000 7f952b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f952415e4c0 0x7f95241f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:40.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.570+0000 7f952b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95240d2fb0 con 0x7f95240b67f0 2024-03-28T13:10:40.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.570+0000 7f952b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95240d3320 con 0x7f95240d37b0 2024-03-28T13:10:40.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.570+0000 7f952b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f95240ba130 con 0x7f952415e4c0 2024-03-28T13:10:40.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.570+0000 7f9529e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95240b67f0 0x7f9524161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:40.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.570+0000 7f9529e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95240b67f0 0x7f9524161020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44208/0 (socket says 172.21.15.19:44208) 2024-03-28T13:10:40.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.570+0000 7f9529e00640 1 -- 172.21.15.19:0/1100663168 learned_addr learned my addr 172.21.15.19:0/1100663168 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:40.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.571+0000 7f9529400640 1 --2- 172.21.15.19:0/1100663168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95240d37b0 0x7f95241f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:40.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.571+0000 7f952a800640 1 --2- 172.21.15.19:0/1100663168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f952415e4c0 0x7f95241f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:40.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.571+0000 7f9529e00640 1 -- 172.21.15.19:0/1100663168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f952415e4c0 msgr2=0x7f95241f3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:40.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.571+0000 7f9529e00640 1 --2- 172.21.15.19:0/1100663168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f952415e4c0 0x7f95241f3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.571+0000 7f9529e00640 1 -- 172.21.15.19:0/1100663168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95240d37b0 msgr2=0x7f95241f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:40.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.571+0000 7f9529e00640 1 --2- 172.21.15.19:0/1100663168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95240d37b0 0x7f95241f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.571+0000 7f9529e00640 1 -- 172.21.15.19:0/1100663168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952415ff80 con 0x7f95240b67f0 2024-03-28T13:10:40.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.571+0000 7f9529e00640 1 --2- 172.21.15.19:0/1100663168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95240b67f0 0x7f9524161020 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f951c009d90 tx=0x7f951c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:40.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.572+0000 7f951a000640 1 -- 172.21.15.19:0/1100663168 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c039050 con 0x7f95240b67f0 2024-03-28T13:10:40.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.572+0000 7f951a000640 1 -- 172.21.15.19:0/1100663168 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f951c03a040 con 0x7f95240b67f0 2024-03-28T13:10:40.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.572+0000 7f952b200640 1 -- 172.21.15.19:0/1100663168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9524160260 con 0x7f95240b67f0 2024-03-28T13:10:40.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.572+0000 7f951a000640 1 -- 172.21.15.19:0/1100663168 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951c009450 con 0x7f95240b67f0 2024-03-28T13:10:40.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.572+0000 7f952b200640 1 -- 172.21.15.19:0/1100663168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9524160820 con 0x7f95240b67f0 2024-03-28T13:10:40.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.573+0000 7f952b200640 1 -- 172.21.15.19:0/1100663168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9524075940 con 0x7f95240b67f0 2024-03-28T13:10:40.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.574+0000 7f951a000640 1 -- 172.21.15.19:0/1100663168 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f951c047020 con 0x7f95240b67f0 2024-03-28T13:10:40.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.575+0000 7f951a000640 1 --2- 172.21.15.19:0/1100663168 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f950806c530 0x7f950806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:40.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.575+0000 7f951a000640 1 -- 172.21.15.19:0/1100663168 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f951c0ada20 con 0x7f95240b67f0 2024-03-28T13:10:40.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.576+0000 7f9529400640 1 --2- 172.21.15.19:0/1100663168 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f950806c530 0x7f950806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:40.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.577+0000 7f9529400640 1 --2- 172.21.15.19:0/1100663168 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f950806c530 0x7f950806e9f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f95240b53d0 tx=0x7f9514008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.579+0000 7f951a000640 1 -- 172.21.15.19:0/1100663168 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f951c04e020 con 0x7f95240b67f0 2024-03-28T13:10:40.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.962+0000 7f952b200640 1 -- 172.21.15.19:0/1100663168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f952406b570 con 0x7f95240b67f0 2024-03-28T13:10:40.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.963+0000 7f951a000640 1 -- 172.21.15.19:0/1100663168 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f951c07c000 con 0x7f95240b67f0 2024-03-28T13:10:40.966 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:10:40.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.967+0000 7f952b200640 1 -- 172.21.15.19:0/1100663168 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f950806c530 msgr2=0x7f950806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:40.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.967+0000 7f952b200640 1 --2- 172.21.15.19:0/1100663168 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f950806c530 0x7f950806e9f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f95240b53d0 tx=0x7f9514008040 comp rx=0 tx=0).stop 2024-03-28T13:10:40.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.967+0000 7f952b200640 1 -- 172.21.15.19:0/1100663168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95240b67f0 msgr2=0x7f9524161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:40.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.967+0000 7f952b200640 1 --2- 172.21.15.19:0/1100663168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95240b67f0 0x7f9524161020 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f951c009d90 tx=0x7f951c004a40 comp rx=0 tx=0).stop 2024-03-28T13:10:40.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.967+0000 7f952b200640 1 -- 172.21.15.19:0/1100663168 shutdown_connections 2024-03-28T13:10:40.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.967+0000 7f952b200640 1 --2- 172.21.15.19:0/1100663168 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f950806c530 0x7f950806e9f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.967+0000 7f952b200640 1 --2- 172.21.15.19:0/1100663168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f952415e4c0 0x7f95241f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.967+0000 7f952b200640 1 --2- 172.21.15.19:0/1100663168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95240d37b0 0x7f95241f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.967+0000 7f952b200640 1 --2- 172.21.15.19:0/1100663168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95240b67f0 0x7f9524161020 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:40.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.967+0000 7f952b200640 1 -- 172.21.15.19:0/1100663168 >> 172.21.15.19:0/1100663168 conn(0x7f95240676a0 msgr2=0x7f952415ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:40.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.968+0000 7f952b200640 1 -- 172.21.15.19:0/1100663168 shutdown_connections 2024-03-28T13:10:40.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:40.968+0000 7f952b200640 1 -- 172.21.15.19:0/1100663168 wait complete. 2024-03-28T13:10:40.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:10:41.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.207+0000 7f4522c00640 1 Processor -- start 2024-03-28T13:10:41.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.207+0000 7f4522c00640 1 -- start start 2024-03-28T13:10:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.210+0000 7f4522c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f451c0d0e70 0x7f451c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.210+0000 7f4522c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f451c15f6e0 0x7f451c0d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.210+0000 7f4522c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f451c0d1790 con 0x7f451c1621d0 2024-03-28T13:10:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.210+0000 7f4522c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f451c0cf6c0 con 0x7f451c0d0e70 2024-03-28T13:10:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.210+0000 7f4522c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f451c0cf9a0 con 0x7f451c15f6e0 2024-03-28T13:10:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.210+0000 7f451be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f451c0d0e70 0x7f451c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.210+0000 7f451be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f451c0d0e70 0x7f451c0cf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44212/0 (socket says 172.21.15.19:44212) 2024-03-28T13:10:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.210+0000 7f451be00640 1 -- 172.21.15.19:0/989651092 learned_addr learned my addr 172.21.15.19:0/989651092 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.211+0000 7f4513e00640 1 --2- 172.21.15.19:0/989651092 >> v2:172.21.15.188:3300/0 conn(0x7f451c15f6e0 0x7f451c0d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.211+0000 7f451be00640 1 -- 172.21.15.19:0/989651092 >> v2:172.21.15.188:3300/0 conn(0x7f451c15f6e0 msgr2=0x7f451c0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:41.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.211+0000 7f451be00640 1 --2- 172.21.15.19:0/989651092 >> v2:172.21.15.188:3300/0 conn(0x7f451c15f6e0 0x7f451c0d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.211+0000 7f451be00640 1 -- 172.21.15.19:0/989651092 >> v1:172.21.15.188:6789/0 conn(0x7f451c1621d0 legacy=0x7f451c162660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:41.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.211+0000 7f451be00640 1 -- 172.21.15.19:0/989651092 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c0cfc80 con 0x7f451c0d0e70 2024-03-28T13:10:41.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.212+0000 7f451be00640 1 --2- 172.21.15.19:0/989651092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f451c0d0e70 0x7f451c0cf050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f451c0b7020 tx=0x7f450c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4c59eb94178c04 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.212+0000 7f451b400640 1 -- 172.21.15.19:0/989651092 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c02e070 con 0x7f451c0d0e70 2024-03-28T13:10:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.213+0000 7f451b400640 1 -- 172.21.15.19:0/989651092 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450c037070 con 0x7f451c0d0e70 2024-03-28T13:10:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.213+0000 7f451b400640 1 -- 172.21.15.19:0/989651092 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c031a00 con 0x7f451c0d0e70 2024-03-28T13:10:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.213+0000 7f4522c00640 1 -- 172.21.15.19:0/989651092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f451c0d0e70 msgr2=0x7f451c0cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:41.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.213+0000 7f4522c00640 1 --2- 172.21.15.19:0/989651092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f451c0d0e70 0x7f451c0cf050 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f451c0b7020 tx=0x7f450c02b880 comp rx=0 tx=0).stop 2024-03-28T13:10:41.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.214+0000 7f4522c00640 1 -- 172.21.15.19:0/989651092 shutdown_connections 2024-03-28T13:10:41.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.214+0000 7f4522c00640 1 --2- 172.21.15.19:0/989651092 >> v2:172.21.15.188:3300/0 conn(0x7f451c15f6e0 0x7f451c0d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.214+0000 7f4522c00640 1 --2- 172.21.15.19:0/989651092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f451c0d0e70 0x7f451c0cf050 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.214+0000 7f4522c00640 1 -- 172.21.15.19:0/989651092 >> 172.21.15.19:0/989651092 conn(0x7f451c08bfb0 msgr2=0x7f451c0d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:41.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.214+0000 7f4522c00640 1 -- 172.21.15.19:0/989651092 shutdown_connections 2024-03-28T13:10:41.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.214+0000 7f4522c00640 1 -- 172.21.15.19:0/989651092 wait complete. 2024-03-28T13:10:41.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.217+0000 7f4522c00640 1 Processor -- start 2024-03-28T13:10:41.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.217+0000 7f4522c00640 1 -- start start 2024-03-28T13:10:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.218+0000 7f4522c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f451c0d0e70 0x7f451c14ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.218+0000 7f4522c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f451c15f6e0 0x7f451c1592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.218+0000 7f4522c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f451c1621d0 0x7f451c159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.218+0000 7f4522c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f451c15fb40 con 0x7f451c1621d0 2024-03-28T13:10:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.218+0000 7f4522c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f451c0d0240 con 0x7f451c15f6e0 2024-03-28T13:10:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.218+0000 7f4522c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f451c15e910 con 0x7f451c0d0e70 2024-03-28T13:10:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.219+0000 7f4520c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f451c1621d0 0x7f451c159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.219+0000 7f4520c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f451c1621d0 0x7f451c159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44226/0 (socket says 172.21.15.19:44226) 2024-03-28T13:10:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.219+0000 7f4520c00640 1 -- 172.21.15.19:0/3120937684 learned_addr learned my addr 172.21.15.19:0/3120937684 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.219+0000 7f451be00640 1 --2- 172.21.15.19:0/3120937684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f451c0d0e70 0x7f451c14ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.219+0000 7f4513e00640 1 --2- 172.21.15.19:0/3120937684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f451c15f6e0 0x7f451c1592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:41.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.219+0000 7f4520c00640 1 -- 172.21.15.19:0/3120937684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f451c0d0e70 msgr2=0x7f451c14ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:41.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.219+0000 7f4520c00640 1 --2- 172.21.15.19:0/3120937684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f451c0d0e70 0x7f451c14ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.219+0000 7f4520c00640 1 -- 172.21.15.19:0/3120937684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f451c15f6e0 msgr2=0x7f451c1592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:41.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.219+0000 7f4520c00640 1 --2- 172.21.15.19:0/3120937684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f451c15f6e0 0x7f451c1592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.219+0000 7f4520c00640 1 -- 172.21.15.19:0/3120937684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c1f4620 con 0x7f451c1621d0 2024-03-28T13:10:41.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.219+0000 7f4513e00640 1 --2- 172.21.15.19:0/3120937684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f451c15f6e0 0x7f451c1592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:41.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.220+0000 7f4520c00640 1 --2- 172.21.15.19:0/3120937684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f451c1621d0 0x7f451c159820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f451400b970 tx=0x7f451400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:41.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.220+0000 7f4518c00640 1 -- 172.21.15.19:0/3120937684 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514019070 con 0x7f451c1621d0 2024-03-28T13:10:41.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.220+0000 7f4518c00640 1 -- 172.21.15.19:0/3120937684 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f451400cb40 con 0x7f451c1621d0 2024-03-28T13:10:41.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.220+0000 7f4522c00640 1 -- 172.21.15.19:0/3120937684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451c148530 con 0x7f451c1621d0 2024-03-28T13:10:41.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.220+0000 7f4518c00640 1 -- 172.21.15.19:0/3120937684 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4514014440 con 0x7f451c1621d0 2024-03-28T13:10:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.220+0000 7f4522c00640 1 -- 172.21.15.19:0/3120937684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451c148af0 con 0x7f451c1621d0 2024-03-28T13:10:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.222+0000 7f4522c00640 1 -- 172.21.15.19:0/3120937684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f451c068cd0 con 0x7f451c1621d0 2024-03-28T13:10:41.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.222+0000 7f4518c00640 1 -- 172.21.15.19:0/3120937684 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f451401c070 con 0x7f451c1621d0 2024-03-28T13:10:41.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.224+0000 7f4518c00640 1 --2- 172.21.15.19:0/3120937684 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f44f806c530 0x7f44f806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:41.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.224+0000 7f4518c00640 1 -- 172.21.15.19:0/3120937684 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f451408b350 con 0x7f451c1621d0 2024-03-28T13:10:41.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.225+0000 7f451be00640 1 --2- 172.21.15.19:0/3120937684 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f44f806c530 0x7f44f806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:41.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.226+0000 7f451be00640 1 --2- 172.21.15.19:0/3120937684 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f44f806c530 0x7f44f806e9f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f450c036000 tx=0x7f450c03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:41.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.227+0000 7f4518c00640 1 -- 172.21.15.19:0/3120937684 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4514059890 con 0x7f451c1621d0 2024-03-28T13:10:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.444+0000 7f4522c00640 1 -- 172.21.15.19:0/3120937684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f451c0d1b80 con 0x7f451c1621d0 2024-03-28T13:10:41.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.445+0000 7f4518c00640 1 -- 172.21.15.19:0/3120937684 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f4514059190 con 0x7f451c1621d0 2024-03-28T13:10:41.445 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:10:41.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.449+0000 7f4522c00640 1 -- 172.21.15.19:0/3120937684 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f44f806c530 msgr2=0x7f44f806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:41.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.449+0000 7f4522c00640 1 --2- 172.21.15.19:0/3120937684 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f44f806c530 0x7f44f806e9f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f450c036000 tx=0x7f450c03d040 comp rx=0 tx=0).stop 2024-03-28T13:10:41.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.449+0000 7f4522c00640 1 -- 172.21.15.19:0/3120937684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f451c1621d0 msgr2=0x7f451c159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:41.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.449+0000 7f4522c00640 1 --2- 172.21.15.19:0/3120937684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f451c1621d0 0x7f451c159820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f451400b970 tx=0x7f451400be30 comp rx=0 tx=0).stop 2024-03-28T13:10:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.449+0000 7f4522c00640 1 -- 172.21.15.19:0/3120937684 shutdown_connections 2024-03-28T13:10:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.449+0000 7f4522c00640 1 --2- 172.21.15.19:0/3120937684 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f44f806c530 0x7f44f806e9f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.449+0000 7f4522c00640 1 --2- 172.21.15.19:0/3120937684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f451c1621d0 0x7f451c159820 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.449+0000 7f4522c00640 1 --2- 172.21.15.19:0/3120937684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f451c15f6e0 0x7f451c1592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.449+0000 7f4522c00640 1 --2- 172.21.15.19:0/3120937684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f451c0d0e70 0x7f451c14ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.449+0000 7f4522c00640 1 -- 172.21.15.19:0/3120937684 >> 172.21.15.19:0/3120937684 conn(0x7f451c08bfb0 msgr2=0x7f451c06d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.449+0000 7f4522c00640 1 -- 172.21.15.19:0/3120937684 shutdown_connections 2024-03-28T13:10:41.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.449+0000 7f4522c00640 1 -- 172.21.15.19:0/3120937684 wait complete. 2024-03-28T13:10:41.468 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:10:41.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.673+0000 7f7fbb200640 1 Processor -- start 2024-03-28T13:10:41.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.673+0000 7f7fbb200640 1 -- start start 2024-03-28T13:10:41.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.675+0000 7f7fbb200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7fbc15e4a0 0x7f7fbc0b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:41.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.675+0000 7f7fbb200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7fbc0cf700 0x7f7fbc0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:41.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.675+0000 7f7fbb200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fbc15e880 con 0x7f7fbc0d2310 2024-03-28T13:10:41.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.675+0000 7f7fbb200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7fbc0d34e0 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.675+0000 7f7fbb200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7fbc0d28d0 con 0x7f7fbc0cf700 2024-03-28T13:10:41.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.675+0000 7f7fb9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7fbc15e4a0 0x7f7fbc0b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:41.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.675+0000 7f7fb9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7fbc15e4a0 0x7f7fbc0b9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44240/0 (socket says 172.21.15.19:44240) 2024-03-28T13:10:41.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.675+0000 7f7fb9e00640 1 -- 172.21.15.19:0/462433134 learned_addr learned my addr 172.21.15.19:0/462433134 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.675+0000 7f7fba800640 1 --2- 172.21.15.19:0/462433134 >> v2:172.21.15.188:3300/0 conn(0x7f7fbc0cf700 0x7f7fbc0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.676+0000 7f7fb9e00640 1 -- 172.21.15.19:0/462433134 >> v2:172.21.15.188:3300/0 conn(0x7f7fbc0cf700 msgr2=0x7f7fbc0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.676+0000 7f7fb9e00640 1 --2- 172.21.15.19:0/462433134 >> v2:172.21.15.188:3300/0 conn(0x7f7fbc0cf700 0x7f7fbc0b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.676+0000 7f7fb9e00640 1 -- 172.21.15.19:0/462433134 >> v1:172.21.15.188:6789/0 conn(0x7f7fbc0d2310 legacy=0x7f7fbc0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.676+0000 7f7fb9e00640 1 -- 172.21.15.19:0/462433134 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc0d2bb0 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.676+0000 7f7fba800640 1 --2- 172.21.15.19:0/462433134 >> v2:172.21.15.188:3300/0 conn(0x7f7fbc0cf700 0x7f7fbc0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:41.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.677+0000 7f7fb9e00640 1 --2- 172.21.15.19:0/462433134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc15e4a0 0x7f7fbc0b9c60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0cfb20 tx=0x7f7fac02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba231f21c7509ca server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:41.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.677+0000 7f7fb9400640 1 -- 172.21.15.19:0/462433134 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac0317c0 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.677+0000 7f7fb9400640 1 -- 172.21.15.19:0/462433134 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fac037070 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.677+0000 7f7fb9400640 1 -- 172.21.15.19:0/462433134 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fac031b80 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.678+0000 7f7fbb200640 1 -- 172.21.15.19:0/462433134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc15e4a0 msgr2=0x7f7fbc0b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:41.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.678+0000 7f7fbb200640 1 --2- 172.21.15.19:0/462433134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc15e4a0 0x7f7fbc0b9c60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc0cfb20 tx=0x7f7fac02b860 comp rx=0 tx=0).stop 2024-03-28T13:10:41.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.678+0000 7f7fbb200640 1 -- 172.21.15.19:0/462433134 shutdown_connections 2024-03-28T13:10:41.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.678+0000 7f7fbb200640 1 --2- 172.21.15.19:0/462433134 >> v2:172.21.15.188:3300/0 conn(0x7f7fbc0cf700 0x7f7fbc0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.678+0000 7f7fbb200640 1 --2- 172.21.15.19:0/462433134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc15e4a0 0x7f7fbc0b9c60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.678+0000 7f7fbb200640 1 -- 172.21.15.19:0/462433134 >> 172.21.15.19:0/462433134 conn(0x7f7fbc0676d0 msgr2=0x7f7fbc067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:41.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.678+0000 7f7fbb200640 1 -- 172.21.15.19:0/462433134 shutdown_connections 2024-03-28T13:10:41.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.678+0000 7f7fbb200640 1 -- 172.21.15.19:0/462433134 wait complete. 2024-03-28T13:10:41.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.680+0000 7f7fbb200640 1 Processor -- start 2024-03-28T13:10:41.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.680+0000 7f7fbb200640 1 -- start start 2024-03-28T13:10:41.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.681+0000 7f7fbb200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fbc0cf700 0x7f7fbc154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:41.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.681+0000 7f7fbb200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fbc0d2310 0x7f7fbc15f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:41.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.681+0000 7f7fbb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc15e4a0 0x7f7fbc1f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:41.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.681+0000 7f7fbb200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc0d3970 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.681+0000 7f7fbb200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7fbc0d3170 con 0x7f7fbc0d2310 2024-03-28T13:10:41.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.681+0000 7f7fbb200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7fbc0ba1a0 con 0x7f7fbc0cf700 2024-03-28T13:10:41.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.681+0000 7f7fba800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc15e4a0 0x7f7fbc1f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:41.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.681+0000 7f7fba800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc15e4a0 0x7f7fbc1f44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44254/0 (socket says 172.21.15.19:44254) 2024-03-28T13:10:41.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.681+0000 7f7fba800640 1 -- 172.21.15.19:0/3074557425 learned_addr learned my addr 172.21.15.19:0/3074557425 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:41.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.682+0000 7f7fb9e00640 1 --2- 172.21.15.19:0/3074557425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fbc0cf700 0x7f7fbc154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:41.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.682+0000 7f7fb1400640 1 --2- 172.21.15.19:0/3074557425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fbc0d2310 0x7f7fbc15f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:41.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.682+0000 7f7fba800640 1 -- 172.21.15.19:0/3074557425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fbc0cf700 msgr2=0x7f7fbc154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:41.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.682+0000 7f7fba800640 1 --2- 172.21.15.19:0/3074557425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fbc0cf700 0x7f7fbc154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.682+0000 7f7fba800640 1 -- 172.21.15.19:0/3074557425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fbc0d2310 msgr2=0x7f7fbc15f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:41.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.682+0000 7f7fba800640 1 --2- 172.21.15.19:0/3074557425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fbc0d2310 0x7f7fbc15f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.682+0000 7f7fba800640 1 -- 172.21.15.19:0/3074557425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fbc0cc0b0 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.682+0000 7f7fba800640 1 --2- 172.21.15.19:0/3074557425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc15e4a0 0x7f7fbc1f44b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400cd40 tx=0x7f7fb400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:41.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.682+0000 7f7fb2a00640 1 -- 172.21.15.19:0/3074557425 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4003b90 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.682+0000 7f7fb2a00640 1 -- 172.21.15.19:0/3074557425 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb4003d50 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.682+0000 7f7fb2a00640 1 -- 172.21.15.19:0/3074557425 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4004260 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.682+0000 7f7fbb200640 1 -- 172.21.15.19:0/3074557425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fbc0cc8d0 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.683+0000 7f7fbb200640 1 -- 172.21.15.19:0/3074557425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fbc15fdd0 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.684+0000 7f7fbb200640 1 -- 172.21.15.19:0/3074557425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fbc160090 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.685+0000 7f7fb2a00640 1 -- 172.21.15.19:0/3074557425 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7fb4004400 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.685+0000 7f7fb2a00640 1 --2- 172.21.15.19:0/3074557425 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f9806c530 0x7f7f9806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:41.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.686+0000 7f7fb2a00640 1 -- 172.21.15.19:0/3074557425 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f7fb408c250 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.686+0000 7f7fb9e00640 1 --2- 172.21.15.19:0/3074557425 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f9806c530 0x7f7f9806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:41.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.687+0000 7f7fb9e00640 1 --2- 172.21.15.19:0/3074557425 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f9806c530 0x7f7f9806e9f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7fac036000 tx=0x7f7fac03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:41.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.695+0000 7f7fb2a00640 1 -- 172.21.15.19:0/3074557425 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7fbc160090 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.982+0000 7f7fbb200640 1 -- 172.21.15.19:0/3074557425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fbc0c9a70 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.983+0000 7f7fb2a00640 1 -- 172.21.15.19:0/3074557425 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f7fbc0c9a70 con 0x7f7fbc15e4a0 2024-03-28T13:10:41.984 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:41.984 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/745791856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":745791856},{"type":"v1","addr":"172.21.15.19:6833","nonce":745791856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/639283827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":639283827},{"type":"v1","addr":"172.21.15.188:6836","nonce":639283827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2168352329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2168352329},{"type":"v1","addr":"172.21.15.188:6837","nonce":2168352329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:09:41.105429+0000","modified":"2024-03-28T13:09:44.203526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/731659331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":731659331},{"type":"v1","addr":"172.21.15.19:6835","nonce":731659331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4254,"qdb_cluster":[4254]},"id":1}]} 2024-03-28T13:10:41.984 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-03-28T13:10:41.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.986+0000 7f7fbb200640 1 -- 172.21.15.19:0/3074557425 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f9806c530 msgr2=0x7f7f9806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:41.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.986+0000 7f7fbb200640 1 --2- 172.21.15.19:0/3074557425 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f9806c530 0x7f7f9806e9f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7fac036000 tx=0x7f7fac03d040 comp rx=0 tx=0).stop 2024-03-28T13:10:41.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.986+0000 7f7fbb200640 1 -- 172.21.15.19:0/3074557425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc15e4a0 msgr2=0x7f7fbc1f44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:41.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.986+0000 7f7fbb200640 1 --2- 172.21.15.19:0/3074557425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc15e4a0 0x7f7fbc1f44b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400cd40 tx=0x7f7fb400a650 comp rx=0 tx=0).stop 2024-03-28T13:10:41.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.986+0000 7f7fbb200640 1 -- 172.21.15.19:0/3074557425 shutdown_connections 2024-03-28T13:10:41.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.986+0000 7f7fbb200640 1 --2- 172.21.15.19:0/3074557425 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f9806c530 0x7f7f9806e9f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.986+0000 7f7fbb200640 1 --2- 172.21.15.19:0/3074557425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fbc15e4a0 0x7f7fbc1f44b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.986+0000 7f7fbb200640 1 --2- 172.21.15.19:0/3074557425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fbc0d2310 0x7f7fbc15f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.986+0000 7f7fbb200640 1 --2- 172.21.15.19:0/3074557425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fbc0cf700 0x7f7fbc154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:41.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.986+0000 7f7fbb200640 1 -- 172.21.15.19:0/3074557425 >> 172.21.15.19:0/3074557425 conn(0x7f7fbc0676d0 msgr2=0x7f7fbc0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:41.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.986+0000 7f7fbb200640 1 -- 172.21.15.19:0/3074557425 shutdown_connections 2024-03-28T13:10:41.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:41.986+0000 7f7fbb200640 1 -- 172.21.15.19:0/3074557425 wait complete. 2024-03-28T13:10:42.007 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:10:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.210+0000 7fd04f400640 1 Processor -- start 2024-03-28T13:10:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.210+0000 7fd04f400640 1 -- start start 2024-03-28T13:10:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.211+0000 7fd04f400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd0480727f0 0x7fd0480b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.211+0000 7fd04f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0480d0780 0x7fd0480d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.211+0000 7fd04f400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd04815b6f0 con 0x7fd04815b310 2024-03-28T13:10:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.211+0000 7fd04f400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fd048072c50 con 0x7fd0480727f0 2024-03-28T13:10:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.212+0000 7fd04f400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd048072e80 con 0x7fd0480d0780 2024-03-28T13:10:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.212+0000 7fd04d400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0480d0780 0x7fd0480d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.212+0000 7fd047e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd0480727f0 0x7fd0480b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.212+0000 7fd04ca00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd04815b310 0x7fd04815f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:51548/0 (socket says 172.21.15.19:51548) 2024-03-28T13:10:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.212+0000 7fd04ca00640 1 -- 172.21.15.19:0/1492720159 learned_addr learned my addr 172.21.15.19:0/1492720159 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.213+0000 7fd047e00640 1 -- 172.21.15.19:0/1492720159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0480d0780 msgr2=0x7fd0480d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.213+0000 7fd047e00640 1 --2- 172.21.15.19:0/1492720159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0480d0780 0x7fd0480d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.213+0000 7fd047e00640 1 -- 172.21.15.19:0/1492720159 >> v1:172.21.15.188:6789/0 conn(0x7fd04815b310 legacy=0x7fd04815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.213+0000 7fd047e00640 1 -- 172.21.15.19:0/1492720159 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd048073160 con 0x7fd0480727f0 2024-03-28T13:10:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.213+0000 7fd04d400640 1 --2- 172.21.15.19:0/1492720159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0480d0780 0x7fd0480d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:10:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.215+0000 7fd047e00640 1 --2- 172.21.15.19:0/1492720159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0480727f0 0x7fd0480b4f90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd034009d90 tx=0x7fd03402bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b473db3972cb8bb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.215+0000 7fd047400640 1 -- 172.21.15.19:0/1492720159 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034039070 con 0x7fd0480727f0 2024-03-28T13:10:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.215+0000 7fd047400640 1 -- 172.21.15.19:0/1492720159 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd03402cb40 con 0x7fd0480727f0 2024-03-28T13:10:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.215+0000 7fd047400640 1 -- 172.21.15.19:0/1492720159 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034034440 con 0x7fd0480727f0 2024-03-28T13:10:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.216+0000 7fd04f400640 1 -- 172.21.15.19:0/1492720159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0480727f0 msgr2=0x7fd0480b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.216+0000 7fd04f400640 1 --2- 172.21.15.19:0/1492720159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0480727f0 0x7fd0480b4f90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd034009d90 tx=0x7fd03402bd90 comp rx=0 tx=0).stop 2024-03-28T13:10:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.216+0000 7fd04f400640 1 -- 172.21.15.19:0/1492720159 shutdown_connections 2024-03-28T13:10:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.216+0000 7fd04f400640 1 --2- 172.21.15.19:0/1492720159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0480727f0 0x7fd0480b4f90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.216+0000 7fd04f400640 1 --2- 172.21.15.19:0/1492720159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0480d0780 0x7fd0480d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.216+0000 7fd04f400640 1 -- 172.21.15.19:0/1492720159 >> 172.21.15.19:0/1492720159 conn(0x7fd04806b990 msgr2=0x7fd04806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.216+0000 7fd04f400640 1 -- 172.21.15.19:0/1492720159 shutdown_connections 2024-03-28T13:10:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.216+0000 7fd04f400640 1 -- 172.21.15.19:0/1492720159 wait complete. 2024-03-28T13:10:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.218+0000 7fd04f400640 1 Processor -- start 2024-03-28T13:10:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.218+0000 7fd04f400640 1 -- start start 2024-03-28T13:10:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.219+0000 7fd04f400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0480727f0 0x7fd0481526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.219+0000 7fd04f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0480d0780 0x7fd0480b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.219+0000 7fd04f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04815b310 0x7fd0480b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.219+0000 7fd04f400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd048073440 con 0x7fd04815b310 2024-03-28T13:10:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.219+0000 7fd04f400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd04815f2c0 con 0x7fd0480727f0 2024-03-28T13:10:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.219+0000 7fd04f400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd0480adc80 con 0x7fd0480d0780 2024-03-28T13:10:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.219+0000 7fd04d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04815b310 0x7fd0480b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.219+0000 7fd04d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04815b310 0x7fd0480b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44270/0 (socket says 172.21.15.19:44270) 2024-03-28T13:10:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.219+0000 7fd04d400640 1 -- 172.21.15.19:0/3052433823 learned_addr learned my addr 172.21.15.19:0/3052433823 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.220+0000 7fd04ca00640 1 --2- 172.21.15.19:0/3052433823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0480727f0 0x7fd0481526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.220+0000 7fd047e00640 1 --2- 172.21.15.19:0/3052433823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0480d0780 0x7fd0480b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.220+0000 7fd04d400640 1 -- 172.21.15.19:0/3052433823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0480d0780 msgr2=0x7fd0480b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.220+0000 7fd04d400640 1 --2- 172.21.15.19:0/3052433823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0480d0780 0x7fd0480b0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.220+0000 7fd04d400640 1 -- 172.21.15.19:0/3052433823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0480727f0 msgr2=0x7fd0481526f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.220+0000 7fd04d400640 1 --2- 172.21.15.19:0/3052433823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0480727f0 0x7fd0481526f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.220+0000 7fd04d400640 1 -- 172.21.15.19:0/3052433823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04809af40 con 0x7fd04815b310 2024-03-28T13:10:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.220+0000 7fd04ca00640 1 --2- 172.21.15.19:0/3052433823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0480727f0 0x7fd0481526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:10:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.220+0000 7fd04d400640 1 --2- 172.21.15.19:0/3052433823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04815b310 0x7fd0480b0ad0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd04000e930 tx=0x7fd04000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.221+0000 7fd044c00640 1 -- 172.21.15.19:0/3052433823 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040017070 con 0x7fd04815b310 2024-03-28T13:10:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.221+0000 7fd044c00640 1 -- 172.21.15.19:0/3052433823 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd04000ca00 con 0x7fd04815b310 2024-03-28T13:10:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.221+0000 7fd044c00640 1 -- 172.21.15.19:0/3052433823 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04000cdd0 con 0x7fd04815b310 2024-03-28T13:10:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.221+0000 7fd04f400640 1 -- 172.21.15.19:0/3052433823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0481555d0 con 0x7fd04815b310 2024-03-28T13:10:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.221+0000 7fd04f400640 1 -- 172.21.15.19:0/3052433823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd048155b90 con 0x7fd04815b310 2024-03-28T13:10:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.222+0000 7fd04f400640 1 -- 172.21.15.19:0/3052433823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0480685c0 con 0x7fd04815b310 2024-03-28T13:10:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.223+0000 7fd044c00640 1 -- 172.21.15.19:0/3052433823 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd040013470 con 0x7fd04815b310 2024-03-28T13:10:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.225+0000 7fd044c00640 1 --2- 172.21.15.19:0/3052433823 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd02806c530 0x7fd02806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:42.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.225+0000 7fd044c00640 1 -- 172.21.15.19:0/3052433823 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7fd04008b170 con 0x7fd04815b310 2024-03-28T13:10:42.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.225+0000 7fd04ca00640 1 --2- 172.21.15.19:0/3052433823 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd02806c530 0x7fd02806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:42.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.226+0000 7fd04ca00640 1 --2- 172.21.15.19:0/3052433823 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd02806c530 0x7fd02806e9f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd038005880 tx=0x7fd038009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:42.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.230+0000 7fd044c00640 1 -- 172.21.15.19:0/3052433823 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd04809af40 con 0x7fd04815b310 2024-03-28T13:10:42.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.439+0000 7fd04f400640 1 -- 172.21.15.19:0/3052433823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd04815ec20 con 0x7fd04815b310 2024-03-28T13:10:42.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.441+0000 7fd044c00640 1 -- 172.21.15.19:0/3052433823 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16704 (secure 0 0 0) 0x7fd0400591b0 con 0x7fd04815b310 2024-03-28T13:10:42.442 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:42.442 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:10:23.142768+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":0,"to":4}]},{"pgid":"3.1b","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:10:42.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.444+0000 7fd04f400640 1 -- 172.21.15.19:0/3052433823 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd02806c530 msgr2=0x7fd02806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:42.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.444+0000 7fd04f400640 1 --2- 172.21.15.19:0/3052433823 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd02806c530 0x7fd02806e9f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd038005880 tx=0x7fd038009040 comp rx=0 tx=0).stop 2024-03-28T13:10:42.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.444+0000 7fd04f400640 1 -- 172.21.15.19:0/3052433823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04815b310 msgr2=0x7fd0480b0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:42.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.444+0000 7fd04f400640 1 --2- 172.21.15.19:0/3052433823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04815b310 0x7fd0480b0ad0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd04000e930 tx=0x7fd04000edf0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.444+0000 7fd04f400640 1 -- 172.21.15.19:0/3052433823 shutdown_connections 2024-03-28T13:10:42.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.444+0000 7fd04f400640 1 --2- 172.21.15.19:0/3052433823 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd02806c530 0x7fd02806e9f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.444+0000 7fd04f400640 1 --2- 172.21.15.19:0/3052433823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd04815b310 0x7fd0480b0ad0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.444+0000 7fd04f400640 1 --2- 172.21.15.19:0/3052433823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0480d0780 0x7fd0480b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.444+0000 7fd04f400640 1 --2- 172.21.15.19:0/3052433823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0480727f0 0x7fd0481526f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.444+0000 7fd04f400640 1 -- 172.21.15.19:0/3052433823 >> 172.21.15.19:0/3052433823 conn(0x7fd04806b990 msgr2=0x7fd0480afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:42.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.445+0000 7fd04f400640 1 -- 172.21.15.19:0/3052433823 shutdown_connections 2024-03-28T13:10:42.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.445+0000 7fd04f400640 1 -- 172.21.15.19:0/3052433823 wait complete. 2024-03-28T13:10:42.466 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-28T13:10:42.466 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:10:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.704+0000 7f3f78800640 1 Processor -- start 2024-03-28T13:10:42.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.704+0000 7f3f78800640 1 -- start start 2024-03-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.705+0000 7f3f78800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3f7015e980 0x7f3f700cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.705+0000 7f3f78800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f700d4f10 0x7f3f700d2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.705+0000 7f3f78800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3f700d2f60 0x7f3f70162390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.705+0000 7f3f78800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3f700d2530 con 0x7f3f7015e980 2024-03-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.705+0000 7f3f78800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3f700d5370 con 0x7f3f700d2f60 2024-03-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.705+0000 7f3f78800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3f700cf630 con 0x7f3f700d4f10 2024-03-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.706+0000 7f3f75e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3f7015e980 0x7f3f700cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.706+0000 7f3f75e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3f7015e980 0x7f3f700cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44276/0 (socket says 172.21.15.19:44276) 2024-03-28T13:10:42.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.706+0000 7f3f75e00640 1 -- 172.21.15.19:0/1696836981 learned_addr learned my addr 172.21.15.19:0/1696836981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.706+0000 7f3f75e00640 1 -- 172.21.15.19:0/1696836981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f700d4f10 msgr2=0x7f3f700d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.706+0000 7f3f76800640 1 --2- 172.21.15.19:0/1696836981 >> v2:172.21.15.188:3300/0 conn(0x7f3f700d2f60 0x7f3f70162390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.706+0000 7f3f75400640 1 --2- 172.21.15.19:0/1696836981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f700d4f10 0x7f3f700d2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.706+0000 7f3f75e00640 1 --2- 172.21.15.19:0/1696836981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f700d4f10 0x7f3f700d2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.706+0000 7f3f75e00640 1 -- 172.21.15.19:0/1696836981 >> v2:172.21.15.188:3300/0 conn(0x7f3f700d2f60 msgr2=0x7f3f70162390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:42.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.706+0000 7f3f75e00640 1 --2- 172.21.15.19:0/1696836981 >> v2:172.21.15.188:3300/0 conn(0x7f3f700d2f60 0x7f3f70162390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.706+0000 7f3f75e00640 1 -- 172.21.15.19:0/1696836981 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f700cf910 con 0x7f3f7015e980 2024-03-28T13:10:42.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.706+0000 7f3f75400640 1 --2- 172.21.15.19:0/1696836981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f700d4f10 0x7f3f700d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:42.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.707+0000 7f3f75e00640 1 --2- 172.21.15.19:0/1696836981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f7015e980 0x7f3f700cefe0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3f700cf520 tx=0x7f3f6402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51335c98413c2779 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.708+0000 7f3f74a00640 1 -- 172.21.15.19:0/1696836981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64035020 con 0x7f3f7015e980 2024-03-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.708+0000 7f3f74a00640 1 -- 172.21.15.19:0/1696836981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f64031870 con 0x7f3f7015e980 2024-03-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.708+0000 7f3f74a00640 1 -- 172.21.15.19:0/1696836981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64031c40 con 0x7f3f7015e980 2024-03-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.708+0000 7f3f78800640 1 -- 172.21.15.19:0/1696836981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f7015e980 msgr2=0x7f3f700cefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.708+0000 7f3f78800640 1 --2- 172.21.15.19:0/1696836981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f7015e980 0x7f3f700cefe0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3f700cf520 tx=0x7f3f6402b910 comp rx=0 tx=0).stop 2024-03-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.709+0000 7f3f78800640 1 -- 172.21.15.19:0/1696836981 shutdown_connections 2024-03-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.709+0000 7f3f78800640 1 --2- 172.21.15.19:0/1696836981 >> v2:172.21.15.188:3300/0 conn(0x7f3f700d2f60 0x7f3f70162390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.709+0000 7f3f78800640 1 --2- 172.21.15.19:0/1696836981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f700d4f10 0x7f3f700d2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.709+0000 7f3f78800640 1 --2- 172.21.15.19:0/1696836981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f7015e980 0x7f3f700cefe0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.709+0000 7f3f78800640 1 -- 172.21.15.19:0/1696836981 >> 172.21.15.19:0/1696836981 conn(0x7f3f7008bfb0 msgr2=0x7f3f700d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:42.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.709+0000 7f3f78800640 1 -- 172.21.15.19:0/1696836981 shutdown_connections 2024-03-28T13:10:42.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.709+0000 7f3f78800640 1 -- 172.21.15.19:0/1696836981 wait complete. 2024-03-28T13:10:42.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.711+0000 7f3f78800640 1 Processor -- start 2024-03-28T13:10:42.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.711+0000 7f3f78800640 1 -- start start 2024-03-28T13:10:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.712+0000 7f3f78800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f700d2f60 0x7f3f7015b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.712+0000 7f3f78800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f700d4f10 0x7f3f701e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.712+0000 7f3f78800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f7015e980 0x7f3f700cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.712+0000 7f3f78800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3f700d01b0 con 0x7f3f700d2f60 2024-03-28T13:10:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.712+0000 7f3f78800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3f700cfed0 con 0x7f3f700d4f10 2024-03-28T13:10:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.712+0000 7f3f78800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3f701628d0 con 0x7f3f7015e980 2024-03-28T13:10:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.712+0000 7f3f75e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f700d2f60 0x7f3f7015b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.712+0000 7f3f75e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f700d2f60 0x7f3f7015b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44290/0 (socket says 172.21.15.19:44290) 2024-03-28T13:10:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.712+0000 7f3f75e00640 1 -- 172.21.15.19:0/812829217 learned_addr learned my addr 172.21.15.19:0/812829217 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.713+0000 7f3f75400640 1 --2- 172.21.15.19:0/812829217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f700d4f10 0x7f3f701e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.713+0000 7f3f75e00640 1 -- 172.21.15.19:0/812829217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f7015e980 msgr2=0x7f3f700cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.713+0000 7f3f75e00640 1 --2- 172.21.15.19:0/812829217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f7015e980 0x7f3f700cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.713+0000 7f3f75e00640 1 -- 172.21.15.19:0/812829217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f700d4f10 msgr2=0x7f3f701e7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.713+0000 7f3f75e00640 1 --2- 172.21.15.19:0/812829217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f700d4f10 0x7f3f701e7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.713+0000 7f3f75e00640 1 -- 172.21.15.19:0/812829217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f700cc660 con 0x7f3f700d2f60 2024-03-28T13:10:42.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.713+0000 7f3f75e00640 1 --2- 172.21.15.19:0/812829217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f700d2f60 0x7f3f7015b4b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3f6402b950 tx=0x7f3f640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:42.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.713+0000 7f3f6a000640 1 -- 172.21.15.19:0/812829217 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64040070 con 0x7f3f700d2f60 2024-03-28T13:10:42.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.713+0000 7f3f6a000640 1 -- 172.21.15.19:0/812829217 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6403a420 con 0x7f3f700d2f60 2024-03-28T13:10:42.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.713+0000 7f3f78800640 1 -- 172.21.15.19:0/812829217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f700cc940 con 0x7f3f700d2f60 2024-03-28T13:10:42.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.713+0000 7f3f6a000640 1 -- 172.21.15.19:0/812829217 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f64035040 con 0x7f3f700d2f60 2024-03-28T13:10:42.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.714+0000 7f3f78800640 1 -- 172.21.15.19:0/812829217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f700ccf00 con 0x7f3f700d2f60 2024-03-28T13:10:42.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.715+0000 7f3f78800640 1 -- 172.21.15.19:0/812829217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f700ce400 con 0x7f3f700d2f60 2024-03-28T13:10:42.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.716+0000 7f3f6a000640 1 -- 172.21.15.19:0/812829217 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3f6404b060 con 0x7f3f700d2f60 2024-03-28T13:10:42.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.717+0000 7f3f6a000640 1 --2- 172.21.15.19:0/812829217 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3f5406c530 0x7f3f5406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:42.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.717+0000 7f3f6a000640 1 -- 172.21.15.19:0/812829217 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f3f640ad640 con 0x7f3f700d2f60 2024-03-28T13:10:42.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.718+0000 7f3f75400640 1 --2- 172.21.15.19:0/812829217 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3f5406c530 0x7f3f5406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:42.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.719+0000 7f3f75400640 1 --2- 172.21.15.19:0/812829217 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3f5406c530 0x7f3f5406e9f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3f70161ce0 tx=0x7f3f60008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:42.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:42.722+0000 7f3f6a000640 1 -- 172.21.15.19:0/812829217 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3f700ce400 con 0x7f3f700d2f60 2024-03-28T13:10:43.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.081+0000 7f3f78800640 1 -- 172.21.15.19:0/812829217 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f700685c0 con 0x7f3f700d2f60 2024-03-28T13:10:43.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.082+0000 7f3f6a000640 1 -- 172.21.15.19:0/812829217 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7f3f6407bc20 con 0x7f3f700d2f60 2024-03-28T13:10:43.082 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:43.082 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/745791856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":745791856},{"type":"v1","addr":"172.21.15.19:6833","nonce":745791856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/639283827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":639283827},{"type":"v1","addr":"172.21.15.188:6836","nonce":639283827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2168352329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2168352329},{"type":"v1","addr":"172.21.15.188:6837","nonce":2168352329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:09:41.105429+0000","modified":"2024-03-28T13:09:44.203526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/731659331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":731659331},{"type":"v1","addr":"172.21.15.19:6835","nonce":731659331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4254,"qdb_cluster":[4254]},"id":1}]} 2024-03-28T13:10:43.082 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-03-28T13:10:43.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.084+0000 7f3f78800640 1 -- 172.21.15.19:0/812829217 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3f5406c530 msgr2=0x7f3f5406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:43.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.084+0000 7f3f78800640 1 --2- 172.21.15.19:0/812829217 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3f5406c530 0x7f3f5406e9f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3f70161ce0 tx=0x7f3f60008040 comp rx=0 tx=0).stop 2024-03-28T13:10:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.084+0000 7f3f78800640 1 -- 172.21.15.19:0/812829217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f700d2f60 msgr2=0x7f3f7015b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.084+0000 7f3f78800640 1 --2- 172.21.15.19:0/812829217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f700d2f60 0x7f3f7015b4b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3f6402b950 tx=0x7f3f640046c0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.084+0000 7f3f78800640 1 -- 172.21.15.19:0/812829217 shutdown_connections 2024-03-28T13:10:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.084+0000 7f3f78800640 1 --2- 172.21.15.19:0/812829217 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3f5406c530 0x7f3f5406e9f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.084+0000 7f3f78800640 1 --2- 172.21.15.19:0/812829217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f7015e980 0x7f3f700cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.084+0000 7f3f78800640 1 --2- 172.21.15.19:0/812829217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f700d4f10 0x7f3f701e7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.084+0000 7f3f78800640 1 --2- 172.21.15.19:0/812829217 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3f700d2f60 0x7f3f7015b4b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.084+0000 7f3f78800640 1 -- 172.21.15.19:0/812829217 >> 172.21.15.19:0/812829217 conn(0x7f3f7008bfb0 msgr2=0x7f3f700c7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:43.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.084+0000 7f3f78800640 1 -- 172.21.15.19:0/812829217 shutdown_connections 2024-03-28T13:10:43.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.084+0000 7f3f78800640 1 -- 172.21.15.19:0/812829217 wait complete. 2024-03-28T13:10:43.104 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:10:43.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.318+0000 7fa72e400640 1 Processor -- start 2024-03-28T13:10:43.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.319+0000 7fa72e400640 1 -- start start 2024-03-28T13:10:43.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.320+0000 7fa72e400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280d4840 0x7fa7280cff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:43.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.320+0000 7fa72e400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa7280d0f20 0x7fa7280d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:43.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.320+0000 7fa72e400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7280d1c90 con 0x7fa72815c2c0 2024-03-28T13:10:43.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.320+0000 7fa72e400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa7280cf160 con 0x7fa7280d0f20 2024-03-28T13:10:43.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.320+0000 7fa72e400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa7280cf440 con 0x7fa7280d4840 2024-03-28T13:10:43.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.320+0000 7fa727400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa72815c2c0 0x7fa7280d4130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37074/0 (socket says 172.21.15.19:37074) 2024-03-28T13:10:43.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.320+0000 7fa727400640 1 -- 172.21.15.19:0/2461363426 learned_addr learned my addr 172.21.15.19:0/2461363426 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:43.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.321+0000 7fa71fe00640 1 --2- 172.21.15.19:0/2461363426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280d4840 0x7fa7280cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:43.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.321+0000 7fa727e00640 1 --2- 172.21.15.19:0/2461363426 >> v2:172.21.15.188:3300/0 conn(0x7fa7280d0f20 0x7fa7280d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:43.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.321+0000 7fa726a00640 1 -- 172.21.15.19:0/2461363426 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4094124267 0 0) 0x7fa7280d1c90 con 0x7fa72815c2c0 2024-03-28T13:10:43.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.321+0000 7fa726a00640 1 -- 172.21.15.19:0/2461363426 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa70c003520 con 0x7fa72815c2c0 2024-03-28T13:10:43.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.321+0000 7fa71fe00640 1 -- 172.21.15.19:0/2461363426 >> v2:172.21.15.188:3300/0 conn(0x7fa7280d0f20 msgr2=0x7fa7280d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:43.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.321+0000 7fa726a00640 1 -- 172.21.15.19:0/2461363426 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 277243909 0 0) 0x7fa70c003520 con 0x7fa72815c2c0 2024-03-28T13:10:43.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.321+0000 7fa71fe00640 1 --2- 172.21.15.19:0/2461363426 >> v2:172.21.15.188:3300/0 conn(0x7fa7280d0f20 0x7fa7280d13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.321+0000 7fa71fe00640 1 -- 172.21.15.19:0/2461363426 >> v1:172.21.15.19:6789/0 conn(0x7fa72815c2c0 legacy=0x7fa7280d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:43.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.321+0000 7fa71fe00640 1 -- 172.21.15.19:0/2461363426 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7280cf720 con 0x7fa7280d4840 2024-03-28T13:10:43.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.323+0000 7fa71fe00640 1 --2- 172.21.15.19:0/2461363426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280d4840 0x7fa7280cff20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa7100097a0 tx=0x7fa71002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddce78b8c579047c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:43.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.323+0000 7fa726a00640 1 -- 172.21.15.19:0/2461363426 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710037070 con 0x7fa7280d4840 2024-03-28T13:10:43.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.323+0000 7fa726a00640 1 -- 172.21.15.19:0/2461363426 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa710030510 con 0x7fa7280d4840 2024-03-28T13:10:43.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.323+0000 7fa726a00640 1 -- 172.21.15.19:0/2461363426 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718001670 con 0x7fa7280d4840 2024-03-28T13:10:43.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.324+0000 7fa72e400640 1 -- 172.21.15.19:0/2461363426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280d4840 msgr2=0x7fa7280cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:43.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.324+0000 7fa72e400640 1 --2- 172.21.15.19:0/2461363426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280d4840 0x7fa7280cff20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa7100097a0 tx=0x7fa71002b790 comp rx=0 tx=0).stop 2024-03-28T13:10:43.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.324+0000 7fa72e400640 1 -- 172.21.15.19:0/2461363426 shutdown_connections 2024-03-28T13:10:43.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.324+0000 7fa72e400640 1 --2- 172.21.15.19:0/2461363426 >> v2:172.21.15.188:3300/0 conn(0x7fa7280d0f20 0x7fa7280d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.324+0000 7fa72e400640 1 --2- 172.21.15.19:0/2461363426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280d4840 0x7fa7280cff20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.324+0000 7fa72e400640 1 -- 172.21.15.19:0/2461363426 >> 172.21.15.19:0/2461363426 conn(0x7fa72808bfb0 msgr2=0x7fa72809d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:43.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.324+0000 7fa72e400640 1 -- 172.21.15.19:0/2461363426 shutdown_connections 2024-03-28T13:10:43.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.324+0000 7fa72e400640 1 -- 172.21.15.19:0/2461363426 wait complete. 2024-03-28T13:10:43.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.325+0000 7fa72e400640 1 Processor -- start 2024-03-28T13:10:43.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.326+0000 7fa72e400640 1 -- start start 2024-03-28T13:10:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.326+0000 7fa72e400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7280d0f20 0x7fa72806df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.326+0000 7fa72e400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280d4840 0x7fa72814b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.326+0000 7fa72e400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72815c2c0 0x7fa7280b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.326+0000 7fa72e400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7280d1a70 con 0x7fa7280d0f20 2024-03-28T13:10:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.326+0000 7fa72e400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa7280d18f0 con 0x7fa72815c2c0 2024-03-28T13:10:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.326+0000 7fa72e400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa7280d1ff0 con 0x7fa7280d4840 2024-03-28T13:10:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.327+0000 7fa727400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7280d0f20 0x7fa72806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.327+0000 7fa727400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7280d0f20 0x7fa72806df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44292/0 (socket says 172.21.15.19:44292) 2024-03-28T13:10:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.327+0000 7fa727400640 1 -- 172.21.15.19:0/457621057 learned_addr learned my addr 172.21.15.19:0/457621057 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.327+0000 7fa71fe00640 1 --2- 172.21.15.19:0/457621057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280d4840 0x7fa72814b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.327+0000 7fa727e00640 1 --2- 172.21.15.19:0/457621057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72815c2c0 0x7fa7280b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.327+0000 7fa727400640 1 -- 172.21.15.19:0/457621057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280d4840 msgr2=0x7fa72814b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.327+0000 7fa727400640 1 --2- 172.21.15.19:0/457621057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280d4840 0x7fa72814b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.327+0000 7fa727400640 1 -- 172.21.15.19:0/457621057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72815c2c0 msgr2=0x7fa7280b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:43.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.327+0000 7fa727400640 1 --2- 172.21.15.19:0/457621057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72815c2c0 0x7fa7280b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.327+0000 7fa727400640 1 -- 172.21.15.19:0/457621057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7280cdec0 con 0x7fa7280d0f20 2024-03-28T13:10:43.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.328+0000 7fa727400640 1 --2- 172.21.15.19:0/457621057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7280d0f20 0x7fa72806df10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa7180017e0 tx=0x7fa71800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:43.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.328+0000 7fa71f400640 1 -- 172.21.15.19:0/457621057 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718005ad0 con 0x7fa7280d0f20 2024-03-28T13:10:43.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.328+0000 7fa71f400640 1 -- 172.21.15.19:0/457621057 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa718005c90 con 0x7fa7280d0f20 2024-03-28T13:10:43.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.328+0000 7fa71f400640 1 -- 172.21.15.19:0/457621057 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7180072b0 con 0x7fa7280d0f20 2024-03-28T13:10:43.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.328+0000 7fa72e400640 1 -- 172.21.15.19:0/457621057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7280ce160 con 0x7fa7280d0f20 2024-03-28T13:10:43.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.328+0000 7fa72e400640 1 -- 172.21.15.19:0/457621057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7280c84c0 con 0x7fa7280d0f20 2024-03-28T13:10:43.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.329+0000 7fa72e400640 1 -- 172.21.15.19:0/457621057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7280685c0 con 0x7fa7280d0f20 2024-03-28T13:10:43.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.330+0000 7fa71f400640 1 -- 172.21.15.19:0/457621057 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa718007450 con 0x7fa7280d0f20 2024-03-28T13:10:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.331+0000 7fa71f400640 1 --2- 172.21.15.19:0/457621057 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa70406c530 0x7fa70406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.331+0000 7fa71f400640 1 -- 172.21.15.19:0/457621057 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7fa71808b4c0 con 0x7fa7280d0f20 2024-03-28T13:10:43.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.332+0000 7fa71fe00640 1 --2- 172.21.15.19:0/457621057 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa70406c530 0x7fa70406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:43.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.333+0000 7fa71fe00640 1 --2- 172.21.15.19:0/457621057 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa70406c530 0x7fa70406e9f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa710034000 tx=0x7fa71002f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:43.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.336+0000 7fa71f400640 1 -- 172.21.15.19:0/457621057 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa718010300 con 0x7fa7280d0f20 2024-03-28T13:10:43.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.575+0000 7fa72e400640 1 -- 172.21.15.19:0/457621057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa72806dcf0 con 0x7fa7280d0f20 2024-03-28T13:10:43.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.576+0000 7fa71f400640 1 -- 172.21.15.19:0/457621057 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5172 (secure 0 0 0) 0x7fa718059a90 con 0x7fa7280d0f20 2024-03-28T13:10:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:43.577 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/745791856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":745791856},{"type":"v1","addr":"172.21.15.19:6833","nonce":745791856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4250,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/639283827","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":639283827},{"type":"v1","addr":"172.21.15.188:6836","nonce":639283827}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/2168352329","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":2168352329},{"type":"v1","addr":"172.21.15.188:6837","nonce":2168352329}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:09:41.105429+0000","modified":"2024-03-28T13:09:44.203526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/731659331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":731659331},{"type":"v1","addr":"172.21.15.19:6835","nonce":731659331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4254,"qdb_cluster":[4254]},"id":1}]} 2024-03-28T13:10:43.577 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 8 2024-03-28T13:10:43.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.579+0000 7fa72e400640 1 -- 172.21.15.19:0/457621057 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa70406c530 msgr2=0x7fa70406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:43.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.579+0000 7fa72e400640 1 --2- 172.21.15.19:0/457621057 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa70406c530 0x7fa70406e9f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa710034000 tx=0x7fa71002f000 comp rx=0 tx=0).stop 2024-03-28T13:10:43.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.579+0000 7fa72e400640 1 -- 172.21.15.19:0/457621057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7280d0f20 msgr2=0x7fa72806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:43.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.579+0000 7fa72e400640 1 --2- 172.21.15.19:0/457621057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7280d0f20 0x7fa72806df10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa7180017e0 tx=0x7fa71800cbe0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.579+0000 7fa72e400640 1 -- 172.21.15.19:0/457621057 shutdown_connections 2024-03-28T13:10:43.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.579+0000 7fa72e400640 1 --2- 172.21.15.19:0/457621057 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa70406c530 0x7fa70406e9f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.579+0000 7fa72e400640 1 --2- 172.21.15.19:0/457621057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72815c2c0 0x7fa7280b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.579+0000 7fa72e400640 1 --2- 172.21.15.19:0/457621057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7280d4840 0x7fa72814b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.579+0000 7fa72e400640 1 --2- 172.21.15.19:0/457621057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7280d0f20 0x7fa72806df10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.579+0000 7fa72e400640 1 -- 172.21.15.19:0/457621057 >> 172.21.15.19:0/457621057 conn(0x7fa72808bfb0 msgr2=0x7fa7280b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:43.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.579+0000 7fa72e400640 1 -- 172.21.15.19:0/457621057 shutdown_connections 2024-03-28T13:10:43.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.579+0000 7fa72e400640 1 -- 172.21.15.19:0/457621057 wait complete. 2024-03-28T13:10:43.599 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:10:43.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.806+0000 7fe148600640 1 Processor -- start 2024-03-28T13:10:43.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.806+0000 7fe148600640 1 -- start start 2024-03-28T13:10:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.808+0000 7fe148600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1400af830 0x7fe140073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.808+0000 7fe148600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe14015fe80 con 0x7fe1400d0650 2024-03-28T13:10:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.808+0000 7fe148600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1400ad120 con 0x7fe14015c390 2024-03-28T13:10:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.808+0000 7fe148600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe14015c770 con 0x7fe1400af830 2024-03-28T13:10:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.808+0000 7fe146600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe1400d0650 0x7fe140071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37088/0 (socket says 172.21.15.19:37088) 2024-03-28T13:10:43.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.808+0000 7fe146600640 1 -- 172.21.15.19:0/3693091171 learned_addr learned my addr 172.21.15.19:0/3693091171 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe145200640 1 --2- 172.21.15.19:0/3693091171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1400af830 0x7fe140073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe13fe00640 1 -- 172.21.15.19:0/3693091171 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3659399163 0 0) 0x7fe14015fe80 con 0x7fe1400d0650 2024-03-28T13:10:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe13fe00640 1 -- 172.21.15.19:0/3693091171 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe128003520 con 0x7fe1400d0650 2024-03-28T13:10:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe13fe00640 1 -- 172.21.15.19:0/3693091171 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 343637985 0 0) 0x7fe128003520 con 0x7fe1400d0650 2024-03-28T13:10:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe13fe00640 1 -- 172.21.15.19:0/3693091171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1400af830 msgr2=0x7fe140073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe13fe00640 1 --2- 172.21.15.19:0/3693091171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1400af830 0x7fe140073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe13fe00640 1 -- 172.21.15.19:0/3693091171 >> v1:172.21.15.188:6789/0 conn(0x7fe14015c390 legacy=0x7fe140160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe13fe00640 1 -- 172.21.15.19:0/3693091171 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1400ad400 con 0x7fe1400d0650 2024-03-28T13:10:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe13fe00640 1 -- 172.21.15.19:0/3693091171 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fe138001670 con 0x7fe1400d0650 2024-03-28T13:10:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe13fe00640 1 -- 172.21.15.19:0/3693091171 >> v1:172.21.15.19:6789/0 conn(0x7fe1400d0650 legacy=0x7fe140071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe13fe00640 1 --2- 172.21.15.19:0/3693091171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1400af830 0x7fe1280039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe13fe00640 1 --2- 172.21.15.19:0/3693091171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe128003fb0 0x7fe128004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe13fe00640 1 --2- 172.21.15.19:0/3693091171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe128004d40 0x7fe1280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe13fe00640 1 -- 172.21.15.19:0/3693091171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe12800bad0 con 0x7fe128003fb0 2024-03-28T13:10:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe13fe00640 1 -- 172.21.15.19:0/3693091171 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe12800bc80 con 0x7fe128004d40 2024-03-28T13:10:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.809+0000 7fe13fe00640 1 -- 172.21.15.19:0/3693091171 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe12800be80 con 0x7fe1400af830 2024-03-28T13:10:43.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.810+0000 7fe145200640 1 --2- 172.21.15.19:0/3693091171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe128003fb0 0x7fe128004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.810+0000 7fe145c00640 1 --2- 172.21.15.19:0/3693091171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1400af830 0x7fe1280039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.810+0000 7fe145200640 1 -- 172.21.15.19:0/3693091171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1400af830 msgr2=0x7fe1280039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.810+0000 7fe145200640 1 --2- 172.21.15.19:0/3693091171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1400af830 0x7fe1280039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.810+0000 7fe145200640 1 -- 172.21.15.19:0/3693091171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe128004d40 msgr2=0x7fe1280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.810+0000 7fe145200640 1 --2- 172.21.15.19:0/3693091171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe128004d40 0x7fe1280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.810+0000 7fe145200640 1 -- 172.21.15.19:0/3693091171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe128003520 con 0x7fe128003fb0 2024-03-28T13:10:43.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.810+0000 7fe145c00640 1 --2- 172.21.15.19:0/3693091171 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe1400af830 0x7fe1280039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.811+0000 7fe145200640 1 --2- 172.21.15.19:0/3693091171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe128003fb0 0x7fe128004800 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009ac0 tx=0x7fe128005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd464e88309dfa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.811+0000 7fe13fe00640 1 -- 172.21.15.19:0/3693091171 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c039070 con 0x7fe128003fb0 2024-03-28T13:10:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.812+0000 7fe13fe00640 1 -- 172.21.15.19:0/3693091171 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe12c02c600 con 0x7fe128003fb0 2024-03-28T13:10:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.812+0000 7fe13fe00640 1 -- 172.21.15.19:0/3693091171 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c009690 con 0x7fe128003fb0 2024-03-28T13:10:43.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.812+0000 7fe148600640 1 -- 172.21.15.19:0/3693091171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe128003fb0 msgr2=0x7fe128004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.812+0000 7fe148600640 1 --2- 172.21.15.19:0/3693091171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe128003fb0 0x7fe128004800 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe12c009ac0 tx=0x7fe128005830 comp rx=0 tx=0).stop 2024-03-28T13:10:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.812+0000 7fe146600640 1 -- 172.21.15.19:0/3693091171 reap_dead start 2024-03-28T13:10:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.812+0000 7fe148600640 1 -- 172.21.15.19:0/3693091171 shutdown_connections 2024-03-28T13:10:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.812+0000 7fe148600640 1 -- 172.21.15.19:0/3693091171 >> 172.21.15.19:0/3693091171 conn(0x7fe14006b990 msgr2=0x7fe14006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.812+0000 7fe148600640 1 -- 172.21.15.19:0/3693091171 shutdown_connections 2024-03-28T13:10:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.812+0000 7fe148600640 1 -- 172.21.15.19:0/3693091171 wait complete. 2024-03-28T13:10:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.814+0000 7fe148600640 1 Processor -- start 2024-03-28T13:10:43.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.814+0000 7fe148600640 1 -- start start 2024-03-28T13:10:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.815+0000 7fe148600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe128003fb0 0x7fe1401514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.815+0000 7fe148600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe140099b00 0x7fe14014e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.815+0000 7fe148600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1400afce0 0x7fe1400b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.815+0000 7fe148600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe140160d50 con 0x7fe140099b00 2024-03-28T13:10:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.815+0000 7fe148600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe140160120 con 0x7fe1400afce0 2024-03-28T13:10:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.815+0000 7fe148600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe1400cf980 con 0x7fe128003fb0 2024-03-28T13:10:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.815+0000 7fe145200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe140099b00 0x7fe14014e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.815+0000 7fe145200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe140099b00 0x7fe14014e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44306/0 (socket says 172.21.15.19:44306) 2024-03-28T13:10:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.815+0000 7fe145200640 1 -- 172.21.15.19:0/3055450768 learned_addr learned my addr 172.21.15.19:0/3055450768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.816+0000 7fe145c00640 1 --2- 172.21.15.19:0/3055450768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe128003fb0 0x7fe1401514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.816+0000 7fe146600640 1 --2- 172.21.15.19:0/3055450768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1400afce0 0x7fe1400b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:43.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.816+0000 7fe145200640 1 -- 172.21.15.19:0/3055450768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe128003fb0 msgr2=0x7fe1401514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.816+0000 7fe145200640 1 --2- 172.21.15.19:0/3055450768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe128003fb0 0x7fe1401514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.816+0000 7fe145200640 1 -- 172.21.15.19:0/3055450768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1400afce0 msgr2=0x7fe1400b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.816+0000 7fe145200640 1 --2- 172.21.15.19:0/3055450768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1400afce0 0x7fe1400b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.816+0000 7fe145200640 1 -- 172.21.15.19:0/3055450768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe140151ab0 con 0x7fe140099b00 2024-03-28T13:10:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.816+0000 7fe145200640 1 --2- 172.21.15.19:0/3055450768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe140099b00 0x7fe14014e000 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe12c037040 tx=0x7fe12c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.817+0000 7fe13d600640 1 -- 172.21.15.19:0/3055450768 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c042020 con 0x7fe140099b00 2024-03-28T13:10:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.817+0000 7fe13d600640 1 -- 172.21.15.19:0/3055450768 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe12c03a040 con 0x7fe140099b00 2024-03-28T13:10:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.817+0000 7fe13d600640 1 -- 172.21.15.19:0/3055450768 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c039070 con 0x7fe140099b00 2024-03-28T13:10:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.817+0000 7fe148600640 1 -- 172.21.15.19:0/3055450768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe140151d90 con 0x7fe140099b00 2024-03-28T13:10:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.817+0000 7fe148600640 1 -- 172.21.15.19:0/3055450768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe140152350 con 0x7fe140099b00 2024-03-28T13:10:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.818+0000 7fe148600640 1 -- 172.21.15.19:0/3055450768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1400b06a0 con 0x7fe140099b00 2024-03-28T13:10:43.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.819+0000 7fe13d600640 1 -- 172.21.15.19:0/3055450768 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe12c045020 con 0x7fe140099b00 2024-03-28T13:10:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.820+0000 7fe13d600640 1 --2- 172.21.15.19:0/3055450768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe12006c530 0x7fe12006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.820+0000 7fe13d600640 1 -- 172.21.15.19:0/3055450768 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7fe12c0acba0 con 0x7fe140099b00 2024-03-28T13:10:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.821+0000 7fe145c00640 1 --2- 172.21.15.19:0/3055450768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe12006c530 0x7fe12006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:43.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.822+0000 7fe145c00640 1 --2- 172.21.15.19:0/3055450768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe12006c530 0x7fe12006e9f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe134001e40 tx=0x7fe134009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:43.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:43.825+0000 7fe13d600640 1 -- 172.21.15.19:0/3055450768 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe12c04c070 con 0x7fe140099b00 2024-03-28T13:10:44.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.029+0000 7fe148600640 1 -- 172.21.15.19:0/3055450768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1400685c0 con 0x7fe140099b00 2024-03-28T13:10:44.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.032+0000 7fe13d600640 1 -- 172.21.15.19:0/3055450768 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+16704 (secure 0 0 0) 0x7fe12c07b170 con 0x7fe140099b00 2024-03-28T13:10:44.032 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:44.033 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":25,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:10:23.142768+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:09:39.243813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-28T13:09:40.092899+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":0,"to":4}]},{"pgid":"3.1b","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:10:44.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.034+0000 7fe148600640 1 -- 172.21.15.19:0/3055450768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe12006c530 msgr2=0x7fe12006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:44.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.034+0000 7fe148600640 1 --2- 172.21.15.19:0/3055450768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe12006c530 0x7fe12006e9f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe134001e40 tx=0x7fe134009040 comp rx=0 tx=0).stop 2024-03-28T13:10:44.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.034+0000 7fe148600640 1 -- 172.21.15.19:0/3055450768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe140099b00 msgr2=0x7fe14014e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:44.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.034+0000 7fe148600640 1 --2- 172.21.15.19:0/3055450768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe140099b00 0x7fe14014e000 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe12c037040 tx=0x7fe12c004730 comp rx=0 tx=0).stop 2024-03-28T13:10:44.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.034+0000 7fe148600640 1 -- 172.21.15.19:0/3055450768 shutdown_connections 2024-03-28T13:10:44.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.035+0000 7fe148600640 1 --2- 172.21.15.19:0/3055450768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe12006c530 0x7fe12006e9f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.035+0000 7fe148600640 1 --2- 172.21.15.19:0/3055450768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe1400afce0 0x7fe1400b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.035+0000 7fe148600640 1 --2- 172.21.15.19:0/3055450768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe140099b00 0x7fe14014e000 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.035+0000 7fe148600640 1 --2- 172.21.15.19:0/3055450768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe128003fb0 0x7fe1401514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.035+0000 7fe148600640 1 -- 172.21.15.19:0/3055450768 >> 172.21.15.19:0/3055450768 conn(0x7fe14006b990 msgr2=0x7fe1400b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:44.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.035+0000 7fe148600640 1 -- 172.21.15.19:0/3055450768 shutdown_connections 2024-03-28T13:10:44.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.035+0000 7fe148600640 1 -- 172.21.15.19:0/3055450768 wait complete. 2024-03-28T13:10:44.054 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T13:10:44.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.264+0000 7f33f8e00640 1 Processor -- start 2024-03-28T13:10:44.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.265+0000 7f33f8e00640 1 -- start start 2024-03-28T13:10:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.268+0000 7f33f8e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f33f4146f60 0x7f33f40ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.268+0000 7f33f8e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f33f4147570 0x7f33f406b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.268+0000 7f33f8e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f406bab0 0x7f33f406f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.268+0000 7f33f8e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f33f40cf030 con 0x7f33f4146f60 2024-03-28T13:10:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.268+0000 7f33f8e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f33f406e320 con 0x7f33f4147570 2024-03-28T13:10:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.268+0000 7f33f8e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f33f406ebc0 con 0x7f33f406bab0 2024-03-28T13:10:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.268+0000 7f33f3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f33f4146f60 0x7f33f40ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:44.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.268+0000 7f33f3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f33f4146f60 0x7f33f40ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44316/0 (socket says 172.21.15.19:44316) 2024-03-28T13:10:44.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.268+0000 7f33f3400640 1 -- 172.21.15.19:0/2770793485 learned_addr learned my addr 172.21.15.19:0/2770793485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:44.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.268+0000 7f33eaa00640 1 --2- 172.21.15.19:0/2770793485 >> v2:172.21.15.188:3300/0 conn(0x7f33f4147570 0x7f33f406b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:44.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.268+0000 7f33f3e00640 1 --2- 172.21.15.19:0/2770793485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f406bab0 0x7f33f406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:44.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.269+0000 7f33f3400640 1 -- 172.21.15.19:0/2770793485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f406bab0 msgr2=0x7f33f406f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:44.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.269+0000 7f33f3400640 1 --2- 172.21.15.19:0/2770793485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f406bab0 0x7f33f406f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.269+0000 7f33f3400640 1 -- 172.21.15.19:0/2770793485 >> v2:172.21.15.188:3300/0 conn(0x7f33f4147570 msgr2=0x7f33f406b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:44.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.269+0000 7f33f3400640 1 --2- 172.21.15.19:0/2770793485 >> v2:172.21.15.188:3300/0 conn(0x7f33f4147570 0x7f33f406b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.269+0000 7f33f3400640 1 -- 172.21.15.19:0/2770793485 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f4147340 con 0x7f33f4146f60 2024-03-28T13:10:44.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.271+0000 7f33f3400640 1 --2- 172.21.15.19:0/2770793485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f4146f60 0x7f33f40ceaf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f33f406f3f0 tx=0x7f33e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9026636350340d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:44.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.271+0000 7f33f2a00640 1 -- 172.21.15.19:0/2770793485 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4035020 con 0x7f33f4146f60 2024-03-28T13:10:44.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.271+0000 7f33f2a00640 1 -- 172.21.15.19:0/2770793485 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e4031870 con 0x7f33f4146f60 2024-03-28T13:10:44.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.271+0000 7f33f2a00640 1 -- 172.21.15.19:0/2770793485 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4031c40 con 0x7f33f4146f60 2024-03-28T13:10:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.272+0000 7f33f8e00640 1 -- 172.21.15.19:0/2770793485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f4146f60 msgr2=0x7f33f40ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.272+0000 7f33f8e00640 1 --2- 172.21.15.19:0/2770793485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f4146f60 0x7f33f40ceaf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f33f406f3f0 tx=0x7f33e402b910 comp rx=0 tx=0).stop 2024-03-28T13:10:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.272+0000 7f33f8e00640 1 -- 172.21.15.19:0/2770793485 shutdown_connections 2024-03-28T13:10:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.272+0000 7f33f8e00640 1 --2- 172.21.15.19:0/2770793485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f406bab0 0x7f33f406f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.272+0000 7f33f8e00640 1 --2- 172.21.15.19:0/2770793485 >> v2:172.21.15.188:3300/0 conn(0x7f33f4147570 0x7f33f406b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.272+0000 7f33f8e00640 1 --2- 172.21.15.19:0/2770793485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f4146f60 0x7f33f40ceaf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.272+0000 7f33f8e00640 1 -- 172.21.15.19:0/2770793485 >> 172.21.15.19:0/2770793485 conn(0x7f33f40b2c90 msgr2=0x7f33f40b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.272+0000 7f33f8e00640 1 -- 172.21.15.19:0/2770793485 shutdown_connections 2024-03-28T13:10:44.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.272+0000 7f33f8e00640 1 -- 172.21.15.19:0/2770793485 wait complete. 2024-03-28T13:10:44.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.275+0000 7f33f8e00640 1 Processor -- start 2024-03-28T13:10:44.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.275+0000 7f33f8e00640 1 -- start start 2024-03-28T13:10:44.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.277+0000 7f33f8e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f406bab0 0x7f33f41426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:44.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.277+0000 7f33f8e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f4146f60 0x7f33f41e4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:44.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.277+0000 7f33f8e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f4147570 0x7f33f4145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:44.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.277+0000 7f33f8e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33f406e850 con 0x7f33f4146f60 2024-03-28T13:10:44.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.277+0000 7f33f8e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f33f406e570 con 0x7f33f4147570 2024-03-28T13:10:44.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.277+0000 7f33f8e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f33f406f050 con 0x7f33f406bab0 2024-03-28T13:10:44.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.277+0000 7f33eaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f4146f60 0x7f33f41e4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:44.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.277+0000 7f33eaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f4146f60 0x7f33f41e4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44318/0 (socket says 172.21.15.19:44318) 2024-03-28T13:10:44.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.277+0000 7f33eaa00640 1 -- 172.21.15.19:0/2033458027 learned_addr learned my addr 172.21.15.19:0/2033458027 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:44.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.277+0000 7f33f3400640 1 --2- 172.21.15.19:0/2033458027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f406bab0 0x7f33f41426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:44.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.277+0000 7f33f3e00640 1 --2- 172.21.15.19:0/2033458027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f4147570 0x7f33f4145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:44.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.278+0000 7f33eaa00640 1 -- 172.21.15.19:0/2033458027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f406bab0 msgr2=0x7f33f41426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:44.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.278+0000 7f33eaa00640 1 --2- 172.21.15.19:0/2033458027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f406bab0 0x7f33f41426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.278+0000 7f33eaa00640 1 -- 172.21.15.19:0/2033458027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f4147570 msgr2=0x7f33f4145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:44.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.278+0000 7f33eaa00640 1 --2- 172.21.15.19:0/2033458027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f4147570 0x7f33f4145ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.278+0000 7f33eaa00640 1 -- 172.21.15.19:0/2033458027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f4144b00 con 0x7f33f4146f60 2024-03-28T13:10:44.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.278+0000 7f33f3e00640 1 --2- 172.21.15.19:0/2033458027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f4147570 0x7f33f4145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:44.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.278+0000 7f33eaa00640 1 --2- 172.21.15.19:0/2033458027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f4146f60 0x7f33f41e4040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f33dc007e40 tx=0x7f33dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:44.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.278+0000 7f33ebe00640 1 -- 172.21.15.19:0/2033458027 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc014620 con 0x7f33f4146f60 2024-03-28T13:10:44.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.278+0000 7f33ebe00640 1 -- 172.21.15.19:0/2033458027 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33dc019070 con 0x7f33f4146f60 2024-03-28T13:10:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.278+0000 7f33ebe00640 1 -- 172.21.15.19:0/2033458027 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc014a00 con 0x7f33f4146f60 2024-03-28T13:10:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.278+0000 7f33f8e00640 1 -- 172.21.15.19:0/2033458027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f4144de0 con 0x7f33f4146f60 2024-03-28T13:10:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.278+0000 7f33f8e00640 1 -- 172.21.15.19:0/2033458027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f41453a0 con 0x7f33f4146f60 2024-03-28T13:10:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.280+0000 7f33f8e00640 1 -- 172.21.15.19:0/2033458027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33f4075940 con 0x7f33f4146f60 2024-03-28T13:10:44.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.281+0000 7f33ebe00640 1 -- 172.21.15.19:0/2033458027 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f33dc014ba0 con 0x7f33f4146f60 2024-03-28T13:10:44.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.282+0000 7f33ebe00640 1 --2- 172.21.15.19:0/2033458027 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33d006c530 0x7f33d006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:44.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.282+0000 7f33ebe00640 1 -- 172.21.15.19:0/2033458027 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5767+0+0 (secure 0 0 0) 0x7f33dc08bfe0 con 0x7f33f4146f60 2024-03-28T13:10:44.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.283+0000 7f33f3400640 1 --2- 172.21.15.19:0/2033458027 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33d006c530 0x7f33d006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:44.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.284+0000 7f33f3400640 1 --2- 172.21.15.19:0/2033458027 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33d006c530 0x7f33d006e9f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f33e4036000 tx=0x7f33e403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:44.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.286+0000 7f33ebe00640 1 -- 172.21.15.19:0/2033458027 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f33dc05a540 con 0x7f33f4146f60 2024-03-28T13:10:44.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.532+0000 7f33f8e00640 1 -- 172.21.15.19:0/2033458027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f33f4072770 con 0x7f33f4146f60 2024-03-28T13:10:44.534 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:10:44.534+0000 7f6b74400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T13:10:44.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.537+0000 7f33ebe00640 1 -- 172.21.15.19:0/2033458027 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f33dc059e40 con 0x7f33f4146f60 2024-03-28T13:10:44.537 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T13:10:44.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.540+0000 7f33f8e00640 1 -- 172.21.15.19:0/2033458027 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33d006c530 msgr2=0x7f33d006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:44.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.540+0000 7f33f8e00640 1 --2- 172.21.15.19:0/2033458027 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33d006c530 0x7f33d006e9f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f33e4036000 tx=0x7f33e403c040 comp rx=0 tx=0).stop 2024-03-28T13:10:44.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.540+0000 7f33f8e00640 1 -- 172.21.15.19:0/2033458027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f4146f60 msgr2=0x7f33f41e4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:44.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.540+0000 7f33f8e00640 1 --2- 172.21.15.19:0/2033458027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f4146f60 0x7f33f41e4040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f33dc007e40 tx=0x7f33dc00e4b0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.540+0000 7f33f8e00640 1 -- 172.21.15.19:0/2033458027 shutdown_connections 2024-03-28T13:10:44.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.540+0000 7f33f8e00640 1 --2- 172.21.15.19:0/2033458027 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33d006c530 0x7f33d006e9f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.540+0000 7f33f8e00640 1 --2- 172.21.15.19:0/2033458027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f4147570 0x7f33f4145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.540+0000 7f33f8e00640 1 --2- 172.21.15.19:0/2033458027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f4146f60 0x7f33f41e4040 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.540+0000 7f33f8e00640 1 --2- 172.21.15.19:0/2033458027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f406bab0 0x7f33f41426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.540+0000 7f33f8e00640 1 -- 172.21.15.19:0/2033458027 >> 172.21.15.19:0/2033458027 conn(0x7f33f40b2c90 msgr2=0x7f33f40b0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:44.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.540+0000 7f33f8e00640 1 -- 172.21.15.19:0/2033458027 shutdown_connections 2024-03-28T13:10:44.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.540+0000 7f33f8e00640 1 -- 172.21.15.19:0/2033458027 wait complete. 2024-03-28T13:10:44.562 INFO:tasks.ceph.mds.c.smithi019.stderr: -8591> 2024-03-28T13:09:47.121+0000 7fe3b8200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:10:44.568 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-28T13:10:44.630 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-03-28T13:10:44.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.758+0000 7f613c000640 1 Processor -- start 2024-03-28T13:10:44.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.758+0000 7f613c000640 1 -- start start 2024-03-28T13:10:44.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.759+0000 7f613c000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f61340af4b0 0x7f6134167460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:44.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.759+0000 7f613c000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61340afab0 con 0x7f61340d0990 2024-03-28T13:10:44.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.759+0000 7f613c000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f613415cc80 con 0x7f6134153580 2024-03-28T13:10:44.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.759+0000 7f613c000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f61340af890 con 0x7f61340af4b0 2024-03-28T13:10:44.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.760+0000 7f6138c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f61340d0990 0x7f61340aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37112/0 (socket says 172.21.15.19:37112) 2024-03-28T13:10:44.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.760+0000 7f6138c00640 1 -- 172.21.15.19:0/3066056049 learned_addr learned my addr 172.21.15.19:0/3066056049 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:44.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.760+0000 7f6133e00640 1 -- 172.21.15.19:0/3066056049 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1322528956 0 0) 0x7f61340afab0 con 0x7f61340d0990 2024-03-28T13:10:44.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.760+0000 7f6133e00640 1 -- 172.21.15.19:0/3066056049 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f611c003520 con 0x7f61340d0990 2024-03-28T13:10:44.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.760+0000 7f613a000640 1 --2- 172.21.15.19:0/3066056049 >> v2:172.21.15.188:3300/0 conn(0x7f61340af4b0 0x7f6134167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:44.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.760+0000 7f6133e00640 1 -- 172.21.15.19:0/3066056049 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3785496462 0 0) 0x7f611c003520 con 0x7f61340d0990 2024-03-28T13:10:44.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.760+0000 7f6133e00640 1 -- 172.21.15.19:0/3066056049 >> v2:172.21.15.188:3300/0 conn(0x7f61340af4b0 msgr2=0x7f6134167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:44.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.760+0000 7f6133e00640 1 --2- 172.21.15.19:0/3066056049 >> v2:172.21.15.188:3300/0 conn(0x7f61340af4b0 0x7f6134167460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.760+0000 7f6133e00640 1 -- 172.21.15.19:0/3066056049 >> v1:172.21.15.188:6789/0 conn(0x7f6134153580 legacy=0x7f61340d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:44.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.760+0000 7f6133e00640 1 -- 172.21.15.19:0/3066056049 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613415da00 con 0x7f61340d0990 2024-03-28T13:10:44.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f6133e00640 1 -- 172.21.15.19:0/3066056049 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f6124001670 con 0x7f61340d0990 2024-03-28T13:10:44.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f6133e00640 1 -- 172.21.15.19:0/3066056049 >> v1:172.21.15.19:6789/0 conn(0x7f61340d0990 legacy=0x7f61340aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:44.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f6133e00640 1 --2- 172.21.15.19:0/3066056049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f611c003a00 0x7f611c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:44.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f6133e00640 1 --2- 172.21.15.19:0/3066056049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c004800 0x7f611c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:44.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f6133e00640 1 --2- 172.21.15.19:0/3066056049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f611c0052e0 0x7f611c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:44.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f6133e00640 1 -- 172.21.15.19:0/3066056049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f61340af890 con 0x7f611c004800 2024-03-28T13:10:44.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f6133e00640 1 -- 172.21.15.19:0/3066056049 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f611c00c200 con 0x7f611c003a00 2024-03-28T13:10:44.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f6133e00640 1 -- 172.21.15.19:0/3066056049 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f611c00c400 con 0x7f611c0052e0 2024-03-28T13:10:44.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f6138c00640 1 --2- 172.21.15.19:0/3066056049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c004800 0x7f611c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:44.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f613a000640 1 --2- 172.21.15.19:0/3066056049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f611c0052e0 0x7f611c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:44.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f6138c00640 1 -- 172.21.15.19:0/3066056049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f611c0052e0 msgr2=0x7f611c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:44.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f6138c00640 1 --2- 172.21.15.19:0/3066056049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f611c0052e0 0x7f611c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f6138c00640 1 -- 172.21.15.19:0/3066056049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f611c003a00 msgr2=0x7f611c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:44.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f6139600640 1 --2- 172.21.15.19:0/3066056049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f611c003a00 0x7f611c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:44.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f6138c00640 1 --2- 172.21.15.19:0/3066056049 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f611c003a00 0x7f611c003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f6138c00640 1 -- 172.21.15.19:0/3066056049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611c003520 con 0x7f611c004800 2024-03-28T13:10:44.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f613a000640 1 --2- 172.21.15.19:0/3066056049 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f611c0052e0 0x7f611c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:44.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.761+0000 7f613a000640 1 -- 172.21.15.19:0/3066056049 reap_dead start 2024-03-28T13:10:44.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.762+0000 7f6138c00640 1 --2- 172.21.15.19:0/3066056049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c004800 0x7f611c004c80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f613406f820 tx=0x7f611c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78a44986f153ac0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:44.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.763+0000 7f6133e00640 1 -- 172.21.15.19:0/3066056049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124007c60 con 0x7f611c004800 2024-03-28T13:10:44.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.763+0000 7f6133e00640 1 -- 172.21.15.19:0/3066056049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6124030070 con 0x7f611c004800 2024-03-28T13:10:44.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.763+0000 7f6133e00640 1 -- 172.21.15.19:0/3066056049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124005db0 con 0x7f611c004800 2024-03-28T13:10:44.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.763+0000 7f613c000640 1 -- 172.21.15.19:0/3066056049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c004800 msgr2=0x7f611c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:44.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.763+0000 7f613c000640 1 --2- 172.21.15.19:0/3066056049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c004800 0x7f611c004c80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f613406f820 tx=0x7f611c005830 comp rx=0 tx=0).stop 2024-03-28T13:10:44.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.763+0000 7f613c000640 1 -- 172.21.15.19:0/3066056049 shutdown_connections 2024-03-28T13:10:44.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.763+0000 7f613c000640 1 --2- 172.21.15.19:0/3066056049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f611c004800 0x7f611c004c80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.763+0000 7f613c000640 1 -- 172.21.15.19:0/3066056049 >> 172.21.15.19:0/3066056049 conn(0x7f613406b750 msgr2=0x7f613406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:44.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.763+0000 7f613c000640 1 -- 172.21.15.19:0/3066056049 shutdown_connections 2024-03-28T13:10:44.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.763+0000 7f613c000640 1 -- 172.21.15.19:0/3066056049 wait complete. 2024-03-28T13:10:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.765+0000 7f613c000640 1 Processor -- start 2024-03-28T13:10:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.765+0000 7f613c000640 1 -- start start 2024-03-28T13:10:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.766+0000 7f613c000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f611c004800 0x7f613414d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.766+0000 7f613c000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6134150b20 0x7f61340b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.766+0000 7f613c000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61340b0620 0x7f6134153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.766+0000 7f613c000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f613415d4e0 con 0x7f6134150b20 2024-03-28T13:10:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.766+0000 7f613c000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f613415d200 con 0x7f61340b0620 2024-03-28T13:10:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.766+0000 7f613c000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6134153960 con 0x7f611c004800 2024-03-28T13:10:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.766+0000 7f6138c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6134150b20 0x7f61340b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.766+0000 7f6138c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6134150b20 0x7f61340b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44350/0 (socket says 172.21.15.19:44350) 2024-03-28T13:10:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.766+0000 7f6138c00640 1 -- 172.21.15.19:0/3089360860 learned_addr learned my addr 172.21.15.19:0/3089360860 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.767+0000 7f6139600640 1 --2- 172.21.15.19:0/3089360860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f611c004800 0x7f613414d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.767+0000 7f6138c00640 1 -- 172.21.15.19:0/3089360860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f611c004800 msgr2=0x7f613414d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.767+0000 7f6138c00640 1 --2- 172.21.15.19:0/3089360860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f611c004800 0x7f613414d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.767+0000 7f6138c00640 1 -- 172.21.15.19:0/3089360860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61340b0620 msgr2=0x7f6134153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.767+0000 7f6138c00640 1 --2- 172.21.15.19:0/3089360860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61340b0620 0x7f6134153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.767+0000 7f6138c00640 1 -- 172.21.15.19:0/3089360860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613414db60 con 0x7f6134150b20 2024-03-28T13:10:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.767+0000 7f6138c00640 1 --2- 172.21.15.19:0/3089360860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6134150b20 0x7f61340b00e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f612400c220 tx=0x7f612402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.767+0000 7f6131600640 1 -- 172.21.15.19:0/3089360860 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124007c60 con 0x7f6134150b20 2024-03-28T13:10:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.767+0000 7f6131600640 1 -- 172.21.15.19:0/3089360860 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f612402cbb0 con 0x7f6134150b20 2024-03-28T13:10:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.767+0000 7f6131600640 1 -- 172.21.15.19:0/3089360860 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124007070 con 0x7f6134150b20 2024-03-28T13:10:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.767+0000 7f613c000640 1 -- 172.21.15.19:0/3089360860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6134156bf0 con 0x7f6134150b20 2024-03-28T13:10:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.768+0000 7f613c000640 1 -- 172.21.15.19:0/3089360860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6134155e30 con 0x7f6134150b20 2024-03-28T13:10:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.769+0000 7f613c000640 1 -- 172.21.15.19:0/3089360860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61340685c0 con 0x7f6134150b20 2024-03-28T13:10:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.770+0000 7f6131600640 1 -- 172.21.15.19:0/3089360860 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6124048020 con 0x7f6134150b20 2024-03-28T13:10:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.770+0000 7f6131600640 1 --2- 172.21.15.19:0/3089360860 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f611806c530 0x7f611806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.771+0000 7f6131600640 1 -- 172.21.15.19:0/3089360860 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5853+0+0 (secure 0 0 0) 0x7f61240acd80 con 0x7f6134150b20 2024-03-28T13:10:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.771+0000 7f6139600640 1 --2- 172.21.15.19:0/3089360860 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f611806c530 0x7f611806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.772+0000 7f6139600640 1 --2- 172.21.15.19:0/3089360860 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f611806c530 0x7f611806e9f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6128000c00 tx=0x7f6128009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:44.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:44.775+0000 7f6131600640 1 -- 172.21.15.19:0/3089360860 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f613414db60 con 0x7f6134150b20 2024-03-28T13:10:45.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.007+0000 7f613c000640 1 -- 172.21.15.19:0/3089360860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f613406eca0 con 0x7f6134150b20 2024-03-28T13:10:45.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.541+0000 7f6131600640 1 -- 172.21.15.19:0/3089360860 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f6124033e50 con 0x7f6134150b20 2024-03-28T13:10:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.544+0000 7f613c000640 1 -- 172.21.15.19:0/3089360860 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f611806c530 msgr2=0x7f611806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.544+0000 7f613c000640 1 --2- 172.21.15.19:0/3089360860 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f611806c530 0x7f611806e9f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6128000c00 tx=0x7f6128009040 comp rx=0 tx=0).stop 2024-03-28T13:10:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.544+0000 7f613c000640 1 -- 172.21.15.19:0/3089360860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6134150b20 msgr2=0x7f61340b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.544+0000 7f613c000640 1 --2- 172.21.15.19:0/3089360860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6134150b20 0x7f61340b00e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f612400c220 tx=0x7f612402e070 comp rx=0 tx=0).stop 2024-03-28T13:10:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.544+0000 7f613c000640 1 -- 172.21.15.19:0/3089360860 shutdown_connections 2024-03-28T13:10:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.544+0000 7f613c000640 1 --2- 172.21.15.19:0/3089360860 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f611806c530 0x7f611806e9f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.544+0000 7f613c000640 1 --2- 172.21.15.19:0/3089360860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61340b0620 0x7f6134153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.544+0000 7f613c000640 1 --2- 172.21.15.19:0/3089360860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6134150b20 0x7f61340b00e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.544+0000 7f613c000640 1 --2- 172.21.15.19:0/3089360860 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f611c004800 0x7f613414d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.544+0000 7f613c000640 1 -- 172.21.15.19:0/3089360860 >> 172.21.15.19:0/3089360860 conn(0x7f613406b750 msgr2=0x7f613406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.545+0000 7f613c000640 1 -- 172.21.15.19:0/3089360860 shutdown_connections 2024-03-28T13:10:45.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.545+0000 7f613c000640 1 -- 172.21.15.19:0/3089360860 wait complete. 2024-03-28T13:10:45.568 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-28T13:10:45.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.797+0000 7f51a3e00640 1 Processor -- start 2024-03-28T13:10:45.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.797+0000 7f51a3e00640 1 -- start start 2024-03-28T13:10:45.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.798+0000 7f51a3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f51a4147c40 0x7f51a40ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:45.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.798+0000 7f51a3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51a406f730 con 0x7f51a406f2a0 2024-03-28T13:10:45.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.798+0000 7f51a3e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51a40adf90 con 0x7f51a4162590 2024-03-28T13:10:45.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.798+0000 7f51a3e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f51a40ae270 con 0x7f51a4147c40 2024-03-28T13:10:45.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.799+0000 7f519a000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f51a406f2a0 0x7f51a4161e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58372/0 (socket says 172.21.15.19:58372) 2024-03-28T13:10:45.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.799+0000 7f519a000640 1 -- 172.21.15.19:0/1410607547 learned_addr learned my addr 172.21.15.19:0/1410607547 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:45.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.799+0000 7f51a2a00640 1 --2- 172.21.15.19:0/1410607547 >> v2:172.21.15.19:3300/0 conn(0x7f51a4147c40 0x7f51a40ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:45.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.799+0000 7f51a2a00640 1 -- 172.21.15.19:0/1410607547 >> v1:172.21.15.188:6789/0 conn(0x7f51a4162590 legacy=0x7f51a40b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:45.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.799+0000 7f51a2a00640 1 -- 172.21.15.19:0/1410607547 >> v1:172.21.15.19:6789/0 conn(0x7f51a406f2a0 legacy=0x7f51a4161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:45.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.800+0000 7f51a2a00640 1 -- 172.21.15.19:0/1410607547 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a40ae550 con 0x7f51a4147c40 2024-03-28T13:10:45.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.799+0000 7f51a2000640 1 -- 172.21.15.19:0/1410607547 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1676941759 0 0) 0x7f51a406f730 con 0x7f51a406f2a0 2024-03-28T13:10:45.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.801+0000 7f51a2a00640 1 --2- 172.21.15.19:0/1410607547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51a4147c40 0x7f51a40ad890 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f51940097a0 tx=0x7f519402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799144e57a411fc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:45.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.801+0000 7f51a2000640 1 -- 172.21.15.19:0/1410607547 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194034070 con 0x7f51a4147c40 2024-03-28T13:10:45.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.801+0000 7f51a2000640 1 -- 172.21.15.19:0/1410607547 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5194030610 con 0x7f51a4147c40 2024-03-28T13:10:45.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.801+0000 7f51a2000640 1 -- 172.21.15.19:0/1410607547 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51940309e0 con 0x7f51a4147c40 2024-03-28T13:10:45.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.801+0000 7f51a3e00640 1 -- 172.21.15.19:0/1410607547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51a4147c40 msgr2=0x7f51a40ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:45.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.801+0000 7f51a3e00640 1 --2- 172.21.15.19:0/1410607547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51a4147c40 0x7f51a40ad890 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f51940097a0 tx=0x7f519402b960 comp rx=0 tx=0).stop 2024-03-28T13:10:45.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.802+0000 7f51a3e00640 1 -- 172.21.15.19:0/1410607547 shutdown_connections 2024-03-28T13:10:45.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.802+0000 7f51a3e00640 1 --2- 172.21.15.19:0/1410607547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51a4147c40 0x7f51a40ad890 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:45.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.802+0000 7f51a3e00640 1 -- 172.21.15.19:0/1410607547 >> 172.21.15.19:0/1410607547 conn(0x7f51a40b1d80 msgr2=0x7f51a40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:45.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.802+0000 7f51a3e00640 1 -- 172.21.15.19:0/1410607547 shutdown_connections 2024-03-28T13:10:45.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.802+0000 7f51a3e00640 1 -- 172.21.15.19:0/1410607547 wait complete. 2024-03-28T13:10:45.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.803+0000 7f51a3e00640 1 Processor -- start 2024-03-28T13:10:45.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.804+0000 7f51a3e00640 1 -- start start 2024-03-28T13:10:45.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.804+0000 7f51a3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a406f2a0 0x7f51a415bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:45.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.804+0000 7f51a3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51a4147c40 0x7f51a41e9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:45.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.805+0000 7f51a3e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a4162590 0x7f51a415d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.805+0000 7f51a3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f51a40af030 con 0x7f51a4147c40 2024-03-28T13:10:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.805+0000 7f51a3e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51a40aeb10 con 0x7f51a4162590 2024-03-28T13:10:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.805+0000 7f51a3e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f51a40af1b0 con 0x7f51a406f2a0 2024-03-28T13:10:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.805+0000 7f519a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51a4147c40 0x7f51a41e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.805+0000 7f519a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51a4147c40 0x7f51a41e9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56718/0 (socket says 172.21.15.19:56718) 2024-03-28T13:10:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.805+0000 7f519a000640 1 -- 172.21.15.19:0/3142210885 learned_addr learned my addr 172.21.15.19:0/3142210885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.805+0000 7f519a000640 1 -- 172.21.15.19:0/3142210885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a406f2a0 msgr2=0x7f51a415bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.805+0000 7f51a2a00640 1 --2- 172.21.15.19:0/3142210885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a406f2a0 0x7f51a415bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.805+0000 7f519a000640 1 --2- 172.21.15.19:0/3142210885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a406f2a0 0x7f51a415bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.805+0000 7f519a000640 1 -- 172.21.15.19:0/3142210885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a4162590 msgr2=0x7f51a415d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.805+0000 7f519a000640 1 --2- 172.21.15.19:0/3142210885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a4162590 0x7f51a415d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.805+0000 7f519a000640 1 -- 172.21.15.19:0/3142210885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a41e8050 con 0x7f51a4147c40 2024-03-28T13:10:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.805+0000 7f51a2a00640 1 --2- 172.21.15.19:0/3142210885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a406f2a0 0x7f51a415bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:45.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.806+0000 7f519a000640 1 --2- 172.21.15.19:0/3142210885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51a4147c40 0x7f51a41e9350 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c420 tx=0x7f518c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:45.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.806+0000 7f519b400640 1 -- 172.21.15.19:0/3142210885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c0074e0 con 0x7f51a4147c40 2024-03-28T13:10:45.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.806+0000 7f519b400640 1 -- 172.21.15.19:0/3142210885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f518c0047e0 con 0x7f51a4147c40 2024-03-28T13:10:45.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.806+0000 7f51a3e00640 1 -- 172.21.15.19:0/3142210885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a41e8330 con 0x7f51a4147c40 2024-03-28T13:10:45.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.806+0000 7f519b400640 1 -- 172.21.15.19:0/3142210885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c004c10 con 0x7f51a4147c40 2024-03-28T13:10:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.806+0000 7f51a3e00640 1 -- 172.21.15.19:0/3142210885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a41e88f0 con 0x7f51a4147c40 2024-03-28T13:10:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.807+0000 7f51a3e00640 1 -- 172.21.15.19:0/3142210885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a4075940 con 0x7f51a4147c40 2024-03-28T13:10:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.808+0000 7f519b400640 1 -- 172.21.15.19:0/3142210885 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f518c011070 con 0x7f51a4147c40 2024-03-28T13:10:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.809+0000 7f519b400640 1 --2- 172.21.15.19:0/3142210885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f518006c530 0x7f518006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:45.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.809+0000 7f519b400640 1 -- 172.21.15.19:0/3142210885 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5853+0+0 (secure 0 0 0) 0x7f518c08b470 con 0x7f51a4147c40 2024-03-28T13:10:45.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.810+0000 7f51a2a00640 1 --2- 172.21.15.19:0/3142210885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f518006c530 0x7f518006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:45.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.811+0000 7f51a2a00640 1 --2- 172.21.15.19:0/3142210885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f518006c530 0x7f518006e9f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5194036000 tx=0x7f519403b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:45.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:45.814+0000 7f519b400640 1 -- 172.21.15.19:0/3142210885 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f518c0598e0 con 0x7f51a4147c40 2024-03-28T13:10:46.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.015+0000 7f51a3e00640 1 -- 172.21.15.19:0/3142210885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f51a406e390 con 0x7f51a4147c40 2024-03-28T13:10:46.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.544+0000 7f519b400640 1 -- 172.21.15.19:0/3142210885 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) ==== 181+0+0 (secure 0 0 0) 0x7f518c0591e0 con 0x7f51a4147c40 2024-03-28T13:10:46.545 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-03-28T13:10:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.548+0000 7f51a3e00640 1 -- 172.21.15.19:0/3142210885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f518006c530 msgr2=0x7f518006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.548+0000 7f51a3e00640 1 --2- 172.21.15.19:0/3142210885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f518006c530 0x7f518006e9f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5194036000 tx=0x7f519403b040 comp rx=0 tx=0).stop 2024-03-28T13:10:46.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.548+0000 7f51a3e00640 1 -- 172.21.15.19:0/3142210885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51a4147c40 msgr2=0x7f51a41e9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.548+0000 7f51a3e00640 1 --2- 172.21.15.19:0/3142210885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51a4147c40 0x7f51a41e9350 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c420 tx=0x7f518c00c8e0 comp rx=0 tx=0).stop 2024-03-28T13:10:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.548+0000 7f51a3e00640 1 -- 172.21.15.19:0/3142210885 shutdown_connections 2024-03-28T13:10:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.548+0000 7f51a3e00640 1 --2- 172.21.15.19:0/3142210885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f518006c530 0x7f518006e9f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.548+0000 7f51a3e00640 1 --2- 172.21.15.19:0/3142210885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51a4162590 0x7f51a415d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.548+0000 7f51a3e00640 1 --2- 172.21.15.19:0/3142210885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51a4147c40 0x7f51a41e9350 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.548+0000 7f51a3e00640 1 --2- 172.21.15.19:0/3142210885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51a406f2a0 0x7f51a415bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.548+0000 7f51a3e00640 1 -- 172.21.15.19:0/3142210885 >> 172.21.15.19:0/3142210885 conn(0x7f51a40b1d80 msgr2=0x7f51a40b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.548+0000 7f51a3e00640 1 -- 172.21.15.19:0/3142210885 shutdown_connections 2024-03-28T13:10:46.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.548+0000 7f51a3e00640 1 -- 172.21.15.19:0/3142210885 wait complete. 2024-03-28T13:10:46.578 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-28T13:10:46.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.803+0000 7fec37000640 1 Processor -- start 2024-03-28T13:10:46.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.803+0000 7fec37000640 1 -- start start 2024-03-28T13:10:46.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.804+0000 7fec37000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec3009dcf0 0x7fec301453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:46.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.804+0000 7fec37000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fec30071710 0x7fec30071b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:46.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.804+0000 7fec37000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec3015cb90 con 0x7fec300acf10 2024-03-28T13:10:46.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.805+0000 7fec37000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fec30145f60 con 0x7fec30071710 2024-03-28T13:10:46.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.805+0000 7fec37000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fec3015c950 con 0x7fec3009dcf0 2024-03-28T13:10:46.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.805+0000 7fec2fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fec300acf10 0x7fec300ad2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58376/0 (socket says 172.21.15.19:58376) 2024-03-28T13:10:46.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.805+0000 7fec2fe00640 1 -- 172.21.15.19:0/3071142118 learned_addr learned my addr 172.21.15.19:0/3071142118 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:46.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.805+0000 7fec35000640 1 --2- 172.21.15.19:0/3071142118 >> v2:172.21.15.188:3300/0 conn(0x7fec30071710 0x7fec30071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.805+0000 7fec2f400640 1 --2- 172.21.15.19:0/3071142118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec3009dcf0 0x7fec301453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.805+0000 7fec2ea00640 1 -- 172.21.15.19:0/3071142118 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3860965949 0 0) 0x7fec3015cb90 con 0x7fec300acf10 2024-03-28T13:10:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2ea00640 1 -- 172.21.15.19:0/3071142118 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec14003520 con 0x7fec300acf10 2024-03-28T13:10:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2ea00640 1 -- 172.21.15.19:0/3071142118 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2844485289 0 0) 0x7fec14003520 con 0x7fec300acf10 2024-03-28T13:10:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2ea00640 1 -- 172.21.15.19:0/3071142118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec3009dcf0 msgr2=0x7fec301453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2ea00640 1 --2- 172.21.15.19:0/3071142118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec3009dcf0 0x7fec301453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2ea00640 1 -- 172.21.15.19:0/3071142118 >> v2:172.21.15.188:3300/0 conn(0x7fec30071710 msgr2=0x7fec30071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2ea00640 1 --2- 172.21.15.19:0/3071142118 >> v2:172.21.15.188:3300/0 conn(0x7fec30071710 0x7fec30071b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2ea00640 1 -- 172.21.15.19:0/3071142118 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3015ce70 con 0x7fec300acf10 2024-03-28T13:10:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2ea00640 1 -- 172.21.15.19:0/3071142118 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fec20001670 con 0x7fec300acf10 2024-03-28T13:10:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2ea00640 1 -- 172.21.15.19:0/3071142118 >> v1:172.21.15.19:6789/0 conn(0x7fec300acf10 legacy=0x7fec300ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:46.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2ea00640 1 --2- 172.21.15.19:0/3071142118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec140039b0 0x7fec14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2ea00640 1 --2- 172.21.15.19:0/3071142118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec3009dcf0 0x7fec14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2ea00640 1 --2- 172.21.15.19:0/3071142118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec14004d40 0x7fec140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2ea00640 1 -- 172.21.15.19:0/3071142118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fec30145f60 con 0x7fec140039b0 2024-03-28T13:10:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2ea00640 1 -- 172.21.15.19:0/3071142118 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fec3015c950 con 0x7fec14004d40 2024-03-28T13:10:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2ea00640 1 -- 172.21.15.19:0/3071142118 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fec1400b9f0 con 0x7fec3009dcf0 2024-03-28T13:10:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.806+0000 7fec2fe00640 1 --2- 172.21.15.19:0/3071142118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec140039b0 0x7fec14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.807+0000 7fec2f400640 1 --2- 172.21.15.19:0/3071142118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec3009dcf0 0x7fec14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.807+0000 7fec35000640 1 --2- 172.21.15.19:0/3071142118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec14004d40 0x7fec140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.807+0000 7fec2fe00640 1 -- 172.21.15.19:0/3071142118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec3009dcf0 msgr2=0x7fec14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.807+0000 7fec2fe00640 1 --2- 172.21.15.19:0/3071142118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec3009dcf0 0x7fec14004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:46.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.807+0000 7fec2fe00640 1 -- 172.21.15.19:0/3071142118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec14004d40 msgr2=0x7fec140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:46.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.807+0000 7fec2fe00640 1 --2- 172.21.15.19:0/3071142118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec14004d40 0x7fec140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:46.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.807+0000 7fec2fe00640 1 -- 172.21.15.19:0/3071142118 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec14003520 con 0x7fec140039b0 2024-03-28T13:10:46.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.808+0000 7fec2fe00640 1 --2- 172.21.15.19:0/3071142118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec140039b0 0x7fec14003dd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fec2000c160 tx=0x7fec14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55982b3f9c1f5d40 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:46.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.808+0000 7fec2ea00640 1 -- 172.21.15.19:0/3071142118 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20007c80 con 0x7fec140039b0 2024-03-28T13:10:46.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.808+0000 7fec2ea00640 1 -- 172.21.15.19:0/3071142118 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec20030070 con 0x7fec140039b0 2024-03-28T13:10:46.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.808+0000 7fec2ea00640 1 -- 172.21.15.19:0/3071142118 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20005d30 con 0x7fec140039b0 2024-03-28T13:10:46.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.809+0000 7fec37000640 1 -- 172.21.15.19:0/3071142118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec140039b0 msgr2=0x7fec14003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:46.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.809+0000 7fec37000640 1 --2- 172.21.15.19:0/3071142118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec140039b0 0x7fec14003dd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fec2000c160 tx=0x7fec14005830 comp rx=0 tx=0).stop 2024-03-28T13:10:46.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.809+0000 7fec35000640 1 -- 172.21.15.19:0/3071142118 reap_dead start 2024-03-28T13:10:46.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.809+0000 7fec37000640 1 -- 172.21.15.19:0/3071142118 shutdown_connections 2024-03-28T13:10:46.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.809+0000 7fec37000640 1 -- 172.21.15.19:0/3071142118 >> 172.21.15.19:0/3071142118 conn(0x7fec3006bba0 msgr2=0x7fec300b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:46.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.809+0000 7fec37000640 1 -- 172.21.15.19:0/3071142118 shutdown_connections 2024-03-28T13:10:46.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.809+0000 7fec37000640 1 -- 172.21.15.19:0/3071142118 wait complete. 2024-03-28T13:10:46.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.811+0000 7fec37000640 1 Processor -- start 2024-03-28T13:10:46.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.811+0000 7fec37000640 1 -- start start 2024-03-28T13:10:46.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.812+0000 7fec37000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec140039b0 0x7fec301357d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:46.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.812+0000 7fec37000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec3013a430 0x7fec3009aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.812+0000 7fec37000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec3009af60 0x7fec3015aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.812+0000 7fec37000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fec3015d430 con 0x7fec3013a430 2024-03-28T13:10:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.812+0000 7fec37000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fec3015d7a0 con 0x7fec3009af60 2024-03-28T13:10:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.812+0000 7fec37000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fec301453f0 con 0x7fec140039b0 2024-03-28T13:10:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.812+0000 7fec2f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec3013a430 0x7fec3009aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.812+0000 7fec2f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec3013a430 0x7fec3009aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56736/0 (socket says 172.21.15.19:56736) 2024-03-28T13:10:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.812+0000 7fec2f400640 1 -- 172.21.15.19:0/3292702707 learned_addr learned my addr 172.21.15.19:0/3292702707 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.813+0000 7fec35000640 1 --2- 172.21.15.19:0/3292702707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec3009af60 0x7fec3015aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.813+0000 7fec2fe00640 1 --2- 172.21.15.19:0/3292702707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec140039b0 0x7fec301357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.813+0000 7fec2f400640 1 -- 172.21.15.19:0/3292702707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec140039b0 msgr2=0x7fec301357d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.813+0000 7fec2f400640 1 --2- 172.21.15.19:0/3292702707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec140039b0 0x7fec301357d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:46.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.813+0000 7fec2f400640 1 -- 172.21.15.19:0/3292702707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec3009af60 msgr2=0x7fec3015aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:46.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.813+0000 7fec2f400640 1 --2- 172.21.15.19:0/3292702707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec3009af60 0x7fec3015aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:46.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.813+0000 7fec2f400640 1 -- 172.21.15.19:0/3292702707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec300b1db0 con 0x7fec3013a430 2024-03-28T13:10:46.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.813+0000 7fec2f400640 1 --2- 172.21.15.19:0/3292702707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec3013a430 0x7fec3009aa20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fec18010860 tx=0x7fec18010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:46.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.813+0000 7fec27e00640 1 -- 172.21.15.19:0/3292702707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1800f040 con 0x7fec3013a430 2024-03-28T13:10:46.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.813+0000 7fec27e00640 1 -- 172.21.15.19:0/3292702707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec1800eb60 con 0x7fec3013a430 2024-03-28T13:10:46.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.813+0000 7fec27e00640 1 -- 172.21.15.19:0/3292702707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1801c400 con 0x7fec3013a430 2024-03-28T13:10:46.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.814+0000 7fec37000640 1 -- 172.21.15.19:0/3292702707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec300b2090 con 0x7fec3013a430 2024-03-28T13:10:46.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.814+0000 7fec37000640 1 -- 172.21.15.19:0/3292702707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec30135d10 con 0x7fec3013a430 2024-03-28T13:10:46.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.815+0000 7fec37000640 1 -- 172.21.15.19:0/3292702707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec30067db0 con 0x7fec3013a430 2024-03-28T13:10:46.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.816+0000 7fec27e00640 1 -- 172.21.15.19:0/3292702707 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fec1801c5a0 con 0x7fec3013a430 2024-03-28T13:10:46.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.817+0000 7fec27e00640 1 --2- 172.21.15.19:0/3292702707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fec0c06c530 0x7fec0c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:46.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.817+0000 7fec27e00640 1 -- 172.21.15.19:0/3292702707 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5415+0+0 (secure 0 0 0) 0x7fec1808bd00 con 0x7fec3013a430 2024-03-28T13:10:46.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.818+0000 7fec2fe00640 1 --2- 172.21.15.19:0/3292702707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fec0c06c530 0x7fec0c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:46.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.819+0000 7fec2fe00640 1 --2- 172.21.15.19:0/3292702707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fec0c06c530 0x7fec0c06e9f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fec20034000 tx=0x7fec20007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:46.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:46.822+0000 7fec27e00640 1 -- 172.21.15.19:0/3292702707 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fec30067db0 con 0x7fec3013a430 2024-03-28T13:10:47.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.029+0000 7fec37000640 1 -- 172.21.15.19:0/3292702707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fec300b2260 con 0x7fec3013a430 2024-03-28T13:10:47.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.562+0000 7fec27e00640 1 -- 172.21.15.19:0/3292702707 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) ==== 169+0+0 (secure 0 0 0) 0x7fec300b2260 con 0x7fec3013a430 2024-03-28T13:10:47.563 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-03-28T13:10:47.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.566+0000 7fec37000640 1 -- 172.21.15.19:0/3292702707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fec0c06c530 msgr2=0x7fec0c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:47.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.566+0000 7fec37000640 1 --2- 172.21.15.19:0/3292702707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fec0c06c530 0x7fec0c06e9f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fec20034000 tx=0x7fec20007000 comp rx=0 tx=0).stop 2024-03-28T13:10:47.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.566+0000 7fec37000640 1 -- 172.21.15.19:0/3292702707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec3013a430 msgr2=0x7fec3009aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:47.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.566+0000 7fec37000640 1 --2- 172.21.15.19:0/3292702707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec3013a430 0x7fec3009aa20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fec18010860 tx=0x7fec18010d20 comp rx=0 tx=0).stop 2024-03-28T13:10:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.566+0000 7fec37000640 1 -- 172.21.15.19:0/3292702707 shutdown_connections 2024-03-28T13:10:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.566+0000 7fec37000640 1 --2- 172.21.15.19:0/3292702707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fec0c06c530 0x7fec0c06e9f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.566+0000 7fec37000640 1 --2- 172.21.15.19:0/3292702707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec3009af60 0x7fec3015aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.566+0000 7fec37000640 1 --2- 172.21.15.19:0/3292702707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec3013a430 0x7fec3009aa20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.566+0000 7fec37000640 1 --2- 172.21.15.19:0/3292702707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec140039b0 0x7fec301357d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.566+0000 7fec37000640 1 -- 172.21.15.19:0/3292702707 >> 172.21.15.19:0/3292702707 conn(0x7fec3006bba0 msgr2=0x7fec300b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.567+0000 7fec37000640 1 -- 172.21.15.19:0/3292702707 shutdown_connections 2024-03-28T13:10:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.567+0000 7fec37000640 1 -- 172.21.15.19:0/3292702707 wait complete. 2024-03-28T13:10:47.595 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-28T13:10:47.595 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-28T13:10:47.596 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-28T13:10:47.596 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:10:47.596 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-28T13:10:47.596 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-28T13:10:47.596 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-28T13:10:47.596 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:10:47.596 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-28T13:10:47.597 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-28T13:10:47.597 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-28T13:10:47.597 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:10:47.597 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-28T13:10:47.597 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-28T13:10:47.597 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-28T13:10:47.597 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:10:47.618 INFO:tasks.ceph.mds.c:Stopped 2024-03-28T13:10:47.619 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-28T13:10:47.620 INFO:tasks.ceph.mds.a:Stopped 2024-03-28T13:10:47.620 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-28T13:10:47.622 INFO:tasks.ceph.mds.d:Stopped 2024-03-28T13:10:47.622 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-28T13:10:47.624 INFO:tasks.ceph.mds.b:Stopped 2024-03-28T13:10:47.625 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-28T13:10:47.666 INFO:tasks.ceph.mds.c:Started 2024-03-28T13:10:47.667 DEBUG:teuthology.parallel:result is None 2024-03-28T13:10:47.670 INFO:tasks.ceph.mds.d:Started 2024-03-28T13:10:47.671 DEBUG:teuthology.parallel:result is None 2024-03-28T13:10:47.672 INFO:tasks.ceph.mds.a:Started 2024-03-28T13:10:47.672 DEBUG:teuthology.parallel:result is None 2024-03-28T13:10:47.677 INFO:tasks.ceph.mds.b:Started 2024-03-28T13:10:47.677 DEBUG:teuthology.parallel:result is None 2024-03-28T13:10:47.677 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-28T13:10:47.823 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-03-28T13:10:47.825 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-03-28T13:10:47.836 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-03-28T13:10:47.836 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-03-28T13:10:47.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.863+0000 7fd902000640 1 Processor -- start 2024-03-28T13:10:47.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.863+0000 7fd902000640 1 -- start start 2024-03-28T13:10:47.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.864+0000 7fd902000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8fc0d2310 0x7fd8fc0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:47.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.864+0000 7fd902000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd8fc0cf530 0x7fd8fc0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:47.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.864+0000 7fd902000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8fc15e880 con 0x7fd8fc15e4a0 2024-03-28T13:10:47.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.864+0000 7fd902000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fd8fc0d34e0 con 0x7fd8fc0cf530 2024-03-28T13:10:47.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.864+0000 7fd902000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd8fc0d28d0 con 0x7fd8fc0d2310 2024-03-28T13:10:47.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.865+0000 7fd900c00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd8fc15e4a0 0x7fd8fc0b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:60224/0 (socket says 172.21.15.19:60224) 2024-03-28T13:10:47.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.865+0000 7fd8fbe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8fc0d2310 0x7fd8fc0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:47.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.865+0000 7fd900c00640 1 -- 172.21.15.19:0/1704038990 learned_addr learned my addr 172.21.15.19:0/1704038990 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:47.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.865+0000 7fd901600640 1 --2- 172.21.15.19:0/1704038990 >> v2:172.21.15.188:3300/0 conn(0x7fd8fc0cf530 0x7fd8fc0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:47.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.865+0000 7fd8fb400640 1 -- 172.21.15.19:0/1704038990 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1625000605 0 0) 0x7fd8fc15e880 con 0x7fd8fc15e4a0 2024-03-28T13:10:47.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.865+0000 7fd8fb400640 1 -- 172.21.15.19:0/1704038990 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8e4003520 con 0x7fd8fc15e4a0 2024-03-28T13:10:47.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.865+0000 7fd8fbe00640 1 -- 172.21.15.19:0/1704038990 >> v2:172.21.15.188:3300/0 conn(0x7fd8fc0cf530 msgr2=0x7fd8fc0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:47.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.865+0000 7fd8fbe00640 1 --2- 172.21.15.19:0/1704038990 >> v2:172.21.15.188:3300/0 conn(0x7fd8fc0cf530 0x7fd8fc0b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:47.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.865+0000 7fd8fbe00640 1 -- 172.21.15.19:0/1704038990 >> v1:172.21.15.188:6789/0 conn(0x7fd8fc15e4a0 legacy=0x7fd8fc0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:47.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.865+0000 7fd8fbe00640 1 -- 172.21.15.19:0/1704038990 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8fc0d2bb0 con 0x7fd8fc0d2310 2024-03-28T13:10:47.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.865+0000 7fd901600640 1 --2- 172.21.15.19:0/1704038990 >> v2:172.21.15.188:3300/0 conn(0x7fd8fc0cf530 0x7fd8fc0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:10:47.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.866+0000 7fd8fbe00640 1 --2- 172.21.15.19:0/1704038990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8fc0d2310 0x7fd8fc0ceff0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80097a0 tx=0x7fd8e802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55b384d822e04cf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:47.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.867+0000 7fd8fb400640 1 -- 172.21.15.19:0/1704038990 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8035020 con 0x7fd8fc0d2310 2024-03-28T13:10:47.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.867+0000 7fd8fb400640 1 -- 172.21.15.19:0/1704038990 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e8031810 con 0x7fd8fc0d2310 2024-03-28T13:10:47.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.867+0000 7fd8fb400640 1 -- 172.21.15.19:0/1704038990 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8031be0 con 0x7fd8fc0d2310 2024-03-28T13:10:47.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.867+0000 7fd902000640 1 -- 172.21.15.19:0/1704038990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8fc0d2310 msgr2=0x7fd8fc0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:47.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.867+0000 7fd902000640 1 --2- 172.21.15.19:0/1704038990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8fc0d2310 0x7fd8fc0ceff0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd8e80097a0 tx=0x7fd8e802b8b0 comp rx=0 tx=0).stop 2024-03-28T13:10:47.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.867+0000 7fd902000640 1 -- 172.21.15.19:0/1704038990 shutdown_connections 2024-03-28T13:10:47.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.867+0000 7fd902000640 1 --2- 172.21.15.19:0/1704038990 >> v2:172.21.15.188:3300/0 conn(0x7fd8fc0cf530 0x7fd8fc0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:47.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.867+0000 7fd902000640 1 --2- 172.21.15.19:0/1704038990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8fc0d2310 0x7fd8fc0ceff0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:47.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.867+0000 7fd902000640 1 -- 172.21.15.19:0/1704038990 >> 172.21.15.19:0/1704038990 conn(0x7fd8fc0676d0 msgr2=0x7fd8fc067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:47.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.867+0000 7fd902000640 1 -- 172.21.15.19:0/1704038990 shutdown_connections 2024-03-28T13:10:47.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.867+0000 7fd902000640 1 -- 172.21.15.19:0/1704038990 wait complete. 2024-03-28T13:10:47.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.869+0000 7fd902000640 1 Processor -- start 2024-03-28T13:10:47.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.869+0000 7fd902000640 1 -- start start 2024-03-28T13:10:47.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.870+0000 7fd902000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8fc0cf530 0x7fd8fc1541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:47.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.870+0000 7fd902000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8fc0d2310 0x7fd8fc0cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:47.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.870+0000 7fd902000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8fc15e4a0 0x7fd8fc15cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:47.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.870+0000 7fd902000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc0d3970 con 0x7fd8fc15e4a0 2024-03-28T13:10:47.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.870+0000 7fd902000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd8fc0d3170 con 0x7fd8fc0cf530 2024-03-28T13:10:47.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.870+0000 7fd902000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd8fc0b7920 con 0x7fd8fc0d2310 2024-03-28T13:10:47.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.871+0000 7fd901600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8fc15e4a0 0x7fd8fc15cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:47.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.871+0000 7fd900c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8fc0cf530 0x7fd8fc1541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:47.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.871+0000 7fd901600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8fc15e4a0 0x7fd8fc15cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56772/0 (socket says 172.21.15.19:56772) 2024-03-28T13:10:47.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.871+0000 7fd901600640 1 -- 172.21.15.19:0/1107663015 learned_addr learned my addr 172.21.15.19:0/1107663015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:47.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.871+0000 7fd8fbe00640 1 --2- 172.21.15.19:0/1107663015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8fc0d2310 0x7fd8fc0cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:47.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.871+0000 7fd901600640 1 -- 172.21.15.19:0/1107663015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8fc0d2310 msgr2=0x7fd8fc0cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:47.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.871+0000 7fd901600640 1 --2- 172.21.15.19:0/1107663015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8fc0d2310 0x7fd8fc0cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:47.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.871+0000 7fd901600640 1 -- 172.21.15.19:0/1107663015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8fc0cf530 msgr2=0x7fd8fc1541e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:47.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.871+0000 7fd901600640 1 --2- 172.21.15.19:0/1107663015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8fc0cf530 0x7fd8fc1541e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:47.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.871+0000 7fd901600640 1 -- 172.21.15.19:0/1107663015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8fc0cca70 con 0x7fd8fc15e4a0 2024-03-28T13:10:47.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.871+0000 7fd901600640 1 --2- 172.21.15.19:0/1107663015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8fc15e4a0 0x7fd8fc15cf40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400efc0 tx=0x7fd8f4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:47.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.872+0000 7fd8f8c00640 1 -- 172.21.15.19:0/1107663015 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4018030 con 0x7fd8fc15e4a0 2024-03-28T13:10:47.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.872+0000 7fd8f8c00640 1 -- 172.21.15.19:0/1107663015 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f400cde0 con 0x7fd8fc15e4a0 2024-03-28T13:10:47.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.872+0000 7fd8f8c00640 1 -- 172.21.15.19:0/1107663015 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4003bb0 con 0x7fd8fc15e4a0 2024-03-28T13:10:47.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.872+0000 7fd902000640 1 -- 172.21.15.19:0/1107663015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8fc0ccd50 con 0x7fd8fc15e4a0 2024-03-28T13:10:47.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.872+0000 7fd902000640 1 -- 172.21.15.19:0/1107663015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8fc0cd310 con 0x7fd8fc15e4a0 2024-03-28T13:10:47.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.873+0000 7fd902000640 1 -- 172.21.15.19:0/1107663015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8fc075940 con 0x7fd8fc15e4a0 2024-03-28T13:10:47.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.874+0000 7fd8f8c00640 1 -- 172.21.15.19:0/1107663015 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd8f4005d10 con 0x7fd8fc15e4a0 2024-03-28T13:10:47.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.875+0000 7fd8f8c00640 1 --2- 172.21.15.19:0/1107663015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd8dc06c530 0x7fd8dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:47.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.875+0000 7fd8f8c00640 1 -- 172.21.15.19:0/1107663015 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4943+0+0 (secure 0 0 0) 0x7fd8f408bd00 con 0x7fd8fc15e4a0 2024-03-28T13:10:47.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.876+0000 7fd900c00640 1 --2- 172.21.15.19:0/1107663015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd8dc06c530 0x7fd8dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:47.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.876+0000 7fd900c00640 1 --2- 172.21.15.19:0/1107663015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd8dc06c530 0x7fd8dc06e9f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00011e0 tx=0x7fd8f0007d30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:47.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:47.880+0000 7fd8f8c00640 1 -- 172.21.15.19:0/1107663015 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd8f4022070 con 0x7fd8fc15e4a0 2024-03-28T13:10:48.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.097+0000 7fd902000640 1 -- 172.21.15.19:0/1107663015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd8fc15c740 con 0x7fd8fc15e4a0 2024-03-28T13:10:48.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.584+0000 7fd8f8c00640 1 -- 172.21.15.19:0/1107663015 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) ==== 97+0+0 (secure 0 0 0) 0x7fd8f4010040 con 0x7fd8fc15e4a0 2024-03-28T13:10:48.584 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-03-28T13:10:48.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.588+0000 7fd902000640 1 -- 172.21.15.19:0/1107663015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd8dc06c530 msgr2=0x7fd8dc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:48.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.588+0000 7fd902000640 1 --2- 172.21.15.19:0/1107663015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd8dc06c530 0x7fd8dc06e9f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00011e0 tx=0x7fd8f0007d30 comp rx=0 tx=0).stop 2024-03-28T13:10:48.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.588+0000 7fd902000640 1 -- 172.21.15.19:0/1107663015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8fc15e4a0 msgr2=0x7fd8fc15cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:48.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.588+0000 7fd902000640 1 --2- 172.21.15.19:0/1107663015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8fc15e4a0 0x7fd8fc15cf40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400efc0 tx=0x7fd8f4009f90 comp rx=0 tx=0).stop 2024-03-28T13:10:48.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.588+0000 7fd902000640 1 -- 172.21.15.19:0/1107663015 shutdown_connections 2024-03-28T13:10:48.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.588+0000 7fd902000640 1 --2- 172.21.15.19:0/1107663015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd8dc06c530 0x7fd8dc06e9f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.588+0000 7fd902000640 1 --2- 172.21.15.19:0/1107663015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd8fc15e4a0 0x7fd8fc15cf40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.588+0000 7fd902000640 1 --2- 172.21.15.19:0/1107663015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8fc0d2310 0x7fd8fc0cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.588+0000 7fd902000640 1 --2- 172.21.15.19:0/1107663015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8fc0cf530 0x7fd8fc1541e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.588+0000 7fd902000640 1 -- 172.21.15.19:0/1107663015 >> 172.21.15.19:0/1107663015 conn(0x7fd8fc0676d0 msgr2=0x7fd8fc15bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.588+0000 7fd902000640 1 -- 172.21.15.19:0/1107663015 shutdown_connections 2024-03-28T13:10:48.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.588+0000 7fd902000640 1 -- 172.21.15.19:0/1107663015 wait complete. 2024-03-28T13:10:48.612 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-28T13:10:48.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.831+0000 7f0d0d200640 1 Processor -- start 2024-03-28T13:10:48.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.831+0000 7f0d0d200640 1 -- start start 2024-03-28T13:10:48.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.833+0000 7f0d0d200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0d080736f0 0x7f0d0815f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:48.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.833+0000 7f0d0d200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d080cfc80 0x7f0d080d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.833+0000 7f0d0d200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d080717c0 con 0x7f0d0815e4d0 2024-03-28T13:10:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.833+0000 7f0d0d200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0d080b3ab0 con 0x7f0d080736f0 2024-03-28T13:10:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.833+0000 7f0d0d200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0d08071dd0 con 0x7f0d080cfc80 2024-03-28T13:10:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.833+0000 7f0d06a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0d080736f0 0x7f0d0815f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.833+0000 7f0d06a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0d080736f0 0x7f0d0815f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56786/0 (socket says 172.21.15.19:56786) 2024-03-28T13:10:48.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.833+0000 7f0d06a00640 1 -- 172.21.15.19:0/222622911 learned_addr learned my addr 172.21.15.19:0/222622911 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:48.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.834+0000 7f0d07e00640 1 --2- 172.21.15.19:0/222622911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d080cfc80 0x7f0d080d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:48.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.834+0000 7f0d06a00640 1 -- 172.21.15.19:0/222622911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d080cfc80 msgr2=0x7f0d080d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:48.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.834+0000 7f0d06a00640 1 --2- 172.21.15.19:0/222622911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d080cfc80 0x7f0d080d0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:48.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.834+0000 7f0d06a00640 1 -- 172.21.15.19:0/222622911 >> v1:172.21.15.188:6789/0 conn(0x7f0d0815e4d0 legacy=0x7f0d080ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.834+0000 7f0d06a00640 1 -- 172.21.15.19:0/222622911 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0815e8b0 con 0x7f0d080736f0 2024-03-28T13:10:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.835+0000 7f0d06a00640 1 --2- 172.21.15.19:0/222622911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d080736f0 0x7f0d0815f500 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009870 tx=0x7f0cf002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96254ff3feb55e16 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.835+0000 7f0d06000640 1 -- 172.21.15.19:0/222622911 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0035020 con 0x7f0d080736f0 2024-03-28T13:10:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.835+0000 7f0d06000640 1 -- 172.21.15.19:0/222622911 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf0031640 con 0x7f0d080736f0 2024-03-28T13:10:48.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.835+0000 7f0d06000640 1 -- 172.21.15.19:0/222622911 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf002d040 con 0x7f0d080736f0 2024-03-28T13:10:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.836+0000 7f0d0d200640 1 -- 172.21.15.19:0/222622911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d080736f0 msgr2=0x7f0d0815f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.836+0000 7f0d0d200640 1 --2- 172.21.15.19:0/222622911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d080736f0 0x7f0d0815f500 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0009870 tx=0x7f0cf002b8f0 comp rx=0 tx=0).stop 2024-03-28T13:10:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.836+0000 7f0d0d200640 1 -- 172.21.15.19:0/222622911 shutdown_connections 2024-03-28T13:10:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.836+0000 7f0d0d200640 1 --2- 172.21.15.19:0/222622911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d080736f0 0x7f0d0815f500 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:48.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.836+0000 7f0d0d200640 1 --2- 172.21.15.19:0/222622911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d080cfc80 0x7f0d080d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.836+0000 7f0d0d200640 1 -- 172.21.15.19:0/222622911 >> 172.21.15.19:0/222622911 conn(0x7f0d0806b570 msgr2=0x7f0d0806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.836+0000 7f0d0d200640 1 -- 172.21.15.19:0/222622911 shutdown_connections 2024-03-28T13:10:48.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.836+0000 7f0d0d200640 1 -- 172.21.15.19:0/222622911 wait complete. 2024-03-28T13:10:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.838+0000 7f0d0d200640 1 Processor -- start 2024-03-28T13:10:48.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.838+0000 7f0d0d200640 1 -- start start 2024-03-28T13:10:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.839+0000 7f0d0d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d080736f0 0x7f0d0815f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.839+0000 7f0d0d200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d080cfc80 0x7f0d08147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.839+0000 7f0d0d200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d0815e4d0 0x7f0d08157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.839+0000 7f0d0d200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08071a60 con 0x7f0d080736f0 2024-03-28T13:10:48.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.839+0000 7f0d0d200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0d080b3fe0 con 0x7f0d0815e4d0 2024-03-28T13:10:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.839+0000 7f0d0d200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0d08072260 con 0x7f0d080cfc80 2024-03-28T13:10:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.839+0000 7f0d07400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d080736f0 0x7f0d0815f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.840+0000 7f0d07400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d080736f0 0x7f0d0815f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56802/0 (socket says 172.21.15.19:56802) 2024-03-28T13:10:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.840+0000 7f0d07400640 1 -- 172.21.15.19:0/1751652381 learned_addr learned my addr 172.21.15.19:0/1751652381 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.840+0000 7f0d06a00640 1 --2- 172.21.15.19:0/1751652381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d080cfc80 0x7f0d08147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:48.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.840+0000 7f0d07e00640 1 --2- 172.21.15.19:0/1751652381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d0815e4d0 0x7f0d08157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.840+0000 7f0d07400640 1 -- 172.21.15.19:0/1751652381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d080cfc80 msgr2=0x7f0d08147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.840+0000 7f0d07400640 1 --2- 172.21.15.19:0/1751652381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d080cfc80 0x7f0d08147ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.840+0000 7f0d07400640 1 -- 172.21.15.19:0/1751652381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d0815e4d0 msgr2=0x7f0d08157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.840+0000 7f0d07400640 1 --2- 172.21.15.19:0/1751652381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d0815e4d0 0x7f0d08157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.840+0000 7f0d07400640 1 -- 172.21.15.19:0/1751652381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0815f9d0 con 0x7f0d080736f0 2024-03-28T13:10:48.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.840+0000 7f0d07400640 1 --2- 172.21.15.19:0/1751652381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d080736f0 0x7f0d0815f490 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800b3f0 tx=0x7f0cf800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.841+0000 7f0cff400640 1 -- 172.21.15.19:0/1751652381 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8015020 con 0x7f0d080736f0 2024-03-28T13:10:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.841+0000 7f0cff400640 1 -- 172.21.15.19:0/1751652381 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf8011830 con 0x7f0d080736f0 2024-03-28T13:10:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.841+0000 7f0cff400640 1 -- 172.21.15.19:0/1751652381 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8011c20 con 0x7f0d080736f0 2024-03-28T13:10:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.841+0000 7f0d0d200640 1 -- 172.21.15.19:0/1751652381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d08148010 con 0x7f0d080736f0 2024-03-28T13:10:48.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.841+0000 7f0d0d200640 1 -- 172.21.15.19:0/1751652381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d08160640 con 0x7f0d080736f0 2024-03-28T13:10:48.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.842+0000 7f0d0d200640 1 -- 172.21.15.19:0/1751652381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d080d0a90 con 0x7f0d080736f0 2024-03-28T13:10:48.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.843+0000 7f0cff400640 1 -- 172.21.15.19:0/1751652381 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0cf8018020 con 0x7f0d080736f0 2024-03-28T13:10:48.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.844+0000 7f0cff400640 1 --2- 172.21.15.19:0/1751652381 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0ce406c530 0x7f0ce406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:48.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.844+0000 7f0cff400640 1 -- 172.21.15.19:0/1751652381 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f0cf808afc0 con 0x7f0d080736f0 2024-03-28T13:10:48.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.845+0000 7f0d06a00640 1 --2- 172.21.15.19:0/1751652381 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0ce406c530 0x7f0ce406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:48.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.846+0000 7f0d06a00640 1 --2- 172.21.15.19:0/1751652381 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0ce406c530 0x7f0ce406e9f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0036000 tx=0x7f0cf003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:48.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:48.849+0000 7f0cff400640 1 -- 172.21.15.19:0/1751652381 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0d080d0a90 con 0x7f0d080736f0 2024-03-28T13:10:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.123+0000 7f0d0d200640 1 -- 172.21.15.19:0/1751652381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0d08075940 con 0x7f0d080736f0 2024-03-28T13:10:49.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.124+0000 7f0cff400640 1 -- 172.21.15.19:0/1751652381 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f0cf801f030 con 0x7f0d080736f0 2024-03-28T13:10:49.124 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:49.124 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:10:49.124 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-03-28T13:10:49.124 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.124 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-03-28T13:10:49.124 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmNRIkABAAq/nJioDZjp1zFWAYOLPBYw==", 2024-03-28T13:10:49.124 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.125 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:10:49.125 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:10:49.125 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:10:49.125 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:10:49.125 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.125 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.125 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-03-28T13:10:49.125 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmc10CIxAAQWCSjH0hW3KUk8/qQm91QA==", 2024-03-28T13:10:49.125 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.125 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:10:49.125 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:10:49.125 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:10:49.126 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:10:49.126 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.126 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.126 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-03-28T13:10:49.126 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmmlUoEhAA4FUGXoZ/h/ve5fThJNdE2Q==", 2024-03-28T13:10:49.126 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.126 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:10:49.126 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:10:49.126 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:10:49.126 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:10:49.126 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.126 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.127 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-03-28T13:10:49.127 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmWDRnMxAA34LiLI79N5fVJnbcPacbQA==", 2024-03-28T13:10:49.127 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.127 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:10:49.127 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:10:49.127 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:10:49.127 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:10:49.127 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.127 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.127 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-03-28T13:10:49.127 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDYawVmxhwWLBAAm7C56UGConFx2D7Lbq2SSA==", 2024-03-28T13:10:49.127 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.127 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:10:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:10:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:10:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-03-28T13:10:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDbawVmC2BiNxAALQrAcXU44m2dTCjks5yy/Q==", 2024-03-28T13:10:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:10:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:10:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:10:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.128 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-03-28T13:10:49.129 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfawVmozrJFRAA7YV1uMucIW7H5SI4KY8aDw==", 2024-03-28T13:10:49.129 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.129 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:10:49.129 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:10:49.129 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:10:49.129 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.129 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.129 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-03-28T13:10:49.129 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiawVmccbTMBAATxk24Bf4LuQSqU92HdwgUg==", 2024-03-28T13:10:49.129 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.130 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:10:49.130 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:10:49.130 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:10:49.130 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.130 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.130 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-03-28T13:10:49.130 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDpawVmdwnILRAAjUEBwnBEeSltu2CLRGhmDQ==", 2024-03-28T13:10:49.130 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.130 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:10:49.130 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:10:49.130 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:10:49.130 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.131 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.131 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-03-28T13:10:49.131 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDtawVmkwW5ARAAZJ/Nd/Wtq7LcbUjeqPNUag==", 2024-03-28T13:10:49.131 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.131 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:10:49.131 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:10:49.131 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:10:49.131 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.131 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.131 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-03-28T13:10:49.131 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDwawVm254wKRAAmCYODqSd2Whn4YKQzywrng==", 2024-03-28T13:10:49.131 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.131 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:10:49.131 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:10:49.132 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:10:49.132 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.132 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.132 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-03-28T13:10:49.132 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDzawVmV8z+MhAA9JeF55K7ph1mQRWalT7XhA==", 2024-03-28T13:10:49.132 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.132 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:10:49.132 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:10:49.132 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:10:49.132 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.132 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.132 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q==", 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmJYGjHBAAAx2TtR0Gb3fafnZU26FCMA==", 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:10:49.133 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmE7V/ERAAYcdvolIMDd5ff+pCn7hN1w==", 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmk6q9JhAAeC3JHQDnK8IXGVoK6qGEFA==", 2024-03-28T13:10:49.134 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.135 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:10:49.135 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:10:49.135 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:10:49.135 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:10:49.135 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.135 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.135 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-03-28T13:10:49.135 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDRawVm9IzFLRAA5i1Yl7LQ7eNNHp7v4H2kog==", 2024-03-28T13:10:49.135 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.135 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:10:49.135 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:10:49.135 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:10:49.135 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:10:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-mds", 2024-03-28T13:10:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQD8awVmf9gUIxAATtWtU+jFRRXNXBtpwirJMA==", 2024-03-28T13:10:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-mds" 2024-03-28T13:10:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-mgr", 2024-03-28T13:10:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQD8awVmp+kUIxAAdRFa8aPOxnu4RHtwAvqKDg==", 2024-03-28T13:10:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-28T13:10:49.136 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.137 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.137 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-osd", 2024-03-28T13:10:49.137 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQD8awVmCPkUIxAA6erZAn8C4M643PUk7xBrrg==", 2024-03-28T13:10:49.137 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.137 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-osd" 2024-03-28T13:10:49.137 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.137 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.137 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-rbd", 2024-03-28T13:10:49.137 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQD8awVmjAgVIxAAwczbA9aNFExwjAkTSjXDBw==", 2024-03-28T13:10:49.137 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.137 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-28T13:10:49.137 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-28T13:10:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQD8awVmHhgVIxAAHTaJ8NvaBiS1Nw93fm/wtA==", 2024-03-28T13:10:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-28T13:10:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.bootstrap-rgw", 2024-03-28T13:10:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQD8awVmHyQVIxAAwYAAFbug/mhMa1dZ0nzj7A==", 2024-03-28T13:10:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-28T13:10:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDSawVm5H1FLxAAtyjOPcCEUQyvTpBdu3Fn3g==", 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDSawVmebzfIRAAA0+qIIaprr4SfEZT6qRooA==", 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:10:49.139 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.140 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:49.140 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-03-28T13:10:49.140 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:10:49.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.127+0000 7f0d0d200640 1 -- 172.21.15.19:0/1751652381 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0ce406c530 msgr2=0x7f0ce406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:49.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.127+0000 7f0d0d200640 1 --2- 172.21.15.19:0/1751652381 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0ce406c530 0x7f0ce406e9f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0036000 tx=0x7f0cf003c040 comp rx=0 tx=0).stop 2024-03-28T13:10:49.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.127+0000 7f0d0d200640 1 -- 172.21.15.19:0/1751652381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d080736f0 msgr2=0x7f0d0815f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.127+0000 7f0d0d200640 1 --2- 172.21.15.19:0/1751652381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d080736f0 0x7f0d0815f490 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800b3f0 tx=0x7f0cf800b8b0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.127+0000 7f0d0d200640 1 -- 172.21.15.19:0/1751652381 shutdown_connections 2024-03-28T13:10:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.127+0000 7f0d0d200640 1 --2- 172.21.15.19:0/1751652381 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0ce406c530 0x7f0ce406e9f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.127+0000 7f0d0d200640 1 --2- 172.21.15.19:0/1751652381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d0815e4d0 0x7f0d08157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.127+0000 7f0d0d200640 1 --2- 172.21.15.19:0/1751652381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d080cfc80 0x7f0d08147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.127+0000 7f0d0d200640 1 --2- 172.21.15.19:0/1751652381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d080736f0 0x7f0d0815f490 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.127+0000 7f0d0d200640 1 -- 172.21.15.19:0/1751652381 >> 172.21.15.19:0/1751652381 conn(0x7f0d0806b570 msgr2=0x7f0d080cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.127+0000 7f0d0d200640 1 -- 172.21.15.19:0/1751652381 shutdown_connections 2024-03-28T13:10:49.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.127+0000 7f0d0d200640 1 -- 172.21.15.19:0/1751652381 wait complete. 2024-03-28T13:10:49.146 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-28T13:10:49.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.371+0000 7f6796c00640 1 Processor -- start 2024-03-28T13:10:49.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.372+0000 7f6796c00640 1 -- start start 2024-03-28T13:10:49.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.373+0000 7f6796c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f679015e4c0 0x7f67900b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:49.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.373+0000 7f6796c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f67900b69c0 0x7f67900b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:49.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.373+0000 7f6796c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67900d2710 con 0x7f67900d37b0 2024-03-28T13:10:49.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.373+0000 7f6796c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f679015e8a0 con 0x7f67900b69c0 2024-03-28T13:10:49.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.373+0000 7f6796c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f67900d4040 con 0x7f679015e4c0 2024-03-28T13:10:49.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.374+0000 7f6796200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f67900b69c0 0x7f67900b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:49.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.374+0000 7f6795800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f679015e4c0 0x7f67900b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:49.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.374+0000 7f6795800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f679015e4c0 0x7f67900b9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:58354/0 (socket says 172.21.15.19:58354) 2024-03-28T13:10:49.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.374+0000 7f6796200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f67900b69c0 0x7f67900b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:51900/0 (socket says 172.21.15.19:51900) 2024-03-28T13:10:49.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.374+0000 7f6795800640 1 -- 172.21.15.19:0/644472606 learned_addr learned my addr 172.21.15.19:0/644472606 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:49.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.374+0000 7f6794e00640 1 --1- 172.21.15.19:0/644472606 >> v1:172.21.15.188:6789/0 conn(0x7f67900d37b0 0x7f67900b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:60250/0 (socket says 172.21.15.19:60250) 2024-03-28T13:10:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.375+0000 7f6795800640 1 -- 172.21.15.19:0/644472606 >> v2:172.21.15.188:3300/0 conn(0x7f67900b69c0 msgr2=0x7f67900b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.375+0000 7f6795800640 1 --2- 172.21.15.19:0/644472606 >> v2:172.21.15.188:3300/0 conn(0x7f67900b69c0 0x7f67900b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.375+0000 7f6795800640 1 -- 172.21.15.19:0/644472606 >> v1:172.21.15.188:6789/0 conn(0x7f67900d37b0 legacy=0x7f67900b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:49.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.375+0000 7f6795800640 1 -- 172.21.15.19:0/644472606 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67900d29f0 con 0x7f679015e4c0 2024-03-28T13:10:49.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.376+0000 7f6795800640 1 --2- 172.21.15.19:0/644472606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f679015e4c0 0x7f67900b9bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6784009870 tx=0x7f678402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8798d2ce660ef1d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.377+0000 7f678fe00640 1 -- 172.21.15.19:0/644472606 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784034070 con 0x7f679015e4c0 2024-03-28T13:10:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.377+0000 7f678fe00640 1 -- 172.21.15.19:0/644472606 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6784030840 con 0x7f679015e4c0 2024-03-28T13:10:49.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.377+0000 7f678fe00640 1 -- 172.21.15.19:0/644472606 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784030c10 con 0x7f679015e4c0 2024-03-28T13:10:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.377+0000 7f6796c00640 1 -- 172.21.15.19:0/644472606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f679015e4c0 msgr2=0x7f67900b9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.377+0000 7f6796c00640 1 --2- 172.21.15.19:0/644472606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f679015e4c0 0x7f67900b9bf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6784009870 tx=0x7f678402b980 comp rx=0 tx=0).stop 2024-03-28T13:10:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.378+0000 7f6796c00640 1 -- 172.21.15.19:0/644472606 shutdown_connections 2024-03-28T13:10:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.378+0000 7f6796c00640 1 --2- 172.21.15.19:0/644472606 >> v2:172.21.15.188:3300/0 conn(0x7f67900b69c0 0x7f67900b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.378+0000 7f6796c00640 1 --2- 172.21.15.19:0/644472606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f679015e4c0 0x7f67900b9bf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.378+0000 7f6796c00640 1 -- 172.21.15.19:0/644472606 >> 172.21.15.19:0/644472606 conn(0x7f67900676a0 msgr2=0x7f6790067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.378+0000 7f6796c00640 1 -- 172.21.15.19:0/644472606 shutdown_connections 2024-03-28T13:10:49.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.378+0000 7f6796c00640 1 -- 172.21.15.19:0/644472606 wait complete. 2024-03-28T13:10:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.379+0000 7f6796c00640 1 Processor -- start 2024-03-28T13:10:49.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.380+0000 7f6796c00640 1 -- start start 2024-03-28T13:10:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.380+0000 7f6796c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67900b69c0 0x7f67901f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.380+0000 7f6796c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67900d37b0 0x7f6790160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.380+0000 7f6796c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f679015e4c0 0x7f6790161050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.380+0000 7f6796c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f67900d2fb0 con 0x7f679015e4c0 2024-03-28T13:10:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.381+0000 7f6796c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f67900d3320 con 0x7f67900d37b0 2024-03-28T13:10:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.381+0000 7f6796c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f67900ba130 con 0x7f67900b69c0 2024-03-28T13:10:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.381+0000 7f6796200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f679015e4c0 0x7f6790161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:49.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.381+0000 7f6796200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f679015e4c0 0x7f6790161050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56818/0 (socket says 172.21.15.19:56818) 2024-03-28T13:10:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.381+0000 7f6796200640 1 -- 172.21.15.19:0/2796141404 learned_addr learned my addr 172.21.15.19:0/2796141404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.381+0000 7f6794e00640 1 --2- 172.21.15.19:0/2796141404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67900d37b0 0x7f6790160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:49.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.381+0000 7f6795800640 1 --2- 172.21.15.19:0/2796141404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67900b69c0 0x7f67901f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.381+0000 7f6796200640 1 -- 172.21.15.19:0/2796141404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67900b69c0 msgr2=0x7f67901f3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.381+0000 7f6796200640 1 --2- 172.21.15.19:0/2796141404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67900b69c0 0x7f67901f3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.381+0000 7f6796200640 1 -- 172.21.15.19:0/2796141404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67900d37b0 msgr2=0x7f6790160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.381+0000 7f6796200640 1 --2- 172.21.15.19:0/2796141404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67900d37b0 0x7f6790160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.381+0000 7f6796200640 1 -- 172.21.15.19:0/2796141404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67900c8990 con 0x7f679015e4c0 2024-03-28T13:10:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.382+0000 7f6796200640 1 --2- 172.21.15.19:0/2796141404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f679015e4c0 0x7f6790161050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f678800efc0 tx=0x7f678800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.382+0000 7f678d600640 1 -- 172.21.15.19:0/2796141404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788018020 con 0x7f679015e4c0 2024-03-28T13:10:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.382+0000 7f678d600640 1 -- 172.21.15.19:0/2796141404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6788014480 con 0x7f679015e4c0 2024-03-28T13:10:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.382+0000 7f678d600640 1 -- 172.21.15.19:0/2796141404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6788014870 con 0x7f679015e4c0 2024-03-28T13:10:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.382+0000 7f6796c00640 1 -- 172.21.15.19:0/2796141404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67900c8c70 con 0x7f679015e4c0 2024-03-28T13:10:49.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.382+0000 7f6796c00640 1 -- 172.21.15.19:0/2796141404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67900c9230 con 0x7f679015e4c0 2024-03-28T13:10:49.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.383+0000 7f6796c00640 1 -- 172.21.15.19:0/2796141404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67900c9430 con 0x7f679015e4c0 2024-03-28T13:10:49.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.385+0000 7f678d600640 1 -- 172.21.15.19:0/2796141404 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6788005d10 con 0x7f679015e4c0 2024-03-28T13:10:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.385+0000 7f678d600640 1 --2- 172.21.15.19:0/2796141404 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f677006c530 0x7f677006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.385+0000 7f678d600640 1 -- 172.21.15.19:0/2796141404 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f678808c320 con 0x7f679015e4c0 2024-03-28T13:10:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.386+0000 7f6795800640 1 --2- 172.21.15.19:0/2796141404 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f677006c530 0x7f677006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.387+0000 7f6795800640 1 --2- 172.21.15.19:0/2796141404 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f677006c530 0x7f677006e9f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6784036000 tx=0x7f678403b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.390+0000 7f678d600640 1 -- 172.21.15.19:0/2796141404 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f67900c9430 con 0x7f679015e4c0 2024-03-28T13:10:49.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.678+0000 7f6796c00640 1 -- 172.21.15.19:0/2796141404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f6790075940 con 0x7f679015e4c0 2024-03-28T13:10:49.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.683+0000 7f678d600640 1 -- 172.21.15.19:0/2796141404 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7f6788022030 con 0x7f679015e4c0 2024-03-28T13:10:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.686+0000 7f6796c00640 1 -- 172.21.15.19:0/2796141404 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f677006c530 msgr2=0x7f677006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.686+0000 7f6796c00640 1 --2- 172.21.15.19:0/2796141404 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f677006c530 0x7f677006e9f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6784036000 tx=0x7f678403b040 comp rx=0 tx=0).stop 2024-03-28T13:10:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.686+0000 7f6796c00640 1 -- 172.21.15.19:0/2796141404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f679015e4c0 msgr2=0x7f6790161050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.686+0000 7f6796c00640 1 --2- 172.21.15.19:0/2796141404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f679015e4c0 0x7f6790161050 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f678800efc0 tx=0x7f678800c490 comp rx=0 tx=0).stop 2024-03-28T13:10:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.687+0000 7f6796c00640 1 -- 172.21.15.19:0/2796141404 shutdown_connections 2024-03-28T13:10:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.687+0000 7f6796c00640 1 --2- 172.21.15.19:0/2796141404 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f677006c530 0x7f677006e9f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.687+0000 7f6796c00640 1 --2- 172.21.15.19:0/2796141404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f679015e4c0 0x7f6790161050 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.687+0000 7f6796c00640 1 --2- 172.21.15.19:0/2796141404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67900d37b0 0x7f6790160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.687+0000 7f6796c00640 1 --2- 172.21.15.19:0/2796141404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67900b69c0 0x7f67901f3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.687+0000 7f6796c00640 1 -- 172.21.15.19:0/2796141404 >> 172.21.15.19:0/2796141404 conn(0x7f67900676a0 msgr2=0x7f67900d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.687+0000 7f6796c00640 1 -- 172.21.15.19:0/2796141404 shutdown_connections 2024-03-28T13:10:49.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.687+0000 7f6796c00640 1 -- 172.21.15.19:0/2796141404 wait complete. 2024-03-28T13:10:49.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-28T13:10:49.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.923+0000 7f1e0ea00640 1 Processor -- start 2024-03-28T13:10:49.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.923+0000 7f1e0ea00640 1 -- start start 2024-03-28T13:10:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.924+0000 7f1e0ea00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1e0815e860 0x7f1e0806d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.925+0000 7f1e0ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1e0806da30 0x7f1e080d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.925+0000 7f1e0ea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e080d4910 0x7f1e080d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.925+0000 7f1e0ea00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1e080cf220 con 0x7f1e0806da30 2024-03-28T13:10:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.925+0000 7f1e0ea00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1e080d4d40 con 0x7f1e0815e860 2024-03-28T13:10:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.925+0000 7f1e0ea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1e0806dfa0 con 0x7f1e080d4910 2024-03-28T13:10:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.925+0000 7f1e07400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1e0806da30 0x7f1e080d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.925+0000 7f1e07400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1e0806da30 0x7f1e080d43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56826/0 (socket says 172.21.15.19:56826) 2024-03-28T13:10:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.925+0000 7f1e07400640 1 -- 172.21.15.19:0/837933102 learned_addr learned my addr 172.21.15.19:0/837933102 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:49.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.925+0000 7f1e07e00640 1 --2- 172.21.15.19:0/837933102 >> v2:172.21.15.188:3300/0 conn(0x7f1e0815e860 0x7f1e0806d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:49.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.925+0000 7f1e0ca00640 1 --2- 172.21.15.19:0/837933102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e080d4910 0x7f1e080d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:49.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.925+0000 7f1e07400640 1 -- 172.21.15.19:0/837933102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e080d4910 msgr2=0x7f1e080d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:49.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.926+0000 7f1e07400640 1 --2- 172.21.15.19:0/837933102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e080d4910 0x7f1e080d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.926+0000 7f1e07400640 1 -- 172.21.15.19:0/837933102 >> v2:172.21.15.188:3300/0 conn(0x7f1e0815e860 msgr2=0x7f1e0806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:49.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.926+0000 7f1e07400640 1 --2- 172.21.15.19:0/837933102 >> v2:172.21.15.188:3300/0 conn(0x7f1e0815e860 0x7f1e0806d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.926+0000 7f1e07400640 1 -- 172.21.15.19:0/837933102 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0806e240 con 0x7f1e0806da30 2024-03-28T13:10:49.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.926+0000 7f1e07e00640 1 --2- 172.21.15.19:0/837933102 >> v2:172.21.15.188:3300/0 conn(0x7f1e0815e860 0x7f1e0806d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:49.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.926+0000 7f1e0ca00640 1 --2- 172.21.15.19:0/837933102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e080d4910 0x7f1e080d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:49.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.927+0000 7f1e07400640 1 --2- 172.21.15.19:0/837933102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0806da30 0x7f1e080d43d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1e080d1fa0 tx=0x7f1df802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1a60124ac9fbbf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.927+0000 7f1e06a00640 1 -- 172.21.15.19:0/837933102 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df802ca70 con 0x7f1e0806da30 2024-03-28T13:10:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.927+0000 7f1e06a00640 1 -- 172.21.15.19:0/837933102 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df802cc10 con 0x7f1e0806da30 2024-03-28T13:10:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.927+0000 7f1e06a00640 1 -- 172.21.15.19:0/837933102 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df80343f0 con 0x7f1e0806da30 2024-03-28T13:10:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.928+0000 7f1e0ea00640 1 -- 172.21.15.19:0/837933102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0806da30 msgr2=0x7f1e080d43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.928+0000 7f1e0ea00640 1 --2- 172.21.15.19:0/837933102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0806da30 0x7f1e080d43d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1e080d1fa0 tx=0x7f1df802bed0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.928+0000 7f1e0ea00640 1 -- 172.21.15.19:0/837933102 shutdown_connections 2024-03-28T13:10:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.928+0000 7f1e0ea00640 1 --2- 172.21.15.19:0/837933102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e080d4910 0x7f1e080d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.928+0000 7f1e0ea00640 1 --2- 172.21.15.19:0/837933102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0806da30 0x7f1e080d43d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.928+0000 7f1e0ea00640 1 --2- 172.21.15.19:0/837933102 >> v2:172.21.15.188:3300/0 conn(0x7f1e0815e860 0x7f1e0806d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.928+0000 7f1e0ea00640 1 -- 172.21.15.19:0/837933102 >> 172.21.15.19:0/837933102 conn(0x7f1e0809d7d0 msgr2=0x7f1e0808bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.928+0000 7f1e0ea00640 1 -- 172.21.15.19:0/837933102 shutdown_connections 2024-03-28T13:10:49.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.928+0000 7f1e0ea00640 1 -- 172.21.15.19:0/837933102 wait complete. 2024-03-28T13:10:49.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.930+0000 7f1e0ea00640 1 Processor -- start 2024-03-28T13:10:49.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.930+0000 7f1e0ea00640 1 -- start start 2024-03-28T13:10:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.931+0000 7f1e0ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0806da30 0x7f1e080b7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.931+0000 7f1e0ea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e080d4910 0x7f1e080b7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.931+0000 7f1e0ea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e0815e860 0x7f1e081f7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.931+0000 7f1e0ea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1e080cedb0 con 0x7f1e0806da30 2024-03-28T13:10:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.931+0000 7f1e0ea00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1e080d2c50 con 0x7f1e0815e860 2024-03-28T13:10:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.931+0000 7f1e0ea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1e080cef30 con 0x7f1e080d4910 2024-03-28T13:10:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.931+0000 7f1e07e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0806da30 0x7f1e080b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.931+0000 7f1e07e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0806da30 0x7f1e080b7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56834/0 (socket says 172.21.15.19:56834) 2024-03-28T13:10:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.931+0000 7f1e07e00640 1 -- 172.21.15.19:0/1629915006 learned_addr learned my addr 172.21.15.19:0/1629915006 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.932+0000 7f1e0ca00640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e0815e860 0x7f1e081f7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.932+0000 7f1e07e00640 1 -- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e080d4910 msgr2=0x7f1e080b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.932+0000 7f1e07400640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e080d4910 0x7f1e080b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:49.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.932+0000 7f1e07e00640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e080d4910 0x7f1e080b7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.932+0000 7f1e07e00640 1 -- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e0815e860 msgr2=0x7f1e081f7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.932+0000 7f1e07e00640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e0815e860 0x7f1e081f7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.932+0000 7f1e07e00640 1 -- 172.21.15.19:0/1629915006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0814eb50 con 0x7f1e0806da30 2024-03-28T13:10:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.932+0000 7f1e0ca00640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e0815e860 0x7f1e081f7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:49.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.932+0000 7f1e07400640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e080d4910 0x7f1e080b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.932+0000 7f1e07e00640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0806da30 0x7f1e080b7020 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1df000c7f0 tx=0x7f1df000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.932+0000 7f1dffe00640 1 -- 172.21.15.19:0/1629915006 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0015070 con 0x7f1e0806da30 2024-03-28T13:10:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.932+0000 7f1dffe00640 1 -- 172.21.15.19:0/1629915006 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df0007820 con 0x7f1e0806da30 2024-03-28T13:10:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.932+0000 7f1dffe00640 1 -- 172.21.15.19:0/1629915006 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0007bd0 con 0x7f1e0806da30 2024-03-28T13:10:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.932+0000 7f1e0ea00640 1 -- 172.21.15.19:0/1629915006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e080b95d0 con 0x7f1e0806da30 2024-03-28T13:10:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.933+0000 7f1e0ea00640 1 -- 172.21.15.19:0/1629915006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e080b9b90 con 0x7f1e0806da30 2024-03-28T13:10:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.934+0000 7f1e0ea00640 1 -- 172.21.15.19:0/1629915006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e080685c0 con 0x7f1e0806da30 2024-03-28T13:10:49.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.935+0000 7f1dffe00640 1 -- 172.21.15.19:0/1629915006 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1df0018070 con 0x7f1e0806da30 2024-03-28T13:10:49.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.936+0000 7f1dffe00640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1de406c530 0x7f1de406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:49.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.936+0000 7f1dffe00640 1 -- 172.21.15.19:0/1629915006 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f1df008b460 con 0x7f1e0806da30 2024-03-28T13:10:49.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.936+0000 7f1e07400640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1de406c530 0x7f1de406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:49.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.937+0000 7f1e07400640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1de406c530 0x7f1de406e9f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1df8035000 tx=0x7f1df8030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:49.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:49.940+0000 7f1dffe00640 1 -- 172.21.15.19:0/1629915006 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1df0059dc0 con 0x7f1e0806da30 2024-03-28T13:10:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.218+0000 7f1e0ea00640 1 -- 172.21.15.19:0/1629915006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f1e0815dfa0 con 0x7f1e0806da30 2024-03-28T13:10:50.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.223+0000 7f1dffe00640 1 -- 172.21.15.19:0/1629915006 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f1df00596c0 con 0x7f1e0806da30 2024-03-28T13:10:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.226+0000 7f1e0ea00640 1 -- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1de406c530 msgr2=0x7f1de406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.226+0000 7f1e0ea00640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1de406c530 0x7f1de406e9f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1df8035000 tx=0x7f1df8030000 comp rx=0 tx=0).stop 2024-03-28T13:10:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.226+0000 7f1e0ea00640 1 -- 172.21.15.19:0/1629915006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0806da30 msgr2=0x7f1e080b7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.226+0000 7f1e0ea00640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0806da30 0x7f1e080b7020 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1df000c7f0 tx=0x7f1df000ccb0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.226+0000 7f1e0ea00640 1 -- 172.21.15.19:0/1629915006 shutdown_connections 2024-03-28T13:10:50.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.226+0000 7f1e0ea00640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1de406c530 0x7f1de406e9f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.226+0000 7f1e0ea00640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e0815e860 0x7f1e081f7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.226+0000 7f1e0ea00640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e080d4910 0x7f1e080b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.226+0000 7f1e0ea00640 1 --2- 172.21.15.19:0/1629915006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1e0806da30 0x7f1e080b7020 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.226+0000 7f1e0ea00640 1 -- 172.21.15.19:0/1629915006 >> 172.21.15.19:0/1629915006 conn(0x7f1e0809d7d0 msgr2=0x7f1e080d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:50.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.226+0000 7f1e0ea00640 1 -- 172.21.15.19:0/1629915006 shutdown_connections 2024-03-28T13:10:50.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.226+0000 7f1e0ea00640 1 -- 172.21.15.19:0/1629915006 wait complete. 2024-03-28T13:10:50.246 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-28T13:10:50.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.471+0000 7f8484600640 1 Processor -- start 2024-03-28T13:10:50.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.471+0000 7f8484600640 1 -- start start 2024-03-28T13:10:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.473+0000 7f8484600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f847c153580 0x7f847c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.473+0000 7f8484600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f847c0d07c0 0x7f847c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.473+0000 7f8484600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f847c0af2e0 0x7f847c06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.473+0000 7f8484600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f847c0d0c00 con 0x7f847c0af2e0 2024-03-28T13:10:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.473+0000 7f8484600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f847c0afab0 con 0x7f847c0d07c0 2024-03-28T13:10:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.473+0000 7f8484600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f847c15cc80 con 0x7f847c153580 2024-03-28T13:10:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.473+0000 7f8482600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f847c0af2e0 0x7f847c06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.473+0000 7f8482600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f847c0af2e0 0x7f847c06d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56840/0 (socket says 172.21.15.19:56840) 2024-03-28T13:10:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.473+0000 7f8482600640 1 -- 172.21.15.19:0/1312632036 learned_addr learned my addr 172.21.15.19:0/1312632036 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.474+0000 7f8481c00640 1 --2- 172.21.15.19:0/1312632036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f847c153580 0x7f847c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.474+0000 7f8481200640 1 --2- 172.21.15.19:0/1312632036 >> v2:172.21.15.188:3300/0 conn(0x7f847c0d07c0 0x7f847c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:50.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.474+0000 7f8482600640 1 -- 172.21.15.19:0/1312632036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f847c153580 msgr2=0x7f847c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:50.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.474+0000 7f8482600640 1 --2- 172.21.15.19:0/1312632036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f847c153580 0x7f847c0d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.474+0000 7f8482600640 1 -- 172.21.15.19:0/1312632036 >> v2:172.21.15.188:3300/0 conn(0x7f847c0d07c0 msgr2=0x7f847c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:50.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.474+0000 7f8482600640 1 --2- 172.21.15.19:0/1312632036 >> v2:172.21.15.188:3300/0 conn(0x7f847c0d07c0 0x7f847c0aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.474+0000 7f8482600640 1 -- 172.21.15.19:0/1312632036 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847c15da00 con 0x7f847c0af2e0 2024-03-28T13:10:50.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.475+0000 7f8482600640 1 --2- 172.21.15.19:0/1312632036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f847c0af2e0 0x7f847c06d5b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f847c06c560 tx=0x7f847402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd5b164858e7200 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:50.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.475+0000 7f847be00640 1 -- 172.21.15.19:0/1312632036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847402c9e0 con 0x7f847c0af2e0 2024-03-28T13:10:50.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.475+0000 7f847be00640 1 -- 172.21.15.19:0/1312632036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f847402cb80 con 0x7f847c0af2e0 2024-03-28T13:10:50.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.475+0000 7f847be00640 1 -- 172.21.15.19:0/1312632036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84740343f0 con 0x7f847c0af2e0 2024-03-28T13:10:50.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.476+0000 7f8484600640 1 -- 172.21.15.19:0/1312632036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f847c0af2e0 msgr2=0x7f847c06d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:50.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.476+0000 7f8484600640 1 --2- 172.21.15.19:0/1312632036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f847c0af2e0 0x7f847c06d5b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f847c06c560 tx=0x7f847402be40 comp rx=0 tx=0).stop 2024-03-28T13:10:50.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.476+0000 7f8484600640 1 -- 172.21.15.19:0/1312632036 shutdown_connections 2024-03-28T13:10:50.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.476+0000 7f8484600640 1 --2- 172.21.15.19:0/1312632036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f847c0af2e0 0x7f847c06d5b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.476+0000 7f8484600640 1 --2- 172.21.15.19:0/1312632036 >> v2:172.21.15.188:3300/0 conn(0x7f847c0d07c0 0x7f847c0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.476+0000 7f8484600640 1 --2- 172.21.15.19:0/1312632036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f847c153580 0x7f847c0d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.476+0000 7f8484600640 1 -- 172.21.15.19:0/1312632036 >> 172.21.15.19:0/1312632036 conn(0x7f847c06b750 msgr2=0x7f847c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:50.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.476+0000 7f8484600640 1 -- 172.21.15.19:0/1312632036 shutdown_connections 2024-03-28T13:10:50.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.476+0000 7f8484600640 1 -- 172.21.15.19:0/1312632036 wait complete. 2024-03-28T13:10:50.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.478+0000 7f8484600640 1 Processor -- start 2024-03-28T13:10:50.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.478+0000 7f8484600640 1 -- start start 2024-03-28T13:10:50.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.479+0000 7f8484600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f847c0af2e0 0x7f847c1483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:50.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.479+0000 7f8484600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f847c0d07c0 0x7f847c0cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:50.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.479+0000 7f8484600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f847c153580 0x7f847c09a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:50.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.479+0000 7f8484600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f847c15d4e0 con 0x7f847c0d07c0 2024-03-28T13:10:50.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.479+0000 7f8484600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f847c15d200 con 0x7f847c0af2e0 2024-03-28T13:10:50.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.479+0000 7f8484600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f847c153960 con 0x7f847c153580 2024-03-28T13:10:50.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.480+0000 7f8481200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f847c0d07c0 0x7f847c0cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:50.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.480+0000 7f8481200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f847c0d07c0 0x7f847c0cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56842/0 (socket says 172.21.15.19:56842) 2024-03-28T13:10:50.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.480+0000 7f8481200640 1 -- 172.21.15.19:0/372897463 learned_addr learned my addr 172.21.15.19:0/372897463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:50.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.480+0000 7f8481c00640 1 --2- 172.21.15.19:0/372897463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f847c0af2e0 0x7f847c1483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:50.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.480+0000 7f8482600640 1 --2- 172.21.15.19:0/372897463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f847c153580 0x7f847c09a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:50.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.480+0000 7f8481200640 1 -- 172.21.15.19:0/372897463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f847c153580 msgr2=0x7f847c09a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:50.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.480+0000 7f8481200640 1 --2- 172.21.15.19:0/372897463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f847c153580 0x7f847c09a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.480+0000 7f8481200640 1 -- 172.21.15.19:0/372897463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f847c0af2e0 msgr2=0x7f847c1483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:50.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.480+0000 7f8481200640 1 --2- 172.21.15.19:0/372897463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f847c0af2e0 0x7f847c1483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.480+0000 7f8481200640 1 -- 172.21.15.19:0/372897463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847c09c2e0 con 0x7f847c0d07c0 2024-03-28T13:10:50.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.481+0000 7f8481200640 1 --2- 172.21.15.19:0/372897463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f847c0d07c0 0x7f847c0cced0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f846800d800 tx=0x7f846800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:50.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.481+0000 7f8479600640 1 -- 172.21.15.19:0/372897463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468018030 con 0x7f847c0d07c0 2024-03-28T13:10:50.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.481+0000 7f8479600640 1 -- 172.21.15.19:0/372897463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84680096a0 con 0x7f847c0d07c0 2024-03-28T13:10:50.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.481+0000 7f8479600640 1 -- 172.21.15.19:0/372897463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468009870 con 0x7f847c0d07c0 2024-03-28T13:10:50.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.481+0000 7f8484600640 1 -- 172.21.15.19:0/372897463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f847c09c5c0 con 0x7f847c0d07c0 2024-03-28T13:10:50.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.481+0000 7f8484600640 1 -- 172.21.15.19:0/372897463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f847c09ce60 con 0x7f847c0d07c0 2024-03-28T13:10:50.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.482+0000 7f8484600640 1 -- 172.21.15.19:0/372897463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f847c0685c0 con 0x7f847c0d07c0 2024-03-28T13:10:50.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.483+0000 7f8479600640 1 -- 172.21.15.19:0/372897463 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f846801b070 con 0x7f847c0d07c0 2024-03-28T13:10:50.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.484+0000 7f8479600640 1 --2- 172.21.15.19:0/372897463 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f845c06c530 0x7f845c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:50.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.484+0000 7f8479600640 1 -- 172.21.15.19:0/372897463 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f846808b6d0 con 0x7f847c0d07c0 2024-03-28T13:10:50.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.485+0000 7f8481c00640 1 --2- 172.21.15.19:0/372897463 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f845c06c530 0x7f845c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:50.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.486+0000 7f8481c00640 1 --2- 172.21.15.19:0/372897463 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f845c06c530 0x7f845c06e9f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8470006fd0 tx=0x7f8470008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:50.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.489+0000 7f8479600640 1 -- 172.21.15.19:0/372897463 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8468059fa0 con 0x7f847c0d07c0 2024-03-28T13:10:50.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.770+0000 7f8484600640 1 -- 172.21.15.19:0/372897463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f847c159ff0 con 0x7f847c0d07c0 2024-03-28T13:10:50.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.774+0000 7f8479600640 1 -- 172.21.15.19:0/372897463 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f84680598a0 con 0x7f847c0d07c0 2024-03-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.776+0000 7f8484600640 1 -- 172.21.15.19:0/372897463 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f845c06c530 msgr2=0x7f845c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.776+0000 7f8484600640 1 --2- 172.21.15.19:0/372897463 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f845c06c530 0x7f845c06e9f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8470006fd0 tx=0x7f8470008040 comp rx=0 tx=0).stop 2024-03-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.776+0000 7f8484600640 1 -- 172.21.15.19:0/372897463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f847c0d07c0 msgr2=0x7f847c0cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.776+0000 7f8484600640 1 --2- 172.21.15.19:0/372897463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f847c0d07c0 0x7f847c0cced0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f846800d800 tx=0x7f846800dcc0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.777+0000 7f8484600640 1 -- 172.21.15.19:0/372897463 shutdown_connections 2024-03-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.777+0000 7f8484600640 1 --2- 172.21.15.19:0/372897463 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f845c06c530 0x7f845c06e9f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.777+0000 7f8484600640 1 --2- 172.21.15.19:0/372897463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f847c153580 0x7f847c09a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.777+0000 7f8484600640 1 --2- 172.21.15.19:0/372897463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f847c0d07c0 0x7f847c0cced0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.777+0000 7f8484600640 1 --2- 172.21.15.19:0/372897463 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f847c0af2e0 0x7f847c1483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:50.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.777+0000 7f8484600640 1 -- 172.21.15.19:0/372897463 >> 172.21.15.19:0/372897463 conn(0x7f847c06b750 msgr2=0x7f847c15a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:50.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.777+0000 7f8484600640 1 -- 172.21.15.19:0/372897463 shutdown_connections 2024-03-28T13:10:50.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:50.777+0000 7f8484600640 1 -- 172.21.15.19:0/372897463 wait complete. 2024-03-28T13:10:50.797 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-28T13:10:51.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.016+0000 7fdcf3000640 1 Processor -- start 2024-03-28T13:10:51.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.016+0000 7fdcf3000640 1 -- start start 2024-03-28T13:10:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.017+0000 7fdcf3000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdcec0d0990 0x7fdcec0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.017+0000 7fdcf3000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdcec0af2e0 0x7fdcec1600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.017+0000 7fdcf3000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcec0afab0 con 0x7fdcec153580 2024-03-28T13:10:51.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.017+0000 7fdcf3000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdcec15cc80 con 0x7fdcec0af2e0 2024-03-28T13:10:51.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.017+0000 7fdcf3000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fdcec15da00 con 0x7fdcec0d0990 2024-03-28T13:10:51.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.017+0000 7fdcebe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdcec153580 0x7fdcec0d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58396/0 (socket says 172.21.15.19:58396) 2024-03-28T13:10:51.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.018+0000 7fdcebe00640 1 -- 172.21.15.19:0/2093877662 learned_addr learned my addr 172.21.15.19:0/2093877662 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.018+0000 7fdceb400640 1 --2- 172.21.15.19:0/2093877662 >> v2:172.21.15.188:3300/0 conn(0x7fdcec0d0990 0x7fdcec0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.018+0000 7fdcf1000640 1 --2- 172.21.15.19:0/2093877662 >> v2:172.21.15.19:3300/0 conn(0x7fdcec0af2e0 0x7fdcec1600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.018+0000 7fdceaa00640 1 -- 172.21.15.19:0/2093877662 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3352095717 0 0) 0x7fdcec0afab0 con 0x7fdcec153580 2024-03-28T13:10:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.018+0000 7fdceaa00640 1 -- 172.21.15.19:0/2093877662 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdcd0003520 con 0x7fdcec153580 2024-03-28T13:10:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.019+0000 7fdcf1000640 1 -- 172.21.15.19:0/2093877662 >> v2:172.21.15.188:3300/0 conn(0x7fdcec0d0990 msgr2=0x7fdcec0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.019+0000 7fdcf1000640 1 --2- 172.21.15.19:0/2093877662 >> v2:172.21.15.188:3300/0 conn(0x7fdcec0d0990 0x7fdcec0aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.019+0000 7fdceaa00640 1 -- 172.21.15.19:0/2093877662 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2566323653 0 0) 0x7fdcd0003520 con 0x7fdcec153580 2024-03-28T13:10:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.019+0000 7fdcf1000640 1 -- 172.21.15.19:0/2093877662 >> v1:172.21.15.19:6789/0 conn(0x7fdcec153580 legacy=0x7fdcec0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.019+0000 7fdcf1000640 1 -- 172.21.15.19:0/2093877662 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec15cf20 con 0x7fdcec0af2e0 2024-03-28T13:10:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.020+0000 7fdcf1000640 1 --2- 172.21.15.19:0/2093877662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcec0af2e0 0x7fdcec1600d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdcec15a6b0 tx=0x7fdce402b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a752357f72f528 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.021+0000 7fdceaa00640 1 -- 172.21.15.19:0/2093877662 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4037070 con 0x7fdcec0af2e0 2024-03-28T13:10:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.021+0000 7fdceaa00640 1 -- 172.21.15.19:0/2093877662 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce4030510 con 0x7fdcec0af2e0 2024-03-28T13:10:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.021+0000 7fdceaa00640 1 -- 172.21.15.19:0/2093877662 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc0019f0 con 0x7fdcec0af2e0 2024-03-28T13:10:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.021+0000 7fdcf3000640 1 -- 172.21.15.19:0/2093877662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcec0af2e0 msgr2=0x7fdcec1600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.021+0000 7fdcf3000640 1 --2- 172.21.15.19:0/2093877662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcec0af2e0 0x7fdcec1600d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdcec15a6b0 tx=0x7fdce402b710 comp rx=0 tx=0).stop 2024-03-28T13:10:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.022+0000 7fdcf3000640 1 -- 172.21.15.19:0/2093877662 shutdown_connections 2024-03-28T13:10:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.022+0000 7fdcf3000640 1 --2- 172.21.15.19:0/2093877662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcec0af2e0 0x7fdcec1600d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.022+0000 7fdcf3000640 1 --2- 172.21.15.19:0/2093877662 >> v2:172.21.15.188:3300/0 conn(0x7fdcec0d0990 0x7fdcec0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.022+0000 7fdcf3000640 1 -- 172.21.15.19:0/2093877662 >> 172.21.15.19:0/2093877662 conn(0x7fdcec06b750 msgr2=0x7fdcec06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.022+0000 7fdcf3000640 1 -- 172.21.15.19:0/2093877662 shutdown_connections 2024-03-28T13:10:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.022+0000 7fdcf3000640 1 -- 172.21.15.19:0/2093877662 wait complete. 2024-03-28T13:10:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.024+0000 7fdcf3000640 1 Processor -- start 2024-03-28T13:10:51.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.024+0000 7fdcf3000640 1 -- start start 2024-03-28T13:10:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.025+0000 7fdcf3000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcec0af2e0 0x7fdcec156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.025+0000 7fdcf3000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcec0d0990 0x7fdcec14b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.025+0000 7fdcf3000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcec153580 0x7fdcec1511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.025+0000 7fdcf3000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec153960 con 0x7fdcec0d0990 2024-03-28T13:10:51.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.025+0000 7fdcf3000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec15d4e0 con 0x7fdcec0af2e0 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.025+0000 7fdcf3000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdcec0af6c0 con 0x7fdcec153580 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.025+0000 7fdceb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcec0d0990 0x7fdcec14b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.025+0000 7fdceb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcec0d0990 0x7fdcec14b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56860/0 (socket says 172.21.15.19:56860) 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.025+0000 7fdceb400640 1 -- 172.21.15.19:0/1012073098 learned_addr learned my addr 172.21.15.19:0/1012073098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.025+0000 7fdceb400640 1 -- 172.21.15.19:0/1012073098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcec153580 msgr2=0x7fdcec1511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.025+0000 7fdcf1000640 1 --2- 172.21.15.19:0/1012073098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcec153580 0x7fdcec1511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.025+0000 7fdcebe00640 1 --2- 172.21.15.19:0/1012073098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcec0af2e0 0x7fdcec156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.026+0000 7fdceb400640 1 --2- 172.21.15.19:0/1012073098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcec153580 0x7fdcec1511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.026+0000 7fdceb400640 1 -- 172.21.15.19:0/1012073098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcec0af2e0 msgr2=0x7fdcec156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.026+0000 7fdceb400640 1 --2- 172.21.15.19:0/1012073098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcec0af2e0 0x7fdcec156030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.026+0000 7fdceb400640 1 -- 172.21.15.19:0/1012073098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec153b80 con 0x7fdcec0d0990 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.026+0000 7fdceb400640 1 --2- 172.21.15.19:0/1012073098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcec0d0990 0x7fdcec14b720 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400e870 tx=0x7fdcd400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.026+0000 7fdce3e00640 1 -- 172.21.15.19:0/1012073098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4017020 con 0x7fdcec0d0990 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.026+0000 7fdce3e00640 1 -- 172.21.15.19:0/1012073098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcd400ee60 con 0x7fdcec0d0990 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.026+0000 7fdce3e00640 1 -- 172.21.15.19:0/1012073098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4004460 con 0x7fdcec0d0990 2024-03-28T13:10:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.026+0000 7fdcf3000640 1 -- 172.21.15.19:0/1012073098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec1516f0 con 0x7fdcec0d0990 2024-03-28T13:10:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.026+0000 7fdcf3000640 1 -- 172.21.15.19:0/1012073098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec154100 con 0x7fdcec0d0990 2024-03-28T13:10:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.028+0000 7fdcf3000640 1 -- 172.21.15.19:0/1012073098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcec0685c0 con 0x7fdcec0d0990 2024-03-28T13:10:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.029+0000 7fdce3e00640 1 -- 172.21.15.19:0/1012073098 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdcd401a020 con 0x7fdcec0d0990 2024-03-28T13:10:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.029+0000 7fdce3e00640 1 --2- 172.21.15.19:0/1012073098 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdcc806c530 0x7fdcc806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.030+0000 7fdce3e00640 1 -- 172.21.15.19:0/1012073098 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fdcd408b450 con 0x7fdcec0d0990 2024-03-28T13:10:51.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.030+0000 7fdcebe00640 1 --2- 172.21.15.19:0/1012073098 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdcc806c530 0x7fdcc806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.032+0000 7fdcebe00640 1 --2- 172.21.15.19:0/1012073098 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdcc806c530 0x7fdcc806e9f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc001560 tx=0x7fdcdc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:51.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.035+0000 7fdce3e00640 1 -- 172.21.15.19:0/1012073098 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdcd4021030 con 0x7fdcec0d0990 2024-03-28T13:10:51.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.316+0000 7fdcf3000640 1 -- 172.21.15.19:0/1012073098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fdcec06eca0 con 0x7fdcec0d0990 2024-03-28T13:10:51.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.320+0000 7fdce3e00640 1 -- 172.21.15.19:0/1012073098 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fdcd4059dc0 con 0x7fdcec0d0990 2024-03-28T13:10:51.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.324+0000 7fdcf3000640 1 -- 172.21.15.19:0/1012073098 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdcc806c530 msgr2=0x7fdcc806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:51.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.324+0000 7fdcf3000640 1 --2- 172.21.15.19:0/1012073098 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdcc806c530 0x7fdcc806e9f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc001560 tx=0x7fdcdc009040 comp rx=0 tx=0).stop 2024-03-28T13:10:51.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.324+0000 7fdcf3000640 1 -- 172.21.15.19:0/1012073098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcec0d0990 msgr2=0x7fdcec14b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:51.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.324+0000 7fdcf3000640 1 --2- 172.21.15.19:0/1012073098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcec0d0990 0x7fdcec14b720 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400e870 tx=0x7fdcd400ed30 comp rx=0 tx=0).stop 2024-03-28T13:10:51.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.324+0000 7fdcf3000640 1 -- 172.21.15.19:0/1012073098 shutdown_connections 2024-03-28T13:10:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.324+0000 7fdcf3000640 1 --2- 172.21.15.19:0/1012073098 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdcc806c530 0x7fdcc806e9f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.324+0000 7fdcf3000640 1 --2- 172.21.15.19:0/1012073098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdcec153580 0x7fdcec1511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.324+0000 7fdcf3000640 1 --2- 172.21.15.19:0/1012073098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdcec0d0990 0x7fdcec14b720 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.324+0000 7fdcf3000640 1 --2- 172.21.15.19:0/1012073098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdcec0af2e0 0x7fdcec156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.324+0000 7fdcf3000640 1 -- 172.21.15.19:0/1012073098 >> 172.21.15.19:0/1012073098 conn(0x7fdcec06b750 msgr2=0x7fdcec0d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.324+0000 7fdcf3000640 1 -- 172.21.15.19:0/1012073098 shutdown_connections 2024-03-28T13:10:51.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.324+0000 7fdcf3000640 1 -- 172.21.15.19:0/1012073098 wait complete. 2024-03-28T13:10:51.342 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-28T13:10:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.554+0000 7f6930000640 1 Processor -- start 2024-03-28T13:10:51.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.554+0000 7f6930000640 1 -- start start 2024-03-28T13:10:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.555+0000 7f6930000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6928153580 0x7f69280d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.556+0000 7f6930000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f69280d07c0 0x7f69280aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.556+0000 7f6930000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69280afab0 con 0x7f69280af2e0 2024-03-28T13:10:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.556+0000 7f6930000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f69280d0c00 con 0x7f69280d07c0 2024-03-28T13:10:51.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.556+0000 7f6930000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f692815cc80 con 0x7f6928153580 2024-03-28T13:10:51.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.556+0000 7f692cc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f69280d07c0 0x7f69280aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:51.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.556+0000 7f692e000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f69280af2e0 0x7f692806d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58408/0 (socket says 172.21.15.19:58408) 2024-03-28T13:10:51.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.556+0000 7f692e000640 1 -- 172.21.15.19:0/4006734279 learned_addr learned my addr 172.21.15.19:0/4006734279 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:51.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.556+0000 7f692d600640 1 --2- 172.21.15.19:0/4006734279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6928153580 0x7f69280d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:51.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.557+0000 7f692cc00640 1 -- 172.21.15.19:0/4006734279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6928153580 msgr2=0x7f69280d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:51.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.557+0000 7f692cc00640 1 --2- 172.21.15.19:0/4006734279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6928153580 0x7f69280d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.557+0000 7f692cc00640 1 -- 172.21.15.19:0/4006734279 >> v1:172.21.15.19:6789/0 conn(0x7f69280af2e0 legacy=0x7f692806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:51.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.557+0000 7f692cc00640 1 -- 172.21.15.19:0/4006734279 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692815da00 con 0x7f69280d07c0 2024-03-28T13:10:51.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.558+0000 7f692cc00640 1 --2- 172.21.15.19:0/4006734279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69280d07c0 0x7f69280aeda0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f69140097a0 tx=0x7f691402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283907c54f76ba80 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:51.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.558+0000 7f6927e00640 1 -- 172.21.15.19:0/4006734279 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914031780 con 0x7f69280d07c0 2024-03-28T13:10:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.559+0000 7f6927e00640 1 -- 172.21.15.19:0/4006734279 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6914037070 con 0x7f69280d07c0 2024-03-28T13:10:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.559+0000 7f6927e00640 1 -- 172.21.15.19:0/4006734279 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914031b40 con 0x7f69280d07c0 2024-03-28T13:10:51.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.559+0000 7f6930000640 1 -- 172.21.15.19:0/4006734279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69280d07c0 msgr2=0x7f69280aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:51.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.559+0000 7f6930000640 1 --2- 172.21.15.19:0/4006734279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69280d07c0 0x7f69280aeda0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f69140097a0 tx=0x7f691402b820 comp rx=0 tx=0).stop 2024-03-28T13:10:51.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.559+0000 7f6930000640 1 -- 172.21.15.19:0/4006734279 shutdown_connections 2024-03-28T13:10:51.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.559+0000 7f6930000640 1 --2- 172.21.15.19:0/4006734279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69280d07c0 0x7f69280aeda0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.559+0000 7f6930000640 1 --2- 172.21.15.19:0/4006734279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6928153580 0x7f69280d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.559+0000 7f6930000640 1 -- 172.21.15.19:0/4006734279 >> 172.21.15.19:0/4006734279 conn(0x7f692806b750 msgr2=0x7f692806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:51.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.560+0000 7f6930000640 1 -- 172.21.15.19:0/4006734279 shutdown_connections 2024-03-28T13:10:51.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.560+0000 7f6930000640 1 -- 172.21.15.19:0/4006734279 wait complete. 2024-03-28T13:10:51.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.561+0000 7f6930000640 1 Processor -- start 2024-03-28T13:10:51.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.562+0000 7f6930000640 1 -- start start 2024-03-28T13:10:51.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.563+0000 7f6930000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69280af2e0 0x7f69280cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:51.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.563+0000 7f6930000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69280d07c0 0x7f692815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:51.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.563+0000 7f6930000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6928153580 0x7f6928152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:51.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.563+0000 7f6930000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f692815d4e0 con 0x7f69280af2e0 2024-03-28T13:10:51.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.563+0000 7f6930000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f692815d200 con 0x7f69280d07c0 2024-03-28T13:10:51.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.563+0000 7f6930000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6928153960 con 0x7f6928153580 2024-03-28T13:10:51.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.563+0000 7f692d600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69280af2e0 0x7f69280cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:51.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.563+0000 7f692d600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69280af2e0 0x7f69280cf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56886/0 (socket says 172.21.15.19:56886) 2024-03-28T13:10:51.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.563+0000 7f692d600640 1 -- 172.21.15.19:0/3793347954 learned_addr learned my addr 172.21.15.19:0/3793347954 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:51.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.564+0000 7f692d600640 1 -- 172.21.15.19:0/3793347954 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6928153580 msgr2=0x7f6928152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:51.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.564+0000 7f692d600640 1 --2- 172.21.15.19:0/3793347954 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6928153580 0x7f6928152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.564+0000 7f692d600640 1 -- 172.21.15.19:0/3793347954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69280d07c0 msgr2=0x7f692815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:51.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.564+0000 7f692cc00640 1 --2- 172.21.15.19:0/3793347954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69280d07c0 0x7f692815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:51.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.564+0000 7f692d600640 1 --2- 172.21.15.19:0/3793347954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69280d07c0 0x7f692815eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.564+0000 7f692d600640 1 -- 172.21.15.19:0/3793347954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69281556c0 con 0x7f69280af2e0 2024-03-28T13:10:51.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.564+0000 7f692cc00640 1 --2- 172.21.15.19:0/3793347954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69280d07c0 0x7f692815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:51.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.564+0000 7f692d600640 1 --2- 172.21.15.19:0/3793347954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69280af2e0 0x7f69280cf220 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f691c00d800 tx=0x7f691c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:51.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.564+0000 7f6925600640 1 -- 172.21.15.19:0/3793347954 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c00b6c0 con 0x7f69280af2e0 2024-03-28T13:10:51.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.564+0000 7f6925600640 1 -- 172.21.15.19:0/3793347954 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f691c019070 con 0x7f69280af2e0 2024-03-28T13:10:51.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.564+0000 7f6925600640 1 -- 172.21.15.19:0/3793347954 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c00ba80 con 0x7f69280af2e0 2024-03-28T13:10:51.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.565+0000 7f6930000640 1 -- 172.21.15.19:0/3793347954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69281569c0 con 0x7f69280af2e0 2024-03-28T13:10:51.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.565+0000 7f6930000640 1 -- 172.21.15.19:0/3793347954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6928155ee0 con 0x7f69280af2e0 2024-03-28T13:10:51.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.566+0000 7f6930000640 1 -- 172.21.15.19:0/3793347954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69280ae420 con 0x7f69280af2e0 2024-03-28T13:10:51.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.567+0000 7f6925600640 1 -- 172.21.15.19:0/3793347954 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f691c00bc20 con 0x7f69280af2e0 2024-03-28T13:10:51.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.568+0000 7f6925600640 1 --2- 172.21.15.19:0/3793347954 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f690806c530 0x7f690806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:51.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.568+0000 7f6925600640 1 -- 172.21.15.19:0/3793347954 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f691c08b7a0 con 0x7f69280af2e0 2024-03-28T13:10:51.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.568+0000 7f692cc00640 1 --2- 172.21.15.19:0/3793347954 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f690806c530 0x7f690806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:51.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.570+0000 7f692cc00640 1 --2- 172.21.15.19:0/3793347954 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f690806c530 0x7f690806e9f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6914036000 tx=0x7f6914008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:51.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.572+0000 7f6925600640 1 -- 172.21.15.19:0/3793347954 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f69280ae420 con 0x7f69280af2e0 2024-03-28T13:10:51.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.854+0000 7f6930000640 1 -- 172.21.15.19:0/3793347954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f69280d05a0 con 0x7f69280af2e0 2024-03-28T13:10:51.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.859+0000 7f6925600640 1 -- 172.21.15.19:0/3793347954 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f69280d05a0 con 0x7f69280af2e0 2024-03-28T13:10:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.861+0000 7f6930000640 1 -- 172.21.15.19:0/3793347954 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f690806c530 msgr2=0x7f690806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.861+0000 7f6930000640 1 --2- 172.21.15.19:0/3793347954 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f690806c530 0x7f690806e9f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6914036000 tx=0x7f6914008030 comp rx=0 tx=0).stop 2024-03-28T13:10:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.861+0000 7f6930000640 1 -- 172.21.15.19:0/3793347954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69280af2e0 msgr2=0x7f69280cf220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.861+0000 7f6930000640 1 --2- 172.21.15.19:0/3793347954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69280af2e0 0x7f69280cf220 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f691c00d800 tx=0x7f691c00dcc0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.861+0000 7f6930000640 1 -- 172.21.15.19:0/3793347954 shutdown_connections 2024-03-28T13:10:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.861+0000 7f6930000640 1 --2- 172.21.15.19:0/3793347954 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f690806c530 0x7f690806e9f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.861+0000 7f6930000640 1 --2- 172.21.15.19:0/3793347954 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6928153580 0x7f6928152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.861+0000 7f6930000640 1 --2- 172.21.15.19:0/3793347954 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69280d07c0 0x7f692815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.861+0000 7f6930000640 1 --2- 172.21.15.19:0/3793347954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69280af2e0 0x7f69280cf220 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.861+0000 7f6930000640 1 -- 172.21.15.19:0/3793347954 >> 172.21.15.19:0/3793347954 conn(0x7f692806b750 msgr2=0x7f692815a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:51.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.862+0000 7f6930000640 1 -- 172.21.15.19:0/3793347954 shutdown_connections 2024-03-28T13:10:51.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:51.862+0000 7f6930000640 1 -- 172.21.15.19:0/3793347954 wait complete. 2024-03-28T13:10:51.882 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-28T13:10:52.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.103+0000 7f35cbe00640 1 Processor -- start 2024-03-28T13:10:52.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.103+0000 7f35cbe00640 1 -- start start 2024-03-28T13:10:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.105+0000 7f35cbe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35cc06dd30 0x7f35cc0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.105+0000 7f35cbe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35cc0ce890 con 0x7f35cc13f190 2024-03-28T13:10:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.105+0000 7f35cbe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35cc0ceb50 con 0x7f35cc0cc880 2024-03-28T13:10:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.105+0000 7f35cbe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f35cc0cd230 con 0x7f35cc06dd30 2024-03-28T13:10:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.105+0000 7f35caa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f35cc13f190 0x7f35cc0cf630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58420/0 (socket says 172.21.15.19:58420) 2024-03-28T13:10:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.105+0000 7f35caa00640 1 -- 172.21.15.19:0/1134777253 learned_addr learned my addr 172.21.15.19:0/1134777253 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:52.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.105+0000 7f35cb400640 1 --2- 172.21.15.19:0/1134777253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35cc06dd30 0x7f35cc0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:52.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35ca000640 1 -- 172.21.15.19:0/1134777253 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1879748824 0 0) 0x7f35cc0ce890 con 0x7f35cc13f190 2024-03-28T13:10:52.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35ca000640 1 -- 172.21.15.19:0/1134777253 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35b0003520 con 0x7f35cc13f190 2024-03-28T13:10:52.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35ca000640 1 -- 172.21.15.19:0/1134777253 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2535410762 0 0) 0x7f35b0003520 con 0x7f35cc13f190 2024-03-28T13:10:52.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35ca000640 1 -- 172.21.15.19:0/1134777253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35cc06dd30 msgr2=0x7f35cc0cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:52.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35ca000640 1 --2- 172.21.15.19:0/1134777253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35cc06dd30 0x7f35cc0cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35ca000640 1 -- 172.21.15.19:0/1134777253 >> v1:172.21.15.188:6789/0 conn(0x7f35cc0cc880 legacy=0x7f35cc06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35ca000640 1 -- 172.21.15.19:0/1134777253 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc0cee30 con 0x7f35cc13f190 2024-03-28T13:10:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35ca000640 1 -- 172.21.15.19:0/1134777253 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f35bc001690 con 0x7f35cc13f190 2024-03-28T13:10:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35ca000640 1 -- 172.21.15.19:0/1134777253 >> v1:172.21.15.19:6789/0 conn(0x7f35cc13f190 legacy=0x7f35cc0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35ca000640 1 --2- 172.21.15.19:0/1134777253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35cc06dd30 0x7f35b00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35ca000640 1 --2- 172.21.15.19:0/1134777253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0003fb0 0x7f35b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35ca000640 1 --2- 172.21.15.19:0/1134777253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b0004d40 0x7f35b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35ca000640 1 -- 172.21.15.19:0/1134777253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc0cd230 con 0x7f35b0003fb0 2024-03-28T13:10:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35ca000640 1 -- 172.21.15.19:0/1134777253 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35b000bb00 con 0x7f35b0004d40 2024-03-28T13:10:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35ca000640 1 -- 172.21.15.19:0/1134777253 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f35b000bd00 con 0x7f35cc06dd30 2024-03-28T13:10:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.106+0000 7f35c2000640 1 --2- 172.21.15.19:0/1134777253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0003fb0 0x7f35b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.107+0000 7f35caa00640 1 --2- 172.21.15.19:0/1134777253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35cc06dd30 0x7f35b00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.107+0000 7f35cb400640 1 --2- 172.21.15.19:0/1134777253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b0004d40 0x7f35b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.107+0000 7f35c2000640 1 -- 172.21.15.19:0/1134777253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35cc06dd30 msgr2=0x7f35b00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:52.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.107+0000 7f35c2000640 1 --2- 172.21.15.19:0/1134777253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35cc06dd30 0x7f35b00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.107+0000 7f35c2000640 1 -- 172.21.15.19:0/1134777253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b0004d40 msgr2=0x7f35b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.107+0000 7f35c2000640 1 --2- 172.21.15.19:0/1134777253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b0004d40 0x7f35b00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.107+0000 7f35c2000640 1 -- 172.21.15.19:0/1134777253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b0003520 con 0x7f35b0003fb0 2024-03-28T13:10:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.107+0000 7f35caa00640 1 --2- 172.21.15.19:0/1134777253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35cc06dd30 0x7f35b00039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.107+0000 7f35cb400640 1 --2- 172.21.15.19:0/1134777253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b0004d40 0x7f35b00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:52.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.108+0000 7f35c2000640 1 --2- 172.21.15.19:0/1134777253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0003fb0 0x7f35b0004800 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f35b4001750 tx=0x7f35b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1e85c4df472a80 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:52.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.108+0000 7f35ca000640 1 -- 172.21.15.19:0/1134777253 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4001370 con 0x7f35b0003fb0 2024-03-28T13:10:52.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.108+0000 7f35ca000640 1 -- 172.21.15.19:0/1134777253 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35b40052d0 con 0x7f35b0003fb0 2024-03-28T13:10:52.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.108+0000 7f35ca000640 1 -- 172.21.15.19:0/1134777253 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4003830 con 0x7f35b0003fb0 2024-03-28T13:10:52.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.109+0000 7f35cbe00640 1 -- 172.21.15.19:0/1134777253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0003fb0 msgr2=0x7f35b0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:52.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.109+0000 7f35cbe00640 1 --2- 172.21.15.19:0/1134777253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0003fb0 0x7f35b0004800 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f35b4001750 tx=0x7f35b0005830 comp rx=0 tx=0).stop 2024-03-28T13:10:52.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.109+0000 7f35cb400640 1 -- 172.21.15.19:0/1134777253 reap_dead start 2024-03-28T13:10:52.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.109+0000 7f35cbe00640 1 -- 172.21.15.19:0/1134777253 shutdown_connections 2024-03-28T13:10:52.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.109+0000 7f35cbe00640 1 -- 172.21.15.19:0/1134777253 >> 172.21.15.19:0/1134777253 conn(0x7f35cc0b1ba0 msgr2=0x7f35cc0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:52.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.109+0000 7f35cbe00640 1 -- 172.21.15.19:0/1134777253 shutdown_connections 2024-03-28T13:10:52.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.109+0000 7f35cbe00640 1 -- 172.21.15.19:0/1134777253 wait complete. 2024-03-28T13:10:52.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.111+0000 7f35cbe00640 1 Processor -- start 2024-03-28T13:10:52.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.111+0000 7f35cbe00640 1 -- start start 2024-03-28T13:10:52.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.112+0000 7f35cbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0003fb0 0x7f35cc1f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:52.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.112+0000 7f35cbe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35cc1e9e10 0x7f35cc15b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:52.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.112+0000 7f35cbe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35cc15bca0 0x7f35cc13e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:52.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.112+0000 7f35cbe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc13f570 con 0x7f35b0003fb0 2024-03-28T13:10:52.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.112+0000 7f35cbe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35cc160c70 con 0x7f35cc1e9e10 2024-03-28T13:10:52.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.112+0000 7f35cbe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f35cc0cccf0 con 0x7f35cc15bca0 2024-03-28T13:10:52.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.112+0000 7f35caa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0003fb0 0x7f35cc1f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:52.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.112+0000 7f35caa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0003fb0 0x7f35cc1f4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56902/0 (socket says 172.21.15.19:56902) 2024-03-28T13:10:52.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.112+0000 7f35caa00640 1 -- 172.21.15.19:0/1204047927 learned_addr learned my addr 172.21.15.19:0/1204047927 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:52.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.113+0000 7f35cb400640 1 --2- 172.21.15.19:0/1204047927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35cc15bca0 0x7f35cc13e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:52.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.113+0000 7f35c2000640 1 --2- 172.21.15.19:0/1204047927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35cc1e9e10 0x7f35cc15b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:52.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.113+0000 7f35caa00640 1 -- 172.21.15.19:0/1204047927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35cc15bca0 msgr2=0x7f35cc13e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:52.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.113+0000 7f35caa00640 1 --2- 172.21.15.19:0/1204047927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35cc15bca0 0x7f35cc13e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.113+0000 7f35caa00640 1 -- 172.21.15.19:0/1204047927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35cc1e9e10 msgr2=0x7f35cc15b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:52.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.113+0000 7f35caa00640 1 --2- 172.21.15.19:0/1204047927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35cc1e9e10 0x7f35cc15b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.113+0000 7f35caa00640 1 -- 172.21.15.19:0/1204047927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35cc1e9050 con 0x7f35b0003fb0 2024-03-28T13:10:52.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.113+0000 7f35caa00640 1 --2- 172.21.15.19:0/1204047927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0003fb0 0x7f35cc1f4db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00d7d0 tx=0x7f35bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:52.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.113+0000 7f35c3400640 1 -- 172.21.15.19:0/1204047927 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc019070 con 0x7f35b0003fb0 2024-03-28T13:10:52.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.113+0000 7f35c3400640 1 -- 172.21.15.19:0/1204047927 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35bc0074e0 con 0x7f35b0003fb0 2024-03-28T13:10:52.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.113+0000 7f35c3400640 1 -- 172.21.15.19:0/1204047927 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35bc004a00 con 0x7f35b0003fb0 2024-03-28T13:10:52.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.113+0000 7f35cbe00640 1 -- 172.21.15.19:0/1204047927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35cc1e9330 con 0x7f35b0003fb0 2024-03-28T13:10:52.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.113+0000 7f35cbe00640 1 -- 172.21.15.19:0/1204047927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35cc1e98f0 con 0x7f35b0003fb0 2024-03-28T13:10:52.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.114+0000 7f35cbe00640 1 -- 172.21.15.19:0/1204047927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35cc075940 con 0x7f35b0003fb0 2024-03-28T13:10:52.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.115+0000 7f35c3400640 1 -- 172.21.15.19:0/1204047927 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f35bc004ba0 con 0x7f35b0003fb0 2024-03-28T13:10:52.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.117+0000 7f35c3400640 1 --2- 172.21.15.19:0/1204047927 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f35a806c560 0x7f35a806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:52.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.117+0000 7f35c3400640 1 -- 172.21.15.19:0/1204047927 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f35bc08ad60 con 0x7f35b0003fb0 2024-03-28T13:10:52.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.117+0000 7f35c2000640 1 --2- 172.21.15.19:0/1204047927 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f35a806c560 0x7f35a806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:52.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.118+0000 7f35c2000640 1 --2- 172.21.15.19:0/1204047927 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f35a806c560 0x7f35a806ea20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f35b4033000 tx=0x7f35b4009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:52.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.122+0000 7f35c3400640 1 -- 172.21.15.19:0/1204047927 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f35bc059630 con 0x7f35b0003fb0 2024-03-28T13:10:52.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.406+0000 7f35cbe00640 1 -- 172.21.15.19:0/1204047927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f35cc0b2870 con 0x7f35b0003fb0 2024-03-28T13:10:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.411+0000 7f35c3400640 1 -- 172.21.15.19:0/1204047927 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f35bc058f30 con 0x7f35b0003fb0 2024-03-28T13:10:52.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.414+0000 7f35cbe00640 1 -- 172.21.15.19:0/1204047927 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f35a806c560 msgr2=0x7f35a806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:52.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.414+0000 7f35cbe00640 1 --2- 172.21.15.19:0/1204047927 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f35a806c560 0x7f35a806ea20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f35b4033000 tx=0x7f35b4009030 comp rx=0 tx=0).stop 2024-03-28T13:10:52.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.414+0000 7f35cbe00640 1 -- 172.21.15.19:0/1204047927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0003fb0 msgr2=0x7f35cc1f4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:52.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.414+0000 7f35cbe00640 1 --2- 172.21.15.19:0/1204047927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0003fb0 0x7f35cc1f4db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00d7d0 tx=0x7f35bc00dc90 comp rx=0 tx=0).stop 2024-03-28T13:10:52.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.414+0000 7f35cbe00640 1 -- 172.21.15.19:0/1204047927 shutdown_connections 2024-03-28T13:10:52.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.414+0000 7f35cbe00640 1 --2- 172.21.15.19:0/1204047927 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f35a806c560 0x7f35a806ea20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.414+0000 7f35cbe00640 1 --2- 172.21.15.19:0/1204047927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35cc15bca0 0x7f35cc13e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.414+0000 7f35cbe00640 1 --2- 172.21.15.19:0/1204047927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35cc1e9e10 0x7f35cc15b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.414+0000 7f35cbe00640 1 --2- 172.21.15.19:0/1204047927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35b0003fb0 0x7f35cc1f4db0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.414+0000 7f35cbe00640 1 -- 172.21.15.19:0/1204047927 >> 172.21.15.19:0/1204047927 conn(0x7f35cc0b1ba0 msgr2=0x7f35cc15f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:52.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.414+0000 7f35cbe00640 1 -- 172.21.15.19:0/1204047927 shutdown_connections 2024-03-28T13:10:52.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.414+0000 7f35cbe00640 1 -- 172.21.15.19:0/1204047927 wait complete. 2024-03-28T13:10:52.433 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:10:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.649+0000 7ff14a200640 1 Processor -- start 2024-03-28T13:10:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.650+0000 7ff14a200640 1 -- start start 2024-03-28T13:10:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.651+0000 7ff14a200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff1441480a0 0x7ff14406ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.651+0000 7ff14a200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff1440726f0 0x7ff1440b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.651+0000 7ff14a200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff144148480 con 0x7ff14406b570 2024-03-28T13:10:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.651+0000 7ff14a200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff14406e5a0 con 0x7ff1440726f0 2024-03-28T13:10:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.651+0000 7ff14a200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff1440b41e0 con 0x7ff1441480a0 2024-03-28T13:10:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.652+0000 7ff13aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff1440726f0 0x7ff1440b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.652+0000 7ff13aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff1440726f0 0x7ff1440b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56904/0 (socket says 172.21.15.19:56904) 2024-03-28T13:10:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.652+0000 7ff13aa00640 1 -- 172.21.15.19:0/419372709 learned_addr learned my addr 172.21.15.19:0/419372709 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:52.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.652+0000 7ff143400640 1 --2- 172.21.15.19:0/419372709 >> v2:172.21.15.188:3300/0 conn(0x7ff1441480a0 0x7ff14406ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:52.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.652+0000 7ff13aa00640 1 -- 172.21.15.19:0/419372709 >> v2:172.21.15.188:3300/0 conn(0x7ff1441480a0 msgr2=0x7ff14406ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:52.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.652+0000 7ff13aa00640 1 --2- 172.21.15.19:0/419372709 >> v2:172.21.15.188:3300/0 conn(0x7ff1441480a0 0x7ff14406ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.652+0000 7ff13aa00640 1 -- 172.21.15.19:0/419372709 >> v1:172.21.15.188:6789/0 conn(0x7ff14406b570 legacy=0x7ff14406b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:52.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.652+0000 7ff13aa00640 1 -- 172.21.15.19:0/419372709 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14406df50 con 0x7ff1440726f0 2024-03-28T13:10:52.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.652+0000 7ff143400640 1 --2- 172.21.15.19:0/419372709 >> v2:172.21.15.188:3300/0 conn(0x7ff1441480a0 0x7ff14406ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:10:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.654+0000 7ff13aa00640 1 --2- 172.21.15.19:0/419372709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1440726f0 0x7ff1440b3890 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff144072b60 tx=0x7ff12c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d594b6a4dc3cd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.654+0000 7ff142a00640 1 -- 172.21.15.19:0/419372709 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c035020 con 0x7ff1440726f0 2024-03-28T13:10:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.655+0000 7ff142a00640 1 -- 172.21.15.19:0/419372709 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff12c031870 con 0x7ff1440726f0 2024-03-28T13:10:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.655+0000 7ff142a00640 1 -- 172.21.15.19:0/419372709 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c031c40 con 0x7ff1440726f0 2024-03-28T13:10:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.655+0000 7ff14a200640 1 -- 172.21.15.19:0/419372709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1440726f0 msgr2=0x7ff1440b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:52.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.655+0000 7ff14a200640 1 --2- 172.21.15.19:0/419372709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1440726f0 0x7ff1440b3890 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff144072b60 tx=0x7ff12c02b910 comp rx=0 tx=0).stop 2024-03-28T13:10:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.655+0000 7ff14a200640 1 -- 172.21.15.19:0/419372709 shutdown_connections 2024-03-28T13:10:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.655+0000 7ff14a200640 1 --2- 172.21.15.19:0/419372709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1440726f0 0x7ff1440b3890 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.655+0000 7ff14a200640 1 --2- 172.21.15.19:0/419372709 >> v2:172.21.15.188:3300/0 conn(0x7ff1441480a0 0x7ff14406ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.655+0000 7ff14a200640 1 -- 172.21.15.19:0/419372709 >> 172.21.15.19:0/419372709 conn(0x7ff1440b2230 msgr2=0x7ff1440b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.656+0000 7ff14a200640 1 -- 172.21.15.19:0/419372709 shutdown_connections 2024-03-28T13:10:52.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.656+0000 7ff14a200640 1 -- 172.21.15.19:0/419372709 wait complete. 2024-03-28T13:10:52.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.657+0000 7ff14a200640 1 Processor -- start 2024-03-28T13:10:52.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.657+0000 7ff14a200640 1 -- start start 2024-03-28T13:10:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.658+0000 7ff14a200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff14406b570 0x7ff14413b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.658+0000 7ff14a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1440726f0 0x7ff144140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.658+0000 7ff14a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1441480a0 0x7ff1441412a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:52.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.658+0000 7ff14a200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1440b44c0 con 0x7ff1440726f0 2024-03-28T13:10:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.658+0000 7ff14a200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff14406ef70 con 0x7ff14406b570 2024-03-28T13:10:52.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.658+0000 7ff14a200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff1440b3dd0 con 0x7ff1441480a0 2024-03-28T13:10:52.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.659+0000 7ff13aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1440726f0 0x7ff144140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:52.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.659+0000 7ff13aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1440726f0 0x7ff144140480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56910/0 (socket says 172.21.15.19:56910) 2024-03-28T13:10:52.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.659+0000 7ff13aa00640 1 -- 172.21.15.19:0/269986596 learned_addr learned my addr 172.21.15.19:0/269986596 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:52.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.659+0000 7ff143400640 1 --2- 172.21.15.19:0/269986596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff14406b570 0x7ff14413b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:52.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.659+0000 7ff143e00640 1 --2- 172.21.15.19:0/269986596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1441480a0 0x7ff1441412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:52.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.659+0000 7ff13aa00640 1 -- 172.21.15.19:0/269986596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1441480a0 msgr2=0x7ff1441412a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:52.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.659+0000 7ff13aa00640 1 --2- 172.21.15.19:0/269986596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1441480a0 0x7ff1441412a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.659+0000 7ff13aa00640 1 -- 172.21.15.19:0/269986596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff14406b570 msgr2=0x7ff14413b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.659+0000 7ff13aa00640 1 --2- 172.21.15.19:0/269986596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff14406b570 0x7ff14413b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.659+0000 7ff13aa00640 1 -- 172.21.15.19:0/269986596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff144156510 con 0x7ff1440726f0 2024-03-28T13:10:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.660+0000 7ff13aa00640 1 --2- 172.21.15.19:0/269986596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1440726f0 0x7ff144140480 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff12c02b950 tx=0x7ff12c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.660+0000 7ff13be00640 1 -- 172.21.15.19:0/269986596 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c040070 con 0x7ff1440726f0 2024-03-28T13:10:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.660+0000 7ff13be00640 1 -- 172.21.15.19:0/269986596 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff12c030d00 con 0x7ff1440726f0 2024-03-28T13:10:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.660+0000 7ff13be00640 1 -- 172.21.15.19:0/269986596 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c035040 con 0x7ff1440726f0 2024-03-28T13:10:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.660+0000 7ff14a200640 1 -- 172.21.15.19:0/269986596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14413de30 con 0x7ff1440726f0 2024-03-28T13:10:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.660+0000 7ff14a200640 1 -- 172.21.15.19:0/269986596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff14413e3f0 con 0x7ff1440726f0 2024-03-28T13:10:52.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.662+0000 7ff14a200640 1 -- 172.21.15.19:0/269986596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1440685c0 con 0x7ff1440726f0 2024-03-28T13:10:52.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.662+0000 7ff13be00640 1 -- 172.21.15.19:0/269986596 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff12c04b060 con 0x7ff1440726f0 2024-03-28T13:10:52.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.663+0000 7ff13be00640 1 --2- 172.21.15.19:0/269986596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff12006c530 0x7ff12006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:52.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.664+0000 7ff13be00640 1 -- 172.21.15.19:0/269986596 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7ff12c0ad140 con 0x7ff1440726f0 2024-03-28T13:10:52.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.664+0000 7ff143400640 1 --2- 172.21.15.19:0/269986596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff12006c530 0x7ff12006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:52.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.665+0000 7ff143400640 1 --2- 172.21.15.19:0/269986596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff12006c530 0x7ff12006e9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff14406ee50 tx=0x7ff134008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:52.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.668+0000 7ff13be00640 1 -- 172.21.15.19:0/269986596 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff12c07bab0 con 0x7ff1440726f0 2024-03-28T13:10:52.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.874+0000 7ff14a200640 1 -- 172.21.15.19:0/269986596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff144146f90 con 0x7ff1440726f0 2024-03-28T13:10:52.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.875+0000 7ff13be00640 1 -- 172.21.15.19:0/269986596 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+12891 (secure 0 0 0) 0x7ff12c07b3b0 con 0x7ff1440726f0 2024-03-28T13:10:52.876 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:52.876 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:10:48.598544+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:10:52.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.878+0000 7ff14a200640 1 -- 172.21.15.19:0/269986596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff12006c530 msgr2=0x7ff12006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:52.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.878+0000 7ff14a200640 1 --2- 172.21.15.19:0/269986596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff12006c530 0x7ff12006e9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff14406ee50 tx=0x7ff134008040 comp rx=0 tx=0).stop 2024-03-28T13:10:52.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.878+0000 7ff14a200640 1 -- 172.21.15.19:0/269986596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1440726f0 msgr2=0x7ff144140480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:52.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.878+0000 7ff14a200640 1 --2- 172.21.15.19:0/269986596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1440726f0 0x7ff144140480 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff12c02b950 tx=0x7ff12c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.878+0000 7ff14a200640 1 -- 172.21.15.19:0/269986596 shutdown_connections 2024-03-28T13:10:52.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.878+0000 7ff14a200640 1 --2- 172.21.15.19:0/269986596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff12006c530 0x7ff12006e9f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.878+0000 7ff14a200640 1 --2- 172.21.15.19:0/269986596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1441480a0 0x7ff1441412a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.878+0000 7ff14a200640 1 --2- 172.21.15.19:0/269986596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1440726f0 0x7ff144140480 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.878+0000 7ff14a200640 1 --2- 172.21.15.19:0/269986596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff14406b570 0x7ff14413b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:52.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.878+0000 7ff14a200640 1 -- 172.21.15.19:0/269986596 >> 172.21.15.19:0/269986596 conn(0x7ff1440b2230 msgr2=0x7ff1440719c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:52.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.878+0000 7ff14a200640 1 -- 172.21.15.19:0/269986596 shutdown_connections 2024-03-28T13:10:52.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:52.879+0000 7ff14a200640 1 -- 172.21.15.19:0/269986596 wait complete. 2024-03-28T13:10:52.897 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:10:52.898 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:10:53.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.127+0000 7ff80f200640 1 Processor -- start 2024-03-28T13:10:53.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.128+0000 7ff80f200640 1 -- start start 2024-03-28T13:10:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.129+0000 7ff80f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff808162920 0x7ff8081670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.129+0000 7ff80f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff808163ac0 0x7ff808071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.129+0000 7ff80f200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff8080d0280 0x7ff8080d0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.129+0000 7ff80f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff808163f20 con 0x7ff808162920 2024-03-28T13:10:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.130+0000 7ff80f200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff808071d60 con 0x7ff8080d0280 2024-03-28T13:10:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.130+0000 7ff80f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff8081665d0 con 0x7ff808163ac0 2024-03-28T13:10:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.130+0000 7ff807e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff808162920 0x7ff8081670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.130+0000 7ff807e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff808162920 0x7ff8081670b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56914/0 (socket says 172.21.15.19:56914) 2024-03-28T13:10:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.130+0000 7ff807e00640 1 -- 172.21.15.19:0/830645030 learned_addr learned my addr 172.21.15.19:0/830645030 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.130+0000 7ff807400640 1 --2- 172.21.15.19:0/830645030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff808163ac0 0x7ff808071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.130+0000 7ff80d200640 1 --2- 172.21.15.19:0/830645030 >> v2:172.21.15.188:3300/0 conn(0x7ff8080d0280 0x7ff8080d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.130+0000 7ff807e00640 1 -- 172.21.15.19:0/830645030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff808163ac0 msgr2=0x7ff808071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.130+0000 7ff807e00640 1 --2- 172.21.15.19:0/830645030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff808163ac0 0x7ff808071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.130+0000 7ff807e00640 1 -- 172.21.15.19:0/830645030 >> v2:172.21.15.188:3300/0 conn(0x7ff8080d0280 msgr2=0x7ff8080d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.130+0000 7ff807e00640 1 --2- 172.21.15.19:0/830645030 >> v2:172.21.15.188:3300/0 conn(0x7ff8080d0280 0x7ff8080d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.130+0000 7ff807e00640 1 -- 172.21.15.19:0/830645030 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8081668b0 con 0x7ff808162920 2024-03-28T13:10:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.130+0000 7ff80d200640 1 --2- 172.21.15.19:0/830645030 >> v2:172.21.15.188:3300/0 conn(0x7ff8080d0280 0x7ff8080d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.132+0000 7ff807e00640 1 --2- 172.21.15.19:0/830645030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff808162920 0x7ff8081670b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff80806e8e0 tx=0x7ff7f802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4deec925b822091 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.132+0000 7ff806a00640 1 -- 172.21.15.19:0/830645030 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f80317d0 con 0x7ff808162920 2024-03-28T13:10:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.132+0000 7ff806a00640 1 -- 172.21.15.19:0/830645030 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7f8037070 con 0x7ff808162920 2024-03-28T13:10:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.132+0000 7ff806a00640 1 -- 172.21.15.19:0/830645030 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8031b90 con 0x7ff808162920 2024-03-28T13:10:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.133+0000 7ff80f200640 1 -- 172.21.15.19:0/830645030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff808162920 msgr2=0x7ff8081670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.133+0000 7ff80f200640 1 --2- 172.21.15.19:0/830645030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff808162920 0x7ff8081670b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff80806e8e0 tx=0x7ff7f802b870 comp rx=0 tx=0).stop 2024-03-28T13:10:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.133+0000 7ff80f200640 1 -- 172.21.15.19:0/830645030 shutdown_connections 2024-03-28T13:10:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.133+0000 7ff80f200640 1 --2- 172.21.15.19:0/830645030 >> v2:172.21.15.188:3300/0 conn(0x7ff8080d0280 0x7ff8080d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.133+0000 7ff80f200640 1 --2- 172.21.15.19:0/830645030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff808163ac0 0x7ff808071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.133+0000 7ff80f200640 1 --2- 172.21.15.19:0/830645030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff808162920 0x7ff8081670b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.133+0000 7ff80f200640 1 -- 172.21.15.19:0/830645030 >> 172.21.15.19:0/830645030 conn(0x7ff80806b750 msgr2=0x7ff80806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:53.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.133+0000 7ff80f200640 1 -- 172.21.15.19:0/830645030 shutdown_connections 2024-03-28T13:10:53.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.133+0000 7ff80f200640 1 -- 172.21.15.19:0/830645030 wait complete. 2024-03-28T13:10:53.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.135+0000 7ff80f200640 1 Processor -- start 2024-03-28T13:10:53.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.135+0000 7ff80f200640 1 -- start start 2024-03-28T13:10:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.136+0000 7ff80f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8080d0280 0x7ff8080b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.136+0000 7ff80f200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff808162920 0x7ff80815c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.136+0000 7ff80f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff808163ac0 0x7ff8081f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.136+0000 7ff80f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff808072040 con 0x7ff8080d0280 2024-03-28T13:10:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.136+0000 7ff80f200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff808162d00 con 0x7ff808162920 2024-03-28T13:10:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.136+0000 7ff80f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff8080721c0 con 0x7ff808163ac0 2024-03-28T13:10:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.136+0000 7ff807e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8080d0280 0x7ff8080b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.136+0000 7ff807e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8080d0280 0x7ff8080b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56930/0 (socket says 172.21.15.19:56930) 2024-03-28T13:10:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.136+0000 7ff807e00640 1 -- 172.21.15.19:0/25001869 learned_addr learned my addr 172.21.15.19:0/25001869 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.137+0000 7ff807e00640 1 -- 172.21.15.19:0/25001869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff808163ac0 msgr2=0x7ff8081f85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.137+0000 7ff807e00640 1 --2- 172.21.15.19:0/25001869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff808163ac0 0x7ff8081f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.137+0000 7ff807e00640 1 -- 172.21.15.19:0/25001869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff808162920 msgr2=0x7ff80815c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.137+0000 7ff807e00640 1 --2- 172.21.15.19:0/25001869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff808162920 0x7ff80815c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.137+0000 7ff807e00640 1 -- 172.21.15.19:0/25001869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8080b3650 con 0x7ff8080d0280 2024-03-28T13:10:53.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.137+0000 7ff807e00640 1 --2- 172.21.15.19:0/25001869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8080d0280 0x7ff8080b2e20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff7f802b8b0 tx=0x7ff7f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:53.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.137+0000 7ff7ffe00640 1 -- 172.21.15.19:0/25001869 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8032870 con 0x7ff8080d0280 2024-03-28T13:10:53.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.137+0000 7ff7ffe00640 1 -- 172.21.15.19:0/25001869 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7f8032a30 con 0x7ff8080d0280 2024-03-28T13:10:53.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.137+0000 7ff7ffe00640 1 -- 172.21.15.19:0/25001869 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8032e20 con 0x7ff8080d0280 2024-03-28T13:10:53.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.137+0000 7ff80f200640 1 -- 172.21.15.19:0/25001869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8080b3930 con 0x7ff8080d0280 2024-03-28T13:10:53.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.138+0000 7ff80f200640 1 -- 172.21.15.19:0/25001869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8080b3360 con 0x7ff8080d0280 2024-03-28T13:10:53.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.139+0000 7ff80f200640 1 -- 172.21.15.19:0/25001869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8080685c0 con 0x7ff8080d0280 2024-03-28T13:10:53.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.140+0000 7ff7ffe00640 1 -- 172.21.15.19:0/25001869 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff7f804c060 con 0x7ff8080d0280 2024-03-28T13:10:53.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.141+0000 7ff7ffe00640 1 --2- 172.21.15.19:0/25001869 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff7e806c560 0x7ff7e806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.141+0000 7ff7ffe00640 1 -- 172.21.15.19:0/25001869 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7ff7f80acea0 con 0x7ff8080d0280 2024-03-28T13:10:53.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.141+0000 7ff807400640 1 --2- 172.21.15.19:0/25001869 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff7e806c560 0x7ff7e806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:53.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.142+0000 7ff807400640 1 --2- 172.21.15.19:0/25001869 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff7e806c560 0x7ff7e806ea20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff80815fef0 tx=0x7ff7f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:53.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.145+0000 7ff7ffe00640 1 -- 172.21.15.19:0/25001869 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff7f8040070 con 0x7ff8080d0280 2024-03-28T13:10:53.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.344+0000 7ff80f200640 1 -- 172.21.15.19:0/25001869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff80806ef60 con 0x7ff8080d0280 2024-03-28T13:10:53.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.345+0000 7ff7ffe00640 1 -- 172.21.15.19:0/25001869 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) ==== 93+0+10 (secure 0 0 0) 0x7ff7f807b800 con 0x7ff8080d0280 2024-03-28T13:10:53.345 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:10:53.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.349+0000 7ff80f200640 1 -- 172.21.15.19:0/25001869 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff7e806c560 msgr2=0x7ff7e806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:53.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.349+0000 7ff80f200640 1 --2- 172.21.15.19:0/25001869 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff7e806c560 0x7ff7e806ea20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff80815fef0 tx=0x7ff7f4008040 comp rx=0 tx=0).stop 2024-03-28T13:10:53.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.349+0000 7ff80f200640 1 -- 172.21.15.19:0/25001869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8080d0280 msgr2=0x7ff8080b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:53.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.349+0000 7ff80f200640 1 --2- 172.21.15.19:0/25001869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8080d0280 0x7ff8080b2e20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff7f802b8b0 tx=0x7ff7f80046c0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.349+0000 7ff80f200640 1 -- 172.21.15.19:0/25001869 shutdown_connections 2024-03-28T13:10:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.349+0000 7ff80f200640 1 --2- 172.21.15.19:0/25001869 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff7e806c560 0x7ff7e806ea20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.349+0000 7ff80f200640 1 --2- 172.21.15.19:0/25001869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff808163ac0 0x7ff8081f85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.349+0000 7ff80f200640 1 --2- 172.21.15.19:0/25001869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff808162920 0x7ff80815c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.349+0000 7ff80f200640 1 --2- 172.21.15.19:0/25001869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff8080d0280 0x7ff8080b2e20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.349+0000 7ff80f200640 1 -- 172.21.15.19:0/25001869 >> 172.21.15.19:0/25001869 conn(0x7ff80806b750 msgr2=0x7ff80815d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.349+0000 7ff80f200640 1 -- 172.21.15.19:0/25001869 shutdown_connections 2024-03-28T13:10:53.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.349+0000 7ff80f200640 1 -- 172.21.15.19:0/25001869 wait complete. 2024-03-28T13:10:53.366 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-28T13:10:53.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.587+0000 7f4d53800640 1 Processor -- start 2024-03-28T13:10:53.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.587+0000 7f4d53800640 1 -- start start 2024-03-28T13:10:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.589+0000 7f4d53800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d4c0d1bf0 0x7f4d4c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.589+0000 7f4d53800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d4c0705f0 0x7f4d4c06e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.589+0000 7f4d53800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d4c15edc0 con 0x7f4d4c070170 2024-03-28T13:10:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.589+0000 7f4d53800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f4d4c0d1fd0 con 0x7f4d4c0705f0 2024-03-28T13:10:53.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.589+0000 7f4d53800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4d4c15eb80 con 0x7f4d4c0d1bf0 2024-03-28T13:10:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.590+0000 7f4d51800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d4c0705f0 0x7f4d4c06e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.590+0000 7f4d50e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d4c0d1bf0 0x7f4d4c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.590+0000 7f4d51800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d4c0705f0 0x7f4d4c06e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:52018/0 (socket says 172.21.15.19:52018) 2024-03-28T13:10:53.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.590+0000 7f4d51800640 1 -- 172.21.15.19:0/2657357458 learned_addr learned my addr 172.21.15.19:0/2657357458 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.591+0000 7f4d51800640 1 -- 172.21.15.19:0/2657357458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d4c0d1bf0 msgr2=0x7f4d4c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.591+0000 7f4d51800640 1 --2- 172.21.15.19:0/2657357458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d4c0d1bf0 0x7f4d4c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.591+0000 7f4d51800640 1 -- 172.21.15.19:0/2657357458 >> v1:172.21.15.188:6789/0 conn(0x7f4d4c070170 legacy=0x7f4d4c06b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.591+0000 7f4d51800640 1 -- 172.21.15.19:0/2657357458 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c0b5e60 con 0x7f4d4c0705f0 2024-03-28T13:10:53.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.591+0000 7f4d50e00640 1 --2- 172.21.15.19:0/2657357458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d4c0d1bf0 0x7f4d4c15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:10:53.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.592+0000 7f4d51800640 1 --2- 172.21.15.19:0/2657357458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d4c0705f0 0x7f4d4c06e750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009e50 tx=0x7f4d4402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c04b24880990e09 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.593+0000 7f4d4be00640 1 -- 172.21.15.19:0/2657357458 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4402cb00 con 0x7f4d4c0705f0 2024-03-28T13:10:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.593+0000 7f4d4be00640 1 -- 172.21.15.19:0/2657357458 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d4402cca0 con 0x7f4d4c0705f0 2024-03-28T13:10:53.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.593+0000 7f4d4be00640 1 -- 172.21.15.19:0/2657357458 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d44034610 con 0x7f4d4c0705f0 2024-03-28T13:10:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.594+0000 7f4d53800640 1 -- 172.21.15.19:0/2657357458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d4c0705f0 msgr2=0x7f4d4c06e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.594+0000 7f4d53800640 1 --2- 172.21.15.19:0/2657357458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d4c0705f0 0x7f4d4c06e750 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009e50 tx=0x7f4d4402bf60 comp rx=0 tx=0).stop 2024-03-28T13:10:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.594+0000 7f4d53800640 1 -- 172.21.15.19:0/2657357458 shutdown_connections 2024-03-28T13:10:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.594+0000 7f4d53800640 1 --2- 172.21.15.19:0/2657357458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d4c0705f0 0x7f4d4c06e750 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.594+0000 7f4d53800640 1 --2- 172.21.15.19:0/2657357458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d4c0d1bf0 0x7f4d4c15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.594+0000 7f4d53800640 1 -- 172.21.15.19:0/2657357458 >> 172.21.15.19:0/2657357458 conn(0x7f4d4c0b6560 msgr2=0x7f4d4c0b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.594+0000 7f4d53800640 1 -- 172.21.15.19:0/2657357458 shutdown_connections 2024-03-28T13:10:53.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.594+0000 7f4d53800640 1 -- 172.21.15.19:0/2657357458 wait complete. 2024-03-28T13:10:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.596+0000 7f4d53800640 1 Processor -- start 2024-03-28T13:10:53.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.596+0000 7f4d53800640 1 -- start start 2024-03-28T13:10:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.597+0000 7f4d53800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d4c070170 0x7f4d4c14b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.597+0000 7f4d53800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d4c0705f0 0x7f4d4c09cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.597+0000 7f4d53800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d4c0d1bf0 0x7f4d4c09d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.597+0000 7f4d53800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c15fae0 con 0x7f4d4c0705f0 2024-03-28T13:10:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.597+0000 7f4d53800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c15f080 con 0x7f4d4c0d1bf0 2024-03-28T13:10:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.597+0000 7f4d53800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4d4c0b5b10 con 0x7f4d4c070170 2024-03-28T13:10:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.597+0000 7f4d43e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d4c0705f0 0x7f4d4c09cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.597+0000 7f4d43e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d4c0705f0 0x7f4d4c09cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56938/0 (socket says 172.21.15.19:56938) 2024-03-28T13:10:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.597+0000 7f4d43e00640 1 -- 172.21.15.19:0/3615098144 learned_addr learned my addr 172.21.15.19:0/3615098144 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:53.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.597+0000 7f4d50e00640 1 --2- 172.21.15.19:0/3615098144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d4c070170 0x7f4d4c14b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:53.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.597+0000 7f4d51800640 1 --2- 172.21.15.19:0/3615098144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d4c0d1bf0 0x7f4d4c09d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:53.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.598+0000 7f4d43e00640 1 -- 172.21.15.19:0/3615098144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d4c070170 msgr2=0x7f4d4c14b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:53.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.598+0000 7f4d43e00640 1 --2- 172.21.15.19:0/3615098144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d4c070170 0x7f4d4c14b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.598+0000 7f4d43e00640 1 -- 172.21.15.19:0/3615098144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d4c0d1bf0 msgr2=0x7f4d4c09d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:53.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.598+0000 7f4d43e00640 1 --2- 172.21.15.19:0/3615098144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d4c0d1bf0 0x7f4d4c09d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.598+0000 7f4d43e00640 1 -- 172.21.15.19:0/3615098144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c158640 con 0x7f4d4c0705f0 2024-03-28T13:10:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.598+0000 7f4d43e00640 1 --2- 172.21.15.19:0/3615098144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d4c0705f0 0x7f4d4c09cce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400c400 tx=0x7f4d3400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.598+0000 7f4d49600640 1 -- 172.21.15.19:0/3615098144 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d340052d0 con 0x7f4d4c0705f0 2024-03-28T13:10:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.598+0000 7f4d49600640 1 -- 172.21.15.19:0/3615098144 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d34005490 con 0x7f4d4c0705f0 2024-03-28T13:10:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.598+0000 7f4d49600640 1 -- 172.21.15.19:0/3615098144 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d34007260 con 0x7f4d4c0705f0 2024-03-28T13:10:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.598+0000 7f4d53800640 1 -- 172.21.15.19:0/3615098144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4c158920 con 0x7f4d4c0705f0 2024-03-28T13:10:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.599+0000 7f4d53800640 1 -- 172.21.15.19:0/3615098144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d4c158ee0 con 0x7f4d4c0705f0 2024-03-28T13:10:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.600+0000 7f4d53800640 1 -- 172.21.15.19:0/3615098144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d4c0685c0 con 0x7f4d4c0705f0 2024-03-28T13:10:53.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.601+0000 7f4d49600640 1 -- 172.21.15.19:0/3615098144 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4d34007400 con 0x7f4d4c0705f0 2024-03-28T13:10:53.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.602+0000 7f4d49600640 1 --2- 172.21.15.19:0/3615098144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d2806c530 0x7f4d2806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:53.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.602+0000 7f4d49600640 1 -- 172.21.15.19:0/3615098144 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7f4d3408a900 con 0x7f4d4c0705f0 2024-03-28T13:10:53.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.603+0000 7f4d50e00640 1 --2- 172.21.15.19:0/3615098144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d2806c530 0x7f4d2806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:53.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.603+0000 7f4d50e00640 1 --2- 172.21.15.19:0/3615098144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d2806c530 0x7f4d2806e9f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009a30 tx=0x7f4d3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:53.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.606+0000 7f4d49600640 1 -- 172.21.15.19:0/3615098144 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4d34059260 con 0x7f4d4c0705f0 2024-03-28T13:10:53.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.812+0000 7f4d53800640 1 -- 172.21.15.19:0/3615098144 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4d4c0afad0 con 0x7f4d4c0705f0 2024-03-28T13:10:53.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.813+0000 7f4d49600640 1 -- 172.21.15.19:0/3615098144 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) ==== 84+0+65 (secure 0 0 0) 0x7f4d34058b60 con 0x7f4d4c0705f0 2024-03-28T13:10:53.813 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:53.813 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-03-28T13:10:53.814 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:10:53.814 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-03-28T13:10:53.814 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-03-28T13:10:53.814 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:10:53.814 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-03-28T13:10:53.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.816+0000 7f4d53800640 1 -- 172.21.15.19:0/3615098144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d2806c530 msgr2=0x7f4d2806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:53.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.816+0000 7f4d53800640 1 --2- 172.21.15.19:0/3615098144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d2806c530 0x7f4d2806e9f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009a30 tx=0x7f4d3c008040 comp rx=0 tx=0).stop 2024-03-28T13:10:53.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.816+0000 7f4d53800640 1 -- 172.21.15.19:0/3615098144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d4c0705f0 msgr2=0x7f4d4c09cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:53.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.816+0000 7f4d53800640 1 --2- 172.21.15.19:0/3615098144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d4c0705f0 0x7f4d4c09cce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400c400 tx=0x7f4d3400c8c0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.816+0000 7f4d53800640 1 -- 172.21.15.19:0/3615098144 shutdown_connections 2024-03-28T13:10:53.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.816+0000 7f4d53800640 1 --2- 172.21.15.19:0/3615098144 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d2806c530 0x7f4d2806e9f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.816+0000 7f4d53800640 1 --2- 172.21.15.19:0/3615098144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d4c0d1bf0 0x7f4d4c09d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.816+0000 7f4d53800640 1 --2- 172.21.15.19:0/3615098144 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d4c0705f0 0x7f4d4c09cce0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.816+0000 7f4d53800640 1 --2- 172.21.15.19:0/3615098144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d4c070170 0x7f4d4c14b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:53.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.816+0000 7f4d53800640 1 -- 172.21.15.19:0/3615098144 >> 172.21.15.19:0/3615098144 conn(0x7f4d4c0b6560 msgr2=0x7f4d4c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:53.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.816+0000 7f4d53800640 1 -- 172.21.15.19:0/3615098144 shutdown_connections 2024-03-28T13:10:53.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:53.816+0000 7f4d53800640 1 -- 172.21.15.19:0/3615098144 wait complete. 2024-03-28T13:10:53.833 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-28T13:10:53.833 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-28T13:10:54.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.064+0000 7fb7f9600640 1 Processor -- start 2024-03-28T13:10:54.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.064+0000 7fb7f9600640 1 -- start start 2024-03-28T13:10:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.066+0000 7fb7f9600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7f415d520 0x7fb7f40cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.066+0000 7fb7f9600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb7f40b4a60 0x7fb7f40d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.066+0000 7fb7f9600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7f40b3fc0 con 0x7fb7f4070fc0 2024-03-28T13:10:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.066+0000 7fb7f9600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb7f4071f90 con 0x7fb7f40b4a60 2024-03-28T13:10:54.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.066+0000 7fb7f9600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f4071cb0 con 0x7fb7f415d520 2024-03-28T13:10:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.067+0000 7fb7f3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb7f4070fc0 0x7fb7f4071430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58422/0 (socket says 172.21.15.19:58422) 2024-03-28T13:10:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.067+0000 7fb7f3400640 1 -- 172.21.15.19:0/2857490960 learned_addr learned my addr 172.21.15.19:0/2857490960 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.067+0000 7fb7f2a00640 1 --2- 172.21.15.19:0/2857490960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7f415d520 0x7fb7f40cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.067+0000 7fb7ebe00640 1 --2- 172.21.15.19:0/2857490960 >> v2:172.21.15.188:3300/0 conn(0x7fb7f40b4a60 0x7fb7f40d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.067+0000 7fb7f2000640 1 -- 172.21.15.19:0/2857490960 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3418069091 0 0) 0x7fb7f40b3fc0 con 0x7fb7f4070fc0 2024-03-28T13:10:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.067+0000 7fb7f2000640 1 -- 172.21.15.19:0/2857490960 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7d8003520 con 0x7fb7f4070fc0 2024-03-28T13:10:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7f2000640 1 -- 172.21.15.19:0/2857490960 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 33703247 0 0) 0x7fb7d8003520 con 0x7fb7f4070fc0 2024-03-28T13:10:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7f2000640 1 -- 172.21.15.19:0/2857490960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7f415d520 msgr2=0x7fb7f40cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7f2000640 1 --2- 172.21.15.19:0/2857490960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7f415d520 0x7fb7f40cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7f2000640 1 -- 172.21.15.19:0/2857490960 >> v2:172.21.15.188:3300/0 conn(0x7fb7f40b4a60 msgr2=0x7fb7f40d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7f2000640 1 --2- 172.21.15.19:0/2857490960 >> v2:172.21.15.188:3300/0 conn(0x7fb7f40b4a60 0x7fb7f40d02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7f2000640 1 -- 172.21.15.19:0/2857490960 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f40b4260 con 0x7fb7f4070fc0 2024-03-28T13:10:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7f2000640 1 -- 172.21.15.19:0/2857490960 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fb7ec001a10 con 0x7fb7f4070fc0 2024-03-28T13:10:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7f2000640 1 -- 172.21.15.19:0/2857490960 >> v1:172.21.15.19:6789/0 conn(0x7fb7f4070fc0 legacy=0x7fb7f4071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7f2000640 1 --2- 172.21.15.19:0/2857490960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7f415d520 0x7fb7d80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7f2000640 1 --2- 172.21.15.19:0/2857490960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7d8003f80 0x7fb7d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7f2000640 1 --2- 172.21.15.19:0/2857490960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d8004d40 0x7fb7d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7f2000640 1 -- 172.21.15.19:0/2857490960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f4071f90 con 0x7fb7d8004d40 2024-03-28T13:10:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7f2000640 1 -- 172.21.15.19:0/2857490960 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f4071cb0 con 0x7fb7d8003f80 2024-03-28T13:10:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7f2000640 1 -- 172.21.15.19:0/2857490960 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb7d800b9c0 con 0x7fb7f415d520 2024-03-28T13:10:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7f3400640 1 --2- 172.21.15.19:0/2857490960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d8004d40 0x7fb7d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.068+0000 7fb7ebe00640 1 --2- 172.21.15.19:0/2857490960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7d8003f80 0x7fb7d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.069+0000 7fb7f2a00640 1 --2- 172.21.15.19:0/2857490960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7f415d520 0x7fb7d80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.069+0000 7fb7f3400640 1 -- 172.21.15.19:0/2857490960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7f415d520 msgr2=0x7fb7d80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.069+0000 7fb7f3400640 1 --2- 172.21.15.19:0/2857490960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7f415d520 0x7fb7d80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.069+0000 7fb7f3400640 1 -- 172.21.15.19:0/2857490960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7d8003f80 msgr2=0x7fb7d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:54.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.069+0000 7fb7f3400640 1 --2- 172.21.15.19:0/2857490960 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7d8003f80 0x7fb7d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.069+0000 7fb7f3400640 1 -- 172.21.15.19:0/2857490960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d8003520 con 0x7fb7d8004d40 2024-03-28T13:10:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.070+0000 7fb7f3400640 1 --2- 172.21.15.19:0/2857490960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d8004d40 0x7fb7d80051c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00a7c0 tx=0x7fb7d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43bf30566b32838 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.070+0000 7fb7f2000640 1 -- 172.21.15.19:0/2857490960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00a410 con 0x7fb7d8004d40 2024-03-28T13:10:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.070+0000 7fb7f2000640 1 -- 172.21.15.19:0/2857490960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7ec034630 con 0x7fb7d8004d40 2024-03-28T13:10:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.070+0000 7fb7f2000640 1 -- 172.21.15.19:0/2857490960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec034a20 con 0x7fb7d8004d40 2024-03-28T13:10:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.071+0000 7fb7f9600640 1 -- 172.21.15.19:0/2857490960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d8004d40 msgr2=0x7fb7d80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.071+0000 7fb7f9600640 1 --2- 172.21.15.19:0/2857490960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d8004d40 0x7fb7d80051c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00a7c0 tx=0x7fb7d8005830 comp rx=0 tx=0).stop 2024-03-28T13:10:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.071+0000 7fb7f3400640 1 -- 172.21.15.19:0/2857490960 reap_dead start 2024-03-28T13:10:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.071+0000 7fb7f9600640 1 -- 172.21.15.19:0/2857490960 shutdown_connections 2024-03-28T13:10:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.071+0000 7fb7f9600640 1 -- 172.21.15.19:0/2857490960 >> 172.21.15.19:0/2857490960 conn(0x7fb7f40d2240 msgr2=0x7fb7f40d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.071+0000 7fb7f9600640 1 -- 172.21.15.19:0/2857490960 shutdown_connections 2024-03-28T13:10:54.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.071+0000 7fb7f9600640 1 -- 172.21.15.19:0/2857490960 wait complete. 2024-03-28T13:10:54.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.073+0000 7fb7f9600640 1 Processor -- start 2024-03-28T13:10:54.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.073+0000 7fb7f9600640 1 -- start start 2024-03-28T13:10:54.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.074+0000 7fb7f9600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d8004d40 0x7fb7f40cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.074+0000 7fb7f9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f40c9280 0x7fb7f4149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.074+0000 7fb7f9600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f40b7080 0x7fb7f40b7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.074+0000 7fb7f9600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f415da80 con 0x7fb7f40c9280 2024-03-28T13:10:54.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.074+0000 7fb7f9600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb7f415d900 con 0x7fb7f40b7080 2024-03-28T13:10:54.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.074+0000 7fb7f9600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb7f40cfd80 con 0x7fb7d8004d40 2024-03-28T13:10:54.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.074+0000 7fb7ebe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f40c9280 0x7fb7f4149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.074+0000 7fb7ebe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f40c9280 0x7fb7f4149e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56952/0 (socket says 172.21.15.19:56952) 2024-03-28T13:10:54.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.074+0000 7fb7ebe00640 1 -- 172.21.15.19:0/1058374254 learned_addr learned my addr 172.21.15.19:0/1058374254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.074+0000 7fb7f3400640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f40b7080 0x7fb7f40b7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.074+0000 7fb7f2a00640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d8004d40 0x7fb7f40cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.075+0000 7fb7ebe00640 1 -- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d8004d40 msgr2=0x7fb7f40cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.075+0000 7fb7ebe00640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d8004d40 0x7fb7f40cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.075+0000 7fb7ebe00640 1 -- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f40b7080 msgr2=0x7fb7f40b7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.075+0000 7fb7ebe00640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f40b7080 0x7fb7f40b7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.075+0000 7fb7ebe00640 1 -- 172.21.15.19:0/1058374254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7f40cc0b0 con 0x7fb7f40c9280 2024-03-28T13:10:54.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.075+0000 7fb7f3400640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f40b7080 0x7fb7f40b7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:54.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.075+0000 7fb7f2a00640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d8004d40 0x7fb7f40cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:54.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.075+0000 7fb7ebe00640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f40c9280 0x7fb7f4149e70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc011600 tx=0x7fb7dc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:54.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.075+0000 7fb7eaa00640 1 -- 172.21.15.19:0/1058374254 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00f920 con 0x7fb7f40c9280 2024-03-28T13:10:54.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.075+0000 7fb7eaa00640 1 -- 172.21.15.19:0/1058374254 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7dc00fac0 con 0x7fb7f40c9280 2024-03-28T13:10:54.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.075+0000 7fb7eaa00640 1 -- 172.21.15.19:0/1058374254 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00fe70 con 0x7fb7f40c9280 2024-03-28T13:10:54.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.075+0000 7fb7f9600640 1 -- 172.21.15.19:0/1058374254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7f40c9820 con 0x7fb7f40c9280 2024-03-28T13:10:54.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.076+0000 7fb7f9600640 1 -- 172.21.15.19:0/1058374254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7f40c9de0 con 0x7fb7f40c9280 2024-03-28T13:10:54.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.077+0000 7fb7f9600640 1 -- 172.21.15.19:0/1058374254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7f40685c0 con 0x7fb7f40c9280 2024-03-28T13:10:54.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.078+0000 7fb7eaa00640 1 -- 172.21.15.19:0/1058374254 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb7dc015460 con 0x7fb7f40c9280 2024-03-28T13:10:54.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.080+0000 7fb7eaa00640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb7d006c560 0x7fb7d006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.080+0000 7fb7eaa00640 1 -- 172.21.15.19:0/1058374254 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4857+0+0 (secure 0 0 0) 0x7fb7dc08b900 con 0x7fb7f40c9280 2024-03-28T13:10:54.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.080+0000 7fb7f2a00640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb7d006c560 0x7fb7d006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.081+0000 7fb7f2a00640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb7d006c560 0x7fb7d006ea20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009a00 tx=0x7fb7e4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:54.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.084+0000 7fb7eaa00640 1 -- 172.21.15.19:0/1058374254 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb7dc019030 con 0x7fb7f40c9280 2024-03-28T13:10:54.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.289+0000 7fb7f9600640 1 -- 172.21.15.19:0/1058374254 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb7f40b2900 con 0x7fb7f40c9280 2024-03-28T13:10:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.708+0000 7fb7eaa00640 1 -- 172.21.15.19:0/1058374254 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) ==== 138+0+0 (secure 0 0 0) 0x7fb7dc05a260 con 0x7fb7f40c9280 2024-03-28T13:10:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-03-28T13:10:54.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.712+0000 7fb7f9600640 1 -- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb7d006c560 msgr2=0x7fb7d006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:54.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.712+0000 7fb7f9600640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb7d006c560 0x7fb7d006ea20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009a00 tx=0x7fb7e4008040 comp rx=0 tx=0).stop 2024-03-28T13:10:54.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.712+0000 7fb7f9600640 1 -- 172.21.15.19:0/1058374254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f40c9280 msgr2=0x7fb7f4149e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:54.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.712+0000 7fb7f9600640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f40c9280 0x7fb7f4149e70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc011600 tx=0x7fb7dc011ac0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.713+0000 7fb7f9600640 1 -- 172.21.15.19:0/1058374254 shutdown_connections 2024-03-28T13:10:54.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.713+0000 7fb7f9600640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb7d006c560 0x7fb7d006ea20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.713+0000 7fb7f9600640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb7f40b7080 0x7fb7f40b7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.713+0000 7fb7f9600640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7f40c9280 0x7fb7f4149e70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.713+0000 7fb7f9600640 1 --2- 172.21.15.19:0/1058374254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb7d8004d40 0x7fb7f40cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.713+0000 7fb7f9600640 1 -- 172.21.15.19:0/1058374254 >> 172.21.15.19:0/1058374254 conn(0x7fb7f40d2240 msgr2=0x7fb7f40b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:54.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.713+0000 7fb7f9600640 1 -- 172.21.15.19:0/1058374254 shutdown_connections 2024-03-28T13:10:54.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.713+0000 7fb7f9600640 1 -- 172.21.15.19:0/1058374254 wait complete. 2024-03-28T13:10:54.739 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-28T13:10:54.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.964+0000 7fa72a400640 1 Processor -- start 2024-03-28T13:10:54.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.964+0000 7fa72a400640 1 -- start start 2024-03-28T13:10:54.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.965+0000 7fa72a400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7241457e0 0x7fa7240b4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.965+0000 7fa72a400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7240b5ed0 con 0x7fa7240b4580 2024-03-28T13:10:54.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.965+0000 7fa72a400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7240b61b0 con 0x7fa72406bc80 2024-03-28T13:10:54.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.965+0000 7fa72a400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa724145bc0 con 0x7fa7241457e0 2024-03-28T13:10:54.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.966+0000 7fa722a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa7240b4580 0x7fa72406b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58426/0 (socket says 172.21.15.19:58426) 2024-03-28T13:10:54.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.966+0000 7fa722a00640 1 -- 172.21.15.19:0/2680000711 learned_addr learned my addr 172.21.15.19:0/2680000711 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.966+0000 7fa723400640 1 --2- 172.21.15.19:0/2680000711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7241457e0 0x7fa7240b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.966+0000 7fa722000640 1 -- 172.21.15.19:0/2680000711 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2056388662 0 0) 0x7fa7240b5ed0 con 0x7fa7240b4580 2024-03-28T13:10:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.966+0000 7fa722000640 1 -- 172.21.15.19:0/2680000711 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa708003520 con 0x7fa7240b4580 2024-03-28T13:10:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa722000640 1 -- 172.21.15.19:0/2680000711 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3522217957 0 0) 0x7fa708003520 con 0x7fa7240b4580 2024-03-28T13:10:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa722000640 1 -- 172.21.15.19:0/2680000711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7241457e0 msgr2=0x7fa7240b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa722000640 1 --2- 172.21.15.19:0/2680000711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7241457e0 0x7fa7240b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa722000640 1 -- 172.21.15.19:0/2680000711 >> v1:172.21.15.188:6789/0 conn(0x7fa72406bc80 legacy=0x7fa7240b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa722000640 1 -- 172.21.15.19:0/2680000711 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7240af870 con 0x7fa7240b4580 2024-03-28T13:10:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa722000640 1 -- 172.21.15.19:0/2680000711 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fa70c001670 con 0x7fa7240b4580 2024-03-28T13:10:54.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa722000640 1 -- 172.21.15.19:0/2680000711 >> v1:172.21.15.19:6789/0 conn(0x7fa7240b4580 legacy=0x7fa72406b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa722000640 1 --2- 172.21.15.19:0/2680000711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7080039e0 0x7fa708003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa722000640 1 --2- 172.21.15.19:0/2680000711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa708004800 0x7fa708004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa722000640 1 --2- 172.21.15.19:0/2680000711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7241457e0 0x7fa708005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa722000640 1 -- 172.21.15.19:0/2680000711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa724145bc0 con 0x7fa708004800 2024-03-28T13:10:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa722000640 1 -- 172.21.15.19:0/2680000711 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa70800bb50 con 0x7fa7080039e0 2024-03-28T13:10:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa722000640 1 -- 172.21.15.19:0/2680000711 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa70800bd50 con 0x7fa7241457e0 2024-03-28T13:10:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa722a00640 1 --2- 172.21.15.19:0/2680000711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa708004800 0x7fa708004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa723400640 1 --2- 172.21.15.19:0/2680000711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7080039e0 0x7fa708003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa723e00640 1 --2- 172.21.15.19:0/2680000711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7241457e0 0x7fa708005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.967+0000 7fa722a00640 1 -- 172.21.15.19:0/2680000711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7241457e0 msgr2=0x7fa708005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.968+0000 7fa722a00640 1 --2- 172.21.15.19:0/2680000711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7241457e0 0x7fa708005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.968+0000 7fa722a00640 1 -- 172.21.15.19:0/2680000711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7080039e0 msgr2=0x7fa708003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.968+0000 7fa722a00640 1 --2- 172.21.15.19:0/2680000711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7080039e0 0x7fa708003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.968+0000 7fa722a00640 1 -- 172.21.15.19:0/2680000711 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa708003520 con 0x7fa708004800 2024-03-28T13:10:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.968+0000 7fa723e00640 1 --2- 172.21.15.19:0/2680000711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7241457e0 0x7fa708005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:54.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.968+0000 7fa723400640 1 --2- 172.21.15.19:0/2680000711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7080039e0 0x7fa708003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.969+0000 7fa722a00640 1 --2- 172.21.15.19:0/2680000711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa708004800 0x7fa708004c80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00c160 tx=0x7fa708005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fe0e0cd0a2cc26 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.969+0000 7fa722000640 1 -- 172.21.15.19:0/2680000711 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c007c60 con 0x7fa708004800 2024-03-28T13:10:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.969+0000 7fa722000640 1 -- 172.21.15.19:0/2680000711 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa70c030070 con 0x7fa708004800 2024-03-28T13:10:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.969+0000 7fa722000640 1 -- 172.21.15.19:0/2680000711 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c005d50 con 0x7fa708004800 2024-03-28T13:10:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.970+0000 7fa72a400640 1 -- 172.21.15.19:0/2680000711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa708004800 msgr2=0x7fa708004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.970+0000 7fa72a400640 1 --2- 172.21.15.19:0/2680000711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa708004800 0x7fa708004c80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00c160 tx=0x7fa708005830 comp rx=0 tx=0).stop 2024-03-28T13:10:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.970+0000 7fa723e00640 1 -- 172.21.15.19:0/2680000711 reap_dead start 2024-03-28T13:10:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.970+0000 7fa72a400640 1 -- 172.21.15.19:0/2680000711 shutdown_connections 2024-03-28T13:10:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.970+0000 7fa72a400640 1 -- 172.21.15.19:0/2680000711 >> 172.21.15.19:0/2680000711 conn(0x7fa7240b65a0 msgr2=0x7fa7240b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.970+0000 7fa72a400640 1 -- 172.21.15.19:0/2680000711 shutdown_connections 2024-03-28T13:10:54.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.970+0000 7fa72a400640 1 -- 172.21.15.19:0/2680000711 wait complete. 2024-03-28T13:10:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.972+0000 7fa72a400640 1 Processor -- start 2024-03-28T13:10:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.972+0000 7fa72a400640 1 -- start start 2024-03-28T13:10:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.973+0000 7fa72a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa708004800 0x7fa72415bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.973+0000 7fa72a400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72409de50 0x7fa72409c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.973+0000 7fa72a400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa72409c830 0x7fa7240b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.973+0000 7fa72a400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7240b0840 con 0x7fa708004800 2024-03-28T13:10:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.973+0000 7fa72a400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa7240afca0 con 0x7fa72409de50 2024-03-28T13:10:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.973+0000 7fa72a400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa7240b09c0 con 0x7fa72409c830 2024-03-28T13:10:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.973+0000 7fa723400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa708004800 0x7fa72415bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.973+0000 7fa723400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa708004800 0x7fa72415bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56960/0 (socket says 172.21.15.19:56960) 2024-03-28T13:10:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.973+0000 7fa723400640 1 -- 172.21.15.19:0/920918028 learned_addr learned my addr 172.21.15.19:0/920918028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.973+0000 7fa723e00640 1 --2- 172.21.15.19:0/920918028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa72409c830 0x7fa7240b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.973+0000 7fa722a00640 1 --2- 172.21.15.19:0/920918028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72409de50 0x7fa72409c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.974+0000 7fa723400640 1 -- 172.21.15.19:0/920918028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa72409c830 msgr2=0x7fa7240b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.974+0000 7fa723400640 1 --2- 172.21.15.19:0/920918028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa72409c830 0x7fa7240b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.974+0000 7fa723400640 1 -- 172.21.15.19:0/920918028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72409de50 msgr2=0x7fa72409c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.974+0000 7fa723400640 1 --2- 172.21.15.19:0/920918028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72409de50 0x7fa72409c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.974+0000 7fa723400640 1 -- 172.21.15.19:0/920918028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa72409cdb0 con 0x7fa708004800 2024-03-28T13:10:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.974+0000 7fa722a00640 1 --2- 172.21.15.19:0/920918028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72409de50 0x7fa72409c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.974+0000 7fa723400640 1 --2- 172.21.15.19:0/920918028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa708004800 0x7fa72415bcb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa71400f350 tx=0x7fa71400f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.974+0000 7fa71b400640 1 -- 172.21.15.19:0/920918028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714003b90 con 0x7fa708004800 2024-03-28T13:10:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.974+0000 7fa71b400640 1 -- 172.21.15.19:0/920918028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa714003d30 con 0x7fa708004800 2024-03-28T13:10:54.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.974+0000 7fa71b400640 1 -- 172.21.15.19:0/920918028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa714004220 con 0x7fa708004800 2024-03-28T13:10:54.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.974+0000 7fa72a400640 1 -- 172.21.15.19:0/920918028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa72409d090 con 0x7fa708004800 2024-03-28T13:10:54.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.975+0000 7fa72a400640 1 -- 172.21.15.19:0/920918028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa72409d650 con 0x7fa708004800 2024-03-28T13:10:54.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.976+0000 7fa72a400640 1 -- 172.21.15.19:0/920918028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa724068cd0 con 0x7fa708004800 2024-03-28T13:10:54.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.978+0000 7fa71b400640 1 -- 172.21.15.19:0/920918028 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa714005d10 con 0x7fa708004800 2024-03-28T13:10:54.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.979+0000 7fa71b400640 1 --2- 172.21.15.19:0/920918028 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa70006c530 0x7fa70006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:54.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.979+0000 7fa71b400640 1 -- 172.21.15.19:0/920918028 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5227+0+0 (secure 0 0 0) 0x7fa71408c3c0 con 0x7fa708004800 2024-03-28T13:10:54.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.979+0000 7fa722a00640 1 --2- 172.21.15.19:0/920918028 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa70006c530 0x7fa70006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:54.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.981+0000 7fa722a00640 1 --2- 172.21.15.19:0/920918028 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa70006c530 0x7fa70006e9f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa70c034000 tx=0x7fa70c009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:54.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:54.983+0000 7fa71b400640 1 -- 172.21.15.19:0/920918028 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa714025070 con 0x7fa708004800 2024-03-28T13:10:55.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.188+0000 7fa72a400640 1 -- 172.21.15.19:0/920918028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa7240d0f40 con 0x7fa708004800 2024-03-28T13:10:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.715+0000 7fa71b400640 1 -- 172.21.15.19:0/920918028 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) ==== 170+0+0 (secure 0 0 0) 0x7fa714012070 con 0x7fa708004800 2024-03-28T13:10:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-03-28T13:10:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.720+0000 7fa72a400640 1 -- 172.21.15.19:0/920918028 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa70006c530 msgr2=0x7fa70006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.720+0000 7fa72a400640 1 --2- 172.21.15.19:0/920918028 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa70006c530 0x7fa70006e9f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa70c034000 tx=0x7fa70c009030 comp rx=0 tx=0).stop 2024-03-28T13:10:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.720+0000 7fa72a400640 1 -- 172.21.15.19:0/920918028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa708004800 msgr2=0x7fa72415bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.720+0000 7fa72a400640 1 --2- 172.21.15.19:0/920918028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa708004800 0x7fa72415bcb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa71400f350 tx=0x7fa71400f810 comp rx=0 tx=0).stop 2024-03-28T13:10:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.720+0000 7fa72a400640 1 -- 172.21.15.19:0/920918028 shutdown_connections 2024-03-28T13:10:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.720+0000 7fa72a400640 1 --2- 172.21.15.19:0/920918028 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa70006c530 0x7fa70006e9f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:55.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.720+0000 7fa72a400640 1 --2- 172.21.15.19:0/920918028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa72409c830 0x7fa7240b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:55.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.720+0000 7fa72a400640 1 --2- 172.21.15.19:0/920918028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa72409de50 0x7fa72409c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:55.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.720+0000 7fa72a400640 1 --2- 172.21.15.19:0/920918028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa708004800 0x7fa72415bcb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:55.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.720+0000 7fa72a400640 1 -- 172.21.15.19:0/920918028 >> 172.21.15.19:0/920918028 conn(0x7fa7240b65a0 msgr2=0x7fa7240b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:55.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.721+0000 7fa72a400640 1 -- 172.21.15.19:0/920918028 shutdown_connections 2024-03-28T13:10:55.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.722+0000 7fa72a400640 1 -- 172.21.15.19:0/920918028 wait complete. 2024-03-28T13:10:55.761 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-28T13:10:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.948+0000 7f58ea600640 1 Processor -- start 2024-03-28T13:10:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.949+0000 7f58ea600640 1 -- start start 2024-03-28T13:10:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.950+0000 7f58ea600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58e415e530 0x7f58e40cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.950+0000 7f58ea600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e4162350 con 0x7f58e406e1b0 2024-03-28T13:10:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.950+0000 7f58ea600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e4162630 con 0x7f58e40cf330 2024-03-28T13:10:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.950+0000 7f58ea600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f58e4162ed0 con 0x7f58e415e530 2024-03-28T13:10:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.951+0000 7f58e9200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58e415e530 0x7f58e40cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.951+0000 7f58e3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f58e406e1b0 0x7f58e40cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35632/0 (socket says 172.21.15.19:35632) 2024-03-28T13:10:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.951+0000 7f58e3e00640 1 -- 172.21.15.19:0/3100754365 learned_addr learned my addr 172.21.15.19:0/3100754365 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.951+0000 7f58e9200640 1 -- 172.21.15.19:0/3100754365 >> v1:172.21.15.188:6789/0 conn(0x7f58e40cf330 legacy=0x7f58e40d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.951+0000 7f58e9200640 1 -- 172.21.15.19:0/3100754365 >> v1:172.21.15.19:6789/0 conn(0x7f58e406e1b0 legacy=0x7f58e40cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.951+0000 7f58e9200640 1 -- 172.21.15.19:0/3100754365 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e4162910 con 0x7f58e415e530 2024-03-28T13:10:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.951+0000 7f58e3400640 1 -- 172.21.15.19:0/3100754365 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2416712084 0 0) 0x7f58e4162350 con 0x7f58e406e1b0 2024-03-28T13:10:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.952+0000 7f58e9200640 1 --2- 172.21.15.19:0/3100754365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e415e530 0x7f58e40cc720 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f58d80097a0 tx=0x7f58d802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91867975e143987 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.953+0000 7f58e3400640 1 -- 172.21.15.19:0/3100754365 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d80315a0 con 0x7f58e415e530 2024-03-28T13:10:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.953+0000 7f58e3400640 1 -- 172.21.15.19:0/3100754365 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d8037070 con 0x7f58e415e530 2024-03-28T13:10:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.953+0000 7f58e3400640 1 -- 172.21.15.19:0/3100754365 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8031960 con 0x7f58e415e530 2024-03-28T13:10:55.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.953+0000 7f58ea600640 1 -- 172.21.15.19:0/3100754365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e415e530 msgr2=0x7f58e40cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:55.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.953+0000 7f58ea600640 1 --2- 172.21.15.19:0/3100754365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e415e530 0x7f58e40cc720 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f58d80097a0 tx=0x7f58d802b840 comp rx=0 tx=0).stop 2024-03-28T13:10:55.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.953+0000 7f58ea600640 1 -- 172.21.15.19:0/3100754365 shutdown_connections 2024-03-28T13:10:55.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.953+0000 7f58ea600640 1 --2- 172.21.15.19:0/3100754365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e415e530 0x7f58e40cc720 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:55.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.953+0000 7f58ea600640 1 -- 172.21.15.19:0/3100754365 >> 172.21.15.19:0/3100754365 conn(0x7f58e4067280 msgr2=0x7f58e4067660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:55.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.954+0000 7f58ea600640 1 -- 172.21.15.19:0/3100754365 shutdown_connections 2024-03-28T13:10:55.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.954+0000 7f58ea600640 1 -- 172.21.15.19:0/3100754365 wait complete. 2024-03-28T13:10:55.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.955+0000 7f58ea600640 1 Processor -- start 2024-03-28T13:10:55.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.955+0000 7f58ea600640 1 -- start start 2024-03-28T13:10:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.956+0000 7f58ea600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e406e1b0 0x7f58e41f7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.956+0000 7f58ea600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58e40cf330 0x7f58e40b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.956+0000 7f58ea600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58e415e530 0x7f58e415c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.956+0000 7f58ea600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58e40cf780 con 0x7f58e406e1b0 2024-03-28T13:10:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.956+0000 7f58ea600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4161fe0 con 0x7f58e415e530 2024-03-28T13:10:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.956+0000 7f58ea600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f58e40cf900 con 0x7f58e40cf330 2024-03-28T13:10:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.957+0000 7f58e9200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e406e1b0 0x7f58e41f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.957+0000 7f58e9200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e406e1b0 0x7f58e41f7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43358/0 (socket says 172.21.15.19:43358) 2024-03-28T13:10:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.957+0000 7f58e9200640 1 -- 172.21.15.19:0/1983835747 learned_addr learned my addr 172.21.15.19:0/1983835747 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.957+0000 7f58e3e00640 1 --2- 172.21.15.19:0/1983835747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58e40cf330 0x7f58e40b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.957+0000 7f58e9c00640 1 --2- 172.21.15.19:0/1983835747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58e415e530 0x7f58e415c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.957+0000 7f58e9200640 1 -- 172.21.15.19:0/1983835747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58e40cf330 msgr2=0x7f58e40b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.957+0000 7f58e9200640 1 --2- 172.21.15.19:0/1983835747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58e40cf330 0x7f58e40b0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.957+0000 7f58e9200640 1 -- 172.21.15.19:0/1983835747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58e415e530 msgr2=0x7f58e415c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.957+0000 7f58e9200640 1 --2- 172.21.15.19:0/1983835747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58e415e530 0x7f58e415c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.957+0000 7f58e9200640 1 -- 172.21.15.19:0/1983835747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e40c6d90 con 0x7f58e406e1b0 2024-03-28T13:10:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.957+0000 7f58e9c00640 1 --2- 172.21.15.19:0/1983835747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58e415e530 0x7f58e415c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.958+0000 7f58e9200640 1 --2- 172.21.15.19:0/1983835747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e406e1b0 0x7f58e41f7700 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f58d8034040 tx=0x7f58d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.958+0000 7f58e0c00640 1 -- 172.21.15.19:0/1983835747 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8032840 con 0x7f58e406e1b0 2024-03-28T13:10:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.958+0000 7f58e0c00640 1 -- 172.21.15.19:0/1983835747 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d80329e0 con 0x7f58e406e1b0 2024-03-28T13:10:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.958+0000 7f58e0c00640 1 -- 172.21.15.19:0/1983835747 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d8032dd0 con 0x7f58e406e1b0 2024-03-28T13:10:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.958+0000 7f58ea600640 1 -- 172.21.15.19:0/1983835747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e40b1130 con 0x7f58e406e1b0 2024-03-28T13:10:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.958+0000 7f58ea600640 1 -- 172.21.15.19:0/1983835747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e40c7030 con 0x7f58e406e1b0 2024-03-28T13:10:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.959+0000 7f58ea600640 1 -- 172.21.15.19:0/1983835747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e4075940 con 0x7f58e406e1b0 2024-03-28T13:10:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.961+0000 7f58e0c00640 1 -- 172.21.15.19:0/1983835747 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f58d803b3f0 con 0x7f58e406e1b0 2024-03-28T13:10:55.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.962+0000 7f58e0c00640 1 --2- 172.21.15.19:0/1983835747 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f58c406c530 0x7f58c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:55.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.962+0000 7f58e0c00640 1 -- 172.21.15.19:0/1983835747 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5609+0+0 (secure 0 0 0) 0x7f58d802f030 con 0x7f58e406e1b0 2024-03-28T13:10:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.963+0000 7f58e3e00640 1 --2- 172.21.15.19:0/1983835747 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f58c406c530 0x7f58c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.964+0000 7f58e3e00640 1 --2- 172.21.15.19:0/1983835747 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f58c406c530 0x7f58c406e9f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f58d0007fd0 tx=0x7f58d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:55.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:55.968+0000 7f58e0c00640 1 -- 172.21.15.19:0/1983835747 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f58d8040070 con 0x7f58e406e1b0 2024-03-28T13:10:56.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.213+0000 7f58ea600640 1 -- 172.21.15.19:0/1983835747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f58e40b1650 con 0x7f58e406e1b0 2024-03-28T13:10:56.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.758+0000 7f58e0c00640 1 -- 172.21.15.19:0/1983835747 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f58d807bce0 con 0x7f58e406e1b0 2024-03-28T13:10:56.762 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 4 and data pool 5 2024-03-28T13:10:56.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.763+0000 7f58ea600640 1 -- 172.21.15.19:0/1983835747 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f58c406c530 msgr2=0x7f58c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:56.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.763+0000 7f58ea600640 1 --2- 172.21.15.19:0/1983835747 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f58c406c530 0x7f58c406e9f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f58d0007fd0 tx=0x7f58d0009040 comp rx=0 tx=0).stop 2024-03-28T13:10:56.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.763+0000 7f58ea600640 1 -- 172.21.15.19:0/1983835747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e406e1b0 msgr2=0x7f58e41f7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:56.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.763+0000 7f58ea600640 1 --2- 172.21.15.19:0/1983835747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e406e1b0 0x7f58e41f7700 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f58d8034040 tx=0x7f58d80046c0 comp rx=0 tx=0).stop 2024-03-28T13:10:56.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.763+0000 7f58ea600640 1 -- 172.21.15.19:0/1983835747 shutdown_connections 2024-03-28T13:10:56.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.763+0000 7f58ea600640 1 --2- 172.21.15.19:0/1983835747 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f58c406c530 0x7f58c406e9f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:56.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.763+0000 7f58ea600640 1 --2- 172.21.15.19:0/1983835747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58e415e530 0x7f58e415c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:56.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.763+0000 7f58ea600640 1 --2- 172.21.15.19:0/1983835747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58e40cf330 0x7f58e40b0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:56.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.763+0000 7f58ea600640 1 --2- 172.21.15.19:0/1983835747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58e406e1b0 0x7f58e41f7700 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:56.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.763+0000 7f58ea600640 1 -- 172.21.15.19:0/1983835747 >> 172.21.15.19:0/1983835747 conn(0x7f58e4067280 msgr2=0x7f58e40cec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:56.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.764+0000 7f58ea600640 1 -- 172.21.15.19:0/1983835747 shutdown_connections 2024-03-28T13:10:56.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.764+0000 7f58ea600640 1 -- 172.21.15.19:0/1983835747 wait complete. 2024-03-28T13:10:56.796 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:10:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.971+0000 7fafb0200640 1 Processor -- start 2024-03-28T13:10:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.971+0000 7fafb0200640 1 -- start start 2024-03-28T13:10:56.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.973+0000 7fafb0200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fafa80af830 0x7fafa8073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:56.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.973+0000 7fafb0200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafa80d0650 0x7fafa80d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:56.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.973+0000 7fafb0200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafa815fe80 con 0x7fafa815c390 2024-03-28T13:10:56.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.973+0000 7fafb0200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fafa80d1000 con 0x7fafa80af830 2024-03-28T13:10:56.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.973+0000 7fafb0200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fafa815c770 con 0x7fafa80d0650 2024-03-28T13:10:56.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.973+0000 7fafad800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fafa815c390 0x7fafa8160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35646/0 (socket says 172.21.15.19:35646) 2024-03-28T13:10:56.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.973+0000 7fafad800640 1 -- 172.21.15.19:0/2870348615 learned_addr learned my addr 172.21.15.19:0/2870348615 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:56.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.974+0000 7faface00640 1 --2- 172.21.15.19:0/2870348615 >> v2:172.21.15.188:3300/0 conn(0x7fafa80af830 0x7fafa8073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:56.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.974+0000 7fafa7e00640 1 -- 172.21.15.19:0/2870348615 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1862310003 0 0) 0x7fafa815fe80 con 0x7fafa815c390 2024-03-28T13:10:56.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.974+0000 7fafa7e00640 1 -- 172.21.15.19:0/2870348615 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf90003520 con 0x7fafa815c390 2024-03-28T13:10:56.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.974+0000 7fafae200640 1 --2- 172.21.15.19:0/2870348615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafa80d0650 0x7fafa80d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:56.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.974+0000 7fafa7e00640 1 -- 172.21.15.19:0/2870348615 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3147806153 0 0) 0x7faf90003520 con 0x7fafa815c390 2024-03-28T13:10:56.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.974+0000 7fafa7e00640 1 -- 172.21.15.19:0/2870348615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafa80d0650 msgr2=0x7fafa80d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:56.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.974+0000 7fafa7e00640 1 --2- 172.21.15.19:0/2870348615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafa80d0650 0x7fafa80d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:56.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.974+0000 7fafa7e00640 1 -- 172.21.15.19:0/2870348615 >> v2:172.21.15.188:3300/0 conn(0x7fafa80af830 msgr2=0x7fafa8073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:56.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.974+0000 7fafa7e00640 1 --2- 172.21.15.19:0/2870348615 >> v2:172.21.15.188:3300/0 conn(0x7fafa80af830 0x7fafa8073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.974+0000 7fafa7e00640 1 -- 172.21.15.19:0/2870348615 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa80ad120 con 0x7fafa815c390 2024-03-28T13:10:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.974+0000 7fafa7e00640 1 -- 172.21.15.19:0/2870348615 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7faf9c001670 con 0x7fafa815c390 2024-03-28T13:10:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.974+0000 7fafae200640 1 --2- 172.21.15.19:0/2870348615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafa80d0650 0x7fafa80d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:10:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.974+0000 7fafa7e00640 1 -- 172.21.15.19:0/2870348615 >> v1:172.21.15.19:6789/0 conn(0x7fafa815c390 legacy=0x7fafa8160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.974+0000 7fafa7e00640 1 --2- 172.21.15.19:0/2870348615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf900039b0 0x7faf90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.974+0000 7fafa7e00640 1 --2- 172.21.15.19:0/2870348615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafa80d0650 0x7faf90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.975+0000 7fafa7e00640 1 --2- 172.21.15.19:0/2870348615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf90004d40 0x7faf900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.975+0000 7fafa7e00640 1 -- 172.21.15.19:0/2870348615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fafa80d1000 con 0x7faf900039b0 2024-03-28T13:10:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.975+0000 7fafa7e00640 1 -- 172.21.15.19:0/2870348615 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faf9000b9c0 con 0x7faf90004d40 2024-03-28T13:10:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.975+0000 7fafa7e00640 1 -- 172.21.15.19:0/2870348615 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7faf9000bb40 con 0x7fafa80d0650 2024-03-28T13:10:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.975+0000 7fafad800640 1 --2- 172.21.15.19:0/2870348615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf900039b0 0x7faf90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.975+0000 7faface00640 1 --2- 172.21.15.19:0/2870348615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafa80d0650 0x7faf90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.975+0000 7fafae200640 1 --2- 172.21.15.19:0/2870348615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf90004d40 0x7faf900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.975+0000 7fafad800640 1 -- 172.21.15.19:0/2870348615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafa80d0650 msgr2=0x7faf90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:56.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.975+0000 7fafad800640 1 --2- 172.21.15.19:0/2870348615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafa80d0650 0x7faf90004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:56.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.975+0000 7fafad800640 1 -- 172.21.15.19:0/2870348615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf90004d40 msgr2=0x7faf900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:56.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.975+0000 7fafad800640 1 --2- 172.21.15.19:0/2870348615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf90004d40 0x7faf900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:56.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.975+0000 7fafad800640 1 -- 172.21.15.19:0/2870348615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf90003520 con 0x7faf900039b0 2024-03-28T13:10:56.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.975+0000 7fafae200640 1 --2- 172.21.15.19:0/2870348615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf90004d40 0x7faf900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:56.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.976+0000 7fafad800640 1 --2- 172.21.15.19:0/2870348615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf900039b0 0x7faf90003dd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00c160 tx=0x7faf90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b59ed936c928b70 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:56.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.977+0000 7fafa7e00640 1 -- 172.21.15.19:0/2870348615 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c007c80 con 0x7faf900039b0 2024-03-28T13:10:56.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.977+0000 7fafa7e00640 1 -- 172.21.15.19:0/2870348615 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf9c030070 con 0x7faf900039b0 2024-03-28T13:10:56.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.977+0000 7fafa7e00640 1 -- 172.21.15.19:0/2870348615 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c005d30 con 0x7faf900039b0 2024-03-28T13:10:56.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.977+0000 7fafb0200640 1 -- 172.21.15.19:0/2870348615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf900039b0 msgr2=0x7faf90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:56.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.977+0000 7fafb0200640 1 --2- 172.21.15.19:0/2870348615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf900039b0 0x7faf90003dd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00c160 tx=0x7faf90005830 comp rx=0 tx=0).stop 2024-03-28T13:10:56.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.977+0000 7fafae200640 1 -- 172.21.15.19:0/2870348615 reap_dead start 2024-03-28T13:10:56.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.977+0000 7fafb0200640 1 -- 172.21.15.19:0/2870348615 shutdown_connections 2024-03-28T13:10:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.977+0000 7fafb0200640 1 -- 172.21.15.19:0/2870348615 >> 172.21.15.19:0/2870348615 conn(0x7fafa806b990 msgr2=0x7fafa806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.977+0000 7fafb0200640 1 -- 172.21.15.19:0/2870348615 shutdown_connections 2024-03-28T13:10:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.977+0000 7fafb0200640 1 -- 172.21.15.19:0/2870348615 wait complete. 2024-03-28T13:10:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.979+0000 7fafb0200640 1 Processor -- start 2024-03-28T13:10:56.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.979+0000 7fafb0200640 1 -- start start 2024-03-28T13:10:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.980+0000 7fafb0200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf900039b0 0x7fafa814c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.980+0000 7fafb0200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafa80b2470 0x7fafa80b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.980+0000 7fafb0200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafa8154df0 0x7fafa8155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.980+0000 7fafb0200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8160120 con 0x7faf900039b0 2024-03-28T13:10:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.980+0000 7fafb0200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fafa80ad6e0 con 0x7fafa80b2470 2024-03-28T13:10:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.980+0000 7fafb0200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fafa8160d50 con 0x7fafa8154df0 2024-03-28T13:10:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.980+0000 7fafad800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf900039b0 0x7fafa814c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.980+0000 7fafad800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf900039b0 0x7fafa814c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43382/0 (socket says 172.21.15.19:43382) 2024-03-28T13:10:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.980+0000 7fafad800640 1 -- 172.21.15.19:0/3827516846 learned_addr learned my addr 172.21.15.19:0/3827516846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.981+0000 7fafae200640 1 --2- 172.21.15.19:0/3827516846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafa8154df0 0x7fafa8155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.981+0000 7faface00640 1 --2- 172.21.15.19:0/3827516846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafa80b2470 0x7fafa80b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.981+0000 7fafad800640 1 -- 172.21.15.19:0/3827516846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafa8154df0 msgr2=0x7fafa8155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.981+0000 7fafad800640 1 --2- 172.21.15.19:0/3827516846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafa8154df0 0x7fafa8155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:56.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.981+0000 7fafad800640 1 -- 172.21.15.19:0/3827516846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafa80b2470 msgr2=0x7fafa80b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.981+0000 7fafad800640 1 --2- 172.21.15.19:0/3827516846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafa80b2470 0x7fafa80b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.981+0000 7fafad800640 1 -- 172.21.15.19:0/3827516846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa8159b30 con 0x7faf900039b0 2024-03-28T13:10:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.981+0000 7fafad800640 1 --2- 172.21.15.19:0/3827516846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf900039b0 0x7fafa814c890 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00c160 tx=0x7fafa80ad2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.981+0000 7fafa5600640 1 -- 172.21.15.19:0/3827516846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c045070 con 0x7faf900039b0 2024-03-28T13:10:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.981+0000 7fafa5600640 1 -- 172.21.15.19:0/3827516846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf9c004bf0 con 0x7faf900039b0 2024-03-28T13:10:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.981+0000 7fafa5600640 1 -- 172.21.15.19:0/3827516846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c007c60 con 0x7faf900039b0 2024-03-28T13:10:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.981+0000 7fafb0200640 1 -- 172.21.15.19:0/3827516846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa8159dd0 con 0x7faf900039b0 2024-03-28T13:10:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.982+0000 7fafb0200640 1 -- 172.21.15.19:0/3827516846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa815a390 con 0x7faf900039b0 2024-03-28T13:10:56.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.983+0000 7fafb0200640 1 -- 172.21.15.19:0/3827516846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa81557b0 con 0x7faf900039b0 2024-03-28T13:10:56.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.983+0000 7fafa5600640 1 -- 172.21.15.19:0/3827516846 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7faf9c007070 con 0x7faf900039b0 2024-03-28T13:10:56.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.984+0000 7fafa5600640 1 --2- 172.21.15.19:0/3827516846 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faf8806c530 0x7faf8806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:56.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.984+0000 7fafa5600640 1 -- 172.21.15.19:0/3827516846 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7faf9c0ac8e0 con 0x7faf900039b0 2024-03-28T13:10:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.985+0000 7faface00640 1 --2- 172.21.15.19:0/3827516846 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faf8806c530 0x7faf8806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.986+0000 7faface00640 1 --2- 172.21.15.19:0/3827516846 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faf8806c530 0x7faf8806e9f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fafa80b39c0 tx=0x7faf94008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:56.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:56.989+0000 7fafa5600640 1 -- 172.21.15.19:0/3827516846 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7faf9c07af70 con 0x7faf900039b0 2024-03-28T13:10:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.182+0000 7fafb0200640 1 -- 172.21.15.19:0/3827516846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fafa80685c0 con 0x7faf900039b0 2024-03-28T13:10:57.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.185+0000 7fafa5600640 1 -- 172.21.15.19:0/3827516846 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16622 (secure 0 0 0) 0x7faf9c07a870 con 0x7faf900039b0 2024-03-28T13:10:57.185 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:57.185 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:10:56.215784+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:10:54.307571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-28T13:10:55.205691+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:10:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.187+0000 7fafb0200640 1 -- 172.21.15.19:0/3827516846 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faf8806c530 msgr2=0x7faf8806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.187+0000 7fafb0200640 1 --2- 172.21.15.19:0/3827516846 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faf8806c530 0x7faf8806e9f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fafa80b39c0 tx=0x7faf94008040 comp rx=0 tx=0).stop 2024-03-28T13:10:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.187+0000 7fafb0200640 1 -- 172.21.15.19:0/3827516846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf900039b0 msgr2=0x7fafa814c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.187+0000 7fafb0200640 1 --2- 172.21.15.19:0/3827516846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf900039b0 0x7fafa814c890 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00c160 tx=0x7fafa80ad2f0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.188+0000 7fafb0200640 1 -- 172.21.15.19:0/3827516846 shutdown_connections 2024-03-28T13:10:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.188+0000 7fafb0200640 1 --2- 172.21.15.19:0/3827516846 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faf8806c530 0x7faf8806e9f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.188+0000 7fafb0200640 1 --2- 172.21.15.19:0/3827516846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafa8154df0 0x7fafa8155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.188+0000 7fafb0200640 1 --2- 172.21.15.19:0/3827516846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafa80b2470 0x7fafa80b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.188+0000 7fafb0200640 1 --2- 172.21.15.19:0/3827516846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf900039b0 0x7fafa814c890 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.188+0000 7fafb0200640 1 -- 172.21.15.19:0/3827516846 >> 172.21.15.19:0/3827516846 conn(0x7fafa806b990 msgr2=0x7fafa806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.188+0000 7fafb0200640 1 -- 172.21.15.19:0/3827516846 shutdown_connections 2024-03-28T13:10:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.188+0000 7fafb0200640 1 -- 172.21.15.19:0/3827516846 wait complete. 2024-03-28T13:10:57.208 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:10:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.426+0000 7f1a0aa00640 1 Processor -- start 2024-03-28T13:10:57.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.426+0000 7f1a0aa00640 1 -- start start 2024-03-28T13:10:57.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.427+0000 7f1a0aa00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1a040af830 0x7f1a04073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:57.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.427+0000 7f1a0aa00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a0415fe80 con 0x7f1a040d0650 2024-03-28T13:10:57.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.427+0000 7f1a0aa00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a040ad120 con 0x7f1a0415c390 2024-03-28T13:10:57.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.427+0000 7f1a0aa00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1a0415c770 con 0x7f1a040af830 2024-03-28T13:10:57.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.427+0000 7f1a08a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1a040d0650 0x7f1a04071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35658/0 (socket says 172.21.15.19:35658) 2024-03-28T13:10:57.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.427+0000 7f1a08a00640 1 -- 172.21.15.19:0/3212624142 learned_addr learned my addr 172.21.15.19:0/3212624142 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.428+0000 7f19fb400640 1 --2- 172.21.15.19:0/3212624142 >> v2:172.21.15.188:3300/0 conn(0x7f1a040af830 0x7f1a04073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.428+0000 7f1a03400640 1 -- 172.21.15.19:0/3212624142 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3916898359 0 0) 0x7f1a0415fe80 con 0x7f1a040d0650 2024-03-28T13:10:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.428+0000 7f1a03400640 1 -- 172.21.15.19:0/3212624142 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19e8003520 con 0x7f1a040d0650 2024-03-28T13:10:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.428+0000 7f1a03400640 1 -- 172.21.15.19:0/3212624142 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 349358606 0 0) 0x7f19e8003520 con 0x7f1a040d0650 2024-03-28T13:10:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.428+0000 7f1a03400640 1 -- 172.21.15.19:0/3212624142 >> v2:172.21.15.188:3300/0 conn(0x7f1a040af830 msgr2=0x7f1a04073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.428+0000 7f1a03400640 1 --2- 172.21.15.19:0/3212624142 >> v2:172.21.15.188:3300/0 conn(0x7f1a040af830 0x7f1a04073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.428+0000 7f1a03400640 1 -- 172.21.15.19:0/3212624142 >> v1:172.21.15.188:6789/0 conn(0x7f1a0415c390 legacy=0x7f1a04160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.428+0000 7f1a03400640 1 -- 172.21.15.19:0/3212624142 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a040ad400 con 0x7f1a040d0650 2024-03-28T13:10:57.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.428+0000 7f1a03400640 1 -- 172.21.15.19:0/3212624142 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f19fc001a10 con 0x7f1a040d0650 2024-03-28T13:10:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.428+0000 7f1a03400640 1 -- 172.21.15.19:0/3212624142 >> v1:172.21.15.19:6789/0 conn(0x7f1a040d0650 legacy=0x7f1a04071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f1a03400640 1 --2- 172.21.15.19:0/3212624142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e8003a00 0x7f19e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f1a03400640 1 --2- 172.21.15.19:0/3212624142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19e8004800 0x7f19e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f1a03400640 1 --2- 172.21.15.19:0/3212624142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19e80051c0 0x7f19e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f1a03400640 1 -- 172.21.15.19:0/3212624142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19e800c090 con 0x7f19e8003a00 2024-03-28T13:10:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f1a03400640 1 -- 172.21.15.19:0/3212624142 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f19e800c240 con 0x7f19e80051c0 2024-03-28T13:10:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f1a03400640 1 -- 172.21.15.19:0/3212624142 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f19e800c440 con 0x7f19e8004800 2024-03-28T13:10:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f1a03e00640 1 --2- 172.21.15.19:0/3212624142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e8003a00 0x7f19e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f19fb400640 1 --2- 172.21.15.19:0/3212624142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19e8004800 0x7f19e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f1a08a00640 1 --2- 172.21.15.19:0/3212624142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19e80051c0 0x7f19e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f1a03e00640 1 -- 172.21.15.19:0/3212624142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19e8004800 msgr2=0x7f19e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f1a03e00640 1 --2- 172.21.15.19:0/3212624142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19e8004800 0x7f19e8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f1a03e00640 1 -- 172.21.15.19:0/3212624142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19e80051c0 msgr2=0x7f19e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f1a03e00640 1 --2- 172.21.15.19:0/3212624142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19e80051c0 0x7f19e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f1a03e00640 1 -- 172.21.15.19:0/3212624142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e8003520 con 0x7f19e8003a00 2024-03-28T13:10:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f1a08a00640 1 --2- 172.21.15.19:0/3212624142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19e80051c0 0x7f19e800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:10:57.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.429+0000 7f1a08a00640 1 -- 172.21.15.19:0/3212624142 reap_dead start 2024-03-28T13:10:57.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.431+0000 7f1a03e00640 1 --2- 172.21.15.19:0/3212624142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e8003a00 0x7f19e8003e20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f19f4001750 tx=0x7f19e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f93c6511f4cfe9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.431+0000 7f1a03400640 1 -- 172.21.15.19:0/3212624142 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4001370 con 0x7f19e8003a00 2024-03-28T13:10:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.431+0000 7f1a03400640 1 -- 172.21.15.19:0/3212624142 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19f40052d0 con 0x7f19e8003a00 2024-03-28T13:10:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.431+0000 7f1a03400640 1 -- 172.21.15.19:0/3212624142 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4003830 con 0x7f19e8003a00 2024-03-28T13:10:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.431+0000 7f1a0aa00640 1 -- 172.21.15.19:0/3212624142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e8003a00 msgr2=0x7f19e8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.431+0000 7f1a0aa00640 1 --2- 172.21.15.19:0/3212624142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e8003a00 0x7f19e8003e20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f19f4001750 tx=0x7f19e8005830 comp rx=0 tx=0).stop 2024-03-28T13:10:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.431+0000 7f1a0aa00640 1 -- 172.21.15.19:0/3212624142 shutdown_connections 2024-03-28T13:10:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.431+0000 7f1a0aa00640 1 --2- 172.21.15.19:0/3212624142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19e8003a00 0x7f19e8003e20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.431+0000 7f1a0aa00640 1 -- 172.21.15.19:0/3212624142 >> 172.21.15.19:0/3212624142 conn(0x7f1a0406b990 msgr2=0x7f1a0406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.432+0000 7f1a0aa00640 1 -- 172.21.15.19:0/3212624142 shutdown_connections 2024-03-28T13:10:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.432+0000 7f1a0aa00640 1 -- 172.21.15.19:0/3212624142 wait complete. 2024-03-28T13:10:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.433+0000 7f1a0aa00640 1 Processor -- start 2024-03-28T13:10:57.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.433+0000 7f1a0aa00640 1 -- start start 2024-03-28T13:10:57.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.434+0000 7f1a0aa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19e8003a00 0x7f1a0415f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:57.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.434+0000 7f1a0aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a04151320 0x7f1a0414f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.434+0000 7f1a0aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a04154b50 0x7f1a04152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.434+0000 7f1a0aa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a040cf980 con 0x7f1a04154b50 2024-03-28T13:10:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.434+0000 7f1a0aa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1a04160120 con 0x7f19e8003a00 2024-03-28T13:10:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.434+0000 7f1a0aa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1a04073790 con 0x7f1a04151320 2024-03-28T13:10:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.435+0000 7f1a08a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a04154b50 0x7f1a04152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.435+0000 7f1a08a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a04154b50 0x7f1a04152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43408/0 (socket says 172.21.15.19:43408) 2024-03-28T13:10:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.435+0000 7f1a08a00640 1 -- 172.21.15.19:0/2679221605 learned_addr learned my addr 172.21.15.19:0/2679221605 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.435+0000 7f19fb400640 1 --2- 172.21.15.19:0/2679221605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a04151320 0x7f1a0414f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.435+0000 7f1a03e00640 1 --2- 172.21.15.19:0/2679221605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19e8003a00 0x7f1a0415f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.435+0000 7f1a08a00640 1 -- 172.21.15.19:0/2679221605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a04151320 msgr2=0x7f1a0414f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.435+0000 7f1a08a00640 1 --2- 172.21.15.19:0/2679221605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a04151320 0x7f1a0414f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.435+0000 7f1a08a00640 1 -- 172.21.15.19:0/2679221605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19e8003a00 msgr2=0x7f1a0415f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.435+0000 7f1a08a00640 1 --2- 172.21.15.19:0/2679221605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19e8003a00 0x7f1a0415f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.435+0000 7f1a08a00640 1 -- 172.21.15.19:0/2679221605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a04151ac0 con 0x7f1a04154b50 2024-03-28T13:10:57.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.436+0000 7f1a08a00640 1 --2- 172.21.15.19:0/2679221605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a04154b50 0x7f1a04152720 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1a040ae580 tx=0x7f19fc00df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.436+0000 7f1a00c00640 1 -- 172.21.15.19:0/2679221605 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc00b800 con 0x7f1a04154b50 2024-03-28T13:10:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.436+0000 7f1a00c00640 1 -- 172.21.15.19:0/2679221605 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19fc00b9a0 con 0x7f1a04154b50 2024-03-28T13:10:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.436+0000 7f1a00c00640 1 -- 172.21.15.19:0/2679221605 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19fc00bd90 con 0x7f1a04154b50 2024-03-28T13:10:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.436+0000 7f1a0aa00640 1 -- 172.21.15.19:0/2679221605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0415c9c0 con 0x7f1a04154b50 2024-03-28T13:10:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.436+0000 7f1a0aa00640 1 -- 172.21.15.19:0/2679221605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a0415cf80 con 0x7f1a04154b50 2024-03-28T13:10:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.437+0000 7f1a0aa00640 1 -- 172.21.15.19:0/2679221605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a040685c0 con 0x7f1a04154b50 2024-03-28T13:10:57.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.438+0000 7f1a00c00640 1 -- 172.21.15.19:0/2679221605 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f19fc01b020 con 0x7f1a04154b50 2024-03-28T13:10:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.439+0000 7f1a00c00640 1 --2- 172.21.15.19:0/2679221605 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f19e006c530 0x7f19e006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:57.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.439+0000 7f1a00c00640 1 -- 172.21.15.19:0/2679221605 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f19fc08b7a0 con 0x7f1a04154b50 2024-03-28T13:10:57.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.440+0000 7f1a03e00640 1 --2- 172.21.15.19:0/2679221605 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f19e006c530 0x7f19e006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:57.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.441+0000 7f1a03e00640 1 --2- 172.21.15.19:0/2679221605 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f19e006c530 0x7f19e006e9f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f19f4033000 tx=0x7f19f4009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:57.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.443+0000 7f1a00c00640 1 -- 172.21.15.19:0/2679221605 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f19fc059760 con 0x7f1a04154b50 2024-03-28T13:10:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.649+0000 7f1a0aa00640 1 -- 172.21.15.19:0/2679221605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a0406f1b0 con 0x7f1a04154b50 2024-03-28T13:10:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.652+0000 7f1a00c00640 1 -- 172.21.15.19:0/2679221605 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16622 (secure 0 0 0) 0x7f19fc05da60 con 0x7f1a04154b50 2024-03-28T13:10:57.652 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:10:57.652 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:10:56.215784+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:10:54.307571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-28T13:10:55.205691+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:10:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.654+0000 7f1a0aa00640 1 -- 172.21.15.19:0/2679221605 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f19e006c530 msgr2=0x7f19e006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.654+0000 7f1a0aa00640 1 --2- 172.21.15.19:0/2679221605 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f19e006c530 0x7f19e006e9f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f19f4033000 tx=0x7f19f4009030 comp rx=0 tx=0).stop 2024-03-28T13:10:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.654+0000 7f1a0aa00640 1 -- 172.21.15.19:0/2679221605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a04154b50 msgr2=0x7f1a04152720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.654+0000 7f1a0aa00640 1 --2- 172.21.15.19:0/2679221605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a04154b50 0x7f1a04152720 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1a040ae580 tx=0x7f19fc00df70 comp rx=0 tx=0).stop 2024-03-28T13:10:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.655+0000 7f1a0aa00640 1 -- 172.21.15.19:0/2679221605 shutdown_connections 2024-03-28T13:10:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.655+0000 7f1a0aa00640 1 --2- 172.21.15.19:0/2679221605 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f19e006c530 0x7f19e006e9f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.655+0000 7f1a0aa00640 1 --2- 172.21.15.19:0/2679221605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a04154b50 0x7f1a04152720 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.655+0000 7f1a0aa00640 1 --2- 172.21.15.19:0/2679221605 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a04151320 0x7f1a0414f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.655+0000 7f1a0aa00640 1 --2- 172.21.15.19:0/2679221605 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19e8003a00 0x7f1a0415f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.655+0000 7f1a0aa00640 1 -- 172.21.15.19:0/2679221605 >> 172.21.15.19:0/2679221605 conn(0x7f1a0406b990 msgr2=0x7f1a040afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.655+0000 7f1a0aa00640 1 -- 172.21.15.19:0/2679221605 shutdown_connections 2024-03-28T13:10:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.655+0000 7f1a0aa00640 1 -- 172.21.15.19:0/2679221605 wait complete. 2024-03-28T13:10:57.676 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-28T13:10:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.897+0000 7fa886400640 1 Processor -- start 2024-03-28T13:10:57.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.897+0000 7fa886400640 1 -- start start 2024-03-28T13:10:57.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.899+0000 7fa886400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa8800cc880 0x7fa8800cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:57.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.899+0000 7fa886400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa88006d620 0x7fa88006da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:57.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.899+0000 7fa886400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8800ce890 con 0x7fa88013f190 2024-03-28T13:10:57.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.899+0000 7fa886400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa8800cd230 con 0x7fa88006d620 2024-03-28T13:10:57.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.899+0000 7fa886400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa8800ceb50 con 0x7fa8800cc880 2024-03-28T13:10:57.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.899+0000 7fa885000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa88013f190 0x7fa8800cf630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35668/0 (socket says 172.21.15.19:35668) 2024-03-28T13:10:57.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.899+0000 7fa885000640 1 -- 172.21.15.19:0/467644470 learned_addr learned my addr 172.21.15.19:0/467644470 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:57.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.899+0000 7fa885a00640 1 --2- 172.21.15.19:0/467644470 >> v2:172.21.15.19:3300/0 conn(0x7fa88006d620 0x7fa88006da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:57.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.899+0000 7fa87fe00640 1 --2- 172.21.15.19:0/467644470 >> v2:172.21.15.188:3300/0 conn(0x7fa8800cc880 0x7fa8800cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:57.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.900+0000 7fa87f400640 1 -- 172.21.15.19:0/467644470 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2173679364 0 0) 0x7fa8800ce890 con 0x7fa88013f190 2024-03-28T13:10:57.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.900+0000 7fa87f400640 1 -- 172.21.15.19:0/467644470 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa868003520 con 0x7fa88013f190 2024-03-28T13:10:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.900+0000 7fa885a00640 1 -- 172.21.15.19:0/467644470 >> v2:172.21.15.188:3300/0 conn(0x7fa8800cc880 msgr2=0x7fa8800cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.900+0000 7fa885a00640 1 --2- 172.21.15.19:0/467644470 >> v2:172.21.15.188:3300/0 conn(0x7fa8800cc880 0x7fa8800cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.900+0000 7fa885a00640 1 -- 172.21.15.19:0/467644470 >> v1:172.21.15.19:6789/0 conn(0x7fa88013f190 legacy=0x7fa8800cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.900+0000 7fa885a00640 1 -- 172.21.15.19:0/467644470 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8800cee30 con 0x7fa88006d620 2024-03-28T13:10:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.901+0000 7fa885a00640 1 --2- 172.21.15.19:0/467644470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa88006d620 0x7fa88006da50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa8800cc7b0 tx=0x7fa87802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88bdfe4ec04b847 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.901+0000 7fa87f400640 1 -- 172.21.15.19:0/467644470 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878039070 con 0x7fa88006d620 2024-03-28T13:10:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.901+0000 7fa87f400640 1 -- 172.21.15.19:0/467644470 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa87802cc70 con 0x7fa88006d620 2024-03-28T13:10:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.901+0000 7fa87f400640 1 -- 172.21.15.19:0/467644470 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878034660 con 0x7fa88006d620 2024-03-28T13:10:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.902+0000 7fa886400640 1 -- 172.21.15.19:0/467644470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa88006d620 msgr2=0x7fa88006da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.902+0000 7fa886400640 1 --2- 172.21.15.19:0/467644470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa88006d620 0x7fa88006da50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa8800cc7b0 tx=0x7fa87802bec0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.902+0000 7fa886400640 1 -- 172.21.15.19:0/467644470 shutdown_connections 2024-03-28T13:10:57.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.902+0000 7fa886400640 1 --2- 172.21.15.19:0/467644470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa88006d620 0x7fa88006da50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.902+0000 7fa886400640 1 --2- 172.21.15.19:0/467644470 >> v2:172.21.15.188:3300/0 conn(0x7fa8800cc880 0x7fa8800cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.902+0000 7fa886400640 1 -- 172.21.15.19:0/467644470 >> 172.21.15.19:0/467644470 conn(0x7fa8800b1ba0 msgr2=0x7fa8800b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:57.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.902+0000 7fa886400640 1 -- 172.21.15.19:0/467644470 shutdown_connections 2024-03-28T13:10:57.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.902+0000 7fa886400640 1 -- 172.21.15.19:0/467644470 wait complete. 2024-03-28T13:10:57.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.903+0000 7fa886400640 1 Processor -- start 2024-03-28T13:10:57.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.904+0000 7fa886400640 1 -- start start 2024-03-28T13:10:57.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.905+0000 7fa886400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa88006d620 0x7fa8801558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:57.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.905+0000 7fa886400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8800cc880 0x7fa8801e77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:57.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.905+0000 7fa886400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88013f190 0x7fa88015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:57.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.905+0000 7fa886400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa88013f570 con 0x7fa8800cc880 2024-03-28T13:10:57.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.905+0000 7fa886400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa880160c70 con 0x7fa88006d620 2024-03-28T13:10:57.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.905+0000 7fa886400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa88006df90 con 0x7fa88013f190 2024-03-28T13:10:57.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.905+0000 7fa87fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8800cc880 0x7fa8801e77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:57.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.905+0000 7fa87fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8800cc880 0x7fa8801e77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43428/0 (socket says 172.21.15.19:43428) 2024-03-28T13:10:57.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.905+0000 7fa87fe00640 1 -- 172.21.15.19:0/296005491 learned_addr learned my addr 172.21.15.19:0/296005491 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:57.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.905+0000 7fa885000640 1 --2- 172.21.15.19:0/296005491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa88006d620 0x7fa8801558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:57.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.905+0000 7fa87fe00640 1 -- 172.21.15.19:0/296005491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88013f190 msgr2=0x7fa88015a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:57.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.905+0000 7fa87fe00640 1 --2- 172.21.15.19:0/296005491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88013f190 0x7fa88015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.905+0000 7fa87fe00640 1 -- 172.21.15.19:0/296005491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa88006d620 msgr2=0x7fa8801558d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:57.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.905+0000 7fa87fe00640 1 --2- 172.21.15.19:0/296005491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa88006d620 0x7fa8801558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:57.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.905+0000 7fa87fe00640 1 -- 172.21.15.19:0/296005491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8801e8280 con 0x7fa8800cc880 2024-03-28T13:10:57.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.906+0000 7fa87fe00640 1 --2- 172.21.15.19:0/296005491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8800cc880 0x7fa8801e77c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00d5f0 tx=0x7fa86c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:57.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.906+0000 7fa87cc00640 1 -- 172.21.15.19:0/296005491 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0125f0 con 0x7fa8800cc880 2024-03-28T13:10:57.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.906+0000 7fa87cc00640 1 -- 172.21.15.19:0/296005491 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa86c019070 con 0x7fa8800cc880 2024-03-28T13:10:57.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.906+0000 7fa87cc00640 1 -- 172.21.15.19:0/296005491 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa86c0129d0 con 0x7fa8800cc880 2024-03-28T13:10:57.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.906+0000 7fa886400640 1 -- 172.21.15.19:0/296005491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8801e8560 con 0x7fa8800cc880 2024-03-28T13:10:57.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.906+0000 7fa886400640 1 -- 172.21.15.19:0/296005491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88013ec10 con 0x7fa8800cc880 2024-03-28T13:10:57.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.907+0000 7fa886400640 1 -- 172.21.15.19:0/296005491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa880075940 con 0x7fa8800cc880 2024-03-28T13:10:57.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.908+0000 7fa87cc00640 1 -- 172.21.15.19:0/296005491 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa86c012b70 con 0x7fa8800cc880 2024-03-28T13:10:57.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.909+0000 7fa87cc00640 1 --2- 172.21.15.19:0/296005491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa86006c530 0x7fa86006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:57.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.909+0000 7fa87cc00640 1 -- 172.21.15.19:0/296005491 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fa86c08baf0 con 0x7fa8800cc880 2024-03-28T13:10:57.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.910+0000 7fa885000640 1 --2- 172.21.15.19:0/296005491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa86006c530 0x7fa86006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:57.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.911+0000 7fa885000640 1 --2- 172.21.15.19:0/296005491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa86006c530 0x7fa86006e9f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa874000f60 tx=0x7fa874009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:57.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:57.914+0000 7fa87cc00640 1 -- 172.21.15.19:0/296005491 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa86c05a100 con 0x7fa8800cc880 2024-03-28T13:10:58.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:58.153+0000 7fa886400640 1 -- 172.21.15.19:0/296005491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa8800b2870 con 0x7fa8800cc880 2024-03-28T13:10:58.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:58.771+0000 7fa87cc00640 1 -- 172.21.15.19:0/296005491 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fa86c059a00 con 0x7fa8800cc880 2024-03-28T13:10:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:58.774+0000 7fa886400640 1 -- 172.21.15.19:0/296005491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa86006c530 msgr2=0x7fa86006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:58.774+0000 7fa886400640 1 --2- 172.21.15.19:0/296005491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa86006c530 0x7fa86006e9f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa874000f60 tx=0x7fa874009040 comp rx=0 tx=0).stop 2024-03-28T13:10:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:58.774+0000 7fa886400640 1 -- 172.21.15.19:0/296005491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8800cc880 msgr2=0x7fa8801e77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:58.774+0000 7fa886400640 1 --2- 172.21.15.19:0/296005491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8800cc880 0x7fa8801e77c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa86c00d5f0 tx=0x7fa86c00dab0 comp rx=0 tx=0).stop 2024-03-28T13:10:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:58.774+0000 7fa886400640 1 -- 172.21.15.19:0/296005491 shutdown_connections 2024-03-28T13:10:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:58.774+0000 7fa886400640 1 --2- 172.21.15.19:0/296005491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa86006c530 0x7fa86006e9f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:58.774+0000 7fa886400640 1 --2- 172.21.15.19:0/296005491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa88013f190 0x7fa88015a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:58.774+0000 7fa886400640 1 --2- 172.21.15.19:0/296005491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8800cc880 0x7fa8801e77c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:58.774+0000 7fa886400640 1 --2- 172.21.15.19:0/296005491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa88006d620 0x7fa8801558d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:58.774+0000 7fa886400640 1 -- 172.21.15.19:0/296005491 >> 172.21.15.19:0/296005491 conn(0x7fa8800b1ba0 msgr2=0x7fa88015f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:58.775+0000 7fa886400640 1 -- 172.21.15.19:0/296005491 shutdown_connections 2024-03-28T13:10:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:58.775+0000 7fa886400640 1 -- 172.21.15.19:0/296005491 wait complete. 2024-03-28T13:10:58.797 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-28T13:10:58.797 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-28T13:10:59.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.011+0000 7f70b2e00640 1 Processor -- start 2024-03-28T13:10:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.011+0000 7f70b2e00640 1 -- start start 2024-03-28T13:10:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.013+0000 7f70b2e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f70ac0cc880 0x7f70ac0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.013+0000 7f70b2e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f70ac06d620 0x7f70ac06da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.013+0000 7f70b2e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70ac0ce890 con 0x7f70ac13f190 2024-03-28T13:10:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.013+0000 7f70b2e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f70ac0cd230 con 0x7f70ac06d620 2024-03-28T13:10:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.013+0000 7f70b2e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f70ac0ceb50 con 0x7f70ac0cc880 2024-03-28T13:10:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.013+0000 7f70b1a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f70ac13f190 0x7f70ac0cf630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35680/0 (socket says 172.21.15.19:35680) 2024-03-28T13:10:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.013+0000 7f70b1a00640 1 -- 172.21.15.19:0/3259875196 learned_addr learned my addr 172.21.15.19:0/3259875196 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:10:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.013+0000 7f70b2400640 1 --2- 172.21.15.19:0/3259875196 >> v2:172.21.15.19:3300/0 conn(0x7f70ac06d620 0x7f70ac06da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.014+0000 7f70b1000640 1 --2- 172.21.15.19:0/3259875196 >> v2:172.21.15.188:3300/0 conn(0x7f70ac0cc880 0x7f70ac0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.014+0000 7f70abe00640 1 -- 172.21.15.19:0/3259875196 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1023300500 0 0) 0x7f70ac0ce890 con 0x7f70ac13f190 2024-03-28T13:10:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.014+0000 7f70abe00640 1 -- 172.21.15.19:0/3259875196 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7094003520 con 0x7f70ac13f190 2024-03-28T13:10:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.014+0000 7f70b2400640 1 -- 172.21.15.19:0/3259875196 >> v2:172.21.15.188:3300/0 conn(0x7f70ac0cc880 msgr2=0x7f70ac0cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:10:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.014+0000 7f70b2400640 1 --2- 172.21.15.19:0/3259875196 >> v2:172.21.15.188:3300/0 conn(0x7f70ac0cc880 0x7f70ac0cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.014+0000 7f70b2400640 1 -- 172.21.15.19:0/3259875196 >> v1:172.21.15.19:6789/0 conn(0x7f70ac13f190 legacy=0x7f70ac0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.014+0000 7f70b2400640 1 -- 172.21.15.19:0/3259875196 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac0cee30 con 0x7f70ac06d620 2024-03-28T13:10:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.015+0000 7f70b2400640 1 --2- 172.21.15.19:0/3259875196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70ac06d620 0x7f70ac06da50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0cc7b0 tx=0x7f70a402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235b22f35b3b1ace server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.015+0000 7f70abe00640 1 -- 172.21.15.19:0/3259875196 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4039070 con 0x7f70ac06d620 2024-03-28T13:10:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.016+0000 7f70abe00640 1 -- 172.21.15.19:0/3259875196 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70a402cc70 con 0x7f70ac06d620 2024-03-28T13:10:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.016+0000 7f70abe00640 1 -- 172.21.15.19:0/3259875196 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4034640 con 0x7f70ac06d620 2024-03-28T13:10:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.016+0000 7f70b2e00640 1 -- 172.21.15.19:0/3259875196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70ac06d620 msgr2=0x7f70ac06da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.016+0000 7f70b2e00640 1 --2- 172.21.15.19:0/3259875196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70ac06d620 0x7f70ac06da50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f70ac0cc7b0 tx=0x7f70a402bec0 comp rx=0 tx=0).stop 2024-03-28T13:10:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.016+0000 7f70b2e00640 1 -- 172.21.15.19:0/3259875196 shutdown_connections 2024-03-28T13:10:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.016+0000 7f70b2e00640 1 --2- 172.21.15.19:0/3259875196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70ac06d620 0x7f70ac06da50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.016+0000 7f70b2e00640 1 --2- 172.21.15.19:0/3259875196 >> v2:172.21.15.188:3300/0 conn(0x7f70ac0cc880 0x7f70ac0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.017+0000 7f70b2e00640 1 -- 172.21.15.19:0/3259875196 >> 172.21.15.19:0/3259875196 conn(0x7f70ac0b1ba0 msgr2=0x7f70ac0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.017+0000 7f70b2e00640 1 -- 172.21.15.19:0/3259875196 shutdown_connections 2024-03-28T13:10:59.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.017+0000 7f70b2e00640 1 -- 172.21.15.19:0/3259875196 wait complete. 2024-03-28T13:10:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.018+0000 7f70b2e00640 1 Processor -- start 2024-03-28T13:10:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.018+0000 7f70b2e00640 1 -- start start 2024-03-28T13:10:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.019+0000 7f70b2e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70ac06d620 0x7f70ac13a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.019+0000 7f70b2e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70ac0cc880 0x7f70ac1572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.019+0000 7f70b2e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70ac13f190 0x7f70ac157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.019+0000 7f70b2e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac13f570 con 0x7f70ac13f190 2024-03-28T13:10:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.019+0000 7f70b2e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f70ac160c70 con 0x7f70ac0cc880 2024-03-28T13:10:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.019+0000 7f70b2e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f70ac06df90 con 0x7f70ac06d620 2024-03-28T13:10:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.020+0000 7f70b2400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70ac13f190 0x7f70ac157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.020+0000 7f70b2400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70ac13f190 0x7f70ac157830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43436/0 (socket says 172.21.15.19:43436) 2024-03-28T13:10:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.020+0000 7f70b2400640 1 -- 172.21.15.19:0/2621379698 learned_addr learned my addr 172.21.15.19:0/2621379698 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:10:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.020+0000 7f70b1a00640 1 --2- 172.21.15.19:0/2621379698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70ac06d620 0x7f70ac13a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.020+0000 7f70b1000640 1 --2- 172.21.15.19:0/2621379698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70ac0cc880 0x7f70ac1572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.020+0000 7f70b2400640 1 -- 172.21.15.19:0/2621379698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70ac06d620 msgr2=0x7f70ac13a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.020+0000 7f70b2400640 1 --2- 172.21.15.19:0/2621379698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70ac06d620 0x7f70ac13a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.020+0000 7f70b2400640 1 -- 172.21.15.19:0/2621379698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70ac0cc880 msgr2=0x7f70ac1572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.020+0000 7f70b2400640 1 --2- 172.21.15.19:0/2621379698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70ac0cc880 0x7f70ac1572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.020+0000 7f70b2400640 1 -- 172.21.15.19:0/2621379698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70ac072620 con 0x7f70ac13f190 2024-03-28T13:10:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.020+0000 7f70b2400640 1 --2- 172.21.15.19:0/2621379698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70ac13f190 0x7f70ac157830 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f70a4037040 tx=0x7f70a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.021+0000 7f70a9600640 1 -- 172.21.15.19:0/2621379698 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4042020 con 0x7f70ac13f190 2024-03-28T13:10:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.021+0000 7f70a9600640 1 -- 172.21.15.19:0/2621379698 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70a403a040 con 0x7f70ac13f190 2024-03-28T13:10:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.021+0000 7f70b2e00640 1 -- 172.21.15.19:0/2621379698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70ac072900 con 0x7f70ac13f190 2024-03-28T13:10:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.021+0000 7f70a9600640 1 -- 172.21.15.19:0/2621379698 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a4039070 con 0x7f70ac13f190 2024-03-28T13:10:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.021+0000 7f70b2e00640 1 -- 172.21.15.19:0/2621379698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70ac072ec0 con 0x7f70ac13f190 2024-03-28T13:10:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.022+0000 7f70b2e00640 1 -- 172.21.15.19:0/2621379698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70ac075940 con 0x7f70ac13f190 2024-03-28T13:10:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.023+0000 7f70a9600640 1 -- 172.21.15.19:0/2621379698 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f70a4045020 con 0x7f70ac13f190 2024-03-28T13:10:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.024+0000 7f70a9600640 1 --2- 172.21.15.19:0/2621379698 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f708c06c530 0x7f708c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:10:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.024+0000 7f70a9600640 1 -- 172.21.15.19:0/2621379698 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f70a40ad670 con 0x7f70ac13f190 2024-03-28T13:10:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.025+0000 7f70b1a00640 1 --2- 172.21.15.19:0/2621379698 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f708c06c530 0x7f708c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:10:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.026+0000 7f70b1a00640 1 --2- 172.21.15.19:0/2621379698 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f708c06c530 0x7f708c06e9f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f70a00011e0 tx=0x7f70a000d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:10:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.028+0000 7f70a9600640 1 -- 172.21.15.19:0/2621379698 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f70a404c070 con 0x7f70ac13f190 2024-03-28T13:10:59.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.265+0000 7f70b2e00640 1 -- 172.21.15.19:0/2621379698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f70ac13c0c0 con 0x7f70ac13f190 2024-03-28T13:10:59.603 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-03-28T13:10:59.602+0000 7f7460000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:10:59.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.782+0000 7f70a9600640 1 -- 172.21.15.19:0/2621379698 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f70a402ce20 con 0x7f70ac13f190 2024-03-28T13:10:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.787+0000 7f70b2e00640 1 -- 172.21.15.19:0/2621379698 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f708c06c530 msgr2=0x7f708c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:59.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.787+0000 7f70b2e00640 1 --2- 172.21.15.19:0/2621379698 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f708c06c530 0x7f708c06e9f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f70a00011e0 tx=0x7f70a000d040 comp rx=0 tx=0).stop 2024-03-28T13:10:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.787+0000 7f70b2e00640 1 -- 172.21.15.19:0/2621379698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70ac13f190 msgr2=0x7f70ac157830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:10:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.787+0000 7f70b2e00640 1 --2- 172.21.15.19:0/2621379698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70ac13f190 0x7f70ac157830 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f70a4037040 tx=0x7f70a4004a40 comp rx=0 tx=0).stop 2024-03-28T13:10:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.787+0000 7f70b2e00640 1 -- 172.21.15.19:0/2621379698 shutdown_connections 2024-03-28T13:10:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.787+0000 7f70b2e00640 1 --2- 172.21.15.19:0/2621379698 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f708c06c530 0x7f708c06e9f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.787+0000 7f70b2e00640 1 --2- 172.21.15.19:0/2621379698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70ac13f190 0x7f70ac157830 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.787+0000 7f70b2e00640 1 --2- 172.21.15.19:0/2621379698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70ac0cc880 0x7f70ac1572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.787+0000 7f70b2e00640 1 --2- 172.21.15.19:0/2621379698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70ac06d620 0x7f70ac13a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:10:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.787+0000 7f70b2e00640 1 -- 172.21.15.19:0/2621379698 >> 172.21.15.19:0/2621379698 conn(0x7f70ac0b1ba0 msgr2=0x7f70ac0af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:10:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.787+0000 7f70b2e00640 1 -- 172.21.15.19:0/2621379698 shutdown_connections 2024-03-28T13:10:59.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:10:59.788+0000 7f70b2e00640 1 -- 172.21.15.19:0/2621379698 wait complete. 2024-03-28T13:10:59.811 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-28T13:10:59.812 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:11:00.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.034+0000 7fdf7be00640 1 Processor -- start 2024-03-28T13:11:00.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.034+0000 7fdf7be00640 1 -- start start 2024-03-28T13:11:00.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.036+0000 7fdf7be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf7c15fc50 0x7fdf7c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:00.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.036+0000 7fdf7be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdf7c0cc490 0x7fdf7c0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:00.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.036+0000 7fdf7be00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf7c0b3150 con 0x7fdf7c0cc0b0 2024-03-28T13:11:00.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.036+0000 7fdf7be00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdf7c15c610 con 0x7fdf7c0cc490 2024-03-28T13:11:00.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.036+0000 7fdf7be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7c1600e0 con 0x7fdf7c15fc50 2024-03-28T13:11:00.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.036+0000 7fdf7b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdf7c0cc490 0x7fdf7c0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:00.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.036+0000 7fdf7b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdf7c0cc490 0x7fdf7c0d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43450/0 (socket says 172.21.15.19:43450) 2024-03-28T13:11:00.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.036+0000 7fdf7b400640 1 -- 172.21.15.19:0/1878359128 learned_addr learned my addr 172.21.15.19:0/1878359128 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:00.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.036+0000 7fdf73e00640 1 --2- 172.21.15.19:0/1878359128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf7c15fc50 0x7fdf7c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:00.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.037+0000 7fdf7b400640 1 -- 172.21.15.19:0/1878359128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf7c15fc50 msgr2=0x7fdf7c0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:00.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.037+0000 7fdf7b400640 1 --2- 172.21.15.19:0/1878359128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf7c15fc50 0x7fdf7c0cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.037+0000 7fdf7b400640 1 -- 172.21.15.19:0/1878359128 >> v1:172.21.15.188:6789/0 conn(0x7fdf7c0cc0b0 legacy=0x7fdf7c06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:00.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.037+0000 7fdf7b400640 1 -- 172.21.15.19:0/1878359128 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c1603c0 con 0x7fdf7c0cc490 2024-03-28T13:11:00.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.038+0000 7fdf7b400640 1 --2- 172.21.15.19:0/1878359128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf7c0cc490 0x7fdf7c0d0300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009e50 tx=0x7fdf7402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf6e3d6abf7af25 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:00.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.038+0000 7fdf7a000640 1 -- 172.21.15.19:0/1878359128 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7402ca70 con 0x7fdf7c0cc490 2024-03-28T13:11:00.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.038+0000 7fdf7a000640 1 -- 172.21.15.19:0/1878359128 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7402cc10 con 0x7fdf7c0cc490 2024-03-28T13:11:00.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.038+0000 7fdf7a000640 1 -- 172.21.15.19:0/1878359128 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf740343f0 con 0x7fdf7c0cc490 2024-03-28T13:11:00.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.039+0000 7fdf7be00640 1 -- 172.21.15.19:0/1878359128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf7c0cc490 msgr2=0x7fdf7c0d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:00.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.039+0000 7fdf7be00640 1 --2- 172.21.15.19:0/1878359128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf7c0cc490 0x7fdf7c0d0300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009e50 tx=0x7fdf7402bed0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.039+0000 7fdf7be00640 1 -- 172.21.15.19:0/1878359128 shutdown_connections 2024-03-28T13:11:00.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.039+0000 7fdf7be00640 1 --2- 172.21.15.19:0/1878359128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf7c0cc490 0x7fdf7c0d0300 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.039+0000 7fdf7be00640 1 --2- 172.21.15.19:0/1878359128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf7c15fc50 0x7fdf7c0cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.039+0000 7fdf7be00640 1 -- 172.21.15.19:0/1878359128 >> 172.21.15.19:0/1878359128 conn(0x7fdf7c0ccad0 msgr2=0x7fdf7c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:00.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.039+0000 7fdf7be00640 1 -- 172.21.15.19:0/1878359128 shutdown_connections 2024-03-28T13:11:00.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.039+0000 7fdf7be00640 1 -- 172.21.15.19:0/1878359128 wait complete. 2024-03-28T13:11:00.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.041+0000 7fdf7be00640 1 Processor -- start 2024-03-28T13:11:00.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.041+0000 7fdf7be00640 1 -- start start 2024-03-28T13:11:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.042+0000 7fdf7be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf7c0cc0b0 0x7fdf7c13e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.042+0000 7fdf7be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf7c0cc490 0x7fdf7c1f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.042+0000 7fdf7be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf7c0b1560 0x7fdf7c1e69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.042+0000 7fdf7be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c15ca70 con 0x7fdf7c0cc0b0 2024-03-28T13:11:00.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.042+0000 7fdf7be00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7c160980 con 0x7fdf7c0cc490 2024-03-28T13:11:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.042+0000 7fdf7be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7c15cbf0 con 0x7fdf7c0b1560 2024-03-28T13:11:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.042+0000 7fdf7aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf7c0cc0b0 0x7fdf7c13e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.042+0000 7fdf7aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf7c0cc0b0 0x7fdf7c13e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43460/0 (socket says 172.21.15.19:43460) 2024-03-28T13:11:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.043+0000 7fdf7aa00640 1 -- 172.21.15.19:0/1286795865 learned_addr learned my addr 172.21.15.19:0/1286795865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.043+0000 7fdf7aa00640 1 -- 172.21.15.19:0/1286795865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf7c0b1560 msgr2=0x7fdf7c1e69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.043+0000 7fdf7b400640 1 --2- 172.21.15.19:0/1286795865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf7c0b1560 0x7fdf7c1e69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.043+0000 7fdf7aa00640 1 --2- 172.21.15.19:0/1286795865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf7c0b1560 0x7fdf7c1e69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.043+0000 7fdf7aa00640 1 -- 172.21.15.19:0/1286795865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf7c0cc490 msgr2=0x7fdf7c1f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.043+0000 7fdf73e00640 1 --2- 172.21.15.19:0/1286795865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf7c0cc490 0x7fdf7c1f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.043+0000 7fdf7aa00640 1 --2- 172.21.15.19:0/1286795865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf7c0cc490 0x7fdf7c1f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.043+0000 7fdf7aa00640 1 -- 172.21.15.19:0/1286795865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf7c0b1d70 con 0x7fdf7c0cc0b0 2024-03-28T13:11:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.043+0000 7fdf73e00640 1 --2- 172.21.15.19:0/1286795865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf7c0cc490 0x7fdf7c1f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:00.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.043+0000 7fdf7aa00640 1 --2- 172.21.15.19:0/1286795865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf7c0cc0b0 0x7fdf7c13e1c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00c460 tx=0x7fdf6c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:00.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.044+0000 7fdf72a00640 1 -- 172.21.15.19:0/1286795865 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c016020 con 0x7fdf7c0cc0b0 2024-03-28T13:11:00.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.044+0000 7fdf72a00640 1 -- 172.21.15.19:0/1286795865 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf6c012870 con 0x7fdf7c0cc0b0 2024-03-28T13:11:00.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.044+0000 7fdf72a00640 1 -- 172.21.15.19:0/1286795865 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c012c60 con 0x7fdf7c0cc0b0 2024-03-28T13:11:00.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.044+0000 7fdf7be00640 1 -- 172.21.15.19:0/1286795865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf7c0b2050 con 0x7fdf7c0cc0b0 2024-03-28T13:11:00.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.044+0000 7fdf7be00640 1 -- 172.21.15.19:0/1286795865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf7c0b2610 con 0x7fdf7c0cc0b0 2024-03-28T13:11:00.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.045+0000 7fdf7be00640 1 -- 172.21.15.19:0/1286795865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf7c075940 con 0x7fdf7c0cc0b0 2024-03-28T13:11:00.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.046+0000 7fdf72a00640 1 -- 172.21.15.19:0/1286795865 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdf6c019020 con 0x7fdf7c0cc0b0 2024-03-28T13:11:00.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.047+0000 7fdf72a00640 1 --2- 172.21.15.19:0/1286795865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf5806c530 0x7fdf5806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:00.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.047+0000 7fdf72a00640 1 -- 172.21.15.19:0/1286795865 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fdf6c08b4d0 con 0x7fdf7c0cc0b0 2024-03-28T13:11:00.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.048+0000 7fdf73e00640 1 --2- 172.21.15.19:0/1286795865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf5806c530 0x7fdf5806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:00.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.049+0000 7fdf73e00640 1 --2- 172.21.15.19:0/1286795865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf5806c530 0x7fdf5806e9f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006fd0 tx=0x7fdf64008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:00.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.052+0000 7fdf72a00640 1 -- 172.21.15.19:0/1286795865 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdf6c020030 con 0x7fdf7c0cc0b0 2024-03-28T13:11:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.297+0000 7fdf7be00640 1 -- 172.21.15.19:0/1286795865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf7c06f2a0 con 0x7fdf7c0cc0b0 2024-03-28T13:11:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.298+0000 7fdf72a00640 1 -- 172.21.15.19:0/1286795865 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7fdf6c059a50 con 0x7fdf7c0cc0b0 2024-03-28T13:11:00.298 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:00.298 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2683971950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2683971950},{"type":"v1","addr":"172.21.15.188:6836","nonce":2683971950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/709274582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":709274582},{"type":"v1","addr":"172.21.15.19:6835","nonce":709274582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5040,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3420318181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3420318181},{"type":"v1","addr":"172.21.15.19:6833","nonce":3420318181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:10:56.216937+0000","modified":"2024-03-28T13:10:59.267258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5061":{"gid":5061,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6837/46284318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":46284318},{"type":"v1","addr":"172.21.15.188:6837","nonce":46284318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5061,"qdb_cluster":[5061]},"id":2}]} 2024-03-28T13:11:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-03-28T13:11:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.300+0000 7fdf7be00640 1 -- 172.21.15.19:0/1286795865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf5806c530 msgr2=0x7fdf5806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.300+0000 7fdf7be00640 1 --2- 172.21.15.19:0/1286795865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf5806c530 0x7fdf5806e9f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006fd0 tx=0x7fdf64008040 comp rx=0 tx=0).stop 2024-03-28T13:11:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.300+0000 7fdf7be00640 1 -- 172.21.15.19:0/1286795865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf7c0cc0b0 msgr2=0x7fdf7c13e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.300+0000 7fdf7be00640 1 --2- 172.21.15.19:0/1286795865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf7c0cc0b0 0x7fdf7c13e1c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c00c460 tx=0x7fdf6c00c920 comp rx=0 tx=0).stop 2024-03-28T13:11:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.300+0000 7fdf7be00640 1 -- 172.21.15.19:0/1286795865 shutdown_connections 2024-03-28T13:11:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.300+0000 7fdf7be00640 1 --2- 172.21.15.19:0/1286795865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf5806c530 0x7fdf5806e9f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.300+0000 7fdf7be00640 1 --2- 172.21.15.19:0/1286795865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf7c0b1560 0x7fdf7c1e69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.300+0000 7fdf7be00640 1 --2- 172.21.15.19:0/1286795865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf7c0cc490 0x7fdf7c1f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.300+0000 7fdf7be00640 1 --2- 172.21.15.19:0/1286795865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf7c0cc0b0 0x7fdf7c13e1c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.300+0000 7fdf7be00640 1 -- 172.21.15.19:0/1286795865 >> 172.21.15.19:0/1286795865 conn(0x7fdf7c0ccad0 msgr2=0x7fdf7c0b5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.301+0000 7fdf7be00640 1 -- 172.21.15.19:0/1286795865 shutdown_connections 2024-03-28T13:11:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.301+0000 7fdf7be00640 1 -- 172.21.15.19:0/1286795865 wait complete. 2024-03-28T13:11:00.321 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:11:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.541+0000 7f398b000640 1 Processor -- start 2024-03-28T13:11:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.542+0000 7f398b000640 1 -- start start 2024-03-28T13:11:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.543+0000 7f398b000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f39840af830 0x7f3984073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.543+0000 7f398b000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f398415fe80 con 0x7f39840d0650 2024-03-28T13:11:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.543+0000 7f398b000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39840ad120 con 0x7f398415c390 2024-03-28T13:11:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.543+0000 7f398b000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f398415c770 con 0x7f39840af830 2024-03-28T13:11:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.544+0000 7f3983400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f39840af830 0x7f3984073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.544+0000 7f3989000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f39840d0650 0x7f3984071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35694/0 (socket says 172.21.15.19:35694) 2024-03-28T13:11:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.544+0000 7f3989000640 1 -- 172.21.15.19:0/416117341 learned_addr learned my addr 172.21.15.19:0/416117341 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.544+0000 7f3983400640 1 -- 172.21.15.19:0/416117341 >> v1:172.21.15.188:6789/0 conn(0x7f398415c390 legacy=0x7f3984160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.544+0000 7f3983400640 1 -- 172.21.15.19:0/416117341 >> v1:172.21.15.19:6789/0 conn(0x7f39840d0650 legacy=0x7f3984071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.544+0000 7f3982a00640 1 -- 172.21.15.19:0/416117341 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 722490530 0 0) 0x7f398415fe80 con 0x7f39840d0650 2024-03-28T13:11:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.544+0000 7f3983400640 1 -- 172.21.15.19:0/416117341 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39840ad400 con 0x7f39840af830 2024-03-28T13:11:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.546+0000 7f3983400640 1 --2- 172.21.15.19:0/416117341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39840af830 0x7f3984073790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f396c0097a0 tx=0x7f396c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f262c6779c204fce server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.546+0000 7f3982a00640 1 -- 172.21.15.19:0/416117341 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c034070 con 0x7f39840af830 2024-03-28T13:11:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.546+0000 7f3982a00640 1 -- 172.21.15.19:0/416117341 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f396c030610 con 0x7f39840af830 2024-03-28T13:11:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.546+0000 7f3982a00640 1 -- 172.21.15.19:0/416117341 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c0309e0 con 0x7f39840af830 2024-03-28T13:11:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.547+0000 7f398b000640 1 -- 172.21.15.19:0/416117341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39840af830 msgr2=0x7f3984073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.547+0000 7f398b000640 1 --2- 172.21.15.19:0/416117341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39840af830 0x7f3984073790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f396c0097a0 tx=0x7f396c02b960 comp rx=0 tx=0).stop 2024-03-28T13:11:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.547+0000 7f398b000640 1 -- 172.21.15.19:0/416117341 shutdown_connections 2024-03-28T13:11:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.547+0000 7f398b000640 1 --2- 172.21.15.19:0/416117341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39840af830 0x7f3984073790 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.547+0000 7f398b000640 1 -- 172.21.15.19:0/416117341 >> 172.21.15.19:0/416117341 conn(0x7f398406b990 msgr2=0x7f398406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.547+0000 7f398b000640 1 -- 172.21.15.19:0/416117341 shutdown_connections 2024-03-28T13:11:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.547+0000 7f398b000640 1 -- 172.21.15.19:0/416117341 wait complete. 2024-03-28T13:11:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.549+0000 7f398b000640 1 Processor -- start 2024-03-28T13:11:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.549+0000 7f398b000640 1 -- start start 2024-03-28T13:11:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.550+0000 7f398b000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39840af830 0x7f398409ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.550+0000 7f398b000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39840d0650 0x7f398409b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.550+0000 7f398b000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398415c390 0x7f39840b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.550+0000 7f398b000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3984160d50 con 0x7f39840af830 2024-03-28T13:11:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.550+0000 7f398b000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3984160120 con 0x7f39840d0650 2024-03-28T13:11:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.550+0000 7f398b000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f39840cf980 con 0x7f398415c390 2024-03-28T13:11:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.550+0000 7f3983e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39840af830 0x7f398409ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.550+0000 7f3983e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39840af830 0x7f398409ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43474/0 (socket says 172.21.15.19:43474) 2024-03-28T13:11:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.550+0000 7f3983e00640 1 -- 172.21.15.19:0/297068084 learned_addr learned my addr 172.21.15.19:0/297068084 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.551+0000 7f3983400640 1 --2- 172.21.15.19:0/297068084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39840d0650 0x7f398409b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.551+0000 7f3983e00640 1 -- 172.21.15.19:0/297068084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398415c390 msgr2=0x7f39840b1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.551+0000 7f3983e00640 1 --2- 172.21.15.19:0/297068084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398415c390 0x7f39840b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.551+0000 7f3983e00640 1 -- 172.21.15.19:0/297068084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39840d0650 msgr2=0x7f398409b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:00.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.551+0000 7f3983e00640 1 --2- 172.21.15.19:0/297068084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39840d0650 0x7f398409b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.551+0000 7f3983e00640 1 -- 172.21.15.19:0/297068084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39840b2980 con 0x7f39840af830 2024-03-28T13:11:00.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.551+0000 7f3983e00640 1 --2- 172.21.15.19:0/297068084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39840af830 0x7f398409ac40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f397400c8a0 tx=0x7f397400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:00.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.551+0000 7f397be00640 1 -- 172.21.15.19:0/297068084 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3974003b70 con 0x7f39840af830 2024-03-28T13:11:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.551+0000 7f397be00640 1 -- 172.21.15.19:0/297068084 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3974003d30 con 0x7f39840af830 2024-03-28T13:11:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.551+0000 7f397be00640 1 -- 172.21.15.19:0/297068084 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397400aa90 con 0x7f39840af830 2024-03-28T13:11:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.551+0000 7f398b000640 1 -- 172.21.15.19:0/297068084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f398415d920 con 0x7f39840af830 2024-03-28T13:11:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.551+0000 7f398b000640 1 -- 172.21.15.19:0/297068084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f398415dee0 con 0x7f39840af830 2024-03-28T13:11:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.553+0000 7f398b000640 1 -- 172.21.15.19:0/297068084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39840685c0 con 0x7f39840af830 2024-03-28T13:11:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.553+0000 7f397be00640 1 -- 172.21.15.19:0/297068084 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f397400ac30 con 0x7f39840af830 2024-03-28T13:11:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.554+0000 7f397be00640 1 --2- 172.21.15.19:0/297068084 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f396006c530 0x7f396006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:00.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.554+0000 7f397be00640 1 -- 172.21.15.19:0/297068084 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f397408b660 con 0x7f39840af830 2024-03-28T13:11:00.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.555+0000 7f3983400640 1 --2- 172.21.15.19:0/297068084 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f396006c530 0x7f396006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.556+0000 7f3983400640 1 --2- 172.21.15.19:0/297068084 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f396006c530 0x7f396006e9f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f396c036000 tx=0x7f396c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:00.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.561+0000 7f397be00640 1 -- 172.21.15.19:0/297068084 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3974059c80 con 0x7f39840af830 2024-03-28T13:11:00.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.765+0000 7f398b000640 1 -- 172.21.15.19:0/297068084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f398406f1b0 con 0x7f39840af830 2024-03-28T13:11:00.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.767+0000 7f397be00640 1 -- 172.21.15.19:0/297068084 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16609 (secure 0 0 0) 0x7f3974059580 con 0x7f39840af830 2024-03-28T13:11:00.768 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:00.768 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":36,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:10:57.758556+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:10:54.307571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-28T13:10:55.205691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:11:00.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.770+0000 7f398b000640 1 -- 172.21.15.19:0/297068084 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f396006c530 msgr2=0x7f396006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:00.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.770+0000 7f398b000640 1 --2- 172.21.15.19:0/297068084 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f396006c530 0x7f396006e9f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f396c036000 tx=0x7f396c008030 comp rx=0 tx=0).stop 2024-03-28T13:11:00.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.770+0000 7f398b000640 1 -- 172.21.15.19:0/297068084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39840af830 msgr2=0x7f398409ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:00.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.770+0000 7f398b000640 1 --2- 172.21.15.19:0/297068084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39840af830 0x7f398409ac40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f397400c8a0 tx=0x7f397400cd60 comp rx=0 tx=0).stop 2024-03-28T13:11:00.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.771+0000 7f398b000640 1 -- 172.21.15.19:0/297068084 shutdown_connections 2024-03-28T13:11:00.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.771+0000 7f398b000640 1 --2- 172.21.15.19:0/297068084 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f396006c530 0x7f396006e9f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.771+0000 7f398b000640 1 --2- 172.21.15.19:0/297068084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398415c390 0x7f39840b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.771+0000 7f398b000640 1 --2- 172.21.15.19:0/297068084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f39840d0650 0x7f398409b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.771+0000 7f398b000640 1 --2- 172.21.15.19:0/297068084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f39840af830 0x7f398409ac40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.771+0000 7f398b000640 1 -- 172.21.15.19:0/297068084 >> 172.21.15.19:0/297068084 conn(0x7f398406b990 msgr2=0x7f398415abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.771+0000 7f398b000640 1 -- 172.21.15.19:0/297068084 shutdown_connections 2024-03-28T13:11:00.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:00.771+0000 7f398b000640 1 -- 172.21.15.19:0/297068084 wait complete. 2024-03-28T13:11:00.790 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-28T13:11:00.791 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:11:01.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.008+0000 7fc532c00640 1 Processor -- start 2024-03-28T13:11:01.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.008+0000 7fc532c00640 1 -- start start 2024-03-28T13:11:01.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.010+0000 7fc532c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c06ea90 0x7fc52c0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:01.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.010+0000 7fc532c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc52c161f30 con 0x7fc52c147a40 2024-03-28T13:11:01.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.010+0000 7fc532c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc52c1621d0 con 0x7fc52c0d0390 2024-03-28T13:11:01.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.010+0000 7fc532c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc52c162b00 con 0x7fc52c06ea90 2024-03-28T13:11:01.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.010+0000 7fc531800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc52c147a40 0x7fc52c0cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35708/0 (socket says 172.21.15.19:35708) 2024-03-28T13:11:01.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.010+0000 7fc531800640 1 -- 172.21.15.19:0/513010940 learned_addr learned my addr 172.21.15.19:0/513010940 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:01.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.011+0000 7fc52be00640 1 -- 172.21.15.19:0/513010940 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3425383784 0 0) 0x7fc52c161f30 con 0x7fc52c147a40 2024-03-28T13:11:01.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.011+0000 7fc532200640 1 --2- 172.21.15.19:0/513010940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c06ea90 0x7fc52c0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.011+0000 7fc52be00640 1 -- 172.21.15.19:0/513010940 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc514003520 con 0x7fc52c147a40 2024-03-28T13:11:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.011+0000 7fc52be00640 1 -- 172.21.15.19:0/513010940 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 100493749 0 0) 0x7fc514003520 con 0x7fc52c147a40 2024-03-28T13:11:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.011+0000 7fc52be00640 1 -- 172.21.15.19:0/513010940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c06ea90 msgr2=0x7fc52c0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.011+0000 7fc52be00640 1 --2- 172.21.15.19:0/513010940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c06ea90 0x7fc52c0ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.011+0000 7fc52be00640 1 -- 172.21.15.19:0/513010940 >> v1:172.21.15.188:6789/0 conn(0x7fc52c0d0390 legacy=0x7fc52c0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.011+0000 7fc52be00640 1 -- 172.21.15.19:0/513010940 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c1624b0 con 0x7fc52c147a40 2024-03-28T13:11:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.011+0000 7fc52be00640 1 -- 172.21.15.19:0/513010940 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fc520001690 con 0x7fc52c147a40 2024-03-28T13:11:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.012+0000 7fc52be00640 1 -- 172.21.15.19:0/513010940 >> v1:172.21.15.19:6789/0 conn(0x7fc52c147a40 legacy=0x7fc52c0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.012+0000 7fc52be00640 1 --2- 172.21.15.19:0/513010940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5140039e0 0x7fc514003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.012+0000 7fc52be00640 1 --2- 172.21.15.19:0/513010940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c06ea90 0x7fc514004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.012+0000 7fc52be00640 1 --2- 172.21.15.19:0/513010940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc514004d40 0x7fc5140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.012+0000 7fc52be00640 1 -- 172.21.15.19:0/513010940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c162b00 con 0x7fc5140039e0 2024-03-28T13:11:01.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.012+0000 7fc52be00640 1 -- 172.21.15.19:0/513010940 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc51400bbd0 con 0x7fc514004d40 2024-03-28T13:11:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.012+0000 7fc52be00640 1 -- 172.21.15.19:0/513010940 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc51400bdd0 con 0x7fc52c06ea90 2024-03-28T13:11:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.012+0000 7fc531800640 1 --2- 172.21.15.19:0/513010940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5140039e0 0x7fc514003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.012+0000 7fc530e00640 1 --2- 172.21.15.19:0/513010940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c06ea90 0x7fc514004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.012+0000 7fc532200640 1 --2- 172.21.15.19:0/513010940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc514004d40 0x7fc5140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.012+0000 7fc531800640 1 -- 172.21.15.19:0/513010940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c06ea90 msgr2=0x7fc514004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.012+0000 7fc531800640 1 --2- 172.21.15.19:0/513010940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52c06ea90 0x7fc514004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.012+0000 7fc531800640 1 -- 172.21.15.19:0/513010940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc514004d40 msgr2=0x7fc5140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.012+0000 7fc531800640 1 --2- 172.21.15.19:0/513010940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc514004d40 0x7fc5140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.013+0000 7fc531800640 1 -- 172.21.15.19:0/513010940 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc514003520 con 0x7fc5140039e0 2024-03-28T13:11:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.013+0000 7fc532200640 1 --2- 172.21.15.19:0/513010940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc514004d40 0x7fc5140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.014+0000 7fc531800640 1 --2- 172.21.15.19:0/513010940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5140039e0 0x7fc514003e00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc52000c0e0 tx=0x7fc514005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d414c2fbeddaa12 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:01.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.014+0000 7fc52be00640 1 -- 172.21.15.19:0/513010940 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520007c40 con 0x7fc5140039e0 2024-03-28T13:11:01.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.014+0000 7fc52be00640 1 -- 172.21.15.19:0/513010940 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc520030070 con 0x7fc5140039e0 2024-03-28T13:11:01.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.014+0000 7fc52be00640 1 -- 172.21.15.19:0/513010940 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc520005d10 con 0x7fc5140039e0 2024-03-28T13:11:01.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.015+0000 7fc532c00640 1 -- 172.21.15.19:0/513010940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5140039e0 msgr2=0x7fc514003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:01.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.015+0000 7fc532c00640 1 --2- 172.21.15.19:0/513010940 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5140039e0 0x7fc514003e00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc52000c0e0 tx=0x7fc514005830 comp rx=0 tx=0).stop 2024-03-28T13:11:01.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.015+0000 7fc532200640 1 -- 172.21.15.19:0/513010940 reap_dead start 2024-03-28T13:11:01.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.015+0000 7fc532c00640 1 -- 172.21.15.19:0/513010940 shutdown_connections 2024-03-28T13:11:01.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.015+0000 7fc532c00640 1 -- 172.21.15.19:0/513010940 >> 172.21.15.19:0/513010940 conn(0x7fc52c0ccad0 msgr2=0x7fc52c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:01.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.015+0000 7fc532c00640 1 -- 172.21.15.19:0/513010940 shutdown_connections 2024-03-28T13:11:01.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.015+0000 7fc532c00640 1 -- 172.21.15.19:0/513010940 wait complete. 2024-03-28T13:11:01.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.017+0000 7fc532c00640 1 Processor -- start 2024-03-28T13:11:01.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.017+0000 7fc532c00640 1 -- start start 2024-03-28T13:11:01.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.018+0000 7fc532c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5140039e0 0x7fc52c0b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:01.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.018+0000 7fc532c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc52c133e40 0x7fc52c1342c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.018+0000 7fc532c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c130c30 0x7fc52c1310b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.018+0000 7fc532c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c147fa0 con 0x7fc52c130c30 2024-03-28T13:11:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.018+0000 7fc532c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c147e20 con 0x7fc52c133e40 2024-03-28T13:11:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.018+0000 7fc532c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc52c0ce380 con 0x7fc5140039e0 2024-03-28T13:11:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.018+0000 7fc532200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c130c30 0x7fc52c1310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.018+0000 7fc532200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c130c30 0x7fc52c1310b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43490/0 (socket says 172.21.15.19:43490) 2024-03-28T13:11:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.018+0000 7fc532200640 1 -- 172.21.15.19:0/2259633061 learned_addr learned my addr 172.21.15.19:0/2259633061 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.018+0000 7fc531800640 1 --2- 172.21.15.19:0/2259633061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5140039e0 0x7fc52c0b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.019+0000 7fc530e00640 1 --2- 172.21.15.19:0/2259633061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc52c133e40 0x7fc52c1342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.019+0000 7fc532200640 1 -- 172.21.15.19:0/2259633061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5140039e0 msgr2=0x7fc52c0b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.019+0000 7fc532200640 1 --2- 172.21.15.19:0/2259633061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5140039e0 0x7fc52c0b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.019+0000 7fc532200640 1 -- 172.21.15.19:0/2259633061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc52c133e40 msgr2=0x7fc52c1342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.019+0000 7fc532200640 1 --2- 172.21.15.19:0/2259633061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc52c133e40 0x7fc52c1342c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.019+0000 7fc532200640 1 -- 172.21.15.19:0/2259633061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c0b5d90 con 0x7fc52c130c30 2024-03-28T13:11:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.019+0000 7fc532200640 1 --2- 172.21.15.19:0/2259633061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c130c30 0x7fc52c1310b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc52c0d0c80 tx=0x7fc524009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.019+0000 7fc529600640 1 -- 172.21.15.19:0/2259633061 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524019070 con 0x7fc52c130c30 2024-03-28T13:11:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.019+0000 7fc529600640 1 -- 172.21.15.19:0/2259633061 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc524007d20 con 0x7fc52c130c30 2024-03-28T13:11:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.019+0000 7fc529600640 1 -- 172.21.15.19:0/2259633061 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524014620 con 0x7fc52c130c30 2024-03-28T13:11:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.019+0000 7fc532c00640 1 -- 172.21.15.19:0/2259633061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc52c0b3390 con 0x7fc52c130c30 2024-03-28T13:11:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.020+0000 7fc532c00640 1 -- 172.21.15.19:0/2259633061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc52c0b57d0 con 0x7fc52c130c30 2024-03-28T13:11:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.021+0000 7fc532c00640 1 -- 172.21.15.19:0/2259633061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52c075940 con 0x7fc52c130c30 2024-03-28T13:11:01.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.022+0000 7fc529600640 1 -- 172.21.15.19:0/2259633061 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc52401c070 con 0x7fc52c130c30 2024-03-28T13:11:01.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.022+0000 7fc529600640 1 --2- 172.21.15.19:0/2259633061 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc50c06c530 0x7fc50c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:01.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.023+0000 7fc529600640 1 -- 172.21.15.19:0/2259633061 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fc52408b120 con 0x7fc52c130c30 2024-03-28T13:11:01.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.023+0000 7fc531800640 1 --2- 172.21.15.19:0/2259633061 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc50c06c530 0x7fc50c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:01.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.024+0000 7fc531800640 1 --2- 172.21.15.19:0/2259633061 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc50c06c530 0x7fc50c06e9f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc520034000 tx=0x7fc520007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:01.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.027+0000 7fc529600640 1 -- 172.21.15.19:0/2259633061 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc524059730 con 0x7fc52c130c30 2024-03-28T13:11:01.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.227+0000 7fc532c00640 1 -- 172.21.15.19:0/2259633061 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc52c0ceb50 con 0x7fc50c06c530 2024-03-28T13:11:01.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.249+0000 7fc529600640 1 -- 172.21.15.19:0/2259633061 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245478 (secure 0 0 0) 0x7fc52c0ceb50 con 0x7fc50c06c530 2024-03-28T13:11:01.252 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:01.254 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:11:01.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.254+0000 7fc532c00640 1 -- 172.21.15.19:0/2259633061 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc50c06c530 msgr2=0x7fc50c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:01.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.254+0000 7fc532c00640 1 --2- 172.21.15.19:0/2259633061 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc50c06c530 0x7fc50c06e9f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc520034000 tx=0x7fc520007000 comp rx=0 tx=0).stop 2024-03-28T13:11:01.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.254+0000 7fc532c00640 1 -- 172.21.15.19:0/2259633061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c130c30 msgr2=0x7fc52c1310b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:01.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.254+0000 7fc532c00640 1 --2- 172.21.15.19:0/2259633061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c130c30 0x7fc52c1310b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc52c0d0c80 tx=0x7fc524009d30 comp rx=0 tx=0).stop 2024-03-28T13:11:01.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.254+0000 7fc532c00640 1 -- 172.21.15.19:0/2259633061 shutdown_connections 2024-03-28T13:11:01.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.254+0000 7fc532c00640 1 --2- 172.21.15.19:0/2259633061 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc50c06c530 0x7fc50c06e9f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.254+0000 7fc532c00640 1 --2- 172.21.15.19:0/2259633061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52c130c30 0x7fc52c1310b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.254+0000 7fc532c00640 1 --2- 172.21.15.19:0/2259633061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc52c133e40 0x7fc52c1342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.254+0000 7fc532c00640 1 --2- 172.21.15.19:0/2259633061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5140039e0 0x7fc52c0b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.254+0000 7fc532c00640 1 -- 172.21.15.19:0/2259633061 >> 172.21.15.19:0/2259633061 conn(0x7fc52c0ccad0 msgr2=0x7fc52c160b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.254+0000 7fc532c00640 1 -- 172.21.15.19:0/2259633061 shutdown_connections 2024-03-28T13:11:01.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.254+0000 7fc532c00640 1 -- 172.21.15.19:0/2259633061 wait complete. 2024-03-28T13:11:01.273 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-03-28T13:11:00.009641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592002,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220296,"kb_used_data":4096,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754424,"statfs":{"total":773094113280,"available":772868530176,"internally_reserved":0,"allocated":4194304,"data_stored":2319098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.296954"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774657+0000","last_change":"2024-03-28T13:10:56.790536+0000","last_active":"2024-03-28T13:10:57.774657+0000","last_peered":"2024-03-28T13:10:57.774657+0000","last_clean":"2024-03-28T13:10:57.774657+0000","last_became_active":"2024-03-28T13:10:56.790180+0000","last_became_peered":"2024-03-28T13:10:56.790180+0000","last_unstale":"2024-03-28T13:10:57.774657+0000","last_undegraded":"2024-03-28T13:10:57.774657+0000","last_fullsized":"2024-03-28T13:10:57.774657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:06:36.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777345+0000","last_change":"2024-03-28T13:10:56.777966+0000","last_active":"2024-03-28T13:10:57.777345+0000","last_peered":"2024-03-28T13:10:57.777345+0000","last_clean":"2024-03-28T13:10:57.777345+0000","last_became_active":"2024-03-28T13:10:56.777658+0000","last_became_peered":"2024-03-28T13:10:56.777658+0000","last_unstale":"2024-03-28T13:10:57.777345+0000","last_undegraded":"2024-03-28T13:10:57.777345+0000","last_fullsized":"2024-03-28T13:10:57.777345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:00:01.855413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774642+0000","last_change":"2024-03-28T13:10:56.783317+0000","last_active":"2024-03-28T13:10:57.774642+0000","last_peered":"2024-03-28T13:10:57.774642+0000","last_clean":"2024-03-28T13:10:57.774642+0000","last_became_active":"2024-03-28T13:10:56.781718+0000","last_became_peered":"2024-03-28T13:10:56.781718+0000","last_unstale":"2024-03-28T13:10:57.774642+0000","last_undegraded":"2024-03-28T13:10:57.774642+0000","last_fullsized":"2024-03-28T13:10:57.774642+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:58:13.880385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783325+0000","last_change":"2024-03-28T13:10:56.779357+0000","last_active":"2024-03-28T13:10:57.783325+0000","last_peered":"2024-03-28T13:10:57.783325+0000","last_clean":"2024-03-28T13:10:57.783325+0000","last_became_active":"2024-03-28T13:10:56.779000+0000","last_became_peered":"2024-03-28T13:10:56.779000+0000","last_unstale":"2024-03-28T13:10:57.783325+0000","last_undegraded":"2024-03-28T13:10:57.783325+0000","last_fullsized":"2024-03-28T13:10:57.783325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:40:07.650748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778583+0000","last_change":"2024-03-28T13:10:56.793373+0000","last_active":"2024-03-28T13:10:57.778583+0000","last_peered":"2024-03-28T13:10:57.778583+0000","last_clean":"2024-03-28T13:10:57.778583+0000","last_became_active":"2024-03-28T13:10:56.789448+0000","last_became_peered":"2024-03-28T13:10:56.789448+0000","last_unstale":"2024-03-28T13:10:57.778583+0000","last_undegraded":"2024-03-28T13:10:57.778583+0000","last_fullsized":"2024-03-28T13:10:57.778583+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:37.170463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.787568+0000","last_change":"2024-03-28T13:10:56.780719+0000","last_active":"2024-03-28T13:10:56.787568+0000","last_peered":"2024-03-28T13:10:56.787568+0000","last_clean":"2024-03-28T13:10:56.787568+0000","last_became_active":"2024-03-28T13:10:56.780369+0000","last_became_peered":"2024-03-28T13:10:56.780369+0000","last_unstale":"2024-03-28T13:10:56.787568+0000","last_undegraded":"2024-03-28T13:10:56.787568+0000","last_fullsized":"2024-03-28T13:10:56.787568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:48:42.794602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777339+0000","last_change":"2024-03-28T13:10:56.774962+0000","last_active":"2024-03-28T13:10:57.777339+0000","last_peered":"2024-03-28T13:10:57.777339+0000","last_clean":"2024-03-28T13:10:57.777339+0000","last_became_active":"2024-03-28T13:10:56.774519+0000","last_became_peered":"2024-03-28T13:10:56.774519+0000","last_unstale":"2024-03-28T13:10:57.777339+0000","last_undegraded":"2024-03-28T13:10:57.777339+0000","last_fullsized":"2024-03-28T13:10:57.777339+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:40.900533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783603+0000","last_change":"2024-03-28T13:10:56.769141+0000","last_active":"2024-03-28T13:10:57.783603+0000","last_peered":"2024-03-28T13:10:57.783603+0000","last_clean":"2024-03-28T13:10:57.783603+0000","last_became_active":"2024-03-28T13:10:56.768873+0000","last_became_peered":"2024-03-28T13:10:56.768873+0000","last_unstale":"2024-03-28T13:10:57.783603+0000","last_undegraded":"2024-03-28T13:10:57.783603+0000","last_fullsized":"2024-03-28T13:10:57.783603+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:30:13.251989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783701+0000","last_change":"2024-03-28T13:10:56.767685+0000","last_active":"2024-03-28T13:10:57.783701+0000","last_peered":"2024-03-28T13:10:57.783701+0000","last_clean":"2024-03-28T13:10:57.783701+0000","last_became_active":"2024-03-28T13:10:56.767320+0000","last_became_peered":"2024-03-28T13:10:56.767320+0000","last_unstale":"2024-03-28T13:10:57.783701+0000","last_undegraded":"2024-03-28T13:10:57.783701+0000","last_fullsized":"2024-03-28T13:10:57.783701+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:45:22.482141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.782025+0000","last_change":"2024-03-28T13:10:56.772701+0000","last_active":"2024-03-28T13:10:56.782025+0000","last_peered":"2024-03-28T13:10:56.782025+0000","last_clean":"2024-03-28T13:10:56.782025+0000","last_became_active":"2024-03-28T13:10:56.772325+0000","last_became_peered":"2024-03-28T13:10:56.772325+0000","last_unstale":"2024-03-28T13:10:56.782025+0000","last_undegraded":"2024-03-28T13:10:56.782025+0000","last_fullsized":"2024-03-28T13:10:56.782025+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:19:57.308914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.779607+0000","last_change":"2024-03-28T13:10:56.768323+0000","last_active":"2024-03-28T13:10:56.779607+0000","last_peered":"2024-03-28T13:10:56.779607+0000","last_clean":"2024-03-28T13:10:56.779607+0000","last_became_active":"2024-03-28T13:10:56.766260+0000","last_became_peered":"2024-03-28T13:10:56.766260+0000","last_unstale":"2024-03-28T13:10:56.779607+0000","last_undegraded":"2024-03-28T13:10:56.779607+0000","last_fullsized":"2024-03-28T13:10:56.779607+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:55:35.027128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778158+0000","last_change":"2024-03-28T13:10:56.780679+0000","last_active":"2024-03-28T13:10:57.778158+0000","last_peered":"2024-03-28T13:10:57.778158+0000","last_clean":"2024-03-28T13:10:57.778158+0000","last_became_active":"2024-03-28T13:10:56.777813+0000","last_became_peered":"2024-03-28T13:10:56.777813+0000","last_unstale":"2024-03-28T13:10:57.778158+0000","last_undegraded":"2024-03-28T13:10:57.778158+0000","last_fullsized":"2024-03-28T13:10:57.778158+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:15:57.044511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778939+0000","last_change":"2024-03-28T13:10:56.781082+0000","last_active":"2024-03-28T13:10:57.778939+0000","last_peered":"2024-03-28T13:10:57.778939+0000","last_clean":"2024-03-28T13:10:57.778939+0000","last_became_active":"2024-03-28T13:10:56.780806+0000","last_became_peered":"2024-03-28T13:10:56.780806+0000","last_unstale":"2024-03-28T13:10:57.778939+0000","last_undegraded":"2024-03-28T13:10:57.778939+0000","last_fullsized":"2024-03-28T13:10:57.778939+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:33.779798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774874+0000","last_change":"2024-03-28T13:10:56.771825+0000","last_active":"2024-03-28T13:10:57.774874+0000","last_peered":"2024-03-28T13:10:57.774874+0000","last_clean":"2024-03-28T13:10:57.774874+0000","last_became_active":"2024-03-28T13:10:56.771505+0000","last_became_peered":"2024-03-28T13:10:56.771505+0000","last_unstale":"2024-03-28T13:10:57.774874+0000","last_undegraded":"2024-03-28T13:10:57.774874+0000","last_fullsized":"2024-03-28T13:10:57.774874+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.778739+0000","last_change":"2024-03-28T13:10:56.769275+0000","last_active":"2024-03-28T13:10:56.778739+0000","last_peered":"2024-03-28T13:10:56.778739+0000","last_clean":"2024-03-28T13:10:56.778739+0000","last_became_active":"2024-03-28T13:10:56.767794+0000","last_became_peered":"2024-03-28T13:10:56.767794+0000","last_unstale":"2024-03-28T13:10:56.778739+0000","last_undegraded":"2024-03-28T13:10:56.778739+0000","last_fullsized":"2024-03-28T13:10:56.778739+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:37:33.635609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-28T13:10:54.716030+0000","last_change":"2024-03-28T13:10:54.716030+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.716030+0000","last_undegraded":"2024-03-28T13:10:54.716030+0000","last_fullsized":"2024-03-28T13:10:54.716030+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.779205+0000","last_change":"2024-03-28T13:10:55.750470+0000","last_active":"2024-03-28T13:10:57.779205+0000","last_peered":"2024-03-28T13:10:57.779205+0000","last_clean":"2024-03-28T13:10:57.779205+0000","last_became_active":"2024-03-28T13:10:55.749676+0000","last_became_peered":"2024-03-28T13:10:55.749676+0000","last_unstale":"2024-03-28T13:10:57.779205+0000","last_undegraded":"2024-03-28T13:10:57.779205+0000","last_fullsized":"2024-03-28T13:10:57.779205+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:55:34.014926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783942+0000","last_change":"2024-03-28T13:10:55.754683+0000","last_active":"2024-03-28T13:10:57.783942+0000","last_peered":"2024-03-28T13:10:57.783942+0000","last_clean":"2024-03-28T13:10:57.783942+0000","last_became_active":"2024-03-28T13:10:55.754344+0000","last_became_peered":"2024-03-28T13:10:55.754344+0000","last_unstale":"2024-03-28T13:10:57.783942+0000","last_undegraded":"2024-03-28T13:10:57.783942+0000","last_fullsized":"2024-03-28T13:10:57.783942+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:22:12.385800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777982+0000","last_change":"2024-03-28T13:10:56.779261+0000","last_active":"2024-03-28T13:10:57.777982+0000","last_peered":"2024-03-28T13:10:57.777982+0000","last_clean":"2024-03-28T13:10:57.777982+0000","last_became_active":"2024-03-28T13:10:56.778899+0000","last_became_peered":"2024-03-28T13:10:56.778899+0000","last_unstale":"2024-03-28T13:10:57.777982+0000","last_undegraded":"2024-03-28T13:10:57.777982+0000","last_fullsized":"2024-03-28T13:10:57.777982+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:44:06.086773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777813+0000","last_change":"2024-03-28T13:10:55.755603+0000","last_active":"2024-03-28T13:10:57.777813+0000","last_peered":"2024-03-28T13:10:57.777813+0000","last_clean":"2024-03-28T13:10:57.777813+0000","last_became_active":"2024-03-28T13:10:55.754244+0000","last_became_peered":"2024-03-28T13:10:55.754244+0000","last_unstale":"2024-03-28T13:10:57.777813+0000","last_undegraded":"2024-03-28T13:10:57.777813+0000","last_fullsized":"2024-03-28T13:10:57.777813+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:30.977315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:54.697274+0000","last_change":"2024-03-28T13:10:54.697274+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.697274+0000","last_undegraded":"2024-03-28T13:10:54.697274+0000","last_fullsized":"2024-03-28T13:10:54.697274+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:54.697274+0000","last_change":"2024-03-28T13:10:54.697274+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.697274+0000","last_undegraded":"2024-03-28T13:10:54.697274+0000","last_fullsized":"2024-03-28T13:10:54.697274+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783436+0000","last_change":"2024-03-28T13:10:55.750960+0000","last_active":"2024-03-28T13:10:57.783436+0000","last_peered":"2024-03-28T13:10:57.783436+0000","last_clean":"2024-03-28T13:10:57.783436+0000","last_became_active":"2024-03-28T13:10:55.750618+0000","last_became_peered":"2024-03-28T13:10:55.750618+0000","last_unstale":"2024-03-28T13:10:57.783436+0000","last_undegraded":"2024-03-28T13:10:57.783436+0000","last_fullsized":"2024-03-28T13:10:57.783436+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:36.018433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776802+0000","last_change":"2024-03-28T13:10:56.775408+0000","last_active":"2024-03-28T13:10:57.776802+0000","last_peered":"2024-03-28T13:10:57.776802+0000","last_clean":"2024-03-28T13:10:57.776802+0000","last_became_active":"2024-03-28T13:10:56.775141+0000","last_became_peered":"2024-03-28T13:10:56.775141+0000","last_unstale":"2024-03-28T13:10:57.776802+0000","last_undegraded":"2024-03-28T13:10:57.776802+0000","last_fullsized":"2024-03-28T13:10:57.776802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:54.697274+0000","last_change":"2024-03-28T13:10:54.697274+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.697274+0000","last_undegraded":"2024-03-28T13:10:54.697274+0000","last_fullsized":"2024-03-28T13:10:54.697274+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.775253+0000","last_change":"2024-03-28T13:10:55.741344+0000","last_active":"2024-03-28T13:10:57.775253+0000","last_peered":"2024-03-28T13:10:57.775253+0000","last_clean":"2024-03-28T13:10:57.775253+0000","last_became_active":"2024-03-28T13:10:55.740955+0000","last_became_peered":"2024-03-28T13:10:55.740955+0000","last_unstale":"2024-03-28T13:10:57.775253+0000","last_undegraded":"2024-03-28T13:10:57.775253+0000","last_fullsized":"2024-03-28T13:10:57.775253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:55:33.765625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778342+0000","last_change":"2024-03-28T13:10:56.792475+0000","last_active":"2024-03-28T13:10:57.778342+0000","last_peered":"2024-03-28T13:10:57.778342+0000","last_clean":"2024-03-28T13:10:57.778342+0000","last_became_active":"2024-03-28T13:10:56.792165+0000","last_became_peered":"2024-03-28T13:10:56.792165+0000","last_unstale":"2024-03-28T13:10:57.778342+0000","last_undegraded":"2024-03-28T13:10:57.778342+0000","last_fullsized":"2024-03-28T13:10:57.778342+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:50:07.655844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774840+0000","last_change":"2024-03-28T13:10:55.749119+0000","last_active":"2024-03-28T13:10:57.774840+0000","last_peered":"2024-03-28T13:10:57.774840+0000","last_clean":"2024-03-28T13:10:57.774840+0000","last_became_active":"2024-03-28T13:10:55.748527+0000","last_became_peered":"2024-03-28T13:10:55.748527+0000","last_unstale":"2024-03-28T13:10:57.774840+0000","last_undegraded":"2024-03-28T13:10:57.774840+0000","last_fullsized":"2024-03-28T13:10:57.774840+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:30.977315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777568+0000","last_change":"2024-03-28T13:10:56.778342+0000","last_active":"2024-03-28T13:10:57.777568+0000","last_peered":"2024-03-28T13:10:57.777568+0000","last_clean":"2024-03-28T13:10:57.777568+0000","last_became_active":"2024-03-28T13:10:56.777259+0000","last_became_peered":"2024-03-28T13:10:56.777259+0000","last_unstale":"2024-03-28T13:10:57.777568+0000","last_undegraded":"2024-03-28T13:10:57.777568+0000","last_fullsized":"2024-03-28T13:10:57.777568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:23.920150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-28T13:10:54.716903+0000","last_change":"2024-03-28T13:10:54.716903+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.716903+0000","last_undegraded":"2024-03-28T13:10:54.716903+0000","last_fullsized":"2024-03-28T13:10:54.716903+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.804651+0000","last_change":"2024-03-28T13:10:55.757421+0000","last_active":"2024-03-28T13:10:56.804651+0000","last_peered":"2024-03-28T13:10:56.804651+0000","last_clean":"2024-03-28T13:10:56.804651+0000","last_became_active":"2024-03-28T13:10:55.757041+0000","last_became_peered":"2024-03-28T13:10:55.757041+0000","last_unstale":"2024-03-28T13:10:56.804651+0000","last_undegraded":"2024-03-28T13:10:56.804651+0000","last_fullsized":"2024-03-28T13:10:56.804651+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:46.007430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.777866+0000","last_change":"2024-03-28T13:10:56.767158+0000","last_active":"2024-03-28T13:10:56.777866+0000","last_peered":"2024-03-28T13:10:56.777866+0000","last_clean":"2024-03-28T13:10:56.777866+0000","last_became_active":"2024-03-28T13:10:56.762220+0000","last_became_peered":"2024-03-28T13:10:56.762220+0000","last_unstale":"2024-03-28T13:10:56.777866+0000","last_undegraded":"2024-03-28T13:10:56.777866+0000","last_fullsized":"2024-03-28T13:10:56.777866+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:36:29.152136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:54.697274+0000","last_change":"2024-03-28T13:10:54.697274+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.697274+0000","last_undegraded":"2024-03-28T13:10:54.697274+0000","last_fullsized":"2024-03-28T13:10:54.697274+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777175+0000","last_change":"2024-03-28T13:10:56.791145+0000","last_active":"2024-03-28T13:10:57.777175+0000","last_peered":"2024-03-28T13:10:57.777175+0000","last_clean":"2024-03-28T13:10:57.777175+0000","last_became_active":"2024-03-28T13:10:56.789400+0000","last_became_peered":"2024-03-28T13:10:56.789400+0000","last_unstale":"2024-03-28T13:10:57.777175+0000","last_undegraded":"2024-03-28T13:10:57.777175+0000","last_fullsized":"2024-03-28T13:10:57.777175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:26.233670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777676+0000","last_change":"2024-03-28T13:10:55.750434+0000","last_active":"2024-03-28T13:10:57.777676+0000","last_peered":"2024-03-28T13:10:57.777676+0000","last_clean":"2024-03-28T13:10:57.777676+0000","last_became_active":"2024-03-28T13:10:55.750086+0000","last_became_peered":"2024-03-28T13:10:55.750086+0000","last_unstale":"2024-03-28T13:10:57.777676+0000","last_undegraded":"2024-03-28T13:10:57.777676+0000","last_fullsized":"2024-03-28T13:10:57.777676+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:37:32.623408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776667+0000","last_change":"2024-03-28T13:10:55.755962+0000","last_active":"2024-03-28T13:10:57.776667+0000","last_peered":"2024-03-28T13:10:57.776667+0000","last_clean":"2024-03-28T13:10:57.776667+0000","last_became_active":"2024-03-28T13:10:55.755652+0000","last_became_peered":"2024-03-28T13:10:55.755652+0000","last_unstale":"2024-03-28T13:10:57.776667+0000","last_undegraded":"2024-03-28T13:10:57.776667+0000","last_fullsized":"2024-03-28T13:10:57.776667+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:51:08.557038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783479+0000","last_change":"2024-03-28T13:10:55.757692+0000","last_active":"2024-03-28T13:10:57.783479+0000","last_peered":"2024-03-28T13:10:57.783479+0000","last_clean":"2024-03-28T13:10:57.783479+0000","last_became_active":"2024-03-28T13:10:55.757463+0000","last_became_peered":"2024-03-28T13:10:55.757463+0000","last_unstale":"2024-03-28T13:10:57.783479+0000","last_undegraded":"2024-03-28T13:10:57.783479+0000","last_fullsized":"2024-03-28T13:10:57.783479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:12:56.307494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783676+0000","last_change":"2024-03-28T13:10:55.757407+0000","last_active":"2024-03-28T13:10:57.783676+0000","last_peered":"2024-03-28T13:10:57.783676+0000","last_clean":"2024-03-28T13:10:57.783676+0000","last_became_active":"2024-03-28T13:10:55.757088+0000","last_became_peered":"2024-03-28T13:10:55.757088+0000","last_unstale":"2024-03-28T13:10:57.783676+0000","last_undegraded":"2024-03-28T13:10:57.783676+0000","last_fullsized":"2024-03-28T13:10:57.783676+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:03:35.637042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:54.697274+0000","last_change":"2024-03-28T13:10:54.697274+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.697274+0000","last_undegraded":"2024-03-28T13:10:54.697274+0000","last_fullsized":"2024-03-28T13:10:54.697274+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774512+0000","last_change":"2024-03-28T13:10:56.774113+0000","last_active":"2024-03-28T13:10:57.774512+0000","last_peered":"2024-03-28T13:10:57.774512+0000","last_clean":"2024-03-28T13:10:57.774512+0000","last_became_active":"2024-03-28T13:10:56.773838+0000","last_became_peered":"2024-03-28T13:10:56.773838+0000","last_unstale":"2024-03-28T13:10:57.774512+0000","last_undegraded":"2024-03-28T13:10:57.774512+0000","last_fullsized":"2024-03-28T13:10:57.774512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:45.087620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-28T13:10:54.717508+0000","last_change":"2024-03-28T13:10:54.717508+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.717508+0000","last_undegraded":"2024-03-28T13:10:54.717508+0000","last_fullsized":"2024-03-28T13:10:54.717508+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783690+0000","last_change":"2024-03-28T13:10:56.779040+0000","last_active":"2024-03-28T13:10:57.783690+0000","last_peered":"2024-03-28T13:10:57.783690+0000","last_clean":"2024-03-28T13:10:57.783690+0000","last_became_active":"2024-03-28T13:10:56.777765+0000","last_became_peered":"2024-03-28T13:10:56.777765+0000","last_unstale":"2024-03-28T13:10:57.783690+0000","last_undegraded":"2024-03-28T13:10:57.783690+0000","last_fullsized":"2024-03-28T13:10:57.783690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:44.685445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-28T13:10:54.721535+0000","last_change":"2024-03-28T13:10:54.721535+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.721535+0000","last_undegraded":"2024-03-28T13:10:54.721535+0000","last_fullsized":"2024-03-28T13:10:54.721535+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.781976+0000","last_change":"2024-03-28T13:10:56.774575+0000","last_active":"2024-03-28T13:10:56.781976+0000","last_peered":"2024-03-28T13:10:56.781976+0000","last_clean":"2024-03-28T13:10:56.781976+0000","last_became_active":"2024-03-28T13:10:56.772544+0000","last_became_peered":"2024-03-28T13:10:56.772544+0000","last_unstale":"2024-03-28T13:10:56.781976+0000","last_undegraded":"2024-03-28T13:10:56.781976+0000","last_fullsized":"2024-03-28T13:10:56.781976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:29:15.008724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.775058+0000","last_change":"2024-03-28T13:10:55.754015+0000","last_active":"2024-03-28T13:10:57.775058+0000","last_peered":"2024-03-28T13:10:57.775058+0000","last_clean":"2024-03-28T13:10:57.775058+0000","last_became_active":"2024-03-28T13:10:55.753701+0000","last_became_peered":"2024-03-28T13:10:55.753701+0000","last_unstale":"2024-03-28T13:10:57.775058+0000","last_undegraded":"2024-03-28T13:10:57.775058+0000","last_fullsized":"2024-03-28T13:10:57.775058+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:29:13.996522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.775071+0000","last_change":"2024-03-28T13:10:56.782871+0000","last_active":"2024-03-28T13:10:57.775071+0000","last_peered":"2024-03-28T13:10:57.775071+0000","last_clean":"2024-03-28T13:10:57.775071+0000","last_became_active":"2024-03-28T13:10:56.782020+0000","last_became_peered":"2024-03-28T13:10:56.782020+0000","last_unstale":"2024-03-28T13:10:57.775071+0000","last_undegraded":"2024-03-28T13:10:57.775071+0000","last_fullsized":"2024-03-28T13:10:57.775071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:54.697274+0000","last_change":"2024-03-28T13:10:54.697274+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.697274+0000","last_undegraded":"2024-03-28T13:10:54.697274+0000","last_fullsized":"2024-03-28T13:10:54.697274+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.787619+0000","last_change":"2024-03-28T13:10:56.780543+0000","last_active":"2024-03-28T13:10:56.787619+0000","last_peered":"2024-03-28T13:10:56.787619+0000","last_clean":"2024-03-28T13:10:56.787619+0000","last_became_active":"2024-03-28T13:10:56.780189+0000","last_became_peered":"2024-03-28T13:10:56.780189+0000","last_unstale":"2024-03-28T13:10:56.787619+0000","last_undegraded":"2024-03-28T13:10:56.787619+0000","last_fullsized":"2024-03-28T13:10:56.787619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:31.989516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-28T13:10:54.720221+0000","last_change":"2024-03-28T13:10:54.720221+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.720221+0000","last_undegraded":"2024-03-28T13:10:54.720221+0000","last_fullsized":"2024-03-28T13:10:54.720221+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-28T13:10:54.715957+0000","last_change":"2024-03-28T13:10:54.715957+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.715957+0000","last_undegraded":"2024-03-28T13:10:54.715957+0000","last_fullsized":"2024-03-28T13:10:54.715957+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783615+0000","last_change":"2024-03-28T13:10:56.775289+0000","last_active":"2024-03-28T13:10:57.783615+0000","last_peered":"2024-03-28T13:10:57.783615+0000","last_clean":"2024-03-28T13:10:57.783615+0000","last_became_active":"2024-03-28T13:10:56.774549+0000","last_became_peered":"2024-03-28T13:10:56.774549+0000","last_unstale":"2024-03-28T13:10:57.783615+0000","last_undegraded":"2024-03-28T13:10:57.783615+0000","last_fullsized":"2024-03-28T13:10:57.783615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:16.475067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.794905+0000","last_change":"2024-03-28T13:10:55.755495+0000","last_active":"2024-03-28T13:10:56.794905+0000","last_peered":"2024-03-28T13:10:56.794905+0000","last_clean":"2024-03-28T13:10:56.794905+0000","last_became_active":"2024-03-28T13:10:55.754987+0000","last_became_peered":"2024-03-28T13:10:55.754987+0000","last_unstale":"2024-03-28T13:10:56.794905+0000","last_undegraded":"2024-03-28T13:10:56.794905+0000","last_fullsized":"2024-03-28T13:10:56.794905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:57:31.432582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783204+0000","last_change":"2024-03-28T13:10:55.748821+0000","last_active":"2024-03-28T13:10:57.783204+0000","last_peered":"2024-03-28T13:10:57.783204+0000","last_clean":"2024-03-28T13:10:57.783204+0000","last_became_active":"2024-03-28T13:10:55.748128+0000","last_became_peered":"2024-03-28T13:10:55.748128+0000","last_unstale":"2024-03-28T13:10:57.783204+0000","last_undegraded":"2024-03-28T13:10:57.783204+0000","last_fullsized":"2024-03-28T13:10:57.783204+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:48:41.782400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783822+0000","last_change":"2024-03-28T13:10:56.785728+0000","last_active":"2024-03-28T13:10:57.783822+0000","last_peered":"2024-03-28T13:10:57.783822+0000","last_clean":"2024-03-28T13:10:57.783822+0000","last_became_active":"2024-03-28T13:10:56.783504+0000","last_became_peered":"2024-03-28T13:10:56.783504+0000","last_unstale":"2024-03-28T13:10:57.783822+0000","last_undegraded":"2024-03-28T13:10:57.783822+0000","last_fullsized":"2024-03-28T13:10:57.783822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:42:20.735994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.747889+0000","last_change":"2024-03-28T13:10:55.757201+0000","last_active":"2024-03-28T13:10:56.747889+0000","last_peered":"2024-03-28T13:10:56.747889+0000","last_clean":"2024-03-28T13:10:56.747889+0000","last_became_active":"2024-03-28T13:10:55.756328+0000","last_became_peered":"2024-03-28T13:10:55.756328+0000","last_unstale":"2024-03-28T13:10:56.747889+0000","last_undegraded":"2024-03-28T13:10:56.747889+0000","last_fullsized":"2024-03-28T13:10:56.747889+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:50:09.927656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.747164+0000","last_change":"2024-03-28T13:10:55.753645+0000","last_active":"2024-03-28T13:10:56.747164+0000","last_peered":"2024-03-28T13:10:56.747164+0000","last_clean":"2024-03-28T13:10:56.747164+0000","last_became_active":"2024-03-28T13:10:55.753103+0000","last_became_peered":"2024-03-28T13:10:55.753103+0000","last_unstale":"2024-03-28T13:10:56.747164+0000","last_undegraded":"2024-03-28T13:10:56.747164+0000","last_fullsized":"2024-03-28T13:10:56.747164+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:38:25.780963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777482+0000","last_change":"2024-03-28T13:10:56.784216+0000","last_active":"2024-03-28T13:10:57.777482+0000","last_peered":"2024-03-28T13:10:57.777482+0000","last_clean":"2024-03-28T13:10:57.777482+0000","last_became_active":"2024-03-28T13:10:56.783936+0000","last_became_peered":"2024-03-28T13:10:56.783936+0000","last_unstale":"2024-03-28T13:10:57.777482+0000","last_undegraded":"2024-03-28T13:10:57.777482+0000","last_fullsized":"2024-03-28T13:10:57.777482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:53.479188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.748281+0000","last_change":"2024-03-28T13:10:55.754934+0000","last_active":"2024-03-28T13:10:56.748281+0000","last_peered":"2024-03-28T13:10:56.748281+0000","last_clean":"2024-03-28T13:10:56.748281+0000","last_became_active":"2024-03-28T13:10:55.754215+0000","last_became_peered":"2024-03-28T13:10:55.754215+0000","last_unstale":"2024-03-28T13:10:56.748281+0000","last_undegraded":"2024-03-28T13:10:56.748281+0000","last_fullsized":"2024-03-28T13:10:56.748281+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:51:08.557038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776768+0000","last_change":"2024-03-28T13:10:56.791043+0000","last_active":"2024-03-28T13:10:57.776768+0000","last_peered":"2024-03-28T13:10:57.776768+0000","last_clean":"2024-03-28T13:10:57.776768+0000","last_became_active":"2024-03-28T13:10:56.787198+0000","last_became_peered":"2024-03-28T13:10:56.787198+0000","last_unstale":"2024-03-28T13:10:57.776768+0000","last_undegraded":"2024-03-28T13:10:57.776768+0000","last_fullsized":"2024-03-28T13:10:57.776768+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:54.697274+0000","last_change":"2024-03-28T13:10:54.697274+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.697274+0000","last_undegraded":"2024-03-28T13:10:54.697274+0000","last_fullsized":"2024-03-28T13:10:54.697274+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776675+0000","last_change":"2024-03-28T13:10:55.746007+0000","last_active":"2024-03-28T13:10:57.776675+0000","last_peered":"2024-03-28T13:10:57.776675+0000","last_clean":"2024-03-28T13:10:57.776675+0000","last_became_active":"2024-03-28T13:10:55.745666+0000","last_became_peered":"2024-03-28T13:10:55.745666+0000","last_unstale":"2024-03-28T13:10:57.776675+0000","last_undegraded":"2024-03-28T13:10:57.776675+0000","last_fullsized":"2024-03-28T13:10:57.776675+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:36:28.139935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-28T13:10:54.715991+0000","last_change":"2024-03-28T13:10:54.715991+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.715991+0000","last_undegraded":"2024-03-28T13:10:54.715991+0000","last_fullsized":"2024-03-28T13:10:54.715991+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":188,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T13:10:54.721334+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:10:54.721334+0000","last_peered":"2024-03-28T13:10:54.721334+0000","last_clean":"2024-03-28T13:10:54.721334+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:10:54.721334+0000","last_undegraded":"2024-03-28T13:10:54.721334+0000","last_fullsized":"2024-03-28T13:10:54.721334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":60,"acting":60,"num_store_stats":4},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":50,"acting":50,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476759,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":348160,"data_stored":130018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476757,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":130018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476757,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":720386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476757,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":130018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":397312,"data_stored":150497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":746135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":156819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":155207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:11:01.278 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:11:01.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:11:01.367 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:11:01.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.527+0000 7f3d2e400640 1 Processor -- start 2024-03-28T13:11:01.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.527+0000 7f3d2e400640 1 -- start start 2024-03-28T13:11:01.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.529+0000 7f3d2e400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2814ba20 0x7f3d280b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:01.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.529+0000 7f3d2e400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3d280cfda0 0x7f3d280d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:01.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.529+0000 7f3d2e400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d280b33e0 con 0x7f3d2814c1b0 2024-03-28T13:11:01.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.529+0000 7f3d2e400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3d280d0710 con 0x7f3d280cfda0 2024-03-28T13:11:01.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.529+0000 7f3d2e400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3d280cde20 con 0x7f3d2814ba20 2024-03-28T13:11:01.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.529+0000 7f3d2da00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3d280cfda0 0x7f3d280d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:01.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.529+0000 7f3d27e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3d2814c1b0 0x7f3d28166290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35712/0 (socket says 172.21.15.19:35712) 2024-03-28T13:11:01.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.529+0000 7f3d27e00640 1 -- 172.21.15.19:0/1484483460 learned_addr learned my addr 172.21.15.19:0/1484483460 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:01.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.530+0000 7f3d2d000640 1 --2- 172.21.15.19:0/1484483460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2814ba20 0x7f3d280b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:01.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.530+0000 7f3d2da00640 1 -- 172.21.15.19:0/1484483460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2814ba20 msgr2=0x7f3d280b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:01.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.530+0000 7f3d2da00640 1 --2- 172.21.15.19:0/1484483460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2814ba20 0x7f3d280b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.530+0000 7f3d27400640 1 -- 172.21.15.19:0/1484483460 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1410495555 0 0) 0x7f3d280b33e0 con 0x7f3d2814c1b0 2024-03-28T13:11:01.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.530+0000 7f3d2da00640 1 -- 172.21.15.19:0/1484483460 >> v1:172.21.15.19:6789/0 conn(0x7f3d2814c1b0 legacy=0x7f3d28166290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:01.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.530+0000 7f3d2da00640 1 -- 172.21.15.19:0/1484483460 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d28166ae0 con 0x7f3d280cfda0 2024-03-28T13:11:01.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.531+0000 7f3d2da00640 1 --2- 172.21.15.19:0/1484483460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d280cfda0 0x7f3d280d01d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009d90 tx=0x7f3d2002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3333cfb0506d68f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:01.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.532+0000 7f3d27400640 1 -- 172.21.15.19:0/1484483460 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20039070 con 0x7f3d280cfda0 2024-03-28T13:11:01.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.532+0000 7f3d27400640 1 -- 172.21.15.19:0/1484483460 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d2002cc60 con 0x7f3d280cfda0 2024-03-28T13:11:01.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.532+0000 7f3d27400640 1 -- 172.21.15.19:0/1484483460 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20034660 con 0x7f3d280cfda0 2024-03-28T13:11:01.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.532+0000 7f3d2e400640 1 -- 172.21.15.19:0/1484483460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d280cfda0 msgr2=0x7f3d280d01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:01.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.532+0000 7f3d2e400640 1 --2- 172.21.15.19:0/1484483460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d280cfda0 0x7f3d280d01d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009d90 tx=0x7f3d2002beb0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.532+0000 7f3d2e400640 1 -- 172.21.15.19:0/1484483460 shutdown_connections 2024-03-28T13:11:01.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.532+0000 7f3d2e400640 1 --2- 172.21.15.19:0/1484483460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d280cfda0 0x7f3d280d01d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.532+0000 7f3d2e400640 1 --2- 172.21.15.19:0/1484483460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2814ba20 0x7f3d280b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.532+0000 7f3d2e400640 1 -- 172.21.15.19:0/1484483460 >> 172.21.15.19:0/1484483460 conn(0x7f3d280ccad0 msgr2=0x7f3d280cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:01.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.532+0000 7f3d2e400640 1 -- 172.21.15.19:0/1484483460 shutdown_connections 2024-03-28T13:11:01.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.533+0000 7f3d2e400640 1 -- 172.21.15.19:0/1484483460 wait complete. 2024-03-28T13:11:01.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.534+0000 7f3d2e400640 1 Processor -- start 2024-03-28T13:11:01.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.534+0000 7f3d2e400640 1 -- start start 2024-03-28T13:11:01.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.535+0000 7f3d2e400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d280cfda0 0x7f3d281f8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:01.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.535+0000 7f3d2e400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2814ba20 0x7f3d28148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:01.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.535+0000 7f3d2e400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d2814c1b0 0x7f3d280c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:01.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.535+0000 7f3d2e400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d280ce0c0 con 0x7f3d2814ba20 2024-03-28T13:11:01.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.535+0000 7f3d2e400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281670a0 con 0x7f3d2814c1b0 2024-03-28T13:11:01.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.535+0000 7f3d2e400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3d2814be00 con 0x7f3d280cfda0 2024-03-28T13:11:01.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.536+0000 7f3d27e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2814ba20 0x7f3d28148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:01.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.536+0000 7f3d27e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2814ba20 0x7f3d28148b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43508/0 (socket says 172.21.15.19:43508) 2024-03-28T13:11:01.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.536+0000 7f3d27e00640 1 -- 172.21.15.19:0/653741749 learned_addr learned my addr 172.21.15.19:0/653741749 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:01.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.536+0000 7f3d2da00640 1 --2- 172.21.15.19:0/653741749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d2814c1b0 0x7f3d280c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:01.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.536+0000 7f3d27e00640 1 -- 172.21.15.19:0/653741749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d280cfda0 msgr2=0x7f3d281f8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:01.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.536+0000 7f3d2d000640 1 --2- 172.21.15.19:0/653741749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d280cfda0 0x7f3d281f8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:01.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.536+0000 7f3d27e00640 1 --2- 172.21.15.19:0/653741749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d280cfda0 0x7f3d281f8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.536+0000 7f3d27e00640 1 -- 172.21.15.19:0/653741749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d2814c1b0 msgr2=0x7f3d280c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:01.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.536+0000 7f3d27e00640 1 --2- 172.21.15.19:0/653741749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d2814c1b0 0x7f3d280c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.536+0000 7f3d27e00640 1 -- 172.21.15.19:0/653741749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d280c73e0 con 0x7f3d2814ba20 2024-03-28T13:11:01.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.537+0000 7f3d27e00640 1 --2- 172.21.15.19:0/653741749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2814ba20 0x7f3d28148b90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400c4d0 tx=0x7f3d1400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:01.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.537+0000 7f3d24c00640 1 -- 172.21.15.19:0/653741749 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d14018070 con 0x7f3d2814ba20 2024-03-28T13:11:01.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.537+0000 7f3d24c00640 1 -- 172.21.15.19:0/653741749 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d14007500 con 0x7f3d2814ba20 2024-03-28T13:11:01.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.537+0000 7f3d24c00640 1 -- 172.21.15.19:0/653741749 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d14004a20 con 0x7f3d2814ba20 2024-03-28T13:11:01.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.537+0000 7f3d2e400640 1 -- 172.21.15.19:0/653741749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d280c7650 con 0x7f3d2814ba20 2024-03-28T13:11:01.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.537+0000 7f3d2e400640 1 -- 172.21.15.19:0/653741749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d280c7c10 con 0x7f3d2814ba20 2024-03-28T13:11:01.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.538+0000 7f3d2e400640 1 -- 172.21.15.19:0/653741749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d28075940 con 0x7f3d2814ba20 2024-03-28T13:11:01.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.539+0000 7f3d24c00640 1 -- 172.21.15.19:0/653741749 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3d14004bc0 con 0x7f3d2814ba20 2024-03-28T13:11:01.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.540+0000 7f3d24c00640 1 --2- 172.21.15.19:0/653741749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d0806c530 0x7f3d0806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:01.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.540+0000 7f3d24c00640 1 -- 172.21.15.19:0/653741749 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f3d1408a980 con 0x7f3d2814ba20 2024-03-28T13:11:01.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.541+0000 7f3d2d000640 1 --2- 172.21.15.19:0/653741749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d0806c530 0x7f3d0806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:01.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.542+0000 7f3d2d000640 1 --2- 172.21.15.19:0/653741749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d0806c530 0x7f3d0806e9f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c006fd0 tx=0x7f3d1c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:01.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.545+0000 7f3d24c00640 1 -- 172.21.15.19:0/653741749 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3d14058fa0 con 0x7f3d2814ba20 2024-03-28T13:11:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.747+0000 7f3d2e400640 1 -- 172.21.15.19:0/653741749 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d28162a80 con 0x7f3d0806c530 2024-03-28T13:11:01.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.769+0000 7f3d24c00640 1 -- 172.21.15.19:0/653741749 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245478 (secure 0 0 0) 0x7f3d28162a80 con 0x7f3d0806c530 2024-03-28T13:11:01.772 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:01.773 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:11:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.774+0000 7f3d2e400640 1 -- 172.21.15.19:0/653741749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d0806c530 msgr2=0x7f3d0806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.774+0000 7f3d2e400640 1 --2- 172.21.15.19:0/653741749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d0806c530 0x7f3d0806e9f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c006fd0 tx=0x7f3d1c008040 comp rx=0 tx=0).stop 2024-03-28T13:11:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.774+0000 7f3d2e400640 1 -- 172.21.15.19:0/653741749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2814ba20 msgr2=0x7f3d28148b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.774+0000 7f3d2e400640 1 --2- 172.21.15.19:0/653741749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2814ba20 0x7f3d28148b90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3d1400c4d0 tx=0x7f3d1400c990 comp rx=0 tx=0).stop 2024-03-28T13:11:01.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.774+0000 7f3d2e400640 1 -- 172.21.15.19:0/653741749 shutdown_connections 2024-03-28T13:11:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.774+0000 7f3d2e400640 1 --2- 172.21.15.19:0/653741749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d0806c530 0x7f3d0806e9f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.774+0000 7f3d2e400640 1 --2- 172.21.15.19:0/653741749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d2814c1b0 0x7f3d280c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.774+0000 7f3d2e400640 1 --2- 172.21.15.19:0/653741749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2814ba20 0x7f3d28148b90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.774+0000 7f3d2e400640 1 --2- 172.21.15.19:0/653741749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d280cfda0 0x7f3d281f8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.774+0000 7f3d2e400640 1 -- 172.21.15.19:0/653741749 >> 172.21.15.19:0/653741749 conn(0x7f3d280ccad0 msgr2=0x7f3d280b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.774+0000 7f3d2e400640 1 -- 172.21.15.19:0/653741749 shutdown_connections 2024-03-28T13:11:01.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:01.774+0000 7f3d2e400640 1 -- 172.21.15.19:0/653741749 wait complete. 2024-03-28T13:11:01.793 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-03-28T13:11:00.009641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592002,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220296,"kb_used_data":4096,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754424,"statfs":{"total":773094113280,"available":772868530176,"internally_reserved":0,"allocated":4194304,"data_stored":2319098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.296954"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774657+0000","last_change":"2024-03-28T13:10:56.790536+0000","last_active":"2024-03-28T13:10:57.774657+0000","last_peered":"2024-03-28T13:10:57.774657+0000","last_clean":"2024-03-28T13:10:57.774657+0000","last_became_active":"2024-03-28T13:10:56.790180+0000","last_became_peered":"2024-03-28T13:10:56.790180+0000","last_unstale":"2024-03-28T13:10:57.774657+0000","last_undegraded":"2024-03-28T13:10:57.774657+0000","last_fullsized":"2024-03-28T13:10:57.774657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:06:36.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777345+0000","last_change":"2024-03-28T13:10:56.777966+0000","last_active":"2024-03-28T13:10:57.777345+0000","last_peered":"2024-03-28T13:10:57.777345+0000","last_clean":"2024-03-28T13:10:57.777345+0000","last_became_active":"2024-03-28T13:10:56.777658+0000","last_became_peered":"2024-03-28T13:10:56.777658+0000","last_unstale":"2024-03-28T13:10:57.777345+0000","last_undegraded":"2024-03-28T13:10:57.777345+0000","last_fullsized":"2024-03-28T13:10:57.777345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:00:01.855413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774642+0000","last_change":"2024-03-28T13:10:56.783317+0000","last_active":"2024-03-28T13:10:57.774642+0000","last_peered":"2024-03-28T13:10:57.774642+0000","last_clean":"2024-03-28T13:10:57.774642+0000","last_became_active":"2024-03-28T13:10:56.781718+0000","last_became_peered":"2024-03-28T13:10:56.781718+0000","last_unstale":"2024-03-28T13:10:57.774642+0000","last_undegraded":"2024-03-28T13:10:57.774642+0000","last_fullsized":"2024-03-28T13:10:57.774642+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:58:13.880385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783325+0000","last_change":"2024-03-28T13:10:56.779357+0000","last_active":"2024-03-28T13:10:57.783325+0000","last_peered":"2024-03-28T13:10:57.783325+0000","last_clean":"2024-03-28T13:10:57.783325+0000","last_became_active":"2024-03-28T13:10:56.779000+0000","last_became_peered":"2024-03-28T13:10:56.779000+0000","last_unstale":"2024-03-28T13:10:57.783325+0000","last_undegraded":"2024-03-28T13:10:57.783325+0000","last_fullsized":"2024-03-28T13:10:57.783325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:40:07.650748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778583+0000","last_change":"2024-03-28T13:10:56.793373+0000","last_active":"2024-03-28T13:10:57.778583+0000","last_peered":"2024-03-28T13:10:57.778583+0000","last_clean":"2024-03-28T13:10:57.778583+0000","last_became_active":"2024-03-28T13:10:56.789448+0000","last_became_peered":"2024-03-28T13:10:56.789448+0000","last_unstale":"2024-03-28T13:10:57.778583+0000","last_undegraded":"2024-03-28T13:10:57.778583+0000","last_fullsized":"2024-03-28T13:10:57.778583+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:37.170463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.787568+0000","last_change":"2024-03-28T13:10:56.780719+0000","last_active":"2024-03-28T13:10:56.787568+0000","last_peered":"2024-03-28T13:10:56.787568+0000","last_clean":"2024-03-28T13:10:56.787568+0000","last_became_active":"2024-03-28T13:10:56.780369+0000","last_became_peered":"2024-03-28T13:10:56.780369+0000","last_unstale":"2024-03-28T13:10:56.787568+0000","last_undegraded":"2024-03-28T13:10:56.787568+0000","last_fullsized":"2024-03-28T13:10:56.787568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:48:42.794602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777339+0000","last_change":"2024-03-28T13:10:56.774962+0000","last_active":"2024-03-28T13:10:57.777339+0000","last_peered":"2024-03-28T13:10:57.777339+0000","last_clean":"2024-03-28T13:10:57.777339+0000","last_became_active":"2024-03-28T13:10:56.774519+0000","last_became_peered":"2024-03-28T13:10:56.774519+0000","last_unstale":"2024-03-28T13:10:57.777339+0000","last_undegraded":"2024-03-28T13:10:57.777339+0000","last_fullsized":"2024-03-28T13:10:57.777339+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:40.900533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783603+0000","last_change":"2024-03-28T13:10:56.769141+0000","last_active":"2024-03-28T13:10:57.783603+0000","last_peered":"2024-03-28T13:10:57.783603+0000","last_clean":"2024-03-28T13:10:57.783603+0000","last_became_active":"2024-03-28T13:10:56.768873+0000","last_became_peered":"2024-03-28T13:10:56.768873+0000","last_unstale":"2024-03-28T13:10:57.783603+0000","last_undegraded":"2024-03-28T13:10:57.783603+0000","last_fullsized":"2024-03-28T13:10:57.783603+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:30:13.251989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783701+0000","last_change":"2024-03-28T13:10:56.767685+0000","last_active":"2024-03-28T13:10:57.783701+0000","last_peered":"2024-03-28T13:10:57.783701+0000","last_clean":"2024-03-28T13:10:57.783701+0000","last_became_active":"2024-03-28T13:10:56.767320+0000","last_became_peered":"2024-03-28T13:10:56.767320+0000","last_unstale":"2024-03-28T13:10:57.783701+0000","last_undegraded":"2024-03-28T13:10:57.783701+0000","last_fullsized":"2024-03-28T13:10:57.783701+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:45:22.482141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.782025+0000","last_change":"2024-03-28T13:10:56.772701+0000","last_active":"2024-03-28T13:10:56.782025+0000","last_peered":"2024-03-28T13:10:56.782025+0000","last_clean":"2024-03-28T13:10:56.782025+0000","last_became_active":"2024-03-28T13:10:56.772325+0000","last_became_peered":"2024-03-28T13:10:56.772325+0000","last_unstale":"2024-03-28T13:10:56.782025+0000","last_undegraded":"2024-03-28T13:10:56.782025+0000","last_fullsized":"2024-03-28T13:10:56.782025+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:19:57.308914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.779607+0000","last_change":"2024-03-28T13:10:56.768323+0000","last_active":"2024-03-28T13:10:56.779607+0000","last_peered":"2024-03-28T13:10:56.779607+0000","last_clean":"2024-03-28T13:10:56.779607+0000","last_became_active":"2024-03-28T13:10:56.766260+0000","last_became_peered":"2024-03-28T13:10:56.766260+0000","last_unstale":"2024-03-28T13:10:56.779607+0000","last_undegraded":"2024-03-28T13:10:56.779607+0000","last_fullsized":"2024-03-28T13:10:56.779607+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:55:35.027128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778158+0000","last_change":"2024-03-28T13:10:56.780679+0000","last_active":"2024-03-28T13:10:57.778158+0000","last_peered":"2024-03-28T13:10:57.778158+0000","last_clean":"2024-03-28T13:10:57.778158+0000","last_became_active":"2024-03-28T13:10:56.777813+0000","last_became_peered":"2024-03-28T13:10:56.777813+0000","last_unstale":"2024-03-28T13:10:57.778158+0000","last_undegraded":"2024-03-28T13:10:57.778158+0000","last_fullsized":"2024-03-28T13:10:57.778158+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:15:57.044511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778939+0000","last_change":"2024-03-28T13:10:56.781082+0000","last_active":"2024-03-28T13:10:57.778939+0000","last_peered":"2024-03-28T13:10:57.778939+0000","last_clean":"2024-03-28T13:10:57.778939+0000","last_became_active":"2024-03-28T13:10:56.780806+0000","last_became_peered":"2024-03-28T13:10:56.780806+0000","last_unstale":"2024-03-28T13:10:57.778939+0000","last_undegraded":"2024-03-28T13:10:57.778939+0000","last_fullsized":"2024-03-28T13:10:57.778939+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:33.779798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774874+0000","last_change":"2024-03-28T13:10:56.771825+0000","last_active":"2024-03-28T13:10:57.774874+0000","last_peered":"2024-03-28T13:10:57.774874+0000","last_clean":"2024-03-28T13:10:57.774874+0000","last_became_active":"2024-03-28T13:10:56.771505+0000","last_became_peered":"2024-03-28T13:10:56.771505+0000","last_unstale":"2024-03-28T13:10:57.774874+0000","last_undegraded":"2024-03-28T13:10:57.774874+0000","last_fullsized":"2024-03-28T13:10:57.774874+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.778739+0000","last_change":"2024-03-28T13:10:56.769275+0000","last_active":"2024-03-28T13:10:56.778739+0000","last_peered":"2024-03-28T13:10:56.778739+0000","last_clean":"2024-03-28T13:10:56.778739+0000","last_became_active":"2024-03-28T13:10:56.767794+0000","last_became_peered":"2024-03-28T13:10:56.767794+0000","last_unstale":"2024-03-28T13:10:56.778739+0000","last_undegraded":"2024-03-28T13:10:56.778739+0000","last_fullsized":"2024-03-28T13:10:56.778739+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:37:33.635609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-28T13:10:54.716030+0000","last_change":"2024-03-28T13:10:54.716030+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.716030+0000","last_undegraded":"2024-03-28T13:10:54.716030+0000","last_fullsized":"2024-03-28T13:10:54.716030+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.779205+0000","last_change":"2024-03-28T13:10:55.750470+0000","last_active":"2024-03-28T13:10:57.779205+0000","last_peered":"2024-03-28T13:10:57.779205+0000","last_clean":"2024-03-28T13:10:57.779205+0000","last_became_active":"2024-03-28T13:10:55.749676+0000","last_became_peered":"2024-03-28T13:10:55.749676+0000","last_unstale":"2024-03-28T13:10:57.779205+0000","last_undegraded":"2024-03-28T13:10:57.779205+0000","last_fullsized":"2024-03-28T13:10:57.779205+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:55:34.014926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783942+0000","last_change":"2024-03-28T13:10:55.754683+0000","last_active":"2024-03-28T13:10:57.783942+0000","last_peered":"2024-03-28T13:10:57.783942+0000","last_clean":"2024-03-28T13:10:57.783942+0000","last_became_active":"2024-03-28T13:10:55.754344+0000","last_became_peered":"2024-03-28T13:10:55.754344+0000","last_unstale":"2024-03-28T13:10:57.783942+0000","last_undegraded":"2024-03-28T13:10:57.783942+0000","last_fullsized":"2024-03-28T13:10:57.783942+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:22:12.385800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777982+0000","last_change":"2024-03-28T13:10:56.779261+0000","last_active":"2024-03-28T13:10:57.777982+0000","last_peered":"2024-03-28T13:10:57.777982+0000","last_clean":"2024-03-28T13:10:57.777982+0000","last_became_active":"2024-03-28T13:10:56.778899+0000","last_became_peered":"2024-03-28T13:10:56.778899+0000","last_unstale":"2024-03-28T13:10:57.777982+0000","last_undegraded":"2024-03-28T13:10:57.777982+0000","last_fullsized":"2024-03-28T13:10:57.777982+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:44:06.086773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777813+0000","last_change":"2024-03-28T13:10:55.755603+0000","last_active":"2024-03-28T13:10:57.777813+0000","last_peered":"2024-03-28T13:10:57.777813+0000","last_clean":"2024-03-28T13:10:57.777813+0000","last_became_active":"2024-03-28T13:10:55.754244+0000","last_became_peered":"2024-03-28T13:10:55.754244+0000","last_unstale":"2024-03-28T13:10:57.777813+0000","last_undegraded":"2024-03-28T13:10:57.777813+0000","last_fullsized":"2024-03-28T13:10:57.777813+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:30.977315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:54.697274+0000","last_change":"2024-03-28T13:10:54.697274+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.697274+0000","last_undegraded":"2024-03-28T13:10:54.697274+0000","last_fullsized":"2024-03-28T13:10:54.697274+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:54.697274+0000","last_change":"2024-03-28T13:10:54.697274+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.697274+0000","last_undegraded":"2024-03-28T13:10:54.697274+0000","last_fullsized":"2024-03-28T13:10:54.697274+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783436+0000","last_change":"2024-03-28T13:10:55.750960+0000","last_active":"2024-03-28T13:10:57.783436+0000","last_peered":"2024-03-28T13:10:57.783436+0000","last_clean":"2024-03-28T13:10:57.783436+0000","last_became_active":"2024-03-28T13:10:55.750618+0000","last_became_peered":"2024-03-28T13:10:55.750618+0000","last_unstale":"2024-03-28T13:10:57.783436+0000","last_undegraded":"2024-03-28T13:10:57.783436+0000","last_fullsized":"2024-03-28T13:10:57.783436+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:36.018433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776802+0000","last_change":"2024-03-28T13:10:56.775408+0000","last_active":"2024-03-28T13:10:57.776802+0000","last_peered":"2024-03-28T13:10:57.776802+0000","last_clean":"2024-03-28T13:10:57.776802+0000","last_became_active":"2024-03-28T13:10:56.775141+0000","last_became_peered":"2024-03-28T13:10:56.775141+0000","last_unstale":"2024-03-28T13:10:57.776802+0000","last_undegraded":"2024-03-28T13:10:57.776802+0000","last_fullsized":"2024-03-28T13:10:57.776802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:54.697274+0000","last_change":"2024-03-28T13:10:54.697274+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.697274+0000","last_undegraded":"2024-03-28T13:10:54.697274+0000","last_fullsized":"2024-03-28T13:10:54.697274+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.775253+0000","last_change":"2024-03-28T13:10:55.741344+0000","last_active":"2024-03-28T13:10:57.775253+0000","last_peered":"2024-03-28T13:10:57.775253+0000","last_clean":"2024-03-28T13:10:57.775253+0000","last_became_active":"2024-03-28T13:10:55.740955+0000","last_became_peered":"2024-03-28T13:10:55.740955+0000","last_unstale":"2024-03-28T13:10:57.775253+0000","last_undegraded":"2024-03-28T13:10:57.775253+0000","last_fullsized":"2024-03-28T13:10:57.775253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:55:33.765625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778342+0000","last_change":"2024-03-28T13:10:56.792475+0000","last_active":"2024-03-28T13:10:57.778342+0000","last_peered":"2024-03-28T13:10:57.778342+0000","last_clean":"2024-03-28T13:10:57.778342+0000","last_became_active":"2024-03-28T13:10:56.792165+0000","last_became_peered":"2024-03-28T13:10:56.792165+0000","last_unstale":"2024-03-28T13:10:57.778342+0000","last_undegraded":"2024-03-28T13:10:57.778342+0000","last_fullsized":"2024-03-28T13:10:57.778342+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:50:07.655844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774840+0000","last_change":"2024-03-28T13:10:55.749119+0000","last_active":"2024-03-28T13:10:57.774840+0000","last_peered":"2024-03-28T13:10:57.774840+0000","last_clean":"2024-03-28T13:10:57.774840+0000","last_became_active":"2024-03-28T13:10:55.748527+0000","last_became_peered":"2024-03-28T13:10:55.748527+0000","last_unstale":"2024-03-28T13:10:57.774840+0000","last_undegraded":"2024-03-28T13:10:57.774840+0000","last_fullsized":"2024-03-28T13:10:57.774840+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:30.977315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777568+0000","last_change":"2024-03-28T13:10:56.778342+0000","last_active":"2024-03-28T13:10:57.777568+0000","last_peered":"2024-03-28T13:10:57.777568+0000","last_clean":"2024-03-28T13:10:57.777568+0000","last_became_active":"2024-03-28T13:10:56.777259+0000","last_became_peered":"2024-03-28T13:10:56.777259+0000","last_unstale":"2024-03-28T13:10:57.777568+0000","last_undegraded":"2024-03-28T13:10:57.777568+0000","last_fullsized":"2024-03-28T13:10:57.777568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:23.920150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-28T13:10:54.716903+0000","last_change":"2024-03-28T13:10:54.716903+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.716903+0000","last_undegraded":"2024-03-28T13:10:54.716903+0000","last_fullsized":"2024-03-28T13:10:54.716903+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.804651+0000","last_change":"2024-03-28T13:10:55.757421+0000","last_active":"2024-03-28T13:10:56.804651+0000","last_peered":"2024-03-28T13:10:56.804651+0000","last_clean":"2024-03-28T13:10:56.804651+0000","last_became_active":"2024-03-28T13:10:55.757041+0000","last_became_peered":"2024-03-28T13:10:55.757041+0000","last_unstale":"2024-03-28T13:10:56.804651+0000","last_undegraded":"2024-03-28T13:10:56.804651+0000","last_fullsized":"2024-03-28T13:10:56.804651+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:46.007430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.777866+0000","last_change":"2024-03-28T13:10:56.767158+0000","last_active":"2024-03-28T13:10:56.777866+0000","last_peered":"2024-03-28T13:10:56.777866+0000","last_clean":"2024-03-28T13:10:56.777866+0000","last_became_active":"2024-03-28T13:10:56.762220+0000","last_became_peered":"2024-03-28T13:10:56.762220+0000","last_unstale":"2024-03-28T13:10:56.777866+0000","last_undegraded":"2024-03-28T13:10:56.777866+0000","last_fullsized":"2024-03-28T13:10:56.777866+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:36:29.152136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:54.697274+0000","last_change":"2024-03-28T13:10:54.697274+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.697274+0000","last_undegraded":"2024-03-28T13:10:54.697274+0000","last_fullsized":"2024-03-28T13:10:54.697274+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777175+0000","last_change":"2024-03-28T13:10:56.791145+0000","last_active":"2024-03-28T13:10:57.777175+0000","last_peered":"2024-03-28T13:10:57.777175+0000","last_clean":"2024-03-28T13:10:57.777175+0000","last_became_active":"2024-03-28T13:10:56.789400+0000","last_became_peered":"2024-03-28T13:10:56.789400+0000","last_unstale":"2024-03-28T13:10:57.777175+0000","last_undegraded":"2024-03-28T13:10:57.777175+0000","last_fullsized":"2024-03-28T13:10:57.777175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:26.233670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777676+0000","last_change":"2024-03-28T13:10:55.750434+0000","last_active":"2024-03-28T13:10:57.777676+0000","last_peered":"2024-03-28T13:10:57.777676+0000","last_clean":"2024-03-28T13:10:57.777676+0000","last_became_active":"2024-03-28T13:10:55.750086+0000","last_became_peered":"2024-03-28T13:10:55.750086+0000","last_unstale":"2024-03-28T13:10:57.777676+0000","last_undegraded":"2024-03-28T13:10:57.777676+0000","last_fullsized":"2024-03-28T13:10:57.777676+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:37:32.623408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776667+0000","last_change":"2024-03-28T13:10:55.755962+0000","last_active":"2024-03-28T13:10:57.776667+0000","last_peered":"2024-03-28T13:10:57.776667+0000","last_clean":"2024-03-28T13:10:57.776667+0000","last_became_active":"2024-03-28T13:10:55.755652+0000","last_became_peered":"2024-03-28T13:10:55.755652+0000","last_unstale":"2024-03-28T13:10:57.776667+0000","last_undegraded":"2024-03-28T13:10:57.776667+0000","last_fullsized":"2024-03-28T13:10:57.776667+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:51:08.557038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783479+0000","last_change":"2024-03-28T13:10:55.757692+0000","last_active":"2024-03-28T13:10:57.783479+0000","last_peered":"2024-03-28T13:10:57.783479+0000","last_clean":"2024-03-28T13:10:57.783479+0000","last_became_active":"2024-03-28T13:10:55.757463+0000","last_became_peered":"2024-03-28T13:10:55.757463+0000","last_unstale":"2024-03-28T13:10:57.783479+0000","last_undegraded":"2024-03-28T13:10:57.783479+0000","last_fullsized":"2024-03-28T13:10:57.783479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:12:56.307494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783676+0000","last_change":"2024-03-28T13:10:55.757407+0000","last_active":"2024-03-28T13:10:57.783676+0000","last_peered":"2024-03-28T13:10:57.783676+0000","last_clean":"2024-03-28T13:10:57.783676+0000","last_became_active":"2024-03-28T13:10:55.757088+0000","last_became_peered":"2024-03-28T13:10:55.757088+0000","last_unstale":"2024-03-28T13:10:57.783676+0000","last_undegraded":"2024-03-28T13:10:57.783676+0000","last_fullsized":"2024-03-28T13:10:57.783676+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:03:35.637042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:54.697274+0000","last_change":"2024-03-28T13:10:54.697274+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.697274+0000","last_undegraded":"2024-03-28T13:10:54.697274+0000","last_fullsized":"2024-03-28T13:10:54.697274+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774512+0000","last_change":"2024-03-28T13:10:56.774113+0000","last_active":"2024-03-28T13:10:57.774512+0000","last_peered":"2024-03-28T13:10:57.774512+0000","last_clean":"2024-03-28T13:10:57.774512+0000","last_became_active":"2024-03-28T13:10:56.773838+0000","last_became_peered":"2024-03-28T13:10:56.773838+0000","last_unstale":"2024-03-28T13:10:57.774512+0000","last_undegraded":"2024-03-28T13:10:57.774512+0000","last_fullsized":"2024-03-28T13:10:57.774512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:45.087620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-28T13:10:54.717508+0000","last_change":"2024-03-28T13:10:54.717508+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.717508+0000","last_undegraded":"2024-03-28T13:10:54.717508+0000","last_fullsized":"2024-03-28T13:10:54.717508+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783690+0000","last_change":"2024-03-28T13:10:56.779040+0000","last_active":"2024-03-28T13:10:57.783690+0000","last_peered":"2024-03-28T13:10:57.783690+0000","last_clean":"2024-03-28T13:10:57.783690+0000","last_became_active":"2024-03-28T13:10:56.777765+0000","last_became_peered":"2024-03-28T13:10:56.777765+0000","last_unstale":"2024-03-28T13:10:57.783690+0000","last_undegraded":"2024-03-28T13:10:57.783690+0000","last_fullsized":"2024-03-28T13:10:57.783690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:44.685445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-28T13:10:54.721535+0000","last_change":"2024-03-28T13:10:54.721535+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.721535+0000","last_undegraded":"2024-03-28T13:10:54.721535+0000","last_fullsized":"2024-03-28T13:10:54.721535+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.781976+0000","last_change":"2024-03-28T13:10:56.774575+0000","last_active":"2024-03-28T13:10:56.781976+0000","last_peered":"2024-03-28T13:10:56.781976+0000","last_clean":"2024-03-28T13:10:56.781976+0000","last_became_active":"2024-03-28T13:10:56.772544+0000","last_became_peered":"2024-03-28T13:10:56.772544+0000","last_unstale":"2024-03-28T13:10:56.781976+0000","last_undegraded":"2024-03-28T13:10:56.781976+0000","last_fullsized":"2024-03-28T13:10:56.781976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:29:15.008724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.775058+0000","last_change":"2024-03-28T13:10:55.754015+0000","last_active":"2024-03-28T13:10:57.775058+0000","last_peered":"2024-03-28T13:10:57.775058+0000","last_clean":"2024-03-28T13:10:57.775058+0000","last_became_active":"2024-03-28T13:10:55.753701+0000","last_became_peered":"2024-03-28T13:10:55.753701+0000","last_unstale":"2024-03-28T13:10:57.775058+0000","last_undegraded":"2024-03-28T13:10:57.775058+0000","last_fullsized":"2024-03-28T13:10:57.775058+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:29:13.996522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.775071+0000","last_change":"2024-03-28T13:10:56.782871+0000","last_active":"2024-03-28T13:10:57.775071+0000","last_peered":"2024-03-28T13:10:57.775071+0000","last_clean":"2024-03-28T13:10:57.775071+0000","last_became_active":"2024-03-28T13:10:56.782020+0000","last_became_peered":"2024-03-28T13:10:56.782020+0000","last_unstale":"2024-03-28T13:10:57.775071+0000","last_undegraded":"2024-03-28T13:10:57.775071+0000","last_fullsized":"2024-03-28T13:10:57.775071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:54.697274+0000","last_change":"2024-03-28T13:10:54.697274+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.697274+0000","last_undegraded":"2024-03-28T13:10:54.697274+0000","last_fullsized":"2024-03-28T13:10:54.697274+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.787619+0000","last_change":"2024-03-28T13:10:56.780543+0000","last_active":"2024-03-28T13:10:56.787619+0000","last_peered":"2024-03-28T13:10:56.787619+0000","last_clean":"2024-03-28T13:10:56.787619+0000","last_became_active":"2024-03-28T13:10:56.780189+0000","last_became_peered":"2024-03-28T13:10:56.780189+0000","last_unstale":"2024-03-28T13:10:56.787619+0000","last_undegraded":"2024-03-28T13:10:56.787619+0000","last_fullsized":"2024-03-28T13:10:56.787619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:31.989516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-28T13:10:54.720221+0000","last_change":"2024-03-28T13:10:54.720221+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.720221+0000","last_undegraded":"2024-03-28T13:10:54.720221+0000","last_fullsized":"2024-03-28T13:10:54.720221+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-28T13:10:54.715957+0000","last_change":"2024-03-28T13:10:54.715957+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.715957+0000","last_undegraded":"2024-03-28T13:10:54.715957+0000","last_fullsized":"2024-03-28T13:10:54.715957+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783615+0000","last_change":"2024-03-28T13:10:56.775289+0000","last_active":"2024-03-28T13:10:57.783615+0000","last_peered":"2024-03-28T13:10:57.783615+0000","last_clean":"2024-03-28T13:10:57.783615+0000","last_became_active":"2024-03-28T13:10:56.774549+0000","last_became_peered":"2024-03-28T13:10:56.774549+0000","last_unstale":"2024-03-28T13:10:57.783615+0000","last_undegraded":"2024-03-28T13:10:57.783615+0000","last_fullsized":"2024-03-28T13:10:57.783615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:16.475067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.794905+0000","last_change":"2024-03-28T13:10:55.755495+0000","last_active":"2024-03-28T13:10:56.794905+0000","last_peered":"2024-03-28T13:10:56.794905+0000","last_clean":"2024-03-28T13:10:56.794905+0000","last_became_active":"2024-03-28T13:10:55.754987+0000","last_became_peered":"2024-03-28T13:10:55.754987+0000","last_unstale":"2024-03-28T13:10:56.794905+0000","last_undegraded":"2024-03-28T13:10:56.794905+0000","last_fullsized":"2024-03-28T13:10:56.794905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:57:31.432582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783204+0000","last_change":"2024-03-28T13:10:55.748821+0000","last_active":"2024-03-28T13:10:57.783204+0000","last_peered":"2024-03-28T13:10:57.783204+0000","last_clean":"2024-03-28T13:10:57.783204+0000","last_became_active":"2024-03-28T13:10:55.748128+0000","last_became_peered":"2024-03-28T13:10:55.748128+0000","last_unstale":"2024-03-28T13:10:57.783204+0000","last_undegraded":"2024-03-28T13:10:57.783204+0000","last_fullsized":"2024-03-28T13:10:57.783204+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:48:41.782400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783822+0000","last_change":"2024-03-28T13:10:56.785728+0000","last_active":"2024-03-28T13:10:57.783822+0000","last_peered":"2024-03-28T13:10:57.783822+0000","last_clean":"2024-03-28T13:10:57.783822+0000","last_became_active":"2024-03-28T13:10:56.783504+0000","last_became_peered":"2024-03-28T13:10:56.783504+0000","last_unstale":"2024-03-28T13:10:57.783822+0000","last_undegraded":"2024-03-28T13:10:57.783822+0000","last_fullsized":"2024-03-28T13:10:57.783822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:42:20.735994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.747889+0000","last_change":"2024-03-28T13:10:55.757201+0000","last_active":"2024-03-28T13:10:56.747889+0000","last_peered":"2024-03-28T13:10:56.747889+0000","last_clean":"2024-03-28T13:10:56.747889+0000","last_became_active":"2024-03-28T13:10:55.756328+0000","last_became_peered":"2024-03-28T13:10:55.756328+0000","last_unstale":"2024-03-28T13:10:56.747889+0000","last_undegraded":"2024-03-28T13:10:56.747889+0000","last_fullsized":"2024-03-28T13:10:56.747889+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:50:09.927656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.747164+0000","last_change":"2024-03-28T13:10:55.753645+0000","last_active":"2024-03-28T13:10:56.747164+0000","last_peered":"2024-03-28T13:10:56.747164+0000","last_clean":"2024-03-28T13:10:56.747164+0000","last_became_active":"2024-03-28T13:10:55.753103+0000","last_became_peered":"2024-03-28T13:10:55.753103+0000","last_unstale":"2024-03-28T13:10:56.747164+0000","last_undegraded":"2024-03-28T13:10:56.747164+0000","last_fullsized":"2024-03-28T13:10:56.747164+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:38:25.780963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777482+0000","last_change":"2024-03-28T13:10:56.784216+0000","last_active":"2024-03-28T13:10:57.777482+0000","last_peered":"2024-03-28T13:10:57.777482+0000","last_clean":"2024-03-28T13:10:57.777482+0000","last_became_active":"2024-03-28T13:10:56.783936+0000","last_became_peered":"2024-03-28T13:10:56.783936+0000","last_unstale":"2024-03-28T13:10:57.777482+0000","last_undegraded":"2024-03-28T13:10:57.777482+0000","last_fullsized":"2024-03-28T13:10:57.777482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:53.479188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.748281+0000","last_change":"2024-03-28T13:10:55.754934+0000","last_active":"2024-03-28T13:10:56.748281+0000","last_peered":"2024-03-28T13:10:56.748281+0000","last_clean":"2024-03-28T13:10:56.748281+0000","last_became_active":"2024-03-28T13:10:55.754215+0000","last_became_peered":"2024-03-28T13:10:55.754215+0000","last_unstale":"2024-03-28T13:10:56.748281+0000","last_undegraded":"2024-03-28T13:10:56.748281+0000","last_fullsized":"2024-03-28T13:10:56.748281+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:51:08.557038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776768+0000","last_change":"2024-03-28T13:10:56.791043+0000","last_active":"2024-03-28T13:10:57.776768+0000","last_peered":"2024-03-28T13:10:57.776768+0000","last_clean":"2024-03-28T13:10:57.776768+0000","last_became_active":"2024-03-28T13:10:56.787198+0000","last_became_peered":"2024-03-28T13:10:56.787198+0000","last_unstale":"2024-03-28T13:10:57.776768+0000","last_undegraded":"2024-03-28T13:10:57.776768+0000","last_fullsized":"2024-03-28T13:10:57.776768+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:54.697274+0000","last_change":"2024-03-28T13:10:54.697274+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.697274+0000","last_undegraded":"2024-03-28T13:10:54.697274+0000","last_fullsized":"2024-03-28T13:10:54.697274+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776675+0000","last_change":"2024-03-28T13:10:55.746007+0000","last_active":"2024-03-28T13:10:57.776675+0000","last_peered":"2024-03-28T13:10:57.776675+0000","last_clean":"2024-03-28T13:10:57.776675+0000","last_became_active":"2024-03-28T13:10:55.745666+0000","last_became_peered":"2024-03-28T13:10:55.745666+0000","last_unstale":"2024-03-28T13:10:57.776675+0000","last_undegraded":"2024-03-28T13:10:57.776675+0000","last_fullsized":"2024-03-28T13:10:57.776675+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:36:28.139935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:10:55.709476+0000","last_change":"2024-03-28T13:10:55.709476+0000","last_active":"2024-03-28T13:10:55.709476+0000","last_peered":"2024-03-28T13:10:55.709476+0000","last_clean":"2024-03-28T13:10:55.709476+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:55.709476+0000","last_undegraded":"2024-03-28T13:10:55.709476+0000","last_fullsized":"2024-03-28T13:10:55.709476+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-28T13:10:54.715991+0000","last_change":"2024-03-28T13:10:54.715991+0000","last_active":"2024-03-28T13:10:54.697274+0000","last_peered":"2024-03-28T13:10:54.697274+0000","last_clean":"2024-03-28T13:10:54.697274+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:10:54.715991+0000","last_undegraded":"2024-03-28T13:10:54.715991+0000","last_fullsized":"2024-03-28T13:10:54.715991+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":188,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-28T13:10:54.721334+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:10:54.721334+0000","last_peered":"2024-03-28T13:10:54.721334+0000","last_clean":"2024-03-28T13:10:54.721334+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:10:54.721334+0000","last_undegraded":"2024-03-28T13:10:54.721334+0000","last_fullsized":"2024-03-28T13:10:54.721334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":60,"acting":60,"num_store_stats":4},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":50,"acting":50,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476759,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":348160,"data_stored":130018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476757,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27364,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":348160,"data_stored":130018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476757,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":720386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476757,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":130018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":397312,"data_stored":150497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":746135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":156819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":155207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:11:01.798 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:11:02.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.060+0000 7f1f20a00640 1 Processor -- start 2024-03-28T13:11:02.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.060+0000 7f1f20a00640 1 -- start start 2024-03-28T13:11:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.062+0000 7f1f20a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1f1c15fc50 0x7f1f1c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.062+0000 7f1f20a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f1c0cc490 0x7f1f1c0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.062+0000 7f1f20a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f1c0b3150 con 0x7f1f1c0cc0b0 2024-03-28T13:11:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.062+0000 7f1f20a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1f1c15c610 con 0x7f1f1c15fc50 2024-03-28T13:11:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.062+0000 7f1f20a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1f1c1600e0 con 0x7f1f1c0cc490 2024-03-28T13:11:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.062+0000 7f1f19e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1f1c15fc50 0x7f1f1c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.062+0000 7f1f1a800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1f1c0cc0b0 0x7f1f1c06dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35714/0 (socket says 172.21.15.19:35714) 2024-03-28T13:11:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.062+0000 7f1f1a800640 1 -- 172.21.15.19:0/749224426 learned_addr learned my addr 172.21.15.19:0/749224426 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:02.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.062+0000 7f1f1b200640 1 --2- 172.21.15.19:0/749224426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f1c0cc490 0x7f1f1c0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:02.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.063+0000 7f1f19400640 1 -- 172.21.15.19:0/749224426 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4013655720 0 0) 0x7f1f1c0b3150 con 0x7f1f1c0cc0b0 2024-03-28T13:11:02.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.063+0000 7f1f19400640 1 -- 172.21.15.19:0/749224426 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f00003520 con 0x7f1f1c0cc0b0 2024-03-28T13:11:02.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.063+0000 7f1f19e00640 1 -- 172.21.15.19:0/749224426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f1c0cc490 msgr2=0x7f1f1c0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:02.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.063+0000 7f1f19e00640 1 --2- 172.21.15.19:0/749224426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f1c0cc490 0x7f1f1c0d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.063+0000 7f1f19e00640 1 -- 172.21.15.19:0/749224426 >> v1:172.21.15.19:6789/0 conn(0x7f1f1c0cc0b0 legacy=0x7f1f1c06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:02.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.063+0000 7f1f19e00640 1 -- 172.21.15.19:0/749224426 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1c1603c0 con 0x7f1f1c15fc50 2024-03-28T13:11:02.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.064+0000 7f1f19e00640 1 --2- 172.21.15.19:0/749224426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f1c15fc50 0x7f1f1c0cfdc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1f080097a0 tx=0x7f1f0802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eee37da5771560a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.064+0000 7f1f19400640 1 -- 172.21.15.19:0/749224426 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08035020 con 0x7f1f1c15fc50 2024-03-28T13:11:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.064+0000 7f1f19400640 1 -- 172.21.15.19:0/749224426 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f08031830 con 0x7f1f1c15fc50 2024-03-28T13:11:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.064+0000 7f1f19400640 1 -- 172.21.15.19:0/749224426 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08031c00 con 0x7f1f1c15fc50 2024-03-28T13:11:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.065+0000 7f1f20a00640 1 -- 172.21.15.19:0/749224426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f1c15fc50 msgr2=0x7f1f1c0cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.065+0000 7f1f20a00640 1 --2- 172.21.15.19:0/749224426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f1c15fc50 0x7f1f1c0cfdc0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1f080097a0 tx=0x7f1f0802b8d0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.065+0000 7f1f20a00640 1 -- 172.21.15.19:0/749224426 shutdown_connections 2024-03-28T13:11:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.065+0000 7f1f20a00640 1 --2- 172.21.15.19:0/749224426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f1c0cc490 0x7f1f1c0d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.065+0000 7f1f20a00640 1 --2- 172.21.15.19:0/749224426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f1c15fc50 0x7f1f1c0cfdc0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.065+0000 7f1f20a00640 1 -- 172.21.15.19:0/749224426 >> 172.21.15.19:0/749224426 conn(0x7f1f1c0ccad0 msgr2=0x7f1f1c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:02.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.065+0000 7f1f20a00640 1 -- 172.21.15.19:0/749224426 shutdown_connections 2024-03-28T13:11:02.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.065+0000 7f1f20a00640 1 -- 172.21.15.19:0/749224426 wait complete. 2024-03-28T13:11:02.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.067+0000 7f1f20a00640 1 Processor -- start 2024-03-28T13:11:02.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.067+0000 7f1f20a00640 1 -- start start 2024-03-28T13:11:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.068+0000 7f1f20a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f1c0cc0b0 0x7f1f1c155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.068+0000 7f1f20a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f1c0cc490 0x7f1f1c12b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.068+0000 7f1f20a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f1c12ccb0 0x7f1f1c12d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.068+0000 7f1f20a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c15ca70 con 0x7f1f1c0cc490 2024-03-28T13:11:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.068+0000 7f1f20a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1f1c160980 con 0x7f1f1c12ccb0 2024-03-28T13:11:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.068+0000 7f1f20a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1f1c15cbf0 con 0x7f1f1c0cc0b0 2024-03-28T13:11:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.068+0000 7f1f19e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f1c0cc490 0x7f1f1c12b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.068+0000 7f1f19e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f1c0cc490 0x7f1f1c12b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43520/0 (socket says 172.21.15.19:43520) 2024-03-28T13:11:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.068+0000 7f1f19e00640 1 -- 172.21.15.19:0/3205034073 learned_addr learned my addr 172.21.15.19:0/3205034073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.069+0000 7f1f1a800640 1 --2- 172.21.15.19:0/3205034073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f1c0cc0b0 0x7f1f1c155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:02.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.069+0000 7f1f1b200640 1 --2- 172.21.15.19:0/3205034073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f1c12ccb0 0x7f1f1c12d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.069+0000 7f1f19e00640 1 -- 172.21.15.19:0/3205034073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f1c0cc0b0 msgr2=0x7f1f1c155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.069+0000 7f1f19e00640 1 --2- 172.21.15.19:0/3205034073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f1c0cc0b0 0x7f1f1c155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.069+0000 7f1f19e00640 1 -- 172.21.15.19:0/3205034073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f1c12ccb0 msgr2=0x7f1f1c12d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.069+0000 7f1f19e00640 1 --2- 172.21.15.19:0/3205034073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f1c12ccb0 0x7f1f1c12d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.069+0000 7f1f19e00640 1 -- 172.21.15.19:0/3205034073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f1c12bc10 con 0x7f1f1c0cc490 2024-03-28T13:11:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.069+0000 7f1f1a800640 1 --2- 172.21.15.19:0/3205034073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f1c0cc0b0 0x7f1f1c155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:02.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.069+0000 7f1f19e00640 1 --2- 172.21.15.19:0/3205034073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f1c0cc490 0x7f1f1c12b150 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1f08034040 tx=0x7f1f080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:02.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.069+0000 7f1f12a00640 1 -- 172.21.15.19:0/3205034073 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08040070 con 0x7f1f1c0cc490 2024-03-28T13:11:02.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.069+0000 7f1f12a00640 1 -- 172.21.15.19:0/3205034073 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f0803a910 con 0x7f1f1c0cc490 2024-03-28T13:11:02.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.069+0000 7f1f12a00640 1 -- 172.21.15.19:0/3205034073 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f08035040 con 0x7f1f1c0cc490 2024-03-28T13:11:02.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.069+0000 7f1f20a00640 1 -- 172.21.15.19:0/3205034073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f1c12bef0 con 0x7f1f1c0cc490 2024-03-28T13:11:02.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.070+0000 7f1f20a00640 1 -- 172.21.15.19:0/3205034073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f1c12c4b0 con 0x7f1f1c0cc490 2024-03-28T13:11:02.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.071+0000 7f1f20a00640 1 -- 172.21.15.19:0/3205034073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f1c075940 con 0x7f1f1c0cc490 2024-03-28T13:11:02.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.072+0000 7f1f12a00640 1 -- 172.21.15.19:0/3205034073 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1f08038020 con 0x7f1f1c0cc490 2024-03-28T13:11:02.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.073+0000 7f1f12a00640 1 --2- 172.21.15.19:0/3205034073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1efc06c530 0x7f1efc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:02.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.073+0000 7f1f12a00640 1 -- 172.21.15.19:0/3205034073 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f1f080ad1b0 con 0x7f1f1c0cc490 2024-03-28T13:11:02.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.073+0000 7f1f1a800640 1 --2- 172.21.15.19:0/3205034073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1efc06c530 0x7f1efc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:02.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.074+0000 7f1f1a800640 1 --2- 172.21.15.19:0/3205034073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1efc06c530 0x7f1efc06e9f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c005880 tx=0x7f1f0c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:02.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.078+0000 7f1f12a00640 1 -- 172.21.15.19:0/3205034073 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1f08031800 con 0x7f1f1c0cc490 2024-03-28T13:11:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.278+0000 7f1f20a00640 1 -- 172.21.15.19:0/3205034073 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1f1c06f2a0 con 0x7f1efc06c530 2024-03-28T13:11:02.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.297+0000 7f1f12a00640 1 -- 172.21.15.19:0/3205034073 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f1f1c06f2a0 con 0x7f1efc06c530 2024-03-28T13:11:02.299 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:11:02.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.303+0000 7f1f20a00640 1 -- 172.21.15.19:0/3205034073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1efc06c530 msgr2=0x7f1efc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:02.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.303+0000 7f1f20a00640 1 --2- 172.21.15.19:0/3205034073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1efc06c530 0x7f1efc06e9f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c005880 tx=0x7f1f0c009040 comp rx=0 tx=0).stop 2024-03-28T13:11:02.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.304+0000 7f1f20a00640 1 -- 172.21.15.19:0/3205034073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f1c0cc490 msgr2=0x7f1f1c12b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:02.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.304+0000 7f1f20a00640 1 --2- 172.21.15.19:0/3205034073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f1c0cc490 0x7f1f1c12b150 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1f08034040 tx=0x7f1f080046c0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.304+0000 7f1f20a00640 1 -- 172.21.15.19:0/3205034073 shutdown_connections 2024-03-28T13:11:02.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.304+0000 7f1f20a00640 1 --2- 172.21.15.19:0/3205034073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1efc06c530 0x7f1efc06e9f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.304+0000 7f1f20a00640 1 --2- 172.21.15.19:0/3205034073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1f1c12ccb0 0x7f1f1c12d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.304+0000 7f1f20a00640 1 --2- 172.21.15.19:0/3205034073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1f1c0cc490 0x7f1f1c12b150 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.304+0000 7f1f20a00640 1 --2- 172.21.15.19:0/3205034073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1f1c0cc0b0 0x7f1f1c155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.304+0000 7f1f20a00640 1 -- 172.21.15.19:0/3205034073 >> 172.21.15.19:0/3205034073 conn(0x7f1f1c0ccad0 msgr2=0x7f1f1c15db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:02.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.304+0000 7f1f20a00640 1 -- 172.21.15.19:0/3205034073 shutdown_connections 2024-03-28T13:11:02.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.304+0000 7f1f20a00640 1 -- 172.21.15.19:0/3205034073 wait complete. 2024-03-28T13:11:02.323 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-03-28T13:11:02.010322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220496,"kb_used_data":4316,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754224,"statfs":{"total":773094113280,"available":772868325376,"internally_reserved":0,"allocated":4419584,"data_stored":2420736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.290953"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771651+0000","last_change":"2024-03-28T13:10:56.775043+0000","last_active":"2024-03-28T13:10:57.771651+0000","last_peered":"2024-03-28T13:10:57.771651+0000","last_clean":"2024-03-28T13:10:57.771651+0000","last_became_active":"2024-03-28T13:10:56.774736+0000","last_became_peered":"2024-03-28T13:10:56.774736+0000","last_unstale":"2024-03-28T13:10:57.771651+0000","last_undegraded":"2024-03-28T13:10:57.771651+0000","last_fullsized":"2024-03-28T13:10:57.771651+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:26.233670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774657+0000","last_change":"2024-03-28T13:10:56.790536+0000","last_active":"2024-03-28T13:10:57.774657+0000","last_peered":"2024-03-28T13:10:57.774657+0000","last_clean":"2024-03-28T13:10:57.774657+0000","last_became_active":"2024-03-28T13:10:56.790180+0000","last_became_peered":"2024-03-28T13:10:56.790180+0000","last_unstale":"2024-03-28T13:10:57.774657+0000","last_undegraded":"2024-03-28T13:10:57.774657+0000","last_fullsized":"2024-03-28T13:10:57.774657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:06:36.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.367194+0000","last_change":"2024-03-28T13:10:56.774875+0000","last_active":"2024-03-28T13:10:59.367194+0000","last_peered":"2024-03-28T13:10:59.367194+0000","last_clean":"2024-03-28T13:10:59.367194+0000","last_became_active":"2024-03-28T13:10:56.774593+0000","last_became_peered":"2024-03-28T13:10:56.774593+0000","last_unstale":"2024-03-28T13:10:59.367194+0000","last_undegraded":"2024-03-28T13:10:59.367194+0000","last_fullsized":"2024-03-28T13:10:59.367194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:50:10.939857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777345+0000","last_change":"2024-03-28T13:10:56.777966+0000","last_active":"2024-03-28T13:10:57.777345+0000","last_peered":"2024-03-28T13:10:57.777345+0000","last_clean":"2024-03-28T13:10:57.777345+0000","last_became_active":"2024-03-28T13:10:56.777658+0000","last_became_peered":"2024-03-28T13:10:56.777658+0000","last_unstale":"2024-03-28T13:10:57.777345+0000","last_undegraded":"2024-03-28T13:10:57.777345+0000","last_fullsized":"2024-03-28T13:10:57.777345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:00:01.855413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774642+0000","last_change":"2024-03-28T13:10:56.783317+0000","last_active":"2024-03-28T13:10:57.774642+0000","last_peered":"2024-03-28T13:10:57.774642+0000","last_clean":"2024-03-28T13:10:57.774642+0000","last_became_active":"2024-03-28T13:10:56.781718+0000","last_became_peered":"2024-03-28T13:10:56.781718+0000","last_unstale":"2024-03-28T13:10:57.774642+0000","last_undegraded":"2024-03-28T13:10:57.774642+0000","last_fullsized":"2024-03-28T13:10:57.774642+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:58:13.880385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783325+0000","last_change":"2024-03-28T13:10:56.779357+0000","last_active":"2024-03-28T13:10:57.783325+0000","last_peered":"2024-03-28T13:10:57.783325+0000","last_clean":"2024-03-28T13:10:57.783325+0000","last_became_active":"2024-03-28T13:10:56.779000+0000","last_became_peered":"2024-03-28T13:10:56.779000+0000","last_unstale":"2024-03-28T13:10:57.783325+0000","last_undegraded":"2024-03-28T13:10:57.783325+0000","last_fullsized":"2024-03-28T13:10:57.783325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:40:07.650748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771017+0000","last_change":"2024-03-28T13:10:56.775965+0000","last_active":"2024-03-28T13:10:57.771017+0000","last_peered":"2024-03-28T13:10:57.771017+0000","last_clean":"2024-03-28T13:10:57.771017+0000","last_became_active":"2024-03-28T13:10:56.775684+0000","last_became_peered":"2024-03-28T13:10:56.775684+0000","last_unstale":"2024-03-28T13:10:57.771017+0000","last_undegraded":"2024-03-28T13:10:57.771017+0000","last_fullsized":"2024-03-28T13:10:57.771017+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:58:13.880385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771007+0000","last_change":"2024-03-28T13:10:56.780593+0000","last_active":"2024-03-28T13:10:57.771007+0000","last_peered":"2024-03-28T13:10:57.771007+0000","last_clean":"2024-03-28T13:10:57.771007+0000","last_became_active":"2024-03-28T13:10:56.780333+0000","last_became_peered":"2024-03-28T13:10:56.780333+0000","last_unstale":"2024-03-28T13:10:57.771007+0000","last_undegraded":"2024-03-28T13:10:57.771007+0000","last_fullsized":"2024-03-28T13:10:57.771007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:45.087620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.367199+0000","last_change":"2024-03-28T13:10:56.781115+0000","last_active":"2024-03-28T13:10:59.367199+0000","last_peered":"2024-03-28T13:10:59.367199+0000","last_clean":"2024-03-28T13:10:59.367199+0000","last_became_active":"2024-03-28T13:10:56.780578+0000","last_became_peered":"2024-03-28T13:10:56.780578+0000","last_unstale":"2024-03-28T13:10:59.367199+0000","last_undegraded":"2024-03-28T13:10:59.367199+0000","last_fullsized":"2024-03-28T13:10:59.367199+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:47.019632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778583+0000","last_change":"2024-03-28T13:10:56.793373+0000","last_active":"2024-03-28T13:10:57.778583+0000","last_peered":"2024-03-28T13:10:57.778583+0000","last_clean":"2024-03-28T13:10:57.778583+0000","last_became_active":"2024-03-28T13:10:56.789448+0000","last_became_peered":"2024-03-28T13:10:56.789448+0000","last_unstale":"2024-03-28T13:10:57.778583+0000","last_undegraded":"2024-03-28T13:10:57.778583+0000","last_fullsized":"2024-03-28T13:10:57.778583+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:37.170463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265827+0000","last_change":"2024-03-28T13:10:56.785075+0000","last_active":"2024-03-28T13:10:58.265827+0000","last_peered":"2024-03-28T13:10:58.265827+0000","last_clean":"2024-03-28T13:10:58.265827+0000","last_became_active":"2024-03-28T13:10:56.784486+0000","last_became_peered":"2024-03-28T13:10:56.784486+0000","last_unstale":"2024-03-28T13:10:58.265827+0000","last_undegraded":"2024-03-28T13:10:58.265827+0000","last_fullsized":"2024-03-28T13:10:58.265827+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:05:28.235846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.367058+0000","last_change":"2024-03-28T13:10:56.771454+0000","last_active":"2024-03-28T13:10:59.367058+0000","last_peered":"2024-03-28T13:10:59.367058+0000","last_clean":"2024-03-28T13:10:59.367058+0000","last_became_active":"2024-03-28T13:10:56.771111+0000","last_became_peered":"2024-03-28T13:10:56.771111+0000","last_unstale":"2024-03-28T13:10:59.367058+0000","last_undegraded":"2024-03-28T13:10:59.367058+0000","last_fullsized":"2024-03-28T13:10:59.367058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:38:26.793165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.787568+0000","last_change":"2024-03-28T13:10:56.780719+0000","last_active":"2024-03-28T13:10:56.787568+0000","last_peered":"2024-03-28T13:10:56.787568+0000","last_clean":"2024-03-28T13:10:56.787568+0000","last_became_active":"2024-03-28T13:10:56.780369+0000","last_became_peered":"2024-03-28T13:10:56.780369+0000","last_unstale":"2024-03-28T13:10:56.787568+0000","last_undegraded":"2024-03-28T13:10:56.787568+0000","last_fullsized":"2024-03-28T13:10:56.787568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:48:42.794602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777339+0000","last_change":"2024-03-28T13:10:56.774962+0000","last_active":"2024-03-28T13:10:57.777339+0000","last_peered":"2024-03-28T13:10:57.777339+0000","last_clean":"2024-03-28T13:10:57.777339+0000","last_became_active":"2024-03-28T13:10:56.774519+0000","last_became_peered":"2024-03-28T13:10:56.774519+0000","last_unstale":"2024-03-28T13:10:57.777339+0000","last_undegraded":"2024-03-28T13:10:57.777339+0000","last_fullsized":"2024-03-28T13:10:57.777339+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:40.900533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029554+0000","last_change":"2024-03-28T13:10:56.776344+0000","last_active":"2024-03-28T13:10:58.029554+0000","last_peered":"2024-03-28T13:10:58.029554+0000","last_clean":"2024-03-28T13:10:58.029554+0000","last_became_active":"2024-03-28T13:10:56.776013+0000","last_became_peered":"2024-03-28T13:10:56.776013+0000","last_unstale":"2024-03-28T13:10:58.029554+0000","last_undegraded":"2024-03-28T13:10:58.029554+0000","last_fullsized":"2024-03-28T13:10:58.029554+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:26.233670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029127+0000","last_change":"2024-03-28T13:10:56.787487+0000","last_active":"2024-03-28T13:10:58.029127+0000","last_peered":"2024-03-28T13:10:58.029127+0000","last_clean":"2024-03-28T13:10:58.029127+0000","last_became_active":"2024-03-28T13:10:56.787012+0000","last_became_peered":"2024-03-28T13:10:56.787012+0000","last_unstale":"2024-03-28T13:10:58.029127+0000","last_undegraded":"2024-03-28T13:10:58.029127+0000","last_fullsized":"2024-03-28T13:10:58.029127+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:06:36.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265751+0000","last_change":"2024-03-28T13:10:56.778750+0000","last_active":"2024-03-28T13:10:58.265751+0000","last_peered":"2024-03-28T13:10:58.265751+0000","last_clean":"2024-03-28T13:10:58.265751+0000","last_became_active":"2024-03-28T13:10:56.778397+0000","last_became_peered":"2024-03-28T13:10:56.778397+0000","last_unstale":"2024-03-28T13:10:58.265751+0000","last_undegraded":"2024-03-28T13:10:58.265751+0000","last_fullsized":"2024-03-28T13:10:58.265751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:09:21.093205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783603+0000","last_change":"2024-03-28T13:10:56.769141+0000","last_active":"2024-03-28T13:10:57.783603+0000","last_peered":"2024-03-28T13:10:57.783603+0000","last_clean":"2024-03-28T13:10:57.783603+0000","last_became_active":"2024-03-28T13:10:56.768873+0000","last_became_peered":"2024-03-28T13:10:56.768873+0000","last_unstale":"2024-03-28T13:10:57.783603+0000","last_undegraded":"2024-03-28T13:10:57.783603+0000","last_fullsized":"2024-03-28T13:10:57.783603+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:30:13.251989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783701+0000","last_change":"2024-03-28T13:10:56.767685+0000","last_active":"2024-03-28T13:10:57.783701+0000","last_peered":"2024-03-28T13:10:57.783701+0000","last_clean":"2024-03-28T13:10:57.783701+0000","last_became_active":"2024-03-28T13:10:56.767320+0000","last_became_peered":"2024-03-28T13:10:56.767320+0000","last_unstale":"2024-03-28T13:10:57.783701+0000","last_undegraded":"2024-03-28T13:10:57.783701+0000","last_fullsized":"2024-03-28T13:10:57.783701+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:45:22.482141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.782025+0000","last_change":"2024-03-28T13:10:56.772701+0000","last_active":"2024-03-28T13:10:56.782025+0000","last_peered":"2024-03-28T13:10:56.782025+0000","last_clean":"2024-03-28T13:10:56.782025+0000","last_became_active":"2024-03-28T13:10:56.772325+0000","last_became_peered":"2024-03-28T13:10:56.772325+0000","last_unstale":"2024-03-28T13:10:56.782025+0000","last_undegraded":"2024-03-28T13:10:56.782025+0000","last_fullsized":"2024-03-28T13:10:56.782025+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:19:57.308914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029451+0000","last_change":"2024-03-28T13:10:56.785391+0000","last_active":"2024-03-28T13:10:58.029451+0000","last_peered":"2024-03-28T13:10:58.029451+0000","last_clean":"2024-03-28T13:10:58.029451+0000","last_became_active":"2024-03-28T13:10:56.782726+0000","last_became_peered":"2024-03-28T13:10:56.782726+0000","last_unstale":"2024-03-28T13:10:58.029451+0000","last_undegraded":"2024-03-28T13:10:58.029451+0000","last_fullsized":"2024-03-28T13:10:58.029451+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.770494+0000","last_change":"2024-03-28T13:10:56.788744+0000","last_active":"2024-03-28T13:10:57.770494+0000","last_peered":"2024-03-28T13:10:57.770494+0000","last_clean":"2024-03-28T13:10:57.770494+0000","last_became_active":"2024-03-28T13:10:56.788421+0000","last_became_peered":"2024-03-28T13:10:56.788421+0000","last_unstale":"2024-03-28T13:10:57.770494+0000","last_undegraded":"2024-03-28T13:10:57.770494+0000","last_fullsized":"2024-03-28T13:10:57.770494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:37:53.859227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.779607+0000","last_change":"2024-03-28T13:10:56.768323+0000","last_active":"2024-03-28T13:10:56.779607+0000","last_peered":"2024-03-28T13:10:56.779607+0000","last_clean":"2024-03-28T13:10:56.779607+0000","last_became_active":"2024-03-28T13:10:56.766260+0000","last_became_peered":"2024-03-28T13:10:56.766260+0000","last_unstale":"2024-03-28T13:10:56.779607+0000","last_undegraded":"2024-03-28T13:10:56.779607+0000","last_fullsized":"2024-03-28T13:10:56.779607+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:55:35.027128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.770677+0000","last_change":"2024-03-28T13:10:56.789986+0000","last_active":"2024-03-28T13:10:57.770677+0000","last_peered":"2024-03-28T13:10:57.770677+0000","last_clean":"2024-03-28T13:10:57.770677+0000","last_became_active":"2024-03-28T13:10:56.789691+0000","last_became_peered":"2024-03-28T13:10:56.789691+0000","last_unstale":"2024-03-28T13:10:57.770677+0000","last_undegraded":"2024-03-28T13:10:57.770677+0000","last_fullsized":"2024-03-28T13:10:57.770677+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:52:10.929698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778158+0000","last_change":"2024-03-28T13:10:56.780679+0000","last_active":"2024-03-28T13:10:57.778158+0000","last_peered":"2024-03-28T13:10:57.778158+0000","last_clean":"2024-03-28T13:10:57.778158+0000","last_became_active":"2024-03-28T13:10:56.777813+0000","last_became_peered":"2024-03-28T13:10:56.777813+0000","last_unstale":"2024-03-28T13:10:57.778158+0000","last_undegraded":"2024-03-28T13:10:57.778158+0000","last_fullsized":"2024-03-28T13:10:57.778158+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:15:57.044511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778939+0000","last_change":"2024-03-28T13:10:56.781082+0000","last_active":"2024-03-28T13:10:57.778939+0000","last_peered":"2024-03-28T13:10:57.778939+0000","last_clean":"2024-03-28T13:10:57.778939+0000","last_became_active":"2024-03-28T13:10:56.780806+0000","last_became_peered":"2024-03-28T13:10:56.780806+0000","last_unstale":"2024-03-28T13:10:57.778939+0000","last_undegraded":"2024-03-28T13:10:57.778939+0000","last_fullsized":"2024-03-28T13:10:57.778939+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:33.779798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774874+0000","last_change":"2024-03-28T13:10:56.771825+0000","last_active":"2024-03-28T13:10:57.774874+0000","last_peered":"2024-03-28T13:10:57.774874+0000","last_clean":"2024-03-28T13:10:57.774874+0000","last_became_active":"2024-03-28T13:10:56.771505+0000","last_became_peered":"2024-03-28T13:10:56.771505+0000","last_unstale":"2024-03-28T13:10:57.774874+0000","last_undegraded":"2024-03-28T13:10:57.774874+0000","last_fullsized":"2024-03-28T13:10:57.774874+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771592+0000","last_change":"2024-03-28T13:10:56.784002+0000","last_active":"2024-03-28T13:10:57.771592+0000","last_peered":"2024-03-28T13:10:57.771592+0000","last_clean":"2024-03-28T13:10:57.771592+0000","last_became_active":"2024-03-28T13:10:56.783731+0000","last_became_peered":"2024-03-28T13:10:56.783731+0000","last_unstale":"2024-03-28T13:10:57.771592+0000","last_undegraded":"2024-03-28T13:10:57.771592+0000","last_fullsized":"2024-03-28T13:10:57.771592+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029694+0000","last_change":"2024-03-28T13:10:56.788313+0000","last_active":"2024-03-28T13:10:58.029694+0000","last_peered":"2024-03-28T13:10:58.029694+0000","last_clean":"2024-03-28T13:10:58.029694+0000","last_became_active":"2024-03-28T13:10:56.787982+0000","last_became_peered":"2024-03-28T13:10:56.787982+0000","last_unstale":"2024-03-28T13:10:58.029694+0000","last_undegraded":"2024-03-28T13:10:58.029694+0000","last_fullsized":"2024-03-28T13:10:58.029694+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:00:01.855413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029188+0000","last_change":"2024-03-28T13:10:56.779668+0000","last_active":"2024-03-28T13:10:58.029188+0000","last_peered":"2024-03-28T13:10:58.029188+0000","last_clean":"2024-03-28T13:10:58.029188+0000","last_became_active":"2024-03-28T13:10:56.778978+0000","last_became_peered":"2024-03-28T13:10:56.778978+0000","last_unstale":"2024-03-28T13:10:58.029188+0000","last_undegraded":"2024-03-28T13:10:58.029188+0000","last_fullsized":"2024-03-28T13:10:58.029188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.770758+0000","last_change":"2024-03-28T13:10:56.788094+0000","last_active":"2024-03-28T13:10:57.770758+0000","last_peered":"2024-03-28T13:10:57.770758+0000","last_clean":"2024-03-28T13:10:57.770758+0000","last_became_active":"2024-03-28T13:10:56.787812+0000","last_became_peered":"2024-03-28T13:10:56.787812+0000","last_unstale":"2024-03-28T13:10:57.770758+0000","last_undegraded":"2024-03-28T13:10:57.770758+0000","last_fullsized":"2024-03-28T13:10:57.770758+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:37:14.452246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265005+0000","last_change":"2024-03-28T13:10:56.771703+0000","last_active":"2024-03-28T13:10:58.265005+0000","last_peered":"2024-03-28T13:10:58.265005+0000","last_clean":"2024-03-28T13:10:58.265005+0000","last_became_active":"2024-03-28T13:10:56.769956+0000","last_became_peered":"2024-03-28T13:10:56.769956+0000","last_unstale":"2024-03-28T13:10:58.265005+0000","last_undegraded":"2024-03-28T13:10:58.265005+0000","last_fullsized":"2024-03-28T13:10:58.265005+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:09:25.565676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.778739+0000","last_change":"2024-03-28T13:10:56.769275+0000","last_active":"2024-03-28T13:10:56.778739+0000","last_peered":"2024-03-28T13:10:56.778739+0000","last_clean":"2024-03-28T13:10:56.778739+0000","last_became_active":"2024-03-28T13:10:56.767794+0000","last_became_peered":"2024-03-28T13:10:56.767794+0000","last_unstale":"2024-03-28T13:10:56.778739+0000","last_undegraded":"2024-03-28T13:10:56.778739+0000","last_fullsized":"2024-03-28T13:10:56.778739+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:37:33.635609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265751+0000","last_change":"2024-03-28T13:10:55.740050+0000","last_active":"2024-03-28T13:10:58.265751+0000","last_peered":"2024-03-28T13:10:58.265751+0000","last_clean":"2024-03-28T13:10:58.265751+0000","last_became_active":"2024-03-28T13:10:55.739731+0000","last_became_peered":"2024-03-28T13:10:55.739731+0000","last_unstale":"2024-03-28T13:10:58.265751+0000","last_undegraded":"2024-03-28T13:10:58.265751+0000","last_fullsized":"2024-03-28T13:10:58.265751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:22.907948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771663+0000","last_change":"2024-03-28T13:10:56.786361+0000","last_active":"2024-03-28T13:10:57.771663+0000","last_peered":"2024-03-28T13:10:57.771663+0000","last_clean":"2024-03-28T13:10:57.771663+0000","last_became_active":"2024-03-28T13:10:56.786029+0000","last_became_peered":"2024-03-28T13:10:56.786029+0000","last_unstale":"2024-03-28T13:10:57.771663+0000","last_undegraded":"2024-03-28T13:10:57.771663+0000","last_fullsized":"2024-03-28T13:10:57.771663+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:40.900533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.779205+0000","last_change":"2024-03-28T13:10:55.750470+0000","last_active":"2024-03-28T13:10:57.779205+0000","last_peered":"2024-03-28T13:10:57.779205+0000","last_clean":"2024-03-28T13:10:57.779205+0000","last_became_active":"2024-03-28T13:10:55.749676+0000","last_became_peered":"2024-03-28T13:10:55.749676+0000","last_unstale":"2024-03-28T13:10:57.779205+0000","last_undegraded":"2024-03-28T13:10:57.779205+0000","last_fullsized":"2024-03-28T13:10:57.779205+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:55:34.014926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771664+0000","last_change":"2024-03-28T13:10:56.793622+0000","last_active":"2024-03-28T13:10:57.771664+0000","last_peered":"2024-03-28T13:10:57.771664+0000","last_clean":"2024-03-28T13:10:57.771664+0000","last_became_active":"2024-03-28T13:10:56.793351+0000","last_became_peered":"2024-03-28T13:10:56.793351+0000","last_unstale":"2024-03-28T13:10:57.771664+0000","last_undegraded":"2024-03-28T13:10:57.771664+0000","last_fullsized":"2024-03-28T13:10:57.771664+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:00:01.855413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783942+0000","last_change":"2024-03-28T13:10:55.754683+0000","last_active":"2024-03-28T13:10:57.783942+0000","last_peered":"2024-03-28T13:10:57.783942+0000","last_clean":"2024-03-28T13:10:57.783942+0000","last_became_active":"2024-03-28T13:10:55.754344+0000","last_became_peered":"2024-03-28T13:10:55.754344+0000","last_unstale":"2024-03-28T13:10:57.783942+0000","last_undegraded":"2024-03-28T13:10:57.783942+0000","last_fullsized":"2024-03-28T13:10:57.783942+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:22:12.385800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777982+0000","last_change":"2024-03-28T13:10:56.779261+0000","last_active":"2024-03-28T13:10:57.777982+0000","last_peered":"2024-03-28T13:10:57.777982+0000","last_clean":"2024-03-28T13:10:57.777982+0000","last_became_active":"2024-03-28T13:10:56.778899+0000","last_became_peered":"2024-03-28T13:10:56.778899+0000","last_unstale":"2024-03-28T13:10:57.777982+0000","last_undegraded":"2024-03-28T13:10:57.777982+0000","last_fullsized":"2024-03-28T13:10:57.777982+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:44:06.086773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777813+0000","last_change":"2024-03-28T13:10:55.755603+0000","last_active":"2024-03-28T13:10:57.777813+0000","last_peered":"2024-03-28T13:10:57.777813+0000","last_clean":"2024-03-28T13:10:57.777813+0000","last_became_active":"2024-03-28T13:10:55.754244+0000","last_became_peered":"2024-03-28T13:10:55.754244+0000","last_unstale":"2024-03-28T13:10:57.777813+0000","last_undegraded":"2024-03-28T13:10:57.777813+0000","last_fullsized":"2024-03-28T13:10:57.777813+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:30.977315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366724+0000","last_change":"2024-03-28T13:10:56.776082+0000","last_active":"2024-03-28T13:10:59.366724+0000","last_peered":"2024-03-28T13:10:59.366724+0000","last_clean":"2024-03-28T13:10:59.366724+0000","last_became_active":"2024-03-28T13:10:56.775495+0000","last_became_peered":"2024-03-28T13:10:56.775495+0000","last_unstale":"2024-03-28T13:10:59.366724+0000","last_undegraded":"2024-03-28T13:10:59.366724+0000","last_fullsized":"2024-03-28T13:10:59.366724+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:39.445190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771537+0000","last_change":"2024-03-28T13:10:55.745761+0000","last_active":"2024-03-28T13:10:57.771537+0000","last_peered":"2024-03-28T13:10:57.771537+0000","last_clean":"2024-03-28T13:10:57.771537+0000","last_became_active":"2024-03-28T13:10:55.745492+0000","last_became_peered":"2024-03-28T13:10:55.745492+0000","last_unstale":"2024-03-28T13:10:57.771537+0000","last_undegraded":"2024-03-28T13:10:57.771537+0000","last_fullsized":"2024-03-28T13:10:57.771537+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:57:31.432582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771652+0000","last_change":"2024-03-28T13:10:56.783694+0000","last_active":"2024-03-28T13:10:57.771652+0000","last_peered":"2024-03-28T13:10:57.771652+0000","last_clean":"2024-03-28T13:10:57.771652+0000","last_became_active":"2024-03-28T13:10:56.783336+0000","last_became_peered":"2024-03-28T13:10:56.783336+0000","last_unstale":"2024-03-28T13:10:57.771652+0000","last_undegraded":"2024-03-28T13:10:57.771652+0000","last_fullsized":"2024-03-28T13:10:57.771652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.770706+0000","last_change":"2024-03-28T13:10:55.753085+0000","last_active":"2024-03-28T13:10:57.770706+0000","last_peered":"2024-03-28T13:10:57.770706+0000","last_clean":"2024-03-28T13:10:57.770706+0000","last_became_active":"2024-03-28T13:10:55.752806+0000","last_became_peered":"2024-03-28T13:10:55.752806+0000","last_unstale":"2024-03-28T13:10:57.770706+0000","last_undegraded":"2024-03-28T13:10:57.770706+0000","last_fullsized":"2024-03-28T13:10:57.770706+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:46.007430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366338+0000","last_change":"2024-03-28T13:10:56.769125+0000","last_active":"2024-03-28T13:10:59.366338+0000","last_peered":"2024-03-28T13:10:59.366338+0000","last_clean":"2024-03-28T13:10:59.366338+0000","last_became_active":"2024-03-28T13:10:56.768806+0000","last_became_peered":"2024-03-28T13:10:56.768806+0000","last_unstale":"2024-03-28T13:10:59.366338+0000","last_undegraded":"2024-03-28T13:10:59.366338+0000","last_fullsized":"2024-03-28T13:10:59.366338+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:57:20.895736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783436+0000","last_change":"2024-03-28T13:10:55.750960+0000","last_active":"2024-03-28T13:10:57.783436+0000","last_peered":"2024-03-28T13:10:57.783436+0000","last_clean":"2024-03-28T13:10:57.783436+0000","last_became_active":"2024-03-28T13:10:55.750618+0000","last_became_peered":"2024-03-28T13:10:55.750618+0000","last_unstale":"2024-03-28T13:10:57.783436+0000","last_undegraded":"2024-03-28T13:10:57.783436+0000","last_fullsized":"2024-03-28T13:10:57.783436+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:36.018433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776802+0000","last_change":"2024-03-28T13:10:56.775408+0000","last_active":"2024-03-28T13:10:57.776802+0000","last_peered":"2024-03-28T13:10:57.776802+0000","last_clean":"2024-03-28T13:10:57.776802+0000","last_became_active":"2024-03-28T13:10:56.775141+0000","last_became_peered":"2024-03-28T13:10:56.775141+0000","last_unstale":"2024-03-28T13:10:57.776802+0000","last_undegraded":"2024-03-28T13:10:57.776802+0000","last_fullsized":"2024-03-28T13:10:57.776802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771390+0000","last_change":"2024-03-28T13:10:55.752431+0000","last_active":"2024-03-28T13:10:57.771390+0000","last_peered":"2024-03-28T13:10:57.771390+0000","last_clean":"2024-03-28T13:10:57.771390+0000","last_became_active":"2024-03-28T13:10:55.752001+0000","last_became_peered":"2024-03-28T13:10:55.752001+0000","last_unstale":"2024-03-28T13:10:57.771390+0000","last_undegraded":"2024-03-28T13:10:57.771390+0000","last_fullsized":"2024-03-28T13:10:57.771390+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:38.432989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265800+0000","last_change":"2024-03-28T13:10:56.775017+0000","last_active":"2024-03-28T13:10:58.265800+0000","last_peered":"2024-03-28T13:10:58.265800+0000","last_clean":"2024-03-28T13:10:58.265800+0000","last_became_active":"2024-03-28T13:10:56.773930+0000","last_became_peered":"2024-03-28T13:10:56.773930+0000","last_unstale":"2024-03-28T13:10:58.265800+0000","last_undegraded":"2024-03-28T13:10:58.265800+0000","last_fullsized":"2024-03-28T13:10:58.265800+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:36:18.347550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.775253+0000","last_change":"2024-03-28T13:10:55.741344+0000","last_active":"2024-03-28T13:10:57.775253+0000","last_peered":"2024-03-28T13:10:57.775253+0000","last_clean":"2024-03-28T13:10:57.775253+0000","last_became_active":"2024-03-28T13:10:55.740955+0000","last_became_peered":"2024-03-28T13:10:55.740955+0000","last_unstale":"2024-03-28T13:10:57.775253+0000","last_undegraded":"2024-03-28T13:10:57.775253+0000","last_fullsized":"2024-03-28T13:10:57.775253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:55:33.765625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778342+0000","last_change":"2024-03-28T13:10:56.792475+0000","last_active":"2024-03-28T13:10:57.778342+0000","last_peered":"2024-03-28T13:10:57.778342+0000","last_clean":"2024-03-28T13:10:57.778342+0000","last_became_active":"2024-03-28T13:10:56.792165+0000","last_became_peered":"2024-03-28T13:10:56.792165+0000","last_unstale":"2024-03-28T13:10:57.778342+0000","last_undegraded":"2024-03-28T13:10:57.778342+0000","last_fullsized":"2024-03-28T13:10:57.778342+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:50:07.655844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774840+0000","last_change":"2024-03-28T13:10:55.749119+0000","last_active":"2024-03-28T13:10:57.774840+0000","last_peered":"2024-03-28T13:10:57.774840+0000","last_clean":"2024-03-28T13:10:57.774840+0000","last_became_active":"2024-03-28T13:10:55.748527+0000","last_became_peered":"2024-03-28T13:10:55.748527+0000","last_unstale":"2024-03-28T13:10:57.774840+0000","last_undegraded":"2024-03-28T13:10:57.774840+0000","last_fullsized":"2024-03-28T13:10:57.774840+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:30.977315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777568+0000","last_change":"2024-03-28T13:10:56.778342+0000","last_active":"2024-03-28T13:10:57.777568+0000","last_peered":"2024-03-28T13:10:57.777568+0000","last_clean":"2024-03-28T13:10:57.777568+0000","last_became_active":"2024-03-28T13:10:56.777259+0000","last_became_peered":"2024-03-28T13:10:56.777259+0000","last_unstale":"2024-03-28T13:10:57.777568+0000","last_undegraded":"2024-03-28T13:10:57.777568+0000","last_fullsized":"2024-03-28T13:10:57.777568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:23.920150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265970+0000","last_change":"2024-03-28T13:10:55.737368+0000","last_active":"2024-03-28T13:10:58.265970+0000","last_peered":"2024-03-28T13:10:58.265970+0000","last_clean":"2024-03-28T13:10:58.265970+0000","last_became_active":"2024-03-28T13:10:55.736922+0000","last_became_peered":"2024-03-28T13:10:55.736922+0000","last_unstale":"2024-03-28T13:10:58.265970+0000","last_undegraded":"2024-03-28T13:10:58.265970+0000","last_fullsized":"2024-03-28T13:10:58.265970+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:39.888331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265780+0000","last_change":"2024-03-28T13:10:56.785370+0000","last_active":"2024-03-28T13:10:58.265780+0000","last_peered":"2024-03-28T13:10:58.265780+0000","last_clean":"2024-03-28T13:10:58.265780+0000","last_became_active":"2024-03-28T13:10:56.784822+0000","last_became_peered":"2024-03-28T13:10:56.784822+0000","last_unstale":"2024-03-28T13:10:58.265780+0000","last_undegraded":"2024-03-28T13:10:58.265780+0000","last_fullsized":"2024-03-28T13:10:58.265780+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:04:04.193445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.804651+0000","last_change":"2024-03-28T13:10:55.757421+0000","last_active":"2024-03-28T13:10:56.804651+0000","last_peered":"2024-03-28T13:10:56.804651+0000","last_clean":"2024-03-28T13:10:56.804651+0000","last_became_active":"2024-03-28T13:10:55.757041+0000","last_became_peered":"2024-03-28T13:10:55.757041+0000","last_unstale":"2024-03-28T13:10:56.804651+0000","last_undegraded":"2024-03-28T13:10:56.804651+0000","last_fullsized":"2024-03-28T13:10:56.804651+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:46.007430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.777866+0000","last_change":"2024-03-28T13:10:56.767158+0000","last_active":"2024-03-28T13:10:56.777866+0000","last_peered":"2024-03-28T13:10:56.777866+0000","last_clean":"2024-03-28T13:10:56.777866+0000","last_became_active":"2024-03-28T13:10:56.762220+0000","last_became_peered":"2024-03-28T13:10:56.762220+0000","last_unstale":"2024-03-28T13:10:56.777866+0000","last_undegraded":"2024-03-28T13:10:56.777866+0000","last_fullsized":"2024-03-28T13:10:56.777866+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:36:29.152136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771783+0000","last_change":"2024-03-28T13:10:55.742612+0000","last_active":"2024-03-28T13:10:57.771783+0000","last_peered":"2024-03-28T13:10:57.771783+0000","last_clean":"2024-03-28T13:10:57.771783+0000","last_became_active":"2024-03-28T13:10:55.742344+0000","last_became_peered":"2024-03-28T13:10:55.742344+0000","last_unstale":"2024-03-28T13:10:57.771783+0000","last_undegraded":"2024-03-28T13:10:57.771783+0000","last_fullsized":"2024-03-28T13:10:57.771783+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:38:25.780963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777175+0000","last_change":"2024-03-28T13:10:56.791145+0000","last_active":"2024-03-28T13:10:57.777175+0000","last_peered":"2024-03-28T13:10:57.777175+0000","last_clean":"2024-03-28T13:10:57.777175+0000","last_became_active":"2024-03-28T13:10:56.789400+0000","last_became_peered":"2024-03-28T13:10:56.789400+0000","last_unstale":"2024-03-28T13:10:57.777175+0000","last_undegraded":"2024-03-28T13:10:57.777175+0000","last_fullsized":"2024-03-28T13:10:57.777175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:26.233670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777676+0000","last_change":"2024-03-28T13:10:55.750434+0000","last_active":"2024-03-28T13:10:57.777676+0000","last_peered":"2024-03-28T13:10:57.777676+0000","last_clean":"2024-03-28T13:10:57.777676+0000","last_became_active":"2024-03-28T13:10:55.750086+0000","last_became_peered":"2024-03-28T13:10:55.750086+0000","last_unstale":"2024-03-28T13:10:57.777676+0000","last_undegraded":"2024-03-28T13:10:57.777676+0000","last_fullsized":"2024-03-28T13:10:57.777676+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:37:32.623408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771101+0000","last_change":"2024-03-28T13:10:56.779238+0000","last_active":"2024-03-28T13:10:57.771101+0000","last_peered":"2024-03-28T13:10:57.771101+0000","last_clean":"2024-03-28T13:10:57.771101+0000","last_became_active":"2024-03-28T13:10:56.773883+0000","last_became_peered":"2024-03-28T13:10:56.773883+0000","last_unstale":"2024-03-28T13:10:57.771101+0000","last_undegraded":"2024-03-28T13:10:57.771101+0000","last_fullsized":"2024-03-28T13:10:57.771101+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:06:36.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776667+0000","last_change":"2024-03-28T13:10:55.755962+0000","last_active":"2024-03-28T13:10:57.776667+0000","last_peered":"2024-03-28T13:10:57.776667+0000","last_clean":"2024-03-28T13:10:57.776667+0000","last_became_active":"2024-03-28T13:10:55.755652+0000","last_became_peered":"2024-03-28T13:10:55.755652+0000","last_unstale":"2024-03-28T13:10:57.776667+0000","last_undegraded":"2024-03-28T13:10:57.776667+0000","last_fullsized":"2024-03-28T13:10:57.776667+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:51:08.557038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265334+0000","last_change":"2024-03-28T13:10:56.772096+0000","last_active":"2024-03-28T13:10:58.265334+0000","last_peered":"2024-03-28T13:10:58.265334+0000","last_clean":"2024-03-28T13:10:58.265334+0000","last_became_active":"2024-03-28T13:10:56.771758+0000","last_became_peered":"2024-03-28T13:10:56.771758+0000","last_unstale":"2024-03-28T13:10:58.265334+0000","last_undegraded":"2024-03-28T13:10:58.265334+0000","last_fullsized":"2024-03-28T13:10:58.265334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:31:34.008079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783479+0000","last_change":"2024-03-28T13:10:55.757692+0000","last_active":"2024-03-28T13:10:57.783479+0000","last_peered":"2024-03-28T13:10:57.783479+0000","last_clean":"2024-03-28T13:10:57.783479+0000","last_became_active":"2024-03-28T13:10:55.757463+0000","last_became_peered":"2024-03-28T13:10:55.757463+0000","last_unstale":"2024-03-28T13:10:57.783479+0000","last_undegraded":"2024-03-28T13:10:57.783479+0000","last_fullsized":"2024-03-28T13:10:57.783479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:12:56.307494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265756+0000","last_change":"2024-03-28T13:10:56.793741+0000","last_active":"2024-03-28T13:10:58.265756+0000","last_peered":"2024-03-28T13:10:58.265756+0000","last_clean":"2024-03-28T13:10:58.265756+0000","last_became_active":"2024-03-28T13:10:56.793473+0000","last_became_peered":"2024-03-28T13:10:56.793473+0000","last_unstale":"2024-03-28T13:10:58.265756+0000","last_undegraded":"2024-03-28T13:10:58.265756+0000","last_fullsized":"2024-03-28T13:10:58.265756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:38:30.086095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783676+0000","last_change":"2024-03-28T13:10:55.757407+0000","last_active":"2024-03-28T13:10:57.783676+0000","last_peered":"2024-03-28T13:10:57.783676+0000","last_clean":"2024-03-28T13:10:57.783676+0000","last_became_active":"2024-03-28T13:10:55.757088+0000","last_became_peered":"2024-03-28T13:10:55.757088+0000","last_unstale":"2024-03-28T13:10:57.783676+0000","last_undegraded":"2024-03-28T13:10:57.783676+0000","last_fullsized":"2024-03-28T13:10:57.783676+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:03:35.637042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.367177+0000","last_change":"2024-03-28T13:10:56.772822+0000","last_active":"2024-03-28T13:10:59.367177+0000","last_peered":"2024-03-28T13:10:59.367177+0000","last_clean":"2024-03-28T13:10:59.367177+0000","last_became_active":"2024-03-28T13:10:56.772511+0000","last_became_peered":"2024-03-28T13:10:56.772511+0000","last_unstale":"2024-03-28T13:10:59.367177+0000","last_undegraded":"2024-03-28T13:10:59.367177+0000","last_fullsized":"2024-03-28T13:10:59.367177+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:57:32.444784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771480+0000","last_change":"2024-03-28T13:10:55.748363+0000","last_active":"2024-03-28T13:10:57.771480+0000","last_peered":"2024-03-28T13:10:57.771480+0000","last_clean":"2024-03-28T13:10:57.771480+0000","last_became_active":"2024-03-28T13:10:55.748084+0000","last_became_peered":"2024-03-28T13:10:55.748084+0000","last_unstale":"2024-03-28T13:10:57.771480+0000","last_undegraded":"2024-03-28T13:10:57.771480+0000","last_fullsized":"2024-03-28T13:10:57.771480+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:58.262597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774512+0000","last_change":"2024-03-28T13:10:56.774113+0000","last_active":"2024-03-28T13:10:57.774512+0000","last_peered":"2024-03-28T13:10:57.774512+0000","last_clean":"2024-03-28T13:10:57.774512+0000","last_became_active":"2024-03-28T13:10:56.773838+0000","last_became_peered":"2024-03-28T13:10:56.773838+0000","last_unstale":"2024-03-28T13:10:57.774512+0000","last_undegraded":"2024-03-28T13:10:57.774512+0000","last_fullsized":"2024-03-28T13:10:57.774512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:45.087620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265939+0000","last_change":"2024-03-28T13:10:55.746355+0000","last_active":"2024-03-28T13:10:58.265939+0000","last_peered":"2024-03-28T13:10:58.265939+0000","last_clean":"2024-03-28T13:10:58.265939+0000","last_became_active":"2024-03-28T13:10:55.744858+0000","last_became_peered":"2024-03-28T13:10:55.744858+0000","last_unstale":"2024-03-28T13:10:58.265939+0000","last_undegraded":"2024-03-28T13:10:58.265939+0000","last_fullsized":"2024-03-28T13:10:58.265939+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:00:00.843211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783690+0000","last_change":"2024-03-28T13:10:56.779040+0000","last_active":"2024-03-28T13:10:57.783690+0000","last_peered":"2024-03-28T13:10:57.783690+0000","last_clean":"2024-03-28T13:10:57.783690+0000","last_became_active":"2024-03-28T13:10:56.777765+0000","last_became_peered":"2024-03-28T13:10:56.777765+0000","last_unstale":"2024-03-28T13:10:57.783690+0000","last_undegraded":"2024-03-28T13:10:57.783690+0000","last_fullsized":"2024-03-28T13:10:57.783690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:44.685445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029121+0000","last_change":"2024-03-28T13:10:55.753629+0000","last_active":"2024-03-28T13:10:58.029121+0000","last_peered":"2024-03-28T13:10:58.029121+0000","last_clean":"2024-03-28T13:10:58.029121+0000","last_became_active":"2024-03-28T13:10:55.753330+0000","last_became_peered":"2024-03-28T13:10:55.753330+0000","last_unstale":"2024-03-28T13:10:58.029121+0000","last_undegraded":"2024-03-28T13:10:58.029121+0000","last_fullsized":"2024-03-28T13:10:58.029121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:30.977315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.781976+0000","last_change":"2024-03-28T13:10:56.774575+0000","last_active":"2024-03-28T13:10:56.781976+0000","last_peered":"2024-03-28T13:10:56.781976+0000","last_clean":"2024-03-28T13:10:56.781976+0000","last_became_active":"2024-03-28T13:10:56.772544+0000","last_became_peered":"2024-03-28T13:10:56.772544+0000","last_unstale":"2024-03-28T13:10:56.781976+0000","last_undegraded":"2024-03-28T13:10:56.781976+0000","last_fullsized":"2024-03-28T13:10:56.781976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:29:15.008724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.775058+0000","last_change":"2024-03-28T13:10:55.754015+0000","last_active":"2024-03-28T13:10:57.775058+0000","last_peered":"2024-03-28T13:10:57.775058+0000","last_clean":"2024-03-28T13:10:57.775058+0000","last_became_active":"2024-03-28T13:10:55.753701+0000","last_became_peered":"2024-03-28T13:10:55.753701+0000","last_unstale":"2024-03-28T13:10:57.775058+0000","last_undegraded":"2024-03-28T13:10:57.775058+0000","last_fullsized":"2024-03-28T13:10:57.775058+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:29:13.996522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.775071+0000","last_change":"2024-03-28T13:10:56.782871+0000","last_active":"2024-03-28T13:10:57.775071+0000","last_peered":"2024-03-28T13:10:57.775071+0000","last_clean":"2024-03-28T13:10:57.775071+0000","last_became_active":"2024-03-28T13:10:56.782020+0000","last_became_peered":"2024-03-28T13:10:56.782020+0000","last_unstale":"2024-03-28T13:10:57.775071+0000","last_undegraded":"2024-03-28T13:10:57.775071+0000","last_fullsized":"2024-03-28T13:10:57.775071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771032+0000","last_change":"2024-03-28T13:10:55.744963+0000","last_active":"2024-03-28T13:10:57.771032+0000","last_peered":"2024-03-28T13:10:57.771032+0000","last_clean":"2024-03-28T13:10:57.771032+0000","last_became_active":"2024-03-28T13:10:55.743998+0000","last_became_peered":"2024-03-28T13:10:55.743998+0000","last_unstale":"2024-03-28T13:10:57.771032+0000","last_undegraded":"2024-03-28T13:10:57.771032+0000","last_fullsized":"2024-03-28T13:10:57.771032+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:50:09.927656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.787619+0000","last_change":"2024-03-28T13:10:56.780543+0000","last_active":"2024-03-28T13:10:56.787619+0000","last_peered":"2024-03-28T13:10:56.787619+0000","last_clean":"2024-03-28T13:10:56.787619+0000","last_became_active":"2024-03-28T13:10:56.780189+0000","last_became_peered":"2024-03-28T13:10:56.780189+0000","last_unstale":"2024-03-28T13:10:56.787619+0000","last_undegraded":"2024-03-28T13:10:56.787619+0000","last_fullsized":"2024-03-28T13:10:56.787619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:31.989516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366777+0000","last_change":"2024-03-28T13:10:55.748693+0000","last_active":"2024-03-28T13:10:59.366777+0000","last_peered":"2024-03-28T13:10:59.366777+0000","last_clean":"2024-03-28T13:10:59.366777+0000","last_became_active":"2024-03-28T13:10:55.748357+0000","last_became_peered":"2024-03-28T13:10:55.748357+0000","last_unstale":"2024-03-28T13:10:59.366777+0000","last_undegraded":"2024-03-28T13:10:59.366777+0000","last_fullsized":"2024-03-28T13:10:59.366777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:16:57.647981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366596+0000","last_change":"2024-03-28T13:10:56.770455+0000","last_active":"2024-03-28T13:10:59.366596+0000","last_peered":"2024-03-28T13:10:59.366596+0000","last_clean":"2024-03-28T13:10:59.366596+0000","last_became_active":"2024-03-28T13:10:56.770174+0000","last_became_peered":"2024-03-28T13:10:56.770174+0000","last_unstale":"2024-03-28T13:10:59.366596+0000","last_undegraded":"2024-03-28T13:10:59.366596+0000","last_fullsized":"2024-03-28T13:10:59.366596+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:59.274798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.264885+0000","last_change":"2024-03-28T13:10:55.743425+0000","last_active":"2024-03-28T13:10:58.264885+0000","last_peered":"2024-03-28T13:10:58.264885+0000","last_clean":"2024-03-28T13:10:58.264885+0000","last_became_active":"2024-03-28T13:10:55.743112+0000","last_became_peered":"2024-03-28T13:10:55.743112+0000","last_unstale":"2024-03-28T13:10:58.264885+0000","last_undegraded":"2024-03-28T13:10:58.264885+0000","last_fullsized":"2024-03-28T13:10:58.264885+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:25.221468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783615+0000","last_change":"2024-03-28T13:10:56.775289+0000","last_active":"2024-03-28T13:10:57.783615+0000","last_peered":"2024-03-28T13:10:57.783615+0000","last_clean":"2024-03-28T13:10:57.783615+0000","last_became_active":"2024-03-28T13:10:56.774549+0000","last_became_peered":"2024-03-28T13:10:56.774549+0000","last_unstale":"2024-03-28T13:10:57.783615+0000","last_undegraded":"2024-03-28T13:10:57.783615+0000","last_fullsized":"2024-03-28T13:10:57.783615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:16.475067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.794905+0000","last_change":"2024-03-28T13:10:55.755495+0000","last_active":"2024-03-28T13:10:56.794905+0000","last_peered":"2024-03-28T13:10:56.794905+0000","last_clean":"2024-03-28T13:10:56.794905+0000","last_became_active":"2024-03-28T13:10:55.754987+0000","last_became_peered":"2024-03-28T13:10:55.754987+0000","last_unstale":"2024-03-28T13:10:56.794905+0000","last_undegraded":"2024-03-28T13:10:56.794905+0000","last_fullsized":"2024-03-28T13:10:56.794905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:57:31.432582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265584+0000","last_change":"2024-03-28T13:10:56.776492+0000","last_active":"2024-03-28T13:10:58.265584+0000","last_peered":"2024-03-28T13:10:58.265584+0000","last_clean":"2024-03-28T13:10:58.265584+0000","last_became_active":"2024-03-28T13:10:56.774511+0000","last_became_peered":"2024-03-28T13:10:56.774511+0000","last_unstale":"2024-03-28T13:10:58.265584+0000","last_undegraded":"2024-03-28T13:10:58.265584+0000","last_fullsized":"2024-03-28T13:10:58.265584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:09:01.845737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783204+0000","last_change":"2024-03-28T13:10:55.748821+0000","last_active":"2024-03-28T13:10:57.783204+0000","last_peered":"2024-03-28T13:10:57.783204+0000","last_clean":"2024-03-28T13:10:57.783204+0000","last_became_active":"2024-03-28T13:10:55.748128+0000","last_became_peered":"2024-03-28T13:10:55.748128+0000","last_unstale":"2024-03-28T13:10:57.783204+0000","last_undegraded":"2024-03-28T13:10:57.783204+0000","last_fullsized":"2024-03-28T13:10:57.783204+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:48:41.782400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783822+0000","last_change":"2024-03-28T13:10:56.785728+0000","last_active":"2024-03-28T13:10:57.783822+0000","last_peered":"2024-03-28T13:10:57.783822+0000","last_clean":"2024-03-28T13:10:57.783822+0000","last_became_active":"2024-03-28T13:10:56.783504+0000","last_became_peered":"2024-03-28T13:10:56.783504+0000","last_unstale":"2024-03-28T13:10:57.783822+0000","last_undegraded":"2024-03-28T13:10:57.783822+0000","last_fullsized":"2024-03-28T13:10:57.783822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:42:20.735994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.747889+0000","last_change":"2024-03-28T13:10:55.757201+0000","last_active":"2024-03-28T13:10:56.747889+0000","last_peered":"2024-03-28T13:10:56.747889+0000","last_clean":"2024-03-28T13:10:56.747889+0000","last_became_active":"2024-03-28T13:10:55.756328+0000","last_became_peered":"2024-03-28T13:10:55.756328+0000","last_unstale":"2024-03-28T13:10:56.747889+0000","last_undegraded":"2024-03-28T13:10:56.747889+0000","last_fullsized":"2024-03-28T13:10:56.747889+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:50:09.927656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029820+0000","last_change":"2024-03-28T13:10:56.789219+0000","last_active":"2024-03-28T13:10:58.029820+0000","last_peered":"2024-03-28T13:10:58.029820+0000","last_clean":"2024-03-28T13:10:58.029820+0000","last_became_active":"2024-03-28T13:10:56.783607+0000","last_became_peered":"2024-03-28T13:10:56.783607+0000","last_unstale":"2024-03-28T13:10:58.029820+0000","last_undegraded":"2024-03-28T13:10:58.029820+0000","last_fullsized":"2024-03-28T13:10:58.029820+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:40.900533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.747164+0000","last_change":"2024-03-28T13:10:55.753645+0000","last_active":"2024-03-28T13:10:56.747164+0000","last_peered":"2024-03-28T13:10:56.747164+0000","last_clean":"2024-03-28T13:10:56.747164+0000","last_became_active":"2024-03-28T13:10:55.753103+0000","last_became_peered":"2024-03-28T13:10:55.753103+0000","last_unstale":"2024-03-28T13:10:56.747164+0000","last_undegraded":"2024-03-28T13:10:56.747164+0000","last_fullsized":"2024-03-28T13:10:56.747164+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:38:25.780963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777482+0000","last_change":"2024-03-28T13:10:56.784216+0000","last_active":"2024-03-28T13:10:57.777482+0000","last_peered":"2024-03-28T13:10:57.777482+0000","last_clean":"2024-03-28T13:10:57.777482+0000","last_became_active":"2024-03-28T13:10:56.783936+0000","last_became_peered":"2024-03-28T13:10:56.783936+0000","last_unstale":"2024-03-28T13:10:57.777482+0000","last_undegraded":"2024-03-28T13:10:57.777482+0000","last_fullsized":"2024-03-28T13:10:57.777482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:53.479188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.748281+0000","last_change":"2024-03-28T13:10:55.754934+0000","last_active":"2024-03-28T13:10:56.748281+0000","last_peered":"2024-03-28T13:10:56.748281+0000","last_clean":"2024-03-28T13:10:56.748281+0000","last_became_active":"2024-03-28T13:10:55.754215+0000","last_became_peered":"2024-03-28T13:10:55.754215+0000","last_unstale":"2024-03-28T13:10:56.748281+0000","last_undegraded":"2024-03-28T13:10:56.748281+0000","last_fullsized":"2024-03-28T13:10:56.748281+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:51:08.557038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776768+0000","last_change":"2024-03-28T13:10:56.791043+0000","last_active":"2024-03-28T13:10:57.776768+0000","last_peered":"2024-03-28T13:10:57.776768+0000","last_clean":"2024-03-28T13:10:57.776768+0000","last_became_active":"2024-03-28T13:10:56.787198+0000","last_became_peered":"2024-03-28T13:10:56.787198+0000","last_unstale":"2024-03-28T13:10:57.776768+0000","last_undegraded":"2024-03-28T13:10:57.776768+0000","last_fullsized":"2024-03-28T13:10:57.776768+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.770893+0000","last_change":"2024-03-28T13:10:55.741761+0000","last_active":"2024-03-28T13:10:57.770893+0000","last_peered":"2024-03-28T13:10:57.770893+0000","last_clean":"2024-03-28T13:10:57.770893+0000","last_became_active":"2024-03-28T13:10:55.741472+0000","last_became_peered":"2024-03-28T13:10:55.741472+0000","last_unstale":"2024-03-28T13:10:57.770893+0000","last_undegraded":"2024-03-28T13:10:57.770893+0000","last_fullsized":"2024-03-28T13:10:57.770893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:57:19.883535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265564+0000","last_change":"2024-03-28T13:10:56.775554+0000","last_active":"2024-03-28T13:10:58.265564+0000","last_peered":"2024-03-28T13:10:58.265564+0000","last_clean":"2024-03-28T13:10:58.265564+0000","last_became_active":"2024-03-28T13:10:56.774272+0000","last_became_peered":"2024-03-28T13:10:56.774272+0000","last_unstale":"2024-03-28T13:10:58.265564+0000","last_undegraded":"2024-03-28T13:10:58.265564+0000","last_fullsized":"2024-03-28T13:10:58.265564+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:53:51.017744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776675+0000","last_change":"2024-03-28T13:10:55.746007+0000","last_active":"2024-03-28T13:10:57.776675+0000","last_peered":"2024-03-28T13:10:57.776675+0000","last_clean":"2024-03-28T13:10:57.776675+0000","last_became_active":"2024-03-28T13:10:55.745666+0000","last_became_peered":"2024-03-28T13:10:55.745666+0000","last_unstale":"2024-03-28T13:10:57.776675+0000","last_undegraded":"2024-03-28T13:10:57.776675+0000","last_fullsized":"2024-03-28T13:10:57.776675+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:36:28.139935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029046+0000","last_change":"2024-03-28T13:10:56.787334+0000","last_active":"2024-03-28T13:10:58.029046+0000","last_peered":"2024-03-28T13:10:58.029046+0000","last_clean":"2024-03-28T13:10:58.029046+0000","last_became_active":"2024-03-28T13:10:56.786996+0000","last_became_peered":"2024-03-28T13:10:56.786996+0000","last_unstale":"2024-03-28T13:10:58.029046+0000","last_undegraded":"2024-03-28T13:10:58.029046+0000","last_fullsized":"2024-03-28T13:10:58.029046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:58:13.880385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.266118+0000","last_change":"2024-03-28T13:10:55.742909+0000","last_active":"2024-03-28T13:10:58.266118+0000","last_peered":"2024-03-28T13:10:58.266118+0000","last_clean":"2024-03-28T13:10:58.266118+0000","last_became_active":"2024-03-28T13:10:55.742605+0000","last_became_peered":"2024-03-28T13:10:55.742605+0000","last_unstale":"2024-03-28T13:10:58.266118+0000","last_undegraded":"2024-03-28T13:10:58.266118+0000","last_fullsized":"2024-03-28T13:10:58.266118+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:52.466987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":194,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366453+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:10:59.366453+0000","last_peered":"2024-03-28T13:10:59.366453+0000","last_clean":"2024-03-28T13:10:59.366453+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:10:59.366453+0000","last_undegraded":"2024-03-28T13:10:59.366453+0000","last_fullsized":"2024-03-28T13:10:59.366453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476760,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":155961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476758,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":155219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476758,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":745623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476758,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":155275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":397312,"data_stored":150497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":746135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":156819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":155207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:11:02.326 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-28T13:11:02.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.589+0000 7f50d2600640 1 Processor -- start 2024-03-28T13:11:02.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.589+0000 7f50d2600640 1 -- start start 2024-03-28T13:11:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.591+0000 7f50d2600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50cc0736f0 0x7f50cc15f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.591+0000 7f50d2600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50cc0cfc80 0x7f50cc0d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.591+0000 7f50d2600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50cc0717c0 con 0x7f50cc15e4d0 2024-03-28T13:11:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.591+0000 7f50d2600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f50cc0b3ab0 con 0x7f50cc0cfc80 2024-03-28T13:11:02.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.591+0000 7f50d2600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f50cc071dd0 con 0x7f50cc0736f0 2024-03-28T13:11:02.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.591+0000 7f50d1200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f50cc15e4d0 0x7f50cc0ade40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35730/0 (socket says 172.21.15.19:35730) 2024-03-28T13:11:02.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.591+0000 7f50d1200640 1 -- 172.21.15.19:0/2399072879 learned_addr learned my addr 172.21.15.19:0/2399072879 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:02.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.591+0000 7f50d1c00640 1 --2- 172.21.15.19:0/2399072879 >> v2:172.21.15.19:3300/0 conn(0x7f50cc0cfc80 0x7f50cc0d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:02.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.591+0000 7f50cbe00640 1 --2- 172.21.15.19:0/2399072879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50cc0736f0 0x7f50cc15f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:02.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.592+0000 7f50cb400640 1 -- 172.21.15.19:0/2399072879 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2382851371 0 0) 0x7f50cc0717c0 con 0x7f50cc15e4d0 2024-03-28T13:11:02.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.592+0000 7f50d1c00640 1 -- 172.21.15.19:0/2399072879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50cc0736f0 msgr2=0x7f50cc15f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.592+0000 7f50d1c00640 1 --2- 172.21.15.19:0/2399072879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50cc0736f0 0x7f50cc15f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.592+0000 7f50d1c00640 1 -- 172.21.15.19:0/2399072879 >> v1:172.21.15.19:6789/0 conn(0x7f50cc15e4d0 legacy=0x7f50cc0ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.592+0000 7f50d1c00640 1 -- 172.21.15.19:0/2399072879 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50cc15e8b0 con 0x7f50cc0cfc80 2024-03-28T13:11:02.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.593+0000 7f50d1c00640 1 --2- 172.21.15.19:0/2399072879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50cc0cfc80 0x7f50cc0d0110 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f50c4009d90 tx=0x7f50c402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b278b0a759874c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:02.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.593+0000 7f50cb400640 1 -- 172.21.15.19:0/2399072879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4039070 con 0x7f50cc0cfc80 2024-03-28T13:11:02.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.594+0000 7f50cb400640 1 -- 172.21.15.19:0/2399072879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c402cc50 con 0x7f50cc0cfc80 2024-03-28T13:11:02.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.594+0000 7f50cb400640 1 -- 172.21.15.19:0/2399072879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4034440 con 0x7f50cc0cfc80 2024-03-28T13:11:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.594+0000 7f50d2600640 1 -- 172.21.15.19:0/2399072879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50cc0cfc80 msgr2=0x7f50cc0d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.594+0000 7f50d2600640 1 --2- 172.21.15.19:0/2399072879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50cc0cfc80 0x7f50cc0d0110 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f50c4009d90 tx=0x7f50c402bea0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.594+0000 7f50d2600640 1 -- 172.21.15.19:0/2399072879 shutdown_connections 2024-03-28T13:11:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.594+0000 7f50d2600640 1 --2- 172.21.15.19:0/2399072879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50cc0cfc80 0x7f50cc0d0110 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.594+0000 7f50d2600640 1 --2- 172.21.15.19:0/2399072879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50cc0736f0 0x7f50cc15f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.594+0000 7f50d2600640 1 -- 172.21.15.19:0/2399072879 >> 172.21.15.19:0/2399072879 conn(0x7f50cc06b570 msgr2=0x7f50cc06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.594+0000 7f50d2600640 1 -- 172.21.15.19:0/2399072879 shutdown_connections 2024-03-28T13:11:02.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.594+0000 7f50d2600640 1 -- 172.21.15.19:0/2399072879 wait complete. 2024-03-28T13:11:02.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.596+0000 7f50d2600640 1 Processor -- start 2024-03-28T13:11:02.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.596+0000 7f50d2600640 1 -- start start 2024-03-28T13:11:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.597+0000 7f50d2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50cc0736f0 0x7f50cc15f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.597+0000 7f50d2600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50cc0cfc80 0x7f50cc1616a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.597+0000 7f50d2600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50cc15e4d0 0x7f50cc149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.597+0000 7f50d2600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc071a60 con 0x7f50cc0736f0 2024-03-28T13:11:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.597+0000 7f50d2600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f50cc0b3fe0 con 0x7f50cc0cfc80 2024-03-28T13:11:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.597+0000 7f50d2600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f50cc072260 con 0x7f50cc15e4d0 2024-03-28T13:11:02.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.598+0000 7f50d1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50cc0736f0 0x7f50cc15f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.598+0000 7f50d1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50cc0736f0 0x7f50cc15f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43540/0 (socket says 172.21.15.19:43540) 2024-03-28T13:11:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.598+0000 7f50d1200640 1 -- 172.21.15.19:0/322353927 learned_addr learned my addr 172.21.15.19:0/322353927 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.598+0000 7f50d1200640 1 -- 172.21.15.19:0/322353927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50cc15e4d0 msgr2=0x7f50cc149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.598+0000 7f50d1200640 1 --2- 172.21.15.19:0/322353927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50cc15e4d0 0x7f50cc149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.598+0000 7f50d1200640 1 -- 172.21.15.19:0/322353927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50cc0cfc80 msgr2=0x7f50cc1616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.598+0000 7f50d1200640 1 --2- 172.21.15.19:0/322353927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50cc0cfc80 0x7f50cc1616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.598+0000 7f50d1200640 1 -- 172.21.15.19:0/322353927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50cc15fb50 con 0x7f50cc0736f0 2024-03-28T13:11:02.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.598+0000 7f50d1200640 1 --2- 172.21.15.19:0/322353927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50cc0736f0 0x7f50cc15f490 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f50c000c430 tx=0x7f50c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.598+0000 7f50c8c00640 1 -- 172.21.15.19:0/322353927 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c00074e0 con 0x7f50cc0736f0 2024-03-28T13:11:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.599+0000 7f50c8c00640 1 -- 172.21.15.19:0/322353927 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c00047e0 con 0x7f50cc0736f0 2024-03-28T13:11:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.599+0000 7f50c8c00640 1 -- 172.21.15.19:0/322353927 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0004bd0 con 0x7f50cc0736f0 2024-03-28T13:11:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.599+0000 7f50d2600640 1 -- 172.21.15.19:0/322353927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50cc15fe30 con 0x7f50cc0736f0 2024-03-28T13:11:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.599+0000 7f50d2600640 1 -- 172.21.15.19:0/322353927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50cc1f4290 con 0x7f50cc0736f0 2024-03-28T13:11:02.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.600+0000 7f50d2600640 1 -- 172.21.15.19:0/322353927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50cc075940 con 0x7f50cc0736f0 2024-03-28T13:11:02.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.601+0000 7f50c8c00640 1 -- 172.21.15.19:0/322353927 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f50c0011070 con 0x7f50cc0736f0 2024-03-28T13:11:02.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.602+0000 7f50c8c00640 1 --2- 172.21.15.19:0/322353927 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50ac06c530 0x7f50ac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:02.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.602+0000 7f50c8c00640 1 -- 172.21.15.19:0/322353927 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f50c008b440 con 0x7f50cc0736f0 2024-03-28T13:11:02.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.602+0000 7f50cbe00640 1 --2- 172.21.15.19:0/322353927 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50ac06c530 0x7f50ac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:02.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.603+0000 7f50cbe00640 1 --2- 172.21.15.19:0/322353927 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50ac06c530 0x7f50ac06e9f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f50b8006a90 tx=0x7f50b8006a20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:02.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.606+0000 7f50c8c00640 1 -- 172.21.15.19:0/322353927 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f50c00599d0 con 0x7f50cc0736f0 2024-03-28T13:11:02.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.885+0000 7f50d2600640 1 -- 172.21.15.19:0/322353927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f50cc15bdb0 con 0x7f50cc0736f0 2024-03-28T13:11:02.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.886+0000 7f50c8c00640 1 -- 172.21.15.19:0/322353927 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1174 (secure 0 0 0) 0x7f50c00592d0 con 0x7f50cc0736f0 2024-03-28T13:11:02.886 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:02.886 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":106,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":8,"num_up_osds":8,"osd_up_since":1711631369,"num_in_osds":8,"osd_in_since":1711631359,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":225787904,"bytes_avail":772868325376,"bytes_total":773094113280,"write_bytes_sec":2278,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":5061}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-03-28T13:10:38.002509+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-28T13:11:02.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.889+0000 7f50d2600640 1 -- 172.21.15.19:0/322353927 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50ac06c530 msgr2=0x7f50ac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:02.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.889+0000 7f50d2600640 1 --2- 172.21.15.19:0/322353927 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50ac06c530 0x7f50ac06e9f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f50b8006a90 tx=0x7f50b8006a20 comp rx=0 tx=0).stop 2024-03-28T13:11:02.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.889+0000 7f50d2600640 1 -- 172.21.15.19:0/322353927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50cc0736f0 msgr2=0x7f50cc15f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:02.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.889+0000 7f50d2600640 1 --2- 172.21.15.19:0/322353927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50cc0736f0 0x7f50cc15f490 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f50c000c430 tx=0x7f50c000c8f0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.889+0000 7f50d2600640 1 -- 172.21.15.19:0/322353927 shutdown_connections 2024-03-28T13:11:02.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.889+0000 7f50d2600640 1 --2- 172.21.15.19:0/322353927 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50ac06c530 0x7f50ac06e9f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.889+0000 7f50d2600640 1 --2- 172.21.15.19:0/322353927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50cc15e4d0 0x7f50cc149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.889+0000 7f50d2600640 1 --2- 172.21.15.19:0/322353927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50cc0cfc80 0x7f50cc1616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.889+0000 7f50d2600640 1 --2- 172.21.15.19:0/322353927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50cc0736f0 0x7f50cc15f490 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.889+0000 7f50d2600640 1 -- 172.21.15.19:0/322353927 >> 172.21.15.19:0/322353927 conn(0x7f50cc06b570 msgr2=0x7f50cc0aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.889+0000 7f50d2600640 1 -- 172.21.15.19:0/322353927 shutdown_connections 2024-03-28T13:11:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:02.889+0000 7f50d2600640 1 -- 172.21.15.19:0/322353927 wait complete. 2024-03-28T13:11:02.905 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-28T13:11:02.905 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:11:03.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.112+0000 7f5a6da00640 1 Processor -- start 2024-03-28T13:11:03.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.112+0000 7f5a6da00640 1 -- start start 2024-03-28T13:11:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.114+0000 7f5a6da00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680d2310 0x7f5a680ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.114+0000 7f5a6da00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a680cf530 0x7f5a680b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.114+0000 7f5a6da00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a6815e880 con 0x7f5a6815e4a0 2024-03-28T13:11:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.114+0000 7f5a6da00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5a680d34e0 con 0x7f5a680cf530 2024-03-28T13:11:03.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.114+0000 7f5a6da00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5a680d28d0 con 0x7f5a680d2310 2024-03-28T13:11:03.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.114+0000 7f5a6d000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a680cf530 0x7f5a680b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:03.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.115+0000 7f5a67400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680d2310 0x7f5a680ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.115+0000 7f5a67e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f5a6815e4a0 0x7f5a680b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:33794/0 (socket says 172.21.15.19:33794) 2024-03-28T13:11:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.115+0000 7f5a6d000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a680cf530 0x7f5a680b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:51080/0 (socket says 172.21.15.19:51080) 2024-03-28T13:11:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.115+0000 7f5a67e00640 1 -- 172.21.15.19:0/617378724 learned_addr learned my addr 172.21.15.19:0/617378724 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.115+0000 7f5a66a00640 1 -- 172.21.15.19:0/617378724 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2687300977 0 0) 0x7f5a6815e880 con 0x7f5a6815e4a0 2024-03-28T13:11:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.115+0000 7f5a66a00640 1 -- 172.21.15.19:0/617378724 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a4c003520 con 0x7f5a6815e4a0 2024-03-28T13:11:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.115+0000 7f5a6d000640 1 -- 172.21.15.19:0/617378724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680d2310 msgr2=0x7f5a680ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.115+0000 7f5a6d000640 1 --2- 172.21.15.19:0/617378724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680d2310 0x7f5a680ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.115+0000 7f5a6d000640 1 -- 172.21.15.19:0/617378724 >> v1:172.21.15.188:6789/0 conn(0x7f5a6815e4a0 legacy=0x7f5a680b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.115+0000 7f5a6d000640 1 -- 172.21.15.19:0/617378724 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a680d2bb0 con 0x7f5a680cf530 2024-03-28T13:11:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.115+0000 7f5a67400640 1 --2- 172.21.15.19:0/617378724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680d2310 0x7f5a680ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:11:03.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.117+0000 7f5a6d000640 1 --2- 172.21.15.19:0/617378724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680cf530 0x7f5a680b73e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009d90 tx=0x7f5a6002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e05fe7e6ef67b46a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:03.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.117+0000 7f5a66a00640 1 -- 172.21.15.19:0/617378724 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60039070 con 0x7f5a680cf530 2024-03-28T13:11:03.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.117+0000 7f5a66a00640 1 -- 172.21.15.19:0/617378724 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a6002cc60 con 0x7f5a680cf530 2024-03-28T13:11:03.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.117+0000 7f5a66a00640 1 -- 172.21.15.19:0/617378724 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60034660 con 0x7f5a680cf530 2024-03-28T13:11:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.118+0000 7f5a6da00640 1 -- 172.21.15.19:0/617378724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680cf530 msgr2=0x7f5a680b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.118+0000 7f5a6da00640 1 --2- 172.21.15.19:0/617378724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680cf530 0x7f5a680b73e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009d90 tx=0x7f5a6002beb0 comp rx=0 tx=0).stop 2024-03-28T13:11:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.118+0000 7f5a6da00640 1 -- 172.21.15.19:0/617378724 shutdown_connections 2024-03-28T13:11:03.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.118+0000 7f5a6da00640 1 --2- 172.21.15.19:0/617378724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680cf530 0x7f5a680b73e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.118+0000 7f5a6da00640 1 --2- 172.21.15.19:0/617378724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a680d2310 0x7f5a680ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.118+0000 7f5a6da00640 1 -- 172.21.15.19:0/617378724 >> 172.21.15.19:0/617378724 conn(0x7f5a680676d0 msgr2=0x7f5a68067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.118+0000 7f5a6da00640 1 -- 172.21.15.19:0/617378724 shutdown_connections 2024-03-28T13:11:03.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.118+0000 7f5a6da00640 1 -- 172.21.15.19:0/617378724 wait complete. 2024-03-28T13:11:03.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.120+0000 7f5a6da00640 1 Processor -- start 2024-03-28T13:11:03.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.120+0000 7f5a6da00640 1 -- start start 2024-03-28T13:11:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.121+0000 7f5a6da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a680cf530 0x7f5a681544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.121+0000 7f5a6da00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680d2310 0x7f5a6815f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.121+0000 7f5a6da00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6815e4a0 0x7f5a6815f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.121+0000 7f5a6da00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5a680d3970 con 0x7f5a680cf530 2024-03-28T13:11:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.121+0000 7f5a6da00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5a680d3170 con 0x7f5a680d2310 2024-03-28T13:11:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.121+0000 7f5a6da00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5a680b7920 con 0x7f5a6815e4a0 2024-03-28T13:11:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.121+0000 7f5a67e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a680cf530 0x7f5a681544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.121+0000 7f5a67e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a680cf530 0x7f5a681544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43546/0 (socket says 172.21.15.19:43546) 2024-03-28T13:11:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.121+0000 7f5a67e00640 1 -- 172.21.15.19:0/3206535481 learned_addr learned my addr 172.21.15.19:0/3206535481 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.122+0000 7f5a67400640 1 --2- 172.21.15.19:0/3206535481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680d2310 0x7f5a6815f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.122+0000 7f5a6d000640 1 --2- 172.21.15.19:0/3206535481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6815e4a0 0x7f5a6815f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:03.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.122+0000 7f5a67e00640 1 -- 172.21.15.19:0/3206535481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6815e4a0 msgr2=0x7f5a6815f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.122+0000 7f5a67e00640 1 --2- 172.21.15.19:0/3206535481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6815e4a0 0x7f5a6815f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.122+0000 7f5a67e00640 1 -- 172.21.15.19:0/3206535481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680d2310 msgr2=0x7f5a6815f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.122+0000 7f5a67e00640 1 --2- 172.21.15.19:0/3206535481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680d2310 0x7f5a6815f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.122+0000 7f5a67e00640 1 -- 172.21.15.19:0/3206535481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a68160010 con 0x7f5a680cf530 2024-03-28T13:11:03.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.122+0000 7f5a67400640 1 --2- 172.21.15.19:0/3206535481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680d2310 0x7f5a6815f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:11:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.122+0000 7f5a67e00640 1 --2- 172.21.15.19:0/3206535481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a680cf530 0x7f5a681544b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800c390 tx=0x7f5a5800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.123+0000 7f5a5fe00640 1 -- 172.21.15.19:0/3206535481 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58017070 con 0x7f5a680cf530 2024-03-28T13:11:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.123+0000 7f5a5fe00640 1 -- 172.21.15.19:0/3206535481 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a58005ce0 con 0x7f5a680cf530 2024-03-28T13:11:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.123+0000 7f5a6da00640 1 -- 172.21.15.19:0/3206535481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a681602f0 con 0x7f5a680cf530 2024-03-28T13:11:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.123+0000 7f5a5fe00640 1 -- 172.21.15.19:0/3206535481 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a580072f0 con 0x7f5a680cf530 2024-03-28T13:11:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.123+0000 7f5a6da00640 1 -- 172.21.15.19:0/3206535481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a681608b0 con 0x7f5a680cf530 2024-03-28T13:11:03.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.124+0000 7f5a6da00640 1 -- 172.21.15.19:0/3206535481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a68075940 con 0x7f5a680cf530 2024-03-28T13:11:03.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.125+0000 7f5a5fe00640 1 -- 172.21.15.19:0/3206535481 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5a5801a070 con 0x7f5a680cf530 2024-03-28T13:11:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.126+0000 7f5a5fe00640 1 --2- 172.21.15.19:0/3206535481 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5a4806c530 0x7f5a4806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.126+0000 7f5a5fe00640 1 -- 172.21.15.19:0/3206535481 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f5a5808aaf0 con 0x7f5a680cf530 2024-03-28T13:11:03.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.127+0000 7f5a67400640 1 --2- 172.21.15.19:0/3206535481 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5a4806c530 0x7f5a4806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:03.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.128+0000 7f5a67400640 1 --2- 172.21.15.19:0/3206535481 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5a4806c530 0x7f5a4806e9f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5a681615a0 tx=0x7f5a54008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:03.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.131+0000 7f5a5fe00640 1 -- 172.21.15.19:0/3206535481 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5a58059100 con 0x7f5a680cf530 2024-03-28T13:11:03.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.334+0000 7f5a6da00640 1 -- 172.21.15.19:0/3206535481 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a68160c20 con 0x7f5a4806c530 2024-03-28T13:11:03.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.353+0000 7f5a5fe00640 1 -- 172.21.15.19:0/3206535481 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f5a68160c20 con 0x7f5a4806c530 2024-03-28T13:11:03.357 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:03.358 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:11:03.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.359+0000 7f5a6da00640 1 -- 172.21.15.19:0/3206535481 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5a4806c530 msgr2=0x7f5a4806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:03.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.359+0000 7f5a6da00640 1 --2- 172.21.15.19:0/3206535481 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5a4806c530 0x7f5a4806e9f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5a681615a0 tx=0x7f5a54008040 comp rx=0 tx=0).stop 2024-03-28T13:11:03.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.359+0000 7f5a6da00640 1 -- 172.21.15.19:0/3206535481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a680cf530 msgr2=0x7f5a681544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:03.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.359+0000 7f5a6da00640 1 --2- 172.21.15.19:0/3206535481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a680cf530 0x7f5a681544b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5a5800c390 tx=0x7f5a5800c850 comp rx=0 tx=0).stop 2024-03-28T13:11:03.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.359+0000 7f5a6da00640 1 -- 172.21.15.19:0/3206535481 shutdown_connections 2024-03-28T13:11:03.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.359+0000 7f5a6da00640 1 --2- 172.21.15.19:0/3206535481 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5a4806c530 0x7f5a4806e9f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:03.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.359+0000 7f5a6da00640 1 --2- 172.21.15.19:0/3206535481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a6815e4a0 0x7f5a6815f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:03.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.359+0000 7f5a6da00640 1 --2- 172.21.15.19:0/3206535481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a680d2310 0x7f5a6815f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:03.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.359+0000 7f5a6da00640 1 --2- 172.21.15.19:0/3206535481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5a680cf530 0x7f5a681544b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:03.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.360+0000 7f5a6da00640 1 -- 172.21.15.19:0/3206535481 >> 172.21.15.19:0/3206535481 conn(0x7f5a680676d0 msgr2=0x7f5a6815bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:03.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.360+0000 7f5a6da00640 1 -- 172.21.15.19:0/3206535481 shutdown_connections 2024-03-28T13:11:03.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:03.360+0000 7f5a6da00640 1 -- 172.21.15.19:0/3206535481 wait complete. 2024-03-28T13:11:03.382 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-03-28T13:11:02.010322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220496,"kb_used_data":4316,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754224,"statfs":{"total":773094113280,"available":772868325376,"internally_reserved":0,"allocated":4419584,"data_stored":2420736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.290953"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771651+0000","last_change":"2024-03-28T13:10:56.775043+0000","last_active":"2024-03-28T13:10:57.771651+0000","last_peered":"2024-03-28T13:10:57.771651+0000","last_clean":"2024-03-28T13:10:57.771651+0000","last_became_active":"2024-03-28T13:10:56.774736+0000","last_became_peered":"2024-03-28T13:10:56.774736+0000","last_unstale":"2024-03-28T13:10:57.771651+0000","last_undegraded":"2024-03-28T13:10:57.771651+0000","last_fullsized":"2024-03-28T13:10:57.771651+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:26.233670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774657+0000","last_change":"2024-03-28T13:10:56.790536+0000","last_active":"2024-03-28T13:10:57.774657+0000","last_peered":"2024-03-28T13:10:57.774657+0000","last_clean":"2024-03-28T13:10:57.774657+0000","last_became_active":"2024-03-28T13:10:56.790180+0000","last_became_peered":"2024-03-28T13:10:56.790180+0000","last_unstale":"2024-03-28T13:10:57.774657+0000","last_undegraded":"2024-03-28T13:10:57.774657+0000","last_fullsized":"2024-03-28T13:10:57.774657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:06:36.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.367194+0000","last_change":"2024-03-28T13:10:56.774875+0000","last_active":"2024-03-28T13:10:59.367194+0000","last_peered":"2024-03-28T13:10:59.367194+0000","last_clean":"2024-03-28T13:10:59.367194+0000","last_became_active":"2024-03-28T13:10:56.774593+0000","last_became_peered":"2024-03-28T13:10:56.774593+0000","last_unstale":"2024-03-28T13:10:59.367194+0000","last_undegraded":"2024-03-28T13:10:59.367194+0000","last_fullsized":"2024-03-28T13:10:59.367194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:50:10.939857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777345+0000","last_change":"2024-03-28T13:10:56.777966+0000","last_active":"2024-03-28T13:10:57.777345+0000","last_peered":"2024-03-28T13:10:57.777345+0000","last_clean":"2024-03-28T13:10:57.777345+0000","last_became_active":"2024-03-28T13:10:56.777658+0000","last_became_peered":"2024-03-28T13:10:56.777658+0000","last_unstale":"2024-03-28T13:10:57.777345+0000","last_undegraded":"2024-03-28T13:10:57.777345+0000","last_fullsized":"2024-03-28T13:10:57.777345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:00:01.855413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774642+0000","last_change":"2024-03-28T13:10:56.783317+0000","last_active":"2024-03-28T13:10:57.774642+0000","last_peered":"2024-03-28T13:10:57.774642+0000","last_clean":"2024-03-28T13:10:57.774642+0000","last_became_active":"2024-03-28T13:10:56.781718+0000","last_became_peered":"2024-03-28T13:10:56.781718+0000","last_unstale":"2024-03-28T13:10:57.774642+0000","last_undegraded":"2024-03-28T13:10:57.774642+0000","last_fullsized":"2024-03-28T13:10:57.774642+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:58:13.880385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783325+0000","last_change":"2024-03-28T13:10:56.779357+0000","last_active":"2024-03-28T13:10:57.783325+0000","last_peered":"2024-03-28T13:10:57.783325+0000","last_clean":"2024-03-28T13:10:57.783325+0000","last_became_active":"2024-03-28T13:10:56.779000+0000","last_became_peered":"2024-03-28T13:10:56.779000+0000","last_unstale":"2024-03-28T13:10:57.783325+0000","last_undegraded":"2024-03-28T13:10:57.783325+0000","last_fullsized":"2024-03-28T13:10:57.783325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:40:07.650748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771017+0000","last_change":"2024-03-28T13:10:56.775965+0000","last_active":"2024-03-28T13:10:57.771017+0000","last_peered":"2024-03-28T13:10:57.771017+0000","last_clean":"2024-03-28T13:10:57.771017+0000","last_became_active":"2024-03-28T13:10:56.775684+0000","last_became_peered":"2024-03-28T13:10:56.775684+0000","last_unstale":"2024-03-28T13:10:57.771017+0000","last_undegraded":"2024-03-28T13:10:57.771017+0000","last_fullsized":"2024-03-28T13:10:57.771017+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:58:13.880385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771007+0000","last_change":"2024-03-28T13:10:56.780593+0000","last_active":"2024-03-28T13:10:57.771007+0000","last_peered":"2024-03-28T13:10:57.771007+0000","last_clean":"2024-03-28T13:10:57.771007+0000","last_became_active":"2024-03-28T13:10:56.780333+0000","last_became_peered":"2024-03-28T13:10:56.780333+0000","last_unstale":"2024-03-28T13:10:57.771007+0000","last_undegraded":"2024-03-28T13:10:57.771007+0000","last_fullsized":"2024-03-28T13:10:57.771007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:45.087620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.367199+0000","last_change":"2024-03-28T13:10:56.781115+0000","last_active":"2024-03-28T13:10:59.367199+0000","last_peered":"2024-03-28T13:10:59.367199+0000","last_clean":"2024-03-28T13:10:59.367199+0000","last_became_active":"2024-03-28T13:10:56.780578+0000","last_became_peered":"2024-03-28T13:10:56.780578+0000","last_unstale":"2024-03-28T13:10:59.367199+0000","last_undegraded":"2024-03-28T13:10:59.367199+0000","last_fullsized":"2024-03-28T13:10:59.367199+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:47.019632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778583+0000","last_change":"2024-03-28T13:10:56.793373+0000","last_active":"2024-03-28T13:10:57.778583+0000","last_peered":"2024-03-28T13:10:57.778583+0000","last_clean":"2024-03-28T13:10:57.778583+0000","last_became_active":"2024-03-28T13:10:56.789448+0000","last_became_peered":"2024-03-28T13:10:56.789448+0000","last_unstale":"2024-03-28T13:10:57.778583+0000","last_undegraded":"2024-03-28T13:10:57.778583+0000","last_fullsized":"2024-03-28T13:10:57.778583+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:37.170463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265827+0000","last_change":"2024-03-28T13:10:56.785075+0000","last_active":"2024-03-28T13:10:58.265827+0000","last_peered":"2024-03-28T13:10:58.265827+0000","last_clean":"2024-03-28T13:10:58.265827+0000","last_became_active":"2024-03-28T13:10:56.784486+0000","last_became_peered":"2024-03-28T13:10:56.784486+0000","last_unstale":"2024-03-28T13:10:58.265827+0000","last_undegraded":"2024-03-28T13:10:58.265827+0000","last_fullsized":"2024-03-28T13:10:58.265827+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:05:28.235846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.367058+0000","last_change":"2024-03-28T13:10:56.771454+0000","last_active":"2024-03-28T13:10:59.367058+0000","last_peered":"2024-03-28T13:10:59.367058+0000","last_clean":"2024-03-28T13:10:59.367058+0000","last_became_active":"2024-03-28T13:10:56.771111+0000","last_became_peered":"2024-03-28T13:10:56.771111+0000","last_unstale":"2024-03-28T13:10:59.367058+0000","last_undegraded":"2024-03-28T13:10:59.367058+0000","last_fullsized":"2024-03-28T13:10:59.367058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:38:26.793165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.787568+0000","last_change":"2024-03-28T13:10:56.780719+0000","last_active":"2024-03-28T13:10:56.787568+0000","last_peered":"2024-03-28T13:10:56.787568+0000","last_clean":"2024-03-28T13:10:56.787568+0000","last_became_active":"2024-03-28T13:10:56.780369+0000","last_became_peered":"2024-03-28T13:10:56.780369+0000","last_unstale":"2024-03-28T13:10:56.787568+0000","last_undegraded":"2024-03-28T13:10:56.787568+0000","last_fullsized":"2024-03-28T13:10:56.787568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:48:42.794602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777339+0000","last_change":"2024-03-28T13:10:56.774962+0000","last_active":"2024-03-28T13:10:57.777339+0000","last_peered":"2024-03-28T13:10:57.777339+0000","last_clean":"2024-03-28T13:10:57.777339+0000","last_became_active":"2024-03-28T13:10:56.774519+0000","last_became_peered":"2024-03-28T13:10:56.774519+0000","last_unstale":"2024-03-28T13:10:57.777339+0000","last_undegraded":"2024-03-28T13:10:57.777339+0000","last_fullsized":"2024-03-28T13:10:57.777339+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:40.900533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029554+0000","last_change":"2024-03-28T13:10:56.776344+0000","last_active":"2024-03-28T13:10:58.029554+0000","last_peered":"2024-03-28T13:10:58.029554+0000","last_clean":"2024-03-28T13:10:58.029554+0000","last_became_active":"2024-03-28T13:10:56.776013+0000","last_became_peered":"2024-03-28T13:10:56.776013+0000","last_unstale":"2024-03-28T13:10:58.029554+0000","last_undegraded":"2024-03-28T13:10:58.029554+0000","last_fullsized":"2024-03-28T13:10:58.029554+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:26.233670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029127+0000","last_change":"2024-03-28T13:10:56.787487+0000","last_active":"2024-03-28T13:10:58.029127+0000","last_peered":"2024-03-28T13:10:58.029127+0000","last_clean":"2024-03-28T13:10:58.029127+0000","last_became_active":"2024-03-28T13:10:56.787012+0000","last_became_peered":"2024-03-28T13:10:56.787012+0000","last_unstale":"2024-03-28T13:10:58.029127+0000","last_undegraded":"2024-03-28T13:10:58.029127+0000","last_fullsized":"2024-03-28T13:10:58.029127+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:06:36.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265751+0000","last_change":"2024-03-28T13:10:56.778750+0000","last_active":"2024-03-28T13:10:58.265751+0000","last_peered":"2024-03-28T13:10:58.265751+0000","last_clean":"2024-03-28T13:10:58.265751+0000","last_became_active":"2024-03-28T13:10:56.778397+0000","last_became_peered":"2024-03-28T13:10:56.778397+0000","last_unstale":"2024-03-28T13:10:58.265751+0000","last_undegraded":"2024-03-28T13:10:58.265751+0000","last_fullsized":"2024-03-28T13:10:58.265751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:09:21.093205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783603+0000","last_change":"2024-03-28T13:10:56.769141+0000","last_active":"2024-03-28T13:10:57.783603+0000","last_peered":"2024-03-28T13:10:57.783603+0000","last_clean":"2024-03-28T13:10:57.783603+0000","last_became_active":"2024-03-28T13:10:56.768873+0000","last_became_peered":"2024-03-28T13:10:56.768873+0000","last_unstale":"2024-03-28T13:10:57.783603+0000","last_undegraded":"2024-03-28T13:10:57.783603+0000","last_fullsized":"2024-03-28T13:10:57.783603+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:30:13.251989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783701+0000","last_change":"2024-03-28T13:10:56.767685+0000","last_active":"2024-03-28T13:10:57.783701+0000","last_peered":"2024-03-28T13:10:57.783701+0000","last_clean":"2024-03-28T13:10:57.783701+0000","last_became_active":"2024-03-28T13:10:56.767320+0000","last_became_peered":"2024-03-28T13:10:56.767320+0000","last_unstale":"2024-03-28T13:10:57.783701+0000","last_undegraded":"2024-03-28T13:10:57.783701+0000","last_fullsized":"2024-03-28T13:10:57.783701+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:45:22.482141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.782025+0000","last_change":"2024-03-28T13:10:56.772701+0000","last_active":"2024-03-28T13:10:56.782025+0000","last_peered":"2024-03-28T13:10:56.782025+0000","last_clean":"2024-03-28T13:10:56.782025+0000","last_became_active":"2024-03-28T13:10:56.772325+0000","last_became_peered":"2024-03-28T13:10:56.772325+0000","last_unstale":"2024-03-28T13:10:56.782025+0000","last_undegraded":"2024-03-28T13:10:56.782025+0000","last_fullsized":"2024-03-28T13:10:56.782025+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:19:57.308914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029451+0000","last_change":"2024-03-28T13:10:56.785391+0000","last_active":"2024-03-28T13:10:58.029451+0000","last_peered":"2024-03-28T13:10:58.029451+0000","last_clean":"2024-03-28T13:10:58.029451+0000","last_became_active":"2024-03-28T13:10:56.782726+0000","last_became_peered":"2024-03-28T13:10:56.782726+0000","last_unstale":"2024-03-28T13:10:58.029451+0000","last_undegraded":"2024-03-28T13:10:58.029451+0000","last_fullsized":"2024-03-28T13:10:58.029451+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.770494+0000","last_change":"2024-03-28T13:10:56.788744+0000","last_active":"2024-03-28T13:10:57.770494+0000","last_peered":"2024-03-28T13:10:57.770494+0000","last_clean":"2024-03-28T13:10:57.770494+0000","last_became_active":"2024-03-28T13:10:56.788421+0000","last_became_peered":"2024-03-28T13:10:56.788421+0000","last_unstale":"2024-03-28T13:10:57.770494+0000","last_undegraded":"2024-03-28T13:10:57.770494+0000","last_fullsized":"2024-03-28T13:10:57.770494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:37:53.859227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.779607+0000","last_change":"2024-03-28T13:10:56.768323+0000","last_active":"2024-03-28T13:10:56.779607+0000","last_peered":"2024-03-28T13:10:56.779607+0000","last_clean":"2024-03-28T13:10:56.779607+0000","last_became_active":"2024-03-28T13:10:56.766260+0000","last_became_peered":"2024-03-28T13:10:56.766260+0000","last_unstale":"2024-03-28T13:10:56.779607+0000","last_undegraded":"2024-03-28T13:10:56.779607+0000","last_fullsized":"2024-03-28T13:10:56.779607+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:55:35.027128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.770677+0000","last_change":"2024-03-28T13:10:56.789986+0000","last_active":"2024-03-28T13:10:57.770677+0000","last_peered":"2024-03-28T13:10:57.770677+0000","last_clean":"2024-03-28T13:10:57.770677+0000","last_became_active":"2024-03-28T13:10:56.789691+0000","last_became_peered":"2024-03-28T13:10:56.789691+0000","last_unstale":"2024-03-28T13:10:57.770677+0000","last_undegraded":"2024-03-28T13:10:57.770677+0000","last_fullsized":"2024-03-28T13:10:57.770677+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:52:10.929698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778158+0000","last_change":"2024-03-28T13:10:56.780679+0000","last_active":"2024-03-28T13:10:57.778158+0000","last_peered":"2024-03-28T13:10:57.778158+0000","last_clean":"2024-03-28T13:10:57.778158+0000","last_became_active":"2024-03-28T13:10:56.777813+0000","last_became_peered":"2024-03-28T13:10:56.777813+0000","last_unstale":"2024-03-28T13:10:57.778158+0000","last_undegraded":"2024-03-28T13:10:57.778158+0000","last_fullsized":"2024-03-28T13:10:57.778158+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:15:57.044511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778939+0000","last_change":"2024-03-28T13:10:56.781082+0000","last_active":"2024-03-28T13:10:57.778939+0000","last_peered":"2024-03-28T13:10:57.778939+0000","last_clean":"2024-03-28T13:10:57.778939+0000","last_became_active":"2024-03-28T13:10:56.780806+0000","last_became_peered":"2024-03-28T13:10:56.780806+0000","last_unstale":"2024-03-28T13:10:57.778939+0000","last_undegraded":"2024-03-28T13:10:57.778939+0000","last_fullsized":"2024-03-28T13:10:57.778939+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:33.779798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774874+0000","last_change":"2024-03-28T13:10:56.771825+0000","last_active":"2024-03-28T13:10:57.774874+0000","last_peered":"2024-03-28T13:10:57.774874+0000","last_clean":"2024-03-28T13:10:57.774874+0000","last_became_active":"2024-03-28T13:10:56.771505+0000","last_became_peered":"2024-03-28T13:10:56.771505+0000","last_unstale":"2024-03-28T13:10:57.774874+0000","last_undegraded":"2024-03-28T13:10:57.774874+0000","last_fullsized":"2024-03-28T13:10:57.774874+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771592+0000","last_change":"2024-03-28T13:10:56.784002+0000","last_active":"2024-03-28T13:10:57.771592+0000","last_peered":"2024-03-28T13:10:57.771592+0000","last_clean":"2024-03-28T13:10:57.771592+0000","last_became_active":"2024-03-28T13:10:56.783731+0000","last_became_peered":"2024-03-28T13:10:56.783731+0000","last_unstale":"2024-03-28T13:10:57.771592+0000","last_undegraded":"2024-03-28T13:10:57.771592+0000","last_fullsized":"2024-03-28T13:10:57.771592+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029694+0000","last_change":"2024-03-28T13:10:56.788313+0000","last_active":"2024-03-28T13:10:58.029694+0000","last_peered":"2024-03-28T13:10:58.029694+0000","last_clean":"2024-03-28T13:10:58.029694+0000","last_became_active":"2024-03-28T13:10:56.787982+0000","last_became_peered":"2024-03-28T13:10:56.787982+0000","last_unstale":"2024-03-28T13:10:58.029694+0000","last_undegraded":"2024-03-28T13:10:58.029694+0000","last_fullsized":"2024-03-28T13:10:58.029694+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:00:01.855413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029188+0000","last_change":"2024-03-28T13:10:56.779668+0000","last_active":"2024-03-28T13:10:58.029188+0000","last_peered":"2024-03-28T13:10:58.029188+0000","last_clean":"2024-03-28T13:10:58.029188+0000","last_became_active":"2024-03-28T13:10:56.778978+0000","last_became_peered":"2024-03-28T13:10:56.778978+0000","last_unstale":"2024-03-28T13:10:58.029188+0000","last_undegraded":"2024-03-28T13:10:58.029188+0000","last_fullsized":"2024-03-28T13:10:58.029188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.770758+0000","last_change":"2024-03-28T13:10:56.788094+0000","last_active":"2024-03-28T13:10:57.770758+0000","last_peered":"2024-03-28T13:10:57.770758+0000","last_clean":"2024-03-28T13:10:57.770758+0000","last_became_active":"2024-03-28T13:10:56.787812+0000","last_became_peered":"2024-03-28T13:10:56.787812+0000","last_unstale":"2024-03-28T13:10:57.770758+0000","last_undegraded":"2024-03-28T13:10:57.770758+0000","last_fullsized":"2024-03-28T13:10:57.770758+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:37:14.452246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265005+0000","last_change":"2024-03-28T13:10:56.771703+0000","last_active":"2024-03-28T13:10:58.265005+0000","last_peered":"2024-03-28T13:10:58.265005+0000","last_clean":"2024-03-28T13:10:58.265005+0000","last_became_active":"2024-03-28T13:10:56.769956+0000","last_became_peered":"2024-03-28T13:10:56.769956+0000","last_unstale":"2024-03-28T13:10:58.265005+0000","last_undegraded":"2024-03-28T13:10:58.265005+0000","last_fullsized":"2024-03-28T13:10:58.265005+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:09:25.565676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.778739+0000","last_change":"2024-03-28T13:10:56.769275+0000","last_active":"2024-03-28T13:10:56.778739+0000","last_peered":"2024-03-28T13:10:56.778739+0000","last_clean":"2024-03-28T13:10:56.778739+0000","last_became_active":"2024-03-28T13:10:56.767794+0000","last_became_peered":"2024-03-28T13:10:56.767794+0000","last_unstale":"2024-03-28T13:10:56.778739+0000","last_undegraded":"2024-03-28T13:10:56.778739+0000","last_fullsized":"2024-03-28T13:10:56.778739+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:37:33.635609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265751+0000","last_change":"2024-03-28T13:10:55.740050+0000","last_active":"2024-03-28T13:10:58.265751+0000","last_peered":"2024-03-28T13:10:58.265751+0000","last_clean":"2024-03-28T13:10:58.265751+0000","last_became_active":"2024-03-28T13:10:55.739731+0000","last_became_peered":"2024-03-28T13:10:55.739731+0000","last_unstale":"2024-03-28T13:10:58.265751+0000","last_undegraded":"2024-03-28T13:10:58.265751+0000","last_fullsized":"2024-03-28T13:10:58.265751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:22.907948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771663+0000","last_change":"2024-03-28T13:10:56.786361+0000","last_active":"2024-03-28T13:10:57.771663+0000","last_peered":"2024-03-28T13:10:57.771663+0000","last_clean":"2024-03-28T13:10:57.771663+0000","last_became_active":"2024-03-28T13:10:56.786029+0000","last_became_peered":"2024-03-28T13:10:56.786029+0000","last_unstale":"2024-03-28T13:10:57.771663+0000","last_undegraded":"2024-03-28T13:10:57.771663+0000","last_fullsized":"2024-03-28T13:10:57.771663+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:40.900533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.779205+0000","last_change":"2024-03-28T13:10:55.750470+0000","last_active":"2024-03-28T13:10:57.779205+0000","last_peered":"2024-03-28T13:10:57.779205+0000","last_clean":"2024-03-28T13:10:57.779205+0000","last_became_active":"2024-03-28T13:10:55.749676+0000","last_became_peered":"2024-03-28T13:10:55.749676+0000","last_unstale":"2024-03-28T13:10:57.779205+0000","last_undegraded":"2024-03-28T13:10:57.779205+0000","last_fullsized":"2024-03-28T13:10:57.779205+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:55:34.014926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771664+0000","last_change":"2024-03-28T13:10:56.793622+0000","last_active":"2024-03-28T13:10:57.771664+0000","last_peered":"2024-03-28T13:10:57.771664+0000","last_clean":"2024-03-28T13:10:57.771664+0000","last_became_active":"2024-03-28T13:10:56.793351+0000","last_became_peered":"2024-03-28T13:10:56.793351+0000","last_unstale":"2024-03-28T13:10:57.771664+0000","last_undegraded":"2024-03-28T13:10:57.771664+0000","last_fullsized":"2024-03-28T13:10:57.771664+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:00:01.855413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783942+0000","last_change":"2024-03-28T13:10:55.754683+0000","last_active":"2024-03-28T13:10:57.783942+0000","last_peered":"2024-03-28T13:10:57.783942+0000","last_clean":"2024-03-28T13:10:57.783942+0000","last_became_active":"2024-03-28T13:10:55.754344+0000","last_became_peered":"2024-03-28T13:10:55.754344+0000","last_unstale":"2024-03-28T13:10:57.783942+0000","last_undegraded":"2024-03-28T13:10:57.783942+0000","last_fullsized":"2024-03-28T13:10:57.783942+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:22:12.385800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777982+0000","last_change":"2024-03-28T13:10:56.779261+0000","last_active":"2024-03-28T13:10:57.777982+0000","last_peered":"2024-03-28T13:10:57.777982+0000","last_clean":"2024-03-28T13:10:57.777982+0000","last_became_active":"2024-03-28T13:10:56.778899+0000","last_became_peered":"2024-03-28T13:10:56.778899+0000","last_unstale":"2024-03-28T13:10:57.777982+0000","last_undegraded":"2024-03-28T13:10:57.777982+0000","last_fullsized":"2024-03-28T13:10:57.777982+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:44:06.086773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777813+0000","last_change":"2024-03-28T13:10:55.755603+0000","last_active":"2024-03-28T13:10:57.777813+0000","last_peered":"2024-03-28T13:10:57.777813+0000","last_clean":"2024-03-28T13:10:57.777813+0000","last_became_active":"2024-03-28T13:10:55.754244+0000","last_became_peered":"2024-03-28T13:10:55.754244+0000","last_unstale":"2024-03-28T13:10:57.777813+0000","last_undegraded":"2024-03-28T13:10:57.777813+0000","last_fullsized":"2024-03-28T13:10:57.777813+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:30.977315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366724+0000","last_change":"2024-03-28T13:10:56.776082+0000","last_active":"2024-03-28T13:10:59.366724+0000","last_peered":"2024-03-28T13:10:59.366724+0000","last_clean":"2024-03-28T13:10:59.366724+0000","last_became_active":"2024-03-28T13:10:56.775495+0000","last_became_peered":"2024-03-28T13:10:56.775495+0000","last_unstale":"2024-03-28T13:10:59.366724+0000","last_undegraded":"2024-03-28T13:10:59.366724+0000","last_fullsized":"2024-03-28T13:10:59.366724+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:39.445190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771537+0000","last_change":"2024-03-28T13:10:55.745761+0000","last_active":"2024-03-28T13:10:57.771537+0000","last_peered":"2024-03-28T13:10:57.771537+0000","last_clean":"2024-03-28T13:10:57.771537+0000","last_became_active":"2024-03-28T13:10:55.745492+0000","last_became_peered":"2024-03-28T13:10:55.745492+0000","last_unstale":"2024-03-28T13:10:57.771537+0000","last_undegraded":"2024-03-28T13:10:57.771537+0000","last_fullsized":"2024-03-28T13:10:57.771537+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:57:31.432582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771652+0000","last_change":"2024-03-28T13:10:56.783694+0000","last_active":"2024-03-28T13:10:57.771652+0000","last_peered":"2024-03-28T13:10:57.771652+0000","last_clean":"2024-03-28T13:10:57.771652+0000","last_became_active":"2024-03-28T13:10:56.783336+0000","last_became_peered":"2024-03-28T13:10:56.783336+0000","last_unstale":"2024-03-28T13:10:57.771652+0000","last_undegraded":"2024-03-28T13:10:57.771652+0000","last_fullsized":"2024-03-28T13:10:57.771652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.770706+0000","last_change":"2024-03-28T13:10:55.753085+0000","last_active":"2024-03-28T13:10:57.770706+0000","last_peered":"2024-03-28T13:10:57.770706+0000","last_clean":"2024-03-28T13:10:57.770706+0000","last_became_active":"2024-03-28T13:10:55.752806+0000","last_became_peered":"2024-03-28T13:10:55.752806+0000","last_unstale":"2024-03-28T13:10:57.770706+0000","last_undegraded":"2024-03-28T13:10:57.770706+0000","last_fullsized":"2024-03-28T13:10:57.770706+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:46.007430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366338+0000","last_change":"2024-03-28T13:10:56.769125+0000","last_active":"2024-03-28T13:10:59.366338+0000","last_peered":"2024-03-28T13:10:59.366338+0000","last_clean":"2024-03-28T13:10:59.366338+0000","last_became_active":"2024-03-28T13:10:56.768806+0000","last_became_peered":"2024-03-28T13:10:56.768806+0000","last_unstale":"2024-03-28T13:10:59.366338+0000","last_undegraded":"2024-03-28T13:10:59.366338+0000","last_fullsized":"2024-03-28T13:10:59.366338+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:57:20.895736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783436+0000","last_change":"2024-03-28T13:10:55.750960+0000","last_active":"2024-03-28T13:10:57.783436+0000","last_peered":"2024-03-28T13:10:57.783436+0000","last_clean":"2024-03-28T13:10:57.783436+0000","last_became_active":"2024-03-28T13:10:55.750618+0000","last_became_peered":"2024-03-28T13:10:55.750618+0000","last_unstale":"2024-03-28T13:10:57.783436+0000","last_undegraded":"2024-03-28T13:10:57.783436+0000","last_fullsized":"2024-03-28T13:10:57.783436+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:36.018433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776802+0000","last_change":"2024-03-28T13:10:56.775408+0000","last_active":"2024-03-28T13:10:57.776802+0000","last_peered":"2024-03-28T13:10:57.776802+0000","last_clean":"2024-03-28T13:10:57.776802+0000","last_became_active":"2024-03-28T13:10:56.775141+0000","last_became_peered":"2024-03-28T13:10:56.775141+0000","last_unstale":"2024-03-28T13:10:57.776802+0000","last_undegraded":"2024-03-28T13:10:57.776802+0000","last_fullsized":"2024-03-28T13:10:57.776802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771390+0000","last_change":"2024-03-28T13:10:55.752431+0000","last_active":"2024-03-28T13:10:57.771390+0000","last_peered":"2024-03-28T13:10:57.771390+0000","last_clean":"2024-03-28T13:10:57.771390+0000","last_became_active":"2024-03-28T13:10:55.752001+0000","last_became_peered":"2024-03-28T13:10:55.752001+0000","last_unstale":"2024-03-28T13:10:57.771390+0000","last_undegraded":"2024-03-28T13:10:57.771390+0000","last_fullsized":"2024-03-28T13:10:57.771390+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:38.432989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265800+0000","last_change":"2024-03-28T13:10:56.775017+0000","last_active":"2024-03-28T13:10:58.265800+0000","last_peered":"2024-03-28T13:10:58.265800+0000","last_clean":"2024-03-28T13:10:58.265800+0000","last_became_active":"2024-03-28T13:10:56.773930+0000","last_became_peered":"2024-03-28T13:10:56.773930+0000","last_unstale":"2024-03-28T13:10:58.265800+0000","last_undegraded":"2024-03-28T13:10:58.265800+0000","last_fullsized":"2024-03-28T13:10:58.265800+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:36:18.347550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.775253+0000","last_change":"2024-03-28T13:10:55.741344+0000","last_active":"2024-03-28T13:10:57.775253+0000","last_peered":"2024-03-28T13:10:57.775253+0000","last_clean":"2024-03-28T13:10:57.775253+0000","last_became_active":"2024-03-28T13:10:55.740955+0000","last_became_peered":"2024-03-28T13:10:55.740955+0000","last_unstale":"2024-03-28T13:10:57.775253+0000","last_undegraded":"2024-03-28T13:10:57.775253+0000","last_fullsized":"2024-03-28T13:10:57.775253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:55:33.765625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778342+0000","last_change":"2024-03-28T13:10:56.792475+0000","last_active":"2024-03-28T13:10:57.778342+0000","last_peered":"2024-03-28T13:10:57.778342+0000","last_clean":"2024-03-28T13:10:57.778342+0000","last_became_active":"2024-03-28T13:10:56.792165+0000","last_became_peered":"2024-03-28T13:10:56.792165+0000","last_unstale":"2024-03-28T13:10:57.778342+0000","last_undegraded":"2024-03-28T13:10:57.778342+0000","last_fullsized":"2024-03-28T13:10:57.778342+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:50:07.655844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774840+0000","last_change":"2024-03-28T13:10:55.749119+0000","last_active":"2024-03-28T13:10:57.774840+0000","last_peered":"2024-03-28T13:10:57.774840+0000","last_clean":"2024-03-28T13:10:57.774840+0000","last_became_active":"2024-03-28T13:10:55.748527+0000","last_became_peered":"2024-03-28T13:10:55.748527+0000","last_unstale":"2024-03-28T13:10:57.774840+0000","last_undegraded":"2024-03-28T13:10:57.774840+0000","last_fullsized":"2024-03-28T13:10:57.774840+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:30.977315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777568+0000","last_change":"2024-03-28T13:10:56.778342+0000","last_active":"2024-03-28T13:10:57.777568+0000","last_peered":"2024-03-28T13:10:57.777568+0000","last_clean":"2024-03-28T13:10:57.777568+0000","last_became_active":"2024-03-28T13:10:56.777259+0000","last_became_peered":"2024-03-28T13:10:56.777259+0000","last_unstale":"2024-03-28T13:10:57.777568+0000","last_undegraded":"2024-03-28T13:10:57.777568+0000","last_fullsized":"2024-03-28T13:10:57.777568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:23.920150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265970+0000","last_change":"2024-03-28T13:10:55.737368+0000","last_active":"2024-03-28T13:10:58.265970+0000","last_peered":"2024-03-28T13:10:58.265970+0000","last_clean":"2024-03-28T13:10:58.265970+0000","last_became_active":"2024-03-28T13:10:55.736922+0000","last_became_peered":"2024-03-28T13:10:55.736922+0000","last_unstale":"2024-03-28T13:10:58.265970+0000","last_undegraded":"2024-03-28T13:10:58.265970+0000","last_fullsized":"2024-03-28T13:10:58.265970+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:39.888331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265780+0000","last_change":"2024-03-28T13:10:56.785370+0000","last_active":"2024-03-28T13:10:58.265780+0000","last_peered":"2024-03-28T13:10:58.265780+0000","last_clean":"2024-03-28T13:10:58.265780+0000","last_became_active":"2024-03-28T13:10:56.784822+0000","last_became_peered":"2024-03-28T13:10:56.784822+0000","last_unstale":"2024-03-28T13:10:58.265780+0000","last_undegraded":"2024-03-28T13:10:58.265780+0000","last_fullsized":"2024-03-28T13:10:58.265780+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:04:04.193445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.804651+0000","last_change":"2024-03-28T13:10:55.757421+0000","last_active":"2024-03-28T13:10:56.804651+0000","last_peered":"2024-03-28T13:10:56.804651+0000","last_clean":"2024-03-28T13:10:56.804651+0000","last_became_active":"2024-03-28T13:10:55.757041+0000","last_became_peered":"2024-03-28T13:10:55.757041+0000","last_unstale":"2024-03-28T13:10:56.804651+0000","last_undegraded":"2024-03-28T13:10:56.804651+0000","last_fullsized":"2024-03-28T13:10:56.804651+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:46.007430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.777866+0000","last_change":"2024-03-28T13:10:56.767158+0000","last_active":"2024-03-28T13:10:56.777866+0000","last_peered":"2024-03-28T13:10:56.777866+0000","last_clean":"2024-03-28T13:10:56.777866+0000","last_became_active":"2024-03-28T13:10:56.762220+0000","last_became_peered":"2024-03-28T13:10:56.762220+0000","last_unstale":"2024-03-28T13:10:56.777866+0000","last_undegraded":"2024-03-28T13:10:56.777866+0000","last_fullsized":"2024-03-28T13:10:56.777866+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:36:29.152136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771783+0000","last_change":"2024-03-28T13:10:55.742612+0000","last_active":"2024-03-28T13:10:57.771783+0000","last_peered":"2024-03-28T13:10:57.771783+0000","last_clean":"2024-03-28T13:10:57.771783+0000","last_became_active":"2024-03-28T13:10:55.742344+0000","last_became_peered":"2024-03-28T13:10:55.742344+0000","last_unstale":"2024-03-28T13:10:57.771783+0000","last_undegraded":"2024-03-28T13:10:57.771783+0000","last_fullsized":"2024-03-28T13:10:57.771783+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:38:25.780963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777175+0000","last_change":"2024-03-28T13:10:56.791145+0000","last_active":"2024-03-28T13:10:57.777175+0000","last_peered":"2024-03-28T13:10:57.777175+0000","last_clean":"2024-03-28T13:10:57.777175+0000","last_became_active":"2024-03-28T13:10:56.789400+0000","last_became_peered":"2024-03-28T13:10:56.789400+0000","last_unstale":"2024-03-28T13:10:57.777175+0000","last_undegraded":"2024-03-28T13:10:57.777175+0000","last_fullsized":"2024-03-28T13:10:57.777175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:26.233670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777676+0000","last_change":"2024-03-28T13:10:55.750434+0000","last_active":"2024-03-28T13:10:57.777676+0000","last_peered":"2024-03-28T13:10:57.777676+0000","last_clean":"2024-03-28T13:10:57.777676+0000","last_became_active":"2024-03-28T13:10:55.750086+0000","last_became_peered":"2024-03-28T13:10:55.750086+0000","last_unstale":"2024-03-28T13:10:57.777676+0000","last_undegraded":"2024-03-28T13:10:57.777676+0000","last_fullsized":"2024-03-28T13:10:57.777676+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:37:32.623408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771101+0000","last_change":"2024-03-28T13:10:56.779238+0000","last_active":"2024-03-28T13:10:57.771101+0000","last_peered":"2024-03-28T13:10:57.771101+0000","last_clean":"2024-03-28T13:10:57.771101+0000","last_became_active":"2024-03-28T13:10:56.773883+0000","last_became_peered":"2024-03-28T13:10:56.773883+0000","last_unstale":"2024-03-28T13:10:57.771101+0000","last_undegraded":"2024-03-28T13:10:57.771101+0000","last_fullsized":"2024-03-28T13:10:57.771101+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:06:36.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776667+0000","last_change":"2024-03-28T13:10:55.755962+0000","last_active":"2024-03-28T13:10:57.776667+0000","last_peered":"2024-03-28T13:10:57.776667+0000","last_clean":"2024-03-28T13:10:57.776667+0000","last_became_active":"2024-03-28T13:10:55.755652+0000","last_became_peered":"2024-03-28T13:10:55.755652+0000","last_unstale":"2024-03-28T13:10:57.776667+0000","last_undegraded":"2024-03-28T13:10:57.776667+0000","last_fullsized":"2024-03-28T13:10:57.776667+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:51:08.557038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265334+0000","last_change":"2024-03-28T13:10:56.772096+0000","last_active":"2024-03-28T13:10:58.265334+0000","last_peered":"2024-03-28T13:10:58.265334+0000","last_clean":"2024-03-28T13:10:58.265334+0000","last_became_active":"2024-03-28T13:10:56.771758+0000","last_became_peered":"2024-03-28T13:10:56.771758+0000","last_unstale":"2024-03-28T13:10:58.265334+0000","last_undegraded":"2024-03-28T13:10:58.265334+0000","last_fullsized":"2024-03-28T13:10:58.265334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:31:34.008079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783479+0000","last_change":"2024-03-28T13:10:55.757692+0000","last_active":"2024-03-28T13:10:57.783479+0000","last_peered":"2024-03-28T13:10:57.783479+0000","last_clean":"2024-03-28T13:10:57.783479+0000","last_became_active":"2024-03-28T13:10:55.757463+0000","last_became_peered":"2024-03-28T13:10:55.757463+0000","last_unstale":"2024-03-28T13:10:57.783479+0000","last_undegraded":"2024-03-28T13:10:57.783479+0000","last_fullsized":"2024-03-28T13:10:57.783479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:12:56.307494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265756+0000","last_change":"2024-03-28T13:10:56.793741+0000","last_active":"2024-03-28T13:10:58.265756+0000","last_peered":"2024-03-28T13:10:58.265756+0000","last_clean":"2024-03-28T13:10:58.265756+0000","last_became_active":"2024-03-28T13:10:56.793473+0000","last_became_peered":"2024-03-28T13:10:56.793473+0000","last_unstale":"2024-03-28T13:10:58.265756+0000","last_undegraded":"2024-03-28T13:10:58.265756+0000","last_fullsized":"2024-03-28T13:10:58.265756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:38:30.086095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783676+0000","last_change":"2024-03-28T13:10:55.757407+0000","last_active":"2024-03-28T13:10:57.783676+0000","last_peered":"2024-03-28T13:10:57.783676+0000","last_clean":"2024-03-28T13:10:57.783676+0000","last_became_active":"2024-03-28T13:10:55.757088+0000","last_became_peered":"2024-03-28T13:10:55.757088+0000","last_unstale":"2024-03-28T13:10:57.783676+0000","last_undegraded":"2024-03-28T13:10:57.783676+0000","last_fullsized":"2024-03-28T13:10:57.783676+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:03:35.637042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.367177+0000","last_change":"2024-03-28T13:10:56.772822+0000","last_active":"2024-03-28T13:10:59.367177+0000","last_peered":"2024-03-28T13:10:59.367177+0000","last_clean":"2024-03-28T13:10:59.367177+0000","last_became_active":"2024-03-28T13:10:56.772511+0000","last_became_peered":"2024-03-28T13:10:56.772511+0000","last_unstale":"2024-03-28T13:10:59.367177+0000","last_undegraded":"2024-03-28T13:10:59.367177+0000","last_fullsized":"2024-03-28T13:10:59.367177+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:57:32.444784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771480+0000","last_change":"2024-03-28T13:10:55.748363+0000","last_active":"2024-03-28T13:10:57.771480+0000","last_peered":"2024-03-28T13:10:57.771480+0000","last_clean":"2024-03-28T13:10:57.771480+0000","last_became_active":"2024-03-28T13:10:55.748084+0000","last_became_peered":"2024-03-28T13:10:55.748084+0000","last_unstale":"2024-03-28T13:10:57.771480+0000","last_undegraded":"2024-03-28T13:10:57.771480+0000","last_fullsized":"2024-03-28T13:10:57.771480+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:58.262597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774512+0000","last_change":"2024-03-28T13:10:56.774113+0000","last_active":"2024-03-28T13:10:57.774512+0000","last_peered":"2024-03-28T13:10:57.774512+0000","last_clean":"2024-03-28T13:10:57.774512+0000","last_became_active":"2024-03-28T13:10:56.773838+0000","last_became_peered":"2024-03-28T13:10:56.773838+0000","last_unstale":"2024-03-28T13:10:57.774512+0000","last_undegraded":"2024-03-28T13:10:57.774512+0000","last_fullsized":"2024-03-28T13:10:57.774512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:45.087620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265939+0000","last_change":"2024-03-28T13:10:55.746355+0000","last_active":"2024-03-28T13:10:58.265939+0000","last_peered":"2024-03-28T13:10:58.265939+0000","last_clean":"2024-03-28T13:10:58.265939+0000","last_became_active":"2024-03-28T13:10:55.744858+0000","last_became_peered":"2024-03-28T13:10:55.744858+0000","last_unstale":"2024-03-28T13:10:58.265939+0000","last_undegraded":"2024-03-28T13:10:58.265939+0000","last_fullsized":"2024-03-28T13:10:58.265939+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:00:00.843211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783690+0000","last_change":"2024-03-28T13:10:56.779040+0000","last_active":"2024-03-28T13:10:57.783690+0000","last_peered":"2024-03-28T13:10:57.783690+0000","last_clean":"2024-03-28T13:10:57.783690+0000","last_became_active":"2024-03-28T13:10:56.777765+0000","last_became_peered":"2024-03-28T13:10:56.777765+0000","last_unstale":"2024-03-28T13:10:57.783690+0000","last_undegraded":"2024-03-28T13:10:57.783690+0000","last_fullsized":"2024-03-28T13:10:57.783690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:44.685445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029121+0000","last_change":"2024-03-28T13:10:55.753629+0000","last_active":"2024-03-28T13:10:58.029121+0000","last_peered":"2024-03-28T13:10:58.029121+0000","last_clean":"2024-03-28T13:10:58.029121+0000","last_became_active":"2024-03-28T13:10:55.753330+0000","last_became_peered":"2024-03-28T13:10:55.753330+0000","last_unstale":"2024-03-28T13:10:58.029121+0000","last_undegraded":"2024-03-28T13:10:58.029121+0000","last_fullsized":"2024-03-28T13:10:58.029121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:30.977315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.781976+0000","last_change":"2024-03-28T13:10:56.774575+0000","last_active":"2024-03-28T13:10:56.781976+0000","last_peered":"2024-03-28T13:10:56.781976+0000","last_clean":"2024-03-28T13:10:56.781976+0000","last_became_active":"2024-03-28T13:10:56.772544+0000","last_became_peered":"2024-03-28T13:10:56.772544+0000","last_unstale":"2024-03-28T13:10:56.781976+0000","last_undegraded":"2024-03-28T13:10:56.781976+0000","last_fullsized":"2024-03-28T13:10:56.781976+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:29:15.008724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.775058+0000","last_change":"2024-03-28T13:10:55.754015+0000","last_active":"2024-03-28T13:10:57.775058+0000","last_peered":"2024-03-28T13:10:57.775058+0000","last_clean":"2024-03-28T13:10:57.775058+0000","last_became_active":"2024-03-28T13:10:55.753701+0000","last_became_peered":"2024-03-28T13:10:55.753701+0000","last_unstale":"2024-03-28T13:10:57.775058+0000","last_undegraded":"2024-03-28T13:10:57.775058+0000","last_fullsized":"2024-03-28T13:10:57.775058+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:29:13.996522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.775071+0000","last_change":"2024-03-28T13:10:56.782871+0000","last_active":"2024-03-28T13:10:57.775071+0000","last_peered":"2024-03-28T13:10:57.775071+0000","last_clean":"2024-03-28T13:10:57.775071+0000","last_became_active":"2024-03-28T13:10:56.782020+0000","last_became_peered":"2024-03-28T13:10:56.782020+0000","last_unstale":"2024-03-28T13:10:57.775071+0000","last_undegraded":"2024-03-28T13:10:57.775071+0000","last_fullsized":"2024-03-28T13:10:57.775071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771032+0000","last_change":"2024-03-28T13:10:55.744963+0000","last_active":"2024-03-28T13:10:57.771032+0000","last_peered":"2024-03-28T13:10:57.771032+0000","last_clean":"2024-03-28T13:10:57.771032+0000","last_became_active":"2024-03-28T13:10:55.743998+0000","last_became_peered":"2024-03-28T13:10:55.743998+0000","last_unstale":"2024-03-28T13:10:57.771032+0000","last_undegraded":"2024-03-28T13:10:57.771032+0000","last_fullsized":"2024-03-28T13:10:57.771032+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:50:09.927656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.787619+0000","last_change":"2024-03-28T13:10:56.780543+0000","last_active":"2024-03-28T13:10:56.787619+0000","last_peered":"2024-03-28T13:10:56.787619+0000","last_clean":"2024-03-28T13:10:56.787619+0000","last_became_active":"2024-03-28T13:10:56.780189+0000","last_became_peered":"2024-03-28T13:10:56.780189+0000","last_unstale":"2024-03-28T13:10:56.787619+0000","last_undegraded":"2024-03-28T13:10:56.787619+0000","last_fullsized":"2024-03-28T13:10:56.787619+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:31.989516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366777+0000","last_change":"2024-03-28T13:10:55.748693+0000","last_active":"2024-03-28T13:10:59.366777+0000","last_peered":"2024-03-28T13:10:59.366777+0000","last_clean":"2024-03-28T13:10:59.366777+0000","last_became_active":"2024-03-28T13:10:55.748357+0000","last_became_peered":"2024-03-28T13:10:55.748357+0000","last_unstale":"2024-03-28T13:10:59.366777+0000","last_undegraded":"2024-03-28T13:10:59.366777+0000","last_fullsized":"2024-03-28T13:10:59.366777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:16:57.647981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366596+0000","last_change":"2024-03-28T13:10:56.770455+0000","last_active":"2024-03-28T13:10:59.366596+0000","last_peered":"2024-03-28T13:10:59.366596+0000","last_clean":"2024-03-28T13:10:59.366596+0000","last_became_active":"2024-03-28T13:10:56.770174+0000","last_became_peered":"2024-03-28T13:10:56.770174+0000","last_unstale":"2024-03-28T13:10:59.366596+0000","last_undegraded":"2024-03-28T13:10:59.366596+0000","last_fullsized":"2024-03-28T13:10:59.366596+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:59.274798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.264885+0000","last_change":"2024-03-28T13:10:55.743425+0000","last_active":"2024-03-28T13:10:58.264885+0000","last_peered":"2024-03-28T13:10:58.264885+0000","last_clean":"2024-03-28T13:10:58.264885+0000","last_became_active":"2024-03-28T13:10:55.743112+0000","last_became_peered":"2024-03-28T13:10:55.743112+0000","last_unstale":"2024-03-28T13:10:58.264885+0000","last_undegraded":"2024-03-28T13:10:58.264885+0000","last_fullsized":"2024-03-28T13:10:58.264885+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:25.221468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783615+0000","last_change":"2024-03-28T13:10:56.775289+0000","last_active":"2024-03-28T13:10:57.783615+0000","last_peered":"2024-03-28T13:10:57.783615+0000","last_clean":"2024-03-28T13:10:57.783615+0000","last_became_active":"2024-03-28T13:10:56.774549+0000","last_became_peered":"2024-03-28T13:10:56.774549+0000","last_unstale":"2024-03-28T13:10:57.783615+0000","last_undegraded":"2024-03-28T13:10:57.783615+0000","last_fullsized":"2024-03-28T13:10:57.783615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:16.475067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.794905+0000","last_change":"2024-03-28T13:10:55.755495+0000","last_active":"2024-03-28T13:10:56.794905+0000","last_peered":"2024-03-28T13:10:56.794905+0000","last_clean":"2024-03-28T13:10:56.794905+0000","last_became_active":"2024-03-28T13:10:55.754987+0000","last_became_peered":"2024-03-28T13:10:55.754987+0000","last_unstale":"2024-03-28T13:10:56.794905+0000","last_undegraded":"2024-03-28T13:10:56.794905+0000","last_fullsized":"2024-03-28T13:10:56.794905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:57:31.432582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265584+0000","last_change":"2024-03-28T13:10:56.776492+0000","last_active":"2024-03-28T13:10:58.265584+0000","last_peered":"2024-03-28T13:10:58.265584+0000","last_clean":"2024-03-28T13:10:58.265584+0000","last_became_active":"2024-03-28T13:10:56.774511+0000","last_became_peered":"2024-03-28T13:10:56.774511+0000","last_unstale":"2024-03-28T13:10:58.265584+0000","last_undegraded":"2024-03-28T13:10:58.265584+0000","last_fullsized":"2024-03-28T13:10:58.265584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:09:01.845737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783204+0000","last_change":"2024-03-28T13:10:55.748821+0000","last_active":"2024-03-28T13:10:57.783204+0000","last_peered":"2024-03-28T13:10:57.783204+0000","last_clean":"2024-03-28T13:10:57.783204+0000","last_became_active":"2024-03-28T13:10:55.748128+0000","last_became_peered":"2024-03-28T13:10:55.748128+0000","last_unstale":"2024-03-28T13:10:57.783204+0000","last_undegraded":"2024-03-28T13:10:57.783204+0000","last_fullsized":"2024-03-28T13:10:57.783204+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:48:41.782400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783822+0000","last_change":"2024-03-28T13:10:56.785728+0000","last_active":"2024-03-28T13:10:57.783822+0000","last_peered":"2024-03-28T13:10:57.783822+0000","last_clean":"2024-03-28T13:10:57.783822+0000","last_became_active":"2024-03-28T13:10:56.783504+0000","last_became_peered":"2024-03-28T13:10:56.783504+0000","last_unstale":"2024-03-28T13:10:57.783822+0000","last_undegraded":"2024-03-28T13:10:57.783822+0000","last_fullsized":"2024-03-28T13:10:57.783822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:42:20.735994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.747889+0000","last_change":"2024-03-28T13:10:55.757201+0000","last_active":"2024-03-28T13:10:56.747889+0000","last_peered":"2024-03-28T13:10:56.747889+0000","last_clean":"2024-03-28T13:10:56.747889+0000","last_became_active":"2024-03-28T13:10:55.756328+0000","last_became_peered":"2024-03-28T13:10:55.756328+0000","last_unstale":"2024-03-28T13:10:56.747889+0000","last_undegraded":"2024-03-28T13:10:56.747889+0000","last_fullsized":"2024-03-28T13:10:56.747889+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:50:09.927656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029820+0000","last_change":"2024-03-28T13:10:56.789219+0000","last_active":"2024-03-28T13:10:58.029820+0000","last_peered":"2024-03-28T13:10:58.029820+0000","last_clean":"2024-03-28T13:10:58.029820+0000","last_became_active":"2024-03-28T13:10:56.783607+0000","last_became_peered":"2024-03-28T13:10:56.783607+0000","last_unstale":"2024-03-28T13:10:58.029820+0000","last_undegraded":"2024-03-28T13:10:58.029820+0000","last_fullsized":"2024-03-28T13:10:58.029820+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:40.900533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.747164+0000","last_change":"2024-03-28T13:10:55.753645+0000","last_active":"2024-03-28T13:10:56.747164+0000","last_peered":"2024-03-28T13:10:56.747164+0000","last_clean":"2024-03-28T13:10:56.747164+0000","last_became_active":"2024-03-28T13:10:55.753103+0000","last_became_peered":"2024-03-28T13:10:55.753103+0000","last_unstale":"2024-03-28T13:10:56.747164+0000","last_undegraded":"2024-03-28T13:10:56.747164+0000","last_fullsized":"2024-03-28T13:10:56.747164+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:38:25.780963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777482+0000","last_change":"2024-03-28T13:10:56.784216+0000","last_active":"2024-03-28T13:10:57.777482+0000","last_peered":"2024-03-28T13:10:57.777482+0000","last_clean":"2024-03-28T13:10:57.777482+0000","last_became_active":"2024-03-28T13:10:56.783936+0000","last_became_peered":"2024-03-28T13:10:56.783936+0000","last_unstale":"2024-03-28T13:10:57.777482+0000","last_undegraded":"2024-03-28T13:10:57.777482+0000","last_fullsized":"2024-03-28T13:10:57.777482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:53.479188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-28T13:10:56.748281+0000","last_change":"2024-03-28T13:10:55.754934+0000","last_active":"2024-03-28T13:10:56.748281+0000","last_peered":"2024-03-28T13:10:56.748281+0000","last_clean":"2024-03-28T13:10:56.748281+0000","last_became_active":"2024-03-28T13:10:55.754215+0000","last_became_peered":"2024-03-28T13:10:55.754215+0000","last_unstale":"2024-03-28T13:10:56.748281+0000","last_undegraded":"2024-03-28T13:10:56.748281+0000","last_fullsized":"2024-03-28T13:10:56.748281+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:51:08.557038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776768+0000","last_change":"2024-03-28T13:10:56.791043+0000","last_active":"2024-03-28T13:10:57.776768+0000","last_peered":"2024-03-28T13:10:57.776768+0000","last_clean":"2024-03-28T13:10:57.776768+0000","last_became_active":"2024-03-28T13:10:56.787198+0000","last_became_peered":"2024-03-28T13:10:56.787198+0000","last_unstale":"2024-03-28T13:10:57.776768+0000","last_undegraded":"2024-03-28T13:10:57.776768+0000","last_fullsized":"2024-03-28T13:10:57.776768+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.770893+0000","last_change":"2024-03-28T13:10:55.741761+0000","last_active":"2024-03-28T13:10:57.770893+0000","last_peered":"2024-03-28T13:10:57.770893+0000","last_clean":"2024-03-28T13:10:57.770893+0000","last_became_active":"2024-03-28T13:10:55.741472+0000","last_became_peered":"2024-03-28T13:10:55.741472+0000","last_unstale":"2024-03-28T13:10:57.770893+0000","last_undegraded":"2024-03-28T13:10:57.770893+0000","last_fullsized":"2024-03-28T13:10:57.770893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:57:19.883535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265564+0000","last_change":"2024-03-28T13:10:56.775554+0000","last_active":"2024-03-28T13:10:58.265564+0000","last_peered":"2024-03-28T13:10:58.265564+0000","last_clean":"2024-03-28T13:10:58.265564+0000","last_became_active":"2024-03-28T13:10:56.774272+0000","last_became_peered":"2024-03-28T13:10:56.774272+0000","last_unstale":"2024-03-28T13:10:58.265564+0000","last_undegraded":"2024-03-28T13:10:58.265564+0000","last_fullsized":"2024-03-28T13:10:58.265564+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:53:51.017744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776675+0000","last_change":"2024-03-28T13:10:55.746007+0000","last_active":"2024-03-28T13:10:57.776675+0000","last_peered":"2024-03-28T13:10:57.776675+0000","last_clean":"2024-03-28T13:10:57.776675+0000","last_became_active":"2024-03-28T13:10:55.745666+0000","last_became_peered":"2024-03-28T13:10:55.745666+0000","last_unstale":"2024-03-28T13:10:57.776675+0000","last_undegraded":"2024-03-28T13:10:57.776675+0000","last_fullsized":"2024-03-28T13:10:57.776675+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:36:28.139935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029046+0000","last_change":"2024-03-28T13:10:56.787334+0000","last_active":"2024-03-28T13:10:58.029046+0000","last_peered":"2024-03-28T13:10:58.029046+0000","last_clean":"2024-03-28T13:10:58.029046+0000","last_became_active":"2024-03-28T13:10:56.786996+0000","last_became_peered":"2024-03-28T13:10:56.786996+0000","last_unstale":"2024-03-28T13:10:58.029046+0000","last_undegraded":"2024-03-28T13:10:58.029046+0000","last_fullsized":"2024-03-28T13:10:58.029046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:58:13.880385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.266118+0000","last_change":"2024-03-28T13:10:55.742909+0000","last_active":"2024-03-28T13:10:58.266118+0000","last_peered":"2024-03-28T13:10:58.266118+0000","last_clean":"2024-03-28T13:10:58.266118+0000","last_became_active":"2024-03-28T13:10:55.742605+0000","last_became_peered":"2024-03-28T13:10:55.742605+0000","last_unstale":"2024-03-28T13:10:58.266118+0000","last_undegraded":"2024-03-28T13:10:58.266118+0000","last_fullsized":"2024-03-28T13:10:58.266118+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:52.466987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":194,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366453+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:10:59.366453+0000","last_peered":"2024-03-28T13:10:59.366453+0000","last_clean":"2024-03-28T13:10:59.366453+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:10:59.366453+0000","last_undegraded":"2024-03-28T13:10:59.366453+0000","last_fullsized":"2024-03-28T13:10:59.366453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476760,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":155961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476758,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":155219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476758,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":745623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476758,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":155275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":397312,"data_stored":150497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509462,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":746135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509462,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":156819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":155207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:11:06.387 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:11:06.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.620+0000 7f03bde00640 1 Processor -- start 2024-03-28T13:11:06.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.620+0000 7f03bde00640 1 -- start start 2024-03-28T13:11:06.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.622+0000 7f03bde00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03b80cc0b0 0x7f03b806dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:06.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.622+0000 7f03bde00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b80d04d0 0x7f03b815ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:06.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.622+0000 7f03bde00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b806e4f0 con 0x7f03b815fc50 2024-03-28T13:11:06.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.622+0000 7f03bde00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f03b815c610 con 0x7f03b80cc0b0 2024-03-28T13:11:06.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.622+0000 7f03bde00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f03b80b3150 con 0x7f03b80d04d0 2024-03-28T13:11:06.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.622+0000 7f03b7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f03b815fc50 0x7f03b80cfdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49774/0 (socket says 172.21.15.19:49774) 2024-03-28T13:11:06.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.622+0000 7f03b7e00640 1 -- 172.21.15.19:0/2073744590 learned_addr learned my addr 172.21.15.19:0/2073744590 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:06.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.622+0000 7f03bca00640 1 --2- 172.21.15.19:0/2073744590 >> v2:172.21.15.19:3300/0 conn(0x7f03b80cc0b0 0x7f03b806dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:06.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.623+0000 7f03bd400640 1 --2- 172.21.15.19:0/2073744590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b80d04d0 0x7f03b815ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.623+0000 7f03b7400640 1 -- 172.21.15.19:0/2073744590 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1768440701 0 0) 0x7f03b806e4f0 con 0x7f03b815fc50 2024-03-28T13:11:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.623+0000 7f03b7400640 1 -- 172.21.15.19:0/2073744590 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03a0003520 con 0x7f03b815fc50 2024-03-28T13:11:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.623+0000 7f03bca00640 1 -- 172.21.15.19:0/2073744590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b80d04d0 msgr2=0x7f03b815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.623+0000 7f03bca00640 1 --2- 172.21.15.19:0/2073744590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b80d04d0 0x7f03b815ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.623+0000 7f03bca00640 1 -- 172.21.15.19:0/2073744590 >> v1:172.21.15.19:6789/0 conn(0x7f03b815fc50 legacy=0x7f03b80cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.623+0000 7f03bca00640 1 -- 172.21.15.19:0/2073744590 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b81600e0 con 0x7f03b80cc0b0 2024-03-28T13:11:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.624+0000 7f03bca00640 1 --2- 172.21.15.19:0/2073744590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b80cc0b0 0x7f03b806dfb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0097a0 tx=0x7f03ac02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc62beb67c265946 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.624+0000 7f03b7400640 1 -- 172.21.15.19:0/2073744590 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac035020 con 0x7f03b80cc0b0 2024-03-28T13:11:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.625+0000 7f03b7400640 1 -- 172.21.15.19:0/2073744590 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ac031830 con 0x7f03b80cc0b0 2024-03-28T13:11:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.625+0000 7f03b7400640 1 -- 172.21.15.19:0/2073744590 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac031be0 con 0x7f03b80cc0b0 2024-03-28T13:11:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.625+0000 7f03bde00640 1 -- 172.21.15.19:0/2073744590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b80cc0b0 msgr2=0x7f03b806dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.625+0000 7f03bde00640 1 --2- 172.21.15.19:0/2073744590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b80cc0b0 0x7f03b806dfb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0097a0 tx=0x7f03ac02b8d0 comp rx=0 tx=0).stop 2024-03-28T13:11:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.625+0000 7f03bde00640 1 -- 172.21.15.19:0/2073744590 shutdown_connections 2024-03-28T13:11:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.625+0000 7f03bde00640 1 --2- 172.21.15.19:0/2073744590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b80d04d0 0x7f03b815ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.625+0000 7f03bde00640 1 --2- 172.21.15.19:0/2073744590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b80cc0b0 0x7f03b806dfb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.625+0000 7f03bde00640 1 -- 172.21.15.19:0/2073744590 >> 172.21.15.19:0/2073744590 conn(0x7f03b80ccad0 msgr2=0x7f03b80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.625+0000 7f03bde00640 1 -- 172.21.15.19:0/2073744590 shutdown_connections 2024-03-28T13:11:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.625+0000 7f03bde00640 1 -- 172.21.15.19:0/2073744590 wait complete. 2024-03-28T13:11:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.627+0000 7f03bde00640 1 Processor -- start 2024-03-28T13:11:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.627+0000 7f03bde00640 1 -- start start 2024-03-28T13:11:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.628+0000 7f03bde00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b80cc0b0 0x7f03b813e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.628+0000 7f03bde00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03b80d04d0 0x7f03b815a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.628+0000 7f03bde00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b80a9480 0x7f03b80a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.628+0000 7f03bde00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03b8160980 con 0x7f03b80cc0b0 2024-03-28T13:11:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.628+0000 7f03bde00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f03b81606a0 con 0x7f03b80d04d0 2024-03-28T13:11:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.628+0000 7f03bde00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f03b80cc490 con 0x7f03b80a9480 2024-03-28T13:11:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.628+0000 7f03bca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b80cc0b0 0x7f03b813e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.628+0000 7f03bca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b80cc0b0 0x7f03b813e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45834/0 (socket says 172.21.15.19:45834) 2024-03-28T13:11:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.628+0000 7f03bca00640 1 -- 172.21.15.19:0/3266545667 learned_addr learned my addr 172.21.15.19:0/3266545667 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.629+0000 7f03b7e00640 1 --2- 172.21.15.19:0/3266545667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03b80d04d0 0x7f03b815a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.629+0000 7f03bca00640 1 -- 172.21.15.19:0/3266545667 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b80a9480 msgr2=0x7f03b80a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.629+0000 7f03bca00640 1 --2- 172.21.15.19:0/3266545667 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b80a9480 0x7f03b80a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.629+0000 7f03bca00640 1 -- 172.21.15.19:0/3266545667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03b80d04d0 msgr2=0x7f03b815a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.629+0000 7f03bca00640 1 --2- 172.21.15.19:0/3266545667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03b80d04d0 0x7f03b815a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.629+0000 7f03bca00640 1 -- 172.21.15.19:0/3266545667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b815a5f0 con 0x7f03b80cc0b0 2024-03-28T13:11:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.629+0000 7f03bca00640 1 --2- 172.21.15.19:0/3266545667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b80cc0b0 0x7f03b813e360 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f03ac02b910 tx=0x7f03ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.629+0000 7f03b4c00640 1 -- 172.21.15.19:0/3266545667 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac035040 con 0x7f03b80cc0b0 2024-03-28T13:11:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.630+0000 7f03b4c00640 1 -- 172.21.15.19:0/3266545667 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ac0044d0 con 0x7f03b80cc0b0 2024-03-28T13:11:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.630+0000 7f03b4c00640 1 -- 172.21.15.19:0/3266545667 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac003810 con 0x7f03b80cc0b0 2024-03-28T13:11:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.630+0000 7f03bde00640 1 -- 172.21.15.19:0/3266545667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b80a9a40 con 0x7f03b80cc0b0 2024-03-28T13:11:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.630+0000 7f03bde00640 1 -- 172.21.15.19:0/3266545667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b815b6c0 con 0x7f03b80cc0b0 2024-03-28T13:11:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.631+0000 7f03bde00640 1 -- 172.21.15.19:0/3266545667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b8075940 con 0x7f03b80cc0b0 2024-03-28T13:11:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.632+0000 7f03b4c00640 1 -- 172.21.15.19:0/3266545667 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f03ac038020 con 0x7f03b80cc0b0 2024-03-28T13:11:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.633+0000 7f03b4c00640 1 --2- 172.21.15.19:0/3266545667 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f039806c530 0x7f039806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.633+0000 7f03b4c00640 1 -- 172.21.15.19:0/3266545667 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f03ac0ad820 con 0x7f03b80cc0b0 2024-03-28T13:11:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.634+0000 7f03b7e00640 1 --2- 172.21.15.19:0/3266545667 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f039806c530 0x7f039806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.635+0000 7f03b7e00640 1 --2- 172.21.15.19:0/3266545667 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f039806c530 0x7f039806e9f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f03b8159db0 tx=0x7f03a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.638+0000 7f03b4c00640 1 -- 172.21.15.19:0/3266545667 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f03b815a5f0 con 0x7f03b80cc0b0 2024-03-28T13:11:06.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.838+0000 7f03bde00640 1 -- 172.21.15.19:0/3266545667 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f03b806f2a0 con 0x7f039806c530 2024-03-28T13:11:06.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.856+0000 7f03b4c00640 1 -- 172.21.15.19:0/3266545667 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251867 (secure 0 0 0) 0x7f03b806f2a0 con 0x7f039806c530 2024-03-28T13:11:06.860 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:06.862 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:11:06.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.861+0000 7f03bde00640 1 -- 172.21.15.19:0/3266545667 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f039806c530 msgr2=0x7f039806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:06.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.861+0000 7f03bde00640 1 --2- 172.21.15.19:0/3266545667 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f039806c530 0x7f039806e9f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f03b8159db0 tx=0x7f03a4009040 comp rx=0 tx=0).stop 2024-03-28T13:11:06.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.861+0000 7f03bde00640 1 -- 172.21.15.19:0/3266545667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b80cc0b0 msgr2=0x7f03b813e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:06.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.861+0000 7f03bde00640 1 --2- 172.21.15.19:0/3266545667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b80cc0b0 0x7f03b813e360 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f03ac02b910 tx=0x7f03ac0046c0 comp rx=0 tx=0).stop 2024-03-28T13:11:06.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.861+0000 7f03bde00640 1 -- 172.21.15.19:0/3266545667 shutdown_connections 2024-03-28T13:11:06.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.861+0000 7f03bde00640 1 --2- 172.21.15.19:0/3266545667 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f039806c530 0x7f039806e9f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:06.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.861+0000 7f03bde00640 1 --2- 172.21.15.19:0/3266545667 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03b80a9480 0x7f03b80a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:06.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.861+0000 7f03bde00640 1 --2- 172.21.15.19:0/3266545667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03b80d04d0 0x7f03b815a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:06.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.861+0000 7f03bde00640 1 --2- 172.21.15.19:0/3266545667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03b80cc0b0 0x7f03b813e360 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:06.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.861+0000 7f03bde00640 1 -- 172.21.15.19:0/3266545667 >> 172.21.15.19:0/3266545667 conn(0x7f03b80ccad0 msgr2=0x7f03b80b59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:06.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.861+0000 7f03bde00640 1 -- 172.21.15.19:0/3266545667 shutdown_connections 2024-03-28T13:11:06.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:06.861+0000 7f03bde00640 1 -- 172.21.15.19:0/3266545667 wait complete. 2024-03-28T13:11:06.882 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-03-28T13:11:06.011810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220500,"kb_used_data":4328,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754220,"statfs":{"total":773094113280,"available":772868321280,"internally_reserved":0,"allocated":4431872,"data_stored":2427036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":221236788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.269731"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771651+0000","last_change":"2024-03-28T13:10:56.775043+0000","last_active":"2024-03-28T13:10:57.771651+0000","last_peered":"2024-03-28T13:10:57.771651+0000","last_clean":"2024-03-28T13:10:57.771651+0000","last_became_active":"2024-03-28T13:10:56.774736+0000","last_became_peered":"2024-03-28T13:10:56.774736+0000","last_unstale":"2024-03-28T13:10:57.771651+0000","last_undegraded":"2024-03-28T13:10:57.771651+0000","last_fullsized":"2024-03-28T13:10:57.771651+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:26.233670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774657+0000","last_change":"2024-03-28T13:10:56.790536+0000","last_active":"2024-03-28T13:10:57.774657+0000","last_peered":"2024-03-28T13:10:57.774657+0000","last_clean":"2024-03-28T13:10:57.774657+0000","last_became_active":"2024-03-28T13:10:56.790180+0000","last_became_peered":"2024-03-28T13:10:56.790180+0000","last_unstale":"2024-03-28T13:10:57.774657+0000","last_undegraded":"2024-03-28T13:10:57.774657+0000","last_fullsized":"2024-03-28T13:10:57.774657+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:06:36.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.367194+0000","last_change":"2024-03-28T13:10:56.774875+0000","last_active":"2024-03-28T13:10:59.367194+0000","last_peered":"2024-03-28T13:10:59.367194+0000","last_clean":"2024-03-28T13:10:59.367194+0000","last_became_active":"2024-03-28T13:10:56.774593+0000","last_became_peered":"2024-03-28T13:10:56.774593+0000","last_unstale":"2024-03-28T13:10:59.367194+0000","last_undegraded":"2024-03-28T13:10:59.367194+0000","last_fullsized":"2024-03-28T13:10:59.367194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:50:10.939857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777345+0000","last_change":"2024-03-28T13:10:56.777966+0000","last_active":"2024-03-28T13:10:57.777345+0000","last_peered":"2024-03-28T13:10:57.777345+0000","last_clean":"2024-03-28T13:10:57.777345+0000","last_became_active":"2024-03-28T13:10:56.777658+0000","last_became_peered":"2024-03-28T13:10:56.777658+0000","last_unstale":"2024-03-28T13:10:57.777345+0000","last_undegraded":"2024-03-28T13:10:57.777345+0000","last_fullsized":"2024-03-28T13:10:57.777345+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:00:01.855413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774642+0000","last_change":"2024-03-28T13:10:56.783317+0000","last_active":"2024-03-28T13:10:57.774642+0000","last_peered":"2024-03-28T13:10:57.774642+0000","last_clean":"2024-03-28T13:10:57.774642+0000","last_became_active":"2024-03-28T13:10:56.781718+0000","last_became_peered":"2024-03-28T13:10:56.781718+0000","last_unstale":"2024-03-28T13:10:57.774642+0000","last_undegraded":"2024-03-28T13:10:57.774642+0000","last_fullsized":"2024-03-28T13:10:57.774642+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:58:13.880385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783325+0000","last_change":"2024-03-28T13:10:56.779357+0000","last_active":"2024-03-28T13:10:57.783325+0000","last_peered":"2024-03-28T13:10:57.783325+0000","last_clean":"2024-03-28T13:10:57.783325+0000","last_became_active":"2024-03-28T13:10:56.779000+0000","last_became_peered":"2024-03-28T13:10:56.779000+0000","last_unstale":"2024-03-28T13:10:57.783325+0000","last_undegraded":"2024-03-28T13:10:57.783325+0000","last_fullsized":"2024-03-28T13:10:57.783325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:40:07.650748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771017+0000","last_change":"2024-03-28T13:10:56.775965+0000","last_active":"2024-03-28T13:10:57.771017+0000","last_peered":"2024-03-28T13:10:57.771017+0000","last_clean":"2024-03-28T13:10:57.771017+0000","last_became_active":"2024-03-28T13:10:56.775684+0000","last_became_peered":"2024-03-28T13:10:56.775684+0000","last_unstale":"2024-03-28T13:10:57.771017+0000","last_undegraded":"2024-03-28T13:10:57.771017+0000","last_fullsized":"2024-03-28T13:10:57.771017+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:58:13.880385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771007+0000","last_change":"2024-03-28T13:10:56.780593+0000","last_active":"2024-03-28T13:10:57.771007+0000","last_peered":"2024-03-28T13:10:57.771007+0000","last_clean":"2024-03-28T13:10:57.771007+0000","last_became_active":"2024-03-28T13:10:56.780333+0000","last_became_peered":"2024-03-28T13:10:56.780333+0000","last_unstale":"2024-03-28T13:10:57.771007+0000","last_undegraded":"2024-03-28T13:10:57.771007+0000","last_fullsized":"2024-03-28T13:10:57.771007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:45.087620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.367199+0000","last_change":"2024-03-28T13:10:56.781115+0000","last_active":"2024-03-28T13:10:59.367199+0000","last_peered":"2024-03-28T13:10:59.367199+0000","last_clean":"2024-03-28T13:10:59.367199+0000","last_became_active":"2024-03-28T13:10:56.780578+0000","last_became_peered":"2024-03-28T13:10:56.780578+0000","last_unstale":"2024-03-28T13:10:59.367199+0000","last_undegraded":"2024-03-28T13:10:59.367199+0000","last_fullsized":"2024-03-28T13:10:59.367199+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:47.019632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778583+0000","last_change":"2024-03-28T13:10:56.793373+0000","last_active":"2024-03-28T13:10:57.778583+0000","last_peered":"2024-03-28T13:10:57.778583+0000","last_clean":"2024-03-28T13:10:57.778583+0000","last_became_active":"2024-03-28T13:10:56.789448+0000","last_became_peered":"2024-03-28T13:10:56.789448+0000","last_unstale":"2024-03-28T13:10:57.778583+0000","last_undegraded":"2024-03-28T13:10:57.778583+0000","last_fullsized":"2024-03-28T13:10:57.778583+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:37.170463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265827+0000","last_change":"2024-03-28T13:10:56.785075+0000","last_active":"2024-03-28T13:10:58.265827+0000","last_peered":"2024-03-28T13:10:58.265827+0000","last_clean":"2024-03-28T13:10:58.265827+0000","last_became_active":"2024-03-28T13:10:56.784486+0000","last_became_peered":"2024-03-28T13:10:56.784486+0000","last_unstale":"2024-03-28T13:10:58.265827+0000","last_undegraded":"2024-03-28T13:10:58.265827+0000","last_fullsized":"2024-03-28T13:10:58.265827+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:05:28.235846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.367058+0000","last_change":"2024-03-28T13:10:56.771454+0000","last_active":"2024-03-28T13:10:59.367058+0000","last_peered":"2024-03-28T13:10:59.367058+0000","last_clean":"2024-03-28T13:10:59.367058+0000","last_became_active":"2024-03-28T13:10:56.771111+0000","last_became_peered":"2024-03-28T13:10:56.771111+0000","last_unstale":"2024-03-28T13:10:59.367058+0000","last_undegraded":"2024-03-28T13:10:59.367058+0000","last_fullsized":"2024-03-28T13:10:59.367058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:38:26.793165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.367978+0000","last_change":"2024-03-28T13:10:56.780719+0000","last_active":"2024-03-28T13:10:59.367978+0000","last_peered":"2024-03-28T13:10:59.367978+0000","last_clean":"2024-03-28T13:10:59.367978+0000","last_became_active":"2024-03-28T13:10:56.780369+0000","last_became_peered":"2024-03-28T13:10:56.780369+0000","last_unstale":"2024-03-28T13:10:59.367978+0000","last_undegraded":"2024-03-28T13:10:59.367978+0000","last_fullsized":"2024-03-28T13:10:59.367978+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:45.087620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777339+0000","last_change":"2024-03-28T13:10:56.774962+0000","last_active":"2024-03-28T13:10:57.777339+0000","last_peered":"2024-03-28T13:10:57.777339+0000","last_clean":"2024-03-28T13:10:57.777339+0000","last_became_active":"2024-03-28T13:10:56.774519+0000","last_became_peered":"2024-03-28T13:10:56.774519+0000","last_unstale":"2024-03-28T13:10:57.777339+0000","last_undegraded":"2024-03-28T13:10:57.777339+0000","last_fullsized":"2024-03-28T13:10:57.777339+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:40.900533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029554+0000","last_change":"2024-03-28T13:10:56.776344+0000","last_active":"2024-03-28T13:10:58.029554+0000","last_peered":"2024-03-28T13:10:58.029554+0000","last_clean":"2024-03-28T13:10:58.029554+0000","last_became_active":"2024-03-28T13:10:56.776013+0000","last_became_peered":"2024-03-28T13:10:56.776013+0000","last_unstale":"2024-03-28T13:10:58.029554+0000","last_undegraded":"2024-03-28T13:10:58.029554+0000","last_fullsized":"2024-03-28T13:10:58.029554+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:26.233670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029127+0000","last_change":"2024-03-28T13:10:56.787487+0000","last_active":"2024-03-28T13:10:58.029127+0000","last_peered":"2024-03-28T13:10:58.029127+0000","last_clean":"2024-03-28T13:10:58.029127+0000","last_became_active":"2024-03-28T13:10:56.787012+0000","last_became_peered":"2024-03-28T13:10:56.787012+0000","last_unstale":"2024-03-28T13:10:58.029127+0000","last_undegraded":"2024-03-28T13:10:58.029127+0000","last_fullsized":"2024-03-28T13:10:58.029127+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:06:36.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265751+0000","last_change":"2024-03-28T13:10:56.778750+0000","last_active":"2024-03-28T13:10:58.265751+0000","last_peered":"2024-03-28T13:10:58.265751+0000","last_clean":"2024-03-28T13:10:58.265751+0000","last_became_active":"2024-03-28T13:10:56.778397+0000","last_became_peered":"2024-03-28T13:10:56.778397+0000","last_unstale":"2024-03-28T13:10:58.265751+0000","last_undegraded":"2024-03-28T13:10:58.265751+0000","last_fullsized":"2024-03-28T13:10:58.265751+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:09:21.093205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783603+0000","last_change":"2024-03-28T13:10:56.769141+0000","last_active":"2024-03-28T13:10:57.783603+0000","last_peered":"2024-03-28T13:10:57.783603+0000","last_clean":"2024-03-28T13:10:57.783603+0000","last_became_active":"2024-03-28T13:10:56.768873+0000","last_became_peered":"2024-03-28T13:10:56.768873+0000","last_unstale":"2024-03-28T13:10:57.783603+0000","last_undegraded":"2024-03-28T13:10:57.783603+0000","last_fullsized":"2024-03-28T13:10:57.783603+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:30:13.251989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783701+0000","last_change":"2024-03-28T13:10:56.767685+0000","last_active":"2024-03-28T13:10:57.783701+0000","last_peered":"2024-03-28T13:10:57.783701+0000","last_clean":"2024-03-28T13:10:57.783701+0000","last_became_active":"2024-03-28T13:10:56.767320+0000","last_became_peered":"2024-03-28T13:10:56.767320+0000","last_unstale":"2024-03-28T13:10:57.783701+0000","last_undegraded":"2024-03-28T13:10:57.783701+0000","last_fullsized":"2024-03-28T13:10:57.783701+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:45:22.482141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.368212+0000","last_change":"2024-03-28T13:10:56.772701+0000","last_active":"2024-03-28T13:10:59.368212+0000","last_peered":"2024-03-28T13:10:59.368212+0000","last_clean":"2024-03-28T13:10:59.368212+0000","last_became_active":"2024-03-28T13:10:56.772325+0000","last_became_peered":"2024-03-28T13:10:56.772325+0000","last_unstale":"2024-03-28T13:10:59.368212+0000","last_undegraded":"2024-03-28T13:10:59.368212+0000","last_fullsized":"2024-03-28T13:10:59.368212+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:58:13.880385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029451+0000","last_change":"2024-03-28T13:10:56.785391+0000","last_active":"2024-03-28T13:10:58.029451+0000","last_peered":"2024-03-28T13:10:58.029451+0000","last_clean":"2024-03-28T13:10:58.029451+0000","last_became_active":"2024-03-28T13:10:56.782726+0000","last_became_peered":"2024-03-28T13:10:56.782726+0000","last_unstale":"2024-03-28T13:10:58.029451+0000","last_undegraded":"2024-03-28T13:10:58.029451+0000","last_fullsized":"2024-03-28T13:10:58.029451+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.770494+0000","last_change":"2024-03-28T13:10:56.788744+0000","last_active":"2024-03-28T13:10:57.770494+0000","last_peered":"2024-03-28T13:10:57.770494+0000","last_clean":"2024-03-28T13:10:57.770494+0000","last_became_active":"2024-03-28T13:10:56.788421+0000","last_became_peered":"2024-03-28T13:10:56.788421+0000","last_unstale":"2024-03-28T13:10:57.770494+0000","last_undegraded":"2024-03-28T13:10:57.770494+0000","last_fullsized":"2024-03-28T13:10:57.770494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:37:53.859227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366913+0000","last_change":"2024-03-28T13:10:56.768323+0000","last_active":"2024-03-28T13:10:59.366913+0000","last_peered":"2024-03-28T13:10:59.366913+0000","last_clean":"2024-03-28T13:10:59.366913+0000","last_became_active":"2024-03-28T13:10:56.766260+0000","last_became_peered":"2024-03-28T13:10:56.766260+0000","last_unstale":"2024-03-28T13:10:59.366913+0000","last_undegraded":"2024-03-28T13:10:59.366913+0000","last_fullsized":"2024-03-28T13:10:59.366913+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:52:10.929698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.770677+0000","last_change":"2024-03-28T13:10:56.789986+0000","last_active":"2024-03-28T13:10:57.770677+0000","last_peered":"2024-03-28T13:10:57.770677+0000","last_clean":"2024-03-28T13:10:57.770677+0000","last_became_active":"2024-03-28T13:10:56.789691+0000","last_became_peered":"2024-03-28T13:10:56.789691+0000","last_unstale":"2024-03-28T13:10:57.770677+0000","last_undegraded":"2024-03-28T13:10:57.770677+0000","last_fullsized":"2024-03-28T13:10:57.770677+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:52:10.929698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778158+0000","last_change":"2024-03-28T13:10:56.780679+0000","last_active":"2024-03-28T13:10:57.778158+0000","last_peered":"2024-03-28T13:10:57.778158+0000","last_clean":"2024-03-28T13:10:57.778158+0000","last_became_active":"2024-03-28T13:10:56.777813+0000","last_became_peered":"2024-03-28T13:10:56.777813+0000","last_unstale":"2024-03-28T13:10:57.778158+0000","last_undegraded":"2024-03-28T13:10:57.778158+0000","last_fullsized":"2024-03-28T13:10:57.778158+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:15:57.044511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778939+0000","last_change":"2024-03-28T13:10:56.781082+0000","last_active":"2024-03-28T13:10:57.778939+0000","last_peered":"2024-03-28T13:10:57.778939+0000","last_clean":"2024-03-28T13:10:57.778939+0000","last_became_active":"2024-03-28T13:10:56.780806+0000","last_became_peered":"2024-03-28T13:10:56.780806+0000","last_unstale":"2024-03-28T13:10:57.778939+0000","last_undegraded":"2024-03-28T13:10:57.778939+0000","last_fullsized":"2024-03-28T13:10:57.778939+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:33.779798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774874+0000","last_change":"2024-03-28T13:10:56.771825+0000","last_active":"2024-03-28T13:10:57.774874+0000","last_peered":"2024-03-28T13:10:57.774874+0000","last_clean":"2024-03-28T13:10:57.774874+0000","last_became_active":"2024-03-28T13:10:56.771505+0000","last_became_peered":"2024-03-28T13:10:56.771505+0000","last_unstale":"2024-03-28T13:10:57.774874+0000","last_undegraded":"2024-03-28T13:10:57.774874+0000","last_fullsized":"2024-03-28T13:10:57.774874+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771592+0000","last_change":"2024-03-28T13:10:56.784002+0000","last_active":"2024-03-28T13:10:57.771592+0000","last_peered":"2024-03-28T13:10:57.771592+0000","last_clean":"2024-03-28T13:10:57.771592+0000","last_became_active":"2024-03-28T13:10:56.783731+0000","last_became_peered":"2024-03-28T13:10:56.783731+0000","last_unstale":"2024-03-28T13:10:57.771592+0000","last_undegraded":"2024-03-28T13:10:57.771592+0000","last_fullsized":"2024-03-28T13:10:57.771592+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029694+0000","last_change":"2024-03-28T13:10:56.788313+0000","last_active":"2024-03-28T13:10:58.029694+0000","last_peered":"2024-03-28T13:10:58.029694+0000","last_clean":"2024-03-28T13:10:58.029694+0000","last_became_active":"2024-03-28T13:10:56.787982+0000","last_became_peered":"2024-03-28T13:10:56.787982+0000","last_unstale":"2024-03-28T13:10:58.029694+0000","last_undegraded":"2024-03-28T13:10:58.029694+0000","last_fullsized":"2024-03-28T13:10:58.029694+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:00:01.855413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029188+0000","last_change":"2024-03-28T13:10:56.779668+0000","last_active":"2024-03-28T13:10:58.029188+0000","last_peered":"2024-03-28T13:10:58.029188+0000","last_clean":"2024-03-28T13:10:58.029188+0000","last_became_active":"2024-03-28T13:10:56.778978+0000","last_became_peered":"2024-03-28T13:10:56.778978+0000","last_unstale":"2024-03-28T13:10:58.029188+0000","last_undegraded":"2024-03-28T13:10:58.029188+0000","last_fullsized":"2024-03-28T13:10:58.029188+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.770758+0000","last_change":"2024-03-28T13:10:56.788094+0000","last_active":"2024-03-28T13:10:57.770758+0000","last_peered":"2024-03-28T13:10:57.770758+0000","last_clean":"2024-03-28T13:10:57.770758+0000","last_became_active":"2024-03-28T13:10:56.787812+0000","last_became_peered":"2024-03-28T13:10:56.787812+0000","last_unstale":"2024-03-28T13:10:57.770758+0000","last_undegraded":"2024-03-28T13:10:57.770758+0000","last_fullsized":"2024-03-28T13:10:57.770758+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:37:14.452246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265005+0000","last_change":"2024-03-28T13:10:56.771703+0000","last_active":"2024-03-28T13:10:58.265005+0000","last_peered":"2024-03-28T13:10:58.265005+0000","last_clean":"2024-03-28T13:10:58.265005+0000","last_became_active":"2024-03-28T13:10:56.769956+0000","last_became_peered":"2024-03-28T13:10:56.769956+0000","last_unstale":"2024-03-28T13:10:58.265005+0000","last_undegraded":"2024-03-28T13:10:58.265005+0000","last_fullsized":"2024-03-28T13:10:58.265005+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:09:25.565676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.368452+0000","last_change":"2024-03-28T13:10:56.769275+0000","last_active":"2024-03-28T13:10:59.368452+0000","last_peered":"2024-03-28T13:10:59.368452+0000","last_clean":"2024-03-28T13:10:59.368452+0000","last_became_active":"2024-03-28T13:10:56.767794+0000","last_became_peered":"2024-03-28T13:10:56.767794+0000","last_unstale":"2024-03-28T13:10:59.368452+0000","last_undegraded":"2024-03-28T13:10:59.368452+0000","last_fullsized":"2024-03-28T13:10:59.368452+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:06:36.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":18,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265751+0000","last_change":"2024-03-28T13:10:55.740050+0000","last_active":"2024-03-28T13:10:58.265751+0000","last_peered":"2024-03-28T13:10:58.265751+0000","last_clean":"2024-03-28T13:10:58.265751+0000","last_became_active":"2024-03-28T13:10:55.739731+0000","last_became_peered":"2024-03-28T13:10:55.739731+0000","last_unstale":"2024-03-28T13:10:58.265751+0000","last_undegraded":"2024-03-28T13:10:58.265751+0000","last_fullsized":"2024-03-28T13:10:58.265751+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:22.907948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771663+0000","last_change":"2024-03-28T13:10:56.786361+0000","last_active":"2024-03-28T13:10:57.771663+0000","last_peered":"2024-03-28T13:10:57.771663+0000","last_clean":"2024-03-28T13:10:57.771663+0000","last_became_active":"2024-03-28T13:10:56.786029+0000","last_became_peered":"2024-03-28T13:10:56.786029+0000","last_unstale":"2024-03-28T13:10:57.771663+0000","last_undegraded":"2024-03-28T13:10:57.771663+0000","last_fullsized":"2024-03-28T13:10:57.771663+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:40.900533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.779205+0000","last_change":"2024-03-28T13:10:55.750470+0000","last_active":"2024-03-28T13:10:57.779205+0000","last_peered":"2024-03-28T13:10:57.779205+0000","last_clean":"2024-03-28T13:10:57.779205+0000","last_became_active":"2024-03-28T13:10:55.749676+0000","last_became_peered":"2024-03-28T13:10:55.749676+0000","last_unstale":"2024-03-28T13:10:57.779205+0000","last_undegraded":"2024-03-28T13:10:57.779205+0000","last_fullsized":"2024-03-28T13:10:57.779205+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:55:34.014926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771664+0000","last_change":"2024-03-28T13:10:56.793622+0000","last_active":"2024-03-28T13:10:57.771664+0000","last_peered":"2024-03-28T13:10:57.771664+0000","last_clean":"2024-03-28T13:10:57.771664+0000","last_became_active":"2024-03-28T13:10:56.793351+0000","last_became_peered":"2024-03-28T13:10:56.793351+0000","last_unstale":"2024-03-28T13:10:57.771664+0000","last_undegraded":"2024-03-28T13:10:57.771664+0000","last_fullsized":"2024-03-28T13:10:57.771664+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:00:01.855413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783942+0000","last_change":"2024-03-28T13:10:55.754683+0000","last_active":"2024-03-28T13:10:57.783942+0000","last_peered":"2024-03-28T13:10:57.783942+0000","last_clean":"2024-03-28T13:10:57.783942+0000","last_became_active":"2024-03-28T13:10:55.754344+0000","last_became_peered":"2024-03-28T13:10:55.754344+0000","last_unstale":"2024-03-28T13:10:57.783942+0000","last_undegraded":"2024-03-28T13:10:57.783942+0000","last_fullsized":"2024-03-28T13:10:57.783942+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:22:12.385800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777982+0000","last_change":"2024-03-28T13:10:56.779261+0000","last_active":"2024-03-28T13:10:57.777982+0000","last_peered":"2024-03-28T13:10:57.777982+0000","last_clean":"2024-03-28T13:10:57.777982+0000","last_became_active":"2024-03-28T13:10:56.778899+0000","last_became_peered":"2024-03-28T13:10:56.778899+0000","last_unstale":"2024-03-28T13:10:57.777982+0000","last_undegraded":"2024-03-28T13:10:57.777982+0000","last_fullsized":"2024-03-28T13:10:57.777982+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:44:06.086773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777813+0000","last_change":"2024-03-28T13:10:55.755603+0000","last_active":"2024-03-28T13:10:57.777813+0000","last_peered":"2024-03-28T13:10:57.777813+0000","last_clean":"2024-03-28T13:10:57.777813+0000","last_became_active":"2024-03-28T13:10:55.754244+0000","last_became_peered":"2024-03-28T13:10:55.754244+0000","last_unstale":"2024-03-28T13:10:57.777813+0000","last_undegraded":"2024-03-28T13:10:57.777813+0000","last_fullsized":"2024-03-28T13:10:57.777813+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:30.977315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366724+0000","last_change":"2024-03-28T13:10:56.776082+0000","last_active":"2024-03-28T13:10:59.366724+0000","last_peered":"2024-03-28T13:10:59.366724+0000","last_clean":"2024-03-28T13:10:59.366724+0000","last_became_active":"2024-03-28T13:10:56.775495+0000","last_became_peered":"2024-03-28T13:10:56.775495+0000","last_unstale":"2024-03-28T13:10:59.366724+0000","last_undegraded":"2024-03-28T13:10:59.366724+0000","last_fullsized":"2024-03-28T13:10:59.366724+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:39.445190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771537+0000","last_change":"2024-03-28T13:10:55.745761+0000","last_active":"2024-03-28T13:10:57.771537+0000","last_peered":"2024-03-28T13:10:57.771537+0000","last_clean":"2024-03-28T13:10:57.771537+0000","last_became_active":"2024-03-28T13:10:55.745492+0000","last_became_peered":"2024-03-28T13:10:55.745492+0000","last_unstale":"2024-03-28T13:10:57.771537+0000","last_undegraded":"2024-03-28T13:10:57.771537+0000","last_fullsized":"2024-03-28T13:10:57.771537+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:57:31.432582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771652+0000","last_change":"2024-03-28T13:10:56.783694+0000","last_active":"2024-03-28T13:10:57.771652+0000","last_peered":"2024-03-28T13:10:57.771652+0000","last_clean":"2024-03-28T13:10:57.771652+0000","last_became_active":"2024-03-28T13:10:56.783336+0000","last_became_peered":"2024-03-28T13:10:56.783336+0000","last_unstale":"2024-03-28T13:10:57.771652+0000","last_undegraded":"2024-03-28T13:10:57.771652+0000","last_fullsized":"2024-03-28T13:10:57.771652+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.770706+0000","last_change":"2024-03-28T13:10:55.753085+0000","last_active":"2024-03-28T13:10:57.770706+0000","last_peered":"2024-03-28T13:10:57.770706+0000","last_clean":"2024-03-28T13:10:57.770706+0000","last_became_active":"2024-03-28T13:10:55.752806+0000","last_became_peered":"2024-03-28T13:10:55.752806+0000","last_unstale":"2024-03-28T13:10:57.770706+0000","last_undegraded":"2024-03-28T13:10:57.770706+0000","last_fullsized":"2024-03-28T13:10:57.770706+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:46.007430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366338+0000","last_change":"2024-03-28T13:10:56.769125+0000","last_active":"2024-03-28T13:10:59.366338+0000","last_peered":"2024-03-28T13:10:59.366338+0000","last_clean":"2024-03-28T13:10:59.366338+0000","last_became_active":"2024-03-28T13:10:56.768806+0000","last_became_peered":"2024-03-28T13:10:56.768806+0000","last_unstale":"2024-03-28T13:10:59.366338+0000","last_undegraded":"2024-03-28T13:10:59.366338+0000","last_fullsized":"2024-03-28T13:10:59.366338+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:57:20.895736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783436+0000","last_change":"2024-03-28T13:10:55.750960+0000","last_active":"2024-03-28T13:10:57.783436+0000","last_peered":"2024-03-28T13:10:57.783436+0000","last_clean":"2024-03-28T13:10:57.783436+0000","last_became_active":"2024-03-28T13:10:55.750618+0000","last_became_peered":"2024-03-28T13:10:55.750618+0000","last_unstale":"2024-03-28T13:10:57.783436+0000","last_undegraded":"2024-03-28T13:10:57.783436+0000","last_fullsized":"2024-03-28T13:10:57.783436+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:10:36.018433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776802+0000","last_change":"2024-03-28T13:10:56.775408+0000","last_active":"2024-03-28T13:10:57.776802+0000","last_peered":"2024-03-28T13:10:57.776802+0000","last_clean":"2024-03-28T13:10:57.776802+0000","last_became_active":"2024-03-28T13:10:56.775141+0000","last_became_peered":"2024-03-28T13:10:56.775141+0000","last_unstale":"2024-03-28T13:10:57.776802+0000","last_undegraded":"2024-03-28T13:10:57.776802+0000","last_fullsized":"2024-03-28T13:10:57.776802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771390+0000","last_change":"2024-03-28T13:10:55.752431+0000","last_active":"2024-03-28T13:10:57.771390+0000","last_peered":"2024-03-28T13:10:57.771390+0000","last_clean":"2024-03-28T13:10:57.771390+0000","last_became_active":"2024-03-28T13:10:55.752001+0000","last_became_peered":"2024-03-28T13:10:55.752001+0000","last_unstale":"2024-03-28T13:10:57.771390+0000","last_undegraded":"2024-03-28T13:10:57.771390+0000","last_fullsized":"2024-03-28T13:10:57.771390+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:38.432989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265800+0000","last_change":"2024-03-28T13:10:56.775017+0000","last_active":"2024-03-28T13:10:58.265800+0000","last_peered":"2024-03-28T13:10:58.265800+0000","last_clean":"2024-03-28T13:10:58.265800+0000","last_became_active":"2024-03-28T13:10:56.773930+0000","last_became_peered":"2024-03-28T13:10:56.773930+0000","last_unstale":"2024-03-28T13:10:58.265800+0000","last_undegraded":"2024-03-28T13:10:58.265800+0000","last_fullsized":"2024-03-28T13:10:58.265800+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:36:18.347550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.775253+0000","last_change":"2024-03-28T13:10:55.741344+0000","last_active":"2024-03-28T13:10:57.775253+0000","last_peered":"2024-03-28T13:10:57.775253+0000","last_clean":"2024-03-28T13:10:57.775253+0000","last_became_active":"2024-03-28T13:10:55.740955+0000","last_became_peered":"2024-03-28T13:10:55.740955+0000","last_unstale":"2024-03-28T13:10:57.775253+0000","last_undegraded":"2024-03-28T13:10:57.775253+0000","last_fullsized":"2024-03-28T13:10:57.775253+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:55:33.765625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.778342+0000","last_change":"2024-03-28T13:10:56.792475+0000","last_active":"2024-03-28T13:10:57.778342+0000","last_peered":"2024-03-28T13:10:57.778342+0000","last_clean":"2024-03-28T13:10:57.778342+0000","last_became_active":"2024-03-28T13:10:56.792165+0000","last_became_peered":"2024-03-28T13:10:56.792165+0000","last_unstale":"2024-03-28T13:10:57.778342+0000","last_undegraded":"2024-03-28T13:10:57.778342+0000","last_fullsized":"2024-03-28T13:10:57.778342+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:50:07.655844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774840+0000","last_change":"2024-03-28T13:10:55.749119+0000","last_active":"2024-03-28T13:10:57.774840+0000","last_peered":"2024-03-28T13:10:57.774840+0000","last_clean":"2024-03-28T13:10:57.774840+0000","last_became_active":"2024-03-28T13:10:55.748527+0000","last_became_peered":"2024-03-28T13:10:55.748527+0000","last_unstale":"2024-03-28T13:10:57.774840+0000","last_undegraded":"2024-03-28T13:10:57.774840+0000","last_fullsized":"2024-03-28T13:10:57.774840+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:30.977315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777568+0000","last_change":"2024-03-28T13:10:56.778342+0000","last_active":"2024-03-28T13:10:57.777568+0000","last_peered":"2024-03-28T13:10:57.777568+0000","last_clean":"2024-03-28T13:10:57.777568+0000","last_became_active":"2024-03-28T13:10:56.777259+0000","last_became_peered":"2024-03-28T13:10:56.777259+0000","last_unstale":"2024-03-28T13:10:57.777568+0000","last_undegraded":"2024-03-28T13:10:57.777568+0000","last_fullsized":"2024-03-28T13:10:57.777568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:49:23.920150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265970+0000","last_change":"2024-03-28T13:10:55.737368+0000","last_active":"2024-03-28T13:10:58.265970+0000","last_peered":"2024-03-28T13:10:58.265970+0000","last_clean":"2024-03-28T13:10:58.265970+0000","last_became_active":"2024-03-28T13:10:55.736922+0000","last_became_peered":"2024-03-28T13:10:55.736922+0000","last_unstale":"2024-03-28T13:10:58.265970+0000","last_undegraded":"2024-03-28T13:10:58.265970+0000","last_fullsized":"2024-03-28T13:10:58.265970+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:39.888331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265780+0000","last_change":"2024-03-28T13:10:56.785370+0000","last_active":"2024-03-28T13:10:58.265780+0000","last_peered":"2024-03-28T13:10:58.265780+0000","last_clean":"2024-03-28T13:10:58.265780+0000","last_became_active":"2024-03-28T13:10:56.784822+0000","last_became_peered":"2024-03-28T13:10:56.784822+0000","last_unstale":"2024-03-28T13:10:58.265780+0000","last_undegraded":"2024-03-28T13:10:58.265780+0000","last_fullsized":"2024-03-28T13:10:58.265780+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:04:04.193445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":20,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.368174+0000","last_change":"2024-03-28T13:10:55.757421+0000","last_active":"2024-03-28T13:10:59.368174+0000","last_peered":"2024-03-28T13:10:59.368174+0000","last_clean":"2024-03-28T13:10:59.368174+0000","last_became_active":"2024-03-28T13:10:55.757041+0000","last_became_peered":"2024-03-28T13:10:55.757041+0000","last_unstale":"2024-03-28T13:10:59.368174+0000","last_undegraded":"2024-03-28T13:10:59.368174+0000","last_fullsized":"2024-03-28T13:10:59.368174+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:17:46.007430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366917+0000","last_change":"2024-03-28T13:10:56.767158+0000","last_active":"2024-03-28T13:10:59.366917+0000","last_peered":"2024-03-28T13:10:59.366917+0000","last_clean":"2024-03-28T13:10:59.366917+0000","last_became_active":"2024-03-28T13:10:56.762220+0000","last_became_peered":"2024-03-28T13:10:56.762220+0000","last_unstale":"2024-03-28T13:10:59.366917+0000","last_undegraded":"2024-03-28T13:10:59.366917+0000","last_fullsized":"2024-03-28T13:10:59.366917+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:37:53.859227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771783+0000","last_change":"2024-03-28T13:10:55.742612+0000","last_active":"2024-03-28T13:10:57.771783+0000","last_peered":"2024-03-28T13:10:57.771783+0000","last_clean":"2024-03-28T13:10:57.771783+0000","last_became_active":"2024-03-28T13:10:55.742344+0000","last_became_peered":"2024-03-28T13:10:55.742344+0000","last_unstale":"2024-03-28T13:10:57.771783+0000","last_undegraded":"2024-03-28T13:10:57.771783+0000","last_fullsized":"2024-03-28T13:10:57.771783+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:38:25.780963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777175+0000","last_change":"2024-03-28T13:10:56.791145+0000","last_active":"2024-03-28T13:10:57.777175+0000","last_peered":"2024-03-28T13:10:57.777175+0000","last_clean":"2024-03-28T13:10:57.777175+0000","last_became_active":"2024-03-28T13:10:56.789400+0000","last_became_peered":"2024-03-28T13:10:56.789400+0000","last_unstale":"2024-03-28T13:10:57.777175+0000","last_undegraded":"2024-03-28T13:10:57.777175+0000","last_fullsized":"2024-03-28T13:10:57.777175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:26.233670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777676+0000","last_change":"2024-03-28T13:10:55.750434+0000","last_active":"2024-03-28T13:10:57.777676+0000","last_peered":"2024-03-28T13:10:57.777676+0000","last_clean":"2024-03-28T13:10:57.777676+0000","last_became_active":"2024-03-28T13:10:55.750086+0000","last_became_peered":"2024-03-28T13:10:55.750086+0000","last_unstale":"2024-03-28T13:10:57.777676+0000","last_undegraded":"2024-03-28T13:10:57.777676+0000","last_fullsized":"2024-03-28T13:10:57.777676+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:37:32.623408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771101+0000","last_change":"2024-03-28T13:10:56.779238+0000","last_active":"2024-03-28T13:10:57.771101+0000","last_peered":"2024-03-28T13:10:57.771101+0000","last_clean":"2024-03-28T13:10:57.771101+0000","last_became_active":"2024-03-28T13:10:56.773883+0000","last_became_peered":"2024-03-28T13:10:56.773883+0000","last_unstale":"2024-03-28T13:10:57.771101+0000","last_undegraded":"2024-03-28T13:10:57.771101+0000","last_fullsized":"2024-03-28T13:10:57.771101+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:06:36.225246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776667+0000","last_change":"2024-03-28T13:10:55.755962+0000","last_active":"2024-03-28T13:10:57.776667+0000","last_peered":"2024-03-28T13:10:57.776667+0000","last_clean":"2024-03-28T13:10:57.776667+0000","last_became_active":"2024-03-28T13:10:55.755652+0000","last_became_peered":"2024-03-28T13:10:55.755652+0000","last_unstale":"2024-03-28T13:10:57.776667+0000","last_undegraded":"2024-03-28T13:10:57.776667+0000","last_fullsized":"2024-03-28T13:10:57.776667+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:51:08.557038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265334+0000","last_change":"2024-03-28T13:10:56.772096+0000","last_active":"2024-03-28T13:10:58.265334+0000","last_peered":"2024-03-28T13:10:58.265334+0000","last_clean":"2024-03-28T13:10:58.265334+0000","last_became_active":"2024-03-28T13:10:56.771758+0000","last_became_peered":"2024-03-28T13:10:56.771758+0000","last_unstale":"2024-03-28T13:10:58.265334+0000","last_undegraded":"2024-03-28T13:10:58.265334+0000","last_fullsized":"2024-03-28T13:10:58.265334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:31:34.008079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":22,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783479+0000","last_change":"2024-03-28T13:10:55.757692+0000","last_active":"2024-03-28T13:10:57.783479+0000","last_peered":"2024-03-28T13:10:57.783479+0000","last_clean":"2024-03-28T13:10:57.783479+0000","last_became_active":"2024-03-28T13:10:55.757463+0000","last_became_peered":"2024-03-28T13:10:55.757463+0000","last_unstale":"2024-03-28T13:10:57.783479+0000","last_undegraded":"2024-03-28T13:10:57.783479+0000","last_fullsized":"2024-03-28T13:10:57.783479+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:12:56.307494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265756+0000","last_change":"2024-03-28T13:10:56.793741+0000","last_active":"2024-03-28T13:10:58.265756+0000","last_peered":"2024-03-28T13:10:58.265756+0000","last_clean":"2024-03-28T13:10:58.265756+0000","last_became_active":"2024-03-28T13:10:56.793473+0000","last_became_peered":"2024-03-28T13:10:56.793473+0000","last_unstale":"2024-03-28T13:10:58.265756+0000","last_undegraded":"2024-03-28T13:10:58.265756+0000","last_fullsized":"2024-03-28T13:10:58.265756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:38:30.086095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783676+0000","last_change":"2024-03-28T13:10:55.757407+0000","last_active":"2024-03-28T13:10:57.783676+0000","last_peered":"2024-03-28T13:10:57.783676+0000","last_clean":"2024-03-28T13:10:57.783676+0000","last_became_active":"2024-03-28T13:10:55.757088+0000","last_became_peered":"2024-03-28T13:10:55.757088+0000","last_unstale":"2024-03-28T13:10:57.783676+0000","last_undegraded":"2024-03-28T13:10:57.783676+0000","last_fullsized":"2024-03-28T13:10:57.783676+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:03:35.637042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.367177+0000","last_change":"2024-03-28T13:10:56.772822+0000","last_active":"2024-03-28T13:10:59.367177+0000","last_peered":"2024-03-28T13:10:59.367177+0000","last_clean":"2024-03-28T13:10:59.367177+0000","last_became_active":"2024-03-28T13:10:56.772511+0000","last_became_peered":"2024-03-28T13:10:56.772511+0000","last_unstale":"2024-03-28T13:10:59.367177+0000","last_undegraded":"2024-03-28T13:10:59.367177+0000","last_fullsized":"2024-03-28T13:10:59.367177+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:57:32.444784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771480+0000","last_change":"2024-03-28T13:10:55.748363+0000","last_active":"2024-03-28T13:10:57.771480+0000","last_peered":"2024-03-28T13:10:57.771480+0000","last_clean":"2024-03-28T13:10:57.771480+0000","last_became_active":"2024-03-28T13:10:55.748084+0000","last_became_peered":"2024-03-28T13:10:55.748084+0000","last_unstale":"2024-03-28T13:10:57.771480+0000","last_undegraded":"2024-03-28T13:10:57.771480+0000","last_fullsized":"2024-03-28T13:10:57.771480+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:58.262597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.774512+0000","last_change":"2024-03-28T13:10:56.774113+0000","last_active":"2024-03-28T13:10:57.774512+0000","last_peered":"2024-03-28T13:10:57.774512+0000","last_clean":"2024-03-28T13:10:57.774512+0000","last_became_active":"2024-03-28T13:10:56.773838+0000","last_became_peered":"2024-03-28T13:10:56.773838+0000","last_unstale":"2024-03-28T13:10:57.774512+0000","last_undegraded":"2024-03-28T13:10:57.774512+0000","last_fullsized":"2024-03-28T13:10:57.774512+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:52:45.087620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265939+0000","last_change":"2024-03-28T13:10:55.746355+0000","last_active":"2024-03-28T13:10:58.265939+0000","last_peered":"2024-03-28T13:10:58.265939+0000","last_clean":"2024-03-28T13:10:58.265939+0000","last_became_active":"2024-03-28T13:10:55.744858+0000","last_became_peered":"2024-03-28T13:10:55.744858+0000","last_unstale":"2024-03-28T13:10:58.265939+0000","last_undegraded":"2024-03-28T13:10:58.265939+0000","last_fullsized":"2024-03-28T13:10:58.265939+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:00:00.843211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783690+0000","last_change":"2024-03-28T13:10:56.779040+0000","last_active":"2024-03-28T13:10:57.783690+0000","last_peered":"2024-03-28T13:10:57.783690+0000","last_clean":"2024-03-28T13:10:57.783690+0000","last_became_active":"2024-03-28T13:10:56.777765+0000","last_became_peered":"2024-03-28T13:10:56.777765+0000","last_unstale":"2024-03-28T13:10:57.783690+0000","last_undegraded":"2024-03-28T13:10:57.783690+0000","last_fullsized":"2024-03-28T13:10:57.783690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:08:44.685445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029121+0000","last_change":"2024-03-28T13:10:55.753629+0000","last_active":"2024-03-28T13:10:58.029121+0000","last_peered":"2024-03-28T13:10:58.029121+0000","last_clean":"2024-03-28T13:10:58.029121+0000","last_became_active":"2024-03-28T13:10:55.753330+0000","last_became_peered":"2024-03-28T13:10:55.753330+0000","last_unstale":"2024-03-28T13:10:58.029121+0000","last_undegraded":"2024-03-28T13:10:58.029121+0000","last_fullsized":"2024-03-28T13:10:58.029121+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:30.977315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.367147+0000","last_change":"2024-03-28T13:10:56.774575+0000","last_active":"2024-03-28T13:10:59.367147+0000","last_peered":"2024-03-28T13:10:59.367147+0000","last_clean":"2024-03-28T13:10:59.367147+0000","last_became_active":"2024-03-28T13:10:56.772544+0000","last_became_peered":"2024-03-28T13:10:56.772544+0000","last_unstale":"2024-03-28T13:10:59.367147+0000","last_undegraded":"2024-03-28T13:10:59.367147+0000","last_fullsized":"2024-03-28T13:10:59.367147+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:37:14.452246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.775058+0000","last_change":"2024-03-28T13:10:55.754015+0000","last_active":"2024-03-28T13:10:57.775058+0000","last_peered":"2024-03-28T13:10:57.775058+0000","last_clean":"2024-03-28T13:10:57.775058+0000","last_became_active":"2024-03-28T13:10:55.753701+0000","last_became_peered":"2024-03-28T13:10:55.753701+0000","last_unstale":"2024-03-28T13:10:57.775058+0000","last_undegraded":"2024-03-28T13:10:57.775058+0000","last_fullsized":"2024-03-28T13:10:57.775058+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:29:13.996522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.775071+0000","last_change":"2024-03-28T13:10:56.782871+0000","last_active":"2024-03-28T13:10:57.775071+0000","last_peered":"2024-03-28T13:10:57.775071+0000","last_clean":"2024-03-28T13:10:57.775071+0000","last_became_active":"2024-03-28T13:10:56.782020+0000","last_became_peered":"2024-03-28T13:10:56.782020+0000","last_unstale":"2024-03-28T13:10:57.775071+0000","last_undegraded":"2024-03-28T13:10:57.775071+0000","last_fullsized":"2024-03-28T13:10:57.775071+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:30.437329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.771032+0000","last_change":"2024-03-28T13:10:55.744963+0000","last_active":"2024-03-28T13:10:57.771032+0000","last_peered":"2024-03-28T13:10:57.771032+0000","last_clean":"2024-03-28T13:10:57.771032+0000","last_became_active":"2024-03-28T13:10:55.743998+0000","last_became_peered":"2024-03-28T13:10:55.743998+0000","last_unstale":"2024-03-28T13:10:57.771032+0000","last_undegraded":"2024-03-28T13:10:57.771032+0000","last_fullsized":"2024-03-28T13:10:57.771032+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:50:09.927656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366833+0000","last_change":"2024-03-28T13:10:56.780543+0000","last_active":"2024-03-28T13:10:59.366833+0000","last_peered":"2024-03-28T13:10:59.366833+0000","last_clean":"2024-03-28T13:10:59.366833+0000","last_became_active":"2024-03-28T13:10:56.780189+0000","last_became_peered":"2024-03-28T13:10:56.780189+0000","last_unstale":"2024-03-28T13:10:59.366833+0000","last_undegraded":"2024-03-28T13:10:59.366833+0000","last_fullsized":"2024-03-28T13:10:59.366833+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:37:52.963800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366777+0000","last_change":"2024-03-28T13:10:55.748693+0000","last_active":"2024-03-28T13:10:59.366777+0000","last_peered":"2024-03-28T13:10:59.366777+0000","last_clean":"2024-03-28T13:10:59.366777+0000","last_became_active":"2024-03-28T13:10:55.748357+0000","last_became_peered":"2024-03-28T13:10:55.748357+0000","last_unstale":"2024-03-28T13:10:59.366777+0000","last_undegraded":"2024-03-28T13:10:59.366777+0000","last_fullsized":"2024-03-28T13:10:59.366777+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:16:57.647981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366596+0000","last_change":"2024-03-28T13:10:56.770455+0000","last_active":"2024-03-28T13:10:59.366596+0000","last_peered":"2024-03-28T13:10:59.366596+0000","last_clean":"2024-03-28T13:10:59.366596+0000","last_became_active":"2024-03-28T13:10:56.770174+0000","last_became_peered":"2024-03-28T13:10:56.770174+0000","last_unstale":"2024-03-28T13:10:59.366596+0000","last_undegraded":"2024-03-28T13:10:59.366596+0000","last_fullsized":"2024-03-28T13:10:59.366596+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:59.274798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.264885+0000","last_change":"2024-03-28T13:10:55.743425+0000","last_active":"2024-03-28T13:10:58.264885+0000","last_peered":"2024-03-28T13:10:58.264885+0000","last_clean":"2024-03-28T13:10:58.264885+0000","last_became_active":"2024-03-28T13:10:55.743112+0000","last_became_peered":"2024-03-28T13:10:55.743112+0000","last_unstale":"2024-03-28T13:10:58.264885+0000","last_undegraded":"2024-03-28T13:10:58.264885+0000","last_fullsized":"2024-03-28T13:10:58.264885+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:21:25.221468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783615+0000","last_change":"2024-03-28T13:10:56.775289+0000","last_active":"2024-03-28T13:10:57.783615+0000","last_peered":"2024-03-28T13:10:57.783615+0000","last_clean":"2024-03-28T13:10:57.783615+0000","last_became_active":"2024-03-28T13:10:56.774549+0000","last_became_peered":"2024-03-28T13:10:56.774549+0000","last_unstale":"2024-03-28T13:10:57.783615+0000","last_undegraded":"2024-03-28T13:10:57.783615+0000","last_fullsized":"2024-03-28T13:10:57.783615+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:30:16.475067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.368246+0000","last_change":"2024-03-28T13:10:55.755495+0000","last_active":"2024-03-28T13:10:59.368246+0000","last_peered":"2024-03-28T13:10:59.368246+0000","last_clean":"2024-03-28T13:10:59.368246+0000","last_became_active":"2024-03-28T13:10:55.754987+0000","last_became_peered":"2024-03-28T13:10:55.754987+0000","last_unstale":"2024-03-28T13:10:59.368246+0000","last_undegraded":"2024-03-28T13:10:59.368246+0000","last_fullsized":"2024-03-28T13:10:59.368246+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:57:31.432582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265584+0000","last_change":"2024-03-28T13:10:56.776492+0000","last_active":"2024-03-28T13:10:58.265584+0000","last_peered":"2024-03-28T13:10:58.265584+0000","last_clean":"2024-03-28T13:10:58.265584+0000","last_became_active":"2024-03-28T13:10:56.774511+0000","last_became_peered":"2024-03-28T13:10:56.774511+0000","last_unstale":"2024-03-28T13:10:58.265584+0000","last_undegraded":"2024-03-28T13:10:58.265584+0000","last_fullsized":"2024-03-28T13:10:58.265584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:09:01.845737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":15,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783204+0000","last_change":"2024-03-28T13:10:55.748821+0000","last_active":"2024-03-28T13:10:57.783204+0000","last_peered":"2024-03-28T13:10:57.783204+0000","last_clean":"2024-03-28T13:10:57.783204+0000","last_became_active":"2024-03-28T13:10:55.748128+0000","last_became_peered":"2024-03-28T13:10:55.748128+0000","last_unstale":"2024-03-28T13:10:57.783204+0000","last_undegraded":"2024-03-28T13:10:57.783204+0000","last_fullsized":"2024-03-28T13:10:57.783204+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:48:41.782400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.783822+0000","last_change":"2024-03-28T13:10:56.785728+0000","last_active":"2024-03-28T13:10:57.783822+0000","last_peered":"2024-03-28T13:10:57.783822+0000","last_clean":"2024-03-28T13:10:57.783822+0000","last_became_active":"2024-03-28T13:10:56.783504+0000","last_became_peered":"2024-03-28T13:10:56.783504+0000","last_unstale":"2024-03-28T13:10:57.783822+0000","last_undegraded":"2024-03-28T13:10:57.783822+0000","last_fullsized":"2024-03-28T13:10:57.783822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:42:20.735994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.368474+0000","last_change":"2024-03-28T13:10:55.757201+0000","last_active":"2024-03-28T13:10:59.368474+0000","last_peered":"2024-03-28T13:10:59.368474+0000","last_clean":"2024-03-28T13:10:59.368474+0000","last_became_active":"2024-03-28T13:10:55.756328+0000","last_became_peered":"2024-03-28T13:10:55.756328+0000","last_unstale":"2024-03-28T13:10:59.368474+0000","last_undegraded":"2024-03-28T13:10:59.368474+0000","last_fullsized":"2024-03-28T13:10:59.368474+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:50:09.927656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029820+0000","last_change":"2024-03-28T13:10:56.789219+0000","last_active":"2024-03-28T13:10:58.029820+0000","last_peered":"2024-03-28T13:10:58.029820+0000","last_clean":"2024-03-28T13:10:58.029820+0000","last_became_active":"2024-03-28T13:10:56.783607+0000","last_became_peered":"2024-03-28T13:10:56.783607+0000","last_unstale":"2024-03-28T13:10:58.029820+0000","last_undegraded":"2024-03-28T13:10:58.029820+0000","last_fullsized":"2024-03-28T13:10:58.029820+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:40.900533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.368147+0000","last_change":"2024-03-28T13:10:55.753645+0000","last_active":"2024-03-28T13:10:59.368147+0000","last_peered":"2024-03-28T13:10:59.368147+0000","last_clean":"2024-03-28T13:10:59.368147+0000","last_became_active":"2024-03-28T13:10:55.753103+0000","last_became_peered":"2024-03-28T13:10:55.753103+0000","last_unstale":"2024-03-28T13:10:59.368147+0000","last_undegraded":"2024-03-28T13:10:59.368147+0000","last_fullsized":"2024-03-28T13:10:59.368147+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:38:25.780963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.777482+0000","last_change":"2024-03-28T13:10:56.784216+0000","last_active":"2024-03-28T13:10:57.777482+0000","last_peered":"2024-03-28T13:10:57.777482+0000","last_clean":"2024-03-28T13:10:57.777482+0000","last_became_active":"2024-03-28T13:10:56.783936+0000","last_became_peered":"2024-03-28T13:10:56.783936+0000","last_unstale":"2024-03-28T13:10:57.777482+0000","last_undegraded":"2024-03-28T13:10:57.777482+0000","last_fullsized":"2024-03-28T13:10:57.777482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:53.479188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.367515+0000","last_change":"2024-03-28T13:10:55.754934+0000","last_active":"2024-03-28T13:10:59.367515+0000","last_peered":"2024-03-28T13:10:59.367515+0000","last_clean":"2024-03-28T13:10:59.367515+0000","last_became_active":"2024-03-28T13:10:55.754215+0000","last_became_peered":"2024-03-28T13:10:55.754215+0000","last_unstale":"2024-03-28T13:10:59.367515+0000","last_undegraded":"2024-03-28T13:10:59.367515+0000","last_fullsized":"2024-03-28T13:10:59.367515+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:51:08.557038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776768+0000","last_change":"2024-03-28T13:10:56.791043+0000","last_active":"2024-03-28T13:10:57.776768+0000","last_peered":"2024-03-28T13:10:57.776768+0000","last_clean":"2024-03-28T13:10:57.776768+0000","last_became_active":"2024-03-28T13:10:56.787198+0000","last_became_peered":"2024-03-28T13:10:56.787198+0000","last_unstale":"2024-03-28T13:10:57.776768+0000","last_undegraded":"2024-03-28T13:10:57.776768+0000","last_fullsized":"2024-03-28T13:10:57.776768+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:25.260123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.770893+0000","last_change":"2024-03-28T13:10:55.741761+0000","last_active":"2024-03-28T13:10:57.770893+0000","last_peered":"2024-03-28T13:10:57.770893+0000","last_clean":"2024-03-28T13:10:57.770893+0000","last_became_active":"2024-03-28T13:10:55.741472+0000","last_became_peered":"2024-03-28T13:10:55.741472+0000","last_unstale":"2024-03-28T13:10:57.770893+0000","last_undegraded":"2024-03-28T13:10:57.770893+0000","last_fullsized":"2024-03-28T13:10:57.770893+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:57:19.883535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.265564+0000","last_change":"2024-03-28T13:10:56.775554+0000","last_active":"2024-03-28T13:10:58.265564+0000","last_peered":"2024-03-28T13:10:58.265564+0000","last_clean":"2024-03-28T13:10:58.265564+0000","last_became_active":"2024-03-28T13:10:56.774272+0000","last_became_peered":"2024-03-28T13:10:56.774272+0000","last_unstale":"2024-03-28T13:10:58.265564+0000","last_undegraded":"2024-03-28T13:10:58.265564+0000","last_fullsized":"2024-03-28T13:10:58.265564+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:53:51.017744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:57.776675+0000","last_change":"2024-03-28T13:10:55.746007+0000","last_active":"2024-03-28T13:10:57.776675+0000","last_peered":"2024-03-28T13:10:57.776675+0000","last_clean":"2024-03-28T13:10:57.776675+0000","last_became_active":"2024-03-28T13:10:55.745666+0000","last_became_peered":"2024-03-28T13:10:55.745666+0000","last_unstale":"2024-03-28T13:10:57.776675+0000","last_undegraded":"2024-03-28T13:10:57.776675+0000","last_fullsized":"2024-03-28T13:10:57.776675+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:36:28.139935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.029046+0000","last_change":"2024-03-28T13:10:56.787334+0000","last_active":"2024-03-28T13:10:58.029046+0000","last_peered":"2024-03-28T13:10:58.029046+0000","last_clean":"2024-03-28T13:10:58.029046+0000","last_became_active":"2024-03-28T13:10:56.786996+0000","last_became_peered":"2024-03-28T13:10:56.786996+0000","last_unstale":"2024-03-28T13:10:58.029046+0000","last_undegraded":"2024-03-28T13:10:58.029046+0000","last_fullsized":"2024-03-28T13:10:58.029046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:55.709476+0000","last_clean_scrub_stamp":"2024-03-28T13:10:55.709476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:58:13.880385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:58.266118+0000","last_change":"2024-03-28T13:10:55.742909+0000","last_active":"2024-03-28T13:10:58.266118+0000","last_peered":"2024-03-28T13:10:58.266118+0000","last_clean":"2024-03-28T13:10:58.266118+0000","last_became_active":"2024-03-28T13:10:55.742605+0000","last_became_peered":"2024-03-28T13:10:55.742605+0000","last_unstale":"2024-03-28T13:10:58.266118+0000","last_undegraded":"2024-03-28T13:10:58.266118+0000","last_fullsized":"2024-03-28T13:10:58.266118+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:10:54.697274+0000","last_clean_scrub_stamp":"2024-03-28T13:10:54.697274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:52.466987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":194,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-28T13:10:59.366453+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:10:59.366453+0000","last_peered":"2024-03-28T13:10:59.366453+0000","last_clean":"2024-03-28T13:10:59.366453+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:10:59.366453+0000","last_undegraded":"2024-03-28T13:10:59.366453+0000","last_fullsized":"2024-03-28T13:10:59.366453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476761,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":155961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476759,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":155219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476759,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":745623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476759,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":155275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":156797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":995328,"data_stored":746135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":156819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542167,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":155207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:11:06.886 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-28T13:11:06.887 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-28T13:11:07.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.152+0000 7fe2b3200640 1 Processor -- start 2024-03-28T13:11:07.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.152+0000 7fe2b3200640 1 -- start start 2024-03-28T13:11:07.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.154+0000 7fe2b3200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe2ac162920 0x7fe2ac1670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:07.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.154+0000 7fe2b3200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe2ac0d0990 0x7fe2ac071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:07.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.154+0000 7fe2b3200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2ac071d60 con 0x7fe2ac163ac0 2024-03-28T13:11:07.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.154+0000 7fe2b3200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe2ac163f20 con 0x7fe2ac162920 2024-03-28T13:11:07.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.154+0000 7fe2b3200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fe2ac1665d0 con 0x7fe2ac0d0990 2024-03-28T13:11:07.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.154+0000 7fe2ab400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe2ac163ac0 0x7fe2ac0d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49778/0 (socket says 172.21.15.19:49778) 2024-03-28T13:11:07.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.154+0000 7fe2ab400640 1 -- 172.21.15.19:0/991953180 learned_addr learned my addr 172.21.15.19:0/991953180 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:07.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.154+0000 7fe2abe00640 1 --2- 172.21.15.19:0/991953180 >> v2:172.21.15.19:3300/0 conn(0x7fe2ac162920 0x7fe2ac1670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:07.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.154+0000 7fe2b1200640 1 --2- 172.21.15.19:0/991953180 >> v2:172.21.15.188:3300/0 conn(0x7fe2ac0d0990 0x7fe2ac071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:07.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.155+0000 7fe2aaa00640 1 -- 172.21.15.19:0/991953180 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 141383952 0 0) 0x7fe2ac071d60 con 0x7fe2ac163ac0 2024-03-28T13:11:07.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.155+0000 7fe2aaa00640 1 -- 172.21.15.19:0/991953180 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe290003520 con 0x7fe2ac163ac0 2024-03-28T13:11:07.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.155+0000 7fe2abe00640 1 -- 172.21.15.19:0/991953180 >> v2:172.21.15.188:3300/0 conn(0x7fe2ac0d0990 msgr2=0x7fe2ac071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:07.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.155+0000 7fe2abe00640 1 --2- 172.21.15.19:0/991953180 >> v2:172.21.15.188:3300/0 conn(0x7fe2ac0d0990 0x7fe2ac071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.155+0000 7fe2abe00640 1 -- 172.21.15.19:0/991953180 >> v1:172.21.15.19:6789/0 conn(0x7fe2ac163ac0 legacy=0x7fe2ac0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:07.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.155+0000 7fe2abe00640 1 -- 172.21.15.19:0/991953180 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac1668b0 con 0x7fe2ac162920 2024-03-28T13:11:07.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.156+0000 7fe2abe00640 1 --2- 172.21.15.19:0/991953180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2ac162920 0x7fe2ac1670b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0cfff0 tx=0x7fe29c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505482eff770025f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:07.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.156+0000 7fe2aaa00640 1 -- 172.21.15.19:0/991953180 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c0315a0 con 0x7fe2ac162920 2024-03-28T13:11:07.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.156+0000 7fe2aaa00640 1 -- 172.21.15.19:0/991953180 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29c037070 con 0x7fe2ac162920 2024-03-28T13:11:07.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.156+0000 7fe2aaa00640 1 -- 172.21.15.19:0/991953180 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29c031960 con 0x7fe2ac162920 2024-03-28T13:11:07.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.157+0000 7fe2b3200640 1 -- 172.21.15.19:0/991953180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2ac162920 msgr2=0x7fe2ac1670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:07.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.157+0000 7fe2b3200640 1 --2- 172.21.15.19:0/991953180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2ac162920 0x7fe2ac1670b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac0cfff0 tx=0x7fe29c02b840 comp rx=0 tx=0).stop 2024-03-28T13:11:07.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.157+0000 7fe2b3200640 1 -- 172.21.15.19:0/991953180 shutdown_connections 2024-03-28T13:11:07.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.157+0000 7fe2b3200640 1 --2- 172.21.15.19:0/991953180 >> v2:172.21.15.188:3300/0 conn(0x7fe2ac0d0990 0x7fe2ac071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.157+0000 7fe2b3200640 1 --2- 172.21.15.19:0/991953180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2ac162920 0x7fe2ac1670b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.157+0000 7fe2b3200640 1 -- 172.21.15.19:0/991953180 >> 172.21.15.19:0/991953180 conn(0x7fe2ac06b750 msgr2=0x7fe2ac06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:07.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.157+0000 7fe2b3200640 1 -- 172.21.15.19:0/991953180 shutdown_connections 2024-03-28T13:11:07.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.157+0000 7fe2b3200640 1 -- 172.21.15.19:0/991953180 wait complete. 2024-03-28T13:11:07.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.159+0000 7fe2b3200640 1 Processor -- start 2024-03-28T13:11:07.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.159+0000 7fe2b3200640 1 -- start start 2024-03-28T13:11:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.160+0000 7fe2b3200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2ac0d0990 0x7fe2ac15b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.160+0000 7fe2b3200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2ac162920 0x7fe2ac15cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.160+0000 7fe2b3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2ac163ac0 0x7fe2ac1fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.160+0000 7fe2b3200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac072040 con 0x7fe2ac163ac0 2024-03-28T13:11:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.160+0000 7fe2b3200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac162d00 con 0x7fe2ac162920 2024-03-28T13:11:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.160+0000 7fe2b3200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ac0721c0 con 0x7fe2ac0d0990 2024-03-28T13:11:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.160+0000 7fe2b1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2ac163ac0 0x7fe2ac1fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.160+0000 7fe2b1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2ac163ac0 0x7fe2ac1fbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45854/0 (socket says 172.21.15.19:45854) 2024-03-28T13:11:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.160+0000 7fe2b1200640 1 -- 172.21.15.19:0/758960669 learned_addr learned my addr 172.21.15.19:0/758960669 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.161+0000 7fe2abe00640 1 --2- 172.21.15.19:0/758960669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2ac0d0990 0x7fe2ac15b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.161+0000 7fe2ab400640 1 --2- 172.21.15.19:0/758960669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2ac162920 0x7fe2ac15cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:07.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.161+0000 7fe2b1200640 1 -- 172.21.15.19:0/758960669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2ac0d0990 msgr2=0x7fe2ac15b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:07.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.161+0000 7fe2b1200640 1 --2- 172.21.15.19:0/758960669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2ac0d0990 0x7fe2ac15b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.161+0000 7fe2b1200640 1 -- 172.21.15.19:0/758960669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2ac162920 msgr2=0x7fe2ac15cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:07.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.161+0000 7fe2b1200640 1 --2- 172.21.15.19:0/758960669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2ac162920 0x7fe2ac15cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.161+0000 7fe2b1200640 1 -- 172.21.15.19:0/758960669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ac15d130 con 0x7fe2ac163ac0 2024-03-28T13:11:07.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.161+0000 7fe2b1200640 1 --2- 172.21.15.19:0/758960669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2ac163ac0 0x7fe2ac1fbcc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4007e70 tx=0x7fe2a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:07.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.161+0000 7fe2a3e00640 1 -- 172.21.15.19:0/758960669 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4014620 con 0x7fe2ac163ac0 2024-03-28T13:11:07.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.162+0000 7fe2a3e00640 1 -- 172.21.15.19:0/758960669 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2a4019070 con 0x7fe2ac163ac0 2024-03-28T13:11:07.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.162+0000 7fe2a3e00640 1 -- 172.21.15.19:0/758960669 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4014a20 con 0x7fe2ac163ac0 2024-03-28T13:11:07.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.162+0000 7fe2b3200640 1 -- 172.21.15.19:0/758960669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2ac1fc630 con 0x7fe2ac163ac0 2024-03-28T13:11:07.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.162+0000 7fe2b3200640 1 -- 172.21.15.19:0/758960669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2ac09c4d0 con 0x7fe2ac163ac0 2024-03-28T13:11:07.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.163+0000 7fe2b3200640 1 -- 172.21.15.19:0/758960669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2ac09c300 con 0x7fe2ac163ac0 2024-03-28T13:11:07.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.164+0000 7fe2a3e00640 1 -- 172.21.15.19:0/758960669 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe2a4014bc0 con 0x7fe2ac163ac0 2024-03-28T13:11:07.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.165+0000 7fe2a3e00640 1 --2- 172.21.15.19:0/758960669 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe28c06c560 0x7fe28c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:07.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.166+0000 7fe2a3e00640 1 -- 172.21.15.19:0/758960669 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fe2a408bfb0 con 0x7fe2ac163ac0 2024-03-28T13:11:07.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.166+0000 7fe2abe00640 1 --2- 172.21.15.19:0/758960669 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe28c06c560 0x7fe28c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:07.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.167+0000 7fe2abe00640 1 --2- 172.21.15.19:0/758960669 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe28c06c560 0x7fe28c06ea20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe29c036000 tx=0x7fe29c03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:07.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.170+0000 7fe2a3e00640 1 -- 172.21.15.19:0/758960669 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe2a405a5c0 con 0x7fe2ac163ac0 2024-03-28T13:11:07.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.445+0000 7fe2b3200640 1 -- 172.21.15.19:0/758960669 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe2ac0685c0 con 0x7fe2ac163ac0 2024-03-28T13:11:07.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.450+0000 7fe2a3e00640 1 -- 172.21.15.19:0/758960669 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7fe2a4059ec0 con 0x7fe2ac163ac0 2024-03-28T13:11:07.450 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-03-28T13:11:07.450 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q== 2024-03-28T13:11:07.450 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-03-28T13:11:07.450 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-03-28T13:11:07.450 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-28T13:11:07.450 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-03-28T13:11:07.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.452+0000 7fe2b3200640 1 -- 172.21.15.19:0/758960669 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe28c06c560 msgr2=0x7fe28c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:07.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.452+0000 7fe2b3200640 1 --2- 172.21.15.19:0/758960669 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe28c06c560 0x7fe28c06ea20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe29c036000 tx=0x7fe29c03d040 comp rx=0 tx=0).stop 2024-03-28T13:11:07.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.452+0000 7fe2b3200640 1 -- 172.21.15.19:0/758960669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2ac163ac0 msgr2=0x7fe2ac1fbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:07.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.452+0000 7fe2b3200640 1 --2- 172.21.15.19:0/758960669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2ac163ac0 0x7fe2ac1fbcc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4007e70 tx=0x7fe2a400e4b0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.453+0000 7fe2b3200640 1 -- 172.21.15.19:0/758960669 shutdown_connections 2024-03-28T13:11:07.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.453+0000 7fe2b3200640 1 --2- 172.21.15.19:0/758960669 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe28c06c560 0x7fe28c06ea20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.453+0000 7fe2b3200640 1 --2- 172.21.15.19:0/758960669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2ac163ac0 0x7fe2ac1fbcc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.453+0000 7fe2b3200640 1 --2- 172.21.15.19:0/758960669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2ac162920 0x7fe2ac15cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.453+0000 7fe2b3200640 1 --2- 172.21.15.19:0/758960669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2ac0d0990 0x7fe2ac15b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.453+0000 7fe2b3200640 1 -- 172.21.15.19:0/758960669 >> 172.21.15.19:0/758960669 conn(0x7fe2ac06b750 msgr2=0x7fe2ac0ae390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:07.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.453+0000 7fe2b3200640 1 -- 172.21.15.19:0/758960669 shutdown_connections 2024-03-28T13:11:07.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.453+0000 7fe2b3200640 1 -- 172.21.15.19:0/758960669 wait complete. 2024-03-28T13:11:07.474 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:11:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.688+0000 7fc18a800640 1 Processor -- start 2024-03-28T13:11:07.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.688+0000 7fc18a800640 1 -- start start 2024-03-28T13:11:07.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.689+0000 7fc18a800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc1840cf1b0 0x7fc1840cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:07.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.689+0000 7fc18a800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc184071460 con 0x7fc1840d00d0 2024-03-28T13:11:07.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.690+0000 7fc18a800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1840721b0 con 0x7fc1841442c0 2024-03-28T13:11:07.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.690+0000 7fc18a800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc184071d20 con 0x7fc1840cf1b0 2024-03-28T13:11:07.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.690+0000 7fc189e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc1840d00d0 0x7fc1840b3b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49780/0 (socket says 172.21.15.19:49780) 2024-03-28T13:11:07.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.690+0000 7fc189e00640 1 -- 172.21.15.19:0/3500110171 learned_addr learned my addr 172.21.15.19:0/3500110171 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:07.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.690+0000 7fc188a00640 1 --2- 172.21.15.19:0/3500110171 >> v2:172.21.15.188:3300/0 conn(0x7fc1840cf1b0 0x7fc1840cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:07.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.690+0000 7fc183e00640 1 -- 172.21.15.19:0/3500110171 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2648260578 0 0) 0x7fc184071460 con 0x7fc1840d00d0 2024-03-28T13:11:07.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.691+0000 7fc183e00640 1 -- 172.21.15.19:0/3500110171 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc16c003520 con 0x7fc1840d00d0 2024-03-28T13:11:07.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.691+0000 7fc188a00640 1 -- 172.21.15.19:0/3500110171 >> v1:172.21.15.188:6789/0 conn(0x7fc1841442c0 legacy=0x7fc1840ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:07.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.691+0000 7fc183e00640 1 -- 172.21.15.19:0/3500110171 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3682663725 0 0) 0x7fc16c003520 con 0x7fc1840d00d0 2024-03-28T13:11:07.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.691+0000 7fc188a00640 1 -- 172.21.15.19:0/3500110171 >> v1:172.21.15.19:6789/0 conn(0x7fc1840d00d0 legacy=0x7fc1840b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:07.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.691+0000 7fc188a00640 1 -- 172.21.15.19:0/3500110171 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1840716d0 con 0x7fc1840cf1b0 2024-03-28T13:11:07.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.692+0000 7fc188a00640 1 --2- 172.21.15.19:0/3500110171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1840cf1b0 0x7fc1840cfb90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc1700097a0 tx=0x7fc17002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=727a85e6c9f1a772 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.693+0000 7fc183e00640 1 -- 172.21.15.19:0/3500110171 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0019f0 con 0x7fc1840cf1b0 2024-03-28T13:11:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.693+0000 7fc183e00640 1 -- 172.21.15.19:0/3500110171 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc170037070 con 0x7fc1840cf1b0 2024-03-28T13:11:07.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.693+0000 7fc183e00640 1 -- 172.21.15.19:0/3500110171 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1700317b0 con 0x7fc1840cf1b0 2024-03-28T13:11:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.693+0000 7fc18a800640 1 -- 172.21.15.19:0/3500110171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1840cf1b0 msgr2=0x7fc1840cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.693+0000 7fc18a800640 1 --2- 172.21.15.19:0/3500110171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1840cf1b0 0x7fc1840cfb90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc1700097a0 tx=0x7fc17002b8b0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.694+0000 7fc18a800640 1 -- 172.21.15.19:0/3500110171 shutdown_connections 2024-03-28T13:11:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.694+0000 7fc18a800640 1 --2- 172.21.15.19:0/3500110171 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1840cf1b0 0x7fc1840cfb90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.694+0000 7fc18a800640 1 -- 172.21.15.19:0/3500110171 >> 172.21.15.19:0/3500110171 conn(0x7fc1840ccad0 msgr2=0x7fc1840cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.694+0000 7fc18a800640 1 -- 172.21.15.19:0/3500110171 shutdown_connections 2024-03-28T13:11:07.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.694+0000 7fc18a800640 1 -- 172.21.15.19:0/3500110171 wait complete. 2024-03-28T13:11:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.695+0000 7fc18a800640 1 Processor -- start 2024-03-28T13:11:07.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.696+0000 7fc18a800640 1 -- start start 2024-03-28T13:11:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.697+0000 7fc18a800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1840cf1b0 0x7fc18415e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.697+0000 7fc18a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1840d00d0 0x7fc1840b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.697+0000 7fc18a800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1841442c0 0x7fc18415a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.697+0000 7fc18a800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1841446a0 con 0x7fc1840d00d0 2024-03-28T13:11:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.697+0000 7fc18a800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc1840d0770 con 0x7fc1841442c0 2024-03-28T13:11:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.697+0000 7fc18a800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc18415e7b0 con 0x7fc1840cf1b0 2024-03-28T13:11:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.697+0000 7fc188a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1840d00d0 0x7fc1840b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.697+0000 7fc188a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1840d00d0 0x7fc1840b58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45858/0 (socket says 172.21.15.19:45858) 2024-03-28T13:11:07.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.697+0000 7fc188a00640 1 -- 172.21.15.19:0/3114703088 learned_addr learned my addr 172.21.15.19:0/3114703088 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:07.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.697+0000 7fc189400640 1 --2- 172.21.15.19:0/3114703088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1840cf1b0 0x7fc18415e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:07.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.697+0000 7fc189e00640 1 --2- 172.21.15.19:0/3114703088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1841442c0 0x7fc18415a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:07.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.698+0000 7fc188a00640 1 -- 172.21.15.19:0/3114703088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1840cf1b0 msgr2=0x7fc18415e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:07.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.698+0000 7fc188a00640 1 --2- 172.21.15.19:0/3114703088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1840cf1b0 0x7fc18415e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.698+0000 7fc188a00640 1 -- 172.21.15.19:0/3114703088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1841442c0 msgr2=0x7fc18415a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:07.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.698+0000 7fc188a00640 1 --2- 172.21.15.19:0/3114703088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1841442c0 0x7fc18415a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.698+0000 7fc188a00640 1 -- 172.21.15.19:0/3114703088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1841f7b20 con 0x7fc1840d00d0 2024-03-28T13:11:07.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.698+0000 7fc188a00640 1 --2- 172.21.15.19:0/3114703088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1840d00d0 0x7fc1840b58b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc170034040 tx=0x7fc1700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:07.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.698+0000 7fc181600640 1 -- 172.21.15.19:0/3114703088 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170004030 con 0x7fc1840d00d0 2024-03-28T13:11:07.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.699+0000 7fc181600640 1 -- 172.21.15.19:0/3114703088 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1700041d0 con 0x7fc1840d00d0 2024-03-28T13:11:07.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.699+0000 7fc18a800640 1 -- 172.21.15.19:0/3114703088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1841f7e00 con 0x7fc1840d00d0 2024-03-28T13:11:07.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.699+0000 7fc181600640 1 -- 172.21.15.19:0/3114703088 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc170003610 con 0x7fc1840d00d0 2024-03-28T13:11:07.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.699+0000 7fc18a800640 1 -- 172.21.15.19:0/3114703088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1841f83c0 con 0x7fc1840d00d0 2024-03-28T13:11:07.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.700+0000 7fc18a800640 1 -- 172.21.15.19:0/3114703088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc184075940 con 0x7fc1840d00d0 2024-03-28T13:11:07.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.701+0000 7fc181600640 1 -- 172.21.15.19:0/3114703088 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc17004c060 con 0x7fc1840d00d0 2024-03-28T13:11:07.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.702+0000 7fc181600640 1 --2- 172.21.15.19:0/3114703088 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc16406c530 0x7fc16406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:07.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.702+0000 7fc181600640 1 -- 172.21.15.19:0/3114703088 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fc1700ad2f0 con 0x7fc1840d00d0 2024-03-28T13:11:07.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.702+0000 7fc189400640 1 --2- 172.21.15.19:0/3114703088 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc16406c530 0x7fc16406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:07.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.703+0000 7fc189400640 1 --2- 172.21.15.19:0/3114703088 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc16406c530 0x7fc16406e9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc178000f60 tx=0x7fc178009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:07.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.707+0000 7fc181600640 1 -- 172.21.15.19:0/3114703088 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc170040070 con 0x7fc1840d00d0 2024-03-28T13:11:07.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.951+0000 7fc18a800640 1 -- 172.21.15.19:0/3114703088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1840b2ae0 con 0x7fc1840d00d0 2024-03-28T13:11:07.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.953+0000 7fc181600640 1 -- 172.21.15.19:0/3114703088 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7fc17007ba40 con 0x7fc1840d00d0 2024-03-28T13:11:07.953 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:07.953 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2683971950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2683971950},{"type":"v1","addr":"172.21.15.188:6836","nonce":2683971950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/709274582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":709274582},{"type":"v1","addr":"172.21.15.19:6835","nonce":709274582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5040,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3420318181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3420318181},{"type":"v1","addr":"172.21.15.19:6833","nonce":3420318181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:10:56.216937+0000","modified":"2024-03-28T13:10:59.267258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5061":{"gid":5061,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6837/46284318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":46284318},{"type":"v1","addr":"172.21.15.188:6837","nonce":46284318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5061,"qdb_cluster":[5061]},"id":2}]} 2024-03-28T13:11:07.953 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-03-28T13:11:07.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.955+0000 7fc18a800640 1 -- 172.21.15.19:0/3114703088 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc16406c530 msgr2=0x7fc16406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:07.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.955+0000 7fc18a800640 1 --2- 172.21.15.19:0/3114703088 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc16406c530 0x7fc16406e9f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc178000f60 tx=0x7fc178009040 comp rx=0 tx=0).stop 2024-03-28T13:11:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.955+0000 7fc18a800640 1 -- 172.21.15.19:0/3114703088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1840d00d0 msgr2=0x7fc1840b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.955+0000 7fc18a800640 1 --2- 172.21.15.19:0/3114703088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1840d00d0 0x7fc1840b58b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc170034040 tx=0x7fc1700046c0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.955+0000 7fc18a800640 1 -- 172.21.15.19:0/3114703088 shutdown_connections 2024-03-28T13:11:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.955+0000 7fc18a800640 1 --2- 172.21.15.19:0/3114703088 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc16406c530 0x7fc16406e9f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.955+0000 7fc18a800640 1 --2- 172.21.15.19:0/3114703088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1841442c0 0x7fc18415a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.955+0000 7fc18a800640 1 --2- 172.21.15.19:0/3114703088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1840d00d0 0x7fc1840b58b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.955+0000 7fc18a800640 1 --2- 172.21.15.19:0/3114703088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1840cf1b0 0x7fc18415e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:07.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.955+0000 7fc18a800640 1 -- 172.21.15.19:0/3114703088 >> 172.21.15.19:0/3114703088 conn(0x7fc1840ccad0 msgr2=0x7fc18406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.956+0000 7fc18a800640 1 -- 172.21.15.19:0/3114703088 shutdown_connections 2024-03-28T13:11:07.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:07.956+0000 7fc18a800640 1 -- 172.21.15.19:0/3114703088 wait complete. 2024-03-28T13:11:08.377 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:10:56.216937+0000', 'modified': '2024-03-28T13:10:59.267258+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5061}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5061': {'gid': 5061, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.188:6837/46284318', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6835', 'nonce': 46284318}, {'type': 'v1', 'addr': '172.21.15.188:6837', 'nonce': 46284318}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5061, 'qdb_cluster': [5061]} 2024-03-28T13:11:08.377 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T13:11:08.377 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-28T13:11:08.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.610+0000 7f88d2a00640 1 Processor -- start 2024-03-28T13:11:08.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.610+0000 7f88d2a00640 1 -- start start 2024-03-28T13:11:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.611+0000 7f88d2a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f88cc0d0280 0x7f88cc0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.611+0000 7f88d2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f88cc162160 0x7f88cc162610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.612+0000 7f88d2a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88cc15ec80 con 0x7f88cc141c10 2024-03-28T13:11:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.612+0000 7f88d2a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f88cc0d0ba0 con 0x7f88cc162160 2024-03-28T13:11:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.612+0000 7f88d2a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f88cc15ef20 con 0x7f88cc0d0280 2024-03-28T13:11:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.612+0000 7f88cb400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f88cc141c10 0x7f88cc071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49792/0 (socket says 172.21.15.19:49792) 2024-03-28T13:11:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.612+0000 7f88cb400640 1 -- 172.21.15.19:0/4170926708 learned_addr learned my addr 172.21.15.19:0/4170926708 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.612+0000 7f88d0a00640 1 --2- 172.21.15.19:0/4170926708 >> v2:172.21.15.19:3300/0 conn(0x7f88cc162160 0x7f88cc162610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.612+0000 7f88cbe00640 1 --2- 172.21.15.19:0/4170926708 >> v2:172.21.15.188:3300/0 conn(0x7f88cc0d0280 0x7f88cc0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.612+0000 7f88caa00640 1 -- 172.21.15.19:0/4170926708 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1694714855 0 0) 0x7f88cc15ec80 con 0x7f88cc141c10 2024-03-28T13:11:08.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.612+0000 7f88caa00640 1 -- 172.21.15.19:0/4170926708 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88b0003520 con 0x7f88cc141c10 2024-03-28T13:11:08.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.613+0000 7f88d0a00640 1 -- 172.21.15.19:0/4170926708 >> v2:172.21.15.188:3300/0 conn(0x7f88cc0d0280 msgr2=0x7f88cc0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:08.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.613+0000 7f88d0a00640 1 --2- 172.21.15.19:0/4170926708 >> v2:172.21.15.188:3300/0 conn(0x7f88cc0d0280 0x7f88cc0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.613+0000 7f88d0a00640 1 -- 172.21.15.19:0/4170926708 >> v1:172.21.15.19:6789/0 conn(0x7f88cc141c10 legacy=0x7f88cc071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:08.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.613+0000 7f88d0a00640 1 -- 172.21.15.19:0/4170926708 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc15f200 con 0x7f88cc162160 2024-03-28T13:11:08.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.614+0000 7f88d0a00640 1 --2- 172.21.15.19:0/4170926708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc162160 0x7f88cc162610 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0d0d80 tx=0x7f88c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e001d4a9b225f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:08.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.614+0000 7f88caa00640 1 -- 172.21.15.19:0/4170926708 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4035020 con 0x7f88cc162160 2024-03-28T13:11:08.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.614+0000 7f88caa00640 1 -- 172.21.15.19:0/4170926708 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88c4031830 con 0x7f88cc162160 2024-03-28T13:11:08.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.614+0000 7f88caa00640 1 -- 172.21.15.19:0/4170926708 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4031be0 con 0x7f88cc162160 2024-03-28T13:11:08.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.615+0000 7f88d2a00640 1 -- 172.21.15.19:0/4170926708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc162160 msgr2=0x7f88cc162610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:08.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.615+0000 7f88d2a00640 1 --2- 172.21.15.19:0/4170926708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc162160 0x7f88cc162610 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0d0d80 tx=0x7f88c402b8d0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.615+0000 7f88d2a00640 1 -- 172.21.15.19:0/4170926708 shutdown_connections 2024-03-28T13:11:08.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.615+0000 7f88d2a00640 1 --2- 172.21.15.19:0/4170926708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc162160 0x7f88cc162610 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.615+0000 7f88d2a00640 1 --2- 172.21.15.19:0/4170926708 >> v2:172.21.15.188:3300/0 conn(0x7f88cc0d0280 0x7f88cc0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.615+0000 7f88d2a00640 1 -- 172.21.15.19:0/4170926708 >> 172.21.15.19:0/4170926708 conn(0x7f88cc06b750 msgr2=0x7f88cc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:08.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.615+0000 7f88d2a00640 1 -- 172.21.15.19:0/4170926708 shutdown_connections 2024-03-28T13:11:08.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.615+0000 7f88d2a00640 1 -- 172.21.15.19:0/4170926708 wait complete. 2024-03-28T13:11:08.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.617+0000 7f88d2a00640 1 Processor -- start 2024-03-28T13:11:08.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.617+0000 7f88d2a00640 1 -- start start 2024-03-28T13:11:08.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.618+0000 7f88d2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc0d0280 0x7f88cc1426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.618+0000 7f88d2a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc141c10 0x7f88cc0b1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.618+0000 7f88d2a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88cc162160 0x7f88cc099790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.618+0000 7f88d2a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc158e90 con 0x7f88cc0d0280 2024-03-28T13:11:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.618+0000 7f88d2a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc15f7c0 con 0x7f88cc162160 2024-03-28T13:11:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.618+0000 7f88d2a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f88cc1616e0 con 0x7f88cc141c10 2024-03-28T13:11:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.618+0000 7f88cbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc0d0280 0x7f88cc1426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.618+0000 7f88cbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc0d0280 0x7f88cc1426d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45884/0 (socket says 172.21.15.19:45884) 2024-03-28T13:11:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.618+0000 7f88cbe00640 1 -- 172.21.15.19:0/153527258 learned_addr learned my addr 172.21.15.19:0/153527258 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.619+0000 7f88cb400640 1 --2- 172.21.15.19:0/153527258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc141c10 0x7f88cc0b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.619+0000 7f88cbe00640 1 -- 172.21.15.19:0/153527258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc141c10 msgr2=0x7f88cc0b1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.619+0000 7f88cbe00640 1 --2- 172.21.15.19:0/153527258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc141c10 0x7f88cc0b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.619+0000 7f88cbe00640 1 -- 172.21.15.19:0/153527258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88cc162160 msgr2=0x7f88cc099790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.619+0000 7f88cbe00640 1 --2- 172.21.15.19:0/153527258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88cc162160 0x7f88cc099790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.619+0000 7f88cbe00640 1 -- 172.21.15.19:0/153527258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc0b44b0 con 0x7f88cc0d0280 2024-03-28T13:11:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.619+0000 7f88cbe00640 1 --2- 172.21.15.19:0/153527258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc0d0280 0x7f88cc1426d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f88bc007ef0 tx=0x7f88bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.619+0000 7f88c3e00640 1 -- 172.21.15.19:0/153527258 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc014620 con 0x7f88cc0d0280 2024-03-28T13:11:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.619+0000 7f88c3e00640 1 -- 172.21.15.19:0/153527258 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88bc019070 con 0x7f88cc0d0280 2024-03-28T13:11:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.619+0000 7f88c3e00640 1 -- 172.21.15.19:0/153527258 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc014a00 con 0x7f88cc0d0280 2024-03-28T13:11:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.619+0000 7f88d2a00640 1 -- 172.21.15.19:0/153527258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88cc0b4730 con 0x7f88cc0d0280 2024-03-28T13:11:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.620+0000 7f88d2a00640 1 -- 172.21.15.19:0/153527258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88cc0b4cf0 con 0x7f88cc0d0280 2024-03-28T13:11:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.622+0000 7f88c3e00640 1 -- 172.21.15.19:0/153527258 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f88bc014ba0 con 0x7f88cc0d0280 2024-03-28T13:11:08.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.623+0000 7f88c3e00640 1 --2- 172.21.15.19:0/153527258 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f88a806c530 0x7f88a806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.623+0000 7f88c3e00640 1 -- 172.21.15.19:0/153527258 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f88bc08bff0 con 0x7f88cc0d0280 2024-03-28T13:11:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.624+0000 7f88d5807740 1 Processor -- start 2024-03-28T13:11:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.624+0000 7f88cb400640 1 --2- 172.21.15.19:0/153527258 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f88a806c530 0x7f88a806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.624+0000 7f88d5807740 1 -- start start 2024-03-28T13:11:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.625+0000 7f88d5807740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed2fd0 0x561abeea5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.625+0000 7f88d5807740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abeed33b0 0x561abeed5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.625+0000 7f88d5807740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abeea64b0 0x561abeed62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.625+0000 7f88d5807740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561abee6d920 con 0x561abeed2fd0 2024-03-28T13:11:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.625+0000 7f88d5807740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561abedbdfa0 con 0x561abeea64b0 2024-03-28T13:11:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.625+0000 7f88d5807740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x561abee0ff90 con 0x561abeed33b0 2024-03-28T13:11:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.625+0000 7f88cb400640 1 --2- 172.21.15.19:0/153527258 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f88a806c530 0x7f88a806e9f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0cfa10 tx=0x7f88b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.625+0000 7f88cbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed2fd0 0x561abeea5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.625+0000 7f88cbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed2fd0 0x561abeea5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45898/0 (socket says 172.21.15.19:45898) 2024-03-28T13:11:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.625+0000 7f88cbe00640 1 -- 172.21.15.19:0/2938718369 learned_addr learned my addr 172.21.15.19:0/2938718369 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.625+0000 7f88cbe00640 1 -- 172.21.15.19:0/2938718369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abeed33b0 msgr2=0x561abeed5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.625+0000 7f88cbe00640 1 --2- 172.21.15.19:0/2938718369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abeed33b0 0x561abeed5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.625+0000 7f88cbe00640 1 -- 172.21.15.19:0/2938718369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abeea64b0 msgr2=0x561abeed62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.625+0000 7f88cbe00640 1 --2- 172.21.15.19:0/2938718369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abeea64b0 0x561abeed62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.625+0000 7f88cbe00640 1 -- 172.21.15.19:0/2938718369 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561abeb242d0 con 0x561abeed2fd0 2024-03-28T13:11:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.626+0000 7f88cbe00640 1 --2- 172.21.15.19:0/2938718369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed2fd0 0x561abeea5f70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0563c0 tx=0x7f88bc059980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6163e3ae44638670 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.626+0000 7f88bbe00640 1 -- 172.21.15.19:0/2938718369 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc072020 con 0x561abeed2fd0 2024-03-28T13:11:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.626+0000 7f88bbe00640 1 -- 172.21.15.19:0/2938718369 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88bc062b40 con 0x561abeed2fd0 2024-03-28T13:11:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.626+0000 7f88bbe00640 1 -- 172.21.15.19:0/2938718369 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0721c0 con 0x561abeed2fd0 2024-03-28T13:11:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.627+0000 7f88d5807740 1 -- 172.21.15.19:0/2938718369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed2fd0 msgr2=0x561abeea5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.627+0000 7f88d5807740 1 --2- 172.21.15.19:0/2938718369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed2fd0 0x561abeea5f70 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0563c0 tx=0x7f88bc059980 comp rx=0 tx=0).stop 2024-03-28T13:11:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.627+0000 7f88d5807740 1 -- 172.21.15.19:0/2938718369 shutdown_connections 2024-03-28T13:11:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.627+0000 7f88d5807740 1 --2- 172.21.15.19:0/2938718369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abeea64b0 0x561abeed62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.627+0000 7f88d5807740 1 --2- 172.21.15.19:0/2938718369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abeed33b0 0x561abeed5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.627+0000 7f88d5807740 1 --2- 172.21.15.19:0/2938718369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed2fd0 0x561abeea5f70 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.627+0000 7f88d5807740 1 -- 172.21.15.19:0/2938718369 >> 172.21.15.19:0/2938718369 conn(0x561abeea81e0 msgr2=0x561abeef0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.627+0000 7f88d5807740 1 -- 172.21.15.19:0/2938718369 shutdown_connections 2024-03-28T13:11:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.627+0000 7f88d5807740 1 -- 172.21.15.19:0/2938718369 wait complete. 2024-03-28T13:11:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.628+0000 7f88d5807740 1 -- start start 2024-03-28T13:11:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.628+0000 7f88d5807740 1 Processor -- start 2024-03-28T13:11:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.629+0000 7f88d5807740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abeed2fd0 0x561abeec3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.629+0000 7f88d5807740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed33b0 0x561abef00940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.629+0000 7f88d5807740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abef00e80 0x561abeecece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.629+0000 7f88d5807740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561abee0bf40 con 0x561abeed33b0 2024-03-28T13:11:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.629+0000 7f88d5807740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561abedb8650 con 0x561abeed2fd0 2024-03-28T13:11:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.629+0000 7f88d5807740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x561abedcba10 con 0x561abef00e80 2024-03-28T13:11:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.629+0000 7f88d5807740 10 client.0 _ll_register_callbacks cb 0x561abee0aff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:11:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.629+0000 7f88cb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed33b0 0x561abef00940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.629+0000 7f88cb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed33b0 0x561abef00940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45910/0 (socket says 172.21.15.19:45910) 2024-03-28T13:11:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.629+0000 7f88cb400640 1 -- 172.21.15.19:0/3778433678 learned_addr learned my addr 172.21.15.19:0/3778433678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.629+0000 7f88cbe00640 1 --2- 172.21.15.19:0/3778433678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abeed2fd0 0x561abeec3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.630+0000 7f88d0a00640 1 --2- 172.21.15.19:0/3778433678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abef00e80 0x561abeecece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.630+0000 7f88cb400640 1 -- 172.21.15.19:0/3778433678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abef00e80 msgr2=0x561abeecece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.630+0000 7f88cb400640 1 --2- 172.21.15.19:0/3778433678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abef00e80 0x561abeecece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.630+0000 7f88cb400640 1 -- 172.21.15.19:0/3778433678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abeed2fd0 msgr2=0x561abeec3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.630+0000 7f88cb400640 1 --2- 172.21.15.19:0/3778433678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abeed2fd0 0x561abeec3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.630+0000 7f88cb400640 1 -- 172.21.15.19:0/3778433678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561abebee6b0 con 0x561abeed33b0 2024-03-28T13:11:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.630+0000 7f88cb400640 1 --2- 172.21.15.19:0/3778433678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed33b0 0x561abef00940 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f88cc148260 tx=0x7f88b4007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4482938ea24b24 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.630+0000 7f88b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:11:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.630+0000 7f88b8c00640 1 -- 172.21.15.19:0/3778433678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4021420 con 0x561abeed33b0 2024-03-28T13:11:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.630+0000 7f88b8c00640 1 -- 172.21.15.19:0/3778433678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f88b4022c70 con 0x561abeed33b0 2024-03-28T13:11:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.630+0000 7f88d5807740 1 -- 172.21.15.19:0/3778433678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88bc062b40 con 0x561abeed33b0 2024-03-28T13:11:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.631+0000 7f88b8c00640 1 -- 172.21.15.19:0/3778433678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f88b4022e70 con 0x561abeed33b0 2024-03-28T13:11:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.631+0000 7f88d5807740 10 client.5265 fetch_fsmap learned FSMap version 17 2024-03-28T13:11:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.631+0000 7f88d5807740 1 -- 172.21.15.19:0/3778433678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x561abef01680 con 0x561abeed33b0 2024-03-28T13:11:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.631+0000 7f88b8c00640 1 -- 172.21.15.19:0/3778433678 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88bc062b40 con 0x561abeed33b0 2024-03-28T13:11:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.631+0000 7f88b8c00640 1 -- 172.21.15.19:0/3778433678 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f88b40215c0 con 0x561abeed33b0 2024-03-28T13:11:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.631+0000 7f88d5807740 10 client.5265 fetch_fsmap finished waiting for FSMap version 17 2024-03-28T13:11:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.631+0000 7f88d5807740 10 client.5265 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T13:11:08.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.631+0000 7f88d5807740 20 client.5265 populate_metadata read hostname 'smithi019' 2024-03-28T13:11:08.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.631+0000 7f88d5807740 1 --2- 172.21.15.19:0/3778433678 >> [v2:172.21.15.188:6835/46284318,v1:172.21.15.188:6837/46284318] conn(0x561abef0c990 0x561abef0ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.631+0000 7f88d5807740 4 client.5265 mds_command: new command op to 5061 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:11:08.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.631+0000 7f88d5807740 1 -- 172.21.15.19:0/3778433678 --> [v2:172.21.15.188:6835/46284318,v1:172.21.15.188:6837/46284318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561abee1d4d0 con 0x561abef0c990 2024-03-28T13:11:08.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.632+0000 7f88cbe00640 1 --2- 172.21.15.19:0/3778433678 >> [v2:172.21.15.188:6835/46284318,v1:172.21.15.188:6837/46284318] conn(0x561abef0c990 0x561abef0ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.633+0000 7f88cbe00640 1 --2- 172.21.15.19:0/3778433678 >> [v2:172.21.15.188:6835/46284318,v1:172.21.15.188:6837/46284318] conn(0x561abef0c990 0x561abef0ed80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47282ff1a227bfe5 server_cookie=d38a0e95cb6262d6 in_seq=0 out_seq=0 2024-03-28T13:11:08.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.633+0000 7f88b8c00640 10 client.5265 ms_handle_connect on v2:172.21.15.188:6835/46284318 2024-03-28T13:11:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.635+0000 7f88b8c00640 1 -- 172.21.15.19:0/3778433678 <== mds.0 v2:172.21.15.188:6835/46284318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561abee1d4d0 con 0x561abef0c990 2024-03-28T13:11:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.635+0000 7f88b8c00640 10 client.5265 handle_command_reply: tid=0 2024-03-28T13:11:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.635+0000 7f88d5807740 1 client.5265 shutdown 2024-03-28T13:11:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.636+0000 7f88d5807740 1 -- 172.21.15.19:0/3778433678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed33b0 msgr2=0x561abef00940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.636+0000 7f88d5807740 1 --2- 172.21.15.19:0/3778433678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed33b0 0x561abef00940 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f88cc148260 tx=0x7f88b4007720 comp rx=0 tx=0).stop 2024-03-28T13:11:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.636+0000 7f88d5807740 1 -- 172.21.15.19:0/3778433678 shutdown_connections 2024-03-28T13:11:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.636+0000 7f88d5807740 1 --2- 172.21.15.19:0/3778433678 >> [v2:172.21.15.188:6835/46284318,v1:172.21.15.188:6837/46284318] conn(0x561abef0c990 0x561abef0ed80 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.636+0000 7f88d5807740 1 --2- 172.21.15.19:0/3778433678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abef00e80 0x561abeecece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.636+0000 7f88d5807740 1 --2- 172.21.15.19:0/3778433678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed33b0 0x561abef00940 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.636+0000 7f88d5807740 1 --2- 172.21.15.19:0/3778433678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abeed2fd0 0x561abeec3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.636+0000 7f88b8c00640 0 client.5265 ms_handle_reset on v2:172.21.15.188:6835/46284318 2024-03-28T13:11:08.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.636+0000 7f88d5807740 1 -- 172.21.15.19:0/3778433678 >> 172.21.15.19:0/3778433678 conn(0x561abeea64b0 msgr2=0x561abeea5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:08.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.636+0000 7f88d5807740 1 -- 172.21.15.19:0/3778433678 shutdown_connections 2024-03-28T13:11:08.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.636+0000 7f88d5807740 1 -- 172.21.15.19:0/3778433678 wait complete. 2024-03-28T13:11:08.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.636+0000 7f88d5807740 20 client.5265 trim_cache size 0 max 16384 2024-03-28T13:11:08.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.656+0000 7f88d5807740 1 Processor -- start 2024-03-28T13:11:08.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.656+0000 7f88d5807740 1 -- start start 2024-03-28T13:11:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.657+0000 7f88d5807740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abeed2fd0 0x561abeed1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.657+0000 7f88d5807740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed33b0 0x561abeeaae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.657+0000 7f88d5807740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abef00e80 0x561abeed8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.657+0000 7f88d5807740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1ffb00 con 0x561abeed33b0 2024-03-28T13:11:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.657+0000 7f88d5807740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc1fee90 con 0x561abef00e80 2024-03-28T13:11:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.657+0000 7f88d5807740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f88cc1ff4b0 con 0x561abeed2fd0 2024-03-28T13:11:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.658+0000 7f88cb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed33b0 0x561abeeaae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.658+0000 7f88cb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed33b0 0x561abeeaae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45912/0 (socket says 172.21.15.19:45912) 2024-03-28T13:11:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.658+0000 7f88cb400640 1 -- 172.21.15.19:0/2987017611 learned_addr learned my addr 172.21.15.19:0/2987017611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.658+0000 7f88d0a00640 1 --2- 172.21.15.19:0/2987017611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abef00e80 0x561abeed8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.658+0000 7f88cbe00640 1 --2- 172.21.15.19:0/2987017611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abeed2fd0 0x561abeed1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.658+0000 7f88cb400640 1 -- 172.21.15.19:0/2987017611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abeed2fd0 msgr2=0x561abeed1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.658+0000 7f88cb400640 1 --2- 172.21.15.19:0/2987017611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abeed2fd0 0x561abeed1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.658+0000 7f88cb400640 1 -- 172.21.15.19:0/2987017611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abef00e80 msgr2=0x561abeed8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.658+0000 7f88cb400640 1 --2- 172.21.15.19:0/2987017611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abef00e80 0x561abeed8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.658+0000 7f88cb400640 1 -- 172.21.15.19:0/2987017611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561abee1aaf0 con 0x561abeed33b0 2024-03-28T13:11:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.658+0000 7f88cb400640 1 --2- 172.21.15.19:0/2987017611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed33b0 0x561abeeaae60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f88b400d1d0 tx=0x7f88b401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f4a3b9088163d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:08.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.659+0000 7f88bb400640 1 -- 172.21.15.19:0/2987017611 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b40215c0 con 0x561abeed33b0 2024-03-28T13:11:08.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.659+0000 7f88bb400640 1 -- 172.21.15.19:0/2987017611 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b4021760 con 0x561abeed33b0 2024-03-28T13:11:08.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.659+0000 7f88bb400640 1 -- 172.21.15.19:0/2987017611 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88b4021d60 con 0x561abeed33b0 2024-03-28T13:11:08.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.659+0000 7f88d5807740 1 -- 172.21.15.19:0/2987017611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed33b0 msgr2=0x561abeeaae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:08.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.659+0000 7f88d5807740 1 --2- 172.21.15.19:0/2987017611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed33b0 0x561abeeaae60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f88b400d1d0 tx=0x7f88b401c000 comp rx=0 tx=0).stop 2024-03-28T13:11:08.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.659+0000 7f88d5807740 1 -- 172.21.15.19:0/2987017611 shutdown_connections 2024-03-28T13:11:08.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.659+0000 7f88d5807740 1 --2- 172.21.15.19:0/2987017611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abef00e80 0x561abeed8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.659+0000 7f88d5807740 1 --2- 172.21.15.19:0/2987017611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed33b0 0x561abeeaae60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.659+0000 7f88d5807740 1 --2- 172.21.15.19:0/2987017611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abeed2fd0 0x561abeed1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.659+0000 7f88d5807740 1 -- 172.21.15.19:0/2987017611 >> 172.21.15.19:0/2987017611 conn(0x561abeea64b0 msgr2=0x561abeea1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:08.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.660+0000 7f88d5807740 1 -- 172.21.15.19:0/2987017611 shutdown_connections 2024-03-28T13:11:08.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.660+0000 7f88d5807740 1 -- 172.21.15.19:0/2987017611 wait complete. 2024-03-28T13:11:08.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.660+0000 7f88d5807740 1 -- start start 2024-03-28T13:11:08.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.661+0000 7f88d5807740 1 Processor -- start 2024-03-28T13:11:08.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88d5807740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed2fd0 0x561abeec2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88d5807740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abeed33b0 0x561abeed1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88d5807740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abef00e80 0x561abeed2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88d5807740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561abeed7f70 con 0x561abeed2fd0 2024-03-28T13:11:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88d5807740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561abef01300 con 0x561abeed33b0 2024-03-28T13:11:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88d5807740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x561abeecfa50 con 0x561abef00e80 2024-03-28T13:11:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88d5807740 10 client.0 _ll_register_callbacks cb 0x7f88b4001330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:11:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88cbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed2fd0 0x561abeec2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88cbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed2fd0 0x561abeec2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45916/0 (socket says 172.21.15.19:45916) 2024-03-28T13:11:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88cbe00640 1 -- 172.21.15.19:0/4261889323 learned_addr learned my addr 172.21.15.19:0/4261889323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88cb400640 1 --2- 172.21.15.19:0/4261889323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abeed33b0 0x561abeed1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88d0a00640 1 --2- 172.21.15.19:0/4261889323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abef00e80 0x561abeed2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88cbe00640 1 -- 172.21.15.19:0/4261889323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abef00e80 msgr2=0x561abeed2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88cbe00640 1 --2- 172.21.15.19:0/4261889323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abef00e80 0x561abeed2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88cbe00640 1 -- 172.21.15.19:0/4261889323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abeed33b0 msgr2=0x561abeed1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88cbe00640 1 --2- 172.21.15.19:0/4261889323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abeed33b0 0x561abeed1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.662+0000 7f88cbe00640 1 -- 172.21.15.19:0/4261889323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561abeee6b80 con 0x561abeed2fd0 2024-03-28T13:11:08.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.663+0000 7f88cbe00640 1 --2- 172.21.15.19:0/4261889323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed2fd0 0x561abeec2660 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f88bc08a850 tx=0x7f88bc016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1846b3673265279 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:08.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.663+0000 7f88b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:11:08.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.663+0000 7f88b8c00640 1 -- 172.21.15.19:0/4261889323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc072040 con 0x561abeed2fd0 2024-03-28T13:11:08.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.663+0000 7f88b8c00640 1 -- 172.21.15.19:0/4261889323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f88bc07ad40 con 0x561abeed2fd0 2024-03-28T13:11:08.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.663+0000 7f88d5807740 1 -- 172.21.15.19:0/4261889323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88b4021760 con 0x561abeed2fd0 2024-03-28T13:11:08.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.663+0000 7f88b8c00640 1 -- 172.21.15.19:0/4261889323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f88bc058340 con 0x561abeed2fd0 2024-03-28T13:11:08.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.663+0000 7f88d5807740 10 client.5271 fetch_fsmap learned FSMap version 17 2024-03-28T13:11:08.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.663+0000 7f88d5807740 1 -- 172.21.15.19:0/4261889323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x561abeee7480 con 0x561abeed2fd0 2024-03-28T13:11:08.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.663+0000 7f88b8c00640 1 -- 172.21.15.19:0/4261889323 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88b4021760 con 0x561abeed2fd0 2024-03-28T13:11:08.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.664+0000 7f88b8c00640 1 -- 172.21.15.19:0/4261889323 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f88bc05d070 con 0x561abeed2fd0 2024-03-28T13:11:08.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.664+0000 7f88d5807740 10 client.5271 fetch_fsmap finished waiting for FSMap version 17 2024-03-28T13:11:08.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.664+0000 7f88d5807740 10 client.5271 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T13:11:08.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.664+0000 7f88d5807740 20 client.5271 populate_metadata read hostname 'smithi019' 2024-03-28T13:11:08.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.664+0000 7f88d5807740 1 --2- 172.21.15.19:0/4261889323 >> [v2:172.21.15.188:6835/46284318,v1:172.21.15.188:6837/46284318] conn(0x561abef583e0 0x561abef09960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.664+0000 7f88d5807740 4 client.5271 mds_command: new command op to 5061 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T13:11:08.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.664+0000 7f88d5807740 1 -- 172.21.15.19:0/4261889323 --> [v2:172.21.15.188:6835/46284318,v1:172.21.15.188:6837/46284318] -- command(tid 0: {"prefix": "status"}) -- 0x561abee17820 con 0x561abef583e0 2024-03-28T13:11:08.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.664+0000 7f88cb400640 1 --2- 172.21.15.19:0/4261889323 >> [v2:172.21.15.188:6835/46284318,v1:172.21.15.188:6837/46284318] conn(0x561abef583e0 0x561abef09960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.665+0000 7f88cb400640 1 --2- 172.21.15.19:0/4261889323 >> [v2:172.21.15.188:6835/46284318,v1:172.21.15.188:6837/46284318] conn(0x561abef583e0 0x561abef09960 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=676f99d481fbe745 server_cookie=c27a9d5b75fa6547 in_seq=0 out_seq=0 2024-03-28T13:11:08.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.665+0000 7f88b8c00640 10 client.5271 ms_handle_connect on v2:172.21.15.188:6835/46284318 2024-03-28T13:11:08.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.666+0000 7f88b8c00640 1 -- 172.21.15.19:0/4261889323 <== mds.0 v2:172.21.15.188:6835/46284318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x561abee17820 con 0x561abef583e0 2024-03-28T13:11:08.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.666+0000 7f88b8c00640 10 client.5271 handle_command_reply: tid=0 2024-03-28T13:11:08.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.666+0000 7f88d5807740 1 client.5271 shutdown 2024-03-28T13:11:08.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.666+0000 7f88d5807740 1 -- 172.21.15.19:0/4261889323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed2fd0 msgr2=0x561abeec2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:08.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.666+0000 7f88d5807740 1 --2- 172.21.15.19:0/4261889323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed2fd0 0x561abeec2660 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f88bc08a850 tx=0x7f88bc016040 comp rx=0 tx=0).stop 2024-03-28T13:11:08.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.666+0000 7f88d5807740 1 -- 172.21.15.19:0/4261889323 shutdown_connections 2024-03-28T13:11:08.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.666+0000 7f88d5807740 1 --2- 172.21.15.19:0/4261889323 >> [v2:172.21.15.188:6835/46284318,v1:172.21.15.188:6837/46284318] conn(0x561abef583e0 0x561abef09960 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.666+0000 7f88d5807740 1 --2- 172.21.15.19:0/4261889323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561abef00e80 0x561abeed2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.666+0000 7f88d5807740 1 --2- 172.21.15.19:0/4261889323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561abeed33b0 0x561abeed1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.666+0000 7f88b8c00640 0 client.5271 ms_handle_reset on v2:172.21.15.188:6835/46284318 2024-03-28T13:11:08.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.666+0000 7f88d5807740 1 --2- 172.21.15.19:0/4261889323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561abeed2fd0 0x561abeec2660 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.666+0000 7f88d5807740 1 -- 172.21.15.19:0/4261889323 >> 172.21.15.19:0/4261889323 conn(0x561abeea64b0 msgr2=0x561abeee8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:08.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.666+0000 7f88d5807740 1 -- 172.21.15.19:0/4261889323 shutdown_connections 2024-03-28T13:11:08.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.666+0000 7f88d5807740 1 -- 172.21.15.19:0/4261889323 wait complete. 2024-03-28T13:11:08.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.666+0000 7f88d5807740 20 client.5271 trim_cache size 0 max 16384 2024-03-28T13:11:08.669 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:11:08.669 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "e6b569d9-f2ce-49dc-bbe7-c0ab33080203", 2024-03-28T13:11:08.669 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-03-28T13:11:08.670 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 5061, 2024-03-28T13:11:08.670 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-03-28T13:11:08.670 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-03-28T13:11:08.670 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-03-28T13:11:08.670 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 11.903254928999999, 2024-03-28T13:11:08.670 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 17, 2024-03-28T13:11:08.670 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 36, 2024-03-28T13:11:08.670 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 36, 2024-03-28T13:11:08.670 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 20.825824587 2024-03-28T13:11:08.670 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:11:08.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.667+0000 7f88d2a00640 1 -- 172.21.15.19:0/153527258 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f88a806c530 msgr2=0x7f88a806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:08.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.667+0000 7f88d2a00640 1 --2- 172.21.15.19:0/153527258 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f88a806c530 0x7f88a806e9f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0cfa10 tx=0x7f88b4009040 comp rx=0 tx=0).stop 2024-03-28T13:11:08.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.667+0000 7f88d2a00640 1 -- 172.21.15.19:0/153527258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc0d0280 msgr2=0x7f88cc1426d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:08.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.667+0000 7f88d2a00640 1 --2- 172.21.15.19:0/153527258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc0d0280 0x7f88cc1426d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f88bc007ef0 tx=0x7f88bc00e4b0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.667+0000 7f88d2a00640 1 -- 172.21.15.19:0/153527258 shutdown_connections 2024-03-28T13:11:08.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.667+0000 7f88d2a00640 1 --2- 172.21.15.19:0/153527258 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f88a806c530 0x7f88a806e9f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.667+0000 7f88d2a00640 1 --2- 172.21.15.19:0/153527258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88cc162160 0x7f88cc099790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.667+0000 7f88d2a00640 1 --2- 172.21.15.19:0/153527258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88cc141c10 0x7f88cc0b1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.667+0000 7f88d2a00640 1 --2- 172.21.15.19:0/153527258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88cc0d0280 0x7f88cc1426d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.667+0000 7f88d2a00640 1 -- 172.21.15.19:0/153527258 >> 172.21.15.19:0/153527258 conn(0x7f88cc06b750 msgr2=0x7f88cc143d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:08.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.667+0000 7f88d2a00640 1 -- 172.21.15.19:0/153527258 shutdown_connections 2024-03-28T13:11:08.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.667+0000 7f88d2a00640 1 -- 172.21.15.19:0/153527258 wait complete. 2024-03-28T13:11:08.684 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:11:08.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.902+0000 7fc539600640 1 Processor -- start 2024-03-28T13:11:08.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.903+0000 7fc539600640 1 -- start start 2024-03-28T13:11:08.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.904+0000 7fc539600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc5340b46d0 0x7fc534162190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.904+0000 7fc539600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc5341626d0 0x7fc5340b5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.904+0000 7fc539600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc534162f50 con 0x7fc534141510 2024-03-28T13:11:08.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.904+0000 7fc539600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc53415fb20 con 0x7fc5340b46d0 2024-03-28T13:11:08.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.904+0000 7fc539600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc534143ed0 con 0x7fc5341626d0 2024-03-28T13:11:08.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.904+0000 7fc533400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc5340b46d0 0x7fc534162190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.904+0000 7fc533400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc5340b46d0 0x7fc534162190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45928/0 (socket says 172.21.15.19:45928) 2024-03-28T13:11:08.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.904+0000 7fc533400640 1 -- 172.21.15.19:0/83164075 learned_addr learned my addr 172.21.15.19:0/83164075 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:08.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.905+0000 7fc538c00640 1 --2- 172.21.15.19:0/83164075 >> v2:172.21.15.188:3300/0 conn(0x7fc5341626d0 0x7fc5340b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.905+0000 7fc533400640 1 -- 172.21.15.19:0/83164075 >> v2:172.21.15.188:3300/0 conn(0x7fc5341626d0 msgr2=0x7fc5340b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:08.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.905+0000 7fc533400640 1 --2- 172.21.15.19:0/83164075 >> v2:172.21.15.188:3300/0 conn(0x7fc5341626d0 0x7fc5340b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.905+0000 7fc533400640 1 -- 172.21.15.19:0/83164075 >> v1:172.21.15.188:6789/0 conn(0x7fc534141510 legacy=0x7fc5340b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:08.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.905+0000 7fc533400640 1 -- 172.21.15.19:0/83164075 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5341441b0 con 0x7fc5340b46d0 2024-03-28T13:11:08.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.906+0000 7fc533400640 1 --2- 172.21.15.19:0/83164075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5340b46d0 0x7fc534162190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc534071d50 tx=0x7fc52402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c45b6b238d9cd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:08.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.906+0000 7fc532a00640 1 -- 172.21.15.19:0/83164075 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52402ca80 con 0x7fc5340b46d0 2024-03-28T13:11:08.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.907+0000 7fc532a00640 1 -- 172.21.15.19:0/83164075 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc52402cc20 con 0x7fc5340b46d0 2024-03-28T13:11:08.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.907+0000 7fc532a00640 1 -- 172.21.15.19:0/83164075 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5240343f0 con 0x7fc5340b46d0 2024-03-28T13:11:08.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.907+0000 7fc539600640 1 -- 172.21.15.19:0/83164075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5340b46d0 msgr2=0x7fc534162190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:08.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.907+0000 7fc539600640 1 --2- 172.21.15.19:0/83164075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5340b46d0 0x7fc534162190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc534071d50 tx=0x7fc52402bee0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.907+0000 7fc539600640 1 -- 172.21.15.19:0/83164075 shutdown_connections 2024-03-28T13:11:08.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.907+0000 7fc539600640 1 --2- 172.21.15.19:0/83164075 >> v2:172.21.15.188:3300/0 conn(0x7fc5341626d0 0x7fc5340b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.907+0000 7fc539600640 1 --2- 172.21.15.19:0/83164075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc5340b46d0 0x7fc534162190 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.907+0000 7fc539600640 1 -- 172.21.15.19:0/83164075 >> 172.21.15.19:0/83164075 conn(0x7fc534067460 msgr2=0x7fc534067870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:08.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.907+0000 7fc539600640 1 -- 172.21.15.19:0/83164075 shutdown_connections 2024-03-28T13:11:08.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.908+0000 7fc539600640 1 -- 172.21.15.19:0/83164075 wait complete. 2024-03-28T13:11:08.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.909+0000 7fc539600640 1 Processor -- start 2024-03-28T13:11:08.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.909+0000 7fc539600640 1 -- start start 2024-03-28T13:11:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.910+0000 7fc539600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5340b46d0 0x7fc5341547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.910+0000 7fc539600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc534141510 0x7fc534158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.910+0000 7fc539600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5341626d0 0x7fc53415e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.910+0000 7fc539600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc534143c50 con 0x7fc534141510 2024-03-28T13:11:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.910+0000 7fc539600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc534143ad0 con 0x7fc5341626d0 2024-03-28T13:11:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.910+0000 7fc539600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc5341449b0 con 0x7fc5340b46d0 2024-03-28T13:11:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.911+0000 7fc533400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc534141510 0x7fc534158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.911+0000 7fc533400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc534141510 0x7fc534158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45936/0 (socket says 172.21.15.19:45936) 2024-03-28T13:11:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.911+0000 7fc533400640 1 -- 172.21.15.19:0/374126591 learned_addr learned my addr 172.21.15.19:0/374126591 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.911+0000 7fc533e00640 1 --2- 172.21.15.19:0/374126591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5340b46d0 0x7fc5341547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.911+0000 7fc533400640 1 -- 172.21.15.19:0/374126591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5340b46d0 msgr2=0x7fc5341547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.911+0000 7fc533400640 1 --2- 172.21.15.19:0/374126591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5340b46d0 0x7fc5341547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.911+0000 7fc533400640 1 -- 172.21.15.19:0/374126591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5341626d0 msgr2=0x7fc53415e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.911+0000 7fc538c00640 1 --2- 172.21.15.19:0/374126591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5341626d0 0x7fc53415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.911+0000 7fc533400640 1 --2- 172.21.15.19:0/374126591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5341626d0 0x7fc53415e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.911+0000 7fc533400640 1 -- 172.21.15.19:0/374126591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5341e7fb0 con 0x7fc534141510 2024-03-28T13:11:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.911+0000 7fc533400640 1 --2- 172.21.15.19:0/374126591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc534141510 0x7fc534158270 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc524009f40 tx=0x7fc524004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.912+0000 7fc52be00640 1 -- 172.21.15.19:0/374126591 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524043030 con 0x7fc534141510 2024-03-28T13:11:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.912+0000 7fc52be00640 1 -- 172.21.15.19:0/374126591 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc52403e450 con 0x7fc534141510 2024-03-28T13:11:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.912+0000 7fc539600640 1 -- 172.21.15.19:0/374126591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5341e8290 con 0x7fc534141510 2024-03-28T13:11:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.912+0000 7fc52be00640 1 -- 172.21.15.19:0/374126591 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc524009510 con 0x7fc534141510 2024-03-28T13:11:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.912+0000 7fc539600640 1 -- 172.21.15.19:0/374126591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5341e8850 con 0x7fc534141510 2024-03-28T13:11:08.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.913+0000 7fc539600640 1 -- 172.21.15.19:0/374126591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc534075940 con 0x7fc534141510 2024-03-28T13:11:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.914+0000 7fc52be00640 1 -- 172.21.15.19:0/374126591 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc524046070 con 0x7fc534141510 2024-03-28T13:11:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.915+0000 7fc52be00640 1 --2- 172.21.15.19:0/374126591 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc51006c530 0x7fc51006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.915+0000 7fc52be00640 1 -- 172.21.15.19:0/374126591 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fc5240adc30 con 0x7fc534141510 2024-03-28T13:11:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.916+0000 7fc533e00640 1 --2- 172.21.15.19:0/374126591 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc51006c530 0x7fc51006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:08.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.917+0000 7fc533e00640 1 --2- 172.21.15.19:0/374126591 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc51006c530 0x7fc51006e9f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc51c007fd0 tx=0x7fc51c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:08.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:08.920+0000 7fc52be00640 1 -- 172.21.15.19:0/374126591 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc52404d070 con 0x7fc534141510 2024-03-28T13:11:09.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09.159+0000 7fc539600640 1 -- 172.21.15.19:0/374126591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc534071db0 con 0x7fc534141510 2024-03-28T13:11:09.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09.160+0000 7fc52be00640 1 -- 172.21.15.19:0/374126591 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7fc52407c1b0 con 0x7fc534141510 2024-03-28T13:11:09.161 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:11:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09.163+0000 7fc539600640 1 -- 172.21.15.19:0/374126591 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc51006c530 msgr2=0x7fc51006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09.163+0000 7fc539600640 1 --2- 172.21.15.19:0/374126591 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc51006c530 0x7fc51006e9f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc51c007fd0 tx=0x7fc51c009040 comp rx=0 tx=0).stop 2024-03-28T13:11:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09.163+0000 7fc539600640 1 -- 172.21.15.19:0/374126591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc534141510 msgr2=0x7fc534158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09.163+0000 7fc539600640 1 --2- 172.21.15.19:0/374126591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc534141510 0x7fc534158270 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc524009f40 tx=0x7fc524004a40 comp rx=0 tx=0).stop 2024-03-28T13:11:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09.163+0000 7fc539600640 1 -- 172.21.15.19:0/374126591 shutdown_connections 2024-03-28T13:11:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09.163+0000 7fc539600640 1 --2- 172.21.15.19:0/374126591 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc51006c530 0x7fc51006e9f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09.163+0000 7fc539600640 1 --2- 172.21.15.19:0/374126591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5341626d0 0x7fc53415e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:09.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09.163+0000 7fc539600640 1 --2- 172.21.15.19:0/374126591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc534141510 0x7fc534158270 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09.163+0000 7fc539600640 1 --2- 172.21.15.19:0/374126591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5340b46d0 0x7fc5341547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09.163+0000 7fc539600640 1 -- 172.21.15.19:0/374126591 >> 172.21.15.19:0/374126591 conn(0x7fc534067460 msgr2=0x7fc5340b3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09.163+0000 7fc539600640 1 -- 172.21.15.19:0/374126591 shutdown_connections 2024-03-28T13:11:09.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09.163+0000 7fc539600640 1 -- 172.21.15.19:0/374126591 wait complete. 2024-03-28T13:11:09.181 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-28T13:11:09.182 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:11:09.182 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:11:09.182 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:11:09.182 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:11:09.182 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-28T13:11:09.182 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-28T13:11:09.182 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:11:09.182 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:11:09.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:09.182 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:11:09.237 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:11:09.237 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:11:09.237 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:11:09.237 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:11:09.237 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:11:09.237 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:11:09.238 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:11:09.238 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:11:09.238 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:11:09.238 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:11:09.238 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:11:09.238 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:11:09.238 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:11:09.238 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:11:09.238 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:11:09.238 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:11:09.238 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42485sec preferred_lft 42485sec 2024-03-28T13:11:09.238 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:11:09.238 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:11:09.239 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:11:09.239 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:11:09.239 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:11:09.239 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:11:09.239 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:11:09.239 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:11:09.239 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:11:09.239 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:11:09.239 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:11:09.239 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:11:09.239 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:11:09.239 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:11:09.239 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:11:09.240 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:11:09.240 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether e6:17:c5:8e:2b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:11:09.240 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::e417:c5ff:fe8e:2bcf/64 scope link 2024-03-28T13:11:09.240 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:11:09.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:09.241 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:11:09.333 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:11:09.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:09.338 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:11:09.395 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:11:09.395 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:11:09.395 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:11:09.395 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:11:09.395 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:11:09.396 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:11:09.396 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:11:09.396 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:11:09.396 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:11:09.396 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:11:09.396 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:11:09.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:11:09.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:11:09.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:09.657 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:11:09.706 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:11:09.706 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:11:09.707 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:11:09.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:09.708 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:11:09.803 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:11:09.803 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:11:09.857 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:11:09.857 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:11:09.858 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:11:09.910 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:11:09.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:09.910 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:11:10.060 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:11:10.060 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:11:10.060 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:11:10.083 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:11:10.083 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:11:10.083 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:11:10.083 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:11:10.084 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:11:10.114 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:11:10.114 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:11:10.114 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:11:10.114 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:11:10.114 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:11:10.114 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:11:10.114 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:11:10.114 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:11:10.121 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:11:10.121 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:11:10.121 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:11:10.121 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:11:10.121 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:11:10.127 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:11:10.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:10.127 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:11:10.218 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:11:10.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:11:10.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:11:10.371 INFO:teuthology.orchestra.run.smithi019.stdout:client4784 2024-03-28T13:11:10.372 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:11:10.372 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:11:10.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:10.425 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-03-28T13:11:10.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:10.501 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-03-28T13:11:10.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:10.557 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-03-28T13:11:10.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:10.614 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-03-28T13:11:10.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:10.730 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-03-28T13:11:10.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:10.794 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-28T13:11:10.870 INFO:teuthology.orchestra.run.smithi019.stdout:./ 2024-03-28T13:11:10.870 INFO:teuthology.orchestra.run.smithi019.stdout:./rfile 2024-03-28T13:11:10.870 INFO:teuthology.orchestra.run.smithi019.stdout:./parentdir 2024-03-28T13:11:10.870 INFO:teuthology.orchestra.run.smithi019.stdout:./parentdir/pfile 2024-03-28T13:11:10.870 INFO:teuthology.orchestra.run.smithi019.stdout:./parentdir/childdir 2024-03-28T13:11:10.870 INFO:teuthology.orchestra.run.smithi019.stdout:./parentdir/childdir/cfile 2024-03-28T13:11:10.872 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:11:10.872 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:11:10.872 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:11:10.872 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:11:10.872 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-28T13:11:10.872 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:11:11.024 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:11:11.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:11:11.038 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:11:11.038 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:11:11.038 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:11:11.038 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:11:11.038 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-03-28T13:11:11.038 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:11:11.230 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627778 2024-03-28T13:11:11.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:11:11.244 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:11:11.245 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:11:11.245 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:11:11.245 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:11:11.245 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-03-28T13:11:11.245 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:11:11.437 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627776 2024-03-28T13:11:11.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:11:11.451 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:11:11.452 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:11:11.452 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:11:11.452 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:11:11.452 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-03-28T13:11:11.452 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:11:11.643 INFO:teuthology.orchestra.run.smithi019.stdout:1099511628279 2024-03-28T13:11:11.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:11:11.658 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:11:11.658 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:11:11.658 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:11:11.658 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:11:11.658 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-03-28T13:11:11.658 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:11:11.839 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627777 2024-03-28T13:11:11.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:11:11.852 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:11:11.853 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:11:11.853 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:11:11.853 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:11:11.853 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-03-28T13:11:11.853 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:11:12.048 INFO:teuthology.orchestra.run.smithi019.stdout:1099511628280 2024-03-28T13:11:12.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:11:12.063 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:11:12.063 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:11:12.161 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:11:12.161 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:11:12.214 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:11:12.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.428+0000 7f63aa000640 1 Processor -- start 2024-03-28T13:11:12.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.429+0000 7f63aa000640 1 -- start start 2024-03-28T13:11:12.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.430+0000 7f63aa000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63a40b4530 0x7f63a40c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:12.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.430+0000 7f63aa000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63a415eae0 con 0x7f63a40d1610 2024-03-28T13:11:12.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.430+0000 7f63aa000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63a40cfeb0 con 0x7f63a415e700 2024-03-28T13:11:12.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.430+0000 7f63aa000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f63a40d0190 con 0x7f63a40b4530 2024-03-28T13:11:12.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.431+0000 7f63a3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f63a40d1610 0x7f63a40c7640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49806/0 (socket says 172.21.15.19:49806) 2024-03-28T13:11:12.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.431+0000 7f63a3e00640 1 -- 172.21.15.19:0/1359404778 learned_addr learned my addr 172.21.15.19:0/1359404778 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.431+0000 7f63a2a00640 1 --2- 172.21.15.19:0/1359404778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63a40b4530 0x7f63a40c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.431+0000 7f63a2000640 1 -- 172.21.15.19:0/1359404778 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3084394901 0 0) 0x7f63a415eae0 con 0x7f63a40d1610 2024-03-28T13:11:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.431+0000 7f63a2000640 1 -- 172.21.15.19:0/1359404778 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6388003520 con 0x7f63a40d1610 2024-03-28T13:11:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.432+0000 7f63a2000640 1 -- 172.21.15.19:0/1359404778 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 583556115 0 0) 0x7f6388003520 con 0x7f63a40d1610 2024-03-28T13:11:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.432+0000 7f63a2000640 1 -- 172.21.15.19:0/1359404778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63a40b4530 msgr2=0x7f63a40c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.432+0000 7f63a2000640 1 --2- 172.21.15.19:0/1359404778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63a40b4530 0x7f63a40c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.432+0000 7f63a2000640 1 -- 172.21.15.19:0/1359404778 >> v1:172.21.15.188:6789/0 conn(0x7f63a415e700 legacy=0x7f63a40b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.432+0000 7f63a2000640 1 -- 172.21.15.19:0/1359404778 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a40d0470 con 0x7f63a40d1610 2024-03-28T13:11:12.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.432+0000 7f63a2000640 1 -- 172.21.15.19:0/1359404778 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f639c0019f0 con 0x7f63a40d1610 2024-03-28T13:11:12.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.432+0000 7f63a2000640 1 -- 172.21.15.19:0/1359404778 >> v1:172.21.15.19:6789/0 conn(0x7f63a40d1610 legacy=0x7f63a40c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:12.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.432+0000 7f63a2000640 1 --2- 172.21.15.19:0/1359404778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63880039e0 0x7f6388003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:12.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.432+0000 7f63a2000640 1 --2- 172.21.15.19:0/1359404778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63a40b4530 0x7f6388004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:12.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.432+0000 7f63a2000640 1 --2- 172.21.15.19:0/1359404778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6388004d40 0x7f63880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:12.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.432+0000 7f63a2000640 1 -- 172.21.15.19:0/1359404778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63a40d0190 con 0x7f63880039e0 2024-03-28T13:11:12.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.432+0000 7f63a2000640 1 -- 172.21.15.19:0/1359404778 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f638800bb80 con 0x7f6388004d40 2024-03-28T13:11:12.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.432+0000 7f63a2000640 1 -- 172.21.15.19:0/1359404778 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f638800bd80 con 0x7f63a40b4530 2024-03-28T13:11:12.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.432+0000 7f63a3400640 1 --2- 172.21.15.19:0/1359404778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63880039e0 0x7f6388003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:12.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.433+0000 7f63a3e00640 1 --2- 172.21.15.19:0/1359404778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6388004d40 0x7f63880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:12.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.433+0000 7f63a2a00640 1 --2- 172.21.15.19:0/1359404778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63a40b4530 0x7f6388004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:12.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.433+0000 7f63a3400640 1 -- 172.21.15.19:0/1359404778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63a40b4530 msgr2=0x7f6388004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:12.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.433+0000 7f63a3400640 1 --2- 172.21.15.19:0/1359404778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63a40b4530 0x7f6388004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:12.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.433+0000 7f63a3400640 1 -- 172.21.15.19:0/1359404778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6388004d40 msgr2=0x7f63880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:12.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.433+0000 7f63a3400640 1 --2- 172.21.15.19:0/1359404778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6388004d40 0x7f63880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:12.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.433+0000 7f63a3400640 1 -- 172.21.15.19:0/1359404778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6388003520 con 0x7f63880039e0 2024-03-28T13:11:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.434+0000 7f63a3400640 1 --2- 172.21.15.19:0/1359404778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63880039e0 0x7f6388003e00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6394001750 tx=0x7f6388005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f444712ca1b6490e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.435+0000 7f63a2000640 1 -- 172.21.15.19:0/1359404778 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6394001370 con 0x7f63880039e0 2024-03-28T13:11:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.435+0000 7f63a2000640 1 -- 172.21.15.19:0/1359404778 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63940052d0 con 0x7f63880039e0 2024-03-28T13:11:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.435+0000 7f63a2000640 1 -- 172.21.15.19:0/1359404778 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6394003830 con 0x7f63880039e0 2024-03-28T13:11:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.435+0000 7f63aa000640 1 -- 172.21.15.19:0/1359404778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63880039e0 msgr2=0x7f6388003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.435+0000 7f63aa000640 1 --2- 172.21.15.19:0/1359404778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63880039e0 0x7f6388003e00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6394001750 tx=0x7f6388005830 comp rx=0 tx=0).stop 2024-03-28T13:11:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.435+0000 7f63a3e00640 1 -- 172.21.15.19:0/1359404778 reap_dead start 2024-03-28T13:11:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.435+0000 7f63aa000640 1 -- 172.21.15.19:0/1359404778 shutdown_connections 2024-03-28T13:11:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.435+0000 7f63aa000640 1 -- 172.21.15.19:0/1359404778 >> 172.21.15.19:0/1359404778 conn(0x7f63a40d2240 msgr2=0x7f63a40d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.436+0000 7f63aa000640 1 -- 172.21.15.19:0/1359404778 shutdown_connections 2024-03-28T13:11:12.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.436+0000 7f63aa000640 1 -- 172.21.15.19:0/1359404778 wait complete. 2024-03-28T13:11:12.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.438+0000 7f63aa000640 1 Processor -- start 2024-03-28T13:11:12.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.438+0000 7f63aa000640 1 -- start start 2024-03-28T13:11:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.440+0000 7f63aa000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63a40b3410 0x7f63a4160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.440+0000 7f63aa000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63a4161830 0x7f63a414c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.440+0000 7f63aa000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a40b37f0 0x7f63a414ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.440+0000 7f63aa000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63a406d1b0 con 0x7f63a40b37f0 2024-03-28T13:11:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.440+0000 7f63aa000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f63a40d0a30 con 0x7f63a4161830 2024-03-28T13:11:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.440+0000 7f63aa000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f63a406d330 con 0x7f63a40b3410 2024-03-28T13:11:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.440+0000 7f63a3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a40b37f0 0x7f63a414ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.440+0000 7f63a3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a40b37f0 0x7f63a414ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45952/0 (socket says 172.21.15.19:45952) 2024-03-28T13:11:12.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.440+0000 7f63a3e00640 1 -- 172.21.15.19:0/763386358 learned_addr learned my addr 172.21.15.19:0/763386358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:12.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.440+0000 7f63a2a00640 1 --2- 172.21.15.19:0/763386358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63a4161830 0x7f63a414c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:12.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.440+0000 7f63a3400640 1 --2- 172.21.15.19:0/763386358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63a40b3410 0x7f63a4160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:12.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.441+0000 7f63a3e00640 1 -- 172.21.15.19:0/763386358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63a40b3410 msgr2=0x7f63a4160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:12.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.441+0000 7f63a3e00640 1 --2- 172.21.15.19:0/763386358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63a40b3410 0x7f63a4160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:12.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.441+0000 7f63a3e00640 1 -- 172.21.15.19:0/763386358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63a4161830 msgr2=0x7f63a414c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:12.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.441+0000 7f63a3e00640 1 --2- 172.21.15.19:0/763386358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63a4161830 0x7f63a414c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:12.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.441+0000 7f63a3e00640 1 -- 172.21.15.19:0/763386358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a41f5500 con 0x7f63a40b37f0 2024-03-28T13:11:12.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.441+0000 7f63a3400640 1 --2- 172.21.15.19:0/763386358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63a40b3410 0x7f63a4160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:12.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.441+0000 7f63a3e00640 1 --2- 172.21.15.19:0/763386358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a40b37f0 0x7f63a414ce90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f63a40692a0 tx=0x7f639c00e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:12.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.441+0000 7f639b400640 1 -- 172.21.15.19:0/763386358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c00ec30 con 0x7f63a40b37f0 2024-03-28T13:11:12.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.441+0000 7f639b400640 1 -- 172.21.15.19:0/763386358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f639c018070 con 0x7f63a40b37f0 2024-03-28T13:11:12.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.441+0000 7f639b400640 1 -- 172.21.15.19:0/763386358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c013420 con 0x7f63a40b37f0 2024-03-28T13:11:12.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.441+0000 7f63aa000640 1 -- 172.21.15.19:0/763386358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a41f57e0 con 0x7f63a40b37f0 2024-03-28T13:11:12.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.441+0000 7f63aa000640 1 -- 172.21.15.19:0/763386358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a41f5da0 con 0x7f63a40b37f0 2024-03-28T13:11:12.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.443+0000 7f63aa000640 1 -- 172.21.15.19:0/763386358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a40685c0 con 0x7f63a40b37f0 2024-03-28T13:11:12.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.444+0000 7f639b400640 1 -- 172.21.15.19:0/763386358 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f639c0135c0 con 0x7f63a40b37f0 2024-03-28T13:11:12.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.445+0000 7f639b400640 1 --2- 172.21.15.19:0/763386358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f638406c580 0x7f638406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:12.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.445+0000 7f639b400640 1 -- 172.21.15.19:0/763386358 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f639c08bb20 con 0x7f63a40b37f0 2024-03-28T13:11:12.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.446+0000 7f63a3400640 1 --2- 172.21.15.19:0/763386358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f638406c580 0x7f638406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:12.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.447+0000 7f63a3400640 1 --2- 172.21.15.19:0/763386358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f638406c580 0x7f638406ea40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6394033000 tx=0x7f6394009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:12.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.450+0000 7f639b400640 1 -- 172.21.15.19:0/763386358 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f639c05a120 con 0x7f63a40b37f0 2024-03-28T13:11:12.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.659+0000 7f63aa000640 1 -- 172.21.15.19:0/763386358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63a415c250 con 0x7f63a40b37f0 2024-03-28T13:11:12.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.662+0000 7f639b400640 1 -- 172.21.15.19:0/763386358 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+16609 (secure 0 0 0) 0x7f639c059a20 con 0x7f63a40b37f0 2024-03-28T13:11:12.663 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:12.663 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":36,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:10:57.758556+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:10:54.307571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-28T13:10:55.205691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:11:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.665+0000 7f63aa000640 1 -- 172.21.15.19:0/763386358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f638406c580 msgr2=0x7f638406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.665+0000 7f63aa000640 1 --2- 172.21.15.19:0/763386358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f638406c580 0x7f638406ea40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6394033000 tx=0x7f6394009030 comp rx=0 tx=0).stop 2024-03-28T13:11:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.665+0000 7f63aa000640 1 -- 172.21.15.19:0/763386358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a40b37f0 msgr2=0x7f63a414ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.665+0000 7f63aa000640 1 --2- 172.21.15.19:0/763386358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a40b37f0 0x7f63a414ce90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f63a40692a0 tx=0x7f639c00e160 comp rx=0 tx=0).stop 2024-03-28T13:11:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.666+0000 7f63aa000640 1 -- 172.21.15.19:0/763386358 shutdown_connections 2024-03-28T13:11:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.666+0000 7f63aa000640 1 --2- 172.21.15.19:0/763386358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f638406c580 0x7f638406ea40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.666+0000 7f63aa000640 1 --2- 172.21.15.19:0/763386358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63a40b37f0 0x7f63a414ce90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.666+0000 7f63aa000640 1 --2- 172.21.15.19:0/763386358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63a4161830 0x7f63a414c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.666+0000 7f63aa000640 1 --2- 172.21.15.19:0/763386358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63a40b3410 0x7f63a4160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:12.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.666+0000 7f63aa000640 1 -- 172.21.15.19:0/763386358 >> 172.21.15.19:0/763386358 conn(0x7f63a40d2240 msgr2=0x7f63a415b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.666+0000 7f63aa000640 1 -- 172.21.15.19:0/763386358 shutdown_connections 2024-03-28T13:11:12.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.666+0000 7f63aa000640 1 -- 172.21.15.19:0/763386358 wait complete. 2024-03-28T13:11:12.686 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:10:54.307571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:10:55.205691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:11:12.686 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:11:12.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.913+0000 7f5866600640 1 Processor -- start 2024-03-28T13:11:12.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.914+0000 7f5866600640 1 -- start start 2024-03-28T13:11:12.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.915+0000 7f5866600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f586013f190 0x7f58600cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:12.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.915+0000 7f5866600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58600cc880 0x7f58600cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:12.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.915+0000 7f5866600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58600ce890 con 0x7f586006d620 2024-03-28T13:11:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.915+0000 7f5866600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f58600cd200 con 0x7f58600cc880 2024-03-28T13:11:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.915+0000 7f5866600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f58600ceb50 con 0x7f586013f190 2024-03-28T13:11:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.915+0000 7f5865c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f586006d620 0x7f586006da70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49814/0 (socket says 172.21.15.19:49814) 2024-03-28T13:11:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.915+0000 7f585fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58600cc880 0x7f58600cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.915+0000 7f5865c00640 1 -- 172.21.15.19:0/3190338277 learned_addr learned my addr 172.21.15.19:0/3190338277 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.916+0000 7f5865200640 1 --2- 172.21.15.19:0/3190338277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f586013f190 0x7f58600cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:12.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.916+0000 7f585fe00640 1 -- 172.21.15.19:0/3190338277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f586013f190 msgr2=0x7f58600cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:12.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.916+0000 7f585fe00640 1 --2- 172.21.15.19:0/3190338277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f586013f190 0x7f58600cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:12.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.916+0000 7f585fe00640 1 -- 172.21.15.19:0/3190338277 >> v1:172.21.15.19:6789/0 conn(0x7f586006d620 legacy=0x7f586006da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:12.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.916+0000 7f585fe00640 1 -- 172.21.15.19:0/3190338277 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58600cee30 con 0x7f58600cc880 2024-03-28T13:11:12.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.916+0000 7f585f400640 1 -- 172.21.15.19:0/3190338277 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1694962731 0 0) 0x7f58600ce890 con 0x7f586006d620 2024-03-28T13:11:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.917+0000 7f585fe00640 1 --2- 172.21.15.19:0/3190338277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58600cc880 0x7f58600cccc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f584c0097a0 tx=0x7f584c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8210b84c29aaea server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.917+0000 7f585f400640 1 -- 172.21.15.19:0/3190338277 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c035020 con 0x7f58600cc880 2024-03-28T13:11:12.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.917+0000 7f585f400640 1 -- 172.21.15.19:0/3190338277 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f584c031810 con 0x7f58600cc880 2024-03-28T13:11:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.917+0000 7f585f400640 1 -- 172.21.15.19:0/3190338277 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c031be0 con 0x7f58600cc880 2024-03-28T13:11:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.918+0000 7f5866600640 1 -- 172.21.15.19:0/3190338277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58600cc880 msgr2=0x7f58600cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.918+0000 7f5866600640 1 --2- 172.21.15.19:0/3190338277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58600cc880 0x7f58600cccc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f584c0097a0 tx=0x7f584c02b8b0 comp rx=0 tx=0).stop 2024-03-28T13:11:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.918+0000 7f5866600640 1 -- 172.21.15.19:0/3190338277 shutdown_connections 2024-03-28T13:11:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.918+0000 7f5866600640 1 --2- 172.21.15.19:0/3190338277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58600cc880 0x7f58600cccc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.918+0000 7f5866600640 1 --2- 172.21.15.19:0/3190338277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f586013f190 0x7f58600cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.918+0000 7f5866600640 1 -- 172.21.15.19:0/3190338277 >> 172.21.15.19:0/3190338277 conn(0x7f58600b1ba0 msgr2=0x7f58600b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.918+0000 7f5866600640 1 -- 172.21.15.19:0/3190338277 shutdown_connections 2024-03-28T13:11:12.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.918+0000 7f5866600640 1 -- 172.21.15.19:0/3190338277 wait complete. 2024-03-28T13:11:12.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.920+0000 7f5866600640 1 Processor -- start 2024-03-28T13:11:12.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.920+0000 7f5866600640 1 -- start start 2024-03-28T13:11:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.921+0000 7f5866600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f586006d620 0x7f58600aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.921+0000 7f5866600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58600cc880 0x7f58601e8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.921+0000 7f5866600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f586013f190 0x7f58601e8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.921+0000 7f5866600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f586013f570 con 0x7f586006d620 2024-03-28T13:11:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.921+0000 7f5866600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5860160c70 con 0x7f58600cc880 2024-03-28T13:11:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.921+0000 7f5866600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f58600cfb70 con 0x7f586013f190 2024-03-28T13:11:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.921+0000 7f5865200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f586006d620 0x7f58600aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.921+0000 7f5865200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f586006d620 0x7f58600aec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45976/0 (socket says 172.21.15.19:45976) 2024-03-28T13:11:12.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.921+0000 7f5865200640 1 -- 172.21.15.19:0/3562438270 learned_addr learned my addr 172.21.15.19:0/3562438270 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.922+0000 7f5865c00640 1 --2- 172.21.15.19:0/3562438270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f586013f190 0x7f58601e8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.922+0000 7f585fe00640 1 --2- 172.21.15.19:0/3562438270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58600cc880 0x7f58601e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.922+0000 7f5865200640 1 -- 172.21.15.19:0/3562438270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f586013f190 msgr2=0x7f58601e8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.922+0000 7f5865200640 1 --2- 172.21.15.19:0/3562438270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f586013f190 0x7f58601e8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.922+0000 7f5865200640 1 -- 172.21.15.19:0/3562438270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58600cc880 msgr2=0x7f58601e8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.922+0000 7f5865200640 1 --2- 172.21.15.19:0/3562438270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58600cc880 0x7f58601e8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.922+0000 7f5865200640 1 -- 172.21.15.19:0/3562438270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5860129cb0 con 0x7f586006d620 2024-03-28T13:11:12.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.922+0000 7f5865200640 1 --2- 172.21.15.19:0/3562438270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f586006d620 0x7f58600aec00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f585400d830 tx=0x7f585400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:12.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.922+0000 7f585cc00640 1 -- 172.21.15.19:0/3562438270 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585400b6c0 con 0x7f586006d620 2024-03-28T13:11:12.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.923+0000 7f585cc00640 1 -- 172.21.15.19:0/3562438270 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5854019070 con 0x7f586006d620 2024-03-28T13:11:12.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.923+0000 7f585cc00640 1 -- 172.21.15.19:0/3562438270 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585400ba80 con 0x7f586006d620 2024-03-28T13:11:12.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.923+0000 7f5866600640 1 -- 172.21.15.19:0/3562438270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5860129f90 con 0x7f586006d620 2024-03-28T13:11:12.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.923+0000 7f5866600640 1 -- 172.21.15.19:0/3562438270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586012a550 con 0x7f586006d620 2024-03-28T13:11:12.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.924+0000 7f5866600640 1 -- 172.21.15.19:0/3562438270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5860075940 con 0x7f586006d620 2024-03-28T13:11:12.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.925+0000 7f585cc00640 1 -- 172.21.15.19:0/3562438270 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f585401e460 con 0x7f586006d620 2024-03-28T13:11:12.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.926+0000 7f585cc00640 1 --2- 172.21.15.19:0/3562438270 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f584006c530 0x7f584006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:12.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.926+0000 7f585cc00640 1 -- 172.21.15.19:0/3562438270 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f585408be00 con 0x7f586006d620 2024-03-28T13:11:12.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.926+0000 7f585fe00640 1 --2- 172.21.15.19:0/3562438270 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f584006c530 0x7f584006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:12.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.928+0000 7f585fe00640 1 --2- 172.21.15.19:0/3562438270 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f584006c530 0x7f584006e9f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f584c036000 tx=0x7f584c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:12.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:12.930+0000 7f585cc00640 1 -- 172.21.15.19:0/3562438270 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f585405a420 con 0x7f586006d620 2024-03-28T13:11:13.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.132+0000 7f5866600640 1 -- 172.21.15.19:0/3562438270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f58600b2870 con 0x7f586006d620 2024-03-28T13:11:13.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.132+0000 7f585cc00640 1 -- 172.21.15.19:0/3562438270 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) ==== 93+0+10 (secure 0 0 0) 0x7f5854059d20 con 0x7f586006d620 2024-03-28T13:11:13.133 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:11:13.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.135+0000 7f5866600640 1 -- 172.21.15.19:0/3562438270 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f584006c530 msgr2=0x7f584006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.135+0000 7f5866600640 1 --2- 172.21.15.19:0/3562438270 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f584006c530 0x7f584006e9f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f584c036000 tx=0x7f584c008030 comp rx=0 tx=0).stop 2024-03-28T13:11:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.135+0000 7f5866600640 1 -- 172.21.15.19:0/3562438270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f586006d620 msgr2=0x7f58600aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.135+0000 7f5866600640 1 --2- 172.21.15.19:0/3562438270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f586006d620 0x7f58600aec00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f585400d830 tx=0x7f585400dcf0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.135+0000 7f5866600640 1 -- 172.21.15.19:0/3562438270 shutdown_connections 2024-03-28T13:11:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.135+0000 7f5866600640 1 --2- 172.21.15.19:0/3562438270 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f584006c530 0x7f584006e9f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.135+0000 7f5866600640 1 --2- 172.21.15.19:0/3562438270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f586013f190 0x7f58601e8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.135+0000 7f5866600640 1 --2- 172.21.15.19:0/3562438270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58600cc880 0x7f58601e8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.135+0000 7f5866600640 1 --2- 172.21.15.19:0/3562438270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f586006d620 0x7f58600aec00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.135+0000 7f5866600640 1 -- 172.21.15.19:0/3562438270 >> 172.21.15.19:0/3562438270 conn(0x7f58600b1ba0 msgr2=0x7f586015b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.135+0000 7f5866600640 1 -- 172.21.15.19:0/3562438270 shutdown_connections 2024-03-28T13:11:13.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.135+0000 7f5866600640 1 -- 172.21.15.19:0/3562438270 wait complete. 2024-03-28T13:11:13.154 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:11:13.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.379+0000 7f1c6f000640 1 Processor -- start 2024-03-28T13:11:13.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.379+0000 7f1c6f000640 1 -- start start 2024-03-28T13:11:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.381+0000 7f1c6f000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1c680cc880 0x7f1c680cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.381+0000 7f1c6f000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c6806d620 0x7f1c6806da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.381+0000 7f1c6f000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c680ce890 con 0x7f1c6813f190 2024-03-28T13:11:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.381+0000 7f1c6f000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1c680cd230 con 0x7f1c680cc880 2024-03-28T13:11:13.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.381+0000 7f1c6f000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1c680ceb50 con 0x7f1c6806d620 2024-03-28T13:11:13.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.382+0000 7f1c6d200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1c680cc880 0x7f1c680cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:13.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.382+0000 7f1c6e600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c6806d620 0x7f1c6806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:13.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.382+0000 7f1c6e600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c6806d620 0x7f1c6806da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:47892/0 (socket says 172.21.15.19:47892) 2024-03-28T13:11:13.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.382+0000 7f1c6d200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1c680cc880 0x7f1c680cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:41620/0 (socket says 172.21.15.19:41620) 2024-03-28T13:11:13.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.382+0000 7f1c6e600640 1 -- 172.21.15.19:0/2288650962 learned_addr learned my addr 172.21.15.19:0/2288650962 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.383+0000 7f1c6e600640 1 -- 172.21.15.19:0/2288650962 >> v2:172.21.15.188:3300/0 conn(0x7f1c680cc880 msgr2=0x7f1c680cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.383+0000 7f1c6e600640 1 --2- 172.21.15.19:0/2288650962 >> v2:172.21.15.188:3300/0 conn(0x7f1c680cc880 0x7f1c680cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.383+0000 7f1c6e600640 1 -- 172.21.15.19:0/2288650962 >> v1:172.21.15.188:6789/0 conn(0x7f1c6813f190 legacy=0x7f1c680cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.383+0000 7f1c6e600640 1 -- 172.21.15.19:0/2288650962 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c680cee30 con 0x7f1c6806d620 2024-03-28T13:11:13.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.383+0000 7f1c6d200640 1 --2- 172.21.15.19:0/2288650962 >> v2:172.21.15.188:3300/0 conn(0x7f1c680cc880 0x7f1c680cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:11:13.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.384+0000 7f1c6e600640 1 --2- 172.21.15.19:0/2288650962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c6806d620 0x7f1c6806da50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009e50 tx=0x7f1c6002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa80d50836436865 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.385+0000 7f1c67e00640 1 -- 172.21.15.19:0/2288650962 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6002cb00 con 0x7f1c6806d620 2024-03-28T13:11:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.385+0000 7f1c67e00640 1 -- 172.21.15.19:0/2288650962 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c6002cca0 con 0x7f1c6806d620 2024-03-28T13:11:13.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.385+0000 7f1c67e00640 1 -- 172.21.15.19:0/2288650962 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60034610 con 0x7f1c6806d620 2024-03-28T13:11:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.386+0000 7f1c6f000640 1 -- 172.21.15.19:0/2288650962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c6806d620 msgr2=0x7f1c6806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.386+0000 7f1c6f000640 1 --2- 172.21.15.19:0/2288650962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c6806d620 0x7f1c6806da50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009e50 tx=0x7f1c6002bf60 comp rx=0 tx=0).stop 2024-03-28T13:11:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.386+0000 7f1c6f000640 1 -- 172.21.15.19:0/2288650962 shutdown_connections 2024-03-28T13:11:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.386+0000 7f1c6f000640 1 --2- 172.21.15.19:0/2288650962 >> v2:172.21.15.188:3300/0 conn(0x7f1c680cc880 0x7f1c680cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.386+0000 7f1c6f000640 1 --2- 172.21.15.19:0/2288650962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c6806d620 0x7f1c6806da50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.386+0000 7f1c6f000640 1 -- 172.21.15.19:0/2288650962 >> 172.21.15.19:0/2288650962 conn(0x7f1c680b1ba0 msgr2=0x7f1c680b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:13.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.386+0000 7f1c6f000640 1 -- 172.21.15.19:0/2288650962 shutdown_connections 2024-03-28T13:11:13.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.386+0000 7f1c6f000640 1 -- 172.21.15.19:0/2288650962 wait complete. 2024-03-28T13:11:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.388+0000 7f1c6f000640 1 Processor -- start 2024-03-28T13:11:13.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.388+0000 7f1c6f000640 1 -- start start 2024-03-28T13:11:13.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.389+0000 7f1c6f000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6806d620 0x7f1c681f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:13.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.389+0000 7f1c6f000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c680cc880 0x7f1c681f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:13.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.389+0000 7f1c6f000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6813f190 0x7f1c681f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:13.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.389+0000 7f1c6f000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6813f570 con 0x7f1c6806d620 2024-03-28T13:11:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.389+0000 7f1c6f000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1c68160c70 con 0x7f1c6813f190 2024-03-28T13:11:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.389+0000 7f1c6f000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1c6806df90 con 0x7f1c680cc880 2024-03-28T13:11:13.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.390+0000 7f1c6dc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6806d620 0x7f1c681f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:13.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.390+0000 7f1c6dc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6806d620 0x7f1c681f36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45982/0 (socket says 172.21.15.19:45982) 2024-03-28T13:11:13.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.390+0000 7f1c6d200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c680cc880 0x7f1c681f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:13.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.390+0000 7f1c6dc00640 1 -- 172.21.15.19:0/2673070594 learned_addr learned my addr 172.21.15.19:0/2673070594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:13.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.390+0000 7f1c6e600640 1 --2- 172.21.15.19:0/2673070594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6813f190 0x7f1c681f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:13.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.390+0000 7f1c6dc00640 1 -- 172.21.15.19:0/2673070594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c680cc880 msgr2=0x7f1c681f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:13.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.390+0000 7f1c6dc00640 1 --2- 172.21.15.19:0/2673070594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c680cc880 0x7f1c681f4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.390+0000 7f1c6dc00640 1 -- 172.21.15.19:0/2673070594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6813f190 msgr2=0x7f1c681f4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:13.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.390+0000 7f1c6dc00640 1 --2- 172.21.15.19:0/2673070594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6813f190 0x7f1c681f4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.390+0000 7f1c6dc00640 1 -- 172.21.15.19:0/2673070594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c681f4ba0 con 0x7f1c6806d620 2024-03-28T13:11:13.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.391+0000 7f1c6dc00640 1 --2- 172.21.15.19:0/2673070594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6806d620 0x7f1c681f36e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00c4d0 tx=0x7f1c5c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:13.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.391+0000 7f1c65600640 1 -- 172.21.15.19:0/2673070594 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0052d0 con 0x7f1c6806d620 2024-03-28T13:11:13.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.391+0000 7f1c65600640 1 -- 172.21.15.19:0/2673070594 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c5c005470 con 0x7f1c6806d620 2024-03-28T13:11:13.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.391+0000 7f1c6f000640 1 -- 172.21.15.19:0/2673070594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c681e5d50 con 0x7f1c6806d620 2024-03-28T13:11:13.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.391+0000 7f1c65600640 1 -- 172.21.15.19:0/2673070594 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c012850 con 0x7f1c6806d620 2024-03-28T13:11:13.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.391+0000 7f1c6f000640 1 -- 172.21.15.19:0/2673070594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c681e6310 con 0x7f1c6806d620 2024-03-28T13:11:13.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.392+0000 7f1c6f000640 1 -- 172.21.15.19:0/2673070594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c68075940 con 0x7f1c6806d620 2024-03-28T13:11:13.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.393+0000 7f1c65600640 1 -- 172.21.15.19:0/2673070594 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1c5c0129f0 con 0x7f1c6806d620 2024-03-28T13:11:13.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.394+0000 7f1c65600640 1 --2- 172.21.15.19:0/2673070594 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1c4806c580 0x7f1c4806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:13.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.394+0000 7f1c65600640 1 -- 172.21.15.19:0/2673070594 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f1c5c08b170 con 0x7f1c6806d620 2024-03-28T13:11:13.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.395+0000 7f1c6d200640 1 --2- 172.21.15.19:0/2673070594 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1c4806c580 0x7f1c4806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:13.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.396+0000 7f1c6d200640 1 --2- 172.21.15.19:0/2673070594 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1c4806c580 0x7f1c4806ea40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009560 tx=0x7f1c54009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:13.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.399+0000 7f1c65600640 1 -- 172.21.15.19:0/2673070594 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1c681f4ba0 con 0x7f1c6806d620 2024-03-28T13:11:13.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.602+0000 7f1c6f000640 1 -- 172.21.15.19:0/2673070594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1c680b2870 con 0x7f1c6806d620 2024-03-28T13:11:13.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.603+0000 7f1c65600640 1 -- 172.21.15.19:0/2673070594 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) ==== 104+0+11 (secure 0 0 0) 0x7f1c5c059120 con 0x7f1c6806d620 2024-03-28T13:11:13.603 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:11:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.605+0000 7f1c6f000640 1 -- 172.21.15.19:0/2673070594 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1c4806c580 msgr2=0x7f1c4806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.605+0000 7f1c6f000640 1 --2- 172.21.15.19:0/2673070594 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1c4806c580 0x7f1c4806ea40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009560 tx=0x7f1c54009190 comp rx=0 tx=0).stop 2024-03-28T13:11:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.605+0000 7f1c6f000640 1 -- 172.21.15.19:0/2673070594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6806d620 msgr2=0x7f1c681f36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.605+0000 7f1c6f000640 1 --2- 172.21.15.19:0/2673070594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6806d620 0x7f1c681f36e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00c4d0 tx=0x7f1c5c00c990 comp rx=0 tx=0).stop 2024-03-28T13:11:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.605+0000 7f1c6f000640 1 -- 172.21.15.19:0/2673070594 shutdown_connections 2024-03-28T13:11:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.605+0000 7f1c6f000640 1 --2- 172.21.15.19:0/2673070594 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1c4806c580 0x7f1c4806ea40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.605+0000 7f1c6f000640 1 --2- 172.21.15.19:0/2673070594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c6813f190 0x7f1c681f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.605+0000 7f1c6f000640 1 --2- 172.21.15.19:0/2673070594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c680cc880 0x7f1c681f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.606+0000 7f1c6f000640 1 --2- 172.21.15.19:0/2673070594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c6806d620 0x7f1c681f36e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.606+0000 7f1c6f000640 1 -- 172.21.15.19:0/2673070594 >> 172.21.15.19:0/2673070594 conn(0x7f1c680b1ba0 msgr2=0x7f1c6813dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.606+0000 7f1c6f000640 1 -- 172.21.15.19:0/2673070594 shutdown_connections 2024-03-28T13:11:13.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.606+0000 7f1c6f000640 1 -- 172.21.15.19:0/2673070594 wait complete. 2024-03-28T13:11:13.625 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:11:13.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.840+0000 7fcfb3000640 1 Processor -- start 2024-03-28T13:11:13.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.840+0000 7fcfb3000640 1 -- start start 2024-03-28T13:11:13.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.841+0000 7fcfb3000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfac167450 0x7fcfac169840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:13.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.841+0000 7fcfb3000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfac15ec80 con 0x7fcfac141c10 2024-03-28T13:11:13.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.841+0000 7fcfb3000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfac15ef20 con 0x7fcfac0d0280 2024-03-28T13:11:13.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.841+0000 7fcfb3000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcfac15f200 con 0x7fcfac167450 2024-03-28T13:11:13.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.842+0000 7fcfab400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcfac141c10 0x7fcfac071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49826/0 (socket says 172.21.15.19:49826) 2024-03-28T13:11:13.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.842+0000 7fcfab400640 1 -- 172.21.15.19:0/237889427 learned_addr learned my addr 172.21.15.19:0/237889427 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:13.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.842+0000 7fcfb1000640 1 --2- 172.21.15.19:0/237889427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfac167450 0x7fcfac169840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:13.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.842+0000 7fcfaaa00640 1 -- 172.21.15.19:0/237889427 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3650320532 0 0) 0x7fcfac15ec80 con 0x7fcfac141c10 2024-03-28T13:11:13.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.842+0000 7fcfaaa00640 1 -- 172.21.15.19:0/237889427 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf90003520 con 0x7fcfac141c10 2024-03-28T13:11:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.843+0000 7fcfaaa00640 1 -- 172.21.15.19:0/237889427 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4205733647 0 0) 0x7fcf90003520 con 0x7fcfac141c10 2024-03-28T13:11:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.843+0000 7fcfaaa00640 1 -- 172.21.15.19:0/237889427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfac167450 msgr2=0x7fcfac169840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.843+0000 7fcfaaa00640 1 --2- 172.21.15.19:0/237889427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfac167450 0x7fcfac169840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.843+0000 7fcfaaa00640 1 -- 172.21.15.19:0/237889427 >> v1:172.21.15.188:6789/0 conn(0x7fcfac0d0280 legacy=0x7fcfac0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.843+0000 7fcfaaa00640 1 -- 172.21.15.19:0/237889427 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfac15f4e0 con 0x7fcfac141c10 2024-03-28T13:11:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.843+0000 7fcfaaa00640 1 -- 172.21.15.19:0/237889427 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fcf94001690 con 0x7fcfac141c10 2024-03-28T13:11:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.843+0000 7fcfaaa00640 1 -- 172.21.15.19:0/237889427 >> v1:172.21.15.19:6789/0 conn(0x7fcfac141c10 legacy=0x7fcfac071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.843+0000 7fcfaaa00640 1 --2- 172.21.15.19:0/237889427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfac167450 0x7fcf900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.843+0000 7fcfaaa00640 1 --2- 172.21.15.19:0/237889427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90003f20 0x7fcf90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.843+0000 7fcfaaa00640 1 --2- 172.21.15.19:0/237889427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf90004d40 0x7fcf900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.843+0000 7fcfaaa00640 1 -- 172.21.15.19:0/237889427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac15f200 con 0x7fcf90003f20 2024-03-28T13:11:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.843+0000 7fcfaaa00640 1 -- 172.21.15.19:0/237889427 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9000ba70 con 0x7fcf90004d40 2024-03-28T13:11:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.843+0000 7fcfaaa00640 1 -- 172.21.15.19:0/237889427 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9000bc70 con 0x7fcfac167450 2024-03-28T13:11:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.843+0000 7fcfab400640 1 --2- 172.21.15.19:0/237889427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90003f20 0x7fcf90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:13.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.844+0000 7fcfb1000640 1 --2- 172.21.15.19:0/237889427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf90004d40 0x7fcf900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:13.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.844+0000 7fcfabe00640 1 --2- 172.21.15.19:0/237889427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfac167450 0x7fcf900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:13.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.844+0000 7fcfab400640 1 -- 172.21.15.19:0/237889427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfac167450 msgr2=0x7fcf900039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:13.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.844+0000 7fcfab400640 1 --2- 172.21.15.19:0/237889427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfac167450 0x7fcf900039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.844+0000 7fcfab400640 1 -- 172.21.15.19:0/237889427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf90004d40 msgr2=0x7fcf900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:13.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.844+0000 7fcfab400640 1 --2- 172.21.15.19:0/237889427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf90004d40 0x7fcf900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.844+0000 7fcfab400640 1 -- 172.21.15.19:0/237889427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf90003520 con 0x7fcf90003f20 2024-03-28T13:11:13.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.844+0000 7fcfb1000640 1 --2- 172.21.15.19:0/237889427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf90004d40 0x7fcf900051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:11:13.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.846+0000 7fcfab400640 1 --2- 172.21.15.19:0/237889427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90003f20 0x7fcf90004800 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcfac148260 tx=0x7fcf90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57166824a3c669a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:13.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.846+0000 7fcfaaa00640 1 -- 172.21.15.19:0/237889427 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94007c60 con 0x7fcf90003f20 2024-03-28T13:11:13.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.846+0000 7fcfaaa00640 1 -- 172.21.15.19:0/237889427 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf94030070 con 0x7fcf90003f20 2024-03-28T13:11:13.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.846+0000 7fcfaaa00640 1 -- 172.21.15.19:0/237889427 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94005cd0 con 0x7fcf90003f20 2024-03-28T13:11:13.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.846+0000 7fcfb3000640 1 -- 172.21.15.19:0/237889427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90003f20 msgr2=0x7fcf90004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:13.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.846+0000 7fcfb3000640 1 --2- 172.21.15.19:0/237889427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90003f20 0x7fcf90004800 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcfac148260 tx=0x7fcf90005830 comp rx=0 tx=0).stop 2024-03-28T13:11:13.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.847+0000 7fcfb1000640 1 -- 172.21.15.19:0/237889427 reap_dead start 2024-03-28T13:11:13.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.847+0000 7fcfb3000640 1 -- 172.21.15.19:0/237889427 shutdown_connections 2024-03-28T13:11:13.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.847+0000 7fcfb3000640 1 -- 172.21.15.19:0/237889427 >> 172.21.15.19:0/237889427 conn(0x7fcfac06b750 msgr2=0x7fcfac06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:13.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.847+0000 7fcfb3000640 1 -- 172.21.15.19:0/237889427 shutdown_connections 2024-03-28T13:11:13.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.847+0000 7fcfb3000640 1 -- 172.21.15.19:0/237889427 wait complete. 2024-03-28T13:11:13.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.848+0000 7fcfb3000640 1 Processor -- start 2024-03-28T13:11:13.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.849+0000 7fcfb3000640 1 -- start start 2024-03-28T13:11:13.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.849+0000 7fcfb3000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90003f20 0x7fcfac152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:13.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.849+0000 7fcfb3000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfac0995d0 0x7fcfac098c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:13.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.850+0000 7fcfb3000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfac156100 0x7fcfac0b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:13.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.850+0000 7fcfb3000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac158e90 con 0x7fcf90003f20 2024-03-28T13:11:13.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.850+0000 7fcfb3000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcfac06f340 con 0x7fcfac156100 2024-03-28T13:11:13.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.850+0000 7fcfb3000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcfac1553d0 con 0x7fcfac0995d0 2024-03-28T13:11:13.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.850+0000 7fcfabe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90003f20 0x7fcfac152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:13.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.850+0000 7fcfabe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90003f20 0x7fcfac152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46008/0 (socket says 172.21.15.19:46008) 2024-03-28T13:11:13.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.850+0000 7fcfabe00640 1 -- 172.21.15.19:0/1291700479 learned_addr learned my addr 172.21.15.19:0/1291700479 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.850+0000 7fcfb1000640 1 --2- 172.21.15.19:0/1291700479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfac156100 0x7fcfac0b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.850+0000 7fcfab400640 1 --2- 172.21.15.19:0/1291700479 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfac0995d0 0x7fcfac098c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.850+0000 7fcfabe00640 1 -- 172.21.15.19:0/1291700479 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfac0995d0 msgr2=0x7fcfac098c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.850+0000 7fcfabe00640 1 --2- 172.21.15.19:0/1291700479 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfac0995d0 0x7fcfac098c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.850+0000 7fcfabe00640 1 -- 172.21.15.19:0/1291700479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfac156100 msgr2=0x7fcfac0b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.850+0000 7fcfabe00640 1 --2- 172.21.15.19:0/1291700479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfac156100 0x7fcfac0b01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.850+0000 7fcfabe00640 1 -- 172.21.15.19:0/1291700479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfac1574b0 con 0x7fcf90003f20 2024-03-28T13:11:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.851+0000 7fcfabe00640 1 --2- 172.21.15.19:0/1291700479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90003f20 0x7fcfac152860 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00e930 tx=0x7fcf9c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.851+0000 7fcfa3e00640 1 -- 172.21.15.19:0/1291700479 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c003b80 con 0x7fcf90003f20 2024-03-28T13:11:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.851+0000 7fcfa3e00640 1 -- 172.21.15.19:0/1291700479 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf9c018070 con 0x7fcf90003f20 2024-03-28T13:11:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.851+0000 7fcfa3e00640 1 -- 172.21.15.19:0/1291700479 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c00c780 con 0x7fcf90003f20 2024-03-28T13:11:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.851+0000 7fcfb3000640 1 -- 172.21.15.19:0/1291700479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfac155620 con 0x7fcf90003f20 2024-03-28T13:11:13.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.851+0000 7fcfb3000640 1 -- 172.21.15.19:0/1291700479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfac155be0 con 0x7fcf90003f20 2024-03-28T13:11:13.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.852+0000 7fcfb3000640 1 -- 172.21.15.19:0/1291700479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfac155de0 con 0x7fcf90003f20 2024-03-28T13:11:13.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.853+0000 7fcfa3e00640 1 -- 172.21.15.19:0/1291700479 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcf9c00c920 con 0x7fcf90003f20 2024-03-28T13:11:13.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.854+0000 7fcfa3e00640 1 --2- 172.21.15.19:0/1291700479 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcf8806c530 0x7fcf8806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:13.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.854+0000 7fcfa3e00640 1 -- 172.21.15.19:0/1291700479 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fcf9c08b7b0 con 0x7fcf90003f20 2024-03-28T13:11:13.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.855+0000 7fcfab400640 1 --2- 172.21.15.19:0/1291700479 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcf8806c530 0x7fcf8806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:13.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.856+0000 7fcfab400640 1 --2- 172.21.15.19:0/1291700479 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcf8806c530 0x7fcf8806e9f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcfac148260 tx=0x7fcf94007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:13.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:13.859+0000 7fcfa3e00640 1 -- 172.21.15.19:0/1291700479 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcfac155de0 con 0x7fcf90003f20 2024-03-28T13:11:14.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.069+0000 7fcfb3000640 1 -- 172.21.15.19:0/1291700479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcfac0685c0 con 0x7fcf90003f20 2024-03-28T13:11:14.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.070+0000 7fcfa3e00640 1 -- 172.21.15.19:0/1291700479 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) ==== 100+0+11 (secure 0 0 0) 0x7fcf9c059e60 con 0x7fcf90003f20 2024-03-28T13:11:14.070 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:11:14.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.073+0000 7fcfb3000640 1 -- 172.21.15.19:0/1291700479 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcf8806c530 msgr2=0x7fcf8806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:14.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.073+0000 7fcfb3000640 1 --2- 172.21.15.19:0/1291700479 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcf8806c530 0x7fcf8806e9f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcfac148260 tx=0x7fcf94007000 comp rx=0 tx=0).stop 2024-03-28T13:11:14.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.073+0000 7fcfb3000640 1 -- 172.21.15.19:0/1291700479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90003f20 msgr2=0x7fcfac152860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:14.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.073+0000 7fcfb3000640 1 --2- 172.21.15.19:0/1291700479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90003f20 0x7fcfac152860 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00e930 tx=0x7fcf9c00edf0 comp rx=0 tx=0).stop 2024-03-28T13:11:14.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.073+0000 7fcfb3000640 1 -- 172.21.15.19:0/1291700479 shutdown_connections 2024-03-28T13:11:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.073+0000 7fcfb3000640 1 --2- 172.21.15.19:0/1291700479 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcf8806c530 0x7fcf8806e9f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.073+0000 7fcfb3000640 1 --2- 172.21.15.19:0/1291700479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcfac156100 0x7fcfac0b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.073+0000 7fcfb3000640 1 --2- 172.21.15.19:0/1291700479 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcfac0995d0 0x7fcfac098c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.073+0000 7fcfb3000640 1 --2- 172.21.15.19:0/1291700479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf90003f20 0x7fcfac152860 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.073+0000 7fcfb3000640 1 -- 172.21.15.19:0/1291700479 >> 172.21.15.19:0/1291700479 conn(0x7fcfac06b750 msgr2=0x7fcfac15fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.073+0000 7fcfb3000640 1 -- 172.21.15.19:0/1291700479 shutdown_connections 2024-03-28T13:11:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.073+0000 7fcfb3000640 1 -- 172.21.15.19:0/1291700479 wait complete. 2024-03-28T13:11:14.092 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:11:14.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.316+0000 7f65fb200640 1 Processor -- start 2024-03-28T13:11:14.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.316+0000 7f65fb200640 1 -- start start 2024-03-28T13:11:14.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.318+0000 7f65fb200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f65f4147a40 0x7f65f40cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:14.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.318+0000 7f65fb200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f65f40d01c0 0x7f65f40cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:14.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.318+0000 7f65fb200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65f4161f30 con 0x7f65f40ce1b0 2024-03-28T13:11:14.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.318+0000 7f65fb200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f65f4162b00 con 0x7f65f40d01c0 2024-03-28T13:11:14.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.318+0000 7f65fb200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f65f41621d0 con 0x7f65f4147a40 2024-03-28T13:11:14.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.318+0000 7f65f9400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f65f40d01c0 0x7f65f40cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:14.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.318+0000 7f65f9400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f65f40d01c0 0x7f65f40cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46010/0 (socket says 172.21.15.19:46010) 2024-03-28T13:11:14.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.318+0000 7f65f9400640 1 -- 172.21.15.19:0/1321086025 learned_addr learned my addr 172.21.15.19:0/1321086025 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:14.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.318+0000 7f65f9e00640 1 --2- 172.21.15.19:0/1321086025 >> v2:172.21.15.188:3300/0 conn(0x7f65f4147a40 0x7f65f40cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:14.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.318+0000 7f65f9400640 1 -- 172.21.15.19:0/1321086025 >> v2:172.21.15.188:3300/0 conn(0x7f65f4147a40 msgr2=0x7f65f40cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:14.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.318+0000 7f65f9400640 1 --2- 172.21.15.19:0/1321086025 >> v2:172.21.15.188:3300/0 conn(0x7f65f4147a40 0x7f65f40cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:14.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.318+0000 7f65f9400640 1 -- 172.21.15.19:0/1321086025 >> v1:172.21.15.188:6789/0 conn(0x7f65f40ce1b0 legacy=0x7f65f40ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:14.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.319+0000 7f65f9400640 1 -- 172.21.15.19:0/1321086025 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f41624b0 con 0x7f65f40d01c0 2024-03-28T13:11:14.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.319+0000 7f65f9e00640 1 --2- 172.21.15.19:0/1321086025 >> v2:172.21.15.188:3300/0 conn(0x7f65f4147a40 0x7f65f40cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:14.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.320+0000 7f65f9400640 1 --2- 172.21.15.19:0/1321086025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65f40d01c0 0x7f65f40cdc70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f65f40cf9f0 tx=0x7f65e002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba750911c97ba1a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:14.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.320+0000 7f65f8a00640 1 -- 172.21.15.19:0/1321086025 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e0035020 con 0x7f65f40d01c0 2024-03-28T13:11:14.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.320+0000 7f65f8a00640 1 -- 172.21.15.19:0/1321086025 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65e0031640 con 0x7f65f40d01c0 2024-03-28T13:11:14.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.320+0000 7f65f8a00640 1 -- 172.21.15.19:0/1321086025 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e002d040 con 0x7f65f40d01c0 2024-03-28T13:11:14.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.321+0000 7f65fb200640 1 -- 172.21.15.19:0/1321086025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65f40d01c0 msgr2=0x7f65f40cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:14.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.321+0000 7f65fb200640 1 --2- 172.21.15.19:0/1321086025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65f40d01c0 0x7f65f40cdc70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f65f40cf9f0 tx=0x7f65e002b8f0 comp rx=0 tx=0).stop 2024-03-28T13:11:14.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.321+0000 7f65fb200640 1 -- 172.21.15.19:0/1321086025 shutdown_connections 2024-03-28T13:11:14.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.321+0000 7f65fb200640 1 --2- 172.21.15.19:0/1321086025 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65f40d01c0 0x7f65f40cdc70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:14.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.321+0000 7f65fb200640 1 --2- 172.21.15.19:0/1321086025 >> v2:172.21.15.188:3300/0 conn(0x7f65f4147a40 0x7f65f40cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:14.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.321+0000 7f65fb200640 1 -- 172.21.15.19:0/1321086025 >> 172.21.15.19:0/1321086025 conn(0x7f65f40ccad0 msgr2=0x7f65f40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:14.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.321+0000 7f65fb200640 1 -- 172.21.15.19:0/1321086025 shutdown_connections 2024-03-28T13:11:14.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.321+0000 7f65fb200640 1 -- 172.21.15.19:0/1321086025 wait complete. 2024-03-28T13:11:14.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.323+0000 7f65fb200640 1 Processor -- start 2024-03-28T13:11:14.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.323+0000 7f65fb200640 1 -- start start 2024-03-28T13:11:14.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.324+0000 7f65fb200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65f40ce1b0 0x7f65f40b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:14.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.324+0000 7f65fb200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65f40d01c0 0x7f65f40b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:14.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.324+0000 7f65fb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65f4147a40 0x7f65f4140af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:14.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.324+0000 7f65fb200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4147e20 con 0x7f65f4147a40 2024-03-28T13:11:14.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.324+0000 7f65fb200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f65f40d0620 con 0x7f65f40d01c0 2024-03-28T13:11:14.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.324+0000 7f65fb200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f65f4147fa0 con 0x7f65f40ce1b0 2024-03-28T13:11:14.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.324+0000 7f65fa800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65f4147a40 0x7f65f4140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:14.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.324+0000 7f65fa800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65f4147a40 0x7f65f4140af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46022/0 (socket says 172.21.15.19:46022) 2024-03-28T13:11:14.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.324+0000 7f65fa800640 1 -- 172.21.15.19:0/120839078 learned_addr learned my addr 172.21.15.19:0/120839078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:14.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.325+0000 7f65f9400640 1 --2- 172.21.15.19:0/120839078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65f40d01c0 0x7f65f40b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:14.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.325+0000 7f65fa800640 1 -- 172.21.15.19:0/120839078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65f40ce1b0 msgr2=0x7f65f40b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:14.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.325+0000 7f65fa800640 1 --2- 172.21.15.19:0/120839078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65f40ce1b0 0x7f65f40b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:14.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.325+0000 7f65fa800640 1 -- 172.21.15.19:0/120839078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65f40d01c0 msgr2=0x7f65f40b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:14.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.325+0000 7f65fa800640 1 --2- 172.21.15.19:0/120839078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65f40d01c0 0x7f65f40b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:14.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.325+0000 7f65fa800640 1 -- 172.21.15.19:0/120839078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f40b33d0 con 0x7f65f4147a40 2024-03-28T13:11:14.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.325+0000 7f65fa800640 1 --2- 172.21.15.19:0/120839078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65f4147a40 0x7f65f4140af0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f65f000c7b0 tx=0x7f65f000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:14.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.325+0000 7f65ee000640 1 -- 172.21.15.19:0/120839078 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0004b60 con 0x7f65f4147a40 2024-03-28T13:11:14.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.325+0000 7f65ee000640 1 -- 172.21.15.19:0/120839078 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65f0004d00 con 0x7f65f4147a40 2024-03-28T13:11:14.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.325+0000 7f65ee000640 1 -- 172.21.15.19:0/120839078 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f000aa90 con 0x7f65f4147a40 2024-03-28T13:11:14.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.325+0000 7f65fb200640 1 -- 172.21.15.19:0/120839078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f40b36b0 con 0x7f65f4147a40 2024-03-28T13:11:14.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.326+0000 7f65fb200640 1 -- 172.21.15.19:0/120839078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f41e7e40 con 0x7f65f4147a40 2024-03-28T13:11:14.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.327+0000 7f65fb200640 1 -- 172.21.15.19:0/120839078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f4075940 con 0x7f65f4147a40 2024-03-28T13:11:14.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.328+0000 7f65ee000640 1 -- 172.21.15.19:0/120839078 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f65f000ac30 con 0x7f65f4147a40 2024-03-28T13:11:14.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.329+0000 7f65ee000640 1 --2- 172.21.15.19:0/120839078 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f65d406c530 0x7f65d406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:14.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.329+0000 7f65ee000640 1 -- 172.21.15.19:0/120839078 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7f65f008b9e0 con 0x7f65f4147a40 2024-03-28T13:11:14.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.329+0000 7f65f9e00640 1 --2- 172.21.15.19:0/120839078 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f65d406c530 0x7f65d406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:14.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.330+0000 7f65f9e00640 1 --2- 172.21.15.19:0/120839078 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f65d406c530 0x7f65d406e9f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f65e8006fd0 tx=0x7f65e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:14.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.334+0000 7f65ee000640 1 -- 172.21.15.19:0/120839078 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f65f0020070 con 0x7f65f4147a40 2024-03-28T13:11:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.558+0000 7f65fb200640 1 -- 172.21.15.19:0/120839078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f65f4071f90 con 0x7f65f4147a40 2024-03-28T13:11:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.559+0000 7f65ee000640 1 -- 172.21.15.19:0/120839078 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) ==== 80+0+0 (secure 0 0 0) 0x7f65f0059ff0 con 0x7f65f4147a40 2024-03-28T13:11:14.559 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-03-28T13:11:14.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.562+0000 7f65fb200640 1 -- 172.21.15.19:0/120839078 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f65d406c530 msgr2=0x7f65d406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:14.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.562+0000 7f65fb200640 1 --2- 172.21.15.19:0/120839078 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f65d406c530 0x7f65d406e9f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f65e8006fd0 tx=0x7f65e8008040 comp rx=0 tx=0).stop 2024-03-28T13:11:14.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.562+0000 7f65fb200640 1 -- 172.21.15.19:0/120839078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65f4147a40 msgr2=0x7f65f4140af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:14.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.562+0000 7f65fb200640 1 --2- 172.21.15.19:0/120839078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65f4147a40 0x7f65f4140af0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f65f000c7b0 tx=0x7f65f000cc70 comp rx=0 tx=0).stop 2024-03-28T13:11:14.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.562+0000 7f65fb200640 1 -- 172.21.15.19:0/120839078 shutdown_connections 2024-03-28T13:11:14.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.562+0000 7f65fb200640 1 --2- 172.21.15.19:0/120839078 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f65d406c530 0x7f65d406e9f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:14.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.562+0000 7f65fb200640 1 --2- 172.21.15.19:0/120839078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f65f4147a40 0x7f65f4140af0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:14.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.562+0000 7f65fb200640 1 --2- 172.21.15.19:0/120839078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65f40d01c0 0x7f65f40b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:14.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.562+0000 7f65fb200640 1 --2- 172.21.15.19:0/120839078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65f40ce1b0 0x7f65f40b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:14.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.562+0000 7f65fb200640 1 -- 172.21.15.19:0/120839078 >> 172.21.15.19:0/120839078 conn(0x7f65f40ccad0 msgr2=0x7f65f415fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:14.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.562+0000 7f65fb200640 1 -- 172.21.15.19:0/120839078 shutdown_connections 2024-03-28T13:11:14.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:14.562+0000 7f65fb200640 1 -- 172.21.15.19:0/120839078 wait complete. 2024-03-28T13:11:14.581 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:11:14.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:14.581 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T13:11:14.752 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:11:14.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:14.753 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:11:14.795 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:11:15.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.026+0000 7fd290c00640 1 Processor -- start 2024-03-28T13:11:15.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.026+0000 7fd290c00640 1 -- start start 2024-03-28T13:11:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.028+0000 7fd290c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28c0727f0 0x7fd28c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.028+0000 7fd290c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd28c15b6f0 con 0x7fd28c15b310 2024-03-28T13:11:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.028+0000 7fd290c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd28c072e80 con 0x7fd28c0d0780 2024-03-28T13:11:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.028+0000 7fd290c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd28c072c50 con 0x7fd28c0727f0 2024-03-28T13:11:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.028+0000 7fd289e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd28c15b310 0x7fd28c15f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49838/0 (socket says 172.21.15.19:49838) 2024-03-28T13:11:15.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.028+0000 7fd289e00640 1 -- 172.21.15.19:0/3133553696 learned_addr learned my addr 172.21.15.19:0/3133553696 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd281400640 1 --2- 172.21.15.19:0/3133553696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28c0727f0 0x7fd28c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3411878420 0 0) 0x7fd28c15b6f0 con 0x7fd28c15b310 2024-03-28T13:11:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd270005730 con 0x7fd28c15b310 2024-03-28T13:11:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 618486740 0 0) 0x7fd28c072e80 con 0x7fd28c0d0780 2024-03-28T13:11:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd28c15b6f0 con 0x7fd28c0d0780 2024-03-28T13:11:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1428237402 0 0) 0x7fd270005730 con 0x7fd28c15b310 2024-03-28T13:11:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28c0727f0 msgr2=0x7fd28c0b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 --2- 172.21.15.19:0/3133553696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28c0727f0 0x7fd28c0b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 >> v1:172.21.15.188:6789/0 conn(0x7fd28c0d0780 legacy=0x7fd28c0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c073160 con 0x7fd28c15b310 2024-03-28T13:11:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fd27c0015f0 con 0x7fd28c15b310 2024-03-28T13:11:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 >> v1:172.21.15.19:6789/0 conn(0x7fd28c15b310 legacy=0x7fd28c15f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 --2- 172.21.15.19:0/3133553696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28c0727f0 0x7fd270005d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:15.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 --2- 172.21.15.19:0/3133553696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270006330 0x7fd270006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 --2- 172.21.15.19:0/3133553696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270006f50 0x7fd2700073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd27000de00 con 0x7fd270006330 2024-03-28T13:11:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd27000dfb0 con 0x7fd270006f50 2024-03-28T13:11:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.029+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd27000e1b0 con 0x7fd28c0727f0 2024-03-28T13:11:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.030+0000 7fd281400640 1 --2- 172.21.15.19:0/3133553696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270006330 0x7fd270006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.030+0000 7fd28a800640 1 --2- 172.21.15.19:0/3133553696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270006f50 0x7fd2700073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.030+0000 7fd289e00640 1 --2- 172.21.15.19:0/3133553696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28c0727f0 0x7fd270005d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.030+0000 7fd281400640 1 -- 172.21.15.19:0/3133553696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28c0727f0 msgr2=0x7fd270005d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.030+0000 7fd281400640 1 --2- 172.21.15.19:0/3133553696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28c0727f0 0x7fd270005d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.030+0000 7fd281400640 1 -- 172.21.15.19:0/3133553696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270006f50 msgr2=0x7fd2700073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.030+0000 7fd281400640 1 --2- 172.21.15.19:0/3133553696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270006f50 0x7fd2700073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.030+0000 7fd281400640 1 -- 172.21.15.19:0/3133553696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd270005730 con 0x7fd270006330 2024-03-28T13:11:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.030+0000 7fd28a800640 1 --2- 172.21.15.19:0/3133553696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270006f50 0x7fd2700073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:15.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.030+0000 7fd289e00640 1 --2- 172.21.15.19:0/3133553696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28c0727f0 0x7fd270005d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.031+0000 7fd281400640 1 --2- 172.21.15.19:0/3133553696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270006330 0x7fd270006a10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd274009ac0 tx=0x7fd270007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b86fd284892ae29 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.032+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274039070 con 0x7fd270006330 2024-03-28T13:11:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.032+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd27402c6b0 con 0x7fd270006330 2024-03-28T13:11:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.032+0000 7fd289400640 1 -- 172.21.15.19:0/3133553696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd274009690 con 0x7fd270006330 2024-03-28T13:11:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.032+0000 7fd290c00640 1 -- 172.21.15.19:0/3133553696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270006330 msgr2=0x7fd270006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.032+0000 7fd290c00640 1 --2- 172.21.15.19:0/3133553696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd270006330 0x7fd270006a10 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd274009ac0 tx=0x7fd270007a40 comp rx=0 tx=0).stop 2024-03-28T13:11:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.032+0000 7fd28a800640 1 -- 172.21.15.19:0/3133553696 reap_dead start 2024-03-28T13:11:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.032+0000 7fd290c00640 1 -- 172.21.15.19:0/3133553696 shutdown_connections 2024-03-28T13:11:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.032+0000 7fd290c00640 1 -- 172.21.15.19:0/3133553696 >> 172.21.15.19:0/3133553696 conn(0x7fd28c06b990 msgr2=0x7fd28c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.032+0000 7fd290c00640 1 -- 172.21.15.19:0/3133553696 shutdown_connections 2024-03-28T13:11:15.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.032+0000 7fd290c00640 1 -- 172.21.15.19:0/3133553696 wait complete. 2024-03-28T13:11:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.034+0000 7fd290c00640 1 Processor -- start 2024-03-28T13:11:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.034+0000 7fd290c00640 1 -- start start 2024-03-28T13:11:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.035+0000 7fd290c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270006330 0x7fd28c150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.035+0000 7fd290c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28c151990 0x7fd28c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.035+0000 7fd290c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd28c0b0840 0x7fd28c0b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.035+0000 7fd290c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c073440 con 0x7fd28c0b0840 2024-03-28T13:11:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.035+0000 7fd290c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c15f2c0 con 0x7fd270006330 2024-03-28T13:11:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.035+0000 7fd290c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd28c0ae390 con 0x7fd28c151990 2024-03-28T13:11:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.036+0000 7fd28a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd28c0b0840 0x7fd28c0b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.036+0000 7fd289e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270006330 0x7fd28c150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.036+0000 7fd289e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270006330 0x7fd28c150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:41678/0 (socket says 172.21.15.19:41678) 2024-03-28T13:11:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.036+0000 7fd289e00640 1 -- 172.21.15.19:0/3752593190 learned_addr learned my addr 172.21.15.19:0/3752593190 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.036+0000 7fd281400640 1 --2- 172.21.15.19:0/3752593190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28c151990 0x7fd28c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:15.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.036+0000 7fd28a800640 1 -- 172.21.15.19:0/3752593190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28c151990 msgr2=0x7fd28c0b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:15.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.036+0000 7fd28a800640 1 --2- 172.21.15.19:0/3752593190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28c151990 0x7fd28c0b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:15.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.036+0000 7fd28a800640 1 -- 172.21.15.19:0/3752593190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270006330 msgr2=0x7fd28c150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:15.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.036+0000 7fd28a800640 1 --2- 172.21.15.19:0/3752593190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270006330 0x7fd28c150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:15.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.036+0000 7fd28a800640 1 -- 172.21.15.19:0/3752593190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c151e20 con 0x7fd28c0b0840 2024-03-28T13:11:15.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.037+0000 7fd28a800640 1 --2- 172.21.15.19:0/3752593190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd28c0b0840 0x7fd28c0b1f60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0b2eb0 tx=0x7fd28400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:15.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.037+0000 7fd282a00640 1 -- 172.21.15.19:0/3752593190 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284018070 con 0x7fd28c0b0840 2024-03-28T13:11:15.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.037+0000 7fd282a00640 1 -- 172.21.15.19:0/3752593190 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd284006d90 con 0x7fd28c0b0840 2024-03-28T13:11:15.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.037+0000 7fd290c00640 1 -- 172.21.15.19:0/3752593190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28c152100 con 0x7fd28c0b0840 2024-03-28T13:11:15.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.037+0000 7fd282a00640 1 -- 172.21.15.19:0/3752593190 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd284003bb0 con 0x7fd28c0b0840 2024-03-28T13:11:15.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.037+0000 7fd290c00640 1 -- 172.21.15.19:0/3752593190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28c1526c0 con 0x7fd28c0b0840 2024-03-28T13:11:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.038+0000 7fd290c00640 1 -- 172.21.15.19:0/3752593190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28c152900 con 0x7fd28c0b0840 2024-03-28T13:11:15.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.039+0000 7fd282a00640 1 -- 172.21.15.19:0/3752593190 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd28401b070 con 0x7fd28c0b0840 2024-03-28T13:11:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.040+0000 7fd282a00640 1 --2- 172.21.15.19:0/3752593190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd26806c530 0x7fd26806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.040+0000 7fd282a00640 1 -- 172.21.15.19:0/3752593190 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7fd28408a8a0 con 0x7fd28c0b0840 2024-03-28T13:11:15.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.041+0000 7fd289e00640 1 --2- 172.21.15.19:0/3752593190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd26806c530 0x7fd26806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:15.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.042+0000 7fd289e00640 1 --2- 172.21.15.19:0/3752593190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd26806c530 0x7fd26806e9f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd27c000ff0 tx=0x7fd27c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:15.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.045+0000 7fd282a00640 1 -- 172.21.15.19:0/3752593190 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd28c152900 con 0x7fd28c0b0840 2024-03-28T13:11:15.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.286+0000 7fd290c00640 1 -- 172.21.15.19:0/3752593190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd28c15ec20 con 0x7fd28c0b0840 2024-03-28T13:11:15.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.288+0000 7fd282a00640 1 -- 172.21.15.19:0/3752593190 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7fd2840587d0 con 0x7fd28c0b0840 2024-03-28T13:11:15.288 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:15.288 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2683971950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2683971950},{"type":"v1","addr":"172.21.15.188:6836","nonce":2683971950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/709274582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":709274582},{"type":"v1","addr":"172.21.15.19:6835","nonce":709274582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5040,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3420318181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3420318181},{"type":"v1","addr":"172.21.15.19:6833","nonce":3420318181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:10:56.216937+0000","modified":"2024-03-28T13:10:59.267258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5061":{"gid":5061,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6837/46284318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":46284318},{"type":"v1","addr":"172.21.15.188:6837","nonce":46284318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5061,"qdb_cluster":[5061]},"id":2}]} 2024-03-28T13:11:15.288 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-03-28T13:11:15.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.290+0000 7fd290c00640 1 -- 172.21.15.19:0/3752593190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd26806c530 msgr2=0x7fd26806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:15.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.290+0000 7fd290c00640 1 --2- 172.21.15.19:0/3752593190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd26806c530 0x7fd26806e9f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd27c000ff0 tx=0x7fd27c009040 comp rx=0 tx=0).stop 2024-03-28T13:11:15.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.290+0000 7fd290c00640 1 -- 172.21.15.19:0/3752593190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd28c0b0840 msgr2=0x7fd28c0b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:15.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.290+0000 7fd290c00640 1 --2- 172.21.15.19:0/3752593190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd28c0b0840 0x7fd28c0b1f60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd28c0b2eb0 tx=0x7fd28400db40 comp rx=0 tx=0).stop 2024-03-28T13:11:15.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.291+0000 7fd290c00640 1 -- 172.21.15.19:0/3752593190 shutdown_connections 2024-03-28T13:11:15.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.291+0000 7fd290c00640 1 --2- 172.21.15.19:0/3752593190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd26806c530 0x7fd26806e9f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:15.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.291+0000 7fd290c00640 1 --2- 172.21.15.19:0/3752593190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd28c0b0840 0x7fd28c0b1f60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:15.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.291+0000 7fd290c00640 1 --2- 172.21.15.19:0/3752593190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28c151990 0x7fd28c0b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:15.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.291+0000 7fd290c00640 1 --2- 172.21.15.19:0/3752593190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd270006330 0x7fd28c150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.291+0000 7fd290c00640 1 -- 172.21.15.19:0/3752593190 >> 172.21.15.19:0/3752593190 conn(0x7fd28c06b990 msgr2=0x7fd28c154a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.291+0000 7fd290c00640 1 -- 172.21.15.19:0/3752593190 shutdown_connections 2024-03-28T13:11:15.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.291+0000 7fd290c00640 1 -- 172.21.15.19:0/3752593190 wait complete. 2024-03-28T13:11:15.310 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-03-28T13:11:15.562 INFO:teuthology.orchestra.run.smithi188.stdout:{"message":"","return_code":0} 2024-03-28T13:11:15.563 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-28T13:11:15.563 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:11:15.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.791+0000 7ffba9600640 1 Processor -- start 2024-03-28T13:11:15.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.791+0000 7ffba9600640 1 -- start start 2024-03-28T13:11:15.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.792+0000 7ffba9600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffba415b310 0x7ffba40727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:15.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.793+0000 7ffba9600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40b4f90 0x7ffba415f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:15.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.793+0000 7ffba9600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba415b6f0 con 0x7ffba40d0780 2024-03-28T13:11:15.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.793+0000 7ffba9600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ffba4072e80 con 0x7ffba415b310 2024-03-28T13:11:15.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.793+0000 7ffba9600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffba4073160 con 0x7ffba40b4f90 2024-03-28T13:11:15.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.793+0000 7ffb9a000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40b4f90 0x7ffba415f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:15.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.793+0000 7ffba2a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffba415b310 0x7ffba40727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:15.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.794+0000 7ffb9a000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40b4f90 0x7ffba415f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:40364/0 (socket says 172.21.15.19:40364) 2024-03-28T13:11:15.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.794+0000 7ffb9a000640 1 -- 172.21.15.19:0/3870402461 learned_addr learned my addr 172.21.15.19:0/3870402461 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:15.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.794+0000 7ffba3400640 1 --1- 172.21.15.19:0/3870402461 >> v1:172.21.15.188:6789/0 conn(0x7ffba40d0780 0x7ffba40adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:55188/0 (socket says 172.21.15.19:55188) 2024-03-28T13:11:15.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.794+0000 7ffb9a000640 1 -- 172.21.15.19:0/3870402461 >> v2:172.21.15.188:3300/0 conn(0x7ffba415b310 msgr2=0x7ffba40727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:15.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.794+0000 7ffb9a000640 1 --2- 172.21.15.19:0/3870402461 >> v2:172.21.15.188:3300/0 conn(0x7ffba415b310 0x7ffba40727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:15.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.794+0000 7ffb9a000640 1 -- 172.21.15.19:0/3870402461 >> v1:172.21.15.188:6789/0 conn(0x7ffba40d0780 legacy=0x7ffba40adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:15.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.794+0000 7ffb9a000640 1 -- 172.21.15.19:0/3870402461 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba415efe0 con 0x7ffba40b4f90 2024-03-28T13:11:15.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.794+0000 7ffba2a00640 1 --2- 172.21.15.19:0/3870402461 >> v2:172.21.15.188:3300/0 conn(0x7ffba415b310 0x7ffba40727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:11:15.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.796+0000 7ffb9a000640 1 --2- 172.21.15.19:0/3870402461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40b4f90 0x7ffba415f7e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00bb10 tx=0x7ffb8c02e030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=527b99c4976b5b81 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.796+0000 7ffba2000640 1 -- 172.21.15.19:0/3870402461 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c03b070 con 0x7ffba40b4f90 2024-03-28T13:11:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.796+0000 7ffba2000640 1 -- 172.21.15.19:0/3870402461 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8c02ede0 con 0x7ffba40b4f90 2024-03-28T13:11:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.796+0000 7ffba2000640 1 -- 172.21.15.19:0/3870402461 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c036660 con 0x7ffba40b4f90 2024-03-28T13:11:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.797+0000 7ffba9600640 1 -- 172.21.15.19:0/3870402461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40b4f90 msgr2=0x7ffba415f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.797+0000 7ffba9600640 1 --2- 172.21.15.19:0/3870402461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40b4f90 0x7ffba415f7e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00bb10 tx=0x7ffb8c02e030 comp rx=0 tx=0).stop 2024-03-28T13:11:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.797+0000 7ffba9600640 1 -- 172.21.15.19:0/3870402461 shutdown_connections 2024-03-28T13:11:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.797+0000 7ffba9600640 1 --2- 172.21.15.19:0/3870402461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40b4f90 0x7ffba415f7e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.797+0000 7ffba9600640 1 --2- 172.21.15.19:0/3870402461 >> v2:172.21.15.188:3300/0 conn(0x7ffba415b310 0x7ffba40727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.797+0000 7ffba9600640 1 -- 172.21.15.19:0/3870402461 >> 172.21.15.19:0/3870402461 conn(0x7ffba406b990 msgr2=0x7ffba406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.797+0000 7ffba9600640 1 -- 172.21.15.19:0/3870402461 shutdown_connections 2024-03-28T13:11:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.797+0000 7ffba9600640 1 -- 172.21.15.19:0/3870402461 wait complete. 2024-03-28T13:11:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.799+0000 7ffba9600640 1 Processor -- start 2024-03-28T13:11:15.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.799+0000 7ffba9600640 1 -- start start 2024-03-28T13:11:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.800+0000 7ffba9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba40b4f90 0x7ffba4152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.800+0000 7ffba9600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40d0780 0x7ffba415dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.800+0000 7ffba9600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba415b310 0x7ffba4157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.800+0000 7ffba9600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffba4073440 con 0x7ffba40b4f90 2024-03-28T13:11:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.800+0000 7ffba9600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffba40b53f0 con 0x7ffba40d0780 2024-03-28T13:11:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.800+0000 7ffba9600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffba415fd20 con 0x7ffba415b310 2024-03-28T13:11:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.800+0000 7ffba2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba40b4f90 0x7ffba4152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.800+0000 7ffba2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba40b4f90 0x7ffba4152800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52072/0 (socket says 172.21.15.19:52072) 2024-03-28T13:11:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.800+0000 7ffba2a00640 1 -- 172.21.15.19:0/551250427 learned_addr learned my addr 172.21.15.19:0/551250427 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.800+0000 7ffba3400640 1 --2- 172.21.15.19:0/551250427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba415b310 0x7ffba4157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.800+0000 7ffb9a000640 1 --2- 172.21.15.19:0/551250427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40d0780 0x7ffba415dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:15.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.800+0000 7ffba2a00640 1 -- 172.21.15.19:0/551250427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba415b310 msgr2=0x7ffba4157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:15.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.801+0000 7ffba2a00640 1 --2- 172.21.15.19:0/551250427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba415b310 0x7ffba4157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:15.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.801+0000 7ffba2a00640 1 -- 172.21.15.19:0/551250427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40d0780 msgr2=0x7ffba415dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:15.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.801+0000 7ffba2a00640 1 --2- 172.21.15.19:0/551250427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40d0780 0x7ffba415dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:15.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.801+0000 7ffba2a00640 1 -- 172.21.15.19:0/551250427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba41581d0 con 0x7ffba40b4f90 2024-03-28T13:11:15.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.801+0000 7ffba2a00640 1 --2- 172.21.15.19:0/551250427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba40b4f90 0x7ffba4152800 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400ebe0 tx=0x7ffb9400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:15.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.801+0000 7ffb9b400640 1 -- 172.21.15.19:0/551250427 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94017070 con 0x7ffba40b4f90 2024-03-28T13:11:15.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.801+0000 7ffb9b400640 1 -- 172.21.15.19:0/551250427 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9400cc30 con 0x7ffba40b4f90 2024-03-28T13:11:15.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.801+0000 7ffb9b400640 1 -- 172.21.15.19:0/551250427 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94013470 con 0x7ffba40b4f90 2024-03-28T13:11:15.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.801+0000 7ffba9600640 1 -- 172.21.15.19:0/551250427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba415e230 con 0x7ffba40b4f90 2024-03-28T13:11:15.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.801+0000 7ffba9600640 1 -- 172.21.15.19:0/551250427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba4157ef0 con 0x7ffba40b4f90 2024-03-28T13:11:15.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.802+0000 7ffba9600640 1 -- 172.21.15.19:0/551250427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba40685c0 con 0x7ffba40b4f90 2024-03-28T13:11:15.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.804+0000 7ffb9b400640 1 -- 172.21.15.19:0/551250427 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ffb9401a070 con 0x7ffba40b4f90 2024-03-28T13:11:15.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.804+0000 7ffb9b400640 1 --2- 172.21.15.19:0/551250427 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8006c530 0x7ffb8006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:15.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.805+0000 7ffb9b400640 1 -- 172.21.15.19:0/551250427 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5709+0+0 (secure 0 0 0) 0x7ffb9408b6b0 con 0x7ffba40b4f90 2024-03-28T13:11:15.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.805+0000 7ffb9a000640 1 --2- 172.21.15.19:0/551250427 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8006c530 0x7ffb8006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:15.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.806+0000 7ffb9a000640 1 --2- 172.21.15.19:0/551250427 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8006c530 0x7ffb8006e9f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c038000 tx=0x7ffb8c032000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:15.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:15.809+0000 7ffb9b400640 1 -- 172.21.15.19:0/551250427 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ffba41581d0 con 0x7ffba40b4f90 2024-03-28T13:11:16.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:16.052+0000 7ffba9600640 1 -- 172.21.15.19:0/551250427 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffba415ec20 con 0x7ffba40b4f90 2024-03-28T13:11:16.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:16.053+0000 7ffb9b400640 1 -- 172.21.15.19:0/551250427 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7ffb94059cc0 con 0x7ffba40b4f90 2024-03-28T13:11:16.053 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:16.053 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2683971950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2683971950},{"type":"v1","addr":"172.21.15.188:6836","nonce":2683971950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/709274582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":709274582},{"type":"v1","addr":"172.21.15.19:6835","nonce":709274582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5040,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3420318181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3420318181},{"type":"v1","addr":"172.21.15.19:6833","nonce":3420318181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:10:56.216937+0000","modified":"2024-03-28T13:10:59.267258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5061":{"gid":5061,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6837/46284318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":46284318},{"type":"v1","addr":"172.21.15.188:6837","nonce":46284318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5061,"qdb_cluster":[5061]},"id":2}]} 2024-03-28T13:11:16.053 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-03-28T13:11:16.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:16.055+0000 7ffba9600640 1 -- 172.21.15.19:0/551250427 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8006c530 msgr2=0x7ffb8006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:16.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:16.055+0000 7ffba9600640 1 --2- 172.21.15.19:0/551250427 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8006c530 0x7ffb8006e9f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c038000 tx=0x7ffb8c032000 comp rx=0 tx=0).stop 2024-03-28T13:11:16.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:16.055+0000 7ffba9600640 1 -- 172.21.15.19:0/551250427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba40b4f90 msgr2=0x7ffba4152800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:16.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:16.055+0000 7ffba9600640 1 --2- 172.21.15.19:0/551250427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba40b4f90 0x7ffba4152800 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400ebe0 tx=0x7ffb9400c520 comp rx=0 tx=0).stop 2024-03-28T13:11:16.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:16.056+0000 7ffba9600640 1 -- 172.21.15.19:0/551250427 shutdown_connections 2024-03-28T13:11:16.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:16.056+0000 7ffba9600640 1 --2- 172.21.15.19:0/551250427 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8006c530 0x7ffb8006e9f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:16.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:16.056+0000 7ffba9600640 1 --2- 172.21.15.19:0/551250427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba415b310 0x7ffba4157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:16.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:16.056+0000 7ffba9600640 1 --2- 172.21.15.19:0/551250427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40d0780 0x7ffba415dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:16.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:16.056+0000 7ffba9600640 1 --2- 172.21.15.19:0/551250427 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba40b4f90 0x7ffba4152800 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:16.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:16.056+0000 7ffba9600640 1 -- 172.21.15.19:0/551250427 >> 172.21.15.19:0/551250427 conn(0x7ffba406b990 msgr2=0x7ffba40b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:16.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:16.056+0000 7ffba9600640 1 -- 172.21.15.19:0/551250427 shutdown_connections 2024-03-28T13:11:16.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:16.056+0000 7ffba9600640 1 -- 172.21.15.19:0/551250427 wait complete. 2024-03-28T13:11:16.073 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json tag path /parentdir mytag 2024-03-28T13:11:16.309 INFO:teuthology.orchestra.run.smithi188.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-03-28T13:11:16.309 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-03-28T13:11:26.310 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-03-28T13:11:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.436+0000 7f3ac58baf40 1 Processor -- start 2024-03-28T13:11:26.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.436+0000 7f3ac58baf40 1 -- start start 2024-03-28T13:11:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.437+0000 7f3ac58baf40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af1c5fa050 0x55af1c60f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.437+0000 7f3ac58baf40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55af1c5fa9d0 0x55af1c60f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.437+0000 7f3ac58baf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af1c5fa430 0x55af1c5f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.437+0000 7f3ac58baf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55af1c6a4560 con 0x55af1c5fa430 2024-03-28T13:11:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.437+0000 7f3ac58baf40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55af1c5712c0 con 0x55af1c5fa050 2024-03-28T13:11:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.437+0000 7f3ac58baf40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55af1c6a0d70 con 0x55af1c5fa9d0 2024-03-28T13:11:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.437+0000 7f3ac3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af1c5fa430 0x55af1c5f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.437+0000 7f3ac3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af1c5fa430 0x55af1c5f7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43340/0 (socket says 172.21.15.19:43340) 2024-03-28T13:11:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.437+0000 7f3ac3400640 1 -- 172.21.15.19:0/1995513434 learned_addr learned my addr 172.21.15.19:0/1995513434 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.437+0000 7f3ac2a00640 1 --2- 172.21.15.19:0/1995513434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af1c5fa050 0x55af1c60f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.437+0000 7f3aba000640 1 --2- 172.21.15.19:0/1995513434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55af1c5fa9d0 0x55af1c60f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.437+0000 7f3ac3400640 1 -- 172.21.15.19:0/1995513434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55af1c5fa9d0 msgr2=0x55af1c60f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.438+0000 7f3ac3400640 1 --2- 172.21.15.19:0/1995513434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55af1c5fa9d0 0x55af1c60f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.438+0000 7f3ac3400640 1 -- 172.21.15.19:0/1995513434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af1c5fa050 msgr2=0x55af1c60f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:26.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.438+0000 7f3ac3400640 1 --2- 172.21.15.19:0/1995513434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af1c5fa050 0x55af1c60f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.438+0000 7f3ac3400640 1 -- 172.21.15.19:0/1995513434 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af1c5f9300 con 0x55af1c5fa430 2024-03-28T13:11:26.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.438+0000 7f3ac3400640 1 --2- 172.21.15.19:0/1995513434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af1c5fa430 0x55af1c5f7d60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3abc057040 tx=0x7f3abc04f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b322c13e10f15adc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.438+0000 7f3ac1600640 1 -- 172.21.15.19:0/1995513434 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc001630 con 0x55af1c5fa430 2024-03-28T13:11:26.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.438+0000 7f3ac1600640 1 -- 172.21.15.19:0/1995513434 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3abc066070 con 0x55af1c5fa430 2024-03-28T13:11:26.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.438+0000 7f3ac1600640 1 -- 172.21.15.19:0/1995513434 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc061680 con 0x55af1c5fa430 2024-03-28T13:11:26.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.439+0000 7f3ac58baf40 1 -- 172.21.15.19:0/1995513434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af1c5fa430 msgr2=0x55af1c5f7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.439+0000 7f3ac58baf40 1 --2- 172.21.15.19:0/1995513434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af1c5fa430 0x55af1c5f7d60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3abc057040 tx=0x7f3abc04f070 comp rx=0 tx=0).stop 2024-03-28T13:11:26.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.439+0000 7f3ac58baf40 1 -- 172.21.15.19:0/1995513434 shutdown_connections 2024-03-28T13:11:26.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.439+0000 7f3ac58baf40 1 --2- 172.21.15.19:0/1995513434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af1c5fa430 0x55af1c5f7d60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.439+0000 7f3ac58baf40 1 --2- 172.21.15.19:0/1995513434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55af1c5fa9d0 0x55af1c60f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.439+0000 7f3ac58baf40 1 --2- 172.21.15.19:0/1995513434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af1c5fa050 0x55af1c60f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.439+0000 7f3ac58baf40 1 -- 172.21.15.19:0/1995513434 >> 172.21.15.19:0/1995513434 conn(0x55af1c5c98a0 msgr2=0x55af1c6111e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.439+0000 7f3ac58baf40 1 -- 172.21.15.19:0/1995513434 shutdown_connections 2024-03-28T13:11:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.439+0000 7f3ac58baf40 1 -- 172.21.15.19:0/1995513434 wait complete. 2024-03-28T13:11:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.440+0000 7f3ac58baf40 1 Processor -- start 2024-03-28T13:11:26.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.440+0000 7f3ac58baf40 1 -- start start 2024-03-28T13:11:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.441+0000 7f3ac58baf40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af1c5fa050 0x55af1c69f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.441+0000 7f3ac58baf40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55af1c5fa430 0x55af1c60f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.441+0000 7f3ac58baf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af1c60f7f0 0x55af1c697f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.441+0000 7f3ac58baf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55af1c6a2ea0 con 0x55af1c60f7f0 2024-03-28T13:11:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.441+0000 7f3ac58baf40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55af1c69d080 con 0x55af1c5fa050 2024-03-28T13:11:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.441+0000 7f3ac58baf40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55af1c6a2b30 con 0x55af1c5fa430 2024-03-28T13:11:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.441+0000 7f3ac3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af1c60f7f0 0x55af1c697f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.441+0000 7f3ac3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af1c60f7f0 0x55af1c697f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43342/0 (socket says 172.21.15.19:43342) 2024-03-28T13:11:26.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.441+0000 7f3ac3400640 1 -- 172.21.15.19:0/3668879700 learned_addr learned my addr 172.21.15.19:0/3668879700 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.441+0000 7f3ac2a00640 1 --2- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af1c5fa050 0x55af1c69f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.442+0000 7f3aba000640 1 --2- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55af1c5fa430 0x55af1c60f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.442+0000 7f3ac3400640 1 -- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55af1c5fa430 msgr2=0x55af1c60f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.442+0000 7f3ac3400640 1 --2- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55af1c5fa430 0x55af1c60f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.442+0000 7f3ac3400640 1 -- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af1c5fa050 msgr2=0x55af1c69f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.442+0000 7f3ac3400640 1 --2- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af1c5fa050 0x55af1c69f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.442+0000 7f3ac3400640 1 -- 172.21.15.19:0/3668879700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af1c72c070 con 0x55af1c60f7f0 2024-03-28T13:11:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.442+0000 7f3ac2a00640 1 --2- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55af1c5fa050 0x55af1c69f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:11:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.442+0000 7f3ac3400640 1 --2- 172.21.15.19:0/3668879700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af1c60f7f0 0x55af1c697f80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3abc063040 tx=0x7f3aa8004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.443+0000 7f3abbe00640 1 -- 172.21.15.19:0/3668879700 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc061680 con 0x55af1c60f7f0 2024-03-28T13:11:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.443+0000 7f3abbe00640 1 -- 172.21.15.19:0/3668879700 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3abc061820 con 0x55af1c60f7f0 2024-03-28T13:11:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.443+0000 7f3ac58baf40 1 -- 172.21.15.19:0/3668879700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55af1c72c350 con 0x55af1c60f7f0 2024-03-28T13:11:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.443+0000 7f3abbe00640 1 -- 172.21.15.19:0/3668879700 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3abc061bf0 con 0x55af1c60f7f0 2024-03-28T13:11:26.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.443+0000 7f3ac58baf40 1 -- 172.21.15.19:0/3668879700 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af1c72c910 con 0x55af1c60f7f0 2024-03-28T13:11:26.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.445+0000 7f3abbe00640 1 -- 172.21.15.19:0/3668879700 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3abc053030 con 0x55af1c60f7f0 2024-03-28T13:11:26.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.446+0000 7f3abbe00640 1 --2- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3aa006af40 0x7f3aa006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.446+0000 7f3abbe00640 1 -- 172.21.15.19:0/3668879700 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f3abc0d0d60 con 0x55af1c60f7f0 2024-03-28T13:11:26.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.446+0000 7f3ac58baf40 1 --2- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x55af1c72d110 0x55af1c736770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.446+0000 7f3ac58baf40 1 -- 172.21.15.19:0/3668879700 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x55af1c736cb0 con 0x55af1c72d110 2024-03-28T13:11:26.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.446+0000 7f3ac2a00640 1 --2- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3aa006af40 0x7f3aa006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.447+0000 7f3aba000640 1 --2- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x55af1c72d110 0x55af1c736770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.448+0000 7f3ac2a00640 1 --2- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3aa006af40 0x7f3aa006d400 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400a0b0 tx=0x7f3ab4007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.448+0000 7f3aba000640 1 --2- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x55af1c72d110 0x55af1c736770 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.449+0000 7f3aba000640 1 -- 172.21.15.19:0/3668879700 <== osd.7 v2:172.21.15.188:6824/861796513 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f3aac009940 con 0x55af1c72d110 2024-03-28T13:11:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.449+0000 7f3ac58baf40 1 -- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x55af1c72d110 msgr2=0x55af1c736770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.449+0000 7f3ac58baf40 1 --2- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x55af1c72d110 0x55af1c736770 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.449+0000 7f3ac58baf40 1 -- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3aa006af40 msgr2=0x7f3aa006d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.449+0000 7f3ac58baf40 1 --2- 172.21.15.19:0/3668879700 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3aa006af40 0x7f3aa006d400 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400a0b0 tx=0x7f3ab4007040 comp rx=0 tx=0).stop 2024-03-28T13:11:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.450+0000 7f3ac58baf40 1 -- 172.21.15.19:0/3668879700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af1c60f7f0 msgr2=0x55af1c697f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.450+0000 7f3ac58baf40 1 --2- 172.21.15.19:0/3668879700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af1c60f7f0 0x55af1c697f80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3abc063040 tx=0x7f3aa8004120 comp rx=0 tx=0).stop 2024-03-28T13:11:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.450+0000 7f3ac3400640 1 -- 172.21.15.19:0/3668879700 reap_dead start 2024-03-28T13:11:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.450+0000 7f3ac58baf40 1 -- 172.21.15.19:0/3668879700 shutdown_connections 2024-03-28T13:11:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.450+0000 7f3ac58baf40 1 -- 172.21.15.19:0/3668879700 >> 172.21.15.19:0/3668879700 conn(0x55af1c5c98a0 msgr2=0x55af1c69e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.450+0000 7f3ac58baf40 1 -- 172.21.15.19:0/3668879700 shutdown_connections 2024-03-28T13:11:26.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.450+0000 7f3ac58baf40 1 -- 172.21.15.19:0/3668879700 wait complete. 2024-03-28T13:11:26.456 INFO:teuthology.orchestra.run.smithi019.stdout:mytag 2024-03-28T13:11:26.456 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-03-28T13:11:26.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.549+0000 7fb540448f40 1 Processor -- start 2024-03-28T13:11:26.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.549+0000 7fb540448f40 1 -- start start 2024-03-28T13:11:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb540448f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5200028a0 0x5639b6590490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb540448f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639b6590a80 0x5639b6593050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb540448f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639b6591b60 0x5639b6591fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb540448f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5639b646a2c0 con 0x7fb5200028a0 2024-03-28T13:11:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb540448f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5639b646e310 con 0x5639b6591b60 2024-03-28T13:11:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb540448f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5639b650fb20 con 0x5639b6590a80 2024-03-28T13:11:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb53ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5200028a0 0x5639b6590490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb53ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5200028a0 0x5639b6590490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43352/0 (socket says 172.21.15.19:43352) 2024-03-28T13:11:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb53ce00640 1 -- 172.21.15.19:0/2307319459 learned_addr learned my addr 172.21.15.19:0/2307319459 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb53d800640 1 --2- 172.21.15.19:0/2307319459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639b6591b60 0x5639b6591fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb53c400640 1 --2- 172.21.15.19:0/2307319459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639b6590a80 0x5639b6593050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb53ce00640 1 -- 172.21.15.19:0/2307319459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639b6590a80 msgr2=0x5639b6593050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb53ce00640 1 --2- 172.21.15.19:0/2307319459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639b6590a80 0x5639b6593050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb53ce00640 1 -- 172.21.15.19:0/2307319459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639b6591b60 msgr2=0x5639b6591fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb53ce00640 1 --2- 172.21.15.19:0/2307319459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639b6591b60 0x5639b6591fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb53ce00640 1 -- 172.21.15.19:0/2307319459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639b6591350 con 0x7fb5200028a0 2024-03-28T13:11:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.550+0000 7fb53d800640 1 --2- 172.21.15.19:0/2307319459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639b6591b60 0x5639b6591fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.551+0000 7fb53ce00640 1 --2- 172.21.15.19:0/2307319459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5200028a0 0x5639b6590490 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0026a0 tx=0x7fb52c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3409e8661cca1829 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.551+0000 7fb53b000640 1 -- 172.21.15.19:0/2307319459 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c017020 con 0x7fb5200028a0 2024-03-28T13:11:26.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.551+0000 7fb53b000640 1 -- 172.21.15.19:0/2307319459 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb52c00e710 con 0x7fb5200028a0 2024-03-28T13:11:26.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.551+0000 7fb53b000640 1 -- 172.21.15.19:0/2307319459 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c00eae0 con 0x7fb5200028a0 2024-03-28T13:11:26.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.551+0000 7fb540448f40 1 -- 172.21.15.19:0/2307319459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5200028a0 msgr2=0x5639b6590490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.551+0000 7fb540448f40 1 --2- 172.21.15.19:0/2307319459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5200028a0 0x5639b6590490 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb52c0026a0 tx=0x7fb52c005d80 comp rx=0 tx=0).stop 2024-03-28T13:11:26.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.552+0000 7fb540448f40 1 -- 172.21.15.19:0/2307319459 shutdown_connections 2024-03-28T13:11:26.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.552+0000 7fb540448f40 1 --2- 172.21.15.19:0/2307319459 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639b6591b60 0x5639b6591fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.552+0000 7fb540448f40 1 --2- 172.21.15.19:0/2307319459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639b6590a80 0x5639b6593050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.552+0000 7fb540448f40 1 --2- 172.21.15.19:0/2307319459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5200028a0 0x5639b6590490 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.552+0000 7fb540448f40 1 -- 172.21.15.19:0/2307319459 >> 172.21.15.19:0/2307319459 conn(0x5639b6511360 msgr2=0x5639b64f23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:26.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.552+0000 7fb540448f40 1 -- 172.21.15.19:0/2307319459 shutdown_connections 2024-03-28T13:11:26.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.552+0000 7fb540448f40 1 -- 172.21.15.19:0/2307319459 wait complete. 2024-03-28T13:11:26.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.552+0000 7fb540448f40 1 Processor -- start 2024-03-28T13:11:26.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.553+0000 7fb540448f40 1 -- start start 2024-03-28T13:11:26.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.553+0000 7fb540448f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b6590a80 0x5639b659c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.553+0000 7fb540448f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639b6591b60 0x5639b658afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.554+0000 7fb540448f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5200028a0 0x5639b650d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.554+0000 7fb540448f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5639b65949e0 con 0x5639b6590a80 2024-03-28T13:11:26.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.554+0000 7fb540448f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5639b64f5040 con 0x5639b6591b60 2024-03-28T13:11:26.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.554+0000 7fb540448f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5639b6593590 con 0x7fb5200028a0 2024-03-28T13:11:26.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.554+0000 7fb53ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b6590a80 0x5639b659c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.554+0000 7fb53ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b6590a80 0x5639b659c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43362/0 (socket says 172.21.15.19:43362) 2024-03-28T13:11:26.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.554+0000 7fb53ce00640 1 -- 172.21.15.19:0/3534951904 learned_addr learned my addr 172.21.15.19:0/3534951904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:26.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.554+0000 7fb53c400640 1 --2- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639b6591b60 0x5639b658afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.554+0000 7fb53ce00640 1 -- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5200028a0 msgr2=0x5639b650d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.554+0000 7fb53ce00640 1 --2- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5200028a0 0x5639b650d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.554+0000 7fb53ce00640 1 -- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639b6591b60 msgr2=0x5639b658afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.554+0000 7fb53ce00640 1 --2- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639b6591b60 0x5639b658afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.554+0000 7fb53ce00640 1 -- 172.21.15.19:0/3534951904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639b658ecd0 con 0x5639b6590a80 2024-03-28T13:11:26.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.554+0000 7fb53c400640 1 --2- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639b6591b60 0x5639b658afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:26.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.555+0000 7fb53ce00640 1 --2- 172.21.15.19:0/3534951904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b6590a80 0x5639b659c410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb52c016040 tx=0x7fb52c003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.555+0000 7fb539c00640 1 -- 172.21.15.19:0/3534951904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c017040 con 0x5639b6590a80 2024-03-28T13:11:26.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.555+0000 7fb539c00640 1 -- 172.21.15.19:0/3534951904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb52c013c60 con 0x5639b6590a80 2024-03-28T13:11:26.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.555+0000 7fb539c00640 1 -- 172.21.15.19:0/3534951904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52c026670 con 0x5639b6590a80 2024-03-28T13:11:26.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.555+0000 7fb540448f40 1 -- 172.21.15.19:0/3534951904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5639b658efb0 con 0x5639b6590a80 2024-03-28T13:11:26.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.555+0000 7fb540448f40 1 -- 172.21.15.19:0/3534951904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639b658f570 con 0x5639b6590a80 2024-03-28T13:11:26.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.557+0000 7fb539c00640 1 -- 172.21.15.19:0/3534951904 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb52c01a020 con 0x5639b6590a80 2024-03-28T13:11:26.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.558+0000 7fb539c00640 1 --2- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb51806af40 0x7fb51806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.558+0000 7fb539c00640 1 -- 172.21.15.19:0/3534951904 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7fb52c08c9b0 con 0x5639b6590a80 2024-03-28T13:11:26.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.559+0000 7fb540448f40 1 --2- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5639b658b8f0 0x5639b658bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.559+0000 7fb540448f40 1 -- 172.21.15.19:0/3534951904 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:1 4.7 4:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x5639b658c290 con 0x5639b658b8f0 2024-03-28T13:11:26.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.559+0000 7fb53c400640 1 --2- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb51806af40 0x7fb51806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.559+0000 7fb53d800640 1 --2- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5639b658b8f0 0x5639b658bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.561+0000 7fb53d800640 1 --2- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5639b658b8f0 0x5639b658bd50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.561+0000 7fb53c400640 1 --2- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb51806af40 0x7fb51806d400 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb524052000 tx=0x7fb524054040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.562+0000 7fb53d800640 1 -- 172.21.15.19:0/3534951904 <== osd.6 v2:172.21.15.188:6816/3597358155 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fb53400d750 con 0x5639b658b8f0 2024-03-28T13:11:26.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.562+0000 7fb540448f40 1 -- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5639b658b8f0 msgr2=0x5639b658bd50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.562+0000 7fb540448f40 1 --2- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5639b658b8f0 0x5639b658bd50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.562+0000 7fb540448f40 1 -- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb51806af40 msgr2=0x7fb51806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.562+0000 7fb540448f40 1 --2- 172.21.15.19:0/3534951904 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb51806af40 0x7fb51806d400 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb524052000 tx=0x7fb524054040 comp rx=0 tx=0).stop 2024-03-28T13:11:26.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.562+0000 7fb540448f40 1 -- 172.21.15.19:0/3534951904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b6590a80 msgr2=0x5639b659c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.562+0000 7fb540448f40 1 --2- 172.21.15.19:0/3534951904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639b6590a80 0x5639b659c410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb52c016040 tx=0x7fb52c003330 comp rx=0 tx=0).stop 2024-03-28T13:11:26.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.562+0000 7fb53d800640 1 -- 172.21.15.19:0/3534951904 reap_dead start 2024-03-28T13:11:26.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.562+0000 7fb540448f40 1 -- 172.21.15.19:0/3534951904 shutdown_connections 2024-03-28T13:11:26.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.562+0000 7fb540448f40 1 -- 172.21.15.19:0/3534951904 >> 172.21.15.19:0/3534951904 conn(0x5639b6511360 msgr2=0x5639b64f3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:26.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.562+0000 7fb540448f40 1 -- 172.21.15.19:0/3534951904 shutdown_connections 2024-03-28T13:11:26.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.562+0000 7fb540448f40 1 -- 172.21.15.19:0/3534951904 wait complete. 2024-03-28T13:11:26.567 INFO:teuthology.orchestra.run.smithi019.stdout:mytag 2024-03-28T13:11:26.568 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-03-28T13:11:26.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.661+0000 7f3dcac92f40 1 Processor -- start 2024-03-28T13:11:26.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.661+0000 7f3dcac92f40 1 -- start start 2024-03-28T13:11:26.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.662+0000 7f3dcac92f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1cf6584c0 0x55c1cf6e4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.662+0000 7f3dcac92f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1cf659b60 0x55c1cf6e67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.662+0000 7f3dcac92f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1cf6e5ad0 0x55c1cf6e75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.662+0000 7f3dcac92f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c1cf658cb0 con 0x55c1cf659b60 2024-03-28T13:11:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.662+0000 7f3dcac92f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c1cf5b72c0 con 0x55c1cf6e5ad0 2024-03-28T13:11:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.662+0000 7f3dcac92f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55c1cf658f50 con 0x55c1cf6584c0 2024-03-28T13:11:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.662+0000 7f3dc7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1cf659b60 0x55c1cf6e67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.662+0000 7f3dc7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1cf659b60 0x55c1cf6e67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43386/0 (socket says 172.21.15.19:43386) 2024-03-28T13:11:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.662+0000 7f3dc7400640 1 -- 172.21.15.19:0/881188923 learned_addr learned my addr 172.21.15.19:0/881188923 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.663+0000 7f3dc7e00640 1 --2- 172.21.15.19:0/881188923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1cf6584c0 0x55c1cf6e4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.663+0000 7f3dc8800640 1 --2- 172.21.15.19:0/881188923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1cf6e5ad0 0x55c1cf6e75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.663+0000 7f3dc7400640 1 -- 172.21.15.19:0/881188923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1cf6584c0 msgr2=0x55c1cf6e4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.663+0000 7f3dc7400640 1 --2- 172.21.15.19:0/881188923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1cf6584c0 0x55c1cf6e4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.663+0000 7f3dc7400640 1 -- 172.21.15.19:0/881188923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1cf6e5ad0 msgr2=0x55c1cf6e75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.663+0000 7f3dc7400640 1 --2- 172.21.15.19:0/881188923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1cf6e5ad0 0x55c1cf6e75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.663+0000 7f3dc7400640 1 -- 172.21.15.19:0/881188923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1cf6f0020 con 0x55c1cf659b60 2024-03-28T13:11:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.663+0000 7f3dc7400640 1 --2- 172.21.15.19:0/881188923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1cf659b60 0x55c1cf6e67c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3db400a440 tx=0x7f3db404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478db3bfb1f243d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.663+0000 7f3dc6000640 1 -- 172.21.15.19:0/881188923 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db4057040 con 0x55c1cf659b60 2024-03-28T13:11:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.664+0000 7f3dc6000640 1 -- 172.21.15.19:0/881188923 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3db4063070 con 0x55c1cf659b60 2024-03-28T13:11:26.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.664+0000 7f3dc6000640 1 -- 172.21.15.19:0/881188923 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db4052da0 con 0x55c1cf659b60 2024-03-28T13:11:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.664+0000 7f3dcac92f40 1 -- 172.21.15.19:0/881188923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1cf659b60 msgr2=0x55c1cf6e67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.664+0000 7f3dcac92f40 1 --2- 172.21.15.19:0/881188923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1cf659b60 0x55c1cf6e67c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3db400a440 tx=0x7f3db404c000 comp rx=0 tx=0).stop 2024-03-28T13:11:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.664+0000 7f3dcac92f40 1 -- 172.21.15.19:0/881188923 shutdown_connections 2024-03-28T13:11:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.664+0000 7f3dcac92f40 1 --2- 172.21.15.19:0/881188923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1cf6e5ad0 0x55c1cf6e75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.664+0000 7f3dcac92f40 1 --2- 172.21.15.19:0/881188923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1cf659b60 0x55c1cf6e67c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.664+0000 7f3dcac92f40 1 --2- 172.21.15.19:0/881188923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1cf6584c0 0x55c1cf6e4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.664+0000 7f3dcac92f40 1 -- 172.21.15.19:0/881188923 >> 172.21.15.19:0/881188923 conn(0x55c1cf63bcf0 msgr2=0x55c1cf6409b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.664+0000 7f3dcac92f40 1 -- 172.21.15.19:0/881188923 shutdown_connections 2024-03-28T13:11:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.664+0000 7f3dcac92f40 1 -- 172.21.15.19:0/881188923 wait complete. 2024-03-28T13:11:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.665+0000 7f3dcac92f40 1 Processor -- start 2024-03-28T13:11:26.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.665+0000 7f3dcac92f40 1 -- start start 2024-03-28T13:11:26.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.666+0000 7f3dcac92f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1cf6584c0 0x55c1cf6e0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.666+0000 7f3dcac92f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1cf659b60 0x55c1cf6dadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.666+0000 7f3dcac92f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1cf6e5ad0 0x55c1cf640780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.666+0000 7f3dcac92f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c1cf659720 con 0x55c1cf6e5ad0 2024-03-28T13:11:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.666+0000 7f3dcac92f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c1cf6e42d0 con 0x55c1cf6584c0 2024-03-28T13:11:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.666+0000 7f3dcac92f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55c1cf6e45b0 con 0x55c1cf659b60 2024-03-28T13:11:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.666+0000 7f3dc8800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1cf6e5ad0 0x55c1cf640780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.667+0000 7f3dc8800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1cf6e5ad0 0x55c1cf640780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43392/0 (socket says 172.21.15.19:43392) 2024-03-28T13:11:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.667+0000 7f3dc8800640 1 -- 172.21.15.19:0/1033108255 learned_addr learned my addr 172.21.15.19:0/1033108255 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.667+0000 7f3dc7e00640 1 --2- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1cf6584c0 0x55c1cf6e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.667+0000 7f3dc7400640 1 --2- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1cf659b60 0x55c1cf6dadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.667+0000 7f3dc8800640 1 -- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1cf659b60 msgr2=0x55c1cf6dadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.667+0000 7f3dc8800640 1 --2- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c1cf659b60 0x55c1cf6dadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.667+0000 7f3dc8800640 1 -- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1cf6584c0 msgr2=0x55c1cf6e0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.667+0000 7f3dc8800640 1 --2- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c1cf6584c0 0x55c1cf6e0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.667+0000 7f3dc8800640 1 -- 172.21.15.19:0/1033108255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1cf6dcdc0 con 0x55c1cf6e5ad0 2024-03-28T13:11:26.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.667+0000 7f3dc8800640 1 --2- 172.21.15.19:0/1033108255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1cf6e5ad0 0x55c1cf640780 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0007e00 tx=0x7f3dc0006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.668+0000 7f3dc4c00640 1 -- 172.21.15.19:0/1033108255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0017070 con 0x55c1cf6e5ad0 2024-03-28T13:11:26.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.668+0000 7f3dc4c00640 1 -- 172.21.15.19:0/1033108255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc0010710 con 0x55c1cf6e5ad0 2024-03-28T13:11:26.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.668+0000 7f3dc4c00640 1 -- 172.21.15.19:0/1033108255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0006e60 con 0x55c1cf6e5ad0 2024-03-28T13:11:26.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.668+0000 7f3dcac92f40 1 -- 172.21.15.19:0/1033108255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c1cf6e37d0 con 0x55c1cf6e5ad0 2024-03-28T13:11:26.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.668+0000 7f3dcac92f40 1 -- 172.21.15.19:0/1033108255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1cf6dd030 con 0x55c1cf6e5ad0 2024-03-28T13:11:26.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.670+0000 7f3dc4c00640 1 -- 172.21.15.19:0/1033108255 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3dc001a070 con 0x55c1cf6e5ad0 2024-03-28T13:11:26.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.671+0000 7f3dc4c00640 1 --2- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3da806af10 0x7f3da806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.671+0000 7f3dc4c00640 1 -- 172.21.15.19:0/1033108255 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f3dc0089a40 con 0x55c1cf6e5ad0 2024-03-28T13:11:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.672+0000 7f3dcac92f40 1 --2- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x55c1cf6e1d30 0x55c1cf6e2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.672+0000 7f3dcac92f40 1 -- 172.21.15.19:0/1033108255 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:1 5.36 5:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x55c1cf6e2650 con 0x55c1cf6e1d30 2024-03-28T13:11:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.672+0000 7f3dc7e00640 1 --2- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3da806af10 0x7f3da806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.672+0000 7f3dc7400640 1 --2- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x55c1cf6e1d30 0x55c1cf6e2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.673+0000 7f3dc7e00640 1 --2- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3da806af10 0x7f3da806d3d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3db8006530 tx=0x7f3db8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.673+0000 7f3dc7400640 1 --2- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x55c1cf6e1d30 0x55c1cf6e2110 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.674+0000 7f3dc7400640 1 -- 172.21.15.19:0/1033108255 <== osd.4 v2:172.21.15.188:6808/1845695132 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f3db405a040 con 0x55c1cf6e1d30 2024-03-28T13:11:26.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.675+0000 7f3dcac92f40 1 -- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x55c1cf6e1d30 msgr2=0x55c1cf6e2110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.675+0000 7f3dcac92f40 1 --2- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x55c1cf6e1d30 0x55c1cf6e2110 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.675+0000 7f3dcac92f40 1 -- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3da806af10 msgr2=0x7f3da806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.675+0000 7f3dcac92f40 1 --2- 172.21.15.19:0/1033108255 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3da806af10 0x7f3da806d3d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3db8006530 tx=0x7f3db8008040 comp rx=0 tx=0).stop 2024-03-28T13:11:26.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.675+0000 7f3dcac92f40 1 -- 172.21.15.19:0/1033108255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1cf6e5ad0 msgr2=0x55c1cf640780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.675+0000 7f3dcac92f40 1 --2- 172.21.15.19:0/1033108255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c1cf6e5ad0 0x55c1cf640780 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0007e00 tx=0x7f3dc0006750 comp rx=0 tx=0).stop 2024-03-28T13:11:26.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.675+0000 7f3dc8800640 1 -- 172.21.15.19:0/1033108255 reap_dead start 2024-03-28T13:11:26.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.675+0000 7f3dcac92f40 1 -- 172.21.15.19:0/1033108255 shutdown_connections 2024-03-28T13:11:26.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.675+0000 7f3dcac92f40 1 -- 172.21.15.19:0/1033108255 >> 172.21.15.19:0/1033108255 conn(0x55c1cf63bcf0 msgr2=0x55c1cf6118b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:26.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.675+0000 7f3dcac92f40 1 -- 172.21.15.19:0/1033108255 shutdown_connections 2024-03-28T13:11:26.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.675+0000 7f3dcac92f40 1 -- 172.21.15.19:0/1033108255 wait complete. 2024-03-28T13:11:26.680 INFO:teuthology.orchestra.run.smithi019.stdout:mytag 2024-03-28T13:11:26.680 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-03-28T13:11:26.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.781+0000 7f67441d6f40 1 Processor -- start 2024-03-28T13:11:26.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.781+0000 7f67441d6f40 1 -- start start 2024-03-28T13:11:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.782+0000 7f67441d6f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56489b4ee510 0x56489b57fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.782+0000 7f67441d6f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56489b57e590 0x56489b571200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.782+0000 7f67441d6f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56489b57bc70 0x56489b4d6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.782+0000 7f67441d6f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56489b582560 con 0x56489b57e590 2024-03-28T13:11:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.782+0000 7f67441d6f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56489b44f2c0 con 0x56489b4ee510 2024-03-28T13:11:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.782+0000 7f67441d6f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x56489b581820 con 0x56489b57bc70 2024-03-28T13:11:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.783+0000 7f6740a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56489b57e590 0x56489b571200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.783+0000 7f6740a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56489b57e590 0x56489b571200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43412/0 (socket says 172.21.15.19:43412) 2024-03-28T13:11:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.783+0000 7f6740a00640 1 -- 172.21.15.19:0/1704161907 learned_addr learned my addr 172.21.15.19:0/1704161907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.783+0000 7f6741e00640 1 --2- 172.21.15.19:0/1704161907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56489b57bc70 0x56489b4d6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.783+0000 7f6741400640 1 --2- 172.21.15.19:0/1704161907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56489b4ee510 0x56489b57fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.783+0000 7f6740a00640 1 -- 172.21.15.19:0/1704161907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56489b57bc70 msgr2=0x56489b4d6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.783+0000 7f6740a00640 1 --2- 172.21.15.19:0/1704161907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56489b57bc70 0x56489b4d6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.783+0000 7f6740a00640 1 -- 172.21.15.19:0/1704161907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56489b4ee510 msgr2=0x56489b57fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.783+0000 7f6740a00640 1 --2- 172.21.15.19:0/1704161907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56489b4ee510 0x56489b57fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.783+0000 7f6740a00640 1 -- 172.21.15.19:0/1704161907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56489b5725a0 con 0x56489b57e590 2024-03-28T13:11:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.784+0000 7f6740a00640 1 --2- 172.21.15.19:0/1704161907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56489b57e590 0x56489b571200 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6730055040 tx=0x7f673004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7d59028ac9584e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.784+0000 7f673b400640 1 -- 172.21.15.19:0/1704161907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6730057040 con 0x56489b57e590 2024-03-28T13:11:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.784+0000 7f673b400640 1 -- 172.21.15.19:0/1704161907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6730062070 con 0x56489b57e590 2024-03-28T13:11:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.784+0000 7f673b400640 1 -- 172.21.15.19:0/1704161907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673005c8c0 con 0x56489b57e590 2024-03-28T13:11:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.784+0000 7f67441d6f40 1 -- 172.21.15.19:0/1704161907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56489b57e590 msgr2=0x56489b571200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.784+0000 7f67441d6f40 1 --2- 172.21.15.19:0/1704161907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56489b57e590 0x56489b571200 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6730055040 tx=0x7f673004c000 comp rx=0 tx=0).stop 2024-03-28T13:11:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.785+0000 7f67441d6f40 1 -- 172.21.15.19:0/1704161907 shutdown_connections 2024-03-28T13:11:26.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.785+0000 7f67441d6f40 1 --2- 172.21.15.19:0/1704161907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56489b57bc70 0x56489b4d6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.785+0000 7f67441d6f40 1 --2- 172.21.15.19:0/1704161907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56489b57e590 0x56489b571200 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.785+0000 7f67441d6f40 1 --2- 172.21.15.19:0/1704161907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56489b4ee510 0x56489b57fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.785+0000 7f67441d6f40 1 -- 172.21.15.19:0/1704161907 >> 172.21.15.19:0/1704161907 conn(0x56489b4a78a0 msgr2=0x56489b4a6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:26.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.785+0000 7f67441d6f40 1 -- 172.21.15.19:0/1704161907 shutdown_connections 2024-03-28T13:11:26.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.785+0000 7f67441d6f40 1 -- 172.21.15.19:0/1704161907 wait complete. 2024-03-28T13:11:26.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.785+0000 7f67441d6f40 1 Processor -- start 2024-03-28T13:11:26.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.786+0000 7f67441d6f40 1 -- start start 2024-03-28T13:11:26.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f67441d6f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56489b4ee510 0x56489b56fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f67441d6f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56489b57e590 0x56489b57c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f67441d6f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56489b4a74c0 0x56489b57c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f67441d6f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56489b572900 con 0x56489b4ee510 2024-03-28T13:11:26.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f67441d6f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56489b5821a0 con 0x56489b57e590 2024-03-28T13:11:26.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f67441d6f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x56489b572b50 con 0x56489b4a74c0 2024-03-28T13:11:26.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f6741400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56489b4ee510 0x56489b56fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f6741400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56489b4ee510 0x56489b56fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43420/0 (socket says 172.21.15.19:43420) 2024-03-28T13:11:26.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f6741400640 1 -- 172.21.15.19:0/499854188 learned_addr learned my addr 172.21.15.19:0/499854188 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:26.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f6740a00640 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56489b57e590 0x56489b57c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f6741e00640 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56489b4a74c0 0x56489b57c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f6741400640 1 -- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56489b4a74c0 msgr2=0x56489b57c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f6741400640 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56489b4a74c0 0x56489b57c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f6741400640 1 -- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56489b57e590 msgr2=0x56489b57c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f6741400640 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56489b57e590 0x56489b57c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f6741400640 1 -- 172.21.15.19:0/499854188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56489b4ab880 con 0x56489b4ee510 2024-03-28T13:11:26.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f6741e00640 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56489b4a74c0 0x56489b57c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:26.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.787+0000 7f6740a00640 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56489b57e590 0x56489b57c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:26.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.788+0000 7f6741400640 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56489b4ee510 0x56489b56fb30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f673400c160 tx=0x7f673400c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.788+0000 7f673a000640 1 -- 172.21.15.19:0/499854188 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734017020 con 0x56489b4ee510 2024-03-28T13:11:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.788+0000 7f673a000640 1 -- 172.21.15.19:0/499854188 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6734013670 con 0x56489b4ee510 2024-03-28T13:11:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.788+0000 7f673a000640 1 -- 172.21.15.19:0/499854188 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734013a60 con 0x56489b4ee510 2024-03-28T13:11:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.788+0000 7f67441d6f40 1 -- 172.21.15.19:0/499854188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56489b4ab2c0 con 0x56489b4ee510 2024-03-28T13:11:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.788+0000 7f67441d6f40 1 -- 172.21.15.19:0/499854188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56489b4a9a60 con 0x56489b4ee510 2024-03-28T13:11:26.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.790+0000 7f673a000640 1 -- 172.21.15.19:0/499854188 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f673401a020 con 0x56489b4ee510 2024-03-28T13:11:26.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.791+0000 7f673a000640 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f672406af10 0x7f672406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.791+0000 7f673a000640 1 -- 172.21.15.19:0/499854188 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f673408a560 con 0x56489b4ee510 2024-03-28T13:11:26.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.792+0000 7f67441d6f40 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x56489b571f00 0x56489b619220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.792+0000 7f67441d6f40 1 -- 172.21.15.19:0/499854188 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:1 5.19 5:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x56489b57e050 con 0x56489b571f00 2024-03-28T13:11:26.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.792+0000 7f6740a00640 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f672406af10 0x7f672406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.792+0000 7f6741e00640 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x56489b571f00 0x56489b619220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.793+0000 7f6740a00640 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f672406af10 0x7f672406d3d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f673000a4f0 tx=0x7f673004f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.793+0000 7f6741e00640 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x56489b571f00 0x56489b619220 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.794+0000 7f6741e00640 1 -- 172.21.15.19:0/499854188 <== osd.4 v2:172.21.15.188:6808/1845695132 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f673c00e470 con 0x56489b571f00 2024-03-28T13:11:26.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.794+0000 7f67441d6f40 1 -- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x56489b571f00 msgr2=0x56489b619220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.794+0000 7f67441d6f40 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x56489b571f00 0x56489b619220 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.794+0000 7f67441d6f40 1 -- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f672406af10 msgr2=0x7f672406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.794+0000 7f67441d6f40 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f672406af10 0x7f672406d3d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f673000a4f0 tx=0x7f673004f000 comp rx=0 tx=0).stop 2024-03-28T13:11:26.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.794+0000 7f67441d6f40 1 -- 172.21.15.19:0/499854188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56489b4ee510 msgr2=0x56489b56fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.794+0000 7f67441d6f40 1 --2- 172.21.15.19:0/499854188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56489b4ee510 0x56489b56fb30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f673400c160 tx=0x7f673400c620 comp rx=0 tx=0).stop 2024-03-28T13:11:26.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.794+0000 7f6741e00640 1 -- 172.21.15.19:0/499854188 reap_dead start 2024-03-28T13:11:26.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.794+0000 7f67441d6f40 1 -- 172.21.15.19:0/499854188 shutdown_connections 2024-03-28T13:11:26.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.794+0000 7f67441d6f40 1 -- 172.21.15.19:0/499854188 >> 172.21.15.19:0/499854188 conn(0x56489b4a78a0 msgr2=0x56489b4a6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:26.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.794+0000 7f67441d6f40 1 -- 172.21.15.19:0/499854188 shutdown_connections 2024-03-28T13:11:26.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.795+0000 7f67441d6f40 1 -- 172.21.15.19:0/499854188 wait complete. 2024-03-28T13:11:26.800 INFO:teuthology.orchestra.run.smithi019.stdout:mytag 2024-03-28T13:11:26.800 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-03-28T13:11:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.902+0000 7f28b000cf40 1 Processor -- start 2024-03-28T13:11:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.902+0000 7f28b000cf40 1 -- start start 2024-03-28T13:11:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28b000cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d34039c20 0x559d340ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28b000cf40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559d340c1240 0x559d340cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28b000cf40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d340cf9b0 0x559d3401d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28b000cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559d33f962c0 con 0x559d34039c20 2024-03-28T13:11:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28b000cf40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559d33f9a310 con 0x559d340cf9b0 2024-03-28T13:11:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28b000cf40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x559d33ff0390 con 0x559d340c1240 2024-03-28T13:11:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28ac800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d34039c20 0x559d340ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28ac800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d34039c20 0x559d340ca6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43426/0 (socket says 172.21.15.19:43426) 2024-03-28T13:11:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28ac800640 1 -- 172.21.15.19:0/940769666 learned_addr learned my addr 172.21.15.19:0/940769666 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28abe00640 1 --2- 172.21.15.19:0/940769666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559d340c1240 0x559d340cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28ad200640 1 --2- 172.21.15.19:0/940769666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d340cf9b0 0x559d3401d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28ac800640 1 -- 172.21.15.19:0/940769666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559d340c1240 msgr2=0x559d340cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28ac800640 1 --2- 172.21.15.19:0/940769666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559d340c1240 0x559d340cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28ac800640 1 -- 172.21.15.19:0/940769666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d340cf9b0 msgr2=0x559d3401d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28ac800640 1 --2- 172.21.15.19:0/940769666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d340cf9b0 0x559d3401d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.903+0000 7f28ac800640 1 -- 172.21.15.19:0/940769666 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d3401ce50 con 0x559d34039c20 2024-03-28T13:11:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.904+0000 7f28ac800640 1 --2- 172.21.15.19:0/940769666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d34039c20 0x559d340ca6f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f289c00b3c0 tx=0x7f289c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc9cf06b546856d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.904+0000 7f28aaa00640 1 -- 172.21.15.19:0/940769666 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c012020 con 0x559d34039c20 2024-03-28T13:11:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.904+0000 7f28aaa00640 1 -- 172.21.15.19:0/940769666 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f289c00e670 con 0x559d34039c20 2024-03-28T13:11:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.904+0000 7f28aaa00640 1 -- 172.21.15.19:0/940769666 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c00eaa0 con 0x559d34039c20 2024-03-28T13:11:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.905+0000 7f28b000cf40 1 -- 172.21.15.19:0/940769666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d34039c20 msgr2=0x559d340ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.905+0000 7f28b000cf40 1 --2- 172.21.15.19:0/940769666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d34039c20 0x559d340ca6f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f289c00b3c0 tx=0x7f289c00b7c0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.905+0000 7f28b000cf40 1 -- 172.21.15.19:0/940769666 shutdown_connections 2024-03-28T13:11:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.905+0000 7f28b000cf40 1 --2- 172.21.15.19:0/940769666 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d340cf9b0 0x559d3401d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.905+0000 7f28b000cf40 1 --2- 172.21.15.19:0/940769666 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559d340c1240 0x559d340cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.905+0000 7f28b000cf40 1 --2- 172.21.15.19:0/940769666 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d34039c20 0x559d340ca6f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.905+0000 7f28b000cf40 1 -- 172.21.15.19:0/940769666 >> 172.21.15.19:0/940769666 conn(0x559d3403d630 msgr2=0x559d33fd9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.905+0000 7f28b000cf40 1 -- 172.21.15.19:0/940769666 shutdown_connections 2024-03-28T13:11:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.905+0000 7f28b000cf40 1 -- 172.21.15.19:0/940769666 wait complete. 2024-03-28T13:11:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.906+0000 7f28b000cf40 1 Processor -- start 2024-03-28T13:11:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.906+0000 7f28b000cf40 1 -- start start 2024-03-28T13:11:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28b000cf40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559d34039c20 0x559d340b75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28b000cf40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d340c1240 0x559d340c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28b000cf40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d340cf9b0 0x559d340c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28b000cf40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559d34021ff0 con 0x559d340cf9b0 2024-03-28T13:11:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28b000cf40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559d3401d0f0 con 0x559d340c1240 2024-03-28T13:11:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28b000cf40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x559d3401e7c0 con 0x559d34039c20 2024-03-28T13:11:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28ad200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d340cf9b0 0x559d340c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28ad200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d340cf9b0 0x559d340c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43442/0 (socket says 172.21.15.19:43442) 2024-03-28T13:11:26.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28ad200640 1 -- 172.21.15.19:0/2315058571 learned_addr learned my addr 172.21.15.19:0/2315058571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:26.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28ac800640 1 --2- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559d34039c20 0x559d340b75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28abe00640 1 --2- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d340c1240 0x559d340c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28ad200640 1 -- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559d34039c20 msgr2=0x559d340b75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28ad200640 1 --2- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559d34039c20 0x559d340b75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28ad200640 1 -- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d340c1240 msgr2=0x559d340c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28ad200640 1 --2- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d340c1240 0x559d340c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.907+0000 7f28ad200640 1 -- 172.21.15.19:0/2315058571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d34021220 con 0x559d340cf9b0 2024-03-28T13:11:26.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.908+0000 7f28abe00640 1 --2- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559d340c1240 0x559d340c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:26.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.908+0000 7f28ad200640 1 --2- 172.21.15.19:0/2315058571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d340cf9b0 0x559d340c88e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f28a4007000 tx=0x7f28a4003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.908+0000 7f28a9600640 1 -- 172.21.15.19:0/2315058571 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4062070 con 0x559d340cf9b0 2024-03-28T13:11:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.908+0000 7f28a9600640 1 -- 172.21.15.19:0/2315058571 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28a405c430 con 0x559d340cf9b0 2024-03-28T13:11:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.908+0000 7f28a9600640 1 -- 172.21.15.19:0/2315058571 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a405c820 con 0x559d340cf9b0 2024-03-28T13:11:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.908+0000 7f28b000cf40 1 -- 172.21.15.19:0/2315058571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559d34020f40 con 0x559d340cf9b0 2024-03-28T13:11:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.908+0000 7f28b000cf40 1 -- 172.21.15.19:0/2315058571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d340ca140 con 0x559d340cf9b0 2024-03-28T13:11:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.911+0000 7f28a9600640 1 -- 172.21.15.19:0/2315058571 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f28a4065070 con 0x559d340cf9b0 2024-03-28T13:11:26.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.911+0000 7f28a9600640 1 --2- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f288c06af60 0x7f288c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.912+0000 7f28a9600640 1 -- 172.21.15.19:0/2315058571 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f28a40cc330 con 0x559d340cf9b0 2024-03-28T13:11:26.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.912+0000 7f28b000cf40 1 --2- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x559d340b7b30 0x559d340b8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:26.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.912+0000 7f28b000cf40 1 -- 172.21.15.19:0/2315058571 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38) -- 0x559d340caca0 con 0x559d340b7b30 2024-03-28T13:11:26.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.912+0000 7f28ac800640 1 --2- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f288c06af60 0x7f288c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.912+0000 7f28abe00640 1 --2- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x559d340b7b30 0x559d340b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:26.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.913+0000 7f28ac800640 1 --2- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f288c06af60 0x7f288c06d420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f289c00b3c0 tx=0x7f289c01b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.914+0000 7f28abe00640 1 --2- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x559d340b7b30 0x559d340b8940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:26.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.914+0000 7f28abe00640 1 -- 172.21.15.19:0/2315058571 <== osd.6 v2:172.21.15.188:6816/3597358155 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f2898010350 con 0x559d340b7b30 2024-03-28T13:11:26.917 INFO:teuthology.orchestra.run.smithi019.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-03-28T13:11:26.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.915+0000 7f28b000cf40 1 -- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x559d340b7b30 msgr2=0x559d340b8940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.915+0000 7f28b000cf40 1 --2- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x559d340b7b30 0x559d340b8940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:26.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.915+0000 7f28b000cf40 1 -- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f288c06af60 msgr2=0x7f288c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.915+0000 7f28b000cf40 1 --2- 172.21.15.19:0/2315058571 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f288c06af60 0x7f288c06d420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f289c00b3c0 tx=0x7f289c01b040 comp rx=0 tx=0).stop 2024-03-28T13:11:26.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.915+0000 7f28b000cf40 1 -- 172.21.15.19:0/2315058571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d340cf9b0 msgr2=0x559d340c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:26.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.915+0000 7f28b000cf40 1 --2- 172.21.15.19:0/2315058571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559d340cf9b0 0x559d340c88e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f28a4007000 tx=0x7f28a4003330 comp rx=0 tx=0).stop 2024-03-28T13:11:26.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.915+0000 7f28ad200640 1 -- 172.21.15.19:0/2315058571 reap_dead start 2024-03-28T13:11:26.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.915+0000 7f28b000cf40 1 -- 172.21.15.19:0/2315058571 shutdown_connections 2024-03-28T13:11:26.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.915+0000 7f28b000cf40 1 -- 172.21.15.19:0/2315058571 >> 172.21.15.19:0/2315058571 conn(0x559d3403d630 msgr2=0x559d33fd9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:26.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.915+0000 7f28b000cf40 1 -- 172.21.15.19:0/2315058571 shutdown_connections 2024-03-28T13:11:26.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:26.915+0000 7f28b000cf40 1 -- 172.21.15.19:0/2315058571 wait complete. 2024-03-28T13:11:26.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T13:11:26.920 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-28T13:11:26.920 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:11:27.035 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:11:27.156 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-28T13:11:27.156 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:11:27.156 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:11:27.156 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:11:27.216 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:11:27.216 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:11:27.216 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:11:27.216 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:11:27.272 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:11:27.272 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:11:27.273 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:11:27.273 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:11:27.330 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:11:27.330 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:11:27.330 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:11:27.330 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:11:27.385 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:11:27.386 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:11:27.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.618+0000 7f7f48400640 1 Processor -- start 2024-03-28T13:11:27.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.618+0000 7f7f48400640 1 -- start start 2024-03-28T13:11:27.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.620+0000 7f7f48400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7f4015f900 0x7f7f4015c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:27.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.620+0000 7f7f48400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7f4015da60 0x7f7f400cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:27.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.620+0000 7f7f48400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4015cd90 con 0x7f7f4015fe90 2024-03-28T13:11:27.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.620+0000 7f7f48400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7f4015d060 con 0x7f7f4015da60 2024-03-28T13:11:27.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.620+0000 7f7f48400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7f4015d340 con 0x7f7f4015f900 2024-03-28T13:11:27.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.620+0000 7f7f46400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7f4015fe90 0x7f7f400ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45464/0 (socket says 172.21.15.19:45464) 2024-03-28T13:11:27.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.620+0000 7f7f46400640 1 -- 172.21.15.19:0/484717841 learned_addr learned my addr 172.21.15.19:0/484717841 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:27.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.620+0000 7f7f45000640 1 --2- 172.21.15.19:0/484717841 >> v2:172.21.15.19:3300/0 conn(0x7f7f4015da60 0x7f7f400cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:27.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.620+0000 7f7f45a00640 1 --2- 172.21.15.19:0/484717841 >> v2:172.21.15.188:3300/0 conn(0x7f7f4015f900 0x7f7f4015c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:27.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.621+0000 7f7f3fe00640 1 -- 172.21.15.19:0/484717841 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3578889998 0 0) 0x7f7f4015cd90 con 0x7f7f4015fe90 2024-03-28T13:11:27.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.621+0000 7f7f3fe00640 1 -- 172.21.15.19:0/484717841 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f28003520 con 0x7f7f4015fe90 2024-03-28T13:11:27.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.621+0000 7f7f45000640 1 -- 172.21.15.19:0/484717841 >> v2:172.21.15.188:3300/0 conn(0x7f7f4015f900 msgr2=0x7f7f4015c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:27.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.621+0000 7f7f45000640 1 --2- 172.21.15.19:0/484717841 >> v2:172.21.15.188:3300/0 conn(0x7f7f4015f900 0x7f7f4015c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:27.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.621+0000 7f7f45000640 1 -- 172.21.15.19:0/484717841 >> v1:172.21.15.19:6789/0 conn(0x7f7f4015fe90 legacy=0x7f7f400ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:27.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.621+0000 7f7f45000640 1 -- 172.21.15.19:0/484717841 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4015d620 con 0x7f7f4015da60 2024-03-28T13:11:27.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.622+0000 7f7f45000640 1 --2- 172.21.15.19:0/484717841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4015da60 0x7f7f400cfe30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7f40070d00 tx=0x7f7f3402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69883e663951effa server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:27.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.622+0000 7f7f3fe00640 1 -- 172.21.15.19:0/484717841 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34039070 con 0x7f7f4015da60 2024-03-28T13:11:27.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.623+0000 7f7f3fe00640 1 -- 172.21.15.19:0/484717841 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3402cc70 con 0x7f7f4015da60 2024-03-28T13:11:27.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.623+0000 7f7f3fe00640 1 -- 172.21.15.19:0/484717841 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34034640 con 0x7f7f4015da60 2024-03-28T13:11:27.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.623+0000 7f7f48400640 1 -- 172.21.15.19:0/484717841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4015da60 msgr2=0x7f7f400cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:27.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.623+0000 7f7f48400640 1 --2- 172.21.15.19:0/484717841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4015da60 0x7f7f400cfe30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7f40070d00 tx=0x7f7f3402bec0 comp rx=0 tx=0).stop 2024-03-28T13:11:27.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.623+0000 7f7f48400640 1 -- 172.21.15.19:0/484717841 shutdown_connections 2024-03-28T13:11:27.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.623+0000 7f7f48400640 1 --2- 172.21.15.19:0/484717841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4015da60 0x7f7f400cfe30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:27.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.623+0000 7f7f48400640 1 --2- 172.21.15.19:0/484717841 >> v2:172.21.15.188:3300/0 conn(0x7f7f4015f900 0x7f7f4015c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:27.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.623+0000 7f7f48400640 1 -- 172.21.15.19:0/484717841 >> 172.21.15.19:0/484717841 conn(0x7f7f400b3e90 msgr2=0x7f7f400b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:27.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.623+0000 7f7f48400640 1 -- 172.21.15.19:0/484717841 shutdown_connections 2024-03-28T13:11:27.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.623+0000 7f7f48400640 1 -- 172.21.15.19:0/484717841 wait complete. 2024-03-28T13:11:27.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.625+0000 7f7f48400640 1 Processor -- start 2024-03-28T13:11:27.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.625+0000 7f7f48400640 1 -- start start 2024-03-28T13:11:27.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.626+0000 7f7f48400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4015da60 0x7f7f400cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:27.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.626+0000 7f7f48400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4015f900 0x7f7f400b7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:27.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.626+0000 7f7f48400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f4015fe90 0x7f7f401f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:27.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.626+0000 7f7f48400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f400d04f0 con 0x7f7f4015f900 2024-03-28T13:11:27.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.626+0000 7f7f48400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7f400d0370 con 0x7f7f4015da60 2024-03-28T13:11:27.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.626+0000 7f7f48400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7f400d0670 con 0x7f7f4015fe90 2024-03-28T13:11:27.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.627+0000 7f7f45000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4015f900 0x7f7f400b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:27.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.627+0000 7f7f45000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4015f900 0x7f7f400b7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43456/0 (socket says 172.21.15.19:43456) 2024-03-28T13:11:27.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.627+0000 7f7f45000640 1 -- 172.21.15.19:0/1812500630 learned_addr learned my addr 172.21.15.19:0/1812500630 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:27.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.627+0000 7f7f45a00640 1 --2- 172.21.15.19:0/1812500630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4015da60 0x7f7f400cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:27.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.627+0000 7f7f45000640 1 -- 172.21.15.19:0/1812500630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f4015fe90 msgr2=0x7f7f401f31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:27.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.627+0000 7f7f45000640 1 --2- 172.21.15.19:0/1812500630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f4015fe90 0x7f7f401f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:27.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.627+0000 7f7f45000640 1 -- 172.21.15.19:0/1812500630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4015da60 msgr2=0x7f7f400cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:27.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.627+0000 7f7f45000640 1 --2- 172.21.15.19:0/1812500630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4015da60 0x7f7f400cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:27.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.627+0000 7f7f45000640 1 -- 172.21.15.19:0/1812500630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f401500a0 con 0x7f7f4015f900 2024-03-28T13:11:27.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.627+0000 7f7f45a00640 1 --2- 172.21.15.19:0/1812500630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4015da60 0x7f7f400cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.627+0000 7f7f45000640 1 --2- 172.21.15.19:0/1812500630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4015f900 0x7f7f400b7590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009e80 tx=0x7f7f34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.628+0000 7f7f3d600640 1 -- 172.21.15.19:0/1812500630 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34042020 con 0x7f7f4015f900 2024-03-28T13:11:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.628+0000 7f7f3d600640 1 -- 172.21.15.19:0/1812500630 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f3403a040 con 0x7f7f4015f900 2024-03-28T13:11:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.628+0000 7f7f3d600640 1 -- 172.21.15.19:0/1812500630 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34039070 con 0x7f7f4015f900 2024-03-28T13:11:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.628+0000 7f7f48400640 1 -- 172.21.15.19:0/1812500630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f400cdff0 con 0x7f7f4015f900 2024-03-28T13:11:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.628+0000 7f7f48400640 1 -- 172.21.15.19:0/1812500630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f400cd230 con 0x7f7f4015f900 2024-03-28T13:11:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.629+0000 7f7f48400640 1 -- 172.21.15.19:0/1812500630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f40068cd0 con 0x7f7f4015f900 2024-03-28T13:11:27.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.630+0000 7f7f3d600640 1 -- 172.21.15.19:0/1812500630 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7f34045020 con 0x7f7f4015f900 2024-03-28T13:11:27.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.631+0000 7f7f3d600640 1 --2- 172.21.15.19:0/1812500630 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f2406c530 0x7f7f2406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:27.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.631+0000 7f7f3d600640 1 -- 172.21.15.19:0/1812500630 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f7f340adb30 con 0x7f7f4015f900 2024-03-28T13:11:27.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.633+0000 7f7f45a00640 1 --2- 172.21.15.19:0/1812500630 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f2406c530 0x7f7f2406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:27.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.634+0000 7f7f45a00640 1 --2- 172.21.15.19:0/1812500630 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f2406c530 0x7f7f2406e9f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009ac0 tx=0x7f7f30008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:27.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.636+0000 7f7f3d600640 1 -- 172.21.15.19:0/1812500630 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7f3404c030 con 0x7f7f4015f900 2024-03-28T13:11:27.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.882+0000 7f7f48400640 1 -- 172.21.15.19:0/1812500630 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f40074930 con 0x7f7f4015f900 2024-03-28T13:11:27.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.883+0000 7f7f3d600640 1 -- 172.21.15.19:0/1812500630 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7f7f401500a0 con 0x7f7f4015f900 2024-03-28T13:11:27.883 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:27.883 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2683971950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2683971950},{"type":"v1","addr":"172.21.15.188:6836","nonce":2683971950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/709274582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":709274582},{"type":"v1","addr":"172.21.15.19:6835","nonce":709274582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5040,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3420318181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3420318181},{"type":"v1","addr":"172.21.15.19:6833","nonce":3420318181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:10:56.216937+0000","modified":"2024-03-28T13:10:59.267258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5061":{"gid":5061,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6837/46284318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":46284318},{"type":"v1","addr":"172.21.15.188:6837","nonce":46284318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5061,"qdb_cluster":[5061]},"id":2}]} 2024-03-28T13:11:27.884 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-03-28T13:11:27.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.885+0000 7f7f48400640 1 -- 172.21.15.19:0/1812500630 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f2406c530 msgr2=0x7f7f2406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:27.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.885+0000 7f7f48400640 1 --2- 172.21.15.19:0/1812500630 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f2406c530 0x7f7f2406e9f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009ac0 tx=0x7f7f30008040 comp rx=0 tx=0).stop 2024-03-28T13:11:27.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.885+0000 7f7f48400640 1 -- 172.21.15.19:0/1812500630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4015f900 msgr2=0x7f7f400b7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:27.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.885+0000 7f7f48400640 1 --2- 172.21.15.19:0/1812500630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4015f900 0x7f7f400b7590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009e80 tx=0x7f7f34004a40 comp rx=0 tx=0).stop 2024-03-28T13:11:27.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.886+0000 7f7f48400640 1 -- 172.21.15.19:0/1812500630 shutdown_connections 2024-03-28T13:11:27.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.886+0000 7f7f48400640 1 --2- 172.21.15.19:0/1812500630 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f2406c530 0x7f7f2406e9f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:27.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.886+0000 7f7f48400640 1 --2- 172.21.15.19:0/1812500630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f4015fe90 0x7f7f401f31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.886+0000 7f7f48400640 1 --2- 172.21.15.19:0/1812500630 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4015f900 0x7f7f400b7590 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.886+0000 7f7f48400640 1 --2- 172.21.15.19:0/1812500630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f4015da60 0x7f7f400cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.886+0000 7f7f48400640 1 -- 172.21.15.19:0/1812500630 >> 172.21.15.19:0/1812500630 conn(0x7f7f400b3e90 msgr2=0x7f7f401602c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.886+0000 7f7f48400640 1 -- 172.21.15.19:0/1812500630 shutdown_connections 2024-03-28T13:11:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:27.886+0000 7f7f48400640 1 -- 172.21.15.19:0/1812500630 wait complete. 2024-03-28T13:11:27.904 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:11:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.120+0000 7f31f8e00640 1 Processor -- start 2024-03-28T13:11:28.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.120+0000 7f31f8e00640 1 -- start start 2024-03-28T13:11:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.121+0000 7f31f8e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31f415fc50 0x7f31f40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.122+0000 7f31f8e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31f40b3150 con 0x7f31f40cc0b0 2024-03-28T13:11:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.122+0000 7f31f8e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31f41600e0 con 0x7f31f40cc490 2024-03-28T13:11:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.122+0000 7f31f8e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f31f415c610 con 0x7f31f415fc50 2024-03-28T13:11:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.122+0000 7f31f3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f31f40cc0b0 0x7f31f406dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45466/0 (socket says 172.21.15.19:45466) 2024-03-28T13:11:28.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.122+0000 7f31f3400640 1 -- 172.21.15.19:0/4100475689 learned_addr learned my addr 172.21.15.19:0/4100475689 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.122+0000 7f31f2a00640 1 --2- 172.21.15.19:0/4100475689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31f415fc50 0x7f31f40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.122+0000 7f31f2000640 1 -- 172.21.15.19:0/4100475689 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2598176005 0 0) 0x7f31f40b3150 con 0x7f31f40cc0b0 2024-03-28T13:11:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.122+0000 7f31f2000640 1 -- 172.21.15.19:0/4100475689 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31d8003520 con 0x7f31f40cc0b0 2024-03-28T13:11:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f2000640 1 -- 172.21.15.19:0/4100475689 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 665086776 0 0) 0x7f31d8003520 con 0x7f31f40cc0b0 2024-03-28T13:11:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f2000640 1 -- 172.21.15.19:0/4100475689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31f415fc50 msgr2=0x7f31f40cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f2000640 1 --2- 172.21.15.19:0/4100475689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31f415fc50 0x7f31f40cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f2000640 1 -- 172.21.15.19:0/4100475689 >> v1:172.21.15.188:6789/0 conn(0x7f31f40cc490 legacy=0x7f31f415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:28.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f2000640 1 -- 172.21.15.19:0/4100475689 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f41603c0 con 0x7f31f40cc0b0 2024-03-28T13:11:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f2000640 1 -- 172.21.15.19:0/4100475689 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f31e4001670 con 0x7f31f40cc0b0 2024-03-28T13:11:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f2000640 1 -- 172.21.15.19:0/4100475689 >> v1:172.21.15.19:6789/0 conn(0x7f31f40cc0b0 legacy=0x7f31f406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f2000640 1 --2- 172.21.15.19:0/4100475689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31d80039e0 0x7f31d8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f2000640 1 --2- 172.21.15.19:0/4100475689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d8004800 0x7f31d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f2000640 1 --2- 172.21.15.19:0/4100475689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31d80051c0 0x7f31d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f2000640 1 -- 172.21.15.19:0/4100475689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f31f415c610 con 0x7f31d8004800 2024-03-28T13:11:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f2000640 1 -- 172.21.15.19:0/4100475689 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f31d800c010 con 0x7f31d80051c0 2024-03-28T13:11:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f2000640 1 -- 172.21.15.19:0/4100475689 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f31d800c210 con 0x7f31d80039e0 2024-03-28T13:11:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f2a00640 1 --2- 172.21.15.19:0/4100475689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31f415fc50 0x7f31f40cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:11:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f2a00640 1 --2- 172.21.15.19:0/4100475689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d8004800 0x7f31d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f3e00640 1 --2- 172.21.15.19:0/4100475689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31d80051c0 0x7f31d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.123+0000 7f31f3400640 1 --2- 172.21.15.19:0/4100475689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31d80039e0 0x7f31d8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.124+0000 7f31f2a00640 1 -- 172.21.15.19:0/4100475689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31d80039e0 msgr2=0x7f31d8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:28.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.124+0000 7f31f2a00640 1 --2- 172.21.15.19:0/4100475689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31d80039e0 0x7f31d8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.124+0000 7f31f2a00640 1 -- 172.21.15.19:0/4100475689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31d80051c0 msgr2=0x7f31d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.124+0000 7f31f2a00640 1 --2- 172.21.15.19:0/4100475689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31d80051c0 0x7f31d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.124+0000 7f31f2a00640 1 -- 172.21.15.19:0/4100475689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d8003520 con 0x7f31d8004800 2024-03-28T13:11:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.125+0000 7f31f2a00640 1 --2- 172.21.15.19:0/4100475689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d8004800 0x7f31d8004c80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009b30 tx=0x7f31d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf96fc22e4df7d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.125+0000 7f31f2000640 1 -- 172.21.15.19:0/4100475689 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e00097d0 con 0x7f31d8004800 2024-03-28T13:11:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.125+0000 7f31f2000640 1 -- 172.21.15.19:0/4100475689 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e0004030 con 0x7f31d8004800 2024-03-28T13:11:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.125+0000 7f31f2000640 1 -- 172.21.15.19:0/4100475689 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e0004400 con 0x7f31d8004800 2024-03-28T13:11:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.126+0000 7f31f8e00640 1 -- 172.21.15.19:0/4100475689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d8004800 msgr2=0x7f31d8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.126+0000 7f31f8e00640 1 --2- 172.21.15.19:0/4100475689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d8004800 0x7f31d8004c80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009b30 tx=0x7f31d8005830 comp rx=0 tx=0).stop 2024-03-28T13:11:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.126+0000 7f31f3e00640 1 -- 172.21.15.19:0/4100475689 reap_dead start 2024-03-28T13:11:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.126+0000 7f31f8e00640 1 -- 172.21.15.19:0/4100475689 shutdown_connections 2024-03-28T13:11:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.126+0000 7f31f8e00640 1 -- 172.21.15.19:0/4100475689 >> 172.21.15.19:0/4100475689 conn(0x7f31f40ccad0 msgr2=0x7f31f40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:28.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.126+0000 7f31f8e00640 1 -- 172.21.15.19:0/4100475689 shutdown_connections 2024-03-28T13:11:28.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.126+0000 7f31f8e00640 1 -- 172.21.15.19:0/4100475689 wait complete. 2024-03-28T13:11:28.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.128+0000 7f31f8e00640 1 Processor -- start 2024-03-28T13:11:28.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.128+0000 7f31f8e00640 1 -- start start 2024-03-28T13:11:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f8e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d8004800 0x7f31f41e5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f8e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31f41e5eb0 0x7f31f412f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f8e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31f412fc70 0x7f31f41300f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f8e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f31f40b5b40 con 0x7f31d8004800 2024-03-28T13:11:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f8e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4160980 con 0x7f31f41e5eb0 2024-03-28T13:11:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f8e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f31f40b5cc0 con 0x7f31f412fc70 2024-03-28T13:11:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d8004800 0x7f31f41e5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d8004800 0x7f31f41e5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43488/0 (socket says 172.21.15.19:43488) 2024-03-28T13:11:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f3400640 1 -- 172.21.15.19:0/1097545206 learned_addr learned my addr 172.21.15.19:0/1097545206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f2a00640 1 --2- 172.21.15.19:0/1097545206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31f41e5eb0 0x7f31f412f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f3e00640 1 --2- 172.21.15.19:0/1097545206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31f412fc70 0x7f31f41300f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f3400640 1 -- 172.21.15.19:0/1097545206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31f412fc70 msgr2=0x7f31f41300f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f3400640 1 --2- 172.21.15.19:0/1097545206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31f412fc70 0x7f31f41300f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f3400640 1 -- 172.21.15.19:0/1097545206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31f41e5eb0 msgr2=0x7f31f412f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f3400640 1 --2- 172.21.15.19:0/1097545206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31f41e5eb0 0x7f31f412f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.129+0000 7f31f3400640 1 -- 172.21.15.19:0/1097545206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f40aecf0 con 0x7f31d8004800 2024-03-28T13:11:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.130+0000 7f31f3400640 1 --2- 172.21.15.19:0/1097545206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d8004800 0x7f31f41e5970 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f31e4007340 tx=0x7f31e4007740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:28.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.130+0000 7f31eb400640 1 -- 172.21.15.19:0/1097545206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e401a070 con 0x7f31d8004800 2024-03-28T13:11:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.130+0000 7f31eb400640 1 -- 172.21.15.19:0/1097545206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e400f9c0 con 0x7f31d8004800 2024-03-28T13:11:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.130+0000 7f31eb400640 1 -- 172.21.15.19:0/1097545206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e400fd90 con 0x7f31d8004800 2024-03-28T13:11:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.130+0000 7f31f8e00640 1 -- 172.21.15.19:0/1097545206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f40aefd0 con 0x7f31d8004800 2024-03-28T13:11:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.130+0000 7f31f8e00640 1 -- 172.21.15.19:0/1097545206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f412db40 con 0x7f31d8004800 2024-03-28T13:11:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.131+0000 7f31f8e00640 1 -- 172.21.15.19:0/1097545206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f4075940 con 0x7f31d8004800 2024-03-28T13:11:28.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.132+0000 7f31eb400640 1 -- 172.21.15.19:0/1097545206 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f31e401d070 con 0x7f31d8004800 2024-03-28T13:11:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.133+0000 7f31eb400640 1 --2- 172.21.15.19:0/1097545206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f31d406c530 0x7f31d406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.133+0000 7f31eb400640 1 -- 172.21.15.19:0/1097545206 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f31e408b300 con 0x7f31d8004800 2024-03-28T13:11:28.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.134+0000 7f31f2a00640 1 --2- 172.21.15.19:0/1097545206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f31d406c530 0x7f31d406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:28.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.135+0000 7f31f2a00640 1 --2- 172.21.15.19:0/1097545206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f31d406c530 0x7f31d406e9f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f31e0035000 tx=0x7f31e0030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:28.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.138+0000 7f31eb400640 1 -- 172.21.15.19:0/1097545206 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f31e4059770 con 0x7f31d8004800 2024-03-28T13:11:28.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.343+0000 7f31f8e00640 1 -- 172.21.15.19:0/1097545206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31f41e6290 con 0x7f31d8004800 2024-03-28T13:11:28.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.345+0000 7f31eb400640 1 -- 172.21.15.19:0/1097545206 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16858 (secure 0 0 0) 0x7f31e4059070 con 0x7f31d8004800 2024-03-28T13:11:28.345 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:28.346 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":38,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:11:23.535441+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:10:54.307571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-28T13:10:55.205691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.0","mappings":[{"from":4,"to":5}]},{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.3e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:11:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.347+0000 7f31f8e00640 1 -- 172.21.15.19:0/1097545206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f31d406c530 msgr2=0x7f31d406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.347+0000 7f31f8e00640 1 --2- 172.21.15.19:0/1097545206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f31d406c530 0x7f31d406e9f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f31e0035000 tx=0x7f31e0030000 comp rx=0 tx=0).stop 2024-03-28T13:11:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.347+0000 7f31f8e00640 1 -- 172.21.15.19:0/1097545206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d8004800 msgr2=0x7f31f41e5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.347+0000 7f31f8e00640 1 --2- 172.21.15.19:0/1097545206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d8004800 0x7f31f41e5970 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f31e4007340 tx=0x7f31e4007740 comp rx=0 tx=0).stop 2024-03-28T13:11:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.347+0000 7f31f8e00640 1 -- 172.21.15.19:0/1097545206 shutdown_connections 2024-03-28T13:11:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.347+0000 7f31f8e00640 1 --2- 172.21.15.19:0/1097545206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f31d406c530 0x7f31d406e9f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.347+0000 7f31f8e00640 1 --2- 172.21.15.19:0/1097545206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31f412fc70 0x7f31f41300f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.347+0000 7f31f8e00640 1 --2- 172.21.15.19:0/1097545206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31f41e5eb0 0x7f31f412f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.347+0000 7f31f8e00640 1 --2- 172.21.15.19:0/1097545206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31d8004800 0x7f31f41e5970 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.347+0000 7f31f8e00640 1 -- 172.21.15.19:0/1097545206 >> 172.21.15.19:0/1097545206 conn(0x7f31f40ccad0 msgr2=0x7f31f40712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.348+0000 7f31f8e00640 1 -- 172.21.15.19:0/1097545206 shutdown_connections 2024-03-28T13:11:28.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.348+0000 7f31f8e00640 1 -- 172.21.15.19:0/1097545206 wait complete. 2024-03-28T13:11:28.365 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:10:54.307571+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:10:55.205691+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:11:28.366 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:11:28.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.588+0000 7f19f5000640 1 Processor -- start 2024-03-28T13:11:28.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.588+0000 7f19f5000640 1 -- start start 2024-03-28T13:11:28.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.589+0000 7f19f5000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19f00d0280 0x7f19f00d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:28.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.589+0000 7f19f5000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f19f0141c10 0x7f19f0071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:28.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.589+0000 7f19f5000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f015ec80 con 0x7f19f0071eb0 2024-03-28T13:11:28.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.589+0000 7f19f5000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f19f00d0ba0 con 0x7f19f0141c10 2024-03-28T13:11:28.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.589+0000 7f19f5000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f19f015ef20 con 0x7f19f00d0280 2024-03-28T13:11:28.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.590+0000 7f19ed400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f19f0141c10 0x7f19f0071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:28.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.590+0000 7f19ed400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f19f0141c10 0x7f19f0071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43492/0 (socket says 172.21.15.19:43492) 2024-03-28T13:11:28.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.590+0000 7f19ed400640 1 -- 172.21.15.19:0/1347611853 learned_addr learned my addr 172.21.15.19:0/1347611853 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:28.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.590+0000 7f19ede00640 1 --2- 172.21.15.19:0/1347611853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19f00d0280 0x7f19f00d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:28.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.590+0000 7f19ed400640 1 -- 172.21.15.19:0/1347611853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19f00d0280 msgr2=0x7f19f00d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:28.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.590+0000 7f19ed400640 1 --2- 172.21.15.19:0/1347611853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19f00d0280 0x7f19f00d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.590+0000 7f19ed400640 1 -- 172.21.15.19:0/1347611853 >> v1:172.21.15.188:6789/0 conn(0x7f19f0071eb0 legacy=0x7f19f006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:28.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.590+0000 7f19ed400640 1 -- 172.21.15.19:0/1347611853 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f015f200 con 0x7f19f0141c10 2024-03-28T13:11:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.591+0000 7f19ed400640 1 --2- 172.21.15.19:0/1347611853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f0141c10 0x7f19f0071970 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009870 tx=0x7f19dc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c463f93ced735aec server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.592+0000 7f19eca00640 1 -- 172.21.15.19:0/1347611853 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc035020 con 0x7f19f0141c10 2024-03-28T13:11:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.592+0000 7f19eca00640 1 -- 172.21.15.19:0/1347611853 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19dc031640 con 0x7f19f0141c10 2024-03-28T13:11:28.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.592+0000 7f19eca00640 1 -- 172.21.15.19:0/1347611853 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc02d040 con 0x7f19f0141c10 2024-03-28T13:11:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.593+0000 7f19f5000640 1 -- 172.21.15.19:0/1347611853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f0141c10 msgr2=0x7f19f0071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.593+0000 7f19f5000640 1 --2- 172.21.15.19:0/1347611853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f0141c10 0x7f19f0071970 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009870 tx=0x7f19dc02b8f0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.593+0000 7f19f5000640 1 -- 172.21.15.19:0/1347611853 shutdown_connections 2024-03-28T13:11:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.593+0000 7f19f5000640 1 --2- 172.21.15.19:0/1347611853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f0141c10 0x7f19f0071970 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.593+0000 7f19f5000640 1 --2- 172.21.15.19:0/1347611853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19f00d0280 0x7f19f00d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.593+0000 7f19f5000640 1 -- 172.21.15.19:0/1347611853 >> 172.21.15.19:0/1347611853 conn(0x7f19f006b750 msgr2=0x7f19f006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.593+0000 7f19f5000640 1 -- 172.21.15.19:0/1347611853 shutdown_connections 2024-03-28T13:11:28.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.593+0000 7f19f5000640 1 -- 172.21.15.19:0/1347611853 wait complete. 2024-03-28T13:11:28.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.594+0000 7f19f5000640 1 Processor -- start 2024-03-28T13:11:28.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.595+0000 7f19f5000640 1 -- start start 2024-03-28T13:11:28.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.595+0000 7f19f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f0071eb0 0x7f19f0154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:28.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.596+0000 7f19f5000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19f00d0280 0x7f19f0155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.596+0000 7f19f5000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19f0141c10 0x7f19f00b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.596+0000 7f19f5000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0158e90 con 0x7f19f0071eb0 2024-03-28T13:11:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.596+0000 7f19f5000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f19f015f7c0 con 0x7f19f00d0280 2024-03-28T13:11:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.596+0000 7f19f5000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f19f0154990 con 0x7f19f0141c10 2024-03-28T13:11:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.596+0000 7f19ede00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f0071eb0 0x7f19f0154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.596+0000 7f19ede00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f0071eb0 0x7f19f0154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43498/0 (socket says 172.21.15.19:43498) 2024-03-28T13:11:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.596+0000 7f19ede00640 1 -- 172.21.15.19:0/3440020015 learned_addr learned my addr 172.21.15.19:0/3440020015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.596+0000 7f19ed400640 1 --2- 172.21.15.19:0/3440020015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19f00d0280 0x7f19f0155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.596+0000 7f19ee800640 1 --2- 172.21.15.19:0/3440020015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19f0141c10 0x7f19f00b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.596+0000 7f19ede00640 1 -- 172.21.15.19:0/3440020015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19f0141c10 msgr2=0x7f19f00b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.596+0000 7f19ede00640 1 --2- 172.21.15.19:0/3440020015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19f0141c10 0x7f19f00b4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.596+0000 7f19ede00640 1 -- 172.21.15.19:0/3440020015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19f00d0280 msgr2=0x7f19f0155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.596+0000 7f19ede00640 1 --2- 172.21.15.19:0/3440020015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19f00d0280 0x7f19f0155dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.596+0000 7f19ede00640 1 -- 172.21.15.19:0/3440020015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f0154c70 con 0x7f19f0071eb0 2024-03-28T13:11:28.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.597+0000 7f19ede00640 1 --2- 172.21.15.19:0/3440020015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f0071eb0 0x7f19f0154450 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f19e000d830 tx=0x7f19e000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:28.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.597+0000 7f19e6000640 1 -- 172.21.15.19:0/3440020015 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e000b6c0 con 0x7f19f0071eb0 2024-03-28T13:11:28.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.597+0000 7f19e6000640 1 -- 172.21.15.19:0/3440020015 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e0019070 con 0x7f19f0071eb0 2024-03-28T13:11:28.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.597+0000 7f19e6000640 1 -- 172.21.15.19:0/3440020015 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e000ba80 con 0x7f19f0071eb0 2024-03-28T13:11:28.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.597+0000 7f19f5000640 1 -- 172.21.15.19:0/3440020015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f01567b0 con 0x7f19f0071eb0 2024-03-28T13:11:28.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.597+0000 7f19f5000640 1 -- 172.21.15.19:0/3440020015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f01551f0 con 0x7f19f0071eb0 2024-03-28T13:11:28.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.598+0000 7f19f5000640 1 -- 172.21.15.19:0/3440020015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f00685c0 con 0x7f19f0071eb0 2024-03-28T13:11:28.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.599+0000 7f19e6000640 1 -- 172.21.15.19:0/3440020015 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f19e001e460 con 0x7f19f0071eb0 2024-03-28T13:11:28.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.601+0000 7f19e6000640 1 --2- 172.21.15.19:0/3440020015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f19d006c530 0x7f19d006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:28.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.601+0000 7f19e6000640 1 -- 172.21.15.19:0/3440020015 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f19e008beb0 con 0x7f19f0071eb0 2024-03-28T13:11:28.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.601+0000 7f19ed400640 1 --2- 172.21.15.19:0/3440020015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f19d006c530 0x7f19d006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:28.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.602+0000 7f19ed400640 1 --2- 172.21.15.19:0/3440020015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f19d006c530 0x7f19d006e9f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f19dc036000 tx=0x7f19dc03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:28.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.605+0000 7f19e6000640 1 -- 172.21.15.19:0/3440020015 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f19e005a3b0 con 0x7f19f0071eb0 2024-03-28T13:11:28.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.815+0000 7f19f5000640 1 -- 172.21.15.19:0/3440020015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f19f00cfdb0 con 0x7f19f0071eb0 2024-03-28T13:11:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.816+0000 7f19e6000640 1 -- 172.21.15.19:0/3440020015 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f19e0059cb0 con 0x7f19f0071eb0 2024-03-28T13:11:28.816 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:11:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.818+0000 7f19f5000640 1 -- 172.21.15.19:0/3440020015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f19d006c530 msgr2=0x7f19d006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.818+0000 7f19f5000640 1 --2- 172.21.15.19:0/3440020015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f19d006c530 0x7f19d006e9f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f19dc036000 tx=0x7f19dc03c040 comp rx=0 tx=0).stop 2024-03-28T13:11:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.818+0000 7f19f5000640 1 -- 172.21.15.19:0/3440020015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f0071eb0 msgr2=0x7f19f0154450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.818+0000 7f19f5000640 1 --2- 172.21.15.19:0/3440020015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f0071eb0 0x7f19f0154450 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f19e000d830 tx=0x7f19e000dcf0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.819+0000 7f19f5000640 1 -- 172.21.15.19:0/3440020015 shutdown_connections 2024-03-28T13:11:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.819+0000 7f19f5000640 1 --2- 172.21.15.19:0/3440020015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f19d006c530 0x7f19d006e9f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.819+0000 7f19f5000640 1 --2- 172.21.15.19:0/3440020015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19f0141c10 0x7f19f00b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.819+0000 7f19f5000640 1 --2- 172.21.15.19:0/3440020015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19f00d0280 0x7f19f0155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.819+0000 7f19f5000640 1 --2- 172.21.15.19:0/3440020015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19f0071eb0 0x7f19f0154450 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.819+0000 7f19f5000640 1 -- 172.21.15.19:0/3440020015 >> 172.21.15.19:0/3440020015 conn(0x7f19f006b750 msgr2=0x7f19f00b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.819+0000 7f19f5000640 1 -- 172.21.15.19:0/3440020015 shutdown_connections 2024-03-28T13:11:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:28.819+0000 7f19f5000640 1 -- 172.21.15.19:0/3440020015 wait complete. 2024-03-28T13:11:28.839 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:11:29.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.060+0000 7f7b84c00640 1 Processor -- start 2024-03-28T13:11:29.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.060+0000 7f7b84c00640 1 -- start start 2024-03-28T13:11:29.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.061+0000 7f7b84c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7b80155930 0x7f7b800d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:29.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.061+0000 7f7b84c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7b800d07c0 0x7f7b800aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:29.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.061+0000 7f7b84c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b800b40b0 con 0x7f7b80155550 2024-03-28T13:11:29.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.061+0000 7f7b84c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7b800d0c10 con 0x7f7b80155930 2024-03-28T13:11:29.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.061+0000 7f7b84c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7b800b4390 con 0x7f7b800d07c0 2024-03-28T13:11:29.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.062+0000 7f7b7de00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7b80155550 0x7f7b8015f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45476/0 (socket says 172.21.15.19:45476) 2024-03-28T13:11:29.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.062+0000 7f7b7de00640 1 -- 172.21.15.19:0/1907704583 learned_addr learned my addr 172.21.15.19:0/1907704583 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:29.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.062+0000 7f7b77e00640 1 --2- 172.21.15.19:0/1907704583 >> v2:172.21.15.19:3300/0 conn(0x7f7b80155930 0x7f7b800d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:29.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.062+0000 7f7b7e800640 1 --2- 172.21.15.19:0/1907704583 >> v2:172.21.15.188:3300/0 conn(0x7f7b800d07c0 0x7f7b800aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:29.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.062+0000 7f7b7d400640 1 -- 172.21.15.19:0/1907704583 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3147558160 0 0) 0x7f7b800b40b0 con 0x7f7b80155550 2024-03-28T13:11:29.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.063+0000 7f7b7d400640 1 -- 172.21.15.19:0/1907704583 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b64005730 con 0x7f7b80155550 2024-03-28T13:11:29.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.063+0000 7f7b77e00640 1 -- 172.21.15.19:0/1907704583 >> v2:172.21.15.188:3300/0 conn(0x7f7b800d07c0 msgr2=0x7f7b800aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:29.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.063+0000 7f7b77e00640 1 --2- 172.21.15.19:0/1907704583 >> v2:172.21.15.188:3300/0 conn(0x7f7b800d07c0 0x7f7b800aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.063+0000 7f7b77e00640 1 -- 172.21.15.19:0/1907704583 >> v1:172.21.15.19:6789/0 conn(0x7f7b80155550 legacy=0x7f7b8015f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.063+0000 7f7b77e00640 1 -- 172.21.15.19:0/1907704583 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b800b4670 con 0x7f7b80155930 2024-03-28T13:11:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.065+0000 7f7b77e00640 1 --2- 172.21.15.19:0/1907704583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b80155930 0x7f7b800d0280 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7b800b3610 tx=0x7f7b6802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119eed3feb58a83a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.065+0000 7f7b7d400640 1 -- 172.21.15.19:0/1907704583 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b680315a0 con 0x7f7b80155930 2024-03-28T13:11:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.065+0000 7f7b7d400640 1 -- 172.21.15.19:0/1907704583 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b68037070 con 0x7f7b80155930 2024-03-28T13:11:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.065+0000 7f7b7d400640 1 -- 172.21.15.19:0/1907704583 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68031940 con 0x7f7b80155930 2024-03-28T13:11:29.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.066+0000 7f7b84c00640 1 -- 172.21.15.19:0/1907704583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b80155930 msgr2=0x7f7b800d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:29.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.066+0000 7f7b84c00640 1 --2- 172.21.15.19:0/1907704583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b80155930 0x7f7b800d0280 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7b800b3610 tx=0x7f7b6802b840 comp rx=0 tx=0).stop 2024-03-28T13:11:29.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.066+0000 7f7b84c00640 1 -- 172.21.15.19:0/1907704583 shutdown_connections 2024-03-28T13:11:29.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.066+0000 7f7b84c00640 1 --2- 172.21.15.19:0/1907704583 >> v2:172.21.15.188:3300/0 conn(0x7f7b800d07c0 0x7f7b800aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.066+0000 7f7b84c00640 1 --2- 172.21.15.19:0/1907704583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b80155930 0x7f7b800d0280 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.066+0000 7f7b84c00640 1 -- 172.21.15.19:0/1907704583 >> 172.21.15.19:0/1907704583 conn(0x7f7b8006b750 msgr2=0x7f7b8006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:29.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.066+0000 7f7b84c00640 1 -- 172.21.15.19:0/1907704583 shutdown_connections 2024-03-28T13:11:29.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.066+0000 7f7b84c00640 1 -- 172.21.15.19:0/1907704583 wait complete. 2024-03-28T13:11:29.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.068+0000 7f7b84c00640 1 Processor -- start 2024-03-28T13:11:29.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.068+0000 7f7b84c00640 1 -- start start 2024-03-28T13:11:29.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.069+0000 7f7b84c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b800d07c0 0x7f7b80099b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:29.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.069+0000 7f7b84c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b80155550 0x7f7b801f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:29.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.069+0000 7f7b84c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b80155930 0x7f7b8015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:29.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.069+0000 7f7b84c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8015ba20 con 0x7f7b800d07c0 2024-03-28T13:11:29.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.069+0000 7f7b84c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b800b4c30 con 0x7f7b80155550 2024-03-28T13:11:29.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.069+0000 7f7b84c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7b8015bba0 con 0x7f7b80155930 2024-03-28T13:11:29.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.069+0000 7f7b7de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b800d07c0 0x7f7b80099b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:29.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.069+0000 7f7b7de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b800d07c0 0x7f7b80099b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43510/0 (socket says 172.21.15.19:43510) 2024-03-28T13:11:29.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.069+0000 7f7b7de00640 1 -- 172.21.15.19:0/3391834174 learned_addr learned my addr 172.21.15.19:0/3391834174 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.070+0000 7f7b7de00640 1 -- 172.21.15.19:0/3391834174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b80155930 msgr2=0x7f7b8015a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.070+0000 7f7b7de00640 1 --2- 172.21.15.19:0/3391834174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b80155930 0x7f7b8015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.070+0000 7f7b7de00640 1 -- 172.21.15.19:0/3391834174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b80155550 msgr2=0x7f7b801f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.070+0000 7f7b7de00640 1 --2- 172.21.15.19:0/3391834174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b80155550 0x7f7b801f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.070+0000 7f7b7de00640 1 -- 172.21.15.19:0/3391834174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8009b730 con 0x7f7b800d07c0 2024-03-28T13:11:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.070+0000 7f7b7de00640 1 --2- 172.21.15.19:0/3391834174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b800d07c0 0x7f7b80099b30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000c460 tx=0x7f7b7000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:29.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.070+0000 7f7b76000640 1 -- 172.21.15.19:0/3391834174 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b70017070 con 0x7f7b800d07c0 2024-03-28T13:11:29.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.071+0000 7f7b76000640 1 -- 172.21.15.19:0/3391834174 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b70005ce0 con 0x7f7b800d07c0 2024-03-28T13:11:29.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.071+0000 7f7b76000640 1 -- 172.21.15.19:0/3391834174 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b700072b0 con 0x7f7b800d07c0 2024-03-28T13:11:29.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.071+0000 7f7b84c00640 1 -- 172.21.15.19:0/3391834174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b8009ba10 con 0x7f7b800d07c0 2024-03-28T13:11:29.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.071+0000 7f7b84c00640 1 -- 172.21.15.19:0/3391834174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b800b03c0 con 0x7f7b800d07c0 2024-03-28T13:11:29.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.072+0000 7f7b84c00640 1 -- 172.21.15.19:0/3391834174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b8009c0d0 con 0x7f7b800d07c0 2024-03-28T13:11:29.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.073+0000 7f7b76000640 1 -- 172.21.15.19:0/3391834174 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7b7001a070 con 0x7f7b800d07c0 2024-03-28T13:11:29.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.074+0000 7f7b76000640 1 --2- 172.21.15.19:0/3391834174 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b5c06c530 0x7f7b5c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:29.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.074+0000 7f7b76000640 1 -- 172.21.15.19:0/3391834174 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f7b7008aa50 con 0x7f7b800d07c0 2024-03-28T13:11:29.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.074+0000 7f7b77e00640 1 --2- 172.21.15.19:0/3391834174 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b5c06c530 0x7f7b5c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:29.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.076+0000 7f7b77e00640 1 --2- 172.21.15.19:0/3391834174 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b5c06c530 0x7f7b5c06e9f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7b68036000 tx=0x7f7b68030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:29.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.078+0000 7f7b76000640 1 -- 172.21.15.19:0/3391834174 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7b70059060 con 0x7f7b800d07c0 2024-03-28T13:11:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.284+0000 7f7b84c00640 1 -- 172.21.15.19:0/3391834174 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7b8009a190 con 0x7f7b800d07c0 2024-03-28T13:11:29.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.284+0000 7f7b76000640 1 -- 172.21.15.19:0/3391834174 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) ==== 104+0+11 (secure 0 0 0) 0x7f7b70058960 con 0x7f7b800d07c0 2024-03-28T13:11:29.284 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:11:29.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.287+0000 7f7b84c00640 1 -- 172.21.15.19:0/3391834174 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b5c06c530 msgr2=0x7f7b5c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:29.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.287+0000 7f7b84c00640 1 --2- 172.21.15.19:0/3391834174 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b5c06c530 0x7f7b5c06e9f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7b68036000 tx=0x7f7b68030000 comp rx=0 tx=0).stop 2024-03-28T13:11:29.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.287+0000 7f7b84c00640 1 -- 172.21.15.19:0/3391834174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b800d07c0 msgr2=0x7f7b80099b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:29.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.287+0000 7f7b84c00640 1 --2- 172.21.15.19:0/3391834174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b800d07c0 0x7f7b80099b30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000c460 tx=0x7f7b7000c920 comp rx=0 tx=0).stop 2024-03-28T13:11:29.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.287+0000 7f7b84c00640 1 -- 172.21.15.19:0/3391834174 shutdown_connections 2024-03-28T13:11:29.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.287+0000 7f7b84c00640 1 --2- 172.21.15.19:0/3391834174 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b5c06c530 0x7f7b5c06e9f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.287+0000 7f7b84c00640 1 --2- 172.21.15.19:0/3391834174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b80155930 0x7f7b8015a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.287+0000 7f7b84c00640 1 --2- 172.21.15.19:0/3391834174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b80155550 0x7f7b801f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.287+0000 7f7b84c00640 1 --2- 172.21.15.19:0/3391834174 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b800d07c0 0x7f7b80099b30 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.287+0000 7f7b84c00640 1 -- 172.21.15.19:0/3391834174 >> 172.21.15.19:0/3391834174 conn(0x7f7b8006b750 msgr2=0x7f7b8006edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:29.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.287+0000 7f7b84c00640 1 -- 172.21.15.19:0/3391834174 shutdown_connections 2024-03-28T13:11:29.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.287+0000 7f7b84c00640 1 -- 172.21.15.19:0/3391834174 wait complete. 2024-03-28T13:11:29.307 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:11:29.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.538+0000 7f0b97e00640 1 Processor -- start 2024-03-28T13:11:29.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.539+0000 7f0b97e00640 1 -- start start 2024-03-28T13:11:29.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.540+0000 7f0b97e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b980b69c0 0x7f0b98168f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:29.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.540+0000 7f0b97e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b980d2710 con 0x7f0b980d37b0 2024-03-28T13:11:29.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.540+0000 7f0b97e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b980d29f0 con 0x7f0b9815e4c0 2024-03-28T13:11:29.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.540+0000 7f0b97e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0b9815e8a0 con 0x7f0b980b69c0 2024-03-28T13:11:29.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.540+0000 7f0b96000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0b980d37b0 0x7f0b980b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45478/0 (socket says 172.21.15.19:45478) 2024-03-28T13:11:29.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.540+0000 7f0b96000640 1 -- 172.21.15.19:0/4140799015 learned_addr learned my addr 172.21.15.19:0/4140799015 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:29.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.540+0000 7f0b97400640 1 --2- 172.21.15.19:0/4140799015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b980b69c0 0x7f0b98168f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:29.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.541+0000 7f0b95600640 1 -- 172.21.15.19:0/4140799015 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3092478852 0 0) 0x7f0b980d2710 con 0x7f0b980d37b0 2024-03-28T13:11:29.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.541+0000 7f0b95600640 1 -- 172.21.15.19:0/4140799015 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b7c003520 con 0x7f0b980d37b0 2024-03-28T13:11:29.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.541+0000 7f0b95600640 1 -- 172.21.15.19:0/4140799015 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2898653794 0 0) 0x7f0b7c003520 con 0x7f0b980d37b0 2024-03-28T13:11:29.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.541+0000 7f0b95600640 1 -- 172.21.15.19:0/4140799015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b980b69c0 msgr2=0x7f0b98168f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:29.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.541+0000 7f0b95600640 1 --2- 172.21.15.19:0/4140799015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b980b69c0 0x7f0b98168f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.541+0000 7f0b95600640 1 -- 172.21.15.19:0/4140799015 >> v1:172.21.15.188:6789/0 conn(0x7f0b9815e4c0 legacy=0x7f0b980b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:29.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.541+0000 7f0b95600640 1 -- 172.21.15.19:0/4140799015 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b980d2cd0 con 0x7f0b980d37b0 2024-03-28T13:11:29.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.541+0000 7f0b95600640 1 -- 172.21.15.19:0/4140799015 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f0b80001670 con 0x7f0b980d37b0 2024-03-28T13:11:29.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.541+0000 7f0b95600640 1 -- 172.21.15.19:0/4140799015 >> v1:172.21.15.19:6789/0 conn(0x7f0b980d37b0 legacy=0x7f0b980b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:29.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.541+0000 7f0b95600640 1 --2- 172.21.15.19:0/4140799015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b980b69c0 0x7f0b7c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:29.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.541+0000 7f0b95600640 1 --2- 172.21.15.19:0/4140799015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7c003f60 0x7f0b7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:29.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.541+0000 7f0b95600640 1 --2- 172.21.15.19:0/4140799015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b7c004d40 0x7f0b7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:29.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.541+0000 7f0b95600640 1 -- 172.21.15.19:0/4140799015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c00ba70 con 0x7f0b7c003f60 2024-03-28T13:11:29.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.541+0000 7f0b95600640 1 -- 172.21.15.19:0/4140799015 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7c00bc20 con 0x7f0b7c004d40 2024-03-28T13:11:29.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.541+0000 7f0b95600640 1 -- 172.21.15.19:0/4140799015 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0b7c00be20 con 0x7f0b980b69c0 2024-03-28T13:11:29.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.542+0000 7f0b96000640 1 --2- 172.21.15.19:0/4140799015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7c003f60 0x7f0b7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:29.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.542+0000 7f0b96a00640 1 --2- 172.21.15.19:0/4140799015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b980b69c0 0x7f0b7c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:29.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.542+0000 7f0b96000640 1 -- 172.21.15.19:0/4140799015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b980b69c0 msgr2=0x7f0b7c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:29.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.542+0000 7f0b96000640 1 --2- 172.21.15.19:0/4140799015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b980b69c0 0x7f0b7c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.542+0000 7f0b97400640 1 --2- 172.21.15.19:0/4140799015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b7c004d40 0x7f0b7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:29.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.542+0000 7f0b96000640 1 -- 172.21.15.19:0/4140799015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b7c004d40 msgr2=0x7f0b7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:29.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.542+0000 7f0b96000640 1 --2- 172.21.15.19:0/4140799015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b7c004d40 0x7f0b7c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.542+0000 7f0b96000640 1 -- 172.21.15.19:0/4140799015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7c003520 con 0x7f0b7c003f60 2024-03-28T13:11:29.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.542+0000 7f0b96a00640 1 --2- 172.21.15.19:0/4140799015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b980b69c0 0x7f0b7c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:29.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.543+0000 7f0b96000640 1 --2- 172.21.15.19:0/4140799015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7c003f60 0x7f0b7c004800 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000c240 tx=0x7f0b7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553ec27b41aac83d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:29.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.543+0000 7f0b95600640 1 -- 172.21.15.19:0/4140799015 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80007d10 con 0x7f0b7c003f60 2024-03-28T13:11:29.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.544+0000 7f0b95600640 1 -- 172.21.15.19:0/4140799015 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b80030070 con 0x7f0b7c003f60 2024-03-28T13:11:29.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.544+0000 7f0b95600640 1 -- 172.21.15.19:0/4140799015 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8002cda0 con 0x7f0b7c003f60 2024-03-28T13:11:29.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.544+0000 7f0b97e00640 1 -- 172.21.15.19:0/4140799015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7c003f60 msgr2=0x7f0b7c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:29.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.544+0000 7f0b97e00640 1 --2- 172.21.15.19:0/4140799015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b7c003f60 0x7f0b7c004800 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000c240 tx=0x7f0b7c005830 comp rx=0 tx=0).stop 2024-03-28T13:11:29.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.544+0000 7f0b97400640 1 -- 172.21.15.19:0/4140799015 reap_dead start 2024-03-28T13:11:29.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.544+0000 7f0b97e00640 1 -- 172.21.15.19:0/4140799015 shutdown_connections 2024-03-28T13:11:29.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.544+0000 7f0b97e00640 1 -- 172.21.15.19:0/4140799015 >> 172.21.15.19:0/4140799015 conn(0x7f0b980676a0 msgr2=0x7f0b98067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:29.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.544+0000 7f0b97e00640 1 -- 172.21.15.19:0/4140799015 shutdown_connections 2024-03-28T13:11:29.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.544+0000 7f0b97e00640 1 -- 172.21.15.19:0/4140799015 wait complete. 2024-03-28T13:11:29.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.546+0000 7f0b97e00640 1 Processor -- start 2024-03-28T13:11:29.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.546+0000 7f0b97e00640 1 -- start start 2024-03-28T13:11:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.547+0000 7f0b97e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b7c003f60 0x7f0b9814f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.547+0000 7f0b97e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b9815c9b0 0x7f0b9815ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.547+0000 7f0b97e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b980cb410 0x7f0b980cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.547+0000 7f0b97e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b980d3320 con 0x7f0b9815c9b0 2024-03-28T13:11:29.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.547+0000 7f0b97e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0b980d4040 con 0x7f0b7c003f60 2024-03-28T13:11:29.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.547+0000 7f0b97e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0b980d2350 con 0x7f0b980cb410 2024-03-28T13:11:29.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.547+0000 7f0b96000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b9815c9b0 0x7f0b9815ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:29.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.548+0000 7f0b96000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b9815c9b0 0x7f0b9815ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43516/0 (socket says 172.21.15.19:43516) 2024-03-28T13:11:29.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.548+0000 7f0b96000640 1 -- 172.21.15.19:0/1159364996 learned_addr learned my addr 172.21.15.19:0/1159364996 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.548+0000 7f0b96a00640 1 --2- 172.21.15.19:0/1159364996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b7c003f60 0x7f0b9814f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.548+0000 7f0b97400640 1 --2- 172.21.15.19:0/1159364996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b980cb410 0x7f0b980cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.548+0000 7f0b96000640 1 -- 172.21.15.19:0/1159364996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b980cb410 msgr2=0x7f0b980cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.548+0000 7f0b96000640 1 --2- 172.21.15.19:0/1159364996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b980cb410 0x7f0b980cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.548+0000 7f0b96000640 1 -- 172.21.15.19:0/1159364996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b7c003f60 msgr2=0x7f0b9814f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.548+0000 7f0b96000640 1 --2- 172.21.15.19:0/1159364996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b7c003f60 0x7f0b9814f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.548+0000 7f0b96000640 1 -- 172.21.15.19:0/1159364996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9814fd60 con 0x7f0b9815c9b0 2024-03-28T13:11:29.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.548+0000 7f0b97400640 1 --2- 172.21.15.19:0/1159364996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b980cb410 0x7f0b980cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:29.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.548+0000 7f0b96000640 1 --2- 172.21.15.19:0/1159364996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b9815c9b0 0x7f0b9815ce10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000c360 tx=0x7f0b980d3290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:29.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.549+0000 7f0b8f400640 1 -- 172.21.15.19:0/1159364996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80046070 con 0x7f0b9815c9b0 2024-03-28T13:11:29.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.549+0000 7f0b8f400640 1 -- 172.21.15.19:0/1159364996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b80004bf0 con 0x7f0b9815c9b0 2024-03-28T13:11:29.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.549+0000 7f0b8f400640 1 -- 172.21.15.19:0/1159364996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b80007d10 con 0x7f0b9815c9b0 2024-03-28T13:11:29.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.549+0000 7f0b97e00640 1 -- 172.21.15.19:0/1159364996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b980cc160 con 0x7f0b9815c9b0 2024-03-28T13:11:29.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.549+0000 7f0b97e00640 1 -- 172.21.15.19:0/1159364996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b980cc720 con 0x7f0b9815c9b0 2024-03-28T13:11:29.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.550+0000 7f0b97e00640 1 -- 172.21.15.19:0/1159364996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b98075940 con 0x7f0b9815c9b0 2024-03-28T13:11:29.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.551+0000 7f0b8f400640 1 -- 172.21.15.19:0/1159364996 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0b80007070 con 0x7f0b9815c9b0 2024-03-28T13:11:29.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.552+0000 7f0b8f400640 1 --2- 172.21.15.19:0/1159364996 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b7406c560 0x7f0b7406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:29.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.552+0000 7f0b8f400640 1 -- 172.21.15.19:0/1159364996 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f0b800ac940 con 0x7f0b9815c9b0 2024-03-28T13:11:29.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.553+0000 7f0b96a00640 1 --2- 172.21.15.19:0/1159364996 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b7406c560 0x7f0b7406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:29.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.554+0000 7f0b96a00640 1 --2- 172.21.15.19:0/1159364996 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b7406c560 0x7f0b7406ea20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0b88001e40 tx=0x7f0b88009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:29.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.556+0000 7f0b8f400640 1 -- 172.21.15.19:0/1159364996 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0b9814fd60 con 0x7f0b9815c9b0 2024-03-28T13:11:29.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.763+0000 7f0b97e00640 1 -- 172.21.15.19:0/1159364996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0b98154f90 con 0x7f0b9815c9b0 2024-03-28T13:11:29.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.764+0000 7f0b8f400640 1 -- 172.21.15.19:0/1159364996 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) ==== 100+0+11 (secure 0 0 0) 0x7f0b8007a8e0 con 0x7f0b9815c9b0 2024-03-28T13:11:29.764 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:11:29.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.766+0000 7f0b97e00640 1 -- 172.21.15.19:0/1159364996 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b7406c560 msgr2=0x7f0b7406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:29.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.766+0000 7f0b97e00640 1 --2- 172.21.15.19:0/1159364996 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b7406c560 0x7f0b7406ea20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0b88001e40 tx=0x7f0b88009040 comp rx=0 tx=0).stop 2024-03-28T13:11:29.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.766+0000 7f0b97e00640 1 -- 172.21.15.19:0/1159364996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b9815c9b0 msgr2=0x7f0b9815ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:29.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.766+0000 7f0b97e00640 1 --2- 172.21.15.19:0/1159364996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b9815c9b0 0x7f0b9815ce10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000c360 tx=0x7f0b980d3290 comp rx=0 tx=0).stop 2024-03-28T13:11:29.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.766+0000 7f0b97e00640 1 -- 172.21.15.19:0/1159364996 shutdown_connections 2024-03-28T13:11:29.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.766+0000 7f0b97e00640 1 --2- 172.21.15.19:0/1159364996 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b7406c560 0x7f0b7406ea20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.766+0000 7f0b97e00640 1 --2- 172.21.15.19:0/1159364996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b980cb410 0x7f0b980cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.766+0000 7f0b97e00640 1 --2- 172.21.15.19:0/1159364996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b9815c9b0 0x7f0b9815ce10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.766+0000 7f0b97e00640 1 --2- 172.21.15.19:0/1159364996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b7c003f60 0x7f0b9814f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:29.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.766+0000 7f0b97e00640 1 -- 172.21.15.19:0/1159364996 >> 172.21.15.19:0/1159364996 conn(0x7f0b980676a0 msgr2=0x7f0b980d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:29.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.766+0000 7f0b97e00640 1 -- 172.21.15.19:0/1159364996 shutdown_connections 2024-03-28T13:11:29.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:29.767+0000 7f0b97e00640 1 -- 172.21.15.19:0/1159364996 wait complete. 2024-03-28T13:11:29.788 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:11:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.010+0000 7fdaf7c00640 1 Processor -- start 2024-03-28T13:11:30.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.011+0000 7fdaf7c00640 1 -- start start 2024-03-28T13:11:30.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.013+0000 7fdaf7c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdaf00b5050 0x7fdaf006ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:30.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.013+0000 7fdaf7c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf015c9e0 con 0x7fdaf00cf960 2024-03-28T13:11:30.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.013+0000 7fdaf7c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaf006f6a0 con 0x7fdaf015c600 2024-03-28T13:11:30.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.013+0000 7fdaf7c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdaf006f470 con 0x7fdaf00b5050 2024-03-28T13:11:30.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.013+0000 7fdaf5c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdaf00b5050 0x7fdaf006ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:30.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.013+0000 7fdaefe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdaf00cf960 0x7fdaf006e820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45486/0 (socket says 172.21.15.19:45486) 2024-03-28T13:11:30.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.013+0000 7fdaefe00640 1 -- 172.21.15.19:0/3156948993 learned_addr learned my addr 172.21.15.19:0/3156948993 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.014+0000 7fdaf5c00640 1 -- 172.21.15.19:0/3156948993 >> v1:172.21.15.188:6789/0 conn(0x7fdaf015c600 legacy=0x7fdaf00cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.014+0000 7fdaf5c00640 1 -- 172.21.15.19:0/3156948993 >> v1:172.21.15.19:6789/0 conn(0x7fdaf00cf960 legacy=0x7fdaf006e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.014+0000 7fdaf5c00640 1 -- 172.21.15.19:0/3156948993 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf006f980 con 0x7fdaf00b5050 2024-03-28T13:11:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.014+0000 7fdaef400640 1 -- 172.21.15.19:0/3156948993 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1975188542 0 0) 0x7fdaf015c9e0 con 0x7fdaf00cf960 2024-03-28T13:11:30.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.015+0000 7fdaf5c00640 1 --2- 172.21.15.19:0/3156948993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf00b5050 0x7fdaf006ef30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdae80097a0 tx=0x7fdae802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9fa403c116e8fa server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:30.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.015+0000 7fdaef400640 1 -- 172.21.15.19:0/3156948993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae802f810 con 0x7fdaf00b5050 2024-03-28T13:11:30.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.015+0000 7fdaef400640 1 -- 172.21.15.19:0/3156948993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdae802f9b0 con 0x7fdaf00b5050 2024-03-28T13:11:30.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.015+0000 7fdaef400640 1 -- 172.21.15.19:0/3156948993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae802fd80 con 0x7fdaf00b5050 2024-03-28T13:11:30.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.016+0000 7fdaf7c00640 1 -- 172.21.15.19:0/3156948993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf00b5050 msgr2=0x7fdaf006ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:30.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.016+0000 7fdaf7c00640 1 --2- 172.21.15.19:0/3156948993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf00b5050 0x7fdaf006ef30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdae80097a0 tx=0x7fdae802b9f0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.016+0000 7fdaf7c00640 1 -- 172.21.15.19:0/3156948993 shutdown_connections 2024-03-28T13:11:30.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.016+0000 7fdaf7c00640 1 --2- 172.21.15.19:0/3156948993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf00b5050 0x7fdaf006ef30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.016+0000 7fdaf7c00640 1 -- 172.21.15.19:0/3156948993 >> 172.21.15.19:0/3156948993 conn(0x7fdaf00b2cb0 msgr2=0x7fdaf00b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.016+0000 7fdaf7c00640 1 -- 172.21.15.19:0/3156948993 shutdown_connections 2024-03-28T13:11:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.016+0000 7fdaf7c00640 1 -- 172.21.15.19:0/3156948993 wait complete. 2024-03-28T13:11:30.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.019+0000 7fdaf7c00640 1 Processor -- start 2024-03-28T13:11:30.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.019+0000 7fdaf7c00640 1 -- start start 2024-03-28T13:11:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.020+0000 7fdaf7c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf00b5050 0x7fdaf0141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.020+0000 7fdaf7c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaf00cf960 0x7fdaf015f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.020+0000 7fdaf7c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdaf015c600 0x7fdaf00af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.020+0000 7fdaf7c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf0070220 con 0x7fdaf00b5050 2024-03-28T13:11:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.020+0000 7fdaf7c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf006ff40 con 0x7fdaf00cf960 2024-03-28T13:11:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.020+0000 7fdaf7c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdaf01670f0 con 0x7fdaf015c600 2024-03-28T13:11:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.020+0000 7fdaf5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf00b5050 0x7fdaf0141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.020+0000 7fdaf5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf00b5050 0x7fdaf0141df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43534/0 (socket says 172.21.15.19:43534) 2024-03-28T13:11:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.020+0000 7fdaf5200640 1 -- 172.21.15.19:0/3216587028 learned_addr learned my addr 172.21.15.19:0/3216587028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.021+0000 7fdaf5200640 1 -- 172.21.15.19:0/3216587028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdaf015c600 msgr2=0x7fdaf00af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.021+0000 7fdaf5200640 1 --2- 172.21.15.19:0/3216587028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdaf015c600 0x7fdaf00af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.021+0000 7fdaf5200640 1 -- 172.21.15.19:0/3216587028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaf00cf960 msgr2=0x7fdaf015f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.021+0000 7fdaf5200640 1 --2- 172.21.15.19:0/3216587028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaf00cf960 0x7fdaf015f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.021+0000 7fdaf5200640 1 -- 172.21.15.19:0/3216587028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf00adf90 con 0x7fdaf00b5050 2024-03-28T13:11:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.021+0000 7fdaf5200640 1 --2- 172.21.15.19:0/3216587028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf00b5050 0x7fdaf0141df0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdad800c430 tx=0x7fdad800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.022+0000 7fdaecc00640 1 -- 172.21.15.19:0/3216587028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad8017070 con 0x7fdaf00b5050 2024-03-28T13:11:30.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.022+0000 7fdaecc00640 1 -- 172.21.15.19:0/3216587028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad80052d0 con 0x7fdaf00b5050 2024-03-28T13:11:30.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.022+0000 7fdaecc00640 1 -- 172.21.15.19:0/3216587028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad8003830 con 0x7fdaf00b5050 2024-03-28T13:11:30.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.022+0000 7fdaf7c00640 1 -- 172.21.15.19:0/3216587028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf00ae230 con 0x7fdaf00b5050 2024-03-28T13:11:30.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.022+0000 7fdaf7c00640 1 -- 172.21.15.19:0/3216587028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf00ae7f0 con 0x7fdaf00b5050 2024-03-28T13:11:30.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.023+0000 7fdaf7c00640 1 -- 172.21.15.19:0/3216587028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaf00685c0 con 0x7fdaf00b5050 2024-03-28T13:11:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.024+0000 7fdaecc00640 1 -- 172.21.15.19:0/3216587028 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdad801a070 con 0x7fdaf00b5050 2024-03-28T13:11:30.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.026+0000 7fdaecc00640 1 --2- 172.21.15.19:0/3216587028 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdacc06c530 0x7fdacc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:30.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.026+0000 7fdaecc00640 1 -- 172.21.15.19:0/3216587028 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7fdad808ab60 con 0x7fdaf00b5050 2024-03-28T13:11:30.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.026+0000 7fdaefe00640 1 --2- 172.21.15.19:0/3216587028 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdacc06c530 0x7fdacc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:30.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.027+0000 7fdaefe00640 1 --2- 172.21.15.19:0/3216587028 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdacc06c530 0x7fdacc06e9f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdae0000f80 tx=0x7fdae0007c80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:30.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.030+0000 7fdaecc00640 1 -- 172.21.15.19:0/3216587028 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdad8059060 con 0x7fdaf00b5050 2024-03-28T13:11:30.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.276+0000 7fdaf7c00640 1 -- 172.21.15.19:0/3216587028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaf015a6d0 con 0x7fdaf00b5050 2024-03-28T13:11:30.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.277+0000 7fdaecc00640 1 -- 172.21.15.19:0/3216587028 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7fdad8058960 con 0x7fdaf00b5050 2024-03-28T13:11:30.278 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:30.278 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2683971950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2683971950},{"type":"v1","addr":"172.21.15.188:6836","nonce":2683971950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/709274582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":709274582},{"type":"v1","addr":"172.21.15.19:6835","nonce":709274582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5040,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3420318181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3420318181},{"type":"v1","addr":"172.21.15.19:6833","nonce":3420318181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:10:56.216937+0000","modified":"2024-03-28T13:10:59.267258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5061":{"gid":5061,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6837/46284318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":46284318},{"type":"v1","addr":"172.21.15.188:6837","nonce":46284318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5061,"qdb_cluster":[5061]},"id":2}]} 2024-03-28T13:11:30.278 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-03-28T13:11:30.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.280+0000 7fdaf7c00640 1 -- 172.21.15.19:0/3216587028 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdacc06c530 msgr2=0x7fdacc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:30.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.280+0000 7fdaf7c00640 1 --2- 172.21.15.19:0/3216587028 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdacc06c530 0x7fdacc06e9f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdae0000f80 tx=0x7fdae0007c80 comp rx=0 tx=0).stop 2024-03-28T13:11:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.280+0000 7fdaf7c00640 1 -- 172.21.15.19:0/3216587028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf00b5050 msgr2=0x7fdaf0141df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.280+0000 7fdaf7c00640 1 --2- 172.21.15.19:0/3216587028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf00b5050 0x7fdaf0141df0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdad800c430 tx=0x7fdad800c8f0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.280+0000 7fdaf7c00640 1 -- 172.21.15.19:0/3216587028 shutdown_connections 2024-03-28T13:11:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.280+0000 7fdaf7c00640 1 --2- 172.21.15.19:0/3216587028 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdacc06c530 0x7fdacc06e9f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.280+0000 7fdaf7c00640 1 --2- 172.21.15.19:0/3216587028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdaf015c600 0x7fdaf00af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.280+0000 7fdaf7c00640 1 --2- 172.21.15.19:0/3216587028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdaf00cf960 0x7fdaf015f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.280+0000 7fdaf7c00640 1 --2- 172.21.15.19:0/3216587028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdaf00b5050 0x7fdaf0141df0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.280+0000 7fdaf7c00640 1 -- 172.21.15.19:0/3216587028 >> 172.21.15.19:0/3216587028 conn(0x7fdaf00b2cb0 msgr2=0x7fdaf00b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.281+0000 7fdaf7c00640 1 -- 172.21.15.19:0/3216587028 shutdown_connections 2024-03-28T13:11:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.281+0000 7fdaf7c00640 1 -- 172.21.15.19:0/3216587028 wait complete. 2024-03-28T13:11:30.301 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:11:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.526+0000 7fac29800640 1 Processor -- start 2024-03-28T13:11:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.526+0000 7fac29800640 1 -- start start 2024-03-28T13:11:30.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.528+0000 7fac29800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fac2414c1b0 0x7fac24166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.528+0000 7fac29800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac240cfda0 0x7fac240d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.528+0000 7fac29800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac240d06f0 con 0x7fac2414ba20 2024-03-28T13:11:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.528+0000 7fac29800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fac240cde20 con 0x7fac2414c1b0 2024-03-28T13:11:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.528+0000 7fac29800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fac24166ae0 con 0x7fac240cfda0 2024-03-28T13:11:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.529+0000 7fac23400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fac2414c1b0 0x7fac24166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.529+0000 7fac23400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fac2414c1b0 0x7fac24166290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:50590/0 (socket says 172.21.15.19:50590) 2024-03-28T13:11:30.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.529+0000 7fac23400640 1 -- 172.21.15.19:0/2899672786 learned_addr learned my addr 172.21.15.19:0/2899672786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:30.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.529+0000 7fac23e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fac2414ba20 0x7fac240b2ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:57744/0 (socket says 172.21.15.19:57744) 2024-03-28T13:11:30.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.529+0000 7fac28e00640 1 --2- 172.21.15.19:0/2899672786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac240cfda0 0x7fac240d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:30.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.530+0000 7fac23400640 1 -- 172.21.15.19:0/2899672786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac240cfda0 msgr2=0x7fac240d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:30.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.530+0000 7fac23400640 1 --2- 172.21.15.19:0/2899672786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac240cfda0 0x7fac240d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.530+0000 7fac23400640 1 -- 172.21.15.19:0/2899672786 >> v1:172.21.15.188:6789/0 conn(0x7fac2414ba20 legacy=0x7fac240b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:30.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.530+0000 7fac23400640 1 -- 172.21.15.19:0/2899672786 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac24166dc0 con 0x7fac2414c1b0 2024-03-28T13:11:30.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.531+0000 7fac23400640 1 --2- 172.21.15.19:0/2899672786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2414c1b0 0x7fac24166290 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fac10009870 tx=0x7fac1002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e56acff7fd73b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:30.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.531+0000 7fac22a00640 1 -- 172.21.15.19:0/2899672786 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10034070 con 0x7fac2414c1b0 2024-03-28T13:11:30.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.531+0000 7fac22a00640 1 -- 172.21.15.19:0/2899672786 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac10030840 con 0x7fac2414c1b0 2024-03-28T13:11:30.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.532+0000 7fac22a00640 1 -- 172.21.15.19:0/2899672786 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac10030c10 con 0x7fac2414c1b0 2024-03-28T13:11:30.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.532+0000 7fac29800640 1 -- 172.21.15.19:0/2899672786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2414c1b0 msgr2=0x7fac24166290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.532+0000 7fac29800640 1 --2- 172.21.15.19:0/2899672786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2414c1b0 0x7fac24166290 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fac10009870 tx=0x7fac1002b980 comp rx=0 tx=0).stop 2024-03-28T13:11:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.532+0000 7fac29800640 1 -- 172.21.15.19:0/2899672786 shutdown_connections 2024-03-28T13:11:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.532+0000 7fac29800640 1 --2- 172.21.15.19:0/2899672786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2414c1b0 0x7fac24166290 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.532+0000 7fac29800640 1 --2- 172.21.15.19:0/2899672786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac240cfda0 0x7fac240d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.532+0000 7fac29800640 1 -- 172.21.15.19:0/2899672786 >> 172.21.15.19:0/2899672786 conn(0x7fac240ccad0 msgr2=0x7fac240cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.532+0000 7fac29800640 1 -- 172.21.15.19:0/2899672786 shutdown_connections 2024-03-28T13:11:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.533+0000 7fac29800640 1 -- 172.21.15.19:0/2899672786 wait complete. 2024-03-28T13:11:30.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.535+0000 7fac29800640 1 Processor -- start 2024-03-28T13:11:30.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.535+0000 7fac29800640 1 -- start start 2024-03-28T13:11:30.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.536+0000 7fac29800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac240cfda0 0x7fac24162240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:30.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.536+0000 7fac29800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac2414ba20 0x7fac241ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:30.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.536+0000 7fac29800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2414c1b0 0x7fac240a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:30.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.536+0000 7fac29800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac241667d0 con 0x7fac240cfda0 2024-03-28T13:11:30.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.536+0000 7fac29800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fac240ce0c0 con 0x7fac2414c1b0 2024-03-28T13:11:30.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.536+0000 7fac29800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fac2414be00 con 0x7fac2414ba20 2024-03-28T13:11:30.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.537+0000 7fac23e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac240cfda0 0x7fac24162240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:30.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.537+0000 7fac23e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac240cfda0 0x7fac24162240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43536/0 (socket says 172.21.15.19:43536) 2024-03-28T13:11:30.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.537+0000 7fac23e00640 1 -- 172.21.15.19:0/2628146131 learned_addr learned my addr 172.21.15.19:0/2628146131 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:30.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.537+0000 7fac23400640 1 --2- 172.21.15.19:0/2628146131 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac2414ba20 0x7fac241ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:30.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.537+0000 7fac28e00640 1 --2- 172.21.15.19:0/2628146131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2414c1b0 0x7fac240a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:30.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.537+0000 7fac23e00640 1 -- 172.21.15.19:0/2628146131 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac2414ba20 msgr2=0x7fac241ecae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:30.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.537+0000 7fac23e00640 1 --2- 172.21.15.19:0/2628146131 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac2414ba20 0x7fac241ecae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.537+0000 7fac23e00640 1 -- 172.21.15.19:0/2628146131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2414c1b0 msgr2=0x7fac240a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:30.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.537+0000 7fac23e00640 1 --2- 172.21.15.19:0/2628146131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2414c1b0 0x7fac240a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.537+0000 7fac23e00640 1 -- 172.21.15.19:0/2628146131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac241fa6e0 con 0x7fac240cfda0 2024-03-28T13:11:30.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.538+0000 7fac23e00640 1 --2- 172.21.15.19:0/2628146131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac240cfda0 0x7fac24162240 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fac1400c3c0 tx=0x7fac1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:30.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.538+0000 7fac1be00640 1 -- 172.21.15.19:0/2628146131 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac140074e0 con 0x7fac240cfda0 2024-03-28T13:11:30.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.538+0000 7fac1be00640 1 -- 172.21.15.19:0/2628146131 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac14007680 con 0x7fac240cfda0 2024-03-28T13:11:30.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.538+0000 7fac1be00640 1 -- 172.21.15.19:0/2628146131 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac14004a00 con 0x7fac240cfda0 2024-03-28T13:11:30.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.538+0000 7fac29800640 1 -- 172.21.15.19:0/2628146131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac241fa9c0 con 0x7fac240cfda0 2024-03-28T13:11:30.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.538+0000 7fac29800640 1 -- 172.21.15.19:0/2628146131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac241fb260 con 0x7fac240cfda0 2024-03-28T13:11:30.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.539+0000 7fac29800640 1 -- 172.21.15.19:0/2628146131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac24075940 con 0x7fac240cfda0 2024-03-28T13:11:30.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.540+0000 7fac1be00640 1 -- 172.21.15.19:0/2628146131 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fac1401a020 con 0x7fac240cfda0 2024-03-28T13:11:30.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.541+0000 7fac1be00640 1 --2- 172.21.15.19:0/2628146131 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac0406c560 0x7fac0406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:30.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.541+0000 7fac1be00640 1 -- 172.21.15.19:0/2628146131 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7fac1408ae80 con 0x7fac240cfda0 2024-03-28T13:11:30.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.542+0000 7fac23400640 1 --2- 172.21.15.19:0/2628146131 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac0406c560 0x7fac0406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:30.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.543+0000 7fac23400640 1 --2- 172.21.15.19:0/2628146131 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac0406c560 0x7fac0406ea20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac10036000 tx=0x7fac1003b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:30.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.546+0000 7fac1be00640 1 -- 172.21.15.19:0/2628146131 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fac240d06f0 con 0x7fac240cfda0 2024-03-28T13:11:30.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.761+0000 7fac29800640 1 -- 172.21.15.19:0/2628146131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac2406f430 con 0x7fac240cfda0 2024-03-28T13:11:30.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.763+0000 7fac1be00640 1 -- 172.21.15.19:0/2628146131 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16858 (secure 0 0 0) 0x7fac14059420 con 0x7fac240cfda0 2024-03-28T13:11:30.764 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:30.764 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":38,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:11:23.535441+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:10:54.307571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-28T13:10:55.205691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.0","mappings":[{"from":4,"to":5}]},{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.3e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:11:30.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.766+0000 7fac29800640 1 -- 172.21.15.19:0/2628146131 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac0406c560 msgr2=0x7fac0406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:30.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.766+0000 7fac29800640 1 --2- 172.21.15.19:0/2628146131 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac0406c560 0x7fac0406ea20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac10036000 tx=0x7fac1003b040 comp rx=0 tx=0).stop 2024-03-28T13:11:30.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.766+0000 7fac29800640 1 -- 172.21.15.19:0/2628146131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac240cfda0 msgr2=0x7fac24162240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:30.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.766+0000 7fac29800640 1 --2- 172.21.15.19:0/2628146131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac240cfda0 0x7fac24162240 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fac1400c3c0 tx=0x7fac1400c880 comp rx=0 tx=0).stop 2024-03-28T13:11:30.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.766+0000 7fac29800640 1 -- 172.21.15.19:0/2628146131 shutdown_connections 2024-03-28T13:11:30.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.766+0000 7fac29800640 1 --2- 172.21.15.19:0/2628146131 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac0406c560 0x7fac0406ea20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.766+0000 7fac29800640 1 --2- 172.21.15.19:0/2628146131 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac2414c1b0 0x7fac240a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.766+0000 7fac29800640 1 --2- 172.21.15.19:0/2628146131 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac2414ba20 0x7fac241ecae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.766+0000 7fac29800640 1 --2- 172.21.15.19:0/2628146131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac240cfda0 0x7fac24162240 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:30.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.766+0000 7fac29800640 1 -- 172.21.15.19:0/2628146131 >> 172.21.15.19:0/2628146131 conn(0x7fac240ccad0 msgr2=0x7fac240cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:30.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.766+0000 7fac29800640 1 -- 172.21.15.19:0/2628146131 shutdown_connections 2024-03-28T13:11:30.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:30.766+0000 7fac29800640 1 -- 172.21.15.19:0/2628146131 wait complete. 2024-03-28T13:11:30.786 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-28T13:11:30.786 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:11:31.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.008+0000 7fcdb3e00640 1 Processor -- start 2024-03-28T13:11:31.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.008+0000 7fcdb3e00640 1 -- start start 2024-03-28T13:11:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.010+0000 7fcdb3e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcdb40cc0b0 0x7fcdb406dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.010+0000 7fcdb3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdb406e4f0 con 0x7fcdb415fc50 2024-03-28T13:11:31.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.010+0000 7fcdb3e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdb40b3150 con 0x7fcdb40d04d0 2024-03-28T13:11:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.010+0000 7fcdb3e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fcdb415c610 con 0x7fcdb40cc0b0 2024-03-28T13:11:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.011+0000 7fcdb2000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcdb415fc50 0x7fcdb40cfdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45498/0 (socket says 172.21.15.19:45498) 2024-03-28T13:11:31.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.011+0000 7fcdb2000640 1 -- 172.21.15.19:0/1070599306 learned_addr learned my addr 172.21.15.19:0/1070599306 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.011+0000 7fcdb2a00640 1 --2- 172.21.15.19:0/1070599306 >> v2:172.21.15.188:3300/0 conn(0x7fcdb40cc0b0 0x7fcdb406dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.011+0000 7fcdb1600640 1 -- 172.21.15.19:0/1070599306 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2954009015 0 0) 0x7fcdb406e4f0 con 0x7fcdb415fc50 2024-03-28T13:11:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.011+0000 7fcdb1600640 1 -- 172.21.15.19:0/1070599306 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd98003520 con 0x7fcdb415fc50 2024-03-28T13:11:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.011+0000 7fcdb1600640 1 -- 172.21.15.19:0/1070599306 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1658825328 0 0) 0x7fcd98003520 con 0x7fcdb415fc50 2024-03-28T13:11:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb1600640 1 -- 172.21.15.19:0/1070599306 >> v2:172.21.15.188:3300/0 conn(0x7fcdb40cc0b0 msgr2=0x7fcdb406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb1600640 1 --2- 172.21.15.19:0/1070599306 >> v2:172.21.15.188:3300/0 conn(0x7fcdb40cc0b0 0x7fcdb406dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb1600640 1 -- 172.21.15.19:0/1070599306 >> v1:172.21.15.188:6789/0 conn(0x7fcdb40d04d0 legacy=0x7fcdb415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:31.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb1600640 1 -- 172.21.15.19:0/1070599306 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb41600e0 con 0x7fcdb415fc50 2024-03-28T13:11:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb1600640 1 -- 172.21.15.19:0/1070599306 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fcda0001670 con 0x7fcdb415fc50 2024-03-28T13:11:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb1600640 1 -- 172.21.15.19:0/1070599306 >> v1:172.21.15.19:6789/0 conn(0x7fcdb415fc50 legacy=0x7fcdb40cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb1600640 1 --2- 172.21.15.19:0/1070599306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd980039b0 0x7fcd98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb1600640 1 --2- 172.21.15.19:0/1070599306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98004800 0x7fcd98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb1600640 1 --2- 172.21.15.19:0/1070599306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd980051c0 0x7fcd9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb1600640 1 -- 172.21.15.19:0/1070599306 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb415c610 con 0x7fcd98004800 2024-03-28T13:11:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb1600640 1 -- 172.21.15.19:0/1070599306 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9800c160 con 0x7fcd980039b0 2024-03-28T13:11:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb1600640 1 -- 172.21.15.19:0/1070599306 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9800c360 con 0x7fcd980051c0 2024-03-28T13:11:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb2000640 1 --2- 172.21.15.19:0/1070599306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98004800 0x7fcd98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb2000640 1 -- 172.21.15.19:0/1070599306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd980051c0 msgr2=0x7fcd9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb2000640 1 --2- 172.21.15.19:0/1070599306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd980051c0 0x7fcd9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb2000640 1 -- 172.21.15.19:0/1070599306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd980039b0 msgr2=0x7fcd98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:31.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb2a00640 1 --2- 172.21.15.19:0/1070599306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd980039b0 0x7fcd98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb2000640 1 --2- 172.21.15.19:0/1070599306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd980039b0 0x7fcd98003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb2000640 1 -- 172.21.15.19:0/1070599306 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd98003520 con 0x7fcd98004800 2024-03-28T13:11:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb3400640 1 -- 172.21.15.19:0/1070599306 reap_dead start 2024-03-28T13:11:31.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.012+0000 7fcdb2a00640 1 --2- 172.21.15.19:0/1070599306 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd980039b0 0x7fcd98003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:31.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.014+0000 7fcdb2000640 1 --2- 172.21.15.19:0/1070599306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98004800 0x7fcd98004c80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcda000c240 tx=0x7fcd98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e03c010f868884 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:31.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.014+0000 7fcdb1600640 1 -- 172.21.15.19:0/1070599306 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0007d10 con 0x7fcd98004800 2024-03-28T13:11:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.014+0000 7fcdb1600640 1 -- 172.21.15.19:0/1070599306 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda0030070 con 0x7fcd98004800 2024-03-28T13:11:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.014+0000 7fcdb1600640 1 -- 172.21.15.19:0/1070599306 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda002cda0 con 0x7fcd98004800 2024-03-28T13:11:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.015+0000 7fcdb3e00640 1 -- 172.21.15.19:0/1070599306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98004800 msgr2=0x7fcd98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.015+0000 7fcdb3e00640 1 --2- 172.21.15.19:0/1070599306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98004800 0x7fcd98004c80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcda000c240 tx=0x7fcd98005830 comp rx=0 tx=0).stop 2024-03-28T13:11:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.015+0000 7fcdb3e00640 1 -- 172.21.15.19:0/1070599306 shutdown_connections 2024-03-28T13:11:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.015+0000 7fcdb3e00640 1 --2- 172.21.15.19:0/1070599306 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98004800 0x7fcd98004c80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.015+0000 7fcdb3e00640 1 -- 172.21.15.19:0/1070599306 >> 172.21.15.19:0/1070599306 conn(0x7fcdb40ccad0 msgr2=0x7fcdb40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.015+0000 7fcdb3e00640 1 -- 172.21.15.19:0/1070599306 shutdown_connections 2024-03-28T13:11:31.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.015+0000 7fcdb3e00640 1 -- 172.21.15.19:0/1070599306 wait complete. 2024-03-28T13:11:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.017+0000 7fcdb3e00640 1 Processor -- start 2024-03-28T13:11:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.017+0000 7fcdb3e00640 1 -- start start 2024-03-28T13:11:31.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.019+0000 7fcdb3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98004800 0x7fcdb415e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.019+0000 7fcdb3e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdb41e8570 0x7fcdb41e89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.019+0000 7fcdb3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdb4155530 0x7fcdb41559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.019+0000 7fcdb3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb4160980 con 0x7fcd98004800 2024-03-28T13:11:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.019+0000 7fcdb3e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb41606a0 con 0x7fcdb41e8570 2024-03-28T13:11:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.019+0000 7fcdb3e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcdb40cc490 con 0x7fcdb4155530 2024-03-28T13:11:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.019+0000 7fcdb2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98004800 0x7fcdb415e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.019+0000 7fcdb2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98004800 0x7fcdb415e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43550/0 (socket says 172.21.15.19:43550) 2024-03-28T13:11:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.019+0000 7fcdb2a00640 1 -- 172.21.15.19:0/39249650 learned_addr learned my addr 172.21.15.19:0/39249650 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.019+0000 7fcdb2a00640 1 -- 172.21.15.19:0/39249650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdb4155530 msgr2=0x7fcdb41559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.019+0000 7fcdb2a00640 1 --2- 172.21.15.19:0/39249650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdb4155530 0x7fcdb41559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.019+0000 7fcdb2a00640 1 -- 172.21.15.19:0/39249650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdb41e8570 msgr2=0x7fcdb41e89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.019+0000 7fcdb2a00640 1 --2- 172.21.15.19:0/39249650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdb41e8570 0x7fcdb41e89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.019+0000 7fcdb2a00640 1 -- 172.21.15.19:0/39249650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb41e9150 con 0x7fcd98004800 2024-03-28T13:11:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.020+0000 7fcdb2a00640 1 --2- 172.21.15.19:0/39249650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98004800 0x7fcdb415e580 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcda40107c0 tx=0x7fcda4010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.020+0000 7fcdab400640 1 -- 172.21.15.19:0/39249650 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda400f040 con 0x7fcd98004800 2024-03-28T13:11:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.020+0000 7fcdab400640 1 -- 172.21.15.19:0/39249650 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcda400eb60 con 0x7fcd98004800 2024-03-28T13:11:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.020+0000 7fcdab400640 1 -- 172.21.15.19:0/39249650 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4010e70 con 0x7fcd98004800 2024-03-28T13:11:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.020+0000 7fcdb3e00640 1 -- 172.21.15.19:0/39249650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb41e9430 con 0x7fcd98004800 2024-03-28T13:11:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.020+0000 7fcdb3e00640 1 -- 172.21.15.19:0/39249650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb41e99f0 con 0x7fcd98004800 2024-03-28T13:11:31.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.022+0000 7fcdb3e00640 1 -- 172.21.15.19:0/39249650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdb4075940 con 0x7fcd98004800 2024-03-28T13:11:31.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.022+0000 7fcdab400640 1 -- 172.21.15.19:0/39249650 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcda401c400 con 0x7fcd98004800 2024-03-28T13:11:31.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.023+0000 7fcdab400640 1 --2- 172.21.15.19:0/39249650 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd9406c560 0x7fcd9406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:31.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.023+0000 7fcdab400640 1 -- 172.21.15.19:0/39249650 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7fcda408be10 con 0x7fcd98004800 2024-03-28T13:11:31.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.024+0000 7fcdb2000640 1 --2- 172.21.15.19:0/39249650 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd9406c560 0x7fcd9406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:31.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.025+0000 7fcdb2000640 1 --2- 172.21.15.19:0/39249650 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd9406c560 0x7fcd9406ea20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcda0034000 tx=0x7fcda0007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:31.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.030+0000 7fcdab400640 1 -- 172.21.15.19:0/39249650 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcda408f030 con 0x7fcd98004800 2024-03-28T13:11:31.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.273+0000 7fcdb3e00640 1 -- 172.21.15.19:0/39249650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdb4145ad0 con 0x7fcd98004800 2024-03-28T13:11:31.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.275+0000 7fcdab400640 1 -- 172.21.15.19:0/39249650 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7fcda4018030 con 0x7fcd98004800 2024-03-28T13:11:31.275 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:31.275 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2683971950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2683971950},{"type":"v1","addr":"172.21.15.188:6836","nonce":2683971950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/709274582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":709274582},{"type":"v1","addr":"172.21.15.19:6835","nonce":709274582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5040,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3420318181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3420318181},{"type":"v1","addr":"172.21.15.19:6833","nonce":3420318181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:10:56.216937+0000","modified":"2024-03-28T13:10:59.267258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5061":{"gid":5061,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6837/46284318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":46284318},{"type":"v1","addr":"172.21.15.188:6837","nonce":46284318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5061,"qdb_cluster":[5061]},"id":2}]} 2024-03-28T13:11:31.275 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-03-28T13:11:31.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.277+0000 7fcdb3e00640 1 -- 172.21.15.19:0/39249650 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd9406c560 msgr2=0x7fcd9406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:31.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.277+0000 7fcdb3e00640 1 --2- 172.21.15.19:0/39249650 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd9406c560 0x7fcd9406ea20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcda0034000 tx=0x7fcda0007000 comp rx=0 tx=0).stop 2024-03-28T13:11:31.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.277+0000 7fcdb3e00640 1 -- 172.21.15.19:0/39249650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98004800 msgr2=0x7fcdb415e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:31.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.277+0000 7fcdb3e00640 1 --2- 172.21.15.19:0/39249650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98004800 0x7fcdb415e580 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcda40107c0 tx=0x7fcda4010c80 comp rx=0 tx=0).stop 2024-03-28T13:11:31.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.278+0000 7fcdb3e00640 1 -- 172.21.15.19:0/39249650 shutdown_connections 2024-03-28T13:11:31.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.278+0000 7fcdb3e00640 1 --2- 172.21.15.19:0/39249650 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd9406c560 0x7fcd9406ea20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.278+0000 7fcdb3e00640 1 --2- 172.21.15.19:0/39249650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdb4155530 0x7fcdb41559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.278+0000 7fcdb3e00640 1 --2- 172.21.15.19:0/39249650 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdb41e8570 0x7fcdb41e89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.278+0000 7fcdb3e00640 1 --2- 172.21.15.19:0/39249650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd98004800 0x7fcdb415e580 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.278+0000 7fcdb3e00640 1 -- 172.21.15.19:0/39249650 >> 172.21.15.19:0/39249650 conn(0x7fcdb40ccad0 msgr2=0x7fcdb4073000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:31.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.278+0000 7fcdb3e00640 1 -- 172.21.15.19:0/39249650 shutdown_connections 2024-03-28T13:11:31.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.278+0000 7fcdb3e00640 1 -- 172.21.15.19:0/39249650 wait complete. 2024-03-28T13:11:31.296 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:11:31.497 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:11:31.507 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:11:31.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.520+0000 7f7263c00640 1 Processor -- start 2024-03-28T13:11:31.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.520+0000 7f7263c00640 1 -- start start 2024-03-28T13:11:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.521+0000 7f7263c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f725c15fa10 0x7f725c0b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.521+0000 7f7263c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f725c0d0c30 0x7f725c0d1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.521+0000 7f7263c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f725c0d3140 con 0x7f725c0d2bf0 2024-03-28T13:11:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.521+0000 7f7263c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f725c0d15c0 con 0x7f725c0d0c30 2024-03-28T13:11:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.521+0000 7f7263c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f725c0d04a0 con 0x7f725c15fa10 2024-03-28T13:11:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.521+0000 7f725be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f725c0d2bf0 0x7f725c0cfd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45512/0 (socket says 172.21.15.19:45512) 2024-03-28T13:11:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.521+0000 7f725be00640 1 -- 172.21.15.19:0/3038366921 learned_addr learned my addr 172.21.15.19:0/3038366921 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.521+0000 7f7261c00640 1 --2- 172.21.15.19:0/3038366921 >> v2:172.21.15.19:3300/0 conn(0x7f725c0d0c30 0x7f725c0d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:31.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.521+0000 7f7261200640 1 --2- 172.21.15.19:0/3038366921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f725c15fa10 0x7f725c0b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.522+0000 7f725b400640 1 -- 172.21.15.19:0/3038366921 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1273028262 0 0) 0x7f725c0d3140 con 0x7f725c0d2bf0 2024-03-28T13:11:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.522+0000 7f725b400640 1 -- 172.21.15.19:0/3038366921 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7244003520 con 0x7f725c0d2bf0 2024-03-28T13:11:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.522+0000 7f7261c00640 1 -- 172.21.15.19:0/3038366921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f725c15fa10 msgr2=0x7f725c0b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.522+0000 7f7261c00640 1 --2- 172.21.15.19:0/3038366921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f725c15fa10 0x7f725c0b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.522+0000 7f7261c00640 1 -- 172.21.15.19:0/3038366921 >> v1:172.21.15.19:6789/0 conn(0x7f725c0d2bf0 legacy=0x7f725c0cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.522+0000 7f7261c00640 1 -- 172.21.15.19:0/3038366921 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c0d3420 con 0x7f725c0d0c30 2024-03-28T13:11:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.522+0000 7f7261200640 1 --2- 172.21.15.19:0/3038366921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f725c15fa10 0x7f725c0b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:11:31.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.523+0000 7f7261c00640 1 --2- 172.21.15.19:0/3038366921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f725c0d0c30 0x7f725c0d1080 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7254009d90 tx=0x7f725402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8b267172e97c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:31.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.524+0000 7f725b400640 1 -- 172.21.15.19:0/3038366921 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7254039070 con 0x7f725c0d0c30 2024-03-28T13:11:31.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.524+0000 7f725b400640 1 -- 172.21.15.19:0/3038366921 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f725402cce0 con 0x7f725c0d0c30 2024-03-28T13:11:31.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.524+0000 7f725b400640 1 -- 172.21.15.19:0/3038366921 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7254034640 con 0x7f725c0d0c30 2024-03-28T13:11:31.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.524+0000 7f7263c00640 1 -- 172.21.15.19:0/3038366921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f725c0d0c30 msgr2=0x7f725c0d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:31.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.524+0000 7f7263c00640 1 --2- 172.21.15.19:0/3038366921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f725c0d0c30 0x7f725c0d1080 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7254009d90 tx=0x7f725402bf30 comp rx=0 tx=0).stop 2024-03-28T13:11:31.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.524+0000 7f7263c00640 1 -- 172.21.15.19:0/3038366921 shutdown_connections 2024-03-28T13:11:31.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.524+0000 7f7263c00640 1 --2- 172.21.15.19:0/3038366921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f725c0d0c30 0x7f725c0d1080 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.524+0000 7f7263c00640 1 --2- 172.21.15.19:0/3038366921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f725c15fa10 0x7f725c0b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.524+0000 7f7263c00640 1 -- 172.21.15.19:0/3038366921 >> 172.21.15.19:0/3038366921 conn(0x7f725c08bfb0 msgr2=0x7f725c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:31.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.524+0000 7f7263c00640 1 -- 172.21.15.19:0/3038366921 shutdown_connections 2024-03-28T13:11:31.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.524+0000 7f7263c00640 1 -- 172.21.15.19:0/3038366921 wait complete. 2024-03-28T13:11:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.526+0000 7f7263c00640 1 Processor -- start 2024-03-28T13:11:31.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.526+0000 7f7263c00640 1 -- start start 2024-03-28T13:11:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.527+0000 7f7263c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f725c0d0c30 0x7f725c1f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.527+0000 7f7263c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f725c0d2bf0 0x7f725c15a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.527+0000 7f7263c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f725c15fa10 0x7f725c0cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.527+0000 7f7263c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f725c0cf870 con 0x7f725c0d2bf0 2024-03-28T13:11:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.527+0000 7f7263c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f725c0cf590 con 0x7f725c0d0c30 2024-03-28T13:11:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.527+0000 7f7263c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f725c0b5ae0 con 0x7f725c15fa10 2024-03-28T13:11:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.527+0000 7f725be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f725c0d2bf0 0x7f725c15a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.528+0000 7f725be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f725c0d2bf0 0x7f725c15a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43572/0 (socket says 172.21.15.19:43572) 2024-03-28T13:11:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.528+0000 7f725be00640 1 -- 172.21.15.19:0/1573853929 learned_addr learned my addr 172.21.15.19:0/1573853929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.528+0000 7f7261200640 1 --2- 172.21.15.19:0/1573853929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f725c0d0c30 0x7f725c1f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.528+0000 7f7261c00640 1 --2- 172.21.15.19:0/1573853929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f725c15fa10 0x7f725c0cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.528+0000 7f725be00640 1 -- 172.21.15.19:0/1573853929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f725c15fa10 msgr2=0x7f725c0cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.528+0000 7f725be00640 1 --2- 172.21.15.19:0/1573853929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f725c15fa10 0x7f725c0cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.528+0000 7f725be00640 1 -- 172.21.15.19:0/1573853929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f725c0d0c30 msgr2=0x7f725c1f87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.528+0000 7f725be00640 1 --2- 172.21.15.19:0/1573853929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f725c0d0c30 0x7f725c1f87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.528+0000 7f725be00640 1 -- 172.21.15.19:0/1573853929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c157620 con 0x7f725c0d2bf0 2024-03-28T13:11:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.528+0000 7f725be00640 1 --2- 172.21.15.19:0/1573853929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f725c0d2bf0 0x7f725c15a410 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f724800c700 tx=0x7f724800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.529+0000 7f7258c00640 1 -- 172.21.15.19:0/1573853929 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248017070 con 0x7f725c0d2bf0 2024-03-28T13:11:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.529+0000 7f7258c00640 1 -- 172.21.15.19:0/1573853929 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7248007290 con 0x7f725c0d2bf0 2024-03-28T13:11:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.529+0000 7f7258c00640 1 -- 172.21.15.19:0/1573853929 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248001d40 con 0x7f725c0d2bf0 2024-03-28T13:11:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.529+0000 7f7263c00640 1 -- 172.21.15.19:0/1573853929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725c157900 con 0x7f725c0d2bf0 2024-03-28T13:11:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.529+0000 7f7263c00640 1 -- 172.21.15.19:0/1573853929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f725c157ec0 con 0x7f725c0d2bf0 2024-03-28T13:11:31.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.530+0000 7f7263c00640 1 -- 172.21.15.19:0/1573853929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f725c068cd0 con 0x7f725c0d2bf0 2024-03-28T13:11:31.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.531+0000 7f7258c00640 1 -- 172.21.15.19:0/1573853929 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f724801a070 con 0x7f725c0d2bf0 2024-03-28T13:11:31.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.532+0000 7f7258c00640 1 --2- 172.21.15.19:0/1573853929 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f723c06c530 0x7f723c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:31.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.532+0000 7f7258c00640 1 -- 172.21.15.19:0/1573853929 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f724808aa80 con 0x7f725c0d2bf0 2024-03-28T13:11:31.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.532+0000 7f7261200640 1 --2- 172.21.15.19:0/1573853929 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f723c06c530 0x7f723c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:31.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.533+0000 7f7261200640 1 --2- 172.21.15.19:0/1573853929 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f723c06c530 0x7f723c06e9f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7250009560 tx=0x7f7250009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:31.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.537+0000 7f7258c00640 1 -- 172.21.15.19:0/1573853929 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f72480589d0 con 0x7f725c0d2bf0 2024-03-28T13:11:31.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.781+0000 7f7263c00640 1 -- 172.21.15.19:0/1573853929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f725c0d09c0 con 0x7f725c0d2bf0 2024-03-28T13:11:31.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.783+0000 7f7258c00640 1 -- 172.21.15.19:0/1573853929 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5178 (secure 0 0 0) 0x7f724805ccd0 con 0x7f725c0d2bf0 2024-03-28T13:11:31.783 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:31.783 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2683971950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2683971950},{"type":"v1","addr":"172.21.15.188:6836","nonce":2683971950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/709274582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":709274582},{"type":"v1","addr":"172.21.15.19:6835","nonce":709274582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5040,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3420318181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3420318181},{"type":"v1","addr":"172.21.15.19:6833","nonce":3420318181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:10:56.216937+0000","modified":"2024-03-28T13:10:59.267258+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5061},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5061":{"gid":5061,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6837/46284318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":46284318},{"type":"v1","addr":"172.21.15.188:6837","nonce":46284318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5061,"qdb_cluster":[5061]},"id":2}]} 2024-03-28T13:11:31.783 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 17 2024-03-28T13:11:31.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.785+0000 7f7263c00640 1 -- 172.21.15.19:0/1573853929 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f723c06c530 msgr2=0x7f723c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:31.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.785+0000 7f7263c00640 1 --2- 172.21.15.19:0/1573853929 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f723c06c530 0x7f723c06e9f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7250009560 tx=0x7f7250009190 comp rx=0 tx=0).stop 2024-03-28T13:11:31.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.785+0000 7f7263c00640 1 -- 172.21.15.19:0/1573853929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f725c0d2bf0 msgr2=0x7f725c15a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:31.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.785+0000 7f7263c00640 1 --2- 172.21.15.19:0/1573853929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f725c0d2bf0 0x7f725c15a410 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f724800c700 tx=0x7f724800cbc0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.785+0000 7f7263c00640 1 -- 172.21.15.19:0/1573853929 shutdown_connections 2024-03-28T13:11:31.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.785+0000 7f7263c00640 1 --2- 172.21.15.19:0/1573853929 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f723c06c530 0x7f723c06e9f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.785+0000 7f7263c00640 1 --2- 172.21.15.19:0/1573853929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f725c15fa10 0x7f725c0cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.785+0000 7f7263c00640 1 --2- 172.21.15.19:0/1573853929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f725c0d2bf0 0x7f725c15a410 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.785+0000 7f7263c00640 1 --2- 172.21.15.19:0/1573853929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f725c0d0c30 0x7f725c1f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:31.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.785+0000 7f7263c00640 1 -- 172.21.15.19:0/1573853929 >> 172.21.15.19:0/1573853929 conn(0x7f725c08bfb0 msgr2=0x7f725c15e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:31.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.785+0000 7f7263c00640 1 -- 172.21.15.19:0/1573853929 shutdown_connections 2024-03-28T13:11:31.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:31.785+0000 7f7263c00640 1 -- 172.21.15.19:0/1573853929 wait complete. 2024-03-28T13:11:31.806 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:11:32.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.032+0000 7f1a3d400640 1 Processor -- start 2024-03-28T13:11:32.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.033+0000 7f1a3d400640 1 -- start start 2024-03-28T13:11:32.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.034+0000 7f1a3d400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3815da60 0x7f1a380cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.034+0000 7f1a3d400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1a3815fe90 0x7f1a380d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.034+0000 7f1a3d400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3815d060 con 0x7f1a3815f900 2024-03-28T13:11:32.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.034+0000 7f1a3d400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1a3815d340 con 0x7f1a3815fe90 2024-03-28T13:11:32.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.034+0000 7f1a3d400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3815d620 con 0x7f1a3815da60 2024-03-28T13:11:32.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.034+0000 7f1a36800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1a3815f900 0x7f1a3815c850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45528/0 (socket says 172.21.15.19:45528) 2024-03-28T13:11:32.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.034+0000 7f1a36800640 1 -- 172.21.15.19:0/254597882 learned_addr learned my addr 172.21.15.19:0/254597882 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:32.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.034+0000 7f1a35400640 1 -- 172.21.15.19:0/254597882 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1581243279 0 0) 0x7f1a3815d060 con 0x7f1a3815f900 2024-03-28T13:11:32.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35400640 1 -- 172.21.15.19:0/254597882 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a1c003520 con 0x7f1a3815f900 2024-03-28T13:11:32.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35e00640 1 --2- 172.21.15.19:0/254597882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3815da60 0x7f1a380cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a37200640 1 --2- 172.21.15.19:0/254597882 >> v2:172.21.15.188:3300/0 conn(0x7f1a3815fe90 0x7f1a380d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35400640 1 -- 172.21.15.19:0/254597882 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3105596113 0 0) 0x7f1a1c003520 con 0x7f1a3815f900 2024-03-28T13:11:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35400640 1 -- 172.21.15.19:0/254597882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3815da60 msgr2=0x7f1a380cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35400640 1 --2- 172.21.15.19:0/254597882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3815da60 0x7f1a380cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35400640 1 -- 172.21.15.19:0/254597882 >> v2:172.21.15.188:3300/0 conn(0x7f1a3815fe90 msgr2=0x7f1a380d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35400640 1 --2- 172.21.15.19:0/254597882 >> v2:172.21.15.188:3300/0 conn(0x7f1a3815fe90 0x7f1a380d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35400640 1 -- 172.21.15.19:0/254597882 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a380c74b0 con 0x7f1a3815f900 2024-03-28T13:11:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35400640 1 -- 172.21.15.19:0/254597882 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f1a20001670 con 0x7f1a3815f900 2024-03-28T13:11:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35400640 1 -- 172.21.15.19:0/254597882 >> v1:172.21.15.19:6789/0 conn(0x7f1a3815f900 legacy=0x7f1a3815c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35400640 1 --2- 172.21.15.19:0/254597882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a1c0039b0 0x7f1a1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35400640 1 --2- 172.21.15.19:0/254597882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3815da60 0x7f1a1c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35400640 1 --2- 172.21.15.19:0/254597882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1c004d40 0x7f1a1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35400640 1 -- 172.21.15.19:0/254597882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3815d340 con 0x7f1a1c0039b0 2024-03-28T13:11:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35400640 1 -- 172.21.15.19:0/254597882 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3815d620 con 0x7f1a1c004d40 2024-03-28T13:11:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a35400640 1 -- 172.21.15.19:0/254597882 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1a1c00ba70 con 0x7f1a3815da60 2024-03-28T13:11:32.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.035+0000 7f1a36800640 1 --2- 172.21.15.19:0/254597882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a1c0039b0 0x7f1a1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.036+0000 7f1a35e00640 1 --2- 172.21.15.19:0/254597882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3815da60 0x7f1a1c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.036+0000 7f1a37200640 1 --2- 172.21.15.19:0/254597882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1c004d40 0x7f1a1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.036+0000 7f1a36800640 1 -- 172.21.15.19:0/254597882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3815da60 msgr2=0x7f1a1c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:32.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.036+0000 7f1a36800640 1 --2- 172.21.15.19:0/254597882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a3815da60 0x7f1a1c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.036+0000 7f1a36800640 1 -- 172.21.15.19:0/254597882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1c004d40 msgr2=0x7f1a1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:32.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.036+0000 7f1a36800640 1 --2- 172.21.15.19:0/254597882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1c004d40 0x7f1a1c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.036+0000 7f1a36800640 1 -- 172.21.15.19:0/254597882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1c003520 con 0x7f1a1c0039b0 2024-03-28T13:11:32.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.037+0000 7f1a36800640 1 --2- 172.21.15.19:0/254597882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a1c0039b0 0x7f1a1c003dd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000c240 tx=0x7f1a1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7f9f26a956c14d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:32.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.037+0000 7f1a35400640 1 -- 172.21.15.19:0/254597882 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20007d50 con 0x7f1a1c0039b0 2024-03-28T13:11:32.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.037+0000 7f1a35400640 1 -- 172.21.15.19:0/254597882 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a20030070 con 0x7f1a1c0039b0 2024-03-28T13:11:32.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.037+0000 7f1a35400640 1 -- 172.21.15.19:0/254597882 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2002cd80 con 0x7f1a1c0039b0 2024-03-28T13:11:32.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.038+0000 7f1a3d400640 1 -- 172.21.15.19:0/254597882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a1c0039b0 msgr2=0x7f1a1c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:32.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.038+0000 7f1a3d400640 1 --2- 172.21.15.19:0/254597882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a1c0039b0 0x7f1a1c003dd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1a2000c240 tx=0x7f1a1c005830 comp rx=0 tx=0).stop 2024-03-28T13:11:32.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.038+0000 7f1a37200640 1 -- 172.21.15.19:0/254597882 reap_dead start 2024-03-28T13:11:32.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.038+0000 7f1a3d400640 1 -- 172.21.15.19:0/254597882 shutdown_connections 2024-03-28T13:11:32.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.038+0000 7f1a3d400640 1 -- 172.21.15.19:0/254597882 >> 172.21.15.19:0/254597882 conn(0x7f1a380b3e90 msgr2=0x7f1a380b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:32.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.038+0000 7f1a3d400640 1 -- 172.21.15.19:0/254597882 shutdown_connections 2024-03-28T13:11:32.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.038+0000 7f1a3d400640 1 -- 172.21.15.19:0/254597882 wait complete. 2024-03-28T13:11:32.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.040+0000 7f1a3d400640 1 Processor -- start 2024-03-28T13:11:32.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.040+0000 7f1a3d400640 1 -- start start 2024-03-28T13:11:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.041+0000 7f1a3d400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1c0039b0 0x7f1a38153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.041+0000 7f1a3d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3806e320 0x7f1a381563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.041+0000 7f1a3d400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a381548c0 0x7f1a38157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.041+0000 7f1a3d400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a380d01c0 con 0x7f1a3806e320 2024-03-28T13:11:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.041+0000 7f1a3d400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1a380d0040 con 0x7f1a1c0039b0 2024-03-28T13:11:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.041+0000 7f1a3d400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1a380ce8d0 con 0x7f1a381548c0 2024-03-28T13:11:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.041+0000 7f1a35e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3806e320 0x7f1a381563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.042+0000 7f1a35e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3806e320 0x7f1a381563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43588/0 (socket says 172.21.15.19:43588) 2024-03-28T13:11:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.042+0000 7f1a35e00640 1 -- 172.21.15.19:0/1939559412 learned_addr learned my addr 172.21.15.19:0/1939559412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.042+0000 7f1a36800640 1 --2- 172.21.15.19:0/1939559412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1c0039b0 0x7f1a38153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.042+0000 7f1a37200640 1 --2- 172.21.15.19:0/1939559412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a381548c0 0x7f1a38157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.042+0000 7f1a35e00640 1 -- 172.21.15.19:0/1939559412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a381548c0 msgr2=0x7f1a38157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.042+0000 7f1a35e00640 1 --2- 172.21.15.19:0/1939559412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a381548c0 0x7f1a38157d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.042+0000 7f1a35e00640 1 -- 172.21.15.19:0/1939559412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1c0039b0 msgr2=0x7f1a38153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.042+0000 7f1a35e00640 1 --2- 172.21.15.19:0/1939559412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1c0039b0 0x7f1a38153930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.042+0000 7f1a35e00640 1 -- 172.21.15.19:0/1939559412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3806d1c0 con 0x7f1a3806e320 2024-03-28T13:11:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.042+0000 7f1a35e00640 1 --2- 172.21.15.19:0/1939559412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3806e320 0x7f1a381563e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1a28012410 tx=0x7f1a28012810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.043+0000 7f1a2ea00640 1 -- 172.21.15.19:0/1939559412 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28017790 con 0x7f1a3806e320 2024-03-28T13:11:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.043+0000 7f1a2ea00640 1 -- 172.21.15.19:0/1939559412 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2801d070 con 0x7f1a3806e320 2024-03-28T13:11:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.043+0000 7f1a2ea00640 1 -- 172.21.15.19:0/1939559412 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28017b50 con 0x7f1a3806e320 2024-03-28T13:11:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.043+0000 7f1a3d400640 1 -- 172.21.15.19:0/1939559412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3806d4a0 con 0x7f1a3806e320 2024-03-28T13:11:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.043+0000 7f1a3d400640 1 -- 172.21.15.19:0/1939559412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a38156920 con 0x7f1a3806e320 2024-03-28T13:11:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.044+0000 7f1a3d400640 1 -- 172.21.15.19:0/1939559412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a38068cd0 con 0x7f1a3806e320 2024-03-28T13:11:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.045+0000 7f1a2ea00640 1 -- 172.21.15.19:0/1939559412 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1a28017cf0 con 0x7f1a3806e320 2024-03-28T13:11:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.046+0000 7f1a2ea00640 1 --2- 172.21.15.19:0/1939559412 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a1406c530 0x7f1a1406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.046+0000 7f1a2ea00640 1 -- 172.21.15.19:0/1939559412 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f1a280901a0 con 0x7f1a3806e320 2024-03-28T13:11:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.047+0000 7f1a36800640 1 --2- 172.21.15.19:0/1939559412 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a1406c530 0x7f1a1406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.048+0000 7f1a36800640 1 --2- 172.21.15.19:0/1939559412 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a1406c530 0x7f1a1406e9f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1a20034000 tx=0x7f1a20007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.051+0000 7f1a2ea00640 1 -- 172.21.15.19:0/1939559412 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1a28015260 con 0x7f1a3806e320 2024-03-28T13:11:32.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.250+0000 7f1a3d400640 1 -- 172.21.15.19:0/1939559412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a380ce690 con 0x7f1a3806e320 2024-03-28T13:11:32.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.252+0000 7f1a2ea00640 1 -- 172.21.15.19:0/1939559412 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16858 (secure 0 0 0) 0x7f1a2805e760 con 0x7f1a3806e320 2024-03-28T13:11:32.253 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:32.253 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":38,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:11:23.535441+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:10:54.307571+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-28T13:10:55.205691+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.0","mappings":[{"from":4,"to":5}]},{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.3e","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:11:32.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.255+0000 7f1a3d400640 1 -- 172.21.15.19:0/1939559412 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a1406c530 msgr2=0x7f1a1406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:32.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.255+0000 7f1a3d400640 1 --2- 172.21.15.19:0/1939559412 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a1406c530 0x7f1a1406e9f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1a20034000 tx=0x7f1a20007000 comp rx=0 tx=0).stop 2024-03-28T13:11:32.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.255+0000 7f1a3d400640 1 -- 172.21.15.19:0/1939559412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3806e320 msgr2=0x7f1a381563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:32.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.255+0000 7f1a3d400640 1 --2- 172.21.15.19:0/1939559412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3806e320 0x7f1a381563e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1a28012410 tx=0x7f1a28012810 comp rx=0 tx=0).stop 2024-03-28T13:11:32.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.255+0000 7f1a3d400640 1 -- 172.21.15.19:0/1939559412 shutdown_connections 2024-03-28T13:11:32.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.255+0000 7f1a3d400640 1 --2- 172.21.15.19:0/1939559412 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a1406c530 0x7f1a1406e9f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.255+0000 7f1a3d400640 1 --2- 172.21.15.19:0/1939559412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a381548c0 0x7f1a38157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.255+0000 7f1a3d400640 1 --2- 172.21.15.19:0/1939559412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a3806e320 0x7f1a381563e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.255+0000 7f1a3d400640 1 --2- 172.21.15.19:0/1939559412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1c0039b0 0x7f1a38153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.255+0000 7f1a3d400640 1 -- 172.21.15.19:0/1939559412 >> 172.21.15.19:0/1939559412 conn(0x7f1a380b3e90 msgr2=0x7f1a380d1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:32.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.256+0000 7f1a3d400640 1 -- 172.21.15.19:0/1939559412 shutdown_connections 2024-03-28T13:11:32.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.256+0000 7f1a3d400640 1 -- 172.21.15.19:0/1939559412 wait complete. 2024-03-28T13:11:32.275 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T13:11:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.492+0000 7f5fb2a00640 1 Processor -- start 2024-03-28T13:11:32.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.492+0000 7f5fb2a00640 1 -- start start 2024-03-28T13:11:32.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.493+0000 7f5fb2a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5fac153580 0x7f5fac0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.493+0000 7f5fb2a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac0af4b0 0x7f5fac1600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.493+0000 7f5fb2a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fac0afab0 con 0x7f5fac0d07c0 2024-03-28T13:11:32.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.493+0000 7f5fb2a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5fac0d0c30 con 0x7f5fac153580 2024-03-28T13:11:32.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.493+0000 7f5fb2a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5fac15cc80 con 0x7f5fac0af4b0 2024-03-28T13:11:32.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.494+0000 7f5fa3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5fac0d07c0 0x7f5fac0aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45530/0 (socket says 172.21.15.19:45530) 2024-03-28T13:11:32.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.494+0000 7f5fa3400640 1 -- 172.21.15.19:0/3485438784 learned_addr learned my addr 172.21.15.19:0/3485438784 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.494+0000 7f5fb0a00640 1 --2- 172.21.15.19:0/3485438784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac0af4b0 0x7f5fac1600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.494+0000 7f5fabe00640 1 --2- 172.21.15.19:0/3485438784 >> v2:172.21.15.188:3300/0 conn(0x7f5fac153580 0x7f5fac0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.494+0000 7f5fab400640 1 -- 172.21.15.19:0/3485438784 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1247979939 0 0) 0x7f5fac0afab0 con 0x7f5fac0d07c0 2024-03-28T13:11:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.494+0000 7f5fab400640 1 -- 172.21.15.19:0/3485438784 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f90003520 con 0x7f5fac0d07c0 2024-03-28T13:11:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.495+0000 7f5fab400640 1 -- 172.21.15.19:0/3485438784 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 680066844 0 0) 0x7f5f90003520 con 0x7f5fac0d07c0 2024-03-28T13:11:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.495+0000 7f5fab400640 1 -- 172.21.15.19:0/3485438784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac0af4b0 msgr2=0x7f5fac1600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.495+0000 7f5fab400640 1 --2- 172.21.15.19:0/3485438784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac0af4b0 0x7f5fac1600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.495+0000 7f5fab400640 1 -- 172.21.15.19:0/3485438784 >> v2:172.21.15.188:3300/0 conn(0x7f5fac153580 msgr2=0x7f5fac0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.495+0000 7f5fab400640 1 --2- 172.21.15.19:0/3485438784 >> v2:172.21.15.188:3300/0 conn(0x7f5fac153580 0x7f5fac0d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.495+0000 7f5fab400640 1 -- 172.21.15.19:0/3485438784 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac15da00 con 0x7f5fac0d07c0 2024-03-28T13:11:32.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.495+0000 7f5fab400640 1 -- 172.21.15.19:0/3485438784 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f5f94001670 con 0x7f5fac0d07c0 2024-03-28T13:11:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.495+0000 7f5fab400640 1 -- 172.21.15.19:0/3485438784 >> v1:172.21.15.19:6789/0 conn(0x7f5fac0d07c0 legacy=0x7f5fac0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.495+0000 7f5fab400640 1 --2- 172.21.15.19:0/3485438784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f900039b0 0x7f5f90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.495+0000 7f5fab400640 1 --2- 172.21.15.19:0/3485438784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac0af4b0 0x7f5f90004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.495+0000 7f5fab400640 1 --2- 172.21.15.19:0/3485438784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f90004d40 0x7f5f900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.495+0000 7f5fab400640 1 -- 172.21.15.19:0/3485438784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac0d0c30 con 0x7f5f90004d40 2024-03-28T13:11:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.495+0000 7f5fab400640 1 -- 172.21.15.19:0/3485438784 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac15cc80 con 0x7f5f900039b0 2024-03-28T13:11:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.495+0000 7f5fab400640 1 -- 172.21.15.19:0/3485438784 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5f9000fe00 con 0x7f5fac0af4b0 2024-03-28T13:11:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.495+0000 7f5fb0a00640 1 --2- 172.21.15.19:0/3485438784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f90004d40 0x7f5f900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.496+0000 7f5fa3400640 1 --2- 172.21.15.19:0/3485438784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac0af4b0 0x7f5f90004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.496+0000 7f5fabe00640 1 --2- 172.21.15.19:0/3485438784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f900039b0 0x7f5f90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.496+0000 7f5fb0a00640 1 -- 172.21.15.19:0/3485438784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac0af4b0 msgr2=0x7f5f90004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.496+0000 7f5fb0a00640 1 --2- 172.21.15.19:0/3485438784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac0af4b0 0x7f5f90004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.496+0000 7f5fb0a00640 1 -- 172.21.15.19:0/3485438784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f900039b0 msgr2=0x7f5f90003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.496+0000 7f5fb0a00640 1 --2- 172.21.15.19:0/3485438784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f900039b0 0x7f5f90003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.496+0000 7f5fb0a00640 1 -- 172.21.15.19:0/3485438784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f90003520 con 0x7f5f90004d40 2024-03-28T13:11:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.497+0000 7f5fb0a00640 1 --2- 172.21.15.19:0/3485438784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f90004d40 0x7f5f900051c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009fc0 tx=0x7f5f90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f64d2370abc9d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.497+0000 7f5fab400640 1 -- 172.21.15.19:0/3485438784 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4039070 con 0x7f5f90004d40 2024-03-28T13:11:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.497+0000 7f5fab400640 1 -- 172.21.15.19:0/3485438784 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fa402cc20 con 0x7f5f90004d40 2024-03-28T13:11:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.497+0000 7f5fab400640 1 -- 172.21.15.19:0/3485438784 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4009c10 con 0x7f5f90004d40 2024-03-28T13:11:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.498+0000 7f5fb2a00640 1 -- 172.21.15.19:0/3485438784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f90004d40 msgr2=0x7f5f900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.498+0000 7f5fb2a00640 1 --2- 172.21.15.19:0/3485438784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f90004d40 0x7f5f900051c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009fc0 tx=0x7f5f90005830 comp rx=0 tx=0).stop 2024-03-28T13:11:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.498+0000 7f5fb0a00640 1 -- 172.21.15.19:0/3485438784 reap_dead start 2024-03-28T13:11:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.498+0000 7f5fb2a00640 1 -- 172.21.15.19:0/3485438784 shutdown_connections 2024-03-28T13:11:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.498+0000 7f5fb2a00640 1 -- 172.21.15.19:0/3485438784 >> 172.21.15.19:0/3485438784 conn(0x7f5fac06b750 msgr2=0x7f5fac06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.498+0000 7f5fb2a00640 1 -- 172.21.15.19:0/3485438784 shutdown_connections 2024-03-28T13:11:32.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.498+0000 7f5fb2a00640 1 -- 172.21.15.19:0/3485438784 wait complete. 2024-03-28T13:11:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.500+0000 7f5fb2a00640 1 Processor -- start 2024-03-28T13:11:32.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.500+0000 7f5fb2a00640 1 -- start start 2024-03-28T13:11:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.501+0000 7f5fb2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f90004d40 0x7f5fac154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.501+0000 7f5fb2a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac150010 0x7f5fac14e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.501+0000 7f5fb2a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fac150550 0x7f5fac0b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.501+0000 7f5fb2a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac15d4e0 con 0x7f5f90004d40 2024-03-28T13:11:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.501+0000 7f5fb2a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac15d200 con 0x7f5fac150550 2024-03-28T13:11:32.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.501+0000 7f5fb2a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5fac153960 con 0x7f5fac150010 2024-03-28T13:11:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.501+0000 7f5fabe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f90004d40 0x7f5fac154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.501+0000 7f5fabe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f90004d40 0x7f5fac154780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43604/0 (socket says 172.21.15.19:43604) 2024-03-28T13:11:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.501+0000 7f5fabe00640 1 -- 172.21.15.19:0/1711351877 learned_addr learned my addr 172.21.15.19:0/1711351877 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.501+0000 7f5fa3400640 1 --2- 172.21.15.19:0/1711351877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac150010 0x7f5fac14e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.501+0000 7f5fabe00640 1 -- 172.21.15.19:0/1711351877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac150010 msgr2=0x7f5fac14e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.501+0000 7f5fabe00640 1 --2- 172.21.15.19:0/1711351877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac150010 0x7f5fac14e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.501+0000 7f5fabe00640 1 -- 172.21.15.19:0/1711351877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fac150550 msgr2=0x7f5fac0b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.501+0000 7f5fabe00640 1 --2- 172.21.15.19:0/1711351877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fac150550 0x7f5fac0b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.501+0000 7f5fabe00640 1 -- 172.21.15.19:0/1711351877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac151020 con 0x7f5f90004d40 2024-03-28T13:11:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.502+0000 7f5fabe00640 1 --2- 172.21.15.19:0/1711351877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f90004d40 0x7f5fac154780 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c011630 tx=0x7f5f9c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.502+0000 7f5fa8c00640 1 -- 172.21.15.19:0/1711351877 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c00f920 con 0x7f5f90004d40 2024-03-28T13:11:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.502+0000 7f5fa8c00640 1 -- 172.21.15.19:0/1711351877 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f9c00fac0 con 0x7f5f90004d40 2024-03-28T13:11:32.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.502+0000 7f5fa8c00640 1 -- 172.21.15.19:0/1711351877 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c00fe70 con 0x7f5f90004d40 2024-03-28T13:11:32.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.502+0000 7f5fb2a00640 1 -- 172.21.15.19:0/1711351877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac150d40 con 0x7f5f90004d40 2024-03-28T13:11:32.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.502+0000 7f5fb2a00640 1 -- 172.21.15.19:0/1711351877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac153f80 con 0x7f5f90004d40 2024-03-28T13:11:32.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.504+0000 7f5fb2a00640 1 -- 172.21.15.19:0/1711351877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fac0d0280 con 0x7f5f90004d40 2024-03-28T13:11:32.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.505+0000 7f5fa8c00640 1 -- 172.21.15.19:0/1711351877 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5f9c015460 con 0x7f5f90004d40 2024-03-28T13:11:32.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.505+0000 7f5fa8c00640 1 --2- 172.21.15.19:0/1711351877 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5f8806c530 0x7f5f8806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:32.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.505+0000 7f5fa8c00640 1 -- 172.21.15.19:0/1711351877 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5825+0+0 (secure 0 0 0) 0x7f5f9c08bc50 con 0x7f5f90004d40 2024-03-28T13:11:32.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.506+0000 7f5fa3400640 1 --2- 172.21.15.19:0/1711351877 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5f8806c530 0x7f5f8806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:32.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.507+0000 7f5fa3400640 1 --2- 172.21.15.19:0/1711351877 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5f8806c530 0x7f5f8806e9f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5fac1f38c0 tx=0x7f5f94000f50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:32.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.510+0000 7f5fa8c00640 1 -- 172.21.15.19:0/1711351877 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5fac0d0280 con 0x7f5f90004d40 2024-03-28T13:11:32.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.761+0000 7f5fb2a00640 1 -- 172.21.15.19:0/1711351877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5fac0685c0 con 0x7f5f90004d40 2024-03-28T13:11:32.762 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:11:32.762+0000 7f6b74400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T13:11:32.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.766+0000 7f5fa8c00640 1 -- 172.21.15.19:0/1711351877 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f5f9c091020 con 0x7f5f90004d40 2024-03-28T13:11:32.766 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T13:11:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.769+0000 7f5fb2a00640 1 -- 172.21.15.19:0/1711351877 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5f8806c530 msgr2=0x7f5f8806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.769+0000 7f5fb2a00640 1 --2- 172.21.15.19:0/1711351877 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5f8806c530 0x7f5f8806e9f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5fac1f38c0 tx=0x7f5f94000f50 comp rx=0 tx=0).stop 2024-03-28T13:11:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.769+0000 7f5fb2a00640 1 -- 172.21.15.19:0/1711351877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f90004d40 msgr2=0x7f5fac154780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.769+0000 7f5fb2a00640 1 --2- 172.21.15.19:0/1711351877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f90004d40 0x7f5fac154780 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c011630 tx=0x7f5f9c011af0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.769+0000 7f5fb2a00640 1 -- 172.21.15.19:0/1711351877 shutdown_connections 2024-03-28T13:11:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.769+0000 7f5fb2a00640 1 --2- 172.21.15.19:0/1711351877 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5f8806c530 0x7f5f8806e9f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.769+0000 7f5fb2a00640 1 --2- 172.21.15.19:0/1711351877 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fac150550 0x7f5fac0b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.769+0000 7f5fb2a00640 1 --2- 172.21.15.19:0/1711351877 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fac150010 0x7f5fac14e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.769+0000 7f5fb2a00640 1 --2- 172.21.15.19:0/1711351877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f90004d40 0x7f5fac154780 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.769+0000 7f5fb2a00640 1 -- 172.21.15.19:0/1711351877 >> 172.21.15.19:0/1711351877 conn(0x7f5fac06b750 msgr2=0x7f5fac06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.770+0000 7f5fb2a00640 1 -- 172.21.15.19:0/1711351877 shutdown_connections 2024-03-28T13:11:32.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:32.770+0000 7f5fb2a00640 1 -- 172.21.15.19:0/1711351877 wait complete. 2024-03-28T13:11:32.789 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-28T13:11:32.801 INFO:tasks.ceph.mds.b.smithi188.stderr: -6998> 2024-03-28T13:10:59.602+0000 7f7460000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:11:32.874 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-03-28T13:11:33.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.006+0000 7f4240c00640 1 Processor -- start 2024-03-28T13:11:33.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.006+0000 7f4240c00640 1 -- start start 2024-03-28T13:11:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.008+0000 7f4240c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f423c0c7810 0x7f423c0b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.008+0000 7f4240c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f423c0d01a0 con 0x7f423c0d1d70 2024-03-28T13:11:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.008+0000 7f4240c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f423c0d0480 con 0x7f423c15e7b0 2024-03-28T13:11:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.008+0000 7f4240c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f423c0d1240 con 0x7f423c0c7810 2024-03-28T13:11:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.008+0000 7f423a800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f423c0d1d70 0x7f423c06ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45536/0 (socket says 172.21.15.19:45536) 2024-03-28T13:11:33.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.008+0000 7f423a800640 1 -- 172.21.15.19:0/1177429984 learned_addr learned my addr 172.21.15.19:0/1177429984 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:33.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.008+0000 7f4239400640 1 --2- 172.21.15.19:0/1177429984 >> v2:172.21.15.188:3300/0 conn(0x7f423c0c7810 0x7f423c0b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:33.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.009+0000 7f4238a00640 1 -- 172.21.15.19:0/1177429984 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2056281437 0 0) 0x7f423c0d01a0 con 0x7f423c0d1d70 2024-03-28T13:11:33.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.009+0000 7f4238a00640 1 -- 172.21.15.19:0/1177429984 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4220003520 con 0x7f423c0d1d70 2024-03-28T13:11:33.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.009+0000 7f4238a00640 1 -- 172.21.15.19:0/1177429984 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3020529454 0 0) 0x7f4220003520 con 0x7f423c0d1d70 2024-03-28T13:11:33.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.009+0000 7f4238a00640 1 -- 172.21.15.19:0/1177429984 >> v2:172.21.15.188:3300/0 conn(0x7f423c0c7810 msgr2=0x7f423c0b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:33.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.009+0000 7f4238a00640 1 --2- 172.21.15.19:0/1177429984 >> v2:172.21.15.188:3300/0 conn(0x7f423c0c7810 0x7f423c0b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.009+0000 7f4238a00640 1 -- 172.21.15.19:0/1177429984 >> v1:172.21.15.188:6789/0 conn(0x7f423c15e7b0 legacy=0x7f423c0c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.009+0000 7f4238a00640 1 -- 172.21.15.19:0/1177429984 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c0d0760 con 0x7f423c0d1d70 2024-03-28T13:11:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.009+0000 7f4238a00640 1 -- 172.21.15.19:0/1177429984 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f4234001670 con 0x7f423c0d1d70 2024-03-28T13:11:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.009+0000 7f4238a00640 1 -- 172.21.15.19:0/1177429984 >> v1:172.21.15.19:6789/0 conn(0x7f423c0d1d70 legacy=0x7f423c06ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.009+0000 7f4238a00640 1 --2- 172.21.15.19:0/1177429984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42200039b0 0x7f4220003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.009+0000 7f4238a00640 1 --2- 172.21.15.19:0/1177429984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4220004800 0x7f4220004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.009+0000 7f4238a00640 1 --2- 172.21.15.19:0/1177429984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42200051c0 0x7f422000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.009+0000 7f4238a00640 1 -- 172.21.15.19:0/1177429984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f423c0d1240 con 0x7f42200051c0 2024-03-28T13:11:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.009+0000 7f4238a00640 1 -- 172.21.15.19:0/1177429984 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f422000c060 con 0x7f42200039b0 2024-03-28T13:11:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.009+0000 7f4238a00640 1 -- 172.21.15.19:0/1177429984 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f422000c260 con 0x7f4220004800 2024-03-28T13:11:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.010+0000 7f423a800640 1 --2- 172.21.15.19:0/1177429984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42200051c0 0x7f422000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.010+0000 7f4239400640 1 --2- 172.21.15.19:0/1177429984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4220004800 0x7f4220004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.010+0000 7f423a800640 1 -- 172.21.15.19:0/1177429984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4220004800 msgr2=0x7f4220004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:33.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.010+0000 7f423a800640 1 --2- 172.21.15.19:0/1177429984 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4220004800 0x7f4220004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:33.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.010+0000 7f423a800640 1 -- 172.21.15.19:0/1177429984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42200039b0 msgr2=0x7f4220003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:33.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.010+0000 7f4239e00640 1 --2- 172.21.15.19:0/1177429984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42200039b0 0x7f4220003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:33.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.010+0000 7f423a800640 1 --2- 172.21.15.19:0/1177429984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42200039b0 0x7f4220003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:33.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.010+0000 7f423a800640 1 -- 172.21.15.19:0/1177429984 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4220003520 con 0x7f42200051c0 2024-03-28T13:11:33.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.010+0000 7f4239e00640 1 --2- 172.21.15.19:0/1177429984 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42200039b0 0x7f4220003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:33.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.011+0000 7f423a800640 1 -- 172.21.15.19:0/1177429984 reap_dead start 2024-03-28T13:11:33.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.011+0000 7f423a800640 1 --2- 172.21.15.19:0/1177429984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42200051c0 0x7f422000b9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f423400c210 tx=0x7f4220005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86783a1c647f7591 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:33.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.012+0000 7f4238a00640 1 -- 172.21.15.19:0/1177429984 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234007d10 con 0x7f42200051c0 2024-03-28T13:11:33.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.012+0000 7f4238a00640 1 -- 172.21.15.19:0/1177429984 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4234030070 con 0x7f42200051c0 2024-03-28T13:11:33.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.012+0000 7f4238a00640 1 -- 172.21.15.19:0/1177429984 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234005cf0 con 0x7f42200051c0 2024-03-28T13:11:33.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.012+0000 7f4240c00640 1 -- 172.21.15.19:0/1177429984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42200051c0 msgr2=0x7f422000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:33.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.012+0000 7f4240c00640 1 --2- 172.21.15.19:0/1177429984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42200051c0 0x7f422000b9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f423400c210 tx=0x7f4220005830 comp rx=0 tx=0).stop 2024-03-28T13:11:33.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.012+0000 7f4240c00640 1 -- 172.21.15.19:0/1177429984 shutdown_connections 2024-03-28T13:11:33.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.012+0000 7f4240c00640 1 --2- 172.21.15.19:0/1177429984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42200051c0 0x7f422000b9c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:33.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.012+0000 7f4240c00640 1 -- 172.21.15.19:0/1177429984 >> 172.21.15.19:0/1177429984 conn(0x7f423c0b3e90 msgr2=0x7f423c0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:33.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.013+0000 7f4240c00640 1 -- 172.21.15.19:0/1177429984 shutdown_connections 2024-03-28T13:11:33.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.013+0000 7f4240c00640 1 -- 172.21.15.19:0/1177429984 wait complete. 2024-03-28T13:11:33.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.014+0000 7f4240c00640 1 Processor -- start 2024-03-28T13:11:33.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.014+0000 7f4240c00640 1 -- start start 2024-03-28T13:11:33.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.015+0000 7f4240c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42200051c0 0x7f423c14eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:33.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.015+0000 7f4240c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f423c0b7550 0x7f423c0b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:33.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.015+0000 7f4240c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c0b4f90 0x7f423c0b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:33.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.015+0000 7f4240c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f423c0cfe30 con 0x7f423c0b7550 2024-03-28T13:11:33.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.015+0000 7f4240c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f423c0d0d20 con 0x7f423c0b4f90 2024-03-28T13:11:33.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.015+0000 7f4240c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f423c0b4980 con 0x7f42200051c0 2024-03-28T13:11:33.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.016+0000 7f4239400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f423c0b7550 0x7f423c0b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:33.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.016+0000 7f4239400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f423c0b7550 0x7f423c0b79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43618/0 (socket says 172.21.15.19:43618) 2024-03-28T13:11:33.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.016+0000 7f4239400640 1 -- 172.21.15.19:0/1812343719 learned_addr learned my addr 172.21.15.19:0/1812343719 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:33.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.016+0000 7f4239400640 1 -- 172.21.15.19:0/1812343719 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42200051c0 msgr2=0x7f423c14eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:33.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.016+0000 7f4239400640 1 --2- 172.21.15.19:0/1812343719 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42200051c0 0x7f423c14eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:33.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.016+0000 7f4239400640 1 -- 172.21.15.19:0/1812343719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c0b4f90 msgr2=0x7f423c0b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:33.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.016+0000 7f4239400640 1 --2- 172.21.15.19:0/1812343719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c0b4f90 0x7f423c0b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:33.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.016+0000 7f4239400640 1 -- 172.21.15.19:0/1812343719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c0cb6c0 con 0x7f423c0b7550 2024-03-28T13:11:33.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.016+0000 7f4239400640 1 --2- 172.21.15.19:0/1812343719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f423c0b7550 0x7f423c0b79b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4224011540 tx=0x7f4224011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:33.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.017+0000 7f4232000640 1 -- 172.21.15.19:0/1812343719 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4224015930 con 0x7f423c0b7550 2024-03-28T13:11:33.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.017+0000 7f4232000640 1 -- 172.21.15.19:0/1812343719 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4224015ad0 con 0x7f423c0b7550 2024-03-28T13:11:33.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.017+0000 7f4240c00640 1 -- 172.21.15.19:0/1812343719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423c0cb9a0 con 0x7f423c0b7550 2024-03-28T13:11:33.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.017+0000 7f4232000640 1 -- 172.21.15.19:0/1812343719 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422400f460 con 0x7f423c0b7550 2024-03-28T13:11:33.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.017+0000 7f4240c00640 1 -- 172.21.15.19:0/1812343719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423c0cbf60 con 0x7f423c0b7550 2024-03-28T13:11:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.018+0000 7f4240c00640 1 -- 172.21.15.19:0/1812343719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f423c068cd0 con 0x7f423c0b7550 2024-03-28T13:11:33.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.019+0000 7f4232000640 1 -- 172.21.15.19:0/1812343719 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f422400f600 con 0x7f423c0b7550 2024-03-28T13:11:33.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.020+0000 7f4232000640 1 --2- 172.21.15.19:0/1812343719 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f421806c560 0x7f421806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:33.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.020+0000 7f4232000640 1 -- 172.21.15.19:0/1812343719 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5911+0+0 (secure 0 0 0) 0x7f422408c0f0 con 0x7f423c0b7550 2024-03-28T13:11:33.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.020+0000 7f4239e00640 1 --2- 172.21.15.19:0/1812343719 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f421806c560 0x7f421806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:33.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.021+0000 7f4239e00640 1 --2- 172.21.15.19:0/1812343719 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f421806c560 0x7f421806ea20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f422c001da0 tx=0x7f422c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:33.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.025+0000 7f4232000640 1 -- 172.21.15.19:0/1812343719 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4224019030 con 0x7f423c0b7550 2024-03-28T13:11:33.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.269+0000 7f4240c00640 1 -- 172.21.15.19:0/1812343719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f423c15cbf0 con 0x7f423c0b7550 2024-03-28T13:11:33.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.770+0000 7f4232000640 1 -- 172.21.15.19:0/1812343719 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f422405a5a0 con 0x7f423c0b7550 2024-03-28T13:11:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.774+0000 7f4240c00640 1 -- 172.21.15.19:0/1812343719 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f421806c560 msgr2=0x7f421806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.774+0000 7f4240c00640 1 --2- 172.21.15.19:0/1812343719 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f421806c560 0x7f421806ea20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f422c001da0 tx=0x7f422c009040 comp rx=0 tx=0).stop 2024-03-28T13:11:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.774+0000 7f4240c00640 1 -- 172.21.15.19:0/1812343719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f423c0b7550 msgr2=0x7f423c0b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.774+0000 7f4240c00640 1 --2- 172.21.15.19:0/1812343719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f423c0b7550 0x7f423c0b79b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4224011540 tx=0x7f4224011a00 comp rx=0 tx=0).stop 2024-03-28T13:11:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.774+0000 7f4240c00640 1 -- 172.21.15.19:0/1812343719 shutdown_connections 2024-03-28T13:11:33.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.774+0000 7f4240c00640 1 --2- 172.21.15.19:0/1812343719 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f421806c560 0x7f421806ea20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:33.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.774+0000 7f4240c00640 1 --2- 172.21.15.19:0/1812343719 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c0b4f90 0x7f423c0b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:33.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.774+0000 7f4240c00640 1 --2- 172.21.15.19:0/1812343719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f423c0b7550 0x7f423c0b79b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:33.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.774+0000 7f4240c00640 1 --2- 172.21.15.19:0/1812343719 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42200051c0 0x7f423c14eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:33.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.774+0000 7f4240c00640 1 -- 172.21.15.19:0/1812343719 >> 172.21.15.19:0/1812343719 conn(0x7f423c0b3e90 msgr2=0x7f423c15d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:33.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.774+0000 7f4240c00640 1 -- 172.21.15.19:0/1812343719 shutdown_connections 2024-03-28T13:11:33.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:33.774+0000 7f4240c00640 1 -- 172.21.15.19:0/1812343719 wait complete. 2024-03-28T13:11:33.799 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-28T13:11:34.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.029+0000 7fbbe3000640 1 Processor -- start 2024-03-28T13:11:34.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.029+0000 7fbbe3000640 1 -- start start 2024-03-28T13:11:34.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.032+0000 7fbbe3000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbdc0b0c50 0x7fbbdc0cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:34.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.032+0000 7fbbe3000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc0b2160 0x7fbbdc0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.032+0000 7fbbe3000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbdc0d0010 con 0x7fbbdc0b1030 2024-03-28T13:11:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.032+0000 7fbbe3000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbbdc0cf790 con 0x7fbbdc0b0c50 2024-03-28T13:11:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.032+0000 7fbbe3000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbbdc0cfa50 con 0x7fbbdc0b2160 2024-03-28T13:11:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.032+0000 7fbbe1000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbbdc0b1030 0x7fbbdc0af9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45546/0 (socket says 172.21.15.19:45546) 2024-03-28T13:11:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.032+0000 7fbbdbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbdc0b0c50 0x7fbbdc0cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.032+0000 7fbbe1000640 1 -- 172.21.15.19:0/275964295 learned_addr learned my addr 172.21.15.19:0/275964295 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:34.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.032+0000 7fbbd3e00640 1 --2- 172.21.15.19:0/275964295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc0b2160 0x7fbbdc0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.032+0000 7fbbdb400640 1 -- 172.21.15.19:0/275964295 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3203583521 0 0) 0x7fbbd4001490 con 0x7fbbdc0b1030 2024-03-28T13:11:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.032+0000 7fbbdbe00640 1 -- 172.21.15.19:0/275964295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc0b2160 msgr2=0x7fbbdc0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.032+0000 7fbbdbe00640 1 --2- 172.21.15.19:0/275964295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc0b2160 0x7fbbdc0b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.032+0000 7fbbdbe00640 1 -- 172.21.15.19:0/275964295 >> v1:172.21.15.19:6789/0 conn(0x7fbbdc0b1030 legacy=0x7fbbdc0af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:34.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.032+0000 7fbbdbe00640 1 -- 172.21.15.19:0/275964295 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbdc0cfd30 con 0x7fbbdc0b0c50 2024-03-28T13:11:34.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.034+0000 7fbbdbe00640 1 --2- 172.21.15.19:0/275964295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbdc0b0c50 0x7fbbdc0cbb90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0097a0 tx=0x7fbbcc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1c14e7c8d12293 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:34.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.034+0000 7fbbdb400640 1 -- 172.21.15.19:0/275964295 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc035020 con 0x7fbbdc0b0c50 2024-03-28T13:11:34.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.034+0000 7fbbdb400640 1 -- 172.21.15.19:0/275964295 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbcc031830 con 0x7fbbdc0b0c50 2024-03-28T13:11:34.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.034+0000 7fbbdb400640 1 -- 172.21.15.19:0/275964295 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc031c00 con 0x7fbbdc0b0c50 2024-03-28T13:11:34.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.034+0000 7fbbe3000640 1 -- 172.21.15.19:0/275964295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbdc0b0c50 msgr2=0x7fbbdc0cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:34.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.034+0000 7fbbe3000640 1 --2- 172.21.15.19:0/275964295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbdc0b0c50 0x7fbbdc0cbb90 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0097a0 tx=0x7fbbcc02b8d0 comp rx=0 tx=0).stop 2024-03-28T13:11:34.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.035+0000 7fbbe3000640 1 -- 172.21.15.19:0/275964295 shutdown_connections 2024-03-28T13:11:34.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.035+0000 7fbbe3000640 1 --2- 172.21.15.19:0/275964295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc0b2160 0x7fbbdc0b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:34.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.035+0000 7fbbe3000640 1 --2- 172.21.15.19:0/275964295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbdc0b0c50 0x7fbbdc0cbb90 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:34.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.035+0000 7fbbe3000640 1 -- 172.21.15.19:0/275964295 >> 172.21.15.19:0/275964295 conn(0x7fbbdc0b2d40 msgr2=0x7fbbdc0b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:34.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.035+0000 7fbbe3000640 1 -- 172.21.15.19:0/275964295 shutdown_connections 2024-03-28T13:11:34.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.035+0000 7fbbe3000640 1 -- 172.21.15.19:0/275964295 wait complete. 2024-03-28T13:11:34.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.037+0000 7fbbe3000640 1 Processor -- start 2024-03-28T13:11:34.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.037+0000 7fbbe3000640 1 -- start start 2024-03-28T13:11:34.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.037+0000 7fbbe3000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbdc0b0c50 0x7fbbdc0c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:34.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.037+0000 7fbbe3000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc0b1030 0x7fbbdc0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:34.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.038+0000 7fbbe3000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbdc0b2160 0x7fbbdc0ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:34.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.038+0000 7fbbe3000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc0b00e0 con 0x7fbbdc0b2160 2024-03-28T13:11:34.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.038+0000 7fbbe3000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc0af4b0 con 0x7fbbdc0b0c50 2024-03-28T13:11:34.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.038+0000 7fbbe3000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbbdc0b0260 con 0x7fbbdc0b1030 2024-03-28T13:11:34.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.038+0000 7fbbe1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbdc0b2160 0x7fbbdc0ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.038+0000 7fbbe1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbdc0b2160 0x7fbbdc0ca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43634/0 (socket says 172.21.15.19:43634) 2024-03-28T13:11:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.038+0000 7fbbe1000640 1 -- 172.21.15.19:0/509710190 learned_addr learned my addr 172.21.15.19:0/509710190 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.038+0000 7fbbdbe00640 1 --2- 172.21.15.19:0/509710190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbdc0b0c50 0x7fbbdc0c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.038+0000 7fbbd3e00640 1 --2- 172.21.15.19:0/509710190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc0b1030 0x7fbbdc0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.039+0000 7fbbe1000640 1 -- 172.21.15.19:0/509710190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc0b1030 msgr2=0x7fbbdc0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.039+0000 7fbbe1000640 1 --2- 172.21.15.19:0/509710190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc0b1030 0x7fbbdc0c9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.039+0000 7fbbe1000640 1 -- 172.21.15.19:0/509710190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbdc0b0c50 msgr2=0x7fbbdc0c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.039+0000 7fbbe1000640 1 --2- 172.21.15.19:0/509710190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbdc0b0c50 0x7fbbdc0c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.039+0000 7fbbe1000640 1 -- 172.21.15.19:0/509710190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbdc0c9060 con 0x7fbbdc0b2160 2024-03-28T13:11:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.039+0000 7fbbe1000640 1 --2- 172.21.15.19:0/509710190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbdc0b2160 0x7fbbdc0ca190 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400c7f0 tx=0x7fbbd400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.039+0000 7fbbd8c00640 1 -- 172.21.15.19:0/509710190 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4018070 con 0x7fbbdc0b2160 2024-03-28T13:11:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.039+0000 7fbbd8c00640 1 -- 172.21.15.19:0/509710190 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd400a5e0 con 0x7fbbdc0b2160 2024-03-28T13:11:34.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.039+0000 7fbbe3000640 1 -- 172.21.15.19:0/509710190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbdc0c4e80 con 0x7fbbdc0b2160 2024-03-28T13:11:34.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.039+0000 7fbbd8c00640 1 -- 172.21.15.19:0/509710190 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400aa10 con 0x7fbbdc0b2160 2024-03-28T13:11:34.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.039+0000 7fbbe3000640 1 -- 172.21.15.19:0/509710190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbdc0c7cc0 con 0x7fbbdc0b2160 2024-03-28T13:11:34.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.041+0000 7fbbe3000640 1 -- 172.21.15.19:0/509710190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbdc0685c0 con 0x7fbbdc0b2160 2024-03-28T13:11:34.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.043+0000 7fbbd8c00640 1 -- 172.21.15.19:0/509710190 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbbd401b070 con 0x7fbbdc0b2160 2024-03-28T13:11:34.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.043+0000 7fbbd8c00640 1 --2- 172.21.15.19:0/509710190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbbb806c560 0x7fbbb806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:34.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.043+0000 7fbbd8c00640 1 -- 172.21.15.19:0/509710190 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5911+0+0 (secure 0 0 0) 0x7fbbd408ada0 con 0x7fbbdc0b2160 2024-03-28T13:11:34.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.044+0000 7fbbdbe00640 1 --2- 172.21.15.19:0/509710190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbbb806c560 0x7fbbb806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:34.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.045+0000 7fbbdbe00640 1 --2- 172.21.15.19:0/509710190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbbb806c560 0x7fbbb806ea20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc036000 tx=0x7fbbcc03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:34.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.048+0000 7fbbd8c00640 1 -- 172.21.15.19:0/509710190 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbbdc0d0010 con 0x7fbbdc0b2160 2024-03-28T13:11:34.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.251+0000 7fbbe3000640 1 -- 172.21.15.19:0/509710190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbbdc0ccc30 con 0x7fbbdc0b2160 2024-03-28T13:11:34.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.779+0000 7fbbd8c00640 1 -- 172.21.15.19:0/509710190 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v40) ==== 181+0+0 (secure 0 0 0) 0x7fbbd4058c00 con 0x7fbbdc0b2160 2024-03-28T13:11:34.779 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-03-28T13:11:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.783+0000 7fbbe3000640 1 -- 172.21.15.19:0/509710190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbbb806c560 msgr2=0x7fbbb806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.783+0000 7fbbe3000640 1 --2- 172.21.15.19:0/509710190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbbb806c560 0x7fbbb806ea20 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc036000 tx=0x7fbbcc03d040 comp rx=0 tx=0).stop 2024-03-28T13:11:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.783+0000 7fbbe3000640 1 -- 172.21.15.19:0/509710190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbdc0b2160 msgr2=0x7fbbdc0ca190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.783+0000 7fbbe3000640 1 --2- 172.21.15.19:0/509710190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbdc0b2160 0x7fbbdc0ca190 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400c7f0 tx=0x7fbbd400ccb0 comp rx=0 tx=0).stop 2024-03-28T13:11:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.784+0000 7fbbe3000640 1 -- 172.21.15.19:0/509710190 shutdown_connections 2024-03-28T13:11:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.784+0000 7fbbe3000640 1 --2- 172.21.15.19:0/509710190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbbb806c560 0x7fbbb806ea20 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.784+0000 7fbbe3000640 1 --2- 172.21.15.19:0/509710190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbdc0b2160 0x7fbbdc0ca190 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.784+0000 7fbbe3000640 1 --2- 172.21.15.19:0/509710190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc0b1030 0x7fbbdc0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.784+0000 7fbbe3000640 1 --2- 172.21.15.19:0/509710190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbdc0b0c50 0x7fbbdc0c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:34.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.784+0000 7fbbe3000640 1 -- 172.21.15.19:0/509710190 >> 172.21.15.19:0/509710190 conn(0x7fbbdc0b2d40 msgr2=0x7fbbdc06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.784+0000 7fbbe3000640 1 -- 172.21.15.19:0/509710190 shutdown_connections 2024-03-28T13:11:34.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:34.784+0000 7fbbe3000640 1 -- 172.21.15.19:0/509710190 wait complete. 2024-03-28T13:11:34.811 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-28T13:11:35.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.033+0000 7f2f19800640 1 Processor -- start 2024-03-28T13:11:35.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.033+0000 7f2f19800640 1 -- start start 2024-03-28T13:11:35.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.034+0000 7f2f19800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f1415e650 0x7f2f140ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:35.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.034+0000 7f2f19800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f140d0550 con 0x7f2f140b3450 2024-03-28T13:11:35.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.034+0000 7f2f19800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f140d0830 con 0x7f2f14070560 2024-03-28T13:11:35.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.034+0000 7f2f19800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2f140d0320 con 0x7f2f1415e650 2024-03-28T13:11:35.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.035+0000 7f2f12000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2f140b3450 0x7f2f1406fe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45556/0 (socket says 172.21.15.19:45556) 2024-03-28T13:11:35.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.035+0000 7f2f12000640 1 -- 172.21.15.19:0/1378926808 learned_addr learned my addr 172.21.15.19:0/1378926808 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:35.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.035+0000 7f2f12a00640 1 --2- 172.21.15.19:0/1378926808 >> v2:172.21.15.19:3300/0 conn(0x7f2f1415e650 0x7f2f140ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:35.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.035+0000 7f2f11600640 1 -- 172.21.15.19:0/1378926808 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2778015786 0 0) 0x7f2f140d0550 con 0x7f2f140b3450 2024-03-28T13:11:35.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.035+0000 7f2f11600640 1 -- 172.21.15.19:0/1378926808 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ef8003520 con 0x7f2f140b3450 2024-03-28T13:11:35.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.035+0000 7f2f12a00640 1 -- 172.21.15.19:0/1378926808 >> v1:172.21.15.188:6789/0 conn(0x7f2f14070560 legacy=0x7f2f140cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:35.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.035+0000 7f2f12a00640 1 -- 172.21.15.19:0/1378926808 >> v1:172.21.15.19:6789/0 conn(0x7f2f140b3450 legacy=0x7f2f1406fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:35.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.035+0000 7f2f12a00640 1 -- 172.21.15.19:0/1378926808 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f140d0b10 con 0x7f2f1415e650 2024-03-28T13:11:35.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.037+0000 7f2f12a00640 1 --2- 172.21.15.19:0/1378926808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f1415e650 0x7f2f140ae020 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2f040097a0 tx=0x7f2f0402b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e25bf28a7d69af server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:35.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.037+0000 7f2f11600640 1 -- 172.21.15.19:0/1378926808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0402f810 con 0x7f2f1415e650 2024-03-28T13:11:35.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.037+0000 7f2f11600640 1 -- 172.21.15.19:0/1378926808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0402f9b0 con 0x7f2f1415e650 2024-03-28T13:11:35.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.037+0000 7f2f11600640 1 -- 172.21.15.19:0/1378926808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0402fd80 con 0x7f2f1415e650 2024-03-28T13:11:35.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.037+0000 7f2f19800640 1 -- 172.21.15.19:0/1378926808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f1415e650 msgr2=0x7f2f140ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:35.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.037+0000 7f2f19800640 1 --2- 172.21.15.19:0/1378926808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f1415e650 0x7f2f140ae020 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2f040097a0 tx=0x7f2f0402b9f0 comp rx=0 tx=0).stop 2024-03-28T13:11:35.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.038+0000 7f2f19800640 1 -- 172.21.15.19:0/1378926808 shutdown_connections 2024-03-28T13:11:35.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.038+0000 7f2f19800640 1 --2- 172.21.15.19:0/1378926808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f1415e650 0x7f2f140ae020 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:35.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.038+0000 7f2f19800640 1 -- 172.21.15.19:0/1378926808 >> 172.21.15.19:0/1378926808 conn(0x7f2f1409aa20 msgr2=0x7f2f140b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:35.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.038+0000 7f2f19800640 1 -- 172.21.15.19:0/1378926808 shutdown_connections 2024-03-28T13:11:35.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.038+0000 7f2f19800640 1 -- 172.21.15.19:0/1378926808 wait complete. 2024-03-28T13:11:35.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.039+0000 7f2f19800640 1 Processor -- start 2024-03-28T13:11:35.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.040+0000 7f2f19800640 1 -- start start 2024-03-28T13:11:35.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.040+0000 7f2f19800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f14070560 0x7f2f14151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:35.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.041+0000 7f2f19800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f140b3450 0x7f2f141591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:35.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.041+0000 7f2f19800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f1415e650 0x7f2f140c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:35.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.041+0000 7f2f19800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1415ea30 con 0x7f2f1415e650 2024-03-28T13:11:35.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.041+0000 7f2f19800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2f140cd830 con 0x7f2f14070560 2024-03-28T13:11:35.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.041+0000 7f2f19800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2f140c65d0 con 0x7f2f140b3450 2024-03-28T13:11:35.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.041+0000 7f2f13400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f1415e650 0x7f2f140c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:35.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.041+0000 7f2f12a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f14070560 0x7f2f14151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:35.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.041+0000 7f2f12000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f140b3450 0x7f2f141591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:35.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.041+0000 7f2f13400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f1415e650 0x7f2f140c5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43662/0 (socket says 172.21.15.19:43662) 2024-03-28T13:11:35.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.041+0000 7f2f13400640 1 -- 172.21.15.19:0/1640824206 learned_addr learned my addr 172.21.15.19:0/1640824206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:35.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.042+0000 7f2f13400640 1 -- 172.21.15.19:0/1640824206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f140b3450 msgr2=0x7f2f141591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:35.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.042+0000 7f2f13400640 1 --2- 172.21.15.19:0/1640824206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f140b3450 0x7f2f141591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:35.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.042+0000 7f2f13400640 1 -- 172.21.15.19:0/1640824206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f14070560 msgr2=0x7f2f14151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.042+0000 7f2f13400640 1 --2- 172.21.15.19:0/1640824206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f14070560 0x7f2f14151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.042+0000 7f2f13400640 1 -- 172.21.15.19:0/1640824206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f140c8370 con 0x7f2f1415e650 2024-03-28T13:11:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.042+0000 7f2f13400640 1 --2- 172.21.15.19:0/1640824206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f1415e650 0x7f2f140c5b50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00c400 tx=0x7f2f0c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.042+0000 7f2f0aa00640 1 -- 172.21.15.19:0/1640824206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c017070 con 0x7f2f1415e650 2024-03-28T13:11:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.042+0000 7f2f0aa00640 1 -- 172.21.15.19:0/1640824206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0c0052d0 con 0x7f2f1415e650 2024-03-28T13:11:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.042+0000 7f2f0aa00640 1 -- 172.21.15.19:0/1640824206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c003810 con 0x7f2f1415e650 2024-03-28T13:11:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.043+0000 7f2f19800640 1 -- 172.21.15.19:0/1640824206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f140c8650 con 0x7f2f1415e650 2024-03-28T13:11:35.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.043+0000 7f2f19800640 1 -- 172.21.15.19:0/1640824206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f14159710 con 0x7f2f1415e650 2024-03-28T13:11:35.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.044+0000 7f2f19800640 1 -- 172.21.15.19:0/1640824206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f14068cd0 con 0x7f2f1415e650 2024-03-28T13:11:35.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.045+0000 7f2f0aa00640 1 -- 172.21.15.19:0/1640824206 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2f0c01a070 con 0x7f2f1415e650 2024-03-28T13:11:35.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.046+0000 7f2f0aa00640 1 --2- 172.21.15.19:0/1640824206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ef406c530 0x7f2ef406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:35.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.046+0000 7f2f0aa00640 1 -- 172.21.15.19:0/1640824206 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5473+0+0 (secure 0 0 0) 0x7f2f0c08a930 con 0x7f2f1415e650 2024-03-28T13:11:35.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.046+0000 7f2f12a00640 1 --2- 172.21.15.19:0/1640824206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ef406c530 0x7f2ef406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:35.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.048+0000 7f2f12a00640 1 --2- 172.21.15.19:0/1640824206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ef406c530 0x7f2ef406e9f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2f04036000 tx=0x7f2f04008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:35.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.051+0000 7f2f0aa00640 1 -- 172.21.15.19:0/1640824206 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2f0c0589d0 con 0x7f2f1415e650 2024-03-28T13:11:35.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.253+0000 7f2f19800640 1 -- 172.21.15.19:0/1640824206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2f1406fff0 con 0x7f2f1415e650 2024-03-28T13:11:35.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.783+0000 7f2f0aa00640 1 -- 172.21.15.19:0/1640824206 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) ==== 169+0+0 (secure 0 0 0) 0x7f2f0c05ccd0 con 0x7f2f1415e650 2024-03-28T13:11:35.783 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-03-28T13:11:35.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.787+0000 7f2f19800640 1 -- 172.21.15.19:0/1640824206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ef406c530 msgr2=0x7f2ef406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:35.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.787+0000 7f2f19800640 1 --2- 172.21.15.19:0/1640824206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ef406c530 0x7f2ef406e9f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2f04036000 tx=0x7f2f04008030 comp rx=0 tx=0).stop 2024-03-28T13:11:35.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.787+0000 7f2f19800640 1 -- 172.21.15.19:0/1640824206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f1415e650 msgr2=0x7f2f140c5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:35.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.787+0000 7f2f19800640 1 --2- 172.21.15.19:0/1640824206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f1415e650 0x7f2f140c5b50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00c400 tx=0x7f2f0c00c8c0 comp rx=0 tx=0).stop 2024-03-28T13:11:35.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.787+0000 7f2f19800640 1 -- 172.21.15.19:0/1640824206 shutdown_connections 2024-03-28T13:11:35.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.787+0000 7f2f19800640 1 --2- 172.21.15.19:0/1640824206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ef406c530 0x7f2ef406e9f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:35.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.787+0000 7f2f19800640 1 --2- 172.21.15.19:0/1640824206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f1415e650 0x7f2f140c5b50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:35.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.787+0000 7f2f19800640 1 --2- 172.21.15.19:0/1640824206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f140b3450 0x7f2f141591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:35.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.787+0000 7f2f19800640 1 --2- 172.21.15.19:0/1640824206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f14070560 0x7f2f14151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:35.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.787+0000 7f2f19800640 1 -- 172.21.15.19:0/1640824206 >> 172.21.15.19:0/1640824206 conn(0x7f2f1409aa20 msgr2=0x7f2f140afcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:35.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.788+0000 7f2f19800640 1 -- 172.21.15.19:0/1640824206 shutdown_connections 2024-03-28T13:11:35.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:35.788+0000 7f2f19800640 1 -- 172.21.15.19:0/1640824206 wait complete. 2024-03-28T13:11:35.818 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-28T13:11:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.020+0000 7efde5800640 1 Processor -- start 2024-03-28T13:11:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.021+0000 7efde5800640 1 -- start start 2024-03-28T13:11:36.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.022+0000 7efde5800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efde0153580 0x7efde00d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:36.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.022+0000 7efde5800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efde00d07c0 0x7efde00aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:36.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.022+0000 7efde5800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efde00af2e0 0x7efde006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:36.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.022+0000 7efde5800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7efde00d0c00 con 0x7efde0153580 2024-03-28T13:11:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.022+0000 7efde5800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7efde00afab0 con 0x7efde00af2e0 2024-03-28T13:11:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.022+0000 7efde5800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efde015cc80 con 0x7efde00d07c0 2024-03-28T13:11:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.023+0000 7efddea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efde0153580 0x7efde00d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.023+0000 7efdd7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efde00d07c0 0x7efde00aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.023+0000 7efddea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efde0153580 0x7efde00d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44224/0 (socket says 172.21.15.19:44224) 2024-03-28T13:11:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.023+0000 7efddea00640 1 -- 172.21.15.19:0/290645436 learned_addr learned my addr 172.21.15.19:0/290645436 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.023+0000 7efddf400640 1 --2- 172.21.15.19:0/290645436 >> v2:172.21.15.188:3300/0 conn(0x7efde00af2e0 0x7efde006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.024+0000 7efddea00640 1 -- 172.21.15.19:0/290645436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efde00d07c0 msgr2=0x7efde00aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.024+0000 7efddea00640 1 --2- 172.21.15.19:0/290645436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efde00d07c0 0x7efde00aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.024+0000 7efddea00640 1 -- 172.21.15.19:0/290645436 >> v2:172.21.15.188:3300/0 conn(0x7efde00af2e0 msgr2=0x7efde006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.024+0000 7efddea00640 1 --2- 172.21.15.19:0/290645436 >> v2:172.21.15.188:3300/0 conn(0x7efde00af2e0 0x7efde006d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.024+0000 7efddea00640 1 -- 172.21.15.19:0/290645436 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde015da00 con 0x7efde0153580 2024-03-28T13:11:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.025+0000 7efddea00640 1 --2- 172.21.15.19:0/290645436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efde0153580 0x7efde00d0280 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7efde00cff70 tx=0x7efdd002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00fa4bae5d5176f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.026+0000 7efdde000640 1 -- 172.21.15.19:0/290645436 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd002e070 con 0x7efde0153580 2024-03-28T13:11:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.026+0000 7efdde000640 1 -- 172.21.15.19:0/290645436 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd0037070 con 0x7efde0153580 2024-03-28T13:11:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.026+0000 7efdde000640 1 -- 172.21.15.19:0/290645436 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0031a00 con 0x7efde0153580 2024-03-28T13:11:36.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.026+0000 7efde5800640 1 -- 172.21.15.19:0/290645436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efde0153580 msgr2=0x7efde00d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:36.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.026+0000 7efde5800640 1 --2- 172.21.15.19:0/290645436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efde0153580 0x7efde00d0280 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7efde00cff70 tx=0x7efdd002b880 comp rx=0 tx=0).stop 2024-03-28T13:11:36.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.027+0000 7efde5800640 1 -- 172.21.15.19:0/290645436 shutdown_connections 2024-03-28T13:11:36.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.027+0000 7efde5800640 1 --2- 172.21.15.19:0/290645436 >> v2:172.21.15.188:3300/0 conn(0x7efde00af2e0 0x7efde006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.027+0000 7efde5800640 1 --2- 172.21.15.19:0/290645436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efde00d07c0 0x7efde00aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.027+0000 7efde5800640 1 --2- 172.21.15.19:0/290645436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efde0153580 0x7efde00d0280 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.027+0000 7efde5800640 1 -- 172.21.15.19:0/290645436 >> 172.21.15.19:0/290645436 conn(0x7efde006b750 msgr2=0x7efde006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:36.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.027+0000 7efde5800640 1 -- 172.21.15.19:0/290645436 shutdown_connections 2024-03-28T13:11:36.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.027+0000 7efde5800640 1 -- 172.21.15.19:0/290645436 wait complete. 2024-03-28T13:11:36.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.029+0000 7efde5800640 1 Processor -- start 2024-03-28T13:11:36.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.029+0000 7efde5800640 1 -- start start 2024-03-28T13:11:36.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.030+0000 7efde5800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efde00af2e0 0x7efde006d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.030+0000 7efde5800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efde00d07c0 0x7efde014fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.030+0000 7efde5800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efde0153580 0x7efde0150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.030+0000 7efde5800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efde015d4e0 con 0x7efde00af2e0 2024-03-28T13:11:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.030+0000 7efde5800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efde015d200 con 0x7efde0153580 2024-03-28T13:11:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.030+0000 7efde5800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efde0153960 con 0x7efde00d07c0 2024-03-28T13:11:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.031+0000 7efddea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efde00af2e0 0x7efde006d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.031+0000 7efddea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efde00af2e0 0x7efde006d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44234/0 (socket says 172.21.15.19:44234) 2024-03-28T13:11:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.031+0000 7efddea00640 1 -- 172.21.15.19:0/1674315649 learned_addr learned my addr 172.21.15.19:0/1674315649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:36.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.031+0000 7efddf400640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efde0153580 0x7efde0150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:36.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.031+0000 7efdd7e00640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efde00d07c0 0x7efde014fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:36.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.031+0000 7efddea00640 1 -- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efde00d07c0 msgr2=0x7efde014fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:36.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.031+0000 7efddea00640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efde00d07c0 0x7efde014fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.031+0000 7efddea00640 1 -- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efde0153580 msgr2=0x7efde0150270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:36.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.031+0000 7efddea00640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efde0153580 0x7efde0150270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.031+0000 7efddea00640 1 -- 172.21.15.19:0/1674315649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde014ec90 con 0x7efde00af2e0 2024-03-28T13:11:36.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.031+0000 7efddf400640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efde0153580 0x7efde0150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:36.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.031+0000 7efdd7e00640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efde00d07c0 0x7efde014fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:36.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.032+0000 7efddea00640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efde00af2e0 0x7efde006d820 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7efdd0000c00 tx=0x7efdd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:36.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.032+0000 7efdd6a00640 1 -- 172.21.15.19:0/1674315649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0032670 con 0x7efde00af2e0 2024-03-28T13:11:36.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.032+0000 7efdd6a00640 1 -- 172.21.15.19:0/1674315649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdd0032810 con 0x7efde00af2e0 2024-03-28T13:11:36.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.032+0000 7efdd6a00640 1 -- 172.21.15.19:0/1674315649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd0032c00 con 0x7efde00af2e0 2024-03-28T13:11:36.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.032+0000 7efde5800640 1 -- 172.21.15.19:0/1674315649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efde014ef70 con 0x7efde00af2e0 2024-03-28T13:11:36.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.032+0000 7efde5800640 1 -- 172.21.15.19:0/1674315649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efde014f530 con 0x7efde00af2e0 2024-03-28T13:11:36.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.033+0000 7efde5800640 1 -- 172.21.15.19:0/1674315649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efde00685c0 con 0x7efde00af2e0 2024-03-28T13:11:36.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.034+0000 7efdd6a00640 1 -- 172.21.15.19:0/1674315649 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efdd004c060 con 0x7efde00af2e0 2024-03-28T13:11:36.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.035+0000 7efdd6a00640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efdbc06c530 0x7efdbc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:36.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.035+0000 7efdd6a00640 1 -- 172.21.15.19:0/1674315649 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7efdd00ace30 con 0x7efde00af2e0 2024-03-28T13:11:36.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.036+0000 7efdd7e00640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efdbc06c530 0x7efdbc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:36.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.037+0000 7efdd7e00640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efdbc06c530 0x7efdbc06e9f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009ac0 tx=0x7efdc8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:36.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.040+0000 7efdd6a00640 1 -- 172.21.15.19:0/1674315649 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efdd0040070 con 0x7efde00af2e0 2024-03-28T13:11:36.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.246+0000 7efde5800640 1 -- 172.21.15.19:0/1674315649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7efde015dc30 con 0x7efde00af2e0 2024-03-28T13:11:36.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.251+0000 7efdd6a00640 1 -- 172.21.15.19:0/1674315649 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7efdd007b6f0 con 0x7efde00af2e0 2024-03-28T13:11:36.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.254+0000 7efde5800640 1 -- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efdbc06c530 msgr2=0x7efdbc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:36.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.254+0000 7efde5800640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efdbc06c530 0x7efdbc06e9f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009ac0 tx=0x7efdc8008040 comp rx=0 tx=0).stop 2024-03-28T13:11:36.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.254+0000 7efde5800640 1 -- 172.21.15.19:0/1674315649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efde00af2e0 msgr2=0x7efde006d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:36.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.254+0000 7efde5800640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efde00af2e0 0x7efde006d820 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7efdd0000c00 tx=0x7efdd00046c0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.254+0000 7efde5800640 1 -- 172.21.15.19:0/1674315649 shutdown_connections 2024-03-28T13:11:36.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.254+0000 7efde5800640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efdbc06c530 0x7efdbc06e9f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.254+0000 7efde5800640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efde0153580 0x7efde0150270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.254+0000 7efde5800640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efde00d07c0 0x7efde014fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.254+0000 7efde5800640 1 --2- 172.21.15.19:0/1674315649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efde00af2e0 0x7efde006d820 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.254+0000 7efde5800640 1 -- 172.21.15.19:0/1674315649 >> 172.21.15.19:0/1674315649 conn(0x7efde006b750 msgr2=0x7efde015beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:36.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.255+0000 7efde5800640 1 -- 172.21.15.19:0/1674315649 shutdown_connections 2024-03-28T13:11:36.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.255+0000 7efde5800640 1 -- 172.21.15.19:0/1674315649 wait complete. 2024-03-28T13:11:36.273 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-03-28T13:11:36.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.493+0000 7f466f800640 1 Processor -- start 2024-03-28T13:11:36.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.493+0000 7f466f800640 1 -- start start 2024-03-28T13:11:36.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.495+0000 7f466f800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4668153580 0x7f46680d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:36.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.495+0000 7f466f800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f46680d07c0 0x7f46680aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:36.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.495+0000 7f466f800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f46680af2e0 0x7f466806d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:36.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.495+0000 7f466f800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f46680d0c00 con 0x7f46680af2e0 2024-03-28T13:11:36.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.495+0000 7f466f800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f46680afab0 con 0x7f46680d07c0 2024-03-28T13:11:36.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.495+0000 7f466f800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f466815cc80 con 0x7f4668153580 2024-03-28T13:11:36.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.495+0000 7f466d800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f46680af2e0 0x7f466806d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:36.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.495+0000 7f466d800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f46680af2e0 0x7f466806d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44242/0 (socket says 172.21.15.19:44242) 2024-03-28T13:11:36.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.495+0000 7f466d800640 1 -- 172.21.15.19:0/2928321495 learned_addr learned my addr 172.21.15.19:0/2928321495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:36.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.496+0000 7f4667e00640 1 --2- 172.21.15.19:0/2928321495 >> v2:172.21.15.188:3300/0 conn(0x7f46680d07c0 0x7f46680aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:36.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.496+0000 7f466ce00640 1 --2- 172.21.15.19:0/2928321495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4668153580 0x7f46680d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:36.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.496+0000 7f466d800640 1 -- 172.21.15.19:0/2928321495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4668153580 msgr2=0x7f46680d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:36.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.496+0000 7f466d800640 1 --2- 172.21.15.19:0/2928321495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4668153580 0x7f46680d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.496+0000 7f466d800640 1 -- 172.21.15.19:0/2928321495 >> v2:172.21.15.188:3300/0 conn(0x7f46680d07c0 msgr2=0x7f46680aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:36.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.496+0000 7f466d800640 1 --2- 172.21.15.19:0/2928321495 >> v2:172.21.15.188:3300/0 conn(0x7f46680d07c0 0x7f46680aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.496+0000 7f466d800640 1 -- 172.21.15.19:0/2928321495 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466815da00 con 0x7f46680af2e0 2024-03-28T13:11:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.496+0000 7f4667e00640 1 --2- 172.21.15.19:0/2928321495 >> v2:172.21.15.188:3300/0 conn(0x7f46680d07c0 0x7f46680aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:11:36.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.498+0000 7f466d800640 1 --2- 172.21.15.19:0/2928321495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46680af2e0 0x7f466806d5b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f466806c560 tx=0x7f466002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9aa5f12b9e8ae12 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:36.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.498+0000 7f4667400640 1 -- 172.21.15.19:0/2928321495 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466002c9f0 con 0x7f46680af2e0 2024-03-28T13:11:36.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.498+0000 7f4667400640 1 -- 172.21.15.19:0/2928321495 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f466002cb90 con 0x7f46680af2e0 2024-03-28T13:11:36.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.498+0000 7f4667400640 1 -- 172.21.15.19:0/2928321495 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46600343f0 con 0x7f46680af2e0 2024-03-28T13:11:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.499+0000 7f466f800640 1 -- 172.21.15.19:0/2928321495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46680af2e0 msgr2=0x7f466806d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.499+0000 7f466f800640 1 --2- 172.21.15.19:0/2928321495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46680af2e0 0x7f466806d5b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f466806c560 tx=0x7f466002be50 comp rx=0 tx=0).stop 2024-03-28T13:11:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.499+0000 7f466f800640 1 -- 172.21.15.19:0/2928321495 shutdown_connections 2024-03-28T13:11:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.499+0000 7f466f800640 1 --2- 172.21.15.19:0/2928321495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46680af2e0 0x7f466806d5b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.499+0000 7f466f800640 1 --2- 172.21.15.19:0/2928321495 >> v2:172.21.15.188:3300/0 conn(0x7f46680d07c0 0x7f46680aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.499+0000 7f466f800640 1 --2- 172.21.15.19:0/2928321495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4668153580 0x7f46680d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.499+0000 7f466f800640 1 -- 172.21.15.19:0/2928321495 >> 172.21.15.19:0/2928321495 conn(0x7f466806b750 msgr2=0x7f466806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.499+0000 7f466f800640 1 -- 172.21.15.19:0/2928321495 shutdown_connections 2024-03-28T13:11:36.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.499+0000 7f466f800640 1 -- 172.21.15.19:0/2928321495 wait complete. 2024-03-28T13:11:36.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.501+0000 7f466f800640 1 Processor -- start 2024-03-28T13:11:36.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.501+0000 7f466f800640 1 -- start start 2024-03-28T13:11:36.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.502+0000 7f466f800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46680af2e0 0x7f46681f1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.502+0000 7f466f800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46680d07c0 0x7f46680cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.502+0000 7f466f800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4668153580 0x7f46680cf720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.502+0000 7f466f800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f466815d4e0 con 0x7f46680af2e0 2024-03-28T13:11:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.502+0000 7f466f800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f466815d200 con 0x7f46680d07c0 2024-03-28T13:11:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.502+0000 7f466f800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4668153960 con 0x7f4668153580 2024-03-28T13:11:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.502+0000 7f466ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46680af2e0 0x7f46681f1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.502+0000 7f466ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46680af2e0 0x7f46681f1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44246/0 (socket says 172.21.15.19:44246) 2024-03-28T13:11:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.502+0000 7f466ce00640 1 -- 172.21.15.19:0/2038434170 learned_addr learned my addr 172.21.15.19:0/2038434170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.502+0000 7f466d800640 1 --2- 172.21.15.19:0/2038434170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4668153580 0x7f46680cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.502+0000 7f4667e00640 1 --2- 172.21.15.19:0/2038434170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46680d07c0 0x7f46680cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.502+0000 7f466ce00640 1 -- 172.21.15.19:0/2038434170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4668153580 msgr2=0x7f46680cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.502+0000 7f466ce00640 1 --2- 172.21.15.19:0/2038434170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4668153580 0x7f46680cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.503+0000 7f466ce00640 1 -- 172.21.15.19:0/2038434170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46680d07c0 msgr2=0x7f46680cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:36.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.503+0000 7f466ce00640 1 --2- 172.21.15.19:0/2038434170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46680d07c0 0x7f46680cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.503+0000 7f466ce00640 1 -- 172.21.15.19:0/2038434170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4668158cc0 con 0x7f46680af2e0 2024-03-28T13:11:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.503+0000 7f4667e00640 1 --2- 172.21.15.19:0/2038434170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46680d07c0 0x7f46680cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.503+0000 7f466ce00640 1 --2- 172.21.15.19:0/2038434170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46680af2e0 0x7f46681f1480 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f465800d800 tx=0x7f465800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:36.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.503+0000 7f4664c00640 1 -- 172.21.15.19:0/2038434170 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465800b6c0 con 0x7f46680af2e0 2024-03-28T13:11:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.503+0000 7f4664c00640 1 -- 172.21.15.19:0/2038434170 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4658019070 con 0x7f46680af2e0 2024-03-28T13:11:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.503+0000 7f4664c00640 1 -- 172.21.15.19:0/2038434170 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465800ba80 con 0x7f46680af2e0 2024-03-28T13:11:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.503+0000 7f466f800640 1 -- 172.21.15.19:0/2038434170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4668158fa0 con 0x7f46680af2e0 2024-03-28T13:11:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.503+0000 7f466f800640 1 -- 172.21.15.19:0/2038434170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4668159560 con 0x7f46680af2e0 2024-03-28T13:11:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.505+0000 7f466f800640 1 -- 172.21.15.19:0/2038434170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46680685c0 con 0x7f46680af2e0 2024-03-28T13:11:36.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.505+0000 7f4664c00640 1 -- 172.21.15.19:0/2038434170 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f465800bc20 con 0x7f46680af2e0 2024-03-28T13:11:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.507+0000 7f4664c00640 1 --2- 172.21.15.19:0/2038434170 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f464806c530 0x7f464806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.507+0000 7f4664c00640 1 -- 172.21.15.19:0/2038434170 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f465808b830 con 0x7f46680af2e0 2024-03-28T13:11:36.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.508+0000 7f4667e00640 1 --2- 172.21.15.19:0/2038434170 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f464806c530 0x7f464806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:36.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.509+0000 7f4667e00640 1 --2- 172.21.15.19:0/2038434170 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f464806c530 0x7f464806e9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4654006fd0 tx=0x7f4654008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:36.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.513+0000 7f4664c00640 1 -- 172.21.15.19:0/2038434170 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f465805a1c0 con 0x7f46680af2e0 2024-03-28T13:11:36.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:36.818+0000 7f466f800640 1 -- 172.21.15.19:0/2038434170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7f46680b2820 con 0x7f46680af2e0 2024-03-28T13:11:37.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:37.807+0000 7f4664c00640 1 -- 172.21.15.19:0/2038434170 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v128) ==== 142+0+0 (secure 0 0 0) 0x7f4658059ac0 con 0x7f46680af2e0 2024-03-28T13:11:37.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:37.811+0000 7f466f800640 1 -- 172.21.15.19:0/2038434170 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f464806c530 msgr2=0x7f464806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:37.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:37.811+0000 7f466f800640 1 --2- 172.21.15.19:0/2038434170 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f464806c530 0x7f464806e9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4654006fd0 tx=0x7f4654008040 comp rx=0 tx=0).stop 2024-03-28T13:11:37.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:37.811+0000 7f466f800640 1 -- 172.21.15.19:0/2038434170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46680af2e0 msgr2=0x7f46681f1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:37.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:37.811+0000 7f466f800640 1 --2- 172.21.15.19:0/2038434170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46680af2e0 0x7f46681f1480 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f465800d800 tx=0x7f465800dcc0 comp rx=0 tx=0).stop 2024-03-28T13:11:37.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:37.812+0000 7f466f800640 1 -- 172.21.15.19:0/2038434170 shutdown_connections 2024-03-28T13:11:37.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:37.812+0000 7f466f800640 1 --2- 172.21.15.19:0/2038434170 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f464806c530 0x7f464806e9f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:37.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:37.812+0000 7f466f800640 1 --2- 172.21.15.19:0/2038434170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4668153580 0x7f46680cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:37.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:37.812+0000 7f466f800640 1 --2- 172.21.15.19:0/2038434170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46680d07c0 0x7f46680cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:37.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:37.812+0000 7f466f800640 1 --2- 172.21.15.19:0/2038434170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46680af2e0 0x7f46681f1480 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:37.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:37.812+0000 7f466f800640 1 -- 172.21.15.19:0/2038434170 >> 172.21.15.19:0/2038434170 conn(0x7f466806b750 msgr2=0x7f466815a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:37.812+0000 7f466f800640 1 -- 172.21.15.19:0/2038434170 shutdown_connections 2024-03-28T13:11:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:37.812+0000 7f466f800640 1 -- 172.21.15.19:0/2038434170 wait complete. 2024-03-28T13:11:37.835 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-03-28T13:11:37.835 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-03-28T13:11:37.835 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-28T13:11:37.836 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:11:38.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.057+0000 7f95eec00640 1 Processor -- start 2024-03-28T13:11:38.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.057+0000 7f95eec00640 1 -- start start 2024-03-28T13:11:38.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.059+0000 7f95eec00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e815d120 0x7f95e8162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:38.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.059+0000 7f95eec00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f95e80af630 0x7f95e8166b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.059+0000 7f95eec00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f95e8167080 0x7f95e80b3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.059+0000 7f95eec00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f95e80723c0 con 0x7f95e80af630 2024-03-28T13:11:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.059+0000 7f95eec00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f95e81633d0 con 0x7f95e8167080 2024-03-28T13:11:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.059+0000 7f95eec00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f95e8163650 con 0x7f95e815d120 2024-03-28T13:11:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.060+0000 7f95df400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f95e80af630 0x7f95e8166b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.060+0000 7f95df400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f95e80af630 0x7f95e8166b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44260/0 (socket says 172.21.15.19:44260) 2024-03-28T13:11:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.060+0000 7f95df400640 1 -- 172.21.15.19:0/2288610639 learned_addr learned my addr 172.21.15.19:0/2288610639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.060+0000 7f95e7e00640 1 --2- 172.21.15.19:0/2288610639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e815d120 0x7f95e8162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.060+0000 7f95ecc00640 1 --2- 172.21.15.19:0/2288610639 >> v2:172.21.15.188:3300/0 conn(0x7f95e8167080 0x7f95e80b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:38.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.060+0000 7f95df400640 1 -- 172.21.15.19:0/2288610639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e815d120 msgr2=0x7f95e8162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.060+0000 7f95df400640 1 --2- 172.21.15.19:0/2288610639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e815d120 0x7f95e8162e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.060+0000 7f95df400640 1 -- 172.21.15.19:0/2288610639 >> v2:172.21.15.188:3300/0 conn(0x7f95e8167080 msgr2=0x7f95e80b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.060+0000 7f95df400640 1 --2- 172.21.15.19:0/2288610639 >> v2:172.21.15.188:3300/0 conn(0x7f95e8167080 0x7f95e80b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.060+0000 7f95df400640 1 -- 172.21.15.19:0/2288610639 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e8163930 con 0x7f95e80af630 2024-03-28T13:11:38.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.060+0000 7f95ecc00640 1 --2- 172.21.15.19:0/2288610639 >> v2:172.21.15.188:3300/0 conn(0x7f95e8167080 0x7f95e80b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:38.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.061+0000 7f95df400640 1 --2- 172.21.15.19:0/2288610639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e80af630 0x7f95e8166b40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f95e815d500 tx=0x7f95d002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255efcd668d7417d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:38.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.062+0000 7f95e7400640 1 -- 172.21.15.19:0/2288610639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0035020 con 0x7f95e80af630 2024-03-28T13:11:38.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.062+0000 7f95e7400640 1 -- 172.21.15.19:0/2288610639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d002d040 con 0x7f95e80af630 2024-03-28T13:11:38.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.062+0000 7f95e7400640 1 -- 172.21.15.19:0/2288610639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0031a80 con 0x7f95e80af630 2024-03-28T13:11:38.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.062+0000 7f95eec00640 1 -- 172.21.15.19:0/2288610639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e80af630 msgr2=0x7f95e8166b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:38.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.062+0000 7f95eec00640 1 --2- 172.21.15.19:0/2288610639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e80af630 0x7f95e8166b40 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f95e815d500 tx=0x7f95d002b900 comp rx=0 tx=0).stop 2024-03-28T13:11:38.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.062+0000 7f95eec00640 1 -- 172.21.15.19:0/2288610639 shutdown_connections 2024-03-28T13:11:38.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.062+0000 7f95eec00640 1 --2- 172.21.15.19:0/2288610639 >> v2:172.21.15.188:3300/0 conn(0x7f95e8167080 0x7f95e80b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.062+0000 7f95eec00640 1 --2- 172.21.15.19:0/2288610639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e80af630 0x7f95e8166b40 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.062+0000 7f95eec00640 1 --2- 172.21.15.19:0/2288610639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e815d120 0x7f95e8162e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.062+0000 7f95eec00640 1 -- 172.21.15.19:0/2288610639 >> 172.21.15.19:0/2288610639 conn(0x7f95e806b990 msgr2=0x7f95e806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:38.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.062+0000 7f95eec00640 1 -- 172.21.15.19:0/2288610639 shutdown_connections 2024-03-28T13:11:38.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.063+0000 7f95eec00640 1 -- 172.21.15.19:0/2288610639 wait complete. 2024-03-28T13:11:38.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.064+0000 7f95eec00640 1 Processor -- start 2024-03-28T13:11:38.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.064+0000 7f95eec00640 1 -- start start 2024-03-28T13:11:38.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.065+0000 7f95eec00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e80af630 0x7f95e815c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:38.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.065+0000 7f95eec00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e815d120 0x7f95e80cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.065+0000 7f95eec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e8167080 0x7f95e80b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.065+0000 7f95eec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95e80b3a40 con 0x7f95e8167080 2024-03-28T13:11:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.065+0000 7f95eec00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95e8166620 con 0x7f95e815d120 2024-03-28T13:11:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.065+0000 7f95eec00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f95e80b3bc0 con 0x7f95e80af630 2024-03-28T13:11:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.066+0000 7f95ecc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e8167080 0x7f95e80b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.066+0000 7f95ecc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e8167080 0x7f95e80b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44264/0 (socket says 172.21.15.19:44264) 2024-03-28T13:11:38.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.066+0000 7f95ecc00640 1 -- 172.21.15.19:0/2085933807 learned_addr learned my addr 172.21.15.19:0/2085933807 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:38.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.066+0000 7f95df400640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e815d120 0x7f95e80cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:38.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.066+0000 7f95e7e00640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e80af630 0x7f95e815c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:38.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.066+0000 7f95ecc00640 1 -- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e80af630 msgr2=0x7f95e815c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:38.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.066+0000 7f95ecc00640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e80af630 0x7f95e815c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.066+0000 7f95ecc00640 1 -- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e815d120 msgr2=0x7f95e80cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:38.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.066+0000 7f95ecc00640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e815d120 0x7f95e80cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.066+0000 7f95ecc00640 1 -- 172.21.15.19:0/2085933807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e81f8c40 con 0x7f95e8167080 2024-03-28T13:11:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.066+0000 7f95e7e00640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e80af630 0x7f95e815c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.066+0000 7f95df400640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e815d120 0x7f95e80cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.066+0000 7f95ecc00640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e8167080 0x7f95e80b1dc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f95e000cef0 tx=0x7f95e000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.067+0000 7f95e4c00640 1 -- 172.21.15.19:0/2085933807 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e001a070 con 0x7f95e8167080 2024-03-28T13:11:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.067+0000 7f95e4c00640 1 -- 172.21.15.19:0/2085933807 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95e000d860 con 0x7f95e8167080 2024-03-28T13:11:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.067+0000 7f95e4c00640 1 -- 172.21.15.19:0/2085933807 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e000dc50 con 0x7f95e8167080 2024-03-28T13:11:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.067+0000 7f95eec00640 1 -- 172.21.15.19:0/2085933807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e81f8f20 con 0x7f95e8167080 2024-03-28T13:11:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.067+0000 7f95eec00640 1 -- 172.21.15.19:0/2085933807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e80cf8d0 con 0x7f95e8167080 2024-03-28T13:11:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.068+0000 7f95eec00640 1 -- 172.21.15.19:0/2085933807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e80685c0 con 0x7f95e8167080 2024-03-28T13:11:38.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.069+0000 7f95e4c00640 1 -- 172.21.15.19:0/2085933807 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f95e001d070 con 0x7f95e8167080 2024-03-28T13:11:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.070+0000 7f95e4c00640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f95c406c530 0x7f95c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.071+0000 7f95e4c00640 1 -- 172.21.15.19:0/2085933807 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f95e008bcb0 con 0x7f95e8167080 2024-03-28T13:11:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.071+0000 7f95e7e00640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f95c406c530 0x7f95c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:38.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.072+0000 7f95e7e00640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f95c406c530 0x7f95c406e9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f95d8006fd0 tx=0x7f95d8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.076+0000 7f95e4c00640 1 -- 172.21.15.19:0/2085933807 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f95e005a680 con 0x7f95e8167080 2024-03-28T13:11:38.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.278+0000 7f95eec00640 1 -- 172.21.15.19:0/2085933807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95e806f3a0 con 0x7f95e8167080 2024-03-28T13:11:38.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.280+0000 7f95e4c00640 1 -- 172.21.15.19:0/2085933807 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+13018 (secure 0 0 0) 0x7f95e0059f80 con 0x7f95e8167080 2024-03-28T13:11:38.280 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:38.280 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":41,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:11:35.779881+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6837/46284318":"2024-03-29T13:11:32.763458+0000","172.21.15.188:6835/46284318":"2024-03-29T13:11:32.763458+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:11:38.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.283+0000 7f95eec00640 1 -- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f95c406c530 msgr2=0x7f95c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:38.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.283+0000 7f95eec00640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f95c406c530 0x7f95c406e9f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f95d8006fd0 tx=0x7f95d8008040 comp rx=0 tx=0).stop 2024-03-28T13:11:38.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.283+0000 7f95eec00640 1 -- 172.21.15.19:0/2085933807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e8167080 msgr2=0x7f95e80b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:38.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.283+0000 7f95eec00640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e8167080 0x7f95e80b1dc0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f95e000cef0 tx=0x7f95e000a710 comp rx=0 tx=0).stop 2024-03-28T13:11:38.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.283+0000 7f95eec00640 1 -- 172.21.15.19:0/2085933807 shutdown_connections 2024-03-28T13:11:38.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.283+0000 7f95eec00640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f95c406c530 0x7f95c406e9f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.283+0000 7f95eec00640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e8167080 0x7f95e80b1dc0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.283+0000 7f95eec00640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95e815d120 0x7f95e80cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.283+0000 7f95eec00640 1 --2- 172.21.15.19:0/2085933807 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95e80af630 0x7f95e815c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.283+0000 7f95eec00640 1 -- 172.21.15.19:0/2085933807 >> 172.21.15.19:0/2085933807 conn(0x7f95e806b990 msgr2=0x7f95e80ad120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:38.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.283+0000 7f95eec00640 1 -- 172.21.15.19:0/2085933807 shutdown_connections 2024-03-28T13:11:38.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.283+0000 7f95eec00640 1 -- 172.21.15.19:0/2085933807 wait complete. 2024-03-28T13:11:38.302 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:11:38.303 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:11:38.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.528+0000 7fcbffe00640 1 Processor -- start 2024-03-28T13:11:38.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.529+0000 7fcbffe00640 1 -- start start 2024-03-28T13:11:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.530+0000 7fcbffe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcbf81412a0 0x7fcbf80b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:38.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.530+0000 7fcbffe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcbf8141680 0x7fcbf80b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:38.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.530+0000 7fcbffe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbf815d7c0 con 0x7fcbf815ca50 2024-03-28T13:11:38.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.530+0000 7fcbffe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcbf8141be0 con 0x7fcbf81412a0 2024-03-28T13:11:38.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.530+0000 7fcbffe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fcbf8141ec0 con 0x7fcbf8141680 2024-03-28T13:11:38.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.530+0000 7fcbfd400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcbf81412a0 0x7fcbf80b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:38.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.531+0000 7fcbfd400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcbf81412a0 0x7fcbf80b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44272/0 (socket says 172.21.15.19:44272) 2024-03-28T13:11:38.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.531+0000 7fcbfd400640 1 -- 172.21.15.19:0/3229298242 learned_addr learned my addr 172.21.15.19:0/3229298242 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.531+0000 7fcbfca00640 1 --2- 172.21.15.19:0/3229298242 >> v2:172.21.15.188:3300/0 conn(0x7fcbf8141680 0x7fcbf80b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.531+0000 7fcbfd400640 1 -- 172.21.15.19:0/3229298242 >> v2:172.21.15.188:3300/0 conn(0x7fcbf8141680 msgr2=0x7fcbf80b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.531+0000 7fcbfd400640 1 --2- 172.21.15.19:0/3229298242 >> v2:172.21.15.188:3300/0 conn(0x7fcbf8141680 0x7fcbf80b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.531+0000 7fcbfd400640 1 -- 172.21.15.19:0/3229298242 >> v1:172.21.15.19:6789/0 conn(0x7fcbf815ca50 legacy=0x7fcbf815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.531+0000 7fcbfd400640 1 -- 172.21.15.19:0/3229298242 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf81421a0 con 0x7fcbf81412a0 2024-03-28T13:11:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.532+0000 7fcbfd400640 1 --2- 172.21.15.19:0/3229298242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf81412a0 0x7fcbf80b43e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80718c0 tx=0x7fcbf002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a150b781ef3f0297 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.533+0000 7fcbf7e00640 1 -- 172.21.15.19:0/3229298242 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf002cb20 con 0x7fcbf81412a0 2024-03-28T13:11:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.533+0000 7fcbf7e00640 1 -- 172.21.15.19:0/3229298242 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf002ccc0 con 0x7fcbf81412a0 2024-03-28T13:11:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.533+0000 7fcbf7e00640 1 -- 172.21.15.19:0/3229298242 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0034610 con 0x7fcbf81412a0 2024-03-28T13:11:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.533+0000 7fcbffe00640 1 -- 172.21.15.19:0/3229298242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf81412a0 msgr2=0x7fcbf80b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:38.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.533+0000 7fcbffe00640 1 --2- 172.21.15.19:0/3229298242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf81412a0 0x7fcbf80b43e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80718c0 tx=0x7fcbf002bf80 comp rx=0 tx=0).stop 2024-03-28T13:11:38.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.533+0000 7fcbffe00640 1 -- 172.21.15.19:0/3229298242 shutdown_connections 2024-03-28T13:11:38.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.533+0000 7fcbffe00640 1 --2- 172.21.15.19:0/3229298242 >> v2:172.21.15.188:3300/0 conn(0x7fcbf8141680 0x7fcbf80b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.533+0000 7fcbffe00640 1 --2- 172.21.15.19:0/3229298242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf81412a0 0x7fcbf80b43e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.533+0000 7fcbffe00640 1 -- 172.21.15.19:0/3229298242 >> 172.21.15.19:0/3229298242 conn(0x7fcbf8067670 msgr2=0x7fcbf8067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:38.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.533+0000 7fcbffe00640 1 -- 172.21.15.19:0/3229298242 shutdown_connections 2024-03-28T13:11:38.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.533+0000 7fcbffe00640 1 -- 172.21.15.19:0/3229298242 wait complete. 2024-03-28T13:11:38.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.535+0000 7fcbffe00640 1 Processor -- start 2024-03-28T13:11:38.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.535+0000 7fcbffe00640 1 -- start start 2024-03-28T13:11:38.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.536+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbf81412a0 0x7fcbf81f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:38.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.536+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf8141680 0x7fcbf8158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:38.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.536+0000 7fcbffe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbf815ca50 0x7fcbf813eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:38.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.536+0000 7fcbffe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf80cec40 con 0x7fcbf8141680 2024-03-28T13:11:38.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.536+0000 7fcbffe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcbf81429a0 con 0x7fcbf81412a0 2024-03-28T13:11:38.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.536+0000 7fcbffe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcbf80cedc0 con 0x7fcbf815ca50 2024-03-28T13:11:38.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.536+0000 7fcbfca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf8141680 0x7fcbf8158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:38.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.537+0000 7fcbfca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf8141680 0x7fcbf8158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44282/0 (socket says 172.21.15.19:44282) 2024-03-28T13:11:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.537+0000 7fcbfca00640 1 -- 172.21.15.19:0/1262383439 learned_addr learned my addr 172.21.15.19:0/1262383439 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.537+0000 7fcbfca00640 1 -- 172.21.15.19:0/1262383439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbf815ca50 msgr2=0x7fcbf813eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.537+0000 7fcbfca00640 1 --2- 172.21.15.19:0/1262383439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbf815ca50 0x7fcbf813eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.537+0000 7fcbfca00640 1 -- 172.21.15.19:0/1262383439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbf81412a0 msgr2=0x7fcbf81f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.537+0000 7fcbfca00640 1 --2- 172.21.15.19:0/1262383439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbf81412a0 0x7fcbf81f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.537+0000 7fcbfca00640 1 -- 172.21.15.19:0/1262383439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbf81f2660 con 0x7fcbf8141680 2024-03-28T13:11:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.537+0000 7fcbfca00640 1 --2- 172.21.15.19:0/1262383439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf8141680 0x7fcbf8158c90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400d800 tx=0x7fcbe400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.538+0000 7fcbf5600640 1 -- 172.21.15.19:0/1262383439 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400b6c0 con 0x7fcbf8141680 2024-03-28T13:11:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.538+0000 7fcbf5600640 1 -- 172.21.15.19:0/1262383439 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbe4019070 con 0x7fcbf8141680 2024-03-28T13:11:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.538+0000 7fcbf5600640 1 -- 172.21.15.19:0/1262383439 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400ba80 con 0x7fcbf8141680 2024-03-28T13:11:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.538+0000 7fcbffe00640 1 -- 172.21.15.19:0/1262383439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbf81f2940 con 0x7fcbf8141680 2024-03-28T13:11:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.538+0000 7fcbffe00640 1 -- 172.21.15.19:0/1262383439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbf813f050 con 0x7fcbf8141680 2024-03-28T13:11:38.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.539+0000 7fcbffe00640 1 -- 172.21.15.19:0/1262383439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbf8074f30 con 0x7fcbf8141680 2024-03-28T13:11:38.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.540+0000 7fcbf5600640 1 -- 172.21.15.19:0/1262383439 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcbe400bc20 con 0x7fcbf8141680 2024-03-28T13:11:38.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.541+0000 7fcbf5600640 1 --2- 172.21.15.19:0/1262383439 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcbd806c530 0x7fcbd806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:38.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.541+0000 7fcbf5600640 1 -- 172.21.15.19:0/1262383439 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7fcbe408b8b0 con 0x7fcbf8141680 2024-03-28T13:11:38.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.542+0000 7fcbfd400640 1 --2- 172.21.15.19:0/1262383439 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcbd806c530 0x7fcbd806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:38.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.543+0000 7fcbfd400640 1 --2- 172.21.15.19:0/1262383439 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcbd806c530 0x7fcbd806e9f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0035000 tx=0x7fcbf002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:38.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.545+0000 7fcbf5600640 1 -- 172.21.15.19:0/1262383439 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcbe405a1c0 con 0x7fcbf8141680 2024-03-28T13:11:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.749+0000 7fcbffe00640 1 -- 172.21.15.19:0/1262383439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcbf80685c0 con 0x7fcbf8141680 2024-03-28T13:11:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.750+0000 7fcbf5600640 1 -- 172.21.15.19:0/1262383439 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7fcbe4059ac0 con 0x7fcbf8141680 2024-03-28T13:11:38.749 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:11:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.752+0000 7fcbffe00640 1 -- 172.21.15.19:0/1262383439 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcbd806c530 msgr2=0x7fcbd806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.752+0000 7fcbffe00640 1 --2- 172.21.15.19:0/1262383439 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcbd806c530 0x7fcbd806e9f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0035000 tx=0x7fcbf002e000 comp rx=0 tx=0).stop 2024-03-28T13:11:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.752+0000 7fcbffe00640 1 -- 172.21.15.19:0/1262383439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf8141680 msgr2=0x7fcbf8158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.752+0000 7fcbffe00640 1 --2- 172.21.15.19:0/1262383439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf8141680 0x7fcbf8158c90 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400d800 tx=0x7fcbe400dcc0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.752+0000 7fcbffe00640 1 -- 172.21.15.19:0/1262383439 shutdown_connections 2024-03-28T13:11:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.752+0000 7fcbffe00640 1 --2- 172.21.15.19:0/1262383439 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcbd806c530 0x7fcbd806e9f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.752+0000 7fcbffe00640 1 --2- 172.21.15.19:0/1262383439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbf815ca50 0x7fcbf813eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.752+0000 7fcbffe00640 1 --2- 172.21.15.19:0/1262383439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcbf8141680 0x7fcbf8158c90 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.752+0000 7fcbffe00640 1 --2- 172.21.15.19:0/1262383439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbf81412a0 0x7fcbf81f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.752+0000 7fcbffe00640 1 -- 172.21.15.19:0/1262383439 >> 172.21.15.19:0/1262383439 conn(0x7fcbf8067670 msgr2=0x7fcbf80cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.753+0000 7fcbffe00640 1 -- 172.21.15.19:0/1262383439 shutdown_connections 2024-03-28T13:11:38.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.753+0000 7fcbffe00640 1 -- 172.21.15.19:0/1262383439 wait complete. 2024-03-28T13:11:38.772 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-03-28T13:11:38.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.993+0000 7f3b50a00640 1 Processor -- start 2024-03-28T13:11:38.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.993+0000 7f3b50a00640 1 -- start start 2024-03-28T13:11:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.995+0000 7f3b50a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3b4c06e030 0x7f3b4c15fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.995+0000 7f3b50a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b4c142880 con 0x7f3b4c13f300 2024-03-28T13:11:38.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.995+0000 7f3b50a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b4c1422c0 con 0x7f3b4c1602d0 2024-03-28T13:11:38.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.995+0000 7f3b50a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3b4c160720 con 0x7f3b4c06e030 2024-03-28T13:11:38.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.995+0000 7f3b4b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3b4c13f300 0x7f3b4c1419e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47582/0 (socket says 172.21.15.19:47582) 2024-03-28T13:11:38.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.995+0000 7f3b4b400640 1 -- 172.21.15.19:0/731122206 learned_addr learned my addr 172.21.15.19:0/731122206 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:38.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.996+0000 7f3b43e00640 1 --2- 172.21.15.19:0/731122206 >> v2:172.21.15.19:3300/0 conn(0x7f3b4c06e030 0x7f3b4c15fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:38.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.996+0000 7f3b4aa00640 1 -- 172.21.15.19:0/731122206 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3518712691 0 0) 0x7f3b4c142880 con 0x7f3b4c13f300 2024-03-28T13:11:38.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.996+0000 7f3b4aa00640 1 -- 172.21.15.19:0/731122206 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b30003520 con 0x7f3b4c13f300 2024-03-28T13:11:38.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.996+0000 7f3b43e00640 1 -- 172.21.15.19:0/731122206 >> v1:172.21.15.188:6789/0 conn(0x7f3b4c1602d0 legacy=0x7f3b4c0cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:38.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.996+0000 7f3b43e00640 1 -- 172.21.15.19:0/731122206 >> v1:172.21.15.19:6789/0 conn(0x7f3b4c13f300 legacy=0x7f3b4c1419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:38.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.996+0000 7f3b43e00640 1 -- 172.21.15.19:0/731122206 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c1425a0 con 0x7f3b4c06e030 2024-03-28T13:11:38.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.997+0000 7f3b43e00640 1 --2- 172.21.15.19:0/731122206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4c06e030 0x7f3b4c15fd90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3b340097a0 tx=0x7f3b3402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd934a6dabada40a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:38.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.998+0000 7f3b4aa00640 1 -- 172.21.15.19:0/731122206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34035020 con 0x7f3b4c06e030 2024-03-28T13:11:38.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.998+0000 7f3b4aa00640 1 -- 172.21.15.19:0/731122206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b34031830 con 0x7f3b4c06e030 2024-03-28T13:11:38.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.998+0000 7f3b4aa00640 1 -- 172.21.15.19:0/731122206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34031c00 con 0x7f3b4c06e030 2024-03-28T13:11:38.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.998+0000 7f3b50a00640 1 -- 172.21.15.19:0/731122206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4c06e030 msgr2=0x7f3b4c15fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:38.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.998+0000 7f3b50a00640 1 --2- 172.21.15.19:0/731122206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4c06e030 0x7f3b4c15fd90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3b340097a0 tx=0x7f3b3402b8d0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.998+0000 7f3b50a00640 1 -- 172.21.15.19:0/731122206 shutdown_connections 2024-03-28T13:11:38.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.998+0000 7f3b50a00640 1 --2- 172.21.15.19:0/731122206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4c06e030 0x7f3b4c15fd90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:38.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.998+0000 7f3b50a00640 1 -- 172.21.15.19:0/731122206 >> 172.21.15.19:0/731122206 conn(0x7f3b4c06b570 msgr2=0x7f3b4c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:38.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.999+0000 7f3b50a00640 1 -- 172.21.15.19:0/731122206 shutdown_connections 2024-03-28T13:11:38.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:38.999+0000 7f3b50a00640 1 -- 172.21.15.19:0/731122206 wait complete. 2024-03-28T13:11:38.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.000+0000 7f3b50a00640 1 Processor -- start 2024-03-28T13:11:38.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.000+0000 7f3b50a00640 1 -- start start 2024-03-28T13:11:39.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.001+0000 7f3b50a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b4c06e030 0x7f3b4c1f1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:39.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.001+0000 7f3b50a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4c13f300 0x7f3b4c1324e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:39.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.001+0000 7f3b50a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b4c1602d0 0x7f3b4c15a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:39.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.001+0000 7f3b50a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c0cfa30 con 0x7f3b4c13f300 2024-03-28T13:11:39.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.001+0000 7f3b50a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4c0cee90 con 0x7f3b4c06e030 2024-03-28T13:11:39.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.001+0000 7f3b50a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4c0cfbb0 con 0x7f3b4c1602d0 2024-03-28T13:11:39.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.002+0000 7f3b43e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4c13f300 0x7f3b4c1324e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:39.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.002+0000 7f3b43e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4c13f300 0x7f3b4c1324e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44296/0 (socket says 172.21.15.19:44296) 2024-03-28T13:11:39.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.002+0000 7f3b43e00640 1 -- 172.21.15.19:0/489895913 learned_addr learned my addr 172.21.15.19:0/489895913 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.002+0000 7f3b4b400640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b4c06e030 0x7f3b4c1f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.002+0000 7f3b4be00640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b4c1602d0 0x7f3b4c15a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.002+0000 7f3b43e00640 1 -- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b4c1602d0 msgr2=0x7f3b4c15a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.002+0000 7f3b43e00640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b4c1602d0 0x7f3b4c15a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.002+0000 7f3b43e00640 1 -- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b4c06e030 msgr2=0x7f3b4c1f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.002+0000 7f3b43e00640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b4c06e030 0x7f3b4c1f1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.002+0000 7f3b43e00640 1 -- 172.21.15.19:0/489895913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4c1f1870 con 0x7f3b4c13f300 2024-03-28T13:11:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.002+0000 7f3b4b400640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b4c06e030 0x7f3b4c1f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:39.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.002+0000 7f3b4be00640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b4c1602d0 0x7f3b4c15a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:39.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.003+0000 7f3b43e00640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4c13f300 0x7f3b4c1324e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3b340098c0 tx=0x7f3b340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:39.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.003+0000 7f3b43400640 1 -- 172.21.15.19:0/489895913 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34041070 con 0x7f3b4c13f300 2024-03-28T13:11:39.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.003+0000 7f3b43400640 1 -- 172.21.15.19:0/489895913 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b3403b920 con 0x7f3b4c13f300 2024-03-28T13:11:39.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.003+0000 7f3b50a00640 1 -- 172.21.15.19:0/489895913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4c157140 con 0x7f3b4c13f300 2024-03-28T13:11:39.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.003+0000 7f3b43400640 1 -- 172.21.15.19:0/489895913 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34035040 con 0x7f3b4c13f300 2024-03-28T13:11:39.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.003+0000 7f3b50a00640 1 -- 172.21.15.19:0/489895913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4c157700 con 0x7f3b4c13f300 2024-03-28T13:11:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.004+0000 7f3b50a00640 1 -- 172.21.15.19:0/489895913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4c075940 con 0x7f3b4c13f300 2024-03-28T13:11:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.005+0000 7f3b43400640 1 -- 172.21.15.19:0/489895913 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3b3404c060 con 0x7f3b4c13f300 2024-03-28T13:11:39.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.007+0000 7f3b43400640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3b2806c530 0x7f3b2806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:39.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.007+0000 7f3b43400640 1 -- 172.21.15.19:0/489895913 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f3b340acd40 con 0x7f3b4c13f300 2024-03-28T13:11:39.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.007+0000 7f3b4b400640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3b2806c530 0x7f3b2806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.009+0000 7f3b4b400640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3b2806c530 0x7f3b2806e9f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c005880 tx=0x7f3b3c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:39.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.011+0000 7f3b43400640 1 -- 172.21.15.19:0/489895913 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3b34031800 con 0x7f3b4c13f300 2024-03-28T13:11:39.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.289+0000 7f3b50a00640 1 -- 172.21.15.19:0/489895913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f3b4c06c580 con 0x7f3b4c13f300 2024-03-28T13:11:39.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.818+0000 7f3b43400640 1 -- 172.21.15.19:0/489895913 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v130) ==== 152+0+0 (secure 0 0 0) 0x7f3b3407b5c0 con 0x7f3b4c13f300 2024-03-28T13:11:39.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.822+0000 7f3b50a00640 1 -- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3b2806c530 msgr2=0x7f3b2806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:39.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.822+0000 7f3b50a00640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3b2806c530 0x7f3b2806e9f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c005880 tx=0x7f3b3c009040 comp rx=0 tx=0).stop 2024-03-28T13:11:39.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.822+0000 7f3b50a00640 1 -- 172.21.15.19:0/489895913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4c13f300 msgr2=0x7f3b4c1324e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:39.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.822+0000 7f3b50a00640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4c13f300 0x7f3b4c1324e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3b340098c0 tx=0x7f3b340046c0 comp rx=0 tx=0).stop 2024-03-28T13:11:39.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.822+0000 7f3b50a00640 1 -- 172.21.15.19:0/489895913 shutdown_connections 2024-03-28T13:11:39.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.822+0000 7f3b50a00640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3b2806c530 0x7f3b2806e9f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:39.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.822+0000 7f3b50a00640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b4c1602d0 0x7f3b4c15a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:39.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.822+0000 7f3b50a00640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b4c13f300 0x7f3b4c1324e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:39.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.822+0000 7f3b50a00640 1 --2- 172.21.15.19:0/489895913 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b4c06e030 0x7f3b4c1f1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:39.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.822+0000 7f3b50a00640 1 -- 172.21.15.19:0/489895913 >> 172.21.15.19:0/489895913 conn(0x7f3b4c06b570 msgr2=0x7f3b4c072870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:39.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.822+0000 7f3b50a00640 1 -- 172.21.15.19:0/489895913 shutdown_connections 2024-03-28T13:11:39.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:39.823+0000 7f3b50a00640 1 -- 172.21.15.19:0/489895913 wait complete. 2024-03-28T13:11:39.848 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-28T13:11:40.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.080+0000 7f98dda00640 1 Processor -- start 2024-03-28T13:11:40.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.081+0000 7f98dda00640 1 -- start start 2024-03-28T13:11:40.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.082+0000 7f98dda00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98d8155550 0x7f98d8155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.082+0000 7f98dda00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d815f800 0x7f98d80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.082+0000 7f98dda00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98d80b40b0 con 0x7f98d80d07c0 2024-03-28T13:11:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.082+0000 7f98dda00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f98d80d0c30 con 0x7f98d8155550 2024-03-28T13:11:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.082+0000 7f98dda00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f98d80b4390 con 0x7f98d815f800 2024-03-28T13:11:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.083+0000 7f98d6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98d8155550 0x7f98d8155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.083+0000 7f98d6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98d8155550 0x7f98d8155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44304/0 (socket says 172.21.15.19:44304) 2024-03-28T13:11:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.083+0000 7f98d6a00640 1 -- 172.21.15.19:0/328186608 learned_addr learned my addr 172.21.15.19:0/328186608 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.083+0000 7f98d6a00640 1 -- 172.21.15.19:0/328186608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d815f800 msgr2=0x7f98d80d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.083+0000 7f98d6a00640 1 --2- 172.21.15.19:0/328186608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d815f800 0x7f98d80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.083+0000 7f98d6a00640 1 -- 172.21.15.19:0/328186608 >> v1:172.21.15.188:6789/0 conn(0x7f98d80d07c0 legacy=0x7f98d80aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.083+0000 7f98d6a00640 1 -- 172.21.15.19:0/328186608 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d80b4670 con 0x7f98d8155550 2024-03-28T13:11:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.084+0000 7f98d6a00640 1 --2- 172.21.15.19:0/328186608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8155550 0x7f98d8155930 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009870 tx=0x7f98c802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1709097648a157e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.085+0000 7f98d5600640 1 -- 172.21.15.19:0/328186608 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c8035020 con 0x7f98d8155550 2024-03-28T13:11:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.085+0000 7f98d5600640 1 -- 172.21.15.19:0/328186608 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c8031640 con 0x7f98d8155550 2024-03-28T13:11:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.085+0000 7f98d5600640 1 -- 172.21.15.19:0/328186608 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c802d040 con 0x7f98d8155550 2024-03-28T13:11:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.085+0000 7f98dda00640 1 -- 172.21.15.19:0/328186608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8155550 msgr2=0x7f98d8155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.085+0000 7f98dda00640 1 --2- 172.21.15.19:0/328186608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8155550 0x7f98d8155930 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009870 tx=0x7f98c802b8f0 comp rx=0 tx=0).stop 2024-03-28T13:11:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.085+0000 7f98dda00640 1 -- 172.21.15.19:0/328186608 shutdown_connections 2024-03-28T13:11:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.085+0000 7f98dda00640 1 --2- 172.21.15.19:0/328186608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d815f800 0x7f98d80d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.085+0000 7f98dda00640 1 --2- 172.21.15.19:0/328186608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8155550 0x7f98d8155930 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.085+0000 7f98dda00640 1 -- 172.21.15.19:0/328186608 >> 172.21.15.19:0/328186608 conn(0x7f98d806b750 msgr2=0x7f98d806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.086+0000 7f98dda00640 1 -- 172.21.15.19:0/328186608 shutdown_connections 2024-03-28T13:11:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.086+0000 7f98dda00640 1 -- 172.21.15.19:0/328186608 wait complete. 2024-03-28T13:11:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.087+0000 7f98dda00640 1 Processor -- start 2024-03-28T13:11:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.087+0000 7f98dda00640 1 -- start start 2024-03-28T13:11:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.088+0000 7f98dda00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d80d07c0 0x7f98d81f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.088+0000 7f98dda00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8155550 0x7f98d814ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.088+0000 7f98dda00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98d815f800 0x7f98d809ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.088+0000 7f98dda00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98d815fc80 con 0x7f98d8155550 2024-03-28T13:11:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.088+0000 7f98dda00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f98d80b4c30 con 0x7f98d815f800 2024-03-28T13:11:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.088+0000 7f98dda00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f98d815ba20 con 0x7f98d80d07c0 2024-03-28T13:11:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.089+0000 7f98d6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8155550 0x7f98d814ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.089+0000 7f98d6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8155550 0x7f98d814ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44312/0 (socket says 172.21.15.19:44312) 2024-03-28T13:11:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.089+0000 7f98d6000640 1 -- 172.21.15.19:0/358037982 learned_addr learned my addr 172.21.15.19:0/358037982 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.089+0000 7f98d6a00640 1 --2- 172.21.15.19:0/358037982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d80d07c0 0x7f98d81f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.089+0000 7f98d7400640 1 --2- 172.21.15.19:0/358037982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98d815f800 0x7f98d809ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.089+0000 7f98d6000640 1 -- 172.21.15.19:0/358037982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d80d07c0 msgr2=0x7f98d81f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.089+0000 7f98d6000640 1 --2- 172.21.15.19:0/358037982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d80d07c0 0x7f98d81f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.089+0000 7f98d6000640 1 -- 172.21.15.19:0/358037982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98d815f800 msgr2=0x7f98d809ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.089+0000 7f98d6000640 1 --2- 172.21.15.19:0/358037982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98d815f800 0x7f98d809ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.089+0000 7f98d6000640 1 -- 172.21.15.19:0/358037982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98d809d910 con 0x7f98d8155550 2024-03-28T13:11:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.090+0000 7f98d6000640 1 --2- 172.21.15.19:0/358037982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8155550 0x7f98d814ea20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f98c000b4d0 tx=0x7f98c000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.090+0000 7f98cea00640 1 -- 172.21.15.19:0/358037982 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0014070 con 0x7f98d8155550 2024-03-28T13:11:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.090+0000 7f98cea00640 1 -- 172.21.15.19:0/358037982 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98c0010870 con 0x7f98d8155550 2024-03-28T13:11:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.090+0000 7f98cea00640 1 -- 172.21.15.19:0/358037982 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0010c60 con 0x7f98d8155550 2024-03-28T13:11:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.090+0000 7f98dda00640 1 -- 172.21.15.19:0/358037982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98d809dbf0 con 0x7f98d8155550 2024-03-28T13:11:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.090+0000 7f98dda00640 1 -- 172.21.15.19:0/358037982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98d809a2c0 con 0x7f98d8155550 2024-03-28T13:11:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.091+0000 7f98dda00640 1 -- 172.21.15.19:0/358037982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98d809a4c0 con 0x7f98d8155550 2024-03-28T13:11:40.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.092+0000 7f98cea00640 1 -- 172.21.15.19:0/358037982 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f98c0017070 con 0x7f98d8155550 2024-03-28T13:11:40.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.093+0000 7f98cea00640 1 --2- 172.21.15.19:0/358037982 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f98b406c530 0x7f98b406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:40.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.094+0000 7f98cea00640 1 -- 172.21.15.19:0/358037982 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f98c008b1d0 con 0x7f98d8155550 2024-03-28T13:11:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.094+0000 7f98d6a00640 1 --2- 172.21.15.19:0/358037982 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f98b406c530 0x7f98b406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.095+0000 7f98d6a00640 1 --2- 172.21.15.19:0/358037982 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f98b406c530 0x7f98b406e9f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f98c8036000 tx=0x7f98c803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.098+0000 7f98cea00640 1 -- 172.21.15.19:0/358037982 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f98d809a4c0 con 0x7f98d8155550 2024-03-28T13:11:40.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.304+0000 7f98dda00640 1 -- 172.21.15.19:0/358037982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f98d80685c0 con 0x7f98d8155550 2024-03-28T13:11:40.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.311+0000 7f98cea00640 1 -- 172.21.15.19:0/358037982 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f98c0059ae0 con 0x7f98d8155550 2024-03-28T13:11:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.314+0000 7f98dda00640 1 -- 172.21.15.19:0/358037982 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f98b406c530 msgr2=0x7f98b406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.314+0000 7f98dda00640 1 --2- 172.21.15.19:0/358037982 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f98b406c530 0x7f98b406e9f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f98c8036000 tx=0x7f98c803c040 comp rx=0 tx=0).stop 2024-03-28T13:11:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.314+0000 7f98dda00640 1 -- 172.21.15.19:0/358037982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8155550 msgr2=0x7f98d814ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.314+0000 7f98dda00640 1 --2- 172.21.15.19:0/358037982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8155550 0x7f98d814ea20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f98c000b4d0 tx=0x7f98c000b990 comp rx=0 tx=0).stop 2024-03-28T13:11:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.314+0000 7f98dda00640 1 -- 172.21.15.19:0/358037982 shutdown_connections 2024-03-28T13:11:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.314+0000 7f98dda00640 1 --2- 172.21.15.19:0/358037982 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f98b406c530 0x7f98b406e9f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.314+0000 7f98dda00640 1 --2- 172.21.15.19:0/358037982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98d815f800 0x7f98d809ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.314+0000 7f98dda00640 1 --2- 172.21.15.19:0/358037982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98d8155550 0x7f98d814ea20 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:40.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.314+0000 7f98dda00640 1 --2- 172.21.15.19:0/358037982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98d80d07c0 0x7f98d81f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:40.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.314+0000 7f98dda00640 1 -- 172.21.15.19:0/358037982 >> 172.21.15.19:0/358037982 conn(0x7f98d806b750 msgr2=0x7f98d806eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:40.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.315+0000 7f98dda00640 1 -- 172.21.15.19:0/358037982 shutdown_connections 2024-03-28T13:11:40.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:40.315+0000 7f98dda00640 1 -- 172.21.15.19:0/358037982 wait complete. 2024-03-28T13:11:40.333 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-28T13:11:40.333 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:11:40.447 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:11:40.570 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-28T13:11:40.570 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:11:40.570 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:11:40.629 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:11:40.629 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:11:40.629 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:11:40.686 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:11:40.687 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:11:40.687 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:11:40.748 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:11:40.748 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:11:40.748 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:11:40.802 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:11:40.803 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-28T13:11:40.803 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:11:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.027+0000 7f15c1000640 1 Processor -- start 2024-03-28T13:11:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.027+0000 7f15c1000640 1 -- start start 2024-03-28T13:11:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.028+0000 7f15c1000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15bc0af830 0x7f15bc073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.028+0000 7f15c1000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15bc0d0650 0x7f15bc0d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.028+0000 7f15c1000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15bc15fe80 con 0x7f15bc15c390 2024-03-28T13:11:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.028+0000 7f15c1000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f15bc15c770 con 0x7f15bc0d0650 2024-03-28T13:11:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.028+0000 7f15c1000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f15bc0ad120 con 0x7f15bc0af830 2024-03-28T13:11:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.028+0000 7f15b9e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f15bc15c390 0x7f15bc160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47592/0 (socket says 172.21.15.19:47592) 2024-03-28T13:11:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.028+0000 7f15b9e00640 1 -- 172.21.15.19:0/4001751853 learned_addr learned my addr 172.21.15.19:0/4001751853 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.028+0000 7f15ba800640 1 --2- 172.21.15.19:0/4001751853 >> v2:172.21.15.19:3300/0 conn(0x7f15bc0d0650 0x7f15bc0d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.029+0000 7f15b8a00640 1 -- 172.21.15.19:0/4001751853 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692891804 0 0) 0x7f15bc15fe80 con 0x7f15bc15c390 2024-03-28T13:11:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.029+0000 7f15b8a00640 1 -- 172.21.15.19:0/4001751853 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15a0003520 con 0x7f15bc15c390 2024-03-28T13:11:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.029+0000 7f15ba800640 1 -- 172.21.15.19:0/4001751853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15bc0af830 msgr2=0x7f15bc073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.029+0000 7f15ba800640 1 --2- 172.21.15.19:0/4001751853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15bc0af830 0x7f15bc073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.029+0000 7f15ba800640 1 -- 172.21.15.19:0/4001751853 >> v1:172.21.15.19:6789/0 conn(0x7f15bc15c390 legacy=0x7f15bc160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.029+0000 7f15ba800640 1 -- 172.21.15.19:0/4001751853 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc0ad400 con 0x7f15bc0d0650 2024-03-28T13:11:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.030+0000 7f15ba800640 1 --2- 172.21.15.19:0/4001751853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0d0650 0x7f15bc0d0b00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009d90 tx=0x7f15b402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56bcfee9c94f451 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.030+0000 7f15b8a00640 1 -- 172.21.15.19:0/4001751853 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4039070 con 0x7f15bc0d0650 2024-03-28T13:11:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.030+0000 7f15b8a00640 1 -- 172.21.15.19:0/4001751853 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15b402cc60 con 0x7f15bc0d0650 2024-03-28T13:11:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.030+0000 7f15b8a00640 1 -- 172.21.15.19:0/4001751853 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4034660 con 0x7f15bc0d0650 2024-03-28T13:11:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.031+0000 7f15c1000640 1 -- 172.21.15.19:0/4001751853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0d0650 msgr2=0x7f15bc0d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.031+0000 7f15c1000640 1 --2- 172.21.15.19:0/4001751853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0d0650 0x7f15bc0d0b00 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009d90 tx=0x7f15b402beb0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.031+0000 7f15c1000640 1 -- 172.21.15.19:0/4001751853 shutdown_connections 2024-03-28T13:11:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.031+0000 7f15c1000640 1 --2- 172.21.15.19:0/4001751853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0d0650 0x7f15bc0d0b00 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.031+0000 7f15c1000640 1 --2- 172.21.15.19:0/4001751853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15bc0af830 0x7f15bc073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.031+0000 7f15c1000640 1 -- 172.21.15.19:0/4001751853 >> 172.21.15.19:0/4001751853 conn(0x7f15bc06b990 msgr2=0x7f15bc06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.031+0000 7f15c1000640 1 -- 172.21.15.19:0/4001751853 shutdown_connections 2024-03-28T13:11:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.031+0000 7f15c1000640 1 -- 172.21.15.19:0/4001751853 wait complete. 2024-03-28T13:11:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.033+0000 7f15c1000640 1 Processor -- start 2024-03-28T13:11:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.033+0000 7f15c1000640 1 -- start start 2024-03-28T13:11:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.034+0000 7f15c1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0af830 0x7f15bc0b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.034+0000 7f15c1000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15bc0d0650 0x7f15bc15a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.034+0000 7f15c1000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15bc15c390 0x7f15bc1f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.034+0000 7f15c1000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc160d50 con 0x7f15bc0af830 2024-03-28T13:11:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.034+0000 7f15c1000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc160120 con 0x7f15bc15c390 2024-03-28T13:11:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.034+0000 7f15c1000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f15bc0cf980 con 0x7f15bc0d0650 2024-03-28T13:11:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.034+0000 7f15b9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0af830 0x7f15bc0b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.034+0000 7f15b9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0af830 0x7f15bc0b3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44336/0 (socket says 172.21.15.19:44336) 2024-03-28T13:11:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.034+0000 7f15b9e00640 1 -- 172.21.15.19:0/1925340235 learned_addr learned my addr 172.21.15.19:0/1925340235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.035+0000 7f15b9400640 1 --2- 172.21.15.19:0/1925340235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15bc0d0650 0x7f15bc15a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.035+0000 7f15ba800640 1 --2- 172.21.15.19:0/1925340235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15bc15c390 0x7f15bc1f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.035+0000 7f15b9e00640 1 -- 172.21.15.19:0/1925340235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15bc0d0650 msgr2=0x7f15bc15a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.035+0000 7f15b9e00640 1 --2- 172.21.15.19:0/1925340235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15bc0d0650 0x7f15bc15a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.035+0000 7f15b9e00640 1 -- 172.21.15.19:0/1925340235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15bc15c390 msgr2=0x7f15bc1f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.035+0000 7f15b9e00640 1 --2- 172.21.15.19:0/1925340235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15bc15c390 0x7f15bc1f3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.035+0000 7f15b9e00640 1 -- 172.21.15.19:0/1925340235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc15b440 con 0x7f15bc0af830 2024-03-28T13:11:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.035+0000 7f15b9e00640 1 --2- 172.21.15.19:0/1925340235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0af830 0x7f15bc0b3d70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00c390 tx=0x7f15ac00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.035+0000 7f15b2000640 1 -- 172.21.15.19:0/1925340235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac017070 con 0x7f15bc0af830 2024-03-28T13:11:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.035+0000 7f15b2000640 1 -- 172.21.15.19:0/1925340235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ac005ce0 con 0x7f15bc0af830 2024-03-28T13:11:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.035+0000 7f15b2000640 1 -- 172.21.15.19:0/1925340235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac0072f0 con 0x7f15bc0af830 2024-03-28T13:11:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.035+0000 7f15c1000640 1 -- 172.21.15.19:0/1925340235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc15aba0 con 0x7f15bc0af830 2024-03-28T13:11:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.036+0000 7f15c1000640 1 -- 172.21.15.19:0/1925340235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc15b160 con 0x7f15bc0af830 2024-03-28T13:11:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.037+0000 7f15c1000640 1 -- 172.21.15.19:0/1925340235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15bc0685c0 con 0x7f15bc0af830 2024-03-28T13:11:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.038+0000 7f15b2000640 1 -- 172.21.15.19:0/1925340235 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f15ac01a070 con 0x7f15bc0af830 2024-03-28T13:11:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.038+0000 7f15b2000640 1 --2- 172.21.15.19:0/1925340235 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f159c06c530 0x7f159c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.039+0000 7f15b2000640 1 -- 172.21.15.19:0/1925340235 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f15ac08a880 con 0x7f15bc0af830 2024-03-28T13:11:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.039+0000 7f15b9400640 1 --2- 172.21.15.19:0/1925340235 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f159c06c530 0x7f159c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.040+0000 7f15b9400640 1 --2- 172.21.15.19:0/1925340235 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f159c06c530 0x7f159c06e9f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f15bc156a20 tx=0x7f15a8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:41.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.043+0000 7f15b2000640 1 -- 172.21.15.19:0/1925340235 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f15ac059100 con 0x7f15bc0af830 2024-03-28T13:11:41.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.290+0000 7f15c1000640 1 -- 172.21.15.19:0/1925340235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15bc06f1b0 con 0x7f15bc0af830 2024-03-28T13:11:41.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.291+0000 7f15b2000640 1 -- 172.21.15.19:0/1925340235 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3872 (secure 0 0 0) 0x7f15ac058a00 con 0x7f15bc0af830 2024-03-28T13:11:41.290 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:41.290 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2683971950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2683971950},{"type":"v1","addr":"172.21.15.188:6836","nonce":2683971950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/709274582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":709274582},{"type":"v1","addr":"172.21.15.19:6835","nonce":709274582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5040,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6833/3420318181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3420318181},{"type":"v1","addr":"172.21.15.19:6833","nonce":3420318181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1215499881","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1215499881},{"type":"v1","addr":"172.21.15.188:6837","nonce":1215499881}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-03-28T13:11:41.291 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 19 2024-03-28T13:11:41.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.294+0000 7f15c1000640 1 -- 172.21.15.19:0/1925340235 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f159c06c530 msgr2=0x7f159c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:41.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.294+0000 7f15c1000640 1 --2- 172.21.15.19:0/1925340235 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f159c06c530 0x7f159c06e9f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f15bc156a20 tx=0x7f15a8008040 comp rx=0 tx=0).stop 2024-03-28T13:11:41.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.294+0000 7f15c1000640 1 -- 172.21.15.19:0/1925340235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0af830 msgr2=0x7f15bc0b3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:41.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.294+0000 7f15c1000640 1 --2- 172.21.15.19:0/1925340235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0af830 0x7f15bc0b3d70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00c390 tx=0x7f15ac00c850 comp rx=0 tx=0).stop 2024-03-28T13:11:41.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.294+0000 7f15c1000640 1 -- 172.21.15.19:0/1925340235 shutdown_connections 2024-03-28T13:11:41.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.294+0000 7f15c1000640 1 --2- 172.21.15.19:0/1925340235 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f159c06c530 0x7f159c06e9f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.294+0000 7f15c1000640 1 --2- 172.21.15.19:0/1925340235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15bc15c390 0x7f15bc1f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.294+0000 7f15c1000640 1 --2- 172.21.15.19:0/1925340235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15bc0d0650 0x7f15bc15a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.294+0000 7f15c1000640 1 --2- 172.21.15.19:0/1925340235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc0af830 0x7f15bc0b3d70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.294+0000 7f15c1000640 1 -- 172.21.15.19:0/1925340235 >> 172.21.15.19:0/1925340235 conn(0x7f15bc06b990 msgr2=0x7f15bc0b4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:41.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.294+0000 7f15c1000640 1 -- 172.21.15.19:0/1925340235 shutdown_connections 2024-03-28T13:11:41.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.295+0000 7f15c1000640 1 -- 172.21.15.19:0/1925340235 wait complete. 2024-03-28T13:11:41.314 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-28T13:11:41.314 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-28T13:11:41.315 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-28T13:11:41.315 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:11:41.315 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-28T13:11:41.315 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-28T13:11:41.315 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-28T13:11:41.315 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:11:41.315 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-28T13:11:41.316 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-28T13:11:41.316 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-28T13:11:41.316 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:11:41.316 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-28T13:11:41.316 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-28T13:11:41.316 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-28T13:11:41.316 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:11:41.336 INFO:tasks.ceph.mds.c:Stopped 2024-03-28T13:11:41.336 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-28T13:11:41.337 INFO:tasks.ceph.mds.b:Stopped 2024-03-28T13:11:41.337 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-28T13:11:41.344 INFO:tasks.ceph.mds.a:Stopped 2024-03-28T13:11:41.344 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-28T13:11:41.345 INFO:tasks.ceph.mds.d:Stopped 2024-03-28T13:11:41.346 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-28T13:11:41.392 INFO:tasks.ceph.mds.c:Started 2024-03-28T13:11:41.392 DEBUG:teuthology.parallel:result is None 2024-03-28T13:11:41.393 INFO:tasks.ceph.mds.b:Started 2024-03-28T13:11:41.393 DEBUG:teuthology.parallel:result is None 2024-03-28T13:11:41.398 INFO:tasks.ceph.mds.a:Started 2024-03-28T13:11:41.398 DEBUG:teuthology.parallel:result is None 2024-03-28T13:11:41.399 INFO:tasks.ceph.mds.d:Started 2024-03-28T13:11:41.399 DEBUG:teuthology.parallel:result is None 2024-03-28T13:11:41.399 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-28T13:11:41.553 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-03-28T13:11:41.554 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-03-28T13:11:41.555 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-03-28T13:11:41.556 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-03-28T13:11:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.594+0000 7f3187e00640 1 Processor -- start 2024-03-28T13:11:41.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.594+0000 7f3187e00640 1 -- start start 2024-03-28T13:11:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.595+0000 7f3187e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f31881442c0 0x7f31880ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.595+0000 7f3187e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f31880cefe0 0x7f31880cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.595+0000 7f3187e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3188071460 con 0x7f31880d00d0 2024-03-28T13:11:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.596+0000 7f3187e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3188071d20 con 0x7f31880cefe0 2024-03-28T13:11:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.596+0000 7f3187e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f31880cf420 con 0x7f31881442c0 2024-03-28T13:11:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.596+0000 7f3187400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f31880d00d0 0x7f318806e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47596/0 (socket says 172.21.15.19:47596) 2024-03-28T13:11:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.596+0000 7f3187400640 1 -- 172.21.15.19:0/3247481979 learned_addr learned my addr 172.21.15.19:0/3247481979 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.596+0000 7f3186000640 1 --2- 172.21.15.19:0/3247481979 >> v2:172.21.15.19:3300/0 conn(0x7f31880cefe0 0x7f31880cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.596+0000 7f3186a00640 1 --2- 172.21.15.19:0/3247481979 >> v2:172.21.15.188:3300/0 conn(0x7f31881442c0 0x7f31880ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.596+0000 7f3185600640 1 -- 172.21.15.19:0/3247481979 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1369743633 0 0) 0x7f3188071460 con 0x7f31880d00d0 2024-03-28T13:11:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.596+0000 7f3186000640 1 -- 172.21.15.19:0/3247481979 >> v2:172.21.15.188:3300/0 conn(0x7f31881442c0 msgr2=0x7f31880ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.596+0000 7f3186000640 1 --2- 172.21.15.19:0/3247481979 >> v2:172.21.15.188:3300/0 conn(0x7f31881442c0 0x7f31880ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.596+0000 7f3186000640 1 -- 172.21.15.19:0/3247481979 >> v1:172.21.15.19:6789/0 conn(0x7f31880d00d0 legacy=0x7f318806e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.596+0000 7f3186000640 1 -- 172.21.15.19:0/3247481979 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31880721b0 con 0x7f31880cefe0 2024-03-28T13:11:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.598+0000 7f3186000640 1 --2- 172.21.15.19:0/3247481979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31880cefe0 0x7f31880cfb90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f31880d0ae0 tx=0x7f317402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5714f0c7e34551bc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.598+0000 7f3185600640 1 -- 172.21.15.19:0/3247481979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3174034070 con 0x7f31880cefe0 2024-03-28T13:11:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.598+0000 7f3185600640 1 -- 172.21.15.19:0/3247481979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3174030610 con 0x7f31880cefe0 2024-03-28T13:11:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.598+0000 7f3185600640 1 -- 172.21.15.19:0/3247481979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31740309e0 con 0x7f31880cefe0 2024-03-28T13:11:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.598+0000 7f3187e00640 1 -- 172.21.15.19:0/3247481979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31880cefe0 msgr2=0x7f31880cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.598+0000 7f3187e00640 1 --2- 172.21.15.19:0/3247481979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31880cefe0 0x7f31880cfb90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f31880d0ae0 tx=0x7f317402b960 comp rx=0 tx=0).stop 2024-03-28T13:11:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.599+0000 7f3187e00640 1 -- 172.21.15.19:0/3247481979 shutdown_connections 2024-03-28T13:11:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.599+0000 7f3187e00640 1 --2- 172.21.15.19:0/3247481979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31880cefe0 0x7f31880cfb90 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.599+0000 7f3187e00640 1 --2- 172.21.15.19:0/3247481979 >> v2:172.21.15.188:3300/0 conn(0x7f31881442c0 0x7f31880ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.599+0000 7f3187e00640 1 -- 172.21.15.19:0/3247481979 >> 172.21.15.19:0/3247481979 conn(0x7f31880ccad0 msgr2=0x7f31880cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.599+0000 7f3187e00640 1 -- 172.21.15.19:0/3247481979 shutdown_connections 2024-03-28T13:11:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.599+0000 7f3187e00640 1 -- 172.21.15.19:0/3247481979 wait complete. 2024-03-28T13:11:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.600+0000 7f3187e00640 1 Processor -- start 2024-03-28T13:11:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.601+0000 7f3187e00640 1 -- start start 2024-03-28T13:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.601+0000 7f3187e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31880cefe0 0x7f31880b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.601+0000 7f3187e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31880d00d0 0x7f318815e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.601+0000 7f3187e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31881442c0 0x7f31880b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.601+0000 7f3187e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f31880d0770 con 0x7f31880cefe0 2024-03-28T13:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.601+0000 7f3187e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f31880719b0 con 0x7f31881442c0 2024-03-28T13:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.601+0000 7f3187e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f31881446a0 con 0x7f31880d00d0 2024-03-28T13:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.602+0000 7f3186a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31880cefe0 0x7f31880b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.602+0000 7f3186a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31880cefe0 0x7f31880b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44394/0 (socket says 172.21.15.19:44394) 2024-03-28T13:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.602+0000 7f3186a00640 1 -- 172.21.15.19:0/91134568 learned_addr learned my addr 172.21.15.19:0/91134568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.602+0000 7f3187400640 1 --2- 172.21.15.19:0/91134568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31881442c0 0x7f31880b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.602+0000 7f3186000640 1 --2- 172.21.15.19:0/91134568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31880d00d0 0x7f318815e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.602+0000 7f3186a00640 1 -- 172.21.15.19:0/91134568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31880d00d0 msgr2=0x7f318815e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.602+0000 7f3186a00640 1 --2- 172.21.15.19:0/91134568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31880d00d0 0x7f318815e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.602+0000 7f3186a00640 1 -- 172.21.15.19:0/91134568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31881442c0 msgr2=0x7f31880b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.602+0000 7f3186a00640 1 --2- 172.21.15.19:0/91134568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31881442c0 0x7f31880b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.602+0000 7f3186a00640 1 -- 172.21.15.19:0/91134568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31880b0c70 con 0x7f31880cefe0 2024-03-28T13:11:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.603+0000 7f3186a00640 1 --2- 172.21.15.19:0/91134568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31880cefe0 0x7f31880b2680 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f317800e9b0 tx=0x7f317800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.603+0000 7f317ea00640 1 -- 172.21.15.19:0/91134568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3178017070 con 0x7f31880cefe0 2024-03-28T13:11:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.603+0000 7f317ea00640 1 -- 172.21.15.19:0/91134568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f317800c9a0 con 0x7f31880cefe0 2024-03-28T13:11:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.603+0000 7f317ea00640 1 -- 172.21.15.19:0/91134568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f317800cd90 con 0x7f31880cefe0 2024-03-28T13:11:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.603+0000 7f3187e00640 1 -- 172.21.15.19:0/91134568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31880b0f50 con 0x7f31880cefe0 2024-03-28T13:11:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.603+0000 7f3187e00640 1 -- 172.21.15.19:0/91134568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31881f5230 con 0x7f31880cefe0 2024-03-28T13:11:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.604+0000 7f3187e00640 1 -- 172.21.15.19:0/91134568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3188075940 con 0x7f31880cefe0 2024-03-28T13:11:41.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.605+0000 7f317ea00640 1 -- 172.21.15.19:0/91134568 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f317801a070 con 0x7f31880cefe0 2024-03-28T13:11:41.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.606+0000 7f317ea00640 1 --2- 172.21.15.19:0/91134568 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f316806c530 0x7f316806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:41.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.606+0000 7f317ea00640 1 -- 172.21.15.19:0/91134568 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f317808b7e0 con 0x7f31880cefe0 2024-03-28T13:11:41.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.606+0000 7f3186000640 1 --2- 172.21.15.19:0/91134568 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f316806c530 0x7f316806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:41.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.607+0000 7f3186000640 1 --2- 172.21.15.19:0/91134568 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f316806c530 0x7f316806e9f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3174036000 tx=0x7f317403b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:41.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.610+0000 7f317ea00640 1 -- 172.21.15.19:0/91134568 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f317805a0f0 con 0x7f31880cefe0 2024-03-28T13:11:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.826+0000 7f3187e00640 1 -- 172.21.15.19:0/91134568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f31880b1470 con 0x7f31880cefe0 2024-03-28T13:11:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.855+0000 7f317ea00640 1 -- 172.21.15.19:0/91134568 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v43) ==== 97+0+0 (secure 0 0 0) 0x7f31780599f0 con 0x7f31880cefe0 2024-03-28T13:11:41.858 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-03-28T13:11:41.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.858+0000 7f3187e00640 1 -- 172.21.15.19:0/91134568 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f316806c530 msgr2=0x7f316806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:41.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.858+0000 7f3187e00640 1 --2- 172.21.15.19:0/91134568 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f316806c530 0x7f316806e9f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3174036000 tx=0x7f317403b040 comp rx=0 tx=0).stop 2024-03-28T13:11:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.858+0000 7f3187e00640 1 -- 172.21.15.19:0/91134568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31880cefe0 msgr2=0x7f31880b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.858+0000 7f3187e00640 1 --2- 172.21.15.19:0/91134568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31880cefe0 0x7f31880b2680 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f317800e9b0 tx=0x7f317800ee70 comp rx=0 tx=0).stop 2024-03-28T13:11:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.859+0000 7f3187e00640 1 -- 172.21.15.19:0/91134568 shutdown_connections 2024-03-28T13:11:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.859+0000 7f3187e00640 1 --2- 172.21.15.19:0/91134568 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f316806c530 0x7f316806e9f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.859+0000 7f3187e00640 1 --2- 172.21.15.19:0/91134568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31881442c0 0x7f31880b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.859+0000 7f3187e00640 1 --2- 172.21.15.19:0/91134568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31880d00d0 0x7f318815e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.859+0000 7f3187e00640 1 --2- 172.21.15.19:0/91134568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31880cefe0 0x7f31880b2680 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.859+0000 7f3187e00640 1 -- 172.21.15.19:0/91134568 >> 172.21.15.19:0/91134568 conn(0x7f31880ccad0 msgr2=0x7f31880b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.859+0000 7f3187e00640 1 -- 172.21.15.19:0/91134568 shutdown_connections 2024-03-28T13:11:41.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:41.859+0000 7f3187e00640 1 -- 172.21.15.19:0/91134568 wait complete. 2024-03-28T13:11:41.882 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-28T13:11:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.106+0000 7f2ea3400640 1 Processor -- start 2024-03-28T13:11:42.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.106+0000 7f2ea3400640 1 -- start start 2024-03-28T13:11:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.108+0000 7f2ea3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2e9c158580 0x7f2e9c1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.108+0000 7f2ea3400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e9c15ede0 con 0x7f2e9c0b4650 2024-03-28T13:11:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.108+0000 7f2ea3400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e9c141d80 con 0x7f2e9c0b4a30 2024-03-28T13:11:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.108+0000 7f2ea3400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2e9c0b6320 con 0x7f2e9c158580 2024-03-28T13:11:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.108+0000 7f2ea0a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2e9c0b4650 0x7f2e9c06c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47612/0 (socket says 172.21.15.19:47612) 2024-03-28T13:11:42.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.108+0000 7f2ea0a00640 1 -- 172.21.15.19:0/1454150074 learned_addr learned my addr 172.21.15.19:0/1454150074 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:42.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.108+0000 7f2ea1400640 1 --2- 172.21.15.19:0/1454150074 >> v2:172.21.15.19:3300/0 conn(0x7f2e9c158580 0x7f2e9c1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:42.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.109+0000 7f2e9b400640 1 -- 172.21.15.19:0/1454150074 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 182020319 0 0) 0x7f2e9c15ede0 con 0x7f2e9c0b4650 2024-03-28T13:11:42.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.109+0000 7f2e9b400640 1 -- 172.21.15.19:0/1454150074 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e80003520 con 0x7f2e9c0b4650 2024-03-28T13:11:42.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.109+0000 7f2ea1400640 1 -- 172.21.15.19:0/1454150074 >> v1:172.21.15.188:6789/0 conn(0x7f2e9c0b4a30 legacy=0x7f2e9c0b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:42.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.109+0000 7f2ea1400640 1 -- 172.21.15.19:0/1454150074 >> v1:172.21.15.19:6789/0 conn(0x7f2e9c0b4650 legacy=0x7f2e9c06c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:42.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.109+0000 7f2ea1400640 1 -- 172.21.15.19:0/1454150074 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9c142060 con 0x7f2e9c158580 2024-03-28T13:11:42.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.110+0000 7f2ea1400640 1 --2- 172.21.15.19:0/1454150074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e9c158580 0x7f2e9c1589a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009d90 tx=0x7f2e9402bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d2b34d4ca9355c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:42.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.110+0000 7f2e9b400640 1 -- 172.21.15.19:0/1454150074 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94039070 con 0x7f2e9c158580 2024-03-28T13:11:42.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.110+0000 7f2e9b400640 1 -- 172.21.15.19:0/1454150074 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e9402ccf0 con 0x7f2e9c158580 2024-03-28T13:11:42.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.110+0000 7f2e9b400640 1 -- 172.21.15.19:0/1454150074 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94034640 con 0x7f2e9c158580 2024-03-28T13:11:42.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.111+0000 7f2ea3400640 1 -- 172.21.15.19:0/1454150074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e9c158580 msgr2=0x7f2e9c1589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:42.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.111+0000 7f2ea3400640 1 --2- 172.21.15.19:0/1454150074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e9c158580 0x7f2e9c1589a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009d90 tx=0x7f2e9402bf40 comp rx=0 tx=0).stop 2024-03-28T13:11:42.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.111+0000 7f2ea3400640 1 -- 172.21.15.19:0/1454150074 shutdown_connections 2024-03-28T13:11:42.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.111+0000 7f2ea3400640 1 --2- 172.21.15.19:0/1454150074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e9c158580 0x7f2e9c1589a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.111+0000 7f2ea3400640 1 -- 172.21.15.19:0/1454150074 >> 172.21.15.19:0/1454150074 conn(0x7f2e9c067670 msgr2=0x7f2e9c067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.111+0000 7f2ea3400640 1 -- 172.21.15.19:0/1454150074 shutdown_connections 2024-03-28T13:11:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.111+0000 7f2ea3400640 1 -- 172.21.15.19:0/1454150074 wait complete. 2024-03-28T13:11:42.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.113+0000 7f2ea3400640 1 Processor -- start 2024-03-28T13:11:42.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.113+0000 7f2ea3400640 1 -- start start 2024-03-28T13:11:42.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.114+0000 7f2ea3400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e9c0b4650 0x7f2e9c0ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:42.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.114+0000 7f2ea3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e9c0b4a30 0x7f2e9c0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:42.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.114+0000 7f2ea3400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e9c158580 0x7f2e9c1597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:42.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.114+0000 7f2ea3400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9c142620 con 0x7f2e9c0b4a30 2024-03-28T13:11:42.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.114+0000 7f2ea3400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9c141b30 con 0x7f2e9c158580 2024-03-28T13:11:42.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.114+0000 7f2ea3400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2e9c142900 con 0x7f2e9c0b4650 2024-03-28T13:11:42.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.114+0000 7f2e9be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e9c0b4a30 0x7f2e9c0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:42.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.114+0000 7f2e9be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e9c0b4a30 0x7f2e9c0cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44414/0 (socket says 172.21.15.19:44414) 2024-03-28T13:11:42.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.114+0000 7f2e9be00640 1 -- 172.21.15.19:0/1537969552 learned_addr learned my addr 172.21.15.19:0/1537969552 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:42.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.115+0000 7f2ea0a00640 1 --2- 172.21.15.19:0/1537969552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e9c0b4650 0x7f2e9c0ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.115+0000 7f2ea1400640 1 --2- 172.21.15.19:0/1537969552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e9c158580 0x7f2e9c1597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.115+0000 7f2e9be00640 1 -- 172.21.15.19:0/1537969552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e9c0b4650 msgr2=0x7f2e9c0ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.115+0000 7f2e9be00640 1 --2- 172.21.15.19:0/1537969552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e9c0b4650 0x7f2e9c0ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.115+0000 7f2e9be00640 1 -- 172.21.15.19:0/1537969552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e9c158580 msgr2=0x7f2e9c1597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.115+0000 7f2e9be00640 1 --2- 172.21.15.19:0/1537969552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e9c158580 0x7f2e9c1597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.115+0000 7f2e9be00640 1 -- 172.21.15.19:0/1537969552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9c0cbc80 con 0x7f2e9c0b4a30 2024-03-28T13:11:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.115+0000 7f2e9be00640 1 --2- 172.21.15.19:0/1537969552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e9c0b4a30 0x7f2e9c0cb580 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400c470 tx=0x7f2e84001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.115+0000 7f2e98c00640 1 -- 172.21.15.19:0/1537969552 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e840052d0 con 0x7f2e9c0b4a30 2024-03-28T13:11:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.115+0000 7f2e98c00640 1 -- 172.21.15.19:0/1537969552 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e84005470 con 0x7f2e9c0b4a30 2024-03-28T13:11:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.115+0000 7f2e98c00640 1 -- 172.21.15.19:0/1537969552 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e84007260 con 0x7f2e9c0b4a30 2024-03-28T13:11:42.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.115+0000 7f2ea3400640 1 -- 172.21.15.19:0/1537969552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e9c0cbf60 con 0x7f2e9c0b4a30 2024-03-28T13:11:42.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.116+0000 7f2ea3400640 1 -- 172.21.15.19:0/1537969552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e9c0cc520 con 0x7f2e9c0b4a30 2024-03-28T13:11:42.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.117+0000 7f2ea3400640 1 -- 172.21.15.19:0/1537969552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e9c074f30 con 0x7f2e9c0b4a30 2024-03-28T13:11:42.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.118+0000 7f2e98c00640 1 -- 172.21.15.19:0/1537969552 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2e84007400 con 0x7f2e9c0b4a30 2024-03-28T13:11:42.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.119+0000 7f2e98c00640 1 --2- 172.21.15.19:0/1537969552 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e7806c530 0x7f2e7806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:42.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.119+0000 7f2e98c00640 1 -- 172.21.15.19:0/1537969552 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4857+0+0 (secure 0 0 0) 0x7f2e8408a960 con 0x7f2e9c0b4a30 2024-03-28T13:11:42.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.119+0000 7f2ea0a00640 1 --2- 172.21.15.19:0/1537969552 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e7806c530 0x7f2e7806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:42.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.120+0000 7f2ea0a00640 1 --2- 172.21.15.19:0/1537969552 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e7806c530 0x7f2e7806e9f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c005880 tx=0x7f2e8c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:42.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.124+0000 7f2e98c00640 1 -- 172.21.15.19:0/1537969552 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2e84059230 con 0x7f2e9c0b4a30 2024-03-28T13:11:42.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.410+0000 7f2ea3400640 1 -- 172.21.15.19:0/1537969552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f2e9c0685c0 con 0x7f2e9c0b4a30 2024-03-28T13:11:42.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.411+0000 7f2e98c00640 1 -- 172.21.15.19:0/1537969552 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7f2e84058b30 con 0x7f2e9c0b4a30 2024-03-28T13:11:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:42.411 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:11:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-03-28T13:11:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-03-28T13:11:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmNRIkABAAq/nJioDZjp1zFWAYOLPBYw==", 2024-03-28T13:11:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:11:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:11:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:11:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:11:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-03-28T13:11:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmc10CIxAAQWCSjH0hW3KUk8/qQm91QA==", 2024-03-28T13:11:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:11:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:11:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:11:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:11:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-03-28T13:11:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmmlUoEhAA4FUGXoZ/h/ve5fThJNdE2Q==", 2024-03-28T13:11:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:11:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:11:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:11:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:11:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-03-28T13:11:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmWDRnMxAA34LiLI79N5fVJnbcPacbQA==", 2024-03-28T13:11:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:11:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:11:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:11:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:11:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-03-28T13:11:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDYawVmxhwWLBAAm7C56UGConFx2D7Lbq2SSA==", 2024-03-28T13:11:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:11:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:11:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:11:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-03-28T13:11:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDbawVmC2BiNxAALQrAcXU44m2dTCjks5yy/Q==", 2024-03-28T13:11:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:11:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:11:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:11:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-03-28T13:11:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfawVmozrJFRAA7YV1uMucIW7H5SI4KY8aDw==", 2024-03-28T13:11:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:11:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:11:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:11:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-03-28T13:11:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiawVmccbTMBAATxk24Bf4LuQSqU92HdwgUg==", 2024-03-28T13:11:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:11:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:11:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:11:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-03-28T13:11:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDpawVmdwnILRAAjUEBwnBEeSltu2CLRGhmDQ==", 2024-03-28T13:11:42.417 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.417 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:11:42.417 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:11:42.417 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:11:42.417 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.417 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-03-28T13:11:42.417 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDtawVmkwW5ARAAZJ/Nd/Wtq7LcbUjeqPNUag==", 2024-03-28T13:11:42.417 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.417 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:11:42.417 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:11:42.417 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:11:42.417 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.418 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-03-28T13:11:42.418 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDwawVm254wKRAAmCYODqSd2Whn4YKQzywrng==", 2024-03-28T13:11:42.418 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.418 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:11:42.418 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:11:42.418 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:11:42.418 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.418 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-03-28T13:11:42.418 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDzawVmV8z+MhAA9JeF55K7ph1mQRWalT7XhA==", 2024-03-28T13:11:42.419 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.419 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:11:42.419 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:11:42.419 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:11:42.419 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.419 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-03-28T13:11:42.419 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q==", 2024-03-28T13:11:42.419 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.419 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:11:42.419 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-03-28T13:11:42.419 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-28T13:11:42.419 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmJYGjHBAAAx2TtR0Gb3fafnZU26FCMA==", 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmE7V/ERAAYcdvolIMDd5ff+pCn7hN1w==", 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.420 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:11:42.421 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:11:42.421 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:11:42.421 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:11:42.421 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.421 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-03-28T13:11:42.421 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmk6q9JhAAeC3JHQDnK8IXGVoK6qGEFA==", 2024-03-28T13:11:42.421 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.421 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:11:42.421 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:11:42.421 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:11:42.421 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:11:42.421 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.422 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-03-28T13:11:42.422 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDRawVm9IzFLRAA5i1Yl7LQ7eNNHp7v4H2kog==", 2024-03-28T13:11:42.422 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.422 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:11:42.422 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:11:42.422 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:11:42.422 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:11:42.422 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.422 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-03-28T13:11:42.422 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDSawVm5H1FLxAAtyjOPcCEUQyvTpBdu3Fn3g==", 2024-03-28T13:11:42.422 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:11:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-03-28T13:11:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:11:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-03-28T13:11:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDSawVmebzfIRAAA0+qIIaprr4SfEZT6qRooA==", 2024-03-28T13:11:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:11:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:11:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-03-28T13:11:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:11:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:42.423 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-03-28T13:11:42.424 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:11:42.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.414+0000 7f2ea3400640 1 -- 172.21.15.19:0/1537969552 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e7806c530 msgr2=0x7f2e7806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:42.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.414+0000 7f2ea3400640 1 --2- 172.21.15.19:0/1537969552 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e7806c530 0x7f2e7806e9f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c005880 tx=0x7f2e8c009040 comp rx=0 tx=0).stop 2024-03-28T13:11:42.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.414+0000 7f2ea3400640 1 -- 172.21.15.19:0/1537969552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e9c0b4a30 msgr2=0x7f2e9c0cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:42.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.414+0000 7f2ea3400640 1 --2- 172.21.15.19:0/1537969552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e9c0b4a30 0x7f2e9c0cb580 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400c470 tx=0x7f2e84001e70 comp rx=0 tx=0).stop 2024-03-28T13:11:42.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.414+0000 7f2ea3400640 1 -- 172.21.15.19:0/1537969552 shutdown_connections 2024-03-28T13:11:42.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.414+0000 7f2ea3400640 1 --2- 172.21.15.19:0/1537969552 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e7806c530 0x7f2e7806e9f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.414+0000 7f2ea3400640 1 --2- 172.21.15.19:0/1537969552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e9c158580 0x7f2e9c1597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.414+0000 7f2ea3400640 1 --2- 172.21.15.19:0/1537969552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e9c0b4a30 0x7f2e9c0cb580 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.414+0000 7f2ea3400640 1 --2- 172.21.15.19:0/1537969552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e9c0b4650 0x7f2e9c0ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.414+0000 7f2ea3400640 1 -- 172.21.15.19:0/1537969552 >> 172.21.15.19:0/1537969552 conn(0x7f2e9c067670 msgr2=0x7f2e9c144440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:42.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.414+0000 7f2ea3400640 1 -- 172.21.15.19:0/1537969552 shutdown_connections 2024-03-28T13:11:42.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.414+0000 7f2ea3400640 1 -- 172.21.15.19:0/1537969552 wait complete. 2024-03-28T13:11:42.434 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:11:42.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.661+0000 7fb56a600640 1 Processor -- start 2024-03-28T13:11:42.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.661+0000 7fb56a600640 1 -- start start 2024-03-28T13:11:42.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.663+0000 7fb56a600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb56414ba20 0x7fb5640b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:42.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.663+0000 7fb56a600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb56414c1b0 0x7fb564166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:42.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.663+0000 7fb56a600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5640b33e0 con 0x7fb5640cfda0 2024-03-28T13:11:42.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.663+0000 7fb56a600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb5640cde20 con 0x7fb56414ba20 2024-03-28T13:11:42.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.663+0000 7fb56a600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb564166ae0 con 0x7fb56414c1b0 2024-03-28T13:11:42.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.663+0000 7fb563e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb56414c1b0 0x7fb564166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:42.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.664+0000 7fb569c00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fb5640cfda0 0x7fb5640ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:52696/0 (socket says 172.21.15.19:52696) 2024-03-28T13:11:42.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.664+0000 7fb569c00640 1 -- 172.21.15.19:0/2086983069 learned_addr learned my addr 172.21.15.19:0/2086983069 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:42.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.664+0000 7fb563e00640 1 --2- 172.21.15.19:0/2086983069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb56414c1b0 0x7fb564166290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:60040/0 (socket says 172.21.15.19:60040) 2024-03-28T13:11:42.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.664+0000 7fb569200640 1 --2- 172.21.15.19:0/2086983069 >> v2:172.21.15.188:3300/0 conn(0x7fb56414ba20 0x7fb5640b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.664+0000 7fb563e00640 1 -- 172.21.15.19:0/2086983069 >> v2:172.21.15.188:3300/0 conn(0x7fb56414ba20 msgr2=0x7fb5640b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.664+0000 7fb563e00640 1 --2- 172.21.15.19:0/2086983069 >> v2:172.21.15.188:3300/0 conn(0x7fb56414ba20 0x7fb5640b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.664+0000 7fb563e00640 1 -- 172.21.15.19:0/2086983069 >> v1:172.21.15.188:6789/0 conn(0x7fb5640cfda0 legacy=0x7fb5640ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.664+0000 7fb563e00640 1 -- 172.21.15.19:0/2086983069 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb564166dc0 con 0x7fb56414c1b0 2024-03-28T13:11:42.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.664+0000 7fb563400640 1 -- 172.21.15.19:0/2086983069 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2632410669 0 0) 0x7fb5640b33e0 con 0x7fb5640cfda0 2024-03-28T13:11:42.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.666+0000 7fb563e00640 1 --2- 172.21.15.19:0/2086983069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb56414c1b0 0x7fb564166290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb5500097a0 tx=0x7fb55002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f906acd7e6645003 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:42.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.667+0000 7fb563400640 1 -- 172.21.15.19:0/2086983069 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550035020 con 0x7fb56414c1b0 2024-03-28T13:11:42.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.667+0000 7fb563400640 1 -- 172.21.15.19:0/2086983069 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb550031830 con 0x7fb56414c1b0 2024-03-28T13:11:42.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.667+0000 7fb563400640 1 -- 172.21.15.19:0/2086983069 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550031c00 con 0x7fb56414c1b0 2024-03-28T13:11:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.667+0000 7fb56a600640 1 -- 172.21.15.19:0/2086983069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb56414c1b0 msgr2=0x7fb564166290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.667+0000 7fb56a600640 1 --2- 172.21.15.19:0/2086983069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb56414c1b0 0x7fb564166290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb5500097a0 tx=0x7fb55002b8d0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.667+0000 7fb56a600640 1 -- 172.21.15.19:0/2086983069 shutdown_connections 2024-03-28T13:11:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.667+0000 7fb56a600640 1 --2- 172.21.15.19:0/2086983069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb56414c1b0 0x7fb564166290 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.667+0000 7fb56a600640 1 --2- 172.21.15.19:0/2086983069 >> v2:172.21.15.188:3300/0 conn(0x7fb56414ba20 0x7fb5640b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.667+0000 7fb56a600640 1 -- 172.21.15.19:0/2086983069 >> 172.21.15.19:0/2086983069 conn(0x7fb5640ccad0 msgr2=0x7fb5640cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.668+0000 7fb56a600640 1 -- 172.21.15.19:0/2086983069 shutdown_connections 2024-03-28T13:11:42.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.668+0000 7fb56a600640 1 -- 172.21.15.19:0/2086983069 wait complete. 2024-03-28T13:11:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.669+0000 7fb56a600640 1 Processor -- start 2024-03-28T13:11:42.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.670+0000 7fb56a600640 1 -- start start 2024-03-28T13:11:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.670+0000 7fb56a600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5640cfda0 0x7fb5641620b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.670+0000 7fb56a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56414ba20 0x7fb5640ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:42.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.671+0000 7fb56a600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb56414c1b0 0x7fb564165320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.671+0000 7fb56a600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb5641667d0 con 0x7fb56414ba20 2024-03-28T13:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.671+0000 7fb56a600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5640ce0c0 con 0x7fb56414c1b0 2024-03-28T13:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.671+0000 7fb56a600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb56414be00 con 0x7fb5640cfda0 2024-03-28T13:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.671+0000 7fb563e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56414ba20 0x7fb5640ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.671+0000 7fb563e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56414ba20 0x7fb5640ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44420/0 (socket says 172.21.15.19:44420) 2024-03-28T13:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.671+0000 7fb563e00640 1 -- 172.21.15.19:0/3674926579 learned_addr learned my addr 172.21.15.19:0/3674926579 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.671+0000 7fb569200640 1 --2- 172.21.15.19:0/3674926579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5640cfda0 0x7fb5641620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.671+0000 7fb569c00640 1 --2- 172.21.15.19:0/3674926579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb56414c1b0 0x7fb564165320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.671+0000 7fb563e00640 1 -- 172.21.15.19:0/3674926579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5640cfda0 msgr2=0x7fb5641620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.671+0000 7fb563e00640 1 --2- 172.21.15.19:0/3674926579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5640cfda0 0x7fb5641620b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.671+0000 7fb563e00640 1 -- 172.21.15.19:0/3674926579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb56414c1b0 msgr2=0x7fb564165320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.671+0000 7fb563e00640 1 --2- 172.21.15.19:0/3674926579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb56414c1b0 0x7fb564165320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.671+0000 7fb563e00640 1 -- 172.21.15.19:0/3674926579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5640ad2e0 con 0x7fb56414ba20 2024-03-28T13:11:42.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.671+0000 7fb569c00640 1 --2- 172.21.15.19:0/3674926579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb56414c1b0 0x7fb564165320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:42.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.672+0000 7fb563e00640 1 --2- 172.21.15.19:0/3674926579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56414ba20 0x7fb5640ae620 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb5500098c0 tx=0x7fb5500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:42.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.672+0000 7fb560c00640 1 -- 172.21.15.19:0/3674926579 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550040070 con 0x7fb56414ba20 2024-03-28T13:11:42.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.672+0000 7fb560c00640 1 -- 172.21.15.19:0/3674926579 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb55003e070 con 0x7fb56414ba20 2024-03-28T13:11:42.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.672+0000 7fb560c00640 1 -- 172.21.15.19:0/3674926579 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550035040 con 0x7fb56414ba20 2024-03-28T13:11:42.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.672+0000 7fb56a600640 1 -- 172.21.15.19:0/3674926579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5640ad580 con 0x7fb56414ba20 2024-03-28T13:11:42.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.672+0000 7fb56a600640 1 -- 172.21.15.19:0/3674926579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5640adb40 con 0x7fb56414ba20 2024-03-28T13:11:42.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.674+0000 7fb56a600640 1 -- 172.21.15.19:0/3674926579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb564075940 con 0x7fb56414ba20 2024-03-28T13:11:42.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.675+0000 7fb560c00640 1 -- 172.21.15.19:0/3674926579 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb550038020 con 0x7fb56414ba20 2024-03-28T13:11:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.677+0000 7fb560c00640 1 --2- 172.21.15.19:0/3674926579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb54406c530 0x7fb54406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.677+0000 7fb560c00640 1 -- 172.21.15.19:0/3674926579 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4857+0+0 (secure 0 0 0) 0x7fb5500accd0 con 0x7fb56414ba20 2024-03-28T13:11:42.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.678+0000 7fb569200640 1 --2- 172.21.15.19:0/3674926579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb54406c530 0x7fb54406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:42.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.679+0000 7fb569200640 1 --2- 172.21.15.19:0/3674926579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb54406c530 0x7fb54406e9f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb558009a30 tx=0x7fb558008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:42.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.682+0000 7fb560c00640 1 -- 172.21.15.19:0/3674926579 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb550031800 con 0x7fb56414ba20 2024-03-28T13:11:42.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.890+0000 7fb56a600640 1 -- 172.21.15.19:0/3674926579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb56406f430 con 0x7fb56414ba20 2024-03-28T13:11:42.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.892+0000 7fb560c00640 1 -- 172.21.15.19:0/3674926579 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+12891 (secure 0 0 0) 0x7fb55007b640 con 0x7fb56414ba20 2024-03-28T13:11:42.894 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:42.894 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":45,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:11:41.868750+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:11:42.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.894+0000 7fb56a600640 1 -- 172.21.15.19:0/3674926579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb54406c530 msgr2=0x7fb54406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:42.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.894+0000 7fb56a600640 1 --2- 172.21.15.19:0/3674926579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb54406c530 0x7fb54406e9f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb558009a30 tx=0x7fb558008040 comp rx=0 tx=0).stop 2024-03-28T13:11:42.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.895+0000 7fb56a600640 1 -- 172.21.15.19:0/3674926579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56414ba20 msgr2=0x7fb5640ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:42.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.895+0000 7fb56a600640 1 --2- 172.21.15.19:0/3674926579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56414ba20 0x7fb5640ae620 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb5500098c0 tx=0x7fb5500046c0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.895+0000 7fb56a600640 1 -- 172.21.15.19:0/3674926579 shutdown_connections 2024-03-28T13:11:42.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.895+0000 7fb56a600640 1 --2- 172.21.15.19:0/3674926579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb54406c530 0x7fb54406e9f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.895+0000 7fb56a600640 1 --2- 172.21.15.19:0/3674926579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb56414c1b0 0x7fb564165320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.895+0000 7fb56a600640 1 --2- 172.21.15.19:0/3674926579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb56414ba20 0x7fb5640ae620 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.895+0000 7fb56a600640 1 --2- 172.21.15.19:0/3674926579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5640cfda0 0x7fb5641620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:42.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.895+0000 7fb56a600640 1 -- 172.21.15.19:0/3674926579 >> 172.21.15.19:0/3674926579 conn(0x7fb5640ccad0 msgr2=0x7fb5640ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:42.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.895+0000 7fb56a600640 1 -- 172.21.15.19:0/3674926579 shutdown_connections 2024-03-28T13:11:42.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:42.895+0000 7fb56a600640 1 -- 172.21.15.19:0/3674926579 wait complete. 2024-03-28T13:11:42.915 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:11:42.915 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:11:43.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.134+0000 7f476ac00640 1 Processor -- start 2024-03-28T13:11:43.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.134+0000 7f476ac00640 1 -- start start 2024-03-28T13:11:43.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.136+0000 7f476ac00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f47640d37b0 0x7f47640b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.136+0000 7f476ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47640b67f0 0x7f47640b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.136+0000 7f476ac00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47640d2710 con 0x7f476415e4c0 2024-03-28T13:11:43.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.136+0000 7f476ac00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f47640b6c60 con 0x7f47640d37b0 2024-03-28T13:11:43.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.136+0000 7f476ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f476415e8a0 con 0x7f47640b67f0 2024-03-28T13:11:43.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.136+0000 7f4769800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f476415e4c0 0x7f47640b9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47624/0 (socket says 172.21.15.19:47624) 2024-03-28T13:11:43.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.136+0000 7f4769800640 1 -- 172.21.15.19:0/598277416 learned_addr learned my addr 172.21.15.19:0/598277416 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:43.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4768e00640 1 --2- 172.21.15.19:0/598277416 >> v2:172.21.15.188:3300/0 conn(0x7f47640d37b0 0x7f47640b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f476a200640 1 --2- 172.21.15.19:0/598277416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47640b67f0 0x7f47640b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 -- 172.21.15.19:0/598277416 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1787193891 0 0) 0x7f47640d2710 con 0x7f476415e4c0 2024-03-28T13:11:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 -- 172.21.15.19:0/598277416 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f474c003520 con 0x7f476415e4c0 2024-03-28T13:11:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 -- 172.21.15.19:0/598277416 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3683330014 0 0) 0x7f474c003520 con 0x7f476415e4c0 2024-03-28T13:11:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 -- 172.21.15.19:0/598277416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47640b67f0 msgr2=0x7f47640b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 --2- 172.21.15.19:0/598277416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47640b67f0 0x7f47640b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 -- 172.21.15.19:0/598277416 >> v2:172.21.15.188:3300/0 conn(0x7f47640d37b0 msgr2=0x7f47640b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:43.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 --2- 172.21.15.19:0/598277416 >> v2:172.21.15.188:3300/0 conn(0x7f47640d37b0 0x7f47640b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 -- 172.21.15.19:0/598277416 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47640d29f0 con 0x7f476415e4c0 2024-03-28T13:11:43.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 -- 172.21.15.19:0/598277416 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f4758001670 con 0x7f476415e4c0 2024-03-28T13:11:43.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 -- 172.21.15.19:0/598277416 >> v1:172.21.15.19:6789/0 conn(0x7f476415e4c0 legacy=0x7f47640b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:43.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 --2- 172.21.15.19:0/598277416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f474c0039b0 0x7f474c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 --2- 172.21.15.19:0/598277416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47640b67f0 0x7f474c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 --2- 172.21.15.19:0/598277416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f474c004d40 0x7f474c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 -- 172.21.15.19:0/598277416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f47640b6c60 con 0x7f474c0039b0 2024-03-28T13:11:43.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 -- 172.21.15.19:0/598277416 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f474c00b9c0 con 0x7f474c004d40 2024-03-28T13:11:43.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.137+0000 7f4763e00640 1 -- 172.21.15.19:0/598277416 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f474c00bb40 con 0x7f47640b67f0 2024-03-28T13:11:43.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.138+0000 7f4769800640 1 --2- 172.21.15.19:0/598277416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f474c0039b0 0x7f474c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.138+0000 7f476a200640 1 --2- 172.21.15.19:0/598277416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f474c004d40 0x7f474c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.138+0000 7f4768e00640 1 --2- 172.21.15.19:0/598277416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47640b67f0 0x7f474c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.138+0000 7f4769800640 1 -- 172.21.15.19:0/598277416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47640b67f0 msgr2=0x7f474c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:43.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.138+0000 7f4769800640 1 --2- 172.21.15.19:0/598277416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47640b67f0 0x7f474c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.138+0000 7f4769800640 1 -- 172.21.15.19:0/598277416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f474c004d40 msgr2=0x7f474c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:43.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.138+0000 7f4769800640 1 --2- 172.21.15.19:0/598277416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f474c004d40 0x7f474c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.138+0000 7f4769800640 1 -- 172.21.15.19:0/598277416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474c003520 con 0x7f474c0039b0 2024-03-28T13:11:43.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.139+0000 7f4769800640 1 --2- 172.21.15.19:0/598277416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f474c0039b0 0x7f474c003dd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f475800c160 tx=0x7f474c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30061b5d01fd9f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:43.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.140+0000 7f4763e00640 1 -- 172.21.15.19:0/598277416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4758007c80 con 0x7f474c0039b0 2024-03-28T13:11:43.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.140+0000 7f4763e00640 1 -- 172.21.15.19:0/598277416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4758030070 con 0x7f474c0039b0 2024-03-28T13:11:43.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.140+0000 7f4763e00640 1 -- 172.21.15.19:0/598277416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4758005d30 con 0x7f474c0039b0 2024-03-28T13:11:43.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.140+0000 7f476ac00640 1 -- 172.21.15.19:0/598277416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f474c0039b0 msgr2=0x7f474c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:43.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.140+0000 7f476ac00640 1 --2- 172.21.15.19:0/598277416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f474c0039b0 0x7f474c003dd0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f475800c160 tx=0x7f474c005830 comp rx=0 tx=0).stop 2024-03-28T13:11:43.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.140+0000 7f476a200640 1 -- 172.21.15.19:0/598277416 reap_dead start 2024-03-28T13:11:43.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.140+0000 7f476ac00640 1 -- 172.21.15.19:0/598277416 shutdown_connections 2024-03-28T13:11:43.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.140+0000 7f476ac00640 1 -- 172.21.15.19:0/598277416 >> 172.21.15.19:0/598277416 conn(0x7f47640676a0 msgr2=0x7f4764067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:43.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.141+0000 7f476ac00640 1 -- 172.21.15.19:0/598277416 shutdown_connections 2024-03-28T13:11:43.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.141+0000 7f476ac00640 1 -- 172.21.15.19:0/598277416 wait complete. 2024-03-28T13:11:43.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.142+0000 7f476ac00640 1 Processor -- start 2024-03-28T13:11:43.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.142+0000 7f476ac00640 1 -- start start 2024-03-28T13:11:43.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.143+0000 7f476ac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f474c0039b0 0x7f47641f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.143+0000 7f476ac00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4764159af0 0x7f47641f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.143+0000 7f476ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47640aaf30 0x7f47640ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.143+0000 7f476ac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f47640d3320 con 0x7f474c0039b0 2024-03-28T13:11:43.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.143+0000 7f476ac00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f47640d4040 con 0x7f4764159af0 2024-03-28T13:11:43.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.143+0000 7f476ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f47640d2fb0 con 0x7f47640aaf30 2024-03-28T13:11:43.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.144+0000 7f4769800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f474c0039b0 0x7f47641f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.144+0000 7f4769800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f474c0039b0 0x7f47641f4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44436/0 (socket says 172.21.15.19:44436) 2024-03-28T13:11:43.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.144+0000 7f4769800640 1 -- 172.21.15.19:0/359663319 learned_addr learned my addr 172.21.15.19:0/359663319 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.144+0000 7f4769800640 1 -- 172.21.15.19:0/359663319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47640aaf30 msgr2=0x7f47640ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.144+0000 7f4769800640 1 --2- 172.21.15.19:0/359663319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47640aaf30 0x7f47640ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.144+0000 7f4768e00640 1 --2- 172.21.15.19:0/359663319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4764159af0 0x7f47641f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.144+0000 7f4769800640 1 -- 172.21.15.19:0/359663319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4764159af0 msgr2=0x7f47641f6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.144+0000 7f4769800640 1 --2- 172.21.15.19:0/359663319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4764159af0 0x7f47641f6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.144+0000 7f4769800640 1 -- 172.21.15.19:0/359663319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47641f4590 con 0x7f474c0039b0 2024-03-28T13:11:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.144+0000 7f4768e00640 1 --2- 172.21.15.19:0/359663319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4764159af0 0x7f47641f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.145+0000 7f4769800640 1 --2- 172.21.15.19:0/359663319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f474c0039b0 0x7f47641f4050 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f475800c280 tx=0x7f47640d3290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.145+0000 7f4761600640 1 -- 172.21.15.19:0/359663319 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475802de60 con 0x7f474c0039b0 2024-03-28T13:11:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.145+0000 7f4761600640 1 -- 172.21.15.19:0/359663319 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4758004bf0 con 0x7f474c0039b0 2024-03-28T13:11:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.145+0000 7f4761600640 1 -- 172.21.15.19:0/359663319 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4758007c60 con 0x7f474c0039b0 2024-03-28T13:11:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.145+0000 7f476ac00640 1 -- 172.21.15.19:0/359663319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47640ab910 con 0x7f474c0039b0 2024-03-28T13:11:43.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.145+0000 7f476ac00640 1 -- 172.21.15.19:0/359663319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47640abed0 con 0x7f474c0039b0 2024-03-28T13:11:43.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.146+0000 7f476ac00640 1 -- 172.21.15.19:0/359663319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47640b8ab0 con 0x7f474c0039b0 2024-03-28T13:11:43.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.148+0000 7f4761600640 1 -- 172.21.15.19:0/359663319 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4758048020 con 0x7f474c0039b0 2024-03-28T13:11:43.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.149+0000 7f4761600640 1 --2- 172.21.15.19:0/359663319 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f474406c530 0x7f474406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.149+0000 7f4761600640 1 -- 172.21.15.19:0/359663319 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4857+0+0 (secure 0 0 0) 0x7f47580ac720 con 0x7f474c0039b0 2024-03-28T13:11:43.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.150+0000 7f4768e00640 1 --2- 172.21.15.19:0/359663319 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f474406c530 0x7f474406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.151+0000 7f4768e00640 1 --2- 172.21.15.19:0/359663319 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f474406c530 0x7f474406e9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f476415a400 tx=0x7f4750008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:43.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.158+0000 7f4761600640 1 -- 172.21.15.19:0/359663319 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f47640b8ab0 con 0x7f474c0039b0 2024-03-28T13:11:43.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.362+0000 7f476ac00640 1 -- 172.21.15.19:0/359663319 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f47640d50a0 con 0x7f474c0039b0 2024-03-28T13:11:43.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.363+0000 7f4761600640 1 -- 172.21.15.19:0/359663319 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) ==== 93+0+10 (secure 0 0 0) 0x7f475807aff0 con 0x7f474c0039b0 2024-03-28T13:11:43.362 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:11:43.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.366+0000 7f476ac00640 1 -- 172.21.15.19:0/359663319 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f474406c530 msgr2=0x7f474406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:43.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.366+0000 7f476ac00640 1 --2- 172.21.15.19:0/359663319 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f474406c530 0x7f474406e9f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f476415a400 tx=0x7f4750008040 comp rx=0 tx=0).stop 2024-03-28T13:11:43.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.366+0000 7f476ac00640 1 -- 172.21.15.19:0/359663319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f474c0039b0 msgr2=0x7f47641f4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:43.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.366+0000 7f476ac00640 1 --2- 172.21.15.19:0/359663319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f474c0039b0 0x7f47641f4050 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f475800c280 tx=0x7f47640d3290 comp rx=0 tx=0).stop 2024-03-28T13:11:43.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.366+0000 7f476ac00640 1 -- 172.21.15.19:0/359663319 shutdown_connections 2024-03-28T13:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.366+0000 7f476ac00640 1 --2- 172.21.15.19:0/359663319 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f474406c530 0x7f474406e9f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.366+0000 7f476ac00640 1 --2- 172.21.15.19:0/359663319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47640aaf30 0x7f47640ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.366+0000 7f476ac00640 1 --2- 172.21.15.19:0/359663319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4764159af0 0x7f47641f6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.366+0000 7f476ac00640 1 --2- 172.21.15.19:0/359663319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f474c0039b0 0x7f47641f4050 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.366+0000 7f476ac00640 1 -- 172.21.15.19:0/359663319 >> 172.21.15.19:0/359663319 conn(0x7f47640676a0 msgr2=0x7f47640d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.366+0000 7f476ac00640 1 -- 172.21.15.19:0/359663319 shutdown_connections 2024-03-28T13:11:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.366+0000 7f476ac00640 1 -- 172.21.15.19:0/359663319 wait complete. 2024-03-28T13:11:43.385 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-28T13:11:43.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.603+0000 7fb446800640 1 Processor -- start 2024-03-28T13:11:43.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.603+0000 7fb446800640 1 -- start start 2024-03-28T13:11:43.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.605+0000 7fb446800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4401604a0 0x7fb440166ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.605+0000 7fb446800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb440142880 con 0x7fb44006e030 2024-03-28T13:11:43.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.605+0000 7fb446800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4401422c0 con 0x7fb44013f300 2024-03-28T13:11:43.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.605+0000 7fb446800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb4400cee90 con 0x7fb4401604a0 2024-03-28T13:11:43.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.605+0000 7fb444a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb44006e030 0x7fb44015fd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47638/0 (socket says 172.21.15.19:47638) 2024-03-28T13:11:43.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.605+0000 7fb444a00640 1 -- 172.21.15.19:0/414560776 learned_addr learned my addr 172.21.15.19:0/414560776 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb445e00640 1 --2- 172.21.15.19:0/414560776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4401604a0 0x7fb440166ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb43fe00640 1 -- 172.21.15.19:0/414560776 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 685469706 0 0) 0x7fb440142880 con 0x7fb44006e030 2024-03-28T13:11:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb43fe00640 1 -- 172.21.15.19:0/414560776 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb428003520 con 0x7fb44006e030 2024-03-28T13:11:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb43fe00640 1 -- 172.21.15.19:0/414560776 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2088710506 0 0) 0x7fb428003520 con 0x7fb44006e030 2024-03-28T13:11:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb43fe00640 1 -- 172.21.15.19:0/414560776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4401604a0 msgr2=0x7fb440166ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb43fe00640 1 --2- 172.21.15.19:0/414560776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4401604a0 0x7fb440166ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb43fe00640 1 -- 172.21.15.19:0/414560776 >> v1:172.21.15.188:6789/0 conn(0x7fb44013f300 legacy=0x7fb4401419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb43fe00640 1 -- 172.21.15.19:0/414560776 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4401425a0 con 0x7fb44006e030 2024-03-28T13:11:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb43fe00640 1 -- 172.21.15.19:0/414560776 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fb42c001670 con 0x7fb44006e030 2024-03-28T13:11:43.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb43fe00640 1 -- 172.21.15.19:0/414560776 >> v1:172.21.15.19:6789/0 conn(0x7fb44006e030 legacy=0x7fb44015fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb43fe00640 1 --2- 172.21.15.19:0/414560776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4280039e0 0x7fb428003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb43fe00640 1 --2- 172.21.15.19:0/414560776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4401604a0 0x7fb428004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb43fe00640 1 --2- 172.21.15.19:0/414560776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb428004d90 0x7fb428005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb43fe00640 1 -- 172.21.15.19:0/414560776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4400cee90 con 0x7fb428004d90 2024-03-28T13:11:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb43fe00640 1 -- 172.21.15.19:0/414560776 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb42800bbb0 con 0x7fb4280039e0 2024-03-28T13:11:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.606+0000 7fb43fe00640 1 -- 172.21.15.19:0/414560776 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb42800bdb0 con 0x7fb4401604a0 2024-03-28T13:11:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.607+0000 7fb445e00640 1 --2- 172.21.15.19:0/414560776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb428004d90 0x7fb428005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.607+0000 7fb444a00640 1 --2- 172.21.15.19:0/414560776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4401604a0 0x7fb428004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.607+0000 7fb445400640 1 --2- 172.21.15.19:0/414560776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4280039e0 0x7fb428003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.607+0000 7fb445e00640 1 -- 172.21.15.19:0/414560776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4401604a0 msgr2=0x7fb428004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:43.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.607+0000 7fb445e00640 1 --2- 172.21.15.19:0/414560776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4401604a0 0x7fb428004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.607+0000 7fb445e00640 1 -- 172.21.15.19:0/414560776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4280039e0 msgr2=0x7fb428003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:43.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.607+0000 7fb445e00640 1 --2- 172.21.15.19:0/414560776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4280039e0 0x7fb428003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.607+0000 7fb445e00640 1 -- 172.21.15.19:0/414560776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb428003520 con 0x7fb428004d90 2024-03-28T13:11:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.608+0000 7fb445e00640 1 --2- 172.21.15.19:0/414560776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb428004d90 0x7fb428005210 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb440071490 tx=0x7fb428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2921ca092e27af17 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.609+0000 7fb43fe00640 1 -- 172.21.15.19:0/414560776 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438039070 con 0x7fb428004d90 2024-03-28T13:11:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.609+0000 7fb43fe00640 1 -- 172.21.15.19:0/414560776 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb43802cc70 con 0x7fb428004d90 2024-03-28T13:11:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.609+0000 7fb43fe00640 1 -- 172.21.15.19:0/414560776 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438009be0 con 0x7fb428004d90 2024-03-28T13:11:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.609+0000 7fb446800640 1 -- 172.21.15.19:0/414560776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb428004d90 msgr2=0x7fb428005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.609+0000 7fb446800640 1 --2- 172.21.15.19:0/414560776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb428004d90 0x7fb428005210 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb440071490 tx=0x7fb428005830 comp rx=0 tx=0).stop 2024-03-28T13:11:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.609+0000 7fb445e00640 1 -- 172.21.15.19:0/414560776 reap_dead start 2024-03-28T13:11:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.609+0000 7fb446800640 1 -- 172.21.15.19:0/414560776 shutdown_connections 2024-03-28T13:11:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.609+0000 7fb446800640 1 -- 172.21.15.19:0/414560776 >> 172.21.15.19:0/414560776 conn(0x7fb44006b570 msgr2=0x7fb44006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.610+0000 7fb446800640 1 -- 172.21.15.19:0/414560776 shutdown_connections 2024-03-28T13:11:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.610+0000 7fb446800640 1 -- 172.21.15.19:0/414560776 wait complete. 2024-03-28T13:11:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.611+0000 7fb446800640 1 Processor -- start 2024-03-28T13:11:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.611+0000 7fb446800640 1 -- start start 2024-03-28T13:11:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.612+0000 7fb446800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb428004d90 0x7fb44013a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.612+0000 7fb446800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb440140730 0x7fb4401f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.612+0000 7fb446800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb44013cd80 0x7fb4401f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.612+0000 7fb446800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4400d04b0 con 0x7fb440140730 2024-03-28T13:11:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.612+0000 7fb446800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb4400d0330 con 0x7fb428004d90 2024-03-28T13:11:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.612+0000 7fb446800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb4400d0630 con 0x7fb44013cd80 2024-03-28T13:11:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.613+0000 7fb444a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb440140730 0x7fb4401f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.613+0000 7fb444a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb440140730 0x7fb4401f7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44452/0 (socket says 172.21.15.19:44452) 2024-03-28T13:11:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.613+0000 7fb444a00640 1 -- 172.21.15.19:0/4071701607 learned_addr learned my addr 172.21.15.19:0/4071701607 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.613+0000 7fb445400640 1 --2- 172.21.15.19:0/4071701607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb428004d90 0x7fb44013a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.613+0000 7fb444a00640 1 -- 172.21.15.19:0/4071701607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb44013cd80 msgr2=0x7fb4401f4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.613+0000 7fb445e00640 1 --2- 172.21.15.19:0/4071701607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb44013cd80 0x7fb4401f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.613+0000 7fb444a00640 1 --2- 172.21.15.19:0/4071701607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb44013cd80 0x7fb4401f4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.613+0000 7fb444a00640 1 -- 172.21.15.19:0/4071701607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb428004d90 msgr2=0x7fb44013a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.613+0000 7fb444a00640 1 --2- 172.21.15.19:0/4071701607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb428004d90 0x7fb44013a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.613+0000 7fb444a00640 1 -- 172.21.15.19:0/4071701607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb440140bb0 con 0x7fb440140730 2024-03-28T13:11:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.613+0000 7fb445400640 1 --2- 172.21.15.19:0/4071701607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb428004d90 0x7fb44013a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.614+0000 7fb444a00640 1 --2- 172.21.15.19:0/4071701607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb440140730 0x7fb4401f7a80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00e760 tx=0x7fb42c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.614+0000 7fb43d600640 1 -- 172.21.15.19:0/4071701607 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c00c8d0 con 0x7fb440140730 2024-03-28T13:11:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.614+0000 7fb43d600640 1 -- 172.21.15.19:0/4071701607 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb42c018070 con 0x7fb440140730 2024-03-28T13:11:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.614+0000 7fb43d600640 1 -- 172.21.15.19:0/4071701607 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42c00ccd0 con 0x7fb440140730 2024-03-28T13:11:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.614+0000 7fb446800640 1 -- 172.21.15.19:0/4071701607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb440140e60 con 0x7fb440140730 2024-03-28T13:11:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.614+0000 7fb446800640 1 -- 172.21.15.19:0/4071701607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb440141420 con 0x7fb440140730 2024-03-28T13:11:43.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.615+0000 7fb446800640 1 -- 172.21.15.19:0/4071701607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb440075940 con 0x7fb440140730 2024-03-28T13:11:43.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.616+0000 7fb43d600640 1 -- 172.21.15.19:0/4071701607 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb42c01d460 con 0x7fb440140730 2024-03-28T13:11:43.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.617+0000 7fb43d600640 1 --2- 172.21.15.19:0/4071701607 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb42006c530 0x7fb42006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:43.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.617+0000 7fb43d600640 1 -- 172.21.15.19:0/4071701607 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4857+0+0 (secure 0 0 0) 0x7fb42c08b310 con 0x7fb440140730 2024-03-28T13:11:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.618+0000 7fb445400640 1 --2- 172.21.15.19:0/4071701607 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb42006c530 0x7fb42006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:43.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.619+0000 7fb445400640 1 --2- 172.21.15.19:0/4071701607 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb42006c530 0x7fb42006e9f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb440141690 tx=0x7fb434009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:43.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.622+0000 7fb43d600640 1 -- 172.21.15.19:0/4071701607 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb42c059c70 con 0x7fb440140730 2024-03-28T13:11:43.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.828+0000 7fb446800640 1 -- 172.21.15.19:0/4071701607 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb44006c580 con 0x7fb440140730 2024-03-28T13:11:43.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.829+0000 7fb43d600640 1 -- 172.21.15.19:0/4071701607 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v45) ==== 84+0+65 (secure 0 0 0) 0x7fb42c059570 con 0x7fb440140730 2024-03-28T13:11:43.828 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:43.828 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-03-28T13:11:43.828 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:43.828 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-03-28T13:11:43.828 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-03-28T13:11:43.829 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:43.829 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-03-28T13:11:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.832+0000 7fb446800640 1 -- 172.21.15.19:0/4071701607 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb42006c530 msgr2=0x7fb42006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.832+0000 7fb446800640 1 --2- 172.21.15.19:0/4071701607 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb42006c530 0x7fb42006e9f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb440141690 tx=0x7fb434009040 comp rx=0 tx=0).stop 2024-03-28T13:11:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.832+0000 7fb446800640 1 -- 172.21.15.19:0/4071701607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb440140730 msgr2=0x7fb4401f7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.832+0000 7fb446800640 1 --2- 172.21.15.19:0/4071701607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb440140730 0x7fb4401f7a80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb42c00e760 tx=0x7fb42c00ec20 comp rx=0 tx=0).stop 2024-03-28T13:11:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.833+0000 7fb446800640 1 -- 172.21.15.19:0/4071701607 shutdown_connections 2024-03-28T13:11:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.833+0000 7fb446800640 1 --2- 172.21.15.19:0/4071701607 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb42006c530 0x7fb42006e9f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.833+0000 7fb446800640 1 --2- 172.21.15.19:0/4071701607 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb44013cd80 0x7fb4401f4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.833+0000 7fb446800640 1 --2- 172.21.15.19:0/4071701607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb440140730 0x7fb4401f7a80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.833+0000 7fb446800640 1 --2- 172.21.15.19:0/4071701607 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb428004d90 0x7fb44013a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.833+0000 7fb446800640 1 -- 172.21.15.19:0/4071701607 >> 172.21.15.19:0/4071701607 conn(0x7fb44006b570 msgr2=0x7fb4400cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:43.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.833+0000 7fb446800640 1 -- 172.21.15.19:0/4071701607 shutdown_connections 2024-03-28T13:11:43.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:43.833+0000 7fb446800640 1 -- 172.21.15.19:0/4071701607 wait complete. 2024-03-28T13:11:43.852 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-28T13:11:43.853 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-28T13:11:44.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.072+0000 7f1d58400640 1 Processor -- start 2024-03-28T13:11:44.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.072+0000 7f1d58400640 1 -- start start 2024-03-28T13:11:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.074+0000 7f1d58400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1d500cc3d0 0x7f1d5006f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.074+0000 7f1d58400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d50072e20 0x7f1d500b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.074+0000 7f1d58400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d5006e4e0 con 0x7f1d50072a40 2024-03-28T13:11:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.074+0000 7f1d58400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1d5006e7c0 con 0x7f1d500cc3d0 2024-03-28T13:11:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.074+0000 7f1d58400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1d5006eaa0 con 0x7f1d50072e20 2024-03-28T13:11:44.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.075+0000 7f1d56400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d50072e20 0x7f1d500b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:44.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.075+0000 7f1d56400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d50072e20 0x7f1d500b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:60114/0 (socket says 172.21.15.19:60114) 2024-03-28T13:11:44.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.075+0000 7f1d55a00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f1d50072a40 0x7f1d5015b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:52712/0 (socket says 172.21.15.19:52712) 2024-03-28T13:11:44.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.075+0000 7f1d56400640 1 -- 172.21.15.19:0/374966690 learned_addr learned my addr 172.21.15.19:0/374966690 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:44.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.075+0000 7f1d55000640 1 --2- 172.21.15.19:0/374966690 >> v2:172.21.15.188:3300/0 conn(0x7f1d500cc3d0 0x7f1d5006f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:44.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.076+0000 7f1d4fe00640 1 -- 172.21.15.19:0/374966690 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2423111050 0 0) 0x7f1d5006e4e0 con 0x7f1d50072a40 2024-03-28T13:11:44.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.076+0000 7f1d4fe00640 1 -- 172.21.15.19:0/374966690 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d38003520 con 0x7f1d50072a40 2024-03-28T13:11:44.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.076+0000 7f1d56400640 1 -- 172.21.15.19:0/374966690 >> v2:172.21.15.188:3300/0 conn(0x7f1d500cc3d0 msgr2=0x7f1d5006f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:44.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.076+0000 7f1d56400640 1 --2- 172.21.15.19:0/374966690 >> v2:172.21.15.188:3300/0 conn(0x7f1d500cc3d0 0x7f1d5006f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:44.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.076+0000 7f1d56400640 1 -- 172.21.15.19:0/374966690 >> v1:172.21.15.188:6789/0 conn(0x7f1d50072a40 legacy=0x7f1d5015b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:44.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.076+0000 7f1d56400640 1 -- 172.21.15.19:0/374966690 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5006ed80 con 0x7f1d50072e20 2024-03-28T13:11:44.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.077+0000 7f1d56400640 1 --2- 172.21.15.19:0/374966690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d50072e20 0x7f1d500b39d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009e50 tx=0x7f1d4802bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce4b4365a541cfef server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:44.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.078+0000 7f1d4fe00640 1 -- 172.21.15.19:0/374966690 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4802caf0 con 0x7f1d50072e20 2024-03-28T13:11:44.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.078+0000 7f1d4fe00640 1 -- 172.21.15.19:0/374966690 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d4802cc90 con 0x7f1d50072e20 2024-03-28T13:11:44.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.078+0000 7f1d4fe00640 1 -- 172.21.15.19:0/374966690 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d480345f0 con 0x7f1d50072e20 2024-03-28T13:11:44.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.079+0000 7f1d58400640 1 -- 172.21.15.19:0/374966690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d50072e20 msgr2=0x7f1d500b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:44.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.079+0000 7f1d58400640 1 --2- 172.21.15.19:0/374966690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d50072e20 0x7f1d500b39d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009e50 tx=0x7f1d4802bf50 comp rx=0 tx=0).stop 2024-03-28T13:11:44.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.079+0000 7f1d58400640 1 -- 172.21.15.19:0/374966690 shutdown_connections 2024-03-28T13:11:44.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.079+0000 7f1d58400640 1 --2- 172.21.15.19:0/374966690 >> v2:172.21.15.188:3300/0 conn(0x7f1d500cc3d0 0x7f1d5006f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:44.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.079+0000 7f1d58400640 1 --2- 172.21.15.19:0/374966690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d50072e20 0x7f1d500b39d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:44.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.079+0000 7f1d58400640 1 -- 172.21.15.19:0/374966690 >> 172.21.15.19:0/374966690 conn(0x7f1d5006bba0 msgr2=0x7f1d500b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:44.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.079+0000 7f1d58400640 1 -- 172.21.15.19:0/374966690 shutdown_connections 2024-03-28T13:11:44.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.079+0000 7f1d58400640 1 -- 172.21.15.19:0/374966690 wait complete. 2024-03-28T13:11:44.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.081+0000 7f1d58400640 1 Processor -- start 2024-03-28T13:11:44.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.081+0000 7f1d58400640 1 -- start start 2024-03-28T13:11:44.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.082+0000 7f1d58400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d50072a40 0x7f1d50071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:44.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.083+0000 7f1d58400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d500cc3d0 0x7f1d5015a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:44.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.083+0000 7f1d58400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d5015a930 0x7f1d50143800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:44.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.083+0000 7f1d58400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d500b3f10 con 0x7f1d50072a40 2024-03-28T13:11:44.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.083+0000 7f1d58400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5006e190 con 0x7f1d5015a930 2024-03-28T13:11:44.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.083+0000 7f1d58400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1d500b4090 con 0x7f1d500cc3d0 2024-03-28T13:11:44.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.083+0000 7f1d55a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d50072a40 0x7f1d50071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.083+0000 7f1d55a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d50072a40 0x7f1d50071880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44454/0 (socket says 172.21.15.19:44454) 2024-03-28T13:11:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.083+0000 7f1d55a00640 1 -- 172.21.15.19:0/1046577839 learned_addr learned my addr 172.21.15.19:0/1046577839 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.083+0000 7f1d56400640 1 --2- 172.21.15.19:0/1046577839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d5015a930 0x7f1d50143800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.083+0000 7f1d55000640 1 --2- 172.21.15.19:0/1046577839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d500cc3d0 0x7f1d5015a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.083+0000 7f1d55a00640 1 -- 172.21.15.19:0/1046577839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d500cc3d0 msgr2=0x7f1d5015a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:44.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.083+0000 7f1d55a00640 1 --2- 172.21.15.19:0/1046577839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d500cc3d0 0x7f1d5015a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:44.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.083+0000 7f1d55a00640 1 -- 172.21.15.19:0/1046577839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d5015a930 msgr2=0x7f1d50143800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:44.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.083+0000 7f1d55a00640 1 --2- 172.21.15.19:0/1046577839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d5015a930 0x7f1d50143800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:44.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.084+0000 7f1d55a00640 1 -- 172.21.15.19:0/1046577839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d500ceed0 con 0x7f1d50072a40 2024-03-28T13:11:44.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.084+0000 7f1d55a00640 1 --2- 172.21.15.19:0/1046577839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d50072a40 0x7f1d50071880 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400c430 tx=0x7f1d4400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:44.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.084+0000 7f1d4d600640 1 -- 172.21.15.19:0/1046577839 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d440074e0 con 0x7f1d50072a40 2024-03-28T13:11:44.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.084+0000 7f1d4d600640 1 -- 172.21.15.19:0/1046577839 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d440047e0 con 0x7f1d50072a40 2024-03-28T13:11:44.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.084+0000 7f1d4d600640 1 -- 172.21.15.19:0/1046577839 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44004bd0 con 0x7f1d50072a40 2024-03-28T13:11:44.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.084+0000 7f1d58400640 1 -- 172.21.15.19:0/1046577839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d500cf1b0 con 0x7f1d50072a40 2024-03-28T13:11:44.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.084+0000 7f1d58400640 1 -- 172.21.15.19:0/1046577839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d500cf770 con 0x7f1d50072a40 2024-03-28T13:11:44.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.085+0000 7f1d58400640 1 -- 172.21.15.19:0/1046577839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d500685c0 con 0x7f1d50072a40 2024-03-28T13:11:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.086+0000 7f1d4d600640 1 -- 172.21.15.19:0/1046577839 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1d4401a020 con 0x7f1d50072a40 2024-03-28T13:11:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.087+0000 7f1d4d600640 1 --2- 172.21.15.19:0/1046577839 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d3006c560 0x7f1d3006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.087+0000 7f1d4d600640 1 -- 172.21.15.19:0/1046577839 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4857+0+0 (secure 0 0 0) 0x7f1d4408ad70 con 0x7f1d50072a40 2024-03-28T13:11:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.088+0000 7f1d55000640 1 --2- 172.21.15.19:0/1046577839 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d3006c560 0x7f1d3006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.089+0000 7f1d55000640 1 --2- 172.21.15.19:0/1046577839 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d3006c560 0x7f1d3006ea20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1d500ae180 tx=0x7f1d3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:44.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.092+0000 7f1d4d600640 1 -- 172.21.15.19:0/1046577839 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1d44010300 con 0x7f1d50072a40 2024-03-28T13:11:44.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.302+0000 7f1d58400640 1 -- 172.21.15.19:0/1046577839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1d500b31a0 con 0x7f1d50072a40 2024-03-28T13:11:44.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.867+0000 7f1d4d600640 1 -- 172.21.15.19:0/1046577839 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v46) ==== 138+0+0 (secure 0 0 0) 0x7f1d44059670 con 0x7f1d50072a40 2024-03-28T13:11:44.867 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-03-28T13:11:44.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.871+0000 7f1d58400640 1 -- 172.21.15.19:0/1046577839 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d3006c560 msgr2=0x7f1d3006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:44.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.871+0000 7f1d58400640 1 --2- 172.21.15.19:0/1046577839 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d3006c560 0x7f1d3006ea20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1d500ae180 tx=0x7f1d3c008040 comp rx=0 tx=0).stop 2024-03-28T13:11:44.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.871+0000 7f1d58400640 1 -- 172.21.15.19:0/1046577839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d50072a40 msgr2=0x7f1d50071880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:44.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.871+0000 7f1d58400640 1 --2- 172.21.15.19:0/1046577839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d50072a40 0x7f1d50071880 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400c430 tx=0x7f1d4400c8f0 comp rx=0 tx=0).stop 2024-03-28T13:11:44.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.871+0000 7f1d58400640 1 -- 172.21.15.19:0/1046577839 shutdown_connections 2024-03-28T13:11:44.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.871+0000 7f1d58400640 1 --2- 172.21.15.19:0/1046577839 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d3006c560 0x7f1d3006ea20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:44.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.871+0000 7f1d58400640 1 --2- 172.21.15.19:0/1046577839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d5015a930 0x7f1d50143800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:44.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.871+0000 7f1d58400640 1 --2- 172.21.15.19:0/1046577839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d500cc3d0 0x7f1d5015a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:44.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.871+0000 7f1d58400640 1 --2- 172.21.15.19:0/1046577839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d50072a40 0x7f1d50071880 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:44.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.871+0000 7f1d58400640 1 -- 172.21.15.19:0/1046577839 >> 172.21.15.19:0/1046577839 conn(0x7f1d5006bba0 msgr2=0x7f1d5014dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:44.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.872+0000 7f1d58400640 1 -- 172.21.15.19:0/1046577839 shutdown_connections 2024-03-28T13:11:44.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:44.872+0000 7f1d58400640 1 -- 172.21.15.19:0/1046577839 wait complete. 2024-03-28T13:11:44.899 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-28T13:11:45.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.119+0000 7fa04e600640 1 Processor -- start 2024-03-28T13:11:45.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.119+0000 7fa04e600640 1 -- start start 2024-03-28T13:11:45.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.121+0000 7fa04e600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa04815e4d0 0x7fa04815f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:45.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.121+0000 7fa04e600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa0480cfc80 0x7fa0480d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:45.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.121+0000 7fa04e600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0480717c0 con 0x7fa0480736f0 2024-03-28T13:11:45.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.121+0000 7fa04e600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa0480b3ab0 con 0x7fa0480cfc80 2024-03-28T13:11:45.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.121+0000 7fa04e600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa048071dd0 con 0x7fa04815e4d0 2024-03-28T13:11:45.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.122+0000 7fa04d200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa04815e4d0 0x7fa04815f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:45.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.122+0000 7fa04dc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa0480cfc80 0x7fa0480d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:45.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.122+0000 7fa047e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fa0480736f0 0x7fa0480ade40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:52718/0 (socket says 172.21.15.19:52718) 2024-03-28T13:11:45.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.122+0000 7fa047e00640 1 -- 172.21.15.19:0/3941101365 learned_addr learned my addr 172.21.15.19:0/3941101365 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:45.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.122+0000 7fa04d200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa04815e4d0 0x7fa04815f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:60130/0 (socket says 172.21.15.19:60130) 2024-03-28T13:11:45.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.122+0000 7fa04dc00640 1 -- 172.21.15.19:0/3941101365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa04815e4d0 msgr2=0x7fa04815f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:45.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.122+0000 7fa04dc00640 1 --2- 172.21.15.19:0/3941101365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa04815e4d0 0x7fa04815f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:45.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.122+0000 7fa04dc00640 1 -- 172.21.15.19:0/3941101365 >> v1:172.21.15.188:6789/0 conn(0x7fa0480736f0 legacy=0x7fa0480ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:45.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.122+0000 7fa04dc00640 1 -- 172.21.15.19:0/3941101365 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04815e8b0 con 0x7fa0480cfc80 2024-03-28T13:11:45.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.122+0000 7fa04d200640 1 --2- 172.21.15.19:0/3941101365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa04815e4d0 0x7fa04815f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:11:45.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.124+0000 7fa04dc00640 1 --2- 172.21.15.19:0/3941101365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0480cfc80 0x7fa0480d0110 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa040009d90 tx=0x7fa04002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eb91e47bc020a64 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:45.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.124+0000 7fa047400640 1 -- 172.21.15.19:0/3941101365 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040039070 con 0x7fa0480cfc80 2024-03-28T13:11:45.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.124+0000 7fa047400640 1 -- 172.21.15.19:0/3941101365 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa04002cc60 con 0x7fa0480cfc80 2024-03-28T13:11:45.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.124+0000 7fa047400640 1 -- 172.21.15.19:0/3941101365 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040034660 con 0x7fa0480cfc80 2024-03-28T13:11:45.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.125+0000 7fa04e600640 1 -- 172.21.15.19:0/3941101365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0480cfc80 msgr2=0x7fa0480d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:45.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.125+0000 7fa04e600640 1 --2- 172.21.15.19:0/3941101365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0480cfc80 0x7fa0480d0110 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa040009d90 tx=0x7fa04002beb0 comp rx=0 tx=0).stop 2024-03-28T13:11:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.125+0000 7fa04e600640 1 -- 172.21.15.19:0/3941101365 shutdown_connections 2024-03-28T13:11:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.125+0000 7fa04e600640 1 --2- 172.21.15.19:0/3941101365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0480cfc80 0x7fa0480d0110 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.125+0000 7fa04e600640 1 --2- 172.21.15.19:0/3941101365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa04815e4d0 0x7fa04815f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.125+0000 7fa04e600640 1 -- 172.21.15.19:0/3941101365 >> 172.21.15.19:0/3941101365 conn(0x7fa04806b570 msgr2=0x7fa04806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.125+0000 7fa04e600640 1 -- 172.21.15.19:0/3941101365 shutdown_connections 2024-03-28T13:11:45.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.125+0000 7fa04e600640 1 -- 172.21.15.19:0/3941101365 wait complete. 2024-03-28T13:11:45.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.128+0000 7fa04e600640 1 Processor -- start 2024-03-28T13:11:45.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.128+0000 7fa04e600640 1 -- start start 2024-03-28T13:11:45.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.129+0000 7fa04e600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0480736f0 0x7fa048160ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:45.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.129+0000 7fa04e600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0480cfc80 0x7fa048155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:45.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.129+0000 7fa04e600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa04815e4d0 0x7fa048158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:45.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.129+0000 7fa04e600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa048071a60 con 0x7fa04815e4d0 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.129+0000 7fa04e600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0480b3fe0 con 0x7fa0480736f0 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.129+0000 7fa04e600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa048072260 con 0x7fa0480cfc80 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.130+0000 7fa04dc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa04815e4d0 0x7fa048158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.130+0000 7fa04dc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa04815e4d0 0x7fa048158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44460/0 (socket says 172.21.15.19:44460) 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.130+0000 7fa04dc00640 1 -- 172.21.15.19:0/2513583051 learned_addr learned my addr 172.21.15.19:0/2513583051 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.130+0000 7fa04d200640 1 --2- 172.21.15.19:0/2513583051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0480736f0 0x7fa048160ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.130+0000 7fa047e00640 1 --2- 172.21.15.19:0/2513583051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0480cfc80 0x7fa048155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.130+0000 7fa04dc00640 1 -- 172.21.15.19:0/2513583051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0480cfc80 msgr2=0x7fa048155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.130+0000 7fa04dc00640 1 --2- 172.21.15.19:0/2513583051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0480cfc80 0x7fa048155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.130+0000 7fa04dc00640 1 -- 172.21.15.19:0/2513583051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0480736f0 msgr2=0x7fa048160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.130+0000 7fa04dc00640 1 --2- 172.21.15.19:0/2513583051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0480736f0 0x7fa048160ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.130+0000 7fa04dc00640 1 -- 172.21.15.19:0/2513583051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04815fe20 con 0x7fa04815e4d0 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.131+0000 7fa04dc00640 1 --2- 172.21.15.19:0/2513583051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa04815e4d0 0x7fa048158740 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa040009d90 tx=0x7fa040004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.131+0000 7fa044c00640 1 -- 172.21.15.19:0/2513583051 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040039050 con 0x7fa04815e4d0 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.131+0000 7fa044c00640 1 -- 172.21.15.19:0/2513583051 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa040045070 con 0x7fa04815e4d0 2024-03-28T13:11:45.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.131+0000 7fa04e600640 1 -- 172.21.15.19:0/2513583051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa048160100 con 0x7fa04815e4d0 2024-03-28T13:11:45.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.131+0000 7fa044c00640 1 -- 172.21.15.19:0/2513583051 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040009450 con 0x7fa04815e4d0 2024-03-28T13:11:45.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.131+0000 7fa04e600640 1 -- 172.21.15.19:0/2513583051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0481606c0 con 0x7fa04815e4d0 2024-03-28T13:11:45.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.132+0000 7fa04e600640 1 -- 172.21.15.19:0/2513583051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa048075940 con 0x7fa04815e4d0 2024-03-28T13:11:45.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.134+0000 7fa044c00640 1 -- 172.21.15.19:0/2513583051 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa04003e850 con 0x7fa04815e4d0 2024-03-28T13:11:45.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.135+0000 7fa044c00640 1 --2- 172.21.15.19:0/2513583051 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa02806c530 0x7fa02806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:45.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.135+0000 7fa044c00640 1 -- 172.21.15.19:0/2513583051 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5227+0+0 (secure 0 0 0) 0x7fa0400aeb90 con 0x7fa04815e4d0 2024-03-28T13:11:45.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.135+0000 7fa04d200640 1 --2- 172.21.15.19:0/2513583051 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa02806c530 0x7fa02806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.137+0000 7fa04d200640 1 --2- 172.21.15.19:0/2513583051 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa02806c530 0x7fa02806e9f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009a30 tx=0x7fa03c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:45.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.139+0000 7fa044c00640 1 -- 172.21.15.19:0/2513583051 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa04007d500 con 0x7fa04815e4d0 2024-03-28T13:11:45.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.343+0000 7fa04e600640 1 -- 172.21.15.19:0/2513583051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa04815beb0 con 0x7fa04815e4d0 2024-03-28T13:11:45.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.884+0000 7fa044c00640 1 -- 172.21.15.19:0/2513583051 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v47) ==== 170+0+0 (secure 0 0 0) 0x7fa04003fe90 con 0x7fa04815e4d0 2024-03-28T13:11:45.884 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-03-28T13:11:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.888+0000 7fa04e600640 1 -- 172.21.15.19:0/2513583051 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa02806c530 msgr2=0x7fa02806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.888+0000 7fa04e600640 1 --2- 172.21.15.19:0/2513583051 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa02806c530 0x7fa02806e9f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009a30 tx=0x7fa03c008040 comp rx=0 tx=0).stop 2024-03-28T13:11:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.888+0000 7fa04e600640 1 -- 172.21.15.19:0/2513583051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa04815e4d0 msgr2=0x7fa048158740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.888+0000 7fa04e600640 1 --2- 172.21.15.19:0/2513583051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa04815e4d0 0x7fa048158740 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa040009d90 tx=0x7fa040004a40 comp rx=0 tx=0).stop 2024-03-28T13:11:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.888+0000 7fa04e600640 1 -- 172.21.15.19:0/2513583051 shutdown_connections 2024-03-28T13:11:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.888+0000 7fa04e600640 1 --2- 172.21.15.19:0/2513583051 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa02806c530 0x7fa02806e9f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.888+0000 7fa04e600640 1 --2- 172.21.15.19:0/2513583051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa04815e4d0 0x7fa048158740 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:45.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.888+0000 7fa04e600640 1 --2- 172.21.15.19:0/2513583051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0480cfc80 0x7fa048155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.888+0000 7fa04e600640 1 --2- 172.21.15.19:0/2513583051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0480736f0 0x7fa048160ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.888+0000 7fa04e600640 1 -- 172.21.15.19:0/2513583051 >> 172.21.15.19:0/2513583051 conn(0x7fa04806b570 msgr2=0x7fa048072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.888+0000 7fa04e600640 1 -- 172.21.15.19:0/2513583051 shutdown_connections 2024-03-28T13:11:45.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:45.888+0000 7fa04e600640 1 -- 172.21.15.19:0/2513583051 wait complete. 2024-03-28T13:11:45.922 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-28T13:11:46.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.113+0000 7f7fa8000640 1 Processor -- start 2024-03-28T13:11:46.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.113+0000 7f7fa8000640 1 -- start start 2024-03-28T13:11:46.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.114+0000 7f7fa8000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7fa0153580 0x7f7fa00d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:46.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.114+0000 7f7fa8000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7fa00d07c0 0x7f7fa00aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:46.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.114+0000 7f7fa8000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fa00af2e0 0x7f7fa006d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.114+0000 7f7fa8000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa00d0c30 con 0x7f7fa00d07c0 2024-03-28T13:11:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.114+0000 7f7fa8000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa00afab0 con 0x7f7fa0153580 2024-03-28T13:11:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.114+0000 7f7fa8000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa015cc80 con 0x7f7fa00af2e0 2024-03-28T13:11:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.115+0000 7f7fa4c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7fa00d07c0 0x7f7fa00aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.115+0000 7f7fa4c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7fa00d07c0 0x7f7fa00aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45328/0 (socket says 172.21.15.19:45328) 2024-03-28T13:11:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.115+0000 7f7fa4c00640 1 -- 172.21.15.19:0/4109106960 learned_addr learned my addr 172.21.15.19:0/4109106960 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.115+0000 7f7fa5600640 1 --2- 172.21.15.19:0/4109106960 >> v2:172.21.15.188:3300/0 conn(0x7f7fa0153580 0x7f7fa00d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.115+0000 7f7fa4c00640 1 -- 172.21.15.19:0/4109106960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fa00af2e0 msgr2=0x7f7fa006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.115+0000 7f7fa6000640 1 --2- 172.21.15.19:0/4109106960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fa00af2e0 0x7f7fa006d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.115+0000 7f7fa4c00640 1 --2- 172.21.15.19:0/4109106960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fa00af2e0 0x7f7fa006d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.115+0000 7f7fa4c00640 1 -- 172.21.15.19:0/4109106960 >> v2:172.21.15.188:3300/0 conn(0x7f7fa0153580 msgr2=0x7f7fa00d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.115+0000 7f7fa4c00640 1 --2- 172.21.15.19:0/4109106960 >> v2:172.21.15.188:3300/0 conn(0x7f7fa0153580 0x7f7fa00d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:46.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.115+0000 7f7fa4c00640 1 -- 172.21.15.19:0/4109106960 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa015da00 con 0x7f7fa00d07c0 2024-03-28T13:11:46.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.115+0000 7f7fa5600640 1 --2- 172.21.15.19:0/4109106960 >> v2:172.21.15.188:3300/0 conn(0x7f7fa0153580 0x7f7fa00d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:46.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.115+0000 7f7fa6000640 1 --2- 172.21.15.19:0/4109106960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fa00af2e0 0x7f7fa006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.116+0000 7f7fa4c00640 1 --2- 172.21.15.19:0/4109106960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa00d07c0 0x7f7fa00aeda0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00b5270 tx=0x7f7f8c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a4c82d9612709c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.117+0000 7f7f9fe00640 1 -- 172.21.15.19:0/4109106960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c02e070 con 0x7f7fa00d07c0 2024-03-28T13:11:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.117+0000 7f7f9fe00640 1 -- 172.21.15.19:0/4109106960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f8c037070 con 0x7f7fa00d07c0 2024-03-28T13:11:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.117+0000 7f7f9fe00640 1 -- 172.21.15.19:0/4109106960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c031a00 con 0x7f7fa00d07c0 2024-03-28T13:11:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.117+0000 7f7fa8000640 1 -- 172.21.15.19:0/4109106960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa00d07c0 msgr2=0x7f7fa00aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.117+0000 7f7fa8000640 1 --2- 172.21.15.19:0/4109106960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa00d07c0 0x7f7fa00aeda0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00b5270 tx=0x7f7f8c02b880 comp rx=0 tx=0).stop 2024-03-28T13:11:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.118+0000 7f7fa8000640 1 -- 172.21.15.19:0/4109106960 shutdown_connections 2024-03-28T13:11:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.118+0000 7f7fa8000640 1 --2- 172.21.15.19:0/4109106960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fa00af2e0 0x7f7fa006d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.118+0000 7f7fa8000640 1 --2- 172.21.15.19:0/4109106960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa00d07c0 0x7f7fa00aeda0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.118+0000 7f7fa8000640 1 --2- 172.21.15.19:0/4109106960 >> v2:172.21.15.188:3300/0 conn(0x7f7fa0153580 0x7f7fa00d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.118+0000 7f7fa8000640 1 -- 172.21.15.19:0/4109106960 >> 172.21.15.19:0/4109106960 conn(0x7f7fa006b750 msgr2=0x7f7fa006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.118+0000 7f7fa8000640 1 -- 172.21.15.19:0/4109106960 shutdown_connections 2024-03-28T13:11:46.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.118+0000 7f7fa8000640 1 -- 172.21.15.19:0/4109106960 wait complete. 2024-03-28T13:11:46.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.119+0000 7f7fa8000640 1 Processor -- start 2024-03-28T13:11:46.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.120+0000 7f7fa8000640 1 -- start start 2024-03-28T13:11:46.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.121+0000 7f7fa8000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fa00af2e0 0x7f7fa0157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:46.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.121+0000 7f7fa8000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fa00d07c0 0x7f7fa0158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:46.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.121+0000 7f7fa8000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa0153580 0x7f7fa0159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.121+0000 7f7fa8000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa015d4e0 con 0x7f7fa0153580 2024-03-28T13:11:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.121+0000 7f7fa8000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa015d200 con 0x7f7fa00d07c0 2024-03-28T13:11:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.121+0000 7f7fa8000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa0153960 con 0x7f7fa00af2e0 2024-03-28T13:11:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.121+0000 7f7fa6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa0153580 0x7f7fa0159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.121+0000 7f7fa6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa0153580 0x7f7fa0159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45338/0 (socket says 172.21.15.19:45338) 2024-03-28T13:11:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.121+0000 7f7fa6000640 1 -- 172.21.15.19:0/762288751 learned_addr learned my addr 172.21.15.19:0/762288751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.121+0000 7f7fa4c00640 1 --2- 172.21.15.19:0/762288751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fa00d07c0 0x7f7fa0158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.121+0000 7f7fa5600640 1 --2- 172.21.15.19:0/762288751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fa00af2e0 0x7f7fa0157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.122+0000 7f7fa6000640 1 -- 172.21.15.19:0/762288751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fa00af2e0 msgr2=0x7f7fa0157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.122+0000 7f7fa6000640 1 --2- 172.21.15.19:0/762288751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fa00af2e0 0x7f7fa0157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.122+0000 7f7fa6000640 1 -- 172.21.15.19:0/762288751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fa00d07c0 msgr2=0x7f7fa0158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.122+0000 7f7fa6000640 1 --2- 172.21.15.19:0/762288751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fa00d07c0 0x7f7fa0158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.122+0000 7f7fa6000640 1 -- 172.21.15.19:0/762288751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa0158440 con 0x7f7fa0153580 2024-03-28T13:11:46.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.122+0000 7f7fa4c00640 1 --2- 172.21.15.19:0/762288751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fa00d07c0 0x7f7fa0158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:11:46.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.122+0000 7f7fa6000640 1 --2- 172.21.15.19:0/762288751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa0153580 0x7f7fa0159ac0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800cd10 tx=0x7f7f9800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:46.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.122+0000 7f7f9d600640 1 -- 172.21.15.19:0/762288751 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9800f680 con 0x7f7fa0153580 2024-03-28T13:11:46.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.122+0000 7f7f9d600640 1 -- 172.21.15.19:0/762288751 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f9800f840 con 0x7f7fa0153580 2024-03-28T13:11:46.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.122+0000 7f7f9d600640 1 -- 172.21.15.19:0/762288751 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9800fc30 con 0x7f7fa0153580 2024-03-28T13:11:46.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.122+0000 7f7fa8000640 1 -- 172.21.15.19:0/762288751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa00ceb90 con 0x7f7fa0153580 2024-03-28T13:11:46.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.123+0000 7f7fa8000640 1 -- 172.21.15.19:0/762288751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa00cf150 con 0x7f7fa0153580 2024-03-28T13:11:46.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.124+0000 7f7fa8000640 1 -- 172.21.15.19:0/762288751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa00685c0 con 0x7f7fa0153580 2024-03-28T13:11:46.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.125+0000 7f7f9d600640 1 -- 172.21.15.19:0/762288751 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7f980213f0 con 0x7f7fa0153580 2024-03-28T13:11:46.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.126+0000 7f7f9d600640 1 --2- 172.21.15.19:0/762288751 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f8006c560 0x7f7f8006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.126+0000 7f7f9d600640 1 -- 172.21.15.19:0/762288751 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5609+0+0 (secure 0 0 0) 0x7f7f9808c3f0 con 0x7f7fa0153580 2024-03-28T13:11:46.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.126+0000 7f7fa5600640 1 --2- 172.21.15.19:0/762288751 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f8006c560 0x7f7f8006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:46.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.128+0000 7f7fa5600640 1 --2- 172.21.15.19:0/762288751 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f8006c560 0x7f7f8006ea20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400a990 tx=0x7f7f94008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:46.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.131+0000 7f7f9d600640 1 -- 172.21.15.19:0/762288751 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7f98014030 con 0x7f7fa0153580 2024-03-28T13:11:46.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.378+0000 7f7fa8000640 1 -- 172.21.15.19:0/762288751 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7fa0154c50 con 0x7f7fa0153580 2024-03-28T13:11:46.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.888+0000 7f7f9d600640 1 -- 172.21.15.19:0/762288751 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) ==== 172+0+0 (secure 0 0 0) 0x7f7f9805a380 con 0x7f7fa0153580 2024-03-28T13:11:46.888 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 6 and data pool 7 2024-03-28T13:11:46.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.893+0000 7f7fa8000640 1 -- 172.21.15.19:0/762288751 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f8006c560 msgr2=0x7f7f8006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:46.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.893+0000 7f7fa8000640 1 --2- 172.21.15.19:0/762288751 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f8006c560 0x7f7f8006ea20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400a990 tx=0x7f7f94008040 comp rx=0 tx=0).stop 2024-03-28T13:11:46.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.893+0000 7f7fa8000640 1 -- 172.21.15.19:0/762288751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa0153580 msgr2=0x7f7fa0159ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:46.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.893+0000 7f7fa8000640 1 --2- 172.21.15.19:0/762288751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa0153580 0x7f7fa0159ac0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800cd10 tx=0x7f7f9800a650 comp rx=0 tx=0).stop 2024-03-28T13:11:46.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.893+0000 7f7fa8000640 1 -- 172.21.15.19:0/762288751 shutdown_connections 2024-03-28T13:11:46.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.893+0000 7f7fa8000640 1 --2- 172.21.15.19:0/762288751 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f8006c560 0x7f7f8006ea20 secure :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400a990 tx=0x7f7f94008040 comp rx=0 tx=0).stop 2024-03-28T13:11:46.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.893+0000 7f7fa8000640 1 --2- 172.21.15.19:0/762288751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fa0153580 0x7f7fa0159ac0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:46.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.893+0000 7f7fa8000640 1 --2- 172.21.15.19:0/762288751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fa00d07c0 0x7f7fa0158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:46.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.893+0000 7f7fa8000640 1 --2- 172.21.15.19:0/762288751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fa00af2e0 0x7f7fa0157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:46.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.893+0000 7f7fa8000640 1 -- 172.21.15.19:0/762288751 >> 172.21.15.19:0/762288751 conn(0x7f7fa006b750 msgr2=0x7f7fa006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:46.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.894+0000 7f7fa8000640 1 -- 172.21.15.19:0/762288751 shutdown_connections 2024-03-28T13:11:46.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:46.894+0000 7f7fa8000640 1 -- 172.21.15.19:0/762288751 wait complete. 2024-03-28T13:11:46.931 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:11:47.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.113+0000 7f5d49e00640 1 Processor -- start 2024-03-28T13:11:47.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.113+0000 7f5d49e00640 1 -- start start 2024-03-28T13:11:47.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.115+0000 7f5d49e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5d4415d120 0x7f5d44162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:47.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.115+0000 7f5d49e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5d440af630 0x7f5d44166b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:47.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.115+0000 7f5d49e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d44167080 0x7f5d440b3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:47.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.115+0000 7f5d49e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5d440723c0 con 0x7f5d4415d120 2024-03-28T13:11:47.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.115+0000 7f5d49e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5d441633d0 con 0x7f5d440af630 2024-03-28T13:11:47.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.115+0000 7f5d49e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5d44163650 con 0x7f5d44167080 2024-03-28T13:11:47.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.115+0000 7f5d42a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5d4415d120 0x7f5d44162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:47.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.115+0000 7f5d42a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5d4415d120 0x7f5d44162e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45346/0 (socket says 172.21.15.19:45346) 2024-03-28T13:11:47.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.115+0000 7f5d42a00640 1 -- 172.21.15.19:0/2518848836 learned_addr learned my addr 172.21.15.19:0/2518848836 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:47.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.116+0000 7f5d43400640 1 --2- 172.21.15.19:0/2518848836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d44167080 0x7f5d440b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.116+0000 7f5d3a000640 1 --2- 172.21.15.19:0/2518848836 >> v2:172.21.15.188:3300/0 conn(0x7f5d440af630 0x7f5d44166b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.116+0000 7f5d42a00640 1 -- 172.21.15.19:0/2518848836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d44167080 msgr2=0x7f5d440b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.116+0000 7f5d42a00640 1 --2- 172.21.15.19:0/2518848836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d44167080 0x7f5d440b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.116+0000 7f5d42a00640 1 -- 172.21.15.19:0/2518848836 >> v2:172.21.15.188:3300/0 conn(0x7f5d440af630 msgr2=0x7f5d44166b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.116+0000 7f5d42a00640 1 --2- 172.21.15.19:0/2518848836 >> v2:172.21.15.188:3300/0 conn(0x7f5d440af630 0x7f5d44166b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.116+0000 7f5d42a00640 1 -- 172.21.15.19:0/2518848836 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d44163930 con 0x7f5d4415d120 2024-03-28T13:11:47.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.116+0000 7f5d3a000640 1 --2- 172.21.15.19:0/2518848836 >> v2:172.21.15.188:3300/0 conn(0x7f5d440af630 0x7f5d44166b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.117+0000 7f5d42a00640 1 --2- 172.21.15.19:0/2518848836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4415d120 0x7f5d44162e90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5d4406ea10 tx=0x7f5d3402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1816234c4a76b3f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.117+0000 7f5d42000640 1 -- 172.21.15.19:0/2518848836 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34035020 con 0x7f5d4415d120 2024-03-28T13:11:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.117+0000 7f5d42000640 1 -- 172.21.15.19:0/2518848836 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d34031870 con 0x7f5d4415d120 2024-03-28T13:11:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.117+0000 7f5d42000640 1 -- 172.21.15.19:0/2518848836 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34031c40 con 0x7f5d4415d120 2024-03-28T13:11:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.118+0000 7f5d49e00640 1 -- 172.21.15.19:0/2518848836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4415d120 msgr2=0x7f5d44162e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.118+0000 7f5d49e00640 1 --2- 172.21.15.19:0/2518848836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4415d120 0x7f5d44162e90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f5d4406ea10 tx=0x7f5d3402b910 comp rx=0 tx=0).stop 2024-03-28T13:11:47.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.118+0000 7f5d49e00640 1 -- 172.21.15.19:0/2518848836 shutdown_connections 2024-03-28T13:11:47.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.118+0000 7f5d49e00640 1 --2- 172.21.15.19:0/2518848836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d44167080 0x7f5d440b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.118+0000 7f5d49e00640 1 --2- 172.21.15.19:0/2518848836 >> v2:172.21.15.188:3300/0 conn(0x7f5d440af630 0x7f5d44166b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.118+0000 7f5d49e00640 1 --2- 172.21.15.19:0/2518848836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d4415d120 0x7f5d44162e90 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.118+0000 7f5d49e00640 1 -- 172.21.15.19:0/2518848836 >> 172.21.15.19:0/2518848836 conn(0x7f5d4406b990 msgr2=0x7f5d4406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:47.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.118+0000 7f5d49e00640 1 -- 172.21.15.19:0/2518848836 shutdown_connections 2024-03-28T13:11:47.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.118+0000 7f5d49e00640 1 -- 172.21.15.19:0/2518848836 wait complete. 2024-03-28T13:11:47.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.120+0000 7f5d49e00640 1 Processor -- start 2024-03-28T13:11:47.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.120+0000 7f5d49e00640 1 -- start start 2024-03-28T13:11:47.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.121+0000 7f5d49e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d440af630 0x7f5d440b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:47.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.121+0000 7f5d49e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4415d120 0x7f5d440b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:47.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.121+0000 7f5d49e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44167080 0x7f5d440cee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:47.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.121+0000 7f5d49e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d440b3a40 con 0x7f5d44167080 2024-03-28T13:11:47.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.121+0000 7f5d49e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5d44166620 con 0x7f5d440af630 2024-03-28T13:11:47.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.121+0000 7f5d49e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5d440b3bc0 con 0x7f5d4415d120 2024-03-28T13:11:47.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.121+0000 7f5d43400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44167080 0x7f5d440cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:47.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.121+0000 7f5d43400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44167080 0x7f5d440cee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45354/0 (socket says 172.21.15.19:45354) 2024-03-28T13:11:47.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.121+0000 7f5d43400640 1 -- 172.21.15.19:0/1616480686 learned_addr learned my addr 172.21.15.19:0/1616480686 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:47.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.122+0000 7f5d3a000640 1 --2- 172.21.15.19:0/1616480686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4415d120 0x7f5d440b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:47.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.122+0000 7f5d43400640 1 -- 172.21.15.19:0/1616480686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4415d120 msgr2=0x7f5d440b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:47.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.122+0000 7f5d43400640 1 --2- 172.21.15.19:0/1616480686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4415d120 0x7f5d440b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.122+0000 7f5d43400640 1 -- 172.21.15.19:0/1616480686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d440af630 msgr2=0x7f5d440b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:47.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.122+0000 7f5d42a00640 1 --2- 172.21.15.19:0/1616480686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d440af630 0x7f5d440b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:47.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.122+0000 7f5d43400640 1 --2- 172.21.15.19:0/1616480686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d440af630 0x7f5d440b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.122+0000 7f5d43400640 1 -- 172.21.15.19:0/1616480686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d440b0ff0 con 0x7f5d44167080 2024-03-28T13:11:47.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.122+0000 7f5d3a000640 1 --2- 172.21.15.19:0/1616480686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4415d120 0x7f5d440b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:47.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.122+0000 7f5d43400640 1 --2- 172.21.15.19:0/1616480686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44167080 0x7f5d440cee10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c007e90 tx=0x7f5d3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:47.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.122+0000 7f5d3be00640 1 -- 172.21.15.19:0/1616480686 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c014420 con 0x7f5d44167080 2024-03-28T13:11:47.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.122+0000 7f5d3be00640 1 -- 172.21.15.19:0/1616480686 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d3c019070 con 0x7f5d44167080 2024-03-28T13:11:47.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.122+0000 7f5d3be00640 1 -- 172.21.15.19:0/1616480686 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c014800 con 0x7f5d44167080 2024-03-28T13:11:47.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.122+0000 7f5d49e00640 1 -- 172.21.15.19:0/1616480686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d440b12d0 con 0x7f5d44167080 2024-03-28T13:11:47.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.123+0000 7f5d49e00640 1 -- 172.21.15.19:0/1616480686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d440b1890 con 0x7f5d44167080 2024-03-28T13:11:47.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.124+0000 7f5d49e00640 1 -- 172.21.15.19:0/1616480686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d440685c0 con 0x7f5d44167080 2024-03-28T13:11:47.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.125+0000 7f5d3be00640 1 -- 172.21.15.19:0/1616480686 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5d3c0149a0 con 0x7f5d44167080 2024-03-28T13:11:47.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.125+0000 7f5d3be00640 1 --2- 172.21.15.19:0/1616480686 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5d2006c530 0x7f5d2006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:47.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.126+0000 7f5d3be00640 1 -- 172.21.15.19:0/1616480686 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f5d3c08bf80 con 0x7f5d44167080 2024-03-28T13:11:47.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.126+0000 7f5d42a00640 1 --2- 172.21.15.19:0/1616480686 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5d2006c530 0x7f5d2006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:47.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.127+0000 7f5d42a00640 1 --2- 172.21.15.19:0/1616480686 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5d2006c530 0x7f5d2006e9f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5d34036000 tx=0x7f5d34008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:47.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.131+0000 7f5d3be00640 1 -- 172.21.15.19:0/1616480686 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5d3c05a5a0 con 0x7f5d44167080 2024-03-28T13:11:47.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.337+0000 7f5d49e00640 1 -- 172.21.15.19:0/1616480686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d440aee90 con 0x7f5d44167080 2024-03-28T13:11:47.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.339+0000 7f5d3be00640 1 -- 172.21.15.19:0/1616480686 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16622 (secure 0 0 0) 0x7f5d3c059ea0 con 0x7f5d44167080 2024-03-28T13:11:47.338 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:47.338 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":48,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:11:46.379407+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:11:44.316258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-28T13:11:45.360154+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:11:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.342+0000 7f5d49e00640 1 -- 172.21.15.19:0/1616480686 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5d2006c530 msgr2=0x7f5d2006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.342+0000 7f5d49e00640 1 --2- 172.21.15.19:0/1616480686 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5d2006c530 0x7f5d2006e9f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5d34036000 tx=0x7f5d34008030 comp rx=0 tx=0).stop 2024-03-28T13:11:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.342+0000 7f5d49e00640 1 -- 172.21.15.19:0/1616480686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44167080 msgr2=0x7f5d440cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:47.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.342+0000 7f5d49e00640 1 --2- 172.21.15.19:0/1616480686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44167080 0x7f5d440cee10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c007e90 tx=0x7f5d3c00e4b0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.342+0000 7f5d49e00640 1 -- 172.21.15.19:0/1616480686 shutdown_connections 2024-03-28T13:11:47.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.342+0000 7f5d49e00640 1 --2- 172.21.15.19:0/1616480686 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5d2006c530 0x7f5d2006e9f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.342+0000 7f5d49e00640 1 --2- 172.21.15.19:0/1616480686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d44167080 0x7f5d440cee10 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.342+0000 7f5d49e00640 1 --2- 172.21.15.19:0/1616480686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4415d120 0x7f5d440b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.342+0000 7f5d49e00640 1 --2- 172.21.15.19:0/1616480686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d440af630 0x7f5d440b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.342+0000 7f5d49e00640 1 -- 172.21.15.19:0/1616480686 >> 172.21.15.19:0/1616480686 conn(0x7f5d4406b990 msgr2=0x7f5d44163e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:47.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.342+0000 7f5d49e00640 1 -- 172.21.15.19:0/1616480686 shutdown_connections 2024-03-28T13:11:47.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.342+0000 7f5d49e00640 1 -- 172.21.15.19:0/1616480686 wait complete. 2024-03-28T13:11:47.361 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:11:47.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.577+0000 7f99fde00640 1 Processor -- start 2024-03-28T13:11:47.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.578+0000 7f99fde00640 1 -- start start 2024-03-28T13:11:47.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.579+0000 7f99fde00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f99f80d0e90 0x7f99f80b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:47.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.579+0000 7f99fde00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f815b6f0 con 0x7f99f815b310 2024-03-28T13:11:47.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.579+0000 7f99fde00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f8072e80 con 0x7f99f80727f0 2024-03-28T13:11:47.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.579+0000 7f99fde00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f99f8072c50 con 0x7f99f80d0e90 2024-03-28T13:11:47.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.579+0000 7f99f6a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f99f815b310 0x7f99f815f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36154/0 (socket says 172.21.15.19:36154) 2024-03-28T13:11:47.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.579+0000 7f99f6a00640 1 -- 172.21.15.19:0/913709274 learned_addr learned my addr 172.21.15.19:0/913709274 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:47.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.580+0000 7f99f5600640 1 -- 172.21.15.19:0/913709274 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3742452373 0 0) 0x7f99f815b6f0 con 0x7f99f815b310 2024-03-28T13:11:47.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.580+0000 7f99f5600640 1 -- 172.21.15.19:0/913709274 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99dc005730 con 0x7f99f815b310 2024-03-28T13:11:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.580+0000 7f99f7400640 1 --2- 172.21.15.19:0/913709274 >> v2:172.21.15.188:3300/0 conn(0x7f99f80d0e90 0x7f99f80b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.580+0000 7f99f5600640 1 -- 172.21.15.19:0/913709274 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4171913899 0 0) 0x7f99dc005730 con 0x7f99f815b310 2024-03-28T13:11:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.580+0000 7f99f5600640 1 -- 172.21.15.19:0/913709274 >> v2:172.21.15.188:3300/0 conn(0x7f99f80d0e90 msgr2=0x7f99f80b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.580+0000 7f99f5600640 1 --2- 172.21.15.19:0/913709274 >> v2:172.21.15.188:3300/0 conn(0x7f99f80d0e90 0x7f99f80b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.580+0000 7f99f5600640 1 -- 172.21.15.19:0/913709274 >> v1:172.21.15.188:6789/0 conn(0x7f99f80727f0 legacy=0x7f99f80d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.580+0000 7f99f5600640 1 -- 172.21.15.19:0/913709274 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f8073160 con 0x7f99f815b310 2024-03-28T13:11:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.580+0000 7f99f5600640 1 -- 172.21.15.19:0/913709274 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f99e8001670 con 0x7f99f815b310 2024-03-28T13:11:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f5600640 1 -- 172.21.15.19:0/913709274 >> v1:172.21.15.19:6789/0 conn(0x7f99f815b310 legacy=0x7f99f815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f5600640 1 --2- 172.21.15.19:0/913709274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99dc005bf0 0x7f99dc006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f5600640 1 --2- 172.21.15.19:0/913709274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99dc006a10 0x7f99dc006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f5600640 1 --2- 172.21.15.19:0/913709274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99dc0075d0 0x7f99dc00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f5600640 1 -- 172.21.15.19:0/913709274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc00e460 con 0x7f99dc006a10 2024-03-28T13:11:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f5600640 1 -- 172.21.15.19:0/913709274 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99dc00e610 con 0x7f99dc005bf0 2024-03-28T13:11:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f5600640 1 -- 172.21.15.19:0/913709274 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f99dc00e810 con 0x7f99dc0075d0 2024-03-28T13:11:47.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f6000640 1 --2- 172.21.15.19:0/913709274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99dc006a10 0x7f99dc006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f7400640 1 --2- 172.21.15.19:0/913709274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99dc0075d0 0x7f99dc00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f6a00640 1 --2- 172.21.15.19:0/913709274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99dc005bf0 0x7f99dc006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f6000640 1 -- 172.21.15.19:0/913709274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99dc0075d0 msgr2=0x7f99dc00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f6000640 1 --2- 172.21.15.19:0/913709274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99dc0075d0 0x7f99dc00dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f6000640 1 -- 172.21.15.19:0/913709274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99dc005bf0 msgr2=0x7f99dc006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f6000640 1 --2- 172.21.15.19:0/913709274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99dc005bf0 0x7f99dc006010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f6000640 1 -- 172.21.15.19:0/913709274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99dc005730 con 0x7f99dc006a10 2024-03-28T13:11:47.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.581+0000 7f99f7400640 1 -- 172.21.15.19:0/913709274 reap_dead start 2024-03-28T13:11:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.583+0000 7f99f6000640 1 --2- 172.21.15.19:0/913709274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99dc006a10 0x7f99dc006e90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f99e0001750 tx=0x7f99dc007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468ad237edb4bb25 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.583+0000 7f99f5600640 1 -- 172.21.15.19:0/913709274 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0001370 con 0x7f99dc006a10 2024-03-28T13:11:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.583+0000 7f99f5600640 1 -- 172.21.15.19:0/913709274 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99e00052d0 con 0x7f99dc006a10 2024-03-28T13:11:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.583+0000 7f99f5600640 1 -- 172.21.15.19:0/913709274 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0003830 con 0x7f99dc006a10 2024-03-28T13:11:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.583+0000 7f99fde00640 1 -- 172.21.15.19:0/913709274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99dc006a10 msgr2=0x7f99dc006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.583+0000 7f99fde00640 1 --2- 172.21.15.19:0/913709274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99dc006a10 0x7f99dc006e90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f99e0001750 tx=0x7f99dc007a40 comp rx=0 tx=0).stop 2024-03-28T13:11:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.584+0000 7f99fde00640 1 -- 172.21.15.19:0/913709274 shutdown_connections 2024-03-28T13:11:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.584+0000 7f99fde00640 1 --2- 172.21.15.19:0/913709274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99dc006a10 0x7f99dc006e90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.584+0000 7f99fde00640 1 -- 172.21.15.19:0/913709274 >> 172.21.15.19:0/913709274 conn(0x7f99f806b990 msgr2=0x7f99f806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:47.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.584+0000 7f99fde00640 1 -- 172.21.15.19:0/913709274 shutdown_connections 2024-03-28T13:11:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.584+0000 7f99fde00640 1 -- 172.21.15.19:0/913709274 wait complete. 2024-03-28T13:11:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.585+0000 7f99fde00640 1 Processor -- start 2024-03-28T13:11:47.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.586+0000 7f99fde00640 1 -- start start 2024-03-28T13:11:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.586+0000 7f99fde00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99dc006a10 0x7f99f80cf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.587+0000 7f99fde00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99f8150a90 0x7f99f814f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.587+0000 7f99fde00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f80ccf20 0x7f99f80cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.587+0000 7f99fde00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99f8073440 con 0x7f99f80ccf20 2024-03-28T13:11:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.587+0000 7f99fde00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99f815f2c0 con 0x7f99dc006a10 2024-03-28T13:11:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.587+0000 7f99fde00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f99f80b4f90 con 0x7f99f8150a90 2024-03-28T13:11:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.587+0000 7f99f7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f80ccf20 0x7f99f80cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.587+0000 7f99f7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f80ccf20 0x7f99f80cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45378/0 (socket says 172.21.15.19:45378) 2024-03-28T13:11:47.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.587+0000 7f99f7400640 1 -- 172.21.15.19:0/2279986303 learned_addr learned my addr 172.21.15.19:0/2279986303 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:47.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.587+0000 7f99f7400640 1 -- 172.21.15.19:0/2279986303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99f8150a90 msgr2=0x7f99f814f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.587+0000 7f99f7400640 1 --2- 172.21.15.19:0/2279986303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99f8150a90 0x7f99f814f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.587+0000 7f99f7400640 1 -- 172.21.15.19:0/2279986303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99dc006a10 msgr2=0x7f99f80cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.587+0000 7f99f7400640 1 --2- 172.21.15.19:0/2279986303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99dc006a10 0x7f99f80cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.587+0000 7f99f7400640 1 -- 172.21.15.19:0/2279986303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f8151060 con 0x7f99f80ccf20 2024-03-28T13:11:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.588+0000 7f99f7400640 1 --2- 172.21.15.19:0/2279986303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f80ccf20 0x7f99f80cd3a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009c30 tx=0x7f99f0007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.588+0000 7f99ef400640 1 -- 172.21.15.19:0/2279986303 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0014410 con 0x7f99f80ccf20 2024-03-28T13:11:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.588+0000 7f99ef400640 1 -- 172.21.15.19:0/2279986303 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f0019070 con 0x7f99f80ccf20 2024-03-28T13:11:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.588+0000 7f99fde00640 1 -- 172.21.15.19:0/2279986303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f8151340 con 0x7f99f80ccf20 2024-03-28T13:11:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.588+0000 7f99ef400640 1 -- 172.21.15.19:0/2279986303 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0014810 con 0x7f99f80ccf20 2024-03-28T13:11:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.588+0000 7f99fde00640 1 -- 172.21.15.19:0/2279986303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f806d890 con 0x7f99f80ccf20 2024-03-28T13:11:47.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.589+0000 7f99fde00640 1 -- 172.21.15.19:0/2279986303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f806da90 con 0x7f99f80ccf20 2024-03-28T13:11:47.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.590+0000 7f99ef400640 1 -- 172.21.15.19:0/2279986303 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f99f00149b0 con 0x7f99f80ccf20 2024-03-28T13:11:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.591+0000 7f99ef400640 1 --2- 172.21.15.19:0/2279986303 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f99d406c560 0x7f99d406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.591+0000 7f99ef400640 1 -- 172.21.15.19:0/2279986303 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5709+0+0 (secure 0 0 0) 0x7f99f008bce0 con 0x7f99f80ccf20 2024-03-28T13:11:47.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.592+0000 7f99f6a00640 1 --2- 172.21.15.19:0/2279986303 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f99d406c560 0x7f99d406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:47.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.593+0000 7f99f6a00640 1 --2- 172.21.15.19:0/2279986303 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f99d406c560 0x7f99d406ea20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f99e80017e0 tx=0x7f99e8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:47.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.596+0000 7f99ef400640 1 -- 172.21.15.19:0/2279986303 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f99f806da90 con 0x7f99f80ccf20 2024-03-28T13:11:47.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.797+0000 7f99fde00640 1 -- 172.21.15.19:0/2279986303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99f806d680 con 0x7f99f80ccf20 2024-03-28T13:11:47.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.800+0000 7f99ef400640 1 -- 172.21.15.19:0/2279986303 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) ==== 74+0+16622 (secure 0 0 0) 0x7f99f005a2f0 con 0x7f99f80ccf20 2024-03-28T13:11:47.800 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:47.800 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":48,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:11:46.379407+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:11:44.316258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-28T13:11:45.360154+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:11:47.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.803+0000 7f99fde00640 1 -- 172.21.15.19:0/2279986303 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f99d406c560 msgr2=0x7f99d406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:47.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.803+0000 7f99fde00640 1 --2- 172.21.15.19:0/2279986303 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f99d406c560 0x7f99d406ea20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f99e80017e0 tx=0x7f99e8009040 comp rx=0 tx=0).stop 2024-03-28T13:11:47.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.803+0000 7f99fde00640 1 -- 172.21.15.19:0/2279986303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f80ccf20 msgr2=0x7f99f80cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:47.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.803+0000 7f99fde00640 1 --2- 172.21.15.19:0/2279986303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f80ccf20 0x7f99f80cd3a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009c30 tx=0x7f99f0007700 comp rx=0 tx=0).stop 2024-03-28T13:11:47.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.803+0000 7f99fde00640 1 -- 172.21.15.19:0/2279986303 shutdown_connections 2024-03-28T13:11:47.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.803+0000 7f99fde00640 1 --2- 172.21.15.19:0/2279986303 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f99d406c560 0x7f99d406ea20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.803+0000 7f99fde00640 1 --2- 172.21.15.19:0/2279986303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f80ccf20 0x7f99f80cd3a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.803+0000 7f99fde00640 1 --2- 172.21.15.19:0/2279986303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99f8150a90 0x7f99f814f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.803+0000 7f99fde00640 1 --2- 172.21.15.19:0/2279986303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99dc006a10 0x7f99f80cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:47.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.803+0000 7f99fde00640 1 -- 172.21.15.19:0/2279986303 >> 172.21.15.19:0/2279986303 conn(0x7f99f806b990 msgr2=0x7f99f806bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:47.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.804+0000 7f99fde00640 1 -- 172.21.15.19:0/2279986303 shutdown_connections 2024-03-28T13:11:47.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:47.804+0000 7f99fde00640 1 -- 172.21.15.19:0/2279986303 wait complete. 2024-03-28T13:11:47.820 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-28T13:11:48.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.026+0000 7f75faa00640 1 Processor -- start 2024-03-28T13:11:48.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.026+0000 7f75faa00640 1 -- start start 2024-03-28T13:11:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.028+0000 7f75faa00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f75f415e4c0 0x7f75f40b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.028+0000 7f75faa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f75f40b69c0 0x7f75f40b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.028+0000 7f75faa00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75f40d2710 con 0x7f75f40d37b0 2024-03-28T13:11:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.028+0000 7f75faa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f75f415e8a0 con 0x7f75f40b69c0 2024-03-28T13:11:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.028+0000 7f75faa00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f75f40d4040 con 0x7f75f415e4c0 2024-03-28T13:11:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.028+0000 7f75fa000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f75f40b69c0 0x7f75f40b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.028+0000 7f75f8c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f75f40d37b0 0x7f75f40b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36160/0 (socket says 172.21.15.19:36160) 2024-03-28T13:11:48.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.028+0000 7f75f8c00640 1 -- 172.21.15.19:0/2983112754 learned_addr learned my addr 172.21.15.19:0/2983112754 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.029+0000 7f75fa000640 1 -- 172.21.15.19:0/2983112754 >> v2:172.21.15.188:3300/0 conn(0x7f75f415e4c0 msgr2=0x7f75f40b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.029+0000 7f75f9600640 1 --2- 172.21.15.19:0/2983112754 >> v2:172.21.15.188:3300/0 conn(0x7f75f415e4c0 0x7f75f40b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.029+0000 7f75fa000640 1 --2- 172.21.15.19:0/2983112754 >> v2:172.21.15.188:3300/0 conn(0x7f75f415e4c0 0x7f75f40b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.029+0000 7f75fa000640 1 -- 172.21.15.19:0/2983112754 >> v1:172.21.15.19:6789/0 conn(0x7f75f40d37b0 legacy=0x7f75f40b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.029+0000 7f75fa000640 1 -- 172.21.15.19:0/2983112754 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f40d29f0 con 0x7f75f40b69c0 2024-03-28T13:11:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.029+0000 7f75f3e00640 1 -- 172.21.15.19:0/2983112754 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2469425518 0 0) 0x7f75f40d2710 con 0x7f75f40d37b0 2024-03-28T13:11:48.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.029+0000 7f75f9600640 1 --2- 172.21.15.19:0/2983112754 >> v2:172.21.15.188:3300/0 conn(0x7f75f415e4c0 0x7f75f40b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:48.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.030+0000 7f75fa000640 1 --2- 172.21.15.19:0/2983112754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75f40b69c0 0x7f75f40b7400 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f75f40d0d70 tx=0x7f75ec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820075cb506b3e92 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:48.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.030+0000 7f75f3e00640 1 -- 172.21.15.19:0/2983112754 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec039070 con 0x7f75f40b69c0 2024-03-28T13:11:48.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.030+0000 7f75f3e00640 1 -- 172.21.15.19:0/2983112754 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75ec02cc80 con 0x7f75f40b69c0 2024-03-28T13:11:48.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.030+0000 7f75f3e00640 1 -- 172.21.15.19:0/2983112754 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ec034660 con 0x7f75f40b69c0 2024-03-28T13:11:48.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.031+0000 7f75faa00640 1 -- 172.21.15.19:0/2983112754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75f40b69c0 msgr2=0x7f75f40b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:48.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.031+0000 7f75faa00640 1 --2- 172.21.15.19:0/2983112754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75f40b69c0 0x7f75f40b7400 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f75f40d0d70 tx=0x7f75ec02bed0 comp rx=0 tx=0).stop 2024-03-28T13:11:48.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.031+0000 7f75faa00640 1 -- 172.21.15.19:0/2983112754 shutdown_connections 2024-03-28T13:11:48.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.031+0000 7f75faa00640 1 --2- 172.21.15.19:0/2983112754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75f40b69c0 0x7f75f40b7400 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:48.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.031+0000 7f75faa00640 1 --2- 172.21.15.19:0/2983112754 >> v2:172.21.15.188:3300/0 conn(0x7f75f415e4c0 0x7f75f40b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:48.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.031+0000 7f75faa00640 1 -- 172.21.15.19:0/2983112754 >> 172.21.15.19:0/2983112754 conn(0x7f75f40676a0 msgr2=0x7f75f4067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:48.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.031+0000 7f75faa00640 1 -- 172.21.15.19:0/2983112754 shutdown_connections 2024-03-28T13:11:48.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.031+0000 7f75faa00640 1 -- 172.21.15.19:0/2983112754 wait complete. 2024-03-28T13:11:48.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.033+0000 7f75faa00640 1 Processor -- start 2024-03-28T13:11:48.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.033+0000 7f75faa00640 1 -- start start 2024-03-28T13:11:48.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.034+0000 7f75faa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75f40b69c0 0x7f75f41540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:48.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.034+0000 7f75faa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75f40d37b0 0x7f75f40cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:48.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.034+0000 7f75faa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f415e4c0 0x7f75f40cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:48.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.034+0000 7f75faa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75f40d2fb0 con 0x7f75f40d37b0 2024-03-28T13:11:48.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.034+0000 7f75faa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f75f40d3320 con 0x7f75f415e4c0 2024-03-28T13:11:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.034+0000 7f75faa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f75f40ba130 con 0x7f75f40b69c0 2024-03-28T13:11:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.034+0000 7f75f8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75f40d37b0 0x7f75f40cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.034+0000 7f75f8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75f40d37b0 0x7f75f40cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45390/0 (socket says 172.21.15.19:45390) 2024-03-28T13:11:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.034+0000 7f75f8c00640 1 -- 172.21.15.19:0/1257223514 learned_addr learned my addr 172.21.15.19:0/1257223514 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.034+0000 7f75fa000640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f415e4c0 0x7f75f40cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.034+0000 7f75f8c00640 1 -- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75f40b69c0 msgr2=0x7f75f41540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.034+0000 7f75f9600640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75f40b69c0 0x7f75f41540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.035+0000 7f75f8c00640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75f40b69c0 0x7f75f41540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.035+0000 7f75f8c00640 1 -- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f415e4c0 msgr2=0x7f75f40cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.035+0000 7f75f8c00640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f415e4c0 0x7f75f40cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.035+0000 7f75f8c00640 1 -- 172.21.15.19:0/1257223514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f41f73a0 con 0x7f75f40d37b0 2024-03-28T13:11:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.035+0000 7f75fa000640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f415e4c0 0x7f75f40cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.035+0000 7f75f9600640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75f40b69c0 0x7f75f41540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:48.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.035+0000 7f75f8c00640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75f40d37b0 0x7f75f40cda30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f75e000c500 tx=0x7f75e000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.035+0000 7f75f1600640 1 -- 172.21.15.19:0/1257223514 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e0018070 con 0x7f75f40d37b0 2024-03-28T13:11:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.035+0000 7f75f1600640 1 -- 172.21.15.19:0/1257223514 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75e0007580 con 0x7f75f40d37b0 2024-03-28T13:11:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.035+0000 7f75faa00640 1 -- 172.21.15.19:0/1257223514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f41f7680 con 0x7f75f40d37b0 2024-03-28T13:11:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.035+0000 7f75f1600640 1 -- 172.21.15.19:0/1257223514 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e0004a20 con 0x7f75f40d37b0 2024-03-28T13:11:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.035+0000 7f75faa00640 1 -- 172.21.15.19:0/1257223514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f41f7c40 con 0x7f75f40d37b0 2024-03-28T13:11:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.037+0000 7f75faa00640 1 -- 172.21.15.19:0/1257223514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75f4159820 con 0x7f75f40d37b0 2024-03-28T13:11:48.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.037+0000 7f75f1600640 1 -- 172.21.15.19:0/1257223514 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f75e001b070 con 0x7f75f40d37b0 2024-03-28T13:11:48.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.038+0000 7f75f1600640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f75d406c530 0x7f75d406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:48.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.038+0000 7f75f1600640 1 -- 172.21.15.19:0/1257223514 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f75e008a950 con 0x7f75f40d37b0 2024-03-28T13:11:48.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.039+0000 7f75f9600640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f75d406c530 0x7f75d406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:48.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.040+0000 7f75f9600640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f75d406c530 0x7f75d406e9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f75e8006fd0 tx=0x7f75e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:48.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.042+0000 7f75f1600640 1 -- 172.21.15.19:0/1257223514 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f75f4159820 con 0x7f75f40d37b0 2024-03-28T13:11:48.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.339+0000 7f75faa00640 1 -- 172.21.15.19:0/1257223514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f75f4075940 con 0x7f75f40d37b0 2024-03-28T13:11:48.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.939+0000 7f75f1600640 1 -- 172.21.15.19:0/1257223514 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f75e0059000 con 0x7f75f40d37b0 2024-03-28T13:11:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.944+0000 7f75faa00640 1 -- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f75d406c530 msgr2=0x7f75d406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.944+0000 7f75faa00640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f75d406c530 0x7f75d406e9f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f75e8006fd0 tx=0x7f75e8008040 comp rx=0 tx=0).stop 2024-03-28T13:11:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.944+0000 7f75faa00640 1 -- 172.21.15.19:0/1257223514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75f40d37b0 msgr2=0x7f75f40cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.944+0000 7f75faa00640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75f40d37b0 0x7f75f40cda30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f75e000c500 tx=0x7f75e000c9c0 comp rx=0 tx=0).stop 2024-03-28T13:11:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.944+0000 7f75faa00640 1 -- 172.21.15.19:0/1257223514 shutdown_connections 2024-03-28T13:11:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.944+0000 7f75faa00640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f75d406c530 0x7f75d406e9f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:48.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.944+0000 7f75faa00640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75f415e4c0 0x7f75f40cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.944+0000 7f75faa00640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75f40d37b0 0x7f75f40cda30 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.944+0000 7f75faa00640 1 --2- 172.21.15.19:0/1257223514 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75f40b69c0 0x7f75f41540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.944+0000 7f75faa00640 1 -- 172.21.15.19:0/1257223514 >> 172.21.15.19:0/1257223514 conn(0x7f75f40676a0 msgr2=0x7f75f415c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.944+0000 7f75faa00640 1 -- 172.21.15.19:0/1257223514 shutdown_connections 2024-03-28T13:11:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:48.944+0000 7f75faa00640 1 -- 172.21.15.19:0/1257223514 wait complete. 2024-03-28T13:11:48.978 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-28T13:11:48.978 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-28T13:11:49.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.203+0000 7f9583200640 1 Processor -- start 2024-03-28T13:11:49.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.203+0000 7f9583200640 1 -- start start 2024-03-28T13:11:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.204+0000 7f9583200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f958415e4c0 0x7f95840b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:49.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.204+0000 7f9583200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f95840d37b0 0x7f95840b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.204+0000 7f9583200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f95840b67f0 0x7f95840d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.204+0000 7f9583200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f958415e8a0 con 0x7f95840b67f0 2024-03-28T13:11:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.204+0000 7f9583200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f95840d4040 con 0x7f95840d37b0 2024-03-28T13:11:49.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.204+0000 7f9583200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f95840d3320 con 0x7f958415e4c0 2024-03-28T13:11:49.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.205+0000 7f9581400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f95840d37b0 0x7f95840b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:49.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.205+0000 7f9582800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f95840b67f0 0x7f95840d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:49.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.205+0000 7f9581e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f958415e4c0 0x7f95840b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:49.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.205+0000 7f9581400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f95840d37b0 0x7f95840b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:58114/0 (socket says 172.21.15.19:58114) 2024-03-28T13:11:49.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.205+0000 7f9581400640 1 -- 172.21.15.19:0/2580152635 learned_addr learned my addr 172.21.15.19:0/2580152635 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:49.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.206+0000 7f9582800640 1 -- 172.21.15.19:0/2580152635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f958415e4c0 msgr2=0x7f95840b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:49.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.206+0000 7f9582800640 1 --2- 172.21.15.19:0/2580152635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f958415e4c0 0x7f95840b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:49.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.206+0000 7f9582800640 1 -- 172.21.15.19:0/2580152635 >> v2:172.21.15.188:3300/0 conn(0x7f95840d37b0 msgr2=0x7f95840b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:49.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.206+0000 7f9582800640 1 --2- 172.21.15.19:0/2580152635 >> v2:172.21.15.188:3300/0 conn(0x7f95840d37b0 0x7f95840b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:49.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.206+0000 7f9582800640 1 -- 172.21.15.19:0/2580152635 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95840d2710 con 0x7f95840b67f0 2024-03-28T13:11:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.208+0000 7f9582800640 1 --2- 172.21.15.19:0/2580152635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95840b67f0 0x7f95840d4260 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f95840d5330 tx=0x7f957c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e292d6e4a2dfa9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.209+0000 7f9580a00640 1 -- 172.21.15.19:0/2580152635 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c0315a0 con 0x7f95840b67f0 2024-03-28T13:11:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.209+0000 7f9580a00640 1 -- 172.21.15.19:0/2580152635 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f957c037070 con 0x7f95840b67f0 2024-03-28T13:11:49.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.209+0000 7f9580a00640 1 -- 172.21.15.19:0/2580152635 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c031960 con 0x7f95840b67f0 2024-03-28T13:11:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.209+0000 7f9583200640 1 -- 172.21.15.19:0/2580152635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95840b67f0 msgr2=0x7f95840d4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.209+0000 7f9583200640 1 --2- 172.21.15.19:0/2580152635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95840b67f0 0x7f95840d4260 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f95840d5330 tx=0x7f957c02b840 comp rx=0 tx=0).stop 2024-03-28T13:11:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.210+0000 7f9583200640 1 -- 172.21.15.19:0/2580152635 shutdown_connections 2024-03-28T13:11:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.210+0000 7f9583200640 1 --2- 172.21.15.19:0/2580152635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95840b67f0 0x7f95840d4260 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.210+0000 7f9583200640 1 --2- 172.21.15.19:0/2580152635 >> v2:172.21.15.188:3300/0 conn(0x7f95840d37b0 0x7f95840b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.210+0000 7f9583200640 1 --2- 172.21.15.19:0/2580152635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f958415e4c0 0x7f95840b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.210+0000 7f9583200640 1 -- 172.21.15.19:0/2580152635 >> 172.21.15.19:0/2580152635 conn(0x7f95840676a0 msgr2=0x7f9584067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.210+0000 7f9583200640 1 -- 172.21.15.19:0/2580152635 shutdown_connections 2024-03-28T13:11:49.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.210+0000 7f9583200640 1 -- 172.21.15.19:0/2580152635 wait complete. 2024-03-28T13:11:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.211+0000 7f9583200640 1 Processor -- start 2024-03-28T13:11:49.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.212+0000 7f9583200640 1 -- start start 2024-03-28T13:11:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.212+0000 7f9583200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95840b67f0 0x7f95841f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9583200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95840d37b0 0x7f9584156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9583200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f958415e4c0 0x7f9584157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9583200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95840d2fb0 con 0x7f95840d37b0 2024-03-28T13:11:49.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9583200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f95840d2cd0 con 0x7f95840b67f0 2024-03-28T13:11:49.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9583200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f95840ba130 con 0x7f958415e4c0 2024-03-28T13:11:49.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9581400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95840d37b0 0x7f9584156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:49.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9581400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95840d37b0 0x7f9584156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45406/0 (socket says 172.21.15.19:45406) 2024-03-28T13:11:49.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9581400640 1 -- 172.21.15.19:0/545623147 learned_addr learned my addr 172.21.15.19:0/545623147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:49.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9581400640 1 -- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f958415e4c0 msgr2=0x7f9584157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:49.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9582800640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f958415e4c0 0x7f9584157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:49.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9581e00640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95840b67f0 0x7f95841f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:49.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9581400640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f958415e4c0 0x7f9584157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:49.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9581400640 1 -- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95840b67f0 msgr2=0x7f95841f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:49.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9581400640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95840b67f0 0x7f95841f37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:49.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9581400640 1 -- 172.21.15.19:0/545623147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9584152260 con 0x7f95840d37b0 2024-03-28T13:11:49.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9582800640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f958415e4c0 0x7f9584157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:49.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.213+0000 7f9581e00640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95840b67f0 0x7f95841f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:49.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.214+0000 7f9581400640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95840d37b0 0x7f9584156fe0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f956c009fd0 tx=0x7f956c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:49.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.214+0000 7f957a000640 1 -- 172.21.15.19:0/545623147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c017070 con 0x7f95840d37b0 2024-03-28T13:11:49.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.214+0000 7f957a000640 1 -- 172.21.15.19:0/545623147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f956c00c940 con 0x7f95840d37b0 2024-03-28T13:11:49.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.214+0000 7f957a000640 1 -- 172.21.15.19:0/545623147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956c00cd10 con 0x7f95840d37b0 2024-03-28T13:11:49.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.214+0000 7f9583200640 1 -- 172.21.15.19:0/545623147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9584157e20 con 0x7f95840d37b0 2024-03-28T13:11:49.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.214+0000 7f9583200640 1 -- 172.21.15.19:0/545623147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95841583e0 con 0x7f95840d37b0 2024-03-28T13:11:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.215+0000 7f9583200640 1 -- 172.21.15.19:0/545623147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9584075940 con 0x7f95840d37b0 2024-03-28T13:11:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.216+0000 7f957a000640 1 -- 172.21.15.19:0/545623147 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f956c01a070 con 0x7f95840d37b0 2024-03-28T13:11:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.217+0000 7f957a000640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f956006c530 0x7f956006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:49.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.217+0000 7f957a000640 1 -- 172.21.15.19:0/545623147 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f956c08b5b0 con 0x7f95840d37b0 2024-03-28T13:11:49.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.218+0000 7f9581e00640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f956006c530 0x7f956006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:49.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.219+0000 7f9581e00640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f956006c530 0x7f956006e9f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9574007440 tx=0x7f957400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:49.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.222+0000 7f957a000640 1 -- 172.21.15.19:0/545623147 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f956c059c60 con 0x7f95840d37b0 2024-03-28T13:11:49.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.469+0000 7f9583200640 1 -- 172.21.15.19:0/545623147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f958406b570 con 0x7f95840d37b0 2024-03-28T13:11:49.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.942+0000 7f957a000640 1 -- 172.21.15.19:0/545623147 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f956c059560 con 0x7f95840d37b0 2024-03-28T13:11:49.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.946+0000 7f9583200640 1 -- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f956006c530 msgr2=0x7f956006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:49.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.946+0000 7f9583200640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f956006c530 0x7f956006e9f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9574007440 tx=0x7f957400b040 comp rx=0 tx=0).stop 2024-03-28T13:11:49.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.946+0000 7f9583200640 1 -- 172.21.15.19:0/545623147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95840d37b0 msgr2=0x7f9584156fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:49.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.946+0000 7f9583200640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95840d37b0 0x7f9584156fe0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f956c009fd0 tx=0x7f956c00ed40 comp rx=0 tx=0).stop 2024-03-28T13:11:49.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.946+0000 7f9583200640 1 -- 172.21.15.19:0/545623147 shutdown_connections 2024-03-28T13:11:49.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.946+0000 7f9583200640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f956006c530 0x7f956006e9f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:49.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.946+0000 7f9583200640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f958415e4c0 0x7f9584157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:49.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.946+0000 7f9583200640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95840d37b0 0x7f9584156fe0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:49.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.946+0000 7f9583200640 1 --2- 172.21.15.19:0/545623147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95840b67f0 0x7f95841f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:49.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.946+0000 7f9583200640 1 -- 172.21.15.19:0/545623147 >> 172.21.15.19:0/545623147 conn(0x7f95840676a0 msgr2=0x7f958415cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:49.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.946+0000 7f9583200640 1 -- 172.21.15.19:0/545623147 shutdown_connections 2024-03-28T13:11:49.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:49.947+0000 7f9583200640 1 -- 172.21.15.19:0/545623147 wait complete. 2024-03-28T13:11:49.970 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-28T13:11:49.970 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:11:50.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.208+0000 7f72bca00640 1 Processor -- start 2024-03-28T13:11:50.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.209+0000 7f72bca00640 1 -- start start 2024-03-28T13:11:50.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.210+0000 7f72bca00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f72b815fc50 0x7f72b80cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:50.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.210+0000 7f72bca00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72b80b3150 con 0x7f72b80cc0b0 2024-03-28T13:11:50.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.210+0000 7f72bca00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72b81600e0 con 0x7f72b80cc490 2024-03-28T13:11:50.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.210+0000 7f72bca00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f72b815c610 con 0x7f72b815fc50 2024-03-28T13:11:50.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.211+0000 7f72b6800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f72b80cc0b0 0x7f72b806dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36170/0 (socket says 172.21.15.19:36170) 2024-03-28T13:11:50.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.211+0000 7f72b6800640 1 -- 172.21.15.19:0/3772861724 learned_addr learned my addr 172.21.15.19:0/3772861724 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:50.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.211+0000 7f72b5e00640 1 --2- 172.21.15.19:0/3772861724 >> v2:172.21.15.188:3300/0 conn(0x7f72b815fc50 0x7f72b80cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:50.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.211+0000 7f72b5400640 1 -- 172.21.15.19:0/3772861724 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1825886159 0 0) 0x7f72b80b3150 con 0x7f72b80cc0b0 2024-03-28T13:11:50.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.211+0000 7f72b5400640 1 -- 172.21.15.19:0/3772861724 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f729c003520 con 0x7f72b80cc0b0 2024-03-28T13:11:50.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.212+0000 7f72b5400640 1 -- 172.21.15.19:0/3772861724 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1412149095 0 0) 0x7f729c003520 con 0x7f72b80cc0b0 2024-03-28T13:11:50.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.212+0000 7f72b5400640 1 -- 172.21.15.19:0/3772861724 >> v2:172.21.15.188:3300/0 conn(0x7f72b815fc50 msgr2=0x7f72b80cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:50.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.212+0000 7f72b5400640 1 --2- 172.21.15.19:0/3772861724 >> v2:172.21.15.188:3300/0 conn(0x7f72b815fc50 0x7f72b80cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.212+0000 7f72b5400640 1 -- 172.21.15.19:0/3772861724 >> v1:172.21.15.188:6789/0 conn(0x7f72b80cc490 legacy=0x7f72b815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:50.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.212+0000 7f72b5400640 1 -- 172.21.15.19:0/3772861724 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b81603c0 con 0x7f72b80cc0b0 2024-03-28T13:11:50.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.212+0000 7f72b5400640 1 -- 172.21.15.19:0/3772861724 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f72a8001670 con 0x7f72b80cc0b0 2024-03-28T13:11:50.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.212+0000 7f72b5400640 1 -- 172.21.15.19:0/3772861724 >> v1:172.21.15.19:6789/0 conn(0x7f72b80cc0b0 legacy=0x7f72b806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:50.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.212+0000 7f72b5400640 1 --2- 172.21.15.19:0/3772861724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f729c0039b0 0x7f729c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:50.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.212+0000 7f72b5400640 1 --2- 172.21.15.19:0/3772861724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f729c004800 0x7f729c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:50.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.212+0000 7f72b5400640 1 --2- 172.21.15.19:0/3772861724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f729c0051c0 0x7f729c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:50.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.212+0000 7f72b5400640 1 -- 172.21.15.19:0/3772861724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72b815c610 con 0x7f729c004800 2024-03-28T13:11:50.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.212+0000 7f72b5400640 1 -- 172.21.15.19:0/3772861724 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f729c00c060 con 0x7f729c0051c0 2024-03-28T13:11:50.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.212+0000 7f72b5400640 1 -- 172.21.15.19:0/3772861724 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f729c00c260 con 0x7f729c0039b0 2024-03-28T13:11:50.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.212+0000 7f72b5e00640 1 --2- 172.21.15.19:0/3772861724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f729c004800 0x7f729c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:50.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.213+0000 7f72b6800640 1 --2- 172.21.15.19:0/3772861724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f729c0039b0 0x7f729c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:50.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.213+0000 7f72b7200640 1 --2- 172.21.15.19:0/3772861724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f729c0051c0 0x7f729c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:50.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.213+0000 7f72b5e00640 1 -- 172.21.15.19:0/3772861724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f729c0039b0 msgr2=0x7f729c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:50.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.213+0000 7f72b5e00640 1 --2- 172.21.15.19:0/3772861724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f729c0039b0 0x7f729c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.213+0000 7f72b5e00640 1 -- 172.21.15.19:0/3772861724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f729c0051c0 msgr2=0x7f729c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:50.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.213+0000 7f72b5e00640 1 --2- 172.21.15.19:0/3772861724 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f729c0051c0 0x7f729c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.213+0000 7f72b5e00640 1 -- 172.21.15.19:0/3772861724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c003520 con 0x7f729c004800 2024-03-28T13:11:50.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.213+0000 7f72b7200640 1 -- 172.21.15.19:0/3772861724 reap_dead start 2024-03-28T13:11:50.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.214+0000 7f72b5e00640 1 --2- 172.21.15.19:0/3772861724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f729c004800 0x7f729c004c80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009ac0 tx=0x7f729c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b29ac8d2d58a6eb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:50.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.214+0000 7f72b5400640 1 -- 172.21.15.19:0/3772861724 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0039070 con 0x7f729c004800 2024-03-28T13:11:50.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.214+0000 7f72b5400640 1 -- 172.21.15.19:0/3772861724 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a002c6b0 con 0x7f729c004800 2024-03-28T13:11:50.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.214+0000 7f72b5400640 1 -- 172.21.15.19:0/3772861724 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0009690 con 0x7f729c004800 2024-03-28T13:11:50.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.215+0000 7f72bca00640 1 -- 172.21.15.19:0/3772861724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f729c004800 msgr2=0x7f729c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:50.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.215+0000 7f72bca00640 1 --2- 172.21.15.19:0/3772861724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f729c004800 0x7f729c004c80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009ac0 tx=0x7f729c005830 comp rx=0 tx=0).stop 2024-03-28T13:11:50.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.215+0000 7f72bca00640 1 -- 172.21.15.19:0/3772861724 shutdown_connections 2024-03-28T13:11:50.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.215+0000 7f72bca00640 1 --2- 172.21.15.19:0/3772861724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f729c004800 0x7f729c004c80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.215+0000 7f72bca00640 1 -- 172.21.15.19:0/3772861724 >> 172.21.15.19:0/3772861724 conn(0x7f72b80ccad0 msgr2=0x7f72b80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:50.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.215+0000 7f72bca00640 1 -- 172.21.15.19:0/3772861724 shutdown_connections 2024-03-28T13:11:50.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.215+0000 7f72bca00640 1 -- 172.21.15.19:0/3772861724 wait complete. 2024-03-28T13:11:50.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.217+0000 7f72bca00640 1 Processor -- start 2024-03-28T13:11:50.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.217+0000 7f72bca00640 1 -- start start 2024-03-28T13:11:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.218+0000 7f72bca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f729c004800 0x7f72b81f8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.218+0000 7f72bca00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b8133700 0x7f72b81f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.218+0000 7f72bca00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b81f8000 0x7f72b8159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.218+0000 7f72bca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72b806dfb0 con 0x7f729c004800 2024-03-28T13:11:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.218+0000 7f72bca00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f72b8160980 con 0x7f72b81f8000 2024-03-28T13:11:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.218+0000 7f72bca00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f72b806e130 con 0x7f72b8133700 2024-03-28T13:11:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.218+0000 7f72b6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f729c004800 0x7f72b81f8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.218+0000 7f72b6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f729c004800 0x7f72b81f8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45414/0 (socket says 172.21.15.19:45414) 2024-03-28T13:11:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.218+0000 7f72b6800640 1 -- 172.21.15.19:0/3158182591 learned_addr learned my addr 172.21.15.19:0/3158182591 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.219+0000 7f72b5e00640 1 --2- 172.21.15.19:0/3158182591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b8133700 0x7f72b81f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:50.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.219+0000 7f72b7200640 1 --2- 172.21.15.19:0/3158182591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b81f8000 0x7f72b8159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:50.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.219+0000 7f72b6800640 1 -- 172.21.15.19:0/3158182591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b8133700 msgr2=0x7f72b81f7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:50.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.219+0000 7f72b6800640 1 --2- 172.21.15.19:0/3158182591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b8133700 0x7f72b81f7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.219+0000 7f72b6800640 1 -- 172.21.15.19:0/3158182591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b81f8000 msgr2=0x7f72b8159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:50.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.219+0000 7f72b6800640 1 --2- 172.21.15.19:0/3158182591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b81f8000 0x7f72b8159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.219+0000 7f72b6800640 1 -- 172.21.15.19:0/3158182591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b8072ab0 con 0x7f729c004800 2024-03-28T13:11:50.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.219+0000 7f72b6800640 1 --2- 172.21.15.19:0/3158182591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f729c004800 0x7f72b81f8660 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f72a800e7b0 tx=0x7f72a800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:50.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.219+0000 7f72aea00640 1 -- 172.21.15.19:0/3158182591 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a800ee60 con 0x7f729c004800 2024-03-28T13:11:50.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.220+0000 7f72aea00640 1 -- 172.21.15.19:0/3158182591 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a8018070 con 0x7f729c004800 2024-03-28T13:11:50.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.220+0000 7f72bca00640 1 -- 172.21.15.19:0/3158182591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b8158b80 con 0x7f729c004800 2024-03-28T13:11:50.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.220+0000 7f72aea00640 1 -- 172.21.15.19:0/3158182591 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a800cab0 con 0x7f729c004800 2024-03-28T13:11:50.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.220+0000 7f72bca00640 1 -- 172.21.15.19:0/3158182591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b8159140 con 0x7f729c004800 2024-03-28T13:11:50.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.221+0000 7f72bca00640 1 -- 172.21.15.19:0/3158182591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b8075940 con 0x7f729c004800 2024-03-28T13:11:50.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.223+0000 7f72aea00640 1 -- 172.21.15.19:0/3158182591 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f72a800cc50 con 0x7f729c004800 2024-03-28T13:11:50.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.224+0000 7f72aea00640 1 --2- 172.21.15.19:0/3158182591 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f729406c530 0x7f729406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:50.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.224+0000 7f72aea00640 1 -- 172.21.15.19:0/3158182591 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f72a808b540 con 0x7f729c004800 2024-03-28T13:11:50.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.224+0000 7f72b5e00640 1 --2- 172.21.15.19:0/3158182591 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f729406c530 0x7f729406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:50.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.225+0000 7f72b5e00640 1 --2- 172.21.15.19:0/3158182591 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f729406c530 0x7f729406e9f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f72a0036000 tx=0x7f72a0030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:50.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.228+0000 7f72aea00640 1 -- 172.21.15.19:0/3158182591 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f72a8059b50 con 0x7f729c004800 2024-03-28T13:11:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.470+0000 7f72bca00640 1 -- 172.21.15.19:0/3158182591 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72b81f83e0 con 0x7f729c004800 2024-03-28T13:11:50.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.470+0000 7f72aea00640 1 -- 172.21.15.19:0/3158182591 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7f72a8059450 con 0x7f729c004800 2024-03-28T13:11:50.471 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:50.471 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2125648249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2125648249},{"type":"v1","addr":"172.21.15.188:6836","nonce":2125648249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/429866161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":429866161},{"type":"v1","addr":"172.21.15.188:6837","nonce":429866161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1422209289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1422209289},{"type":"v1","addr":"172.21.15.19:6834","nonce":1422209289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:11:46.379929+0000","modified":"2024-03-28T13:11:49.470791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5538},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5538":{"gid":5538,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/179338886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":179338886},{"type":"v1","addr":"172.21.15.19:6835","nonce":179338886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5538,"qdb_cluster":[5538]},"id":3}]} 2024-03-28T13:11:50.471 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-03-28T13:11:50.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.473+0000 7f72bca00640 1 -- 172.21.15.19:0/3158182591 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f729406c530 msgr2=0x7f729406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:50.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.473+0000 7f72bca00640 1 --2- 172.21.15.19:0/3158182591 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f729406c530 0x7f729406e9f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f72a0036000 tx=0x7f72a0030000 comp rx=0 tx=0).stop 2024-03-28T13:11:50.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.473+0000 7f72bca00640 1 -- 172.21.15.19:0/3158182591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f729c004800 msgr2=0x7f72b81f8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:50.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.473+0000 7f72bca00640 1 --2- 172.21.15.19:0/3158182591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f729c004800 0x7f72b81f8660 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f72a800e7b0 tx=0x7f72a800ec70 comp rx=0 tx=0).stop 2024-03-28T13:11:50.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.473+0000 7f72bca00640 1 -- 172.21.15.19:0/3158182591 shutdown_connections 2024-03-28T13:11:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.473+0000 7f72bca00640 1 --2- 172.21.15.19:0/3158182591 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f729406c530 0x7f729406e9f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.473+0000 7f72bca00640 1 --2- 172.21.15.19:0/3158182591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f72b81f8000 0x7f72b8159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.473+0000 7f72bca00640 1 --2- 172.21.15.19:0/3158182591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f72b8133700 0x7f72b81f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.473+0000 7f72bca00640 1 --2- 172.21.15.19:0/3158182591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f729c004800 0x7f72b81f8660 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.473+0000 7f72bca00640 1 -- 172.21.15.19:0/3158182591 >> 172.21.15.19:0/3158182591 conn(0x7f72b80ccad0 msgr2=0x7f72b80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.473+0000 7f72bca00640 1 -- 172.21.15.19:0/3158182591 shutdown_connections 2024-03-28T13:11:50.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.473+0000 7f72bca00640 1 -- 172.21.15.19:0/3158182591 wait complete. 2024-03-28T13:11:50.491 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:11:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.704+0000 7f7d2fe00640 1 Processor -- start 2024-03-28T13:11:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.705+0000 7f7d2fe00640 1 -- start start 2024-03-28T13:11:50.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.706+0000 7f7d2fe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7d3015e530 0x7f7d300cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:50.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.706+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d300cf330 0x7f7d300d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:50.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.706+0000 7f7d2fe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d30162350 con 0x7f7d3006e1b0 2024-03-28T13:11:50.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.706+0000 7f7d2fe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7d30162ed0 con 0x7f7d3015e530 2024-03-28T13:11:50.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.706+0000 7f7d2fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7d30162630 con 0x7f7d300cf330 2024-03-28T13:11:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.707+0000 7f7d2f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d300cf330 0x7f7d300d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.707+0000 7f7d2f400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d300cf330 0x7f7d300d1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:40114/0 (socket says 172.21.15.19:40114) 2024-03-28T13:11:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.707+0000 7f7d2f400640 1 -- 172.21.15.19:0/1897324613 learned_addr learned my addr 172.21.15.19:0/1897324613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.707+0000 7f7d2ea00640 1 --2- 172.21.15.19:0/1897324613 >> v2:172.21.15.188:3300/0 conn(0x7f7d3015e530 0x7f7d300cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.708+0000 7f7d2ea00640 1 -- 172.21.15.19:0/1897324613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d300cf330 msgr2=0x7f7d300d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.708+0000 7f7d2ea00640 1 --2- 172.21.15.19:0/1897324613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d300cf330 0x7f7d300d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.708+0000 7f7d2ea00640 1 -- 172.21.15.19:0/1897324613 >> v1:172.21.15.188:6789/0 conn(0x7f7d3006e1b0 legacy=0x7f7d300cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.708+0000 7f7d2ea00640 1 -- 172.21.15.19:0/1897324613 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d30162910 con 0x7f7d3015e530 2024-03-28T13:11:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.708+0000 7f7d2f400640 1 --2- 172.21.15.19:0/1897324613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d300cf330 0x7f7d300d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:11:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.710+0000 7f7d2ea00640 1 --2- 172.21.15.19:0/1897324613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d3015e530 0x7f7d300cc720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7d200097a0 tx=0x7f7d2002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1596d8e46e97f759 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.710+0000 7f7d2d600640 1 -- 172.21.15.19:0/1897324613 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d200315a0 con 0x7f7d3015e530 2024-03-28T13:11:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.710+0000 7f7d2d600640 1 -- 172.21.15.19:0/1897324613 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d20037070 con 0x7f7d3015e530 2024-03-28T13:11:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.710+0000 7f7d2d600640 1 -- 172.21.15.19:0/1897324613 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d20031960 con 0x7f7d3015e530 2024-03-28T13:11:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.711+0000 7f7d2fe00640 1 -- 172.21.15.19:0/1897324613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d3015e530 msgr2=0x7f7d300cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.711+0000 7f7d2fe00640 1 --2- 172.21.15.19:0/1897324613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d3015e530 0x7f7d300cc720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7d200097a0 tx=0x7f7d2002b840 comp rx=0 tx=0).stop 2024-03-28T13:11:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.711+0000 7f7d2fe00640 1 -- 172.21.15.19:0/1897324613 shutdown_connections 2024-03-28T13:11:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.711+0000 7f7d2fe00640 1 --2- 172.21.15.19:0/1897324613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d300cf330 0x7f7d300d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.711+0000 7f7d2fe00640 1 --2- 172.21.15.19:0/1897324613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d3015e530 0x7f7d300cc720 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.711+0000 7f7d2fe00640 1 -- 172.21.15.19:0/1897324613 >> 172.21.15.19:0/1897324613 conn(0x7f7d30067280 msgr2=0x7f7d30067660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.711+0000 7f7d2fe00640 1 -- 172.21.15.19:0/1897324613 shutdown_connections 2024-03-28T13:11:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.711+0000 7f7d2fe00640 1 -- 172.21.15.19:0/1897324613 wait complete. 2024-03-28T13:11:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.713+0000 7f7d2fe00640 1 Processor -- start 2024-03-28T13:11:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.713+0000 7f7d2fe00640 1 -- start start 2024-03-28T13:11:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.714+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d3006e1b0 0x7f7d300b7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.714+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d300cf330 0x7f7d300b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.714+0000 7f7d2fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3015e530 0x7f7d301529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.714+0000 7f7d2fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7d300cf7c0 con 0x7f7d3015e530 2024-03-28T13:11:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.714+0000 7f7d2fe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7d30161fe0 con 0x7f7d3006e1b0 2024-03-28T13:11:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.714+0000 7f7d2fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7d300cf940 con 0x7f7d300cf330 2024-03-28T13:11:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.714+0000 7f7d2f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3015e530 0x7f7d301529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.714+0000 7f7d2f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3015e530 0x7f7d301529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45418/0 (socket says 172.21.15.19:45418) 2024-03-28T13:11:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.714+0000 7f7d2f400640 1 -- 172.21.15.19:0/2582935747 learned_addr learned my addr 172.21.15.19:0/2582935747 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.714+0000 7f7d2ea00640 1 --2- 172.21.15.19:0/2582935747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d3006e1b0 0x7f7d300b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.714+0000 7f7d2e000640 1 --2- 172.21.15.19:0/2582935747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d300cf330 0x7f7d300b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.715+0000 7f7d2f400640 1 -- 172.21.15.19:0/2582935747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d300cf330 msgr2=0x7f7d300b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.715+0000 7f7d2f400640 1 --2- 172.21.15.19:0/2582935747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d300cf330 0x7f7d300b54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.715+0000 7f7d2f400640 1 -- 172.21.15.19:0/2582935747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d3006e1b0 msgr2=0x7f7d300b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.715+0000 7f7d2f400640 1 --2- 172.21.15.19:0/2582935747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d3006e1b0 0x7f7d300b7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.715+0000 7f7d2f400640 1 -- 172.21.15.19:0/2582935747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d300b7a80 con 0x7f7d3015e530 2024-03-28T13:11:50.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.715+0000 7f7d2f400640 1 --2- 172.21.15.19:0/2582935747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3015e530 0x7f7d301529e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800efc0 tx=0x7f7d2800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:50.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.715+0000 7f7d26a00640 1 -- 172.21.15.19:0/2582935747 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2800cbe0 con 0x7f7d3015e530 2024-03-28T13:11:50.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.715+0000 7f7d26a00640 1 -- 172.21.15.19:0/2582935747 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d2800cd80 con 0x7f7d3015e530 2024-03-28T13:11:50.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.715+0000 7f7d26a00640 1 -- 172.21.15.19:0/2582935747 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d28003bb0 con 0x7f7d3015e530 2024-03-28T13:11:50.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.715+0000 7f7d2fe00640 1 -- 172.21.15.19:0/2582935747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d300b4870 con 0x7f7d3015e530 2024-03-28T13:11:50.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.716+0000 7f7d2fe00640 1 -- 172.21.15.19:0/2582935747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d300c93f0 con 0x7f7d3015e530 2024-03-28T13:11:50.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.717+0000 7f7d2fe00640 1 -- 172.21.15.19:0/2582935747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d30075940 con 0x7f7d3015e530 2024-03-28T13:11:50.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.717+0000 7f7d26a00640 1 -- 172.21.15.19:0/2582935747 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7d2801b030 con 0x7f7d3015e530 2024-03-28T13:11:50.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.718+0000 7f7d26a00640 1 --2- 172.21.15.19:0/2582935747 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7d0c06c560 0x7f7d0c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:50.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.718+0000 7f7d26a00640 1 -- 172.21.15.19:0/2582935747 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f7d2808ba70 con 0x7f7d3015e530 2024-03-28T13:11:50.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.719+0000 7f7d2ea00640 1 --2- 172.21.15.19:0/2582935747 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7d0c06c560 0x7f7d0c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:50.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.720+0000 7f7d2ea00640 1 --2- 172.21.15.19:0/2582935747 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7d0c06c560 0x7f7d0c06ea20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7d20036000 tx=0x7f7d2003d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:50.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.723+0000 7f7d26a00640 1 -- 172.21.15.19:0/2582935747 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7d28022020 con 0x7f7d3015e530 2024-03-28T13:11:50.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.947+0000 7f7d2fe00640 1 -- 172.21.15.19:0/2582935747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d30070280 con 0x7f7d3015e530 2024-03-28T13:11:50.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.950+0000 7f7d26a00640 1 -- 172.21.15.19:0/2582935747 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16609 (secure 0 0 0) 0x7f7d28010040 con 0x7f7d3015e530 2024-03-28T13:11:50.950 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:50.951 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":49,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:11:47.888435+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:11:44.316258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-28T13:11:45.360154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:11:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.953+0000 7f7d2fe00640 1 -- 172.21.15.19:0/2582935747 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7d0c06c560 msgr2=0x7f7d0c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.953+0000 7f7d2fe00640 1 --2- 172.21.15.19:0/2582935747 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7d0c06c560 0x7f7d0c06ea20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7d20036000 tx=0x7f7d2003d040 comp rx=0 tx=0).stop 2024-03-28T13:11:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.953+0000 7f7d2fe00640 1 -- 172.21.15.19:0/2582935747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3015e530 msgr2=0x7f7d301529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.953+0000 7f7d2fe00640 1 --2- 172.21.15.19:0/2582935747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3015e530 0x7f7d301529e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800efc0 tx=0x7f7d2800c490 comp rx=0 tx=0).stop 2024-03-28T13:11:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.953+0000 7f7d2fe00640 1 -- 172.21.15.19:0/2582935747 shutdown_connections 2024-03-28T13:11:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.953+0000 7f7d2fe00640 1 --2- 172.21.15.19:0/2582935747 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7d0c06c560 0x7f7d0c06ea20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.953+0000 7f7d2fe00640 1 --2- 172.21.15.19:0/2582935747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d3015e530 0x7f7d301529e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.953+0000 7f7d2fe00640 1 --2- 172.21.15.19:0/2582935747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d300cf330 0x7f7d300b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.953+0000 7f7d2fe00640 1 --2- 172.21.15.19:0/2582935747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d3006e1b0 0x7f7d300b7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:50.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.953+0000 7f7d2fe00640 1 -- 172.21.15.19:0/2582935747 >> 172.21.15.19:0/2582935747 conn(0x7f7d30067280 msgr2=0x7f7d300b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:50.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.953+0000 7f7d2fe00640 1 -- 172.21.15.19:0/2582935747 shutdown_connections 2024-03-28T13:11:50.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:50.953+0000 7f7d2fe00640 1 -- 172.21.15.19:0/2582935747 wait complete. 2024-03-28T13:11:50.980 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-28T13:11:50.980 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:11:51.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.201+0000 7f4030e00640 1 Processor -- start 2024-03-28T13:11:51.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.201+0000 7f4030e00640 1 -- start start 2024-03-28T13:11:51.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.203+0000 7f4030e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402c0d2310 0x7f402c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:51.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.203+0000 7f4030e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f402c0cf530 0x7f402c0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:51.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.203+0000 7f4030e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f402c15e880 con 0x7f402c15e4a0 2024-03-28T13:11:51.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.203+0000 7f4030e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f402c0d34e0 con 0x7f402c0cf530 2024-03-28T13:11:51.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.203+0000 7f4030e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f402c0d28d0 con 0x7f402c0d2310 2024-03-28T13:11:51.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.204+0000 7f402aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402c0d2310 0x7f402c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:51.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.204+0000 7f402aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402c0d2310 0x7f402c0ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:40124/0 (socket says 172.21.15.19:40124) 2024-03-28T13:11:51.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.204+0000 7f402aa00640 1 -- 172.21.15.19:0/283177031 learned_addr learned my addr 172.21.15.19:0/283177031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:51.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.204+0000 7f402be00640 1 --2- 172.21.15.19:0/283177031 >> v2:172.21.15.188:3300/0 conn(0x7f402c0cf530 0x7f402c0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:51.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.205+0000 7f402be00640 1 -- 172.21.15.19:0/283177031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402c0d2310 msgr2=0x7f402c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:51.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.205+0000 7f402be00640 1 --2- 172.21.15.19:0/283177031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402c0d2310 0x7f402c0ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:51.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.205+0000 7f402be00640 1 -- 172.21.15.19:0/283177031 >> v1:172.21.15.188:6789/0 conn(0x7f402c15e4a0 legacy=0x7f402c0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:51.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.205+0000 7f402be00640 1 -- 172.21.15.19:0/283177031 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c0d2bb0 con 0x7f402c0cf530 2024-03-28T13:11:51.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.206+0000 7f402be00640 1 --2- 172.21.15.19:0/283177031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402c0cf530 0x7f402c0b73e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4024009d90 tx=0x7f402402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ab95604709984a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:51.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.207+0000 7f402a000640 1 -- 172.21.15.19:0/283177031 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024039070 con 0x7f402c0cf530 2024-03-28T13:11:51.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.207+0000 7f402a000640 1 -- 172.21.15.19:0/283177031 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f402402cc50 con 0x7f402c0cf530 2024-03-28T13:11:51.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.207+0000 7f402a000640 1 -- 172.21.15.19:0/283177031 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024034440 con 0x7f402c0cf530 2024-03-28T13:11:51.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.208+0000 7f4030e00640 1 -- 172.21.15.19:0/283177031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402c0cf530 msgr2=0x7f402c0b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:51.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.208+0000 7f4030e00640 1 --2- 172.21.15.19:0/283177031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402c0cf530 0x7f402c0b73e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4024009d90 tx=0x7f402402bea0 comp rx=0 tx=0).stop 2024-03-28T13:11:51.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.208+0000 7f4030e00640 1 -- 172.21.15.19:0/283177031 shutdown_connections 2024-03-28T13:11:51.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.208+0000 7f4030e00640 1 --2- 172.21.15.19:0/283177031 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402c0cf530 0x7f402c0b73e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:51.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.208+0000 7f4030e00640 1 --2- 172.21.15.19:0/283177031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402c0d2310 0x7f402c0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:51.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.208+0000 7f4030e00640 1 -- 172.21.15.19:0/283177031 >> 172.21.15.19:0/283177031 conn(0x7f402c0676d0 msgr2=0x7f402c067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:51.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.208+0000 7f4030e00640 1 -- 172.21.15.19:0/283177031 shutdown_connections 2024-03-28T13:11:51.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.208+0000 7f4030e00640 1 -- 172.21.15.19:0/283177031 wait complete. 2024-03-28T13:11:51.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.211+0000 7f4030e00640 1 Processor -- start 2024-03-28T13:11:51.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.211+0000 7f4030e00640 1 -- start start 2024-03-28T13:11:51.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.213+0000 7f4030e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402c0cf530 0x7f402c1f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.213+0000 7f4030e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f402c0d2310 0x7f402c0c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.213+0000 7f4030e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402c15e4a0 0x7f402c160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.213+0000 7f4030e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f402c0d3970 con 0x7f402c0d2310 2024-03-28T13:11:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.213+0000 7f4030e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f402c0d3170 con 0x7f402c0cf530 2024-03-28T13:11:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.213+0000 7f4030e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f402c0b7920 con 0x7f402c15e4a0 2024-03-28T13:11:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.213+0000 7f402aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f402c0d2310 0x7f402c0c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.213+0000 7f402aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f402c0d2310 0x7f402c0c97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45426/0 (socket says 172.21.15.19:45426) 2024-03-28T13:11:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.213+0000 7f402aa00640 1 -- 172.21.15.19:0/4234156934 learned_addr learned my addr 172.21.15.19:0/4234156934 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.213+0000 7f402be00640 1 --2- 172.21.15.19:0/4234156934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402c15e4a0 0x7f402c160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.213+0000 7f402b400640 1 --2- 172.21.15.19:0/4234156934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402c0cf530 0x7f402c1f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.213+0000 7f402aa00640 1 -- 172.21.15.19:0/4234156934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402c15e4a0 msgr2=0x7f402c160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.213+0000 7f402aa00640 1 --2- 172.21.15.19:0/4234156934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402c15e4a0 0x7f402c160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.213+0000 7f402aa00640 1 -- 172.21.15.19:0/4234156934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402c0cf530 msgr2=0x7f402c1f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.214+0000 7f402aa00640 1 --2- 172.21.15.19:0/4234156934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402c0cf530 0x7f402c1f4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.214+0000 7f402aa00640 1 -- 172.21.15.19:0/4234156934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c156510 con 0x7f402c0d2310 2024-03-28T13:11:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.214+0000 7f402b400640 1 --2- 172.21.15.19:0/4234156934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402c0cf530 0x7f402c1f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.214+0000 7f402aa00640 1 --2- 172.21.15.19:0/4234156934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f402c0d2310 0x7f402c0c97c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f401800e8b0 tx=0x7f401800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.214+0000 7f4023400640 1 -- 172.21.15.19:0/4234156934 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018017070 con 0x7f402c0d2310 2024-03-28T13:11:51.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.214+0000 7f4023400640 1 -- 172.21.15.19:0/4234156934 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f401800c940 con 0x7f402c0d2310 2024-03-28T13:11:51.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.214+0000 7f4023400640 1 -- 172.21.15.19:0/4234156934 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401800cd10 con 0x7f402c0d2310 2024-03-28T13:11:51.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.214+0000 7f4030e00640 1 -- 172.21.15.19:0/4234156934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402c1567f0 con 0x7f402c0d2310 2024-03-28T13:11:51.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.214+0000 7f4030e00640 1 -- 172.21.15.19:0/4234156934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402c156db0 con 0x7f402c0d2310 2024-03-28T13:11:51.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.216+0000 7f4030e00640 1 -- 172.21.15.19:0/4234156934 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f402c075940 con 0x7f402c0d2310 2024-03-28T13:11:51.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.217+0000 7f4023400640 1 -- 172.21.15.19:0/4234156934 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f401801a070 con 0x7f402c0d2310 2024-03-28T13:11:51.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.217+0000 7f4023400640 1 --2- 172.21.15.19:0/4234156934 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f400c06c530 0x7f400c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:51.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.218+0000 7f4023400640 1 -- 172.21.15.19:0/4234156934 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f401808ba30 con 0x7f402c0d2310 2024-03-28T13:11:51.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.218+0000 7f402b400640 1 --2- 172.21.15.19:0/4234156934 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f400c06c530 0x7f400c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.219+0000 7f402b400640 1 --2- 172.21.15.19:0/4234156934 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f400c06c530 0x7f400c06e9f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f401c000f60 tx=0x7f401c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:51.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.227+0000 7f4023400640 1 -- 172.21.15.19:0/4234156934 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f401805a040 con 0x7f402c0d2310 2024-03-28T13:11:51.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.427+0000 7f4030e00640 1 -- 172.21.15.19:0/4234156934 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f402c0d4110 con 0x7f400c06c530 2024-03-28T13:11:51.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.447+0000 7f4023400640 1 -- 172.21.15.19:0/4234156934 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246112 (secure 0 0 0) 0x7f402c0d4110 con 0x7f400c06c530 2024-03-28T13:11:51.452 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:51.455 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:11:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.453+0000 7f4030e00640 1 -- 172.21.15.19:0/4234156934 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f400c06c530 msgr2=0x7f400c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.453+0000 7f4030e00640 1 --2- 172.21.15.19:0/4234156934 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f400c06c530 0x7f400c06e9f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f401c000f60 tx=0x7f401c009040 comp rx=0 tx=0).stop 2024-03-28T13:11:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.453+0000 7f4030e00640 1 -- 172.21.15.19:0/4234156934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f402c0d2310 msgr2=0x7f402c0c97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.453+0000 7f4030e00640 1 --2- 172.21.15.19:0/4234156934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f402c0d2310 0x7f402c0c97c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f401800e8b0 tx=0x7f401800ed70 comp rx=0 tx=0).stop 2024-03-28T13:11:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.453+0000 7f4030e00640 1 -- 172.21.15.19:0/4234156934 shutdown_connections 2024-03-28T13:11:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.453+0000 7f4030e00640 1 --2- 172.21.15.19:0/4234156934 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f400c06c530 0x7f400c06e9f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:51.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.453+0000 7f4030e00640 1 --2- 172.21.15.19:0/4234156934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402c15e4a0 0x7f402c160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.453+0000 7f4030e00640 1 --2- 172.21.15.19:0/4234156934 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f402c0d2310 0x7f402c0c97c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.453+0000 7f4030e00640 1 --2- 172.21.15.19:0/4234156934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f402c0cf530 0x7f402c1f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.453+0000 7f4030e00640 1 -- 172.21.15.19:0/4234156934 >> 172.21.15.19:0/4234156934 conn(0x7f402c0676d0 msgr2=0x7f402c0cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.454+0000 7f4030e00640 1 -- 172.21.15.19:0/4234156934 shutdown_connections 2024-03-28T13:11:51.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.454+0000 7f4030e00640 1 -- 172.21.15.19:0/4234156934 wait complete. 2024-03-28T13:11:51.473 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-03-28T13:11:50.027753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592618,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":190,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225728,"kb_used_data":5384,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748992,"statfs":{"total":773094113280,"available":772862967808,"internally_reserved":0,"allocated":5513216,"data_stored":2910480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":225496628},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2250,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.155732"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899850+0000","last_change":"2024-03-28T13:11:46.935132+0000","last_active":"2024-03-28T13:11:47.899850+0000","last_peered":"2024-03-28T13:11:47.899850+0000","last_clean":"2024-03-28T13:11:47.899850+0000","last_became_active":"2024-03-28T13:11:46.934845+0000","last_became_peered":"2024-03-28T13:11:46.934845+0000","last_unstale":"2024-03-28T13:11:47.899850+0000","last_undegraded":"2024-03-28T13:11:47.899850+0000","last_fullsized":"2024-03-28T13:11:47.899850+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:51:37.394011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.310520+0000","last_change":"2024-03-28T13:11:46.941771+0000","last_active":"2024-03-28T13:11:48.310520+0000","last_peered":"2024-03-28T13:11:48.310520+0000","last_clean":"2024-03-28T13:11:48.310520+0000","last_became_active":"2024-03-28T13:11:46.940315+0000","last_became_peered":"2024-03-28T13:11:46.940315+0000","last_unstale":"2024-03-28T13:11:48.310520+0000","last_undegraded":"2024-03-28T13:11:48.310520+0000","last_fullsized":"2024-03-28T13:11:48.310520+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:03:02.471429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899660+0000","last_change":"2024-03-28T13:11:46.935947+0000","last_active":"2024-03-28T13:11:47.899660+0000","last_peered":"2024-03-28T13:11:47.899660+0000","last_clean":"2024-03-28T13:11:47.899660+0000","last_became_active":"2024-03-28T13:11:46.935483+0000","last_became_peered":"2024-03-28T13:11:46.935483+0000","last_unstale":"2024-03-28T13:11:47.899660+0000","last_undegraded":"2024-03-28T13:11:47.899660+0000","last_fullsized":"2024-03-28T13:11:47.899660+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:22:58.039744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311564+0000","last_change":"2024-03-28T13:11:46.943250+0000","last_active":"2024-03-28T13:11:48.311564+0000","last_peered":"2024-03-28T13:11:48.311564+0000","last_clean":"2024-03-28T13:11:48.311564+0000","last_became_active":"2024-03-28T13:11:46.942810+0000","last_became_peered":"2024-03-28T13:11:46.942810+0000","last_unstale":"2024-03-28T13:11:48.311564+0000","last_undegraded":"2024-03-28T13:11:48.311564+0000","last_fullsized":"2024-03-28T13:11:48.311564+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:49:47.662466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312572+0000","last_change":"2024-03-28T13:11:46.930353+0000","last_active":"2024-03-28T13:11:48.312572+0000","last_peered":"2024-03-28T13:11:48.312572+0000","last_clean":"2024-03-28T13:11:48.312572+0000","last_became_active":"2024-03-28T13:11:46.930052+0000","last_became_peered":"2024-03-28T13:11:46.930052+0000","last_unstale":"2024-03-28T13:11:48.312572+0000","last_undegraded":"2024-03-28T13:11:48.312572+0000","last_fullsized":"2024-03-28T13:11:48.312572+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:43.386902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900563+0000","last_change":"2024-03-28T13:11:46.929838+0000","last_active":"2024-03-28T13:11:47.900563+0000","last_peered":"2024-03-28T13:11:47.900563+0000","last_clean":"2024-03-28T13:11:47.900563+0000","last_became_active":"2024-03-28T13:11:46.928355+0000","last_became_peered":"2024-03-28T13:11:46.928355+0000","last_unstale":"2024-03-28T13:11:47.900563+0000","last_undegraded":"2024-03-28T13:11:47.900563+0000","last_fullsized":"2024-03-28T13:11:47.900563+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:13:37.250641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.310842+0000","last_change":"2024-03-28T13:11:46.941054+0000","last_active":"2024-03-28T13:11:48.310842+0000","last_peered":"2024-03-28T13:11:48.310842+0000","last_clean":"2024-03-28T13:11:48.310842+0000","last_became_active":"2024-03-28T13:11:46.940744+0000","last_became_peered":"2024-03-28T13:11:46.940744+0000","last_unstale":"2024-03-28T13:11:48.310842+0000","last_undegraded":"2024-03-28T13:11:48.310842+0000","last_fullsized":"2024-03-28T13:11:48.310842+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:09:38.289283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899554+0000","last_change":"2024-03-28T13:11:46.938966+0000","last_active":"2024-03-28T13:11:47.899554+0000","last_peered":"2024-03-28T13:11:47.899554+0000","last_clean":"2024-03-28T13:11:47.899554+0000","last_became_active":"2024-03-28T13:11:46.938618+0000","last_became_peered":"2024-03-28T13:11:46.938618+0000","last_unstale":"2024-03-28T13:11:47.899554+0000","last_undegraded":"2024-03-28T13:11:47.899554+0000","last_fullsized":"2024-03-28T13:11:47.899554+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:11.680247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900082+0000","last_change":"2024-03-28T13:11:46.926823+0000","last_active":"2024-03-28T13:11:47.900082+0000","last_peered":"2024-03-28T13:11:47.900082+0000","last_clean":"2024-03-28T13:11:47.900082+0000","last_became_active":"2024-03-28T13:11:46.926500+0000","last_became_peered":"2024-03-28T13:11:46.926500+0000","last_unstale":"2024-03-28T13:11:47.900082+0000","last_undegraded":"2024-03-28T13:11:47.900082+0000","last_fullsized":"2024-03-28T13:11:47.900082+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:43:15.464354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312278+0000","last_change":"2024-03-28T13:11:46.931175+0000","last_active":"2024-03-28T13:11:48.312278+0000","last_peered":"2024-03-28T13:11:48.312278+0000","last_clean":"2024-03-28T13:11:48.312278+0000","last_became_active":"2024-03-28T13:11:46.930846+0000","last_became_peered":"2024-03-28T13:11:46.930846+0000","last_unstale":"2024-03-28T13:11:48.312278+0000","last_undegraded":"2024-03-28T13:11:48.312278+0000","last_fullsized":"2024-03-28T13:11:48.312278+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:34:41.880929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900445+0000","last_change":"2024-03-28T13:11:46.930815+0000","last_active":"2024-03-28T13:11:47.900445+0000","last_peered":"2024-03-28T13:11:47.900445+0000","last_clean":"2024-03-28T13:11:47.900445+0000","last_became_active":"2024-03-28T13:11:46.930132+0000","last_became_peered":"2024-03-28T13:11:46.930132+0000","last_unstale":"2024-03-28T13:11:47.900445+0000","last_undegraded":"2024-03-28T13:11:47.900445+0000","last_fullsized":"2024-03-28T13:11:47.900445+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:14:03.582306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.925513+0000","last_change":"2024-03-28T13:11:46.916936+0000","last_active":"2024-03-28T13:11:46.925513+0000","last_peered":"2024-03-28T13:11:46.925513+0000","last_clean":"2024-03-28T13:11:46.925513+0000","last_became_active":"2024-03-28T13:11:46.916561+0000","last_became_peered":"2024-03-28T13:11:46.916561+0000","last_unstale":"2024-03-28T13:11:46.925513+0000","last_undegraded":"2024-03-28T13:11:46.925513+0000","last_fullsized":"2024-03-28T13:11:46.925513+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:45:18.443944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900136+0000","last_change":"2024-03-28T13:11:46.938100+0000","last_active":"2024-03-28T13:11:47.900136+0000","last_peered":"2024-03-28T13:11:47.900136+0000","last_clean":"2024-03-28T13:11:47.900136+0000","last_became_active":"2024-03-28T13:11:46.937802+0000","last_became_peered":"2024-03-28T13:11:46.937802+0000","last_unstale":"2024-03-28T13:11:47.900136+0000","last_undegraded":"2024-03-28T13:11:47.900136+0000","last_fullsized":"2024-03-28T13:11:47.900136+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:02:26.883141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.310298+0000","last_change":"2024-03-28T13:11:46.923320+0000","last_active":"2024-03-28T13:11:48.310298+0000","last_peered":"2024-03-28T13:11:48.310298+0000","last_clean":"2024-03-28T13:11:48.310298+0000","last_became_active":"2024-03-28T13:11:46.920809+0000","last_became_peered":"2024-03-28T13:11:46.920809+0000","last_unstale":"2024-03-28T13:11:48.310298+0000","last_undegraded":"2024-03-28T13:11:48.310298+0000","last_fullsized":"2024-03-28T13:11:48.310298+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:09:37.323636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900233+0000","last_change":"2024-03-28T13:11:46.935032+0000","last_active":"2024-03-28T13:11:47.900233+0000","last_peered":"2024-03-28T13:11:47.900233+0000","last_clean":"2024-03-28T13:11:47.900233+0000","last_became_active":"2024-03-28T13:11:46.931213+0000","last_became_peered":"2024-03-28T13:11:46.931213+0000","last_unstale":"2024-03-28T13:11:47.900233+0000","last_undegraded":"2024-03-28T13:11:47.900233+0000","last_fullsized":"2024-03-28T13:11:47.900233+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:15:07.394420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.939599+0000","last_change":"2024-03-28T13:11:46.937952+0000","last_active":"2024-03-28T13:11:46.939599+0000","last_peered":"2024-03-28T13:11:46.939599+0000","last_clean":"2024-03-28T13:11:46.939599+0000","last_became_active":"2024-03-28T13:11:46.937671+0000","last_became_peered":"2024-03-28T13:11:46.937671+0000","last_unstale":"2024-03-28T13:11:46.939599+0000","last_undegraded":"2024-03-28T13:11:46.939599+0000","last_fullsized":"2024-03-28T13:11:46.939599+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:41:11.684448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"48'4","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311840+0000","last_change":"2024-03-28T13:11:45.920678+0000","last_active":"2024-03-28T13:11:48.311840+0000","last_peered":"2024-03-28T13:11:48.311840+0000","last_clean":"2024-03-28T13:11:48.311840+0000","last_became_active":"2024-03-28T13:11:45.920319+0000","last_became_peered":"2024-03-28T13:11:45.920319+0000","last_unstale":"2024-03-28T13:11:48.311840+0000","last_undegraded":"2024-03-28T13:11:48.311840+0000","last_fullsized":"2024-03-28T13:11:48.311840+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:46:08.557688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-03-28T13:11:44.876373+0000","last_change":"2024-03-28T13:11:44.876373+0000","last_active":"2024-03-28T13:11:44.853516+0000","last_peered":"2024-03-28T13:11:44.853516+0000","last_clean":"2024-03-28T13:11:44.853516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:44.876373+0000","last_undegraded":"2024-03-28T13:11:44.876373+0000","last_fullsized":"2024-03-28T13:11:44.876373+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311121+0000","last_change":"2024-03-28T13:11:45.916445+0000","last_active":"2024-03-28T13:11:48.311121+0000","last_peered":"2024-03-28T13:11:48.311121+0000","last_clean":"2024-03-28T13:11:48.311121+0000","last_became_active":"2024-03-28T13:11:45.915977+0000","last_became_peered":"2024-03-28T13:11:45.915977+0000","last_unstale":"2024-03-28T13:11:48.311121+0000","last_undegraded":"2024-03-28T13:11:48.311121+0000","last_fullsized":"2024-03-28T13:11:48.311121+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:37:07.491590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900819+0000","last_change":"2024-03-28T13:11:46.938178+0000","last_active":"2024-03-28T13:11:47.900819+0000","last_peered":"2024-03-28T13:11:47.900819+0000","last_clean":"2024-03-28T13:11:47.900819+0000","last_became_active":"2024-03-28T13:11:46.937858+0000","last_became_peered":"2024-03-28T13:11:46.937858+0000","last_unstale":"2024-03-28T13:11:47.900819+0000","last_undegraded":"2024-03-28T13:11:47.900819+0000","last_fullsized":"2024-03-28T13:11:47.900819+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:34:41.880929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-03-28T13:11:44.875791+0000","last_change":"2024-03-28T13:11:44.875791+0000","last_active":"2024-03-28T13:11:44.853516+0000","last_peered":"2024-03-28T13:11:44.853516+0000","last_clean":"2024-03-28T13:11:44.853516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:44.875791+0000","last_undegraded":"2024-03-28T13:11:44.875791+0000","last_fullsized":"2024-03-28T13:11:44.875791+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311556+0000","last_change":"2024-03-28T13:11:45.917510+0000","last_active":"2024-03-28T13:11:48.311556+0000","last_peered":"2024-03-28T13:11:48.311556+0000","last_clean":"2024-03-28T13:11:48.311556+0000","last_became_active":"2024-03-28T13:11:45.916797+0000","last_became_peered":"2024-03-28T13:11:45.916797+0000","last_unstale":"2024-03-28T13:11:48.311556+0000","last_undegraded":"2024-03-28T13:11:48.311556+0000","last_fullsized":"2024-03-28T13:11:48.311556+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:09:50.989777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311012+0000","last_change":"2024-03-28T13:11:46.936730+0000","last_active":"2024-03-28T13:11:48.311012+0000","last_peered":"2024-03-28T13:11:48.311012+0000","last_clean":"2024-03-28T13:11:48.311012+0000","last_became_active":"2024-03-28T13:11:46.936463+0000","last_became_peered":"2024-03-28T13:11:46.936463+0000","last_unstale":"2024-03-28T13:11:48.311012+0000","last_undegraded":"2024-03-28T13:11:48.311012+0000","last_fullsized":"2024-03-28T13:11:48.311012+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:14:03.582306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-03-28T13:11:44.876586+0000","last_change":"2024-03-28T13:11:44.876586+0000","last_active":"2024-03-28T13:11:44.853516+0000","last_peered":"2024-03-28T13:11:44.853516+0000","last_clean":"2024-03-28T13:11:44.853516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:44.876586+0000","last_undegraded":"2024-03-28T13:11:44.876586+0000","last_fullsized":"2024-03-28T13:11:44.876586+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312729+0000","last_change":"2024-03-28T13:11:46.945709+0000","last_active":"2024-03-28T13:11:48.312729+0000","last_peered":"2024-03-28T13:11:48.312729+0000","last_clean":"2024-03-28T13:11:48.312729+0000","last_became_active":"2024-03-28T13:11:46.945294+0000","last_became_peered":"2024-03-28T13:11:46.945294+0000","last_unstale":"2024-03-28T13:11:48.312729+0000","last_undegraded":"2024-03-28T13:11:48.312729+0000","last_fullsized":"2024-03-28T13:11:48.312729+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:33:07.264658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-03-28T13:11:44.882987+0000","last_change":"2024-03-28T13:11:44.882987+0000","last_active":"2024-03-28T13:11:44.853516+0000","last_peered":"2024-03-28T13:11:44.853516+0000","last_clean":"2024-03-28T13:11:44.853516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:44.882987+0000","last_undegraded":"2024-03-28T13:11:44.882987+0000","last_fullsized":"2024-03-28T13:11:44.882987+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.310895+0000","last_change":"2024-03-28T13:11:45.919677+0000","last_active":"2024-03-28T13:11:48.310895+0000","last_peered":"2024-03-28T13:11:48.310895+0000","last_clean":"2024-03-28T13:11:48.310895+0000","last_became_active":"2024-03-28T13:11:45.918752+0000","last_became_peered":"2024-03-28T13:11:45.918752+0000","last_unstale":"2024-03-28T13:11:48.310895+0000","last_undegraded":"2024-03-28T13:11:48.310895+0000","last_fullsized":"2024-03-28T13:11:48.310895+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:46:35.043997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-03-28T13:11:44.882442+0000","last_change":"2024-03-28T13:11:44.882442+0000","last_active":"2024-03-28T13:11:44.853516+0000","last_peered":"2024-03-28T13:11:44.853516+0000","last_clean":"2024-03-28T13:11:44.853516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:44.882442+0000","last_undegraded":"2024-03-28T13:11:44.882442+0000","last_fullsized":"2024-03-28T13:11:44.882442+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312815+0000","last_change":"2024-03-28T13:11:45.919230+0000","last_active":"2024-03-28T13:11:48.312815+0000","last_peered":"2024-03-28T13:11:48.312815+0000","last_clean":"2024-03-28T13:11:48.312815+0000","last_became_active":"2024-03-28T13:11:45.918843+0000","last_became_peered":"2024-03-28T13:11:45.918843+0000","last_unstale":"2024-03-28T13:11:48.312815+0000","last_undegraded":"2024-03-28T13:11:48.312815+0000","last_fullsized":"2024-03-28T13:11:48.312815+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:10.665269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899900+0000","last_change":"2024-03-28T13:11:46.938713+0000","last_active":"2024-03-28T13:11:47.899900+0000","last_peered":"2024-03-28T13:11:47.899900+0000","last_clean":"2024-03-28T13:11:47.899900+0000","last_became_active":"2024-03-28T13:11:46.937364+0000","last_became_peered":"2024-03-28T13:11:46.937364+0000","last_unstale":"2024-03-28T13:11:47.899900+0000","last_undegraded":"2024-03-28T13:11:47.899900+0000","last_fullsized":"2024-03-28T13:11:47.899900+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:16:23.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-03-28T13:11:44.880505+0000","last_change":"2024-03-28T13:11:44.880505+0000","last_active":"2024-03-28T13:11:44.853516+0000","last_peered":"2024-03-28T13:11:44.853516+0000","last_clean":"2024-03-28T13:11:44.853516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:44.880505+0000","last_undegraded":"2024-03-28T13:11:44.880505+0000","last_fullsized":"2024-03-28T13:11:44.880505+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"48'6","reported_seq":20,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312587+0000","last_change":"2024-03-28T13:11:45.916407+0000","last_active":"2024-03-28T13:11:48.312587+0000","last_peered":"2024-03-28T13:11:48.312587+0000","last_clean":"2024-03-28T13:11:48.312587+0000","last_became_active":"2024-03-28T13:11:45.916093+0000","last_became_peered":"2024-03-28T13:11:45.916093+0000","last_unstale":"2024-03-28T13:11:48.312587+0000","last_undegraded":"2024-03-28T13:11:48.312587+0000","last_fullsized":"2024-03-28T13:11:48.312587+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:58:42.454595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"48'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.939353+0000","last_change":"2024-03-28T13:11:45.923185+0000","last_active":"2024-03-28T13:11:46.939353+0000","last_peered":"2024-03-28T13:11:46.939353+0000","last_clean":"2024-03-28T13:11:46.939353+0000","last_became_active":"2024-03-28T13:11:45.922652+0000","last_became_peered":"2024-03-28T13:11:45.922652+0000","last_unstale":"2024-03-28T13:11:46.939353+0000","last_undegraded":"2024-03-28T13:11:46.939353+0000","last_fullsized":"2024-03-28T13:11:46.939353+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:42:27.769753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.921870+0000","last_change":"2024-03-28T13:11:46.911170+0000","last_active":"2024-03-28T13:11:46.921870+0000","last_peered":"2024-03-28T13:11:46.921870+0000","last_clean":"2024-03-28T13:11:46.921870+0000","last_became_active":"2024-03-28T13:11:46.910443+0000","last_became_peered":"2024-03-28T13:11:46.910443+0000","last_unstale":"2024-03-28T13:11:46.921870+0000","last_undegraded":"2024-03-28T13:11:46.921870+0000","last_fullsized":"2024-03-28T13:11:46.921870+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:14:18.385803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-03-28T13:11:44.883065+0000","last_change":"2024-03-28T13:11:44.883065+0000","last_active":"2024-03-28T13:11:44.853516+0000","last_peered":"2024-03-28T13:11:44.853516+0000","last_clean":"2024-03-28T13:11:44.853516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:44.883065+0000","last_undegraded":"2024-03-28T13:11:44.883065+0000","last_fullsized":"2024-03-28T13:11:44.883065+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.947527+0000","last_change":"2024-03-28T13:11:46.946271+0000","last_active":"2024-03-28T13:11:46.947527+0000","last_peered":"2024-03-28T13:11:46.947527+0000","last_clean":"2024-03-28T13:11:46.947527+0000","last_became_active":"2024-03-28T13:11:46.945925+0000","last_became_peered":"2024-03-28T13:11:46.945925+0000","last_unstale":"2024-03-28T13:11:46.947527+0000","last_undegraded":"2024-03-28T13:11:46.947527+0000","last_fullsized":"2024-03-28T13:11:46.947527+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:57:46.465180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311961+0000","last_change":"2024-03-28T13:11:45.916288+0000","last_active":"2024-03-28T13:11:48.311961+0000","last_peered":"2024-03-28T13:11:48.311961+0000","last_clean":"2024-03-28T13:11:48.311961+0000","last_became_active":"2024-03-28T13:11:45.915835+0000","last_became_peered":"2024-03-28T13:11:45.915835+0000","last_unstale":"2024-03-28T13:11:48.311961+0000","last_undegraded":"2024-03-28T13:11:48.311961+0000","last_fullsized":"2024-03-28T13:11:48.311961+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:10:10.237245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"48'8","reported_seq":22,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311190+0000","last_change":"2024-03-28T13:11:45.917197+0000","last_active":"2024-03-28T13:11:48.311190+0000","last_peered":"2024-03-28T13:11:48.311190+0000","last_clean":"2024-03-28T13:11:48.311190+0000","last_became_active":"2024-03-28T13:11:45.916838+0000","last_became_peered":"2024-03-28T13:11:45.916838+0000","last_unstale":"2024-03-28T13:11:48.311190+0000","last_undegraded":"2024-03-28T13:11:48.311190+0000","last_fullsized":"2024-03-28T13:11:48.311190+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:49:46.647488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.930219+0000","last_change":"2024-03-28T13:11:46.927028+0000","last_active":"2024-03-28T13:11:46.930219+0000","last_peered":"2024-03-28T13:11:46.930219+0000","last_clean":"2024-03-28T13:11:46.930219+0000","last_became_active":"2024-03-28T13:11:46.926743+0000","last_became_peered":"2024-03-28T13:11:46.926743+0000","last_unstale":"2024-03-28T13:11:46.930219+0000","last_undegraded":"2024-03-28T13:11:46.930219+0000","last_fullsized":"2024-03-28T13:11:46.930219+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:28:34.523887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900059+0000","last_change":"2024-03-28T13:11:45.914263+0000","last_active":"2024-03-28T13:11:47.900059+0000","last_peered":"2024-03-28T13:11:47.900059+0000","last_clean":"2024-03-28T13:11:47.900059+0000","last_became_active":"2024-03-28T13:11:45.913970+0000","last_became_peered":"2024-03-28T13:11:45.913970+0000","last_unstale":"2024-03-28T13:11:47.900059+0000","last_undegraded":"2024-03-28T13:11:47.900059+0000","last_fullsized":"2024-03-28T13:11:47.900059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:09:50.989777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.939356+0000","last_change":"2024-03-28T13:11:46.936470+0000","last_active":"2024-03-28T13:11:46.939356+0000","last_peered":"2024-03-28T13:11:46.939356+0000","last_clean":"2024-03-28T13:11:46.939356+0000","last_became_active":"2024-03-28T13:11:46.936152+0000","last_became_peered":"2024-03-28T13:11:46.936152+0000","last_unstale":"2024-03-28T13:11:46.939356+0000","last_undegraded":"2024-03-28T13:11:46.939356+0000","last_fullsized":"2024-03-28T13:11:46.939356+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:04.460078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-03-28T13:11:44.882973+0000","last_change":"2024-03-28T13:11:44.882973+0000","last_active":"2024-03-28T13:11:44.853516+0000","last_peered":"2024-03-28T13:11:44.853516+0000","last_clean":"2024-03-28T13:11:44.853516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:44.882973+0000","last_undegraded":"2024-03-28T13:11:44.882973+0000","last_fullsized":"2024-03-28T13:11:44.882973+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-03-28T13:11:44.881901+0000","last_change":"2024-03-28T13:11:44.881901+0000","last_active":"2024-03-28T13:11:44.853516+0000","last_peered":"2024-03-28T13:11:44.853516+0000","last_clean":"2024-03-28T13:11:44.853516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:44.881901+0000","last_undegraded":"2024-03-28T13:11:44.881901+0000","last_fullsized":"2024-03-28T13:11:44.881901+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.310519+0000","last_change":"2024-03-28T13:11:45.915563+0000","last_active":"2024-03-28T13:11:48.310519+0000","last_peered":"2024-03-28T13:11:48.310519+0000","last_clean":"2024-03-28T13:11:48.310519+0000","last_became_active":"2024-03-28T13:11:45.915232+0000","last_became_peered":"2024-03-28T13:11:45.915232+0000","last_unstale":"2024-03-28T13:11:48.310519+0000","last_undegraded":"2024-03-28T13:11:48.310519+0000","last_fullsized":"2024-03-28T13:11:48.310519+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:39:19.230135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311251+0000","last_change":"2024-03-28T13:11:46.933297+0000","last_active":"2024-03-28T13:11:48.311251+0000","last_peered":"2024-03-28T13:11:48.311251+0000","last_clean":"2024-03-28T13:11:48.311251+0000","last_became_active":"2024-03-28T13:11:46.933007+0000","last_became_peered":"2024-03-28T13:11:46.933007+0000","last_unstale":"2024-03-28T13:11:48.311251+0000","last_undegraded":"2024-03-28T13:11:48.311251+0000","last_fullsized":"2024-03-28T13:11:48.311251+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:28:12.201099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312096+0000","last_change":"2024-03-28T13:11:45.923927+0000","last_active":"2024-03-28T13:11:48.312096+0000","last_peered":"2024-03-28T13:11:48.312096+0000","last_clean":"2024-03-28T13:11:48.312096+0000","last_became_active":"2024-03-28T13:11:45.923513+0000","last_became_peered":"2024-03-28T13:11:45.923513+0000","last_unstale":"2024-03-28T13:11:48.312096+0000","last_undegraded":"2024-03-28T13:11:48.312096+0000","last_fullsized":"2024-03-28T13:11:48.312096+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:09:37.274305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311266+0000","last_change":"2024-03-28T13:11:46.920550+0000","last_active":"2024-03-28T13:11:48.311266+0000","last_peered":"2024-03-28T13:11:48.311266+0000","last_clean":"2024-03-28T13:11:48.311266+0000","last_became_active":"2024-03-28T13:11:46.918773+0000","last_became_peered":"2024-03-28T13:11:46.918773+0000","last_unstale":"2024-03-28T13:11:48.311266+0000","last_undegraded":"2024-03-28T13:11:48.311266+0000","last_fullsized":"2024-03-28T13:11:48.311266+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:58:43.469573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-03-28T13:11:44.881650+0000","last_change":"2024-03-28T13:11:44.881650+0000","last_active":"2024-03-28T13:11:44.853516+0000","last_peered":"2024-03-28T13:11:44.853516+0000","last_clean":"2024-03-28T13:11:44.853516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:44.881650+0000","last_undegraded":"2024-03-28T13:11:44.881650+0000","last_fullsized":"2024-03-28T13:11:44.881650+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900067+0000","last_change":"2024-03-28T13:11:46.935009+0000","last_active":"2024-03-28T13:11:47.900067+0000","last_peered":"2024-03-28T13:11:47.900067+0000","last_clean":"2024-03-28T13:11:47.900067+0000","last_became_active":"2024-03-28T13:11:46.930504+0000","last_became_peered":"2024-03-28T13:11:46.930504+0000","last_unstale":"2024-03-28T13:11:47.900067+0000","last_undegraded":"2024-03-28T13:11:47.900067+0000","last_fullsized":"2024-03-28T13:11:47.900067+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:23:39.500731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311557+0000","last_change":"2024-03-28T13:11:45.923921+0000","last_active":"2024-03-28T13:11:48.311557+0000","last_peered":"2024-03-28T13:11:48.311557+0000","last_clean":"2024-03-28T13:11:48.311557+0000","last_became_active":"2024-03-28T13:11:45.923492+0000","last_became_peered":"2024-03-28T13:11:45.923492+0000","last_unstale":"2024-03-28T13:11:48.311557+0000","last_undegraded":"2024-03-28T13:11:48.311557+0000","last_fullsized":"2024-03-28T13:11:48.311557+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:03:01.456451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899417+0000","last_change":"2024-03-28T13:11:46.936793+0000","last_active":"2024-03-28T13:11:47.899417+0000","last_peered":"2024-03-28T13:11:47.899417+0000","last_clean":"2024-03-28T13:11:47.899417+0000","last_became_active":"2024-03-28T13:11:46.936578+0000","last_became_peered":"2024-03-28T13:11:46.936578+0000","last_unstale":"2024-03-28T13:11:47.899417+0000","last_undegraded":"2024-03-28T13:11:47.899417+0000","last_fullsized":"2024-03-28T13:11:47.899417+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:46:36.058976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-03-28T13:11:44.875851+0000","last_change":"2024-03-28T13:11:44.875851+0000","last_active":"2024-03-28T13:11:44.853516+0000","last_peered":"2024-03-28T13:11:44.853516+0000","last_clean":"2024-03-28T13:11:44.853516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:44.875851+0000","last_undegraded":"2024-03-28T13:11:44.875851+0000","last_fullsized":"2024-03-28T13:11:44.875851+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"49'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.903593+0000","last_change":"2024-03-28T13:11:45.914641+0000","last_active":"2024-03-28T13:11:48.903593+0000","last_peered":"2024-03-28T13:11:48.903593+0000","last_clean":"2024-03-28T13:11:48.903593+0000","last_became_active":"2024-03-28T13:11:45.912854+0000","last_became_peered":"2024-03-28T13:11:45.912854+0000","last_unstale":"2024-03-28T13:11:48.903593+0000","last_undegraded":"2024-03-28T13:11:48.903593+0000","last_fullsized":"2024-03-28T13:11:48.903593+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:10:10.237245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312572+0000","last_change":"2024-03-28T13:11:46.934436+0000","last_active":"2024-03-28T13:11:48.312572+0000","last_peered":"2024-03-28T13:11:48.312572+0000","last_clean":"2024-03-28T13:11:48.312572+0000","last_became_active":"2024-03-28T13:11:46.934112+0000","last_became_peered":"2024-03-28T13:11:46.934112+0000","last_unstale":"2024-03-28T13:11:48.312572+0000","last_undegraded":"2024-03-28T13:11:48.312572+0000","last_fullsized":"2024-03-28T13:11:48.312572+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:07.509367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-03-28T13:11:44.882692+0000","last_change":"2024-03-28T13:11:44.882692+0000","last_active":"2024-03-28T13:11:44.853516+0000","last_peered":"2024-03-28T13:11:44.853516+0000","last_clean":"2024-03-28T13:11:44.853516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:44.882692+0000","last_undegraded":"2024-03-28T13:11:44.882692+0000","last_fullsized":"2024-03-28T13:11:44.882692+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.896678+0000","last_change":"2024-03-28T13:11:45.925284+0000","last_active":"2024-03-28T13:11:46.896678+0000","last_peered":"2024-03-28T13:11:46.896678+0000","last_clean":"2024-03-28T13:11:46.896678+0000","last_became_active":"2024-03-28T13:11:45.924633+0000","last_became_peered":"2024-03-28T13:11:45.924633+0000","last_unstale":"2024-03-28T13:11:46.896678+0000","last_undegraded":"2024-03-28T13:11:46.896678+0000","last_fullsized":"2024-03-28T13:11:46.896678+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:07:22.061520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311437+0000","last_change":"2024-03-28T13:11:46.932551+0000","last_active":"2024-03-28T13:11:48.311437+0000","last_peered":"2024-03-28T13:11:48.311437+0000","last_clean":"2024-03-28T13:11:48.311437+0000","last_became_active":"2024-03-28T13:11:46.930144+0000","last_became_peered":"2024-03-28T13:11:46.930144+0000","last_unstale":"2024-03-28T13:11:48.311437+0000","last_undegraded":"2024-03-28T13:11:48.311437+0000","last_fullsized":"2024-03-28T13:11:48.311437+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:13:37.250641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-03-28T13:11:44.881737+0000","last_change":"2024-03-28T13:11:44.881737+0000","last_active":"2024-03-28T13:11:44.853516+0000","last_peered":"2024-03-28T13:11:44.853516+0000","last_clean":"2024-03-28T13:11:44.853516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:44.881737+0000","last_undegraded":"2024-03-28T13:11:44.881737+0000","last_fullsized":"2024-03-28T13:11:44.881737+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900467+0000","last_change":"2024-03-28T13:11:46.932761+0000","last_active":"2024-03-28T13:11:47.900467+0000","last_peered":"2024-03-28T13:11:47.900467+0000","last_clean":"2024-03-28T13:11:47.900467+0000","last_became_active":"2024-03-28T13:11:46.929710+0000","last_became_peered":"2024-03-28T13:11:46.929710+0000","last_unstale":"2024-03-28T13:11:47.900467+0000","last_undegraded":"2024-03-28T13:11:47.900467+0000","last_fullsized":"2024-03-28T13:11:47.900467+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:28:12.201099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-03-28T13:11:44.875794+0000","last_change":"2024-03-28T13:11:44.875794+0000","last_active":"2024-03-28T13:11:44.853516+0000","last_peered":"2024-03-28T13:11:44.853516+0000","last_clean":"2024-03-28T13:11:44.853516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:44.875794+0000","last_undegraded":"2024-03-28T13:11:44.875794+0000","last_fullsized":"2024-03-28T13:11:44.875794+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.310584+0000","last_change":"2024-03-28T13:11:45.917018+0000","last_active":"2024-03-28T13:11:48.310584+0000","last_peered":"2024-03-28T13:11:48.310584+0000","last_clean":"2024-03-28T13:11:48.310584+0000","last_became_active":"2024-03-28T13:11:45.916556+0000","last_became_peered":"2024-03-28T13:11:45.916556+0000","last_unstale":"2024-03-28T13:11:48.310584+0000","last_undegraded":"2024-03-28T13:11:48.310584+0000","last_fullsized":"2024-03-28T13:11:48.310584+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:04:53.337486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-28T13:11:45.868495+0000","last_change":"2024-03-28T13:11:45.868495+0000","last_active":"2024-03-28T13:11:45.868495+0000","last_peered":"2024-03-28T13:11:45.868495+0000","last_clean":"2024-03-28T13:11:45.868495+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:45.868495+0000","last_undegraded":"2024-03-28T13:11:45.868495+0000","last_fullsized":"2024-03-28T13:11:45.868495+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.897166+0000","last_change":"2024-03-28T13:11:45.920898+0000","last_active":"2024-03-28T13:11:46.897166+0000","last_peered":"2024-03-28T13:11:46.897166+0000","last_clean":"2024-03-28T13:11:46.897166+0000","last_became_active":"2024-03-28T13:11:45.920582+0000","last_became_peered":"2024-03-28T13:11:45.920582+0000","last_unstale":"2024-03-28T13:11:46.897166+0000","last_undegraded":"2024-03-28T13:11:46.897166+0000","last_fullsized":"2024-03-28T13:11:46.897166+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:53:04.504198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900230+0000","last_change":"2024-03-28T13:11:46.939335+0000","last_active":"2024-03-28T13:11:47.900230+0000","last_peered":"2024-03-28T13:11:47.900230+0000","last_clean":"2024-03-28T13:11:47.900230+0000","last_became_active":"2024-03-28T13:11:46.938778+0000","last_became_peered":"2024-03-28T13:11:46.938778+0000","last_unstale":"2024-03-28T13:11:47.900230+0000","last_undegraded":"2024-03-28T13:11:47.900230+0000","last_fullsized":"2024-03-28T13:11:47.900230+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:03:36.192099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2024-03-28T13:11:44.875952+0000","last_change":"2024-03-28T13:11:44.875952+0000","last_active":"2024-03-28T13:11:44.853516+0000","last_peered":"2024-03-28T13:11:44.853516+0000","last_clean":"2024-03-28T13:11:44.853516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:11:44.875952+0000","last_undegraded":"2024-03-28T13:11:44.875952+0000","last_fullsized":"2024-03-28T13:11:44.875952+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":212,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-03-28T13:11:44.885526+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:11:44.885526+0000","last_peered":"2024-03-28T13:11:44.885526+0000","last_clean":"2024-03-28T13:11:44.885526+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:11:44.885526+0000","last_undegraded":"2024-03-28T13:11:44.885526+0000","last_fullsized":"2024-03-28T13:11:44.885526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2250,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27,"ondisk_log_size":27,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476769,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":516096,"data_stored":205700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476767,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":205700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476767,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1110016,"data_stored":796068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476767,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27524,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":516096,"data_stored":205700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509471,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":561152,"data_stored":225183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1163264,"data_stored":821257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":226271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542176,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":557056,"data_stored":224601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:11:51.485 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:11:51.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.735+0000 7f6aa2c00640 1 Processor -- start 2024-03-28T13:11:51.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.736+0000 7f6aa2c00640 1 -- start start 2024-03-28T13:11:51.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.737+0000 7f6aa2c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c0cc0b0 0x7f6a9c06dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:51.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.737+0000 7f6aa2c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a9c15fc50 0x7f6a9c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:51.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.737+0000 7f6aa2c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a9c06e4f0 con 0x7f6a9c0cc490 2024-03-28T13:11:51.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.737+0000 7f6aa2c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f6a9c15c610 con 0x7f6a9c15fc50 2024-03-28T13:11:51.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.737+0000 7f6aa2c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6a9c0b3150 con 0x7f6a9c0cc0b0 2024-03-28T13:11:51.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.738+0000 7f6aa2200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6a9c0cc490 0x7f6a9c15ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36176/0 (socket says 172.21.15.19:36176) 2024-03-28T13:11:51.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.738+0000 7f6aa2200640 1 -- 172.21.15.19:0/2662567435 learned_addr learned my addr 172.21.15.19:0/2662567435 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.738+0000 7f6aa0e00640 1 --2- 172.21.15.19:0/2662567435 >> v2:172.21.15.188:3300/0 conn(0x7f6a9c15fc50 0x7f6a9c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.738+0000 7f6aa1800640 1 --2- 172.21.15.19:0/2662567435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c0cc0b0 0x7f6a9c06dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.739+0000 7f6a9be00640 1 -- 172.21.15.19:0/2662567435 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2714148206 0 0) 0x7f6a9c06e4f0 con 0x7f6a9c0cc490 2024-03-28T13:11:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.739+0000 7f6a9be00640 1 -- 172.21.15.19:0/2662567435 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a84003520 con 0x7f6a9c0cc490 2024-03-28T13:11:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.739+0000 7f6aa0e00640 1 -- 172.21.15.19:0/2662567435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c0cc0b0 msgr2=0x7f6a9c06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.739+0000 7f6aa0e00640 1 --2- 172.21.15.19:0/2662567435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c0cc0b0 0x7f6a9c06dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.739+0000 7f6aa0e00640 1 -- 172.21.15.19:0/2662567435 >> v1:172.21.15.19:6789/0 conn(0x7f6a9c0cc490 legacy=0x7f6a9c15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:51.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.739+0000 7f6a9be00640 1 -- 172.21.15.19:0/2662567435 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 504535646 0 0) 0x7f6a84003520 con 0x7f6a9c0cc490 2024-03-28T13:11:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.739+0000 7f6aa0e00640 1 -- 172.21.15.19:0/2662567435 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c1600e0 con 0x7f6a9c15fc50 2024-03-28T13:11:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.739+0000 7f6aa1800640 1 --2- 172.21.15.19:0/2662567435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c0cc0b0 0x7f6a9c06dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:11:51.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.740+0000 7f6aa0e00640 1 --2- 172.21.15.19:0/2662567435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9c15fc50 0x7f6a9c0cfdc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6a880097a0 tx=0x7f6a8802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8fedc37645fe016 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:51.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.741+0000 7f6a9be00640 1 -- 172.21.15.19:0/2662567435 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88037070 con 0x7f6a9c15fc50 2024-03-28T13:11:51.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.741+0000 7f6a9be00640 1 -- 172.21.15.19:0/2662567435 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a88030510 con 0x7f6a9c15fc50 2024-03-28T13:11:51.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.741+0000 7f6a9be00640 1 -- 172.21.15.19:0/2662567435 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a94001670 con 0x7f6a9c15fc50 2024-03-28T13:11:51.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.741+0000 7f6aa2c00640 1 -- 172.21.15.19:0/2662567435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9c15fc50 msgr2=0x7f6a9c0cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:51.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.741+0000 7f6aa2c00640 1 --2- 172.21.15.19:0/2662567435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9c15fc50 0x7f6a9c0cfdc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6a880097a0 tx=0x7f6a8802b790 comp rx=0 tx=0).stop 2024-03-28T13:11:51.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.742+0000 7f6aa2c00640 1 -- 172.21.15.19:0/2662567435 shutdown_connections 2024-03-28T13:11:51.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.742+0000 7f6aa2c00640 1 --2- 172.21.15.19:0/2662567435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9c15fc50 0x7f6a9c0cfdc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:51.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.742+0000 7f6aa2c00640 1 --2- 172.21.15.19:0/2662567435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c0cc0b0 0x7f6a9c06dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:51.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.742+0000 7f6aa2c00640 1 -- 172.21.15.19:0/2662567435 >> 172.21.15.19:0/2662567435 conn(0x7f6a9c0ccad0 msgr2=0x7f6a9c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:51.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.742+0000 7f6aa2c00640 1 -- 172.21.15.19:0/2662567435 shutdown_connections 2024-03-28T13:11:51.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.742+0000 7f6aa2c00640 1 -- 172.21.15.19:0/2662567435 wait complete. 2024-03-28T13:11:51.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.743+0000 7f6aa2c00640 1 Processor -- start 2024-03-28T13:11:51.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.744+0000 7f6aa2c00640 1 -- start start 2024-03-28T13:11:51.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.745+0000 7f6aa2c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a9c0cc0b0 0x7f6a9c0b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:51.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.745+0000 7f6aa2c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c0cc490 0x7f6a9c137b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.745+0000 7f6aa2c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9c138920 0x7f6a9c1e73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.745+0000 7f6aa2c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c160980 con 0x7f6a9c0cc0b0 2024-03-28T13:11:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.745+0000 7f6aa2c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9c1606a0 con 0x7f6a9c138920 2024-03-28T13:11:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.745+0000 7f6aa2c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6a9c0d0300 con 0x7f6a9c0cc490 2024-03-28T13:11:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.745+0000 7f6aa1800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a9c0cc0b0 0x7f6a9c0b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.745+0000 7f6aa1800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a9c0cc0b0 0x7f6a9c0b1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45442/0 (socket says 172.21.15.19:45442) 2024-03-28T13:11:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.745+0000 7f6aa1800640 1 -- 172.21.15.19:0/3316435482 learned_addr learned my addr 172.21.15.19:0/3316435482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.745+0000 7f6aa0e00640 1 --2- 172.21.15.19:0/3316435482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c0cc490 0x7f6a9c137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.745+0000 7f6aa2200640 1 --2- 172.21.15.19:0/3316435482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9c138920 0x7f6a9c1e73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.746+0000 7f6aa1800640 1 -- 172.21.15.19:0/3316435482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c0cc490 msgr2=0x7f6a9c137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.746+0000 7f6aa1800640 1 --2- 172.21.15.19:0/3316435482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c0cc490 0x7f6a9c137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.746+0000 7f6aa1800640 1 -- 172.21.15.19:0/3316435482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9c138920 msgr2=0x7f6a9c1e73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.746+0000 7f6aa1800640 1 --2- 172.21.15.19:0/3316435482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9c138920 0x7f6a9c1e73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:51.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.746+0000 7f6aa1800640 1 -- 172.21.15.19:0/3316435482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9c138470 con 0x7f6a9c0cc0b0 2024-03-28T13:11:51.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.746+0000 7f6aa0e00640 1 --2- 172.21.15.19:0/3316435482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c0cc490 0x7f6a9c137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:11:51.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.746+0000 7f6aa1800640 1 --2- 172.21.15.19:0/3316435482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a9c0cc0b0 0x7f6a9c0b1dd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000efc0 tx=0x7f6a9000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:51.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.746+0000 7f6a99600640 1 -- 172.21.15.19:0/3316435482 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90018020 con 0x7f6a9c0cc0b0 2024-03-28T13:11:51.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.746+0000 7f6a99600640 1 -- 172.21.15.19:0/3316435482 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a90014680 con 0x7f6a9c0cc0b0 2024-03-28T13:11:51.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.746+0000 7f6a99600640 1 -- 172.21.15.19:0/3316435482 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a90014a50 con 0x7f6a9c0cc0b0 2024-03-28T13:11:51.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.746+0000 7f6aa2c00640 1 -- 172.21.15.19:0/3316435482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9c0cf6c0 con 0x7f6a9c0cc0b0 2024-03-28T13:11:51.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.746+0000 7f6aa2c00640 1 -- 172.21.15.19:0/3316435482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a9c072370 con 0x7f6a9c0cc0b0 2024-03-28T13:11:51.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.748+0000 7f6aa2c00640 1 -- 172.21.15.19:0/3316435482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a9c075940 con 0x7f6a9c0cc0b0 2024-03-28T13:11:51.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.749+0000 7f6a99600640 1 -- 172.21.15.19:0/3316435482 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6a9001b020 con 0x7f6a9c0cc0b0 2024-03-28T13:11:51.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.749+0000 7f6a99600640 1 --2- 172.21.15.19:0/3316435482 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a7c06c530 0x7f6a7c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:51.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.750+0000 7f6a99600640 1 -- 172.21.15.19:0/3316435482 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f6a9008c6c0 con 0x7f6a9c0cc0b0 2024-03-28T13:11:51.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.750+0000 7f6aa0e00640 1 --2- 172.21.15.19:0/3316435482 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a7c06c530 0x7f6a7c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:51.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.751+0000 7f6aa0e00640 1 --2- 172.21.15.19:0/3316435482 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a7c06c530 0x7f6a7c06e9f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6a88034000 tx=0x7f6a8802f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:51.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:51.759+0000 7f6a99600640 1 -- 172.21.15.19:0/3316435482 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6a90022020 con 0x7f6a9c0cc0b0 2024-03-28T13:11:52.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.053+0000 7f6aa2c00640 1 -- 172.21.15.19:0/3316435482 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a9c138750 con 0x7f6a7c06c530 2024-03-28T13:11:52.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.072+0000 7f6a99600640 1 -- 172.21.15.19:0/3316435482 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251950 (secure 0 0 0) 0x7f6a9c138750 con 0x7f6a7c06c530 2024-03-28T13:11:52.076 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:52.077 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:11:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.077+0000 7f6aa2c00640 1 -- 172.21.15.19:0/3316435482 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a7c06c530 msgr2=0x7f6a7c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.077+0000 7f6aa2c00640 1 --2- 172.21.15.19:0/3316435482 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a7c06c530 0x7f6a7c06e9f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6a88034000 tx=0x7f6a8802f000 comp rx=0 tx=0).stop 2024-03-28T13:11:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.077+0000 7f6aa2c00640 1 -- 172.21.15.19:0/3316435482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a9c0cc0b0 msgr2=0x7f6a9c0b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.077+0000 7f6aa2c00640 1 --2- 172.21.15.19:0/3316435482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a9c0cc0b0 0x7f6a9c0b1dd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6a9000efc0 tx=0x7f6a9000c490 comp rx=0 tx=0).stop 2024-03-28T13:11:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.077+0000 7f6aa2c00640 1 -- 172.21.15.19:0/3316435482 shutdown_connections 2024-03-28T13:11:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.077+0000 7f6aa2c00640 1 --2- 172.21.15.19:0/3316435482 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a7c06c530 0x7f6a7c06e9f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.077+0000 7f6aa2c00640 1 --2- 172.21.15.19:0/3316435482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9c138920 0x7f6a9c1e73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.077+0000 7f6aa2c00640 1 --2- 172.21.15.19:0/3316435482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a9c0cc490 0x7f6a9c137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.077+0000 7f6aa2c00640 1 --2- 172.21.15.19:0/3316435482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a9c0cc0b0 0x7f6a9c0b1dd0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.077+0000 7f6aa2c00640 1 -- 172.21.15.19:0/3316435482 >> 172.21.15.19:0/3316435482 conn(0x7f6a9c0ccad0 msgr2=0x7f6a9c15ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:52.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.077+0000 7f6aa2c00640 1 -- 172.21.15.19:0/3316435482 shutdown_connections 2024-03-28T13:11:52.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.077+0000 7f6aa2c00640 1 -- 172.21.15.19:0/3316435482 wait complete. 2024-03-28T13:11:52.098 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-03-28T13:11:52.028694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225964,"kb_used_data":5604,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748756,"statfs":{"total":773094113280,"available":772862726144,"internally_reserved":0,"allocated":5738496,"data_stored":3013592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":225496628},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.140370"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312381+0000","last_change":"2024-03-28T13:11:46.916176+0000","last_active":"2024-03-28T13:11:48.312381+0000","last_peered":"2024-03-28T13:11:48.312381+0000","last_clean":"2024-03-28T13:11:48.312381+0000","last_became_active":"2024-03-28T13:11:46.915866+0000","last_became_peered":"2024-03-28T13:11:46.915866+0000","last_unstale":"2024-03-28T13:11:48.312381+0000","last_undegraded":"2024-03-28T13:11:48.312381+0000","last_fullsized":"2024-03-28T13:11:48.312381+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:51:37.394011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.310079+0000","last_change":"2024-03-28T13:11:46.933764+0000","last_active":"2024-03-28T13:11:48.310079+0000","last_peered":"2024-03-28T13:11:48.310079+0000","last_clean":"2024-03-28T13:11:48.310079+0000","last_became_active":"2024-03-28T13:11:46.933467+0000","last_became_peered":"2024-03-28T13:11:46.933467+0000","last_unstale":"2024-03-28T13:11:48.310079+0000","last_undegraded":"2024-03-28T13:11:48.310079+0000","last_fullsized":"2024-03-28T13:11:48.310079+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:46:36.058976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.897089+0000","last_change":"2024-03-28T13:11:46.934791+0000","last_active":"2024-03-28T13:11:47.897089+0000","last_peered":"2024-03-28T13:11:47.897089+0000","last_clean":"2024-03-28T13:11:47.897089+0000","last_became_active":"2024-03-28T13:11:46.934522+0000","last_became_peered":"2024-03-28T13:11:46.934522+0000","last_unstale":"2024-03-28T13:11:47.897089+0000","last_undegraded":"2024-03-28T13:11:47.897089+0000","last_fullsized":"2024-03-28T13:11:47.897089+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:32:24.167098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899850+0000","last_change":"2024-03-28T13:11:46.935132+0000","last_active":"2024-03-28T13:11:47.899850+0000","last_peered":"2024-03-28T13:11:47.899850+0000","last_clean":"2024-03-28T13:11:47.899850+0000","last_became_active":"2024-03-28T13:11:46.934845+0000","last_became_peered":"2024-03-28T13:11:46.934845+0000","last_unstale":"2024-03-28T13:11:47.899850+0000","last_undegraded":"2024-03-28T13:11:47.899850+0000","last_fullsized":"2024-03-28T13:11:47.899850+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:51:37.394011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311492+0000","last_change":"2024-03-28T13:11:46.921565+0000","last_active":"2024-03-28T13:11:48.311492+0000","last_peered":"2024-03-28T13:11:48.311492+0000","last_clean":"2024-03-28T13:11:48.311492+0000","last_became_active":"2024-03-28T13:11:46.921100+0000","last_became_peered":"2024-03-28T13:11:46.921100+0000","last_unstale":"2024-03-28T13:11:48.311492+0000","last_undegraded":"2024-03-28T13:11:48.311492+0000","last_fullsized":"2024-03-28T13:11:48.311492+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:49:47.662466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.310520+0000","last_change":"2024-03-28T13:11:46.941771+0000","last_active":"2024-03-28T13:11:48.310520+0000","last_peered":"2024-03-28T13:11:48.310520+0000","last_clean":"2024-03-28T13:11:48.310520+0000","last_became_active":"2024-03-28T13:11:46.940315+0000","last_became_peered":"2024-03-28T13:11:46.940315+0000","last_unstale":"2024-03-28T13:11:48.310520+0000","last_undegraded":"2024-03-28T13:11:48.310520+0000","last_fullsized":"2024-03-28T13:11:48.310520+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:03:02.471429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.898460+0000","last_change":"2024-03-28T13:11:46.942957+0000","last_active":"2024-03-28T13:11:47.898460+0000","last_peered":"2024-03-28T13:11:47.898460+0000","last_clean":"2024-03-28T13:11:47.898460+0000","last_became_active":"2024-03-28T13:11:46.941329+0000","last_became_peered":"2024-03-28T13:11:46.941329+0000","last_unstale":"2024-03-28T13:11:47.898460+0000","last_undegraded":"2024-03-28T13:11:47.898460+0000","last_fullsized":"2024-03-28T13:11:47.898460+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:15.846948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899660+0000","last_change":"2024-03-28T13:11:46.935947+0000","last_active":"2024-03-28T13:11:47.899660+0000","last_peered":"2024-03-28T13:11:47.899660+0000","last_clean":"2024-03-28T13:11:47.899660+0000","last_became_active":"2024-03-28T13:11:46.935483+0000","last_became_peered":"2024-03-28T13:11:46.935483+0000","last_unstale":"2024-03-28T13:11:47.899660+0000","last_undegraded":"2024-03-28T13:11:47.899660+0000","last_fullsized":"2024-03-28T13:11:47.899660+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:22:58.039744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311564+0000","last_change":"2024-03-28T13:11:46.943250+0000","last_active":"2024-03-28T13:11:48.311564+0000","last_peered":"2024-03-28T13:11:48.311564+0000","last_clean":"2024-03-28T13:11:48.311564+0000","last_became_active":"2024-03-28T13:11:46.942810+0000","last_became_peered":"2024-03-28T13:11:46.942810+0000","last_unstale":"2024-03-28T13:11:48.311564+0000","last_undegraded":"2024-03-28T13:11:48.311564+0000","last_fullsized":"2024-03-28T13:11:48.311564+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:49:47.662466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312572+0000","last_change":"2024-03-28T13:11:46.930353+0000","last_active":"2024-03-28T13:11:48.312572+0000","last_peered":"2024-03-28T13:11:48.312572+0000","last_clean":"2024-03-28T13:11:48.312572+0000","last_became_active":"2024-03-28T13:11:46.930052+0000","last_became_peered":"2024-03-28T13:11:46.930052+0000","last_unstale":"2024-03-28T13:11:48.312572+0000","last_undegraded":"2024-03-28T13:11:48.312572+0000","last_fullsized":"2024-03-28T13:11:48.312572+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:43.386902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900563+0000","last_change":"2024-03-28T13:11:46.929838+0000","last_active":"2024-03-28T13:11:47.900563+0000","last_peered":"2024-03-28T13:11:47.900563+0000","last_clean":"2024-03-28T13:11:47.900563+0000","last_became_active":"2024-03-28T13:11:46.928355+0000","last_became_peered":"2024-03-28T13:11:46.928355+0000","last_unstale":"2024-03-28T13:11:47.900563+0000","last_undegraded":"2024-03-28T13:11:47.900563+0000","last_fullsized":"2024-03-28T13:11:47.900563+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:13:37.250641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.309469+0000","last_change":"2024-03-28T13:11:46.906251+0000","last_active":"2024-03-28T13:11:48.309469+0000","last_peered":"2024-03-28T13:11:48.309469+0000","last_clean":"2024-03-28T13:11:48.309469+0000","last_became_active":"2024-03-28T13:11:46.905920+0000","last_became_peered":"2024-03-28T13:11:46.905920+0000","last_unstale":"2024-03-28T13:11:48.309469+0000","last_undegraded":"2024-03-28T13:11:48.309469+0000","last_fullsized":"2024-03-28T13:11:48.309469+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:09:38.289283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.310842+0000","last_change":"2024-03-28T13:11:46.941054+0000","last_active":"2024-03-28T13:11:48.310842+0000","last_peered":"2024-03-28T13:11:48.310842+0000","last_clean":"2024-03-28T13:11:48.310842+0000","last_became_active":"2024-03-28T13:11:46.940744+0000","last_became_peered":"2024-03-28T13:11:46.940744+0000","last_unstale":"2024-03-28T13:11:48.310842+0000","last_undegraded":"2024-03-28T13:11:48.310842+0000","last_fullsized":"2024-03-28T13:11:48.310842+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:09:38.289283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899554+0000","last_change":"2024-03-28T13:11:46.938966+0000","last_active":"2024-03-28T13:11:47.899554+0000","last_peered":"2024-03-28T13:11:47.899554+0000","last_clean":"2024-03-28T13:11:47.899554+0000","last_became_active":"2024-03-28T13:11:46.938618+0000","last_became_peered":"2024-03-28T13:11:46.938618+0000","last_unstale":"2024-03-28T13:11:47.899554+0000","last_undegraded":"2024-03-28T13:11:47.899554+0000","last_fullsized":"2024-03-28T13:11:47.899554+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:11.680247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900082+0000","last_change":"2024-03-28T13:11:46.926823+0000","last_active":"2024-03-28T13:11:47.900082+0000","last_peered":"2024-03-28T13:11:47.900082+0000","last_clean":"2024-03-28T13:11:47.900082+0000","last_became_active":"2024-03-28T13:11:46.926500+0000","last_became_peered":"2024-03-28T13:11:46.926500+0000","last_unstale":"2024-03-28T13:11:47.900082+0000","last_undegraded":"2024-03-28T13:11:47.900082+0000","last_fullsized":"2024-03-28T13:11:47.900082+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:43:15.464354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311620+0000","last_change":"2024-03-28T13:11:46.938852+0000","last_active":"2024-03-28T13:11:48.311620+0000","last_peered":"2024-03-28T13:11:48.311620+0000","last_clean":"2024-03-28T13:11:48.311620+0000","last_became_active":"2024-03-28T13:11:46.938659+0000","last_became_peered":"2024-03-28T13:11:46.938659+0000","last_unstale":"2024-03-28T13:11:48.311620+0000","last_undegraded":"2024-03-28T13:11:48.311620+0000","last_fullsized":"2024-03-28T13:11:48.311620+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:46:36.058976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.310482+0000","last_change":"2024-03-28T13:11:46.917132+0000","last_active":"2024-03-28T13:11:48.310482+0000","last_peered":"2024-03-28T13:11:48.310482+0000","last_clean":"2024-03-28T13:11:48.310482+0000","last_became_active":"2024-03-28T13:11:46.916636+0000","last_became_peered":"2024-03-28T13:11:46.916636+0000","last_unstale":"2024-03-28T13:11:48.310482+0000","last_undegraded":"2024-03-28T13:11:48.310482+0000","last_fullsized":"2024-03-28T13:11:48.310482+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:11.680247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.897209+0000","last_change":"2024-03-28T13:11:46.940658+0000","last_active":"2024-03-28T13:11:47.897209+0000","last_peered":"2024-03-28T13:11:47.897209+0000","last_clean":"2024-03-28T13:11:47.897209+0000","last_became_active":"2024-03-28T13:11:46.940097+0000","last_became_peered":"2024-03-28T13:11:46.940097+0000","last_unstale":"2024-03-28T13:11:47.897209+0000","last_undegraded":"2024-03-28T13:11:47.897209+0000","last_fullsized":"2024-03-28T13:11:47.897209+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:54:41.176763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311964+0000","last_change":"2024-03-28T13:11:46.921398+0000","last_active":"2024-03-28T13:11:48.311964+0000","last_peered":"2024-03-28T13:11:48.311964+0000","last_clean":"2024-03-28T13:11:48.311964+0000","last_became_active":"2024-03-28T13:11:46.921083+0000","last_became_peered":"2024-03-28T13:11:46.921083+0000","last_unstale":"2024-03-28T13:11:48.311964+0000","last_undegraded":"2024-03-28T13:11:48.311964+0000","last_fullsized":"2024-03-28T13:11:48.311964+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:22:58.039744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312278+0000","last_change":"2024-03-28T13:11:46.931175+0000","last_active":"2024-03-28T13:11:48.312278+0000","last_peered":"2024-03-28T13:11:48.312278+0000","last_clean":"2024-03-28T13:11:48.312278+0000","last_became_active":"2024-03-28T13:11:46.930846+0000","last_became_peered":"2024-03-28T13:11:46.930846+0000","last_unstale":"2024-03-28T13:11:48.312278+0000","last_undegraded":"2024-03-28T13:11:48.312278+0000","last_fullsized":"2024-03-28T13:11:48.312278+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:34:41.880929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900445+0000","last_change":"2024-03-28T13:11:46.930815+0000","last_active":"2024-03-28T13:11:47.900445+0000","last_peered":"2024-03-28T13:11:47.900445+0000","last_clean":"2024-03-28T13:11:47.900445+0000","last_became_active":"2024-03-28T13:11:46.930132+0000","last_became_peered":"2024-03-28T13:11:46.930132+0000","last_unstale":"2024-03-28T13:11:47.900445+0000","last_undegraded":"2024-03-28T13:11:47.900445+0000","last_fullsized":"2024-03-28T13:11:47.900445+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:14:03.582306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.925513+0000","last_change":"2024-03-28T13:11:46.916936+0000","last_active":"2024-03-28T13:11:46.925513+0000","last_peered":"2024-03-28T13:11:46.925513+0000","last_clean":"2024-03-28T13:11:46.925513+0000","last_became_active":"2024-03-28T13:11:46.916561+0000","last_became_peered":"2024-03-28T13:11:46.916561+0000","last_unstale":"2024-03-28T13:11:46.925513+0000","last_undegraded":"2024-03-28T13:11:46.925513+0000","last_fullsized":"2024-03-28T13:11:46.925513+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:45:18.443944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.898714+0000","last_change":"2024-03-28T13:11:46.912064+0000","last_active":"2024-03-28T13:11:47.898714+0000","last_peered":"2024-03-28T13:11:47.898714+0000","last_clean":"2024-03-28T13:11:47.898714+0000","last_became_active":"2024-03-28T13:11:46.911776+0000","last_became_peered":"2024-03-28T13:11:46.911776+0000","last_unstale":"2024-03-28T13:11:47.898714+0000","last_undegraded":"2024-03-28T13:11:47.898714+0000","last_fullsized":"2024-03-28T13:11:47.898714+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:19:51.184803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.309226+0000","last_change":"2024-03-28T13:11:46.928169+0000","last_active":"2024-03-28T13:11:48.309226+0000","last_peered":"2024-03-28T13:11:48.309226+0000","last_clean":"2024-03-28T13:11:48.309226+0000","last_became_active":"2024-03-28T13:11:46.927661+0000","last_became_peered":"2024-03-28T13:11:46.927661+0000","last_unstale":"2024-03-28T13:11:48.309226+0000","last_undegraded":"2024-03-28T13:11:48.309226+0000","last_fullsized":"2024-03-28T13:11:48.309226+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:07.765070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312391+0000","last_change":"2024-03-28T13:11:46.915104+0000","last_active":"2024-03-28T13:11:48.312391+0000","last_peered":"2024-03-28T13:11:48.312391+0000","last_clean":"2024-03-28T13:11:48.312391+0000","last_became_active":"2024-03-28T13:11:46.914733+0000","last_became_peered":"2024-03-28T13:11:46.914733+0000","last_unstale":"2024-03-28T13:11:48.312391+0000","last_undegraded":"2024-03-28T13:11:48.312391+0000","last_fullsized":"2024-03-28T13:11:48.312391+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:16:23.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900136+0000","last_change":"2024-03-28T13:11:46.938100+0000","last_active":"2024-03-28T13:11:47.900136+0000","last_peered":"2024-03-28T13:11:47.900136+0000","last_clean":"2024-03-28T13:11:47.900136+0000","last_became_active":"2024-03-28T13:11:46.937802+0000","last_became_peered":"2024-03-28T13:11:46.937802+0000","last_unstale":"2024-03-28T13:11:47.900136+0000","last_undegraded":"2024-03-28T13:11:47.900136+0000","last_fullsized":"2024-03-28T13:11:47.900136+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:02:26.883141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899266+0000","last_change":"2024-03-28T13:11:46.929307+0000","last_active":"2024-03-28T13:11:47.899266+0000","last_peered":"2024-03-28T13:11:47.899266+0000","last_clean":"2024-03-28T13:11:47.899266+0000","last_became_active":"2024-03-28T13:11:46.928997+0000","last_became_peered":"2024-03-28T13:11:46.928997+0000","last_unstale":"2024-03-28T13:11:47.899266+0000","last_undegraded":"2024-03-28T13:11:47.899266+0000","last_fullsized":"2024-03-28T13:11:47.899266+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:50:07.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899309+0000","last_change":"2024-03-28T13:11:46.908891+0000","last_active":"2024-03-28T13:11:47.899309+0000","last_peered":"2024-03-28T13:11:47.899309+0000","last_clean":"2024-03-28T13:11:47.899309+0000","last_became_active":"2024-03-28T13:11:46.908565+0000","last_became_peered":"2024-03-28T13:11:46.908565+0000","last_unstale":"2024-03-28T13:11:47.899309+0000","last_undegraded":"2024-03-28T13:11:47.899309+0000","last_fullsized":"2024-03-28T13:11:47.899309+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:24:28.942924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311833+0000","last_change":"2024-03-28T13:11:46.931727+0000","last_active":"2024-03-28T13:11:48.311833+0000","last_peered":"2024-03-28T13:11:48.311833+0000","last_clean":"2024-03-28T13:11:48.311833+0000","last_became_active":"2024-03-28T13:11:46.931034+0000","last_became_peered":"2024-03-28T13:11:46.931034+0000","last_unstale":"2024-03-28T13:11:48.311833+0000","last_undegraded":"2024-03-28T13:11:48.311833+0000","last_fullsized":"2024-03-28T13:11:48.311833+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:11.680247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.309285+0000","last_change":"2024-03-28T13:11:46.906427+0000","last_active":"2024-03-28T13:11:48.309285+0000","last_peered":"2024-03-28T13:11:48.309285+0000","last_clean":"2024-03-28T13:11:48.309285+0000","last_became_active":"2024-03-28T13:11:46.906111+0000","last_became_peered":"2024-03-28T13:11:46.906111+0000","last_unstale":"2024-03-28T13:11:48.309285+0000","last_undegraded":"2024-03-28T13:11:48.309285+0000","last_fullsized":"2024-03-28T13:11:48.309285+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:03:02.471429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.310298+0000","last_change":"2024-03-28T13:11:46.923320+0000","last_active":"2024-03-28T13:11:48.310298+0000","last_peered":"2024-03-28T13:11:48.310298+0000","last_clean":"2024-03-28T13:11:48.310298+0000","last_became_active":"2024-03-28T13:11:46.920809+0000","last_became_peered":"2024-03-28T13:11:46.920809+0000","last_unstale":"2024-03-28T13:11:48.310298+0000","last_undegraded":"2024-03-28T13:11:48.310298+0000","last_fullsized":"2024-03-28T13:11:48.310298+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:09:37.323636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900233+0000","last_change":"2024-03-28T13:11:46.935032+0000","last_active":"2024-03-28T13:11:47.900233+0000","last_peered":"2024-03-28T13:11:47.900233+0000","last_clean":"2024-03-28T13:11:47.900233+0000","last_became_active":"2024-03-28T13:11:46.931213+0000","last_became_peered":"2024-03-28T13:11:46.931213+0000","last_unstale":"2024-03-28T13:11:47.900233+0000","last_undegraded":"2024-03-28T13:11:47.900233+0000","last_fullsized":"2024-03-28T13:11:47.900233+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:15:07.394420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.939599+0000","last_change":"2024-03-28T13:11:46.937952+0000","last_active":"2024-03-28T13:11:46.939599+0000","last_peered":"2024-03-28T13:11:46.939599+0000","last_clean":"2024-03-28T13:11:46.939599+0000","last_became_active":"2024-03-28T13:11:46.937671+0000","last_became_peered":"2024-03-28T13:11:46.937671+0000","last_unstale":"2024-03-28T13:11:46.939599+0000","last_undegraded":"2024-03-28T13:11:46.939599+0000","last_fullsized":"2024-03-28T13:11:46.939599+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:41:11.684448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"48'4","reported_seq":18,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311840+0000","last_change":"2024-03-28T13:11:45.920678+0000","last_active":"2024-03-28T13:11:48.311840+0000","last_peered":"2024-03-28T13:11:48.311840+0000","last_clean":"2024-03-28T13:11:48.311840+0000","last_became_active":"2024-03-28T13:11:45.920319+0000","last_became_peered":"2024-03-28T13:11:45.920319+0000","last_unstale":"2024-03-28T13:11:48.311840+0000","last_undegraded":"2024-03-28T13:11:48.311840+0000","last_fullsized":"2024-03-28T13:11:48.311840+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:46:08.557688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312621+0000","last_change":"2024-03-28T13:11:46.924030+0000","last_active":"2024-03-28T13:11:48.312621+0000","last_peered":"2024-03-28T13:11:48.312621+0000","last_clean":"2024-03-28T13:11:48.312621+0000","last_became_active":"2024-03-28T13:11:46.923706+0000","last_became_peered":"2024-03-28T13:11:46.923706+0000","last_unstale":"2024-03-28T13:11:48.312621+0000","last_undegraded":"2024-03-28T13:11:48.312621+0000","last_fullsized":"2024-03-28T13:11:48.312621+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:23:39.500731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899436+0000","last_change":"2024-03-28T13:11:45.916652+0000","last_active":"2024-03-28T13:11:47.899436+0000","last_peered":"2024-03-28T13:11:47.899436+0000","last_clean":"2024-03-28T13:11:47.899436+0000","last_became_active":"2024-03-28T13:11:45.916355+0000","last_became_peered":"2024-03-28T13:11:45.916355+0000","last_unstale":"2024-03-28T13:11:47.899436+0000","last_undegraded":"2024-03-28T13:11:47.899436+0000","last_fullsized":"2024-03-28T13:11:47.899436+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:58:28.887733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.309541+0000","last_change":"2024-03-28T13:11:46.933003+0000","last_active":"2024-03-28T13:11:48.309541+0000","last_peered":"2024-03-28T13:11:48.309541+0000","last_clean":"2024-03-28T13:11:48.309541+0000","last_became_active":"2024-03-28T13:11:46.932767+0000","last_became_peered":"2024-03-28T13:11:46.932767+0000","last_unstale":"2024-03-28T13:11:48.309541+0000","last_undegraded":"2024-03-28T13:11:48.309541+0000","last_fullsized":"2024-03-28T13:11:48.309541+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:58:43.469573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311121+0000","last_change":"2024-03-28T13:11:45.916445+0000","last_active":"2024-03-28T13:11:48.311121+0000","last_peered":"2024-03-28T13:11:48.311121+0000","last_clean":"2024-03-28T13:11:48.311121+0000","last_became_active":"2024-03-28T13:11:45.915977+0000","last_became_peered":"2024-03-28T13:11:45.915977+0000","last_unstale":"2024-03-28T13:11:48.311121+0000","last_undegraded":"2024-03-28T13:11:48.311121+0000","last_fullsized":"2024-03-28T13:11:48.311121+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:37:07.491590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900819+0000","last_change":"2024-03-28T13:11:46.938178+0000","last_active":"2024-03-28T13:11:47.900819+0000","last_peered":"2024-03-28T13:11:47.900819+0000","last_clean":"2024-03-28T13:11:47.900819+0000","last_became_active":"2024-03-28T13:11:46.937858+0000","last_became_peered":"2024-03-28T13:11:46.937858+0000","last_unstale":"2024-03-28T13:11:47.900819+0000","last_undegraded":"2024-03-28T13:11:47.900819+0000","last_fullsized":"2024-03-28T13:11:47.900819+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:34:41.880929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899172+0000","last_change":"2024-03-28T13:11:45.916697+0000","last_active":"2024-03-28T13:11:47.899172+0000","last_peered":"2024-03-28T13:11:47.899172+0000","last_clean":"2024-03-28T13:11:47.899172+0000","last_became_active":"2024-03-28T13:11:45.916441+0000","last_became_peered":"2024-03-28T13:11:45.916441+0000","last_unstale":"2024-03-28T13:11:47.899172+0000","last_undegraded":"2024-03-28T13:11:47.899172+0000","last_fullsized":"2024-03-28T13:11:47.899172+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:33:06.249680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.898765+0000","last_change":"2024-03-28T13:11:46.928234+0000","last_active":"2024-03-28T13:11:47.898765+0000","last_peered":"2024-03-28T13:11:47.898765+0000","last_clean":"2024-03-28T13:11:47.898765+0000","last_became_active":"2024-03-28T13:11:46.927899+0000","last_became_peered":"2024-03-28T13:11:46.927899+0000","last_unstale":"2024-03-28T13:11:47.898765+0000","last_undegraded":"2024-03-28T13:11:47.898765+0000","last_fullsized":"2024-03-28T13:11:47.898765+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:41:28.018217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311556+0000","last_change":"2024-03-28T13:11:45.917510+0000","last_active":"2024-03-28T13:11:48.311556+0000","last_peered":"2024-03-28T13:11:48.311556+0000","last_clean":"2024-03-28T13:11:48.311556+0000","last_became_active":"2024-03-28T13:11:45.916797+0000","last_became_peered":"2024-03-28T13:11:45.916797+0000","last_unstale":"2024-03-28T13:11:48.311556+0000","last_undegraded":"2024-03-28T13:11:48.311556+0000","last_fullsized":"2024-03-28T13:11:48.311556+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:09:50.989777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311012+0000","last_change":"2024-03-28T13:11:46.936730+0000","last_active":"2024-03-28T13:11:48.311012+0000","last_peered":"2024-03-28T13:11:48.311012+0000","last_clean":"2024-03-28T13:11:48.311012+0000","last_became_active":"2024-03-28T13:11:46.936463+0000","last_became_peered":"2024-03-28T13:11:46.936463+0000","last_unstale":"2024-03-28T13:11:48.311012+0000","last_undegraded":"2024-03-28T13:11:48.311012+0000","last_fullsized":"2024-03-28T13:11:48.311012+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:14:03.582306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.898637+0000","last_change":"2024-03-28T13:11:45.921482+0000","last_active":"2024-03-28T13:11:47.898637+0000","last_peered":"2024-03-28T13:11:47.898637+0000","last_clean":"2024-03-28T13:11:47.898637+0000","last_became_active":"2024-03-28T13:11:45.918997+0000","last_became_peered":"2024-03-28T13:11:45.918997+0000","last_unstale":"2024-03-28T13:11:47.898637+0000","last_undegraded":"2024-03-28T13:11:47.898637+0000","last_fullsized":"2024-03-28T13:11:47.898637+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:38:40.871007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312729+0000","last_change":"2024-03-28T13:11:46.945709+0000","last_active":"2024-03-28T13:11:48.312729+0000","last_peered":"2024-03-28T13:11:48.312729+0000","last_clean":"2024-03-28T13:11:48.312729+0000","last_became_active":"2024-03-28T13:11:46.945294+0000","last_became_peered":"2024-03-28T13:11:46.945294+0000","last_unstale":"2024-03-28T13:11:48.312729+0000","last_undegraded":"2024-03-28T13:11:48.312729+0000","last_fullsized":"2024-03-28T13:11:48.312729+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:33:07.264658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311327+0000","last_change":"2024-03-28T13:11:45.921244+0000","last_active":"2024-03-28T13:11:48.311327+0000","last_peered":"2024-03-28T13:11:48.311327+0000","last_clean":"2024-03-28T13:11:48.311327+0000","last_became_active":"2024-03-28T13:11:45.920315+0000","last_became_peered":"2024-03-28T13:11:45.920315+0000","last_unstale":"2024-03-28T13:11:48.311327+0000","last_undegraded":"2024-03-28T13:11:48.311327+0000","last_fullsized":"2024-03-28T13:11:48.311327+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:39:19.230135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.898684+0000","last_change":"2024-03-28T13:11:46.920925+0000","last_active":"2024-03-28T13:11:47.898684+0000","last_peered":"2024-03-28T13:11:47.898684+0000","last_clean":"2024-03-28T13:11:47.898684+0000","last_became_active":"2024-03-28T13:11:46.920562+0000","last_became_peered":"2024-03-28T13:11:46.920562+0000","last_unstale":"2024-03-28T13:11:47.898684+0000","last_undegraded":"2024-03-28T13:11:47.898684+0000","last_fullsized":"2024-03-28T13:11:47.898684+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:10:15.791360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.310895+0000","last_change":"2024-03-28T13:11:45.919677+0000","last_active":"2024-03-28T13:11:48.310895+0000","last_peered":"2024-03-28T13:11:48.310895+0000","last_clean":"2024-03-28T13:11:48.310895+0000","last_became_active":"2024-03-28T13:11:45.918752+0000","last_became_peered":"2024-03-28T13:11:45.918752+0000","last_unstale":"2024-03-28T13:11:48.310895+0000","last_undegraded":"2024-03-28T13:11:48.310895+0000","last_fullsized":"2024-03-28T13:11:48.310895+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:46:35.043997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.896995+0000","last_change":"2024-03-28T13:11:46.932409+0000","last_active":"2024-03-28T13:11:47.896995+0000","last_peered":"2024-03-28T13:11:47.896995+0000","last_clean":"2024-03-28T13:11:47.896995+0000","last_became_active":"2024-03-28T13:11:46.932085+0000","last_became_peered":"2024-03-28T13:11:46.932085+0000","last_unstale":"2024-03-28T13:11:47.896995+0000","last_undegraded":"2024-03-28T13:11:47.896995+0000","last_fullsized":"2024-03-28T13:11:47.896995+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:10:15.724695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.896661+0000","last_change":"2024-03-28T13:11:45.915205+0000","last_active":"2024-03-28T13:11:47.896661+0000","last_peered":"2024-03-28T13:11:47.896661+0000","last_clean":"2024-03-28T13:11:47.896661+0000","last_became_active":"2024-03-28T13:11:45.914155+0000","last_became_peered":"2024-03-28T13:11:45.914155+0000","last_unstale":"2024-03-28T13:11:47.896661+0000","last_undegraded":"2024-03-28T13:11:47.896661+0000","last_fullsized":"2024-03-28T13:11:47.896661+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:48:42.623229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.896650+0000","last_change":"2024-03-28T13:11:46.926099+0000","last_active":"2024-03-28T13:11:47.896650+0000","last_peered":"2024-03-28T13:11:47.896650+0000","last_clean":"2024-03-28T13:11:47.896650+0000","last_became_active":"2024-03-28T13:11:46.925819+0000","last_became_peered":"2024-03-28T13:11:46.925819+0000","last_unstale":"2024-03-28T13:11:47.896650+0000","last_undegraded":"2024-03-28T13:11:47.896650+0000","last_fullsized":"2024-03-28T13:11:47.896650+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:23:22.567680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312815+0000","last_change":"2024-03-28T13:11:45.919230+0000","last_active":"2024-03-28T13:11:48.312815+0000","last_peered":"2024-03-28T13:11:48.312815+0000","last_clean":"2024-03-28T13:11:48.312815+0000","last_became_active":"2024-03-28T13:11:45.918843+0000","last_became_peered":"2024-03-28T13:11:45.918843+0000","last_unstale":"2024-03-28T13:11:48.312815+0000","last_undegraded":"2024-03-28T13:11:48.312815+0000","last_fullsized":"2024-03-28T13:11:48.312815+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:10.665269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899900+0000","last_change":"2024-03-28T13:11:46.938713+0000","last_active":"2024-03-28T13:11:47.899900+0000","last_peered":"2024-03-28T13:11:47.899900+0000","last_clean":"2024-03-28T13:11:47.899900+0000","last_became_active":"2024-03-28T13:11:46.937364+0000","last_became_peered":"2024-03-28T13:11:46.937364+0000","last_unstale":"2024-03-28T13:11:47.899900+0000","last_undegraded":"2024-03-28T13:11:47.899900+0000","last_fullsized":"2024-03-28T13:11:47.899900+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:16:23.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"48'3","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.309582+0000","last_change":"2024-03-28T13:11:45.918985+0000","last_active":"2024-03-28T13:11:48.309582+0000","last_peered":"2024-03-28T13:11:48.309582+0000","last_clean":"2024-03-28T13:11:48.309582+0000","last_became_active":"2024-03-28T13:11:45.918675+0000","last_became_peered":"2024-03-28T13:11:45.918675+0000","last_unstale":"2024-03-28T13:11:48.309582+0000","last_undegraded":"2024-03-28T13:11:48.309582+0000","last_fullsized":"2024-03-28T13:11:48.309582+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:54:40.161785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899456+0000","last_change":"2024-03-28T13:11:46.942828+0000","last_active":"2024-03-28T13:11:47.899456+0000","last_peered":"2024-03-28T13:11:47.899456+0000","last_clean":"2024-03-28T13:11:47.899456+0000","last_became_active":"2024-03-28T13:11:46.941947+0000","last_became_peered":"2024-03-28T13:11:46.941947+0000","last_unstale":"2024-03-28T13:11:47.899456+0000","last_undegraded":"2024-03-28T13:11:47.899456+0000","last_fullsized":"2024-03-28T13:11:47.899456+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:53:21.650433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"48'6","reported_seq":20,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312587+0000","last_change":"2024-03-28T13:11:45.916407+0000","last_active":"2024-03-28T13:11:48.312587+0000","last_peered":"2024-03-28T13:11:48.312587+0000","last_clean":"2024-03-28T13:11:48.312587+0000","last_became_active":"2024-03-28T13:11:45.916093+0000","last_became_peered":"2024-03-28T13:11:45.916093+0000","last_unstale":"2024-03-28T13:11:48.312587+0000","last_undegraded":"2024-03-28T13:11:48.312587+0000","last_fullsized":"2024-03-28T13:11:48.312587+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:58:42.454595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.896586+0000","last_change":"2024-03-28T13:11:46.936652+0000","last_active":"2024-03-28T13:11:47.896586+0000","last_peered":"2024-03-28T13:11:47.896586+0000","last_clean":"2024-03-28T13:11:47.896586+0000","last_became_active":"2024-03-28T13:11:46.936376+0000","last_became_peered":"2024-03-28T13:11:46.936376+0000","last_unstale":"2024-03-28T13:11:47.896586+0000","last_undegraded":"2024-03-28T13:11:47.896586+0000","last_fullsized":"2024-03-28T13:11:47.896586+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:43:37.798582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"48'2","reported_seq":14,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.939353+0000","last_change":"2024-03-28T13:11:45.923185+0000","last_active":"2024-03-28T13:11:46.939353+0000","last_peered":"2024-03-28T13:11:46.939353+0000","last_clean":"2024-03-28T13:11:46.939353+0000","last_became_active":"2024-03-28T13:11:45.922652+0000","last_became_peered":"2024-03-28T13:11:45.922652+0000","last_unstale":"2024-03-28T13:11:46.939353+0000","last_undegraded":"2024-03-28T13:11:46.939353+0000","last_fullsized":"2024-03-28T13:11:46.939353+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:42:27.769753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.921870+0000","last_change":"2024-03-28T13:11:46.911170+0000","last_active":"2024-03-28T13:11:46.921870+0000","last_peered":"2024-03-28T13:11:46.921870+0000","last_clean":"2024-03-28T13:11:46.921870+0000","last_became_active":"2024-03-28T13:11:46.910443+0000","last_became_peered":"2024-03-28T13:11:46.910443+0000","last_unstale":"2024-03-28T13:11:46.921870+0000","last_undegraded":"2024-03-28T13:11:46.921870+0000","last_fullsized":"2024-03-28T13:11:46.921870+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:14:18.385803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311296+0000","last_change":"2024-03-28T13:11:45.916601+0000","last_active":"2024-03-28T13:11:48.311296+0000","last_peered":"2024-03-28T13:11:48.311296+0000","last_clean":"2024-03-28T13:11:48.311296+0000","last_became_active":"2024-03-28T13:11:45.915689+0000","last_became_peered":"2024-03-28T13:11:45.915689+0000","last_unstale":"2024-03-28T13:11:48.311296+0000","last_undegraded":"2024-03-28T13:11:48.311296+0000","last_fullsized":"2024-03-28T13:11:48.311296+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:37:07.491590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.947527+0000","last_change":"2024-03-28T13:11:46.946271+0000","last_active":"2024-03-28T13:11:46.947527+0000","last_peered":"2024-03-28T13:11:46.947527+0000","last_clean":"2024-03-28T13:11:46.947527+0000","last_became_active":"2024-03-28T13:11:46.945925+0000","last_became_peered":"2024-03-28T13:11:46.945925+0000","last_unstale":"2024-03-28T13:11:46.947527+0000","last_undegraded":"2024-03-28T13:11:46.947527+0000","last_fullsized":"2024-03-28T13:11:46.947527+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:57:46.465180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311961+0000","last_change":"2024-03-28T13:11:45.916288+0000","last_active":"2024-03-28T13:11:48.311961+0000","last_peered":"2024-03-28T13:11:48.311961+0000","last_clean":"2024-03-28T13:11:48.311961+0000","last_became_active":"2024-03-28T13:11:45.915835+0000","last_became_peered":"2024-03-28T13:11:45.915835+0000","last_unstale":"2024-03-28T13:11:48.311961+0000","last_undegraded":"2024-03-28T13:11:48.311961+0000","last_fullsized":"2024-03-28T13:11:48.311961+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:10:10.237245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.309073+0000","last_change":"2024-03-28T13:11:46.915314+0000","last_active":"2024-03-28T13:11:48.309073+0000","last_peered":"2024-03-28T13:11:48.309073+0000","last_clean":"2024-03-28T13:11:48.309073+0000","last_became_active":"2024-03-28T13:11:46.913818+0000","last_became_peered":"2024-03-28T13:11:46.913818+0000","last_unstale":"2024-03-28T13:11:48.309073+0000","last_undegraded":"2024-03-28T13:11:48.309073+0000","last_fullsized":"2024-03-28T13:11:48.309073+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:33:27.743555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"48'8","reported_seq":22,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311190+0000","last_change":"2024-03-28T13:11:45.917197+0000","last_active":"2024-03-28T13:11:48.311190+0000","last_peered":"2024-03-28T13:11:48.311190+0000","last_clean":"2024-03-28T13:11:48.311190+0000","last_became_active":"2024-03-28T13:11:45.916838+0000","last_became_peered":"2024-03-28T13:11:45.916838+0000","last_unstale":"2024-03-28T13:11:48.311190+0000","last_undegraded":"2024-03-28T13:11:48.311190+0000","last_fullsized":"2024-03-28T13:11:48.311190+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:49:46.647488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.930219+0000","last_change":"2024-03-28T13:11:46.927028+0000","last_active":"2024-03-28T13:11:46.930219+0000","last_peered":"2024-03-28T13:11:46.930219+0000","last_clean":"2024-03-28T13:11:46.930219+0000","last_became_active":"2024-03-28T13:11:46.926743+0000","last_became_peered":"2024-03-28T13:11:46.926743+0000","last_unstale":"2024-03-28T13:11:46.930219+0000","last_undegraded":"2024-03-28T13:11:46.930219+0000","last_fullsized":"2024-03-28T13:11:46.930219+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:28:34.523887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900059+0000","last_change":"2024-03-28T13:11:45.914263+0000","last_active":"2024-03-28T13:11:47.900059+0000","last_peered":"2024-03-28T13:11:47.900059+0000","last_clean":"2024-03-28T13:11:47.900059+0000","last_became_active":"2024-03-28T13:11:45.913970+0000","last_became_peered":"2024-03-28T13:11:45.913970+0000","last_unstale":"2024-03-28T13:11:47.900059+0000","last_undegraded":"2024-03-28T13:11:47.900059+0000","last_fullsized":"2024-03-28T13:11:47.900059+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:09:50.989777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.939356+0000","last_change":"2024-03-28T13:11:46.936470+0000","last_active":"2024-03-28T13:11:46.939356+0000","last_peered":"2024-03-28T13:11:46.939356+0000","last_clean":"2024-03-28T13:11:46.939356+0000","last_became_active":"2024-03-28T13:11:46.936152+0000","last_became_peered":"2024-03-28T13:11:46.936152+0000","last_unstale":"2024-03-28T13:11:46.939356+0000","last_undegraded":"2024-03-28T13:11:46.939356+0000","last_fullsized":"2024-03-28T13:11:46.939356+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:04.460078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.897048+0000","last_change":"2024-03-28T13:11:45.922323+0000","last_active":"2024-03-28T13:11:47.897048+0000","last_peered":"2024-03-28T13:11:47.897048+0000","last_clean":"2024-03-28T13:11:47.897048+0000","last_became_active":"2024-03-28T13:11:45.921911+0000","last_became_peered":"2024-03-28T13:11:45.921911+0000","last_unstale":"2024-03-28T13:11:47.897048+0000","last_undegraded":"2024-03-28T13:11:47.897048+0000","last_fullsized":"2024-03-28T13:11:47.897048+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:58:30.044573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.309625+0000","last_change":"2024-03-28T13:11:46.940575+0000","last_active":"2024-03-28T13:11:48.309625+0000","last_peered":"2024-03-28T13:11:48.309625+0000","last_clean":"2024-03-28T13:11:48.309625+0000","last_became_active":"2024-03-28T13:11:46.939564+0000","last_became_peered":"2024-03-28T13:11:46.939564+0000","last_unstale":"2024-03-28T13:11:48.309625+0000","last_undegraded":"2024-03-28T13:11:48.309625+0000","last_fullsized":"2024-03-28T13:11:48.309625+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:49:47.662466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"48'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.897104+0000","last_change":"2024-03-28T13:11:45.915876+0000","last_active":"2024-03-28T13:11:47.897104+0000","last_peered":"2024-03-28T13:11:47.897104+0000","last_clean":"2024-03-28T13:11:47.897104+0000","last_became_active":"2024-03-28T13:11:45.914520+0000","last_became_peered":"2024-03-28T13:11:45.914520+0000","last_unstale":"2024-03-28T13:11:47.897104+0000","last_undegraded":"2024-03-28T13:11:47.897104+0000","last_fullsized":"2024-03-28T13:11:47.897104+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:00:50.999453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.897004+0000","last_change":"2024-03-28T13:11:46.933334+0000","last_active":"2024-03-28T13:11:47.897004+0000","last_peered":"2024-03-28T13:11:47.897004+0000","last_clean":"2024-03-28T13:11:47.897004+0000","last_became_active":"2024-03-28T13:11:46.932919+0000","last_became_peered":"2024-03-28T13:11:46.932919+0000","last_unstale":"2024-03-28T13:11:47.897004+0000","last_undegraded":"2024-03-28T13:11:47.897004+0000","last_fullsized":"2024-03-28T13:11:47.897004+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:36:21.877515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.310519+0000","last_change":"2024-03-28T13:11:45.915563+0000","last_active":"2024-03-28T13:11:48.310519+0000","last_peered":"2024-03-28T13:11:48.310519+0000","last_clean":"2024-03-28T13:11:48.310519+0000","last_became_active":"2024-03-28T13:11:45.915232+0000","last_became_peered":"2024-03-28T13:11:45.915232+0000","last_unstale":"2024-03-28T13:11:48.310519+0000","last_undegraded":"2024-03-28T13:11:48.310519+0000","last_fullsized":"2024-03-28T13:11:48.310519+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:39:19.230135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311251+0000","last_change":"2024-03-28T13:11:46.933297+0000","last_active":"2024-03-28T13:11:48.311251+0000","last_peered":"2024-03-28T13:11:48.311251+0000","last_clean":"2024-03-28T13:11:48.311251+0000","last_became_active":"2024-03-28T13:11:46.933007+0000","last_became_peered":"2024-03-28T13:11:46.933007+0000","last_unstale":"2024-03-28T13:11:48.311251+0000","last_undegraded":"2024-03-28T13:11:48.311251+0000","last_fullsized":"2024-03-28T13:11:48.311251+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:28:12.201099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312096+0000","last_change":"2024-03-28T13:11:45.923927+0000","last_active":"2024-03-28T13:11:48.312096+0000","last_peered":"2024-03-28T13:11:48.312096+0000","last_clean":"2024-03-28T13:11:48.312096+0000","last_became_active":"2024-03-28T13:11:45.923513+0000","last_became_peered":"2024-03-28T13:11:45.923513+0000","last_unstale":"2024-03-28T13:11:48.312096+0000","last_undegraded":"2024-03-28T13:11:48.312096+0000","last_fullsized":"2024-03-28T13:11:48.312096+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:09:37.274305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311266+0000","last_change":"2024-03-28T13:11:46.920550+0000","last_active":"2024-03-28T13:11:48.311266+0000","last_peered":"2024-03-28T13:11:48.311266+0000","last_clean":"2024-03-28T13:11:48.311266+0000","last_became_active":"2024-03-28T13:11:46.918773+0000","last_became_peered":"2024-03-28T13:11:46.918773+0000","last_unstale":"2024-03-28T13:11:48.311266+0000","last_undegraded":"2024-03-28T13:11:48.311266+0000","last_fullsized":"2024-03-28T13:11:48.311266+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:58:43.469573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311532+0000","last_change":"2024-03-28T13:11:45.917843+0000","last_active":"2024-03-28T13:11:48.311532+0000","last_peered":"2024-03-28T13:11:48.311532+0000","last_clean":"2024-03-28T13:11:48.311532+0000","last_became_active":"2024-03-28T13:11:45.917567+0000","last_became_peered":"2024-03-28T13:11:45.917567+0000","last_unstale":"2024-03-28T13:11:48.311532+0000","last_undegraded":"2024-03-28T13:11:48.311532+0000","last_fullsized":"2024-03-28T13:11:48.311532+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:04:53.337486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900067+0000","last_change":"2024-03-28T13:11:46.935009+0000","last_active":"2024-03-28T13:11:47.900067+0000","last_peered":"2024-03-28T13:11:47.900067+0000","last_clean":"2024-03-28T13:11:47.900067+0000","last_became_active":"2024-03-28T13:11:46.930504+0000","last_became_peered":"2024-03-28T13:11:46.930504+0000","last_unstale":"2024-03-28T13:11:47.900067+0000","last_undegraded":"2024-03-28T13:11:47.900067+0000","last_fullsized":"2024-03-28T13:11:47.900067+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:23:39.500731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311557+0000","last_change":"2024-03-28T13:11:45.923921+0000","last_active":"2024-03-28T13:11:48.311557+0000","last_peered":"2024-03-28T13:11:48.311557+0000","last_clean":"2024-03-28T13:11:48.311557+0000","last_became_active":"2024-03-28T13:11:45.923492+0000","last_became_peered":"2024-03-28T13:11:45.923492+0000","last_unstale":"2024-03-28T13:11:48.311557+0000","last_undegraded":"2024-03-28T13:11:48.311557+0000","last_fullsized":"2024-03-28T13:11:48.311557+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:03:01.456451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899417+0000","last_change":"2024-03-28T13:11:46.936793+0000","last_active":"2024-03-28T13:11:47.899417+0000","last_peered":"2024-03-28T13:11:47.899417+0000","last_clean":"2024-03-28T13:11:47.899417+0000","last_became_active":"2024-03-28T13:11:46.936578+0000","last_became_peered":"2024-03-28T13:11:46.936578+0000","last_unstale":"2024-03-28T13:11:47.899417+0000","last_undegraded":"2024-03-28T13:11:47.899417+0000","last_fullsized":"2024-03-28T13:11:47.899417+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:46:36.058976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.898208+0000","last_change":"2024-03-28T13:11:45.923390+0000","last_active":"2024-03-28T13:11:47.898208+0000","last_peered":"2024-03-28T13:11:47.898208+0000","last_clean":"2024-03-28T13:11:47.898208+0000","last_became_active":"2024-03-28T13:11:45.922999+0000","last_became_peered":"2024-03-28T13:11:45.922999+0000","last_unstale":"2024-03-28T13:11:47.898208+0000","last_undegraded":"2024-03-28T13:11:47.898208+0000","last_fullsized":"2024-03-28T13:11:47.898208+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:42.371923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899695+0000","last_change":"2024-03-28T13:11:46.929903+0000","last_active":"2024-03-28T13:11:47.899695+0000","last_peered":"2024-03-28T13:11:47.899695+0000","last_clean":"2024-03-28T13:11:47.899695+0000","last_became_active":"2024-03-28T13:11:46.929617+0000","last_became_peered":"2024-03-28T13:11:46.929617+0000","last_unstale":"2024-03-28T13:11:47.899695+0000","last_undegraded":"2024-03-28T13:11:47.899695+0000","last_fullsized":"2024-03-28T13:11:47.899695+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:21:36.912694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"49'1","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.903593+0000","last_change":"2024-03-28T13:11:45.914641+0000","last_active":"2024-03-28T13:11:48.903593+0000","last_peered":"2024-03-28T13:11:48.903593+0000","last_clean":"2024-03-28T13:11:48.903593+0000","last_became_active":"2024-03-28T13:11:45.912854+0000","last_became_peered":"2024-03-28T13:11:45.912854+0000","last_unstale":"2024-03-28T13:11:48.903593+0000","last_undegraded":"2024-03-28T13:11:48.903593+0000","last_fullsized":"2024-03-28T13:11:48.903593+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:10:10.237245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312572+0000","last_change":"2024-03-28T13:11:46.934436+0000","last_active":"2024-03-28T13:11:48.312572+0000","last_peered":"2024-03-28T13:11:48.312572+0000","last_clean":"2024-03-28T13:11:48.312572+0000","last_became_active":"2024-03-28T13:11:46.934112+0000","last_became_peered":"2024-03-28T13:11:46.934112+0000","last_unstale":"2024-03-28T13:11:48.312572+0000","last_undegraded":"2024-03-28T13:11:48.312572+0000","last_fullsized":"2024-03-28T13:11:48.312572+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:07.509367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.312017+0000","last_change":"2024-03-28T13:11:45.919514+0000","last_active":"2024-03-28T13:11:48.312017+0000","last_peered":"2024-03-28T13:11:48.312017+0000","last_clean":"2024-03-28T13:11:48.312017+0000","last_became_active":"2024-03-28T13:11:45.917518+0000","last_became_peered":"2024-03-28T13:11:45.917518+0000","last_unstale":"2024-03-28T13:11:48.312017+0000","last_undegraded":"2024-03-28T13:11:48.312017+0000","last_fullsized":"2024-03-28T13:11:48.312017+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:06:17.379887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899748+0000","last_change":"2024-03-28T13:11:46.942950+0000","last_active":"2024-03-28T13:11:47.899748+0000","last_peered":"2024-03-28T13:11:47.899748+0000","last_clean":"2024-03-28T13:11:47.899748+0000","last_became_active":"2024-03-28T13:11:46.942670+0000","last_became_peered":"2024-03-28T13:11:46.942670+0000","last_unstale":"2024-03-28T13:11:47.899748+0000","last_undegraded":"2024-03-28T13:11:47.899748+0000","last_fullsized":"2024-03-28T13:11:47.899748+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:15:09.957549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.896678+0000","last_change":"2024-03-28T13:11:45.925284+0000","last_active":"2024-03-28T13:11:46.896678+0000","last_peered":"2024-03-28T13:11:46.896678+0000","last_clean":"2024-03-28T13:11:46.896678+0000","last_became_active":"2024-03-28T13:11:45.924633+0000","last_became_peered":"2024-03-28T13:11:45.924633+0000","last_unstale":"2024-03-28T13:11:46.896678+0000","last_undegraded":"2024-03-28T13:11:46.896678+0000","last_fullsized":"2024-03-28T13:11:46.896678+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:07:22.061520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311437+0000","last_change":"2024-03-28T13:11:46.932551+0000","last_active":"2024-03-28T13:11:48.311437+0000","last_peered":"2024-03-28T13:11:48.311437+0000","last_clean":"2024-03-28T13:11:48.311437+0000","last_became_active":"2024-03-28T13:11:46.930144+0000","last_became_peered":"2024-03-28T13:11:46.930144+0000","last_unstale":"2024-03-28T13:11:48.311437+0000","last_undegraded":"2024-03-28T13:11:48.311437+0000","last_fullsized":"2024-03-28T13:11:48.311437+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:13:37.250641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.311683+0000","last_change":"2024-03-28T13:11:45.920609+0000","last_active":"2024-03-28T13:11:48.311683+0000","last_peered":"2024-03-28T13:11:48.311683+0000","last_clean":"2024-03-28T13:11:48.311683+0000","last_became_active":"2024-03-28T13:11:45.920311+0000","last_became_peered":"2024-03-28T13:11:45.920311+0000","last_unstale":"2024-03-28T13:11:48.311683+0000","last_undegraded":"2024-03-28T13:11:48.311683+0000","last_fullsized":"2024-03-28T13:11:48.311683+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:46:08.557688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900467+0000","last_change":"2024-03-28T13:11:46.932761+0000","last_active":"2024-03-28T13:11:47.900467+0000","last_peered":"2024-03-28T13:11:47.900467+0000","last_clean":"2024-03-28T13:11:47.900467+0000","last_became_active":"2024-03-28T13:11:46.929710+0000","last_became_peered":"2024-03-28T13:11:46.929710+0000","last_unstale":"2024-03-28T13:11:47.900467+0000","last_undegraded":"2024-03-28T13:11:47.900467+0000","last_fullsized":"2024-03-28T13:11:47.900467+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:28:12.201099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.898783+0000","last_change":"2024-03-28T13:11:45.919674+0000","last_active":"2024-03-28T13:11:47.898783+0000","last_peered":"2024-03-28T13:11:47.898783+0000","last_clean":"2024-03-28T13:11:47.898783+0000","last_became_active":"2024-03-28T13:11:45.919418+0000","last_became_peered":"2024-03-28T13:11:45.919418+0000","last_unstale":"2024-03-28T13:11:47.898783+0000","last_undegraded":"2024-03-28T13:11:47.898783+0000","last_fullsized":"2024-03-28T13:11:47.898783+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:04:50.855872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.896293+0000","last_change":"2024-03-28T13:11:46.940655+0000","last_active":"2024-03-28T13:11:47.896293+0000","last_peered":"2024-03-28T13:11:47.896293+0000","last_clean":"2024-03-28T13:11:47.896293+0000","last_became_active":"2024-03-28T13:11:46.940096+0000","last_became_peered":"2024-03-28T13:11:46.940096+0000","last_unstale":"2024-03-28T13:11:47.896293+0000","last_undegraded":"2024-03-28T13:11:47.896293+0000","last_fullsized":"2024-03-28T13:11:47.896293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:51:42.559863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.310584+0000","last_change":"2024-03-28T13:11:45.917018+0000","last_active":"2024-03-28T13:11:48.310584+0000","last_peered":"2024-03-28T13:11:48.310584+0000","last_clean":"2024-03-28T13:11:48.310584+0000","last_became_active":"2024-03-28T13:11:45.916556+0000","last_became_peered":"2024-03-28T13:11:45.916556+0000","last_unstale":"2024-03-28T13:11:48.310584+0000","last_undegraded":"2024-03-28T13:11:48.310584+0000","last_fullsized":"2024-03-28T13:11:48.310584+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:04:53.337486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:48.309238+0000","last_change":"2024-03-28T13:11:46.925281+0000","last_active":"2024-03-28T13:11:48.309238+0000","last_peered":"2024-03-28T13:11:48.309238+0000","last_clean":"2024-03-28T13:11:48.309238+0000","last_became_active":"2024-03-28T13:11:46.924972+0000","last_became_peered":"2024-03-28T13:11:46.924972+0000","last_unstale":"2024-03-28T13:11:48.309238+0000","last_undegraded":"2024-03-28T13:11:48.309238+0000","last_fullsized":"2024-03-28T13:11:48.309238+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:09:37.323636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-28T13:11:46.897166+0000","last_change":"2024-03-28T13:11:45.920898+0000","last_active":"2024-03-28T13:11:46.897166+0000","last_peered":"2024-03-28T13:11:46.897166+0000","last_clean":"2024-03-28T13:11:46.897166+0000","last_became_active":"2024-03-28T13:11:45.920582+0000","last_became_peered":"2024-03-28T13:11:45.920582+0000","last_unstale":"2024-03-28T13:11:46.897166+0000","last_undegraded":"2024-03-28T13:11:46.897166+0000","last_fullsized":"2024-03-28T13:11:46.897166+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:53:04.504198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.900230+0000","last_change":"2024-03-28T13:11:46.939335+0000","last_active":"2024-03-28T13:11:47.900230+0000","last_peered":"2024-03-28T13:11:47.900230+0000","last_clean":"2024-03-28T13:11:47.900230+0000","last_became_active":"2024-03-28T13:11:46.938778+0000","last_became_peered":"2024-03-28T13:11:46.938778+0000","last_unstale":"2024-03-28T13:11:47.900230+0000","last_undegraded":"2024-03-28T13:11:47.900230+0000","last_fullsized":"2024-03-28T13:11:47.900230+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:45.868495+0000","last_clean_scrub_stamp":"2024-03-28T13:11:45.868495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:03:36.192099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"48'2","reported_seq":16,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.899775+0000","last_change":"2024-03-28T13:11:45.921207+0000","last_active":"2024-03-28T13:11:47.899775+0000","last_peered":"2024-03-28T13:11:47.899775+0000","last_clean":"2024-03-28T13:11:47.899775+0000","last_became_active":"2024-03-28T13:11:45.920198+0000","last_became_peered":"2024-03-28T13:11:45.920198+0000","last_unstale":"2024-03-28T13:11:47.899775+0000","last_undegraded":"2024-03-28T13:11:47.899775+0000","last_fullsized":"2024-03-28T13:11:47.899775+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:11:44.853516+0000","last_clean_scrub_stamp":"2024-03-28T13:11:44.853516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:33:13.397378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":218,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-28T13:11:47.896029+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:11:47.896029+0000","last_peered":"2024-03-28T13:11:47.896029+0000","last_clean":"2024-03-28T13:11:47.896029+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:11:47.896029+0000","last_undegraded":"2024-03-28T13:11:47.896029+0000","last_fullsized":"2024-03-28T13:11:47.896029+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476770,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":577536,"data_stored":232479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476768,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":230957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476768,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1163264,"data_stored":821305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476768,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27588,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344252,"statfs":{"total":96636764160,"available":96608514048,"internally_reserved":0,"allocated":573440,"data_stored":231539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509471,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":561152,"data_stored":225183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509472,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1163264,"data_stored":821257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509472,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":573440,"data_stored":226271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542176,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":557056,"data_stored":224601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:11:52.102 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-28T13:11:52.102 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-28T13:11:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.376+0000 7f13d9c00640 1 Processor -- start 2024-03-28T13:11:52.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.376+0000 7f13d9c00640 1 -- start start 2024-03-28T13:11:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.377+0000 7f13d9c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f13d40b4650 0x7f13d406c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.377+0000 7f13d9c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d40b4a30 0x7f13d40b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.378+0000 7f13d9c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13d415ede0 con 0x7f13d4158580 2024-03-28T13:11:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.378+0000 7f13d9c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f13d4141b30 con 0x7f13d40b4650 2024-03-28T13:11:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.378+0000 7f13d9c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f13d4141d80 con 0x7f13d40b4a30 2024-03-28T13:11:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.378+0000 7f13d3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f13d4158580 0x7f13d41589d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36180/0 (socket says 172.21.15.19:36180) 2024-03-28T13:11:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.378+0000 7f13d3400640 1 -- 172.21.15.19:0/938956591 learned_addr learned my addr 172.21.15.19:0/938956591 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:52.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.378+0000 7f13d2a00640 1 --2- 172.21.15.19:0/938956591 >> v2:172.21.15.19:3300/0 conn(0x7f13d40b4650 0x7f13d406c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.378+0000 7f13d2000640 1 --2- 172.21.15.19:0/938956591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d40b4a30 0x7f13d40b5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.378+0000 7f13d2a00640 1 -- 172.21.15.19:0/938956591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d40b4a30 msgr2=0x7f13d40b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.378+0000 7f13d2a00640 1 --2- 172.21.15.19:0/938956591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d40b4a30 0x7f13d40b5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.378+0000 7f13d2a00640 1 -- 172.21.15.19:0/938956591 >> v1:172.21.15.19:6789/0 conn(0x7f13d4158580 legacy=0x7f13d41589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:52.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.379+0000 7f13d2a00640 1 -- 172.21.15.19:0/938956591 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d4142060 con 0x7f13d40b4650 2024-03-28T13:11:52.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.380+0000 7f13d2a00640 1 --2- 172.21.15.19:0/938956591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13d40b4650 0x7f13d406c680 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f13c40097a0 tx=0x7f13c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac62fa867aec663 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:52.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.381+0000 7f13d1600640 1 -- 172.21.15.19:0/938956591 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4035020 con 0x7f13d40b4650 2024-03-28T13:11:52.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.381+0000 7f13d1600640 1 -- 172.21.15.19:0/938956591 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c4031830 con 0x7f13d40b4650 2024-03-28T13:11:52.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.381+0000 7f13d1600640 1 -- 172.21.15.19:0/938956591 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4031c00 con 0x7f13d40b4650 2024-03-28T13:11:52.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.381+0000 7f13d9c00640 1 -- 172.21.15.19:0/938956591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13d40b4650 msgr2=0x7f13d406c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:52.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.381+0000 7f13d9c00640 1 --2- 172.21.15.19:0/938956591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13d40b4650 0x7f13d406c680 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f13c40097a0 tx=0x7f13c402b8d0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.382+0000 7f13d9c00640 1 -- 172.21.15.19:0/938956591 shutdown_connections 2024-03-28T13:11:52.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.382+0000 7f13d9c00640 1 --2- 172.21.15.19:0/938956591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d40b4a30 0x7f13d40b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.382+0000 7f13d9c00640 1 --2- 172.21.15.19:0/938956591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13d40b4650 0x7f13d406c680 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.382+0000 7f13d9c00640 1 -- 172.21.15.19:0/938956591 >> 172.21.15.19:0/938956591 conn(0x7f13d4067670 msgr2=0x7f13d4067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:52.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.382+0000 7f13d9c00640 1 -- 172.21.15.19:0/938956591 shutdown_connections 2024-03-28T13:11:52.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.382+0000 7f13d9c00640 1 -- 172.21.15.19:0/938956591 wait complete. 2024-03-28T13:11:52.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.383+0000 7f13d9c00640 1 Processor -- start 2024-03-28T13:11:52.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.384+0000 7f13d9c00640 1 -- start start 2024-03-28T13:11:52.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.384+0000 7f13d9c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d40b4650 0x7f13d41f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:52.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.384+0000 7f13d9c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d40b4a30 0x7f13d40cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:52.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.384+0000 7f13d9c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13d4158580 0x7f13d41408a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:52.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.385+0000 7f13d9c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13d4142900 con 0x7f13d4158580 2024-03-28T13:11:52.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.385+0000 7f13d9c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13d4142620 con 0x7f13d40b4a30 2024-03-28T13:11:52.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.385+0000 7f13d9c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f13d406cbc0 con 0x7f13d40b4650 2024-03-28T13:11:52.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.385+0000 7f13d3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13d4158580 0x7f13d41408a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:52.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.385+0000 7f13d3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13d4158580 0x7f13d41408a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45456/0 (socket says 172.21.15.19:45456) 2024-03-28T13:11:52.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.385+0000 7f13d3400640 1 -- 172.21.15.19:0/1485368112 learned_addr learned my addr 172.21.15.19:0/1485368112 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:52.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.385+0000 7f13d3400640 1 -- 172.21.15.19:0/1485368112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d40b4650 msgr2=0x7f13d41f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:52.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.385+0000 7f13d3400640 1 --2- 172.21.15.19:0/1485368112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d40b4650 0x7f13d41f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.385+0000 7f13d3400640 1 -- 172.21.15.19:0/1485368112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d40b4a30 msgr2=0x7f13d40cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:52.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.385+0000 7f13d2000640 1 --2- 172.21.15.19:0/1485368112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d40b4a30 0x7f13d40cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:52.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.385+0000 7f13d3400640 1 --2- 172.21.15.19:0/1485368112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d40b4a30 0x7f13d40cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.385+0000 7f13d3400640 1 -- 172.21.15.19:0/1485368112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d40ccc90 con 0x7f13d4158580 2024-03-28T13:11:52.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.386+0000 7f13d3400640 1 --2- 172.21.15.19:0/1485368112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13d4158580 0x7f13d41408a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f13cc001eb0 tx=0x7f13cc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:52.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.386+0000 7f13caa00640 1 -- 172.21.15.19:0/1485368112 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc017070 con 0x7f13d4158580 2024-03-28T13:11:52.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.386+0000 7f13caa00640 1 -- 172.21.15.19:0/1485368112 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13cc005ce0 con 0x7f13d4158580 2024-03-28T13:11:52.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.386+0000 7f13caa00640 1 -- 172.21.15.19:0/1485368112 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc007310 con 0x7f13d4158580 2024-03-28T13:11:52.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.386+0000 7f13d9c00640 1 -- 172.21.15.19:0/1485368112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d40ccf70 con 0x7f13d4158580 2024-03-28T13:11:52.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.386+0000 7f13d9c00640 1 -- 172.21.15.19:0/1485368112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d40cd530 con 0x7f13d4158580 2024-03-28T13:11:52.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.387+0000 7f13d9c00640 1 -- 172.21.15.19:0/1485368112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d4074f30 con 0x7f13d4158580 2024-03-28T13:11:52.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.388+0000 7f13caa00640 1 -- 172.21.15.19:0/1485368112 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f13cc01a070 con 0x7f13d4158580 2024-03-28T13:11:52.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.390+0000 7f13caa00640 1 --2- 172.21.15.19:0/1485368112 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13b006c530 0x7f13b006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:52.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.390+0000 7f13caa00640 1 -- 172.21.15.19:0/1485368112 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f13cc08aa70 con 0x7f13d4158580 2024-03-28T13:11:52.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.391+0000 7f13d2a00640 1 --2- 172.21.15.19:0/1485368112 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13b006c530 0x7f13b006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:52.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.392+0000 7f13d2a00640 1 --2- 172.21.15.19:0/1485368112 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13b006c530 0x7f13b006e9f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f13c4036000 tx=0x7f13c4030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:52.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.395+0000 7f13caa00640 1 -- 172.21.15.19:0/1485368112 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f13cc059090 con 0x7f13d4158580 2024-03-28T13:11:52.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.673+0000 7f13d9c00640 1 -- 172.21.15.19:0/1485368112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f13d40685c0 con 0x7f13d4158580 2024-03-28T13:11:52.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.677+0000 7f13caa00640 1 -- 172.21.15.19:0/1485368112 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7f13cc058990 con 0x7f13d4158580 2024-03-28T13:11:52.677 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-03-28T13:11:52.677 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q== 2024-03-28T13:11:52.677 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-03-28T13:11:52.677 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-03-28T13:11:52.677 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-28T13:11:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-03-28T13:11:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.680+0000 7f13d9c00640 1 -- 172.21.15.19:0/1485368112 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13b006c530 msgr2=0x7f13b006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.680+0000 7f13d9c00640 1 --2- 172.21.15.19:0/1485368112 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13b006c530 0x7f13b006e9f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f13c4036000 tx=0x7f13c4030000 comp rx=0 tx=0).stop 2024-03-28T13:11:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.680+0000 7f13d9c00640 1 -- 172.21.15.19:0/1485368112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13d4158580 msgr2=0x7f13d41408a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.680+0000 7f13d9c00640 1 --2- 172.21.15.19:0/1485368112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13d4158580 0x7f13d41408a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f13cc001eb0 tx=0x7f13cc00c890 comp rx=0 tx=0).stop 2024-03-28T13:11:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.681+0000 7f13d9c00640 1 -- 172.21.15.19:0/1485368112 shutdown_connections 2024-03-28T13:11:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.681+0000 7f13d9c00640 1 --2- 172.21.15.19:0/1485368112 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13b006c530 0x7f13b006e9f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.681+0000 7f13d9c00640 1 --2- 172.21.15.19:0/1485368112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13d4158580 0x7f13d41408a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.681+0000 7f13d9c00640 1 --2- 172.21.15.19:0/1485368112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d40b4a30 0x7f13d40cc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.681+0000 7f13d9c00640 1 --2- 172.21.15.19:0/1485368112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d40b4650 0x7f13d41f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.681+0000 7f13d9c00640 1 -- 172.21.15.19:0/1485368112 >> 172.21.15.19:0/1485368112 conn(0x7f13d4067670 msgr2=0x7f13d40b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.681+0000 7f13d9c00640 1 -- 172.21.15.19:0/1485368112 shutdown_connections 2024-03-28T13:11:52.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.681+0000 7f13d9c00640 1 -- 172.21.15.19:0/1485368112 wait complete. 2024-03-28T13:11:52.700 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:11:52.861 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-03-28T13:11:52.861+0000 7fad70e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:11:52.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.907+0000 7f27e3800640 1 Processor -- start 2024-03-28T13:11:52.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.907+0000 7f27e3800640 1 -- start start 2024-03-28T13:11:52.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.908+0000 7f27e3800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27dc15c390 0x7f27dc073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:52.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.908+0000 7f27e3800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f27dc0af830 0x7f27dc160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:52.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.908+0000 7f27e3800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27dc15fe80 con 0x7f27dc160b80 2024-03-28T13:11:52.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.908+0000 7f27e3800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f27dc15c770 con 0x7f27dc0af830 2024-03-28T13:11:52.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.908+0000 7f27e3800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f27dc0ad120 con 0x7f27dc15c390 2024-03-28T13:11:52.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.909+0000 7f27dbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f27dc0af830 0x7f27dc160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:52.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.909+0000 7f27e1800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f27dc160b80 0x7f27dc0d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36186/0 (socket says 172.21.15.19:36186) 2024-03-28T13:11:52.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.909+0000 7f27e1800640 1 -- 172.21.15.19:0/1083636681 learned_addr learned my addr 172.21.15.19:0/1083636681 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:52.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.909+0000 7f27e0e00640 1 --2- 172.21.15.19:0/1083636681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27dc15c390 0x7f27dc073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:52.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.909+0000 7f27dbe00640 1 -- 172.21.15.19:0/1083636681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27dc15c390 msgr2=0x7f27dc073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:52.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.909+0000 7f27dbe00640 1 --2- 172.21.15.19:0/1083636681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27dc15c390 0x7f27dc073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.909+0000 7f27dbe00640 1 -- 172.21.15.19:0/1083636681 >> v1:172.21.15.19:6789/0 conn(0x7f27dc160b80 legacy=0x7f27dc0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:52.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.909+0000 7f27dbe00640 1 -- 172.21.15.19:0/1083636681 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27dc0ad400 con 0x7f27dc0af830 2024-03-28T13:11:52.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.910+0000 7f27dbe00640 1 --2- 172.21.15.19:0/1083636681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27dc0af830 0x7f27dc160640 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f27c80097a0 tx=0x7f27c802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33aa8d7935e7df3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:52.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.911+0000 7f27db400640 1 -- 172.21.15.19:0/1083636681 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8035020 con 0x7f27dc0af830 2024-03-28T13:11:52.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.911+0000 7f27db400640 1 -- 172.21.15.19:0/1083636681 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27c8031810 con 0x7f27dc0af830 2024-03-28T13:11:52.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.911+0000 7f27db400640 1 -- 172.21.15.19:0/1083636681 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8031be0 con 0x7f27dc0af830 2024-03-28T13:11:52.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.911+0000 7f27e3800640 1 -- 172.21.15.19:0/1083636681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27dc0af830 msgr2=0x7f27dc160640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:52.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.911+0000 7f27e3800640 1 --2- 172.21.15.19:0/1083636681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27dc0af830 0x7f27dc160640 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f27c80097a0 tx=0x7f27c802b8b0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.912+0000 7f27e3800640 1 -- 172.21.15.19:0/1083636681 shutdown_connections 2024-03-28T13:11:52.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.912+0000 7f27e3800640 1 --2- 172.21.15.19:0/1083636681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27dc0af830 0x7f27dc160640 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.912+0000 7f27e3800640 1 --2- 172.21.15.19:0/1083636681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27dc15c390 0x7f27dc073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.912+0000 7f27e3800640 1 -- 172.21.15.19:0/1083636681 >> 172.21.15.19:0/1083636681 conn(0x7f27dc06b990 msgr2=0x7f27dc06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:52.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.912+0000 7f27e3800640 1 -- 172.21.15.19:0/1083636681 shutdown_connections 2024-03-28T13:11:52.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.912+0000 7f27e3800640 1 -- 172.21.15.19:0/1083636681 wait complete. 2024-03-28T13:11:52.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.913+0000 7f27e3800640 1 Processor -- start 2024-03-28T13:11:52.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.914+0000 7f27e3800640 1 -- start start 2024-03-28T13:11:52.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.914+0000 7f27e3800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27dc0af830 0x7f27dc14a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:52.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.914+0000 7f27e3800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27dc15c390 0x7f27dc14ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:52.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.914+0000 7f27e3800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27dc160b80 0x7f27dc1f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:52.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.914+0000 7f27e3800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc0cf980 con 0x7f27dc15c390 2024-03-28T13:11:52.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.914+0000 7f27e3800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc160120 con 0x7f27dc160b80 2024-03-28T13:11:52.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.914+0000 7f27e3800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f27dc0d0d60 con 0x7f27dc0af830 2024-03-28T13:11:52.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.915+0000 7f27dbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27dc15c390 0x7f27dc14ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:52.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.915+0000 7f27dbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27dc15c390 0x7f27dc14ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45466/0 (socket says 172.21.15.19:45466) 2024-03-28T13:11:52.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.915+0000 7f27dbe00640 1 -- 172.21.15.19:0/2577247073 learned_addr learned my addr 172.21.15.19:0/2577247073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:52.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.915+0000 7f27e0e00640 1 --2- 172.21.15.19:0/2577247073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27dc0af830 0x7f27dc14a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:52.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.915+0000 7f27dbe00640 1 -- 172.21.15.19:0/2577247073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27dc0af830 msgr2=0x7f27dc14a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:52.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.915+0000 7f27dbe00640 1 --2- 172.21.15.19:0/2577247073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27dc0af830 0x7f27dc14a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.915+0000 7f27e1800640 1 --2- 172.21.15.19:0/2577247073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27dc160b80 0x7f27dc1f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:52.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.915+0000 7f27dbe00640 1 -- 172.21.15.19:0/2577247073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27dc160b80 msgr2=0x7f27dc1f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:52.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.915+0000 7f27dbe00640 1 --2- 172.21.15.19:0/2577247073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27dc160b80 0x7f27dc1f3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:52.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.915+0000 7f27dbe00640 1 -- 172.21.15.19:0/2577247073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27dc14ae40 con 0x7f27dc15c390 2024-03-28T13:11:52.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.916+0000 7f27dbe00640 1 --2- 172.21.15.19:0/2577247073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27dc15c390 0x7f27dc14ce40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f27c802bec0 tx=0x7f27c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:52.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.916+0000 7f27d8c00640 1 -- 172.21.15.19:0/2577247073 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8041070 con 0x7f27dc15c390 2024-03-28T13:11:52.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.916+0000 7f27d8c00640 1 -- 172.21.15.19:0/2577247073 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27c803f070 con 0x7f27dc15c390 2024-03-28T13:11:52.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.916+0000 7f27d8c00640 1 -- 172.21.15.19:0/2577247073 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8035040 con 0x7f27dc15c390 2024-03-28T13:11:52.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.916+0000 7f27e3800640 1 -- 172.21.15.19:0/2577247073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27dc14bda0 con 0x7f27dc15c390 2024-03-28T13:11:52.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.916+0000 7f27e3800640 1 -- 172.21.15.19:0/2577247073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27dc14c360 con 0x7f27dc15c390 2024-03-28T13:11:52.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.917+0000 7f27e3800640 1 -- 172.21.15.19:0/2577247073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27dc0685c0 con 0x7f27dc15c390 2024-03-28T13:11:52.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.918+0000 7f27d8c00640 1 -- 172.21.15.19:0/2577247073 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f27c8038020 con 0x7f27dc15c390 2024-03-28T13:11:52.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.919+0000 7f27d8c00640 1 --2- 172.21.15.19:0/2577247073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f27bc06c530 0x7f27bc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:52.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.919+0000 7f27d8c00640 1 -- 172.21.15.19:0/2577247073 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f27c80ad560 con 0x7f27dc15c390 2024-03-28T13:11:52.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.920+0000 7f27e0e00640 1 --2- 172.21.15.19:0/2577247073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f27bc06c530 0x7f27bc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:52.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.921+0000 7f27e0e00640 1 --2- 172.21.15.19:0/2577247073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f27bc06c530 0x7f27bc06e9f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f27d0006fd0 tx=0x7f27d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:52.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:52.924+0000 7f27d8c00640 1 -- 172.21.15.19:0/2577247073 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f27c807bae0 con 0x7f27dc15c390 2024-03-28T13:11:53.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.173+0000 7f27e3800640 1 -- 172.21.15.19:0/2577247073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27dc158540 con 0x7f27dc15c390 2024-03-28T13:11:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.174+0000 7f27d8c00640 1 -- 172.21.15.19:0/2577247073 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7f27c807b3e0 con 0x7f27dc15c390 2024-03-28T13:11:53.174 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:53.174 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2125648249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2125648249},{"type":"v1","addr":"172.21.15.188:6836","nonce":2125648249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/429866161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":429866161},{"type":"v1","addr":"172.21.15.188:6837","nonce":429866161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1422209289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1422209289},{"type":"v1","addr":"172.21.15.19:6834","nonce":1422209289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:11:46.379929+0000","modified":"2024-03-28T13:11:49.470791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5538},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5538":{"gid":5538,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/179338886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":179338886},{"type":"v1","addr":"172.21.15.19:6835","nonce":179338886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5538,"qdb_cluster":[5538]},"id":3}]} 2024-03-28T13:11:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-03-28T13:11:53.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.176+0000 7f27e3800640 1 -- 172.21.15.19:0/2577247073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f27bc06c530 msgr2=0x7f27bc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:53.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.176+0000 7f27e3800640 1 --2- 172.21.15.19:0/2577247073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f27bc06c530 0x7f27bc06e9f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f27d0006fd0 tx=0x7f27d0008040 comp rx=0 tx=0).stop 2024-03-28T13:11:53.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.176+0000 7f27e3800640 1 -- 172.21.15.19:0/2577247073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27dc15c390 msgr2=0x7f27dc14ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:53.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.176+0000 7f27e3800640 1 --2- 172.21.15.19:0/2577247073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27dc15c390 0x7f27dc14ce40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f27c802bec0 tx=0x7f27c80046c0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.177+0000 7f27e3800640 1 -- 172.21.15.19:0/2577247073 shutdown_connections 2024-03-28T13:11:53.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.177+0000 7f27e3800640 1 --2- 172.21.15.19:0/2577247073 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f27bc06c530 0x7f27bc06e9f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.177+0000 7f27e3800640 1 --2- 172.21.15.19:0/2577247073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27dc160b80 0x7f27dc1f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.177+0000 7f27e3800640 1 --2- 172.21.15.19:0/2577247073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27dc15c390 0x7f27dc14ce40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.177+0000 7f27e3800640 1 --2- 172.21.15.19:0/2577247073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27dc0af830 0x7f27dc14a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.177+0000 7f27e3800640 1 -- 172.21.15.19:0/2577247073 >> 172.21.15.19:0/2577247073 conn(0x7f27dc06b990 msgr2=0x7f27dc14b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:53.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.177+0000 7f27e3800640 1 -- 172.21.15.19:0/2577247073 shutdown_connections 2024-03-28T13:11:53.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.177+0000 7f27e3800640 1 -- 172.21.15.19:0/2577247073 wait complete. 2024-03-28T13:11:53.600 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:11:46.379929+0000', 'modified': '2024-03-28T13:11:49.470791+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5538}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5538': {'gid': 5538, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6835/179338886', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 179338886}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 179338886}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5538, 'qdb_cluster': [5538]} 2024-03-28T13:11:53.600 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T13:11:53.600 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-28T13:11:53.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.819+0000 7f4a91800640 1 Processor -- start 2024-03-28T13:11:53.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.820+0000 7f4a91800640 1 -- start start 2024-03-28T13:11:53.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.821+0000 7f4a91800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4a8c15e4a0 0x7f4a8c0b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.821+0000 7f4a91800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a8c0d2310 0x7f4a8c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.821+0000 7f4a91800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a8c0cf530 0x7f4a8c0d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.821+0000 7f4a91800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4a8c0d34e0 con 0x7f4a8c15e4a0 2024-03-28T13:11:53.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.821+0000 7f4a91800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f4a8c15e880 con 0x7f4a8c0cf530 2024-03-28T13:11:53.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.821+0000 7f4a91800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8c0d28d0 con 0x7f4a8c0d2310 2024-03-28T13:11:53.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.822+0000 7f4a8be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4a8c15e4a0 0x7f4a8c0b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:53.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.822+0000 7f4a8be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4a8c15e4a0 0x7f4a8c0b9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45482/0 (socket says 172.21.15.19:45482) 2024-03-28T13:11:53.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.822+0000 7f4a8be00640 1 -- 172.21.15.19:0/3727860700 learned_addr learned my addr 172.21.15.19:0/3727860700 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.822+0000 7f4a8be00640 1 -- 172.21.15.19:0/3727860700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a8c0d2310 msgr2=0x7f4a8c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.822+0000 7f4a8be00640 1 --2- 172.21.15.19:0/3727860700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a8c0d2310 0x7f4a8c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.822+0000 7f4a8be00640 1 -- 172.21.15.19:0/3727860700 >> v2:172.21.15.188:3300/0 conn(0x7f4a8c0cf530 msgr2=0x7f4a8c0d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.822+0000 7f4a8be00640 1 --2- 172.21.15.19:0/3727860700 >> v2:172.21.15.188:3300/0 conn(0x7f4a8c0cf530 0x7f4a8c0d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.822+0000 7f4a8be00640 1 -- 172.21.15.19:0/3727860700 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c0d2bb0 con 0x7f4a8c15e4a0 2024-03-28T13:11:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.823+0000 7f4a8be00640 1 --2- 172.21.15.19:0/3727860700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c15e4a0 0x7f4a8c0b9c60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0d50f0 tx=0x7f4a7c02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2182c7c59c3555c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.824+0000 7f4a8aa00640 1 -- 172.21.15.19:0/3727860700 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c035020 con 0x7f4a8c15e4a0 2024-03-28T13:11:53.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.824+0000 7f4a8aa00640 1 -- 172.21.15.19:0/3727860700 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7c031880 con 0x7f4a8c15e4a0 2024-03-28T13:11:53.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.824+0000 7f4a8aa00640 1 -- 172.21.15.19:0/3727860700 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c031c50 con 0x7f4a8c15e4a0 2024-03-28T13:11:53.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.824+0000 7f4a91800640 1 -- 172.21.15.19:0/3727860700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c15e4a0 msgr2=0x7f4a8c0b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:53.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.824+0000 7f4a91800640 1 --2- 172.21.15.19:0/3727860700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c15e4a0 0x7f4a8c0b9c60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0d50f0 tx=0x7f4a7c02b920 comp rx=0 tx=0).stop 2024-03-28T13:11:53.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.824+0000 7f4a91800640 1 -- 172.21.15.19:0/3727860700 shutdown_connections 2024-03-28T13:11:53.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.824+0000 7f4a91800640 1 --2- 172.21.15.19:0/3727860700 >> v2:172.21.15.188:3300/0 conn(0x7f4a8c0cf530 0x7f4a8c0d3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.824+0000 7f4a91800640 1 --2- 172.21.15.19:0/3727860700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a8c0d2310 0x7f4a8c0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.824+0000 7f4a91800640 1 --2- 172.21.15.19:0/3727860700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c15e4a0 0x7f4a8c0b9c60 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.824+0000 7f4a91800640 1 -- 172.21.15.19:0/3727860700 >> 172.21.15.19:0/3727860700 conn(0x7f4a8c0676d0 msgr2=0x7f4a8c067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:53.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.824+0000 7f4a91800640 1 -- 172.21.15.19:0/3727860700 shutdown_connections 2024-03-28T13:11:53.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.825+0000 7f4a91800640 1 -- 172.21.15.19:0/3727860700 wait complete. 2024-03-28T13:11:53.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.826+0000 7f4a91800640 1 Processor -- start 2024-03-28T13:11:53.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.826+0000 7f4a91800640 1 -- start start 2024-03-28T13:11:53.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.827+0000 7f4a91800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a8c0cf530 0x7f4a8c150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.827+0000 7f4a91800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a8c0d2310 0x7f4a8c0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.827+0000 7f4a91800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c15e4a0 0x7f4a8c0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.827+0000 7f4a91800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c0d3970 con 0x7f4a8c15e4a0 2024-03-28T13:11:53.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.827+0000 7f4a91800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c0d3170 con 0x7f4a8c0d2310 2024-03-28T13:11:53.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.827+0000 7f4a91800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8c0ba1a0 con 0x7f4a8c0cf530 2024-03-28T13:11:53.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.827+0000 7f4a90e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c15e4a0 0x7f4a8c0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:53.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.827+0000 7f4a90e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c15e4a0 0x7f4a8c0c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45494/0 (socket says 172.21.15.19:45494) 2024-03-28T13:11:53.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.827+0000 7f4a90e00640 1 -- 172.21.15.19:0/594551309 learned_addr learned my addr 172.21.15.19:0/594551309 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:53.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.828+0000 7f4a90e00640 1 -- 172.21.15.19:0/594551309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a8c0cf530 msgr2=0x7f4a8c150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:53.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.828+0000 7f4a90e00640 1 --2- 172.21.15.19:0/594551309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a8c0cf530 0x7f4a8c150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.828+0000 7f4a90e00640 1 -- 172.21.15.19:0/594551309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a8c0d2310 msgr2=0x7f4a8c0c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:53.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.828+0000 7f4a90e00640 1 --2- 172.21.15.19:0/594551309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a8c0d2310 0x7f4a8c0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.828+0000 7f4a90e00640 1 -- 172.21.15.19:0/594551309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8c0c7f60 con 0x7f4a8c15e4a0 2024-03-28T13:11:53.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.828+0000 7f4a90e00640 1 --2- 172.21.15.19:0/594551309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c15e4a0 0x7f4a8c0c9df0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400b9a0 tx=0x7f4a8400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:53.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.828+0000 7f4a83e00640 1 -- 172.21.15.19:0/594551309 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84019070 con 0x7f4a8c15e4a0 2024-03-28T13:11:53.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.829+0000 7f4a83e00640 1 -- 172.21.15.19:0/594551309 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a8400cbc0 con 0x7f4a8c15e4a0 2024-03-28T13:11:53.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.829+0000 7f4a83e00640 1 -- 172.21.15.19:0/594551309 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a84014440 con 0x7f4a8c15e4a0 2024-03-28T13:11:53.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.829+0000 7f4a91800640 1 -- 172.21.15.19:0/594551309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8c0c8240 con 0x7f4a8c15e4a0 2024-03-28T13:11:53.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.829+0000 7f4a91800640 1 -- 172.21.15.19:0/594551309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a8c0c8800 con 0x7f4a8c15e4a0 2024-03-28T13:11:53.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.831+0000 7f4a83e00640 1 -- 172.21.15.19:0/594551309 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4a8401c070 con 0x7f4a8c15e4a0 2024-03-28T13:11:53.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.832+0000 7f4a83e00640 1 --2- 172.21.15.19:0/594551309 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4a6c06c560 0x7f4a6c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.832+0000 7f4a83e00640 1 -- 172.21.15.19:0/594551309 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f4a8408b4c0 con 0x7f4a8c15e4a0 2024-03-28T13:11:53.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.832+0000 7f4a95ba0740 1 Processor -- start 2024-03-28T13:11:53.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.832+0000 7f4a8be00640 1 --2- 172.21.15.19:0/594551309 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4a6c06c560 0x7f4a6c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:53.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.832+0000 7f4a95ba0740 1 -- start start 2024-03-28T13:11:53.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.833+0000 7f4a95ba0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fca151d0 0x5618fc9e8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.833+0000 7f4a95ba0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca155b0 0x5618fca17f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.833+0000 7f4a95ba0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fc9e86b0 0x5618fca184a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.833+0000 7f4a95ba0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5618fc9af920 con 0x5618fca155b0 2024-03-28T13:11:53.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.833+0000 7f4a95ba0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5618fc8fffa0 con 0x5618fc9e86b0 2024-03-28T13:11:53.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.833+0000 7f4a95ba0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5618fc951f90 con 0x5618fca151d0 2024-03-28T13:11:53.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.833+0000 7f4a8be00640 1 --2- 172.21.15.19:0/594551309 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4a6c06c560 0x7f4a6c06ea20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c036000 tx=0x7f4a7c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:53.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.834+0000 7f4a8b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca155b0 0x5618fca17f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:53.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.834+0000 7f4a8b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca155b0 0x5618fca17f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45508/0 (socket says 172.21.15.19:45508) 2024-03-28T13:11:53.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.834+0000 7f4a8b400640 1 -- 172.21.15.19:0/785611742 learned_addr learned my addr 172.21.15.19:0/785611742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:53.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.834+0000 7f4a8be00640 1 --2- 172.21.15.19:0/785611742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fca151d0 0x5618fc9e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:53.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.834+0000 7f4a90e00640 1 --2- 172.21.15.19:0/785611742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fc9e86b0 0x5618fca184a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:53.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.834+0000 7f4a8b400640 1 -- 172.21.15.19:0/785611742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fca151d0 msgr2=0x5618fc9e8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:53.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.834+0000 7f4a8b400640 1 --2- 172.21.15.19:0/785611742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fca151d0 0x5618fc9e8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.834+0000 7f4a8b400640 1 -- 172.21.15.19:0/785611742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fc9e86b0 msgr2=0x5618fca184a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:53.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.834+0000 7f4a8b400640 1 --2- 172.21.15.19:0/785611742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fc9e86b0 0x5618fca184a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.834+0000 7f4a8b400640 1 -- 172.21.15.19:0/785611742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618fc6662d0 con 0x5618fca155b0 2024-03-28T13:11:53.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.834+0000 7f4a8b400640 1 --2- 172.21.15.19:0/785611742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca155b0 0x5618fca17f60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009780 tx=0x7f4a78009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6972ce76bad80629 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.835+0000 7f4a80c00640 1 -- 172.21.15.19:0/785611742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78015020 con 0x5618fca155b0 2024-03-28T13:11:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.835+0000 7f4a80c00640 1 -- 172.21.15.19:0/785611742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7800bad0 con 0x5618fca155b0 2024-03-28T13:11:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.835+0000 7f4a80c00640 1 -- 172.21.15.19:0/785611742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78011420 con 0x5618fca155b0 2024-03-28T13:11:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.835+0000 7f4a95ba0740 1 -- 172.21.15.19:0/785611742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca155b0 msgr2=0x5618fca17f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.835+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/785611742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca155b0 0x5618fca17f60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009780 tx=0x7f4a78009c40 comp rx=0 tx=0).stop 2024-03-28T13:11:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.835+0000 7f4a95ba0740 1 -- 172.21.15.19:0/785611742 shutdown_connections 2024-03-28T13:11:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.835+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/785611742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fc9e86b0 0x5618fca184a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.835+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/785611742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca155b0 0x5618fca17f60 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.835+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/785611742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fca151d0 0x5618fc9e8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.835+0000 7f4a95ba0740 1 -- 172.21.15.19:0/785611742 >> 172.21.15.19:0/785611742 conn(0x5618fc9ea3e0 msgr2=0x5618fca32f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.835+0000 7f4a95ba0740 1 -- 172.21.15.19:0/785611742 shutdown_connections 2024-03-28T13:11:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.835+0000 7f4a95ba0740 1 -- 172.21.15.19:0/785611742 wait complete. 2024-03-28T13:11:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.836+0000 7f4a95ba0740 1 -- start start 2024-03-28T13:11:53.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.836+0000 7f4a95ba0740 1 Processor -- start 2024-03-28T13:11:53.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.837+0000 7f4a95ba0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca1cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.837+0000 7f4a95ba0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fca155b0 0x5618fca02000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.837+0000 7f4a95ba0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fc9fff00 0x5618fca438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.837+0000 7f4a95ba0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5618fc94df40 con 0x5618fca151d0 2024-03-28T13:11:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.837+0000 7f4a95ba0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5618fc8fa650 con 0x5618fca155b0 2024-03-28T13:11:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.837+0000 7f4a95ba0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5618fc90da10 con 0x5618fc9fff00 2024-03-28T13:11:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.837+0000 7f4a95ba0740 10 client.0 _ll_register_callbacks cb 0x5618fc94cff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:11:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.838+0000 7f4a8be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca1cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.838+0000 7f4a8be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca1cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45524/0 (socket says 172.21.15.19:45524) 2024-03-28T13:11:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.838+0000 7f4a8be00640 1 -- 172.21.15.19:0/292550538 learned_addr learned my addr 172.21.15.19:0/292550538 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.838+0000 7f4a8be00640 1 -- 172.21.15.19:0/292550538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fc9fff00 msgr2=0x5618fca438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.838+0000 7f4a8be00640 1 --2- 172.21.15.19:0/292550538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fc9fff00 0x5618fca438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.838+0000 7f4a8be00640 1 -- 172.21.15.19:0/292550538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fca155b0 msgr2=0x5618fca02000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.838+0000 7f4a8be00640 1 --2- 172.21.15.19:0/292550538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fca155b0 0x5618fca02000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.838+0000 7f4a8be00640 1 -- 172.21.15.19:0/292550538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618fc7306b0 con 0x5618fca151d0 2024-03-28T13:11:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.838+0000 7f4a8be00640 1 --2- 172.21.15.19:0/292550538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca1cc90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009930 tx=0x7f4a7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183436f93739eb1a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:53.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.839+0000 7f4a75600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:11:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.839+0000 7f4a75600640 1 -- 172.21.15.19:0/292550538 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c052060 con 0x5618fca151d0 2024-03-28T13:11:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.839+0000 7f4a95ba0740 1 -- 172.21.15.19:0/292550538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a7800bad0 con 0x5618fca151d0 2024-03-28T13:11:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.839+0000 7f4a75600640 1 -- 172.21.15.19:0/292550538 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f4a7c044600 con 0x5618fca151d0 2024-03-28T13:11:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.839+0000 7f4a75600640 1 -- 172.21.15.19:0/292550538 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a7c044800 con 0x5618fca151d0 2024-03-28T13:11:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.839+0000 7f4a95ba0740 10 client.5661 fetch_fsmap learned FSMap version 26 2024-03-28T13:11:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.839+0000 7f4a95ba0740 1 -- 172.21.15.19:0/292550538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5618fca442f0 con 0x5618fca151d0 2024-03-28T13:11:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.839+0000 7f4a75600640 1 -- 172.21.15.19:0/292550538 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7800bad0 con 0x5618fca151d0 2024-03-28T13:11:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.840+0000 7f4a75600640 1 -- 172.21.15.19:0/292550538 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f4a7c0449a0 con 0x5618fca151d0 2024-03-28T13:11:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.840+0000 7f4a95ba0740 10 client.5661 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T13:11:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.840+0000 7f4a95ba0740 10 client.5661 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:11:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.840+0000 7f4a95ba0740 20 client.5661 populate_metadata read hostname 'smithi019' 2024-03-28T13:11:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.840+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/292550538 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x5618fca16f40 0x5618fca33350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.840+0000 7f4a95ba0740 4 client.5661 mds_command: new command op to 5538 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:11:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.840+0000 7f4a95ba0740 1 -- 172.21.15.19:0/292550538 --> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618fc95f4d0 con 0x5618fca16f40 2024-03-28T13:11:53.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.840+0000 7f4a8b400640 1 --2- 172.21.15.19:0/292550538 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x5618fca16f40 0x5618fca33350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.840+0000 7f4a8b400640 1 --2- 172.21.15.19:0/292550538 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x5618fca16f40 0x5618fca33350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f3c4b4121f10999 server_cookie=c8765a9582940dd4 in_seq=0 out_seq=0 2024-03-28T13:11:53.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.840+0000 7f4a75600640 10 client.5661 ms_handle_connect on v2:172.21.15.19:6833/179338886 2024-03-28T13:11:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.841+0000 7f4a75600640 1 -- 172.21.15.19:0/292550538 <== mds.0 v2:172.21.15.19:6833/179338886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5618fc95f4d0 con 0x5618fca16f40 2024-03-28T13:11:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.841+0000 7f4a75600640 10 client.5661 handle_command_reply: tid=0 2024-03-28T13:11:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.841+0000 7f4a95ba0740 1 client.5661 shutdown 2024-03-28T13:11:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.842+0000 7f4a95ba0740 1 -- 172.21.15.19:0/292550538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 msgr2=0x5618fca1cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.842+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/292550538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca1cc90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009930 tx=0x7f4a7c004730 comp rx=0 tx=0).stop 2024-03-28T13:11:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.842+0000 7f4a95ba0740 1 -- 172.21.15.19:0/292550538 shutdown_connections 2024-03-28T13:11:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.842+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/292550538 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x5618fca16f40 0x5618fca33350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.842+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/292550538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fc9fff00 0x5618fca438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.842+0000 7f4a75600640 0 client.5661 ms_handle_reset on v2:172.21.15.19:6833/179338886 2024-03-28T13:11:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.842+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/292550538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fca155b0 0x5618fca02000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.842+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/292550538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca1cc90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.842+0000 7f4a95ba0740 1 -- 172.21.15.19:0/292550538 >> 172.21.15.19:0/292550538 conn(0x5618fc9e86b0 msgr2=0x5618fc9e7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:53.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.842+0000 7f4a95ba0740 1 -- 172.21.15.19:0/292550538 shutdown_connections 2024-03-28T13:11:53.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.842+0000 7f4a95ba0740 1 -- 172.21.15.19:0/292550538 wait complete. 2024-03-28T13:11:53.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.842+0000 7f4a95ba0740 20 client.5661 trim_cache size 0 max 16384 2024-03-28T13:11:53.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.861+0000 7f4a95ba0740 1 Processor -- start 2024-03-28T13:11:53.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.861+0000 7f4a95ba0740 1 -- start start 2024-03-28T13:11:53.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.862+0000 7f4a95ba0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fc9fff00 0x5618fca136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.862+0000 7f4a95ba0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca1bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.862+0000 7f4a95ba0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fca155b0 0x5618fc9ff210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.862+0000 7f4a95ba0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c1ffac0 con 0x5618fca151d0 2024-03-28T13:11:53.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.862+0000 7f4a95ba0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a8c15f860 con 0x5618fc9fff00 2024-03-28T13:11:53.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.862+0000 7f4a95ba0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4a8c1f80d0 con 0x5618fca155b0 2024-03-28T13:11:53.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.862+0000 7f4a8b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca1bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:53.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.863+0000 7f4a8b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca1bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45534/0 (socket says 172.21.15.19:45534) 2024-03-28T13:11:53.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.863+0000 7f4a8b400640 1 -- 172.21.15.19:0/3628092465 learned_addr learned my addr 172.21.15.19:0/3628092465 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:53.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.863+0000 7f4a8b400640 1 -- 172.21.15.19:0/3628092465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fca155b0 msgr2=0x5618fc9ff210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:53.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.863+0000 7f4a8b400640 1 --2- 172.21.15.19:0/3628092465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fca155b0 0x5618fc9ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.863+0000 7f4a8b400640 1 -- 172.21.15.19:0/3628092465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fc9fff00 msgr2=0x5618fca136a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:53.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.863+0000 7f4a8b400640 1 --2- 172.21.15.19:0/3628092465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fc9fff00 0x5618fca136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.863+0000 7f4a8b400640 1 -- 172.21.15.19:0/3628092465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618fca149d0 con 0x5618fca151d0 2024-03-28T13:11:53.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.863+0000 7f4a8b400640 1 --2- 172.21.15.19:0/3628092465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca1bec0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009750 tx=0x7f4a78010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d699004cd364e9df server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:53.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.863+0000 7f4a77e00640 1 -- 172.21.15.19:0/3628092465 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78029070 con 0x5618fca151d0 2024-03-28T13:11:53.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.863+0000 7f4a77e00640 1 -- 172.21.15.19:0/3628092465 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a78024bd0 con 0x5618fca151d0 2024-03-28T13:11:53.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.863+0000 7f4a77e00640 1 -- 172.21.15.19:0/3628092465 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78015040 con 0x5618fca151d0 2024-03-28T13:11:53.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.864+0000 7f4a95ba0740 1 -- 172.21.15.19:0/3628092465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 msgr2=0x5618fca1bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:53.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.864+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/3628092465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca1bec0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4a78009750 tx=0x7f4a78010000 comp rx=0 tx=0).stop 2024-03-28T13:11:53.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.864+0000 7f4a95ba0740 1 -- 172.21.15.19:0/3628092465 shutdown_connections 2024-03-28T13:11:53.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.864+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/3628092465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fca155b0 0x5618fc9ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.864+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/3628092465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca1bec0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.864+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/3628092465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fc9fff00 0x5618fca136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.864+0000 7f4a95ba0740 1 -- 172.21.15.19:0/3628092465 >> 172.21.15.19:0/3628092465 conn(0x5618fc9e86b0 msgr2=0x5618fca1caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:53.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.864+0000 7f4a95ba0740 1 -- 172.21.15.19:0/3628092465 shutdown_connections 2024-03-28T13:11:53.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.864+0000 7f4a95ba0740 1 -- 172.21.15.19:0/3628092465 wait complete. 2024-03-28T13:11:53.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.865+0000 7f4a95ba0740 1 -- start start 2024-03-28T13:11:53.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.865+0000 7f4a95ba0740 1 Processor -- start 2024-03-28T13:11:53.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.866+0000 7f4a95ba0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fc9fff00 0x5618fca9bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.866+0000 7f4a95ba0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca9b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.866+0000 7f4a95ba0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fca155b0 0x5618fca13410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.866+0000 7f4a95ba0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5618fca16dc0 con 0x5618fca151d0 2024-03-28T13:11:53.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.866+0000 7f4a95ba0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5618fc94df40 con 0x5618fca155b0 2024-03-28T13:11:53.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.866+0000 7f4a95ba0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5618fca17400 con 0x5618fc9fff00 2024-03-28T13:11:53.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.866+0000 7f4a95ba0740 10 client.0 _ll_register_callbacks cb 0x7f4a7c0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:11:53.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.866+0000 7f4a8b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca9b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:53.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.866+0000 7f4a8b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca9b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45550/0 (socket says 172.21.15.19:45550) 2024-03-28T13:11:53.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.866+0000 7f4a8b400640 1 -- 172.21.15.19:0/1509426201 learned_addr learned my addr 172.21.15.19:0/1509426201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.867+0000 7f4a8b400640 1 -- 172.21.15.19:0/1509426201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fc9fff00 msgr2=0x5618fca9bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.867+0000 7f4a8b400640 1 --2- 172.21.15.19:0/1509426201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fc9fff00 0x5618fca9bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.867+0000 7f4a8b400640 1 -- 172.21.15.19:0/1509426201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fca155b0 msgr2=0x5618fca13410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.867+0000 7f4a90e00640 1 --2- 172.21.15.19:0/1509426201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fca155b0 0x5618fca13410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.867+0000 7f4a8b400640 1 --2- 172.21.15.19:0/1509426201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fca155b0 0x5618fca13410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.867+0000 7f4a8b400640 1 -- 172.21.15.19:0/1509426201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618fca53180 con 0x5618fca151d0 2024-03-28T13:11:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.867+0000 7f4a8b400640 1 --2- 172.21.15.19:0/1509426201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca9b8d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4a78027040 tx=0x7f4a78004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab70a15631454f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.867+0000 7f4a75600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:11:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.867+0000 7f4a75600640 1 -- 172.21.15.19:0/1509426201 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a78034060 con 0x5618fca151d0 2024-03-28T13:11:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.867+0000 7f4a75600640 1 -- 172.21.15.19:0/1509426201 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f4a7800bb90 con 0x5618fca151d0 2024-03-28T13:11:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.867+0000 7f4a95ba0740 1 -- 172.21.15.19:0/1509426201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a78024bd0 con 0x5618fca151d0 2024-03-28T13:11:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.867+0000 7f4a75600640 1 -- 172.21.15.19:0/1509426201 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4a78029070 con 0x5618fca151d0 2024-03-28T13:11:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.867+0000 7f4a95ba0740 10 client.5667 fetch_fsmap learned FSMap version 26 2024-03-28T13:11:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.867+0000 7f4a95ba0740 1 -- 172.21.15.19:0/1509426201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x5618fca9c5a0 con 0x5618fca151d0 2024-03-28T13:11:53.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.868+0000 7f4a75600640 1 -- 172.21.15.19:0/1509426201 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a78024bd0 con 0x5618fca151d0 2024-03-28T13:11:53.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.868+0000 7f4a75600640 1 -- 172.21.15.19:0/1509426201 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f4a78024410 con 0x5618fca151d0 2024-03-28T13:11:53.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.868+0000 7f4a95ba0740 10 client.5667 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T13:11:53.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.868+0000 7f4a95ba0740 10 client.5667 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:11:53.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.868+0000 7f4a95ba0740 20 client.5667 populate_metadata read hostname 'smithi019' 2024-03-28T13:11:53.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.868+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/1509426201 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x5618fc9ff320 0x5618fc9ff700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:53.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.868+0000 7f4a95ba0740 4 client.5667 mds_command: new command op to 5538 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T13:11:53.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.868+0000 7f4a95ba0740 1 -- 172.21.15.19:0/1509426201 --> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] -- command(tid 0: {"prefix": "status"}) -- 0x5618fc959820 con 0x5618fc9ff320 2024-03-28T13:11:53.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.868+0000 7f4a8be00640 1 --2- 172.21.15.19:0/1509426201 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x5618fc9ff320 0x5618fc9ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:53.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a8be00640 1 --2- 172.21.15.19:0/1509426201 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x5618fc9ff320 0x5618fc9ff700 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12928e8fdd513a54 server_cookie=3d1c86d2433d7255 in_seq=0 out_seq=0 2024-03-28T13:11:53.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a75600640 10 client.5667 ms_handle_connect on v2:172.21.15.19:6833/179338886 2024-03-28T13:11:53.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a75600640 1 -- 172.21.15.19:0/1509426201 <== mds.0 v2:172.21.15.19:6833/179338886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5618fc959820 con 0x5618fc9ff320 2024-03-28T13:11:53.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a75600640 10 client.5667 handle_command_reply: tid=0 2024-03-28T13:11:53.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a95ba0740 1 client.5667 shutdown 2024-03-28T13:11:53.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a95ba0740 1 -- 172.21.15.19:0/1509426201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 msgr2=0x5618fca9b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:53.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/1509426201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca9b8d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4a78027040 tx=0x7f4a78004840 comp rx=0 tx=0).stop 2024-03-28T13:11:53.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a95ba0740 1 -- 172.21.15.19:0/1509426201 shutdown_connections 2024-03-28T13:11:53.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/1509426201 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x5618fc9ff320 0x5618fc9ff700 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/1509426201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5618fca155b0 0x5618fca13410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/1509426201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5618fca151d0 0x5618fca9b8d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a95ba0740 1 --2- 172.21.15.19:0/1509426201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5618fc9fff00 0x5618fca9bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a95ba0740 1 -- 172.21.15.19:0/1509426201 >> 172.21.15.19:0/1509426201 conn(0x5618fc9e86b0 msgr2=0x5618fca1caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:53.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a75600640 0 client.5667 ms_handle_reset on v2:172.21.15.19:6833/179338886 2024-03-28T13:11:53.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a95ba0740 1 -- 172.21.15.19:0/1509426201 shutdown_connections 2024-03-28T13:11:53.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.869+0000 7f4a95ba0740 1 -- 172.21.15.19:0/1509426201 wait complete. 2024-03-28T13:11:53.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.870+0000 7f4a95ba0740 20 client.5667 trim_cache size 0 max 16384 2024-03-28T13:11:53.873 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:11:53.873 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "e6b569d9-f2ce-49dc-bbe7-c0ab33080203", 2024-03-28T13:11:53.873 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-03-28T13:11:53.873 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 5538, 2024-03-28T13:11:53.873 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-03-28T13:11:53.873 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-03-28T13:11:53.873 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-03-28T13:11:53.873 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 6.9801824960000003, 2024-03-28T13:11:53.873 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 26, 2024-03-28T13:11:53.873 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 49, 2024-03-28T13:11:53.873 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 49, 2024-03-28T13:11:53.873 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 12.310329898999999 2024-03-28T13:11:53.873 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:11:53.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.870+0000 7f4a91800640 1 -- 172.21.15.19:0/594551309 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4a6c06c560 msgr2=0x7f4a6c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:53.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.870+0000 7f4a91800640 1 --2- 172.21.15.19:0/594551309 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4a6c06c560 0x7f4a6c06ea20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c036000 tx=0x7f4a7c030000 comp rx=0 tx=0).stop 2024-03-28T13:11:53.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.870+0000 7f4a91800640 1 -- 172.21.15.19:0/594551309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c15e4a0 msgr2=0x7f4a8c0c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:53.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.870+0000 7f4a91800640 1 --2- 172.21.15.19:0/594551309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c15e4a0 0x7f4a8c0c9df0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400b9a0 tx=0x7f4a8400be60 comp rx=0 tx=0).stop 2024-03-28T13:11:53.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.870+0000 7f4a91800640 1 -- 172.21.15.19:0/594551309 shutdown_connections 2024-03-28T13:11:53.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.870+0000 7f4a91800640 1 --2- 172.21.15.19:0/594551309 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4a6c06c560 0x7f4a6c06ea20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.870+0000 7f4a91800640 1 --2- 172.21.15.19:0/594551309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a8c15e4a0 0x7f4a8c0c9df0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.870+0000 7f4a91800640 1 --2- 172.21.15.19:0/594551309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a8c0d2310 0x7f4a8c0c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.870+0000 7f4a91800640 1 --2- 172.21.15.19:0/594551309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a8c0cf530 0x7f4a8c150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:53.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.870+0000 7f4a91800640 1 -- 172.21.15.19:0/594551309 >> 172.21.15.19:0/594551309 conn(0x7f4a8c0676d0 msgr2=0x7f4a8c0b73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:53.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.870+0000 7f4a91800640 1 -- 172.21.15.19:0/594551309 shutdown_connections 2024-03-28T13:11:53.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:53.870+0000 7f4a91800640 1 -- 172.21.15.19:0/594551309 wait complete. 2024-03-28T13:11:53.889 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:11:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.092+0000 7f3d32c00640 1 Processor -- start 2024-03-28T13:11:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.093+0000 7f3d32c00640 1 -- start start 2024-03-28T13:11:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.094+0000 7f3d32c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c0d1380 0x7f3d2c0d17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:54.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.094+0000 7f3d32c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d2c0d2fd0 con 0x7f3d2c0d3a70 2024-03-28T13:11:54.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.094+0000 7f3d32c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d2c0cfa70 con 0x7f3d2c15e420 2024-03-28T13:11:54.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.094+0000 7f3d32c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3d2c0d1d20 con 0x7f3d2c0d1380 2024-03-28T13:11:54.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.095+0000 7f3d32200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3d2c0d3a70 0x7f3d2c0d3ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36198/0 (socket says 172.21.15.19:36198) 2024-03-28T13:11:54.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.095+0000 7f3d32200640 1 -- 172.21.15.19:0/2963695847 learned_addr learned my addr 172.21.15.19:0/2963695847 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.095+0000 7f3d30e00640 1 --2- 172.21.15.19:0/2963695847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c0d1380 0x7f3d2c0d17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.095+0000 7f3d2be00640 1 -- 172.21.15.19:0/2963695847 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 745216374 0 0) 0x7f3d2c0d2fd0 con 0x7f3d2c0d3a70 2024-03-28T13:11:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.095+0000 7f3d2be00640 1 -- 172.21.15.19:0/2963695847 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d14003520 con 0x7f3d2c0d3a70 2024-03-28T13:11:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.095+0000 7f3d2be00640 1 -- 172.21.15.19:0/2963695847 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2882015613 0 0) 0x7f3d14003520 con 0x7f3d2c0d3a70 2024-03-28T13:11:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.095+0000 7f3d2be00640 1 -- 172.21.15.19:0/2963695847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c0d1380 msgr2=0x7f3d2c0d17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.095+0000 7f3d2be00640 1 --2- 172.21.15.19:0/2963695847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c0d1380 0x7f3d2c0d17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.095+0000 7f3d2be00640 1 -- 172.21.15.19:0/2963695847 >> v1:172.21.15.188:6789/0 conn(0x7f3d2c15e420 legacy=0x7f3d2c0b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.095+0000 7f3d2be00640 1 -- 172.21.15.19:0/2963695847 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2c0cf790 con 0x7f3d2c0d3a70 2024-03-28T13:11:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.095+0000 7f3d2be00640 1 -- 172.21.15.19:0/2963695847 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f3d240019f0 con 0x7f3d2c0d3a70 2024-03-28T13:11:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.095+0000 7f3d30e00640 1 --2- 172.21.15.19:0/2963695847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c0d1380 0x7f3d2c0d17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:11:54.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.096+0000 7f3d2be00640 1 -- 172.21.15.19:0/2963695847 >> v1:172.21.15.19:6789/0 conn(0x7f3d2c0d3a70 legacy=0x7f3d2c0d3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.096+0000 7f3d2be00640 1 --2- 172.21.15.19:0/2963695847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d140039e0 0x7f3d14003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.096+0000 7f3d2be00640 1 --2- 172.21.15.19:0/2963695847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c0d1380 0x7f3d14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.096+0000 7f3d2be00640 1 --2- 172.21.15.19:0/2963695847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d14004d40 0x7f3d140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.096+0000 7f3d2be00640 1 -- 172.21.15.19:0/2963695847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1400bb60 con 0x7f3d140039e0 2024-03-28T13:11:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.096+0000 7f3d2be00640 1 -- 172.21.15.19:0/2963695847 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d1400bd10 con 0x7f3d14004d40 2024-03-28T13:11:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.096+0000 7f3d2be00640 1 -- 172.21.15.19:0/2963695847 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3d1400bf10 con 0x7f3d2c0d1380 2024-03-28T13:11:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.096+0000 7f3d31800640 1 --2- 172.21.15.19:0/2963695847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d140039e0 0x7f3d14003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.096+0000 7f3d30e00640 1 --2- 172.21.15.19:0/2963695847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c0d1380 0x7f3d14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.096+0000 7f3d32200640 1 --2- 172.21.15.19:0/2963695847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d14004d40 0x7f3d140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.097+0000 7f3d31800640 1 -- 172.21.15.19:0/2963695847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c0d1380 msgr2=0x7f3d14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.097+0000 7f3d31800640 1 --2- 172.21.15.19:0/2963695847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c0d1380 0x7f3d14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.097+0000 7f3d31800640 1 -- 172.21.15.19:0/2963695847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d14004d40 msgr2=0x7f3d140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.097+0000 7f3d31800640 1 --2- 172.21.15.19:0/2963695847 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d14004d40 0x7f3d140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.097+0000 7f3d31800640 1 -- 172.21.15.19:0/2963695847 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d14003520 con 0x7f3d140039e0 2024-03-28T13:11:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.098+0000 7f3d31800640 1 --2- 172.21.15.19:0/2963695847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d140039e0 0x7f3d14003e00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3d20001720 tx=0x7f3d14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4076b30d5ff0b17a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.098+0000 7f3d2be00640 1 -- 172.21.15.19:0/2963695847 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20001370 con 0x7f3d140039e0 2024-03-28T13:11:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.098+0000 7f3d2be00640 1 -- 172.21.15.19:0/2963695847 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d200052d0 con 0x7f3d140039e0 2024-03-28T13:11:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.098+0000 7f3d2be00640 1 -- 172.21.15.19:0/2963695847 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20003830 con 0x7f3d140039e0 2024-03-28T13:11:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.099+0000 7f3d32c00640 1 -- 172.21.15.19:0/2963695847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d140039e0 msgr2=0x7f3d14003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.099+0000 7f3d32c00640 1 --2- 172.21.15.19:0/2963695847 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d140039e0 0x7f3d14003e00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3d20001720 tx=0x7f3d14005830 comp rx=0 tx=0).stop 2024-03-28T13:11:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.099+0000 7f3d32200640 1 -- 172.21.15.19:0/2963695847 reap_dead start 2024-03-28T13:11:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.099+0000 7f3d32c00640 1 -- 172.21.15.19:0/2963695847 shutdown_connections 2024-03-28T13:11:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.099+0000 7f3d32c00640 1 -- 172.21.15.19:0/2963695847 >> 172.21.15.19:0/2963695847 conn(0x7f3d2c067670 msgr2=0x7f3d2c067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.099+0000 7f3d32c00640 1 -- 172.21.15.19:0/2963695847 shutdown_connections 2024-03-28T13:11:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.099+0000 7f3d32c00640 1 -- 172.21.15.19:0/2963695847 wait complete. 2024-03-28T13:11:54.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.100+0000 7f3d32c00640 1 Processor -- start 2024-03-28T13:11:54.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.101+0000 7f3d32c00640 1 -- start start 2024-03-28T13:11:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.101+0000 7f3d32c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d140039e0 0x7f3d2c1613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.101+0000 7f3d32c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c155600 0x7f3d2c0ca650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.101+0000 7f3d32c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c1f4eb0 0x7f3d2c1549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.101+0000 7f3d32c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c15e800 con 0x7f3d2c155600 2024-03-28T13:11:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.101+0000 7f3d32c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c0d3550 con 0x7f3d140039e0 2024-03-28T13:11:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.101+0000 7f3d32c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3d2c0cf420 con 0x7f3d2c1f4eb0 2024-03-28T13:11:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.102+0000 7f3d30e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c155600 0x7f3d2c0ca650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.102+0000 7f3d30e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c155600 0x7f3d2c0ca650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45572/0 (socket says 172.21.15.19:45572) 2024-03-28T13:11:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.102+0000 7f3d30e00640 1 -- 172.21.15.19:0/2252406177 learned_addr learned my addr 172.21.15.19:0/2252406177 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.102+0000 7f3d31800640 1 --2- 172.21.15.19:0/2252406177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d140039e0 0x7f3d2c1613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.102+0000 7f3d32200640 1 --2- 172.21.15.19:0/2252406177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c1f4eb0 0x7f3d2c1549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.102+0000 7f3d30e00640 1 -- 172.21.15.19:0/2252406177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c1f4eb0 msgr2=0x7f3d2c1549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.102+0000 7f3d30e00640 1 --2- 172.21.15.19:0/2252406177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c1f4eb0 0x7f3d2c1549c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.102+0000 7f3d30e00640 1 -- 172.21.15.19:0/2252406177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d140039e0 msgr2=0x7f3d2c1613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.102+0000 7f3d30e00640 1 --2- 172.21.15.19:0/2252406177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d140039e0 0x7f3d2c1613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.102+0000 7f3d30e00640 1 -- 172.21.15.19:0/2252406177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2c15e980 con 0x7f3d2c155600 2024-03-28T13:11:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.103+0000 7f3d30e00640 1 --2- 172.21.15.19:0/2252406177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c155600 0x7f3d2c0ca650 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3d18010860 tx=0x7f3d18010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.103+0000 7f3d29600640 1 -- 172.21.15.19:0/2252406177 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1801a070 con 0x7f3d2c155600 2024-03-28T13:11:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.103+0000 7f3d29600640 1 -- 172.21.15.19:0/2252406177 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d1800cb40 con 0x7f3d2c155600 2024-03-28T13:11:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.103+0000 7f3d29600640 1 -- 172.21.15.19:0/2252406177 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1800f040 con 0x7f3d2c155600 2024-03-28T13:11:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.103+0000 7f3d32c00640 1 -- 172.21.15.19:0/2252406177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d2c15ec20 con 0x7f3d2c155600 2024-03-28T13:11:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.103+0000 7f3d32c00640 1 -- 172.21.15.19:0/2252406177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d2c1f46b0 con 0x7f3d2c155600 2024-03-28T13:11:54.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.104+0000 7f3d32c00640 1 -- 172.21.15.19:0/2252406177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d2c075940 con 0x7f3d2c155600 2024-03-28T13:11:54.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.106+0000 7f3d29600640 1 -- 172.21.15.19:0/2252406177 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3d18015470 con 0x7f3d2c155600 2024-03-28T13:11:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.107+0000 7f3d29600640 1 --2- 172.21.15.19:0/2252406177 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d0c06c560 0x7f3d0c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.107+0000 7f3d29600640 1 -- 172.21.15.19:0/2252406177 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f3d1808bbd0 con 0x7f3d2c155600 2024-03-28T13:11:54.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.108+0000 7f3d31800640 1 --2- 172.21.15.19:0/2252406177 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d0c06c560 0x7f3d0c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:54.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.109+0000 7f3d31800640 1 --2- 172.21.15.19:0/2252406177 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d0c06c560 0x7f3d0c06ea20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3d20033000 tx=0x7f3d20009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:54.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.111+0000 7f3d29600640 1 -- 172.21.15.19:0/2252406177 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3d1805a1e0 con 0x7f3d2c155600 2024-03-28T13:11:54.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.482+0000 7f3d32c00640 1 -- 172.21.15.19:0/2252406177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3d2c0d4b00 con 0x7f3d2c155600 2024-03-28T13:11:54.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.483+0000 7f3d29600640 1 -- 172.21.15.19:0/2252406177 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f3d18059ae0 con 0x7f3d2c155600 2024-03-28T13:11:54.483 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:11:54.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.485+0000 7f3d32c00640 1 -- 172.21.15.19:0/2252406177 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d0c06c560 msgr2=0x7f3d0c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:54.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.485+0000 7f3d32c00640 1 --2- 172.21.15.19:0/2252406177 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d0c06c560 0x7f3d0c06ea20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3d20033000 tx=0x7f3d20009030 comp rx=0 tx=0).stop 2024-03-28T13:11:54.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.485+0000 7f3d32c00640 1 -- 172.21.15.19:0/2252406177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c155600 msgr2=0x7f3d2c0ca650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:54.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.485+0000 7f3d32c00640 1 --2- 172.21.15.19:0/2252406177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c155600 0x7f3d2c0ca650 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3d18010860 tx=0x7f3d18010d20 comp rx=0 tx=0).stop 2024-03-28T13:11:54.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.485+0000 7f3d32c00640 1 -- 172.21.15.19:0/2252406177 shutdown_connections 2024-03-28T13:11:54.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.485+0000 7f3d32c00640 1 --2- 172.21.15.19:0/2252406177 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d0c06c560 0x7f3d0c06ea20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:54.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.485+0000 7f3d32c00640 1 --2- 172.21.15.19:0/2252406177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d2c1f4eb0 0x7f3d2c1549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:54.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.485+0000 7f3d32c00640 1 --2- 172.21.15.19:0/2252406177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d2c155600 0x7f3d2c0ca650 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:54.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.485+0000 7f3d32c00640 1 --2- 172.21.15.19:0/2252406177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d140039e0 0x7f3d2c1613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:54.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.485+0000 7f3d32c00640 1 -- 172.21.15.19:0/2252406177 >> 172.21.15.19:0/2252406177 conn(0x7f3d2c067670 msgr2=0x7f3d2c161f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:54.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.485+0000 7f3d32c00640 1 -- 172.21.15.19:0/2252406177 shutdown_connections 2024-03-28T13:11:54.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54.486+0000 7f3d32c00640 1 -- 172.21.15.19:0/2252406177 wait complete. 2024-03-28T13:11:54.505 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-28T13:11:54.506 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:11:54.506 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:11:54.506 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:11:54.506 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:11:54.506 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-28T13:11:54.506 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-28T13:11:54.506 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:11:54.506 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:11:54.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:54.507 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:11:54.556 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:11:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:11:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:11:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:11:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:11:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:11:54.557 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:11:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:11:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:11:54.557 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:11:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:11:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:11:54.557 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:11:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:11:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:11:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:11:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42440sec preferred_lft 42440sec 2024-03-28T13:11:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:11:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:11:54.558 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:11:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:11:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:11:54.558 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:11:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:11:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:11:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:11:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:11:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:11:54.558 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:11:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:11:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:11:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:11:54.559 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:11:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether e6:17:c5:8e:2b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:11:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::e417:c5ff:fe8e:2bcf/64 scope link 2024-03-28T13:11:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:11:54.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:54.559 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:11:54.648 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:11:54.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:54.652 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:11:54.705 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:11:54.706 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:11:54.706 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:11:54.706 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:11:54.706 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:11:54.707 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:11:54.707 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:11:54.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:11:54.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:11:54.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:11:54.707 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:11:54.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:11:54.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:11:54.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:54.980 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:11:55.026 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:11:55.026 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:11:55.027 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:11:55.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:55.027 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:11:55.160 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:11:55.160 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:11:55.208 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:11:55.208 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:11:55.208 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:11:55.306 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:11:55.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:55.306 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:11:55.459 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:11:55.459 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:11:55.459 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:11:55.483 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:11:55.483 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:11:55.483 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:11:55.483 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:11:55.483 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:11:55.510 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:11:55.510 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:11:55.510 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:11:55.510 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:11:55.510 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:11:55.510 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:11:55.510 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:11:55.510 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:11:55.512 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:11:55.512 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:11:55.512 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:11:55.512 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:11:55.512 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:11:55.515 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:11:55.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:55.516 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:11:55.594 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:11:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:11:55.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:11:55.715 INFO:teuthology.orchestra.run.smithi019.stdout:client4997 2024-03-28T13:11:55.715 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:11:55.715 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:11:55.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:55.764 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-03-28T13:11:55.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:55.873 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-03-28T13:11:55.929 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:11:55.929 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:11:55.929 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:11:55.929 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:11:55.929 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-03-28T13:11:55.929 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:11:56.063 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627777 2024-03-28T13:11:56.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:11:56.077 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:11:56.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.352+0000 7fd627000640 1 Processor -- start 2024-03-28T13:11:56.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.353+0000 7fd627000640 1 -- start start 2024-03-28T13:11:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.354+0000 7fd627000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd62015e6e0 0x7fd6200d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:56.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.354+0000 7fd627000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd62006b570 0x7fd62006ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.354+0000 7fd627000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6200d08c0 con 0x7fd6200b0580 2024-03-28T13:11:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.354+0000 7fd627000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd620162f90 con 0x7fd62006b570 2024-03-28T13:11:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.354+0000 7fd627000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd6200d0660 con 0x7fd62015e6e0 2024-03-28T13:11:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.355+0000 7fd626600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd62006b570 0x7fd62006ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.355+0000 7fd625200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd6200b0580 0x7fd620162880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50858/0 (socket says 172.21.15.19:50858) 2024-03-28T13:11:56.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.355+0000 7fd625200640 1 -- 172.21.15.19:0/3568079845 learned_addr learned my addr 172.21.15.19:0/3568079845 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.355+0000 7fd625c00640 1 --2- 172.21.15.19:0/3568079845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd62015e6e0 0x7fd6200d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.355+0000 7fd61fe00640 1 -- 172.21.15.19:0/3568079845 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1647163105 0 0) 0x7fd6200d08c0 con 0x7fd6200b0580 2024-03-28T13:11:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.355+0000 7fd61fe00640 1 -- 172.21.15.19:0/3568079845 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd608003520 con 0x7fd6200b0580 2024-03-28T13:11:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.355+0000 7fd626600640 1 -- 172.21.15.19:0/3568079845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd62015e6e0 msgr2=0x7fd6200d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.355+0000 7fd626600640 1 --2- 172.21.15.19:0/3568079845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd62015e6e0 0x7fd6200d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.355+0000 7fd626600640 1 -- 172.21.15.19:0/3568079845 >> v1:172.21.15.19:6789/0 conn(0x7fd6200b0580 legacy=0x7fd620162880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:56.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.355+0000 7fd626600640 1 -- 172.21.15.19:0/3568079845 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6200d0ba0 con 0x7fd62006b570 2024-03-28T13:11:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.357+0000 7fd626600640 1 --2- 172.21.15.19:0/3568079845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd62006b570 0x7fd62006ba00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd618009d90 tx=0x7fd61802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46df9d06ea3b4a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.357+0000 7fd61fe00640 1 -- 172.21.15.19:0/3568079845 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618039070 con 0x7fd62006b570 2024-03-28T13:11:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.357+0000 7fd61fe00640 1 -- 172.21.15.19:0/3568079845 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd61802cc50 con 0x7fd62006b570 2024-03-28T13:11:56.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.357+0000 7fd61fe00640 1 -- 172.21.15.19:0/3568079845 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618034440 con 0x7fd62006b570 2024-03-28T13:11:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.358+0000 7fd627000640 1 -- 172.21.15.19:0/3568079845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd62006b570 msgr2=0x7fd62006ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.358+0000 7fd627000640 1 --2- 172.21.15.19:0/3568079845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd62006b570 0x7fd62006ba00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd618009d90 tx=0x7fd61802bea0 comp rx=0 tx=0).stop 2024-03-28T13:11:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.358+0000 7fd627000640 1 -- 172.21.15.19:0/3568079845 shutdown_connections 2024-03-28T13:11:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.358+0000 7fd627000640 1 --2- 172.21.15.19:0/3568079845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd62006b570 0x7fd62006ba00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.358+0000 7fd627000640 1 --2- 172.21.15.19:0/3568079845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd62015e6e0 0x7fd6200d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.358+0000 7fd627000640 1 -- 172.21.15.19:0/3568079845 >> 172.21.15.19:0/3568079845 conn(0x7fd62009aa20 msgr2=0x7fd6200b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.358+0000 7fd627000640 1 -- 172.21.15.19:0/3568079845 shutdown_connections 2024-03-28T13:11:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.358+0000 7fd627000640 1 -- 172.21.15.19:0/3568079845 wait complete. 2024-03-28T13:11:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.360+0000 7fd627000640 1 Processor -- start 2024-03-28T13:11:56.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.360+0000 7fd627000640 1 -- start start 2024-03-28T13:11:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.361+0000 7fd627000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd62006b570 0x7fd6200b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.361+0000 7fd627000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6200b0580 0x7fd6200b6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.361+0000 7fd627000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd62015e6e0 0x7fd6200b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.361+0000 7fd627000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd6200b0a00 con 0x7fd62006b570 2024-03-28T13:11:56.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.361+0000 7fd627000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd620162080 con 0x7fd62015e6e0 2024-03-28T13:11:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.361+0000 7fd627000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd62015eac0 con 0x7fd6200b0580 2024-03-28T13:11:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.362+0000 7fd625c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd62006b570 0x7fd6200b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.362+0000 7fd625c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd62006b570 0x7fd6200b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41852/0 (socket says 172.21.15.19:41852) 2024-03-28T13:11:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.362+0000 7fd625c00640 1 -- 172.21.15.19:0/3360273725 learned_addr learned my addr 172.21.15.19:0/3360273725 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.362+0000 7fd625200640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6200b0580 0x7fd6200b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.362+0000 7fd625c00640 1 -- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6200b0580 msgr2=0x7fd6200b6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.362+0000 7fd626600640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd62015e6e0 0x7fd6200b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.362+0000 7fd625c00640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6200b0580 0x7fd6200b6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.362+0000 7fd625c00640 1 -- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd62015e6e0 msgr2=0x7fd6200b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.362+0000 7fd625c00640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd62015e6e0 0x7fd6200b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.362+0000 7fd625c00640 1 -- 172.21.15.19:0/3360273725 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6200c5880 con 0x7fd62006b570 2024-03-28T13:11:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.362+0000 7fd626600640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd62015e6e0 0x7fd6200b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.362+0000 7fd625200640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6200b0580 0x7fd6200b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:56.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.363+0000 7fd625c00640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd62006b570 0x7fd6200b2b60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd61400e870 tx=0x7fd61400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.363+0000 7fd61d600640 1 -- 172.21.15.19:0/3360273725 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd614017070 con 0x7fd62006b570 2024-03-28T13:11:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.363+0000 7fd61d600640 1 -- 172.21.15.19:0/3360273725 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd61400c940 con 0x7fd62006b570 2024-03-28T13:11:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.363+0000 7fd61d600640 1 -- 172.21.15.19:0/3360273725 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61400cd30 con 0x7fd62006b570 2024-03-28T13:11:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.363+0000 7fd627000640 1 -- 172.21.15.19:0/3360273725 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6200c5b60 con 0x7fd62006b570 2024-03-28T13:11:56.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.363+0000 7fd627000640 1 -- 172.21.15.19:0/3360273725 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6200c6120 con 0x7fd62006b570 2024-03-28T13:11:56.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.364+0000 7fd627000640 1 -- 172.21.15.19:0/3360273725 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd620075940 con 0x7fd62006b570 2024-03-28T13:11:56.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.366+0000 7fd61d600640 1 -- 172.21.15.19:0/3360273725 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd61401a070 con 0x7fd62006b570 2024-03-28T13:11:56.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.368+0000 7fd61d600640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd60006c530 0x7fd60006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:56.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.368+0000 7fd61d600640 1 -- 172.21.15.19:0/3360273725 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7fd61408b900 con 0x7fd62006b570 2024-03-28T13:11:56.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.369+0000 7fd625200640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd60006c530 0x7fd60006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:56.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.370+0000 7fd625200640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd60006c530 0x7fd60006e9f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd6200703e0 tx=0x7fd60c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:56.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.373+0000 7fd61d600640 1 -- 172.21.15.19:0/3360273725 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd614059f10 con 0x7fd62006b570 2024-03-28T13:11:56.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.616+0000 7fd627000640 1 -- 172.21.15.19:0/3360273725 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd62006de30 con 0x7fd62006b570 2024-03-28T13:11:56.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.617+0000 7fd61d600640 1 -- 172.21.15.19:0/3360273725 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7fd61408b020 con 0x7fd62006b570 2024-03-28T13:11:56.617 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:56.617 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2125648249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2125648249},{"type":"v1","addr":"172.21.15.188:6836","nonce":2125648249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/429866161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":429866161},{"type":"v1","addr":"172.21.15.188:6837","nonce":429866161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1422209289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1422209289},{"type":"v1","addr":"172.21.15.19:6834","nonce":1422209289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:11:46.379929+0000","modified":"2024-03-28T13:11:49.470791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5538},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5538":{"gid":5538,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/179338886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":179338886},{"type":"v1","addr":"172.21.15.19:6835","nonce":179338886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5538,"qdb_cluster":[5538]},"id":3}]} 2024-03-28T13:11:56.617 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-03-28T13:11:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.619+0000 7fd627000640 1 -- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd60006c530 msgr2=0x7fd60006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.619+0000 7fd627000640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd60006c530 0x7fd60006e9f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd6200703e0 tx=0x7fd60c009040 comp rx=0 tx=0).stop 2024-03-28T13:11:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.619+0000 7fd627000640 1 -- 172.21.15.19:0/3360273725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd62006b570 msgr2=0x7fd6200b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.619+0000 7fd627000640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd62006b570 0x7fd6200b2b60 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd61400e870 tx=0x7fd61400ed30 comp rx=0 tx=0).stop 2024-03-28T13:11:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.620+0000 7fd627000640 1 -- 172.21.15.19:0/3360273725 shutdown_connections 2024-03-28T13:11:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.620+0000 7fd627000640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd60006c530 0x7fd60006e9f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.620+0000 7fd627000640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd62015e6e0 0x7fd6200b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.620+0000 7fd627000640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6200b0580 0x7fd6200b6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.620+0000 7fd627000640 1 --2- 172.21.15.19:0/3360273725 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd62006b570 0x7fd6200b2b60 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:56.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.620+0000 7fd627000640 1 -- 172.21.15.19:0/3360273725 >> 172.21.15.19:0/3360273725 conn(0x7fd62009aa20 msgr2=0x7fd62015da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:56.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.620+0000 7fd627000640 1 -- 172.21.15.19:0/3360273725 shutdown_connections 2024-03-28T13:11:56.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:56.620+0000 7fd627000640 1 -- 172.21.15.19:0/3360273725 wait complete. 2024-03-28T13:11:56.639 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-03-28T13:11:56.913 INFO:teuthology.orchestra.run.smithi019.stdout:{"message":"","return_code":0} 2024-03-28T13:11:56.914 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-28T13:11:56.914 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-03-28T13:11:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.012+0000 7f5553c38f40 1 Processor -- start 2024-03-28T13:11:57.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.012+0000 7f5553c38f40 1 -- start start 2024-03-28T13:11:57.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5553c38f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55600e061c20 0x55600e0495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5553c38f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55600e0e9240 0x55600e045520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5553c38f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55600e0efec0 0x55600e0e4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5553c38f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55600e0188d0 con 0x55600e061c20 2024-03-28T13:11:57.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5553c38f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55600dfbe2c0 con 0x55600e0e9240 2024-03-28T13:11:57.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5553c38f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55600e018390 con 0x55600e0efec0 2024-03-28T13:11:57.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5550600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55600e061c20 0x55600e0495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5550600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55600e061c20 0x55600e0495a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41862/0 (socket says 172.21.15.19:41862) 2024-03-28T13:11:57.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5550600640 1 -- 172.21.15.19:0/2515482481 learned_addr learned my addr 172.21.15.19:0/2515482481 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5551000640 1 --2- 172.21.15.19:0/2515482481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55600e0efec0 0x55600e0e4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f554fc00640 1 --2- 172.21.15.19:0/2515482481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55600e0e9240 0x55600e045520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5550600640 1 -- 172.21.15.19:0/2515482481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55600e0efec0 msgr2=0x55600e0e4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5550600640 1 --2- 172.21.15.19:0/2515482481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55600e0efec0 0x55600e0e4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5550600640 1 -- 172.21.15.19:0/2515482481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55600e0e9240 msgr2=0x55600e045520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5550600640 1 --2- 172.21.15.19:0/2515482481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55600e0e9240 0x55600e045520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5550600640 1 -- 172.21.15.19:0/2515482481 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55600e045a60 con 0x55600e061c20 2024-03-28T13:11:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.013+0000 7f5551000640 1 --2- 172.21.15.19:0/2515482481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55600e0efec0 0x55600e0e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.014+0000 7f5550600640 1 --2- 172.21.15.19:0/2515482481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55600e061c20 0x55600e0495a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5540001db0 tx=0x7f554000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f930ec96113c4c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.014+0000 7f554e800640 1 -- 172.21.15.19:0/2515482481 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554000bd80 con 0x55600e061c20 2024-03-28T13:11:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.014+0000 7f554e800640 1 -- 172.21.15.19:0/2515482481 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f554000f400 con 0x55600e061c20 2024-03-28T13:11:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.014+0000 7f554e800640 1 -- 172.21.15.19:0/2515482481 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554000f7f0 con 0x55600e061c20 2024-03-28T13:11:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.015+0000 7f5553c38f40 1 -- 172.21.15.19:0/2515482481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55600e061c20 msgr2=0x55600e0495a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.015+0000 7f5553c38f40 1 --2- 172.21.15.19:0/2515482481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55600e061c20 0x55600e0495a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5540001db0 tx=0x7f554000b5b0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.015+0000 7f5553c38f40 1 -- 172.21.15.19:0/2515482481 shutdown_connections 2024-03-28T13:11:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.015+0000 7f5553c38f40 1 --2- 172.21.15.19:0/2515482481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55600e0efec0 0x55600e0e4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.015+0000 7f5553c38f40 1 --2- 172.21.15.19:0/2515482481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55600e0e9240 0x55600e045520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.015+0000 7f5553c38f40 1 --2- 172.21.15.19:0/2515482481 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55600e061c20 0x55600e0495a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.015+0000 7f5553c38f40 1 -- 172.21.15.19:0/2515482481 >> 172.21.15.19:0/2515482481 conn(0x55600e065630 msgr2=0x55600e0ef070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.015+0000 7f5553c38f40 1 -- 172.21.15.19:0/2515482481 shutdown_connections 2024-03-28T13:11:57.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.015+0000 7f5553c38f40 1 -- 172.21.15.19:0/2515482481 wait complete. 2024-03-28T13:11:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.016+0000 7f5553c38f40 1 Processor -- start 2024-03-28T13:11:57.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.016+0000 7f5553c38f40 1 -- start start 2024-03-28T13:11:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.017+0000 7f5553c38f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55600e061c20 0x55600e061490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.017+0000 7f5553c38f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55600e0e9240 0x55600e0dfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.017+0000 7f5553c38f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55600e0efec0 0x55600e016aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.017+0000 7f5553c38f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55600e0462d0 con 0x55600e061c20 2024-03-28T13:11:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.017+0000 7f5553c38f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55600e049250 con 0x55600e0e9240 2024-03-28T13:11:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.017+0000 7f5553c38f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55600e0e03a0 con 0x55600e0efec0 2024-03-28T13:11:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.017+0000 7f5550600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55600e061c20 0x55600e061490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.017+0000 7f5550600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55600e061c20 0x55600e061490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41866/0 (socket says 172.21.15.19:41866) 2024-03-28T13:11:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.017+0000 7f5550600640 1 -- 172.21.15.19:0/569406552 learned_addr learned my addr 172.21.15.19:0/569406552 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.018+0000 7f5550600640 1 -- 172.21.15.19:0/569406552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55600e0efec0 msgr2=0x55600e016aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.018+0000 7f5551000640 1 --2- 172.21.15.19:0/569406552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55600e0efec0 0x55600e016aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.018+0000 7f554fc00640 1 --2- 172.21.15.19:0/569406552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55600e0e9240 0x55600e0dfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.018+0000 7f5550600640 1 --2- 172.21.15.19:0/569406552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55600e0efec0 0x55600e016aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.018+0000 7f5550600640 1 -- 172.21.15.19:0/569406552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55600e0e9240 msgr2=0x55600e0dfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.018+0000 7f5550600640 1 --2- 172.21.15.19:0/569406552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55600e0e9240 0x55600e0dfaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.018+0000 7f5550600640 1 -- 172.21.15.19:0/569406552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55600e049640 con 0x55600e061c20 2024-03-28T13:11:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.018+0000 7f5551000640 1 --2- 172.21.15.19:0/569406552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55600e0efec0 0x55600e016aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.018+0000 7f5550600640 1 --2- 172.21.15.19:0/569406552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55600e061c20 0x55600e061490 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f554000bab0 tx=0x7f5540000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.018+0000 7f554d400640 1 -- 172.21.15.19:0/569406552 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5540020070 con 0x55600e061c20 2024-03-28T13:11:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.018+0000 7f554d400640 1 -- 172.21.15.19:0/569406552 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f554001c440 con 0x55600e061c20 2024-03-28T13:11:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.018+0000 7f554d400640 1 -- 172.21.15.19:0/569406552 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554001c890 con 0x55600e061c20 2024-03-28T13:11:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.018+0000 7f5553c38f40 1 -- 172.21.15.19:0/569406552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55600e0f1c40 con 0x55600e061c20 2024-03-28T13:11:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.019+0000 7f5553c38f40 1 -- 172.21.15.19:0/569406552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55600e0e31f0 con 0x55600e061c20 2024-03-28T13:11:57.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.021+0000 7f554d400640 1 -- 172.21.15.19:0/569406552 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5540023070 con 0x55600e061c20 2024-03-28T13:11:57.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.022+0000 7f554d400640 1 --2- 172.21.15.19:0/569406552 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f553006af40 0x7f553006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.022+0000 7f554d400640 1 -- 172.21.15.19:0/569406552 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f554008c0a0 con 0x55600e061c20 2024-03-28T13:11:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.022+0000 7f5553c38f40 1 --2- 172.21.15.19:0/569406552 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x55600e017660 0x55600e188450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.022+0000 7f5553c38f40 1 -- 172.21.15.19:0/569406552 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49) -- 0x55600e0e3710 con 0x55600e017660 2024-03-28T13:11:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.022+0000 7f5551000640 1 --2- 172.21.15.19:0/569406552 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x55600e017660 0x55600e188450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.022+0000 7f554fc00640 1 --2- 172.21.15.19:0/569406552 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f553006af40 0x7f553006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.023+0000 7f5551000640 1 --2- 172.21.15.19:0/569406552 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x55600e017660 0x55600e188450 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.023+0000 7f5551000640 1 -- 172.21.15.19:0/569406552 <== osd.3 v2:172.21.15.19:6819/3265515865 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f55480525c0 con 0x55600e017660 2024-03-28T13:11:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.023+0000 7f554fc00640 1 --2- 172.21.15.19:0/569406552 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f553006af40 0x7f553006d400 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55600e049ef0 tx=0x7f553c005a00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.023+0000 7f5553c38f40 1 -- 172.21.15.19:0/569406552 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x55600e017660 msgr2=0x55600e188450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.024+0000 7f5553c38f40 1 --2- 172.21.15.19:0/569406552 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x55600e017660 0x55600e188450 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.024+0000 7f5553c38f40 1 -- 172.21.15.19:0/569406552 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f553006af40 msgr2=0x7f553006d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.024+0000 7f5553c38f40 1 --2- 172.21.15.19:0/569406552 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f553006af40 0x7f553006d400 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55600e049ef0 tx=0x7f553c005a00 comp rx=0 tx=0).stop 2024-03-28T13:11:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.024+0000 7f5553c38f40 1 -- 172.21.15.19:0/569406552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55600e061c20 msgr2=0x55600e061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.024+0000 7f5553c38f40 1 --2- 172.21.15.19:0/569406552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55600e061c20 0x55600e061490 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f554000bab0 tx=0x7f5540000bc0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.024+0000 7f5551000640 1 -- 172.21.15.19:0/569406552 reap_dead start 2024-03-28T13:11:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.024+0000 7f5553c38f40 1 -- 172.21.15.19:0/569406552 shutdown_connections 2024-03-28T13:11:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.024+0000 7f5553c38f40 1 -- 172.21.15.19:0/569406552 >> 172.21.15.19:0/569406552 conn(0x55600e065630 msgr2=0x55600e0ef070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.024+0000 7f5553c38f40 1 -- 172.21.15.19:0/569406552 shutdown_connections 2024-03-28T13:11:57.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.024+0000 7f5553c38f40 1 -- 172.21.15.19:0/569406552 wait complete. 2024-03-28T13:11:57.028 INFO:teuthology.orchestra.run.smithi019.stdout:Yalphaparent_a 2024-03-28T13:11:57.028 DEBUG:teuthology.orchestra.run.smithi019:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-03-28T13:11:57.196 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:11:57.196 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511627777, 2024-03-28T13:11:57.197 INFO:teuthology.orchestra.run.smithi019.stdout: "ancestors": [ 2024-03-28T13:11:57.197 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:57.197 INFO:teuthology.orchestra.run.smithi019.stdout: "dirino": 1099511627776, 2024-03-28T13:11:57.197 INFO:teuthology.orchestra.run.smithi019.stdout: "dname": "alpha", 2024-03-28T13:11:57.197 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 2 2024-03-28T13:11:57.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:11:57.197 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:11:57.197 INFO:teuthology.orchestra.run.smithi019.stdout: "dirino": 1, 2024-03-28T13:11:57.197 INFO:teuthology.orchestra.run.smithi019.stdout: "dname": "parent_a", 2024-03-28T13:11:57.197 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 4 2024-03-28T13:11:57.197 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:57.197 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-03-28T13:11:57.197 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 7, 2024-03-28T13:11:57.198 INFO:teuthology.orchestra.run.smithi019.stdout: "old_pools": [] 2024-03-28T13:11:57.198 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:11:57.198 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:57.216 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-03-28T13:11:57.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.330+0000 7fdc55043f40 1 Processor -- start 2024-03-28T13:11:57.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.330+0000 7fdc55043f40 1 -- start start 2024-03-28T13:11:57.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.331+0000 7fdc55043f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562051a814f0 0x562051b51490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.331+0000 7fdc55043f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562051ac65d0 0x562051b504f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.331+0000 7fdc55043f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562051ac2d70 0x562051aac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.331+0000 7fdc55043f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562051ac5050 con 0x562051ac65d0 2024-03-28T13:11:57.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.331+0000 7fdc55043f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562051a242c0 con 0x562051a814f0 2024-03-28T13:11:57.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.331+0000 7fdc55043f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x562051ac2650 con 0x562051ac2d70 2024-03-28T13:11:57.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.331+0000 7fdc51000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562051ac65d0 0x562051b504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.331+0000 7fdc51000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562051ac65d0 0x562051b504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41880/0 (socket says 172.21.15.19:41880) 2024-03-28T13:11:57.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.331+0000 7fdc51000640 1 -- 172.21.15.19:0/3988009908 learned_addr learned my addr 172.21.15.19:0/3988009908 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.331+0000 7fdc52400640 1 --2- 172.21.15.19:0/3988009908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562051ac2d70 0x562051aac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.332+0000 7fdc51000640 1 -- 172.21.15.19:0/3988009908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562051ac2d70 msgr2=0x562051aac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.332+0000 7fdc51000640 1 --2- 172.21.15.19:0/3988009908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562051ac2d70 0x562051aac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.332+0000 7fdc51a00640 1 --2- 172.21.15.19:0/3988009908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562051a814f0 0x562051b51490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.332+0000 7fdc51000640 1 -- 172.21.15.19:0/3988009908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562051a814f0 msgr2=0x562051b51490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.332+0000 7fdc51000640 1 --2- 172.21.15.19:0/3988009908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562051a814f0 0x562051b51490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.332+0000 7fdc51000640 1 -- 172.21.15.19:0/3988009908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562051a812f0 con 0x562051ac65d0 2024-03-28T13:11:57.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.332+0000 7fdc51a00640 1 --2- 172.21.15.19:0/3988009908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562051a814f0 0x562051b51490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.332+0000 7fdc51000640 1 --2- 172.21.15.19:0/3988009908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562051ac65d0 0x562051b504f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdc40055040 tx=0x7fdc4004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c65eff5bf924f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.332+0000 7fdc4b400640 1 -- 172.21.15.19:0/3988009908 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40057070 con 0x562051ac65d0 2024-03-28T13:11:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.332+0000 7fdc4b400640 1 -- 172.21.15.19:0/3988009908 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc40061070 con 0x562051ac65d0 2024-03-28T13:11:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.332+0000 7fdc4b400640 1 -- 172.21.15.19:0/3988009908 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40052940 con 0x562051ac65d0 2024-03-28T13:11:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.333+0000 7fdc55043f40 1 -- 172.21.15.19:0/3988009908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562051ac65d0 msgr2=0x562051b504f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.333+0000 7fdc55043f40 1 --2- 172.21.15.19:0/3988009908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562051ac65d0 0x562051b504f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdc40055040 tx=0x7fdc4004c000 comp rx=0 tx=0).stop 2024-03-28T13:11:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.333+0000 7fdc55043f40 1 -- 172.21.15.19:0/3988009908 shutdown_connections 2024-03-28T13:11:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.333+0000 7fdc55043f40 1 --2- 172.21.15.19:0/3988009908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562051ac2d70 0x562051aac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.333+0000 7fdc55043f40 1 --2- 172.21.15.19:0/3988009908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562051ac65d0 0x562051b504f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.333+0000 7fdc55043f40 1 --2- 172.21.15.19:0/3988009908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562051a814f0 0x562051b51490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.333+0000 7fdc55043f40 1 -- 172.21.15.19:0/3988009908 >> 172.21.15.19:0/3988009908 conn(0x562051aa8dc0 msgr2=0x562051aad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.333+0000 7fdc55043f40 1 -- 172.21.15.19:0/3988009908 shutdown_connections 2024-03-28T13:11:57.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.333+0000 7fdc55043f40 1 -- 172.21.15.19:0/3988009908 wait complete. 2024-03-28T13:11:57.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.334+0000 7fdc55043f40 1 Processor -- start 2024-03-28T13:11:57.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.334+0000 7fdc55043f40 1 -- start start 2024-03-28T13:11:57.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.335+0000 7fdc55043f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562051a814f0 0x562051b3ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.335+0000 7fdc55043f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562051ac2d70 0x562051b434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.335+0000 7fdc55043f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562051ac65d0 0x562051b43a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.335+0000 7fdc55043f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562051b51db0 con 0x562051ac2d70 2024-03-28T13:11:57.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.335+0000 7fdc55043f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562051b50a30 con 0x562051a814f0 2024-03-28T13:11:57.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.335+0000 7fdc55043f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x562051a81050 con 0x562051ac65d0 2024-03-28T13:11:57.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.335+0000 7fdc51000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562051ac2d70 0x562051b434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.335+0000 7fdc51000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562051ac2d70 0x562051b434e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41890/0 (socket says 172.21.15.19:41890) 2024-03-28T13:11:57.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.335+0000 7fdc51000640 1 -- 172.21.15.19:0/872913975 learned_addr learned my addr 172.21.15.19:0/872913975 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.336+0000 7fdc51000640 1 -- 172.21.15.19:0/872913975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562051ac65d0 msgr2=0x562051b43a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.336+0000 7fdc51000640 1 --2- 172.21.15.19:0/872913975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562051ac65d0 0x562051b43a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.336+0000 7fdc51000640 1 -- 172.21.15.19:0/872913975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562051a814f0 msgr2=0x562051b3ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.336+0000 7fdc51000640 1 --2- 172.21.15.19:0/872913975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562051a814f0 0x562051b3ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.336+0000 7fdc51000640 1 -- 172.21.15.19:0/872913975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562051b41040 con 0x562051ac2d70 2024-03-28T13:11:57.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.336+0000 7fdc51000640 1 --2- 172.21.15.19:0/872913975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562051ac2d70 0x562051b434e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdc400445a0 tx=0x7fdc40002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.336+0000 7fdc4a000640 1 -- 172.21.15.19:0/872913975 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4006b070 con 0x562051ac2d70 2024-03-28T13:11:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.336+0000 7fdc4a000640 1 -- 172.21.15.19:0/872913975 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc40058040 con 0x562051ac2d70 2024-03-28T13:11:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.336+0000 7fdc55043f40 1 -- 172.21.15.19:0/872913975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562051aadae0 con 0x562051ac2d70 2024-03-28T13:11:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.337+0000 7fdc4a000640 1 -- 172.21.15.19:0/872913975 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40057050 con 0x562051ac2d70 2024-03-28T13:11:57.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.337+0000 7fdc55043f40 1 -- 172.21.15.19:0/872913975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562051b3baf0 con 0x562051ac2d70 2024-03-28T13:11:57.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.339+0000 7fdc4a000640 1 -- 172.21.15.19:0/872913975 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdc4006e070 con 0x562051ac2d70 2024-03-28T13:11:57.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.339+0000 7fdc4a000640 1 --2- 172.21.15.19:0/872913975 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc3406af10 0x7fdc3406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.340+0000 7fdc4a000640 1 -- 172.21.15.19:0/872913975 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7fdc400cd1c0 con 0x562051ac2d70 2024-03-28T13:11:57.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.340+0000 7fdc55043f40 1 --2- 172.21.15.19:0/872913975 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x562051b3c470 0x562051bfaca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.340+0000 7fdc55043f40 1 -- 172.21.15.19:0/872913975 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49) -- 0x562051bfb1e0 con 0x562051b3c470 2024-03-28T13:11:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.340+0000 7fdc51a00640 1 --2- 172.21.15.19:0/872913975 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc3406af10 0x7fdc3406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.340+0000 7fdc52400640 1 --2- 172.21.15.19:0/872913975 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x562051b3c470 0x562051bfaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.341+0000 7fdc52400640 1 --2- 172.21.15.19:0/872913975 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x562051b3c470 0x562051bfaca0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.341+0000 7fdc51a00640 1 --2- 172.21.15.19:0/872913975 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc3406af10 0x7fdc3406d3d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x562051ac52a0 tx=0x7fdc44006420 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.343+0000 7fdc52400640 1 -- 172.21.15.19:0/872913975 <== osd.3 v2:172.21.15.19:6819/3265515865 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v49'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdc4c00e660 con 0x562051b3c470 2024-03-28T13:11:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.344+0000 7fdc55043f40 1 -- 172.21.15.19:0/872913975 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x562051b3c470 msgr2=0x562051bfaca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.344+0000 7fdc55043f40 1 --2- 172.21.15.19:0/872913975 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x562051b3c470 0x562051bfaca0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.344+0000 7fdc55043f40 1 -- 172.21.15.19:0/872913975 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc3406af10 msgr2=0x7fdc3406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.344+0000 7fdc55043f40 1 --2- 172.21.15.19:0/872913975 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc3406af10 0x7fdc3406d3d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x562051ac52a0 tx=0x7fdc44006420 comp rx=0 tx=0).stop 2024-03-28T13:11:57.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.344+0000 7fdc55043f40 1 -- 172.21.15.19:0/872913975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562051ac2d70 msgr2=0x562051b434e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.344+0000 7fdc55043f40 1 --2- 172.21.15.19:0/872913975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562051ac2d70 0x562051b434e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdc400445a0 tx=0x7fdc40002fb0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.344+0000 7fdc52400640 1 -- 172.21.15.19:0/872913975 reap_dead start 2024-03-28T13:11:57.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.344+0000 7fdc55043f40 1 -- 172.21.15.19:0/872913975 shutdown_connections 2024-03-28T13:11:57.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.344+0000 7fdc55043f40 1 -- 172.21.15.19:0/872913975 >> 172.21.15.19:0/872913975 conn(0x562051aa8dc0 msgr2=0x562051aa91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.344+0000 7fdc55043f40 1 -- 172.21.15.19:0/872913975 shutdown_connections 2024-03-28T13:11:57.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.344+0000 7fdc55043f40 1 -- 172.21.15.19:0/872913975 wait complete. 2024-03-28T13:11:57.352 DEBUG:teuthology.orchestra.run.smithi019:> sudo daemon-helper kill ceph --cluster ceph -w 2024-03-28T13:11:57.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:57.358 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-03-28T13:11:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.567+0000 7fa0b5600640 1 Processor -- start 2024-03-28T13:11:57.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.568+0000 7fa0b5600640 1 -- start start 2024-03-28T13:11:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.570+0000 7fa0b5600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0b015e420 0x7fa0b00b9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.570+0000 7fa0b5600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa0b00b6310 0x7fa0b00b6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.570+0000 7fa0b5600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b00d23b0 con 0x7fa0b00d3d50 2024-03-28T13:11:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.570+0000 7fa0b5600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b00d2fc0 con 0x7fa0b015e420 2024-03-28T13:11:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.570+0000 7fa0b5600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa0b00b6c80 con 0x7fa0b00b6310 2024-03-28T13:11:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.570+0000 7fa0afe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0b015e420 0x7fa0b00b9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.571+0000 7fa0afe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0b015e420 0x7fa0b00b9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41904/0 (socket says 172.21.15.19:41904) 2024-03-28T13:11:57.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.571+0000 7fa0afe00640 1 -- 172.21.15.19:0/3978732663 learned_addr learned my addr 172.21.15.19:0/3978732663 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.571+0000 7fa0b4c00640 1 --2- 172.21.15.19:0/3978732663 >> v2:172.21.15.188:3300/0 conn(0x7fa0b00b6310 0x7fa0b00b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.571+0000 7fa0afe00640 1 -- 172.21.15.19:0/3978732663 >> v2:172.21.15.188:3300/0 conn(0x7fa0b00b6310 msgr2=0x7fa0b00b6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.571+0000 7fa0afe00640 1 --2- 172.21.15.19:0/3978732663 >> v2:172.21.15.188:3300/0 conn(0x7fa0b00b6310 0x7fa0b00b6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.571+0000 7fa0afe00640 1 -- 172.21.15.19:0/3978732663 >> v1:172.21.15.188:6789/0 conn(0x7fa0b00d3d50 legacy=0x7fa0b00d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.571+0000 7fa0afe00640 1 -- 172.21.15.19:0/3978732663 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b00d2690 con 0x7fa0b015e420 2024-03-28T13:11:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.571+0000 7fa0b4c00640 1 --2- 172.21.15.19:0/3978732663 >> v2:172.21.15.188:3300/0 conn(0x7fa0b00b6310 0x7fa0b00b6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.572+0000 7fa0afe00640 1 --2- 172.21.15.19:0/3978732663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b015e420 0x7fa0b00b9c50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00d4170 tx=0x7fa0a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ea7922fbc749a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.573+0000 7fa0aea00640 1 -- 172.21.15.19:0/3978732663 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0035020 con 0x7fa0b015e420 2024-03-28T13:11:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.573+0000 7fa0aea00640 1 -- 172.21.15.19:0/3978732663 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a0031870 con 0x7fa0b015e420 2024-03-28T13:11:57.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.573+0000 7fa0aea00640 1 -- 172.21.15.19:0/3978732663 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0031c40 con 0x7fa0b015e420 2024-03-28T13:11:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.573+0000 7fa0b5600640 1 -- 172.21.15.19:0/3978732663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b015e420 msgr2=0x7fa0b00b9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.573+0000 7fa0b5600640 1 --2- 172.21.15.19:0/3978732663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b015e420 0x7fa0b00b9c50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00d4170 tx=0x7fa0a002b910 comp rx=0 tx=0).stop 2024-03-28T13:11:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.574+0000 7fa0b5600640 1 -- 172.21.15.19:0/3978732663 shutdown_connections 2024-03-28T13:11:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.574+0000 7fa0b5600640 1 --2- 172.21.15.19:0/3978732663 >> v2:172.21.15.188:3300/0 conn(0x7fa0b00b6310 0x7fa0b00b6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.574+0000 7fa0b5600640 1 --2- 172.21.15.19:0/3978732663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b015e420 0x7fa0b00b9c50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.574+0000 7fa0b5600640 1 -- 172.21.15.19:0/3978732663 >> 172.21.15.19:0/3978732663 conn(0x7fa0b00676a0 msgr2=0x7fa0b0067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.574+0000 7fa0b5600640 1 -- 172.21.15.19:0/3978732663 shutdown_connections 2024-03-28T13:11:57.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.574+0000 7fa0b5600640 1 -- 172.21.15.19:0/3978732663 wait complete. 2024-03-28T13:11:57.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.576+0000 7fa0b5600640 1 Processor -- start 2024-03-28T13:11:57.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.576+0000 7fa0b5600640 1 -- start start 2024-03-28T13:11:57.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.577+0000 7fa0b5600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0b00b6310 0x7fa0b00c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.577+0000 7fa0b5600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b00d3d50 0x7fa0b00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.577+0000 7fa0b5600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0b015e420 0x7fa0b00b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.577+0000 7fa0b5600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b00ba190 con 0x7fa0b00d3d50 2024-03-28T13:11:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.577+0000 7fa0b5600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b00d2c50 con 0x7fa0b015e420 2024-03-28T13:11:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.577+0000 7fa0b5600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b015e800 con 0x7fa0b00b6310 2024-03-28T13:11:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.577+0000 7fa0af400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b00d3d50 0x7fa0b00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.577+0000 7fa0af400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b00d3d50 0x7fa0b00b4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41918/0 (socket says 172.21.15.19:41918) 2024-03-28T13:11:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.577+0000 7fa0af400640 1 -- 172.21.15.19:0/1046257455 learned_addr learned my addr 172.21.15.19:0/1046257455 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.578+0000 7fa0b4c00640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0b015e420 0x7fa0b00b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.578+0000 7fa0afe00640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0b00b6310 0x7fa0b00c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.578+0000 7f5c2f000640 1 Processor -- start 2024-03-28T13:11:57.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.578+0000 7f5c2f000640 1 -- start start 2024-03-28T13:11:57.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.578+0000 7fa0af400640 1 -- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0b00b6310 msgr2=0x7fa0b00c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.578+0000 7fa0af400640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0b00b6310 0x7fa0b00c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.578+0000 7fa0af400640 1 -- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0b015e420 msgr2=0x7fa0b00b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.578+0000 7fa0af400640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0b015e420 0x7fa0b00b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.578+0000 7fa0af400640 1 -- 172.21.15.19:0/1046257455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b01f4c70 con 0x7fa0b00d3d50 2024-03-28T13:11:57.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.578+0000 7fa0afe00640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0b00b6310 0x7fa0b00c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:11:57.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.578+0000 7fa0b4c00640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0b015e420 0x7fa0b00b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:11:57.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.578+0000 7fa0af400640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b00d3d50 0x7fa0b00b4e70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c7f0 tx=0x7fa09800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.579+0000 7fa0a7e00640 1 -- 172.21.15.19:0/1046257455 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098016020 con 0x7fa0b00d3d50 2024-03-28T13:11:57.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.579+0000 7fa0a7e00640 1 -- 172.21.15.19:0/1046257455 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa09800aa50 con 0x7fa0b00d3d50 2024-03-28T13:11:57.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.579+0000 7fa0b5600640 1 -- 172.21.15.19:0/1046257455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b01f76d0 con 0x7fa0b00d3d50 2024-03-28T13:11:57.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.579+0000 7fa0a7e00640 1 -- 172.21.15.19:0/1046257455 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09800ae40 con 0x7fa0b00d3d50 2024-03-28T13:11:57.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.579+0000 7fa0b5600640 1 -- 172.21.15.19:0/1046257455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b00ca480 con 0x7fa0b00d3d50 2024-03-28T13:11:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.579+0000 7f5c2f000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5c280aebf0 0x7f5c280af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.579+0000 7f5c2f000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5c28071850 0x7f5c28071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.579+0000 7f5c2f000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c2806e4e0 con 0x7f5c28145ce0 2024-03-28T13:11:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.579+0000 7f5c2f000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5c2806e7c0 con 0x7f5c28071850 2024-03-28T13:11:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.579+0000 7f5c2f000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5c2806eaa0 con 0x7f5c280aebf0 2024-03-28T13:11:57.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.580+0000 7f5c2d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5c28071850 0x7f5c28071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.580+0000 7f5c27400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5c280aebf0 0x7f5c280af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.580+0000 7f5c2d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5c28071850 0x7f5c28071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41920/0 (socket says 172.21.15.19:41920) 2024-03-28T13:11:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.580+0000 7f5c2d000640 1 -- 172.21.15.19:0/1183896659 learned_addr learned my addr 172.21.15.19:0/1183896659 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.581+0000 7f5c2d000640 1 -- 172.21.15.19:0/1183896659 >> v2:172.21.15.188:3300/0 conn(0x7f5c280aebf0 msgr2=0x7f5c280af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.581+0000 7f5c2d000640 1 --2- 172.21.15.19:0/1183896659 >> v2:172.21.15.188:3300/0 conn(0x7f5c280aebf0 0x7f5c280af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.581+0000 7f5c2d000640 1 -- 172.21.15.19:0/1183896659 >> v1:172.21.15.19:6789/0 conn(0x7f5c28145ce0 legacy=0x7f5c280acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.581+0000 7f5c2d000640 1 -- 172.21.15.19:0/1183896659 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2806ed80 con 0x7f5c28071850 2024-03-28T13:11:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.582+0000 7f5c2d000640 1 --2- 172.21.15.19:0/1183896659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c28071850 0x7f5c28071ca0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5c2806cf20 tx=0x7f5c2002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea36c5a9bb36894 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.582+0000 7f5c26a00640 1 -- 172.21.15.19:0/1183896659 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2002cb10 con 0x7f5c28071850 2024-03-28T13:11:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.582+0000 7f5c26a00640 1 -- 172.21.15.19:0/1183896659 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c2002ccb0 con 0x7f5c28071850 2024-03-28T13:11:57.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.582+0000 7f5c26a00640 1 -- 172.21.15.19:0/1183896659 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2002ce60 con 0x7f5c28071850 2024-03-28T13:11:57.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.582+0000 7fa0a7e00640 1 -- 172.21.15.19:0/1046257455 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa098019020 con 0x7fa0b00d3d50 2024-03-28T13:11:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.583+0000 7f5c2f000640 1 -- 172.21.15.19:0/1183896659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c28071850 msgr2=0x7f5c28071ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.583+0000 7f5c2f000640 1 --2- 172.21.15.19:0/1183896659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c28071850 0x7f5c28071ca0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5c2806cf20 tx=0x7f5c2002bf70 comp rx=0 tx=0).stop 2024-03-28T13:11:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.583+0000 7f5c2f000640 1 -- 172.21.15.19:0/1183896659 shutdown_connections 2024-03-28T13:11:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.583+0000 7f5c2f000640 1 --2- 172.21.15.19:0/1183896659 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c28071850 0x7f5c28071ca0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.583+0000 7f5c2f000640 1 --2- 172.21.15.19:0/1183896659 >> v2:172.21.15.188:3300/0 conn(0x7f5c280aebf0 0x7f5c280af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.583+0000 7f5c2f000640 1 -- 172.21.15.19:0/1183896659 >> 172.21.15.19:0/1183896659 conn(0x7f5c2806b570 msgr2=0x7f5c2806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.584+0000 7f5c2f000640 1 -- 172.21.15.19:0/1183896659 shutdown_connections 2024-03-28T13:11:57.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.584+0000 7f5c2f000640 1 -- 172.21.15.19:0/1183896659 wait complete. 2024-03-28T13:11:57.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.584+0000 7fa0a7e00640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa08c06c530 0x7fa08c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.584+0000 7fa0b998b740 1 Processor -- start 2024-03-28T13:11:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.584+0000 7fa0a7e00640 1 -- 172.21.15.19:0/1046257455 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7fa09808b4b0 con 0x7fa0b00d3d50 2024-03-28T13:11:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.584+0000 7fa0b998b740 1 -- start start 2024-03-28T13:11:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.585+0000 7fa0afe00640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa08c06c530 0x7fa08c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.585+0000 7fa0b998b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b161788180 0x55b161788560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.585+0000 7fa0b998b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16178dfa0 0x55b16178c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.585+0000 7fa0b998b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617d5890 0x55b1617d5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.585+0000 7fa0b998b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b1616f2280 con 0x55b16178dfa0 2024-03-28T13:11:57.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.585+0000 7fa0b998b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b16169a6f0 con 0x55b161788180 2024-03-28T13:11:57.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.585+0000 7fa0b998b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55b1616ee230 con 0x55b1617d5890 2024-03-28T13:11:57.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.585+0000 7fa0af400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16178dfa0 0x55b16178c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.586+0000 7fa0af400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16178dfa0 0x55b16178c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41936/0 (socket says 172.21.15.19:41936) 2024-03-28T13:11:57.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.586+0000 7fa0af400640 1 -- 172.21.15.19:0/1081024614 learned_addr learned my addr 172.21.15.19:0/1081024614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.586+0000 7fa0afe00640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa08c06c530 0x7fa08c06e9f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0036000 tx=0x7fa0a003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.586+0000 7fa0afe00640 1 --2- 172.21.15.19:0/1081024614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b161788180 0x55b161788560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.586+0000 7fa0b4c00640 1 --2- 172.21.15.19:0/1081024614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617d5890 0x55b1617d5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.586+0000 7fa0af400640 1 -- 172.21.15.19:0/1081024614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617d5890 msgr2=0x55b1617d5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.586+0000 7fa0af400640 1 --2- 172.21.15.19:0/1081024614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617d5890 0x55b1617d5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.586+0000 7fa0af400640 1 -- 172.21.15.19:0/1081024614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b161788180 msgr2=0x55b161788560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.586+0000 7fa0af400640 1 --2- 172.21.15.19:0/1081024614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b161788180 0x55b161788560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.586+0000 7fa0af400640 1 -- 172.21.15.19:0/1081024614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1614ce7e0 con 0x55b16178dfa0 2024-03-28T13:11:57.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.586+0000 7fa0af400640 1 --2- 172.21.15.19:0/1081024614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16178dfa0 0x55b16178c1d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa098055820 tx=0x7fa098059500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee835ac429a3e80 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.586+0000 7f5c2f000640 1 Processor -- start 2024-03-28T13:11:57.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.587+0000 7f5c2f000640 1 -- start start 2024-03-28T13:11:57.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.587+0000 7fa09fe00640 1 -- 172.21.15.19:0/1081024614 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098071020 con 0x55b16178dfa0 2024-03-28T13:11:57.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.587+0000 7fa09fe00640 1 -- 172.21.15.19:0/1081024614 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0980576c0 con 0x55b16178dfa0 2024-03-28T13:11:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.587+0000 7fa09fe00640 1 -- 172.21.15.19:0/1081024614 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09808b070 con 0x55b16178dfa0 2024-03-28T13:11:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.587+0000 7fa0b998b740 1 -- 172.21.15.19:0/1081024614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16178dfa0 msgr2=0x55b16178c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.587+0000 7fa0b998b740 1 --2- 172.21.15.19:0/1081024614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16178dfa0 0x55b16178c1d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa098055820 tx=0x7fa098059500 comp rx=0 tx=0).stop 2024-03-28T13:11:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.588+0000 7fa0b998b740 1 -- 172.21.15.19:0/1081024614 shutdown_connections 2024-03-28T13:11:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.588+0000 7fa0b998b740 1 --2- 172.21.15.19:0/1081024614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617d5890 0x55b1617d5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.588+0000 7fa0b998b740 1 --2- 172.21.15.19:0/1081024614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b16178dfa0 0x55b16178c1d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.588+0000 7fa0b998b740 1 --2- 172.21.15.19:0/1081024614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b161788180 0x55b161788560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.588+0000 7fa0b998b740 1 -- 172.21.15.19:0/1081024614 >> 172.21.15.19:0/1081024614 conn(0x55b16178a200 msgr2=0x55b16178bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.588+0000 7fa0b998b740 1 -- 172.21.15.19:0/1081024614 shutdown_connections 2024-03-28T13:11:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.588+0000 7fa0b998b740 1 -- 172.21.15.19:0/1081024614 wait complete. 2024-03-28T13:11:57.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.588+0000 7f5c2f000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c28071850 0x7f5c2813eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.588+0000 7f5c2f000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c280aebf0 0x7f5c2813bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.588+0000 7fa0b998b740 1 -- start start 2024-03-28T13:11:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.588+0000 7f5c2f000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c28145ce0 0x7f5c2813c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.588+0000 7f5c2f000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5c280ad700 con 0x7f5c28071850 2024-03-28T13:11:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.588+0000 7f5c2f000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2806f060 con 0x7f5c280aebf0 2024-03-28T13:11:57.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.588+0000 7f5c2f000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5c280af570 con 0x7f5c28145ce0 2024-03-28T13:11:57.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.589+0000 7fa0b998b740 1 Processor -- start 2024-03-28T13:11:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.589+0000 7f5c27e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c28071850 0x7f5c2813eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.589+0000 7f5c27e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c28071850 0x7f5c2813eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41944/0 (socket says 172.21.15.19:41944) 2024-03-28T13:11:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.589+0000 7f5c27e00640 1 -- 172.21.15.19:0/338571930 learned_addr learned my addr 172.21.15.19:0/338571930 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.589+0000 7f5c27400640 1 --2- 172.21.15.19:0/338571930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c280aebf0 0x7f5c2813bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.589+0000 7f5c2d000640 1 --2- 172.21.15.19:0/338571930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c28145ce0 0x7f5c2813c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.589+0000 7f5c27e00640 1 -- 172.21.15.19:0/338571930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c28145ce0 msgr2=0x7f5c2813c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.589+0000 7f5c27e00640 1 --2- 172.21.15.19:0/338571930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c28145ce0 0x7f5c2813c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.589+0000 7f5c27e00640 1 -- 172.21.15.19:0/338571930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c280aebf0 msgr2=0x7f5c2813bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.589+0000 7f5c27e00640 1 --2- 172.21.15.19:0/338571930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c280aebf0 0x7f5c2813bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.589+0000 7f5c27e00640 1 -- 172.21.15.19:0/338571930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2813d020 con 0x7f5c28071850 2024-03-28T13:11:57.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.590+0000 7f5c27e00640 1 --2- 172.21.15.19:0/338571930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5c28071850 0x7f5c2813eff0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800c3c0 tx=0x7f5c1800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.590+0000 7f5c1fe00640 1 -- 172.21.15.19:0/338571930 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c180052d0 con 0x7f5c28071850 2024-03-28T13:11:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.590+0000 7f5c1fe00640 1 -- 172.21.15.19:0/338571930 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c18005490 con 0x7f5c28071850 2024-03-28T13:11:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.590+0000 7f5c1fe00640 1 -- 172.21.15.19:0/338571930 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c18007260 con 0x7f5c28071850 2024-03-28T13:11:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.590+0000 7f5c2f000640 1 -- 172.21.15.19:0/338571930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2813d300 con 0x7f5c28071850 2024-03-28T13:11:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.590+0000 7fa0b998b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b16178dfa0 0x55b1617e3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.590+0000 7fa0b998b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617d5890 0x55b1617b8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.590+0000 7fa0b998b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617accf0 0x55b1617ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.590+0000 7f5c2f000640 1 -- 172.21.15.19:0/338571930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2813d8c0 con 0x7f5c28071850 2024-03-28T13:11:57.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.590+0000 7fa0b998b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b1617b48f0 con 0x55b1617d5890 2024-03-28T13:11:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.590+0000 7fa0b998b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b1616adaa0 con 0x55b16178dfa0 2024-03-28T13:11:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.590+0000 7fa0b998b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55b1617b4b40 con 0x55b1617accf0 2024-03-28T13:11:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.590+0000 7fa0b998b740 10 client.0 _ll_register_callbacks cb 0x55b1616ad490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:11:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.591+0000 7fa0afe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b16178dfa0 0x55b1617e3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.591+0000 7fa0af400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617d5890 0x55b1617b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.591+0000 7fa0afe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b16178dfa0 0x55b1617e3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:57790/0 (socket says 172.21.15.19:57790) 2024-03-28T13:11:57.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.591+0000 7fa0afe00640 1 -- 172.21.15.19:0/394614978 learned_addr learned my addr 172.21.15.19:0/394614978 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.591+0000 7fa0b4c00640 1 --2- 172.21.15.19:0/394614978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617accf0 0x55b1617ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.591+0000 7f5c2f000640 1 -- 172.21.15.19:0/338571930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f5c280685c0 con 0x7f5c28071850 2024-03-28T13:11:57.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.591+0000 7fa0af400640 1 -- 172.21.15.19:0/394614978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617accf0 msgr2=0x55b1617ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.591+0000 7fa0af400640 1 --2- 172.21.15.19:0/394614978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617accf0 0x55b1617ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.592+0000 7fa0af400640 1 -- 172.21.15.19:0/394614978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b16178dfa0 msgr2=0x55b1617e3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.592+0000 7fa0af400640 1 --2- 172.21.15.19:0/394614978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b16178dfa0 0x55b1617e3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.592+0000 7fa0af400640 1 -- 172.21.15.19:0/394614978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b16152ef70 con 0x55b1617d5890 2024-03-28T13:11:57.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.592+0000 7fa0b4c00640 1 --2- 172.21.15.19:0/394614978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617accf0 0x55b1617ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:11:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.592+0000 7fa0af400640 1 --2- 172.21.15.19:0/394614978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617d5890 0x55b1617b8820 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa098061a70 tx=0x7fa098003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401ad54b42e85108 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.592+0000 7fa09cc00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:11:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.592+0000 7fa09cc00640 1 -- 172.21.15.19:0/394614978 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098071050 con 0x55b1617d5890 2024-03-28T13:11:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.592+0000 7fa09cc00640 1 -- 172.21.15.19:0/394614978 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7fa09806dbe0 con 0x55b1617d5890 2024-03-28T13:11:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.592+0000 7fa0b998b740 1 -- 172.21.15.19:0/394614978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0980576c0 con 0x55b1617d5890 2024-03-28T13:11:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.593+0000 7fa09cc00640 1 -- 172.21.15.19:0/394614978 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa09807d050 con 0x55b1617d5890 2024-03-28T13:11:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.593+0000 7fa0b998b740 10 client.5718 fetch_fsmap learned FSMap version 26 2024-03-28T13:11:57.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.593+0000 7fa0b998b740 1 -- 172.21.15.19:0/394614978 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55b1613c6170 con 0x55b1617d5890 2024-03-28T13:11:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.593+0000 7fa09cc00640 1 -- 172.21.15.19:0/394614978 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa09806de00 con 0x55b1617d5890 2024-03-28T13:11:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.593+0000 7fa09cc00640 1 -- 172.21.15.19:0/394614978 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fa098080020 con 0x55b1617d5890 2024-03-28T13:11:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.593+0000 7fa0b998b740 10 client.5718 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T13:11:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.593+0000 7fa0b998b740 10 client.5718 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-03-28T13:11:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.593+0000 7fa0b998b740 20 client.5718 populate_metadata read hostname 'smithi019' 2024-03-28T13:11:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.593+0000 7fa0b998b740 1 --2- 172.21.15.19:0/394614978 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55b1617ae610 0x55b1617e4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.593+0000 7fa0b998b740 4 client.5718 mds_command: new command op to 5538 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:11:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.593+0000 7fa0b998b740 1 -- 172.21.15.19:0/394614978 --> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa0980576c0 con 0x55b1617ae610 2024-03-28T13:11:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.593+0000 7fa0afe00640 1 --2- 172.21.15.19:0/394614978 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55b1617ae610 0x55b1617e4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.594+0000 7fa0afe00640 1 --2- 172.21.15.19:0/394614978 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55b1617ae610 0x55b1617e4620 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4d8a8b7a4f9723c server_cookie=a992dd0cab94063e in_seq=0 out_seq=0 2024-03-28T13:11:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.594+0000 7fa09cc00640 10 client.5718 ms_handle_connect on v2:172.21.15.19:6833/179338886 2024-03-28T13:11:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.594+0000 7f5c1fe00640 1 -- 172.21.15.19:0/338571930 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5c18007400 con 0x7f5c28071850 2024-03-28T13:11:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.595+0000 7fa09cc00640 1 -- 172.21.15.19:0/394614978 <== mds.0 v2:172.21.15.19:6833/179338886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fa0980576c0 con 0x55b1617ae610 2024-03-28T13:11:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.595+0000 7fa09cc00640 10 client.5718 handle_command_reply: tid=0 2024-03-28T13:11:57.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.595+0000 7fa0b998b740 1 client.5718 shutdown 2024-03-28T13:11:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.595+0000 7f5c1fe00640 1 --2- 172.21.15.19:0/338571930 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5c0806c530 0x7f5c0806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7f5c1fe00640 1 -- 172.21.15.19:0/338571930 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f5c1808b2f0 con 0x7f5c28071850 2024-03-28T13:11:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7fa0b998b740 1 -- 172.21.15.19:0/394614978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617d5890 msgr2=0x55b1617b8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7fa0b998b740 1 --2- 172.21.15.19:0/394614978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617d5890 0x55b1617b8820 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa098061a70 tx=0x7fa098003a70 comp rx=0 tx=0).stop 2024-03-28T13:11:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7fa0b998b740 1 -- 172.21.15.19:0/394614978 shutdown_connections 2024-03-28T13:11:57.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7f5c1fe00640 1 -- 172.21.15.19:0/338571930 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+501 (secure 0 0 0) 0x7f5c1808b6c0 con 0x7f5c28071850 2024-03-28T13:11:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7fa0b998b740 1 --2- 172.21.15.19:0/394614978 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55b1617ae610 0x55b1617e4620 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7fa0b998b740 1 --2- 172.21.15.19:0/394614978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617accf0 0x55b1617ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7fa0b998b740 1 --2- 172.21.15.19:0/394614978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617d5890 0x55b1617b8820 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7fa0b998b740 1 --2- 172.21.15.19:0/394614978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b16178dfa0 0x55b1617e3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7fa0b998b740 1 -- 172.21.15.19:0/394614978 >> 172.21.15.19:0/394614978 conn(0x55b161788180 msgr2=0x55b161787620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7fa09cc00640 0 client.5718 ms_handle_reset on v2:172.21.15.19:6833/179338886 2024-03-28T13:11:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7fa0b998b740 1 -- 172.21.15.19:0/394614978 shutdown_connections 2024-03-28T13:11:57.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7fa0b998b740 1 -- 172.21.15.19:0/394614978 wait complete. 2024-03-28T13:11:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7f5c2f000640 1 -- 172.21.15.19:0/338571930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f5c280b03b0 con 0x7f5c28071850 2024-03-28T13:11:57.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7f5c27400640 1 --2- 172.21.15.19:0/338571930 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5c0806c530 0x7f5c0806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.596+0000 7fa0b998b740 20 client.5718 trim_cache size 0 max 16384 2024-03-28T13:11:57.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.597+0000 7f5c27400640 1 --2- 172.21.15.19:0/338571930 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5c0806c530 0x7f5c0806e9f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5c281456f0 tx=0x7f5c14009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.616+0000 7fa0b998b740 1 Processor -- start 2024-03-28T13:11:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.616+0000 7fa0b998b740 1 -- start start 2024-03-28T13:11:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.617+0000 7fa0b998b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b16178dfa0 0x55b1617ebd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.617+0000 7fa0b998b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617accf0 0x55b1617ec290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.617+0000 7fa0b998b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617d5890 0x55b1617d4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.617+0000 7fa0b998b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b0200870 con 0x55b1617accf0 2024-03-28T13:11:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.617+0000 7fa0b998b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b015b6c0 con 0x55b16178dfa0 2024-03-28T13:11:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.617+0000 7fa0b998b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b015aa20 con 0x55b1617d5890 2024-03-28T13:11:57.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.617+0000 7fa0af400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617accf0 0x55b1617ec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.617+0000 7fa0af400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617accf0 0x55b1617ec290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41970/0 (socket says 172.21.15.19:41970) 2024-03-28T13:11:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.617+0000 7fa0af400640 1 -- 172.21.15.19:0/3901224760 learned_addr learned my addr 172.21.15.19:0/3901224760 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.617+0000 7fa0b4c00640 1 --2- 172.21.15.19:0/3901224760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617d5890 0x55b1617d4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.617+0000 7fa0af400640 1 -- 172.21.15.19:0/3901224760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617d5890 msgr2=0x55b1617d4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.617+0000 7fa0af400640 1 --2- 172.21.15.19:0/3901224760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617d5890 0x55b1617d4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.617+0000 7fa0af400640 1 -- 172.21.15.19:0/3901224760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b16178dfa0 msgr2=0x55b1617ebd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.617+0000 7fa0af400640 1 --2- 172.21.15.19:0/3901224760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b16178dfa0 0x55b1617ebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.617+0000 7fa0af400640 1 -- 172.21.15.19:0/3901224760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1617ad4e0 con 0x55b1617accf0 2024-03-28T13:11:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.618+0000 7fa0af400640 1 --2- 172.21.15.19:0/3901224760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617accf0 0x55b1617ec290 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa098055820 tx=0x7fa09806e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a871839fc72d6493 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.618+0000 7fa09f400640 1 -- 172.21.15.19:0/3901224760 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098071050 con 0x55b1617accf0 2024-03-28T13:11:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.618+0000 7fa09f400640 1 -- 172.21.15.19:0/3901224760 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0980830e0 con 0x55b1617accf0 2024-03-28T13:11:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.618+0000 7fa09f400640 1 -- 172.21.15.19:0/3901224760 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098058f00 con 0x55b1617accf0 2024-03-28T13:11:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.619+0000 7fa0b998b740 1 -- 172.21.15.19:0/3901224760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617accf0 msgr2=0x55b1617ec290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.619+0000 7fa0b998b740 1 --2- 172.21.15.19:0/3901224760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617accf0 0x55b1617ec290 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa098055820 tx=0x7fa09806e000 comp rx=0 tx=0).stop 2024-03-28T13:11:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.619+0000 7fa0b998b740 1 -- 172.21.15.19:0/3901224760 shutdown_connections 2024-03-28T13:11:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.619+0000 7fa0b998b740 1 --2- 172.21.15.19:0/3901224760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617d5890 0x55b1617d4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.619+0000 7fa0b998b740 1 --2- 172.21.15.19:0/3901224760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617accf0 0x55b1617ec290 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.619+0000 7fa0b998b740 1 --2- 172.21.15.19:0/3901224760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b16178dfa0 0x55b1617ebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.619+0000 7fa0b998b740 1 -- 172.21.15.19:0/3901224760 >> 172.21.15.19:0/3901224760 conn(0x55b161788180 msgr2=0x55b16178b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.619+0000 7fa0b998b740 1 -- 172.21.15.19:0/3901224760 shutdown_connections 2024-03-28T13:11:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.619+0000 7fa0b998b740 1 -- 172.21.15.19:0/3901224760 wait complete. 2024-03-28T13:11:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.620+0000 7fa0b998b740 1 -- start start 2024-03-28T13:11:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.620+0000 7fa0b998b740 1 Processor -- start 2024-03-28T13:11:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.621+0000 7fa0b998b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b16178dfa0 0x55b1617af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.621+0000 7fa0b998b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617accf0 0x55b161832790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.621+0000 7fa0b998b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617d5890 0x55b16184ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.621+0000 7fa0b998b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b1617b9480 con 0x55b1617accf0 2024-03-28T13:11:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.621+0000 7fa0b998b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b015b070 con 0x55b16178dfa0 2024-03-28T13:11:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.621+0000 7fa0b998b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55b161782c50 con 0x55b1617d5890 2024-03-28T13:11:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.621+0000 7fa0b998b740 10 client.0 _ll_register_callbacks cb 0x7fa098059b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:11:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.621+0000 7fa0af400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617accf0 0x55b161832790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.621+0000 7fa0af400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617accf0 0x55b161832790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41978/0 (socket says 172.21.15.19:41978) 2024-03-28T13:11:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.621+0000 7fa0af400640 1 -- 172.21.15.19:0/4118109836 learned_addr learned my addr 172.21.15.19:0/4118109836 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.622+0000 7fa0b4c00640 1 --2- 172.21.15.19:0/4118109836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617d5890 0x55b16184ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.622+0000 7fa0afe00640 1 --2- 172.21.15.19:0/4118109836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b16178dfa0 0x55b1617af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.622+0000 7fa0af400640 1 -- 172.21.15.19:0/4118109836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617d5890 msgr2=0x55b16184ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.622+0000 7fa0af400640 1 --2- 172.21.15.19:0/4118109836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617d5890 0x55b16184ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.622+0000 7fa0af400640 1 -- 172.21.15.19:0/4118109836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b16178dfa0 msgr2=0x55b1617af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.622+0000 7fa0af400640 1 --2- 172.21.15.19:0/4118109836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b16178dfa0 0x55b1617af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.622+0000 7fa0af400640 1 -- 172.21.15.19:0/4118109836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1617888d0 con 0x55b1617accf0 2024-03-28T13:11:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.622+0000 7fa0af400640 1 --2- 172.21.15.19:0/4118109836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617accf0 0x55b161832790 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa09805d930 tx=0x7fa098055790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df7eecbb8dd81816 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.622+0000 7fa09cc00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:11:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.623+0000 7fa09cc00640 1 -- 172.21.15.19:0/4118109836 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098071070 con 0x55b1617accf0 2024-03-28T13:11:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.623+0000 7fa09cc00640 1 -- 172.21.15.19:0/4118109836 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7fa098061d10 con 0x55b1617accf0 2024-03-28T13:11:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.623+0000 7fa0b998b740 1 -- 172.21.15.19:0/4118109836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0980830e0 con 0x55b1617accf0 2024-03-28T13:11:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.623+0000 7fa09cc00640 1 -- 172.21.15.19:0/4118109836 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa09806c050 con 0x55b1617accf0 2024-03-28T13:11:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.623+0000 7fa0b998b740 10 client.5724 fetch_fsmap learned FSMap version 26 2024-03-28T13:11:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.623+0000 7fa0b998b740 1 -- 172.21.15.19:0/4118109836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55b1617888d0 con 0x55b1617accf0 2024-03-28T13:11:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.623+0000 7fa09cc00640 1 -- 172.21.15.19:0/4118109836 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0980830e0 con 0x55b1617accf0 2024-03-28T13:11:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.624+0000 7fa09cc00640 1 -- 172.21.15.19:0/4118109836 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fa098080070 con 0x55b1617accf0 2024-03-28T13:11:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.624+0000 7fa0b998b740 10 client.5724 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T13:11:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.624+0000 7fa0b998b740 10 client.5724 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-03-28T13:11:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.624+0000 7fa0b998b740 20 client.5724 populate_metadata read hostname 'smithi019' 2024-03-28T13:11:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.624+0000 7fa0b998b740 1 --2- 172.21.15.19:0/4118109836 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55b16184e300 0x55b1617e19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.624+0000 7fa0b998b740 4 client.5724 mds_command: new command op to 5538 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-03-28T13:11:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.624+0000 7fa0b998b740 1 -- 172.21.15.19:0/4118109836 --> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55b1617afd00 con 0x55b16184e300 2024-03-28T13:11:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.624+0000 7fa0afe00640 1 --2- 172.21.15.19:0/4118109836 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55b16184e300 0x55b1617e19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.625+0000 7fa0afe00640 1 --2- 172.21.15.19:0/4118109836 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55b16184e300 0x55b1617e19c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3b3816d8d21dbcb server_cookie=60f173bb94061262 in_seq=0 out_seq=0 2024-03-28T13:11:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.625+0000 7fa09cc00640 10 client.5724 ms_handle_connect on v2:172.21.15.19:6833/179338886 2024-03-28T13:11:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.626+0000 7fa09cc00640 1 -- 172.21.15.19:0/4118109836 <== mds.0 v2:172.21.15.19:6833/179338886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b1617afd00 con 0x55b16184e300 2024-03-28T13:11:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.626+0000 7fa09cc00640 10 client.5724 handle_command_reply: tid=0 2024-03-28T13:11:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.626+0000 7fa0b998b740 1 client.5724 shutdown 2024-03-28T13:11:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.626+0000 7fa0b998b740 1 -- 172.21.15.19:0/4118109836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617accf0 msgr2=0x55b161832790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.626+0000 7fa0b998b740 1 --2- 172.21.15.19:0/4118109836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617accf0 0x55b161832790 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa09805d930 tx=0x7fa098055790 comp rx=0 tx=0).stop 2024-03-28T13:11:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.626+0000 7fa0b998b740 1 -- 172.21.15.19:0/4118109836 shutdown_connections 2024-03-28T13:11:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.626+0000 7fa0b998b740 1 --2- 172.21.15.19:0/4118109836 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55b16184e300 0x55b1617e19c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.626+0000 7fa0b998b740 1 --2- 172.21.15.19:0/4118109836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b1617d5890 0x55b16184ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.626+0000 7fa0b998b740 1 --2- 172.21.15.19:0/4118109836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b1617accf0 0x55b161832790 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.626+0000 7fa0b998b740 1 --2- 172.21.15.19:0/4118109836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b16178dfa0 0x55b1617af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.626+0000 7fa09cc00640 0 client.5724 ms_handle_reset on v2:172.21.15.19:6833/179338886 2024-03-28T13:11:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.626+0000 7fa0b998b740 1 -- 172.21.15.19:0/4118109836 >> 172.21.15.19:0/4118109836 conn(0x55b161788180 msgr2=0x55b1617b6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.656 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-03-28T13:11:57.626+0000 7fad6be00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={4997=0-4194304@1} caps={4997=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=4997 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55cd7dfccc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-28T13:11:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.626+0000 7fa0b998b740 1 -- 172.21.15.19:0/4118109836 shutdown_connections 2024-03-28T13:11:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.626+0000 7fa0b998b740 1 -- 172.21.15.19:0/4118109836 wait complete. 2024-03-28T13:11:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.627+0000 7fa0b998b740 20 client.5724 trim_cache size 0 max 16384 2024-03-28T13:11:57.658 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:11:57.658 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-03-28T13:11:57.658 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "d27d8124-e0b0-47cd-b6cb-3c45edf91f32", 2024-03-28T13:11:57.658 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-03-28T13:11:57.658 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:11:57.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.627+0000 7fa0b5600640 1 -- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa08c06c530 msgr2=0x7fa08c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.627+0000 7fa0b5600640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa08c06c530 0x7fa08c06e9f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0036000 tx=0x7fa0a003c040 comp rx=0 tx=0).stop 2024-03-28T13:11:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.627+0000 7fa0b5600640 1 -- 172.21.15.19:0/1046257455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b00d3d50 msgr2=0x7fa0b00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.627+0000 7fa0b5600640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b00d3d50 0x7fa0b00b4e70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa09800c7f0 tx=0x7fa09800ccb0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.627+0000 7fa0b5600640 1 -- 172.21.15.19:0/1046257455 shutdown_connections 2024-03-28T13:11:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.627+0000 7fa0b5600640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa08c06c530 0x7fa08c06e9f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.627+0000 7fa0b5600640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0b015e420 0x7fa0b00b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.627+0000 7fa0b5600640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0b00d3d50 0x7fa0b00b4e70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.627+0000 7fa0b5600640 1 --2- 172.21.15.19:0/1046257455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0b00b6310 0x7fa0b00c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.627+0000 7fa0b5600640 1 -- 172.21.15.19:0/1046257455 >> 172.21.15.19:0/1046257455 conn(0x7fa0b00676a0 msgr2=0x7fa0b00b7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.628+0000 7fa0b5600640 1 -- 172.21.15.19:0/1046257455 shutdown_connections 2024-03-28T13:11:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.628+0000 7fa0b5600640 1 -- 172.21.15.19:0/1046257455 wait complete. 2024-03-28T13:11:57.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:11:57.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-03-28T13:11:57.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.869+0000 7fcd2b200640 1 Processor -- start 2024-03-28T13:11:57.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.869+0000 7fcd2b200640 1 -- start start 2024-03-28T13:11:57.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.871+0000 7fcd2b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd24141c10 0x7fcd24071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.871+0000 7fcd2b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd24071eb0 0x7fcd24162160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.871+0000 7fcd2b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd2415ec80 con 0x7fcd240d0280 2024-03-28T13:11:57.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.871+0000 7fcd2b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcd2415ef20 con 0x7fcd24141c10 2024-03-28T13:11:57.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.871+0000 7fcd2b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcd2415f200 con 0x7fcd24071eb0 2024-03-28T13:11:57.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.871+0000 7fcd23e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcd240d0280 0x7fcd240d0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50894/0 (socket says 172.21.15.19:50894) 2024-03-28T13:11:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.871+0000 7fcd23e00640 1 -- 172.21.15.19:0/2998455158 learned_addr learned my addr 172.21.15.19:0/2998455158 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:11:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.871+0000 7fcd23400640 1 --2- 172.21.15.19:0/2998455158 >> v2:172.21.15.19:3300/0 conn(0x7fcd24141c10 0x7fcd24071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.871+0000 7fcd29200640 1 --2- 172.21.15.19:0/2998455158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd24071eb0 0x7fcd24162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.872+0000 7fcd23400640 1 -- 172.21.15.19:0/2998455158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd24071eb0 msgr2=0x7fcd24162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.872+0000 7fcd23400640 1 --2- 172.21.15.19:0/2998455158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd24071eb0 0x7fcd24162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.872+0000 7fcd23400640 1 -- 172.21.15.19:0/2998455158 >> v1:172.21.15.19:6789/0 conn(0x7fcd240d0280 legacy=0x7fcd240d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.872+0000 7fcd23400640 1 -- 172.21.15.19:0/2998455158 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2415f4e0 con 0x7fcd24141c10 2024-03-28T13:11:57.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.872+0000 7fcd22a00640 1 -- 172.21.15.19:0/2998455158 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3358357471 0 0) 0x7fcd2415ec80 con 0x7fcd240d0280 2024-03-28T13:11:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.873+0000 7fcd23400640 1 --2- 172.21.15.19:0/2998455158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd24141c10 0x7fcd24071970 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fcd100097a0 tx=0x7fcd1002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b552415b013b3c7c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.873+0000 7fcd22a00640 1 -- 172.21.15.19:0/2998455158 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10035020 con 0x7fcd24141c10 2024-03-28T13:11:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.873+0000 7fcd22a00640 1 -- 172.21.15.19:0/2998455158 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd10031810 con 0x7fcd24141c10 2024-03-28T13:11:57.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.873+0000 7fcd22a00640 1 -- 172.21.15.19:0/2998455158 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10031be0 con 0x7fcd24141c10 2024-03-28T13:11:57.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.874+0000 7fcd2b200640 1 -- 172.21.15.19:0/2998455158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd24141c10 msgr2=0x7fcd24071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.874+0000 7fcd2b200640 1 --2- 172.21.15.19:0/2998455158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd24141c10 0x7fcd24071970 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fcd100097a0 tx=0x7fcd1002b8b0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.874+0000 7fcd2b200640 1 -- 172.21.15.19:0/2998455158 shutdown_connections 2024-03-28T13:11:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.874+0000 7fcd2b200640 1 --2- 172.21.15.19:0/2998455158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd24071eb0 0x7fcd24162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.874+0000 7fcd2b200640 1 --2- 172.21.15.19:0/2998455158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd24141c10 0x7fcd24071970 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.874+0000 7fcd2b200640 1 -- 172.21.15.19:0/2998455158 >> 172.21.15.19:0/2998455158 conn(0x7fcd2406b750 msgr2=0x7fcd2406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.874+0000 7fcd2b200640 1 -- 172.21.15.19:0/2998455158 shutdown_connections 2024-03-28T13:11:57.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.874+0000 7fcd2b200640 1 -- 172.21.15.19:0/2998455158 wait complete. 2024-03-28T13:11:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.876+0000 7fcd2b200640 1 Processor -- start 2024-03-28T13:11:57.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.876+0000 7fcd2b200640 1 -- start start 2024-03-28T13:11:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.877+0000 7fcd2b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd24071eb0 0x7fcd24099ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.877+0000 7fcd2b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd240d0280 0x7fcd240b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.877+0000 7fcd2b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd24141c10 0x7fcd240b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.877+0000 7fcd2b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd24158e90 con 0x7fcd24141c10 2024-03-28T13:11:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.877+0000 7fcd2b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2406f340 con 0x7fcd240d0280 2024-03-28T13:11:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.877+0000 7fcd2b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcd241f58d0 con 0x7fcd24071eb0 2024-03-28T13:11:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.878+0000 7fcd29200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd24141c10 0x7fcd240b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.878+0000 7fcd29200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd24141c10 0x7fcd240b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42002/0 (socket says 172.21.15.19:42002) 2024-03-28T13:11:57.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.878+0000 7fcd29200640 1 -- 172.21.15.19:0/2797223543 learned_addr learned my addr 172.21.15.19:0/2797223543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.878+0000 7fcd23e00640 1 --2- 172.21.15.19:0/2797223543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd24071eb0 0x7fcd24099ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.878+0000 7fcd23400640 1 --2- 172.21.15.19:0/2797223543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd240d0280 0x7fcd240b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.879+0000 7fcd29200640 1 -- 172.21.15.19:0/2797223543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd24071eb0 msgr2=0x7fcd24099ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.879+0000 7fcd29200640 1 --2- 172.21.15.19:0/2797223543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd24071eb0 0x7fcd24099ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.879+0000 7fcd29200640 1 -- 172.21.15.19:0/2797223543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd240d0280 msgr2=0x7fcd240b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.879+0000 7fcd29200640 1 --2- 172.21.15.19:0/2797223543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd240d0280 0x7fcd240b1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.879+0000 7fcd29200640 1 -- 172.21.15.19:0/2797223543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd241f5bb0 con 0x7fcd24141c10 2024-03-28T13:11:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.879+0000 7fcd29200640 1 --2- 172.21.15.19:0/2797223543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd24141c10 0x7fcd240b2460 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c007e40 tx=0x7fcd1c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.879+0000 7fcd1be00640 1 -- 172.21.15.19:0/2797223543 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c014620 con 0x7fcd24141c10 2024-03-28T13:11:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.879+0000 7fcd1be00640 1 -- 172.21.15.19:0/2797223543 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd1c019070 con 0x7fcd24141c10 2024-03-28T13:11:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.879+0000 7fcd1be00640 1 -- 172.21.15.19:0/2797223543 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c014a20 con 0x7fcd24141c10 2024-03-28T13:11:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.879+0000 7fcd2b200640 1 -- 172.21.15.19:0/2797223543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd241f5e90 con 0x7fcd24141c10 2024-03-28T13:11:57.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.880+0000 7fcd2b200640 1 -- 172.21.15.19:0/2797223543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd241f6450 con 0x7fcd24141c10 2024-03-28T13:11:57.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.882+0000 7fcd1be00640 1 -- 172.21.15.19:0/2797223543 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcd1c014bc0 con 0x7fcd24141c10 2024-03-28T13:11:57.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.883+0000 7fcd1be00640 1 --2- 172.21.15.19:0/2797223543 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd0406c580 0x7fcd0406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.884+0000 7fcd1be00640 1 -- 172.21.15.19:0/2797223543 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7fcd1c08bf00 con 0x7fcd24141c10 2024-03-28T13:11:57.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.884+0000 7fcd2dfb8740 1 Processor -- start 2024-03-28T13:11:57.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.884+0000 7fcd2dfb8740 1 -- start start 2024-03-28T13:11:57.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.884+0000 7fcd23e00640 1 --2- 172.21.15.19:0/2797223543 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd0406c580 0x7fcd0406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.885+0000 7fcd2dfb8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb4628e560 0x55eb46261500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.885+0000 7fcd2dfb8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb4628e940 0x55eb462912f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.885+0000 7fcd2dfb8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb46261a40 0x55eb46291830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.885+0000 7fcd2dfb8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eb461cc6b0 con 0x55eb46261a40 2024-03-28T13:11:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.885+0000 7fcd2dfb8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eb46174750 con 0x55eb4628e940 2024-03-28T13:11:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.885+0000 7fcd2dfb8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55eb461c8660 con 0x55eb4628e560 2024-03-28T13:11:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.885+0000 7fcd29200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb46261a40 0x55eb46291830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.885+0000 7fcd29200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb46261a40 0x55eb46291830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42018/0 (socket says 172.21.15.19:42018) 2024-03-28T13:11:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.885+0000 7fcd29200640 1 -- 172.21.15.19:0/1286920326 learned_addr learned my addr 172.21.15.19:0/1286920326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.885+0000 7fcd23e00640 1 --2- 172.21.15.19:0/1286920326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb4628e560 0x55eb46261500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.886+0000 7fcd23400640 1 --2- 172.21.15.19:0/1286920326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb4628e940 0x55eb462912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.886+0000 7fcd29200640 1 -- 172.21.15.19:0/1286920326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb4628e560 msgr2=0x55eb46261500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.886+0000 7fcd29200640 1 --2- 172.21.15.19:0/1286920326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb4628e560 0x55eb46261500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.886+0000 7fcd29200640 1 -- 172.21.15.19:0/1286920326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb4628e940 msgr2=0x55eb462912f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.886+0000 7fcd23e00640 1 --2- 172.21.15.19:0/2797223543 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd0406c580 0x7fcd0406ea40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcd14000f60 tx=0x7fcd14007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.886+0000 7fcd29200640 1 --2- 172.21.15.19:0/1286920326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb4628e940 0x55eb462912f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.886+0000 7fcd29200640 1 -- 172.21.15.19:0/1286920326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb45faa7d0 con 0x55eb46261a40 2024-03-28T13:11:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.886+0000 7fcd29200640 1 --2- 172.21.15.19:0/1286920326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb46261a40 0x55eb46291830 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0562f0 tx=0x7fcd1c05a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf2fa69f462690d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.886+0000 7fcd18c00640 1 -- 172.21.15.19:0/1286920326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c05e240 con 0x55eb46261a40 2024-03-28T13:11:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.886+0000 7fcd18c00640 1 -- 172.21.15.19:0/1286920326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd1c062a70 con 0x55eb46261a40 2024-03-28T13:11:57.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.886+0000 7fcd18c00640 1 -- 172.21.15.19:0/1286920326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c058190 con 0x55eb46261a40 2024-03-28T13:11:57.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.887+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/1286920326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb46261a40 msgr2=0x55eb46291830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.887+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/1286920326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb46261a40 0x55eb46291830 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c0562f0 tx=0x7fcd1c05a590 comp rx=0 tx=0).stop 2024-03-28T13:11:57.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.887+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/1286920326 shutdown_connections 2024-03-28T13:11:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.887+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/1286920326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb46261a40 0x55eb46291830 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.887+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/1286920326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb4628e940 0x55eb462912f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.887+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/1286920326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb4628e560 0x55eb46261500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.887+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/1286920326 >> 172.21.15.19:0/1286920326 conn(0x55eb46263770 msgr2=0x55eb462ac2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.887+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/1286920326 shutdown_connections 2024-03-28T13:11:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.887+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/1286920326 wait complete. 2024-03-28T13:11:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.888+0000 7fcd2dfb8740 1 -- start start 2024-03-28T13:11:57.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.889+0000 7fcd2dfb8740 1 Processor -- start 2024-03-28T13:11:57.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.890+0000 7fcd2dfb8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb4628e560 0x55eb4628dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.890+0000 7fcd2dfb8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4628e940 0x55eb46299a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.890+0000 7fcd2dfb8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb46299f60 0x55eb46293bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.890+0000 7fcd2dfb8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eb46265e10 con 0x55eb4628e940 2024-03-28T13:11:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.890+0000 7fcd2dfb8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eb46187a80 con 0x55eb4628e560 2024-03-28T13:11:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.890+0000 7fcd2dfb8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55eb46268650 con 0x55eb46299f60 2024-03-28T13:11:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.891+0000 7fcd2dfb8740 10 client.0 _ll_register_callbacks cb 0x55eb46187470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:11:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.891+0000 7fcd23400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4628e940 0x55eb46299a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.891+0000 7fcd23400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4628e940 0x55eb46299a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42020/0 (socket says 172.21.15.19:42020) 2024-03-28T13:11:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.891+0000 7fcd23400640 1 -- 172.21.15.19:0/123873457 learned_addr learned my addr 172.21.15.19:0/123873457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.891+0000 7fcd23e00640 1 --2- 172.21.15.19:0/123873457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb4628e560 0x55eb4628dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.891+0000 7fcd29200640 1 --2- 172.21.15.19:0/123873457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb46299f60 0x55eb46293bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.891+0000 7fcd23400640 1 -- 172.21.15.19:0/123873457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb46299f60 msgr2=0x55eb46293bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.891+0000 7fcd23400640 1 --2- 172.21.15.19:0/123873457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb46299f60 0x55eb46293bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.891+0000 7fcd23400640 1 -- 172.21.15.19:0/123873457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb4628e560 msgr2=0x55eb4628dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.891+0000 7fcd23400640 1 --2- 172.21.15.19:0/123873457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb4628e560 0x55eb4628dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.891+0000 7fcd23400640 1 -- 172.21.15.19:0/123873457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb45ee03f0 con 0x55eb4628e940 2024-03-28T13:11:57.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.891+0000 7fcd23e00640 1 --2- 172.21.15.19:0/123873457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb4628e560 0x55eb4628dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:11:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.892+0000 7fcd23400640 1 --2- 172.21.15.19:0/123873457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4628e940 0x55eb46299a20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fcd100097a0 tx=0x7fcd10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dbac4b2eaa7864 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.892+0000 7fcd0d600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:11:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.892+0000 7fcd0d600640 1 -- 172.21.15.19:0/123873457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10049060 con 0x55eb4628e940 2024-03-28T13:11:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.892+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/123873457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd1c062a70 con 0x55eb4628e940 2024-03-28T13:11:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.892+0000 7fcd0d600640 1 -- 172.21.15.19:0/123873457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7fcd10003bd0 con 0x55eb4628e940 2024-03-28T13:11:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.892+0000 7fcd0d600640 1 -- 172.21.15.19:0/123873457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcd1004f050 con 0x55eb4628e940 2024-03-28T13:11:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.892+0000 7fcd2dfb8740 10 client.5739 fetch_fsmap learned FSMap version 26 2024-03-28T13:11:57.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.892+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/123873457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55eb45ea00f0 con 0x55eb4628e940 2024-03-28T13:11:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.893+0000 7fcd0d600640 1 -- 172.21.15.19:0/123873457 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd1c062a70 con 0x55eb4628e940 2024-03-28T13:11:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.893+0000 7fcd0d600640 1 -- 172.21.15.19:0/123873457 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fcd10052020 con 0x55eb4628e940 2024-03-28T13:11:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.893+0000 7fcd2dfb8740 10 client.5739 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T13:11:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.893+0000 7fcd2dfb8740 10 client.5739 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-03-28T13:11:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.893+0000 7fcd2dfb8740 20 client.5739 populate_metadata read hostname 'smithi019' 2024-03-28T13:11:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.893+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/123873457 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55eb46297560 0x55eb462bffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.893+0000 7fcd2dfb8740 4 client.5739 mds_command: new command op to 5538 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:11:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.893+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/123873457 --> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb461d9bf0 con 0x55eb46297560 2024-03-28T13:11:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.893+0000 7fcd23e00640 1 --2- 172.21.15.19:0/123873457 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55eb46297560 0x55eb462bffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.894+0000 7fcd23e00640 1 --2- 172.21.15.19:0/123873457 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55eb46297560 0x55eb462bffc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37b25eefe024fdce server_cookie=ac80f6f3dba8fcab in_seq=0 out_seq=0 2024-03-28T13:11:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.894+0000 7fcd0d600640 10 client.5739 ms_handle_connect on v2:172.21.15.19:6833/179338886 2024-03-28T13:11:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.895+0000 7fcd0d600640 1 -- 172.21.15.19:0/123873457 <== mds.0 v2:172.21.15.19:6833/179338886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eb461d9bf0 con 0x55eb46297560 2024-03-28T13:11:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.895+0000 7fcd0d600640 10 client.5739 handle_command_reply: tid=0 2024-03-28T13:11:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.895+0000 7fcd2dfb8740 1 client.5739 shutdown 2024-03-28T13:11:57.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.895+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/123873457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4628e940 msgr2=0x55eb46299a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.895+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/123873457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4628e940 0x55eb46299a20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fcd100097a0 tx=0x7fcd10004730 comp rx=0 tx=0).stop 2024-03-28T13:11:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.895+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/123873457 shutdown_connections 2024-03-28T13:11:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.895+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/123873457 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55eb46297560 0x55eb462bffc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.895+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/123873457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb46299f60 0x55eb46293bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.895+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/123873457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4628e940 0x55eb46299a20 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.895+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/123873457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb4628e560 0x55eb4628dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.895+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/123873457 >> 172.21.15.19:0/123873457 conn(0x55eb46261a40 msgr2=0x55eb46265550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.895+0000 7fcd0d600640 0 client.5739 ms_handle_reset on v2:172.21.15.19:6833/179338886 2024-03-28T13:11:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.895+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/123873457 shutdown_connections 2024-03-28T13:11:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.895+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/123873457 wait complete. 2024-03-28T13:11:57.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.895+0000 7fcd2dfb8740 20 client.5739 trim_cache size 0 max 16384 2024-03-28T13:11:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.916+0000 7fcd2dfb8740 1 Processor -- start 2024-03-28T13:11:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.917+0000 7fcd2dfb8740 1 -- start start 2024-03-28T13:11:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.917+0000 7fcd2dfb8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb4628e560 0x55eb462af060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.917+0000 7fcd2dfb8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb4628e940 0x55eb462af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.917+0000 7fcd2dfb8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb46299f60 0x55eb46260390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.917+0000 7fcd2dfb8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd24200930 con 0x55eb46299f60 2024-03-28T13:11:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.917+0000 7fcd2dfb8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcd241f7630 con 0x55eb4628e560 2024-03-28T13:11:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.917+0000 7fcd2dfb8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcd241f69c0 con 0x55eb4628e940 2024-03-28T13:11:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.918+0000 7fcd29200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb46299f60 0x55eb46260390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.918+0000 7fcd29200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb46299f60 0x55eb46260390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42030/0 (socket says 172.21.15.19:42030) 2024-03-28T13:11:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.918+0000 7fcd29200640 1 -- 172.21.15.19:0/3505320972 learned_addr learned my addr 172.21.15.19:0/3505320972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.918+0000 7fcd29200640 1 -- 172.21.15.19:0/3505320972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb4628e940 msgr2=0x55eb462af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.918+0000 7fcd29200640 1 --2- 172.21.15.19:0/3505320972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb4628e940 0x55eb462af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.918+0000 7fcd29200640 1 -- 172.21.15.19:0/3505320972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb4628e560 msgr2=0x55eb462af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.918+0000 7fcd29200640 1 --2- 172.21.15.19:0/3505320972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb4628e560 0x55eb462af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.918+0000 7fcd29200640 1 -- 172.21.15.19:0/3505320972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb462687f0 con 0x55eb46299f60 2024-03-28T13:11:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.918+0000 7fcd29200640 1 --2- 172.21.15.19:0/3505320972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb46299f60 0x55eb46260390 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c070040 tx=0x7fcd1c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defae179d5ea107a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.919+0000 7fcd0fe00640 1 -- 172.21.15.19:0/3505320972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c076070 con 0x55eb46299f60 2024-03-28T13:11:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.919+0000 7fcd0fe00640 1 -- 172.21.15.19:0/3505320972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd1c062a70 con 0x55eb46299f60 2024-03-28T13:11:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.919+0000 7fcd0fe00640 1 -- 172.21.15.19:0/3505320972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c05e240 con 0x55eb46299f60 2024-03-28T13:11:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.919+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/3505320972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb46299f60 msgr2=0x55eb46260390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.919+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/3505320972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb46299f60 0x55eb46260390 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c070040 tx=0x7fcd1c01f000 comp rx=0 tx=0).stop 2024-03-28T13:11:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.919+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/3505320972 shutdown_connections 2024-03-28T13:11:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.919+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/3505320972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb46299f60 0x55eb46260390 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.919+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/3505320972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb4628e940 0x55eb462af5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.919+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/3505320972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb4628e560 0x55eb462af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.919+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/3505320972 >> 172.21.15.19:0/3505320972 conn(0x55eb46261a40 msgr2=0x55eb4628d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.919+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/3505320972 shutdown_connections 2024-03-28T13:11:57.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.919+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/3505320972 wait complete. 2024-03-28T13:11:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.920+0000 7fcd2dfb8740 1 -- start start 2024-03-28T13:11:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.920+0000 7fcd2dfb8740 1 Processor -- start 2024-03-28T13:11:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.921+0000 7fcd2dfb8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4628e560 0x55eb46323300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.921+0000 7fcd2dfb8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb4628e940 0x55eb46292810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.921+0000 7fcd2dfb8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb46299f60 0x55eb46325d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.921+0000 7fcd2dfb8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd241f6fe0 con 0x55eb4628e560 2024-03-28T13:11:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.921+0000 7fcd2dfb8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eb461c87e0 con 0x55eb46299f60 2024-03-28T13:11:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.921+0000 7fcd2dfb8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55eb46326a10 con 0x55eb4628e940 2024-03-28T13:11:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.921+0000 7fcd2dfb8740 10 client.0 _ll_register_callbacks cb 0x7fcd10004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:11:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.921+0000 7fcd23e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4628e560 0x55eb46323300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.922+0000 7fcd23e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4628e560 0x55eb46323300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42040/0 (socket says 172.21.15.19:42040) 2024-03-28T13:11:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.922+0000 7fcd23e00640 1 -- 172.21.15.19:0/286492589 learned_addr learned my addr 172.21.15.19:0/286492589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:11:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.922+0000 7fcd23e00640 1 -- 172.21.15.19:0/286492589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb4628e940 msgr2=0x55eb46292810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.922+0000 7fcd23e00640 1 --2- 172.21.15.19:0/286492589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb4628e940 0x55eb46292810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.922+0000 7fcd23e00640 1 -- 172.21.15.19:0/286492589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb46299f60 msgr2=0x55eb46325d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:11:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.922+0000 7fcd23e00640 1 --2- 172.21.15.19:0/286492589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb46299f60 0x55eb46325d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.922+0000 7fcd23e00640 1 -- 172.21.15.19:0/286492589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb46292d50 con 0x55eb4628e560 2024-03-28T13:11:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.922+0000 7fcd23e00640 1 --2- 172.21.15.19:0/286492589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4628e560 0x55eb46323300 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fcd14001cd0 tx=0x7fcd14001080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0b6699bccd876b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:11:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.922+0000 7fcd0d600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:11:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.923+0000 7fcd0d600640 1 -- 172.21.15.19:0/286492589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14026e60 con 0x55eb4628e560 2024-03-28T13:11:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.923+0000 7fcd0d600640 1 -- 172.21.15.19:0/286492589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7fcd14022aa0 con 0x55eb4628e560 2024-03-28T13:11:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.923+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/286492589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd1c062a70 con 0x55eb4628e560 2024-03-28T13:11:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.923+0000 7fcd0d600640 1 -- 172.21.15.19:0/286492589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcd1401b040 con 0x55eb4628e560 2024-03-28T13:11:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.923+0000 7fcd2dfb8740 10 client.5745 fetch_fsmap learned FSMap version 26 2024-03-28T13:11:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.923+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/286492589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55eb46326750 con 0x55eb4628e560 2024-03-28T13:11:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.923+0000 7fcd0d600640 1 -- 172.21.15.19:0/286492589 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd1c062a70 con 0x55eb4628e560 2024-03-28T13:11:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.923+0000 7fcd0d600640 1 -- 172.21.15.19:0/286492589 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7fcd14022ca0 con 0x55eb4628e560 2024-03-28T13:11:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.923+0000 7fcd2dfb8740 10 client.5745 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T13:11:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.923+0000 7fcd2dfb8740 10 client.5745 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-03-28T13:11:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.923+0000 7fcd2dfb8740 20 client.5745 populate_metadata read hostname 'smithi019' 2024-03-28T13:11:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.923+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/286492589 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55eb462c6d30 0x55eb462bcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:11:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.923+0000 7fcd2dfb8740 4 client.5745 mds_command: new command op to 5538 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:11:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.923+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/286492589 --> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eb461d3f40 con 0x55eb462c6d30 2024-03-28T13:11:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.924+0000 7fcd23400640 1 --2- 172.21.15.19:0/286492589 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55eb462c6d30 0x55eb462bcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:11:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.924+0000 7fcd23400640 1 --2- 172.21.15.19:0/286492589 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55eb462c6d30 0x55eb462bcf70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e04dde88b2b426 server_cookie=ece03abeb6e43bad in_seq=0 out_seq=0 2024-03-28T13:11:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.924+0000 7fcd0d600640 10 client.5745 ms_handle_connect on v2:172.21.15.19:6833/179338886 2024-03-28T13:11:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.924+0000 7fcd0d600640 1 -- 172.21.15.19:0/286492589 <== mds.0 v2:172.21.15.19:6833/179338886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55eb461d3f40 con 0x55eb462c6d30 2024-03-28T13:11:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.924+0000 7fcd0d600640 10 client.5745 handle_command_reply: tid=0 2024-03-28T13:11:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.924+0000 7fcd2dfb8740 1 client.5745 shutdown 2024-03-28T13:11:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.925+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/286492589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4628e560 msgr2=0x55eb46323300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.925+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/286492589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4628e560 0x55eb46323300 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fcd14001cd0 tx=0x7fcd14001080 comp rx=0 tx=0).stop 2024-03-28T13:11:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.925+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/286492589 shutdown_connections 2024-03-28T13:11:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.925+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/286492589 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55eb462c6d30 0x55eb462bcf70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.925+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/286492589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eb46299f60 0x55eb46325d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.925+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/286492589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eb4628e940 0x55eb46292810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.925+0000 7fcd2dfb8740 1 --2- 172.21.15.19:0/286492589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eb4628e560 0x55eb46323300 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.925+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/286492589 >> 172.21.15.19:0/286492589 conn(0x55eb46261a40 msgr2=0x55eb462af060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.925+0000 7fcd0d600640 0 client.5745 ms_handle_reset on v2:172.21.15.19:6833/179338886 2024-03-28T13:11:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.925+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/286492589 shutdown_connections 2024-03-28T13:11:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.925+0000 7fcd2dfb8740 1 -- 172.21.15.19:0/286492589 wait complete. 2024-03-28T13:11:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.925+0000 7fcd2dfb8740 20 client.5745 trim_cache size 0 max 16384 2024-03-28T13:11:57.928 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:11:57.928 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T13:11:57.928 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-03-28T13:11:57.928 INFO:teuthology.orchestra.run.smithi019.stdout: "d27d8124-e0b0-47cd-b6cb-3c45edf91f32": { 2024-03-28T13:11:57.928 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-03-28T13:11:57.928 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "d27d8124-e0b0-47cd-b6cb-3c45edf91f32", 2024-03-28T13:11:57.928 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,repair" 2024-03-28T13:11:57.929 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:57.929 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:11:57.929 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:11:57.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.926+0000 7fcd2b200640 1 -- 172.21.15.19:0/2797223543 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd0406c580 msgr2=0x7fcd0406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.926+0000 7fcd2b200640 1 --2- 172.21.15.19:0/2797223543 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd0406c580 0x7fcd0406ea40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcd14000f60 tx=0x7fcd14007d50 comp rx=0 tx=0).stop 2024-03-28T13:11:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.926+0000 7fcd2b200640 1 -- 172.21.15.19:0/2797223543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd24141c10 msgr2=0x7fcd240b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:11:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.926+0000 7fcd2b200640 1 --2- 172.21.15.19:0/2797223543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd24141c10 0x7fcd240b2460 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c007e40 tx=0x7fcd1c00e4b0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.926+0000 7fcd2b200640 1 -- 172.21.15.19:0/2797223543 shutdown_connections 2024-03-28T13:11:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.926+0000 7fcd2b200640 1 --2- 172.21.15.19:0/2797223543 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd0406c580 0x7fcd0406ea40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.926+0000 7fcd2b200640 1 --2- 172.21.15.19:0/2797223543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd24141c10 0x7fcd240b2460 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.926+0000 7fcd2b200640 1 --2- 172.21.15.19:0/2797223543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd240d0280 0x7fcd240b1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.926+0000 7fcd2b200640 1 --2- 172.21.15.19:0/2797223543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd24071eb0 0x7fcd24099ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:11:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.926+0000 7fcd2b200640 1 -- 172.21.15.19:0/2797223543 >> 172.21.15.19:0/2797223543 conn(0x7fcd2406b750 msgr2=0x7fcd24142590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:11:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.926+0000 7fcd2b200640 1 -- 172.21.15.19:0/2797223543 shutdown_connections 2024-03-28T13:11:57.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:57.926+0000 7fcd2b200640 1 -- 172.21.15.19:0/2797223543 wait complete. 2024-03-28T13:11:57.942 INFO:tasks.cephfs.filesystem:scrub status for tag:d27d8124-e0b0-47cd-b6cb-3c45edf91f32 - {'path': '/', 'tag': 'd27d8124-e0b0-47cd-b6cb-3c45edf91f32', 'options': 'recursive,repair'} 2024-03-28T13:11:59.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:11:59.103+0000 7f5c1fe00640 1 -- 172.21.15.19:0/338571930 <== mon.0 v2:172.21.15.19:3300/0 7 ==== log(6 entries from seq 1 at 2024-03-28T13:11:57.626698+0000) ==== 1078+0+0 (secure 0 0 0) 0x7f5c1808b0d0 con 0x7f5c28071850 2024-03-28T13:11:59.104 INFO:teuthology.orchestra.run.smithi019.stdout: cluster: 2024-03-28T13:11:59.104 INFO:teuthology.orchestra.run.smithi019.stdout: id: e6b569d9-f2ce-49dc-bbe7-c0ab33080203 2024-03-28T13:11:59.104 INFO:teuthology.orchestra.run.smithi019.stdout: health: HEALTH_OK 2024-03-28T13:11:59.104 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:59.104 INFO:teuthology.orchestra.run.smithi019.stdout: services: 2024-03-28T13:11:59.104 INFO:teuthology.orchestra.run.smithi019.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-03-28T13:11:59.104 INFO:teuthology.orchestra.run.smithi019.stdout: mgr: x(active, since 2m), standbys: y 2024-03-28T13:11:59.104 INFO:teuthology.orchestra.run.smithi019.stdout: mds: 1/1 daemons up, 3 standby 2024-03-28T13:11:59.104 INFO:teuthology.orchestra.run.smithi019.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-03-28T13:11:59.104 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:59.104 INFO:teuthology.orchestra.run.smithi019.stdout: data: 2024-03-28T13:11:59.105 INFO:teuthology.orchestra.run.smithi019.stdout: volumes: 1/1 healthy 2024-03-28T13:11:59.105 INFO:teuthology.orchestra.run.smithi019.stdout: pools: 3 pools, 97 pgs 2024-03-28T13:11:59.105 INFO:teuthology.orchestra.run.smithi019.stdout: objects: 24 objects, 579 KiB 2024-03-28T13:11:59.105 INFO:teuthology.orchestra.run.smithi019.stdout: usage: 221 MiB used, 720 GiB / 720 GiB avail 2024-03-28T13:11:59.105 INFO:teuthology.orchestra.run.smithi019.stdout: pgs: 97 active+clean 2024-03-28T13:11:59.105 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:59.105 INFO:teuthology.orchestra.run.smithi019.stdout: io: 2024-03-28T13:11:59.105 INFO:teuthology.orchestra.run.smithi019.stdout: client: 1.5 KiB/s wr, 0 op/s rd, 4 op/s wr 2024-03-28T13:11:59.105 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:59.105 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:11:59.105 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:11:57.626698+0000 mds.a [INF] scrub queued for path: / 2024-03-28T13:11:59.105 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:11:57.626703+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-03-28T13:11:59.105 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:11:57.626710+0000 mds.a [INF] scrub summary: active paths [/] 2024-03-28T13:11:59.105 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:11:57.627829+0000 mds.a [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-03-28T13:11:59.105 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:11:57.627842+0000 mds.a [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-03-28T13:11:59.106 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:11:57.629409+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-03-28T13:12:01.627 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:12:01.635 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:12:03.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:03.116+0000 7f5c1fe00640 1 -- 172.21.15.19:0/338571930 <== mon.0 v2:172.21.15.19:3300/0 8 ==== log(1 entries from seq 7 at 2024-03-28T13:12:01.564636+0000) ==== 189+0+0 (secure 0 0 0) 0x7f5c18057350 con 0x7f5c28071850 2024-03-28T13:12:03.116 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:12:01.564636+0000 mds.a [INF] scrub summary: idle 2024-03-28T13:12:23.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:23.178+0000 7f5c1fe00640 1 -- 172.21.15.19:0/338571930 <== mon.0 v2:172.21.15.19:3300/0 9 ==== log(10 entries from seq 131 at 2024-03-28T13:12:22.564139+0000) ==== 2739+0+0 (secure 0 0 0) 0x7f5c180575b0 con 0x7f5c28071850 2024-03-28T13:12:24.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:24.205+0000 7f5c1fe00640 1 -- 172.21.15.19:0/338571930 <== mon.0 v2:172.21.15.19:3300/0 10 ==== log(5 entries from seq 455 at 2024-03-28T13:12:23.186542+0000) ==== 1336+0+0 (secure 0 0 0) 0x7f5c18089890 con 0x7f5c28071850 2024-03-28T13:12:27.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:12:27.944 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-03-28T13:12:28.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.171+0000 7f0c8a400640 1 Processor -- start 2024-03-28T13:12:28.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.171+0000 7f0c8a400640 1 -- start start 2024-03-28T13:12:28.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.173+0000 7f0c8a400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0c8415e660 0x7f0c8406f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.173+0000 7f0c8a400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c840ae6f0 con 0x7f0c8406cc90 2024-03-28T13:12:28.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.173+0000 7f0c8a400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c8406fe10 con 0x7f0c8406f8b0 2024-03-28T13:12:28.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.173+0000 7f0c8a400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0c840703d0 con 0x7f0c8415e660 2024-03-28T13:12:28.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.173+0000 7f0c83e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0c8406cc90 0x7f0c840af170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44978/0 (socket says 172.21.15.19:44978) 2024-03-28T13:12:28.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.173+0000 7f0c83400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0c8415e660 0x7f0c8406f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.173+0000 7f0c83e00640 1 -- 172.21.15.19:0/1242144692 learned_addr learned my addr 172.21.15.19:0/1242144692 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:28.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.174+0000 7f0c83400640 1 -- 172.21.15.19:0/1242144692 >> v1:172.21.15.188:6789/0 conn(0x7f0c8406f8b0 legacy=0x7f0c8406c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:28.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.174+0000 7f0c83400640 1 -- 172.21.15.19:0/1242144692 >> v1:172.21.15.19:6789/0 conn(0x7f0c8406cc90 legacy=0x7f0c840af170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.174+0000 7f0c83400640 1 -- 172.21.15.19:0/1242144692 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c840700f0 con 0x7f0c8415e660 2024-03-28T13:12:28.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.175+0000 7f0c83400640 1 --2- 172.21.15.19:0/1242144692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8415e660 0x7f0c8406f370 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0c740097a0 tx=0x7f0c7402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a84d79b24bd4f7e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:28.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.175+0000 7f0c82000640 1 -- 172.21.15.19:0/1242144692 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c740315a0 con 0x7f0c8415e660 2024-03-28T13:12:28.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.175+0000 7f0c82000640 1 -- 172.21.15.19:0/1242144692 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c74037070 con 0x7f0c8415e660 2024-03-28T13:12:28.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.175+0000 7f0c82000640 1 -- 172.21.15.19:0/1242144692 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74031960 con 0x7f0c8415e660 2024-03-28T13:12:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.176+0000 7f0c8a400640 1 -- 172.21.15.19:0/1242144692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8415e660 msgr2=0x7f0c8406f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.176+0000 7f0c8a400640 1 --2- 172.21.15.19:0/1242144692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8415e660 0x7f0c8406f370 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0c740097a0 tx=0x7f0c7402b840 comp rx=0 tx=0).stop 2024-03-28T13:12:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.176+0000 7f0c8a400640 1 -- 172.21.15.19:0/1242144692 shutdown_connections 2024-03-28T13:12:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.176+0000 7f0c8a400640 1 --2- 172.21.15.19:0/1242144692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8415e660 0x7f0c8406f370 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.176+0000 7f0c8a400640 1 -- 172.21.15.19:0/1242144692 >> 172.21.15.19:0/1242144692 conn(0x7f0c84071e20 msgr2=0x7f0c84072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.176+0000 7f0c8a400640 1 -- 172.21.15.19:0/1242144692 shutdown_connections 2024-03-28T13:12:28.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.176+0000 7f0c8a400640 1 -- 172.21.15.19:0/1242144692 wait complete. 2024-03-28T13:12:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.178+0000 7f0c8a400640 1 Processor -- start 2024-03-28T13:12:28.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.178+0000 7f0c8a400640 1 -- start start 2024-03-28T13:12:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.180+0000 7f0c8a400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8406cc90 0x7f0c84145680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.180+0000 7f0c8a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8406f8b0 0x7f0c841463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.180+0000 7f0c8a400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8415e660 0x7f0c84160910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.180+0000 7f0c8a400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8415ea40 con 0x7f0c8406f8b0 2024-03-28T13:12:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.180+0000 7f0c8a400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c840aec50 con 0x7f0c8415e660 2024-03-28T13:12:28.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.180+0000 7f0c8a400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0c840cea90 con 0x7f0c8406cc90 2024-03-28T13:12:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.180+0000 7f0c82a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8406f8b0 0x7f0c841463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.180+0000 7f0c82a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8406f8b0 0x7f0c841463f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45980/0 (socket says 172.21.15.19:45980) 2024-03-28T13:12:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.180+0000 7f0c82a00640 1 -- 172.21.15.19:0/3651365738 learned_addr learned my addr 172.21.15.19:0/3651365738 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.180+0000 7f0c83400640 1 --2- 172.21.15.19:0/3651365738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8406cc90 0x7f0c84145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.180+0000 7f0c83e00640 1 --2- 172.21.15.19:0/3651365738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8415e660 0x7f0c84160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.181+0000 7f0c82a00640 1 -- 172.21.15.19:0/3651365738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8406cc90 msgr2=0x7f0c84145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.181+0000 7f0c82a00640 1 --2- 172.21.15.19:0/3651365738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8406cc90 0x7f0c84145680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.181+0000 7f0c82a00640 1 -- 172.21.15.19:0/3651365738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8415e660 msgr2=0x7f0c84160910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.181+0000 7f0c82a00640 1 --2- 172.21.15.19:0/3651365738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8415e660 0x7f0c84160910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.181+0000 7f0c82a00640 1 -- 172.21.15.19:0/3651365738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c84151520 con 0x7f0c8406f8b0 2024-03-28T13:12:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.181+0000 7f0c83400640 1 --2- 172.21.15.19:0/3651365738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8406cc90 0x7f0c84145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.181+0000 7f0c82a00640 1 --2- 172.21.15.19:0/3651365738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8406f8b0 0x7f0c841463f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000c430 tx=0x7f0c7000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.181+0000 7f0c7b400640 1 -- 172.21.15.19:0/3651365738 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70011790 con 0x7f0c8406f8b0 2024-03-28T13:12:28.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.181+0000 7f0c7b400640 1 -- 172.21.15.19:0/3651365738 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c70017070 con 0x7f0c8406f8b0 2024-03-28T13:12:28.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.181+0000 7f0c8a400640 1 -- 172.21.15.19:0/3651365738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c84151800 con 0x7f0c8406f8b0 2024-03-28T13:12:28.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.181+0000 7f0c7b400640 1 -- 172.21.15.19:0/3651365738 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70011bb0 con 0x7f0c8406f8b0 2024-03-28T13:12:28.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.181+0000 7f0c8a400640 1 -- 172.21.15.19:0/3651365738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c84151dc0 con 0x7f0c8406f8b0 2024-03-28T13:12:28.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.184+0000 7f0c7b400640 1 -- 172.21.15.19:0/3651365738 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0c7001c460 con 0x7f0c8406f8b0 2024-03-28T13:12:28.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.185+0000 7f0c7b400640 1 --2- 172.21.15.19:0/3651365738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c6406c530 0x7f0c6406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.185+0000 7f0c7b400640 1 -- 172.21.15.19:0/3651365738 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7f0c7008b750 con 0x7f0c8406f8b0 2024-03-28T13:12:28.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.185+0000 7f0c83400640 1 --2- 172.21.15.19:0/3651365738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c6406c530 0x7f0c6406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.186+0000 7f0c8cfeb740 1 Processor -- start 2024-03-28T13:12:28.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.186+0000 7f0c8cfeb740 1 -- start start 2024-03-28T13:12:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.187+0000 7f0c83400640 1 --2- 172.21.15.19:0/3651365738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c6406c530 0x7f0c6406e9f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0c74036000 tx=0x7f0c74008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.187+0000 7f0c8cfeb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31560 0x55e208c04500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.187+0000 7f0c8cfeb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31940 0x55e208c342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.187+0000 7f0c8cfeb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c04a40 0x55e208c34830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.187+0000 7f0c8cfeb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e208b6f6b0 con 0x55e208c31940 2024-03-28T13:12:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.187+0000 7f0c8cfeb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e208b17750 con 0x55e208c04a40 2024-03-28T13:12:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.187+0000 7f0c8cfeb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55e208b6b660 con 0x55e208c31560 2024-03-28T13:12:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.187+0000 7f0c82a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31940 0x55e208c342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.187+0000 7f0c82a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31940 0x55e208c342f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45984/0 (socket says 172.21.15.19:45984) 2024-03-28T13:12:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.187+0000 7f0c82a00640 1 -- 172.21.15.19:0/247483415 learned_addr learned my addr 172.21.15.19:0/247483415 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.187+0000 7f0c83400640 1 --2- 172.21.15.19:0/247483415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31560 0x55e208c04500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.188+0000 7f0c82a00640 1 -- 172.21.15.19:0/247483415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31560 msgr2=0x55e208c04500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.188+0000 7f0c82a00640 1 --2- 172.21.15.19:0/247483415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31560 0x55e208c04500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.188+0000 7f0c83e00640 1 --2- 172.21.15.19:0/247483415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c04a40 0x55e208c34830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.188+0000 7f0c82a00640 1 -- 172.21.15.19:0/247483415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c04a40 msgr2=0x55e208c34830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.188+0000 7f0c82a00640 1 --2- 172.21.15.19:0/247483415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c04a40 0x55e208c34830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.188+0000 7f0c82a00640 1 -- 172.21.15.19:0/247483415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20894d7d0 con 0x55e208c31940 2024-03-28T13:12:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.188+0000 7f0c83e00640 1 --2- 172.21.15.19:0/247483415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c04a40 0x55e208c34830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.188+0000 7f0c82a00640 1 --2- 172.21.15.19:0/247483415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31940 0x55e208c342f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0c700039d0 tx=0x7f0c70059570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e1a29a87716eb4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.188+0000 7f0c6fe00640 1 -- 172.21.15.19:0/247483415 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c700577b0 con 0x55e208c31940 2024-03-28T13:12:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.188+0000 7f0c6fe00640 1 -- 172.21.15.19:0/247483415 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c70011d50 con 0x55e208c31940 2024-03-28T13:12:28.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.188+0000 7f0c6fe00640 1 -- 172.21.15.19:0/247483415 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70061c90 con 0x55e208c31940 2024-03-28T13:12:28.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.189+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/247483415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31940 msgr2=0x55e208c342f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.189+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/247483415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31940 0x55e208c342f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0c700039d0 tx=0x7f0c70059570 comp rx=0 tx=0).stop 2024-03-28T13:12:28.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.189+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/247483415 shutdown_connections 2024-03-28T13:12:28.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.189+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/247483415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c04a40 0x55e208c34830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.189+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/247483415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31940 0x55e208c342f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.189+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/247483415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31560 0x55e208c04500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.189+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/247483415 >> 172.21.15.19:0/247483415 conn(0x55e208c06770 msgr2=0x55e208c4f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.189+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/247483415 shutdown_connections 2024-03-28T13:12:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.189+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/247483415 wait complete. 2024-03-28T13:12:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.190+0000 7f0c8cfeb740 1 -- start start 2024-03-28T13:12:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.190+0000 7f0c8cfeb740 1 Processor -- start 2024-03-28T13:12:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.191+0000 7f0c8cfeb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31560 0x55e208c2ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.191+0000 7f0c8cfeb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31940 0x55e208c305c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.191+0000 7f0c8cfeb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c5fc40 0x55e208c2b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.191+0000 7f0c8cfeb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e208c08e10 con 0x55e208c31560 2024-03-28T13:12:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.191+0000 7f0c8cfeb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e208b2aa80 con 0x55e208c5fc40 2024-03-28T13:12:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.191+0000 7f0c8cfeb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55e208c0b650 con 0x55e208c31940 2024-03-28T13:12:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.191+0000 7f0c8cfeb740 10 client.0 _ll_register_callbacks cb 0x55e208b2a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:12:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.191+0000 7f0c83400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31560 0x55e208c2ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.191+0000 7f0c83400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31560 0x55e208c2ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45994/0 (socket says 172.21.15.19:45994) 2024-03-28T13:12:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.191+0000 7f0c83400640 1 -- 172.21.15.19:0/1612707628 learned_addr learned my addr 172.21.15.19:0/1612707628 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.192+0000 7f0c82a00640 1 --2- 172.21.15.19:0/1612707628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31940 0x55e208c305c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.192+0000 7f0c83e00640 1 --2- 172.21.15.19:0/1612707628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c5fc40 0x55e208c2b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.192+0000 7f0c83400640 1 -- 172.21.15.19:0/1612707628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31940 msgr2=0x55e208c305c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.192+0000 7f0c83400640 1 --2- 172.21.15.19:0/1612707628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31940 0x55e208c305c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.192+0000 7f0c83400640 1 -- 172.21.15.19:0/1612707628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c5fc40 msgr2=0x55e208c2b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.192+0000 7f0c83400640 1 --2- 172.21.15.19:0/1612707628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c5fc40 0x55e208c2b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.192+0000 7f0c83400640 1 -- 172.21.15.19:0/1612707628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2088833f0 con 0x55e208c31560 2024-03-28T13:12:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.192+0000 7f0c83e00640 1 --2- 172.21.15.19:0/1612707628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c5fc40 0x55e208c2b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.192+0000 7f0c83400640 1 --2- 172.21.15.19:0/1612707628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31560 0x55e208c2ac90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0c7402be50 tx=0x7f0c74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af52fb7624b60e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.192+0000 7f0c6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:12:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.193+0000 7f0c6cc00640 1 -- 172.21.15.19:0/1612707628 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74051640 con 0x55e208c31560 2024-03-28T13:12:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.193+0000 7f0c6cc00640 1 -- 172.21.15.19:0/1612707628 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7f0c74047410 con 0x55e208c31560 2024-03-28T13:12:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.193+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/1612707628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c70011d50 con 0x55e208c31560 2024-03-28T13:12:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.193+0000 7f0c6cc00640 1 -- 172.21.15.19:0/1612707628 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0c74047610 con 0x55e208c31560 2024-03-28T13:12:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.193+0000 7f0c8cfeb740 10 client.5760 fetch_fsmap learned FSMap version 26 2024-03-28T13:12:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.193+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/1612707628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55e2088430f0 con 0x55e208c31560 2024-03-28T13:12:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.193+0000 7f0c6cc00640 1 -- 172.21.15.19:0/1612707628 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c70011d50 con 0x55e208c31560 2024-03-28T13:12:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.193+0000 7f0c6cc00640 1 -- 172.21.15.19:0/1612707628 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f0c7404f070 con 0x55e208c31560 2024-03-28T13:12:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.193+0000 7f0c8cfeb740 10 client.5760 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T13:12:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.193+0000 7f0c8cfeb740 10 client.5760 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-03-28T13:12:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.193+0000 7f0c8cfeb740 20 client.5760 populate_metadata read hostname 'smithi019' 2024-03-28T13:12:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.193+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/1612707628 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55e208c29ad0 0x55e208c63230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.193+0000 7f0c8cfeb740 4 client.5760 mds_command: new command op to 5538 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:12:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.193+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/1612707628 --> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e208b7cbf0 con 0x55e208c29ad0 2024-03-28T13:12:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.194+0000 7f0c82a00640 1 --2- 172.21.15.19:0/1612707628 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55e208c29ad0 0x55e208c63230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.194+0000 7f0c82a00640 1 --2- 172.21.15.19:0/1612707628 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55e208c29ad0 0x55e208c63230 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3761fe41a7700130 server_cookie=26a594a9e6f0a215 in_seq=0 out_seq=0 2024-03-28T13:12:28.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.194+0000 7f0c6cc00640 10 client.5760 ms_handle_connect on v2:172.21.15.19:6833/179338886 2024-03-28T13:12:28.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.195+0000 7f0c6cc00640 1 -- 172.21.15.19:0/1612707628 <== mds.0 v2:172.21.15.19:6833/179338886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e208b7cbf0 con 0x55e208c29ad0 2024-03-28T13:12:28.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.195+0000 7f0c6cc00640 10 client.5760 handle_command_reply: tid=0 2024-03-28T13:12:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.195+0000 7f0c8cfeb740 1 client.5760 shutdown 2024-03-28T13:12:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.195+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/1612707628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31560 msgr2=0x55e208c2ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.195+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/1612707628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31560 0x55e208c2ac90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0c7402be50 tx=0x7f0c74004730 comp rx=0 tx=0).stop 2024-03-28T13:12:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.196+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/1612707628 shutdown_connections 2024-03-28T13:12:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.196+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/1612707628 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55e208c29ad0 0x55e208c63230 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.196+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/1612707628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c5fc40 0x55e208c2b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.196+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/1612707628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31940 0x55e208c305c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.196+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/1612707628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31560 0x55e208c2ac90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.196+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/1612707628 >> 172.21.15.19:0/1612707628 conn(0x55e208c04a40 msgr2=0x55e208c08410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.196+0000 7f0c6cc00640 0 client.5760 ms_handle_reset on v2:172.21.15.19:6833/179338886 2024-03-28T13:12:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.196+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/1612707628 shutdown_connections 2024-03-28T13:12:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.196+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/1612707628 wait complete. 2024-03-28T13:12:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.196+0000 7f0c8cfeb740 20 client.5760 trim_cache size 0 max 16384 2024-03-28T13:12:28.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.216+0000 7f0c8cfeb740 1 Processor -- start 2024-03-28T13:12:28.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.216+0000 7f0c8cfeb740 1 -- start start 2024-03-28T13:12:28.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.217+0000 7f0c8cfeb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c31560 0x55e208c2f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.217+0000 7f0c8cfeb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31940 0x55e208c2c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.217+0000 7f0c8cfeb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c5fc40 0x55e208cbf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.217+0000 7f0c8cfeb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84200dc0 con 0x55e208c5fc40 2024-03-28T13:12:28.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.217+0000 7f0c8cfeb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8415ff20 con 0x55e208c31560 2024-03-28T13:12:28.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.217+0000 7f0c8cfeb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0c841f89f0 con 0x55e208c31940 2024-03-28T13:12:28.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.217+0000 7f0c83e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c5fc40 0x55e208cbf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.218+0000 7f0c83e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c5fc40 0x55e208cbf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46000/0 (socket says 172.21.15.19:46000) 2024-03-28T13:12:28.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.218+0000 7f0c83e00640 1 -- 172.21.15.19:0/2326371576 learned_addr learned my addr 172.21.15.19:0/2326371576 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.218+0000 7f0c83e00640 1 -- 172.21.15.19:0/2326371576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31940 msgr2=0x55e208c2c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.218+0000 7f0c83e00640 1 --2- 172.21.15.19:0/2326371576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31940 0x55e208c2c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.218+0000 7f0c83400640 1 --2- 172.21.15.19:0/2326371576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c31560 0x55e208c2f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.218+0000 7f0c83e00640 1 -- 172.21.15.19:0/2326371576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c31560 msgr2=0x55e208c2f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.218+0000 7f0c83e00640 1 --2- 172.21.15.19:0/2326371576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c31560 0x55e208c2f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.218+0000 7f0c83e00640 1 -- 172.21.15.19:0/2326371576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e208c2fcd0 con 0x55e208c5fc40 2024-03-28T13:12:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.218+0000 7f0c83e00640 1 --2- 172.21.15.19:0/2326371576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c5fc40 0x55e208cbf5a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c002030 tx=0x7f0c7c00f600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154c408bad12a3f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.218+0000 7f0c6f400640 1 -- 172.21.15.19:0/2326371576 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c0072d0 con 0x55e208c5fc40 2024-03-28T13:12:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.219+0000 7f0c6f400640 1 -- 172.21.15.19:0/2326371576 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c7c018070 con 0x55e208c5fc40 2024-03-28T13:12:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.219+0000 7f0c6f400640 1 -- 172.21.15.19:0/2326371576 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c0076b0 con 0x55e208c5fc40 2024-03-28T13:12:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.219+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/2326371576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c5fc40 msgr2=0x55e208cbf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.219+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/2326371576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c5fc40 0x55e208cbf5a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c002030 tx=0x7f0c7c00f600 comp rx=0 tx=0).stop 2024-03-28T13:12:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.219+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/2326371576 shutdown_connections 2024-03-28T13:12:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.219+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/2326371576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c5fc40 0x55e208cbf5a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.219+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/2326371576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31940 0x55e208c2c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.219+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/2326371576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c31560 0x55e208c2f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.219+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/2326371576 >> 172.21.15.19:0/2326371576 conn(0x55e208c04a40 msgr2=0x55e208c303d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.219+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/2326371576 shutdown_connections 2024-03-28T13:12:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.219+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/2326371576 wait complete. 2024-03-28T13:12:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.220+0000 7f0c8cfeb740 1 -- start start 2024-03-28T13:12:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.220+0000 7f0c8cfeb740 1 Processor -- start 2024-03-28T13:12:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.221+0000 7f0c8cfeb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31560 0x55e208cbe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.221+0000 7f0c8cfeb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31940 0x55e208c27fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.221+0000 7f0c8cfeb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c5fc40 0x55e208c4ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.221+0000 7f0c8cfeb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e208c349d0 con 0x55e208c31940 2024-03-28T13:12:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.221+0000 7f0c8cfeb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c8415f8d0 con 0x55e208c5fc40 2024-03-28T13:12:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.221+0000 7f0c8cfeb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55e208c4ff90 con 0x55e208c31560 2024-03-28T13:12:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.221+0000 7f0c8cfeb740 10 client.0 _ll_register_callbacks cb 0x7f0c74031e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:12:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.222+0000 7f0c82a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31940 0x55e208c27fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.222+0000 7f0c82a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31940 0x55e208c27fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46012/0 (socket says 172.21.15.19:46012) 2024-03-28T13:12:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.222+0000 7f0c82a00640 1 -- 172.21.15.19:0/4117131357 learned_addr learned my addr 172.21.15.19:0/4117131357 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.222+0000 7f0c83400640 1 --2- 172.21.15.19:0/4117131357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31560 0x55e208cbe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.222+0000 7f0c82a00640 1 -- 172.21.15.19:0/4117131357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31560 msgr2=0x55e208cbe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.222+0000 7f0c82a00640 1 --2- 172.21.15.19:0/4117131357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31560 0x55e208cbe0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.222+0000 7f0c82a00640 1 -- 172.21.15.19:0/4117131357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c5fc40 msgr2=0x55e208c4ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.222+0000 7f0c83e00640 1 --2- 172.21.15.19:0/4117131357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c5fc40 0x55e208c4ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.222+0000 7f0c82a00640 1 --2- 172.21.15.19:0/4117131357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c5fc40 0x55e208c4ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.222+0000 7f0c82a00640 1 -- 172.21.15.19:0/4117131357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e208c69e40 con 0x55e208c31940 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.222+0000 7f0c82a00640 1 --2- 172.21.15.19:0/4117131357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31940 0x55e208c27fd0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0c70059d60 tx=0x7f0c70089a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc61937902aff4c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.222+0000 7f0c6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.223+0000 7f0c6cc00640 1 -- 172.21.15.19:0/4117131357 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c70088040 con 0x55e208c31940 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.223+0000 7f0c6cc00640 1 -- 172.21.15.19:0/4117131357 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7f0c70057550 con 0x55e208c31940 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.223+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/4117131357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c7c018070 con 0x55e208c31940 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.223+0000 7f0c6cc00640 1 -- 172.21.15.19:0/4117131357 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f0c700577b0 con 0x55e208c31940 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.223+0000 7f0c8cfeb740 10 client.5766 fetch_fsmap learned FSMap version 26 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.223+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/4117131357 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55e208c69b10 con 0x55e208c31940 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.223+0000 7f0c6cc00640 1 -- 172.21.15.19:0/4117131357 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c7c018070 con 0x55e208c31940 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.223+0000 7f0c6cc00640 1 -- 172.21.15.19:0/4117131357 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f0c70055b20 con 0x55e208c31940 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.223+0000 7f0c8cfeb740 10 client.5766 fetch_fsmap finished waiting for FSMap version 26 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.223+0000 7f0c8cfeb740 10 client.5766 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.223+0000 7f0c8cfeb740 20 client.5766 populate_metadata read hostname 'smithi019' 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.223+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/4117131357 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55e208c726c0 0x55e208c2ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.223+0000 7f0c8cfeb740 4 client.5766 mds_command: new command op to 5538 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:12:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.223+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/4117131357 --> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e208b76f40 con 0x55e208c726c0 2024-03-28T13:12:28.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.224+0000 7f0c83400640 1 --2- 172.21.15.19:0/4117131357 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55e208c726c0 0x55e208c2ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.224+0000 7f0c83400640 1 --2- 172.21.15.19:0/4117131357 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55e208c726c0 0x55e208c2ab90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d33e71b5043fe67e server_cookie=85ae8d5976cbfd0a in_seq=0 out_seq=0 2024-03-28T13:12:28.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.224+0000 7f0c6cc00640 10 client.5766 ms_handle_connect on v2:172.21.15.19:6833/179338886 2024-03-28T13:12:28.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.225+0000 7f0c6cc00640 1 -- 172.21.15.19:0/4117131357 <== mds.0 v2:172.21.15.19:6833/179338886 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e208b76f40 con 0x55e208c726c0 2024-03-28T13:12:28.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.225+0000 7f0c6cc00640 10 client.5766 handle_command_reply: tid=0 2024-03-28T13:12:28.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.225+0000 7f0c8cfeb740 1 client.5766 shutdown 2024-03-28T13:12:28.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.225+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/4117131357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31940 msgr2=0x55e208c27fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.225+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/4117131357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31940 0x55e208c27fd0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0c70059d60 tx=0x7f0c70089a20 comp rx=0 tx=0).stop 2024-03-28T13:12:28.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.225+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/4117131357 shutdown_connections 2024-03-28T13:12:28.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.225+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/4117131357 >> [v2:172.21.15.19:6833/179338886,v1:172.21.15.19:6835/179338886] conn(0x55e208c726c0 0x55e208c2ab90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.225+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/4117131357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e208c5fc40 0x55e208c4ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.225+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/4117131357 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e208c31940 0x55e208c27fd0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.225+0000 7f0c8cfeb740 1 --2- 172.21.15.19:0/4117131357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e208c31560 0x55e208cbe0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.225+0000 7f0c6cc00640 0 client.5766 ms_handle_reset on v2:172.21.15.19:6833/179338886 2024-03-28T13:12:28.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.225+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/4117131357 >> 172.21.15.19:0/4117131357 conn(0x55e208c04a40 msgr2=0x55e208c2e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:28.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.225+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/4117131357 shutdown_connections 2024-03-28T13:12:28.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.225+0000 7f0c8cfeb740 1 -- 172.21.15.19:0/4117131357 wait complete. 2024-03-28T13:12:28.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.225+0000 7f0c8cfeb740 20 client.5766 trim_cache size 0 max 16384 2024-03-28T13:12:28.228 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:12:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-03-28T13:12:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-03-28T13:12:28.228 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:12:28.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.226+0000 7f0c8a400640 1 -- 172.21.15.19:0/3651365738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c6406c530 msgr2=0x7f0c6406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.226+0000 7f0c8a400640 1 --2- 172.21.15.19:0/3651365738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c6406c530 0x7f0c6406e9f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0c74036000 tx=0x7f0c74008030 comp rx=0 tx=0).stop 2024-03-28T13:12:28.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.226+0000 7f0c8a400640 1 -- 172.21.15.19:0/3651365738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8406f8b0 msgr2=0x7f0c841463f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.226+0000 7f0c8a400640 1 --2- 172.21.15.19:0/3651365738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8406f8b0 0x7f0c841463f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000c430 tx=0x7f0c7000c8f0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.226+0000 7f0c8a400640 1 -- 172.21.15.19:0/3651365738 shutdown_connections 2024-03-28T13:12:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.226+0000 7f0c8a400640 1 --2- 172.21.15.19:0/3651365738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c6406c530 0x7f0c6406e9f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.226+0000 7f0c8a400640 1 --2- 172.21.15.19:0/3651365738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c8415e660 0x7f0c84160910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.226+0000 7f0c8a400640 1 --2- 172.21.15.19:0/3651365738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c8406f8b0 0x7f0c841463f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.226+0000 7f0c8a400640 1 --2- 172.21.15.19:0/3651365738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c8406cc90 0x7f0c84145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.226+0000 7f0c8a400640 1 -- 172.21.15.19:0/3651365738 >> 172.21.15.19:0/3651365738 conn(0x7f0c84071e20 msgr2=0x7f0c8406bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.226+0000 7f0c8a400640 1 -- 172.21.15.19:0/3651365738 shutdown_connections 2024-03-28T13:12:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.226+0000 7f0c8a400640 1 -- 172.21.15.19:0/3651365738 wait complete. 2024-03-28T13:12:28.248 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T13:12:28.348 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:12:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.557+0000 7fc527200640 1 Processor -- start 2024-03-28T13:12:28.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.557+0000 7fc527200640 1 -- start start 2024-03-28T13:12:28.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.558+0000 7fc527200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5280d1ba0 0x7fc5280cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.558+0000 7fc527200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc52815e7e0 con 0x7fc5280d2770 2024-03-28T13:12:28.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.558+0000 7fc527200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc528162300 con 0x7fc52815e400 2024-03-28T13:12:28.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.558+0000 7fc527200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc5281625e0 con 0x7fc5280d1ba0 2024-03-28T13:12:28.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.559+0000 7fc51d400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc5280d2770 0x7fc5280cec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44992/0 (socket says 172.21.15.19:44992) 2024-03-28T13:12:28.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.559+0000 7fc51d400640 1 -- 172.21.15.19:0/2573160578 learned_addr learned my addr 172.21.15.19:0/2573160578 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.559+0000 7fc526800640 1 --2- 172.21.15.19:0/2573160578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5280d1ba0 0x7fc5280cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.559+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1668304786 0 0) 0x7fc52815e7e0 con 0x7fc5280d2770 2024-03-28T13:12:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc50c003520 con 0x7fc5280d2770 2024-03-28T13:12:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1157574799 0 0) 0x7fc528162300 con 0x7fc52815e400 2024-03-28T13:12:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc52815e7e0 con 0x7fc52815e400 2024-03-28T13:12:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1967546880 0 0) 0x7fc50c003520 con 0x7fc5280d2770 2024-03-28T13:12:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5280d1ba0 msgr2=0x7fc5280cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:28.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 --2- 172.21.15.19:0/2573160578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5280d1ba0 0x7fc5280cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 >> v1:172.21.15.188:6789/0 conn(0x7fc52815e400 legacy=0x7fc5280d2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5281628c0 con 0x7fc5280d2770 2024-03-28T13:12:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fc510001670 con 0x7fc5280d2770 2024-03-28T13:12:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 >> v1:172.21.15.19:6789/0 conn(0x7fc5280d2770 legacy=0x7fc5280cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 --2- 172.21.15.19:0/2573160578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc50c003b70 0x7fc50c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 --2- 172.21.15.19:0/2573160578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc50c004800 0x7fc50c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 --2- 172.21.15.19:0/2573160578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5280d1ba0 0x7fc50c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5281625e0 con 0x7fc50c004800 2024-03-28T13:12:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc50c00baa0 con 0x7fc50c003b70 2024-03-28T13:12:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc50c00bca0 con 0x7fc5280d1ba0 2024-03-28T13:12:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.560+0000 7fc51d400640 1 --2- 172.21.15.19:0/2573160578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc50c004800 0x7fc50c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.561+0000 7fc525e00640 1 --2- 172.21.15.19:0/2573160578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc50c003b70 0x7fc50c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.561+0000 7fc51d400640 1 -- 172.21.15.19:0/2573160578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5280d1ba0 msgr2=0x7fc50c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:28.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.561+0000 7fc51d400640 1 --2- 172.21.15.19:0/2573160578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5280d1ba0 0x7fc50c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.561+0000 7fc51d400640 1 -- 172.21.15.19:0/2573160578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc50c003b70 msgr2=0x7fc50c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:28.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.561+0000 7fc51d400640 1 --2- 172.21.15.19:0/2573160578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc50c003b70 0x7fc50c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.561+0000 7fc51d400640 1 -- 172.21.15.19:0/2573160578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50c003520 con 0x7fc50c004800 2024-03-28T13:12:28.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.562+0000 7fc51d400640 1 --2- 172.21.15.19:0/2573160578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc50c004800 0x7fc50c004c80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc510007fc0 tx=0x7fc50c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd45fc8e067a7a62 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:28.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.563+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510007be0 con 0x7fc50c004800 2024-03-28T13:12:28.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.563+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc510030070 con 0x7fc50c004800 2024-03-28T13:12:28.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.563+0000 7fc525400640 1 -- 172.21.15.19:0/2573160578 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc510005d30 con 0x7fc50c004800 2024-03-28T13:12:28.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.563+0000 7fc527200640 1 -- 172.21.15.19:0/2573160578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc50c004800 msgr2=0x7fc50c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.563+0000 7fc527200640 1 --2- 172.21.15.19:0/2573160578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc50c004800 0x7fc50c004c80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc510007fc0 tx=0x7fc50c005830 comp rx=0 tx=0).stop 2024-03-28T13:12:28.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.563+0000 7fc526800640 1 -- 172.21.15.19:0/2573160578 reap_dead start 2024-03-28T13:12:28.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.563+0000 7fc527200640 1 -- 172.21.15.19:0/2573160578 shutdown_connections 2024-03-28T13:12:28.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.563+0000 7fc527200640 1 -- 172.21.15.19:0/2573160578 >> 172.21.15.19:0/2573160578 conn(0x7fc528067280 msgr2=0x7fc528067660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:28.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.564+0000 7fc527200640 1 -- 172.21.15.19:0/2573160578 shutdown_connections 2024-03-28T13:12:28.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.564+0000 7fc527200640 1 -- 172.21.15.19:0/2573160578 wait complete. 2024-03-28T13:12:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.567+0000 7fc527200640 1 Processor -- start 2024-03-28T13:12:28.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.567+0000 7fc527200640 1 -- start start 2024-03-28T13:12:28.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.568+0000 7fc527200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52815ad60 0x7fc5281605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.568+0000 7fc527200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52806cb40 0x7fc52806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.568+0000 7fc527200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc52815e9e0 0x7fc52815ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.568+0000 7fc527200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc5281620e0 con 0x7fc52815ad60 2024-03-28T13:12:28.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.568+0000 7fc527200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc528161f60 con 0x7fc52815e9e0 2024-03-28T13:12:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.568+0000 7fc527200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc52806e2b0 con 0x7fc52806cb40 2024-03-28T13:12:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.569+0000 7fc525e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52815ad60 0x7fc5281605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.569+0000 7fc525e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52815ad60 0x7fc5281605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46032/0 (socket says 172.21.15.19:46032) 2024-03-28T13:12:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.569+0000 7fc525e00640 1 -- 172.21.15.19:0/3265860322 learned_addr learned my addr 172.21.15.19:0/3265860322 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.569+0000 7fc526800640 1 --2- 172.21.15.19:0/3265860322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc52815e9e0 0x7fc52815ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.569+0000 7fc525e00640 1 -- 172.21.15.19:0/3265860322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52806cb40 msgr2=0x7fc52806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.569+0000 7fc525e00640 1 --2- 172.21.15.19:0/3265860322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52806cb40 0x7fc52806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.569+0000 7fc525e00640 1 -- 172.21.15.19:0/3265860322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc52815e9e0 msgr2=0x7fc52815ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.569+0000 7fc525e00640 1 --2- 172.21.15.19:0/3265860322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc52815e9e0 0x7fc52815ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.569+0000 7fc525e00640 1 -- 172.21.15.19:0/3265860322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5281617e0 con 0x7fc52815ad60 2024-03-28T13:12:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.570+0000 7fc525e00640 1 --2- 172.21.15.19:0/3265860322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52815ad60 0x7fc5281605f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc51800f7d0 tx=0x7fc51800fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.570+0000 7fc51ea00640 1 -- 172.21.15.19:0/3265860322 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51801a070 con 0x7fc52815ad60 2024-03-28T13:12:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.570+0000 7fc51ea00640 1 -- 172.21.15.19:0/3265860322 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc51800fe80 con 0x7fc52815ad60 2024-03-28T13:12:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.570+0000 7fc51ea00640 1 -- 172.21.15.19:0/3265860322 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518007050 con 0x7fc52815ad60 2024-03-28T13:12:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.570+0000 7fc527200640 1 -- 172.21.15.19:0/3265860322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5280b6c90 con 0x7fc52815ad60 2024-03-28T13:12:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.570+0000 7fc527200640 1 -- 172.21.15.19:0/3265860322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5280b5ed0 con 0x7fc52815ad60 2024-03-28T13:12:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.571+0000 7fc527200640 1 -- 172.21.15.19:0/3265860322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52806d500 con 0x7fc52815ad60 2024-03-28T13:12:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.572+0000 7fc51ea00640 1 -- 172.21.15.19:0/3265860322 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc5180071f0 con 0x7fc52815ad60 2024-03-28T13:12:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.574+0000 7fc51ea00640 1 --2- 172.21.15.19:0/3265860322 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc50406c530 0x7fc50406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.574+0000 7fc51ea00640 1 -- 172.21.15.19:0/3265860322 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7fc51808af10 con 0x7fc52815ad60 2024-03-28T13:12:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.575+0000 7fc51d400640 1 --2- 172.21.15.19:0/3265860322 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc50406c530 0x7fc50406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:28.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.576+0000 7fc51d400640 1 --2- 172.21.15.19:0/3265860322 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc50406c530 0x7fc50406e9f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc510034000 tx=0x7fc510007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:28.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.578+0000 7fc51ea00640 1 -- 172.21.15.19:0/3265860322 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc518059490 con 0x7fc52815ad60 2024-03-28T13:12:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.824+0000 7fc527200640 1 -- 172.21.15.19:0/3265860322 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc528075940 con 0x7fc52815ad60 2024-03-28T13:12:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.825+0000 7fc51ea00640 1 -- 172.21.15.19:0/3265860322 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7fc518058d90 con 0x7fc52815ad60 2024-03-28T13:12:28.826 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:28.826 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2125648249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2125648249},{"type":"v1","addr":"172.21.15.188:6836","nonce":2125648249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/429866161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":429866161},{"type":"v1","addr":"172.21.15.188:6837","nonce":429866161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1422209289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1422209289},{"type":"v1","addr":"172.21.15.19:6834","nonce":1422209289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:11:46.379929+0000","modified":"2024-03-28T13:11:49.470791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5538},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5538":{"gid":5538,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/179338886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":179338886},{"type":"v1","addr":"172.21.15.19:6835","nonce":179338886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5538,"qdb_cluster":[5538]},"id":3}]} 2024-03-28T13:12:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-03-28T13:12:28.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.828+0000 7fc527200640 1 -- 172.21.15.19:0/3265860322 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc50406c530 msgr2=0x7fc50406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.828+0000 7fc527200640 1 --2- 172.21.15.19:0/3265860322 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc50406c530 0x7fc50406e9f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc510034000 tx=0x7fc510007000 comp rx=0 tx=0).stop 2024-03-28T13:12:28.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.828+0000 7fc527200640 1 -- 172.21.15.19:0/3265860322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52815ad60 msgr2=0x7fc5281605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:28.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.828+0000 7fc527200640 1 --2- 172.21.15.19:0/3265860322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52815ad60 0x7fc5281605f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc51800f7d0 tx=0x7fc51800fc90 comp rx=0 tx=0).stop 2024-03-28T13:12:28.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.828+0000 7fc527200640 1 -- 172.21.15.19:0/3265860322 shutdown_connections 2024-03-28T13:12:28.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.828+0000 7fc527200640 1 --2- 172.21.15.19:0/3265860322 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc50406c530 0x7fc50406e9f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.828+0000 7fc527200640 1 --2- 172.21.15.19:0/3265860322 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc52815e9e0 0x7fc52815ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.828+0000 7fc527200640 1 --2- 172.21.15.19:0/3265860322 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc52806cb40 0x7fc52806cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.828+0000 7fc527200640 1 --2- 172.21.15.19:0/3265860322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc52815ad60 0x7fc5281605f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:28.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.828+0000 7fc527200640 1 -- 172.21.15.19:0/3265860322 >> 172.21.15.19:0/3265860322 conn(0x7fc528067280 msgr2=0x7fc52815c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:28.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.828+0000 7fc527200640 1 -- 172.21.15.19:0/3265860322 shutdown_connections 2024-03-28T13:12:28.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:28.828+0000 7fc527200640 1 -- 172.21.15.19:0/3265860322 wait complete. 2024-03-28T13:12:28.854 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-03-28T13:12:29.113 INFO:teuthology.orchestra.run.smithi019.stdout:{"message":"","return_code":0} 2024-03-28T13:12:29.113 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-28T13:12:29.113 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-03-28T13:12:29.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.223+0000 7f2f4c440f40 1 Processor -- start 2024-03-28T13:12:29.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.224+0000 7f2f4c440f40 1 -- start start 2024-03-28T13:12:29.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f4c440f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56266b436240 0x56266b423d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:29.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f4c440f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266b43c270 0x56266b3968c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:29.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f4c440f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56266b43cbe0 0x56266b43d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:29.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f4c440f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56266b3af160 con 0x56266b43c270 2024-03-28T13:12:29.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f4c440f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56266b30b2c0 con 0x56266b43cbe0 2024-03-28T13:12:29.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f4c440f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x56266b3658d0 con 0x56266b436240 2024-03-28T13:12:29.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f48400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266b43c270 0x56266b3968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:29.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f48400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266b43c270 0x56266b3968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46048/0 (socket says 172.21.15.19:46048) 2024-03-28T13:12:29.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f48400640 1 -- 172.21.15.19:0/3103915551 learned_addr learned my addr 172.21.15.19:0/3103915551 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:29.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f49800640 1 --2- 172.21.15.19:0/3103915551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56266b43cbe0 0x56266b43d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:29.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f48e00640 1 --2- 172.21.15.19:0/3103915551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56266b436240 0x56266b423d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:29.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f48400640 1 -- 172.21.15.19:0/3103915551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56266b436240 msgr2=0x56266b423d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:29.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f48400640 1 --2- 172.21.15.19:0/3103915551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56266b436240 0x56266b423d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:29.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f48400640 1 -- 172.21.15.19:0/3103915551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56266b43cbe0 msgr2=0x56266b43d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:29.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f48400640 1 --2- 172.21.15.19:0/3103915551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56266b43cbe0 0x56266b43d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:29.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f48400640 1 -- 172.21.15.19:0/3103915551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56266b3641f0 con 0x56266b43c270 2024-03-28T13:12:29.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.225+0000 7f2f48e00640 1 --2- 172.21.15.19:0/3103915551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56266b436240 0x56266b423d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.226+0000 7f2f48400640 1 --2- 172.21.15.19:0/3103915551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266b43c270 0x56266b3968c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000a770 tx=0x7f2f3000a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ba2ae9e3c78012 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.226+0000 7f2f47000640 1 -- 172.21.15.19:0/3103915551 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30018070 con 0x56266b43c270 2024-03-28T13:12:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.226+0000 7f2f47000640 1 -- 172.21.15.19:0/3103915551 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f30007600 con 0x56266b43c270 2024-03-28T13:12:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.226+0000 7f2f47000640 1 -- 172.21.15.19:0/3103915551 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f300079b0 con 0x56266b43c270 2024-03-28T13:12:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.227+0000 7f2f4c440f40 1 -- 172.21.15.19:0/3103915551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266b43c270 msgr2=0x56266b3968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.227+0000 7f2f4c440f40 1 --2- 172.21.15.19:0/3103915551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266b43c270 0x56266b3968c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000a770 tx=0x7f2f3000a860 comp rx=0 tx=0).stop 2024-03-28T13:12:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.227+0000 7f2f4c440f40 1 -- 172.21.15.19:0/3103915551 shutdown_connections 2024-03-28T13:12:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.227+0000 7f2f4c440f40 1 --2- 172.21.15.19:0/3103915551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56266b43cbe0 0x56266b43d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.227+0000 7f2f4c440f40 1 --2- 172.21.15.19:0/3103915551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266b43c270 0x56266b3968c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.227+0000 7f2f4c440f40 1 --2- 172.21.15.19:0/3103915551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56266b436240 0x56266b423d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.227+0000 7f2f4c440f40 1 -- 172.21.15.19:0/3103915551 >> 172.21.15.19:0/3103915551 conn(0x56266b3b2630 msgr2=0x56266b393730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.227+0000 7f2f4c440f40 1 -- 172.21.15.19:0/3103915551 shutdown_connections 2024-03-28T13:12:29.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.227+0000 7f2f4c440f40 1 -- 172.21.15.19:0/3103915551 wait complete. 2024-03-28T13:12:29.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.228+0000 7f2f4c440f40 1 Processor -- start 2024-03-28T13:12:29.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.228+0000 7f2f4c440f40 1 -- start start 2024-03-28T13:12:29.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.229+0000 7f2f4c440f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56266b436240 0x56266b42fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:29.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.229+0000 7f2f4c440f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56266b43c270 0x56266b41e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:29.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.229+0000 7f2f4c440f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266b43cbe0 0x56266b396d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:29.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.229+0000 7f2f4c440f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56266b431940 con 0x56266b43cbe0 2024-03-28T13:12:29.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.229+0000 7f2f4c440f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56266b4323c0 con 0x56266b43c270 2024-03-28T13:12:29.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.229+0000 7f2f4c440f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x56266b431c20 con 0x56266b436240 2024-03-28T13:12:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.229+0000 7f2f49800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266b43cbe0 0x56266b396d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.229+0000 7f2f49800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266b43cbe0 0x56266b396d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46052/0 (socket says 172.21.15.19:46052) 2024-03-28T13:12:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.229+0000 7f2f49800640 1 -- 172.21.15.19:0/3629874081 learned_addr learned my addr 172.21.15.19:0/3629874081 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.229+0000 7f2f48e00640 1 --2- 172.21.15.19:0/3629874081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56266b436240 0x56266b42fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.230+0000 7f2f48400640 1 --2- 172.21.15.19:0/3629874081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56266b43c270 0x56266b41e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.230+0000 7f2f49800640 1 -- 172.21.15.19:0/3629874081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56266b436240 msgr2=0x56266b42fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.230+0000 7f2f49800640 1 --2- 172.21.15.19:0/3629874081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56266b436240 0x56266b42fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.230+0000 7f2f49800640 1 -- 172.21.15.19:0/3629874081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56266b43c270 msgr2=0x56266b41e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.230+0000 7f2f49800640 1 --2- 172.21.15.19:0/3629874081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56266b43c270 0x56266b41e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.230+0000 7f2f49800640 1 -- 172.21.15.19:0/3629874081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56266b420300 con 0x56266b43cbe0 2024-03-28T13:12:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.230+0000 7f2f49800640 1 --2- 172.21.15.19:0/3629874081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266b43cbe0 0x56266b396d60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2f400041b0 tx=0x7f2f4000d530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.230+0000 7f2f45c00640 1 -- 172.21.15.19:0/3629874081 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40018070 con 0x56266b43cbe0 2024-03-28T13:12:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.230+0000 7f2f45c00640 1 -- 172.21.15.19:0/3629874081 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f400030d0 con 0x56266b43cbe0 2024-03-28T13:12:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.230+0000 7f2f4c440f40 1 -- 172.21.15.19:0/3629874081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56266b4205e0 con 0x56266b43cbe0 2024-03-28T13:12:29.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.230+0000 7f2f45c00640 1 -- 172.21.15.19:0/3629874081 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f400014a0 con 0x56266b43cbe0 2024-03-28T13:12:29.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.231+0000 7f2f4c440f40 1 -- 172.21.15.19:0/3629874081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56266b420ba0 con 0x56266b43cbe0 2024-03-28T13:12:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.234+0000 7f2f45c00640 1 -- 172.21.15.19:0/3629874081 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2f4001b070 con 0x56266b43cbe0 2024-03-28T13:12:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.235+0000 7f2f45c00640 1 --2- 172.21.15.19:0/3629874081 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2f2406af10 0x7f2f2406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:29.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.235+0000 7f2f45c00640 1 -- 172.21.15.19:0/3629874081 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7f2f40089a60 con 0x56266b43cbe0 2024-03-28T13:12:29.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.235+0000 7f2f4c440f40 1 --2- 172.21.15.19:0/3629874081 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x56266b41e8e0 0x56266b434430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:29.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.235+0000 7f2f4c440f40 1 -- 172.21.15.19:0/3629874081 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51) -- 0x56266b434970 con 0x56266b41e8e0 2024-03-28T13:12:29.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.235+0000 7f2f48e00640 1 --2- 172.21.15.19:0/3629874081 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2f2406af10 0x7f2f2406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:29.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.236+0000 7f2f48400640 1 --2- 172.21.15.19:0/3629874081 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x56266b41e8e0 0x56266b434430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:29.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.236+0000 7f2f48400640 1 --2- 172.21.15.19:0/3629874081 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x56266b41e8e0 0x56266b434430 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:29.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.237+0000 7f2f48400640 1 -- 172.21.15.19:0/3629874081 <== osd.3 v2:172.21.15.19:6819/3265515865 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f2f30007b30 con 0x56266b41e8e0 2024-03-28T13:12:29.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.237+0000 7f2f4c440f40 1 -- 172.21.15.19:0/3629874081 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x56266b41e8e0 msgr2=0x56266b434430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:29.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.237+0000 7f2f4c440f40 1 --2- 172.21.15.19:0/3629874081 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x56266b41e8e0 0x56266b434430 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.237+0000 7f2f48e00640 1 --2- 172.21.15.19:0/3629874081 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2f2406af10 0x7f2f2406d3d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2f38054000 tx=0x7f2f3804e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.237+0000 7f2f4c440f40 1 -- 172.21.15.19:0/3629874081 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2f2406af10 msgr2=0x7f2f2406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.237+0000 7f2f4c440f40 1 --2- 172.21.15.19:0/3629874081 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2f2406af10 0x7f2f2406d3d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2f38054000 tx=0x7f2f3804e000 comp rx=0 tx=0).stop 2024-03-28T13:12:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.237+0000 7f2f4c440f40 1 -- 172.21.15.19:0/3629874081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266b43cbe0 msgr2=0x56266b396d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.237+0000 7f2f4c440f40 1 --2- 172.21.15.19:0/3629874081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56266b43cbe0 0x56266b396d60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2f400041b0 tx=0x7f2f4000d530 comp rx=0 tx=0).stop 2024-03-28T13:12:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.237+0000 7f2f49800640 1 -- 172.21.15.19:0/3629874081 reap_dead start 2024-03-28T13:12:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.237+0000 7f2f4c440f40 1 -- 172.21.15.19:0/3629874081 shutdown_connections 2024-03-28T13:12:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.237+0000 7f2f4c440f40 1 -- 172.21.15.19:0/3629874081 >> 172.21.15.19:0/3629874081 conn(0x56266b3b2630 msgr2=0x56266b362e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.237+0000 7f2f4c440f40 1 -- 172.21.15.19:0/3629874081 shutdown_connections 2024-03-28T13:12:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.237+0000 7f2f4c440f40 1 -- 172.21.15.19:0/3629874081 wait complete. 2024-03-28T13:12:29.245 INFO:teuthology.orchestra.run.smithi019.stdout:Yalphaparent_a 2024-03-28T13:12:29.245 DEBUG:teuthology.orchestra.run.smithi019:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-03-28T13:12:29.353 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:12:29.353 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511627777, 2024-03-28T13:12:29.353 INFO:teuthology.orchestra.run.smithi019.stdout: "ancestors": [ 2024-03-28T13:12:29.353 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:29.353 INFO:teuthology.orchestra.run.smithi019.stdout: "dirino": 1099511627776, 2024-03-28T13:12:29.353 INFO:teuthology.orchestra.run.smithi019.stdout: "dname": "alpha", 2024-03-28T13:12:29.353 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 4 2024-03-28T13:12:29.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:29.353 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:29.354 INFO:teuthology.orchestra.run.smithi019.stdout: "dirino": 1, 2024-03-28T13:12:29.354 INFO:teuthology.orchestra.run.smithi019.stdout: "dname": "parent_a", 2024-03-28T13:12:29.354 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 7 2024-03-28T13:12:29.354 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:29.354 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-03-28T13:12:29.354 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 7, 2024-03-28T13:12:29.354 INFO:teuthology.orchestra.run.smithi019.stdout: "old_pools": [] 2024-03-28T13:12:29.354 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:12:29.354 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:29.360 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-28T13:12:29.360 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:12:29.475 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:12:29.610 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-28T13:12:29.611 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:12:29.611 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:12:29.611 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:12:29.671 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:12:29.671 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:12:29.721 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:12:29.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.983+0000 7f084aa00640 1 Processor -- start 2024-03-28T13:12:29.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.983+0000 7f084aa00640 1 -- start start 2024-03-28T13:12:29.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.985+0000 7f084aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f08440cc0b0 0x7f084406dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:29.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.985+0000 7f084aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08440d04d0 0x7f084415ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:29.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.985+0000 7f084aa00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f084406e4f0 con 0x7f084415fc50 2024-03-28T13:12:29.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.985+0000 7f084aa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f084415c610 con 0x7f08440cc0b0 2024-03-28T13:12:29.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.985+0000 7f084aa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f08440b3150 con 0x7f08440d04d0 2024-03-28T13:12:29.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.985+0000 7f0849600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f08440cc0b0 0x7f084406dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:29.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.985+0000 7f0848c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f084415fc50 0x7f08440cfdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45004/0 (socket says 172.21.15.19:45004) 2024-03-28T13:12:29.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.985+0000 7f0848c00640 1 -- 172.21.15.19:0/351722251 learned_addr learned my addr 172.21.15.19:0/351722251 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:29.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.986+0000 7f084a000640 1 --2- 172.21.15.19:0/351722251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08440d04d0 0x7f084415ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:29.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.986+0000 7f0849600640 1 -- 172.21.15.19:0/351722251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08440d04d0 msgr2=0x7f084415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:29.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.986+0000 7f0849600640 1 --2- 172.21.15.19:0/351722251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08440d04d0 0x7f084415ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:29.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.986+0000 7f0849600640 1 -- 172.21.15.19:0/351722251 >> v1:172.21.15.19:6789/0 conn(0x7f084415fc50 legacy=0x7f08440cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:29.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.986+0000 7f0849600640 1 -- 172.21.15.19:0/351722251 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08441600e0 con 0x7f08440cc0b0 2024-03-28T13:12:29.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.986+0000 7f0843e00640 1 -- 172.21.15.19:0/351722251 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1793076444 0 0) 0x7f084406e4f0 con 0x7f084415fc50 2024-03-28T13:12:29.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.987+0000 7f0849600640 1 --2- 172.21.15.19:0/351722251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08440cc0b0 0x7f084406dfb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f08380097a0 tx=0x7f083802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a7fcf57b037380 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:29.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.988+0000 7f0843e00640 1 -- 172.21.15.19:0/351722251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838035020 con 0x7f08440cc0b0 2024-03-28T13:12:29.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.988+0000 7f0843e00640 1 -- 172.21.15.19:0/351722251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0838031830 con 0x7f08440cc0b0 2024-03-28T13:12:29.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.988+0000 7f0843e00640 1 -- 172.21.15.19:0/351722251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838031c00 con 0x7f08440cc0b0 2024-03-28T13:12:29.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.988+0000 7f084aa00640 1 -- 172.21.15.19:0/351722251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08440cc0b0 msgr2=0x7f084406dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:29.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.988+0000 7f084aa00640 1 --2- 172.21.15.19:0/351722251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08440cc0b0 0x7f084406dfb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f08380097a0 tx=0x7f083802b8d0 comp rx=0 tx=0).stop 2024-03-28T13:12:29.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.988+0000 7f084aa00640 1 -- 172.21.15.19:0/351722251 shutdown_connections 2024-03-28T13:12:29.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.988+0000 7f084aa00640 1 --2- 172.21.15.19:0/351722251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08440d04d0 0x7f084415ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:29.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.988+0000 7f084aa00640 1 --2- 172.21.15.19:0/351722251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08440cc0b0 0x7f084406dfb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:29.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.988+0000 7f084aa00640 1 -- 172.21.15.19:0/351722251 >> 172.21.15.19:0/351722251 conn(0x7f08440ccad0 msgr2=0x7f08440cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:29.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.988+0000 7f084aa00640 1 -- 172.21.15.19:0/351722251 shutdown_connections 2024-03-28T13:12:29.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.988+0000 7f084aa00640 1 -- 172.21.15.19:0/351722251 wait complete. 2024-03-28T13:12:29.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.990+0000 7f084aa00640 1 Processor -- start 2024-03-28T13:12:29.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.990+0000 7f084aa00640 1 -- start start 2024-03-28T13:12:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.991+0000 7f084aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08440cc0b0 0x7f0844139b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.991+0000 7f084aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08440d04d0 0x7f0844155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.991+0000 7f084aa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0844154f20 0x7f0844135af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.991+0000 7f084aa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0844160980 con 0x7f08440cc0b0 2024-03-28T13:12:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.991+0000 7f084aa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f08441606a0 con 0x7f0844154f20 2024-03-28T13:12:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.991+0000 7f084aa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f08440cc490 con 0x7f08440d04d0 2024-03-28T13:12:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.991+0000 7f0849600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08440cc0b0 0x7f0844139b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:29.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.991+0000 7f0849600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08440cc0b0 0x7f0844139b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46066/0 (socket says 172.21.15.19:46066) 2024-03-28T13:12:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.991+0000 7f0849600640 1 -- 172.21.15.19:0/2649917785 learned_addr learned my addr 172.21.15.19:0/2649917785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.992+0000 7f0848c00640 1 --2- 172.21.15.19:0/2649917785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08440d04d0 0x7f0844155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.992+0000 7f084a000640 1 --2- 172.21.15.19:0/2649917785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0844154f20 0x7f0844135af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.992+0000 7f0849600640 1 -- 172.21.15.19:0/2649917785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08440d04d0 msgr2=0x7f0844155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.992+0000 7f0849600640 1 --2- 172.21.15.19:0/2649917785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08440d04d0 0x7f0844155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.992+0000 7f0849600640 1 -- 172.21.15.19:0/2649917785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0844154f20 msgr2=0x7f0844135af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.992+0000 7f0849600640 1 --2- 172.21.15.19:0/2649917785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0844154f20 0x7f0844135af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.992+0000 7f0849600640 1 -- 172.21.15.19:0/2649917785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084413a160 con 0x7f08440cc0b0 2024-03-28T13:12:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.992+0000 7f0848c00640 1 --2- 172.21.15.19:0/2649917785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08440d04d0 0x7f0844155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:29.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.992+0000 7f0849600640 1 --2- 172.21.15.19:0/2649917785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08440cc0b0 0x7f0844139b80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0838000c00 tx=0x7f08380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.993+0000 7f0841600640 1 -- 172.21.15.19:0/2649917785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838035070 con 0x7f08440cc0b0 2024-03-28T13:12:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.993+0000 7f0841600640 1 -- 172.21.15.19:0/2649917785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0838004230 con 0x7f08440cc0b0 2024-03-28T13:12:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.993+0000 7f0841600640 1 -- 172.21.15.19:0/2649917785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083803a6f0 con 0x7f08440cc0b0 2024-03-28T13:12:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.993+0000 7f084aa00640 1 -- 172.21.15.19:0/2649917785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084413a440 con 0x7f08440cc0b0 2024-03-28T13:12:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.993+0000 7f084aa00640 1 -- 172.21.15.19:0/2649917785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f084413aa00 con 0x7f08440cc0b0 2024-03-28T13:12:29.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.994+0000 7f084aa00640 1 -- 172.21.15.19:0/2649917785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0844075940 con 0x7f08440cc0b0 2024-03-28T13:12:29.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.995+0000 7f0841600640 1 -- 172.21.15.19:0/2649917785 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0838038020 con 0x7f08440cc0b0 2024-03-28T13:12:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.996+0000 7f0841600640 1 --2- 172.21.15.19:0/2649917785 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f082406c580 0x7f082406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.996+0000 7f0841600640 1 -- 172.21.15.19:0/2649917785 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7f08380ad410 con 0x7f08440cc0b0 2024-03-28T13:12:29.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.997+0000 7f0848c00640 1 --2- 172.21.15.19:0/2649917785 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f082406c580 0x7f082406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:29.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:29.998+0000 7f0848c00640 1 --2- 172.21.15.19:0/2649917785 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f082406c580 0x7f082406ea40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f084413ee20 tx=0x7f0830009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:30.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.001+0000 7f0841600640 1 -- 172.21.15.19:0/2649917785 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0838040070 con 0x7f08440cc0b0 2024-03-28T13:12:30.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.215+0000 7f084aa00640 1 -- 172.21.15.19:0/2649917785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f084415f870 con 0x7f08440cc0b0 2024-03-28T13:12:30.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.219+0000 7f0841600640 1 -- 172.21.15.19:0/2649917785 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16900 (secure 0 0 0) 0x7f083807ba10 con 0x7f08440cc0b0 2024-03-28T13:12:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:30.219 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":51,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:12:24.186519+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:11:44.316258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-28T13:11:45.360154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.0","mappings":[{"from":5,"to":0}]},{"pgid":"7.6","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.39","mappings":[{"from":5,"to":7}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:12:30.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.221+0000 7f084aa00640 1 -- 172.21.15.19:0/2649917785 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f082406c580 msgr2=0x7f082406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:30.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.221+0000 7f084aa00640 1 --2- 172.21.15.19:0/2649917785 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f082406c580 0x7f082406ea40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f084413ee20 tx=0x7f0830009040 comp rx=0 tx=0).stop 2024-03-28T13:12:30.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.221+0000 7f084aa00640 1 -- 172.21.15.19:0/2649917785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08440cc0b0 msgr2=0x7f0844139b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:30.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.221+0000 7f084aa00640 1 --2- 172.21.15.19:0/2649917785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08440cc0b0 0x7f0844139b80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0838000c00 tx=0x7f08380046c0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.222+0000 7f084aa00640 1 -- 172.21.15.19:0/2649917785 shutdown_connections 2024-03-28T13:12:30.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.222+0000 7f084aa00640 1 --2- 172.21.15.19:0/2649917785 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f082406c580 0x7f082406ea40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.222+0000 7f084aa00640 1 --2- 172.21.15.19:0/2649917785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0844154f20 0x7f0844135af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.222+0000 7f084aa00640 1 --2- 172.21.15.19:0/2649917785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08440d04d0 0x7f0844155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.222+0000 7f084aa00640 1 --2- 172.21.15.19:0/2649917785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08440cc0b0 0x7f0844139b80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.222+0000 7f084aa00640 1 -- 172.21.15.19:0/2649917785 >> 172.21.15.19:0/2649917785 conn(0x7f08440ccad0 msgr2=0x7f08440b59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.222+0000 7f084aa00640 1 -- 172.21.15.19:0/2649917785 shutdown_connections 2024-03-28T13:12:30.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.222+0000 7f084aa00640 1 -- 172.21.15.19:0/2649917785 wait complete. 2024-03-28T13:12:30.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:11:44.316258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:11:45.360154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:12:30.241 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:12:30.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.453+0000 7f7e49e00640 1 Processor -- start 2024-03-28T13:12:30.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.453+0000 7f7e49e00640 1 -- start start 2024-03-28T13:12:30.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.454+0000 7f7e49e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e440d3e40 0x7f7e440b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:30.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.454+0000 7f7e49e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7e440cef20 0x7f7e440cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:30.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.454+0000 7f7e49e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e440d2da0 con 0x7f7e4415e460 2024-03-28T13:12:30.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.454+0000 7f7e49e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7e440b8320 con 0x7f7e440cef20 2024-03-28T13:12:30.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.454+0000 7f7e49e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7e440d3080 con 0x7f7e440d3e40 2024-03-28T13:12:30.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.455+0000 7f7e43e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e440d3e40 0x7f7e440b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:30.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.455+0000 7f7e48a00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f7e4415e460 0x7f7e440d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:49164/0 (socket says 172.21.15.19:49164) 2024-03-28T13:12:30.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.455+0000 7f7e48a00640 1 -- 172.21.15.19:0/2400519513 learned_addr learned my addr 172.21.15.19:0/2400519513 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:30.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.455+0000 7f7e49400640 1 --2- 172.21.15.19:0/2400519513 >> v2:172.21.15.188:3300/0 conn(0x7f7e440cef20 0x7f7e440cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:30.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.456+0000 7f7e49400640 1 -- 172.21.15.19:0/2400519513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e440d3e40 msgr2=0x7f7e440b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:30.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.456+0000 7f7e49400640 1 --2- 172.21.15.19:0/2400519513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e440d3e40 0x7f7e440b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.456+0000 7f7e49400640 1 -- 172.21.15.19:0/2400519513 >> v1:172.21.15.188:6789/0 conn(0x7f7e4415e460 legacy=0x7f7e440d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:30.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.456+0000 7f7e49400640 1 -- 172.21.15.19:0/2400519513 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e440d3360 con 0x7f7e440cef20 2024-03-28T13:12:30.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.457+0000 7f7e49400640 1 --2- 172.21.15.19:0/2400519513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e440cef20 0x7f7e440cf3b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009d90 tx=0x7f7e3c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=719030e29cb3c5b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:30.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.458+0000 7f7e43400640 1 -- 172.21.15.19:0/2400519513 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c039070 con 0x7f7e440cef20 2024-03-28T13:12:30.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.458+0000 7f7e43400640 1 -- 172.21.15.19:0/2400519513 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e3c02cc50 con 0x7f7e440cef20 2024-03-28T13:12:30.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.458+0000 7f7e43400640 1 -- 172.21.15.19:0/2400519513 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c034440 con 0x7f7e440cef20 2024-03-28T13:12:30.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.458+0000 7f7e49e00640 1 -- 172.21.15.19:0/2400519513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e440cef20 msgr2=0x7f7e440cf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:30.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.458+0000 7f7e49e00640 1 --2- 172.21.15.19:0/2400519513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e440cef20 0x7f7e440cf3b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009d90 tx=0x7f7e3c02bea0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.458+0000 7f7e49e00640 1 -- 172.21.15.19:0/2400519513 shutdown_connections 2024-03-28T13:12:30.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.458+0000 7f7e49e00640 1 --2- 172.21.15.19:0/2400519513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e440cef20 0x7f7e440cf3b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.458+0000 7f7e49e00640 1 --2- 172.21.15.19:0/2400519513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e440d3e40 0x7f7e440b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.458+0000 7f7e49e00640 1 -- 172.21.15.19:0/2400519513 >> 172.21.15.19:0/2400519513 conn(0x7f7e440678b0 msgr2=0x7f7e440d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:30.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.459+0000 7f7e49e00640 1 -- 172.21.15.19:0/2400519513 shutdown_connections 2024-03-28T13:12:30.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.459+0000 7f7e49e00640 1 -- 172.21.15.19:0/2400519513 wait complete. 2024-03-28T13:12:30.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.460+0000 7f7e49e00640 1 Processor -- start 2024-03-28T13:12:30.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.461+0000 7f7e49e00640 1 -- start start 2024-03-28T13:12:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.461+0000 7f7e49e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e440cef20 0x7f7e440ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.462+0000 7f7e49e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e440d3e40 0x7f7e4415b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.462+0000 7f7e49e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e4415e460 0x7f7e440d4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.462+0000 7f7e49e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e440b7310 con 0x7f7e440cef20 2024-03-28T13:12:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.462+0000 7f7e49e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7e440d3920 con 0x7f7e4415e460 2024-03-28T13:12:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.462+0000 7f7e49e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7e440b7490 con 0x7f7e440d3e40 2024-03-28T13:12:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.462+0000 7f7e48a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e440cef20 0x7f7e440ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:30.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.462+0000 7f7e48a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e440cef20 0x7f7e440ac9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46076/0 (socket says 172.21.15.19:46076) 2024-03-28T13:12:30.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.462+0000 7f7e48a00640 1 -- 172.21.15.19:0/1169559560 learned_addr learned my addr 172.21.15.19:0/1169559560 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:30.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.462+0000 7f7e49400640 1 --2- 172.21.15.19:0/1169559560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e4415e460 0x7f7e440d4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:30.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.462+0000 7f7e43e00640 1 --2- 172.21.15.19:0/1169559560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e440d3e40 0x7f7e4415b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:30.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.463+0000 7f7e48a00640 1 -- 172.21.15.19:0/1169559560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e440d3e40 msgr2=0x7f7e4415b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:30.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.463+0000 7f7e48a00640 1 --2- 172.21.15.19:0/1169559560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e440d3e40 0x7f7e4415b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.463+0000 7f7e48a00640 1 -- 172.21.15.19:0/1169559560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e4415e460 msgr2=0x7f7e440d4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:30.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.463+0000 7f7e48a00640 1 --2- 172.21.15.19:0/1169559560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e4415e460 0x7f7e440d4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.463+0000 7f7e48a00640 1 -- 172.21.15.19:0/1169559560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e440cd280 con 0x7f7e440cef20 2024-03-28T13:12:30.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.463+0000 7f7e43e00640 1 --2- 172.21.15.19:0/1169559560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e440d3e40 0x7f7e4415b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:12:30.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.463+0000 7f7e48a00640 1 --2- 172.21.15.19:0/1169559560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e440cef20 0x7f7e440ac9d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800c3c0 tx=0x7f7e3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:30.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.464+0000 7f7e40c00640 1 -- 172.21.15.19:0/1169559560 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e38017070 con 0x7f7e440cef20 2024-03-28T13:12:30.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.464+0000 7f7e40c00640 1 -- 172.21.15.19:0/1169559560 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e38005ce0 con 0x7f7e440cef20 2024-03-28T13:12:30.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.464+0000 7f7e49e00640 1 -- 172.21.15.19:0/1169559560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e440cd560 con 0x7f7e440cef20 2024-03-28T13:12:30.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.464+0000 7f7e40c00640 1 -- 172.21.15.19:0/1169559560 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e380072f0 con 0x7f7e440cef20 2024-03-28T13:12:30.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.464+0000 7f7e49e00640 1 -- 172.21.15.19:0/1169559560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e440acf10 con 0x7f7e440cef20 2024-03-28T13:12:30.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.465+0000 7f7e49e00640 1 -- 172.21.15.19:0/1169559560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e44075940 con 0x7f7e440cef20 2024-03-28T13:12:30.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.466+0000 7f7e40c00640 1 -- 172.21.15.19:0/1169559560 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7e3801a070 con 0x7f7e440cef20 2024-03-28T13:12:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.467+0000 7f7e40c00640 1 --2- 172.21.15.19:0/1169559560 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e2406c530 0x7f7e2406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.467+0000 7f7e40c00640 1 -- 172.21.15.19:0/1169559560 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7f7e3808aaf0 con 0x7f7e440cef20 2024-03-28T13:12:30.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.468+0000 7f7e43e00640 1 --2- 172.21.15.19:0/1169559560 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e2406c530 0x7f7e2406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:30.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.469+0000 7f7e43e00640 1 --2- 172.21.15.19:0/1169559560 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e2406c530 0x7f7e2406e9f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7e440d4e40 tx=0x7f7e30008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:30.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.476+0000 7f7e40c00640 1 -- 172.21.15.19:0/1169559560 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7e38058990 con 0x7f7e440cef20 2024-03-28T13:12:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.699+0000 7f7e49e00640 1 -- 172.21.15.19:0/1169559560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7e440cd8f0 con 0x7f7e440cef20 2024-03-28T13:12:30.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.699+0000 7f7e40c00640 1 -- 172.21.15.19:0/1169559560 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) ==== 93+0+10 (secure 0 0 0) 0x7f7e3805de20 con 0x7f7e440cef20 2024-03-28T13:12:30.699 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:12:30.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.702+0000 7f7e49e00640 1 -- 172.21.15.19:0/1169559560 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e2406c530 msgr2=0x7f7e2406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:30.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.702+0000 7f7e49e00640 1 --2- 172.21.15.19:0/1169559560 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e2406c530 0x7f7e2406e9f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7e440d4e40 tx=0x7f7e30008040 comp rx=0 tx=0).stop 2024-03-28T13:12:30.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.702+0000 7f7e49e00640 1 -- 172.21.15.19:0/1169559560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e440cef20 msgr2=0x7f7e440ac9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:30.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.702+0000 7f7e49e00640 1 --2- 172.21.15.19:0/1169559560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e440cef20 0x7f7e440ac9d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7e3800c3c0 tx=0x7f7e3800c880 comp rx=0 tx=0).stop 2024-03-28T13:12:30.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.702+0000 7f7e49e00640 1 -- 172.21.15.19:0/1169559560 shutdown_connections 2024-03-28T13:12:30.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.702+0000 7f7e49e00640 1 --2- 172.21.15.19:0/1169559560 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e2406c530 0x7f7e2406e9f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.702+0000 7f7e49e00640 1 --2- 172.21.15.19:0/1169559560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e4415e460 0x7f7e440d4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.702+0000 7f7e49e00640 1 --2- 172.21.15.19:0/1169559560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e440d3e40 0x7f7e4415b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.702+0000 7f7e49e00640 1 --2- 172.21.15.19:0/1169559560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e440cef20 0x7f7e440ac9d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.702+0000 7f7e49e00640 1 -- 172.21.15.19:0/1169559560 >> 172.21.15.19:0/1169559560 conn(0x7f7e440678b0 msgr2=0x7f7e440d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.702+0000 7f7e49e00640 1 -- 172.21.15.19:0/1169559560 shutdown_connections 2024-03-28T13:12:30.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.702+0000 7f7e49e00640 1 -- 172.21.15.19:0/1169559560 wait complete. 2024-03-28T13:12:30.722 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:12:30.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.942+0000 7fc1f2c00640 1 Processor -- start 2024-03-28T13:12:30.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.943+0000 7fc1f2c00640 1 -- start start 2024-03-28T13:12:30.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.944+0000 7fc1f2c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec15e4c0 0x7fc1ec0b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:30.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.944+0000 7fc1f2c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc1ec0b69c0 0x7fc1ec0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:30.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.944+0000 7fc1f2c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1ec0d2710 con 0x7fc1ec0d37b0 2024-03-28T13:12:30.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.944+0000 7fc1f2c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc1ec15e8a0 con 0x7fc1ec0b69c0 2024-03-28T13:12:30.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.944+0000 7fc1f2c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ec0d4040 con 0x7fc1ec15e4c0 2024-03-28T13:12:30.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.944+0000 7fc1f0e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc1ec0d37b0 0x7fc1ec0b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45020/0 (socket says 172.21.15.19:45020) 2024-03-28T13:12:30.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.944+0000 7fc1f0e00640 1 -- 172.21.15.19:0/1247294798 learned_addr learned my addr 172.21.15.19:0/1247294798 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.945+0000 7fc1f1800640 1 --2- 172.21.15.19:0/1247294798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec15e4c0 0x7fc1ec0b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.945+0000 7fc1f2200640 1 --2- 172.21.15.19:0/1247294798 >> v2:172.21.15.188:3300/0 conn(0x7fc1ec0b69c0 0x7fc1ec0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.945+0000 7fc1ebe00640 1 -- 172.21.15.19:0/1247294798 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1683228913 0 0) 0x7fc1ec0d2710 con 0x7fc1ec0d37b0 2024-03-28T13:12:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.945+0000 7fc1ebe00640 1 -- 172.21.15.19:0/1247294798 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1d4003520 con 0x7fc1ec0d37b0 2024-03-28T13:12:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.945+0000 7fc1ebe00640 1 -- 172.21.15.19:0/1247294798 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3580987687 0 0) 0x7fc1d4003520 con 0x7fc1ec0d37b0 2024-03-28T13:12:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.945+0000 7fc1ebe00640 1 -- 172.21.15.19:0/1247294798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec15e4c0 msgr2=0x7fc1ec0b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.945+0000 7fc1ebe00640 1 --2- 172.21.15.19:0/1247294798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec15e4c0 0x7fc1ec0b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.945+0000 7fc1ebe00640 1 -- 172.21.15.19:0/1247294798 >> v2:172.21.15.188:3300/0 conn(0x7fc1ec0b69c0 msgr2=0x7fc1ec0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.945+0000 7fc1ebe00640 1 --2- 172.21.15.19:0/1247294798 >> v2:172.21.15.188:3300/0 conn(0x7fc1ec0b69c0 0x7fc1ec0b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.945+0000 7fc1ebe00640 1 -- 172.21.15.19:0/1247294798 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ec0d29f0 con 0x7fc1ec0d37b0 2024-03-28T13:12:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.946+0000 7fc1ebe00640 1 -- 172.21.15.19:0/1247294798 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fc1d8001670 con 0x7fc1ec0d37b0 2024-03-28T13:12:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.946+0000 7fc1ebe00640 1 -- 172.21.15.19:0/1247294798 >> v1:172.21.15.19:6789/0 conn(0x7fc1ec0d37b0 legacy=0x7fc1ec0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.946+0000 7fc1ebe00640 1 --2- 172.21.15.19:0/1247294798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1d40039b0 0x7fc1d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:30.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.946+0000 7fc1ebe00640 1 --2- 172.21.15.19:0/1247294798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec15e4c0 0x7fc1d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.946+0000 7fc1ebe00640 1 --2- 172.21.15.19:0/1247294798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1d4004d40 0x7fc1d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.946+0000 7fc1ebe00640 1 -- 172.21.15.19:0/1247294798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec0d4040 con 0x7fc1d4004d40 2024-03-28T13:12:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.946+0000 7fc1ebe00640 1 -- 172.21.15.19:0/1247294798 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d400b9c0 con 0x7fc1d40039b0 2024-03-28T13:12:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.946+0000 7fc1ebe00640 1 -- 172.21.15.19:0/1247294798 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d400bb40 con 0x7fc1ec15e4c0 2024-03-28T13:12:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.946+0000 7fc1f2200640 1 --2- 172.21.15.19:0/1247294798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1d4004d40 0x7fc1d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.946+0000 7fc1f1800640 1 --2- 172.21.15.19:0/1247294798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1d40039b0 0x7fc1d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.946+0000 7fc1f0e00640 1 --2- 172.21.15.19:0/1247294798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec15e4c0 0x7fc1d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.946+0000 7fc1f2200640 1 -- 172.21.15.19:0/1247294798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec15e4c0 msgr2=0x7fc1d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.947+0000 7fc1f2200640 1 --2- 172.21.15.19:0/1247294798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec15e4c0 0x7fc1d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.947+0000 7fc1f2200640 1 -- 172.21.15.19:0/1247294798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1d40039b0 msgr2=0x7fc1d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.947+0000 7fc1f2200640 1 --2- 172.21.15.19:0/1247294798 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1d40039b0 0x7fc1d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.947+0000 7fc1f2200640 1 -- 172.21.15.19:0/1247294798 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d4003520 con 0x7fc1d4004d40 2024-03-28T13:12:30.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.948+0000 7fc1f2200640 1 --2- 172.21.15.19:0/1247294798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1d4004d40 0x7fc1d40051c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec08d5d0 tx=0x7fc1d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3e2eff6806b0ed server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:30.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.948+0000 7fc1ebe00640 1 -- 172.21.15.19:0/1247294798 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4039070 con 0x7fc1d4004d40 2024-03-28T13:12:30.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.948+0000 7fc1ebe00640 1 -- 172.21.15.19:0/1247294798 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1e402cbd0 con 0x7fc1d4004d40 2024-03-28T13:12:30.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.948+0000 7fc1ebe00640 1 -- 172.21.15.19:0/1247294798 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4009c10 con 0x7fc1d4004d40 2024-03-28T13:12:30.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.949+0000 7fc1f2c00640 1 -- 172.21.15.19:0/1247294798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1d4004d40 msgr2=0x7fc1d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:30.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.949+0000 7fc1f2c00640 1 --2- 172.21.15.19:0/1247294798 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1d4004d40 0x7fc1d40051c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec08d5d0 tx=0x7fc1d4005830 comp rx=0 tx=0).stop 2024-03-28T13:12:30.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.949+0000 7fc1f2200640 1 -- 172.21.15.19:0/1247294798 reap_dead start 2024-03-28T13:12:30.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.949+0000 7fc1f2c00640 1 -- 172.21.15.19:0/1247294798 shutdown_connections 2024-03-28T13:12:30.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.949+0000 7fc1f2c00640 1 -- 172.21.15.19:0/1247294798 >> 172.21.15.19:0/1247294798 conn(0x7fc1ec0676a0 msgr2=0x7fc1ec067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:30.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.949+0000 7fc1f2c00640 1 -- 172.21.15.19:0/1247294798 shutdown_connections 2024-03-28T13:12:30.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.949+0000 7fc1f2c00640 1 -- 172.21.15.19:0/1247294798 wait complete. 2024-03-28T13:12:30.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.951+0000 7fc1f2c00640 1 Processor -- start 2024-03-28T13:12:30.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.951+0000 7fc1f2c00640 1 -- start start 2024-03-28T13:12:30.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.952+0000 7fc1f2c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec146340 0x7fc1ec0cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:30.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.952+0000 7fc1f2c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1ec0cbc10 0x7fc1ec160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:30.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.952+0000 7fc1f2c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ec161090 0x7fc1ec1f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:30.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.952+0000 7fc1f2c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec0d2fb0 con 0x7fc1ec0cbc10 2024-03-28T13:12:30.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.952+0000 7fc1f2c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ec0d3320 con 0x7fc1ec161090 2024-03-28T13:12:30.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.952+0000 7fc1f2c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc1ec0d2350 con 0x7fc1ec146340 2024-03-28T13:12:30.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.952+0000 7fc1f0e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1ec0cbc10 0x7fc1ec160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:30.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.952+0000 7fc1f0e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1ec0cbc10 0x7fc1ec160b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46080/0 (socket says 172.21.15.19:46080) 2024-03-28T13:12:30.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.952+0000 7fc1f0e00640 1 -- 172.21.15.19:0/2060239042 learned_addr learned my addr 172.21.15.19:0/2060239042 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.952+0000 7fc1f0e00640 1 -- 172.21.15.19:0/2060239042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec146340 msgr2=0x7fc1ec0cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.952+0000 7fc1f0e00640 1 --2- 172.21.15.19:0/2060239042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec146340 0x7fc1ec0cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.952+0000 7fc1f0e00640 1 -- 172.21.15.19:0/2060239042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ec161090 msgr2=0x7fc1ec1f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.952+0000 7fc1f0e00640 1 --2- 172.21.15.19:0/2060239042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ec161090 0x7fc1ec1f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.952+0000 7fc1f0e00640 1 -- 172.21.15.19:0/2060239042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ec0cccf0 con 0x7fc1ec0cbc10 2024-03-28T13:12:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.953+0000 7fc1f0e00640 1 --2- 172.21.15.19:0/2060239042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1ec0cbc10 0x7fc1ec160b50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800eb30 tx=0x7fc1d8005e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.953+0000 7fc1e9600640 1 -- 172.21.15.19:0/2060239042 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8019070 con 0x7fc1ec0cbc10 2024-03-28T13:12:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.953+0000 7fc1e9600640 1 -- 172.21.15.19:0/2060239042 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d800c910 con 0x7fc1ec0cbc10 2024-03-28T13:12:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.953+0000 7fc1f2c00640 1 -- 172.21.15.19:0/2060239042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ec0cc170 con 0x7fc1ec0cbc10 2024-03-28T13:12:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.953+0000 7fc1e9600640 1 -- 172.21.15.19:0/2060239042 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d800cce0 con 0x7fc1ec0cbc10 2024-03-28T13:12:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.953+0000 7fc1f2c00640 1 -- 172.21.15.19:0/2060239042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ec0cc730 con 0x7fc1ec0cbc10 2024-03-28T13:12:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.955+0000 7fc1f2c00640 1 -- 172.21.15.19:0/2060239042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ec1614b0 con 0x7fc1ec0cbc10 2024-03-28T13:12:30.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.956+0000 7fc1e9600640 1 -- 172.21.15.19:0/2060239042 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc1d801c070 con 0x7fc1ec0cbc10 2024-03-28T13:12:30.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.957+0000 7fc1e9600640 1 --2- 172.21.15.19:0/2060239042 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc1cc06c530 0x7fc1cc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:30.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.957+0000 7fc1e9600640 1 -- 172.21.15.19:0/2060239042 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7fc1d808b120 con 0x7fc1ec0cbc10 2024-03-28T13:12:30.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.957+0000 7fc1f1800640 1 --2- 172.21.15.19:0/2060239042 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc1cc06c530 0x7fc1cc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:30.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.958+0000 7fc1f1800640 1 --2- 172.21.15.19:0/2060239042 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc1cc06c530 0x7fc1cc06e9f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009890 tx=0x7fc1e0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:30.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:30.961+0000 7fc1e9600640 1 -- 172.21.15.19:0/2060239042 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc1d80596a0 con 0x7fc1ec0cbc10 2024-03-28T13:12:31.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.169+0000 7fc1f2c00640 1 -- 172.21.15.19:0/2060239042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc1ec075940 con 0x7fc1ec0cbc10 2024-03-28T13:12:31.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.170+0000 7fc1e9600640 1 -- 172.21.15.19:0/2060239042 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) ==== 104+0+11 (secure 0 0 0) 0x7fc1d8058fa0 con 0x7fc1ec0cbc10 2024-03-28T13:12:31.170 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:12:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.173+0000 7fc1f2c00640 1 -- 172.21.15.19:0/2060239042 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc1cc06c530 msgr2=0x7fc1cc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.173+0000 7fc1f2c00640 1 --2- 172.21.15.19:0/2060239042 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc1cc06c530 0x7fc1cc06e9f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0009890 tx=0x7fc1e0008040 comp rx=0 tx=0).stop 2024-03-28T13:12:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.173+0000 7fc1f2c00640 1 -- 172.21.15.19:0/2060239042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1ec0cbc10 msgr2=0x7fc1ec160b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.173+0000 7fc1f2c00640 1 --2- 172.21.15.19:0/2060239042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1ec0cbc10 0x7fc1ec160b50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800eb30 tx=0x7fc1d8005e10 comp rx=0 tx=0).stop 2024-03-28T13:12:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.173+0000 7fc1f2c00640 1 -- 172.21.15.19:0/2060239042 shutdown_connections 2024-03-28T13:12:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.173+0000 7fc1f2c00640 1 --2- 172.21.15.19:0/2060239042 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc1cc06c530 0x7fc1cc06e9f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.173+0000 7fc1f2c00640 1 --2- 172.21.15.19:0/2060239042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1ec161090 0x7fc1ec1f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.173+0000 7fc1f2c00640 1 --2- 172.21.15.19:0/2060239042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc1ec0cbc10 0x7fc1ec160b50 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.173+0000 7fc1f2c00640 1 --2- 172.21.15.19:0/2060239042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1ec146340 0x7fc1ec0cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.173+0000 7fc1f2c00640 1 -- 172.21.15.19:0/2060239042 >> 172.21.15.19:0/2060239042 conn(0x7fc1ec0676a0 msgr2=0x7fc1ec0cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.173+0000 7fc1f2c00640 1 -- 172.21.15.19:0/2060239042 shutdown_connections 2024-03-28T13:12:31.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.173+0000 7fc1f2c00640 1 -- 172.21.15.19:0/2060239042 wait complete. 2024-03-28T13:12:31.191 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:12:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.402+0000 7fa840600640 1 Processor -- start 2024-03-28T13:12:31.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.402+0000 7fa840600640 1 -- start start 2024-03-28T13:12:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.403+0000 7fa840600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa838153580 0x7fa8380d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.403+0000 7fa840600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8380d07c0 0x7fa8380aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.404+0000 7fa840600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8380afab0 con 0x7fa8380af2e0 2024-03-28T13:12:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.404+0000 7fa840600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa8380d0c00 con 0x7fa838153580 2024-03-28T13:12:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.404+0000 7fa840600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa83815cc80 con 0x7fa8380d07c0 2024-03-28T13:12:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.404+0000 7fa83dc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa838153580 0x7fa8380d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.404+0000 7fa83dc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa838153580 0x7fa8380d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46092/0 (socket says 172.21.15.19:46092) 2024-03-28T13:12:31.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.404+0000 7fa83dc00640 1 -- 172.21.15.19:0/3218254214 learned_addr learned my addr 172.21.15.19:0/3218254214 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.404+0000 7fa83d200640 1 --2- 172.21.15.19:0/3218254214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8380d07c0 0x7fa8380aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.404+0000 7fa83dc00640 1 -- 172.21.15.19:0/3218254214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8380d07c0 msgr2=0x7fa8380aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.404+0000 7fa83dc00640 1 --2- 172.21.15.19:0/3218254214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8380d07c0 0x7fa8380aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.404+0000 7fa83dc00640 1 -- 172.21.15.19:0/3218254214 >> v1:172.21.15.188:6789/0 conn(0x7fa8380af2e0 legacy=0x7fa83806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:31.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.405+0000 7fa83dc00640 1 -- 172.21.15.19:0/3218254214 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83815da00 con 0x7fa838153580 2024-03-28T13:12:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.406+0000 7fa83dc00640 1 --2- 172.21.15.19:0/3218254214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa838153580 0x7fa8380d0280 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009870 tx=0x7fa82c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b810a10eb73c0555 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.406+0000 7fa837e00640 1 -- 172.21.15.19:0/3218254214 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c035020 con 0x7fa838153580 2024-03-28T13:12:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.406+0000 7fa837e00640 1 -- 172.21.15.19:0/3218254214 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa82c031640 con 0x7fa838153580 2024-03-28T13:12:31.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.406+0000 7fa837e00640 1 -- 172.21.15.19:0/3218254214 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c02d040 con 0x7fa838153580 2024-03-28T13:12:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.407+0000 7fa840600640 1 -- 172.21.15.19:0/3218254214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa838153580 msgr2=0x7fa8380d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.407+0000 7fa840600640 1 --2- 172.21.15.19:0/3218254214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa838153580 0x7fa8380d0280 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009870 tx=0x7fa82c02b8f0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.407+0000 7fa840600640 1 -- 172.21.15.19:0/3218254214 shutdown_connections 2024-03-28T13:12:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.407+0000 7fa840600640 1 --2- 172.21.15.19:0/3218254214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8380d07c0 0x7fa8380aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.407+0000 7fa840600640 1 --2- 172.21.15.19:0/3218254214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa838153580 0x7fa8380d0280 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.407+0000 7fa840600640 1 -- 172.21.15.19:0/3218254214 >> 172.21.15.19:0/3218254214 conn(0x7fa83806b750 msgr2=0x7fa83806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.407+0000 7fa840600640 1 -- 172.21.15.19:0/3218254214 shutdown_connections 2024-03-28T13:12:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.407+0000 7fa840600640 1 -- 172.21.15.19:0/3218254214 wait complete. 2024-03-28T13:12:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.409+0000 7fa840600640 1 Processor -- start 2024-03-28T13:12:31.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.409+0000 7fa840600640 1 -- start start 2024-03-28T13:12:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.410+0000 7fa840600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8380af2e0 0x7fa83815a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.410+0000 7fa840600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8380d07c0 0x7fa8380731a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.410+0000 7fa840600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa838153580 0x7fa83814b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.410+0000 7fa840600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa83815d4e0 con 0x7fa8380af2e0 2024-03-28T13:12:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.410+0000 7fa840600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa83815d200 con 0x7fa838153580 2024-03-28T13:12:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.410+0000 7fa840600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa838153960 con 0x7fa8380d07c0 2024-03-28T13:12:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.410+0000 7fa83dc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8380af2e0 0x7fa83815a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.410+0000 7fa83dc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8380af2e0 0x7fa83815a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46104/0 (socket says 172.21.15.19:46104) 2024-03-28T13:12:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.410+0000 7fa83dc00640 1 -- 172.21.15.19:0/2689878165 learned_addr learned my addr 172.21.15.19:0/2689878165 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.410+0000 7fa83d200640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8380d07c0 0x7fa8380731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.410+0000 7fa83dc00640 1 -- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8380d07c0 msgr2=0x7fa8380731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.410+0000 7fa83e600640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa838153580 0x7fa83814b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.410+0000 7fa83dc00640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8380d07c0 0x7fa8380731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.410+0000 7fa83dc00640 1 -- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa838153580 msgr2=0x7fa83814b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.411+0000 7fa83dc00640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa838153580 0x7fa83814b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.411+0000 7fa83dc00640 1 -- 172.21.15.19:0/2689878165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83815f640 con 0x7fa8380af2e0 2024-03-28T13:12:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.411+0000 7fa83d200640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8380d07c0 0x7fa8380731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.411+0000 7fa83e600640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa838153580 0x7fa83814b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.411+0000 7fa83dc00640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8380af2e0 0x7fa83815a180 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa82c000c00 tx=0x7fa82c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.411+0000 7fa835600640 1 -- 172.21.15.19:0/2689878165 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c040070 con 0x7fa8380af2e0 2024-03-28T13:12:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.411+0000 7fa835600640 1 -- 172.21.15.19:0/2689878165 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa82c0399f0 con 0x7fa8380af2e0 2024-03-28T13:12:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.411+0000 7fa835600640 1 -- 172.21.15.19:0/2689878165 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c035040 con 0x7fa8380af2e0 2024-03-28T13:12:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.411+0000 7fa840600640 1 -- 172.21.15.19:0/2689878165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83815f8e0 con 0x7fa8380af2e0 2024-03-28T13:12:31.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.411+0000 7fa840600640 1 -- 172.21.15.19:0/2689878165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8380729a0 con 0x7fa8380af2e0 2024-03-28T13:12:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.413+0000 7fa840600640 1 -- 172.21.15.19:0/2689878165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8380685c0 con 0x7fa8380af2e0 2024-03-28T13:12:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.414+0000 7fa835600640 1 -- 172.21.15.19:0/2689878165 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa82c04b060 con 0x7fa8380af2e0 2024-03-28T13:12:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.414+0000 7fa835600640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa81806c530 0x7fa81806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:31.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.415+0000 7fa835600640 1 -- 172.21.15.19:0/2689878165 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7fa82c0ad090 con 0x7fa8380af2e0 2024-03-28T13:12:31.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.415+0000 7fa83d200640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa81806c530 0x7fa81806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:31.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.417+0000 7fa83d200640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa81806c530 0x7fa81806e9f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa824006fd0 tx=0x7fa824008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:31.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.419+0000 7fa835600640 1 -- 172.21.15.19:0/2689878165 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa82c07b6b0 con 0x7fa8380af2e0 2024-03-28T13:12:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.623+0000 7fa840600640 1 -- 172.21.15.19:0/2689878165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa83815acf0 con 0x7fa8380af2e0 2024-03-28T13:12:31.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.623+0000 7fa835600640 1 -- 172.21.15.19:0/2689878165 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) ==== 100+0+11 (secure 0 0 0) 0x7fa82c07afb0 con 0x7fa8380af2e0 2024-03-28T13:12:31.627 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:12:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.626+0000 7fa840600640 1 -- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa81806c530 msgr2=0x7fa81806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.626+0000 7fa840600640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa81806c530 0x7fa81806e9f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa824006fd0 tx=0x7fa824008040 comp rx=0 tx=0).stop 2024-03-28T13:12:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.626+0000 7fa840600640 1 -- 172.21.15.19:0/2689878165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8380af2e0 msgr2=0x7fa83815a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.626+0000 7fa840600640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8380af2e0 0x7fa83815a180 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa82c000c00 tx=0x7fa82c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.626+0000 7fa840600640 1 -- 172.21.15.19:0/2689878165 shutdown_connections 2024-03-28T13:12:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.626+0000 7fa840600640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa81806c530 0x7fa81806e9f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.626+0000 7fa840600640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa838153580 0x7fa83814b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.626+0000 7fa840600640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8380d07c0 0x7fa8380731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.626+0000 7fa840600640 1 --2- 172.21.15.19:0/2689878165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8380af2e0 0x7fa83815a180 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.626+0000 7fa840600640 1 -- 172.21.15.19:0/2689878165 >> 172.21.15.19:0/2689878165 conn(0x7fa83806b750 msgr2=0x7fa83815b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.626+0000 7fa840600640 1 -- 172.21.15.19:0/2689878165 shutdown_connections 2024-03-28T13:12:31.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.626+0000 7fa840600640 1 -- 172.21.15.19:0/2689878165 wait complete. 2024-03-28T13:12:31.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:12:31.790 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:12:31.798 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:12:31.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.866+0000 7f379b800640 1 Processor -- start 2024-03-28T13:12:31.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.866+0000 7f379b800640 1 -- start start 2024-03-28T13:12:31.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.868+0000 7f379b800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3794153580 0x7f37940d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:31.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.868+0000 7f379b800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37940afab0 con 0x7f37940af4b0 2024-03-28T13:12:31.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.868+0000 7f379b800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f379415cc80 con 0x7f37940d07c0 2024-03-28T13:12:31.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.868+0000 7f379b800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f37940d0c30 con 0x7f3794153580 2024-03-28T13:12:31.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.868+0000 7f3799800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f37940af4b0 0x7f37941600d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45036/0 (socket says 172.21.15.19:45036) 2024-03-28T13:12:31.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.868+0000 7f3799800640 1 -- 172.21.15.19:0/1070692203 learned_addr learned my addr 172.21.15.19:0/1070692203 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.869+0000 7f3798e00640 1 --2- 172.21.15.19:0/1070692203 >> v2:172.21.15.188:3300/0 conn(0x7f3794153580 0x7f37940d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.869+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1346733996 0 0) 0x7f37940afab0 con 0x7f37940af4b0 2024-03-28T13:12:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.869+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f377c003520 con 0x7f37940af4b0 2024-03-28T13:12:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.869+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3331384372 0 0) 0x7f379415cc80 con 0x7f37940d07c0 2024-03-28T13:12:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.869+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37940afab0 con 0x7f37940d07c0 2024-03-28T13:12:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 300823000 0 0) 0x7f377c003520 con 0x7f37940af4b0 2024-03-28T13:12:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 >> v2:172.21.15.188:3300/0 conn(0x7f3794153580 msgr2=0x7f37940d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3793400640 1 --2- 172.21.15.19:0/1070692203 >> v2:172.21.15.188:3300/0 conn(0x7f3794153580 0x7f37940d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 >> v1:172.21.15.188:6789/0 conn(0x7f37940d07c0 legacy=0x7f37940aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f379415da00 con 0x7f37940af4b0 2024-03-28T13:12:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f378c0019f0 con 0x7f37940af4b0 2024-03-28T13:12:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 >> v1:172.21.15.19:6789/0 conn(0x7f37940af4b0 legacy=0x7f37941600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:31.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3793400640 1 --2- 172.21.15.19:0/1070692203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003b90 0x7f377c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3793400640 1 --2- 172.21.15.19:0/1070692203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c004800 0x7f377c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3793400640 1 --2- 172.21.15.19:0/1070692203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f377c005210 0x7f377c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37940d0c30 con 0x7f377c003b90 2024-03-28T13:12:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f377c00c140 con 0x7f377c005210 2024-03-28T13:12:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f377c00c340 con 0x7f377c004800 2024-03-28T13:12:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3798e00640 1 --2- 172.21.15.19:0/1070692203 >> v2:172.21.15.188:3300/0 conn(0x7f3794153580 0x7f37940d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:12:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3798e00640 1 --2- 172.21.15.19:0/1070692203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003b90 0x7f377c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.870+0000 7f3793e00640 1 --2- 172.21.15.19:0/1070692203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c004800 0x7f377c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.871+0000 7f3799800640 1 --2- 172.21.15.19:0/1070692203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f377c005210 0x7f377c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.871+0000 7f3798e00640 1 -- 172.21.15.19:0/1070692203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c004800 msgr2=0x7f377c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.871+0000 7f3798e00640 1 --2- 172.21.15.19:0/1070692203 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f377c004800 0x7f377c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.871+0000 7f3798e00640 1 -- 172.21.15.19:0/1070692203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f377c005210 msgr2=0x7f377c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.871+0000 7f3798e00640 1 --2- 172.21.15.19:0/1070692203 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f377c005210 0x7f377c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.871+0000 7f3798e00640 1 -- 172.21.15.19:0/1070692203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c003520 con 0x7f377c003b90 2024-03-28T13:12:31.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.871+0000 7f3799800640 1 -- 172.21.15.19:0/1070692203 reap_dead start 2024-03-28T13:12:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.872+0000 7f3798e00640 1 --2- 172.21.15.19:0/1070692203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003b90 0x7f377c003f90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3788009c70 tx=0x7f377c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b851d8e72d7cc8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.873+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788009840 con 0x7f377c003b90 2024-03-28T13:12:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.873+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3788004020 con 0x7f377c003b90 2024-03-28T13:12:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.873+0000 7f3793400640 1 -- 172.21.15.19:0/1070692203 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788004450 con 0x7f377c003b90 2024-03-28T13:12:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.874+0000 7f379b800640 1 -- 172.21.15.19:0/1070692203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003b90 msgr2=0x7f377c003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.874+0000 7f379b800640 1 --2- 172.21.15.19:0/1070692203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003b90 0x7f377c003f90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3788009c70 tx=0x7f377c005830 comp rx=0 tx=0).stop 2024-03-28T13:12:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.874+0000 7f379b800640 1 -- 172.21.15.19:0/1070692203 shutdown_connections 2024-03-28T13:12:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.874+0000 7f379b800640 1 --2- 172.21.15.19:0/1070692203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003b90 0x7f377c003f90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.874+0000 7f379b800640 1 -- 172.21.15.19:0/1070692203 >> 172.21.15.19:0/1070692203 conn(0x7f379406b750 msgr2=0x7f379406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.874+0000 7f379b800640 1 -- 172.21.15.19:0/1070692203 shutdown_connections 2024-03-28T13:12:31.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.874+0000 7f379b800640 1 -- 172.21.15.19:0/1070692203 wait complete. 2024-03-28T13:12:31.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.877+0000 7f379b800640 1 Processor -- start 2024-03-28T13:12:31.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.877+0000 7f379b800640 1 -- start start 2024-03-28T13:12:31.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.879+0000 7f379b800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003b90 0x7f3794159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:31.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.879+0000 7f379b800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3794150d70 0x7f37941505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:31.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.879+0000 7f379b800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37940ce960 0x7f37940cee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:31.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.879+0000 7f379b800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f379415d4e0 con 0x7f377c003b90 2024-03-28T13:12:31.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.879+0000 7f379b800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f379415d200 con 0x7f3794150d70 2024-03-28T13:12:31.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.879+0000 7f379b800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3794153960 con 0x7f37940ce960 2024-03-28T13:12:31.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.879+0000 7f3798e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003b90 0x7f3794159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:31.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.879+0000 7f3798e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003b90 0x7f3794159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46124/0 (socket says 172.21.15.19:46124) 2024-03-28T13:12:31.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.879+0000 7f3798e00640 1 -- 172.21.15.19:0/1055939136 learned_addr learned my addr 172.21.15.19:0/1055939136 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:31.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.879+0000 7f3799800640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37940ce960 0x7f37940cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:31.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.879+0000 7f3793e00640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3794150d70 0x7f37941505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:31.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.880+0000 7f3798e00640 1 -- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37940ce960 msgr2=0x7f37940cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:31.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.880+0000 7f3798e00640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37940ce960 0x7f37940cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.880+0000 7f3798e00640 1 -- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3794150d70 msgr2=0x7f37941505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:31.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.880+0000 7f3798e00640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3794150d70 0x7f37941505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:31.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.880+0000 7f3798e00640 1 -- 172.21.15.19:0/1055939136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3794150b20 con 0x7f377c003b90 2024-03-28T13:12:31.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.880+0000 7f3799800640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37940ce960 0x7f37940cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:12:31.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.880+0000 7f3793e00640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3794150d70 0x7f37941505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:12:31.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.880+0000 7f3798e00640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003b90 0x7f3794159e60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3788036040 tx=0x7f3788004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:31.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.880+0000 7f3790c00640 1 -- 172.21.15.19:0/1055939136 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788043020 con 0x7f377c003b90 2024-03-28T13:12:31.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.881+0000 7f3790c00640 1 -- 172.21.15.19:0/1055939136 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f378802c8f0 con 0x7f377c003b90 2024-03-28T13:12:31.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.881+0000 7f379b800640 1 -- 172.21.15.19:0/1055939136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3794157eb0 con 0x7f377c003b90 2024-03-28T13:12:31.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.881+0000 7f3790c00640 1 -- 172.21.15.19:0/1055939136 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378802cce0 con 0x7f377c003b90 2024-03-28T13:12:31.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.881+0000 7f379b800640 1 -- 172.21.15.19:0/1055939136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3794158470 con 0x7f377c003b90 2024-03-28T13:12:31.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.882+0000 7f379b800640 1 -- 172.21.15.19:0/1055939136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37940685c0 con 0x7f377c003b90 2024-03-28T13:12:31.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.883+0000 7f3790c00640 1 -- 172.21.15.19:0/1055939136 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3788046020 con 0x7f377c003b90 2024-03-28T13:12:31.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.885+0000 7f3790c00640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f377406c530 0x7f377406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:31.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.885+0000 7f3790c00640 1 -- 172.21.15.19:0/1055939136 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7f37880ad8b0 con 0x7f377c003b90 2024-03-28T13:12:31.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.885+0000 7f3793e00640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f377406c530 0x7f377406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:31.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.887+0000 7f3793e00640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f377406c530 0x7f377406e9f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3780007ae0 tx=0x7f3780009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:31.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:31.890+0000 7f3790c00640 1 -- 172.21.15.19:0/1055939136 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f378804d030 con 0x7f377c003b90 2024-03-28T13:12:32.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.109+0000 7f379b800640 1 -- 172.21.15.19:0/1055939136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f379406eca0 con 0x7f377c003b90 2024-03-28T13:12:32.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.109+0000 7f3790c00640 1 -- 172.21.15.19:0/1055939136 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v51) ==== 80+0+0 (secure 0 0 0) 0x7f378807bed0 con 0x7f377c003b90 2024-03-28T13:12:32.109 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-03-28T13:12:32.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.112+0000 7f379b800640 1 -- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f377406c530 msgr2=0x7f377406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:32.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.112+0000 7f379b800640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f377406c530 0x7f377406e9f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3780007ae0 tx=0x7f3780009040 comp rx=0 tx=0).stop 2024-03-28T13:12:32.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.112+0000 7f379b800640 1 -- 172.21.15.19:0/1055939136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003b90 msgr2=0x7f3794159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:32.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.112+0000 7f379b800640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003b90 0x7f3794159e60 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3788036040 tx=0x7f3788004730 comp rx=0 tx=0).stop 2024-03-28T13:12:32.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.112+0000 7f379b800640 1 -- 172.21.15.19:0/1055939136 shutdown_connections 2024-03-28T13:12:32.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.112+0000 7f379b800640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f377406c530 0x7f377406e9f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:32.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.112+0000 7f379b800640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37940ce960 0x7f37940cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:32.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.112+0000 7f379b800640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3794150d70 0x7f37941505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:32.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.112+0000 7f379b800640 1 --2- 172.21.15.19:0/1055939136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f377c003b90 0x7f3794159e60 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:32.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.112+0000 7f379b800640 1 -- 172.21.15.19:0/1055939136 >> 172.21.15.19:0/1055939136 conn(0x7f379406b750 msgr2=0x7f379406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:32.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.112+0000 7f379b800640 1 -- 172.21.15.19:0/1055939136 shutdown_connections 2024-03-28T13:12:32.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.112+0000 7f379b800640 1 -- 172.21.15.19:0/1055939136 wait complete. 2024-03-28T13:12:32.131 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:12:32.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:12:32.131 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T13:12:32.238 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:12:32.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:12:32.239 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:12:32.295 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:12:32.295 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:12:32.295 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:12:32.335 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:12:32.335 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:12:32.335 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:12:32.335 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:12:32.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:12:32.374 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:12:32.374 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:12:32.374 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:12:32.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:12:32.419 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:12:32.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.639+0000 7fbe69c00640 1 Processor -- start 2024-03-28T13:12:32.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.639+0000 7fbe69c00640 1 -- start start 2024-03-28T13:12:32.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.640+0000 7fbe69c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbe6415e3b0 0x7fbe640d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:32.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.640+0000 7fbe69c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe640d2cf0 con 0x7fbe640d3d90 2024-03-28T13:12:32.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.640+0000 7fbe69c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe640d2fd0 con 0x7fbe640b63c0 2024-03-28T13:12:32.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.640+0000 7fbe69c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbe640d32b0 con 0x7fbe6415e3b0 2024-03-28T13:12:32.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.641+0000 7fbe63e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbe6415e3b0 0x7fbe640d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:32.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.641+0000 7fbe63400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbe640d3d90 0x7fbe640b5cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45052/0 (socket says 172.21.15.19:45052) 2024-03-28T13:12:32.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.641+0000 7fbe63400640 1 -- 172.21.15.19:0/40180885 learned_addr learned my addr 172.21.15.19:0/40180885 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:32.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.641+0000 7fbe63e00640 1 -- 172.21.15.19:0/40180885 >> v1:172.21.15.188:6789/0 conn(0x7fbe640b63c0 legacy=0x7fbe640b7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:32.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.641+0000 7fbe63e00640 1 -- 172.21.15.19:0/40180885 >> v1:172.21.15.19:6789/0 conn(0x7fbe640d3d90 legacy=0x7fbe640b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:32.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.641+0000 7fbe62a00640 1 -- 172.21.15.19:0/40180885 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2146003075 0 0) 0x7fbe640d2cf0 con 0x7fbe640d3d90 2024-03-28T13:12:32.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.642+0000 7fbe63e00640 1 -- 172.21.15.19:0/40180885 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe640d3590 con 0x7fbe6415e3b0 2024-03-28T13:12:32.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.643+0000 7fbe63e00640 1 --2- 172.21.15.19:0/40180885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6415e3b0 0x7fbe640d22e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbe540097d0 tx=0x7fbe5402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d8b3264852e3ee server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:32.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.643+0000 7fbe62a00640 1 -- 172.21.15.19:0/40180885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54034070 con 0x7fbe6415e3b0 2024-03-28T13:12:32.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.643+0000 7fbe62a00640 1 -- 172.21.15.19:0/40180885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe54030610 con 0x7fbe6415e3b0 2024-03-28T13:12:32.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.643+0000 7fbe62a00640 1 -- 172.21.15.19:0/40180885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe540309e0 con 0x7fbe6415e3b0 2024-03-28T13:12:32.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.644+0000 7fbe69c00640 1 -- 172.21.15.19:0/40180885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6415e3b0 msgr2=0x7fbe640d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:32.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.644+0000 7fbe69c00640 1 --2- 172.21.15.19:0/40180885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6415e3b0 0x7fbe640d22e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbe540097d0 tx=0x7fbe5402b960 comp rx=0 tx=0).stop 2024-03-28T13:12:32.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.644+0000 7fbe69c00640 1 -- 172.21.15.19:0/40180885 shutdown_connections 2024-03-28T13:12:32.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.644+0000 7fbe69c00640 1 --2- 172.21.15.19:0/40180885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6415e3b0 0x7fbe640d22e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:32.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.644+0000 7fbe69c00640 1 -- 172.21.15.19:0/40180885 >> 172.21.15.19:0/40180885 conn(0x7fbe640678b0 msgr2=0x7fbe640d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:32.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.644+0000 7fbe69c00640 1 -- 172.21.15.19:0/40180885 shutdown_connections 2024-03-28T13:12:32.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.644+0000 7fbe69c00640 1 -- 172.21.15.19:0/40180885 wait complete. 2024-03-28T13:12:32.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.647+0000 7fbe69c00640 1 Processor -- start 2024-03-28T13:12:32.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.647+0000 7fbe69c00640 1 -- start start 2024-03-28T13:12:32.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.649+0000 7fbe69c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe640b63c0 0x7fbe640cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:32.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.649+0000 7fbe69c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe640d3d90 0x7fbe641e8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:32.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.649+0000 7fbe69c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6415e3b0 0x7fbe64155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:32.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.649+0000 7fbe69c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe640ba030 con 0x7fbe6415e3b0 2024-03-28T13:12:32.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.649+0000 7fbe69c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6415e790 con 0x7fbe640b63c0 2024-03-28T13:12:32.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.649+0000 7fbe69c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbe640ba1b0 con 0x7fbe640d3d90 2024-03-28T13:12:32.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.649+0000 7fbe69200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6415e3b0 0x7fbe64155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:32.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.649+0000 7fbe69200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6415e3b0 0x7fbe64155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46136/0 (socket says 172.21.15.19:46136) 2024-03-28T13:12:32.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.649+0000 7fbe69200640 1 -- 172.21.15.19:0/788882874 learned_addr learned my addr 172.21.15.19:0/788882874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:32.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.649+0000 7fbe63e00640 1 --2- 172.21.15.19:0/788882874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe640b63c0 0x7fbe640cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:32.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.649+0000 7fbe63400640 1 --2- 172.21.15.19:0/788882874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe640d3d90 0x7fbe641e8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:32.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.650+0000 7fbe69200640 1 -- 172.21.15.19:0/788882874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe640d3d90 msgr2=0x7fbe641e8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:32.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.650+0000 7fbe69200640 1 --2- 172.21.15.19:0/788882874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe640d3d90 0x7fbe641e8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:32.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.650+0000 7fbe69200640 1 -- 172.21.15.19:0/788882874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe640b63c0 msgr2=0x7fbe640cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:32.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.650+0000 7fbe69200640 1 --2- 172.21.15.19:0/788882874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe640b63c0 0x7fbe640cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:32.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.650+0000 7fbe69200640 1 -- 172.21.15.19:0/788882874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe640c7e00 con 0x7fbe6415e3b0 2024-03-28T13:12:32.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.650+0000 7fbe69200640 1 --2- 172.21.15.19:0/788882874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6415e3b0 0x7fbe64155b00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00c410 tx=0x7fbe5c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:32.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.650+0000 7fbe5be00640 1 -- 172.21.15.19:0/788882874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c011790 con 0x7fbe6415e3b0 2024-03-28T13:12:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.651+0000 7fbe5be00640 1 -- 172.21.15.19:0/788882874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe5c017070 con 0x7fbe6415e3b0 2024-03-28T13:12:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.651+0000 7fbe5be00640 1 -- 172.21.15.19:0/788882874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c011b70 con 0x7fbe6415e3b0 2024-03-28T13:12:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.651+0000 7fbe69c00640 1 -- 172.21.15.19:0/788882874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe640c80a0 con 0x7fbe6415e3b0 2024-03-28T13:12:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.651+0000 7fbe69c00640 1 -- 172.21.15.19:0/788882874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe640c8660 con 0x7fbe6415e3b0 2024-03-28T13:12:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.652+0000 7fbe69c00640 1 -- 172.21.15.19:0/788882874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe64075940 con 0x7fbe6415e3b0 2024-03-28T13:12:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.653+0000 7fbe5be00640 1 -- 172.21.15.19:0/788882874 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbe5c011d10 con 0x7fbe6415e3b0 2024-03-28T13:12:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.654+0000 7fbe5be00640 1 --2- 172.21.15.19:0/788882874 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbe4006c530 0x7fbe4006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.655+0000 7fbe5be00640 1 -- 172.21.15.19:0/788882874 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7fbe5c08b2b0 con 0x7fbe6415e3b0 2024-03-28T13:12:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.655+0000 7fbe63e00640 1 --2- 172.21.15.19:0/788882874 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbe4006c530 0x7fbe4006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.656+0000 7fbe63e00640 1 --2- 172.21.15.19:0/788882874 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbe4006c530 0x7fbe4006e9f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbe54036000 tx=0x7fbe5403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:32.659+0000 7fbe5be00640 1 -- 172.21.15.19:0/788882874 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbe5c021020 con 0x7fbe6415e3b0 2024-03-28T13:12:33.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.032+0000 7fbe69c00640 1 -- 172.21.15.19:0/788882874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe640ce910 con 0x7fbe6415e3b0 2024-03-28T13:12:33.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.033+0000 7fbe5be00640 1 -- 172.21.15.19:0/788882874 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7fbe5c059830 con 0x7fbe6415e3b0 2024-03-28T13:12:33.034 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:33.034 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2125648249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2125648249},{"type":"v1","addr":"172.21.15.188:6836","nonce":2125648249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/429866161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":429866161},{"type":"v1","addr":"172.21.15.188:6837","nonce":429866161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1422209289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1422209289},{"type":"v1","addr":"172.21.15.19:6834","nonce":1422209289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:11:46.379929+0000","modified":"2024-03-28T13:11:49.470791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5538},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5538":{"gid":5538,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/179338886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":179338886},{"type":"v1","addr":"172.21.15.19:6835","nonce":179338886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5538,"qdb_cluster":[5538]},"id":3}]} 2024-03-28T13:12:33.034 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-03-28T13:12:33.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.036+0000 7fbe69c00640 1 -- 172.21.15.19:0/788882874 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbe4006c530 msgr2=0x7fbe4006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:33.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.036+0000 7fbe69c00640 1 --2- 172.21.15.19:0/788882874 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbe4006c530 0x7fbe4006e9f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbe54036000 tx=0x7fbe5403c040 comp rx=0 tx=0).stop 2024-03-28T13:12:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.036+0000 7fbe69c00640 1 -- 172.21.15.19:0/788882874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6415e3b0 msgr2=0x7fbe64155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.036+0000 7fbe69c00640 1 --2- 172.21.15.19:0/788882874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6415e3b0 0x7fbe64155b00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00c410 tx=0x7fbe5c00c8d0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.036+0000 7fbe69c00640 1 -- 172.21.15.19:0/788882874 shutdown_connections 2024-03-28T13:12:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.036+0000 7fbe69c00640 1 --2- 172.21.15.19:0/788882874 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbe4006c530 0x7fbe4006e9f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.036+0000 7fbe69c00640 1 --2- 172.21.15.19:0/788882874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe6415e3b0 0x7fbe64155b00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.036+0000 7fbe69c00640 1 --2- 172.21.15.19:0/788882874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe640d3d90 0x7fbe641e8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.036+0000 7fbe69c00640 1 --2- 172.21.15.19:0/788882874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe640b63c0 0x7fbe640cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.036+0000 7fbe69c00640 1 -- 172.21.15.19:0/788882874 >> 172.21.15.19:0/788882874 conn(0x7fbe640678b0 msgr2=0x7fbe640b7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.036+0000 7fbe69c00640 1 -- 172.21.15.19:0/788882874 shutdown_connections 2024-03-28T13:12:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.036+0000 7fbe69c00640 1 -- 172.21.15.19:0/788882874 wait complete. 2024-03-28T13:12:33.054 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:12:33.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.259+0000 7feb5a200640 1 Processor -- start 2024-03-28T13:12:33.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.259+0000 7feb5a200640 1 -- start start 2024-03-28T13:12:33.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.261+0000 7feb5a200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb5415e4a0 0x7feb540b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:33.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.261+0000 7feb5a200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feb540d2310 0x7feb540ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:33.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.261+0000 7feb5a200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb5415e880 con 0x7feb540cf530 2024-03-28T13:12:33.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.261+0000 7feb5a200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7feb540cf9a0 con 0x7feb5415e4a0 2024-03-28T13:12:33.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.261+0000 7feb5a200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7feb540d34e0 con 0x7feb540d2310 2024-03-28T13:12:33.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.261+0000 7feb58e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb5415e4a0 0x7feb540b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:33.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.261+0000 7feb58e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb5415e4a0 0x7feb540b9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46150/0 (socket says 172.21.15.19:46150) 2024-03-28T13:12:33.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.261+0000 7feb58e00640 1 -- 172.21.15.19:0/2176941905 learned_addr learned my addr 172.21.15.19:0/2176941905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:33.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.262+0000 7feb53e00640 1 --2- 172.21.15.19:0/2176941905 >> v2:172.21.15.188:3300/0 conn(0x7feb540d2310 0x7feb540ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:33.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.262+0000 7feb58e00640 1 -- 172.21.15.19:0/2176941905 >> v2:172.21.15.188:3300/0 conn(0x7feb540d2310 msgr2=0x7feb540ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:33.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.262+0000 7feb58e00640 1 --2- 172.21.15.19:0/2176941905 >> v2:172.21.15.188:3300/0 conn(0x7feb540d2310 0x7feb540ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.262+0000 7feb58e00640 1 -- 172.21.15.19:0/2176941905 >> v1:172.21.15.188:6789/0 conn(0x7feb540cf530 legacy=0x7feb540d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:33.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.262+0000 7feb58e00640 1 -- 172.21.15.19:0/2176941905 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb540d28d0 con 0x7feb5415e4a0 2024-03-28T13:12:33.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.263+0000 7feb58e00640 1 --2- 172.21.15.19:0/2176941905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb5415e4a0 0x7feb540b9c60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7feb540d3660 tx=0x7feb4802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afbbbea5dee1116 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:33.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.264+0000 7feb53400640 1 -- 172.21.15.19:0/2176941905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4802e070 con 0x7feb5415e4a0 2024-03-28T13:12:33.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.264+0000 7feb53400640 1 -- 172.21.15.19:0/2176941905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb48037070 con 0x7feb5415e4a0 2024-03-28T13:12:33.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.264+0000 7feb53400640 1 -- 172.21.15.19:0/2176941905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48031a00 con 0x7feb5415e4a0 2024-03-28T13:12:33.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.264+0000 7feb5a200640 1 -- 172.21.15.19:0/2176941905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb5415e4a0 msgr2=0x7feb540b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.264+0000 7feb5a200640 1 --2- 172.21.15.19:0/2176941905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb5415e4a0 0x7feb540b9c60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7feb540d3660 tx=0x7feb4802b880 comp rx=0 tx=0).stop 2024-03-28T13:12:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.264+0000 7feb5a200640 1 -- 172.21.15.19:0/2176941905 shutdown_connections 2024-03-28T13:12:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.264+0000 7feb5a200640 1 --2- 172.21.15.19:0/2176941905 >> v2:172.21.15.188:3300/0 conn(0x7feb540d2310 0x7feb540ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.264+0000 7feb5a200640 1 --2- 172.21.15.19:0/2176941905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb5415e4a0 0x7feb540b9c60 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.264+0000 7feb5a200640 1 -- 172.21.15.19:0/2176941905 >> 172.21.15.19:0/2176941905 conn(0x7feb540676d0 msgr2=0x7feb54067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.264+0000 7feb5a200640 1 -- 172.21.15.19:0/2176941905 shutdown_connections 2024-03-28T13:12:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.264+0000 7feb5a200640 1 -- 172.21.15.19:0/2176941905 wait complete. 2024-03-28T13:12:33.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.266+0000 7feb5a200640 1 Processor -- start 2024-03-28T13:12:33.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.266+0000 7feb5a200640 1 -- start start 2024-03-28T13:12:33.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.267+0000 7feb5a200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb540cf530 0x7feb540c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:33.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.267+0000 7feb5a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb540d2310 0x7feb541604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:33.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.267+0000 7feb5a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb5415e4a0 0x7feb54155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:33.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.267+0000 7feb5a200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feb540d3170 con 0x7feb540d2310 2024-03-28T13:12:33.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.267+0000 7feb5a200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feb540d2e90 con 0x7feb540cf530 2024-03-28T13:12:33.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.267+0000 7feb5a200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feb540d3970 con 0x7feb5415e4a0 2024-03-28T13:12:33.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.268+0000 7feb53e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb540d2310 0x7feb541604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:33.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.268+0000 7feb53e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb540d2310 0x7feb541604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46162/0 (socket says 172.21.15.19:46162) 2024-03-28T13:12:33.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.268+0000 7feb53e00640 1 -- 172.21.15.19:0/2737813188 learned_addr learned my addr 172.21.15.19:0/2737813188 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:33.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.268+0000 7feb58e00640 1 --2- 172.21.15.19:0/2737813188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb540cf530 0x7feb540c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:33.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.268+0000 7feb59800640 1 --2- 172.21.15.19:0/2737813188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb5415e4a0 0x7feb54155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:33.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.268+0000 7feb53e00640 1 -- 172.21.15.19:0/2737813188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb5415e4a0 msgr2=0x7feb54155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:33.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.268+0000 7feb53e00640 1 --2- 172.21.15.19:0/2737813188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb5415e4a0 0x7feb54155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.268+0000 7feb53e00640 1 -- 172.21.15.19:0/2737813188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb540cf530 msgr2=0x7feb540c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:33.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.268+0000 7feb53e00640 1 --2- 172.21.15.19:0/2737813188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb540cf530 0x7feb540c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.268+0000 7feb53e00640 1 -- 172.21.15.19:0/2737813188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb54160e30 con 0x7feb540d2310 2024-03-28T13:12:33.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.268+0000 7feb53e00640 1 --2- 172.21.15.19:0/2737813188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb540d2310 0x7feb541604e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7feb4000d800 tx=0x7feb4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:33.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.269+0000 7feb50c00640 1 -- 172.21.15.19:0/2737813188 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4000b6c0 con 0x7feb540d2310 2024-03-28T13:12:33.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.269+0000 7feb50c00640 1 -- 172.21.15.19:0/2737813188 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb40019070 con 0x7feb540d2310 2024-03-28T13:12:33.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.269+0000 7feb50c00640 1 -- 172.21.15.19:0/2737813188 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4000ba80 con 0x7feb540d2310 2024-03-28T13:12:33.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.269+0000 7feb5a200640 1 -- 172.21.15.19:0/2737813188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb54161110 con 0x7feb540d2310 2024-03-28T13:12:33.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.269+0000 7feb5a200640 1 -- 172.21.15.19:0/2737813188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb54152820 con 0x7feb540d2310 2024-03-28T13:12:33.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.270+0000 7feb5a200640 1 -- 172.21.15.19:0/2737813188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb54075940 con 0x7feb540d2310 2024-03-28T13:12:33.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.271+0000 7feb50c00640 1 -- 172.21.15.19:0/2737813188 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7feb4000bc20 con 0x7feb540d2310 2024-03-28T13:12:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.272+0000 7feb50c00640 1 --2- 172.21.15.19:0/2737813188 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb3406c560 0x7feb3406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.272+0000 7feb50c00640 1 -- 172.21.15.19:0/2737813188 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7feb4008bbe0 con 0x7feb540d2310 2024-03-28T13:12:33.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.273+0000 7feb58e00640 1 --2- 172.21.15.19:0/2737813188 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb3406c560 0x7feb3406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:33.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.274+0000 7feb58e00640 1 --2- 172.21.15.19:0/2737813188 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb3406c560 0x7feb3406ea20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feb48036000 tx=0x7feb4803d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:33.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.277+0000 7feb50c00640 1 -- 172.21.15.19:0/2737813188 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7feb4001c360 con 0x7feb540d2310 2024-03-28T13:12:33.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.481+0000 7feb5a200640 1 -- 172.21.15.19:0/2737813188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb54152c20 con 0x7feb540d2310 2024-03-28T13:12:33.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.485+0000 7feb50c00640 1 -- 172.21.15.19:0/2737813188 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16900 (secure 0 0 0) 0x7feb4005a170 con 0x7feb540d2310 2024-03-28T13:12:33.485 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:33.485 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":51,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:12:24.186519+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:11:44.316258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-28T13:11:45.360154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.0","mappings":[{"from":5,"to":0}]},{"pgid":"7.6","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.39","mappings":[{"from":5,"to":7}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:12:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.487+0000 7feb5a200640 1 -- 172.21.15.19:0/2737813188 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb3406c560 msgr2=0x7feb3406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.487+0000 7feb5a200640 1 --2- 172.21.15.19:0/2737813188 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb3406c560 0x7feb3406ea20 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feb48036000 tx=0x7feb4803d040 comp rx=0 tx=0).stop 2024-03-28T13:12:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.487+0000 7feb5a200640 1 -- 172.21.15.19:0/2737813188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb540d2310 msgr2=0x7feb541604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.487+0000 7feb5a200640 1 --2- 172.21.15.19:0/2737813188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb540d2310 0x7feb541604e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7feb4000d800 tx=0x7feb4000dcc0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.487+0000 7feb5a200640 1 -- 172.21.15.19:0/2737813188 shutdown_connections 2024-03-28T13:12:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.487+0000 7feb5a200640 1 --2- 172.21.15.19:0/2737813188 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb3406c560 0x7feb3406ea20 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.487+0000 7feb5a200640 1 --2- 172.21.15.19:0/2737813188 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb5415e4a0 0x7feb54155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.487+0000 7feb5a200640 1 --2- 172.21.15.19:0/2737813188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb540d2310 0x7feb541604e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.487+0000 7feb5a200640 1 --2- 172.21.15.19:0/2737813188 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb540cf530 0x7feb540c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.487+0000 7feb5a200640 1 -- 172.21.15.19:0/2737813188 >> 172.21.15.19:0/2737813188 conn(0x7feb540676d0 msgr2=0x7feb540b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:33.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.488+0000 7feb5a200640 1 -- 172.21.15.19:0/2737813188 shutdown_connections 2024-03-28T13:12:33.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.488+0000 7feb5a200640 1 -- 172.21.15.19:0/2737813188 wait complete. 2024-03-28T13:12:33.506 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:11:44.316258+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:11:45.360154+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:12:33.507 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:12:33.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.732+0000 7f4852c00640 1 Processor -- start 2024-03-28T13:12:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.733+0000 7f4852c00640 1 -- start start 2024-03-28T13:12:33.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.734+0000 7f4852c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484c0d0990 0x7f484c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:33.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.734+0000 7f4852c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f484c0afab0 con 0x7f484c153580 2024-03-28T13:12:33.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.734+0000 7f4852c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f484c15cc80 con 0x7f484c0af2e0 2024-03-28T13:12:33.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.734+0000 7f4852c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f484c15da00 con 0x7f484c0d0990 2024-03-28T13:12:33.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.735+0000 7f484be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f484c153580 0x7f484c0d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45066/0 (socket says 172.21.15.19:45066) 2024-03-28T13:12:33.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.735+0000 7f484be00640 1 -- 172.21.15.19:0/3555891097 learned_addr learned my addr 172.21.15.19:0/3555891097 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:33.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.735+0000 7f484b400640 1 --2- 172.21.15.19:0/3555891097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484c0d0990 0x7f484c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.735+0000 7f484aa00640 1 -- 172.21.15.19:0/3555891097 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2358669157 0 0) 0x7f484c0afab0 con 0x7f484c153580 2024-03-28T13:12:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.735+0000 7f484aa00640 1 -- 172.21.15.19:0/3555891097 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f482c003520 con 0x7f484c153580 2024-03-28T13:12:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.735+0000 7f484aa00640 1 -- 172.21.15.19:0/3555891097 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1320622218 0 0) 0x7f482c003520 con 0x7f484c153580 2024-03-28T13:12:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.735+0000 7f484aa00640 1 -- 172.21.15.19:0/3555891097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484c0d0990 msgr2=0x7f484c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.735+0000 7f484aa00640 1 --2- 172.21.15.19:0/3555891097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484c0d0990 0x7f484c0aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.735+0000 7f484aa00640 1 -- 172.21.15.19:0/3555891097 >> v1:172.21.15.188:6789/0 conn(0x7f484c0af2e0 legacy=0x7f484c1600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.735+0000 7f484aa00640 1 -- 172.21.15.19:0/3555891097 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484c15cf20 con 0x7f484c153580 2024-03-28T13:12:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.735+0000 7f484aa00640 1 -- 172.21.15.19:0/3555891097 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f483c001670 con 0x7f484c153580 2024-03-28T13:12:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.736+0000 7f484aa00640 1 -- 172.21.15.19:0/3555891097 >> v1:172.21.15.19:6789/0 conn(0x7f484c153580 legacy=0x7f484c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.736+0000 7f484aa00640 1 --2- 172.21.15.19:0/3555891097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482c0039e0 0x7f482c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.736+0000 7f484aa00640 1 --2- 172.21.15.19:0/3555891097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484c0d0990 0x7f482c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.736+0000 7f484aa00640 1 --2- 172.21.15.19:0/3555891097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f482c004d40 0x7f482c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.736+0000 7f484aa00640 1 -- 172.21.15.19:0/3555891097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f484c15da00 con 0x7f482c004d40 2024-03-28T13:12:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.736+0000 7f484aa00640 1 -- 172.21.15.19:0/3555891097 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f482c00baf0 con 0x7f482c0039e0 2024-03-28T13:12:33.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.736+0000 7f484aa00640 1 -- 172.21.15.19:0/3555891097 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f482c00bcf0 con 0x7f484c0d0990 2024-03-28T13:12:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.736+0000 7f4850c00640 1 --2- 172.21.15.19:0/3555891097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f482c004d40 0x7f482c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.736+0000 7f484be00640 1 --2- 172.21.15.19:0/3555891097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482c0039e0 0x7f482c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.736+0000 7f484b400640 1 --2- 172.21.15.19:0/3555891097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484c0d0990 0x7f482c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.736+0000 7f4850c00640 1 -- 172.21.15.19:0/3555891097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484c0d0990 msgr2=0x7f482c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.736+0000 7f4850c00640 1 --2- 172.21.15.19:0/3555891097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484c0d0990 0x7f482c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.737+0000 7f4850c00640 1 -- 172.21.15.19:0/3555891097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482c0039e0 msgr2=0x7f482c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.737+0000 7f4850c00640 1 --2- 172.21.15.19:0/3555891097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f482c0039e0 0x7f482c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.737+0000 7f4850c00640 1 -- 172.21.15.19:0/3555891097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482c003520 con 0x7f482c004d40 2024-03-28T13:12:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.738+0000 7f4850c00640 1 --2- 172.21.15.19:0/3555891097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f482c004d40 0x7f482c0051c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4844001750 tx=0x7f482c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945040fcf076a3cf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.738+0000 7f484aa00640 1 -- 172.21.15.19:0/3555891097 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844001370 con 0x7f482c004d40 2024-03-28T13:12:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.738+0000 7f484aa00640 1 -- 172.21.15.19:0/3555891097 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48440052d0 con 0x7f482c004d40 2024-03-28T13:12:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.738+0000 7f484aa00640 1 -- 172.21.15.19:0/3555891097 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844003830 con 0x7f482c004d40 2024-03-28T13:12:33.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.738+0000 7f4852c00640 1 -- 172.21.15.19:0/3555891097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f482c004d40 msgr2=0x7f482c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.738+0000 7f4852c00640 1 --2- 172.21.15.19:0/3555891097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f482c004d40 0x7f482c0051c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4844001750 tx=0x7f482c005830 comp rx=0 tx=0).stop 2024-03-28T13:12:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.738+0000 7f4850c00640 1 -- 172.21.15.19:0/3555891097 reap_dead start 2024-03-28T13:12:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.739+0000 7f4852c00640 1 -- 172.21.15.19:0/3555891097 shutdown_connections 2024-03-28T13:12:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.739+0000 7f4852c00640 1 -- 172.21.15.19:0/3555891097 >> 172.21.15.19:0/3555891097 conn(0x7f484c06b750 msgr2=0x7f484c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.739+0000 7f4852c00640 1 -- 172.21.15.19:0/3555891097 shutdown_connections 2024-03-28T13:12:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.739+0000 7f4852c00640 1 -- 172.21.15.19:0/3555891097 wait complete. 2024-03-28T13:12:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.741+0000 7f4852c00640 1 Processor -- start 2024-03-28T13:12:33.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.741+0000 7f4852c00640 1 -- start start 2024-03-28T13:12:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.742+0000 7f4852c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f482c004d40 0x7f484c0cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.742+0000 7f4852c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f484c0cfb10 0x7f484c15b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.742+0000 7f4852c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484c1ee050 0x7f484c1ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:33.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.742+0000 7f4852c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f484c153960 con 0x7f482c004d40 2024-03-28T13:12:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.742+0000 7f4852c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f484c15d4e0 con 0x7f484c0cfb10 2024-03-28T13:12:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.742+0000 7f4852c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f484c0af710 con 0x7f484c1ee050 2024-03-28T13:12:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.742+0000 7f484be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f482c004d40 0x7f484c0cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.742+0000 7f484be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f482c004d40 0x7f484c0cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46186/0 (socket says 172.21.15.19:46186) 2024-03-28T13:12:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.742+0000 7f484be00640 1 -- 172.21.15.19:0/3806026480 learned_addr learned my addr 172.21.15.19:0/3806026480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.743+0000 7f484b400640 1 --2- 172.21.15.19:0/3806026480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f484c0cfb10 0x7f484c15b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.743+0000 7f4850c00640 1 --2- 172.21.15.19:0/3806026480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484c1ee050 0x7f484c1ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.743+0000 7f484be00640 1 -- 172.21.15.19:0/3806026480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484c1ee050 msgr2=0x7f484c1ee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.743+0000 7f484be00640 1 --2- 172.21.15.19:0/3806026480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484c1ee050 0x7f484c1ee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.743+0000 7f484be00640 1 -- 172.21.15.19:0/3806026480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f484c0cfb10 msgr2=0x7f484c15b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.743+0000 7f484be00640 1 --2- 172.21.15.19:0/3806026480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f484c0cfb10 0x7f484c15b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.743+0000 7f484be00640 1 -- 172.21.15.19:0/3806026480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484c1eea10 con 0x7f482c004d40 2024-03-28T13:12:33.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.743+0000 7f4850c00640 1 --2- 172.21.15.19:0/3806026480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484c1ee050 0x7f484c1ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.743+0000 7f484be00640 1 --2- 172.21.15.19:0/3806026480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f482c004d40 0x7f484c0cd2d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f483c00e8a0 tx=0x7f483c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.743+0000 7f4843e00640 1 -- 172.21.15.19:0/3806026480 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c00c850 con 0x7f482c004d40 2024-03-28T13:12:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.743+0000 7f4843e00640 1 -- 172.21.15.19:0/3806026480 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f483c018070 con 0x7f482c004d40 2024-03-28T13:12:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.743+0000 7f4852c00640 1 -- 172.21.15.19:0/3806026480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f484c1f4640 con 0x7f482c004d40 2024-03-28T13:12:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.743+0000 7f4843e00640 1 -- 172.21.15.19:0/3806026480 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c00cc50 con 0x7f482c004d40 2024-03-28T13:12:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.744+0000 7f4852c00640 1 -- 172.21.15.19:0/3806026480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f484c1f4c00 con 0x7f482c004d40 2024-03-28T13:12:33.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.745+0000 7f4852c00640 1 -- 172.21.15.19:0/3806026480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f484c06ff20 con 0x7f482c004d40 2024-03-28T13:12:33.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.746+0000 7f4843e00640 1 -- 172.21.15.19:0/3806026480 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f483c01d460 con 0x7f482c004d40 2024-03-28T13:12:33.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.747+0000 7f4843e00640 1 --2- 172.21.15.19:0/3806026480 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f482806c530 0x7f482806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:33.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.747+0000 7f4843e00640 1 -- 172.21.15.19:0/3806026480 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7f483c08b6d0 con 0x7f482c004d40 2024-03-28T13:12:33.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.747+0000 7f484b400640 1 --2- 172.21.15.19:0/3806026480 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f482806c530 0x7f482806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:33.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.749+0000 7f484b400640 1 --2- 172.21.15.19:0/3806026480 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f482806c530 0x7f482806e9f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f484c153150 tx=0x7f483400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:33.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.751+0000 7f4843e00640 1 -- 172.21.15.19:0/3806026480 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f484c06ff20 con 0x7f482c004d40 2024-03-28T13:12:33.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.956+0000 7f4852c00640 1 -- 172.21.15.19:0/3806026480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f484c152ea0 con 0x7f482c004d40 2024-03-28T13:12:33.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.957+0000 7f4843e00640 1 -- 172.21.15.19:0/3806026480 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) ==== 93+0+10 (secure 0 0 0) 0x7f484c1eea10 con 0x7f482c004d40 2024-03-28T13:12:33.957 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:12:33.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.959+0000 7f4852c00640 1 -- 172.21.15.19:0/3806026480 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f482806c530 msgr2=0x7f482806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.959+0000 7f4852c00640 1 --2- 172.21.15.19:0/3806026480 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f482806c530 0x7f482806e9f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f484c153150 tx=0x7f483400b040 comp rx=0 tx=0).stop 2024-03-28T13:12:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.959+0000 7f4852c00640 1 -- 172.21.15.19:0/3806026480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f482c004d40 msgr2=0x7f484c0cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.959+0000 7f4852c00640 1 --2- 172.21.15.19:0/3806026480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f482c004d40 0x7f484c0cd2d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f483c00e8a0 tx=0x7f483c00ed60 comp rx=0 tx=0).stop 2024-03-28T13:12:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.959+0000 7f4852c00640 1 -- 172.21.15.19:0/3806026480 shutdown_connections 2024-03-28T13:12:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.959+0000 7f4852c00640 1 --2- 172.21.15.19:0/3806026480 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f482806c530 0x7f482806e9f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.959+0000 7f4852c00640 1 --2- 172.21.15.19:0/3806026480 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484c1ee050 0x7f484c1ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.959+0000 7f4852c00640 1 --2- 172.21.15.19:0/3806026480 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f484c0cfb10 0x7f484c15b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.959+0000 7f4852c00640 1 --2- 172.21.15.19:0/3806026480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f482c004d40 0x7f484c0cd2d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.959+0000 7f4852c00640 1 -- 172.21.15.19:0/3806026480 >> 172.21.15.19:0/3806026480 conn(0x7f484c06b750 msgr2=0x7f484c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.959+0000 7f4852c00640 1 -- 172.21.15.19:0/3806026480 shutdown_connections 2024-03-28T13:12:33.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:33.959+0000 7f4852c00640 1 -- 172.21.15.19:0/3806026480 wait complete. 2024-03-28T13:12:33.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:12:34.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.195+0000 7fd2a9e00640 1 Processor -- start 2024-03-28T13:12:34.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.195+0000 7fd2a9e00640 1 -- start start 2024-03-28T13:12:34.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.196+0000 7fd2a9e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd2a415e4c0 0x7fd2a40b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:34.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.196+0000 7fd2a9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2a40d37b0 0x7fd2a40b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:34.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.196+0000 7fd2a9e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2a40b67f0 0x7fd2a40d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:34.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.196+0000 7fd2a9e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a40b6c60 con 0x7fd2a40d37b0 2024-03-28T13:12:34.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.196+0000 7fd2a9e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fd2a415e8a0 con 0x7fd2a415e4c0 2024-03-28T13:12:34.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.196+0000 7fd2a9e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a40d4040 con 0x7fd2a40b67f0 2024-03-28T13:12:34.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.197+0000 7fd29be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2a40d37b0 0x7fd2a40b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:34.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.197+0000 7fd29be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd2a40d37b0 0x7fd2a40b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46194/0 (socket says 172.21.15.19:46194) 2024-03-28T13:12:34.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.197+0000 7fd29be00640 1 -- 172.21.15.19:0/2431003299 learned_addr learned my addr 172.21.15.19:0/2431003299 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.197+0000 7fd2a9400640 1 --2- 172.21.15.19:0/2431003299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2a40b67f0 0x7fd2a40d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.197+0000 7fd2a8a00640 1 --2- 172.21.15.19:0/2431003299 >> v2:172.21.15.188:3300/0 conn(0x7fd2a415e4c0 0x7fd2a40b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.197+0000 7fd29be00640 1 -- 172.21.15.19:0/2431003299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2a40b67f0 msgr2=0x7fd2a40d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.197+0000 7fd29be00640 1 --2- 172.21.15.19:0/2431003299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2a40b67f0 0x7fd2a40d4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.197+0000 7fd29be00640 1 -- 172.21.15.19:0/2431003299 >> v2:172.21.15.188:3300/0 conn(0x7fd2a415e4c0 msgr2=0x7fd2a40b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.197+0000 7fd29be00640 1 --2- 172.21.15.19:0/2431003299 >> v2:172.21.15.188:3300/0 conn(0x7fd2a415e4c0 0x7fd2a40b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.197+0000 7fd29be00640 1 -- 172.21.15.19:0/2431003299 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a40d2710 con 0x7fd2a40d37b0 2024-03-28T13:12:34.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.197+0000 7fd2a8a00640 1 --2- 172.21.15.19:0/2431003299 >> v2:172.21.15.188:3300/0 conn(0x7fd2a415e4c0 0x7fd2a40b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:34.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.198+0000 7fd29be00640 1 --2- 172.21.15.19:0/2431003299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a40d37b0 0x7fd2a40b62b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd2a40d0d70 tx=0x7fd28c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98fc956579a9419 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:34.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.199+0000 7fd2a3e00640 1 -- 172.21.15.19:0/2431003299 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c035020 con 0x7fd2a40d37b0 2024-03-28T13:12:34.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.199+0000 7fd2a3e00640 1 -- 172.21.15.19:0/2431003299 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd28c02d040 con 0x7fd2a40d37b0 2024-03-28T13:12:34.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.199+0000 7fd2a3e00640 1 -- 172.21.15.19:0/2431003299 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28c031a80 con 0x7fd2a40d37b0 2024-03-28T13:12:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.199+0000 7fd2a9e00640 1 -- 172.21.15.19:0/2431003299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a40d37b0 msgr2=0x7fd2a40b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.199+0000 7fd2a9e00640 1 --2- 172.21.15.19:0/2431003299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a40d37b0 0x7fd2a40b62b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd2a40d0d70 tx=0x7fd28c02b900 comp rx=0 tx=0).stop 2024-03-28T13:12:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.200+0000 7fd2a9e00640 1 -- 172.21.15.19:0/2431003299 shutdown_connections 2024-03-28T13:12:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.200+0000 7fd2a9e00640 1 --2- 172.21.15.19:0/2431003299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2a40b67f0 0x7fd2a40d4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.200+0000 7fd2a9e00640 1 --2- 172.21.15.19:0/2431003299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a40d37b0 0x7fd2a40b62b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.200+0000 7fd2a9e00640 1 --2- 172.21.15.19:0/2431003299 >> v2:172.21.15.188:3300/0 conn(0x7fd2a415e4c0 0x7fd2a40b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.200+0000 7fd2a9e00640 1 -- 172.21.15.19:0/2431003299 >> 172.21.15.19:0/2431003299 conn(0x7fd2a40676a0 msgr2=0x7fd2a4067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.200+0000 7fd2a9e00640 1 -- 172.21.15.19:0/2431003299 shutdown_connections 2024-03-28T13:12:34.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.200+0000 7fd2a9e00640 1 -- 172.21.15.19:0/2431003299 wait complete. 2024-03-28T13:12:34.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.201+0000 7fd2a9e00640 1 Processor -- start 2024-03-28T13:12:34.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.202+0000 7fd2a9e00640 1 -- start start 2024-03-28T13:12:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.202+0000 7fd2a9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a40b67f0 0x7fd2a40cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.202+0000 7fd2a9e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2a40d37b0 0x7fd2a41f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.203+0000 7fd2a9e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2a415e4c0 0x7fd2a41f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.203+0000 7fd2a9e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a40d2cd0 con 0x7fd2a40b67f0 2024-03-28T13:12:34.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.203+0000 7fd2a9e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a40d3320 con 0x7fd2a415e4c0 2024-03-28T13:12:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.203+0000 7fd2a9e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a40d2fb0 con 0x7fd2a40d37b0 2024-03-28T13:12:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.203+0000 7fd2a8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a40b67f0 0x7fd2a40cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.203+0000 7fd2a8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a40b67f0 0x7fd2a40cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46210/0 (socket says 172.21.15.19:46210) 2024-03-28T13:12:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.203+0000 7fd2a8a00640 1 -- 172.21.15.19:0/4146404015 learned_addr learned my addr 172.21.15.19:0/4146404015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.203+0000 7fd29be00640 1 --2- 172.21.15.19:0/4146404015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2a40d37b0 0x7fd2a41f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.203+0000 7fd2a8a00640 1 -- 172.21.15.19:0/4146404015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2a40d37b0 msgr2=0x7fd2a41f3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.203+0000 7fd2a9400640 1 --2- 172.21.15.19:0/4146404015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2a415e4c0 0x7fd2a41f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.203+0000 7fd2a8a00640 1 --2- 172.21.15.19:0/4146404015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2a40d37b0 0x7fd2a41f3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.203+0000 7fd2a8a00640 1 -- 172.21.15.19:0/4146404015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2a415e4c0 msgr2=0x7fd2a41f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.203+0000 7fd2a8a00640 1 --2- 172.21.15.19:0/4146404015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2a415e4c0 0x7fd2a41f4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.203+0000 7fd2a8a00640 1 -- 172.21.15.19:0/4146404015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a415f790 con 0x7fd2a40b67f0 2024-03-28T13:12:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.203+0000 7fd29be00640 1 --2- 172.21.15.19:0/4146404015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2a40d37b0 0x7fd2a41f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.204+0000 7fd2a8a00640 1 --2- 172.21.15.19:0/4146404015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a40b67f0 0x7fd2a40cb3a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd29400c820 tx=0x7fd29400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.204+0000 7fd2a1600640 1 -- 172.21.15.19:0/4146404015 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294015070 con 0x7fd2a40b67f0 2024-03-28T13:12:34.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.204+0000 7fd2a1600640 1 -- 172.21.15.19:0/4146404015 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd294007a30 con 0x7fd2a40b67f0 2024-03-28T13:12:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.204+0000 7fd2a1600640 1 -- 172.21.15.19:0/4146404015 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294007de0 con 0x7fd2a40b67f0 2024-03-28T13:12:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.204+0000 7fd2a9e00640 1 -- 172.21.15.19:0/4146404015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a41f4850 con 0x7fd2a40b67f0 2024-03-28T13:12:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.204+0000 7fd2a9e00640 1 -- 172.21.15.19:0/4146404015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a415fa10 con 0x7fd2a40b67f0 2024-03-28T13:12:34.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.205+0000 7fd2a9e00640 1 -- 172.21.15.19:0/4146404015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a40cfcb0 con 0x7fd2a40b67f0 2024-03-28T13:12:34.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.206+0000 7fd2a1600640 1 -- 172.21.15.19:0/4146404015 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd294018070 con 0x7fd2a40b67f0 2024-03-28T13:12:34.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.207+0000 7fd2a1600640 1 --2- 172.21.15.19:0/4146404015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd28006c530 0x7fd28006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:34.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.207+0000 7fd2a1600640 1 -- 172.21.15.19:0/4146404015 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7fd29408b580 con 0x7fd2a40b67f0 2024-03-28T13:12:34.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.208+0000 7fd29be00640 1 --2- 172.21.15.19:0/4146404015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd28006c530 0x7fd28006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:34.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.209+0000 7fd29be00640 1 --2- 172.21.15.19:0/4146404015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd28006c530 0x7fd28006e9f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd28c036000 tx=0x7fd28c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:34.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.212+0000 7fd2a1600640 1 -- 172.21.15.19:0/4146404015 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd2a40cfcb0 con 0x7fd2a40b67f0 2024-03-28T13:12:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.420+0000 7fd2a9e00640 1 -- 172.21.15.19:0/4146404015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd2a40b6490 con 0x7fd2a40b67f0 2024-03-28T13:12:34.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.421+0000 7fd2a1600640 1 -- 172.21.15.19:0/4146404015 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) ==== 104+0+11 (secure 0 0 0) 0x7fd2a40b6490 con 0x7fd2a40b67f0 2024-03-28T13:12:34.421 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:12:34.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.423+0000 7fd2a9e00640 1 -- 172.21.15.19:0/4146404015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd28006c530 msgr2=0x7fd28006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:34.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.423+0000 7fd2a9e00640 1 --2- 172.21.15.19:0/4146404015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd28006c530 0x7fd28006e9f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd28c036000 tx=0x7fd28c03c040 comp rx=0 tx=0).stop 2024-03-28T13:12:34.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.424+0000 7fd2a9e00640 1 -- 172.21.15.19:0/4146404015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a40b67f0 msgr2=0x7fd2a40cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:34.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.424+0000 7fd2a9e00640 1 --2- 172.21.15.19:0/4146404015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a40b67f0 0x7fd2a40cb3a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd29400c820 tx=0x7fd29400cce0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.424+0000 7fd2a9e00640 1 -- 172.21.15.19:0/4146404015 shutdown_connections 2024-03-28T13:12:34.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.424+0000 7fd2a9e00640 1 --2- 172.21.15.19:0/4146404015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd28006c530 0x7fd28006e9f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.424+0000 7fd2a9e00640 1 --2- 172.21.15.19:0/4146404015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2a415e4c0 0x7fd2a41f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.424+0000 7fd2a9e00640 1 --2- 172.21.15.19:0/4146404015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2a40d37b0 0x7fd2a41f3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.424+0000 7fd2a9e00640 1 --2- 172.21.15.19:0/4146404015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2a40b67f0 0x7fd2a40cb3a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.424+0000 7fd2a9e00640 1 -- 172.21.15.19:0/4146404015 >> 172.21.15.19:0/4146404015 conn(0x7fd2a40676a0 msgr2=0x7fd2a415d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:34.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.424+0000 7fd2a9e00640 1 -- 172.21.15.19:0/4146404015 shutdown_connections 2024-03-28T13:12:34.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.424+0000 7fd2a9e00640 1 -- 172.21.15.19:0/4146404015 wait complete. 2024-03-28T13:12:34.444 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:12:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.661+0000 7fdfdf200640 1 Processor -- start 2024-03-28T13:12:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.661+0000 7fdfdf200640 1 -- start start 2024-03-28T13:12:34.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.663+0000 7fdfdf200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdfe0147c80 0x7fdfe006b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:34.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.663+0000 7fdfdf200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe006bab0 0x7fdfe00702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.663+0000 7fdfdf200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdfe00707e0 0x7fdfe0161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.663+0000 7fdfdf200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdfe006bf10 con 0x7fdfe00707e0 2024-03-28T13:12:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.663+0000 7fdfdf200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fdfe0070c70 con 0x7fdfe0147c80 2024-03-28T13:12:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.663+0000 7fdfdf200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe00d0f20 con 0x7fdfe006bab0 2024-03-28T13:12:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.663+0000 7fdfde800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdfe00707e0 0x7fdfe0161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.663+0000 7fdfde800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdfe00707e0 0x7fdfe0161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46214/0 (socket says 172.21.15.19:46214) 2024-03-28T13:12:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.663+0000 7fdfde800640 1 -- 172.21.15.19:0/3840043223 learned_addr learned my addr 172.21.15.19:0/3840043223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.663+0000 7fdfdd400640 1 --2- 172.21.15.19:0/3840043223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe006bab0 0x7fdfe00702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.664+0000 7fdfdde00640 1 --2- 172.21.15.19:0/3840043223 >> v2:172.21.15.188:3300/0 conn(0x7fdfe0147c80 0x7fdfe006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.664+0000 7fdfde800640 1 -- 172.21.15.19:0/3840043223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe006bab0 msgr2=0x7fdfe00702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.664+0000 7fdfde800640 1 --2- 172.21.15.19:0/3840043223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe006bab0 0x7fdfe00702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.664+0000 7fdfde800640 1 -- 172.21.15.19:0/3840043223 >> v2:172.21.15.188:3300/0 conn(0x7fdfe0147c80 msgr2=0x7fdfe006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.664+0000 7fdfde800640 1 --2- 172.21.15.19:0/3840043223 >> v2:172.21.15.188:3300/0 conn(0x7fdfe0147c80 0x7fdfe006b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.664+0000 7fdfde800640 1 -- 172.21.15.19:0/3840043223 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe00cfe80 con 0x7fdfe00707e0 2024-03-28T13:12:34.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.664+0000 7fdfdde00640 1 --2- 172.21.15.19:0/3840043223 >> v2:172.21.15.188:3300/0 conn(0x7fdfe0147c80 0x7fdfe006b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:34.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.665+0000 7fdfde800640 1 --2- 172.21.15.19:0/3840043223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe00707e0 0x7fdfe0161f70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdfe006e130 tx=0x7fdfd802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be81156c1f498777 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:34.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.665+0000 7fdfdca00640 1 -- 172.21.15.19:0/3840043223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd802ca80 con 0x7fdfe00707e0 2024-03-28T13:12:34.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.665+0000 7fdfdca00640 1 -- 172.21.15.19:0/3840043223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfd802cc20 con 0x7fdfe00707e0 2024-03-28T13:12:34.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.665+0000 7fdfdca00640 1 -- 172.21.15.19:0/3840043223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd80343f0 con 0x7fdfe00707e0 2024-03-28T13:12:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.666+0000 7fdfdf200640 1 -- 172.21.15.19:0/3840043223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe00707e0 msgr2=0x7fdfe0161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.666+0000 7fdfdf200640 1 --2- 172.21.15.19:0/3840043223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe00707e0 0x7fdfe0161f70 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fdfe006e130 tx=0x7fdfd802bee0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.666+0000 7fdfdf200640 1 -- 172.21.15.19:0/3840043223 shutdown_connections 2024-03-28T13:12:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.666+0000 7fdfdf200640 1 --2- 172.21.15.19:0/3840043223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe00707e0 0x7fdfe0161f70 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.666+0000 7fdfdf200640 1 --2- 172.21.15.19:0/3840043223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe006bab0 0x7fdfe00702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.666+0000 7fdfdf200640 1 --2- 172.21.15.19:0/3840043223 >> v2:172.21.15.188:3300/0 conn(0x7fdfe0147c80 0x7fdfe006b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.666+0000 7fdfdf200640 1 -- 172.21.15.19:0/3840043223 >> 172.21.15.19:0/3840043223 conn(0x7fdfe006e730 msgr2=0x7fdfe006eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.666+0000 7fdfdf200640 1 -- 172.21.15.19:0/3840043223 shutdown_connections 2024-03-28T13:12:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.666+0000 7fdfdf200640 1 -- 172.21.15.19:0/3840043223 wait complete. 2024-03-28T13:12:34.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.668+0000 7fdfdf200640 1 Processor -- start 2024-03-28T13:12:34.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.668+0000 7fdfdf200640 1 -- start start 2024-03-28T13:12:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.669+0000 7fdfdf200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe006bab0 0x7fdfe00b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.669+0000 7fdfdf200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe00707e0 0x7fdfe01f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.669+0000 7fdfdf200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe0147c80 0x7fdfe015d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.669+0000 7fdfdf200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe00d0440 con 0x7fdfe006bab0 2024-03-28T13:12:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.669+0000 7fdfdf200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe00d0a90 con 0x7fdfe0147c80 2024-03-28T13:12:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.669+0000 7fdfdf200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe00d0720 con 0x7fdfe00707e0 2024-03-28T13:12:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.669+0000 7fdfdde00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe006bab0 0x7fdfe00b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.669+0000 7fdfdde00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe006bab0 0x7fdfe00b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46218/0 (socket says 172.21.15.19:46218) 2024-03-28T13:12:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.669+0000 7fdfdde00640 1 -- 172.21.15.19:0/3057348848 learned_addr learned my addr 172.21.15.19:0/3057348848 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.669+0000 7fdfdd400640 1 --2- 172.21.15.19:0/3057348848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe00707e0 0x7fdfe01f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.669+0000 7fdfde800640 1 --2- 172.21.15.19:0/3057348848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe0147c80 0x7fdfe015d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.670+0000 7fdfdde00640 1 -- 172.21.15.19:0/3057348848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe00707e0 msgr2=0x7fdfe01f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.670+0000 7fdfdde00640 1 --2- 172.21.15.19:0/3057348848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe00707e0 0x7fdfe01f4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.670+0000 7fdfdde00640 1 -- 172.21.15.19:0/3057348848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe0147c80 msgr2=0x7fdfe015d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:34.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.670+0000 7fdfdde00640 1 --2- 172.21.15.19:0/3057348848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe0147c80 0x7fdfe015d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.670+0000 7fdfdde00640 1 -- 172.21.15.19:0/3057348848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe01f54e0 con 0x7fdfe006bab0 2024-03-28T13:12:34.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.670+0000 7fdfde800640 1 --2- 172.21.15.19:0/3057348848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe0147c80 0x7fdfe015d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:34.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.670+0000 7fdfdde00640 1 --2- 172.21.15.19:0/3057348848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe006bab0 0x7fdfe00b4180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000c7f0 tx=0x7fdfd000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:34.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.670+0000 7fdfd6000640 1 -- 172.21.15.19:0/3057348848 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0015070 con 0x7fdfe006bab0 2024-03-28T13:12:34.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.670+0000 7fdfd6000640 1 -- 172.21.15.19:0/3057348848 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfd0007a30 con 0x7fdfe006bab0 2024-03-28T13:12:34.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.670+0000 7fdfd6000640 1 -- 172.21.15.19:0/3057348848 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0007e00 con 0x7fdfe006bab0 2024-03-28T13:12:34.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.670+0000 7fdfdf200640 1 -- 172.21.15.19:0/3057348848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe00b4820 con 0x7fdfe006bab0 2024-03-28T13:12:34.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.671+0000 7fdfdf200640 1 -- 172.21.15.19:0/3057348848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe00b4de0 con 0x7fdfe006bab0 2024-03-28T13:12:34.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.672+0000 7fdfdf200640 1 -- 172.21.15.19:0/3057348848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe0075940 con 0x7fdfe006bab0 2024-03-28T13:12:34.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.674+0000 7fdfd6000640 1 -- 172.21.15.19:0/3057348848 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdfd0018070 con 0x7fdfe006bab0 2024-03-28T13:12:34.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.675+0000 7fdfd6000640 1 --2- 172.21.15.19:0/3057348848 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdfc006c530 0x7fdfc006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.675+0000 7fdfd6000640 1 -- 172.21.15.19:0/3057348848 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7fdfd008b6e0 con 0x7fdfe006bab0 2024-03-28T13:12:34.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.675+0000 7fdfdd400640 1 --2- 172.21.15.19:0/3057348848 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdfc006c530 0x7fdfc006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:34.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.676+0000 7fdfdd400640 1 --2- 172.21.15.19:0/3057348848 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdfc006c530 0x7fdfc006e9f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc006fd0 tx=0x7fdfcc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:34.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.679+0000 7fdfd6000640 1 -- 172.21.15.19:0/3057348848 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdfd0059c70 con 0x7fdfe006bab0 2024-03-28T13:12:34.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.882+0000 7fdfdf200640 1 -- 172.21.15.19:0/3057348848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fdfe0158ac0 con 0x7fdfe006bab0 2024-03-28T13:12:34.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.883+0000 7fdfd6000640 1 -- 172.21.15.19:0/3057348848 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) ==== 100+0+11 (secure 0 0 0) 0x7fdfd0059570 con 0x7fdfe006bab0 2024-03-28T13:12:34.883 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:12:34.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.885+0000 7fdfdf200640 1 -- 172.21.15.19:0/3057348848 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdfc006c530 msgr2=0x7fdfc006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:34.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.886+0000 7fdfdf200640 1 --2- 172.21.15.19:0/3057348848 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdfc006c530 0x7fdfc006e9f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc006fd0 tx=0x7fdfcc008040 comp rx=0 tx=0).stop 2024-03-28T13:12:34.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.886+0000 7fdfdf200640 1 -- 172.21.15.19:0/3057348848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe006bab0 msgr2=0x7fdfe00b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:34.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.886+0000 7fdfdf200640 1 --2- 172.21.15.19:0/3057348848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe006bab0 0x7fdfe00b4180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fdfd000c7f0 tx=0x7fdfd000cbf0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.886+0000 7fdfdf200640 1 -- 172.21.15.19:0/3057348848 shutdown_connections 2024-03-28T13:12:34.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.886+0000 7fdfdf200640 1 --2- 172.21.15.19:0/3057348848 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdfc006c530 0x7fdfc006e9f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.886+0000 7fdfdf200640 1 --2- 172.21.15.19:0/3057348848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe0147c80 0x7fdfe015d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.886+0000 7fdfdf200640 1 --2- 172.21.15.19:0/3057348848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe00707e0 0x7fdfe01f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.886+0000 7fdfdf200640 1 --2- 172.21.15.19:0/3057348848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe006bab0 0x7fdfe00b4180 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:34.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.886+0000 7fdfdf200640 1 -- 172.21.15.19:0/3057348848 >> 172.21.15.19:0/3057348848 conn(0x7fdfe006e730 msgr2=0x7fdfe0146ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:34.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.886+0000 7fdfdf200640 1 -- 172.21.15.19:0/3057348848 shutdown_connections 2024-03-28T13:12:34.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:34.886+0000 7fdfdf200640 1 -- 172.21.15.19:0/3057348848 wait complete. 2024-03-28T13:12:34.905 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:12:35.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.108+0000 7f1302800640 1 Processor -- start 2024-03-28T13:12:35.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.109+0000 7f1302800640 1 -- start start 2024-03-28T13:12:35.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.110+0000 7f1302800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12fc0ce040 0x7f12fc06e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:35.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.110+0000 7f1302800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12fc0735a0 con 0x7f12fc1525f0 2024-03-28T13:12:35.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.110+0000 7f1302800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12fc0727e0 con 0x7f12fc15c460 2024-03-28T13:12:35.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.110+0000 7f1302800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f12fc073110 con 0x7f12fc0ce040 2024-03-28T13:12:35.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.111+0000 7f12faa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f12fc0ce040 0x7f12fc06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:35.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.111+0000 7f12fb400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f12fc1525f0 0x7f12fc0cd930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45072/0 (socket says 172.21.15.19:45072) 2024-03-28T13:12:35.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.111+0000 7f12fb400640 1 -- 172.21.15.19:0/3715115505 learned_addr learned my addr 172.21.15.19:0/3715115505 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:35.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.111+0000 7f12faa00640 1 -- 172.21.15.19:0/3715115505 >> v1:172.21.15.188:6789/0 conn(0x7f12fc15c460 legacy=0x7f12fc165390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:35.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.111+0000 7f12faa00640 1 -- 172.21.15.19:0/3715115505 >> v1:172.21.15.19:6789/0 conn(0x7f12fc1525f0 legacy=0x7f12fc0cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:35.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.111+0000 7f12fa000640 1 -- 172.21.15.19:0/3715115505 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3315596755 0 0) 0x7f12fc0735a0 con 0x7f12fc1525f0 2024-03-28T13:12:35.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.111+0000 7f12faa00640 1 -- 172.21.15.19:0/3715115505 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12fc072ac0 con 0x7f12fc0ce040 2024-03-28T13:12:35.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.113+0000 7f12faa00640 1 --2- 172.21.15.19:0/3715115505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12fc0ce040 0x7f12fc06e710 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f12e80097a0 tx=0x7f12e802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce5eb48a1dccdcf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:35.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.113+0000 7f12fa000640 1 -- 172.21.15.19:0/3715115505 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8038070 con 0x7f12fc0ce040 2024-03-28T13:12:35.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.113+0000 7f12fa000640 1 -- 172.21.15.19:0/3715115505 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12e8031700 con 0x7f12fc0ce040 2024-03-28T13:12:35.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.113+0000 7f12fa000640 1 -- 172.21.15.19:0/3715115505 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8031ad0 con 0x7f12fc0ce040 2024-03-28T13:12:35.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.114+0000 7f1302800640 1 -- 172.21.15.19:0/3715115505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12fc0ce040 msgr2=0x7f12fc06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:35.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.114+0000 7f1302800640 1 --2- 172.21.15.19:0/3715115505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12fc0ce040 0x7f12fc06e710 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f12e80097a0 tx=0x7f12e802b7a0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.114+0000 7f1302800640 1 -- 172.21.15.19:0/3715115505 shutdown_connections 2024-03-28T13:12:35.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.114+0000 7f1302800640 1 --2- 172.21.15.19:0/3715115505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12fc0ce040 0x7f12fc06e710 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.114+0000 7f1302800640 1 -- 172.21.15.19:0/3715115505 >> 172.21.15.19:0/3715115505 conn(0x7f12fc0b2cb0 msgr2=0x7f12fc0b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:35.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.114+0000 7f1302800640 1 -- 172.21.15.19:0/3715115505 shutdown_connections 2024-03-28T13:12:35.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.114+0000 7f1302800640 1 -- 172.21.15.19:0/3715115505 wait complete. 2024-03-28T13:12:35.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.116+0000 7f1302800640 1 Processor -- start 2024-03-28T13:12:35.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.116+0000 7f1302800640 1 -- start start 2024-03-28T13:12:35.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.117+0000 7f1302800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12fc0ce040 0x7f12fc144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:35.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.117+0000 7f1302800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12fc1525f0 0x7f12fc0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:35.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.117+0000 7f1302800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12fc15c460 0x7f12fc14f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:35.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.117+0000 7f1302800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc15c8b0 con 0x7f12fc0ce040 2024-03-28T13:12:35.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.117+0000 7f1302800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc06eda0 con 0x7f12fc15c460 2024-03-28T13:12:35.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.117+0000 7f1302800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f12fc166ab0 con 0x7f12fc1525f0 2024-03-28T13:12:35.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.117+0000 7f12fb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12fc0ce040 0x7f12fc144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:35.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.117+0000 7f12fb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12fc0ce040 0x7f12fc144db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46242/0 (socket says 172.21.15.19:46242) 2024-03-28T13:12:35.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.117+0000 7f12fb400640 1 -- 172.21.15.19:0/3893836136 learned_addr learned my addr 172.21.15.19:0/3893836136 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:35.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.118+0000 7f12faa00640 1 --2- 172.21.15.19:0/3893836136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12fc1525f0 0x7f12fc0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:35.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.118+0000 7f12fb400640 1 -- 172.21.15.19:0/3893836136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12fc1525f0 msgr2=0x7f12fc0aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:35.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.118+0000 7f12fbe00640 1 --2- 172.21.15.19:0/3893836136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12fc15c460 0x7f12fc14f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.118+0000 7f12fb400640 1 --2- 172.21.15.19:0/3893836136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12fc1525f0 0x7f12fc0aea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.118+0000 7f12fb400640 1 -- 172.21.15.19:0/3893836136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12fc15c460 msgr2=0x7f12fc14f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.118+0000 7f12fb400640 1 --2- 172.21.15.19:0/3893836136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12fc15c460 0x7f12fc14f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.118+0000 7f12fb400640 1 -- 172.21.15.19:0/3893836136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12fc148d80 con 0x7f12fc0ce040 2024-03-28T13:12:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.118+0000 7f12fbe00640 1 --2- 172.21.15.19:0/3893836136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12fc15c460 0x7f12fc14f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.118+0000 7f12fb400640 1 --2- 172.21.15.19:0/3893836136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12fc0ce040 0x7f12fc144db0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00c4d0 tx=0x7f12ec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.119+0000 7f12f3400640 1 -- 172.21.15.19:0/3893836136 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec018070 con 0x7f12fc0ce040 2024-03-28T13:12:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.119+0000 7f12f3400640 1 -- 172.21.15.19:0/3893836136 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12ec007500 con 0x7f12fc0ce040 2024-03-28T13:12:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.119+0000 7f12f3400640 1 -- 172.21.15.19:0/3893836136 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec004a20 con 0x7f12fc0ce040 2024-03-28T13:12:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.119+0000 7f1302800640 1 -- 172.21.15.19:0/3893836136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12fc149060 con 0x7f12fc0ce040 2024-03-28T13:12:35.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.119+0000 7f1302800640 1 -- 172.21.15.19:0/3893836136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12fc149620 con 0x7f12fc0ce040 2024-03-28T13:12:35.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.120+0000 7f1302800640 1 -- 172.21.15.19:0/3893836136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12fc0685c0 con 0x7f12fc0ce040 2024-03-28T13:12:35.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.121+0000 7f12f3400640 1 -- 172.21.15.19:0/3893836136 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f12ec004bc0 con 0x7f12fc0ce040 2024-03-28T13:12:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.122+0000 7f12f3400640 1 --2- 172.21.15.19:0/3893836136 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f12dc06c530 0x7f12dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.122+0000 7f12f3400640 1 -- 172.21.15.19:0/3893836136 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7f12ec08ab30 con 0x7f12fc0ce040 2024-03-28T13:12:35.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.123+0000 7f12faa00640 1 --2- 172.21.15.19:0/3893836136 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f12dc06c530 0x7f12dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:35.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.124+0000 7f12faa00640 1 --2- 172.21.15.19:0/3893836136 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f12dc06c530 0x7f12dc06e9f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f12e8035000 tx=0x7f12e8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:35.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.127+0000 7f12f3400640 1 -- 172.21.15.19:0/3893836136 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f12ec059020 con 0x7f12fc0ce040 2024-03-28T13:12:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.375+0000 7f1302800640 1 -- 172.21.15.19:0/3893836136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12fc0b3ac0 con 0x7f12fc0ce040 2024-03-28T13:12:35.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.377+0000 7f12f3400640 1 -- 172.21.15.19:0/3893836136 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7f12ec058920 con 0x7f12fc0ce040 2024-03-28T13:12:35.377 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:35.377 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2125648249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2125648249},{"type":"v1","addr":"172.21.15.188:6836","nonce":2125648249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/429866161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":429866161},{"type":"v1","addr":"172.21.15.188:6837","nonce":429866161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1422209289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1422209289},{"type":"v1","addr":"172.21.15.19:6834","nonce":1422209289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:11:46.379929+0000","modified":"2024-03-28T13:11:49.470791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5538},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5538":{"gid":5538,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/179338886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":179338886},{"type":"v1","addr":"172.21.15.19:6835","nonce":179338886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5538,"qdb_cluster":[5538]},"id":3}]} 2024-03-28T13:12:35.377 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-03-28T13:12:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.379+0000 7f1302800640 1 -- 172.21.15.19:0/3893836136 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f12dc06c530 msgr2=0x7f12dc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.379+0000 7f1302800640 1 --2- 172.21.15.19:0/3893836136 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f12dc06c530 0x7f12dc06e9f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f12e8035000 tx=0x7f12e8008030 comp rx=0 tx=0).stop 2024-03-28T13:12:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.379+0000 7f1302800640 1 -- 172.21.15.19:0/3893836136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12fc0ce040 msgr2=0x7f12fc144db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.379+0000 7f1302800640 1 --2- 172.21.15.19:0/3893836136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12fc0ce040 0x7f12fc144db0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00c4d0 tx=0x7f12ec00c990 comp rx=0 tx=0).stop 2024-03-28T13:12:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.380+0000 7f1302800640 1 -- 172.21.15.19:0/3893836136 shutdown_connections 2024-03-28T13:12:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.380+0000 7f1302800640 1 --2- 172.21.15.19:0/3893836136 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f12dc06c530 0x7f12dc06e9f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.380+0000 7f1302800640 1 --2- 172.21.15.19:0/3893836136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12fc15c460 0x7f12fc14f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.380+0000 7f1302800640 1 --2- 172.21.15.19:0/3893836136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f12fc1525f0 0x7f12fc0aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.380+0000 7f1302800640 1 --2- 172.21.15.19:0/3893836136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12fc0ce040 0x7f12fc144db0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.380+0000 7f1302800640 1 -- 172.21.15.19:0/3893836136 >> 172.21.15.19:0/3893836136 conn(0x7f12fc0b2cb0 msgr2=0x7f12fc14b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:35.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.380+0000 7f1302800640 1 -- 172.21.15.19:0/3893836136 shutdown_connections 2024-03-28T13:12:35.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.380+0000 7f1302800640 1 -- 172.21.15.19:0/3893836136 wait complete. 2024-03-28T13:12:35.398 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:12:35.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.612+0000 7faeab200640 1 Processor -- start 2024-03-28T13:12:35.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.612+0000 7faeab200640 1 -- start start 2024-03-28T13:12:35.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.615+0000 7faeab200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faeac0d2310 0x7faeac0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:35.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.615+0000 7faeab200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faeac0cf530 0x7faeac0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:35.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.615+0000 7faeab200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeac15e880 con 0x7faeac15e4a0 2024-03-28T13:12:35.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.615+0000 7faeab200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7faeac0cf9a0 con 0x7faeac0cf530 2024-03-28T13:12:35.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.615+0000 7faeab200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7faeac0d34e0 con 0x7faeac0d2310 2024-03-28T13:12:35.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.616+0000 7faeaa800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faeac0cf530 0x7faeac0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:35.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.616+0000 7faeaa800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7faeac0cf530 0x7faeac0b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42606/0 (socket says 172.21.15.19:42606) 2024-03-28T13:12:35.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.616+0000 7faeaa800640 1 -- 172.21.15.19:0/2571161669 learned_addr learned my addr 172.21.15.19:0/2571161669 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:35.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.616+0000 7faea9400640 1 --2- 172.21.15.19:0/2571161669 >> v2:172.21.15.188:3300/0 conn(0x7faeac0d2310 0x7faeac0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:35.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.616+0000 7faeaa800640 1 -- 172.21.15.19:0/2571161669 >> v2:172.21.15.188:3300/0 conn(0x7faeac0d2310 msgr2=0x7faeac0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:35.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.616+0000 7faeaa800640 1 --2- 172.21.15.19:0/2571161669 >> v2:172.21.15.188:3300/0 conn(0x7faeac0d2310 0x7faeac0ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.616+0000 7faeaa800640 1 -- 172.21.15.19:0/2571161669 >> v1:172.21.15.188:6789/0 conn(0x7faeac15e4a0 legacy=0x7faeac0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:35.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.616+0000 7faeaa800640 1 -- 172.21.15.19:0/2571161669 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac0d28d0 con 0x7faeac0cf530 2024-03-28T13:12:35.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.617+0000 7faeaa800640 1 --2- 172.21.15.19:0/2571161669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeac0cf530 0x7faeac0b73e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7faeac0d3660 tx=0x7faea402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c668a34ae5d711b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:35.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.618+0000 7faea8a00640 1 -- 172.21.15.19:0/2571161669 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea402c9f0 con 0x7faeac0cf530 2024-03-28T13:12:35.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.618+0000 7faea8a00640 1 -- 172.21.15.19:0/2571161669 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea402cb90 con 0x7faeac0cf530 2024-03-28T13:12:35.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.618+0000 7faea8a00640 1 -- 172.21.15.19:0/2571161669 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea40343f0 con 0x7faeac0cf530 2024-03-28T13:12:35.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.619+0000 7faeab200640 1 -- 172.21.15.19:0/2571161669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeac0cf530 msgr2=0x7faeac0b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:35.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.619+0000 7faeab200640 1 --2- 172.21.15.19:0/2571161669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeac0cf530 0x7faeac0b73e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7faeac0d3660 tx=0x7faea402be50 comp rx=0 tx=0).stop 2024-03-28T13:12:35.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.619+0000 7faeab200640 1 -- 172.21.15.19:0/2571161669 shutdown_connections 2024-03-28T13:12:35.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.619+0000 7faeab200640 1 --2- 172.21.15.19:0/2571161669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeac0cf530 0x7faeac0b73e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.619+0000 7faeab200640 1 --2- 172.21.15.19:0/2571161669 >> v2:172.21.15.188:3300/0 conn(0x7faeac0d2310 0x7faeac0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.619+0000 7faeab200640 1 -- 172.21.15.19:0/2571161669 >> 172.21.15.19:0/2571161669 conn(0x7faeac0676d0 msgr2=0x7faeac067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:35.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.619+0000 7faeab200640 1 -- 172.21.15.19:0/2571161669 shutdown_connections 2024-03-28T13:12:35.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.619+0000 7faeab200640 1 -- 172.21.15.19:0/2571161669 wait complete. 2024-03-28T13:12:35.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.623+0000 7faeab200640 1 Processor -- start 2024-03-28T13:12:35.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.623+0000 7faeab200640 1 -- start start 2024-03-28T13:12:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.625+0000 7faeab200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeac0cf530 0x7faeac1f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.625+0000 7faeab200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeac0d2310 0x7faeac0c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:35.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.625+0000 7faeab200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeac15e4a0 0x7faeac160780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.625+0000 7faeab200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faeac0d3170 con 0x7faeac0cf530 2024-03-28T13:12:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.625+0000 7faeab200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faeac0d2e90 con 0x7faeac0d2310 2024-03-28T13:12:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.625+0000 7faeab200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7faeac0d3970 con 0x7faeac15e4a0 2024-03-28T13:12:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.626+0000 7faea9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeac0cf530 0x7faeac1f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.626+0000 7faea9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeac0cf530 0x7faeac1f7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42614/0 (socket says 172.21.15.19:42614) 2024-03-28T13:12:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.626+0000 7faea9e00640 1 -- 172.21.15.19:0/3884913317 learned_addr learned my addr 172.21.15.19:0/3884913317 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.626+0000 7faea9400640 1 --2- 172.21.15.19:0/3884913317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeac0d2310 0x7faeac0c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.626+0000 7faeaa800640 1 --2- 172.21.15.19:0/3884913317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeac15e4a0 0x7faeac160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.626+0000 7faea9e00640 1 -- 172.21.15.19:0/3884913317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeac15e4a0 msgr2=0x7faeac160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.626+0000 7faea9e00640 1 --2- 172.21.15.19:0/3884913317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeac15e4a0 0x7faeac160780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.626+0000 7faea9e00640 1 -- 172.21.15.19:0/3884913317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeac0d2310 msgr2=0x7faeac0c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.626+0000 7faea9e00640 1 --2- 172.21.15.19:0/3884913317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeac0d2310 0x7faeac0c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.626+0000 7faea9e00640 1 -- 172.21.15.19:0/3884913317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac1613d0 con 0x7faeac0cf530 2024-03-28T13:12:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.626+0000 7faeaa800640 1 --2- 172.21.15.19:0/3884913317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeac15e4a0 0x7faeac160780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:35.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.627+0000 7faea9e00640 1 --2- 172.21.15.19:0/3884913317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeac0cf530 0x7faeac1f7f90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00b3f0 tx=0x7fae9c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:35.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.627+0000 7faea2000640 1 -- 172.21.15.19:0/3884913317 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c015020 con 0x7faeac0cf530 2024-03-28T13:12:35.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.627+0000 7faea2000640 1 -- 172.21.15.19:0/3884913317 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae9c011830 con 0x7faeac0cf530 2024-03-28T13:12:35.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.627+0000 7faea2000640 1 -- 172.21.15.19:0/3884913317 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9c011c20 con 0x7faeac0cf530 2024-03-28T13:12:35.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.627+0000 7faeab200640 1 -- 172.21.15.19:0/3884913317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeac1f74b0 con 0x7faeac0cf530 2024-03-28T13:12:35.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.627+0000 7faeab200640 1 -- 172.21.15.19:0/3884913317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeac1f7a70 con 0x7faeac0cf530 2024-03-28T13:12:35.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.629+0000 7faeab200640 1 -- 172.21.15.19:0/3884913317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeac1f7c70 con 0x7faeac0cf530 2024-03-28T13:12:35.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.629+0000 7faea2000640 1 -- 172.21.15.19:0/3884913317 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fae9c018020 con 0x7faeac0cf530 2024-03-28T13:12:35.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.630+0000 7faea2000640 1 --2- 172.21.15.19:0/3884913317 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fae8c06c530 0x7fae8c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:35.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.631+0000 7faea2000640 1 -- 172.21.15.19:0/3884913317 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7fae9c08b430 con 0x7faeac0cf530 2024-03-28T13:12:35.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.631+0000 7faea9400640 1 --2- 172.21.15.19:0/3884913317 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fae8c06c530 0x7fae8c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:35.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.632+0000 7faea9400640 1 --2- 172.21.15.19:0/3884913317 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fae8c06c530 0x7fae8c06e9f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faeac153190 tx=0x7fae98008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:35.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.640+0000 7faea2000640 1 -- 172.21.15.19:0/3884913317 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7faeac1f7c70 con 0x7faeac0cf530 2024-03-28T13:12:35.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.843+0000 7faeab200640 1 -- 172.21.15.19:0/3884913317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faeac075940 con 0x7faeac0cf530 2024-03-28T13:12:35.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.846+0000 7faea2000640 1 -- 172.21.15.19:0/3884913317 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16900 (secure 0 0 0) 0x7fae9c01f030 con 0x7faeac0cf530 2024-03-28T13:12:35.846 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:35.846 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":51,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:12:24.186519+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:11:44.316258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-28T13:11:45.360154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.0","mappings":[{"from":5,"to":0}]},{"pgid":"7.6","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.39","mappings":[{"from":5,"to":7}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:12:35.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.848+0000 7faeab200640 1 -- 172.21.15.19:0/3884913317 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fae8c06c530 msgr2=0x7fae8c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:35.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.848+0000 7faeab200640 1 --2- 172.21.15.19:0/3884913317 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fae8c06c530 0x7fae8c06e9f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faeac153190 tx=0x7fae98008040 comp rx=0 tx=0).stop 2024-03-28T13:12:35.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.848+0000 7faeab200640 1 -- 172.21.15.19:0/3884913317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeac0cf530 msgr2=0x7faeac1f7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:35.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.848+0000 7faeab200640 1 --2- 172.21.15.19:0/3884913317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeac0cf530 0x7faeac1f7f90 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fae9c00b3f0 tx=0x7fae9c00b8b0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.849+0000 7faeab200640 1 -- 172.21.15.19:0/3884913317 shutdown_connections 2024-03-28T13:12:35.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.849+0000 7faeab200640 1 --2- 172.21.15.19:0/3884913317 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fae8c06c530 0x7fae8c06e9f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.849+0000 7faeab200640 1 --2- 172.21.15.19:0/3884913317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faeac15e4a0 0x7faeac160780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.849+0000 7faeab200640 1 --2- 172.21.15.19:0/3884913317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faeac0d2310 0x7faeac0c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.849+0000 7faeab200640 1 --2- 172.21.15.19:0/3884913317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faeac0cf530 0x7faeac1f7f90 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:35.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.849+0000 7faeab200640 1 -- 172.21.15.19:0/3884913317 >> 172.21.15.19:0/3884913317 conn(0x7faeac0676d0 msgr2=0x7faeac15cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:35.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.849+0000 7faeab200640 1 -- 172.21.15.19:0/3884913317 shutdown_connections 2024-03-28T13:12:35.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:35.849+0000 7faeab200640 1 -- 172.21.15.19:0/3884913317 wait complete. 2024-03-28T13:12:35.877 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-28T13:12:35.877 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:12:36.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.087+0000 7f1fef200640 1 Processor -- start 2024-03-28T13:12:36.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.087+0000 7f1fef200640 1 -- start start 2024-03-28T13:12:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.089+0000 7f1fef200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ff015e4a0 0x7f1ff00b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.089+0000 7f1fef200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff00cf700 0x7f1ff00b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.089+0000 7f1fef200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ff015e880 con 0x7f1ff00d2310 2024-03-28T13:12:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.089+0000 7f1fef200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1ff00d34e0 con 0x7f1ff015e4a0 2024-03-28T13:12:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.089+0000 7f1fef200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff00d28d0 con 0x7f1ff00cf700 2024-03-28T13:12:36.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.090+0000 7f1fee800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff00cf700 0x7f1ff00b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:36.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.090+0000 7f1fee800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff00cf700 0x7f1ff00b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:41460/0 (socket says 172.21.15.19:41460) 2024-03-28T13:12:36.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.090+0000 7f1fee800640 1 -- 172.21.15.19:0/769968670 learned_addr learned my addr 172.21.15.19:0/769968670 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:36.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.090+0000 7f1fede00640 1 --2- 172.21.15.19:0/769968670 >> v2:172.21.15.188:3300/0 conn(0x7f1ff015e4a0 0x7f1ff00b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:36.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.091+0000 7f1fed400640 1 -- 172.21.15.19:0/769968670 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2295030260 0 0) 0x7f1ff015e880 con 0x7f1ff00d2310 2024-03-28T13:12:36.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.091+0000 7f1fed400640 1 -- 172.21.15.19:0/769968670 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fd4003520 con 0x7f1ff00d2310 2024-03-28T13:12:36.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.091+0000 7f1fede00640 1 -- 172.21.15.19:0/769968670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff00cf700 msgr2=0x7f1ff00b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:36.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.091+0000 7f1fede00640 1 --2- 172.21.15.19:0/769968670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff00cf700 0x7f1ff00b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.091+0000 7f1fede00640 1 -- 172.21.15.19:0/769968670 >> v1:172.21.15.188:6789/0 conn(0x7f1ff00d2310 legacy=0x7f1ff00ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:36.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.091+0000 7f1fede00640 1 -- 172.21.15.19:0/769968670 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff00d2bb0 con 0x7f1ff015e4a0 2024-03-28T13:12:36.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.091+0000 7f1fee800640 1 --2- 172.21.15.19:0/769968670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff00cf700 0x7f1ff00b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:12:36.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.092+0000 7f1fede00640 1 --2- 172.21.15.19:0/769968670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff015e4a0 0x7f1ff00b9c60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00d3660 tx=0x7f1fe002be00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a775460eeac2a64c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:36.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.093+0000 7f1fed400640 1 -- 172.21.15.19:0/769968670 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0039070 con 0x7f1ff015e4a0 2024-03-28T13:12:36.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.093+0000 7f1fed400640 1 -- 172.21.15.19:0/769968670 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe002cbb0 con 0x7f1ff015e4a0 2024-03-28T13:12:36.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.093+0000 7f1fed400640 1 -- 172.21.15.19:0/769968670 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0034440 con 0x7f1ff015e4a0 2024-03-28T13:12:36.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.093+0000 7f1fef200640 1 -- 172.21.15.19:0/769968670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff015e4a0 msgr2=0x7f1ff00b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:36.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.093+0000 7f1fef200640 1 --2- 172.21.15.19:0/769968670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff015e4a0 0x7f1ff00b9c60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00d3660 tx=0x7f1fe002be00 comp rx=0 tx=0).stop 2024-03-28T13:12:36.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.093+0000 7f1fef200640 1 -- 172.21.15.19:0/769968670 shutdown_connections 2024-03-28T13:12:36.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.093+0000 7f1fef200640 1 --2- 172.21.15.19:0/769968670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff00cf700 0x7f1ff00b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.093+0000 7f1fef200640 1 --2- 172.21.15.19:0/769968670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff015e4a0 0x7f1ff00b9c60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.093+0000 7f1fef200640 1 -- 172.21.15.19:0/769968670 >> 172.21.15.19:0/769968670 conn(0x7f1ff00676d0 msgr2=0x7f1ff0067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:36.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.094+0000 7f1fef200640 1 -- 172.21.15.19:0/769968670 shutdown_connections 2024-03-28T13:12:36.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.094+0000 7f1fef200640 1 -- 172.21.15.19:0/769968670 wait complete. 2024-03-28T13:12:36.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.095+0000 7f1fef200640 1 Processor -- start 2024-03-28T13:12:36.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.095+0000 7f1fef200640 1 -- start start 2024-03-28T13:12:36.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.096+0000 7f1fef200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff00cf700 0x7f1ff0143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:36.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.096+0000 7f1fef200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff00d2310 0x7f1ff015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:36.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.096+0000 7f1fef200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ff015e4a0 0x7f1ff015efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:36.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.096+0000 7f1fef200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff00d3970 con 0x7f1ff015e4a0 2024-03-28T13:12:36.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.096+0000 7f1fef200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff00d3170 con 0x7f1ff00cf700 2024-03-28T13:12:36.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.096+0000 7f1fef200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff00ba1a0 con 0x7f1ff00d2310 2024-03-28T13:12:36.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.097+0000 7f1fee800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ff015e4a0 0x7f1ff015efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:36.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.097+0000 7f1fee800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ff015e4a0 0x7f1ff015efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42628/0 (socket says 172.21.15.19:42628) 2024-03-28T13:12:36.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.097+0000 7f1fee800640 1 -- 172.21.15.19:0/1425440965 learned_addr learned my addr 172.21.15.19:0/1425440965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:36.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.097+0000 7f1fede00640 1 --2- 172.21.15.19:0/1425440965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff00cf700 0x7f1ff0143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:36.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.097+0000 7f1fe5400640 1 --2- 172.21.15.19:0/1425440965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff00d2310 0x7f1ff015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:36.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.097+0000 7f1fee800640 1 -- 172.21.15.19:0/1425440965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff00d2310 msgr2=0x7f1ff015ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:36.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.097+0000 7f1fee800640 1 --2- 172.21.15.19:0/1425440965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff00d2310 0x7f1ff015ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.097+0000 7f1fee800640 1 -- 172.21.15.19:0/1425440965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff00cf700 msgr2=0x7f1ff0143070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:36.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.097+0000 7f1fee800640 1 --2- 172.21.15.19:0/1425440965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff00cf700 0x7f1ff0143070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.097+0000 7f1fee800640 1 -- 172.21.15.19:0/1425440965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff01f3e50 con 0x7f1ff015e4a0 2024-03-28T13:12:36.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.098+0000 7f1fee800640 1 --2- 172.21.15.19:0/1425440965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ff015e4a0 0x7f1ff015efc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8006e00 tx=0x7f1fe800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:36.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.098+0000 7f1fe6a00640 1 -- 172.21.15.19:0/1425440965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8004030 con 0x7f1ff015e4a0 2024-03-28T13:12:36.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.098+0000 7f1fe6a00640 1 -- 172.21.15.19:0/1425440965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fe80041d0 con 0x7f1ff015e4a0 2024-03-28T13:12:36.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.098+0000 7f1fe6a00640 1 -- 172.21.15.19:0/1425440965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe800c910 con 0x7f1ff015e4a0 2024-03-28T13:12:36.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.098+0000 7f1fef200640 1 -- 172.21.15.19:0/1425440965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff00b8490 con 0x7f1ff015e4a0 2024-03-28T13:12:36.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.098+0000 7f1fef200640 1 -- 172.21.15.19:0/1425440965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff00b8a50 con 0x7f1ff015e4a0 2024-03-28T13:12:36.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.099+0000 7f1fef200640 1 -- 172.21.15.19:0/1425440965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff0075940 con 0x7f1ff015e4a0 2024-03-28T13:12:36.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.101+0000 7f1fe6a00640 1 -- 172.21.15.19:0/1425440965 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1fe800cab0 con 0x7f1ff015e4a0 2024-03-28T13:12:36.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.102+0000 7f1fe6a00640 1 --2- 172.21.15.19:0/1425440965 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1fcc06c560 0x7f1fcc06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:36.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.102+0000 7f1fe6a00640 1 -- 172.21.15.19:0/1425440965 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7f1fe808bb20 con 0x7f1ff015e4a0 2024-03-28T13:12:36.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.103+0000 7f1fede00640 1 --2- 172.21.15.19:0/1425440965 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1fcc06c560 0x7f1fcc06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:36.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.104+0000 7f1fede00640 1 --2- 172.21.15.19:0/1425440965 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1fcc06c560 0x7f1fcc06ea20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0036000 tx=0x7f1fe0030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:36.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.109+0000 7f1fe6a00640 1 -- 172.21.15.19:0/1425440965 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1fe8017020 con 0x7f1ff015e4a0 2024-03-28T13:12:36.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.353+0000 7f1fef200640 1 -- 172.21.15.19:0/1425440965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ff00b3eb0 con 0x7f1ff015e4a0 2024-03-28T13:12:36.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.354+0000 7f1fe6a00640 1 -- 172.21.15.19:0/1425440965 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7f1fe805a010 con 0x7f1ff015e4a0 2024-03-28T13:12:36.354 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:36.355 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2125648249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2125648249},{"type":"v1","addr":"172.21.15.188:6836","nonce":2125648249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/429866161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":429866161},{"type":"v1","addr":"172.21.15.188:6837","nonce":429866161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1422209289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1422209289},{"type":"v1","addr":"172.21.15.19:6834","nonce":1422209289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:11:46.379929+0000","modified":"2024-03-28T13:11:49.470791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5538},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5538":{"gid":5538,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/179338886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":179338886},{"type":"v1","addr":"172.21.15.19:6835","nonce":179338886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5538,"qdb_cluster":[5538]},"id":3}]} 2024-03-28T13:12:36.355 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-03-28T13:12:36.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.357+0000 7f1fef200640 1 -- 172.21.15.19:0/1425440965 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1fcc06c560 msgr2=0x7f1fcc06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:36.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.357+0000 7f1fef200640 1 --2- 172.21.15.19:0/1425440965 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1fcc06c560 0x7f1fcc06ea20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0036000 tx=0x7f1fe0030000 comp rx=0 tx=0).stop 2024-03-28T13:12:36.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.357+0000 7f1fef200640 1 -- 172.21.15.19:0/1425440965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ff015e4a0 msgr2=0x7f1ff015efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:36.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.357+0000 7f1fef200640 1 --2- 172.21.15.19:0/1425440965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ff015e4a0 0x7f1ff015efc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8006e00 tx=0x7f1fe800ed40 comp rx=0 tx=0).stop 2024-03-28T13:12:36.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.357+0000 7f1fef200640 1 -- 172.21.15.19:0/1425440965 shutdown_connections 2024-03-28T13:12:36.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.357+0000 7f1fef200640 1 --2- 172.21.15.19:0/1425440965 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1fcc06c560 0x7f1fcc06ea20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.357+0000 7f1fef200640 1 --2- 172.21.15.19:0/1425440965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ff015e4a0 0x7f1ff015efc0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.357+0000 7f1fef200640 1 --2- 172.21.15.19:0/1425440965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ff00d2310 0x7f1ff015ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.357+0000 7f1fef200640 1 --2- 172.21.15.19:0/1425440965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ff00cf700 0x7f1ff0143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.357+0000 7f1fef200640 1 -- 172.21.15.19:0/1425440965 >> 172.21.15.19:0/1425440965 conn(0x7f1ff00676d0 msgr2=0x7f1ff015bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:36.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.357+0000 7f1fef200640 1 -- 172.21.15.19:0/1425440965 shutdown_connections 2024-03-28T13:12:36.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.357+0000 7f1fef200640 1 -- 172.21.15.19:0/1425440965 wait complete. 2024-03-28T13:12:36.376 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:12:36.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.578+0000 7f912e000640 1 Processor -- start 2024-03-28T13:12:36.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.578+0000 7f912e000640 1 -- start start 2024-03-28T13:12:36.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.580+0000 7f912e000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f912815c390 0x7f9128073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:36.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.580+0000 7f912e000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f91280af830 0x7f9128160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:36.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.580+0000 7f912e000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9128160b80 0x7f91280d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:36.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.580+0000 7f912e000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f912815c770 con 0x7f9128160b80 2024-03-28T13:12:36.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.580+0000 7f912e000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f912815fe80 con 0x7f91280af830 2024-03-28T13:12:36.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.580+0000 7f912e000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f91280ad120 con 0x7f912815c390 2024-03-28T13:12:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.580+0000 7f9127e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9128160b80 0x7f91280d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.580+0000 7f9127e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9128160b80 0x7f91280d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42630/0 (socket says 172.21.15.19:42630) 2024-03-28T13:12:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.580+0000 7f9127e00640 1 -- 172.21.15.19:0/3681841883 learned_addr learned my addr 172.21.15.19:0/3681841883 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.580+0000 7f9126a00640 1 --2- 172.21.15.19:0/3681841883 >> v2:172.21.15.188:3300/0 conn(0x7f91280af830 0x7f9128160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.580+0000 7f9127400640 1 --2- 172.21.15.19:0/3681841883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f912815c390 0x7f9128073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.581+0000 7f9127e00640 1 -- 172.21.15.19:0/3681841883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f912815c390 msgr2=0x7f9128073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.581+0000 7f9127e00640 1 --2- 172.21.15.19:0/3681841883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f912815c390 0x7f9128073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.581+0000 7f9127e00640 1 -- 172.21.15.19:0/3681841883 >> v2:172.21.15.188:3300/0 conn(0x7f91280af830 msgr2=0x7f9128160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.581+0000 7f9127e00640 1 --2- 172.21.15.19:0/3681841883 >> v2:172.21.15.188:3300/0 conn(0x7f91280af830 0x7f9128160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.581+0000 7f9127e00640 1 -- 172.21.15.19:0/3681841883 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91280ad400 con 0x7f9128160b80 2024-03-28T13:12:36.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.582+0000 7f9127e00640 1 --2- 172.21.15.19:0/3681841883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9128160b80 0x7f91280d0650 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f91280d11c0 tx=0x7f912002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4376c6f2c824e565 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:36.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.583+0000 7f9126000640 1 -- 172.21.15.19:0/3681841883 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912002ca70 con 0x7f9128160b80 2024-03-28T13:12:36.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.583+0000 7f9126000640 1 -- 172.21.15.19:0/3681841883 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f912002cc10 con 0x7f9128160b80 2024-03-28T13:12:36.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.583+0000 7f9126000640 1 -- 172.21.15.19:0/3681841883 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91200343f0 con 0x7f9128160b80 2024-03-28T13:12:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.583+0000 7f912e000640 1 -- 172.21.15.19:0/3681841883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9128160b80 msgr2=0x7f91280d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.583+0000 7f912e000640 1 --2- 172.21.15.19:0/3681841883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9128160b80 0x7f91280d0650 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f91280d11c0 tx=0x7f912002bed0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.583+0000 7f912e000640 1 -- 172.21.15.19:0/3681841883 shutdown_connections 2024-03-28T13:12:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.583+0000 7f912e000640 1 --2- 172.21.15.19:0/3681841883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9128160b80 0x7f91280d0650 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.583+0000 7f912e000640 1 --2- 172.21.15.19:0/3681841883 >> v2:172.21.15.188:3300/0 conn(0x7f91280af830 0x7f9128160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.583+0000 7f912e000640 1 --2- 172.21.15.19:0/3681841883 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f912815c390 0x7f9128073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.583+0000 7f912e000640 1 -- 172.21.15.19:0/3681841883 >> 172.21.15.19:0/3681841883 conn(0x7f912806b990 msgr2=0x7f912806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:36.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.583+0000 7f912e000640 1 -- 172.21.15.19:0/3681841883 shutdown_connections 2024-03-28T13:12:36.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.583+0000 7f912e000640 1 -- 172.21.15.19:0/3681841883 wait complete. 2024-03-28T13:12:36.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.585+0000 7f912e000640 1 Processor -- start 2024-03-28T13:12:36.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.585+0000 7f912e000640 1 -- start start 2024-03-28T13:12:36.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.586+0000 7f912e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91280af830 0x7f91280b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:36.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.586+0000 7f912e000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f912815c390 0x7f912814a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:36.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.586+0000 7f912e000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9128160b80 0x7f912809abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:36.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.586+0000 7f912e000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91280cf980 con 0x7f91280af830 2024-03-28T13:12:36.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.586+0000 7f912e000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9128160120 con 0x7f912815c390 2024-03-28T13:12:36.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.586+0000 7f912e000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f91280d0b90 con 0x7f9128160b80 2024-03-28T13:12:36.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.587+0000 7f9127400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91280af830 0x7f91280b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:36.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.587+0000 7f9127400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91280af830 0x7f91280b1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42636/0 (socket says 172.21.15.19:42636) 2024-03-28T13:12:36.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.587+0000 7f9127400640 1 -- 172.21.15.19:0/3692232221 learned_addr learned my addr 172.21.15.19:0/3692232221 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:36.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.587+0000 7f9127e00640 1 --2- 172.21.15.19:0/3692232221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9128160b80 0x7f912809abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:36.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.587+0000 7f9127400640 1 -- 172.21.15.19:0/3692232221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9128160b80 msgr2=0x7f912809abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:36.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.587+0000 7f9127400640 1 --2- 172.21.15.19:0/3692232221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9128160b80 0x7f912809abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.587+0000 7f9127400640 1 -- 172.21.15.19:0/3692232221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f912815c390 msgr2=0x7f912814a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:36.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.587+0000 7f9126a00640 1 --2- 172.21.15.19:0/3692232221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f912815c390 0x7f912814a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:36.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.587+0000 7f9127400640 1 --2- 172.21.15.19:0/3692232221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f912815c390 0x7f912814a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.587+0000 7f9127400640 1 -- 172.21.15.19:0/3692232221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912809db30 con 0x7f91280af830 2024-03-28T13:12:36.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.587+0000 7f9127e00640 1 --2- 172.21.15.19:0/3692232221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9128160b80 0x7f912809abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:36.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.588+0000 7f9127400640 1 --2- 172.21.15.19:0/3692232221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91280af830 0x7f91280b1740 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f911800d800 tx=0x7f911800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:36.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.588+0000 7f911f400640 1 -- 172.21.15.19:0/3692232221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911800b6c0 con 0x7f91280af830 2024-03-28T13:12:36.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.588+0000 7f911f400640 1 -- 172.21.15.19:0/3692232221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9118019070 con 0x7f91280af830 2024-03-28T13:12:36.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.588+0000 7f911f400640 1 -- 172.21.15.19:0/3692232221 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911800ba80 con 0x7f91280af830 2024-03-28T13:12:36.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.588+0000 7f912e000640 1 -- 172.21.15.19:0/3692232221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912809de10 con 0x7f91280af830 2024-03-28T13:12:36.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.588+0000 7f912e000640 1 -- 172.21.15.19:0/3692232221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f912815dc90 con 0x7f91280af830 2024-03-28T13:12:36.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.590+0000 7f912e000640 1 -- 172.21.15.19:0/3692232221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91280685c0 con 0x7f91280af830 2024-03-28T13:12:36.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.590+0000 7f911f400640 1 -- 172.21.15.19:0/3692232221 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f911800bc20 con 0x7f91280af830 2024-03-28T13:12:36.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.591+0000 7f911f400640 1 --2- 172.21.15.19:0/3692232221 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f910806c530 0x7f910806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:36.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.591+0000 7f911f400640 1 -- 172.21.15.19:0/3692232221 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7f911808bd00 con 0x7f91280af830 2024-03-28T13:12:36.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.592+0000 7f9126a00640 1 --2- 172.21.15.19:0/3692232221 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f910806c530 0x7f910806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:36.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.593+0000 7f9126a00640 1 --2- 172.21.15.19:0/3692232221 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f910806c530 0x7f910806e9f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91281f7cf0 tx=0x7f9114008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:36.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.595+0000 7f911f400640 1 -- 172.21.15.19:0/3692232221 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f911805a160 con 0x7f91280af830 2024-03-28T13:12:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.867+0000 7f912e000640 1 -- 172.21.15.19:0/3692232221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f912806f1b0 con 0x7f91280af830 2024-03-28T13:12:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.868+0000 7f911f400640 1 -- 172.21.15.19:0/3692232221 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5181 (secure 0 0 0) 0x7f9118059a60 con 0x7f91280af830 2024-03-28T13:12:36.868 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:36.868 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2125648249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2125648249},{"type":"v1","addr":"172.21.15.188:6836","nonce":2125648249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/429866161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":429866161},{"type":"v1","addr":"172.21.15.188:6837","nonce":429866161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1422209289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1422209289},{"type":"v1","addr":"172.21.15.19:6834","nonce":1422209289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:11:46.379929+0000","modified":"2024-03-28T13:11:49.470791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5538},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5538":{"gid":5538,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/179338886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":179338886},{"type":"v1","addr":"172.21.15.19:6835","nonce":179338886}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5538,"qdb_cluster":[5538]},"id":3}]} 2024-03-28T13:12:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 26 2024-03-28T13:12:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.870+0000 7f912e000640 1 -- 172.21.15.19:0/3692232221 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f910806c530 msgr2=0x7f910806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.870+0000 7f912e000640 1 --2- 172.21.15.19:0/3692232221 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f910806c530 0x7f910806e9f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f91281f7cf0 tx=0x7f9114008040 comp rx=0 tx=0).stop 2024-03-28T13:12:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.870+0000 7f912e000640 1 -- 172.21.15.19:0/3692232221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91280af830 msgr2=0x7f91280b1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.870+0000 7f912e000640 1 --2- 172.21.15.19:0/3692232221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91280af830 0x7f91280b1740 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f911800d800 tx=0x7f911800dcc0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.870+0000 7f912e000640 1 -- 172.21.15.19:0/3692232221 shutdown_connections 2024-03-28T13:12:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.870+0000 7f912e000640 1 --2- 172.21.15.19:0/3692232221 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f910806c530 0x7f910806e9f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.870+0000 7f912e000640 1 --2- 172.21.15.19:0/3692232221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9128160b80 0x7f912809abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.870+0000 7f912e000640 1 --2- 172.21.15.19:0/3692232221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f912815c390 0x7f912814a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.870+0000 7f912e000640 1 --2- 172.21.15.19:0/3692232221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91280af830 0x7f91280b1740 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.870+0000 7f912e000640 1 -- 172.21.15.19:0/3692232221 >> 172.21.15.19:0/3692232221 conn(0x7f912806b990 msgr2=0x7f912815a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.871+0000 7f912e000640 1 -- 172.21.15.19:0/3692232221 shutdown_connections 2024-03-28T13:12:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:36.871+0000 7f912e000640 1 -- 172.21.15.19:0/3692232221 wait complete. 2024-03-28T13:12:36.890 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:12:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.132+0000 7f892fe00640 1 Processor -- start 2024-03-28T13:12:37.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.132+0000 7f892fe00640 1 -- start start 2024-03-28T13:12:37.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.134+0000 7f892fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89300cc0b0 0x7f893006dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:37.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.134+0000 7f892fe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f89300d04d0 0x7f893015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:37.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.134+0000 7f892fe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f893006e4f0 con 0x7f893015fc50 2024-03-28T13:12:37.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.134+0000 7f892fe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f893015c610 con 0x7f89300d04d0 2024-03-28T13:12:37.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.134+0000 7f892fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f89300b3150 con 0x7f89300cc0b0 2024-03-28T13:12:37.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.134+0000 7f892ea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89300cc0b0 0x7f893006dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:37.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.135+0000 7f892ea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89300cc0b0 0x7f893006dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:41486/0 (socket says 172.21.15.19:41486) 2024-03-28T13:12:37.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.135+0000 7f892e000640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f893015fc50 0x7f89300cfdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:46894/0 (socket says 172.21.15.19:46894) 2024-03-28T13:12:37.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.135+0000 7f892ea00640 1 -- 172.21.15.19:0/4005355492 learned_addr learned my addr 172.21.15.19:0/4005355492 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:37.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.135+0000 7f892f400640 1 --2- 172.21.15.19:0/4005355492 >> v2:172.21.15.188:3300/0 conn(0x7f89300d04d0 0x7f893015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:37.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.135+0000 7f892d600640 1 -- 172.21.15.19:0/4005355492 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1405155134 0 0) 0x7f893006e4f0 con 0x7f893015fc50 2024-03-28T13:12:37.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.135+0000 7f892ea00640 1 -- 172.21.15.19:0/4005355492 >> v2:172.21.15.188:3300/0 conn(0x7f89300d04d0 msgr2=0x7f893015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:37.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.135+0000 7f892ea00640 1 --2- 172.21.15.19:0/4005355492 >> v2:172.21.15.188:3300/0 conn(0x7f89300d04d0 0x7f893015ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.135+0000 7f892ea00640 1 -- 172.21.15.19:0/4005355492 >> v1:172.21.15.188:6789/0 conn(0x7f893015fc50 legacy=0x7f89300cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:37.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.135+0000 7f892ea00640 1 -- 172.21.15.19:0/4005355492 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89301600e0 con 0x7f89300cc0b0 2024-03-28T13:12:37.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.137+0000 7f892ea00640 1 --2- 172.21.15.19:0/4005355492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89300cc0b0 0x7f893006dfb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8920009870 tx=0x7f892002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ed777e88ca5fa2d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:37.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.137+0000 7f892d600640 1 -- 172.21.15.19:0/4005355492 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920034070 con 0x7f89300cc0b0 2024-03-28T13:12:37.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.137+0000 7f892d600640 1 -- 172.21.15.19:0/4005355492 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8920030840 con 0x7f89300cc0b0 2024-03-28T13:12:37.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.137+0000 7f892d600640 1 -- 172.21.15.19:0/4005355492 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920030c10 con 0x7f89300cc0b0 2024-03-28T13:12:37.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.138+0000 7f892fe00640 1 -- 172.21.15.19:0/4005355492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89300cc0b0 msgr2=0x7f893006dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:37.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.138+0000 7f892fe00640 1 --2- 172.21.15.19:0/4005355492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89300cc0b0 0x7f893006dfb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8920009870 tx=0x7f892002b980 comp rx=0 tx=0).stop 2024-03-28T13:12:37.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.138+0000 7f892fe00640 1 -- 172.21.15.19:0/4005355492 shutdown_connections 2024-03-28T13:12:37.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.138+0000 7f892fe00640 1 --2- 172.21.15.19:0/4005355492 >> v2:172.21.15.188:3300/0 conn(0x7f89300d04d0 0x7f893015ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.138+0000 7f892fe00640 1 --2- 172.21.15.19:0/4005355492 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89300cc0b0 0x7f893006dfb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.138+0000 7f892fe00640 1 -- 172.21.15.19:0/4005355492 >> 172.21.15.19:0/4005355492 conn(0x7f89300ccad0 msgr2=0x7f89300cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:37.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.138+0000 7f892fe00640 1 -- 172.21.15.19:0/4005355492 shutdown_connections 2024-03-28T13:12:37.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.138+0000 7f892fe00640 1 -- 172.21.15.19:0/4005355492 wait complete. 2024-03-28T13:12:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.140+0000 7f892fe00640 1 Processor -- start 2024-03-28T13:12:37.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.141+0000 7f892fe00640 1 -- start start 2024-03-28T13:12:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.142+0000 7f892fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89300cc0b0 0x7f89301555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.142+0000 7f892fe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89300d04d0 0x7f893013c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.142+0000 7f892fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f893013e180 0x7f893012d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.142+0000 7f892fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8930160980 con 0x7f89300cc0b0 2024-03-28T13:12:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.142+0000 7f892fe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f89301606a0 con 0x7f89300d04d0 2024-03-28T13:12:37.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.142+0000 7f892fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f89300cc490 con 0x7f893013e180 2024-03-28T13:12:37.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.142+0000 7f892ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89300cc0b0 0x7f89301555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:37.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.142+0000 7f892ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89300cc0b0 0x7f89301555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42640/0 (socket says 172.21.15.19:42640) 2024-03-28T13:12:37.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.142+0000 7f892ea00640 1 -- 172.21.15.19:0/4151715119 learned_addr learned my addr 172.21.15.19:0/4151715119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.143+0000 7f892e000640 1 --2- 172.21.15.19:0/4151715119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89300d04d0 0x7f893013c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.143+0000 7f892f400640 1 --2- 172.21.15.19:0/4151715119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f893013e180 0x7f893012d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.143+0000 7f892ea00640 1 -- 172.21.15.19:0/4151715119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f893013e180 msgr2=0x7f893012d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.143+0000 7f892ea00640 1 --2- 172.21.15.19:0/4151715119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f893013e180 0x7f893012d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.143+0000 7f892ea00640 1 -- 172.21.15.19:0/4151715119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89300d04d0 msgr2=0x7f893013c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.143+0000 7f892ea00640 1 --2- 172.21.15.19:0/4151715119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89300d04d0 0x7f893013c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.143+0000 7f892ea00640 1 -- 172.21.15.19:0/4151715119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893013cb00 con 0x7f89300cc0b0 2024-03-28T13:12:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.143+0000 7f892e000640 1 --2- 172.21.15.19:0/4151715119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89300d04d0 0x7f893013c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.143+0000 7f892ea00640 1 --2- 172.21.15.19:0/4151715119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89300cc0b0 0x7f89301555e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8920000c00 tx=0x7f89200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:37.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.144+0000 7f8927400640 1 -- 172.21.15.19:0/4151715119 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920034040 con 0x7f89300cc0b0 2024-03-28T13:12:37.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.144+0000 7f8927400640 1 -- 172.21.15.19:0/4151715119 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8920030640 con 0x7f89300cc0b0 2024-03-28T13:12:37.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.144+0000 7f8927400640 1 -- 172.21.15.19:0/4151715119 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920039d40 con 0x7f89300cc0b0 2024-03-28T13:12:37.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.144+0000 7f892fe00640 1 -- 172.21.15.19:0/4151715119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893013cde0 con 0x7f89300cc0b0 2024-03-28T13:12:37.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.144+0000 7f892fe00640 1 -- 172.21.15.19:0/4151715119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893013d3a0 con 0x7f89300cc0b0 2024-03-28T13:12:37.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.145+0000 7f892fe00640 1 -- 172.21.15.19:0/4151715119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8930075940 con 0x7f89300cc0b0 2024-03-28T13:12:37.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.146+0000 7f8927400640 1 -- 172.21.15.19:0/4151715119 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f892004b060 con 0x7f89300cc0b0 2024-03-28T13:12:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.147+0000 7f8927400640 1 --2- 172.21.15.19:0/4151715119 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f891006c530 0x7f891006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.147+0000 7f8927400640 1 -- 172.21.15.19:0/4151715119 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7f89200ad3f0 con 0x7f89300cc0b0 2024-03-28T13:12:37.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.148+0000 7f892e000640 1 --2- 172.21.15.19:0/4151715119 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f891006c530 0x7f891006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:37.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.149+0000 7f892e000640 1 --2- 172.21.15.19:0/4151715119 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f891006c530 0x7f891006e9f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f893015bd80 tx=0x7f891c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:37.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.152+0000 7f8927400640 1 -- 172.21.15.19:0/4151715119 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f892007b980 con 0x7f89300cc0b0 2024-03-28T13:12:37.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.361+0000 7f892fe00640 1 -- 172.21.15.19:0/4151715119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f893015beb0 con 0x7f89300cc0b0 2024-03-28T13:12:37.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.363+0000 7f8927400640 1 -- 172.21.15.19:0/4151715119 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+16900 (secure 0 0 0) 0x7f892007b280 con 0x7f89300cc0b0 2024-03-28T13:12:37.363 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:37.364 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":51,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:12:24.186519+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:11:44.316258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-28T13:11:45.360154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.0","mappings":[{"from":5,"to":0}]},{"pgid":"7.6","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.39","mappings":[{"from":5,"to":7}]},{"pgid":"7.3a","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:12:37.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.366+0000 7f892fe00640 1 -- 172.21.15.19:0/4151715119 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f891006c530 msgr2=0x7f891006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:37.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.366+0000 7f892fe00640 1 --2- 172.21.15.19:0/4151715119 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f891006c530 0x7f891006e9f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f893015bd80 tx=0x7f891c009040 comp rx=0 tx=0).stop 2024-03-28T13:12:37.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.366+0000 7f892fe00640 1 -- 172.21.15.19:0/4151715119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89300cc0b0 msgr2=0x7f89301555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:37.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.366+0000 7f892fe00640 1 --2- 172.21.15.19:0/4151715119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89300cc0b0 0x7f89301555e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8920000c00 tx=0x7f89200046c0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.366+0000 7f892fe00640 1 -- 172.21.15.19:0/4151715119 shutdown_connections 2024-03-28T13:12:37.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.366+0000 7f892fe00640 1 --2- 172.21.15.19:0/4151715119 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f891006c530 0x7f891006e9f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.366+0000 7f892fe00640 1 --2- 172.21.15.19:0/4151715119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f893013e180 0x7f893012d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.366+0000 7f892fe00640 1 --2- 172.21.15.19:0/4151715119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89300d04d0 0x7f893013c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.366+0000 7f892fe00640 1 --2- 172.21.15.19:0/4151715119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89300cc0b0 0x7f89301555e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.366+0000 7f892fe00640 1 -- 172.21.15.19:0/4151715119 >> 172.21.15.19:0/4151715119 conn(0x7f89300ccad0 msgr2=0x7f8930072f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:37.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.366+0000 7f892fe00640 1 -- 172.21.15.19:0/4151715119 shutdown_connections 2024-03-28T13:12:37.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.366+0000 7f892fe00640 1 -- 172.21.15.19:0/4151715119 wait complete. 2024-03-28T13:12:37.385 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T13:12:37.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.602+0000 7fb914200640 1 Processor -- start 2024-03-28T13:12:37.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.602+0000 7fb914200640 1 -- start start 2024-03-28T13:12:37.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.603+0000 7fb914200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb90c0d0280 0x7fb90c0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:37.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.603+0000 7fb914200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb90c141c10 0x7fb90c071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:37.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.604+0000 7fb914200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb90c15ec80 con 0x7fb90c071eb0 2024-03-28T13:12:37.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.604+0000 7fb914200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb90c0d0ba0 con 0x7fb90c141c10 2024-03-28T13:12:37.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.604+0000 7fb914200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb90c15ef20 con 0x7fb90c0d0280 2024-03-28T13:12:37.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.604+0000 7fb912200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb90c071eb0 0x7fb90c06d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43070/0 (socket says 172.21.15.19:43070) 2024-03-28T13:12:37.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.604+0000 7fb912200640 1 -- 172.21.15.19:0/3077098069 learned_addr learned my addr 172.21.15.19:0/3077098069 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:37.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.604+0000 7fb910e00640 1 --2- 172.21.15.19:0/3077098069 >> v2:172.21.15.19:3300/0 conn(0x7fb90c141c10 0x7fb90c071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:37.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.604+0000 7fb911800640 1 --2- 172.21.15.19:0/3077098069 >> v2:172.21.15.188:3300/0 conn(0x7fb90c0d0280 0x7fb90c0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:37.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.605+0000 7fb910e00640 1 -- 172.21.15.19:0/3077098069 >> v2:172.21.15.188:3300/0 conn(0x7fb90c0d0280 msgr2=0x7fb90c0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:37.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.605+0000 7fb910e00640 1 --2- 172.21.15.19:0/3077098069 >> v2:172.21.15.188:3300/0 conn(0x7fb90c0d0280 0x7fb90c0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.605+0000 7fb910e00640 1 -- 172.21.15.19:0/3077098069 >> v1:172.21.15.19:6789/0 conn(0x7fb90c071eb0 legacy=0x7fb90c06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.605+0000 7fb910e00640 1 -- 172.21.15.19:0/3077098069 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90c15f200 con 0x7fb90c141c10 2024-03-28T13:12:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.605+0000 7fb90be00640 1 -- 172.21.15.19:0/3077098069 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2905455738 0 0) 0x7fb90c15ec80 con 0x7fb90c071eb0 2024-03-28T13:12:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.606+0000 7fb910e00640 1 --2- 172.21.15.19:0/3077098069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c141c10 0x7fb90c071970 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0d1050 tx=0x7fb8f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5413a8645cdb7764 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.606+0000 7fb90be00640 1 -- 172.21.15.19:0/3077098069 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8035020 con 0x7fb90c141c10 2024-03-28T13:12:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.606+0000 7fb90be00640 1 -- 172.21.15.19:0/3077098069 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f8031830 con 0x7fb90c141c10 2024-03-28T13:12:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.606+0000 7fb90be00640 1 -- 172.21.15.19:0/3077098069 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8031c00 con 0x7fb90c141c10 2024-03-28T13:12:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.607+0000 7fb914200640 1 -- 172.21.15.19:0/3077098069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c141c10 msgr2=0x7fb90c071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.607+0000 7fb914200640 1 --2- 172.21.15.19:0/3077098069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c141c10 0x7fb90c071970 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0d1050 tx=0x7fb8f802b8d0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.607+0000 7fb914200640 1 -- 172.21.15.19:0/3077098069 shutdown_connections 2024-03-28T13:12:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.607+0000 7fb914200640 1 --2- 172.21.15.19:0/3077098069 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c141c10 0x7fb90c071970 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.607+0000 7fb914200640 1 --2- 172.21.15.19:0/3077098069 >> v2:172.21.15.188:3300/0 conn(0x7fb90c0d0280 0x7fb90c0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.607+0000 7fb914200640 1 -- 172.21.15.19:0/3077098069 >> 172.21.15.19:0/3077098069 conn(0x7fb90c06b750 msgr2=0x7fb90c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.607+0000 7fb914200640 1 -- 172.21.15.19:0/3077098069 shutdown_connections 2024-03-28T13:12:37.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.607+0000 7fb914200640 1 -- 172.21.15.19:0/3077098069 wait complete. 2024-03-28T13:12:37.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.609+0000 7fb914200640 1 Processor -- start 2024-03-28T13:12:37.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.609+0000 7fb914200640 1 -- start start 2024-03-28T13:12:37.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.610+0000 7fb914200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90c071eb0 0x7fb90c1f4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:37.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.610+0000 7fb914200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90c0d0280 0x7fb90c09c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.610+0000 7fb914200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c141c10 0x7fb90c09cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.610+0000 7fb914200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c158e90 con 0x7fb90c141c10 2024-03-28T13:12:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.610+0000 7fb914200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c15f7c0 con 0x7fb90c071eb0 2024-03-28T13:12:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.610+0000 7fb914200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb90c1f7780 con 0x7fb90c0d0280 2024-03-28T13:12:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.610+0000 7fb912200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c141c10 0x7fb90c09cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.610+0000 7fb912200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c141c10 0x7fb90c09cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42646/0 (socket says 172.21.15.19:42646) 2024-03-28T13:12:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.610+0000 7fb912200640 1 -- 172.21.15.19:0/1686194975 learned_addr learned my addr 172.21.15.19:0/1686194975 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.610+0000 7fb910e00640 1 --2- 172.21.15.19:0/1686194975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90c0d0280 0x7fb90c09c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.611+0000 7fb911800640 1 --2- 172.21.15.19:0/1686194975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90c071eb0 0x7fb90c1f4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.611+0000 7fb912200640 1 -- 172.21.15.19:0/1686194975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90c0d0280 msgr2=0x7fb90c09c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.611+0000 7fb912200640 1 --2- 172.21.15.19:0/1686194975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90c0d0280 0x7fb90c09c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.611+0000 7fb912200640 1 -- 172.21.15.19:0/1686194975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90c071eb0 msgr2=0x7fb90c1f4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.611+0000 7fb912200640 1 --2- 172.21.15.19:0/1686194975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90c071eb0 0x7fb90c1f4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.611+0000 7fb912200640 1 -- 172.21.15.19:0/1686194975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90c09a750 con 0x7fb90c141c10 2024-03-28T13:12:37.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.611+0000 7fb912200640 1 --2- 172.21.15.19:0/1686194975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c141c10 0x7fb90c09cc40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb90400c7a0 tx=0x7fb90400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:37.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.611+0000 7fb909600640 1 -- 172.21.15.19:0/1686194975 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904018070 con 0x7fb90c141c10 2024-03-28T13:12:37.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.611+0000 7fb909600640 1 -- 172.21.15.19:0/1686194975 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb90400a640 con 0x7fb90c141c10 2024-03-28T13:12:37.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.612+0000 7fb914200640 1 -- 172.21.15.19:0/1686194975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90c09aa30 con 0x7fb90c141c10 2024-03-28T13:12:37.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.612+0000 7fb909600640 1 -- 172.21.15.19:0/1686194975 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90400aab0 con 0x7fb90c141c10 2024-03-28T13:12:37.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.612+0000 7fb914200640 1 -- 172.21.15.19:0/1686194975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb90c09aff0 con 0x7fb90c141c10 2024-03-28T13:12:37.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.613+0000 7fb914200640 1 -- 172.21.15.19:0/1686194975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb90c0685c0 con 0x7fb90c141c10 2024-03-28T13:12:37.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.614+0000 7fb909600640 1 -- 172.21.15.19:0/1686194975 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb90401b070 con 0x7fb90c141c10 2024-03-28T13:12:37.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.615+0000 7fb909600640 1 --2- 172.21.15.19:0/1686194975 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb8ec06c530 0x7fb8ec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:37.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.615+0000 7fb909600640 1 -- 172.21.15.19:0/1686194975 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5854+0+0 (secure 0 0 0) 0x7fb90408ac70 con 0x7fb90c141c10 2024-03-28T13:12:37.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.615+0000 7fb911800640 1 --2- 172.21.15.19:0/1686194975 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb8ec06c530 0x7fb8ec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:37.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.617+0000 7fb911800640 1 --2- 172.21.15.19:0/1686194975 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb8ec06c530 0x7fb8ec06e9f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb900006fd0 tx=0x7fb900008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:37.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.620+0000 7fb909600640 1 -- 172.21.15.19:0/1686194975 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb904058c40 con 0x7fb90c141c10 2024-03-28T13:12:37.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.864+0000 7fb914200640 1 -- 172.21.15.19:0/1686194975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb90c0cfdb0 con 0x7fb90c141c10 2024-03-28T13:12:37.865 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:12:37.866+0000 7f6b74400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T13:12:37.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.869+0000 7fb909600640 1 -- 172.21.15.19:0/1686194975 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7fb90405cf40 con 0x7fb90c141c10 2024-03-28T13:12:37.869 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T13:12:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.872+0000 7fb914200640 1 -- 172.21.15.19:0/1686194975 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb8ec06c530 msgr2=0x7fb8ec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.872+0000 7fb914200640 1 --2- 172.21.15.19:0/1686194975 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb8ec06c530 0x7fb8ec06e9f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb900006fd0 tx=0x7fb900008040 comp rx=0 tx=0).stop 2024-03-28T13:12:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.873+0000 7fb914200640 1 -- 172.21.15.19:0/1686194975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c141c10 msgr2=0x7fb90c09cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.873+0000 7fb914200640 1 --2- 172.21.15.19:0/1686194975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c141c10 0x7fb90c09cc40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb90400c7a0 tx=0x7fb90400cc60 comp rx=0 tx=0).stop 2024-03-28T13:12:37.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.874+0000 7fb914200640 1 -- 172.21.15.19:0/1686194975 shutdown_connections 2024-03-28T13:12:37.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.874+0000 7fb914200640 1 --2- 172.21.15.19:0/1686194975 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb8ec06c530 0x7fb8ec06e9f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.874+0000 7fb914200640 1 --2- 172.21.15.19:0/1686194975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c141c10 0x7fb90c09cc40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.874+0000 7fb914200640 1 --2- 172.21.15.19:0/1686194975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90c0d0280 0x7fb90c09c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.874+0000 7fb914200640 1 --2- 172.21.15.19:0/1686194975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90c071eb0 0x7fb90c1f4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:37.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.874+0000 7fb914200640 1 -- 172.21.15.19:0/1686194975 >> 172.21.15.19:0/1686194975 conn(0x7fb90c06b750 msgr2=0x7fb90c15fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:37.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.875+0000 7fb914200640 1 -- 172.21.15.19:0/1686194975 shutdown_connections 2024-03-28T13:12:37.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:37.875+0000 7fb914200640 1 -- 172.21.15.19:0/1686194975 wait complete. 2024-03-28T13:12:37.903 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-28T13:12:37.909 INFO:tasks.ceph.mds.a.smithi019.stderr: -6168> 2024-03-28T13:11:52.861+0000 7fad70e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:12:37.918 INFO:tasks.ceph.mds.a.smithi019.stderr: -3336> 2024-03-28T13:11:57.626+0000 7fad6be00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={4997=0-4194304@1} caps={4997=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=4997 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55cd7dfccc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-28T13:12:37.984 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-03-28T13:12:38.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.099+0000 7fbba1400640 1 Processor -- start 2024-03-28T13:12:38.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.100+0000 7fbba1400640 1 -- start start 2024-03-28T13:12:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.101+0000 7fbba1400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbb9c0d0280 0x7fbb9c0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.101+0000 7fbba1400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbb9c162160 0x7fbb9c162610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.101+0000 7fbba1400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb9c15ec80 con 0x7fbb9c141c10 2024-03-28T13:12:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.101+0000 7fbba1400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbb9c0d0ba0 con 0x7fbb9c0d0280 2024-03-28T13:12:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.101+0000 7fbba1400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fbb9c15ef20 con 0x7fbb9c162160 2024-03-28T13:12:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.101+0000 7fbb9a800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbb9c0d0280 0x7fbb9c0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.101+0000 7fbb99e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbb9c141c10 0x7fbb9c071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43092/0 (socket says 172.21.15.19:43092) 2024-03-28T13:12:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.101+0000 7fbb99e00640 1 -- 172.21.15.19:0/1757956325 learned_addr learned my addr 172.21.15.19:0/1757956325 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.102+0000 7fbb9b200640 1 --2- 172.21.15.19:0/1757956325 >> v2:172.21.15.188:3300/0 conn(0x7fbb9c162160 0x7fbb9c162610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.102+0000 7fbb9a800640 1 -- 172.21.15.19:0/1757956325 >> v2:172.21.15.188:3300/0 conn(0x7fbb9c162160 msgr2=0x7fbb9c162610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.102+0000 7fbb9a800640 1 --2- 172.21.15.19:0/1757956325 >> v2:172.21.15.188:3300/0 conn(0x7fbb9c162160 0x7fbb9c162610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:38.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.102+0000 7fbb9a800640 1 -- 172.21.15.19:0/1757956325 >> v1:172.21.15.19:6789/0 conn(0x7fbb9c141c10 legacy=0x7fbb9c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:38.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.102+0000 7fbb9a800640 1 -- 172.21.15.19:0/1757956325 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c15f200 con 0x7fbb9c0d0280 2024-03-28T13:12:38.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.102+0000 7fbb99400640 1 -- 172.21.15.19:0/1757956325 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2702840279 0 0) 0x7fbb9c15ec80 con 0x7fbb9c141c10 2024-03-28T13:12:38.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.103+0000 7fbb9a800640 1 --2- 172.21.15.19:0/1757956325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c0d0280 0x7fbb9c0d0660 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0d0d80 tx=0x7fbb8c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1061269e730a57a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:38.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.103+0000 7fbb99400640 1 -- 172.21.15.19:0/1757956325 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c039070 con 0x7fbb9c0d0280 2024-03-28T13:12:38.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.103+0000 7fbb99400640 1 -- 172.21.15.19:0/1757956325 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb8c02cc60 con 0x7fbb9c0d0280 2024-03-28T13:12:38.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.103+0000 7fbb99400640 1 -- 172.21.15.19:0/1757956325 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c034660 con 0x7fbb9c0d0280 2024-03-28T13:12:38.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.104+0000 7fbba1400640 1 -- 172.21.15.19:0/1757956325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c0d0280 msgr2=0x7fbb9c0d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:38.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.104+0000 7fbba1400640 1 --2- 172.21.15.19:0/1757956325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c0d0280 0x7fbb9c0d0660 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0d0d80 tx=0x7fbb8c02beb0 comp rx=0 tx=0).stop 2024-03-28T13:12:38.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.104+0000 7fbba1400640 1 -- 172.21.15.19:0/1757956325 shutdown_connections 2024-03-28T13:12:38.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.104+0000 7fbba1400640 1 --2- 172.21.15.19:0/1757956325 >> v2:172.21.15.188:3300/0 conn(0x7fbb9c162160 0x7fbb9c162610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:38.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.104+0000 7fbba1400640 1 --2- 172.21.15.19:0/1757956325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c0d0280 0x7fbb9c0d0660 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:38.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.104+0000 7fbba1400640 1 -- 172.21.15.19:0/1757956325 >> 172.21.15.19:0/1757956325 conn(0x7fbb9c06b750 msgr2=0x7fbb9c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:38.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.104+0000 7fbba1400640 1 -- 172.21.15.19:0/1757956325 shutdown_connections 2024-03-28T13:12:38.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.104+0000 7fbba1400640 1 -- 172.21.15.19:0/1757956325 wait complete. 2024-03-28T13:12:38.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.106+0000 7fbba1400640 1 Processor -- start 2024-03-28T13:12:38.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.106+0000 7fbba1400640 1 -- start start 2024-03-28T13:12:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.107+0000 7fbba1400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb9c0d0280 0x7fbb9c155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.107+0000 7fbba1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c141c10 0x7fbb9c1613e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.108+0000 7fbba1400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb9c162160 0x7fbb9c09ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.108+0000 7fbba1400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c158e90 con 0x7fbb9c141c10 2024-03-28T13:12:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.108+0000 7fbba1400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c15f7c0 con 0x7fbb9c162160 2024-03-28T13:12:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.108+0000 7fbba1400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbb9c142510 con 0x7fbb9c0d0280 2024-03-28T13:12:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.108+0000 7fbb99e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c141c10 0x7fbb9c1613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.108+0000 7fbb99e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c141c10 0x7fbb9c1613e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42678/0 (socket says 172.21.15.19:42678) 2024-03-28T13:12:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.108+0000 7fbb99e00640 1 -- 172.21.15.19:0/3079120349 learned_addr learned my addr 172.21.15.19:0/3079120349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.108+0000 7fbb9a800640 1 --2- 172.21.15.19:0/3079120349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb9c0d0280 0x7fbb9c155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.108+0000 7fbb9b200640 1 --2- 172.21.15.19:0/3079120349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb9c162160 0x7fbb9c09ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.108+0000 7fbb99e00640 1 -- 172.21.15.19:0/3079120349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb9c0d0280 msgr2=0x7fbb9c155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.108+0000 7fbb99e00640 1 --2- 172.21.15.19:0/3079120349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb9c0d0280 0x7fbb9c155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.108+0000 7fbb99e00640 1 -- 172.21.15.19:0/3079120349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb9c162160 msgr2=0x7fbb9c09ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.108+0000 7fbb99e00640 1 --2- 172.21.15.19:0/3079120349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb9c162160 0x7fbb9c09ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.109+0000 7fbb99e00640 1 -- 172.21.15.19:0/3079120349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c142230 con 0x7fbb9c141c10 2024-03-28T13:12:38.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.109+0000 7fbb9a800640 1 --2- 172.21.15.19:0/3079120349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb9c0d0280 0x7fbb9c155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:12:38.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.109+0000 7fbb99e00640 1 --2- 172.21.15.19:0/3079120349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c141c10 0x7fbb9c1613e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800c470 tx=0x7fbb8800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:38.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.109+0000 7fbb92a00640 1 -- 172.21.15.19:0/3079120349 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88018070 con 0x7fbb9c141c10 2024-03-28T13:12:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.109+0000 7fbb92a00640 1 -- 172.21.15.19:0/3079120349 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb880074e0 con 0x7fbb9c141c10 2024-03-28T13:12:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.109+0000 7fbba1400640 1 -- 172.21.15.19:0/3079120349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9c09cd30 con 0x7fbb9c141c10 2024-03-28T13:12:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.109+0000 7fbb92a00640 1 -- 172.21.15.19:0/3079120349 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb88004a20 con 0x7fbb9c141c10 2024-03-28T13:12:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.109+0000 7fbba1400640 1 -- 172.21.15.19:0/3079120349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9c099a00 con 0x7fbb9c141c10 2024-03-28T13:12:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.111+0000 7fbba1400640 1 -- 172.21.15.19:0/3079120349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb9c0685c0 con 0x7fbb9c141c10 2024-03-28T13:12:38.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.111+0000 7fbb92a00640 1 -- 172.21.15.19:0/3079120349 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbb8801b070 con 0x7fbb9c141c10 2024-03-28T13:12:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.113+0000 7fbb92a00640 1 --2- 172.21.15.19:0/3079120349 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbb7c06c530 0x7fbb7c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:38.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.113+0000 7fbb92a00640 1 -- 172.21.15.19:0/3079120349 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5940+0+0 (secure 0 0 0) 0x7fbb8808aa60 con 0x7fbb9c141c10 2024-03-28T13:12:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.114+0000 7fbb9a800640 1 --2- 172.21.15.19:0/3079120349 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbb7c06c530 0x7fbb7c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:38.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.115+0000 7fbb9a800640 1 --2- 172.21.15.19:0/3079120349 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbb7c06c530 0x7fbb7c06e9f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c036000 tx=0x7fbb8c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:38.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.118+0000 7fbb92a00640 1 -- 172.21.15.19:0/3079120349 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbb88058f80 con 0x7fbb9c141c10 2024-03-28T13:12:38.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.358+0000 7fbba1400640 1 -- 172.21.15.19:0/3079120349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbb9c0af030 con 0x7fbb9c141c10 2024-03-28T13:12:38.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.880+0000 7fbb92a00640 1 -- 172.21.15.19:0/3079120349 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7fbb88058880 con 0x7fbb9c141c10 2024-03-28T13:12:38.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.883+0000 7fbba1400640 1 -- 172.21.15.19:0/3079120349 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbb7c06c530 msgr2=0x7fbb7c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:38.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.883+0000 7fbba1400640 1 --2- 172.21.15.19:0/3079120349 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbb7c06c530 0x7fbb7c06e9f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c036000 tx=0x7fbb8c030000 comp rx=0 tx=0).stop 2024-03-28T13:12:38.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.883+0000 7fbba1400640 1 -- 172.21.15.19:0/3079120349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c141c10 msgr2=0x7fbb9c1613e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:38.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.883+0000 7fbba1400640 1 --2- 172.21.15.19:0/3079120349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c141c10 0x7fbb9c1613e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800c470 tx=0x7fbb8800c930 comp rx=0 tx=0).stop 2024-03-28T13:12:38.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.884+0000 7fbba1400640 1 -- 172.21.15.19:0/3079120349 shutdown_connections 2024-03-28T13:12:38.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.884+0000 7fbba1400640 1 --2- 172.21.15.19:0/3079120349 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbb7c06c530 0x7fbb7c06e9f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:38.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.884+0000 7fbba1400640 1 --2- 172.21.15.19:0/3079120349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb9c162160 0x7fbb9c09ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:38.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.884+0000 7fbba1400640 1 --2- 172.21.15.19:0/3079120349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb9c141c10 0x7fbb9c1613e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:38.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.884+0000 7fbba1400640 1 --2- 172.21.15.19:0/3079120349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb9c0d0280 0x7fbb9c155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:38.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.884+0000 7fbba1400640 1 -- 172.21.15.19:0/3079120349 >> 172.21.15.19:0/3079120349 conn(0x7fbb9c06b750 msgr2=0x7fbb9c0b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:38.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.884+0000 7fbba1400640 1 -- 172.21.15.19:0/3079120349 shutdown_connections 2024-03-28T13:12:38.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:38.884+0000 7fbba1400640 1 -- 172.21.15.19:0/3079120349 wait complete. 2024-03-28T13:12:38.907 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-28T13:12:39.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.181+0000 7f7573200640 1 Processor -- start 2024-03-28T13:12:39.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.182+0000 7f7573200640 1 -- start start 2024-03-28T13:12:39.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.183+0000 7f7573200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f757406ea70 0x7f75741630e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:39.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.183+0000 7f7573200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7574072e50 con 0x7f7574162d00 2024-03-28T13:12:39.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.183+0000 7f7573200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7574073130 con 0x7f7574163620 2024-03-28T13:12:39.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.183+0000 7f7573200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f757406d5d0 con 0x7f757406ea70 2024-03-28T13:12:39.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.183+0000 7f7571e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7574162d00 0x7f7574072420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43098/0 (socket says 172.21.15.19:43098) 2024-03-28T13:12:39.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.183+0000 7f7571e00640 1 -- 172.21.15.19:0/3351458057 learned_addr learned my addr 172.21.15.19:0/3351458057 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:39.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7569400640 1 --2- 172.21.15.19:0/3351458057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f757406ea70 0x7f75741630e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:39.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7571400640 1 -- 172.21.15.19:0/3351458057 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2727443063 0 0) 0x7f7574072e50 con 0x7f7574162d00 2024-03-28T13:12:39.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7571400640 1 -- 172.21.15.19:0/3351458057 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7558003520 con 0x7f7574162d00 2024-03-28T13:12:39.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7571400640 1 -- 172.21.15.19:0/3351458057 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2489139771 0 0) 0x7f7558003520 con 0x7f7574162d00 2024-03-28T13:12:39.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7571400640 1 -- 172.21.15.19:0/3351458057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f757406ea70 msgr2=0x7f75741630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:39.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7571400640 1 --2- 172.21.15.19:0/3351458057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f757406ea70 0x7f75741630e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:39.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7571400640 1 -- 172.21.15.19:0/3351458057 >> v1:172.21.15.188:6789/0 conn(0x7f7574163620 legacy=0x7f757416c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:39.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7571400640 1 -- 172.21.15.19:0/3351458057 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7574073410 con 0x7f7574162d00 2024-03-28T13:12:39.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7571400640 1 -- 172.21.15.19:0/3351458057 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f7564001670 con 0x7f7574162d00 2024-03-28T13:12:39.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7571400640 1 -- 172.21.15.19:0/3351458057 >> v1:172.21.15.19:6789/0 conn(0x7f7574162d00 legacy=0x7f7574072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:39.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7571400640 1 --2- 172.21.15.19:0/3351458057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f757406ea70 0x7f75580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:39.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7571400640 1 --2- 172.21.15.19:0/3351458057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7558004010 0x7f7558004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:39.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7571400640 1 --2- 172.21.15.19:0/3351458057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7558004d40 0x7f75580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:39.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7571400640 1 -- 172.21.15.19:0/3351458057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f757406d5d0 con 0x7f7558004010 2024-03-28T13:12:39.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7571400640 1 -- 172.21.15.19:0/3351458057 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f755800bb80 con 0x7f7558004d40 2024-03-28T13:12:39.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.184+0000 7f7571400640 1 -- 172.21.15.19:0/3351458057 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f755800bd80 con 0x7f757406ea70 2024-03-28T13:12:39.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.185+0000 7f7569400640 1 --2- 172.21.15.19:0/3351458057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7558004010 0x7f7558004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:39.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.185+0000 7f7571e00640 1 --2- 172.21.15.19:0/3351458057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f757406ea70 0x7f75580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:39.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.185+0000 7f7572800640 1 --2- 172.21.15.19:0/3351458057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7558004d40 0x7f75580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:39.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.185+0000 7f7569400640 1 -- 172.21.15.19:0/3351458057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f757406ea70 msgr2=0x7f75580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:39.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.185+0000 7f7569400640 1 --2- 172.21.15.19:0/3351458057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f757406ea70 0x7f75580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:39.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.185+0000 7f7569400640 1 -- 172.21.15.19:0/3351458057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7558004d40 msgr2=0x7f75580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:39.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.185+0000 7f7569400640 1 --2- 172.21.15.19:0/3351458057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7558004d40 0x7f75580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:39.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.185+0000 7f7569400640 1 -- 172.21.15.19:0/3351458057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7558003520 con 0x7f7558004010 2024-03-28T13:12:39.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.185+0000 7f7571e00640 1 --2- 172.21.15.19:0/3351458057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f757406ea70 0x7f75580039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:39.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.186+0000 7f7569400640 1 --2- 172.21.15.19:0/3351458057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7558004010 0x7f7558004800 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f755c00ba30 tx=0x7f7558005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07d624aed8e3107 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:39.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.187+0000 7f7571400640 1 -- 172.21.15.19:0/3351458057 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c036020 con 0x7f7558004010 2024-03-28T13:12:39.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.187+0000 7f7571400640 1 -- 172.21.15.19:0/3351458057 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f755c032640 con 0x7f7558004010 2024-03-28T13:12:39.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.187+0000 7f7571400640 1 -- 172.21.15.19:0/3351458057 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c00b600 con 0x7f7558004010 2024-03-28T13:12:39.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.187+0000 7f7573200640 1 -- 172.21.15.19:0/3351458057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7558004010 msgr2=0x7f7558004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:39.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.187+0000 7f7573200640 1 --2- 172.21.15.19:0/3351458057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7558004010 0x7f7558004800 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f755c00ba30 tx=0x7f7558005830 comp rx=0 tx=0).stop 2024-03-28T13:12:39.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.187+0000 7f7572800640 1 -- 172.21.15.19:0/3351458057 reap_dead start 2024-03-28T13:12:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.188+0000 7f7573200640 1 -- 172.21.15.19:0/3351458057 shutdown_connections 2024-03-28T13:12:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.188+0000 7f7573200640 1 -- 172.21.15.19:0/3351458057 >> 172.21.15.19:0/3351458057 conn(0x7f75740b1d80 msgr2=0x7f75740b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.188+0000 7f7573200640 1 -- 172.21.15.19:0/3351458057 shutdown_connections 2024-03-28T13:12:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.188+0000 7f7573200640 1 -- 172.21.15.19:0/3351458057 wait complete. 2024-03-28T13:12:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.190+0000 7f7573200640 1 Processor -- start 2024-03-28T13:12:39.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.190+0000 7f7573200640 1 -- start start 2024-03-28T13:12:39.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.191+0000 7f7573200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7558004010 0x7f757414c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:39.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.191+0000 7f7573200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7574161530 0x7f75741619b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:39.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.191+0000 7f7573200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75740a9610 0x7f75740a9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.191+0000 7f7573200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7574072b30 con 0x7f75740a9610 2024-03-28T13:12:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.191+0000 7f7573200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7574163ab0 con 0x7f7574161530 2024-03-28T13:12:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.191+0000 7f7573200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f75740b3a70 con 0x7f7558004010 2024-03-28T13:12:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.191+0000 7f7572800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75740a9610 0x7f75740a9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.191+0000 7f7572800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75740a9610 0x7f75740a9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42690/0 (socket says 172.21.15.19:42690) 2024-03-28T13:12:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.191+0000 7f7572800640 1 -- 172.21.15.19:0/4160619684 learned_addr learned my addr 172.21.15.19:0/4160619684 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.191+0000 7f7569400640 1 --2- 172.21.15.19:0/4160619684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7574161530 0x7f75741619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.191+0000 7f7571e00640 1 --2- 172.21.15.19:0/4160619684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7558004010 0x7f757414c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.192+0000 7f7572800640 1 -- 172.21.15.19:0/4160619684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7558004010 msgr2=0x7f757414c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.192+0000 7f7572800640 1 --2- 172.21.15.19:0/4160619684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7558004010 0x7f757414c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.192+0000 7f7572800640 1 -- 172.21.15.19:0/4160619684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7574161530 msgr2=0x7f75741619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.192+0000 7f7572800640 1 --2- 172.21.15.19:0/4160619684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7574161530 0x7f75741619b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:39.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.192+0000 7f7572800640 1 -- 172.21.15.19:0/4160619684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75740aa370 con 0x7f75740a9610 2024-03-28T13:12:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.192+0000 7f7572800640 1 --2- 172.21.15.19:0/4160619684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75740a9610 0x7f75740a9a90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7574163a00 tx=0x7f756c00ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.192+0000 7f756aa00640 1 -- 172.21.15.19:0/4160619684 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c003b80 con 0x7f75740a9610 2024-03-28T13:12:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.192+0000 7f756aa00640 1 -- 172.21.15.19:0/4160619684 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756c018070 con 0x7f75740a9610 2024-03-28T13:12:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.192+0000 7f756aa00640 1 -- 172.21.15.19:0/4160619684 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0050d0 con 0x7f75740a9610 2024-03-28T13:12:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.192+0000 7f7573200640 1 -- 172.21.15.19:0/4160619684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757414ce90 con 0x7f75740a9610 2024-03-28T13:12:39.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.192+0000 7f7573200640 1 -- 172.21.15.19:0/4160619684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f757414d450 con 0x7f75740a9610 2024-03-28T13:12:39.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.194+0000 7f7573200640 1 -- 172.21.15.19:0/4160619684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7574075940 con 0x7f75740a9610 2024-03-28T13:12:39.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.195+0000 7f756aa00640 1 -- 172.21.15.19:0/4160619684 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f756c005270 con 0x7f75740a9610 2024-03-28T13:12:39.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.196+0000 7f756aa00640 1 --2- 172.21.15.19:0/4160619684 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f755006c560 0x7f755006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:39.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.196+0000 7f756aa00640 1 -- 172.21.15.19:0/4160619684 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5940+0+0 (secure 0 0 0) 0x7f756c08b3c0 con 0x7f75740a9610 2024-03-28T13:12:39.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.196+0000 7f7571e00640 1 --2- 172.21.15.19:0/4160619684 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f755006c560 0x7f755006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:39.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.197+0000 7f7571e00640 1 --2- 172.21.15.19:0/4160619684 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f755006c560 0x7f755006ea20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7564001200 tx=0x7f7564009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:39.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.200+0000 7f756aa00640 1 -- 172.21.15.19:0/4160619684 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f756c059980 con 0x7f75740a9610 2024-03-28T13:12:39.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:39.404+0000 7f7573200640 1 -- 172.21.15.19:0/4160619684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f75740b1660 con 0x7f75740a9610 2024-03-28T13:12:40.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.333+0000 7f756aa00640 1 -- 172.21.15.19:0/4160619684 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) ==== 181+0+0 (secure 0 0 0) 0x7f756c059280 con 0x7f75740a9610 2024-03-28T13:12:40.334 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-03-28T13:12:40.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.336+0000 7f7573200640 1 -- 172.21.15.19:0/4160619684 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f755006c560 msgr2=0x7f755006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:40.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.337+0000 7f7573200640 1 --2- 172.21.15.19:0/4160619684 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f755006c560 0x7f755006ea20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7564001200 tx=0x7f7564009040 comp rx=0 tx=0).stop 2024-03-28T13:12:40.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.337+0000 7f7573200640 1 -- 172.21.15.19:0/4160619684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75740a9610 msgr2=0x7f75740a9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:40.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.337+0000 7f7573200640 1 --2- 172.21.15.19:0/4160619684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75740a9610 0x7f75740a9a90 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f7574163a00 tx=0x7f756c00ea00 comp rx=0 tx=0).stop 2024-03-28T13:12:40.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.337+0000 7f7573200640 1 -- 172.21.15.19:0/4160619684 shutdown_connections 2024-03-28T13:12:40.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.337+0000 7f7573200640 1 --2- 172.21.15.19:0/4160619684 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f755006c560 0x7f755006ea20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:40.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.337+0000 7f7573200640 1 --2- 172.21.15.19:0/4160619684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75740a9610 0x7f75740a9a90 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:40.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.337+0000 7f7573200640 1 --2- 172.21.15.19:0/4160619684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7574161530 0x7f75741619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:40.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.337+0000 7f7573200640 1 --2- 172.21.15.19:0/4160619684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7558004010 0x7f757414c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:40.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.337+0000 7f7573200640 1 -- 172.21.15.19:0/4160619684 >> 172.21.15.19:0/4160619684 conn(0x7f75740b1d80 msgr2=0x7f7574072420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:40.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.337+0000 7f7573200640 1 -- 172.21.15.19:0/4160619684 shutdown_connections 2024-03-28T13:12:40.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.337+0000 7f7573200640 1 -- 172.21.15.19:0/4160619684 wait complete. 2024-03-28T13:12:40.364 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-28T13:12:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.579+0000 7fcd56600640 1 Processor -- start 2024-03-28T13:12:40.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.579+0000 7fcd56600640 1 -- start start 2024-03-28T13:12:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.580+0000 7fcd56600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd500acf10 0x7fcd5009dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.581+0000 7fcd56600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd500ad2f0 0x7fcd501453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.581+0000 7fcd56600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd5015cb90 con 0x7fcd5015ff70 2024-03-28T13:12:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.581+0000 7fcd56600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcd50145f60 con 0x7fcd500acf10 2024-03-28T13:12:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.581+0000 7fcd56600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcd5015c950 con 0x7fcd500ad2f0 2024-03-28T13:12:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.581+0000 7fcd4f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd500acf10 0x7fcd5009dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.581+0000 7fcd4f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd500acf10 0x7fcd5009dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42702/0 (socket says 172.21.15.19:42702) 2024-03-28T13:12:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.581+0000 7fcd4f400640 1 -- 172.21.15.19:0/309748555 learned_addr learned my addr 172.21.15.19:0/309748555 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.581+0000 7fcd4ea00640 1 --2- 172.21.15.19:0/309748555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd500ad2f0 0x7fcd501453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.582+0000 7fcd4f400640 1 -- 172.21.15.19:0/309748555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd500ad2f0 msgr2=0x7fcd501453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.582+0000 7fcd4f400640 1 --2- 172.21.15.19:0/309748555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd500ad2f0 0x7fcd501453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.582+0000 7fcd4f400640 1 -- 172.21.15.19:0/309748555 >> v1:172.21.15.188:6789/0 conn(0x7fcd5015ff70 legacy=0x7fcd501603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.582+0000 7fcd4f400640 1 -- 172.21.15.19:0/309748555 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5015ce70 con 0x7fcd500acf10 2024-03-28T13:12:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.582+0000 7fcd4ea00640 1 --2- 172.21.15.19:0/309748555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd500ad2f0 0x7fcd501453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.583+0000 7fcd4f400640 1 --2- 172.21.15.19:0/309748555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd500acf10 0x7fcd5009dcf0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009870 tx=0x7fcd4002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f083bb7b6f082d0b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.583+0000 7fcd4e000640 1 -- 172.21.15.19:0/309748555 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd400317c0 con 0x7fcd500acf10 2024-03-28T13:12:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.583+0000 7fcd4e000640 1 -- 172.21.15.19:0/309748555 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd40037070 con 0x7fcd500acf10 2024-03-28T13:12:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.583+0000 7fcd4e000640 1 -- 172.21.15.19:0/309748555 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40031b80 con 0x7fcd500acf10 2024-03-28T13:12:40.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.584+0000 7fcd56600640 1 -- 172.21.15.19:0/309748555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd500acf10 msgr2=0x7fcd5009dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:40.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.584+0000 7fcd56600640 1 --2- 172.21.15.19:0/309748555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd500acf10 0x7fcd5009dcf0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009870 tx=0x7fcd4002b860 comp rx=0 tx=0).stop 2024-03-28T13:12:40.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.584+0000 7fcd56600640 1 -- 172.21.15.19:0/309748555 shutdown_connections 2024-03-28T13:12:40.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.584+0000 7fcd56600640 1 --2- 172.21.15.19:0/309748555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd500ad2f0 0x7fcd501453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:40.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.584+0000 7fcd56600640 1 --2- 172.21.15.19:0/309748555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd500acf10 0x7fcd5009dcf0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:40.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.584+0000 7fcd56600640 1 -- 172.21.15.19:0/309748555 >> 172.21.15.19:0/309748555 conn(0x7fcd5006bba0 msgr2=0x7fcd500b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:40.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.584+0000 7fcd56600640 1 -- 172.21.15.19:0/309748555 shutdown_connections 2024-03-28T13:12:40.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.584+0000 7fcd56600640 1 -- 172.21.15.19:0/309748555 wait complete. 2024-03-28T13:12:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.585+0000 7fcd56600640 1 Processor -- start 2024-03-28T13:12:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.586+0000 7fcd56600640 1 -- start start 2024-03-28T13:12:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd56600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd500acf10 0x7fcd5006dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd56600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd500ad2f0 0x7fcd5015bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd56600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd5015ff70 0x7fcd5013b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd56600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5015d430 con 0x7fcd500acf10 2024-03-28T13:12:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd56600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5015d7a0 con 0x7fcd500ad2f0 2024-03-28T13:12:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd56600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcd500ad770 con 0x7fcd5015ff70 2024-03-28T13:12:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd4f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd500acf10 0x7fcd5006dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd4f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd500acf10 0x7fcd5006dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42714/0 (socket says 172.21.15.19:42714) 2024-03-28T13:12:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd4f400640 1 -- 172.21.15.19:0/4127880963 learned_addr learned my addr 172.21.15.19:0/4127880963 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd4f400640 1 -- 172.21.15.19:0/4127880963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd5015ff70 msgr2=0x7fcd5013b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd4fe00640 1 --2- 172.21.15.19:0/4127880963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd5015ff70 0x7fcd5013b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd4ea00640 1 --2- 172.21.15.19:0/4127880963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd500ad2f0 0x7fcd5015bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd4f400640 1 --2- 172.21.15.19:0/4127880963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd5015ff70 0x7fcd5013b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:40.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd4f400640 1 -- 172.21.15.19:0/4127880963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd500ad2f0 msgr2=0x7fcd5015bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd4f400640 1 --2- 172.21.15.19:0/4127880963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd500ad2f0 0x7fcd5015bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd4f400640 1 -- 172.21.15.19:0/4127880963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd501f8460 con 0x7fcd500acf10 2024-03-28T13:12:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.587+0000 7fcd4ea00640 1 --2- 172.21.15.19:0/4127880963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd500ad2f0 0x7fcd5015bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.588+0000 7fcd4f400640 1 --2- 172.21.15.19:0/4127880963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd500acf10 0x7fcd5006dfb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fcd40000c00 tx=0x7fcd400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.588+0000 7fcd47400640 1 -- 172.21.15.19:0/4127880963 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40032860 con 0x7fcd500acf10 2024-03-28T13:12:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.588+0000 7fcd47400640 1 -- 172.21.15.19:0/4127880963 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd40032a20 con 0x7fcd500acf10 2024-03-28T13:12:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.588+0000 7fcd47400640 1 -- 172.21.15.19:0/4127880963 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40032e10 con 0x7fcd500acf10 2024-03-28T13:12:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.588+0000 7fcd56600640 1 -- 172.21.15.19:0/4127880963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd501f8740 con 0x7fcd500acf10 2024-03-28T13:12:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.588+0000 7fcd56600640 1 -- 172.21.15.19:0/4127880963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd501f8d00 con 0x7fcd500acf10 2024-03-28T13:12:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.590+0000 7fcd56600640 1 -- 172.21.15.19:0/4127880963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5009c1d0 con 0x7fcd500acf10 2024-03-28T13:12:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.591+0000 7fcd47400640 1 -- 172.21.15.19:0/4127880963 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcd4004c060 con 0x7fcd500acf10 2024-03-28T13:12:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.592+0000 7fcd47400640 1 --2- 172.21.15.19:0/4127880963 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd3006c530 0x7fcd3006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.592+0000 7fcd47400640 1 -- 172.21.15.19:0/4127880963 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5502+0+0 (secure 0 0 0) 0x7fcd400ad0c0 con 0x7fcd500acf10 2024-03-28T13:12:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.593+0000 7fcd4ea00640 1 --2- 172.21.15.19:0/4127880963 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd3006c530 0x7fcd3006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.594+0000 7fcd4ea00640 1 --2- 172.21.15.19:0/4127880963 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd3006c530 0x7fcd3006e9f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c006fd0 tx=0x7fcd3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.596+0000 7fcd47400640 1 -- 172.21.15.19:0/4127880963 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcd40040070 con 0x7fcd500acf10 2024-03-28T13:12:40.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:40.802+0000 7fcd56600640 1 -- 172.21.15.19:0/4127880963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcd500685c0 con 0x7fcd500acf10 2024-03-28T13:12:41.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.337+0000 7fcd47400640 1 -- 172.21.15.19:0/4127880963 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) ==== 169+0+0 (secure 0 0 0) 0x7fcd4004c350 con 0x7fcd500acf10 2024-03-28T13:12:41.337 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-03-28T13:12:41.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.341+0000 7fcd56600640 1 -- 172.21.15.19:0/4127880963 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd3006c530 msgr2=0x7fcd3006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:41.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.341+0000 7fcd56600640 1 --2- 172.21.15.19:0/4127880963 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd3006c530 0x7fcd3006e9f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c006fd0 tx=0x7fcd3c008040 comp rx=0 tx=0).stop 2024-03-28T13:12:41.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.341+0000 7fcd56600640 1 -- 172.21.15.19:0/4127880963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd500acf10 msgr2=0x7fcd5006dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.341+0000 7fcd56600640 1 --2- 172.21.15.19:0/4127880963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd500acf10 0x7fcd5006dfb0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fcd40000c00 tx=0x7fcd400046c0 comp rx=0 tx=0).stop 2024-03-28T13:12:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.341+0000 7fcd56600640 1 -- 172.21.15.19:0/4127880963 shutdown_connections 2024-03-28T13:12:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.341+0000 7fcd56600640 1 --2- 172.21.15.19:0/4127880963 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd3006c530 0x7fcd3006e9f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.341+0000 7fcd56600640 1 --2- 172.21.15.19:0/4127880963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd5015ff70 0x7fcd5013b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.341+0000 7fcd56600640 1 --2- 172.21.15.19:0/4127880963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd500ad2f0 0x7fcd5015bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.341+0000 7fcd56600640 1 --2- 172.21.15.19:0/4127880963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd500acf10 0x7fcd5006dfb0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.341+0000 7fcd56600640 1 -- 172.21.15.19:0/4127880963 >> 172.21.15.19:0/4127880963 conn(0x7fcd5006bba0 msgr2=0x7fcd50071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.341+0000 7fcd56600640 1 -- 172.21.15.19:0/4127880963 shutdown_connections 2024-03-28T13:12:41.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.341+0000 7fcd56600640 1 -- 172.21.15.19:0/4127880963 wait complete. 2024-03-28T13:12:41.369 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-28T13:12:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.585+0000 7ffaa5a00640 1 Processor -- start 2024-03-28T13:12:41.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.585+0000 7ffaa5a00640 1 -- start start 2024-03-28T13:12:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.587+0000 7ffaa5a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffaa00cc0b0 0x7ffaa006dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.587+0000 7ffaa5a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffaa015fc50 0x7ffaa00cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.587+0000 7ffaa5a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaa006e4f0 con 0x7ffaa00cc490 2024-03-28T13:12:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.587+0000 7ffaa5a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ffaa015c610 con 0x7ffaa015fc50 2024-03-28T13:12:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.587+0000 7ffaa5a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ffaa00b3150 con 0x7ffaa00cc0b0 2024-03-28T13:12:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.587+0000 7ffa9f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffaa015fc50 0x7ffaa00cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.587+0000 7ffa9f400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ffaa015fc50 0x7ffaa00cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42730/0 (socket says 172.21.15.19:42730) 2024-03-28T13:12:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.587+0000 7ffa9f400640 1 -- 172.21.15.19:0/810914384 learned_addr learned my addr 172.21.15.19:0/810914384 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.588+0000 7ffa9f400640 1 -- 172.21.15.19:0/810914384 >> v2:172.21.15.188:3300/0 conn(0x7ffaa00cc0b0 msgr2=0x7ffaa006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.588+0000 7ffa9fe00640 1 --2- 172.21.15.19:0/810914384 >> v2:172.21.15.188:3300/0 conn(0x7ffaa00cc0b0 0x7ffaa006dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:41.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.588+0000 7ffa9f400640 1 --2- 172.21.15.19:0/810914384 >> v2:172.21.15.188:3300/0 conn(0x7ffaa00cc0b0 0x7ffaa006dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:41.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.588+0000 7ffa9f400640 1 -- 172.21.15.19:0/810914384 >> v1:172.21.15.188:6789/0 conn(0x7ffaa00cc490 legacy=0x7ffaa015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:41.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.588+0000 7ffa9f400640 1 -- 172.21.15.19:0/810914384 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa01600e0 con 0x7ffaa015fc50 2024-03-28T13:12:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.589+0000 7ffa9f400640 1 --2- 172.21.15.19:0/810914384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa015fc50 0x7ffaa00cfdc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00cfd10 tx=0x7ffa8c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305dfdb213fe205d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.589+0000 7ffa9ea00640 1 -- 172.21.15.19:0/810914384 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c035020 con 0x7ffaa015fc50 2024-03-28T13:12:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.589+0000 7ffa9ea00640 1 -- 172.21.15.19:0/810914384 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa8c031870 con 0x7ffaa015fc50 2024-03-28T13:12:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.589+0000 7ffa9ea00640 1 -- 172.21.15.19:0/810914384 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c031c40 con 0x7ffaa015fc50 2024-03-28T13:12:41.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.590+0000 7ffaa5a00640 1 -- 172.21.15.19:0/810914384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa015fc50 msgr2=0x7ffaa00cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.590+0000 7ffaa5a00640 1 --2- 172.21.15.19:0/810914384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa015fc50 0x7ffaa00cfdc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00cfd10 tx=0x7ffa8c02b910 comp rx=0 tx=0).stop 2024-03-28T13:12:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.590+0000 7ffaa5a00640 1 -- 172.21.15.19:0/810914384 shutdown_connections 2024-03-28T13:12:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.590+0000 7ffaa5a00640 1 --2- 172.21.15.19:0/810914384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa015fc50 0x7ffaa00cfdc0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.590+0000 7ffaa5a00640 1 --2- 172.21.15.19:0/810914384 >> v2:172.21.15.188:3300/0 conn(0x7ffaa00cc0b0 0x7ffaa006dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.590+0000 7ffaa5a00640 1 -- 172.21.15.19:0/810914384 >> 172.21.15.19:0/810914384 conn(0x7ffaa00ccad0 msgr2=0x7ffaa00cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.590+0000 7ffaa5a00640 1 -- 172.21.15.19:0/810914384 shutdown_connections 2024-03-28T13:12:41.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.590+0000 7ffaa5a00640 1 -- 172.21.15.19:0/810914384 wait complete. 2024-03-28T13:12:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.592+0000 7ffaa5a00640 1 Processor -- start 2024-03-28T13:12:41.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.592+0000 7ffaa5a00640 1 -- start start 2024-03-28T13:12:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.593+0000 7ffaa5a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa00cc0b0 0x7ffaa013dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.593+0000 7ffaa5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00cc490 0x7ffaa013f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.593+0000 7ffaa5a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa01f4e40 0x7ffaa0072440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.593+0000 7ffaa5a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa0160980 con 0x7ffaa00cc490 2024-03-28T13:12:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.593+0000 7ffaa5a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa01606a0 con 0x7ffaa00cc0b0 2024-03-28T13:12:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.593+0000 7ffaa5a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa00d0300 con 0x7ffaa01f4e40 2024-03-28T13:12:41.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.593+0000 7ffa9f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00cc490 0x7ffaa013f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.593+0000 7ffa9f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00cc490 0x7ffaa013f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42732/0 (socket says 172.21.15.19:42732) 2024-03-28T13:12:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.593+0000 7ffa9f400640 1 -- 172.21.15.19:0/275073193 learned_addr learned my addr 172.21.15.19:0/275073193 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.594+0000 7ffa9fe00640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa00cc0b0 0x7ffaa013dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.594+0000 7ffaa5000640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa01f4e40 0x7ffaa0072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.594+0000 7ffa9f400640 1 -- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa01f4e40 msgr2=0x7ffaa0072440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.594+0000 7ffa9f400640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa01f4e40 0x7ffaa0072440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.594+0000 7ffa9f400640 1 -- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa00cc0b0 msgr2=0x7ffaa013dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.594+0000 7ffa9f400640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa00cc0b0 0x7ffaa013dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:41.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.594+0000 7ffa9f400640 1 -- 172.21.15.19:0/275073193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa013fbb0 con 0x7ffaa00cc490 2024-03-28T13:12:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.594+0000 7ffaa5000640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa01f4e40 0x7ffaa0072440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.594+0000 7ffa9fe00640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa00cc0b0 0x7ffaa013dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:12:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.594+0000 7ffa9f400640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00cc490 0x7ffaa013f670 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c000c00 tx=0x7ffa8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.595+0000 7ffa97e00640 1 -- 172.21.15.19:0/275073193 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c040070 con 0x7ffaa00cc490 2024-03-28T13:12:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.595+0000 7ffa97e00640 1 -- 172.21.15.19:0/275073193 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa8c03a420 con 0x7ffaa00cc490 2024-03-28T13:12:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.595+0000 7ffa97e00640 1 -- 172.21.15.19:0/275073193 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c035040 con 0x7ffaa00cc490 2024-03-28T13:12:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.595+0000 7ffaa5a00640 1 -- 172.21.15.19:0/275073193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa00b1000 con 0x7ffaa00cc490 2024-03-28T13:12:41.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.595+0000 7ffaa5a00640 1 -- 172.21.15.19:0/275073193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa00b15c0 con 0x7ffaa00cc490 2024-03-28T13:12:41.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.596+0000 7ffaa5a00640 1 -- 172.21.15.19:0/275073193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa0075940 con 0x7ffaa00cc490 2024-03-28T13:12:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.597+0000 7ffa97e00640 1 -- 172.21.15.19:0/275073193 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ffa8c04b060 con 0x7ffaa00cc490 2024-03-28T13:12:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.598+0000 7ffa97e00640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffa8006c560 0x7ffa8006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.599+0000 7ffa97e00640 1 -- 172.21.15.19:0/275073193 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7ffa8c0ad330 con 0x7ffaa00cc490 2024-03-28T13:12:41.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.599+0000 7ffa9fe00640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffa8006c560 0x7ffa8006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:41.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.600+0000 7ffa9fe00640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffa8006c560 0x7ffa8006ea20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000a9d0 tx=0x7ffa90008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:41.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.603+0000 7ffa97e00640 1 -- 172.21.15.19:0/275073193 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ffa8c07bc50 con 0x7ffaa00cc490 2024-03-28T13:12:41.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.809+0000 7ffaa5a00640 1 -- 172.21.15.19:0/275073193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7ffaa0073830 con 0x7ffaa00cc490 2024-03-28T13:12:41.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.817+0000 7ffa97e00640 1 -- 172.21.15.19:0/275073193 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7ffa8c07b550 con 0x7ffaa00cc490 2024-03-28T13:12:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.819+0000 7ffaa5a00640 1 -- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffa8006c560 msgr2=0x7ffa8006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.819+0000 7ffaa5a00640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffa8006c560 0x7ffa8006ea20 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000a9d0 tx=0x7ffa90008040 comp rx=0 tx=0).stop 2024-03-28T13:12:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.819+0000 7ffaa5a00640 1 -- 172.21.15.19:0/275073193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00cc490 msgr2=0x7ffaa013f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.819+0000 7ffaa5a00640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00cc490 0x7ffaa013f670 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c000c00 tx=0x7ffa8c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:12:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.820+0000 7ffaa5a00640 1 -- 172.21.15.19:0/275073193 shutdown_connections 2024-03-28T13:12:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.820+0000 7ffaa5a00640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffa8006c560 0x7ffa8006ea20 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.820+0000 7ffaa5a00640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaa01f4e40 0x7ffaa0072440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.820+0000 7ffaa5a00640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffaa00cc490 0x7ffaa013f670 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.820+0000 7ffaa5a00640 1 --2- 172.21.15.19:0/275073193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaa00cc0b0 0x7ffaa013dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.820+0000 7ffaa5a00640 1 -- 172.21.15.19:0/275073193 >> 172.21.15.19:0/275073193 conn(0x7ffaa00ccad0 msgr2=0x7ffaa015dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.820+0000 7ffaa5a00640 1 -- 172.21.15.19:0/275073193 shutdown_connections 2024-03-28T13:12:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:41.820+0000 7ffaa5a00640 1 -- 172.21.15.19:0/275073193 wait complete. 2024-03-28T13:12:41.841 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-03-28T13:12:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.062+0000 7f3d7ec00640 1 Processor -- start 2024-03-28T13:12:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.062+0000 7f3d7ec00640 1 -- start start 2024-03-28T13:12:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.064+0000 7f3d7ec00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d780d0990 0x7f3d781670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.064+0000 7f3d7ec00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d78071d60 con 0x7f3d78162920 2024-03-28T13:12:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.064+0000 7f3d7ec00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d781665d0 con 0x7f3d78163ac0 2024-03-28T13:12:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.064+0000 7f3d7ec00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3d78163f50 con 0x7f3d780d0990 2024-03-28T13:12:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.064+0000 7f3d77e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3d78162920 0x7f3d780d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43102/0 (socket says 172.21.15.19:43102) 2024-03-28T13:12:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.064+0000 7f3d77e00640 1 -- 172.21.15.19:0/2961668762 learned_addr learned my addr 172.21.15.19:0/2961668762 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.065+0000 7f3d76a00640 1 -- 172.21.15.19:0/2961668762 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 924890360 0 0) 0x7f3d78071d60 con 0x7f3d78162920 2024-03-28T13:12:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.065+0000 7f3d76a00640 1 -- 172.21.15.19:0/2961668762 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d5c003520 con 0x7f3d78162920 2024-03-28T13:12:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.065+0000 7f3d77400640 1 --2- 172.21.15.19:0/2961668762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d780d0990 0x7f3d781670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.065+0000 7f3d76a00640 1 -- 172.21.15.19:0/2961668762 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 27518665 0 0) 0x7f3d5c003520 con 0x7f3d78162920 2024-03-28T13:12:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.065+0000 7f3d76a00640 1 -- 172.21.15.19:0/2961668762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d780d0990 msgr2=0x7f3d781670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.065+0000 7f3d76a00640 1 --2- 172.21.15.19:0/2961668762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d780d0990 0x7f3d781670b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.065+0000 7f3d76a00640 1 -- 172.21.15.19:0/2961668762 >> v1:172.21.15.188:6789/0 conn(0x7f3d78163ac0 legacy=0x7f3d780ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:42.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.065+0000 7f3d76a00640 1 -- 172.21.15.19:0/2961668762 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d781668b0 con 0x7f3d78162920 2024-03-28T13:12:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.065+0000 7f3d76a00640 1 -- 172.21.15.19:0/2961668762 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f3d680019f0 con 0x7f3d78162920 2024-03-28T13:12:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.065+0000 7f3d76a00640 1 -- 172.21.15.19:0/2961668762 >> v1:172.21.15.19:6789/0 conn(0x7f3d78162920 legacy=0x7f3d780d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.065+0000 7f3d76a00640 1 --2- 172.21.15.19:0/2961668762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d5c0039e0 0x7f3d5c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.066+0000 7f3d76a00640 1 --2- 172.21.15.19:0/2961668762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d5c004800 0x7f3d5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.066+0000 7f3d76a00640 1 --2- 172.21.15.19:0/2961668762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d5c0052e0 0x7f3d5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.066+0000 7f3d76a00640 1 -- 172.21.15.19:0/2961668762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c00c0e0 con 0x7f3d5c004800 2024-03-28T13:12:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.066+0000 7f3d76a00640 1 -- 172.21.15.19:0/2961668762 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c00c290 con 0x7f3d5c0039e0 2024-03-28T13:12:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.066+0000 7f3d76a00640 1 -- 172.21.15.19:0/2961668762 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3d5c00c490 con 0x7f3d5c0052e0 2024-03-28T13:12:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.066+0000 7f3d77400640 1 --2- 172.21.15.19:0/2961668762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d780d0990 0x7f3d781670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:12:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.066+0000 7f3d77400640 1 --2- 172.21.15.19:0/2961668762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d5c004800 0x7f3d5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.066+0000 7f3d77e00640 1 --2- 172.21.15.19:0/2961668762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d5c0039e0 0x7f3d5c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.066+0000 7f3d7cc00640 1 --2- 172.21.15.19:0/2961668762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d5c0052e0 0x7f3d5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.066+0000 7f3d77400640 1 -- 172.21.15.19:0/2961668762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d5c0052e0 msgr2=0x7f3d5c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:42.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.066+0000 7f3d77400640 1 --2- 172.21.15.19:0/2961668762 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d5c0052e0 0x7f3d5c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.066+0000 7f3d77400640 1 -- 172.21.15.19:0/2961668762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d5c0039e0 msgr2=0x7f3d5c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.066+0000 7f3d77400640 1 --2- 172.21.15.19:0/2961668762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d5c0039e0 0x7f3d5c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.067+0000 7f3d77400640 1 -- 172.21.15.19:0/2961668762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5c003520 con 0x7f3d5c004800 2024-03-28T13:12:42.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.067+0000 7f3d77e00640 1 --2- 172.21.15.19:0/2961668762 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d5c0039e0 0x7f3d5c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:12:42.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.068+0000 7f3d77400640 1 --2- 172.21.15.19:0/2961668762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d5c004800 0x7f3d5c004c80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3d60006fd0 tx=0x7f3d5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e1e0db315ed258 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:42.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.068+0000 7f3d76a00640 1 -- 172.21.15.19:0/2961668762 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6000f040 con 0x7f3d5c004800 2024-03-28T13:12:42.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.068+0000 7f3d76a00640 1 -- 172.21.15.19:0/2961668762 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d60033600 con 0x7f3d5c004800 2024-03-28T13:12:42.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.068+0000 7f3d76a00640 1 -- 172.21.15.19:0/2961668762 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d600339d0 con 0x7f3d5c004800 2024-03-28T13:12:42.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.069+0000 7f3d7ec00640 1 -- 172.21.15.19:0/2961668762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d5c004800 msgr2=0x7f3d5c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:42.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.069+0000 7f3d7ec00640 1 --2- 172.21.15.19:0/2961668762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d5c004800 0x7f3d5c004c80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3d60006fd0 tx=0x7f3d5c005830 comp rx=0 tx=0).stop 2024-03-28T13:12:42.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.069+0000 7f3d7cc00640 1 -- 172.21.15.19:0/2961668762 reap_dead start 2024-03-28T13:12:42.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.069+0000 7f3d7ec00640 1 -- 172.21.15.19:0/2961668762 shutdown_connections 2024-03-28T13:12:42.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.069+0000 7f3d7ec00640 1 -- 172.21.15.19:0/2961668762 >> 172.21.15.19:0/2961668762 conn(0x7f3d7806b750 msgr2=0x7f3d7806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:42.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.069+0000 7f3d7ec00640 1 -- 172.21.15.19:0/2961668762 shutdown_connections 2024-03-28T13:12:42.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.069+0000 7f3d7ec00640 1 -- 172.21.15.19:0/2961668762 wait complete. 2024-03-28T13:12:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.071+0000 7f3d7ec00640 1 Processor -- start 2024-03-28T13:12:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.071+0000 7f3d7ec00640 1 -- start start 2024-03-28T13:12:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.072+0000 7f3d7ec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d5c004800 0x7f3d78154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:42.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.072+0000 7f3d7ec00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d7809a190 0x7f3d7809a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.072+0000 7f3d7ec00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d781f88c0 0x7f3d781f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.072+0000 7f3d7ec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3d78072040 con 0x7f3d5c004800 2024-03-28T13:12:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.072+0000 7f3d7ec00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3d78162d00 con 0x7f3d7809a190 2024-03-28T13:12:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.072+0000 7f3d7ec00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3d780721c0 con 0x7f3d781f88c0 2024-03-28T13:12:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.072+0000 7f3d77e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d5c004800 0x7f3d78154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.072+0000 7f3d77e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d5c004800 0x7f3d78154590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42740/0 (socket says 172.21.15.19:42740) 2024-03-28T13:12:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.072+0000 7f3d77e00640 1 -- 172.21.15.19:0/120881918 learned_addr learned my addr 172.21.15.19:0/120881918 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.072+0000 7f3d77e00640 1 -- 172.21.15.19:0/120881918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d781f88c0 msgr2=0x7f3d781f8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.072+0000 7f3d77e00640 1 --2- 172.21.15.19:0/120881918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d781f88c0 0x7f3d781f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.072+0000 7f3d77e00640 1 -- 172.21.15.19:0/120881918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d7809a190 msgr2=0x7f3d7809a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:42.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.072+0000 7f3d77e00640 1 --2- 172.21.15.19:0/120881918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d7809a190 0x7f3d7809a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:42.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.072+0000 7f3d77e00640 1 -- 172.21.15.19:0/120881918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7809ad50 con 0x7f3d5c004800 2024-03-28T13:12:42.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.073+0000 7f3d77e00640 1 --2- 172.21.15.19:0/120881918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d5c004800 0x7f3d78154590 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3d68007b20 tx=0x7f3d6800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.073+0000 7f3d6fe00640 1 -- 172.21.15.19:0/120881918 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d68013420 con 0x7f3d5c004800 2024-03-28T13:12:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.073+0000 7f3d6fe00640 1 -- 172.21.15.19:0/120881918 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d68018070 con 0x7f3d5c004800 2024-03-28T13:12:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.073+0000 7f3d6fe00640 1 -- 172.21.15.19:0/120881918 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d680137e0 con 0x7f3d5c004800 2024-03-28T13:12:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.073+0000 7f3d7ec00640 1 -- 172.21.15.19:0/120881918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7809b030 con 0x7f3d5c004800 2024-03-28T13:12:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.073+0000 7f3d7ec00640 1 -- 172.21.15.19:0/120881918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d7809b5f0 con 0x7f3d5c004800 2024-03-28T13:12:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.074+0000 7f3d7ec00640 1 -- 172.21.15.19:0/120881918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d780685c0 con 0x7f3d5c004800 2024-03-28T13:12:42.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.075+0000 7f3d6fe00640 1 -- 172.21.15.19:0/120881918 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3d68013980 con 0x7f3d5c004800 2024-03-28T13:12:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.076+0000 7f3d6fe00640 1 --2- 172.21.15.19:0/120881918 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d5406c530 0x7f3d5406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:42.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.076+0000 7f3d6fe00640 1 -- 172.21.15.19:0/120881918 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7f3d6808bb50 con 0x7f3d5c004800 2024-03-28T13:12:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.077+0000 7f3d77400640 1 --2- 172.21.15.19:0/120881918 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d5406c530 0x7f3d5406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:42.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.078+0000 7f3d77400640 1 --2- 172.21.15.19:0/120881918 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d5406c530 0x7f3d5406e9f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3d60036000 tx=0x7f3d60030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:42.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.081+0000 7f3d6fe00640 1 -- 172.21.15.19:0/120881918 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3d6805a460 con 0x7f3d5c004800 2024-03-28T13:12:42.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:42.359+0000 7f3d7ec00640 1 -- 172.21.15.19:0/120881918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f3d7806ef60 con 0x7f3d5c004800 2024-03-28T13:12:43.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.363+0000 7f3d6fe00640 1 -- 172.21.15.19:0/120881918 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v178) ==== 149+0+0 (secure 0 0 0) 0x7f3d68059d60 con 0x7f3d5c004800 2024-03-28T13:12:43.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.367+0000 7f3d7ec00640 1 -- 172.21.15.19:0/120881918 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d5406c530 msgr2=0x7f3d5406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.367+0000 7f3d7ec00640 1 --2- 172.21.15.19:0/120881918 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d5406c530 0x7f3d5406e9f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3d60036000 tx=0x7f3d60030000 comp rx=0 tx=0).stop 2024-03-28T13:12:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.367+0000 7f3d7ec00640 1 -- 172.21.15.19:0/120881918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d5c004800 msgr2=0x7f3d78154590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.367+0000 7f3d7ec00640 1 --2- 172.21.15.19:0/120881918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d5c004800 0x7f3d78154590 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3d68007b20 tx=0x7f3d6800e530 comp rx=0 tx=0).stop 2024-03-28T13:12:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.367+0000 7f3d7ec00640 1 -- 172.21.15.19:0/120881918 shutdown_connections 2024-03-28T13:12:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.367+0000 7f3d7ec00640 1 --2- 172.21.15.19:0/120881918 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3d5406c530 0x7f3d5406e9f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.367+0000 7f3d7ec00640 1 --2- 172.21.15.19:0/120881918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3d781f88c0 0x7f3d781f8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.367+0000 7f3d7ec00640 1 --2- 172.21.15.19:0/120881918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3d7809a190 0x7f3d7809a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.367+0000 7f3d7ec00640 1 --2- 172.21.15.19:0/120881918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3d5c004800 0x7f3d78154590 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.367+0000 7f3d7ec00640 1 -- 172.21.15.19:0/120881918 >> 172.21.15.19:0/120881918 conn(0x7f3d7806b750 msgr2=0x7f3d7806d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.367+0000 7f3d7ec00640 1 -- 172.21.15.19:0/120881918 shutdown_connections 2024-03-28T13:12:43.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.368+0000 7f3d7ec00640 1 -- 172.21.15.19:0/120881918 wait complete. 2024-03-28T13:12:43.389 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-03-28T13:12:43.389 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-03-28T13:12:43.390 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-28T13:12:43.390 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:12:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.610+0000 7f4026600640 1 Processor -- start 2024-03-28T13:12:43.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.610+0000 7f4026600640 1 -- start start 2024-03-28T13:12:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.611+0000 7f4026600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f402015fc50 0x7f40200cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.611+0000 7f4026600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40200cc490 0x7f40200d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:43.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.611+0000 7f4026600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40200b3150 con 0x7f40200cc0b0 2024-03-28T13:12:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.611+0000 7f4026600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f402015c610 con 0x7f402015fc50 2024-03-28T13:12:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.611+0000 7f4026600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f40201600e0 con 0x7f40200cc490 2024-03-28T13:12:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.612+0000 7f4025200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f40200cc0b0 0x7f402006dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43104/0 (socket says 172.21.15.19:43104) 2024-03-28T13:12:43.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.612+0000 7f4025200640 1 -- 172.21.15.19:0/1898801075 learned_addr learned my addr 172.21.15.19:0/1898801075 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.612+0000 7f401fe00640 1 --2- 172.21.15.19:0/1898801075 >> v2:172.21.15.188:3300/0 conn(0x7f402015fc50 0x7f40200cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:43.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.612+0000 7f4025c00640 1 --2- 172.21.15.19:0/1898801075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40200cc490 0x7f40200d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.612+0000 7f401f400640 1 -- 172.21.15.19:0/1898801075 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3293077685 0 0) 0x7f40200b3150 con 0x7f40200cc0b0 2024-03-28T13:12:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.612+0000 7f401f400640 1 -- 172.21.15.19:0/1898801075 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4008003520 con 0x7f40200cc0b0 2024-03-28T13:12:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.612+0000 7f401f400640 1 -- 172.21.15.19:0/1898801075 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1255593634 0 0) 0x7f4008003520 con 0x7f40200cc0b0 2024-03-28T13:12:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.612+0000 7f401f400640 1 -- 172.21.15.19:0/1898801075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40200cc490 msgr2=0x7f40200d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.612+0000 7f401f400640 1 --2- 172.21.15.19:0/1898801075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40200cc490 0x7f40200d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f401f400640 1 -- 172.21.15.19:0/1898801075 >> v2:172.21.15.188:3300/0 conn(0x7f402015fc50 msgr2=0x7f40200cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f401f400640 1 --2- 172.21.15.19:0/1898801075 >> v2:172.21.15.188:3300/0 conn(0x7f402015fc50 0x7f40200cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:43.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f401f400640 1 -- 172.21.15.19:0/1898801075 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40201603c0 con 0x7f40200cc0b0 2024-03-28T13:12:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f401f400640 1 -- 172.21.15.19:0/1898801075 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f4014001670 con 0x7f40200cc0b0 2024-03-28T13:12:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f4025c00640 1 --2- 172.21.15.19:0/1898801075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40200cc490 0x7f40200d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:12:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f401f400640 1 -- 172.21.15.19:0/1898801075 >> v1:172.21.15.19:6789/0 conn(0x7f40200cc0b0 legacy=0x7f402006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f401f400640 1 --2- 172.21.15.19:0/1898801075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40080039b0 0x7f4008003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f401f400640 1 --2- 172.21.15.19:0/1898801075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4008004800 0x7f4008004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f401f400640 1 --2- 172.21.15.19:0/1898801075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40200cc490 0x7f40080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f401f400640 1 -- 172.21.15.19:0/1898801075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f402015c610 con 0x7f40080039b0 2024-03-28T13:12:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f401f400640 1 -- 172.21.15.19:0/1898801075 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f40201600e0 con 0x7f4008004800 2024-03-28T13:12:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f401f400640 1 -- 172.21.15.19:0/1898801075 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f400800b9c0 con 0x7f40200cc490 2024-03-28T13:12:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f4025200640 1 --2- 172.21.15.19:0/1898801075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40080039b0 0x7f4008003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f401fe00640 1 --2- 172.21.15.19:0/1898801075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4008004800 0x7f4008004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f4025200640 1 -- 172.21.15.19:0/1898801075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40200cc490 msgr2=0x7f40080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f4025200640 1 --2- 172.21.15.19:0/1898801075 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40200cc490 0x7f40080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f4025200640 1 -- 172.21.15.19:0/1898801075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4008004800 msgr2=0x7f4008004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:43.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f4025200640 1 --2- 172.21.15.19:0/1898801075 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4008004800 0x7f4008004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:43.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.613+0000 7f4025200640 1 -- 172.21.15.19:0/1898801075 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4008003520 con 0x7f40080039b0 2024-03-28T13:12:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.615+0000 7f4025200640 1 --2- 172.21.15.19:0/1898801075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40080039b0 0x7f4008003dd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f401400c270 tx=0x7f4008005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d1835ebacd0140 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.615+0000 7f401f400640 1 -- 172.21.15.19:0/1898801075 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014007d30 con 0x7f40080039b0 2024-03-28T13:12:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.615+0000 7f401f400640 1 -- 172.21.15.19:0/1898801075 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4014030070 con 0x7f40080039b0 2024-03-28T13:12:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.615+0000 7f401f400640 1 -- 172.21.15.19:0/1898801075 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401402cd60 con 0x7f40080039b0 2024-03-28T13:12:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.615+0000 7f4026600640 1 -- 172.21.15.19:0/1898801075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40080039b0 msgr2=0x7f4008003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.615+0000 7f4026600640 1 --2- 172.21.15.19:0/1898801075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40080039b0 0x7f4008003dd0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f401400c270 tx=0x7f4008005830 comp rx=0 tx=0).stop 2024-03-28T13:12:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.615+0000 7f4025c00640 1 -- 172.21.15.19:0/1898801075 reap_dead start 2024-03-28T13:12:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.616+0000 7f4026600640 1 -- 172.21.15.19:0/1898801075 shutdown_connections 2024-03-28T13:12:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.616+0000 7f4026600640 1 -- 172.21.15.19:0/1898801075 >> 172.21.15.19:0/1898801075 conn(0x7f40200ccad0 msgr2=0x7f40200cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.616+0000 7f4026600640 1 -- 172.21.15.19:0/1898801075 shutdown_connections 2024-03-28T13:12:43.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.616+0000 7f4026600640 1 -- 172.21.15.19:0/1898801075 wait complete. 2024-03-28T13:12:43.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.617+0000 7f4026600640 1 Processor -- start 2024-03-28T13:12:43.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.618+0000 7f4026600640 1 -- start start 2024-03-28T13:12:43.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.619+0000 7f4026600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40080039b0 0x7f402015f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:43.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.619+0000 7f4026600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402013b3e0 0x7f4020152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:43.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.619+0000 7f4026600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40201f43c0 0x7f40200723f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:43.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.619+0000 7f4026600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40200d0300 con 0x7f40080039b0 2024-03-28T13:12:43.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.619+0000 7f4026600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4020160980 con 0x7f40201f43c0 2024-03-28T13:12:43.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.619+0000 7f4026600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f40200d0480 con 0x7f402013b3e0 2024-03-28T13:12:43.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.619+0000 7f4025200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40080039b0 0x7f402015f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:43.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.619+0000 7f4025200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40080039b0 0x7f402015f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42752/0 (socket says 172.21.15.19:42752) 2024-03-28T13:12:43.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.619+0000 7f4025200640 1 -- 172.21.15.19:0/3573983628 learned_addr learned my addr 172.21.15.19:0/3573983628 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:43.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.619+0000 7f401fe00640 1 --2- 172.21.15.19:0/3573983628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402013b3e0 0x7f4020152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:43.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.619+0000 7f4025c00640 1 --2- 172.21.15.19:0/3573983628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40201f43c0 0x7f40200723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:43.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.619+0000 7f4025200640 1 -- 172.21.15.19:0/3573983628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402013b3e0 msgr2=0x7f4020152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:43.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.619+0000 7f4025200640 1 --2- 172.21.15.19:0/3573983628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402013b3e0 0x7f4020152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:43.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.619+0000 7f4025200640 1 -- 172.21.15.19:0/3573983628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40201f43c0 msgr2=0x7f40200723f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:43.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.619+0000 7f4025200640 1 --2- 172.21.15.19:0/3573983628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40201f43c0 0x7f40200723f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:43.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.620+0000 7f4025200640 1 -- 172.21.15.19:0/3573983628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402015f770 con 0x7f40080039b0 2024-03-28T13:12:43.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.620+0000 7f4025c00640 1 --2- 172.21.15.19:0/3573983628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40201f43c0 0x7f40200723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:43.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.620+0000 7f4025200640 1 --2- 172.21.15.19:0/3573983628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40080039b0 0x7f402015f230 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f401400c270 tx=0x7f4020160590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:43.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.620+0000 7f401cc00640 1 -- 172.21.15.19:0/3573983628 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014045070 con 0x7f40080039b0 2024-03-28T13:12:43.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.620+0000 7f401cc00640 1 -- 172.21.15.19:0/3573983628 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4014004c10 con 0x7f40080039b0 2024-03-28T13:12:43.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.620+0000 7f401cc00640 1 -- 172.21.15.19:0/3573983628 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4014007070 con 0x7f40080039b0 2024-03-28T13:12:43.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.620+0000 7f4026600640 1 -- 172.21.15.19:0/3573983628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402015ee10 con 0x7f40080039b0 2024-03-28T13:12:43.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.620+0000 7f4026600640 1 -- 172.21.15.19:0/3573983628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402013b8e0 con 0x7f40080039b0 2024-03-28T13:12:43.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.622+0000 7f4026600640 1 -- 172.21.15.19:0/3573983628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4020145840 con 0x7f40080039b0 2024-03-28T13:12:43.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.622+0000 7f401cc00640 1 -- 172.21.15.19:0/3573983628 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4014007210 con 0x7f40080039b0 2024-03-28T13:12:43.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.623+0000 7f401cc00640 1 --2- 172.21.15.19:0/3573983628 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f400006c5b0 0x7f400006ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:43.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.623+0000 7f401cc00640 1 -- 172.21.15.19:0/3573983628 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7f40140ac570 con 0x7f40080039b0 2024-03-28T13:12:43.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.624+0000 7f401fe00640 1 --2- 172.21.15.19:0/3573983628 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f400006c5b0 0x7f400006ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:43.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.625+0000 7f401fe00640 1 --2- 172.21.15.19:0/3573983628 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f400006c5b0 0x7f400006ea70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f40200b31b0 tx=0x7f400c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:43.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.630+0000 7f401cc00640 1 -- 172.21.15.19:0/3573983628 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4020145840 con 0x7f40080039b0 2024-03-28T13:12:43.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.834+0000 7f4026600640 1 -- 172.21.15.19:0/3573983628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4020075940 con 0x7f40080039b0 2024-03-28T13:12:43.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.836+0000 7f401cc00640 1 -- 172.21.15.19:0/3573983628 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+13018 (secure 0 0 0) 0x7f401407ae70 con 0x7f40080039b0 2024-03-28T13:12:43.838 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:43.838 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":54,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:12:41.334108+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/179338886":"2024-03-29T13:12:37.866571+0000","172.21.15.19:6833/179338886":"2024-03-29T13:12:37.866571+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:12:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.839+0000 7f4026600640 1 -- 172.21.15.19:0/3573983628 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f400006c5b0 msgr2=0x7f400006ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.839+0000 7f4026600640 1 --2- 172.21.15.19:0/3573983628 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f400006c5b0 0x7f400006ea70 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f40200b31b0 tx=0x7f400c008040 comp rx=0 tx=0).stop 2024-03-28T13:12:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.839+0000 7f4026600640 1 -- 172.21.15.19:0/3573983628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40080039b0 msgr2=0x7f402015f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.839+0000 7f4026600640 1 --2- 172.21.15.19:0/3573983628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40080039b0 0x7f402015f230 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f401400c270 tx=0x7f4020160590 comp rx=0 tx=0).stop 2024-03-28T13:12:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.839+0000 7f4026600640 1 -- 172.21.15.19:0/3573983628 shutdown_connections 2024-03-28T13:12:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.839+0000 7f4026600640 1 --2- 172.21.15.19:0/3573983628 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f400006c5b0 0x7f400006ea70 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.839+0000 7f4026600640 1 --2- 172.21.15.19:0/3573983628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40201f43c0 0x7f40200723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.839+0000 7f4026600640 1 --2- 172.21.15.19:0/3573983628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f402013b3e0 0x7f4020152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.839+0000 7f4026600640 1 --2- 172.21.15.19:0/3573983628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40080039b0 0x7f402015f230 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.839+0000 7f4026600640 1 -- 172.21.15.19:0/3573983628 >> 172.21.15.19:0/3573983628 conn(0x7f40200ccad0 msgr2=0x7f40200ad890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.839+0000 7f4026600640 1 -- 172.21.15.19:0/3573983628 shutdown_connections 2024-03-28T13:12:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:43.839+0000 7f4026600640 1 -- 172.21.15.19:0/3573983628 wait complete. 2024-03-28T13:12:43.861 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:12:43.862 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:12:44.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.084+0000 7fe451000640 1 Processor -- start 2024-03-28T13:12:44.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.084+0000 7fe451000640 1 -- start start 2024-03-28T13:12:44.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.086+0000 7fe451000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe44c13f190 0x7fe44c0cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:44.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.086+0000 7fe451000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe44c06dd30 0x7fe44c0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:44.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.086+0000 7fe451000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe44c0ce890 con 0x7fe44c0cc880 2024-03-28T13:12:44.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.086+0000 7fe451000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe44c0cd200 con 0x7fe44c06dd30 2024-03-28T13:12:44.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.086+0000 7fe451000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe44c0ceb50 con 0x7fe44c13f190 2024-03-28T13:12:44.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.086+0000 7fe44be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe44c06dd30 0x7fe44c0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:44.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.086+0000 7fe443e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe44c0cc880 0x7fe44c06d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43118/0 (socket says 172.21.15.19:43118) 2024-03-28T13:12:44.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.086+0000 7fe443e00640 1 -- 172.21.15.19:0/2812252734 learned_addr learned my addr 172.21.15.19:0/2812252734 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:44.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.086+0000 7fe44b400640 1 --2- 172.21.15.19:0/2812252734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe44c13f190 0x7fe44c0cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:44.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.087+0000 7fe44aa00640 1 -- 172.21.15.19:0/2812252734 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 91600595 0 0) 0x7fe44c0ce890 con 0x7fe44c0cc880 2024-03-28T13:12:44.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.087+0000 7fe44aa00640 1 -- 172.21.15.19:0/2812252734 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe42c003520 con 0x7fe44c0cc880 2024-03-28T13:12:44.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.087+0000 7fe44be00640 1 -- 172.21.15.19:0/2812252734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe44c13f190 msgr2=0x7fe44c0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:44.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.087+0000 7fe44be00640 1 --2- 172.21.15.19:0/2812252734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe44c13f190 0x7fe44c0cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.087+0000 7fe44be00640 1 -- 172.21.15.19:0/2812252734 >> v1:172.21.15.19:6789/0 conn(0x7fe44c0cc880 legacy=0x7fe44c06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:44.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.087+0000 7fe44be00640 1 -- 172.21.15.19:0/2812252734 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c0cee30 con 0x7fe44c06dd30 2024-03-28T13:12:44.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.088+0000 7fe44be00640 1 --2- 172.21.15.19:0/2812252734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe44c06dd30 0x7fe44c0cccc0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe444009d90 tx=0x7fe44402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f4e442f399a429 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:44.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.089+0000 7fe44aa00640 1 -- 172.21.15.19:0/2812252734 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444039070 con 0x7fe44c06dd30 2024-03-28T13:12:44.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.089+0000 7fe44aa00640 1 -- 172.21.15.19:0/2812252734 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe44402cc50 con 0x7fe44c06dd30 2024-03-28T13:12:44.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.089+0000 7fe44aa00640 1 -- 172.21.15.19:0/2812252734 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe444034440 con 0x7fe44c06dd30 2024-03-28T13:12:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.089+0000 7fe451000640 1 -- 172.21.15.19:0/2812252734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe44c06dd30 msgr2=0x7fe44c0cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.089+0000 7fe451000640 1 --2- 172.21.15.19:0/2812252734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe44c06dd30 0x7fe44c0cccc0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe444009d90 tx=0x7fe44402bea0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.089+0000 7fe451000640 1 -- 172.21.15.19:0/2812252734 shutdown_connections 2024-03-28T13:12:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.089+0000 7fe451000640 1 --2- 172.21.15.19:0/2812252734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe44c06dd30 0x7fe44c0cccc0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.089+0000 7fe451000640 1 --2- 172.21.15.19:0/2812252734 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe44c13f190 0x7fe44c0cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.089+0000 7fe451000640 1 -- 172.21.15.19:0/2812252734 >> 172.21.15.19:0/2812252734 conn(0x7fe44c0b1ba0 msgr2=0x7fe44c0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.090+0000 7fe451000640 1 -- 172.21.15.19:0/2812252734 shutdown_connections 2024-03-28T13:12:44.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.090+0000 7fe451000640 1 -- 172.21.15.19:0/2812252734 wait complete. 2024-03-28T13:12:44.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.091+0000 7fe451000640 1 Processor -- start 2024-03-28T13:12:44.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.091+0000 7fe451000640 1 -- start start 2024-03-28T13:12:44.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.092+0000 7fe451000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe44c06dd30 0x7fe44c1f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.092+0000 7fe451000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe44c0cc880 0x7fe44c0aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.092+0000 7fe451000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe44c13f190 0x7fe44c135790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.092+0000 7fe451000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c13f570 con 0x7fe44c0cc880 2024-03-28T13:12:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.092+0000 7fe451000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe44c160c70 con 0x7fe44c13f190 2024-03-28T13:12:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.092+0000 7fe451000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe44c0cfb70 con 0x7fe44c06dd30 2024-03-28T13:12:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.093+0000 7fe443e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe44c0cc880 0x7fe44c0aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.093+0000 7fe443e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe44c0cc880 0x7fe44c0aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42776/0 (socket says 172.21.15.19:42776) 2024-03-28T13:12:44.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.093+0000 7fe443e00640 1 -- 172.21.15.19:0/858345493 learned_addr learned my addr 172.21.15.19:0/858345493 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:44.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.093+0000 7fe44b400640 1 --2- 172.21.15.19:0/858345493 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe44c06dd30 0x7fe44c1f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:44.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.093+0000 7fe443e00640 1 -- 172.21.15.19:0/858345493 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe44c06dd30 msgr2=0x7fe44c1f4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:44.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.093+0000 7fe44be00640 1 --2- 172.21.15.19:0/858345493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe44c13f190 0x7fe44c135790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:44.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.093+0000 7fe443e00640 1 --2- 172.21.15.19:0/858345493 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe44c06dd30 0x7fe44c1f4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.093+0000 7fe443e00640 1 -- 172.21.15.19:0/858345493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe44c13f190 msgr2=0x7fe44c135790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:44.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.093+0000 7fe443e00640 1 --2- 172.21.15.19:0/858345493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe44c13f190 0x7fe44c135790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.093+0000 7fe443e00640 1 -- 172.21.15.19:0/858345493 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe44c0a9860 con 0x7fe44c0cc880 2024-03-28T13:12:44.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.093+0000 7fe443e00640 1 --2- 172.21.15.19:0/858345493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe44c0cc880 0x7fe44c0aeac0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe43400c460 tx=0x7fe43400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:44.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.094+0000 7fe443400640 1 -- 172.21.15.19:0/858345493 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434017070 con 0x7fe44c0cc880 2024-03-28T13:12:44.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.094+0000 7fe443400640 1 -- 172.21.15.19:0/858345493 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe434005ce0 con 0x7fe44c0cc880 2024-03-28T13:12:44.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.094+0000 7fe443400640 1 -- 172.21.15.19:0/858345493 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe434007290 con 0x7fe44c0cc880 2024-03-28T13:12:44.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.094+0000 7fe451000640 1 -- 172.21.15.19:0/858345493 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe44c0a9b40 con 0x7fe44c0cc880 2024-03-28T13:12:44.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.094+0000 7fe451000640 1 -- 172.21.15.19:0/858345493 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44c0aa100 con 0x7fe44c0cc880 2024-03-28T13:12:44.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.095+0000 7fe451000640 1 -- 172.21.15.19:0/858345493 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44c075940 con 0x7fe44c0cc880 2024-03-28T13:12:44.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.096+0000 7fe443400640 1 -- 172.21.15.19:0/858345493 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe43401a070 con 0x7fe44c0cc880 2024-03-28T13:12:44.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.097+0000 7fe443400640 1 --2- 172.21.15.19:0/858345493 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe42806c530 0x7fe42806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:44.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.097+0000 7fe443400640 1 -- 172.21.15.19:0/858345493 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7fe43408a900 con 0x7fe44c0cc880 2024-03-28T13:12:44.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.098+0000 7fe44b400640 1 --2- 172.21.15.19:0/858345493 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe42806c530 0x7fe42806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:44.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.099+0000 7fe44b400640 1 --2- 172.21.15.19:0/858345493 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe42806c530 0x7fe42806e9f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe43c006fd0 tx=0x7fe43c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:44.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.101+0000 7fe443400640 1 -- 172.21.15.19:0/858345493 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe4340590f0 con 0x7fe44c0cc880 2024-03-28T13:12:44.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.306+0000 7fe451000640 1 -- 172.21.15.19:0/858345493 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe44c0aa2d0 con 0x7fe44c0cc880 2024-03-28T13:12:44.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.307+0000 7fe443400640 1 -- 172.21.15.19:0/858345493 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) ==== 93+0+10 (secure 0 0 0) 0x7fe4340589f0 con 0x7fe44c0cc880 2024-03-28T13:12:44.307 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:12:44.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.309+0000 7fe451000640 1 -- 172.21.15.19:0/858345493 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe42806c530 msgr2=0x7fe42806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:44.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.309+0000 7fe451000640 1 --2- 172.21.15.19:0/858345493 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe42806c530 0x7fe42806e9f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe43c006fd0 tx=0x7fe43c008040 comp rx=0 tx=0).stop 2024-03-28T13:12:44.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.309+0000 7fe451000640 1 -- 172.21.15.19:0/858345493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe44c0cc880 msgr2=0x7fe44c0aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:44.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.309+0000 7fe451000640 1 --2- 172.21.15.19:0/858345493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe44c0cc880 0x7fe44c0aeac0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe43400c460 tx=0x7fe43400c920 comp rx=0 tx=0).stop 2024-03-28T13:12:44.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.309+0000 7fe451000640 1 -- 172.21.15.19:0/858345493 shutdown_connections 2024-03-28T13:12:44.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.309+0000 7fe451000640 1 --2- 172.21.15.19:0/858345493 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe42806c530 0x7fe42806e9f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.309+0000 7fe451000640 1 --2- 172.21.15.19:0/858345493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe44c13f190 0x7fe44c135790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.309+0000 7fe451000640 1 --2- 172.21.15.19:0/858345493 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe44c0cc880 0x7fe44c0aeac0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.309+0000 7fe451000640 1 --2- 172.21.15.19:0/858345493 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe44c06dd30 0x7fe44c1f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.309+0000 7fe451000640 1 -- 172.21.15.19:0/858345493 >> 172.21.15.19:0/858345493 conn(0x7fe44c0b1ba0 msgr2=0x7fe44c15e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:44.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.310+0000 7fe451000640 1 -- 172.21.15.19:0/858345493 shutdown_connections 2024-03-28T13:12:44.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.310+0000 7fe451000640 1 -- 172.21.15.19:0/858345493 wait complete. 2024-03-28T13:12:44.330 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-03-28T13:12:44.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.548+0000 7fef74a00640 1 Processor -- start 2024-03-28T13:12:44.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.548+0000 7fef74a00640 1 -- start start 2024-03-28T13:12:44.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.549+0000 7fef74a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fef70155550 0x7fef70155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:44.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.550+0000 7fef74a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef700d0990 0x7fef700aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:44.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.550+0000 7fef74a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef700b40b0 con 0x7fef7015f800 2024-03-28T13:12:44.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.550+0000 7fef74a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fef700b4390 con 0x7fef70155550 2024-03-28T13:12:44.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.550+0000 7fef74a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fef700b4670 con 0x7fef700d0990 2024-03-28T13:12:44.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.550+0000 7fef6d400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fef7015f800 0x7fef700d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43134/0 (socket says 172.21.15.19:43134) 2024-03-28T13:12:44.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.550+0000 7fef6d400640 1 -- 172.21.15.19:0/1994467135 learned_addr learned my addr 172.21.15.19:0/1994467135 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:44.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.550+0000 7fef6de00640 1 --2- 172.21.15.19:0/1994467135 >> v2:172.21.15.19:3300/0 conn(0x7fef70155550 0x7fef70155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:44.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.550+0000 7fef6e800640 1 --2- 172.21.15.19:0/1994467135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef700d0990 0x7fef700aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:44.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.551+0000 7fef6ca00640 1 -- 172.21.15.19:0/1994467135 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1614186091 0 0) 0x7fef700b40b0 con 0x7fef7015f800 2024-03-28T13:12:44.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.551+0000 7fef6ca00640 1 -- 172.21.15.19:0/1994467135 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef54003520 con 0x7fef7015f800 2024-03-28T13:12:44.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.551+0000 7fef6de00640 1 -- 172.21.15.19:0/1994467135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef700d0990 msgr2=0x7fef700aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:44.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.551+0000 7fef6de00640 1 --2- 172.21.15.19:0/1994467135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef700d0990 0x7fef700aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.551+0000 7fef6de00640 1 -- 172.21.15.19:0/1994467135 >> v1:172.21.15.19:6789/0 conn(0x7fef7015f800 legacy=0x7fef700d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:44.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.551+0000 7fef6de00640 1 -- 172.21.15.19:0/1994467135 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef700b4950 con 0x7fef70155550 2024-03-28T13:12:44.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.552+0000 7fef6de00640 1 --2- 172.21.15.19:0/1994467135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef70155550 0x7fef70155930 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fef600097a0 tx=0x7fef6002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4044b2263b063981 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:44.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.552+0000 7fef6ca00640 1 -- 172.21.15.19:0/1994467135 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60035020 con 0x7fef70155550 2024-03-28T13:12:44.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.552+0000 7fef6ca00640 1 -- 172.21.15.19:0/1994467135 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef60031810 con 0x7fef70155550 2024-03-28T13:12:44.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.552+0000 7fef6ca00640 1 -- 172.21.15.19:0/1994467135 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60031be0 con 0x7fef70155550 2024-03-28T13:12:44.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.553+0000 7fef74a00640 1 -- 172.21.15.19:0/1994467135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef70155550 msgr2=0x7fef70155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:44.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.553+0000 7fef74a00640 1 --2- 172.21.15.19:0/1994467135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef70155550 0x7fef70155930 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fef600097a0 tx=0x7fef6002b8b0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.553+0000 7fef74a00640 1 -- 172.21.15.19:0/1994467135 shutdown_connections 2024-03-28T13:12:44.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.553+0000 7fef74a00640 1 --2- 172.21.15.19:0/1994467135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef700d0990 0x7fef700aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.553+0000 7fef74a00640 1 --2- 172.21.15.19:0/1994467135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef70155550 0x7fef70155930 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.553+0000 7fef74a00640 1 -- 172.21.15.19:0/1994467135 >> 172.21.15.19:0/1994467135 conn(0x7fef7006b750 msgr2=0x7fef7006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:44.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.553+0000 7fef74a00640 1 -- 172.21.15.19:0/1994467135 shutdown_connections 2024-03-28T13:12:44.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.553+0000 7fef74a00640 1 -- 172.21.15.19:0/1994467135 wait complete. 2024-03-28T13:12:44.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.555+0000 7fef74a00640 1 Processor -- start 2024-03-28T13:12:44.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.555+0000 7fef74a00640 1 -- start start 2024-03-28T13:12:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.556+0000 7fef74a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef700d0990 0x7fef7015e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.556+0000 7fef74a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef70155550 0x7fef700cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.556+0000 7fef74a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef7015f800 0x7fef7015a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.556+0000 7fef74a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fef7015ba20 con 0x7fef7015f800 2024-03-28T13:12:44.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.556+0000 7fef74a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fef7015fc80 con 0x7fef700d0990 2024-03-28T13:12:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.556+0000 7fef74a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fef7015bba0 con 0x7fef70155550 2024-03-28T13:12:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.556+0000 7fef6e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef7015f800 0x7fef7015a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.556+0000 7fef6e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef7015f800 0x7fef7015a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42798/0 (socket says 172.21.15.19:42798) 2024-03-28T13:12:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.556+0000 7fef6e800640 1 -- 172.21.15.19:0/1894647237 learned_addr learned my addr 172.21.15.19:0/1894647237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.556+0000 7fef6d400640 1 --2- 172.21.15.19:0/1894647237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef70155550 0x7fef700cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.556+0000 7fef6de00640 1 --2- 172.21.15.19:0/1894647237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef700d0990 0x7fef7015e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.557+0000 7fef6e800640 1 -- 172.21.15.19:0/1894647237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef70155550 msgr2=0x7fef700cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.557+0000 7fef6e800640 1 --2- 172.21.15.19:0/1894647237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef70155550 0x7fef700cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.557+0000 7fef6e800640 1 -- 172.21.15.19:0/1894647237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef700d0990 msgr2=0x7fef7015e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.557+0000 7fef6e800640 1 --2- 172.21.15.19:0/1894647237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef700d0990 0x7fef7015e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.557+0000 7fef6e800640 1 -- 172.21.15.19:0/1894647237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7015d830 con 0x7fef7015f800 2024-03-28T13:12:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.557+0000 7fef6de00640 1 --2- 172.21.15.19:0/1894647237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef700d0990 0x7fef7015e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.557+0000 7fef6e800640 1 --2- 172.21.15.19:0/1894647237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef7015f800 0x7fef7015a2d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fef6800efc0 tx=0x7fef6800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.557+0000 7fef66000640 1 -- 172.21.15.19:0/1894647237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68018020 con 0x7fef7015f800 2024-03-28T13:12:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.557+0000 7fef66000640 1 -- 172.21.15.19:0/1894647237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef68014480 con 0x7fef7015f800 2024-03-28T13:12:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.557+0000 7fef74a00640 1 -- 172.21.15.19:0/1894647237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7015dd00 con 0x7fef7015f800 2024-03-28T13:12:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.557+0000 7fef66000640 1 -- 172.21.15.19:0/1894647237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68009e60 con 0x7fef7015f800 2024-03-28T13:12:44.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.557+0000 7fef74a00640 1 -- 172.21.15.19:0/1894647237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef700cf3b0 con 0x7fef7015f800 2024-03-28T13:12:44.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.558+0000 7fef74a00640 1 -- 172.21.15.19:0/1894647237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef700685c0 con 0x7fef7015f800 2024-03-28T13:12:44.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.559+0000 7fef66000640 1 -- 172.21.15.19:0/1894647237 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fef68005d10 con 0x7fef7015f800 2024-03-28T13:12:44.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.560+0000 7fef66000640 1 --2- 172.21.15.19:0/1894647237 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fef5006c560 0x7fef5006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:44.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.561+0000 7fef66000640 1 -- 172.21.15.19:0/1894647237 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7fef6808c0b0 con 0x7fef7015f800 2024-03-28T13:12:44.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.561+0000 7fef6de00640 1 --2- 172.21.15.19:0/1894647237 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fef5006c560 0x7fef5006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:44.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.562+0000 7fef6de00640 1 --2- 172.21.15.19:0/1894647237 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fef5006c560 0x7fef5006ea20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fef60036000 tx=0x7fef60008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:44.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.565+0000 7fef66000640 1 -- 172.21.15.19:0/1894647237 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fef68022030 con 0x7fef7015f800 2024-03-28T13:12:44.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:44.851+0000 7fef74a00640 1 -- 172.21.15.19:0/1894647237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7fef700b3ad0 con 0x7fef7015f800 2024-03-28T13:12:45.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.377+0000 7fef66000640 1 -- 172.21.15.19:0/1894647237 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v180) ==== 172+0+0 (secure 0 0 0) 0x7fef68010040 con 0x7fef7015f800 2024-03-28T13:12:45.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.381+0000 7fef74a00640 1 -- 172.21.15.19:0/1894647237 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fef5006c560 msgr2=0x7fef5006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:45.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.381+0000 7fef74a00640 1 --2- 172.21.15.19:0/1894647237 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fef5006c560 0x7fef5006ea20 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fef60036000 tx=0x7fef60008030 comp rx=0 tx=0).stop 2024-03-28T13:12:45.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.381+0000 7fef74a00640 1 -- 172.21.15.19:0/1894647237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef7015f800 msgr2=0x7fef7015a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:45.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.381+0000 7fef74a00640 1 --2- 172.21.15.19:0/1894647237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef7015f800 0x7fef7015a2d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fef6800efc0 tx=0x7fef6800c490 comp rx=0 tx=0).stop 2024-03-28T13:12:45.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.381+0000 7fef74a00640 1 -- 172.21.15.19:0/1894647237 shutdown_connections 2024-03-28T13:12:45.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.381+0000 7fef74a00640 1 --2- 172.21.15.19:0/1894647237 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fef5006c560 0x7fef5006ea20 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:45.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.381+0000 7fef74a00640 1 --2- 172.21.15.19:0/1894647237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef7015f800 0x7fef7015a2d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:45.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.381+0000 7fef74a00640 1 --2- 172.21.15.19:0/1894647237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef70155550 0x7fef700cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:45.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.381+0000 7fef74a00640 1 --2- 172.21.15.19:0/1894647237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef700d0990 0x7fef7015e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:45.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.381+0000 7fef74a00640 1 -- 172.21.15.19:0/1894647237 >> 172.21.15.19:0/1894647237 conn(0x7fef7006b750 msgr2=0x7fef700cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:45.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.382+0000 7fef74a00640 1 -- 172.21.15.19:0/1894647237 shutdown_connections 2024-03-28T13:12:45.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.382+0000 7fef74a00640 1 -- 172.21.15.19:0/1894647237 wait complete. 2024-03-28T13:12:45.403 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-28T13:12:45.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.628+0000 7fcd49200640 1 Processor -- start 2024-03-28T13:12:45.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.628+0000 7fcd49200640 1 -- start start 2024-03-28T13:12:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.629+0000 7fcd49200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcd44155930 0x7fcd440d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.629+0000 7fcd49200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd440d07c0 0x7fcd440aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.629+0000 7fcd49200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd440b40b0 con 0x7fcd44155550 2024-03-28T13:12:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.629+0000 7fcd49200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcd440d0c30 con 0x7fcd44155930 2024-03-28T13:12:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.629+0000 7fcd49200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcd440b4390 con 0x7fcd440d07c0 2024-03-28T13:12:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.630+0000 7fcd41e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fcd44155550 0x7fcd4415f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44736/0 (socket says 172.21.15.19:44736) 2024-03-28T13:12:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.630+0000 7fcd41e00640 1 -- 172.21.15.19:0/2683648307 learned_addr learned my addr 172.21.15.19:0/2683648307 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.630+0000 7fcd41400640 1 --2- 172.21.15.19:0/2683648307 >> v2:172.21.15.19:3300/0 conn(0x7fcd44155930 0x7fcd440d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:45.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.630+0000 7fcd42800640 1 --2- 172.21.15.19:0/2683648307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd440d07c0 0x7fcd440aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:45.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.630+0000 7fcd40a00640 1 -- 172.21.15.19:0/2683648307 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1177179282 0 0) 0x7fcd440b40b0 con 0x7fcd44155550 2024-03-28T13:12:45.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.630+0000 7fcd40a00640 1 -- 172.21.15.19:0/2683648307 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd28003520 con 0x7fcd44155550 2024-03-28T13:12:45.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.630+0000 7fcd41400640 1 -- 172.21.15.19:0/2683648307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd440d07c0 msgr2=0x7fcd440aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:45.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.630+0000 7fcd41400640 1 --2- 172.21.15.19:0/2683648307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd440d07c0 0x7fcd440aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:45.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.630+0000 7fcd41400640 1 -- 172.21.15.19:0/2683648307 >> v1:172.21.15.19:6789/0 conn(0x7fcd44155550 legacy=0x7fcd4415f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:45.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.630+0000 7fcd41400640 1 -- 172.21.15.19:0/2683648307 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd440b4670 con 0x7fcd44155930 2024-03-28T13:12:45.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.630+0000 7fcd42800640 1 --2- 172.21.15.19:0/2683648307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd440d07c0 0x7fcd440aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:12:45.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.632+0000 7fcd41400640 1 --2- 172.21.15.19:0/2683648307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd44155930 0x7fcd440d0280 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0097a0 tx=0x7fcd2c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164b40daf3f128d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:45.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.632+0000 7fcd40a00640 1 -- 172.21.15.19:0/2683648307 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c035020 con 0x7fcd44155930 2024-03-28T13:12:45.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.632+0000 7fcd40a00640 1 -- 172.21.15.19:0/2683648307 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd2c031810 con 0x7fcd44155930 2024-03-28T13:12:45.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.632+0000 7fcd40a00640 1 -- 172.21.15.19:0/2683648307 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c031be0 con 0x7fcd44155930 2024-03-28T13:12:45.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.632+0000 7fcd49200640 1 -- 172.21.15.19:0/2683648307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd44155930 msgr2=0x7fcd440d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:45.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.632+0000 7fcd49200640 1 --2- 172.21.15.19:0/2683648307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd44155930 0x7fcd440d0280 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0097a0 tx=0x7fcd2c02b8b0 comp rx=0 tx=0).stop 2024-03-28T13:12:45.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.633+0000 7fcd49200640 1 -- 172.21.15.19:0/2683648307 shutdown_connections 2024-03-28T13:12:45.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.633+0000 7fcd49200640 1 --2- 172.21.15.19:0/2683648307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd440d07c0 0x7fcd440aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:45.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.633+0000 7fcd49200640 1 --2- 172.21.15.19:0/2683648307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd44155930 0x7fcd440d0280 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:45.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.633+0000 7fcd49200640 1 -- 172.21.15.19:0/2683648307 >> 172.21.15.19:0/2683648307 conn(0x7fcd4406b750 msgr2=0x7fcd4406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.633+0000 7fcd49200640 1 -- 172.21.15.19:0/2683648307 shutdown_connections 2024-03-28T13:12:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.633+0000 7fcd49200640 1 -- 172.21.15.19:0/2683648307 wait complete. 2024-03-28T13:12:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.634+0000 7fcd49200640 1 Processor -- start 2024-03-28T13:12:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.634+0000 7fcd49200640 1 -- start start 2024-03-28T13:12:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.635+0000 7fcd49200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd440d07c0 0x7fcd44154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.635+0000 7fcd49200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd44155550 0x7fcd44072690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.636+0000 7fcd49200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd44155930 0x7fcd44099820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.636+0000 7fcd49200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd4415ba20 con 0x7fcd44155550 2024-03-28T13:12:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.636+0000 7fcd49200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcd440b4c30 con 0x7fcd440d07c0 2024-03-28T13:12:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.636+0000 7fcd49200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcd4415bba0 con 0x7fcd44155930 2024-03-28T13:12:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.636+0000 7fcd41400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd44155550 0x7fcd44072690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.636+0000 7fcd41400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd44155550 0x7fcd44072690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35616/0 (socket says 172.21.15.19:35616) 2024-03-28T13:12:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.636+0000 7fcd41400640 1 -- 172.21.15.19:0/2389799738 learned_addr learned my addr 172.21.15.19:0/2389799738 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:45.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.636+0000 7fcd42800640 1 --2- 172.21.15.19:0/2389799738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd44155930 0x7fcd44099820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:45.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.636+0000 7fcd41e00640 1 --2- 172.21.15.19:0/2389799738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd440d07c0 0x7fcd44154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:45.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.636+0000 7fcd41400640 1 -- 172.21.15.19:0/2389799738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd44155930 msgr2=0x7fcd44099820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:45.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.636+0000 7fcd41400640 1 --2- 172.21.15.19:0/2389799738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd44155930 0x7fcd44099820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:45.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.636+0000 7fcd41400640 1 -- 172.21.15.19:0/2389799738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd440d07c0 msgr2=0x7fcd44154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:45.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.636+0000 7fcd41400640 1 --2- 172.21.15.19:0/2389799738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd440d07c0 0x7fcd44154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:45.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.636+0000 7fcd41400640 1 -- 172.21.15.19:0/2389799738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4415d450 con 0x7fcd44155550 2024-03-28T13:12:45.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.637+0000 7fcd41400640 1 --2- 172.21.15.19:0/2389799738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd44155550 0x7fcd44072690 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c02bec0 tx=0x7fcd2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:45.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.637+0000 7fcd3a000640 1 -- 172.21.15.19:0/2389799738 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c041070 con 0x7fcd44155550 2024-03-28T13:12:45.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.637+0000 7fcd3a000640 1 -- 172.21.15.19:0/2389799738 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd2c03ad00 con 0x7fcd44155550 2024-03-28T13:12:45.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.637+0000 7fcd49200640 1 -- 172.21.15.19:0/2389799738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4415d730 con 0x7fcd44155550 2024-03-28T13:12:45.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.637+0000 7fcd3a000640 1 -- 172.21.15.19:0/2389799738 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c035040 con 0x7fcd44155550 2024-03-28T13:12:45.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.637+0000 7fcd49200640 1 -- 172.21.15.19:0/2389799738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd44154d50 con 0x7fcd44155550 2024-03-28T13:12:45.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.638+0000 7fcd49200640 1 -- 172.21.15.19:0/2389799738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd440685c0 con 0x7fcd44155550 2024-03-28T13:12:45.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.639+0000 7fcd3a000640 1 -- 172.21.15.19:0/2389799738 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcd2c04c060 con 0x7fcd44155550 2024-03-28T13:12:45.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.640+0000 7fcd3a000640 1 --2- 172.21.15.19:0/2389799738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd2006c560 0x7fcd2006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:45.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.640+0000 7fcd3a000640 1 -- 172.21.15.19:0/2389799738 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7fcd2c0acea0 con 0x7fcd44155550 2024-03-28T13:12:45.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.641+0000 7fcd41e00640 1 --2- 172.21.15.19:0/2389799738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd2006c560 0x7fcd2006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:45.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.642+0000 7fcd41e00640 1 --2- 172.21.15.19:0/2389799738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd2006c560 0x7fcd2006ea20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcd4415eb80 tx=0x7fcd34009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:45.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.645+0000 7fcd3a000640 1 -- 172.21.15.19:0/2389799738 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcd2c04b2c0 con 0x7fcd44155550 2024-03-28T13:12:45.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.851+0000 7fcd49200640 1 -- 172.21.15.19:0/2389799738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fcd4406e160 con 0x7fcd44155550 2024-03-28T13:12:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.859+0000 7fcd3a000640 1 -- 172.21.15.19:0/2389799738 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7fcd2c07b850 con 0x7fcd44155550 2024-03-28T13:12:45.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.862+0000 7fcd49200640 1 -- 172.21.15.19:0/2389799738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd2006c560 msgr2=0x7fcd2006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:45.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.862+0000 7fcd49200640 1 --2- 172.21.15.19:0/2389799738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd2006c560 0x7fcd2006ea20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcd4415eb80 tx=0x7fcd34009040 comp rx=0 tx=0).stop 2024-03-28T13:12:45.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.862+0000 7fcd49200640 1 -- 172.21.15.19:0/2389799738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd44155550 msgr2=0x7fcd44072690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:45.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.862+0000 7fcd49200640 1 --2- 172.21.15.19:0/2389799738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd44155550 0x7fcd44072690 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c02bec0 tx=0x7fcd2c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:12:45.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.862+0000 7fcd49200640 1 -- 172.21.15.19:0/2389799738 shutdown_connections 2024-03-28T13:12:45.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.862+0000 7fcd49200640 1 --2- 172.21.15.19:0/2389799738 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcd2006c560 0x7fcd2006ea20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:45.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.862+0000 7fcd49200640 1 --2- 172.21.15.19:0/2389799738 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd44155930 0x7fcd44099820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:45.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.862+0000 7fcd49200640 1 --2- 172.21.15.19:0/2389799738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd44155550 0x7fcd44072690 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:45.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.862+0000 7fcd49200640 1 --2- 172.21.15.19:0/2389799738 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd440d07c0 0x7fcd44154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:45.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.862+0000 7fcd49200640 1 -- 172.21.15.19:0/2389799738 >> 172.21.15.19:0/2389799738 conn(0x7fcd4406b750 msgr2=0x7fcd4406d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:45.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.862+0000 7fcd49200640 1 -- 172.21.15.19:0/2389799738 shutdown_connections 2024-03-28T13:12:45.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:45.862+0000 7fcd49200640 1 -- 172.21.15.19:0/2389799738 wait complete. 2024-03-28T13:12:45.882 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-28T13:12:45.882 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:12:45.999 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:12:46.120 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-28T13:12:46.121 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:12:46.121 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:12:46.181 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:12:46.181 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:12:46.182 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:12:46.238 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:12:46.238 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:12:46.238 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:12:46.298 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:12:46.299 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:12:46.299 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:12:46.355 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:12:46.355 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-28T13:12:46.355 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:12:46.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.589+0000 7f438fc00640 1 Processor -- start 2024-03-28T13:12:46.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.589+0000 7f438fc00640 1 -- start start 2024-03-28T13:12:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.590+0000 7f438fc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f43880d0e90 0x7f43880b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.590+0000 7f438fc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f438815b6f0 con 0x7f438815b310 2024-03-28T13:12:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.590+0000 7f438fc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4388072e80 con 0x7f43880727f0 2024-03-28T13:12:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.590+0000 7f438fc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f4388072c50 con 0x7f43880d0e90 2024-03-28T13:12:46.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.591+0000 7f438d200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f438815b310 0x7f438815f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44746/0 (socket says 172.21.15.19:44746) 2024-03-28T13:12:46.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.591+0000 7f438d200640 1 -- 172.21.15.19:0/923996853 learned_addr learned my addr 172.21.15.19:0/923996853 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:46.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.591+0000 7f4387400640 1 -- 172.21.15.19:0/923996853 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1963928367 0 0) 0x7f438815b6f0 con 0x7f438815b310 2024-03-28T13:12:46.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.591+0000 7f4387400640 1 -- 172.21.15.19:0/923996853 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f436c005730 con 0x7f438815b310 2024-03-28T13:12:46.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.591+0000 7f438dc00640 1 --2- 172.21.15.19:0/923996853 >> v2:172.21.15.188:3300/0 conn(0x7f43880d0e90 0x7f43880b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:46.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.591+0000 7f4387400640 1 -- 172.21.15.19:0/923996853 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 924205508 0 0) 0x7f436c005730 con 0x7f438815b310 2024-03-28T13:12:46.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.591+0000 7f4387400640 1 -- 172.21.15.19:0/923996853 >> v2:172.21.15.188:3300/0 conn(0x7f43880d0e90 msgr2=0x7f43880b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:46.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.591+0000 7f4387400640 1 --2- 172.21.15.19:0/923996853 >> v2:172.21.15.188:3300/0 conn(0x7f43880d0e90 0x7f43880b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:46.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.591+0000 7f4387400640 1 -- 172.21.15.19:0/923996853 >> v1:172.21.15.188:6789/0 conn(0x7f43880727f0 legacy=0x7f43880d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:46.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.591+0000 7f4387400640 1 -- 172.21.15.19:0/923996853 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4388073160 con 0x7f438815b310 2024-03-28T13:12:46.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.591+0000 7f4387400640 1 -- 172.21.15.19:0/923996853 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f4378001670 con 0x7f438815b310 2024-03-28T13:12:46.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.591+0000 7f4387400640 1 -- 172.21.15.19:0/923996853 >> v1:172.21.15.19:6789/0 conn(0x7f438815b310 legacy=0x7f438815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:46.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.591+0000 7f4387400640 1 --2- 172.21.15.19:0/923996853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f436c005c40 0x7f436c006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:46.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.591+0000 7f4387400640 1 --2- 172.21.15.19:0/923996853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f436c006a10 0x7f436c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:46.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.592+0000 7f4387400640 1 --2- 172.21.15.19:0/923996853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f436c00bbc0 0x7f436c00dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:46.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.592+0000 7f4387400640 1 -- 172.21.15.19:0/923996853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f436c00e5a0 con 0x7f436c00bbc0 2024-03-28T13:12:46.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.592+0000 7f4387400640 1 -- 172.21.15.19:0/923996853 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f436c00e720 con 0x7f436c005c40 2024-03-28T13:12:46.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.592+0000 7f4387400640 1 -- 172.21.15.19:0/923996853 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f436c00e920 con 0x7f436c006a10 2024-03-28T13:12:46.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.592+0000 7f438dc00640 1 --2- 172.21.15.19:0/923996853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f436c00bbc0 0x7f436c00dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:46.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.592+0000 7f4387e00640 1 --2- 172.21.15.19:0/923996853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f436c006a10 0x7f436c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:46.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.592+0000 7f438d200640 1 --2- 172.21.15.19:0/923996853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f436c005c40 0x7f436c006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:46.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.592+0000 7f438dc00640 1 -- 172.21.15.19:0/923996853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f436c006a10 msgr2=0x7f436c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:46.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.592+0000 7f438dc00640 1 --2- 172.21.15.19:0/923996853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f436c006a10 0x7f436c006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:46.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.592+0000 7f438dc00640 1 -- 172.21.15.19:0/923996853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f436c005c40 msgr2=0x7f436c006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:46.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.592+0000 7f438dc00640 1 --2- 172.21.15.19:0/923996853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f436c005c40 0x7f436c006040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:46.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.592+0000 7f438dc00640 1 -- 172.21.15.19:0/923996853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f436c005730 con 0x7f436c00bbc0 2024-03-28T13:12:46.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.593+0000 7f438dc00640 1 -- 172.21.15.19:0/923996853 reap_dead start 2024-03-28T13:12:46.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.594+0000 7f438dc00640 1 --2- 172.21.15.19:0/923996853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f436c00bbc0 0x7f436c00dfb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f43880b3b80 tx=0x7f436c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc84a2079d77277 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:46.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.594+0000 7f4387400640 1 -- 172.21.15.19:0/923996853 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4380007b30 con 0x7f436c00bbc0 2024-03-28T13:12:46.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.594+0000 7f4387400640 1 -- 172.21.15.19:0/923996853 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4380033600 con 0x7f436c00bbc0 2024-03-28T13:12:46.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.594+0000 7f4387400640 1 -- 172.21.15.19:0/923996853 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43800339d0 con 0x7f436c00bbc0 2024-03-28T13:12:46.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.595+0000 7f438fc00640 1 -- 172.21.15.19:0/923996853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f436c00bbc0 msgr2=0x7f436c00dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:46.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.595+0000 7f438fc00640 1 --2- 172.21.15.19:0/923996853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f436c00bbc0 0x7f436c00dfb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f43880b3b80 tx=0x7f436c007a40 comp rx=0 tx=0).stop 2024-03-28T13:12:46.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.595+0000 7f438fc00640 1 -- 172.21.15.19:0/923996853 shutdown_connections 2024-03-28T13:12:46.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.595+0000 7f438fc00640 1 --2- 172.21.15.19:0/923996853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f436c00bbc0 0x7f436c00dfb0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:46.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.595+0000 7f438fc00640 1 -- 172.21.15.19:0/923996853 >> 172.21.15.19:0/923996853 conn(0x7f438806b990 msgr2=0x7f438806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:46.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.595+0000 7f438fc00640 1 -- 172.21.15.19:0/923996853 shutdown_connections 2024-03-28T13:12:46.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.595+0000 7f438fc00640 1 -- 172.21.15.19:0/923996853 wait complete. 2024-03-28T13:12:46.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.597+0000 7f438fc00640 1 Processor -- start 2024-03-28T13:12:46.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.597+0000 7f438fc00640 1 -- start start 2024-03-28T13:12:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438fc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f436c00bbc0 0x7f43880cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438fc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4388151cc0 0x7f438815e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438fc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438815e5d0 0x7f43880b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438fc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4388073440 con 0x7f436c00bbc0 2024-03-28T13:12:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438fc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f438815f2c0 con 0x7f4388151cc0 2024-03-28T13:12:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438fc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f43880b4f90 con 0x7f438815e5d0 2024-03-28T13:12:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f436c00bbc0 0x7f43880cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f436c00bbc0 0x7f43880cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35640/0 (socket says 172.21.15.19:35640) 2024-03-28T13:12:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438d200640 1 -- 172.21.15.19:0/659387238 learned_addr learned my addr 172.21.15.19:0/659387238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438dc00640 1 --2- 172.21.15.19:0/659387238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438815e5d0 0x7f43880b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438d200640 1 -- 172.21.15.19:0/659387238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438815e5d0 msgr2=0x7f43880b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438d200640 1 --2- 172.21.15.19:0/659387238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438815e5d0 0x7f43880b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438d200640 1 -- 172.21.15.19:0/659387238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4388151cc0 msgr2=0x7f438815e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438d200640 1 --2- 172.21.15.19:0/659387238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4388151cc0 0x7f438815e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438d200640 1 -- 172.21.15.19:0/659387238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438815cb30 con 0x7f436c00bbc0 2024-03-28T13:12:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.598+0000 7f438dc00640 1 --2- 172.21.15.19:0/659387238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438815e5d0 0x7f43880b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.599+0000 7f438d200640 1 --2- 172.21.15.19:0/659387238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f436c00bbc0 0x7f43880cc8c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f437800d7a0 tx=0x7f437800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.599+0000 7f4384c00640 1 -- 172.21.15.19:0/659387238 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4378019070 con 0x7f436c00bbc0 2024-03-28T13:12:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.599+0000 7f4384c00640 1 -- 172.21.15.19:0/659387238 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43780074e0 con 0x7f436c00bbc0 2024-03-28T13:12:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.599+0000 7f4384c00640 1 -- 172.21.15.19:0/659387238 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4378004a00 con 0x7f436c00bbc0 2024-03-28T13:12:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.599+0000 7f438fc00640 1 -- 172.21.15.19:0/659387238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438815c2b0 con 0x7f436c00bbc0 2024-03-28T13:12:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.599+0000 7f438fc00640 1 -- 172.21.15.19:0/659387238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f438815c570 con 0x7f436c00bbc0 2024-03-28T13:12:46.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.600+0000 7f438fc00640 1 -- 172.21.15.19:0/659387238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43880685c0 con 0x7f436c00bbc0 2024-03-28T13:12:46.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.601+0000 7f4384c00640 1 -- 172.21.15.19:0/659387238 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4378004ba0 con 0x7f436c00bbc0 2024-03-28T13:12:46.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.602+0000 7f4384c00640 1 --2- 172.21.15.19:0/659387238 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f436406c530 0x7f436406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:46.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.602+0000 7f4384c00640 1 -- 172.21.15.19:0/659387238 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7f437808ace0 con 0x7f436c00bbc0 2024-03-28T13:12:46.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.603+0000 7f4387e00640 1 --2- 172.21.15.19:0/659387238 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f436406c530 0x7f436406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:46.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.604+0000 7f4387e00640 1 --2- 172.21.15.19:0/659387238 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f436406c530 0x7f436406e9f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f43880ca0c0 tx=0x7f4370009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.607+0000 7f4384c00640 1 -- 172.21.15.19:0/659387238 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f43780595f0 con 0x7f436c00bbc0 2024-03-28T13:12:46.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.855+0000 7f438fc00640 1 -- 172.21.15.19:0/659387238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43880d0010 con 0x7f436c00bbc0 2024-03-28T13:12:46.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.856+0000 7f4384c00640 1 -- 172.21.15.19:0/659387238 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3872 (secure 0 0 0) 0x7f4378058ef0 con 0x7f436c00bbc0 2024-03-28T13:12:46.856 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:46.856 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4916,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/2125648249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2125648249},{"type":"v1","addr":"172.21.15.188:6836","nonce":2125648249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4925,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/429866161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":429866161},{"type":"v1","addr":"172.21.15.188:6837","nonce":429866161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/1422209289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1422209289},{"type":"v1","addr":"172.21.15.19:6834","nonce":1422209289}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5886,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2560009894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2560009894},{"type":"v1","addr":"172.21.15.19:6835","nonce":2560009894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-03-28T13:12:46.856 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-03-28T13:12:46.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.859+0000 7f438fc00640 1 -- 172.21.15.19:0/659387238 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f436406c530 msgr2=0x7f436406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:46.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.859+0000 7f438fc00640 1 --2- 172.21.15.19:0/659387238 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f436406c530 0x7f436406e9f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f43880ca0c0 tx=0x7f4370009040 comp rx=0 tx=0).stop 2024-03-28T13:12:46.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.859+0000 7f438fc00640 1 -- 172.21.15.19:0/659387238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f436c00bbc0 msgr2=0x7f43880cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:46.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.859+0000 7f438fc00640 1 --2- 172.21.15.19:0/659387238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f436c00bbc0 0x7f43880cc8c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f437800d7a0 tx=0x7f437800dc60 comp rx=0 tx=0).stop 2024-03-28T13:12:46.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.859+0000 7f438fc00640 1 -- 172.21.15.19:0/659387238 shutdown_connections 2024-03-28T13:12:46.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.859+0000 7f438fc00640 1 --2- 172.21.15.19:0/659387238 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f436406c530 0x7f436406e9f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:46.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.859+0000 7f438fc00640 1 --2- 172.21.15.19:0/659387238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f438815e5d0 0x7f43880b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:46.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.859+0000 7f438fc00640 1 --2- 172.21.15.19:0/659387238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4388151cc0 0x7f438815e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:46.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.859+0000 7f438fc00640 1 --2- 172.21.15.19:0/659387238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f436c00bbc0 0x7f43880cc8c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:46.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.859+0000 7f438fc00640 1 -- 172.21.15.19:0/659387238 >> 172.21.15.19:0/659387238 conn(0x7f438806b990 msgr2=0x7f438806bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:46.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.859+0000 7f438fc00640 1 -- 172.21.15.19:0/659387238 shutdown_connections 2024-03-28T13:12:46.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:46.859+0000 7f438fc00640 1 -- 172.21.15.19:0/659387238 wait complete. 2024-03-28T13:12:46.879 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-28T13:12:46.879 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-28T13:12:46.880 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-28T13:12:46.880 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:12:46.880 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-28T13:12:46.880 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-28T13:12:46.880 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-28T13:12:46.880 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:12:46.881 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-28T13:12:46.881 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-28T13:12:46.881 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-28T13:12:46.881 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:12:46.881 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-28T13:12:46.881 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-28T13:12:46.881 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-28T13:12:46.881 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:12:46.897 INFO:tasks.ceph.mds.a:Stopped 2024-03-28T13:12:46.897 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-28T13:12:46.899 INFO:tasks.ceph.mds.c:Stopped 2024-03-28T13:12:46.899 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-28T13:12:46.901 INFO:tasks.ceph.mds.b:Stopped 2024-03-28T13:12:46.901 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-28T13:12:46.902 INFO:tasks.ceph.mds.d:Stopped 2024-03-28T13:12:46.903 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-28T13:12:46.945 INFO:tasks.ceph.mds.a:Started 2024-03-28T13:12:46.946 DEBUG:teuthology.parallel:result is None 2024-03-28T13:12:46.949 INFO:tasks.ceph.mds.b:Started 2024-03-28T13:12:46.950 DEBUG:teuthology.parallel:result is None 2024-03-28T13:12:46.951 INFO:tasks.ceph.mds.c:Started 2024-03-28T13:12:46.952 DEBUG:teuthology.parallel:result is None 2024-03-28T13:12:46.956 INFO:tasks.ceph.mds.d:Started 2024-03-28T13:12:46.956 DEBUG:teuthology.parallel:result is None 2024-03-28T13:12:46.956 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-28T13:12:47.101 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-03-28T13:12:47.102 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-03-28T13:12:47.109 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-03-28T13:12:47.115 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-03-28T13:12:47.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.144+0000 7f44ae600640 1 Processor -- start 2024-03-28T13:12:47.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.144+0000 7f44ae600640 1 -- start start 2024-03-28T13:12:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.145+0000 7f44ae600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80b4650 0x7f44a806c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.145+0000 7f44ae600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44a815ede0 con 0x7f44a80b4a30 2024-03-28T13:12:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.145+0000 7f44ae600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44a8141d80 con 0x7f44a8158580 2024-03-28T13:12:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.145+0000 7f44ae600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f44a80b6320 con 0x7f44a80b4650 2024-03-28T13:12:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.145+0000 7f44a6a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f44a80b4a30 0x7f44a80b5c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44768/0 (socket says 172.21.15.19:44768) 2024-03-28T13:12:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.146+0000 7f44a6a00640 1 -- 172.21.15.19:0/3117077255 learned_addr learned my addr 172.21.15.19:0/3117077255 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:47.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.146+0000 7f44a7400640 1 --2- 172.21.15.19:0/3117077255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80b4650 0x7f44a806c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:47.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.146+0000 7f44a6000640 1 -- 172.21.15.19:0/3117077255 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2175271874 0 0) 0x7f44a8141d80 con 0x7f44a8158580 2024-03-28T13:12:47.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.146+0000 7f44a6000640 1 -- 172.21.15.19:0/3117077255 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f448c003520 con 0x7f44a8158580 2024-03-28T13:12:47.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.146+0000 7f44a6000640 1 -- 172.21.15.19:0/3117077255 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4237110795 0 0) 0x7f44a815ede0 con 0x7f44a80b4a30 2024-03-28T13:12:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.146+0000 7f44a6000640 1 -- 172.21.15.19:0/3117077255 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44a8141d80 con 0x7f44a80b4a30 2024-03-28T13:12:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.146+0000 7f44a7400640 1 -- 172.21.15.19:0/3117077255 >> v1:172.21.15.188:6789/0 conn(0x7f44a8158580 legacy=0x7f44a81589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.146+0000 7f44a7400640 1 -- 172.21.15.19:0/3117077255 >> v1:172.21.15.19:6789/0 conn(0x7f44a80b4a30 legacy=0x7f44a80b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.146+0000 7f44a7400640 1 -- 172.21.15.19:0/3117077255 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a8142060 con 0x7f44a80b4650 2024-03-28T13:12:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.147+0000 7f44a7400640 1 --2- 172.21.15.19:0/3117077255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80b4650 0x7f44a806c680 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44980097d0 tx=0x7f449802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e306234dc3da97e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.148+0000 7f44a6000640 1 -- 172.21.15.19:0/3117077255 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498035020 con 0x7f44a80b4650 2024-03-28T13:12:47.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.148+0000 7f44a6000640 1 -- 172.21.15.19:0/3117077255 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4498031800 con 0x7f44a80b4650 2024-03-28T13:12:47.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.148+0000 7f44a6000640 1 -- 172.21.15.19:0/3117077255 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498031bb0 con 0x7f44a80b4650 2024-03-28T13:12:47.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.148+0000 7f44ae600640 1 -- 172.21.15.19:0/3117077255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80b4650 msgr2=0x7f44a806c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:47.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.148+0000 7f44ae600640 1 --2- 172.21.15.19:0/3117077255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80b4650 0x7f44a806c680 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44980097d0 tx=0x7f449802b8a0 comp rx=0 tx=0).stop 2024-03-28T13:12:47.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.148+0000 7f44ae600640 1 -- 172.21.15.19:0/3117077255 shutdown_connections 2024-03-28T13:12:47.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.148+0000 7f44ae600640 1 --2- 172.21.15.19:0/3117077255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80b4650 0x7f44a806c680 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:47.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.148+0000 7f44ae600640 1 -- 172.21.15.19:0/3117077255 >> 172.21.15.19:0/3117077255 conn(0x7f44a8067670 msgr2=0x7f44a8067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:47.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.148+0000 7f44ae600640 1 -- 172.21.15.19:0/3117077255 shutdown_connections 2024-03-28T13:12:47.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.148+0000 7f44ae600640 1 -- 172.21.15.19:0/3117077255 wait complete. 2024-03-28T13:12:47.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.150+0000 7f44ae600640 1 Processor -- start 2024-03-28T13:12:47.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.150+0000 7f44ae600640 1 -- start start 2024-03-28T13:12:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.151+0000 7f44ae600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80b4650 0x7f44a8140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.151+0000 7f44ae600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44a80b4a30 0x7f44a815a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.151+0000 7f44ae600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44a8158580 0x7f44a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.151+0000 7f44ae600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8142620 con 0x7f44a80b4a30 2024-03-28T13:12:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.151+0000 7f44ae600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f44a8141b30 con 0x7f44a8158580 2024-03-28T13:12:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.151+0000 7f44ae600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f44a8142900 con 0x7f44a80b4650 2024-03-28T13:12:47.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.151+0000 7f44a6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44a80b4a30 0x7f44a815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:47.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.151+0000 7f44a6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44a80b4a30 0x7f44a815a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35684/0 (socket says 172.21.15.19:35684) 2024-03-28T13:12:47.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.151+0000 7f44a6a00640 1 -- 172.21.15.19:0/466620443 learned_addr learned my addr 172.21.15.19:0/466620443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:47.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.151+0000 7f44a7e00640 1 --2- 172.21.15.19:0/466620443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44a8158580 0x7f44a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:47.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.152+0000 7f44a7400640 1 --2- 172.21.15.19:0/466620443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80b4650 0x7f44a8140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:47.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.152+0000 7f44a6a00640 1 -- 172.21.15.19:0/466620443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80b4650 msgr2=0x7f44a8140d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:47.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.152+0000 7f44a6a00640 1 --2- 172.21.15.19:0/466620443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80b4650 0x7f44a8140d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:47.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.152+0000 7f44a6a00640 1 -- 172.21.15.19:0/466620443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44a8158580 msgr2=0x7f44a806d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:47.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.152+0000 7f44a6a00640 1 --2- 172.21.15.19:0/466620443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44a8158580 0x7f44a806d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:47.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.152+0000 7f44a6a00640 1 -- 172.21.15.19:0/466620443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a815a750 con 0x7f44a80b4a30 2024-03-28T13:12:47.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.152+0000 7f44a7e00640 1 --2- 172.21.15.19:0/466620443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44a8158580 0x7f44a806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:12:47.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.152+0000 7f44a6a00640 1 --2- 172.21.15.19:0/466620443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44a80b4a30 0x7f44a815a210 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f449400c530 tx=0x7f449400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:47.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.152+0000 7f449f400640 1 -- 172.21.15.19:0/466620443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494017070 con 0x7f44a80b4a30 2024-03-28T13:12:47.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.153+0000 7f449f400640 1 -- 172.21.15.19:0/466620443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4494005ce0 con 0x7f44a80b4a30 2024-03-28T13:12:47.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.153+0000 7f449f400640 1 -- 172.21.15.19:0/466620443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494007270 con 0x7f44a80b4a30 2024-03-28T13:12:47.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.153+0000 7f44ae600640 1 -- 172.21.15.19:0/466620443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a8159170 con 0x7f44a80b4a30 2024-03-28T13:12:47.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.153+0000 7f44ae600640 1 -- 172.21.15.19:0/466620443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a8159730 con 0x7f44a80b4a30 2024-03-28T13:12:47.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.154+0000 7f44ae600640 1 -- 172.21.15.19:0/466620443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44a80685c0 con 0x7f44a80b4a30 2024-03-28T13:12:47.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.155+0000 7f449f400640 1 -- 172.21.15.19:0/466620443 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f449401a070 con 0x7f44a80b4a30 2024-03-28T13:12:47.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.156+0000 7f449f400640 1 --2- 172.21.15.19:0/466620443 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f448806c530 0x7f448806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:47.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.156+0000 7f449f400640 1 -- 172.21.15.19:0/466620443 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4943+0+0 (secure 0 0 0) 0x7f449408a6b0 con 0x7f44a80b4a30 2024-03-28T13:12:47.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.157+0000 7f44a7400640 1 --2- 172.21.15.19:0/466620443 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f448806c530 0x7f448806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:47.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.157+0000 7f44a7400640 1 --2- 172.21.15.19:0/466620443 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f448806c530 0x7f448806e9f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4498036000 tx=0x7f4498008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:47.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.161+0000 7f449f400640 1 -- 172.21.15.19:0/466620443 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4494058980 con 0x7f44a80b4a30 2024-03-28T13:12:47.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:47.374+0000 7f44ae600640 1 -- 172.21.15.19:0/466620443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f44a80b2960 con 0x7f44a80b4a30 2024-03-28T13:12:48.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.422+0000 7f449f400640 1 -- 172.21.15.19:0/466620443 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) ==== 97+0+0 (secure 0 0 0) 0x7f449405cc80 con 0x7f44a80b4a30 2024-03-28T13:12:48.423 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-03-28T13:12:48.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.426+0000 7f44ae600640 1 -- 172.21.15.19:0/466620443 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f448806c530 msgr2=0x7f448806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:48.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.426+0000 7f44ae600640 1 --2- 172.21.15.19:0/466620443 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f448806c530 0x7f448806e9f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4498036000 tx=0x7f4498008030 comp rx=0 tx=0).stop 2024-03-28T13:12:48.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.426+0000 7f44ae600640 1 -- 172.21.15.19:0/466620443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44a80b4a30 msgr2=0x7f44a815a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:48.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.426+0000 7f44ae600640 1 --2- 172.21.15.19:0/466620443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44a80b4a30 0x7f44a815a210 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f449400c530 tx=0x7f449400c9f0 comp rx=0 tx=0).stop 2024-03-28T13:12:48.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.426+0000 7f44ae600640 1 -- 172.21.15.19:0/466620443 shutdown_connections 2024-03-28T13:12:48.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.426+0000 7f44ae600640 1 --2- 172.21.15.19:0/466620443 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f448806c530 0x7f448806e9f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:48.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.426+0000 7f44ae600640 1 --2- 172.21.15.19:0/466620443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44a8158580 0x7f44a806d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:48.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.426+0000 7f44ae600640 1 --2- 172.21.15.19:0/466620443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44a80b4a30 0x7f44a815a210 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:48.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.426+0000 7f44ae600640 1 --2- 172.21.15.19:0/466620443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44a80b4650 0x7f44a8140d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:48.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.426+0000 7f44ae600640 1 -- 172.21.15.19:0/466620443 >> 172.21.15.19:0/466620443 conn(0x7f44a8067670 msgr2=0x7f44a8161070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:48.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.426+0000 7f44ae600640 1 -- 172.21.15.19:0/466620443 shutdown_connections 2024-03-28T13:12:48.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.427+0000 7f44ae600640 1 -- 172.21.15.19:0/466620443 wait complete. 2024-03-28T13:12:48.450 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-28T13:12:48.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.671+0000 7f8b84600640 1 Processor -- start 2024-03-28T13:12:48.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.672+0000 7f8b84600640 1 -- start start 2024-03-28T13:12:48.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.673+0000 7f8b84600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b7c15e7b0 0x7f8b7c0b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:48.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.673+0000 7f8b84600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b7c0c7810 0x7f8b7c06ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:48.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.673+0000 7f8b84600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b7c0d01a0 con 0x7f8b7c0d1d70 2024-03-28T13:12:48.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.673+0000 7f8b84600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8b7c0d1240 con 0x7f8b7c15e7b0 2024-03-28T13:12:48.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.673+0000 7f8b84600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7c0d0480 con 0x7f8b7c0c7810 2024-03-28T13:12:48.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.674+0000 7f8b81c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b7c15e7b0 0x7f8b7c0b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:48.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.674+0000 7f8b81c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8b7c15e7b0 0x7f8b7c0b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35694/0 (socket says 172.21.15.19:35694) 2024-03-28T13:12:48.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.674+0000 7f8b81c00640 1 -- 172.21.15.19:0/2868717313 learned_addr learned my addr 172.21.15.19:0/2868717313 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:48.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.674+0000 7f8b82600640 1 --2- 172.21.15.19:0/2868717313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b7c0c7810 0x7f8b7c06ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:48.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.674+0000 7f8b81c00640 1 -- 172.21.15.19:0/2868717313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b7c0c7810 msgr2=0x7f8b7c06ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:48.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.674+0000 7f8b81c00640 1 --2- 172.21.15.19:0/2868717313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b7c0c7810 0x7f8b7c06ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:48.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.674+0000 7f8b81c00640 1 -- 172.21.15.19:0/2868717313 >> v1:172.21.15.188:6789/0 conn(0x7f8b7c0d1d70 legacy=0x7f8b7c0c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:48.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.674+0000 7f8b81c00640 1 -- 172.21.15.19:0/2868717313 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7c0d0760 con 0x7f8b7c15e7b0 2024-03-28T13:12:48.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.676+0000 7f8b81c00640 1 --2- 172.21.15.19:0/2868717313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b7c15e7b0 0x7f8b7c0b4980 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009870 tx=0x7f8b7002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91d07825d3ebe4b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:48.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.677+0000 7f8b7be00640 1 -- 172.21.15.19:0/2868717313 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70034070 con 0x7f8b7c15e7b0 2024-03-28T13:12:48.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.677+0000 7f8b7be00640 1 -- 172.21.15.19:0/2868717313 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b70030840 con 0x7f8b7c15e7b0 2024-03-28T13:12:48.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.677+0000 7f8b7be00640 1 -- 172.21.15.19:0/2868717313 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70030c10 con 0x7f8b7c15e7b0 2024-03-28T13:12:48.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.677+0000 7f8b84600640 1 -- 172.21.15.19:0/2868717313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b7c15e7b0 msgr2=0x7f8b7c0b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:48.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.677+0000 7f8b84600640 1 --2- 172.21.15.19:0/2868717313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b7c15e7b0 0x7f8b7c0b4980 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009870 tx=0x7f8b7002b980 comp rx=0 tx=0).stop 2024-03-28T13:12:48.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.677+0000 7f8b84600640 1 -- 172.21.15.19:0/2868717313 shutdown_connections 2024-03-28T13:12:48.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.677+0000 7f8b84600640 1 --2- 172.21.15.19:0/2868717313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b7c0c7810 0x7f8b7c06ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:48.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.677+0000 7f8b84600640 1 --2- 172.21.15.19:0/2868717313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b7c15e7b0 0x7f8b7c0b4980 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:48.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.677+0000 7f8b84600640 1 -- 172.21.15.19:0/2868717313 >> 172.21.15.19:0/2868717313 conn(0x7f8b7c0b3e90 msgr2=0x7f8b7c0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:48.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.677+0000 7f8b84600640 1 -- 172.21.15.19:0/2868717313 shutdown_connections 2024-03-28T13:12:48.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.677+0000 7f8b84600640 1 -- 172.21.15.19:0/2868717313 wait complete. 2024-03-28T13:12:48.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.679+0000 7f8b84600640 1 Processor -- start 2024-03-28T13:12:48.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.679+0000 7f8b84600640 1 -- start start 2024-03-28T13:12:48.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.680+0000 7f8b84600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b7c0c7810 0x7f8b7c1506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:48.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.680+0000 7f8b84600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b7c0d1d70 0x7f8b7c0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:48.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.680+0000 7f8b84600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b7c15e7b0 0x7f8b7c154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:48.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.680+0000 7f8b84600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c0cfe30 con 0x7f8b7c0c7810 2024-03-28T13:12:48.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.680+0000 7f8b84600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7c0d0d20 con 0x7f8b7c0d1d70 2024-03-28T13:12:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.680+0000 7f8b84600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7c06cf70 con 0x7f8b7c15e7b0 2024-03-28T13:12:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.680+0000 7f8b81c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b7c0c7810 0x7f8b7c1506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.681+0000 7f8b81c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b7c0c7810 0x7f8b7c1506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35708/0 (socket says 172.21.15.19:35708) 2024-03-28T13:12:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.681+0000 7f8b81c00640 1 -- 172.21.15.19:0/2811096931 learned_addr learned my addr 172.21.15.19:0/2811096931 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.681+0000 7f8b81c00640 1 -- 172.21.15.19:0/2811096931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b7c15e7b0 msgr2=0x7f8b7c154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.681+0000 7f8b82600640 1 --2- 172.21.15.19:0/2811096931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b7c15e7b0 0x7f8b7c154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.681+0000 7f8b81200640 1 --2- 172.21.15.19:0/2811096931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b7c0d1d70 0x7f8b7c0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.681+0000 7f8b81c00640 1 --2- 172.21.15.19:0/2811096931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b7c15e7b0 0x7f8b7c154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.681+0000 7f8b81c00640 1 -- 172.21.15.19:0/2811096931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b7c0d1d70 msgr2=0x7f8b7c0ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.681+0000 7f8b81c00640 1 --2- 172.21.15.19:0/2811096931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b7c0d1d70 0x7f8b7c0ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.681+0000 7f8b81c00640 1 -- 172.21.15.19:0/2811096931 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7c158370 con 0x7f8b7c0c7810 2024-03-28T13:12:48.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.681+0000 7f8b81200640 1 --2- 172.21.15.19:0/2811096931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b7c0d1d70 0x7f8b7c0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:48.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.681+0000 7f8b81c00640 1 --2- 172.21.15.19:0/2811096931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b7c0c7810 0x7f8b7c1506f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8b70000c00 tx=0x7f8b700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:48.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.682+0000 7f8b79600640 1 -- 172.21.15.19:0/2811096931 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70034040 con 0x7f8b7c0c7810 2024-03-28T13:12:48.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.682+0000 7f8b79600640 1 -- 172.21.15.19:0/2811096931 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b70039d90 con 0x7f8b7c0c7810 2024-03-28T13:12:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.682+0000 7f8b79600640 1 -- 172.21.15.19:0/2811096931 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70043610 con 0x7f8b7c0c7810 2024-03-28T13:12:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.682+0000 7f8b84600640 1 -- 172.21.15.19:0/2811096931 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7c158650 con 0x7f8b7c0c7810 2024-03-28T13:12:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.682+0000 7f8b84600640 1 -- 172.21.15.19:0/2811096931 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b7c161260 con 0x7f8b7c0c7810 2024-03-28T13:12:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.683+0000 7f8b84600640 1 -- 172.21.15.19:0/2811096931 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b7c068cd0 con 0x7f8b7c0c7810 2024-03-28T13:12:48.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.684+0000 7f8b79600640 1 -- 172.21.15.19:0/2811096931 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8b7004a060 con 0x7f8b7c0c7810 2024-03-28T13:12:48.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.686+0000 7f8b79600640 1 --2- 172.21.15.19:0/2811096931 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8b6006c530 0x7f8b6006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:48.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.686+0000 7f8b79600640 1 -- 172.21.15.19:0/2811096931 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7f8b700ad4a0 con 0x7f8b7c0c7810 2024-03-28T13:12:48.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.686+0000 7f8b81200640 1 --2- 172.21.15.19:0/2811096931 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8b6006c530 0x7f8b6006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:48.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.688+0000 7f8b81200640 1 --2- 172.21.15.19:0/2811096931 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8b6006c530 0x7f8b6006e9f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0ca140 tx=0x7f8b6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:48.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.691+0000 7f8b79600640 1 -- 172.21.15.19:0/2811096931 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8b7003e070 con 0x7f8b7c0c7810 2024-03-28T13:12:48.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.968+0000 7f8b84600640 1 -- 172.21.15.19:0/2811096931 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8b7c159ea0 con 0x7f8b7c0c7810 2024-03-28T13:12:48.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.969+0000 7f8b79600640 1 -- 172.21.15.19:0/2811096931 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+5304 (secure 0 0 0) 0x7f8b7002d330 con 0x7f8b7c0c7810 2024-03-28T13:12:48.970 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:48.970 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:12:48.970 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-03-28T13:12:48.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.970 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-03-28T13:12:48.970 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmNRIkABAAq/nJioDZjp1zFWAYOLPBYw==", 2024-03-28T13:12:48.970 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.970 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:12:48.970 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:12:48.970 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmc10CIxAAQWCSjH0hW3KUk8/qQm91QA==", 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-03-28T13:12:48.971 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmmlUoEhAA4FUGXoZ/h/ve5fThJNdE2Q==", 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmWDRnMxAA34LiLI79N5fVJnbcPacbQA==", 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDYawVmxhwWLBAAm7C56UGConFx2D7Lbq2SSA==", 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDbawVmC2BiNxAALQrAcXU44m2dTCjks5yy/Q==", 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:12:48.973 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:12:48.974 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.974 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-03-28T13:12:48.974 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfawVmozrJFRAA7YV1uMucIW7H5SI4KY8aDw==", 2024-03-28T13:12:48.974 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.974 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:12:48.974 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:12:48.974 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:12:48.974 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.974 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-03-28T13:12:48.975 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiawVmccbTMBAATxk24Bf4LuQSqU92HdwgUg==", 2024-03-28T13:12:48.975 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.975 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:12:48.975 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:12:48.975 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:12:48.975 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.975 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.975 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-03-28T13:12:48.975 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDpawVmdwnILRAAjUEBwnBEeSltu2CLRGhmDQ==", 2024-03-28T13:12:48.975 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.975 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:12:48.975 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:12:48.975 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:12:48.976 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.976 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-03-28T13:12:48.976 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDtawVmkwW5ARAAZJ/Nd/Wtq7LcbUjeqPNUag==", 2024-03-28T13:12:48.976 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.976 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:12:48.976 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:12:48.976 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:12:48.976 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.976 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-03-28T13:12:48.976 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDwawVm254wKRAAmCYODqSd2Whn4YKQzywrng==", 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDzawVmV8z+MhAA9JeF55K7ph1mQRWalT7XhA==", 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.978 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-03-28T13:12:48.978 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q==", 2024-03-28T13:12:48.978 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.978 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:12:48.978 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-03-28T13:12:48.978 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-28T13:12:48.978 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.978 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-03-28T13:12:48.978 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmJYGjHBAAAx2TtR0Gb3fafnZU26FCMA==", 2024-03-28T13:12:48.978 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.978 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:12:48.978 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:12:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:12:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:12:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-03-28T13:12:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmE7V/ERAAYcdvolIMDd5ff+pCn7hN1w==", 2024-03-28T13:12:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:12:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:12:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:12:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:12:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.980 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-03-28T13:12:48.980 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmk6q9JhAAeC3JHQDnK8IXGVoK6qGEFA==", 2024-03-28T13:12:48.980 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.980 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:12:48.980 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:12:48.980 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:12:48.980 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:12:48.980 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.980 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.980 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-03-28T13:12:48.980 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDRawVm9IzFLRAA5i1Yl7LQ7eNNHp7v4H2kog==", 2024-03-28T13:12:48.980 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.980 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:12:48.980 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:12:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:12:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:12:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-03-28T13:12:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDSawVm5H1FLxAAtyjOPcCEUQyvTpBdu3Fn3g==", 2024-03-28T13:12:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:12:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-03-28T13:12:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:12:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:12:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:48.981 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-03-28T13:12:48.982 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDSawVmebzfIRAAA0+qIIaprr4SfEZT6qRooA==", 2024-03-28T13:12:48.982 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:12:48.982 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:12:48.982 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-03-28T13:12:48.982 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:12:48.982 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.982 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:48.982 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-03-28T13:12:48.982 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:12:48.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.972+0000 7f8b84600640 1 -- 172.21.15.19:0/2811096931 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8b6006c530 msgr2=0x7f8b6006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:48.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.972+0000 7f8b84600640 1 --2- 172.21.15.19:0/2811096931 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8b6006c530 0x7f8b6006e9f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c0ca140 tx=0x7f8b6c009040 comp rx=0 tx=0).stop 2024-03-28T13:12:48.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.972+0000 7f8b84600640 1 -- 172.21.15.19:0/2811096931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b7c0c7810 msgr2=0x7f8b7c1506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:48.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.972+0000 7f8b84600640 1 --2- 172.21.15.19:0/2811096931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b7c0c7810 0x7f8b7c1506f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8b70000c00 tx=0x7f8b700046c0 comp rx=0 tx=0).stop 2024-03-28T13:12:48.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.972+0000 7f8b84600640 1 -- 172.21.15.19:0/2811096931 shutdown_connections 2024-03-28T13:12:48.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.972+0000 7f8b84600640 1 --2- 172.21.15.19:0/2811096931 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8b6006c530 0x7f8b6006e9f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:48.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.972+0000 7f8b84600640 1 --2- 172.21.15.19:0/2811096931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b7c15e7b0 0x7f8b7c154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:48.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.972+0000 7f8b84600640 1 --2- 172.21.15.19:0/2811096931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b7c0d1d70 0x7f8b7c0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:48.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.972+0000 7f8b84600640 1 --2- 172.21.15.19:0/2811096931 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b7c0c7810 0x7f8b7c1506f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:48.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.972+0000 7f8b84600640 1 -- 172.21.15.19:0/2811096931 >> 172.21.15.19:0/2811096931 conn(0x7f8b7c0b3e90 msgr2=0x7f8b7c15a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:48.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.972+0000 7f8b84600640 1 -- 172.21.15.19:0/2811096931 shutdown_connections 2024-03-28T13:12:48.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:48.972+0000 7f8b84600640 1 -- 172.21.15.19:0/2811096931 wait complete. 2024-03-28T13:12:48.992 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:12:49.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.218+0000 7fdc49c00640 1 Processor -- start 2024-03-28T13:12:49.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.218+0000 7fdc49c00640 1 -- start start 2024-03-28T13:12:49.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.219+0000 7fdc49c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdc4414ba20 0x7fdc440b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:49.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.219+0000 7fdc49c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc440b33e0 con 0x7fdc4414c1b0 2024-03-28T13:12:49.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.219+0000 7fdc49c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc440cde20 con 0x7fdc440cfda0 2024-03-28T13:12:49.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.219+0000 7fdc49c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fdc44166ae0 con 0x7fdc4414ba20 2024-03-28T13:12:49.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.219+0000 7fdc43400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdc4414c1b0 0x7fdc44166290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44784/0 (socket says 172.21.15.19:44784) 2024-03-28T13:12:49.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.220+0000 7fdc43400640 1 -- 172.21.15.19:0/2909582570 learned_addr learned my addr 172.21.15.19:0/2909582570 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:49.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.220+0000 7fdc43e00640 1 --2- 172.21.15.19:0/2909582570 >> v2:172.21.15.188:3300/0 conn(0x7fdc4414ba20 0x7fdc440b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:49.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.220+0000 7fdc42a00640 1 -- 172.21.15.19:0/2909582570 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1152123092 0 0) 0x7fdc440b33e0 con 0x7fdc4414c1b0 2024-03-28T13:12:49.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.220+0000 7fdc42a00640 1 -- 172.21.15.19:0/2909582570 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc28003520 con 0x7fdc4414c1b0 2024-03-28T13:12:49.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.220+0000 7fdc42a00640 1 -- 172.21.15.19:0/2909582570 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2892393963 0 0) 0x7fdc28003520 con 0x7fdc4414c1b0 2024-03-28T13:12:49.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.220+0000 7fdc42a00640 1 -- 172.21.15.19:0/2909582570 >> v2:172.21.15.188:3300/0 conn(0x7fdc4414ba20 msgr2=0x7fdc440b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:49.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.220+0000 7fdc42a00640 1 --2- 172.21.15.19:0/2909582570 >> v2:172.21.15.188:3300/0 conn(0x7fdc4414ba20 0x7fdc440b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.220+0000 7fdc42a00640 1 -- 172.21.15.19:0/2909582570 >> v1:172.21.15.188:6789/0 conn(0x7fdc440cfda0 legacy=0x7fdc440d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:49.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.220+0000 7fdc42a00640 1 -- 172.21.15.19:0/2909582570 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc44166dc0 con 0x7fdc4414c1b0 2024-03-28T13:12:49.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.220+0000 7fdc42a00640 1 -- 172.21.15.19:0/2909582570 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fdc30001670 con 0x7fdc4414c1b0 2024-03-28T13:12:49.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc42a00640 1 -- 172.21.15.19:0/2909582570 >> v1:172.21.15.19:6789/0 conn(0x7fdc4414c1b0 legacy=0x7fdc44166290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:49.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc42a00640 1 --2- 172.21.15.19:0/2909582570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc28003a00 0x7fdc28003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:49.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc42a00640 1 --2- 172.21.15.19:0/2909582570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc28004800 0x7fdc28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:49.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc42a00640 1 --2- 172.21.15.19:0/2909582570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc280051c0 0x7fdc2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:49.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc42a00640 1 -- 172.21.15.19:0/2909582570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc44166ae0 con 0x7fdc28004800 2024-03-28T13:12:49.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc42a00640 1 -- 172.21.15.19:0/2909582570 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2800c0c0 con 0x7fdc28003a00 2024-03-28T13:12:49.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc42a00640 1 -- 172.21.15.19:0/2909582570 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdc2800c2c0 con 0x7fdc280051c0 2024-03-28T13:12:49.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc43400640 1 --2- 172.21.15.19:0/2909582570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc28004800 0x7fdc28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:49.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc43e00640 1 --2- 172.21.15.19:0/2909582570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc28003a00 0x7fdc28003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:49.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc43400640 1 -- 172.21.15.19:0/2909582570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc280051c0 msgr2=0x7fdc2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:49.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc43400640 1 --2- 172.21.15.19:0/2909582570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc280051c0 0x7fdc2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc43400640 1 -- 172.21.15.19:0/2909582570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc28003a00 msgr2=0x7fdc28003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:49.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc43400640 1 --2- 172.21.15.19:0/2909582570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc28003a00 0x7fdc28003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc43400640 1 -- 172.21.15.19:0/2909582570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc28003520 con 0x7fdc28004800 2024-03-28T13:12:49.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc43e00640 1 --2- 172.21.15.19:0/2909582570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc28003a00 0x7fdc28003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:49.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.221+0000 7fdc49200640 1 -- 172.21.15.19:0/2909582570 reap_dead start 2024-03-28T13:12:49.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.223+0000 7fdc43400640 1 --2- 172.21.15.19:0/2909582570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc28004800 0x7fdc28004c80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000c0e0 tx=0x7fdc28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d81c31e4e876d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:49.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.223+0000 7fdc42a00640 1 -- 172.21.15.19:0/2909582570 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30007c20 con 0x7fdc28004800 2024-03-28T13:12:49.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.223+0000 7fdc42a00640 1 -- 172.21.15.19:0/2909582570 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc30030070 con 0x7fdc28004800 2024-03-28T13:12:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.223+0000 7fdc42a00640 1 -- 172.21.15.19:0/2909582570 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30005d70 con 0x7fdc28004800 2024-03-28T13:12:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.223+0000 7fdc49c00640 1 -- 172.21.15.19:0/2909582570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc28004800 msgr2=0x7fdc28004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.223+0000 7fdc49c00640 1 --2- 172.21.15.19:0/2909582570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc28004800 0x7fdc28004c80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000c0e0 tx=0x7fdc28005830 comp rx=0 tx=0).stop 2024-03-28T13:12:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.224+0000 7fdc49c00640 1 -- 172.21.15.19:0/2909582570 shutdown_connections 2024-03-28T13:12:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.224+0000 7fdc49c00640 1 --2- 172.21.15.19:0/2909582570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc28004800 0x7fdc28004c80 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.224+0000 7fdc49c00640 1 -- 172.21.15.19:0/2909582570 >> 172.21.15.19:0/2909582570 conn(0x7fdc440ccad0 msgr2=0x7fdc440cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.224+0000 7fdc49c00640 1 -- 172.21.15.19:0/2909582570 shutdown_connections 2024-03-28T13:12:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.224+0000 7fdc49c00640 1 -- 172.21.15.19:0/2909582570 wait complete. 2024-03-28T13:12:49.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.226+0000 7fdc49c00640 1 Processor -- start 2024-03-28T13:12:49.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.226+0000 7fdc49c00640 1 -- start start 2024-03-28T13:12:49.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.227+0000 7fdc49c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc28004800 0x7fdc44148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:49.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.227+0000 7fdc49c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc44161e50 0x7fdc44165700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.227+0000 7fdc49c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441ee2a0 0x7fdc441ee750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.227+0000 7fdc49c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4414be00 con 0x7fdc441ee2a0 2024-03-28T13:12:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.227+0000 7fdc49c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdc440ce0c0 con 0x7fdc44161e50 2024-03-28T13:12:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.227+0000 7fdc49c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdc4414bf80 con 0x7fdc28004800 2024-03-28T13:12:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.227+0000 7fdc49200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441ee2a0 0x7fdc441ee750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.227+0000 7fdc49200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441ee2a0 0x7fdc441ee750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35712/0 (socket says 172.21.15.19:35712) 2024-03-28T13:12:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.227+0000 7fdc49200640 1 -- 172.21.15.19:0/2388278444 learned_addr learned my addr 172.21.15.19:0/2388278444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.227+0000 7fdc43e00640 1 --2- 172.21.15.19:0/2388278444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc28004800 0x7fdc44148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.227+0000 7fdc43400640 1 --2- 172.21.15.19:0/2388278444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc44161e50 0x7fdc44165700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.227+0000 7fdc49200640 1 -- 172.21.15.19:0/2388278444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc28004800 msgr2=0x7fdc44148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.228+0000 7fdc49200640 1 --2- 172.21.15.19:0/2388278444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc28004800 0x7fdc44148890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.228+0000 7fdc49200640 1 -- 172.21.15.19:0/2388278444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc44161e50 msgr2=0x7fdc44165700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.228+0000 7fdc49200640 1 --2- 172.21.15.19:0/2388278444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc44161e50 0x7fdc44165700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.228+0000 7fdc49200640 1 -- 172.21.15.19:0/2388278444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc44162f60 con 0x7fdc441ee2a0 2024-03-28T13:12:49.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.228+0000 7fdc49200640 1 --2- 172.21.15.19:0/2388278444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441ee2a0 0x7fdc441ee750 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdc440cc8c0 tx=0x7fdc3c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:49.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.228+0000 7fdc3be00640 1 -- 172.21.15.19:0/2388278444 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c005d40 con 0x7fdc441ee2a0 2024-03-28T13:12:49.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.228+0000 7fdc3be00640 1 -- 172.21.15.19:0/2388278444 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc3c00a640 con 0x7fdc441ee2a0 2024-03-28T13:12:49.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.228+0000 7fdc3be00640 1 -- 172.21.15.19:0/2388278444 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00aa30 con 0x7fdc441ee2a0 2024-03-28T13:12:49.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.228+0000 7fdc49c00640 1 -- 172.21.15.19:0/2388278444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc44163240 con 0x7fdc441ee2a0 2024-03-28T13:12:49.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.228+0000 7fdc49c00640 1 -- 172.21.15.19:0/2388278444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc44163800 con 0x7fdc441ee2a0 2024-03-28T13:12:49.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.230+0000 7fdc49c00640 1 -- 172.21.15.19:0/2388278444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc44075940 con 0x7fdc441ee2a0 2024-03-28T13:12:49.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.230+0000 7fdc3be00640 1 -- 172.21.15.19:0/2388278444 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdc3c011070 con 0x7fdc441ee2a0 2024-03-28T13:12:49.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.231+0000 7fdc3be00640 1 --2- 172.21.15.19:0/2388278444 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc2406c560 0x7fdc2406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:49.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.232+0000 7fdc3be00640 1 -- 172.21.15.19:0/2388278444 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7fdc3c08a340 con 0x7fdc441ee2a0 2024-03-28T13:12:49.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.232+0000 7fdc43e00640 1 --2- 172.21.15.19:0/2388278444 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc2406c560 0x7fdc2406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:49.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.233+0000 7fdc43e00640 1 --2- 172.21.15.19:0/2388278444 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc2406c560 0x7fdc2406ea20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdc4406a480 tx=0x7fdc34008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:49.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.236+0000 7fdc3be00640 1 -- 172.21.15.19:0/2388278444 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdc3c01b030 con 0x7fdc441ee2a0 2024-03-28T13:12:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.444+0000 7fdc49c00640 1 -- 172.21.15.19:0/2388278444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc44162a80 con 0x7fdc441ee2a0 2024-03-28T13:12:49.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.446+0000 7fdc3be00640 1 -- 172.21.15.19:0/2388278444 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+12891 (secure 0 0 0) 0x7fdc3c058cb0 con 0x7fdc441ee2a0 2024-03-28T13:12:49.446 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:49.446 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":59,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:12:48.419862+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:12:49.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.448+0000 7fdc49c00640 1 -- 172.21.15.19:0/2388278444 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc2406c560 msgr2=0x7fdc2406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.448+0000 7fdc49c00640 1 --2- 172.21.15.19:0/2388278444 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc2406c560 0x7fdc2406ea20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdc4406a480 tx=0x7fdc34008040 comp rx=0 tx=0).stop 2024-03-28T13:12:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.448+0000 7fdc49c00640 1 -- 172.21.15.19:0/2388278444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441ee2a0 msgr2=0x7fdc441ee750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.448+0000 7fdc49c00640 1 --2- 172.21.15.19:0/2388278444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441ee2a0 0x7fdc441ee750 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdc440cc8c0 tx=0x7fdc3c00cb90 comp rx=0 tx=0).stop 2024-03-28T13:12:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.448+0000 7fdc49c00640 1 -- 172.21.15.19:0/2388278444 shutdown_connections 2024-03-28T13:12:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.448+0000 7fdc49c00640 1 --2- 172.21.15.19:0/2388278444 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdc2406c560 0x7fdc2406ea20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.448+0000 7fdc49c00640 1 --2- 172.21.15.19:0/2388278444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdc441ee2a0 0x7fdc441ee750 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.448+0000 7fdc49c00640 1 --2- 172.21.15.19:0/2388278444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc44161e50 0x7fdc44165700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.448+0000 7fdc49c00640 1 --2- 172.21.15.19:0/2388278444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc28004800 0x7fdc44148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.448+0000 7fdc49c00640 1 -- 172.21.15.19:0/2388278444 >> 172.21.15.19:0/2388278444 conn(0x7fdc440ccad0 msgr2=0x7fdc440afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.449+0000 7fdc49c00640 1 -- 172.21.15.19:0/2388278444 shutdown_connections 2024-03-28T13:12:49.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.449+0000 7fdc49c00640 1 -- 172.21.15.19:0/2388278444 wait complete. 2024-03-28T13:12:49.471 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:12:49.471 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:12:49.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.699+0000 7fd06e000640 1 Processor -- start 2024-03-28T13:12:49.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.699+0000 7fd06e000640 1 -- start start 2024-03-28T13:12:49.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.701+0000 7fd06e000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06815bf50 0x7fd0680b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:49.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.701+0000 7fd06e000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd0680b3cf0 0x7fd06806d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:49.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.701+0000 7fd06e000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd06815c330 con 0x7fd06806db60 2024-03-28T13:12:49.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.701+0000 7fd06e000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd06806dff0 con 0x7fd0680b3cf0 2024-03-28T13:12:49.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.701+0000 7fd06e000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd06806ee40 con 0x7fd06815bf50 2024-03-28T13:12:49.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.701+0000 7fd067e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd0680b3cf0 0x7fd06806d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:49.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.701+0000 7fd067e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd0680b3cf0 0x7fd06806d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35728/0 (socket says 172.21.15.19:35728) 2024-03-28T13:12:49.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.701+0000 7fd067e00640 1 -- 172.21.15.19:0/1389696426 learned_addr learned my addr 172.21.15.19:0/1389696426 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:49.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.702+0000 7fd06cc00640 1 --2- 172.21.15.19:0/1389696426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06815bf50 0x7fd0680b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:49.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.702+0000 7fd067e00640 1 -- 172.21.15.19:0/1389696426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06815bf50 msgr2=0x7fd0680b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:49.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.702+0000 7fd067e00640 1 --2- 172.21.15.19:0/1389696426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06815bf50 0x7fd0680b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.702+0000 7fd067e00640 1 -- 172.21.15.19:0/1389696426 >> v1:172.21.15.188:6789/0 conn(0x7fd06806db60 legacy=0x7fd068161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:49.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.702+0000 7fd067e00640 1 -- 172.21.15.19:0/1389696426 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06806e230 con 0x7fd0680b3cf0 2024-03-28T13:12:49.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.702+0000 7fd06cc00640 1 --2- 172.21.15.19:0/1389696426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06815bf50 0x7fd0680b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:49.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.703+0000 7fd067e00640 1 --2- 172.21.15.19:0/1389696426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0680b3cf0 0x7fd06806d620 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd058009870 tx=0x7fd05802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7239d0691a5db6c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:49.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.703+0000 7fd067400640 1 -- 172.21.15.19:0/1389696426 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058034070 con 0x7fd0680b3cf0 2024-03-28T13:12:49.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.703+0000 7fd067400640 1 -- 172.21.15.19:0/1389696426 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd058030840 con 0x7fd0680b3cf0 2024-03-28T13:12:49.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.703+0000 7fd067400640 1 -- 172.21.15.19:0/1389696426 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058030c10 con 0x7fd0680b3cf0 2024-03-28T13:12:49.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.704+0000 7fd06e000640 1 -- 172.21.15.19:0/1389696426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0680b3cf0 msgr2=0x7fd06806d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:49.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.704+0000 7fd06e000640 1 --2- 172.21.15.19:0/1389696426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0680b3cf0 0x7fd06806d620 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd058009870 tx=0x7fd05802b980 comp rx=0 tx=0).stop 2024-03-28T13:12:49.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.704+0000 7fd06e000640 1 -- 172.21.15.19:0/1389696426 shutdown_connections 2024-03-28T13:12:49.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.704+0000 7fd06e000640 1 --2- 172.21.15.19:0/1389696426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0680b3cf0 0x7fd06806d620 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.704+0000 7fd06e000640 1 --2- 172.21.15.19:0/1389696426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06815bf50 0x7fd0680b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.704+0000 7fd06e000640 1 -- 172.21.15.19:0/1389696426 >> 172.21.15.19:0/1389696426 conn(0x7fd0680b1ba0 msgr2=0x7fd0680b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.704+0000 7fd06e000640 1 -- 172.21.15.19:0/1389696426 shutdown_connections 2024-03-28T13:12:49.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.704+0000 7fd06e000640 1 -- 172.21.15.19:0/1389696426 wait complete. 2024-03-28T13:12:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.706+0000 7fd06e000640 1 Processor -- start 2024-03-28T13:12:49.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.706+0000 7fd06e000640 1 -- start start 2024-03-28T13:12:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.707+0000 7fd06e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd06806db60 0x7fd068145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.707+0000 7fd06e000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0680b3cf0 0x7fd0680ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.707+0000 7fd06e000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06815bf50 0x7fd0680c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.707+0000 7fd06e000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd06815ed30 con 0x7fd06806db60 2024-03-28T13:12:49.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.707+0000 7fd06e000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd0680b4130 con 0x7fd0680b3cf0 2024-03-28T13:12:49.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.707+0000 7fd06e000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd06806e7f0 con 0x7fd06815bf50 2024-03-28T13:12:49.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.708+0000 7fd06cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd06806db60 0x7fd068145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:49.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.708+0000 7fd06cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd06806db60 0x7fd068145f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35732/0 (socket says 172.21.15.19:35732) 2024-03-28T13:12:49.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.708+0000 7fd06cc00640 1 -- 172.21.15.19:0/354255193 learned_addr learned my addr 172.21.15.19:0/354255193 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:49.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.708+0000 7fd06d600640 1 --2- 172.21.15.19:0/354255193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06815bf50 0x7fd0680c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:49.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.708+0000 7fd067e00640 1 --2- 172.21.15.19:0/354255193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0680b3cf0 0x7fd0680ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:49.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.708+0000 7fd06cc00640 1 -- 172.21.15.19:0/354255193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06815bf50 msgr2=0x7fd0680c4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:49.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.708+0000 7fd06cc00640 1 --2- 172.21.15.19:0/354255193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06815bf50 0x7fd0680c4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.708+0000 7fd06cc00640 1 -- 172.21.15.19:0/354255193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0680b3cf0 msgr2=0x7fd0680ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.708+0000 7fd06cc00640 1 --2- 172.21.15.19:0/354255193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0680b3cf0 0x7fd0680ae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.708+0000 7fd06cc00640 1 -- 172.21.15.19:0/354255193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0680adeb0 con 0x7fd06806db60 2024-03-28T13:12:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.708+0000 7fd06d600640 1 --2- 172.21.15.19:0/354255193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06815bf50 0x7fd0680c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.709+0000 7fd06cc00640 1 --2- 172.21.15.19:0/354255193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd06806db60 0x7fd068145f00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00c7f0 tx=0x7fd05c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.709+0000 7fd064c00640 1 -- 172.21.15.19:0/354255193 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c015070 con 0x7fd06806db60 2024-03-28T13:12:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.709+0000 7fd064c00640 1 -- 172.21.15.19:0/354255193 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd05c007820 con 0x7fd06806db60 2024-03-28T13:12:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.709+0000 7fd06e000640 1 -- 172.21.15.19:0/354255193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0681495a0 con 0x7fd06806db60 2024-03-28T13:12:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.709+0000 7fd064c00640 1 -- 172.21.15.19:0/354255193 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c007bd0 con 0x7fd06806db60 2024-03-28T13:12:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.709+0000 7fd06e000640 1 -- 172.21.15.19:0/354255193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0680ae450 con 0x7fd06806db60 2024-03-28T13:12:49.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.710+0000 7fd06e000640 1 -- 172.21.15.19:0/354255193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd068075940 con 0x7fd06806db60 2024-03-28T13:12:49.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.711+0000 7fd064c00640 1 -- 172.21.15.19:0/354255193 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd05c018070 con 0x7fd06806db60 2024-03-28T13:12:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.713+0000 7fd064c00640 1 --2- 172.21.15.19:0/354255193 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd04c06c530 0x7fd04c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.713+0000 7fd064c00640 1 -- 172.21.15.19:0/354255193 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7fd05c08b4f0 con 0x7fd06806db60 2024-03-28T13:12:49.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.713+0000 7fd067e00640 1 --2- 172.21.15.19:0/354255193 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd04c06c530 0x7fd04c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:49.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.715+0000 7fd067e00640 1 --2- 172.21.15.19:0/354255193 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd04c06c530 0x7fd04c06e9f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd058036000 tx=0x7fd05803b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:49.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.717+0000 7fd064c00640 1 -- 172.21.15.19:0/354255193 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd05c059e50 con 0x7fd06806db60 2024-03-28T13:12:49.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.923+0000 7fd06e000640 1 -- 172.21.15.19:0/354255193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd06815eeb0 con 0x7fd06806db60 2024-03-28T13:12:49.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.924+0000 7fd064c00640 1 -- 172.21.15.19:0/354255193 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7fd05c059750 con 0x7fd06806db60 2024-03-28T13:12:49.924 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:12:49.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.927+0000 7fd06e000640 1 -- 172.21.15.19:0/354255193 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd04c06c530 msgr2=0x7fd04c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:49.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.927+0000 7fd06e000640 1 --2- 172.21.15.19:0/354255193 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd04c06c530 0x7fd04c06e9f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd058036000 tx=0x7fd05803b040 comp rx=0 tx=0).stop 2024-03-28T13:12:49.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.927+0000 7fd06e000640 1 -- 172.21.15.19:0/354255193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd06806db60 msgr2=0x7fd068145f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:49.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.927+0000 7fd06e000640 1 --2- 172.21.15.19:0/354255193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd06806db60 0x7fd068145f00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00c7f0 tx=0x7fd05c00ccb0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.927+0000 7fd06e000640 1 -- 172.21.15.19:0/354255193 shutdown_connections 2024-03-28T13:12:49.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.927+0000 7fd06e000640 1 --2- 172.21.15.19:0/354255193 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd04c06c530 0x7fd04c06e9f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.927+0000 7fd06e000640 1 --2- 172.21.15.19:0/354255193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd06815bf50 0x7fd0680c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.927+0000 7fd06e000640 1 --2- 172.21.15.19:0/354255193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0680b3cf0 0x7fd0680ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.927+0000 7fd06e000640 1 --2- 172.21.15.19:0/354255193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd06806db60 0x7fd068145f00 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.927+0000 7fd06e000640 1 -- 172.21.15.19:0/354255193 >> 172.21.15.19:0/354255193 conn(0x7fd0680b1ba0 msgr2=0x7fd068162900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.927+0000 7fd06e000640 1 -- 172.21.15.19:0/354255193 shutdown_connections 2024-03-28T13:12:49.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:49.927+0000 7fd06e000640 1 -- 172.21.15.19:0/354255193 wait complete. 2024-03-28T13:12:49.947 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-28T13:12:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.173+0000 7fb105600640 1 Processor -- start 2024-03-28T13:12:50.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.173+0000 7fb105600640 1 -- start start 2024-03-28T13:12:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.175+0000 7fb105600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb100144ad0 0x7fb1000b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.175+0000 7fb105600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb10015fb70 con 0x7fb10015f710 2024-03-28T13:12:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.175+0000 7fb105600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb100144eb0 con 0x7fb100072f50 2024-03-28T13:12:50.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.175+0000 7fb105600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb1000cdad0 con 0x7fb100144ad0 2024-03-28T13:12:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.176+0000 7fb0f7400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb10015f710 0x7fb100072840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44794/0 (socket says 172.21.15.19:44794) 2024-03-28T13:12:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.176+0000 7fb0f7400640 1 -- 172.21.15.19:0/4024798336 learned_addr learned my addr 172.21.15.19:0/4024798336 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.176+0000 7fb0ffe00640 1 --2- 172.21.15.19:0/4024798336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb100144ad0 0x7fb1000b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.176+0000 7fb0ff400640 1 -- 172.21.15.19:0/4024798336 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3478153940 0 0) 0x7fb10015fb70 con 0x7fb10015f710 2024-03-28T13:12:50.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.176+0000 7fb0ff400640 1 -- 172.21.15.19:0/4024798336 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0e4003520 con 0x7fb10015f710 2024-03-28T13:12:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.177+0000 7fb0ff400640 1 -- 172.21.15.19:0/4024798336 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3008172763 0 0) 0x7fb0e4003520 con 0x7fb10015f710 2024-03-28T13:12:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.177+0000 7fb0ff400640 1 -- 172.21.15.19:0/4024798336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb100144ad0 msgr2=0x7fb1000b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.177+0000 7fb0ff400640 1 --2- 172.21.15.19:0/4024798336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb100144ad0 0x7fb1000b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.177+0000 7fb0ff400640 1 -- 172.21.15.19:0/4024798336 >> v1:172.21.15.188:6789/0 conn(0x7fb100072f50 legacy=0x7fb1000cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.177+0000 7fb0ff400640 1 -- 172.21.15.19:0/4024798336 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1000b36f0 con 0x7fb10015f710 2024-03-28T13:12:50.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.177+0000 7fb0ff400640 1 -- 172.21.15.19:0/4024798336 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fb0e80019f0 con 0x7fb10015f710 2024-03-28T13:12:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.177+0000 7fb0ff400640 1 -- 172.21.15.19:0/4024798336 >> v1:172.21.15.19:6789/0 conn(0x7fb10015f710 legacy=0x7fb100072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.177+0000 7fb0ff400640 1 --2- 172.21.15.19:0/4024798336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0e40039e0 0x7fb0e4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.177+0000 7fb0ff400640 1 --2- 172.21.15.19:0/4024798336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb100144ad0 0x7fb0e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.177+0000 7fb0ff400640 1 --2- 172.21.15.19:0/4024798336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0e4004d40 0x7fb0e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.177+0000 7fb0ff400640 1 -- 172.21.15.19:0/4024798336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1000cdad0 con 0x7fb0e4004d40 2024-03-28T13:12:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.177+0000 7fb0ff400640 1 -- 172.21.15.19:0/4024798336 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e400ba70 con 0x7fb0e40039e0 2024-03-28T13:12:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.177+0000 7fb0ff400640 1 -- 172.21.15.19:0/4024798336 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb0e400bc70 con 0x7fb100144ad0 2024-03-28T13:12:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.177+0000 7fb104c00640 1 --2- 172.21.15.19:0/4024798336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0e4004d40 0x7fb0e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.178+0000 7fb0ffe00640 1 --2- 172.21.15.19:0/4024798336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0e40039e0 0x7fb0e4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.178+0000 7fb0f7400640 1 --2- 172.21.15.19:0/4024798336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb100144ad0 0x7fb0e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.178+0000 7fb104c00640 1 -- 172.21.15.19:0/4024798336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb100144ad0 msgr2=0x7fb0e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.178+0000 7fb104c00640 1 --2- 172.21.15.19:0/4024798336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb100144ad0 0x7fb0e4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.178+0000 7fb104c00640 1 -- 172.21.15.19:0/4024798336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0e40039e0 msgr2=0x7fb0e4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.178+0000 7fb104c00640 1 --2- 172.21.15.19:0/4024798336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0e40039e0 0x7fb0e4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:50.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.178+0000 7fb104c00640 1 -- 172.21.15.19:0/4024798336 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e4003520 con 0x7fb0e4004d40 2024-03-28T13:12:50.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.178+0000 7fb0f7400640 1 --2- 172.21.15.19:0/4024798336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb100144ad0 0x7fb0e4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.179+0000 7fb104c00640 1 --2- 172.21.15.19:0/4024798336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0e4004d40 0x7fb0e40051c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb10006ffd0 tx=0x7fb0e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4634986635318e18 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.179+0000 7fb0ff400640 1 -- 172.21.15.19:0/4024798336 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8001370 con 0x7fb0e4004d40 2024-03-28T13:12:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.179+0000 7fb0ff400640 1 -- 172.21.15.19:0/4024798336 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f80052d0 con 0x7fb0e4004d40 2024-03-28T13:12:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.179+0000 7fb0ff400640 1 -- 172.21.15.19:0/4024798336 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f8003830 con 0x7fb0e4004d40 2024-03-28T13:12:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.180+0000 7fb105600640 1 -- 172.21.15.19:0/4024798336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0e4004d40 msgr2=0x7fb0e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.180+0000 7fb105600640 1 --2- 172.21.15.19:0/4024798336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0e4004d40 0x7fb0e40051c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb10006ffd0 tx=0x7fb0e4005830 comp rx=0 tx=0).stop 2024-03-28T13:12:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.180+0000 7fb104c00640 1 -- 172.21.15.19:0/4024798336 reap_dead start 2024-03-28T13:12:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.180+0000 7fb105600640 1 -- 172.21.15.19:0/4024798336 shutdown_connections 2024-03-28T13:12:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.180+0000 7fb105600640 1 -- 172.21.15.19:0/4024798336 >> 172.21.15.19:0/4024798336 conn(0x7fb10006b570 msgr2=0x7fb10006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.180+0000 7fb105600640 1 -- 172.21.15.19:0/4024798336 shutdown_connections 2024-03-28T13:12:50.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.180+0000 7fb105600640 1 -- 172.21.15.19:0/4024798336 wait complete. 2024-03-28T13:12:50.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.182+0000 7fb105600640 1 Processor -- start 2024-03-28T13:12:50.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.182+0000 7fb105600640 1 -- start start 2024-03-28T13:12:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.183+0000 7fb105600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1001356c0 0x7fb1000abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.183+0000 7fb105600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1000ac310 0x7fb1000d0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.183+0000 7fb105600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb100135aa0 0x7fb1000d0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.183+0000 7fb105600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1000b3f90 con 0x7fb1000ac310 2024-03-28T13:12:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.183+0000 7fb105600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1000b3cb0 con 0x7fb1001356c0 2024-03-28T13:12:50.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.183+0000 7fb105600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb1000b2ff0 con 0x7fb100135aa0 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.183+0000 7fb0f7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1000ac310 0x7fb1000d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.183+0000 7fb0f7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1000ac310 0x7fb1000d0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35744/0 (socket says 172.21.15.19:35744) 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.183+0000 7fb0f7400640 1 -- 172.21.15.19:0/903859358 learned_addr learned my addr 172.21.15.19:0/903859358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.183+0000 7fb104c00640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb100135aa0 0x7fb1000d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.183+0000 7fb0ffe00640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1001356c0 0x7fb1000abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.183+0000 7fb0f7400640 1 -- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb100135aa0 msgr2=0x7fb1000d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.184+0000 7fb0f7400640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb100135aa0 0x7fb1000d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.184+0000 7fb0f7400640 1 -- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1001356c0 msgr2=0x7fb1000abdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.184+0000 7fb0f7400640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1001356c0 0x7fb1000abdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.184+0000 7fb0f7400640 1 -- 172.21.15.19:0/903859358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1000a9520 con 0x7fb1000ac310 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.184+0000 7fb104c00640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb100135aa0 0x7fb1000d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.184+0000 7fb0ffe00640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1001356c0 0x7fb1000abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.184+0000 7fb0f7400640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1000ac310 0x7fb1000d0040 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8002230 tx=0x7fb0e800b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.184+0000 7fb0fcc00640 1 -- 172.21.15.19:0/903859358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8019070 con 0x7fb1000ac310 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.184+0000 7fb0fcc00640 1 -- 172.21.15.19:0/903859358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e80072d0 con 0x7fb1000ac310 2024-03-28T13:12:50.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.184+0000 7fb0fcc00640 1 -- 172.21.15.19:0/903859358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80076e0 con 0x7fb1000ac310 2024-03-28T13:12:50.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.184+0000 7fb105600640 1 -- 172.21.15.19:0/903859358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1000ac7b0 con 0x7fb1000ac310 2024-03-28T13:12:50.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.184+0000 7fb105600640 1 -- 172.21.15.19:0/903859358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1000aca90 con 0x7fb1000ac310 2024-03-28T13:12:50.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.186+0000 7fb105600640 1 -- 172.21.15.19:0/903859358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb100075940 con 0x7fb1000ac310 2024-03-28T13:12:50.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.186+0000 7fb0fcc00640 1 -- 172.21.15.19:0/903859358 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb0e8007880 con 0x7fb1000ac310 2024-03-28T13:12:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.187+0000 7fb0fcc00640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb0dc06c410 0x7fb0dc06e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.187+0000 7fb0fcc00640 1 -- 172.21.15.19:0/903859358 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7fb0e808ba40 con 0x7fb1000ac310 2024-03-28T13:12:50.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.188+0000 7fb0ffe00640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb0dc06c410 0x7fb0dc06e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:50.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.189+0000 7fb0ffe00640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb0dc06c410 0x7fb0dc06e8d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009a00 tx=0x7fb0f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:50.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.192+0000 7fb0fcc00640 1 -- 172.21.15.19:0/903859358 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb0e805a3a0 con 0x7fb1000ac310 2024-03-28T13:12:50.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.396+0000 7fb105600640 1 -- 172.21.15.19:0/903859358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb1000b4240 con 0x7fb1000ac310 2024-03-28T13:12:50.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.397+0000 7fb0fcc00640 1 -- 172.21.15.19:0/903859358 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v59) ==== 84+0+65 (secure 0 0 0) 0x7fb0e8059ca0 con 0x7fb1000ac310 2024-03-28T13:12:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:50.397 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-03-28T13:12:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:12:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-03-28T13:12:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-03-28T13:12:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:12:50.398 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-03-28T13:12:50.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.400+0000 7fb105600640 1 -- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb0dc06c410 msgr2=0x7fb0dc06e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:50.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.400+0000 7fb105600640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb0dc06c410 0x7fb0dc06e8d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009a00 tx=0x7fb0f0008040 comp rx=0 tx=0).stop 2024-03-28T13:12:50.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.400+0000 7fb105600640 1 -- 172.21.15.19:0/903859358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1000ac310 msgr2=0x7fb1000d0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.400+0000 7fb105600640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1000ac310 0x7fb1000d0040 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8002230 tx=0x7fb0e800b560 comp rx=0 tx=0).stop 2024-03-28T13:12:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.400+0000 7fb105600640 1 -- 172.21.15.19:0/903859358 shutdown_connections 2024-03-28T13:12:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.400+0000 7fb105600640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb0dc06c410 0x7fb0dc06e8d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.400+0000 7fb105600640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb100135aa0 0x7fb1000d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.400+0000 7fb105600640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1000ac310 0x7fb1000d0040 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.400+0000 7fb105600640 1 --2- 172.21.15.19:0/903859358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1001356c0 0x7fb1000abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.400+0000 7fb105600640 1 -- 172.21.15.19:0/903859358 >> 172.21.15.19:0/903859358 conn(0x7fb10006b570 msgr2=0x7fb10015dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.400+0000 7fb105600640 1 -- 172.21.15.19:0/903859358 shutdown_connections 2024-03-28T13:12:50.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.401+0000 7fb105600640 1 -- 172.21.15.19:0/903859358 wait complete. 2024-03-28T13:12:50.419 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-28T13:12:50.419 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-28T13:12:50.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.649+0000 7f917ac00640 1 Processor -- start 2024-03-28T13:12:50.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.649+0000 7f917ac00640 1 -- start start 2024-03-28T13:12:50.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.651+0000 7f917ac00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f917415e4d0 0x7f917406e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.651+0000 7f917ac00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9174071e40 con 0x7f917406f6e0 2024-03-28T13:12:50.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.651+0000 7f917ac00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9174072120 con 0x7f9174071610 2024-03-28T13:12:50.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.651+0000 7f917ac00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9174072400 con 0x7f917415e4d0 2024-03-28T13:12:50.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.651+0000 7f9178c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f917406f6e0 0x7f91740acf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44806/0 (socket says 172.21.15.19:44806) 2024-03-28T13:12:50.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.652+0000 7f9178c00640 1 -- 172.21.15.19:0/380902059 learned_addr learned my addr 172.21.15.19:0/380902059 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:50.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.652+0000 7f9173e00640 1 --2- 172.21.15.19:0/380902059 >> v2:172.21.15.188:3300/0 conn(0x7f917415e4d0 0x7f917406e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:50.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.652+0000 7f9173400640 1 -- 172.21.15.19:0/380902059 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4124604664 0 0) 0x7f9174071e40 con 0x7f917406f6e0 2024-03-28T13:12:50.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.652+0000 7f9173400640 1 -- 172.21.15.19:0/380902059 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9158003520 con 0x7f917406f6e0 2024-03-28T13:12:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.652+0000 7f9173400640 1 -- 172.21.15.19:0/380902059 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 236919479 0 0) 0x7f9158003520 con 0x7f917406f6e0 2024-03-28T13:12:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.652+0000 7f9173400640 1 -- 172.21.15.19:0/380902059 >> v2:172.21.15.188:3300/0 conn(0x7f917415e4d0 msgr2=0x7f917406e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.652+0000 7f9173400640 1 --2- 172.21.15.19:0/380902059 >> v2:172.21.15.188:3300/0 conn(0x7f917415e4d0 0x7f917406e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.652+0000 7f9173400640 1 -- 172.21.15.19:0/380902059 >> v1:172.21.15.188:6789/0 conn(0x7f9174071610 legacy=0x7f917406efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.652+0000 7f9173400640 1 -- 172.21.15.19:0/380902059 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91740726e0 con 0x7f917406f6e0 2024-03-28T13:12:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.652+0000 7f9173400640 1 -- 172.21.15.19:0/380902059 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f916c0019f0 con 0x7f917406f6e0 2024-03-28T13:12:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.653+0000 7f9173400640 1 -- 172.21.15.19:0/380902059 >> v1:172.21.15.19:6789/0 conn(0x7f917406f6e0 legacy=0x7f91740acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.653+0000 7f9173400640 1 --2- 172.21.15.19:0/380902059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91580039b0 0x7f9158003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.653+0000 7f9173400640 1 --2- 172.21.15.19:0/380902059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9158004800 0x7f9158004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.653+0000 7f9173400640 1 --2- 172.21.15.19:0/380902059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91580051c0 0x7f915800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.653+0000 7f9173400640 1 -- 172.21.15.19:0/380902059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9174072400 con 0x7f91580039b0 2024-03-28T13:12:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.653+0000 7f9173400640 1 -- 172.21.15.19:0/380902059 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f915800c160 con 0x7f91580051c0 2024-03-28T13:12:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.653+0000 7f9173400640 1 -- 172.21.15.19:0/380902059 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f915800c360 con 0x7f9158004800 2024-03-28T13:12:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.653+0000 7f9173e00640 1 --2- 172.21.15.19:0/380902059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91580039b0 0x7f9158003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:50.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.653+0000 7f9178c00640 1 --2- 172.21.15.19:0/380902059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91580051c0 0x7f915800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:50.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.653+0000 7f9173e00640 1 -- 172.21.15.19:0/380902059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9158004800 msgr2=0x7f9158004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:50.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.653+0000 7f9173e00640 1 --2- 172.21.15.19:0/380902059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9158004800 0x7f9158004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:50.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.653+0000 7f9173e00640 1 -- 172.21.15.19:0/380902059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91580051c0 msgr2=0x7f915800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:50.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.653+0000 7f9173e00640 1 --2- 172.21.15.19:0/380902059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91580051c0 0x7f915800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:50.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.653+0000 7f9173e00640 1 -- 172.21.15.19:0/380902059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9158003520 con 0x7f91580039b0 2024-03-28T13:12:50.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.654+0000 7f9178c00640 1 --2- 172.21.15.19:0/380902059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91580051c0 0x7f915800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:50.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.654+0000 7f9178c00640 1 -- 172.21.15.19:0/380902059 reap_dead start 2024-03-28T13:12:50.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.655+0000 7f9173e00640 1 --2- 172.21.15.19:0/380902059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91580039b0 0x7f9158003dd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9164009ac0 tx=0x7f9158005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101d5b43ae303e31 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:50.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.655+0000 7f9173400640 1 -- 172.21.15.19:0/380902059 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164039070 con 0x7f91580039b0 2024-03-28T13:12:50.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.655+0000 7f9173400640 1 -- 172.21.15.19:0/380902059 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f916402c6b0 con 0x7f91580039b0 2024-03-28T13:12:50.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.655+0000 7f9173400640 1 -- 172.21.15.19:0/380902059 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164009690 con 0x7f91580039b0 2024-03-28T13:12:50.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.655+0000 7f917ac00640 1 -- 172.21.15.19:0/380902059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91580039b0 msgr2=0x7f9158003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:50.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.655+0000 7f917ac00640 1 --2- 172.21.15.19:0/380902059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91580039b0 0x7f9158003dd0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f9164009ac0 tx=0x7f9158005830 comp rx=0 tx=0).stop 2024-03-28T13:12:50.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.655+0000 7f917ac00640 1 -- 172.21.15.19:0/380902059 shutdown_connections 2024-03-28T13:12:50.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.655+0000 7f917ac00640 1 --2- 172.21.15.19:0/380902059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91580039b0 0x7f9158003dd0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:50.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.655+0000 7f917ac00640 1 -- 172.21.15.19:0/380902059 >> 172.21.15.19:0/380902059 conn(0x7f917406bba0 msgr2=0x7f91740b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.656+0000 7f917ac00640 1 -- 172.21.15.19:0/380902059 shutdown_connections 2024-03-28T13:12:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.656+0000 7f917ac00640 1 -- 172.21.15.19:0/380902059 wait complete. 2024-03-28T13:12:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.657+0000 7f917ac00640 1 Processor -- start 2024-03-28T13:12:50.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.658+0000 7f917ac00640 1 -- start start 2024-03-28T13:12:50.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.658+0000 7f917ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91580039b0 0x7f91740ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.658+0000 7f917ac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f917415eac0 0x7f91741529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.658+0000 7f917ac00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f917409b3e0 0x7f91741542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.659+0000 7f917ac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9174072ee0 con 0x7f917415eac0 2024-03-28T13:12:50.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.659+0000 7f917ac00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f917415e8b0 con 0x7f917409b3e0 2024-03-28T13:12:50.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.659+0000 7f917ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9174071aa0 con 0x7f91580039b0 2024-03-28T13:12:50.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.659+0000 7f916be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f917415eac0 0x7f91741529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:50.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.659+0000 7f916be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f917415eac0 0x7f91741529b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35748/0 (socket says 172.21.15.19:35748) 2024-03-28T13:12:50.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.659+0000 7f916be00640 1 -- 172.21.15.19:0/1034689038 learned_addr learned my addr 172.21.15.19:0/1034689038 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:50.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.659+0000 7f916be00640 1 -- 172.21.15.19:0/1034689038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91580039b0 msgr2=0x7f91740ae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:50.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.659+0000 7f916be00640 1 --2- 172.21.15.19:0/1034689038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91580039b0 0x7f91740ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:50.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.659+0000 7f916be00640 1 -- 172.21.15.19:0/1034689038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f917409b3e0 msgr2=0x7f91741542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:50.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.659+0000 7f916be00640 1 --2- 172.21.15.19:0/1034689038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f917409b3e0 0x7f91741542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:50.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.659+0000 7f916be00640 1 -- 172.21.15.19:0/1034689038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917409bb50 con 0x7f917415eac0 2024-03-28T13:12:50.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.659+0000 7f916be00640 1 --2- 172.21.15.19:0/1034689038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f917415eac0 0x7f91741529b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c530 tx=0x7f915c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:50.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.660+0000 7f9170c00640 1 -- 172.21.15.19:0/1034689038 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c0052d0 con 0x7f917415eac0 2024-03-28T13:12:50.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.660+0000 7f9170c00640 1 -- 172.21.15.19:0/1034689038 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f915c005470 con 0x7f917415eac0 2024-03-28T13:12:50.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.660+0000 7f9170c00640 1 -- 172.21.15.19:0/1034689038 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c012850 con 0x7f917415eac0 2024-03-28T13:12:50.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.660+0000 7f917ac00640 1 -- 172.21.15.19:0/1034689038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917409be30 con 0x7f917415eac0 2024-03-28T13:12:50.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.660+0000 7f917ac00640 1 -- 172.21.15.19:0/1034689038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f917409c3f0 con 0x7f917415eac0 2024-03-28T13:12:50.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.661+0000 7f917ac00640 1 -- 172.21.15.19:0/1034689038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91740685c0 con 0x7f917415eac0 2024-03-28T13:12:50.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.662+0000 7f9170c00640 1 -- 172.21.15.19:0/1034689038 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f915c0129f0 con 0x7f917415eac0 2024-03-28T13:12:50.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.663+0000 7f9170c00640 1 --2- 172.21.15.19:0/1034689038 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f915006c530 0x7f915006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:50.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.663+0000 7f9170c00640 1 -- 172.21.15.19:0/1034689038 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7f915c08b000 con 0x7f917415eac0 2024-03-28T13:12:50.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.664+0000 7f9173e00640 1 --2- 172.21.15.19:0/1034689038 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f915006c530 0x7f915006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:50.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.665+0000 7f9173e00640 1 --2- 172.21.15.19:0/1034689038 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f915006c530 0x7f915006e9f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9164036000 tx=0x7f916402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:50.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.668+0000 7f9170c00640 1 -- 172.21.15.19:0/1034689038 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f915c059a00 con 0x7f917415eac0 2024-03-28T13:12:50.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:50.870+0000 7f917ac00640 1 -- 172.21.15.19:0/1034689038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f91740af870 con 0x7f917415eac0 2024-03-28T13:12:51.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.449+0000 7f9170c00640 1 -- 172.21.15.19:0/1034689038 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v60) ==== 138+0+0 (secure 0 0 0) 0x7f915c059300 con 0x7f917415eac0 2024-03-28T13:12:51.450 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-03-28T13:12:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.453+0000 7f917ac00640 1 -- 172.21.15.19:0/1034689038 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f915006c530 msgr2=0x7f915006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.453+0000 7f917ac00640 1 --2- 172.21.15.19:0/1034689038 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f915006c530 0x7f915006e9f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9164036000 tx=0x7f916402e000 comp rx=0 tx=0).stop 2024-03-28T13:12:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.453+0000 7f917ac00640 1 -- 172.21.15.19:0/1034689038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f917415eac0 msgr2=0x7f91741529b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:51.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.453+0000 7f917ac00640 1 --2- 172.21.15.19:0/1034689038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f917415eac0 0x7f91741529b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f915c00c530 tx=0x7f915c00c9f0 comp rx=0 tx=0).stop 2024-03-28T13:12:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.453+0000 7f917ac00640 1 -- 172.21.15.19:0/1034689038 shutdown_connections 2024-03-28T13:12:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.453+0000 7f917ac00640 1 --2- 172.21.15.19:0/1034689038 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f915006c530 0x7f915006e9f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.453+0000 7f917ac00640 1 --2- 172.21.15.19:0/1034689038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f917409b3e0 0x7f91741542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.453+0000 7f917ac00640 1 --2- 172.21.15.19:0/1034689038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f917415eac0 0x7f91741529b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.453+0000 7f917ac00640 1 --2- 172.21.15.19:0/1034689038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91580039b0 0x7f91740ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.453+0000 7f917ac00640 1 -- 172.21.15.19:0/1034689038 >> 172.21.15.19:0/1034689038 conn(0x7f917406bba0 msgr2=0x7f9174158c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.453+0000 7f917ac00640 1 -- 172.21.15.19:0/1034689038 shutdown_connections 2024-03-28T13:12:51.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.453+0000 7f917ac00640 1 -- 172.21.15.19:0/1034689038 wait complete. 2024-03-28T13:12:51.483 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-28T13:12:51.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.702+0000 7f97f6400640 1 Processor -- start 2024-03-28T13:12:51.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.702+0000 7f97f6400640 1 -- start start 2024-03-28T13:12:51.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.704+0000 7f97f6400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f97f00d0280 0x7f97f00d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:51.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.704+0000 7f97f6400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f0141c10 0x7f97f0071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:51.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.704+0000 7f97f6400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97f015ec80 con 0x7f97f0071eb0 2024-03-28T13:12:51.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.704+0000 7f97f6400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f97f00d0ba0 con 0x7f97f00d0280 2024-03-28T13:12:51.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.704+0000 7f97f6400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f97f015ef20 con 0x7f97f0141c10 2024-03-28T13:12:51.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.705+0000 7f97e6a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f0141c10 0x7f97f0071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:51.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.705+0000 7f97ef400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f97f00d0280 0x7f97f00d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:51.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.705+0000 7f97e6a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f0141c10 0x7f97f0071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:53804/0 (socket says 172.21.15.19:53804) 2024-03-28T13:12:51.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.705+0000 7f97e6a00640 1 -- 172.21.15.19:0/1128398840 learned_addr learned my addr 172.21.15.19:0/1128398840 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:51.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.706+0000 7f97ef400640 1 -- 172.21.15.19:0/1128398840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f0141c10 msgr2=0x7f97f0071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:51.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.706+0000 7f97ef400640 1 --2- 172.21.15.19:0/1128398840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f0141c10 0x7f97f0071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:51.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.706+0000 7f97ef400640 1 -- 172.21.15.19:0/1128398840 >> v1:172.21.15.188:6789/0 conn(0x7f97f0071eb0 legacy=0x7f97f006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:51.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.706+0000 7f97ef400640 1 -- 172.21.15.19:0/1128398840 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f015f200 con 0x7f97f00d0280 2024-03-28T13:12:51.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.707+0000 7f97ef400640 1 --2- 172.21.15.19:0/1128398840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97f00d0280 0x7f97f00d0660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f97e00097a0 tx=0x7f97e002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae7fd7f8dc7b9929 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.708+0000 7f97eea00640 1 -- 172.21.15.19:0/1128398840 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0035020 con 0x7f97f00d0280 2024-03-28T13:12:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.708+0000 7f97eea00640 1 -- 172.21.15.19:0/1128398840 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e0031830 con 0x7f97f00d0280 2024-03-28T13:12:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.708+0000 7f97eea00640 1 -- 172.21.15.19:0/1128398840 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0031c00 con 0x7f97f00d0280 2024-03-28T13:12:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.709+0000 7f97f6400640 1 -- 172.21.15.19:0/1128398840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97f00d0280 msgr2=0x7f97f00d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.709+0000 7f97f6400640 1 --2- 172.21.15.19:0/1128398840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97f00d0280 0x7f97f00d0660 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f97e00097a0 tx=0x7f97e002b8d0 comp rx=0 tx=0).stop 2024-03-28T13:12:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.709+0000 7f97f6400640 1 -- 172.21.15.19:0/1128398840 shutdown_connections 2024-03-28T13:12:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.709+0000 7f97f6400640 1 --2- 172.21.15.19:0/1128398840 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f0141c10 0x7f97f0071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.709+0000 7f97f6400640 1 --2- 172.21.15.19:0/1128398840 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97f00d0280 0x7f97f00d0660 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.709+0000 7f97f6400640 1 -- 172.21.15.19:0/1128398840 >> 172.21.15.19:0/1128398840 conn(0x7f97f006b750 msgr2=0x7f97f006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:51.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.709+0000 7f97f6400640 1 -- 172.21.15.19:0/1128398840 shutdown_connections 2024-03-28T13:12:51.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.709+0000 7f97f6400640 1 -- 172.21.15.19:0/1128398840 wait complete. 2024-03-28T13:12:51.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.711+0000 7f97f6400640 1 Processor -- start 2024-03-28T13:12:51.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.711+0000 7f97f6400640 1 -- start start 2024-03-28T13:12:51.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.712+0000 7f97f6400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f0071eb0 0x7f97f01528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:51.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.712+0000 7f97f6400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f00d0280 0x7f97f009dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:51.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.712+0000 7f97f6400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97f0141c10 0x7f97f00ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:51.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.712+0000 7f97f6400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97f0158e90 con 0x7f97f0071eb0 2024-03-28T13:12:51.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.712+0000 7f97f6400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97f015f7c0 con 0x7f97f0141c10 2024-03-28T13:12:51.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.712+0000 7f97f6400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f97f00aec30 con 0x7f97f00d0280 2024-03-28T13:12:51.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.712+0000 7f97ef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f0071eb0 0x7f97f01528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:51.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.713+0000 7f97ef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f0071eb0 0x7f97f01528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35762/0 (socket says 172.21.15.19:35762) 2024-03-28T13:12:51.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.713+0000 7f97ef400640 1 -- 172.21.15.19:0/1709151714 learned_addr learned my addr 172.21.15.19:0/1709151714 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:51.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.713+0000 7f97e6a00640 1 --2- 172.21.15.19:0/1709151714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f00d0280 0x7f97f009dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:51.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.713+0000 7f97efe00640 1 --2- 172.21.15.19:0/1709151714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97f0141c10 0x7f97f00ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:51.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.713+0000 7f97ef400640 1 -- 172.21.15.19:0/1709151714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f00d0280 msgr2=0x7f97f009dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:51.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.713+0000 7f97ef400640 1 --2- 172.21.15.19:0/1709151714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f00d0280 0x7f97f009dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:51.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.713+0000 7f97ef400640 1 -- 172.21.15.19:0/1709151714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97f0141c10 msgr2=0x7f97f00ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:51.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.713+0000 7f97ef400640 1 --2- 172.21.15.19:0/1709151714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97f0141c10 0x7f97f00ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:51.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.713+0000 7f97ef400640 1 -- 172.21.15.19:0/1709151714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f00aeea0 con 0x7f97f0071eb0 2024-03-28T13:12:51.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.714+0000 7f97ef400640 1 --2- 172.21.15.19:0/1709151714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f0071eb0 0x7f97f01528e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009890 tx=0x7f97e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:51.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.714+0000 7f97e7e00640 1 -- 172.21.15.19:0/1709151714 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0035040 con 0x7f97f0071eb0 2024-03-28T13:12:51.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.714+0000 7f97e7e00640 1 -- 172.21.15.19:0/1709151714 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e003b920 con 0x7f97f0071eb0 2024-03-28T13:12:51.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.714+0000 7f97f6400640 1 -- 172.21.15.19:0/1709151714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f00af180 con 0x7f97f0071eb0 2024-03-28T13:12:51.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.714+0000 7f97e7e00640 1 -- 172.21.15.19:0/1709151714 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e003bd10 con 0x7f97f0071eb0 2024-03-28T13:12:51.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.714+0000 7f97f6400640 1 -- 172.21.15.19:0/1709151714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f01533b0 con 0x7f97f0071eb0 2024-03-28T13:12:51.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.716+0000 7f97f6400640 1 -- 172.21.15.19:0/1709151714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f00685c0 con 0x7f97f0071eb0 2024-03-28T13:12:51.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.717+0000 7f97e7e00640 1 -- 172.21.15.19:0/1709151714 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f97e0038020 con 0x7f97f0071eb0 2024-03-28T13:12:51.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.718+0000 7f97e7e00640 1 --2- 172.21.15.19:0/1709151714 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f97cc06c530 0x7f97cc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:51.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.718+0000 7f97e7e00640 1 -- 172.21.15.19:0/1709151714 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5227+0+0 (secure 0 0 0) 0x7f97e00ad5f0 con 0x7f97f0071eb0 2024-03-28T13:12:51.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.718+0000 7f97e6a00640 1 --2- 172.21.15.19:0/1709151714 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f97cc06c530 0x7f97cc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:51.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.720+0000 7f97e6a00640 1 --2- 172.21.15.19:0/1709151714 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f97cc06c530 0x7f97cc06e9f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f97f00afa60 tx=0x7f97d8009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:51.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.724+0000 7f97e7e00640 1 -- 172.21.15.19:0/1709151714 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f97e007be00 con 0x7f97f0071eb0 2024-03-28T13:12:51.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:51.930+0000 7f97f6400640 1 -- 172.21.15.19:0/1709151714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f97f00b1940 con 0x7f97f0071eb0 2024-03-28T13:12:52.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.455+0000 7f97e7e00640 1 -- 172.21.15.19:0/1709151714 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v61) ==== 170+0+0 (secure 0 0 0) 0x7f97e0031800 con 0x7f97f0071eb0 2024-03-28T13:12:52.456 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-03-28T13:12:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.460+0000 7f97f6400640 1 -- 172.21.15.19:0/1709151714 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f97cc06c530 msgr2=0x7f97cc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.460+0000 7f97f6400640 1 --2- 172.21.15.19:0/1709151714 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f97cc06c530 0x7f97cc06e9f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f97f00afa60 tx=0x7f97d8009240 comp rx=0 tx=0).stop 2024-03-28T13:12:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.460+0000 7f97f6400640 1 -- 172.21.15.19:0/1709151714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f0071eb0 msgr2=0x7f97f01528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.460+0000 7f97f6400640 1 --2- 172.21.15.19:0/1709151714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f0071eb0 0x7f97f01528e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009890 tx=0x7f97e00046c0 comp rx=0 tx=0).stop 2024-03-28T13:12:52.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.461+0000 7f97f6400640 1 -- 172.21.15.19:0/1709151714 shutdown_connections 2024-03-28T13:12:52.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.461+0000 7f97f6400640 1 --2- 172.21.15.19:0/1709151714 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f97cc06c530 0x7f97cc06e9f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:52.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.461+0000 7f97f6400640 1 --2- 172.21.15.19:0/1709151714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97f0141c10 0x7f97f00ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:52.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.461+0000 7f97f6400640 1 --2- 172.21.15.19:0/1709151714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97f00d0280 0x7f97f009dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:52.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.461+0000 7f97f6400640 1 --2- 172.21.15.19:0/1709151714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97f0071eb0 0x7f97f01528e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:52.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.461+0000 7f97f6400640 1 -- 172.21.15.19:0/1709151714 >> 172.21.15.19:0/1709151714 conn(0x7f97f006b750 msgr2=0x7f97f0147280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:52.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.461+0000 7f97f6400640 1 -- 172.21.15.19:0/1709151714 shutdown_connections 2024-03-28T13:12:52.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.461+0000 7f97f6400640 1 -- 172.21.15.19:0/1709151714 wait complete. 2024-03-28T13:12:52.501 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-28T13:12:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.683+0000 7ff537400640 1 Processor -- start 2024-03-28T13:12:52.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.683+0000 7ff537400640 1 -- start start 2024-03-28T13:12:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.685+0000 7ff537400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff530153580 0x7ff5300d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.685+0000 7ff537400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5300af4b0 0x7ff5301600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.685+0000 7ff537400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5300afab0 con 0x7ff5300d07c0 2024-03-28T13:12:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.685+0000 7ff537400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff5300d0c00 con 0x7ff5300af4b0 2024-03-28T13:12:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.685+0000 7ff537400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff53015cc80 con 0x7ff530153580 2024-03-28T13:12:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.686+0000 7ff534a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff530153580 0x7ff5300d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.686+0000 7ff535400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5300af4b0 0x7ff5301600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.686+0000 7ff52fe00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ff5300d07c0 0x7ff5300aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:34736/0 (socket says 172.21.15.19:34736) 2024-03-28T13:12:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.686+0000 7ff52fe00640 1 -- 172.21.15.19:0/1561276677 learned_addr learned my addr 172.21.15.19:0/1561276677 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.686+0000 7ff535400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5300af4b0 0x7ff5301600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:38344/0 (socket says 172.21.15.19:38344) 2024-03-28T13:12:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.686+0000 7ff52f400640 1 -- 172.21.15.19:0/1561276677 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 681016002 0 0) 0x7ff5300afab0 con 0x7ff5300d07c0 2024-03-28T13:12:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.686+0000 7ff52f400640 1 -- 172.21.15.19:0/1561276677 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff514003520 con 0x7ff5300d07c0 2024-03-28T13:12:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.687+0000 7ff535400640 1 -- 172.21.15.19:0/1561276677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff530153580 msgr2=0x7ff5300d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.687+0000 7ff535400640 1 --2- 172.21.15.19:0/1561276677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff530153580 0x7ff5300d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.687+0000 7ff535400640 1 -- 172.21.15.19:0/1561276677 >> v1:172.21.15.188:6789/0 conn(0x7ff5300d07c0 legacy=0x7ff5300aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.687+0000 7ff535400640 1 -- 172.21.15.19:0/1561276677 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53015da00 con 0x7ff5300af4b0 2024-03-28T13:12:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.688+0000 7ff535400640 1 --2- 172.21.15.19:0/1561276677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5300af4b0 0x7ff5301600d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff5280097a0 tx=0x7ff52802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74163ee9f0a1acf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.689+0000 7ff52f400640 1 -- 172.21.15.19:0/1561276677 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5280315a0 con 0x7ff5300af4b0 2024-03-28T13:12:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.689+0000 7ff52f400640 1 -- 172.21.15.19:0/1561276677 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff528037070 con 0x7ff5300af4b0 2024-03-28T13:12:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.689+0000 7ff52f400640 1 -- 172.21.15.19:0/1561276677 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff528031960 con 0x7ff5300af4b0 2024-03-28T13:12:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.690+0000 7ff537400640 1 -- 172.21.15.19:0/1561276677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5300af4b0 msgr2=0x7ff5301600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.690+0000 7ff537400640 1 --2- 172.21.15.19:0/1561276677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5300af4b0 0x7ff5301600d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff5280097a0 tx=0x7ff52802b840 comp rx=0 tx=0).stop 2024-03-28T13:12:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.690+0000 7ff537400640 1 -- 172.21.15.19:0/1561276677 shutdown_connections 2024-03-28T13:12:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.690+0000 7ff537400640 1 --2- 172.21.15.19:0/1561276677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5300af4b0 0x7ff5301600d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.690+0000 7ff537400640 1 --2- 172.21.15.19:0/1561276677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff530153580 0x7ff5300d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.690+0000 7ff537400640 1 -- 172.21.15.19:0/1561276677 >> 172.21.15.19:0/1561276677 conn(0x7ff53006b750 msgr2=0x7ff53006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.690+0000 7ff537400640 1 -- 172.21.15.19:0/1561276677 shutdown_connections 2024-03-28T13:12:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.690+0000 7ff537400640 1 -- 172.21.15.19:0/1561276677 wait complete. 2024-03-28T13:12:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.691+0000 7ff537400640 1 Processor -- start 2024-03-28T13:12:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.692+0000 7ff537400640 1 -- start start 2024-03-28T13:12:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.692+0000 7ff537400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5300af4b0 0x7ff53014b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.693+0000 7ff537400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5300d07c0 0x7ff5301503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.693+0000 7ff537400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff530153580 0x7ff5300cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.693+0000 7ff537400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff53015d4e0 con 0x7ff5300d07c0 2024-03-28T13:12:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.693+0000 7ff537400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff53015d200 con 0x7ff5300af4b0 2024-03-28T13:12:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.693+0000 7ff537400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff530153960 con 0x7ff530153580 2024-03-28T13:12:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.693+0000 7ff52fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5300d07c0 0x7ff5301503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.693+0000 7ff52fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5300d07c0 0x7ff5301503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35768/0 (socket says 172.21.15.19:35768) 2024-03-28T13:12:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.693+0000 7ff52fe00640 1 -- 172.21.15.19:0/57002570 learned_addr learned my addr 172.21.15.19:0/57002570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.693+0000 7ff534a00640 1 --2- 172.21.15.19:0/57002570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5300af4b0 0x7ff53014b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.693+0000 7ff535400640 1 --2- 172.21.15.19:0/57002570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff530153580 0x7ff5300cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.693+0000 7ff52fe00640 1 -- 172.21.15.19:0/57002570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff530153580 msgr2=0x7ff5300cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.693+0000 7ff52fe00640 1 --2- 172.21.15.19:0/57002570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff530153580 0x7ff5300cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.693+0000 7ff52fe00640 1 -- 172.21.15.19:0/57002570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5300af4b0 msgr2=0x7ff53014b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.693+0000 7ff52fe00640 1 --2- 172.21.15.19:0/57002570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5300af4b0 0x7ff53014b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:52.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.693+0000 7ff52fe00640 1 -- 172.21.15.19:0/57002570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff530150e00 con 0x7ff5300d07c0 2024-03-28T13:12:52.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.694+0000 7ff52fe00640 1 --2- 172.21.15.19:0/57002570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5300d07c0 0x7ff5301503d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff51800c390 tx=0x7ff51800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:52.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.694+0000 7ff52cc00640 1 -- 172.21.15.19:0/57002570 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518017070 con 0x7ff5300d07c0 2024-03-28T13:12:52.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.694+0000 7ff52cc00640 1 -- 172.21.15.19:0/57002570 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff518005ce0 con 0x7ff5300d07c0 2024-03-28T13:12:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.694+0000 7ff52cc00640 1 -- 172.21.15.19:0/57002570 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5180072f0 con 0x7ff5300d07c0 2024-03-28T13:12:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.694+0000 7ff537400640 1 -- 172.21.15.19:0/57002570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5301510e0 con 0x7ff5300d07c0 2024-03-28T13:12:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.694+0000 7ff537400640 1 -- 172.21.15.19:0/57002570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff53014c5e0 con 0x7ff5300d07c0 2024-03-28T13:12:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.695+0000 7ff537400640 1 -- 172.21.15.19:0/57002570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff53006e600 con 0x7ff5300d07c0 2024-03-28T13:12:52.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.697+0000 7ff52cc00640 1 -- 172.21.15.19:0/57002570 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff51801a070 con 0x7ff5300d07c0 2024-03-28T13:12:52.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.697+0000 7ff52cc00640 1 --2- 172.21.15.19:0/57002570 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff50c06c560 0x7ff50c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:52.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.698+0000 7ff52cc00640 1 -- 172.21.15.19:0/57002570 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5609+0+0 (secure 0 0 0) 0x7ff51808aa80 con 0x7ff5300d07c0 2024-03-28T13:12:52.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.698+0000 7ff534a00640 1 --2- 172.21.15.19:0/57002570 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff50c06c560 0x7ff50c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:52.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.699+0000 7ff534a00640 1 --2- 172.21.15.19:0/57002570 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff50c06c560 0x7ff50c06ea20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff5200074a0 tx=0x7ff52000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:52.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.703+0000 7ff52cc00640 1 -- 172.21.15.19:0/57002570 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff53006e600 con 0x7ff5300d07c0 2024-03-28T13:12:52.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:52.945+0000 7ff537400640 1 -- 172.21.15.19:0/57002570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff5300685c0 con 0x7ff5300d07c0 2024-03-28T13:12:53.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.502+0000 7ff52cc00640 1 -- 172.21.15.19:0/57002570 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v30) ==== 172+0+0 (secure 0 0 0) 0x7ff51808ce20 con 0x7ff5300d07c0 2024-03-28T13:12:53.508 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 8 and data pool 9 2024-03-28T13:12:53.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.512+0000 7ff537400640 1 -- 172.21.15.19:0/57002570 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff50c06c560 msgr2=0x7ff50c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:53.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.512+0000 7ff537400640 1 --2- 172.21.15.19:0/57002570 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff50c06c560 0x7ff50c06ea20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff5200074a0 tx=0x7ff52000b040 comp rx=0 tx=0).stop 2024-03-28T13:12:53.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.512+0000 7ff537400640 1 -- 172.21.15.19:0/57002570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5300d07c0 msgr2=0x7ff5301503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:53.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.512+0000 7ff537400640 1 --2- 172.21.15.19:0/57002570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5300d07c0 0x7ff5301503d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff51800c390 tx=0x7ff51800c850 comp rx=0 tx=0).stop 2024-03-28T13:12:53.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.513+0000 7ff537400640 1 -- 172.21.15.19:0/57002570 shutdown_connections 2024-03-28T13:12:53.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.513+0000 7ff537400640 1 --2- 172.21.15.19:0/57002570 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff50c06c560 0x7ff50c06ea20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:53.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.513+0000 7ff537400640 1 --2- 172.21.15.19:0/57002570 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff530153580 0x7ff5300cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:53.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.513+0000 7ff537400640 1 --2- 172.21.15.19:0/57002570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5300d07c0 0x7ff5301503d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:53.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.513+0000 7ff537400640 1 --2- 172.21.15.19:0/57002570 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5300af4b0 0x7ff53014b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:53.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.513+0000 7ff537400640 1 -- 172.21.15.19:0/57002570 >> 172.21.15.19:0/57002570 conn(0x7ff53006b750 msgr2=0x7ff5300aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.514+0000 7ff537400640 1 -- 172.21.15.19:0/57002570 shutdown_connections 2024-03-28T13:12:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.514+0000 7ff537400640 1 -- 172.21.15.19:0/57002570 wait complete. 2024-03-28T13:12:53.548 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:12:53.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.728+0000 7f8045c00640 1 Processor -- start 2024-03-28T13:12:53.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.728+0000 7f8045c00640 1 -- start start 2024-03-28T13:12:53.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.729+0000 7f8045c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f80400727f0 0x7f80400b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:53.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.729+0000 7f8045c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f80400d0780 0x7f80400d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:53.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.729+0000 7f8045c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f804015b6f0 con 0x7f804015b310 2024-03-28T13:12:53.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.729+0000 7f8045c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8040072c50 con 0x7f80400d0780 2024-03-28T13:12:53.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.729+0000 7f8045c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f8040072e80 con 0x7f80400727f0 2024-03-28T13:12:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.730+0000 7f803ea00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f804015b310 0x7f804015f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44822/0 (socket says 172.21.15.19:44822) 2024-03-28T13:12:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.730+0000 7f803ea00640 1 -- 172.21.15.19:0/3970050253 learned_addr learned my addr 172.21.15.19:0/3970050253 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.730+0000 7f803f400640 1 --2- 172.21.15.19:0/3970050253 >> v2:172.21.15.19:3300/0 conn(0x7f80400d0780 0x7f80400d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.730+0000 7f803d600640 1 -- 172.21.15.19:0/3970050253 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2926303782 0 0) 0x7f804015b6f0 con 0x7f804015b310 2024-03-28T13:12:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.730+0000 7f803e000640 1 --2- 172.21.15.19:0/3970050253 >> v2:172.21.15.188:3300/0 conn(0x7f80400727f0 0x7f80400b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.730+0000 7f803d600640 1 -- 172.21.15.19:0/3970050253 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8024005730 con 0x7f804015b310 2024-03-28T13:12:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.730+0000 7f803f400640 1 -- 172.21.15.19:0/3970050253 >> v2:172.21.15.188:3300/0 conn(0x7f80400727f0 msgr2=0x7f80400b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.730+0000 7f803f400640 1 --2- 172.21.15.19:0/3970050253 >> v2:172.21.15.188:3300/0 conn(0x7f80400727f0 0x7f80400b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.730+0000 7f803f400640 1 -- 172.21.15.19:0/3970050253 >> v1:172.21.15.19:6789/0 conn(0x7f804015b310 legacy=0x7f804015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.730+0000 7f803f400640 1 -- 172.21.15.19:0/3970050253 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8040073160 con 0x7f80400d0780 2024-03-28T13:12:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.731+0000 7f803f400640 1 --2- 172.21.15.19:0/3970050253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80400d0780 0x7f80400d0c10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8040071200 tx=0x7f803802bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4df8a432c9e511 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.732+0000 7f803d600640 1 -- 172.21.15.19:0/3970050253 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038039070 con 0x7f80400d0780 2024-03-28T13:12:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.732+0000 7f803d600640 1 -- 172.21.15.19:0/3970050253 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f803802cb60 con 0x7f80400d0780 2024-03-28T13:12:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.732+0000 7f803d600640 1 -- 172.21.15.19:0/3970050253 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038034440 con 0x7f80400d0780 2024-03-28T13:12:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.732+0000 7f8045c00640 1 -- 172.21.15.19:0/3970050253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80400d0780 msgr2=0x7f80400d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.732+0000 7f8045c00640 1 --2- 172.21.15.19:0/3970050253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80400d0780 0x7f80400d0c10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8040071200 tx=0x7f803802bdb0 comp rx=0 tx=0).stop 2024-03-28T13:12:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.732+0000 7f8045c00640 1 -- 172.21.15.19:0/3970050253 shutdown_connections 2024-03-28T13:12:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.732+0000 7f8045c00640 1 --2- 172.21.15.19:0/3970050253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80400d0780 0x7f80400d0c10 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.732+0000 7f8045c00640 1 --2- 172.21.15.19:0/3970050253 >> v2:172.21.15.188:3300/0 conn(0x7f80400727f0 0x7f80400b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.732+0000 7f8045c00640 1 -- 172.21.15.19:0/3970050253 >> 172.21.15.19:0/3970050253 conn(0x7f804006b990 msgr2=0x7f804006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.733+0000 7f8045c00640 1 -- 172.21.15.19:0/3970050253 shutdown_connections 2024-03-28T13:12:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.733+0000 7f8045c00640 1 -- 172.21.15.19:0/3970050253 wait complete. 2024-03-28T13:12:53.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.734+0000 7f8045c00640 1 Processor -- start 2024-03-28T13:12:53.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.735+0000 7f8045c00640 1 -- start start 2024-03-28T13:12:53.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.736+0000 7f8045c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80400727f0 0x7f8040151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:53.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.736+0000 7f8045c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80400d0780 0x7f804015c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:53.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.736+0000 7f8045c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f804015b310 0x7f80400ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.736+0000 7f8045c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8040073440 con 0x7f804015b310 2024-03-28T13:12:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.736+0000 7f8045c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f804015f2c0 con 0x7f80400727f0 2024-03-28T13:12:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.736+0000 7f8045c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f80400adc80 con 0x7f80400d0780 2024-03-28T13:12:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.736+0000 7f803f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f804015b310 0x7f80400ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.736+0000 7f803f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f804015b310 0x7f80400ce720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35792/0 (socket says 172.21.15.19:35792) 2024-03-28T13:12:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.736+0000 7f803f400640 1 -- 172.21.15.19:0/2704328155 learned_addr learned my addr 172.21.15.19:0/2704328155 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.736+0000 7f803ea00640 1 --2- 172.21.15.19:0/2704328155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80400727f0 0x7f8040151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.736+0000 7f803e000640 1 --2- 172.21.15.19:0/2704328155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80400d0780 0x7f804015c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.736+0000 7f803f400640 1 -- 172.21.15.19:0/2704328155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80400d0780 msgr2=0x7f804015c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.736+0000 7f803f400640 1 --2- 172.21.15.19:0/2704328155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80400d0780 0x7f804015c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.736+0000 7f803f400640 1 -- 172.21.15.19:0/2704328155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80400727f0 msgr2=0x7f8040151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.736+0000 7f803f400640 1 --2- 172.21.15.19:0/2704328155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80400727f0 0x7f8040151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:53.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.737+0000 7f803f400640 1 -- 172.21.15.19:0/2704328155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804015ca50 con 0x7f804015b310 2024-03-28T13:12:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.737+0000 7f803ea00640 1 --2- 172.21.15.19:0/2704328155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80400727f0 0x7f8040151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.737+0000 7f803f400640 1 --2- 172.21.15.19:0/2704328155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f804015b310 0x7f80400ce720 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8038009e80 tx=0x7f8038004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.737+0000 7f8036a00640 1 -- 172.21.15.19:0/2704328155 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038039050 con 0x7f804015b310 2024-03-28T13:12:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.737+0000 7f8036a00640 1 -- 172.21.15.19:0/2704328155 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8038045070 con 0x7f804015b310 2024-03-28T13:12:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.737+0000 7f8036a00640 1 -- 172.21.15.19:0/2704328155 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803803e830 con 0x7f804015b310 2024-03-28T13:12:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.737+0000 7f8045c00640 1 -- 172.21.15.19:0/2704328155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804015cd30 con 0x7f804015b310 2024-03-28T13:12:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.737+0000 7f8045c00640 1 -- 172.21.15.19:0/2704328155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f804015c5f0 con 0x7f804015b310 2024-03-28T13:12:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.738+0000 7f8045c00640 1 -- 172.21.15.19:0/2704328155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80400685c0 con 0x7f804015b310 2024-03-28T13:12:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.739+0000 7f8036a00640 1 -- 172.21.15.19:0/2704328155 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f803802cd10 con 0x7f804015b310 2024-03-28T13:12:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.740+0000 7f8036a00640 1 --2- 172.21.15.19:0/2704328155 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f802006c530 0x7f802006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.740+0000 7f8036a00640 1 -- 172.21.15.19:0/2704328155 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f80380ae200 con 0x7f804015b310 2024-03-28T13:12:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.741+0000 7f803ea00640 1 --2- 172.21.15.19:0/2704328155 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f802006c530 0x7f802006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:53.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.742+0000 7f803ea00640 1 --2- 172.21.15.19:0/2704328155 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f802006c530 0x7f802006e9f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8030005880 tx=0x7f8030009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:53.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.745+0000 7f8036a00640 1 -- 172.21.15.19:0/2704328155 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8038032e20 con 0x7f804015b310 2024-03-28T13:12:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.951+0000 7f8045c00640 1 -- 172.21.15.19:0/2704328155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f804015ec20 con 0x7f804015b310 2024-03-28T13:12:53.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.953+0000 7f8036a00640 1 -- 172.21.15.19:0/2704328155 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16678 (secure 0 0 0) 0x7f803807c930 con 0x7f804015b310 2024-03-28T13:12:53.954 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:53.954 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":62,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:12:52.946924+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:12:50.888331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-28T13:12:51.946012+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:12:53.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.956+0000 7f8045c00640 1 -- 172.21.15.19:0/2704328155 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f802006c530 msgr2=0x7f802006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.956+0000 7f8045c00640 1 --2- 172.21.15.19:0/2704328155 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f802006c530 0x7f802006e9f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f8030005880 tx=0x7f8030009040 comp rx=0 tx=0).stop 2024-03-28T13:12:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.956+0000 7f8045c00640 1 -- 172.21.15.19:0/2704328155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f804015b310 msgr2=0x7f80400ce720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.956+0000 7f8045c00640 1 --2- 172.21.15.19:0/2704328155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f804015b310 0x7f80400ce720 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8038009e80 tx=0x7f8038004a40 comp rx=0 tx=0).stop 2024-03-28T13:12:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.956+0000 7f8045c00640 1 -- 172.21.15.19:0/2704328155 shutdown_connections 2024-03-28T13:12:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.956+0000 7f8045c00640 1 --2- 172.21.15.19:0/2704328155 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f802006c530 0x7f802006e9f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.956+0000 7f8045c00640 1 --2- 172.21.15.19:0/2704328155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f804015b310 0x7f80400ce720 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.956+0000 7f8045c00640 1 --2- 172.21.15.19:0/2704328155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80400d0780 0x7f804015c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.956+0000 7f8045c00640 1 --2- 172.21.15.19:0/2704328155 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80400727f0 0x7f8040151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.956+0000 7f8045c00640 1 -- 172.21.15.19:0/2704328155 >> 172.21.15.19:0/2704328155 conn(0x7f804006b990 msgr2=0x7f80400afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.956+0000 7f8045c00640 1 -- 172.21.15.19:0/2704328155 shutdown_connections 2024-03-28T13:12:53.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:53.956+0000 7f8045c00640 1 -- 172.21.15.19:0/2704328155 wait complete. 2024-03-28T13:12:53.976 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:12:54.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.191+0000 7f75b5c00640 1 Processor -- start 2024-03-28T13:12:54.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.191+0000 7f75b5c00640 1 -- start start 2024-03-28T13:12:54.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.193+0000 7f75b5c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75b00d0390 0x7f75b00cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:54.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.193+0000 7f75b5c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f75b00ce1b0 0x7f75b006da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:54.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.193+0000 7f75b5c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b0161f30 con 0x7f75b0147a40 2024-03-28T13:12:54.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.193+0000 7f75b5c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f75b0162b00 con 0x7f75b00ce1b0 2024-03-28T13:12:54.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.193+0000 7f75b5c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f75b01621d0 con 0x7f75b00d0390 2024-03-28T13:12:54.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.193+0000 7f75b5200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f75b00ce1b0 0x7f75b006da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:54.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.193+0000 7f75afe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f75b0147a40 0x7f75b00cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:44830/0 (socket says 172.21.15.19:44830) 2024-03-28T13:12:54.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.193+0000 7f75afe00640 1 -- 172.21.15.19:0/2082196102 learned_addr learned my addr 172.21.15.19:0/2082196102 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:54.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.194+0000 7f75af400640 1 --2- 172.21.15.19:0/2082196102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75b00d0390 0x7f75b00cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:54.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.194+0000 7f75b5200640 1 -- 172.21.15.19:0/2082196102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75b00d0390 msgr2=0x7f75b00cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:54.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.194+0000 7f75b5200640 1 --2- 172.21.15.19:0/2082196102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75b00d0390 0x7f75b00cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:54.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.194+0000 7f75b5200640 1 -- 172.21.15.19:0/2082196102 >> v1:172.21.15.19:6789/0 conn(0x7f75b0147a40 legacy=0x7f75b00cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:54.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.194+0000 7f75b5200640 1 -- 172.21.15.19:0/2082196102 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b01624b0 con 0x7f75b00ce1b0 2024-03-28T13:12:54.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.195+0000 7f75b5200640 1 --2- 172.21.15.19:0/2082196102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00ce1b0 0x7f75b006da80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009d90 tx=0x7f75a802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58720593ea03fd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:54.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.196+0000 7f75aea00640 1 -- 172.21.15.19:0/2082196102 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8039070 con 0x7f75b00ce1b0 2024-03-28T13:12:54.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.196+0000 7f75aea00640 1 -- 172.21.15.19:0/2082196102 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a802cc60 con 0x7f75b00ce1b0 2024-03-28T13:12:54.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.196+0000 7f75aea00640 1 -- 172.21.15.19:0/2082196102 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8034660 con 0x7f75b00ce1b0 2024-03-28T13:12:54.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.196+0000 7f75b5c00640 1 -- 172.21.15.19:0/2082196102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00ce1b0 msgr2=0x7f75b006da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:54.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.196+0000 7f75b5c00640 1 --2- 172.21.15.19:0/2082196102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00ce1b0 0x7f75b006da80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009d90 tx=0x7f75a802beb0 comp rx=0 tx=0).stop 2024-03-28T13:12:54.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.196+0000 7f75b5c00640 1 -- 172.21.15.19:0/2082196102 shutdown_connections 2024-03-28T13:12:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.196+0000 7f75b5c00640 1 --2- 172.21.15.19:0/2082196102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00ce1b0 0x7f75b006da80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.196+0000 7f75b5c00640 1 --2- 172.21.15.19:0/2082196102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75b00d0390 0x7f75b00cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.196+0000 7f75b5c00640 1 -- 172.21.15.19:0/2082196102 >> 172.21.15.19:0/2082196102 conn(0x7f75b00ccad0 msgr2=0x7f75b00cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.196+0000 7f75b5c00640 1 -- 172.21.15.19:0/2082196102 shutdown_connections 2024-03-28T13:12:54.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.197+0000 7f75b5c00640 1 -- 172.21.15.19:0/2082196102 wait complete. 2024-03-28T13:12:54.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.198+0000 7f75b5c00640 1 Processor -- start 2024-03-28T13:12:54.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.198+0000 7f75b5c00640 1 -- start start 2024-03-28T13:12:54.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.199+0000 7f75b5c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00ce1b0 0x7f75b0141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:54.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.199+0000 7f75b5c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75b00d0390 0x7f75b013f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.199+0000 7f75b5c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75b0147a40 0x7f75b013d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.199+0000 7f75b5c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75b0147fa0 con 0x7f75b00ce1b0 2024-03-28T13:12:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.199+0000 7f75b5c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f75b0147e20 con 0x7f75b0147a40 2024-03-28T13:12:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.199+0000 7f75b5c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f75b00ce640 con 0x7f75b00d0390 2024-03-28T13:12:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.200+0000 7f75afe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00ce1b0 0x7f75b0141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.200+0000 7f75afe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00ce1b0 0x7f75b0141aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35816/0 (socket says 172.21.15.19:35816) 2024-03-28T13:12:54.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.200+0000 7f75afe00640 1 -- 172.21.15.19:0/4276327865 learned_addr learned my addr 172.21.15.19:0/4276327865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:54.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.200+0000 7f75b5200640 1 --2- 172.21.15.19:0/4276327865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75b0147a40 0x7f75b013d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:54.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.200+0000 7f75afe00640 1 -- 172.21.15.19:0/4276327865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75b00d0390 msgr2=0x7f75b013f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:54.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.200+0000 7f75afe00640 1 --2- 172.21.15.19:0/4276327865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75b00d0390 0x7f75b013f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:54.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.200+0000 7f75afe00640 1 -- 172.21.15.19:0/4276327865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75b0147a40 msgr2=0x7f75b013d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:54.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.200+0000 7f75afe00640 1 --2- 172.21.15.19:0/4276327865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75b0147a40 0x7f75b013d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:54.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.200+0000 7f75afe00640 1 -- 172.21.15.19:0/4276327865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b014d8a0 con 0x7f75b00ce1b0 2024-03-28T13:12:54.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.200+0000 7f75afe00640 1 --2- 172.21.15.19:0/4276327865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00ce1b0 0x7f75b0141aa0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f75a000c3c0 tx=0x7f75a000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:54.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.201+0000 7f75a7e00640 1 -- 172.21.15.19:0/4276327865 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0017070 con 0x7f75b00ce1b0 2024-03-28T13:12:54.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.201+0000 7f75a7e00640 1 -- 172.21.15.19:0/4276327865 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a0005ce0 con 0x7f75b00ce1b0 2024-03-28T13:12:54.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.201+0000 7f75a7e00640 1 -- 172.21.15.19:0/4276327865 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a0007310 con 0x7f75b00ce1b0 2024-03-28T13:12:54.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.201+0000 7f75b5c00640 1 -- 172.21.15.19:0/4276327865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b0141fe0 con 0x7f75b00ce1b0 2024-03-28T13:12:54.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.201+0000 7f75b5c00640 1 -- 172.21.15.19:0/4276327865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b014db40 con 0x7f75b00ce1b0 2024-03-28T13:12:54.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.202+0000 7f75b5c00640 1 -- 172.21.15.19:0/4276327865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b0075940 con 0x7f75b00ce1b0 2024-03-28T13:12:54.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.203+0000 7f75a7e00640 1 -- 172.21.15.19:0/4276327865 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f75a001a070 con 0x7f75b00ce1b0 2024-03-28T13:12:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.204+0000 7f75a7e00640 1 --2- 172.21.15.19:0/4276327865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f759006c580 0x7f759006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.204+0000 7f75a7e00640 1 -- 172.21.15.19:0/4276327865 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f75a008aab0 con 0x7f75b00ce1b0 2024-03-28T13:12:54.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.205+0000 7f75af400640 1 --2- 172.21.15.19:0/4276327865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f759006c580 0x7f759006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:54.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.206+0000 7f75af400640 1 --2- 172.21.15.19:0/4276327865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f759006c580 0x7f759006ea40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f759c006a90 tx=0x7f759c006a20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:54.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.208+0000 7f75a7e00640 1 -- 172.21.15.19:0/4276327865 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f75a00590b0 con 0x7f75b00ce1b0 2024-03-28T13:12:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.414+0000 7f75b5c00640 1 -- 172.21.15.19:0/4276327865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f75b0161690 con 0x7f75b00ce1b0 2024-03-28T13:12:54.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.416+0000 7f75a7e00640 1 -- 172.21.15.19:0/4276327865 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16678 (secure 0 0 0) 0x7f75a00589b0 con 0x7f75b00ce1b0 2024-03-28T13:12:54.417 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:54.417 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":62,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:12:52.946924+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:12:50.888331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-28T13:12:51.946012+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:12:54.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.419+0000 7f75b5c00640 1 -- 172.21.15.19:0/4276327865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f759006c580 msgr2=0x7f759006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:54.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.419+0000 7f75b5c00640 1 --2- 172.21.15.19:0/4276327865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f759006c580 0x7f759006ea40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f759c006a90 tx=0x7f759c006a20 comp rx=0 tx=0).stop 2024-03-28T13:12:54.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.419+0000 7f75b5c00640 1 -- 172.21.15.19:0/4276327865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00ce1b0 msgr2=0x7f75b0141aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:54.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.419+0000 7f75b5c00640 1 --2- 172.21.15.19:0/4276327865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00ce1b0 0x7f75b0141aa0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f75a000c3c0 tx=0x7f75a000c880 comp rx=0 tx=0).stop 2024-03-28T13:12:54.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.419+0000 7f75b5c00640 1 -- 172.21.15.19:0/4276327865 shutdown_connections 2024-03-28T13:12:54.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.419+0000 7f75b5c00640 1 --2- 172.21.15.19:0/4276327865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f759006c580 0x7f759006ea40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:54.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.419+0000 7f75b5c00640 1 --2- 172.21.15.19:0/4276327865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75b0147a40 0x7f75b013d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:54.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.419+0000 7f75b5c00640 1 --2- 172.21.15.19:0/4276327865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75b00d0390 0x7f75b013f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:54.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.419+0000 7f75b5c00640 1 --2- 172.21.15.19:0/4276327865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f75b00ce1b0 0x7f75b0141aa0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:54.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.419+0000 7f75b5c00640 1 -- 172.21.15.19:0/4276327865 >> 172.21.15.19:0/4276327865 conn(0x7f75b00ccad0 msgr2=0x7f75b01424c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:54.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.420+0000 7f75b5c00640 1 -- 172.21.15.19:0/4276327865 shutdown_connections 2024-03-28T13:12:54.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.420+0000 7f75b5c00640 1 -- 172.21.15.19:0/4276327865 wait complete. 2024-03-28T13:12:54.441 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-28T13:12:54.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.665+0000 7f802fa00640 1 Processor -- start 2024-03-28T13:12:54.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.666+0000 7f802fa00640 1 -- start start 2024-03-28T13:12:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.667+0000 7f802fa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8028162a30 0x7f80280b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.668+0000 7f802fa00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f80280d18b0 0x7f80280c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.668+0000 7f802fa00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80280d2030 con 0x7f80280c7640 2024-03-28T13:12:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.668+0000 7f802fa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f80280c7a70 con 0x7f8028162a30 2024-03-28T13:12:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.668+0000 7f802fa00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f8028166670 con 0x7f80280d18b0 2024-03-28T13:12:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.668+0000 7f802d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8028162a30 0x7f80280b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.668+0000 7f802d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8028162a30 0x7f80280b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35820/0 (socket says 172.21.15.19:35820) 2024-03-28T13:12:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.668+0000 7f802d000640 1 -- 172.21.15.19:0/3847751128 learned_addr learned my addr 172.21.15.19:0/3847751128 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:54.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.668+0000 7f8027e00640 1 --2- 172.21.15.19:0/3847751128 >> v2:172.21.15.188:3300/0 conn(0x7f80280d18b0 0x7f80280c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:54.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.668+0000 7f802d000640 1 -- 172.21.15.19:0/3847751128 >> v2:172.21.15.188:3300/0 conn(0x7f80280d18b0 msgr2=0x7f80280c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:54.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.669+0000 7f802d000640 1 --2- 172.21.15.19:0/3847751128 >> v2:172.21.15.188:3300/0 conn(0x7f80280d18b0 0x7f80280c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.669+0000 7f802d000640 1 -- 172.21.15.19:0/3847751128 >> v1:172.21.15.188:6789/0 conn(0x7f80280c7640 legacy=0x7f8028071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.669+0000 7f802d000640 1 -- 172.21.15.19:0/3847751128 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8028166950 con 0x7f8028162a30 2024-03-28T13:12:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.670+0000 7f802d000640 1 --2- 172.21.15.19:0/3847751128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8028162a30 0x7f80280b4980 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f80280708e0 tx=0x7f801c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeadfb7485fb1423 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.670+0000 7f8027400640 1 -- 172.21.15.19:0/3847751128 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c02e070 con 0x7f8028162a30 2024-03-28T13:12:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.670+0000 7f8027400640 1 -- 172.21.15.19:0/3847751128 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f801c037070 con 0x7f8028162a30 2024-03-28T13:12:54.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.670+0000 7f8027400640 1 -- 172.21.15.19:0/3847751128 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c031a00 con 0x7f8028162a30 2024-03-28T13:12:54.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.671+0000 7f802fa00640 1 -- 172.21.15.19:0/3847751128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8028162a30 msgr2=0x7f80280b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:54.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.671+0000 7f802fa00640 1 --2- 172.21.15.19:0/3847751128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8028162a30 0x7f80280b4980 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f80280708e0 tx=0x7f801c02b880 comp rx=0 tx=0).stop 2024-03-28T13:12:54.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.671+0000 7f802fa00640 1 -- 172.21.15.19:0/3847751128 shutdown_connections 2024-03-28T13:12:54.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.671+0000 7f802fa00640 1 --2- 172.21.15.19:0/3847751128 >> v2:172.21.15.188:3300/0 conn(0x7f80280d18b0 0x7f80280c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:54.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.671+0000 7f802fa00640 1 --2- 172.21.15.19:0/3847751128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8028162a30 0x7f80280b4980 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:54.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.671+0000 7f802fa00640 1 -- 172.21.15.19:0/3847751128 >> 172.21.15.19:0/3847751128 conn(0x7f80280b3e90 msgr2=0x7f80280b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:54.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.671+0000 7f802fa00640 1 -- 172.21.15.19:0/3847751128 shutdown_connections 2024-03-28T13:12:54.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.671+0000 7f802fa00640 1 -- 172.21.15.19:0/3847751128 wait complete. 2024-03-28T13:12:54.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.673+0000 7f802fa00640 1 Processor -- start 2024-03-28T13:12:54.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.673+0000 7f802fa00640 1 -- start start 2024-03-28T13:12:54.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.674+0000 7f802fa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80280c7640 0x7f8028161850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:54.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.674+0000 7f802fa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80280d18b0 0x7f802815ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:54.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.674+0000 7f802fa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8028162a30 0x7f8028162f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:54.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.674+0000 7f802fa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8028162e10 con 0x7f8028162a30 2024-03-28T13:12:54.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.674+0000 7f802fa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8028166f10 con 0x7f80280d18b0 2024-03-28T13:12:54.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.674+0000 7f802fa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f802806da40 con 0x7f80280c7640 2024-03-28T13:12:54.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.674+0000 7f802da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8028162a30 0x7f8028162f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:54.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.674+0000 7f802da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8028162a30 0x7f8028162f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35836/0 (socket says 172.21.15.19:35836) 2024-03-28T13:12:54.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.674+0000 7f802da00640 1 -- 172.21.15.19:0/3814620838 learned_addr learned my addr 172.21.15.19:0/3814620838 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:54.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.675+0000 7f8027e00640 1 --2- 172.21.15.19:0/3814620838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80280d18b0 0x7f802815ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:54.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.675+0000 7f802d000640 1 --2- 172.21.15.19:0/3814620838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80280c7640 0x7f8028161850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:54.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.675+0000 7f802da00640 1 -- 172.21.15.19:0/3814620838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80280c7640 msgr2=0x7f8028161850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:54.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.675+0000 7f802da00640 1 --2- 172.21.15.19:0/3814620838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80280c7640 0x7f8028161850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:54.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.675+0000 7f802da00640 1 -- 172.21.15.19:0/3814620838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80280d18b0 msgr2=0x7f802815ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:54.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.675+0000 7f802da00640 1 --2- 172.21.15.19:0/3814620838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80280d18b0 0x7f802815ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:54.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.675+0000 7f802da00640 1 -- 172.21.15.19:0/3814620838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8028161410 con 0x7f8028162a30 2024-03-28T13:12:54.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.675+0000 7f802da00640 1 --2- 172.21.15.19:0/3814620838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8028162a30 0x7f8028162f90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f802000c7b0 tx=0x7f802000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.676+0000 7f8024c00640 1 -- 172.21.15.19:0/3814620838 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020017070 con 0x7f8028162a30 2024-03-28T13:12:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.676+0000 7f8024c00640 1 -- 172.21.15.19:0/3814620838 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8020003da0 con 0x7f8028162a30 2024-03-28T13:12:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.676+0000 7f8024c00640 1 -- 172.21.15.19:0/3814620838 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80200052b0 con 0x7f8028162a30 2024-03-28T13:12:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.676+0000 7f802fa00640 1 -- 172.21.15.19:0/3814620838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8028160750 con 0x7f8028162a30 2024-03-28T13:12:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.676+0000 7f802fa00640 1 -- 172.21.15.19:0/3814620838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8028160cd0 con 0x7f8028162a30 2024-03-28T13:12:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.677+0000 7f802fa00640 1 -- 172.21.15.19:0/3814620838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8028068cd0 con 0x7f8028162a30 2024-03-28T13:12:54.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.678+0000 7f8024c00640 1 -- 172.21.15.19:0/3814620838 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8020005450 con 0x7f8028162a30 2024-03-28T13:12:54.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.679+0000 7f8024c00640 1 --2- 172.21.15.19:0/3814620838 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f800806c530 0x7f800806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.679+0000 7f8024c00640 1 -- 172.21.15.19:0/3814620838 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f802008ac20 con 0x7f8028162a30 2024-03-28T13:12:54.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.679+0000 7f802d000640 1 --2- 172.21.15.19:0/3814620838 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f800806c530 0x7f800806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:54.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.681+0000 7f802d000640 1 --2- 172.21.15.19:0/3814620838 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f800806c530 0x7f800806e9f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f801c036000 tx=0x7f801c03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:54.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.688+0000 7f8024c00640 1 -- 172.21.15.19:0/3814620838 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8020059240 con 0x7f8028162a30 2024-03-28T13:12:54.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:54.930+0000 7f802fa00640 1 -- 172.21.15.19:0/3814620838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f802815eb40 con 0x7f8028162a30 2024-03-28T13:12:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.521+0000 7f8024c00640 1 -- 172.21.15.19:0/3814620838 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) ==== 118+0+0 (secure 0 0 0) 0x7f8020058b40 con 0x7f8028162a30 2024-03-28T13:12:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.526+0000 7f802fa00640 1 -- 172.21.15.19:0/3814620838 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f800806c530 msgr2=0x7f800806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.526+0000 7f802fa00640 1 --2- 172.21.15.19:0/3814620838 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f800806c530 0x7f800806e9f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f801c036000 tx=0x7f801c03d040 comp rx=0 tx=0).stop 2024-03-28T13:12:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.526+0000 7f802fa00640 1 -- 172.21.15.19:0/3814620838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8028162a30 msgr2=0x7f8028162f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.526+0000 7f802fa00640 1 --2- 172.21.15.19:0/3814620838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8028162a30 0x7f8028162f90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f802000c7b0 tx=0x7f802000cc70 comp rx=0 tx=0).stop 2024-03-28T13:12:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.526+0000 7f802fa00640 1 -- 172.21.15.19:0/3814620838 shutdown_connections 2024-03-28T13:12:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.526+0000 7f802fa00640 1 --2- 172.21.15.19:0/3814620838 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f800806c530 0x7f800806e9f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.526+0000 7f802fa00640 1 --2- 172.21.15.19:0/3814620838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8028162a30 0x7f8028162f90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.526+0000 7f802fa00640 1 --2- 172.21.15.19:0/3814620838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80280d18b0 0x7f802815ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.526+0000 7f802fa00640 1 --2- 172.21.15.19:0/3814620838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80280c7640 0x7f8028161850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.526+0000 7f802fa00640 1 -- 172.21.15.19:0/3814620838 >> 172.21.15.19:0/3814620838 conn(0x7f80280b3e90 msgr2=0x7f80280b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.526+0000 7f802fa00640 1 -- 172.21.15.19:0/3814620838 shutdown_connections 2024-03-28T13:12:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.526+0000 7f802fa00640 1 -- 172.21.15.19:0/3814620838 wait complete. 2024-03-28T13:12:55.547 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-28T13:12:55.547 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-28T13:12:55.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.765+0000 7f5685600640 1 Processor -- start 2024-03-28T13:12:55.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.766+0000 7f5685600640 1 -- start start 2024-03-28T13:12:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.768+0000 7f5685600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f568013f190 0x7f56800cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.768+0000 7f5685600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56800ce890 con 0x7f568006dd30 2024-03-28T13:12:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.768+0000 7f5685600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56800ceb50 con 0x7f56800cc880 2024-03-28T13:12:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.768+0000 7f5685600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f56800cee30 con 0x7f568013f190 2024-03-28T13:12:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.768+0000 7f567fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f568013f190 0x7f56800cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.768+0000 7f5684c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f568006dd30 0x7f568015ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52164/0 (socket says 172.21.15.19:52164) 2024-03-28T13:12:55.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.768+0000 7f5684c00640 1 -- 172.21.15.19:0/2789185307 learned_addr learned my addr 172.21.15.19:0/2789185307 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.769+0000 7f567fe00640 1 -- 172.21.15.19:0/2789185307 >> v1:172.21.15.188:6789/0 conn(0x7f56800cc880 legacy=0x7f568006d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.769+0000 7f567fe00640 1 -- 172.21.15.19:0/2789185307 >> v1:172.21.15.19:6789/0 conn(0x7f568006dd30 legacy=0x7f568015ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.769+0000 7f567fe00640 1 -- 172.21.15.19:0/2789185307 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56800cf110 con 0x7f568013f190 2024-03-28T13:12:55.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.769+0000 7f567ea00640 1 -- 172.21.15.19:0/2789185307 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4024843063 0 0) 0x7f56800ce890 con 0x7f568006dd30 2024-03-28T13:12:55.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.770+0000 7f567fe00640 1 --2- 172.21.15.19:0/2789185307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568013f190 0x7f56800cf630 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5670009d90 tx=0x7f567002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52dc57ec34b0cb75 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:55.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.770+0000 7f567ea00640 1 -- 172.21.15.19:0/2789185307 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670039070 con 0x7f568013f190 2024-03-28T13:12:55.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.770+0000 7f567ea00640 1 -- 172.21.15.19:0/2789185307 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f567002cc60 con 0x7f568013f190 2024-03-28T13:12:55.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.770+0000 7f567ea00640 1 -- 172.21.15.19:0/2789185307 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670034660 con 0x7f568013f190 2024-03-28T13:12:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.771+0000 7f5685600640 1 -- 172.21.15.19:0/2789185307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568013f190 msgr2=0x7f56800cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.771+0000 7f5685600640 1 --2- 172.21.15.19:0/2789185307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568013f190 0x7f56800cf630 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5670009d90 tx=0x7f567002beb0 comp rx=0 tx=0).stop 2024-03-28T13:12:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.771+0000 7f5685600640 1 -- 172.21.15.19:0/2789185307 shutdown_connections 2024-03-28T13:12:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.771+0000 7f5685600640 1 --2- 172.21.15.19:0/2789185307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568013f190 0x7f56800cf630 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.771+0000 7f5685600640 1 -- 172.21.15.19:0/2789185307 >> 172.21.15.19:0/2789185307 conn(0x7f56800b1ba0 msgr2=0x7f56800b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.771+0000 7f5685600640 1 -- 172.21.15.19:0/2789185307 shutdown_connections 2024-03-28T13:12:55.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.771+0000 7f5685600640 1 -- 172.21.15.19:0/2789185307 wait complete. 2024-03-28T13:12:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.773+0000 7f5685600640 1 Processor -- start 2024-03-28T13:12:55.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.773+0000 7f5685600640 1 -- start start 2024-03-28T13:12:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.774+0000 7f5685600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568006dd30 0x7f56801552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.774+0000 7f5685600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56800cc880 0x7f56801f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.774+0000 7f5685600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f568013f190 0x7f568013bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.774+0000 7f5685600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f56800cfb70 con 0x7f568006dd30 2024-03-28T13:12:55.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.774+0000 7f5685600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f568013f570 con 0x7f568013f190 2024-03-28T13:12:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.774+0000 7f5685600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5680160620 con 0x7f56800cc880 2024-03-28T13:12:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.774+0000 7f567fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568006dd30 0x7f56801552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.774+0000 7f567fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568006dd30 0x7f56801552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33764/0 (socket says 172.21.15.19:33764) 2024-03-28T13:12:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.774+0000 7f567fe00640 1 -- 172.21.15.19:0/2771481622 learned_addr learned my addr 172.21.15.19:0/2771481622 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.774+0000 7f567fe00640 1 -- 172.21.15.19:0/2771481622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56800cc880 msgr2=0x7f56801f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.774+0000 7f567fe00640 1 --2- 172.21.15.19:0/2771481622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56800cc880 0x7f56801f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.774+0000 7f567fe00640 1 -- 172.21.15.19:0/2771481622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f568013f190 msgr2=0x7f568013bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.774+0000 7f567fe00640 1 --2- 172.21.15.19:0/2771481622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f568013f190 0x7f568013bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.774+0000 7f567fe00640 1 -- 172.21.15.19:0/2771481622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568013ade0 con 0x7f568006dd30 2024-03-28T13:12:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.775+0000 7f567fe00640 1 --2- 172.21.15.19:0/2771481622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568006dd30 0x7f56801552e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5670037040 tx=0x7f5670004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.775+0000 7f5677e00640 1 -- 172.21.15.19:0/2771481622 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670039040 con 0x7f568006dd30 2024-03-28T13:12:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.775+0000 7f5677e00640 1 -- 172.21.15.19:0/2771481622 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5670003b90 con 0x7f568006dd30 2024-03-28T13:12:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.775+0000 7f5677e00640 1 -- 172.21.15.19:0/2771481622 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670004030 con 0x7f568006dd30 2024-03-28T13:12:55.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.775+0000 7f5685600640 1 -- 172.21.15.19:0/2771481622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56801f4a20 con 0x7f568006dd30 2024-03-28T13:12:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.775+0000 7f5685600640 1 -- 172.21.15.19:0/2771481622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568013cdb0 con 0x7f568006dd30 2024-03-28T13:12:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.777+0000 7f5685600640 1 -- 172.21.15.19:0/2771481622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56800b3d30 con 0x7f568006dd30 2024-03-28T13:12:55.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.777+0000 7f5677e00640 1 -- 172.21.15.19:0/2771481622 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f567002cb20 con 0x7f568006dd30 2024-03-28T13:12:55.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.778+0000 7f5677e00640 1 --2- 172.21.15.19:0/2771481622 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f566006c530 0x7f566006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:55.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.779+0000 7f5677e00640 1 -- 172.21.15.19:0/2771481622 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f56700aea80 con 0x7f568006dd30 2024-03-28T13:12:55.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.779+0000 7f567f400640 1 --2- 172.21.15.19:0/2771481622 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f566006c530 0x7f566006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:55.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.780+0000 7f567f400640 1 --2- 172.21.15.19:0/2771481622 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f566006c530 0x7f566006e9f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f56800d0bf0 tx=0x7f566c0079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:55.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:55.783+0000 7f5677e00640 1 -- 172.21.15.19:0/2771481622 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f56800b3d30 con 0x7f568006dd30 2024-03-28T13:12:56.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.030+0000 7f5685600640 1 -- 172.21.15.19:0/2771481622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5680075940 con 0x7f568006dd30 2024-03-28T13:12:56.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.534+0000 7f5677e00640 1 -- 172.21.15.19:0/2771481622 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f56700b0e20 con 0x7f568006dd30 2024-03-28T13:12:56.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.538+0000 7f5685600640 1 -- 172.21.15.19:0/2771481622 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f566006c530 msgr2=0x7f566006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:56.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.538+0000 7f5685600640 1 --2- 172.21.15.19:0/2771481622 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f566006c530 0x7f566006e9f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f56800d0bf0 tx=0x7f566c0079b0 comp rx=0 tx=0).stop 2024-03-28T13:12:56.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.538+0000 7f5685600640 1 -- 172.21.15.19:0/2771481622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568006dd30 msgr2=0x7f56801552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:56.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.538+0000 7f5685600640 1 --2- 172.21.15.19:0/2771481622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568006dd30 0x7f56801552e0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5670037040 tx=0x7f5670004a40 comp rx=0 tx=0).stop 2024-03-28T13:12:56.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.539+0000 7f5685600640 1 -- 172.21.15.19:0/2771481622 shutdown_connections 2024-03-28T13:12:56.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.539+0000 7f5685600640 1 --2- 172.21.15.19:0/2771481622 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f566006c530 0x7f566006e9f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:56.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.539+0000 7f5685600640 1 --2- 172.21.15.19:0/2771481622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f568013f190 0x7f568013bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:56.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.539+0000 7f5685600640 1 --2- 172.21.15.19:0/2771481622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56800cc880 0x7f56801f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:56.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.539+0000 7f5685600640 1 --2- 172.21.15.19:0/2771481622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568006dd30 0x7f56801552e0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:56.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.539+0000 7f5685600640 1 -- 172.21.15.19:0/2771481622 >> 172.21.15.19:0/2771481622 conn(0x7f56800b1ba0 msgr2=0x7f56800b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:56.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.539+0000 7f5685600640 1 -- 172.21.15.19:0/2771481622 shutdown_connections 2024-03-28T13:12:56.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.539+0000 7f5685600640 1 -- 172.21.15.19:0/2771481622 wait complete. 2024-03-28T13:12:56.560 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-28T13:12:56.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:12:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.773+0000 7f8ecda00640 1 Processor -- start 2024-03-28T13:12:56.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.773+0000 7f8ecda00640 1 -- start start 2024-03-28T13:12:56.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.775+0000 7f8ecda00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8ec8070250 0x7f8ec80d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:56.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.775+0000 7f8ecda00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ec80d26e0 0x7f8ec80d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:56.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.775+0000 7f8ecda00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ec80d2ed0 con 0x7f8ec813f100 2024-03-28T13:12:56.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.775+0000 7f8ecda00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f8ec8142270 con 0x7f8ec8070250 2024-03-28T13:12:56.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.775+0000 7f8ecda00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec813f4e0 con 0x7f8ec80d26e0 2024-03-28T13:12:56.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.775+0000 7f8ec7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8ec813f100 0x7f8ec806fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52174/0 (socket says 172.21.15.19:52174) 2024-03-28T13:12:56.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.775+0000 7f8ec7e00640 1 -- 172.21.15.19:0/3784865974 learned_addr learned my addr 172.21.15.19:0/3784865974 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:56.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.775+0000 7f8ec7400640 1 --2- 172.21.15.19:0/3784865974 >> v2:172.21.15.188:3300/0 conn(0x7f8ec8070250 0x7f8ec80d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:56.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.775+0000 7f8ecd000640 1 --2- 172.21.15.19:0/3784865974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ec80d26e0 0x7f8ec80d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:56.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 -- 172.21.15.19:0/3784865974 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1468828849 0 0) 0x7f8ec80d2ed0 con 0x7f8ec813f100 2024-03-28T13:12:56.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 -- 172.21.15.19:0/3784865974 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8eac003520 con 0x7f8ec813f100 2024-03-28T13:12:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 -- 172.21.15.19:0/3784865974 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3237280411 0 0) 0x7f8eac003520 con 0x7f8ec813f100 2024-03-28T13:12:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 -- 172.21.15.19:0/3784865974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ec80d26e0 msgr2=0x7f8ec80d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 --2- 172.21.15.19:0/3784865974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ec80d26e0 0x7f8ec80d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 -- 172.21.15.19:0/3784865974 >> v2:172.21.15.188:3300/0 conn(0x7f8ec8070250 msgr2=0x7f8ec80d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 --2- 172.21.15.19:0/3784865974 >> v2:172.21.15.188:3300/0 conn(0x7f8ec8070250 0x7f8ec80d21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 -- 172.21.15.19:0/3784865974 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec8141940 con 0x7f8ec813f100 2024-03-28T13:12:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 -- 172.21.15.19:0/3784865974 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f8eb8001670 con 0x7f8ec813f100 2024-03-28T13:12:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec7400640 1 --2- 172.21.15.19:0/3784865974 >> v2:172.21.15.188:3300/0 conn(0x7f8ec8070250 0x7f8ec80d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:12:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 -- 172.21.15.19:0/3784865974 >> v1:172.21.15.19:6789/0 conn(0x7f8ec813f100 legacy=0x7f8ec806fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 --2- 172.21.15.19:0/3784865974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eac0039b0 0x7f8eac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 --2- 172.21.15.19:0/3784865974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8eac004800 0x7f8eac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 --2- 172.21.15.19:0/3784865974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ec80d26e0 0x7f8eac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 -- 172.21.15.19:0/3784865974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8142270 con 0x7f8eac0039b0 2024-03-28T13:12:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 -- 172.21.15.19:0/3784865974 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec813f4e0 con 0x7f8eac004800 2024-03-28T13:12:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.776+0000 7f8ec6a00640 1 -- 172.21.15.19:0/3784865974 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8eac00b9c0 con 0x7f8ec80d26e0 2024-03-28T13:12:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.777+0000 7f8ec7e00640 1 --2- 172.21.15.19:0/3784865974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eac0039b0 0x7f8eac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.777+0000 7f8ec7400640 1 --2- 172.21.15.19:0/3784865974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8eac004800 0x7f8eac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.777+0000 7f8ecd000640 1 --2- 172.21.15.19:0/3784865974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ec80d26e0 0x7f8eac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.777+0000 7f8ec7e00640 1 -- 172.21.15.19:0/3784865974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ec80d26e0 msgr2=0x7f8eac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.777+0000 7f8ec7e00640 1 --2- 172.21.15.19:0/3784865974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ec80d26e0 0x7f8eac0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.777+0000 7f8ec7e00640 1 -- 172.21.15.19:0/3784865974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8eac004800 msgr2=0x7f8eac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.777+0000 7f8ec7e00640 1 --2- 172.21.15.19:0/3784865974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8eac004800 0x7f8eac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.777+0000 7f8ec7e00640 1 -- 172.21.15.19:0/3784865974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eac003520 con 0x7f8eac0039b0 2024-03-28T13:12:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.779+0000 7f8ec7e00640 1 --2- 172.21.15.19:0/3784865974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eac0039b0 0x7f8eac003dd0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800c290 tx=0x7f8eac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caea83c5293b4693 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.779+0000 7f8ec6a00640 1 -- 172.21.15.19:0/3784865974 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8007d50 con 0x7f8eac0039b0 2024-03-28T13:12:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.779+0000 7f8ec6a00640 1 -- 172.21.15.19:0/3784865974 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb8030070 con 0x7f8eac0039b0 2024-03-28T13:12:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.779+0000 7f8ec6a00640 1 -- 172.21.15.19:0/3784865974 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb802cd80 con 0x7f8eac0039b0 2024-03-28T13:12:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.780+0000 7f8ecda00640 1 -- 172.21.15.19:0/3784865974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eac0039b0 msgr2=0x7f8eac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.780+0000 7f8ecda00640 1 --2- 172.21.15.19:0/3784865974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8eac0039b0 0x7f8eac003dd0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800c290 tx=0x7f8eac005830 comp rx=0 tx=0).stop 2024-03-28T13:12:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.780+0000 7f8ecd000640 1 -- 172.21.15.19:0/3784865974 reap_dead start 2024-03-28T13:12:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.780+0000 7f8ecda00640 1 -- 172.21.15.19:0/3784865974 shutdown_connections 2024-03-28T13:12:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.780+0000 7f8ecda00640 1 -- 172.21.15.19:0/3784865974 >> 172.21.15.19:0/3784865974 conn(0x7f8ec8067460 msgr2=0x7f8ec8067840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.780+0000 7f8ecda00640 1 -- 172.21.15.19:0/3784865974 shutdown_connections 2024-03-28T13:12:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.780+0000 7f8ecda00640 1 -- 172.21.15.19:0/3784865974 wait complete. 2024-03-28T13:12:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.782+0000 7f8ecda00640 1 Processor -- start 2024-03-28T13:12:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.782+0000 7f8ecda00640 1 -- start start 2024-03-28T13:12:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.783+0000 7f8ecda00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8eac0039b0 0x7f8ec81440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.783+0000 7f8ecda00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec815e1e0 0x7f8ec815d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.783+0000 7f8ecda00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ec80b38e0 0x7f8ec80b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.783+0000 7f8ecda00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8142700 con 0x7f8ec815e1e0 2024-03-28T13:12:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.783+0000 7f8ecda00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8ec8141f00 con 0x7f8eac0039b0 2024-03-28T13:12:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.783+0000 7f8ecda00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8ec80d2b70 con 0x7f8ec80b38e0 2024-03-28T13:12:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.783+0000 7f8ec7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec815e1e0 0x7f8ec815d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.783+0000 7f8ec7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec815e1e0 0x7f8ec815d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33778/0 (socket says 172.21.15.19:33778) 2024-03-28T13:12:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.783+0000 7f8ec7400640 1 -- 172.21.15.19:0/939802653 learned_addr learned my addr 172.21.15.19:0/939802653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.783+0000 7f8ec7e00640 1 --2- 172.21.15.19:0/939802653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8eac0039b0 0x7f8ec81440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.783+0000 7f8ecd000640 1 --2- 172.21.15.19:0/939802653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ec80b38e0 0x7f8ec80b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:56.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.784+0000 7f8ec7400640 1 -- 172.21.15.19:0/939802653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ec80b38e0 msgr2=0x7f8ec80b3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:56.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.784+0000 7f8ec7400640 1 --2- 172.21.15.19:0/939802653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ec80b38e0 0x7f8ec80b3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:56.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.784+0000 7f8ec7400640 1 -- 172.21.15.19:0/939802653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8eac0039b0 msgr2=0x7f8ec81440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:56.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.784+0000 7f8ec7400640 1 --2- 172.21.15.19:0/939802653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8eac0039b0 0x7f8ec81440f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:56.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.784+0000 7f8ec7400640 1 -- 172.21.15.19:0/939802653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ec813d8c0 con 0x7f8ec815e1e0 2024-03-28T13:12:56.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.784+0000 7f8ec7400640 1 --2- 172.21.15.19:0/939802653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec815e1e0 0x7f8ec815d1c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8eb00114e0 tx=0x7f8eb00119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:56.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.784+0000 7f8ebfe00640 1 -- 172.21.15.19:0/939802653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb001a020 con 0x7f8ec815e1e0 2024-03-28T13:12:56.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.784+0000 7f8ebfe00640 1 -- 172.21.15.19:0/939802653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb0016680 con 0x7f8ec815e1e0 2024-03-28T13:12:56.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.784+0000 7f8ecda00640 1 -- 172.21.15.19:0/939802653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ec813dba0 con 0x7f8ec815e1e0 2024-03-28T13:12:56.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.784+0000 7f8ebfe00640 1 -- 172.21.15.19:0/939802653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0016a50 con 0x7f8ec815e1e0 2024-03-28T13:12:56.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.785+0000 7f8ecda00640 1 -- 172.21.15.19:0/939802653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ec815d700 con 0x7f8ec815e1e0 2024-03-28T13:12:56.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.786+0000 7f8ecda00640 1 -- 172.21.15.19:0/939802653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ec8075940 con 0x7f8ec815e1e0 2024-03-28T13:12:56.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.788+0000 7f8ebfe00640 1 -- 172.21.15.19:0/939802653 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8eb0005d10 con 0x7f8ec815e1e0 2024-03-28T13:12:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.789+0000 7f8ebfe00640 1 --2- 172.21.15.19:0/939802653 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8ea406c530 0x7f8ea406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.789+0000 7f8ebfe00640 1 -- 172.21.15.19:0/939802653 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f8eb008c6a0 con 0x7f8ec815e1e0 2024-03-28T13:12:56.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.789+0000 7f8ec7e00640 1 --2- 172.21.15.19:0/939802653 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8ea406c530 0x7f8ea406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:56.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.791+0000 7f8ec7e00640 1 --2- 172.21.15.19:0/939802653 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8ea406c530 0x7f8ea406e9f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8034000 tx=0x7f8eb8007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:56.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:56.794+0000 7f8ebfe00640 1 -- 172.21.15.19:0/939802653 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8eb0024030 con 0x7f8ec815e1e0 2024-03-28T13:12:57.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.037+0000 7f8ecda00640 1 -- 172.21.15.19:0/939802653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ec8071d40 con 0x7f8ec815e1e0 2024-03-28T13:12:57.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.038+0000 7f8ebfe00640 1 -- 172.21.15.19:0/939802653 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5187 (secure 0 0 0) 0x7f8eb005ad50 con 0x7f8ec815e1e0 2024-03-28T13:12:57.038 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:57.038 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1607201514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1607201514},{"type":"v1","addr":"172.21.15.188:6837","nonce":1607201514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1206740614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1206740614},{"type":"v1","addr":"172.21.15.188:6835","nonce":1206740614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2018671345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2018671345},{"type":"v1","addr":"172.21.15.19:6834","nonce":2018671345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:12:52.947573+0000","modified":"2024-03-28T13:12:56.032174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3341654519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3341654519},{"type":"v1","addr":"172.21.15.19:6835","nonce":3341654519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":4}]} 2024-03-28T13:12:57.039 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 34 2024-03-28T13:12:57.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.041+0000 7f8ecda00640 1 -- 172.21.15.19:0/939802653 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8ea406c530 msgr2=0x7f8ea406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:57.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.041+0000 7f8ecda00640 1 --2- 172.21.15.19:0/939802653 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8ea406c530 0x7f8ea406e9f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8034000 tx=0x7f8eb8007000 comp rx=0 tx=0).stop 2024-03-28T13:12:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.041+0000 7f8ecda00640 1 -- 172.21.15.19:0/939802653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec815e1e0 msgr2=0x7f8ec815d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.041+0000 7f8ecda00640 1 --2- 172.21.15.19:0/939802653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec815e1e0 0x7f8ec815d1c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8eb00114e0 tx=0x7f8eb00119a0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.041+0000 7f8ecda00640 1 -- 172.21.15.19:0/939802653 shutdown_connections 2024-03-28T13:12:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.041+0000 7f8ecda00640 1 --2- 172.21.15.19:0/939802653 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8ea406c530 0x7f8ea406e9f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.041+0000 7f8ecda00640 1 --2- 172.21.15.19:0/939802653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ec80b38e0 0x7f8ec80b3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.041+0000 7f8ecda00640 1 --2- 172.21.15.19:0/939802653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ec815e1e0 0x7f8ec815d1c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.041+0000 7f8ecda00640 1 --2- 172.21.15.19:0/939802653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8eac0039b0 0x7f8ec81440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.041+0000 7f8ecda00640 1 -- 172.21.15.19:0/939802653 >> 172.21.15.19:0/939802653 conn(0x7f8ec8067460 msgr2=0x7f8ec80b6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.041+0000 7f8ecda00640 1 -- 172.21.15.19:0/939802653 shutdown_connections 2024-03-28T13:12:57.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.041+0000 7f8ecda00640 1 -- 172.21.15.19:0/939802653 wait complete. 2024-03-28T13:12:57.061 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:12:57.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.277+0000 7fb365c00640 1 Processor -- start 2024-03-28T13:12:57.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.277+0000 7fb365c00640 1 -- start start 2024-03-28T13:12:57.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.279+0000 7fb365c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb36006ea90 0x7fb3600ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:57.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.279+0000 7fb365c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb360161f30 con 0x7fb360147a40 2024-03-28T13:12:57.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.279+0000 7fb365c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3601621d0 con 0x7fb3600d0390 2024-03-28T13:12:57.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.279+0000 7fb365c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb360162b00 con 0x7fb36006ea90 2024-03-28T13:12:57.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.279+0000 7fb35fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb360147a40 0x7fb3600cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52176/0 (socket says 172.21.15.19:52176) 2024-03-28T13:12:57.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.279+0000 7fb35fe00640 1 -- 172.21.15.19:0/606505018 learned_addr learned my addr 172.21.15.19:0/606505018 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:57.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb365200640 1 --2- 172.21.15.19:0/606505018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb36006ea90 0x7fb3600ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:57.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35ea00640 1 -- 172.21.15.19:0/606505018 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2137141139 0 0) 0x7fb360161f30 con 0x7fb360147a40 2024-03-28T13:12:57.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35ea00640 1 -- 172.21.15.19:0/606505018 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb344003520 con 0x7fb360147a40 2024-03-28T13:12:57.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35ea00640 1 -- 172.21.15.19:0/606505018 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 777352578 0 0) 0x7fb344003520 con 0x7fb360147a40 2024-03-28T13:12:57.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35ea00640 1 -- 172.21.15.19:0/606505018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb36006ea90 msgr2=0x7fb3600ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:57.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35ea00640 1 --2- 172.21.15.19:0/606505018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb36006ea90 0x7fb3600ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35ea00640 1 -- 172.21.15.19:0/606505018 >> v1:172.21.15.188:6789/0 conn(0x7fb3600d0390 legacy=0x7fb3600cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35ea00640 1 -- 172.21.15.19:0/606505018 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3601624b0 con 0x7fb360147a40 2024-03-28T13:12:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35ea00640 1 -- 172.21.15.19:0/606505018 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fb350001a10 con 0x7fb360147a40 2024-03-28T13:12:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35ea00640 1 -- 172.21.15.19:0/606505018 >> v1:172.21.15.19:6789/0 conn(0x7fb360147a40 legacy=0x7fb3600cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35ea00640 1 --2- 172.21.15.19:0/606505018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3440039e0 0x7fb344003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35ea00640 1 --2- 172.21.15.19:0/606505018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb36006ea90 0x7fb344004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35ea00640 1 --2- 172.21.15.19:0/606505018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb344004d40 0x7fb3440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35ea00640 1 -- 172.21.15.19:0/606505018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb360162b00 con 0x7fb3440039e0 2024-03-28T13:12:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35ea00640 1 -- 172.21.15.19:0/606505018 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb34400baa0 con 0x7fb344004d40 2024-03-28T13:12:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35ea00640 1 -- 172.21.15.19:0/606505018 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb34400bca0 con 0x7fb36006ea90 2024-03-28T13:12:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.280+0000 7fb35fe00640 1 --2- 172.21.15.19:0/606505018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3440039e0 0x7fb344003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.281+0000 7fb365200640 1 --2- 172.21.15.19:0/606505018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb344004d40 0x7fb3440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.281+0000 7fb35f400640 1 --2- 172.21.15.19:0/606505018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb36006ea90 0x7fb344004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:57.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.281+0000 7fb35fe00640 1 -- 172.21.15.19:0/606505018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb36006ea90 msgr2=0x7fb344004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.281+0000 7fb35fe00640 1 --2- 172.21.15.19:0/606505018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb36006ea90 0x7fb344004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.281+0000 7fb35fe00640 1 -- 172.21.15.19:0/606505018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb344004d40 msgr2=0x7fb3440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.281+0000 7fb35fe00640 1 --2- 172.21.15.19:0/606505018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb344004d40 0x7fb3440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.281+0000 7fb35fe00640 1 -- 172.21.15.19:0/606505018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb344003520 con 0x7fb3440039e0 2024-03-28T13:12:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.281+0000 7fb365200640 1 --2- 172.21.15.19:0/606505018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb344004d40 0x7fb3440051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.281+0000 7fb35f400640 1 --2- 172.21.15.19:0/606505018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb36006ea90 0x7fb344004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.282+0000 7fb35fe00640 1 --2- 172.21.15.19:0/606505018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3440039e0 0x7fb344003e00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb3600cc8c0 tx=0x7fb344005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45456c420c785ebb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.282+0000 7fb35ea00640 1 -- 172.21.15.19:0/606505018 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb35000a480 con 0x7fb3440039e0 2024-03-28T13:12:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.282+0000 7fb35ea00640 1 -- 172.21.15.19:0/606505018 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb350034650 con 0x7fb3440039e0 2024-03-28T13:12:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.282+0000 7fb35ea00640 1 -- 172.21.15.19:0/606505018 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350034a40 con 0x7fb3440039e0 2024-03-28T13:12:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.283+0000 7fb365c00640 1 -- 172.21.15.19:0/606505018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3440039e0 msgr2=0x7fb344003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.283+0000 7fb365c00640 1 --2- 172.21.15.19:0/606505018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3440039e0 0x7fb344003e00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb3600cc8c0 tx=0x7fb344005830 comp rx=0 tx=0).stop 2024-03-28T13:12:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.283+0000 7fb365200640 1 -- 172.21.15.19:0/606505018 reap_dead start 2024-03-28T13:12:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.283+0000 7fb365c00640 1 -- 172.21.15.19:0/606505018 shutdown_connections 2024-03-28T13:12:57.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.283+0000 7fb365c00640 1 -- 172.21.15.19:0/606505018 >> 172.21.15.19:0/606505018 conn(0x7fb3600ccad0 msgr2=0x7fb3600cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:57.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.283+0000 7fb365c00640 1 -- 172.21.15.19:0/606505018 shutdown_connections 2024-03-28T13:12:57.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.283+0000 7fb365c00640 1 -- 172.21.15.19:0/606505018 wait complete. 2024-03-28T13:12:57.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.286+0000 7fb365c00640 1 Processor -- start 2024-03-28T13:12:57.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.286+0000 7fb365c00640 1 -- start start 2024-03-28T13:12:57.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.288+0000 7fb365c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3440039e0 0x7fb3601f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:57.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.288+0000 7fb365c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb360139350 0x7fb3601397d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:57.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.288+0000 7fb365c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb36013b3f0 0x7fb36013b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:57.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.288+0000 7fb365c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb360147fa0 con 0x7fb360139350 2024-03-28T13:12:57.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.288+0000 7fb365c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb360147e20 con 0x7fb36013b3f0 2024-03-28T13:12:57.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.288+0000 7fb365c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb3600ce380 con 0x7fb3440039e0 2024-03-28T13:12:57.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.288+0000 7fb35f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb360139350 0x7fb3601397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:57.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.288+0000 7fb35f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb360139350 0x7fb3601397d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33794/0 (socket says 172.21.15.19:33794) 2024-03-28T13:12:57.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.288+0000 7fb35f400640 1 -- 172.21.15.19:0/11086384 learned_addr learned my addr 172.21.15.19:0/11086384 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.289+0000 7fb35fe00640 1 --2- 172.21.15.19:0/11086384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3440039e0 0x7fb3601f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.289+0000 7fb365200640 1 --2- 172.21.15.19:0/11086384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb36013b3f0 0x7fb36013b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.289+0000 7fb35f400640 1 -- 172.21.15.19:0/11086384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3440039e0 msgr2=0x7fb3601f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.289+0000 7fb35f400640 1 --2- 172.21.15.19:0/11086384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3440039e0 0x7fb3601f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.289+0000 7fb35f400640 1 -- 172.21.15.19:0/11086384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb36013b3f0 msgr2=0x7fb36013b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.289+0000 7fb35f400640 1 --2- 172.21.15.19:0/11086384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb36013b3f0 0x7fb36013b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.289+0000 7fb35f400640 1 -- 172.21.15.19:0/11086384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36013c1e0 con 0x7fb360139350 2024-03-28T13:12:57.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.289+0000 7fb365200640 1 --2- 172.21.15.19:0/11086384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb36013b3f0 0x7fb36013b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:57.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.289+0000 7fb35f400640 1 --2- 172.21.15.19:0/11086384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb360139350 0x7fb3601397d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00d7d0 tx=0x7fb34c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:57.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.289+0000 7fb357e00640 1 -- 172.21.15.19:0/11086384 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c005070 con 0x7fb360139350 2024-03-28T13:12:57.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.289+0000 7fb357e00640 1 -- 172.21.15.19:0/11086384 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb34c005230 con 0x7fb360139350 2024-03-28T13:12:57.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.289+0000 7fb357e00640 1 -- 172.21.15.19:0/11086384 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c007030 con 0x7fb360139350 2024-03-28T13:12:57.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.289+0000 7fb365c00640 1 -- 172.21.15.19:0/11086384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb36013c4c0 con 0x7fb360139350 2024-03-28T13:12:57.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.290+0000 7fb365c00640 1 -- 172.21.15.19:0/11086384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3601f5c50 con 0x7fb360139350 2024-03-28T13:12:57.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.291+0000 7fb365c00640 1 -- 172.21.15.19:0/11086384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb36013c6c0 con 0x7fb360139350 2024-03-28T13:12:57.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.292+0000 7fb357e00640 1 -- 172.21.15.19:0/11086384 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb34c01a020 con 0x7fb360139350 2024-03-28T13:12:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.293+0000 7fb357e00640 1 --2- 172.21.15.19:0/11086384 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb34006c560 0x7fb34006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.293+0000 7fb357e00640 1 -- 172.21.15.19:0/11086384 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fb34c08af80 con 0x7fb360139350 2024-03-28T13:12:57.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.293+0000 7fb35fe00640 1 --2- 172.21.15.19:0/11086384 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb34006c560 0x7fb34006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:57.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.295+0000 7fb35fe00640 1 --2- 172.21.15.19:0/11086384 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb34006c560 0x7fb34006ea20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb350034000 tx=0x7fb350009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:57.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.298+0000 7fb357e00640 1 -- 172.21.15.19:0/11086384 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb34c059500 con 0x7fb360139350 2024-03-28T13:12:57.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.501+0000 7fb365c00640 1 -- 172.21.15.19:0/11086384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb360075940 con 0x7fb360139350 2024-03-28T13:12:57.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.504+0000 7fb357e00640 1 -- 172.21.15.19:0/11086384 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16665 (secure 0 0 0) 0x7fb34c058e00 con 0x7fb360139350 2024-03-28T13:12:57.505 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:57.505 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":63,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:12:54.499829+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:12:50.888331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-28T13:12:51.946012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:12:57.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.507+0000 7fb365c00640 1 -- 172.21.15.19:0/11086384 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb34006c560 msgr2=0x7fb34006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:57.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.507+0000 7fb365c00640 1 --2- 172.21.15.19:0/11086384 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb34006c560 0x7fb34006ea20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb350034000 tx=0x7fb350009030 comp rx=0 tx=0).stop 2024-03-28T13:12:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.507+0000 7fb365c00640 1 -- 172.21.15.19:0/11086384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb360139350 msgr2=0x7fb3601397d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.507+0000 7fb365c00640 1 --2- 172.21.15.19:0/11086384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb360139350 0x7fb3601397d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00d7d0 tx=0x7fb34c00dc90 comp rx=0 tx=0).stop 2024-03-28T13:12:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.507+0000 7fb365c00640 1 -- 172.21.15.19:0/11086384 shutdown_connections 2024-03-28T13:12:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.507+0000 7fb365c00640 1 --2- 172.21.15.19:0/11086384 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb34006c560 0x7fb34006ea20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.507+0000 7fb365c00640 1 --2- 172.21.15.19:0/11086384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb36013b3f0 0x7fb36013b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.507+0000 7fb365c00640 1 --2- 172.21.15.19:0/11086384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb360139350 0x7fb3601397d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.507+0000 7fb365c00640 1 --2- 172.21.15.19:0/11086384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3440039e0 0x7fb3601f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.507+0000 7fb365c00640 1 -- 172.21.15.19:0/11086384 >> 172.21.15.19:0/11086384 conn(0x7fb3600ccad0 msgr2=0x7fb36015f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.507+0000 7fb365c00640 1 -- 172.21.15.19:0/11086384 shutdown_connections 2024-03-28T13:12:57.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.508+0000 7fb365c00640 1 -- 172.21.15.19:0/11086384 wait complete. 2024-03-28T13:12:57.529 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-28T13:12:57.529 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:12:57.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.738+0000 7f1b95600640 1 Processor -- start 2024-03-28T13:12:57.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.738+0000 7f1b95600640 1 -- start start 2024-03-28T13:12:57.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.740+0000 7f1b95600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b900cc0b0 0x7f1b9006dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.740+0000 7f1b95600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1b9015fc50 0x7f1b900cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.741+0000 7f1b95600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b9006e4f0 con 0x7f1b900cc490 2024-03-28T13:12:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.741+0000 7f1b95600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1b9015c610 con 0x7f1b9015fc50 2024-03-28T13:12:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.741+0000 7f1b95600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1b900b3150 con 0x7f1b900cc0b0 2024-03-28T13:12:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.741+0000 7f1b87e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1b9015fc50 0x7f1b900cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.741+0000 7f1b87e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1b9015fc50 0x7f1b900cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33806/0 (socket says 172.21.15.19:33806) 2024-03-28T13:12:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.741+0000 7f1b87e00640 1 -- 172.21.15.19:0/3973284660 learned_addr learned my addr 172.21.15.19:0/3973284660 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.741+0000 7f1b87e00640 1 -- 172.21.15.19:0/3973284660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b900cc0b0 msgr2=0x7f1b9006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.741+0000 7f1b8fe00640 1 --2- 172.21.15.19:0/3973284660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b900cc0b0 0x7f1b9006dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.742+0000 7f1b87e00640 1 --2- 172.21.15.19:0/3973284660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b900cc0b0 0x7f1b9006dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.742+0000 7f1b87e00640 1 -- 172.21.15.19:0/3973284660 >> v1:172.21.15.188:6789/0 conn(0x7f1b900cc490 legacy=0x7f1b9015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:57.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.742+0000 7f1b87e00640 1 -- 172.21.15.19:0/3973284660 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b901600e0 con 0x7f1b9015fc50 2024-03-28T13:12:57.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.742+0000 7f1b8fe00640 1 --2- 172.21.15.19:0/3973284660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b900cc0b0 0x7f1b9006dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:57.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.743+0000 7f1b87e00640 1 --2- 172.21.15.19:0/3973284660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b9015fc50 0x7f1b900cfdc0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009870 tx=0x7f1b7802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84ffa0d74169ec6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:57.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.743+0000 7f1b8f400640 1 -- 172.21.15.19:0/3973284660 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78034070 con 0x7f1b9015fc50 2024-03-28T13:12:57.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.743+0000 7f1b8f400640 1 -- 172.21.15.19:0/3973284660 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b78030840 con 0x7f1b9015fc50 2024-03-28T13:12:57.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.743+0000 7f1b8f400640 1 -- 172.21.15.19:0/3973284660 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78030c10 con 0x7f1b9015fc50 2024-03-28T13:12:57.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.744+0000 7f1b95600640 1 -- 172.21.15.19:0/3973284660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b9015fc50 msgr2=0x7f1b900cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.744+0000 7f1b95600640 1 --2- 172.21.15.19:0/3973284660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b9015fc50 0x7f1b900cfdc0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009870 tx=0x7f1b7802b980 comp rx=0 tx=0).stop 2024-03-28T13:12:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.744+0000 7f1b95600640 1 -- 172.21.15.19:0/3973284660 shutdown_connections 2024-03-28T13:12:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.744+0000 7f1b95600640 1 --2- 172.21.15.19:0/3973284660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b9015fc50 0x7f1b900cfdc0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.744+0000 7f1b95600640 1 --2- 172.21.15.19:0/3973284660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b900cc0b0 0x7f1b9006dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.744+0000 7f1b95600640 1 -- 172.21.15.19:0/3973284660 >> 172.21.15.19:0/3973284660 conn(0x7f1b900ccad0 msgr2=0x7f1b900cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.744+0000 7f1b95600640 1 -- 172.21.15.19:0/3973284660 shutdown_connections 2024-03-28T13:12:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.744+0000 7f1b95600640 1 -- 172.21.15.19:0/3973284660 wait complete. 2024-03-28T13:12:57.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.747+0000 7f1b95600640 1 Processor -- start 2024-03-28T13:12:57.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.747+0000 7f1b95600640 1 -- start start 2024-03-28T13:12:57.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.749+0000 7f1b95600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b900cc0b0 0x7f1b901f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:57.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.749+0000 7f1b95600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b900cc490 0x7f1b9015b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:57.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.749+0000 7f1b95600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b9015b670 0x7f1b900ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:57.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.749+0000 7f1b95600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b90160980 con 0x7f1b900cc0b0 2024-03-28T13:12:57.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.749+0000 7f1b95600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1b901606a0 con 0x7f1b9015b670 2024-03-28T13:12:57.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.749+0000 7f1b95600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1b900d0300 con 0x7f1b900cc490 2024-03-28T13:12:57.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.749+0000 7f1b8fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b900cc0b0 0x7f1b901f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:57.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.749+0000 7f1b8fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b900cc0b0 0x7f1b901f75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33820/0 (socket says 172.21.15.19:33820) 2024-03-28T13:12:57.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.749+0000 7f1b8fe00640 1 -- 172.21.15.19:0/3536992487 learned_addr learned my addr 172.21.15.19:0/3536992487 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:57.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.750+0000 7f1b87e00640 1 --2- 172.21.15.19:0/3536992487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b900cc490 0x7f1b9015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.750+0000 7f1b8fe00640 1 -- 172.21.15.19:0/3536992487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b900cc490 msgr2=0x7f1b9015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.750+0000 7f1b94c00640 1 --2- 172.21.15.19:0/3536992487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b9015b670 0x7f1b900ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.750+0000 7f1b8fe00640 1 --2- 172.21.15.19:0/3536992487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b900cc490 0x7f1b9015b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.750+0000 7f1b8fe00640 1 -- 172.21.15.19:0/3536992487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b9015b670 msgr2=0x7f1b900ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.750+0000 7f1b8fe00640 1 --2- 172.21.15.19:0/3536992487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b9015b670 0x7f1b900ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.750+0000 7f1b8fe00640 1 -- 172.21.15.19:0/3536992487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b900cfa10 con 0x7f1b900cc0b0 2024-03-28T13:12:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.750+0000 7f1b94c00640 1 --2- 172.21.15.19:0/3536992487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b9015b670 0x7f1b900ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.750+0000 7f1b8fe00640 1 --2- 172.21.15.19:0/3536992487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b900cc0b0 0x7f1b901f75a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000c7f0 tx=0x7f1b8000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.750+0000 7f1b8cc00640 1 -- 172.21.15.19:0/3536992487 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80015070 con 0x7f1b900cc0b0 2024-03-28T13:12:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.751+0000 7f1b8cc00640 1 -- 172.21.15.19:0/3536992487 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b80007a30 con 0x7f1b900cc0b0 2024-03-28T13:12:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.751+0000 7f1b8cc00640 1 -- 172.21.15.19:0/3536992487 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b80007e00 con 0x7f1b900cc0b0 2024-03-28T13:12:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.751+0000 7f1b95600640 1 -- 172.21.15.19:0/3536992487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b900cf450 con 0x7f1b900cc0b0 2024-03-28T13:12:57.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.751+0000 7f1b95600640 1 -- 172.21.15.19:0/3536992487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b900afc10 con 0x7f1b900cc0b0 2024-03-28T13:12:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.752+0000 7f1b95600640 1 -- 172.21.15.19:0/3536992487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b900b4430 con 0x7f1b900cc0b0 2024-03-28T13:12:57.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.753+0000 7f1b8cc00640 1 -- 172.21.15.19:0/3536992487 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1b80018070 con 0x7f1b900cc0b0 2024-03-28T13:12:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.755+0000 7f1b8cc00640 1 --2- 172.21.15.19:0/3536992487 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1b6c06c560 0x7f1b6c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:57.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.755+0000 7f1b8cc00640 1 -- 172.21.15.19:0/3536992487 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f1b8008b6a0 con 0x7f1b900cc0b0 2024-03-28T13:12:57.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.755+0000 7f1b87e00640 1 --2- 172.21.15.19:0/3536992487 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1b6c06c560 0x7f1b6c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:57.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.756+0000 7f1b87e00640 1 --2- 172.21.15.19:0/3536992487 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1b6c06c560 0x7f1b6c06ea20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1b78036000 tx=0x7f1b7803b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:57.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:57.759+0000 7f1b8cc00640 1 -- 172.21.15.19:0/3536992487 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1b900b4430 con 0x7f1b900cc0b0 2024-03-28T13:12:57.878 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-03-28T13:12:57.878+0000 7fdf43600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:12:58.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.074+0000 7f1b95600640 1 -- 172.21.15.19:0/3536992487 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1b90075940 con 0x7f1b6c06c560 2024-03-28T13:12:58.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.093+0000 7f1b8cc00640 1 -- 172.21.15.19:0/3536992487 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251923 (secure 0 0 0) 0x7f1b90075940 con 0x7f1b6c06c560 2024-03-28T13:12:58.098 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:58.100 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:12:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.098+0000 7f1b95600640 1 -- 172.21.15.19:0/3536992487 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1b6c06c560 msgr2=0x7f1b6c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.098+0000 7f1b95600640 1 --2- 172.21.15.19:0/3536992487 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1b6c06c560 0x7f1b6c06ea20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1b78036000 tx=0x7f1b7803b040 comp rx=0 tx=0).stop 2024-03-28T13:12:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.098+0000 7f1b95600640 1 -- 172.21.15.19:0/3536992487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b900cc0b0 msgr2=0x7f1b901f75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.098+0000 7f1b95600640 1 --2- 172.21.15.19:0/3536992487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b900cc0b0 0x7f1b901f75a0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000c7f0 tx=0x7f1b8000cbf0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.099+0000 7f1b95600640 1 -- 172.21.15.19:0/3536992487 shutdown_connections 2024-03-28T13:12:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.099+0000 7f1b95600640 1 --2- 172.21.15.19:0/3536992487 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1b6c06c560 0x7f1b6c06ea20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.099+0000 7f1b95600640 1 --2- 172.21.15.19:0/3536992487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b9015b670 0x7f1b900ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.099+0000 7f1b95600640 1 --2- 172.21.15.19:0/3536992487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b900cc490 0x7f1b9015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.099+0000 7f1b95600640 1 --2- 172.21.15.19:0/3536992487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b900cc0b0 0x7f1b901f75a0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.099+0000 7f1b95600640 1 -- 172.21.15.19:0/3536992487 >> 172.21.15.19:0/3536992487 conn(0x7f1b900ccad0 msgr2=0x7f1b9015cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.099+0000 7f1b95600640 1 -- 172.21.15.19:0/3536992487 shutdown_connections 2024-03-28T13:12:58.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.099+0000 7f1b95600640 1 -- 172.21.15.19:0/3536992487 wait complete. 2024-03-28T13:12:58.126 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-03-28T13:12:58.054165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243916,"kb_used_data":6928,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730804,"statfs":{"total":773094113280,"available":772844343296,"internally_reserved":0,"allocated":7094272,"data_stored":3623592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":242535988},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001998"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.528724+0000","last_change":"2024-03-28T13:12:53.525879+0000","last_active":"2024-03-28T13:12:53.528724+0000","last_peered":"2024-03-28T13:12:53.528724+0000","last_clean":"2024-03-28T13:12:53.528724+0000","last_became_active":"2024-03-28T13:12:53.525279+0000","last_became_peered":"2024-03-28T13:12:53.525279+0000","last_unstale":"2024-03-28T13:12:53.528724+0000","last_undegraded":"2024-03-28T13:12:53.528724+0000","last_fullsized":"2024-03-28T13:12:53.528724+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:46:18.555821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.532157+0000","last_change":"2024-03-28T13:12:53.530114+0000","last_active":"2024-03-28T13:12:56.532157+0000","last_peered":"2024-03-28T13:12:56.532157+0000","last_clean":"2024-03-28T13:12:56.532157+0000","last_became_active":"2024-03-28T13:12:53.529820+0000","last_became_peered":"2024-03-28T13:12:53.529820+0000","last_unstale":"2024-03-28T13:12:56.532157+0000","last_undegraded":"2024-03-28T13:12:56.532157+0000","last_fullsized":"2024-03-28T13:12:56.532157+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:55:07.763584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523109+0000","last_change":"2024-03-28T13:12:53.517605+0000","last_active":"2024-03-28T13:12:54.523109+0000","last_peered":"2024-03-28T13:12:54.523109+0000","last_clean":"2024-03-28T13:12:54.523109+0000","last_became_active":"2024-03-28T13:12:53.517285+0000","last_became_peered":"2024-03-28T13:12:53.517285+0000","last_unstale":"2024-03-28T13:12:54.523109+0000","last_undegraded":"2024-03-28T13:12:54.523109+0000","last_fullsized":"2024-03-28T13:12:54.523109+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:27:53.920916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513660+0000","last_change":"2024-03-28T13:12:53.519742+0000","last_active":"2024-03-28T13:12:54.513660+0000","last_peered":"2024-03-28T13:12:54.513660+0000","last_clean":"2024-03-28T13:12:54.513660+0000","last_became_active":"2024-03-28T13:12:53.519465+0000","last_became_peered":"2024-03-28T13:12:53.519465+0000","last_unstale":"2024-03-28T13:12:54.513660+0000","last_undegraded":"2024-03-28T13:12:54.513660+0000","last_fullsized":"2024-03-28T13:12:54.513660+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:29:47.754192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.526802+0000","last_change":"2024-03-28T13:12:53.520736+0000","last_active":"2024-03-28T13:12:53.526802+0000","last_peered":"2024-03-28T13:12:53.526802+0000","last_clean":"2024-03-28T13:12:53.526802+0000","last_became_active":"2024-03-28T13:12:53.520435+0000","last_became_peered":"2024-03-28T13:12:53.520435+0000","last_unstale":"2024-03-28T13:12:53.526802+0000","last_undegraded":"2024-03-28T13:12:53.526802+0000","last_fullsized":"2024-03-28T13:12:53.526802+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:54:28.232197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.525167+0000","last_change":"2024-03-28T13:12:53.518578+0000","last_active":"2024-03-28T13:12:53.525167+0000","last_peered":"2024-03-28T13:12:53.525167+0000","last_clean":"2024-03-28T13:12:53.525167+0000","last_became_active":"2024-03-28T13:12:53.518270+0000","last_became_peered":"2024-03-28T13:12:53.518270+0000","last_unstale":"2024-03-28T13:12:53.525167+0000","last_undegraded":"2024-03-28T13:12:53.525167+0000","last_fullsized":"2024-03-28T13:12:53.525167+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:35.524688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513081+0000","last_change":"2024-03-28T13:12:53.514524+0000","last_active":"2024-03-28T13:12:54.513081+0000","last_peered":"2024-03-28T13:12:54.513081+0000","last_clean":"2024-03-28T13:12:54.513081+0000","last_became_active":"2024-03-28T13:12:53.514203+0000","last_became_peered":"2024-03-28T13:12:53.514203+0000","last_unstale":"2024-03-28T13:12:54.513081+0000","last_undegraded":"2024-03-28T13:12:54.513081+0000","last_fullsized":"2024-03-28T13:12:54.513081+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:16:40.025156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.516516+0000","last_change":"2024-03-28T13:12:53.501343+0000","last_active":"2024-03-28T13:12:53.516516+0000","last_peered":"2024-03-28T13:12:53.516516+0000","last_clean":"2024-03-28T13:12:53.516516+0000","last_became_active":"2024-03-28T13:12:53.500816+0000","last_became_peered":"2024-03-28T13:12:53.500816+0000","last_unstale":"2024-03-28T13:12:53.516516+0000","last_undegraded":"2024-03-28T13:12:53.516516+0000","last_fullsized":"2024-03-28T13:12:53.516516+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:41:30.265558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.529988+0000","last_change":"2024-03-28T13:12:53.525511+0000","last_active":"2024-03-28T13:12:53.529988+0000","last_peered":"2024-03-28T13:12:53.529988+0000","last_clean":"2024-03-28T13:12:53.529988+0000","last_became_active":"2024-03-28T13:12:53.525233+0000","last_became_peered":"2024-03-28T13:12:53.525233+0000","last_unstale":"2024-03-28T13:12:53.529988+0000","last_undegraded":"2024-03-28T13:12:53.529988+0000","last_fullsized":"2024-03-28T13:12:53.529988+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:51:17.865300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.526884+0000","last_change":"2024-03-28T13:12:53.521826+0000","last_active":"2024-03-28T13:12:53.526884+0000","last_peered":"2024-03-28T13:12:53.526884+0000","last_clean":"2024-03-28T13:12:53.526884+0000","last_became_active":"2024-03-28T13:12:53.521479+0000","last_became_peered":"2024-03-28T13:12:53.521479+0000","last_unstale":"2024-03-28T13:12:53.526884+0000","last_undegraded":"2024-03-28T13:12:53.526884+0000","last_fullsized":"2024-03-28T13:12:53.526884+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:16:16.539313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.532901+0000","last_change":"2024-03-28T13:12:53.526182+0000","last_active":"2024-03-28T13:12:53.532901+0000","last_peered":"2024-03-28T13:12:53.532901+0000","last_clean":"2024-03-28T13:12:53.532901+0000","last_became_active":"2024-03-28T13:12:53.525868+0000","last_became_peered":"2024-03-28T13:12:53.525868+0000","last_unstale":"2024-03-28T13:12:53.532901+0000","last_undegraded":"2024-03-28T13:12:53.532901+0000","last_fullsized":"2024-03-28T13:12:53.532901+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:16:16.539313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523498+0000","last_change":"2024-03-28T13:12:53.519176+0000","last_active":"2024-03-28T13:12:53.523498+0000","last_peered":"2024-03-28T13:12:53.523498+0000","last_clean":"2024-03-28T13:12:53.523498+0000","last_became_active":"2024-03-28T13:12:53.518852+0000","last_became_peered":"2024-03-28T13:12:53.518852+0000","last_unstale":"2024-03-28T13:12:53.523498+0000","last_undegraded":"2024-03-28T13:12:53.523498+0000","last_fullsized":"2024-03-28T13:12:53.523498+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:34:20.994120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.553831+0000","last_change":"2024-03-28T13:12:53.525369+0000","last_active":"2024-03-28T13:12:54.553831+0000","last_peered":"2024-03-28T13:12:54.553831+0000","last_clean":"2024-03-28T13:12:54.553831+0000","last_became_active":"2024-03-28T13:12:53.524809+0000","last_became_peered":"2024-03-28T13:12:53.524809+0000","last_unstale":"2024-03-28T13:12:54.553831+0000","last_undegraded":"2024-03-28T13:12:54.553831+0000","last_fullsized":"2024-03-28T13:12:54.553831+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:44.888405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.522846+0000","last_change":"2024-03-28T13:12:53.527839+0000","last_active":"2024-03-28T13:12:54.522846+0000","last_peered":"2024-03-28T13:12:54.522846+0000","last_clean":"2024-03-28T13:12:54.522846+0000","last_became_active":"2024-03-28T13:12:53.527486+0000","last_became_peered":"2024-03-28T13:12:53.527486+0000","last_unstale":"2024-03-28T13:12:54.522846+0000","last_undegraded":"2024-03-28T13:12:54.522846+0000","last_fullsized":"2024-03-28T13:12:54.522846+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:41:30.265558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.553642+0000","last_change":"2024-03-28T13:12:53.530948+0000","last_active":"2024-03-28T13:12:54.553642+0000","last_peered":"2024-03-28T13:12:54.553642+0000","last_clean":"2024-03-28T13:12:54.553642+0000","last_became_active":"2024-03-28T13:12:53.530682+0000","last_became_peered":"2024-03-28T13:12:53.530682+0000","last_unstale":"2024-03-28T13:12:54.553642+0000","last_undegraded":"2024-03-28T13:12:54.553642+0000","last_fullsized":"2024-03-28T13:12:54.553642+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:26:05.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.524962+0000","last_change":"2024-03-28T13:12:53.516104+0000","last_active":"2024-03-28T13:12:53.524962+0000","last_peered":"2024-03-28T13:12:53.524962+0000","last_clean":"2024-03-28T13:12:53.524962+0000","last_became_active":"2024-03-28T13:12:53.515238+0000","last_became_peered":"2024-03-28T13:12:53.515238+0000","last_unstale":"2024-03-28T13:12:53.524962+0000","last_undegraded":"2024-03-28T13:12:53.524962+0000","last_fullsized":"2024-03-28T13:12:53.524962+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:42:34.599981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.534771+0000","last_change":"2024-03-28T13:12:53.528585+0000","last_active":"2024-03-28T13:12:53.534771+0000","last_peered":"2024-03-28T13:12:53.534771+0000","last_clean":"2024-03-28T13:12:53.534771+0000","last_became_active":"2024-03-28T13:12:53.528264+0000","last_became_peered":"2024-03-28T13:12:53.528264+0000","last_unstale":"2024-03-28T13:12:53.534771+0000","last_undegraded":"2024-03-28T13:12:53.534771+0000","last_fullsized":"2024-03-28T13:12:53.534771+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:26:05.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523073+0000","last_change":"2024-03-28T13:12:53.529601+0000","last_active":"2024-03-28T13:12:54.523073+0000","last_peered":"2024-03-28T13:12:54.523073+0000","last_clean":"2024-03-28T13:12:54.523073+0000","last_became_active":"2024-03-28T13:12:53.529250+0000","last_became_peered":"2024-03-28T13:12:53.529250+0000","last_unstale":"2024-03-28T13:12:54.523073+0000","last_undegraded":"2024-03-28T13:12:54.523073+0000","last_fullsized":"2024-03-28T13:12:54.523073+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:02:06.529277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.531676+0000","last_change":"2024-03-28T13:12:53.530792+0000","last_active":"2024-03-28T13:12:56.531676+0000","last_peered":"2024-03-28T13:12:56.531676+0000","last_clean":"2024-03-28T13:12:56.531676+0000","last_became_active":"2024-03-28T13:12:53.530071+0000","last_became_peered":"2024-03-28T13:12:53.530071+0000","last_unstale":"2024-03-28T13:12:56.531676+0000","last_undegraded":"2024-03-28T13:12:56.531676+0000","last_fullsized":"2024-03-28T13:12:56.531676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:06:17.410323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.535355+0000","last_change":"2024-03-28T13:12:53.526799+0000","last_active":"2024-03-28T13:12:53.535355+0000","last_peered":"2024-03-28T13:12:53.535355+0000","last_clean":"2024-03-28T13:12:53.535355+0000","last_became_active":"2024-03-28T13:12:53.526520+0000","last_became_peered":"2024-03-28T13:12:53.526520+0000","last_unstale":"2024-03-28T13:12:53.535355+0000","last_undegraded":"2024-03-28T13:12:53.535355+0000","last_fullsized":"2024-03-28T13:12:53.535355+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:46:18.555821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.553773+0000","last_change":"2024-03-28T13:12:53.520815+0000","last_active":"2024-03-28T13:12:54.553773+0000","last_peered":"2024-03-28T13:12:54.553773+0000","last_clean":"2024-03-28T13:12:54.553773+0000","last_became_active":"2024-03-28T13:12:53.520536+0000","last_became_peered":"2024-03-28T13:12:53.520536+0000","last_unstale":"2024-03-28T13:12:54.553773+0000","last_undegraded":"2024-03-28T13:12:54.553773+0000","last_fullsized":"2024-03-28T13:12:54.553773+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:18:34.253144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.538076+0000","last_change":"2024-03-28T13:12:53.535551+0000","last_active":"2024-03-28T13:12:53.538076+0000","last_peered":"2024-03-28T13:12:53.538076+0000","last_clean":"2024-03-28T13:12:53.538076+0000","last_became_active":"2024-03-28T13:12:53.533181+0000","last_became_peered":"2024-03-28T13:12:53.533181+0000","last_unstale":"2024-03-28T13:12:53.538076+0000","last_undegraded":"2024-03-28T13:12:53.538076+0000","last_fullsized":"2024-03-28T13:12:53.538076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:44.888405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513088+0000","last_change":"2024-03-28T13:12:53.533475+0000","last_active":"2024-03-28T13:12:54.513088+0000","last_peered":"2024-03-28T13:12:54.513088+0000","last_clean":"2024-03-28T13:12:54.513088+0000","last_became_active":"2024-03-28T13:12:53.532757+0000","last_became_peered":"2024-03-28T13:12:53.532757+0000","last_unstale":"2024-03-28T13:12:54.513088+0000","last_undegraded":"2024-03-28T13:12:54.513088+0000","last_fullsized":"2024-03-28T13:12:54.513088+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:55:36.036411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.517789+0000","last_change":"2024-03-28T13:12:53.514611+0000","last_active":"2024-03-28T13:12:53.517789+0000","last_peered":"2024-03-28T13:12:53.517789+0000","last_clean":"2024-03-28T13:12:53.517789+0000","last_became_active":"2024-03-28T13:12:53.511618+0000","last_became_peered":"2024-03-28T13:12:53.511618+0000","last_unstale":"2024-03-28T13:12:53.517789+0000","last_undegraded":"2024-03-28T13:12:53.517789+0000","last_fullsized":"2024-03-28T13:12:53.517789+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:36.484476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.531699+0000","last_change":"2024-03-28T13:12:53.532707+0000","last_active":"2024-03-28T13:12:56.531699+0000","last_peered":"2024-03-28T13:12:56.531699+0000","last_clean":"2024-03-28T13:12:56.531699+0000","last_became_active":"2024-03-28T13:12:53.529446+0000","last_became_peered":"2024-03-28T13:12:53.529446+0000","last_unstale":"2024-03-28T13:12:56.531699+0000","last_undegraded":"2024-03-28T13:12:56.531699+0000","last_fullsized":"2024-03-28T13:12:56.531699+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:19:19.903928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.554016+0000","last_change":"2024-03-28T13:12:53.524731+0000","last_active":"2024-03-28T13:12:54.554016+0000","last_peered":"2024-03-28T13:12:54.554016+0000","last_clean":"2024-03-28T13:12:54.554016+0000","last_became_active":"2024-03-28T13:12:53.524429+0000","last_became_peered":"2024-03-28T13:12:53.524429+0000","last_unstale":"2024-03-28T13:12:54.554016+0000","last_undegraded":"2024-03-28T13:12:54.554016+0000","last_fullsized":"2024-03-28T13:12:54.554016+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:41:30.265558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.531997+0000","last_change":"2024-03-28T13:12:53.534582+0000","last_active":"2024-03-28T13:12:56.531997+0000","last_peered":"2024-03-28T13:12:56.531997+0000","last_clean":"2024-03-28T13:12:56.531997+0000","last_became_active":"2024-03-28T13:12:53.533780+0000","last_became_peered":"2024-03-28T13:12:53.533780+0000","last_unstale":"2024-03-28T13:12:56.531997+0000","last_undegraded":"2024-03-28T13:12:56.531997+0000","last_fullsized":"2024-03-28T13:12:56.531997+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:53:53.398050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523778+0000","last_change":"2024-03-28T13:12:53.514047+0000","last_active":"2024-03-28T13:12:53.523778+0000","last_peered":"2024-03-28T13:12:53.523778+0000","last_clean":"2024-03-28T13:12:53.523778+0000","last_became_active":"2024-03-28T13:12:53.513768+0000","last_became_peered":"2024-03-28T13:12:53.513768+0000","last_unstale":"2024-03-28T13:12:53.523778+0000","last_undegraded":"2024-03-28T13:12:53.523778+0000","last_fullsized":"2024-03-28T13:12:53.523778+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:29:02.546768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.516679+0000","last_change":"2024-03-28T13:12:53.503745+0000","last_active":"2024-03-28T13:12:53.516679+0000","last_peered":"2024-03-28T13:12:53.516679+0000","last_clean":"2024-03-28T13:12:53.516679+0000","last_became_active":"2024-03-28T13:12:53.503222+0000","last_became_peered":"2024-03-28T13:12:53.503222+0000","last_unstale":"2024-03-28T13:12:53.516679+0000","last_undegraded":"2024-03-28T13:12:53.516679+0000","last_fullsized":"2024-03-28T13:12:53.516679+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:02:06.529277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513385+0000","last_change":"2024-03-28T13:12:53.526488+0000","last_active":"2024-03-28T13:12:54.513385+0000","last_peered":"2024-03-28T13:12:54.513385+0000","last_clean":"2024-03-28T13:12:54.513385+0000","last_became_active":"2024-03-28T13:12:53.526193+0000","last_became_peered":"2024-03-28T13:12:53.526193+0000","last_unstale":"2024-03-28T13:12:54.513385+0000","last_undegraded":"2024-03-28T13:12:54.513385+0000","last_fullsized":"2024-03-28T13:12:54.513385+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:54:18.665506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.530877+0000","last_change":"2024-03-28T13:12:53.527407+0000","last_active":"2024-03-28T13:12:56.530877+0000","last_peered":"2024-03-28T13:12:56.530877+0000","last_clean":"2024-03-28T13:12:56.530877+0000","last_became_active":"2024-03-28T13:12:53.527110+0000","last_became_peered":"2024-03-28T13:12:53.527110+0000","last_unstale":"2024-03-28T13:12:56.530877+0000","last_undegraded":"2024-03-28T13:12:56.530877+0000","last_fullsized":"2024-03-28T13:12:56.530877+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:09:31.626615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.536760+0000","last_change":"2024-03-28T13:12:53.533474+0000","last_active":"2024-03-28T13:12:53.536760+0000","last_peered":"2024-03-28T13:12:53.536760+0000","last_clean":"2024-03-28T13:12:53.536760+0000","last_became_active":"2024-03-28T13:12:53.533211+0000","last_became_peered":"2024-03-28T13:12:53.533211+0000","last_unstale":"2024-03-28T13:12:53.536760+0000","last_undegraded":"2024-03-28T13:12:53.536760+0000","last_fullsized":"2024-03-28T13:12:53.536760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:18:34.253144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.526861+0000","last_change":"2024-03-28T13:12:53.521211+0000","last_active":"2024-03-28T13:12:53.526861+0000","last_peered":"2024-03-28T13:12:53.526861+0000","last_clean":"2024-03-28T13:12:53.526861+0000","last_became_active":"2024-03-28T13:12:53.518832+0000","last_became_peered":"2024-03-28T13:12:53.518832+0000","last_unstale":"2024-03-28T13:12:53.526861+0000","last_undegraded":"2024-03-28T13:12:53.526861+0000","last_fullsized":"2024-03-28T13:12:53.526861+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:43.494458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"62'4","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.532064+0000","last_change":"2024-03-28T13:12:52.497633+0000","last_active":"2024-03-28T13:12:53.532064+0000","last_peered":"2024-03-28T13:12:53.532064+0000","last_clean":"2024-03-28T13:12:53.532064+0000","last_became_active":"2024-03-28T13:12:52.497326+0000","last_became_peered":"2024-03-28T13:12:52.497326+0000","last_unstale":"2024-03-28T13:12:53.532064+0000","last_undegraded":"2024-03-28T13:12:53.532064+0000","last_fullsized":"2024-03-28T13:12:53.532064+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:54:27.217786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.534799+0000","last_change":"2024-03-28T13:12:53.529202+0000","last_active":"2024-03-28T13:12:53.534799+0000","last_peered":"2024-03-28T13:12:53.534799+0000","last_clean":"2024-03-28T13:12:53.534799+0000","last_became_active":"2024-03-28T13:12:53.526303+0000","last_became_peered":"2024-03-28T13:12:53.526303+0000","last_unstale":"2024-03-28T13:12:53.534799+0000","last_undegraded":"2024-03-28T13:12:53.534799+0000","last_fullsized":"2024-03-28T13:12:53.534799+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:18:34.253144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.532432+0000","last_change":"2024-03-28T13:12:52.490509+0000","last_active":"2024-03-28T13:12:56.532432+0000","last_peered":"2024-03-28T13:12:56.532432+0000","last_clean":"2024-03-28T13:12:56.532432+0000","last_became_active":"2024-03-28T13:12:52.490224+0000","last_became_peered":"2024-03-28T13:12:52.490224+0000","last_unstale":"2024-03-28T13:12:56.532432+0000","last_undegraded":"2024-03-28T13:12:56.532432+0000","last_fullsized":"2024-03-28T13:12:56.532432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:05.546468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.532373+0000","last_change":"2024-03-28T13:12:53.523604+0000","last_active":"2024-03-28T13:12:53.532373+0000","last_peered":"2024-03-28T13:12:53.532373+0000","last_clean":"2024-03-28T13:12:53.532373+0000","last_became_active":"2024-03-28T13:12:53.523220+0000","last_became_peered":"2024-03-28T13:12:53.523220+0000","last_unstale":"2024-03-28T13:12:53.532373+0000","last_undegraded":"2024-03-28T13:12:53.532373+0000","last_fullsized":"2024-03-28T13:12:53.532373+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:48:21.538965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.538179+0000","last_change":"2024-03-28T13:12:52.497064+0000","last_active":"2024-03-28T13:12:53.538179+0000","last_peered":"2024-03-28T13:12:53.538179+0000","last_clean":"2024-03-28T13:12:53.538179+0000","last_became_active":"2024-03-28T13:12:52.496174+0000","last_became_peered":"2024-03-28T13:12:52.496174+0000","last_unstale":"2024-03-28T13:12:53.538179+0000","last_undegraded":"2024-03-28T13:12:53.538179+0000","last_fullsized":"2024-03-28T13:12:53.538179+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:30.940424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.534691+0000","last_change":"2024-03-28T13:12:53.529791+0000","last_active":"2024-03-28T13:12:53.534691+0000","last_peered":"2024-03-28T13:12:53.534691+0000","last_clean":"2024-03-28T13:12:53.534691+0000","last_became_active":"2024-03-28T13:12:53.529522+0000","last_became_peered":"2024-03-28T13:12:53.529522+0000","last_unstale":"2024-03-28T13:12:53.534691+0000","last_undegraded":"2024-03-28T13:12:53.534691+0000","last_fullsized":"2024-03-28T13:12:53.534691+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:44.888405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"62'2","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513761+0000","last_change":"2024-03-28T13:12:52.472395+0000","last_active":"2024-03-28T13:12:54.513761+0000","last_peered":"2024-03-28T13:12:54.513761+0000","last_clean":"2024-03-28T13:12:54.513761+0000","last_became_active":"2024-03-28T13:12:52.471810+0000","last_became_peered":"2024-03-28T13:12:52.471810+0000","last_unstale":"2024-03-28T13:12:54.513761+0000","last_undegraded":"2024-03-28T13:12:54.513761+0000","last_fullsized":"2024-03-28T13:12:54.513761+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:47:30.323323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523777+0000","last_change":"2024-03-28T13:12:53.517800+0000","last_active":"2024-03-28T13:12:53.523777+0000","last_peered":"2024-03-28T13:12:53.523777+0000","last_clean":"2024-03-28T13:12:53.523777+0000","last_became_active":"2024-03-28T13:12:53.517514+0000","last_became_peered":"2024-03-28T13:12:53.517514+0000","last_unstale":"2024-03-28T13:12:53.523777+0000","last_undegraded":"2024-03-28T13:12:53.523777+0000","last_fullsized":"2024-03-28T13:12:53.523777+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:50:31.090629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.522731+0000","last_change":"2024-03-28T13:12:52.498789+0000","last_active":"2024-03-28T13:12:54.522731+0000","last_peered":"2024-03-28T13:12:54.522731+0000","last_clean":"2024-03-28T13:12:54.522731+0000","last_became_active":"2024-03-28T13:12:52.498391+0000","last_became_peered":"2024-03-28T13:12:52.498391+0000","last_unstale":"2024-03-28T13:12:54.522731+0000","last_undegraded":"2024-03-28T13:12:54.522731+0000","last_fullsized":"2024-03-28T13:12:54.522731+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:21.358714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.535449+0000","last_change":"2024-03-28T13:12:53.523100+0000","last_active":"2024-03-28T13:12:53.535449+0000","last_peered":"2024-03-28T13:12:53.535449+0000","last_clean":"2024-03-28T13:12:53.535449+0000","last_became_active":"2024-03-28T13:12:53.522846+0000","last_became_peered":"2024-03-28T13:12:53.522846+0000","last_unstale":"2024-03-28T13:12:53.535449+0000","last_undegraded":"2024-03-28T13:12:53.535449+0000","last_fullsized":"2024-03-28T13:12:53.535449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:26:05.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.533964+0000","last_change":"2024-03-28T13:12:52.493590+0000","last_active":"2024-03-28T13:12:53.533964+0000","last_peered":"2024-03-28T13:12:53.533964+0000","last_clean":"2024-03-28T13:12:53.533964+0000","last_became_active":"2024-03-28T13:12:52.493306+0000","last_became_peered":"2024-03-28T13:12:52.493306+0000","last_unstale":"2024-03-28T13:12:53.533964+0000","last_undegraded":"2024-03-28T13:12:53.533964+0000","last_fullsized":"2024-03-28T13:12:53.533964+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:36:31.168233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.522490+0000","last_change":"2024-03-28T13:12:53.517696+0000","last_active":"2024-03-28T13:12:54.522490+0000","last_peered":"2024-03-28T13:12:54.522490+0000","last_clean":"2024-03-28T13:12:54.522490+0000","last_became_active":"2024-03-28T13:12:53.513400+0000","last_became_peered":"2024-03-28T13:12:53.513400+0000","last_unstale":"2024-03-28T13:12:54.522490+0000","last_undegraded":"2024-03-28T13:12:54.522490+0000","last_fullsized":"2024-03-28T13:12:54.522490+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:44.888405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.486660+0000","last_change":"2024-03-28T13:12:52.490236+0000","last_active":"2024-03-28T13:12:53.486660+0000","last_peered":"2024-03-28T13:12:53.486660+0000","last_clean":"2024-03-28T13:12:53.486660+0000","last_became_active":"2024-03-28T13:12:52.489880+0000","last_became_peered":"2024-03-28T13:12:52.489880+0000","last_unstale":"2024-03-28T13:12:53.486660+0000","last_undegraded":"2024-03-28T13:12:53.486660+0000","last_fullsized":"2024-03-28T13:12:53.486660+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:12:13.076721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.525049+0000","last_change":"2024-03-28T13:12:53.515864+0000","last_active":"2024-03-28T13:12:53.525049+0000","last_peered":"2024-03-28T13:12:53.525049+0000","last_clean":"2024-03-28T13:12:53.525049+0000","last_became_active":"2024-03-28T13:12:53.515528+0000","last_became_peered":"2024-03-28T13:12:53.515528+0000","last_unstale":"2024-03-28T13:12:53.525049+0000","last_undegraded":"2024-03-28T13:12:53.525049+0000","last_fullsized":"2024-03-28T13:12:53.525049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:20:57.766567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.553494+0000","last_change":"2024-03-28T13:12:52.497223+0000","last_active":"2024-03-28T13:12:54.553494+0000","last_peered":"2024-03-28T13:12:54.553494+0000","last_clean":"2024-03-28T13:12:54.553494+0000","last_became_active":"2024-03-28T13:12:52.496968+0000","last_became_peered":"2024-03-28T13:12:52.496968+0000","last_unstale":"2024-03-28T13:12:54.553494+0000","last_undegraded":"2024-03-28T13:12:54.553494+0000","last_fullsized":"2024-03-28T13:12:54.553494+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:21.414302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.554104+0000","last_change":"2024-03-28T13:12:53.521865+0000","last_active":"2024-03-28T13:12:54.554104+0000","last_peered":"2024-03-28T13:12:54.554104+0000","last_clean":"2024-03-28T13:12:54.554104+0000","last_became_active":"2024-03-28T13:12:53.521470+0000","last_became_peered":"2024-03-28T13:12:53.521470+0000","last_unstale":"2024-03-28T13:12:54.554104+0000","last_undegraded":"2024-03-28T13:12:54.554104+0000","last_fullsized":"2024-03-28T13:12:54.554104+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:02:06.529277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.489279+0000","last_change":"2024-03-28T13:12:52.493670+0000","last_active":"2024-03-28T13:12:53.489279+0000","last_peered":"2024-03-28T13:12:53.489279+0000","last_clean":"2024-03-28T13:12:53.489279+0000","last_became_active":"2024-03-28T13:12:52.493461+0000","last_became_peered":"2024-03-28T13:12:52.493461+0000","last_unstale":"2024-03-28T13:12:53.489279+0000","last_undegraded":"2024-03-28T13:12:53.489279+0000","last_fullsized":"2024-03-28T13:12:53.489279+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:34:12.832012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523240+0000","last_change":"2024-03-28T13:12:53.529215+0000","last_active":"2024-03-28T13:12:54.523240+0000","last_peered":"2024-03-28T13:12:54.523240+0000","last_clean":"2024-03-28T13:12:54.523240+0000","last_became_active":"2024-03-28T13:12:53.528645+0000","last_became_peered":"2024-03-28T13:12:53.528645+0000","last_unstale":"2024-03-28T13:12:54.523240+0000","last_undegraded":"2024-03-28T13:12:54.523240+0000","last_fullsized":"2024-03-28T13:12:54.523240+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:29:02.546768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.492468+0000","last_change":"2024-03-28T13:12:52.497766+0000","last_active":"2024-03-28T13:12:53.492468+0000","last_peered":"2024-03-28T13:12:53.492468+0000","last_clean":"2024-03-28T13:12:53.492468+0000","last_became_active":"2024-03-28T13:12:52.497455+0000","last_became_peered":"2024-03-28T13:12:52.497455+0000","last_unstale":"2024-03-28T13:12:53.492468+0000","last_undegraded":"2024-03-28T13:12:53.492468+0000","last_fullsized":"2024-03-28T13:12:53.492468+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:18:33.238734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523838+0000","last_change":"2024-03-28T13:12:53.518732+0000","last_active":"2024-03-28T13:12:53.523838+0000","last_peered":"2024-03-28T13:12:53.523838+0000","last_clean":"2024-03-28T13:12:53.523838+0000","last_became_active":"2024-03-28T13:12:53.518456+0000","last_became_peered":"2024-03-28T13:12:53.518456+0000","last_unstale":"2024-03-28T13:12:53.523838+0000","last_undegraded":"2024-03-28T13:12:53.523838+0000","last_fullsized":"2024-03-28T13:12:53.523838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:48:29.629780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"62'3","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513707+0000","last_change":"2024-03-28T13:12:52.471608+0000","last_active":"2024-03-28T13:12:54.513707+0000","last_peered":"2024-03-28T13:12:54.513707+0000","last_clean":"2024-03-28T13:12:54.513707+0000","last_became_active":"2024-03-28T13:12:52.469801+0000","last_became_peered":"2024-03-28T13:12:52.469801+0000","last_unstale":"2024-03-28T13:12:54.513707+0000","last_undegraded":"2024-03-28T13:12:54.513707+0000","last_fullsized":"2024-03-28T13:12:54.513707+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:21:22.397053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513592+0000","last_change":"2024-03-28T13:12:53.533647+0000","last_active":"2024-03-28T13:12:54.513592+0000","last_peered":"2024-03-28T13:12:54.513592+0000","last_clean":"2024-03-28T13:12:54.513592+0000","last_became_active":"2024-03-28T13:12:53.533358+0000","last_became_peered":"2024-03-28T13:12:53.533358+0000","last_unstale":"2024-03-28T13:12:54.513592+0000","last_undegraded":"2024-03-28T13:12:54.513592+0000","last_fullsized":"2024-03-28T13:12:54.513592+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:31:13.215911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"62'6","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.553833+0000","last_change":"2024-03-28T13:12:52.489670+0000","last_active":"2024-03-28T13:12:54.553833+0000","last_peered":"2024-03-28T13:12:54.553833+0000","last_clean":"2024-03-28T13:12:54.553833+0000","last_became_active":"2024-03-28T13:12:52.489326+0000","last_became_peered":"2024-03-28T13:12:52.489326+0000","last_unstale":"2024-03-28T13:12:54.553833+0000","last_undegraded":"2024-03-28T13:12:54.553833+0000","last_fullsized":"2024-03-28T13:12:54.553833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:21.358714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.534598+0000","last_change":"2024-03-28T13:12:53.532827+0000","last_active":"2024-03-28T13:12:53.534598+0000","last_peered":"2024-03-28T13:12:53.534598+0000","last_clean":"2024-03-28T13:12:53.534598+0000","last_became_active":"2024-03-28T13:12:53.532369+0000","last_became_peered":"2024-03-28T13:12:53.532369+0000","last_unstale":"2024-03-28T13:12:53.534598+0000","last_undegraded":"2024-03-28T13:12:53.534598+0000","last_fullsized":"2024-03-28T13:12:53.534598+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:10:42.898595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.537856+0000","last_change":"2024-03-28T13:12:52.500541+0000","last_active":"2024-03-28T13:12:53.537856+0000","last_peered":"2024-03-28T13:12:53.537856+0000","last_clean":"2024-03-28T13:12:53.537856+0000","last_became_active":"2024-03-28T13:12:52.499747+0000","last_became_peered":"2024-03-28T13:12:52.499747+0000","last_unstale":"2024-03-28T13:12:53.537856+0000","last_undegraded":"2024-03-28T13:12:53.537856+0000","last_fullsized":"2024-03-28T13:12:53.537856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:46:17.541410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.522763+0000","last_change":"2024-03-28T13:12:53.515617+0000","last_active":"2024-03-28T13:12:54.522763+0000","last_peered":"2024-03-28T13:12:54.522763+0000","last_clean":"2024-03-28T13:12:54.522763+0000","last_became_active":"2024-03-28T13:12:53.515344+0000","last_became_peered":"2024-03-28T13:12:53.515344+0000","last_unstale":"2024-03-28T13:12:54.522763+0000","last_undegraded":"2024-03-28T13:12:54.522763+0000","last_fullsized":"2024-03-28T13:12:54.522763+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:56.402530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513712+0000","last_change":"2024-03-28T13:12:52.486702+0000","last_active":"2024-03-28T13:12:54.513712+0000","last_peered":"2024-03-28T13:12:54.513712+0000","last_clean":"2024-03-28T13:12:54.513712+0000","last_became_active":"2024-03-28T13:12:52.486386+0000","last_became_peered":"2024-03-28T13:12:52.486386+0000","last_unstale":"2024-03-28T13:12:54.513712+0000","last_undegraded":"2024-03-28T13:12:54.513712+0000","last_fullsized":"2024-03-28T13:12:54.513712+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:09.555954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.514793+0000","last_change":"2024-03-28T13:12:53.511880+0000","last_active":"2024-03-28T13:12:53.514793+0000","last_peered":"2024-03-28T13:12:53.514793+0000","last_clean":"2024-03-28T13:12:53.514793+0000","last_became_active":"2024-03-28T13:12:53.511268+0000","last_became_peered":"2024-03-28T13:12:53.511268+0000","last_unstale":"2024-03-28T13:12:53.514793+0000","last_undegraded":"2024-03-28T13:12:53.514793+0000","last_fullsized":"2024-03-28T13:12:53.514793+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:05:58.452615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513338+0000","last_change":"2024-03-28T13:12:52.486944+0000","last_active":"2024-03-28T13:12:54.513338+0000","last_peered":"2024-03-28T13:12:54.513338+0000","last_clean":"2024-03-28T13:12:54.513338+0000","last_became_active":"2024-03-28T13:12:52.485574+0000","last_became_peered":"2024-03-28T13:12:52.485574+0000","last_unstale":"2024-03-28T13:12:54.513338+0000","last_undegraded":"2024-03-28T13:12:54.513338+0000","last_fullsized":"2024-03-28T13:12:54.513338+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:50:54.412377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513359+0000","last_change":"2024-03-28T13:12:53.533711+0000","last_active":"2024-03-28T13:12:54.513359+0000","last_peered":"2024-03-28T13:12:54.513359+0000","last_clean":"2024-03-28T13:12:54.513359+0000","last_became_active":"2024-03-28T13:12:53.533128+0000","last_became_peered":"2024-03-28T13:12:53.533128+0000","last_unstale":"2024-03-28T13:12:54.513359+0000","last_undegraded":"2024-03-28T13:12:54.513359+0000","last_fullsized":"2024-03-28T13:12:54.513359+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:51:57.502055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"62'8","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.512980+0000","last_change":"2024-03-28T13:12:52.481249+0000","last_active":"2024-03-28T13:12:54.512980+0000","last_peered":"2024-03-28T13:12:54.512980+0000","last_clean":"2024-03-28T13:12:54.512980+0000","last_became_active":"2024-03-28T13:12:52.480966+0000","last_became_peered":"2024-03-28T13:12:52.480966+0000","last_unstale":"2024-03-28T13:12:54.512980+0000","last_undegraded":"2024-03-28T13:12:54.512980+0000","last_fullsized":"2024-03-28T13:12:54.512980+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:47:56.291512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523183+0000","last_change":"2024-03-28T13:12:53.516944+0000","last_active":"2024-03-28T13:12:54.523183+0000","last_peered":"2024-03-28T13:12:54.523183+0000","last_clean":"2024-03-28T13:12:54.523183+0000","last_became_active":"2024-03-28T13:12:53.516647+0000","last_became_peered":"2024-03-28T13:12:53.516647+0000","last_unstale":"2024-03-28T13:12:54.523183+0000","last_undegraded":"2024-03-28T13:12:54.523183+0000","last_fullsized":"2024-03-28T13:12:54.523183+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:02:51.661702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.531969+0000","last_change":"2024-03-28T13:12:52.499069+0000","last_active":"2024-03-28T13:12:56.531969+0000","last_peered":"2024-03-28T13:12:56.531969+0000","last_clean":"2024-03-28T13:12:56.531969+0000","last_became_active":"2024-03-28T13:12:52.498808+0000","last_became_peered":"2024-03-28T13:12:52.498808+0000","last_unstale":"2024-03-28T13:12:56.531969+0000","last_undegraded":"2024-03-28T13:12:56.531969+0000","last_fullsized":"2024-03-28T13:12:56.531969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:21:56.447486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523426+0000","last_change":"2024-03-28T13:12:53.528894+0000","last_active":"2024-03-28T13:12:54.523426+0000","last_peered":"2024-03-28T13:12:54.523426+0000","last_clean":"2024-03-28T13:12:54.523426+0000","last_became_active":"2024-03-28T13:12:53.528275+0000","last_became_peered":"2024-03-28T13:12:53.528275+0000","last_unstale":"2024-03-28T13:12:54.523426+0000","last_undegraded":"2024-03-28T13:12:54.523426+0000","last_fullsized":"2024-03-28T13:12:54.523426+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:50:31.090629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.537729+0000","last_change":"2024-03-28T13:12:52.498458+0000","last_active":"2024-03-28T13:12:53.537729+0000","last_peered":"2024-03-28T13:12:53.537729+0000","last_clean":"2024-03-28T13:12:53.537729+0000","last_became_active":"2024-03-28T13:12:52.497234+0000","last_became_peered":"2024-03-28T13:12:52.497234+0000","last_unstale":"2024-03-28T13:12:53.537729+0000","last_undegraded":"2024-03-28T13:12:53.537729+0000","last_fullsized":"2024-03-28T13:12:53.537729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:43.873994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.532733+0000","last_change":"2024-03-28T13:12:53.531284+0000","last_active":"2024-03-28T13:12:56.532733+0000","last_peered":"2024-03-28T13:12:56.532733+0000","last_clean":"2024-03-28T13:12:56.532733+0000","last_became_active":"2024-03-28T13:12:53.530318+0000","last_became_peered":"2024-03-28T13:12:53.530318+0000","last_unstale":"2024-03-28T13:12:56.532733+0000","last_undegraded":"2024-03-28T13:12:56.532733+0000","last_fullsized":"2024-03-28T13:12:56.532733+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:46:31.798711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523193+0000","last_change":"2024-03-28T13:12:52.489367+0000","last_active":"2024-03-28T13:12:54.523193+0000","last_peered":"2024-03-28T13:12:54.523193+0000","last_clean":"2024-03-28T13:12:54.523193+0000","last_became_active":"2024-03-28T13:12:52.488219+0000","last_became_peered":"2024-03-28T13:12:52.488219+0000","last_unstale":"2024-03-28T13:12:54.523193+0000","last_undegraded":"2024-03-28T13:12:54.523193+0000","last_fullsized":"2024-03-28T13:12:54.523193+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:30.940424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513591+0000","last_change":"2024-03-28T13:12:53.517878+0000","last_active":"2024-03-28T13:12:54.513591+0000","last_peered":"2024-03-28T13:12:54.513591+0000","last_clean":"2024-03-28T13:12:54.513591+0000","last_became_active":"2024-03-28T13:12:53.517601+0000","last_became_peered":"2024-03-28T13:12:53.517601+0000","last_unstale":"2024-03-28T13:12:54.513591+0000","last_undegraded":"2024-03-28T13:12:54.513591+0000","last_fullsized":"2024-03-28T13:12:54.513591+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:07:06.162755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.494129+0000","last_change":"2024-03-28T13:12:52.492132+0000","last_active":"2024-03-28T13:12:53.494129+0000","last_peered":"2024-03-28T13:12:53.494129+0000","last_clean":"2024-03-28T13:12:53.494129+0000","last_became_active":"2024-03-28T13:12:52.491829+0000","last_became_peered":"2024-03-28T13:12:52.491829+0000","last_unstale":"2024-03-28T13:12:53.494129+0000","last_undegraded":"2024-03-28T13:12:53.494129+0000","last_fullsized":"2024-03-28T13:12:53.494129+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:55.388119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523687+0000","last_change":"2024-03-28T13:12:53.519732+0000","last_active":"2024-03-28T13:12:53.523687+0000","last_peered":"2024-03-28T13:12:53.523687+0000","last_clean":"2024-03-28T13:12:53.523687+0000","last_became_active":"2024-03-28T13:12:53.519417+0000","last_became_peered":"2024-03-28T13:12:53.519417+0000","last_unstale":"2024-03-28T13:12:53.523687+0000","last_undegraded":"2024-03-28T13:12:53.523687+0000","last_fullsized":"2024-03-28T13:12:53.523687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:40:22.156766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.539505+0000","last_change":"2024-03-28T13:12:52.497898+0000","last_active":"2024-03-28T13:12:53.539505+0000","last_peered":"2024-03-28T13:12:53.539505+0000","last_clean":"2024-03-28T13:12:53.539505+0000","last_became_active":"2024-03-28T13:12:52.497054+0000","last_became_peered":"2024-03-28T13:12:52.497054+0000","last_unstale":"2024-03-28T13:12:53.539505+0000","last_undegraded":"2024-03-28T13:12:53.539505+0000","last_fullsized":"2024-03-28T13:12:53.539505+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:42.480048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.521670+0000","last_change":"2024-03-28T13:12:53.512993+0000","last_active":"2024-03-28T13:12:53.521670+0000","last_peered":"2024-03-28T13:12:53.521670+0000","last_clean":"2024-03-28T13:12:53.521670+0000","last_became_active":"2024-03-28T13:12:53.512788+0000","last_became_peered":"2024-03-28T13:12:53.512788+0000","last_unstale":"2024-03-28T13:12:53.521670+0000","last_undegraded":"2024-03-28T13:12:53.521670+0000","last_fullsized":"2024-03-28T13:12:53.521670+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:27:53.920916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.491141+0000","last_change":"2024-03-28T13:12:52.500236+0000","last_active":"2024-03-28T13:12:53.491141+0000","last_peered":"2024-03-28T13:12:53.491141+0000","last_clean":"2024-03-28T13:12:53.491141+0000","last_became_active":"2024-03-28T13:12:52.499795+0000","last_became_peered":"2024-03-28T13:12:52.499795+0000","last_unstale":"2024-03-28T13:12:53.491141+0000","last_undegraded":"2024-03-28T13:12:53.491141+0000","last_fullsized":"2024-03-28T13:12:53.491141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:16:15.524903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523835+0000","last_change":"2024-03-28T13:12:53.515564+0000","last_active":"2024-03-28T13:12:53.523835+0000","last_peered":"2024-03-28T13:12:53.523835+0000","last_clean":"2024-03-28T13:12:53.523835+0000","last_became_active":"2024-03-28T13:12:53.514826+0000","last_became_peered":"2024-03-28T13:12:53.514826+0000","last_unstale":"2024-03-28T13:12:53.523835+0000","last_undegraded":"2024-03-28T13:12:53.523835+0000","last_fullsized":"2024-03-28T13:12:53.523835+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:20:59.923272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.489947+0000","last_change":"2024-03-28T13:12:52.492600+0000","last_active":"2024-03-28T13:12:53.489947+0000","last_peered":"2024-03-28T13:12:53.489947+0000","last_clean":"2024-03-28T13:12:53.489947+0000","last_became_active":"2024-03-28T13:12:52.492285+0000","last_became_peered":"2024-03-28T13:12:52.492285+0000","last_unstale":"2024-03-28T13:12:53.489947+0000","last_undegraded":"2024-03-28T13:12:53.489947+0000","last_fullsized":"2024-03-28T13:12:53.489947+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:39:47.453340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523338+0000","last_change":"2024-03-28T13:12:53.524081+0000","last_active":"2024-03-28T13:12:54.523338+0000","last_peered":"2024-03-28T13:12:54.523338+0000","last_clean":"2024-03-28T13:12:54.523338+0000","last_became_active":"2024-03-28T13:12:53.523708+0000","last_became_peered":"2024-03-28T13:12:53.523708+0000","last_unstale":"2024-03-28T13:12:54.523338+0000","last_undegraded":"2024-03-28T13:12:54.523338+0000","last_fullsized":"2024-03-28T13:12:54.523338+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:20:59.923272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"62'2","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.532229+0000","last_change":"2024-03-28T13:12:52.492725+0000","last_active":"2024-03-28T13:12:56.532229+0000","last_peered":"2024-03-28T13:12:56.532229+0000","last_clean":"2024-03-28T13:12:56.532229+0000","last_became_active":"2024-03-28T13:12:52.491883+0000","last_became_peered":"2024-03-28T13:12:52.491883+0000","last_unstale":"2024-03-28T13:12:56.532229+0000","last_undegraded":"2024-03-28T13:12:56.532229+0000","last_fullsized":"2024-03-28T13:12:56.532229+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:58:32.915798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523342+0000","last_change":"2024-03-28T13:12:53.520223+0000","last_active":"2024-03-28T13:12:54.523342+0000","last_peered":"2024-03-28T13:12:54.523342+0000","last_clean":"2024-03-28T13:12:54.523342+0000","last_became_active":"2024-03-28T13:12:53.513629+0000","last_became_peered":"2024-03-28T13:12:53.513629+0000","last_unstale":"2024-03-28T13:12:54.523342+0000","last_undegraded":"2024-03-28T13:12:54.523342+0000","last_fullsized":"2024-03-28T13:12:54.523342+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:49:35.695939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.509738+0000","last_change":"2024-03-28T13:12:52.495889+0000","last_active":"2024-03-28T13:12:53.509738+0000","last_peered":"2024-03-28T13:12:53.509738+0000","last_clean":"2024-03-28T13:12:53.509738+0000","last_became_active":"2024-03-28T13:12:52.495589+0000","last_became_peered":"2024-03-28T13:12:52.495589+0000","last_unstale":"2024-03-28T13:12:53.509738+0000","last_undegraded":"2024-03-28T13:12:53.509738+0000","last_fullsized":"2024-03-28T13:12:53.509738+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:21.358714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.532854+0000","last_change":"2024-03-28T13:12:53.534280+0000","last_active":"2024-03-28T13:12:56.532854+0000","last_peered":"2024-03-28T13:12:56.532854+0000","last_clean":"2024-03-28T13:12:56.532854+0000","last_became_active":"2024-03-28T13:12:53.534031+0000","last_became_peered":"2024-03-28T13:12:53.534031+0000","last_unstale":"2024-03-28T13:12:56.532854+0000","last_undegraded":"2024-03-28T13:12:56.532854+0000","last_fullsized":"2024-03-28T13:12:56.532854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:28:58.765737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.488213+0000","last_change":"2024-03-28T13:12:52.491050+0000","last_active":"2024-03-28T13:12:53.488213+0000","last_peered":"2024-03-28T13:12:53.488213+0000","last_clean":"2024-03-28T13:12:53.488213+0000","last_became_active":"2024-03-28T13:12:52.490722+0000","last_became_peered":"2024-03-28T13:12:52.490722+0000","last_unstale":"2024-03-28T13:12:53.488213+0000","last_undegraded":"2024-03-28T13:12:53.488213+0000","last_fullsized":"2024-03-28T13:12:53.488213+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:12:48.954255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523979+0000","last_change":"2024-03-28T13:12:53.515816+0000","last_active":"2024-03-28T13:12:53.523979+0000","last_peered":"2024-03-28T13:12:53.523979+0000","last_clean":"2024-03-28T13:12:53.523979+0000","last_became_active":"2024-03-28T13:12:53.515487+0000","last_became_peered":"2024-03-28T13:12:53.515487+0000","last_unstale":"2024-03-28T13:12:53.523979+0000","last_undegraded":"2024-03-28T13:12:53.523979+0000","last_fullsized":"2024-03-28T13:12:53.523979+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:49:35.695939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.522863+0000","last_change":"2024-03-28T13:12:52.481784+0000","last_active":"2024-03-28T13:12:54.522863+0000","last_peered":"2024-03-28T13:12:54.522863+0000","last_clean":"2024-03-28T13:12:54.522863+0000","last_became_active":"2024-03-28T13:12:52.481499+0000","last_became_peered":"2024-03-28T13:12:52.481499+0000","last_unstale":"2024-03-28T13:12:54.522863+0000","last_undegraded":"2024-03-28T13:12:54.522863+0000","last_fullsized":"2024-03-28T13:12:54.522863+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:36:31.168233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.525144+0000","last_change":"2024-03-28T13:12:53.517879+0000","last_active":"2024-03-28T13:12:53.525144+0000","last_peered":"2024-03-28T13:12:53.525144+0000","last_clean":"2024-03-28T13:12:53.525144+0000","last_became_active":"2024-03-28T13:12:53.517550+0000","last_became_peered":"2024-03-28T13:12:53.517550+0000","last_unstale":"2024-03-28T13:12:53.525144+0000","last_undegraded":"2024-03-28T13:12:53.525144+0000","last_fullsized":"2024-03-28T13:12:53.525144+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:51:13.898620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.531064+0000","last_change":"2024-03-28T13:12:52.499023+0000","last_active":"2024-03-28T13:12:56.531064+0000","last_peered":"2024-03-28T13:12:56.531064+0000","last_clean":"2024-03-28T13:12:56.531064+0000","last_became_active":"2024-03-28T13:12:52.497835+0000","last_became_peered":"2024-03-28T13:12:52.497835+0000","last_unstale":"2024-03-28T13:12:56.531064+0000","last_undegraded":"2024-03-28T13:12:56.531064+0000","last_fullsized":"2024-03-28T13:12:56.531064+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:42.508336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.553970+0000","last_change":"2024-03-28T13:12:53.525102+0000","last_active":"2024-03-28T13:12:54.553970+0000","last_peered":"2024-03-28T13:12:54.553970+0000","last_clean":"2024-03-28T13:12:54.553970+0000","last_became_active":"2024-03-28T13:12:53.524410+0000","last_became_peered":"2024-03-28T13:12:53.524410+0000","last_unstale":"2024-03-28T13:12:54.553970+0000","last_undegraded":"2024-03-28T13:12:54.553970+0000","last_fullsized":"2024-03-28T13:12:54.553970+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:56.402530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.522654+0000","last_change":"2024-03-28T13:12:52.492624+0000","last_active":"2024-03-28T13:12:54.522654+0000","last_peered":"2024-03-28T13:12:54.522654+0000","last_clean":"2024-03-28T13:12:54.522654+0000","last_became_active":"2024-03-28T13:12:52.492151+0000","last_became_peered":"2024-03-28T13:12:52.492151+0000","last_unstale":"2024-03-28T13:12:54.522654+0000","last_undegraded":"2024-03-28T13:12:54.522654+0000","last_fullsized":"2024-03-28T13:12:54.522654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:21.414302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.521781+0000","last_change":"2024-03-28T13:12:53.511121+0000","last_active":"2024-03-28T13:12:53.521781+0000","last_peered":"2024-03-28T13:12:53.521781+0000","last_clean":"2024-03-28T13:12:53.521781+0000","last_became_active":"2024-03-28T13:12:53.510092+0000","last_became_peered":"2024-03-28T13:12:53.510092+0000","last_unstale":"2024-03-28T13:12:53.521781+0000","last_undegraded":"2024-03-28T13:12:53.521781+0000","last_fullsized":"2024-03-28T13:12:53.521781+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:02:51.661702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.509614+0000","last_change":"2024-03-28T13:12:52.495791+0000","last_active":"2024-03-28T13:12:53.509614+0000","last_peered":"2024-03-28T13:12:53.509614+0000","last_clean":"2024-03-28T13:12:53.509614+0000","last_became_active":"2024-03-28T13:12:52.495465+0000","last_became_peered":"2024-03-28T13:12:52.495465+0000","last_unstale":"2024-03-28T13:12:53.509614+0000","last_undegraded":"2024-03-28T13:12:53.509614+0000","last_fullsized":"2024-03-28T13:12:53.509614+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:21.414302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.530881+0000","last_change":"2024-03-28T13:12:53.532522+0000","last_active":"2024-03-28T13:12:56.530881+0000","last_peered":"2024-03-28T13:12:56.530881+0000","last_clean":"2024-03-28T13:12:56.530881+0000","last_became_active":"2024-03-28T13:12:53.532132+0000","last_became_peered":"2024-03-28T13:12:53.532132+0000","last_unstale":"2024-03-28T13:12:56.530881+0000","last_undegraded":"2024-03-28T13:12:56.530881+0000","last_fullsized":"2024-03-28T13:12:56.530881+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:25:06.549357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.492125+0000","last_change":"2024-03-28T13:12:52.487641+0000","last_active":"2024-03-28T13:12:53.492125+0000","last_peered":"2024-03-28T13:12:53.492125+0000","last_clean":"2024-03-28T13:12:53.492125+0000","last_became_active":"2024-03-28T13:12:52.487320+0000","last_became_peered":"2024-03-28T13:12:52.487320+0000","last_unstale":"2024-03-28T13:12:53.492125+0000","last_undegraded":"2024-03-28T13:12:53.492125+0000","last_fullsized":"2024-03-28T13:12:53.492125+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:26:04.005993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523841+0000","last_change":"2024-03-28T13:12:53.520321+0000","last_active":"2024-03-28T13:12:53.523841+0000","last_peered":"2024-03-28T13:12:53.523841+0000","last_clean":"2024-03-28T13:12:53.523841+0000","last_became_active":"2024-03-28T13:12:53.520018+0000","last_became_peered":"2024-03-28T13:12:53.520018+0000","last_unstale":"2024-03-28T13:12:53.523841+0000","last_undegraded":"2024-03-28T13:12:53.523841+0000","last_fullsized":"2024-03-28T13:12:53.523841+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:28:01.110960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.553606+0000","last_change":"2024-03-28T13:12:52.484733+0000","last_active":"2024-03-28T13:12:54.553606+0000","last_peered":"2024-03-28T13:12:54.553606+0000","last_clean":"2024-03-28T13:12:54.553606+0000","last_became_active":"2024-03-28T13:12:52.483907+0000","last_became_peered":"2024-03-28T13:12:52.483907+0000","last_unstale":"2024-03-28T13:12:54.553606+0000","last_undegraded":"2024-03-28T13:12:54.553606+0000","last_fullsized":"2024-03-28T13:12:54.553606+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:20:56.752156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":241,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.488175+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:12:53.488175+0000","last_peered":"2024-03-28T13:12:53.488175+0000","last_clean":"2024-03-28T13:12:53.488175+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:12:53.488175+0000","last_undegraded":"2024-03-28T13:12:53.488175+0000","last_fullsized":"2024-03-28T13:12:53.488175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476783,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":737280,"data_stored":305832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476781,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":753664,"data_stored":307514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476781,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1331200,"data_stored":896692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476781,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":745472,"data_stored":307354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":737280,"data_stored":305764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":889335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":720896,"data_stored":298967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":753664,"data_stored":312134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:12:58.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:12:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.377+0000 7f2d48a00640 1 Processor -- start 2024-03-28T13:12:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.377+0000 7f2d48a00640 1 -- start start 2024-03-28T13:12:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.378+0000 7f2d48a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d4415da60 0x7f2d440cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.378+0000 7f2d48a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2d4415fe90 0x7f2d440d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.378+0000 7f2d48a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d4415d060 con 0x7f2d4415f900 2024-03-28T13:12:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.378+0000 7f2d48a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f2d4415d340 con 0x7f2d4415fe90 2024-03-28T13:12:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.378+0000 7f2d48a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2d4415d620 con 0x7f2d4415da60 2024-03-28T13:12:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.379+0000 7f2d41e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2d4415f900 0x7f2d4415c850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52182/0 (socket says 172.21.15.19:52182) 2024-03-28T13:12:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.379+0000 7f2d41e00640 1 -- 172.21.15.19:0/58153236 learned_addr learned my addr 172.21.15.19:0/58153236 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:12:58.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.379+0000 7f2d40a00640 1 -- 172.21.15.19:0/58153236 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3588958314 0 0) 0x7f2d4415d060 con 0x7f2d4415f900 2024-03-28T13:12:58.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.379+0000 7f2d40a00640 1 -- 172.21.15.19:0/58153236 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d28003520 con 0x7f2d4415f900 2024-03-28T13:12:58.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.379+0000 7f2d42800640 1 --2- 172.21.15.19:0/58153236 >> v2:172.21.15.188:3300/0 conn(0x7f2d4415fe90 0x7f2d440d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:58.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.379+0000 7f2d41400640 1 --2- 172.21.15.19:0/58153236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d4415da60 0x7f2d440cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:58.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.380+0000 7f2d40a00640 1 -- 172.21.15.19:0/58153236 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1930559632 0 0) 0x7f2d28003520 con 0x7f2d4415f900 2024-03-28T13:12:58.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.380+0000 7f2d40a00640 1 -- 172.21.15.19:0/58153236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d4415da60 msgr2=0x7f2d440cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:58.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.380+0000 7f2d40a00640 1 --2- 172.21.15.19:0/58153236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d4415da60 0x7f2d440cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.380+0000 7f2d40a00640 1 -- 172.21.15.19:0/58153236 >> v2:172.21.15.188:3300/0 conn(0x7f2d4415fe90 msgr2=0x7f2d440d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:58.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.380+0000 7f2d40a00640 1 --2- 172.21.15.19:0/58153236 >> v2:172.21.15.188:3300/0 conn(0x7f2d4415fe90 0x7f2d440d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.380+0000 7f2d40a00640 1 -- 172.21.15.19:0/58153236 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d440c74b0 con 0x7f2d4415f900 2024-03-28T13:12:58.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.380+0000 7f2d40a00640 1 -- 172.21.15.19:0/58153236 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f2d34001670 con 0x7f2d4415f900 2024-03-28T13:12:58.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.380+0000 7f2d40a00640 1 -- 172.21.15.19:0/58153236 >> v1:172.21.15.19:6789/0 conn(0x7f2d4415f900 legacy=0x7f2d4415c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:58.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.380+0000 7f2d40a00640 1 --2- 172.21.15.19:0/58153236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d280039b0 0x7f2d28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:58.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.380+0000 7f2d40a00640 1 --2- 172.21.15.19:0/58153236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d28004800 0x7f2d28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:58.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.380+0000 7f2d40a00640 1 --2- 172.21.15.19:0/58153236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d4415da60 0x7f2d280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:58.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.380+0000 7f2d40a00640 1 -- 172.21.15.19:0/58153236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4415d340 con 0x7f2d280039b0 2024-03-28T13:12:58.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.380+0000 7f2d40a00640 1 -- 172.21.15.19:0/58153236 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4415d620 con 0x7f2d28004800 2024-03-28T13:12:58.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.380+0000 7f2d40a00640 1 -- 172.21.15.19:0/58153236 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2800ba10 con 0x7f2d4415da60 2024-03-28T13:12:58.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.380+0000 7f2d41e00640 1 --2- 172.21.15.19:0/58153236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d280039b0 0x7f2d28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:58.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.381+0000 7f2d42800640 1 --2- 172.21.15.19:0/58153236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d4415da60 0x7f2d280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:58.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.381+0000 7f2d41400640 1 --2- 172.21.15.19:0/58153236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d28004800 0x7f2d28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:58.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.381+0000 7f2d41e00640 1 -- 172.21.15.19:0/58153236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d4415da60 msgr2=0x7f2d280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:58.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.381+0000 7f2d41e00640 1 --2- 172.21.15.19:0/58153236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d4415da60 0x7f2d280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.381+0000 7f2d41e00640 1 -- 172.21.15.19:0/58153236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d28004800 msgr2=0x7f2d28004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:58.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.381+0000 7f2d41e00640 1 --2- 172.21.15.19:0/58153236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d28004800 0x7f2d28004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.381+0000 7f2d41e00640 1 -- 172.21.15.19:0/58153236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d28003520 con 0x7f2d280039b0 2024-03-28T13:12:58.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.381+0000 7f2d42800640 1 --2- 172.21.15.19:0/58153236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d4415da60 0x7f2d280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:58.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.381+0000 7f2d41400640 1 --2- 172.21.15.19:0/58153236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d28004800 0x7f2d28004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:12:58.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.382+0000 7f2d41e00640 1 --2- 172.21.15.19:0/58153236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d280039b0 0x7f2d28003dd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400c190 tx=0x7f2d28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4338ece58fe6dc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:58.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.382+0000 7f2d40a00640 1 -- 172.21.15.19:0/58153236 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34007c80 con 0x7f2d280039b0 2024-03-28T13:12:58.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.382+0000 7f2d40a00640 1 -- 172.21.15.19:0/58153236 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d34030070 con 0x7f2d280039b0 2024-03-28T13:12:58.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.382+0000 7f2d40a00640 1 -- 172.21.15.19:0/58153236 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34005d30 con 0x7f2d280039b0 2024-03-28T13:12:58.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.383+0000 7f2d48a00640 1 -- 172.21.15.19:0/58153236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d280039b0 msgr2=0x7f2d28003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:58.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.383+0000 7f2d48a00640 1 --2- 172.21.15.19:0/58153236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d280039b0 0x7f2d28003dd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400c190 tx=0x7f2d28005830 comp rx=0 tx=0).stop 2024-03-28T13:12:58.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.383+0000 7f2d42800640 1 -- 172.21.15.19:0/58153236 reap_dead start 2024-03-28T13:12:58.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.383+0000 7f2d48a00640 1 -- 172.21.15.19:0/58153236 shutdown_connections 2024-03-28T13:12:58.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.383+0000 7f2d48a00640 1 -- 172.21.15.19:0/58153236 >> 172.21.15.19:0/58153236 conn(0x7f2d440b3e90 msgr2=0x7f2d440b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:58.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.383+0000 7f2d48a00640 1 -- 172.21.15.19:0/58153236 shutdown_connections 2024-03-28T13:12:58.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.383+0000 7f2d48a00640 1 -- 172.21.15.19:0/58153236 wait complete. 2024-03-28T13:12:58.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.385+0000 7f2d48a00640 1 Processor -- start 2024-03-28T13:12:58.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.385+0000 7f2d48a00640 1 -- start start 2024-03-28T13:12:58.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.386+0000 7f2d48a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d280039b0 0x7f2d44153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:58.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.386+0000 7f2d48a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d440b3320 0x7f2d440b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:58.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.386+0000 7f2d48a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d440b6ab0 0x7f2d440b6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:58.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.386+0000 7f2d48a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2d440d01c0 con 0x7f2d440b6ab0 2024-03-28T13:12:58.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.386+0000 7f2d48a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2d440d0040 con 0x7f2d440b3320 2024-03-28T13:12:58.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.386+0000 7f2d48a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2d440ce8d0 con 0x7f2d280039b0 2024-03-28T13:12:58.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.387+0000 7f2d42800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d440b6ab0 0x7f2d440b6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:58.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.387+0000 7f2d42800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d440b6ab0 0x7f2d440b6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33846/0 (socket says 172.21.15.19:33846) 2024-03-28T13:12:58.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.387+0000 7f2d42800640 1 -- 172.21.15.19:0/3918649639 learned_addr learned my addr 172.21.15.19:0/3918649639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:58.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.387+0000 7f2d41e00640 1 --2- 172.21.15.19:0/3918649639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d280039b0 0x7f2d44153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:58.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.387+0000 7f2d41400640 1 --2- 172.21.15.19:0/3918649639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d440b3320 0x7f2d440b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:58.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.387+0000 7f2d42800640 1 -- 172.21.15.19:0/3918649639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d280039b0 msgr2=0x7f2d44153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:58.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.387+0000 7f2d42800640 1 --2- 172.21.15.19:0/3918649639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d280039b0 0x7f2d44153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.387+0000 7f2d42800640 1 -- 172.21.15.19:0/3918649639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d440b3320 msgr2=0x7f2d440b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:58.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.387+0000 7f2d42800640 1 --2- 172.21.15.19:0/3918649639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d440b3320 0x7f2d440b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.387+0000 7f2d42800640 1 -- 172.21.15.19:0/3918649639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d440b7470 con 0x7f2d440b6ab0 2024-03-28T13:12:58.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.387+0000 7f2d41e00640 1 --2- 172.21.15.19:0/3918649639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d280039b0 0x7f2d44153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:12:58.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.387+0000 7f2d42800640 1 --2- 172.21.15.19:0/3918649639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d440b6ab0 0x7f2d440b6f30 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2d4406f3c0 tx=0x7f2d3c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:58.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.388+0000 7f2d3a000640 1 -- 172.21.15.19:0/3918649639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c016620 con 0x7f2d440b6ab0 2024-03-28T13:12:58.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.388+0000 7f2d3a000640 1 -- 172.21.15.19:0/3918649639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d3c01b070 con 0x7f2d440b6ab0 2024-03-28T13:12:58.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.388+0000 7f2d3a000640 1 -- 172.21.15.19:0/3918649639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c0169e0 con 0x7f2d440b6ab0 2024-03-28T13:12:58.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.388+0000 7f2d48a00640 1 -- 172.21.15.19:0/3918649639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d440c94d0 con 0x7f2d440b6ab0 2024-03-28T13:12:58.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.388+0000 7f2d48a00640 1 -- 172.21.15.19:0/3918649639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d440c9a90 con 0x7f2d440b6ab0 2024-03-28T13:12:58.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.389+0000 7f2d48a00640 1 -- 172.21.15.19:0/3918649639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d44068cd0 con 0x7f2d440b6ab0 2024-03-28T13:12:58.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.390+0000 7f2d3a000640 1 -- 172.21.15.19:0/3918649639 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2d3c005d10 con 0x7f2d440b6ab0 2024-03-28T13:12:58.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.391+0000 7f2d3a000640 1 --2- 172.21.15.19:0/3918649639 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2d2406c560 0x7f2d2406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:58.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.391+0000 7f2d3a000640 1 -- 172.21.15.19:0/3918649639 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f2d3c08bf50 con 0x7f2d440b6ab0 2024-03-28T13:12:58.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.392+0000 7f2d41e00640 1 --2- 172.21.15.19:0/3918649639 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2d2406c560 0x7f2d2406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:58.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.393+0000 7f2d41e00640 1 --2- 172.21.15.19:0/3918649639 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2d2406c560 0x7f2d2406ea20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2d34034000 tx=0x7f2d34007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:58.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.395+0000 7f2d3a000640 1 -- 172.21.15.19:0/3918649639 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2d3c05a570 con 0x7f2d440b6ab0 2024-03-28T13:12:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.596+0000 7f2d48a00640 1 -- 172.21.15.19:0/3918649639 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d44158c50 con 0x7f2d2406c560 2024-03-28T13:12:58.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.616+0000 7f2d3a000640 1 -- 172.21.15.19:0/3918649639 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251923 (secure 0 0 0) 0x7f2d34082830 con 0x7f2d2406c560 2024-03-28T13:12:58.620 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:58.622 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:12:58.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.622+0000 7f2d48a00640 1 -- 172.21.15.19:0/3918649639 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2d2406c560 msgr2=0x7f2d2406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:58.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.622+0000 7f2d48a00640 1 --2- 172.21.15.19:0/3918649639 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2d2406c560 0x7f2d2406ea20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2d34034000 tx=0x7f2d34007000 comp rx=0 tx=0).stop 2024-03-28T13:12:58.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.623+0000 7f2d48a00640 1 -- 172.21.15.19:0/3918649639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d440b6ab0 msgr2=0x7f2d440b6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:58.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.623+0000 7f2d48a00640 1 --2- 172.21.15.19:0/3918649639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d440b6ab0 0x7f2d440b6f30 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2d4406f3c0 tx=0x7f2d3c00f520 comp rx=0 tx=0).stop 2024-03-28T13:12:58.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.623+0000 7f2d48a00640 1 -- 172.21.15.19:0/3918649639 shutdown_connections 2024-03-28T13:12:58.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.623+0000 7f2d48a00640 1 --2- 172.21.15.19:0/3918649639 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2d2406c560 0x7f2d2406ea20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.623+0000 7f2d48a00640 1 --2- 172.21.15.19:0/3918649639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d440b6ab0 0x7f2d440b6f30 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.623+0000 7f2d48a00640 1 --2- 172.21.15.19:0/3918649639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d440b3320 0x7f2d440b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.623+0000 7f2d48a00640 1 --2- 172.21.15.19:0/3918649639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d280039b0 0x7f2d44153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.623+0000 7f2d48a00640 1 -- 172.21.15.19:0/3918649639 >> 172.21.15.19:0/3918649639 conn(0x7f2d440b3e90 msgr2=0x7f2d440d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:58.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.623+0000 7f2d48a00640 1 -- 172.21.15.19:0/3918649639 shutdown_connections 2024-03-28T13:12:58.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.623+0000 7f2d48a00640 1 -- 172.21.15.19:0/3918649639 wait complete. 2024-03-28T13:12:58.641 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-03-28T13:12:58.054165+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243916,"kb_used_data":6928,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730804,"statfs":{"total":773094113280,"available":772844343296,"internally_reserved":0,"allocated":7094272,"data_stored":3623592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":242535988},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001998"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.528724+0000","last_change":"2024-03-28T13:12:53.525879+0000","last_active":"2024-03-28T13:12:53.528724+0000","last_peered":"2024-03-28T13:12:53.528724+0000","last_clean":"2024-03-28T13:12:53.528724+0000","last_became_active":"2024-03-28T13:12:53.525279+0000","last_became_peered":"2024-03-28T13:12:53.525279+0000","last_unstale":"2024-03-28T13:12:53.528724+0000","last_undegraded":"2024-03-28T13:12:53.528724+0000","last_fullsized":"2024-03-28T13:12:53.528724+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:46:18.555821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.532157+0000","last_change":"2024-03-28T13:12:53.530114+0000","last_active":"2024-03-28T13:12:56.532157+0000","last_peered":"2024-03-28T13:12:56.532157+0000","last_clean":"2024-03-28T13:12:56.532157+0000","last_became_active":"2024-03-28T13:12:53.529820+0000","last_became_peered":"2024-03-28T13:12:53.529820+0000","last_unstale":"2024-03-28T13:12:56.532157+0000","last_undegraded":"2024-03-28T13:12:56.532157+0000","last_fullsized":"2024-03-28T13:12:56.532157+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:55:07.763584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523109+0000","last_change":"2024-03-28T13:12:53.517605+0000","last_active":"2024-03-28T13:12:54.523109+0000","last_peered":"2024-03-28T13:12:54.523109+0000","last_clean":"2024-03-28T13:12:54.523109+0000","last_became_active":"2024-03-28T13:12:53.517285+0000","last_became_peered":"2024-03-28T13:12:53.517285+0000","last_unstale":"2024-03-28T13:12:54.523109+0000","last_undegraded":"2024-03-28T13:12:54.523109+0000","last_fullsized":"2024-03-28T13:12:54.523109+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:27:53.920916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513660+0000","last_change":"2024-03-28T13:12:53.519742+0000","last_active":"2024-03-28T13:12:54.513660+0000","last_peered":"2024-03-28T13:12:54.513660+0000","last_clean":"2024-03-28T13:12:54.513660+0000","last_became_active":"2024-03-28T13:12:53.519465+0000","last_became_peered":"2024-03-28T13:12:53.519465+0000","last_unstale":"2024-03-28T13:12:54.513660+0000","last_undegraded":"2024-03-28T13:12:54.513660+0000","last_fullsized":"2024-03-28T13:12:54.513660+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:29:47.754192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.526802+0000","last_change":"2024-03-28T13:12:53.520736+0000","last_active":"2024-03-28T13:12:53.526802+0000","last_peered":"2024-03-28T13:12:53.526802+0000","last_clean":"2024-03-28T13:12:53.526802+0000","last_became_active":"2024-03-28T13:12:53.520435+0000","last_became_peered":"2024-03-28T13:12:53.520435+0000","last_unstale":"2024-03-28T13:12:53.526802+0000","last_undegraded":"2024-03-28T13:12:53.526802+0000","last_fullsized":"2024-03-28T13:12:53.526802+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:54:28.232197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.525167+0000","last_change":"2024-03-28T13:12:53.518578+0000","last_active":"2024-03-28T13:12:53.525167+0000","last_peered":"2024-03-28T13:12:53.525167+0000","last_clean":"2024-03-28T13:12:53.525167+0000","last_became_active":"2024-03-28T13:12:53.518270+0000","last_became_peered":"2024-03-28T13:12:53.518270+0000","last_unstale":"2024-03-28T13:12:53.525167+0000","last_undegraded":"2024-03-28T13:12:53.525167+0000","last_fullsized":"2024-03-28T13:12:53.525167+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:25:35.524688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513081+0000","last_change":"2024-03-28T13:12:53.514524+0000","last_active":"2024-03-28T13:12:54.513081+0000","last_peered":"2024-03-28T13:12:54.513081+0000","last_clean":"2024-03-28T13:12:54.513081+0000","last_became_active":"2024-03-28T13:12:53.514203+0000","last_became_peered":"2024-03-28T13:12:53.514203+0000","last_unstale":"2024-03-28T13:12:54.513081+0000","last_undegraded":"2024-03-28T13:12:54.513081+0000","last_fullsized":"2024-03-28T13:12:54.513081+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:16:40.025156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.516516+0000","last_change":"2024-03-28T13:12:53.501343+0000","last_active":"2024-03-28T13:12:53.516516+0000","last_peered":"2024-03-28T13:12:53.516516+0000","last_clean":"2024-03-28T13:12:53.516516+0000","last_became_active":"2024-03-28T13:12:53.500816+0000","last_became_peered":"2024-03-28T13:12:53.500816+0000","last_unstale":"2024-03-28T13:12:53.516516+0000","last_undegraded":"2024-03-28T13:12:53.516516+0000","last_fullsized":"2024-03-28T13:12:53.516516+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:41:30.265558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.529988+0000","last_change":"2024-03-28T13:12:53.525511+0000","last_active":"2024-03-28T13:12:53.529988+0000","last_peered":"2024-03-28T13:12:53.529988+0000","last_clean":"2024-03-28T13:12:53.529988+0000","last_became_active":"2024-03-28T13:12:53.525233+0000","last_became_peered":"2024-03-28T13:12:53.525233+0000","last_unstale":"2024-03-28T13:12:53.529988+0000","last_undegraded":"2024-03-28T13:12:53.529988+0000","last_fullsized":"2024-03-28T13:12:53.529988+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:51:17.865300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.526884+0000","last_change":"2024-03-28T13:12:53.521826+0000","last_active":"2024-03-28T13:12:53.526884+0000","last_peered":"2024-03-28T13:12:53.526884+0000","last_clean":"2024-03-28T13:12:53.526884+0000","last_became_active":"2024-03-28T13:12:53.521479+0000","last_became_peered":"2024-03-28T13:12:53.521479+0000","last_unstale":"2024-03-28T13:12:53.526884+0000","last_undegraded":"2024-03-28T13:12:53.526884+0000","last_fullsized":"2024-03-28T13:12:53.526884+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:16:16.539313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.532901+0000","last_change":"2024-03-28T13:12:53.526182+0000","last_active":"2024-03-28T13:12:53.532901+0000","last_peered":"2024-03-28T13:12:53.532901+0000","last_clean":"2024-03-28T13:12:53.532901+0000","last_became_active":"2024-03-28T13:12:53.525868+0000","last_became_peered":"2024-03-28T13:12:53.525868+0000","last_unstale":"2024-03-28T13:12:53.532901+0000","last_undegraded":"2024-03-28T13:12:53.532901+0000","last_fullsized":"2024-03-28T13:12:53.532901+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:16:16.539313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523498+0000","last_change":"2024-03-28T13:12:53.519176+0000","last_active":"2024-03-28T13:12:53.523498+0000","last_peered":"2024-03-28T13:12:53.523498+0000","last_clean":"2024-03-28T13:12:53.523498+0000","last_became_active":"2024-03-28T13:12:53.518852+0000","last_became_peered":"2024-03-28T13:12:53.518852+0000","last_unstale":"2024-03-28T13:12:53.523498+0000","last_undegraded":"2024-03-28T13:12:53.523498+0000","last_fullsized":"2024-03-28T13:12:53.523498+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:34:20.994120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.553831+0000","last_change":"2024-03-28T13:12:53.525369+0000","last_active":"2024-03-28T13:12:54.553831+0000","last_peered":"2024-03-28T13:12:54.553831+0000","last_clean":"2024-03-28T13:12:54.553831+0000","last_became_active":"2024-03-28T13:12:53.524809+0000","last_became_peered":"2024-03-28T13:12:53.524809+0000","last_unstale":"2024-03-28T13:12:54.553831+0000","last_undegraded":"2024-03-28T13:12:54.553831+0000","last_fullsized":"2024-03-28T13:12:54.553831+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:44.888405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.522846+0000","last_change":"2024-03-28T13:12:53.527839+0000","last_active":"2024-03-28T13:12:54.522846+0000","last_peered":"2024-03-28T13:12:54.522846+0000","last_clean":"2024-03-28T13:12:54.522846+0000","last_became_active":"2024-03-28T13:12:53.527486+0000","last_became_peered":"2024-03-28T13:12:53.527486+0000","last_unstale":"2024-03-28T13:12:54.522846+0000","last_undegraded":"2024-03-28T13:12:54.522846+0000","last_fullsized":"2024-03-28T13:12:54.522846+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:41:30.265558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.553642+0000","last_change":"2024-03-28T13:12:53.530948+0000","last_active":"2024-03-28T13:12:54.553642+0000","last_peered":"2024-03-28T13:12:54.553642+0000","last_clean":"2024-03-28T13:12:54.553642+0000","last_became_active":"2024-03-28T13:12:53.530682+0000","last_became_peered":"2024-03-28T13:12:53.530682+0000","last_unstale":"2024-03-28T13:12:54.553642+0000","last_undegraded":"2024-03-28T13:12:54.553642+0000","last_fullsized":"2024-03-28T13:12:54.553642+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:26:05.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.524962+0000","last_change":"2024-03-28T13:12:53.516104+0000","last_active":"2024-03-28T13:12:53.524962+0000","last_peered":"2024-03-28T13:12:53.524962+0000","last_clean":"2024-03-28T13:12:53.524962+0000","last_became_active":"2024-03-28T13:12:53.515238+0000","last_became_peered":"2024-03-28T13:12:53.515238+0000","last_unstale":"2024-03-28T13:12:53.524962+0000","last_undegraded":"2024-03-28T13:12:53.524962+0000","last_fullsized":"2024-03-28T13:12:53.524962+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:42:34.599981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.534771+0000","last_change":"2024-03-28T13:12:53.528585+0000","last_active":"2024-03-28T13:12:53.534771+0000","last_peered":"2024-03-28T13:12:53.534771+0000","last_clean":"2024-03-28T13:12:53.534771+0000","last_became_active":"2024-03-28T13:12:53.528264+0000","last_became_peered":"2024-03-28T13:12:53.528264+0000","last_unstale":"2024-03-28T13:12:53.534771+0000","last_undegraded":"2024-03-28T13:12:53.534771+0000","last_fullsized":"2024-03-28T13:12:53.534771+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:26:05.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523073+0000","last_change":"2024-03-28T13:12:53.529601+0000","last_active":"2024-03-28T13:12:54.523073+0000","last_peered":"2024-03-28T13:12:54.523073+0000","last_clean":"2024-03-28T13:12:54.523073+0000","last_became_active":"2024-03-28T13:12:53.529250+0000","last_became_peered":"2024-03-28T13:12:53.529250+0000","last_unstale":"2024-03-28T13:12:54.523073+0000","last_undegraded":"2024-03-28T13:12:54.523073+0000","last_fullsized":"2024-03-28T13:12:54.523073+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:02:06.529277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.531676+0000","last_change":"2024-03-28T13:12:53.530792+0000","last_active":"2024-03-28T13:12:56.531676+0000","last_peered":"2024-03-28T13:12:56.531676+0000","last_clean":"2024-03-28T13:12:56.531676+0000","last_became_active":"2024-03-28T13:12:53.530071+0000","last_became_peered":"2024-03-28T13:12:53.530071+0000","last_unstale":"2024-03-28T13:12:56.531676+0000","last_undegraded":"2024-03-28T13:12:56.531676+0000","last_fullsized":"2024-03-28T13:12:56.531676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:06:17.410323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.535355+0000","last_change":"2024-03-28T13:12:53.526799+0000","last_active":"2024-03-28T13:12:53.535355+0000","last_peered":"2024-03-28T13:12:53.535355+0000","last_clean":"2024-03-28T13:12:53.535355+0000","last_became_active":"2024-03-28T13:12:53.526520+0000","last_became_peered":"2024-03-28T13:12:53.526520+0000","last_unstale":"2024-03-28T13:12:53.535355+0000","last_undegraded":"2024-03-28T13:12:53.535355+0000","last_fullsized":"2024-03-28T13:12:53.535355+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:46:18.555821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.553773+0000","last_change":"2024-03-28T13:12:53.520815+0000","last_active":"2024-03-28T13:12:54.553773+0000","last_peered":"2024-03-28T13:12:54.553773+0000","last_clean":"2024-03-28T13:12:54.553773+0000","last_became_active":"2024-03-28T13:12:53.520536+0000","last_became_peered":"2024-03-28T13:12:53.520536+0000","last_unstale":"2024-03-28T13:12:54.553773+0000","last_undegraded":"2024-03-28T13:12:54.553773+0000","last_fullsized":"2024-03-28T13:12:54.553773+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:18:34.253144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.538076+0000","last_change":"2024-03-28T13:12:53.535551+0000","last_active":"2024-03-28T13:12:53.538076+0000","last_peered":"2024-03-28T13:12:53.538076+0000","last_clean":"2024-03-28T13:12:53.538076+0000","last_became_active":"2024-03-28T13:12:53.533181+0000","last_became_peered":"2024-03-28T13:12:53.533181+0000","last_unstale":"2024-03-28T13:12:53.538076+0000","last_undegraded":"2024-03-28T13:12:53.538076+0000","last_fullsized":"2024-03-28T13:12:53.538076+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:44.888405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513088+0000","last_change":"2024-03-28T13:12:53.533475+0000","last_active":"2024-03-28T13:12:54.513088+0000","last_peered":"2024-03-28T13:12:54.513088+0000","last_clean":"2024-03-28T13:12:54.513088+0000","last_became_active":"2024-03-28T13:12:53.532757+0000","last_became_peered":"2024-03-28T13:12:53.532757+0000","last_unstale":"2024-03-28T13:12:54.513088+0000","last_undegraded":"2024-03-28T13:12:54.513088+0000","last_fullsized":"2024-03-28T13:12:54.513088+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:55:36.036411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.517789+0000","last_change":"2024-03-28T13:12:53.514611+0000","last_active":"2024-03-28T13:12:53.517789+0000","last_peered":"2024-03-28T13:12:53.517789+0000","last_clean":"2024-03-28T13:12:53.517789+0000","last_became_active":"2024-03-28T13:12:53.511618+0000","last_became_peered":"2024-03-28T13:12:53.511618+0000","last_unstale":"2024-03-28T13:12:53.517789+0000","last_undegraded":"2024-03-28T13:12:53.517789+0000","last_fullsized":"2024-03-28T13:12:53.517789+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:59:36.484476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.531699+0000","last_change":"2024-03-28T13:12:53.532707+0000","last_active":"2024-03-28T13:12:56.531699+0000","last_peered":"2024-03-28T13:12:56.531699+0000","last_clean":"2024-03-28T13:12:56.531699+0000","last_became_active":"2024-03-28T13:12:53.529446+0000","last_became_peered":"2024-03-28T13:12:53.529446+0000","last_unstale":"2024-03-28T13:12:56.531699+0000","last_undegraded":"2024-03-28T13:12:56.531699+0000","last_fullsized":"2024-03-28T13:12:56.531699+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:19:19.903928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.554016+0000","last_change":"2024-03-28T13:12:53.524731+0000","last_active":"2024-03-28T13:12:54.554016+0000","last_peered":"2024-03-28T13:12:54.554016+0000","last_clean":"2024-03-28T13:12:54.554016+0000","last_became_active":"2024-03-28T13:12:53.524429+0000","last_became_peered":"2024-03-28T13:12:53.524429+0000","last_unstale":"2024-03-28T13:12:54.554016+0000","last_undegraded":"2024-03-28T13:12:54.554016+0000","last_fullsized":"2024-03-28T13:12:54.554016+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:41:30.265558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.531997+0000","last_change":"2024-03-28T13:12:53.534582+0000","last_active":"2024-03-28T13:12:56.531997+0000","last_peered":"2024-03-28T13:12:56.531997+0000","last_clean":"2024-03-28T13:12:56.531997+0000","last_became_active":"2024-03-28T13:12:53.533780+0000","last_became_peered":"2024-03-28T13:12:53.533780+0000","last_unstale":"2024-03-28T13:12:56.531997+0000","last_undegraded":"2024-03-28T13:12:56.531997+0000","last_fullsized":"2024-03-28T13:12:56.531997+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:53:53.398050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523778+0000","last_change":"2024-03-28T13:12:53.514047+0000","last_active":"2024-03-28T13:12:53.523778+0000","last_peered":"2024-03-28T13:12:53.523778+0000","last_clean":"2024-03-28T13:12:53.523778+0000","last_became_active":"2024-03-28T13:12:53.513768+0000","last_became_peered":"2024-03-28T13:12:53.513768+0000","last_unstale":"2024-03-28T13:12:53.523778+0000","last_undegraded":"2024-03-28T13:12:53.523778+0000","last_fullsized":"2024-03-28T13:12:53.523778+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:29:02.546768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.516679+0000","last_change":"2024-03-28T13:12:53.503745+0000","last_active":"2024-03-28T13:12:53.516679+0000","last_peered":"2024-03-28T13:12:53.516679+0000","last_clean":"2024-03-28T13:12:53.516679+0000","last_became_active":"2024-03-28T13:12:53.503222+0000","last_became_peered":"2024-03-28T13:12:53.503222+0000","last_unstale":"2024-03-28T13:12:53.516679+0000","last_undegraded":"2024-03-28T13:12:53.516679+0000","last_fullsized":"2024-03-28T13:12:53.516679+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:02:06.529277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513385+0000","last_change":"2024-03-28T13:12:53.526488+0000","last_active":"2024-03-28T13:12:54.513385+0000","last_peered":"2024-03-28T13:12:54.513385+0000","last_clean":"2024-03-28T13:12:54.513385+0000","last_became_active":"2024-03-28T13:12:53.526193+0000","last_became_peered":"2024-03-28T13:12:53.526193+0000","last_unstale":"2024-03-28T13:12:54.513385+0000","last_undegraded":"2024-03-28T13:12:54.513385+0000","last_fullsized":"2024-03-28T13:12:54.513385+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:54:18.665506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.530877+0000","last_change":"2024-03-28T13:12:53.527407+0000","last_active":"2024-03-28T13:12:56.530877+0000","last_peered":"2024-03-28T13:12:56.530877+0000","last_clean":"2024-03-28T13:12:56.530877+0000","last_became_active":"2024-03-28T13:12:53.527110+0000","last_became_peered":"2024-03-28T13:12:53.527110+0000","last_unstale":"2024-03-28T13:12:56.530877+0000","last_undegraded":"2024-03-28T13:12:56.530877+0000","last_fullsized":"2024-03-28T13:12:56.530877+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:09:31.626615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.536760+0000","last_change":"2024-03-28T13:12:53.533474+0000","last_active":"2024-03-28T13:12:53.536760+0000","last_peered":"2024-03-28T13:12:53.536760+0000","last_clean":"2024-03-28T13:12:53.536760+0000","last_became_active":"2024-03-28T13:12:53.533211+0000","last_became_peered":"2024-03-28T13:12:53.533211+0000","last_unstale":"2024-03-28T13:12:53.536760+0000","last_undegraded":"2024-03-28T13:12:53.536760+0000","last_fullsized":"2024-03-28T13:12:53.536760+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:18:34.253144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.526861+0000","last_change":"2024-03-28T13:12:53.521211+0000","last_active":"2024-03-28T13:12:53.526861+0000","last_peered":"2024-03-28T13:12:53.526861+0000","last_clean":"2024-03-28T13:12:53.526861+0000","last_became_active":"2024-03-28T13:12:53.518832+0000","last_became_peered":"2024-03-28T13:12:53.518832+0000","last_unstale":"2024-03-28T13:12:53.526861+0000","last_undegraded":"2024-03-28T13:12:53.526861+0000","last_fullsized":"2024-03-28T13:12:53.526861+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:43.494458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"62'4","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.532064+0000","last_change":"2024-03-28T13:12:52.497633+0000","last_active":"2024-03-28T13:12:53.532064+0000","last_peered":"2024-03-28T13:12:53.532064+0000","last_clean":"2024-03-28T13:12:53.532064+0000","last_became_active":"2024-03-28T13:12:52.497326+0000","last_became_peered":"2024-03-28T13:12:52.497326+0000","last_unstale":"2024-03-28T13:12:53.532064+0000","last_undegraded":"2024-03-28T13:12:53.532064+0000","last_fullsized":"2024-03-28T13:12:53.532064+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:54:27.217786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.534799+0000","last_change":"2024-03-28T13:12:53.529202+0000","last_active":"2024-03-28T13:12:53.534799+0000","last_peered":"2024-03-28T13:12:53.534799+0000","last_clean":"2024-03-28T13:12:53.534799+0000","last_became_active":"2024-03-28T13:12:53.526303+0000","last_became_peered":"2024-03-28T13:12:53.526303+0000","last_unstale":"2024-03-28T13:12:53.534799+0000","last_undegraded":"2024-03-28T13:12:53.534799+0000","last_fullsized":"2024-03-28T13:12:53.534799+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:18:34.253144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.532432+0000","last_change":"2024-03-28T13:12:52.490509+0000","last_active":"2024-03-28T13:12:56.532432+0000","last_peered":"2024-03-28T13:12:56.532432+0000","last_clean":"2024-03-28T13:12:56.532432+0000","last_became_active":"2024-03-28T13:12:52.490224+0000","last_became_peered":"2024-03-28T13:12:52.490224+0000","last_unstale":"2024-03-28T13:12:56.532432+0000","last_undegraded":"2024-03-28T13:12:56.532432+0000","last_fullsized":"2024-03-28T13:12:56.532432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:05.546468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.532373+0000","last_change":"2024-03-28T13:12:53.523604+0000","last_active":"2024-03-28T13:12:53.532373+0000","last_peered":"2024-03-28T13:12:53.532373+0000","last_clean":"2024-03-28T13:12:53.532373+0000","last_became_active":"2024-03-28T13:12:53.523220+0000","last_became_peered":"2024-03-28T13:12:53.523220+0000","last_unstale":"2024-03-28T13:12:53.532373+0000","last_undegraded":"2024-03-28T13:12:53.532373+0000","last_fullsized":"2024-03-28T13:12:53.532373+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:48:21.538965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.538179+0000","last_change":"2024-03-28T13:12:52.497064+0000","last_active":"2024-03-28T13:12:53.538179+0000","last_peered":"2024-03-28T13:12:53.538179+0000","last_clean":"2024-03-28T13:12:53.538179+0000","last_became_active":"2024-03-28T13:12:52.496174+0000","last_became_peered":"2024-03-28T13:12:52.496174+0000","last_unstale":"2024-03-28T13:12:53.538179+0000","last_undegraded":"2024-03-28T13:12:53.538179+0000","last_fullsized":"2024-03-28T13:12:53.538179+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:30.940424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.534691+0000","last_change":"2024-03-28T13:12:53.529791+0000","last_active":"2024-03-28T13:12:53.534691+0000","last_peered":"2024-03-28T13:12:53.534691+0000","last_clean":"2024-03-28T13:12:53.534691+0000","last_became_active":"2024-03-28T13:12:53.529522+0000","last_became_peered":"2024-03-28T13:12:53.529522+0000","last_unstale":"2024-03-28T13:12:53.534691+0000","last_undegraded":"2024-03-28T13:12:53.534691+0000","last_fullsized":"2024-03-28T13:12:53.534691+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:44.888405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"62'2","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513761+0000","last_change":"2024-03-28T13:12:52.472395+0000","last_active":"2024-03-28T13:12:54.513761+0000","last_peered":"2024-03-28T13:12:54.513761+0000","last_clean":"2024-03-28T13:12:54.513761+0000","last_became_active":"2024-03-28T13:12:52.471810+0000","last_became_peered":"2024-03-28T13:12:52.471810+0000","last_unstale":"2024-03-28T13:12:54.513761+0000","last_undegraded":"2024-03-28T13:12:54.513761+0000","last_fullsized":"2024-03-28T13:12:54.513761+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:47:30.323323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523777+0000","last_change":"2024-03-28T13:12:53.517800+0000","last_active":"2024-03-28T13:12:53.523777+0000","last_peered":"2024-03-28T13:12:53.523777+0000","last_clean":"2024-03-28T13:12:53.523777+0000","last_became_active":"2024-03-28T13:12:53.517514+0000","last_became_peered":"2024-03-28T13:12:53.517514+0000","last_unstale":"2024-03-28T13:12:53.523777+0000","last_undegraded":"2024-03-28T13:12:53.523777+0000","last_fullsized":"2024-03-28T13:12:53.523777+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:50:31.090629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.522731+0000","last_change":"2024-03-28T13:12:52.498789+0000","last_active":"2024-03-28T13:12:54.522731+0000","last_peered":"2024-03-28T13:12:54.522731+0000","last_clean":"2024-03-28T13:12:54.522731+0000","last_became_active":"2024-03-28T13:12:52.498391+0000","last_became_peered":"2024-03-28T13:12:52.498391+0000","last_unstale":"2024-03-28T13:12:54.522731+0000","last_undegraded":"2024-03-28T13:12:54.522731+0000","last_fullsized":"2024-03-28T13:12:54.522731+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:21.358714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.535449+0000","last_change":"2024-03-28T13:12:53.523100+0000","last_active":"2024-03-28T13:12:53.535449+0000","last_peered":"2024-03-28T13:12:53.535449+0000","last_clean":"2024-03-28T13:12:53.535449+0000","last_became_active":"2024-03-28T13:12:53.522846+0000","last_became_peered":"2024-03-28T13:12:53.522846+0000","last_unstale":"2024-03-28T13:12:53.535449+0000","last_undegraded":"2024-03-28T13:12:53.535449+0000","last_fullsized":"2024-03-28T13:12:53.535449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:26:05.020404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"62'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.533964+0000","last_change":"2024-03-28T13:12:52.493590+0000","last_active":"2024-03-28T13:12:53.533964+0000","last_peered":"2024-03-28T13:12:53.533964+0000","last_clean":"2024-03-28T13:12:53.533964+0000","last_became_active":"2024-03-28T13:12:52.493306+0000","last_became_peered":"2024-03-28T13:12:52.493306+0000","last_unstale":"2024-03-28T13:12:53.533964+0000","last_undegraded":"2024-03-28T13:12:53.533964+0000","last_fullsized":"2024-03-28T13:12:53.533964+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:36:31.168233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.522490+0000","last_change":"2024-03-28T13:12:53.517696+0000","last_active":"2024-03-28T13:12:54.522490+0000","last_peered":"2024-03-28T13:12:54.522490+0000","last_clean":"2024-03-28T13:12:54.522490+0000","last_became_active":"2024-03-28T13:12:53.513400+0000","last_became_peered":"2024-03-28T13:12:53.513400+0000","last_unstale":"2024-03-28T13:12:54.522490+0000","last_undegraded":"2024-03-28T13:12:54.522490+0000","last_fullsized":"2024-03-28T13:12:54.522490+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:44.888405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.486660+0000","last_change":"2024-03-28T13:12:52.490236+0000","last_active":"2024-03-28T13:12:53.486660+0000","last_peered":"2024-03-28T13:12:53.486660+0000","last_clean":"2024-03-28T13:12:53.486660+0000","last_became_active":"2024-03-28T13:12:52.489880+0000","last_became_peered":"2024-03-28T13:12:52.489880+0000","last_unstale":"2024-03-28T13:12:53.486660+0000","last_undegraded":"2024-03-28T13:12:53.486660+0000","last_fullsized":"2024-03-28T13:12:53.486660+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:12:13.076721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.525049+0000","last_change":"2024-03-28T13:12:53.515864+0000","last_active":"2024-03-28T13:12:53.525049+0000","last_peered":"2024-03-28T13:12:53.525049+0000","last_clean":"2024-03-28T13:12:53.525049+0000","last_became_active":"2024-03-28T13:12:53.515528+0000","last_became_peered":"2024-03-28T13:12:53.515528+0000","last_unstale":"2024-03-28T13:12:53.525049+0000","last_undegraded":"2024-03-28T13:12:53.525049+0000","last_fullsized":"2024-03-28T13:12:53.525049+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:20:57.766567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.553494+0000","last_change":"2024-03-28T13:12:52.497223+0000","last_active":"2024-03-28T13:12:54.553494+0000","last_peered":"2024-03-28T13:12:54.553494+0000","last_clean":"2024-03-28T13:12:54.553494+0000","last_became_active":"2024-03-28T13:12:52.496968+0000","last_became_peered":"2024-03-28T13:12:52.496968+0000","last_unstale":"2024-03-28T13:12:54.553494+0000","last_undegraded":"2024-03-28T13:12:54.553494+0000","last_fullsized":"2024-03-28T13:12:54.553494+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:21.414302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.554104+0000","last_change":"2024-03-28T13:12:53.521865+0000","last_active":"2024-03-28T13:12:54.554104+0000","last_peered":"2024-03-28T13:12:54.554104+0000","last_clean":"2024-03-28T13:12:54.554104+0000","last_became_active":"2024-03-28T13:12:53.521470+0000","last_became_peered":"2024-03-28T13:12:53.521470+0000","last_unstale":"2024-03-28T13:12:54.554104+0000","last_undegraded":"2024-03-28T13:12:54.554104+0000","last_fullsized":"2024-03-28T13:12:54.554104+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:02:06.529277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.489279+0000","last_change":"2024-03-28T13:12:52.493670+0000","last_active":"2024-03-28T13:12:53.489279+0000","last_peered":"2024-03-28T13:12:53.489279+0000","last_clean":"2024-03-28T13:12:53.489279+0000","last_became_active":"2024-03-28T13:12:52.493461+0000","last_became_peered":"2024-03-28T13:12:52.493461+0000","last_unstale":"2024-03-28T13:12:53.489279+0000","last_undegraded":"2024-03-28T13:12:53.489279+0000","last_fullsized":"2024-03-28T13:12:53.489279+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:34:12.832012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523240+0000","last_change":"2024-03-28T13:12:53.529215+0000","last_active":"2024-03-28T13:12:54.523240+0000","last_peered":"2024-03-28T13:12:54.523240+0000","last_clean":"2024-03-28T13:12:54.523240+0000","last_became_active":"2024-03-28T13:12:53.528645+0000","last_became_peered":"2024-03-28T13:12:53.528645+0000","last_unstale":"2024-03-28T13:12:54.523240+0000","last_undegraded":"2024-03-28T13:12:54.523240+0000","last_fullsized":"2024-03-28T13:12:54.523240+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:29:02.546768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.492468+0000","last_change":"2024-03-28T13:12:52.497766+0000","last_active":"2024-03-28T13:12:53.492468+0000","last_peered":"2024-03-28T13:12:53.492468+0000","last_clean":"2024-03-28T13:12:53.492468+0000","last_became_active":"2024-03-28T13:12:52.497455+0000","last_became_peered":"2024-03-28T13:12:52.497455+0000","last_unstale":"2024-03-28T13:12:53.492468+0000","last_undegraded":"2024-03-28T13:12:53.492468+0000","last_fullsized":"2024-03-28T13:12:53.492468+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:18:33.238734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523838+0000","last_change":"2024-03-28T13:12:53.518732+0000","last_active":"2024-03-28T13:12:53.523838+0000","last_peered":"2024-03-28T13:12:53.523838+0000","last_clean":"2024-03-28T13:12:53.523838+0000","last_became_active":"2024-03-28T13:12:53.518456+0000","last_became_peered":"2024-03-28T13:12:53.518456+0000","last_unstale":"2024-03-28T13:12:53.523838+0000","last_undegraded":"2024-03-28T13:12:53.523838+0000","last_fullsized":"2024-03-28T13:12:53.523838+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:48:29.629780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"62'3","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513707+0000","last_change":"2024-03-28T13:12:52.471608+0000","last_active":"2024-03-28T13:12:54.513707+0000","last_peered":"2024-03-28T13:12:54.513707+0000","last_clean":"2024-03-28T13:12:54.513707+0000","last_became_active":"2024-03-28T13:12:52.469801+0000","last_became_peered":"2024-03-28T13:12:52.469801+0000","last_unstale":"2024-03-28T13:12:54.513707+0000","last_undegraded":"2024-03-28T13:12:54.513707+0000","last_fullsized":"2024-03-28T13:12:54.513707+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:21:22.397053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513592+0000","last_change":"2024-03-28T13:12:53.533647+0000","last_active":"2024-03-28T13:12:54.513592+0000","last_peered":"2024-03-28T13:12:54.513592+0000","last_clean":"2024-03-28T13:12:54.513592+0000","last_became_active":"2024-03-28T13:12:53.533358+0000","last_became_peered":"2024-03-28T13:12:53.533358+0000","last_unstale":"2024-03-28T13:12:54.513592+0000","last_undegraded":"2024-03-28T13:12:54.513592+0000","last_fullsized":"2024-03-28T13:12:54.513592+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:31:13.215911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"62'6","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.553833+0000","last_change":"2024-03-28T13:12:52.489670+0000","last_active":"2024-03-28T13:12:54.553833+0000","last_peered":"2024-03-28T13:12:54.553833+0000","last_clean":"2024-03-28T13:12:54.553833+0000","last_became_active":"2024-03-28T13:12:52.489326+0000","last_became_peered":"2024-03-28T13:12:52.489326+0000","last_unstale":"2024-03-28T13:12:54.553833+0000","last_undegraded":"2024-03-28T13:12:54.553833+0000","last_fullsized":"2024-03-28T13:12:54.553833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:21.358714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.534598+0000","last_change":"2024-03-28T13:12:53.532827+0000","last_active":"2024-03-28T13:12:53.534598+0000","last_peered":"2024-03-28T13:12:53.534598+0000","last_clean":"2024-03-28T13:12:53.534598+0000","last_became_active":"2024-03-28T13:12:53.532369+0000","last_became_peered":"2024-03-28T13:12:53.532369+0000","last_unstale":"2024-03-28T13:12:53.534598+0000","last_undegraded":"2024-03-28T13:12:53.534598+0000","last_fullsized":"2024-03-28T13:12:53.534598+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:10:42.898595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.537856+0000","last_change":"2024-03-28T13:12:52.500541+0000","last_active":"2024-03-28T13:12:53.537856+0000","last_peered":"2024-03-28T13:12:53.537856+0000","last_clean":"2024-03-28T13:12:53.537856+0000","last_became_active":"2024-03-28T13:12:52.499747+0000","last_became_peered":"2024-03-28T13:12:52.499747+0000","last_unstale":"2024-03-28T13:12:53.537856+0000","last_undegraded":"2024-03-28T13:12:53.537856+0000","last_fullsized":"2024-03-28T13:12:53.537856+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:46:17.541410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.522763+0000","last_change":"2024-03-28T13:12:53.515617+0000","last_active":"2024-03-28T13:12:54.522763+0000","last_peered":"2024-03-28T13:12:54.522763+0000","last_clean":"2024-03-28T13:12:54.522763+0000","last_became_active":"2024-03-28T13:12:53.515344+0000","last_became_peered":"2024-03-28T13:12:53.515344+0000","last_unstale":"2024-03-28T13:12:54.522763+0000","last_undegraded":"2024-03-28T13:12:54.522763+0000","last_fullsized":"2024-03-28T13:12:54.522763+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:56.402530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513712+0000","last_change":"2024-03-28T13:12:52.486702+0000","last_active":"2024-03-28T13:12:54.513712+0000","last_peered":"2024-03-28T13:12:54.513712+0000","last_clean":"2024-03-28T13:12:54.513712+0000","last_became_active":"2024-03-28T13:12:52.486386+0000","last_became_peered":"2024-03-28T13:12:52.486386+0000","last_unstale":"2024-03-28T13:12:54.513712+0000","last_undegraded":"2024-03-28T13:12:54.513712+0000","last_fullsized":"2024-03-28T13:12:54.513712+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:09.555954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.514793+0000","last_change":"2024-03-28T13:12:53.511880+0000","last_active":"2024-03-28T13:12:53.514793+0000","last_peered":"2024-03-28T13:12:53.514793+0000","last_clean":"2024-03-28T13:12:53.514793+0000","last_became_active":"2024-03-28T13:12:53.511268+0000","last_became_peered":"2024-03-28T13:12:53.511268+0000","last_unstale":"2024-03-28T13:12:53.514793+0000","last_undegraded":"2024-03-28T13:12:53.514793+0000","last_fullsized":"2024-03-28T13:12:53.514793+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:05:58.452615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513338+0000","last_change":"2024-03-28T13:12:52.486944+0000","last_active":"2024-03-28T13:12:54.513338+0000","last_peered":"2024-03-28T13:12:54.513338+0000","last_clean":"2024-03-28T13:12:54.513338+0000","last_became_active":"2024-03-28T13:12:52.485574+0000","last_became_peered":"2024-03-28T13:12:52.485574+0000","last_unstale":"2024-03-28T13:12:54.513338+0000","last_undegraded":"2024-03-28T13:12:54.513338+0000","last_fullsized":"2024-03-28T13:12:54.513338+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:50:54.412377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513359+0000","last_change":"2024-03-28T13:12:53.533711+0000","last_active":"2024-03-28T13:12:54.513359+0000","last_peered":"2024-03-28T13:12:54.513359+0000","last_clean":"2024-03-28T13:12:54.513359+0000","last_became_active":"2024-03-28T13:12:53.533128+0000","last_became_peered":"2024-03-28T13:12:53.533128+0000","last_unstale":"2024-03-28T13:12:54.513359+0000","last_undegraded":"2024-03-28T13:12:54.513359+0000","last_fullsized":"2024-03-28T13:12:54.513359+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:51:57.502055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"62'8","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.512980+0000","last_change":"2024-03-28T13:12:52.481249+0000","last_active":"2024-03-28T13:12:54.512980+0000","last_peered":"2024-03-28T13:12:54.512980+0000","last_clean":"2024-03-28T13:12:54.512980+0000","last_became_active":"2024-03-28T13:12:52.480966+0000","last_became_peered":"2024-03-28T13:12:52.480966+0000","last_unstale":"2024-03-28T13:12:54.512980+0000","last_undegraded":"2024-03-28T13:12:54.512980+0000","last_fullsized":"2024-03-28T13:12:54.512980+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:47:56.291512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523183+0000","last_change":"2024-03-28T13:12:53.516944+0000","last_active":"2024-03-28T13:12:54.523183+0000","last_peered":"2024-03-28T13:12:54.523183+0000","last_clean":"2024-03-28T13:12:54.523183+0000","last_became_active":"2024-03-28T13:12:53.516647+0000","last_became_peered":"2024-03-28T13:12:53.516647+0000","last_unstale":"2024-03-28T13:12:54.523183+0000","last_undegraded":"2024-03-28T13:12:54.523183+0000","last_fullsized":"2024-03-28T13:12:54.523183+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:02:51.661702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.531969+0000","last_change":"2024-03-28T13:12:52.499069+0000","last_active":"2024-03-28T13:12:56.531969+0000","last_peered":"2024-03-28T13:12:56.531969+0000","last_clean":"2024-03-28T13:12:56.531969+0000","last_became_active":"2024-03-28T13:12:52.498808+0000","last_became_peered":"2024-03-28T13:12:52.498808+0000","last_unstale":"2024-03-28T13:12:56.531969+0000","last_undegraded":"2024-03-28T13:12:56.531969+0000","last_fullsized":"2024-03-28T13:12:56.531969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:21:56.447486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523426+0000","last_change":"2024-03-28T13:12:53.528894+0000","last_active":"2024-03-28T13:12:54.523426+0000","last_peered":"2024-03-28T13:12:54.523426+0000","last_clean":"2024-03-28T13:12:54.523426+0000","last_became_active":"2024-03-28T13:12:53.528275+0000","last_became_peered":"2024-03-28T13:12:53.528275+0000","last_unstale":"2024-03-28T13:12:54.523426+0000","last_undegraded":"2024-03-28T13:12:54.523426+0000","last_fullsized":"2024-03-28T13:12:54.523426+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:50:31.090629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.537729+0000","last_change":"2024-03-28T13:12:52.498458+0000","last_active":"2024-03-28T13:12:53.537729+0000","last_peered":"2024-03-28T13:12:53.537729+0000","last_clean":"2024-03-28T13:12:53.537729+0000","last_became_active":"2024-03-28T13:12:52.497234+0000","last_became_peered":"2024-03-28T13:12:52.497234+0000","last_unstale":"2024-03-28T13:12:53.537729+0000","last_undegraded":"2024-03-28T13:12:53.537729+0000","last_fullsized":"2024-03-28T13:12:53.537729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:43.873994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.532733+0000","last_change":"2024-03-28T13:12:53.531284+0000","last_active":"2024-03-28T13:12:56.532733+0000","last_peered":"2024-03-28T13:12:56.532733+0000","last_clean":"2024-03-28T13:12:56.532733+0000","last_became_active":"2024-03-28T13:12:53.530318+0000","last_became_peered":"2024-03-28T13:12:53.530318+0000","last_unstale":"2024-03-28T13:12:56.532733+0000","last_undegraded":"2024-03-28T13:12:56.532733+0000","last_fullsized":"2024-03-28T13:12:56.532733+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:46:31.798711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523193+0000","last_change":"2024-03-28T13:12:52.489367+0000","last_active":"2024-03-28T13:12:54.523193+0000","last_peered":"2024-03-28T13:12:54.523193+0000","last_clean":"2024-03-28T13:12:54.523193+0000","last_became_active":"2024-03-28T13:12:52.488219+0000","last_became_peered":"2024-03-28T13:12:52.488219+0000","last_unstale":"2024-03-28T13:12:54.523193+0000","last_undegraded":"2024-03-28T13:12:54.523193+0000","last_fullsized":"2024-03-28T13:12:54.523193+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:30.940424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.513591+0000","last_change":"2024-03-28T13:12:53.517878+0000","last_active":"2024-03-28T13:12:54.513591+0000","last_peered":"2024-03-28T13:12:54.513591+0000","last_clean":"2024-03-28T13:12:54.513591+0000","last_became_active":"2024-03-28T13:12:53.517601+0000","last_became_peered":"2024-03-28T13:12:53.517601+0000","last_unstale":"2024-03-28T13:12:54.513591+0000","last_undegraded":"2024-03-28T13:12:54.513591+0000","last_fullsized":"2024-03-28T13:12:54.513591+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:07:06.162755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.494129+0000","last_change":"2024-03-28T13:12:52.492132+0000","last_active":"2024-03-28T13:12:53.494129+0000","last_peered":"2024-03-28T13:12:53.494129+0000","last_clean":"2024-03-28T13:12:53.494129+0000","last_became_active":"2024-03-28T13:12:52.491829+0000","last_became_peered":"2024-03-28T13:12:52.491829+0000","last_unstale":"2024-03-28T13:12:53.494129+0000","last_undegraded":"2024-03-28T13:12:53.494129+0000","last_fullsized":"2024-03-28T13:12:53.494129+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:55.388119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523687+0000","last_change":"2024-03-28T13:12:53.519732+0000","last_active":"2024-03-28T13:12:53.523687+0000","last_peered":"2024-03-28T13:12:53.523687+0000","last_clean":"2024-03-28T13:12:53.523687+0000","last_became_active":"2024-03-28T13:12:53.519417+0000","last_became_peered":"2024-03-28T13:12:53.519417+0000","last_unstale":"2024-03-28T13:12:53.523687+0000","last_undegraded":"2024-03-28T13:12:53.523687+0000","last_fullsized":"2024-03-28T13:12:53.523687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:40:22.156766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"62'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.539505+0000","last_change":"2024-03-28T13:12:52.497898+0000","last_active":"2024-03-28T13:12:53.539505+0000","last_peered":"2024-03-28T13:12:53.539505+0000","last_clean":"2024-03-28T13:12:53.539505+0000","last_became_active":"2024-03-28T13:12:52.497054+0000","last_became_peered":"2024-03-28T13:12:52.497054+0000","last_unstale":"2024-03-28T13:12:53.539505+0000","last_undegraded":"2024-03-28T13:12:53.539505+0000","last_fullsized":"2024-03-28T13:12:53.539505+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:42.480048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.521670+0000","last_change":"2024-03-28T13:12:53.512993+0000","last_active":"2024-03-28T13:12:53.521670+0000","last_peered":"2024-03-28T13:12:53.521670+0000","last_clean":"2024-03-28T13:12:53.521670+0000","last_became_active":"2024-03-28T13:12:53.512788+0000","last_became_peered":"2024-03-28T13:12:53.512788+0000","last_unstale":"2024-03-28T13:12:53.521670+0000","last_undegraded":"2024-03-28T13:12:53.521670+0000","last_fullsized":"2024-03-28T13:12:53.521670+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:27:53.920916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.491141+0000","last_change":"2024-03-28T13:12:52.500236+0000","last_active":"2024-03-28T13:12:53.491141+0000","last_peered":"2024-03-28T13:12:53.491141+0000","last_clean":"2024-03-28T13:12:53.491141+0000","last_became_active":"2024-03-28T13:12:52.499795+0000","last_became_peered":"2024-03-28T13:12:52.499795+0000","last_unstale":"2024-03-28T13:12:53.491141+0000","last_undegraded":"2024-03-28T13:12:53.491141+0000","last_fullsized":"2024-03-28T13:12:53.491141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:16:15.524903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523835+0000","last_change":"2024-03-28T13:12:53.515564+0000","last_active":"2024-03-28T13:12:53.523835+0000","last_peered":"2024-03-28T13:12:53.523835+0000","last_clean":"2024-03-28T13:12:53.523835+0000","last_became_active":"2024-03-28T13:12:53.514826+0000","last_became_peered":"2024-03-28T13:12:53.514826+0000","last_unstale":"2024-03-28T13:12:53.523835+0000","last_undegraded":"2024-03-28T13:12:53.523835+0000","last_fullsized":"2024-03-28T13:12:53.523835+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:20:59.923272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.489947+0000","last_change":"2024-03-28T13:12:52.492600+0000","last_active":"2024-03-28T13:12:53.489947+0000","last_peered":"2024-03-28T13:12:53.489947+0000","last_clean":"2024-03-28T13:12:53.489947+0000","last_became_active":"2024-03-28T13:12:52.492285+0000","last_became_peered":"2024-03-28T13:12:52.492285+0000","last_unstale":"2024-03-28T13:12:53.489947+0000","last_undegraded":"2024-03-28T13:12:53.489947+0000","last_fullsized":"2024-03-28T13:12:53.489947+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:39:47.453340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523338+0000","last_change":"2024-03-28T13:12:53.524081+0000","last_active":"2024-03-28T13:12:54.523338+0000","last_peered":"2024-03-28T13:12:54.523338+0000","last_clean":"2024-03-28T13:12:54.523338+0000","last_became_active":"2024-03-28T13:12:53.523708+0000","last_became_peered":"2024-03-28T13:12:53.523708+0000","last_unstale":"2024-03-28T13:12:54.523338+0000","last_undegraded":"2024-03-28T13:12:54.523338+0000","last_fullsized":"2024-03-28T13:12:54.523338+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:20:59.923272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"62'2","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.532229+0000","last_change":"2024-03-28T13:12:52.492725+0000","last_active":"2024-03-28T13:12:56.532229+0000","last_peered":"2024-03-28T13:12:56.532229+0000","last_clean":"2024-03-28T13:12:56.532229+0000","last_became_active":"2024-03-28T13:12:52.491883+0000","last_became_peered":"2024-03-28T13:12:52.491883+0000","last_unstale":"2024-03-28T13:12:56.532229+0000","last_undegraded":"2024-03-28T13:12:56.532229+0000","last_fullsized":"2024-03-28T13:12:56.532229+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:58:32.915798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.523342+0000","last_change":"2024-03-28T13:12:53.520223+0000","last_active":"2024-03-28T13:12:54.523342+0000","last_peered":"2024-03-28T13:12:54.523342+0000","last_clean":"2024-03-28T13:12:54.523342+0000","last_became_active":"2024-03-28T13:12:53.513629+0000","last_became_peered":"2024-03-28T13:12:53.513629+0000","last_unstale":"2024-03-28T13:12:54.523342+0000","last_undegraded":"2024-03-28T13:12:54.523342+0000","last_fullsized":"2024-03-28T13:12:54.523342+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:49:35.695939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.509738+0000","last_change":"2024-03-28T13:12:52.495889+0000","last_active":"2024-03-28T13:12:53.509738+0000","last_peered":"2024-03-28T13:12:53.509738+0000","last_clean":"2024-03-28T13:12:53.509738+0000","last_became_active":"2024-03-28T13:12:52.495589+0000","last_became_peered":"2024-03-28T13:12:52.495589+0000","last_unstale":"2024-03-28T13:12:53.509738+0000","last_undegraded":"2024-03-28T13:12:53.509738+0000","last_fullsized":"2024-03-28T13:12:53.509738+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:21.358714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.532854+0000","last_change":"2024-03-28T13:12:53.534280+0000","last_active":"2024-03-28T13:12:56.532854+0000","last_peered":"2024-03-28T13:12:56.532854+0000","last_clean":"2024-03-28T13:12:56.532854+0000","last_became_active":"2024-03-28T13:12:53.534031+0000","last_became_peered":"2024-03-28T13:12:53.534031+0000","last_unstale":"2024-03-28T13:12:56.532854+0000","last_undegraded":"2024-03-28T13:12:56.532854+0000","last_fullsized":"2024-03-28T13:12:56.532854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:28:58.765737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.488213+0000","last_change":"2024-03-28T13:12:52.491050+0000","last_active":"2024-03-28T13:12:53.488213+0000","last_peered":"2024-03-28T13:12:53.488213+0000","last_clean":"2024-03-28T13:12:53.488213+0000","last_became_active":"2024-03-28T13:12:52.490722+0000","last_became_peered":"2024-03-28T13:12:52.490722+0000","last_unstale":"2024-03-28T13:12:53.488213+0000","last_undegraded":"2024-03-28T13:12:53.488213+0000","last_fullsized":"2024-03-28T13:12:53.488213+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:12:48.954255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523979+0000","last_change":"2024-03-28T13:12:53.515816+0000","last_active":"2024-03-28T13:12:53.523979+0000","last_peered":"2024-03-28T13:12:53.523979+0000","last_clean":"2024-03-28T13:12:53.523979+0000","last_became_active":"2024-03-28T13:12:53.515487+0000","last_became_peered":"2024-03-28T13:12:53.515487+0000","last_unstale":"2024-03-28T13:12:53.523979+0000","last_undegraded":"2024-03-28T13:12:53.523979+0000","last_fullsized":"2024-03-28T13:12:53.523979+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:49:35.695939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.522863+0000","last_change":"2024-03-28T13:12:52.481784+0000","last_active":"2024-03-28T13:12:54.522863+0000","last_peered":"2024-03-28T13:12:54.522863+0000","last_clean":"2024-03-28T13:12:54.522863+0000","last_became_active":"2024-03-28T13:12:52.481499+0000","last_became_peered":"2024-03-28T13:12:52.481499+0000","last_unstale":"2024-03-28T13:12:54.522863+0000","last_undegraded":"2024-03-28T13:12:54.522863+0000","last_fullsized":"2024-03-28T13:12:54.522863+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:36:31.168233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.525144+0000","last_change":"2024-03-28T13:12:53.517879+0000","last_active":"2024-03-28T13:12:53.525144+0000","last_peered":"2024-03-28T13:12:53.525144+0000","last_clean":"2024-03-28T13:12:53.525144+0000","last_became_active":"2024-03-28T13:12:53.517550+0000","last_became_peered":"2024-03-28T13:12:53.517550+0000","last_unstale":"2024-03-28T13:12:53.525144+0000","last_undegraded":"2024-03-28T13:12:53.525144+0000","last_fullsized":"2024-03-28T13:12:53.525144+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:51:13.898620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.531064+0000","last_change":"2024-03-28T13:12:52.499023+0000","last_active":"2024-03-28T13:12:56.531064+0000","last_peered":"2024-03-28T13:12:56.531064+0000","last_clean":"2024-03-28T13:12:56.531064+0000","last_became_active":"2024-03-28T13:12:52.497835+0000","last_became_peered":"2024-03-28T13:12:52.497835+0000","last_unstale":"2024-03-28T13:12:56.531064+0000","last_undegraded":"2024-03-28T13:12:56.531064+0000","last_fullsized":"2024-03-28T13:12:56.531064+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:20:42.508336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.553970+0000","last_change":"2024-03-28T13:12:53.525102+0000","last_active":"2024-03-28T13:12:54.553970+0000","last_peered":"2024-03-28T13:12:54.553970+0000","last_clean":"2024-03-28T13:12:54.553970+0000","last_became_active":"2024-03-28T13:12:53.524410+0000","last_became_peered":"2024-03-28T13:12:53.524410+0000","last_unstale":"2024-03-28T13:12:54.553970+0000","last_undegraded":"2024-03-28T13:12:54.553970+0000","last_fullsized":"2024-03-28T13:12:54.553970+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:56.402530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.522654+0000","last_change":"2024-03-28T13:12:52.492624+0000","last_active":"2024-03-28T13:12:54.522654+0000","last_peered":"2024-03-28T13:12:54.522654+0000","last_clean":"2024-03-28T13:12:54.522654+0000","last_became_active":"2024-03-28T13:12:52.492151+0000","last_became_peered":"2024-03-28T13:12:52.492151+0000","last_unstale":"2024-03-28T13:12:54.522654+0000","last_undegraded":"2024-03-28T13:12:54.522654+0000","last_fullsized":"2024-03-28T13:12:54.522654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:21.414302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.521781+0000","last_change":"2024-03-28T13:12:53.511121+0000","last_active":"2024-03-28T13:12:53.521781+0000","last_peered":"2024-03-28T13:12:53.521781+0000","last_clean":"2024-03-28T13:12:53.521781+0000","last_became_active":"2024-03-28T13:12:53.510092+0000","last_became_peered":"2024-03-28T13:12:53.510092+0000","last_unstale":"2024-03-28T13:12:53.521781+0000","last_undegraded":"2024-03-28T13:12:53.521781+0000","last_fullsized":"2024-03-28T13:12:53.521781+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:02:51.661702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.509614+0000","last_change":"2024-03-28T13:12:52.495791+0000","last_active":"2024-03-28T13:12:53.509614+0000","last_peered":"2024-03-28T13:12:53.509614+0000","last_clean":"2024-03-28T13:12:53.509614+0000","last_became_active":"2024-03-28T13:12:52.495465+0000","last_became_peered":"2024-03-28T13:12:52.495465+0000","last_unstale":"2024-03-28T13:12:53.509614+0000","last_undegraded":"2024-03-28T13:12:53.509614+0000","last_fullsized":"2024-03-28T13:12:53.509614+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:31:21.414302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:56.530881+0000","last_change":"2024-03-28T13:12:53.532522+0000","last_active":"2024-03-28T13:12:56.530881+0000","last_peered":"2024-03-28T13:12:56.530881+0000","last_clean":"2024-03-28T13:12:56.530881+0000","last_became_active":"2024-03-28T13:12:53.532132+0000","last_became_peered":"2024-03-28T13:12:53.532132+0000","last_unstale":"2024-03-28T13:12:56.530881+0000","last_undegraded":"2024-03-28T13:12:56.530881+0000","last_fullsized":"2024-03-28T13:12:56.530881+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:25:06.549357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.492125+0000","last_change":"2024-03-28T13:12:52.487641+0000","last_active":"2024-03-28T13:12:53.492125+0000","last_peered":"2024-03-28T13:12:53.492125+0000","last_clean":"2024-03-28T13:12:53.492125+0000","last_became_active":"2024-03-28T13:12:52.487320+0000","last_became_peered":"2024-03-28T13:12:52.487320+0000","last_unstale":"2024-03-28T13:12:53.492125+0000","last_undegraded":"2024-03-28T13:12:53.492125+0000","last_fullsized":"2024-03-28T13:12:53.492125+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:26:04.005993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.523841+0000","last_change":"2024-03-28T13:12:53.520321+0000","last_active":"2024-03-28T13:12:53.523841+0000","last_peered":"2024-03-28T13:12:53.523841+0000","last_clean":"2024-03-28T13:12:53.523841+0000","last_became_active":"2024-03-28T13:12:53.520018+0000","last_became_peered":"2024-03-28T13:12:53.520018+0000","last_unstale":"2024-03-28T13:12:53.523841+0000","last_undegraded":"2024-03-28T13:12:53.523841+0000","last_fullsized":"2024-03-28T13:12:53.523841+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:52.450259+0000","last_clean_scrub_stamp":"2024-03-28T13:12:52.450259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:28:01.110960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-28T13:12:54.553606+0000","last_change":"2024-03-28T13:12:52.484733+0000","last_active":"2024-03-28T13:12:54.553606+0000","last_peered":"2024-03-28T13:12:54.553606+0000","last_clean":"2024-03-28T13:12:54.553606+0000","last_became_active":"2024-03-28T13:12:52.483907+0000","last_became_peered":"2024-03-28T13:12:52.483907+0000","last_unstale":"2024-03-28T13:12:54.553606+0000","last_undegraded":"2024-03-28T13:12:54.553606+0000","last_fullsized":"2024-03-28T13:12:54.553606+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:12:51.435849+0000","last_clean_scrub_stamp":"2024-03-28T13:12:51.435849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:20:56.752156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":241,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-28T13:12:53.488175+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:12:53.488175+0000","last_peered":"2024-03-28T13:12:53.488175+0000","last_clean":"2024-03-28T13:12:53.488175+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:12:53.488175+0000","last_undegraded":"2024-03-28T13:12:53.488175+0000","last_fullsized":"2024-03-28T13:12:53.488175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476783,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":737280,"data_stored":305832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476781,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":753664,"data_stored":307514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476781,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1331200,"data_stored":896692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476781,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":745472,"data_stored":307354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":737280,"data_stored":305764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509485,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1314816,"data_stored":889335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509485,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":720896,"data_stored":298967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":753664,"data_stored":312134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:12:58.645 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-28T13:12:58.645 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-28T13:12:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.919+0000 7f3c35c00640 1 Processor -- start 2024-03-28T13:12:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.920+0000 7f3c35c00640 1 -- start start 2024-03-28T13:12:58.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.921+0000 7f3c35c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c30155550 0x7f3c30155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:58.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.921+0000 7f3c35c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3c3015f800 0x7f3c300d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:58.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.921+0000 7f3c35c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c300b40b0 con 0x7f3c300d07c0 2024-03-28T13:12:58.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.921+0000 7f3c35c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3c300d0c30 con 0x7f3c3015f800 2024-03-28T13:12:58.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.921+0000 7f3c35c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3c300b4390 con 0x7f3c30155550 2024-03-28T13:12:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.922+0000 7f3c2e000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3c3015f800 0x7f3c300d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.922+0000 7f3c2e000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3c3015f800 0x7f3c300d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33850/0 (socket says 172.21.15.19:33850) 2024-03-28T13:12:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.922+0000 7f3c2e000640 1 -- 172.21.15.19:0/3518780909 learned_addr learned my addr 172.21.15.19:0/3518780909 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.922+0000 7f3c2ea00640 1 --2- 172.21.15.19:0/3518780909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c30155550 0x7f3c30155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.922+0000 7f3c2e000640 1 -- 172.21.15.19:0/3518780909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c30155550 msgr2=0x7f3c30155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:58.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.922+0000 7f3c2e000640 1 --2- 172.21.15.19:0/3518780909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c30155550 0x7f3c30155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.922+0000 7f3c2e000640 1 -- 172.21.15.19:0/3518780909 >> v1:172.21.15.188:6789/0 conn(0x7f3c300d07c0 legacy=0x7f3c300aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:58.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.922+0000 7f3c2e000640 1 -- 172.21.15.19:0/3518780909 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c300b4670 con 0x7f3c3015f800 2024-03-28T13:12:58.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.924+0000 7f3c2e000640 1 --2- 172.21.15.19:0/3518780909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c3015f800 0x7f3c300d0280 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3c20009e50 tx=0x7f3c2002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85fde833a971b5c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:58.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.924+0000 7f3c2d600640 1 -- 172.21.15.19:0/3518780909 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2002ca70 con 0x7f3c3015f800 2024-03-28T13:12:58.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.924+0000 7f3c2d600640 1 -- 172.21.15.19:0/3518780909 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c2002cc10 con 0x7f3c3015f800 2024-03-28T13:12:58.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.924+0000 7f3c2d600640 1 -- 172.21.15.19:0/3518780909 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c200343f0 con 0x7f3c3015f800 2024-03-28T13:12:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.925+0000 7f3c35c00640 1 -- 172.21.15.19:0/3518780909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c3015f800 msgr2=0x7f3c300d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.925+0000 7f3c35c00640 1 --2- 172.21.15.19:0/3518780909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c3015f800 0x7f3c300d0280 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3c20009e50 tx=0x7f3c2002bed0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.925+0000 7f3c35c00640 1 -- 172.21.15.19:0/3518780909 shutdown_connections 2024-03-28T13:12:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.925+0000 7f3c35c00640 1 --2- 172.21.15.19:0/3518780909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c3015f800 0x7f3c300d0280 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.925+0000 7f3c35c00640 1 --2- 172.21.15.19:0/3518780909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c30155550 0x7f3c30155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.925+0000 7f3c35c00640 1 -- 172.21.15.19:0/3518780909 >> 172.21.15.19:0/3518780909 conn(0x7f3c3006b750 msgr2=0x7f3c3006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.925+0000 7f3c35c00640 1 -- 172.21.15.19:0/3518780909 shutdown_connections 2024-03-28T13:12:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.925+0000 7f3c35c00640 1 -- 172.21.15.19:0/3518780909 wait complete. 2024-03-28T13:12:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.927+0000 7f3c35c00640 1 Processor -- start 2024-03-28T13:12:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.927+0000 7f3c35c00640 1 -- start start 2024-03-28T13:12:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.928+0000 7f3c35c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c300d07c0 0x7f3c3015cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.928+0000 7f3c35c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c30155550 0x7f3c301421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.928+0000 7f3c35c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c3015f800 0x7f3c300af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.928+0000 7f3c35c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3015fc80 con 0x7f3c3015f800 2024-03-28T13:12:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.928+0000 7f3c35c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3c300b4c30 con 0x7f3c300d07c0 2024-03-28T13:12:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.928+0000 7f3c35c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3c3015ba20 con 0x7f3c30155550 2024-03-28T13:12:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.928+0000 7f3c2f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c3015f800 0x7f3c300af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.928+0000 7f3c2f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c3015f800 0x7f3c300af7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33862/0 (socket says 172.21.15.19:33862) 2024-03-28T13:12:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.928+0000 7f3c2f400640 1 -- 172.21.15.19:0/4128802912 learned_addr learned my addr 172.21.15.19:0/4128802912 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:58.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.929+0000 7f3c2f400640 1 -- 172.21.15.19:0/4128802912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c30155550 msgr2=0x7f3c301421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:58.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.929+0000 7f3c2f400640 1 --2- 172.21.15.19:0/4128802912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c30155550 0x7f3c301421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.929+0000 7f3c2f400640 1 -- 172.21.15.19:0/4128802912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c300d07c0 msgr2=0x7f3c3015cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:58.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.929+0000 7f3c2f400640 1 --2- 172.21.15.19:0/4128802912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c300d07c0 0x7f3c3015cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:58.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.929+0000 7f3c2f400640 1 -- 172.21.15.19:0/4128802912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c300af0a0 con 0x7f3c3015f800 2024-03-28T13:12:58.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.929+0000 7f3c2f400640 1 --2- 172.21.15.19:0/4128802912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c3015f800 0x7f3c300af7a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800b440 tx=0x7f3c2800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:58.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.929+0000 7f3c26a00640 1 -- 172.21.15.19:0/4128802912 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28015020 con 0x7f3c3015f800 2024-03-28T13:12:58.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.929+0000 7f3c26a00640 1 -- 172.21.15.19:0/4128802912 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c28011830 con 0x7f3c3015f800 2024-03-28T13:12:58.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.929+0000 7f3c35c00640 1 -- 172.21.15.19:0/4128802912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c30154fe0 con 0x7f3c3015f800 2024-03-28T13:12:58.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.929+0000 7f3c26a00640 1 -- 172.21.15.19:0/4128802912 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c28011c20 con 0x7f3c3015f800 2024-03-28T13:12:58.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.930+0000 7f3c35c00640 1 -- 172.21.15.19:0/4128802912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c30168c50 con 0x7f3c3015f800 2024-03-28T13:12:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.931+0000 7f3c35c00640 1 -- 172.21.15.19:0/4128802912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c300685c0 con 0x7f3c3015f800 2024-03-28T13:12:58.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.932+0000 7f3c26a00640 1 -- 172.21.15.19:0/4128802912 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3c28018020 con 0x7f3c3015f800 2024-03-28T13:12:58.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.933+0000 7f3c26a00640 1 --2- 172.21.15.19:0/4128802912 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c0c06c530 0x7f3c0c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:58.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.933+0000 7f3c26a00640 1 -- 172.21.15.19:0/4128802912 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f3c2808b290 con 0x7f3c3015f800 2024-03-28T13:12:58.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.934+0000 7f3c2ea00640 1 --2- 172.21.15.19:0/4128802912 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c0c06c530 0x7f3c0c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:58.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.935+0000 7f3c2ea00640 1 --2- 172.21.15.19:0/4128802912 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c0c06c530 0x7f3c0c06e9f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3c18006fd0 tx=0x7f3c18008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:58.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:58.937+0000 7f3c26a00640 1 -- 172.21.15.19:0/4128802912 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3c2801f030 con 0x7f3c3015f800 2024-03-28T13:12:59.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.216+0000 7f3c35c00640 1 -- 172.21.15.19:0/4128802912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3c300b3ad0 con 0x7f3c3015f800 2024-03-28T13:12:59.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.221+0000 7f3c26a00640 1 -- 172.21.15.19:0/4128802912 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f3c280172c0 con 0x7f3c3015f800 2024-03-28T13:12:59.221 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-03-28T13:12:59.221 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q== 2024-03-28T13:12:59.221 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-03-28T13:12:59.221 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-03-28T13:12:59.221 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-28T13:12:59.222 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-03-28T13:12:59.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.224+0000 7f3c35c00640 1 -- 172.21.15.19:0/4128802912 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c0c06c530 msgr2=0x7f3c0c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:59.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.224+0000 7f3c35c00640 1 --2- 172.21.15.19:0/4128802912 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c0c06c530 0x7f3c0c06e9f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3c18006fd0 tx=0x7f3c18008040 comp rx=0 tx=0).stop 2024-03-28T13:12:59.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.224+0000 7f3c35c00640 1 -- 172.21.15.19:0/4128802912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c3015f800 msgr2=0x7f3c300af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:59.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.224+0000 7f3c35c00640 1 --2- 172.21.15.19:0/4128802912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c3015f800 0x7f3c300af7a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800b440 tx=0x7f3c2800b900 comp rx=0 tx=0).stop 2024-03-28T13:12:59.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.225+0000 7f3c35c00640 1 -- 172.21.15.19:0/4128802912 shutdown_connections 2024-03-28T13:12:59.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.225+0000 7f3c35c00640 1 --2- 172.21.15.19:0/4128802912 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c0c06c530 0x7f3c0c06e9f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:59.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.225+0000 7f3c35c00640 1 --2- 172.21.15.19:0/4128802912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c3015f800 0x7f3c300af7a0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:59.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.225+0000 7f3c35c00640 1 --2- 172.21.15.19:0/4128802912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c30155550 0x7f3c301421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:59.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.225+0000 7f3c35c00640 1 --2- 172.21.15.19:0/4128802912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c300d07c0 0x7f3c3015cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:59.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.225+0000 7f3c35c00640 1 -- 172.21.15.19:0/4128802912 >> 172.21.15.19:0/4128802912 conn(0x7f3c3006b750 msgr2=0x7f3c3015d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:59.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.225+0000 7f3c35c00640 1 -- 172.21.15.19:0/4128802912 shutdown_connections 2024-03-28T13:12:59.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.225+0000 7f3c35c00640 1 -- 172.21.15.19:0/4128802912 wait complete. 2024-03-28T13:12:59.253 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:12:59.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.464+0000 7f8744000640 1 Processor -- start 2024-03-28T13:12:59.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.464+0000 7f8744000640 1 -- start start 2024-03-28T13:12:59.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.465+0000 7f8744000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f873c0af830 0x7f873c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:59.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.465+0000 7f8744000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f873c0d0650 0x7f873c0d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:59.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.465+0000 7f8744000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f873c15fe80 con 0x7f873c15c390 2024-03-28T13:12:59.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.465+0000 7f8744000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f873c0d1000 con 0x7f873c0af830 2024-03-28T13:12:59.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.465+0000 7f8744000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f873c15c770 con 0x7f873c0d0650 2024-03-28T13:12:59.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.466+0000 7f8740c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f873c0af830 0x7f873c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:59.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.466+0000 7f8740c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f873c0af830 0x7f873c073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:50926/0 (socket says 172.21.15.19:50926) 2024-03-28T13:12:59.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.466+0000 7f8740c00640 1 -- 172.21.15.19:0/3807883155 learned_addr learned my addr 172.21.15.19:0/3807883155 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:59.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.466+0000 7f8742000640 1 --2- 172.21.15.19:0/3807883155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f873c0d0650 0x7f873c0d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:59.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.467+0000 7f873be00640 1 -- 172.21.15.19:0/3807883155 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2359917431 0 0) 0x7f873c15fe80 con 0x7f873c15c390 2024-03-28T13:12:59.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.467+0000 7f873be00640 1 -- 172.21.15.19:0/3807883155 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8724003520 con 0x7f873c15c390 2024-03-28T13:12:59.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.467+0000 7f8742000640 1 -- 172.21.15.19:0/3807883155 >> v2:172.21.15.188:3300/0 conn(0x7f873c0af830 msgr2=0x7f873c073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:12:59.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.467+0000 7f8742000640 1 --2- 172.21.15.19:0/3807883155 >> v2:172.21.15.188:3300/0 conn(0x7f873c0af830 0x7f873c073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:59.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.467+0000 7f8742000640 1 -- 172.21.15.19:0/3807883155 >> v1:172.21.15.188:6789/0 conn(0x7f873c15c390 legacy=0x7f873c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:59.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.467+0000 7f8742000640 1 -- 172.21.15.19:0/3807883155 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873c0ad120 con 0x7f873c0d0650 2024-03-28T13:12:59.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.467+0000 7f8740c00640 1 --2- 172.21.15.19:0/3807883155 >> v2:172.21.15.188:3300/0 conn(0x7f873c0af830 0x7f873c073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:12:59.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.468+0000 7f8742000640 1 --2- 172.21.15.19:0/3807883155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f873c0d0650 0x7f873c0d0ac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f87340097a0 tx=0x7f873402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54fc971552550a7d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:59.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.469+0000 7f873be00640 1 -- 172.21.15.19:0/3807883155 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734035020 con 0x7f873c0d0650 2024-03-28T13:12:59.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.469+0000 7f873be00640 1 -- 172.21.15.19:0/3807883155 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8734031810 con 0x7f873c0d0650 2024-03-28T13:12:59.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.469+0000 7f873be00640 1 -- 172.21.15.19:0/3807883155 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8734031bc0 con 0x7f873c0d0650 2024-03-28T13:12:59.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.470+0000 7f8744000640 1 -- 172.21.15.19:0/3807883155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f873c0d0650 msgr2=0x7f873c0d0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:59.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.470+0000 7f8744000640 1 --2- 172.21.15.19:0/3807883155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f873c0d0650 0x7f873c0d0ac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f87340097a0 tx=0x7f873402b8b0 comp rx=0 tx=0).stop 2024-03-28T13:12:59.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.470+0000 7f8744000640 1 -- 172.21.15.19:0/3807883155 shutdown_connections 2024-03-28T13:12:59.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.470+0000 7f8744000640 1 --2- 172.21.15.19:0/3807883155 >> v2:172.21.15.188:3300/0 conn(0x7f873c0af830 0x7f873c073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:59.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.470+0000 7f8744000640 1 --2- 172.21.15.19:0/3807883155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f873c0d0650 0x7f873c0d0ac0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:59.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.470+0000 7f8744000640 1 -- 172.21.15.19:0/3807883155 >> 172.21.15.19:0/3807883155 conn(0x7f873c06b990 msgr2=0x7f873c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:59.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.470+0000 7f8744000640 1 -- 172.21.15.19:0/3807883155 shutdown_connections 2024-03-28T13:12:59.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.470+0000 7f8744000640 1 -- 172.21.15.19:0/3807883155 wait complete. 2024-03-28T13:12:59.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.471+0000 7f8744000640 1 Processor -- start 2024-03-28T13:12:59.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.472+0000 7f8744000640 1 -- start start 2024-03-28T13:12:59.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.472+0000 7f8744000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f873c0af830 0x7f873c15d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:59.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.473+0000 7f8744000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f873c0d0650 0x7f873c099590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:59.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.473+0000 7f8744000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f873c15c390 0x7f873c097a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:59.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.473+0000 7f8744000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f873c160120 con 0x7f873c0af830 2024-03-28T13:12:59.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.473+0000 7f8744000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f873c0ad6e0 con 0x7f873c0d0650 2024-03-28T13:12:59.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.473+0000 7f8744000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f873c160d50 con 0x7f873c15c390 2024-03-28T13:12:59.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.473+0000 7f8741600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f873c0af830 0x7f873c15d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:59.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.473+0000 7f8741600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f873c0af830 0x7f873c15d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33872/0 (socket says 172.21.15.19:33872) 2024-03-28T13:12:59.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.473+0000 7f8741600640 1 -- 172.21.15.19:0/1414679405 learned_addr learned my addr 172.21.15.19:0/1414679405 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:12:59.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.473+0000 7f8740c00640 1 --2- 172.21.15.19:0/1414679405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f873c0d0650 0x7f873c099590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:59.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.473+0000 7f8741600640 1 -- 172.21.15.19:0/1414679405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f873c15c390 msgr2=0x7f873c097a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:59.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.473+0000 7f8741600640 1 --2- 172.21.15.19:0/1414679405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f873c15c390 0x7f873c097a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:59.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.473+0000 7f8741600640 1 -- 172.21.15.19:0/1414679405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f873c0d0650 msgr2=0x7f873c099590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:59.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.473+0000 7f8741600640 1 --2- 172.21.15.19:0/1414679405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f873c0d0650 0x7f873c099590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:59.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.473+0000 7f8741600640 1 -- 172.21.15.19:0/1414679405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873c0984f0 con 0x7f873c0af830 2024-03-28T13:12:59.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.474+0000 7f8741600640 1 --2- 172.21.15.19:0/1414679405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f873c0af830 0x7f873c15d0c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f873000c7a0 tx=0x7f873000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:59.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.474+0000 7f8739600640 1 -- 172.21.15.19:0/1414679405 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8730003b70 con 0x7f873c0af830 2024-03-28T13:12:59.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.474+0000 7f8739600640 1 -- 172.21.15.19:0/1414679405 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8730003d30 con 0x7f873c0af830 2024-03-28T13:12:59.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.474+0000 7f8739600640 1 -- 172.21.15.19:0/1414679405 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873000aa30 con 0x7f873c0af830 2024-03-28T13:12:59.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.474+0000 7f8744000640 1 -- 172.21.15.19:0/1414679405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873c0987d0 con 0x7f873c0af830 2024-03-28T13:12:59.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.474+0000 7f8744000640 1 -- 172.21.15.19:0/1414679405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f873c098d90 con 0x7f873c0af830 2024-03-28T13:12:59.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.475+0000 7f8744000640 1 -- 172.21.15.19:0/1414679405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f873c0685c0 con 0x7f873c0af830 2024-03-28T13:12:59.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.477+0000 7f8739600640 1 -- 172.21.15.19:0/1414679405 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f873000abd0 con 0x7f873c0af830 2024-03-28T13:12:59.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.479+0000 7f8739600640 1 --2- 172.21.15.19:0/1414679405 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f871c06c530 0x7f871c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:12:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.479+0000 7f8739600640 1 -- 172.21.15.19:0/1414679405 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f873008b6b0 con 0x7f873c0af830 2024-03-28T13:12:59.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.479+0000 7f8740c00640 1 --2- 172.21.15.19:0/1414679405 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f871c06c530 0x7f871c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:12:59.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.481+0000 7f8740c00640 1 --2- 172.21.15.19:0/1414679405 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f871c06c530 0x7f871c06e9f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f873c15b570 tx=0x7f8728008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:12:59.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.484+0000 7f8739600640 1 -- 172.21.15.19:0/1414679405 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8730059c30 con 0x7f873c0af830 2024-03-28T13:12:59.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.727+0000 7f8744000640 1 -- 172.21.15.19:0/1414679405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f873c14bb20 con 0x7f873c0af830 2024-03-28T13:12:59.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.727+0000 7f8739600640 1 -- 172.21.15.19:0/1414679405 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5187 (secure 0 0 0) 0x7f8730059530 con 0x7f873c0af830 2024-03-28T13:12:59.728 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:12:59.728 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1607201514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1607201514},{"type":"v1","addr":"172.21.15.188:6837","nonce":1607201514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1206740614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1206740614},{"type":"v1","addr":"172.21.15.188:6835","nonce":1206740614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2018671345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2018671345},{"type":"v1","addr":"172.21.15.19:6834","nonce":2018671345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:12:52.947573+0000","modified":"2024-03-28T13:12:56.032174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3341654519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3341654519},{"type":"v1","addr":"172.21.15.19:6835","nonce":3341654519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":4}]} 2024-03-28T13:12:59.728 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 34 2024-03-28T13:12:59.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.730+0000 7f8744000640 1 -- 172.21.15.19:0/1414679405 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f871c06c530 msgr2=0x7f871c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:59.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.730+0000 7f8744000640 1 --2- 172.21.15.19:0/1414679405 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f871c06c530 0x7f871c06e9f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f873c15b570 tx=0x7f8728008040 comp rx=0 tx=0).stop 2024-03-28T13:12:59.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.730+0000 7f8744000640 1 -- 172.21.15.19:0/1414679405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f873c0af830 msgr2=0x7f873c15d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:12:59.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.730+0000 7f8744000640 1 --2- 172.21.15.19:0/1414679405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f873c0af830 0x7f873c15d0c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f873000c7a0 tx=0x7f873000cc60 comp rx=0 tx=0).stop 2024-03-28T13:12:59.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.730+0000 7f8744000640 1 -- 172.21.15.19:0/1414679405 shutdown_connections 2024-03-28T13:12:59.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.730+0000 7f8744000640 1 --2- 172.21.15.19:0/1414679405 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f871c06c530 0x7f871c06e9f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:59.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.730+0000 7f8744000640 1 --2- 172.21.15.19:0/1414679405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f873c15c390 0x7f873c097a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:59.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.730+0000 7f8744000640 1 --2- 172.21.15.19:0/1414679405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f873c0d0650 0x7f873c099590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:59.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.730+0000 7f8744000640 1 --2- 172.21.15.19:0/1414679405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f873c0af830 0x7f873c15d0c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:12:59.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.730+0000 7f8744000640 1 -- 172.21.15.19:0/1414679405 >> 172.21.15.19:0/1414679405 conn(0x7f873c06b990 msgr2=0x7f873c14f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:12:59.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.730+0000 7f8744000640 1 -- 172.21.15.19:0/1414679405 shutdown_connections 2024-03-28T13:12:59.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:12:59.730+0000 7f8744000640 1 -- 172.21.15.19:0/1414679405 wait complete. 2024-03-28T13:13:00.154 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:12:52.947573+0000', 'modified': '2024-03-28T13:12:56.032174+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5970': {'gid': 5970, 'name': 'a', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6835/3341654519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 3341654519}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 3341654519}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5970, 'qdb_cluster': [5970]} 2024-03-28T13:13:00.154 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T13:13:00.154 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-28T13:13:00.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.378+0000 7f2bdfe00640 1 Processor -- start 2024-03-28T13:13:00.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.378+0000 7f2bdfe00640 1 -- start start 2024-03-28T13:13:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.380+0000 7f2bdfe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2be006b570 0x7f2be006f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.380+0000 7f2bdfe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2be006b950 0x7f2be00b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.380+0000 7f2bdfe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2be015fde0 con 0x7f2be0141900 2024-03-28T13:13:00.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.380+0000 7f2bdfe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2be01609f0 con 0x7f2be006b570 2024-03-28T13:13:00.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.380+0000 7f2bdfe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f2be01600c0 con 0x7f2be006b950 2024-03-28T13:13:00.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.380+0000 7f2bdea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2be006b570 0x7f2be006f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.381+0000 7f2bdea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2be006b570 0x7f2be006f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33878/0 (socket says 172.21.15.19:33878) 2024-03-28T13:13:00.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.381+0000 7f2bdea00640 1 -- 172.21.15.19:0/2584863714 learned_addr learned my addr 172.21.15.19:0/2584863714 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:00.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.381+0000 7f2bde000640 1 --2- 172.21.15.19:0/2584863714 >> v2:172.21.15.188:3300/0 conn(0x7f2be006b950 0x7f2be00b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.381+0000 7f2bdea00640 1 -- 172.21.15.19:0/2584863714 >> v2:172.21.15.188:3300/0 conn(0x7f2be006b950 msgr2=0x7f2be00b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:00.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.381+0000 7f2bdea00640 1 --2- 172.21.15.19:0/2584863714 >> v2:172.21.15.188:3300/0 conn(0x7f2be006b950 0x7f2be00b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.381+0000 7f2bdea00640 1 -- 172.21.15.19:0/2584863714 >> v1:172.21.15.188:6789/0 conn(0x7f2be0141900 legacy=0x7f2be0141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:00.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.381+0000 7f2bdea00640 1 -- 172.21.15.19:0/2584863714 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be01603a0 con 0x7f2be006b570 2024-03-28T13:13:00.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.381+0000 7f2bde000640 1 --2- 172.21.15.19:0/2584863714 >> v2:172.21.15.188:3300/0 conn(0x7f2be006b950 0x7f2be00b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:00.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.382+0000 7f2bdea00640 1 --2- 172.21.15.19:0/2584863714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be006b570 0x7f2be006f970 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009870 tx=0x7f2bd002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e923865e922498f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:00.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.383+0000 7f2bdd600640 1 -- 172.21.15.19:0/2584863714 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0035020 con 0x7f2be006b570 2024-03-28T13:13:00.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.383+0000 7f2bdd600640 1 -- 172.21.15.19:0/2584863714 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd002d040 con 0x7f2be006b570 2024-03-28T13:13:00.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.383+0000 7f2bdd600640 1 -- 172.21.15.19:0/2584863714 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0031a80 con 0x7f2be006b570 2024-03-28T13:13:00.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.383+0000 7f2bdfe00640 1 -- 172.21.15.19:0/2584863714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be006b570 msgr2=0x7f2be006f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:00.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.383+0000 7f2bdfe00640 1 --2- 172.21.15.19:0/2584863714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be006b570 0x7f2be006f970 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009870 tx=0x7f2bd002b900 comp rx=0 tx=0).stop 2024-03-28T13:13:00.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.384+0000 7f2bdfe00640 1 -- 172.21.15.19:0/2584863714 shutdown_connections 2024-03-28T13:13:00.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.384+0000 7f2bdfe00640 1 --2- 172.21.15.19:0/2584863714 >> v2:172.21.15.188:3300/0 conn(0x7f2be006b950 0x7f2be00b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.384+0000 7f2bdfe00640 1 --2- 172.21.15.19:0/2584863714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be006b570 0x7f2be006f970 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.384+0000 7f2bdfe00640 1 -- 172.21.15.19:0/2584863714 >> 172.21.15.19:0/2584863714 conn(0x7f2be00b2c90 msgr2=0x7f2be00b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:00.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.384+0000 7f2bdfe00640 1 -- 172.21.15.19:0/2584863714 shutdown_connections 2024-03-28T13:13:00.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.384+0000 7f2bdfe00640 1 -- 172.21.15.19:0/2584863714 wait complete. 2024-03-28T13:13:00.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.386+0000 7f2bdfe00640 1 Processor -- start 2024-03-28T13:13:00.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.386+0000 7f2bdfe00640 1 -- start start 2024-03-28T13:13:00.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.387+0000 7f2bdfe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2be006b570 0x7f2be01f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.387+0000 7f2bdfe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2be006b950 0x7f2be01e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.387+0000 7f2bdfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be0141900 0x7f2be00cf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.387+0000 7f2bdfe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0070cb0 con 0x7f2be0141900 2024-03-28T13:13:00.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.387+0000 7f2bdfe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2be006bde0 con 0x7f2be006b950 2024-03-28T13:13:00.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.387+0000 7f2bdfe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2be0070e30 con 0x7f2be006b570 2024-03-28T13:13:00.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.388+0000 7f2bdf400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be0141900 0x7f2be00cf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.388+0000 7f2bdf400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be0141900 0x7f2be00cf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33884/0 (socket says 172.21.15.19:33884) 2024-03-28T13:13:00.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.388+0000 7f2bdf400640 1 -- 172.21.15.19:0/1961915917 learned_addr learned my addr 172.21.15.19:0/1961915917 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.388+0000 7f2bde000640 1 --2- 172.21.15.19:0/1961915917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2be006b950 0x7f2be01e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.388+0000 7f2bdea00640 1 --2- 172.21.15.19:0/1961915917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2be006b570 0x7f2be01f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.388+0000 7f2bdf400640 1 -- 172.21.15.19:0/1961915917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2be006b570 msgr2=0x7f2be01f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.388+0000 7f2bdf400640 1 --2- 172.21.15.19:0/1961915917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2be006b570 0x7f2be01f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.388+0000 7f2bdf400640 1 -- 172.21.15.19:0/1961915917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2be006b950 msgr2=0x7f2be01e7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.388+0000 7f2bdf400640 1 --2- 172.21.15.19:0/1961915917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2be006b950 0x7f2be01e7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.388+0000 7f2bdf400640 1 -- 172.21.15.19:0/1961915917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be01e82f0 con 0x7f2be0141900 2024-03-28T13:13:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.388+0000 7f2bdea00640 1 --2- 172.21.15.19:0/1961915917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2be006b570 0x7f2be01f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.389+0000 7f2bdf400640 1 --2- 172.21.15.19:0/1961915917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be0141900 0x7f2be00cf7f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800b970 tx=0x7f2bd800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.389+0000 7f2bd6a00640 1 -- 172.21.15.19:0/1961915917 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8019070 con 0x7f2be0141900 2024-03-28T13:13:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.389+0000 7f2bd6a00640 1 -- 172.21.15.19:0/1961915917 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd800cb40 con 0x7f2be0141900 2024-03-28T13:13:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.389+0000 7f2bd6a00640 1 -- 172.21.15.19:0/1961915917 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8014440 con 0x7f2be0141900 2024-03-28T13:13:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.389+0000 7f2bdfe00640 1 -- 172.21.15.19:0/1961915917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be01e8050 con 0x7f2be0141900 2024-03-28T13:13:00.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.389+0000 7f2bdfe00640 1 -- 172.21.15.19:0/1961915917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be015f400 con 0x7f2be0141900 2024-03-28T13:13:00.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.392+0000 7f2bd6a00640 1 -- 172.21.15.19:0/1961915917 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2bd801c070 con 0x7f2be0141900 2024-03-28T13:13:00.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.392+0000 7f2bd6a00640 1 --2- 172.21.15.19:0/1961915917 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2bc006c530 0x7f2bc006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.393+0000 7f2bd6a00640 1 -- 172.21.15.19:0/1961915917 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f2bd808b280 con 0x7f2be0141900 2024-03-28T13:13:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.393+0000 7f2bdea00640 1 --2- 172.21.15.19:0/1961915917 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2bc006c530 0x7f2bc006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.393+0000 7f2be8800740 1 Processor -- start 2024-03-28T13:13:00.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.394+0000 7f2be8800740 1 -- start start 2024-03-28T13:13:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.394+0000 7f2be8800740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a321d0 0x5653f9a05170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.394+0000 7f2be8800740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a325b0 0x5653f9a34f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.394+0000 7f2be8800740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a056b0 0x5653f9a354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2be8800740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5653f99cc920 con 0x5653f9a056b0 2024-03-28T13:13:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2be8800740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5653f991cfa0 con 0x5653f9a325b0 2024-03-28T13:13:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2be8800740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5653f996ef90 con 0x5653f9a321d0 2024-03-28T13:13:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2bdf400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a056b0 0x5653f9a354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2bdf400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a056b0 0x5653f9a354a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33896/0 (socket says 172.21.15.19:33896) 2024-03-28T13:13:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2bdf400640 1 -- 172.21.15.19:0/950148946 learned_addr learned my addr 172.21.15.19:0/950148946 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2bdea00640 1 --2- 172.21.15.19:0/1961915917 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2bc006c530 0x7f2bc006e9f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0036000 tx=0x7f2bd0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2bde000640 1 --2- 172.21.15.19:0/950148946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a325b0 0x5653f9a34f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2bdea00640 1 --2- 172.21.15.19:0/950148946 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a321d0 0x5653f9a05170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2bdf400640 1 -- 172.21.15.19:0/950148946 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a321d0 msgr2=0x5653f9a05170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2bdf400640 1 --2- 172.21.15.19:0/950148946 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a321d0 0x5653f9a05170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2bdf400640 1 -- 172.21.15.19:0/950148946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a325b0 msgr2=0x5653f9a34f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2bdf400640 1 --2- 172.21.15.19:0/950148946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a325b0 0x5653f9a34f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2bdf400640 1 -- 172.21.15.19:0/950148946 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f96832d0 con 0x5653f9a056b0 2024-03-28T13:13:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2bde000640 1 --2- 172.21.15.19:0/950148946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a325b0 0x5653f9a34f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.395+0000 7f2bdea00640 1 --2- 172.21.15.19:0/950148946 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a321d0 0x5653f9a05170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.396+0000 7f2bdf400640 1 --2- 172.21.15.19:0/950148946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a056b0 0x5653f9a354a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80897e0 tx=0x7f2bd8059a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58b51a2650ed9f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.396+0000 7f2bcb400640 1 -- 172.21.15.19:0/950148946 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd808b0d0 con 0x5653f9a056b0 2024-03-28T13:13:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.396+0000 7f2bcb400640 1 -- 172.21.15.19:0/950148946 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd8061e10 con 0x5653f9a056b0 2024-03-28T13:13:00.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.396+0000 7f2bcb400640 1 -- 172.21.15.19:0/950148946 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd80558a0 con 0x5653f9a056b0 2024-03-28T13:13:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.397+0000 7f2be8800740 1 -- 172.21.15.19:0/950148946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a056b0 msgr2=0x5653f9a354a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.397+0000 7f2be8800740 1 --2- 172.21.15.19:0/950148946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a056b0 0x5653f9a354a0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80897e0 tx=0x7f2bd8059a70 comp rx=0 tx=0).stop 2024-03-28T13:13:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.397+0000 7f2be8800740 1 -- 172.21.15.19:0/950148946 shutdown_connections 2024-03-28T13:13:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.397+0000 7f2be8800740 1 --2- 172.21.15.19:0/950148946 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a056b0 0x5653f9a354a0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.397+0000 7f2be8800740 1 --2- 172.21.15.19:0/950148946 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a325b0 0x5653f9a34f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.397+0000 7f2be8800740 1 --2- 172.21.15.19:0/950148946 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a321d0 0x5653f9a05170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.397+0000 7f2be8800740 1 -- 172.21.15.19:0/950148946 >> 172.21.15.19:0/950148946 conn(0x5653f9a073e0 msgr2=0x5653f9a4ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.397+0000 7f2be8800740 1 -- 172.21.15.19:0/950148946 shutdown_connections 2024-03-28T13:13:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.397+0000 7f2be8800740 1 -- 172.21.15.19:0/950148946 wait complete. 2024-03-28T13:13:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.398+0000 7f2be8800740 1 -- start start 2024-03-28T13:13:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.398+0000 7f2be8800740 1 Processor -- start 2024-03-28T13:13:00.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.399+0000 7f2be8800740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a321d0 0x5653f9a33e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.399+0000 7f2be8800740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a325b0 0x5653f9a34340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.399+0000 7f2be8800740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 0x5653f9a39890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.399+0000 7f2be8800740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5653f996af40 con 0x5653f9a325b0 2024-03-28T13:13:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.399+0000 7f2be8800740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5653f9917650 con 0x5653f9a38a60 2024-03-28T13:13:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.399+0000 7f2be8800740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5653f992aa10 con 0x5653f9a321d0 2024-03-28T13:13:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.399+0000 7f2be8800740 10 client.0 _ll_register_callbacks cb 0x5653f9969ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:13:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.399+0000 7f2bde000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a325b0 0x5653f9a34340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.399+0000 7f2bde000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a325b0 0x5653f9a34340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33912/0 (socket says 172.21.15.19:33912) 2024-03-28T13:13:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.399+0000 7f2bde000640 1 -- 172.21.15.19:0/3597364391 learned_addr learned my addr 172.21.15.19:0/3597364391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.399+0000 7f2bdea00640 1 --2- 172.21.15.19:0/3597364391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a321d0 0x5653f9a33e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.399+0000 7f2bdf400640 1 --2- 172.21.15.19:0/3597364391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 0x5653f9a39890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.400+0000 7f2bde000640 1 -- 172.21.15.19:0/3597364391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a321d0 msgr2=0x5653f9a33e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.400+0000 7f2bde000640 1 --2- 172.21.15.19:0/3597364391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a321d0 0x5653f9a33e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.400+0000 7f2bde000640 1 -- 172.21.15.19:0/3597364391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 msgr2=0x5653f9a39890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.400+0000 7f2bde000640 1 --2- 172.21.15.19:0/3597364391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 0x5653f9a39890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.400+0000 7f2bde000640 1 -- 172.21.15.19:0/3597364391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f974d6b0 con 0x5653f9a325b0 2024-03-28T13:13:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.400+0000 7f2bdf400640 1 --2- 172.21.15.19:0/3597364391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 0x5653f9a39890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.400+0000 7f2bde000640 1 --2- 172.21.15.19:0/3597364391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a325b0 0x5653f9a34340 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc010580 tx=0x7f2bcc010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbead16c7b0ddccf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.400+0000 7f2bbfe00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:13:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.400+0000 7f2bbfe00640 1 -- 172.21.15.19:0/3597364391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc004020 con 0x5653f9a325b0 2024-03-28T13:13:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.400+0000 7f2bbfe00640 1 -- 172.21.15.19:0/3597364391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f2bcc0041c0 con 0x5653f9a325b0 2024-03-28T13:13:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.400+0000 7f2be8800740 1 -- 172.21.15.19:0/3597364391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd8061e10 con 0x5653f9a325b0 2024-03-28T13:13:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.400+0000 7f2bbfe00640 1 -- 172.21.15.19:0/3597364391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2bcc023050 con 0x5653f9a325b0 2024-03-28T13:13:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.401+0000 7f2be8800740 10 client.6096 fetch_fsmap learned FSMap version 34 2024-03-28T13:13:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.401+0000 7f2be8800740 1 -- 172.21.15.19:0/3597364391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5653f9a1d2e0 con 0x5653f9a325b0 2024-03-28T13:13:00.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.401+0000 7f2bbfe00640 1 -- 172.21.15.19:0/3597364391 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd8061e10 con 0x5653f9a325b0 2024-03-28T13:13:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.401+0000 7f2bbfe00640 1 -- 172.21.15.19:0/3597364391 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f2bcc026020 con 0x5653f9a325b0 2024-03-28T13:13:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.401+0000 7f2be8800740 10 client.6096 fetch_fsmap finished waiting for FSMap version 34 2024-03-28T13:13:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.401+0000 7f2be8800740 10 client.6096 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:13:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.401+0000 7f2be8800740 20 client.6096 populate_metadata read hostname 'smithi019' 2024-03-28T13:13:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.401+0000 7f2be8800740 1 --2- 172.21.15.19:0/3597364391 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x5653f9a1e690 0x5653f9a1f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.401+0000 7f2be8800740 4 client.6096 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:13:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.401+0000 7f2be8800740 1 -- 172.21.15.19:0/3597364391 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653f997c4d0 con 0x5653f9a1e690 2024-03-28T13:13:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.401+0000 7f2bdea00640 1 --2- 172.21.15.19:0/3597364391 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x5653f9a1e690 0x5653f9a1f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.402+0000 7f2bdea00640 1 --2- 172.21.15.19:0/3597364391 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x5653f9a1e690 0x5653f9a1f7e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a2ed0da26689b95 server_cookie=18d82126c770e569 in_seq=0 out_seq=0 2024-03-28T13:13:00.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.402+0000 7f2bbfe00640 10 client.6096 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.403+0000 7f2bbfe00640 1 -- 172.21.15.19:0/3597364391 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653f997c4d0 con 0x5653f9a1e690 2024-03-28T13:13:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.403+0000 7f2bbfe00640 10 client.6096 handle_command_reply: tid=0 2024-03-28T13:13:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.403+0000 7f2be8800740 1 client.6096 shutdown 2024-03-28T13:13:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.403+0000 7f2be8800740 1 -- 172.21.15.19:0/3597364391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a325b0 msgr2=0x5653f9a34340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.403+0000 7f2be8800740 1 --2- 172.21.15.19:0/3597364391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a325b0 0x5653f9a34340 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc010580 tx=0x7f2bcc010a40 comp rx=0 tx=0).stop 2024-03-28T13:13:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.403+0000 7f2be8800740 1 -- 172.21.15.19:0/3597364391 shutdown_connections 2024-03-28T13:13:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.403+0000 7f2be8800740 1 --2- 172.21.15.19:0/3597364391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 0x5653f9a39890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.403+0000 7f2be8800740 1 --2- 172.21.15.19:0/3597364391 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x5653f9a1e690 0x5653f9a1f7e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.403+0000 7f2be8800740 1 --2- 172.21.15.19:0/3597364391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a325b0 0x5653f9a34340 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.404+0000 7f2be8800740 1 --2- 172.21.15.19:0/3597364391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a321d0 0x5653f9a33e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.404+0000 7f2bbfe00640 0 client.6096 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.404+0000 7f2be8800740 1 -- 172.21.15.19:0/3597364391 >> 172.21.15.19:0/3597364391 conn(0x5653f9a056b0 msgr2=0x5653f9a04890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.404+0000 7f2be8800740 1 -- 172.21.15.19:0/3597364391 shutdown_connections 2024-03-28T13:13:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.404+0000 7f2be8800740 1 -- 172.21.15.19:0/3597364391 wait complete. 2024-03-28T13:13:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.404+0000 7f2be8800740 20 client.6096 trim_cache size 0 max 16384 2024-03-28T13:13:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.424+0000 7f2be8800740 1 Processor -- start 2024-03-28T13:13:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.424+0000 7f2be8800740 1 -- start start 2024-03-28T13:13:00.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.425+0000 7f2be8800740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a321d0 0x5653f9a35010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.425+0000 7f2be8800740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a325b0 0x5653f9a61f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.425+0000 7f2be8800740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 0x5653f9a624a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.425+0000 7f2be8800740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2be015e420 con 0x5653f9a321d0 2024-03-28T13:13:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.425+0000 7f2be8800740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2be015d780 con 0x5653f9a38a60 2024-03-28T13:13:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.425+0000 7f2be8800740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2be015ddd0 con 0x5653f9a325b0 2024-03-28T13:13:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.425+0000 7f2bdea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a321d0 0x5653f9a35010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.425+0000 7f2bdea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a321d0 0x5653f9a35010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33920/0 (socket says 172.21.15.19:33920) 2024-03-28T13:13:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.425+0000 7f2bdea00640 1 -- 172.21.15.19:0/2439584327 learned_addr learned my addr 172.21.15.19:0/2439584327 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.425+0000 7f2bdf400640 1 --2- 172.21.15.19:0/2439584327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 0x5653f9a624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.425+0000 7f2bde000640 1 --2- 172.21.15.19:0/2439584327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a325b0 0x5653f9a61f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.426+0000 7f2bdea00640 1 -- 172.21.15.19:0/2439584327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a325b0 msgr2=0x5653f9a61f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.426+0000 7f2bdea00640 1 --2- 172.21.15.19:0/2439584327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a325b0 0x5653f9a61f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.426+0000 7f2bdea00640 1 -- 172.21.15.19:0/2439584327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 msgr2=0x5653f9a624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.426+0000 7f2bdea00640 1 --2- 172.21.15.19:0/2439584327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 0x5653f9a624a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.426+0000 7f2bdea00640 1 -- 172.21.15.19:0/2439584327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f9a39c00 con 0x5653f9a321d0 2024-03-28T13:13:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.426+0000 7f2bdea00640 1 --2- 172.21.15.19:0/2439584327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a321d0 0x5653f9a35010 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00090b0 tx=0x7f2bd0030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4cc290dbc08ceb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.426+0000 7f2bcaa00640 1 -- 172.21.15.19:0/2439584327 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0035040 con 0x5653f9a321d0 2024-03-28T13:13:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.426+0000 7f2bcaa00640 1 -- 172.21.15.19:0/2439584327 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd00044d0 con 0x5653f9a321d0 2024-03-28T13:13:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.426+0000 7f2bcaa00640 1 -- 172.21.15.19:0/2439584327 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0003be0 con 0x5653f9a321d0 2024-03-28T13:13:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.427+0000 7f2be8800740 1 -- 172.21.15.19:0/2439584327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a321d0 msgr2=0x5653f9a35010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.427+0000 7f2be8800740 1 --2- 172.21.15.19:0/2439584327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a321d0 0x5653f9a35010 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00090b0 tx=0x7f2bd0030000 comp rx=0 tx=0).stop 2024-03-28T13:13:00.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.427+0000 7f2be8800740 1 -- 172.21.15.19:0/2439584327 shutdown_connections 2024-03-28T13:13:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.427+0000 7f2be8800740 1 --2- 172.21.15.19:0/2439584327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 0x5653f9a624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.427+0000 7f2be8800740 1 --2- 172.21.15.19:0/2439584327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a325b0 0x5653f9a61f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.427+0000 7f2be8800740 1 --2- 172.21.15.19:0/2439584327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a321d0 0x5653f9a35010 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.427+0000 7f2be8800740 1 -- 172.21.15.19:0/2439584327 >> 172.21.15.19:0/2439584327 conn(0x5653f9a056b0 msgr2=0x5653f9a38f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.427+0000 7f2be8800740 1 -- 172.21.15.19:0/2439584327 shutdown_connections 2024-03-28T13:13:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.427+0000 7f2be8800740 1 -- 172.21.15.19:0/2439584327 wait complete. 2024-03-28T13:13:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.428+0000 7f2be8800740 1 -- start start 2024-03-28T13:13:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.429+0000 7f2be8800740 1 Processor -- start 2024-03-28T13:13:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.430+0000 7f2be8800740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a321d0 0x5653f9a32d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.430+0000 7f2be8800740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a325b0 0x5653f9a33280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.430+0000 7f2be8800740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 0x5653f9a04000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.430+0000 7f2be8800740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5653f9a1e510 con 0x5653f9a321d0 2024-03-28T13:13:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.430+0000 7f2be8800740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5653f9a1d840 con 0x5653f9a38a60 2024-03-28T13:13:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.430+0000 7f2be8800740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5653f9a1eb00 con 0x5653f9a325b0 2024-03-28T13:13:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.430+0000 7f2be8800740 10 client.0 _ll_register_callbacks cb 0x7f2bcc00e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:13:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.431+0000 7f2bdea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a321d0 0x5653f9a32d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.431+0000 7f2bdea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a321d0 0x5653f9a32d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33932/0 (socket says 172.21.15.19:33932) 2024-03-28T13:13:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.431+0000 7f2bdea00640 1 -- 172.21.15.19:0/567540418 learned_addr learned my addr 172.21.15.19:0/567540418 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.431+0000 7f2bdf400640 1 --2- 172.21.15.19:0/567540418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 0x5653f9a04000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.431+0000 7f2bdea00640 1 -- 172.21.15.19:0/567540418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a325b0 msgr2=0x5653f9a33280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.431+0000 7f2bdea00640 1 --2- 172.21.15.19:0/567540418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a325b0 0x5653f9a33280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.431+0000 7f2bdea00640 1 -- 172.21.15.19:0/567540418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 msgr2=0x5653f9a04000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.431+0000 7f2bdea00640 1 --2- 172.21.15.19:0/567540418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 0x5653f9a04000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.431+0000 7f2bdea00640 1 -- 172.21.15.19:0/567540418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f9a0aa90 con 0x5653f9a321d0 2024-03-28T13:13:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.431+0000 7f2bdf400640 1 --2- 172.21.15.19:0/567540418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 0x5653f9a04000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.431+0000 7f2bdea00640 1 --2- 172.21.15.19:0/567540418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a321d0 0x5653f9a32d40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009990 tx=0x7f2bd0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307bc1a2d7dd1726 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.431+0000 7f2bbfe00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:13:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.432+0000 7f2bbfe00640 1 -- 172.21.15.19:0/567540418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0056020 con 0x5653f9a321d0 2024-03-28T13:13:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.432+0000 7f2bbfe00640 1 -- 172.21.15.19:0/567540418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f2bd00522d0 con 0x5653f9a321d0 2024-03-28T13:13:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.432+0000 7f2be8800740 1 -- 172.21.15.19:0/567540418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd00044d0 con 0x5653f9a321d0 2024-03-28T13:13:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.432+0000 7f2bbfe00640 1 -- 172.21.15.19:0/567540418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2bd0035070 con 0x5653f9a321d0 2024-03-28T13:13:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.432+0000 7f2be8800740 10 client.6102 fetch_fsmap learned FSMap version 34 2024-03-28T13:13:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.432+0000 7f2be8800740 1 -- 172.21.15.19:0/567540418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5653f9a74030 con 0x5653f9a321d0 2024-03-28T13:13:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.432+0000 7f2bbfe00640 1 -- 172.21.15.19:0/567540418 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bd00044d0 con 0x5653f9a321d0 2024-03-28T13:13:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.432+0000 7f2bbfe00640 1 -- 172.21.15.19:0/567540418 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f2bd0052510 con 0x5653f9a321d0 2024-03-28T13:13:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.432+0000 7f2be8800740 10 client.6102 fetch_fsmap finished waiting for FSMap version 34 2024-03-28T13:13:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.432+0000 7f2be8800740 10 client.6102 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:13:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.432+0000 7f2be8800740 20 client.6102 populate_metadata read hostname 'smithi019' 2024-03-28T13:13:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.432+0000 7f2be8800740 1 --2- 172.21.15.19:0/567540418 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x5653f9a72f50 0x5653f9a73330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.433+0000 7f2be8800740 4 client.6102 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T13:13:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.433+0000 7f2be8800740 1 -- 172.21.15.19:0/567540418 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "status"}) -- 0x5653f9976820 con 0x5653f9a72f50 2024-03-28T13:13:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.433+0000 7f2bde000640 1 --2- 172.21.15.19:0/567540418 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x5653f9a72f50 0x5653f9a73330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.433+0000 7f2bde000640 1 --2- 172.21.15.19:0/567540418 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x5653f9a72f50 0x5653f9a73330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=442c62e96b731ee0 server_cookie=5b7ce3e25ab82b6a in_seq=0 out_seq=0 2024-03-28T13:13:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.433+0000 7f2bbfe00640 10 client.6102 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.434+0000 7f2bbfe00640 1 -- 172.21.15.19:0/567540418 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x5653f9976820 con 0x5653f9a72f50 2024-03-28T13:13:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.434+0000 7f2bbfe00640 10 client.6102 handle_command_reply: tid=0 2024-03-28T13:13:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.434+0000 7f2be8800740 1 client.6102 shutdown 2024-03-28T13:13:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.434+0000 7f2be8800740 1 -- 172.21.15.19:0/567540418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a321d0 msgr2=0x5653f9a32d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.434+0000 7f2be8800740 1 --2- 172.21.15.19:0/567540418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a321d0 0x5653f9a32d40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009990 tx=0x7f2bd0004730 comp rx=0 tx=0).stop 2024-03-28T13:13:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.434+0000 7f2be8800740 1 -- 172.21.15.19:0/567540418 shutdown_connections 2024-03-28T13:13:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.434+0000 7f2be8800740 1 --2- 172.21.15.19:0/567540418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5653f9a38a60 0x5653f9a04000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.434+0000 7f2be8800740 1 --2- 172.21.15.19:0/567540418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5653f9a325b0 0x5653f9a33280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.434+0000 7f2be8800740 1 --2- 172.21.15.19:0/567540418 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x5653f9a72f50 0x5653f9a73330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.434+0000 7f2be8800740 1 --2- 172.21.15.19:0/567540418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5653f9a321d0 0x5653f9a32d40 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.434+0000 7f2be8800740 1 -- 172.21.15.19:0/567540418 >> 172.21.15.19:0/567540418 conn(0x5653f9a056b0 msgr2=0x5653f9a35010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.434+0000 7f2bbfe00640 0 client.6102 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.434+0000 7f2be8800740 1 -- 172.21.15.19:0/567540418 shutdown_connections 2024-03-28T13:13:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.434+0000 7f2be8800740 1 -- 172.21.15.19:0/567540418 wait complete. 2024-03-28T13:13:00.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.434+0000 7f2be8800740 20 client.6102 trim_cache size 0 max 16384 2024-03-28T13:13:00.437 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:13:00.437 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "e6b569d9-f2ce-49dc-bbe7-c0ab33080203", 2024-03-28T13:13:00.437 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-03-28T13:13:00.437 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 5970, 2024-03-28T13:13:00.437 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-03-28T13:13:00.437 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-03-28T13:13:00.438 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-03-28T13:13:00.438 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 6.930099427, 2024-03-28T13:13:00.438 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 34, 2024-03-28T13:13:00.438 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 63, 2024-03-28T13:13:00.438 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 63, 2024-03-28T13:13:00.438 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 13.327070748000001 2024-03-28T13:13:00.438 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:13:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.435+0000 7f2bdfe00640 1 -- 172.21.15.19:0/1961915917 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2bc006c530 msgr2=0x7f2bc006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.435+0000 7f2bdfe00640 1 --2- 172.21.15.19:0/1961915917 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2bc006c530 0x7f2bc006e9f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0036000 tx=0x7f2bd0008030 comp rx=0 tx=0).stop 2024-03-28T13:13:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.435+0000 7f2bdfe00640 1 -- 172.21.15.19:0/1961915917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be0141900 msgr2=0x7f2be00cf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.435+0000 7f2bdfe00640 1 --2- 172.21.15.19:0/1961915917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be0141900 0x7f2be00cf7f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800b970 tx=0x7f2bd800be30 comp rx=0 tx=0).stop 2024-03-28T13:13:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.435+0000 7f2bdfe00640 1 -- 172.21.15.19:0/1961915917 shutdown_connections 2024-03-28T13:13:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.435+0000 7f2bdfe00640 1 --2- 172.21.15.19:0/1961915917 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2bc006c530 0x7f2bc006e9f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.435+0000 7f2bdfe00640 1 --2- 172.21.15.19:0/1961915917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be0141900 0x7f2be00cf7f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.435+0000 7f2bdfe00640 1 --2- 172.21.15.19:0/1961915917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2be006b950 0x7f2be01e7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.435+0000 7f2bdfe00640 1 --2- 172.21.15.19:0/1961915917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2be006b570 0x7f2be01f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.435+0000 7f2bdfe00640 1 -- 172.21.15.19:0/1961915917 >> 172.21.15.19:0/1961915917 conn(0x7f2be00b2c90 msgr2=0x7f2be0073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:00.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.435+0000 7f2bdfe00640 1 -- 172.21.15.19:0/1961915917 shutdown_connections 2024-03-28T13:13:00.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.435+0000 7f2bdfe00640 1 -- 172.21.15.19:0/1961915917 wait complete. 2024-03-28T13:13:00.462 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:13:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.672+0000 7f07acc00640 1 Processor -- start 2024-03-28T13:13:00.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.672+0000 7f07acc00640 1 -- start start 2024-03-28T13:13:00.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.674+0000 7f07acc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f07a813f300 0x7f07a806e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.674+0000 7f07acc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f07a81419e0 0x7f07a815fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.674+0000 7f07acc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07a8142880 con 0x7f07a8141e40 2024-03-28T13:13:00.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.674+0000 7f07acc00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f07a80cee90 con 0x7f07a813f300 2024-03-28T13:13:00.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.674+0000 7f07acc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f07a81422c0 con 0x7f07a81419e0 2024-03-28T13:13:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.674+0000 7f07a7400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f07a813f300 0x7f07a806e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.674+0000 7f07a7400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f07a813f300 0x7f07a806e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33934/0 (socket says 172.21.15.19:33934) 2024-03-28T13:13:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.674+0000 7f07a7400640 1 -- 172.21.15.19:0/2700190417 learned_addr learned my addr 172.21.15.19:0/2700190417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.675+0000 7f07a7400640 1 -- 172.21.15.19:0/2700190417 >> v2:172.21.15.188:3300/0 conn(0x7f07a81419e0 msgr2=0x7f07a815fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.675+0000 7f07a7400640 1 --2- 172.21.15.19:0/2700190417 >> v2:172.21.15.188:3300/0 conn(0x7f07a81419e0 0x7f07a815fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.675+0000 7f07a7400640 1 -- 172.21.15.19:0/2700190417 >> v1:172.21.15.188:6789/0 conn(0x7f07a8141e40 legacy=0x7f07a815c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:00.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.675+0000 7f07a7400640 1 -- 172.21.15.19:0/2700190417 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a81425a0 con 0x7f07a813f300 2024-03-28T13:13:00.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.677+0000 7f07a7400640 1 --2- 172.21.15.19:0/2700190417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a813f300 0x7f07a806e030 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f07a80cf010 tx=0x7f079802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213e10a05e7353a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:00.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.677+0000 7f07a6000640 1 -- 172.21.15.19:0/2700190417 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079802ca70 con 0x7f07a813f300 2024-03-28T13:13:00.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.677+0000 7f07a6000640 1 -- 172.21.15.19:0/2700190417 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079802cc10 con 0x7f07a813f300 2024-03-28T13:13:00.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.677+0000 7f07a6000640 1 -- 172.21.15.19:0/2700190417 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07980343f0 con 0x7f07a813f300 2024-03-28T13:13:00.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.678+0000 7f07acc00640 1 -- 172.21.15.19:0/2700190417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a813f300 msgr2=0x7f07a806e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:00.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.678+0000 7f07acc00640 1 --2- 172.21.15.19:0/2700190417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a813f300 0x7f07a806e030 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f07a80cf010 tx=0x7f079802bed0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.678+0000 7f07acc00640 1 -- 172.21.15.19:0/2700190417 shutdown_connections 2024-03-28T13:13:00.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.678+0000 7f07acc00640 1 --2- 172.21.15.19:0/2700190417 >> v2:172.21.15.188:3300/0 conn(0x7f07a81419e0 0x7f07a815fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.678+0000 7f07acc00640 1 --2- 172.21.15.19:0/2700190417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a813f300 0x7f07a806e030 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.678+0000 7f07acc00640 1 -- 172.21.15.19:0/2700190417 >> 172.21.15.19:0/2700190417 conn(0x7f07a806b570 msgr2=0x7f07a806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:00.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.678+0000 7f07acc00640 1 -- 172.21.15.19:0/2700190417 shutdown_connections 2024-03-28T13:13:00.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.678+0000 7f07acc00640 1 -- 172.21.15.19:0/2700190417 wait complete. 2024-03-28T13:13:00.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.680+0000 7f07acc00640 1 Processor -- start 2024-03-28T13:13:00.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.681+0000 7f07acc00640 1 -- start start 2024-03-28T13:13:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.682+0000 7f07acc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07a813f300 0x7f07a80b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.682+0000 7f07acc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a81419e0 0x7f07a81e8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.682+0000 7f07acc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07a8141e40 0x7f07a813aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.682+0000 7f07acc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07a80d04b0 con 0x7f07a81419e0 2024-03-28T13:13:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.682+0000 7f07acc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07a80d0330 con 0x7f07a813f300 2024-03-28T13:13:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.682+0000 7f07acc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f07a80d0630 con 0x7f07a8141e40 2024-03-28T13:13:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.682+0000 7f07a6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a81419e0 0x7f07a81e8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.682+0000 7f07a6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a81419e0 0x7f07a81e8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33946/0 (socket says 172.21.15.19:33946) 2024-03-28T13:13:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.682+0000 7f07a6a00640 1 -- 172.21.15.19:0/959391792 learned_addr learned my addr 172.21.15.19:0/959391792 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.683+0000 7f07a7400640 1 --2- 172.21.15.19:0/959391792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07a813f300 0x7f07a80b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.683+0000 7f07a7e00640 1 --2- 172.21.15.19:0/959391792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07a8141e40 0x7f07a813aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.683+0000 7f07a6a00640 1 -- 172.21.15.19:0/959391792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07a8141e40 msgr2=0x7f07a813aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.683+0000 7f07a6a00640 1 --2- 172.21.15.19:0/959391792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07a8141e40 0x7f07a813aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.683+0000 7f07a6a00640 1 -- 172.21.15.19:0/959391792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07a813f300 msgr2=0x7f07a80b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:00.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.683+0000 7f07a6a00640 1 --2- 172.21.15.19:0/959391792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07a813f300 0x7f07a80b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.683+0000 7f07a6a00640 1 -- 172.21.15.19:0/959391792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a80b3910 con 0x7f07a81419e0 2024-03-28T13:13:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.683+0000 7f07a6a00640 1 --2- 172.21.15.19:0/959391792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a81419e0 0x7f07a81e8720 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f079000b450 tx=0x7f079000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.684+0000 7f079f400640 1 -- 172.21.15.19:0/959391792 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790015020 con 0x7f07a81419e0 2024-03-28T13:13:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.684+0000 7f079f400640 1 -- 172.21.15.19:0/959391792 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0790011890 con 0x7f07a81419e0 2024-03-28T13:13:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.684+0000 7f079f400640 1 -- 172.21.15.19:0/959391792 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790011c80 con 0x7f07a81419e0 2024-03-28T13:13:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.684+0000 7f07acc00640 1 -- 172.21.15.19:0/959391792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a813f6e0 con 0x7f07a81419e0 2024-03-28T13:13:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.684+0000 7f07acc00640 1 -- 172.21.15.19:0/959391792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a813fca0 con 0x7f07a81419e0 2024-03-28T13:13:00.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.685+0000 7f07acc00640 1 -- 172.21.15.19:0/959391792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a8075940 con 0x7f07a81419e0 2024-03-28T13:13:00.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.687+0000 7f079f400640 1 -- 172.21.15.19:0/959391792 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0790018020 con 0x7f07a81419e0 2024-03-28T13:13:00.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.688+0000 7f079f400640 1 --2- 172.21.15.19:0/959391792 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f078406c530 0x7f078406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:00.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.688+0000 7f079f400640 1 -- 172.21.15.19:0/959391792 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f079008b8c0 con 0x7f07a81419e0 2024-03-28T13:13:00.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.689+0000 7f07a7400640 1 --2- 172.21.15.19:0/959391792 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f078406c530 0x7f078406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:00.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.690+0000 7f07a7400640 1 --2- 172.21.15.19:0/959391792 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f078406c530 0x7f078406e9f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0798035000 tx=0x7f0798030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:00.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.694+0000 7f079f400640 1 -- 172.21.15.19:0/959391792 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f079001f020 con 0x7f07a81419e0 2024-03-28T13:13:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.936+0000 7f07acc00640 1 -- 172.21.15.19:0/959391792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f07a8141070 con 0x7f07a81419e0 2024-03-28T13:13:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.937+0000 7f079f400640 1 -- 172.21.15.19:0/959391792 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) ==== 53+0+73 (secure 0 0 0) 0x7f079000d040 con 0x7f07a81419e0 2024-03-28T13:13:00.936 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:13:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.939+0000 7f07acc00640 1 -- 172.21.15.19:0/959391792 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f078406c530 msgr2=0x7f078406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.939+0000 7f07acc00640 1 --2- 172.21.15.19:0/959391792 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f078406c530 0x7f078406e9f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0798035000 tx=0x7f0798030000 comp rx=0 tx=0).stop 2024-03-28T13:13:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.939+0000 7f07acc00640 1 -- 172.21.15.19:0/959391792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a81419e0 msgr2=0x7f07a81e8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:00.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.939+0000 7f07acc00640 1 --2- 172.21.15.19:0/959391792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a81419e0 0x7f07a81e8720 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f079000b450 tx=0x7f079000b910 comp rx=0 tx=0).stop 2024-03-28T13:13:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.939+0000 7f07acc00640 1 -- 172.21.15.19:0/959391792 shutdown_connections 2024-03-28T13:13:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.939+0000 7f07acc00640 1 --2- 172.21.15.19:0/959391792 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f078406c530 0x7f078406e9f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.939+0000 7f07acc00640 1 --2- 172.21.15.19:0/959391792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07a8141e40 0x7f07a813aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.939+0000 7f07acc00640 1 --2- 172.21.15.19:0/959391792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a81419e0 0x7f07a81e8720 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.939+0000 7f07acc00640 1 --2- 172.21.15.19:0/959391792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07a813f300 0x7f07a80b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.939+0000 7f07acc00640 1 -- 172.21.15.19:0/959391792 >> 172.21.15.19:0/959391792 conn(0x7f07a806b570 msgr2=0x7f07a8073110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.939+0000 7f07acc00640 1 -- 172.21.15.19:0/959391792 shutdown_connections 2024-03-28T13:13:00.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:00.939+0000 7f07acc00640 1 -- 172.21.15.19:0/959391792 wait complete. 2024-03-28T13:13:00.959 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-28T13:13:00.959 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:13:00.959 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:13:00.959 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:13:00.959 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:13:00.960 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-28T13:13:00.960 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-28T13:13:00.960 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:13:00.960 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:13:00.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:13:00.960 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:13:01.007 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:13:01.007 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:13:01.008 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:13:01.008 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:13:01.008 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:13:01.008 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:13:01.008 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:13:01.008 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:13:01.008 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:13:01.008 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:13:01.008 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:13:01.008 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:13:01.008 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:13:01.008 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:13:01.008 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:13:01.008 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:13:01.008 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42374sec preferred_lft 42374sec 2024-03-28T13:13:01.009 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:13:01.009 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:13:01.009 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:13:01.009 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:13:01.009 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:13:01.009 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:13:01.009 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:13:01.009 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:13:01.009 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:13:01.009 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:13:01.009 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:13:01.009 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:13:01.009 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:13:01.009 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:13:01.010 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:13:01.010 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:13:01.010 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether e6:17:c5:8e:2b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:13:01.010 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::e417:c5ff:fe8e:2bcf/64 scope link 2024-03-28T13:13:01.010 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:13:01.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:13:01.011 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:13:01.100 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:13:01.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:13:01.105 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:13:01.160 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:13:01.160 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:13:01.160 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:13:01.160 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:13:01.160 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:13:01.160 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:13:01.160 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:13:01.160 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:13:01.160 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:13:01.161 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:13:01.161 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:13:01.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:13:01.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:13:01.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:13:01.409 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:13:01.457 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:13:01.457 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:13:01.457 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:13:01.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:13:01.458 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:13:01.553 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:13:01.553 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:13:01.606 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:13:01.606 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:13:01.606 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:13:01.650 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:13:01.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:13:01.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:13:01.830 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:13:01.830 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:13:01.830 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:13:01.852 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:13:01.852 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:13:01.852 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:13:01.852 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:13:01.852 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:13:01.876 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:13:01.877 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:13:01.877 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:13:01.877 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:13:01.877 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:13:01.877 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:13:01.877 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:13:01.877 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:13:01.878 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:13:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:13:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:13:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:13:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:13:01.883 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:13:01.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:13:01.883 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:13:01.933 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:13:01.940 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:13:01.986 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:13:02.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:13:02.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:13:02.106 INFO:teuthology.orchestra.run.smithi019.stdout:client6111 2024-03-28T13:13:02.106 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:13:02.106 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:13:02.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:13:02.201 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-03-28T13:13:02.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:13:02.256 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-03-28T13:13:02.307 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:13:02.307 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:13:02.307 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:13:02.307 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:13:02.307 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-03-28T13:13:02.307 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:13:02.447 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627777 2024-03-28T13:13:02.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:13:02.461 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:13:02.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.723+0000 7f0c54e00640 1 Processor -- start 2024-03-28T13:13:02.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.723+0000 7f0c54e00640 1 -- start start 2024-03-28T13:13:02.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.725+0000 7f0c54e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c5015e4a0 0x7f0c500b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:02.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.725+0000 7f0c54e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0c500d2310 0x7f0c500ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:02.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.725+0000 7f0c54e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c5015e880 con 0x7f0c500cf530 2024-03-28T13:13:02.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.725+0000 7f0c54e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f0c500d34e0 con 0x7f0c500d2310 2024-03-28T13:13:02.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.725+0000 7f0c54e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0c500d28d0 con 0x7f0c5015e4a0 2024-03-28T13:13:02.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.725+0000 7f0c4fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0c500cf530 0x7f0c500d3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52184/0 (socket says 172.21.15.19:52184) 2024-03-28T13:13:02.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.725+0000 7f0c4fe00640 1 -- 172.21.15.19:0/2448959670 learned_addr learned my addr 172.21.15.19:0/2448959670 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:13:02.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.726+0000 7f0c4f400640 1 --2- 172.21.15.19:0/2448959670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c5015e4a0 0x7f0c500b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:02.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.726+0000 7f0c4ea00640 1 -- 172.21.15.19:0/2448959670 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 334690624 0 0) 0x7f0c5015e880 con 0x7f0c500cf530 2024-03-28T13:13:02.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.726+0000 7f0c4ea00640 1 -- 172.21.15.19:0/2448959670 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c34003520 con 0x7f0c500cf530 2024-03-28T13:13:02.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.726+0000 7f0c47e00640 1 --2- 172.21.15.19:0/2448959670 >> v2:172.21.15.188:3300/0 conn(0x7f0c500d2310 0x7f0c500ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:02.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.726+0000 7f0c4ea00640 1 -- 172.21.15.19:0/2448959670 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2235647157 0 0) 0x7f0c34003520 con 0x7f0c500cf530 2024-03-28T13:13:02.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.726+0000 7f0c4ea00640 1 -- 172.21.15.19:0/2448959670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c5015e4a0 msgr2=0x7f0c500b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:02.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.726+0000 7f0c4ea00640 1 --2- 172.21.15.19:0/2448959670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c5015e4a0 0x7f0c500b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:02.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.726+0000 7f0c4ea00640 1 -- 172.21.15.19:0/2448959670 >> v2:172.21.15.188:3300/0 conn(0x7f0c500d2310 msgr2=0x7f0c500ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:02.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.726+0000 7f0c4ea00640 1 --2- 172.21.15.19:0/2448959670 >> v2:172.21.15.188:3300/0 conn(0x7f0c500d2310 0x7f0c500ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:02.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.726+0000 7f0c4ea00640 1 -- 172.21.15.19:0/2448959670 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c500d2bb0 con 0x7f0c500cf530 2024-03-28T13:13:02.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.726+0000 7f0c4ea00640 1 -- 172.21.15.19:0/2448959670 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f0c480019f0 con 0x7f0c500cf530 2024-03-28T13:13:02.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.727+0000 7f0c4ea00640 1 -- 172.21.15.19:0/2448959670 >> v1:172.21.15.19:6789/0 conn(0x7f0c500cf530 legacy=0x7f0c500d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:02.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.727+0000 7f0c4ea00640 1 --2- 172.21.15.19:0/2448959670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c340039b0 0x7f0c34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:02.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.727+0000 7f0c4ea00640 1 --2- 172.21.15.19:0/2448959670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c5015e4a0 0x7f0c34004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:02.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.727+0000 7f0c4ea00640 1 --2- 172.21.15.19:0/2448959670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c34004d40 0x7f0c340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:02.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.727+0000 7f0c4ea00640 1 -- 172.21.15.19:0/2448959670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c500d34e0 con 0x7f0c34004d40 2024-03-28T13:13:02.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.727+0000 7f0c4ea00640 1 -- 172.21.15.19:0/2448959670 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c500d28d0 con 0x7f0c340039b0 2024-03-28T13:13:02.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.727+0000 7f0c4ea00640 1 -- 172.21.15.19:0/2448959670 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3400b9c0 con 0x7f0c5015e4a0 2024-03-28T13:13:02.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.727+0000 7f0c4fe00640 1 --2- 172.21.15.19:0/2448959670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c34004d40 0x7f0c340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:02.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.727+0000 7f0c4f400640 1 --2- 172.21.15.19:0/2448959670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c340039b0 0x7f0c34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:02.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.727+0000 7f0c47e00640 1 --2- 172.21.15.19:0/2448959670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c5015e4a0 0x7f0c34004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:02.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.727+0000 7f0c4fe00640 1 -- 172.21.15.19:0/2448959670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c5015e4a0 msgr2=0x7f0c34004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:02.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.727+0000 7f0c4fe00640 1 --2- 172.21.15.19:0/2448959670 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c5015e4a0 0x7f0c34004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:02.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.727+0000 7f0c4fe00640 1 -- 172.21.15.19:0/2448959670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c340039b0 msgr2=0x7f0c34003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:02.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.727+0000 7f0c4fe00640 1 --2- 172.21.15.19:0/2448959670 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c340039b0 0x7f0c34003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:02.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.727+0000 7f0c4fe00640 1 -- 172.21.15.19:0/2448959670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c34003520 con 0x7f0c34004d40 2024-03-28T13:13:02.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.729+0000 7f0c4fe00640 1 --2- 172.21.15.19:0/2448959670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c34004d40 0x7f0c340051c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800a880 tx=0x7f0c34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7b645cec0955f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:02.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.729+0000 7f0c4ea00640 1 -- 172.21.15.19:0/2448959670 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4800a4a0 con 0x7f0c34004d40 2024-03-28T13:13:02.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.729+0000 7f0c4ea00640 1 -- 172.21.15.19:0/2448959670 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c48034650 con 0x7f0c34004d40 2024-03-28T13:13:02.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.729+0000 7f0c4ea00640 1 -- 172.21.15.19:0/2448959670 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c48034a40 con 0x7f0c34004d40 2024-03-28T13:13:02.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.729+0000 7f0c54e00640 1 -- 172.21.15.19:0/2448959670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c34004d40 msgr2=0x7f0c340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:02.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.729+0000 7f0c54e00640 1 --2- 172.21.15.19:0/2448959670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c34004d40 0x7f0c340051c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800a880 tx=0x7f0c34005830 comp rx=0 tx=0).stop 2024-03-28T13:13:02.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.729+0000 7f0c4fe00640 1 -- 172.21.15.19:0/2448959670 reap_dead start 2024-03-28T13:13:02.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.729+0000 7f0c54e00640 1 -- 172.21.15.19:0/2448959670 shutdown_connections 2024-03-28T13:13:02.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.729+0000 7f0c54e00640 1 -- 172.21.15.19:0/2448959670 >> 172.21.15.19:0/2448959670 conn(0x7f0c500676d0 msgr2=0x7f0c50067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:02.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.730+0000 7f0c54e00640 1 -- 172.21.15.19:0/2448959670 shutdown_connections 2024-03-28T13:13:02.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.730+0000 7f0c54e00640 1 -- 172.21.15.19:0/2448959670 wait complete. 2024-03-28T13:13:02.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.731+0000 7f0c54e00640 1 Processor -- start 2024-03-28T13:13:02.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.732+0000 7f0c54e00640 1 -- start start 2024-03-28T13:13:02.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.732+0000 7f0c54e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c34004d40 0x7f0c500ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:02.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.732+0000 7f0c54e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c50150b50 0x7f0c50153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:02.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.733+0000 7f0c54e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c50155250 0x7f0c501556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:02.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.733+0000 7f0c54e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c500d3970 con 0x7f0c50150b50 2024-03-28T13:13:02.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.733+0000 7f0c54e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c500d3170 con 0x7f0c50155250 2024-03-28T13:13:02.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.733+0000 7f0c54e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0c500b83f0 con 0x7f0c34004d40 2024-03-28T13:13:02.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.733+0000 7f0c47e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c50150b50 0x7f0c50153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:02.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.733+0000 7f0c47e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c50150b50 0x7f0c50153650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33962/0 (socket says 172.21.15.19:33962) 2024-03-28T13:13:02.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.733+0000 7f0c47e00640 1 -- 172.21.15.19:0/2633193526 learned_addr learned my addr 172.21.15.19:0/2633193526 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:02.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.733+0000 7f0c4f400640 1 --2- 172.21.15.19:0/2633193526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c34004d40 0x7f0c500ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:02.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.733+0000 7f0c47e00640 1 -- 172.21.15.19:0/2633193526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c34004d40 msgr2=0x7f0c500ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:02.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.733+0000 7f0c47e00640 1 --2- 172.21.15.19:0/2633193526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c34004d40 0x7f0c500ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:02.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.733+0000 7f0c47e00640 1 -- 172.21.15.19:0/2633193526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c50155250 msgr2=0x7f0c501556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:02.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.733+0000 7f0c47e00640 1 --2- 172.21.15.19:0/2633193526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c50155250 0x7f0c501556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:02.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.733+0000 7f0c47e00640 1 -- 172.21.15.19:0/2633193526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c500ca6b0 con 0x7f0c50150b50 2024-03-28T13:13:02.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.733+0000 7f0c4f400640 1 --2- 172.21.15.19:0/2633193526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c34004d40 0x7f0c500ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:02.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.733+0000 7f0c47e00640 1 --2- 172.21.15.19:0/2633193526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c50150b50 0x7f0c50153650 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0c38011730 tx=0x7f0c38011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:02.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.734+0000 7f0c47400640 1 -- 172.21.15.19:0/2633193526 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3800f8e0 con 0x7f0c50150b50 2024-03-28T13:13:02.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.734+0000 7f0c47400640 1 -- 172.21.15.19:0/2633193526 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3800fa80 con 0x7f0c50150b50 2024-03-28T13:13:02.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.734+0000 7f0c47400640 1 -- 172.21.15.19:0/2633193526 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3800fe30 con 0x7f0c50150b50 2024-03-28T13:13:02.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.734+0000 7f0c54e00640 1 -- 172.21.15.19:0/2633193526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5015f270 con 0x7f0c50150b50 2024-03-28T13:13:02.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.734+0000 7f0c54e00640 1 -- 172.21.15.19:0/2633193526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c50154180 con 0x7f0c50150b50 2024-03-28T13:13:02.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.736+0000 7f0c54e00640 1 -- 172.21.15.19:0/2633193526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c50075940 con 0x7f0c50150b50 2024-03-28T13:13:02.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.736+0000 7f0c47400640 1 -- 172.21.15.19:0/2633193526 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0c38015460 con 0x7f0c50150b50 2024-03-28T13:13:02.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.737+0000 7f0c47400640 1 --2- 172.21.15.19:0/2633193526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c2c06c530 0x7f0c2c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:02.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.738+0000 7f0c47400640 1 -- 172.21.15.19:0/2633193526 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f0c3808bc60 con 0x7f0c50150b50 2024-03-28T13:13:02.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.738+0000 7f0c4f400640 1 --2- 172.21.15.19:0/2633193526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c2c06c530 0x7f0c2c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:02.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.739+0000 7f0c4f400640 1 --2- 172.21.15.19:0/2633193526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c2c06c530 0x7f0c2c06e9f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009980 tx=0x7f0c40008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:02.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.743+0000 7f0c47400640 1 -- 172.21.15.19:0/2633193526 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0c500ca6b0 con 0x7f0c50150b50 2024-03-28T13:13:02.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.985+0000 7f0c54e00640 1 -- 172.21.15.19:0/2633193526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c5006cc00 con 0x7f0c50150b50 2024-03-28T13:13:02.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.986+0000 7f0c47400640 1 -- 172.21.15.19:0/2633193526 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5187 (secure 0 0 0) 0x7f0c38019030 con 0x7f0c50150b50 2024-03-28T13:13:02.986 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:13:02.987 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1607201514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1607201514},{"type":"v1","addr":"172.21.15.188:6837","nonce":1607201514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1206740614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1206740614},{"type":"v1","addr":"172.21.15.188:6835","nonce":1206740614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2018671345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2018671345},{"type":"v1","addr":"172.21.15.19:6834","nonce":2018671345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:12:52.947573+0000","modified":"2024-03-28T13:12:56.032174+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6835/3341654519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3341654519},{"type":"v1","addr":"172.21.15.19:6835","nonce":3341654519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":4}]} 2024-03-28T13:13:02.987 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 34 2024-03-28T13:13:02.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.989+0000 7f0c54e00640 1 -- 172.21.15.19:0/2633193526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c2c06c530 msgr2=0x7f0c2c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:02.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.989+0000 7f0c54e00640 1 --2- 172.21.15.19:0/2633193526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c2c06c530 0x7f0c2c06e9f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009980 tx=0x7f0c40008040 comp rx=0 tx=0).stop 2024-03-28T13:13:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.989+0000 7f0c54e00640 1 -- 172.21.15.19:0/2633193526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c50150b50 msgr2=0x7f0c50153650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.989+0000 7f0c54e00640 1 --2- 172.21.15.19:0/2633193526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c50150b50 0x7f0c50153650 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0c38011730 tx=0x7f0c38011bf0 comp rx=0 tx=0).stop 2024-03-28T13:13:02.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.989+0000 7f0c54e00640 1 -- 172.21.15.19:0/2633193526 shutdown_connections 2024-03-28T13:13:02.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.989+0000 7f0c54e00640 1 --2- 172.21.15.19:0/2633193526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0c2c06c530 0x7f0c2c06e9f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:02.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.989+0000 7f0c54e00640 1 --2- 172.21.15.19:0/2633193526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c50155250 0x7f0c501556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:02.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.989+0000 7f0c54e00640 1 --2- 172.21.15.19:0/2633193526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c50150b50 0x7f0c50153650 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:02.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.989+0000 7f0c54e00640 1 --2- 172.21.15.19:0/2633193526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c34004d40 0x7f0c500ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:02.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.989+0000 7f0c54e00640 1 -- 172.21.15.19:0/2633193526 >> 172.21.15.19:0/2633193526 conn(0x7f0c500676d0 msgr2=0x7f0c500d3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:02.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.989+0000 7f0c54e00640 1 -- 172.21.15.19:0/2633193526 shutdown_connections 2024-03-28T13:13:02.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:02.989+0000 7f0c54e00640 1 -- 172.21.15.19:0/2633193526 wait complete. 2024-03-28T13:13:03.010 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-03-28T13:13:03.269 INFO:teuthology.orchestra.run.smithi019.stdout:{"message":"","return_code":0} 2024-03-28T13:13:03.269 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-28T13:13:03.269 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-03-28T13:13:03.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.367+0000 7fac343b7f40 1 Processor -- start 2024-03-28T13:13:03.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.367+0000 7fac343b7f40 1 -- start start 2024-03-28T13:13:03.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.368+0000 7fac343b7f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0f19bdc20 0x55b0f1a43150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.368+0000 7fac343b7f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b0f1a45240 0x55b0f19a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.368+0000 7fac343b7f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b0f1a4bec0 0x55b0f1a40ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.368+0000 7fac343b7f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b0f19748d0 con 0x55b0f19bdc20 2024-03-28T13:13:03.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.368+0000 7fac343b7f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b0f191a2c0 con 0x55b0f1a4bec0 2024-03-28T13:13:03.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.368+0000 7fac343b7f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55b0f1974390 con 0x55b0f1a45240 2024-03-28T13:13:03.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.369+0000 7fac31600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0f19bdc20 0x55b0f1a43150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.369+0000 7fac31600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0f19bdc20 0x55b0f1a43150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33976/0 (socket says 172.21.15.19:33976) 2024-03-28T13:13:03.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.369+0000 7fac31600640 1 -- 172.21.15.19:0/471349906 learned_addr learned my addr 172.21.15.19:0/471349906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:03.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.369+0000 7fac31600640 1 -- 172.21.15.19:0/471349906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b0f1a45240 msgr2=0x55b0f19a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:03.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.369+0000 7fac31600640 1 --2- 172.21.15.19:0/471349906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b0f1a45240 0x55b0f19a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.369+0000 7fac31600640 1 -- 172.21.15.19:0/471349906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b0f1a4bec0 msgr2=0x55b0f1a40ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:03.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.369+0000 7fac31600640 1 --2- 172.21.15.19:0/471349906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b0f1a4bec0 0x55b0f1a40ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.369+0000 7fac31600640 1 -- 172.21.15.19:0/471349906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0f1972ff0 con 0x55b0f19bdc20 2024-03-28T13:13:03.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.369+0000 7fac31600640 1 --2- 172.21.15.19:0/471349906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0f19bdc20 0x55b0f1a43150 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fac2400a590 tx=0x7fac2400aa50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39966a059dbc761b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:03.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.370+0000 7fac2b400640 1 -- 172.21.15.19:0/471349906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2400b980 con 0x55b0f19bdc20 2024-03-28T13:13:03.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.370+0000 7fac2b400640 1 -- 172.21.15.19:0/471349906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac2400f070 con 0x55b0f19bdc20 2024-03-28T13:13:03.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.370+0000 7fac2b400640 1 -- 172.21.15.19:0/471349906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2400bd80 con 0x55b0f19bdc20 2024-03-28T13:13:03.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.370+0000 7fac343b7f40 1 -- 172.21.15.19:0/471349906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0f19bdc20 msgr2=0x55b0f1a43150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.370+0000 7fac343b7f40 1 --2- 172.21.15.19:0/471349906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0f19bdc20 0x55b0f1a43150 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fac2400a590 tx=0x7fac2400aa50 comp rx=0 tx=0).stop 2024-03-28T13:13:03.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.370+0000 7fac343b7f40 1 -- 172.21.15.19:0/471349906 shutdown_connections 2024-03-28T13:13:03.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.370+0000 7fac343b7f40 1 --2- 172.21.15.19:0/471349906 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b0f1a4bec0 0x55b0f1a40ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.370+0000 7fac343b7f40 1 --2- 172.21.15.19:0/471349906 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b0f1a45240 0x55b0f19a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.370+0000 7fac343b7f40 1 --2- 172.21.15.19:0/471349906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0f19bdc20 0x55b0f1a43150 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.370+0000 7fac343b7f40 1 -- 172.21.15.19:0/471349906 >> 172.21.15.19:0/471349906 conn(0x55b0f19c1630 msgr2=0x55b0f1a4b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:03.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.371+0000 7fac343b7f40 1 -- 172.21.15.19:0/471349906 shutdown_connections 2024-03-28T13:13:03.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.371+0000 7fac343b7f40 1 -- 172.21.15.19:0/471349906 wait complete. 2024-03-28T13:13:03.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.371+0000 7fac343b7f40 1 Processor -- start 2024-03-28T13:13:03.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.371+0000 7fac343b7f40 1 -- start start 2024-03-28T13:13:03.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.372+0000 7fac343b7f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0f19bdc20 0x55b0f1a39640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.372+0000 7fac343b7f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b0f1a45240 0x55b0f1a3fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.372+0000 7fac343b7f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b0f1a4bec0 0x55b0f1971920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.372+0000 7fac343b7f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b0f19a5b70 con 0x55b0f19bdc20 2024-03-28T13:13:03.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.372+0000 7fac343b7f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b0f1a3d060 con 0x55b0f1a45240 2024-03-28T13:13:03.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.372+0000 7fac343b7f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55b0f1a3c730 con 0x55b0f1a4bec0 2024-03-28T13:13:03.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.373+0000 7fac31600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0f19bdc20 0x55b0f1a39640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.373+0000 7fac31600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0f19bdc20 0x55b0f1a39640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33990/0 (socket says 172.21.15.19:33990) 2024-03-28T13:13:03.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.373+0000 7fac31600640 1 -- 172.21.15.19:0/3842451113 learned_addr learned my addr 172.21.15.19:0/3842451113 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:03.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.373+0000 7fac30c00640 1 --2- 172.21.15.19:0/3842451113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b0f1a45240 0x55b0f1a3fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.373+0000 7fac31600640 1 -- 172.21.15.19:0/3842451113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b0f1a4bec0 msgr2=0x55b0f1971920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.373+0000 7fac31600640 1 --2- 172.21.15.19:0/3842451113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b0f1a4bec0 0x55b0f1971920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.373+0000 7fac31600640 1 -- 172.21.15.19:0/3842451113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b0f1a45240 msgr2=0x55b0f1a3fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.373+0000 7fac31600640 1 --2- 172.21.15.19:0/3842451113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b0f1a45240 0x55b0f1a3fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.373+0000 7fac31600640 1 -- 172.21.15.19:0/3842451113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0f1a3dc40 con 0x55b0f19bdc20 2024-03-28T13:13:03.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.374+0000 7fac31600640 1 --2- 172.21.15.19:0/3842451113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0f19bdc20 0x55b0f1a39640 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x55b0f19a2c40 tx=0x7fac24002fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:03.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.374+0000 7fac2a000640 1 -- 172.21.15.19:0/3842451113 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24014a80 con 0x55b0f19bdc20 2024-03-28T13:13:03.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.374+0000 7fac2a000640 1 -- 172.21.15.19:0/3842451113 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac24024070 con 0x55b0f19bdc20 2024-03-28T13:13:03.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.374+0000 7fac343b7f40 1 -- 172.21.15.19:0/3842451113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b0f19a10f0 con 0x55b0f19bdc20 2024-03-28T13:13:03.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.374+0000 7fac2a000640 1 -- 172.21.15.19:0/3842451113 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac24008440 con 0x55b0f19bdc20 2024-03-28T13:13:03.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.374+0000 7fac343b7f40 1 -- 172.21.15.19:0/3842451113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0f1a3dec0 con 0x55b0f19bdc20 2024-03-28T13:13:03.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.377+0000 7fac2a000640 1 -- 172.21.15.19:0/3842451113 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fac240085e0 con 0x55b0f19bdc20 2024-03-28T13:13:03.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.378+0000 7fac2a000640 1 --2- 172.21.15.19:0/3842451113 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac1406af40 0x7fac1406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.378+0000 7fac2a000640 1 -- 172.21.15.19:0/3842451113 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fac2408c1d0 con 0x55b0f19bdc20 2024-03-28T13:13:03.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.378+0000 7fac343b7f40 1 --2- 172.21.15.19:0/3842451113 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x55b0f1a403a0 0x55b0f1ae3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.378+0000 7fac343b7f40 1 -- 172.21.15.19:0/3842451113 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63) -- 0x55b0f1ae3fb0 con 0x55b0f1a403a0 2024-03-28T13:13:03.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.379+0000 7fac30c00640 1 --2- 172.21.15.19:0/3842451113 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac1406af40 0x7fac1406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.379+0000 7fac32000640 1 --2- 172.21.15.19:0/3842451113 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x55b0f1a403a0 0x55b0f1ae3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.380+0000 7fac32000640 1 --2- 172.21.15.19:0/3842451113 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x55b0f1a403a0 0x55b0f1ae3a70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:03.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.380+0000 7fac30c00640 1 --2- 172.21.15.19:0/3842451113 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac1406af40 0x7fac1406d400 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fac20054000 tx=0x7fac2004c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.380+0000 7fac32000640 1 -- 172.21.15.19:0/3842451113 <== osd.2 v2:172.21.15.19:6816/1693606238 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7fac2c00b470 con 0x55b0f1a403a0 2024-03-28T13:13:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.380+0000 7fac343b7f40 1 -- 172.21.15.19:0/3842451113 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x55b0f1a403a0 msgr2=0x55b0f1ae3a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.380+0000 7fac343b7f40 1 --2- 172.21.15.19:0/3842451113 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x55b0f1a403a0 0x55b0f1ae3a70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.380+0000 7fac343b7f40 1 -- 172.21.15.19:0/3842451113 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac1406af40 msgr2=0x7fac1406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.380+0000 7fac343b7f40 1 --2- 172.21.15.19:0/3842451113 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac1406af40 0x7fac1406d400 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fac20054000 tx=0x7fac2004c000 comp rx=0 tx=0).stop 2024-03-28T13:13:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.380+0000 7fac343b7f40 1 -- 172.21.15.19:0/3842451113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0f19bdc20 msgr2=0x55b0f1a39640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.380+0000 7fac343b7f40 1 --2- 172.21.15.19:0/3842451113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b0f19bdc20 0x55b0f1a39640 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x55b0f19a2c40 tx=0x7fac24002fd0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.380+0000 7fac32000640 1 -- 172.21.15.19:0/3842451113 reap_dead start 2024-03-28T13:13:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.381+0000 7fac343b7f40 1 -- 172.21.15.19:0/3842451113 shutdown_connections 2024-03-28T13:13:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.381+0000 7fac343b7f40 1 -- 172.21.15.19:0/3842451113 >> 172.21.15.19:0/3842451113 conn(0x55b0f19c1630 msgr2=0x55b0f19a4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.381+0000 7fac343b7f40 1 -- 172.21.15.19:0/3842451113 shutdown_connections 2024-03-28T13:13:03.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.381+0000 7fac343b7f40 1 -- 172.21.15.19:0/3842451113 wait complete. 2024-03-28T13:13:03.385 INFO:teuthology.orchestra.run.smithi019.stdout:Xfiledir_test 2024-03-28T13:13:03.385 DEBUG:teuthology.orchestra.run.smithi019:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-03-28T13:13:03.476 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:13:03.476 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511627777, 2024-03-28T13:13:03.476 INFO:teuthology.orchestra.run.smithi019.stdout: "ancestors": [ 2024-03-28T13:13:03.476 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:13:03.476 INFO:teuthology.orchestra.run.smithi019.stdout: "dirino": 1099511627776, 2024-03-28T13:13:03.476 INFO:teuthology.orchestra.run.smithi019.stdout: "dname": "file", 2024-03-28T13:13:03.476 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 2 2024-03-28T13:13:03.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:13:03.476 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:13:03.476 INFO:teuthology.orchestra.run.smithi019.stdout: "dirino": 1, 2024-03-28T13:13:03.477 INFO:teuthology.orchestra.run.smithi019.stdout: "dname": "dir_test", 2024-03-28T13:13:03.477 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 4 2024-03-28T13:13:03.477 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:13:03.477 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-03-28T13:13:03.477 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 9, 2024-03-28T13:13:03.477 INFO:teuthology.orchestra.run.smithi019.stdout: "old_pools": [] 2024-03-28T13:13:03.477 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:13:03.477 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:13:03.481 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-03-28T13:13:03.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.585+0000 7f7bf5451f40 1 Processor -- start 2024-03-28T13:13:03.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.585+0000 7f7bf5451f40 1 -- start start 2024-03-28T13:13:03.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.586+0000 7f7bf5451f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c11c09d70 0x559c11c948d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.586+0000 7f7bf5451f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c11c95870 0x559c11c98180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.586+0000 7f7bf5451f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c11c95c50 0x559c11c896c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.586+0000 7f7bf5451f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559c11c087d0 con 0x559c11c95c50 2024-03-28T13:13:03.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.586+0000 7f7bf5451f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559c11b672c0 con 0x559c11c09d70 2024-03-28T13:13:03.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.586+0000 7f7bf5451f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x559c11bc48d0 con 0x559c11c95870 2024-03-28T13:13:03.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.586+0000 7f7bf2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c11c95c50 0x559c11c896c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.586+0000 7f7bf2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c11c95c50 0x559c11c896c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34004/0 (socket says 172.21.15.19:34004) 2024-03-28T13:13:03.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.586+0000 7f7bf2a00640 1 -- 172.21.15.19:0/3832607422 learned_addr learned my addr 172.21.15.19:0/3832607422 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:03.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.587+0000 7f7bf2000640 1 --2- 172.21.15.19:0/3832607422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c11c09d70 0x559c11c948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.587+0000 7f7bf2a00640 1 -- 172.21.15.19:0/3832607422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c11c95870 msgr2=0x559c11c98180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:03.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.587+0000 7f7bf2a00640 1 --2- 172.21.15.19:0/3832607422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c11c95870 0x559c11c98180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.587+0000 7f7bf2a00640 1 -- 172.21.15.19:0/3832607422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c11c09d70 msgr2=0x559c11c948d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:03.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.587+0000 7f7bf2a00640 1 --2- 172.21.15.19:0/3832607422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c11c09d70 0x559c11c948d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.587+0000 7f7bf2a00640 1 -- 172.21.15.19:0/3832607422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c11bec1a0 con 0x559c11c95c50 2024-03-28T13:13:03.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.587+0000 7f7bf2a00640 1 --2- 172.21.15.19:0/3832607422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c11c95c50 0x559c11c896c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00c470 tx=0x7f7bec00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9ee9564159d843 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:03.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.587+0000 7f7bf0c00640 1 -- 172.21.15.19:0/3832607422 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec0116b0 con 0x559c11c95c50 2024-03-28T13:13:03.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.587+0000 7f7bf0c00640 1 -- 172.21.15.19:0/3832607422 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bec017070 con 0x559c11c95c50 2024-03-28T13:13:03.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.587+0000 7f7bf0c00640 1 -- 172.21.15.19:0/3832607422 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec011ab0 con 0x559c11c95c50 2024-03-28T13:13:03.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.588+0000 7f7bf5451f40 1 -- 172.21.15.19:0/3832607422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c11c95c50 msgr2=0x559c11c896c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.588+0000 7f7bf5451f40 1 --2- 172.21.15.19:0/3832607422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c11c95c50 0x559c11c896c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00c470 tx=0x7f7bec00c930 comp rx=0 tx=0).stop 2024-03-28T13:13:03.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.588+0000 7f7bf5451f40 1 -- 172.21.15.19:0/3832607422 shutdown_connections 2024-03-28T13:13:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.588+0000 7f7bf5451f40 1 --2- 172.21.15.19:0/3832607422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c11c95c50 0x559c11c896c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.588+0000 7f7bf5451f40 1 --2- 172.21.15.19:0/3832607422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c11c95870 0x559c11c98180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.588+0000 7f7bf5451f40 1 --2- 172.21.15.19:0/3832607422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c11c09d70 0x559c11c948d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.588+0000 7f7bf5451f40 1 -- 172.21.15.19:0/3832607422 >> 172.21.15.19:0/3832607422 conn(0x559c11bebdc0 msgr2=0x559c11beb520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.588+0000 7f7bf5451f40 1 -- 172.21.15.19:0/3832607422 shutdown_connections 2024-03-28T13:13:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.588+0000 7f7bf5451f40 1 -- 172.21.15.19:0/3832607422 wait complete. 2024-03-28T13:13:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.589+0000 7f7bf5451f40 1 Processor -- start 2024-03-28T13:13:03.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.589+0000 7f7bf5451f40 1 -- start start 2024-03-28T13:13:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.590+0000 7f7bf5451f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c11c09d70 0x559c11c86330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.590+0000 7f7bf5451f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c11c95870 0x559c11c99470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.590+0000 7f7bf5451f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c11c95c50 0x559c11c999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.590+0000 7f7bf5451f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559c11bec440 con 0x559c11c95c50 2024-03-28T13:13:03.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.590+0000 7f7bf5451f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559c11c97630 con 0x559c11c95870 2024-03-28T13:13:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.590+0000 7f7bf5451f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x559c11bec720 con 0x559c11c09d70 2024-03-28T13:13:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.590+0000 7f7bf2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c11c95c50 0x559c11c999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.590+0000 7f7bf2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c11c95c50 0x559c11c999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34010/0 (socket says 172.21.15.19:34010) 2024-03-28T13:13:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.590+0000 7f7bf2a00640 1 -- 172.21.15.19:0/1413362985 learned_addr learned my addr 172.21.15.19:0/1413362985 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.591+0000 7f7bebe00640 1 --2- 172.21.15.19:0/1413362985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c11c95870 0x559c11c99470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.591+0000 7f7bf2a00640 1 -- 172.21.15.19:0/1413362985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c11c09d70 msgr2=0x559c11c86330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.591+0000 7f7bf2000640 1 --2- 172.21.15.19:0/1413362985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c11c09d70 0x559c11c86330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.591+0000 7f7bf2a00640 1 --2- 172.21.15.19:0/1413362985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c11c09d70 0x559c11c86330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.591+0000 7f7bf2a00640 1 -- 172.21.15.19:0/1413362985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c11c95870 msgr2=0x559c11c99470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.591+0000 7f7bf2a00640 1 --2- 172.21.15.19:0/1413362985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c11c95870 0x559c11c99470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.591+0000 7f7bf2a00640 1 -- 172.21.15.19:0/1413362985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c11c01cd0 con 0x559c11c95c50 2024-03-28T13:13:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.591+0000 7f7bebe00640 1 --2- 172.21.15.19:0/1413362985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c11c95870 0x559c11c99470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.591+0000 7f7bf2a00640 1 --2- 172.21.15.19:0/1413362985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c11c95c50 0x559c11c999b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7bec005fd0 tx=0x7f7bec0033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.591+0000 7f7beaa00640 1 -- 172.21.15.19:0/1413362985 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec012930 con 0x559c11c95c50 2024-03-28T13:13:03.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.592+0000 7f7beaa00640 1 -- 172.21.15.19:0/1413362985 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bec012ad0 con 0x559c11c95c50 2024-03-28T13:13:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.592+0000 7f7beaa00640 1 -- 172.21.15.19:0/1413362985 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec026430 con 0x559c11c95c50 2024-03-28T13:13:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.592+0000 7f7bf5451f40 1 -- 172.21.15.19:0/1413362985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559c11beb770 con 0x559c11c95c50 2024-03-28T13:13:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.592+0000 7f7bf5451f40 1 -- 172.21.15.19:0/1413362985 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c11c93800 con 0x559c11c95c50 2024-03-28T13:13:03.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.594+0000 7f7beaa00640 1 -- 172.21.15.19:0/1413362985 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7bec02f430 con 0x559c11c95c50 2024-03-28T13:13:03.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.594+0000 7f7beaa00640 1 --2- 172.21.15.19:0/1413362985 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7bd006af10 0x7f7bd006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.595+0000 7f7beaa00640 1 -- 172.21.15.19:0/1413362985 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f7bec00e020 con 0x559c11c95c50 2024-03-28T13:13:03.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.595+0000 7f7bf5451f40 1 --2- 172.21.15.19:0/1413362985 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x559c11bef250 0x559c11d3dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.595+0000 7f7bf5451f40 1 -- 172.21.15.19:0/1413362985 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63) -- 0x559c11c87bf0 con 0x559c11bef250 2024-03-28T13:13:03.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.595+0000 7f7bebe00640 1 --2- 172.21.15.19:0/1413362985 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x559c11bef250 0x559c11d3dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.595+0000 7f7bf2000640 1 --2- 172.21.15.19:0/1413362985 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7bd006af10 0x7f7bd006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.596+0000 7f7bebe00640 1 --2- 172.21.15.19:0/1413362985 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x559c11bef250 0x559c11d3dae0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:03.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.596+0000 7f7bf2000640 1 --2- 172.21.15.19:0/1413362985 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7bd006af10 0x7f7bd006d3d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7be4054000 tx=0x7f7be404e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.598+0000 7f7bebe00640 1 -- 172.21.15.19:0/1413362985 <== osd.2 v2:172.21.15.19:6816/1693606238 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v63'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7bdc0093d0 con 0x559c11bef250 2024-03-28T13:13:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.598+0000 7f7bf5451f40 1 -- 172.21.15.19:0/1413362985 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x559c11bef250 msgr2=0x559c11d3dae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.598+0000 7f7bf5451f40 1 --2- 172.21.15.19:0/1413362985 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x559c11bef250 0x559c11d3dae0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.598+0000 7f7bf5451f40 1 -- 172.21.15.19:0/1413362985 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7bd006af10 msgr2=0x7f7bd006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.598+0000 7f7bf5451f40 1 --2- 172.21.15.19:0/1413362985 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7bd006af10 0x7f7bd006d3d0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7be4054000 tx=0x7f7be404e000 comp rx=0 tx=0).stop 2024-03-28T13:13:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.598+0000 7f7bf5451f40 1 -- 172.21.15.19:0/1413362985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c11c95c50 msgr2=0x559c11c999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.598+0000 7f7bf5451f40 1 --2- 172.21.15.19:0/1413362985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c11c95c50 0x559c11c999b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f7bec005fd0 tx=0x7f7bec0033a0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.598+0000 7f7bf2a00640 1 -- 172.21.15.19:0/1413362985 reap_dead start 2024-03-28T13:13:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.598+0000 7f7bf5451f40 1 -- 172.21.15.19:0/1413362985 shutdown_connections 2024-03-28T13:13:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.598+0000 7f7bf5451f40 1 -- 172.21.15.19:0/1413362985 >> 172.21.15.19:0/1413362985 conn(0x559c11bebdc0 msgr2=0x559c11c07930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.598+0000 7f7bf5451f40 1 -- 172.21.15.19:0/1413362985 shutdown_connections 2024-03-28T13:13:03.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.598+0000 7f7bf5451f40 1 -- 172.21.15.19:0/1413362985 wait complete. 2024-03-28T13:13:03.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:13:03.603 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-03-28T13:13:03.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.813+0000 7f9ee7800640 1 Processor -- start 2024-03-28T13:13:03.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.813+0000 7f9ee7800640 1 -- start start 2024-03-28T13:13:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.816+0000 7f9ee7800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee00b3270 0x7f9ee00d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.816+0000 7f9ee7800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ee006d370 con 0x7f9ee0161f10 2024-03-28T13:13:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.816+0000 7f9ee7800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ee00d0f00 con 0x7f9ee006e110 2024-03-28T13:13:03.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.816+0000 7f9ee7800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee006d650 con 0x7f9ee00b3270 2024-03-28T13:13:03.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.817+0000 7f9ee4e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9ee0161f10 0x7f9ee006da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52200/0 (socket says 172.21.15.19:52200) 2024-03-28T13:13:03.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.817+0000 7f9ee4e00640 1 -- 172.21.15.19:0/3116698693 learned_addr learned my addr 172.21.15.19:0/3116698693 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:13:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.817+0000 7f9ee5800640 1 --2- 172.21.15.19:0/3116698693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee00b3270 0x7f9ee00d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.817+0000 7f9edf400640 1 -- 172.21.15.19:0/3116698693 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2081480687 0 0) 0x7f9ee006d370 con 0x7f9ee0161f10 2024-03-28T13:13:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.817+0000 7f9edf400640 1 -- 172.21.15.19:0/3116698693 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ec4003520 con 0x7f9ee0161f10 2024-03-28T13:13:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9edf400640 1 -- 172.21.15.19:0/3116698693 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3111632571 0 0) 0x7f9ec4003520 con 0x7f9ee0161f10 2024-03-28T13:13:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9edf400640 1 -- 172.21.15.19:0/3116698693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee00b3270 msgr2=0x7f9ee00d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9edf400640 1 --2- 172.21.15.19:0/3116698693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee00b3270 0x7f9ee00d19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9edf400640 1 -- 172.21.15.19:0/3116698693 >> v1:172.21.15.188:6789/0 conn(0x7f9ee006e110 legacy=0x7f9ee00d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9edf400640 1 -- 172.21.15.19:0/3116698693 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee00d11e0 con 0x7f9ee0161f10 2024-03-28T13:13:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9edf400640 1 -- 172.21.15.19:0/3116698693 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f9ed00019f0 con 0x7f9ee0161f10 2024-03-28T13:13:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9edf400640 1 -- 172.21.15.19:0/3116698693 >> v1:172.21.15.19:6789/0 conn(0x7f9ee0161f10 legacy=0x7f9ee006da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9edf400640 1 --2- 172.21.15.19:0/3116698693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ec40039e0 0x7f9ec4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9edf400640 1 --2- 172.21.15.19:0/3116698693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee00b3270 0x7f9ec4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9edf400640 1 --2- 172.21.15.19:0/3116698693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ec4004d40 0x7f9ec40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9edf400640 1 -- 172.21.15.19:0/3116698693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee006d650 con 0x7f9ec40039e0 2024-03-28T13:13:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9edf400640 1 -- 172.21.15.19:0/3116698693 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec400ba70 con 0x7f9ec4004d40 2024-03-28T13:13:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9edf400640 1 -- 172.21.15.19:0/3116698693 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9ec400bc70 con 0x7f9ee00b3270 2024-03-28T13:13:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/3116698693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ec40039e0 0x7f9ec4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9ee4e00640 1 -- 172.21.15.19:0/3116698693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee00b3270 msgr2=0x7f9ec4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/3116698693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee00b3270 0x7f9ec4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.818+0000 7f9ee4e00640 1 -- 172.21.15.19:0/3116698693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ec4004d40 msgr2=0x7f9ec40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.819+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/3116698693 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ec4004d40 0x7f9ec40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.819+0000 7f9ee4e00640 1 -- 172.21.15.19:0/3116698693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec4003520 con 0x7f9ec40039e0 2024-03-28T13:13:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.820+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/3116698693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ec40039e0 0x7f9ec4003e00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0002230 tx=0x7f9ec4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8e1d33060845e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.820+0000 7f9edf400640 1 -- 172.21.15.19:0/3116698693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed000a4a0 con 0x7f9ec40039e0 2024-03-28T13:13:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.820+0000 7f9edf400640 1 -- 172.21.15.19:0/3116698693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed0034650 con 0x7f9ec40039e0 2024-03-28T13:13:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.820+0000 7f9edf400640 1 -- 172.21.15.19:0/3116698693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0034a40 con 0x7f9ec40039e0 2024-03-28T13:13:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.821+0000 7f9ee7800640 1 -- 172.21.15.19:0/3116698693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ec40039e0 msgr2=0x7f9ec4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.821+0000 7f9ee7800640 1 --2- 172.21.15.19:0/3116698693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ec40039e0 0x7f9ec4003e00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0002230 tx=0x7f9ec4005830 comp rx=0 tx=0).stop 2024-03-28T13:13:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.821+0000 7f9ee5800640 1 -- 172.21.15.19:0/3116698693 reap_dead start 2024-03-28T13:13:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.821+0000 7f9ee7800640 1 -- 172.21.15.19:0/3116698693 shutdown_connections 2024-03-28T13:13:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.821+0000 7f9ee7800640 1 -- 172.21.15.19:0/3116698693 >> 172.21.15.19:0/3116698693 conn(0x7f9ee008bfb0 msgr2=0x7f9ee00d4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.821+0000 7f9ee7800640 1 -- 172.21.15.19:0/3116698693 shutdown_connections 2024-03-28T13:13:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.821+0000 7f9ee7800640 1 -- 172.21.15.19:0/3116698693 wait complete. 2024-03-28T13:13:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.823+0000 7f9ee7800640 1 Processor -- start 2024-03-28T13:13:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.823+0000 7f9ee7800640 1 -- start start 2024-03-28T13:13:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.824+0000 7f9ee7800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ec40039e0 0x7f9ee00a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.824+0000 7f9ee7800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee015e2f0 0x7f9ee015f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.824+0000 7f9ee7800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee00b8260 0x7f9ee01f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.824+0000 7f9ee7800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee006cff0 con 0x7f9ec40039e0 2024-03-28T13:13:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.824+0000 7f9ee7800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee01622f0 con 0x7f9ee015e2f0 2024-03-28T13:13:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.824+0000 7f9ee7800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee00d19e0 con 0x7f9ee00b8260 2024-03-28T13:13:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.824+0000 7f9ee4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ec40039e0 0x7f9ee00a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.824+0000 7f9ee4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ec40039e0 0x7f9ee00a1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34024/0 (socket says 172.21.15.19:34024) 2024-03-28T13:13:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.824+0000 7f9ee4e00640 1 -- 172.21.15.19:0/1139518054 learned_addr learned my addr 172.21.15.19:0/1139518054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.825+0000 7f9ee5800640 1 --2- 172.21.15.19:0/1139518054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee00b8260 0x7f9ee01f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.825+0000 7f9edfe00640 1 --2- 172.21.15.19:0/1139518054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee015e2f0 0x7f9ee015f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.825+0000 7f9ee4e00640 1 -- 172.21.15.19:0/1139518054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee00b8260 msgr2=0x7f9ee01f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.825+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/1139518054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee00b8260 0x7f9ee01f5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.825+0000 7f9ee4e00640 1 -- 172.21.15.19:0/1139518054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee015e2f0 msgr2=0x7f9ee015f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.825+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/1139518054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee015e2f0 0x7f9ee015f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.825+0000 7f9ee4e00640 1 -- 172.21.15.19:0/1139518054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee00b71c0 con 0x7f9ec40039e0 2024-03-28T13:13:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.825+0000 7f9edfe00640 1 --2- 172.21.15.19:0/1139518054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee015e2f0 0x7f9ee015f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.825+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/1139518054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ec40039e0 0x7f9ee00a1c60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9ed002cc40 tx=0x7f9ed0001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.825+0000 7f9edcc00640 1 -- 172.21.15.19:0/1139518054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0046070 con 0x7f9ec40039e0 2024-03-28T13:13:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.825+0000 7f9edcc00640 1 -- 172.21.15.19:0/1139518054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed003a040 con 0x7f9ec40039e0 2024-03-28T13:13:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.825+0000 7f9ee7800640 1 -- 172.21.15.19:0/1139518054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee00b74a0 con 0x7f9ec40039e0 2024-03-28T13:13:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.825+0000 7f9edcc00640 1 -- 172.21.15.19:0/1139518054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed00019d0 con 0x7f9ec40039e0 2024-03-28T13:13:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.826+0000 7f9ee7800640 1 -- 172.21.15.19:0/1139518054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee00b7a60 con 0x7f9ec40039e0 2024-03-28T13:13:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.828+0000 7f9edcc00640 1 -- 172.21.15.19:0/1139518054 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9ed0049070 con 0x7f9ec40039e0 2024-03-28T13:13:03.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.829+0000 7f9edcc00640 1 --2- 172.21.15.19:0/1139518054 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ebc06c530 0x7f9ebc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.829+0000 7f9edcc00640 1 -- 172.21.15.19:0/1139518054 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f9ed00ad4a0 con 0x7f9ec40039e0 2024-03-28T13:13:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.829+0000 7f9edfe00640 1 --2- 172.21.15.19:0/1139518054 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ebc06c530 0x7f9ebc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.830+0000 7f9eea459740 1 Processor -- start 2024-03-28T13:13:03.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.830+0000 7f9eea459740 1 -- start start 2024-03-28T13:13:03.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.831+0000 7f9edfe00640 1 --2- 172.21.15.19:0/1139518054 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ebc06c530 0x7f9ebc06e9f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00ce6f0 tx=0x7f9ec8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:03.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.831+0000 7f9eea459740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 0x55a67f1bbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.831+0000 7f9eea459740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f1e9320 0x55a67f1ebcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.831+0000 7f9eea459740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1bc420 0x55a67f1ec210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.831+0000 7f9eea459740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a67f125360 con 0x55a67f1bc420 2024-03-28T13:13:03.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.831+0000 7f9eea459740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a67f0cd750 con 0x55a67f1e9320 2024-03-28T13:13:03.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.831+0000 7f9eea459740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55a67f121310 con 0x55a67f1e8f40 2024-03-28T13:13:03.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.831+0000 7f9ee5800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1bc420 0x55a67f1ec210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.831+0000 7f9ee5800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1bc420 0x55a67f1ec210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34032/0 (socket says 172.21.15.19:34032) 2024-03-28T13:13:03.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.831+0000 7f9ee5800640 1 -- 172.21.15.19:0/2329111359 learned_addr learned my addr 172.21.15.19:0/2329111359 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:03.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.832+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/2329111359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 0x55a67f1bbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.832+0000 7f9edfe00640 1 --2- 172.21.15.19:0/2329111359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f1e9320 0x55a67f1ebcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.832+0000 7f9ee5800640 1 -- 172.21.15.19:0/2329111359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 msgr2=0x55a67f1bbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:03.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.832+0000 7f9ee5800640 1 --2- 172.21.15.19:0/2329111359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 0x55a67f1bbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.832+0000 7f9ee5800640 1 -- 172.21.15.19:0/2329111359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f1e9320 msgr2=0x55a67f1ebcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:03.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.832+0000 7f9ee5800640 1 --2- 172.21.15.19:0/2329111359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f1e9320 0x55a67f1ebcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.832+0000 7f9ee5800640 1 -- 172.21.15.19:0/2329111359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a67f0eeda0 con 0x55a67f1bc420 2024-03-28T13:13:03.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.832+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/2329111359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 0x55a67f1bbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:03.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.832+0000 7f9edfe00640 1 --2- 172.21.15.19:0/2329111359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f1e9320 0x55a67f1ebcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:03.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.832+0000 7f9ee5800640 1 --2- 172.21.15.19:0/2329111359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1bc420 0x55a67f1ec210 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9ee008d900 tx=0x7f9ed80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62abd1b371fae432 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:03.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.833+0000 7f9ed5600640 1 -- 172.21.15.19:0/2329111359 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed800f040 con 0x55a67f1bc420 2024-03-28T13:13:03.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.833+0000 7f9ed5600640 1 -- 172.21.15.19:0/2329111359 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed801a070 con 0x55a67f1bc420 2024-03-28T13:13:03.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.833+0000 7f9ed5600640 1 -- 172.21.15.19:0/2329111359 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80106c0 con 0x55a67f1bc420 2024-03-28T13:13:03.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.833+0000 7f9eea459740 1 -- 172.21.15.19:0/2329111359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1bc420 msgr2=0x55a67f1ec210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.833+0000 7f9eea459740 1 --2- 172.21.15.19:0/2329111359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1bc420 0x55a67f1ec210 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9ee008d900 tx=0x7f9ed80046c0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.833+0000 7f9eea459740 1 -- 172.21.15.19:0/2329111359 shutdown_connections 2024-03-28T13:13:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.833+0000 7f9eea459740 1 --2- 172.21.15.19:0/2329111359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1bc420 0x55a67f1ec210 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.833+0000 7f9eea459740 1 --2- 172.21.15.19:0/2329111359 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f1e9320 0x55a67f1ebcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.833+0000 7f9eea459740 1 --2- 172.21.15.19:0/2329111359 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 0x55a67f1bbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.833+0000 7f9eea459740 1 -- 172.21.15.19:0/2329111359 >> 172.21.15.19:0/2329111359 conn(0x55a67f1be150 msgr2=0x55a67f206cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.833+0000 7f9eea459740 1 -- 172.21.15.19:0/2329111359 shutdown_connections 2024-03-28T13:13:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.834+0000 7f9eea459740 1 -- 172.21.15.19:0/2329111359 wait complete. 2024-03-28T13:13:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.834+0000 7f9eea459740 1 -- start start 2024-03-28T13:13:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.834+0000 7f9eea459740 1 Processor -- start 2024-03-28T13:13:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.835+0000 7f9eea459740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 0x55a67f219da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.835+0000 7f9eea459740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e9320 0x55a67f1eded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.835+0000 7f9eea459740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f206ea0 0x55a67f207320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.835+0000 7f9eea459740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a67f1c07f0 con 0x55a67f1e9320 2024-03-28T13:13:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.835+0000 7f9eea459740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a67f0e0b10 con 0x55a67f206ea0 2024-03-28T13:13:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.835+0000 7f9eea459740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55a67f1c3030 con 0x55a67f1e8f40 2024-03-28T13:13:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.836+0000 7f9eea459740 10 client.0 _ll_register_callbacks cb 0x55a67f11bed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:13:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.836+0000 7f9edfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e9320 0x55a67f1eded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.836+0000 7f9edfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e9320 0x55a67f1eded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34042/0 (socket says 172.21.15.19:34042) 2024-03-28T13:13:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.836+0000 7f9edfe00640 1 -- 172.21.15.19:0/2768650782 learned_addr learned my addr 172.21.15.19:0/2768650782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.836+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/2768650782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 0x55a67f219da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.836+0000 7f9ee5800640 1 --2- 172.21.15.19:0/2768650782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f206ea0 0x55a67f207320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.836+0000 7f9edfe00640 1 -- 172.21.15.19:0/2768650782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 msgr2=0x55a67f219da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.836+0000 7f9edfe00640 1 --2- 172.21.15.19:0/2768650782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 0x55a67f219da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.836+0000 7f9edfe00640 1 -- 172.21.15.19:0/2768650782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f206ea0 msgr2=0x55a67f207320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.836+0000 7f9edfe00640 1 --2- 172.21.15.19:0/2768650782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f206ea0 0x55a67f207320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.836+0000 7f9edfe00640 1 -- 172.21.15.19:0/2768650782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a67ef057b0 con 0x55a67f1e9320 2024-03-28T13:13:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.836+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/2768650782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 0x55a67f219da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.836+0000 7f9edfe00640 1 --2- 172.21.15.19:0/2768650782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e9320 0x55a67f1eded0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800dab0 tx=0x7f9ec8001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cbb255c72397e23 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.836+0000 7f9ece000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:13:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.837+0000 7f9ece000640 1 -- 172.21.15.19:0/2768650782 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8022460 con 0x55a67f1e9320 2024-03-28T13:13:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.837+0000 7f9ece000640 1 -- 172.21.15.19:0/2768650782 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f9ec801f9b0 con 0x55a67f1e9320 2024-03-28T13:13:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.837+0000 7f9eea459740 1 -- 172.21.15.19:0/2768650782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ed801a070 con 0x55a67f1e9320 2024-03-28T13:13:03.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.837+0000 7f9ece000640 1 -- 172.21.15.19:0/2768650782 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9ec8028070 con 0x55a67f1e9320 2024-03-28T13:13:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.837+0000 7f9eea459740 10 client.6147 fetch_fsmap learned FSMap version 34 2024-03-28T13:13:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.837+0000 7f9eea459740 1 -- 172.21.15.19:0/2768650782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55a67ef65f40 con 0x55a67f1e9320 2024-03-28T13:13:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.837+0000 7f9ece000640 1 -- 172.21.15.19:0/2768650782 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ed801a070 con 0x55a67f1e9320 2024-03-28T13:13:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.837+0000 7f9ece000640 1 -- 172.21.15.19:0/2768650782 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f9ec801fbb0 con 0x55a67f1e9320 2024-03-28T13:13:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.837+0000 7f9eea459740 10 client.6147 fetch_fsmap finished waiting for FSMap version 34 2024-03-28T13:13:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.837+0000 7f9eea459740 10 client.6147 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-03-28T13:13:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.838+0000 7f9eea459740 20 client.6147 populate_metadata read hostname 'smithi019' 2024-03-28T13:13:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.838+0000 7f9eea459740 1 --2- 172.21.15.19:0/2768650782 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55a67f205cb0 0x55a67f2060c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.838+0000 7f9eea459740 4 client.6147 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:13:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.838+0000 7f9eea459740 1 -- 172.21.15.19:0/2768650782 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a67f1328a0 con 0x55a67f205cb0 2024-03-28T13:13:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.838+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/2768650782 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55a67f205cb0 0x55a67f2060c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.838+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/2768650782 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55a67f205cb0 0x55a67f2060c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=880396c027302a8f server_cookie=321d567f7fbef60f in_seq=0 out_seq=0 2024-03-28T13:13:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.838+0000 7f9ece000640 10 client.6147 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.839+0000 7f9ece000640 1 -- 172.21.15.19:0/2768650782 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a67f1328a0 con 0x55a67f205cb0 2024-03-28T13:13:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.839+0000 7f9ece000640 10 client.6147 handle_command_reply: tid=0 2024-03-28T13:13:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.839+0000 7f9eea459740 1 client.6147 shutdown 2024-03-28T13:13:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.839+0000 7f9eea459740 1 -- 172.21.15.19:0/2768650782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e9320 msgr2=0x55a67f1eded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.839+0000 7f9eea459740 1 --2- 172.21.15.19:0/2768650782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e9320 0x55a67f1eded0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800dab0 tx=0x7f9ec8001190 comp rx=0 tx=0).stop 2024-03-28T13:13:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.839+0000 7f9eea459740 1 -- 172.21.15.19:0/2768650782 shutdown_connections 2024-03-28T13:13:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.839+0000 7f9eea459740 1 --2- 172.21.15.19:0/2768650782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f206ea0 0x55a67f207320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.839+0000 7f9eea459740 1 --2- 172.21.15.19:0/2768650782 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55a67f205cb0 0x55a67f2060c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.839+0000 7f9eea459740 1 --2- 172.21.15.19:0/2768650782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e9320 0x55a67f1eded0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.839+0000 7f9eea459740 1 --2- 172.21.15.19:0/2768650782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 0x55a67f219da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.840+0000 7f9eea459740 1 -- 172.21.15.19:0/2768650782 >> 172.21.15.19:0/2768650782 conn(0x55a67f1bc420 msgr2=0x55a67f1bfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.840+0000 7f9ece000640 0 client.6147 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.840+0000 7f9eea459740 1 -- 172.21.15.19:0/2768650782 shutdown_connections 2024-03-28T13:13:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.840+0000 7f9eea459740 1 -- 172.21.15.19:0/2768650782 wait complete. 2024-03-28T13:13:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.840+0000 7f9eea459740 20 client.6147 trim_cache size 0 max 16384 2024-03-28T13:13:03.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.859+0000 7f9eea459740 1 Processor -- start 2024-03-28T13:13:03.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.859+0000 7f9eea459740 1 -- start start 2024-03-28T13:13:03.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.860+0000 7f9eea459740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 0x55a67f1ede90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.860+0000 7f9eea459740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e9320 0x55a67f1f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.860+0000 7f9eea459740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f206ea0 0x55a67f1c1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.860+0000 7f9eea459740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee0201410 con 0x55a67f1e9320 2024-03-28T13:13:03.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.860+0000 7f9eea459740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee0200410 con 0x55a67f206ea0 2024-03-28T13:13:03.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.860+0000 7f9eea459740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee01f8420 con 0x55a67f1e8f40 2024-03-28T13:13:03.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.860+0000 7f9edfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e9320 0x55a67f1f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.860+0000 7f9edfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e9320 0x55a67f1f0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34054/0 (socket says 172.21.15.19:34054) 2024-03-28T13:13:03.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.860+0000 7f9edfe00640 1 -- 172.21.15.19:0/1813680660 learned_addr learned my addr 172.21.15.19:0/1813680660 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:03.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.861+0000 7f9edfe00640 1 -- 172.21.15.19:0/1813680660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 msgr2=0x55a67f1ede90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:03.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.861+0000 7f9edfe00640 1 --2- 172.21.15.19:0/1813680660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 0x55a67f1ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.861+0000 7f9edfe00640 1 -- 172.21.15.19:0/1813680660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f206ea0 msgr2=0x55a67f1c1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:03.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.861+0000 7f9edfe00640 1 --2- 172.21.15.19:0/1813680660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f206ea0 0x55a67f1c1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.861+0000 7f9edfe00640 1 -- 172.21.15.19:0/1813680660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a67edf90a0 con 0x55a67f1e9320 2024-03-28T13:13:03.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.861+0000 7f9edfe00640 1 --2- 172.21.15.19:0/1813680660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e9320 0x55a67f1f0ab0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800d670 tx=0x7f9ec8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44690428037cf855 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:03.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.861+0000 7f9ed4c00640 1 -- 172.21.15.19:0/1813680660 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8007030 con 0x55a67f1e9320 2024-03-28T13:13:03.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.861+0000 7f9ed4c00640 1 -- 172.21.15.19:0/1813680660 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec80071d0 con 0x55a67f1e9320 2024-03-28T13:13:03.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.861+0000 7f9ed4c00640 1 -- 172.21.15.19:0/1813680660 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec80075c0 con 0x55a67f1e9320 2024-03-28T13:13:03.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.862+0000 7f9eea459740 1 -- 172.21.15.19:0/1813680660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e9320 msgr2=0x55a67f1f0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.862+0000 7f9eea459740 1 --2- 172.21.15.19:0/1813680660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e9320 0x55a67f1f0ab0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f9ec800d670 tx=0x7f9ec8014000 comp rx=0 tx=0).stop 2024-03-28T13:13:03.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.862+0000 7f9eea459740 1 -- 172.21.15.19:0/1813680660 shutdown_connections 2024-03-28T13:13:03.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.862+0000 7f9eea459740 1 --2- 172.21.15.19:0/1813680660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f206ea0 0x55a67f1c1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.862+0000 7f9eea459740 1 --2- 172.21.15.19:0/1813680660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e9320 0x55a67f1f0ab0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.862+0000 7f9eea459740 1 --2- 172.21.15.19:0/1813680660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f1e8f40 0x55a67f1ede90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.862+0000 7f9eea459740 1 -- 172.21.15.19:0/1813680660 >> 172.21.15.19:0/1813680660 conn(0x55a67f1bc420 msgr2=0x55a67f1eb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:03.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.862+0000 7f9eea459740 1 -- 172.21.15.19:0/1813680660 shutdown_connections 2024-03-28T13:13:03.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.862+0000 7f9eea459740 1 -- 172.21.15.19:0/1813680660 wait complete. 2024-03-28T13:13:03.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.863+0000 7f9eea459740 1 -- start start 2024-03-28T13:13:03.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.863+0000 7f9eea459740 1 Processor -- start 2024-03-28T13:13:03.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.864+0000 7f9eea459740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e8f40 0x55a67f278260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.864+0000 7f9eea459740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f1e9320 0x55a67f1e5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.864+0000 7f9eea459740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f206ea0 0x55a67f1e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.864+0000 7f9eea459740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a67f2084c0 con 0x55a67f1e8f40 2024-03-28T13:13:03.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.864+0000 7f9eea459740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee01ffe20 con 0x55a67f1e9320 2024-03-28T13:13:03.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.864+0000 7f9eea459740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55a67f1e7160 con 0x55a67f206ea0 2024-03-28T13:13:03.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.864+0000 7f9eea459740 10 client.0 _ll_register_callbacks cb 0x7f9ec8001da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:13:03.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.864+0000 7f9ee4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e8f40 0x55a67f278260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.864+0000 7f9ee4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e8f40 0x55a67f278260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34064/0 (socket says 172.21.15.19:34064) 2024-03-28T13:13:03.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.864+0000 7f9ee4e00640 1 -- 172.21.15.19:0/2247790694 learned_addr learned my addr 172.21.15.19:0/2247790694 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:03.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.865+0000 7f9ee4e00640 1 -- 172.21.15.19:0/2247790694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f206ea0 msgr2=0x55a67f1e63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:03.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.865+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/2247790694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f206ea0 0x55a67f1e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.865+0000 7f9ee4e00640 1 -- 172.21.15.19:0/2247790694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f1e9320 msgr2=0x55a67f1e5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:03.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.865+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/2247790694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f1e9320 0x55a67f1e5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.865+0000 7f9ee4e00640 1 -- 172.21.15.19:0/2247790694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a67f1e8000 con 0x55a67f1e8f40 2024-03-28T13:13:03.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.865+0000 7f9ee4e00640 1 --2- 172.21.15.19:0/2247790694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e8f40 0x55a67f278260 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9ed007b0e0 tx=0x7f9ed007b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a13bfcb64dc58ca server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:03.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.865+0000 7f9ece000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:13:03.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.865+0000 7f9ece000640 1 -- 172.21.15.19:0/2247790694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0083fc0 con 0x55a67f1e8f40 2024-03-28T13:13:03.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.865+0000 7f9ece000640 1 -- 172.21.15.19:0/2247790694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f9ed0077a50 con 0x55a67f1e8f40 2024-03-28T13:13:03.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.865+0000 7f9eea459740 1 -- 172.21.15.19:0/2247790694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ec80071d0 con 0x55a67f1e8f40 2024-03-28T13:13:03.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.866+0000 7f9ece000640 1 -- 172.21.15.19:0/2247790694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9ed0083b30 con 0x55a67f1e8f40 2024-03-28T13:13:03.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.866+0000 7f9eea459740 10 client.6153 fetch_fsmap learned FSMap version 34 2024-03-28T13:13:03.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.866+0000 7f9eea459740 1 -- 172.21.15.19:0/2247790694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x55a67f21a170 con 0x55a67f1e8f40 2024-03-28T13:13:03.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.866+0000 7f9ece000640 1 -- 172.21.15.19:0/2247790694 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec80071d0 con 0x55a67f1e8f40 2024-03-28T13:13:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.866+0000 7f9ece000640 1 -- 172.21.15.19:0/2247790694 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f9ed007a830 con 0x55a67f1e8f40 2024-03-28T13:13:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.866+0000 7f9eea459740 10 client.6153 fetch_fsmap finished waiting for FSMap version 34 2024-03-28T13:13:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.866+0000 7f9eea459740 10 client.6153 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-03-28T13:13:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.866+0000 7f9eea459740 20 client.6153 populate_metadata read hostname 'smithi019' 2024-03-28T13:13:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.866+0000 7f9eea459740 1 --2- 172.21.15.19:0/2247790694 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55a67f1bad90 0x55a67f1bb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.866+0000 7f9eea459740 4 client.6153 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-03-28T13:13:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.866+0000 7f9eea459740 1 -- 172.21.15.19:0/2247790694 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x55a67f1ef6b0 con 0x55a67f1bad90 2024-03-28T13:13:03.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.866+0000 7f9edfe00640 1 --2- 172.21.15.19:0/2247790694 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55a67f1bad90 0x55a67f1bb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.867+0000 7f9edfe00640 1 --2- 172.21.15.19:0/2247790694 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55a67f1bad90 0x55a67f1bb170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7987ad3b8edf4a9a server_cookie=dcb6136478917b6f in_seq=0 out_seq=0 2024-03-28T13:13:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.867+0000 7f9ece000640 10 client.6153 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.868+0000 7f9ece000640 1 -- 172.21.15.19:0/2247790694 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a67f1ef6b0 con 0x55a67f1bad90 2024-03-28T13:13:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.868+0000 7f9ece000640 10 client.6153 handle_command_reply: tid=0 2024-03-28T13:13:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.868+0000 7f9eea459740 1 client.6153 shutdown 2024-03-28T13:13:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.868+0000 7f9eea459740 1 -- 172.21.15.19:0/2247790694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e8f40 msgr2=0x55a67f278260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.868+0000 7f9eea459740 1 --2- 172.21.15.19:0/2247790694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e8f40 0x55a67f278260 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9ed007b0e0 tx=0x7f9ed007b4a0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.868+0000 7f9eea459740 1 -- 172.21.15.19:0/2247790694 shutdown_connections 2024-03-28T13:13:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.868+0000 7f9eea459740 1 --2- 172.21.15.19:0/2247790694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a67f206ea0 0x55a67f1e63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.868+0000 7f9eea459740 1 --2- 172.21.15.19:0/2247790694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a67f1e9320 0x55a67f1e5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.868+0000 7f9eea459740 1 --2- 172.21.15.19:0/2247790694 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55a67f1bad90 0x55a67f1bb170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.868+0000 7f9eea459740 1 --2- 172.21.15.19:0/2247790694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a67f1e8f40 0x55a67f278260 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.868+0000 7f9eea459740 1 -- 172.21.15.19:0/2247790694 >> 172.21.15.19:0/2247790694 conn(0x55a67f1bc420 msgr2=0x55a67f2066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:03.870 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-03-28T13:13:03.868+0000 7fdf3e600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6111=0-4194304@1} caps={6111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6111 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55cd5fb58c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-28T13:13:03.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.868+0000 7f9ece000640 0 client.6153 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:03.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.868+0000 7f9eea459740 1 -- 172.21.15.19:0/2247790694 shutdown_connections 2024-03-28T13:13:03.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.868+0000 7f9eea459740 1 -- 172.21.15.19:0/2247790694 wait complete. 2024-03-28T13:13:03.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.869+0000 7f9eea459740 20 client.6153 trim_cache size 0 max 16384 2024-03-28T13:13:03.871 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:13:03.871 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-03-28T13:13:03.871 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "00bf361b-ff31-44ba-a159-7136b2ac8406", 2024-03-28T13:13:03.871 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-03-28T13:13:03.871 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:13:03.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.869+0000 7f9ee7800640 1 -- 172.21.15.19:0/1139518054 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ebc06c530 msgr2=0x7f9ebc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.869+0000 7f9ee7800640 1 --2- 172.21.15.19:0/1139518054 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ebc06c530 0x7f9ebc06e9f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00ce6f0 tx=0x7f9ec8009040 comp rx=0 tx=0).stop 2024-03-28T13:13:03.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.869+0000 7f9ee7800640 1 -- 172.21.15.19:0/1139518054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ec40039e0 msgr2=0x7f9ee00a1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:03.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.869+0000 7f9ee7800640 1 --2- 172.21.15.19:0/1139518054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ec40039e0 0x7f9ee00a1c60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9ed002cc40 tx=0x7f9ed0001dc0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.869+0000 7f9ee7800640 1 -- 172.21.15.19:0/1139518054 shutdown_connections 2024-03-28T13:13:03.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.869+0000 7f9ee7800640 1 --2- 172.21.15.19:0/1139518054 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ebc06c530 0x7f9ebc06e9f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.869+0000 7f9ee7800640 1 --2- 172.21.15.19:0/1139518054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ee00b8260 0x7f9ee01f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.869+0000 7f9ee7800640 1 --2- 172.21.15.19:0/1139518054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ee015e2f0 0x7f9ee015f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.869+0000 7f9ee7800640 1 --2- 172.21.15.19:0/1139518054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ec40039e0 0x7f9ee00a1c60 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:03.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.869+0000 7f9ee7800640 1 -- 172.21.15.19:0/1139518054 >> 172.21.15.19:0/1139518054 conn(0x7f9ee008bfb0 msgr2=0x7f9ee006d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:03.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.869+0000 7f9ee7800640 1 -- 172.21.15.19:0/1139518054 shutdown_connections 2024-03-28T13:13:03.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:03.870+0000 7f9ee7800640 1 -- 172.21.15.19:0/1139518054 wait complete. 2024-03-28T13:13:03.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:13:03.887 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-28T13:13:04.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.090+0000 7f511ee00640 1 Processor -- start 2024-03-28T13:13:04.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.090+0000 7f511ee00640 1 -- start start 2024-03-28T13:13:04.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.092+0000 7f511ee00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f51180d43d0 0x7f51180d47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.092+0000 7f511ee00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51180cf6b0 0x7f51180cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.092+0000 7f511ee00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51180d4d20 con 0x7f511815e860 2024-03-28T13:13:04.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.092+0000 7f511ee00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f51180cf220 con 0x7f51180d43d0 2024-03-28T13:13:04.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.092+0000 7f511ee00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f511806dfa0 con 0x7f51180cf6b0 2024-03-28T13:13:04.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.093+0000 7f510fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f51180d43d0 0x7f51180d47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.093+0000 7f510fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f51180d43d0 0x7f51180d47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34080/0 (socket says 172.21.15.19:34080) 2024-03-28T13:13:04.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.093+0000 7f510fe00640 1 -- 172.21.15.19:0/2780269954 learned_addr learned my addr 172.21.15.19:0/2780269954 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:04.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.093+0000 7f5117e00640 1 --1- 172.21.15.19:0/2780269954 >> v1:172.21.15.19:6789/0 conn(0x7f511815e860 0x7f511806d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52212/0 (socket says 172.21.15.19:52212) 2024-03-28T13:13:04.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.093+0000 7f511ce00640 1 --2- 172.21.15.19:0/2780269954 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51180cf6b0 0x7f51180cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.093+0000 7f510fe00640 1 -- 172.21.15.19:0/2780269954 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51180cf6b0 msgr2=0x7f51180cfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:04.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.093+0000 7f510fe00640 1 --2- 172.21.15.19:0/2780269954 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51180cf6b0 0x7f51180cfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.093+0000 7f510fe00640 1 -- 172.21.15.19:0/2780269954 >> v1:172.21.15.19:6789/0 conn(0x7f511815e860 legacy=0x7f511806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:04.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.093+0000 7f510fe00640 1 -- 172.21.15.19:0/2780269954 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511806e240 con 0x7f51180d43d0 2024-03-28T13:13:04.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.093+0000 7f5117400640 1 -- 172.21.15.19:0/2780269954 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3405645581 0 0) 0x7f5108001110 con 0x7f511815e860 2024-03-28T13:13:04.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.094+0000 7f510fe00640 1 --2- 172.21.15.19:0/2780269954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51180d43d0 0x7f51180d47e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5100009870 tx=0x7f510002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec68adbbe7481e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:04.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.095+0000 7f5117400640 1 -- 172.21.15.19:0/2780269954 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100034070 con 0x7f51180d43d0 2024-03-28T13:13:04.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.095+0000 7f5117400640 1 -- 172.21.15.19:0/2780269954 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5100030840 con 0x7f51180d43d0 2024-03-28T13:13:04.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.095+0000 7f5117400640 1 -- 172.21.15.19:0/2780269954 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100030c10 con 0x7f51180d43d0 2024-03-28T13:13:04.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.095+0000 7f511ee00640 1 -- 172.21.15.19:0/2780269954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51180d43d0 msgr2=0x7f51180d47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:04.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.095+0000 7f511ee00640 1 --2- 172.21.15.19:0/2780269954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51180d43d0 0x7f51180d47e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5100009870 tx=0x7f510002b980 comp rx=0 tx=0).stop 2024-03-28T13:13:04.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.095+0000 7f511ee00640 1 -- 172.21.15.19:0/2780269954 shutdown_connections 2024-03-28T13:13:04.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.095+0000 7f511ee00640 1 --2- 172.21.15.19:0/2780269954 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51180cf6b0 0x7f51180cfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.095+0000 7f511ee00640 1 --2- 172.21.15.19:0/2780269954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51180d43d0 0x7f51180d47e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.095+0000 7f511ee00640 1 -- 172.21.15.19:0/2780269954 >> 172.21.15.19:0/2780269954 conn(0x7f511809d7d0 msgr2=0x7f511808bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:04.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.096+0000 7f511ee00640 1 -- 172.21.15.19:0/2780269954 shutdown_connections 2024-03-28T13:13:04.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.096+0000 7f511ee00640 1 -- 172.21.15.19:0/2780269954 wait complete. 2024-03-28T13:13:04.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.098+0000 7f511ee00640 1 Processor -- start 2024-03-28T13:13:04.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.099+0000 7f511ee00640 1 -- start start 2024-03-28T13:13:04.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.100+0000 7f511ee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51180cf6b0 0x7f51181528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.100+0000 7f511ee00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51180d43d0 0x7f51181606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.100+0000 7f511ee00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f511815e860 0x7f51180b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.100+0000 7f511ee00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f511806dc00 con 0x7f51180cf6b0 2024-03-28T13:13:04.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.100+0000 7f511ee00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51180d2c50 con 0x7f511815e860 2024-03-28T13:13:04.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.100+0000 7f511ee00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f511806dd80 con 0x7f51180d43d0 2024-03-28T13:13:04.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.100+0000 7f5117e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51180cf6b0 0x7f51181528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.101+0000 7f5117e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51180cf6b0 0x7f51181528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34088/0 (socket says 172.21.15.19:34088) 2024-03-28T13:13:04.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.101+0000 7f5117e00640 1 -- 172.21.15.19:0/1868589014 learned_addr learned my addr 172.21.15.19:0/1868589014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:04.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.101+0000 7f510fe00640 1 --2- 172.21.15.19:0/1868589014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51180d43d0 0x7f51181606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.101+0000 7f511ce00640 1 --2- 172.21.15.19:0/1868589014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f511815e860 0x7f51180b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.101+0000 7f5117e00640 1 -- 172.21.15.19:0/1868589014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51180d43d0 msgr2=0x7f51181606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:04.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.101+0000 7f5117e00640 1 --2- 172.21.15.19:0/1868589014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51180d43d0 0x7f51181606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.101+0000 7f5117e00640 1 -- 172.21.15.19:0/1868589014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f511815e860 msgr2=0x7f51180b8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:04.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.101+0000 7f5117e00640 1 --2- 172.21.15.19:0/1868589014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f511815e860 0x7f51180b8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.101+0000 7f5117e00640 1 -- 172.21.15.19:0/1868589014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5118152e30 con 0x7f51180cf6b0 2024-03-28T13:13:04.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.101+0000 7f5117e00640 1 --2- 172.21.15.19:0/1868589014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51180cf6b0 0x7f51181528f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f510800c440 tx=0x7f510800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:04.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.102+0000 7f5114c00640 1 -- 172.21.15.19:0/1868589014 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108018070 con 0x7f51180cf6b0 2024-03-28T13:13:04.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.102+0000 7f5114c00640 1 -- 172.21.15.19:0/1868589014 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51080074a0 con 0x7f51180cf6b0 2024-03-28T13:13:04.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.102+0000 7f5114c00640 1 -- 172.21.15.19:0/1868589014 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108004a20 con 0x7f51180cf6b0 2024-03-28T13:13:04.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.102+0000 7f511ee00640 1 -- 172.21.15.19:0/1868589014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51180c9bc0 con 0x7f51180cf6b0 2024-03-28T13:13:04.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.102+0000 7f511ee00640 1 -- 172.21.15.19:0/1868589014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51180c8e00 con 0x7f51180cf6b0 2024-03-28T13:13:04.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.104+0000 7f5114c00640 1 -- 172.21.15.19:0/1868589014 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f510801b070 con 0x7f51180cf6b0 2024-03-28T13:13:04.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.106+0000 7f5114c00640 1 --2- 172.21.15.19:0/1868589014 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50f406c560 0x7f50f406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.106+0000 7f5114c00640 1 -- 172.21.15.19:0/1868589014 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f510808a870 con 0x7f51180cf6b0 2024-03-28T13:13:04.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.106+0000 7f510fe00640 1 --2- 172.21.15.19:0/1868589014 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50f406c560 0x7f50f406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.107+0000 7f51219f5740 1 Processor -- start 2024-03-28T13:13:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.107+0000 7f51219f5740 1 -- start start 2024-03-28T13:13:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.107+0000 7f51219f5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c6560 0x557c3a599500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.107+0000 7f51219f5740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c6940 0x557c3a5c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.108+0000 7f51219f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a599a40 0x557c3a5c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.108+0000 7f51219f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557c3a5046b0 con 0x557c3a599a40 2024-03-28T13:13:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.108+0000 7f51219f5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557c3a4ac750 con 0x557c3a5c6560 2024-03-28T13:13:04.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.108+0000 7f51219f5740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x557c3a500660 con 0x557c3a5c6940 2024-03-28T13:13:04.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.108+0000 7f510fe00640 1 --2- 172.21.15.19:0/1868589014 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50f406c560 0x7f50f406ea20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5100036000 tx=0x7f510003b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:04.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.108+0000 7f511ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a599a40 0x557c3a5c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.108+0000 7f511ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a599a40 0x557c3a5c9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34096/0 (socket says 172.21.15.19:34096) 2024-03-28T13:13:04.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.108+0000 7f511ce00640 1 -- 172.21.15.19:0/3093385881 learned_addr learned my addr 172.21.15.19:0/3093385881 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:04.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.108+0000 7f5117e00640 1 --2- 172.21.15.19:0/3093385881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c6560 0x557c3a599500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.108+0000 7f510fe00640 1 --2- 172.21.15.19:0/3093385881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c6940 0x557c3a5c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.108+0000 7f511ce00640 1 -- 172.21.15.19:0/3093385881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c6940 msgr2=0x557c3a5c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:04.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.108+0000 7f511ce00640 1 --2- 172.21.15.19:0/3093385881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c6940 0x557c3a5c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.108+0000 7f511ce00640 1 -- 172.21.15.19:0/3093385881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c6560 msgr2=0x557c3a599500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:04.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.108+0000 7f511ce00640 1 --2- 172.21.15.19:0/3093385881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c6560 0x557c3a599500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.108+0000 7f511ce00640 1 -- 172.21.15.19:0/3093385881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c3a2e27d0 con 0x557c3a599a40 2024-03-28T13:13:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.109+0000 7f511ce00640 1 --2- 172.21.15.19:0/3093385881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a599a40 0x557c3a5c9830 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5110007360 tx=0x7f5110007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d050516750e5b6c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.109+0000 7f510cc00640 1 -- 172.21.15.19:0/3093385881 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110013780 con 0x557c3a599a40 2024-03-28T13:13:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.109+0000 7f510cc00640 1 -- 172.21.15.19:0/3093385881 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5110019070 con 0x557c3a599a40 2024-03-28T13:13:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.109+0000 7f510cc00640 1 -- 172.21.15.19:0/3093385881 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110013b40 con 0x557c3a599a40 2024-03-28T13:13:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.110+0000 7f51219f5740 1 -- 172.21.15.19:0/3093385881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a599a40 msgr2=0x557c3a5c9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.110+0000 7f51219f5740 1 --2- 172.21.15.19:0/3093385881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a599a40 0x557c3a5c9830 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5110007360 tx=0x7f5110007820 comp rx=0 tx=0).stop 2024-03-28T13:13:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.110+0000 7f51219f5740 1 -- 172.21.15.19:0/3093385881 shutdown_connections 2024-03-28T13:13:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.110+0000 7f51219f5740 1 --2- 172.21.15.19:0/3093385881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a599a40 0x557c3a5c9830 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.110+0000 7f51219f5740 1 --2- 172.21.15.19:0/3093385881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c6940 0x557c3a5c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.110+0000 7f51219f5740 1 --2- 172.21.15.19:0/3093385881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c6560 0x557c3a599500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.110+0000 7f51219f5740 1 -- 172.21.15.19:0/3093385881 >> 172.21.15.19:0/3093385881 conn(0x557c3a59b770 msgr2=0x557c3a5e42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:04.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.110+0000 7f51219f5740 1 -- 172.21.15.19:0/3093385881 shutdown_connections 2024-03-28T13:13:04.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.110+0000 7f51219f5740 1 -- 172.21.15.19:0/3093385881 wait complete. 2024-03-28T13:13:04.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.111+0000 7f51219f5740 1 -- start start 2024-03-28T13:13:04.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.111+0000 7f51219f5740 1 Processor -- start 2024-03-28T13:13:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.112+0000 7f51219f5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c6560 0x557c3a5c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.112+0000 7f51219f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6940 0x557c3a5cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.112+0000 7f51219f5740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c2060 0x557c3a5cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.112+0000 7f51219f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557c3a59de10 con 0x557c3a5c6940 2024-03-28T13:13:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.112+0000 7f51219f5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557c3a4bfa80 con 0x557c3a5c6560 2024-03-28T13:13:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.112+0000 7f51219f5740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x557c3a5a0650 con 0x557c3a5c2060 2024-03-28T13:13:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.112+0000 7f51219f5740 10 client.0 _ll_register_callbacks cb 0x557c3a4bf470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:13:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.112+0000 7f510fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6940 0x557c3a5cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.113+0000 7f510fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6940 0x557c3a5cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34100/0 (socket says 172.21.15.19:34100) 2024-03-28T13:13:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.113+0000 7f510fe00640 1 -- 172.21.15.19:0/123403584 learned_addr learned my addr 172.21.15.19:0/123403584 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.113+0000 7f5117e00640 1 --2- 172.21.15.19:0/123403584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c6560 0x557c3a5c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.113+0000 7f511ce00640 1 --2- 172.21.15.19:0/123403584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c2060 0x557c3a5cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.113+0000 7f510fe00640 1 -- 172.21.15.19:0/123403584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c2060 msgr2=0x557c3a5cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.113+0000 7f510fe00640 1 --2- 172.21.15.19:0/123403584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c2060 0x557c3a5cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.113+0000 7f510fe00640 1 -- 172.21.15.19:0/123403584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c6560 msgr2=0x557c3a5c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.113+0000 7f510fe00640 1 --2- 172.21.15.19:0/123403584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c6560 0x557c3a5c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.113+0000 7f510fe00640 1 -- 172.21.15.19:0/123403584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c3a2183f0 con 0x557c3a5c6940 2024-03-28T13:13:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.113+0000 7f510fe00640 1 --2- 172.21.15.19:0/123403584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6940 0x557c3a5cbde0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5100009990 tx=0x7f510002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10572f32d65b1028 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.113+0000 7f5105600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:13:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.114+0000 7f5105600640 1 -- 172.21.15.19:0/123403584 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100034040 con 0x557c3a5c6940 2024-03-28T13:13:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.114+0000 7f5105600640 1 -- 172.21.15.19:0/123403584 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f5100043810 con 0x557c3a5c6940 2024-03-28T13:13:04.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.114+0000 7f51219f5740 1 -- 172.21.15.19:0/123403584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5110019070 con 0x557c3a5c6940 2024-03-28T13:13:04.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.114+0000 7f5105600640 1 -- 172.21.15.19:0/123403584 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5100043a10 con 0x557c3a5c6940 2024-03-28T13:13:04.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.114+0000 7f51219f5740 10 client.6168 fetch_fsmap learned FSMap version 34 2024-03-28T13:13:04.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.114+0000 7f51219f5740 1 -- 172.21.15.19:0/123403584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x557c3a1d80f0 con 0x557c3a5c6940 2024-03-28T13:13:04.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.114+0000 7f5105600640 1 -- 172.21.15.19:0/123403584 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5110019070 con 0x557c3a5c6940 2024-03-28T13:13:04.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.114+0000 7f5105600640 1 -- 172.21.15.19:0/123403584 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f510005f020 con 0x557c3a5c6940 2024-03-28T13:13:04.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.114+0000 7f51219f5740 10 client.6168 fetch_fsmap finished waiting for FSMap version 34 2024-03-28T13:13:04.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.114+0000 7f51219f5740 10 client.6168 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-03-28T13:13:04.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.114+0000 7f51219f5740 20 client.6168 populate_metadata read hostname 'smithi019' 2024-03-28T13:13:04.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.114+0000 7f51219f5740 1 --2- 172.21.15.19:0/123403584 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x557c3a5bf4c0 0x557c3a5bf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.114+0000 7f51219f5740 4 client.6168 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:13:04.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.114+0000 7f51219f5740 1 -- 172.21.15.19:0/123403584 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c3a511bf0 con 0x557c3a5bf4c0 2024-03-28T13:13:04.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.115+0000 7f5117e00640 1 --2- 172.21.15.19:0/123403584 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x557c3a5bf4c0 0x557c3a5bf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.115+0000 7f5117e00640 1 --2- 172.21.15.19:0/123403584 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x557c3a5bf4c0 0x557c3a5bf940 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a5e810cfb5c580d server_cookie=2f1e3a8296ae64da in_seq=0 out_seq=0 2024-03-28T13:13:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.115+0000 7f5105600640 10 client.6168 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.116+0000 7f5105600640 1 -- 172.21.15.19:0/123403584 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c3a511bf0 con 0x557c3a5bf4c0 2024-03-28T13:13:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.116+0000 7f5105600640 10 client.6168 handle_command_reply: tid=0 2024-03-28T13:13:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.116+0000 7f51219f5740 1 client.6168 shutdown 2024-03-28T13:13:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.116+0000 7f51219f5740 1 -- 172.21.15.19:0/123403584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6940 msgr2=0x557c3a5cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.116+0000 7f51219f5740 1 --2- 172.21.15.19:0/123403584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6940 0x557c3a5cbde0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f5100009990 tx=0x7f510002e000 comp rx=0 tx=0).stop 2024-03-28T13:13:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.116+0000 7f51219f5740 1 -- 172.21.15.19:0/123403584 shutdown_connections 2024-03-28T13:13:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.116+0000 7f51219f5740 1 --2- 172.21.15.19:0/123403584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c2060 0x557c3a5cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.116+0000 7f51219f5740 1 --2- 172.21.15.19:0/123403584 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x557c3a5bf4c0 0x557c3a5bf940 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.116+0000 7f51219f5740 1 --2- 172.21.15.19:0/123403584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6940 0x557c3a5cbde0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.116+0000 7f51219f5740 1 --2- 172.21.15.19:0/123403584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c6560 0x557c3a5c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.116+0000 7f51219f5740 1 -- 172.21.15.19:0/123403584 >> 172.21.15.19:0/123403584 conn(0x557c3a599a40 msgr2=0x557c3a59d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.116+0000 7f5105600640 0 client.6168 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.117+0000 7f51219f5740 1 -- 172.21.15.19:0/123403584 shutdown_connections 2024-03-28T13:13:04.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.117+0000 7f51219f5740 1 -- 172.21.15.19:0/123403584 wait complete. 2024-03-28T13:13:04.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.117+0000 7f51219f5740 20 client.6168 trim_cache size 0 max 16384 2024-03-28T13:13:04.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.136+0000 7f51219f5740 1 Processor -- start 2024-03-28T13:13:04.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.136+0000 7f51219f5740 1 -- start start 2024-03-28T13:13:04.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.137+0000 7f51219f5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c2060 0x557c3a598710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.137+0000 7f51219f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6560 0x557c3a5e76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.137+0000 7f51219f5740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c6940 0x557c3a607020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.137+0000 7f51219f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5118200b20 con 0x557c3a5c6560 2024-03-28T13:13:04.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.137+0000 7f51219f5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51181ffb40 con 0x557c3a5c2060 2024-03-28T13:13:04.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.137+0000 7f51219f5740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f51180cbe50 con 0x557c3a5c6940 2024-03-28T13:13:04.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.137+0000 7f510fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6560 0x557c3a5e76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.137+0000 7f510fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6560 0x557c3a5e76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34110/0 (socket says 172.21.15.19:34110) 2024-03-28T13:13:04.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.137+0000 7f510fe00640 1 -- 172.21.15.19:0/2346633828 learned_addr learned my addr 172.21.15.19:0/2346633828 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:04.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.137+0000 7f510fe00640 1 -- 172.21.15.19:0/2346633828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c6940 msgr2=0x557c3a607020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:04.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.137+0000 7f510fe00640 1 --2- 172.21.15.19:0/2346633828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c6940 0x557c3a607020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.138+0000 7f510fe00640 1 -- 172.21.15.19:0/2346633828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c2060 msgr2=0x557c3a598710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:04.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.138+0000 7f510fe00640 1 --2- 172.21.15.19:0/2346633828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c2060 0x557c3a598710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.138+0000 7f510fe00640 1 -- 172.21.15.19:0/2346633828 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c3a5c9a00 con 0x557c3a5c6560 2024-03-28T13:13:04.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.138+0000 7f510fe00640 1 --2- 172.21.15.19:0/2346633828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6560 0x557c3a5e76b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f51000306e0 tx=0x7f5100009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8317ad8f02aaaa59 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:04.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.138+0000 7f5107e00640 1 -- 172.21.15.19:0/2346633828 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100043610 con 0x557c3a5c6560 2024-03-28T13:13:04.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.138+0000 7f5107e00640 1 -- 172.21.15.19:0/2346633828 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5100054070 con 0x557c3a5c6560 2024-03-28T13:13:04.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.138+0000 7f5107e00640 1 -- 172.21.15.19:0/2346633828 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100004030 con 0x557c3a5c6560 2024-03-28T13:13:04.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.139+0000 7f51219f5740 1 -- 172.21.15.19:0/2346633828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6560 msgr2=0x557c3a5e76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:04.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.139+0000 7f51219f5740 1 --2- 172.21.15.19:0/2346633828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6560 0x557c3a5e76b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f51000306e0 tx=0x7f5100009240 comp rx=0 tx=0).stop 2024-03-28T13:13:04.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.139+0000 7f51219f5740 1 -- 172.21.15.19:0/2346633828 shutdown_connections 2024-03-28T13:13:04.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.139+0000 7f51219f5740 1 --2- 172.21.15.19:0/2346633828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c6940 0x557c3a607020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.139+0000 7f51219f5740 1 --2- 172.21.15.19:0/2346633828 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6560 0x557c3a5e76b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.139+0000 7f51219f5740 1 --2- 172.21.15.19:0/2346633828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c2060 0x557c3a598710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.139+0000 7f51219f5740 1 -- 172.21.15.19:0/2346633828 >> 172.21.15.19:0/2346633828 conn(0x557c3a599a40 msgr2=0x557c3a5e49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:04.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.139+0000 7f51219f5740 1 -- 172.21.15.19:0/2346633828 shutdown_connections 2024-03-28T13:13:04.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.139+0000 7f51219f5740 1 -- 172.21.15.19:0/2346633828 wait complete. 2024-03-28T13:13:04.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.140+0000 7f51219f5740 1 -- start start 2024-03-28T13:13:04.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.140+0000 7f51219f5740 1 Processor -- start 2024-03-28T13:13:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.141+0000 7f51219f5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c2060 0x557c3a6473c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.141+0000 7f51219f5740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c6560 0x557c3a5cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.141+0000 7f51219f5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6940 0x557c3a5ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.141+0000 7f51219f5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557c3a59b010 con 0x557c3a5c6940 2024-03-28T13:13:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.141+0000 7f51219f5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51180cc4a0 con 0x557c3a5c2060 2024-03-28T13:13:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.141+0000 7f51219f5740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x557c3a538890 con 0x557c3a5c6560 2024-03-28T13:13:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.141+0000 7f51219f5740 10 client.0 _ll_register_callbacks cb 0x7f5100030c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:13:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.141+0000 7f511ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6940 0x557c3a5ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.141+0000 7f511ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6940 0x557c3a5ccf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34124/0 (socket says 172.21.15.19:34124) 2024-03-28T13:13:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.141+0000 7f511ce00640 1 -- 172.21.15.19:0/781134917 learned_addr learned my addr 172.21.15.19:0/781134917 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.141+0000 7f510fe00640 1 --2- 172.21.15.19:0/781134917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c6560 0x557c3a5cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.141+0000 7f5117e00640 1 --2- 172.21.15.19:0/781134917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c2060 0x557c3a6473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.142+0000 7f511ce00640 1 -- 172.21.15.19:0/781134917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c6560 msgr2=0x557c3a5cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.142+0000 7f511ce00640 1 --2- 172.21.15.19:0/781134917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c6560 0x557c3a5cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.142+0000 7f511ce00640 1 -- 172.21.15.19:0/781134917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c2060 msgr2=0x557c3a6473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.142+0000 7f511ce00640 1 --2- 172.21.15.19:0/781134917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c2060 0x557c3a6473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.142+0000 7f511ce00640 1 -- 172.21.15.19:0/781134917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c3a65e080 con 0x557c3a5c6940 2024-03-28T13:13:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.142+0000 7f510fe00640 1 --2- 172.21.15.19:0/781134917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c6560 0x557c3a5cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.142+0000 7f5117e00640 1 --2- 172.21.15.19:0/781134917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c2060 0x557c3a6473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.142+0000 7f511ce00640 1 --2- 172.21.15.19:0/781134917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6940 0x557c3a5ccf40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5110009f30 tx=0x7f5110004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36f2875ffd97d7a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.142+0000 7f5105600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:13:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.142+0000 7f5105600640 1 -- 172.21.15.19:0/781134917 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110003e60 con 0x557c3a5c6940 2024-03-28T13:13:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.142+0000 7f5105600640 1 -- 172.21.15.19:0/781134917 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f5110029ae0 con 0x557c3a5c6940 2024-03-28T13:13:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.142+0000 7f51219f5740 1 -- 172.21.15.19:0/781134917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5100054070 con 0x557c3a5c6940 2024-03-28T13:13:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.143+0000 7f5105600640 1 -- 172.21.15.19:0/781134917 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5110030050 con 0x557c3a5c6940 2024-03-28T13:13:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.143+0000 7f51219f5740 10 client.6174 fetch_fsmap learned FSMap version 34 2024-03-28T13:13:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.143+0000 7f51219f5740 1 -- 172.21.15.19:0/781134917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x557c3a5c1500 con 0x557c3a5c6940 2024-03-28T13:13:04.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.143+0000 7f5105600640 1 -- 172.21.15.19:0/781134917 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5100054070 con 0x557c3a5c6940 2024-03-28T13:13:04.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.143+0000 7f5105600640 1 -- 172.21.15.19:0/781134917 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 3494+0+0 (secure 0 0 0) 0x7f5110033020 con 0x557c3a5c6940 2024-03-28T13:13:04.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.144+0000 7f51219f5740 10 client.6174 fetch_fsmap finished waiting for FSMap version 34 2024-03-28T13:13:04.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.144+0000 7f51219f5740 10 client.6174 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-03-28T13:13:04.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.144+0000 7f51219f5740 20 client.6174 populate_metadata read hostname 'smithi019' 2024-03-28T13:13:04.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.144+0000 7f51219f5740 1 --2- 172.21.15.19:0/781134917 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x557c3a5c1b90 0x557c3a5bce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:04.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.144+0000 7f51219f5740 4 client.6174 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:13:04.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.144+0000 7f51219f5740 1 -- 172.21.15.19:0/781134917 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c3a50bf40 con 0x557c3a5c1b90 2024-03-28T13:13:04.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.144+0000 7f5117e00640 1 --2- 172.21.15.19:0/781134917 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x557c3a5c1b90 0x557c3a5bce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:04.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.144+0000 7f5117e00640 1 --2- 172.21.15.19:0/781134917 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x557c3a5c1b90 0x557c3a5bce20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bed307a1a5eaebc2 server_cookie=878abc2e2c6845df in_seq=0 out_seq=0 2024-03-28T13:13:04.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.144+0000 7f5105600640 10 client.6174 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:04.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.145+0000 7f5105600640 1 -- 172.21.15.19:0/781134917 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+253 (crc 0 0 0) 0x557c3a50bf40 con 0x557c3a5c1b90 2024-03-28T13:13:04.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.145+0000 7f5105600640 10 client.6174 handle_command_reply: tid=0 2024-03-28T13:13:04.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.145+0000 7f51219f5740 1 client.6174 shutdown 2024-03-28T13:13:04.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.145+0000 7f51219f5740 1 -- 172.21.15.19:0/781134917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6940 msgr2=0x557c3a5ccf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:04.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.145+0000 7f51219f5740 1 --2- 172.21.15.19:0/781134917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6940 0x557c3a5ccf40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5110009f30 tx=0x7f5110004a40 comp rx=0 tx=0).stop 2024-03-28T13:13:04.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.145+0000 7f51219f5740 1 -- 172.21.15.19:0/781134917 shutdown_connections 2024-03-28T13:13:04.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.145+0000 7f51219f5740 1 --2- 172.21.15.19:0/781134917 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x557c3a5c1b90 0x557c3a5bce20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.145+0000 7f51219f5740 1 --2- 172.21.15.19:0/781134917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557c3a5c6940 0x557c3a5ccf40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.145+0000 7f51219f5740 1 --2- 172.21.15.19:0/781134917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557c3a5c6560 0x557c3a5cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.145+0000 7f51219f5740 1 --2- 172.21.15.19:0/781134917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557c3a5c2060 0x557c3a6473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.145+0000 7f5105600640 0 client.6174 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:04.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.145+0000 7f51219f5740 1 -- 172.21.15.19:0/781134917 >> 172.21.15.19:0/781134917 conn(0x557c3a599a40 msgr2=0x557c3a594a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:04.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.145+0000 7f51219f5740 1 -- 172.21.15.19:0/781134917 shutdown_connections 2024-03-28T13:13:04.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.145+0000 7f51219f5740 1 -- 172.21.15.19:0/781134917 wait complete. 2024-03-28T13:13:04.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.145+0000 7f51219f5740 20 client.6174 trim_cache size 0 max 16384 2024-03-28T13:13:04.148 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:13:04.148 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T13:13:04.148 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-03-28T13:13:04.149 INFO:teuthology.orchestra.run.smithi019.stdout: "00bf361b-ff31-44ba-a159-7136b2ac8406": { 2024-03-28T13:13:04.149 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-03-28T13:13:04.149 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "00bf361b-ff31-44ba-a159-7136b2ac8406", 2024-03-28T13:13:04.149 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive" 2024-03-28T13:13:04.149 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:13:04.149 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:13:04.149 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:13:04.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.146+0000 7f511ee00640 1 -- 172.21.15.19:0/1868589014 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50f406c560 msgr2=0x7f50f406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:04.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.146+0000 7f511ee00640 1 --2- 172.21.15.19:0/1868589014 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50f406c560 0x7f50f406ea20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5100036000 tx=0x7f510003b040 comp rx=0 tx=0).stop 2024-03-28T13:13:04.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.146+0000 7f511ee00640 1 -- 172.21.15.19:0/1868589014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51180cf6b0 msgr2=0x7f51181528f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:04.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.146+0000 7f511ee00640 1 --2- 172.21.15.19:0/1868589014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51180cf6b0 0x7f51181528f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f510800c440 tx=0x7f510800c900 comp rx=0 tx=0).stop 2024-03-28T13:13:04.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.146+0000 7f511ee00640 1 -- 172.21.15.19:0/1868589014 shutdown_connections 2024-03-28T13:13:04.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.146+0000 7f511ee00640 1 --2- 172.21.15.19:0/1868589014 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50f406c560 0x7f50f406ea20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.146+0000 7f511ee00640 1 --2- 172.21.15.19:0/1868589014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f511815e860 0x7f51180b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.146+0000 7f511ee00640 1 --2- 172.21.15.19:0/1868589014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51180d43d0 0x7f51181606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.146+0000 7f511ee00640 1 --2- 172.21.15.19:0/1868589014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51180cf6b0 0x7f51181528f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:04.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.146+0000 7f511ee00640 1 -- 172.21.15.19:0/1868589014 >> 172.21.15.19:0/1868589014 conn(0x7f511809d7d0 msgr2=0x7f511806cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:04.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.146+0000 7f511ee00640 1 -- 172.21.15.19:0/1868589014 shutdown_connections 2024-03-28T13:13:04.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:04.146+0000 7f511ee00640 1 -- 172.21.15.19:0/1868589014 wait complete. 2024-03-28T13:13:04.168 INFO:tasks.cephfs.filesystem:scrub status for tag:00bf361b-ff31-44ba-a159-7136b2ac8406 - {'path': '/', 'tag': '00bf361b-ff31-44ba-a159-7136b2ac8406', 'options': 'recursive'} 2024-03-28T13:13:09.594 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:13:09.593+0000 7f6b77600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-28T13:13:32.111 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:13:32.119 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:13:34.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:13:34.169 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-28T13:13:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.395+0000 7f46c1c00640 1 Processor -- start 2024-03-28T13:13:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.395+0000 7f46c1c00640 1 -- start start 2024-03-28T13:13:34.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.397+0000 7f46c1c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f46bc0d1580 0x7f46bc0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.397+0000 7f46c1c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46bc0b4db0 0x7f46bc0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.397+0000 7f46c1c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46bc0b5ce0 con 0x7f46bc161f60 2024-03-28T13:13:34.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.397+0000 7f46c1c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f46bc0d27f0 con 0x7f46bc0d1580 2024-03-28T13:13:34.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.397+0000 7f46c1c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f46bc0d2a90 con 0x7f46bc0b4db0 2024-03-28T13:13:34.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.397+0000 7f46bb400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46bc0b4db0 0x7f46bc0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.397+0000 7f46b2000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f46bc0d1580 0x7f46bc0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.398+0000 7f46bb400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46bc0b4db0 0x7f46bc0cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:58134/0 (socket says 172.21.15.19:58134) 2024-03-28T13:13:34.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.398+0000 7f46bb400640 1 -- 172.21.15.19:0/2525507097 learned_addr learned my addr 172.21.15.19:0/2525507097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:34.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.398+0000 7f46b2000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f46bc0d1580 0x7f46bc0cf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:41516/0 (socket says 172.21.15.19:41516) 2024-03-28T13:13:34.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.398+0000 7f46b2000640 1 -- 172.21.15.19:0/2525507097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46bc0b4db0 msgr2=0x7f46bc0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:34.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.398+0000 7f46b2000640 1 --2- 172.21.15.19:0/2525507097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46bc0b4db0 0x7f46bc0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.398+0000 7f46b2000640 1 -- 172.21.15.19:0/2525507097 >> v1:172.21.15.188:6789/0 conn(0x7f46bc161f60 legacy=0x7f46bc0d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.398+0000 7f46b2000640 1 -- 172.21.15.19:0/2525507097 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46bc162340 con 0x7f46bc0d1580 2024-03-28T13:13:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.400+0000 7f46b2000640 1 --2- 172.21.15.19:0/2525507097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46bc0d1580 0x7f46bc0cf0d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f46a40097a0 tx=0x7f46a402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=862b52a073d05910 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:34.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.401+0000 7f46ba000640 1 -- 172.21.15.19:0/2525507097 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4035020 con 0x7f46bc0d1580 2024-03-28T13:13:34.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.401+0000 7f46ba000640 1 -- 172.21.15.19:0/2525507097 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a4031830 con 0x7f46bc0d1580 2024-03-28T13:13:34.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.401+0000 7f46ba000640 1 -- 172.21.15.19:0/2525507097 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4031c00 con 0x7f46bc0d1580 2024-03-28T13:13:34.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.402+0000 7f46c1c00640 1 -- 172.21.15.19:0/2525507097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46bc0d1580 msgr2=0x7f46bc0cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.402+0000 7f46c1c00640 1 --2- 172.21.15.19:0/2525507097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46bc0d1580 0x7f46bc0cf0d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f46a40097a0 tx=0x7f46a402b8d0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.402+0000 7f46c1c00640 1 -- 172.21.15.19:0/2525507097 shutdown_connections 2024-03-28T13:13:34.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.402+0000 7f46c1c00640 1 --2- 172.21.15.19:0/2525507097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46bc0d1580 0x7f46bc0cf0d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.402+0000 7f46c1c00640 1 --2- 172.21.15.19:0/2525507097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46bc0b4db0 0x7f46bc0cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.402+0000 7f46c1c00640 1 -- 172.21.15.19:0/2525507097 >> 172.21.15.19:0/2525507097 conn(0x7f46bc08bfb0 msgr2=0x7f46bc0d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:34.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.402+0000 7f46c1c00640 1 -- 172.21.15.19:0/2525507097 shutdown_connections 2024-03-28T13:13:34.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.402+0000 7f46c1c00640 1 -- 172.21.15.19:0/2525507097 wait complete. 2024-03-28T13:13:34.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.404+0000 7f46c1c00640 1 Processor -- start 2024-03-28T13:13:34.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.404+0000 7f46c1c00640 1 -- start start 2024-03-28T13:13:34.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.405+0000 7f46c1c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46bc0b4db0 0x7f46bc1f86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.405+0000 7f46c1c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46bc0d1580 0x7f46bc0b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.405+0000 7f46c1c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46bc161f60 0x7f46bc158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.405+0000 7f46c1c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc0d3310 con 0x7f46bc0b4db0 2024-03-28T13:13:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.405+0000 7f46c1c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc0d3030 con 0x7f46bc161f60 2024-03-28T13:13:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.405+0000 7f46c1c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f46bc0b3170 con 0x7f46bc0d1580 2024-03-28T13:13:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.405+0000 7f46baa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46bc0b4db0 0x7f46bc1f86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.405+0000 7f46baa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46bc0b4db0 0x7f46bc1f86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41612/0 (socket says 172.21.15.19:41612) 2024-03-28T13:13:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.405+0000 7f46baa00640 1 -- 172.21.15.19:0/335122390 learned_addr learned my addr 172.21.15.19:0/335122390 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.405+0000 7f46bb400640 1 --2- 172.21.15.19:0/335122390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46bc161f60 0x7f46bc158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.405+0000 7f46b2000640 1 --2- 172.21.15.19:0/335122390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46bc0d1580 0x7f46bc0b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.406+0000 7f46baa00640 1 -- 172.21.15.19:0/335122390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46bc0d1580 msgr2=0x7f46bc0b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.406+0000 7f46baa00640 1 --2- 172.21.15.19:0/335122390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46bc0d1580 0x7f46bc0b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.406+0000 7f46baa00640 1 -- 172.21.15.19:0/335122390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46bc161f60 msgr2=0x7f46bc158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.406+0000 7f46baa00640 1 --2- 172.21.15.19:0/335122390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46bc161f60 0x7f46bc158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.406+0000 7f46baa00640 1 -- 172.21.15.19:0/335122390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46bc1f8c00 con 0x7f46bc0b4db0 2024-03-28T13:13:34.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.406+0000 7f46baa00640 1 --2- 172.21.15.19:0/335122390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46bc0b4db0 0x7f46bc1f86c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00c760 tx=0x7f46ac00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:34.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.406+0000 7f46b3400640 1 -- 172.21.15.19:0/335122390 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac016020 con 0x7f46bc0b4db0 2024-03-28T13:13:34.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.406+0000 7f46b3400640 1 -- 172.21.15.19:0/335122390 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ac006d50 con 0x7f46bc0b4db0 2024-03-28T13:13:34.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.406+0000 7f46b3400640 1 -- 172.21.15.19:0/335122390 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac003bb0 con 0x7f46bc0b4db0 2024-03-28T13:13:34.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.406+0000 7f46c1c00640 1 -- 172.21.15.19:0/335122390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46bc0b9390 con 0x7f46bc0b4db0 2024-03-28T13:13:34.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.407+0000 7f46c1c00640 1 -- 172.21.15.19:0/335122390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46bc0cb7e0 con 0x7f46bc0b4db0 2024-03-28T13:13:34.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.410+0000 7f46b3400640 1 -- 172.21.15.19:0/335122390 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f46ac00a850 con 0x7f46bc0b4db0 2024-03-28T13:13:34.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.410+0000 7f46c49b0740 1 Processor -- start 2024-03-28T13:13:34.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.411+0000 7f46c49b0740 1 -- start start 2024-03-28T13:13:34.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.411+0000 7f46b3400640 1 --2- 172.21.15.19:0/335122390 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f469806c530 0x7f469806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.411+0000 7f46b3400640 1 -- 172.21.15.19:0/335122390 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f46ac08b3d0 con 0x7f46bc0b4db0 2024-03-28T13:13:34.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.412+0000 7f46c49b0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9daf560 0x560bd9d82500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.412+0000 7f46c49b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf940 0x560bd9db22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.412+0000 7f46c49b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9d82a40 0x560bd9db2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.412+0000 7f46c49b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560bd9ced6b0 con 0x560bd9daf940 2024-03-28T13:13:34.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.412+0000 7f46c49b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560bd9c95750 con 0x560bd9d82a40 2024-03-28T13:13:34.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.412+0000 7f46c49b0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x560bd9ce9660 con 0x560bd9daf560 2024-03-28T13:13:34.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.412+0000 7f46bb400640 1 --2- 172.21.15.19:0/335122390 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f469806c530 0x7f469806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.412+0000 7f46b2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf940 0x560bd9db22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.412+0000 7f46b2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf940 0x560bd9db22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41614/0 (socket says 172.21.15.19:41614) 2024-03-28T13:13:34.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.412+0000 7f46b2000640 1 -- 172.21.15.19:0/4226746098 learned_addr learned my addr 172.21.15.19:0/4226746098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:34.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.412+0000 7f46baa00640 1 --2- 172.21.15.19:0/4226746098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9daf560 0x560bd9d82500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.412+0000 7f46bb400640 1 --2- 172.21.15.19:0/4226746098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9d82a40 0x560bd9db2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.413+0000 7f46b2000640 1 -- 172.21.15.19:0/4226746098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9daf560 msgr2=0x560bd9d82500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:34.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.413+0000 7f46b2000640 1 --2- 172.21.15.19:0/4226746098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9daf560 0x560bd9d82500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.413+0000 7f46b2000640 1 -- 172.21.15.19:0/4226746098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9d82a40 msgr2=0x560bd9db2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:34.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.413+0000 7f46b2000640 1 --2- 172.21.15.19:0/4226746098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9d82a40 0x560bd9db2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.413+0000 7f46b2000640 1 -- 172.21.15.19:0/4226746098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd9acb7d0 con 0x560bd9daf940 2024-03-28T13:13:34.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.413+0000 7f46b2000640 1 --2- 172.21.15.19:0/4226746098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf940 0x560bd9db22f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f46a402bee0 tx=0x7f46a4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3370e18e220d951c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:34.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.413+0000 7f46ab400640 1 -- 172.21.15.19:0/4226746098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4035040 con 0x560bd9daf940 2024-03-28T13:13:34.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.413+0000 7f46bb400640 1 --2- 172.21.15.19:0/335122390 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f469806c530 0x7f469806e9f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x560bd9d85700 tx=0x7f46b4009570 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:34.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.413+0000 7f46ab400640 1 -- 172.21.15.19:0/4226746098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a40317f0 con 0x560bd9daf940 2024-03-28T13:13:34.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.413+0000 7f46ab400640 1 -- 172.21.15.19:0/4226746098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4031bc0 con 0x560bd9daf940 2024-03-28T13:13:34.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.414+0000 7f46c49b0740 1 -- 172.21.15.19:0/4226746098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf940 msgr2=0x560bd9db22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.414+0000 7f46c49b0740 1 --2- 172.21.15.19:0/4226746098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf940 0x560bd9db22f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f46a402bee0 tx=0x7f46a4030000 comp rx=0 tx=0).stop 2024-03-28T13:13:34.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.414+0000 7f46c49b0740 1 -- 172.21.15.19:0/4226746098 shutdown_connections 2024-03-28T13:13:34.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.414+0000 7f46c49b0740 1 --2- 172.21.15.19:0/4226746098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9d82a40 0x560bd9db2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.414+0000 7f46c49b0740 1 --2- 172.21.15.19:0/4226746098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf940 0x560bd9db22f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.414+0000 7f46c49b0740 1 --2- 172.21.15.19:0/4226746098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9daf560 0x560bd9d82500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.414+0000 7f46c49b0740 1 -- 172.21.15.19:0/4226746098 >> 172.21.15.19:0/4226746098 conn(0x560bd9d84770 msgr2=0x560bd9dcd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:34.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.414+0000 7f46c49b0740 1 -- 172.21.15.19:0/4226746098 shutdown_connections 2024-03-28T13:13:34.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.414+0000 7f46c49b0740 1 -- 172.21.15.19:0/4226746098 wait complete. 2024-03-28T13:13:34.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.415+0000 7f46c49b0740 1 -- start start 2024-03-28T13:13:34.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.416+0000 7f46c49b0740 1 Processor -- start 2024-03-28T13:13:34.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46c49b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf560 0x560bd9ddd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46c49b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf940 0x560bd9dcd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46c49b0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9db8c70 0x560bd9db90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46c49b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560bd9d86e10 con 0x560bd9daf940 2024-03-28T13:13:34.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46c49b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560bd9ca8a80 con 0x560bd9daf560 2024-03-28T13:13:34.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46c49b0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x560bd9d89650 con 0x560bd9db8c70 2024-03-28T13:13:34.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46c49b0740 10 client.0 _ll_register_callbacks cb 0x560bd9ca8470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:13:34.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46b2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf940 0x560bd9dcd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46b2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf940 0x560bd9dcd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41616/0 (socket says 172.21.15.19:41616) 2024-03-28T13:13:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46b2000640 1 -- 172.21.15.19:0/415469799 learned_addr learned my addr 172.21.15.19:0/415469799 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46baa00640 1 --2- 172.21.15.19:0/415469799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf560 0x560bd9ddd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46bb400640 1 --2- 172.21.15.19:0/415469799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9db8c70 0x560bd9db90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46b2000640 1 -- 172.21.15.19:0/415469799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9db8c70 msgr2=0x560bd9db90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46b2000640 1 --2- 172.21.15.19:0/415469799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9db8c70 0x560bd9db90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46b2000640 1 -- 172.21.15.19:0/415469799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf560 msgr2=0x560bd9ddd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46b2000640 1 --2- 172.21.15.19:0/415469799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf560 0x560bd9ddd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.417+0000 7f46b2000640 1 -- 172.21.15.19:0/415469799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bd9a013f0 con 0x560bd9daf940 2024-03-28T13:13:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.418+0000 7f46b2000640 1 --2- 172.21.15.19:0/415469799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf940 0x560bd9dcd6b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f46a40097a0 tx=0x7f46a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e17043ed9e9103 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.418+0000 7f469fe00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:13:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.418+0000 7f469fe00640 1 -- 172.21.15.19:0/415469799 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a4035030 con 0x560bd9daf940 2024-03-28T13:13:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.418+0000 7f469fe00640 1 -- 172.21.15.19:0/415469799 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f46a40319a0 con 0x560bd9daf940 2024-03-28T13:13:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.418+0000 7f46c49b0740 1 -- 172.21.15.19:0/415469799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46a40317f0 con 0x560bd9daf940 2024-03-28T13:13:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.418+0000 7f469fe00640 1 -- 172.21.15.19:0/415469799 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f46a4057050 con 0x560bd9daf940 2024-03-28T13:13:34.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.418+0000 7f46c49b0740 10 client.6183 fetch_fsmap learned FSMap version 35 2024-03-28T13:13:34.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.418+0000 7f46c49b0740 1 -- 172.21.15.19:0/415469799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x560bd99c10f0 con 0x560bd9daf940 2024-03-28T13:13:34.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.419+0000 7f469fe00640 1 -- 172.21.15.19:0/415469799 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a40317f0 con 0x560bd9daf940 2024-03-28T13:13:34.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.419+0000 7f469fe00640 1 -- 172.21.15.19:0/415469799 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f46a405a020 con 0x560bd9daf940 2024-03-28T13:13:34.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.419+0000 7f46c49b0740 10 client.6183 fetch_fsmap finished waiting for FSMap version 35 2024-03-28T13:13:34.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.419+0000 7f46c49b0740 10 client.6183 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-03-28T13:13:34.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.419+0000 7f46c49b0740 20 client.6183 populate_metadata read hostname 'smithi019' 2024-03-28T13:13:34.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.419+0000 7f46c49b0740 1 --2- 172.21.15.19:0/415469799 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x560bd9db7860 0x560bd9db7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.419+0000 7f46c49b0740 4 client.6183 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:13:34.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.419+0000 7f46c49b0740 1 -- 172.21.15.19:0/415469799 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bd9cfabf0 con 0x560bd9db7860 2024-03-28T13:13:34.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.419+0000 7f46baa00640 1 --2- 172.21.15.19:0/415469799 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x560bd9db7860 0x560bd9db7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.420+0000 7f46baa00640 1 --2- 172.21.15.19:0/415469799 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x560bd9db7860 0x560bd9db7d10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28af1d93dec1f5d0 server_cookie=18d7d47ba01f3afb in_seq=0 out_seq=0 2024-03-28T13:13:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.420+0000 7f469fe00640 10 client.6183 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.421+0000 7f469fe00640 1 -- 172.21.15.19:0/415469799 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560bd9cfabf0 con 0x560bd9db7860 2024-03-28T13:13:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.421+0000 7f469fe00640 10 client.6183 handle_command_reply: tid=0 2024-03-28T13:13:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.421+0000 7f46c49b0740 1 client.6183 shutdown 2024-03-28T13:13:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.421+0000 7f46c49b0740 1 -- 172.21.15.19:0/415469799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf940 msgr2=0x560bd9dcd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.421+0000 7f46c49b0740 1 --2- 172.21.15.19:0/415469799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf940 0x560bd9dcd6b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f46a40097a0 tx=0x7f46a4004730 comp rx=0 tx=0).stop 2024-03-28T13:13:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.421+0000 7f46c49b0740 1 -- 172.21.15.19:0/415469799 shutdown_connections 2024-03-28T13:13:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.421+0000 7f46c49b0740 1 --2- 172.21.15.19:0/415469799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9db8c70 0x560bd9db90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.421+0000 7f46c49b0740 1 --2- 172.21.15.19:0/415469799 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x560bd9db7860 0x560bd9db7d10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.421+0000 7f46c49b0740 1 --2- 172.21.15.19:0/415469799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf940 0x560bd9dcd6b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.421+0000 7f46c49b0740 1 --2- 172.21.15.19:0/415469799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf560 0x560bd9ddd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.421+0000 7f469fe00640 0 client.6183 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.421+0000 7f46c49b0740 1 -- 172.21.15.19:0/415469799 >> 172.21.15.19:0/415469799 conn(0x560bd9d82a40 msgr2=0x560bd9d864e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.421+0000 7f46c49b0740 1 -- 172.21.15.19:0/415469799 shutdown_connections 2024-03-28T13:13:34.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.421+0000 7f46c49b0740 1 -- 172.21.15.19:0/415469799 wait complete. 2024-03-28T13:13:34.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.422+0000 7f46c49b0740 20 client.6183 trim_cache size 0 max 16384 2024-03-28T13:13:34.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.461+0000 7f46c49b0740 1 Processor -- start 2024-03-28T13:13:34.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.461+0000 7f46c49b0740 1 -- start start 2024-03-28T13:13:34.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.463+0000 7f46c49b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf560 0x560bd9df00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.463+0000 7f46c49b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf940 0x560bd9db1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.463+0000 7f46c49b0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9db8c70 0x560bd9db2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.463+0000 7f46c49b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc200d70 con 0x560bd9daf560 2024-03-28T13:13:34.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.463+0000 7f46c49b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc0cd260 con 0x560bd9daf940 2024-03-28T13:13:34.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.463+0000 7f46c49b0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f46bc0b81e0 con 0x560bd9db8c70 2024-03-28T13:13:34.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.463+0000 7f46baa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf560 0x560bd9df00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.463+0000 7f46baa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf560 0x560bd9df00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41618/0 (socket says 172.21.15.19:41618) 2024-03-28T13:13:34.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.463+0000 7f46baa00640 1 -- 172.21.15.19:0/1197595976 learned_addr learned my addr 172.21.15.19:0/1197595976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:34.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.464+0000 7f46bb400640 1 --2- 172.21.15.19:0/1197595976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9db8c70 0x560bd9db2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.464+0000 7f46b2000640 1 --2- 172.21.15.19:0/1197595976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf940 0x560bd9db1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.464+0000 7f46baa00640 1 -- 172.21.15.19:0/1197595976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9db8c70 msgr2=0x560bd9db2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.464+0000 7f46baa00640 1 --2- 172.21.15.19:0/1197595976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9db8c70 0x560bd9db2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.464+0000 7f46baa00640 1 -- 172.21.15.19:0/1197595976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf940 msgr2=0x560bd9db1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.464+0000 7f46baa00640 1 --2- 172.21.15.19:0/1197595976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf940 0x560bd9db1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.464+0000 7f46baa00640 1 -- 172.21.15.19:0/1197595976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd9db82a0 con 0x560bd9daf560 2024-03-28T13:13:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.464+0000 7f46bb400640 1 --2- 172.21.15.19:0/1197595976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9db8c70 0x560bd9db2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.464+0000 7f46baa00640 1 --2- 172.21.15.19:0/1197595976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf560 0x560bd9df00f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00a430 tx=0x7f46ac08b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee667a985a5f4042 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.464+0000 7f46aaa00640 1 -- 172.21.15.19:0/1197595976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac061e80 con 0x560bd9daf560 2024-03-28T13:13:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.464+0000 7f46aaa00640 1 -- 172.21.15.19:0/1197595976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ac089930 con 0x560bd9daf560 2024-03-28T13:13:34.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.464+0000 7f46aaa00640 1 -- 172.21.15.19:0/1197595976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac08b0b0 con 0x560bd9daf560 2024-03-28T13:13:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.465+0000 7f46c49b0740 1 -- 172.21.15.19:0/1197595976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf560 msgr2=0x560bd9df00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.465+0000 7f46c49b0740 1 --2- 172.21.15.19:0/1197595976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf560 0x560bd9df00f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00a430 tx=0x7f46ac08b040 comp rx=0 tx=0).stop 2024-03-28T13:13:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.465+0000 7f46c49b0740 1 -- 172.21.15.19:0/1197595976 shutdown_connections 2024-03-28T13:13:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.465+0000 7f46c49b0740 1 --2- 172.21.15.19:0/1197595976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9db8c70 0x560bd9db2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.465+0000 7f46c49b0740 1 --2- 172.21.15.19:0/1197595976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf940 0x560bd9db1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.465+0000 7f46c49b0740 1 --2- 172.21.15.19:0/1197595976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9daf560 0x560bd9df00f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.465+0000 7f46c49b0740 1 -- 172.21.15.19:0/1197595976 >> 172.21.15.19:0/1197595976 conn(0x560bd9d82a40 msgr2=0x560bd9d82f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.465+0000 7f46c49b0740 1 -- 172.21.15.19:0/1197595976 shutdown_connections 2024-03-28T13:13:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.465+0000 7f46c49b0740 1 -- 172.21.15.19:0/1197595976 wait complete. 2024-03-28T13:13:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.466+0000 7f46c49b0740 1 -- start start 2024-03-28T13:13:34.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.466+0000 7f46c49b0740 1 Processor -- start 2024-03-28T13:13:34.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.467+0000 7f46c49b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf560 0x560bd9ddba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.467+0000 7f46c49b0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9daf940 0x560bd9db1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.467+0000 7f46c49b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9db8c70 0x560bd9db2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.467+0000 7f46c49b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560bd9db5920 con 0x560bd9db8c70 2024-03-28T13:13:34.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.467+0000 7f46c49b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f46bc0ccc10 con 0x560bd9daf560 2024-03-28T13:13:34.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.467+0000 7f46c49b0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x560bd9d88740 con 0x560bd9daf940 2024-03-28T13:13:34.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.467+0000 7f46c49b0740 10 client.0 _ll_register_callbacks cb 0x7f46a4009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:13:34.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.467+0000 7f46bb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9db8c70 0x560bd9db2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.467+0000 7f46bb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9db8c70 0x560bd9db2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41624/0 (socket says 172.21.15.19:41624) 2024-03-28T13:13:34.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.467+0000 7f46bb400640 1 -- 172.21.15.19:0/152790885 learned_addr learned my addr 172.21.15.19:0/152790885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:34.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.468+0000 7f46b2000640 1 --2- 172.21.15.19:0/152790885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9daf940 0x560bd9db1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.468+0000 7f46baa00640 1 --2- 172.21.15.19:0/152790885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf560 0x560bd9ddba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.468+0000 7f46bb400640 1 -- 172.21.15.19:0/152790885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9daf940 msgr2=0x560bd9db1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.468+0000 7f46bb400640 1 --2- 172.21.15.19:0/152790885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9daf940 0x560bd9db1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.468+0000 7f46bb400640 1 -- 172.21.15.19:0/152790885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf560 msgr2=0x560bd9ddba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.468+0000 7f46bb400640 1 --2- 172.21.15.19:0/152790885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf560 0x560bd9ddba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.468+0000 7f46bb400640 1 -- 172.21.15.19:0/152790885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bd9d891e0 con 0x560bd9db8c70 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.468+0000 7f46b2000640 1 --2- 172.21.15.19:0/152790885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9daf940 0x560bd9db1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.468+0000 7f46baa00640 1 --2- 172.21.15.19:0/152790885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf560 0x560bd9ddba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.468+0000 7f46bb400640 1 --2- 172.21.15.19:0/152790885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9db8c70 0x560bd9db2330 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009160 tx=0x7f46b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cede847f2a22ae65 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.468+0000 7f469fe00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.468+0000 7f469fe00640 1 -- 172.21.15.19:0/152790885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4029060 con 0x560bd9db8c70 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.468+0000 7f469fe00640 1 -- 172.21.15.19:0/152790885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f46b4024650 con 0x560bd9db8c70 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.468+0000 7f46c49b0740 1 -- 172.21.15.19:0/152790885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46ac089930 con 0x560bd9db8c70 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.469+0000 7f469fe00640 1 -- 172.21.15.19:0/152790885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f46b40096a0 con 0x560bd9db8c70 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.469+0000 7f46c49b0740 10 client.6189 fetch_fsmap learned FSMap version 35 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.469+0000 7f46c49b0740 1 -- 172.21.15.19:0/152790885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x560bd9de29c0 con 0x560bd9db8c70 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.469+0000 7f469fe00640 1 -- 172.21.15.19:0/152790885 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ac089930 con 0x560bd9db8c70 2024-03-28T13:13:34.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.469+0000 7f469fe00640 1 -- 172.21.15.19:0/152790885 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f46b4024850 con 0x560bd9db8c70 2024-03-28T13:13:34.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.469+0000 7f46c49b0740 10 client.6189 fetch_fsmap finished waiting for FSMap version 35 2024-03-28T13:13:34.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.469+0000 7f46c49b0740 10 client.6189 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-03-28T13:13:34.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.469+0000 7f46c49b0740 20 client.6189 populate_metadata read hostname 'smithi019' 2024-03-28T13:13:34.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.469+0000 7f46c49b0740 1 --2- 172.21.15.19:0/152790885 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x560bd9e46240 0x560bd9dac130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.469+0000 7f46c49b0740 4 client.6189 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:13:34.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.469+0000 7f46c49b0740 1 -- 172.21.15.19:0/152790885 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560bd9cf4f40 con 0x560bd9e46240 2024-03-28T13:13:34.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.470+0000 7f46baa00640 1 --2- 172.21.15.19:0/152790885 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x560bd9e46240 0x560bd9dac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.470+0000 7f46baa00640 1 --2- 172.21.15.19:0/152790885 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x560bd9e46240 0x560bd9dac130 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a550c5281e620c9 server_cookie=f18fb5d0dd984b55 in_seq=0 out_seq=0 2024-03-28T13:13:34.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.470+0000 7f469fe00640 10 client.6189 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:34.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.470+0000 7f469fe00640 1 -- 172.21.15.19:0/152790885 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560bd9cf4f40 con 0x560bd9e46240 2024-03-28T13:13:34.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.470+0000 7f469fe00640 10 client.6189 handle_command_reply: tid=0 2024-03-28T13:13:34.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.470+0000 7f46c49b0740 1 client.6189 shutdown 2024-03-28T13:13:34.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.470+0000 7f46c49b0740 1 -- 172.21.15.19:0/152790885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9db8c70 msgr2=0x560bd9db2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.470+0000 7f46c49b0740 1 --2- 172.21.15.19:0/152790885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9db8c70 0x560bd9db2330 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f46b4009160 tx=0x7f46b40046c0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.471+0000 7f46c49b0740 1 -- 172.21.15.19:0/152790885 shutdown_connections 2024-03-28T13:13:34.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.471+0000 7f46c49b0740 1 --2- 172.21.15.19:0/152790885 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x560bd9e46240 0x560bd9dac130 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.471+0000 7f46c49b0740 1 --2- 172.21.15.19:0/152790885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560bd9db8c70 0x560bd9db2330 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.471+0000 7f46c49b0740 1 --2- 172.21.15.19:0/152790885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560bd9daf940 0x560bd9db1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.471+0000 7f46c49b0740 1 --2- 172.21.15.19:0/152790885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560bd9daf560 0x560bd9ddba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.471+0000 7f469fe00640 0 client.6189 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:34.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.471+0000 7f46c49b0740 1 -- 172.21.15.19:0/152790885 >> 172.21.15.19:0/152790885 conn(0x560bd9d82a40 msgr2=0x560bd9d7db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:34.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.471+0000 7f46c49b0740 1 -- 172.21.15.19:0/152790885 shutdown_connections 2024-03-28T13:13:34.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.471+0000 7f46c49b0740 1 -- 172.21.15.19:0/152790885 wait complete. 2024-03-28T13:13:34.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.471+0000 7f46c49b0740 20 client.6189 trim_cache size 0 max 16384 2024-03-28T13:13:34.474 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:13:34.475 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-03-28T13:13:34.475 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-03-28T13:13:34.475 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:13:34.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.472+0000 7f46c1c00640 1 -- 172.21.15.19:0/335122390 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f469806c530 msgr2=0x7f469806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.472+0000 7f46c1c00640 1 --2- 172.21.15.19:0/335122390 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f469806c530 0x7f469806e9f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x560bd9d85700 tx=0x7f46b4009570 comp rx=0 tx=0).stop 2024-03-28T13:13:34.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.472+0000 7f46c1c00640 1 -- 172.21.15.19:0/335122390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46bc0b4db0 msgr2=0x7f46bc1f86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.472+0000 7f46c1c00640 1 --2- 172.21.15.19:0/335122390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46bc0b4db0 0x7f46bc1f86c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00c760 tx=0x7f46ac00cc20 comp rx=0 tx=0).stop 2024-03-28T13:13:34.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.472+0000 7f46c1c00640 1 -- 172.21.15.19:0/335122390 shutdown_connections 2024-03-28T13:13:34.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.472+0000 7f46c1c00640 1 --2- 172.21.15.19:0/335122390 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f469806c530 0x7f469806e9f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.472+0000 7f46c1c00640 1 --2- 172.21.15.19:0/335122390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46bc161f60 0x7f46bc158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.472+0000 7f46c1c00640 1 --2- 172.21.15.19:0/335122390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46bc0d1580 0x7f46bc0b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.472+0000 7f46c1c00640 1 --2- 172.21.15.19:0/335122390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46bc0b4db0 0x7f46bc1f86c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.472+0000 7f46c1c00640 1 -- 172.21.15.19:0/335122390 >> 172.21.15.19:0/335122390 conn(0x7f46bc08bfb0 msgr2=0x7f46bc160f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:34.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.472+0000 7f46c1c00640 1 -- 172.21.15.19:0/335122390 shutdown_connections 2024-03-28T13:13:34.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.472+0000 7f46c1c00640 1 -- 172.21.15.19:0/335122390 wait complete. 2024-03-28T13:13:34.491 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T13:13:34.491 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:13:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.703+0000 7f4f83000640 1 Processor -- start 2024-03-28T13:13:34.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.703+0000 7f4f83000640 1 -- start start 2024-03-28T13:13:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.705+0000 7f4f83000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f7c0d0e90 0x7f4f7c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.705+0000 7f4f83000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7c15b6f0 con 0x7f4f7c0727f0 2024-03-28T13:13:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.705+0000 7f4f83000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7c072e80 con 0x7f4f7c15b310 2024-03-28T13:13:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.705+0000 7f4f83000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7c072c50 con 0x7f4f7c0d0e90 2024-03-28T13:13:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.705+0000 7f4f7b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4f7c0727f0 0x7f4f7c0d0780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57940/0 (socket says 172.21.15.19:57940) 2024-03-28T13:13:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.705+0000 7f4f7b400640 1 -- 172.21.15.19:0/2849487948 learned_addr learned my addr 172.21.15.19:0/2849487948 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:13:34.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.706+0000 7f4f81000640 1 --2- 172.21.15.19:0/2849487948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f7c0d0e90 0x7f4f7c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.706+0000 7f4f7aa00640 1 -- 172.21.15.19:0/2849487948 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4147793572 0 0) 0x7f4f7c15b6f0 con 0x7f4f7c0727f0 2024-03-28T13:13:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.706+0000 7f4f7aa00640 1 -- 172.21.15.19:0/2849487948 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f60003520 con 0x7f4f7c0727f0 2024-03-28T13:13:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.706+0000 7f4f81000640 1 -- 172.21.15.19:0/2849487948 >> v1:172.21.15.188:6789/0 conn(0x7f4f7c15b310 legacy=0x7f4f7c15f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.706+0000 7f4f7aa00640 1 -- 172.21.15.19:0/2849487948 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3992445652 0 0) 0x7f4f60003520 con 0x7f4f7c0727f0 2024-03-28T13:13:34.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.706+0000 7f4f81000640 1 -- 172.21.15.19:0/2849487948 >> v1:172.21.15.19:6789/0 conn(0x7f4f7c0727f0 legacy=0x7f4f7c0d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.706+0000 7f4f81000640 1 -- 172.21.15.19:0/2849487948 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7c073160 con 0x7f4f7c0d0e90 2024-03-28T13:13:34.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.708+0000 7f4f81000640 1 --2- 172.21.15.19:0/2849487948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f7c0d0e90 0x7f4f7c0b4f90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009dc0 tx=0x7f4f7402bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d8d3a76eff4160 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:34.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.709+0000 7f4f7aa00640 1 -- 172.21.15.19:0/2849487948 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68001670 con 0x7f4f7c0d0e90 2024-03-28T13:13:34.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.709+0000 7f4f7aa00640 1 -- 172.21.15.19:0/2849487948 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f7402d660 con 0x7f4f7c0d0e90 2024-03-28T13:13:34.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.709+0000 7f4f7aa00640 1 -- 172.21.15.19:0/2849487948 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7402da10 con 0x7f4f7c0d0e90 2024-03-28T13:13:34.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.709+0000 7f4f83000640 1 -- 172.21.15.19:0/2849487948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f7c0d0e90 msgr2=0x7f4f7c0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.709+0000 7f4f83000640 1 --2- 172.21.15.19:0/2849487948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f7c0d0e90 0x7f4f7c0b4f90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009dc0 tx=0x7f4f7402bb20 comp rx=0 tx=0).stop 2024-03-28T13:13:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.710+0000 7f4f83000640 1 -- 172.21.15.19:0/2849487948 shutdown_connections 2024-03-28T13:13:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.710+0000 7f4f83000640 1 --2- 172.21.15.19:0/2849487948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f7c0d0e90 0x7f4f7c0b4f90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.710+0000 7f4f83000640 1 -- 172.21.15.19:0/2849487948 >> 172.21.15.19:0/2849487948 conn(0x7f4f7c06b990 msgr2=0x7f4f7c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.710+0000 7f4f83000640 1 -- 172.21.15.19:0/2849487948 shutdown_connections 2024-03-28T13:13:34.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.710+0000 7f4f83000640 1 -- 172.21.15.19:0/2849487948 wait complete. 2024-03-28T13:13:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.711+0000 7f4f83000640 1 Processor -- start 2024-03-28T13:13:34.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.712+0000 7f4f83000640 1 -- start start 2024-03-28T13:13:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.712+0000 7f4f83000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f7c0727f0 0x7f4f7c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.712+0000 7f4f83000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f7c0d0e90 0x7f4f7c14f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.713+0000 7f4f83000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f7c15b310 0x7f4f7c1f9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.713+0000 7f4f83000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c073440 con 0x7f4f7c15b310 2024-03-28T13:13:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.713+0000 7f4f83000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7c15f2c0 con 0x7f4f7c0727f0 2024-03-28T13:13:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.713+0000 7f4f83000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7c0adc80 con 0x7f4f7c0d0e90 2024-03-28T13:13:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.713+0000 7f4f81000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f7c15b310 0x7f4f7c1f9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.713+0000 7f4f81000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f7c15b310 0x7f4f7c1f9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41634/0 (socket says 172.21.15.19:41634) 2024-03-28T13:13:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.713+0000 7f4f81000640 1 -- 172.21.15.19:0/3514334616 learned_addr learned my addr 172.21.15.19:0/3514334616 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.713+0000 7f4f7be00640 1 --2- 172.21.15.19:0/3514334616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f7c0727f0 0x7f4f7c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.713+0000 7f4f7b400640 1 --2- 172.21.15.19:0/3514334616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f7c0d0e90 0x7f4f7c14f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.713+0000 7f4f81000640 1 -- 172.21.15.19:0/3514334616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f7c0d0e90 msgr2=0x7f4f7c14f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.713+0000 7f4f81000640 1 --2- 172.21.15.19:0/3514334616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f7c0d0e90 0x7f4f7c14f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.713+0000 7f4f81000640 1 -- 172.21.15.19:0/3514334616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f7c0727f0 msgr2=0x7f4f7c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.713+0000 7f4f81000640 1 --2- 172.21.15.19:0/3514334616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f7c0727f0 0x7f4f7c0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.714+0000 7f4f81000640 1 -- 172.21.15.19:0/3514334616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f7c0cdde0 con 0x7f4f7c15b310 2024-03-28T13:13:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.714+0000 7f4f81000640 1 --2- 172.21.15.19:0/3514334616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f7c15b310 0x7f4f7c1f9ca0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009eb0 tx=0x7f4f74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.714+0000 7f4f73e00640 1 -- 172.21.15.19:0/3514334616 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74047070 con 0x7f4f7c15b310 2024-03-28T13:13:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.714+0000 7f4f73e00640 1 -- 172.21.15.19:0/3514334616 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f74004250 con 0x7f4f7c15b310 2024-03-28T13:13:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.714+0000 7f4f83000640 1 -- 172.21.15.19:0/3514334616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f7c0ce080 con 0x7f4f7c15b310 2024-03-28T13:13:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.714+0000 7f4f73e00640 1 -- 172.21.15.19:0/3514334616 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f74009450 con 0x7f4f7c15b310 2024-03-28T13:13:34.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.714+0000 7f4f83000640 1 -- 172.21.15.19:0/3514334616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f7c0ce640 con 0x7f4f7c15b310 2024-03-28T13:13:34.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.716+0000 7f4f83000640 1 -- 172.21.15.19:0/3514334616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f7c0685c0 con 0x7f4f7c15b310 2024-03-28T13:13:34.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.717+0000 7f4f73e00640 1 -- 172.21.15.19:0/3514334616 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4f7404a020 con 0x7f4f7c15b310 2024-03-28T13:13:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.718+0000 7f4f73e00640 1 --2- 172.21.15.19:0/3514334616 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4f5c06c5b0 0x7f4f5c06ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:34.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.718+0000 7f4f73e00640 1 -- 172.21.15.19:0/3514334616 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f4f740ad070 con 0x7f4f7c15b310 2024-03-28T13:13:34.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.719+0000 7f4f7be00640 1 --2- 172.21.15.19:0/3514334616 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4f5c06c5b0 0x7f4f5c06ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:34.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.720+0000 7f4f7be00640 1 --2- 172.21.15.19:0/3514334616 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4f5c06c5b0 0x7f4f5c06ea70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c000f60 tx=0x7f4f6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:34.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.723+0000 7f4f73e00640 1 -- 172.21.15.19:0/3514334616 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4f74033030 con 0x7f4f7c15b310 2024-03-28T13:13:34.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.969+0000 7f4f83000640 1 -- 172.21.15.19:0/3514334616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f7c15ec20 con 0x7f4f7c15b310 2024-03-28T13:13:34.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.970+0000 7f4f73e00640 1 -- 172.21.15.19:0/3514334616 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5187 (secure 0 0 0) 0x7f4f7407b630 con 0x7f4f7c15b310 2024-03-28T13:13:34.970 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:13:34.970 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1607201514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1607201514},{"type":"v1","addr":"172.21.15.188:6837","nonce":1607201514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1206740614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1206740614},{"type":"v1","addr":"172.21.15.188:6835","nonce":1206740614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2018671345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2018671345},{"type":"v1","addr":"172.21.15.19:6834","nonce":2018671345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:12:52.947573+0000","modified":"2024-03-28T13:13:09.546366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":7,"addr":"172.21.15.19:6835/3341654519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3341654519},{"type":"v1","addr":"172.21.15.19:6835","nonce":3341654519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":4}]} 2024-03-28T13:13:34.971 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 35 2024-03-28T13:13:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.974+0000 7f4f83000640 1 -- 172.21.15.19:0/3514334616 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4f5c06c5b0 msgr2=0x7f4f5c06ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.974+0000 7f4f83000640 1 --2- 172.21.15.19:0/3514334616 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4f5c06c5b0 0x7f4f5c06ea70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c000f60 tx=0x7f4f6c009040 comp rx=0 tx=0).stop 2024-03-28T13:13:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.974+0000 7f4f83000640 1 -- 172.21.15.19:0/3514334616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f7c15b310 msgr2=0x7f4f7c1f9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.974+0000 7f4f83000640 1 --2- 172.21.15.19:0/3514334616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f7c15b310 0x7f4f7c1f9ca0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009eb0 tx=0x7f4f74004a40 comp rx=0 tx=0).stop 2024-03-28T13:13:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.974+0000 7f4f83000640 1 -- 172.21.15.19:0/3514334616 shutdown_connections 2024-03-28T13:13:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.974+0000 7f4f83000640 1 --2- 172.21.15.19:0/3514334616 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4f5c06c5b0 0x7f4f5c06ea70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.974+0000 7f4f83000640 1 --2- 172.21.15.19:0/3514334616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f7c15b310 0x7f4f7c1f9ca0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.974+0000 7f4f83000640 1 --2- 172.21.15.19:0/3514334616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f7c0d0e90 0x7f4f7c14f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.974+0000 7f4f83000640 1 --2- 172.21.15.19:0/3514334616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f7c0727f0 0x7f4f7c0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.974+0000 7f4f83000640 1 -- 172.21.15.19:0/3514334616 >> 172.21.15.19:0/3514334616 conn(0x7f4f7c06b990 msgr2=0x7f4f7c0b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:34.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.974+0000 7f4f83000640 1 -- 172.21.15.19:0/3514334616 shutdown_connections 2024-03-28T13:13:34.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:34.974+0000 7f4f83000640 1 -- 172.21.15.19:0/3514334616 wait complete. 2024-03-28T13:13:34.994 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-03-28T13:13:35.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.214+0000 7f33f5a00640 1 Processor -- start 2024-03-28T13:13:35.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.214+0000 7f33f5a00640 1 -- start start 2024-03-28T13:13:35.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.216+0000 7f33f5a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f33f015e460 0x7f33f00b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.216+0000 7f33f5a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33f00d2da0 con 0x7f33f00d3e40 2024-03-28T13:13:35.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.216+0000 7f33f5a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33f00d3080 con 0x7f33f00cef20 2024-03-28T13:13:35.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.216+0000 7f33f5a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f33f00d3360 con 0x7f33f015e460 2024-03-28T13:13:35.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.216+0000 7f33ef400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f33f00d3e40 0x7f33f00d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57952/0 (socket says 172.21.15.19:57952) 2024-03-28T13:13:35.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.216+0000 7f33ef400640 1 -- 172.21.15.19:0/3807135213 learned_addr learned my addr 172.21.15.19:0/3807135213 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:13:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33efe00640 1 --2- 172.21.15.19:0/3807135213 >> v2:172.21.15.188:3300/0 conn(0x7f33f015e460 0x7f33f00b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33eea00640 1 -- 172.21.15.19:0/3807135213 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 361432965 0 0) 0x7f33f00d2da0 con 0x7f33f00d3e40 2024-03-28T13:13:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33eea00640 1 -- 172.21.15.19:0/3807135213 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33d4003520 con 0x7f33f00d3e40 2024-03-28T13:13:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33eea00640 1 -- 172.21.15.19:0/3807135213 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 926226242 0 0) 0x7f33d4003520 con 0x7f33f00d3e40 2024-03-28T13:13:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33eea00640 1 -- 172.21.15.19:0/3807135213 >> v2:172.21.15.188:3300/0 conn(0x7f33f015e460 msgr2=0x7f33f00b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33eea00640 1 --2- 172.21.15.19:0/3807135213 >> v2:172.21.15.188:3300/0 conn(0x7f33f015e460 0x7f33f00b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33eea00640 1 -- 172.21.15.19:0/3807135213 >> v1:172.21.15.188:6789/0 conn(0x7f33f00cef20 legacy=0x7f33f00b7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33eea00640 1 -- 172.21.15.19:0/3807135213 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f00d3640 con 0x7f33f00d3e40 2024-03-28T13:13:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33eea00640 1 -- 172.21.15.19:0/3807135213 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f33dc0017f0 con 0x7f33f00d3e40 2024-03-28T13:13:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33eea00640 1 -- 172.21.15.19:0/3807135213 >> v1:172.21.15.19:6789/0 conn(0x7f33f00d3e40 legacy=0x7f33f00d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33eea00640 1 --2- 172.21.15.19:0/3807135213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33d4003a00 0x7f33d4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33eea00640 1 --2- 172.21.15.19:0/3807135213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4004800 0x7f33d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33eea00640 1 --2- 172.21.15.19:0/3807135213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33d40051c0 0x7f33d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33eea00640 1 -- 172.21.15.19:0/3807135213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33f00d3360 con 0x7f33d4004800 2024-03-28T13:13:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33eea00640 1 -- 172.21.15.19:0/3807135213 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f33d400c0c0 con 0x7f33d4003a00 2024-03-28T13:13:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.217+0000 7f33eea00640 1 -- 172.21.15.19:0/3807135213 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f33d400c2c0 con 0x7f33d40051c0 2024-03-28T13:13:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.218+0000 7f33ef400640 1 --2- 172.21.15.19:0/3807135213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4004800 0x7f33d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.218+0000 7f33efe00640 1 --2- 172.21.15.19:0/3807135213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33d4003a00 0x7f33d4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.218+0000 7f33f5000640 1 --2- 172.21.15.19:0/3807135213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33d40051c0 0x7f33d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.218+0000 7f33ef400640 1 -- 172.21.15.19:0/3807135213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33d40051c0 msgr2=0x7f33d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.218+0000 7f33ef400640 1 --2- 172.21.15.19:0/3807135213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33d40051c0 0x7f33d400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.218+0000 7f33ef400640 1 -- 172.21.15.19:0/3807135213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33d4003a00 msgr2=0x7f33d4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.218+0000 7f33ef400640 1 --2- 172.21.15.19:0/3807135213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33d4003a00 0x7f33d4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.218+0000 7f33ef400640 1 -- 172.21.15.19:0/3807135213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d4003520 con 0x7f33d4004800 2024-03-28T13:13:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.218+0000 7f33f5000640 1 -- 172.21.15.19:0/3807135213 reap_dead start 2024-03-28T13:13:35.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.219+0000 7f33ef400640 1 --2- 172.21.15.19:0/3807135213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4004800 0x7f33d4004c80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00c0e0 tx=0x7f33d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cfe55eb477d1a70 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:35.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.220+0000 7f33eea00640 1 -- 172.21.15.19:0/3807135213 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc007c30 con 0x7f33d4004800 2024-03-28T13:13:35.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.220+0000 7f33eea00640 1 -- 172.21.15.19:0/3807135213 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33dc030070 con 0x7f33d4004800 2024-03-28T13:13:35.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.220+0000 7f33eea00640 1 -- 172.21.15.19:0/3807135213 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc007530 con 0x7f33d4004800 2024-03-28T13:13:35.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.220+0000 7f33f5a00640 1 -- 172.21.15.19:0/3807135213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4004800 msgr2=0x7f33d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.220+0000 7f33f5a00640 1 --2- 172.21.15.19:0/3807135213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4004800 0x7f33d4004c80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00c0e0 tx=0x7f33d4005830 comp rx=0 tx=0).stop 2024-03-28T13:13:35.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.220+0000 7f33f5a00640 1 -- 172.21.15.19:0/3807135213 shutdown_connections 2024-03-28T13:13:35.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.220+0000 7f33f5a00640 1 --2- 172.21.15.19:0/3807135213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33d4004800 0x7f33d4004c80 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.220+0000 7f33f5a00640 1 -- 172.21.15.19:0/3807135213 >> 172.21.15.19:0/3807135213 conn(0x7f33f00678b0 msgr2=0x7f33f00d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:35.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.220+0000 7f33f5a00640 1 -- 172.21.15.19:0/3807135213 shutdown_connections 2024-03-28T13:13:35.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.220+0000 7f33f5a00640 1 -- 172.21.15.19:0/3807135213 wait complete. 2024-03-28T13:13:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.222+0000 7f33f5a00640 1 Processor -- start 2024-03-28T13:13:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.222+0000 7f33f5a00640 1 -- start start 2024-03-28T13:13:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.223+0000 7f33f5a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33d4004800 0x7f33f0150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.223+0000 7f33f5a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f00cc690 0x7f33f00cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.223+0000 7f33f5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f01e35e0 0x7f33f00c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.223+0000 7f33f5a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33f00b9fa0 con 0x7f33f01e35e0 2024-03-28T13:13:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.223+0000 7f33f5a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f33f00b9e20 con 0x7f33d4004800 2024-03-28T13:13:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.223+0000 7f33f5a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f33f00ba120 con 0x7f33f00cc690 2024-03-28T13:13:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.224+0000 7f33f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f01e35e0 0x7f33f00c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.224+0000 7f33f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f01e35e0 0x7f33f00c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41644/0 (socket says 172.21.15.19:41644) 2024-03-28T13:13:35.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.224+0000 7f33f5000640 1 -- 172.21.15.19:0/4229864612 learned_addr learned my addr 172.21.15.19:0/4229864612 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.224+0000 7f33efe00640 1 --2- 172.21.15.19:0/4229864612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33d4004800 0x7f33f0150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.224+0000 7f33ef400640 1 --2- 172.21.15.19:0/4229864612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f00cc690 0x7f33f00cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.224+0000 7f33f5000640 1 -- 172.21.15.19:0/4229864612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f00cc690 msgr2=0x7f33f00cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.224+0000 7f33f5000640 1 --2- 172.21.15.19:0/4229864612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f00cc690 0x7f33f00cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.224+0000 7f33f5000640 1 -- 172.21.15.19:0/4229864612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33d4004800 msgr2=0x7f33f0150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.224+0000 7f33f5000640 1 --2- 172.21.15.19:0/4229864612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33d4004800 0x7f33f0150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.224+0000 7f33f5000640 1 -- 172.21.15.19:0/4229864612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f01e3a60 con 0x7f33f01e35e0 2024-03-28T13:13:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.225+0000 7f33f5000640 1 --2- 172.21.15.19:0/4229864612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f01e35e0 0x7f33f00c8dd0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f33f00d1fb0 tx=0x7f33e800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.225+0000 7f33e7e00640 1 -- 172.21.15.19:0/4229864612 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8018070 con 0x7f33f01e35e0 2024-03-28T13:13:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.225+0000 7f33e7e00640 1 -- 172.21.15.19:0/4229864612 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e8003d80 con 0x7f33f01e35e0 2024-03-28T13:13:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.225+0000 7f33f5a00640 1 -- 172.21.15.19:0/4229864612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f00ccc50 con 0x7f33f01e35e0 2024-03-28T13:13:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.225+0000 7f33e7e00640 1 -- 172.21.15.19:0/4229864612 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e80052b0 con 0x7f33f01e35e0 2024-03-28T13:13:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.225+0000 7f33f5a00640 1 -- 172.21.15.19:0/4229864612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f00cd210 con 0x7f33f01e35e0 2024-03-28T13:13:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.227+0000 7f33e7e00640 1 -- 172.21.15.19:0/4229864612 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f33e801b070 con 0x7f33f01e35e0 2024-03-28T13:13:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.228+0000 7f33e7e00640 1 --2- 172.21.15.19:0/4229864612 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33d006c560 0x7f33d006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.228+0000 7f33e7e00640 1 -- 172.21.15.19:0/4229864612 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f33e808ae20 con 0x7f33f01e35e0 2024-03-28T13:13:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.229+0000 7f33efe00640 1 --2- 172.21.15.19:0/4229864612 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33d006c560 0x7f33d006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.229+0000 7f33f9d87740 1 Processor -- start 2024-03-28T13:13:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.229+0000 7f33f9d87740 1 -- start start 2024-03-28T13:13:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.230+0000 7f33f9d87740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8cd8a0 0x55abcd8cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.230+0000 7f33f9d87740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd89d8c0 0x55abcd8cab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.230+0000 7f33f9d87740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8a1420 0x55abcd8cb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.230+0000 7f33f9d87740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55abcd8075a0 con 0x55abcd8a1420 2024-03-28T13:13:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.230+0000 7f33f9d87740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55abcd7af390 con 0x55abcd89d8c0 2024-03-28T13:13:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.230+0000 7f33f9d87740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55abcd803550 con 0x55abcd8cd8a0 2024-03-28T13:13:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.230+0000 7f33efe00640 1 --2- 172.21.15.19:0/4229864612 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33d006c560 0x7f33d006ea20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f33e00099d0 tx=0x7f33e0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.231+0000 7f33f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8a1420 0x55abcd8cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.231+0000 7f33f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8a1420 0x55abcd8cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41650/0 (socket says 172.21.15.19:41650) 2024-03-28T13:13:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.231+0000 7f33f5000640 1 -- 172.21.15.19:0/2236314687 learned_addr learned my addr 172.21.15.19:0/2236314687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.231+0000 7f33efe00640 1 --2- 172.21.15.19:0/2236314687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8cd8a0 0x55abcd8cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.231+0000 7f33ef400640 1 --2- 172.21.15.19:0/2236314687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd89d8c0 0x55abcd8cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.231+0000 7f33f5000640 1 -- 172.21.15.19:0/2236314687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8cd8a0 msgr2=0x55abcd8cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.231+0000 7f33f5000640 1 --2- 172.21.15.19:0/2236314687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8cd8a0 0x55abcd8cdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.231+0000 7f33f5000640 1 -- 172.21.15.19:0/2236314687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd89d8c0 msgr2=0x55abcd8cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.231+0000 7f33f5000640 1 --2- 172.21.15.19:0/2236314687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd89d8c0 0x55abcd8cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.231+0000 7f33f5000640 1 -- 172.21.15.19:0/2236314687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abcd5e77c0 con 0x55abcd8a1420 2024-03-28T13:13:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.231+0000 7f33f5000640 1 --2- 172.21.15.19:0/2236314687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8a1420 0x55abcd8cb050 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f33e8061310 tx=0x7f33e8059410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97902fea9867837 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.232+0000 7f33e4c00640 1 -- 172.21.15.19:0/2236314687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8061840 con 0x55abcd8a1420 2024-03-28T13:13:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.232+0000 7f33e4c00640 1 -- 172.21.15.19:0/2236314687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e8056ed0 con 0x55abcd8a1420 2024-03-28T13:13:35.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.232+0000 7f33e4c00640 1 -- 172.21.15.19:0/2236314687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8082bd0 con 0x55abcd8a1420 2024-03-28T13:13:35.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.232+0000 7f33f9d87740 1 -- 172.21.15.19:0/2236314687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8a1420 msgr2=0x55abcd8cb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.232+0000 7f33f9d87740 1 --2- 172.21.15.19:0/2236314687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8a1420 0x55abcd8cb050 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f33e8061310 tx=0x7f33e8059410 comp rx=0 tx=0).stop 2024-03-28T13:13:35.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.232+0000 7f33f9d87740 1 -- 172.21.15.19:0/2236314687 shutdown_connections 2024-03-28T13:13:35.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.232+0000 7f33f9d87740 1 --2- 172.21.15.19:0/2236314687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8a1420 0x55abcd8cb050 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.232+0000 7f33f9d87740 1 --2- 172.21.15.19:0/2236314687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd89d8c0 0x55abcd8cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.232+0000 7f33f9d87740 1 --2- 172.21.15.19:0/2236314687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8cd8a0 0x55abcd8cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.232+0000 7f33f9d87740 1 -- 172.21.15.19:0/2236314687 >> 172.21.15.19:0/2236314687 conn(0x55abcd89fb30 msgr2=0x55abcd8e86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:35.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.232+0000 7f33f9d87740 1 -- 172.21.15.19:0/2236314687 shutdown_connections 2024-03-28T13:13:35.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.232+0000 7f33f9d87740 1 -- 172.21.15.19:0/2236314687 wait complete. 2024-03-28T13:13:35.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.233+0000 7f33f9d87740 1 -- start start 2024-03-28T13:13:35.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.233+0000 7f33f9d87740 1 Processor -- start 2024-03-28T13:13:35.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.234+0000 7f33f9d87740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8a1420 0x55abcd8e8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.234+0000 7f33f9d87740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8cd8a0 0x55abcd8d1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.234+0000 7f33f9d87740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8ce4f0 0x55abcd8b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.234+0000 7f33f9d87740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55abcd8a25d0 con 0x55abcd8cd8a0 2024-03-28T13:13:35.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.235+0000 7f33f9d87740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55abcd7c2780 con 0x55abcd8ce4f0 2024-03-28T13:13:35.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.235+0000 7f33f9d87740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55abcd8a19c0 con 0x55abcd8a1420 2024-03-28T13:13:35.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.235+0000 7f33f9d87740 10 client.0 _ll_register_callbacks cb 0x55abcd7fefb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:13:35.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.235+0000 7f33ef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8cd8a0 0x55abcd8d1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.235+0000 7f33ef400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8cd8a0 0x55abcd8d1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41660/0 (socket says 172.21.15.19:41660) 2024-03-28T13:13:35.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.235+0000 7f33ef400640 1 -- 172.21.15.19:0/190931148 learned_addr learned my addr 172.21.15.19:0/190931148 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:35.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.235+0000 7f33efe00640 1 --2- 172.21.15.19:0/190931148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8a1420 0x55abcd8e8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.235+0000 7f33f5000640 1 --2- 172.21.15.19:0/190931148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8ce4f0 0x55abcd8b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.235+0000 7f33ef400640 1 -- 172.21.15.19:0/190931148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8a1420 msgr2=0x55abcd8e8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:35.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.235+0000 7f33ef400640 1 --2- 172.21.15.19:0/190931148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8a1420 0x55abcd8e8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.235+0000 7f33ef400640 1 -- 172.21.15.19:0/190931148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8ce4f0 msgr2=0x55abcd8b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:35.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.235+0000 7f33ef400640 1 --2- 172.21.15.19:0/190931148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8ce4f0 0x55abcd8b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.235+0000 7f33ef400640 1 -- 172.21.15.19:0/190931148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abcd647f50 con 0x55abcd8cd8a0 2024-03-28T13:13:35.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.236+0000 7f33ef400640 1 --2- 172.21.15.19:0/190931148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8cd8a0 0x55abcd8d1480 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f33dc000ef0 tx=0x7f33dc02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac46cb4e06bce45 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:35.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.236+0000 7f33d9600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:13:35.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.236+0000 7f33d9600640 1 -- 172.21.15.19:0/190931148 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33dc02cbe0 con 0x55abcd8cd8a0 2024-03-28T13:13:35.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.236+0000 7f33d9600640 1 -- 172.21.15.19:0/190931148 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f33dc02cd80 con 0x55abcd8cd8a0 2024-03-28T13:13:35.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.236+0000 7f33f9d87740 1 -- 172.21.15.19:0/190931148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e8056ed0 con 0x55abcd8cd8a0 2024-03-28T13:13:35.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.236+0000 7f33d9600640 1 -- 172.21.15.19:0/190931148 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f33dc04d050 con 0x55abcd8cd8a0 2024-03-28T13:13:35.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.236+0000 7f33f9d87740 10 client.6207 fetch_fsmap learned FSMap version 35 2024-03-28T13:13:35.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.236+0000 7f33f9d87740 1 -- 172.21.15.19:0/190931148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55abcd4db0b0 con 0x55abcd8cd8a0 2024-03-28T13:13:35.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.237+0000 7f33d9600640 1 -- 172.21.15.19:0/190931148 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e8056ed0 con 0x55abcd8cd8a0 2024-03-28T13:13:35.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.237+0000 7f33d9600640 1 -- 172.21.15.19:0/190931148 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f33dc050020 con 0x55abcd8cd8a0 2024-03-28T13:13:35.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.237+0000 7f33f9d87740 10 client.6207 fetch_fsmap finished waiting for FSMap version 35 2024-03-28T13:13:35.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.237+0000 7f33f9d87740 10 client.6207 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:13:35.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.237+0000 7f33f9d87740 20 client.6207 populate_metadata read hostname 'smithi019' 2024-03-28T13:13:35.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.237+0000 7f33f9d87740 1 --2- 172.21.15.19:0/190931148 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55abcd8fbc40 0x55abcd8fc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.237+0000 7f33f9d87740 4 client.6207 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:13:35.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.237+0000 7f33f9d87740 1 -- 172.21.15.19:0/190931148 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abcd7db870 con 0x55abcd8fbc40 2024-03-28T13:13:35.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.237+0000 7f33efe00640 1 --2- 172.21.15.19:0/190931148 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55abcd8fbc40 0x55abcd8fc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.238+0000 7f33efe00640 1 --2- 172.21.15.19:0/190931148 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55abcd8fbc40 0x55abcd8fc020 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34b830d47e3b944d server_cookie=33d92adc1db5b013 in_seq=0 out_seq=0 2024-03-28T13:13:35.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.238+0000 7f33d9600640 10 client.6207 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:35.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.238+0000 7f33d9600640 1 -- 172.21.15.19:0/190931148 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55abcd7db870 con 0x55abcd8fbc40 2024-03-28T13:13:35.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.238+0000 7f33d9600640 10 client.6207 handle_command_reply: tid=0 2024-03-28T13:13:35.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.239+0000 7f33f9d87740 1 client.6207 shutdown 2024-03-28T13:13:35.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.239+0000 7f33f9d87740 1 -- 172.21.15.19:0/190931148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8cd8a0 msgr2=0x55abcd8d1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.239+0000 7f33f9d87740 1 --2- 172.21.15.19:0/190931148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8cd8a0 0x55abcd8d1480 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f33dc000ef0 tx=0x7f33dc02e070 comp rx=0 tx=0).stop 2024-03-28T13:13:35.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.239+0000 7f33f9d87740 1 -- 172.21.15.19:0/190931148 shutdown_connections 2024-03-28T13:13:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.239+0000 7f33f9d87740 1 --2- 172.21.15.19:0/190931148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8ce4f0 0x55abcd8b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.239+0000 7f33f9d87740 1 --2- 172.21.15.19:0/190931148 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55abcd8fbc40 0x55abcd8fc020 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.239+0000 7f33f9d87740 1 --2- 172.21.15.19:0/190931148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8cd8a0 0x55abcd8d1480 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.239+0000 7f33f9d87740 1 --2- 172.21.15.19:0/190931148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8a1420 0x55abcd8e8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.239+0000 7f33d9600640 0 client.6207 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.239+0000 7f33f9d87740 1 -- 172.21.15.19:0/190931148 >> 172.21.15.19:0/190931148 conn(0x55abcd89d8c0 msgr2=0x55abcd8e9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.239+0000 7f33f9d87740 1 -- 172.21.15.19:0/190931148 shutdown_connections 2024-03-28T13:13:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.239+0000 7f33f9d87740 1 -- 172.21.15.19:0/190931148 wait complete. 2024-03-28T13:13:35.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.239+0000 7f33f9d87740 20 client.6207 trim_cache size 0 max 16384 2024-03-28T13:13:35.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.269+0000 7f33f9d87740 1 Processor -- start 2024-03-28T13:13:35.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.269+0000 7f33f9d87740 1 -- start start 2024-03-28T13:13:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.271+0000 7f33f9d87740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8a1420 0x55abcd8a3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.271+0000 7f33f9d87740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8cd8a0 0x55abcd8ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.271+0000 7f33f9d87740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8ce4f0 0x55abcd8cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.271+0000 7f33f9d87740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33f0200090 con 0x55abcd8a1420 2024-03-28T13:13:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.271+0000 7f33f9d87740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f33f00cbf80 con 0x55abcd8ce4f0 2024-03-28T13:13:35.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.271+0000 7f33f9d87740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f33f01ffa40 con 0x55abcd8cd8a0 2024-03-28T13:13:35.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.271+0000 7f33efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8a1420 0x55abcd8a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.271+0000 7f33efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8a1420 0x55abcd8a3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41666/0 (socket says 172.21.15.19:41666) 2024-03-28T13:13:35.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.271+0000 7f33efe00640 1 -- 172.21.15.19:0/3867960953 learned_addr learned my addr 172.21.15.19:0/3867960953 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:35.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.272+0000 7f33f5000640 1 --2- 172.21.15.19:0/3867960953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8ce4f0 0x55abcd8cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.272+0000 7f33ef400640 1 --2- 172.21.15.19:0/3867960953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8cd8a0 0x55abcd8ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.272+0000 7f33efe00640 1 -- 172.21.15.19:0/3867960953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8cd8a0 msgr2=0x55abcd8ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:35.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.272+0000 7f33efe00640 1 --2- 172.21.15.19:0/3867960953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8cd8a0 0x55abcd8ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.272+0000 7f33efe00640 1 -- 172.21.15.19:0/3867960953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8ce4f0 msgr2=0x55abcd8cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:35.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.272+0000 7f33efe00640 1 --2- 172.21.15.19:0/3867960953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8ce4f0 0x55abcd8cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.272+0000 7f33efe00640 1 -- 172.21.15.19:0/3867960953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abcd8fb180 con 0x55abcd8a1420 2024-03-28T13:13:35.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.272+0000 7f33f5000640 1 --2- 172.21.15.19:0/3867960953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8ce4f0 0x55abcd8cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:35.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.272+0000 7f33efe00640 1 --2- 172.21.15.19:0/3867960953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8a1420 0x55abcd8a3130 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55abcd8cc0d0 tx=0x7f33e0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940813346c2bce10 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:35.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.272+0000 7f33dbe00640 1 -- 172.21.15.19:0/3867960953 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0015be0 con 0x55abcd8a1420 2024-03-28T13:13:35.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.272+0000 7f33dbe00640 1 -- 172.21.15.19:0/3867960953 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e001b040 con 0x55abcd8a1420 2024-03-28T13:13:35.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.273+0000 7f33dbe00640 1 -- 172.21.15.19:0/3867960953 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e0004020 con 0x55abcd8a1420 2024-03-28T13:13:35.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.273+0000 7f33f9d87740 1 -- 172.21.15.19:0/3867960953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8a1420 msgr2=0x55abcd8a3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.273+0000 7f33f9d87740 1 --2- 172.21.15.19:0/3867960953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8a1420 0x55abcd8a3130 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x55abcd8cc0d0 tx=0x7f33e0015000 comp rx=0 tx=0).stop 2024-03-28T13:13:35.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.273+0000 7f33f9d87740 1 -- 172.21.15.19:0/3867960953 shutdown_connections 2024-03-28T13:13:35.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.273+0000 7f33f9d87740 1 --2- 172.21.15.19:0/3867960953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8ce4f0 0x55abcd8cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.273+0000 7f33f9d87740 1 --2- 172.21.15.19:0/3867960953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8cd8a0 0x55abcd8ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.273+0000 7f33f9d87740 1 --2- 172.21.15.19:0/3867960953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8a1420 0x55abcd8a3130 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.273+0000 7f33f9d87740 1 -- 172.21.15.19:0/3867960953 >> 172.21.15.19:0/3867960953 conn(0x55abcd89d8c0 msgr2=0x55abcd8e8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:35.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.274+0000 7f33f9d87740 1 -- 172.21.15.19:0/3867960953 shutdown_connections 2024-03-28T13:13:35.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.274+0000 7f33f9d87740 1 -- 172.21.15.19:0/3867960953 wait complete. 2024-03-28T13:13:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.274+0000 7f33f9d87740 1 -- start start 2024-03-28T13:13:35.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.274+0000 7f33f9d87740 1 Processor -- start 2024-03-28T13:13:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.275+0000 7f33f9d87740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8a1420 0x55abcd8ebbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.275+0000 7f33f9d87740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8cd8a0 0x55abcd8ec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.276+0000 7f33f9d87740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8ce4f0 0x55abcd90af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.276+0000 7f33f9d87740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55abcd8b2640 con 0x55abcd8ce4f0 2024-03-28T13:13:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.276+0000 7f33f9d87740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55abcd8d19c0 con 0x55abcd8cd8a0 2024-03-28T13:13:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.276+0000 7f33f9d87740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55abcd8ce970 con 0x55abcd8a1420 2024-03-28T13:13:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.276+0000 7f33f9d87740 10 client.0 _ll_register_callbacks cb 0x7f33d40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:13:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.276+0000 7f33f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8ce4f0 0x55abcd90af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.276+0000 7f33f5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8ce4f0 0x55abcd90af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41674/0 (socket says 172.21.15.19:41674) 2024-03-28T13:13:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.276+0000 7f33f5000640 1 -- 172.21.15.19:0/3951660949 learned_addr learned my addr 172.21.15.19:0/3951660949 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.276+0000 7f33efe00640 1 --2- 172.21.15.19:0/3951660949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8a1420 0x55abcd8ebbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.276+0000 7f33ef400640 1 --2- 172.21.15.19:0/3951660949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8cd8a0 0x55abcd8ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.276+0000 7f33f5000640 1 -- 172.21.15.19:0/3951660949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8a1420 msgr2=0x55abcd8ebbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.276+0000 7f33f5000640 1 --2- 172.21.15.19:0/3951660949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8a1420 0x55abcd8ebbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.276+0000 7f33f5000640 1 -- 172.21.15.19:0/3951660949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8cd8a0 msgr2=0x55abcd8ec100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.276+0000 7f33f5000640 1 --2- 172.21.15.19:0/3951660949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8cd8a0 0x55abcd8ec100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.276+0000 7f33f5000640 1 -- 172.21.15.19:0/3951660949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abcd8c6480 con 0x55abcd8ce4f0 2024-03-28T13:13:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.277+0000 7f33f5000640 1 --2- 172.21.15.19:0/3951660949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8ce4f0 0x55abcd90af30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f33e8061310 tx=0x7f33e8015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f136d394c82503d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.277+0000 7f33d9600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:13:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.277+0000 7f33d9600640 1 -- 172.21.15.19:0/3951660949 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e8082dd0 con 0x55abcd8ce4f0 2024-03-28T13:13:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.277+0000 7f33d9600640 1 -- 172.21.15.19:0/3951660949 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f33e8058020 con 0x55abcd8ce4f0 2024-03-28T13:13:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.277+0000 7f33f9d87740 1 -- 172.21.15.19:0/3951660949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33e001b040 con 0x55abcd8ce4f0 2024-03-28T13:13:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.277+0000 7f33d9600640 1 -- 172.21.15.19:0/3951660949 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f33e8058240 con 0x55abcd8ce4f0 2024-03-28T13:13:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.277+0000 7f33f9d87740 10 client.6213 fetch_fsmap learned FSMap version 35 2024-03-28T13:13:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.277+0000 7f33f9d87740 1 -- 172.21.15.19:0/3951660949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55abcd8e9260 con 0x55abcd8ce4f0 2024-03-28T13:13:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.277+0000 7f33d9600640 1 -- 172.21.15.19:0/3951660949 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e001b040 con 0x55abcd8ce4f0 2024-03-28T13:13:35.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.278+0000 7f33d9600640 1 -- 172.21.15.19:0/3951660949 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f33e8054920 con 0x55abcd8ce4f0 2024-03-28T13:13:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.278+0000 7f33f9d87740 10 client.6213 fetch_fsmap finished waiting for FSMap version 35 2024-03-28T13:13:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.278+0000 7f33f9d87740 10 client.6213 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:13:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.278+0000 7f33f9d87740 20 client.6213 populate_metadata read hostname 'smithi019' 2024-03-28T13:13:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.278+0000 7f33f9d87740 1 --2- 172.21.15.19:0/3951660949 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55abcd8c95c0 0x55abcd90b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.278+0000 7f33f9d87740 4 client.6213 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-28T13:13:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.278+0000 7f33f9d87740 1 -- 172.21.15.19:0/3951660949 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55abcd8d0230 con 0x55abcd8c95c0 2024-03-28T13:13:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.278+0000 7f33efe00640 1 --2- 172.21.15.19:0/3951660949 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55abcd8c95c0 0x55abcd90b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.279+0000 7f33d9600640 10 client.6213 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.279+0000 7f33efe00640 1 --2- 172.21.15.19:0/3951660949 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55abcd8c95c0 0x55abcd90b470 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a4645196dc117a server_cookie=c70a17f3b32fe38a in_seq=0 out_seq=0 2024-03-28T13:13:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.279+0000 7f33d9600640 1 -- 172.21.15.19:0/3951660949 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+141 (crc 0 0 0) 0x55abcd8d0230 con 0x55abcd8c95c0 2024-03-28T13:13:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.279+0000 7f33d9600640 10 client.6213 handle_command_reply: tid=0 2024-03-28T13:13:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.279+0000 7f33f9d87740 1 client.6213 shutdown 2024-03-28T13:13:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.279+0000 7f33f9d87740 1 -- 172.21.15.19:0/3951660949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8ce4f0 msgr2=0x55abcd90af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.279+0000 7f33f9d87740 1 --2- 172.21.15.19:0/3951660949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8ce4f0 0x55abcd90af30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f33e8061310 tx=0x7f33e8015040 comp rx=0 tx=0).stop 2024-03-28T13:13:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.279+0000 7f33f9d87740 1 -- 172.21.15.19:0/3951660949 shutdown_connections 2024-03-28T13:13:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.279+0000 7f33f9d87740 1 --2- 172.21.15.19:0/3951660949 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55abcd8c95c0 0x55abcd90b470 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.279+0000 7f33f9d87740 1 --2- 172.21.15.19:0/3951660949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55abcd8ce4f0 0x55abcd90af30 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.279+0000 7f33f9d87740 1 --2- 172.21.15.19:0/3951660949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55abcd8cd8a0 0x55abcd8ec100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.279+0000 7f33f9d87740 1 --2- 172.21.15.19:0/3951660949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55abcd8a1420 0x55abcd8ebbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.279+0000 7f33d9600640 0 client.6213 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.279+0000 7f33f9d87740 1 -- 172.21.15.19:0/3951660949 >> 172.21.15.19:0/3951660949 conn(0x55abcd89d8c0 msgr2=0x55abcd8e8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.279+0000 7f33f9d87740 1 -- 172.21.15.19:0/3951660949 shutdown_connections 2024-03-28T13:13:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.280+0000 7f33f9d87740 1 -- 172.21.15.19:0/3951660949 wait complete. 2024-03-28T13:13:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.280+0000 7f33f9d87740 20 client.6213 trim_cache size 0 max 16384 2024-03-28T13:13:35.283 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:13:35.284 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-03-28T13:13:35.284 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:13:35.284 INFO:teuthology.orchestra.run.smithi019.stdout: "damage_type": "backtrace", 2024-03-28T13:13:35.284 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 3633501573, 2024-03-28T13:13:35.284 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511627777, 2024-03-28T13:13:35.284 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/dir_test/file" 2024-03-28T13:13:35.284 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:13:35.284 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-03-28T13:13:35.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.280+0000 7f33f5a00640 1 -- 172.21.15.19:0/4229864612 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33d006c560 msgr2=0x7f33d006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.280+0000 7f33f5a00640 1 --2- 172.21.15.19:0/4229864612 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33d006c560 0x7f33d006ea20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f33e00099d0 tx=0x7f33e0008040 comp rx=0 tx=0).stop 2024-03-28T13:13:35.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.280+0000 7f33f5a00640 1 -- 172.21.15.19:0/4229864612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f01e35e0 msgr2=0x7f33f00c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.280+0000 7f33f5a00640 1 --2- 172.21.15.19:0/4229864612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f01e35e0 0x7f33f00c8dd0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f33f00d1fb0 tx=0x7f33e800d9d0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.280+0000 7f33f5a00640 1 -- 172.21.15.19:0/4229864612 shutdown_connections 2024-03-28T13:13:35.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.280+0000 7f33f5a00640 1 --2- 172.21.15.19:0/4229864612 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33d006c560 0x7f33d006ea20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.280+0000 7f33f5a00640 1 --2- 172.21.15.19:0/4229864612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f33f01e35e0 0x7f33f00c8dd0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.280+0000 7f33f5a00640 1 --2- 172.21.15.19:0/4229864612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f00cc690 0x7f33f00cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.280+0000 7f33f5a00640 1 --2- 172.21.15.19:0/4229864612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33d4004800 0x7f33f0150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.280+0000 7f33f5a00640 1 -- 172.21.15.19:0/4229864612 >> 172.21.15.19:0/4229864612 conn(0x7f33f00678b0 msgr2=0x7f33f00d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:35.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.281+0000 7f33f5a00640 1 -- 172.21.15.19:0/4229864612 shutdown_connections 2024-03-28T13:13:35.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.281+0000 7f33f5a00640 1 -- 172.21.15.19:0/4229864612 wait complete. 2024-03-28T13:13:35.297 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-28T13:13:35.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.505+0000 7feb2be00640 1 Processor -- start 2024-03-28T13:13:35.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.506+0000 7feb2be00640 1 -- start start 2024-03-28T13:13:35.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.507+0000 7feb2be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb2c15e420 0x7feb2c0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.507+0000 7feb2be00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feb2c0d1a90 0x7feb2c0d3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.507+0000 7feb2be00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb2c0d2fd0 con 0x7feb2c0b3460 2024-03-28T13:13:35.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.507+0000 7feb2be00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7feb2c0cfa70 con 0x7feb2c15e420 2024-03-28T13:13:35.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.507+0000 7feb2be00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7feb2c0cf790 con 0x7feb2c0d1a90 2024-03-28T13:13:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.508+0000 7feb2aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb2c15e420 0x7feb2c0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.508+0000 7feb2aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb2c15e420 0x7feb2c0b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44530/0 (socket says 172.21.15.19:44530) 2024-03-28T13:13:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.508+0000 7feb2aa00640 1 -- 172.21.15.19:0/1738410454 learned_addr learned my addr 172.21.15.19:0/1738410454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.508+0000 7feb2b400640 1 --2- 172.21.15.19:0/1738410454 >> v2:172.21.15.188:3300/0 conn(0x7feb2c0d1a90 0x7feb2c0d3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.508+0000 7feb2aa00640 1 -- 172.21.15.19:0/1738410454 >> v2:172.21.15.188:3300/0 conn(0x7feb2c0d1a90 msgr2=0x7feb2c0d3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.508+0000 7feb2aa00640 1 --2- 172.21.15.19:0/1738410454 >> v2:172.21.15.188:3300/0 conn(0x7feb2c0d1a90 0x7feb2c0d3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.508+0000 7feb2aa00640 1 -- 172.21.15.19:0/1738410454 >> v1:172.21.15.19:6789/0 conn(0x7feb2c0b3460 legacy=0x7feb2c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.508+0000 7feb2aa00640 1 -- 172.21.15.19:0/1738410454 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c0d3270 con 0x7feb2c15e420 2024-03-28T13:13:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.508+0000 7feb2b400640 1 --2- 172.21.15.19:0/1738410454 >> v2:172.21.15.188:3300/0 conn(0x7feb2c0d1a90 0x7feb2c0d3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.509+0000 7feb2aa00640 1 --2- 172.21.15.19:0/1738410454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb2c15e420 0x7feb2c0b2f20 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7feb2c161f30 tx=0x7feb1c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea96150a1c29f5a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:35.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.510+0000 7feb2a000640 1 -- 172.21.15.19:0/1738410454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c034070 con 0x7feb2c15e420 2024-03-28T13:13:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.510+0000 7feb2a000640 1 -- 172.21.15.19:0/1738410454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb1c030860 con 0x7feb2c15e420 2024-03-28T13:13:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.510+0000 7feb2a000640 1 -- 172.21.15.19:0/1738410454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c030c30 con 0x7feb2c15e420 2024-03-28T13:13:35.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.510+0000 7feb2be00640 1 -- 172.21.15.19:0/1738410454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb2c15e420 msgr2=0x7feb2c0b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.510+0000 7feb2be00640 1 --2- 172.21.15.19:0/1738410454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb2c15e420 0x7feb2c0b2f20 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7feb2c161f30 tx=0x7feb1c02b9a0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.510+0000 7feb2be00640 1 -- 172.21.15.19:0/1738410454 shutdown_connections 2024-03-28T13:13:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.510+0000 7feb2be00640 1 --2- 172.21.15.19:0/1738410454 >> v2:172.21.15.188:3300/0 conn(0x7feb2c0d1a90 0x7feb2c0d3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.510+0000 7feb2be00640 1 --2- 172.21.15.19:0/1738410454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb2c15e420 0x7feb2c0b2f20 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.511+0000 7feb2be00640 1 -- 172.21.15.19:0/1738410454 >> 172.21.15.19:0/1738410454 conn(0x7feb2c067670 msgr2=0x7feb2c067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.511+0000 7feb2be00640 1 -- 172.21.15.19:0/1738410454 shutdown_connections 2024-03-28T13:13:35.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.511+0000 7feb2be00640 1 -- 172.21.15.19:0/1738410454 wait complete. 2024-03-28T13:13:35.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.512+0000 7feb2be00640 1 Processor -- start 2024-03-28T13:13:35.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.512+0000 7feb2be00640 1 -- start start 2024-03-28T13:13:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.513+0000 7feb2be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb2c0b3460 0x7feb2c152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.513+0000 7feb2be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb2c0d1a90 0x7feb2c0c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.513+0000 7feb2be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb2c15e420 0x7feb2c0ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.513+0000 7feb2be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c0cf420 con 0x7feb2c15e420 2024-03-28T13:13:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.513+0000 7feb2be00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feb2c15e800 con 0x7feb2c0d1a90 2024-03-28T13:13:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.513+0000 7feb2be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feb2c0d3fb0 con 0x7feb2c0b3460 2024-03-28T13:13:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.514+0000 7feb2b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb2c15e420 0x7feb2c0ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.514+0000 7feb2b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb2c15e420 0x7feb2c0ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44532/0 (socket says 172.21.15.19:44532) 2024-03-28T13:13:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.514+0000 7feb2b400640 1 -- 172.21.15.19:0/1959893951 learned_addr learned my addr 172.21.15.19:0/1959893951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.514+0000 7feb2b400640 1 -- 172.21.15.19:0/1959893951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb2c0b3460 msgr2=0x7feb2c152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.514+0000 7feb2b400640 1 --2- 172.21.15.19:0/1959893951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb2c0b3460 0x7feb2c152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.514+0000 7feb2b400640 1 -- 172.21.15.19:0/1959893951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb2c0d1a90 msgr2=0x7feb2c0c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.514+0000 7feb2b400640 1 --2- 172.21.15.19:0/1959893951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb2c0d1a90 0x7feb2c0c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.514+0000 7feb2b400640 1 -- 172.21.15.19:0/1959893951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2c152ea0 con 0x7feb2c15e420 2024-03-28T13:13:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.515+0000 7feb2b400640 1 --2- 172.21.15.19:0/1959893951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb2c15e420 0x7feb2c0ca2b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7feb2400cef0 tx=0x7feb2400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.515+0000 7feb22a00640 1 -- 172.21.15.19:0/1959893951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2401a070 con 0x7feb2c15e420 2024-03-28T13:13:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.515+0000 7feb22a00640 1 -- 172.21.15.19:0/1959893951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb2400d860 con 0x7feb2c15e420 2024-03-28T13:13:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.515+0000 7feb22a00640 1 -- 172.21.15.19:0/1959893951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2400dc50 con 0x7feb2c15e420 2024-03-28T13:13:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.515+0000 7feb2be00640 1 -- 172.21.15.19:0/1959893951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb2c1f4bf0 con 0x7feb2c15e420 2024-03-28T13:13:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.515+0000 7feb2be00640 1 -- 172.21.15.19:0/1959893951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb2c153520 con 0x7feb2c15e420 2024-03-28T13:13:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.516+0000 7feb2be00640 1 -- 172.21.15.19:0/1959893951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb2c075940 con 0x7feb2c15e420 2024-03-28T13:13:35.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.517+0000 7feb22a00640 1 -- 172.21.15.19:0/1959893951 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7feb2401d070 con 0x7feb2c15e420 2024-03-28T13:13:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.518+0000 7feb22a00640 1 --2- 172.21.15.19:0/1959893951 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb0806c530 0x7feb0806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.518+0000 7feb22a00640 1 -- 172.21.15.19:0/1959893951 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7feb2408c080 con 0x7feb2c15e420 2024-03-28T13:13:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.519+0000 7feb2aa00640 1 --2- 172.21.15.19:0/1959893951 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb0806c530 0x7feb0806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:35.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.520+0000 7feb2aa00640 1 --2- 172.21.15.19:0/1959893951 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb0806c530 0x7feb0806e9f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7feb1c036000 tx=0x7feb1c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:35.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.523+0000 7feb22a00640 1 -- 172.21.15.19:0/1959893951 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7feb2c152ea0 con 0x7feb2c15e420 2024-03-28T13:13:35.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.773+0000 7feb2be00640 1 -- 172.21.15.19:0/1959893951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7feb2c154000 con 0x7feb2c15e420 2024-03-28T13:13:35.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.773+0000 7feb22a00640 1 -- 172.21.15.19:0/1959893951 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7feb2405a680 con 0x7feb2c15e420 2024-03-28T13:13:35.775 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:13:35.775 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-28T13:13:35.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.776+0000 7feb2be00640 1 -- 172.21.15.19:0/1959893951 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb0806c530 msgr2=0x7feb0806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.776+0000 7feb2be00640 1 --2- 172.21.15.19:0/1959893951 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb0806c530 0x7feb0806e9f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7feb1c036000 tx=0x7feb1c030000 comp rx=0 tx=0).stop 2024-03-28T13:13:35.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.776+0000 7feb2be00640 1 -- 172.21.15.19:0/1959893951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb2c15e420 msgr2=0x7feb2c0ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:35.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.776+0000 7feb2be00640 1 --2- 172.21.15.19:0/1959893951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb2c15e420 0x7feb2c0ca2b0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7feb2400cef0 tx=0x7feb2400a710 comp rx=0 tx=0).stop 2024-03-28T13:13:35.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.776+0000 7feb2be00640 1 -- 172.21.15.19:0/1959893951 shutdown_connections 2024-03-28T13:13:35.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.776+0000 7feb2be00640 1 --2- 172.21.15.19:0/1959893951 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb0806c530 0x7feb0806e9f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.776+0000 7feb2be00640 1 --2- 172.21.15.19:0/1959893951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb2c15e420 0x7feb2c0ca2b0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.776+0000 7feb2be00640 1 --2- 172.21.15.19:0/1959893951 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb2c0d1a90 0x7feb2c0c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.776+0000 7feb2be00640 1 --2- 172.21.15.19:0/1959893951 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb2c0b3460 0x7feb2c152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.776+0000 7feb2be00640 1 -- 172.21.15.19:0/1959893951 >> 172.21.15.19:0/1959893951 conn(0x7feb2c067670 msgr2=0x7feb2c1627e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.776+0000 7feb2be00640 1 -- 172.21.15.19:0/1959893951 shutdown_connections 2024-03-28T13:13:35.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:35.776+0000 7feb2be00640 1 -- 172.21.15.19:0/1959893951 wait complete. 2024-03-28T13:13:35.796 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-28T13:13:35.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:13:35.796 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-03-28T13:13:36.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.018+0000 7f913d000640 1 Processor -- start 2024-03-28T13:13:36.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.018+0000 7f913d000640 1 -- start start 2024-03-28T13:13:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.021+0000 7f913d000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91380acf10 0x7f913809dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.022+0000 7f913d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f91380ad2f0 0x7f91381453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.022+0000 7f913d000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f913815cb90 con 0x7f913815ff70 2024-03-28T13:13:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.022+0000 7f913d000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9138145f60 con 0x7f91380ad2f0 2024-03-28T13:13:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.022+0000 7f913d000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f913815c950 con 0x7f91380acf10 2024-03-28T13:13:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.022+0000 7f9135400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f91380ad2f0 0x7f91381453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.022+0000 7f9135400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f91380ad2f0 0x7f91381453f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44548/0 (socket says 172.21.15.19:44548) 2024-03-28T13:13:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.022+0000 7f9135400640 1 -- 172.21.15.19:0/3885330531 learned_addr learned my addr 172.21.15.19:0/3885330531 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.022+0000 7f9135e00640 1 --2- 172.21.15.19:0/3885330531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91380acf10 0x7f913809dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.023+0000 7f9135400640 1 -- 172.21.15.19:0/3885330531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91380acf10 msgr2=0x7f913809dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.023+0000 7f9135400640 1 --2- 172.21.15.19:0/3885330531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91380acf10 0x7f913809dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.023+0000 7f9135400640 1 -- 172.21.15.19:0/3885330531 >> v1:172.21.15.188:6789/0 conn(0x7f913815ff70 legacy=0x7f91381603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.023+0000 7f9135400640 1 -- 172.21.15.19:0/3885330531 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913815ce70 con 0x7f91380ad2f0 2024-03-28T13:13:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.024+0000 7f9135400640 1 --2- 172.21.15.19:0/3885330531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91380ad2f0 0x7f91381453f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9124009870 tx=0x7f912402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6594e8628ff7446a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.025+0000 7f9134a00640 1 -- 172.21.15.19:0/3885330531 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124035020 con 0x7f91380ad2f0 2024-03-28T13:13:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.025+0000 7f9134a00640 1 -- 172.21.15.19:0/3885330531 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9124031640 con 0x7f91380ad2f0 2024-03-28T13:13:36.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.025+0000 7f9134a00640 1 -- 172.21.15.19:0/3885330531 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912402d040 con 0x7f91380ad2f0 2024-03-28T13:13:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.025+0000 7f913d000640 1 -- 172.21.15.19:0/3885330531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91380ad2f0 msgr2=0x7f91381453f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.025+0000 7f913d000640 1 --2- 172.21.15.19:0/3885330531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91380ad2f0 0x7f91381453f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9124009870 tx=0x7f912402b8f0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.025+0000 7f913d000640 1 -- 172.21.15.19:0/3885330531 shutdown_connections 2024-03-28T13:13:36.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.025+0000 7f913d000640 1 --2- 172.21.15.19:0/3885330531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91380ad2f0 0x7f91381453f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.025+0000 7f913d000640 1 --2- 172.21.15.19:0/3885330531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91380acf10 0x7f913809dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.025+0000 7f913d000640 1 -- 172.21.15.19:0/3885330531 >> 172.21.15.19:0/3885330531 conn(0x7f913806bba0 msgr2=0x7f91380b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:36.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.025+0000 7f913d000640 1 -- 172.21.15.19:0/3885330531 shutdown_connections 2024-03-28T13:13:36.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.026+0000 7f913d000640 1 -- 172.21.15.19:0/3885330531 wait complete. 2024-03-28T13:13:36.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.027+0000 7f913d000640 1 Processor -- start 2024-03-28T13:13:36.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.027+0000 7f913d000640 1 -- start start 2024-03-28T13:13:36.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.028+0000 7f913d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91380acf10 0x7f913809cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.028+0000 7f913d000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91380ad2f0 0x7f913813d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.028+0000 7f913d000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913815ff70 0x7f91381f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.028+0000 7f913d000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f913815d430 con 0x7f91380acf10 2024-03-28T13:13:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.028+0000 7f913d000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f913815d7a0 con 0x7f913815ff70 2024-03-28T13:13:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.028+0000 7f913d000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f91380ad770 con 0x7f91380ad2f0 2024-03-28T13:13:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.029+0000 7f9135e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91380acf10 0x7f913809cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.029+0000 7f9135e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91380acf10 0x7f913809cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44562/0 (socket says 172.21.15.19:44562) 2024-03-28T13:13:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.029+0000 7f9135e00640 1 -- 172.21.15.19:0/3528725090 learned_addr learned my addr 172.21.15.19:0/3528725090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.029+0000 7f9135400640 1 --2- 172.21.15.19:0/3528725090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91380ad2f0 0x7f913813d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.029+0000 7f9136800640 1 --2- 172.21.15.19:0/3528725090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913815ff70 0x7f91381f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.029+0000 7f9135e00640 1 -- 172.21.15.19:0/3528725090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91380ad2f0 msgr2=0x7f913813d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.029+0000 7f9135e00640 1 --2- 172.21.15.19:0/3528725090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91380ad2f0 0x7f913813d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.029+0000 7f9135e00640 1 -- 172.21.15.19:0/3528725090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913815ff70 msgr2=0x7f91381f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.029+0000 7f9135e00640 1 --2- 172.21.15.19:0/3528725090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913815ff70 0x7f91381f41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.029+0000 7f9135e00640 1 -- 172.21.15.19:0/3528725090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913806d4d0 con 0x7f91380acf10 2024-03-28T13:13:36.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.029+0000 7f9136800640 1 --2- 172.21.15.19:0/3528725090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913815ff70 0x7f91381f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:36.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.030+0000 7f9135e00640 1 --2- 172.21.15.19:0/3528725090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91380acf10 0x7f913809cf20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f912800d800 tx=0x7f912800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:36.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.030+0000 7f912e000640 1 -- 172.21.15.19:0/3528725090 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912800b6c0 con 0x7f91380acf10 2024-03-28T13:13:36.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.030+0000 7f912e000640 1 -- 172.21.15.19:0/3528725090 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9128019070 con 0x7f91380acf10 2024-03-28T13:13:36.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.030+0000 7f912e000640 1 -- 172.21.15.19:0/3528725090 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912800bac0 con 0x7f91380acf10 2024-03-28T13:13:36.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.030+0000 7f913d000640 1 -- 172.21.15.19:0/3528725090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913809d7e0 con 0x7f91380acf10 2024-03-28T13:13:36.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.030+0000 7f913d000640 1 -- 172.21.15.19:0/3528725090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f913806da70 con 0x7f91380acf10 2024-03-28T13:13:36.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.034+0000 7f912e000640 1 -- 172.21.15.19:0/3528725090 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f912800bc60 con 0x7f91380acf10 2024-03-28T13:13:36.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.034+0000 7f912e000640 1 --2- 172.21.15.19:0/3528725090 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f911806c560 0x7f911806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.035+0000 7f912e000640 1 -- 172.21.15.19:0/3528725090 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f912808bbf0 con 0x7f91380acf10 2024-03-28T13:13:36.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.035+0000 7f913fe0a740 1 Processor -- start 2024-03-28T13:13:36.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.035+0000 7f913fe0a740 1 -- start start 2024-03-28T13:13:36.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.035+0000 7f9135400640 1 --2- 172.21.15.19:0/3528725090 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f911806c560 0x7f911806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.036+0000 7f913fe0a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf906750 0x55d1cf8d96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.036+0000 7f913fe0a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906b30 0x55d1cf9094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.036+0000 7f913fe0a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf8d9c30 0x55d1cf909a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.036+0000 7f913fe0a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d1cf843320 con 0x55d1cf8d9c30 2024-03-28T13:13:36.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.036+0000 7f913fe0a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d1cf7eb780 con 0x55d1cf906b30 2024-03-28T13:13:36.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.036+0000 7f913fe0a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55d1cf83f2d0 con 0x55d1cf906750 2024-03-28T13:13:36.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.037+0000 7f9136800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf8d9c30 0x55d1cf909a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.037+0000 7f9135400640 1 --2- 172.21.15.19:0/3528725090 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f911806c560 0x7f911806ea20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9124036000 tx=0x7f9124008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:36.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.037+0000 7f9136800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf8d9c30 0x55d1cf909a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44564/0 (socket says 172.21.15.19:44564) 2024-03-28T13:13:36.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.037+0000 7f9136800640 1 -- 172.21.15.19:0/623502440 learned_addr learned my addr 172.21.15.19:0/623502440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:36.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.037+0000 7f9135400640 1 --2- 172.21.15.19:0/623502440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906b30 0x55d1cf9094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.037+0000 7f9135e00640 1 --2- 172.21.15.19:0/623502440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf906750 0x55d1cf8d96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.037+0000 7f9136800640 1 -- 172.21.15.19:0/623502440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf906750 msgr2=0x55d1cf8d96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.037+0000 7f9136800640 1 --2- 172.21.15.19:0/623502440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf906750 0x55d1cf8d96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.037+0000 7f9136800640 1 -- 172.21.15.19:0/623502440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906b30 msgr2=0x55d1cf9094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.037+0000 7f9136800640 1 --2- 172.21.15.19:0/623502440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906b30 0x55d1cf9094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.037+0000 7f9136800640 1 -- 172.21.15.19:0/623502440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1cf623940 con 0x55d1cf8d9c30 2024-03-28T13:13:36.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.037+0000 7f9136800640 1 --2- 172.21.15.19:0/623502440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf8d9c30 0x55d1cf909a20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9130001dd0 tx=0x7f913000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8344e550336edb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:36.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.038+0000 7f9122a00640 1 -- 172.21.15.19:0/623502440 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130018070 con 0x55d1cf8d9c30 2024-03-28T13:13:36.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.038+0000 7f9122a00640 1 -- 172.21.15.19:0/623502440 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f913000ddd0 con 0x55d1cf8d9c30 2024-03-28T13:13:36.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.038+0000 7f9122a00640 1 -- 172.21.15.19:0/623502440 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130003bb0 con 0x55d1cf8d9c30 2024-03-28T13:13:36.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.038+0000 7f913fe0a740 1 -- 172.21.15.19:0/623502440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf8d9c30 msgr2=0x55d1cf909a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.038+0000 7f913fe0a740 1 --2- 172.21.15.19:0/623502440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf8d9c30 0x55d1cf909a20 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9130001dd0 tx=0x7f913000d520 comp rx=0 tx=0).stop 2024-03-28T13:13:36.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.039+0000 7f913fe0a740 1 -- 172.21.15.19:0/623502440 shutdown_connections 2024-03-28T13:13:36.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.039+0000 7f913fe0a740 1 --2- 172.21.15.19:0/623502440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf8d9c30 0x55d1cf909a20 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.039+0000 7f913fe0a740 1 --2- 172.21.15.19:0/623502440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906b30 0x55d1cf9094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.039+0000 7f913fe0a740 1 --2- 172.21.15.19:0/623502440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf906750 0x55d1cf8d96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.039+0000 7f913fe0a740 1 -- 172.21.15.19:0/623502440 >> 172.21.15.19:0/623502440 conn(0x55d1cf8db960 msgr2=0x55d1cf9244e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.039+0000 7f913fe0a740 1 -- 172.21.15.19:0/623502440 shutdown_connections 2024-03-28T13:13:36.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.039+0000 7f913fe0a740 1 -- 172.21.15.19:0/623502440 wait complete. 2024-03-28T13:13:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.040+0000 7f913fe0a740 1 -- start start 2024-03-28T13:13:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.040+0000 7f913fe0a740 1 Processor -- start 2024-03-28T13:13:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.042+0000 7f913fe0a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906750 0x55d1cf8fe850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.042+0000 7f913fe0a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf906b30 0x55d1cf8ff4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.042+0000 7f913fe0a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf924f80 0x55d1cf90b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.042+0000 7f913fe0a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d1cf8de000 con 0x55d1cf924f80 2024-03-28T13:13:36.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.042+0000 7f913fe0a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d1cf7feb30 con 0x55d1cf906750 2024-03-28T13:13:36.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.042+0000 7f913fe0a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55d1cf8e0840 con 0x55d1cf906b30 2024-03-28T13:13:36.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.042+0000 7f913fe0a740 10 client.0 _ll_register_callbacks cb 0x55d1cf7fe520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:13:36.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.042+0000 7f9136800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf924f80 0x55d1cf90b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.042+0000 7f9136800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf924f80 0x55d1cf90b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44576/0 (socket says 172.21.15.19:44576) 2024-03-28T13:13:36.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.042+0000 7f9136800640 1 -- 172.21.15.19:0/2752952430 learned_addr learned my addr 172.21.15.19:0/2752952430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.043+0000 7f9136800640 1 -- 172.21.15.19:0/2752952430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf906b30 msgr2=0x55d1cf8ff4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.043+0000 7f9136800640 1 --2- 172.21.15.19:0/2752952430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf906b30 0x55d1cf8ff4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.043+0000 7f9136800640 1 -- 172.21.15.19:0/2752952430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906750 msgr2=0x55d1cf8fe850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.043+0000 7f9136800640 1 --2- 172.21.15.19:0/2752952430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906750 0x55d1cf8fe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.043+0000 7f9136800640 1 -- 172.21.15.19:0/2752952430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1cf6840d0 con 0x55d1cf924f80 2024-03-28T13:13:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.043+0000 7f9136800640 1 --2- 172.21.15.19:0/2752952430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf924f80 0x55d1cf90b2a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f913000da20 tx=0x7f9130001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53509de5561b70e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.043+0000 7f9117400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:13:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.043+0000 7f9117400640 1 -- 172.21.15.19:0/2752952430 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130022020 con 0x55d1cf924f80 2024-03-28T13:13:36.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.043+0000 7f9117400640 1 -- 172.21.15.19:0/2752952430 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f91300050d0 con 0x55d1cf924f80 2024-03-28T13:13:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.044+0000 7f913fe0a740 1 -- 172.21.15.19:0/2752952430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f913000ddd0 con 0x55d1cf924f80 2024-03-28T13:13:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.044+0000 7f9117400640 1 -- 172.21.15.19:0/2752952430 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f913002c050 con 0x55d1cf924f80 2024-03-28T13:13:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.044+0000 7f913fe0a740 10 client.6231 fetch_fsmap learned FSMap version 35 2024-03-28T13:13:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.044+0000 7f913fe0a740 1 -- 172.21.15.19:0/2752952430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55d1cf517230 con 0x55d1cf924f80 2024-03-28T13:13:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.044+0000 7f9117400640 1 -- 172.21.15.19:0/2752952430 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f913000ddd0 con 0x55d1cf924f80 2024-03-28T13:13:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.044+0000 7f9117400640 1 -- 172.21.15.19:0/2752952430 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f913002f020 con 0x55d1cf924f80 2024-03-28T13:13:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.044+0000 7f913fe0a740 10 client.6231 fetch_fsmap finished waiting for FSMap version 35 2024-03-28T13:13:36.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.044+0000 7f913fe0a740 10 client.6231 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-03-28T13:13:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.044+0000 7f913fe0a740 20 client.6231 populate_metadata read hostname 'smithi019' 2024-03-28T13:13:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.044+0000 7f913fe0a740 1 --2- 172.21.15.19:0/2752952430 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55d1cf902010 0x55d1cf937ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.044+0000 7f913fe0a740 4 client.6231 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:13:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.044+0000 7f913fe0a740 1 -- 172.21.15.19:0/2752952430 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1cf850860 con 0x55d1cf902010 2024-03-28T13:13:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.045+0000 7f9135e00640 1 --2- 172.21.15.19:0/2752952430 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55d1cf902010 0x55d1cf937ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.045+0000 7f9135e00640 1 --2- 172.21.15.19:0/2752952430 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55d1cf902010 0x55d1cf937ec0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c98d7210244487db server_cookie=68b1f6d254c08256 in_seq=0 out_seq=0 2024-03-28T13:13:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.045+0000 7f9117400640 10 client.6231 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.046+0000 7f9117400640 1 -- 172.21.15.19:0/2752952430 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1cf850860 con 0x55d1cf902010 2024-03-28T13:13:36.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.046+0000 7f9117400640 10 client.6231 handle_command_reply: tid=0 2024-03-28T13:13:36.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.046+0000 7f913fe0a740 1 client.6231 shutdown 2024-03-28T13:13:36.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.046+0000 7f913fe0a740 1 -- 172.21.15.19:0/2752952430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf924f80 msgr2=0x55d1cf90b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.046+0000 7f913fe0a740 1 --2- 172.21.15.19:0/2752952430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf924f80 0x55d1cf90b2a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f913000da20 tx=0x7f9130001250 comp rx=0 tx=0).stop 2024-03-28T13:13:36.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.046+0000 7f913fe0a740 1 -- 172.21.15.19:0/2752952430 shutdown_connections 2024-03-28T13:13:36.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.046+0000 7f913fe0a740 1 --2- 172.21.15.19:0/2752952430 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55d1cf902010 0x55d1cf937ec0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.046+0000 7f913fe0a740 1 --2- 172.21.15.19:0/2752952430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf924f80 0x55d1cf90b2a0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.046+0000 7f913fe0a740 1 --2- 172.21.15.19:0/2752952430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf906b30 0x55d1cf8ff4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.046+0000 7f913fe0a740 1 --2- 172.21.15.19:0/2752952430 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906750 0x55d1cf8fe850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.046+0000 7f913fe0a740 1 -- 172.21.15.19:0/2752952430 >> 172.21.15.19:0/2752952430 conn(0x55d1cf8d9c30 msgr2=0x55d1cf8dd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:36.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.046+0000 7f9117400640 0 client.6231 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:36.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.046+0000 7f913fe0a740 1 -- 172.21.15.19:0/2752952430 shutdown_connections 2024-03-28T13:13:36.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.047+0000 7f913fe0a740 1 -- 172.21.15.19:0/2752952430 wait complete. 2024-03-28T13:13:36.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.047+0000 7f913fe0a740 20 client.6231 trim_cache size 0 max 16384 2024-03-28T13:13:36.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.068+0000 7f913fe0a740 1 Processor -- start 2024-03-28T13:13:36.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.068+0000 7f913fe0a740 1 -- start start 2024-03-28T13:13:36.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.069+0000 7f913fe0a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf906750 0x55d1cf8d5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.069+0000 7f913fe0a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906b30 0x55d1cf90b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.069+0000 7f913fe0a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf924f80 0x55d1cf900e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.069+0000 7f913fe0a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9138200c10 con 0x55d1cf906750 2024-03-28T13:13:36.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.069+0000 7f913fe0a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f91381ffc30 con 0x55d1cf906b30 2024-03-28T13:13:36.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.069+0000 7f913fe0a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f91381ff030 con 0x55d1cf924f80 2024-03-28T13:13:36.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.069+0000 7f9135e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf906750 0x55d1cf8d5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.069+0000 7f9135e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf906750 0x55d1cf8d5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44584/0 (socket says 172.21.15.19:44584) 2024-03-28T13:13:36.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.069+0000 7f9135e00640 1 -- 172.21.15.19:0/2843521165 learned_addr learned my addr 172.21.15.19:0/2843521165 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:36.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.069+0000 7f9135400640 1 --2- 172.21.15.19:0/2843521165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906b30 0x55d1cf90b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.069+0000 7f9136800640 1 --2- 172.21.15.19:0/2843521165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf924f80 0x55d1cf900e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.069+0000 7f9135e00640 1 -- 172.21.15.19:0/2843521165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf924f80 msgr2=0x55d1cf900e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.069+0000 7f9135e00640 1 --2- 172.21.15.19:0/2843521165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf924f80 0x55d1cf900e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.070+0000 7f9135e00640 1 -- 172.21.15.19:0/2843521165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906b30 msgr2=0x55d1cf90b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.070+0000 7f9135e00640 1 --2- 172.21.15.19:0/2843521165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906b30 0x55d1cf90b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.070+0000 7f9135e00640 1 -- 172.21.15.19:0/2843521165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1cf8ffd30 con 0x55d1cf906750 2024-03-28T13:13:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.070+0000 7f9135e00640 1 --2- 172.21.15.19:0/2843521165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf906750 0x55d1cf8d5030 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9128055f20 tx=0x7f9128059b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46be7d53ed9913 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.070+0000 7f9122000640 1 -- 172.21.15.19:0/2843521165 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912808a150 con 0x55d1cf906750 2024-03-28T13:13:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.070+0000 7f9122000640 1 -- 172.21.15.19:0/2843521165 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9128057dc0 con 0x55d1cf906750 2024-03-28T13:13:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.070+0000 7f9122000640 1 -- 172.21.15.19:0/2843521165 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128062210 con 0x55d1cf906750 2024-03-28T13:13:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.071+0000 7f913fe0a740 1 -- 172.21.15.19:0/2843521165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf906750 msgr2=0x55d1cf8d5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.071+0000 7f913fe0a740 1 --2- 172.21.15.19:0/2843521165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf906750 0x55d1cf8d5030 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9128055f20 tx=0x7f9128059b80 comp rx=0 tx=0).stop 2024-03-28T13:13:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.071+0000 7f913fe0a740 1 -- 172.21.15.19:0/2843521165 shutdown_connections 2024-03-28T13:13:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.071+0000 7f913fe0a740 1 --2- 172.21.15.19:0/2843521165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf924f80 0x55d1cf900e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.071+0000 7f913fe0a740 1 --2- 172.21.15.19:0/2843521165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906b30 0x55d1cf90b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.071+0000 7f913fe0a740 1 --2- 172.21.15.19:0/2843521165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf906750 0x55d1cf8d5030 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.071+0000 7f913fe0a740 1 -- 172.21.15.19:0/2843521165 >> 172.21.15.19:0/2843521165 conn(0x55d1cf8d9c30 msgr2=0x55d1cf8df780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:36.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.071+0000 7f913fe0a740 1 -- 172.21.15.19:0/2843521165 shutdown_connections 2024-03-28T13:13:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.071+0000 7f913fe0a740 1 -- 172.21.15.19:0/2843521165 wait complete. 2024-03-28T13:13:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.072+0000 7f913fe0a740 1 -- start start 2024-03-28T13:13:36.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.072+0000 7f913fe0a740 1 Processor -- start 2024-03-28T13:13:36.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.073+0000 7f913fe0a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906750 0x55d1cf90d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.073+0000 7f913fe0a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf906b30 0x55d1cf9814f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.073+0000 7f913fe0a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf924f80 0x55d1cf9371b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.073+0000 7f913fe0a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d1cf907860 con 0x55d1cf906b30 2024-03-28T13:13:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.073+0000 7f913fe0a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f91381ff5e0 con 0x55d1cf906750 2024-03-28T13:13:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.073+0000 7f913fe0a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55d1cf8ff410 con 0x55d1cf924f80 2024-03-28T13:13:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.073+0000 7f913fe0a740 10 client.0 _ll_register_callbacks cb 0x7f91300021b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:13:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.074+0000 7f9135400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf906b30 0x55d1cf9814f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.074+0000 7f9135400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf906b30 0x55d1cf9814f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44598/0 (socket says 172.21.15.19:44598) 2024-03-28T13:13:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.074+0000 7f9135400640 1 -- 172.21.15.19:0/1629905697 learned_addr learned my addr 172.21.15.19:0/1629905697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.074+0000 7f9135e00640 1 --2- 172.21.15.19:0/1629905697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906750 0x55d1cf90d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.074+0000 7f9136800640 1 --2- 172.21.15.19:0/1629905697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf924f80 0x55d1cf9371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.074+0000 7f9135400640 1 -- 172.21.15.19:0/1629905697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf924f80 msgr2=0x55d1cf9371b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.074+0000 7f9135400640 1 --2- 172.21.15.19:0/1629905697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf924f80 0x55d1cf9371b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.074+0000 7f9135400640 1 -- 172.21.15.19:0/1629905697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906750 msgr2=0x55d1cf90d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.074+0000 7f9135400640 1 --2- 172.21.15.19:0/1629905697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906750 0x55d1cf90d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.074+0000 7f9135400640 1 -- 172.21.15.19:0/1629905697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1cf947e20 con 0x55d1cf906b30 2024-03-28T13:13:36.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.074+0000 7f9135e00640 1 --2- 172.21.15.19:0/1629905697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906750 0x55d1cf90d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:13:36.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.075+0000 7f9135400640 1 --2- 172.21.15.19:0/1629905697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf906b30 0x55d1cf9814f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9124009630 tx=0x7f9124004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09835786f5fe742 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:36.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.075+0000 7f9117400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:13:36.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.075+0000 7f9117400640 1 -- 172.21.15.19:0/1629905697 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124035040 con 0x55d1cf906b30 2024-03-28T13:13:36.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.075+0000 7f9117400640 1 -- 172.21.15.19:0/1629905697 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f912403ad00 con 0x55d1cf906b30 2024-03-28T13:13:36.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.075+0000 7f913fe0a740 1 -- 172.21.15.19:0/1629905697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9128057dc0 con 0x55d1cf906b30 2024-03-28T13:13:36.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.075+0000 7f9117400640 1 -- 172.21.15.19:0/1629905697 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f912404f400 con 0x55d1cf906b30 2024-03-28T13:13:36.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.075+0000 7f913fe0a740 10 client.6237 fetch_fsmap learned FSMap version 35 2024-03-28T13:13:36.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.075+0000 7f913fe0a740 1 -- 172.21.15.19:0/1629905697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55d1cf90b1f0 con 0x55d1cf906b30 2024-03-28T13:13:36.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.075+0000 7f9117400640 1 -- 172.21.15.19:0/1629905697 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9128057dc0 con 0x55d1cf906b30 2024-03-28T13:13:36.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.076+0000 7f9117400640 1 -- 172.21.15.19:0/1629905697 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f912404f5a0 con 0x55d1cf906b30 2024-03-28T13:13:36.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.076+0000 7f913fe0a740 10 client.6237 fetch_fsmap finished waiting for FSMap version 35 2024-03-28T13:13:36.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.076+0000 7f913fe0a740 10 client.6237 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-03-28T13:13:36.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.076+0000 7f913fe0a740 20 client.6237 populate_metadata read hostname 'smithi019' 2024-03-28T13:13:36.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.076+0000 7f913fe0a740 1 --2- 172.21.15.19:0/1629905697 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55d1cf9376f0 0x55d1cf981f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.076+0000 7f913fe0a740 4 client.6237 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-03-28T13:13:36.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.076+0000 7f913fe0a740 1 -- 172.21.15.19:0/1629905697 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x55d1cf84abb0 con 0x55d1cf9376f0 2024-03-28T13:13:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.076+0000 7f9135e00640 1 --2- 172.21.15.19:0/1629905697 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55d1cf9376f0 0x55d1cf981f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.077+0000 7f9135e00640 1 --2- 172.21.15.19:0/1629905697 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55d1cf9376f0 0x55d1cf981f00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58cfac1571182069 server_cookie=41b4350db53e8865 in_seq=0 out_seq=0 2024-03-28T13:13:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.077+0000 7f9117400640 10 client.6237 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.077+0000 7f9117400640 1 -- 172.21.15.19:0/1629905697 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d1cf84abb0 con 0x55d1cf9376f0 2024-03-28T13:13:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.077+0000 7f9117400640 10 client.6237 handle_command_reply: tid=0 2024-03-28T13:13:36.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.077+0000 7f913fe0a740 1 client.6237 shutdown 2024-03-28T13:13:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.078+0000 7f913fe0a740 1 -- 172.21.15.19:0/1629905697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf906b30 msgr2=0x55d1cf9814f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.078+0000 7f913fe0a740 1 --2- 172.21.15.19:0/1629905697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf906b30 0x55d1cf9814f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9124009630 tx=0x7f9124004730 comp rx=0 tx=0).stop 2024-03-28T13:13:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.078+0000 7f913fe0a740 1 -- 172.21.15.19:0/1629905697 shutdown_connections 2024-03-28T13:13:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.078+0000 7f913fe0a740 1 --2- 172.21.15.19:0/1629905697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d1cf924f80 0x55d1cf9371b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.078+0000 7f913fe0a740 1 --2- 172.21.15.19:0/1629905697 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55d1cf9376f0 0x55d1cf981f00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.078+0000 7f913fe0a740 1 --2- 172.21.15.19:0/1629905697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d1cf906b30 0x55d1cf9814f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.078+0000 7f913fe0a740 1 --2- 172.21.15.19:0/1629905697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d1cf906750 0x55d1cf90d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.078+0000 7f913fe0a740 1 -- 172.21.15.19:0/1629905697 >> 172.21.15.19:0/1629905697 conn(0x55d1cf8d9c30 msgr2=0x55d1cf90a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:36.080 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-03-28T13:13:36.078+0000 7fdf3e600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-03-28T13:13:02.306279+0000 1=1+0) caps={6111=pAsLsXsFs/-@2},l=6111(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x55cd5fb58c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-28T13:13:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.078+0000 7f9117400640 0 client.6237 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.078+0000 7f913fe0a740 1 -- 172.21.15.19:0/1629905697 shutdown_connections 2024-03-28T13:13:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.078+0000 7f913fe0a740 1 -- 172.21.15.19:0/1629905697 wait complete. 2024-03-28T13:13:36.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.078+0000 7f913fe0a740 20 client.6237 trim_cache size 0 max 16384 2024-03-28T13:13:36.081 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:13:36.082 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-03-28T13:13:36.082 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "113b93b3-052c-4446-81c8-a582b6c64ff4", 2024-03-28T13:13:36.082 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-03-28T13:13:36.082 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:13:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.079+0000 7f913d000640 1 -- 172.21.15.19:0/3528725090 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f911806c560 msgr2=0x7f911806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.079+0000 7f913d000640 1 --2- 172.21.15.19:0/3528725090 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f911806c560 0x7f911806ea20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9124036000 tx=0x7f9124008030 comp rx=0 tx=0).stop 2024-03-28T13:13:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.079+0000 7f913d000640 1 -- 172.21.15.19:0/3528725090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91380acf10 msgr2=0x7f913809cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.079+0000 7f913d000640 1 --2- 172.21.15.19:0/3528725090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91380acf10 0x7f913809cf20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f912800d800 tx=0x7f912800dcc0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.079+0000 7f913d000640 1 -- 172.21.15.19:0/3528725090 shutdown_connections 2024-03-28T13:13:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.079+0000 7f913d000640 1 --2- 172.21.15.19:0/3528725090 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f911806c560 0x7f911806ea20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.079+0000 7f913d000640 1 --2- 172.21.15.19:0/3528725090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913815ff70 0x7f91381f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.079+0000 7f913d000640 1 --2- 172.21.15.19:0/3528725090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91380ad2f0 0x7f913813d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.079+0000 7f913d000640 1 --2- 172.21.15.19:0/3528725090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91380acf10 0x7f913809cf20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.079+0000 7f913d000640 1 -- 172.21.15.19:0/3528725090 >> 172.21.15.19:0/3528725090 conn(0x7f913806bba0 msgr2=0x7f91380720b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.079+0000 7f913d000640 1 -- 172.21.15.19:0/3528725090 shutdown_connections 2024-03-28T13:13:36.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.079+0000 7f913d000640 1 -- 172.21.15.19:0/3528725090 wait complete. 2024-03-28T13:13:36.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:13:36.105 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-28T13:13:36.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.318+0000 7f6ee6c00640 1 Processor -- start 2024-03-28T13:13:36.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.319+0000 7f6ee6c00640 1 -- start start 2024-03-28T13:13:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.320+0000 7f6ee6c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6ee006c580 0x7f6ee006ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.320+0000 7f6ee6c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ee006ea40 0x7f6ee006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.320+0000 7f6ee6c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee0067900 con 0x7f6ee01590c0 2024-03-28T13:13:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.321+0000 7f6ee6c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6ee006e5b0 con 0x7f6ee006ea40 2024-03-28T13:13:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.321+0000 7f6ee6c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f6ee00b47d0 con 0x7f6ee006c580 2024-03-28T13:13:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.321+0000 7f6ee4c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ee006ea40 0x7f6ee006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.321+0000 7f6ee4c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ee006ea40 0x7f6ee006eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44600/0 (socket says 172.21.15.19:44600) 2024-03-28T13:13:36.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.321+0000 7f6ee4c00640 1 -- 172.21.15.19:0/659220636 learned_addr learned my addr 172.21.15.19:0/659220636 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.321+0000 7f6edf400640 1 --2- 172.21.15.19:0/659220636 >> v2:172.21.15.188:3300/0 conn(0x7f6ee006c580 0x7f6ee006ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.322+0000 7f6ee4c00640 1 -- 172.21.15.19:0/659220636 >> v2:172.21.15.188:3300/0 conn(0x7f6ee006c580 msgr2=0x7f6ee006ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.322+0000 7f6ee4c00640 1 --2- 172.21.15.19:0/659220636 >> v2:172.21.15.188:3300/0 conn(0x7f6ee006c580 0x7f6ee006ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.322+0000 7f6ee4c00640 1 -- 172.21.15.19:0/659220636 >> v1:172.21.15.188:6789/0 conn(0x7f6ee01590c0 legacy=0x7f6ee015f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.322+0000 7f6ee4c00640 1 -- 172.21.15.19:0/659220636 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee00b4ab0 con 0x7f6ee006ea40 2024-03-28T13:13:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.322+0000 7f6edf400640 1 --2- 172.21.15.19:0/659220636 >> v2:172.21.15.188:3300/0 conn(0x7f6ee006c580 0x7f6ee006ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:13:36.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.323+0000 7f6ee4c00640 1 --2- 172.21.15.19:0/659220636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ee006ea40 0x7f6ee006eef0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0073710 tx=0x7f6ed802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cdb0305e9dd5e2c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.324+0000 7f6edea00640 1 -- 172.21.15.19:0/659220636 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8034070 con 0x7f6ee006ea40 2024-03-28T13:13:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.324+0000 7f6edea00640 1 -- 172.21.15.19:0/659220636 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed8030860 con 0x7f6ee006ea40 2024-03-28T13:13:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.324+0000 7f6edea00640 1 -- 172.21.15.19:0/659220636 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8030c30 con 0x7f6ee006ea40 2024-03-28T13:13:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.324+0000 7f6ee6c00640 1 -- 172.21.15.19:0/659220636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ee006ea40 msgr2=0x7f6ee006eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.324+0000 7f6ee6c00640 1 --2- 172.21.15.19:0/659220636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ee006ea40 0x7f6ee006eef0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0073710 tx=0x7f6ed802b9a0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.324+0000 7f6ee6c00640 1 -- 172.21.15.19:0/659220636 shutdown_connections 2024-03-28T13:13:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.324+0000 7f6ee6c00640 1 --2- 172.21.15.19:0/659220636 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ee006ea40 0x7f6ee006eef0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.324+0000 7f6ee6c00640 1 --2- 172.21.15.19:0/659220636 >> v2:172.21.15.188:3300/0 conn(0x7f6ee006c580 0x7f6ee006ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.324+0000 7f6ee6c00640 1 -- 172.21.15.19:0/659220636 >> 172.21.15.19:0/659220636 conn(0x7f6ee0071e60 msgr2=0x7f6ee0072270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:36.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.324+0000 7f6ee6c00640 1 -- 172.21.15.19:0/659220636 shutdown_connections 2024-03-28T13:13:36.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.324+0000 7f6ee6c00640 1 -- 172.21.15.19:0/659220636 wait complete. 2024-03-28T13:13:36.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.327+0000 7f6ee6c00640 1 Processor -- start 2024-03-28T13:13:36.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.327+0000 7f6ee6c00640 1 -- start start 2024-03-28T13:13:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.329+0000 7f6ee6c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ee006c580 0x7f6ee0143280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.329+0000 7f6ee6c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee006ea40 0x7f6ee0150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.329+0000 7f6ee6c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee01590c0 0x7f6ee01537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.329+0000 7f6ee6c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee00b4380 con 0x7f6ee006c580 2024-03-28T13:13:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.329+0000 7f6ee6c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee006e240 con 0x7f6ee01590c0 2024-03-28T13:13:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.329+0000 7f6ee6c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee00b4500 con 0x7f6ee006ea40 2024-03-28T13:13:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.329+0000 7f6edfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ee006c580 0x7f6ee0143280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.329+0000 7f6edfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ee006c580 0x7f6ee0143280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44616/0 (socket says 172.21.15.19:44616) 2024-03-28T13:13:36.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.329+0000 7f6edfe00640 1 -- 172.21.15.19:0/1135509451 learned_addr learned my addr 172.21.15.19:0/1135509451 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.330+0000 7f6ee4c00640 1 --2- 172.21.15.19:0/1135509451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee01590c0 0x7f6ee01537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.330+0000 7f6edf400640 1 --2- 172.21.15.19:0/1135509451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee006ea40 0x7f6ee0150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.330+0000 7f6edfe00640 1 -- 172.21.15.19:0/1135509451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee006ea40 msgr2=0x7f6ee0150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.330+0000 7f6edfe00640 1 --2- 172.21.15.19:0/1135509451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee006ea40 0x7f6ee0150e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.330+0000 7f6edfe00640 1 -- 172.21.15.19:0/1135509451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee01590c0 msgr2=0x7f6ee01537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.330+0000 7f6edfe00640 1 --2- 172.21.15.19:0/1135509451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee01590c0 0x7f6ee01537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.330+0000 7f6edfe00640 1 -- 172.21.15.19:0/1135509451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee0152180 con 0x7f6ee006c580 2024-03-28T13:13:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.330+0000 7f6ee4c00640 1 --2- 172.21.15.19:0/1135509451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee01590c0 0x7f6ee01537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.330+0000 7f6edfe00640 1 --2- 172.21.15.19:0/1135509451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ee006c580 0x7f6ee0143280 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000c730 tx=0x7f6ed000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.330+0000 7f6ed7e00640 1 -- 172.21.15.19:0/1135509451 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0016020 con 0x7f6ee006c580 2024-03-28T13:13:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.331+0000 7f6ed7e00640 1 -- 172.21.15.19:0/1135509451 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed0006d50 con 0x7f6ee006c580 2024-03-28T13:13:36.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.331+0000 7f6ed7e00640 1 -- 172.21.15.19:0/1135509451 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0003bb0 con 0x7f6ee006c580 2024-03-28T13:13:36.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.331+0000 7f6ee6c00640 1 -- 172.21.15.19:0/1135509451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee0152420 con 0x7f6ee006c580 2024-03-28T13:13:36.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.331+0000 7f6ee6c00640 1 -- 172.21.15.19:0/1135509451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee01529e0 con 0x7f6ee006c580 2024-03-28T13:13:36.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.334+0000 7f6ed7e00640 1 -- 172.21.15.19:0/1135509451 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6ed000a850 con 0x7f6ee006c580 2024-03-28T13:13:36.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.335+0000 7f6ee997c740 1 Processor -- start 2024-03-28T13:13:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.335+0000 7f6ee997c740 1 -- start start 2024-03-28T13:13:36.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.335+0000 7f6ed7e00640 1 --2- 172.21.15.19:0/1135509451 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6ebc06c530 0x7f6ebc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.335+0000 7f6ed7e00640 1 -- 172.21.15.19:0/1135509451 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f6ed008b400 con 0x7f6ee006c580 2024-03-28T13:13:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.336+0000 7f6ee997c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd560 0x55c9f4db0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.336+0000 7f6ee997c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd940 0x55c9f4de02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.336+0000 7f6ee997c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4db0a40 0x55c9f4de0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.336+0000 7f6ee997c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9f4d1b6b0 con 0x55c9f4ddd940 2024-03-28T13:13:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.336+0000 7f6ee997c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c9f4cc3750 con 0x55c9f4db0a40 2024-03-28T13:13:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.336+0000 7f6ee997c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55c9f4d17660 con 0x55c9f4ddd560 2024-03-28T13:13:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.336+0000 7f6edf400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd940 0x55c9f4de02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.336+0000 7f6edf400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd940 0x55c9f4de02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44618/0 (socket says 172.21.15.19:44618) 2024-03-28T13:13:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.336+0000 7f6edf400640 1 -- 172.21.15.19:0/721194484 learned_addr learned my addr 172.21.15.19:0/721194484 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.336+0000 7f6ee4c00640 1 --2- 172.21.15.19:0/1135509451 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6ebc06c530 0x7f6ebc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.336+0000 7f6edfe00640 1 --2- 172.21.15.19:0/721194484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd560 0x55c9f4db0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.336+0000 7f6ee4c00640 1 --2- 172.21.15.19:0/721194484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4db0a40 0x55c9f4de0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.336+0000 7f6edf400640 1 -- 172.21.15.19:0/721194484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd560 msgr2=0x55c9f4db0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.337+0000 7f6edf400640 1 --2- 172.21.15.19:0/721194484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd560 0x55c9f4db0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.337+0000 7f6edf400640 1 -- 172.21.15.19:0/721194484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4db0a40 msgr2=0x55c9f4de0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.337+0000 7f6edf400640 1 --2- 172.21.15.19:0/721194484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4db0a40 0x55c9f4de0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.337+0000 7f6edf400640 1 -- 172.21.15.19:0/721194484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f4af97d0 con 0x55c9f4ddd940 2024-03-28T13:13:36.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.337+0000 7f6edf400640 1 --2- 172.21.15.19:0/721194484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd940 0x55c9f4de02f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6ee004f920 tx=0x7f6ec800dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e8c94daa4d3000 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.337+0000 7f6ecfe00640 1 -- 172.21.15.19:0/721194484 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8018070 con 0x55c9f4ddd940 2024-03-28T13:13:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.337+0000 7f6ecfe00640 1 -- 172.21.15.19:0/721194484 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec800f960 con 0x55c9f4ddd940 2024-03-28T13:13:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.337+0000 7f6ecfe00640 1 -- 172.21.15.19:0/721194484 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec800fd10 con 0x55c9f4ddd940 2024-03-28T13:13:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.338+0000 7f6ee997c740 1 -- 172.21.15.19:0/721194484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd940 msgr2=0x55c9f4de02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.338+0000 7f6ee997c740 1 --2- 172.21.15.19:0/721194484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd940 0x55c9f4de02f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6ee004f920 tx=0x7f6ec800dc80 comp rx=0 tx=0).stop 2024-03-28T13:13:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.338+0000 7f6ee4c00640 1 --2- 172.21.15.19:0/1135509451 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6ebc06c530 0x7f6ebc06e9f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8036000 tx=0x7f6ed8030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.338+0000 7f6ee997c740 1 -- 172.21.15.19:0/721194484 shutdown_connections 2024-03-28T13:13:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.338+0000 7f6ee997c740 1 --2- 172.21.15.19:0/721194484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4db0a40 0x55c9f4de0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.338+0000 7f6ee997c740 1 --2- 172.21.15.19:0/721194484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd940 0x55c9f4de02f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.338+0000 7f6ee997c740 1 --2- 172.21.15.19:0/721194484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd560 0x55c9f4db0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.338+0000 7f6ee997c740 1 -- 172.21.15.19:0/721194484 >> 172.21.15.19:0/721194484 conn(0x55c9f4db2770 msgr2=0x55c9f4dfb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.338+0000 7f6ee997c740 1 -- 172.21.15.19:0/721194484 shutdown_connections 2024-03-28T13:13:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.338+0000 7f6ee997c740 1 -- 172.21.15.19:0/721194484 wait complete. 2024-03-28T13:13:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.339+0000 7f6ee997c740 1 -- start start 2024-03-28T13:13:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.339+0000 7f6ee997c740 1 Processor -- start 2024-03-28T13:13:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.341+0000 7f6ee997c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd560 0x55c9f4de73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.341+0000 7f6ee997c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd940 0x55c9f4de1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.341+0000 7f6ee997c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4e0ebc0 0x55c9f4dfb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.341+0000 7f6ee997c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9f4db4e10 con 0x55c9f4ddd560 2024-03-28T13:13:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.341+0000 7f6ee997c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c9f4cd6a80 con 0x55c9f4e0ebc0 2024-03-28T13:13:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.341+0000 7f6ee997c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55c9f4db7650 con 0x55c9f4ddd940 2024-03-28T13:13:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.341+0000 7f6ee997c740 10 client.0 _ll_register_callbacks cb 0x55c9f4cd6470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:13:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.342+0000 7f6edfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd560 0x55c9f4de73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.342+0000 7f6edfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd560 0x55c9f4de73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44624/0 (socket says 172.21.15.19:44624) 2024-03-28T13:13:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.342+0000 7f6edfe00640 1 -- 172.21.15.19:0/599734566 learned_addr learned my addr 172.21.15.19:0/599734566 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:36.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.342+0000 7f6ee4c00640 1 --2- 172.21.15.19:0/599734566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4e0ebc0 0x55c9f4dfb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.342+0000 7f6edf400640 1 --2- 172.21.15.19:0/599734566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd940 0x55c9f4de1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.342+0000 7f6edfe00640 1 -- 172.21.15.19:0/599734566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd940 msgr2=0x55c9f4de1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.342+0000 7f6edfe00640 1 --2- 172.21.15.19:0/599734566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd940 0x55c9f4de1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.342+0000 7f6edfe00640 1 -- 172.21.15.19:0/599734566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4e0ebc0 msgr2=0x55c9f4dfb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.342+0000 7f6edfe00640 1 --2- 172.21.15.19:0/599734566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4e0ebc0 0x55c9f4dfb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.342+0000 7f6edfe00640 1 -- 172.21.15.19:0/599734566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f4a2f3f0 con 0x55c9f4ddd560 2024-03-28T13:13:36.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.342+0000 7f6edfe00640 1 --2- 172.21.15.19:0/599734566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd560 0x55c9f4de73a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0058fd0 tx=0x7f6ed00592d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d610a2986407bdf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:36.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.342+0000 7f6eccc00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:13:36.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.343+0000 7f6eccc00640 1 -- 172.21.15.19:0/599734566 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0055940 con 0x55c9f4ddd560 2024-03-28T13:13:36.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.343+0000 7f6eccc00640 1 -- 172.21.15.19:0/599734566 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f6ed000ab40 con 0x55c9f4ddd560 2024-03-28T13:13:36.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.343+0000 7f6ee997c740 1 -- 172.21.15.19:0/599734566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec800f960 con 0x55c9f4ddd560 2024-03-28T13:13:36.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.343+0000 7f6eccc00640 1 -- 172.21.15.19:0/599734566 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6ed0055ae0 con 0x55c9f4ddd560 2024-03-28T13:13:36.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.343+0000 7f6ee997c740 10 client.6249 fetch_fsmap learned FSMap version 35 2024-03-28T13:13:36.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.343+0000 7f6ee997c740 1 -- 172.21.15.19:0/599734566 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55c9f49ef0f0 con 0x55c9f4ddd560 2024-03-28T13:13:36.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.343+0000 7f6eccc00640 1 -- 172.21.15.19:0/599734566 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec800f960 con 0x55c9f4ddd560 2024-03-28T13:13:36.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.343+0000 7f6eccc00640 1 -- 172.21.15.19:0/599734566 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f6ed005d150 con 0x55c9f4ddd560 2024-03-28T13:13:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.343+0000 7f6ee997c740 10 client.6249 fetch_fsmap finished waiting for FSMap version 35 2024-03-28T13:13:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.344+0000 7f6ee997c740 10 client.6249 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-03-28T13:13:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.344+0000 7f6ee997c740 20 client.6249 populate_metadata read hostname 'smithi019' 2024-03-28T13:13:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.344+0000 7f6ee997c740 1 --2- 172.21.15.19:0/599734566 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55c9f4de50f0 0x55c9f4de5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.344+0000 7f6ee997c740 4 client.6249 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:13:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.344+0000 7f6ee997c740 1 -- 172.21.15.19:0/599734566 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9f4d28bf0 con 0x55c9f4de50f0 2024-03-28T13:13:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.344+0000 7f6edf400640 1 --2- 172.21.15.19:0/599734566 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55c9f4de50f0 0x55c9f4de5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.345+0000 7f6edf400640 1 --2- 172.21.15.19:0/599734566 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55c9f4de50f0 0x55c9f4de5530 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5721a321e634edcd server_cookie=3521544b69c5ba1f in_seq=0 out_seq=0 2024-03-28T13:13:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.345+0000 7f6eccc00640 10 client.6249 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.345+0000 7f6eccc00640 1 -- 172.21.15.19:0/599734566 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c9f4d28bf0 con 0x55c9f4de50f0 2024-03-28T13:13:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.345+0000 7f6eccc00640 10 client.6249 handle_command_reply: tid=0 2024-03-28T13:13:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.345+0000 7f6ee997c740 1 client.6249 shutdown 2024-03-28T13:13:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.346+0000 7f6ee997c740 1 -- 172.21.15.19:0/599734566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd560 msgr2=0x55c9f4de73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.346+0000 7f6ee997c740 1 --2- 172.21.15.19:0/599734566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd560 0x55c9f4de73a0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0058fd0 tx=0x7f6ed00592d0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.346+0000 7f6ee997c740 1 -- 172.21.15.19:0/599734566 shutdown_connections 2024-03-28T13:13:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.346+0000 7f6ee997c740 1 --2- 172.21.15.19:0/599734566 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4e0ebc0 0x55c9f4dfb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.346+0000 7f6ee997c740 1 --2- 172.21.15.19:0/599734566 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd940 0x55c9f4de1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.346+0000 7f6ee997c740 1 --2- 172.21.15.19:0/599734566 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55c9f4de50f0 0x55c9f4de5530 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.346+0000 7f6ee997c740 1 --2- 172.21.15.19:0/599734566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd560 0x55c9f4de73a0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.346+0000 7f6ee997c740 1 -- 172.21.15.19:0/599734566 >> 172.21.15.19:0/599734566 conn(0x55c9f4db0a40 msgr2=0x55c9f4db44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.346+0000 7f6eccc00640 0 client.6249 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.346+0000 7f6ee997c740 1 -- 172.21.15.19:0/599734566 shutdown_connections 2024-03-28T13:13:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.346+0000 7f6ee997c740 1 -- 172.21.15.19:0/599734566 wait complete. 2024-03-28T13:13:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.346+0000 7f6ee997c740 20 client.6249 trim_cache size 0 max 16384 2024-03-28T13:13:36.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.365+0000 7f6ee997c740 1 Processor -- start 2024-03-28T13:13:36.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.365+0000 7f6ee997c740 1 -- start start 2024-03-28T13:13:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.366+0000 7f6ee997c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4ddd560 0x55c9f4de8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.366+0000 7f6ee997c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd940 0x55c9f4dfebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.366+0000 7f6ee997c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4e0ebc0 0x55c9f4daf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.366+0000 7f6ee997c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee0200c00 con 0x55c9f4e0ebc0 2024-03-28T13:13:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.366+0000 7f6ee997c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee014f5e0 con 0x55c9f4ddd560 2024-03-28T13:13:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.366+0000 7f6ee997c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee01f8960 con 0x55c9f4ddd940 2024-03-28T13:13:36.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.367+0000 7f6ee4c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4e0ebc0 0x55c9f4daf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.367+0000 7f6ee4c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4e0ebc0 0x55c9f4daf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44640/0 (socket says 172.21.15.19:44640) 2024-03-28T13:13:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.367+0000 7f6ee4c00640 1 -- 172.21.15.19:0/324397243 learned_addr learned my addr 172.21.15.19:0/324397243 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.367+0000 7f6edfe00640 1 --2- 172.21.15.19:0/324397243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4ddd560 0x55c9f4de8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.367+0000 7f6edf400640 1 --2- 172.21.15.19:0/324397243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd940 0x55c9f4dfebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.367+0000 7f6ee4c00640 1 -- 172.21.15.19:0/324397243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd940 msgr2=0x55c9f4dfebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.367+0000 7f6ee4c00640 1 --2- 172.21.15.19:0/324397243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd940 0x55c9f4dfebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.367+0000 7f6ee4c00640 1 -- 172.21.15.19:0/324397243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4ddd560 msgr2=0x55c9f4de8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.367+0000 7f6ee4c00640 1 --2- 172.21.15.19:0/324397243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4ddd560 0x55c9f4de8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.367+0000 7f6ee4c00640 1 -- 172.21.15.19:0/324397243 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f4d177e0 con 0x55c9f4e0ebc0 2024-03-28T13:13:36.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.367+0000 7f6edfe00640 1 --2- 172.21.15.19:0/324397243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4ddd560 0x55c9f4de8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:13:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.368+0000 7f6ee4c00640 1 --2- 172.21.15.19:0/324397243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4e0ebc0 0x55c9f4daf390 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009930 tx=0x7f6ed8031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a763ab04e355b756 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.368+0000 7f6ecf400640 1 -- 172.21.15.19:0/324397243 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8053070 con 0x55c9f4e0ebc0 2024-03-28T13:13:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.368+0000 7f6ecf400640 1 -- 172.21.15.19:0/324397243 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed8039c90 con 0x55c9f4e0ebc0 2024-03-28T13:13:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.368+0000 7f6ecf400640 1 -- 172.21.15.19:0/324397243 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8034040 con 0x55c9f4e0ebc0 2024-03-28T13:13:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.369+0000 7f6ee997c740 1 -- 172.21.15.19:0/324397243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4e0ebc0 msgr2=0x55c9f4daf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.369+0000 7f6ee997c740 1 --2- 172.21.15.19:0/324397243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4e0ebc0 0x55c9f4daf390 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8009930 tx=0x7f6ed8031000 comp rx=0 tx=0).stop 2024-03-28T13:13:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.369+0000 7f6ee997c740 1 -- 172.21.15.19:0/324397243 shutdown_connections 2024-03-28T13:13:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.369+0000 7f6ee997c740 1 --2- 172.21.15.19:0/324397243 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4e0ebc0 0x55c9f4daf390 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.369+0000 7f6ee997c740 1 --2- 172.21.15.19:0/324397243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd940 0x55c9f4dfebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.369+0000 7f6ee997c740 1 --2- 172.21.15.19:0/324397243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4ddd560 0x55c9f4de8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.369+0000 7f6ee997c740 1 -- 172.21.15.19:0/324397243 >> 172.21.15.19:0/324397243 conn(0x55c9f4db0a40 msgr2=0x55c9f4de71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:36.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.369+0000 7f6ee997c740 1 -- 172.21.15.19:0/324397243 shutdown_connections 2024-03-28T13:13:36.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.369+0000 7f6ee997c740 1 -- 172.21.15.19:0/324397243 wait complete. 2024-03-28T13:13:36.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.369+0000 7f6ee997c740 1 -- start start 2024-03-28T13:13:36.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.370+0000 7f6ee997c740 1 Processor -- start 2024-03-28T13:13:36.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6ee997c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd560 0x55c9f4e0e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6ee997c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd940 0x55c9f4e60610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6ee997c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4e0ebc0 0x55c9f4ddb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6ee997c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9f4dfb280 con 0x55c9f4ddd560 2024-03-28T13:13:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6ee997c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee014ef90 con 0x55c9f4e0ebc0 2024-03-28T13:13:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6ee997c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55c9f4de56a0 con 0x55c9f4ddd940 2024-03-28T13:13:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6ee997c740 10 client.0 _ll_register_callbacks cb 0x7f6ed0019400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:13:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6edfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd560 0x55c9f4e0e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6edfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd560 0x55c9f4e0e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44650/0 (socket says 172.21.15.19:44650) 2024-03-28T13:13:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6edfe00640 1 -- 172.21.15.19:0/3731727387 learned_addr learned my addr 172.21.15.19:0/3731727387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:13:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6edf400640 1 --2- 172.21.15.19:0/3731727387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd940 0x55c9f4e60610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6edfe00640 1 -- 172.21.15.19:0/3731727387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd940 msgr2=0x55c9f4e60610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6ee4c00640 1 --2- 172.21.15.19:0/3731727387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4e0ebc0 0x55c9f4ddb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6edfe00640 1 --2- 172.21.15.19:0/3731727387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd940 0x55c9f4e60610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6edfe00640 1 -- 172.21.15.19:0/3731727387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4e0ebc0 msgr2=0x55c9f4ddb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:13:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6edfe00640 1 --2- 172.21.15.19:0/3731727387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4e0ebc0 0x55c9f4ddb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.371+0000 7f6edfe00640 1 -- 172.21.15.19:0/3731727387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f4e63290 con 0x55c9f4ddd560 2024-03-28T13:13:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.372+0000 7f6edfe00640 1 --2- 172.21.15.19:0/3731727387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd560 0x55c9f4e0e600 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6ed005d5d0 tx=0x7f6ed00897b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc212d87dc0ab7f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:13:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.372+0000 7f6eccc00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:13:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.372+0000 7f6eccc00640 1 -- 172.21.15.19:0/3731727387 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0055940 con 0x55c9f4ddd560 2024-03-28T13:13:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.372+0000 7f6eccc00640 1 -- 172.21.15.19:0/3731727387 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f6ed0058720 con 0x55c9f4ddd560 2024-03-28T13:13:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.372+0000 7f6ee997c740 1 -- 172.21.15.19:0/3731727387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ed8039c90 con 0x55c9f4ddd560 2024-03-28T13:13:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.372+0000 7f6eccc00640 1 -- 172.21.15.19:0/3731727387 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6ed0055570 con 0x55c9f4ddd560 2024-03-28T13:13:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.372+0000 7f6ee997c740 10 client.6255 fetch_fsmap learned FSMap version 35 2024-03-28T13:13:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.372+0000 7f6ee997c740 1 -- 172.21.15.19:0/3731727387 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55c9f4ddc280 con 0x55c9f4ddd560 2024-03-28T13:13:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.373+0000 7f6eccc00640 1 -- 172.21.15.19:0/3731727387 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed8039c90 con 0x55c9f4ddd560 2024-03-28T13:13:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.373+0000 7f6eccc00640 1 -- 172.21.15.19:0/3731727387 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 35) ==== 3494+0+0 (secure 0 0 0) 0x7f6ed006daf0 con 0x55c9f4ddd560 2024-03-28T13:13:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.373+0000 7f6ee997c740 10 client.6255 fetch_fsmap finished waiting for FSMap version 35 2024-03-28T13:13:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.373+0000 7f6ee997c740 10 client.6255 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-03-28T13:13:36.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.373+0000 7f6ee997c740 20 client.6255 populate_metadata read hostname 'smithi019' 2024-03-28T13:13:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.373+0000 7f6ee997c740 1 --2- 172.21.15.19:0/3731727387 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55c9f4e1f520 0x55c9f4ddbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:13:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.373+0000 7f6ee997c740 4 client.6255 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:13:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.373+0000 7f6ee997c740 1 -- 172.21.15.19:0/3731727387 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c9f4d22f40 con 0x55c9f4e1f520 2024-03-28T13:13:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.373+0000 7f6edf400640 1 --2- 172.21.15.19:0/3731727387 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55c9f4e1f520 0x55c9f4ddbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:13:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.373+0000 7f6edf400640 1 --2- 172.21.15.19:0/3731727387 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55c9f4e1f520 0x55c9f4ddbc80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55b1bb9cb862f0e2 server_cookie=48b89774faddf803 in_seq=0 out_seq=0 2024-03-28T13:13:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.373+0000 7f6eccc00640 10 client.6255 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.374+0000 7f6eccc00640 1 -- 172.21.15.19:0/3731727387 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55c9f4d22f40 con 0x55c9f4e1f520 2024-03-28T13:13:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.374+0000 7f6eccc00640 10 client.6255 handle_command_reply: tid=0 2024-03-28T13:13:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.374+0000 7f6ee997c740 1 client.6255 shutdown 2024-03-28T13:13:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.374+0000 7f6ee997c740 1 -- 172.21.15.19:0/3731727387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd560 msgr2=0x55c9f4e0e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.374+0000 7f6ee997c740 1 --2- 172.21.15.19:0/3731727387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd560 0x55c9f4e0e600 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6ed005d5d0 tx=0x7f6ed00897b0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.374+0000 7f6ee997c740 1 -- 172.21.15.19:0/3731727387 shutdown_connections 2024-03-28T13:13:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.374+0000 7f6ee997c740 1 --2- 172.21.15.19:0/3731727387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c9f4e0ebc0 0x55c9f4ddb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.374+0000 7f6ee997c740 1 --2- 172.21.15.19:0/3731727387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c9f4ddd940 0x55c9f4e60610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.374+0000 7f6ee997c740 1 --2- 172.21.15.19:0/3731727387 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55c9f4e1f520 0x55c9f4ddbc80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.374+0000 7f6ee997c740 1 --2- 172.21.15.19:0/3731727387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f4ddd560 0x55c9f4e0e600 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.374+0000 7f6eccc00640 0 client.6255 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:13:36.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.374+0000 7f6ee997c740 1 -- 172.21.15.19:0/3731727387 >> 172.21.15.19:0/3731727387 conn(0x55c9f4db0a40 msgr2=0x55c9f4de5ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:36.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.374+0000 7f6ee997c740 1 -- 172.21.15.19:0/3731727387 shutdown_connections 2024-03-28T13:13:36.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.374+0000 7f6ee997c740 1 -- 172.21.15.19:0/3731727387 wait complete. 2024-03-28T13:13:36.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.374+0000 7f6ee997c740 20 client.6255 trim_cache size 0 max 16384 2024-03-28T13:13:36.379 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:13:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T13:13:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-03-28T13:13:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "113b93b3-052c-4446-81c8-a582b6c64ff4": { 2024-03-28T13:13:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-03-28T13:13:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "113b93b3-052c-4446-81c8-a582b6c64ff4", 2024-03-28T13:13:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,repair,force" 2024-03-28T13:13:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:13:36.379 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:13:36.379 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:13:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.375+0000 7f6ee6c00640 1 -- 172.21.15.19:0/1135509451 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6ebc06c530 msgr2=0x7f6ebc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.375+0000 7f6ee6c00640 1 --2- 172.21.15.19:0/1135509451 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6ebc06c530 0x7f6ebc06e9f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8036000 tx=0x7f6ed8030000 comp rx=0 tx=0).stop 2024-03-28T13:13:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.375+0000 7f6ee6c00640 1 -- 172.21.15.19:0/1135509451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ee006c580 msgr2=0x7f6ee0143280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:13:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.375+0000 7f6ee6c00640 1 --2- 172.21.15.19:0/1135509451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ee006c580 0x7f6ee0143280 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000c730 tx=0x7f6ed000cbf0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.375+0000 7f6ee6c00640 1 -- 172.21.15.19:0/1135509451 shutdown_connections 2024-03-28T13:13:36.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.375+0000 7f6ee6c00640 1 --2- 172.21.15.19:0/1135509451 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6ebc06c530 0x7f6ebc06e9f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.375+0000 7f6ee6c00640 1 --2- 172.21.15.19:0/1135509451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ee01590c0 0x7f6ee01537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.375+0000 7f6ee6c00640 1 --2- 172.21.15.19:0/1135509451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ee006ea40 0x7f6ee0150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.375+0000 7f6ee6c00640 1 --2- 172.21.15.19:0/1135509451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ee006c580 0x7f6ee0143280 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:13:36.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.375+0000 7f6ee6c00640 1 -- 172.21.15.19:0/1135509451 >> 172.21.15.19:0/1135509451 conn(0x7f6ee0071e60 msgr2=0x7f6ee00af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:13:36.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.375+0000 7f6ee6c00640 1 -- 172.21.15.19:0/1135509451 shutdown_connections 2024-03-28T13:13:36.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:13:36.375+0000 7f6ee6c00640 1 -- 172.21.15.19:0/1135509451 wait complete. 2024-03-28T13:13:36.392 INFO:tasks.cephfs.filesystem:scrub status for tag:113b93b3-052c-4446-81c8-a582b6c64ff4 - {'path': '/', 'tag': '113b93b3-052c-4446-81c8-a582b6c64ff4', 'options': 'recursive,repair,force'} 2024-03-28T13:14:02.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:14:02.267 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:14:06.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:06.393 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-28T13:14:06.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.622+0000 7f0979c00640 1 Processor -- start 2024-03-28T13:14:06.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.622+0000 7f0979c00640 1 -- start start 2024-03-28T13:14:06.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.623+0000 7f0979c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f09740d0e70 0x7f09740cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.623+0000 7f0979c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f097415f6e0 0x7f09740d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.623+0000 7f0979c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f09741621d0 0x7f0974162660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.623+0000 7f0979c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f09740d1790 con 0x7f09741621d0 2024-03-28T13:14:06.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.623+0000 7f0979c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f09740cf6c0 con 0x7f09740d0e70 2024-03-28T13:14:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.623+0000 7f0979c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f09740cf9a0 con 0x7f097415f6e0 2024-03-28T13:14:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.623+0000 7f0973400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f09741621d0 0x7f0974162660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.623+0000 7f0973400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f09741621d0 0x7f0974162660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48400/0 (socket says 172.21.15.19:48400) 2024-03-28T13:14:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.623+0000 7f0973400640 1 -- 172.21.15.19:0/204975283 learned_addr learned my addr 172.21.15.19:0/204975283 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.624+0000 7f0972a00640 1 --2- 172.21.15.19:0/204975283 >> v2:172.21.15.188:3300/0 conn(0x7f09740d0e70 0x7f09740cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.624+0000 7f0973400640 1 -- 172.21.15.19:0/204975283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f097415f6e0 msgr2=0x7f09740d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.624+0000 7f0972000640 1 --2- 172.21.15.19:0/204975283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f097415f6e0 0x7f09740d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.624+0000 7f0973400640 1 --2- 172.21.15.19:0/204975283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f097415f6e0 0x7f09740d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.624+0000 7f0973400640 1 -- 172.21.15.19:0/204975283 >> v2:172.21.15.188:3300/0 conn(0x7f09740d0e70 msgr2=0x7f09740cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.624+0000 7f0973400640 1 --2- 172.21.15.19:0/204975283 >> v2:172.21.15.188:3300/0 conn(0x7f09740d0e70 0x7f09740cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.624+0000 7f0973400640 1 -- 172.21.15.19:0/204975283 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09740cfc80 con 0x7f09741621d0 2024-03-28T13:14:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.624+0000 7f0972a00640 1 --2- 172.21.15.19:0/204975283 >> v2:172.21.15.188:3300/0 conn(0x7f09740d0e70 0x7f09740cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.624+0000 7f0972000640 1 --2- 172.21.15.19:0/204975283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f097415f6e0 0x7f09740d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.625+0000 7f0973400640 1 --2- 172.21.15.19:0/204975283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09741621d0 0x7f0974162660 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f097408cf80 tx=0x7f096c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77abc9e2fff9479d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.625+0000 7f0971600640 1 -- 172.21.15.19:0/204975283 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c02c9f0 con 0x7f09741621d0 2024-03-28T13:14:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.625+0000 7f0971600640 1 -- 172.21.15.19:0/204975283 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096c02cb90 con 0x7f09741621d0 2024-03-28T13:14:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.625+0000 7f0971600640 1 -- 172.21.15.19:0/204975283 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c0343f0 con 0x7f09741621d0 2024-03-28T13:14:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.626+0000 7f0979c00640 1 -- 172.21.15.19:0/204975283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09741621d0 msgr2=0x7f0974162660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.626+0000 7f0979c00640 1 --2- 172.21.15.19:0/204975283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09741621d0 0x7f0974162660 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f097408cf80 tx=0x7f096c02be50 comp rx=0 tx=0).stop 2024-03-28T13:14:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.626+0000 7f0979c00640 1 -- 172.21.15.19:0/204975283 shutdown_connections 2024-03-28T13:14:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.626+0000 7f0979c00640 1 --2- 172.21.15.19:0/204975283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09741621d0 0x7f0974162660 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.626+0000 7f0979c00640 1 --2- 172.21.15.19:0/204975283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f097415f6e0 0x7f09740d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.626+0000 7f0979c00640 1 --2- 172.21.15.19:0/204975283 >> v2:172.21.15.188:3300/0 conn(0x7f09740d0e70 0x7f09740cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.626+0000 7f0979c00640 1 -- 172.21.15.19:0/204975283 >> 172.21.15.19:0/204975283 conn(0x7f097408bfb0 msgr2=0x7f09740d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.626+0000 7f0979c00640 1 -- 172.21.15.19:0/204975283 shutdown_connections 2024-03-28T13:14:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.626+0000 7f0979c00640 1 -- 172.21.15.19:0/204975283 wait complete. 2024-03-28T13:14:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.629+0000 7f0979c00640 1 Processor -- start 2024-03-28T13:14:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.629+0000 7f0979c00640 1 -- start start 2024-03-28T13:14:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.631+0000 7f0979c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09740d0e70 0x7f09741f19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.631+0000 7f0979c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f097415f6e0 0x7f09741f8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.631+0000 7f0979c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09741621d0 0x7f09740c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.631+0000 7f0979c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f097415fb40 con 0x7f097415f6e0 2024-03-28T13:14:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.631+0000 7f0979c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f09740d0240 con 0x7f09740d0e70 2024-03-28T13:14:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.631+0000 7f0979c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f097415e910 con 0x7f09741621d0 2024-03-28T13:14:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.631+0000 7f0972000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f097415f6e0 0x7f09741f8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.631+0000 7f0972000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f097415f6e0 0x7f09741f8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48414/0 (socket says 172.21.15.19:48414) 2024-03-28T13:14:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.631+0000 7f0972000640 1 -- 172.21.15.19:0/2522774911 learned_addr learned my addr 172.21.15.19:0/2522774911 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.632+0000 7f0972a00640 1 --2- 172.21.15.19:0/2522774911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09740d0e70 0x7f09741f19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.632+0000 7f0973400640 1 --2- 172.21.15.19:0/2522774911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09741621d0 0x7f09740c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.632+0000 7f0972000640 1 -- 172.21.15.19:0/2522774911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09741621d0 msgr2=0x7f09740c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.632+0000 7f0972000640 1 --2- 172.21.15.19:0/2522774911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09741621d0 0x7f09740c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.632+0000 7f0972000640 1 -- 172.21.15.19:0/2522774911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09740d0e70 msgr2=0x7f09741f19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:06.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.632+0000 7f0972000640 1 --2- 172.21.15.19:0/2522774911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09740d0e70 0x7f09741f19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.632+0000 7f0972000640 1 -- 172.21.15.19:0/2522774911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09741f8940 con 0x7f097415f6e0 2024-03-28T13:14:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.632+0000 7f0972a00640 1 --2- 172.21.15.19:0/2522774911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09740d0e70 0x7f09741f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.632+0000 7f0972000640 1 --2- 172.21.15.19:0/2522774911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f097415f6e0 0x7f09741f8400 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f095c00c7f0 tx=0x7f095c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.633+0000 7f096aa00640 1 -- 172.21.15.19:0/2522774911 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c015070 con 0x7f097415f6e0 2024-03-28T13:14:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.633+0000 7f096aa00640 1 -- 172.21.15.19:0/2522774911 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c007a30 con 0x7f097415f6e0 2024-03-28T13:14:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.633+0000 7f096aa00640 1 -- 172.21.15.19:0/2522774911 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c007e00 con 0x7f097415f6e0 2024-03-28T13:14:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.633+0000 7f0979c00640 1 -- 172.21.15.19:0/2522774911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09741f45f0 con 0x7f097415f6e0 2024-03-28T13:14:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.633+0000 7f0979c00640 1 -- 172.21.15.19:0/2522774911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09741f4b90 con 0x7f097415f6e0 2024-03-28T13:14:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.636+0000 7f096aa00640 1 -- 172.21.15.19:0/2522774911 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f095c018070 con 0x7f097415f6e0 2024-03-28T13:14:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.637+0000 7f097c9b0740 1 Processor -- start 2024-03-28T13:14:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.637+0000 7f096aa00640 1 --2- 172.21.15.19:0/2522774911 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f095006c530 0x7f095006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.637+0000 7f097c9b0740 1 -- start start 2024-03-28T13:14:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.638+0000 7f096aa00640 1 -- 172.21.15.19:0/2522774911 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f095c08b640 con 0x7f097415f6e0 2024-03-28T13:14:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.638+0000 7f0973400640 1 --2- 172.21.15.19:0/2522774911 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f095006c530 0x7f095006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.638+0000 7f097c9b0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1ec4e0 0x55e1fa207ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.638+0000 7f097c9b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fd8c0 0x55e1fa1ec8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.638+0000 7f097c9b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa209c30 0x55e1fa20a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.638+0000 7f097c9b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e1fa1276b0 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.638+0000 7f097c9b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e1fa0cf750 con 0x55e1fa209c30 2024-03-28T13:14:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.638+0000 7f097c9b0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55e1fa123660 con 0x55e1fa1ec4e0 2024-03-28T13:14:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.639+0000 7f0972000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fd8c0 0x55e1fa1ec8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.639+0000 7f0972000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fd8c0 0x55e1fa1ec8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48416/0 (socket says 172.21.15.19:48416) 2024-03-28T13:14:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.639+0000 7f0972000640 1 -- 172.21.15.19:0/4084873447 learned_addr learned my addr 172.21.15.19:0/4084873447 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.639+0000 7f0972a00640 1 --2- 172.21.15.19:0/4084873447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1ec4e0 0x55e1fa207ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.639+0000 7f0973400640 1 --2- 172.21.15.19:0/4084873447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa209c30 0x55e1fa20a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.639+0000 7f0972000640 1 -- 172.21.15.19:0/4084873447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1ec4e0 msgr2=0x55e1fa207ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.639+0000 7f0972000640 1 --2- 172.21.15.19:0/4084873447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1ec4e0 0x55e1fa207ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.639+0000 7f0972000640 1 -- 172.21.15.19:0/4084873447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa209c30 msgr2=0x55e1fa20a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.639+0000 7f0972000640 1 --2- 172.21.15.19:0/4084873447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa209c30 0x55e1fa20a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.639+0000 7f0972000640 1 -- 172.21.15.19:0/4084873447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1f9f057d0 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.639+0000 7f0972a00640 1 --2- 172.21.15.19:0/4084873447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1ec4e0 0x55e1fa207ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.640+0000 7f0973400640 1 --2- 172.21.15.19:0/2522774911 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f095006c530 0x7f095006e9f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f096c035000 tx=0x7f096c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.640+0000 7f0972000640 1 --2- 172.21.15.19:0/4084873447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fd8c0 0x55e1fa1ec8c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f095c061bc0 tx=0x7f095c059cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb72ea83fd6217a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.640+0000 7f0963400640 1 -- 172.21.15.19:0/4084873447 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c01f040 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.640+0000 7f0963400640 1 -- 172.21.15.19:0/4084873447 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c0620f0 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.640+0000 7f0963400640 1 -- 172.21.15.19:0/4084873447 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c055b80 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.641+0000 7f097c9b0740 1 -- 172.21.15.19:0/4084873447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fd8c0 msgr2=0x55e1fa1ec8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:06.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.641+0000 7f097c9b0740 1 --2- 172.21.15.19:0/4084873447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fd8c0 0x55e1fa1ec8c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f095c061bc0 tx=0x7f095c059cd0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.641+0000 7f097c9b0740 1 -- 172.21.15.19:0/4084873447 shutdown_connections 2024-03-28T13:14:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.641+0000 7f097c9b0740 1 --2- 172.21.15.19:0/4084873447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa209c30 0x55e1fa20a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.641+0000 7f097c9b0740 1 --2- 172.21.15.19:0/4084873447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fd8c0 0x55e1fa1ec8c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.641+0000 7f097c9b0740 1 --2- 172.21.15.19:0/4084873447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1ec4e0 0x55e1fa207ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.641+0000 7f097c9b0740 1 -- 172.21.15.19:0/4084873447 >> 172.21.15.19:0/4084873447 conn(0x55e1fa1be770 msgr2=0x55e1fa2072f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.641+0000 7f097c9b0740 1 -- 172.21.15.19:0/4084873447 shutdown_connections 2024-03-28T13:14:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.641+0000 7f097c9b0740 1 -- 172.21.15.19:0/4084873447 wait complete. 2024-03-28T13:14:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.642+0000 7f097c9b0740 1 -- start start 2024-03-28T13:14:06.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.642+0000 7f097c9b0740 1 Processor -- start 2024-03-28T13:14:06.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.644+0000 7f097c9b0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1fd8c0 0x55e1fa1ea790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.644+0000 7f097c9b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa209c30 0x55e1fa1ffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.644+0000 7f097c9b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fa570 0x55e1fa1fa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.644+0000 7f097c9b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e1fa1bc500 con 0x55e1fa1fa570 2024-03-28T13:14:06.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.644+0000 7f097c9b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e1fa0e2a80 con 0x55e1fa209c30 2024-03-28T13:14:06.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.644+0000 7f097c9b0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55e1fa1c0b10 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.644+0000 7f097c9b0740 10 client.0 _ll_register_callbacks cb 0x55e1fa0e2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:14:06.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.644+0000 7f0973400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fa570 0x55e1fa1fa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.645+0000 7f0973400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fa570 0x55e1fa1fa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48422/0 (socket says 172.21.15.19:48422) 2024-03-28T13:14:06.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.645+0000 7f0973400640 1 -- 172.21.15.19:0/3282240 learned_addr learned my addr 172.21.15.19:0/3282240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:06.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.645+0000 7f0972000640 1 --2- 172.21.15.19:0/3282240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa209c30 0x55e1fa1ffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.645+0000 7f0972a00640 1 --2- 172.21.15.19:0/3282240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1fd8c0 0x55e1fa1ea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.645+0000 7f0973400640 1 -- 172.21.15.19:0/3282240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1fd8c0 msgr2=0x55e1fa1ea790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:06.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.645+0000 7f0973400640 1 --2- 172.21.15.19:0/3282240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1fd8c0 0x55e1fa1ea790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.645+0000 7f0973400640 1 -- 172.21.15.19:0/3282240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa209c30 msgr2=0x55e1fa1ffbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:06.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.645+0000 7f0973400640 1 --2- 172.21.15.19:0/3282240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa209c30 0x55e1fa1ffbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.645+0000 7f0973400640 1 -- 172.21.15.19:0/3282240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1f9e3b3f0 con 0x55e1fa1fa570 2024-03-28T13:14:06.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.645+0000 7f0973400640 1 --2- 172.21.15.19:0/3282240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fa570 0x55e1fa1fa9f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f096c02cd10 tx=0x7f096c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b82707d738308c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:06.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.645+0000 7f0957e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:14:06.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.646+0000 7f0957e00640 1 -- 172.21.15.19:0/3282240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c04f020 con 0x55e1fa1fa570 2024-03-28T13:14:06.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.646+0000 7f097c9b0740 1 -- 172.21.15.19:0/3282240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f095c0620f0 con 0x55e1fa1fa570 2024-03-28T13:14:06.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.646+0000 7f0957e00640 1 -- 172.21.15.19:0/3282240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f096c04b830 con 0x55e1fa1fa570 2024-03-28T13:14:06.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.646+0000 7f0957e00640 1 -- 172.21.15.19:0/3282240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f096c047070 con 0x55e1fa1fa570 2024-03-28T13:14:06.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.646+0000 7f097c9b0740 10 client.6267 fetch_fsmap learned FSMap version 36 2024-03-28T13:14:06.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.646+0000 7f097c9b0740 1 -- 172.21.15.19:0/3282240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55e1f9dfb0f0 con 0x55e1fa1fa570 2024-03-28T13:14:06.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.647+0000 7f0957e00640 1 -- 172.21.15.19:0/3282240 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c0620f0 con 0x55e1fa1fa570 2024-03-28T13:14:06.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.647+0000 7f0957e00640 1 -- 172.21.15.19:0/3282240 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f096c05c020 con 0x55e1fa1fa570 2024-03-28T13:14:06.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.647+0000 7f097c9b0740 10 client.6267 fetch_fsmap finished waiting for FSMap version 36 2024-03-28T13:14:06.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.647+0000 7f097c9b0740 10 client.6267 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-03-28T13:14:06.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.647+0000 7f097c9b0740 20 client.6267 populate_metadata read hostname 'smithi019' 2024-03-28T13:14:06.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.647+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3282240 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55e1fa223ef0 0x55e1fa220470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.647+0000 7f097c9b0740 4 client.6267 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:14:06.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.647+0000 7f097c9b0740 1 -- 172.21.15.19:0/3282240 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1fa134bf0 con 0x55e1fa223ef0 2024-03-28T13:14:06.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.647+0000 7f0972a00640 1 --2- 172.21.15.19:0/3282240 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55e1fa223ef0 0x55e1fa220470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.648+0000 7f0972a00640 1 --2- 172.21.15.19:0/3282240 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55e1fa223ef0 0x55e1fa220470 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eeb23ab833adc81 server_cookie=8740238feabf4de7 in_seq=0 out_seq=0 2024-03-28T13:14:06.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.648+0000 7f0957e00640 10 client.6267 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:14:06.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.648+0000 7f0957e00640 1 -- 172.21.15.19:0/3282240 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1fa134bf0 con 0x55e1fa223ef0 2024-03-28T13:14:06.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.648+0000 7f0957e00640 10 client.6267 handle_command_reply: tid=0 2024-03-28T13:14:06.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.649+0000 7f097c9b0740 1 client.6267 shutdown 2024-03-28T13:14:06.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.649+0000 7f097c9b0740 1 -- 172.21.15.19:0/3282240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fa570 msgr2=0x55e1fa1fa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.649+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3282240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fa570 0x55e1fa1fa9f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f096c02cd10 tx=0x7f096c004ab0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.649+0000 7f097c9b0740 1 -- 172.21.15.19:0/3282240 shutdown_connections 2024-03-28T13:14:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.649+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3282240 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55e1fa223ef0 0x55e1fa220470 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.649+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3282240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fa570 0x55e1fa1fa9f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.649+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3282240 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa209c30 0x55e1fa1ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.649+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3282240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1fd8c0 0x55e1fa1ea790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.649+0000 7f0957e00640 0 client.6267 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:14:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.649+0000 7f097c9b0740 1 -- 172.21.15.19:0/3282240 >> 172.21.15.19:0/3282240 conn(0x55e1fa1ec4e0 msgr2=0x55e1fa2072f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:06.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.649+0000 7f097c9b0740 1 -- 172.21.15.19:0/3282240 shutdown_connections 2024-03-28T13:14:06.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.649+0000 7f097c9b0740 1 -- 172.21.15.19:0/3282240 wait complete. 2024-03-28T13:14:06.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.649+0000 7f097c9b0740 20 client.6267 trim_cache size 0 max 16384 2024-03-28T13:14:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.668+0000 7f097c9b0740 1 Processor -- start 2024-03-28T13:14:06.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.669+0000 7f097c9b0740 1 -- start start 2024-03-28T13:14:06.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.669+0000 7f097c9b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa1fa570 0x55e1fa1ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.669+0000 7f097c9b0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1fd8c0 0x55e1fa1eb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.669+0000 7f097c9b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa209c30 0x55e1fa1eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.669+0000 7f097c9b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f09740706f0 con 0x55e1fa209c30 2024-03-28T13:14:06.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.669+0000 7f097c9b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f097406df30 con 0x55e1fa1fa570 2024-03-28T13:14:06.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.669+0000 7f097c9b0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f097406d290 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.670+0000 7f0973400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa209c30 0x55e1fa1eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.670+0000 7f0973400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa209c30 0x55e1fa1eb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48434/0 (socket says 172.21.15.19:48434) 2024-03-28T13:14:06.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.670+0000 7f0973400640 1 -- 172.21.15.19:0/3128171150 learned_addr learned my addr 172.21.15.19:0/3128171150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:06.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.670+0000 7f0972000640 1 --2- 172.21.15.19:0/3128171150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1fd8c0 0x55e1fa1eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.670+0000 7f0972a00640 1 --2- 172.21.15.19:0/3128171150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa1fa570 0x55e1fa1ef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.670+0000 7f0973400640 1 -- 172.21.15.19:0/3128171150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1fd8c0 msgr2=0x55e1fa1eb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:06.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.670+0000 7f0973400640 1 --2- 172.21.15.19:0/3128171150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1fd8c0 0x55e1fa1eb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.670+0000 7f0973400640 1 -- 172.21.15.19:0/3128171150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa1fa570 msgr2=0x55e1fa1ef8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:06.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.670+0000 7f0973400640 1 --2- 172.21.15.19:0/3128171150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa1fa570 0x55e1fa1ef8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.670+0000 7f0973400640 1 -- 172.21.15.19:0/3128171150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1fa1be450 con 0x55e1fa209c30 2024-03-28T13:14:06.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.671+0000 7f0973400640 1 --2- 172.21.15.19:0/3128171150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa209c30 0x55e1fa1eb760 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f096c02c640 tx=0x7f096c049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ae8e2043306f35 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:06.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.671+0000 7f0962a00640 1 -- 172.21.15.19:0/3128171150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c04f040 con 0x55e1fa209c30 2024-03-28T13:14:06.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.671+0000 7f0962a00640 1 -- 172.21.15.19:0/3128171150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096c04b630 con 0x55e1fa209c30 2024-03-28T13:14:06.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.671+0000 7f0962a00640 1 -- 172.21.15.19:0/3128171150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c003990 con 0x55e1fa209c30 2024-03-28T13:14:06.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.672+0000 7f097c9b0740 1 -- 172.21.15.19:0/3128171150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa209c30 msgr2=0x55e1fa1eb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:06.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.672+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3128171150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa209c30 0x55e1fa1eb760 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f096c02c640 tx=0x7f096c049000 comp rx=0 tx=0).stop 2024-03-28T13:14:06.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.672+0000 7f097c9b0740 1 -- 172.21.15.19:0/3128171150 shutdown_connections 2024-03-28T13:14:06.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.672+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3128171150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa209c30 0x55e1fa1eb760 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.672+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3128171150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa1fd8c0 0x55e1fa1eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.672+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3128171150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa1fa570 0x55e1fa1ef8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.672+0000 7f097c9b0740 1 -- 172.21.15.19:0/3128171150 >> 172.21.15.19:0/3128171150 conn(0x55e1fa1ec4e0 msgr2=0x55e1fa1b7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:06.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.672+0000 7f097c9b0740 1 -- 172.21.15.19:0/3128171150 shutdown_connections 2024-03-28T13:14:06.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.672+0000 7f097c9b0740 1 -- 172.21.15.19:0/3128171150 wait complete. 2024-03-28T13:14:06.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.672+0000 7f097c9b0740 1 -- start start 2024-03-28T13:14:06.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.673+0000 7f097c9b0740 1 Processor -- start 2024-03-28T13:14:06.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.674+0000 7f097c9b0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa1fa570 0x55e1fa1efca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.674+0000 7f097c9b0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fd8c0 0x55e1fa2898e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.674+0000 7f097c9b0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa209c30 0x55e1fa289e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.674+0000 7f097c9b0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e1fa2005a0 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.674+0000 7f097c9b0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f097406d8e0 con 0x55e1fa1fa570 2024-03-28T13:14:06.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.674+0000 7f097c9b0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55e1fa1ecec0 con 0x55e1fa209c30 2024-03-28T13:14:06.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.674+0000 7f097c9b0740 10 client.0 _ll_register_callbacks cb 0x7f096c040450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:14:06.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.674+0000 7f0972000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fd8c0 0x55e1fa2898e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.674+0000 7f0972000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fd8c0 0x55e1fa2898e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48444/0 (socket says 172.21.15.19:48444) 2024-03-28T13:14:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.674+0000 7f0972000640 1 -- 172.21.15.19:0/3028256140 learned_addr learned my addr 172.21.15.19:0/3028256140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.674+0000 7f0972a00640 1 --2- 172.21.15.19:0/3028256140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa1fa570 0x55e1fa1efca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.674+0000 7f0973400640 1 --2- 172.21.15.19:0/3028256140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa209c30 0x55e1fa289e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.675+0000 7f0972000640 1 -- 172.21.15.19:0/3028256140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa209c30 msgr2=0x55e1fa289e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.675+0000 7f0972000640 1 --2- 172.21.15.19:0/3028256140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa209c30 0x55e1fa289e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.675+0000 7f0972000640 1 -- 172.21.15.19:0/3028256140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa1fa570 msgr2=0x55e1fa1efca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.675+0000 7f0972000640 1 --2- 172.21.15.19:0/3028256140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa1fa570 0x55e1fa1efca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.675+0000 7f0972000640 1 -- 172.21.15.19:0/3028256140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1fa1c28d0 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.675+0000 7f0973400640 1 --2- 172.21.15.19:0/3028256140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa209c30 0x55e1fa289e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.675+0000 7f0972a00640 1 --2- 172.21.15.19:0/3028256140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa1fa570 0x55e1fa1efca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:06.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.675+0000 7f0972000640 1 --2- 172.21.15.19:0/3028256140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fd8c0 0x55e1fa2898e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f095c055970 tx=0x7f095c08b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8f8f0b4bb4aa03 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.675+0000 7f0957e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:14:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.675+0000 7f0957e00640 1 -- 172.21.15.19:0/3028256140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c077070 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.675+0000 7f0957e00640 1 -- 172.21.15.19:0/3028256140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f095c058960 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.675+0000 7f097c9b0740 1 -- 172.21.15.19:0/3028256140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f096c04b630 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.675+0000 7f0957e00640 1 -- 172.21.15.19:0/3028256140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f095c01f040 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.675+0000 7f097c9b0740 10 client.6273 fetch_fsmap learned FSMap version 36 2024-03-28T13:14:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.675+0000 7f097c9b0740 1 -- 172.21.15.19:0/3028256140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55e1fa1c25c0 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.676+0000 7f0957e00640 1 -- 172.21.15.19:0/3028256140 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096c04b630 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.676+0000 7f0957e00640 1 -- 172.21.15.19:0/3028256140 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f095c071b00 con 0x55e1fa1fd8c0 2024-03-28T13:14:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.676+0000 7f097c9b0740 10 client.6273 fetch_fsmap finished waiting for FSMap version 36 2024-03-28T13:14:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.676+0000 7f097c9b0740 10 client.6273 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-03-28T13:14:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.676+0000 7f097c9b0740 20 client.6273 populate_metadata read hostname 'smithi019' 2024-03-28T13:14:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.676+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3028256140 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55e1fa226b60 0x55e1fa28a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.676+0000 7f097c9b0740 4 client.6273 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:14:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.676+0000 7f097c9b0740 1 -- 172.21.15.19:0/3028256140 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e1fa12ef40 con 0x55e1fa226b60 2024-03-28T13:14:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.676+0000 7f0972a00640 1 --2- 172.21.15.19:0/3028256140 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55e1fa226b60 0x55e1fa28a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.677+0000 7f0972a00640 1 --2- 172.21.15.19:0/3028256140 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55e1fa226b60 0x55e1fa28a720 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3292a0d026ada34c server_cookie=a1e7f107fd657794 in_seq=0 out_seq=0 2024-03-28T13:14:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.677+0000 7f0957e00640 10 client.6273 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:14:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.677+0000 7f0957e00640 1 -- 172.21.15.19:0/3028256140 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e1fa12ef40 con 0x55e1fa226b60 2024-03-28T13:14:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.677+0000 7f0957e00640 10 client.6273 handle_command_reply: tid=0 2024-03-28T13:14:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.677+0000 7f097c9b0740 1 client.6273 shutdown 2024-03-28T13:14:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.677+0000 7f097c9b0740 1 -- 172.21.15.19:0/3028256140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fd8c0 msgr2=0x55e1fa2898e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.677+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3028256140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fd8c0 0x55e1fa2898e0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f095c055970 tx=0x7f095c08b050 comp rx=0 tx=0).stop 2024-03-28T13:14:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.677+0000 7f097c9b0740 1 -- 172.21.15.19:0/3028256140 shutdown_connections 2024-03-28T13:14:06.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.677+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3028256140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1fa209c30 0x55e1fa289e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.678+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3028256140 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x55e1fa226b60 0x55e1fa28a720 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.678+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3028256140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1fa1fd8c0 0x55e1fa2898e0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.678+0000 7f097c9b0740 1 --2- 172.21.15.19:0/3028256140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1fa1fa570 0x55e1fa1efca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.678+0000 7f097c9b0740 1 -- 172.21.15.19:0/3028256140 >> 172.21.15.19:0/3028256140 conn(0x55e1fa1ec4e0 msgr2=0x55e1fa1b7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:06.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.678+0000 7f0957e00640 0 client.6273 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:14:06.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.678+0000 7f097c9b0740 1 -- 172.21.15.19:0/3028256140 shutdown_connections 2024-03-28T13:14:06.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.678+0000 7f097c9b0740 1 -- 172.21.15.19:0/3028256140 wait complete. 2024-03-28T13:14:06.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.678+0000 7f097c9b0740 20 client.6273 trim_cache size 0 max 16384 2024-03-28T13:14:06.681 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:14:06.681 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-03-28T13:14:06.682 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-03-28T13:14:06.682 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:14:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.678+0000 7f0979c00640 1 -- 172.21.15.19:0/2522774911 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f095006c530 msgr2=0x7f095006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.678+0000 7f0979c00640 1 --2- 172.21.15.19:0/2522774911 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f095006c530 0x7f095006e9f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f096c035000 tx=0x7f096c02e000 comp rx=0 tx=0).stop 2024-03-28T13:14:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.678+0000 7f0979c00640 1 -- 172.21.15.19:0/2522774911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f097415f6e0 msgr2=0x7f09741f8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.678+0000 7f0979c00640 1 --2- 172.21.15.19:0/2522774911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f097415f6e0 0x7f09741f8400 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f095c00c7f0 tx=0x7f095c00cbf0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.679+0000 7f0979c00640 1 -- 172.21.15.19:0/2522774911 shutdown_connections 2024-03-28T13:14:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.679+0000 7f0979c00640 1 --2- 172.21.15.19:0/2522774911 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f095006c530 0x7f095006e9f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.679+0000 7f0979c00640 1 --2- 172.21.15.19:0/2522774911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09741621d0 0x7f09740c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.679+0000 7f0979c00640 1 --2- 172.21.15.19:0/2522774911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f097415f6e0 0x7f09741f8400 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.679+0000 7f0979c00640 1 --2- 172.21.15.19:0/2522774911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09740d0e70 0x7f09741f19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.679+0000 7f0979c00640 1 -- 172.21.15.19:0/2522774911 >> 172.21.15.19:0/2522774911 conn(0x7f097408bfb0 msgr2=0x7f0974160e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.679+0000 7f0979c00640 1 -- 172.21.15.19:0/2522774911 shutdown_connections 2024-03-28T13:14:06.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.679+0000 7f0979c00640 1 -- 172.21.15.19:0/2522774911 wait complete. 2024-03-28T13:14:06.696 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T13:14:06.697 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:14:06.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.902+0000 7f2f84a00640 1 Processor -- start 2024-03-28T13:14:06.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.902+0000 7f2f84a00640 1 -- start start 2024-03-28T13:14:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.904+0000 7f2f84a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f8015fc50 0x7f2f800cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.904+0000 7f2f84a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f800b3150 con 0x7f2f800cc490 2024-03-28T13:14:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.904+0000 7f2f84a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f801600e0 con 0x7f2f800cc0b0 2024-03-28T13:14:06.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.904+0000 7f2f84a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2f8015c610 con 0x7f2f8015fc50 2024-03-28T13:14:06.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.904+0000 7f2f75e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f8015fc50 0x7f2f800cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.904+0000 7f2f7f200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2f800cc490 0x7f2f8015ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51066/0 (socket says 172.21.15.19:51066) 2024-03-28T13:14:06.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.904+0000 7f2f75e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2f8015fc50 0x7f2f800cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48456/0 (socket says 172.21.15.19:48456) 2024-03-28T13:14:06.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.904+0000 7f2f7f200640 1 -- 172.21.15.19:0/910744498 learned_addr learned my addr 172.21.15.19:0/910744498 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:06.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.905+0000 7f2f75e00640 1 -- 172.21.15.19:0/910744498 >> v1:172.21.15.188:6789/0 conn(0x7f2f800cc0b0 legacy=0x7f2f8006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:06.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.905+0000 7f2f75e00640 1 -- 172.21.15.19:0/910744498 >> v1:172.21.15.19:6789/0 conn(0x7f2f800cc490 legacy=0x7f2f8015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:06.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.905+0000 7f2f75e00640 1 -- 172.21.15.19:0/910744498 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f801603c0 con 0x7f2f8015fc50 2024-03-28T13:14:06.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.907+0000 7f2f75e00640 1 --2- 172.21.15.19:0/910744498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f8015fc50 0x7f2f800cfdc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2f680097a0 tx=0x7f2f6802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3803ce7c23fbc80 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:06.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.907+0000 7f2f7de00640 1 -- 172.21.15.19:0/910744498 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f68035020 con 0x7f2f8015fc50 2024-03-28T13:14:06.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.907+0000 7f2f7de00640 1 -- 172.21.15.19:0/910744498 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f68031830 con 0x7f2f8015fc50 2024-03-28T13:14:06.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.907+0000 7f2f7de00640 1 -- 172.21.15.19:0/910744498 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f68031c00 con 0x7f2f8015fc50 2024-03-28T13:14:06.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.908+0000 7f2f84a00640 1 -- 172.21.15.19:0/910744498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f8015fc50 msgr2=0x7f2f800cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:06.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.908+0000 7f2f84a00640 1 --2- 172.21.15.19:0/910744498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f8015fc50 0x7f2f800cfdc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2f680097a0 tx=0x7f2f6802b8d0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.908+0000 7f2f84a00640 1 -- 172.21.15.19:0/910744498 shutdown_connections 2024-03-28T13:14:06.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.908+0000 7f2f84a00640 1 --2- 172.21.15.19:0/910744498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f8015fc50 0x7f2f800cfdc0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.908+0000 7f2f84a00640 1 -- 172.21.15.19:0/910744498 >> 172.21.15.19:0/910744498 conn(0x7f2f800ccad0 msgr2=0x7f2f800cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:06.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.908+0000 7f2f84a00640 1 -- 172.21.15.19:0/910744498 shutdown_connections 2024-03-28T13:14:06.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.908+0000 7f2f84a00640 1 -- 172.21.15.19:0/910744498 wait complete. 2024-03-28T13:14:06.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.909+0000 7f2f84a00640 1 Processor -- start 2024-03-28T13:14:06.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.910+0000 7f2f84a00640 1 -- start start 2024-03-28T13:14:06.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f84a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f800cc0b0 0x7f2f801e7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f84a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f800cc490 0x7f2f800cf490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f84a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f8015b7a0 0x7f2f8013c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f84a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f800d0300 con 0x7f2f800cc490 2024-03-28T13:14:06.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f84a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2f80160980 con 0x7f2f8015b7a0 2024-03-28T13:14:06.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f84a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2f800d0480 con 0x7f2f800cc0b0 2024-03-28T13:14:06.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f75e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f800cc490 0x7f2f800cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f75e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f800cc490 0x7f2f800cf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48458/0 (socket says 172.21.15.19:48458) 2024-03-28T13:14:06.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f75e00640 1 -- 172.21.15.19:0/2872115823 learned_addr learned my addr 172.21.15.19:0/2872115823 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:06.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f7f200640 1 --2- 172.21.15.19:0/2872115823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f8015b7a0 0x7f2f8013c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f75e00640 1 -- 172.21.15.19:0/2872115823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f800cc0b0 msgr2=0x7f2f801e7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:06.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f7e800640 1 --2- 172.21.15.19:0/2872115823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f800cc0b0 0x7f2f801e7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f75e00640 1 --2- 172.21.15.19:0/2872115823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f800cc0b0 0x7f2f801e7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f75e00640 1 -- 172.21.15.19:0/2872115823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f8015b7a0 msgr2=0x7f2f8013c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:06.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f75e00640 1 --2- 172.21.15.19:0/2872115823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f8015b7a0 0x7f2f8013c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:06.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.911+0000 7f2f75e00640 1 -- 172.21.15.19:0/2872115823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f800cf9d0 con 0x7f2f800cc490 2024-03-28T13:14:06.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.912+0000 7f2f7f200640 1 --2- 172.21.15.19:0/2872115823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f8015b7a0 0x7f2f8013c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:06.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.912+0000 7f2f75e00640 1 --2- 172.21.15.19:0/2872115823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f800cc490 0x7f2f800cf490 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2f68000c00 tx=0x7f2f680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:06.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.912+0000 7f2f77400640 1 -- 172.21.15.19:0/2872115823 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6803aa70 con 0x7f2f800cc490 2024-03-28T13:14:06.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.912+0000 7f2f77400640 1 -- 172.21.15.19:0/2872115823 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f6803ac10 con 0x7f2f800cc490 2024-03-28T13:14:06.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.912+0000 7f2f77400640 1 -- 172.21.15.19:0/2872115823 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f680443f0 con 0x7f2f800cc490 2024-03-28T13:14:06.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.912+0000 7f2f84a00640 1 -- 172.21.15.19:0/2872115823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f800ce410 con 0x7f2f800cc490 2024-03-28T13:14:06.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.912+0000 7f2f84a00640 1 -- 172.21.15.19:0/2872115823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f800ce9b0 con 0x7f2f800cc490 2024-03-28T13:14:06.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.914+0000 7f2f84a00640 1 -- 172.21.15.19:0/2872115823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f800cf180 con 0x7f2f800cc490 2024-03-28T13:14:06.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.914+0000 7f2f77400640 1 -- 172.21.15.19:0/2872115823 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2f6804b060 con 0x7f2f800cc490 2024-03-28T13:14:06.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.915+0000 7f2f77400640 1 --2- 172.21.15.19:0/2872115823 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2f5c06c530 0x7f2f5c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:06.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.915+0000 7f2f77400640 1 -- 172.21.15.19:0/2872115823 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f2f680ad460 con 0x7f2f800cc490 2024-03-28T13:14:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.916+0000 7f2f7e800640 1 --2- 172.21.15.19:0/2872115823 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2f5c06c530 0x7f2f5c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:06.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.917+0000 7f2f7e800640 1 --2- 172.21.15.19:0/2872115823 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2f5c06c530 0x7f2f5c06e9f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2f70007fd0 tx=0x7f2f70009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:06.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:06.921+0000 7f2f77400640 1 -- 172.21.15.19:0/2872115823 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2f6803f070 con 0x7f2f800cc490 2024-03-28T13:14:07.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.163+0000 7f2f84a00640 1 -- 172.21.15.19:0/2872115823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f80075940 con 0x7f2f800cc490 2024-03-28T13:14:07.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.163+0000 7f2f77400640 1 -- 172.21.15.19:0/2872115823 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5188 (secure 0 0 0) 0x7f2f6807ba30 con 0x7f2f800cc490 2024-03-28T13:14:07.164 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:07.164 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1607201514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1607201514},{"type":"v1","addr":"172.21.15.188:6837","nonce":1607201514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1206740614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1206740614},{"type":"v1","addr":"172.21.15.188:6835","nonce":1206740614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2018671345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2018671345},{"type":"v1","addr":"172.21.15.19:6834","nonce":2018671345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:12:52.947573+0000","modified":"2024-03-28T13:13:37.546087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":14,"addr":"172.21.15.19:6835/3341654519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3341654519},{"type":"v1","addr":"172.21.15.19:6835","nonce":3341654519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":4}]} 2024-03-28T13:14:07.164 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 36 2024-03-28T13:14:07.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.166+0000 7f2f84a00640 1 -- 172.21.15.19:0/2872115823 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2f5c06c530 msgr2=0x7f2f5c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.166+0000 7f2f84a00640 1 --2- 172.21.15.19:0/2872115823 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2f5c06c530 0x7f2f5c06e9f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2f70007fd0 tx=0x7f2f70009040 comp rx=0 tx=0).stop 2024-03-28T13:14:07.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.166+0000 7f2f84a00640 1 -- 172.21.15.19:0/2872115823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f800cc490 msgr2=0x7f2f800cf490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.166+0000 7f2f84a00640 1 --2- 172.21.15.19:0/2872115823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f800cc490 0x7f2f800cf490 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2f68000c00 tx=0x7f2f680046c0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.166+0000 7f2f84a00640 1 -- 172.21.15.19:0/2872115823 shutdown_connections 2024-03-28T13:14:07.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.166+0000 7f2f84a00640 1 --2- 172.21.15.19:0/2872115823 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2f5c06c530 0x7f2f5c06e9f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.166+0000 7f2f84a00640 1 --2- 172.21.15.19:0/2872115823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f8015b7a0 0x7f2f8013c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.166+0000 7f2f84a00640 1 --2- 172.21.15.19:0/2872115823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2f800cc490 0x7f2f800cf490 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.166+0000 7f2f84a00640 1 --2- 172.21.15.19:0/2872115823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f800cc0b0 0x7f2f801e7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.166+0000 7f2f84a00640 1 -- 172.21.15.19:0/2872115823 >> 172.21.15.19:0/2872115823 conn(0x7f2f800ccad0 msgr2=0x7f2f800afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:07.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.166+0000 7f2f84a00640 1 -- 172.21.15.19:0/2872115823 shutdown_connections 2024-03-28T13:14:07.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.166+0000 7f2f84a00640 1 -- 172.21.15.19:0/2872115823 wait complete. 2024-03-28T13:14:07.185 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-03-28T13:14:07.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.397+0000 7f0ac1600640 1 Processor -- start 2024-03-28T13:14:07.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.397+0000 7f0ac1600640 1 -- start start 2024-03-28T13:14:07.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.399+0000 7f0ac1600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0abc15e5d0 0x7f0abc073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.399+0000 7f0ac1600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0abc072e00 0x7f0abc167fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.399+0000 7f0ac1600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0abc0724c0 con 0x7f0abc072030 2024-03-28T13:14:07.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.399+0000 7f0ac1600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f0abc0727a0 con 0x7f0abc15e5d0 2024-03-28T13:14:07.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.399+0000 7f0ac1600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0abc0ae100 con 0x7f0abc072e00 2024-03-28T13:14:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.400+0000 7f0abbe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0abc15e5d0 0x7f0abc073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.400+0000 7f0ac0c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0abc072e00 0x7f0abc167fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.400+0000 7f0ac0c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0abc072e00 0x7f0abc167fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:33552/0 (socket says 172.21.15.19:33552) 2024-03-28T13:14:07.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.400+0000 7f0ac0c00640 1 -- 172.21.15.19:0/2201555099 learned_addr learned my addr 172.21.15.19:0/2201555099 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:07.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.401+0000 7f0ac0c00640 1 -- 172.21.15.19:0/2201555099 >> v2:172.21.15.188:3300/0 conn(0x7f0abc15e5d0 msgr2=0x7f0abc073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:07.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.401+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/2201555099 >> v2:172.21.15.188:3300/0 conn(0x7f0abc15e5d0 0x7f0abc073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.401+0000 7f0ac0c00640 1 -- 172.21.15.19:0/2201555099 >> v1:172.21.15.188:6789/0 conn(0x7f0abc072030 legacy=0x7f0abc0b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.401+0000 7f0ac0c00640 1 -- 172.21.15.19:0/2201555099 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc0ae3e0 con 0x7f0abc072e00 2024-03-28T13:14:07.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.402+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/2201555099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0abc072e00 0x7f0abc167fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009e50 tx=0x7f0ab402bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ca3b02c0a78d657 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:07.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.403+0000 7f0abaa00640 1 -- 172.21.15.19:0/2201555099 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab402ca80 con 0x7f0abc072e00 2024-03-28T13:14:07.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.403+0000 7f0abaa00640 1 -- 172.21.15.19:0/2201555099 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab402cc20 con 0x7f0abc072e00 2024-03-28T13:14:07.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.403+0000 7f0abaa00640 1 -- 172.21.15.19:0/2201555099 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40343f0 con 0x7f0abc072e00 2024-03-28T13:14:07.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.403+0000 7f0ac1600640 1 -- 172.21.15.19:0/2201555099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0abc072e00 msgr2=0x7f0abc167fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.403+0000 7f0ac1600640 1 --2- 172.21.15.19:0/2201555099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0abc072e00 0x7f0abc167fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009e50 tx=0x7f0ab402bee0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.403+0000 7f0ac1600640 1 -- 172.21.15.19:0/2201555099 shutdown_connections 2024-03-28T13:14:07.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.403+0000 7f0ac1600640 1 --2- 172.21.15.19:0/2201555099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0abc072e00 0x7f0abc167fe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.403+0000 7f0ac1600640 1 --2- 172.21.15.19:0/2201555099 >> v2:172.21.15.188:3300/0 conn(0x7f0abc15e5d0 0x7f0abc073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.403+0000 7f0ac1600640 1 -- 172.21.15.19:0/2201555099 >> 172.21.15.19:0/2201555099 conn(0x7f0abc0b1ba0 msgr2=0x7f0abc0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:07.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.404+0000 7f0ac1600640 1 -- 172.21.15.19:0/2201555099 shutdown_connections 2024-03-28T13:14:07.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.404+0000 7f0ac1600640 1 -- 172.21.15.19:0/2201555099 wait complete. 2024-03-28T13:14:07.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.406+0000 7f0ac1600640 1 Processor -- start 2024-03-28T13:14:07.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.406+0000 7f0ac1600640 1 -- start start 2024-03-28T13:14:07.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.407+0000 7f0ac1600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0abc072e00 0x7f0abc14cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.407+0000 7f0ac1600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0abc156390 0x7f0abc1567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.407+0000 7f0ac1600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0abc15f3d0 0x7f0abc15f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.407+0000 7f0ac1600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc06d620 con 0x7f0abc15f3d0 2024-03-28T13:14:07.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.407+0000 7f0ac1600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc0b4140 con 0x7f0abc072e00 2024-03-28T13:14:07.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.407+0000 7f0ac1600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0abc06d7a0 con 0x7f0abc156390 2024-03-28T13:14:07.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.408+0000 7f0ac0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0abc15f3d0 0x7f0abc15f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.408+0000 7f0ac0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0abc15f3d0 0x7f0abc15f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48474/0 (socket says 172.21.15.19:48474) 2024-03-28T13:14:07.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.408+0000 7f0ac0c00640 1 -- 172.21.15.19:0/711974082 learned_addr learned my addr 172.21.15.19:0/711974082 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:07.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.408+0000 7f0abbe00640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0abc072e00 0x7f0abc14cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.408+0000 7f0abb400640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0abc156390 0x7f0abc1567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.408+0000 7f0ac0c00640 1 -- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0abc156390 msgr2=0x7f0abc1567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.408+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0abc156390 0x7f0abc1567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.408+0000 7f0ac0c00640 1 -- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0abc072e00 msgr2=0x7f0abc14cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.408+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0abc072e00 0x7f0abc14cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.408+0000 7f0ac0c00640 1 -- 172.21.15.19:0/711974082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc14d440 con 0x7f0abc15f3d0 2024-03-28T13:14:07.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.408+0000 7f0abbe00640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0abc072e00 0x7f0abc14cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:07.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.408+0000 7f0abb400640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0abc156390 0x7f0abc1567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:07.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.408+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0abc15f3d0 0x7f0abc15f850 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0ab402c4f0 tx=0x7f0ab4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:07.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.409+0000 7f0ab3e00640 1 -- 172.21.15.19:0/711974082 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4043020 con 0x7f0abc15f3d0 2024-03-28T13:14:07.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.409+0000 7f0ab3e00640 1 -- 172.21.15.19:0/711974082 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab403e450 con 0x7f0abc15f3d0 2024-03-28T13:14:07.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.409+0000 7f0ac1600640 1 -- 172.21.15.19:0/711974082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0abc14eeb0 con 0x7f0abc15f3d0 2024-03-28T13:14:07.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.409+0000 7f0ab3e00640 1 -- 172.21.15.19:0/711974082 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4009510 con 0x7f0abc15f3d0 2024-03-28T13:14:07.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.409+0000 7f0ac1600640 1 -- 172.21.15.19:0/711974082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0abc157160 con 0x7f0abc15f3d0 2024-03-28T13:14:07.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.411+0000 7f0ab3e00640 1 -- 172.21.15.19:0/711974082 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0ab4005d10 con 0x7f0abc15f3d0 2024-03-28T13:14:07.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.413+0000 7f0ab3e00640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0a9c06c530 0x7f0a9c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.413+0000 7f0ab3e00640 1 -- 172.21.15.19:0/711974082 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f0ab40ae1d0 con 0x7f0abc15f3d0 2024-03-28T13:14:07.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.413+0000 7f0abbe00640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0a9c06c530 0x7f0a9c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.414+0000 7f0ac5828740 1 Processor -- start 2024-03-28T13:14:07.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.414+0000 7f0ac5828740 1 -- start start 2024-03-28T13:14:07.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.415+0000 7f0abbe00640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0a9c06c530 0x7f0a9c06e9f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009a20 tx=0x7f0aac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:07.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.415+0000 7f0ac5828740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 0x555e87b66c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.415+0000 7f0ac5828740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b368c0 0x555e87b63b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.415+0000 7f0ac5828740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b3a420 0x555e87b64050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.415+0000 7f0ac5828740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555e87aa05a0 con 0x555e87b3a420 2024-03-28T13:14:07.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.415+0000 7f0ac5828740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555e87a48390 con 0x555e87b368c0 2024-03-28T13:14:07.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.415+0000 7f0ac5828740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x555e87a9c550 con 0x555e87b668a0 2024-03-28T13:14:07.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.415+0000 7f0ac0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b3a420 0x555e87b64050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.415+0000 7f0ac0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b3a420 0x555e87b64050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48476/0 (socket says 172.21.15.19:48476) 2024-03-28T13:14:07.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.415+0000 7f0ac0c00640 1 -- 172.21.15.19:0/3631965038 learned_addr learned my addr 172.21.15.19:0/3631965038 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:07.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.416+0000 7f0abbe00640 1 --2- 172.21.15.19:0/3631965038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 0x555e87b66c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.416+0000 7f0abb400640 1 --2- 172.21.15.19:0/3631965038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b368c0 0x555e87b63b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.416+0000 7f0ac0c00640 1 -- 172.21.15.19:0/3631965038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 msgr2=0x555e87b66c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:07.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.416+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/3631965038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 0x555e87b66c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.416+0000 7f0ac0c00640 1 -- 172.21.15.19:0/3631965038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b368c0 msgr2=0x555e87b63b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:07.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.416+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/3631965038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b368c0 0x555e87b63b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.416+0000 7f0ac0c00640 1 -- 172.21.15.19:0/3631965038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e878807c0 con 0x555e87b3a420 2024-03-28T13:14:07.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.416+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/3631965038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b3a420 0x555e87b64050 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4084760 tx=0x7f0ab407bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f02949c88f4118 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:07.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.416+0000 7f0aa7e00640 1 -- 172.21.15.19:0/3631965038 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40aa070 con 0x555e87b3a420 2024-03-28T13:14:07.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.416+0000 7f0aa7e00640 1 -- 172.21.15.19:0/3631965038 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab407c020 con 0x555e87b3a420 2024-03-28T13:14:07.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.416+0000 7f0aa7e00640 1 -- 172.21.15.19:0/3631965038 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40803d0 con 0x555e87b3a420 2024-03-28T13:14:07.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.417+0000 7f0ac5828740 1 -- 172.21.15.19:0/3631965038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b3a420 msgr2=0x555e87b64050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.417+0000 7f0ac5828740 1 --2- 172.21.15.19:0/3631965038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b3a420 0x555e87b64050 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4084760 tx=0x7f0ab407bb60 comp rx=0 tx=0).stop 2024-03-28T13:14:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.417+0000 7f0ac5828740 1 -- 172.21.15.19:0/3631965038 shutdown_connections 2024-03-28T13:14:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.417+0000 7f0ac5828740 1 --2- 172.21.15.19:0/3631965038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b3a420 0x555e87b64050 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.417+0000 7f0ac5828740 1 --2- 172.21.15.19:0/3631965038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b368c0 0x555e87b63b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.417+0000 7f0ac5828740 1 --2- 172.21.15.19:0/3631965038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 0x555e87b66c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.417+0000 7f0ac5828740 1 -- 172.21.15.19:0/3631965038 >> 172.21.15.19:0/3631965038 conn(0x555e87b38b30 msgr2=0x555e87b816b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.417+0000 7f0ac5828740 1 -- 172.21.15.19:0/3631965038 shutdown_connections 2024-03-28T13:14:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.417+0000 7f0ac5828740 1 -- 172.21.15.19:0/3631965038 wait complete. 2024-03-28T13:14:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.418+0000 7f0ac5828740 1 -- start start 2024-03-28T13:14:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.418+0000 7f0ac5828740 1 Processor -- start 2024-03-28T13:14:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.419+0000 7f0ac5828740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b3a420 0x555e87b60e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.419+0000 7f0ac5828740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b668a0 0x555e87b81e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.419+0000 7f0ac5828740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b79ec0 0x555e87b6a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.419+0000 7f0ac5828740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555e87a48390 con 0x555e87b79ec0 2024-03-28T13:14:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.419+0000 7f0ac5828740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555e87a5b780 con 0x555e87b668a0 2024-03-28T13:14:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.419+0000 7f0ac5828740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x555e87aa05a0 con 0x555e87b3a420 2024-03-28T13:14:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.419+0000 7f0ac5828740 10 client.0 _ll_register_callbacks cb 0x555e87a97fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:14:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.420+0000 7f0ac0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b79ec0 0x555e87b6a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.420+0000 7f0ac0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b79ec0 0x555e87b6a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48478/0 (socket says 172.21.15.19:48478) 2024-03-28T13:14:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.420+0000 7f0ac0c00640 1 -- 172.21.15.19:0/1490876619 learned_addr learned my addr 172.21.15.19:0/1490876619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.420+0000 7f0abbe00640 1 --2- 172.21.15.19:0/1490876619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b3a420 0x555e87b60e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.420+0000 7f0abb400640 1 --2- 172.21.15.19:0/1490876619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b668a0 0x555e87b81e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.420+0000 7f0ac0c00640 1 -- 172.21.15.19:0/1490876619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b3a420 msgr2=0x555e87b60e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.420+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/1490876619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b3a420 0x555e87b60e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.420+0000 7f0ac0c00640 1 -- 172.21.15.19:0/1490876619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b668a0 msgr2=0x555e87b81e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.420+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/1490876619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b668a0 0x555e87b81e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.420+0000 7f0ac0c00640 1 -- 172.21.15.19:0/1490876619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e878e0f50 con 0x555e87b79ec0 2024-03-28T13:14:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.420+0000 7f0abbe00640 1 --2- 172.21.15.19:0/1490876619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b3a420 0x555e87b60e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.420+0000 7f0abb400640 1 --2- 172.21.15.19:0/1490876619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b668a0 0x555e87b81e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.420+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/1490876619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b79ec0 0x555e87b6a100 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4084760 tx=0x7f0ab4009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c82940b2f811d22 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.420+0000 7f0aa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:14:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.421+0000 7f0aa4c00640 1 -- 172.21.15.19:0/1490876619 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40aa070 con 0x555e87b79ec0 2024-03-28T13:14:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.421+0000 7f0aa4c00640 1 -- 172.21.15.19:0/1490876619 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f0ab40800d0 con 0x555e87b79ec0 2024-03-28T13:14:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.421+0000 7f0ac5828740 1 -- 172.21.15.19:0/1490876619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab407c020 con 0x555e87b79ec0 2024-03-28T13:14:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.421+0000 7f0aa4c00640 1 -- 172.21.15.19:0/1490876619 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0ab402ca90 con 0x555e87b79ec0 2024-03-28T13:14:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.421+0000 7f0ac5828740 10 client.6291 fetch_fsmap learned FSMap version 36 2024-03-28T13:14:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.421+0000 7f0ac5828740 1 -- 172.21.15.19:0/1490876619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x555e877740b0 con 0x555e87b79ec0 2024-03-28T13:14:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.421+0000 7f0aa4c00640 1 -- 172.21.15.19:0/1490876619 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab407c020 con 0x555e87b79ec0 2024-03-28T13:14:07.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.421+0000 7f0aa4c00640 1 -- 172.21.15.19:0/1490876619 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f0ab4050060 con 0x555e87b79ec0 2024-03-28T13:14:07.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.421+0000 7f0ac5828740 10 client.6291 fetch_fsmap finished waiting for FSMap version 36 2024-03-28T13:14:07.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.421+0000 7f0ac5828740 10 client.6291 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:14:07.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.421+0000 7f0ac5828740 20 client.6291 populate_metadata read hostname 'smithi019' 2024-03-28T13:14:07.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.421+0000 7f0ac5828740 1 --2- 172.21.15.19:0/1490876619 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x555e87b78d40 0x555e87b67d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.422+0000 7f0ac5828740 4 client.6291 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:14:07.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.422+0000 7f0ac5828740 1 -- 172.21.15.19:0/1490876619 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e87a74870 con 0x555e87b78d40 2024-03-28T13:14:07.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.422+0000 7f0abbe00640 1 --2- 172.21.15.19:0/1490876619 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x555e87b78d40 0x555e87b67d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.422+0000 7f0abbe00640 1 --2- 172.21.15.19:0/1490876619 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x555e87b78d40 0x555e87b67d80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b16d01eb1180921d server_cookie=5d79ac2fc849bc5b in_seq=0 out_seq=0 2024-03-28T13:14:07.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.422+0000 7f0aa4c00640 10 client.6291 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:14:07.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.423+0000 7f0aa4c00640 1 -- 172.21.15.19:0/1490876619 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e87a74870 con 0x555e87b78d40 2024-03-28T13:14:07.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.423+0000 7f0aa4c00640 10 client.6291 handle_command_reply: tid=0 2024-03-28T13:14:07.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.423+0000 7f0ac5828740 1 client.6291 shutdown 2024-03-28T13:14:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.423+0000 7f0ac5828740 1 -- 172.21.15.19:0/1490876619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b79ec0 msgr2=0x555e87b6a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.423+0000 7f0ac5828740 1 --2- 172.21.15.19:0/1490876619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b79ec0 0x555e87b6a100 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4084760 tx=0x7f0ab4009810 comp rx=0 tx=0).stop 2024-03-28T13:14:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.424+0000 7f0ac5828740 1 -- 172.21.15.19:0/1490876619 shutdown_connections 2024-03-28T13:14:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.424+0000 7f0ac5828740 1 --2- 172.21.15.19:0/1490876619 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x555e87b78d40 0x555e87b67d80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.424+0000 7f0ac5828740 1 --2- 172.21.15.19:0/1490876619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b79ec0 0x555e87b6a100 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.424+0000 7f0ac5828740 1 --2- 172.21.15.19:0/1490876619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b668a0 0x555e87b81e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.424+0000 7f0ac5828740 1 --2- 172.21.15.19:0/1490876619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b3a420 0x555e87b60e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.424+0000 7f0aa4c00640 0 client.6291 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:14:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.424+0000 7f0ac5828740 1 -- 172.21.15.19:0/1490876619 >> 172.21.15.19:0/1490876619 conn(0x555e87b368c0 msgr2=0x555e87b84bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.424+0000 7f0ac5828740 1 -- 172.21.15.19:0/1490876619 shutdown_connections 2024-03-28T13:14:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.424+0000 7f0ac5828740 1 -- 172.21.15.19:0/1490876619 wait complete. 2024-03-28T13:14:07.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.424+0000 7f0ac5828740 20 client.6291 trim_cache size 0 max 16384 2024-03-28T13:14:07.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.439+0000 7f0ac5828740 1 Processor -- start 2024-03-28T13:14:07.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.439+0000 7f0ac5828740 1 -- start start 2024-03-28T13:14:07.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.439+0000 7f0ac5828740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b3a420 0x555e87b83b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0ac5828740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 0x555e87b5fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0ac5828740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b79ec0 0x555e87b66c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0ac5828740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1ffbb0 con 0x555e87b3a420 2024-03-28T13:14:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0ac5828740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1f8cb0 con 0x555e87b79ec0 2024-03-28T13:14:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0ac5828740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0abc1ff590 con 0x555e87b668a0 2024-03-28T13:14:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0abbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b3a420 0x555e87b83b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0abbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b3a420 0x555e87b83b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48490/0 (socket says 172.21.15.19:48490) 2024-03-28T13:14:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0abbe00640 1 -- 172.21.15.19:0/1591023941 learned_addr learned my addr 172.21.15.19:0/1591023941 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/1591023941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b79ec0 0x555e87b66c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0abb400640 1 --2- 172.21.15.19:0/1591023941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 0x555e87b5fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0abbe00640 1 -- 172.21.15.19:0/1591023941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 msgr2=0x555e87b5fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0abbe00640 1 --2- 172.21.15.19:0/1591023941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 0x555e87b5fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0abbe00640 1 -- 172.21.15.19:0/1591023941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b79ec0 msgr2=0x555e87b66c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0abbe00640 1 --2- 172.21.15.19:0/1591023941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b79ec0 0x555e87b66c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0abbe00640 1 -- 172.21.15.19:0/1591023941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e87a43ae0 con 0x555e87b3a420 2024-03-28T13:14:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/1591023941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b79ec0 0x555e87b66c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.440+0000 7f0abb400640 1 --2- 172.21.15.19:0/1591023941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 0x555e87b5fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:07.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.441+0000 7f0abbe00640 1 --2- 172.21.15.19:0/1591023941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b3a420 0x555e87b83b80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x555e87b78560 tx=0x7f0aac013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c770a4ba4af436a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.441+0000 7f0aa7400640 1 -- 172.21.15.19:0/1591023941 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac00f040 con 0x555e87b3a420 2024-03-28T13:14:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.441+0000 7f0aa7400640 1 -- 172.21.15.19:0/1591023941 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aac010d00 con 0x555e87b3a420 2024-03-28T13:14:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.441+0000 7f0aa7400640 1 -- 172.21.15.19:0/1591023941 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac003d00 con 0x555e87b3a420 2024-03-28T13:14:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.442+0000 7f0ac5828740 1 -- 172.21.15.19:0/1591023941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b3a420 msgr2=0x555e87b83b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.442+0000 7f0ac5828740 1 --2- 172.21.15.19:0/1591023941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b3a420 0x555e87b83b80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x555e87b78560 tx=0x7f0aac013000 comp rx=0 tx=0).stop 2024-03-28T13:14:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.442+0000 7f0ac5828740 1 -- 172.21.15.19:0/1591023941 shutdown_connections 2024-03-28T13:14:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.442+0000 7f0ac5828740 1 --2- 172.21.15.19:0/1591023941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b79ec0 0x555e87b66c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.442+0000 7f0ac5828740 1 --2- 172.21.15.19:0/1591023941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 0x555e87b5fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.442+0000 7f0ac5828740 1 --2- 172.21.15.19:0/1591023941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b3a420 0x555e87b83b80 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.442+0000 7f0ac5828740 1 -- 172.21.15.19:0/1591023941 >> 172.21.15.19:0/1591023941 conn(0x555e87b368c0 msgr2=0x555e87b84bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.442+0000 7f0ac5828740 1 -- 172.21.15.19:0/1591023941 shutdown_connections 2024-03-28T13:14:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.442+0000 7f0ac5828740 1 -- 172.21.15.19:0/1591023941 wait complete. 2024-03-28T13:14:07.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.442+0000 7f0ac5828740 1 -- start start 2024-03-28T13:14:07.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.443+0000 7f0ac5828740 1 Processor -- start 2024-03-28T13:14:07.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0ac5828740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b3a420 0x555e87b602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0ac5828740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 0x555e87b61c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0ac5828740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b79ec0 0x555e87bfa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0ac5828740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555e87a48390 con 0x555e87b79ec0 2024-03-28T13:14:07.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0ac5828740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555e87aa05a0 con 0x555e87b3a420 2024-03-28T13:14:07.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0ac5828740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x555e87a5b780 con 0x555e87b668a0 2024-03-28T13:14:07.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0ac5828740 10 client.0 _ll_register_callbacks cb 0x7f0ab40a8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:14:07.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0ac0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b79ec0 0x555e87bfa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0ac0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b79ec0 0x555e87bfa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48504/0 (socket says 172.21.15.19:48504) 2024-03-28T13:14:07.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0ac0c00640 1 -- 172.21.15.19:0/7007923 learned_addr learned my addr 172.21.15.19:0/7007923 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0abbe00640 1 --2- 172.21.15.19:0/7007923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b3a420 0x555e87b602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0abb400640 1 --2- 172.21.15.19:0/7007923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 0x555e87b61c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0ac0c00640 1 -- 172.21.15.19:0/7007923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 msgr2=0x555e87b61c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/7007923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 0x555e87b61c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0ac0c00640 1 -- 172.21.15.19:0/7007923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b3a420 msgr2=0x555e87b602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/7007923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b3a420 0x555e87b602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0ac0c00640 1 -- 172.21.15.19:0/7007923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e87b62fa0 con 0x555e87b79ec0 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.444+0000 7f0abbe00640 1 --2- 172.21.15.19:0/7007923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b3a420 0x555e87b602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.445+0000 7f0ac0c00640 1 --2- 172.21.15.19:0/7007923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b79ec0 0x555e87bfa1b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0ab407bfb0 tx=0x7f0ab4080000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1cd3f32ccbefdc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.445+0000 7f0aa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.445+0000 7f0aa4c00640 1 -- 172.21.15.19:0/7007923 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab40aa040 con 0x555e87b79ec0 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.445+0000 7f0aa4c00640 1 -- 172.21.15.19:0/7007923 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f0ab40ac310 con 0x555e87b79ec0 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.445+0000 7f0ac5828740 1 -- 172.21.15.19:0/7007923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0aac010d00 con 0x555e87b79ec0 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.445+0000 7f0aa4c00640 1 -- 172.21.15.19:0/7007923 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0ab407b980 con 0x555e87b79ec0 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.445+0000 7f0ac5828740 10 client.6297 fetch_fsmap learned FSMap version 36 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.445+0000 7f0ac5828740 1 -- 172.21.15.19:0/7007923 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x555e87b62fa0 con 0x555e87b79ec0 2024-03-28T13:14:07.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.446+0000 7f0aa4c00640 1 -- 172.21.15.19:0/7007923 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aac010d00 con 0x555e87b79ec0 2024-03-28T13:14:07.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.446+0000 7f0aa4c00640 1 -- 172.21.15.19:0/7007923 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 3494+0+0 (secure 0 0 0) 0x7f0ab407c280 con 0x555e87b79ec0 2024-03-28T13:14:07.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.446+0000 7f0ac5828740 10 client.6297 fetch_fsmap finished waiting for FSMap version 36 2024-03-28T13:14:07.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.446+0000 7f0ac5828740 10 client.6297 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:14:07.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.446+0000 7f0ac5828740 20 client.6297 populate_metadata read hostname 'smithi019' 2024-03-28T13:14:07.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.446+0000 7f0ac5828740 1 --2- 172.21.15.19:0/7007923 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x555e87b68590 0x555e87b5a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.446+0000 7f0ac5828740 4 client.6297 mds_command: new command op to 5970 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-28T13:14:07.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.446+0000 7f0ac5828740 1 -- 172.21.15.19:0/7007923 --> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x555e87aadae0 con 0x555e87b68590 2024-03-28T13:14:07.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.446+0000 7f0abbe00640 1 --2- 172.21.15.19:0/7007923 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x555e87b68590 0x555e87b5a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0abbe00640 1 --2- 172.21.15.19:0/7007923 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x555e87b68590 0x555e87b5a1c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f741ae2c153b999 server_cookie=da33f1af27b41b8b in_seq=0 out_seq=0 2024-03-28T13:14:07.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0aa4c00640 10 client.6297 ms_handle_connect on v2:172.21.15.19:6833/3341654519 2024-03-28T13:14:07.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0aa4c00640 1 -- 172.21.15.19:0/7007923 <== mds.0 v2:172.21.15.19:6833/3341654519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e87aadae0 con 0x555e87b68590 2024-03-28T13:14:07.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0aa4c00640 10 client.6297 handle_command_reply: tid=0 2024-03-28T13:14:07.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0ac5828740 1 client.6297 shutdown 2024-03-28T13:14:07.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0ac5828740 1 -- 172.21.15.19:0/7007923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b79ec0 msgr2=0x555e87bfa1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0ac5828740 1 --2- 172.21.15.19:0/7007923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b79ec0 0x555e87bfa1b0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0ab407bfb0 tx=0x7f0ab4080000 comp rx=0 tx=0).stop 2024-03-28T13:14:07.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0ac5828740 1 -- 172.21.15.19:0/7007923 shutdown_connections 2024-03-28T13:14:07.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0ac5828740 1 --2- 172.21.15.19:0/7007923 >> [v2:172.21.15.19:6833/3341654519,v1:172.21.15.19:6835/3341654519] conn(0x555e87b68590 0x555e87b5a1c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0ac5828740 1 --2- 172.21.15.19:0/7007923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555e87b79ec0 0x555e87bfa1b0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0ac5828740 1 --2- 172.21.15.19:0/7007923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555e87b668a0 0x555e87b61c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0ac5828740 1 --2- 172.21.15.19:0/7007923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555e87b3a420 0x555e87b602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0ac5828740 1 -- 172.21.15.19:0/7007923 >> 172.21.15.19:0/7007923 conn(0x555e87b368c0 msgr2=0x555e87b3d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:07.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0aa4c00640 0 client.6297 ms_handle_reset on v2:172.21.15.19:6833/3341654519 2024-03-28T13:14:07.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0ac5828740 1 -- 172.21.15.19:0/7007923 shutdown_connections 2024-03-28T13:14:07.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.447+0000 7f0ac5828740 1 -- 172.21.15.19:0/7007923 wait complete. 2024-03-28T13:14:07.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.448+0000 7f0ac5828740 20 client.6297 trim_cache size 0 max 16384 2024-03-28T13:14:07.450 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:07.451 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-03-28T13:14:07.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.448+0000 7f0ac1600640 1 -- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0a9c06c530 msgr2=0x7f0a9c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.448+0000 7f0ac1600640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0a9c06c530 0x7f0a9c06e9f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009a20 tx=0x7f0aac008040 comp rx=0 tx=0).stop 2024-03-28T13:14:07.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.448+0000 7f0ac1600640 1 -- 172.21.15.19:0/711974082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0abc15f3d0 msgr2=0x7f0abc15f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.448+0000 7f0ac1600640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0abc15f3d0 0x7f0abc15f850 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0ab402c4f0 tx=0x7f0ab4004a40 comp rx=0 tx=0).stop 2024-03-28T13:14:07.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.449+0000 7f0ac1600640 1 -- 172.21.15.19:0/711974082 shutdown_connections 2024-03-28T13:14:07.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.449+0000 7f0ac1600640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0a9c06c530 0x7f0a9c06e9f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.449+0000 7f0ac1600640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0abc15f3d0 0x7f0abc15f850 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.449+0000 7f0ac1600640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0abc156390 0x7f0abc1567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.449+0000 7f0ac1600640 1 --2- 172.21.15.19:0/711974082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0abc072e00 0x7f0abc14cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.449+0000 7f0ac1600640 1 -- 172.21.15.19:0/711974082 >> 172.21.15.19:0/711974082 conn(0x7f0abc0b1ba0 msgr2=0x7f0abc15d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:07.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.449+0000 7f0ac1600640 1 -- 172.21.15.19:0/711974082 shutdown_connections 2024-03-28T13:14:07.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.449+0000 7f0ac1600640 1 -- 172.21.15.19:0/711974082 wait complete. 2024-03-28T13:14:07.467 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-28T13:14:07.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.664+0000 7f4560000640 1 Processor -- start 2024-03-28T13:14:07.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.665+0000 7f4560000640 1 -- start start 2024-03-28T13:14:07.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.666+0000 7f4560000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f45580d0990 0x7f45581670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.666+0000 7f4560000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4558163ee0 con 0x7f4558162920 2024-03-28T13:14:07.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.666+0000 7f4560000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4558071d60 con 0x7f4558163ac0 2024-03-28T13:14:07.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.666+0000 7f4560000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f45581665d0 con 0x7f45580d0990 2024-03-28T13:14:07.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.666+0000 7f455d600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4558162920 0x7f45580d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51072/0 (socket says 172.21.15.19:51072) 2024-03-28T13:14:07.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.666+0000 7f455d600640 1 -- 172.21.15.19:0/415506529 learned_addr learned my addr 172.21.15.19:0/415506529 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:07.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.667+0000 7f455cc00640 1 --2- 172.21.15.19:0/415506529 >> v2:172.21.15.188:3300/0 conn(0x7f45580d0990 0x7f45581670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.667+0000 7f4557e00640 1 -- 172.21.15.19:0/415506529 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3270082111 0 0) 0x7f4558163ee0 con 0x7f4558162920 2024-03-28T13:14:07.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.667+0000 7f4557e00640 1 -- 172.21.15.19:0/415506529 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4540003520 con 0x7f4558162920 2024-03-28T13:14:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.668+0000 7f4557e00640 1 -- 172.21.15.19:0/415506529 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 15484347 0 0) 0x7f4540003520 con 0x7f4558162920 2024-03-28T13:14:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.668+0000 7f455cc00640 1 -- 172.21.15.19:0/415506529 >> v1:172.21.15.188:6789/0 conn(0x7f4558163ac0 legacy=0x7f45580ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.668+0000 7f455cc00640 1 -- 172.21.15.19:0/415506529 >> v1:172.21.15.19:6789/0 conn(0x7f4558162920 legacy=0x7f45580d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.668+0000 7f455cc00640 1 -- 172.21.15.19:0/415506529 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45581668b0 con 0x7f45580d0990 2024-03-28T13:14:07.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.669+0000 7f455cc00640 1 --2- 172.21.15.19:0/415506529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45580d0990 0x7f45581670b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f45440097a0 tx=0x7f454402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c33e93ba75b27c2c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:07.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.670+0000 7f4557e00640 1 -- 172.21.15.19:0/415506529 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544037070 con 0x7f45580d0990 2024-03-28T13:14:07.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.670+0000 7f4557e00640 1 -- 172.21.15.19:0/415506529 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4544030510 con 0x7f45580d0990 2024-03-28T13:14:07.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.670+0000 7f4557e00640 1 -- 172.21.15.19:0/415506529 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c001690 con 0x7f45580d0990 2024-03-28T13:14:07.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.670+0000 7f4560000640 1 -- 172.21.15.19:0/415506529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45580d0990 msgr2=0x7f45581670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.670+0000 7f4560000640 1 --2- 172.21.15.19:0/415506529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45580d0990 0x7f45581670b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f45440097a0 tx=0x7f454402b790 comp rx=0 tx=0).stop 2024-03-28T13:14:07.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.671+0000 7f4560000640 1 -- 172.21.15.19:0/415506529 shutdown_connections 2024-03-28T13:14:07.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.671+0000 7f4560000640 1 --2- 172.21.15.19:0/415506529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45580d0990 0x7f45581670b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.671+0000 7f4560000640 1 -- 172.21.15.19:0/415506529 >> 172.21.15.19:0/415506529 conn(0x7f455806b750 msgr2=0x7f455806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:07.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.671+0000 7f4560000640 1 -- 172.21.15.19:0/415506529 shutdown_connections 2024-03-28T13:14:07.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.671+0000 7f4560000640 1 -- 172.21.15.19:0/415506529 wait complete. 2024-03-28T13:14:07.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.674+0000 7f4560000640 1 Processor -- start 2024-03-28T13:14:07.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.674+0000 7f4560000640 1 -- start start 2024-03-28T13:14:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.675+0000 7f4560000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45580d0990 0x7f455814c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.675+0000 7f4560000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4558162920 0x7f45580ce750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.675+0000 7f4560000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4558163ac0 0x7f45580ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.675+0000 7f4560000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4558072040 con 0x7f4558162920 2024-03-28T13:14:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.675+0000 7f4560000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4558162d00 con 0x7f4558163ac0 2024-03-28T13:14:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.675+0000 7f4560000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f45580721c0 con 0x7f45580d0990 2024-03-28T13:14:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.675+0000 7f455cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4558162920 0x7f45580ce750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.675+0000 7f455cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4558162920 0x7f45580ce750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48516/0 (socket says 172.21.15.19:48516) 2024-03-28T13:14:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.676+0000 7f455cc00640 1 -- 172.21.15.19:0/2021122079 learned_addr learned my addr 172.21.15.19:0/2021122079 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.676+0000 7f455d600640 1 --2- 172.21.15.19:0/2021122079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45580d0990 0x7f455814c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.676+0000 7f455e000640 1 --2- 172.21.15.19:0/2021122079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4558163ac0 0x7f45580ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.676+0000 7f455cc00640 1 -- 172.21.15.19:0/2021122079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45580d0990 msgr2=0x7f455814c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.676+0000 7f455cc00640 1 --2- 172.21.15.19:0/2021122079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45580d0990 0x7f455814c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.676+0000 7f455cc00640 1 -- 172.21.15.19:0/2021122079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4558163ac0 msgr2=0x7f45580ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.676+0000 7f455cc00640 1 --2- 172.21.15.19:0/2021122079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4558163ac0 0x7f45580ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.676+0000 7f455cc00640 1 -- 172.21.15.19:0/2021122079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45580cf210 con 0x7f4558162920 2024-03-28T13:14:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.676+0000 7f455d600640 1 --2- 172.21.15.19:0/2021122079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45580d0990 0x7f455814c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:14:07.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.677+0000 7f455cc00640 1 --2- 172.21.15.19:0/2021122079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4558162920 0x7f45580ce750 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f45440098c0 tx=0x7f45440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:07.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.677+0000 7f4555600640 1 -- 172.21.15.19:0/2021122079 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544042070 con 0x7f4558162920 2024-03-28T13:14:07.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.677+0000 7f4555600640 1 -- 172.21.15.19:0/2021122079 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f454403e870 con 0x7f4558162920 2024-03-28T13:14:07.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.677+0000 7f4555600640 1 -- 172.21.15.19:0/2021122079 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544037040 con 0x7f4558162920 2024-03-28T13:14:07.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.677+0000 7f4560000640 1 -- 172.21.15.19:0/2021122079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4558164330 con 0x7f4558162920 2024-03-28T13:14:07.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.677+0000 7f4560000640 1 -- 172.21.15.19:0/2021122079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45581648f0 con 0x7f4558162920 2024-03-28T13:14:07.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.678+0000 7f4560000640 1 -- 172.21.15.19:0/2021122079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4558164140 con 0x7f4558162920 2024-03-28T13:14:07.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.679+0000 7f4555600640 1 -- 172.21.15.19:0/2021122079 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4544045070 con 0x7f4558162920 2024-03-28T13:14:07.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.680+0000 7f4555600640 1 --2- 172.21.15.19:0/2021122079 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f453806c530 0x7f453806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:07.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.680+0000 7f4555600640 1 -- 172.21.15.19:0/2021122079 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f45440ad390 con 0x7f4558162920 2024-03-28T13:14:07.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.681+0000 7f455d600640 1 --2- 172.21.15.19:0/2021122079 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f453806c530 0x7f453806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:07.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.682+0000 7f455d600640 1 --2- 172.21.15.19:0/2021122079 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f453806c530 0x7f453806e9f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f454c001800 tx=0x7f454c0016f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:07.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.685+0000 7f4555600640 1 -- 172.21.15.19:0/2021122079 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f454407b8d0 con 0x7f4558162920 2024-03-28T13:14:07.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.928+0000 7f4560000640 1 -- 172.21.15.19:0/2021122079 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f45580685c0 con 0x7f4558162920 2024-03-28T13:14:07.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.929+0000 7f4555600640 1 -- 172.21.15.19:0/2021122079 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f454407b1d0 con 0x7f4558162920 2024-03-28T13:14:07.929 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:07.929 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-28T13:14:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.932+0000 7f4560000640 1 -- 172.21.15.19:0/2021122079 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f453806c530 msgr2=0x7f453806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.932+0000 7f4560000640 1 --2- 172.21.15.19:0/2021122079 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f453806c530 0x7f453806e9f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f454c001800 tx=0x7f454c0016f0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.932+0000 7f4560000640 1 -- 172.21.15.19:0/2021122079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4558162920 msgr2=0x7f45580ce750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.932+0000 7f4560000640 1 --2- 172.21.15.19:0/2021122079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4558162920 0x7f45580ce750 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f45440098c0 tx=0x7f45440046c0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.932+0000 7f4560000640 1 -- 172.21.15.19:0/2021122079 shutdown_connections 2024-03-28T13:14:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.932+0000 7f4560000640 1 --2- 172.21.15.19:0/2021122079 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f453806c530 0x7f453806e9f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.932+0000 7f4560000640 1 --2- 172.21.15.19:0/2021122079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4558163ac0 0x7f45580ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.932+0000 7f4560000640 1 --2- 172.21.15.19:0/2021122079 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4558162920 0x7f45580ce750 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.932+0000 7f4560000640 1 --2- 172.21.15.19:0/2021122079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45580d0990 0x7f455814c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:07.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.932+0000 7f4560000640 1 -- 172.21.15.19:0/2021122079 >> 172.21.15.19:0/2021122079 conn(0x7f455806b750 msgr2=0x7f455806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:07.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.932+0000 7f4560000640 1 -- 172.21.15.19:0/2021122079 shutdown_connections 2024-03-28T13:14:07.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:07.932+0000 7f4560000640 1 -- 172.21.15.19:0/2021122079 wait complete. 2024-03-28T13:14:07.952 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-28T13:14:07.952 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-28T13:14:07.953 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:14:08.065 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:14:08.185 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-28T13:14:08.185 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:14:08.186 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:14:08.186 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:14:08.241 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:14:08.241 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:14:08.292 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:14:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.560+0000 7f9023c00640 1 Processor -- start 2024-03-28T13:14:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.561+0000 7f9023c00640 1 -- start start 2024-03-28T13:14:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.562+0000 7f9023c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f901c0d2bf0 0x7f901c0cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.562+0000 7f9023c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f901c0d0c30 0x7f901c0d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.562+0000 7f9023c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f901c0d3140 con 0x7f901c15fa10 2024-03-28T13:14:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.562+0000 7f9023c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f901c0d15e0 con 0x7f901c0d2bf0 2024-03-28T13:14:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.562+0000 7f9023c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f901c0d3420 con 0x7f901c0d0c30 2024-03-28T13:14:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.562+0000 7f901be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f901c0d2bf0 0x7f901c0cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.563+0000 7f901be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f901c0d2bf0 0x7f901c0cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48520/0 (socket says 172.21.15.19:48520) 2024-03-28T13:14:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.563+0000 7f901be00640 1 -- 172.21.15.19:0/379356584 learned_addr learned my addr 172.21.15.19:0/379356584 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.563+0000 7f9021c00640 1 --2- 172.21.15.19:0/379356584 >> v2:172.21.15.188:3300/0 conn(0x7f901c0d0c30 0x7f901c0d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.563+0000 7f901be00640 1 -- 172.21.15.19:0/379356584 >> v2:172.21.15.188:3300/0 conn(0x7f901c0d0c30 msgr2=0x7f901c0d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.563+0000 7f901be00640 1 --2- 172.21.15.19:0/379356584 >> v2:172.21.15.188:3300/0 conn(0x7f901c0d0c30 0x7f901c0d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.563+0000 7f901be00640 1 -- 172.21.15.19:0/379356584 >> v1:172.21.15.188:6789/0 conn(0x7f901c15fa10 legacy=0x7f901c0b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.563+0000 7f901be00640 1 -- 172.21.15.19:0/379356584 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f901c0cf2b0 con 0x7f901c0d2bf0 2024-03-28T13:14:08.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.564+0000 7f901be00640 1 --2- 172.21.15.19:0/379356584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f901c0d2bf0 0x7f901c0cfd90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f901c0cb4d0 tx=0x7f900802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b5e19a6d07ad52 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:08.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.564+0000 7f901b400640 1 -- 172.21.15.19:0/379356584 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008004230 con 0x7f901c0d2bf0 2024-03-28T13:14:08.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.564+0000 7f901b400640 1 -- 172.21.15.19:0/379356584 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90080043d0 con 0x7f901c0d2bf0 2024-03-28T13:14:08.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.565+0000 7f901b400640 1 -- 172.21.15.19:0/379356584 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008030840 con 0x7f901c0d2bf0 2024-03-28T13:14:08.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.565+0000 7f9023c00640 1 -- 172.21.15.19:0/379356584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f901c0d2bf0 msgr2=0x7f901c0cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:08.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.565+0000 7f9023c00640 1 --2- 172.21.15.19:0/379356584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f901c0d2bf0 0x7f901c0cfd90 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f901c0cb4d0 tx=0x7f900802b980 comp rx=0 tx=0).stop 2024-03-28T13:14:08.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.565+0000 7f9023c00640 1 -- 172.21.15.19:0/379356584 shutdown_connections 2024-03-28T13:14:08.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.565+0000 7f9023c00640 1 --2- 172.21.15.19:0/379356584 >> v2:172.21.15.188:3300/0 conn(0x7f901c0d0c30 0x7f901c0d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:08.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.565+0000 7f9023c00640 1 --2- 172.21.15.19:0/379356584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f901c0d2bf0 0x7f901c0cfd90 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:08.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.565+0000 7f9023c00640 1 -- 172.21.15.19:0/379356584 >> 172.21.15.19:0/379356584 conn(0x7f901c08bfb0 msgr2=0x7f901c0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:08.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.565+0000 7f9023c00640 1 -- 172.21.15.19:0/379356584 shutdown_connections 2024-03-28T13:14:08.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.566+0000 7f9023c00640 1 -- 172.21.15.19:0/379356584 wait complete. 2024-03-28T13:14:08.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.567+0000 7f9023c00640 1 Processor -- start 2024-03-28T13:14:08.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.567+0000 7f9023c00640 1 -- start start 2024-03-28T13:14:08.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.568+0000 7f9023c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f901c0d0c30 0x7f901c158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:08.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.568+0000 7f9023c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f901c0d2bf0 0x7f901c1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:08.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.568+0000 7f9023c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f901c15fa10 0x7f901c1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:08.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.568+0000 7f9023c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f901c0d02d0 con 0x7f901c15fa10 2024-03-28T13:14:08.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.568+0000 7f9023c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f901c0cf870 con 0x7f901c0d2bf0 2024-03-28T13:14:08.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.568+0000 7f9023c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f901c0d0450 con 0x7f901c0d0c30 2024-03-28T13:14:08.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.569+0000 7f9021c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f901c15fa10 0x7f901c1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:08.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.569+0000 7f9021c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f901c15fa10 0x7f901c1597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48536/0 (socket says 172.21.15.19:48536) 2024-03-28T13:14:08.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.569+0000 7f9021c00640 1 -- 172.21.15.19:0/171109976 learned_addr learned my addr 172.21.15.19:0/171109976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:08.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.569+0000 7f901be00640 1 --2- 172.21.15.19:0/171109976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f901c0d2bf0 0x7f901c1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:08.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.569+0000 7f9021200640 1 --2- 172.21.15.19:0/171109976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f901c0d0c30 0x7f901c158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:08.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.569+0000 7f9021c00640 1 -- 172.21.15.19:0/171109976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f901c0d0c30 msgr2=0x7f901c158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:08.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.570+0000 7f9021c00640 1 --2- 172.21.15.19:0/171109976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f901c0d0c30 0x7f901c158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.570+0000 7f9021c00640 1 -- 172.21.15.19:0/171109976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f901c0d2bf0 msgr2=0x7f901c1592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.570+0000 7f9021c00640 1 --2- 172.21.15.19:0/171109976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f901c0d2bf0 0x7f901c1592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.570+0000 7f9021c00640 1 -- 172.21.15.19:0/171109976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f901c15d080 con 0x7f901c15fa10 2024-03-28T13:14:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.570+0000 7f9021c00640 1 --2- 172.21.15.19:0/171109976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f901c15fa10 0x7f901c1597f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f901400cda0 tx=0x7f901400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.570+0000 7f9018c00640 1 -- 172.21.15.19:0/171109976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014003b90 con 0x7f901c15fa10 2024-03-28T13:14:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.570+0000 7f9018c00640 1 -- 172.21.15.19:0/171109976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9014003d30 con 0x7f901c15fa10 2024-03-28T13:14:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.570+0000 7f9023c00640 1 -- 172.21.15.19:0/171109976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f901c15d360 con 0x7f901c15fa10 2024-03-28T13:14:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.570+0000 7f9018c00640 1 -- 172.21.15.19:0/171109976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9014004240 con 0x7f901c15fa10 2024-03-28T13:14:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.570+0000 7f9023c00640 1 -- 172.21.15.19:0/171109976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f901c158570 con 0x7f901c15fa10 2024-03-28T13:14:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.572+0000 7f9023c00640 1 -- 172.21.15.19:0/171109976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f901c0d3da0 con 0x7f901c15fa10 2024-03-28T13:14:08.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.573+0000 7f9018c00640 1 -- 172.21.15.19:0/171109976 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f90140043e0 con 0x7f901c15fa10 2024-03-28T13:14:08.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.573+0000 7f9018c00640 1 --2- 172.21.15.19:0/171109976 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8ffc06c530 0x7f8ffc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:08.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.574+0000 7f9018c00640 1 -- 172.21.15.19:0/171109976 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f901408c330 con 0x7f901c15fa10 2024-03-28T13:14:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.574+0000 7f9021200640 1 --2- 172.21.15.19:0/171109976 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8ffc06c530 0x7f8ffc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:08.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.575+0000 7f9021200640 1 --2- 172.21.15.19:0/171109976 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8ffc06c530 0x7f8ffc06e9f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9010006fd0 tx=0x7f9010008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:08.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.580+0000 7f9018c00640 1 -- 172.21.15.19:0/171109976 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f901c0d3da0 con 0x7f901c15fa10 2024-03-28T13:14:08.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.782+0000 7f9023c00640 1 -- 172.21.15.19:0/171109976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f901c068cd0 con 0x7f901c15fa10 2024-03-28T13:14:08.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.784+0000 7f9018c00640 1 -- 172.21.15.19:0/171109976 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16761 (secure 0 0 0) 0x7f901401d340 con 0x7f901c15fa10 2024-03-28T13:14:08.784 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:08.784 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:13:24.671229+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:12:50.888331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-28T13:12:51.946012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.14","mappings":[{"from":6,"to":4}]},{"pgid":"9.2c","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:14:08.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.786+0000 7f9023c00640 1 -- 172.21.15.19:0/171109976 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8ffc06c530 msgr2=0x7f8ffc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.786+0000 7f9023c00640 1 --2- 172.21.15.19:0/171109976 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8ffc06c530 0x7f8ffc06e9f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9010006fd0 tx=0x7f9010008040 comp rx=0 tx=0).stop 2024-03-28T13:14:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.786+0000 7f9023c00640 1 -- 172.21.15.19:0/171109976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f901c15fa10 msgr2=0x7f901c1597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.786+0000 7f9023c00640 1 --2- 172.21.15.19:0/171109976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f901c15fa10 0x7f901c1597f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f901400cda0 tx=0x7f901400a660 comp rx=0 tx=0).stop 2024-03-28T13:14:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.786+0000 7f9023c00640 1 -- 172.21.15.19:0/171109976 shutdown_connections 2024-03-28T13:14:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.786+0000 7f9023c00640 1 --2- 172.21.15.19:0/171109976 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8ffc06c530 0x7f8ffc06e9f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.786+0000 7f9023c00640 1 --2- 172.21.15.19:0/171109976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f901c15fa10 0x7f901c1597f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.786+0000 7f9023c00640 1 --2- 172.21.15.19:0/171109976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f901c0d2bf0 0x7f901c1592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.786+0000 7f9023c00640 1 --2- 172.21.15.19:0/171109976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f901c0d0c30 0x7f901c158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.787+0000 7f9023c00640 1 -- 172.21.15.19:0/171109976 >> 172.21.15.19:0/171109976 conn(0x7f901c08bfb0 msgr2=0x7f901c157730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.787+0000 7f9023c00640 1 -- 172.21.15.19:0/171109976 shutdown_connections 2024-03-28T13:14:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:08.787+0000 7f9023c00640 1 -- 172.21.15.19:0/171109976 wait complete. 2024-03-28T13:14:08.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:12:50.888331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:12:51.946012+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:14:08.806 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:14:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.023+0000 7f49efa00640 1 Processor -- start 2024-03-28T13:14:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.023+0000 7f49efa00640 1 -- start start 2024-03-28T13:14:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.025+0000 7f49efa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49e8153580 0x7f49e80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.025+0000 7f49efa00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f49e80d07c0 0x7f49e80aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.025+0000 7f49efa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e80af2e0 0x7f49e806d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.025+0000 7f49efa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f49e80d0c30 con 0x7f49e8153580 2024-03-28T13:14:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.025+0000 7f49efa00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f49e80afab0 con 0x7f49e80d07c0 2024-03-28T13:14:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.025+0000 7f49efa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f49e815cc80 con 0x7f49e80af2e0 2024-03-28T13:14:09.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.025+0000 7f49ed000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49e8153580 0x7f49e80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.025+0000 7f49ed000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49e8153580 0x7f49e80d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48540/0 (socket says 172.21.15.19:48540) 2024-03-28T13:14:09.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.025+0000 7f49ed000640 1 -- 172.21.15.19:0/742466712 learned_addr learned my addr 172.21.15.19:0/742466712 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:09.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.025+0000 7f49e7e00640 1 --2- 172.21.15.19:0/742466712 >> v2:172.21.15.188:3300/0 conn(0x7f49e80d07c0 0x7f49e80aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.026+0000 7f49ed000640 1 -- 172.21.15.19:0/742466712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e80af2e0 msgr2=0x7f49e806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:09.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.026+0000 7f49ed000640 1 --2- 172.21.15.19:0/742466712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e80af2e0 0x7f49e806d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.026+0000 7f49ed000640 1 -- 172.21.15.19:0/742466712 >> v2:172.21.15.188:3300/0 conn(0x7f49e80d07c0 msgr2=0x7f49e80aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:09.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.026+0000 7f49ed000640 1 --2- 172.21.15.19:0/742466712 >> v2:172.21.15.188:3300/0 conn(0x7f49e80d07c0 0x7f49e80aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.026+0000 7f49ed000640 1 -- 172.21.15.19:0/742466712 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e815da00 con 0x7f49e8153580 2024-03-28T13:14:09.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.026+0000 7f49e7e00640 1 --2- 172.21.15.19:0/742466712 >> v2:172.21.15.188:3300/0 conn(0x7f49e80d07c0 0x7f49e80aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:09.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.027+0000 7f49ed000640 1 --2- 172.21.15.19:0/742466712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e8153580 0x7f49e80d0280 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f49e80cff70 tx=0x7f49d802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc697b01b0bd66 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:09.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.027+0000 7f49e7400640 1 -- 172.21.15.19:0/742466712 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d80317d0 con 0x7f49e8153580 2024-03-28T13:14:09.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.027+0000 7f49e7400640 1 -- 172.21.15.19:0/742466712 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d8037070 con 0x7f49e8153580 2024-03-28T13:14:09.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.027+0000 7f49e7400640 1 -- 172.21.15.19:0/742466712 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8031b90 con 0x7f49e8153580 2024-03-28T13:14:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.027+0000 7f49efa00640 1 -- 172.21.15.19:0/742466712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e8153580 msgr2=0x7f49e80d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.028+0000 7f49efa00640 1 --2- 172.21.15.19:0/742466712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e8153580 0x7f49e80d0280 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f49e80cff70 tx=0x7f49d802b870 comp rx=0 tx=0).stop 2024-03-28T13:14:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.028+0000 7f49efa00640 1 -- 172.21.15.19:0/742466712 shutdown_connections 2024-03-28T13:14:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.028+0000 7f49efa00640 1 --2- 172.21.15.19:0/742466712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e80af2e0 0x7f49e806d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.028+0000 7f49efa00640 1 --2- 172.21.15.19:0/742466712 >> v2:172.21.15.188:3300/0 conn(0x7f49e80d07c0 0x7f49e80aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.028+0000 7f49efa00640 1 --2- 172.21.15.19:0/742466712 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e8153580 0x7f49e80d0280 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.028+0000 7f49efa00640 1 -- 172.21.15.19:0/742466712 >> 172.21.15.19:0/742466712 conn(0x7f49e806b750 msgr2=0x7f49e806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.028+0000 7f49efa00640 1 -- 172.21.15.19:0/742466712 shutdown_connections 2024-03-28T13:14:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.028+0000 7f49efa00640 1 -- 172.21.15.19:0/742466712 wait complete. 2024-03-28T13:14:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.030+0000 7f49efa00640 1 Processor -- start 2024-03-28T13:14:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.030+0000 7f49efa00640 1 -- start start 2024-03-28T13:14:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.031+0000 7f49efa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e80af2e0 0x7f49e80ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.031+0000 7f49efa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80d07c0 0x7f49e809c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.031+0000 7f49efa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8153580 0x7f49e80cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.031+0000 7f49efa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49e815d4e0 con 0x7f49e80af2e0 2024-03-28T13:14:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.031+0000 7f49efa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49e815d200 con 0x7f49e80d07c0 2024-03-28T13:14:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.031+0000 7f49efa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f49e8153960 con 0x7f49e8153580 2024-03-28T13:14:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.031+0000 7f49ed000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e80af2e0 0x7f49e80ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.031+0000 7f49ed000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e80af2e0 0x7f49e80ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48550/0 (socket says 172.21.15.19:48550) 2024-03-28T13:14:09.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.031+0000 7f49ed000640 1 -- 172.21.15.19:0/1301836822 learned_addr learned my addr 172.21.15.19:0/1301836822 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.032+0000 7f49e7e00640 1 --2- 172.21.15.19:0/1301836822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80d07c0 0x7f49e809c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.032+0000 7f49eda00640 1 --2- 172.21.15.19:0/1301836822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8153580 0x7f49e80cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.032+0000 7f49ed000640 1 -- 172.21.15.19:0/1301836822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8153580 msgr2=0x7f49e80cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.032+0000 7f49ed000640 1 --2- 172.21.15.19:0/1301836822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8153580 0x7f49e80cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.032+0000 7f49ed000640 1 -- 172.21.15.19:0/1301836822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80d07c0 msgr2=0x7f49e809c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.032+0000 7f49ed000640 1 --2- 172.21.15.19:0/1301836822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80d07c0 0x7f49e809c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.032+0000 7f49ed000640 1 -- 172.21.15.19:0/1301836822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e809d210 con 0x7f49e80af2e0 2024-03-28T13:14:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.032+0000 7f49e7e00640 1 --2- 172.21.15.19:0/1301836822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80d07c0 0x7f49e809c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.032+0000 7f49ed000640 1 --2- 172.21.15.19:0/1301836822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e80af2e0 0x7f49e80ceff0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f49d802b8b0 tx=0x7f49d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.032+0000 7f49e4c00640 1 -- 172.21.15.19:0/1301836822 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8032870 con 0x7f49e80af2e0 2024-03-28T13:14:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.032+0000 7f49e4c00640 1 -- 172.21.15.19:0/1301836822 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d8032a10 con 0x7f49e80af2e0 2024-03-28T13:14:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.032+0000 7f49e4c00640 1 -- 172.21.15.19:0/1301836822 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8032e00 con 0x7f49e80af2e0 2024-03-28T13:14:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.032+0000 7f49efa00640 1 -- 172.21.15.19:0/1301836822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e809d4f0 con 0x7f49e80af2e0 2024-03-28T13:14:09.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.033+0000 7f49efa00640 1 -- 172.21.15.19:0/1301836822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e809cde0 con 0x7f49e80af2e0 2024-03-28T13:14:09.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.034+0000 7f49efa00640 1 -- 172.21.15.19:0/1301836822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e80685c0 con 0x7f49e80af2e0 2024-03-28T13:14:09.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.035+0000 7f49e4c00640 1 -- 172.21.15.19:0/1301836822 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f49d804c060 con 0x7f49e80af2e0 2024-03-28T13:14:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.036+0000 7f49e4c00640 1 --2- 172.21.15.19:0/1301836822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49c806c580 0x7f49c806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.036+0000 7f49e4c00640 1 -- 172.21.15.19:0/1301836822 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f49d80ad4b0 con 0x7f49e80af2e0 2024-03-28T13:14:09.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.036+0000 7f49e7e00640 1 --2- 172.21.15.19:0/1301836822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49c806c580 0x7f49c806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.037+0000 7f49e7e00640 1 --2- 172.21.15.19:0/1301836822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49c806c580 0x7f49c806ea40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f49e815c220 tx=0x7f49d4007490 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:09.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.044+0000 7f49e4c00640 1 -- 172.21.15.19:0/1301836822 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f49d8040070 con 0x7f49e80af2e0 2024-03-28T13:14:09.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.245+0000 7f49efa00640 1 -- 172.21.15.19:0/1301836822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f49e815bbd0 con 0x7f49e80af2e0 2024-03-28T13:14:09.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.246+0000 7f49e4c00640 1 -- 172.21.15.19:0/1301836822 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) ==== 93+0+10 (secure 0 0 0) 0x7f49d807b4b0 con 0x7f49e80af2e0 2024-03-28T13:14:09.246 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:14:09.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.249+0000 7f49efa00640 1 -- 172.21.15.19:0/1301836822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49c806c580 msgr2=0x7f49c806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.249+0000 7f49efa00640 1 --2- 172.21.15.19:0/1301836822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49c806c580 0x7f49c806ea40 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f49e815c220 tx=0x7f49d4007490 comp rx=0 tx=0).stop 2024-03-28T13:14:09.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.249+0000 7f49efa00640 1 -- 172.21.15.19:0/1301836822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e80af2e0 msgr2=0x7f49e80ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.249+0000 7f49efa00640 1 --2- 172.21.15.19:0/1301836822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e80af2e0 0x7f49e80ceff0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f49d802b8b0 tx=0x7f49d80046c0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.249+0000 7f49efa00640 1 -- 172.21.15.19:0/1301836822 shutdown_connections 2024-03-28T13:14:09.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.249+0000 7f49efa00640 1 --2- 172.21.15.19:0/1301836822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49c806c580 0x7f49c806ea40 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.249+0000 7f49efa00640 1 --2- 172.21.15.19:0/1301836822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8153580 0x7f49e80cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.249+0000 7f49efa00640 1 --2- 172.21.15.19:0/1301836822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80d07c0 0x7f49e809c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.249+0000 7f49efa00640 1 --2- 172.21.15.19:0/1301836822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e80af2e0 0x7f49e80ceff0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.249+0000 7f49efa00640 1 -- 172.21.15.19:0/1301836822 >> 172.21.15.19:0/1301836822 conn(0x7f49e806b750 msgr2=0x7f49e806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:09.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.249+0000 7f49efa00640 1 -- 172.21.15.19:0/1301836822 shutdown_connections 2024-03-28T13:14:09.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.249+0000 7f49efa00640 1 -- 172.21.15.19:0/1301836822 wait complete. 2024-03-28T13:14:09.266 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:14:09.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.480+0000 7f1d76e00640 1 Processor -- start 2024-03-28T13:14:09.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.481+0000 7f1d76e00640 1 -- start start 2024-03-28T13:14:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.482+0000 7f1d76e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1d700d0e70 0x7f1d700cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.483+0000 7f1d76e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d700cf6c0 con 0x7f1d700d2110 2024-03-28T13:14:09.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.483+0000 7f1d76e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d700cf9a0 con 0x7f1d7015f6e0 2024-03-28T13:14:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.483+0000 7f1d76e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1d700cfc80 con 0x7f1d700d0e70 2024-03-28T13:14:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.483+0000 7f1d74e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1d700d2110 0x7f1d700d25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51086/0 (socket says 172.21.15.19:51086) 2024-03-28T13:14:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.483+0000 7f1d74e00640 1 -- 172.21.15.19:0/2954087238 learned_addr learned my addr 172.21.15.19:0/2954087238 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.483+0000 7f1d6fe00640 1 --2- 172.21.15.19:0/2954087238 >> v2:172.21.15.188:3300/0 conn(0x7f1d700d0e70 0x7f1d700cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.483+0000 7f1d6f400640 1 -- 172.21.15.19:0/2954087238 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 668050925 0 0) 0x7f1d700cf6c0 con 0x7f1d700d2110 2024-03-28T13:14:09.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.484+0000 7f1d6f400640 1 -- 172.21.15.19:0/2954087238 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d50003520 con 0x7f1d700d2110 2024-03-28T13:14:09.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.484+0000 7f1d6f400640 1 -- 172.21.15.19:0/2954087238 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3640448752 0 0) 0x7f1d50003520 con 0x7f1d700d2110 2024-03-28T13:14:09.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.484+0000 7f1d6fe00640 1 -- 172.21.15.19:0/2954087238 >> v1:172.21.15.188:6789/0 conn(0x7f1d7015f6e0 legacy=0x7f1d700d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.484+0000 7f1d6fe00640 1 -- 172.21.15.19:0/2954087238 >> v1:172.21.15.19:6789/0 conn(0x7f1d700d2110 legacy=0x7f1d700d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.484+0000 7f1d6fe00640 1 -- 172.21.15.19:0/2954087238 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d700cff60 con 0x7f1d700d0e70 2024-03-28T13:14:09.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.485+0000 7f1d6fe00640 1 --2- 172.21.15.19:0/2954087238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d700d0e70 0x7f1d700cf050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1d600097a0 tx=0x7f1d6002b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13dbd137399a36a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:09.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.486+0000 7f1d6f400640 1 -- 172.21.15.19:0/2954087238 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60037070 con 0x7f1d700d0e70 2024-03-28T13:14:09.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.486+0000 7f1d6f400640 1 -- 172.21.15.19:0/2954087238 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d60030510 con 0x7f1d700d0e70 2024-03-28T13:14:09.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.486+0000 7f1d6f400640 1 -- 172.21.15.19:0/2954087238 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d680019f0 con 0x7f1d700d0e70 2024-03-28T13:14:09.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.486+0000 7f1d76e00640 1 -- 172.21.15.19:0/2954087238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d700d0e70 msgr2=0x7f1d700cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.486+0000 7f1d76e00640 1 --2- 172.21.15.19:0/2954087238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d700d0e70 0x7f1d700cf050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1d600097a0 tx=0x7f1d6002b790 comp rx=0 tx=0).stop 2024-03-28T13:14:09.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.486+0000 7f1d76e00640 1 -- 172.21.15.19:0/2954087238 shutdown_connections 2024-03-28T13:14:09.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.486+0000 7f1d76e00640 1 --2- 172.21.15.19:0/2954087238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d700d0e70 0x7f1d700cf050 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.486+0000 7f1d76e00640 1 -- 172.21.15.19:0/2954087238 >> 172.21.15.19:0/2954087238 conn(0x7f1d7008bfb0 msgr2=0x7f1d700d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:09.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.486+0000 7f1d76e00640 1 -- 172.21.15.19:0/2954087238 shutdown_connections 2024-03-28T13:14:09.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.486+0000 7f1d76e00640 1 -- 172.21.15.19:0/2954087238 wait complete. 2024-03-28T13:14:09.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.488+0000 7f1d76e00640 1 Processor -- start 2024-03-28T13:14:09.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.488+0000 7f1d76e00640 1 -- start start 2024-03-28T13:14:09.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.489+0000 7f1d76e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d700d0e70 0x7f1d70154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.489+0000 7f1d76e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d700d2110 0x7f1d700cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.489+0000 7f1d76e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d7015f6e0 0x7f1d700c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.489+0000 7f1d76e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7015e910 con 0x7f1d700d2110 2024-03-28T13:14:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.489+0000 7f1d76e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7015fb40 con 0x7f1d7015f6e0 2024-03-28T13:14:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.489+0000 7f1d76e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1d7015ea90 con 0x7f1d700d0e70 2024-03-28T13:14:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.490+0000 7f1d67e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d700d2110 0x7f1d700cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.490+0000 7f1d67e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d700d2110 0x7f1d700cb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48560/0 (socket says 172.21.15.19:48560) 2024-03-28T13:14:09.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.490+0000 7f1d67e00640 1 -- 172.21.15.19:0/4186663815 learned_addr learned my addr 172.21.15.19:0/4186663815 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.490+0000 7f1d6fe00640 1 --2- 172.21.15.19:0/4186663815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d700d0e70 0x7f1d70154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.490+0000 7f1d67e00640 1 -- 172.21.15.19:0/4186663815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d700d0e70 msgr2=0x7f1d70154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.490+0000 7f1d67e00640 1 --2- 172.21.15.19:0/4186663815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d700d0e70 0x7f1d70154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.490+0000 7f1d67e00640 1 -- 172.21.15.19:0/4186663815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d7015f6e0 msgr2=0x7f1d700c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.490+0000 7f1d67e00640 1 --2- 172.21.15.19:0/4186663815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d7015f6e0 0x7f1d700c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.490+0000 7f1d67e00640 1 -- 172.21.15.19:0/4186663815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d700ca870 con 0x7f1d700d2110 2024-03-28T13:14:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.491+0000 7f1d67e00640 1 --2- 172.21.15.19:0/4186663815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d700d2110 0x7f1d700cb530 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c450 tx=0x7f1d5800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:09.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.491+0000 7f1d6cc00640 1 -- 172.21.15.19:0/4186663815 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d580052d0 con 0x7f1d700d2110 2024-03-28T13:14:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.491+0000 7f1d6cc00640 1 -- 172.21.15.19:0/4186663815 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d58005470 con 0x7f1d700d2110 2024-03-28T13:14:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.491+0000 7f1d76e00640 1 -- 172.21.15.19:0/4186663815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d700cab50 con 0x7f1d700d2110 2024-03-28T13:14:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.491+0000 7f1d6cc00640 1 -- 172.21.15.19:0/4186663815 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d58012870 con 0x7f1d700d2110 2024-03-28T13:14:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.491+0000 7f1d76e00640 1 -- 172.21.15.19:0/4186663815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d700c8d50 con 0x7f1d700d2110 2024-03-28T13:14:09.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.492+0000 7f1d76e00640 1 -- 172.21.15.19:0/4186663815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d70068cd0 con 0x7f1d700d2110 2024-03-28T13:14:09.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.494+0000 7f1d6cc00640 1 -- 172.21.15.19:0/4186663815 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1d58012a10 con 0x7f1d700d2110 2024-03-28T13:14:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.495+0000 7f1d6cc00640 1 --2- 172.21.15.19:0/4186663815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d4c06c530 0x7f1d4c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.496+0000 7f1d6cc00640 1 -- 172.21.15.19:0/4186663815 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f1d5808b350 con 0x7f1d700d2110 2024-03-28T13:14:09.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.496+0000 7f1d6fe00640 1 --2- 172.21.15.19:0/4186663815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d4c06c530 0x7f1d4c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.497+0000 7f1d6fe00640 1 --2- 172.21.15.19:0/4186663815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d4c06c530 0x7f1d4c06e9f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1d60034000 tx=0x7f1d6002f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:09.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.505+0000 7f1d6cc00640 1 -- 172.21.15.19:0/4186663815 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1d580598a0 con 0x7f1d700d2110 2024-03-28T13:14:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.711+0000 7f1d76e00640 1 -- 172.21.15.19:0/4186663815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1d700d1b80 con 0x7f1d700d2110 2024-03-28T13:14:09.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.712+0000 7f1d6cc00640 1 -- 172.21.15.19:0/4186663815 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) ==== 104+0+11 (secure 0 0 0) 0x7f1d580591a0 con 0x7f1d700d2110 2024-03-28T13:14:09.711 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:14:09.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.714+0000 7f1d76e00640 1 -- 172.21.15.19:0/4186663815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d4c06c530 msgr2=0x7f1d4c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.714+0000 7f1d76e00640 1 --2- 172.21.15.19:0/4186663815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d4c06c530 0x7f1d4c06e9f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1d60034000 tx=0x7f1d6002f000 comp rx=0 tx=0).stop 2024-03-28T13:14:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.714+0000 7f1d76e00640 1 -- 172.21.15.19:0/4186663815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d700d2110 msgr2=0x7f1d700cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.714+0000 7f1d76e00640 1 --2- 172.21.15.19:0/4186663815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d700d2110 0x7f1d700cb530 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1d5800c450 tx=0x7f1d5800c910 comp rx=0 tx=0).stop 2024-03-28T13:14:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.714+0000 7f1d76e00640 1 -- 172.21.15.19:0/4186663815 shutdown_connections 2024-03-28T13:14:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.714+0000 7f1d76e00640 1 --2- 172.21.15.19:0/4186663815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d4c06c530 0x7f1d4c06e9f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.714+0000 7f1d76e00640 1 --2- 172.21.15.19:0/4186663815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d7015f6e0 0x7f1d700c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.714+0000 7f1d76e00640 1 --2- 172.21.15.19:0/4186663815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d700d2110 0x7f1d700cb530 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.714+0000 7f1d76e00640 1 --2- 172.21.15.19:0/4186663815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d700d0e70 0x7f1d70154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.714+0000 7f1d76e00640 1 -- 172.21.15.19:0/4186663815 >> 172.21.15.19:0/4186663815 conn(0x7f1d7008bfb0 msgr2=0x7f1d7006de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.715+0000 7f1d76e00640 1 -- 172.21.15.19:0/4186663815 shutdown_connections 2024-03-28T13:14:09.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.715+0000 7f1d76e00640 1 -- 172.21.15.19:0/4186663815 wait complete. 2024-03-28T13:14:09.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:14:09.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.948+0000 7f691f200640 1 Processor -- start 2024-03-28T13:14:09.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.949+0000 7f691f200640 1 -- start start 2024-03-28T13:14:09.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.952+0000 7f691f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69180d0990 0x7f69180aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.952+0000 7f691f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69180b40b0 con 0x7f6918155930 2024-03-28T13:14:09.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.952+0000 7f691f200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69180b4390 con 0x7f6918155550 2024-03-28T13:14:09.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.952+0000 7f691f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f69180b4670 con 0x7f69180d0990 2024-03-28T13:14:09.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.952+0000 7f6917400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6918155930 0x7f69180d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51092/0 (socket says 172.21.15.19:51092) 2024-03-28T13:14:09.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.952+0000 7f6917400640 1 -- 172.21.15.19:0/2515215577 learned_addr learned my addr 172.21.15.19:0/2515215577 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:09.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.952+0000 7f691d200640 1 --2- 172.21.15.19:0/2515215577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69180d0990 0x7f69180aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.953+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 841287747 0 0) 0x7f69180b40b0 con 0x7f6918155930 2024-03-28T13:14:09.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.953+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68fc003520 con 0x7f6918155930 2024-03-28T13:14:09.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.953+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3901140655 0 0) 0x7f69180b4390 con 0x7f6918155550 2024-03-28T13:14:09.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.953+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69180b40b0 con 0x7f6918155550 2024-03-28T13:14:09.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.953+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3726389833 0 0) 0x7f68fc003520 con 0x7f6918155930 2024-03-28T13:14:09.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.953+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69180d0990 msgr2=0x7f69180aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:09.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.953+0000 7f6916a00640 1 --2- 172.21.15.19:0/2515215577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69180d0990 0x7f69180aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.953+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 >> v1:172.21.15.188:6789/0 conn(0x7f6918155550 legacy=0x7f691815f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.953+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69180b4950 con 0x7f6918155930 2024-03-28T13:14:09.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.953+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f6904001670 con 0x7f6918155930 2024-03-28T13:14:09.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.953+0000 7f691d200640 1 --2- 172.21.15.19:0/2515215577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69180d0990 0x7f69180aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:14:09.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.953+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 >> v1:172.21.15.19:6789/0 conn(0x7f6918155930 legacy=0x7f69180d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.953+0000 7f6916a00640 1 --2- 172.21.15.19:0/2515215577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68fc003b70 0x7f68fc003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.953+0000 7f6916a00640 1 --2- 172.21.15.19:0/2515215577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69180d0990 0x7f68fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.954+0000 7f6916a00640 1 --2- 172.21.15.19:0/2515215577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68fc004d40 0x7f68fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.954+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f69180b4670 con 0x7f68fc003b70 2024-03-28T13:14:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.954+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc00ba70 con 0x7f68fc004d40 2024-03-28T13:14:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.954+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f68fc00de30 con 0x7f69180d0990 2024-03-28T13:14:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.954+0000 7f6917e00640 1 --2- 172.21.15.19:0/2515215577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68fc003b70 0x7f68fc003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.954+0000 7f6917400640 1 --2- 172.21.15.19:0/2515215577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69180d0990 0x7f68fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.954+0000 7f691d200640 1 --2- 172.21.15.19:0/2515215577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68fc004d40 0x7f68fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.954+0000 7f6917e00640 1 -- 172.21.15.19:0/2515215577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69180d0990 msgr2=0x7f68fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.954+0000 7f6917e00640 1 --2- 172.21.15.19:0/2515215577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69180d0990 0x7f68fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.955+0000 7f6917e00640 1 -- 172.21.15.19:0/2515215577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68fc004d40 msgr2=0x7f68fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.955+0000 7f6917e00640 1 --2- 172.21.15.19:0/2515215577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68fc004d40 0x7f68fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.955+0000 7f6917e00640 1 -- 172.21.15.19:0/2515215577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc003520 con 0x7f68fc003b70 2024-03-28T13:14:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.955+0000 7f6917400640 1 --2- 172.21.15.19:0/2515215577 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69180d0990 0x7f68fc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:14:09.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.955+0000 7f691d200640 1 --2- 172.21.15.19:0/2515215577 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68fc004d40 0x7f68fc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:09.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.956+0000 7f6917e00640 1 --2- 172.21.15.19:0/2515215577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68fc003b70 0x7f68fc003f70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f69080018d0 tx=0x7f68fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c257c0271755be server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:09.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.956+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6908037070 con 0x7f68fc003b70 2024-03-28T13:14:09.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.956+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69080050d0 con 0x7f68fc003b70 2024-03-28T13:14:09.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.956+0000 7f6916a00640 1 -- 172.21.15.19:0/2515215577 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69080014f0 con 0x7f68fc003b70 2024-03-28T13:14:09.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.957+0000 7f691f200640 1 -- 172.21.15.19:0/2515215577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68fc003b70 msgr2=0x7f68fc003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.957+0000 7f691f200640 1 --2- 172.21.15.19:0/2515215577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68fc003b70 0x7f68fc003f70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f69080018d0 tx=0x7f68fc005830 comp rx=0 tx=0).stop 2024-03-28T13:14:09.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.957+0000 7f691d200640 1 -- 172.21.15.19:0/2515215577 reap_dead start 2024-03-28T13:14:09.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.957+0000 7f691f200640 1 -- 172.21.15.19:0/2515215577 shutdown_connections 2024-03-28T13:14:09.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.957+0000 7f691f200640 1 -- 172.21.15.19:0/2515215577 >> 172.21.15.19:0/2515215577 conn(0x7f691806b750 msgr2=0x7f691806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:09.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.957+0000 7f691f200640 1 -- 172.21.15.19:0/2515215577 shutdown_connections 2024-03-28T13:14:09.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.957+0000 7f691f200640 1 -- 172.21.15.19:0/2515215577 wait complete. 2024-03-28T13:14:09.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.959+0000 7f691f200640 1 Processor -- start 2024-03-28T13:14:09.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.959+0000 7f691f200640 1 -- start start 2024-03-28T13:14:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.960+0000 7f691f200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68fc003b70 0x7f69180cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.960+0000 7f691f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f691806e0e0 0x7f69180992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.960+0000 7f691f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6918153ee0 0x7f6918154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.960+0000 7f691f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f691815bba0 con 0x7f691806e0e0 2024-03-28T13:14:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.960+0000 7f691f200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f691815ba20 con 0x7f68fc003b70 2024-03-28T13:14:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.960+0000 7f691f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f69180b3cf0 con 0x7f6918153ee0 2024-03-28T13:14:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.960+0000 7f6917400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f691806e0e0 0x7f69180992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.960+0000 7f6917400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f691806e0e0 0x7f69180992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48578/0 (socket says 172.21.15.19:48578) 2024-03-28T13:14:09.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.960+0000 7f6917400640 1 -- 172.21.15.19:0/1869549658 learned_addr learned my addr 172.21.15.19:0/1869549658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:09.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.960+0000 7f6917400640 1 -- 172.21.15.19:0/1869549658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6918153ee0 msgr2=0x7f6918154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.960+0000 7f6917400640 1 --2- 172.21.15.19:0/1869549658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6918153ee0 0x7f6918154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.960+0000 7f6917400640 1 -- 172.21.15.19:0/1869549658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68fc003b70 msgr2=0x7f69180cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:09.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.961+0000 7f6917e00640 1 --2- 172.21.15.19:0/1869549658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68fc003b70 0x7f69180cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.961+0000 7f6917400640 1 --2- 172.21.15.19:0/1869549658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68fc003b70 0x7f69180cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:09.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.961+0000 7f6917400640 1 -- 172.21.15.19:0/1869549658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69180cdde0 con 0x7f691806e0e0 2024-03-28T13:14:09.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.961+0000 7f6917400640 1 --2- 172.21.15.19:0/1869549658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f691806e0e0 0x7f69180992f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f69040017e0 tx=0x7f690400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:09.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.961+0000 7f690fe00640 1 -- 172.21.15.19:0/1869549658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690400eea0 con 0x7f691806e0e0 2024-03-28T13:14:09.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.961+0000 7f690fe00640 1 -- 172.21.15.19:0/1869549658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6904018070 con 0x7f691806e0e0 2024-03-28T13:14:09.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.961+0000 7f690fe00640 1 -- 172.21.15.19:0/1869549658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690400ca70 con 0x7f691806e0e0 2024-03-28T13:14:09.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.961+0000 7f691f200640 1 -- 172.21.15.19:0/1869549658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69180ce0c0 con 0x7f691806e0e0 2024-03-28T13:14:09.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.962+0000 7f691f200640 1 -- 172.21.15.19:0/1869549658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69180ce680 con 0x7f691806e0e0 2024-03-28T13:14:09.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.963+0000 7f691f200640 1 -- 172.21.15.19:0/1869549658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69180685c0 con 0x7f691806e0e0 2024-03-28T13:14:09.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.964+0000 7f690fe00640 1 -- 172.21.15.19:0/1869549658 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f690400cc10 con 0x7f691806e0e0 2024-03-28T13:14:09.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.964+0000 7f690fe00640 1 --2- 172.21.15.19:0/1869549658 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f68f806c530 0x7f68f806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:09.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.965+0000 7f690fe00640 1 -- 172.21.15.19:0/1869549658 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f690408b550 con 0x7f691806e0e0 2024-03-28T13:14:09.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.965+0000 7f6917e00640 1 --2- 172.21.15.19:0/1869549658 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f68f806c530 0x7f68f806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:09.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.966+0000 7f6917e00640 1 --2- 172.21.15.19:0/1869549658 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f68f806c530 0x7f68f806e9f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6908034000 tx=0x7f6908031000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:09.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:09.969+0000 7f690fe00640 1 -- 172.21.15.19:0/1869549658 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6904059a90 con 0x7f691806e0e0 2024-03-28T13:14:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.296+0000 7f691f200640 1 -- 172.21.15.19:0/1869549658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f69180b3ad0 con 0x7f691806e0e0 2024-03-28T13:14:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.297+0000 7f690fe00640 1 -- 172.21.15.19:0/1869549658 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) ==== 100+0+11 (secure 0 0 0) 0x7f6904059390 con 0x7f691806e0e0 2024-03-28T13:14:10.296 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:14:10.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.299+0000 7f691f200640 1 -- 172.21.15.19:0/1869549658 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f68f806c530 msgr2=0x7f68f806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:10.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.299+0000 7f691f200640 1 --2- 172.21.15.19:0/1869549658 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f68f806c530 0x7f68f806e9f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6908034000 tx=0x7f6908031000 comp rx=0 tx=0).stop 2024-03-28T13:14:10.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.299+0000 7f691f200640 1 -- 172.21.15.19:0/1869549658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f691806e0e0 msgr2=0x7f69180992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:10.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.299+0000 7f691f200640 1 --2- 172.21.15.19:0/1869549658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f691806e0e0 0x7f69180992f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f69040017e0 tx=0x7f690400eb90 comp rx=0 tx=0).stop 2024-03-28T13:14:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.299+0000 7f691f200640 1 -- 172.21.15.19:0/1869549658 shutdown_connections 2024-03-28T13:14:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.299+0000 7f691f200640 1 --2- 172.21.15.19:0/1869549658 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f68f806c530 0x7f68f806e9f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.299+0000 7f691f200640 1 --2- 172.21.15.19:0/1869549658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6918153ee0 0x7f6918154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.299+0000 7f691f200640 1 --2- 172.21.15.19:0/1869549658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f691806e0e0 0x7f69180992f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.299+0000 7f691f200640 1 --2- 172.21.15.19:0/1869549658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68fc003b70 0x7f69180cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.299+0000 7f691f200640 1 -- 172.21.15.19:0/1869549658 >> 172.21.15.19:0/1869549658 conn(0x7f691806b750 msgr2=0x7f691806d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.299+0000 7f691f200640 1 -- 172.21.15.19:0/1869549658 shutdown_connections 2024-03-28T13:14:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.299+0000 7f691f200640 1 -- 172.21.15.19:0/1869549658 wait complete. 2024-03-28T13:14:10.317 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:14:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.542+0000 7f43b5c00640 1 Processor -- start 2024-03-28T13:14:10.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.542+0000 7f43b5c00640 1 -- start start 2024-03-28T13:14:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.544+0000 7f43b5c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f43b015fc50 0x7f43b00cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.544+0000 7f43b5c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00cc490 0x7f43b00d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.544+0000 7f43b5c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43b00b3150 con 0x7f43b00cc0b0 2024-03-28T13:14:10.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.544+0000 7f43b5c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f43b015c610 con 0x7f43b015fc50 2024-03-28T13:14:10.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.544+0000 7f43b5c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f43b01600e0 con 0x7f43b00cc490 2024-03-28T13:14:10.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.545+0000 7f43afe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f43b00cc0b0 0x7f43b006dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51104/0 (socket says 172.21.15.19:51104) 2024-03-28T13:14:10.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.545+0000 7f43afe00640 1 -- 172.21.15.19:0/2798249448 learned_addr learned my addr 172.21.15.19:0/2798249448 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.545+0000 7f43b5200640 1 --2- 172.21.15.19:0/2798249448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00cc490 0x7f43b00d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.545+0000 7f43af400640 1 --2- 172.21.15.19:0/2798249448 >> v2:172.21.15.188:3300/0 conn(0x7f43b015fc50 0x7f43b00cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.545+0000 7f43aea00640 1 -- 172.21.15.19:0/2798249448 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2108740990 0 0) 0x7f43b00b3150 con 0x7f43b00cc0b0 2024-03-28T13:14:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.545+0000 7f43aea00640 1 -- 172.21.15.19:0/2798249448 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4394003520 con 0x7f43b00cc0b0 2024-03-28T13:14:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.546+0000 7f43aea00640 1 -- 172.21.15.19:0/2798249448 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2461817352 0 0) 0x7f4394003520 con 0x7f43b00cc0b0 2024-03-28T13:14:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.546+0000 7f43aea00640 1 -- 172.21.15.19:0/2798249448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00cc490 msgr2=0x7f43b00d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.546+0000 7f43aea00640 1 --2- 172.21.15.19:0/2798249448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00cc490 0x7f43b00d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.546+0000 7f43aea00640 1 -- 172.21.15.19:0/2798249448 >> v2:172.21.15.188:3300/0 conn(0x7f43b015fc50 msgr2=0x7f43b00cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.546+0000 7f43aea00640 1 --2- 172.21.15.19:0/2798249448 >> v2:172.21.15.188:3300/0 conn(0x7f43b015fc50 0x7f43b00cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.546+0000 7f43aea00640 1 -- 172.21.15.19:0/2798249448 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b01603c0 con 0x7f43b00cc0b0 2024-03-28T13:14:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.546+0000 7f43aea00640 1 -- 172.21.15.19:0/2798249448 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f43a0001670 con 0x7f43b00cc0b0 2024-03-28T13:14:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.546+0000 7f43aea00640 1 -- 172.21.15.19:0/2798249448 >> v1:172.21.15.19:6789/0 conn(0x7f43b00cc0b0 legacy=0x7f43b006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.546+0000 7f43aea00640 1 --2- 172.21.15.19:0/2798249448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43940039b0 0x7f4394003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:10.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.546+0000 7f43aea00640 1 --2- 172.21.15.19:0/2798249448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00cc490 0x7f4394004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:10.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.546+0000 7f43aea00640 1 --2- 172.21.15.19:0/2798249448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4394004d40 0x7f43940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:10.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.546+0000 7f43aea00640 1 -- 172.21.15.19:0/2798249448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43b015c610 con 0x7f4394004d40 2024-03-28T13:14:10.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.546+0000 7f43aea00640 1 -- 172.21.15.19:0/2798249448 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43b01600e0 con 0x7f43940039b0 2024-03-28T13:14:10.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.546+0000 7f43aea00640 1 -- 172.21.15.19:0/2798249448 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f439400b9c0 con 0x7f43b00cc490 2024-03-28T13:14:10.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.546+0000 7f43b5200640 1 --2- 172.21.15.19:0/2798249448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4394004d40 0x7f43940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:10.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.547+0000 7f43b5200640 1 -- 172.21.15.19:0/2798249448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00cc490 msgr2=0x7f4394004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:10.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.547+0000 7f43b5200640 1 --2- 172.21.15.19:0/2798249448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00cc490 0x7f4394004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:10.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.547+0000 7f43b5200640 1 -- 172.21.15.19:0/2798249448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43940039b0 msgr2=0x7f4394003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:10.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.547+0000 7f43b5200640 1 --2- 172.21.15.19:0/2798249448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43940039b0 0x7f4394003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:10.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.547+0000 7f43b5200640 1 -- 172.21.15.19:0/2798249448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4394003520 con 0x7f4394004d40 2024-03-28T13:14:10.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.548+0000 7f43b5200640 1 --2- 172.21.15.19:0/2798249448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4394004d40 0x7f43940051c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009f90 tx=0x7f4394005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339cb13599fa5234 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:10.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.548+0000 7f43aea00640 1 -- 172.21.15.19:0/2798249448 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8039070 con 0x7f4394004d40 2024-03-28T13:14:10.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.548+0000 7f43aea00640 1 -- 172.21.15.19:0/2798249448 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a802cc20 con 0x7f4394004d40 2024-03-28T13:14:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.548+0000 7f43aea00640 1 -- 172.21.15.19:0/2798249448 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8009c10 con 0x7f4394004d40 2024-03-28T13:14:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.549+0000 7f43b5c00640 1 -- 172.21.15.19:0/2798249448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4394004d40 msgr2=0x7f43940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.549+0000 7f43b5c00640 1 --2- 172.21.15.19:0/2798249448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4394004d40 0x7f43940051c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009f90 tx=0x7f4394005830 comp rx=0 tx=0).stop 2024-03-28T13:14:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.549+0000 7f43b5200640 1 -- 172.21.15.19:0/2798249448 reap_dead start 2024-03-28T13:14:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.549+0000 7f43b5c00640 1 -- 172.21.15.19:0/2798249448 shutdown_connections 2024-03-28T13:14:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.549+0000 7f43b5c00640 1 -- 172.21.15.19:0/2798249448 >> 172.21.15.19:0/2798249448 conn(0x7f43b00ccad0 msgr2=0x7f43b00cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.549+0000 7f43b5c00640 1 -- 172.21.15.19:0/2798249448 shutdown_connections 2024-03-28T13:14:10.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.549+0000 7f43b5c00640 1 -- 172.21.15.19:0/2798249448 wait complete. 2024-03-28T13:14:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.551+0000 7f43b5c00640 1 Processor -- start 2024-03-28T13:14:10.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.551+0000 7f43b5c00640 1 -- start start 2024-03-28T13:14:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.552+0000 7f43b5c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4394004d40 0x7f43b013a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.552+0000 7f43b5c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b015f4d0 0x7f43b0137860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.552+0000 7f43b5c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43b015e510 0x7f43b015e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.552+0000 7f43b5c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43b00d0300 con 0x7f43b015e510 2024-03-28T13:14:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.552+0000 7f43b5c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43b0160980 con 0x7f43b015f4d0 2024-03-28T13:14:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.552+0000 7f43b5c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f43b00d0480 con 0x7f4394004d40 2024-03-28T13:14:10.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.552+0000 7f43b5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43b015e510 0x7f43b015e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.552+0000 7f43b5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43b015e510 0x7f43b015e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48594/0 (socket says 172.21.15.19:48594) 2024-03-28T13:14:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.552+0000 7f43b5200640 1 -- 172.21.15.19:0/2490306580 learned_addr learned my addr 172.21.15.19:0/2490306580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.552+0000 7f43af400640 1 --2- 172.21.15.19:0/2490306580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b015f4d0 0x7f43b0137860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.552+0000 7f43afe00640 1 --2- 172.21.15.19:0/2490306580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4394004d40 0x7f43b013a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.552+0000 7f43b5200640 1 -- 172.21.15.19:0/2490306580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4394004d40 msgr2=0x7f43b013a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.552+0000 7f43b5200640 1 --2- 172.21.15.19:0/2490306580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4394004d40 0x7f43b013a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.552+0000 7f43b5200640 1 -- 172.21.15.19:0/2490306580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b015f4d0 msgr2=0x7f43b0137860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.553+0000 7f43b5200640 1 --2- 172.21.15.19:0/2490306580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b015f4d0 0x7f43b0137860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:10.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.553+0000 7f43b5200640 1 -- 172.21.15.19:0/2490306580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b013a9a0 con 0x7f43b015e510 2024-03-28T13:14:10.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.553+0000 7f43b5200640 1 --2- 172.21.15.19:0/2490306580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43b015e510 0x7f43b015e9c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f43a8037040 tx=0x7f43a8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:10.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.553+0000 7f43a7e00640 1 -- 172.21.15.19:0/2490306580 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8039070 con 0x7f43b015e510 2024-03-28T13:14:10.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.553+0000 7f43a7e00640 1 -- 172.21.15.19:0/2490306580 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a8044070 con 0x7f43b015e510 2024-03-28T13:14:10.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.553+0000 7f43a7e00640 1 -- 172.21.15.19:0/2490306580 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a803e830 con 0x7f43b015e510 2024-03-28T13:14:10.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.553+0000 7f43b5c00640 1 -- 172.21.15.19:0/2490306580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b013ac80 con 0x7f43b015e510 2024-03-28T13:14:10.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.553+0000 7f43b5c00640 1 -- 172.21.15.19:0/2490306580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b013b240 con 0x7f43b015e510 2024-03-28T13:14:10.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.554+0000 7f43b5c00640 1 -- 172.21.15.19:0/2490306580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b0075940 con 0x7f43b015e510 2024-03-28T13:14:10.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.556+0000 7f43a7e00640 1 -- 172.21.15.19:0/2490306580 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f43a803e9d0 con 0x7f43b015e510 2024-03-28T13:14:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.556+0000 7f43a7e00640 1 --2- 172.21.15.19:0/2490306580 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f438c06c560 0x7f438c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:10.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.556+0000 7f43a7e00640 1 -- 172.21.15.19:0/2490306580 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f43a80ada00 con 0x7f43b015e510 2024-03-28T13:14:10.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.557+0000 7f43afe00640 1 --2- 172.21.15.19:0/2490306580 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f438c06c560 0x7f438c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:10.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.558+0000 7f43afe00640 1 --2- 172.21.15.19:0/2490306580 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f438c06c560 0x7f438c06ea20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f43a00011e0 tx=0x7f43a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:10.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.561+0000 7f43a7e00640 1 -- 172.21.15.19:0/2490306580 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f43a807c070 con 0x7f43b015e510 2024-03-28T13:14:10.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.779+0000 7f43b5c00640 1 -- 172.21.15.19:0/2490306580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f43b00b29c0 con 0x7f43b015e510 2024-03-28T13:14:10.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.780+0000 7f43a7e00640 1 -- 172.21.15.19:0/2490306580 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) ==== 80+0+0 (secure 0 0 0) 0x7f43a807b970 con 0x7f43b015e510 2024-03-28T13:14:10.780 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-03-28T13:14:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.783+0000 7f43b5c00640 1 -- 172.21.15.19:0/2490306580 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f438c06c560 msgr2=0x7f438c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.783+0000 7f43b5c00640 1 --2- 172.21.15.19:0/2490306580 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f438c06c560 0x7f438c06ea20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f43a00011e0 tx=0x7f43a0009040 comp rx=0 tx=0).stop 2024-03-28T13:14:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.783+0000 7f43b5c00640 1 -- 172.21.15.19:0/2490306580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43b015e510 msgr2=0x7f43b015e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.783+0000 7f43b5c00640 1 --2- 172.21.15.19:0/2490306580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43b015e510 0x7f43b015e9c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f43a8037040 tx=0x7f43a8009750 comp rx=0 tx=0).stop 2024-03-28T13:14:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.783+0000 7f43b5c00640 1 -- 172.21.15.19:0/2490306580 shutdown_connections 2024-03-28T13:14:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.783+0000 7f43b5c00640 1 --2- 172.21.15.19:0/2490306580 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f438c06c560 0x7f438c06ea20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.783+0000 7f43b5c00640 1 --2- 172.21.15.19:0/2490306580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43b015e510 0x7f43b015e9c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.783+0000 7f43b5c00640 1 --2- 172.21.15.19:0/2490306580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b015f4d0 0x7f43b0137860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.783+0000 7f43b5c00640 1 --2- 172.21.15.19:0/2490306580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4394004d40 0x7f43b013a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:10.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.783+0000 7f43b5c00640 1 -- 172.21.15.19:0/2490306580 >> 172.21.15.19:0/2490306580 conn(0x7f43b00ccad0 msgr2=0x7f43b0071f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.783+0000 7f43b5c00640 1 -- 172.21.15.19:0/2490306580 shutdown_connections 2024-03-28T13:14:10.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:10.783+0000 7f43b5c00640 1 -- 172.21.15.19:0/2490306580 wait complete. 2024-03-28T13:14:10.801 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:14:10.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:10.801 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T13:14:10.937 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:14:10.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:10.937 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:14:10.986 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:14:10.986 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:14:10.986 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:14:11.025 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:14:11.025 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:14:11.025 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:14:11.025 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:14:11.063 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:14:11.063 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:14:11.063 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:14:11.063 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:14:11.110 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:14:11.110 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:14:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.309+0000 7f140d600640 1 Processor -- start 2024-03-28T13:14:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.309+0000 7f140d600640 1 -- start start 2024-03-28T13:14:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.310+0000 7f140d600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14080b63c0 0x7f1408168f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.310+0000 7f140d600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14080d2cf0 con 0x7f140815e3b0 2024-03-28T13:14:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.310+0000 7f140d600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14080d2fd0 con 0x7f14080d3d90 2024-03-28T13:14:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.310+0000 7f140d600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f14080d32b0 con 0x7f14080b63c0 2024-03-28T13:14:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.311+0000 7f1407e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f140815e3b0 0x7f14080d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51106/0 (socket says 172.21.15.19:51106) 2024-03-28T13:14:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.311+0000 7f1407e00640 1 -- 172.21.15.19:0/1054986850 learned_addr learned my addr 172.21.15.19:0/1054986850 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.311+0000 7f1406a00640 1 -- 172.21.15.19:0/1054986850 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3994393599 0 0) 0x7f14080d2cf0 con 0x7f140815e3b0 2024-03-28T13:14:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.311+0000 7f1406a00640 1 -- 172.21.15.19:0/1054986850 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13ec003520 con 0x7f140815e3b0 2024-03-28T13:14:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.311+0000 7f140cc00640 1 --2- 172.21.15.19:0/1054986850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14080b63c0 0x7f1408168f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.311+0000 7f1406a00640 1 -- 172.21.15.19:0/1054986850 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1698244226 0 0) 0x7f13ec003520 con 0x7f140815e3b0 2024-03-28T13:14:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.311+0000 7f1406a00640 1 -- 172.21.15.19:0/1054986850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14080b63c0 msgr2=0x7f1408168f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.311+0000 7f1406a00640 1 --2- 172.21.15.19:0/1054986850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14080b63c0 0x7f1408168f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.311+0000 7f1406a00640 1 -- 172.21.15.19:0/1054986850 >> v1:172.21.15.188:6789/0 conn(0x7f14080d3d90 legacy=0x7f14080b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.311+0000 7f1406a00640 1 -- 172.21.15.19:0/1054986850 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14080d3590 con 0x7f140815e3b0 2024-03-28T13:14:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.311+0000 7f1406a00640 1 -- 172.21.15.19:0/1054986850 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f13f8001670 con 0x7f140815e3b0 2024-03-28T13:14:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.311+0000 7f140cc00640 1 --2- 172.21.15.19:0/1054986850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14080b63c0 0x7f1408168f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:14:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.311+0000 7f1406a00640 1 -- 172.21.15.19:0/1054986850 >> v1:172.21.15.19:6789/0 conn(0x7f140815e3b0 legacy=0x7f14080d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.312+0000 7f1406a00640 1 --2- 172.21.15.19:0/1054986850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14080b63c0 0x7f13ec0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.312+0000 7f1406a00640 1 --2- 172.21.15.19:0/1054986850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13ec004020 0x7f13ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:11.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.312+0000 7f1406a00640 1 --2- 172.21.15.19:0/1054986850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13ec004d40 0x7f13ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.312+0000 7f1406a00640 1 -- 172.21.15.19:0/1054986850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14080d32b0 con 0x7f13ec004d40 2024-03-28T13:14:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.312+0000 7f1406a00640 1 -- 172.21.15.19:0/1054986850 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec00bc10 con 0x7f13ec004020 2024-03-28T13:14:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.312+0000 7f1406a00640 1 -- 172.21.15.19:0/1054986850 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f13ec00be10 con 0x7f14080b63c0 2024-03-28T13:14:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.312+0000 7f140cc00640 1 --2- 172.21.15.19:0/1054986850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13ec004d40 0x7f13ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.312+0000 7f1407400640 1 --2- 172.21.15.19:0/1054986850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13ec004020 0x7f13ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.312+0000 7f1407e00640 1 --2- 172.21.15.19:0/1054986850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14080b63c0 0x7f13ec0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.312+0000 7f140cc00640 1 -- 172.21.15.19:0/1054986850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14080b63c0 msgr2=0x7f13ec0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.312+0000 7f140cc00640 1 --2- 172.21.15.19:0/1054986850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14080b63c0 0x7f13ec0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.312+0000 7f140cc00640 1 -- 172.21.15.19:0/1054986850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13ec004020 msgr2=0x7f13ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.312+0000 7f140cc00640 1 --2- 172.21.15.19:0/1054986850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13ec004020 0x7f13ec004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.312+0000 7f140cc00640 1 -- 172.21.15.19:0/1054986850 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ec003520 con 0x7f13ec004d40 2024-03-28T13:14:11.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.312+0000 7f1407e00640 1 --2- 172.21.15.19:0/1054986850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14080b63c0 0x7f13ec0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.314+0000 7f140cc00640 1 --2- 172.21.15.19:0/1054986850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13ec004d40 0x7f13ec0051c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f14000073e0 tx=0x7f13ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79574211ba0a13ea server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.314+0000 7f1406a00640 1 -- 172.21.15.19:0/1054986850 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400039070 con 0x7f13ec004d40 2024-03-28T13:14:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.314+0000 7f1406a00640 1 -- 172.21.15.19:0/1054986850 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f140002cd30 con 0x7f13ec004d40 2024-03-28T13:14:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.314+0000 7f1406a00640 1 -- 172.21.15.19:0/1054986850 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400009c80 con 0x7f13ec004d40 2024-03-28T13:14:11.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.315+0000 7f140d600640 1 -- 172.21.15.19:0/1054986850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13ec004d40 msgr2=0x7f13ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.315+0000 7f140d600640 1 --2- 172.21.15.19:0/1054986850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13ec004d40 0x7f13ec0051c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f14000073e0 tx=0x7f13ec005830 comp rx=0 tx=0).stop 2024-03-28T13:14:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.315+0000 7f140cc00640 1 -- 172.21.15.19:0/1054986850 reap_dead start 2024-03-28T13:14:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.315+0000 7f140d600640 1 -- 172.21.15.19:0/1054986850 shutdown_connections 2024-03-28T13:14:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.315+0000 7f140d600640 1 -- 172.21.15.19:0/1054986850 >> 172.21.15.19:0/1054986850 conn(0x7f14080678b0 msgr2=0x7f14080d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.315+0000 7f140d600640 1 -- 172.21.15.19:0/1054986850 shutdown_connections 2024-03-28T13:14:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.315+0000 7f140d600640 1 -- 172.21.15.19:0/1054986850 wait complete. 2024-03-28T13:14:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.317+0000 7f140d600640 1 Processor -- start 2024-03-28T13:14:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.317+0000 7f140d600640 1 -- start start 2024-03-28T13:14:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.318+0000 7f140d600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13ec004d40 0x7f1408154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.318+0000 7f140d600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f140815b410 0x7f14080c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.318+0000 7f140d600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14080c99d0 0x7f14080c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.318+0000 7f140d600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f140815e790 con 0x7f13ec004d40 2024-03-28T13:14:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.318+0000 7f140d600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f14080d29f0 con 0x7f14080c99d0 2024-03-28T13:14:11.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.318+0000 7f140d600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f14080ba030 con 0x7f140815b410 2024-03-28T13:14:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.318+0000 7f1407e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13ec004d40 0x7f1408154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.318+0000 7f1407e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13ec004d40 0x7f1408154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48614/0 (socket says 172.21.15.19:48614) 2024-03-28T13:14:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.318+0000 7f1407e00640 1 -- 172.21.15.19:0/1944171270 learned_addr learned my addr 172.21.15.19:0/1944171270 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.319+0000 7f1407e00640 1 -- 172.21.15.19:0/1944171270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f140815b410 msgr2=0x7f14080c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.319+0000 7f1407e00640 1 --2- 172.21.15.19:0/1944171270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f140815b410 0x7f14080c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.319+0000 7f1407e00640 1 -- 172.21.15.19:0/1944171270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14080c99d0 msgr2=0x7f14080c9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.319+0000 7f1407e00640 1 --2- 172.21.15.19:0/1944171270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14080c99d0 0x7f14080c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.319+0000 7f1407e00640 1 -- 172.21.15.19:0/1944171270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1408154a50 con 0x7f13ec004d40 2024-03-28T13:14:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.319+0000 7f1407e00640 1 --2- 172.21.15.19:0/1944171270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13ec004d40 0x7f1408154510 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f13f800d940 tx=0x7f13f800de00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.319+0000 7f13ffe00640 1 -- 172.21.15.19:0/1944171270 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8019070 con 0x7f13ec004d40 2024-03-28T13:14:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.319+0000 7f13ffe00640 1 -- 172.21.15.19:0/1944171270 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f80074e0 con 0x7f13ec004d40 2024-03-28T13:14:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.319+0000 7f13ffe00640 1 -- 172.21.15.19:0/1944171270 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f80049e0 con 0x7f13ec004d40 2024-03-28T13:14:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.319+0000 7f140d600640 1 -- 172.21.15.19:0/1944171270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14080c9010 con 0x7f13ec004d40 2024-03-28T13:14:11.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.319+0000 7f140d600640 1 -- 172.21.15.19:0/1944171270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14080c95d0 con 0x7f13ec004d40 2024-03-28T13:14:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.320+0000 7f140d600640 1 -- 172.21.15.19:0/1944171270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1408075940 con 0x7f13ec004d40 2024-03-28T13:14:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.321+0000 7f13ffe00640 1 -- 172.21.15.19:0/1944171270 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f13f801c070 con 0x7f13ec004d40 2024-03-28T13:14:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.322+0000 7f13ffe00640 1 --2- 172.21.15.19:0/1944171270 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13e406c530 0x7f13e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.322+0000 7f13ffe00640 1 -- 172.21.15.19:0/1944171270 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f13f808b2c0 con 0x7f13ec004d40 2024-03-28T13:14:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.323+0000 7f1407400640 1 --2- 172.21.15.19:0/1944171270 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13e406c530 0x7f13e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:11.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.324+0000 7f1407400640 1 --2- 172.21.15.19:0/1944171270 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13e406c530 0x7f13e406e9f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f13f0007fd0 tx=0x7f13f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:11.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.327+0000 7f13ffe00640 1 -- 172.21.15.19:0/1944171270 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f13f8059890 con 0x7f13ec004d40 2024-03-28T13:14:11.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.569+0000 7f140d600640 1 -- 172.21.15.19:0/1944171270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14080cefb0 con 0x7f13ec004d40 2024-03-28T13:14:11.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.570+0000 7f13ffe00640 1 -- 172.21.15.19:0/1944171270 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5188 (secure 0 0 0) 0x7f13f8059190 con 0x7f13ec004d40 2024-03-28T13:14:11.570 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:11.570 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1607201514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1607201514},{"type":"v1","addr":"172.21.15.188:6837","nonce":1607201514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1206740614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1206740614},{"type":"v1","addr":"172.21.15.188:6835","nonce":1206740614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2018671345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2018671345},{"type":"v1","addr":"172.21.15.19:6834","nonce":2018671345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:12:52.947573+0000","modified":"2024-03-28T13:13:37.546087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":14,"addr":"172.21.15.19:6835/3341654519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3341654519},{"type":"v1","addr":"172.21.15.19:6835","nonce":3341654519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":4}]} 2024-03-28T13:14:11.571 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 36 2024-03-28T13:14:11.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.573+0000 7f140d600640 1 -- 172.21.15.19:0/1944171270 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13e406c530 msgr2=0x7f13e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:11.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.573+0000 7f140d600640 1 --2- 172.21.15.19:0/1944171270 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13e406c530 0x7f13e406e9f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f13f0007fd0 tx=0x7f13f0009040 comp rx=0 tx=0).stop 2024-03-28T13:14:11.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.573+0000 7f140d600640 1 -- 172.21.15.19:0/1944171270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13ec004d40 msgr2=0x7f1408154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:11.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.573+0000 7f140d600640 1 --2- 172.21.15.19:0/1944171270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13ec004d40 0x7f1408154510 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f13f800d940 tx=0x7f13f800de00 comp rx=0 tx=0).stop 2024-03-28T13:14:11.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.573+0000 7f140d600640 1 -- 172.21.15.19:0/1944171270 shutdown_connections 2024-03-28T13:14:11.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.573+0000 7f140d600640 1 --2- 172.21.15.19:0/1944171270 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f13e406c530 0x7f13e406e9f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:11.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.573+0000 7f140d600640 1 --2- 172.21.15.19:0/1944171270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14080c99d0 0x7f14080c9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:11.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.573+0000 7f140d600640 1 --2- 172.21.15.19:0/1944171270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f140815b410 0x7f14080c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:11.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.573+0000 7f140d600640 1 --2- 172.21.15.19:0/1944171270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13ec004d40 0x7f1408154510 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:11.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.573+0000 7f140d600640 1 -- 172.21.15.19:0/1944171270 >> 172.21.15.19:0/1944171270 conn(0x7f14080678b0 msgr2=0x7f14080d4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:11.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.573+0000 7f140d600640 1 -- 172.21.15.19:0/1944171270 shutdown_connections 2024-03-28T13:14:11.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.573+0000 7f140d600640 1 -- 172.21.15.19:0/1944171270 wait complete. 2024-03-28T13:14:11.593 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:14:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.825+0000 7f58dda00640 1 Processor -- start 2024-03-28T13:14:11.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.825+0000 7f58dda00640 1 -- start start 2024-03-28T13:14:11.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.828+0000 7f58dda00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f58d80d1950 0x7f58d80d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:11.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.828+0000 7f58dda00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58d80d2d60 0x7f58d80cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:11.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.828+0000 7f58dda00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58d815ebc0 con 0x7f58d815e7e0 2024-03-28T13:14:11.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.829+0000 7f58dda00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f58d80d1db0 con 0x7f58d80d2d60 2024-03-28T13:14:11.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.829+0000 7f58dda00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f58d806d1c0 con 0x7f58d80d1950 2024-03-28T13:14:11.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.829+0000 7f58d7400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58d80d2d60 0x7f58d80cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:11.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.829+0000 7f58d6000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f58d80d1950 0x7f58d80d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:11.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.829+0000 7f58d7400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f58d80d2d60 0x7f58d80cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48620/0 (socket says 172.21.15.19:48620) 2024-03-28T13:14:11.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.829+0000 7f58d7400640 1 -- 172.21.15.19:0/2877448544 learned_addr learned my addr 172.21.15.19:0/2877448544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.829+0000 7f58d7400640 1 -- 172.21.15.19:0/2877448544 >> v2:172.21.15.188:3300/0 conn(0x7f58d80d1950 msgr2=0x7f58d80d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.829+0000 7f58d7400640 1 --2- 172.21.15.19:0/2877448544 >> v2:172.21.15.188:3300/0 conn(0x7f58d80d1950 0x7f58d80d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.829+0000 7f58d7400640 1 -- 172.21.15.19:0/2877448544 >> v1:172.21.15.188:6789/0 conn(0x7f58d815e7e0 legacy=0x7f58d806ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.829+0000 7f58d7400640 1 -- 172.21.15.19:0/2877448544 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d806d4a0 con 0x7f58d80d2d60 2024-03-28T13:14:11.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.831+0000 7f58d7400640 1 --2- 172.21.15.19:0/2877448544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80d2d60 0x7f58d80cfb20 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f58d80b6f50 tx=0x7f58d002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61da08795b5e0a97 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:11.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.831+0000 7f58d5600640 1 -- 172.21.15.19:0/2877448544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d002c9f0 con 0x7f58d80d2d60 2024-03-28T13:14:11.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.831+0000 7f58d5600640 1 -- 172.21.15.19:0/2877448544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d002cb90 con 0x7f58d80d2d60 2024-03-28T13:14:11.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.831+0000 7f58d5600640 1 -- 172.21.15.19:0/2877448544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d00343f0 con 0x7f58d80d2d60 2024-03-28T13:14:11.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.832+0000 7f58dda00640 1 -- 172.21.15.19:0/2877448544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80d2d60 msgr2=0x7f58d80cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:11.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.832+0000 7f58dda00640 1 --2- 172.21.15.19:0/2877448544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80d2d60 0x7f58d80cfb20 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f58d80b6f50 tx=0x7f58d002be50 comp rx=0 tx=0).stop 2024-03-28T13:14:11.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.832+0000 7f58dda00640 1 -- 172.21.15.19:0/2877448544 shutdown_connections 2024-03-28T13:14:11.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.832+0000 7f58dda00640 1 --2- 172.21.15.19:0/2877448544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80d2d60 0x7f58d80cfb20 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:11.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.832+0000 7f58dda00640 1 --2- 172.21.15.19:0/2877448544 >> v2:172.21.15.188:3300/0 conn(0x7f58d80d1950 0x7f58d80d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:11.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.832+0000 7f58dda00640 1 -- 172.21.15.19:0/2877448544 >> 172.21.15.19:0/2877448544 conn(0x7f58d809d7c0 msgr2=0x7f58d806bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:11.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.832+0000 7f58dda00640 1 -- 172.21.15.19:0/2877448544 shutdown_connections 2024-03-28T13:14:11.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.832+0000 7f58dda00640 1 -- 172.21.15.19:0/2877448544 wait complete. 2024-03-28T13:14:11.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.834+0000 7f58dda00640 1 Processor -- start 2024-03-28T13:14:11.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.834+0000 7f58dda00640 1 -- start start 2024-03-28T13:14:11.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.835+0000 7f58dda00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58d80d1950 0x7f58d814f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:11.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.835+0000 7f58dda00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80d2d60 0x7f58d80ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:11.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.835+0000 7f58dda00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58d815e7e0 0x7f58d814e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:11.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.835+0000 7f58dda00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58d80cefd0 con 0x7f58d80d2d60 2024-03-28T13:14:11.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.835+0000 7f58dda00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f58d80cf600 con 0x7f58d815e7e0 2024-03-28T13:14:11.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.835+0000 7f58dda00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f58d80d0060 con 0x7f58d80d1950 2024-03-28T13:14:11.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.835+0000 7f58d6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80d2d60 0x7f58d80ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:11.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.836+0000 7f58d6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80d2d60 0x7f58d80ce2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48634/0 (socket says 172.21.15.19:48634) 2024-03-28T13:14:11.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.836+0000 7f58d6000640 1 -- 172.21.15.19:0/4031021592 learned_addr learned my addr 172.21.15.19:0/4031021592 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:11.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.836+0000 7f58d6000640 1 -- 172.21.15.19:0/4031021592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58d80d1950 msgr2=0x7f58d814f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:11.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.836+0000 7f58d6a00640 1 --2- 172.21.15.19:0/4031021592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58d80d1950 0x7f58d814f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:11.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.836+0000 7f58d7400640 1 --2- 172.21.15.19:0/4031021592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58d815e7e0 0x7f58d814e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:11.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.836+0000 7f58d6000640 1 --2- 172.21.15.19:0/4031021592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58d80d1950 0x7f58d814f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:11.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.836+0000 7f58d6000640 1 -- 172.21.15.19:0/4031021592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58d815e7e0 msgr2=0x7f58d814e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:11.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.836+0000 7f58d6000640 1 --2- 172.21.15.19:0/4031021592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58d815e7e0 0x7f58d814e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:11.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.836+0000 7f58d6000640 1 -- 172.21.15.19:0/4031021592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d80cbae0 con 0x7f58d80d2d60 2024-03-28T13:14:11.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.836+0000 7f58d6000640 1 --2- 172.21.15.19:0/4031021592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80d2d60 0x7f58d80ce2e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f58c400e9b0 tx=0x7f58c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:11.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.837+0000 7f58cea00640 1 -- 172.21.15.19:0/4031021592 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4017070 con 0x7f58d80d2d60 2024-03-28T13:14:11.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.837+0000 7f58cea00640 1 -- 172.21.15.19:0/4031021592 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c400c9a0 con 0x7f58d80d2d60 2024-03-28T13:14:11.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.837+0000 7f58dda00640 1 -- 172.21.15.19:0/4031021592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d80b88c0 con 0x7f58d80d2d60 2024-03-28T13:14:11.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.837+0000 7f58cea00640 1 -- 172.21.15.19:0/4031021592 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c400cd90 con 0x7f58d80d2d60 2024-03-28T13:14:11.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.837+0000 7f58dda00640 1 -- 172.21.15.19:0/4031021592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d80b8e80 con 0x7f58d80d2d60 2024-03-28T13:14:11.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.838+0000 7f58dda00640 1 -- 172.21.15.19:0/4031021592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d80b2ec0 con 0x7f58d80d2d60 2024-03-28T13:14:11.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.839+0000 7f58cea00640 1 -- 172.21.15.19:0/4031021592 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f58c401a070 con 0x7f58d80d2d60 2024-03-28T13:14:11.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.841+0000 7f58cea00640 1 --2- 172.21.15.19:0/4031021592 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f58b806c530 0x7f58b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:11.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.841+0000 7f58cea00640 1 -- 172.21.15.19:0/4031021592 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f58c408b740 con 0x7f58d80d2d60 2024-03-28T13:14:11.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.841+0000 7f58d6a00640 1 --2- 172.21.15.19:0/4031021592 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f58b806c530 0x7f58b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:11.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.843+0000 7f58d6a00640 1 --2- 172.21.15.19:0/4031021592 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f58b806c530 0x7f58b806e9f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f58c800a750 tx=0x7f58c8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:11.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:11.845+0000 7f58cea00640 1 -- 172.21.15.19:0/4031021592 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f58d80b2ec0 con 0x7f58d80d2d60 2024-03-28T13:14:12.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.052+0000 7f58dda00640 1 -- 172.21.15.19:0/4031021592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58d80685c0 con 0x7f58d80d2d60 2024-03-28T13:14:12.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.054+0000 7f58cea00640 1 -- 172.21.15.19:0/4031021592 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16761 (secure 0 0 0) 0x7f58c4059d10 con 0x7f58d80d2d60 2024-03-28T13:14:12.055 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:12.055 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:13:24.671229+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:12:50.888331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-28T13:12:51.946012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.14","mappings":[{"from":6,"to":4}]},{"pgid":"9.2c","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:14:12.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.057+0000 7f58dda00640 1 -- 172.21.15.19:0/4031021592 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f58b806c530 msgr2=0x7f58b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:12.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.057+0000 7f58dda00640 1 --2- 172.21.15.19:0/4031021592 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f58b806c530 0x7f58b806e9f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f58c800a750 tx=0x7f58c8008040 comp rx=0 tx=0).stop 2024-03-28T13:14:12.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.057+0000 7f58dda00640 1 -- 172.21.15.19:0/4031021592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80d2d60 msgr2=0x7f58d80ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:12.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.057+0000 7f58dda00640 1 --2- 172.21.15.19:0/4031021592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80d2d60 0x7f58d80ce2e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f58c400e9b0 tx=0x7f58c400ee70 comp rx=0 tx=0).stop 2024-03-28T13:14:12.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.057+0000 7f58dda00640 1 -- 172.21.15.19:0/4031021592 shutdown_connections 2024-03-28T13:14:12.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.057+0000 7f58dda00640 1 --2- 172.21.15.19:0/4031021592 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f58b806c530 0x7f58b806e9f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.057+0000 7f58dda00640 1 --2- 172.21.15.19:0/4031021592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58d815e7e0 0x7f58d814e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.057+0000 7f58dda00640 1 --2- 172.21.15.19:0/4031021592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58d80d2d60 0x7f58d80ce2e0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.057+0000 7f58dda00640 1 --2- 172.21.15.19:0/4031021592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58d80d1950 0x7f58d814f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.057+0000 7f58dda00640 1 -- 172.21.15.19:0/4031021592 >> 172.21.15.19:0/4031021592 conn(0x7f58d809d7c0 msgr2=0x7f58d80b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:12.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.057+0000 7f58dda00640 1 -- 172.21.15.19:0/4031021592 shutdown_connections 2024-03-28T13:14:12.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.057+0000 7f58dda00640 1 -- 172.21.15.19:0/4031021592 wait complete. 2024-03-28T13:14:12.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:12:50.888331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:12:51.946012+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:14:12.078 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:14:12.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.304+0000 7f9ba7e00640 1 Processor -- start 2024-03-28T13:14:12.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.304+0000 7f9ba7e00640 1 -- start start 2024-03-28T13:14:12.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.305+0000 7f9ba7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ba815e460 0x7f9ba80b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.305+0000 7f9ba7e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ba80d2da0 con 0x7f9ba80cef20 2024-03-28T13:14:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.305+0000 7f9ba7e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ba80d3080 con 0x7f9ba80d3e40 2024-03-28T13:14:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.305+0000 7f9ba7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba80d3360 con 0x7f9ba815e460 2024-03-28T13:14:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.306+0000 7f9ba7400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9ba80cef20 0x7f9ba80b7310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51112/0 (socket says 172.21.15.19:51112) 2024-03-28T13:14:12.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.306+0000 7f9ba7400640 1 -- 172.21.15.19:0/1177339439 learned_addr learned my addr 172.21.15.19:0/1177339439 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:12.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.306+0000 7f9ba6a00640 1 --2- 172.21.15.19:0/1177339439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ba815e460 0x7f9ba80b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:12.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.306+0000 7f9ba6000640 1 -- 172.21.15.19:0/1177339439 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3741404479 0 0) 0x7f9ba80d2da0 con 0x7f9ba80cef20 2024-03-28T13:14:12.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.306+0000 7f9ba6000640 1 -- 172.21.15.19:0/1177339439 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b8c003520 con 0x7f9ba80cef20 2024-03-28T13:14:12.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.307+0000 7f9ba6000640 1 -- 172.21.15.19:0/1177339439 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2553753505 0 0) 0x7f9b8c003520 con 0x7f9ba80cef20 2024-03-28T13:14:12.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.307+0000 7f9ba6000640 1 -- 172.21.15.19:0/1177339439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ba815e460 msgr2=0x7f9ba80b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:12.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.307+0000 7f9ba6000640 1 --2- 172.21.15.19:0/1177339439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ba815e460 0x7f9ba80b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.307+0000 7f9ba6000640 1 -- 172.21.15.19:0/1177339439 >> v1:172.21.15.188:6789/0 conn(0x7f9ba80d3e40 legacy=0x7f9ba80d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:12.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.307+0000 7f9ba6000640 1 -- 172.21.15.19:0/1177339439 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba80d3640 con 0x7f9ba80cef20 2024-03-28T13:14:12.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.307+0000 7f9ba6000640 1 -- 172.21.15.19:0/1177339439 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f9ba00019f0 con 0x7f9ba80cef20 2024-03-28T13:14:12.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.307+0000 7f9ba6000640 1 -- 172.21.15.19:0/1177339439 >> v1:172.21.15.19:6789/0 conn(0x7f9ba80cef20 legacy=0x7f9ba80b7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:12.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.307+0000 7f9ba6000640 1 --2- 172.21.15.19:0/1177339439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b8c0039e0 0x7f9b8c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:12.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.307+0000 7f9ba6000640 1 --2- 172.21.15.19:0/1177339439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c004800 0x7f9b8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:12.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.307+0000 7f9ba6000640 1 --2- 172.21.15.19:0/1177339439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ba815e460 0x7f9b8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:12.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.307+0000 7f9ba6000640 1 -- 172.21.15.19:0/1177339439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba80d3360 con 0x7f9b8c004800 2024-03-28T13:14:12.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.307+0000 7f9ba6000640 1 -- 172.21.15.19:0/1177339439 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c00baa0 con 0x7f9b8c0039e0 2024-03-28T13:14:12.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.307+0000 7f9ba6000640 1 -- 172.21.15.19:0/1177339439 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c00bca0 con 0x7f9ba815e460 2024-03-28T13:14:12.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.307+0000 7f9b9fe00640 1 --2- 172.21.15.19:0/1177339439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c004800 0x7f9b8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:12.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.308+0000 7f9ba7400640 1 --2- 172.21.15.19:0/1177339439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ba815e460 0x7f9b8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:12.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.308+0000 7f9ba6a00640 1 --2- 172.21.15.19:0/1177339439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b8c0039e0 0x7f9b8c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:12.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.308+0000 7f9b9fe00640 1 -- 172.21.15.19:0/1177339439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ba815e460 msgr2=0x7f9b8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:12.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.308+0000 7f9b9fe00640 1 --2- 172.21.15.19:0/1177339439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ba815e460 0x7f9b8c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.308+0000 7f9b9fe00640 1 -- 172.21.15.19:0/1177339439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b8c0039e0 msgr2=0x7f9b8c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:12.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.308+0000 7f9b9fe00640 1 --2- 172.21.15.19:0/1177339439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9b8c0039e0 0x7f9b8c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.308+0000 7f9b9fe00640 1 -- 172.21.15.19:0/1177339439 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c003520 con 0x7f9b8c004800 2024-03-28T13:14:12.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.309+0000 7f9b9fe00640 1 --2- 172.21.15.19:0/1177339439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c004800 0x7f9b8c004c80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9b90007fd0 tx=0x7f9b8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18289b7c434e01ec server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:12.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.309+0000 7f9ba6000640 1 -- 172.21.15.19:0/1177339439 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b900013c0 con 0x7f9b8c004800 2024-03-28T13:14:12.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.309+0000 7f9ba6000640 1 -- 172.21.15.19:0/1177339439 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b900052d0 con 0x7f9b8c004800 2024-03-28T13:14:12.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.309+0000 7f9ba6000640 1 -- 172.21.15.19:0/1177339439 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b90003640 con 0x7f9b8c004800 2024-03-28T13:14:12.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.310+0000 7f9ba7e00640 1 -- 172.21.15.19:0/1177339439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c004800 msgr2=0x7f9b8c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:12.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.310+0000 7f9ba7e00640 1 --2- 172.21.15.19:0/1177339439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c004800 0x7f9b8c004c80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9b90007fd0 tx=0x7f9b8c005830 comp rx=0 tx=0).stop 2024-03-28T13:14:12.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.310+0000 7f9ba7400640 1 -- 172.21.15.19:0/1177339439 reap_dead start 2024-03-28T13:14:12.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.310+0000 7f9ba7e00640 1 -- 172.21.15.19:0/1177339439 shutdown_connections 2024-03-28T13:14:12.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.310+0000 7f9ba7e00640 1 -- 172.21.15.19:0/1177339439 >> 172.21.15.19:0/1177339439 conn(0x7f9ba80678b0 msgr2=0x7f9ba80d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:12.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.310+0000 7f9ba7e00640 1 -- 172.21.15.19:0/1177339439 shutdown_connections 2024-03-28T13:14:12.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.310+0000 7f9ba7e00640 1 -- 172.21.15.19:0/1177339439 wait complete. 2024-03-28T13:14:12.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.313+0000 7f9ba7e00640 1 Processor -- start 2024-03-28T13:14:12.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.313+0000 7f9ba7e00640 1 -- start start 2024-03-28T13:14:12.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.314+0000 7f9ba7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c004800 0x7f9ba815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:12.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.314+0000 7f9ba7e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ba80b4af0 0x7f9ba80b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:12.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.314+0000 7f9ba7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ba8148800 0x7f9ba8148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:12.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.315+0000 7f9ba7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba80b9fa0 con 0x7f9b8c004800 2024-03-28T13:14:12.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.315+0000 7f9ba7e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba80b9e20 con 0x7f9ba80b4af0 2024-03-28T13:14:12.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.315+0000 7f9ba7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba80ba120 con 0x7f9ba8148800 2024-03-28T13:14:12.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.315+0000 7f9ba6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c004800 0x7f9ba815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:12.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.315+0000 7f9ba6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c004800 0x7f9ba815cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48648/0 (socket says 172.21.15.19:48648) 2024-03-28T13:14:12.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.315+0000 7f9ba6a00640 1 -- 172.21.15.19:0/3971293010 learned_addr learned my addr 172.21.15.19:0/3971293010 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:12.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.315+0000 7f9ba6a00640 1 -- 172.21.15.19:0/3971293010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ba8148800 msgr2=0x7f9ba8148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:12.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.315+0000 7f9ba6a00640 1 --2- 172.21.15.19:0/3971293010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ba8148800 0x7f9ba8148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.315+0000 7f9ba6a00640 1 -- 172.21.15.19:0/3971293010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ba80b4af0 msgr2=0x7f9ba80b4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:12.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.315+0000 7f9b9fe00640 1 --2- 172.21.15.19:0/3971293010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ba80b4af0 0x7f9ba80b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:12.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.315+0000 7f9ba6a00640 1 --2- 172.21.15.19:0/3971293010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ba80b4af0 0x7f9ba80b4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.315+0000 7f9ba6a00640 1 -- 172.21.15.19:0/3971293010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba8149410 con 0x7f9b8c004800 2024-03-28T13:14:12.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.315+0000 7f9ba6a00640 1 --2- 172.21.15.19:0/3971293010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c004800 0x7f9ba815cb80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9b98011630 tx=0x7f9b98011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:12.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.316+0000 7f9b9f400640 1 -- 172.21.15.19:0/3971293010 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98015870 con 0x7f9b8c004800 2024-03-28T13:14:12.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.316+0000 7f9b9f400640 1 -- 172.21.15.19:0/3971293010 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b98015a10 con 0x7f9b8c004800 2024-03-28T13:14:12.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.316+0000 7f9b9f400640 1 -- 172.21.15.19:0/3971293010 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98015dc0 con 0x7f9b8c004800 2024-03-28T13:14:12.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.316+0000 7f9ba7e00640 1 -- 172.21.15.19:0/3971293010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba81496f0 con 0x7f9b8c004800 2024-03-28T13:14:12.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.316+0000 7f9ba7e00640 1 -- 172.21.15.19:0/3971293010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba8149cb0 con 0x7f9b8c004800 2024-03-28T13:14:12.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.317+0000 7f9ba7e00640 1 -- 172.21.15.19:0/3971293010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba8075940 con 0x7f9b8c004800 2024-03-28T13:14:12.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.318+0000 7f9b9f400640 1 -- 172.21.15.19:0/3971293010 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9b9800f450 con 0x7f9b8c004800 2024-03-28T13:14:12.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.320+0000 7f9b9f400640 1 --2- 172.21.15.19:0/3971293010 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9b8406c560 0x7f9b8406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:12.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.320+0000 7f9b9f400640 1 -- 172.21.15.19:0/3971293010 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f9b9808c0b0 con 0x7f9b8c004800 2024-03-28T13:14:12.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.320+0000 7f9b9fe00640 1 --2- 172.21.15.19:0/3971293010 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9b8406c560 0x7f9b8406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:12.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.321+0000 7f9b9fe00640 1 --2- 172.21.15.19:0/3971293010 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9b8406c560 0x7f9b8406ea20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9b90033000 tx=0x7f9b90009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.323+0000 7f9b9f400640 1 -- 172.21.15.19:0/3971293010 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9b98019030 con 0x7f9b8c004800 2024-03-28T13:14:12.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.553+0000 7f9ba7e00640 1 -- 172.21.15.19:0/3971293010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9ba80ca5a0 con 0x7f9b8c004800 2024-03-28T13:14:12.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.554+0000 7f9b9f400640 1 -- 172.21.15.19:0/3971293010 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) ==== 93+0+10 (secure 0 0 0) 0x7f9b9805a5f0 con 0x7f9b8c004800 2024-03-28T13:14:12.554 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:14:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.557+0000 7f9ba7e00640 1 -- 172.21.15.19:0/3971293010 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9b8406c560 msgr2=0x7f9b8406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.557+0000 7f9ba7e00640 1 --2- 172.21.15.19:0/3971293010 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9b8406c560 0x7f9b8406ea20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9b90033000 tx=0x7f9b90009030 comp rx=0 tx=0).stop 2024-03-28T13:14:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.557+0000 7f9ba7e00640 1 -- 172.21.15.19:0/3971293010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c004800 msgr2=0x7f9ba815cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.557+0000 7f9ba7e00640 1 --2- 172.21.15.19:0/3971293010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c004800 0x7f9ba815cb80 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9b98011630 tx=0x7f9b98011af0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.557+0000 7f9ba7e00640 1 -- 172.21.15.19:0/3971293010 shutdown_connections 2024-03-28T13:14:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.557+0000 7f9ba7e00640 1 --2- 172.21.15.19:0/3971293010 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9b8406c560 0x7f9b8406ea20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.557+0000 7f9ba7e00640 1 --2- 172.21.15.19:0/3971293010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ba8148800 0x7f9ba8148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.557+0000 7f9ba7e00640 1 --2- 172.21.15.19:0/3971293010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ba80b4af0 0x7f9ba80b4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.557+0000 7f9ba7e00640 1 --2- 172.21.15.19:0/3971293010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9b8c004800 0x7f9ba815cb80 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.557+0000 7f9ba7e00640 1 -- 172.21.15.19:0/3971293010 >> 172.21.15.19:0/3971293010 conn(0x7f9ba80678b0 msgr2=0x7f9ba80c9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:12.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.557+0000 7f9ba7e00640 1 -- 172.21.15.19:0/3971293010 shutdown_connections 2024-03-28T13:14:12.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.557+0000 7f9ba7e00640 1 -- 172.21.15.19:0/3971293010 wait complete. 2024-03-28T13:14:12.575 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:14:12.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.788+0000 7f2a75200640 1 Processor -- start 2024-03-28T13:14:12.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.789+0000 7f2a75200640 1 -- start start 2024-03-28T13:14:12.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.790+0000 7f2a75200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2a7006ea20 0x7f2a7006bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:12.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.790+0000 7f2a75200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a7015ed90 con 0x7f2a700afd50 2024-03-28T13:14:12.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.791+0000 7f2a75200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a7015f070 con 0x7f2a700b3e70 2024-03-28T13:14:12.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.791+0000 7f2a75200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2a7015f350 con 0x7f2a7006ea20 2024-03-28T13:14:12.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.791+0000 7f2a6fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2a7006ea20 0x7f2a7006bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:12.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.791+0000 7f2a6f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2a700afd50 0x7f2a700b3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51124/0 (socket says 172.21.15.19:51124) 2024-03-28T13:14:12.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.791+0000 7f2a6f400640 1 -- 172.21.15.19:0/2670809633 learned_addr learned my addr 172.21.15.19:0/2670809633 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:12.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.792+0000 7f2a6e000640 1 -- 172.21.15.19:0/2670809633 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1054083059 0 0) 0x7f2a7015ed90 con 0x7f2a700afd50 2024-03-28T13:14:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.792+0000 7f2a6fe00640 1 -- 172.21.15.19:0/2670809633 >> v1:172.21.15.188:6789/0 conn(0x7f2a700b3e70 legacy=0x7f2a7006b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.792+0000 7f2a6fe00640 1 -- 172.21.15.19:0/2670809633 >> v1:172.21.15.19:6789/0 conn(0x7f2a700afd50 legacy=0x7f2a700b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.792+0000 7f2a6fe00640 1 -- 172.21.15.19:0/2670809633 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7015f630 con 0x7f2a7006ea20 2024-03-28T13:14:12.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.793+0000 7f2a6fe00640 1 --2- 172.21.15.19:0/2670809633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a7006ea20 0x7f2a7006bfb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009d90 tx=0x7f2a6802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d43640e91d01e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.793+0000 7f2a6e000640 1 -- 172.21.15.19:0/2670809633 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68039070 con 0x7f2a7006ea20 2024-03-28T13:14:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.793+0000 7f2a6e000640 1 -- 172.21.15.19:0/2670809633 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6802ccf0 con 0x7f2a7006ea20 2024-03-28T13:14:12.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.793+0000 7f2a6e000640 1 -- 172.21.15.19:0/2670809633 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68034660 con 0x7f2a7006ea20 2024-03-28T13:14:12.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.794+0000 7f2a75200640 1 -- 172.21.15.19:0/2670809633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a7006ea20 msgr2=0x7f2a7006bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:12.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.794+0000 7f2a75200640 1 --2- 172.21.15.19:0/2670809633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a7006ea20 0x7f2a7006bfb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2a68009d90 tx=0x7f2a6802bf40 comp rx=0 tx=0).stop 2024-03-28T13:14:12.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.794+0000 7f2a75200640 1 -- 172.21.15.19:0/2670809633 shutdown_connections 2024-03-28T13:14:12.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.794+0000 7f2a75200640 1 --2- 172.21.15.19:0/2670809633 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a7006ea20 0x7f2a7006bfb0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.794+0000 7f2a75200640 1 -- 172.21.15.19:0/2670809633 >> 172.21.15.19:0/2670809633 conn(0x7f2a700d0770 msgr2=0x7f2a700d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:12.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.794+0000 7f2a75200640 1 -- 172.21.15.19:0/2670809633 shutdown_connections 2024-03-28T13:14:12.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.794+0000 7f2a75200640 1 -- 172.21.15.19:0/2670809633 wait complete. 2024-03-28T13:14:12.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.795+0000 7f2a75200640 1 Processor -- start 2024-03-28T13:14:12.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.796+0000 7f2a75200640 1 -- start start 2024-03-28T13:14:12.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a75200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7006ea20 0x7f2a70152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:12.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a75200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a700afd50 0x7f2a7014f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:12.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a75200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a700b3e70 0x7f2a700cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:12.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a75200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7006fa20 con 0x7f2a700afd50 2024-03-28T13:14:12.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a75200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a700b0130 con 0x7f2a700b3e70 2024-03-28T13:14:12.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a75200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7006fba0 con 0x7f2a7006ea20 2024-03-28T13:14:12.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a6ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a700afd50 0x7f2a7014f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:12.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a6ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a700afd50 0x7f2a7014f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48672/0 (socket says 172.21.15.19:48672) 2024-03-28T13:14:12.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a6ea00640 1 -- 172.21.15.19:0/2046347713 learned_addr learned my addr 172.21.15.19:0/2046347713 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:12.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a6f400640 1 --2- 172.21.15.19:0/2046347713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7006ea20 0x7f2a70152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:12.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a6fe00640 1 --2- 172.21.15.19:0/2046347713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a700b3e70 0x7f2a700cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:12.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a6ea00640 1 -- 172.21.15.19:0/2046347713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7006ea20 msgr2=0x7f2a70152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:12.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a6ea00640 1 --2- 172.21.15.19:0/2046347713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7006ea20 0x7f2a70152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a6ea00640 1 -- 172.21.15.19:0/2046347713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a700b3e70 msgr2=0x7f2a700cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:12.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a6ea00640 1 --2- 172.21.15.19:0/2046347713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a700b3e70 0x7f2a700cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:12.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a6ea00640 1 -- 172.21.15.19:0/2046347713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a70156c00 con 0x7f2a700afd50 2024-03-28T13:14:12.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.797+0000 7f2a6fe00640 1 --2- 172.21.15.19:0/2046347713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a700b3e70 0x7f2a700cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:12.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.798+0000 7f2a6ea00640 1 --2- 172.21.15.19:0/2046347713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a700afd50 0x7f2a7014f760 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00c400 tx=0x7f2a5c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:12.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.798+0000 7f2a67400640 1 -- 172.21.15.19:0/2046347713 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c017070 con 0x7f2a700afd50 2024-03-28T13:14:12.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.798+0000 7f2a67400640 1 -- 172.21.15.19:0/2046347713 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a5c0052d0 con 0x7f2a700afd50 2024-03-28T13:14:12.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.798+0000 7f2a67400640 1 -- 172.21.15.19:0/2046347713 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c003830 con 0x7f2a700afd50 2024-03-28T13:14:12.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.798+0000 7f2a75200640 1 -- 172.21.15.19:0/2046347713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a700ce1b0 con 0x7f2a700afd50 2024-03-28T13:14:12.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.798+0000 7f2a75200640 1 -- 172.21.15.19:0/2046347713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a700ce770 con 0x7f2a700afd50 2024-03-28T13:14:12.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.800+0000 7f2a75200640 1 -- 172.21.15.19:0/2046347713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a70075940 con 0x7f2a700afd50 2024-03-28T13:14:12.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.801+0000 7f2a67400640 1 -- 172.21.15.19:0/2046347713 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2a5c01a070 con 0x7f2a700afd50 2024-03-28T13:14:12.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.802+0000 7f2a67400640 1 --2- 172.21.15.19:0/2046347713 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a5006c530 0x7f2a5006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:12.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.802+0000 7f2a67400640 1 -- 172.21.15.19:0/2046347713 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f2a5c08aab0 con 0x7f2a700afd50 2024-03-28T13:14:12.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.802+0000 7f2a6f400640 1 --2- 172.21.15.19:0/2046347713 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a5006c530 0x7f2a5006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:12.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.803+0000 7f2a6f400640 1 --2- 172.21.15.19:0/2046347713 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a5006c530 0x7f2a5006e9f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2a60000f60 tx=0x7f2a60007b90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:12.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:12.811+0000 7f2a67400640 1 -- 172.21.15.19:0/2046347713 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2a70156c00 con 0x7f2a700afd50 2024-03-28T13:14:13.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.015+0000 7f2a75200640 1 -- 172.21.15.19:0/2046347713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2a70067590 con 0x7f2a700afd50 2024-03-28T13:14:13.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.016+0000 7f2a67400640 1 -- 172.21.15.19:0/2046347713 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) ==== 104+0+11 (secure 0 0 0) 0x7f2a5c059090 con 0x7f2a700afd50 2024-03-28T13:14:13.016 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:14:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.019+0000 7f2a75200640 1 -- 172.21.15.19:0/2046347713 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a5006c530 msgr2=0x7f2a5006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.019+0000 7f2a75200640 1 --2- 172.21.15.19:0/2046347713 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a5006c530 0x7f2a5006e9f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2a60000f60 tx=0x7f2a60007b90 comp rx=0 tx=0).stop 2024-03-28T13:14:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.019+0000 7f2a75200640 1 -- 172.21.15.19:0/2046347713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a700afd50 msgr2=0x7f2a7014f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.019+0000 7f2a75200640 1 --2- 172.21.15.19:0/2046347713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a700afd50 0x7f2a7014f760 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00c400 tx=0x7f2a5c00c8c0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.019+0000 7f2a75200640 1 -- 172.21.15.19:0/2046347713 shutdown_connections 2024-03-28T13:14:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.019+0000 7f2a75200640 1 --2- 172.21.15.19:0/2046347713 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a5006c530 0x7f2a5006e9f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.019+0000 7f2a75200640 1 --2- 172.21.15.19:0/2046347713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a700b3e70 0x7f2a700cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.019+0000 7f2a75200640 1 --2- 172.21.15.19:0/2046347713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a700afd50 0x7f2a7014f760 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.019+0000 7f2a75200640 1 --2- 172.21.15.19:0/2046347713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a7006ea20 0x7f2a70152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.019+0000 7f2a75200640 1 -- 172.21.15.19:0/2046347713 >> 172.21.15.19:0/2046347713 conn(0x7f2a700d0770 msgr2=0x7f2a7006e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.019+0000 7f2a75200640 1 -- 172.21.15.19:0/2046347713 shutdown_connections 2024-03-28T13:14:13.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.019+0000 7f2a75200640 1 -- 172.21.15.19:0/2046347713 wait complete. 2024-03-28T13:14:13.036 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:14:13.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.245+0000 7fb5e7e00640 1 Processor -- start 2024-03-28T13:14:13.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.246+0000 7fb5e7e00640 1 -- start start 2024-03-28T13:14:13.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.248+0000 7fb5e7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb5e815e460 0x7fb5e80b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:13.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.249+0000 7fb5e7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80d3e40 0x7fb5e80d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:13.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.249+0000 7fb5e7e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb5e80cef20 0x7fb5e80cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:13.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.249+0000 7fb5e7e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb5e80cf8b0 con 0x7fb5e815e460 2024-03-28T13:14:13.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.249+0000 7fb5e7e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb5e80d2da0 con 0x7fb5e80cef20 2024-03-28T13:14:13.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.249+0000 7fb5e7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e80d3080 con 0x7fb5e80d3e40 2024-03-28T13:14:13.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.249+0000 7fb5e6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb5e815e460 0x7fb5e80b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:13.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.249+0000 7fb5e6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb5e815e460 0x7fb5e80b9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48676/0 (socket says 172.21.15.19:48676) 2024-03-28T13:14:13.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.249+0000 7fb5e6a00640 1 -- 172.21.15.19:0/2600980507 learned_addr learned my addr 172.21.15.19:0/2600980507 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:13.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.249+0000 7fb5dfe00640 1 --2- 172.21.15.19:0/2600980507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80d3e40 0x7fb5e80d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:13.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.249+0000 7fb5e6a00640 1 -- 172.21.15.19:0/2600980507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80d3e40 msgr2=0x7fb5e80d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:13.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.249+0000 7fb5e6a00640 1 --2- 172.21.15.19:0/2600980507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80d3e40 0x7fb5e80d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.250+0000 7fb5e6a00640 1 -- 172.21.15.19:0/2600980507 >> v2:172.21.15.188:3300/0 conn(0x7fb5e80cef20 msgr2=0x7fb5e80cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:13.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.249+0000 7fb5e7400640 1 --2- 172.21.15.19:0/2600980507 >> v2:172.21.15.188:3300/0 conn(0x7fb5e80cef20 0x7fb5e80cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:13.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.250+0000 7fb5e6a00640 1 --2- 172.21.15.19:0/2600980507 >> v2:172.21.15.188:3300/0 conn(0x7fb5e80cef20 0x7fb5e80cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.250+0000 7fb5e6a00640 1 -- 172.21.15.19:0/2600980507 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e80d3360 con 0x7fb5e815e460 2024-03-28T13:14:13.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.250+0000 7fb5e7400640 1 --2- 172.21.15.19:0/2600980507 >> v2:172.21.15.188:3300/0 conn(0x7fb5e80cef20 0x7fb5e80cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:13.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.251+0000 7fb5e6a00640 1 --2- 172.21.15.19:0/2600980507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5e815e460 0x7fb5e80b9e20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80cb2b0 tx=0x7fb5d802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ead7b47a47718c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:13.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.251+0000 7fb5e6000640 1 -- 172.21.15.19:0/2600980507 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8035020 con 0x7fb5e815e460 2024-03-28T13:14:13.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.251+0000 7fb5e6000640 1 -- 172.21.15.19:0/2600980507 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d802d040 con 0x7fb5e815e460 2024-03-28T13:14:13.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.251+0000 7fb5e6000640 1 -- 172.21.15.19:0/2600980507 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8031a80 con 0x7fb5e815e460 2024-03-28T13:14:13.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.252+0000 7fb5e7e00640 1 -- 172.21.15.19:0/2600980507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5e815e460 msgr2=0x7fb5e80b9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:13.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.252+0000 7fb5e7e00640 1 --2- 172.21.15.19:0/2600980507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5e815e460 0x7fb5e80b9e20 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80cb2b0 tx=0x7fb5d802b900 comp rx=0 tx=0).stop 2024-03-28T13:14:13.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.252+0000 7fb5e7e00640 1 -- 172.21.15.19:0/2600980507 shutdown_connections 2024-03-28T13:14:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.252+0000 7fb5e7e00640 1 --2- 172.21.15.19:0/2600980507 >> v2:172.21.15.188:3300/0 conn(0x7fb5e80cef20 0x7fb5e80cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.252+0000 7fb5e7e00640 1 --2- 172.21.15.19:0/2600980507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80d3e40 0x7fb5e80d2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.252+0000 7fb5e7e00640 1 --2- 172.21.15.19:0/2600980507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5e815e460 0x7fb5e80b9e20 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.252+0000 7fb5e7e00640 1 -- 172.21.15.19:0/2600980507 >> 172.21.15.19:0/2600980507 conn(0x7fb5e80678b0 msgr2=0x7fb5e80d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.252+0000 7fb5e7e00640 1 -- 172.21.15.19:0/2600980507 shutdown_connections 2024-03-28T13:14:13.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.252+0000 7fb5e7e00640 1 -- 172.21.15.19:0/2600980507 wait complete. 2024-03-28T13:14:13.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.254+0000 7fb5e7e00640 1 Processor -- start 2024-03-28T13:14:13.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.254+0000 7fb5e7e00640 1 -- start start 2024-03-28T13:14:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.256+0000 7fb5e7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5e80cef20 0x7fb5e81e4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.256+0000 7fb5e7e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e80d3e40 0x7fb5e815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:13.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.256+0000 7fb5e7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e815e460 0x7fb5e81e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.256+0000 7fb5e7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e80d2a90 con 0x7fb5e80cef20 2024-03-28T13:14:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.256+0000 7fb5e7e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e80d3920 con 0x7fb5e80d3e40 2024-03-28T13:14:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.256+0000 7fb5e7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e806c610 con 0x7fb5e815e460 2024-03-28T13:14:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.256+0000 7fb5e6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5e80cef20 0x7fb5e81e4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.256+0000 7fb5e6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5e80cef20 0x7fb5e81e4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48692/0 (socket says 172.21.15.19:48692) 2024-03-28T13:14:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.256+0000 7fb5e6a00640 1 -- 172.21.15.19:0/3609181372 learned_addr learned my addr 172.21.15.19:0/3609181372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.257+0000 7fb5dfe00640 1 --2- 172.21.15.19:0/3609181372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e80d3e40 0x7fb5e815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.257+0000 7fb5e7400640 1 --2- 172.21.15.19:0/3609181372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e815e460 0x7fb5e81e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.257+0000 7fb5e6a00640 1 -- 172.21.15.19:0/3609181372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e815e460 msgr2=0x7fb5e81e6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.257+0000 7fb5e6a00640 1 --2- 172.21.15.19:0/3609181372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e815e460 0x7fb5e81e6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.257+0000 7fb5e6a00640 1 -- 172.21.15.19:0/3609181372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e80d3e40 msgr2=0x7fb5e815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.257+0000 7fb5e6a00640 1 --2- 172.21.15.19:0/3609181372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e80d3e40 0x7fb5e815a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.257+0000 7fb5e6a00640 1 -- 172.21.15.19:0/3609181372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e81e6540 con 0x7fb5e80cef20 2024-03-28T13:14:13.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.257+0000 7fb5e6a00640 1 --2- 172.21.15.19:0/3609181372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5e80cef20 0x7fb5e81e4280 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb5d802b940 tx=0x7fb5d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:13.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.257+0000 7fb5dea00640 1 -- 172.21.15.19:0/3609181372 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8040070 con 0x7fb5e80cef20 2024-03-28T13:14:13.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.257+0000 7fb5dea00640 1 -- 172.21.15.19:0/3609181372 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5d8039c30 con 0x7fb5e80cef20 2024-03-28T13:14:13.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.257+0000 7fb5dea00640 1 -- 172.21.15.19:0/3609181372 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8035040 con 0x7fb5e80cef20 2024-03-28T13:14:13.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.257+0000 7fb5e7e00640 1 -- 172.21.15.19:0/3609181372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e81e4b20 con 0x7fb5e80cef20 2024-03-28T13:14:13.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.258+0000 7fb5e7e00640 1 -- 172.21.15.19:0/3609181372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e81e4e00 con 0x7fb5e80cef20 2024-03-28T13:14:13.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.259+0000 7fb5e7e00640 1 -- 172.21.15.19:0/3609181372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e8075940 con 0x7fb5e80cef20 2024-03-28T13:14:13.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.261+0000 7fb5dea00640 1 -- 172.21.15.19:0/3609181372 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb5d804b060 con 0x7fb5e80cef20 2024-03-28T13:14:13.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.261+0000 7fb5dea00640 1 --2- 172.21.15.19:0/3609181372 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb5c406c530 0x7fb5c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:13.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.262+0000 7fb5dea00640 1 -- 172.21.15.19:0/3609181372 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7fb5d80ad4c0 con 0x7fb5e80cef20 2024-03-28T13:14:13.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.262+0000 7fb5dfe00640 1 --2- 172.21.15.19:0/3609181372 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb5c406c530 0x7fb5c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:13.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.264+0000 7fb5dfe00640 1 --2- 172.21.15.19:0/3609181372 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb5c406c530 0x7fb5c406e9f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80cada0 tx=0x7fb5d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:13.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.267+0000 7fb5dea00640 1 -- 172.21.15.19:0/3609181372 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb5d807ba90 con 0x7fb5e80cef20 2024-03-28T13:14:13.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.492+0000 7fb5e7e00640 1 -- 172.21.15.19:0/3609181372 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb5e81e6310 con 0x7fb5e80cef20 2024-03-28T13:14:13.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.493+0000 7fb5dea00640 1 -- 172.21.15.19:0/3609181372 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) ==== 100+0+11 (secure 0 0 0) 0x7fb5d807b390 con 0x7fb5e80cef20 2024-03-28T13:14:13.493 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:14:13.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.495+0000 7fb5e7e00640 1 -- 172.21.15.19:0/3609181372 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb5c406c530 msgr2=0x7fb5c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:13.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.495+0000 7fb5e7e00640 1 --2- 172.21.15.19:0/3609181372 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb5c406c530 0x7fb5c406e9f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80cada0 tx=0x7fb5d0008040 comp rx=0 tx=0).stop 2024-03-28T13:14:13.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.495+0000 7fb5e7e00640 1 -- 172.21.15.19:0/3609181372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5e80cef20 msgr2=0x7fb5e81e4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:13.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.495+0000 7fb5e7e00640 1 --2- 172.21.15.19:0/3609181372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5e80cef20 0x7fb5e81e4280 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb5d802b940 tx=0x7fb5d80046c0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.495+0000 7fb5e7e00640 1 -- 172.21.15.19:0/3609181372 shutdown_connections 2024-03-28T13:14:13.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.495+0000 7fb5e7e00640 1 --2- 172.21.15.19:0/3609181372 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb5c406c530 0x7fb5c406e9f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.495+0000 7fb5e7e00640 1 --2- 172.21.15.19:0/3609181372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e815e460 0x7fb5e81e6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.495+0000 7fb5e7e00640 1 --2- 172.21.15.19:0/3609181372 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e80d3e40 0x7fb5e815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.495+0000 7fb5e7e00640 1 --2- 172.21.15.19:0/3609181372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5e80cef20 0x7fb5e81e4280 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.495+0000 7fb5e7e00640 1 -- 172.21.15.19:0/3609181372 >> 172.21.15.19:0/3609181372 conn(0x7fb5e80678b0 msgr2=0x7fb5e806ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:13.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.496+0000 7fb5e7e00640 1 -- 172.21.15.19:0/3609181372 shutdown_connections 2024-03-28T13:14:13.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.496+0000 7fb5e7e00640 1 -- 172.21.15.19:0/3609181372 wait complete. 2024-03-28T13:14:13.515 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:14:13.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.738+0000 7f23e1c00640 1 Processor -- start 2024-03-28T13:14:13.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.739+0000 7f23e1c00640 1 -- start start 2024-03-28T13:14:13.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.740+0000 7f23e1c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f23dc15c390 0x7f23dc073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:13.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.740+0000 7f23e1c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f23dc0af830 0x7f23dc160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:13.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.740+0000 7f23e1c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23dc15fe80 con 0x7f23dc160b80 2024-03-28T13:14:13.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.740+0000 7f23e1c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f23dc15c770 con 0x7f23dc15c390 2024-03-28T13:14:13.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.740+0000 7f23e1c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f23dc0ad120 con 0x7f23dc0af830 2024-03-28T13:14:13.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.740+0000 7f23daa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f23dc15c390 0x7f23dc073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:13.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.740+0000 7f23db400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f23dc160b80 0x7f23dc0d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51130/0 (socket says 172.21.15.19:51130) 2024-03-28T13:14:13.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.740+0000 7f23db400640 1 -- 172.21.15.19:0/3187128539 learned_addr learned my addr 172.21.15.19:0/3187128539 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:13.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.741+0000 7f23d2000640 1 --2- 172.21.15.19:0/3187128539 >> v2:172.21.15.188:3300/0 conn(0x7f23dc0af830 0x7f23dc160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:13.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.741+0000 7f23daa00640 1 -- 172.21.15.19:0/3187128539 >> v2:172.21.15.188:3300/0 conn(0x7f23dc0af830 msgr2=0x7f23dc160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:13.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.741+0000 7f23daa00640 1 --2- 172.21.15.19:0/3187128539 >> v2:172.21.15.188:3300/0 conn(0x7f23dc0af830 0x7f23dc160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.741+0000 7f23daa00640 1 -- 172.21.15.19:0/3187128539 >> v1:172.21.15.19:6789/0 conn(0x7f23dc160b80 legacy=0x7f23dc0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:13.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.741+0000 7f23daa00640 1 -- 172.21.15.19:0/3187128539 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc0ad400 con 0x7f23dc15c390 2024-03-28T13:14:13.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.742+0000 7f23daa00640 1 --2- 172.21.15.19:0/3187128539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23dc15c390 0x7f23dc073790 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f23dc06eaa0 tx=0x7f23cc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196c092021e6a0a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:13.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.742+0000 7f23da000640 1 -- 172.21.15.19:0/3187128539 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc035020 con 0x7f23dc15c390 2024-03-28T13:14:13.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.742+0000 7f23da000640 1 -- 172.21.15.19:0/3187128539 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23cc031830 con 0x7f23dc15c390 2024-03-28T13:14:13.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.742+0000 7f23da000640 1 -- 172.21.15.19:0/3187128539 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc031c00 con 0x7f23dc15c390 2024-03-28T13:14:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.743+0000 7f23e1c00640 1 -- 172.21.15.19:0/3187128539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23dc15c390 msgr2=0x7f23dc073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.743+0000 7f23e1c00640 1 --2- 172.21.15.19:0/3187128539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23dc15c390 0x7f23dc073790 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f23dc06eaa0 tx=0x7f23cc02b8d0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.743+0000 7f23e1c00640 1 -- 172.21.15.19:0/3187128539 shutdown_connections 2024-03-28T13:14:13.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.743+0000 7f23e1c00640 1 --2- 172.21.15.19:0/3187128539 >> v2:172.21.15.188:3300/0 conn(0x7f23dc0af830 0x7f23dc160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.743+0000 7f23e1c00640 1 --2- 172.21.15.19:0/3187128539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23dc15c390 0x7f23dc073790 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.743+0000 7f23e1c00640 1 -- 172.21.15.19:0/3187128539 >> 172.21.15.19:0/3187128539 conn(0x7f23dc06b990 msgr2=0x7f23dc06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.743+0000 7f23e1c00640 1 -- 172.21.15.19:0/3187128539 shutdown_connections 2024-03-28T13:14:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.743+0000 7f23e1c00640 1 -- 172.21.15.19:0/3187128539 wait complete. 2024-03-28T13:14:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.745+0000 7f23e1c00640 1 Processor -- start 2024-03-28T13:14:13.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.745+0000 7f23e1c00640 1 -- start start 2024-03-28T13:14:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.746+0000 7f23e1c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23dc0af830 0x7f23dc150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.746+0000 7f23e1c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23dc15c390 0x7f23dc0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.746+0000 7f23e1c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23dc160b80 0x7f23dc159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.746+0000 7f23e1c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc0cf980 con 0x7f23dc0af830 2024-03-28T13:14:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.746+0000 7f23e1c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc160120 con 0x7f23dc160b80 2024-03-28T13:14:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.746+0000 7f23e1c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f23dc0d0d60 con 0x7f23dc15c390 2024-03-28T13:14:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.747+0000 7f23daa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23dc0af830 0x7f23dc150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.747+0000 7f23daa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23dc0af830 0x7f23dc150840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48698/0 (socket says 172.21.15.19:48698) 2024-03-28T13:14:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.747+0000 7f23daa00640 1 -- 172.21.15.19:0/2168112403 learned_addr learned my addr 172.21.15.19:0/2168112403 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.747+0000 7f23db400640 1 --2- 172.21.15.19:0/2168112403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23dc160b80 0x7f23dc159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.747+0000 7f23d2000640 1 --2- 172.21.15.19:0/2168112403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23dc15c390 0x7f23dc0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.747+0000 7f23daa00640 1 -- 172.21.15.19:0/2168112403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23dc15c390 msgr2=0x7f23dc0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.747+0000 7f23daa00640 1 --2- 172.21.15.19:0/2168112403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23dc15c390 0x7f23dc0b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.747+0000 7f23daa00640 1 -- 172.21.15.19:0/2168112403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23dc160b80 msgr2=0x7f23dc159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.747+0000 7f23daa00640 1 --2- 172.21.15.19:0/2168112403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23dc160b80 0x7f23dc159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.747+0000 7f23daa00640 1 -- 172.21.15.19:0/2168112403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc1f7ef0 con 0x7f23dc0af830 2024-03-28T13:14:13.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.748+0000 7f23daa00640 1 --2- 172.21.15.19:0/2168112403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23dc0af830 0x7f23dc150840 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009890 tx=0x7f23cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:13.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.748+0000 7f23d3400640 1 -- 172.21.15.19:0/2168112403 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc041070 con 0x7f23dc0af830 2024-03-28T13:14:13.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.748+0000 7f23d3400640 1 -- 172.21.15.19:0/2168112403 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23cc03aa30 con 0x7f23dc0af830 2024-03-28T13:14:13.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.748+0000 7f23d3400640 1 -- 172.21.15.19:0/2168112403 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc035040 con 0x7f23dc0af830 2024-03-28T13:14:13.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.748+0000 7f23e1c00640 1 -- 172.21.15.19:0/2168112403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23dc1f81d0 con 0x7f23dc0af830 2024-03-28T13:14:13.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.748+0000 7f23e1c00640 1 -- 172.21.15.19:0/2168112403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23dc1f8790 con 0x7f23dc0af830 2024-03-28T13:14:13.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.749+0000 7f23e1c00640 1 -- 172.21.15.19:0/2168112403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23dc0685c0 con 0x7f23dc0af830 2024-03-28T13:14:13.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.750+0000 7f23d3400640 1 -- 172.21.15.19:0/2168112403 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f23cc038020 con 0x7f23dc0af830 2024-03-28T13:14:13.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.752+0000 7f23d3400640 1 --2- 172.21.15.19:0/2168112403 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f23b806c530 0x7f23b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:13.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.752+0000 7f23d3400640 1 -- 172.21.15.19:0/2168112403 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f23cc0ad1e0 con 0x7f23dc0af830 2024-03-28T13:14:13.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.752+0000 7f23d2000640 1 --2- 172.21.15.19:0/2168112403 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f23b806c530 0x7f23b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:13.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.754+0000 7f23d2000640 1 --2- 172.21.15.19:0/2168112403 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f23b806c530 0x7f23b806e9f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0b5420 tx=0x7f23c4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:13.757+0000 7f23d3400640 1 -- 172.21.15.19:0/2168112403 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f23cc02f2e0 con 0x7f23dc0af830 2024-03-28T13:14:14.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.003+0000 7f23e1c00640 1 -- 172.21.15.19:0/2168112403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23dc06f1b0 con 0x7f23dc0af830 2024-03-28T13:14:14.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.005+0000 7f23d3400640 1 -- 172.21.15.19:0/2168112403 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5188 (secure 0 0 0) 0x7f23cc07b720 con 0x7f23dc0af830 2024-03-28T13:14:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:14.005 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1607201514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1607201514},{"type":"v1","addr":"172.21.15.188:6837","nonce":1607201514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1206740614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1206740614},{"type":"v1","addr":"172.21.15.188:6835","nonce":1206740614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2018671345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2018671345},{"type":"v1","addr":"172.21.15.19:6834","nonce":2018671345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:12:52.947573+0000","modified":"2024-03-28T13:13:37.546087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":14,"addr":"172.21.15.19:6835/3341654519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3341654519},{"type":"v1","addr":"172.21.15.19:6835","nonce":3341654519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":4}]} 2024-03-28T13:14:14.005 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 36 2024-03-28T13:14:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.008+0000 7f23e1c00640 1 -- 172.21.15.19:0/2168112403 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f23b806c530 msgr2=0x7f23b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.008+0000 7f23e1c00640 1 --2- 172.21.15.19:0/2168112403 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f23b806c530 0x7f23b806e9f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0b5420 tx=0x7f23c4008040 comp rx=0 tx=0).stop 2024-03-28T13:14:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.008+0000 7f23e1c00640 1 -- 172.21.15.19:0/2168112403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23dc0af830 msgr2=0x7f23dc150840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.008+0000 7f23e1c00640 1 --2- 172.21.15.19:0/2168112403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23dc0af830 0x7f23dc150840 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f23cc009890 tx=0x7f23cc0046c0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.008+0000 7f23e1c00640 1 -- 172.21.15.19:0/2168112403 shutdown_connections 2024-03-28T13:14:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.008+0000 7f23e1c00640 1 --2- 172.21.15.19:0/2168112403 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f23b806c530 0x7f23b806e9f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.008+0000 7f23e1c00640 1 --2- 172.21.15.19:0/2168112403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23dc160b80 0x7f23dc159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.008+0000 7f23e1c00640 1 --2- 172.21.15.19:0/2168112403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23dc15c390 0x7f23dc0b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.008+0000 7f23e1c00640 1 --2- 172.21.15.19:0/2168112403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f23dc0af830 0x7f23dc150840 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.008+0000 7f23e1c00640 1 -- 172.21.15.19:0/2168112403 >> 172.21.15.19:0/2168112403 conn(0x7f23dc06b990 msgr2=0x7f23dc0afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.008+0000 7f23e1c00640 1 -- 172.21.15.19:0/2168112403 shutdown_connections 2024-03-28T13:14:14.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.008+0000 7f23e1c00640 1 -- 172.21.15.19:0/2168112403 wait complete. 2024-03-28T13:14:14.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:14:14.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.242+0000 7f6db3e00640 1 Processor -- start 2024-03-28T13:14:14.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.242+0000 7f6db3e00640 1 -- start start 2024-03-28T13:14:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.245+0000 7f6db3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6db40d2310 0x7f6db40ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.245+0000 7f6db3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6db40cf530 0x7f6db40b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.245+0000 7f6db3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6db415e880 con 0x7f6db415e4a0 2024-03-28T13:14:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.245+0000 7f6db3e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6db40d34e0 con 0x7f6db40d2310 2024-03-28T13:14:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.245+0000 7f6db3e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6db40d28d0 con 0x7f6db40cf530 2024-03-28T13:14:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.245+0000 7f6db2a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6db415e4a0 0x7f6db40b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51144/0 (socket says 172.21.15.19:51144) 2024-03-28T13:14:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.245+0000 7f6db2a00640 1 -- 172.21.15.19:0/3869331155 learned_addr learned my addr 172.21.15.19:0/3869331155 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.245+0000 7f6db2000640 1 --2- 172.21.15.19:0/3869331155 >> v2:172.21.15.19:3300/0 conn(0x7f6db40d2310 0x7f6db40ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.246+0000 7f6db3400640 1 --2- 172.21.15.19:0/3869331155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6db40cf530 0x7f6db40b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.246+0000 7f6db1600640 1 -- 172.21.15.19:0/3869331155 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1430148364 0 0) 0x7f6db415e880 con 0x7f6db415e4a0 2024-03-28T13:14:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.246+0000 7f6db1600640 1 -- 172.21.15.19:0/3869331155 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d98003520 con 0x7f6db415e4a0 2024-03-28T13:14:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.246+0000 7f6db2000640 1 -- 172.21.15.19:0/3869331155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6db40cf530 msgr2=0x7f6db40b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.246+0000 7f6db2000640 1 --2- 172.21.15.19:0/3869331155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6db40cf530 0x7f6db40b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.246+0000 7f6db2000640 1 -- 172.21.15.19:0/3869331155 >> v1:172.21.15.19:6789/0 conn(0x7f6db415e4a0 legacy=0x7f6db40b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.246+0000 7f6db2000640 1 -- 172.21.15.19:0/3869331155 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db40d2bb0 con 0x7f6db40d2310 2024-03-28T13:14:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.247+0000 7f6db2000640 1 --2- 172.21.15.19:0/3869331155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db40d2310 0x7f6db40ceff0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0097a0 tx=0x7f6d9c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbff1bc3e2fb41b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.247+0000 7f6db1600640 1 -- 172.21.15.19:0/3869331155 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c0315a0 con 0x7f6db40d2310 2024-03-28T13:14:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.247+0000 7f6db1600640 1 -- 172.21.15.19:0/3869331155 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d9c037070 con 0x7f6db40d2310 2024-03-28T13:14:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.247+0000 7f6db1600640 1 -- 172.21.15.19:0/3869331155 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c031960 con 0x7f6db40d2310 2024-03-28T13:14:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.248+0000 7f6db3e00640 1 -- 172.21.15.19:0/3869331155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db40d2310 msgr2=0x7f6db40ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.248+0000 7f6db3e00640 1 --2- 172.21.15.19:0/3869331155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db40d2310 0x7f6db40ceff0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c0097a0 tx=0x7f6d9c02b840 comp rx=0 tx=0).stop 2024-03-28T13:14:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.248+0000 7f6db3e00640 1 -- 172.21.15.19:0/3869331155 shutdown_connections 2024-03-28T13:14:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.248+0000 7f6db3e00640 1 --2- 172.21.15.19:0/3869331155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6db40cf530 0x7f6db40b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.248+0000 7f6db3e00640 1 --2- 172.21.15.19:0/3869331155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db40d2310 0x7f6db40ceff0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.248+0000 7f6db3e00640 1 -- 172.21.15.19:0/3869331155 >> 172.21.15.19:0/3869331155 conn(0x7f6db40676d0 msgr2=0x7f6db4067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.248+0000 7f6db3e00640 1 -- 172.21.15.19:0/3869331155 shutdown_connections 2024-03-28T13:14:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.248+0000 7f6db3e00640 1 -- 172.21.15.19:0/3869331155 wait complete. 2024-03-28T13:14:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.251+0000 7f6db3e00640 1 Processor -- start 2024-03-28T13:14:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.251+0000 7f6db3e00640 1 -- start start 2024-03-28T13:14:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.253+0000 7f6db3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6db40cf530 0x7f6db4161530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.253+0000 7f6db3e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6db40d2310 0x7f6db40aac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.253+0000 7f6db3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db415e4a0 0x7f6db414a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.253+0000 7f6db3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6db40d3970 con 0x7f6db415e4a0 2024-03-28T13:14:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.253+0000 7f6db3e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6db40d3170 con 0x7f6db40d2310 2024-03-28T13:14:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.253+0000 7f6db3e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6db40b7920 con 0x7f6db40cf530 2024-03-28T13:14:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.253+0000 7f6db3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db415e4a0 0x7f6db414a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.253+0000 7f6db3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db415e4a0 0x7f6db414a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48714/0 (socket says 172.21.15.19:48714) 2024-03-28T13:14:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.253+0000 7f6db3400640 1 -- 172.21.15.19:0/3739620535 learned_addr learned my addr 172.21.15.19:0/3739620535 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.254+0000 7f6db2a00640 1 --2- 172.21.15.19:0/3739620535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6db40cf530 0x7f6db4161530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.254+0000 7f6db2000640 1 --2- 172.21.15.19:0/3739620535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6db40d2310 0x7f6db40aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.254+0000 7f6db3400640 1 -- 172.21.15.19:0/3739620535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6db40cf530 msgr2=0x7f6db4161530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.254+0000 7f6db3400640 1 --2- 172.21.15.19:0/3739620535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6db40cf530 0x7f6db4161530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.254+0000 7f6db3400640 1 -- 172.21.15.19:0/3739620535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6db40d2310 msgr2=0x7f6db40aac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.254+0000 7f6db3400640 1 --2- 172.21.15.19:0/3739620535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6db40d2310 0x7f6db40aac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.254+0000 7f6db3400640 1 -- 172.21.15.19:0/3739620535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db414b0d0 con 0x7f6db415e4a0 2024-03-28T13:14:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.254+0000 7f6db2000640 1 --2- 172.21.15.19:0/3739620535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6db40d2310 0x7f6db40aac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.254+0000 7f6db3400640 1 --2- 172.21.15.19:0/3739620535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db415e4a0 0x7f6db414a520 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00d830 tx=0x7f6dac00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.254+0000 7f6daaa00640 1 -- 172.21.15.19:0/3739620535 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac00b6c0 con 0x7f6db415e4a0 2024-03-28T13:14:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.254+0000 7f6daaa00640 1 -- 172.21.15.19:0/3739620535 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dac019070 con 0x7f6db415e4a0 2024-03-28T13:14:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.254+0000 7f6db3e00640 1 -- 172.21.15.19:0/3739620535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db414b3b0 con 0x7f6db415e4a0 2024-03-28T13:14:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.254+0000 7f6daaa00640 1 -- 172.21.15.19:0/3739620535 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac00bac0 con 0x7f6db415e4a0 2024-03-28T13:14:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.255+0000 7f6db3e00640 1 -- 172.21.15.19:0/3739620535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db414aa60 con 0x7f6db415e4a0 2024-03-28T13:14:14.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.256+0000 7f6db3e00640 1 -- 172.21.15.19:0/3739620535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db4075940 con 0x7f6db415e4a0 2024-03-28T13:14:14.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.257+0000 7f6daaa00640 1 -- 172.21.15.19:0/3739620535 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6dac01e460 con 0x7f6db415e4a0 2024-03-28T13:14:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.258+0000 7f6daaa00640 1 --2- 172.21.15.19:0/3739620535 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6d9006c530 0x7f6d9006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.259+0000 7f6daaa00640 1 -- 172.21.15.19:0/3739620535 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f6dac08be20 con 0x7f6db415e4a0 2024-03-28T13:14:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.259+0000 7f6db2a00640 1 --2- 172.21.15.19:0/3739620535 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6d9006c530 0x7f6d9006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:14.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.260+0000 7f6db2a00640 1 --2- 172.21.15.19:0/3739620535 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6d9006c530 0x7f6d9006e9f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6da40012e0 tx=0x7f6da4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:14.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.262+0000 7f6daaa00640 1 -- 172.21.15.19:0/3739620535 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6dac05a3f0 con 0x7f6db415e4a0 2024-03-28T13:14:14.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.482+0000 7f6db3e00640 1 -- 172.21.15.19:0/3739620535 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6db415adb0 con 0x7f6db415e4a0 2024-03-28T13:14:14.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.484+0000 7f6daaa00640 1 -- 172.21.15.19:0/3739620535 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16761 (secure 0 0 0) 0x7f6dac059cf0 con 0x7f6db415e4a0 2024-03-28T13:14:14.484 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:14.484 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:13:24.671229+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:12:50.888331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-28T13:12:51.946012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.14","mappings":[{"from":6,"to":4}]},{"pgid":"9.2c","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:14:14.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.486+0000 7f6db3e00640 1 -- 172.21.15.19:0/3739620535 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6d9006c530 msgr2=0x7f6d9006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:14.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.486+0000 7f6db3e00640 1 --2- 172.21.15.19:0/3739620535 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6d9006c530 0x7f6d9006e9f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6da40012e0 tx=0x7f6da4009040 comp rx=0 tx=0).stop 2024-03-28T13:14:14.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.486+0000 7f6db3e00640 1 -- 172.21.15.19:0/3739620535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db415e4a0 msgr2=0x7f6db414a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:14.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.486+0000 7f6db3e00640 1 --2- 172.21.15.19:0/3739620535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db415e4a0 0x7f6db414a520 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00d830 tx=0x7f6dac00dcf0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.486+0000 7f6db3e00640 1 -- 172.21.15.19:0/3739620535 shutdown_connections 2024-03-28T13:14:14.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.486+0000 7f6db3e00640 1 --2- 172.21.15.19:0/3739620535 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6d9006c530 0x7f6d9006e9f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.487+0000 7f6db3e00640 1 --2- 172.21.15.19:0/3739620535 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6db415e4a0 0x7f6db414a520 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.487+0000 7f6db3e00640 1 --2- 172.21.15.19:0/3739620535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6db40d2310 0x7f6db40aac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.487+0000 7f6db3e00640 1 --2- 172.21.15.19:0/3739620535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6db40cf530 0x7f6db4161530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.487+0000 7f6db3e00640 1 -- 172.21.15.19:0/3739620535 >> 172.21.15.19:0/3739620535 conn(0x7f6db40676d0 msgr2=0x7f6db415bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:14.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.487+0000 7f6db3e00640 1 -- 172.21.15.19:0/3739620535 shutdown_connections 2024-03-28T13:14:14.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.487+0000 7f6db3e00640 1 -- 172.21.15.19:0/3739620535 wait complete. 2024-03-28T13:14:14.505 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-28T13:14:14.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:14:14.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.719+0000 7fb689c00640 1 Processor -- start 2024-03-28T13:14:14.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.720+0000 7fb689c00640 1 -- start start 2024-03-28T13:14:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.721+0000 7fb689c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb68415e530 0x7fb6840cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.721+0000 7fb689c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb68406e1b0 0x7fb6840cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.721+0000 7fb689c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6840cf160 0x7fb6840cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.721+0000 7fb689c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb684162ed0 con 0x7fb68415e530 2024-03-28T13:14:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.721+0000 7fb689c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb684162350 con 0x7fb68406e1b0 2024-03-28T13:14:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.721+0000 7fb689c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb684162630 con 0x7fb6840cf160 2024-03-28T13:14:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.722+0000 7fb683e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb68415e530 0x7fb6840cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.722+0000 7fb683e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb68415e530 0x7fb6840cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48718/0 (socket says 172.21.15.19:48718) 2024-03-28T13:14:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.722+0000 7fb683e00640 1 -- 172.21.15.19:0/3637988433 learned_addr learned my addr 172.21.15.19:0/3637988433 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.722+0000 7fb67be00640 1 --2- 172.21.15.19:0/3637988433 >> v2:172.21.15.188:3300/0 conn(0x7fb68406e1b0 0x7fb6840cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.722+0000 7fb689200640 1 --2- 172.21.15.19:0/3637988433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6840cf160 0x7fb6840cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.722+0000 7fb683e00640 1 -- 172.21.15.19:0/3637988433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6840cf160 msgr2=0x7fb6840cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.722+0000 7fb683e00640 1 --2- 172.21.15.19:0/3637988433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6840cf160 0x7fb6840cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.722+0000 7fb683e00640 1 -- 172.21.15.19:0/3637988433 >> v2:172.21.15.188:3300/0 conn(0x7fb68406e1b0 msgr2=0x7fb6840cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.722+0000 7fb683e00640 1 --2- 172.21.15.19:0/3637988433 >> v2:172.21.15.188:3300/0 conn(0x7fb68406e1b0 0x7fb6840cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.722+0000 7fb683e00640 1 -- 172.21.15.19:0/3637988433 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb684162910 con 0x7fb68415e530 2024-03-28T13:14:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.722+0000 7fb67be00640 1 --2- 172.21.15.19:0/3637988433 >> v2:172.21.15.188:3300/0 conn(0x7fb68406e1b0 0x7fb6840cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:14:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.722+0000 7fb689200640 1 --2- 172.21.15.19:0/3637988433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6840cf160 0x7fb6840cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:14:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.724+0000 7fb683e00640 1 --2- 172.21.15.19:0/3637988433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb68415e530 0x7fb6840cc720 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb6840c5b70 tx=0x7fb67402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f330cbd845d89a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.724+0000 7fb683400640 1 -- 172.21.15.19:0/3637988433 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb67402e070 con 0x7fb68415e530 2024-03-28T13:14:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.724+0000 7fb683400640 1 -- 172.21.15.19:0/3637988433 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb674037070 con 0x7fb68415e530 2024-03-28T13:14:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.724+0000 7fb683400640 1 -- 172.21.15.19:0/3637988433 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674031a00 con 0x7fb68415e530 2024-03-28T13:14:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.725+0000 7fb689c00640 1 -- 172.21.15.19:0/3637988433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb68415e530 msgr2=0x7fb6840cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.725+0000 7fb689c00640 1 --2- 172.21.15.19:0/3637988433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb68415e530 0x7fb6840cc720 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb6840c5b70 tx=0x7fb67402b880 comp rx=0 tx=0).stop 2024-03-28T13:14:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.725+0000 7fb689c00640 1 -- 172.21.15.19:0/3637988433 shutdown_connections 2024-03-28T13:14:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.725+0000 7fb689c00640 1 --2- 172.21.15.19:0/3637988433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6840cf160 0x7fb6840cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.725+0000 7fb689c00640 1 --2- 172.21.15.19:0/3637988433 >> v2:172.21.15.188:3300/0 conn(0x7fb68406e1b0 0x7fb6840cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.725+0000 7fb689c00640 1 --2- 172.21.15.19:0/3637988433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb68415e530 0x7fb6840cc720 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.725+0000 7fb689c00640 1 -- 172.21.15.19:0/3637988433 >> 172.21.15.19:0/3637988433 conn(0x7fb684067280 msgr2=0x7fb684067660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:14.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.725+0000 7fb689c00640 1 -- 172.21.15.19:0/3637988433 shutdown_connections 2024-03-28T13:14:14.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.725+0000 7fb689c00640 1 -- 172.21.15.19:0/3637988433 wait complete. 2024-03-28T13:14:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.727+0000 7fb689c00640 1 Processor -- start 2024-03-28T13:14:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.727+0000 7fb689c00640 1 -- start start 2024-03-28T13:14:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.728+0000 7fb689c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb68406e1b0 0x7fb684158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.728+0000 7fb689c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6840cf160 0x7fb684152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.728+0000 7fb689c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb68415e530 0x7fb684156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.728+0000 7fb689c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb68406c680 con 0x7fb68406e1b0 2024-03-28T13:14:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.728+0000 7fb689c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb684161fe0 con 0x7fb68415e530 2024-03-28T13:14:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.728+0000 7fb689c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb68406c800 con 0x7fb6840cf160 2024-03-28T13:14:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.728+0000 7fb683e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb68406e1b0 0x7fb684158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.728+0000 7fb683e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb68406e1b0 0x7fb684158770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48722/0 (socket says 172.21.15.19:48722) 2024-03-28T13:14:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.728+0000 7fb683e00640 1 -- 172.21.15.19:0/2225539886 learned_addr learned my addr 172.21.15.19:0/2225539886 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.729+0000 7fb67be00640 1 --2- 172.21.15.19:0/2225539886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6840cf160 0x7fb684152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.729+0000 7fb683e00640 1 -- 172.21.15.19:0/2225539886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6840cf160 msgr2=0x7fb684152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.729+0000 7fb683e00640 1 --2- 172.21.15.19:0/2225539886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6840cf160 0x7fb684152750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.729+0000 7fb689200640 1 --2- 172.21.15.19:0/2225539886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb68415e530 0x7fb684156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.729+0000 7fb683e00640 1 -- 172.21.15.19:0/2225539886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb68415e530 msgr2=0x7fb684156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.729+0000 7fb683e00640 1 --2- 172.21.15.19:0/2225539886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb68415e530 0x7fb684156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.729+0000 7fb683e00640 1 -- 172.21.15.19:0/2225539886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6841576d0 con 0x7fb68406e1b0 2024-03-28T13:14:14.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.729+0000 7fb689200640 1 --2- 172.21.15.19:0/2225539886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb68415e530 0x7fb684156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.729+0000 7fb683e00640 1 --2- 172.21.15.19:0/2225539886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb68406e1b0 0x7fb684158770 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb67402b8c0 tx=0x7fb6740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.730+0000 7fb680c00640 1 -- 172.21.15.19:0/2225539886 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674032670 con 0x7fb68406e1b0 2024-03-28T13:14:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.730+0000 7fb680c00640 1 -- 172.21.15.19:0/2225539886 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb674032850 con 0x7fb68406e1b0 2024-03-28T13:14:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.730+0000 7fb680c00640 1 -- 172.21.15.19:0/2225539886 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb674032c40 con 0x7fb68406e1b0 2024-03-28T13:14:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.730+0000 7fb689c00640 1 -- 172.21.15.19:0/2225539886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6841579b0 con 0x7fb68406e1b0 2024-03-28T13:14:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.730+0000 7fb689c00640 1 -- 172.21.15.19:0/2225539886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb684157f70 con 0x7fb68406e1b0 2024-03-28T13:14:14.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.731+0000 7fb689c00640 1 -- 172.21.15.19:0/2225539886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb684075940 con 0x7fb68406e1b0 2024-03-28T13:14:14.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.733+0000 7fb680c00640 1 -- 172.21.15.19:0/2225539886 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb6740453f0 con 0x7fb68406e1b0 2024-03-28T13:14:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.734+0000 7fb680c00640 1 --2- 172.21.15.19:0/2225539886 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb66006c530 0x7fb66006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.734+0000 7fb680c00640 1 -- 172.21.15.19:0/2225539886 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7fb6740ad4a0 con 0x7fb68406e1b0 2024-03-28T13:14:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.735+0000 7fb67be00640 1 --2- 172.21.15.19:0/2225539886 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb66006c530 0x7fb66006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:14.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.736+0000 7fb67be00640 1 --2- 172.21.15.19:0/2225539886 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb66006c530 0x7fb66006e9f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb66c006fd0 tx=0x7fb66c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:14.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.739+0000 7fb680c00640 1 -- 172.21.15.19:0/2225539886 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb674040070 con 0x7fb68406e1b0 2024-03-28T13:14:14.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.986+0000 7fb689c00640 1 -- 172.21.15.19:0/2225539886 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb684070280 con 0x7fb68406e1b0 2024-03-28T13:14:14.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.987+0000 7fb680c00640 1 -- 172.21.15.19:0/2225539886 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5188 (secure 0 0 0) 0x7fb67407bba0 con 0x7fb68406e1b0 2024-03-28T13:14:14.987 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:14.987 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1607201514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1607201514},{"type":"v1","addr":"172.21.15.188:6837","nonce":1607201514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1206740614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1206740614},{"type":"v1","addr":"172.21.15.188:6835","nonce":1206740614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2018671345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2018671345},{"type":"v1","addr":"172.21.15.19:6834","nonce":2018671345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:12:52.947573+0000","modified":"2024-03-28T13:13:37.546087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":14,"addr":"172.21.15.19:6835/3341654519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3341654519},{"type":"v1","addr":"172.21.15.19:6835","nonce":3341654519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":4}]} 2024-03-28T13:14:14.988 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 36 2024-03-28T13:14:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.990+0000 7fb689c00640 1 -- 172.21.15.19:0/2225539886 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb66006c530 msgr2=0x7fb66006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.990+0000 7fb689c00640 1 --2- 172.21.15.19:0/2225539886 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb66006c530 0x7fb66006e9f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb66c006fd0 tx=0x7fb66c008040 comp rx=0 tx=0).stop 2024-03-28T13:14:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.990+0000 7fb689c00640 1 -- 172.21.15.19:0/2225539886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb68406e1b0 msgr2=0x7fb684158770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.990+0000 7fb689c00640 1 --2- 172.21.15.19:0/2225539886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb68406e1b0 0x7fb684158770 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb67402b8c0 tx=0x7fb6740046c0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.990+0000 7fb689c00640 1 -- 172.21.15.19:0/2225539886 shutdown_connections 2024-03-28T13:14:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.990+0000 7fb689c00640 1 --2- 172.21.15.19:0/2225539886 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb66006c530 0x7fb66006e9f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.990+0000 7fb689c00640 1 --2- 172.21.15.19:0/2225539886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb68415e530 0x7fb684156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.990+0000 7fb689c00640 1 --2- 172.21.15.19:0/2225539886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6840cf160 0x7fb684152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.990+0000 7fb689c00640 1 --2- 172.21.15.19:0/2225539886 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb68406e1b0 0x7fb684158770 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.990+0000 7fb689c00640 1 -- 172.21.15.19:0/2225539886 >> 172.21.15.19:0/2225539886 conn(0x7fb684067280 msgr2=0x7fb684160820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.990+0000 7fb689c00640 1 -- 172.21.15.19:0/2225539886 shutdown_connections 2024-03-28T13:14:14.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:14.990+0000 7fb689c00640 1 -- 172.21.15.19:0/2225539886 wait complete. 2024-03-28T13:14:15.011 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:14:15.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.224+0000 7f9ed0e00640 1 Processor -- start 2024-03-28T13:14:15.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.224+0000 7f9ed0e00640 1 -- start start 2024-03-28T13:14:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.226+0000 7f9ed0e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9ecc15fc50 0x7f9ecc0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.226+0000 7f9ed0e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ecc0cc490 0x7f9ecc0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.226+0000 7f9ed0e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ecc0b3150 con 0x7f9ecc0cc0b0 2024-03-28T13:14:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.226+0000 7f9ed0e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9ecc15c610 con 0x7f9ecc15fc50 2024-03-28T13:14:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.226+0000 7f9ed0e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9ecc1600e0 con 0x7f9ecc0cc490 2024-03-28T13:14:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.226+0000 7f9ecb400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9ecc0cc0b0 0x7f9ecc06dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51148/0 (socket says 172.21.15.19:51148) 2024-03-28T13:14:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.226+0000 7f9ecb400640 1 -- 172.21.15.19:0/2771974589 learned_addr learned my addr 172.21.15.19:0/2771974589 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:15.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.227+0000 7f9ecbe00640 1 --2- 172.21.15.19:0/2771974589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ecc0cc490 0x7f9ecc0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:15.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.227+0000 7f9ecaa00640 1 -- 172.21.15.19:0/2771974589 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3339494492 0 0) 0x7f9ecc0b3150 con 0x7f9ecc0cc0b0 2024-03-28T13:14:15.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.227+0000 7f9ecaa00640 1 -- 172.21.15.19:0/2771974589 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9eb0003520 con 0x7f9ecc0cc0b0 2024-03-28T13:14:15.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.227+0000 7f9ec3e00640 1 --2- 172.21.15.19:0/2771974589 >> v2:172.21.15.188:3300/0 conn(0x7f9ecc15fc50 0x7f9ecc0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:15.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.227+0000 7f9ecaa00640 1 -- 172.21.15.19:0/2771974589 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1627299098 0 0) 0x7f9eb0003520 con 0x7f9ecc0cc0b0 2024-03-28T13:14:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.227+0000 7f9ecaa00640 1 -- 172.21.15.19:0/2771974589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ecc0cc490 msgr2=0x7f9ecc0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.227+0000 7f9ecaa00640 1 --2- 172.21.15.19:0/2771974589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ecc0cc490 0x7f9ecc0d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.227+0000 7f9ecaa00640 1 -- 172.21.15.19:0/2771974589 >> v2:172.21.15.188:3300/0 conn(0x7f9ecc15fc50 msgr2=0x7f9ecc0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.227+0000 7f9ecaa00640 1 --2- 172.21.15.19:0/2771974589 >> v2:172.21.15.188:3300/0 conn(0x7f9ecc15fc50 0x7f9ecc0cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.227+0000 7f9ecaa00640 1 -- 172.21.15.19:0/2771974589 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc1603c0 con 0x7f9ecc0cc0b0 2024-03-28T13:14:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.227+0000 7f9ecaa00640 1 -- 172.21.15.19:0/2771974589 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f9ebc001670 con 0x7f9ecc0cc0b0 2024-03-28T13:14:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.227+0000 7f9ecaa00640 1 -- 172.21.15.19:0/2771974589 >> v1:172.21.15.19:6789/0 conn(0x7f9ecc0cc0b0 legacy=0x7f9ecc06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.227+0000 7f9ecaa00640 1 --2- 172.21.15.19:0/2771974589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eb00039b0 0x7f9eb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.228+0000 7f9ecaa00640 1 --2- 172.21.15.19:0/2771974589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ecc0cc490 0x7f9eb0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.228+0000 7f9ecaa00640 1 --2- 172.21.15.19:0/2771974589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eb0004d40 0x7f9eb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.228+0000 7f9ecaa00640 1 -- 172.21.15.19:0/2771974589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc15c610 con 0x7f9eb00039b0 2024-03-28T13:14:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.228+0000 7f9ecaa00640 1 -- 172.21.15.19:0/2771974589 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc1600e0 con 0x7f9eb0004d40 2024-03-28T13:14:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.228+0000 7f9ecaa00640 1 -- 172.21.15.19:0/2771974589 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9eb000ba40 con 0x7f9ecc0cc490 2024-03-28T13:14:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.228+0000 7f9ecb400640 1 --2- 172.21.15.19:0/2771974589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eb00039b0 0x7f9eb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.228+0000 7f9ecb400640 1 -- 172.21.15.19:0/2771974589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ecc0cc490 msgr2=0x7f9eb0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:15.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.228+0000 7f9ecbe00640 1 --2- 172.21.15.19:0/2771974589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eb0004d40 0x7f9eb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:15.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.228+0000 7f9ecb400640 1 --2- 172.21.15.19:0/2771974589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ecc0cc490 0x7f9eb0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.228+0000 7f9ecb400640 1 -- 172.21.15.19:0/2771974589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eb0004d40 msgr2=0x7f9eb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:15.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.228+0000 7f9ecb400640 1 --2- 172.21.15.19:0/2771974589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eb0004d40 0x7f9eb00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.228+0000 7f9ecb400640 1 -- 172.21.15.19:0/2771974589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eb0003520 con 0x7f9eb00039b0 2024-03-28T13:14:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.229+0000 7f9ecb400640 1 --2- 172.21.15.19:0/2771974589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eb00039b0 0x7f9eb0003dd0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00c190 tx=0x7f9eb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51632ed2c7a2dc13 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.230+0000 7f9ecaa00640 1 -- 172.21.15.19:0/2771974589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc007c60 con 0x7f9eb00039b0 2024-03-28T13:14:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.230+0000 7f9ecaa00640 1 -- 172.21.15.19:0/2771974589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc030070 con 0x7f9eb00039b0 2024-03-28T13:14:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.230+0000 7f9ecaa00640 1 -- 172.21.15.19:0/2771974589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc005d30 con 0x7f9eb00039b0 2024-03-28T13:14:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.230+0000 7f9ed0e00640 1 -- 172.21.15.19:0/2771974589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eb00039b0 msgr2=0x7f9eb0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.230+0000 7f9ed0e00640 1 --2- 172.21.15.19:0/2771974589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eb00039b0 0x7f9eb0003dd0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00c190 tx=0x7f9eb0005830 comp rx=0 tx=0).stop 2024-03-28T13:14:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.230+0000 7f9ecbe00640 1 -- 172.21.15.19:0/2771974589 reap_dead start 2024-03-28T13:14:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.230+0000 7f9ed0e00640 1 -- 172.21.15.19:0/2771974589 shutdown_connections 2024-03-28T13:14:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.230+0000 7f9ed0e00640 1 -- 172.21.15.19:0/2771974589 >> 172.21.15.19:0/2771974589 conn(0x7f9ecc0ccad0 msgr2=0x7f9ecc0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.230+0000 7f9ed0e00640 1 -- 172.21.15.19:0/2771974589 shutdown_connections 2024-03-28T13:14:15.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.231+0000 7f9ed0e00640 1 -- 172.21.15.19:0/2771974589 wait complete. 2024-03-28T13:14:15.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.232+0000 7f9ed0e00640 1 Processor -- start 2024-03-28T13:14:15.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.232+0000 7f9ed0e00640 1 -- start start 2024-03-28T13:14:15.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.233+0000 7f9ed0e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb00039b0 0x7f9ecc156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.233+0000 7f9ed0e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ecc1f8430 0x7f9ecc0b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.233+0000 7f9ed0e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ecc0ae4a0 0x7f9ecc0ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.233+0000 7f9ed0e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc0d0300 con 0x7f9ecc1f8430 2024-03-28T13:14:15.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.233+0000 7f9ed0e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc160980 con 0x7f9ecc0ae4a0 2024-03-28T13:14:15.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.233+0000 7f9ed0e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9ecc0d0480 con 0x7f9eb00039b0 2024-03-28T13:14:15.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.234+0000 7f9ec3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ecc1f8430 0x7f9ecc0b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:15.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.234+0000 7f9ec3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ecc1f8430 0x7f9ecc0b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48754/0 (socket says 172.21.15.19:48754) 2024-03-28T13:14:15.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.234+0000 7f9ec3e00640 1 -- 172.21.15.19:0/1072715060 learned_addr learned my addr 172.21.15.19:0/1072715060 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:15.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.234+0000 7f9ecb400640 1 --2- 172.21.15.19:0/1072715060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb00039b0 0x7f9ecc156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:15.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.234+0000 7f9ecbe00640 1 --2- 172.21.15.19:0/1072715060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ecc0ae4a0 0x7f9ecc0ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:15.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.234+0000 7f9ec3e00640 1 -- 172.21.15.19:0/1072715060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb00039b0 msgr2=0x7f9ecc156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:15.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.234+0000 7f9ec3e00640 1 --2- 172.21.15.19:0/1072715060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb00039b0 0x7f9ecc156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.234+0000 7f9ec3e00640 1 -- 172.21.15.19:0/1072715060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ecc0ae4a0 msgr2=0x7f9ecc0ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:15.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.234+0000 7f9ec3e00640 1 --2- 172.21.15.19:0/1072715060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ecc0ae4a0 0x7f9ecc0ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.234+0000 7f9ec3e00640 1 -- 172.21.15.19:0/1072715060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc15c9d0 con 0x7f9ecc1f8430 2024-03-28T13:14:15.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.235+0000 7f9ec3e00640 1 --2- 172.21.15.19:0/1072715060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ecc1f8430 0x7f9ecc0b18d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400d800 tx=0x7f9eb400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:15.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.235+0000 7f9ec3400640 1 -- 172.21.15.19:0/1072715060 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb400b640 con 0x7f9ecc1f8430 2024-03-28T13:14:15.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.235+0000 7f9ec3400640 1 -- 172.21.15.19:0/1072715060 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eb4019070 con 0x7f9ecc1f8430 2024-03-28T13:14:15.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.235+0000 7f9ec3400640 1 -- 172.21.15.19:0/1072715060 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb400ba20 con 0x7f9ecc1f8430 2024-03-28T13:14:15.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.235+0000 7f9ed0e00640 1 -- 172.21.15.19:0/1072715060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ecc156e60 con 0x7f9ecc1f8430 2024-03-28T13:14:15.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.235+0000 7f9ed0e00640 1 -- 172.21.15.19:0/1072715060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ecc0b2520 con 0x7f9ecc1f8430 2024-03-28T13:14:15.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.236+0000 7f9ed0e00640 1 -- 172.21.15.19:0/1072715060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ecc075940 con 0x7f9ecc1f8430 2024-03-28T13:14:15.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.237+0000 7f9ec3400640 1 -- 172.21.15.19:0/1072715060 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9eb400bbc0 con 0x7f9ecc1f8430 2024-03-28T13:14:15.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.238+0000 7f9ec3400640 1 --2- 172.21.15.19:0/1072715060 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ea806c530 0x7f9ea806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.239+0000 7f9ec3400640 1 -- 172.21.15.19:0/1072715060 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f9eb408ba90 con 0x7f9ecc1f8430 2024-03-28T13:14:15.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.239+0000 7f9ecb400640 1 --2- 172.21.15.19:0/1072715060 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ea806c530 0x7f9ea806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:15.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.240+0000 7f9ecb400640 1 --2- 172.21.15.19:0/1072715060 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ea806c530 0x7f9ea806e9f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc034000 tx=0x7f9ebc007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:15.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.243+0000 7f9ec3400640 1 -- 172.21.15.19:0/1072715060 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9ecc156e60 con 0x7f9ecc1f8430 2024-03-28T13:14:15.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.486+0000 7f9ed0e00640 1 -- 172.21.15.19:0/1072715060 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ecc073830 con 0x7f9ecc1f8430 2024-03-28T13:14:15.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.486+0000 7f9ec3400640 1 -- 172.21.15.19:0/1072715060 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5188 (secure 0 0 0) 0x7f9eb405a060 con 0x7f9ecc1f8430 2024-03-28T13:14:15.487 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:15.487 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1607201514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1607201514},{"type":"v1","addr":"172.21.15.188:6837","nonce":1607201514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1206740614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1206740614},{"type":"v1","addr":"172.21.15.188:6835","nonce":1206740614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2018671345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2018671345},{"type":"v1","addr":"172.21.15.19:6834","nonce":2018671345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:12:52.947573+0000","modified":"2024-03-28T13:13:37.546087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"a","rank":0,"incarnation":30,"state":"up:active","state_seq":14,"addr":"172.21.15.19:6835/3341654519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3341654519},{"type":"v1","addr":"172.21.15.19:6835","nonce":3341654519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5970,"qdb_cluster":[5970]},"id":4}]} 2024-03-28T13:14:15.487 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 36 2024-03-28T13:14:15.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.489+0000 7f9ed0e00640 1 -- 172.21.15.19:0/1072715060 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ea806c530 msgr2=0x7f9ea806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:15.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.489+0000 7f9ed0e00640 1 --2- 172.21.15.19:0/1072715060 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ea806c530 0x7f9ea806e9f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc034000 tx=0x7f9ebc007000 comp rx=0 tx=0).stop 2024-03-28T13:14:15.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.489+0000 7f9ed0e00640 1 -- 172.21.15.19:0/1072715060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ecc1f8430 msgr2=0x7f9ecc0b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:15.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.489+0000 7f9ed0e00640 1 --2- 172.21.15.19:0/1072715060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ecc1f8430 0x7f9ecc0b18d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400d800 tx=0x7f9eb400dcc0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.489+0000 7f9ed0e00640 1 -- 172.21.15.19:0/1072715060 shutdown_connections 2024-03-28T13:14:15.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.489+0000 7f9ed0e00640 1 --2- 172.21.15.19:0/1072715060 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ea806c530 0x7f9ea806e9f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.489+0000 7f9ed0e00640 1 --2- 172.21.15.19:0/1072715060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ecc0ae4a0 0x7f9ecc0ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.489+0000 7f9ed0e00640 1 --2- 172.21.15.19:0/1072715060 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ecc1f8430 0x7f9ecc0b18d0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.489+0000 7f9ed0e00640 1 --2- 172.21.15.19:0/1072715060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb00039b0 0x7f9ecc156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.489+0000 7f9ed0e00640 1 -- 172.21.15.19:0/1072715060 >> 172.21.15.19:0/1072715060 conn(0x7f9ecc0ccad0 msgr2=0x7f9ecc0cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:15.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.489+0000 7f9ed0e00640 1 -- 172.21.15.19:0/1072715060 shutdown_connections 2024-03-28T13:14:15.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.489+0000 7f9ed0e00640 1 -- 172.21.15.19:0/1072715060 wait complete. 2024-03-28T13:14:15.510 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:14:15.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.729+0000 7f0bbf200640 1 Processor -- start 2024-03-28T13:14:15.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.729+0000 7f0bbf200640 1 -- start start 2024-03-28T13:14:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.731+0000 7f0bbf200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0bc015e6e0 0x7f0bc00d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.731+0000 7f0bbf200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0bc00b0580 0x7f0bc0162880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.731+0000 7f0bbf200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc006b570 0x7f0bc006ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.731+0000 7f0bbf200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc00d0660 con 0x7f0bc00b0580 2024-03-28T13:14:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.731+0000 7f0bbf200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc00d08c0 con 0x7f0bc015e6e0 2024-03-28T13:14:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.731+0000 7f0bbf200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc00d0ba0 con 0x7f0bc006b570 2024-03-28T13:14:15.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.731+0000 7f0bbd400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0bc00b0580 0x7f0bc0162880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:15.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.731+0000 7f0bbd400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0bc00b0580 0x7f0bc0162880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50956/0 (socket says 172.21.15.19:50956) 2024-03-28T13:14:15.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.731+0000 7f0bbd400640 1 -- 172.21.15.19:0/3699422914 learned_addr learned my addr 172.21.15.19:0/3699422914 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:15.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.732+0000 7f0bbe800640 1 --2- 172.21.15.19:0/3699422914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc006b570 0x7f0bc006ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:15.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.732+0000 7f0bbde00640 1 --2- 172.21.15.19:0/3699422914 >> v2:172.21.15.188:3300/0 conn(0x7f0bc015e6e0 0x7f0bc00d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:15.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.732+0000 7f0bbd400640 1 -- 172.21.15.19:0/3699422914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc006b570 msgr2=0x7f0bc006ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:15.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.732+0000 7f0bbd400640 1 --2- 172.21.15.19:0/3699422914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc006b570 0x7f0bc006ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.732+0000 7f0bbd400640 1 -- 172.21.15.19:0/3699422914 >> v2:172.21.15.188:3300/0 conn(0x7f0bc015e6e0 msgr2=0x7f0bc00d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:15.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.732+0000 7f0bbd400640 1 --2- 172.21.15.19:0/3699422914 >> v2:172.21.15.188:3300/0 conn(0x7f0bc015e6e0 0x7f0bc00d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.732+0000 7f0bbd400640 1 -- 172.21.15.19:0/3699422914 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc00d0e80 con 0x7f0bc00b0580 2024-03-28T13:14:15.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.732+0000 7f0bbe800640 1 --2- 172.21.15.19:0/3699422914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc006b570 0x7f0bc006ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:15.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.732+0000 7f0bbde00640 1 --2- 172.21.15.19:0/3699422914 >> v2:172.21.15.188:3300/0 conn(0x7f0bc015e6e0 0x7f0bc00d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:15.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.734+0000 7f0bbd400640 1 --2- 172.21.15.19:0/3699422914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bc00b0580 0x7f0bc0162880 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0bc006e320 tx=0x7f0bac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9808075c108260 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:15.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.734+0000 7f0bbca00640 1 -- 172.21.15.19:0/3699422914 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac035020 con 0x7f0bc00b0580 2024-03-28T13:14:15.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.734+0000 7f0bbca00640 1 -- 172.21.15.19:0/3699422914 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bac031870 con 0x7f0bc00b0580 2024-03-28T13:14:15.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.734+0000 7f0bbca00640 1 -- 172.21.15.19:0/3699422914 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac031c40 con 0x7f0bc00b0580 2024-03-28T13:14:15.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.735+0000 7f0bbf200640 1 -- 172.21.15.19:0/3699422914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bc00b0580 msgr2=0x7f0bc0162880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:15.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.735+0000 7f0bbf200640 1 --2- 172.21.15.19:0/3699422914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bc00b0580 0x7f0bc0162880 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f0bc006e320 tx=0x7f0bac02b910 comp rx=0 tx=0).stop 2024-03-28T13:14:15.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.735+0000 7f0bbf200640 1 -- 172.21.15.19:0/3699422914 shutdown_connections 2024-03-28T13:14:15.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.735+0000 7f0bbf200640 1 --2- 172.21.15.19:0/3699422914 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc006b570 0x7f0bc006ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.735+0000 7f0bbf200640 1 --2- 172.21.15.19:0/3699422914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bc00b0580 0x7f0bc0162880 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.735+0000 7f0bbf200640 1 --2- 172.21.15.19:0/3699422914 >> v2:172.21.15.188:3300/0 conn(0x7f0bc015e6e0 0x7f0bc00d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.735+0000 7f0bbf200640 1 -- 172.21.15.19:0/3699422914 >> 172.21.15.19:0/3699422914 conn(0x7f0bc009aa20 msgr2=0x7f0bc00b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:15.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.735+0000 7f0bbf200640 1 -- 172.21.15.19:0/3699422914 shutdown_connections 2024-03-28T13:14:15.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.735+0000 7f0bbf200640 1 -- 172.21.15.19:0/3699422914 wait complete. 2024-03-28T13:14:15.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.738+0000 7f0bbf200640 1 Processor -- start 2024-03-28T13:14:15.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.738+0000 7f0bbf200640 1 -- start start 2024-03-28T13:14:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.740+0000 7f0bbf200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bc006b570 0x7f0bc01f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.740+0000 7f0bbf200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bc00b0580 0x7f0bc00ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.740+0000 7f0bbf200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc015e6e0 0x7f0bc00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.740+0000 7f0bbf200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc0162dc0 con 0x7f0bc00b0580 2024-03-28T13:14:15.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.740+0000 7f0bbf200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc00b0a00 con 0x7f0bc006b570 2024-03-28T13:14:15.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.740+0000 7f0bbf200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc0162f40 con 0x7f0bc015e6e0 2024-03-28T13:14:15.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.740+0000 7f0bbd400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bc00b0580 0x7f0bc00ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:15.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.740+0000 7f0bbd400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bc00b0580 0x7f0bc00ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50958/0 (socket says 172.21.15.19:50958) 2024-03-28T13:14:15.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.740+0000 7f0bbd400640 1 -- 172.21.15.19:0/1769131648 learned_addr learned my addr 172.21.15.19:0/1769131648 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:15.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.741+0000 7f0bbde00640 1 --2- 172.21.15.19:0/1769131648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bc006b570 0x7f0bc01f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:15.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.741+0000 7f0bbe800640 1 --2- 172.21.15.19:0/1769131648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc015e6e0 0x7f0bc00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.741+0000 7f0bbd400640 1 -- 172.21.15.19:0/1769131648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc015e6e0 msgr2=0x7f0bc00b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.741+0000 7f0bbd400640 1 --2- 172.21.15.19:0/1769131648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc015e6e0 0x7f0bc00b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.741+0000 7f0bbd400640 1 -- 172.21.15.19:0/1769131648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bc006b570 msgr2=0x7f0bc01f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.741+0000 7f0bbd400640 1 --2- 172.21.15.19:0/1769131648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bc006b570 0x7f0bc01f7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.741+0000 7f0bbd400640 1 -- 172.21.15.19:0/1769131648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc00ae990 con 0x7f0bc00b0580 2024-03-28T13:14:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.741+0000 7f0bbd400640 1 --2- 172.21.15.19:0/1769131648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bc00b0580 0x7f0bc00ae020 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0bac02b950 tx=0x7f0bac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.742+0000 7f0bb6000640 1 -- 172.21.15.19:0/1769131648 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac040070 con 0x7f0bc00b0580 2024-03-28T13:14:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.742+0000 7f0bb6000640 1 -- 172.21.15.19:0/1769131648 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bac03a420 con 0x7f0bc00b0580 2024-03-28T13:14:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.742+0000 7f0bb6000640 1 -- 172.21.15.19:0/1769131648 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac035040 con 0x7f0bc00b0580 2024-03-28T13:14:15.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.742+0000 7f0bbf200640 1 -- 172.21.15.19:0/1769131648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc00aec70 con 0x7f0bc00b0580 2024-03-28T13:14:15.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.742+0000 7f0bbf200640 1 -- 172.21.15.19:0/1769131648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc00ae560 con 0x7f0bc00b0580 2024-03-28T13:14:15.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.743+0000 7f0bbf200640 1 -- 172.21.15.19:0/1769131648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc0075940 con 0x7f0bc00b0580 2024-03-28T13:14:15.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.744+0000 7f0bb6000640 1 -- 172.21.15.19:0/1769131648 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0bac038020 con 0x7f0bc00b0580 2024-03-28T13:14:15.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.745+0000 7f0bb6000640 1 --2- 172.21.15.19:0/1769131648 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0ba006c530 0x7f0ba006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:15.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.745+0000 7f0bb6000640 1 -- 172.21.15.19:0/1769131648 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f0bac0ad730 con 0x7f0bc00b0580 2024-03-28T13:14:15.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.746+0000 7f0bbde00640 1 --2- 172.21.15.19:0/1769131648 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0ba006c530 0x7f0ba006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:15.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.747+0000 7f0bbde00640 1 --2- 172.21.15.19:0/1769131648 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0ba006c530 0x7f0ba006e9f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00c4be0 tx=0x7f0bb0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.750+0000 7f0bb6000640 1 -- 172.21.15.19:0/1769131648 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0bac07bd00 con 0x7f0bc00b0580 2024-03-28T13:14:15.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.954+0000 7f0bbf200640 1 -- 172.21.15.19:0/1769131648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bc006de30 con 0x7f0bc00b0580 2024-03-28T13:14:15.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.957+0000 7f0bb6000640 1 -- 172.21.15.19:0/1769131648 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+16761 (secure 0 0 0) 0x7f0bac07b600 con 0x7f0bc00b0580 2024-03-28T13:14:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:15.957 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":65,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:13:24.671229+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:12:50.888331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-28T13:12:51.946012+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.14","mappings":[{"from":6,"to":4}]},{"pgid":"9.2c","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:14:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.959+0000 7f0bbf200640 1 -- 172.21.15.19:0/1769131648 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0ba006c530 msgr2=0x7f0ba006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.959+0000 7f0bbf200640 1 --2- 172.21.15.19:0/1769131648 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0ba006c530 0x7f0ba006e9f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0bc00c4be0 tx=0x7f0bb0008040 comp rx=0 tx=0).stop 2024-03-28T13:14:15.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.959+0000 7f0bbf200640 1 -- 172.21.15.19:0/1769131648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bc00b0580 msgr2=0x7f0bc00ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.959+0000 7f0bbf200640 1 --2- 172.21.15.19:0/1769131648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bc00b0580 0x7f0bc00ae020 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0bac02b950 tx=0x7f0bac0046c0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.959+0000 7f0bbf200640 1 -- 172.21.15.19:0/1769131648 shutdown_connections 2024-03-28T13:14:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.959+0000 7f0bbf200640 1 --2- 172.21.15.19:0/1769131648 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0ba006c530 0x7f0ba006e9f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.959+0000 7f0bbf200640 1 --2- 172.21.15.19:0/1769131648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0bc015e6e0 0x7f0bc00b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.959+0000 7f0bbf200640 1 --2- 172.21.15.19:0/1769131648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0bc00b0580 0x7f0bc00ae020 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.959+0000 7f0bbf200640 1 --2- 172.21.15.19:0/1769131648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0bc006b570 0x7f0bc01f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.959+0000 7f0bbf200640 1 -- 172.21.15.19:0/1769131648 >> 172.21.15.19:0/1769131648 conn(0x7f0bc009aa20 msgr2=0x7f0bc006f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.959+0000 7f0bbf200640 1 -- 172.21.15.19:0/1769131648 shutdown_connections 2024-03-28T13:14:15.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:15.959+0000 7f0bbf200640 1 -- 172.21.15.19:0/1769131648 wait complete. 2024-03-28T13:14:15.982 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T13:14:16.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.212+0000 7f162f200640 1 Processor -- start 2024-03-28T13:14:16.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.213+0000 7f162f200640 1 -- start start 2024-03-28T13:14:16.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.214+0000 7f162f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f163015fc50 0x7f16300cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:16.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.214+0000 7f162f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f16300cc490 0x7f16300d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:16.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.214+0000 7f162f200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16300b3150 con 0x7f16300cc0b0 2024-03-28T13:14:16.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.214+0000 7f162f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f163015c610 con 0x7f16300cc490 2024-03-28T13:14:16.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.214+0000 7f162f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f16301600e0 con 0x7f163015fc50 2024-03-28T13:14:16.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.215+0000 7f162e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f16300cc490 0x7f16300d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:16.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.215+0000 7f162e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f16300cc490 0x7f16300d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50968/0 (socket says 172.21.15.19:50968) 2024-03-28T13:14:16.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.215+0000 7f162e800640 1 -- 172.21.15.19:0/2004158970 learned_addr learned my addr 172.21.15.19:0/2004158970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:16.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.215+0000 7f1625400640 1 --2- 172.21.15.19:0/2004158970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f163015fc50 0x7f16300cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:16.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.215+0000 7f162e800640 1 -- 172.21.15.19:0/2004158970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f163015fc50 msgr2=0x7f16300cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:16.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.215+0000 7f162e800640 1 --2- 172.21.15.19:0/2004158970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f163015fc50 0x7f16300cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.215+0000 7f162e800640 1 -- 172.21.15.19:0/2004158970 >> v1:172.21.15.188:6789/0 conn(0x7f16300cc0b0 legacy=0x7f163006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:16.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.215+0000 7f162e800640 1 -- 172.21.15.19:0/2004158970 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16301603c0 con 0x7f16300cc490 2024-03-28T13:14:16.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.215+0000 7f1625400640 1 --2- 172.21.15.19:0/2004158970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f163015fc50 0x7f16300cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:16.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.217+0000 7f162e800640 1 --2- 172.21.15.19:0/2004158970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16300cc490 0x7f16300d0300 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1628009870 tx=0x7f162802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4237b2502459c5a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:16.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.217+0000 7f162d400640 1 -- 172.21.15.19:0/2004158970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628034070 con 0x7f16300cc490 2024-03-28T13:14:16.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.217+0000 7f162d400640 1 -- 172.21.15.19:0/2004158970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1628030840 con 0x7f16300cc490 2024-03-28T13:14:16.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.217+0000 7f162d400640 1 -- 172.21.15.19:0/2004158970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628030c10 con 0x7f16300cc490 2024-03-28T13:14:16.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.218+0000 7f162f200640 1 -- 172.21.15.19:0/2004158970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16300cc490 msgr2=0x7f16300d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:16.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.218+0000 7f162f200640 1 --2- 172.21.15.19:0/2004158970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16300cc490 0x7f16300d0300 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1628009870 tx=0x7f162802b980 comp rx=0 tx=0).stop 2024-03-28T13:14:16.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.218+0000 7f162f200640 1 -- 172.21.15.19:0/2004158970 shutdown_connections 2024-03-28T13:14:16.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.218+0000 7f162f200640 1 --2- 172.21.15.19:0/2004158970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16300cc490 0x7f16300d0300 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.218+0000 7f162f200640 1 --2- 172.21.15.19:0/2004158970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f163015fc50 0x7f16300cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.218+0000 7f162f200640 1 -- 172.21.15.19:0/2004158970 >> 172.21.15.19:0/2004158970 conn(0x7f16300ccad0 msgr2=0x7f16300cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:16.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.218+0000 7f162f200640 1 -- 172.21.15.19:0/2004158970 shutdown_connections 2024-03-28T13:14:16.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.218+0000 7f162f200640 1 -- 172.21.15.19:0/2004158970 wait complete. 2024-03-28T13:14:16.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.220+0000 7f162f200640 1 Processor -- start 2024-03-28T13:14:16.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.220+0000 7f162f200640 1 -- start start 2024-03-28T13:14:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.221+0000 7f162f200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16300cc0b0 0x7f16300b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.221+0000 7f162f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16300cc490 0x7f1630157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.221+0000 7f162f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1630137200 0x7f163013b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.221+0000 7f162f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f163015ca70 con 0x7f16300cc490 2024-03-28T13:14:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.221+0000 7f162f200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1630160980 con 0x7f16300cc0b0 2024-03-28T13:14:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.221+0000 7f162f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f163015cbf0 con 0x7f1630137200 2024-03-28T13:14:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.221+0000 7f1625400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16300cc490 0x7f1630157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.221+0000 7f1625400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16300cc490 0x7f1630157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50984/0 (socket says 172.21.15.19:50984) 2024-03-28T13:14:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.221+0000 7f1625400640 1 -- 172.21.15.19:0/4177408445 learned_addr learned my addr 172.21.15.19:0/4177408445 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.221+0000 7f162e800640 1 --2- 172.21.15.19:0/4177408445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1630137200 0x7f163013b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:16.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.221+0000 7f162de00640 1 --2- 172.21.15.19:0/4177408445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16300cc0b0 0x7f16300b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:16.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.222+0000 7f1625400640 1 -- 172.21.15.19:0/4177408445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1630137200 msgr2=0x7f163013b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.222+0000 7f1625400640 1 --2- 172.21.15.19:0/4177408445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1630137200 0x7f163013b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.222+0000 7f1625400640 1 -- 172.21.15.19:0/4177408445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16300cc0b0 msgr2=0x7f16300b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.222+0000 7f1625400640 1 --2- 172.21.15.19:0/4177408445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16300cc0b0 0x7f16300b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.222+0000 7f1625400640 1 -- 172.21.15.19:0/4177408445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163013a560 con 0x7f16300cc490 2024-03-28T13:14:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.222+0000 7f162de00640 1 --2- 172.21.15.19:0/4177408445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16300cc0b0 0x7f16300b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:14:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.222+0000 7f1625400640 1 --2- 172.21.15.19:0/4177408445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16300cc490 0x7f1630157e80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f161800c7f0 tx=0x7f161800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.222+0000 7f1626a00640 1 -- 172.21.15.19:0/4177408445 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618015070 con 0x7f16300cc490 2024-03-28T13:14:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.223+0000 7f1626a00640 1 -- 172.21.15.19:0/4177408445 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1618007820 con 0x7f16300cc490 2024-03-28T13:14:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.223+0000 7f162f200640 1 -- 172.21.15.19:0/4177408445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163015af60 con 0x7f16300cc490 2024-03-28T13:14:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.223+0000 7f1626a00640 1 -- 172.21.15.19:0/4177408445 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618007bd0 con 0x7f16300cc490 2024-03-28T13:14:16.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.223+0000 7f162f200640 1 -- 172.21.15.19:0/4177408445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f163015b4e0 con 0x7f16300cc490 2024-03-28T13:14:16.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.224+0000 7f162f200640 1 -- 172.21.15.19:0/4177408445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1630075940 con 0x7f16300cc490 2024-03-28T13:14:16.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.226+0000 7f1626a00640 1 -- 172.21.15.19:0/4177408445 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1618018070 con 0x7f16300cc490 2024-03-28T13:14:16.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.227+0000 7f1626a00640 1 --2- 172.21.15.19:0/4177408445 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160c06c560 0x7f160c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:16.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.227+0000 7f1626a00640 1 -- 172.21.15.19:0/4177408445 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5767+0+0 (secure 0 0 0) 0x7f161808b7d0 con 0x7f16300cc490 2024-03-28T13:14:16.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.227+0000 7f162de00640 1 --2- 172.21.15.19:0/4177408445 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160c06c560 0x7f160c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:16.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.229+0000 7f162de00640 1 --2- 172.21.15.19:0/4177408445 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160c06c560 0x7f160c06ea20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1620001e00 tx=0x7f1620009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:16.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.232+0000 7f1626a00640 1 -- 172.21.15.19:0/4177408445 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f16180596c0 con 0x7f16300cc490 2024-03-28T13:14:16.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.480+0000 7f162f200640 1 -- 172.21.15.19:0/4177408445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f163006f2a0 con 0x7f16300cc490 2024-03-28T13:14:16.480 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:14:16.481+0000 7f6b74400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T13:14:16.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.484+0000 7f1626a00640 1 -- 172.21.15.19:0/4177408445 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) ==== 161+0+0 (secure 0 0 0) 0x7f161805d9c0 con 0x7f16300cc490 2024-03-28T13:14:16.484 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T13:14:16.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.488+0000 7f162f200640 1 -- 172.21.15.19:0/4177408445 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160c06c560 msgr2=0x7f160c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:16.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.488+0000 7f162f200640 1 --2- 172.21.15.19:0/4177408445 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160c06c560 0x7f160c06ea20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1620001e00 tx=0x7f1620009040 comp rx=0 tx=0).stop 2024-03-28T13:14:16.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.488+0000 7f162f200640 1 -- 172.21.15.19:0/4177408445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16300cc490 msgr2=0x7f1630157e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:16.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.488+0000 7f162f200640 1 --2- 172.21.15.19:0/4177408445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16300cc490 0x7f1630157e80 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f161800c7f0 tx=0x7f161800ccb0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.489+0000 7f162f200640 1 -- 172.21.15.19:0/4177408445 shutdown_connections 2024-03-28T13:14:16.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.489+0000 7f162f200640 1 --2- 172.21.15.19:0/4177408445 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160c06c560 0x7f160c06ea20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.489+0000 7f162f200640 1 --2- 172.21.15.19:0/4177408445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1630137200 0x7f163013b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.489+0000 7f162f200640 1 --2- 172.21.15.19:0/4177408445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16300cc490 0x7f1630157e80 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.489+0000 7f162f200640 1 --2- 172.21.15.19:0/4177408445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16300cc0b0 0x7f16300b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.489+0000 7f162f200640 1 -- 172.21.15.19:0/4177408445 >> 172.21.15.19:0/4177408445 conn(0x7f16300ccad0 msgr2=0x7f16300727a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:16.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.491+0000 7f162f200640 1 -- 172.21.15.19:0/4177408445 shutdown_connections 2024-03-28T13:14:16.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.491+0000 7f162f200640 1 -- 172.21.15.19:0/4177408445 wait complete. 2024-03-28T13:14:16.500 INFO:tasks.ceph.mds.a.smithi019.stderr: -7603> 2024-03-28T13:12:57.878+0000 7fdf43600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:14:16.503 INFO:tasks.ceph.mds.a.smithi019.stderr: -5011> 2024-03-28T13:13:03.868+0000 7fdf3e600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={6111=0-4194304@1} caps={6111=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6111 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55cd5fb58c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-28T13:14:16.506 INFO:tasks.ceph.mds.a.smithi019.stderr: -3021> 2024-03-28T13:13:36.078+0000 7fdf3e600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-03-28T13:13:02.306279+0000 1=1+0) caps={6111=pAsLsXsFs/-@2},l=6111(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x55cd5fb58c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-28T13:14:16.515 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-28T13:14:16.555 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-03-28T13:14:16.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.705+0000 7fa66e200640 1 Processor -- start 2024-03-28T13:14:16.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.705+0000 7fa66e200640 1 -- start start 2024-03-28T13:14:16.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.706+0000 7fa66e200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6680d0280 0x7fa6680d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:16.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.707+0000 7fa66e200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa668162160 0x7fa6681625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:16.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.707+0000 7fa66e200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa66815ec80 con 0x7fa668141c10 2024-03-28T13:14:16.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.707+0000 7fa66e200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa6680d0ba0 con 0x7fa668162160 2024-03-28T13:14:16.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.707+0000 7fa66e200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa66815ef20 con 0x7fa6680d0280 2024-03-28T13:14:16.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.707+0000 7fa667400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6680d0280 0x7fa6680d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:16.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.707+0000 7fa667e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa668162160 0x7fa6681625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:16.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.707+0000 7fa667e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa668162160 0x7fa6681625e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:35570/0 (socket says 172.21.15.19:35570) 2024-03-28T13:14:16.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.707+0000 7fa667e00640 1 -- 172.21.15.19:0/959398434 learned_addr learned my addr 172.21.15.19:0/959398434 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:16.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.708+0000 7fa667400640 1 -- 172.21.15.19:0/959398434 >> v2:172.21.15.188:3300/0 conn(0x7fa668162160 msgr2=0x7fa6681625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:16.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.708+0000 7fa667400640 1 --2- 172.21.15.19:0/959398434 >> v2:172.21.15.188:3300/0 conn(0x7fa668162160 0x7fa6681625e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.708+0000 7fa667400640 1 -- 172.21.15.19:0/959398434 >> v1:172.21.15.188:6789/0 conn(0x7fa668141c10 legacy=0x7fa668071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:16.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.708+0000 7fa667400640 1 -- 172.21.15.19:0/959398434 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66815f200 con 0x7fa6680d0280 2024-03-28T13:14:16.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.708+0000 7fa667e00640 1 --2- 172.21.15.19:0/959398434 >> v2:172.21.15.188:3300/0 conn(0x7fa668162160 0x7fa6681625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:14:16.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.709+0000 7fa667400640 1 --2- 172.21.15.19:0/959398434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6680d0280 0x7fa6680d0660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa6580097a0 tx=0x7fa65802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19d3325389cda2c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:16.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.710+0000 7fa666000640 1 -- 172.21.15.19:0/959398434 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658035020 con 0x7fa6680d0280 2024-03-28T13:14:16.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.710+0000 7fa666000640 1 -- 172.21.15.19:0/959398434 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa658031810 con 0x7fa6680d0280 2024-03-28T13:14:16.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.710+0000 7fa666000640 1 -- 172.21.15.19:0/959398434 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658031be0 con 0x7fa6680d0280 2024-03-28T13:14:16.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.710+0000 7fa66e200640 1 -- 172.21.15.19:0/959398434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6680d0280 msgr2=0x7fa6680d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:16.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.710+0000 7fa66e200640 1 --2- 172.21.15.19:0/959398434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6680d0280 0x7fa6680d0660 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa6580097a0 tx=0x7fa65802b8b0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.711+0000 7fa66e200640 1 -- 172.21.15.19:0/959398434 shutdown_connections 2024-03-28T13:14:16.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.711+0000 7fa66e200640 1 --2- 172.21.15.19:0/959398434 >> v2:172.21.15.188:3300/0 conn(0x7fa668162160 0x7fa6681625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.711+0000 7fa66e200640 1 --2- 172.21.15.19:0/959398434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6680d0280 0x7fa6680d0660 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.711+0000 7fa66e200640 1 -- 172.21.15.19:0/959398434 >> 172.21.15.19:0/959398434 conn(0x7fa66806b750 msgr2=0x7fa66806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:16.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.711+0000 7fa66e200640 1 -- 172.21.15.19:0/959398434 shutdown_connections 2024-03-28T13:14:16.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.711+0000 7fa66e200640 1 -- 172.21.15.19:0/959398434 wait complete. 2024-03-28T13:14:16.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.712+0000 7fa66e200640 1 Processor -- start 2024-03-28T13:14:16.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.713+0000 7fa66e200640 1 -- start start 2024-03-28T13:14:16.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.714+0000 7fa66e200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6680d0280 0x7fa668157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:16.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.714+0000 7fa66e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa668141c10 0x7fa66815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:16.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.714+0000 7fa66e200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa668162160 0x7fa66815ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:16.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.714+0000 7fa66e200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa668158e90 con 0x7fa668141c10 2024-03-28T13:14:16.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.714+0000 7fa66e200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa66815f7c0 con 0x7fa668162160 2024-03-28T13:14:16.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.714+0000 7fa66e200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa66809d730 con 0x7fa6680d0280 2024-03-28T13:14:16.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.714+0000 7fa666a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa668141c10 0x7fa66815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:16.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.714+0000 7fa666a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa668141c10 0x7fa66815a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51012/0 (socket says 172.21.15.19:51012) 2024-03-28T13:14:16.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.714+0000 7fa666a00640 1 -- 172.21.15.19:0/1125972454 learned_addr learned my addr 172.21.15.19:0/1125972454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:16.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.714+0000 7fa667e00640 1 --2- 172.21.15.19:0/1125972454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa668162160 0x7fa66815ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:16.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.714+0000 7fa667400640 1 --2- 172.21.15.19:0/1125972454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6680d0280 0x7fa668157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.715+0000 7fa666a00640 1 -- 172.21.15.19:0/1125972454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6680d0280 msgr2=0x7fa668157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.715+0000 7fa666a00640 1 --2- 172.21.15.19:0/1125972454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6680d0280 0x7fa668157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.715+0000 7fa666a00640 1 -- 172.21.15.19:0/1125972454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa668162160 msgr2=0x7fa66815ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.715+0000 7fa666a00640 1 --2- 172.21.15.19:0/1125972454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa668162160 0x7fa66815ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.715+0000 7fa666a00640 1 -- 172.21.15.19:0/1125972454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66809d450 con 0x7fa668141c10 2024-03-28T13:14:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.715+0000 7fa666a00640 1 --2- 172.21.15.19:0/1125972454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa668141c10 0x7fa66815a7a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa65400c3d0 tx=0x7fa65400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.715+0000 7fa65f400640 1 -- 172.21.15.19:0/1125972454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6540052d0 con 0x7fa668141c10 2024-03-28T13:14:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.715+0000 7fa65f400640 1 -- 172.21.15.19:0/1125972454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa654005490 con 0x7fa668141c10 2024-03-28T13:14:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.715+0000 7fa66e200640 1 -- 172.21.15.19:0/1125972454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66815d480 con 0x7fa668141c10 2024-03-28T13:14:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.715+0000 7fa65f400640 1 -- 172.21.15.19:0/1125972454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654007260 con 0x7fa668141c10 2024-03-28T13:14:16.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.715+0000 7fa66e200640 1 -- 172.21.15.19:0/1125972454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa66815da40 con 0x7fa668141c10 2024-03-28T13:14:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.716+0000 7fa66e200640 1 -- 172.21.15.19:0/1125972454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6680685c0 con 0x7fa668141c10 2024-03-28T13:14:16.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.717+0000 7fa65f400640 1 -- 172.21.15.19:0/1125972454 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa654007400 con 0x7fa668141c10 2024-03-28T13:14:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.718+0000 7fa65f400640 1 --2- 172.21.15.19:0/1125972454 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa64806c530 0x7fa64806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:16.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.718+0000 7fa65f400640 1 -- 172.21.15.19:0/1125972454 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5853+0+0 (secure 0 0 0) 0x7fa65408acd0 con 0x7fa668141c10 2024-03-28T13:14:16.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.719+0000 7fa667400640 1 --2- 172.21.15.19:0/1125972454 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa64806c530 0x7fa64806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:16.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.720+0000 7fa667400640 1 --2- 172.21.15.19:0/1125972454 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa64806c530 0x7fa64806e9f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa658036000 tx=0x7fa658008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:16.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.723+0000 7fa65f400640 1 -- 172.21.15.19:0/1125972454 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa6540591d0 con 0x7fa668141c10 2024-03-28T13:14:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:16.966+0000 7fa66e200640 1 -- 172.21.15.19:0/1125972454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa66815dc10 con 0x7fa668141c10 2024-03-28T13:14:17.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.487+0000 7fa65f400640 1 -- 172.21.15.19:0/1125972454 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7fa654058ad0 con 0x7fa668141c10 2024-03-28T13:14:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.491+0000 7fa66e200640 1 -- 172.21.15.19:0/1125972454 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa64806c530 msgr2=0x7fa64806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.491+0000 7fa66e200640 1 --2- 172.21.15.19:0/1125972454 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa64806c530 0x7fa64806e9f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa658036000 tx=0x7fa658008030 comp rx=0 tx=0).stop 2024-03-28T13:14:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.491+0000 7fa66e200640 1 -- 172.21.15.19:0/1125972454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa668141c10 msgr2=0x7fa66815a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.491+0000 7fa66e200640 1 --2- 172.21.15.19:0/1125972454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa668141c10 0x7fa66815a7a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa65400c3d0 tx=0x7fa65400c890 comp rx=0 tx=0).stop 2024-03-28T13:14:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.491+0000 7fa66e200640 1 -- 172.21.15.19:0/1125972454 shutdown_connections 2024-03-28T13:14:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.491+0000 7fa66e200640 1 --2- 172.21.15.19:0/1125972454 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa64806c530 0x7fa64806e9f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.491+0000 7fa66e200640 1 --2- 172.21.15.19:0/1125972454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa668162160 0x7fa66815ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.491+0000 7fa66e200640 1 --2- 172.21.15.19:0/1125972454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa668141c10 0x7fa66815a7a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.491+0000 7fa66e200640 1 --2- 172.21.15.19:0/1125972454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6680d0280 0x7fa668157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.491+0000 7fa66e200640 1 -- 172.21.15.19:0/1125972454 >> 172.21.15.19:0/1125972454 conn(0x7fa66806b750 msgr2=0x7fa6680b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:17.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.491+0000 7fa66e200640 1 -- 172.21.15.19:0/1125972454 shutdown_connections 2024-03-28T13:14:17.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.491+0000 7fa66e200640 1 -- 172.21.15.19:0/1125972454 wait complete. 2024-03-28T13:14:17.514 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-28T13:14:17.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.787+0000 7f8cbc400640 1 Processor -- start 2024-03-28T13:14:17.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.787+0000 7f8cbc400640 1 -- start start 2024-03-28T13:14:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.789+0000 7f8cbc400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cb40acf10 0x7f8cb409dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.789+0000 7f8cbc400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cb40ad2f0 0x7f8cb41453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.789+0000 7f8cbc400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cb415cb90 con 0x7f8cb415ff70 2024-03-28T13:14:17.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.789+0000 7f8cbc400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8cb4145f60 con 0x7f8cb40acf10 2024-03-28T13:14:17.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.789+0000 7f8cbc400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb415c950 con 0x7f8cb40ad2f0 2024-03-28T13:14:17.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.789+0000 7f8cb9a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cb40acf10 0x7f8cb409dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:17.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.789+0000 7f8cb9a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8cb40acf10 0x7f8cb409dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51020/0 (socket says 172.21.15.19:51020) 2024-03-28T13:14:17.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.789+0000 7f8cb9a00640 1 -- 172.21.15.19:0/2995908991 learned_addr learned my addr 172.21.15.19:0/2995908991 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:17.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.790+0000 7f8cb9000640 1 --2- 172.21.15.19:0/2995908991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cb40ad2f0 0x7f8cb41453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:17.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.790+0000 7f8cb9a00640 1 -- 172.21.15.19:0/2995908991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cb40ad2f0 msgr2=0x7f8cb41453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:17.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.790+0000 7f8cb9a00640 1 --2- 172.21.15.19:0/2995908991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cb40ad2f0 0x7f8cb41453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:17.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.790+0000 7f8cb9a00640 1 -- 172.21.15.19:0/2995908991 >> v1:172.21.15.188:6789/0 conn(0x7f8cb415ff70 legacy=0x7f8cb41603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:17.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.790+0000 7f8cb9a00640 1 -- 172.21.15.19:0/2995908991 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb415ce70 con 0x7f8cb40acf10 2024-03-28T13:14:17.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.792+0000 7f8cb9a00640 1 --2- 172.21.15.19:0/2995908991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40acf10 0x7f8cb409dcf0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009870 tx=0x7f8ca802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446e6f6c4af1c602 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:17.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.792+0000 7f8cb3e00640 1 -- 172.21.15.19:0/2995908991 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80317c0 con 0x7f8cb40acf10 2024-03-28T13:14:17.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.792+0000 7f8cb3e00640 1 -- 172.21.15.19:0/2995908991 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ca8037070 con 0x7f8cb40acf10 2024-03-28T13:14:17.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.792+0000 7f8cb3e00640 1 -- 172.21.15.19:0/2995908991 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8031b80 con 0x7f8cb40acf10 2024-03-28T13:14:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.793+0000 7f8cbc400640 1 -- 172.21.15.19:0/2995908991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40acf10 msgr2=0x7f8cb409dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.793+0000 7f8cbc400640 1 --2- 172.21.15.19:0/2995908991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40acf10 0x7f8cb409dcf0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009870 tx=0x7f8ca802b860 comp rx=0 tx=0).stop 2024-03-28T13:14:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.793+0000 7f8cbc400640 1 -- 172.21.15.19:0/2995908991 shutdown_connections 2024-03-28T13:14:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.793+0000 7f8cbc400640 1 --2- 172.21.15.19:0/2995908991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cb40ad2f0 0x7f8cb41453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.793+0000 7f8cbc400640 1 --2- 172.21.15.19:0/2995908991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40acf10 0x7f8cb409dcf0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.793+0000 7f8cbc400640 1 -- 172.21.15.19:0/2995908991 >> 172.21.15.19:0/2995908991 conn(0x7f8cb406bba0 msgr2=0x7f8cb40b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.793+0000 7f8cbc400640 1 -- 172.21.15.19:0/2995908991 shutdown_connections 2024-03-28T13:14:17.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.793+0000 7f8cbc400640 1 -- 172.21.15.19:0/2995908991 wait complete. 2024-03-28T13:14:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.796+0000 7f8cbc400640 1 Processor -- start 2024-03-28T13:14:17.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.796+0000 7f8cbc400640 1 -- start start 2024-03-28T13:14:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.797+0000 7f8cbc400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cb40acf10 0x7f8cb413af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.797+0000 7f8cbc400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40ad2f0 0x7f8cb40b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.797+0000 7f8cbc400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cb415ff70 0x7f8cb40aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.797+0000 7f8cbc400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb415d430 con 0x7f8cb40ad2f0 2024-03-28T13:14:17.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.797+0000 7f8cbc400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb415d7a0 con 0x7f8cb415ff70 2024-03-28T13:14:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.797+0000 7f8cbc400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8cb40ad770 con 0x7f8cb40acf10 2024-03-28T13:14:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.797+0000 7f8cb9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40ad2f0 0x7f8cb40b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.797+0000 7f8cb9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40ad2f0 0x7f8cb40b1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51036/0 (socket says 172.21.15.19:51036) 2024-03-28T13:14:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.797+0000 7f8cb9000640 1 -- 172.21.15.19:0/2528946739 learned_addr learned my addr 172.21.15.19:0/2528946739 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.798+0000 7f8cb9000640 1 -- 172.21.15.19:0/2528946739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cb40acf10 msgr2=0x7f8cb413af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.798+0000 7f8cb9000640 1 --2- 172.21.15.19:0/2528946739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cb40acf10 0x7f8cb413af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.798+0000 7f8cb9000640 1 -- 172.21.15.19:0/2528946739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cb415ff70 msgr2=0x7f8cb40aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.798+0000 7f8cb9000640 1 --2- 172.21.15.19:0/2528946739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cb415ff70 0x7f8cb40aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.798+0000 7f8cb9000640 1 -- 172.21.15.19:0/2528946739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb413a4b0 con 0x7f8cb40ad2f0 2024-03-28T13:14:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.798+0000 7f8cb9000640 1 --2- 172.21.15.19:0/2528946739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40ad2f0 0x7f8cb40b1c80 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000d800 tx=0x7f8ca000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:17.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.798+0000 7f8cb1600640 1 -- 172.21.15.19:0/2528946739 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca000b6c0 con 0x7f8cb40ad2f0 2024-03-28T13:14:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.798+0000 7f8cb1600640 1 -- 172.21.15.19:0/2528946739 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ca0019070 con 0x7f8cb40ad2f0 2024-03-28T13:14:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.798+0000 7f8cb1600640 1 -- 172.21.15.19:0/2528946739 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca000ba80 con 0x7f8cb40ad2f0 2024-03-28T13:14:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.799+0000 7f8cbc400640 1 -- 172.21.15.19:0/2528946739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb41525c0 con 0x7f8cb40ad2f0 2024-03-28T13:14:17.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.799+0000 7f8cbc400640 1 -- 172.21.15.19:0/2528946739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb413a720 con 0x7f8cb40ad2f0 2024-03-28T13:14:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.800+0000 7f8cbc400640 1 -- 172.21.15.19:0/2528946739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb40685c0 con 0x7f8cb40ad2f0 2024-03-28T13:14:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.801+0000 7f8cb1600640 1 -- 172.21.15.19:0/2528946739 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8ca000bc20 con 0x7f8cb40ad2f0 2024-03-28T13:14:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.802+0000 7f8cb1600640 1 --2- 172.21.15.19:0/2528946739 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8c9406c530 0x7f8c9406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:17.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.802+0000 7f8cb1600640 1 -- 172.21.15.19:0/2528946739 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5853+0+0 (secure 0 0 0) 0x7f8ca008bc30 con 0x7f8cb40ad2f0 2024-03-28T13:14:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.802+0000 7f8cb9a00640 1 --2- 172.21.15.19:0/2528946739 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8c9406c530 0x7f8c9406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:17.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.803+0000 7f8cb9a00640 1 --2- 172.21.15.19:0/2528946739 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8c9406c530 0x7f8c9406e9f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8036000 tx=0x7f8ca8030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:17.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:17.806+0000 7f8cb1600640 1 -- 172.21.15.19:0/2528946739 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8cb41525c0 con 0x7f8cb40ad2f0 2024-03-28T13:14:18.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:18.007+0000 7f8cbc400640 1 -- 172.21.15.19:0/2528946739 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8cb40b2900 con 0x7f8cb40ad2f0 2024-03-28T13:14:18.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:18.914+0000 7f8cb1600640 1 -- 172.21.15.19:0/2528946739 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) ==== 181+0+0 (secure 0 0 0) 0x7f8ca005a1c0 con 0x7f8cb40ad2f0 2024-03-28T13:14:18.914 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-03-28T13:14:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:18.917+0000 7f8cbc400640 1 -- 172.21.15.19:0/2528946739 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8c9406c530 msgr2=0x7f8c9406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:18.917+0000 7f8cbc400640 1 --2- 172.21.15.19:0/2528946739 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8c9406c530 0x7f8c9406e9f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8036000 tx=0x7f8ca8030000 comp rx=0 tx=0).stop 2024-03-28T13:14:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:18.917+0000 7f8cbc400640 1 -- 172.21.15.19:0/2528946739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40ad2f0 msgr2=0x7f8cb40b1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:18.917+0000 7f8cbc400640 1 --2- 172.21.15.19:0/2528946739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40ad2f0 0x7f8cb40b1c80 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000d800 tx=0x7f8ca000dcc0 comp rx=0 tx=0).stop 2024-03-28T13:14:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:18.918+0000 7f8cbc400640 1 -- 172.21.15.19:0/2528946739 shutdown_connections 2024-03-28T13:14:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:18.918+0000 7f8cbc400640 1 --2- 172.21.15.19:0/2528946739 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8c9406c530 0x7f8c9406e9f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:18.918+0000 7f8cbc400640 1 --2- 172.21.15.19:0/2528946739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cb415ff70 0x7f8cb40aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:18.918+0000 7f8cbc400640 1 --2- 172.21.15.19:0/2528946739 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb40ad2f0 0x7f8cb40b1c80 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:18.918+0000 7f8cbc400640 1 --2- 172.21.15.19:0/2528946739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cb40acf10 0x7f8cb413af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:18.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:18.918+0000 7f8cbc400640 1 -- 172.21.15.19:0/2528946739 >> 172.21.15.19:0/2528946739 conn(0x7f8cb406bba0 msgr2=0x7f8cb415e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:18.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:18.918+0000 7f8cbc400640 1 -- 172.21.15.19:0/2528946739 shutdown_connections 2024-03-28T13:14:18.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:18.918+0000 7f8cbc400640 1 -- 172.21.15.19:0/2528946739 wait complete. 2024-03-28T13:14:18.942 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-28T13:14:19.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.146+0000 7fb2cb200640 1 Processor -- start 2024-03-28T13:14:19.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.147+0000 7fb2cb200640 1 -- start start 2024-03-28T13:14:19.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.148+0000 7fb2cb200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2cc1637f0 0x7fb2cc16d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.148+0000 7fb2cb200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2cc072e50 con 0x7fb2cc06ea70 2024-03-28T13:14:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.148+0000 7fb2cb200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2cc073130 con 0x7fb2cc162d00 2024-03-28T13:14:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.148+0000 7fb2cb200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb2cc06d5d0 con 0x7fb2cc1637f0 2024-03-28T13:14:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.148+0000 7fb2c3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb2cc06ea70 0x7fb2cc1630e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49340/0 (socket says 172.21.15.19:49340) 2024-03-28T13:14:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.148+0000 7fb2c3e00640 1 -- 172.21.15.19:0/886905033 learned_addr learned my addr 172.21.15.19:0/886905033 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2ca800640 1 --2- 172.21.15.19:0/886905033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2cc1637f0 0x7fb2cc16d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2c9400640 1 -- 172.21.15.19:0/886905033 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4276052406 0 0) 0x7fb2cc072e50 con 0x7fb2cc06ea70 2024-03-28T13:14:19.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2c9400640 1 -- 172.21.15.19:0/886905033 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2b0003520 con 0x7fb2cc06ea70 2024-03-28T13:14:19.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2c9400640 1 -- 172.21.15.19:0/886905033 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3270897736 0 0) 0x7fb2b0003520 con 0x7fb2cc06ea70 2024-03-28T13:14:19.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2c9400640 1 -- 172.21.15.19:0/886905033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2cc1637f0 msgr2=0x7fb2cc16d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:19.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2c9400640 1 --2- 172.21.15.19:0/886905033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2cc1637f0 0x7fb2cc16d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:19.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2c9400640 1 -- 172.21.15.19:0/886905033 >> v1:172.21.15.188:6789/0 conn(0x7fb2cc162d00 legacy=0x7fb2cc072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:19.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2c9400640 1 -- 172.21.15.19:0/886905033 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2cc073410 con 0x7fb2cc06ea70 2024-03-28T13:14:19.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2c9400640 1 -- 172.21.15.19:0/886905033 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fb2b4001670 con 0x7fb2cc06ea70 2024-03-28T13:14:19.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2c9400640 1 -- 172.21.15.19:0/886905033 >> v1:172.21.15.19:6789/0 conn(0x7fb2cc06ea70 legacy=0x7fb2cc1630e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:19.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2c9400640 1 --2- 172.21.15.19:0/886905033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2b00039e0 0x7fb2b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:19.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2c9400640 1 --2- 172.21.15.19:0/886905033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2b0004800 0x7fb2b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2c9400640 1 --2- 172.21.15.19:0/886905033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2cc1637f0 0x7fb2b0005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2c9400640 1 -- 172.21.15.19:0/886905033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc06d5d0 con 0x7fb2b0004800 2024-03-28T13:14:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2c9400640 1 -- 172.21.15.19:0/886905033 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b000bb50 con 0x7fb2b00039e0 2024-03-28T13:14:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.149+0000 7fb2c9400640 1 -- 172.21.15.19:0/886905033 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b000bd50 con 0x7fb2cc1637f0 2024-03-28T13:14:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.150+0000 7fb2c3e00640 1 --2- 172.21.15.19:0/886905033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2b0004800 0x7fb2b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.150+0000 7fb2c9e00640 1 --2- 172.21.15.19:0/886905033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2b00039e0 0x7fb2b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.150+0000 7fb2ca800640 1 --2- 172.21.15.19:0/886905033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2cc1637f0 0x7fb2b0005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.150+0000 7fb2c3e00640 1 -- 172.21.15.19:0/886905033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2cc1637f0 msgr2=0x7fb2b0005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.150+0000 7fb2c3e00640 1 --2- 172.21.15.19:0/886905033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2cc1637f0 0x7fb2b0005250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.150+0000 7fb2c3e00640 1 -- 172.21.15.19:0/886905033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2b00039e0 msgr2=0x7fb2b0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.150+0000 7fb2c3e00640 1 --2- 172.21.15.19:0/886905033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2b00039e0 0x7fb2b0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.150+0000 7fb2c3e00640 1 -- 172.21.15.19:0/886905033 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b0003520 con 0x7fb2b0004800 2024-03-28T13:14:19.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.151+0000 7fb2c3e00640 1 --2- 172.21.15.19:0/886905033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2b0004800 0x7fb2b0004c80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400c190 tx=0x7fb2b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d261444a98f50c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.152+0000 7fb2c9400640 1 -- 172.21.15.19:0/886905033 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4007c60 con 0x7fb2b0004800 2024-03-28T13:14:19.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.152+0000 7fb2c9400640 1 -- 172.21.15.19:0/886905033 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b4030070 con 0x7fb2b0004800 2024-03-28T13:14:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.152+0000 7fb2c9400640 1 -- 172.21.15.19:0/886905033 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4005d70 con 0x7fb2b0004800 2024-03-28T13:14:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.152+0000 7fb2cb200640 1 -- 172.21.15.19:0/886905033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2b0004800 msgr2=0x7fb2b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.152+0000 7fb2cb200640 1 --2- 172.21.15.19:0/886905033 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2b0004800 0x7fb2b0004c80 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400c190 tx=0x7fb2b0005830 comp rx=0 tx=0).stop 2024-03-28T13:14:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.152+0000 7fb2ca800640 1 -- 172.21.15.19:0/886905033 reap_dead start 2024-03-28T13:14:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.152+0000 7fb2cb200640 1 -- 172.21.15.19:0/886905033 shutdown_connections 2024-03-28T13:14:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.152+0000 7fb2cb200640 1 -- 172.21.15.19:0/886905033 >> 172.21.15.19:0/886905033 conn(0x7fb2cc0b1d80 msgr2=0x7fb2cc0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.152+0000 7fb2cb200640 1 -- 172.21.15.19:0/886905033 shutdown_connections 2024-03-28T13:14:19.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.153+0000 7fb2cb200640 1 -- 172.21.15.19:0/886905033 wait complete. 2024-03-28T13:14:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.154+0000 7fb2cb200640 1 Processor -- start 2024-03-28T13:14:19.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.155+0000 7fb2cb200640 1 -- start start 2024-03-28T13:14:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.155+0000 7fb2cb200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2b0004800 0x7fb2cc1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.155+0000 7fb2cb200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2cc15b460 0x7fb2cc15b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.156+0000 7fb2cb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2cc1583b0 0x7fb2cc158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.156+0000 7fb2cb200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc0b3a70 con 0x7fb2cc1583b0 2024-03-28T13:14:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.156+0000 7fb2cb200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb2cc072b30 con 0x7fb2cc15b460 2024-03-28T13:14:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.156+0000 7fb2cb200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb2cc0b3bf0 con 0x7fb2b0004800 2024-03-28T13:14:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.156+0000 7fb2ca800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2cc1583b0 0x7fb2cc158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.156+0000 7fb2ca800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2cc1583b0 0x7fb2cc158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51060/0 (socket says 172.21.15.19:51060) 2024-03-28T13:14:19.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.156+0000 7fb2ca800640 1 -- 172.21.15.19:0/3661783746 learned_addr learned my addr 172.21.15.19:0/3661783746 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.156+0000 7fb2c9e00640 1 --2- 172.21.15.19:0/3661783746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2b0004800 0x7fb2cc1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.156+0000 7fb2c3e00640 1 --2- 172.21.15.19:0/3661783746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2cc15b460 0x7fb2cc15b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.156+0000 7fb2ca800640 1 -- 172.21.15.19:0/3661783746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2b0004800 msgr2=0x7fb2cc1529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.156+0000 7fb2ca800640 1 --2- 172.21.15.19:0/3661783746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2b0004800 0x7fb2cc1529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.156+0000 7fb2ca800640 1 -- 172.21.15.19:0/3661783746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2cc15b460 msgr2=0x7fb2cc15b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.156+0000 7fb2ca800640 1 --2- 172.21.15.19:0/3661783746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2cc15b460 0x7fb2cc15b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.156+0000 7fb2ca800640 1 -- 172.21.15.19:0/3661783746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2cc1590e0 con 0x7fb2cc1583b0 2024-03-28T13:14:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.156+0000 7fb2c3e00640 1 --2- 172.21.15.19:0/3661783746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2cc15b460 0x7fb2cc15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.157+0000 7fb2ca800640 1 --2- 172.21.15.19:0/3661783746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2cc1583b0 0x7fb2cc158830 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc071210 tx=0x7fb2c4009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.157+0000 7fb2c2000640 1 -- 172.21.15.19:0/3661783746 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4019070 con 0x7fb2cc1583b0 2024-03-28T13:14:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.157+0000 7fb2c2000640 1 -- 172.21.15.19:0/3661783746 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2c4007d40 con 0x7fb2cc1583b0 2024-03-28T13:14:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.157+0000 7fb2cb200640 1 -- 172.21.15.19:0/3661783746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2cc1593c0 con 0x7fb2cc1583b0 2024-03-28T13:14:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.157+0000 7fb2c2000640 1 -- 172.21.15.19:0/3661783746 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4014640 con 0x7fb2cc1583b0 2024-03-28T13:14:19.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.157+0000 7fb2cb200640 1 -- 172.21.15.19:0/3661783746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2cc159980 con 0x7fb2cc1583b0 2024-03-28T13:14:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.158+0000 7fb2cb200640 1 -- 172.21.15.19:0/3661783746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2cc075940 con 0x7fb2cc1583b0 2024-03-28T13:14:19.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.159+0000 7fb2c2000640 1 -- 172.21.15.19:0/3661783746 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb2c401c070 con 0x7fb2cc1583b0 2024-03-28T13:14:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.160+0000 7fb2c2000640 1 --2- 172.21.15.19:0/3661783746 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb2a806c560 0x7fb2a806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:19.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.160+0000 7fb2c2000640 1 -- 172.21.15.19:0/3661783746 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5415+0+0 (secure 0 0 0) 0x7fb2c408b1c0 con 0x7fb2cc1583b0 2024-03-28T13:14:19.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.161+0000 7fb2c9e00640 1 --2- 172.21.15.19:0/3661783746 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb2a806c560 0x7fb2a806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:19.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.162+0000 7fb2c9e00640 1 --2- 172.21.15.19:0/3661783746 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb2a806c560 0x7fb2a806ea20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0b1400 tx=0x7fb2bc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:19.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.164+0000 7fb2c2000640 1 -- 172.21.15.19:0/3661783746 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb2c4059860 con 0x7fb2cc1583b0 2024-03-28T13:14:19.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.372+0000 7fb2cb200640 1 -- 172.21.15.19:0/3661783746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb2cc0b1710 con 0x7fb2cc1583b0 2024-03-28T13:14:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.917+0000 7fb2c2000640 1 -- 172.21.15.19:0/3661783746 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) ==== 169+0+0 (secure 0 0 0) 0x7fb2c4059160 con 0x7fb2cc1583b0 2024-03-28T13:14:19.918 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-03-28T13:14:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.922+0000 7fb2cb200640 1 -- 172.21.15.19:0/3661783746 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb2a806c560 msgr2=0x7fb2a806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.922+0000 7fb2cb200640 1 --2- 172.21.15.19:0/3661783746 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb2a806c560 0x7fb2a806ea20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0b1400 tx=0x7fb2bc009040 comp rx=0 tx=0).stop 2024-03-28T13:14:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.922+0000 7fb2cb200640 1 -- 172.21.15.19:0/3661783746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2cc1583b0 msgr2=0x7fb2cc158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:19.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.922+0000 7fb2cb200640 1 --2- 172.21.15.19:0/3661783746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2cc1583b0 0x7fb2cc158830 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc071210 tx=0x7fb2c4009d30 comp rx=0 tx=0).stop 2024-03-28T13:14:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.924+0000 7fb2cb200640 1 -- 172.21.15.19:0/3661783746 shutdown_connections 2024-03-28T13:14:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.924+0000 7fb2cb200640 1 --2- 172.21.15.19:0/3661783746 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb2a806c560 0x7fb2a806ea20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.924+0000 7fb2cb200640 1 --2- 172.21.15.19:0/3661783746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2cc1583b0 0x7fb2cc158830 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.924+0000 7fb2cb200640 1 --2- 172.21.15.19:0/3661783746 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2cc15b460 0x7fb2cc15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.924+0000 7fb2cb200640 1 --2- 172.21.15.19:0/3661783746 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2b0004800 0x7fb2cc1529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:19.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.924+0000 7fb2cb200640 1 -- 172.21.15.19:0/3661783746 >> 172.21.15.19:0/3661783746 conn(0x7fb2cc0b1d80 msgr2=0x7fb2cc1659e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:19.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.925+0000 7fb2cb200640 1 -- 172.21.15.19:0/3661783746 shutdown_connections 2024-03-28T13:14:19.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:19.926+0000 7fb2cb200640 1 -- 172.21.15.19:0/3661783746 wait complete. 2024-03-28T13:14:19.954 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-28T13:14:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.157+0000 7f6e7e800640 1 Processor -- start 2024-03-28T13:14:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.157+0000 7f6e7e800640 1 -- start start 2024-03-28T13:14:20.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.159+0000 7f6e7e800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815eca0 0x7f6e780b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.159+0000 7f6e7e800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6e7815ac50 0x7f6e780b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.159+0000 7f6e7e800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e780b4990 con 0x7f6e7815a870 2024-03-28T13:14:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.159+0000 7f6e7e800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f6e7806d5b0 con 0x7f6e7815ac50 2024-03-28T13:14:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.159+0000 7f6e7e800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6e7806db70 con 0x7f6e7815eca0 2024-03-28T13:14:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.159+0000 7f6e76a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815eca0 0x7f6e780b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.160+0000 7f6e77e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6e7815ac50 0x7f6e780b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.160+0000 7f6e77e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6e7815ac50 0x7f6e780b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:35618/0 (socket says 172.21.15.19:35618) 2024-03-28T13:14:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.160+0000 7f6e77e00640 1 -- 172.21.15.19:0/4172822204 learned_addr learned my addr 172.21.15.19:0/4172822204 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.160+0000 7f6e76a00640 1 -- 172.21.15.19:0/4172822204 >> v2:172.21.15.188:3300/0 conn(0x7f6e7815ac50 msgr2=0x7f6e780b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.160+0000 7f6e76a00640 1 --2- 172.21.15.19:0/4172822204 >> v2:172.21.15.188:3300/0 conn(0x7f6e7815ac50 0x7f6e780b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.160+0000 7f6e76a00640 1 -- 172.21.15.19:0/4172822204 >> v1:172.21.15.188:6789/0 conn(0x7f6e7815a870 legacy=0x7f6e7815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:20.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.160+0000 7f6e76a00640 1 -- 172.21.15.19:0/4172822204 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e780b4c00 con 0x7f6e7815eca0 2024-03-28T13:14:20.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.162+0000 7f6e76a00640 1 --2- 172.21.15.19:0/4172822204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815eca0 0x7f6e780b3f10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e600097a0 tx=0x7f6e6002dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aaaf1658ef28e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:20.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.163+0000 7f6e76000640 1 -- 172.21.15.19:0/4172822204 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6002f8a0 con 0x7f6e7815eca0 2024-03-28T13:14:20.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.163+0000 7f6e76000640 1 -- 172.21.15.19:0/4172822204 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e6002fa40 con 0x7f6e7815eca0 2024-03-28T13:14:20.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.163+0000 7f6e76000640 1 -- 172.21.15.19:0/4172822204 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6002fe10 con 0x7f6e7815eca0 2024-03-28T13:14:20.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.164+0000 7f6e7e800640 1 -- 172.21.15.19:0/4172822204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815eca0 msgr2=0x7f6e780b3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:20.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.164+0000 7f6e7e800640 1 --2- 172.21.15.19:0/4172822204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815eca0 0x7f6e780b3f10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6e600097a0 tx=0x7f6e6002dae0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.164+0000 7f6e7e800640 1 -- 172.21.15.19:0/4172822204 shutdown_connections 2024-03-28T13:14:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.164+0000 7f6e7e800640 1 --2- 172.21.15.19:0/4172822204 >> v2:172.21.15.188:3300/0 conn(0x7f6e7815ac50 0x7f6e780b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.164+0000 7f6e7e800640 1 --2- 172.21.15.19:0/4172822204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815eca0 0x7f6e780b3f10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.164+0000 7f6e7e800640 1 -- 172.21.15.19:0/4172822204 >> 172.21.15.19:0/4172822204 conn(0x7f6e7806b750 msgr2=0x7f6e7806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.164+0000 7f6e7e800640 1 -- 172.21.15.19:0/4172822204 shutdown_connections 2024-03-28T13:14:20.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.164+0000 7f6e7e800640 1 -- 172.21.15.19:0/4172822204 wait complete. 2024-03-28T13:14:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.166+0000 7f6e7e800640 1 Processor -- start 2024-03-28T13:14:20.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.166+0000 7f6e7e800640 1 -- start start 2024-03-28T13:14:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.167+0000 7f6e7e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e7815a870 0x7f6e7815cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.167+0000 7f6e7e800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815ac50 0x7f6e7814d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.167+0000 7f6e7e800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e7815eca0 0x7f6e7814d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.167+0000 7f6e7e800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6e780d0280 con 0x7f6e7815a870 2024-03-28T13:14:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.167+0000 7f6e7e800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7806d800 con 0x7f6e7815eca0 2024-03-28T13:14:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.167+0000 7f6e7e800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6e780d0400 con 0x7f6e7815ac50 2024-03-28T13:14:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.167+0000 7f6e77400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e7815a870 0x7f6e7815cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:20.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.167+0000 7f6e77400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e7815a870 0x7f6e7815cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51064/0 (socket says 172.21.15.19:51064) 2024-03-28T13:14:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.167+0000 7f6e77400640 1 -- 172.21.15.19:0/3397166321 learned_addr learned my addr 172.21.15.19:0/3397166321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.167+0000 7f6e76a00640 1 --2- 172.21.15.19:0/3397166321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815ac50 0x7f6e7814d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.167+0000 7f6e77400640 1 -- 172.21.15.19:0/3397166321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815ac50 msgr2=0x7f6e7814d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.167+0000 7f6e77400640 1 --2- 172.21.15.19:0/3397166321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815ac50 0x7f6e7814d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.167+0000 7f6e77400640 1 -- 172.21.15.19:0/3397166321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e7815eca0 msgr2=0x7f6e7814d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.167+0000 7f6e77e00640 1 --2- 172.21.15.19:0/3397166321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e7815eca0 0x7f6e7814d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.168+0000 7f6e77400640 1 --2- 172.21.15.19:0/3397166321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e7815eca0 0x7f6e7814d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.168+0000 7f6e77400640 1 -- 172.21.15.19:0/3397166321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7815d690 con 0x7f6e7815a870 2024-03-28T13:14:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.168+0000 7f6e77e00640 1 --2- 172.21.15.19:0/3397166321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e7815eca0 0x7f6e7814d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.168+0000 7f6e77400640 1 --2- 172.21.15.19:0/3397166321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e7815a870 0x7f6e7815cfd0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800c400 tx=0x7f6e6800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.168+0000 7f6e6f400640 1 -- 172.21.15.19:0/3397166321 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e680052d0 con 0x7f6e7815a870 2024-03-28T13:14:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.168+0000 7f6e6f400640 1 -- 172.21.15.19:0/3397166321 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e68005490 con 0x7f6e7815a870 2024-03-28T13:14:20.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.168+0000 7f6e6f400640 1 -- 172.21.15.19:0/3397166321 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68007260 con 0x7f6e7815a870 2024-03-28T13:14:20.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.168+0000 7f6e7e800640 1 -- 172.21.15.19:0/3397166321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7815d970 con 0x7f6e7815a870 2024-03-28T13:14:20.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.168+0000 7f6e7e800640 1 -- 172.21.15.19:0/3397166321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e780ccd00 con 0x7f6e7815a870 2024-03-28T13:14:20.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.170+0000 7f6e7e800640 1 -- 172.21.15.19:0/3397166321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e780685c0 con 0x7f6e7815a870 2024-03-28T13:14:20.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.170+0000 7f6e6f400640 1 -- 172.21.15.19:0/3397166321 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6e68007400 con 0x7f6e7815a870 2024-03-28T13:14:20.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.171+0000 7f6e6f400640 1 --2- 172.21.15.19:0/3397166321 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6e5406c530 0x7f6e5406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:20.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.171+0000 7f6e6f400640 1 -- 172.21.15.19:0/3397166321 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f6e6808a9f0 con 0x7f6e7815a870 2024-03-28T13:14:20.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.172+0000 7f6e76a00640 1 --2- 172.21.15.19:0/3397166321 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6e5406c530 0x7f6e5406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:20.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.173+0000 7f6e76a00640 1 --2- 172.21.15.19:0/3397166321 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6e5406c530 0x7f6e5406e9f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6e60038000 tx=0x7f6e60008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:20.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.176+0000 7f6e6f400640 1 -- 172.21.15.19:0/3397166321 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6e68059270 con 0x7f6e7815a870 2024-03-28T13:14:20.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.381+0000 7f6e7e800640 1 -- 172.21.15.19:0/3397166321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f6e780b0ca0 con 0x7f6e7815a870 2024-03-28T13:14:20.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.388+0000 7f6e6f400640 1 -- 172.21.15.19:0/3397166321 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f6e68058b70 con 0x7f6e7815a870 2024-03-28T13:14:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.391+0000 7f6e7e800640 1 -- 172.21.15.19:0/3397166321 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6e5406c530 msgr2=0x7f6e5406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.391+0000 7f6e7e800640 1 --2- 172.21.15.19:0/3397166321 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6e5406c530 0x7f6e5406e9f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6e60038000 tx=0x7f6e60008030 comp rx=0 tx=0).stop 2024-03-28T13:14:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.391+0000 7f6e7e800640 1 -- 172.21.15.19:0/3397166321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e7815a870 msgr2=0x7f6e7815cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.391+0000 7f6e7e800640 1 --2- 172.21.15.19:0/3397166321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e7815a870 0x7f6e7815cfd0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800c400 tx=0x7f6e6800c8c0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.391+0000 7f6e7e800640 1 -- 172.21.15.19:0/3397166321 shutdown_connections 2024-03-28T13:14:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.391+0000 7f6e7e800640 1 --2- 172.21.15.19:0/3397166321 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6e5406c530 0x7f6e5406e9f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.391+0000 7f6e7e800640 1 --2- 172.21.15.19:0/3397166321 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e7815eca0 0x7f6e7814d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.391+0000 7f6e7e800640 1 --2- 172.21.15.19:0/3397166321 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e7815ac50 0x7f6e7814d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.391+0000 7f6e7e800640 1 --2- 172.21.15.19:0/3397166321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e7815a870 0x7f6e7815cfd0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.391+0000 7f6e7e800640 1 -- 172.21.15.19:0/3397166321 >> 172.21.15.19:0/3397166321 conn(0x7f6e7806b750 msgr2=0x7f6e7814af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:20.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.391+0000 7f6e7e800640 1 -- 172.21.15.19:0/3397166321 shutdown_connections 2024-03-28T13:14:20.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.391+0000 7f6e7e800640 1 -- 172.21.15.19:0/3397166321 wait complete. 2024-03-28T13:14:20.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-03-28T13:14:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.637+0000 7f3bb5200640 1 Processor -- start 2024-03-28T13:14:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.637+0000 7f3bb5200640 1 -- start start 2024-03-28T13:14:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.638+0000 7f3bb5200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3bb0144ad0 0x7f3bb00b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.638+0000 7f3bb5200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3bb015f710 0x7f3bb0072840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.638+0000 7f3bb5200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb0073190 con 0x7f3bb0072d80 2024-03-28T13:14:20.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.639+0000 7f3bb5200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb00cdad0 con 0x7f3bb0144ad0 2024-03-28T13:14:20.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.639+0000 7f3bb5200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb015fba0 con 0x7f3bb015f710 2024-03-28T13:14:20.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.639+0000 7f3bafe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3bb0072d80 0x7f3bb00714e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49354/0 (socket says 172.21.15.19:49354) 2024-03-28T13:14:20.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.639+0000 7f3bafe00640 1 -- 172.21.15.19:0/2008296451 learned_addr learned my addr 172.21.15.19:0/2008296451 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:20.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.639+0000 7f3baf400640 1 --2- 172.21.15.19:0/2008296451 >> v2:172.21.15.19:3300/0 conn(0x7f3bb0144ad0 0x7f3bb00b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:20.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.639+0000 7f3baea00640 1 --2- 172.21.15.19:0/2008296451 >> v2:172.21.15.188:3300/0 conn(0x7f3bb015f710 0x7f3bb0072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:20.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.640+0000 7f3baf400640 1 -- 172.21.15.19:0/2008296451 >> v2:172.21.15.188:3300/0 conn(0x7f3bb015f710 msgr2=0x7f3bb0072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:20.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.640+0000 7f3baf400640 1 --2- 172.21.15.19:0/2008296451 >> v2:172.21.15.188:3300/0 conn(0x7f3bb015f710 0x7f3bb0072840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.640+0000 7f3baf400640 1 -- 172.21.15.19:0/2008296451 >> v1:172.21.15.19:6789/0 conn(0x7f3bb0072d80 legacy=0x7f3bb00714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:20.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.640+0000 7f3bae000640 1 -- 172.21.15.19:0/2008296451 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3798113771 0 0) 0x7f3ba8001490 con 0x7f3bb0072d80 2024-03-28T13:14:20.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.640+0000 7f3baf400640 1 -- 172.21.15.19:0/2008296451 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb0144eb0 con 0x7f3bb0144ad0 2024-03-28T13:14:20.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.641+0000 7f3baf400640 1 --2- 172.21.15.19:0/2008296451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bb0144ad0 0x7f3bb00b2ff0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00b4d90 tx=0x7f3ba002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c21bbfc24f45410 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:20.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.641+0000 7f3bae000640 1 -- 172.21.15.19:0/2008296451 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0035020 con 0x7f3bb0144ad0 2024-03-28T13:14:20.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.641+0000 7f3bae000640 1 -- 172.21.15.19:0/2008296451 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba0031830 con 0x7f3bb0144ad0 2024-03-28T13:14:20.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.641+0000 7f3bae000640 1 -- 172.21.15.19:0/2008296451 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0031c00 con 0x7f3bb0144ad0 2024-03-28T13:14:20.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.642+0000 7f3bb5200640 1 -- 172.21.15.19:0/2008296451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bb0144ad0 msgr2=0x7f3bb00b2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:20.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.642+0000 7f3bb5200640 1 --2- 172.21.15.19:0/2008296451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bb0144ad0 0x7f3bb00b2ff0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00b4d90 tx=0x7f3ba002b8d0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.642+0000 7f3bb5200640 1 -- 172.21.15.19:0/2008296451 shutdown_connections 2024-03-28T13:14:20.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.642+0000 7f3bb5200640 1 --2- 172.21.15.19:0/2008296451 >> v2:172.21.15.188:3300/0 conn(0x7f3bb015f710 0x7f3bb0072840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.642+0000 7f3bb5200640 1 --2- 172.21.15.19:0/2008296451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bb0144ad0 0x7f3bb00b2ff0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.642+0000 7f3bb5200640 1 -- 172.21.15.19:0/2008296451 >> 172.21.15.19:0/2008296451 conn(0x7f3bb006b570 msgr2=0x7f3bb006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:20.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.642+0000 7f3bb5200640 1 -- 172.21.15.19:0/2008296451 shutdown_connections 2024-03-28T13:14:20.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.642+0000 7f3bb5200640 1 -- 172.21.15.19:0/2008296451 wait complete. 2024-03-28T13:14:20.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.644+0000 7f3bb5200640 1 Processor -- start 2024-03-28T13:14:20.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.644+0000 7f3bb5200640 1 -- start start 2024-03-28T13:14:20.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.645+0000 7f3bb5200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bb0072d80 0x7f3bb00cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:20.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.645+0000 7f3bb5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bb0144ad0 0x7f3bb0139890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.645+0000 7f3bb5200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bb015f710 0x7f3bb015eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.645+0000 7f3bb5200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb00b3cb0 con 0x7f3bb0144ad0 2024-03-28T13:14:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.645+0000 7f3bb5200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb00b39d0 con 0x7f3bb015f710 2024-03-28T13:14:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.645+0000 7f3bb5200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb00b3f90 con 0x7f3bb0072d80 2024-03-28T13:14:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.645+0000 7f3baea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bb0144ad0 0x7f3bb0139890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.645+0000 7f3baea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bb0144ad0 0x7f3bb0139890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51086/0 (socket says 172.21.15.19:51086) 2024-03-28T13:14:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.645+0000 7f3baea00640 1 -- 172.21.15.19:0/1260394517 learned_addr learned my addr 172.21.15.19:0/1260394517 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.645+0000 7f3bafe00640 1 --2- 172.21.15.19:0/1260394517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bb015f710 0x7f3bb015eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.645+0000 7f3baea00640 1 -- 172.21.15.19:0/1260394517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bb0072d80 msgr2=0x7f3bb00cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.645+0000 7f3baf400640 1 --2- 172.21.15.19:0/1260394517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bb0072d80 0x7f3bb00cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.645+0000 7f3baea00640 1 --2- 172.21.15.19:0/1260394517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bb0072d80 0x7f3bb00cfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.646+0000 7f3baea00640 1 -- 172.21.15.19:0/1260394517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bb015f710 msgr2=0x7f3bb015eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:20.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.646+0000 7f3baea00640 1 --2- 172.21.15.19:0/1260394517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bb015f710 0x7f3bb015eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.646+0000 7f3baea00640 1 -- 172.21.15.19:0/1260394517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb0142530 con 0x7f3bb0144ad0 2024-03-28T13:14:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.646+0000 7f3bafe00640 1 --2- 172.21.15.19:0/1260394517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bb015f710 0x7f3bb015eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:20.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.646+0000 7f3baea00640 1 --2- 172.21.15.19:0/1260394517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bb0144ad0 0x7f3bb0139890 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800e870 tx=0x7f3b9800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.646+0000 7f3ba7400640 1 -- 172.21.15.19:0/1260394517 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98017070 con 0x7f3bb0144ad0 2024-03-28T13:14:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.646+0000 7f3ba7400640 1 -- 172.21.15.19:0/1260394517 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b9800c940 con 0x7f3bb0144ad0 2024-03-28T13:14:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.646+0000 7f3ba7400640 1 -- 172.21.15.19:0/1260394517 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800cd30 con 0x7f3bb0144ad0 2024-03-28T13:14:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.646+0000 7f3bb5200640 1 -- 172.21.15.19:0/1260394517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb01427d0 con 0x7f3bb0144ad0 2024-03-28T13:14:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.646+0000 7f3bb5200640 1 -- 172.21.15.19:0/1260394517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb0142d90 con 0x7f3bb0144ad0 2024-03-28T13:14:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.647+0000 7f3bb5200640 1 -- 172.21.15.19:0/1260394517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb0075940 con 0x7f3bb0144ad0 2024-03-28T13:14:20.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.648+0000 7f3ba7400640 1 -- 172.21.15.19:0/1260394517 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3b9801a070 con 0x7f3bb0144ad0 2024-03-28T13:14:20.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.650+0000 7f3ba7400640 1 --2- 172.21.15.19:0/1260394517 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3b8c06c560 0x7f3b8c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.650+0000 7f3ba7400640 1 -- 172.21.15.19:0/1260394517 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f3b9808b330 con 0x7f3bb0144ad0 2024-03-28T13:14:20.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.650+0000 7f3baf400640 1 --2- 172.21.15.19:0/1260394517 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3b8c06c560 0x7f3b8c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:20.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.652+0000 7f3baf400640 1 --2- 172.21.15.19:0/1260394517 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3b8c06c560 0x7f3b8c06ea20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0036000 tx=0x7f3ba0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:20.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.655+0000 7f3ba7400640 1 -- 172.21.15.19:0/1260394517 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3b98059bb0 con 0x7f3bb0144ad0 2024-03-28T13:14:20.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:20.939+0000 7f3bb5200640 1 -- 172.21.15.19:0/1260394517 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f3bb00b4240 con 0x7f3bb0144ad0 2024-03-28T13:14:21.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:21.937+0000 7f3ba7400640 1 -- 172.21.15.19:0/1260394517 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v248) ==== 169+0+0 (secure 0 0 0) 0x7f3b980594b0 con 0x7f3bb0144ad0 2024-03-28T13:14:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:21.940+0000 7f3bb5200640 1 -- 172.21.15.19:0/1260394517 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3b8c06c560 msgr2=0x7f3b8c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:21.940+0000 7f3bb5200640 1 --2- 172.21.15.19:0/1260394517 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3b8c06c560 0x7f3b8c06ea20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0036000 tx=0x7f3ba0008030 comp rx=0 tx=0).stop 2024-03-28T13:14:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:21.940+0000 7f3bb5200640 1 -- 172.21.15.19:0/1260394517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bb0144ad0 msgr2=0x7f3bb0139890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:21.940+0000 7f3bb5200640 1 --2- 172.21.15.19:0/1260394517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bb0144ad0 0x7f3bb0139890 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800e870 tx=0x7f3b9800ed30 comp rx=0 tx=0).stop 2024-03-28T13:14:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:21.941+0000 7f3bb5200640 1 -- 172.21.15.19:0/1260394517 shutdown_connections 2024-03-28T13:14:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:21.941+0000 7f3bb5200640 1 --2- 172.21.15.19:0/1260394517 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3b8c06c560 0x7f3b8c06ea20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:21.941+0000 7f3bb5200640 1 --2- 172.21.15.19:0/1260394517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bb015f710 0x7f3bb015eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:21.941+0000 7f3bb5200640 1 --2- 172.21.15.19:0/1260394517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bb0144ad0 0x7f3bb0139890 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:21.941+0000 7f3bb5200640 1 --2- 172.21.15.19:0/1260394517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bb0072d80 0x7f3bb00cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:21.941+0000 7f3bb5200640 1 -- 172.21.15.19:0/1260394517 >> 172.21.15.19:0/1260394517 conn(0x7f3bb006b570 msgr2=0x7f3bb00b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:21.941+0000 7f3bb5200640 1 -- 172.21.15.19:0/1260394517 shutdown_connections 2024-03-28T13:14:21.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:21.941+0000 7f3bb5200640 1 -- 172.21.15.19:0/1260394517 wait complete. 2024-03-28T13:14:21.961 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-03-28T13:14:21.961 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-03-28T13:14:21.961 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-28T13:14:21.962 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:14:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.185+0000 7f2109600640 1 Processor -- start 2024-03-28T13:14:22.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.185+0000 7f2109600640 1 -- start start 2024-03-28T13:14:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.187+0000 7f2109600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f210414c1b0 0x7f2104166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.187+0000 7f2109600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21040cde20 con 0x7f210414ba20 2024-03-28T13:14:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.187+0000 7f2109600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2104166ae0 con 0x7f21040cfda0 2024-03-28T13:14:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.187+0000 7f2109600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2104166dc0 con 0x7f210414c1b0 2024-03-28T13:14:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.187+0000 7f2103e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f210414ba20 0x7f21040b2ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49358/0 (socket says 172.21.15.19:49358) 2024-03-28T13:14:22.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.187+0000 7f2103e00640 1 -- 172.21.15.19:0/1809604656 learned_addr learned my addr 172.21.15.19:0/1809604656 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.187+0000 7f2103400640 1 --2- 172.21.15.19:0/1809604656 >> v2:172.21.15.19:3300/0 conn(0x7f210414c1b0 0x7f2104166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.188+0000 7f2102a00640 1 -- 172.21.15.19:0/1809604656 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2606088928 0 0) 0x7f21040cde20 con 0x7f210414ba20 2024-03-28T13:14:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.188+0000 7f2103400640 1 -- 172.21.15.19:0/1809604656 >> v1:172.21.15.188:6789/0 conn(0x7f21040cfda0 legacy=0x7f21040d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.188+0000 7f2103400640 1 -- 172.21.15.19:0/1809604656 >> v1:172.21.15.19:6789/0 conn(0x7f210414ba20 legacy=0x7f21040b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:22.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.188+0000 7f2103400640 1 -- 172.21.15.19:0/1809604656 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21041670a0 con 0x7f210414c1b0 2024-03-28T13:14:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.189+0000 7f2103400640 1 --2- 172.21.15.19:0/1809604656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f210414c1b0 0x7f2104166290 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0097a0 tx=0x7f20ec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2215c4c900016 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.189+0000 7f2102a00640 1 -- 172.21.15.19:0/1809604656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec035020 con 0x7f210414c1b0 2024-03-28T13:14:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.189+0000 7f2102a00640 1 -- 172.21.15.19:0/1809604656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20ec031830 con 0x7f210414c1b0 2024-03-28T13:14:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.189+0000 7f2102a00640 1 -- 172.21.15.19:0/1809604656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec031c00 con 0x7f210414c1b0 2024-03-28T13:14:22.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.190+0000 7f2109600640 1 -- 172.21.15.19:0/1809604656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f210414c1b0 msgr2=0x7f2104166290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:22.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.190+0000 7f2109600640 1 --2- 172.21.15.19:0/1809604656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f210414c1b0 0x7f2104166290 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0097a0 tx=0x7f20ec02b8d0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.190+0000 7f2109600640 1 -- 172.21.15.19:0/1809604656 shutdown_connections 2024-03-28T13:14:22.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.190+0000 7f2109600640 1 --2- 172.21.15.19:0/1809604656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f210414c1b0 0x7f2104166290 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.190+0000 7f2109600640 1 -- 172.21.15.19:0/1809604656 >> 172.21.15.19:0/1809604656 conn(0x7f21040ccad0 msgr2=0x7f21040cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:22.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.190+0000 7f2109600640 1 -- 172.21.15.19:0/1809604656 shutdown_connections 2024-03-28T13:14:22.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.190+0000 7f2109600640 1 -- 172.21.15.19:0/1809604656 wait complete. 2024-03-28T13:14:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.192+0000 7f2109600640 1 Processor -- start 2024-03-28T13:14:22.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.192+0000 7f2109600640 1 -- start start 2024-03-28T13:14:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.193+0000 7f2109600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21040cfda0 0x7f2104161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.193+0000 7f2109600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f210414ba20 0x7f2104164ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:22.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.193+0000 7f2109600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210414c1b0 0x7f21040b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.193+0000 7f2109600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f210414be00 con 0x7f210414ba20 2024-03-28T13:14:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.193+0000 7f2109600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f21041667d0 con 0x7f210414c1b0 2024-03-28T13:14:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.193+0000 7f2109600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f210414bf80 con 0x7f21040cfda0 2024-03-28T13:14:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.194+0000 7f2103400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f210414ba20 0x7f2104164ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.194+0000 7f2103400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f210414ba20 0x7f2104164ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51106/0 (socket says 172.21.15.19:51106) 2024-03-28T13:14:22.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.194+0000 7f2103400640 1 -- 172.21.15.19:0/3931728435 learned_addr learned my addr 172.21.15.19:0/3931728435 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.194+0000 7f2108c00640 1 --2- 172.21.15.19:0/3931728435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210414c1b0 0x7f21040b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:22.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.194+0000 7f2103e00640 1 --2- 172.21.15.19:0/3931728435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21040cfda0 0x7f2104161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:22.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.194+0000 7f2103400640 1 -- 172.21.15.19:0/3931728435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21040cfda0 msgr2=0x7f2104161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:22.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.194+0000 7f2103400640 1 --2- 172.21.15.19:0/3931728435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21040cfda0 0x7f2104161fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.194+0000 7f2103400640 1 -- 172.21.15.19:0/3931728435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210414c1b0 msgr2=0x7f21040b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:22.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.194+0000 7f2103400640 1 --2- 172.21.15.19:0/3931728435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210414c1b0 0x7f21040b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.194+0000 7f2103400640 1 -- 172.21.15.19:0/3931728435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21041fc7a0 con 0x7f210414ba20 2024-03-28T13:14:22.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.194+0000 7f2103e00640 1 --2- 172.21.15.19:0/3931728435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21040cfda0 0x7f2104161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:22.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.195+0000 7f2103400640 1 --2- 172.21.15.19:0/3931728435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f210414ba20 0x7f2104164ba0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0097a0 tx=0x7f20ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:22.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.195+0000 7f20fbe00640 1 -- 172.21.15.19:0/3931728435 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec004230 con 0x7f210414ba20 2024-03-28T13:14:22.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.195+0000 7f20fbe00640 1 -- 172.21.15.19:0/3931728435 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20ec0043d0 con 0x7f210414ba20 2024-03-28T13:14:22.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.195+0000 7f20fbe00640 1 -- 172.21.15.19:0/3931728435 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec03cad0 con 0x7f210414ba20 2024-03-28T13:14:22.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.195+0000 7f2109600640 1 -- 172.21.15.19:0/3931728435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21041fca80 con 0x7f210414ba20 2024-03-28T13:14:22.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.195+0000 7f2109600640 1 -- 172.21.15.19:0/3931728435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21041643a0 con 0x7f210414ba20 2024-03-28T13:14:22.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.197+0000 7f2109600640 1 -- 172.21.15.19:0/3931728435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2104075940 con 0x7f210414ba20 2024-03-28T13:14:22.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.197+0000 7f20fbe00640 1 -- 172.21.15.19:0/3931728435 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f20ec038020 con 0x7f210414ba20 2024-03-28T13:14:22.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.198+0000 7f20fbe00640 1 --2- 172.21.15.19:0/3931728435 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f20e006c530 0x7f20e006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:22.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.198+0000 7f20fbe00640 1 -- 172.21.15.19:0/3931728435 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f20ec0ad6c0 con 0x7f210414ba20 2024-03-28T13:14:22.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.199+0000 7f2103e00640 1 --2- 172.21.15.19:0/3931728435 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f20e006c530 0x7f20e006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.200+0000 7f2103e00640 1 --2- 172.21.15.19:0/3931728435 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f20e006c530 0x7f20e006e9f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f20f4007fd0 tx=0x7f20f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:22.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.204+0000 7f20fbe00640 1 -- 172.21.15.19:0/3931728435 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f20ec041070 con 0x7f210414ba20 2024-03-28T13:14:22.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.410+0000 7f2109600640 1 -- 172.21.15.19:0/3931728435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2104162a80 con 0x7f210414ba20 2024-03-28T13:14:22.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.412+0000 7f20fbe00640 1 -- 172.21.15.19:0/3931728435 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+13020 (secure 0 0 0) 0x7f20ec07bfd0 con 0x7f210414ba20 2024-03-28T13:14:22.412 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:22.412 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":68,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:14:19.914962+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3341654519":"2024-03-29T13:14:16.481838+0000","172.21.15.19:6833/3341654519":"2024-03-29T13:14:16.481838+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:14:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.414+0000 7f2109600640 1 -- 172.21.15.19:0/3931728435 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f20e006c530 msgr2=0x7f20e006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.415+0000 7f2109600640 1 --2- 172.21.15.19:0/3931728435 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f20e006c530 0x7f20e006e9f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f20f4007fd0 tx=0x7f20f4009040 comp rx=0 tx=0).stop 2024-03-28T13:14:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.415+0000 7f2109600640 1 -- 172.21.15.19:0/3931728435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f210414ba20 msgr2=0x7f2104164ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.415+0000 7f2109600640 1 --2- 172.21.15.19:0/3931728435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f210414ba20 0x7f2104164ba0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0097a0 tx=0x7f20ec0046c0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.415+0000 7f2109600640 1 -- 172.21.15.19:0/3931728435 shutdown_connections 2024-03-28T13:14:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.415+0000 7f2109600640 1 --2- 172.21.15.19:0/3931728435 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f20e006c530 0x7f20e006e9f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.415+0000 7f2109600640 1 --2- 172.21.15.19:0/3931728435 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f210414c1b0 0x7f21040b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.415+0000 7f2109600640 1 --2- 172.21.15.19:0/3931728435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f210414ba20 0x7f2104164ba0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.415+0000 7f2109600640 1 --2- 172.21.15.19:0/3931728435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21040cfda0 0x7f2104161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.415+0000 7f2109600640 1 -- 172.21.15.19:0/3931728435 >> 172.21.15.19:0/3931728435 conn(0x7f21040ccad0 msgr2=0x7f2104149ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:22.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.415+0000 7f2109600640 1 -- 172.21.15.19:0/3931728435 shutdown_connections 2024-03-28T13:14:22.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.415+0000 7f2109600640 1 -- 172.21.15.19:0/3931728435 wait complete. 2024-03-28T13:14:22.434 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:14:22.434 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:14:22.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.659+0000 7fcf1ea00640 1 Processor -- start 2024-03-28T13:14:22.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.659+0000 7fcf1ea00640 1 -- start start 2024-03-28T13:14:22.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.660+0000 7fcf1ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf1815e4c0 0x7fcf180b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:22.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.660+0000 7fcf1ea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf180d37b0 0x7fcf180b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:22.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.660+0000 7fcf1ea00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcf180b67f0 0x7fcf180d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.660+0000 7fcf1ea00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcf1815e8a0 con 0x7fcf1815e4c0 2024-03-28T13:14:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.660+0000 7fcf1ea00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fcf180d4040 con 0x7fcf180b67f0 2024-03-28T13:14:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.660+0000 7fcf1ea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcf180d3320 con 0x7fcf180d37b0 2024-03-28T13:14:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.661+0000 7fcf1d600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf1815e4c0 0x7fcf180b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.661+0000 7fcf1d600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf1815e4c0 0x7fcf180b9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51120/0 (socket says 172.21.15.19:51120) 2024-03-28T13:14:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.661+0000 7fcf1d600640 1 -- 172.21.15.19:0/2130271430 learned_addr learned my addr 172.21.15.19:0/2130271430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.661+0000 7fcf1d600640 1 -- 172.21.15.19:0/2130271430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf180d37b0 msgr2=0x7fcf180b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.661+0000 7fcf1cc00640 1 --2- 172.21.15.19:0/2130271430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf180d37b0 0x7fcf180b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.661+0000 7fcf1e000640 1 --2- 172.21.15.19:0/2130271430 >> v2:172.21.15.188:3300/0 conn(0x7fcf180b67f0 0x7fcf180d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.661+0000 7fcf1d600640 1 --2- 172.21.15.19:0/2130271430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf180d37b0 0x7fcf180b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.661+0000 7fcf1d600640 1 -- 172.21.15.19:0/2130271430 >> v2:172.21.15.188:3300/0 conn(0x7fcf180b67f0 msgr2=0x7fcf180d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:22.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.661+0000 7fcf1d600640 1 --2- 172.21.15.19:0/2130271430 >> v2:172.21.15.188:3300/0 conn(0x7fcf180b67f0 0x7fcf180d4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.661+0000 7fcf1d600640 1 -- 172.21.15.19:0/2130271430 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf180d2710 con 0x7fcf1815e4c0 2024-03-28T13:14:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.661+0000 7fcf1e000640 1 --2- 172.21.15.19:0/2130271430 >> v2:172.21.15.188:3300/0 conn(0x7fcf180b67f0 0x7fcf180d4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.661+0000 7fcf1cc00640 1 --2- 172.21.15.19:0/2130271430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf180d37b0 0x7fcf180b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.662+0000 7fcf1d600640 1 --2- 172.21.15.19:0/2130271430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf1815e4c0 0x7fcf180b9bf0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fcf180d1710 tx=0x7fcf0c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7d6a30679b4b50 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.663+0000 7fcf17e00640 1 -- 172.21.15.19:0/2130271430 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c035020 con 0x7fcf1815e4c0 2024-03-28T13:14:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.663+0000 7fcf17e00640 1 -- 172.21.15.19:0/2130271430 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0c02d040 con 0x7fcf1815e4c0 2024-03-28T13:14:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.663+0000 7fcf17e00640 1 -- 172.21.15.19:0/2130271430 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c031a80 con 0x7fcf1815e4c0 2024-03-28T13:14:22.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.663+0000 7fcf1ea00640 1 -- 172.21.15.19:0/2130271430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf1815e4c0 msgr2=0x7fcf180b9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.663+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/2130271430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf1815e4c0 0x7fcf180b9bf0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fcf180d1710 tx=0x7fcf0c02b900 comp rx=0 tx=0).stop 2024-03-28T13:14:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.664+0000 7fcf1ea00640 1 -- 172.21.15.19:0/2130271430 shutdown_connections 2024-03-28T13:14:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.664+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/2130271430 >> v2:172.21.15.188:3300/0 conn(0x7fcf180b67f0 0x7fcf180d4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.664+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/2130271430 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf180d37b0 0x7fcf180b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.664+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/2130271430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf1815e4c0 0x7fcf180b9bf0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.664+0000 7fcf1ea00640 1 -- 172.21.15.19:0/2130271430 >> 172.21.15.19:0/2130271430 conn(0x7fcf180676a0 msgr2=0x7fcf18067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:22.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.664+0000 7fcf1ea00640 1 -- 172.21.15.19:0/2130271430 shutdown_connections 2024-03-28T13:14:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.664+0000 7fcf1ea00640 1 -- 172.21.15.19:0/2130271430 wait complete. 2024-03-28T13:14:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.666+0000 7fcf1ea00640 1 Processor -- start 2024-03-28T13:14:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.666+0000 7fcf1ea00640 1 -- start start 2024-03-28T13:14:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.667+0000 7fcf1ea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf180b67f0 0x7fcf18154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.667+0000 7fcf1ea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf180d37b0 0x7fcf18160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.667+0000 7fcf1ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf1815e4c0 0x7fcf180cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.667+0000 7fcf1ea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcf180d2fb0 con 0x7fcf1815e4c0 2024-03-28T13:14:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.667+0000 7fcf1ea00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf180d2cd0 con 0x7fcf180b67f0 2024-03-28T13:14:22.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.667+0000 7fcf1ea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcf180ba130 con 0x7fcf180d37b0 2024-03-28T13:14:22.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.667+0000 7fcf1e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf1815e4c0 0x7fcf180cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:22.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.667+0000 7fcf1e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf1815e4c0 0x7fcf180cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51122/0 (socket says 172.21.15.19:51122) 2024-03-28T13:14:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.667+0000 7fcf1e000640 1 -- 172.21.15.19:0/1967880206 learned_addr learned my addr 172.21.15.19:0/1967880206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.667+0000 7fcf1d600640 1 --2- 172.21.15.19:0/1967880206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf180b67f0 0x7fcf18154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.668+0000 7fcf1cc00640 1 --2- 172.21.15.19:0/1967880206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf180d37b0 0x7fcf18160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.668+0000 7fcf1e000640 1 -- 172.21.15.19:0/1967880206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf180d37b0 msgr2=0x7fcf18160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.668+0000 7fcf1e000640 1 --2- 172.21.15.19:0/1967880206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf180d37b0 0x7fcf18160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.668+0000 7fcf1e000640 1 -- 172.21.15.19:0/1967880206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf180b67f0 msgr2=0x7fcf18154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.668+0000 7fcf1e000640 1 --2- 172.21.15.19:0/1967880206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf180b67f0 0x7fcf18154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.668+0000 7fcf1e000640 1 -- 172.21.15.19:0/1967880206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf18154db0 con 0x7fcf1815e4c0 2024-03-28T13:14:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.668+0000 7fcf1cc00640 1 --2- 172.21.15.19:0/1967880206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf180d37b0 0x7fcf18160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.668+0000 7fcf1e000640 1 --2- 172.21.15.19:0/1967880206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf1815e4c0 0x7fcf180cd790 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000cef0 tx=0x7fcf1000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:22.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.668+0000 7fcf15600640 1 -- 172.21.15.19:0/1967880206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1001a070 con 0x7fcf1815e4c0 2024-03-28T13:14:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.668+0000 7fcf15600640 1 -- 172.21.15.19:0/1967880206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf1000d860 con 0x7fcf1815e4c0 2024-03-28T13:14:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.668+0000 7fcf15600640 1 -- 172.21.15.19:0/1967880206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1000dc50 con 0x7fcf1815e4c0 2024-03-28T13:14:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.668+0000 7fcf1ea00640 1 -- 172.21.15.19:0/1967880206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf181f7480 con 0x7fcf1815e4c0 2024-03-28T13:14:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.669+0000 7fcf1ea00640 1 -- 172.21.15.19:0/1967880206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf181f7a00 con 0x7fcf1815e4c0 2024-03-28T13:14:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.670+0000 7fcf1ea00640 1 -- 172.21.15.19:0/1967880206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf18075940 con 0x7fcf1815e4c0 2024-03-28T13:14:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.671+0000 7fcf15600640 1 -- 172.21.15.19:0/1967880206 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcf1001d070 con 0x7fcf1815e4c0 2024-03-28T13:14:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.671+0000 7fcf15600640 1 --2- 172.21.15.19:0/1967880206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcef806c530 0x7fcef806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.672+0000 7fcf15600640 1 -- 172.21.15.19:0/1967880206 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7fcf1008bcb0 con 0x7fcf1815e4c0 2024-03-28T13:14:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.672+0000 7fcf1d600640 1 --2- 172.21.15.19:0/1967880206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcef806c530 0x7fcef806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:22.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.673+0000 7fcf1d600640 1 --2- 172.21.15.19:0/1967880206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcef806c530 0x7fcef806e9f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c036000 tx=0x7fcf0c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:22.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.676+0000 7fcf15600640 1 -- 172.21.15.19:0/1967880206 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcf18154db0 con 0x7fcf1815e4c0 2024-03-28T13:14:22.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.881+0000 7fcf1ea00640 1 -- 172.21.15.19:0/1967880206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf18067cd0 con 0x7fcf1815e4c0 2024-03-28T13:14:22.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.882+0000 7fcf15600640 1 -- 172.21.15.19:0/1967880206 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) ==== 93+0+10 (secure 0 0 0) 0x7fcf1005a680 con 0x7fcf1815e4c0 2024-03-28T13:14:22.882 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:14:22.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.884+0000 7fcf1ea00640 1 -- 172.21.15.19:0/1967880206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcef806c530 msgr2=0x7fcef806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:22.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.884+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/1967880206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcef806c530 0x7fcef806e9f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c036000 tx=0x7fcf0c03c040 comp rx=0 tx=0).stop 2024-03-28T13:14:22.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.884+0000 7fcf1ea00640 1 -- 172.21.15.19:0/1967880206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf1815e4c0 msgr2=0x7fcf180cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:22.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.884+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/1967880206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf1815e4c0 0x7fcf180cd790 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcf1000cef0 tx=0x7fcf1000a710 comp rx=0 tx=0).stop 2024-03-28T13:14:22.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.884+0000 7fcf1ea00640 1 -- 172.21.15.19:0/1967880206 shutdown_connections 2024-03-28T13:14:22.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.884+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/1967880206 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcef806c530 0x7fcef806e9f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.884+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/1967880206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf1815e4c0 0x7fcf180cd790 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.884+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/1967880206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf180d37b0 0x7fcf18160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.884+0000 7fcf1ea00640 1 --2- 172.21.15.19:0/1967880206 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf180b67f0 0x7fcf18154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.884+0000 7fcf1ea00640 1 -- 172.21.15.19:0/1967880206 >> 172.21.15.19:0/1967880206 conn(0x7fcf180676a0 msgr2=0x7fcf1815b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.884+0000 7fcf1ea00640 1 -- 172.21.15.19:0/1967880206 shutdown_connections 2024-03-28T13:14:22.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:22.884+0000 7fcf1ea00640 1 -- 172.21.15.19:0/1967880206 wait complete. 2024-03-28T13:14:22.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-03-28T13:14:23.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.117+0000 7f549b200640 1 Processor -- start 2024-03-28T13:14:23.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.118+0000 7f549b200640 1 -- start start 2024-03-28T13:14:23.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.119+0000 7f549b200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f549c144d40 0x7f549c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.119+0000 7f549b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f549c0cd740 0x7f549c0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.120+0000 7f549b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f549c0d0d50 con 0x7f549c144960 2024-03-28T13:14:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.120+0000 7f549b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f549c0cdb90 con 0x7f549c0cd740 2024-03-28T13:14:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.120+0000 7f549b200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f549c0d1030 con 0x7f549c144d40 2024-03-28T13:14:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.120+0000 7f5499e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f549c144960 0x7f549c0730a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49368/0 (socket says 172.21.15.19:49368) 2024-03-28T13:14:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.120+0000 7f5499e00640 1 -- 172.21.15.19:0/4147914965 learned_addr learned my addr 172.21.15.19:0/4147914965 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:23.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.120+0000 7f549a800640 1 --2- 172.21.15.19:0/4147914965 >> v2:172.21.15.19:3300/0 conn(0x7f549c0cd740 0x7f549c0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:23.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.120+0000 7f5493e00640 1 --2- 172.21.15.19:0/4147914965 >> v2:172.21.15.188:3300/0 conn(0x7f549c144d40 0x7f549c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:23.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.120+0000 7f5499400640 1 -- 172.21.15.19:0/4147914965 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2321287450 0 0) 0x7f549c0d0d50 con 0x7f549c144960 2024-03-28T13:14:23.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.120+0000 7f5499400640 1 -- 172.21.15.19:0/4147914965 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5480003520 con 0x7f549c144960 2024-03-28T13:14:23.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.120+0000 7f549a800640 1 -- 172.21.15.19:0/4147914965 >> v2:172.21.15.188:3300/0 conn(0x7f549c144d40 msgr2=0x7f549c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:23.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.121+0000 7f549a800640 1 --2- 172.21.15.19:0/4147914965 >> v2:172.21.15.188:3300/0 conn(0x7f549c144d40 0x7f549c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:23.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.121+0000 7f549a800640 1 -- 172.21.15.19:0/4147914965 >> v1:172.21.15.19:6789/0 conn(0x7f549c144960 legacy=0x7f549c0730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:23.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.121+0000 7f549a800640 1 -- 172.21.15.19:0/4147914965 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549c0d1310 con 0x7f549c0cd740 2024-03-28T13:14:23.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.122+0000 7f549a800640 1 --2- 172.21.15.19:0/4147914965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549c0cd740 0x7f549c0ceaf0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f549c073d20 tx=0x7f549402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cef456bedef573 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:23.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.122+0000 7f5499400640 1 -- 172.21.15.19:0/4147914965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494039070 con 0x7f549c0cd740 2024-03-28T13:14:23.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.122+0000 7f5499400640 1 -- 172.21.15.19:0/4147914965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f549402cc80 con 0x7f549c0cd740 2024-03-28T13:14:23.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.122+0000 7f5499400640 1 -- 172.21.15.19:0/4147914965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494034640 con 0x7f549c0cd740 2024-03-28T13:14:23.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.123+0000 7f549b200640 1 -- 172.21.15.19:0/4147914965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549c0cd740 msgr2=0x7f549c0ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:23.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.123+0000 7f549b200640 1 --2- 172.21.15.19:0/4147914965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549c0cd740 0x7f549c0ceaf0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f549c073d20 tx=0x7f549402bed0 comp rx=0 tx=0).stop 2024-03-28T13:14:23.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.123+0000 7f549b200640 1 -- 172.21.15.19:0/4147914965 shutdown_connections 2024-03-28T13:14:23.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.123+0000 7f549b200640 1 --2- 172.21.15.19:0/4147914965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549c0cd740 0x7f549c0ceaf0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:23.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.123+0000 7f549b200640 1 --2- 172.21.15.19:0/4147914965 >> v2:172.21.15.188:3300/0 conn(0x7f549c144d40 0x7f549c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:23.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.123+0000 7f549b200640 1 -- 172.21.15.19:0/4147914965 >> 172.21.15.19:0/4147914965 conn(0x7f549c06c680 msgr2=0x7f549c06ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:23.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.123+0000 7f549b200640 1 -- 172.21.15.19:0/4147914965 shutdown_connections 2024-03-28T13:14:23.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.123+0000 7f549b200640 1 -- 172.21.15.19:0/4147914965 wait complete. 2024-03-28T13:14:23.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.125+0000 7f549b200640 1 Processor -- start 2024-03-28T13:14:23.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.125+0000 7f549b200640 1 -- start start 2024-03-28T13:14:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.126+0000 7f549b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549c0cd740 0x7f549c13c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.126+0000 7f549b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f549c144960 0x7f549c13def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.126+0000 7f549b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f549c144d40 0x7f549c157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.126+0000 7f549b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f549c0737b0 con 0x7f549c0cd740 2024-03-28T13:14:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.126+0000 7f549b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f549c0d18d0 con 0x7f549c144960 2024-03-28T13:14:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.126+0000 7f549b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f549c0b0820 con 0x7f549c144d40 2024-03-28T13:14:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.126+0000 7f5499e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549c0cd740 0x7f549c13c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.126+0000 7f5499e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549c0cd740 0x7f549c13c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51140/0 (socket says 172.21.15.19:51140) 2024-03-28T13:14:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.126+0000 7f5499e00640 1 -- 172.21.15.19:0/2694590392 learned_addr learned my addr 172.21.15.19:0/2694590392 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.127+0000 7f5499e00640 1 -- 172.21.15.19:0/2694590392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f549c144d40 msgr2=0x7f549c157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.127+0000 7f5499e00640 1 --2- 172.21.15.19:0/2694590392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f549c144d40 0x7f549c157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.127+0000 7f5499e00640 1 -- 172.21.15.19:0/2694590392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f549c144960 msgr2=0x7f549c13def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.127+0000 7f5499e00640 1 --2- 172.21.15.19:0/2694590392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f549c144960 0x7f549c13def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:23.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.127+0000 7f5499e00640 1 -- 172.21.15.19:0/2694590392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549c1575c0 con 0x7f549c0cd740 2024-03-28T13:14:23.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.127+0000 7f5499e00640 1 --2- 172.21.15.19:0/2694590392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549c0cd740 0x7f549c13c0d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f548c00c460 tx=0x7f548c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:23.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.127+0000 7f5492000640 1 -- 172.21.15.19:0/2694590392 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c017070 con 0x7f549c0cd740 2024-03-28T13:14:23.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.127+0000 7f5492000640 1 -- 172.21.15.19:0/2694590392 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f548c005ce0 con 0x7f549c0cd740 2024-03-28T13:14:23.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.127+0000 7f5492000640 1 -- 172.21.15.19:0/2694590392 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c0072f0 con 0x7f549c0cd740 2024-03-28T13:14:23.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.127+0000 7f549b200640 1 -- 172.21.15.19:0/2694590392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549c0ae430 con 0x7f549c0cd740 2024-03-28T13:14:23.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.128+0000 7f549b200640 1 -- 172.21.15.19:0/2694590392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549c0ae9f0 con 0x7f549c0cd740 2024-03-28T13:14:23.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.129+0000 7f549b200640 1 -- 172.21.15.19:0/2694590392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f549c075940 con 0x7f549c0cd740 2024-03-28T13:14:23.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.130+0000 7f5492000640 1 -- 172.21.15.19:0/2694590392 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f548c01a070 con 0x7f549c0cd740 2024-03-28T13:14:23.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.130+0000 7f5492000640 1 --2- 172.21.15.19:0/2694590392 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f547806c530 0x7f547806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:23.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.131+0000 7f5492000640 1 -- 172.21.15.19:0/2694590392 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f548c08a8e0 con 0x7f549c0cd740 2024-03-28T13:14:23.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.131+0000 7f5493e00640 1 --2- 172.21.15.19:0/2694590392 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f547806c530 0x7f547806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:23.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.132+0000 7f5493e00640 1 --2- 172.21.15.19:0/2694590392 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f547806c530 0x7f547806e9f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5484006fd0 tx=0x7f5484008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:23.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.135+0000 7f5492000640 1 -- 172.21.15.19:0/2694590392 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f548c059160 con 0x7f549c0cd740 2024-03-28T13:14:23.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.410+0000 7f549b200640 1 -- 172.21.15.19:0/2694590392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f549c0727a0 con 0x7f549c0cd740 2024-03-28T13:14:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.950+0000 7f5492000640 1 -- 172.21.15.19:0/2694590392 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v250) ==== 169+0+0 (secure 0 0 0) 0x7f548c058a60 con 0x7f549c0cd740 2024-03-28T13:14:23.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.953+0000 7f549b200640 1 -- 172.21.15.19:0/2694590392 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f547806c530 msgr2=0x7f547806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:23.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.953+0000 7f549b200640 1 --2- 172.21.15.19:0/2694590392 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f547806c530 0x7f547806e9f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5484006fd0 tx=0x7f5484008040 comp rx=0 tx=0).stop 2024-03-28T13:14:23.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.953+0000 7f549b200640 1 -- 172.21.15.19:0/2694590392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549c0cd740 msgr2=0x7f549c13c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:23.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.953+0000 7f549b200640 1 --2- 172.21.15.19:0/2694590392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549c0cd740 0x7f549c13c0d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f548c00c460 tx=0x7f548c00c920 comp rx=0 tx=0).stop 2024-03-28T13:14:23.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.953+0000 7f549b200640 1 -- 172.21.15.19:0/2694590392 shutdown_connections 2024-03-28T13:14:23.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.953+0000 7f549b200640 1 --2- 172.21.15.19:0/2694590392 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f547806c530 0x7f547806e9f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:23.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.953+0000 7f549b200640 1 --2- 172.21.15.19:0/2694590392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f549c144d40 0x7f549c157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:23.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.953+0000 7f549b200640 1 --2- 172.21.15.19:0/2694590392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f549c144960 0x7f549c13def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:23.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.953+0000 7f549b200640 1 --2- 172.21.15.19:0/2694590392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f549c0cd740 0x7f549c13c0d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:23.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.953+0000 7f549b200640 1 -- 172.21.15.19:0/2694590392 >> 172.21.15.19:0/2694590392 conn(0x7f549c06c680 msgr2=0x7f549c15e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:23.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.953+0000 7f549b200640 1 -- 172.21.15.19:0/2694590392 shutdown_connections 2024-03-28T13:14:23.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:23.954+0000 7f549b200640 1 -- 172.21.15.19:0/2694590392 wait complete. 2024-03-28T13:14:23.974 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-28T13:14:24.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.210+0000 7f5409000640 1 Processor -- start 2024-03-28T13:14:24.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.210+0000 7f5409000640 1 -- start start 2024-03-28T13:14:24.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.211+0000 7f5409000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f54040d0990 0x7f54040aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:24.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.212+0000 7f5409000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54040b40b0 con 0x7f5404155930 2024-03-28T13:14:24.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.212+0000 7f5409000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54040b4390 con 0x7f5404155550 2024-03-28T13:14:24.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.212+0000 7f5409000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f54040b4670 con 0x7f54040d0990 2024-03-28T13:14:24.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.212+0000 7f5402800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f54040d0990 0x7f54040aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:24.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.212+0000 7f5401400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5404155930 0x7f54040d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49370/0 (socket says 172.21.15.19:49370) 2024-03-28T13:14:24.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.212+0000 7f5401400640 1 -- 172.21.15.19:0/2194691075 learned_addr learned my addr 172.21.15.19:0/2194691075 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:24.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.213+0000 7f5402800640 1 -- 172.21.15.19:0/2194691075 >> v1:172.21.15.188:6789/0 conn(0x7f5404155550 legacy=0x7f540415f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:24.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.213+0000 7f5402800640 1 -- 172.21.15.19:0/2194691075 >> v1:172.21.15.19:6789/0 conn(0x7f5404155930 legacy=0x7f54040d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:24.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.213+0000 7f5402800640 1 -- 172.21.15.19:0/2194691075 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54040b4950 con 0x7f54040d0990 2024-03-28T13:14:24.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.213+0000 7f5400a00640 1 -- 172.21.15.19:0/2194691075 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1199135563 0 0) 0x7f54040b40b0 con 0x7f5404155930 2024-03-28T13:14:24.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.214+0000 7f5402800640 1 --2- 172.21.15.19:0/2194691075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54040d0990 0x7f54040aede0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0097a0 tx=0x7f53fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c01eb81143bc4b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:24.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.214+0000 7f5400a00640 1 -- 172.21.15.19:0/2194691075 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc035020 con 0x7f54040d0990 2024-03-28T13:14:24.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.214+0000 7f5400a00640 1 -- 172.21.15.19:0/2194691075 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53fc031830 con 0x7f54040d0990 2024-03-28T13:14:24.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.214+0000 7f5400a00640 1 -- 172.21.15.19:0/2194691075 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53fc031c00 con 0x7f54040d0990 2024-03-28T13:14:24.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.215+0000 7f5409000640 1 -- 172.21.15.19:0/2194691075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54040d0990 msgr2=0x7f54040aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:24.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.215+0000 7f5409000640 1 --2- 172.21.15.19:0/2194691075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54040d0990 0x7f54040aede0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0097a0 tx=0x7f53fc02b8d0 comp rx=0 tx=0).stop 2024-03-28T13:14:24.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.215+0000 7f5409000640 1 -- 172.21.15.19:0/2194691075 shutdown_connections 2024-03-28T13:14:24.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.215+0000 7f5409000640 1 --2- 172.21.15.19:0/2194691075 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54040d0990 0x7f54040aede0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:24.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.215+0000 7f5409000640 1 -- 172.21.15.19:0/2194691075 >> 172.21.15.19:0/2194691075 conn(0x7f540406b750 msgr2=0x7f540406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:24.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.215+0000 7f5409000640 1 -- 172.21.15.19:0/2194691075 shutdown_connections 2024-03-28T13:14:24.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.215+0000 7f5409000640 1 -- 172.21.15.19:0/2194691075 wait complete. 2024-03-28T13:14:24.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.217+0000 7f5409000640 1 Processor -- start 2024-03-28T13:14:24.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.217+0000 7f5409000640 1 -- start start 2024-03-28T13:14:24.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.218+0000 7f5409000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54040d0990 0x7f54041442f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:24.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.218+0000 7f5409000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5404155550 0x7f5404152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:24.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.218+0000 7f5409000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5404155930 0x7f5404152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:24.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.218+0000 7f5409000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f540415bba0 con 0x7f5404155550 2024-03-28T13:14:24.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.218+0000 7f5409000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f540415ba20 con 0x7f5404155930 2024-03-28T13:14:24.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.218+0000 7f5409000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f54040b3cf0 con 0x7f54040d0990 2024-03-28T13:14:24.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.218+0000 7f5401400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5404155550 0x7f5404152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:24.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.218+0000 7f5401400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5404155550 0x7f5404152900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51168/0 (socket says 172.21.15.19:51168) 2024-03-28T13:14:24.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.218+0000 7f5401400640 1 -- 172.21.15.19:0/666378999 learned_addr learned my addr 172.21.15.19:0/666378999 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:24.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.218+0000 7f5401e00640 1 --2- 172.21.15.19:0/666378999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54040d0990 0x7f54041442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:24.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.219+0000 7f5402800640 1 --2- 172.21.15.19:0/666378999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5404155930 0x7f5404152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:24.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.219+0000 7f5401400640 1 -- 172.21.15.19:0/666378999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54040d0990 msgr2=0x7f54041442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:24.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.219+0000 7f5401400640 1 --2- 172.21.15.19:0/666378999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54040d0990 0x7f54041442f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:24.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.219+0000 7f5401400640 1 -- 172.21.15.19:0/666378999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5404155930 msgr2=0x7f5404152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:24.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.219+0000 7f5401400640 1 --2- 172.21.15.19:0/666378999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5404155930 0x7f5404152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:24.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.219+0000 7f5401400640 1 -- 172.21.15.19:0/666378999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54040b0660 con 0x7f5404155550 2024-03-28T13:14:24.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.219+0000 7f5401400640 1 --2- 172.21.15.19:0/666378999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5404155550 0x7f5404152900 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00c4a0 tx=0x7f53ec00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:24.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.219+0000 7f53fa000640 1 -- 172.21.15.19:0/666378999 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec018070 con 0x7f5404155550 2024-03-28T13:14:24.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.219+0000 7f53fa000640 1 -- 172.21.15.19:0/666378999 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ec0074e0 con 0x7f5404155550 2024-03-28T13:14:24.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.219+0000 7f5409000640 1 -- 172.21.15.19:0/666378999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54040b0940 con 0x7f5404155550 2024-03-28T13:14:24.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.219+0000 7f53fa000640 1 -- 172.21.15.19:0/666378999 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec004a20 con 0x7f5404155550 2024-03-28T13:14:24.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.219+0000 7f5409000640 1 -- 172.21.15.19:0/666378999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5404144830 con 0x7f5404155550 2024-03-28T13:14:24.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.221+0000 7f5409000640 1 -- 172.21.15.19:0/666378999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54040685c0 con 0x7f5404155550 2024-03-28T13:14:24.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.221+0000 7f53fa000640 1 -- 172.21.15.19:0/666378999 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f53ec004bc0 con 0x7f5404155550 2024-03-28T13:14:24.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.222+0000 7f53fa000640 1 --2- 172.21.15.19:0/666378999 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f53e006c530 0x7f53e006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:24.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.222+0000 7f53fa000640 1 -- 172.21.15.19:0/666378999 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f53ec08a830 con 0x7f5404155550 2024-03-28T13:14:24.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.223+0000 7f5401e00640 1 --2- 172.21.15.19:0/666378999 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f53e006c530 0x7f53e006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:24.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.224+0000 7f5401e00640 1 --2- 172.21.15.19:0/666378999 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f53e006c530 0x7f53e006e9f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f53f40012e0 tx=0x7f53f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:24.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.227+0000 7f53fa000640 1 -- 172.21.15.19:0/666378999 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f53ec0590c0 con 0x7f5404155550 2024-03-28T13:14:24.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.431+0000 7f5409000640 1 -- 172.21.15.19:0/666378999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5404073080 con 0x7f5404155550 2024-03-28T13:14:24.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.439+0000 7f53fa000640 1 -- 172.21.15.19:0/666378999 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f53ec0589c0 con 0x7f5404155550 2024-03-28T13:14:24.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.442+0000 7f5409000640 1 -- 172.21.15.19:0/666378999 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f53e006c530 msgr2=0x7f53e006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.442+0000 7f5409000640 1 --2- 172.21.15.19:0/666378999 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f53e006c530 0x7f53e006e9f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f53f40012e0 tx=0x7f53f4009040 comp rx=0 tx=0).stop 2024-03-28T13:14:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.442+0000 7f5409000640 1 -- 172.21.15.19:0/666378999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5404155550 msgr2=0x7f5404152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.442+0000 7f5409000640 1 --2- 172.21.15.19:0/666378999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5404155550 0x7f5404152900 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00c4a0 tx=0x7f53ec00c960 comp rx=0 tx=0).stop 2024-03-28T13:14:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.442+0000 7f5409000640 1 -- 172.21.15.19:0/666378999 shutdown_connections 2024-03-28T13:14:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.442+0000 7f5409000640 1 --2- 172.21.15.19:0/666378999 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f53e006c530 0x7f53e006e9f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.442+0000 7f5409000640 1 --2- 172.21.15.19:0/666378999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5404155930 0x7f5404152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.442+0000 7f5409000640 1 --2- 172.21.15.19:0/666378999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5404155550 0x7f5404152900 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.442+0000 7f5409000640 1 --2- 172.21.15.19:0/666378999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54040d0990 0x7f54041442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.442+0000 7f5409000640 1 -- 172.21.15.19:0/666378999 >> 172.21.15.19:0/666378999 conn(0x7f540406b750 msgr2=0x7f540415c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.443+0000 7f5409000640 1 -- 172.21.15.19:0/666378999 shutdown_connections 2024-03-28T13:14:24.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:24.443+0000 7f5409000640 1 -- 172.21.15.19:0/666378999 wait complete. 2024-03-28T13:14:24.464 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-28T13:14:24.465 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:14:24.584 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:14:24.704 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-28T13:14:24.704 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:14:24.704 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:14:24.764 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:14:24.764 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:14:24.764 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:14:24.821 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:14:24.821 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:14:24.821 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:14:24.879 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:14:24.880 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:14:24.880 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:14:24.936 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:14:24.937 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-28T13:14:24.937 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:14:25.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.168+0000 7f7efa400640 1 Processor -- start 2024-03-28T13:14:25.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.169+0000 7f7efa400640 1 -- start start 2024-03-28T13:14:25.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.170+0000 7f7efa400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ef40cf700 0x7f7ef4168fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:25.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.170+0000 7f7efa400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ef415e880 con 0x7f7ef40d2310 2024-03-28T13:14:25.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.170+0000 7f7efa400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ef40d28d0 con 0x7f7ef415e4a0 2024-03-28T13:14:25.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.170+0000 7f7efa400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef40d34e0 con 0x7f7ef40cf700 2024-03-28T13:14:25.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.171+0000 7f7ef3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7ef40d2310 0x7f7ef40ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:49378/0 (socket says 172.21.15.19:49378) 2024-03-28T13:14:25.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.171+0000 7f7ef9a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ef40cf700 0x7f7ef4168fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:25.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.171+0000 7f7ef3e00640 1 -- 172.21.15.19:0/4099696242 learned_addr learned my addr 172.21.15.19:0/4099696242 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:25.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.171+0000 7f7ef9a00640 1 -- 172.21.15.19:0/4099696242 >> v1:172.21.15.188:6789/0 conn(0x7f7ef415e4a0 legacy=0x7f7ef40b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:25.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.171+0000 7f7ef9a00640 1 -- 172.21.15.19:0/4099696242 >> v1:172.21.15.19:6789/0 conn(0x7f7ef40d2310 legacy=0x7f7ef40ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:25.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.171+0000 7f7ef9a00640 1 -- 172.21.15.19:0/4099696242 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef40d2bb0 con 0x7f7ef40cf700 2024-03-28T13:14:25.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.172+0000 7f7ef9a00640 1 --2- 172.21.15.19:0/4099696242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ef40cf700 0x7f7ef4168fc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009d90 tx=0x7f7eec02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e563d6f665e3180 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:25.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.173+0000 7f7ef3400640 1 -- 172.21.15.19:0/4099696242 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec039070 con 0x7f7ef40cf700 2024-03-28T13:14:25.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.173+0000 7f7ef3400640 1 -- 172.21.15.19:0/4099696242 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec02cbd0 con 0x7f7ef40cf700 2024-03-28T13:14:25.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.173+0000 7f7ef3400640 1 -- 172.21.15.19:0/4099696242 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec034440 con 0x7f7ef40cf700 2024-03-28T13:14:25.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.173+0000 7f7efa400640 1 -- 172.21.15.19:0/4099696242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ef40cf700 msgr2=0x7f7ef4168fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:25.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.173+0000 7f7efa400640 1 --2- 172.21.15.19:0/4099696242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ef40cf700 0x7f7ef4168fc0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009d90 tx=0x7f7eec02be20 comp rx=0 tx=0).stop 2024-03-28T13:14:25.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.173+0000 7f7efa400640 1 -- 172.21.15.19:0/4099696242 shutdown_connections 2024-03-28T13:14:25.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.173+0000 7f7efa400640 1 --2- 172.21.15.19:0/4099696242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ef40cf700 0x7f7ef4168fc0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:25.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.173+0000 7f7efa400640 1 -- 172.21.15.19:0/4099696242 >> 172.21.15.19:0/4099696242 conn(0x7f7ef40676d0 msgr2=0x7f7ef4067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:25.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.174+0000 7f7efa400640 1 -- 172.21.15.19:0/4099696242 shutdown_connections 2024-03-28T13:14:25.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.174+0000 7f7efa400640 1 -- 172.21.15.19:0/4099696242 wait complete. 2024-03-28T13:14:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.175+0000 7f7efa400640 1 Processor -- start 2024-03-28T13:14:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.175+0000 7f7efa400640 1 -- start start 2024-03-28T13:14:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.176+0000 7f7efa400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ef40cf700 0x7f7ef40c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.176+0000 7f7efa400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ef40d2310 0x7f7ef4160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.176+0000 7f7efa400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ef415e4a0 0x7f7ef4158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.176+0000 7f7efa400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef40d3970 con 0x7f7ef415e4a0 2024-03-28T13:14:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.176+0000 7f7efa400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef40d3170 con 0x7f7ef40d2310 2024-03-28T13:14:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.176+0000 7f7efa400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef40b83f0 con 0x7f7ef40cf700 2024-03-28T13:14:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.177+0000 7f7ef9a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ef415e4a0 0x7f7ef4158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.177+0000 7f7ef9a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ef415e4a0 0x7f7ef4158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51174/0 (socket says 172.21.15.19:51174) 2024-03-28T13:14:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.177+0000 7f7ef9a00640 1 -- 172.21.15.19:0/1082540236 learned_addr learned my addr 172.21.15.19:0/1082540236 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.177+0000 7f7ef9a00640 1 -- 172.21.15.19:0/1082540236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ef40cf700 msgr2=0x7f7ef40c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:25.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.177+0000 7f7ef9a00640 1 --2- 172.21.15.19:0/1082540236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ef40cf700 0x7f7ef40c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:25.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.177+0000 7f7ef9a00640 1 -- 172.21.15.19:0/1082540236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ef40d2310 msgr2=0x7f7ef4160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:25.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.177+0000 7f7ef9a00640 1 --2- 172.21.15.19:0/1082540236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ef40d2310 0x7f7ef4160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:25.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.177+0000 7f7ef9a00640 1 -- 172.21.15.19:0/1082540236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef4160ad0 con 0x7f7ef415e4a0 2024-03-28T13:14:25.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.177+0000 7f7ef9a00640 1 --2- 172.21.15.19:0/1082540236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ef415e4a0 0x7f7ef4158070 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7eec037040 tx=0x7f7eec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:25.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.178+0000 7f7ef0c00640 1 -- 172.21.15.19:0/1082540236 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec039050 con 0x7f7ef415e4a0 2024-03-28T13:14:25.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.178+0000 7f7ef0c00640 1 -- 172.21.15.19:0/1082540236 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec03a040 con 0x7f7ef415e4a0 2024-03-28T13:14:25.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.178+0000 7f7ef0c00640 1 -- 172.21.15.19:0/1082540236 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec03e610 con 0x7f7ef415e4a0 2024-03-28T13:14:25.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.178+0000 7f7efa400640 1 -- 172.21.15.19:0/1082540236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef4160db0 con 0x7f7ef415e4a0 2024-03-28T13:14:25.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.178+0000 7f7efa400640 1 -- 172.21.15.19:0/1082540236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef4161370 con 0x7f7ef415e4a0 2024-03-28T13:14:25.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.179+0000 7f7efa400640 1 -- 172.21.15.19:0/1082540236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef4075940 con 0x7f7ef415e4a0 2024-03-28T13:14:25.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.180+0000 7f7ef0c00640 1 -- 172.21.15.19:0/1082540236 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7eec047020 con 0x7f7ef415e4a0 2024-03-28T13:14:25.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.181+0000 7f7ef0c00640 1 --2- 172.21.15.19:0/1082540236 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ed406c530 0x7f7ed406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:25.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.181+0000 7f7ef0c00640 1 -- 172.21.15.19:0/1082540236 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f7eec0ad7e0 con 0x7f7ef415e4a0 2024-03-28T13:14:25.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.181+0000 7f7ef9000640 1 --2- 172.21.15.19:0/1082540236 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ed406c530 0x7f7ed406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:25.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.183+0000 7f7ef9000640 1 --2- 172.21.15.19:0/1082540236 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ed406c530 0x7f7ed406e9f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8006fd0 tx=0x7f7ee8006bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:25.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.185+0000 7f7ef0c00640 1 -- 172.21.15.19:0/1082540236 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7eec04e020 con 0x7f7ef415e4a0 2024-03-28T13:14:25.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.433+0000 7f7efa400640 1 -- 172.21.15.19:0/1082540236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ef415d070 con 0x7f7ef415e4a0 2024-03-28T13:14:25.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.434+0000 7f7ef0c00640 1 -- 172.21.15.19:0/1082540236 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+3875 (secure 0 0 0) 0x7f7eec07c1c0 con 0x7f7ef415e4a0 2024-03-28T13:14:25.434 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:25.434 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5156,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1607201514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6836","nonce":1607201514},{"type":"v1","addr":"172.21.15.188:6837","nonce":1607201514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5404,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/1206740614","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1206740614},{"type":"v1","addr":"172.21.15.188:6835","nonce":1206740614}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5967,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2018671345","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2018671345},{"type":"v1","addr":"172.21.15.19:6834","nonce":2018671345}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":6417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1652184518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1652184518},{"type":"v1","addr":"172.21.15.19:6835","nonce":1652184518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":38}],"filesystems":[]} 2024-03-28T13:14:25.435 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 38 2024-03-28T13:14:25.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.437+0000 7f7efa400640 1 -- 172.21.15.19:0/1082540236 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ed406c530 msgr2=0x7f7ed406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:25.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.437+0000 7f7efa400640 1 --2- 172.21.15.19:0/1082540236 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ed406c530 0x7f7ed406e9f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8006fd0 tx=0x7f7ee8006bc0 comp rx=0 tx=0).stop 2024-03-28T13:14:25.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.437+0000 7f7efa400640 1 -- 172.21.15.19:0/1082540236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ef415e4a0 msgr2=0x7f7ef4158070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:25.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.437+0000 7f7efa400640 1 --2- 172.21.15.19:0/1082540236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ef415e4a0 0x7f7ef4158070 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f7eec037040 tx=0x7f7eec004a40 comp rx=0 tx=0).stop 2024-03-28T13:14:25.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.437+0000 7f7efa400640 1 -- 172.21.15.19:0/1082540236 shutdown_connections 2024-03-28T13:14:25.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.437+0000 7f7efa400640 1 --2- 172.21.15.19:0/1082540236 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ed406c530 0x7f7ed406e9f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:25.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.437+0000 7f7efa400640 1 --2- 172.21.15.19:0/1082540236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ef415e4a0 0x7f7ef4158070 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:25.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.437+0000 7f7efa400640 1 --2- 172.21.15.19:0/1082540236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ef40d2310 0x7f7ef4160380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:25.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.437+0000 7f7efa400640 1 --2- 172.21.15.19:0/1082540236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ef40cf700 0x7f7ef40c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:25.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.437+0000 7f7efa400640 1 -- 172.21.15.19:0/1082540236 >> 172.21.15.19:0/1082540236 conn(0x7f7ef40676d0 msgr2=0x7f7ef40ca6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:25.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.437+0000 7f7efa400640 1 -- 172.21.15.19:0/1082540236 shutdown_connections 2024-03-28T13:14:25.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.438+0000 7f7efa400640 1 -- 172.21.15.19:0/1082540236 wait complete. 2024-03-28T13:14:25.459 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-28T13:14:25.459 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-28T13:14:25.459 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-28T13:14:25.459 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:14:25.459 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-28T13:14:25.459 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-28T13:14:25.460 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-28T13:14:25.460 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:14:25.460 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-28T13:14:25.460 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-28T13:14:25.460 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-28T13:14:25.460 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:14:25.460 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-28T13:14:25.460 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-28T13:14:25.461 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-28T13:14:25.461 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:14:25.479 INFO:tasks.ceph.mds.b:Stopped 2024-03-28T13:14:25.480 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-28T13:14:25.482 INFO:tasks.ceph.mds.d:Stopped 2024-03-28T13:14:25.482 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-28T13:14:25.487 INFO:tasks.ceph.mds.c:Stopped 2024-03-28T13:14:25.487 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-28T13:14:25.488 INFO:tasks.ceph.mds.a:Stopped 2024-03-28T13:14:25.488 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-28T13:14:25.528 INFO:tasks.ceph.mds.b:Started 2024-03-28T13:14:25.529 DEBUG:teuthology.parallel:result is None 2024-03-28T13:14:25.535 INFO:tasks.ceph.mds.d:Started 2024-03-28T13:14:25.535 DEBUG:teuthology.parallel:result is None 2024-03-28T13:14:25.535 INFO:tasks.ceph.mds.c:Started 2024-03-28T13:14:25.536 DEBUG:teuthology.parallel:result is None 2024-03-28T13:14:25.542 INFO:tasks.ceph.mds.a:Started 2024-03-28T13:14:25.542 DEBUG:teuthology.parallel:result is None 2024-03-28T13:14:25.542 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-28T13:14:25.688 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-03-28T13:14:25.688 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-03-28T13:14:25.704 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-03-28T13:14:25.705 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-03-28T13:14:25.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.745+0000 7f0e6a200640 1 Processor -- start 2024-03-28T13:14:25.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.745+0000 7f0e6a200640 1 -- start start 2024-03-28T13:14:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.746+0000 7f0e6a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e640d2310 0x7f0e640ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.746+0000 7f0e6a200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0e640cf530 0x7f0e640b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.746+0000 7f0e6a200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e6415e880 con 0x7f0e6415e4a0 2024-03-28T13:14:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.746+0000 7f0e6a200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f0e640d34e0 con 0x7f0e640cf530 2024-03-28T13:14:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.746+0000 7f0e6a200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0e640d28d0 con 0x7f0e640d2310 2024-03-28T13:14:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.747+0000 7f0e69800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0e640cf530 0x7f0e640b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.747+0000 7f0e68e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f0e6415e4a0 0x7f0e640b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:42120/0 (socket says 172.21.15.19:42120) 2024-03-28T13:14:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.747+0000 7f0e68e00640 1 -- 172.21.15.19:0/895113296 learned_addr learned my addr 172.21.15.19:0/895113296 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.747+0000 7f0e63e00640 1 --2- 172.21.15.19:0/895113296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e640d2310 0x7f0e640ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.747+0000 7f0e63400640 1 -- 172.21.15.19:0/895113296 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2627892746 0 0) 0x7f0e6415e880 con 0x7f0e6415e4a0 2024-03-28T13:14:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.747+0000 7f0e63400640 1 -- 172.21.15.19:0/895113296 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e4c003520 con 0x7f0e6415e4a0 2024-03-28T13:14:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.747+0000 7f0e63e00640 1 -- 172.21.15.19:0/895113296 >> v2:172.21.15.188:3300/0 conn(0x7f0e640cf530 msgr2=0x7f0e640b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.747+0000 7f0e63e00640 1 --2- 172.21.15.19:0/895113296 >> v2:172.21.15.188:3300/0 conn(0x7f0e640cf530 0x7f0e640b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.747+0000 7f0e63e00640 1 -- 172.21.15.19:0/895113296 >> v1:172.21.15.188:6789/0 conn(0x7f0e6415e4a0 legacy=0x7f0e640b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.747+0000 7f0e63e00640 1 -- 172.21.15.19:0/895113296 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e640d2bb0 con 0x7f0e640d2310 2024-03-28T13:14:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.747+0000 7f0e69800640 1 --2- 172.21.15.19:0/895113296 >> v2:172.21.15.188:3300/0 conn(0x7f0e640cf530 0x7f0e640b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:14:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.749+0000 7f0e63e00640 1 --2- 172.21.15.19:0/895113296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e640d2310 0x7f0e640ceff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0e500097a0 tx=0x7f0e5002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a97fde616dc4e7bd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.749+0000 7f0e63400640 1 -- 172.21.15.19:0/895113296 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50035020 con 0x7f0e640d2310 2024-03-28T13:14:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.749+0000 7f0e63400640 1 -- 172.21.15.19:0/895113296 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e50031810 con 0x7f0e640d2310 2024-03-28T13:14:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.749+0000 7f0e63400640 1 -- 172.21.15.19:0/895113296 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e50031bc0 con 0x7f0e640d2310 2024-03-28T13:14:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.749+0000 7f0e6a200640 1 -- 172.21.15.19:0/895113296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e640d2310 msgr2=0x7f0e640ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.749+0000 7f0e6a200640 1 --2- 172.21.15.19:0/895113296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e640d2310 0x7f0e640ceff0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0e500097a0 tx=0x7f0e5002b8b0 comp rx=0 tx=0).stop 2024-03-28T13:14:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.750+0000 7f0e6a200640 1 -- 172.21.15.19:0/895113296 shutdown_connections 2024-03-28T13:14:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.750+0000 7f0e6a200640 1 --2- 172.21.15.19:0/895113296 >> v2:172.21.15.188:3300/0 conn(0x7f0e640cf530 0x7f0e640b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.750+0000 7f0e6a200640 1 --2- 172.21.15.19:0/895113296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e640d2310 0x7f0e640ceff0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.750+0000 7f0e6a200640 1 -- 172.21.15.19:0/895113296 >> 172.21.15.19:0/895113296 conn(0x7f0e640676d0 msgr2=0x7f0e64067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.750+0000 7f0e6a200640 1 -- 172.21.15.19:0/895113296 shutdown_connections 2024-03-28T13:14:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.750+0000 7f0e6a200640 1 -- 172.21.15.19:0/895113296 wait complete. 2024-03-28T13:14:25.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.751+0000 7f0e6a200640 1 Processor -- start 2024-03-28T13:14:25.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.752+0000 7f0e6a200640 1 -- start start 2024-03-28T13:14:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.752+0000 7f0e6a200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e640cf530 0x7f0e64155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.752+0000 7f0e6a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e640d2310 0x7f0e641614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.752+0000 7f0e6a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6415e4a0 0x7f0e64154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.752+0000 7f0e6a200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0e640d3970 con 0x7f0e6415e4a0 2024-03-28T13:14:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.752+0000 7f0e6a200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0e640d3170 con 0x7f0e640cf530 2024-03-28T13:14:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.752+0000 7f0e6a200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0e640b7920 con 0x7f0e640d2310 2024-03-28T13:14:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.753+0000 7f0e69800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6415e4a0 0x7f0e64154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.753+0000 7f0e69800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6415e4a0 0x7f0e64154300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40620/0 (socket says 172.21.15.19:40620) 2024-03-28T13:14:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.753+0000 7f0e69800640 1 -- 172.21.15.19:0/1962644377 learned_addr learned my addr 172.21.15.19:0/1962644377 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.753+0000 7f0e63e00640 1 --2- 172.21.15.19:0/1962644377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e640d2310 0x7f0e641614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.753+0000 7f0e68e00640 1 --2- 172.21.15.19:0/1962644377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e640cf530 0x7f0e64155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.753+0000 7f0e69800640 1 -- 172.21.15.19:0/1962644377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e640d2310 msgr2=0x7f0e641614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.753+0000 7f0e69800640 1 --2- 172.21.15.19:0/1962644377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e640d2310 0x7f0e641614c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.753+0000 7f0e69800640 1 -- 172.21.15.19:0/1962644377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e640cf530 msgr2=0x7f0e64155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.753+0000 7f0e69800640 1 --2- 172.21.15.19:0/1962644377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e640cf530 0x7f0e64155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.753+0000 7f0e69800640 1 -- 172.21.15.19:0/1962644377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e64154840 con 0x7f0e6415e4a0 2024-03-28T13:14:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.753+0000 7f0e68e00640 1 --2- 172.21.15.19:0/1962644377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e640cf530 0x7f0e64155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.754+0000 7f0e69800640 1 --2- 172.21.15.19:0/1962644377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6415e4a0 0x7f0e64154300 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00efc0 tx=0x7f0e5c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.754+0000 7f0e60c00640 1 -- 172.21.15.19:0/1962644377 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c018030 con 0x7f0e6415e4a0 2024-03-28T13:14:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.754+0000 7f0e60c00640 1 -- 172.21.15.19:0/1962644377 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e5c00cde0 con 0x7f0e6415e4a0 2024-03-28T13:14:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.754+0000 7f0e6a200640 1 -- 172.21.15.19:0/1962644377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e64155700 con 0x7f0e6415e4a0 2024-03-28T13:14:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.754+0000 7f0e60c00640 1 -- 172.21.15.19:0/1962644377 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c003bb0 con 0x7f0e6415e4a0 2024-03-28T13:14:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.754+0000 7f0e6a200640 1 -- 172.21.15.19:0/1962644377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e64155cc0 con 0x7f0e6415e4a0 2024-03-28T13:14:25.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.755+0000 7f0e6a200640 1 -- 172.21.15.19:0/1962644377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e64075940 con 0x7f0e6415e4a0 2024-03-28T13:14:25.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.756+0000 7f0e60c00640 1 -- 172.21.15.19:0/1962644377 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0e5c005d10 con 0x7f0e6415e4a0 2024-03-28T13:14:25.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.757+0000 7f0e60c00640 1 --2- 172.21.15.19:0/1962644377 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0e4406c530 0x7f0e4406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:25.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.757+0000 7f0e60c00640 1 -- 172.21.15.19:0/1962644377 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 4943+0+0 (secure 0 0 0) 0x7f0e5c08bed0 con 0x7f0e6415e4a0 2024-03-28T13:14:25.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.758+0000 7f0e68e00640 1 --2- 172.21.15.19:0/1962644377 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0e4406c530 0x7f0e4406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:25.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.759+0000 7f0e68e00640 1 --2- 172.21.15.19:0/1962644377 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0e4406c530 0x7f0e4406e9f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0e58005880 tx=0x7f0e58009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:25.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.762+0000 7f0e60c00640 1 -- 172.21.15.19:0/1962644377 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0e5c022070 con 0x7f0e6415e4a0 2024-03-28T13:14:25.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:25.979+0000 7f0e6a200640 1 -- 172.21.15.19:0/1962644377 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0e6415da30 con 0x7f0e6415e4a0 2024-03-28T13:14:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.015+0000 7f0e60c00640 1 -- 172.21.15.19:0/1962644377 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v73) ==== 97+0+0 (secure 0 0 0) 0x7f0e5c010040 con 0x7f0e6415e4a0 2024-03-28T13:14:27.015 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-03-28T13:14:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.019+0000 7f0e6a200640 1 -- 172.21.15.19:0/1962644377 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0e4406c530 msgr2=0x7f0e4406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.019+0000 7f0e6a200640 1 --2- 172.21.15.19:0/1962644377 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0e4406c530 0x7f0e4406e9f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0e58005880 tx=0x7f0e58009040 comp rx=0 tx=0).stop 2024-03-28T13:14:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.019+0000 7f0e6a200640 1 -- 172.21.15.19:0/1962644377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6415e4a0 msgr2=0x7f0e64154300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.019+0000 7f0e6a200640 1 --2- 172.21.15.19:0/1962644377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6415e4a0 0x7f0e64154300 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00efc0 tx=0x7f0e5c009f90 comp rx=0 tx=0).stop 2024-03-28T13:14:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.019+0000 7f0e6a200640 1 -- 172.21.15.19:0/1962644377 shutdown_connections 2024-03-28T13:14:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.019+0000 7f0e6a200640 1 --2- 172.21.15.19:0/1962644377 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0e4406c530 0x7f0e4406e9f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.019+0000 7f0e6a200640 1 --2- 172.21.15.19:0/1962644377 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0e6415e4a0 0x7f0e64154300 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.019+0000 7f0e6a200640 1 --2- 172.21.15.19:0/1962644377 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e640d2310 0x7f0e641614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.019+0000 7f0e6a200640 1 --2- 172.21.15.19:0/1962644377 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e640cf530 0x7f0e64155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.019+0000 7f0e6a200640 1 -- 172.21.15.19:0/1962644377 >> 172.21.15.19:0/1962644377 conn(0x7f0e640676d0 msgr2=0x7f0e640d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.019+0000 7f0e6a200640 1 -- 172.21.15.19:0/1962644377 shutdown_connections 2024-03-28T13:14:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.019+0000 7f0e6a200640 1 -- 172.21.15.19:0/1962644377 wait complete. 2024-03-28T13:14:27.042 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-28T13:14:27.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.246+0000 7f7713e00640 1 Processor -- start 2024-03-28T13:14:27.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.247+0000 7f7713e00640 1 -- start start 2024-03-28T13:14:27.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.248+0000 7f7713e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f77140d1380 0x7f77140d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.248+0000 7f7713e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77140d2fd0 con 0x7f77140d3a70 2024-03-28T13:14:27.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.248+0000 7f7713e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77140cfa70 con 0x7f771415e420 2024-03-28T13:14:27.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.248+0000 7f7713e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f77140cf790 con 0x7f77140d1380 2024-03-28T13:14:27.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.248+0000 7f7713400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f77140d3a70 0x7f77140d3e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58566/0 (socket says 172.21.15.19:58566) 2024-03-28T13:14:27.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.248+0000 7f7713400640 1 -- 172.21.15.19:0/2756496412 learned_addr learned my addr 172.21.15.19:0/2756496412 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:27.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.249+0000 7f7712000640 1 -- 172.21.15.19:0/2756496412 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1878660726 0 0) 0x7f77140d2fd0 con 0x7f77140d3a70 2024-03-28T13:14:27.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.249+0000 7f770a000640 1 --2- 172.21.15.19:0/2756496412 >> v2:172.21.15.188:3300/0 conn(0x7f77140d1380 0x7f77140d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:27.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.249+0000 7f7712000640 1 -- 172.21.15.19:0/2756496412 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76f8003520 con 0x7f77140d3a70 2024-03-28T13:14:27.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.249+0000 7f7712000640 1 -- 172.21.15.19:0/2756496412 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 731672505 0 0) 0x7f76f8003520 con 0x7f77140d3a70 2024-03-28T13:14:27.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.249+0000 7f7712000640 1 -- 172.21.15.19:0/2756496412 >> v2:172.21.15.188:3300/0 conn(0x7f77140d1380 msgr2=0x7f77140d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.249+0000 7f7712000640 1 --2- 172.21.15.19:0/2756496412 >> v2:172.21.15.188:3300/0 conn(0x7f77140d1380 0x7f77140d1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.249+0000 7f7712000640 1 -- 172.21.15.19:0/2756496412 >> v1:172.21.15.188:6789/0 conn(0x7f771415e420 legacy=0x7f77140b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.249+0000 7f7712000640 1 -- 172.21.15.19:0/2756496412 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77140d3270 con 0x7f77140d3a70 2024-03-28T13:14:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.249+0000 7f7712000640 1 -- 172.21.15.19:0/2756496412 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f770c0019f0 con 0x7f77140d3a70 2024-03-28T13:14:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.249+0000 7f7712000640 1 -- 172.21.15.19:0/2756496412 >> v1:172.21.15.19:6789/0 conn(0x7f77140d3a70 legacy=0x7f77140d3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.250+0000 7f7712000640 1 --2- 172.21.15.19:0/2756496412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76f8003a00 0x7f76f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.250+0000 7f7712000640 1 --2- 172.21.15.19:0/2756496412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76f8004800 0x7f76f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.250+0000 7f7712000640 1 --2- 172.21.15.19:0/2756496412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8005210 0x7f76f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.250+0000 7f7712000640 1 -- 172.21.15.19:0/2756496412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77140cf790 con 0x7f76f8005210 2024-03-28T13:14:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.250+0000 7f7712000640 1 -- 172.21.15.19:0/2756496412 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f76f800c150 con 0x7f76f8004800 2024-03-28T13:14:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.250+0000 7f7712000640 1 -- 172.21.15.19:0/2756496412 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f76f800c350 con 0x7f76f8003a00 2024-03-28T13:14:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.250+0000 7f7713400640 1 --2- 172.21.15.19:0/2756496412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8005210 0x7f76f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.250+0000 7f7713400640 1 -- 172.21.15.19:0/2756496412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76f8003a00 msgr2=0x7f76f8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.250+0000 7f7713400640 1 --2- 172.21.15.19:0/2756496412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76f8003a00 0x7f76f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.250+0000 7f7713400640 1 -- 172.21.15.19:0/2756496412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76f8004800 msgr2=0x7f76f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:27.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.250+0000 7f7713400640 1 --2- 172.21.15.19:0/2756496412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76f8004800 0x7f76f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.250+0000 7f7713400640 1 -- 172.21.15.19:0/2756496412 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f8003520 con 0x7f76f8005210 2024-03-28T13:14:27.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.251+0000 7f7713400640 1 -- 172.21.15.19:0/2756496412 reap_dead start 2024-03-28T13:14:27.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.251+0000 7f7713400640 1 --2- 172.21.15.19:0/2756496412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8005210 0x7f76f800b9c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f770c00a790 tx=0x7f76f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe37139daaf95c92 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:27.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.252+0000 7f7712000640 1 -- 172.21.15.19:0/2756496412 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c00a3b0 con 0x7f76f8005210 2024-03-28T13:14:27.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.252+0000 7f7712000640 1 -- 172.21.15.19:0/2756496412 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f770c034670 con 0x7f76f8005210 2024-03-28T13:14:27.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.252+0000 7f7712000640 1 -- 172.21.15.19:0/2756496412 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770c034a60 con 0x7f76f8005210 2024-03-28T13:14:27.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.252+0000 7f7713e00640 1 -- 172.21.15.19:0/2756496412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8005210 msgr2=0x7f76f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:27.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.252+0000 7f7713e00640 1 --2- 172.21.15.19:0/2756496412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8005210 0x7f76f800b9c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f770c00a790 tx=0x7f76f8005830 comp rx=0 tx=0).stop 2024-03-28T13:14:27.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.253+0000 7f7713e00640 1 -- 172.21.15.19:0/2756496412 shutdown_connections 2024-03-28T13:14:27.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.253+0000 7f7713e00640 1 --2- 172.21.15.19:0/2756496412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8005210 0x7f76f800b9c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.253+0000 7f7713e00640 1 -- 172.21.15.19:0/2756496412 >> 172.21.15.19:0/2756496412 conn(0x7f7714067670 msgr2=0x7f7714067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:27.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.253+0000 7f7713e00640 1 -- 172.21.15.19:0/2756496412 shutdown_connections 2024-03-28T13:14:27.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.253+0000 7f7713e00640 1 -- 172.21.15.19:0/2756496412 wait complete. 2024-03-28T13:14:27.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.254+0000 7f7713e00640 1 Processor -- start 2024-03-28T13:14:27.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.255+0000 7f7713e00640 1 -- start start 2024-03-28T13:14:27.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.255+0000 7f7713e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8005210 0x7f7714152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.255+0000 7f7713e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77140cd320 0x7f77140b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.255+0000 7f7713e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77140c9f20 0x7f77140ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.255+0000 7f7713e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77140cf420 con 0x7f76f8005210 2024-03-28T13:14:27.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.255+0000 7f7713e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f771415e800 con 0x7f77140c9f20 2024-03-28T13:14:27.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.255+0000 7f7713e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f77140d1810 con 0x7f77140cd320 2024-03-28T13:14:27.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.256+0000 7f7712a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8005210 0x7f7714152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:27.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.256+0000 7f7712a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8005210 0x7f7714152a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40630/0 (socket says 172.21.15.19:40630) 2024-03-28T13:14:27.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.256+0000 7f7712a00640 1 -- 172.21.15.19:0/2592129827 learned_addr learned my addr 172.21.15.19:0/2592129827 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:27.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.256+0000 7f770a000640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77140cd320 0x7f77140b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:27.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.256+0000 7f7713400640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77140c9f20 0x7f77140ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:27.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.256+0000 7f7712a00640 1 -- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77140cd320 msgr2=0x7f77140b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:27.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.256+0000 7f7712a00640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77140cd320 0x7f77140b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.256+0000 7f7712a00640 1 -- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77140c9f20 msgr2=0x7f77140ca3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:27.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.256+0000 7f7712a00640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77140c9f20 0x7f77140ca3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.256+0000 7f7712a00640 1 -- 172.21.15.19:0/2592129827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77140cc280 con 0x7f76f8005210 2024-03-28T13:14:27.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.256+0000 7f770a000640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77140cd320 0x7f77140b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:27.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.256+0000 7f7713400640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77140c9f20 0x7f77140ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:27.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.257+0000 7f7712a00640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8005210 0x7f7714152a60 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f770400c450 tx=0x7f770400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:27.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.257+0000 7f770b400640 1 -- 172.21.15.19:0/2592129827 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704017070 con 0x7f76f8005210 2024-03-28T13:14:27.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.257+0000 7f770b400640 1 -- 172.21.15.19:0/2592129827 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77040052d0 con 0x7f76f8005210 2024-03-28T13:14:27.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.257+0000 7f770b400640 1 -- 172.21.15.19:0/2592129827 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704003810 con 0x7f76f8005210 2024-03-28T13:14:27.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.257+0000 7f7713e00640 1 -- 172.21.15.19:0/2592129827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77140cc560 con 0x7f76f8005210 2024-03-28T13:14:27.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.257+0000 7f7713e00640 1 -- 172.21.15.19:0/2592129827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77140ccb20 con 0x7f76f8005210 2024-03-28T13:14:27.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.258+0000 7f7713e00640 1 -- 172.21.15.19:0/2592129827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7714075940 con 0x7f76f8005210 2024-03-28T13:14:27.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.259+0000 7f770b400640 1 -- 172.21.15.19:0/2592129827 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f770401a070 con 0x7f76f8005210 2024-03-28T13:14:27.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.260+0000 7f770b400640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f76f006c560 0x7f76f006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.261+0000 7f770b400640 1 -- 172.21.15.19:0/2592129827 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4857+0+0 (secure 0 0 0) 0x7f770408a740 con 0x7f76f8005210 2024-03-28T13:14:27.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.261+0000 7f770a000640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f76f006c560 0x7f76f006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:27.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.263+0000 7f770a000640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f76f006c560 0x7f76f006ea20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f77140ce7c0 tx=0x7f76fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:27.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.266+0000 7f770b400640 1 -- 172.21.15.19:0/2592129827 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f77040590a0 con 0x7f76f8005210 2024-03-28T13:14:27.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.543+0000 7f7713e00640 1 -- 172.21.15.19:0/2592129827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7714160ec0 con 0x7f76f8005210 2024-03-28T13:14:27.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.544+0000 7f770b400640 1 -- 172.21.15.19:0/2592129827 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7f77040589a0 con 0x7f76f8005210 2024-03-28T13:14:27.544 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:27.544 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:14:27.544 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-03-28T13:14:27.544 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.544 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-03-28T13:14:27.544 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmNRIkABAAq/nJioDZjp1zFWAYOLPBYw==", 2024-03-28T13:14:27.544 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmc10CIxAAQWCSjH0hW3KUk8/qQm91QA==", 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.546 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.546 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-03-28T13:14:27.546 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmmlUoEhAA4FUGXoZ/h/ve5fThJNdE2Q==", 2024-03-28T13:14:27.546 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.546 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:14:27.546 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:14:27.546 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:14:27.546 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:14:27.546 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.546 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.546 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-03-28T13:14:27.546 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmWDRnMxAA34LiLI79N5fVJnbcPacbQA==", 2024-03-28T13:14:27.546 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.547 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:14:27.547 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:14:27.547 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:14:27.547 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:14:27.547 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.547 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.547 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-03-28T13:14:27.547 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDYawVmxhwWLBAAm7C56UGConFx2D7Lbq2SSA==", 2024-03-28T13:14:27.547 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.547 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:14:27.547 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:14:27.547 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:14:27.547 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-03-28T13:14:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDbawVmC2BiNxAALQrAcXU44m2dTCjks5yy/Q==", 2024-03-28T13:14:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:14:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:14:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:14:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-03-28T13:14:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfawVmozrJFRAA7YV1uMucIW7H5SI4KY8aDw==", 2024-03-28T13:14:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:14:27.548 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiawVmccbTMBAATxk24Bf4LuQSqU92HdwgUg==", 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDpawVmdwnILRAAjUEBwnBEeSltu2CLRGhmDQ==", 2024-03-28T13:14:27.549 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.550 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:14:27.550 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:14:27.550 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:14:27.550 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.550 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.550 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-03-28T13:14:27.550 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDtawVmkwW5ARAAZJ/Nd/Wtq7LcbUjeqPNUag==", 2024-03-28T13:14:27.550 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.550 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:14:27.550 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:14:27.550 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:14:27.550 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDwawVm254wKRAAmCYODqSd2Whn4YKQzywrng==", 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDzawVmV8z+MhAA9JeF55K7ph1mQRWalT7XhA==", 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:14:27.551 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:14:27.552 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.552 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.552 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-03-28T13:14:27.552 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q==", 2024-03-28T13:14:27.552 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.552 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:14:27.552 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-03-28T13:14:27.552 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-28T13:14:27.552 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.552 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.552 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-03-28T13:14:27.553 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmJYGjHBAAAx2TtR0Gb3fafnZU26FCMA==", 2024-03-28T13:14:27.553 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.553 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:14:27.553 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:14:27.553 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:14:27.553 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:14:27.553 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.553 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.553 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-03-28T13:14:27.553 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmE7V/ERAAYcdvolIMDd5ff+pCn7hN1w==", 2024-03-28T13:14:27.553 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.553 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:14:27.554 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:14:27.554 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:14:27.554 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:14:27.554 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.554 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.554 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-03-28T13:14:27.554 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmk6q9JhAAeC3JHQDnK8IXGVoK6qGEFA==", 2024-03-28T13:14:27.554 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.554 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:14:27.554 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:14:27.554 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:14:27.554 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:14:27.555 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.555 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.555 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-03-28T13:14:27.555 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDRawVm9IzFLRAA5i1Yl7LQ7eNNHp7v4H2kog==", 2024-03-28T13:14:27.555 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.555 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:14:27.555 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:14:27.555 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:14:27.555 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:14:27.555 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.555 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.555 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-03-28T13:14:27.555 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDSawVm5H1FLxAAtyjOPcCEUQyvTpBdu3Fn3g==", 2024-03-28T13:14:27.556 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.556 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:14:27.556 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-03-28T13:14:27.556 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:14:27.556 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:14:27.556 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:27.556 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-03-28T13:14:27.556 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDSawVmebzfIRAAA0+qIIaprr4SfEZT6qRooA==", 2024-03-28T13:14:27.556 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:14:27.556 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:14:27.556 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-03-28T13:14:27.556 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:14:27.556 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.556 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:27.557 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-03-28T13:14:27.557 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:14:27.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.547+0000 7f7713e00640 1 -- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f76f006c560 msgr2=0x7f76f006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:27.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.547+0000 7f7713e00640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f76f006c560 0x7f76f006ea20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f77140ce7c0 tx=0x7f76fc008040 comp rx=0 tx=0).stop 2024-03-28T13:14:27.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.547+0000 7f7713e00640 1 -- 172.21.15.19:0/2592129827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8005210 msgr2=0x7f7714152a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:27.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.547+0000 7f7713e00640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8005210 0x7f7714152a60 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f770400c450 tx=0x7f770400c910 comp rx=0 tx=0).stop 2024-03-28T13:14:27.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.547+0000 7f7713e00640 1 -- 172.21.15.19:0/2592129827 shutdown_connections 2024-03-28T13:14:27.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.547+0000 7f7713e00640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f76f006c560 0x7f76f006ea20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.547+0000 7f7713e00640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77140c9f20 0x7f77140ca3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.547+0000 7f7713e00640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77140cd320 0x7f77140b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.547+0000 7f7713e00640 1 --2- 172.21.15.19:0/2592129827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f8005210 0x7f7714152a60 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.547+0000 7f7713e00640 1 -- 172.21.15.19:0/2592129827 >> 172.21.15.19:0/2592129827 conn(0x7f7714067670 msgr2=0x7f77141589e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:27.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.547+0000 7f7713e00640 1 -- 172.21.15.19:0/2592129827 shutdown_connections 2024-03-28T13:14:27.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.547+0000 7f7713e00640 1 -- 172.21.15.19:0/2592129827 wait complete. 2024-03-28T13:14:27.566 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:14:27.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.766+0000 7f2f09400640 1 Processor -- start 2024-03-28T13:14:27.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.766+0000 7f2f09400640 1 -- start start 2024-03-28T13:14:27.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.768+0000 7f2f09400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0415c390 0x7f2f04073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.768+0000 7f2f09400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2f040af830 0x7f2f04160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.768+0000 7f2f09400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f0415fe80 con 0x7f2f04160b80 2024-03-28T13:14:27.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.768+0000 7f2f09400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f2f0415c770 con 0x7f2f040af830 2024-03-28T13:14:27.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.768+0000 7f2f09400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2f040ad120 con 0x7f2f0415c390 2024-03-28T13:14:27.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.768+0000 7f2f03200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2f04160b80 0x7f2f040d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58582/0 (socket says 172.21.15.19:58582) 2024-03-28T13:14:27.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.768+0000 7f2f03200640 1 -- 172.21.15.19:0/1352459537 learned_addr learned my addr 172.21.15.19:0/1352459537 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.768+0000 7f2efbe00640 1 --2- 172.21.15.19:0/1352459537 >> v2:172.21.15.188:3300/0 conn(0x7f2f040af830 0x7f2f04160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.768+0000 7f2f02800640 1 --2- 172.21.15.19:0/1352459537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0415c390 0x7f2f04073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.768+0000 7f2f01e00640 1 -- 172.21.15.19:0/1352459537 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1075986140 0 0) 0x7f2f0415fe80 con 0x7f2f04160b80 2024-03-28T13:14:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2f01e00640 1 -- 172.21.15.19:0/1352459537 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ee8003520 con 0x7f2f04160b80 2024-03-28T13:14:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2f01e00640 1 -- 172.21.15.19:0/1352459537 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3087366422 0 0) 0x7f2ee8003520 con 0x7f2f04160b80 2024-03-28T13:14:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2f01e00640 1 -- 172.21.15.19:0/1352459537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0415c390 msgr2=0x7f2f04073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2f01e00640 1 --2- 172.21.15.19:0/1352459537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0415c390 0x7f2f04073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2f01e00640 1 -- 172.21.15.19:0/1352459537 >> v2:172.21.15.188:3300/0 conn(0x7f2f040af830 msgr2=0x7f2f04160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2f01e00640 1 --2- 172.21.15.19:0/1352459537 >> v2:172.21.15.188:3300/0 conn(0x7f2f040af830 0x7f2f04160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2f01e00640 1 -- 172.21.15.19:0/1352459537 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f040ad400 con 0x7f2f04160b80 2024-03-28T13:14:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2f01e00640 1 -- 172.21.15.19:0/1352459537 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f2efc0019f0 con 0x7f2f04160b80 2024-03-28T13:14:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2f01e00640 1 -- 172.21.15.19:0/1352459537 >> v1:172.21.15.19:6789/0 conn(0x7f2f04160b80 legacy=0x7f2f040d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2f01e00640 1 --2- 172.21.15.19:0/1352459537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee80039b0 0x7f2ee8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2f01e00640 1 --2- 172.21.15.19:0/1352459537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ee8004800 0x7f2ee8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2f01e00640 1 --2- 172.21.15.19:0/1352459537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0415c390 0x7f2ee80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2f01e00640 1 -- 172.21.15.19:0/1352459537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f040ad120 con 0x7f2ee8004800 2024-03-28T13:14:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2f01e00640 1 -- 172.21.15.19:0/1352459537 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee800b9c0 con 0x7f2ee80039b0 2024-03-28T13:14:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2f01e00640 1 -- 172.21.15.19:0/1352459537 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee800bb40 con 0x7f2f0415c390 2024-03-28T13:14:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.769+0000 7f2efbe00640 1 --2- 172.21.15.19:0/1352459537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ee8004800 0x7f2ee8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.770+0000 7f2f02800640 1 --2- 172.21.15.19:0/1352459537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee80039b0 0x7f2ee8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:27.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.770+0000 7f2efbe00640 1 -- 172.21.15.19:0/1352459537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0415c390 msgr2=0x7f2ee80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:27.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.770+0000 7f2f03200640 1 --2- 172.21.15.19:0/1352459537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0415c390 0x7f2ee80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:27.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.770+0000 7f2efbe00640 1 --2- 172.21.15.19:0/1352459537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0415c390 0x7f2ee80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.770+0000 7f2efbe00640 1 -- 172.21.15.19:0/1352459537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee80039b0 msgr2=0x7f2ee8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:27.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.770+0000 7f2efbe00640 1 --2- 172.21.15.19:0/1352459537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ee80039b0 0x7f2ee8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.770+0000 7f2efbe00640 1 -- 172.21.15.19:0/1352459537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee8003520 con 0x7f2ee8004800 2024-03-28T13:14:27.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.770+0000 7f2f03200640 1 --2- 172.21.15.19:0/1352459537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0415c390 0x7f2ee80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:27.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.771+0000 7f2efbe00640 1 --2- 172.21.15.19:0/1352459537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ee8004800 0x7f2ee8004c80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009ac0 tx=0x7f2ee8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8f3f4ce4cf706d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:27.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.771+0000 7f2f01e00640 1 -- 172.21.15.19:0/1352459537 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec039070 con 0x7f2ee8004800 2024-03-28T13:14:27.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.771+0000 7f2f01e00640 1 -- 172.21.15.19:0/1352459537 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eec02c710 con 0x7f2ee8004800 2024-03-28T13:14:27.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.771+0000 7f2f01e00640 1 -- 172.21.15.19:0/1352459537 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec009690 con 0x7f2ee8004800 2024-03-28T13:14:27.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.772+0000 7f2f09400640 1 -- 172.21.15.19:0/1352459537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ee8004800 msgr2=0x7f2ee8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:27.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.772+0000 7f2f09400640 1 --2- 172.21.15.19:0/1352459537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ee8004800 0x7f2ee8004c80 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2eec009ac0 tx=0x7f2ee8005830 comp rx=0 tx=0).stop 2024-03-28T13:14:27.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.772+0000 7f2f03200640 1 -- 172.21.15.19:0/1352459537 reap_dead start 2024-03-28T13:14:27.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.772+0000 7f2f09400640 1 -- 172.21.15.19:0/1352459537 shutdown_connections 2024-03-28T13:14:27.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.772+0000 7f2f09400640 1 -- 172.21.15.19:0/1352459537 >> 172.21.15.19:0/1352459537 conn(0x7f2f0406b990 msgr2=0x7f2f0406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:27.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.772+0000 7f2f09400640 1 -- 172.21.15.19:0/1352459537 shutdown_connections 2024-03-28T13:14:27.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.772+0000 7f2f09400640 1 -- 172.21.15.19:0/1352459537 wait complete. 2024-03-28T13:14:27.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.774+0000 7f2f09400640 1 Processor -- start 2024-03-28T13:14:27.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.774+0000 7f2f09400640 1 -- start start 2024-03-28T13:14:27.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.775+0000 7f2f09400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ee8004800 0x7f2f041f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.775+0000 7f2f09400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f0415d7c0 0x7f2f041f8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.775+0000 7f2f09400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0415dc40 0x7f2f041f85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.775+0000 7f2f09400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2f040cf980 con 0x7f2ee8004800 2024-03-28T13:14:27.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.775+0000 7f2f09400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2f04160120 con 0x7f2f0415d7c0 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.775+0000 7f2f09400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2f040d0d60 con 0x7f2f0415dc40 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.775+0000 7f2f02800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ee8004800 0x7f2f041f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.775+0000 7f2f02800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ee8004800 0x7f2f041f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40650/0 (socket says 172.21.15.19:40650) 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.775+0000 7f2f02800640 1 -- 172.21.15.19:0/1405693922 learned_addr learned my addr 172.21.15.19:0/1405693922 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.775+0000 7f2efbe00640 1 --2- 172.21.15.19:0/1405693922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f0415d7c0 0x7f2f041f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.775+0000 7f2f03200640 1 --2- 172.21.15.19:0/1405693922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0415dc40 0x7f2f041f85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.775+0000 7f2f02800640 1 -- 172.21.15.19:0/1405693922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0415dc40 msgr2=0x7f2f041f85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.776+0000 7f2f02800640 1 --2- 172.21.15.19:0/1405693922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0415dc40 0x7f2f041f85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.776+0000 7f2f02800640 1 -- 172.21.15.19:0/1405693922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f0415d7c0 msgr2=0x7f2f041f8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.776+0000 7f2f02800640 1 --2- 172.21.15.19:0/1405693922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f0415d7c0 0x7f2f041f8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.776+0000 7f2f02800640 1 -- 172.21.15.19:0/1405693922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0415e210 con 0x7f2ee8004800 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.776+0000 7f2efbe00640 1 --2- 172.21.15.19:0/1405693922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f0415d7c0 0x7f2f041f8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.776+0000 7f2f02800640 1 --2- 172.21.15.19:0/1405693922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ee8004800 0x7f2f041f4310 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4011600 tx=0x7f2ef4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.776+0000 7f2efaa00640 1 -- 172.21.15.19:0/1405693922 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef400f920 con 0x7f2ee8004800 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.776+0000 7f2efaa00640 1 -- 172.21.15.19:0/1405693922 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef400fac0 con 0x7f2ee8004800 2024-03-28T13:14:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.776+0000 7f2efaa00640 1 -- 172.21.15.19:0/1405693922 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef400fe70 con 0x7f2ee8004800 2024-03-28T13:14:27.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.776+0000 7f2f09400640 1 -- 172.21.15.19:0/1405693922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0415e4f0 con 0x7f2ee8004800 2024-03-28T13:14:27.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.776+0000 7f2f09400640 1 -- 172.21.15.19:0/1405693922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0415eab0 con 0x7f2ee8004800 2024-03-28T13:14:27.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.778+0000 7f2f09400640 1 -- 172.21.15.19:0/1405693922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f040685c0 con 0x7f2ee8004800 2024-03-28T13:14:27.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.779+0000 7f2efaa00640 1 -- 172.21.15.19:0/1405693922 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2ef4015460 con 0x7f2ee8004800 2024-03-28T13:14:27.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.779+0000 7f2efaa00640 1 --2- 172.21.15.19:0/1405693922 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ee006c530 0x7f2ee006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:27.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.780+0000 7f2efaa00640 1 -- 172.21.15.19:0/1405693922 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4857+0+0 (secure 0 0 0) 0x7f2ef408bba0 con 0x7f2ee8004800 2024-03-28T13:14:27.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.780+0000 7f2efbe00640 1 --2- 172.21.15.19:0/1405693922 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ee006c530 0x7f2ee006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:27.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.781+0000 7f2efbe00640 1 --2- 172.21.15.19:0/1405693922 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ee006c530 0x7f2ee006e9f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2eec036000 tx=0x7f2eec030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:27.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.784+0000 7f2efaa00640 1 -- 172.21.15.19:0/1405693922 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2ef4019030 con 0x7f2ee8004800 2024-03-28T13:14:27.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.992+0000 7f2f09400640 1 -- 172.21.15.19:0/1405693922 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f0406f1b0 con 0x7f2ee8004800 2024-03-28T13:14:27.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:27.996+0000 7f2efaa00640 1 -- 172.21.15.19:0/1405693922 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+12891 (secure 0 0 0) 0x7f2ef405a5a0 con 0x7f2ee8004800 2024-03-28T13:14:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:27.996 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":73,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:14:27.005555+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:14:28.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.000+0000 7f2f09400640 1 -- 172.21.15.19:0/1405693922 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ee006c530 msgr2=0x7f2ee006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:28.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.000+0000 7f2f09400640 1 --2- 172.21.15.19:0/1405693922 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ee006c530 0x7f2ee006e9f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2eec036000 tx=0x7f2eec030000 comp rx=0 tx=0).stop 2024-03-28T13:14:28.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.000+0000 7f2f09400640 1 -- 172.21.15.19:0/1405693922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ee8004800 msgr2=0x7f2f041f4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:28.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.000+0000 7f2f09400640 1 --2- 172.21.15.19:0/1405693922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ee8004800 0x7f2f041f4310 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4011600 tx=0x7f2ef4011ac0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.000+0000 7f2f09400640 1 -- 172.21.15.19:0/1405693922 shutdown_connections 2024-03-28T13:14:28.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.000+0000 7f2f09400640 1 --2- 172.21.15.19:0/1405693922 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ee006c530 0x7f2ee006e9f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.000+0000 7f2f09400640 1 --2- 172.21.15.19:0/1405693922 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f0415dc40 0x7f2f041f85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.000+0000 7f2f09400640 1 --2- 172.21.15.19:0/1405693922 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f0415d7c0 0x7f2f041f8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.000+0000 7f2f09400640 1 --2- 172.21.15.19:0/1405693922 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ee8004800 0x7f2f041f4310 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.000+0000 7f2f09400640 1 -- 172.21.15.19:0/1405693922 >> 172.21.15.19:0/1405693922 conn(0x7f2f0406b990 msgr2=0x7f2f04157690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:28.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.000+0000 7f2f09400640 1 -- 172.21.15.19:0/1405693922 shutdown_connections 2024-03-28T13:14:28.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.000+0000 7f2f09400640 1 -- 172.21.15.19:0/1405693922 wait complete. 2024-03-28T13:14:28.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:14:28.021 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:14:28.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.241+0000 7f7886000640 1 Processor -- start 2024-03-28T13:14:28.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.242+0000 7f7886000640 1 -- start start 2024-03-28T13:14:28.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.243+0000 7f7886000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f78800afd50 0x7f788006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:28.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.243+0000 7f7886000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f78800b3760 0x7f78800b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:28.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.244+0000 7f7886000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7880161f50 0x7f78801623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:28.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.244+0000 7f7886000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f78800b4100 con 0x7f78800afd50 2024-03-28T13:14:28.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.244+0000 7f7886000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f788015ed90 con 0x7f78800b3760 2024-03-28T13:14:28.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.244+0000 7f7886000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f788015f070 con 0x7f7880161f50 2024-03-28T13:14:28.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.244+0000 7f7884c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f78800afd50 0x7f788006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:28.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.244+0000 7f7884c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f78800afd50 0x7f788006ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40652/0 (socket says 172.21.15.19:40652) 2024-03-28T13:14:28.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.244+0000 7f7884c00640 1 -- 172.21.15.19:0/2246764638 learned_addr learned my addr 172.21.15.19:0/2246764638 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:28.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.244+0000 7f7884c00640 1 -- 172.21.15.19:0/2246764638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7880161f50 msgr2=0x7f78801623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:28.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.244+0000 7f7884c00640 1 --2- 172.21.15.19:0/2246764638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7880161f50 0x7f78801623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.244+0000 7f7884c00640 1 -- 172.21.15.19:0/2246764638 >> v2:172.21.15.188:3300/0 conn(0x7f78800b3760 msgr2=0x7f78800b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:28.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.244+0000 7f7884c00640 1 --2- 172.21.15.19:0/2246764638 >> v2:172.21.15.188:3300/0 conn(0x7f78800b3760 0x7f78800b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.244+0000 7f7884c00640 1 -- 172.21.15.19:0/2246764638 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788015f350 con 0x7f78800afd50 2024-03-28T13:14:28.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.245+0000 7f7884c00640 1 --2- 172.21.15.19:0/2246764638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78800afd50 0x7f788006ea20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f78800b5770 tx=0x7f787002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42375ff0fa1b592f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:28.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.246+0000 7f787f400640 1 -- 172.21.15.19:0/2246764638 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787002ca80 con 0x7f78800afd50 2024-03-28T13:14:28.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.246+0000 7f787f400640 1 -- 172.21.15.19:0/2246764638 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f787002cc20 con 0x7f78800afd50 2024-03-28T13:14:28.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.246+0000 7f787f400640 1 -- 172.21.15.19:0/2246764638 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78700343f0 con 0x7f78800afd50 2024-03-28T13:14:28.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.247+0000 7f7886000640 1 -- 172.21.15.19:0/2246764638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78800afd50 msgr2=0x7f788006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:28.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.247+0000 7f7886000640 1 --2- 172.21.15.19:0/2246764638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78800afd50 0x7f788006ea20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f78800b5770 tx=0x7f787002bee0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.247+0000 7f7886000640 1 -- 172.21.15.19:0/2246764638 shutdown_connections 2024-03-28T13:14:28.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.247+0000 7f7886000640 1 --2- 172.21.15.19:0/2246764638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7880161f50 0x7f78801623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.247+0000 7f7886000640 1 --2- 172.21.15.19:0/2246764638 >> v2:172.21.15.188:3300/0 conn(0x7f78800b3760 0x7f78800b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.247+0000 7f7886000640 1 --2- 172.21.15.19:0/2246764638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78800afd50 0x7f788006ea20 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.247+0000 7f7886000640 1 -- 172.21.15.19:0/2246764638 >> 172.21.15.19:0/2246764638 conn(0x7f78800d0770 msgr2=0x7f78800d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:28.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.247+0000 7f7886000640 1 -- 172.21.15.19:0/2246764638 shutdown_connections 2024-03-28T13:14:28.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.247+0000 7f7886000640 1 -- 172.21.15.19:0/2246764638 wait complete. 2024-03-28T13:14:28.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.249+0000 7f7886000640 1 Processor -- start 2024-03-28T13:14:28.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.249+0000 7f7886000640 1 -- start start 2024-03-28T13:14:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.250+0000 7f7886000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78800afd50 0x7f7880153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.250+0000 7f7886000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78800b3760 0x7f78801606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.250+0000 7f7886000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7880161f50 0x7f7880150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.250+0000 7f7886000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f788006fa20 con 0x7f78800afd50 2024-03-28T13:14:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.250+0000 7f7886000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f78800b0130 con 0x7f7880161f50 2024-03-28T13:14:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.250+0000 7f7886000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f788006fba0 con 0x7f78800b3760 2024-03-28T13:14:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.250+0000 7f7884c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78800afd50 0x7f7880153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.250+0000 7f7884c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78800afd50 0x7f7880153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40668/0 (socket says 172.21.15.19:40668) 2024-03-28T13:14:28.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.250+0000 7f7884c00640 1 -- 172.21.15.19:0/3562171283 learned_addr learned my addr 172.21.15.19:0/3562171283 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:28.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.250+0000 7f7885600640 1 --2- 172.21.15.19:0/3562171283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7880161f50 0x7f7880150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.251+0000 7f7884c00640 1 -- 172.21.15.19:0/3562171283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78800b3760 msgr2=0x7f78801606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.251+0000 7f7884c00640 1 --2- 172.21.15.19:0/3562171283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78800b3760 0x7f78801606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.251+0000 7f7884c00640 1 -- 172.21.15.19:0/3562171283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7880161f50 msgr2=0x7f7880150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.251+0000 7f7884c00640 1 --2- 172.21.15.19:0/3562171283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7880161f50 0x7f7880150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.251+0000 7f7884c00640 1 -- 172.21.15.19:0/3562171283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7880153710 con 0x7f78800afd50 2024-03-28T13:14:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.251+0000 7f7884c00640 1 --2- 172.21.15.19:0/3562171283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78800afd50 0x7f7880153130 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f78700093e0 tx=0x7f7870004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.251+0000 7f787cc00640 1 -- 172.21.15.19:0/3562171283 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870043020 con 0x7f78800afd50 2024-03-28T13:14:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.251+0000 7f787cc00640 1 -- 172.21.15.19:0/3562171283 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f787003e450 con 0x7f78800afd50 2024-03-28T13:14:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.251+0000 7f787cc00640 1 -- 172.21.15.19:0/3562171283 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787003e840 con 0x7f78800afd50 2024-03-28T13:14:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.251+0000 7f7886000640 1 -- 172.21.15.19:0/3562171283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788015fbf0 con 0x7f78800afd50 2024-03-28T13:14:28.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.252+0000 7f7886000640 1 -- 172.21.15.19:0/3562171283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78801601b0 con 0x7f78800afd50 2024-03-28T13:14:28.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.253+0000 7f7886000640 1 -- 172.21.15.19:0/3562171283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7880075940 con 0x7f78800afd50 2024-03-28T13:14:28.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.253+0000 7f787cc00640 1 -- 172.21.15.19:0/3562171283 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7870046020 con 0x7f78800afd50 2024-03-28T13:14:28.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.254+0000 7f787cc00640 1 --2- 172.21.15.19:0/3562171283 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f785c06c530 0x7f785c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:28.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.255+0000 7f787cc00640 1 -- 172.21.15.19:0/3562171283 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4857+0+0 (secure 0 0 0) 0x7f78700add20 con 0x7f78800afd50 2024-03-28T13:14:28.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.255+0000 7f787fe00640 1 --2- 172.21.15.19:0/3562171283 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f785c06c530 0x7f785c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:28.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.256+0000 7f787fe00640 1 --2- 172.21.15.19:0/3562171283 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f785c06c530 0x7f785c06e9f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f78800afab0 tx=0x7f7868006c90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:28.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.259+0000 7f787cc00640 1 -- 172.21.15.19:0/3562171283 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f78700b2070 con 0x7f78800afd50 2024-03-28T13:14:28.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.463+0000 7f7886000640 1 -- 172.21.15.19:0/3562171283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7880067590 con 0x7f78800afd50 2024-03-28T13:14:28.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.464+0000 7f787cc00640 1 -- 172.21.15.19:0/3562171283 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f787004d030 con 0x7f78800afd50 2024-03-28T13:14:28.464 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:14:28.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.467+0000 7f7886000640 1 -- 172.21.15.19:0/3562171283 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f785c06c530 msgr2=0x7f785c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:28.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.467+0000 7f7886000640 1 --2- 172.21.15.19:0/3562171283 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f785c06c530 0x7f785c06e9f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f78800afab0 tx=0x7f7868006c90 comp rx=0 tx=0).stop 2024-03-28T13:14:28.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.467+0000 7f7886000640 1 -- 172.21.15.19:0/3562171283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78800afd50 msgr2=0x7f7880153130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:28.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.467+0000 7f7886000640 1 --2- 172.21.15.19:0/3562171283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78800afd50 0x7f7880153130 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f78700093e0 tx=0x7f7870004a40 comp rx=0 tx=0).stop 2024-03-28T13:14:28.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.467+0000 7f7886000640 1 -- 172.21.15.19:0/3562171283 shutdown_connections 2024-03-28T13:14:28.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.467+0000 7f7886000640 1 --2- 172.21.15.19:0/3562171283 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f785c06c530 0x7f785c06e9f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.467+0000 7f7886000640 1 --2- 172.21.15.19:0/3562171283 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7880161f50 0x7f7880150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.467+0000 7f7886000640 1 --2- 172.21.15.19:0/3562171283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78800b3760 0x7f78801606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.467+0000 7f7886000640 1 --2- 172.21.15.19:0/3562171283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78800afd50 0x7f7880153130 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.467+0000 7f7886000640 1 -- 172.21.15.19:0/3562171283 >> 172.21.15.19:0/3562171283 conn(0x7f78800d0770 msgr2=0x7f788006e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:28.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.468+0000 7f7886000640 1 -- 172.21.15.19:0/3562171283 shutdown_connections 2024-03-28T13:14:28.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.468+0000 7f7886000640 1 -- 172.21.15.19:0/3562171283 wait complete. 2024-03-28T13:14:28.488 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-28T13:14:28.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.708+0000 7f2e13400640 1 Processor -- start 2024-03-28T13:14:28.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.708+0000 7f2e13400640 1 -- start start 2024-03-28T13:14:28.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.710+0000 7f2e13400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2e0c162920 0x7f2e0c1670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:28.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.710+0000 7f2e13400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2e0c0d0990 0x7f2e0c071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:28.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.710+0000 7f2e13400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e0c071d60 con 0x7f2e0c163ac0 2024-03-28T13:14:28.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.710+0000 7f2e13400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2e0c163f20 con 0x7f2e0c162920 2024-03-28T13:14:28.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.710+0000 7f2e13400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f2e0c1665d0 con 0x7f2e0c0d0990 2024-03-28T13:14:28.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.711+0000 7f2e0be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2e0c163ac0 0x7f2e0c0d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58592/0 (socket says 172.21.15.19:58592) 2024-03-28T13:14:28.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.711+0000 7f2e0be00640 1 -- 172.21.15.19:0/1242446351 learned_addr learned my addr 172.21.15.19:0/1242446351 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:28.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.711+0000 7f2e10a00640 1 --2- 172.21.15.19:0/1242446351 >> v2:172.21.15.19:3300/0 conn(0x7f2e0c162920 0x7f2e0c1670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:28.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.711+0000 7f2e11400640 1 --2- 172.21.15.19:0/1242446351 >> v2:172.21.15.188:3300/0 conn(0x7f2e0c0d0990 0x7f2e0c071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:28.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.711+0000 7f2e0b400640 1 -- 172.21.15.19:0/1242446351 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3760673152 0 0) 0x7f2e0c071d60 con 0x7f2e0c163ac0 2024-03-28T13:14:28.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.711+0000 7f2e0b400640 1 -- 172.21.15.19:0/1242446351 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2df4003520 con 0x7f2e0c163ac0 2024-03-28T13:14:28.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.711+0000 7f2e10a00640 1 -- 172.21.15.19:0/1242446351 >> v2:172.21.15.188:3300/0 conn(0x7f2e0c0d0990 msgr2=0x7f2e0c071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:28.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.711+0000 7f2e10a00640 1 --2- 172.21.15.19:0/1242446351 >> v2:172.21.15.188:3300/0 conn(0x7f2e0c0d0990 0x7f2e0c071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.711+0000 7f2e10a00640 1 -- 172.21.15.19:0/1242446351 >> v1:172.21.15.19:6789/0 conn(0x7f2e0c163ac0 legacy=0x7f2e0c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:28.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.711+0000 7f2e10a00640 1 -- 172.21.15.19:0/1242446351 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c1668b0 con 0x7f2e0c162920 2024-03-28T13:14:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.713+0000 7f2e10a00640 1 --2- 172.21.15.19:0/1242446351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e0c162920 0x7f2e0c1670b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0cfff0 tx=0x7f2e0002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a945c9b5a85a91 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.713+0000 7f2e0b400640 1 -- 172.21.15.19:0/1242446351 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e000315a0 con 0x7f2e0c162920 2024-03-28T13:14:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.713+0000 7f2e0b400640 1 -- 172.21.15.19:0/1242446351 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e00037070 con 0x7f2e0c162920 2024-03-28T13:14:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.713+0000 7f2e0b400640 1 -- 172.21.15.19:0/1242446351 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00031940 con 0x7f2e0c162920 2024-03-28T13:14:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.714+0000 7f2e13400640 1 -- 172.21.15.19:0/1242446351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e0c162920 msgr2=0x7f2e0c1670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.714+0000 7f2e13400640 1 --2- 172.21.15.19:0/1242446351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e0c162920 0x7f2e0c1670b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0cfff0 tx=0x7f2e0002b840 comp rx=0 tx=0).stop 2024-03-28T13:14:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.714+0000 7f2e13400640 1 -- 172.21.15.19:0/1242446351 shutdown_connections 2024-03-28T13:14:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.714+0000 7f2e13400640 1 --2- 172.21.15.19:0/1242446351 >> v2:172.21.15.188:3300/0 conn(0x7f2e0c0d0990 0x7f2e0c071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.714+0000 7f2e13400640 1 --2- 172.21.15.19:0/1242446351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e0c162920 0x7f2e0c1670b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.714+0000 7f2e13400640 1 -- 172.21.15.19:0/1242446351 >> 172.21.15.19:0/1242446351 conn(0x7f2e0c06b750 msgr2=0x7f2e0c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.714+0000 7f2e13400640 1 -- 172.21.15.19:0/1242446351 shutdown_connections 2024-03-28T13:14:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.714+0000 7f2e13400640 1 -- 172.21.15.19:0/1242446351 wait complete. 2024-03-28T13:14:28.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.716+0000 7f2e13400640 1 Processor -- start 2024-03-28T13:14:28.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.716+0000 7f2e13400640 1 -- start start 2024-03-28T13:14:28.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.717+0000 7f2e13400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e0c0d0990 0x7f2e0c09aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:28.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.717+0000 7f2e13400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e0c162920 0x7f2e0c0b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:28.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.717+0000 7f2e13400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e0c163ac0 0x7f2e0c0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:28.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.717+0000 7f2e13400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c072040 con 0x7f2e0c0d0990 2024-03-28T13:14:28.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.717+0000 7f2e13400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0c162d00 con 0x7f2e0c163ac0 2024-03-28T13:14:28.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.717+0000 7f2e13400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2e0c0721c0 con 0x7f2e0c162920 2024-03-28T13:14:28.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.717+0000 7f2e10a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e0c0d0990 0x7f2e0c09aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:28.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.717+0000 7f2e10a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e0c0d0990 0x7f2e0c09aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40682/0 (socket says 172.21.15.19:40682) 2024-03-28T13:14:28.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.717+0000 7f2e10a00640 1 -- 172.21.15.19:0/1408307952 learned_addr learned my addr 172.21.15.19:0/1408307952 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.717+0000 7f2e10a00640 1 -- 172.21.15.19:0/1408307952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e0c162920 msgr2=0x7f2e0c0b3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.717+0000 7f2e10a00640 1 --2- 172.21.15.19:0/1408307952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e0c162920 0x7f2e0c0b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.717+0000 7f2e10a00640 1 -- 172.21.15.19:0/1408307952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e0c163ac0 msgr2=0x7f2e0c0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.717+0000 7f2e10a00640 1 --2- 172.21.15.19:0/1408307952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e0c163ac0 0x7f2e0c0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.717+0000 7f2e10a00640 1 -- 172.21.15.19:0/1408307952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0c160c50 con 0x7f2e0c0d0990 2024-03-28T13:14:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.718+0000 7f2e10a00640 1 --2- 172.21.15.19:0/1408307952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e0c0d0990 0x7f2e0c09aba0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2e0002b880 tx=0x7f2e000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.718+0000 7f2e08c00640 1 -- 172.21.15.19:0/1408307952 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00032630 con 0x7f2e0c0d0990 2024-03-28T13:14:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.718+0000 7f2e08c00640 1 -- 172.21.15.19:0/1408307952 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e000327d0 con 0x7f2e0c0d0990 2024-03-28T13:14:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.718+0000 7f2e08c00640 1 -- 172.21.15.19:0/1408307952 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e00032ba0 con 0x7f2e0c0d0990 2024-03-28T13:14:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.718+0000 7f2e13400640 1 -- 172.21.15.19:0/1408307952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0c160f30 con 0x7f2e0c0d0990 2024-03-28T13:14:28.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.718+0000 7f2e13400640 1 -- 172.21.15.19:0/1408307952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e0c1614f0 con 0x7f2e0c0d0990 2024-03-28T13:14:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.719+0000 7f2e13400640 1 -- 172.21.15.19:0/1408307952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e0c0685c0 con 0x7f2e0c0d0990 2024-03-28T13:14:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.720+0000 7f2e08c00640 1 -- 172.21.15.19:0/1408307952 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2e0004e430 con 0x7f2e0c0d0990 2024-03-28T13:14:28.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.721+0000 7f2e08c00640 1 --2- 172.21.15.19:0/1408307952 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2dec06c530 0x7f2dec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:28.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.721+0000 7f2e08c00640 1 -- 172.21.15.19:0/1408307952 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4857+0+0 (secure 0 0 0) 0x7f2e0002f030 con 0x7f2e0c0d0990 2024-03-28T13:14:28.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.722+0000 7f2e0be00640 1 --2- 172.21.15.19:0/1408307952 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2dec06c530 0x7f2dec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:28.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.723+0000 7f2e0be00640 1 --2- 172.21.15.19:0/1408307952 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2dec06c530 0x7f2dec06e9f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c1fb920 tx=0x7f2df80078f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:28.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.726+0000 7f2e08c00640 1 -- 172.21.15.19:0/1408307952 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2e00040070 con 0x7f2e0c0d0990 2024-03-28T13:14:28.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.936+0000 7f2e13400640 1 -- 172.21.15.19:0/1408307952 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2e0c15fe10 con 0x7f2e0c0d0990 2024-03-28T13:14:28.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.937+0000 7f2e08c00640 1 -- 172.21.15.19:0/1408307952 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7f2e0007bc50 con 0x7f2e0c0d0990 2024-03-28T13:14:28.937 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:28.937 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-03-28T13:14:28.937 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:14:28.937 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-03-28T13:14:28.937 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-03-28T13:14:28.938 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:28.938 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-03-28T13:14:28.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.940+0000 7f2e13400640 1 -- 172.21.15.19:0/1408307952 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2dec06c530 msgr2=0x7f2dec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:28.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.940+0000 7f2e13400640 1 --2- 172.21.15.19:0/1408307952 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2dec06c530 0x7f2dec06e9f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c1fb920 tx=0x7f2df80078f0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.940+0000 7f2e13400640 1 -- 172.21.15.19:0/1408307952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e0c0d0990 msgr2=0x7f2e0c09aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:28.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.940+0000 7f2e13400640 1 --2- 172.21.15.19:0/1408307952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e0c0d0990 0x7f2e0c09aba0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f2e0002b880 tx=0x7f2e000046c0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.940+0000 7f2e13400640 1 -- 172.21.15.19:0/1408307952 shutdown_connections 2024-03-28T13:14:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.940+0000 7f2e13400640 1 --2- 172.21.15.19:0/1408307952 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2dec06c530 0x7f2dec06e9f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.940+0000 7f2e13400640 1 --2- 172.21.15.19:0/1408307952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e0c163ac0 0x7f2e0c0b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.940+0000 7f2e13400640 1 --2- 172.21.15.19:0/1408307952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2e0c162920 0x7f2e0c0b3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.940+0000 7f2e13400640 1 --2- 172.21.15.19:0/1408307952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e0c0d0990 0x7f2e0c09aba0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.940+0000 7f2e13400640 1 -- 172.21.15.19:0/1408307952 >> 172.21.15.19:0/1408307952 conn(0x7f2e0c06b750 msgr2=0x7f2e0c0aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.940+0000 7f2e13400640 1 -- 172.21.15.19:0/1408307952 shutdown_connections 2024-03-28T13:14:28.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:28.940+0000 7f2e13400640 1 -- 172.21.15.19:0/1408307952 wait complete. 2024-03-28T13:14:28.960 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-28T13:14:28.961 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-28T13:14:29.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.187+0000 7f9d36400640 1 Processor -- start 2024-03-28T13:14:29.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.187+0000 7f9d36400640 1 -- start start 2024-03-28T13:14:29.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.189+0000 7f9d36400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9d3013f190 0x7f9d300ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:29.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.189+0000 7f9d36400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d30141fa0 0x7f9d3006db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:29.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.189+0000 7f9d36400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d300cfbf0 con 0x7f9d300ced40 2024-03-28T13:14:29.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.189+0000 7f9d36400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9d3013f570 con 0x7f9d3013f190 2024-03-28T13:14:29.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.189+0000 7f9d36400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9d300cf350 con 0x7f9d30141fa0 2024-03-28T13:14:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.189+0000 7f9d2fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9d300ced40 0x7f9d30141890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58598/0 (socket says 172.21.15.19:58598) 2024-03-28T13:14:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.190+0000 7f9d2fe00640 1 -- 172.21.15.19:0/3360288238 learned_addr learned my addr 172.21.15.19:0/3360288238 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.189+0000 7f9d35000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9d3013f190 0x7f9d300ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.190+0000 7f9d35a00640 1 --2- 172.21.15.19:0/3360288238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d30141fa0 0x7f9d3006db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.190+0000 7f9d35000640 1 -- 172.21.15.19:0/3360288238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d30141fa0 msgr2=0x7f9d3006db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.190+0000 7f9d35000640 1 --2- 172.21.15.19:0/3360288238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d30141fa0 0x7f9d3006db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.190+0000 7f9d35000640 1 -- 172.21.15.19:0/3360288238 >> v1:172.21.15.19:6789/0 conn(0x7f9d300ced40 legacy=0x7f9d30141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.190+0000 7f9d35000640 1 -- 172.21.15.19:0/3360288238 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d300cf630 con 0x7f9d3013f190 2024-03-28T13:14:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.191+0000 7f9d35000640 1 --2- 172.21.15.19:0/3360288238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d3013f190 0x7f9d300ce800 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009d90 tx=0x7f9d2002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f2f98fe3f2315c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.192+0000 7f9d2f400640 1 -- 172.21.15.19:0/3360288238 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20039070 con 0x7f9d3013f190 2024-03-28T13:14:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.192+0000 7f9d2f400640 1 -- 172.21.15.19:0/3360288238 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d2002cc50 con 0x7f9d3013f190 2024-03-28T13:14:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.192+0000 7f9d2f400640 1 -- 172.21.15.19:0/3360288238 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20034440 con 0x7f9d3013f190 2024-03-28T13:14:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.192+0000 7f9d36400640 1 -- 172.21.15.19:0/3360288238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d3013f190 msgr2=0x7f9d300ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.192+0000 7f9d36400640 1 --2- 172.21.15.19:0/3360288238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d3013f190 0x7f9d300ce800 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f9d20009d90 tx=0x7f9d2002bea0 comp rx=0 tx=0).stop 2024-03-28T13:14:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.192+0000 7f9d36400640 1 -- 172.21.15.19:0/3360288238 shutdown_connections 2024-03-28T13:14:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.192+0000 7f9d36400640 1 --2- 172.21.15.19:0/3360288238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d30141fa0 0x7f9d3006db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.192+0000 7f9d36400640 1 --2- 172.21.15.19:0/3360288238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d3013f190 0x7f9d300ce800 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.193+0000 7f9d36400640 1 -- 172.21.15.19:0/3360288238 >> 172.21.15.19:0/3360288238 conn(0x7f9d300b1d80 msgr2=0x7f9d300b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.193+0000 7f9d36400640 1 -- 172.21.15.19:0/3360288238 shutdown_connections 2024-03-28T13:14:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.193+0000 7f9d36400640 1 -- 172.21.15.19:0/3360288238 wait complete. 2024-03-28T13:14:29.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.194+0000 7f9d36400640 1 Processor -- start 2024-03-28T13:14:29.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.195+0000 7f9d36400640 1 -- start start 2024-03-28T13:14:29.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.195+0000 7f9d36400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d300ced40 0x7f9d30140980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:29.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.196+0000 7f9d36400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d3013f190 0x7f9d301538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:29.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.196+0000 7f9d36400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d30141fa0 0x7f9d3013a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:29.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.196+0000 7f9d36400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9d30142550 con 0x7f9d300ced40 2024-03-28T13:14:29.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.196+0000 7f9d36400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9d3006d670 con 0x7f9d3013f190 2024-03-28T13:14:29.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.196+0000 7f9d36400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9d301426d0 con 0x7f9d30141fa0 2024-03-28T13:14:29.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.196+0000 7f9d35000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d300ced40 0x7f9d30140980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.196+0000 7f9d35000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d300ced40 0x7f9d30140980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40712/0 (socket says 172.21.15.19:40712) 2024-03-28T13:14:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.196+0000 7f9d35000640 1 -- 172.21.15.19:0/3693133310 learned_addr learned my addr 172.21.15.19:0/3693133310 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.196+0000 7f9d35000640 1 -- 172.21.15.19:0/3693133310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d30141fa0 msgr2=0x7f9d3013a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.196+0000 7f9d35000640 1 --2- 172.21.15.19:0/3693133310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d30141fa0 0x7f9d3013a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.196+0000 7f9d35000640 1 -- 172.21.15.19:0/3693133310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d3013f190 msgr2=0x7f9d301538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.196+0000 7f9d35000640 1 --2- 172.21.15.19:0/3693133310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d3013f190 0x7f9d301538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.196+0000 7f9d35000640 1 -- 172.21.15.19:0/3693133310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d30071850 con 0x7f9d300ced40 2024-03-28T13:14:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.197+0000 7f9d35000640 1 --2- 172.21.15.19:0/3693133310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d300ced40 0x7f9d30140980 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9d20037040 tx=0x7f9d20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.197+0000 7f9d2cc00640 1 -- 172.21.15.19:0/3693133310 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20039050 con 0x7f9d300ced40 2024-03-28T13:14:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.197+0000 7f9d2cc00640 1 -- 172.21.15.19:0/3693133310 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d20045070 con 0x7f9d300ced40 2024-03-28T13:14:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.197+0000 7f9d2cc00640 1 -- 172.21.15.19:0/3693133310 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2003e610 con 0x7f9d300ced40 2024-03-28T13:14:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.197+0000 7f9d36400640 1 -- 172.21.15.19:0/3693133310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d30071b30 con 0x7f9d300ced40 2024-03-28T13:14:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.197+0000 7f9d36400640 1 -- 172.21.15.19:0/3693133310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d301f8670 con 0x7f9d300ced40 2024-03-28T13:14:29.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.198+0000 7f9d36400640 1 -- 172.21.15.19:0/3693133310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d30075940 con 0x7f9d300ced40 2024-03-28T13:14:29.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.199+0000 7f9d2cc00640 1 -- 172.21.15.19:0/3693133310 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9d2003e7b0 con 0x7f9d300ced40 2024-03-28T13:14:29.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.200+0000 7f9d2cc00640 1 --2- 172.21.15.19:0/3693133310 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9d1006c530 0x7f9d1006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:29.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.201+0000 7f9d2cc00640 1 -- 172.21.15.19:0/3693133310 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4857+0+0 (secure 0 0 0) 0x7f9d200ae320 con 0x7f9d300ced40 2024-03-28T13:14:29.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.201+0000 7f9d2fe00640 1 --2- 172.21.15.19:0/3693133310 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9d1006c530 0x7f9d1006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:29.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.202+0000 7f9d2fe00640 1 --2- 172.21.15.19:0/3693133310 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9d1006c530 0x7f9d1006e9f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c005880 tx=0x7f9d1c007a20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:29.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.205+0000 7f9d2cc00640 1 -- 172.21.15.19:0/3693133310 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9d2007cd10 con 0x7f9d300ced40 2024-03-28T13:14:29.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:29.410+0000 7f9d36400640 1 -- 172.21.15.19:0/3693133310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9d301563e0 con 0x7f9d300ced40 2024-03-28T13:14:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.017+0000 7f9d2cc00640 1 -- 172.21.15.19:0/3693133310 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7f9d2007c610 con 0x7f9d300ced40 2024-03-28T13:14:30.017 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-03-28T13:14:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.021+0000 7f9d36400640 1 -- 172.21.15.19:0/3693133310 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9d1006c530 msgr2=0x7f9d1006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.021+0000 7f9d36400640 1 --2- 172.21.15.19:0/3693133310 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9d1006c530 0x7f9d1006e9f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c005880 tx=0x7f9d1c007a20 comp rx=0 tx=0).stop 2024-03-28T13:14:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.021+0000 7f9d36400640 1 -- 172.21.15.19:0/3693133310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d300ced40 msgr2=0x7f9d30140980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.021+0000 7f9d36400640 1 --2- 172.21.15.19:0/3693133310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d300ced40 0x7f9d30140980 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f9d20037040 tx=0x7f9d20004a40 comp rx=0 tx=0).stop 2024-03-28T13:14:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.021+0000 7f9d36400640 1 -- 172.21.15.19:0/3693133310 shutdown_connections 2024-03-28T13:14:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.021+0000 7f9d36400640 1 --2- 172.21.15.19:0/3693133310 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9d1006c530 0x7f9d1006e9f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.021+0000 7f9d36400640 1 --2- 172.21.15.19:0/3693133310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d30141fa0 0x7f9d3013a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.021+0000 7f9d36400640 1 --2- 172.21.15.19:0/3693133310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d3013f190 0x7f9d301538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.021+0000 7f9d36400640 1 --2- 172.21.15.19:0/3693133310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9d300ced40 0x7f9d30140980 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.021+0000 7f9d36400640 1 -- 172.21.15.19:0/3693133310 >> 172.21.15.19:0/3693133310 conn(0x7f9d300b1d80 msgr2=0x7f9d3013b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.021+0000 7f9d36400640 1 -- 172.21.15.19:0/3693133310 shutdown_connections 2024-03-28T13:14:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.021+0000 7f9d36400640 1 -- 172.21.15.19:0/3693133310 wait complete. 2024-03-28T13:14:30.049 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-28T13:14:30.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.261+0000 7f37fac00640 1 Processor -- start 2024-03-28T13:14:30.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.261+0000 7f37fac00640 1 -- start start 2024-03-28T13:14:30.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.263+0000 7f37fac00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f37f406e030 0x7f37f415fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:30.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.263+0000 7f37fac00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f37f41602d0 0x7f37f40cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:30.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.263+0000 7f37fac00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f4142880 con 0x7f37f413f300 2024-03-28T13:14:30.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.263+0000 7f37fac00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f37f4160720 con 0x7f37f406e030 2024-03-28T13:14:30.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.263+0000 7f37fac00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f37f40cee90 con 0x7f37f41602d0 2024-03-28T13:14:30.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.263+0000 7f37f8e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f37f406e030 0x7f37f415fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:30.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.263+0000 7f37f9800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f37f413f300 0x7f37f41419e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58600/0 (socket says 172.21.15.19:58600) 2024-03-28T13:14:30.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.263+0000 7f37f9800640 1 -- 172.21.15.19:0/4218982505 learned_addr learned my addr 172.21.15.19:0/4218982505 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:30.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.264+0000 7f37fa200640 1 --2- 172.21.15.19:0/4218982505 >> v2:172.21.15.188:3300/0 conn(0x7f37f41602d0 0x7f37f40cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:30.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.264+0000 7f37f8e00640 1 -- 172.21.15.19:0/4218982505 >> v2:172.21.15.188:3300/0 conn(0x7f37f41602d0 msgr2=0x7f37f40cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:30.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.264+0000 7f37f8e00640 1 --2- 172.21.15.19:0/4218982505 >> v2:172.21.15.188:3300/0 conn(0x7f37f41602d0 0x7f37f40cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:30.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.264+0000 7f37f8e00640 1 -- 172.21.15.19:0/4218982505 >> v1:172.21.15.19:6789/0 conn(0x7f37f413f300 legacy=0x7f37f41419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:30.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.264+0000 7f37f8e00640 1 -- 172.21.15.19:0/4218982505 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f41422c0 con 0x7f37f406e030 2024-03-28T13:14:30.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.265+0000 7f37f8e00640 1 --2- 172.21.15.19:0/4218982505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f406e030 0x7f37f415fd90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f37f40b30a0 tx=0x7f37e002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b7c6d95de33ad9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:30.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.265+0000 7f37f3e00640 1 -- 172.21.15.19:0/4218982505 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0035020 con 0x7f37f406e030 2024-03-28T13:14:30.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.265+0000 7f37f3e00640 1 -- 172.21.15.19:0/4218982505 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e0031830 con 0x7f37f406e030 2024-03-28T13:14:30.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.265+0000 7f37f3e00640 1 -- 172.21.15.19:0/4218982505 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0031c00 con 0x7f37f406e030 2024-03-28T13:14:30.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.266+0000 7f37fac00640 1 -- 172.21.15.19:0/4218982505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f406e030 msgr2=0x7f37f415fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:30.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.266+0000 7f37fac00640 1 --2- 172.21.15.19:0/4218982505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f406e030 0x7f37f415fd90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f37f40b30a0 tx=0x7f37e002b8d0 comp rx=0 tx=0).stop 2024-03-28T13:14:30.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.266+0000 7f37fac00640 1 -- 172.21.15.19:0/4218982505 shutdown_connections 2024-03-28T13:14:30.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.266+0000 7f37fac00640 1 --2- 172.21.15.19:0/4218982505 >> v2:172.21.15.188:3300/0 conn(0x7f37f41602d0 0x7f37f40cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:30.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.266+0000 7f37fac00640 1 --2- 172.21.15.19:0/4218982505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f406e030 0x7f37f415fd90 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:30.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.266+0000 7f37fac00640 1 -- 172.21.15.19:0/4218982505 >> 172.21.15.19:0/4218982505 conn(0x7f37f406b570 msgr2=0x7f37f406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:30.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.266+0000 7f37fac00640 1 -- 172.21.15.19:0/4218982505 shutdown_connections 2024-03-28T13:14:30.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.266+0000 7f37fac00640 1 -- 172.21.15.19:0/4218982505 wait complete. 2024-03-28T13:14:30.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.268+0000 7f37fac00640 1 Processor -- start 2024-03-28T13:14:30.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.268+0000 7f37fac00640 1 -- start start 2024-03-28T13:14:30.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.269+0000 7f37fac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f406e030 0x7f37f40b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:30.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.269+0000 7f37fac00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37f413f300 0x7f37f40b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:30.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.269+0000 7f37fac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37f41602d0 0x7f37f40ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:30.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.269+0000 7f37fac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37f40cf860 con 0x7f37f406e030 2024-03-28T13:14:30.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.269+0000 7f37fac00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f37f40ceb20 con 0x7f37f413f300 2024-03-28T13:14:30.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.269+0000 7f37fac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f37f40cf9e0 con 0x7f37f41602d0 2024-03-28T13:14:30.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.269+0000 7f37f9800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f406e030 0x7f37f40b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:30.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.270+0000 7f37f9800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f406e030 0x7f37f40b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40726/0 (socket says 172.21.15.19:40726) 2024-03-28T13:14:30.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.270+0000 7f37f9800640 1 -- 172.21.15.19:0/3777246393 learned_addr learned my addr 172.21.15.19:0/3777246393 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:30.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.270+0000 7f37f8e00640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37f413f300 0x7f37f40b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:30.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.270+0000 7f37f9800640 1 -- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37f41602d0 msgr2=0x7f37f40ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:30.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.270+0000 7f37fa200640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37f41602d0 0x7f37f40ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:30.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.270+0000 7f37f9800640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37f41602d0 0x7f37f40ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:30.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.270+0000 7f37f9800640 1 -- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37f413f300 msgr2=0x7f37f40b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:30.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.270+0000 7f37f9800640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37f413f300 0x7f37f40b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:30.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.270+0000 7f37f9800640 1 -- 172.21.15.19:0/3777246393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f40ae060 con 0x7f37f406e030 2024-03-28T13:14:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.270+0000 7f37f8e00640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37f413f300 0x7f37f40b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.270+0000 7f37fa200640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37f41602d0 0x7f37f40ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.271+0000 7f37f9800640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f406e030 0x7f37f40b1ed0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f37e800c430 tx=0x7f37e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.271+0000 7f37f1600640 1 -- 172.21.15.19:0/3777246393 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8017070 con 0x7f37f406e030 2024-03-28T13:14:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.271+0000 7f37f1600640 1 -- 172.21.15.19:0/3777246393 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e8005ce0 con 0x7f37f406e030 2024-03-28T13:14:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.271+0000 7f37fac00640 1 -- 172.21.15.19:0/3777246393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f40ae340 con 0x7f37f406e030 2024-03-28T13:14:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.271+0000 7f37f1600640 1 -- 172.21.15.19:0/3777246393 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e80072f0 con 0x7f37f406e030 2024-03-28T13:14:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.271+0000 7f37fac00640 1 -- 172.21.15.19:0/3777246393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f40a9760 con 0x7f37f406e030 2024-03-28T13:14:30.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.272+0000 7f37fac00640 1 -- 172.21.15.19:0/3777246393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f4075940 con 0x7f37f406e030 2024-03-28T13:14:30.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.273+0000 7f37f1600640 1 -- 172.21.15.19:0/3777246393 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f37e801a070 con 0x7f37f406e030 2024-03-28T13:14:30.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.274+0000 7f37f1600640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f37d406c530 0x7f37d406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:30.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.274+0000 7f37f1600640 1 -- 172.21.15.19:0/3777246393 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5227+0+0 (secure 0 0 0) 0x7f37e808a8d0 con 0x7f37f406e030 2024-03-28T13:14:30.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.275+0000 7f37f8e00640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f37d406c530 0x7f37d406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:30.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.276+0000 7f37f8e00640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f37d406c530 0x7f37d406e9f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f37e0036000 tx=0x7f37e0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.280+0000 7f37f1600640 1 -- 172.21.15.19:0/3777246393 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f37e80590e0 con 0x7f37f406e030 2024-03-28T13:14:30.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:30.480+0000 7f37fac00640 1 -- 172.21.15.19:0/3777246393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f37f40cf640 con 0x7f37f406e030 2024-03-28T13:14:31.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.032+0000 7f37f1600640 1 -- 172.21.15.19:0/3777246393 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) ==== 170+0+0 (secure 0 0 0) 0x7f37e80589e0 con 0x7f37f406e030 2024-03-28T13:14:31.032 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-03-28T13:14:31.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.036+0000 7f37fac00640 1 -- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f37d406c530 msgr2=0x7f37d406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:31.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.036+0000 7f37fac00640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f37d406c530 0x7f37d406e9f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f37e0036000 tx=0x7f37e0008030 comp rx=0 tx=0).stop 2024-03-28T13:14:31.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.036+0000 7f37fac00640 1 -- 172.21.15.19:0/3777246393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f406e030 msgr2=0x7f37f40b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.036+0000 7f37fac00640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f406e030 0x7f37f40b1ed0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f37e800c430 tx=0x7f37e800c8f0 comp rx=0 tx=0).stop 2024-03-28T13:14:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.036+0000 7f37fac00640 1 -- 172.21.15.19:0/3777246393 shutdown_connections 2024-03-28T13:14:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.036+0000 7f37fac00640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f37d406c530 0x7f37d406e9f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.037+0000 7f37fac00640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f37f41602d0 0x7f37f40ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.037+0000 7f37fac00640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37f413f300 0x7f37f40b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.037+0000 7f37fac00640 1 --2- 172.21.15.19:0/3777246393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f406e030 0x7f37f40b1ed0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.037+0000 7f37fac00640 1 -- 172.21.15.19:0/3777246393 >> 172.21.15.19:0/3777246393 conn(0x7f37f406b570 msgr2=0x7f37f40cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.037+0000 7f37fac00640 1 -- 172.21.15.19:0/3777246393 shutdown_connections 2024-03-28T13:14:31.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.037+0000 7f37fac00640 1 -- 172.21.15.19:0/3777246393 wait complete. 2024-03-28T13:14:31.079 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-28T13:14:31.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.263+0000 7fde6ee00640 1 Processor -- start 2024-03-28T13:14:31.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.263+0000 7fde6ee00640 1 -- start start 2024-03-28T13:14:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.265+0000 7fde6ee00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fde68155930 0x7fde680d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.265+0000 7fde6ee00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde680d07c0 0x7fde680aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.265+0000 7fde6ee00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde680b40b0 con 0x7fde68155550 2024-03-28T13:14:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.265+0000 7fde6ee00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fde680d0c10 con 0x7fde680d07c0 2024-03-28T13:14:31.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.265+0000 7fde6ee00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fde680b4390 con 0x7fde68155930 2024-03-28T13:14:31.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.265+0000 7fde6ce00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde680d07c0 0x7fde680aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:31.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.265+0000 7fde6ce00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde680d07c0 0x7fde680aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40730/0 (socket says 172.21.15.19:40730) 2024-03-28T13:14:31.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.265+0000 7fde6ce00640 1 -- 172.21.15.19:0/2875729614 learned_addr learned my addr 172.21.15.19:0/2875729614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:31.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.265+0000 7fde67400640 1 --2- 172.21.15.19:0/2875729614 >> v2:172.21.15.188:3300/0 conn(0x7fde68155930 0x7fde680d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:31.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.265+0000 7fde6ce00640 1 -- 172.21.15.19:0/2875729614 >> v2:172.21.15.188:3300/0 conn(0x7fde68155930 msgr2=0x7fde680d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:31.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.265+0000 7fde6ce00640 1 --2- 172.21.15.19:0/2875729614 >> v2:172.21.15.188:3300/0 conn(0x7fde68155930 0x7fde680d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:31.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.265+0000 7fde6ce00640 1 -- 172.21.15.19:0/2875729614 >> v1:172.21.15.188:6789/0 conn(0x7fde68155550 legacy=0x7fde6815f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:31.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.265+0000 7fde6ce00640 1 -- 172.21.15.19:0/2875729614 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde680b4670 con 0x7fde680d07c0 2024-03-28T13:14:31.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.267+0000 7fde6ce00640 1 --2- 172.21.15.19:0/2875729614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde680d07c0 0x7fde680aede0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fde68073970 tx=0x7fde6002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef345a52ef5e5c7e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:31.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.267+0000 7fde66a00640 1 -- 172.21.15.19:0/2875729614 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6002c9f0 con 0x7fde680d07c0 2024-03-28T13:14:31.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.267+0000 7fde66a00640 1 -- 172.21.15.19:0/2875729614 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde6002cb90 con 0x7fde680d07c0 2024-03-28T13:14:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.267+0000 7fde66a00640 1 -- 172.21.15.19:0/2875729614 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde600343f0 con 0x7fde680d07c0 2024-03-28T13:14:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.267+0000 7fde6ee00640 1 -- 172.21.15.19:0/2875729614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde680d07c0 msgr2=0x7fde680aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.267+0000 7fde6ee00640 1 --2- 172.21.15.19:0/2875729614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde680d07c0 0x7fde680aede0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fde68073970 tx=0x7fde6002be50 comp rx=0 tx=0).stop 2024-03-28T13:14:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.268+0000 7fde6ee00640 1 -- 172.21.15.19:0/2875729614 shutdown_connections 2024-03-28T13:14:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.268+0000 7fde6ee00640 1 --2- 172.21.15.19:0/2875729614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde680d07c0 0x7fde680aede0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.268+0000 7fde6ee00640 1 --2- 172.21.15.19:0/2875729614 >> v2:172.21.15.188:3300/0 conn(0x7fde68155930 0x7fde680d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.268+0000 7fde6ee00640 1 -- 172.21.15.19:0/2875729614 >> 172.21.15.19:0/2875729614 conn(0x7fde6806b750 msgr2=0x7fde6806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.268+0000 7fde6ee00640 1 -- 172.21.15.19:0/2875729614 shutdown_connections 2024-03-28T13:14:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.268+0000 7fde6ee00640 1 -- 172.21.15.19:0/2875729614 wait complete. 2024-03-28T13:14:31.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.269+0000 7fde6ee00640 1 Processor -- start 2024-03-28T13:14:31.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.269+0000 7fde6ee00640 1 -- start start 2024-03-28T13:14:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.270+0000 7fde6ee00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde680d07c0 0x7fde68151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.270+0000 7fde6ee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde68155550 0x7fde681f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.270+0000 7fde6ee00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde68155930 0x7fde680b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.271+0000 7fde6ee00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde6815ba20 con 0x7fde68155550 2024-03-28T13:14:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.271+0000 7fde6ee00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fde680b4c30 con 0x7fde68155930 2024-03-28T13:14:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.271+0000 7fde6ee00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fde6815bba0 con 0x7fde680d07c0 2024-03-28T13:14:31.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.271+0000 7fde67400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde68155550 0x7fde681f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:31.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.271+0000 7fde67400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde68155550 0x7fde681f8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40732/0 (socket says 172.21.15.19:40732) 2024-03-28T13:14:31.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.271+0000 7fde67400640 1 -- 172.21.15.19:0/2621240006 learned_addr learned my addr 172.21.15.19:0/2621240006 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:31.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.271+0000 7fde6ce00640 1 --2- 172.21.15.19:0/2621240006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde68155930 0x7fde680b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:31.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.271+0000 7fde67e00640 1 --2- 172.21.15.19:0/2621240006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde680d07c0 0x7fde68151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:31.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.271+0000 7fde67400640 1 -- 172.21.15.19:0/2621240006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde680d07c0 msgr2=0x7fde68151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:31.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.271+0000 7fde67400640 1 --2- 172.21.15.19:0/2621240006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde680d07c0 0x7fde68151a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:31.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.271+0000 7fde67400640 1 -- 172.21.15.19:0/2621240006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde68155930 msgr2=0x7fde680b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:31.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.271+0000 7fde67400640 1 --2- 172.21.15.19:0/2621240006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde68155930 0x7fde680b0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:31.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.271+0000 7fde67400640 1 -- 172.21.15.19:0/2621240006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde68152280 con 0x7fde68155550 2024-03-28T13:14:31.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.272+0000 7fde67400640 1 --2- 172.21.15.19:0/2621240006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde68155550 0x7fde681f8680 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fde5400d800 tx=0x7fde5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:31.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.272+0000 7fde5fe00640 1 -- 172.21.15.19:0/2621240006 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5400b6c0 con 0x7fde68155550 2024-03-28T13:14:31.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.272+0000 7fde5fe00640 1 -- 172.21.15.19:0/2621240006 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde54019070 con 0x7fde68155550 2024-03-28T13:14:31.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.272+0000 7fde5fe00640 1 -- 172.21.15.19:0/2621240006 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5400ba80 con 0x7fde68155550 2024-03-28T13:14:31.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.272+0000 7fde6ee00640 1 -- 172.21.15.19:0/2621240006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde68152560 con 0x7fde68155550 2024-03-28T13:14:31.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.272+0000 7fde6ee00640 1 -- 172.21.15.19:0/2621240006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde68152b20 con 0x7fde68155550 2024-03-28T13:14:31.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.273+0000 7fde6ee00640 1 -- 172.21.15.19:0/2621240006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde68144db0 con 0x7fde68155550 2024-03-28T13:14:31.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.275+0000 7fde5fe00640 1 -- 172.21.15.19:0/2621240006 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fde5400bc20 con 0x7fde68155550 2024-03-28T13:14:31.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.275+0000 7fde5fe00640 1 --2- 172.21.15.19:0/2621240006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fde4806c530 0x7fde4806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:31.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.275+0000 7fde5fe00640 1 -- 172.21.15.19:0/2621240006 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5609+0+0 (secure 0 0 0) 0x7fde5408bb20 con 0x7fde68155550 2024-03-28T13:14:31.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.276+0000 7fde67e00640 1 --2- 172.21.15.19:0/2621240006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fde4806c530 0x7fde4806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:31.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.277+0000 7fde67e00640 1 --2- 172.21.15.19:0/2621240006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fde4806c530 0x7fde4806e9f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fde58003d90 tx=0x7fde58009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:31.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.281+0000 7fde5fe00640 1 -- 172.21.15.19:0/2621240006 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fde68144db0 con 0x7fde68155550 2024-03-28T13:14:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:31.523+0000 7fde6ee00640 1 -- 172.21.15.19:0/2621240006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fde6815a830 con 0x7fde68155550 2024-03-28T13:14:32.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.036+0000 7fde5fe00640 1 -- 172.21.15.19:0/2621240006 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v41) ==== 174+0+0 (secure 0 0 0) 0x7fde6815a830 con 0x7fde68155550 2024-03-28T13:14:32.037 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 10 and data pool 11 2024-03-28T13:14:32.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.041+0000 7fde6ee00640 1 -- 172.21.15.19:0/2621240006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fde4806c530 msgr2=0x7fde4806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:32.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.041+0000 7fde6ee00640 1 --2- 172.21.15.19:0/2621240006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fde4806c530 0x7fde4806e9f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fde58003d90 tx=0x7fde58009040 comp rx=0 tx=0).stop 2024-03-28T13:14:32.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.041+0000 7fde6ee00640 1 -- 172.21.15.19:0/2621240006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde68155550 msgr2=0x7fde681f8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:32.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.041+0000 7fde6ee00640 1 --2- 172.21.15.19:0/2621240006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde68155550 0x7fde681f8680 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fde5400d800 tx=0x7fde5400dcc0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.041+0000 7fde6ee00640 1 -- 172.21.15.19:0/2621240006 shutdown_connections 2024-03-28T13:14:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.041+0000 7fde6ee00640 1 --2- 172.21.15.19:0/2621240006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fde4806c530 0x7fde4806e9f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.041+0000 7fde6ee00640 1 --2- 172.21.15.19:0/2621240006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde68155930 0x7fde680b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.041+0000 7fde6ee00640 1 --2- 172.21.15.19:0/2621240006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde68155550 0x7fde681f8680 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.041+0000 7fde6ee00640 1 --2- 172.21.15.19:0/2621240006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde680d07c0 0x7fde68151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.041+0000 7fde6ee00640 1 -- 172.21.15.19:0/2621240006 >> 172.21.15.19:0/2621240006 conn(0x7fde6806b750 msgr2=0x7fde6815cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.042+0000 7fde6ee00640 1 -- 172.21.15.19:0/2621240006 shutdown_connections 2024-03-28T13:14:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.042+0000 7fde6ee00640 1 -- 172.21.15.19:0/2621240006 wait complete. 2024-03-28T13:14:32.094 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:14:32.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.267+0000 7fc376000640 1 Processor -- start 2024-03-28T13:14:32.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.267+0000 7fc376000640 1 -- start start 2024-03-28T13:14:32.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.269+0000 7fc376000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc37015c390 0x7fc370073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:32.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.269+0000 7fc376000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3700d0650 0x7fc3700d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:32.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.269+0000 7fc376000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc37015fe80 con 0x7fc3700af830 2024-03-28T13:14:32.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.269+0000 7fc376000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc37015c770 con 0x7fc37015c390 2024-03-28T13:14:32.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.269+0000 7fc376000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc3700ad120 con 0x7fc3700d0650 2024-03-28T13:14:32.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.269+0000 7fc367e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc3700af830 0x7fc370160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58612/0 (socket says 172.21.15.19:58612) 2024-03-28T13:14:32.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.269+0000 7fc367e00640 1 -- 172.21.15.19:0/2366098971 learned_addr learned my addr 172.21.15.19:0/2366098971 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:32.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.269+0000 7fc36fe00640 1 --2- 172.21.15.19:0/2366098971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3700d0650 0x7fc3700d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:32.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.269+0000 7fc36ea00640 1 -- 172.21.15.19:0/2366098971 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1863295431 0 0) 0x7fc37015fe80 con 0x7fc3700af830 2024-03-28T13:14:32.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36ea00640 1 -- 172.21.15.19:0/2366098971 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc354003520 con 0x7fc3700af830 2024-03-28T13:14:32.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36f400640 1 --2- 172.21.15.19:0/2366098971 >> v2:172.21.15.188:3300/0 conn(0x7fc37015c390 0x7fc370073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:32.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36ea00640 1 -- 172.21.15.19:0/2366098971 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4042250143 0 0) 0x7fc354003520 con 0x7fc3700af830 2024-03-28T13:14:32.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36ea00640 1 -- 172.21.15.19:0/2366098971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3700d0650 msgr2=0x7fc3700d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:32.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36ea00640 1 --2- 172.21.15.19:0/2366098971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3700d0650 0x7fc3700d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36ea00640 1 -- 172.21.15.19:0/2366098971 >> v2:172.21.15.188:3300/0 conn(0x7fc37015c390 msgr2=0x7fc370073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36ea00640 1 --2- 172.21.15.19:0/2366098971 >> v2:172.21.15.188:3300/0 conn(0x7fc37015c390 0x7fc370073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36ea00640 1 -- 172.21.15.19:0/2366098971 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3700ad400 con 0x7fc3700af830 2024-03-28T13:14:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36ea00640 1 -- 172.21.15.19:0/2366098971 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fc358001670 con 0x7fc3700af830 2024-03-28T13:14:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36ea00640 1 -- 172.21.15.19:0/2366098971 >> v1:172.21.15.19:6789/0 conn(0x7fc3700af830 legacy=0x7fc370160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36ea00640 1 --2- 172.21.15.19:0/2366098971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3540039b0 0x7fc354003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36ea00640 1 --2- 172.21.15.19:0/2366098971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc354004800 0x7fc354004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36ea00640 1 --2- 172.21.15.19:0/2366098971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3700d0650 0x7fc3540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36ea00640 1 -- 172.21.15.19:0/2366098971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3700ad120 con 0x7fc354004800 2024-03-28T13:14:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36ea00640 1 -- 172.21.15.19:0/2366098971 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc35400b9c0 con 0x7fc3540039b0 2024-03-28T13:14:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36ea00640 1 -- 172.21.15.19:0/2366098971 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc35400bb90 con 0x7fc3700d0650 2024-03-28T13:14:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc36f400640 1 --2- 172.21.15.19:0/2366098971 >> v2:172.21.15.188:3300/0 conn(0x7fc37015c390 0x7fc370073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:14:32.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.270+0000 7fc367e00640 1 --2- 172.21.15.19:0/2366098971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc354004800 0x7fc354004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:32.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.271+0000 7fc367e00640 1 -- 172.21.15.19:0/2366098971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3700d0650 msgr2=0x7fc3540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:32.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.271+0000 7fc36fe00640 1 --2- 172.21.15.19:0/2366098971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3700d0650 0x7fc3540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:32.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.271+0000 7fc367e00640 1 --2- 172.21.15.19:0/2366098971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3700d0650 0x7fc3540051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.271+0000 7fc367e00640 1 -- 172.21.15.19:0/2366098971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3540039b0 msgr2=0x7fc354003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:32.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.271+0000 7fc367e00640 1 --2- 172.21.15.19:0/2366098971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3540039b0 0x7fc354003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.271+0000 7fc367e00640 1 -- 172.21.15.19:0/2366098971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc354003520 con 0x7fc354004800 2024-03-28T13:14:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.272+0000 7fc367e00640 1 --2- 172.21.15.19:0/2366098971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc354004800 0x7fc354004c80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc35800c160 tx=0x7fc354005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b581d73b68ae7112 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.272+0000 7fc36ea00640 1 -- 172.21.15.19:0/2366098971 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358007c80 con 0x7fc354004800 2024-03-28T13:14:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.272+0000 7fc36ea00640 1 -- 172.21.15.19:0/2366098971 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc358030070 con 0x7fc354004800 2024-03-28T13:14:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.272+0000 7fc36ea00640 1 -- 172.21.15.19:0/2366098971 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358005d30 con 0x7fc354004800 2024-03-28T13:14:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.273+0000 7fc376000640 1 -- 172.21.15.19:0/2366098971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc354004800 msgr2=0x7fc354004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:32.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.273+0000 7fc376000640 1 --2- 172.21.15.19:0/2366098971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc354004800 0x7fc354004c80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fc35800c160 tx=0x7fc354005830 comp rx=0 tx=0).stop 2024-03-28T13:14:32.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.273+0000 7fc36fe00640 1 -- 172.21.15.19:0/2366098971 reap_dead start 2024-03-28T13:14:32.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.273+0000 7fc376000640 1 -- 172.21.15.19:0/2366098971 shutdown_connections 2024-03-28T13:14:32.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.273+0000 7fc376000640 1 -- 172.21.15.19:0/2366098971 >> 172.21.15.19:0/2366098971 conn(0x7fc37006b990 msgr2=0x7fc37006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:32.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.273+0000 7fc376000640 1 -- 172.21.15.19:0/2366098971 shutdown_connections 2024-03-28T13:14:32.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.273+0000 7fc376000640 1 -- 172.21.15.19:0/2366098971 wait complete. 2024-03-28T13:14:32.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.274+0000 7fc376000640 1 Processor -- start 2024-03-28T13:14:32.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.275+0000 7fc376000640 1 -- start start 2024-03-28T13:14:32.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.275+0000 7fc376000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc354004800 0x7fc3701487a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:32.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.275+0000 7fc376000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3701599e0 0x7fc37015d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:32.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.275+0000 7fc376000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc370153870 0x7fc37006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:32.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.276+0000 7fc376000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc370160d50 con 0x7fc354004800 2024-03-28T13:14:32.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.276+0000 7fc376000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc370160120 con 0x7fc370153870 2024-03-28T13:14:32.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.276+0000 7fc376000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc3700cf980 con 0x7fc3701599e0 2024-03-28T13:14:32.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.276+0000 7fc36f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc354004800 0x7fc3701487a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:32.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.276+0000 7fc36f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc354004800 0x7fc3701487a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40754/0 (socket says 172.21.15.19:40754) 2024-03-28T13:14:32.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.276+0000 7fc36f400640 1 -- 172.21.15.19:0/498082494 learned_addr learned my addr 172.21.15.19:0/498082494 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:32.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.276+0000 7fc36f400640 1 -- 172.21.15.19:0/498082494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3701599e0 msgr2=0x7fc37015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:32.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.276+0000 7fc36f400640 1 --2- 172.21.15.19:0/498082494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3701599e0 0x7fc37015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.276+0000 7fc36f400640 1 -- 172.21.15.19:0/498082494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc370153870 msgr2=0x7fc37006d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:32.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.276+0000 7fc36f400640 1 --2- 172.21.15.19:0/498082494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc370153870 0x7fc37006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.276+0000 7fc36f400640 1 -- 172.21.15.19:0/498082494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37015dbc0 con 0x7fc354004800 2024-03-28T13:14:32.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.276+0000 7fc36f400640 1 --2- 172.21.15.19:0/498082494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc354004800 0x7fc3701487a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc36000e7e0 tx=0x7fc36000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:32.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.277+0000 7fc367400640 1 -- 172.21.15.19:0/498082494 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36000f040 con 0x7fc354004800 2024-03-28T13:14:32.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.277+0000 7fc367400640 1 -- 172.21.15.19:0/498082494 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc360004030 con 0x7fc354004800 2024-03-28T13:14:32.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.277+0000 7fc367400640 1 -- 172.21.15.19:0/498082494 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360004420 con 0x7fc354004800 2024-03-28T13:14:32.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.277+0000 7fc376000640 1 -- 172.21.15.19:0/498082494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc370151360 con 0x7fc354004800 2024-03-28T13:14:32.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.277+0000 7fc376000640 1 -- 172.21.15.19:0/498082494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc370151920 con 0x7fc354004800 2024-03-28T13:14:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.278+0000 7fc376000640 1 -- 172.21.15.19:0/498082494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3700685c0 con 0x7fc354004800 2024-03-28T13:14:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.279+0000 7fc367400640 1 -- 172.21.15.19:0/498082494 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc36000c650 con 0x7fc354004800 2024-03-28T13:14:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.280+0000 7fc367400640 1 --2- 172.21.15.19:0/498082494 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc34c06c530 0x7fc34c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:32.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.280+0000 7fc367400640 1 -- 172.21.15.19:0/498082494 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7fc36008b770 con 0x7fc354004800 2024-03-28T13:14:32.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.280+0000 7fc367e00640 1 --2- 172.21.15.19:0/498082494 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc34c06c530 0x7fc34c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:32.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.281+0000 7fc367e00640 1 --2- 172.21.15.19:0/498082494 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc34c06c530 0x7fc34c06e9f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc358034000 tx=0x7fc358039040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:32.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.284+0000 7fc367400640 1 -- 172.21.15.19:0/498082494 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc360017020 con 0x7fc354004800 2024-03-28T13:14:32.451 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:14:32.459 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:14:32.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.484+0000 7fc376000640 1 -- 172.21.15.19:0/498082494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc37006f1b0 con 0x7fc354004800 2024-03-28T13:14:32.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.486+0000 7fc367400640 1 -- 172.21.15.19:0/498082494 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+16621 (secure 0 0 0) 0x7fc360059d80 con 0x7fc354004800 2024-03-28T13:14:32.486 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:32.486 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":76,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:14:31.525443+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:14:29.424080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-28T13:14:30.494667+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:14:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.489+0000 7fc376000640 1 -- 172.21.15.19:0/498082494 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc34c06c530 msgr2=0x7fc34c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.489+0000 7fc376000640 1 --2- 172.21.15.19:0/498082494 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc34c06c530 0x7fc34c06e9f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc358034000 tx=0x7fc358039040 comp rx=0 tx=0).stop 2024-03-28T13:14:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.489+0000 7fc376000640 1 -- 172.21.15.19:0/498082494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc354004800 msgr2=0x7fc3701487a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.489+0000 7fc376000640 1 --2- 172.21.15.19:0/498082494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc354004800 0x7fc3701487a0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fc36000e7e0 tx=0x7fc36000eca0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.489+0000 7fc376000640 1 -- 172.21.15.19:0/498082494 shutdown_connections 2024-03-28T13:14:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.489+0000 7fc376000640 1 --2- 172.21.15.19:0/498082494 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc34c06c530 0x7fc34c06e9f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.489+0000 7fc376000640 1 --2- 172.21.15.19:0/498082494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc370153870 0x7fc37006d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.489+0000 7fc376000640 1 --2- 172.21.15.19:0/498082494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3701599e0 0x7fc37015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.489+0000 7fc376000640 1 --2- 172.21.15.19:0/498082494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc354004800 0x7fc3701487a0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.489+0000 7fc376000640 1 -- 172.21.15.19:0/498082494 >> 172.21.15.19:0/498082494 conn(0x7fc37006b990 msgr2=0x7fc370156400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.489+0000 7fc376000640 1 -- 172.21.15.19:0/498082494 shutdown_connections 2024-03-28T13:14:32.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.489+0000 7fc376000640 1 -- 172.21.15.19:0/498082494 wait complete. 2024-03-28T13:14:32.507 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:14:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.743+0000 7fce17a00640 1 Processor -- start 2024-03-28T13:14:32.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.743+0000 7fce17a00640 1 -- start start 2024-03-28T13:14:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.744+0000 7fce17a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fce100af830 0x7fce10073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.744+0000 7fce17a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fce100d0650 0x7fce100d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.744+0000 7fce17a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce1015fe80 con 0x7fce1015c390 2024-03-28T13:14:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.744+0000 7fce17a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fce1015c770 con 0x7fce100af830 2024-03-28T13:14:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.744+0000 7fce17a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fce100d0fc0 con 0x7fce100d0650 2024-03-28T13:14:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.745+0000 7fce0fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fce100af830 0x7fce10073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.745+0000 7fce0fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fce100af830 0x7fce10073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40766/0 (socket says 172.21.15.19:40766) 2024-03-28T13:14:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.745+0000 7fce0fe00640 1 -- 172.21.15.19:0/1977961984 learned_addr learned my addr 172.21.15.19:0/1977961984 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.745+0000 7fce15a00640 1 --2- 172.21.15.19:0/1977961984 >> v2:172.21.15.188:3300/0 conn(0x7fce100d0650 0x7fce100d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.745+0000 7fce0fe00640 1 -- 172.21.15.19:0/1977961984 >> v2:172.21.15.188:3300/0 conn(0x7fce100d0650 msgr2=0x7fce100d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.745+0000 7fce0fe00640 1 --2- 172.21.15.19:0/1977961984 >> v2:172.21.15.188:3300/0 conn(0x7fce100d0650 0x7fce100d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.745+0000 7fce0fe00640 1 -- 172.21.15.19:0/1977961984 >> v1:172.21.15.188:6789/0 conn(0x7fce1015c390 legacy=0x7fce10160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.745+0000 7fce0fe00640 1 -- 172.21.15.19:0/1977961984 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce100ad120 con 0x7fce100af830 2024-03-28T13:14:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.747+0000 7fce0fe00640 1 --2- 172.21.15.19:0/1977961984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce100af830 0x7fce10073790 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fce1006eaa0 tx=0x7fcdf802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f99cb18f5643af2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:32.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.747+0000 7fce0f400640 1 -- 172.21.15.19:0/1977961984 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf802ca70 con 0x7fce100af830 2024-03-28T13:14:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.747+0000 7fce0f400640 1 -- 172.21.15.19:0/1977961984 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf802cc10 con 0x7fce100af830 2024-03-28T13:14:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.747+0000 7fce0f400640 1 -- 172.21.15.19:0/1977961984 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80343f0 con 0x7fce100af830 2024-03-28T13:14:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.748+0000 7fce17a00640 1 -- 172.21.15.19:0/1977961984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce100af830 msgr2=0x7fce10073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.748+0000 7fce17a00640 1 --2- 172.21.15.19:0/1977961984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce100af830 0x7fce10073790 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fce1006eaa0 tx=0x7fcdf802bed0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.748+0000 7fce17a00640 1 -- 172.21.15.19:0/1977961984 shutdown_connections 2024-03-28T13:14:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.748+0000 7fce17a00640 1 --2- 172.21.15.19:0/1977961984 >> v2:172.21.15.188:3300/0 conn(0x7fce100d0650 0x7fce100d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.748+0000 7fce17a00640 1 --2- 172.21.15.19:0/1977961984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce100af830 0x7fce10073790 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.748+0000 7fce17a00640 1 -- 172.21.15.19:0/1977961984 >> 172.21.15.19:0/1977961984 conn(0x7fce1006b990 msgr2=0x7fce1006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.748+0000 7fce17a00640 1 -- 172.21.15.19:0/1977961984 shutdown_connections 2024-03-28T13:14:32.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.748+0000 7fce17a00640 1 -- 172.21.15.19:0/1977961984 wait complete. 2024-03-28T13:14:32.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.749+0000 7fce17a00640 1 Processor -- start 2024-03-28T13:14:32.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.750+0000 7fce17a00640 1 -- start start 2024-03-28T13:14:32.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.750+0000 7fce17a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce100af830 0x7fce100afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:32.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.750+0000 7fce17a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce100d0650 0x7fce100b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:32.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.750+0000 7fce17a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce1015c390 0x7fce101f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:32.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.750+0000 7fce17a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce10160120 con 0x7fce100d0650 2024-03-28T13:14:32.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.750+0000 7fce17a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fce100ad6e0 con 0x7fce1015c390 2024-03-28T13:14:32.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.750+0000 7fce17a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fce10160d50 con 0x7fce100af830 2024-03-28T13:14:32.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.751+0000 7fce0fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce100d0650 0x7fce100b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:32.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.751+0000 7fce0fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce100d0650 0x7fce100b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40778/0 (socket says 172.21.15.19:40778) 2024-03-28T13:14:32.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.751+0000 7fce0fe00640 1 -- 172.21.15.19:0/994783004 learned_addr learned my addr 172.21.15.19:0/994783004 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:32.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.751+0000 7fce0fe00640 1 -- 172.21.15.19:0/994783004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce100af830 msgr2=0x7fce100afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:32.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.751+0000 7fce0fe00640 1 --2- 172.21.15.19:0/994783004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce100af830 0x7fce100afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.751+0000 7fce0fe00640 1 -- 172.21.15.19:0/994783004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce1015c390 msgr2=0x7fce101f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:32.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.751+0000 7fce0fe00640 1 --2- 172.21.15.19:0/994783004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce1015c390 0x7fce101f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.751+0000 7fce0fe00640 1 -- 172.21.15.19:0/994783004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce100b38f0 con 0x7fce100d0650 2024-03-28T13:14:32.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.751+0000 7fce0fe00640 1 --2- 172.21.15.19:0/994783004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce100d0650 0x7fce100b4370 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009e50 tx=0x7fcdf8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:32.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.752+0000 7fce0cc00640 1 -- 172.21.15.19:0/994783004 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf8043020 con 0x7fce100d0650 2024-03-28T13:14:32.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.752+0000 7fce0cc00640 1 -- 172.21.15.19:0/994783004 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf80094d0 con 0x7fce100d0650 2024-03-28T13:14:32.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.752+0000 7fce0cc00640 1 -- 172.21.15.19:0/994783004 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf803e670 con 0x7fce100d0650 2024-03-28T13:14:32.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.752+0000 7fce17a00640 1 -- 172.21.15.19:0/994783004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce101f5350 con 0x7fce100d0650 2024-03-28T13:14:32.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.752+0000 7fce17a00640 1 -- 172.21.15.19:0/994783004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce100b3b70 con 0x7fce100d0650 2024-03-28T13:14:32.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.753+0000 7fce17a00640 1 -- 172.21.15.19:0/994783004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce100685c0 con 0x7fce100d0650 2024-03-28T13:14:32.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.754+0000 7fce0cc00640 1 -- 172.21.15.19:0/994783004 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcdf8046020 con 0x7fce100d0650 2024-03-28T13:14:32.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.755+0000 7fce0cc00640 1 --2- 172.21.15.19:0/994783004 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcdec06c530 0x7fcdec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:32.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.755+0000 7fce0cc00640 1 -- 172.21.15.19:0/994783004 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5709+0+0 (secure 0 0 0) 0x7fcdf80adf60 con 0x7fce100d0650 2024-03-28T13:14:32.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.755+0000 7fce15000640 1 --2- 172.21.15.19:0/994783004 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcdec06c530 0x7fcdec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:32.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.756+0000 7fce15000640 1 --2- 172.21.15.19:0/994783004 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcdec06c530 0x7fcdec06e9f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fce00001560 tx=0x7fce00007990 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:32.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.762+0000 7fce0cc00640 1 -- 172.21.15.19:0/994783004 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcdf804d030 con 0x7fce100d0650 2024-03-28T13:14:32.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.959+0000 7fce17a00640 1 -- 172.21.15.19:0/994783004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce1006e350 con 0x7fce100d0650 2024-03-28T13:14:32.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.962+0000 7fce0cc00640 1 -- 172.21.15.19:0/994783004 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+16621 (secure 0 0 0) 0x7fce100b38f0 con 0x7fce100d0650 2024-03-28T13:14:32.963 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:32.963 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":76,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:14:31.525443+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:14:29.424080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-28T13:14:30.494667+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:14:32.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.964+0000 7fce17a00640 1 -- 172.21.15.19:0/994783004 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcdec06c530 msgr2=0x7fcdec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:32.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.964+0000 7fce17a00640 1 --2- 172.21.15.19:0/994783004 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcdec06c530 0x7fcdec06e9f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fce00001560 tx=0x7fce00007990 comp rx=0 tx=0).stop 2024-03-28T13:14:32.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.965+0000 7fce17a00640 1 -- 172.21.15.19:0/994783004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce100d0650 msgr2=0x7fce100b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:32.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.965+0000 7fce17a00640 1 --2- 172.21.15.19:0/994783004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce100d0650 0x7fce100b4370 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009e50 tx=0x7fcdf8004a40 comp rx=0 tx=0).stop 2024-03-28T13:14:32.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.965+0000 7fce17a00640 1 -- 172.21.15.19:0/994783004 shutdown_connections 2024-03-28T13:14:32.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.965+0000 7fce17a00640 1 --2- 172.21.15.19:0/994783004 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcdec06c530 0x7fcdec06e9f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.965+0000 7fce17a00640 1 --2- 172.21.15.19:0/994783004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce1015c390 0x7fce101f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.965+0000 7fce17a00640 1 --2- 172.21.15.19:0/994783004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce100d0650 0x7fce100b4370 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.965+0000 7fce17a00640 1 --2- 172.21.15.19:0/994783004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce100af830 0x7fce100afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:32.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.965+0000 7fce17a00640 1 -- 172.21.15.19:0/994783004 >> 172.21.15.19:0/994783004 conn(0x7fce1006b990 msgr2=0x7fce10155560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:32.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.965+0000 7fce17a00640 1 -- 172.21.15.19:0/994783004 shutdown_connections 2024-03-28T13:14:32.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:32.965+0000 7fce17a00640 1 -- 172.21.15.19:0/994783004 wait complete. 2024-03-28T13:14:32.983 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-28T13:14:33.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.208+0000 7f04fe200640 1 Processor -- start 2024-03-28T13:14:33.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.208+0000 7f04fe200640 1 -- start start 2024-03-28T13:14:33.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.210+0000 7f04fe200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f04f813f190 0x7f04f80cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:33.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.210+0000 7f04fe200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f04f806dd30 0x7f04f80cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:33.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.210+0000 7f04fe200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04f80ce890 con 0x7f04f80cc880 2024-03-28T13:14:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.210+0000 7f04fe200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f04f80cd230 con 0x7f04f806dd30 2024-03-28T13:14:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.210+0000 7f04fe200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f04f80ceb50 con 0x7f04f813f190 2024-03-28T13:14:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.210+0000 7f04fd800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f04f806dd30 0x7f04f80cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.210+0000 7f04f7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f04f80cc880 0x7f04f806d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58622/0 (socket says 172.21.15.19:58622) 2024-03-28T13:14:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.210+0000 7f04f7e00640 1 -- 172.21.15.19:0/1619660677 learned_addr learned my addr 172.21.15.19:0/1619660677 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.210+0000 7f04fce00640 1 --2- 172.21.15.19:0/1619660677 >> v2:172.21.15.188:3300/0 conn(0x7f04f813f190 0x7f04f80cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.211+0000 7f04fd800640 1 -- 172.21.15.19:0/1619660677 >> v2:172.21.15.188:3300/0 conn(0x7f04f813f190 msgr2=0x7f04f80cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.211+0000 7f04fd800640 1 --2- 172.21.15.19:0/1619660677 >> v2:172.21.15.188:3300/0 conn(0x7f04f813f190 0x7f04f80cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.211+0000 7f04fd800640 1 -- 172.21.15.19:0/1619660677 >> v1:172.21.15.19:6789/0 conn(0x7f04f80cc880 legacy=0x7f04f806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:33.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.211+0000 7f04fd800640 1 -- 172.21.15.19:0/1619660677 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f80cee30 con 0x7f04f806dd30 2024-03-28T13:14:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.212+0000 7f04fd800640 1 --2- 172.21.15.19:0/1619660677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04f806dd30 0x7f04f80cccf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f04f806f660 tx=0x7f04f002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a7787ff3140898 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.212+0000 7f04f7400640 1 -- 172.21.15.19:0/1619660677 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0039070 con 0x7f04f806dd30 2024-03-28T13:14:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.212+0000 7f04f7400640 1 -- 172.21.15.19:0/1619660677 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f002cc70 con 0x7f04f806dd30 2024-03-28T13:14:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.212+0000 7f04f7400640 1 -- 172.21.15.19:0/1619660677 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f0034660 con 0x7f04f806dd30 2024-03-28T13:14:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.213+0000 7f04fe200640 1 -- 172.21.15.19:0/1619660677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04f806dd30 msgr2=0x7f04f80cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.213+0000 7f04fe200640 1 --2- 172.21.15.19:0/1619660677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04f806dd30 0x7f04f80cccf0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f04f806f660 tx=0x7f04f002bec0 comp rx=0 tx=0).stop 2024-03-28T13:14:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.213+0000 7f04fe200640 1 -- 172.21.15.19:0/1619660677 shutdown_connections 2024-03-28T13:14:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.213+0000 7f04fe200640 1 --2- 172.21.15.19:0/1619660677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04f806dd30 0x7f04f80cccf0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.213+0000 7f04fe200640 1 --2- 172.21.15.19:0/1619660677 >> v2:172.21.15.188:3300/0 conn(0x7f04f813f190 0x7f04f80cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.213+0000 7f04fe200640 1 -- 172.21.15.19:0/1619660677 >> 172.21.15.19:0/1619660677 conn(0x7f04f80b1ba0 msgr2=0x7f04f80b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:33.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.213+0000 7f04fe200640 1 -- 172.21.15.19:0/1619660677 shutdown_connections 2024-03-28T13:14:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.213+0000 7f04fe200640 1 -- 172.21.15.19:0/1619660677 wait complete. 2024-03-28T13:14:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.215+0000 7f04fe200640 1 Processor -- start 2024-03-28T13:14:33.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.215+0000 7f04fe200640 1 -- start start 2024-03-28T13:14:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.216+0000 7f04fe200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04f806dd30 0x7f04f81559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.216+0000 7f04fe200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04f80cc880 0x7f04f81e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:33.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.216+0000 7f04fe200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04f813f190 0x7f04f815af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.216+0000 7f04fe200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04f813f570 con 0x7f04f806dd30 2024-03-28T13:14:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.216+0000 7f04fe200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04f8160c70 con 0x7f04f813f190 2024-03-28T13:14:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.216+0000 7f04fe200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f04f80cfb70 con 0x7f04f80cc880 2024-03-28T13:14:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.216+0000 7f04fce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04f806dd30 0x7f04f81559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.216+0000 7f04fce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04f806dd30 0x7f04f81559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40786/0 (socket says 172.21.15.19:40786) 2024-03-28T13:14:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.216+0000 7f04fce00640 1 -- 172.21.15.19:0/242099009 learned_addr learned my addr 172.21.15.19:0/242099009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.217+0000 7f04fd800640 1 --2- 172.21.15.19:0/242099009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04f813f190 0x7f04f815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.217+0000 7f04f7e00640 1 --2- 172.21.15.19:0/242099009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04f80cc880 0x7f04f81e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.217+0000 7f04fce00640 1 -- 172.21.15.19:0/242099009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04f80cc880 msgr2=0x7f04f81e7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:33.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.217+0000 7f04fce00640 1 --2- 172.21.15.19:0/242099009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04f80cc880 0x7f04f81e7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.217+0000 7f04fce00640 1 -- 172.21.15.19:0/242099009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04f813f190 msgr2=0x7f04f815af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.217+0000 7f04fce00640 1 --2- 172.21.15.19:0/242099009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04f813f190 0x7f04f815af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.217+0000 7f04fce00640 1 -- 172.21.15.19:0/242099009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f81e6070 con 0x7f04f806dd30 2024-03-28T13:14:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.217+0000 7f04fd800640 1 --2- 172.21.15.19:0/242099009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04f813f190 0x7f04f815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:33.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.217+0000 7f04fce00640 1 --2- 172.21.15.19:0/242099009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04f806dd30 0x7f04f81559f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00d830 tx=0x7f04ec00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.217+0000 7f04f4c00640 1 -- 172.21.15.19:0/242099009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec00b6c0 con 0x7f04f806dd30 2024-03-28T13:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.217+0000 7f04f4c00640 1 -- 172.21.15.19:0/242099009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ec019070 con 0x7f04f806dd30 2024-03-28T13:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.217+0000 7f04f4c00640 1 -- 172.21.15.19:0/242099009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec00ba80 con 0x7f04f806dd30 2024-03-28T13:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.217+0000 7f04fe200640 1 -- 172.21.15.19:0/242099009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f81e6350 con 0x7f04f806dd30 2024-03-28T13:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.218+0000 7f04fe200640 1 -- 172.21.15.19:0/242099009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f81e6910 con 0x7f04f806dd30 2024-03-28T13:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.219+0000 7f04fe200640 1 -- 172.21.15.19:0/242099009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04f8075940 con 0x7f04f806dd30 2024-03-28T13:14:33.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.220+0000 7f04f4c00640 1 -- 172.21.15.19:0/242099009 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f04ec01e460 con 0x7f04f806dd30 2024-03-28T13:14:33.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.221+0000 7f04f4c00640 1 --2- 172.21.15.19:0/242099009 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f04d806c530 0x7f04d806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:33.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.221+0000 7f04f4c00640 1 -- 172.21.15.19:0/242099009 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f04ec08bd90 con 0x7f04f806dd30 2024-03-28T13:14:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.222+0000 7f04f7e00640 1 --2- 172.21.15.19:0/242099009 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f04d806c530 0x7f04d806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:33.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.223+0000 7f04f7e00640 1 --2- 172.21.15.19:0/242099009 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f04d806c530 0x7f04d806e9f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f04f81f77e0 tx=0x7f04e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:33.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.225+0000 7f04f4c00640 1 -- 172.21.15.19:0/242099009 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f04ec05a3b0 con 0x7f04f806dd30 2024-03-28T13:14:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:33.468+0000 7f04fe200640 1 -- 172.21.15.19:0/242099009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f04f806d7c0 con 0x7f04f806dd30 2024-03-28T13:14:34.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.077+0000 7f04f4c00640 1 -- 172.21.15.19:0/242099009 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v44) ==== 118+0+0 (secure 0 0 0) 0x7f04ec059cb0 con 0x7f04f806dd30 2024-03-28T13:14:34.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.080+0000 7f04fe200640 1 -- 172.21.15.19:0/242099009 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f04d806c530 msgr2=0x7f04d806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:34.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.080+0000 7f04fe200640 1 --2- 172.21.15.19:0/242099009 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f04d806c530 0x7f04d806e9f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f04f81f77e0 tx=0x7f04e4009040 comp rx=0 tx=0).stop 2024-03-28T13:14:34.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.080+0000 7f04fe200640 1 -- 172.21.15.19:0/242099009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04f806dd30 msgr2=0x7f04f81559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:34.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.081+0000 7f04fe200640 1 --2- 172.21.15.19:0/242099009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04f806dd30 0x7f04f81559f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f04ec00d830 tx=0x7f04ec00dcf0 comp rx=0 tx=0).stop 2024-03-28T13:14:34.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.081+0000 7f04fe200640 1 -- 172.21.15.19:0/242099009 shutdown_connections 2024-03-28T13:14:34.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.081+0000 7f04fe200640 1 --2- 172.21.15.19:0/242099009 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f04d806c530 0x7f04d806e9f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:34.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.081+0000 7f04fe200640 1 --2- 172.21.15.19:0/242099009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04f813f190 0x7f04f815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:34.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.081+0000 7f04fe200640 1 --2- 172.21.15.19:0/242099009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04f80cc880 0x7f04f81e7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:34.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.081+0000 7f04fe200640 1 --2- 172.21.15.19:0/242099009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04f806dd30 0x7f04f81559f0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:34.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.081+0000 7f04fe200640 1 -- 172.21.15.19:0/242099009 >> 172.21.15.19:0/242099009 conn(0x7f04f80b1ba0 msgr2=0x7f04f80af760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:34.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.081+0000 7f04fe200640 1 -- 172.21.15.19:0/242099009 shutdown_connections 2024-03-28T13:14:34.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.081+0000 7f04fe200640 1 -- 172.21.15.19:0/242099009 wait complete. 2024-03-28T13:14:34.102 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-28T13:14:34.102 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-28T13:14:34.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.312+0000 7f3c01800640 1 Processor -- start 2024-03-28T13:14:34.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.313+0000 7f3c01800640 1 -- start start 2024-03-28T13:14:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.314+0000 7f3c01800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3bfc15e4c0 0x7f3bfc0b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.314+0000 7f3c01800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3bfc0d37b0 0x7f3bfc0b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.314+0000 7f3c01800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bfc0d2710 con 0x7f3bfc0b67f0 2024-03-28T13:14:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.314+0000 7f3c01800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3bfc15e8a0 con 0x7f3bfc15e4c0 2024-03-28T13:14:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.314+0000 7f3c01800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3bfc0d4040 con 0x7f3bfc0d37b0 2024-03-28T13:14:34.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.314+0000 7f3bfbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3bfc15e4c0 0x7f3bfc0b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:34.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.314+0000 7f3c00e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3bfc0b67f0 0x7f3bfc0d4260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58634/0 (socket says 172.21.15.19:58634) 2024-03-28T13:14:34.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.314+0000 7f3c00e00640 1 -- 172.21.15.19:0/2921020941 learned_addr learned my addr 172.21.15.19:0/2921020941 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:34.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.315+0000 7f3bfb400640 1 --2- 172.21.15.19:0/2921020941 >> v2:172.21.15.188:3300/0 conn(0x7f3bfc0d37b0 0x7f3bfc0b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:34.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.315+0000 7f3bfbe00640 1 -- 172.21.15.19:0/2921020941 >> v2:172.21.15.188:3300/0 conn(0x7f3bfc0d37b0 msgr2=0x7f3bfc0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:34.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.315+0000 7f3bfbe00640 1 --2- 172.21.15.19:0/2921020941 >> v2:172.21.15.188:3300/0 conn(0x7f3bfc0d37b0 0x7f3bfc0b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:34.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.315+0000 7f3bfbe00640 1 -- 172.21.15.19:0/2921020941 >> v1:172.21.15.19:6789/0 conn(0x7f3bfc0b67f0 legacy=0x7f3bfc0d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:34.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.315+0000 7f3bfbe00640 1 -- 172.21.15.19:0/2921020941 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bfc0d29f0 con 0x7f3bfc15e4c0 2024-03-28T13:14:34.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.316+0000 7f3bfbe00640 1 --2- 172.21.15.19:0/2921020941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bfc15e4c0 0x7f3bfc0b9bf0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0b99b0 tx=0x7f3bec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1285134d73961d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:34.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.316+0000 7f3bfaa00640 1 -- 172.21.15.19:0/2921020941 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec035020 con 0x7f3bfc15e4c0 2024-03-28T13:14:34.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.316+0000 7f3bfaa00640 1 -- 172.21.15.19:0/2921020941 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bec031830 con 0x7f3bfc15e4c0 2024-03-28T13:14:34.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.316+0000 7f3bfaa00640 1 -- 172.21.15.19:0/2921020941 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec031c00 con 0x7f3bfc15e4c0 2024-03-28T13:14:34.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.317+0000 7f3c01800640 1 -- 172.21.15.19:0/2921020941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bfc15e4c0 msgr2=0x7f3bfc0b9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:34.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.317+0000 7f3c01800640 1 --2- 172.21.15.19:0/2921020941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bfc15e4c0 0x7f3bfc0b9bf0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc0b99b0 tx=0x7f3bec02b8d0 comp rx=0 tx=0).stop 2024-03-28T13:14:34.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.317+0000 7f3c01800640 1 -- 172.21.15.19:0/2921020941 shutdown_connections 2024-03-28T13:14:34.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.317+0000 7f3c01800640 1 --2- 172.21.15.19:0/2921020941 >> v2:172.21.15.188:3300/0 conn(0x7f3bfc0d37b0 0x7f3bfc0b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:34.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.317+0000 7f3c01800640 1 --2- 172.21.15.19:0/2921020941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bfc15e4c0 0x7f3bfc0b9bf0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:34.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.317+0000 7f3c01800640 1 -- 172.21.15.19:0/2921020941 >> 172.21.15.19:0/2921020941 conn(0x7f3bfc0676a0 msgr2=0x7f3bfc067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:34.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.317+0000 7f3c01800640 1 -- 172.21.15.19:0/2921020941 shutdown_connections 2024-03-28T13:14:34.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.317+0000 7f3c01800640 1 -- 172.21.15.19:0/2921020941 wait complete. 2024-03-28T13:14:34.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.319+0000 7f3c01800640 1 Processor -- start 2024-03-28T13:14:34.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.319+0000 7f3c01800640 1 -- start start 2024-03-28T13:14:34.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.320+0000 7f3c01800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bfc0b67f0 0x7f3bfc159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:34.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.320+0000 7f3c01800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bfc0d37b0 0x7f3bfc161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:34.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.320+0000 7f3c01800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bfc15e4c0 0x7f3bfc1540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:34.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.320+0000 7f3c01800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3bfc0b6c20 con 0x7f3bfc0b67f0 2024-03-28T13:14:34.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.320+0000 7f3c01800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3bfc0d3320 con 0x7f3bfc15e4c0 2024-03-28T13:14:34.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.320+0000 7f3c01800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3bfc0d2fb0 con 0x7f3bfc0d37b0 2024-03-28T13:14:34.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.320+0000 7f3bfbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bfc0b67f0 0x7f3bfc159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:34.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.320+0000 7f3bfbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bfc0b67f0 0x7f3bfc159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40816/0 (socket says 172.21.15.19:40816) 2024-03-28T13:14:34.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.320+0000 7f3bfbe00640 1 -- 172.21.15.19:0/1786690440 learned_addr learned my addr 172.21.15.19:0/1786690440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:34.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.321+0000 7f3c00e00640 1 --2- 172.21.15.19:0/1786690440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bfc15e4c0 0x7f3bfc1540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:34.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.321+0000 7f3bfbe00640 1 -- 172.21.15.19:0/1786690440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bfc0d37b0 msgr2=0x7f3bfc161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:34.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.321+0000 7f3bfbe00640 1 --2- 172.21.15.19:0/1786690440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bfc0d37b0 0x7f3bfc161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:34.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.321+0000 7f3bfbe00640 1 -- 172.21.15.19:0/1786690440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bfc15e4c0 msgr2=0x7f3bfc1540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:34.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.321+0000 7f3bfbe00640 1 --2- 172.21.15.19:0/1786690440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bfc15e4c0 0x7f3bfc1540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:34.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.321+0000 7f3bfbe00640 1 -- 172.21.15.19:0/1786690440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bfc15ec80 con 0x7f3bfc0b67f0 2024-03-28T13:14:34.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.321+0000 7f3bfbe00640 1 --2- 172.21.15.19:0/1786690440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bfc0b67f0 0x7f3bfc159cc0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009890 tx=0x7f3bec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:34.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.322+0000 7f3bf3e00640 1 -- 172.21.15.19:0/1786690440 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec041070 con 0x7f3bfc0b67f0 2024-03-28T13:14:34.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.322+0000 7f3bf3e00640 1 -- 172.21.15.19:0/1786690440 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bec03aa30 con 0x7f3bfc0b67f0 2024-03-28T13:14:34.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.322+0000 7f3bf3e00640 1 -- 172.21.15.19:0/1786690440 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec035040 con 0x7f3bfc0b67f0 2024-03-28T13:14:34.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.322+0000 7f3c01800640 1 -- 172.21.15.19:0/1786690440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bfc15ef60 con 0x7f3bfc0b67f0 2024-03-28T13:14:34.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.322+0000 7f3c01800640 1 -- 172.21.15.19:0/1786690440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bfc15f520 con 0x7f3bfc0b67f0 2024-03-28T13:14:34.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.323+0000 7f3c01800640 1 -- 172.21.15.19:0/1786690440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bfc075940 con 0x7f3bfc0b67f0 2024-03-28T13:14:34.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.324+0000 7f3bf3e00640 1 -- 172.21.15.19:0/1786690440 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3bec038020 con 0x7f3bfc0b67f0 2024-03-28T13:14:34.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.325+0000 7f3bf3e00640 1 --2- 172.21.15.19:0/1786690440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3bdc06c530 0x7f3bdc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:34.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.326+0000 7f3bf3e00640 1 -- 172.21.15.19:0/1786690440 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f3bec0ad1f0 con 0x7f3bfc0b67f0 2024-03-28T13:14:34.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.326+0000 7f3bfb400640 1 --2- 172.21.15.19:0/1786690440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3bdc06c530 0x7f3bdc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:34.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.327+0000 7f3bfb400640 1 --2- 172.21.15.19:0/1786690440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3bdc06c530 0x7f3bdc06e9f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc158840 tx=0x7f3be8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:34.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.330+0000 7f3bf3e00640 1 -- 172.21.15.19:0/1786690440 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3bec07b770 con 0x7f3bfc0b67f0 2024-03-28T13:14:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:34.571+0000 7f3c01800640 1 -- 172.21.15.19:0/1786690440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3bfc0b9330 con 0x7f3bfc0b67f0 2024-03-28T13:14:35.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.091+0000 7f3bf3e00640 1 -- 172.21.15.19:0/1786690440 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v45) ==== 122+0+0 (secure 0 0 0) 0x7f3bec07b070 con 0x7f3bfc0b67f0 2024-03-28T13:14:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.094+0000 7f3c01800640 1 -- 172.21.15.19:0/1786690440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3bdc06c530 msgr2=0x7f3bdc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.094+0000 7f3c01800640 1 --2- 172.21.15.19:0/1786690440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3bdc06c530 0x7f3bdc06e9f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc158840 tx=0x7f3be8008040 comp rx=0 tx=0).stop 2024-03-28T13:14:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.094+0000 7f3c01800640 1 -- 172.21.15.19:0/1786690440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bfc0b67f0 msgr2=0x7f3bfc159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.094+0000 7f3c01800640 1 --2- 172.21.15.19:0/1786690440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bfc0b67f0 0x7f3bfc159cc0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009890 tx=0x7f3bec0046c0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.094+0000 7f3c01800640 1 -- 172.21.15.19:0/1786690440 shutdown_connections 2024-03-28T13:14:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.094+0000 7f3c01800640 1 --2- 172.21.15.19:0/1786690440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3bdc06c530 0x7f3bdc06e9f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.094+0000 7f3c01800640 1 --2- 172.21.15.19:0/1786690440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bfc15e4c0 0x7f3bfc1540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.094+0000 7f3c01800640 1 --2- 172.21.15.19:0/1786690440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bfc0d37b0 0x7f3bfc161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.094+0000 7f3c01800640 1 --2- 172.21.15.19:0/1786690440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3bfc0b67f0 0x7f3bfc159cc0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.094+0000 7f3c01800640 1 -- 172.21.15.19:0/1786690440 >> 172.21.15.19:0/1786690440 conn(0x7f3bfc0676a0 msgr2=0x7f3bfc15d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:35.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.094+0000 7f3c01800640 1 -- 172.21.15.19:0/1786690440 shutdown_connections 2024-03-28T13:14:35.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.094+0000 7f3c01800640 1 -- 172.21.15.19:0/1786690440 wait complete. 2024-03-28T13:14:35.113 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-28T13:14:35.113 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:14:35.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.311+0000 7fad59800640 1 Processor -- start 2024-03-28T13:14:35.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.312+0000 7fad59800640 1 -- start start 2024-03-28T13:14:35.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.313+0000 7fad59800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fad5415eab0 0x7fad5406c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:35.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.313+0000 7fad59800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad540af490 0x7fad5406cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:35.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.313+0000 7fad59800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad540b53e0 con 0x7fad540b5990 2024-03-28T13:14:35.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.313+0000 7fad59800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fad540b56c0 con 0x7fad5415eab0 2024-03-28T13:14:35.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.313+0000 7fad59800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fad540adc80 con 0x7fad540af490 2024-03-28T13:14:35.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.314+0000 7fad52a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fad5415eab0 0x7fad5406c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:35.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.314+0000 7fad52a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fad5415eab0 0x7fad5406c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:51362/0 (socket says 172.21.15.19:51362) 2024-03-28T13:14:35.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.314+0000 7fad52a00640 1 -- 172.21.15.19:0/473669684 learned_addr learned my addr 172.21.15.19:0/473669684 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:35.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.314+0000 7fad53400640 1 --2- 172.21.15.19:0/473669684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad540af490 0x7fad5406cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.315+0000 7fad52a00640 1 -- 172.21.15.19:0/473669684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad540af490 msgr2=0x7fad5406cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.315+0000 7fad52a00640 1 --2- 172.21.15.19:0/473669684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad540af490 0x7fad5406cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.315+0000 7fad52a00640 1 -- 172.21.15.19:0/473669684 >> v1:172.21.15.188:6789/0 conn(0x7fad540b5990 legacy=0x7fad540b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:35.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.315+0000 7fad52a00640 1 -- 172.21.15.19:0/473669684 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad54072a00 con 0x7fad5415eab0 2024-03-28T13:14:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.316+0000 7fad52a00640 1 --2- 172.21.15.19:0/473669684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad5415eab0 0x7fad5406c580 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fad44009e50 tx=0x7fad4402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bf3e4a3a281cacc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.317+0000 7fad52000640 1 -- 172.21.15.19:0/473669684 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4402cb00 con 0x7fad5415eab0 2024-03-28T13:14:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.317+0000 7fad52000640 1 -- 172.21.15.19:0/473669684 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad4402cca0 con 0x7fad5415eab0 2024-03-28T13:14:35.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.317+0000 7fad52000640 1 -- 172.21.15.19:0/473669684 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44034610 con 0x7fad5415eab0 2024-03-28T13:14:35.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.317+0000 7fad59800640 1 -- 172.21.15.19:0/473669684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad5415eab0 msgr2=0x7fad5406c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:35.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.317+0000 7fad59800640 1 --2- 172.21.15.19:0/473669684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad5415eab0 0x7fad5406c580 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fad44009e50 tx=0x7fad4402bf60 comp rx=0 tx=0).stop 2024-03-28T13:14:35.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.318+0000 7fad59800640 1 -- 172.21.15.19:0/473669684 shutdown_connections 2024-03-28T13:14:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.318+0000 7fad59800640 1 --2- 172.21.15.19:0/473669684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad540af490 0x7fad5406cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.318+0000 7fad59800640 1 --2- 172.21.15.19:0/473669684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad5415eab0 0x7fad5406c580 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.318+0000 7fad59800640 1 -- 172.21.15.19:0/473669684 >> 172.21.15.19:0/473669684 conn(0x7fad54071df0 msgr2=0x7fad540721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.318+0000 7fad59800640 1 -- 172.21.15.19:0/473669684 shutdown_connections 2024-03-28T13:14:35.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.318+0000 7fad59800640 1 -- 172.21.15.19:0/473669684 wait complete. 2024-03-28T13:14:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.320+0000 7fad59800640 1 Processor -- start 2024-03-28T13:14:35.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.320+0000 7fad59800640 1 -- start start 2024-03-28T13:14:35.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.321+0000 7fad59800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad540af490 0x7fad5415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.321+0000 7fad59800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad540b5990 0x7fad54153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.321+0000 7fad59800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad5415eab0 0x7fad541551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.321+0000 7fad59800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad540b4430 con 0x7fad540af490 2024-03-28T13:14:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.321+0000 7fad59800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fad540b42b0 con 0x7fad5415eab0 2024-03-28T13:14:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.321+0000 7fad59800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fad540b45b0 con 0x7fad540b5990 2024-03-28T13:14:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.321+0000 7fad52a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad540af490 0x7fad5415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.321+0000 7fad52a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad540af490 0x7fad5415ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40820/0 (socket says 172.21.15.19:40820) 2024-03-28T13:14:35.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.321+0000 7fad52a00640 1 -- 172.21.15.19:0/1415637637 learned_addr learned my addr 172.21.15.19:0/1415637637 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.321+0000 7fad53400640 1 --2- 172.21.15.19:0/1415637637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad5415eab0 0x7fad541551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.321+0000 7fad52a00640 1 -- 172.21.15.19:0/1415637637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad540b5990 msgr2=0x7fad54153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.321+0000 7fad52a00640 1 --2- 172.21.15.19:0/1415637637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad540b5990 0x7fad54153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.321+0000 7fad52a00640 1 -- 172.21.15.19:0/1415637637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad5415eab0 msgr2=0x7fad541551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.321+0000 7fad52a00640 1 --2- 172.21.15.19:0/1415637637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad5415eab0 0x7fad541551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.321+0000 7fad52a00640 1 -- 172.21.15.19:0/1415637637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad54155720 con 0x7fad540af490 2024-03-28T13:14:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.322+0000 7fad52a00640 1 --2- 172.21.15.19:0/1415637637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad540af490 0x7fad5415ad30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fad44036040 tx=0x7fad44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.322+0000 7fad4b400640 1 -- 172.21.15.19:0/1415637637 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44043020 con 0x7fad540af490 2024-03-28T13:14:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.322+0000 7fad4b400640 1 -- 172.21.15.19:0/1415637637 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad4402ce50 con 0x7fad540af490 2024-03-28T13:14:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.322+0000 7fad59800640 1 -- 172.21.15.19:0/1415637637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad54156b30 con 0x7fad540af490 2024-03-28T13:14:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.322+0000 7fad4b400640 1 -- 172.21.15.19:0/1415637637 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44009510 con 0x7fad540af490 2024-03-28T13:14:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.322+0000 7fad59800640 1 -- 172.21.15.19:0/1415637637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad541549e0 con 0x7fad540af490 2024-03-28T13:14:35.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.324+0000 7fad59800640 1 -- 172.21.15.19:0/1415637637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad540685c0 con 0x7fad540af490 2024-03-28T13:14:35.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.325+0000 7fad4b400640 1 -- 172.21.15.19:0/1415637637 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fad44046020 con 0x7fad540af490 2024-03-28T13:14:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.325+0000 7fad4b400640 1 --2- 172.21.15.19:0/1415637637 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fad3006c560 0x7fad3006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:35.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.326+0000 7fad4b400640 1 -- 172.21.15.19:0/1415637637 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7fad440adc10 con 0x7fad540af490 2024-03-28T13:14:35.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.326+0000 7fad4a000640 1 --2- 172.21.15.19:0/1415637637 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fad3006c560 0x7fad3006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:35.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.327+0000 7fad4a000640 1 --2- 172.21.15.19:0/1415637637 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fad3006c560 0x7fad3006ea20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fad3c000f60 tx=0x7fad3c0079d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:35.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.330+0000 7fad4b400640 1 -- 172.21.15.19:0/1415637637 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fad4404d030 con 0x7fad540af490 2024-03-28T13:14:35.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.577+0000 7fad59800640 1 -- 172.21.15.19:0/1415637637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad54074790 con 0x7fad540af490 2024-03-28T13:14:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.578+0000 7fad4b400640 1 -- 172.21.15.19:0/1415637637 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5186 (secure 0 0 0) 0x7fad4403c350 con 0x7fad540af490 2024-03-28T13:14:35.578 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:35.578 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/144726418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":144726418},{"type":"v1","addr":"172.21.15.188:6836","nonce":144726418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5363,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2642581607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2642581607},{"type":"v1","addr":"172.21.15.19:6835","nonce":2642581607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1721039756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1721039756},{"type":"v1","addr":"172.21.15.188:6837","nonce":1721039756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:14:31.526577+0000","modified":"2024-03-28T13:14:34.573581+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6507},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6507":{"gid":6507,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/1189161682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1189161682},{"type":"v1","addr":"172.21.15.19:6834","nonce":1189161682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6507,"qdb_cluster":[6507]},"id":5}]} 2024-03-28T13:14:35.578 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 45 2024-03-28T13:14:35.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.580+0000 7fad59800640 1 -- 172.21.15.19:0/1415637637 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fad3006c560 msgr2=0x7fad3006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:35.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.580+0000 7fad59800640 1 --2- 172.21.15.19:0/1415637637 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fad3006c560 0x7fad3006ea20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fad3c000f60 tx=0x7fad3c0079d0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.580+0000 7fad59800640 1 -- 172.21.15.19:0/1415637637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad540af490 msgr2=0x7fad5415ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:35.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.580+0000 7fad59800640 1 --2- 172.21.15.19:0/1415637637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad540af490 0x7fad5415ad30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fad44036040 tx=0x7fad44004a40 comp rx=0 tx=0).stop 2024-03-28T13:14:35.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.580+0000 7fad59800640 1 -- 172.21.15.19:0/1415637637 shutdown_connections 2024-03-28T13:14:35.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.581+0000 7fad59800640 1 --2- 172.21.15.19:0/1415637637 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fad3006c560 0x7fad3006ea20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.581+0000 7fad59800640 1 --2- 172.21.15.19:0/1415637637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad5415eab0 0x7fad541551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.581+0000 7fad59800640 1 --2- 172.21.15.19:0/1415637637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad540b5990 0x7fad54153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.581+0000 7fad59800640 1 --2- 172.21.15.19:0/1415637637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad540af490 0x7fad5415ad30 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.581+0000 7fad59800640 1 -- 172.21.15.19:0/1415637637 >> 172.21.15.19:0/1415637637 conn(0x7fad54071df0 msgr2=0x7fad5415dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:35.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.581+0000 7fad59800640 1 -- 172.21.15.19:0/1415637637 shutdown_connections 2024-03-28T13:14:35.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.581+0000 7fad59800640 1 -- 172.21.15.19:0/1415637637 wait complete. 2024-03-28T13:14:35.600 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:14:35.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.813+0000 7f29e3e00640 1 Processor -- start 2024-03-28T13:14:35.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.813+0000 7f29e3e00640 1 -- start start 2024-03-28T13:14:35.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.814+0000 7f29e3e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f29e40d2310 0x7f29e40ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:35.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.814+0000 7f29e3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29e415e880 con 0x7f29e40cf530 2024-03-28T13:14:35.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.814+0000 7f29e3e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29e40d28d0 con 0x7f29e415e4a0 2024-03-28T13:14:35.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.814+0000 7f29e3e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f29e40cf9a0 con 0x7f29e40d2310 2024-03-28T13:14:35.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.815+0000 7f29e3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f29e40cf530 0x7f29e40b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48160/0 (socket says 172.21.15.19:48160) 2024-03-28T13:14:35.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.815+0000 7f29e3400640 1 -- 172.21.15.19:0/4128609059 learned_addr learned my addr 172.21.15.19:0/4128609059 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:35.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.815+0000 7f29e2000640 1 --2- 172.21.15.19:0/4128609059 >> v2:172.21.15.188:3300/0 conn(0x7f29e40d2310 0x7f29e40ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:35.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.815+0000 7f29e1600640 1 -- 172.21.15.19:0/4128609059 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3547699871 0 0) 0x7f29e415e880 con 0x7f29e40cf530 2024-03-28T13:14:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.815+0000 7f29e1600640 1 -- 172.21.15.19:0/4128609059 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29c8003520 con 0x7f29e40cf530 2024-03-28T13:14:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.816+0000 7f29e1600640 1 -- 172.21.15.19:0/4128609059 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1800378802 0 0) 0x7f29c8003520 con 0x7f29e40cf530 2024-03-28T13:14:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.816+0000 7f29e1600640 1 -- 172.21.15.19:0/4128609059 >> v2:172.21.15.188:3300/0 conn(0x7f29e40d2310 msgr2=0x7f29e40ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.816+0000 7f29e1600640 1 --2- 172.21.15.19:0/4128609059 >> v2:172.21.15.188:3300/0 conn(0x7f29e40d2310 0x7f29e40ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.816+0000 7f29e1600640 1 -- 172.21.15.19:0/4128609059 >> v1:172.21.15.188:6789/0 conn(0x7f29e415e4a0 legacy=0x7f29e40b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.816+0000 7f29e1600640 1 -- 172.21.15.19:0/4128609059 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e40d2bb0 con 0x7f29e40cf530 2024-03-28T13:14:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.816+0000 7f29e1600640 1 -- 172.21.15.19:0/4128609059 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f29dc001670 con 0x7f29e40cf530 2024-03-28T13:14:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.816+0000 7f29e1600640 1 -- 172.21.15.19:0/4128609059 >> v1:172.21.15.19:6789/0 conn(0x7f29e40cf530 legacy=0x7f29e40b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.816+0000 7f29e1600640 1 --2- 172.21.15.19:0/4128609059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c80039b0 0x7f29c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.816+0000 7f29e1600640 1 --2- 172.21.15.19:0/4128609059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c8004800 0x7f29c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.816+0000 7f29e1600640 1 --2- 172.21.15.19:0/4128609059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c80051c0 0x7f29c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.816+0000 7f29e1600640 1 -- 172.21.15.19:0/4128609059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29e40cf9a0 con 0x7f29c80039b0 2024-03-28T13:14:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.816+0000 7f29e1600640 1 -- 172.21.15.19:0/4128609059 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f29c800c160 con 0x7f29c80051c0 2024-03-28T13:14:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.816+0000 7f29e1600640 1 -- 172.21.15.19:0/4128609059 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f29c800c360 con 0x7f29c8004800 2024-03-28T13:14:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.816+0000 7f29e2a00640 1 --2- 172.21.15.19:0/4128609059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c80039b0 0x7f29c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:35.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.816+0000 7f29e3400640 1 --2- 172.21.15.19:0/4128609059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c80051c0 0x7f29c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:35.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.817+0000 7f29e2000640 1 --2- 172.21.15.19:0/4128609059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c8004800 0x7f29c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:35.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.817+0000 7f29e2a00640 1 -- 172.21.15.19:0/4128609059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c8004800 msgr2=0x7f29c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:35.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.817+0000 7f29e2a00640 1 --2- 172.21.15.19:0/4128609059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c8004800 0x7f29c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.817+0000 7f29e2a00640 1 -- 172.21.15.19:0/4128609059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c80051c0 msgr2=0x7f29c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:35.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.817+0000 7f29e2a00640 1 --2- 172.21.15.19:0/4128609059 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c80051c0 0x7f29c800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.817+0000 7f29e2a00640 1 -- 172.21.15.19:0/4128609059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c8003520 con 0x7f29c80039b0 2024-03-28T13:14:35.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.817+0000 7f29e3400640 1 -- 172.21.15.19:0/4128609059 reap_dead start 2024-03-28T13:14:35.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.819+0000 7f29e2a00640 1 --2- 172.21.15.19:0/4128609059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c80039b0 0x7f29c8003dd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f29d4001750 tx=0x7f29c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c0f1a35d6dde38 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:35.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.819+0000 7f29e1600640 1 -- 172.21.15.19:0/4128609059 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4001370 con 0x7f29c80039b0 2024-03-28T13:14:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.819+0000 7f29e1600640 1 -- 172.21.15.19:0/4128609059 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29d40052d0 con 0x7f29c80039b0 2024-03-28T13:14:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.819+0000 7f29e1600640 1 -- 172.21.15.19:0/4128609059 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4003830 con 0x7f29c80039b0 2024-03-28T13:14:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.819+0000 7f29e3e00640 1 -- 172.21.15.19:0/4128609059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c80039b0 msgr2=0x7f29c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.819+0000 7f29e3e00640 1 --2- 172.21.15.19:0/4128609059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c80039b0 0x7f29c8003dd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f29d4001750 tx=0x7f29c8005830 comp rx=0 tx=0).stop 2024-03-28T13:14:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.820+0000 7f29e3e00640 1 -- 172.21.15.19:0/4128609059 shutdown_connections 2024-03-28T13:14:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.820+0000 7f29e3e00640 1 --2- 172.21.15.19:0/4128609059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29c80039b0 0x7f29c8003dd0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.820+0000 7f29e3e00640 1 -- 172.21.15.19:0/4128609059 >> 172.21.15.19:0/4128609059 conn(0x7f29e40676d0 msgr2=0x7f29e4067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.820+0000 7f29e3e00640 1 -- 172.21.15.19:0/4128609059 shutdown_connections 2024-03-28T13:14:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.820+0000 7f29e3e00640 1 -- 172.21.15.19:0/4128609059 wait complete. 2024-03-28T13:14:35.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.821+0000 7f29e3e00640 1 Processor -- start 2024-03-28T13:14:35.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.822+0000 7f29e3e00640 1 -- start start 2024-03-28T13:14:35.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.822+0000 7f29e3e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c80039b0 0x7f29e4148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:35.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.823+0000 7f29e3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e4155560 0x7f29e41f7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:35.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.823+0000 7f29e3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e4158b90 0x7f29e40b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:35.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.823+0000 7f29e3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f29e40d3170 con 0x7f29e4158b90 2024-03-28T13:14:35.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.823+0000 7f29e3e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f29e40d34e0 con 0x7f29c80039b0 2024-03-28T13:14:35.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.823+0000 7f29e3e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f29e40d3970 con 0x7f29e4155560 2024-03-28T13:14:35.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.823+0000 7f29e3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e4158b90 0x7f29e40b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:35.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.823+0000 7f29e3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e4158b90 0x7f29e40b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48058/0 (socket says 172.21.15.19:48058) 2024-03-28T13:14:35.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.823+0000 7f29e3400640 1 -- 172.21.15.19:0/4147584216 learned_addr learned my addr 172.21.15.19:0/4147584216 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:35.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.823+0000 7f29e2a00640 1 --2- 172.21.15.19:0/4147584216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c80039b0 0x7f29e4148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:35.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.823+0000 7f29e3400640 1 -- 172.21.15.19:0/4147584216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e4155560 msgr2=0x7f29e41f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:35.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.823+0000 7f29e3400640 1 --2- 172.21.15.19:0/4147584216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e4155560 0x7f29e41f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.823+0000 7f29e3400640 1 -- 172.21.15.19:0/4147584216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c80039b0 msgr2=0x7f29e4148db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:35.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.823+0000 7f29e3400640 1 --2- 172.21.15.19:0/4147584216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c80039b0 0x7f29e4148db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:35.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.823+0000 7f29e3400640 1 -- 172.21.15.19:0/4147584216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e41f7fc0 con 0x7f29e4158b90 2024-03-28T13:14:35.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.824+0000 7f29e3400640 1 --2- 172.21.15.19:0/4147584216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e4158b90 0x7f29e40b42d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f29e406d5c0 tx=0x7f29dc00e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:35.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.824+0000 7f29daa00640 1 -- 172.21.15.19:0/4147584216 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc019070 con 0x7f29e4158b90 2024-03-28T13:14:35.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.824+0000 7f29daa00640 1 -- 172.21.15.19:0/4147584216 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29dc010510 con 0x7f29e4158b90 2024-03-28T13:14:35.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.824+0000 7f29e3e00640 1 -- 172.21.15.19:0/4147584216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e41f82a0 con 0x7f29e4158b90 2024-03-28T13:14:35.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.824+0000 7f29daa00640 1 -- 172.21.15.19:0/4147584216 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29dc0108e0 con 0x7f29e4158b90 2024-03-28T13:14:35.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.824+0000 7f29e3e00640 1 -- 172.21.15.19:0/4147584216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e4148890 con 0x7f29e4158b90 2024-03-28T13:14:35.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.825+0000 7f29e3e00640 1 -- 172.21.15.19:0/4147584216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e4075940 con 0x7f29e4158b90 2024-03-28T13:14:35.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.827+0000 7f29daa00640 1 -- 172.21.15.19:0/4147584216 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f29dc01c070 con 0x7f29e4158b90 2024-03-28T13:14:35.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.829+0000 7f29daa00640 1 --2- 172.21.15.19:0/4147584216 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f29c006c530 0x7f29c006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:35.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.829+0000 7f29daa00640 1 -- 172.21.15.19:0/4147584216 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f29dc08abb0 con 0x7f29e4158b90 2024-03-28T13:14:35.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.829+0000 7f29e2a00640 1 --2- 172.21.15.19:0/4147584216 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f29c006c530 0x7f29c006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:35.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.831+0000 7f29e2a00640 1 --2- 172.21.15.19:0/4147584216 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f29c006c530 0x7f29c006e9f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f29d4033000 tx=0x7f29d4009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:35.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:35.834+0000 7f29daa00640 1 -- 172.21.15.19:0/4147584216 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f29dc058b80 con 0x7f29e4158b90 2024-03-28T13:14:36.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.102+0000 7f29e3e00640 1 -- 172.21.15.19:0/4147584216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29e4067e80 con 0x7f29e4158b90 2024-03-28T13:14:36.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.105+0000 7f29daa00640 1 -- 172.21.15.19:0/4147584216 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+16608 (secure 0 0 0) 0x7f29dc01fe20 con 0x7f29e4158b90 2024-03-28T13:14:36.105 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:36.106 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":77,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:14:33.055510+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:14:29.424080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-28T13:14:30.494667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:14:36.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.108+0000 7f29e3e00640 1 -- 172.21.15.19:0/4147584216 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f29c006c530 msgr2=0x7f29c006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:36.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.108+0000 7f29e3e00640 1 --2- 172.21.15.19:0/4147584216 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f29c006c530 0x7f29c006e9f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f29d4033000 tx=0x7f29d4009030 comp rx=0 tx=0).stop 2024-03-28T13:14:36.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.108+0000 7f29e3e00640 1 -- 172.21.15.19:0/4147584216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e4158b90 msgr2=0x7f29e40b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:36.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.108+0000 7f29e3e00640 1 --2- 172.21.15.19:0/4147584216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e4158b90 0x7f29e40b42d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f29e406d5c0 tx=0x7f29dc00e890 comp rx=0 tx=0).stop 2024-03-28T13:14:36.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.108+0000 7f29e3e00640 1 -- 172.21.15.19:0/4147584216 shutdown_connections 2024-03-28T13:14:36.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.108+0000 7f29e3e00640 1 --2- 172.21.15.19:0/4147584216 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f29c006c530 0x7f29c006e9f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.108+0000 7f29e3e00640 1 --2- 172.21.15.19:0/4147584216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29e4158b90 0x7f29e40b42d0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.108+0000 7f29e3e00640 1 --2- 172.21.15.19:0/4147584216 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29e4155560 0x7f29e41f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.108+0000 7f29e3e00640 1 --2- 172.21.15.19:0/4147584216 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c80039b0 0x7f29e4148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.108+0000 7f29e3e00640 1 -- 172.21.15.19:0/4147584216 >> 172.21.15.19:0/4147584216 conn(0x7f29e40676d0 msgr2=0x7f29e415cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:36.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.109+0000 7f29e3e00640 1 -- 172.21.15.19:0/4147584216 shutdown_connections 2024-03-28T13:14:36.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.109+0000 7f29e3e00640 1 -- 172.21.15.19:0/4147584216 wait complete. 2024-03-28T13:14:36.128 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-28T13:14:36.128 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:14:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.335+0000 7f5dccc00640 1 Processor -- start 2024-03-28T13:14:36.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.335+0000 7f5dccc00640 1 -- start start 2024-03-28T13:14:36.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.336+0000 7f5dccc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5dc815e4a0 0x7f5dc80b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.337+0000 7f5dccc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc815e880 con 0x7f5dc80cf700 2024-03-28T13:14:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.337+0000 7f5dccc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dc80d28d0 con 0x7f5dc80d2310 2024-03-28T13:14:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.337+0000 7f5dccc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5dc80d34e0 con 0x7f5dc815e4a0 2024-03-28T13:14:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.337+0000 7f5dc7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5dc80cf700 0x7f5dc80b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48166/0 (socket says 172.21.15.19:48166) 2024-03-28T13:14:36.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.337+0000 7f5dc7e00640 1 -- 172.21.15.19:0/2915773053 learned_addr learned my addr 172.21.15.19:0/2915773053 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.337+0000 7f5dc7400640 1 --2- 172.21.15.19:0/2915773053 >> v2:172.21.15.188:3300/0 conn(0x7f5dc815e4a0 0x7f5dc80b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc6000640 1 -- 172.21.15.19:0/2915773053 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 84576741 0 0) 0x7f5dc815e880 con 0x7f5dc80cf700 2024-03-28T13:14:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc6000640 1 -- 172.21.15.19:0/2915773053 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5dac003520 con 0x7f5dc80cf700 2024-03-28T13:14:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc6000640 1 -- 172.21.15.19:0/2915773053 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3002579121 0 0) 0x7f5dac003520 con 0x7f5dc80cf700 2024-03-28T13:14:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc6000640 1 -- 172.21.15.19:0/2915773053 >> v2:172.21.15.188:3300/0 conn(0x7f5dc815e4a0 msgr2=0x7f5dc80b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc6000640 1 --2- 172.21.15.19:0/2915773053 >> v2:172.21.15.188:3300/0 conn(0x7f5dc815e4a0 0x7f5dc80b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc6000640 1 -- 172.21.15.19:0/2915773053 >> v1:172.21.15.188:6789/0 conn(0x7f5dc80d2310 legacy=0x7f5dc80ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc6000640 1 -- 172.21.15.19:0/2915773053 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc80d2bb0 con 0x7f5dc80cf700 2024-03-28T13:14:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc6000640 1 -- 172.21.15.19:0/2915773053 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f5dc0001a10 con 0x7f5dc80cf700 2024-03-28T13:14:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc6000640 1 -- 172.21.15.19:0/2915773053 >> v1:172.21.15.19:6789/0 conn(0x7f5dc80cf700 legacy=0x7f5dc80b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc6000640 1 --2- 172.21.15.19:0/2915773053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dac0039b0 0x7f5dac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc6000640 1 --2- 172.21.15.19:0/2915773053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dac004800 0x7f5dac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc6000640 1 --2- 172.21.15.19:0/2915773053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dac0051c0 0x7f5dac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc6000640 1 -- 172.21.15.19:0/2915773053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc80d34e0 con 0x7f5dac0039b0 2024-03-28T13:14:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc6000640 1 -- 172.21.15.19:0/2915773053 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac00c160 con 0x7f5dac0051c0 2024-03-28T13:14:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc6000640 1 -- 172.21.15.19:0/2915773053 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5dac00c360 con 0x7f5dac004800 2024-03-28T13:14:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.338+0000 7f5dc7400640 1 --2- 172.21.15.19:0/2915773053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dac0039b0 0x7f5dac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.339+0000 7f5dc7400640 1 -- 172.21.15.19:0/2915773053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dac004800 msgr2=0x7f5dac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.339+0000 7f5dc7400640 1 --2- 172.21.15.19:0/2915773053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dac004800 0x7f5dac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.339+0000 7f5dc7400640 1 -- 172.21.15.19:0/2915773053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dac0051c0 msgr2=0x7f5dac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.339+0000 7f5dc7400640 1 --2- 172.21.15.19:0/2915773053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dac0051c0 0x7f5dac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.339+0000 7f5dc7400640 1 -- 172.21.15.19:0/2915773053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dac003520 con 0x7f5dac0039b0 2024-03-28T13:14:36.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.339+0000 7f5dc7e00640 1 -- 172.21.15.19:0/2915773053 reap_dead start 2024-03-28T13:14:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.341+0000 7f5dc7400640 1 --2- 172.21.15.19:0/2915773053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dac0039b0 0x7f5dac003dd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009ac0 tx=0x7f5dac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9947efa91e716c45 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.341+0000 7f5dc6000640 1 -- 172.21.15.19:0/2915773053 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8039070 con 0x7f5dac0039b0 2024-03-28T13:14:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.341+0000 7f5dc6000640 1 -- 172.21.15.19:0/2915773053 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db802c710 con 0x7f5dac0039b0 2024-03-28T13:14:36.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.341+0000 7f5dc6000640 1 -- 172.21.15.19:0/2915773053 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8009690 con 0x7f5dac0039b0 2024-03-28T13:14:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.342+0000 7f5dccc00640 1 -- 172.21.15.19:0/2915773053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dac0039b0 msgr2=0x7f5dac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.342+0000 7f5dccc00640 1 --2- 172.21.15.19:0/2915773053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dac0039b0 0x7f5dac003dd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009ac0 tx=0x7f5dac005830 comp rx=0 tx=0).stop 2024-03-28T13:14:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.342+0000 7f5dccc00640 1 -- 172.21.15.19:0/2915773053 shutdown_connections 2024-03-28T13:14:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.342+0000 7f5dccc00640 1 --2- 172.21.15.19:0/2915773053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dac0039b0 0x7f5dac003dd0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.342+0000 7f5dccc00640 1 -- 172.21.15.19:0/2915773053 >> 172.21.15.19:0/2915773053 conn(0x7f5dc80676d0 msgr2=0x7f5dc8067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.342+0000 7f5dccc00640 1 -- 172.21.15.19:0/2915773053 shutdown_connections 2024-03-28T13:14:36.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.342+0000 7f5dccc00640 1 -- 172.21.15.19:0/2915773053 wait complete. 2024-03-28T13:14:36.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.344+0000 7f5dccc00640 1 Processor -- start 2024-03-28T13:14:36.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.344+0000 7f5dccc00640 1 -- start start 2024-03-28T13:14:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.345+0000 7f5dccc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dac0039b0 0x7f5dc80ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.345+0000 7f5dccc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dc8156240 0x7f5dc80cd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.345+0000 7f5dccc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dc80cd860 0x7f5dc8157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.345+0000 7f5dccc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc80d3970 con 0x7f5dac0039b0 2024-03-28T13:14:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.345+0000 7f5dccc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc80d3170 con 0x7f5dc8156240 2024-03-28T13:14:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.345+0000 7f5dccc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc80ceff0 con 0x7f5dc80cd860 2024-03-28T13:14:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.345+0000 7f5dc7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dac0039b0 0x7f5dc80ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.345+0000 7f5dc7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dac0039b0 0x7f5dc80ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48062/0 (socket says 172.21.15.19:48062) 2024-03-28T13:14:36.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.345+0000 7f5dc7400640 1 -- 172.21.15.19:0/4064853115 learned_addr learned my addr 172.21.15.19:0/4064853115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.346+0000 7f5dc6a00640 1 --2- 172.21.15.19:0/4064853115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dc8156240 0x7f5dc80cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.346+0000 7f5dc7e00640 1 --2- 172.21.15.19:0/4064853115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dc80cd860 0x7f5dc8157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.346+0000 7f5dc7400640 1 -- 172.21.15.19:0/4064853115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dc80cd860 msgr2=0x7f5dc8157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.346+0000 7f5dc7400640 1 --2- 172.21.15.19:0/4064853115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dc80cd860 0x7f5dc8157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.346+0000 7f5dc7400640 1 -- 172.21.15.19:0/4064853115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dc8156240 msgr2=0x7f5dc80cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.346+0000 7f5dc7400640 1 --2- 172.21.15.19:0/4064853115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dc8156240 0x7f5dc80cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.346+0000 7f5dc7400640 1 -- 172.21.15.19:0/4064853115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc8159500 con 0x7f5dac0039b0 2024-03-28T13:14:36.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.346+0000 7f5dc7400640 1 --2- 172.21.15.19:0/4064853115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dac0039b0 0x7f5dc80ccde0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f5db8000c00 tx=0x7f5db8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.347+0000 7f5dbf400640 1 -- 172.21.15.19:0/4064853115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8039050 con 0x7f5dac0039b0 2024-03-28T13:14:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.347+0000 7f5dbf400640 1 -- 172.21.15.19:0/4064853115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5db8045070 con 0x7f5dac0039b0 2024-03-28T13:14:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.347+0000 7f5dbf400640 1 -- 172.21.15.19:0/4064853115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db80044e0 con 0x7f5dac0039b0 2024-03-28T13:14:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.347+0000 7f5dccc00640 1 -- 172.21.15.19:0/4064853115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc80cdce0 con 0x7f5dac0039b0 2024-03-28T13:14:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.347+0000 7f5dccc00640 1 -- 172.21.15.19:0/4064853115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc815f130 con 0x7f5dac0039b0 2024-03-28T13:14:36.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.348+0000 7f5dccc00640 1 -- 172.21.15.19:0/4064853115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc815f370 con 0x7f5dac0039b0 2024-03-28T13:14:36.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.350+0000 7f5dbf400640 1 -- 172.21.15.19:0/4064853115 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5db8003a50 con 0x7f5dac0039b0 2024-03-28T13:14:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.351+0000 7f5dbf400640 1 --2- 172.21.15.19:0/4064853115 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5da806c560 0x7f5da806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.352+0000 7f5dbf400640 1 -- 172.21.15.19:0/4064853115 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f5db80ad6b0 con 0x7f5dac0039b0 2024-03-28T13:14:36.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.352+0000 7f5dc6a00640 1 --2- 172.21.15.19:0/4064853115 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5da806c560 0x7f5da806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:36.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.353+0000 7f5dc6a00640 1 --2- 172.21.15.19:0/4064853115 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5da806c560 0x7f5da806ea20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5dc81f7620 tx=0x7f5db4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:36.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.356+0000 7f5dbf400640 1 -- 172.21.15.19:0/4064853115 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5dc815f370 con 0x7f5dac0039b0 2024-03-28T13:14:36.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.555+0000 7f5dccc00640 1 -- 172.21.15.19:0/4064853115 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5dc8075940 con 0x7f5da806c560 2024-03-28T13:14:36.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.575+0000 7f5dbf400640 1 -- 172.21.15.19:0/4064853115 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252062 (secure 0 0 0) 0x7f5dc8075940 con 0x7f5da806c560 2024-03-28T13:14:36.579 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:36.581 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:14:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.580+0000 7f5dccc00640 1 -- 172.21.15.19:0/4064853115 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5da806c560 msgr2=0x7f5da806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.580+0000 7f5dccc00640 1 --2- 172.21.15.19:0/4064853115 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5da806c560 0x7f5da806ea20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5dc81f7620 tx=0x7f5db4009040 comp rx=0 tx=0).stop 2024-03-28T13:14:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.580+0000 7f5dccc00640 1 -- 172.21.15.19:0/4064853115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dac0039b0 msgr2=0x7f5dc80ccde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.580+0000 7f5dccc00640 1 --2- 172.21.15.19:0/4064853115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dac0039b0 0x7f5dc80ccde0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f5db8000c00 tx=0x7f5db8004730 comp rx=0 tx=0).stop 2024-03-28T13:14:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.580+0000 7f5dccc00640 1 -- 172.21.15.19:0/4064853115 shutdown_connections 2024-03-28T13:14:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.580+0000 7f5dccc00640 1 --2- 172.21.15.19:0/4064853115 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5da806c560 0x7f5da806ea20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.580+0000 7f5dccc00640 1 --2- 172.21.15.19:0/4064853115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dc80cd860 0x7f5dc8157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.580+0000 7f5dccc00640 1 --2- 172.21.15.19:0/4064853115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dc8156240 0x7f5dc80cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.580+0000 7f5dccc00640 1 --2- 172.21.15.19:0/4064853115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dac0039b0 0x7f5dc80ccde0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.580+0000 7f5dccc00640 1 -- 172.21.15.19:0/4064853115 >> 172.21.15.19:0/4064853115 conn(0x7f5dc80676d0 msgr2=0x7f5dc80d3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.581+0000 7f5dccc00640 1 -- 172.21.15.19:0/4064853115 shutdown_connections 2024-03-28T13:14:36.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.581+0000 7f5dccc00640 1 -- 172.21.15.19:0/4064853115 wait complete. 2024-03-28T13:14:36.598 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-03-28T13:14:36.092766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257900,"kb_used_data":8432,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716820,"statfs":{"total":773094113280,"available":772830023680,"internally_reserved":0,"allocated":8634368,"data_stored":4314612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001925"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071743+0000","last_change":"2024-03-28T13:14:32.086179+0000","last_active":"2024-03-28T13:14:33.071743+0000","last_peered":"2024-03-28T13:14:33.071743+0000","last_clean":"2024-03-28T13:14:33.071743+0000","last_became_active":"2024-03-28T13:14:32.085893+0000","last_became_peered":"2024-03-28T13:14:32.085893+0000","last_unstale":"2024-03-28T13:14:33.071743+0000","last_undegraded":"2024-03-28T13:14:33.071743+0000","last_fullsized":"2024-03-28T13:14:33.071743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:26.383672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059640+0000","last_change":"2024-03-28T13:14:32.081691+0000","last_active":"2024-03-28T13:14:34.059640+0000","last_peered":"2024-03-28T13:14:34.059640+0000","last_clean":"2024-03-28T13:14:34.059640+0000","last_became_active":"2024-03-28T13:14:32.081401+0000","last_became_peered":"2024-03-28T13:14:32.081401+0000","last_unstale":"2024-03-28T13:14:34.059640+0000","last_undegraded":"2024-03-28T13:14:34.059640+0000","last_fullsized":"2024-03-28T13:14:34.059640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:51:05.489280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.072572+0000","last_change":"2024-03-28T13:14:32.068576+0000","last_active":"2024-03-28T13:14:32.072572+0000","last_peered":"2024-03-28T13:14:32.072572+0000","last_clean":"2024-03-28T13:14:32.072572+0000","last_became_active":"2024-03-28T13:14:32.068252+0000","last_became_peered":"2024-03-28T13:14:32.068252+0000","last_unstale":"2024-03-28T13:14:32.072572+0000","last_undegraded":"2024-03-28T13:14:32.072572+0000","last_fullsized":"2024-03-28T13:14:32.072572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:58.024115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.079244+0000","last_change":"2024-03-28T13:14:32.072558+0000","last_active":"2024-03-28T13:14:32.079244+0000","last_peered":"2024-03-28T13:14:32.079244+0000","last_clean":"2024-03-28T13:14:32.079244+0000","last_became_active":"2024-03-28T13:14:32.072269+0000","last_became_peered":"2024-03-28T13:14:32.072269+0000","last_unstale":"2024-03-28T13:14:32.079244+0000","last_undegraded":"2024-03-28T13:14:32.079244+0000","last_fullsized":"2024-03-28T13:14:32.079244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:43:26.658980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.072854+0000","last_change":"2024-03-28T13:14:32.080653+0000","last_active":"2024-03-28T13:14:33.072854+0000","last_peered":"2024-03-28T13:14:33.072854+0000","last_clean":"2024-03-28T13:14:33.072854+0000","last_became_active":"2024-03-28T13:14:32.078004+0000","last_became_peered":"2024-03-28T13:14:32.078004+0000","last_unstale":"2024-03-28T13:14:33.072854+0000","last_undegraded":"2024-03-28T13:14:33.072854+0000","last_fullsized":"2024-03-28T13:14:33.072854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:13:02.556681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060070+0000","last_change":"2024-03-28T13:14:32.061384+0000","last_active":"2024-03-28T13:14:34.060070+0000","last_peered":"2024-03-28T13:14:34.060070+0000","last_clean":"2024-03-28T13:14:34.060070+0000","last_became_active":"2024-03-28T13:14:32.058089+0000","last_became_peered":"2024-03-28T13:14:32.058089+0000","last_unstale":"2024-03-28T13:14:34.060070+0000","last_undegraded":"2024-03-28T13:14:34.060070+0000","last_fullsized":"2024-03-28T13:14:34.060070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:13:02.512771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351156+0000","last_change":"2024-03-28T13:14:32.088103+0000","last_active":"2024-03-28T13:14:33.351156+0000","last_peered":"2024-03-28T13:14:33.351156+0000","last_clean":"2024-03-28T13:14:33.351156+0000","last_became_active":"2024-03-28T13:14:32.087761+0000","last_became_peered":"2024-03-28T13:14:32.087761+0000","last_unstale":"2024-03-28T13:14:33.351156+0000","last_undegraded":"2024-03-28T13:14:33.351156+0000","last_fullsized":"2024-03-28T13:14:33.351156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:14.605529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.069742+0000","last_change":"2024-03-28T13:14:32.076227+0000","last_active":"2024-03-28T13:14:33.069742+0000","last_peered":"2024-03-28T13:14:33.069742+0000","last_clean":"2024-03-28T13:14:33.069742+0000","last_became_active":"2024-03-28T13:14:32.075917+0000","last_became_peered":"2024-03-28T13:14:32.075917+0000","last_unstale":"2024-03-28T13:14:33.069742+0000","last_undegraded":"2024-03-28T13:14:33.069742+0000","last_fullsized":"2024-03-28T13:14:33.069742+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:51:38.097780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.082358+0000","last_change":"2024-03-28T13:14:32.077975+0000","last_active":"2024-03-28T13:14:32.082358+0000","last_peered":"2024-03-28T13:14:32.082358+0000","last_clean":"2024-03-28T13:14:32.082358+0000","last_became_active":"2024-03-28T13:14:32.077659+0000","last_became_peered":"2024-03-28T13:14:32.077659+0000","last_unstale":"2024-03-28T13:14:32.082358+0000","last_undegraded":"2024-03-28T13:14:32.082358+0000","last_fullsized":"2024-03-28T13:14:32.082358+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:56:27.147994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.352143+0000","last_change":"2024-03-28T13:14:32.088520+0000","last_active":"2024-03-28T13:14:33.352143+0000","last_peered":"2024-03-28T13:14:33.352143+0000","last_clean":"2024-03-28T13:14:33.352143+0000","last_became_active":"2024-03-28T13:14:32.087092+0000","last_became_peered":"2024-03-28T13:14:32.087092+0000","last_unstale":"2024-03-28T13:14:33.352143+0000","last_undegraded":"2024-03-28T13:14:33.352143+0000","last_fullsized":"2024-03-28T13:14:33.352143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:49:40.704170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071258+0000","last_change":"2024-03-28T13:14:32.076370+0000","last_active":"2024-03-28T13:14:33.071258+0000","last_peered":"2024-03-28T13:14:33.071258+0000","last_clean":"2024-03-28T13:14:33.071258+0000","last_became_active":"2024-03-28T13:14:32.076064+0000","last_became_peered":"2024-03-28T13:14:32.076064+0000","last_unstale":"2024-03-28T13:14:33.071258+0000","last_undegraded":"2024-03-28T13:14:33.071258+0000","last_fullsized":"2024-03-28T13:14:33.071258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:15:13.865834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060521+0000","last_change":"2024-03-28T13:14:32.083698+0000","last_active":"2024-03-28T13:14:34.060521+0000","last_peered":"2024-03-28T13:14:34.060521+0000","last_clean":"2024-03-28T13:14:34.060521+0000","last_became_active":"2024-03-28T13:14:32.083098+0000","last_became_peered":"2024-03-28T13:14:32.083098+0000","last_unstale":"2024-03-28T13:14:34.060521+0000","last_undegraded":"2024-03-28T13:14:34.060521+0000","last_fullsized":"2024-03-28T13:14:34.060521+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:07.232388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.087368+0000","last_change":"2024-03-28T13:14:32.084201+0000","last_active":"2024-03-28T13:14:32.087368+0000","last_peered":"2024-03-28T13:14:32.087368+0000","last_clean":"2024-03-28T13:14:32.087368+0000","last_became_active":"2024-03-28T13:14:32.083907+0000","last_became_peered":"2024-03-28T13:14:32.083907+0000","last_unstale":"2024-03-28T13:14:32.087368+0000","last_undegraded":"2024-03-28T13:14:32.087368+0000","last_fullsized":"2024-03-28T13:14:32.087368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:42:16.554792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059944+0000","last_change":"2024-03-28T13:14:32.082254+0000","last_active":"2024-03-28T13:14:34.059944+0000","last_peered":"2024-03-28T13:14:34.059944+0000","last_clean":"2024-03-28T13:14:34.059944+0000","last_became_active":"2024-03-28T13:14:32.081983+0000","last_became_peered":"2024-03-28T13:14:32.081983+0000","last_unstale":"2024-03-28T13:14:34.059944+0000","last_undegraded":"2024-03-28T13:14:34.059944+0000","last_fullsized":"2024-03-28T13:14:34.059944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:13:02.512771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060114+0000","last_change":"2024-03-28T13:14:32.076292+0000","last_active":"2024-03-28T13:14:34.060114+0000","last_peered":"2024-03-28T13:14:34.060114+0000","last_clean":"2024-03-28T13:14:34.060114+0000","last_became_active":"2024-03-28T13:14:32.075976+0000","last_became_peered":"2024-03-28T13:14:32.075976+0000","last_unstale":"2024-03-28T13:14:34.060114+0000","last_undegraded":"2024-03-28T13:14:34.060114+0000","last_fullsized":"2024-03-28T13:14:34.060114+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:18.345000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.069756+0000","last_change":"2024-03-28T13:14:32.060377+0000","last_active":"2024-03-28T13:14:33.069756+0000","last_peered":"2024-03-28T13:14:33.069756+0000","last_clean":"2024-03-28T13:14:33.069756+0000","last_became_active":"2024-03-28T13:14:32.059330+0000","last_became_peered":"2024-03-28T13:14:32.059330+0000","last_unstale":"2024-03-28T13:14:33.069756+0000","last_undegraded":"2024-03-28T13:14:33.069756+0000","last_fullsized":"2024-03-28T13:14:33.069756+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:41.183649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.074029+0000","last_change":"2024-03-28T13:14:32.083257+0000","last_active":"2024-03-28T13:14:33.074029+0000","last_peered":"2024-03-28T13:14:33.074029+0000","last_clean":"2024-03-28T13:14:33.074029+0000","last_became_active":"2024-03-28T13:14:32.082865+0000","last_became_peered":"2024-03-28T13:14:32.082865+0000","last_unstale":"2024-03-28T13:14:33.074029+0000","last_undegraded":"2024-03-28T13:14:33.074029+0000","last_fullsized":"2024-03-28T13:14:33.074029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:20:39.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.074011+0000","last_change":"2024-03-28T13:14:32.092110+0000","last_active":"2024-03-28T13:14:33.074011+0000","last_peered":"2024-03-28T13:14:33.074011+0000","last_clean":"2024-03-28T13:14:33.074011+0000","last_became_active":"2024-03-28T13:14:32.091778+0000","last_became_peered":"2024-03-28T13:14:32.091778+0000","last_unstale":"2024-03-28T13:14:33.074011+0000","last_undegraded":"2024-03-28T13:14:33.074011+0000","last_fullsized":"2024-03-28T13:14:33.074011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:13:42.425225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071088+0000","last_change":"2024-03-28T13:14:32.058245+0000","last_active":"2024-03-28T13:14:33.071088+0000","last_peered":"2024-03-28T13:14:33.071088+0000","last_clean":"2024-03-28T13:14:33.071088+0000","last_became_active":"2024-03-28T13:14:32.057857+0000","last_became_peered":"2024-03-28T13:14:32.057857+0000","last_unstale":"2024-03-28T13:14:33.071088+0000","last_undegraded":"2024-03-28T13:14:33.071088+0000","last_fullsized":"2024-03-28T13:14:33.071088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:25:34.731536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.073300+0000","last_change":"2024-03-28T13:14:32.086692+0000","last_active":"2024-03-28T13:14:33.073300+0000","last_peered":"2024-03-28T13:14:33.073300+0000","last_clean":"2024-03-28T13:14:33.073300+0000","last_became_active":"2024-03-28T13:14:32.084876+0000","last_became_peered":"2024-03-28T13:14:32.084876+0000","last_unstale":"2024-03-28T13:14:33.073300+0000","last_undegraded":"2024-03-28T13:14:33.073300+0000","last_fullsized":"2024-03-28T13:14:33.073300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:40:05.574117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059044+0000","last_change":"2024-03-28T13:14:32.073579+0000","last_active":"2024-03-28T13:14:34.059044+0000","last_peered":"2024-03-28T13:14:34.059044+0000","last_clean":"2024-03-28T13:14:34.059044+0000","last_became_active":"2024-03-28T13:14:32.073300+0000","last_became_peered":"2024-03-28T13:14:32.073300+0000","last_unstale":"2024-03-28T13:14:34.059044+0000","last_undegraded":"2024-03-28T13:14:34.059044+0000","last_fullsized":"2024-03-28T13:14:34.059044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:18.345000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.061332+0000","last_change":"2024-03-28T13:14:32.077547+0000","last_active":"2024-03-28T13:14:34.061332+0000","last_peered":"2024-03-28T13:14:34.061332+0000","last_clean":"2024-03-28T13:14:34.061332+0000","last_became_active":"2024-03-28T13:14:32.075519+0000","last_became_peered":"2024-03-28T13:14:32.075519+0000","last_unstale":"2024-03-28T13:14:34.061332+0000","last_undegraded":"2024-03-28T13:14:34.061332+0000","last_fullsized":"2024-03-28T13:14:34.061332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:18.345000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351266+0000","last_change":"2024-03-28T13:14:32.087926+0000","last_active":"2024-03-28T13:14:33.351266+0000","last_peered":"2024-03-28T13:14:33.351266+0000","last_clean":"2024-03-28T13:14:33.351266+0000","last_became_active":"2024-03-28T13:14:32.087454+0000","last_became_peered":"2024-03-28T13:14:32.087454+0000","last_unstale":"2024-03-28T13:14:33.351266+0000","last_undegraded":"2024-03-28T13:14:33.351266+0000","last_fullsized":"2024-03-28T13:14:33.351266+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:53:36.071173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.069858+0000","last_change":"2024-03-28T13:14:32.055538+0000","last_active":"2024-03-28T13:14:33.069858+0000","last_peered":"2024-03-28T13:14:33.069858+0000","last_clean":"2024-03-28T13:14:33.069858+0000","last_became_active":"2024-03-28T13:14:32.055203+0000","last_became_peered":"2024-03-28T13:14:32.055203+0000","last_unstale":"2024-03-28T13:14:33.069858+0000","last_undegraded":"2024-03-28T13:14:33.069858+0000","last_fullsized":"2024-03-28T13:14:33.069858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:23:14.381420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351899+0000","last_change":"2024-03-28T13:14:32.085509+0000","last_active":"2024-03-28T13:14:33.351899+0000","last_peered":"2024-03-28T13:14:33.351899+0000","last_clean":"2024-03-28T13:14:33.351899+0000","last_became_active":"2024-03-28T13:14:32.084987+0000","last_became_peered":"2024-03-28T13:14:32.084987+0000","last_unstale":"2024-03-28T13:14:33.351899+0000","last_undegraded":"2024-03-28T13:14:33.351899+0000","last_fullsized":"2024-03-28T13:14:33.351899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:31:26.323310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071201+0000","last_change":"2024-03-28T13:14:32.078943+0000","last_active":"2024-03-28T13:14:33.071201+0000","last_peered":"2024-03-28T13:14:33.071201+0000","last_clean":"2024-03-28T13:14:33.071201+0000","last_became_active":"2024-03-28T13:14:32.078592+0000","last_became_peered":"2024-03-28T13:14:32.078592+0000","last_unstale":"2024-03-28T13:14:33.071201+0000","last_undegraded":"2024-03-28T13:14:33.071201+0000","last_fullsized":"2024-03-28T13:14:33.071201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:15:08.895008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.073473+0000","last_change":"2024-03-28T13:14:32.086509+0000","last_active":"2024-03-28T13:14:33.073473+0000","last_peered":"2024-03-28T13:14:33.073473+0000","last_clean":"2024-03-28T13:14:33.073473+0000","last_became_active":"2024-03-28T13:14:32.085203+0000","last_became_peered":"2024-03-28T13:14:32.085203+0000","last_unstale":"2024-03-28T13:14:33.073473+0000","last_undegraded":"2024-03-28T13:14:33.073473+0000","last_fullsized":"2024-03-28T13:14:33.073473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:37:48.508041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059488+0000","last_change":"2024-03-28T13:14:32.071537+0000","last_active":"2024-03-28T13:14:34.059488+0000","last_peered":"2024-03-28T13:14:34.059488+0000","last_clean":"2024-03-28T13:14:34.059488+0000","last_became_active":"2024-03-28T13:14:32.071228+0000","last_became_peered":"2024-03-28T13:14:32.071228+0000","last_unstale":"2024-03-28T13:14:34.059488+0000","last_undegraded":"2024-03-28T13:14:34.059488+0000","last_fullsized":"2024-03-28T13:14:34.059488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:07.232388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.072402+0000","last_change":"2024-03-28T13:14:32.065606+0000","last_active":"2024-03-28T13:14:32.072402+0000","last_peered":"2024-03-28T13:14:32.072402+0000","last_clean":"2024-03-28T13:14:32.072402+0000","last_became_active":"2024-03-28T13:14:32.065296+0000","last_became_peered":"2024-03-28T13:14:32.065296+0000","last_unstale":"2024-03-28T13:14:32.072402+0000","last_undegraded":"2024-03-28T13:14:32.072402+0000","last_fullsized":"2024-03-28T13:14:32.072402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:50:02.228041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.070729+0000","last_change":"2024-03-28T13:14:32.057766+0000","last_active":"2024-03-28T13:14:33.070729+0000","last_peered":"2024-03-28T13:14:33.070729+0000","last_clean":"2024-03-28T13:14:33.070729+0000","last_became_active":"2024-03-28T13:14:32.057492+0000","last_became_peered":"2024-03-28T13:14:32.057492+0000","last_unstale":"2024-03-28T13:14:33.070729+0000","last_undegraded":"2024-03-28T13:14:33.070729+0000","last_fullsized":"2024-03-28T13:14:33.070729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:31:36.795818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060520+0000","last_change":"2024-03-28T13:14:32.078346+0000","last_active":"2024-03-28T13:14:34.060520+0000","last_peered":"2024-03-28T13:14:34.060520+0000","last_clean":"2024-03-28T13:14:34.060520+0000","last_became_active":"2024-03-28T13:14:32.078001+0000","last_became_peered":"2024-03-28T13:14:32.078001+0000","last_unstale":"2024-03-28T13:14:34.060520+0000","last_undegraded":"2024-03-28T13:14:34.060520+0000","last_fullsized":"2024-03-28T13:14:34.060520+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:54:12.565079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.058780+0000","last_change":"2024-03-28T13:14:32.072888+0000","last_active":"2024-03-28T13:14:34.058780+0000","last_peered":"2024-03-28T13:14:34.058780+0000","last_clean":"2024-03-28T13:14:34.058780+0000","last_became_active":"2024-03-28T13:14:32.070877+0000","last_became_peered":"2024-03-28T13:14:32.070877+0000","last_unstale":"2024-03-28T13:14:34.058780+0000","last_undegraded":"2024-03-28T13:14:34.058780+0000","last_fullsized":"2024-03-28T13:14:34.058780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:38:49.112282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071297+0000","last_change":"2024-03-28T13:14:32.068332+0000","last_active":"2024-03-28T13:14:33.071297+0000","last_peered":"2024-03-28T13:14:33.071297+0000","last_clean":"2024-03-28T13:14:33.071297+0000","last_became_active":"2024-03-28T13:14:32.067416+0000","last_became_peered":"2024-03-28T13:14:32.067416+0000","last_unstale":"2024-03-28T13:14:33.071297+0000","last_undegraded":"2024-03-28T13:14:33.071297+0000","last_fullsized":"2024-03-28T13:14:33.071297+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:12:32.741089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"76'4","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.061470+0000","last_change":"2024-03-28T13:14:31.064033+0000","last_active":"2024-03-28T13:14:34.061470+0000","last_peered":"2024-03-28T13:14:34.061470+0000","last_clean":"2024-03-28T13:14:34.061470+0000","last_became_active":"2024-03-28T13:14:31.063708+0000","last_became_peered":"2024-03-28T13:14:31.063708+0000","last_unstale":"2024-03-28T13:14:34.061470+0000","last_undegraded":"2024-03-28T13:14:34.061470+0000","last_fullsized":"2024-03-28T13:14:34.061470+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:02:36.298761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.071833+0000","last_change":"2024-03-28T13:14:32.062836+0000","last_active":"2024-03-28T13:14:32.071833+0000","last_peered":"2024-03-28T13:14:32.071833+0000","last_clean":"2024-03-28T13:14:32.071833+0000","last_became_active":"2024-03-28T13:14:32.062485+0000","last_became_peered":"2024-03-28T13:14:32.062485+0000","last_unstale":"2024-03-28T13:14:32.071833+0000","last_undegraded":"2024-03-28T13:14:32.071833+0000","last_fullsized":"2024-03-28T13:14:32.071833+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:50:21.514416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.073658+0000","last_change":"2024-03-28T13:14:31.065269+0000","last_active":"2024-03-28T13:14:33.073658+0000","last_peered":"2024-03-28T13:14:33.073658+0000","last_clean":"2024-03-28T13:14:33.073658+0000","last_became_active":"2024-03-28T13:14:31.064858+0000","last_became_peered":"2024-03-28T13:14:31.064858+0000","last_unstale":"2024-03-28T13:14:33.073658+0000","last_undegraded":"2024-03-28T13:14:33.073658+0000","last_fullsized":"2024-03-28T13:14:33.073658+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:02:11.688822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.061822+0000","last_change":"2024-03-28T13:14:32.075328+0000","last_active":"2024-03-28T13:14:34.061822+0000","last_peered":"2024-03-28T13:14:34.061822+0000","last_clean":"2024-03-28T13:14:34.061822+0000","last_became_active":"2024-03-28T13:14:32.075030+0000","last_became_peered":"2024-03-28T13:14:32.075030+0000","last_unstale":"2024-03-28T13:14:34.061822+0000","last_undegraded":"2024-03-28T13:14:34.061822+0000","last_fullsized":"2024-03-28T13:14:34.061822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:51:05.489280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071315+0000","last_change":"2024-03-28T13:14:31.072743+0000","last_active":"2024-03-28T13:14:33.071315+0000","last_peered":"2024-03-28T13:14:33.071315+0000","last_clean":"2024-03-28T13:14:33.071315+0000","last_became_active":"2024-03-28T13:14:31.072467+0000","last_became_peered":"2024-03-28T13:14:31.072467+0000","last_unstale":"2024-03-28T13:14:33.071315+0000","last_undegraded":"2024-03-28T13:14:33.071315+0000","last_fullsized":"2024-03-28T13:14:33.071315+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:20:38.508327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351445+0000","last_change":"2024-03-28T13:14:32.085623+0000","last_active":"2024-03-28T13:14:33.351445+0000","last_peered":"2024-03-28T13:14:33.351445+0000","last_clean":"2024-03-28T13:14:33.351445+0000","last_became_active":"2024-03-28T13:14:32.085346+0000","last_became_peered":"2024-03-28T13:14:32.085346+0000","last_unstale":"2024-03-28T13:14:33.351445+0000","last_undegraded":"2024-03-28T13:14:33.351445+0000","last_fullsized":"2024-03-28T13:14:33.351445+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:55:57.234625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.061906+0000","last_change":"2024-03-28T13:14:31.067452+0000","last_active":"2024-03-28T13:14:34.061906+0000","last_peered":"2024-03-28T13:14:34.061906+0000","last_clean":"2024-03-28T13:14:34.061906+0000","last_became_active":"2024-03-28T13:14:31.067158+0000","last_became_peered":"2024-03-28T13:14:31.067158+0000","last_unstale":"2024-03-28T13:14:34.061906+0000","last_undegraded":"2024-03-28T13:14:34.061906+0000","last_fullsized":"2024-03-28T13:14:34.061906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:02:56.474915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.070861+0000","last_change":"2024-03-28T13:14:32.077291+0000","last_active":"2024-03-28T13:14:33.070861+0000","last_peered":"2024-03-28T13:14:33.070861+0000","last_clean":"2024-03-28T13:14:33.070861+0000","last_became_active":"2024-03-28T13:14:32.076998+0000","last_became_peered":"2024-03-28T13:14:32.076998+0000","last_unstale":"2024-03-28T13:14:33.070861+0000","last_undegraded":"2024-03-28T13:14:33.070861+0000","last_fullsized":"2024-03-28T13:14:33.070861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:49:39.311094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059895+0000","last_change":"2024-03-28T13:14:31.065639+0000","last_active":"2024-03-28T13:14:34.059895+0000","last_peered":"2024-03-28T13:14:34.059895+0000","last_clean":"2024-03-28T13:14:34.059895+0000","last_became_active":"2024-03-28T13:14:31.064089+0000","last_became_peered":"2024-03-28T13:14:31.064089+0000","last_unstale":"2024-03-28T13:14:34.059895+0000","last_undegraded":"2024-03-28T13:14:34.059895+0000","last_fullsized":"2024-03-28T13:14:34.059895+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:02:56.474915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.061690+0000","last_change":"2024-03-28T13:14:32.067061+0000","last_active":"2024-03-28T13:14:34.061690+0000","last_peered":"2024-03-28T13:14:34.061690+0000","last_clean":"2024-03-28T13:14:34.061690+0000","last_became_active":"2024-03-28T13:14:32.066753+0000","last_became_peered":"2024-03-28T13:14:32.066753+0000","last_unstale":"2024-03-28T13:14:34.061690+0000","last_undegraded":"2024-03-28T13:14:34.061690+0000","last_fullsized":"2024-03-28T13:14:34.061690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:07.232388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351098+0000","last_change":"2024-03-28T13:14:31.069586+0000","last_active":"2024-03-28T13:14:33.351098+0000","last_peered":"2024-03-28T13:14:33.351098+0000","last_clean":"2024-03-28T13:14:33.351098+0000","last_became_active":"2024-03-28T13:14:31.068909+0000","last_became_peered":"2024-03-28T13:14:31.068909+0000","last_unstale":"2024-03-28T13:14:33.351098+0000","last_undegraded":"2024-03-28T13:14:33.351098+0000","last_fullsized":"2024-03-28T13:14:33.351098+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:05:48.122751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.074654+0000","last_change":"2024-03-28T13:14:32.071627+0000","last_active":"2024-03-28T13:14:33.074654+0000","last_peered":"2024-03-28T13:14:33.074654+0000","last_clean":"2024-03-28T13:14:33.074654+0000","last_became_active":"2024-03-28T13:14:32.071305+0000","last_became_peered":"2024-03-28T13:14:32.071305+0000","last_unstale":"2024-03-28T13:14:33.074654+0000","last_undegraded":"2024-03-28T13:14:33.074654+0000","last_fullsized":"2024-03-28T13:14:33.074654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:38:49.112282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.075221+0000","last_change":"2024-03-28T13:14:31.060736+0000","last_active":"2024-03-28T13:14:33.075221+0000","last_peered":"2024-03-28T13:14:33.075221+0000","last_clean":"2024-03-28T13:14:33.075221+0000","last_became_active":"2024-03-28T13:14:31.060380+0000","last_became_peered":"2024-03-28T13:14:31.060380+0000","last_unstale":"2024-03-28T13:14:33.075221+0000","last_undegraded":"2024-03-28T13:14:33.075221+0000","last_fullsized":"2024-03-28T13:14:33.075221+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:02:56.474915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059572+0000","last_change":"2024-03-28T13:14:32.086680+0000","last_active":"2024-03-28T13:14:34.059572+0000","last_peered":"2024-03-28T13:14:34.059572+0000","last_clean":"2024-03-28T13:14:34.059572+0000","last_became_active":"2024-03-28T13:14:32.086324+0000","last_became_peered":"2024-03-28T13:14:32.086324+0000","last_unstale":"2024-03-28T13:14:34.059572+0000","last_undegraded":"2024-03-28T13:14:34.059572+0000","last_fullsized":"2024-03-28T13:14:34.059572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:38:49.112282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.058919+0000","last_change":"2024-03-28T13:14:31.065869+0000","last_active":"2024-03-28T13:14:34.058919+0000","last_peered":"2024-03-28T13:14:34.058919+0000","last_clean":"2024-03-28T13:14:34.058919+0000","last_became_active":"2024-03-28T13:14:31.065341+0000","last_became_peered":"2024-03-28T13:14:31.065341+0000","last_unstale":"2024-03-28T13:14:34.058919+0000","last_undegraded":"2024-03-28T13:14:34.058919+0000","last_fullsized":"2024-03-28T13:14:34.058919+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:31:25.306578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.352293+0000","last_change":"2024-03-28T13:14:32.087235+0000","last_active":"2024-03-28T13:14:33.352293+0000","last_peered":"2024-03-28T13:14:33.352293+0000","last_clean":"2024-03-28T13:14:33.352293+0000","last_became_active":"2024-03-28T13:14:32.083492+0000","last_became_peered":"2024-03-28T13:14:32.083492+0000","last_unstale":"2024-03-28T13:14:33.352293+0000","last_undegraded":"2024-03-28T13:14:33.352293+0000","last_fullsized":"2024-03-28T13:14:33.352293+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:02:37.315493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.070946+0000","last_change":"2024-03-28T13:14:31.074537+0000","last_active":"2024-03-28T13:14:33.070946+0000","last_peered":"2024-03-28T13:14:33.070946+0000","last_clean":"2024-03-28T13:14:33.070946+0000","last_became_active":"2024-03-28T13:14:31.074035+0000","last_became_peered":"2024-03-28T13:14:31.074035+0000","last_unstale":"2024-03-28T13:14:33.070946+0000","last_undegraded":"2024-03-28T13:14:33.070946+0000","last_fullsized":"2024-03-28T13:14:33.070946+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:19:23.878452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059817+0000","last_change":"2024-03-28T13:14:32.077047+0000","last_active":"2024-03-28T13:14:34.059817+0000","last_peered":"2024-03-28T13:14:34.059817+0000","last_clean":"2024-03-28T13:14:34.059817+0000","last_became_active":"2024-03-28T13:14:32.073440+0000","last_became_peered":"2024-03-28T13:14:32.073440+0000","last_unstale":"2024-03-28T13:14:34.059817+0000","last_undegraded":"2024-03-28T13:14:34.059817+0000","last_fullsized":"2024-03-28T13:14:34.059817+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:03:49.306204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060158+0000","last_change":"2024-03-28T13:14:31.070699+0000","last_active":"2024-03-28T13:14:34.060158+0000","last_peered":"2024-03-28T13:14:34.060158+0000","last_clean":"2024-03-28T13:14:34.060158+0000","last_became_active":"2024-03-28T13:14:31.069564+0000","last_became_peered":"2024-03-28T13:14:31.069564+0000","last_unstale":"2024-03-28T13:14:34.060158+0000","last_undegraded":"2024-03-28T13:14:34.060158+0000","last_fullsized":"2024-03-28T13:14:34.060158+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:08:43.715141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.070397+0000","last_change":"2024-03-28T13:14:32.072375+0000","last_active":"2024-03-28T13:14:33.070397+0000","last_peered":"2024-03-28T13:14:33.070397+0000","last_clean":"2024-03-28T13:14:33.070397+0000","last_became_active":"2024-03-28T13:14:32.070561+0000","last_became_peered":"2024-03-28T13:14:32.070561+0000","last_unstale":"2024-03-28T13:14:33.070397+0000","last_undegraded":"2024-03-28T13:14:33.070397+0000","last_fullsized":"2024-03-28T13:14:33.070397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:27:03.194232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"76'3","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060131+0000","last_change":"2024-03-28T13:14:31.064949+0000","last_active":"2024-03-28T13:14:34.060131+0000","last_peered":"2024-03-28T13:14:34.060131+0000","last_clean":"2024-03-28T13:14:34.060131+0000","last_became_active":"2024-03-28T13:14:31.064648+0000","last_became_peered":"2024-03-28T13:14:31.064648+0000","last_unstale":"2024-03-28T13:14:34.060131+0000","last_undegraded":"2024-03-28T13:14:34.060131+0000","last_fullsized":"2024-03-28T13:14:34.060131+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:02:36.298761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.073791+0000","last_change":"2024-03-28T13:14:32.079935+0000","last_active":"2024-03-28T13:14:33.073791+0000","last_peered":"2024-03-28T13:14:33.073791+0000","last_clean":"2024-03-28T13:14:33.073791+0000","last_became_active":"2024-03-28T13:14:32.079596+0000","last_became_peered":"2024-03-28T13:14:32.079596+0000","last_unstale":"2024-03-28T13:14:33.073791+0000","last_undegraded":"2024-03-28T13:14:33.073791+0000","last_fullsized":"2024-03-28T13:14:33.073791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:02.366971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"76'6","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.075721+0000","last_change":"2024-03-28T13:14:31.065080+0000","last_active":"2024-03-28T13:14:33.075721+0000","last_peered":"2024-03-28T13:14:33.075721+0000","last_clean":"2024-03-28T13:14:33.075721+0000","last_became_active":"2024-03-28T13:14:31.064811+0000","last_became_peered":"2024-03-28T13:14:31.064811+0000","last_unstale":"2024-03-28T13:14:33.075721+0000","last_undegraded":"2024-03-28T13:14:33.075721+0000","last_fullsized":"2024-03-28T13:14:33.075721+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:02:36.298761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.062837+0000","last_change":"2024-03-28T13:14:32.076640+0000","last_active":"2024-03-28T13:14:34.062837+0000","last_peered":"2024-03-28T13:14:34.062837+0000","last_clean":"2024-03-28T13:14:34.062837+0000","last_became_active":"2024-03-28T13:14:32.076248+0000","last_became_peered":"2024-03-28T13:14:32.076248+0000","last_unstale":"2024-03-28T13:14:34.062837+0000","last_undegraded":"2024-03-28T13:14:34.062837+0000","last_fullsized":"2024-03-28T13:14:34.062837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:03:49.306204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.076056+0000","last_change":"2024-03-28T13:14:31.076277+0000","last_active":"2024-03-28T13:14:33.076056+0000","last_peered":"2024-03-28T13:14:33.076056+0000","last_clean":"2024-03-28T13:14:33.076056+0000","last_became_active":"2024-03-28T13:14:31.075975+0000","last_became_peered":"2024-03-28T13:14:31.075975+0000","last_unstale":"2024-03-28T13:14:33.076056+0000","last_undegraded":"2024-03-28T13:14:33.076056+0000","last_fullsized":"2024-03-28T13:14:33.076056+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:54:15.392118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351602+0000","last_change":"2024-03-28T13:14:32.078492+0000","last_active":"2024-03-28T13:14:33.351602+0000","last_peered":"2024-03-28T13:14:33.351602+0000","last_clean":"2024-03-28T13:14:33.351602+0000","last_became_active":"2024-03-28T13:14:32.078117+0000","last_became_peered":"2024-03-28T13:14:32.078117+0000","last_unstale":"2024-03-28T13:14:33.351602+0000","last_undegraded":"2024-03-28T13:14:33.351602+0000","last_fullsized":"2024-03-28T13:14:33.351602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:51.785029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351256+0000","last_change":"2024-03-28T13:14:31.066069+0000","last_active":"2024-03-28T13:14:33.351256+0000","last_peered":"2024-03-28T13:14:33.351256+0000","last_clean":"2024-03-28T13:14:33.351256+0000","last_became_active":"2024-03-28T13:14:31.065761+0000","last_became_peered":"2024-03-28T13:14:31.065761+0000","last_unstale":"2024-03-28T13:14:33.351256+0000","last_undegraded":"2024-03-28T13:14:33.351256+0000","last_fullsized":"2024-03-28T13:14:33.351256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:52:32.979174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.072531+0000","last_change":"2024-03-28T13:14:32.082547+0000","last_active":"2024-03-28T13:14:33.072531+0000","last_peered":"2024-03-28T13:14:33.072531+0000","last_clean":"2024-03-28T13:14:33.072531+0000","last_became_active":"2024-03-28T13:14:32.082178+0000","last_became_peered":"2024-03-28T13:14:32.082178+0000","last_unstale":"2024-03-28T13:14:33.072531+0000","last_undegraded":"2024-03-28T13:14:33.072531+0000","last_fullsized":"2024-03-28T13:14:33.072531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:51:59.278001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351233+0000","last_change":"2024-03-28T13:14:31.064922+0000","last_active":"2024-03-28T13:14:33.351233+0000","last_peered":"2024-03-28T13:14:33.351233+0000","last_clean":"2024-03-28T13:14:33.351233+0000","last_became_active":"2024-03-28T13:14:31.064550+0000","last_became_peered":"2024-03-28T13:14:31.064550+0000","last_unstale":"2024-03-28T13:14:33.351233+0000","last_undegraded":"2024-03-28T13:14:33.351233+0000","last_fullsized":"2024-03-28T13:14:33.351233+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:49:34.858308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.072022+0000","last_change":"2024-03-28T13:14:32.063187+0000","last_active":"2024-03-28T13:14:32.072022+0000","last_peered":"2024-03-28T13:14:32.072022+0000","last_clean":"2024-03-28T13:14:32.072022+0000","last_became_active":"2024-03-28T13:14:32.059334+0000","last_became_peered":"2024-03-28T13:14:32.059334+0000","last_unstale":"2024-03-28T13:14:32.072022+0000","last_undegraded":"2024-03-28T13:14:32.072022+0000","last_fullsized":"2024-03-28T13:14:32.072022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:23:50.277081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"76'8","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.350916+0000","last_change":"2024-03-28T13:14:31.065268+0000","last_active":"2024-03-28T13:14:33.350916+0000","last_peered":"2024-03-28T13:14:33.350916+0000","last_clean":"2024-03-28T13:14:33.350916+0000","last_became_active":"2024-03-28T13:14:31.064627+0000","last_became_peered":"2024-03-28T13:14:31.064627+0000","last_unstale":"2024-03-28T13:14:33.350916+0000","last_undegraded":"2024-03-28T13:14:33.350916+0000","last_fullsized":"2024-03-28T13:14:33.350916+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:55:57.078531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060014+0000","last_change":"2024-03-28T13:14:32.089562+0000","last_active":"2024-03-28T13:14:34.060014+0000","last_peered":"2024-03-28T13:14:34.060014+0000","last_clean":"2024-03-28T13:14:34.060014+0000","last_became_active":"2024-03-28T13:14:32.088982+0000","last_became_peered":"2024-03-28T13:14:32.088982+0000","last_unstale":"2024-03-28T13:14:34.060014+0000","last_undegraded":"2024-03-28T13:14:34.060014+0000","last_fullsized":"2024-03-28T13:14:34.060014+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:37.922598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071476+0000","last_change":"2024-03-28T13:14:31.076047+0000","last_active":"2024-03-28T13:14:33.071476+0000","last_peered":"2024-03-28T13:14:33.071476+0000","last_clean":"2024-03-28T13:14:33.071476+0000","last_became_active":"2024-03-28T13:14:31.075629+0000","last_became_peered":"2024-03-28T13:14:31.075629+0000","last_unstale":"2024-03-28T13:14:33.071476+0000","last_undegraded":"2024-03-28T13:14:33.071476+0000","last_fullsized":"2024-03-28T13:14:33.071476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:44:56.601232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060103+0000","last_change":"2024-03-28T13:14:32.076254+0000","last_active":"2024-03-28T13:14:34.060103+0000","last_peered":"2024-03-28T13:14:34.060103+0000","last_clean":"2024-03-28T13:14:34.060103+0000","last_became_active":"2024-03-28T13:14:32.075986+0000","last_became_peered":"2024-03-28T13:14:32.075986+0000","last_unstale":"2024-03-28T13:14:34.060103+0000","last_undegraded":"2024-03-28T13:14:34.060103+0000","last_fullsized":"2024-03-28T13:14:34.060103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:02:12.705555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060257+0000","last_change":"2024-03-28T13:14:31.062734+0000","last_active":"2024-03-28T13:14:34.060257+0000","last_peered":"2024-03-28T13:14:34.060257+0000","last_clean":"2024-03-28T13:14:34.060257+0000","last_became_active":"2024-03-28T13:14:31.061798+0000","last_became_peered":"2024-03-28T13:14:31.061798+0000","last_unstale":"2024-03-28T13:14:34.060257+0000","last_undegraded":"2024-03-28T13:14:34.060257+0000","last_fullsized":"2024-03-28T13:14:34.060257+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:31:25.306578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.081679+0000","last_change":"2024-03-28T13:14:32.077657+0000","last_active":"2024-03-28T13:14:32.081679+0000","last_peered":"2024-03-28T13:14:32.081679+0000","last_clean":"2024-03-28T13:14:32.081679+0000","last_became_active":"2024-03-28T13:14:32.077363+0000","last_became_peered":"2024-03-28T13:14:32.077363+0000","last_unstale":"2024-03-28T13:14:32.081679+0000","last_undegraded":"2024-03-28T13:14:32.081679+0000","last_fullsized":"2024-03-28T13:14:32.081679+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:01:24.339090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059811+0000","last_change":"2024-03-28T13:14:31.074229+0000","last_active":"2024-03-28T13:14:34.059811+0000","last_peered":"2024-03-28T13:14:34.059811+0000","last_clean":"2024-03-28T13:14:34.059811+0000","last_became_active":"2024-03-28T13:14:31.073867+0000","last_became_peered":"2024-03-28T13:14:31.073867+0000","last_unstale":"2024-03-28T13:14:34.059811+0000","last_undegraded":"2024-03-28T13:14:34.059811+0000","last_fullsized":"2024-03-28T13:14:34.059811+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:50.768297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.072253+0000","last_change":"2024-03-28T13:14:32.061744+0000","last_active":"2024-03-28T13:14:32.072253+0000","last_peered":"2024-03-28T13:14:32.072253+0000","last_clean":"2024-03-28T13:14:32.072253+0000","last_became_active":"2024-03-28T13:14:32.061009+0000","last_became_peered":"2024-03-28T13:14:32.061009+0000","last_unstale":"2024-03-28T13:14:32.072253+0000","last_undegraded":"2024-03-28T13:14:32.072253+0000","last_fullsized":"2024-03-28T13:14:32.072253+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:28:36.463799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.062016+0000","last_change":"2024-03-28T13:14:31.073410+0000","last_active":"2024-03-28T13:14:34.062016+0000","last_peered":"2024-03-28T13:14:34.062016+0000","last_clean":"2024-03-28T13:14:34.062016+0000","last_became_active":"2024-03-28T13:14:31.072154+0000","last_became_peered":"2024-03-28T13:14:31.072154+0000","last_unstale":"2024-03-28T13:14:34.062016+0000","last_undegraded":"2024-03-28T13:14:34.062016+0000","last_fullsized":"2024-03-28T13:14:34.062016+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:54:15.392118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.075775+0000","last_change":"2024-03-28T13:14:32.071396+0000","last_active":"2024-03-28T13:14:33.075775+0000","last_peered":"2024-03-28T13:14:33.075775+0000","last_clean":"2024-03-28T13:14:33.075775+0000","last_became_active":"2024-03-28T13:14:32.071106+0000","last_became_peered":"2024-03-28T13:14:32.071106+0000","last_unstale":"2024-03-28T13:14:33.075775+0000","last_undegraded":"2024-03-28T13:14:33.075775+0000","last_fullsized":"2024-03-28T13:14:33.075775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:07.232388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.062082+0000","last_change":"2024-03-28T13:14:31.065461+0000","last_active":"2024-03-28T13:14:34.062082+0000","last_peered":"2024-03-28T13:14:34.062082+0000","last_clean":"2024-03-28T13:14:34.062082+0000","last_became_active":"2024-03-28T13:14:31.064893+0000","last_became_peered":"2024-03-28T13:14:31.064893+0000","last_unstale":"2024-03-28T13:14:34.062082+0000","last_undegraded":"2024-03-28T13:14:34.062082+0000","last_fullsized":"2024-03-28T13:14:34.062082+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:16:26.795499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.075018+0000","last_change":"2024-03-28T13:14:32.076674+0000","last_active":"2024-03-28T13:14:33.075018+0000","last_peered":"2024-03-28T13:14:33.075018+0000","last_clean":"2024-03-28T13:14:33.075018+0000","last_became_active":"2024-03-28T13:14:32.076253+0000","last_became_peered":"2024-03-28T13:14:32.076253+0000","last_unstale":"2024-03-28T13:14:33.075018+0000","last_undegraded":"2024-03-28T13:14:33.075018+0000","last_fullsized":"2024-03-28T13:14:33.075018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:13:02.512771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059220+0000","last_change":"2024-03-28T13:14:31.062652+0000","last_active":"2024-03-28T13:14:34.059220+0000","last_peered":"2024-03-28T13:14:34.059220+0000","last_clean":"2024-03-28T13:14:34.059220+0000","last_became_active":"2024-03-28T13:14:31.062344+0000","last_became_peered":"2024-03-28T13:14:31.062344+0000","last_unstale":"2024-03-28T13:14:34.059220+0000","last_undegraded":"2024-03-28T13:14:34.059220+0000","last_fullsized":"2024-03-28T13:14:34.059220+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:13.588796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.061519+0000","last_change":"2024-03-28T13:14:32.073821+0000","last_active":"2024-03-28T13:14:34.061519+0000","last_peered":"2024-03-28T13:14:34.061519+0000","last_clean":"2024-03-28T13:14:34.061519+0000","last_became_active":"2024-03-28T13:14:32.073506+0000","last_became_peered":"2024-03-28T13:14:32.073506+0000","last_unstale":"2024-03-28T13:14:34.061519+0000","last_undegraded":"2024-03-28T13:14:34.061519+0000","last_fullsized":"2024-03-28T13:14:34.061519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:51:05.489280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.073440+0000","last_change":"2024-03-28T13:14:31.065339+0000","last_active":"2024-03-28T13:14:33.073440+0000","last_peered":"2024-03-28T13:14:33.073440+0000","last_clean":"2024-03-28T13:14:33.073440+0000","last_became_active":"2024-03-28T13:14:31.064911+0000","last_became_peered":"2024-03-28T13:14:31.064911+0000","last_unstale":"2024-03-28T13:14:33.073440+0000","last_undegraded":"2024-03-28T13:14:33.073440+0000","last_fullsized":"2024-03-28T13:14:33.073440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:54:11.548347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.073299+0000","last_change":"2024-03-28T13:14:32.085353+0000","last_active":"2024-03-28T13:14:33.073299+0000","last_peered":"2024-03-28T13:14:33.073299+0000","last_clean":"2024-03-28T13:14:33.073299+0000","last_became_active":"2024-03-28T13:14:32.085002+0000","last_became_peered":"2024-03-28T13:14:32.085002+0000","last_unstale":"2024-03-28T13:14:33.073299+0000","last_undegraded":"2024-03-28T13:14:33.073299+0000","last_fullsized":"2024-03-28T13:14:33.073299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:01:29.028951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071652+0000","last_change":"2024-03-28T13:14:31.074316+0000","last_active":"2024-03-28T13:14:33.071652+0000","last_peered":"2024-03-28T13:14:33.071652+0000","last_clean":"2024-03-28T13:14:33.071652+0000","last_became_active":"2024-03-28T13:14:31.073739+0000","last_became_peered":"2024-03-28T13:14:31.073739+0000","last_unstale":"2024-03-28T13:14:33.071652+0000","last_undegraded":"2024-03-28T13:14:33.071652+0000","last_fullsized":"2024-03-28T13:14:33.071652+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:25.449532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.352169+0000","last_change":"2024-03-28T13:14:32.078863+0000","last_active":"2024-03-28T13:14:33.352169+0000","last_peered":"2024-03-28T13:14:33.352169+0000","last_clean":"2024-03-28T13:14:33.352169+0000","last_became_active":"2024-03-28T13:14:32.078574+0000","last_became_peered":"2024-03-28T13:14:32.078574+0000","last_unstale":"2024-03-28T13:14:33.352169+0000","last_undegraded":"2024-03-28T13:14:33.352169+0000","last_fullsized":"2024-03-28T13:14:33.352169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:57:40.546077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"77'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071347+0000","last_change":"2024-03-28T13:14:31.070077+0000","last_active":"2024-03-28T13:14:33.071347+0000","last_peered":"2024-03-28T13:14:33.071347+0000","last_clean":"2024-03-28T13:14:33.071347+0000","last_became_active":"2024-03-28T13:14:31.069793+0000","last_became_peered":"2024-03-28T13:14:31.069793+0000","last_unstale":"2024-03-28T13:14:33.071347+0000","last_undegraded":"2024-03-28T13:14:33.071347+0000","last_fullsized":"2024-03-28T13:14:33.071347+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:05:53.956158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060031+0000","last_change":"2024-03-28T13:14:32.060724+0000","last_active":"2024-03-28T13:14:34.060031+0000","last_peered":"2024-03-28T13:14:34.060031+0000","last_clean":"2024-03-28T13:14:34.060031+0000","last_became_active":"2024-03-28T13:14:32.060456+0000","last_became_peered":"2024-03-28T13:14:32.060456+0000","last_unstale":"2024-03-28T13:14:34.060031+0000","last_undegraded":"2024-03-28T13:14:34.060031+0000","last_fullsized":"2024-03-28T13:14:34.060031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:34:50.268674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060422+0000","last_change":"2024-03-28T13:14:31.068193+0000","last_active":"2024-03-28T13:14:34.060422+0000","last_peered":"2024-03-28T13:14:34.060422+0000","last_clean":"2024-03-28T13:14:34.060422+0000","last_became_active":"2024-03-28T13:14:31.067862+0000","last_became_peered":"2024-03-28T13:14:31.067862+0000","last_unstale":"2024-03-28T13:14:34.060422+0000","last_undegraded":"2024-03-28T13:14:34.060422+0000","last_fullsized":"2024-03-28T13:14:34.060422+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:49:39.687438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351608+0000","last_change":"2024-03-28T13:14:32.072068+0000","last_active":"2024-03-28T13:14:33.351608+0000","last_peered":"2024-03-28T13:14:33.351608+0000","last_clean":"2024-03-28T13:14:33.351608+0000","last_became_active":"2024-03-28T13:14:32.071797+0000","last_became_peered":"2024-03-28T13:14:32.071797+0000","last_unstale":"2024-03-28T13:14:33.351608+0000","last_undegraded":"2024-03-28T13:14:33.351608+0000","last_fullsized":"2024-03-28T13:14:33.351608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:08:44.731874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060785+0000","last_change":"2024-03-28T13:14:31.067793+0000","last_active":"2024-03-28T13:14:34.060785+0000","last_peered":"2024-03-28T13:14:34.060785+0000","last_clean":"2024-03-28T13:14:34.060785+0000","last_became_active":"2024-03-28T13:14:31.067177+0000","last_became_peered":"2024-03-28T13:14:31.067177+0000","last_unstale":"2024-03-28T13:14:34.060785+0000","last_undegraded":"2024-03-28T13:14:34.060785+0000","last_fullsized":"2024-03-28T13:14:34.060785+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:57:39.529345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.072900+0000","last_change":"2024-03-28T13:14:32.078690+0000","last_active":"2024-03-28T13:14:33.072900+0000","last_peered":"2024-03-28T13:14:33.072900+0000","last_clean":"2024-03-28T13:14:33.072900+0000","last_became_active":"2024-03-28T13:14:32.078358+0000","last_became_peered":"2024-03-28T13:14:32.078358+0000","last_unstale":"2024-03-28T13:14:33.072900+0000","last_undegraded":"2024-03-28T13:14:33.072900+0000","last_fullsized":"2024-03-28T13:14:33.072900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:58:03.118568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.061020+0000","last_change":"2024-03-28T13:14:31.058379+0000","last_active":"2024-03-28T13:14:34.061020+0000","last_peered":"2024-03-28T13:14:34.061020+0000","last_clean":"2024-03-28T13:14:34.061020+0000","last_became_active":"2024-03-28T13:14:31.057370+0000","last_became_peered":"2024-03-28T13:14:31.057370+0000","last_unstale":"2024-03-28T13:14:34.061020+0000","last_undegraded":"2024-03-28T13:14:34.061020+0000","last_fullsized":"2024-03-28T13:14:34.061020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:54:15.392118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.070064+0000","last_change":"2024-03-28T13:14:32.062403+0000","last_active":"2024-03-28T13:14:33.070064+0000","last_peered":"2024-03-28T13:14:33.070064+0000","last_clean":"2024-03-28T13:14:33.070064+0000","last_became_active":"2024-03-28T13:14:32.062099+0000","last_became_peered":"2024-03-28T13:14:32.062099+0000","last_unstale":"2024-03-28T13:14:33.070064+0000","last_undegraded":"2024-03-28T13:14:33.070064+0000","last_fullsized":"2024-03-28T13:14:33.070064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:12:11.052471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.073153+0000","last_change":"2024-03-28T13:14:31.067860+0000","last_active":"2024-03-28T13:14:33.073153+0000","last_peered":"2024-03-28T13:14:33.073153+0000","last_clean":"2024-03-28T13:14:33.073153+0000","last_became_active":"2024-03-28T13:14:31.067560+0000","last_became_peered":"2024-03-28T13:14:31.067560+0000","last_unstale":"2024-03-28T13:14:33.073153+0000","last_undegraded":"2024-03-28T13:14:33.073153+0000","last_fullsized":"2024-03-28T13:14:33.073153+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:34:49.251941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.075113+0000","last_change":"2024-03-28T13:14:32.068020+0000","last_active":"2024-03-28T13:14:33.075113+0000","last_peered":"2024-03-28T13:14:33.075113+0000","last_clean":"2024-03-28T13:14:33.075113+0000","last_became_active":"2024-03-28T13:14:32.067680+0000","last_became_peered":"2024-03-28T13:14:32.067680+0000","last_unstale":"2024-03-28T13:14:33.075113+0000","last_undegraded":"2024-03-28T13:14:33.075113+0000","last_fullsized":"2024-03-28T13:14:33.075113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:18.345000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059398+0000","last_change":"2024-03-28T13:14:31.072958+0000","last_active":"2024-03-28T13:14:34.059398+0000","last_peered":"2024-03-28T13:14:34.059398+0000","last_clean":"2024-03-28T13:14:34.059398+0000","last_became_active":"2024-03-28T13:14:31.072289+0000","last_became_peered":"2024-03-28T13:14:31.072289+0000","last_unstale":"2024-03-28T13:14:34.059398+0000","last_undegraded":"2024-03-28T13:14:34.059398+0000","last_fullsized":"2024-03-28T13:14:34.059398+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:53:35.054440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.058930+0000","last_change":"2024-03-28T13:14:32.072121+0000","last_active":"2024-03-28T13:14:34.058930+0000","last_peered":"2024-03-28T13:14:34.058930+0000","last_clean":"2024-03-28T13:14:34.058930+0000","last_became_active":"2024-03-28T13:14:32.071381+0000","last_became_peered":"2024-03-28T13:14:32.071381+0000","last_unstale":"2024-03-28T13:14:34.058930+0000","last_undegraded":"2024-03-28T13:14:34.058930+0000","last_fullsized":"2024-03-28T13:14:34.058930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:13:02.512771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059268+0000","last_change":"2024-03-28T13:14:31.066873+0000","last_active":"2024-03-28T13:14:34.059268+0000","last_peered":"2024-03-28T13:14:34.059268+0000","last_clean":"2024-03-28T13:14:34.059268+0000","last_became_active":"2024-03-28T13:14:31.065469+0000","last_became_peered":"2024-03-28T13:14:31.065469+0000","last_unstale":"2024-03-28T13:14:34.059268+0000","last_undegraded":"2024-03-28T13:14:34.059268+0000","last_fullsized":"2024-03-28T13:14:34.059268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:49:39.687438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.074325+0000","last_change":"2024-03-28T13:14:32.082048+0000","last_active":"2024-03-28T13:14:33.074325+0000","last_peered":"2024-03-28T13:14:33.074325+0000","last_clean":"2024-03-28T13:14:33.074325+0000","last_became_active":"2024-03-28T13:14:32.081621+0000","last_became_peered":"2024-03-28T13:14:32.081621+0000","last_unstale":"2024-03-28T13:14:33.074325+0000","last_undegraded":"2024-03-28T13:14:33.074325+0000","last_fullsized":"2024-03-28T13:14:33.074325+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:05:54.972890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351660+0000","last_change":"2024-03-28T13:14:31.065484+0000","last_active":"2024-03-28T13:14:33.351660+0000","last_peered":"2024-03-28T13:14:33.351660+0000","last_clean":"2024-03-28T13:14:33.351660+0000","last_became_active":"2024-03-28T13:14:31.065151+0000","last_became_peered":"2024-03-28T13:14:31.065151+0000","last_unstale":"2024-03-28T13:14:33.351660+0000","last_undegraded":"2024-03-28T13:14:33.351660+0000","last_fullsized":"2024-03-28T13:14:33.351660+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:49:08.890119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":270,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351748+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:14:33.351748+0000","last_peered":"2024-03-28T13:14:33.351748+0000","last_clean":"2024-03-28T13:14:33.351748+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:14:33.351748+0000","last_undegraded":"2024-03-28T13:14:33.351748+0000","last_fullsized":"2024-03-28T13:14:33.351748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476803,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":937984,"data_stored":392833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476801,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":929792,"data_stored":393289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476801,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1519616,"data_stored":983075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476801,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":929792,"data_stored":392777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509504,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":394353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509505,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1523712,"data_stored":983165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509505,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":925696,"data_stored":387101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":925696,"data_stored":388019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:14:36.603 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:14:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.853+0000 7f8793e00640 1 Processor -- start 2024-03-28T13:14:36.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.853+0000 7f8793e00640 1 -- start start 2024-03-28T13:14:36.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.855+0000 7f8793e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f878c15aa90 0x7f878c15b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.855+0000 7f8793e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f878c15bee0 con 0x7f878c0717f0 2024-03-28T13:14:36.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.855+0000 7f8793e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f878c0b3910 con 0x7f878c072100 2024-03-28T13:14:36.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.855+0000 7f8793e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f878c0b5190 con 0x7f878c15aa90 2024-03-28T13:14:36.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.855+0000 7f8790a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f878c0717f0 0x7f878c0b46d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48174/0 (socket says 172.21.15.19:48174) 2024-03-28T13:14:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.855+0000 7f8790a00640 1 -- 172.21.15.19:0/1511219390 learned_addr learned my addr 172.21.15.19:0/1511219390 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.856+0000 7f8791400640 1 --2- 172.21.15.19:0/1511219390 >> v2:172.21.15.188:3300/0 conn(0x7f878c15aa90 0x7f878c15b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.856+0000 7f878be00640 1 -- 172.21.15.19:0/1511219390 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3356889812 0 0) 0x7f8778001110 con 0x7f878c0717f0 2024-03-28T13:14:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.856+0000 7f878be00640 1 -- 172.21.15.19:0/1511219390 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8774003440 con 0x7f878c0717f0 2024-03-28T13:14:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.856+0000 7f878be00640 1 -- 172.21.15.19:0/1511219390 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4060281696 0 0) 0x7f8774003440 con 0x7f878c0717f0 2024-03-28T13:14:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.856+0000 7f878be00640 1 -- 172.21.15.19:0/1511219390 >> v2:172.21.15.188:3300/0 conn(0x7f878c15aa90 msgr2=0x7f878c15b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.856+0000 7f878be00640 1 --2- 172.21.15.19:0/1511219390 >> v2:172.21.15.188:3300/0 conn(0x7f878c15aa90 0x7f878c15b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.856+0000 7f878be00640 1 -- 172.21.15.19:0/1511219390 >> v1:172.21.15.188:6789/0 conn(0x7f878c072100 legacy=0x7f878c072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.856+0000 7f878be00640 1 -- 172.21.15.19:0/1511219390 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878c0b3bf0 con 0x7f878c0717f0 2024-03-28T13:14:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.856+0000 7f878be00640 1 -- 172.21.15.19:0/1511219390 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f8778001840 con 0x7f878c0717f0 2024-03-28T13:14:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.856+0000 7f878be00640 1 -- 172.21.15.19:0/1511219390 >> v1:172.21.15.19:6789/0 conn(0x7f878c0717f0 legacy=0x7f878c0b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.857+0000 7f878be00640 1 --2- 172.21.15.19:0/1511219390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8774003af0 0x7f8774003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.857+0000 7f878be00640 1 --2- 172.21.15.19:0/1511219390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87740088a0 0x7f877400ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.857+0000 7f878be00640 1 --2- 172.21.15.19:0/1511219390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f877400b510 0x7f877400d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.857+0000 7f878be00640 1 -- 172.21.15.19:0/1511219390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f878c0b5190 con 0x7f87740088a0 2024-03-28T13:14:36.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.857+0000 7f878be00640 1 -- 172.21.15.19:0/1511219390 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f877400e2b0 con 0x7f877400b510 2024-03-28T13:14:36.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.857+0000 7f878be00640 1 -- 172.21.15.19:0/1511219390 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f877400e4b0 con 0x7f8774003af0 2024-03-28T13:14:36.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.857+0000 7f8790a00640 1 --2- 172.21.15.19:0/1511219390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87740088a0 0x7f877400ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:36.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.857+0000 7f8791400640 1 --2- 172.21.15.19:0/1511219390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8774003af0 0x7f8774003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:36.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.857+0000 7f8791e00640 1 --2- 172.21.15.19:0/1511219390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f877400b510 0x7f877400d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:36.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.858+0000 7f8790a00640 1 -- 172.21.15.19:0/1511219390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8774003af0 msgr2=0x7f8774003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:36.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.858+0000 7f8790a00640 1 --2- 172.21.15.19:0/1511219390 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8774003af0 0x7f8774003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.858+0000 7f8790a00640 1 -- 172.21.15.19:0/1511219390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f877400b510 msgr2=0x7f877400d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:36.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.858+0000 7f8790a00640 1 --2- 172.21.15.19:0/1511219390 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f877400b510 0x7f877400d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.858+0000 7f8790a00640 1 -- 172.21.15.19:0/1511219390 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8774003440 con 0x7f87740088a0 2024-03-28T13:14:36.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.858+0000 7f8791e00640 1 -- 172.21.15.19:0/1511219390 reap_dead start 2024-03-28T13:14:36.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.859+0000 7f8790a00640 1 --2- 172.21.15.19:0/1511219390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87740088a0 0x7f877400ad10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f877800c2f0 tx=0x7f8774003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee77197be09ba6b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:36.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.860+0000 7f878be00640 1 -- 172.21.15.19:0/1511219390 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778007e00 con 0x7f87740088a0 2024-03-28T13:14:36.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.860+0000 7f878be00640 1 -- 172.21.15.19:0/1511219390 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8778030070 con 0x7f87740088a0 2024-03-28T13:14:36.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.860+0000 7f878be00640 1 -- 172.21.15.19:0/1511219390 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877802ce30 con 0x7f87740088a0 2024-03-28T13:14:36.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.860+0000 7f8793e00640 1 -- 172.21.15.19:0/1511219390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87740088a0 msgr2=0x7f877400ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:36.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.860+0000 7f8793e00640 1 --2- 172.21.15.19:0/1511219390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87740088a0 0x7f877400ad10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f877800c2f0 tx=0x7f8774003730 comp rx=0 tx=0).stop 2024-03-28T13:14:36.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.860+0000 7f8793e00640 1 -- 172.21.15.19:0/1511219390 shutdown_connections 2024-03-28T13:14:36.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.860+0000 7f8793e00640 1 --2- 172.21.15.19:0/1511219390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87740088a0 0x7f877400ad10 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.860+0000 7f8793e00640 1 -- 172.21.15.19:0/1511219390 >> 172.21.15.19:0/1511219390 conn(0x7f878c06b990 msgr2=0x7f878c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:36.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.860+0000 7f8793e00640 1 -- 172.21.15.19:0/1511219390 shutdown_connections 2024-03-28T13:14:36.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.860+0000 7f8793e00640 1 -- 172.21.15.19:0/1511219390 wait complete. 2024-03-28T13:14:36.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.863+0000 7f8793e00640 1 Processor -- start 2024-03-28T13:14:36.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.864+0000 7f8793e00640 1 -- start start 2024-03-28T13:14:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.865+0000 7f8793e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87740088a0 0x7f878c147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.865+0000 7f8793e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f878c15c460 0x7f878c15c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.865+0000 7f8793e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f878c0caa30 0x7f878c0caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.865+0000 7f8793e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f878c071cc0 con 0x7f878c15c460 2024-03-28T13:14:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.865+0000 7f8793e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f878c0b4240 con 0x7f878c0caa30 2024-03-28T13:14:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.865+0000 7f8793e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f878c15b9a0 con 0x7f87740088a0 2024-03-28T13:14:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.865+0000 7f8790a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f878c15c460 0x7f878c15c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.866+0000 7f8790a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f878c15c460 0x7f878c15c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48084/0 (socket says 172.21.15.19:48084) 2024-03-28T13:14:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.866+0000 7f8790a00640 1 -- 172.21.15.19:0/1619743292 learned_addr learned my addr 172.21.15.19:0/1619743292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.866+0000 7f8791400640 1 --2- 172.21.15.19:0/1619743292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87740088a0 0x7f878c147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:36.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.866+0000 7f8791e00640 1 --2- 172.21.15.19:0/1619743292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f878c0caa30 0x7f878c0caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.866+0000 7f8790a00640 1 -- 172.21.15.19:0/1619743292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87740088a0 msgr2=0x7f878c147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.866+0000 7f8790a00640 1 --2- 172.21.15.19:0/1619743292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87740088a0 0x7f878c147c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.866+0000 7f8790a00640 1 -- 172.21.15.19:0/1619743292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f878c0caa30 msgr2=0x7f878c0caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.866+0000 7f8790a00640 1 --2- 172.21.15.19:0/1619743292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f878c0caa30 0x7f878c0caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.866+0000 7f8790a00640 1 -- 172.21.15.19:0/1619743292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878c0cb3f0 con 0x7f878c15c460 2024-03-28T13:14:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.867+0000 7f8790a00640 1 --2- 172.21.15.19:0/1619743292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f878c15c460 0x7f878c15c8c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f877800c410 tx=0x7f878c0b3ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.867+0000 7f8789600640 1 -- 172.21.15.19:0/1619743292 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778004bf0 con 0x7f878c15c460 2024-03-28T13:14:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.867+0000 7f8789600640 1 -- 172.21.15.19:0/1619743292 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8778004d90 con 0x7f878c15c460 2024-03-28T13:14:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.867+0000 7f8789600640 1 -- 172.21.15.19:0/1619743292 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8778007290 con 0x7f878c15c460 2024-03-28T13:14:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.867+0000 7f8793e00640 1 -- 172.21.15.19:0/1619743292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878c0cb680 con 0x7f878c15c460 2024-03-28T13:14:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.867+0000 7f8793e00640 1 -- 172.21.15.19:0/1619743292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878c0cbc40 con 0x7f878c15c460 2024-03-28T13:14:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.868+0000 7f8793e00640 1 -- 172.21.15.19:0/1619743292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f878c15ce00 con 0x7f878c15c460 2024-03-28T13:14:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.870+0000 7f8789600640 1 -- 172.21.15.19:0/1619743292 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8778048020 con 0x7f878c15c460 2024-03-28T13:14:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.871+0000 7f8789600640 1 --2- 172.21.15.19:0/1619743292 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f876c06c530 0x7f876c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.871+0000 7f8789600640 1 -- 172.21.15.19:0/1619743292 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f87780acaf0 con 0x7f878c15c460 2024-03-28T13:14:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.872+0000 7f8791400640 1 --2- 172.21.15.19:0/1619743292 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f876c06c530 0x7f876c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.873+0000 7f8791400640 1 --2- 172.21.15.19:0/1619743292 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f876c06c530 0x7f876c06e9f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f878c06a480 tx=0x7f8780008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:36.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:36.876+0000 7f8789600640 1 -- 172.21.15.19:0/1619743292 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f878c15bee0 con 0x7f878c15c460 2024-03-28T13:14:36.998 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-03-28T13:14:36.998+0000 7fae96200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:14:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.075+0000 7f8793e00640 1 -- 172.21.15.19:0/1619743292 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f878c0685c0 con 0x7f876c06c530 2024-03-28T13:14:37.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.094+0000 7f8789600640 1 -- 172.21.15.19:0/1619743292 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252062 (secure 0 0 0) 0x7f878c0685c0 con 0x7f876c06c530 2024-03-28T13:14:37.098 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:37.099 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:14:37.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.101+0000 7f8793e00640 1 -- 172.21.15.19:0/1619743292 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f876c06c530 msgr2=0x7f876c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:37.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.101+0000 7f8793e00640 1 --2- 172.21.15.19:0/1619743292 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f876c06c530 0x7f876c06e9f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f878c06a480 tx=0x7f8780008040 comp rx=0 tx=0).stop 2024-03-28T13:14:37.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.101+0000 7f8793e00640 1 -- 172.21.15.19:0/1619743292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f878c15c460 msgr2=0x7f878c15c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:37.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.101+0000 7f8793e00640 1 --2- 172.21.15.19:0/1619743292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f878c15c460 0x7f878c15c8c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f877800c410 tx=0x7f878c0b3ae0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.101+0000 7f8793e00640 1 -- 172.21.15.19:0/1619743292 shutdown_connections 2024-03-28T13:14:37.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.101+0000 7f8793e00640 1 --2- 172.21.15.19:0/1619743292 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f876c06c530 0x7f876c06e9f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.101+0000 7f8793e00640 1 --2- 172.21.15.19:0/1619743292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f878c0caa30 0x7f878c0caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.101+0000 7f8793e00640 1 --2- 172.21.15.19:0/1619743292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f878c15c460 0x7f878c15c8c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.101+0000 7f8793e00640 1 --2- 172.21.15.19:0/1619743292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87740088a0 0x7f878c147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.101+0000 7f8793e00640 1 -- 172.21.15.19:0/1619743292 >> 172.21.15.19:0/1619743292 conn(0x7f878c06b990 msgr2=0x7f878c06bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:37.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.101+0000 7f8793e00640 1 -- 172.21.15.19:0/1619743292 shutdown_connections 2024-03-28T13:14:37.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.101+0000 7f8793e00640 1 -- 172.21.15.19:0/1619743292 wait complete. 2024-03-28T13:14:37.123 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-03-28T13:14:36.092766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257900,"kb_used_data":8432,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716820,"statfs":{"total":773094113280,"available":772830023680,"internally_reserved":0,"allocated":8634368,"data_stored":4314612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001925"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071743+0000","last_change":"2024-03-28T13:14:32.086179+0000","last_active":"2024-03-28T13:14:33.071743+0000","last_peered":"2024-03-28T13:14:33.071743+0000","last_clean":"2024-03-28T13:14:33.071743+0000","last_became_active":"2024-03-28T13:14:32.085893+0000","last_became_peered":"2024-03-28T13:14:32.085893+0000","last_unstale":"2024-03-28T13:14:33.071743+0000","last_undegraded":"2024-03-28T13:14:33.071743+0000","last_fullsized":"2024-03-28T13:14:33.071743+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:38:26.383672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059640+0000","last_change":"2024-03-28T13:14:32.081691+0000","last_active":"2024-03-28T13:14:34.059640+0000","last_peered":"2024-03-28T13:14:34.059640+0000","last_clean":"2024-03-28T13:14:34.059640+0000","last_became_active":"2024-03-28T13:14:32.081401+0000","last_became_peered":"2024-03-28T13:14:32.081401+0000","last_unstale":"2024-03-28T13:14:34.059640+0000","last_undegraded":"2024-03-28T13:14:34.059640+0000","last_fullsized":"2024-03-28T13:14:34.059640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:51:05.489280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.072572+0000","last_change":"2024-03-28T13:14:32.068576+0000","last_active":"2024-03-28T13:14:32.072572+0000","last_peered":"2024-03-28T13:14:32.072572+0000","last_clean":"2024-03-28T13:14:32.072572+0000","last_became_active":"2024-03-28T13:14:32.068252+0000","last_became_peered":"2024-03-28T13:14:32.068252+0000","last_unstale":"2024-03-28T13:14:32.072572+0000","last_undegraded":"2024-03-28T13:14:32.072572+0000","last_fullsized":"2024-03-28T13:14:32.072572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:58.024115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.079244+0000","last_change":"2024-03-28T13:14:32.072558+0000","last_active":"2024-03-28T13:14:32.079244+0000","last_peered":"2024-03-28T13:14:32.079244+0000","last_clean":"2024-03-28T13:14:32.079244+0000","last_became_active":"2024-03-28T13:14:32.072269+0000","last_became_peered":"2024-03-28T13:14:32.072269+0000","last_unstale":"2024-03-28T13:14:32.079244+0000","last_undegraded":"2024-03-28T13:14:32.079244+0000","last_fullsized":"2024-03-28T13:14:32.079244+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:43:26.658980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.072854+0000","last_change":"2024-03-28T13:14:32.080653+0000","last_active":"2024-03-28T13:14:33.072854+0000","last_peered":"2024-03-28T13:14:33.072854+0000","last_clean":"2024-03-28T13:14:33.072854+0000","last_became_active":"2024-03-28T13:14:32.078004+0000","last_became_peered":"2024-03-28T13:14:32.078004+0000","last_unstale":"2024-03-28T13:14:33.072854+0000","last_undegraded":"2024-03-28T13:14:33.072854+0000","last_fullsized":"2024-03-28T13:14:33.072854+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:13:02.556681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060070+0000","last_change":"2024-03-28T13:14:32.061384+0000","last_active":"2024-03-28T13:14:34.060070+0000","last_peered":"2024-03-28T13:14:34.060070+0000","last_clean":"2024-03-28T13:14:34.060070+0000","last_became_active":"2024-03-28T13:14:32.058089+0000","last_became_peered":"2024-03-28T13:14:32.058089+0000","last_unstale":"2024-03-28T13:14:34.060070+0000","last_undegraded":"2024-03-28T13:14:34.060070+0000","last_fullsized":"2024-03-28T13:14:34.060070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:13:02.512771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351156+0000","last_change":"2024-03-28T13:14:32.088103+0000","last_active":"2024-03-28T13:14:33.351156+0000","last_peered":"2024-03-28T13:14:33.351156+0000","last_clean":"2024-03-28T13:14:33.351156+0000","last_became_active":"2024-03-28T13:14:32.087761+0000","last_became_peered":"2024-03-28T13:14:32.087761+0000","last_unstale":"2024-03-28T13:14:33.351156+0000","last_undegraded":"2024-03-28T13:14:33.351156+0000","last_fullsized":"2024-03-28T13:14:33.351156+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:14.605529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.069742+0000","last_change":"2024-03-28T13:14:32.076227+0000","last_active":"2024-03-28T13:14:33.069742+0000","last_peered":"2024-03-28T13:14:33.069742+0000","last_clean":"2024-03-28T13:14:33.069742+0000","last_became_active":"2024-03-28T13:14:32.075917+0000","last_became_peered":"2024-03-28T13:14:32.075917+0000","last_unstale":"2024-03-28T13:14:33.069742+0000","last_undegraded":"2024-03-28T13:14:33.069742+0000","last_fullsized":"2024-03-28T13:14:33.069742+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:51:38.097780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.082358+0000","last_change":"2024-03-28T13:14:32.077975+0000","last_active":"2024-03-28T13:14:32.082358+0000","last_peered":"2024-03-28T13:14:32.082358+0000","last_clean":"2024-03-28T13:14:32.082358+0000","last_became_active":"2024-03-28T13:14:32.077659+0000","last_became_peered":"2024-03-28T13:14:32.077659+0000","last_unstale":"2024-03-28T13:14:32.082358+0000","last_undegraded":"2024-03-28T13:14:32.082358+0000","last_fullsized":"2024-03-28T13:14:32.082358+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:56:27.147994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.352143+0000","last_change":"2024-03-28T13:14:32.088520+0000","last_active":"2024-03-28T13:14:33.352143+0000","last_peered":"2024-03-28T13:14:33.352143+0000","last_clean":"2024-03-28T13:14:33.352143+0000","last_became_active":"2024-03-28T13:14:32.087092+0000","last_became_peered":"2024-03-28T13:14:32.087092+0000","last_unstale":"2024-03-28T13:14:33.352143+0000","last_undegraded":"2024-03-28T13:14:33.352143+0000","last_fullsized":"2024-03-28T13:14:33.352143+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:49:40.704170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071258+0000","last_change":"2024-03-28T13:14:32.076370+0000","last_active":"2024-03-28T13:14:33.071258+0000","last_peered":"2024-03-28T13:14:33.071258+0000","last_clean":"2024-03-28T13:14:33.071258+0000","last_became_active":"2024-03-28T13:14:32.076064+0000","last_became_peered":"2024-03-28T13:14:32.076064+0000","last_unstale":"2024-03-28T13:14:33.071258+0000","last_undegraded":"2024-03-28T13:14:33.071258+0000","last_fullsized":"2024-03-28T13:14:33.071258+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:15:13.865834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060521+0000","last_change":"2024-03-28T13:14:32.083698+0000","last_active":"2024-03-28T13:14:34.060521+0000","last_peered":"2024-03-28T13:14:34.060521+0000","last_clean":"2024-03-28T13:14:34.060521+0000","last_became_active":"2024-03-28T13:14:32.083098+0000","last_became_peered":"2024-03-28T13:14:32.083098+0000","last_unstale":"2024-03-28T13:14:34.060521+0000","last_undegraded":"2024-03-28T13:14:34.060521+0000","last_fullsized":"2024-03-28T13:14:34.060521+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:07.232388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.087368+0000","last_change":"2024-03-28T13:14:32.084201+0000","last_active":"2024-03-28T13:14:32.087368+0000","last_peered":"2024-03-28T13:14:32.087368+0000","last_clean":"2024-03-28T13:14:32.087368+0000","last_became_active":"2024-03-28T13:14:32.083907+0000","last_became_peered":"2024-03-28T13:14:32.083907+0000","last_unstale":"2024-03-28T13:14:32.087368+0000","last_undegraded":"2024-03-28T13:14:32.087368+0000","last_fullsized":"2024-03-28T13:14:32.087368+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:42:16.554792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059944+0000","last_change":"2024-03-28T13:14:32.082254+0000","last_active":"2024-03-28T13:14:34.059944+0000","last_peered":"2024-03-28T13:14:34.059944+0000","last_clean":"2024-03-28T13:14:34.059944+0000","last_became_active":"2024-03-28T13:14:32.081983+0000","last_became_peered":"2024-03-28T13:14:32.081983+0000","last_unstale":"2024-03-28T13:14:34.059944+0000","last_undegraded":"2024-03-28T13:14:34.059944+0000","last_fullsized":"2024-03-28T13:14:34.059944+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:13:02.512771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060114+0000","last_change":"2024-03-28T13:14:32.076292+0000","last_active":"2024-03-28T13:14:34.060114+0000","last_peered":"2024-03-28T13:14:34.060114+0000","last_clean":"2024-03-28T13:14:34.060114+0000","last_became_active":"2024-03-28T13:14:32.075976+0000","last_became_peered":"2024-03-28T13:14:32.075976+0000","last_unstale":"2024-03-28T13:14:34.060114+0000","last_undegraded":"2024-03-28T13:14:34.060114+0000","last_fullsized":"2024-03-28T13:14:34.060114+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:18.345000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.069756+0000","last_change":"2024-03-28T13:14:32.060377+0000","last_active":"2024-03-28T13:14:33.069756+0000","last_peered":"2024-03-28T13:14:33.069756+0000","last_clean":"2024-03-28T13:14:33.069756+0000","last_became_active":"2024-03-28T13:14:32.059330+0000","last_became_peered":"2024-03-28T13:14:32.059330+0000","last_unstale":"2024-03-28T13:14:33.069756+0000","last_undegraded":"2024-03-28T13:14:33.069756+0000","last_fullsized":"2024-03-28T13:14:33.069756+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:11:41.183649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.074029+0000","last_change":"2024-03-28T13:14:32.083257+0000","last_active":"2024-03-28T13:14:33.074029+0000","last_peered":"2024-03-28T13:14:33.074029+0000","last_clean":"2024-03-28T13:14:33.074029+0000","last_became_active":"2024-03-28T13:14:32.082865+0000","last_became_peered":"2024-03-28T13:14:32.082865+0000","last_unstale":"2024-03-28T13:14:33.074029+0000","last_undegraded":"2024-03-28T13:14:33.074029+0000","last_fullsized":"2024-03-28T13:14:33.074029+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:20:39.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.074011+0000","last_change":"2024-03-28T13:14:32.092110+0000","last_active":"2024-03-28T13:14:33.074011+0000","last_peered":"2024-03-28T13:14:33.074011+0000","last_clean":"2024-03-28T13:14:33.074011+0000","last_became_active":"2024-03-28T13:14:32.091778+0000","last_became_peered":"2024-03-28T13:14:32.091778+0000","last_unstale":"2024-03-28T13:14:33.074011+0000","last_undegraded":"2024-03-28T13:14:33.074011+0000","last_fullsized":"2024-03-28T13:14:33.074011+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:13:42.425225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071088+0000","last_change":"2024-03-28T13:14:32.058245+0000","last_active":"2024-03-28T13:14:33.071088+0000","last_peered":"2024-03-28T13:14:33.071088+0000","last_clean":"2024-03-28T13:14:33.071088+0000","last_became_active":"2024-03-28T13:14:32.057857+0000","last_became_peered":"2024-03-28T13:14:32.057857+0000","last_unstale":"2024-03-28T13:14:33.071088+0000","last_undegraded":"2024-03-28T13:14:33.071088+0000","last_fullsized":"2024-03-28T13:14:33.071088+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:25:34.731536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.073300+0000","last_change":"2024-03-28T13:14:32.086692+0000","last_active":"2024-03-28T13:14:33.073300+0000","last_peered":"2024-03-28T13:14:33.073300+0000","last_clean":"2024-03-28T13:14:33.073300+0000","last_became_active":"2024-03-28T13:14:32.084876+0000","last_became_peered":"2024-03-28T13:14:32.084876+0000","last_unstale":"2024-03-28T13:14:33.073300+0000","last_undegraded":"2024-03-28T13:14:33.073300+0000","last_fullsized":"2024-03-28T13:14:33.073300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:40:05.574117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059044+0000","last_change":"2024-03-28T13:14:32.073579+0000","last_active":"2024-03-28T13:14:34.059044+0000","last_peered":"2024-03-28T13:14:34.059044+0000","last_clean":"2024-03-28T13:14:34.059044+0000","last_became_active":"2024-03-28T13:14:32.073300+0000","last_became_peered":"2024-03-28T13:14:32.073300+0000","last_unstale":"2024-03-28T13:14:34.059044+0000","last_undegraded":"2024-03-28T13:14:34.059044+0000","last_fullsized":"2024-03-28T13:14:34.059044+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:18.345000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.061332+0000","last_change":"2024-03-28T13:14:32.077547+0000","last_active":"2024-03-28T13:14:34.061332+0000","last_peered":"2024-03-28T13:14:34.061332+0000","last_clean":"2024-03-28T13:14:34.061332+0000","last_became_active":"2024-03-28T13:14:32.075519+0000","last_became_peered":"2024-03-28T13:14:32.075519+0000","last_unstale":"2024-03-28T13:14:34.061332+0000","last_undegraded":"2024-03-28T13:14:34.061332+0000","last_fullsized":"2024-03-28T13:14:34.061332+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:18.345000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351266+0000","last_change":"2024-03-28T13:14:32.087926+0000","last_active":"2024-03-28T13:14:33.351266+0000","last_peered":"2024-03-28T13:14:33.351266+0000","last_clean":"2024-03-28T13:14:33.351266+0000","last_became_active":"2024-03-28T13:14:32.087454+0000","last_became_peered":"2024-03-28T13:14:32.087454+0000","last_unstale":"2024-03-28T13:14:33.351266+0000","last_undegraded":"2024-03-28T13:14:33.351266+0000","last_fullsized":"2024-03-28T13:14:33.351266+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:53:36.071173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.069858+0000","last_change":"2024-03-28T13:14:32.055538+0000","last_active":"2024-03-28T13:14:33.069858+0000","last_peered":"2024-03-28T13:14:33.069858+0000","last_clean":"2024-03-28T13:14:33.069858+0000","last_became_active":"2024-03-28T13:14:32.055203+0000","last_became_peered":"2024-03-28T13:14:32.055203+0000","last_unstale":"2024-03-28T13:14:33.069858+0000","last_undegraded":"2024-03-28T13:14:33.069858+0000","last_fullsized":"2024-03-28T13:14:33.069858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:23:14.381420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351899+0000","last_change":"2024-03-28T13:14:32.085509+0000","last_active":"2024-03-28T13:14:33.351899+0000","last_peered":"2024-03-28T13:14:33.351899+0000","last_clean":"2024-03-28T13:14:33.351899+0000","last_became_active":"2024-03-28T13:14:32.084987+0000","last_became_peered":"2024-03-28T13:14:32.084987+0000","last_unstale":"2024-03-28T13:14:33.351899+0000","last_undegraded":"2024-03-28T13:14:33.351899+0000","last_fullsized":"2024-03-28T13:14:33.351899+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:31:26.323310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071201+0000","last_change":"2024-03-28T13:14:32.078943+0000","last_active":"2024-03-28T13:14:33.071201+0000","last_peered":"2024-03-28T13:14:33.071201+0000","last_clean":"2024-03-28T13:14:33.071201+0000","last_became_active":"2024-03-28T13:14:32.078592+0000","last_became_peered":"2024-03-28T13:14:32.078592+0000","last_unstale":"2024-03-28T13:14:33.071201+0000","last_undegraded":"2024-03-28T13:14:33.071201+0000","last_fullsized":"2024-03-28T13:14:33.071201+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:15:08.895008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.073473+0000","last_change":"2024-03-28T13:14:32.086509+0000","last_active":"2024-03-28T13:14:33.073473+0000","last_peered":"2024-03-28T13:14:33.073473+0000","last_clean":"2024-03-28T13:14:33.073473+0000","last_became_active":"2024-03-28T13:14:32.085203+0000","last_became_peered":"2024-03-28T13:14:32.085203+0000","last_unstale":"2024-03-28T13:14:33.073473+0000","last_undegraded":"2024-03-28T13:14:33.073473+0000","last_fullsized":"2024-03-28T13:14:33.073473+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:37:48.508041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059488+0000","last_change":"2024-03-28T13:14:32.071537+0000","last_active":"2024-03-28T13:14:34.059488+0000","last_peered":"2024-03-28T13:14:34.059488+0000","last_clean":"2024-03-28T13:14:34.059488+0000","last_became_active":"2024-03-28T13:14:32.071228+0000","last_became_peered":"2024-03-28T13:14:32.071228+0000","last_unstale":"2024-03-28T13:14:34.059488+0000","last_undegraded":"2024-03-28T13:14:34.059488+0000","last_fullsized":"2024-03-28T13:14:34.059488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:07.232388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.072402+0000","last_change":"2024-03-28T13:14:32.065606+0000","last_active":"2024-03-28T13:14:32.072402+0000","last_peered":"2024-03-28T13:14:32.072402+0000","last_clean":"2024-03-28T13:14:32.072402+0000","last_became_active":"2024-03-28T13:14:32.065296+0000","last_became_peered":"2024-03-28T13:14:32.065296+0000","last_unstale":"2024-03-28T13:14:32.072402+0000","last_undegraded":"2024-03-28T13:14:32.072402+0000","last_fullsized":"2024-03-28T13:14:32.072402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:50:02.228041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.070729+0000","last_change":"2024-03-28T13:14:32.057766+0000","last_active":"2024-03-28T13:14:33.070729+0000","last_peered":"2024-03-28T13:14:33.070729+0000","last_clean":"2024-03-28T13:14:33.070729+0000","last_became_active":"2024-03-28T13:14:32.057492+0000","last_became_peered":"2024-03-28T13:14:32.057492+0000","last_unstale":"2024-03-28T13:14:33.070729+0000","last_undegraded":"2024-03-28T13:14:33.070729+0000","last_fullsized":"2024-03-28T13:14:33.070729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:31:36.795818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060520+0000","last_change":"2024-03-28T13:14:32.078346+0000","last_active":"2024-03-28T13:14:34.060520+0000","last_peered":"2024-03-28T13:14:34.060520+0000","last_clean":"2024-03-28T13:14:34.060520+0000","last_became_active":"2024-03-28T13:14:32.078001+0000","last_became_peered":"2024-03-28T13:14:32.078001+0000","last_unstale":"2024-03-28T13:14:34.060520+0000","last_undegraded":"2024-03-28T13:14:34.060520+0000","last_fullsized":"2024-03-28T13:14:34.060520+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:54:12.565079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.058780+0000","last_change":"2024-03-28T13:14:32.072888+0000","last_active":"2024-03-28T13:14:34.058780+0000","last_peered":"2024-03-28T13:14:34.058780+0000","last_clean":"2024-03-28T13:14:34.058780+0000","last_became_active":"2024-03-28T13:14:32.070877+0000","last_became_peered":"2024-03-28T13:14:32.070877+0000","last_unstale":"2024-03-28T13:14:34.058780+0000","last_undegraded":"2024-03-28T13:14:34.058780+0000","last_fullsized":"2024-03-28T13:14:34.058780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:38:49.112282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071297+0000","last_change":"2024-03-28T13:14:32.068332+0000","last_active":"2024-03-28T13:14:33.071297+0000","last_peered":"2024-03-28T13:14:33.071297+0000","last_clean":"2024-03-28T13:14:33.071297+0000","last_became_active":"2024-03-28T13:14:32.067416+0000","last_became_peered":"2024-03-28T13:14:32.067416+0000","last_unstale":"2024-03-28T13:14:33.071297+0000","last_undegraded":"2024-03-28T13:14:33.071297+0000","last_fullsized":"2024-03-28T13:14:33.071297+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:12:32.741089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"76'4","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.061470+0000","last_change":"2024-03-28T13:14:31.064033+0000","last_active":"2024-03-28T13:14:34.061470+0000","last_peered":"2024-03-28T13:14:34.061470+0000","last_clean":"2024-03-28T13:14:34.061470+0000","last_became_active":"2024-03-28T13:14:31.063708+0000","last_became_peered":"2024-03-28T13:14:31.063708+0000","last_unstale":"2024-03-28T13:14:34.061470+0000","last_undegraded":"2024-03-28T13:14:34.061470+0000","last_fullsized":"2024-03-28T13:14:34.061470+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:02:36.298761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.071833+0000","last_change":"2024-03-28T13:14:32.062836+0000","last_active":"2024-03-28T13:14:32.071833+0000","last_peered":"2024-03-28T13:14:32.071833+0000","last_clean":"2024-03-28T13:14:32.071833+0000","last_became_active":"2024-03-28T13:14:32.062485+0000","last_became_peered":"2024-03-28T13:14:32.062485+0000","last_unstale":"2024-03-28T13:14:32.071833+0000","last_undegraded":"2024-03-28T13:14:32.071833+0000","last_fullsized":"2024-03-28T13:14:32.071833+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:50:21.514416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.073658+0000","last_change":"2024-03-28T13:14:31.065269+0000","last_active":"2024-03-28T13:14:33.073658+0000","last_peered":"2024-03-28T13:14:33.073658+0000","last_clean":"2024-03-28T13:14:33.073658+0000","last_became_active":"2024-03-28T13:14:31.064858+0000","last_became_peered":"2024-03-28T13:14:31.064858+0000","last_unstale":"2024-03-28T13:14:33.073658+0000","last_undegraded":"2024-03-28T13:14:33.073658+0000","last_fullsized":"2024-03-28T13:14:33.073658+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:02:11.688822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.061822+0000","last_change":"2024-03-28T13:14:32.075328+0000","last_active":"2024-03-28T13:14:34.061822+0000","last_peered":"2024-03-28T13:14:34.061822+0000","last_clean":"2024-03-28T13:14:34.061822+0000","last_became_active":"2024-03-28T13:14:32.075030+0000","last_became_peered":"2024-03-28T13:14:32.075030+0000","last_unstale":"2024-03-28T13:14:34.061822+0000","last_undegraded":"2024-03-28T13:14:34.061822+0000","last_fullsized":"2024-03-28T13:14:34.061822+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:51:05.489280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071315+0000","last_change":"2024-03-28T13:14:31.072743+0000","last_active":"2024-03-28T13:14:33.071315+0000","last_peered":"2024-03-28T13:14:33.071315+0000","last_clean":"2024-03-28T13:14:33.071315+0000","last_became_active":"2024-03-28T13:14:31.072467+0000","last_became_peered":"2024-03-28T13:14:31.072467+0000","last_unstale":"2024-03-28T13:14:33.071315+0000","last_undegraded":"2024-03-28T13:14:33.071315+0000","last_fullsized":"2024-03-28T13:14:33.071315+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:20:38.508327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351445+0000","last_change":"2024-03-28T13:14:32.085623+0000","last_active":"2024-03-28T13:14:33.351445+0000","last_peered":"2024-03-28T13:14:33.351445+0000","last_clean":"2024-03-28T13:14:33.351445+0000","last_became_active":"2024-03-28T13:14:32.085346+0000","last_became_peered":"2024-03-28T13:14:32.085346+0000","last_unstale":"2024-03-28T13:14:33.351445+0000","last_undegraded":"2024-03-28T13:14:33.351445+0000","last_fullsized":"2024-03-28T13:14:33.351445+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:55:57.234625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.061906+0000","last_change":"2024-03-28T13:14:31.067452+0000","last_active":"2024-03-28T13:14:34.061906+0000","last_peered":"2024-03-28T13:14:34.061906+0000","last_clean":"2024-03-28T13:14:34.061906+0000","last_became_active":"2024-03-28T13:14:31.067158+0000","last_became_peered":"2024-03-28T13:14:31.067158+0000","last_unstale":"2024-03-28T13:14:34.061906+0000","last_undegraded":"2024-03-28T13:14:34.061906+0000","last_fullsized":"2024-03-28T13:14:34.061906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:02:56.474915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.070861+0000","last_change":"2024-03-28T13:14:32.077291+0000","last_active":"2024-03-28T13:14:33.070861+0000","last_peered":"2024-03-28T13:14:33.070861+0000","last_clean":"2024-03-28T13:14:33.070861+0000","last_became_active":"2024-03-28T13:14:32.076998+0000","last_became_peered":"2024-03-28T13:14:32.076998+0000","last_unstale":"2024-03-28T13:14:33.070861+0000","last_undegraded":"2024-03-28T13:14:33.070861+0000","last_fullsized":"2024-03-28T13:14:33.070861+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:49:39.311094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059895+0000","last_change":"2024-03-28T13:14:31.065639+0000","last_active":"2024-03-28T13:14:34.059895+0000","last_peered":"2024-03-28T13:14:34.059895+0000","last_clean":"2024-03-28T13:14:34.059895+0000","last_became_active":"2024-03-28T13:14:31.064089+0000","last_became_peered":"2024-03-28T13:14:31.064089+0000","last_unstale":"2024-03-28T13:14:34.059895+0000","last_undegraded":"2024-03-28T13:14:34.059895+0000","last_fullsized":"2024-03-28T13:14:34.059895+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:02:56.474915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.061690+0000","last_change":"2024-03-28T13:14:32.067061+0000","last_active":"2024-03-28T13:14:34.061690+0000","last_peered":"2024-03-28T13:14:34.061690+0000","last_clean":"2024-03-28T13:14:34.061690+0000","last_became_active":"2024-03-28T13:14:32.066753+0000","last_became_peered":"2024-03-28T13:14:32.066753+0000","last_unstale":"2024-03-28T13:14:34.061690+0000","last_undegraded":"2024-03-28T13:14:34.061690+0000","last_fullsized":"2024-03-28T13:14:34.061690+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:07.232388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351098+0000","last_change":"2024-03-28T13:14:31.069586+0000","last_active":"2024-03-28T13:14:33.351098+0000","last_peered":"2024-03-28T13:14:33.351098+0000","last_clean":"2024-03-28T13:14:33.351098+0000","last_became_active":"2024-03-28T13:14:31.068909+0000","last_became_peered":"2024-03-28T13:14:31.068909+0000","last_unstale":"2024-03-28T13:14:33.351098+0000","last_undegraded":"2024-03-28T13:14:33.351098+0000","last_fullsized":"2024-03-28T13:14:33.351098+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:05:48.122751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.074654+0000","last_change":"2024-03-28T13:14:32.071627+0000","last_active":"2024-03-28T13:14:33.074654+0000","last_peered":"2024-03-28T13:14:33.074654+0000","last_clean":"2024-03-28T13:14:33.074654+0000","last_became_active":"2024-03-28T13:14:32.071305+0000","last_became_peered":"2024-03-28T13:14:32.071305+0000","last_unstale":"2024-03-28T13:14:33.074654+0000","last_undegraded":"2024-03-28T13:14:33.074654+0000","last_fullsized":"2024-03-28T13:14:33.074654+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:38:49.112282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.075221+0000","last_change":"2024-03-28T13:14:31.060736+0000","last_active":"2024-03-28T13:14:33.075221+0000","last_peered":"2024-03-28T13:14:33.075221+0000","last_clean":"2024-03-28T13:14:33.075221+0000","last_became_active":"2024-03-28T13:14:31.060380+0000","last_became_peered":"2024-03-28T13:14:31.060380+0000","last_unstale":"2024-03-28T13:14:33.075221+0000","last_undegraded":"2024-03-28T13:14:33.075221+0000","last_fullsized":"2024-03-28T13:14:33.075221+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:02:56.474915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059572+0000","last_change":"2024-03-28T13:14:32.086680+0000","last_active":"2024-03-28T13:14:34.059572+0000","last_peered":"2024-03-28T13:14:34.059572+0000","last_clean":"2024-03-28T13:14:34.059572+0000","last_became_active":"2024-03-28T13:14:32.086324+0000","last_became_peered":"2024-03-28T13:14:32.086324+0000","last_unstale":"2024-03-28T13:14:34.059572+0000","last_undegraded":"2024-03-28T13:14:34.059572+0000","last_fullsized":"2024-03-28T13:14:34.059572+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:38:49.112282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.058919+0000","last_change":"2024-03-28T13:14:31.065869+0000","last_active":"2024-03-28T13:14:34.058919+0000","last_peered":"2024-03-28T13:14:34.058919+0000","last_clean":"2024-03-28T13:14:34.058919+0000","last_became_active":"2024-03-28T13:14:31.065341+0000","last_became_peered":"2024-03-28T13:14:31.065341+0000","last_unstale":"2024-03-28T13:14:34.058919+0000","last_undegraded":"2024-03-28T13:14:34.058919+0000","last_fullsized":"2024-03-28T13:14:34.058919+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:31:25.306578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.352293+0000","last_change":"2024-03-28T13:14:32.087235+0000","last_active":"2024-03-28T13:14:33.352293+0000","last_peered":"2024-03-28T13:14:33.352293+0000","last_clean":"2024-03-28T13:14:33.352293+0000","last_became_active":"2024-03-28T13:14:32.083492+0000","last_became_peered":"2024-03-28T13:14:32.083492+0000","last_unstale":"2024-03-28T13:14:33.352293+0000","last_undegraded":"2024-03-28T13:14:33.352293+0000","last_fullsized":"2024-03-28T13:14:33.352293+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:02:37.315493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.070946+0000","last_change":"2024-03-28T13:14:31.074537+0000","last_active":"2024-03-28T13:14:33.070946+0000","last_peered":"2024-03-28T13:14:33.070946+0000","last_clean":"2024-03-28T13:14:33.070946+0000","last_became_active":"2024-03-28T13:14:31.074035+0000","last_became_peered":"2024-03-28T13:14:31.074035+0000","last_unstale":"2024-03-28T13:14:33.070946+0000","last_undegraded":"2024-03-28T13:14:33.070946+0000","last_fullsized":"2024-03-28T13:14:33.070946+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:19:23.878452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059817+0000","last_change":"2024-03-28T13:14:32.077047+0000","last_active":"2024-03-28T13:14:34.059817+0000","last_peered":"2024-03-28T13:14:34.059817+0000","last_clean":"2024-03-28T13:14:34.059817+0000","last_became_active":"2024-03-28T13:14:32.073440+0000","last_became_peered":"2024-03-28T13:14:32.073440+0000","last_unstale":"2024-03-28T13:14:34.059817+0000","last_undegraded":"2024-03-28T13:14:34.059817+0000","last_fullsized":"2024-03-28T13:14:34.059817+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:03:49.306204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060158+0000","last_change":"2024-03-28T13:14:31.070699+0000","last_active":"2024-03-28T13:14:34.060158+0000","last_peered":"2024-03-28T13:14:34.060158+0000","last_clean":"2024-03-28T13:14:34.060158+0000","last_became_active":"2024-03-28T13:14:31.069564+0000","last_became_peered":"2024-03-28T13:14:31.069564+0000","last_unstale":"2024-03-28T13:14:34.060158+0000","last_undegraded":"2024-03-28T13:14:34.060158+0000","last_fullsized":"2024-03-28T13:14:34.060158+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:08:43.715141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.070397+0000","last_change":"2024-03-28T13:14:32.072375+0000","last_active":"2024-03-28T13:14:33.070397+0000","last_peered":"2024-03-28T13:14:33.070397+0000","last_clean":"2024-03-28T13:14:33.070397+0000","last_became_active":"2024-03-28T13:14:32.070561+0000","last_became_peered":"2024-03-28T13:14:32.070561+0000","last_unstale":"2024-03-28T13:14:33.070397+0000","last_undegraded":"2024-03-28T13:14:33.070397+0000","last_fullsized":"2024-03-28T13:14:33.070397+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:27:03.194232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"76'3","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060131+0000","last_change":"2024-03-28T13:14:31.064949+0000","last_active":"2024-03-28T13:14:34.060131+0000","last_peered":"2024-03-28T13:14:34.060131+0000","last_clean":"2024-03-28T13:14:34.060131+0000","last_became_active":"2024-03-28T13:14:31.064648+0000","last_became_peered":"2024-03-28T13:14:31.064648+0000","last_unstale":"2024-03-28T13:14:34.060131+0000","last_undegraded":"2024-03-28T13:14:34.060131+0000","last_fullsized":"2024-03-28T13:14:34.060131+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:02:36.298761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.073791+0000","last_change":"2024-03-28T13:14:32.079935+0000","last_active":"2024-03-28T13:14:33.073791+0000","last_peered":"2024-03-28T13:14:33.073791+0000","last_clean":"2024-03-28T13:14:33.073791+0000","last_became_active":"2024-03-28T13:14:32.079596+0000","last_became_peered":"2024-03-28T13:14:32.079596+0000","last_unstale":"2024-03-28T13:14:33.073791+0000","last_undegraded":"2024-03-28T13:14:33.073791+0000","last_fullsized":"2024-03-28T13:14:33.073791+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:02.366971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"76'6","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.075721+0000","last_change":"2024-03-28T13:14:31.065080+0000","last_active":"2024-03-28T13:14:33.075721+0000","last_peered":"2024-03-28T13:14:33.075721+0000","last_clean":"2024-03-28T13:14:33.075721+0000","last_became_active":"2024-03-28T13:14:31.064811+0000","last_became_peered":"2024-03-28T13:14:31.064811+0000","last_unstale":"2024-03-28T13:14:33.075721+0000","last_undegraded":"2024-03-28T13:14:33.075721+0000","last_fullsized":"2024-03-28T13:14:33.075721+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:02:36.298761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.062837+0000","last_change":"2024-03-28T13:14:32.076640+0000","last_active":"2024-03-28T13:14:34.062837+0000","last_peered":"2024-03-28T13:14:34.062837+0000","last_clean":"2024-03-28T13:14:34.062837+0000","last_became_active":"2024-03-28T13:14:32.076248+0000","last_became_peered":"2024-03-28T13:14:32.076248+0000","last_unstale":"2024-03-28T13:14:34.062837+0000","last_undegraded":"2024-03-28T13:14:34.062837+0000","last_fullsized":"2024-03-28T13:14:34.062837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:03:49.306204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.076056+0000","last_change":"2024-03-28T13:14:31.076277+0000","last_active":"2024-03-28T13:14:33.076056+0000","last_peered":"2024-03-28T13:14:33.076056+0000","last_clean":"2024-03-28T13:14:33.076056+0000","last_became_active":"2024-03-28T13:14:31.075975+0000","last_became_peered":"2024-03-28T13:14:31.075975+0000","last_unstale":"2024-03-28T13:14:33.076056+0000","last_undegraded":"2024-03-28T13:14:33.076056+0000","last_fullsized":"2024-03-28T13:14:33.076056+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:54:15.392118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351602+0000","last_change":"2024-03-28T13:14:32.078492+0000","last_active":"2024-03-28T13:14:33.351602+0000","last_peered":"2024-03-28T13:14:33.351602+0000","last_clean":"2024-03-28T13:14:33.351602+0000","last_became_active":"2024-03-28T13:14:32.078117+0000","last_became_peered":"2024-03-28T13:14:32.078117+0000","last_unstale":"2024-03-28T13:14:33.351602+0000","last_undegraded":"2024-03-28T13:14:33.351602+0000","last_fullsized":"2024-03-28T13:14:33.351602+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:51.785029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351256+0000","last_change":"2024-03-28T13:14:31.066069+0000","last_active":"2024-03-28T13:14:33.351256+0000","last_peered":"2024-03-28T13:14:33.351256+0000","last_clean":"2024-03-28T13:14:33.351256+0000","last_became_active":"2024-03-28T13:14:31.065761+0000","last_became_peered":"2024-03-28T13:14:31.065761+0000","last_unstale":"2024-03-28T13:14:33.351256+0000","last_undegraded":"2024-03-28T13:14:33.351256+0000","last_fullsized":"2024-03-28T13:14:33.351256+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:52:32.979174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.072531+0000","last_change":"2024-03-28T13:14:32.082547+0000","last_active":"2024-03-28T13:14:33.072531+0000","last_peered":"2024-03-28T13:14:33.072531+0000","last_clean":"2024-03-28T13:14:33.072531+0000","last_became_active":"2024-03-28T13:14:32.082178+0000","last_became_peered":"2024-03-28T13:14:32.082178+0000","last_unstale":"2024-03-28T13:14:33.072531+0000","last_undegraded":"2024-03-28T13:14:33.072531+0000","last_fullsized":"2024-03-28T13:14:33.072531+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:51:59.278001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351233+0000","last_change":"2024-03-28T13:14:31.064922+0000","last_active":"2024-03-28T13:14:33.351233+0000","last_peered":"2024-03-28T13:14:33.351233+0000","last_clean":"2024-03-28T13:14:33.351233+0000","last_became_active":"2024-03-28T13:14:31.064550+0000","last_became_peered":"2024-03-28T13:14:31.064550+0000","last_unstale":"2024-03-28T13:14:33.351233+0000","last_undegraded":"2024-03-28T13:14:33.351233+0000","last_fullsized":"2024-03-28T13:14:33.351233+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:49:34.858308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.072022+0000","last_change":"2024-03-28T13:14:32.063187+0000","last_active":"2024-03-28T13:14:32.072022+0000","last_peered":"2024-03-28T13:14:32.072022+0000","last_clean":"2024-03-28T13:14:32.072022+0000","last_became_active":"2024-03-28T13:14:32.059334+0000","last_became_peered":"2024-03-28T13:14:32.059334+0000","last_unstale":"2024-03-28T13:14:32.072022+0000","last_undegraded":"2024-03-28T13:14:32.072022+0000","last_fullsized":"2024-03-28T13:14:32.072022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:23:50.277081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"76'8","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.350916+0000","last_change":"2024-03-28T13:14:31.065268+0000","last_active":"2024-03-28T13:14:33.350916+0000","last_peered":"2024-03-28T13:14:33.350916+0000","last_clean":"2024-03-28T13:14:33.350916+0000","last_became_active":"2024-03-28T13:14:31.064627+0000","last_became_peered":"2024-03-28T13:14:31.064627+0000","last_unstale":"2024-03-28T13:14:33.350916+0000","last_undegraded":"2024-03-28T13:14:33.350916+0000","last_fullsized":"2024-03-28T13:14:33.350916+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:55:57.078531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060014+0000","last_change":"2024-03-28T13:14:32.089562+0000","last_active":"2024-03-28T13:14:34.060014+0000","last_peered":"2024-03-28T13:14:34.060014+0000","last_clean":"2024-03-28T13:14:34.060014+0000","last_became_active":"2024-03-28T13:14:32.088982+0000","last_became_peered":"2024-03-28T13:14:32.088982+0000","last_unstale":"2024-03-28T13:14:34.060014+0000","last_undegraded":"2024-03-28T13:14:34.060014+0000","last_fullsized":"2024-03-28T13:14:34.060014+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:37.922598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071476+0000","last_change":"2024-03-28T13:14:31.076047+0000","last_active":"2024-03-28T13:14:33.071476+0000","last_peered":"2024-03-28T13:14:33.071476+0000","last_clean":"2024-03-28T13:14:33.071476+0000","last_became_active":"2024-03-28T13:14:31.075629+0000","last_became_peered":"2024-03-28T13:14:31.075629+0000","last_unstale":"2024-03-28T13:14:33.071476+0000","last_undegraded":"2024-03-28T13:14:33.071476+0000","last_fullsized":"2024-03-28T13:14:33.071476+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:44:56.601232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060103+0000","last_change":"2024-03-28T13:14:32.076254+0000","last_active":"2024-03-28T13:14:34.060103+0000","last_peered":"2024-03-28T13:14:34.060103+0000","last_clean":"2024-03-28T13:14:34.060103+0000","last_became_active":"2024-03-28T13:14:32.075986+0000","last_became_peered":"2024-03-28T13:14:32.075986+0000","last_unstale":"2024-03-28T13:14:34.060103+0000","last_undegraded":"2024-03-28T13:14:34.060103+0000","last_fullsized":"2024-03-28T13:14:34.060103+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:02:12.705555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060257+0000","last_change":"2024-03-28T13:14:31.062734+0000","last_active":"2024-03-28T13:14:34.060257+0000","last_peered":"2024-03-28T13:14:34.060257+0000","last_clean":"2024-03-28T13:14:34.060257+0000","last_became_active":"2024-03-28T13:14:31.061798+0000","last_became_peered":"2024-03-28T13:14:31.061798+0000","last_unstale":"2024-03-28T13:14:34.060257+0000","last_undegraded":"2024-03-28T13:14:34.060257+0000","last_fullsized":"2024-03-28T13:14:34.060257+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:31:25.306578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.081679+0000","last_change":"2024-03-28T13:14:32.077657+0000","last_active":"2024-03-28T13:14:32.081679+0000","last_peered":"2024-03-28T13:14:32.081679+0000","last_clean":"2024-03-28T13:14:32.081679+0000","last_became_active":"2024-03-28T13:14:32.077363+0000","last_became_peered":"2024-03-28T13:14:32.077363+0000","last_unstale":"2024-03-28T13:14:32.081679+0000","last_undegraded":"2024-03-28T13:14:32.081679+0000","last_fullsized":"2024-03-28T13:14:32.081679+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:01:24.339090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"76'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059811+0000","last_change":"2024-03-28T13:14:31.074229+0000","last_active":"2024-03-28T13:14:34.059811+0000","last_peered":"2024-03-28T13:14:34.059811+0000","last_clean":"2024-03-28T13:14:34.059811+0000","last_became_active":"2024-03-28T13:14:31.073867+0000","last_became_peered":"2024-03-28T13:14:31.073867+0000","last_unstale":"2024-03-28T13:14:34.059811+0000","last_undegraded":"2024-03-28T13:14:34.059811+0000","last_fullsized":"2024-03-28T13:14:34.059811+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:32:50.768297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-28T13:14:32.072253+0000","last_change":"2024-03-28T13:14:32.061744+0000","last_active":"2024-03-28T13:14:32.072253+0000","last_peered":"2024-03-28T13:14:32.072253+0000","last_clean":"2024-03-28T13:14:32.072253+0000","last_became_active":"2024-03-28T13:14:32.061009+0000","last_became_peered":"2024-03-28T13:14:32.061009+0000","last_unstale":"2024-03-28T13:14:32.072253+0000","last_undegraded":"2024-03-28T13:14:32.072253+0000","last_fullsized":"2024-03-28T13:14:32.072253+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:28:36.463799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.062016+0000","last_change":"2024-03-28T13:14:31.073410+0000","last_active":"2024-03-28T13:14:34.062016+0000","last_peered":"2024-03-28T13:14:34.062016+0000","last_clean":"2024-03-28T13:14:34.062016+0000","last_became_active":"2024-03-28T13:14:31.072154+0000","last_became_peered":"2024-03-28T13:14:31.072154+0000","last_unstale":"2024-03-28T13:14:34.062016+0000","last_undegraded":"2024-03-28T13:14:34.062016+0000","last_fullsized":"2024-03-28T13:14:34.062016+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:54:15.392118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.075775+0000","last_change":"2024-03-28T13:14:32.071396+0000","last_active":"2024-03-28T13:14:33.075775+0000","last_peered":"2024-03-28T13:14:33.075775+0000","last_clean":"2024-03-28T13:14:33.075775+0000","last_became_active":"2024-03-28T13:14:32.071106+0000","last_became_peered":"2024-03-28T13:14:32.071106+0000","last_unstale":"2024-03-28T13:14:33.075775+0000","last_undegraded":"2024-03-28T13:14:33.075775+0000","last_fullsized":"2024-03-28T13:14:33.075775+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:30:07.232388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.062082+0000","last_change":"2024-03-28T13:14:31.065461+0000","last_active":"2024-03-28T13:14:34.062082+0000","last_peered":"2024-03-28T13:14:34.062082+0000","last_clean":"2024-03-28T13:14:34.062082+0000","last_became_active":"2024-03-28T13:14:31.064893+0000","last_became_peered":"2024-03-28T13:14:31.064893+0000","last_unstale":"2024-03-28T13:14:34.062082+0000","last_undegraded":"2024-03-28T13:14:34.062082+0000","last_fullsized":"2024-03-28T13:14:34.062082+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:16:26.795499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.075018+0000","last_change":"2024-03-28T13:14:32.076674+0000","last_active":"2024-03-28T13:14:33.075018+0000","last_peered":"2024-03-28T13:14:33.075018+0000","last_clean":"2024-03-28T13:14:33.075018+0000","last_became_active":"2024-03-28T13:14:32.076253+0000","last_became_peered":"2024-03-28T13:14:32.076253+0000","last_unstale":"2024-03-28T13:14:33.075018+0000","last_undegraded":"2024-03-28T13:14:33.075018+0000","last_fullsized":"2024-03-28T13:14:33.075018+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:13:02.512771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059220+0000","last_change":"2024-03-28T13:14:31.062652+0000","last_active":"2024-03-28T13:14:34.059220+0000","last_peered":"2024-03-28T13:14:34.059220+0000","last_clean":"2024-03-28T13:14:34.059220+0000","last_became_active":"2024-03-28T13:14:31.062344+0000","last_became_peered":"2024-03-28T13:14:31.062344+0000","last_unstale":"2024-03-28T13:14:34.059220+0000","last_undegraded":"2024-03-28T13:14:34.059220+0000","last_fullsized":"2024-03-28T13:14:34.059220+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:57:13.588796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.061519+0000","last_change":"2024-03-28T13:14:32.073821+0000","last_active":"2024-03-28T13:14:34.061519+0000","last_peered":"2024-03-28T13:14:34.061519+0000","last_clean":"2024-03-28T13:14:34.061519+0000","last_became_active":"2024-03-28T13:14:32.073506+0000","last_became_peered":"2024-03-28T13:14:32.073506+0000","last_unstale":"2024-03-28T13:14:34.061519+0000","last_undegraded":"2024-03-28T13:14:34.061519+0000","last_fullsized":"2024-03-28T13:14:34.061519+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:51:05.489280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.073440+0000","last_change":"2024-03-28T13:14:31.065339+0000","last_active":"2024-03-28T13:14:33.073440+0000","last_peered":"2024-03-28T13:14:33.073440+0000","last_clean":"2024-03-28T13:14:33.073440+0000","last_became_active":"2024-03-28T13:14:31.064911+0000","last_became_peered":"2024-03-28T13:14:31.064911+0000","last_unstale":"2024-03-28T13:14:33.073440+0000","last_undegraded":"2024-03-28T13:14:33.073440+0000","last_fullsized":"2024-03-28T13:14:33.073440+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:54:11.548347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.073299+0000","last_change":"2024-03-28T13:14:32.085353+0000","last_active":"2024-03-28T13:14:33.073299+0000","last_peered":"2024-03-28T13:14:33.073299+0000","last_clean":"2024-03-28T13:14:33.073299+0000","last_became_active":"2024-03-28T13:14:32.085002+0000","last_became_peered":"2024-03-28T13:14:32.085002+0000","last_unstale":"2024-03-28T13:14:33.073299+0000","last_undegraded":"2024-03-28T13:14:33.073299+0000","last_fullsized":"2024-03-28T13:14:33.073299+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:01:29.028951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071652+0000","last_change":"2024-03-28T13:14:31.074316+0000","last_active":"2024-03-28T13:14:33.071652+0000","last_peered":"2024-03-28T13:14:33.071652+0000","last_clean":"2024-03-28T13:14:33.071652+0000","last_became_active":"2024-03-28T13:14:31.073739+0000","last_became_peered":"2024-03-28T13:14:31.073739+0000","last_unstale":"2024-03-28T13:14:33.071652+0000","last_undegraded":"2024-03-28T13:14:33.071652+0000","last_fullsized":"2024-03-28T13:14:33.071652+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:25.449532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.352169+0000","last_change":"2024-03-28T13:14:32.078863+0000","last_active":"2024-03-28T13:14:33.352169+0000","last_peered":"2024-03-28T13:14:33.352169+0000","last_clean":"2024-03-28T13:14:33.352169+0000","last_became_active":"2024-03-28T13:14:32.078574+0000","last_became_peered":"2024-03-28T13:14:32.078574+0000","last_unstale":"2024-03-28T13:14:33.352169+0000","last_undegraded":"2024-03-28T13:14:33.352169+0000","last_fullsized":"2024-03-28T13:14:33.352169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:57:40.546077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"77'1","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.071347+0000","last_change":"2024-03-28T13:14:31.070077+0000","last_active":"2024-03-28T13:14:33.071347+0000","last_peered":"2024-03-28T13:14:33.071347+0000","last_clean":"2024-03-28T13:14:33.071347+0000","last_became_active":"2024-03-28T13:14:31.069793+0000","last_became_peered":"2024-03-28T13:14:31.069793+0000","last_unstale":"2024-03-28T13:14:33.071347+0000","last_undegraded":"2024-03-28T13:14:33.071347+0000","last_fullsized":"2024-03-28T13:14:33.071347+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:05:53.956158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060031+0000","last_change":"2024-03-28T13:14:32.060724+0000","last_active":"2024-03-28T13:14:34.060031+0000","last_peered":"2024-03-28T13:14:34.060031+0000","last_clean":"2024-03-28T13:14:34.060031+0000","last_became_active":"2024-03-28T13:14:32.060456+0000","last_became_peered":"2024-03-28T13:14:32.060456+0000","last_unstale":"2024-03-28T13:14:34.060031+0000","last_undegraded":"2024-03-28T13:14:34.060031+0000","last_fullsized":"2024-03-28T13:14:34.060031+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:34:50.268674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060422+0000","last_change":"2024-03-28T13:14:31.068193+0000","last_active":"2024-03-28T13:14:34.060422+0000","last_peered":"2024-03-28T13:14:34.060422+0000","last_clean":"2024-03-28T13:14:34.060422+0000","last_became_active":"2024-03-28T13:14:31.067862+0000","last_became_peered":"2024-03-28T13:14:31.067862+0000","last_unstale":"2024-03-28T13:14:34.060422+0000","last_undegraded":"2024-03-28T13:14:34.060422+0000","last_fullsized":"2024-03-28T13:14:34.060422+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:49:39.687438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351608+0000","last_change":"2024-03-28T13:14:32.072068+0000","last_active":"2024-03-28T13:14:33.351608+0000","last_peered":"2024-03-28T13:14:33.351608+0000","last_clean":"2024-03-28T13:14:33.351608+0000","last_became_active":"2024-03-28T13:14:32.071797+0000","last_became_peered":"2024-03-28T13:14:32.071797+0000","last_unstale":"2024-03-28T13:14:33.351608+0000","last_undegraded":"2024-03-28T13:14:33.351608+0000","last_fullsized":"2024-03-28T13:14:33.351608+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:08:44.731874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.060785+0000","last_change":"2024-03-28T13:14:31.067793+0000","last_active":"2024-03-28T13:14:34.060785+0000","last_peered":"2024-03-28T13:14:34.060785+0000","last_clean":"2024-03-28T13:14:34.060785+0000","last_became_active":"2024-03-28T13:14:31.067177+0000","last_became_peered":"2024-03-28T13:14:31.067177+0000","last_unstale":"2024-03-28T13:14:34.060785+0000","last_undegraded":"2024-03-28T13:14:34.060785+0000","last_fullsized":"2024-03-28T13:14:34.060785+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:57:39.529345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.072900+0000","last_change":"2024-03-28T13:14:32.078690+0000","last_active":"2024-03-28T13:14:33.072900+0000","last_peered":"2024-03-28T13:14:33.072900+0000","last_clean":"2024-03-28T13:14:33.072900+0000","last_became_active":"2024-03-28T13:14:32.078358+0000","last_became_peered":"2024-03-28T13:14:32.078358+0000","last_unstale":"2024-03-28T13:14:33.072900+0000","last_undegraded":"2024-03-28T13:14:33.072900+0000","last_fullsized":"2024-03-28T13:14:33.072900+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:58:03.118568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.061020+0000","last_change":"2024-03-28T13:14:31.058379+0000","last_active":"2024-03-28T13:14:34.061020+0000","last_peered":"2024-03-28T13:14:34.061020+0000","last_clean":"2024-03-28T13:14:34.061020+0000","last_became_active":"2024-03-28T13:14:31.057370+0000","last_became_peered":"2024-03-28T13:14:31.057370+0000","last_unstale":"2024-03-28T13:14:34.061020+0000","last_undegraded":"2024-03-28T13:14:34.061020+0000","last_fullsized":"2024-03-28T13:14:34.061020+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:54:15.392118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.070064+0000","last_change":"2024-03-28T13:14:32.062403+0000","last_active":"2024-03-28T13:14:33.070064+0000","last_peered":"2024-03-28T13:14:33.070064+0000","last_clean":"2024-03-28T13:14:33.070064+0000","last_became_active":"2024-03-28T13:14:32.062099+0000","last_became_peered":"2024-03-28T13:14:32.062099+0000","last_unstale":"2024-03-28T13:14:33.070064+0000","last_undegraded":"2024-03-28T13:14:33.070064+0000","last_fullsized":"2024-03-28T13:14:33.070064+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:12:11.052471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.073153+0000","last_change":"2024-03-28T13:14:31.067860+0000","last_active":"2024-03-28T13:14:33.073153+0000","last_peered":"2024-03-28T13:14:33.073153+0000","last_clean":"2024-03-28T13:14:33.073153+0000","last_became_active":"2024-03-28T13:14:31.067560+0000","last_became_peered":"2024-03-28T13:14:31.067560+0000","last_unstale":"2024-03-28T13:14:33.073153+0000","last_undegraded":"2024-03-28T13:14:33.073153+0000","last_fullsized":"2024-03-28T13:14:33.073153+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:34:49.251941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.075113+0000","last_change":"2024-03-28T13:14:32.068020+0000","last_active":"2024-03-28T13:14:33.075113+0000","last_peered":"2024-03-28T13:14:33.075113+0000","last_clean":"2024-03-28T13:14:33.075113+0000","last_became_active":"2024-03-28T13:14:32.067680+0000","last_became_peered":"2024-03-28T13:14:32.067680+0000","last_unstale":"2024-03-28T13:14:33.075113+0000","last_undegraded":"2024-03-28T13:14:33.075113+0000","last_fullsized":"2024-03-28T13:14:33.075113+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:55:18.345000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059398+0000","last_change":"2024-03-28T13:14:31.072958+0000","last_active":"2024-03-28T13:14:34.059398+0000","last_peered":"2024-03-28T13:14:34.059398+0000","last_clean":"2024-03-28T13:14:34.059398+0000","last_became_active":"2024-03-28T13:14:31.072289+0000","last_became_peered":"2024-03-28T13:14:31.072289+0000","last_unstale":"2024-03-28T13:14:34.059398+0000","last_undegraded":"2024-03-28T13:14:34.059398+0000","last_fullsized":"2024-03-28T13:14:34.059398+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:53:35.054440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.058930+0000","last_change":"2024-03-28T13:14:32.072121+0000","last_active":"2024-03-28T13:14:34.058930+0000","last_peered":"2024-03-28T13:14:34.058930+0000","last_clean":"2024-03-28T13:14:34.058930+0000","last_became_active":"2024-03-28T13:14:32.071381+0000","last_became_peered":"2024-03-28T13:14:32.071381+0000","last_unstale":"2024-03-28T13:14:34.058930+0000","last_undegraded":"2024-03-28T13:14:34.058930+0000","last_fullsized":"2024-03-28T13:14:34.058930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:13:02.512771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:34.059268+0000","last_change":"2024-03-28T13:14:31.066873+0000","last_active":"2024-03-28T13:14:34.059268+0000","last_peered":"2024-03-28T13:14:34.059268+0000","last_clean":"2024-03-28T13:14:34.059268+0000","last_became_active":"2024-03-28T13:14:31.065469+0000","last_became_peered":"2024-03-28T13:14:31.065469+0000","last_unstale":"2024-03-28T13:14:34.059268+0000","last_undegraded":"2024-03-28T13:14:34.059268+0000","last_fullsized":"2024-03-28T13:14:34.059268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:49:39.687438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.074325+0000","last_change":"2024-03-28T13:14:32.082048+0000","last_active":"2024-03-28T13:14:33.074325+0000","last_peered":"2024-03-28T13:14:33.074325+0000","last_clean":"2024-03-28T13:14:33.074325+0000","last_became_active":"2024-03-28T13:14:32.081621+0000","last_became_peered":"2024-03-28T13:14:32.081621+0000","last_unstale":"2024-03-28T13:14:33.074325+0000","last_undegraded":"2024-03-28T13:14:33.074325+0000","last_fullsized":"2024-03-28T13:14:33.074325+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:31.019377+0000","last_clean_scrub_stamp":"2024-03-28T13:14:31.019377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:05:54.972890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"76'2","reported_seq":16,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351660+0000","last_change":"2024-03-28T13:14:31.065484+0000","last_active":"2024-03-28T13:14:33.351660+0000","last_peered":"2024-03-28T13:14:33.351660+0000","last_clean":"2024-03-28T13:14:33.351660+0000","last_became_active":"2024-03-28T13:14:31.065151+0000","last_became_peered":"2024-03-28T13:14:31.065151+0000","last_unstale":"2024-03-28T13:14:33.351660+0000","last_undegraded":"2024-03-28T13:14:33.351660+0000","last_fullsized":"2024-03-28T13:14:33.351660+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:14:30.002645+0000","last_clean_scrub_stamp":"2024-03-28T13:14:30.002645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:49:08.890119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":270,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-28T13:14:33.351748+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:14:33.351748+0000","last_peered":"2024-03-28T13:14:33.351748+0000","last_clean":"2024-03-28T13:14:33.351748+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:14:33.351748+0000","last_undegraded":"2024-03-28T13:14:33.351748+0000","last_fullsized":"2024-03-28T13:14:33.351748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476803,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":937984,"data_stored":392833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476801,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":929792,"data_stored":393289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476801,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1519616,"data_stored":983075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476801,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":929792,"data_stored":392777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509504,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":942080,"data_stored":394353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509505,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1523712,"data_stored":983165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509505,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":925696,"data_stored":387101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":925696,"data_stored":388019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:14:37.127 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-28T13:14:37.127 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-28T13:14:37.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.393+0000 7f5f0ce00640 1 Processor -- start 2024-03-28T13:14:37.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.393+0000 7f5f0ce00640 1 -- start start 2024-03-28T13:14:37.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.396+0000 7f5f0ce00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5f080b0670 0x7f5f0815f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:37.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.397+0000 7f5f0ce00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f08070120 0x7f5f0806b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.397+0000 7f5f0ce00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f080b5b10 con 0x7f5f0806e770 2024-03-28T13:14:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.397+0000 7f5f0ce00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5f080b63e0 con 0x7f5f080b0670 2024-03-28T13:14:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.397+0000 7f5f0ce00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0806c3f0 con 0x7f5f08070120 2024-03-28T13:14:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.397+0000 7f5efd400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f08070120 0x7f5f0806b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.397+0000 7f5f05e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5f080b0670 0x7f5f0815f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.397+0000 7f5efd400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f08070120 0x7f5f0806b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:45314/0 (socket says 172.21.15.19:45314) 2024-03-28T13:14:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.397+0000 7f5efd400640 1 -- 172.21.15.19:0/2197373251 learned_addr learned my addr 172.21.15.19:0/2197373251 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.398+0000 7f5f05e00640 1 -- 172.21.15.19:0/2197373251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f08070120 msgr2=0x7f5f0806b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.398+0000 7f5f05e00640 1 --2- 172.21.15.19:0/2197373251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f08070120 0x7f5f0806b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.398+0000 7f5f05e00640 1 -- 172.21.15.19:0/2197373251 >> v1:172.21.15.188:6789/0 conn(0x7f5f0806e770 legacy=0x7f5f080d1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.398+0000 7f5f05e00640 1 -- 172.21.15.19:0/2197373251 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0815fdf0 con 0x7f5f080b0670 2024-03-28T13:14:37.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.399+0000 7f5f05e00640 1 --2- 172.21.15.19:0/2197373251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f080b0670 0x7f5f0815f8b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80097d0 tx=0x7f5ef802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14618c93cf2480f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.400+0000 7f5f05400640 1 -- 172.21.15.19:0/2197373251 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8035020 con 0x7f5f080b0670 2024-03-28T13:14:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.400+0000 7f5f05400640 1 -- 172.21.15.19:0/2197373251 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef8031810 con 0x7f5f080b0670 2024-03-28T13:14:37.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.400+0000 7f5f05400640 1 -- 172.21.15.19:0/2197373251 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8031be0 con 0x7f5f080b0670 2024-03-28T13:14:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.401+0000 7f5f0ce00640 1 -- 172.21.15.19:0/2197373251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f080b0670 msgr2=0x7f5f0815f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.401+0000 7f5f0ce00640 1 --2- 172.21.15.19:0/2197373251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f080b0670 0x7f5f0815f8b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80097d0 tx=0x7f5ef802b8b0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.401+0000 7f5f0ce00640 1 -- 172.21.15.19:0/2197373251 shutdown_connections 2024-03-28T13:14:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.401+0000 7f5f0ce00640 1 --2- 172.21.15.19:0/2197373251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f08070120 0x7f5f0806b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.401+0000 7f5f0ce00640 1 --2- 172.21.15.19:0/2197373251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f080b0670 0x7f5f0815f8b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.401+0000 7f5f0ce00640 1 -- 172.21.15.19:0/2197373251 >> 172.21.15.19:0/2197373251 conn(0x7f5f080b65a0 msgr2=0x7f5f080b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.401+0000 7f5f0ce00640 1 -- 172.21.15.19:0/2197373251 shutdown_connections 2024-03-28T13:14:37.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.401+0000 7f5f0ce00640 1 -- 172.21.15.19:0/2197373251 wait complete. 2024-03-28T13:14:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.403+0000 7f5f0ce00640 1 Processor -- start 2024-03-28T13:14:37.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.403+0000 7f5f0ce00640 1 -- start start 2024-03-28T13:14:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.404+0000 7f5f0ce00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0806e770 0x7f5f08141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.404+0000 7f5f0ce00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f08070120 0x7f5f080d0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.404+0000 7f5f0ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f080b0670 0x7f5f080aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.404+0000 7f5f0ce00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f080b6070 con 0x7f5f080b0670 2024-03-28T13:14:37.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.404+0000 7f5f0ce00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5f080b5d90 con 0x7f5f0806e770 2024-03-28T13:14:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.404+0000 7f5f0ce00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5f080b0a50 con 0x7f5f08070120 2024-03-28T13:14:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.404+0000 7f5f06800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f080b0670 0x7f5f080aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.404+0000 7f5f06800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f080b0670 0x7f5f080aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48100/0 (socket says 172.21.15.19:48100) 2024-03-28T13:14:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.404+0000 7f5f06800640 1 -- 172.21.15.19:0/3803130589 learned_addr learned my addr 172.21.15.19:0/3803130589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.405+0000 7f5f05e00640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0806e770 0x7f5f08141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.405+0000 7f5efd400640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f08070120 0x7f5f080d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.405+0000 7f5f06800640 1 -- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f08070120 msgr2=0x7f5f080d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.405+0000 7f5f06800640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f08070120 0x7f5f080d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.405+0000 7f5f06800640 1 -- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0806e770 msgr2=0x7f5f08141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.405+0000 7f5f06800640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0806e770 0x7f5f08141ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.405+0000 7f5f06800640 1 -- 172.21.15.19:0/3803130589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f08147200 con 0x7f5f080b0670 2024-03-28T13:14:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.405+0000 7f5efd400640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f08070120 0x7f5f080d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:14:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.405+0000 7f5f05e00640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0806e770 0x7f5f08141ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:14:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.405+0000 7f5f06800640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f080b0670 0x7f5f080aecb0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000c760 tx=0x7f5f0000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.406+0000 7f5efea00640 1 -- 172.21.15.19:0/3803130589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f00003b70 con 0x7f5f080b0670 2024-03-28T13:14:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.406+0000 7f5efea00640 1 -- 172.21.15.19:0/3803130589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f00003d10 con 0x7f5f080b0670 2024-03-28T13:14:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.406+0000 7f5efea00640 1 -- 172.21.15.19:0/3803130589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f0000aa90 con 0x7f5f080b0670 2024-03-28T13:14:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.406+0000 7f5f0ce00640 1 -- 172.21.15.19:0/3803130589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f081465c0 con 0x7f5f080b0670 2024-03-28T13:14:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.406+0000 7f5f0ce00640 1 -- 172.21.15.19:0/3803130589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f080cdc80 con 0x7f5f080b0670 2024-03-28T13:14:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.407+0000 7f5f0ce00640 1 -- 172.21.15.19:0/3803130589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f08068cd0 con 0x7f5f080b0670 2024-03-28T13:14:37.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.408+0000 7f5efea00640 1 -- 172.21.15.19:0/3803130589 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5f0000ac30 con 0x7f5f080b0670 2024-03-28T13:14:37.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.409+0000 7f5efea00640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ee406c560 0x7f5ee406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:37.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.409+0000 7f5efea00640 1 -- 172.21.15.19:0/3803130589 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f5f0008b5a0 con 0x7f5f080b0670 2024-03-28T13:14:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.410+0000 7f5f05e00640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ee406c560 0x7f5ee406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:37.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.411+0000 7f5f05e00640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ee406c560 0x7f5ee406ea20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8036000 tx=0x7f5ef803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:37.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.414+0000 7f5efea00640 1 -- 172.21.15.19:0/3803130589 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5f080b5b10 con 0x7f5f080b0670 2024-03-28T13:14:37.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.697+0000 7f5f0ce00640 1 -- 172.21.15.19:0/3803130589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5f08160500 con 0x7f5f080b0670 2024-03-28T13:14:37.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.700+0000 7f5efea00640 1 -- 172.21.15.19:0/3803130589 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f5f00059bc0 con 0x7f5f080b0670 2024-03-28T13:14:37.700 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-03-28T13:14:37.701 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q== 2024-03-28T13:14:37.701 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-03-28T13:14:37.701 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-03-28T13:14:37.701 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-28T13:14:37.701 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-03-28T13:14:37.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.703+0000 7f5f0ce00640 1 -- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ee406c560 msgr2=0x7f5ee406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:37.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.703+0000 7f5f0ce00640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ee406c560 0x7f5ee406ea20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8036000 tx=0x7f5ef803c040 comp rx=0 tx=0).stop 2024-03-28T13:14:37.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.703+0000 7f5f0ce00640 1 -- 172.21.15.19:0/3803130589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f080b0670 msgr2=0x7f5f080aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:37.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.703+0000 7f5f0ce00640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f080b0670 0x7f5f080aecb0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5f0000c760 tx=0x7f5f0000cc20 comp rx=0 tx=0).stop 2024-03-28T13:14:37.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.703+0000 7f5f0ce00640 1 -- 172.21.15.19:0/3803130589 shutdown_connections 2024-03-28T13:14:37.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.703+0000 7f5f0ce00640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ee406c560 0x7f5ee406ea20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.703+0000 7f5f0ce00640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f080b0670 0x7f5f080aecb0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.703+0000 7f5f0ce00640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f08070120 0x7f5f080d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.703+0000 7f5f0ce00640 1 --2- 172.21.15.19:0/3803130589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f0806e770 0x7f5f08141ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.703+0000 7f5f0ce00640 1 -- 172.21.15.19:0/3803130589 >> 172.21.15.19:0/3803130589 conn(0x7f5f080b65a0 msgr2=0x7f5f08158940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:37.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.703+0000 7f5f0ce00640 1 -- 172.21.15.19:0/3803130589 shutdown_connections 2024-03-28T13:14:37.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.703+0000 7f5f0ce00640 1 -- 172.21.15.19:0/3803130589 wait complete. 2024-03-28T13:14:37.721 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:14:37.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.918+0000 7fa961c00640 1 Processor -- start 2024-03-28T13:14:37.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.918+0000 7fa961c00640 1 -- start start 2024-03-28T13:14:37.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.920+0000 7fa961c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa95c15c390 0x7fa95c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:37.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.920+0000 7fa961c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa95c0d0650 0x7fa95c0d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:37.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.920+0000 7fa961c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa95c15fe80 con 0x7fa95c0af830 2024-03-28T13:14:37.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.920+0000 7fa961c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa95c15c770 con 0x7fa95c15c390 2024-03-28T13:14:37.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.920+0000 7fa961c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa95c0ad120 con 0x7fa95c0d0650 2024-03-28T13:14:37.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.920+0000 7fa95aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa95c15c390 0x7fa95c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:37.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.920+0000 7fa95aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa95c15c390 0x7fa95c073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48110/0 (socket says 172.21.15.19:48110) 2024-03-28T13:14:37.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.920+0000 7fa95aa00640 1 -- 172.21.15.19:0/2452544246 learned_addr learned my addr 172.21.15.19:0/2452544246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:37.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.920+0000 7fa95b400640 1 --2- 172.21.15.19:0/2452544246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa95c0d0650 0x7fa95c0d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:37.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.921+0000 7fa95aa00640 1 -- 172.21.15.19:0/2452544246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa95c0d0650 msgr2=0x7fa95c0d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:37.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.921+0000 7fa95aa00640 1 --2- 172.21.15.19:0/2452544246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa95c0d0650 0x7fa95c0d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.921+0000 7fa95aa00640 1 -- 172.21.15.19:0/2452544246 >> v1:172.21.15.188:6789/0 conn(0x7fa95c0af830 legacy=0x7fa95c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:37.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.921+0000 7fa95aa00640 1 -- 172.21.15.19:0/2452544246 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95c0ad400 con 0x7fa95c15c390 2024-03-28T13:14:37.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.922+0000 7fa95aa00640 1 --2- 172.21.15.19:0/2452544246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c15c390 0x7fa95c073790 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009870 tx=0x7fa94c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b2e337a3ff2667 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:37.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.923+0000 7fa95a000640 1 -- 172.21.15.19:0/2452544246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c035020 con 0x7fa95c15c390 2024-03-28T13:14:37.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.923+0000 7fa95a000640 1 -- 172.21.15.19:0/2452544246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94c031640 con 0x7fa95c15c390 2024-03-28T13:14:37.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.923+0000 7fa95a000640 1 -- 172.21.15.19:0/2452544246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c02d040 con 0x7fa95c15c390 2024-03-28T13:14:37.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.923+0000 7fa961c00640 1 -- 172.21.15.19:0/2452544246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c15c390 msgr2=0x7fa95c073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:37.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.923+0000 7fa961c00640 1 --2- 172.21.15.19:0/2452544246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c15c390 0x7fa95c073790 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009870 tx=0x7fa94c02b8f0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.923+0000 7fa961c00640 1 -- 172.21.15.19:0/2452544246 shutdown_connections 2024-03-28T13:14:37.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.923+0000 7fa961c00640 1 --2- 172.21.15.19:0/2452544246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa95c0d0650 0x7fa95c0d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.923+0000 7fa961c00640 1 --2- 172.21.15.19:0/2452544246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c15c390 0x7fa95c073790 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.923+0000 7fa961c00640 1 -- 172.21.15.19:0/2452544246 >> 172.21.15.19:0/2452544246 conn(0x7fa95c06b990 msgr2=0x7fa95c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:37.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.924+0000 7fa961c00640 1 -- 172.21.15.19:0/2452544246 shutdown_connections 2024-03-28T13:14:37.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.924+0000 7fa961c00640 1 -- 172.21.15.19:0/2452544246 wait complete. 2024-03-28T13:14:37.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.925+0000 7fa961c00640 1 Processor -- start 2024-03-28T13:14:37.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.926+0000 7fa961c00640 1 -- start start 2024-03-28T13:14:37.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.926+0000 7fa961c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c0af830 0x7fa95c14fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:37.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.926+0000 7fa961c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa95c0d0650 0x7fa95c152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:37.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa961c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa95c15c390 0x7fa95c15d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:37.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa961c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c160d50 con 0x7fa95c0af830 2024-03-28T13:14:37.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa961c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa95c160120 con 0x7fa95c0d0650 2024-03-28T13:14:37.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa961c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa95c0cf980 con 0x7fa95c15c390 2024-03-28T13:14:37.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa95aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c0af830 0x7fa95c14fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:37.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa95aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c0af830 0x7fa95c14fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48112/0 (socket says 172.21.15.19:48112) 2024-03-28T13:14:37.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa95aa00640 1 -- 172.21.15.19:0/1996276887 learned_addr learned my addr 172.21.15.19:0/1996276887 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:37.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa95b400640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa95c15c390 0x7fa95c15d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:37.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa952000640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa95c0d0650 0x7fa95c152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:37.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa95aa00640 1 -- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa95c15c390 msgr2=0x7fa95c15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:37.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa95aa00640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa95c15c390 0x7fa95c15d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa95aa00640 1 -- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa95c0d0650 msgr2=0x7fa95c152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:37.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa95aa00640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa95c0d0650 0x7fa95c152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:37.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa95aa00640 1 -- 172.21.15.19:0/1996276887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95c154a10 con 0x7fa95c0af830 2024-03-28T13:14:37.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa95b400640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa95c15c390 0x7fa95c15d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:37.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.927+0000 7fa952000640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa95c0d0650 0x7fa95c152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:37.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.928+0000 7fa95aa00640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c0af830 0x7fa95c14fba0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa94c02b930 tx=0x7fa94c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:37.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.928+0000 7fa953400640 1 -- 172.21.15.19:0/1996276887 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c040070 con 0x7fa95c0af830 2024-03-28T13:14:37.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.928+0000 7fa953400640 1 -- 172.21.15.19:0/1996276887 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa94c0399f0 con 0x7fa95c0af830 2024-03-28T13:14:37.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.928+0000 7fa953400640 1 -- 172.21.15.19:0/1996276887 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c035040 con 0x7fa95c0af830 2024-03-28T13:14:37.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.928+0000 7fa961c00640 1 -- 172.21.15.19:0/1996276887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95c15daf0 con 0x7fa95c0af830 2024-03-28T13:14:37.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.928+0000 7fa961c00640 1 -- 172.21.15.19:0/1996276887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa95c154cd0 con 0x7fa95c0af830 2024-03-28T13:14:37.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.930+0000 7fa961c00640 1 -- 172.21.15.19:0/1996276887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa95c0685c0 con 0x7fa95c0af830 2024-03-28T13:14:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.931+0000 7fa953400640 1 -- 172.21.15.19:0/1996276887 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa94c04b060 con 0x7fa95c0af830 2024-03-28T13:14:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.932+0000 7fa953400640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa93806c530 0x7fa93806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.932+0000 7fa953400640 1 -- 172.21.15.19:0/1996276887 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7fa94c0acf80 con 0x7fa95c0af830 2024-03-28T13:14:37.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.933+0000 7fa952000640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa93806c530 0x7fa93806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:37.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.934+0000 7fa952000640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa93806c530 0x7fa93806e9f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa95c1577d0 tx=0x7fa944008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:37.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:37.937+0000 7fa953400640 1 -- 172.21.15.19:0/1996276887 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa94c07b630 con 0x7fa95c0af830 2024-03-28T13:14:38.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.182+0000 7fa961c00640 1 -- 172.21.15.19:0/1996276887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa95c0b4610 con 0x7fa95c0af830 2024-03-28T13:14:38.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.184+0000 7fa953400640 1 -- 172.21.15.19:0/1996276887 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5186 (secure 0 0 0) 0x7fa94c07af30 con 0x7fa95c0af830 2024-03-28T13:14:38.184 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:38.184 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/144726418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":144726418},{"type":"v1","addr":"172.21.15.188:6836","nonce":144726418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5363,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2642581607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2642581607},{"type":"v1","addr":"172.21.15.19:6835","nonce":2642581607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1721039756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1721039756},{"type":"v1","addr":"172.21.15.188:6837","nonce":1721039756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:14:31.526577+0000","modified":"2024-03-28T13:14:34.573581+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6507},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6507":{"gid":6507,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/1189161682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1189161682},{"type":"v1","addr":"172.21.15.19:6834","nonce":1189161682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6507,"qdb_cluster":[6507]},"id":5}]} 2024-03-28T13:14:38.184 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 45 2024-03-28T13:14:38.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.186+0000 7fa961c00640 1 -- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa93806c530 msgr2=0x7fa93806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:38.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.186+0000 7fa961c00640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa93806c530 0x7fa93806e9f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa95c1577d0 tx=0x7fa944008040 comp rx=0 tx=0).stop 2024-03-28T13:14:38.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.186+0000 7fa961c00640 1 -- 172.21.15.19:0/1996276887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c0af830 msgr2=0x7fa95c14fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:38.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.186+0000 7fa961c00640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c0af830 0x7fa95c14fba0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fa94c02b930 tx=0x7fa94c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.187+0000 7fa961c00640 1 -- 172.21.15.19:0/1996276887 shutdown_connections 2024-03-28T13:14:38.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.187+0000 7fa961c00640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa93806c530 0x7fa93806e9f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.187+0000 7fa961c00640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa95c15c390 0x7fa95c15d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.187+0000 7fa961c00640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa95c0d0650 0x7fa95c152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.187+0000 7fa961c00640 1 --2- 172.21.15.19:0/1996276887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa95c0af830 0x7fa95c14fba0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.187+0000 7fa961c00640 1 -- 172.21.15.19:0/1996276887 >> 172.21.15.19:0/1996276887 conn(0x7fa95c06b990 msgr2=0x7fa95c0726c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:38.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.187+0000 7fa961c00640 1 -- 172.21.15.19:0/1996276887 shutdown_connections 2024-03-28T13:14:38.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.187+0000 7fa961c00640 1 -- 172.21.15.19:0/1996276887 wait complete. 2024-03-28T13:14:38.609 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:14:31.526577+0000', 'modified': '2024-03-28T13:14:34.573581+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6507}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6507': {'gid': 6507, 'name': 'c', 'rank': 0, 'incarnation': 41, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6834/1189161682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 1189161682}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 1189161682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6507, 'qdb_cluster': [6507]} 2024-03-28T13:14:38.609 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T13:14:38.610 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-28T13:14:38.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.842+0000 7f02dea00640 1 Processor -- start 2024-03-28T13:14:38.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.842+0000 7f02dea00640 1 -- start start 2024-03-28T13:14:38.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.843+0000 7f02dea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f02d80d3e20 0x7f02d80b9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.843+0000 7f02dea00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f02d80cef00 0x7f02d80cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.843+0000 7f02dea00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02d80d2d80 con 0x7f02d815e440 2024-03-28T13:14:38.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.843+0000 7f02dea00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f02d80cf8b0 con 0x7f02d80d3e20 2024-03-28T13:14:38.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.843+0000 7f02dea00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f02d80b8300 con 0x7f02d80cef00 2024-03-28T13:14:38.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.844+0000 7f02dcc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f02d80d3e20 0x7f02d80b9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.844+0000 7f02dd600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f02d815e440 0x7f02d80d2530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48190/0 (socket says 172.21.15.19:48190) 2024-03-28T13:14:38.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.844+0000 7f02dd600640 1 -- 172.21.15.19:0/1948877575 learned_addr learned my addr 172.21.15.19:0/1948877575 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:38.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.844+0000 7f02dcc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f02d80d3e20 0x7f02d80b9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48122/0 (socket says 172.21.15.19:48122) 2024-03-28T13:14:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.844+0000 7f02dcc00640 1 -- 172.21.15.19:0/1948877575 >> v2:172.21.15.188:3300/0 conn(0x7f02d80cef00 msgr2=0x7f02d80cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.844+0000 7f02dcc00640 1 --2- 172.21.15.19:0/1948877575 >> v2:172.21.15.188:3300/0 conn(0x7f02d80cef00 0x7f02d80cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.844+0000 7f02dcc00640 1 -- 172.21.15.19:0/1948877575 >> v1:172.21.15.19:6789/0 conn(0x7f02d815e440 legacy=0x7f02d80d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.844+0000 7f02dcc00640 1 -- 172.21.15.19:0/1948877575 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d80d3060 con 0x7f02d80d3e20 2024-03-28T13:14:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.844+0000 7f02d7e00640 1 -- 172.21.15.19:0/1948877575 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2694939044 0 0) 0x7f02d80d2d80 con 0x7f02d815e440 2024-03-28T13:14:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.845+0000 7f02dcc00640 1 --2- 172.21.15.19:0/1948877575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d80d3e20 0x7f02d80b9e00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f02d80d0640 tx=0x7f02c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3945a0d730d853 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.846+0000 7f02d7e00640 1 -- 172.21.15.19:0/1948877575 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c4035020 con 0x7f02d80d3e20 2024-03-28T13:14:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.846+0000 7f02d7e00640 1 -- 172.21.15.19:0/1948877575 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c4031830 con 0x7f02d80d3e20 2024-03-28T13:14:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.846+0000 7f02d7e00640 1 -- 172.21.15.19:0/1948877575 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c4031c00 con 0x7f02d80d3e20 2024-03-28T13:14:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.846+0000 7f02dea00640 1 -- 172.21.15.19:0/1948877575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d80d3e20 msgr2=0x7f02d80b9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.846+0000 7f02dea00640 1 --2- 172.21.15.19:0/1948877575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d80d3e20 0x7f02d80b9e00 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f02d80d0640 tx=0x7f02c402b8d0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.846+0000 7f02dea00640 1 -- 172.21.15.19:0/1948877575 shutdown_connections 2024-03-28T13:14:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.846+0000 7f02dea00640 1 --2- 172.21.15.19:0/1948877575 >> v2:172.21.15.188:3300/0 conn(0x7f02d80cef00 0x7f02d80cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.846+0000 7f02dea00640 1 --2- 172.21.15.19:0/1948877575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d80d3e20 0x7f02d80b9e00 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.846+0000 7f02dea00640 1 -- 172.21.15.19:0/1948877575 >> 172.21.15.19:0/1948877575 conn(0x7f02d80678b0 msgr2=0x7f02d80d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.847+0000 7f02dea00640 1 -- 172.21.15.19:0/1948877575 shutdown_connections 2024-03-28T13:14:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.847+0000 7f02dea00640 1 -- 172.21.15.19:0/1948877575 wait complete. 2024-03-28T13:14:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.848+0000 7f02dea00640 1 Processor -- start 2024-03-28T13:14:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.848+0000 7f02dea00640 1 -- start start 2024-03-28T13:14:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.849+0000 7f02dea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d80cef00 0x7f02d81530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.849+0000 7f02dea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d80d3e20 0x7f02d8161320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.849+0000 7f02dea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02d815e440 0x7f02d8161860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.850+0000 7f02dea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02d80d3900 con 0x7f02d80cef00 2024-03-28T13:14:38.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.850+0000 7f02dea00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f02d80d3620 con 0x7f02d80d3e20 2024-03-28T13:14:38.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.850+0000 7f02dea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f02d80b72f0 con 0x7f02d815e440 2024-03-28T13:14:38.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.850+0000 7f02dd600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d80cef00 0x7f02d81530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.850+0000 7f02dd600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d80cef00 0x7f02d81530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48130/0 (socket says 172.21.15.19:48130) 2024-03-28T13:14:38.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.850+0000 7f02dd600640 1 -- 172.21.15.19:0/3623476604 learned_addr learned my addr 172.21.15.19:0/3623476604 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:38.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.850+0000 7f02dcc00640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d80d3e20 0x7f02d8161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.850+0000 7f02dd600640 1 -- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02d815e440 msgr2=0x7f02d8161860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.850+0000 7f02de000640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02d815e440 0x7f02d8161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.850+0000 7f02dd600640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02d815e440 0x7f02d8161860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.850+0000 7f02dd600640 1 -- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d80d3e20 msgr2=0x7f02d8161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.850+0000 7f02dd600640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d80d3e20 0x7f02d8161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.850+0000 7f02dd600640 1 -- 172.21.15.19:0/3623476604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d815f9d0 con 0x7f02d80cef00 2024-03-28T13:14:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.850+0000 7f02dcc00640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d80d3e20 0x7f02d8161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.850+0000 7f02de000640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02d815e440 0x7f02d8161860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.851+0000 7f02dd600640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d80cef00 0x7f02d81530e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00c7c0 tx=0x7f02cc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.851+0000 7f02d5600640 1 -- 172.21.15.19:0/3623476604 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc018070 con 0x7f02d80cef00 2024-03-28T13:14:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.851+0000 7f02d5600640 1 -- 172.21.15.19:0/3623476604 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02cc00a640 con 0x7f02d80cef00 2024-03-28T13:14:38.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.851+0000 7f02d5600640 1 -- 172.21.15.19:0/3623476604 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc00aa30 con 0x7f02d80cef00 2024-03-28T13:14:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.851+0000 7f02dea00640 1 -- 172.21.15.19:0/3623476604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d815fc70 con 0x7f02d80cef00 2024-03-28T13:14:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.851+0000 7f02dea00640 1 -- 172.21.15.19:0/3623476604 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d8160230 con 0x7f02d80cef00 2024-03-28T13:14:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.853+0000 7f02d5600640 1 -- 172.21.15.19:0/3623476604 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f02cc00abd0 con 0x7f02d80cef00 2024-03-28T13:14:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.854+0000 7f02d5600640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f02b806c530 0x7f02b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.854+0000 7f02d5600640 1 -- 172.21.15.19:0/3623476604 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f02cc08adc0 con 0x7f02d80cef00 2024-03-28T13:14:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.855+0000 7f02dcc00640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f02b806c530 0x7f02b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.856+0000 7f02dcc00640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f02b806c530 0x7f02b806e9f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f02c4036000 tx=0x7f02c4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.856+0000 7f02e2da1740 1 Processor -- start 2024-03-28T13:14:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.856+0000 7f02e2da1740 1 -- start start 2024-03-28T13:14:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02e2da1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec31d0 0x5585f5e96170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02e2da1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5ec35b0 0x5585f5ec5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02e2da1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5e966b0 0x5585f5ec64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02e2da1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5585f5e5d920 con 0x5585f5ec31d0 2024-03-28T13:14:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02e2da1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5585f5dadfa0 con 0x5585f5ec35b0 2024-03-28T13:14:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02e2da1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5585f5dfff90 con 0x5585f5e966b0 2024-03-28T13:14:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02dd600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec31d0 0x5585f5e96170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02dd600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec31d0 0x5585f5e96170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48140/0 (socket says 172.21.15.19:48140) 2024-03-28T13:14:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02dd600640 1 -- 172.21.15.19:0/570978363 learned_addr learned my addr 172.21.15.19:0/570978363 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02dcc00640 1 --2- 172.21.15.19:0/570978363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5ec35b0 0x5585f5ec5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02de000640 1 --2- 172.21.15.19:0/570978363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5e966b0 0x5585f5ec64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02dd600640 1 -- 172.21.15.19:0/570978363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5e966b0 msgr2=0x5585f5ec64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02dd600640 1 --2- 172.21.15.19:0/570978363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5e966b0 0x5585f5ec64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02dd600640 1 -- 172.21.15.19:0/570978363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5ec35b0 msgr2=0x5585f5ec5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02dd600640 1 --2- 172.21.15.19:0/570978363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5ec35b0 0x5585f5ec5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02dd600640 1 -- 172.21.15.19:0/570978363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585f5b142d0 con 0x5585f5ec31d0 2024-03-28T13:14:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.858+0000 7f02dcc00640 1 --2- 172.21.15.19:0/570978363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5ec35b0 0x5585f5ec5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.859+0000 7f02dd600640 1 --2- 172.21.15.19:0/570978363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec31d0 0x5585f5e96170 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f02cc061380 tx=0x7f02cc054f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bdcb019bc12c2f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.859+0000 7f02c9c00640 1 -- 172.21.15.19:0/570978363 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc0618b0 con 0x5585f5ec31d0 2024-03-28T13:14:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.859+0000 7f02c9c00640 1 -- 172.21.15.19:0/570978363 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02cc056fd0 con 0x5585f5ec31d0 2024-03-28T13:14:38.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.859+0000 7f02c9c00640 1 -- 172.21.15.19:0/570978363 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc058120 con 0x5585f5ec31d0 2024-03-28T13:14:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.860+0000 7f02e2da1740 1 -- 172.21.15.19:0/570978363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec31d0 msgr2=0x5585f5e96170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.860+0000 7f02e2da1740 1 --2- 172.21.15.19:0/570978363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec31d0 0x5585f5e96170 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f02cc061380 tx=0x7f02cc054f70 comp rx=0 tx=0).stop 2024-03-28T13:14:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.860+0000 7f02e2da1740 1 -- 172.21.15.19:0/570978363 shutdown_connections 2024-03-28T13:14:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.860+0000 7f02e2da1740 1 --2- 172.21.15.19:0/570978363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5e966b0 0x5585f5ec64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.860+0000 7f02e2da1740 1 --2- 172.21.15.19:0/570978363 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5ec35b0 0x5585f5ec5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.860+0000 7f02e2da1740 1 --2- 172.21.15.19:0/570978363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec31d0 0x5585f5e96170 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.860+0000 7f02e2da1740 1 -- 172.21.15.19:0/570978363 >> 172.21.15.19:0/570978363 conn(0x5585f5e983e0 msgr2=0x5585f5ee0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.860+0000 7f02e2da1740 1 -- 172.21.15.19:0/570978363 shutdown_connections 2024-03-28T13:14:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.860+0000 7f02e2da1740 1 -- 172.21.15.19:0/570978363 wait complete. 2024-03-28T13:14:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.861+0000 7f02e2da1740 1 -- start start 2024-03-28T13:14:38.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.861+0000 7f02e2da1740 1 Processor -- start 2024-03-28T13:14:38.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.863+0000 7f02e2da1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5ec31d0 0x5585f5eb78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.863+0000 7f02e2da1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5ec35b0 0x5585f5ed8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.863+0000 7f02e2da1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5edb0f0 0x5585f5eb4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.863+0000 7f02e2da1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5585f5dfbf40 con 0x5585f5edb0f0 2024-03-28T13:14:38.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.863+0000 7f02e2da1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5585f5da8650 con 0x5585f5ec35b0 2024-03-28T13:14:38.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.863+0000 7f02e2da1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5585f5dbba10 con 0x5585f5ec31d0 2024-03-28T13:14:38.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.863+0000 7f02e2da1740 10 client.0 _ll_register_callbacks cb 0x5585f5dfaff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:14:38.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.864+0000 7f02de000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5edb0f0 0x5585f5eb4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.864+0000 7f02de000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5edb0f0 0x5585f5eb4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48142/0 (socket says 172.21.15.19:48142) 2024-03-28T13:14:38.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.864+0000 7f02de000640 1 -- 172.21.15.19:0/922982030 learned_addr learned my addr 172.21.15.19:0/922982030 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:38.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.864+0000 7f02dcc00640 1 --2- 172.21.15.19:0/922982030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5ec35b0 0x5585f5ed8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.864+0000 7f02dd600640 1 --2- 172.21.15.19:0/922982030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5ec31d0 0x5585f5eb78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.864+0000 7f02de000640 1 -- 172.21.15.19:0/922982030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5ec31d0 msgr2=0x5585f5eb78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:38.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.864+0000 7f02de000640 1 --2- 172.21.15.19:0/922982030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5ec31d0 0x5585f5eb78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.864+0000 7f02de000640 1 -- 172.21.15.19:0/922982030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5ec35b0 msgr2=0x5585f5ed8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:38.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.864+0000 7f02de000640 1 --2- 172.21.15.19:0/922982030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5ec35b0 0x5585f5ed8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.864+0000 7f02de000640 1 -- 172.21.15.19:0/922982030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585f5bde6b0 con 0x5585f5edb0f0 2024-03-28T13:14:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.864+0000 7f02de000640 1 --2- 172.21.15.19:0/922982030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5edb0f0 0x5585f5eb4e50 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f02d000d1f0 tx=0x7f02d000d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ae2f551493d8fc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.864+0000 7f02be000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:14:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.865+0000 7f02be000640 1 -- 172.21.15.19:0/922982030 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d0016070 con 0x5585f5edb0f0 2024-03-28T13:14:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.865+0000 7f02be000640 1 -- 172.21.15.19:0/922982030 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f02d0012870 con 0x5585f5edb0f0 2024-03-28T13:14:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.865+0000 7f02e2da1740 1 -- 172.21.15.19:0/922982030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02cc056fd0 con 0x5585f5edb0f0 2024-03-28T13:14:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.865+0000 7f02be000640 1 -- 172.21.15.19:0/922982030 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f02d0021050 con 0x5585f5edb0f0 2024-03-28T13:14:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.865+0000 7f02e2da1740 10 client.6639 fetch_fsmap learned FSMap version 45 2024-03-28T13:14:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.865+0000 7f02e2da1740 1 -- 172.21.15.19:0/922982030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x5585f5ed9be0 con 0x5585f5edb0f0 2024-03-28T13:14:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.865+0000 7f02be000640 1 -- 172.21.15.19:0/922982030 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02cc056fd0 con 0x5585f5edb0f0 2024-03-28T13:14:38.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.865+0000 7f02be000640 1 -- 172.21.15.19:0/922982030 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7f02d0024020 con 0x5585f5edb0f0 2024-03-28T13:14:38.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.865+0000 7f02e2da1740 10 client.6639 fetch_fsmap finished waiting for FSMap version 45 2024-03-28T13:14:38.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.866+0000 7f02e2da1740 10 client.6639 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T13:14:38.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.866+0000 7f02e2da1740 20 client.6639 populate_metadata read hostname 'smithi019' 2024-03-28T13:14:38.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.866+0000 7f02e2da1740 1 --2- 172.21.15.19:0/922982030 >> [v2:172.21.15.19:6832/1189161682,v1:172.21.15.19:6834/1189161682] conn(0x5585f5efcff0 0x5585f5eff3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.866+0000 7f02e2da1740 4 client.6639 mds_command: new command op to 6507 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:14:38.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.866+0000 7f02e2da1740 1 -- 172.21.15.19:0/922982030 --> [v2:172.21.15.19:6832/1189161682,v1:172.21.15.19:6834/1189161682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585f5e0d4d0 con 0x5585f5efcff0 2024-03-28T13:14:38.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.866+0000 7f02dd600640 1 --2- 172.21.15.19:0/922982030 >> [v2:172.21.15.19:6832/1189161682,v1:172.21.15.19:6834/1189161682] conn(0x5585f5efcff0 0x5585f5eff3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.866+0000 7f02dd600640 1 --2- 172.21.15.19:0/922982030 >> [v2:172.21.15.19:6832/1189161682,v1:172.21.15.19:6834/1189161682] conn(0x5585f5efcff0 0x5585f5eff3e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63f655c6f31b4f4e server_cookie=b4e39fa4448019ca in_seq=0 out_seq=0 2024-03-28T13:14:38.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.866+0000 7f02be000640 10 client.6639 ms_handle_connect on v2:172.21.15.19:6832/1189161682 2024-03-28T13:14:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.868+0000 7f02be000640 1 -- 172.21.15.19:0/922982030 <== mds.0 v2:172.21.15.19:6832/1189161682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585f5e0d4d0 con 0x5585f5efcff0 2024-03-28T13:14:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.868+0000 7f02be000640 10 client.6639 handle_command_reply: tid=0 2024-03-28T13:14:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.868+0000 7f02e2da1740 1 client.6639 shutdown 2024-03-28T13:14:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.868+0000 7f02e2da1740 1 -- 172.21.15.19:0/922982030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5edb0f0 msgr2=0x5585f5eb4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:38.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.868+0000 7f02e2da1740 1 --2- 172.21.15.19:0/922982030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5edb0f0 0x5585f5eb4e50 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f02d000d1f0 tx=0x7f02d000d6b0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.868+0000 7f02e2da1740 1 -- 172.21.15.19:0/922982030 shutdown_connections 2024-03-28T13:14:38.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.868+0000 7f02e2da1740 1 --2- 172.21.15.19:0/922982030 >> [v2:172.21.15.19:6832/1189161682,v1:172.21.15.19:6834/1189161682] conn(0x5585f5efcff0 0x5585f5eff3e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.868+0000 7f02e2da1740 1 --2- 172.21.15.19:0/922982030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5edb0f0 0x5585f5eb4e50 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.868+0000 7f02e2da1740 1 --2- 172.21.15.19:0/922982030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5ec35b0 0x5585f5ed8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.868+0000 7f02e2da1740 1 --2- 172.21.15.19:0/922982030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5ec31d0 0x5585f5eb78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.868+0000 7f02be000640 0 client.6639 ms_handle_reset on v2:172.21.15.19:6832/1189161682 2024-03-28T13:14:38.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.868+0000 7f02e2da1740 1 -- 172.21.15.19:0/922982030 >> 172.21.15.19:0/922982030 conn(0x5585f5e966b0 msgr2=0x5585f5e95890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:38.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.869+0000 7f02e2da1740 1 -- 172.21.15.19:0/922982030 shutdown_connections 2024-03-28T13:14:38.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.869+0000 7f02e2da1740 1 -- 172.21.15.19:0/922982030 wait complete. 2024-03-28T13:14:38.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.869+0000 7f02e2da1740 20 client.6639 trim_cache size 0 max 16384 2024-03-28T13:14:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.899+0000 7f02e2da1740 1 Processor -- start 2024-03-28T13:14:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.899+0000 7f02e2da1740 1 -- start start 2024-03-28T13:14:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.900+0000 7f02e2da1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5ec31d0 0x5585f5eb4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.900+0000 7f02e2da1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec35b0 0x5585f5ec2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.900+0000 7f02e2da1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5edb0f0 0x5585f5ec2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.900+0000 7f02e2da1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f02d81ff780 con 0x5585f5ec35b0 2024-03-28T13:14:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.900+0000 7f02e2da1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f02d80acb30 con 0x5585f5edb0f0 2024-03-28T13:14:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.900+0000 7f02e2da1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f02d80ad180 con 0x5585f5ec31d0 2024-03-28T13:14:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.900+0000 7f02dcc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec35b0 0x5585f5ec2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.900+0000 7f02dcc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec35b0 0x5585f5ec2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48156/0 (socket says 172.21.15.19:48156) 2024-03-28T13:14:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.900+0000 7f02dcc00640 1 -- 172.21.15.19:0/2959665856 learned_addr learned my addr 172.21.15.19:0/2959665856 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.901+0000 7f02de000640 1 --2- 172.21.15.19:0/2959665856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5edb0f0 0x5585f5ec2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.901+0000 7f02dd600640 1 --2- 172.21.15.19:0/2959665856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5ec31d0 0x5585f5eb4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.901+0000 7f02dcc00640 1 -- 172.21.15.19:0/2959665856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5ec31d0 msgr2=0x5585f5eb4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.901+0000 7f02dcc00640 1 --2- 172.21.15.19:0/2959665856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5ec31d0 0x5585f5eb4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.901+0000 7f02dcc00640 1 -- 172.21.15.19:0/2959665856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5edb0f0 msgr2=0x5585f5ec2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.901+0000 7f02dcc00640 1 --2- 172.21.15.19:0/2959665856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5edb0f0 0x5585f5ec2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.901+0000 7f02dcc00640 1 -- 172.21.15.19:0/2959665856 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585f5e9ac40 con 0x5585f5ec35b0 2024-03-28T13:14:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.901+0000 7f02de000640 1 --2- 172.21.15.19:0/2959665856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5edb0f0 0x5585f5ec2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.901+0000 7f02dcc00640 1 --2- 172.21.15.19:0/2959665856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec35b0 0x5585f5ec2300 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f02c402bee0 tx=0x7f02c4030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88dd184ab390e11 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.901+0000 7f02c9200640 1 -- 172.21.15.19:0/2959665856 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c403ae40 con 0x5585f5ec35b0 2024-03-28T13:14:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.901+0000 7f02c9200640 1 -- 172.21.15.19:0/2959665856 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c4031cb0 con 0x5585f5ec35b0 2024-03-28T13:14:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.901+0000 7f02c9200640 1 -- 172.21.15.19:0/2959665856 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c4031e60 con 0x5585f5ec35b0 2024-03-28T13:14:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.902+0000 7f02e2da1740 1 -- 172.21.15.19:0/2959665856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec35b0 msgr2=0x5585f5ec2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.902+0000 7f02e2da1740 1 --2- 172.21.15.19:0/2959665856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec35b0 0x5585f5ec2300 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f02c402bee0 tx=0x7f02c4030000 comp rx=0 tx=0).stop 2024-03-28T13:14:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.902+0000 7f02e2da1740 1 -- 172.21.15.19:0/2959665856 shutdown_connections 2024-03-28T13:14:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.902+0000 7f02e2da1740 1 --2- 172.21.15.19:0/2959665856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5edb0f0 0x5585f5ec2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.902+0000 7f02e2da1740 1 --2- 172.21.15.19:0/2959665856 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec35b0 0x5585f5ec2300 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.902+0000 7f02e2da1740 1 --2- 172.21.15.19:0/2959665856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5ec31d0 0x5585f5eb4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.902+0000 7f02e2da1740 1 -- 172.21.15.19:0/2959665856 >> 172.21.15.19:0/2959665856 conn(0x5585f5e966b0 msgr2=0x5585f5e91460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.902+0000 7f02e2da1740 1 -- 172.21.15.19:0/2959665856 shutdown_connections 2024-03-28T13:14:38.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.902+0000 7f02e2da1740 1 -- 172.21.15.19:0/2959665856 wait complete. 2024-03-28T13:14:38.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.903+0000 7f02e2da1740 1 -- start start 2024-03-28T13:14:38.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.904+0000 7f02e2da1740 1 Processor -- start 2024-03-28T13:14:38.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.905+0000 7f02e2da1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec31d0 0x5585f5eb4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.905+0000 7f02e2da1740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5ec35b0 0x5585f5eb60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.905+0000 7f02e2da1740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5edb0f0 0x5585f5ec2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.905+0000 7f02e2da1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5585f5edaa20 con 0x5585f5ec31d0 2024-03-28T13:14:38.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.905+0000 7f02e2da1740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5585f5ed9ec0 con 0x5585f5edb0f0 2024-03-28T13:14:38.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.905+0000 7f02e2da1740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5585f5ec97e0 con 0x5585f5ec35b0 2024-03-28T13:14:38.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.905+0000 7f02e2da1740 10 client.0 _ll_register_callbacks cb 0x7f02d0006ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:14:38.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.906+0000 7f02dd600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec31d0 0x5585f5eb4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.906+0000 7f02dd600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec31d0 0x5585f5eb4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48166/0 (socket says 172.21.15.19:48166) 2024-03-28T13:14:38.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.906+0000 7f02dd600640 1 -- 172.21.15.19:0/1385674452 learned_addr learned my addr 172.21.15.19:0/1385674452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:38.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.906+0000 7f02dd600640 1 -- 172.21.15.19:0/1385674452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5ec35b0 msgr2=0x5585f5eb60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:38.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.906+0000 7f02dd600640 1 --2- 172.21.15.19:0/1385674452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5ec35b0 0x5585f5eb60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.906+0000 7f02dd600640 1 -- 172.21.15.19:0/1385674452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5edb0f0 msgr2=0x5585f5ec2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.906+0000 7f02dd600640 1 --2- 172.21.15.19:0/1385674452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5edb0f0 0x5585f5ec2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.906+0000 7f02dd600640 1 -- 172.21.15.19:0/1385674452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585f5edc6a0 con 0x5585f5ec31d0 2024-03-28T13:14:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.906+0000 7f02dd600640 1 --2- 172.21.15.19:0/1385674452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec31d0 0x5585f5eb4c70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f02cc055130 tx=0x7f02cc05b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd997cbafae0f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.906+0000 7f02be000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:14:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.907+0000 7f02be000640 1 -- 172.21.15.19:0/1385674452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc055340 con 0x5585f5ec31d0 2024-03-28T13:14:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.907+0000 7f02be000640 1 -- 172.21.15.19:0/1385674452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f02cc058120 con 0x5585f5ec31d0 2024-03-28T13:14:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.907+0000 7f02e2da1740 1 -- 172.21.15.19:0/1385674452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02c4031cb0 con 0x5585f5ec31d0 2024-03-28T13:14:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.907+0000 7f02be000640 1 -- 172.21.15.19:0/1385674452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f02cc080050 con 0x5585f5ec31d0 2024-03-28T13:14:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.907+0000 7f02e2da1740 10 client.6645 fetch_fsmap learned FSMap version 45 2024-03-28T13:14:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.907+0000 7f02e2da1740 1 -- 172.21.15.19:0/1385674452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=2+}) -- 0x5585f5e9b750 con 0x5585f5ec31d0 2024-03-28T13:14:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.907+0000 7f02be000640 1 -- 172.21.15.19:0/1385674452 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c4031cb0 con 0x5585f5ec31d0 2024-03-28T13:14:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.907+0000 7f02be000640 1 -- 172.21.15.19:0/1385674452 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 45) ==== 3494+0+0 (secure 0 0 0) 0x7f02cc05f290 con 0x5585f5ec31d0 2024-03-28T13:14:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.907+0000 7f02e2da1740 10 client.6645 fetch_fsmap finished waiting for FSMap version 45 2024-03-28T13:14:38.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.907+0000 7f02e2da1740 10 client.6645 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T13:14:38.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.907+0000 7f02e2da1740 20 client.6645 populate_metadata read hostname 'smithi019' 2024-03-28T13:14:38.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.907+0000 7f02e2da1740 1 --2- 172.21.15.19:0/1385674452 >> [v2:172.21.15.19:6832/1189161682,v1:172.21.15.19:6834/1189161682] conn(0x5585f5f5afd0 0x5585f5ec2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:38.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.907+0000 7f02e2da1740 4 client.6645 mds_command: new command op to 6507 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T13:14:38.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.907+0000 7f02e2da1740 1 -- 172.21.15.19:0/1385674452 --> [v2:172.21.15.19:6832/1189161682,v1:172.21.15.19:6834/1189161682] -- command(tid 0: {"prefix": "status"}) -- 0x5585f5e07820 con 0x5585f5f5afd0 2024-03-28T13:14:38.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.908+0000 7f02dcc00640 1 --2- 172.21.15.19:0/1385674452 >> [v2:172.21.15.19:6832/1189161682,v1:172.21.15.19:6834/1189161682] conn(0x5585f5f5afd0 0x5585f5ec2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:38.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.908+0000 7f02dcc00640 1 --2- 172.21.15.19:0/1385674452 >> [v2:172.21.15.19:6832/1189161682,v1:172.21.15.19:6834/1189161682] conn(0x5585f5f5afd0 0x5585f5ec2840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8646724ddcdd082d server_cookie=de59553ffc75d910 in_seq=0 out_seq=0 2024-03-28T13:14:38.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.908+0000 7f02be000640 10 client.6645 ms_handle_connect on v2:172.21.15.19:6832/1189161682 2024-03-28T13:14:38.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.908+0000 7f02be000640 1 -- 172.21.15.19:0/1385674452 <== mds.0 v2:172.21.15.19:6832/1189161682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5585f5e07820 con 0x5585f5f5afd0 2024-03-28T13:14:38.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.908+0000 7f02be000640 10 client.6645 handle_command_reply: tid=0 2024-03-28T13:14:38.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.908+0000 7f02e2da1740 1 client.6645 shutdown 2024-03-28T13:14:38.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.909+0000 7f02e2da1740 1 -- 172.21.15.19:0/1385674452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec31d0 msgr2=0x5585f5eb4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:38.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.909+0000 7f02e2da1740 1 --2- 172.21.15.19:0/1385674452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec31d0 0x5585f5eb4c70 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f02cc055130 tx=0x7f02cc05b070 comp rx=0 tx=0).stop 2024-03-28T13:14:38.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.909+0000 7f02e2da1740 1 -- 172.21.15.19:0/1385674452 shutdown_connections 2024-03-28T13:14:38.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.909+0000 7f02e2da1740 1 --2- 172.21.15.19:0/1385674452 >> [v2:172.21.15.19:6832/1189161682,v1:172.21.15.19:6834/1189161682] conn(0x5585f5f5afd0 0x5585f5ec2840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.909+0000 7f02e2da1740 1 --2- 172.21.15.19:0/1385674452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5585f5edb0f0 0x5585f5ec2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.909+0000 7f02e2da1740 1 --2- 172.21.15.19:0/1385674452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5585f5ec35b0 0x5585f5eb60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.909+0000 7f02e2da1740 1 --2- 172.21.15.19:0/1385674452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585f5ec31d0 0x5585f5eb4c70 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.909+0000 7f02be000640 0 client.6645 ms_handle_reset on v2:172.21.15.19:6832/1189161682 2024-03-28T13:14:38.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.909+0000 7f02e2da1740 1 -- 172.21.15.19:0/1385674452 >> 172.21.15.19:0/1385674452 conn(0x5585f5e966b0 msgr2=0x5585f5eb7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:38.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.909+0000 7f02e2da1740 1 -- 172.21.15.19:0/1385674452 shutdown_connections 2024-03-28T13:14:38.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.909+0000 7f02e2da1740 1 -- 172.21.15.19:0/1385674452 wait complete. 2024-03-28T13:14:38.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.909+0000 7f02e2da1740 20 client.6645 trim_cache size 0 max 16384 2024-03-28T13:14:38.912 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:14:38.912 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "e6b569d9-f2ce-49dc-bbe7-c0ab33080203", 2024-03-28T13:14:38.912 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-03-28T13:14:38.912 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 6507, 2024-03-28T13:14:38.912 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-03-28T13:14:38.912 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-03-28T13:14:38.912 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-03-28T13:14:38.912 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 6.8707910549999998, 2024-03-28T13:14:38.912 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 45, 2024-03-28T13:14:38.912 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 77, 2024-03-28T13:14:38.912 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 77, 2024-03-28T13:14:38.912 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 13.199703522 2024-03-28T13:14:38.912 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:14:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.910+0000 7f02dea00640 1 -- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f02b806c530 msgr2=0x7f02b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.910+0000 7f02dea00640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f02b806c530 0x7f02b806e9f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f02c4036000 tx=0x7f02c4008030 comp rx=0 tx=0).stop 2024-03-28T13:14:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.910+0000 7f02dea00640 1 -- 172.21.15.19:0/3623476604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d80cef00 msgr2=0x7f02d81530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.910+0000 7f02dea00640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d80cef00 0x7f02d81530e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00c7c0 tx=0x7f02cc00cc80 comp rx=0 tx=0).stop 2024-03-28T13:14:38.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.910+0000 7f02dea00640 1 -- 172.21.15.19:0/3623476604 shutdown_connections 2024-03-28T13:14:38.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.910+0000 7f02dea00640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f02b806c530 0x7f02b806e9f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.910+0000 7f02dea00640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02d815e440 0x7f02d8161860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.910+0000 7f02dea00640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02d80d3e20 0x7f02d8161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.910+0000 7f02dea00640 1 --2- 172.21.15.19:0/3623476604 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02d80cef00 0x7f02d81530e0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:38.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.910+0000 7f02dea00640 1 -- 172.21.15.19:0/3623476604 >> 172.21.15.19:0/3623476604 conn(0x7f02d80678b0 msgr2=0x7f02d80c75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:38.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.910+0000 7f02dea00640 1 -- 172.21.15.19:0/3623476604 shutdown_connections 2024-03-28T13:14:38.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:38.910+0000 7f02dea00640 1 -- 172.21.15.19:0/3623476604 wait complete. 2024-03-28T13:14:38.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:14:39.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.160+0000 7fa6ab200640 1 Processor -- start 2024-03-28T13:14:39.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.160+0000 7fa6ab200640 1 -- start start 2024-03-28T13:14:39.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.161+0000 7fa6ab200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa6ac0cfef0 0x7fa6ac0d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:39.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.162+0000 7fa6ab200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa6ac0cd420 0x7fa6ac0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:39.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.162+0000 7fa6ab200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6ac06df70 con 0x7fa6ac14c360 2024-03-28T13:14:39.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.162+0000 7fa6ab200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa6ac0d0890 con 0x7fa6ac0cd420 2024-03-28T13:14:39.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.162+0000 7fa6ab200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa6ac14c740 con 0x7fa6ac0cfef0 2024-03-28T13:14:39.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.162+0000 7fa6aa800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa6ac0cd420 0x7fa6ac0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:39.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.162+0000 7fa6aa800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa6ac0cd420 0x7fa6ac0cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48174/0 (socket says 172.21.15.19:48174) 2024-03-28T13:14:39.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.162+0000 7fa6aa800640 1 -- 172.21.15.19:0/2015186815 learned_addr learned my addr 172.21.15.19:0/2015186815 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:39.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.162+0000 7fa6a3e00640 1 --2- 172.21.15.19:0/2015186815 >> v2:172.21.15.188:3300/0 conn(0x7fa6ac0cfef0 0x7fa6ac0d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:39.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.163+0000 7fa6aa800640 1 -- 172.21.15.19:0/2015186815 >> v2:172.21.15.188:3300/0 conn(0x7fa6ac0cfef0 msgr2=0x7fa6ac0d0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:39.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.163+0000 7fa6aa800640 1 --2- 172.21.15.19:0/2015186815 >> v2:172.21.15.188:3300/0 conn(0x7fa6ac0cfef0 0x7fa6ac0d0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:39.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.163+0000 7fa6aa800640 1 -- 172.21.15.19:0/2015186815 >> v1:172.21.15.188:6789/0 conn(0x7fa6ac14c360 legacy=0x7fa6ac06d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:39.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.163+0000 7fa6aa800640 1 -- 172.21.15.19:0/2015186815 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac06e250 con 0x7fa6ac0cd420 2024-03-28T13:14:39.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.163+0000 7fa6a3e00640 1 --2- 172.21.15.19:0/2015186815 >> v2:172.21.15.188:3300/0 conn(0x7fa6ac0cfef0 0x7fa6ac0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:39.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.164+0000 7fa6aa800640 1 --2- 172.21.15.19:0/2015186815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ac0cd420 0x7fa6ac0cd860 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac06c430 tx=0x7fa6a402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bb6e16cf855bbc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:39.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.165+0000 7fa6a9400640 1 -- 172.21.15.19:0/2015186815 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a402ca80 con 0x7fa6ac0cd420 2024-03-28T13:14:39.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.165+0000 7fa6a9400640 1 -- 172.21.15.19:0/2015186815 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a402cc20 con 0x7fa6ac0cd420 2024-03-28T13:14:39.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.165+0000 7fa6a9400640 1 -- 172.21.15.19:0/2015186815 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a40343f0 con 0x7fa6ac0cd420 2024-03-28T13:14:39.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.165+0000 7fa6ab200640 1 -- 172.21.15.19:0/2015186815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ac0cd420 msgr2=0x7fa6ac0cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:39.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.165+0000 7fa6ab200640 1 --2- 172.21.15.19:0/2015186815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ac0cd420 0x7fa6ac0cd860 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac06c430 tx=0x7fa6a402bee0 comp rx=0 tx=0).stop 2024-03-28T13:14:39.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.166+0000 7fa6ab200640 1 -- 172.21.15.19:0/2015186815 shutdown_connections 2024-03-28T13:14:39.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.166+0000 7fa6ab200640 1 --2- 172.21.15.19:0/2015186815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ac0cd420 0x7fa6ac0cd860 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:39.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.166+0000 7fa6ab200640 1 --2- 172.21.15.19:0/2015186815 >> v2:172.21.15.188:3300/0 conn(0x7fa6ac0cfef0 0x7fa6ac0d0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:39.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.166+0000 7fa6ab200640 1 -- 172.21.15.19:0/2015186815 >> 172.21.15.19:0/2015186815 conn(0x7fa6ac06b570 msgr2=0x7fa6ac06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:39.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.166+0000 7fa6ab200640 1 -- 172.21.15.19:0/2015186815 shutdown_connections 2024-03-28T13:14:39.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.166+0000 7fa6ab200640 1 -- 172.21.15.19:0/2015186815 wait complete. 2024-03-28T13:14:39.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.168+0000 7fa6ab200640 1 Processor -- start 2024-03-28T13:14:39.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.168+0000 7fa6ab200640 1 -- start start 2024-03-28T13:14:39.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.169+0000 7fa6ab200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6ac0cd420 0x7fa6ac1ecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:39.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.169+0000 7fa6ab200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ac0cfef0 0x7fa6ac0c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:39.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.169+0000 7fa6ab200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ac14c360 0x7fa6ac0c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:39.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.169+0000 7fa6ab200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac0ccf00 con 0x7fa6ac0cfef0 2024-03-28T13:14:39.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.169+0000 7fa6ab200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac0cc9d0 con 0x7fa6ac0cd420 2024-03-28T13:14:39.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.169+0000 7fa6ab200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ac06dc00 con 0x7fa6ac14c360 2024-03-28T13:14:39.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.169+0000 7fa6a3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ac0cfef0 0x7fa6ac0c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:39.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.169+0000 7fa6a3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ac0cfef0 0x7fa6ac0c6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48176/0 (socket says 172.21.15.19:48176) 2024-03-28T13:14:39.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.169+0000 7fa6a3e00640 1 -- 172.21.15.19:0/161500646 learned_addr learned my addr 172.21.15.19:0/161500646 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:39.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.169+0000 7fa6a9e00640 1 --2- 172.21.15.19:0/161500646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6ac0cd420 0x7fa6ac1ecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:39.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.169+0000 7fa6aa800640 1 --2- 172.21.15.19:0/161500646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ac14c360 0x7fa6ac0c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:39.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.170+0000 7fa6a3e00640 1 -- 172.21.15.19:0/161500646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ac14c360 msgr2=0x7fa6ac0c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:39.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.170+0000 7fa6a3e00640 1 --2- 172.21.15.19:0/161500646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ac14c360 0x7fa6ac0c7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:39.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.170+0000 7fa6a3e00640 1 -- 172.21.15.19:0/161500646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6ac0cd420 msgr2=0x7fa6ac1ecfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:39.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.170+0000 7fa6a3e00640 1 --2- 172.21.15.19:0/161500646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6ac0cd420 0x7fa6ac1ecfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:39.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.170+0000 7fa6a3e00640 1 -- 172.21.15.19:0/161500646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac1483e0 con 0x7fa6ac0cfef0 2024-03-28T13:14:39.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.170+0000 7fa6a9e00640 1 --2- 172.21.15.19:0/161500646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6ac0cd420 0x7fa6ac1ecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:39.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.170+0000 7fa6a3e00640 1 --2- 172.21.15.19:0/161500646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ac0cfef0 0x7fa6ac0c6d30 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c7f0 tx=0x7fa69400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:39.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.170+0000 7fa6a2000640 1 -- 172.21.15.19:0/161500646 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694015070 con 0x7fa6ac0cfef0 2024-03-28T13:14:39.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.170+0000 7fa6a2000640 1 -- 172.21.15.19:0/161500646 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa694007a30 con 0x7fa6ac0cfef0 2024-03-28T13:14:39.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.170+0000 7fa6a2000640 1 -- 172.21.15.19:0/161500646 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa694007e00 con 0x7fa6ac0cfef0 2024-03-28T13:14:39.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.170+0000 7fa6ab200640 1 -- 172.21.15.19:0/161500646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6ac0c78a0 con 0x7fa6ac0cfef0 2024-03-28T13:14:39.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.170+0000 7fa6ab200640 1 -- 172.21.15.19:0/161500646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6ac1ec7e0 con 0x7fa6ac0cfef0 2024-03-28T13:14:39.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.172+0000 7fa6ab200640 1 -- 172.21.15.19:0/161500646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ac075940 con 0x7fa6ac0cfef0 2024-03-28T13:14:39.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.172+0000 7fa6a2000640 1 -- 172.21.15.19:0/161500646 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa694018070 con 0x7fa6ac0cfef0 2024-03-28T13:14:39.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.173+0000 7fa6a2000640 1 --2- 172.21.15.19:0/161500646 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa68806c530 0x7fa68806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:39.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.173+0000 7fa6a2000640 1 -- 172.21.15.19:0/161500646 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7fa69408b6c0 con 0x7fa6ac0cfef0 2024-03-28T13:14:39.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.174+0000 7fa6a9e00640 1 --2- 172.21.15.19:0/161500646 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa68806c530 0x7fa68806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:39.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.175+0000 7fa6a9e00640 1 --2- 172.21.15.19:0/161500646 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa68806c530 0x7fa68806e9f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa69c007fd0 tx=0x7fa69c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:39.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.178+0000 7fa6a2000640 1 -- 172.21.15.19:0/161500646 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa694059cd0 con 0x7fa6ac0cfef0 2024-03-28T13:14:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.421+0000 7fa6ab200640 1 -- 172.21.15.19:0/161500646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa6ac06c580 con 0x7fa6ac0cfef0 2024-03-28T13:14:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.421+0000 7fa6a2000640 1 -- 172.21.15.19:0/161500646 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v45) ==== 53+0+73 (secure 0 0 0) 0x7fa6940595d0 con 0x7fa6ac0cfef0 2024-03-28T13:14:39.421 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:14:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.424+0000 7fa6ab200640 1 -- 172.21.15.19:0/161500646 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa68806c530 msgr2=0x7fa68806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.424+0000 7fa6ab200640 1 --2- 172.21.15.19:0/161500646 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa68806c530 0x7fa68806e9f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa69c007fd0 tx=0x7fa69c009040 comp rx=0 tx=0).stop 2024-03-28T13:14:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.424+0000 7fa6ab200640 1 -- 172.21.15.19:0/161500646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ac0cfef0 msgr2=0x7fa6ac0c6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.424+0000 7fa6ab200640 1 --2- 172.21.15.19:0/161500646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ac0cfef0 0x7fa6ac0c6d30 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c7f0 tx=0x7fa69400cbf0 comp rx=0 tx=0).stop 2024-03-28T13:14:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.424+0000 7fa6ab200640 1 -- 172.21.15.19:0/161500646 shutdown_connections 2024-03-28T13:14:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.424+0000 7fa6ab200640 1 --2- 172.21.15.19:0/161500646 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa68806c530 0x7fa68806e9f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.424+0000 7fa6ab200640 1 --2- 172.21.15.19:0/161500646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6ac14c360 0x7fa6ac0c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.424+0000 7fa6ab200640 1 --2- 172.21.15.19:0/161500646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6ac0cfef0 0x7fa6ac0c6d30 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.424+0000 7fa6ab200640 1 --2- 172.21.15.19:0/161500646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6ac0cd420 0x7fa6ac1ecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.424+0000 7fa6ab200640 1 -- 172.21.15.19:0/161500646 >> 172.21.15.19:0/161500646 conn(0x7fa6ac06b570 msgr2=0x7fa6ac0cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.425+0000 7fa6ab200640 1 -- 172.21.15.19:0/161500646 shutdown_connections 2024-03-28T13:14:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39.425+0000 7fa6ab200640 1 -- 172.21.15.19:0/161500646 wait complete. 2024-03-28T13:14:39.444 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-28T13:14:39.444 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:14:39.444 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:14:39.444 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:14:39.445 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:14:39.445 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-28T13:14:39.445 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-28T13:14:39.445 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:14:39.445 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:14:39.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:39.445 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:14:39.491 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:14:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:14:39.491 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:14:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:14:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:14:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:14:39.492 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:14:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:14:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:14:39.492 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:14:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:14:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:14:39.492 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:14:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:14:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:14:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:14:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42275sec preferred_lft 42275sec 2024-03-28T13:14:39.492 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:14:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:14:39.493 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:14:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:14:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:14:39.493 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:14:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:14:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:14:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:14:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:14:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:14:39.493 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:14:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:14:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:14:39.493 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:14:39.494 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:14:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether e6:17:c5:8e:2b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:14:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::e417:c5ff:fe8e:2bcf/64 scope link 2024-03-28T13:14:39.494 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:14:39.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:39.495 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:14:39.627 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:14:39.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:39.634 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:14:39.686 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:14:39.687 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:14:39.687 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:14:39.687 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:14:39.687 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:14:39.687 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:14:39.687 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:14:39.687 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:14:39.687 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:14:39.688 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:14:39.688 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:14:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:14:39.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:14:39.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:39.940 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:14:39.992 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:14:39.992 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:14:39.993 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:14:39.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:39.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:14:40.087 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:14:40.087 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:14:40.139 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:14:40.139 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:14:40.140 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:14:40.187 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:14:40.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:40.187 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:14:40.317 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:14:40.317 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:14:40.317 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:14:40.337 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:14:40.337 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:14:40.337 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:14:40.337 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:14:40.337 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:14:40.365 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:14:40.365 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:14:40.366 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:14:40.366 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:14:40.366 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:14:40.366 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:14:40.366 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:14:40.366 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:14:40.368 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:14:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:14:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:14:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:14:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:14:40.372 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:14:40.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:40.372 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:14:40.453 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:14:40.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:14:40.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:14:40.584 INFO:teuthology.orchestra.run.smithi019.stdout:client5773 2024-03-28T13:14:40.585 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:14:40.585 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:14:40.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:40.634 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-03-28T13:14:40.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:40.736 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-03-28T13:14:40.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:40.792 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-03-28T13:14:40.847 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:14:40.847 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:14:40.848 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:14:40.848 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:14:40.848 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-03-28T13:14:40.848 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:14:40.993 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627776 2024-03-28T13:14:40.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:14:41.008 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:14:41.008 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:14:41.103 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:14:41.104 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:14:41.153 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:14:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.409+0000 7fc172e00640 1 Processor -- start 2024-03-28T13:14:41.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.410+0000 7fc172e00640 1 -- start start 2024-03-28T13:14:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.411+0000 7fc172e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc16c13f100 0x7fc16c06fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:41.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.411+0000 7fc172e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc16c070080 0x7fc16c0d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.411+0000 7fc172e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc16c0704a0 con 0x7fc16c0d26e0 2024-03-28T13:14:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.411+0000 7fc172e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc16c142270 con 0x7fc16c13f100 2024-03-28T13:14:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.411+0000 7fc172e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc16c0d2ed0 con 0x7fc16c070080 2024-03-28T13:14:41.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.412+0000 7fc172400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc16c0d26e0 0x7fc16c15fd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48196/0 (socket says 172.21.15.19:48196) 2024-03-28T13:14:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.412+0000 7fc171a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc16c13f100 0x7fc16c06fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.412+0000 7fc172400640 1 -- 172.21.15.19:0/1530160820 learned_addr learned my addr 172.21.15.19:0/1530160820 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.412+0000 7fc171000640 1 --2- 172.21.15.19:0/1530160820 >> v2:172.21.15.188:3300/0 conn(0x7fc16c070080 0x7fc16c0d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.412+0000 7fc171a00640 1 -- 172.21.15.19:0/1530160820 >> v2:172.21.15.188:3300/0 conn(0x7fc16c070080 msgr2=0x7fc16c0d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.412+0000 7fc171a00640 1 --2- 172.21.15.19:0/1530160820 >> v2:172.21.15.188:3300/0 conn(0x7fc16c070080 0x7fc16c0d21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.412+0000 7fc171a00640 1 -- 172.21.15.19:0/1530160820 >> v1:172.21.15.19:6789/0 conn(0x7fc16c0d26e0 legacy=0x7fc16c15fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.412+0000 7fc171a00640 1 -- 172.21.15.19:0/1530160820 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16c13f4e0 con 0x7fc16c13f100 2024-03-28T13:14:41.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.412+0000 7fc16be00640 1 -- 172.21.15.19:0/1530160820 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 395090656 0 0) 0x7fc16c0704a0 con 0x7fc16c0d26e0 2024-03-28T13:14:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.414+0000 7fc171a00640 1 --2- 172.21.15.19:0/1530160820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c13f100 0x7fc16c06fb40 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc16c071130 tx=0x7fc16002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5c82dcbdb55c17 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.414+0000 7fc16be00640 1 -- 172.21.15.19:0/1530160820 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160035020 con 0x7fc16c13f100 2024-03-28T13:14:41.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.414+0000 7fc16be00640 1 -- 172.21.15.19:0/1530160820 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc160031830 con 0x7fc16c13f100 2024-03-28T13:14:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.414+0000 7fc16be00640 1 -- 172.21.15.19:0/1530160820 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160031c00 con 0x7fc16c13f100 2024-03-28T13:14:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.414+0000 7fc172e00640 1 -- 172.21.15.19:0/1530160820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c13f100 msgr2=0x7fc16c06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.414+0000 7fc172e00640 1 --2- 172.21.15.19:0/1530160820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c13f100 0x7fc16c06fb40 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fc16c071130 tx=0x7fc16002b8d0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.415+0000 7fc172e00640 1 -- 172.21.15.19:0/1530160820 shutdown_connections 2024-03-28T13:14:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.415+0000 7fc172e00640 1 --2- 172.21.15.19:0/1530160820 >> v2:172.21.15.188:3300/0 conn(0x7fc16c070080 0x7fc16c0d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.415+0000 7fc172e00640 1 --2- 172.21.15.19:0/1530160820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c13f100 0x7fc16c06fb40 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.415+0000 7fc172e00640 1 -- 172.21.15.19:0/1530160820 >> 172.21.15.19:0/1530160820 conn(0x7fc16c067460 msgr2=0x7fc16c067840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.415+0000 7fc172e00640 1 -- 172.21.15.19:0/1530160820 shutdown_connections 2024-03-28T13:14:41.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.415+0000 7fc172e00640 1 -- 172.21.15.19:0/1530160820 wait complete. 2024-03-28T13:14:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.416+0000 7fc172e00640 1 Processor -- start 2024-03-28T13:14:41.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.417+0000 7fc172e00640 1 -- start start 2024-03-28T13:14:41.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc172e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c070080 0x7fc16c13dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:41.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc172e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0d26e0 0x7fc16c0b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:41.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc172e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c13f100 0x7fc16c140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:41.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc172e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c141f00 con 0x7fc16c13f100 2024-03-28T13:14:41.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc172e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c141c20 con 0x7fc16c070080 2024-03-28T13:14:41.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc172e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc16c142700 con 0x7fc16c0d26e0 2024-03-28T13:14:41.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc172400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c13f100 0x7fc16c140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:41.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc172400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c13f100 0x7fc16c140720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48186/0 (socket says 172.21.15.19:48186) 2024-03-28T13:14:41.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc172400640 1 -- 172.21.15.19:0/1777776588 learned_addr learned my addr 172.21.15.19:0/1777776588 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:41.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc171a00640 1 --2- 172.21.15.19:0/1777776588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c070080 0x7fc16c13dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:41.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc171000640 1 --2- 172.21.15.19:0/1777776588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0d26e0 0x7fc16c0b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:41.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc172400640 1 -- 172.21.15.19:0/1777776588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0d26e0 msgr2=0x7fc16c0b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:41.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc172400640 1 --2- 172.21.15.19:0/1777776588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0d26e0 0x7fc16c0b3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc172400640 1 -- 172.21.15.19:0/1777776588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c070080 msgr2=0x7fc16c13dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:41.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc172400640 1 --2- 172.21.15.19:0/1777776588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c070080 0x7fc16c13dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc172400640 1 -- 172.21.15.19:0/1777776588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16c0b3b10 con 0x7fc16c13f100 2024-03-28T13:14:41.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.418+0000 7fc171a00640 1 --2- 172.21.15.19:0/1777776588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c070080 0x7fc16c13dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:41.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.419+0000 7fc172400640 1 --2- 172.21.15.19:0/1777776588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c13f100 0x7fc16c140720 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc16400c7a0 tx=0x7fc16400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:41.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.419+0000 7fc169600640 1 -- 172.21.15.19:0/1777776588 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc164018070 con 0x7fc16c13f100 2024-03-28T13:14:41.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.419+0000 7fc169600640 1 -- 172.21.15.19:0/1777776588 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc16400a640 con 0x7fc16c13f100 2024-03-28T13:14:41.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.419+0000 7fc169600640 1 -- 172.21.15.19:0/1777776588 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16400aab0 con 0x7fc16c13f100 2024-03-28T13:14:41.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.419+0000 7fc172e00640 1 -- 172.21.15.19:0/1777776588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16c0b3df0 con 0x7fc16c13f100 2024-03-28T13:14:41.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.419+0000 7fc172e00640 1 -- 172.21.15.19:0/1777776588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16c0b43b0 con 0x7fc16c13f100 2024-03-28T13:14:41.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.420+0000 7fc172e00640 1 -- 172.21.15.19:0/1777776588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc16c0685c0 con 0x7fc16c13f100 2024-03-28T13:14:41.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.422+0000 7fc169600640 1 -- 172.21.15.19:0/1777776588 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc16400ac50 con 0x7fc16c13f100 2024-03-28T13:14:41.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.423+0000 7fc169600640 1 --2- 172.21.15.19:0/1777776588 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc14c06c530 0x7fc14c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:41.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.423+0000 7fc169600640 1 -- 172.21.15.19:0/1777776588 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7fc16408acd0 con 0x7fc16c13f100 2024-03-28T13:14:41.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.423+0000 7fc171a00640 1 --2- 172.21.15.19:0/1777776588 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc14c06c530 0x7fc14c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:41.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.425+0000 7fc171a00640 1 --2- 172.21.15.19:0/1777776588 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc14c06c530 0x7fc14c06e9f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc160036000 tx=0x7fc160008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:41.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.432+0000 7fc169600640 1 -- 172.21.15.19:0/1777776588 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc16c0685c0 con 0x7fc16c13f100 2024-03-28T13:14:41.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.639+0000 7fc172e00640 1 -- 172.21.15.19:0/1777776588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc16c075940 con 0x7fc16c13f100 2024-03-28T13:14:41.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.642+0000 7fc169600640 1 -- 172.21.15.19:0/1777776588 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+16608 (secure 0 0 0) 0x7fc164059380 con 0x7fc16c13f100 2024-03-28T13:14:41.642 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:41.642 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":77,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:14:33.055510+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:14:29.424080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-28T13:14:30.494667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:14:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.645+0000 7fc172e00640 1 -- 172.21.15.19:0/1777776588 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc14c06c530 msgr2=0x7fc14c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.645+0000 7fc172e00640 1 --2- 172.21.15.19:0/1777776588 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc14c06c530 0x7fc14c06e9f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc160036000 tx=0x7fc160008030 comp rx=0 tx=0).stop 2024-03-28T13:14:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.645+0000 7fc172e00640 1 -- 172.21.15.19:0/1777776588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c13f100 msgr2=0x7fc16c140720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:41.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.645+0000 7fc172e00640 1 --2- 172.21.15.19:0/1777776588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c13f100 0x7fc16c140720 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fc16400c7a0 tx=0x7fc16400cc60 comp rx=0 tx=0).stop 2024-03-28T13:14:41.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.646+0000 7fc172e00640 1 -- 172.21.15.19:0/1777776588 shutdown_connections 2024-03-28T13:14:41.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.646+0000 7fc172e00640 1 --2- 172.21.15.19:0/1777776588 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc14c06c530 0x7fc14c06e9f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.646+0000 7fc172e00640 1 --2- 172.21.15.19:0/1777776588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc16c13f100 0x7fc16c140720 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.646+0000 7fc172e00640 1 --2- 172.21.15.19:0/1777776588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0d26e0 0x7fc16c0b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.646+0000 7fc172e00640 1 --2- 172.21.15.19:0/1777776588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c070080 0x7fc16c13dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.646+0000 7fc172e00640 1 -- 172.21.15.19:0/1777776588 >> 172.21.15.19:0/1777776588 conn(0x7fc16c067460 msgr2=0x7fc16c0b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:41.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.646+0000 7fc172e00640 1 -- 172.21.15.19:0/1777776588 shutdown_connections 2024-03-28T13:14:41.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.646+0000 7fc172e00640 1 -- 172.21.15.19:0/1777776588 wait complete. 2024-03-28T13:14:41.666 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:14:29.424080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:14:30.494667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:14:41.666 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:14:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.880+0000 7fac60400640 1 Processor -- start 2024-03-28T13:14:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.880+0000 7fac60400640 1 -- start start 2024-03-28T13:14:41.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.882+0000 7fac60400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fac5815e6d0 0x7fac580d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:41.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.882+0000 7fac60400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fac580d4760 0x7fac580cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:41.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.882+0000 7fac60400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac580ceef0 con 0x7fac580d0cb0 2024-03-28T13:14:41.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.882+0000 7fac60400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fac5815eab0 con 0x7fac580d4760 2024-03-28T13:14:41.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.882+0000 7fac60400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fac580d1220 con 0x7fac5815e6d0 2024-03-28T13:14:41.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.882+0000 7fac5e400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fac580d4760 0x7fac580cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:41.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.882+0000 7fac5d000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fac580d0cb0 0x7fac580d4050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48202/0 (socket says 172.21.15.19:48202) 2024-03-28T13:14:41.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.882+0000 7fac5d000640 1 -- 172.21.15.19:0/1489667162 learned_addr learned my addr 172.21.15.19:0/1489667162 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:41.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.883+0000 7fac5da00640 1 --2- 172.21.15.19:0/1489667162 >> v2:172.21.15.188:3300/0 conn(0x7fac5815e6d0 0x7fac580d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:41.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.883+0000 7fac5e400640 1 -- 172.21.15.19:0/1489667162 >> v2:172.21.15.188:3300/0 conn(0x7fac5815e6d0 msgr2=0x7fac580d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:41.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.883+0000 7fac5e400640 1 --2- 172.21.15.19:0/1489667162 >> v2:172.21.15.188:3300/0 conn(0x7fac5815e6d0 0x7fac580d21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.883+0000 7fac5e400640 1 -- 172.21.15.19:0/1489667162 >> v1:172.21.15.19:6789/0 conn(0x7fac580d0cb0 legacy=0x7fac580d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:41.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.883+0000 7fac5e400640 1 -- 172.21.15.19:0/1489667162 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac580d1500 con 0x7fac580d4760 2024-03-28T13:14:41.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.883+0000 7fac57e00640 1 -- 172.21.15.19:0/1489667162 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3506486590 0 0) 0x7fac580ceef0 con 0x7fac580d0cb0 2024-03-28T13:14:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.884+0000 7fac5e400640 1 --2- 172.21.15.19:0/1489667162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac580d4760 0x7fac580cf3d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fac5808cdc0 tx=0x7fac5002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728841cfd51233da server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.884+0000 7fac57e00640 1 -- 172.21.15.19:0/1489667162 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50039070 con 0x7fac580d4760 2024-03-28T13:14:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.884+0000 7fac57e00640 1 -- 172.21.15.19:0/1489667162 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac5002cc80 con 0x7fac580d4760 2024-03-28T13:14:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.884+0000 7fac57e00640 1 -- 172.21.15.19:0/1489667162 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50034660 con 0x7fac580d4760 2024-03-28T13:14:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.885+0000 7fac60400640 1 -- 172.21.15.19:0/1489667162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac580d4760 msgr2=0x7fac580cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:41.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.885+0000 7fac60400640 1 --2- 172.21.15.19:0/1489667162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac580d4760 0x7fac580cf3d0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fac5808cdc0 tx=0x7fac5002bed0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.885+0000 7fac60400640 1 -- 172.21.15.19:0/1489667162 shutdown_connections 2024-03-28T13:14:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.885+0000 7fac60400640 1 --2- 172.21.15.19:0/1489667162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac580d4760 0x7fac580cf3d0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.885+0000 7fac60400640 1 --2- 172.21.15.19:0/1489667162 >> v2:172.21.15.188:3300/0 conn(0x7fac5815e6d0 0x7fac580d21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.885+0000 7fac60400640 1 -- 172.21.15.19:0/1489667162 >> 172.21.15.19:0/1489667162 conn(0x7fac5809d7b0 msgr2=0x7fac5808bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.885+0000 7fac60400640 1 -- 172.21.15.19:0/1489667162 shutdown_connections 2024-03-28T13:14:41.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.885+0000 7fac60400640 1 -- 172.21.15.19:0/1489667162 wait complete. 2024-03-28T13:14:41.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.887+0000 7fac60400640 1 Processor -- start 2024-03-28T13:14:41.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.887+0000 7fac60400640 1 -- start start 2024-03-28T13:14:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.888+0000 7fac60400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac580d0cb0 0x7fac58150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.888+0000 7fac60400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac580d4760 0x7fac5809f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.888+0000 7fac60400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac5815e6d0 0x7fac5809f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.888+0000 7fac60400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fac580d2720 con 0x7fac580d0cb0 2024-03-28T13:14:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.888+0000 7fac60400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fac580d1ac0 con 0x7fac580d4760 2024-03-28T13:14:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.888+0000 7fac60400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fac5806c890 con 0x7fac5815e6d0 2024-03-28T13:14:41.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.889+0000 7fac5da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac580d0cb0 0x7fac58150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:41.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.889+0000 7fac5da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac580d0cb0 0x7fac58150890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48212/0 (socket says 172.21.15.19:48212) 2024-03-28T13:14:41.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.889+0000 7fac5da00640 1 -- 172.21.15.19:0/1277134768 learned_addr learned my addr 172.21.15.19:0/1277134768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:41.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.889+0000 7fac5d000640 1 --2- 172.21.15.19:0/1277134768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac580d4760 0x7fac5809f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:41.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.889+0000 7fac5e400640 1 --2- 172.21.15.19:0/1277134768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac5815e6d0 0x7fac5809f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:41.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.889+0000 7fac5da00640 1 -- 172.21.15.19:0/1277134768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac5815e6d0 msgr2=0x7fac5809f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:41.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.889+0000 7fac5da00640 1 --2- 172.21.15.19:0/1277134768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac5815e6d0 0x7fac5809f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.889+0000 7fac5da00640 1 -- 172.21.15.19:0/1277134768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac580d4760 msgr2=0x7fac5809f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:41.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.889+0000 7fac5da00640 1 --2- 172.21.15.19:0/1277134768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac580d4760 0x7fac5809f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:41.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.889+0000 7fac5da00640 1 -- 172.21.15.19:0/1277134768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac580a1280 con 0x7fac580d0cb0 2024-03-28T13:14:41.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.890+0000 7fac5da00640 1 --2- 172.21.15.19:0/1277134768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac580d0cb0 0x7fac58150890 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00d800 tx=0x7fac4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:41.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.890+0000 7fac55600640 1 -- 172.21.15.19:0/1277134768 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c00b6c0 con 0x7fac580d0cb0 2024-03-28T13:14:41.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.890+0000 7fac55600640 1 -- 172.21.15.19:0/1277134768 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac4c019070 con 0x7fac580d0cb0 2024-03-28T13:14:41.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.890+0000 7fac55600640 1 -- 172.21.15.19:0/1277134768 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c00ba80 con 0x7fac580d0cb0 2024-03-28T13:14:41.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.890+0000 7fac60400640 1 -- 172.21.15.19:0/1277134768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac580a1560 con 0x7fac580d0cb0 2024-03-28T13:14:41.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.890+0000 7fac60400640 1 -- 172.21.15.19:0/1277134768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac580a1b20 con 0x7fac580d0cb0 2024-03-28T13:14:41.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.891+0000 7fac60400640 1 -- 172.21.15.19:0/1277134768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac580685c0 con 0x7fac580d0cb0 2024-03-28T13:14:41.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.892+0000 7fac55600640 1 -- 172.21.15.19:0/1277134768 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fac4c00bc20 con 0x7fac580d0cb0 2024-03-28T13:14:41.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.893+0000 7fac55600640 1 --2- 172.21.15.19:0/1277134768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac3806c530 0x7fac3806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:41.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.893+0000 7fac55600640 1 -- 172.21.15.19:0/1277134768 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7fac4c08bcb0 con 0x7fac580d0cb0 2024-03-28T13:14:41.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.894+0000 7fac5d000640 1 --2- 172.21.15.19:0/1277134768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac3806c530 0x7fac3806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:41.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.895+0000 7fac5d000640 1 --2- 172.21.15.19:0/1277134768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac3806c530 0x7fac3806e9f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fac5815def0 tx=0x7fac44009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:41.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:41.898+0000 7fac55600640 1 -- 172.21.15.19:0/1277134768 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fac4c05a1a0 con 0x7fac580d0cb0 2024-03-28T13:14:42.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.138+0000 7fac60400640 1 -- 172.21.15.19:0/1277134768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fac5806c670 con 0x7fac580d0cb0 2024-03-28T13:14:42.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.139+0000 7fac55600640 1 -- 172.21.15.19:0/1277134768 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fac4c059aa0 con 0x7fac580d0cb0 2024-03-28T13:14:42.139 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:14:42.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.141+0000 7fac60400640 1 -- 172.21.15.19:0/1277134768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac3806c530 msgr2=0x7fac3806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:42.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.141+0000 7fac60400640 1 --2- 172.21.15.19:0/1277134768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac3806c530 0x7fac3806e9f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fac5815def0 tx=0x7fac44009040 comp rx=0 tx=0).stop 2024-03-28T13:14:42.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.141+0000 7fac60400640 1 -- 172.21.15.19:0/1277134768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac580d0cb0 msgr2=0x7fac58150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:42.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.141+0000 7fac60400640 1 --2- 172.21.15.19:0/1277134768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac580d0cb0 0x7fac58150890 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00d800 tx=0x7fac4c00dcc0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.141+0000 7fac60400640 1 -- 172.21.15.19:0/1277134768 shutdown_connections 2024-03-28T13:14:42.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.141+0000 7fac60400640 1 --2- 172.21.15.19:0/1277134768 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fac3806c530 0x7fac3806e9f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.141+0000 7fac60400640 1 --2- 172.21.15.19:0/1277134768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fac5815e6d0 0x7fac5809f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.141+0000 7fac60400640 1 --2- 172.21.15.19:0/1277134768 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fac580d4760 0x7fac5809f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.141+0000 7fac60400640 1 --2- 172.21.15.19:0/1277134768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fac580d0cb0 0x7fac58150890 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.141+0000 7fac60400640 1 -- 172.21.15.19:0/1277134768 >> 172.21.15.19:0/1277134768 conn(0x7fac5809d7b0 msgr2=0x7fac5806dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:42.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.142+0000 7fac60400640 1 -- 172.21.15.19:0/1277134768 shutdown_connections 2024-03-28T13:14:42.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.142+0000 7fac60400640 1 -- 172.21.15.19:0/1277134768 wait complete. 2024-03-28T13:14:42.159 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:14:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.377+0000 7fe244a00640 1 Processor -- start 2024-03-28T13:14:42.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.377+0000 7fe244a00640 1 -- start start 2024-03-28T13:14:42.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.379+0000 7fe244a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe2400d0990 0x7fe2400aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:42.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.379+0000 7fe244a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2400af2e0 0x7fe2401600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:42.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.379+0000 7fe244a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2400afab0 con 0x7fe240153580 2024-03-28T13:14:42.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.379+0000 7fe244a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe24015cc80 con 0x7fe2400d0990 2024-03-28T13:14:42.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.379+0000 7fe244a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe24015da00 con 0x7fe2400af2e0 2024-03-28T13:14:42.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.379+0000 7fe23d400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe2400d0990 0x7fe2400aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:42.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.379+0000 7fe23d400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe2400d0990 0x7fe2400aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48222/0 (socket says 172.21.15.19:48222) 2024-03-28T13:14:42.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.379+0000 7fe23d400640 1 -- 172.21.15.19:0/632426150 learned_addr learned my addr 172.21.15.19:0/632426150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:42.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.380+0000 7fe23e800640 1 --2- 172.21.15.19:0/632426150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2400af2e0 0x7fe2401600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:42.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.380+0000 7fe23d400640 1 -- 172.21.15.19:0/632426150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2400af2e0 msgr2=0x7fe2401600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:42.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.380+0000 7fe23d400640 1 --2- 172.21.15.19:0/632426150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2400af2e0 0x7fe2401600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.380+0000 7fe23d400640 1 -- 172.21.15.19:0/632426150 >> v1:172.21.15.188:6789/0 conn(0x7fe240153580 legacy=0x7fe2400d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:42.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.380+0000 7fe23d400640 1 -- 172.21.15.19:0/632426150 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24015cf20 con 0x7fe2400d0990 2024-03-28T13:14:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.381+0000 7fe23d400640 1 --2- 172.21.15.19:0/632426150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2400d0990 0x7fe2400aeda0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009870 tx=0x7fe22c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b9f02beda8d93 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.381+0000 7fe23ca00640 1 -- 172.21.15.19:0/632426150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c0317c0 con 0x7fe2400d0990 2024-03-28T13:14:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.381+0000 7fe23ca00640 1 -- 172.21.15.19:0/632426150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe22c037070 con 0x7fe2400d0990 2024-03-28T13:14:42.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.381+0000 7fe23ca00640 1 -- 172.21.15.19:0/632426150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe22c031b80 con 0x7fe2400d0990 2024-03-28T13:14:42.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.382+0000 7fe244a00640 1 -- 172.21.15.19:0/632426150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2400d0990 msgr2=0x7fe2400aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:42.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.382+0000 7fe244a00640 1 --2- 172.21.15.19:0/632426150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2400d0990 0x7fe2400aeda0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009870 tx=0x7fe22c02b860 comp rx=0 tx=0).stop 2024-03-28T13:14:42.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.382+0000 7fe244a00640 1 -- 172.21.15.19:0/632426150 shutdown_connections 2024-03-28T13:14:42.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.382+0000 7fe244a00640 1 --2- 172.21.15.19:0/632426150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2400d0990 0x7fe2400aeda0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.382+0000 7fe244a00640 1 --2- 172.21.15.19:0/632426150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2400af2e0 0x7fe2401600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.382+0000 7fe244a00640 1 -- 172.21.15.19:0/632426150 >> 172.21.15.19:0/632426150 conn(0x7fe24006b750 msgr2=0x7fe24006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:42.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.383+0000 7fe244a00640 1 -- 172.21.15.19:0/632426150 shutdown_connections 2024-03-28T13:14:42.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.383+0000 7fe244a00640 1 -- 172.21.15.19:0/632426150 wait complete. 2024-03-28T13:14:42.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.384+0000 7fe244a00640 1 Processor -- start 2024-03-28T13:14:42.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.384+0000 7fe244a00640 1 -- start start 2024-03-28T13:14:42.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.385+0000 7fe244a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2400af2e0 0x7fe2400caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:42.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.385+0000 7fe244a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2400d0990 0x7fe24014fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:42.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.385+0000 7fe244a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe240153580 0x7fe240150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:42.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.385+0000 7fe244a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe240153960 con 0x7fe2400af2e0 2024-03-28T13:14:42.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.385+0000 7fe244a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe24015d4e0 con 0x7fe240153580 2024-03-28T13:14:42.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.385+0000 7fe244a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe2400af710 con 0x7fe2400d0990 2024-03-28T13:14:42.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.386+0000 7fe23de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2400af2e0 0x7fe2400caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:42.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.386+0000 7fe23de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2400af2e0 0x7fe2400caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48224/0 (socket says 172.21.15.19:48224) 2024-03-28T13:14:42.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.386+0000 7fe23de00640 1 -- 172.21.15.19:0/3167628089 learned_addr learned my addr 172.21.15.19:0/3167628089 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:42.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.386+0000 7fe23de00640 1 -- 172.21.15.19:0/3167628089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2400d0990 msgr2=0x7fe24014fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:42.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.386+0000 7fe23de00640 1 --2- 172.21.15.19:0/3167628089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2400d0990 0x7fe24014fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.386+0000 7fe23de00640 1 -- 172.21.15.19:0/3167628089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe240153580 msgr2=0x7fe240150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:42.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.386+0000 7fe23de00640 1 --2- 172.21.15.19:0/3167628089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe240153580 0x7fe240150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.386+0000 7fe23de00640 1 -- 172.21.15.19:0/3167628089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2400cbb50 con 0x7fe2400af2e0 2024-03-28T13:14:42.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.386+0000 7fe23de00640 1 --2- 172.21.15.19:0/3167628089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2400af2e0 0x7fe2400caad0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe23000c460 tx=0x7fe23000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:42.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.387+0000 7fe236000640 1 -- 172.21.15.19:0/3167628089 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230016020 con 0x7fe2400af2e0 2024-03-28T13:14:42.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.387+0000 7fe236000640 1 -- 172.21.15.19:0/3167628089 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe230012870 con 0x7fe2400af2e0 2024-03-28T13:14:42.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.387+0000 7fe236000640 1 -- 172.21.15.19:0/3167628089 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230012c60 con 0x7fe2400af2e0 2024-03-28T13:14:42.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.387+0000 7fe244a00640 1 -- 172.21.15.19:0/3167628089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe240150380 con 0x7fe2400af2e0 2024-03-28T13:14:42.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.387+0000 7fe244a00640 1 -- 172.21.15.19:0/3167628089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2400c9d10 con 0x7fe2400af2e0 2024-03-28T13:14:42.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.388+0000 7fe244a00640 1 -- 172.21.15.19:0/3167628089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2400685c0 con 0x7fe2400af2e0 2024-03-28T13:14:42.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.389+0000 7fe236000640 1 -- 172.21.15.19:0/3167628089 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe230019020 con 0x7fe2400af2e0 2024-03-28T13:14:42.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.390+0000 7fe236000640 1 --2- 172.21.15.19:0/3167628089 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe22006c530 0x7fe22006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:42.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.390+0000 7fe236000640 1 -- 172.21.15.19:0/3167628089 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7fe23008b170 con 0x7fe2400af2e0 2024-03-28T13:14:42.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.391+0000 7fe23d400640 1 --2- 172.21.15.19:0/3167628089 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe22006c530 0x7fe22006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:42.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.392+0000 7fe23d400640 1 --2- 172.21.15.19:0/3167628089 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe22006c530 0x7fe22006e9f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe22c036000 tx=0x7fe22c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:42.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.394+0000 7fe236000640 1 -- 172.21.15.19:0/3167628089 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe230020070 con 0x7fe2400af2e0 2024-03-28T13:14:42.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.597+0000 7fe244a00640 1 -- 172.21.15.19:0/3167628089 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe2400b4d50 con 0x7fe2400af2e0 2024-03-28T13:14:42.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.598+0000 7fe236000640 1 -- 172.21.15.19:0/3167628089 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7fe2400cbb50 con 0x7fe2400af2e0 2024-03-28T13:14:42.598 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:14:42.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.601+0000 7fe244a00640 1 -- 172.21.15.19:0/3167628089 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe22006c530 msgr2=0x7fe22006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:42.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.601+0000 7fe244a00640 1 --2- 172.21.15.19:0/3167628089 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe22006c530 0x7fe22006e9f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe22c036000 tx=0x7fe22c030000 comp rx=0 tx=0).stop 2024-03-28T13:14:42.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.601+0000 7fe244a00640 1 -- 172.21.15.19:0/3167628089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2400af2e0 msgr2=0x7fe2400caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:42.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.601+0000 7fe244a00640 1 --2- 172.21.15.19:0/3167628089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2400af2e0 0x7fe2400caad0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe23000c460 tx=0x7fe23000c920 comp rx=0 tx=0).stop 2024-03-28T13:14:42.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.601+0000 7fe244a00640 1 -- 172.21.15.19:0/3167628089 shutdown_connections 2024-03-28T13:14:42.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.601+0000 7fe244a00640 1 --2- 172.21.15.19:0/3167628089 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe22006c530 0x7fe22006e9f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.601+0000 7fe244a00640 1 --2- 172.21.15.19:0/3167628089 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe240153580 0x7fe240150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.601+0000 7fe244a00640 1 --2- 172.21.15.19:0/3167628089 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2400d0990 0x7fe24014fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.601+0000 7fe244a00640 1 --2- 172.21.15.19:0/3167628089 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2400af2e0 0x7fe2400caad0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.601+0000 7fe244a00640 1 -- 172.21.15.19:0/3167628089 >> 172.21.15.19:0/3167628089 conn(0x7fe24006b750 msgr2=0x7fe24006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:42.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.601+0000 7fe244a00640 1 -- 172.21.15.19:0/3167628089 shutdown_connections 2024-03-28T13:14:42.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.601+0000 7fe244a00640 1 -- 172.21.15.19:0/3167628089 wait complete. 2024-03-28T13:14:42.621 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:14:42.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.838+0000 7f62af200640 1 Processor -- start 2024-03-28T13:14:42.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.839+0000 7f62af200640 1 -- start start 2024-03-28T13:14:42.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.840+0000 7f62af200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62b006bc80 0x7f62b00702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:42.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.840+0000 7f62af200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62b00cfe80 con 0x7f62b00707e0 2024-03-28T13:14:42.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.840+0000 7f62af200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62b00d0160 con 0x7f62b0147c80 2024-03-28T13:14:42.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.840+0000 7f62af200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f62b0070c70 con 0x7f62b006bc80 2024-03-28T13:14:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.841+0000 7f62ae800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f62b00707e0 0x7f62b00724f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48204/0 (socket says 172.21.15.19:48204) 2024-03-28T13:14:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.841+0000 7f62ae800640 1 -- 172.21.15.19:0/1955948628 learned_addr learned my addr 172.21.15.19:0/1955948628 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.841+0000 7f62ad400640 1 --2- 172.21.15.19:0/1955948628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62b006bc80 0x7f62b00702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.841+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3419778292 0 0) 0x7f62b00cfe80 con 0x7f62b00707e0 2024-03-28T13:14:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.841+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6294003520 con 0x7f62b00707e0 2024-03-28T13:14:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2851377601 0 0) 0x7f62b00d0160 con 0x7f62b0147c80 2024-03-28T13:14:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62b00cfe80 con 0x7f62b0147c80 2024-03-28T13:14:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 402629399 0 0) 0x7f6294003520 con 0x7f62b00707e0 2024-03-28T13:14:42.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62b006bc80 msgr2=0x7f62b00702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62aca00640 1 --2- 172.21.15.19:0/1955948628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62b006bc80 0x7f62b00702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 >> v1:172.21.15.188:6789/0 conn(0x7f62b0147c80 legacy=0x7f62b006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b00d0440 con 0x7f62b00707e0 2024-03-28T13:14:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f62a80019f0 con 0x7f62b00707e0 2024-03-28T13:14:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 >> v1:172.21.15.19:6789/0 conn(0x7f62b00707e0 legacy=0x7f62b00724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62aca00640 1 --2- 172.21.15.19:0/1955948628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6294003b70 0x7f6294003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62aca00640 1 --2- 172.21.15.19:0/1955948628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6294004800 0x7f6294004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62aca00640 1 --2- 172.21.15.19:0/1955948628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62b006bc80 0x7f62940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f629400ba70 con 0x7f6294003b70 2024-03-28T13:14:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f629400bbf0 con 0x7f6294004800 2024-03-28T13:14:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f629400bdc0 con 0x7f62b006bc80 2024-03-28T13:14:42.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.842+0000 7f62ade00640 1 --2- 172.21.15.19:0/1955948628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6294003b70 0x7f6294003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.843+0000 7f62ad400640 1 --2- 172.21.15.19:0/1955948628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6294004800 0x7f6294004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.843+0000 7f62ae800640 1 --2- 172.21.15.19:0/1955948628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62b006bc80 0x7f62940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.843+0000 7f62ade00640 1 -- 172.21.15.19:0/1955948628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62b006bc80 msgr2=0x7f62940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.843+0000 7f62ade00640 1 --2- 172.21.15.19:0/1955948628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62b006bc80 0x7f62940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.843+0000 7f62ade00640 1 -- 172.21.15.19:0/1955948628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6294004800 msgr2=0x7f6294004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.843+0000 7f62ade00640 1 --2- 172.21.15.19:0/1955948628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6294004800 0x7f6294004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.843+0000 7f62ade00640 1 -- 172.21.15.19:0/1955948628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6294003520 con 0x7f6294003b70 2024-03-28T13:14:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.843+0000 7f62ae800640 1 --2- 172.21.15.19:0/1955948628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62b006bc80 0x7f62940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.843+0000 7f62ad400640 1 --2- 172.21.15.19:0/1955948628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6294004800 0x7f6294004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.844+0000 7f62ade00640 1 --2- 172.21.15.19:0/1955948628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6294003b70 0x7f6294003f70 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f62a00018d0 tx=0x7f6294005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fd1ff9788b86e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.844+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a00014f0 con 0x7f6294003b70 2024-03-28T13:14:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.844+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a002f050 con 0x7f6294003b70 2024-03-28T13:14:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.844+0000 7f62aca00640 1 -- 172.21.15.19:0/1955948628 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a00035f0 con 0x7f6294003b70 2024-03-28T13:14:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.845+0000 7f62af200640 1 -- 172.21.15.19:0/1955948628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6294003b70 msgr2=0x7f6294003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.845+0000 7f62af200640 1 --2- 172.21.15.19:0/1955948628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6294003b70 0x7f6294003f70 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f62a00018d0 tx=0x7f6294005830 comp rx=0 tx=0).stop 2024-03-28T13:14:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.845+0000 7f62ae800640 1 -- 172.21.15.19:0/1955948628 reap_dead start 2024-03-28T13:14:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.845+0000 7f62af200640 1 -- 172.21.15.19:0/1955948628 shutdown_connections 2024-03-28T13:14:42.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.845+0000 7f62af200640 1 -- 172.21.15.19:0/1955948628 >> 172.21.15.19:0/1955948628 conn(0x7f62b006e730 msgr2=0x7f62b006eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.845+0000 7f62af200640 1 -- 172.21.15.19:0/1955948628 shutdown_connections 2024-03-28T13:14:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.845+0000 7f62af200640 1 -- 172.21.15.19:0/1955948628 wait complete. 2024-03-28T13:14:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.847+0000 7f62af200640 1 Processor -- start 2024-03-28T13:14:42.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.847+0000 7f62af200640 1 -- start start 2024-03-28T13:14:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.848+0000 7f62af200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6294003b70 0x7f62b0159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.849+0000 7f62af200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b01e6ba0 0x7f62b01e7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.849+0000 7f62af200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62b0157630 0x7f62b0157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.849+0000 7f62af200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62b00d0a90 con 0x7f62b01e6ba0 2024-03-28T13:14:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.849+0000 7f62af200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f62b00d0f20 con 0x7f62b0157630 2024-03-28T13:14:42.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.849+0000 7f62af200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f62b00cfb50 con 0x7f6294003b70 2024-03-28T13:14:42.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.849+0000 7f62ad400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b01e6ba0 0x7f62b01e7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:42.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.849+0000 7f62ad400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b01e6ba0 0x7f62b01e7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48230/0 (socket says 172.21.15.19:48230) 2024-03-28T13:14:42.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.849+0000 7f62ad400640 1 -- 172.21.15.19:0/3700632433 learned_addr learned my addr 172.21.15.19:0/3700632433 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:42.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.849+0000 7f62ade00640 1 --2- 172.21.15.19:0/3700632433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6294003b70 0x7f62b0159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:42.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.849+0000 7f62ae800640 1 --2- 172.21.15.19:0/3700632433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62b0157630 0x7f62b0157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:42.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.849+0000 7f62ad400640 1 -- 172.21.15.19:0/3700632433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6294003b70 msgr2=0x7f62b0159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:42.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.849+0000 7f62ad400640 1 --2- 172.21.15.19:0/3700632433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6294003b70 0x7f62b0159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.849+0000 7f62ad400640 1 -- 172.21.15.19:0/3700632433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62b0157630 msgr2=0x7f62b0157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:42.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.849+0000 7f62ad400640 1 --2- 172.21.15.19:0/3700632433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62b0157630 0x7f62b0157ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:42.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.849+0000 7f62ad400640 1 -- 172.21.15.19:0/3700632433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b01e7660 con 0x7f62b01e6ba0 2024-03-28T13:14:42.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.850+0000 7f62ad400640 1 --2- 172.21.15.19:0/3700632433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b01e6ba0 0x7f62b01e7020 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f629c0108b0 tx=0x7f629c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:42.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.850+0000 7f62a6000640 1 -- 172.21.15.19:0/3700632433 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c00f040 con 0x7f62b01e6ba0 2024-03-28T13:14:42.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.850+0000 7f62a6000640 1 -- 172.21.15.19:0/3700632433 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f629c00eb60 con 0x7f62b01e6ba0 2024-03-28T13:14:42.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.850+0000 7f62af200640 1 -- 172.21.15.19:0/3700632433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b01e7940 con 0x7f62b01e6ba0 2024-03-28T13:14:42.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.850+0000 7f62a6000640 1 -- 172.21.15.19:0/3700632433 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c01c400 con 0x7f62b01e6ba0 2024-03-28T13:14:42.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.850+0000 7f62af200640 1 -- 172.21.15.19:0/3700632433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b01e7f00 con 0x7f62b01e6ba0 2024-03-28T13:14:42.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.851+0000 7f62af200640 1 -- 172.21.15.19:0/3700632433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b0075940 con 0x7f62b01e6ba0 2024-03-28T13:14:42.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.853+0000 7f62a6000640 1 -- 172.21.15.19:0/3700632433 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f629c01c5a0 con 0x7f62b01e6ba0 2024-03-28T13:14:42.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.854+0000 7f62a6000640 1 --2- 172.21.15.19:0/3700632433 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f629006c530 0x7f629006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:42.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.854+0000 7f62a6000640 1 -- 172.21.15.19:0/3700632433 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f629c08bea0 con 0x7f62b01e6ba0 2024-03-28T13:14:42.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.855+0000 7f62ade00640 1 --2- 172.21.15.19:0/3700632433 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f629006c530 0x7f629006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:42.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.856+0000 7f62ade00640 1 --2- 172.21.15.19:0/3700632433 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f629006c530 0x7f629006e9f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f62a0034000 tx=0x7f62a0031000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:42.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:42.858+0000 7f62a6000640 1 -- 172.21.15.19:0/3700632433 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f629c018030 con 0x7f62b01e6ba0 2024-03-28T13:14:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.080+0000 7f62af200640 1 -- 172.21.15.19:0/3700632433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f62b0146f90 con 0x7f62b01e6ba0 2024-03-28T13:14:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.081+0000 7f62a6000640 1 -- 172.21.15.19:0/3700632433 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f629c05a420 con 0x7f62b01e6ba0 2024-03-28T13:14:43.081 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:14:43.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.084+0000 7f62af200640 1 -- 172.21.15.19:0/3700632433 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f629006c530 msgr2=0x7f629006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:43.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.084+0000 7f62af200640 1 --2- 172.21.15.19:0/3700632433 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f629006c530 0x7f629006e9f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f62a0034000 tx=0x7f62a0031000 comp rx=0 tx=0).stop 2024-03-28T13:14:43.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.084+0000 7f62af200640 1 -- 172.21.15.19:0/3700632433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b01e6ba0 msgr2=0x7f62b01e7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:43.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.084+0000 7f62af200640 1 --2- 172.21.15.19:0/3700632433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b01e6ba0 0x7f62b01e7020 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f629c0108b0 tx=0x7f629c010d70 comp rx=0 tx=0).stop 2024-03-28T13:14:43.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.084+0000 7f62af200640 1 -- 172.21.15.19:0/3700632433 shutdown_connections 2024-03-28T13:14:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.084+0000 7f62af200640 1 --2- 172.21.15.19:0/3700632433 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f629006c530 0x7f629006e9f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.084+0000 7f62af200640 1 --2- 172.21.15.19:0/3700632433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62b0157630 0x7f62b0157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.084+0000 7f62af200640 1 --2- 172.21.15.19:0/3700632433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62b01e6ba0 0x7f62b01e7020 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.084+0000 7f62af200640 1 --2- 172.21.15.19:0/3700632433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6294003b70 0x7f62b0159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.084+0000 7f62af200640 1 -- 172.21.15.19:0/3700632433 >> 172.21.15.19:0/3700632433 conn(0x7f62b006e730 msgr2=0x7f62b015fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.084+0000 7f62af200640 1 -- 172.21.15.19:0/3700632433 shutdown_connections 2024-03-28T13:14:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.084+0000 7f62af200640 1 -- 172.21.15.19:0/3700632433 wait complete. 2024-03-28T13:14:43.104 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:14:43.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.324+0000 7f422fa00640 1 Processor -- start 2024-03-28T13:14:43.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.324+0000 7f422fa00640 1 -- start start 2024-03-28T13:14:43.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.326+0000 7f422fa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f422806c580 0x7f422813e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:43.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.326+0000 7f422fa00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f422815f110 0x7f42280b4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:43.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.326+0000 7f422fa00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42280b3a30 con 0x7f422806c960 2024-03-28T13:14:43.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.326+0000 7f422fa00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f4228155830 con 0x7f422815f110 2024-03-28T13:14:43.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.326+0000 7f422fa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f42280b42f0 con 0x7f422806c580 2024-03-28T13:14:43.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.326+0000 7f4227e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f422806c960 0x7f422815ea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48220/0 (socket says 172.21.15.19:48220) 2024-03-28T13:14:43.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.326+0000 7f4227e00640 1 -- 172.21.15.19:0/2984966934 learned_addr learned my addr 172.21.15.19:0/2984966934 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.326+0000 7f422da00640 1 --2- 172.21.15.19:0/2984966934 >> v2:172.21.15.188:3300/0 conn(0x7f422815f110 0x7f42280b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.327+0000 7f422d000640 1 --2- 172.21.15.19:0/2984966934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f422806c580 0x7f422813e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.327+0000 7f4227400640 1 -- 172.21.15.19:0/2984966934 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3990184507 0 0) 0x7f42280b3a30 con 0x7f422806c960 2024-03-28T13:14:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.327+0000 7f4227400640 1 -- 172.21.15.19:0/2984966934 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f420c003520 con 0x7f422806c960 2024-03-28T13:14:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.327+0000 7f422da00640 1 -- 172.21.15.19:0/2984966934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f422806c580 msgr2=0x7f422813e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.327+0000 7f422da00640 1 --2- 172.21.15.19:0/2984966934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f422806c580 0x7f422813e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.327+0000 7f422da00640 1 -- 172.21.15.19:0/2984966934 >> v1:172.21.15.19:6789/0 conn(0x7f422806c960 legacy=0x7f422815ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.327+0000 7f422da00640 1 -- 172.21.15.19:0/2984966934 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42280b3ca0 con 0x7f422815f110 2024-03-28T13:14:43.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.329+0000 7f422da00640 1 --2- 172.21.15.19:0/2984966934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f422815f110 0x7f42280b4780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f42200097a0 tx=0x7f422002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22d76fca507b556d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:43.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.329+0000 7f4227400640 1 -- 172.21.15.19:0/2984966934 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220031780 con 0x7f422815f110 2024-03-28T13:14:43.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.329+0000 7f4227400640 1 -- 172.21.15.19:0/2984966934 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4220037070 con 0x7f422815f110 2024-03-28T13:14:43.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.329+0000 7f4227400640 1 -- 172.21.15.19:0/2984966934 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220031b40 con 0x7f422815f110 2024-03-28T13:14:43.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.330+0000 7f422fa00640 1 -- 172.21.15.19:0/2984966934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f422815f110 msgr2=0x7f42280b4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:43.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.330+0000 7f422fa00640 1 --2- 172.21.15.19:0/2984966934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f422815f110 0x7f42280b4780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f42200097a0 tx=0x7f422002b820 comp rx=0 tx=0).stop 2024-03-28T13:14:43.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.330+0000 7f422fa00640 1 -- 172.21.15.19:0/2984966934 shutdown_connections 2024-03-28T13:14:43.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.330+0000 7f422fa00640 1 --2- 172.21.15.19:0/2984966934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f422815f110 0x7f42280b4780 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:43.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.330+0000 7f422fa00640 1 --2- 172.21.15.19:0/2984966934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f422806c580 0x7f422813e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:43.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.330+0000 7f422fa00640 1 -- 172.21.15.19:0/2984966934 >> 172.21.15.19:0/2984966934 conn(0x7f4228071e60 msgr2=0x7f4228072270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.330+0000 7f422fa00640 1 -- 172.21.15.19:0/2984966934 shutdown_connections 2024-03-28T13:14:43.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.330+0000 7f422fa00640 1 -- 172.21.15.19:0/2984966934 wait complete. 2024-03-28T13:14:43.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.332+0000 7f422fa00640 1 Processor -- start 2024-03-28T13:14:43.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.332+0000 7f422fa00640 1 -- start start 2024-03-28T13:14:43.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.333+0000 7f422fa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f422806c580 0x7f4228143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:43.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.333+0000 7f422fa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f422806c960 0x7f4228143b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:43.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.333+0000 7f422fa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f422815f110 0x7f422813c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:43.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.333+0000 7f422fa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f422806cdc0 con 0x7f422815f110 2024-03-28T13:14:43.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.333+0000 7f422fa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f422815f780 con 0x7f422806c580 2024-03-28T13:14:43.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.333+0000 7f422fa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f422813df10 con 0x7f422806c960 2024-03-28T13:14:43.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.334+0000 7f422da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f422815f110 0x7f422813c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:43.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.334+0000 7f422da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f422815f110 0x7f422813c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48232/0 (socket says 172.21.15.19:48232) 2024-03-28T13:14:43.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.334+0000 7f422da00640 1 -- 172.21.15.19:0/715999484 learned_addr learned my addr 172.21.15.19:0/715999484 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:43.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.334+0000 7f422d000640 1 --2- 172.21.15.19:0/715999484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f422806c580 0x7f4228143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:43.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.334+0000 7f4227e00640 1 --2- 172.21.15.19:0/715999484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f422806c960 0x7f4228143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:43.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.334+0000 7f422da00640 1 -- 172.21.15.19:0/715999484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f422806c960 msgr2=0x7f4228143b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:43.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.334+0000 7f422da00640 1 --2- 172.21.15.19:0/715999484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f422806c960 0x7f4228143b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:43.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.334+0000 7f422da00640 1 -- 172.21.15.19:0/715999484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f422806c580 msgr2=0x7f4228143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:43.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.334+0000 7f422da00640 1 --2- 172.21.15.19:0/715999484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f422806c580 0x7f4228143610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:43.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.334+0000 7f422da00640 1 -- 172.21.15.19:0/715999484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422814a840 con 0x7f422815f110 2024-03-28T13:14:43.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.334+0000 7f422d000640 1 --2- 172.21.15.19:0/715999484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f422806c580 0x7f4228143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:14:43.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.335+0000 7f422da00640 1 --2- 172.21.15.19:0/715999484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f422815f110 0x7f422813c6a0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f42200097a0 tx=0x7f42200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:43.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.335+0000 7f4224c00640 1 -- 172.21.15.19:0/715999484 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220032610 con 0x7f422815f110 2024-03-28T13:14:43.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.335+0000 7f4224c00640 1 -- 172.21.15.19:0/715999484 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42200327b0 con 0x7f422815f110 2024-03-28T13:14:43.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.335+0000 7f4224c00640 1 -- 172.21.15.19:0/715999484 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220032ba0 con 0x7f422815f110 2024-03-28T13:14:43.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.335+0000 7f422fa00640 1 -- 172.21.15.19:0/715999484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42281fc2e0 con 0x7f422815f110 2024-03-28T13:14:43.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.335+0000 7f422fa00640 1 -- 172.21.15.19:0/715999484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42281fc8a0 con 0x7f422815f110 2024-03-28T13:14:43.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.336+0000 7f422fa00640 1 -- 172.21.15.19:0/715999484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42281fcaa0 con 0x7f422815f110 2024-03-28T13:14:43.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.337+0000 7f4224c00640 1 -- 172.21.15.19:0/715999484 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f422004c060 con 0x7f422815f110 2024-03-28T13:14:43.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.338+0000 7f4224c00640 1 --2- 172.21.15.19:0/715999484 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f420406c530 0x7f420406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:43.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.338+0000 7f4224c00640 1 -- 172.21.15.19:0/715999484 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f42200acb70 con 0x7f422815f110 2024-03-28T13:14:43.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.339+0000 7f422d000640 1 --2- 172.21.15.19:0/715999484 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f420406c530 0x7f420406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:43.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.340+0000 7f422d000640 1 --2- 172.21.15.19:0/715999484 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f420406c530 0x7f420406e9f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f42180074c0 tx=0x7f421800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:43.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.343+0000 7f4224c00640 1 -- 172.21.15.19:0/715999484 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f42281fcaa0 con 0x7f422815f110 2024-03-28T13:14:43.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.560+0000 7f422fa00640 1 -- 172.21.15.19:0/715999484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f42280685c0 con 0x7f422815f110 2024-03-28T13:14:43.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.561+0000 7f4224c00640 1 -- 172.21.15.19:0/715999484 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v77) ==== 80+0+0 (secure 0 0 0) 0x7f4220040070 con 0x7f422815f110 2024-03-28T13:14:43.562 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-03-28T13:14:43.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.564+0000 7f422fa00640 1 -- 172.21.15.19:0/715999484 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f420406c530 msgr2=0x7f420406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:43.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.564+0000 7f422fa00640 1 --2- 172.21.15.19:0/715999484 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f420406c530 0x7f420406e9f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f42180074c0 tx=0x7f421800b040 comp rx=0 tx=0).stop 2024-03-28T13:14:43.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.564+0000 7f422fa00640 1 -- 172.21.15.19:0/715999484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f422815f110 msgr2=0x7f422813c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:43.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.564+0000 7f422fa00640 1 --2- 172.21.15.19:0/715999484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f422815f110 0x7f422813c6a0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f42200097a0 tx=0x7f42200046c0 comp rx=0 tx=0).stop 2024-03-28T13:14:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.564+0000 7f422fa00640 1 -- 172.21.15.19:0/715999484 shutdown_connections 2024-03-28T13:14:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.564+0000 7f422fa00640 1 --2- 172.21.15.19:0/715999484 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f420406c530 0x7f420406e9f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.564+0000 7f422fa00640 1 --2- 172.21.15.19:0/715999484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f422815f110 0x7f422813c6a0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.564+0000 7f422fa00640 1 --2- 172.21.15.19:0/715999484 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f422806c960 0x7f4228143b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.564+0000 7f422fa00640 1 --2- 172.21.15.19:0/715999484 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f422806c580 0x7f4228143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.564+0000 7f422fa00640 1 -- 172.21.15.19:0/715999484 >> 172.21.15.19:0/715999484 conn(0x7f4228071e60 msgr2=0x7f422813f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.564+0000 7f422fa00640 1 -- 172.21.15.19:0/715999484 shutdown_connections 2024-03-28T13:14:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.564+0000 7f422fa00640 1 -- 172.21.15.19:0/715999484 wait complete. 2024-03-28T13:14:43.583 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:14:43.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:43.584 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T13:14:43.730 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:14:43.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:43.730 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:14:43.785 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:14:43.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.997+0000 7f0971800640 1 Processor -- start 2024-03-28T13:14:43.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.997+0000 7f0971800640 1 -- start start 2024-03-28T13:14:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.999+0000 7f0971800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f096c0d2310 0x7f096c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.999+0000 7f0971800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f096c0cf530 0x7f096c0b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.999+0000 7f0971800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f096c15e880 con 0x7f096c15e4a0 2024-03-28T13:14:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.999+0000 7f0971800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f096c0cf9a0 con 0x7f096c0d2310 2024-03-28T13:14:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:43.999+0000 7f0971800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f096c0d34e0 con 0x7f096c0cf530 2024-03-28T13:14:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.000+0000 7f096b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f096c0d2310 0x7f096c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.000+0000 7f096be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f096c15e4a0 0x7f096c0b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48228/0 (socket says 172.21.15.19:48228) 2024-03-28T13:14:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.000+0000 7f096be00640 1 -- 172.21.15.19:0/3783847979 learned_addr learned my addr 172.21.15.19:0/3783847979 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.000+0000 7f0970e00640 1 --2- 172.21.15.19:0/3783847979 >> v2:172.21.15.188:3300/0 conn(0x7f096c0cf530 0x7f096c0b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.000+0000 7f096b400640 1 -- 172.21.15.19:0/3783847979 >> v2:172.21.15.188:3300/0 conn(0x7f096c0cf530 msgr2=0x7f096c0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.000+0000 7f096b400640 1 --2- 172.21.15.19:0/3783847979 >> v2:172.21.15.188:3300/0 conn(0x7f096c0cf530 0x7f096c0b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.000+0000 7f096b400640 1 -- 172.21.15.19:0/3783847979 >> v1:172.21.15.19:6789/0 conn(0x7f096c15e4a0 legacy=0x7f096c0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.000+0000 7f096b400640 1 -- 172.21.15.19:0/3783847979 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096c0d28d0 con 0x7f096c0d2310 2024-03-28T13:14:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.000+0000 7f096aa00640 1 -- 172.21.15.19:0/3783847979 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3428065137 0 0) 0x7f096c15e880 con 0x7f096c15e4a0 2024-03-28T13:14:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.000+0000 7f0970e00640 1 --2- 172.21.15.19:0/3783847979 >> v2:172.21.15.188:3300/0 conn(0x7f096c0cf530 0x7f096c0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.001+0000 7f096b400640 1 --2- 172.21.15.19:0/3783847979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f096c0d2310 0x7f096c0ceff0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f096c0d3660 tx=0x7f095402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e541ba76e786e55 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.002+0000 7f096aa00640 1 -- 172.21.15.19:0/3783847979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09540315a0 con 0x7f096c0d2310 2024-03-28T13:14:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.002+0000 7f096aa00640 1 -- 172.21.15.19:0/3783847979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0954037070 con 0x7f096c0d2310 2024-03-28T13:14:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.002+0000 7f096aa00640 1 -- 172.21.15.19:0/3783847979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954031960 con 0x7f096c0d2310 2024-03-28T13:14:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.002+0000 7f0971800640 1 -- 172.21.15.19:0/3783847979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f096c0d2310 msgr2=0x7f096c0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.002+0000 7f0971800640 1 --2- 172.21.15.19:0/3783847979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f096c0d2310 0x7f096c0ceff0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f096c0d3660 tx=0x7f095402b840 comp rx=0 tx=0).stop 2024-03-28T13:14:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.003+0000 7f0971800640 1 -- 172.21.15.19:0/3783847979 shutdown_connections 2024-03-28T13:14:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.003+0000 7f0971800640 1 --2- 172.21.15.19:0/3783847979 >> v2:172.21.15.188:3300/0 conn(0x7f096c0cf530 0x7f096c0b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.003+0000 7f0971800640 1 --2- 172.21.15.19:0/3783847979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f096c0d2310 0x7f096c0ceff0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.003+0000 7f0971800640 1 -- 172.21.15.19:0/3783847979 >> 172.21.15.19:0/3783847979 conn(0x7f096c0676d0 msgr2=0x7f096c067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.003+0000 7f0971800640 1 -- 172.21.15.19:0/3783847979 shutdown_connections 2024-03-28T13:14:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.003+0000 7f0971800640 1 -- 172.21.15.19:0/3783847979 wait complete. 2024-03-28T13:14:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.004+0000 7f0971800640 1 Processor -- start 2024-03-28T13:14:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.005+0000 7f0971800640 1 -- start start 2024-03-28T13:14:44.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.005+0000 7f0971800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f096c0cf530 0x7f096c152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:44.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.005+0000 7f0971800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f096c0d2310 0x7f096c0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:44.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.005+0000 7f0971800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f096c15e4a0 0x7f096c0c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:44.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.006+0000 7f0971800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f096c0d3170 con 0x7f096c0d2310 2024-03-28T13:14:44.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.006+0000 7f0971800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f096c0d2e90 con 0x7f096c15e4a0 2024-03-28T13:14:44.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.006+0000 7f0971800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f096c0d3970 con 0x7f096c0cf530 2024-03-28T13:14:44.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.006+0000 7f096b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f096c0d2310 0x7f096c0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:44.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.006+0000 7f096b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f096c0d2310 0x7f096c0c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48250/0 (socket says 172.21.15.19:48250) 2024-03-28T13:14:44.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.006+0000 7f096b400640 1 -- 172.21.15.19:0/616110981 learned_addr learned my addr 172.21.15.19:0/616110981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.006+0000 7f096be00640 1 --2- 172.21.15.19:0/616110981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f096c0cf530 0x7f096c152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.006+0000 7f0970e00640 1 --2- 172.21.15.19:0/616110981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f096c15e4a0 0x7f096c0c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.006+0000 7f096b400640 1 -- 172.21.15.19:0/616110981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f096c0cf530 msgr2=0x7f096c152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.006+0000 7f096b400640 1 --2- 172.21.15.19:0/616110981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f096c0cf530 0x7f096c152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.006+0000 7f096b400640 1 -- 172.21.15.19:0/616110981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f096c15e4a0 msgr2=0x7f096c0c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.006+0000 7f096b400640 1 --2- 172.21.15.19:0/616110981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f096c15e4a0 0x7f096c0c6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.006+0000 7f096b400640 1 -- 172.21.15.19:0/616110981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096c1532b0 con 0x7f096c0d2310 2024-03-28T13:14:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.006+0000 7f096be00640 1 --2- 172.21.15.19:0/616110981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f096c0cf530 0x7f096c152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.007+0000 7f096b400640 1 --2- 172.21.15.19:0/616110981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f096c0d2310 0x7f096c0c7dc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f0954009860 tx=0x7f09540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.007+0000 7f0963e00640 1 -- 172.21.15.19:0/616110981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954032840 con 0x7f096c0d2310 2024-03-28T13:14:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.007+0000 7f0963e00640 1 -- 172.21.15.19:0/616110981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0954032a00 con 0x7f096c0d2310 2024-03-28T13:14:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.007+0000 7f0963e00640 1 -- 172.21.15.19:0/616110981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954032df0 con 0x7f096c0d2310 2024-03-28T13:14:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.007+0000 7f0971800640 1 -- 172.21.15.19:0/616110981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096c152fd0 con 0x7f096c0d2310 2024-03-28T13:14:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.007+0000 7f0971800640 1 -- 172.21.15.19:0/616110981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f096c151d50 con 0x7f096c0d2310 2024-03-28T13:14:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.009+0000 7f0971800640 1 -- 172.21.15.19:0/616110981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f096c075940 con 0x7f096c0d2310 2024-03-28T13:14:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.009+0000 7f0963e00640 1 -- 172.21.15.19:0/616110981 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f095404c020 con 0x7f096c0d2310 2024-03-28T13:14:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.010+0000 7f0963e00640 1 --2- 172.21.15.19:0/616110981 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f094806c560 0x7f094806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.010+0000 7f0963e00640 1 -- 172.21.15.19:0/616110981 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f095402f020 con 0x7f096c0d2310 2024-03-28T13:14:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.011+0000 7f096be00640 1 --2- 172.21.15.19:0/616110981 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f094806c560 0x7f094806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.012+0000 7f096be00640 1 --2- 172.21.15.19:0/616110981 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f094806c560 0x7f094806ea20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0964007fd0 tx=0x7f0964009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:44.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.015+0000 7f0963e00640 1 -- 172.21.15.19:0/616110981 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0954040070 con 0x7f096c0d2310 2024-03-28T13:14:44.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.256+0000 7f0971800640 1 -- 172.21.15.19:0/616110981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f096c0b48e0 con 0x7f096c0d2310 2024-03-28T13:14:44.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.257+0000 7f0963e00640 1 -- 172.21.15.19:0/616110981 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+5186 (secure 0 0 0) 0x7f095407bb30 con 0x7f096c0d2310 2024-03-28T13:14:44.257 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:44.257 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/144726418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":144726418},{"type":"v1","addr":"172.21.15.188:6836","nonce":144726418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5363,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2642581607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2642581607},{"type":"v1","addr":"172.21.15.19:6835","nonce":2642581607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1721039756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1721039756},{"type":"v1","addr":"172.21.15.188:6837","nonce":1721039756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:14:31.526577+0000","modified":"2024-03-28T13:14:34.573581+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6507},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6507":{"gid":6507,"name":"c","rank":0,"incarnation":41,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/1189161682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1189161682},{"type":"v1","addr":"172.21.15.19:6834","nonce":1189161682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6507,"qdb_cluster":[6507]},"id":5}]} 2024-03-28T13:14:44.257 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 45 2024-03-28T13:14:44.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.259+0000 7f0971800640 1 -- 172.21.15.19:0/616110981 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f094806c560 msgr2=0x7f094806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:44.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.259+0000 7f0971800640 1 --2- 172.21.15.19:0/616110981 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f094806c560 0x7f094806ea20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0964007fd0 tx=0x7f0964009040 comp rx=0 tx=0).stop 2024-03-28T13:14:44.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.259+0000 7f0971800640 1 -- 172.21.15.19:0/616110981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f096c0d2310 msgr2=0x7f096c0c7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:44.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.259+0000 7f0971800640 1 --2- 172.21.15.19:0/616110981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f096c0d2310 0x7f096c0c7dc0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f0954009860 tx=0x7f09540046c0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.259+0000 7f0971800640 1 -- 172.21.15.19:0/616110981 shutdown_connections 2024-03-28T13:14:44.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.259+0000 7f0971800640 1 --2- 172.21.15.19:0/616110981 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f094806c560 0x7f094806ea20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.259+0000 7f0971800640 1 --2- 172.21.15.19:0/616110981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f096c15e4a0 0x7f096c0c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.259+0000 7f0971800640 1 --2- 172.21.15.19:0/616110981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f096c0d2310 0x7f096c0c7dc0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.259+0000 7f0971800640 1 --2- 172.21.15.19:0/616110981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f096c0cf530 0x7f096c152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.259+0000 7f0971800640 1 -- 172.21.15.19:0/616110981 >> 172.21.15.19:0/616110981 conn(0x7f096c0676d0 msgr2=0x7f096c0b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:44.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.260+0000 7f0971800640 1 -- 172.21.15.19:0/616110981 shutdown_connections 2024-03-28T13:14:44.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.260+0000 7f0971800640 1 -- 172.21.15.19:0/616110981 wait complete. 2024-03-28T13:14:44.278 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-28T13:14:44.535 INFO:teuthology.orchestra.run.smithi019.stdout:{"message":"","return_code":0} 2024-03-28T13:14:44.535 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-28T13:14:44.535 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T13:14:44.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.744+0000 7f5e36a00640 1 Processor -- start 2024-03-28T13:14:44.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.745+0000 7f5e36a00640 1 -- start start 2024-03-28T13:14:44.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.747+0000 7f5e36a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5e30160a30 0x7f5e3006b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:44.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.747+0000 7f5e36a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e30145d60 0x7f5e3006bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:44.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.747+0000 7f5e36a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e300b5ed0 con 0x7f5e301457e0 2024-03-28T13:14:44.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.747+0000 7f5e36a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5e30145bc0 con 0x7f5e30160a30 2024-03-28T13:14:44.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.747+0000 7f5e36a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5e300afca0 con 0x7f5e30145d60 2024-03-28T13:14:44.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.747+0000 7f5e2fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5e301457e0 0x7f5e300b4040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48230/0 (socket says 172.21.15.19:48230) 2024-03-28T13:14:44.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.747+0000 7f5e2fe00640 1 -- 172.21.15.19:0/445243767 learned_addr learned my addr 172.21.15.19:0/445243767 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.747+0000 7f5e2f400640 1 --2- 172.21.15.19:0/445243767 >> v2:172.21.15.188:3300/0 conn(0x7f5e30160a30 0x7f5e3006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e34a00640 1 --2- 172.21.15.19:0/445243767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e30145d60 0x7f5e3006bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 -- 172.21.15.19:0/445243767 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1831651916 0 0) 0x7f5e300b5ed0 con 0x7f5e301457e0 2024-03-28T13:14:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 -- 172.21.15.19:0/445243767 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e14003520 con 0x7f5e301457e0 2024-03-28T13:14:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 -- 172.21.15.19:0/445243767 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2378146988 0 0) 0x7f5e14003520 con 0x7f5e301457e0 2024-03-28T13:14:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 -- 172.21.15.19:0/445243767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e30145d60 msgr2=0x7f5e3006bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:44.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 --2- 172.21.15.19:0/445243767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e30145d60 0x7f5e3006bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 -- 172.21.15.19:0/445243767 >> v2:172.21.15.188:3300/0 conn(0x7f5e30160a30 msgr2=0x7f5e3006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:44.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 --2- 172.21.15.19:0/445243767 >> v2:172.21.15.188:3300/0 conn(0x7f5e30160a30 0x7f5e3006b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 -- 172.21.15.19:0/445243767 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e300b61b0 con 0x7f5e301457e0 2024-03-28T13:14:44.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 -- 172.21.15.19:0/445243767 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f5e200019f0 con 0x7f5e301457e0 2024-03-28T13:14:44.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 -- 172.21.15.19:0/445243767 >> v1:172.21.15.19:6789/0 conn(0x7f5e301457e0 legacy=0x7f5e300b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:44.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 --2- 172.21.15.19:0/445243767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039b0 0x7f5e14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:44.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 --2- 172.21.15.19:0/445243767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e30145d60 0x7f5e14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:44.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 --2- 172.21.15.19:0/445243767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e14004d40 0x7f5e140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:44.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 -- 172.21.15.19:0/445243767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e30145bc0 con 0x7f5e140039b0 2024-03-28T13:14:44.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 -- 172.21.15.19:0/445243767 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5e300afca0 con 0x7f5e14004d40 2024-03-28T13:14:44.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.748+0000 7f5e2ea00640 1 -- 172.21.15.19:0/445243767 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5e1400ba70 con 0x7f5e30145d60 2024-03-28T13:14:44.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.749+0000 7f5e2fe00640 1 --2- 172.21.15.19:0/445243767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039b0 0x7f5e14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:44.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.749+0000 7f5e2fe00640 1 -- 172.21.15.19:0/445243767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e30145d60 msgr2=0x7f5e14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:44.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.749+0000 7f5e2fe00640 1 --2- 172.21.15.19:0/445243767 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e30145d60 0x7f5e14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.749+0000 7f5e34a00640 1 --2- 172.21.15.19:0/445243767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e14004d40 0x7f5e140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:44.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.749+0000 7f5e2fe00640 1 -- 172.21.15.19:0/445243767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e14004d40 msgr2=0x7f5e140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:44.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.749+0000 7f5e2fe00640 1 --2- 172.21.15.19:0/445243767 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e14004d40 0x7f5e140051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.749+0000 7f5e2fe00640 1 -- 172.21.15.19:0/445243767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e14003520 con 0x7f5e140039b0 2024-03-28T13:14:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.750+0000 7f5e2fe00640 1 --2- 172.21.15.19:0/445243767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039b0 0x7f5e14003dd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000a8a0 tx=0x7f5e14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93183db76c991ff8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.751+0000 7f5e2ea00640 1 -- 172.21.15.19:0/445243767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e2000a4c0 con 0x7f5e140039b0 2024-03-28T13:14:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.751+0000 7f5e2ea00640 1 -- 172.21.15.19:0/445243767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e20034650 con 0x7f5e140039b0 2024-03-28T13:14:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.751+0000 7f5e2ea00640 1 -- 172.21.15.19:0/445243767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e20034a40 con 0x7f5e140039b0 2024-03-28T13:14:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.751+0000 7f5e36a00640 1 -- 172.21.15.19:0/445243767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039b0 msgr2=0x7f5e14003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.751+0000 7f5e36a00640 1 --2- 172.21.15.19:0/445243767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e140039b0 0x7f5e14003dd0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000a8a0 tx=0x7f5e14005830 comp rx=0 tx=0).stop 2024-03-28T13:14:44.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.751+0000 7f5e34a00640 1 -- 172.21.15.19:0/445243767 reap_dead start 2024-03-28T13:14:44.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.751+0000 7f5e36a00640 1 -- 172.21.15.19:0/445243767 shutdown_connections 2024-03-28T13:14:44.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.751+0000 7f5e36a00640 1 -- 172.21.15.19:0/445243767 >> 172.21.15.19:0/445243767 conn(0x7f5e300b65a0 msgr2=0x7f5e300b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:44.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.751+0000 7f5e36a00640 1 -- 172.21.15.19:0/445243767 shutdown_connections 2024-03-28T13:14:44.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.751+0000 7f5e36a00640 1 -- 172.21.15.19:0/445243767 wait complete. 2024-03-28T13:14:44.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.753+0000 7f5e36a00640 1 Processor -- start 2024-03-28T13:14:44.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.753+0000 7f5e36a00640 1 -- start start 2024-03-28T13:14:44.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.754+0000 7f5e36a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e140039b0 0x7f5e3006ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:44.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.754+0000 7f5e36a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e301e7750 0x7f5e3013fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:44.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.754+0000 7f5e36a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e301e7c30 0x7f5e301e80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:44.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.754+0000 7f5e36a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e300af9f0 con 0x7f5e301e7750 2024-03-28T13:14:44.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.754+0000 7f5e36a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5e300af870 con 0x7f5e301e7c30 2024-03-28T13:14:44.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.754+0000 7f5e36a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5e300ae1e0 con 0x7f5e140039b0 2024-03-28T13:14:44.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.754+0000 7f5e2f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e301e7750 0x7f5e3013fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:44.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.755+0000 7f5e2f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e301e7750 0x7f5e3013fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48274/0 (socket says 172.21.15.19:48274) 2024-03-28T13:14:44.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.755+0000 7f5e2f400640 1 -- 172.21.15.19:0/3238737476 learned_addr learned my addr 172.21.15.19:0/3238737476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:44.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.755+0000 7f5e34a00640 1 --2- 172.21.15.19:0/3238737476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e301e7c30 0x7f5e301e80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:44.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.755+0000 7f5e2fe00640 1 --2- 172.21.15.19:0/3238737476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e140039b0 0x7f5e3006ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:44.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.755+0000 7f5e2f400640 1 -- 172.21.15.19:0/3238737476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e140039b0 msgr2=0x7f5e3006ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:44.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.755+0000 7f5e2f400640 1 --2- 172.21.15.19:0/3238737476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e140039b0 0x7f5e3006ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.755+0000 7f5e2f400640 1 -- 172.21.15.19:0/3238737476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e301e7c30 msgr2=0x7f5e301e80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:44.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.755+0000 7f5e2f400640 1 --2- 172.21.15.19:0/3238737476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e301e7c30 0x7f5e301e80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:44.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.755+0000 7f5e2f400640 1 -- 172.21.15.19:0/3238737476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e3006e950 con 0x7f5e301e7750 2024-03-28T13:14:44.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.755+0000 7f5e2f400640 1 --2- 172.21.15.19:0/3238737476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e301e7750 0x7f5e3013fa90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00e860 tx=0x7f5e1c00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:44.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.756+0000 7f5e27e00640 1 -- 172.21.15.19:0/3238737476 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c017070 con 0x7f5e301e7750 2024-03-28T13:14:44.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.756+0000 7f5e36a00640 1 -- 172.21.15.19:0/3238737476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e3006ec30 con 0x7f5e301e7750 2024-03-28T13:14:44.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.756+0000 7f5e27e00640 1 -- 172.21.15.19:0/3238737476 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e1c00c940 con 0x7f5e301e7750 2024-03-28T13:14:44.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.756+0000 7f5e27e00640 1 -- 172.21.15.19:0/3238737476 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00cd10 con 0x7f5e301e7750 2024-03-28T13:14:44.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.756+0000 7f5e36a00640 1 -- 172.21.15.19:0/3238737476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e3006f1f0 con 0x7f5e301e7750 2024-03-28T13:14:44.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.757+0000 7f5e36a00640 1 -- 172.21.15.19:0/3238737476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e30068780 con 0x7f5e301e7750 2024-03-28T13:14:44.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.758+0000 7f5e27e00640 1 -- 172.21.15.19:0/3238737476 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5e1c01a070 con 0x7f5e301e7750 2024-03-28T13:14:44.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.759+0000 7f5e27e00640 1 --2- 172.21.15.19:0/3238737476 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5e1006c530 0x7f5e1006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:44.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.759+0000 7f5e27e00640 1 -- 172.21.15.19:0/3238737476 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5709+0+0 (secure 0 0 0) 0x7f5e1c08ba90 con 0x7f5e301e7750 2024-03-28T13:14:44.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.759+0000 7f5e2fe00640 1 --2- 172.21.15.19:0/3238737476 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5e1006c530 0x7f5e1006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:44.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.761+0000 7f5e2fe00640 1 --2- 172.21.15.19:0/3238737476 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5e1006c530 0x7f5e1006e9f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5e20034000 tx=0x7f5e20009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:44.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:44.764+0000 7f5e27e00640 1 -- 172.21.15.19:0/3238737476 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5e30068780 con 0x7f5e301e7750 2024-03-28T13:14:45.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.006+0000 7f5e36a00640 1 -- 172.21.15.19:0/3238737476 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5e30068cd0 con 0x7f5e301e7750 2024-03-28T13:14:45.006 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:14:45.007+0000 7f6b74400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T13:14:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.010+0000 7f5e27e00640 1 -- 172.21.15.19:0/3238737476 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) ==== 161+0+0 (secure 0 0 0) 0x7f5e1c059f90 con 0x7f5e301e7750 2024-03-28T13:14:45.010 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T13:14:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.013+0000 7f5e36a00640 1 -- 172.21.15.19:0/3238737476 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5e1006c530 msgr2=0x7f5e1006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:45.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.013+0000 7f5e36a00640 1 --2- 172.21.15.19:0/3238737476 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5e1006c530 0x7f5e1006e9f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5e20034000 tx=0x7f5e20009030 comp rx=0 tx=0).stop 2024-03-28T13:14:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.013+0000 7f5e36a00640 1 -- 172.21.15.19:0/3238737476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e301e7750 msgr2=0x7f5e3013fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.013+0000 7f5e36a00640 1 --2- 172.21.15.19:0/3238737476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e301e7750 0x7f5e3013fa90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00e860 tx=0x7f5e1c00ed20 comp rx=0 tx=0).stop 2024-03-28T13:14:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.013+0000 7f5e36a00640 1 -- 172.21.15.19:0/3238737476 shutdown_connections 2024-03-28T13:14:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.013+0000 7f5e36a00640 1 --2- 172.21.15.19:0/3238737476 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5e1006c530 0x7f5e1006e9f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:45.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.013+0000 7f5e36a00640 1 --2- 172.21.15.19:0/3238737476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e301e7c30 0x7f5e301e80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.013+0000 7f5e36a00640 1 --2- 172.21.15.19:0/3238737476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e301e7750 0x7f5e3013fa90 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.013+0000 7f5e36a00640 1 --2- 172.21.15.19:0/3238737476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e140039b0 0x7f5e3006ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.013+0000 7f5e36a00640 1 -- 172.21.15.19:0/3238737476 >> 172.21.15.19:0/3238737476 conn(0x7f5e300b65a0 msgr2=0x7f5e300b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.013+0000 7f5e36a00640 1 -- 172.21.15.19:0/3238737476 shutdown_connections 2024-03-28T13:14:45.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.013+0000 7f5e36a00640 1 -- 172.21.15.19:0/3238737476 wait complete. 2024-03-28T13:14:45.036 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-03-28T13:14:45.041 INFO:tasks.ceph.mds.c.smithi019.stderr: -4406> 2024-03-28T13:14:36.998+0000 7fae96200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:14:45.106 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-03-28T13:14:45.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.134+0000 7f0053a65f40 1 Processor -- start 2024-03-28T13:14:45.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.134+0000 7f0053a65f40 1 -- start start 2024-03-28T13:14:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.135+0000 7f0053a65f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00380044b0 0x55d7cf88ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.135+0000 7f0053a65f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7cf917d30 0x55d7cf875360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.135+0000 7f0053a65f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d7cf9181b0 0x55d7cf8758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.135+0000 7f0053a65f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d7cf88f140 con 0x7f00380044b0 2024-03-28T13:14:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.135+0000 7f0053a65f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d7cf7ee2c0 con 0x55d7cf917d30 2024-03-28T13:14:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.135+0000 7f0053a65f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55d7cf91c470 con 0x55d7cf9181b0 2024-03-28T13:14:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.135+0000 7f0050c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00380044b0 0x55d7cf88ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.135+0000 7f0050c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00380044b0 0x55d7cf88ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48318/0 (socket says 172.21.15.19:48318) 2024-03-28T13:14:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.135+0000 7f0050c00640 1 -- 172.21.15.19:0/2031565409 learned_addr learned my addr 172.21.15.19:0/2031565409 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.135+0000 7f0050c00640 1 -- 172.21.15.19:0/2031565409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d7cf9181b0 msgr2=0x55d7cf8758a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.135+0000 7f0050c00640 1 --2- 172.21.15.19:0/2031565409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d7cf9181b0 0x55d7cf8758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.135+0000 7f0050c00640 1 -- 172.21.15.19:0/2031565409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7cf917d30 msgr2=0x55d7cf875360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.135+0000 7f0050c00640 1 --2- 172.21.15.19:0/2031565409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7cf917d30 0x55d7cf875360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.135+0000 7f0050c00640 1 -- 172.21.15.19:0/2031565409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7cf91aa60 con 0x7f00380044b0 2024-03-28T13:14:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.136+0000 7f0050c00640 1 --2- 172.21.15.19:0/2031565409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00380044b0 0x55d7cf88ddf0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f004401d2d0 tx=0x7f004404c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3574db093fdde761 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.136+0000 7f004aa00640 1 -- 172.21.15.19:0/2031565409 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004400a340 con 0x7f00380044b0 2024-03-28T13:14:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.136+0000 7f004aa00640 1 -- 172.21.15.19:0/2031565409 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00440547b0 con 0x7f00380044b0 2024-03-28T13:14:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.136+0000 7f004aa00640 1 -- 172.21.15.19:0/2031565409 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0044054ba0 con 0x7f00380044b0 2024-03-28T13:14:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.136+0000 7f0053a65f40 1 -- 172.21.15.19:0/2031565409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00380044b0 msgr2=0x55d7cf88ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:45.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.136+0000 7f0053a65f40 1 --2- 172.21.15.19:0/2031565409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00380044b0 0x55d7cf88ddf0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f004401d2d0 tx=0x7f004404c070 comp rx=0 tx=0).stop 2024-03-28T13:14:45.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.136+0000 7f0053a65f40 1 -- 172.21.15.19:0/2031565409 shutdown_connections 2024-03-28T13:14:45.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.136+0000 7f0053a65f40 1 --2- 172.21.15.19:0/2031565409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d7cf9181b0 0x55d7cf8758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:45.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.136+0000 7f0053a65f40 1 --2- 172.21.15.19:0/2031565409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7cf917d30 0x55d7cf875360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:45.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.136+0000 7f0053a65f40 1 --2- 172.21.15.19:0/2031565409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00380044b0 0x55d7cf88ddf0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:45.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.136+0000 7f0053a65f40 1 -- 172.21.15.19:0/2031565409 >> 172.21.15.19:0/2031565409 conn(0x55d7cf871c20 msgr2=0x55d7cf91e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:45.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.136+0000 7f0053a65f40 1 -- 172.21.15.19:0/2031565409 shutdown_connections 2024-03-28T13:14:45.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.136+0000 7f0053a65f40 1 -- 172.21.15.19:0/2031565409 wait complete. 2024-03-28T13:14:45.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.137+0000 7f0053a65f40 1 Processor -- start 2024-03-28T13:14:45.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.137+0000 7f0053a65f40 1 -- start start 2024-03-28T13:14:45.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.138+0000 7f0053a65f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7cf917d30 0x55d7cf9b8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:45.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.138+0000 7f0053a65f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7cf9181b0 0x55d7cf848f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:45.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.138+0000 7f0053a65f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00380044b0 0x55d7cf88c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:45.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.138+0000 7f0053a65f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d7cf891b60 con 0x55d7cf9181b0 2024-03-28T13:14:45.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.138+0000 7f0053a65f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d7cf88d750 con 0x55d7cf917d30 2024-03-28T13:14:45.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.138+0000 7f0053a65f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55d7cf88d3e0 con 0x7f00380044b0 2024-03-28T13:14:45.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.139+0000 7f004be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7cf9181b0 0x55d7cf848f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:45.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.139+0000 7f004be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7cf9181b0 0x55d7cf848f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48322/0 (socket says 172.21.15.19:48322) 2024-03-28T13:14:45.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.139+0000 7f004be00640 1 -- 172.21.15.19:0/345559100 learned_addr learned my addr 172.21.15.19:0/345559100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:45.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.139+0000 7f0050c00640 1 --2- 172.21.15.19:0/345559100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7cf917d30 0x55d7cf9b8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:45.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.139+0000 7f004be00640 1 -- 172.21.15.19:0/345559100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00380044b0 msgr2=0x55d7cf88c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:45.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.139+0000 7f004be00640 1 --2- 172.21.15.19:0/345559100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00380044b0 0x55d7cf88c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:45.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.139+0000 7f004be00640 1 -- 172.21.15.19:0/345559100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7cf917d30 msgr2=0x55d7cf9b8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:45.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.139+0000 7f004be00640 1 --2- 172.21.15.19:0/345559100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7cf917d30 0x55d7cf9b8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:45.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.139+0000 7f004be00640 1 -- 172.21.15.19:0/345559100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7cf9b9230 con 0x55d7cf9181b0 2024-03-28T13:14:45.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.139+0000 7f0050c00640 1 --2- 172.21.15.19:0/345559100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d7cf917d30 0x55d7cf9b8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:45.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.139+0000 7f004be00640 1 --2- 172.21.15.19:0/345559100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7cf9181b0 0x55d7cf848f90 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f003c00bd40 tx=0x7f003c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:45.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.140+0000 7f0049600640 1 -- 172.21.15.19:0/345559100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c019070 con 0x55d7cf9181b0 2024-03-28T13:14:45.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.140+0000 7f0049600640 1 -- 172.21.15.19:0/345559100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f003c007a10 con 0x55d7cf9181b0 2024-03-28T13:14:45.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.140+0000 7f0049600640 1 -- 172.21.15.19:0/345559100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003c007e00 con 0x55d7cf9181b0 2024-03-28T13:14:45.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.140+0000 7f0053a65f40 1 -- 172.21.15.19:0/345559100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d7cf9215e0 con 0x55d7cf9181b0 2024-03-28T13:14:45.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.140+0000 7f0053a65f40 1 -- 172.21.15.19:0/345559100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7cf921ba0 con 0x55d7cf9181b0 2024-03-28T13:14:45.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.142+0000 7f0049600640 1 -- 172.21.15.19:0/345559100 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f003c01c070 con 0x55d7cf9181b0 2024-03-28T13:14:45.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.143+0000 7f0049600640 1 --2- 172.21.15.19:0/345559100 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f003006af10 0x7f003006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:45.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.143+0000 7f0049600640 1 -- 172.21.15.19:0/345559100 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7f003c0894e0 con 0x55d7cf9181b0 2024-03-28T13:14:45.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.143+0000 7f0053a65f40 1 --2- 172.21.15.19:0/345559100 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x55d7cf88cc40 0x55d7cf9c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:45.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.143+0000 7f0053a65f40 1 -- 172.21.15.19:0/345559100 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e78) -- 0x55d7cf9c3e00 con 0x55d7cf88cc40 2024-03-28T13:14:45.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.143+0000 7f0050c00640 1 --2- 172.21.15.19:0/345559100 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f003006af10 0x7f003006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:45.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.143+0000 7f0051600640 1 --2- 172.21.15.19:0/345559100 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x55d7cf88cc40 0x55d7cf9c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:45.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.144+0000 7f0051600640 1 --2- 172.21.15.19:0/345559100 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x55d7cf88cc40 0x55d7cf9c38c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:45.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.144+0000 7f0050c00640 1 --2- 172.21.15.19:0/345559100 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f003006af10 0x7f003006d3d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0044061000 tx=0x7f0044050000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:45.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.149+0000 7f0051600640 1 -- 172.21.15.19:0/345559100 <== osd.1 v2:172.21.15.19:6801/1998544260 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v78'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f004c00d1c0 con 0x55d7cf88cc40 2024-03-28T13:14:45.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.149+0000 7f0053a65f40 1 -- 172.21.15.19:0/345559100 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x55d7cf88cc40 msgr2=0x55d7cf9c38c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:45.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.149+0000 7f0053a65f40 1 --2- 172.21.15.19:0/345559100 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x55d7cf88cc40 0x55d7cf9c38c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:45.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.149+0000 7f0053a65f40 1 -- 172.21.15.19:0/345559100 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f003006af10 msgr2=0x7f003006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:45.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.149+0000 7f0053a65f40 1 --2- 172.21.15.19:0/345559100 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f003006af10 0x7f003006d3d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0044061000 tx=0x7f0044050000 comp rx=0 tx=0).stop 2024-03-28T13:14:45.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.149+0000 7f0053a65f40 1 -- 172.21.15.19:0/345559100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7cf9181b0 msgr2=0x55d7cf848f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:45.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.149+0000 7f0053a65f40 1 --2- 172.21.15.19:0/345559100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d7cf9181b0 0x55d7cf848f90 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f003c00bd40 tx=0x7f003c00be30 comp rx=0 tx=0).stop 2024-03-28T13:14:45.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.149+0000 7f0051600640 1 -- 172.21.15.19:0/345559100 reap_dead start 2024-03-28T13:14:45.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.149+0000 7f0053a65f40 1 -- 172.21.15.19:0/345559100 shutdown_connections 2024-03-28T13:14:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.149+0000 7f0053a65f40 1 -- 172.21.15.19:0/345559100 >> 172.21.15.19:0/345559100 conn(0x55d7cf871c20 msgr2=0x55d7cf88de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.149+0000 7f0053a65f40 1 -- 172.21.15.19:0/345559100 shutdown_connections 2024-03-28T13:14:45.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.150+0000 7f0053a65f40 1 -- 172.21.15.19:0/345559100 wait complete. 2024-03-28T13:14:45.154 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-28T13:14:45.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.346+0000 7f5447e00640 1 Processor -- start 2024-03-28T13:14:45.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.347+0000 7f5447e00640 1 -- start start 2024-03-28T13:14:45.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.348+0000 7f5447e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f544813f190 0x7f54480cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:45.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.348+0000 7f5447e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54480ce890 con 0x7f54480cc880 2024-03-28T13:14:45.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.348+0000 7f5447e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54480ceb50 con 0x7f544806dd30 2024-03-28T13:14:45.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.348+0000 7f5447e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f54480cee30 con 0x7f544813f190 2024-03-28T13:14:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.349+0000 7f5446000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f54480cc880 0x7f544806d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48262/0 (socket says 172.21.15.19:48262) 2024-03-28T13:14:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.349+0000 7f5446000640 1 -- 172.21.15.19:0/2995948363 learned_addr learned my addr 172.21.15.19:0/2995948363 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:45.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.349+0000 7f5446a00640 1 --2- 172.21.15.19:0/2995948363 >> v2:172.21.15.19:3300/0 conn(0x7f544813f190 0x7f54480cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.349+0000 7f5445600640 1 -- 172.21.15.19:0/2995948363 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1851879306 0 0) 0x7f54480ce890 con 0x7f54480cc880 2024-03-28T13:14:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.349+0000 7f5445600640 1 -- 172.21.15.19:0/2995948363 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f542c003520 con 0x7f54480cc880 2024-03-28T13:14:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.349+0000 7f5446a00640 1 -- 172.21.15.19:0/2995948363 >> v1:172.21.15.188:6789/0 conn(0x7f544806dd30 legacy=0x7f544815ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.350+0000 7f5446a00640 1 -- 172.21.15.19:0/2995948363 >> v1:172.21.15.19:6789/0 conn(0x7f54480cc880 legacy=0x7f544806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:45.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.350+0000 7f5446a00640 1 -- 172.21.15.19:0/2995948363 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54480cf110 con 0x7f544813f190 2024-03-28T13:14:45.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.351+0000 7f5446a00640 1 --2- 172.21.15.19:0/2995948363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f544813f190 0x7f54480cf630 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f54380097a0 tx=0x7f543802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490b2f21565ad40f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:45.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.351+0000 7f5445600640 1 -- 172.21.15.19:0/2995948363 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438034070 con 0x7f544813f190 2024-03-28T13:14:45.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.351+0000 7f5445600640 1 -- 172.21.15.19:0/2995948363 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5438030610 con 0x7f544813f190 2024-03-28T13:14:45.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.351+0000 7f5445600640 1 -- 172.21.15.19:0/2995948363 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54380309c0 con 0x7f544813f190 2024-03-28T13:14:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.352+0000 7f5447e00640 1 -- 172.21.15.19:0/2995948363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f544813f190 msgr2=0x7f54480cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.352+0000 7f5447e00640 1 --2- 172.21.15.19:0/2995948363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f544813f190 0x7f54480cf630 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f54380097a0 tx=0x7f543802b960 comp rx=0 tx=0).stop 2024-03-28T13:14:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.352+0000 7f5447e00640 1 -- 172.21.15.19:0/2995948363 shutdown_connections 2024-03-28T13:14:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.352+0000 7f5447e00640 1 --2- 172.21.15.19:0/2995948363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f544813f190 0x7f54480cf630 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.352+0000 7f5447e00640 1 -- 172.21.15.19:0/2995948363 >> 172.21.15.19:0/2995948363 conn(0x7f54480b1ba0 msgr2=0x7f54480b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.352+0000 7f5447e00640 1 -- 172.21.15.19:0/2995948363 shutdown_connections 2024-03-28T13:14:45.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.352+0000 7f5447e00640 1 -- 172.21.15.19:0/2995948363 wait complete. 2024-03-28T13:14:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.354+0000 7f5447e00640 1 Processor -- start 2024-03-28T13:14:45.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.354+0000 7f5447e00640 1 -- start start 2024-03-28T13:14:45.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.355+0000 7f5447e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f544806dd30 0x7f54480ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:45.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.355+0000 7f5447e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54480cc880 0x7f5448155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:45.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.355+0000 7f5447e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f544813f190 0x7f54480b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.355+0000 7f5447e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f54480cfb70 con 0x7f544813f190 2024-03-28T13:14:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.355+0000 7f5447e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f544813f570 con 0x7f54480cc880 2024-03-28T13:14:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.355+0000 7f5447e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5448160620 con 0x7f544806dd30 2024-03-28T13:14:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.355+0000 7f5447400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f544813f190 0x7f54480b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.355+0000 7f5447400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f544813f190 0x7f54480b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48332/0 (socket says 172.21.15.19:48332) 2024-03-28T13:14:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.355+0000 7f5447400640 1 -- 172.21.15.19:0/2178863128 learned_addr learned my addr 172.21.15.19:0/2178863128 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.355+0000 7f5446a00640 1 --2- 172.21.15.19:0/2178863128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f544806dd30 0x7f54480ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.356+0000 7f5446000640 1 --2- 172.21.15.19:0/2178863128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54480cc880 0x7f5448155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.356+0000 7f5447400640 1 -- 172.21.15.19:0/2178863128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f544806dd30 msgr2=0x7f54480ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.356+0000 7f5447400640 1 --2- 172.21.15.19:0/2178863128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f544806dd30 0x7f54480ac170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.356+0000 7f5447400640 1 -- 172.21.15.19:0/2178863128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54480cc880 msgr2=0x7f5448155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.356+0000 7f5447400640 1 --2- 172.21.15.19:0/2178863128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54480cc880 0x7f5448155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.356+0000 7f5447400640 1 -- 172.21.15.19:0/2178863128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54480aafc0 con 0x7f544813f190 2024-03-28T13:14:45.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.356+0000 7f5447400640 1 --2- 172.21.15.19:0/2178863128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f544813f190 0x7f54480b0e10 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f544000c6b0 tx=0x7f544000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.356+0000 7f543ea00640 1 -- 172.21.15.19:0/2178863128 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440003b70 con 0x7f544813f190 2024-03-28T13:14:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.356+0000 7f543ea00640 1 -- 172.21.15.19:0/2178863128 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5440003d30 con 0x7f544813f190 2024-03-28T13:14:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.356+0000 7f5447e00640 1 -- 172.21.15.19:0/2178863128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54480a9f20 con 0x7f544813f190 2024-03-28T13:14:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.356+0000 7f543ea00640 1 -- 172.21.15.19:0/2178863128 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440005310 con 0x7f544813f190 2024-03-28T13:14:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.357+0000 7f5447e00640 1 -- 172.21.15.19:0/2178863128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54480aa4e0 con 0x7f544813f190 2024-03-28T13:14:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.358+0000 7f5447e00640 1 -- 172.21.15.19:0/2178863128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5448075940 con 0x7f544813f190 2024-03-28T13:14:45.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.359+0000 7f543ea00640 1 -- 172.21.15.19:0/2178863128 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f54400054b0 con 0x7f544813f190 2024-03-28T13:14:45.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.360+0000 7f543ea00640 1 --2- 172.21.15.19:0/2178863128 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f542806c530 0x7f542806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:45.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.360+0000 7f543ea00640 1 -- 172.21.15.19:0/2178863128 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7f544008b5f0 con 0x7f544813f190 2024-03-28T13:14:45.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.360+0000 7f5446a00640 1 --2- 172.21.15.19:0/2178863128 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f542806c530 0x7f542806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:45.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.361+0000 7f5446a00640 1 --2- 172.21.15.19:0/2178863128 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f542806c530 0x7f542806e9f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5438036000 tx=0x7f5438008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:45.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.364+0000 7f543ea00640 1 -- 172.21.15.19:0/2178863128 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f54480aafc0 con 0x7f544813f190 2024-03-28T13:14:45.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:45.612+0000 7f5447e00640 1 -- 172.21.15.19:0/2178863128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f54480b2870 con 0x7f544813f190 2024-03-28T13:14:46.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.014+0000 7f543ea00640 1 -- 172.21.15.19:0/2178863128 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v47) ==== 162+0+0 (secure 0 0 0) 0x7f5440059ba0 con 0x7f544813f190 2024-03-28T13:14:46.014 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-28T13:14:46.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.017+0000 7f5447e00640 1 -- 172.21.15.19:0/2178863128 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f542806c530 msgr2=0x7f542806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:46.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.017+0000 7f5447e00640 1 --2- 172.21.15.19:0/2178863128 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f542806c530 0x7f542806e9f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5438036000 tx=0x7f5438008030 comp rx=0 tx=0).stop 2024-03-28T13:14:46.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.017+0000 7f5447e00640 1 -- 172.21.15.19:0/2178863128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f544813f190 msgr2=0x7f54480b0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:46.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.017+0000 7f5447e00640 1 --2- 172.21.15.19:0/2178863128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f544813f190 0x7f54480b0e10 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f544000c6b0 tx=0x7f544000cb70 comp rx=0 tx=0).stop 2024-03-28T13:14:46.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.018+0000 7f5447e00640 1 -- 172.21.15.19:0/2178863128 shutdown_connections 2024-03-28T13:14:46.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.018+0000 7f5447e00640 1 --2- 172.21.15.19:0/2178863128 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f542806c530 0x7f542806e9f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:46.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.018+0000 7f5447e00640 1 --2- 172.21.15.19:0/2178863128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f544813f190 0x7f54480b0e10 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:46.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.018+0000 7f5447e00640 1 --2- 172.21.15.19:0/2178863128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54480cc880 0x7f5448155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:46.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.018+0000 7f5447e00640 1 --2- 172.21.15.19:0/2178863128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f544806dd30 0x7f54480ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:46.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.018+0000 7f5447e00640 1 -- 172.21.15.19:0/2178863128 >> 172.21.15.19:0/2178863128 conn(0x7f54480b1ba0 msgr2=0x7f54480b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:46.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.018+0000 7f5447e00640 1 -- 172.21.15.19:0/2178863128 shutdown_connections 2024-03-28T13:14:46.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.018+0000 7f5447e00640 1 -- 172.21.15.19:0/2178863128 wait complete. 2024-03-28T13:14:46.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:14:46.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.235+0000 7f2a3a600640 1 Processor -- start 2024-03-28T13:14:46.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.235+0000 7f2a3a600640 1 -- start start 2024-03-28T13:14:46.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.236+0000 7f2a3a600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a3415b310 0x7f2a340727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:46.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.237+0000 7f2a3a600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a340d0780 0x7f2a340d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:46.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.237+0000 7f2a3a600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a3415b6f0 con 0x7f2a340b4f90 2024-03-28T13:14:46.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.237+0000 7f2a3a600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f2a34072e80 con 0x7f2a3415b310 2024-03-28T13:14:46.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.237+0000 7f2a3a600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2a34073160 con 0x7f2a340d0780 2024-03-28T13:14:46.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.237+0000 7f2a33400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a3415b310 0x7f2a340727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:46.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.237+0000 7f2a33e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a340d0780 0x7f2a340d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:46.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.237+0000 7f2a33400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a3415b310 0x7f2a340727f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:34100/0 (socket says 172.21.15.19:34100) 2024-03-28T13:14:46.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.237+0000 7f2a33400640 1 -- 172.21.15.19:0/769582894 learned_addr learned my addr 172.21.15.19:0/769582894 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:46.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.238+0000 7f2a33400640 1 -- 172.21.15.19:0/769582894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a340d0780 msgr2=0x7f2a340d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:46.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.238+0000 7f2a33400640 1 --2- 172.21.15.19:0/769582894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a340d0780 0x7f2a340d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:46.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.238+0000 7f2a33400640 1 -- 172.21.15.19:0/769582894 >> v1:172.21.15.188:6789/0 conn(0x7f2a340b4f90 legacy=0x7f2a3415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:46.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.238+0000 7f2a33400640 1 -- 172.21.15.19:0/769582894 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3415efe0 con 0x7f2a3415b310 2024-03-28T13:14:46.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.239+0000 7f2a33400640 1 --2- 172.21.15.19:0/769582894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a3415b310 0x7f2a340727f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009870 tx=0x7f2a2402b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f11d879ccef253d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:46.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.240+0000 7f2a32000640 1 -- 172.21.15.19:0/769582894 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a240317c0 con 0x7f2a3415b310 2024-03-28T13:14:46.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.240+0000 7f2a32000640 1 -- 172.21.15.19:0/769582894 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a24037070 con 0x7f2a3415b310 2024-03-28T13:14:46.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.240+0000 7f2a32000640 1 -- 172.21.15.19:0/769582894 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24031b80 con 0x7f2a3415b310 2024-03-28T13:14:46.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.240+0000 7f2a3a600640 1 -- 172.21.15.19:0/769582894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a3415b310 msgr2=0x7f2a340727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:46.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.240+0000 7f2a3a600640 1 --2- 172.21.15.19:0/769582894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a3415b310 0x7f2a340727f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2a24009870 tx=0x7f2a2402b860 comp rx=0 tx=0).stop 2024-03-28T13:14:46.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.240+0000 7f2a3a600640 1 -- 172.21.15.19:0/769582894 shutdown_connections 2024-03-28T13:14:46.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.240+0000 7f2a3a600640 1 --2- 172.21.15.19:0/769582894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a340d0780 0x7f2a340d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:46.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.240+0000 7f2a3a600640 1 --2- 172.21.15.19:0/769582894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a3415b310 0x7f2a340727f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:46.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.240+0000 7f2a3a600640 1 -- 172.21.15.19:0/769582894 >> 172.21.15.19:0/769582894 conn(0x7f2a3406b990 msgr2=0x7f2a3406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:46.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.241+0000 7f2a3a600640 1 -- 172.21.15.19:0/769582894 shutdown_connections 2024-03-28T13:14:46.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.241+0000 7f2a3a600640 1 -- 172.21.15.19:0/769582894 wait complete. 2024-03-28T13:14:46.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.242+0000 7f2a3a600640 1 Processor -- start 2024-03-28T13:14:46.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.242+0000 7f2a3a600640 1 -- start start 2024-03-28T13:14:46.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.243+0000 7f2a3a600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a340b4f90 0x7f2a3415dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:46.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.243+0000 7f2a3a600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a340d0780 0x7f2a341f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:46.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.243+0000 7f2a3a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3415b310 0x7f2a34153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:46.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.243+0000 7f2a3a600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a34073440 con 0x7f2a3415b310 2024-03-28T13:14:46.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.243+0000 7f2a3a600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a340b53f0 con 0x7f2a340d0780 2024-03-28T13:14:46.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.243+0000 7f2a3a600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2a340adc80 con 0x7f2a340b4f90 2024-03-28T13:14:46.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.244+0000 7f2a33e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3415b310 0x7f2a34153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:46.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.244+0000 7f2a33e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3415b310 0x7f2a34153910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47200/0 (socket says 172.21.15.19:47200) 2024-03-28T13:14:46.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.244+0000 7f2a33e00640 1 -- 172.21.15.19:0/709760064 learned_addr learned my addr 172.21.15.19:0/709760064 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:46.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.244+0000 7f2a33400640 1 --2- 172.21.15.19:0/709760064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a340b4f90 0x7f2a3415dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.244+0000 7f2a32a00640 1 --2- 172.21.15.19:0/709760064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a340d0780 0x7f2a341f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.244+0000 7f2a33e00640 1 -- 172.21.15.19:0/709760064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a340b4f90 msgr2=0x7f2a3415dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.244+0000 7f2a33e00640 1 --2- 172.21.15.19:0/709760064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a340b4f90 0x7f2a3415dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.244+0000 7f2a33e00640 1 -- 172.21.15.19:0/709760064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a340d0780 msgr2=0x7f2a341f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.244+0000 7f2a33e00640 1 --2- 172.21.15.19:0/709760064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a340d0780 0x7f2a341f3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.244+0000 7f2a33e00640 1 -- 172.21.15.19:0/709760064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a341f4110 con 0x7f2a3415b310 2024-03-28T13:14:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.245+0000 7f2a33e00640 1 --2- 172.21.15.19:0/709760064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3415b310 0x7f2a34153910 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00efc0 tx=0x7f2a2c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.245+0000 7f2a2b400640 1 -- 172.21.15.19:0/709760064 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c018020 con 0x7f2a3415b310 2024-03-28T13:14:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.245+0000 7f2a2b400640 1 -- 172.21.15.19:0/709760064 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a2c014480 con 0x7f2a3415b310 2024-03-28T13:14:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.245+0000 7f2a2b400640 1 -- 172.21.15.19:0/709760064 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c014870 con 0x7f2a3415b310 2024-03-28T13:14:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.245+0000 7f2a3a600640 1 -- 172.21.15.19:0/709760064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a34153130 con 0x7f2a3415b310 2024-03-28T13:14:46.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.245+0000 7f2a3a600640 1 -- 172.21.15.19:0/709760064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a341f4690 con 0x7f2a3415b310 2024-03-28T13:14:46.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.246+0000 7f2a3a600640 1 -- 172.21.15.19:0/709760064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a340685c0 con 0x7f2a3415b310 2024-03-28T13:14:46.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.247+0000 7f2a2b400640 1 -- 172.21.15.19:0/709760064 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2a2c005d10 con 0x7f2a3415b310 2024-03-28T13:14:46.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.248+0000 7f2a2b400640 1 --2- 172.21.15.19:0/709760064 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a1406c530 0x7f2a1406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:46.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.248+0000 7f2a2b400640 1 -- 172.21.15.19:0/709760064 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7f2a2c08c350 con 0x7f2a3415b310 2024-03-28T13:14:46.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.249+0000 7f2a33400640 1 --2- 172.21.15.19:0/709760064 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a1406c530 0x7f2a1406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:46.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.250+0000 7f2a33400640 1 --2- 172.21.15.19:0/709760064 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a1406c530 0x7f2a1406e9f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2a24036000 tx=0x7f2a2403d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:46.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.253+0000 7f2a2b400640 1 -- 172.21.15.19:0/709760064 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2a2c022070 con 0x7f2a3415b310 2024-03-28T13:14:46.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.497+0000 7f2a3a600640 1 -- 172.21.15.19:0/709760064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a340d0010 con 0x7f2a3415b310 2024-03-28T13:14:46.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.498+0000 7f2a2b400640 1 -- 172.21.15.19:0/709760064 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+5180 (secure 0 0 0) 0x7f2a2c010040 con 0x7f2a3415b310 2024-03-28T13:14:46.498 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:46.499 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/144726418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":144726418},{"type":"v1","addr":"172.21.15.188:6836","nonce":144726418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5363,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2642581607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2642581607},{"type":"v1","addr":"172.21.15.19:6835","nonce":2642581607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1721039756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1721039756},{"type":"v1","addr":"172.21.15.188:6837","nonce":1721039756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:14:31.526577+0000","modified":"2024-03-28T13:14:46.020239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"c","rank":0,"incarnation":48,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6834/2130552083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2130552083},{"type":"v1","addr":"172.21.15.19:6834","nonce":2130552083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-03-28T13:14:46.499 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 48 2024-03-28T13:14:46.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.501+0000 7f2a3a600640 1 -- 172.21.15.19:0/709760064 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a1406c530 msgr2=0x7f2a1406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:46.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.501+0000 7f2a3a600640 1 --2- 172.21.15.19:0/709760064 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a1406c530 0x7f2a1406e9f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2a24036000 tx=0x7f2a2403d040 comp rx=0 tx=0).stop 2024-03-28T13:14:46.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.501+0000 7f2a3a600640 1 -- 172.21.15.19:0/709760064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3415b310 msgr2=0x7f2a34153910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:46.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.501+0000 7f2a3a600640 1 --2- 172.21.15.19:0/709760064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3415b310 0x7f2a34153910 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c00efc0 tx=0x7f2a2c00c490 comp rx=0 tx=0).stop 2024-03-28T13:14:46.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.501+0000 7f2a3a600640 1 -- 172.21.15.19:0/709760064 shutdown_connections 2024-03-28T13:14:46.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.501+0000 7f2a3a600640 1 --2- 172.21.15.19:0/709760064 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a1406c530 0x7f2a1406e9f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:46.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.501+0000 7f2a3a600640 1 --2- 172.21.15.19:0/709760064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a3415b310 0x7f2a34153910 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:46.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.501+0000 7f2a3a600640 1 --2- 172.21.15.19:0/709760064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a340d0780 0x7f2a341f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:46.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.501+0000 7f2a3a600640 1 --2- 172.21.15.19:0/709760064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a340b4f90 0x7f2a3415dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:46.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.501+0000 7f2a3a600640 1 -- 172.21.15.19:0/709760064 >> 172.21.15.19:0/709760064 conn(0x7f2a3406b990 msgr2=0x7f2a340b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:46.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.501+0000 7f2a3a600640 1 -- 172.21.15.19:0/709760064 shutdown_connections 2024-03-28T13:14:46.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:46.502+0000 7f2a3a600640 1 -- 172.21.15.19:0/709760064 wait complete. 2024-03-28T13:14:46.923 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:14:31.526577+0000', 'modified': '2024-03-28T13:14:46.020239+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 78, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6708}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6708': {'gid': 6708, 'name': 'c', 'rank': 0, 'incarnation': 48, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6834/2130552083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 2130552083}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 2130552083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-28T13:14:46.923 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6708:up:replay 2024-03-28T13:14:47.924 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:14:48.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.164+0000 7fca64a00640 1 Processor -- start 2024-03-28T13:14:48.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.164+0000 7fca64a00640 1 -- start start 2024-03-28T13:14:48.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.166+0000 7fca64a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca6006cc90 0x7fca600ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:48.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.166+0000 7fca64a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fca600b5990 0x7fca600ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:48.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.166+0000 7fca64a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca600b53e0 con 0x7fca6015eab0 2024-03-28T13:14:48.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.166+0000 7fca64a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fca600b56c0 con 0x7fca600b5990 2024-03-28T13:14:48.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.166+0000 7fca64a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fca600adc80 con 0x7fca6006cc90 2024-03-28T13:14:48.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.166+0000 7fca5e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fca600b5990 0x7fca600ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:48.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.166+0000 7fca5de00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fca6015eab0 0x7fca6006c580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51524/0 (socket says 172.21.15.19:51524) 2024-03-28T13:14:48.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.166+0000 7fca5de00640 1 -- 172.21.15.19:0/773726979 learned_addr learned my addr 172.21.15.19:0/773726979 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:48.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.166+0000 7fca55400640 1 --2- 172.21.15.19:0/773726979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca6006cc90 0x7fca600ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:48.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.166+0000 7fca5e800640 1 -- 172.21.15.19:0/773726979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca6006cc90 msgr2=0x7fca600ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:48.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.166+0000 7fca5e800640 1 --2- 172.21.15.19:0/773726979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca6006cc90 0x7fca600ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:48.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.166+0000 7fca5e800640 1 -- 172.21.15.19:0/773726979 >> v1:172.21.15.19:6789/0 conn(0x7fca6015eab0 legacy=0x7fca6006c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:48.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.166+0000 7fca5d400640 1 -- 172.21.15.19:0/773726979 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3454805207 0 0) 0x7fca600b53e0 con 0x7fca6015eab0 2024-03-28T13:14:48.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.167+0000 7fca5e800640 1 -- 172.21.15.19:0/773726979 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca600b49a0 con 0x7fca600b5990 2024-03-28T13:14:48.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.168+0000 7fca5e800640 1 --2- 172.21.15.19:0/773726979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca600b5990 0x7fca600ae9c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fca58009d90 tx=0x7fca5802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835016aef0c42c2d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:48.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.168+0000 7fca5d400640 1 -- 172.21.15.19:0/773726979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58039070 con 0x7fca600b5990 2024-03-28T13:14:48.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.168+0000 7fca5d400640 1 -- 172.21.15.19:0/773726979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca5802cc50 con 0x7fca600b5990 2024-03-28T13:14:48.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.168+0000 7fca5d400640 1 -- 172.21.15.19:0/773726979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58034440 con 0x7fca600b5990 2024-03-28T13:14:48.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.168+0000 7fca64a00640 1 -- 172.21.15.19:0/773726979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca600b5990 msgr2=0x7fca600ae9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:48.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.168+0000 7fca64a00640 1 --2- 172.21.15.19:0/773726979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca600b5990 0x7fca600ae9c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fca58009d90 tx=0x7fca5802bea0 comp rx=0 tx=0).stop 2024-03-28T13:14:48.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.169+0000 7fca64a00640 1 -- 172.21.15.19:0/773726979 shutdown_connections 2024-03-28T13:14:48.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.169+0000 7fca64a00640 1 --2- 172.21.15.19:0/773726979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca600b5990 0x7fca600ae9c0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:48.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.169+0000 7fca64a00640 1 --2- 172.21.15.19:0/773726979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca6006cc90 0x7fca600ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:48.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.169+0000 7fca64a00640 1 -- 172.21.15.19:0/773726979 >> 172.21.15.19:0/773726979 conn(0x7fca60071df0 msgr2=0x7fca600721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:48.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.169+0000 7fca64a00640 1 -- 172.21.15.19:0/773726979 shutdown_connections 2024-03-28T13:14:48.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.169+0000 7fca64a00640 1 -- 172.21.15.19:0/773726979 wait complete. 2024-03-28T13:14:48.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.170+0000 7fca64a00640 1 Processor -- start 2024-03-28T13:14:48.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.171+0000 7fca64a00640 1 -- start start 2024-03-28T13:14:48.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca64a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca6006cc90 0x7fca600b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:48.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca64a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca600b5990 0x7fca601f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:48.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca64a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca6015eab0 0x7fca601f3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:48.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca64a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca60072d90 con 0x7fca6015eab0 2024-03-28T13:14:48.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca64a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fca60072c10 con 0x7fca6006cc90 2024-03-28T13:14:48.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca64a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fca600aef00 con 0x7fca600b5990 2024-03-28T13:14:48.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca5e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca6015eab0 0x7fca601f3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:48.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca5e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca6015eab0 0x7fca601f3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47222/0 (socket says 172.21.15.19:47222) 2024-03-28T13:14:48.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca5e800640 1 -- 172.21.15.19:0/2032864208 learned_addr learned my addr 172.21.15.19:0/2032864208 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:48.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca5de00640 1 --2- 172.21.15.19:0/2032864208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca6006cc90 0x7fca600b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:48.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca55400640 1 --2- 172.21.15.19:0/2032864208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca600b5990 0x7fca601f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:48.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca5e800640 1 -- 172.21.15.19:0/2032864208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca600b5990 msgr2=0x7fca601f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:48.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca5e800640 1 --2- 172.21.15.19:0/2032864208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca600b5990 0x7fca601f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:48.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca5e800640 1 -- 172.21.15.19:0/2032864208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca6006cc90 msgr2=0x7fca600b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:48.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca5e800640 1 --2- 172.21.15.19:0/2032864208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca6006cc90 0x7fca600b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:48.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.172+0000 7fca5e800640 1 -- 172.21.15.19:0/2032864208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca601f83f0 con 0x7fca6015eab0 2024-03-28T13:14:48.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.173+0000 7fca5e800640 1 --2- 172.21.15.19:0/2032864208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca6015eab0 0x7fca601f3b10 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fca58009eb0 tx=0x7fca58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:48.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.173+0000 7fca56a00640 1 -- 172.21.15.19:0/2032864208 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58039050 con 0x7fca6015eab0 2024-03-28T13:14:48.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.173+0000 7fca56a00640 1 -- 172.21.15.19:0/2032864208 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca58045070 con 0x7fca6015eab0 2024-03-28T13:14:48.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.173+0000 7fca56a00640 1 -- 172.21.15.19:0/2032864208 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5803e610 con 0x7fca6015eab0 2024-03-28T13:14:48.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.173+0000 7fca64a00640 1 -- 172.21.15.19:0/2032864208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6014ef00 con 0x7fca6015eab0 2024-03-28T13:14:48.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.173+0000 7fca64a00640 1 -- 172.21.15.19:0/2032864208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca600b44d0 con 0x7fca6015eab0 2024-03-28T13:14:48.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.174+0000 7fca64a00640 1 -- 172.21.15.19:0/2032864208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca600685c0 con 0x7fca6015eab0 2024-03-28T13:14:48.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.176+0000 7fca56a00640 1 -- 172.21.15.19:0/2032864208 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fca5803e7b0 con 0x7fca6015eab0 2024-03-28T13:14:48.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.177+0000 7fca56a00640 1 --2- 172.21.15.19:0/2032864208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca3c06c530 0x7fca3c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:48.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.177+0000 7fca56a00640 1 -- 172.21.15.19:0/2032864208 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7fca580ae240 con 0x7fca6015eab0 2024-03-28T13:14:48.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.177+0000 7fca5de00640 1 --2- 172.21.15.19:0/2032864208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca3c06c530 0x7fca3c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:48.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.178+0000 7fca5de00640 1 --2- 172.21.15.19:0/2032864208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca3c06c530 0x7fca3c06e9f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fca50000f60 tx=0x7fca50007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:48.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.181+0000 7fca56a00640 1 -- 172.21.15.19:0/2032864208 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fca58032e20 con 0x7fca6015eab0 2024-03-28T13:14:48.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.425+0000 7fca64a00640 1 -- 172.21.15.19:0/2032864208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca600ae6d0 con 0x7fca6015eab0 2024-03-28T13:14:48.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.426+0000 7fca56a00640 1 -- 172.21.15.19:0/2032864208 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+5180 (secure 0 0 0) 0x7fca5807c7f0 con 0x7fca6015eab0 2024-03-28T13:14:48.426 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:48.426 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/144726418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":144726418},{"type":"v1","addr":"172.21.15.188:6836","nonce":144726418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5363,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2642581607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2642581607},{"type":"v1","addr":"172.21.15.19:6835","nonce":2642581607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1721039756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1721039756},{"type":"v1","addr":"172.21.15.188:6837","nonce":1721039756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:14:31.526577+0000","modified":"2024-03-28T13:14:47.039000+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"c","rank":0,"incarnation":48,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.19:6834/2130552083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2130552083},{"type":"v1","addr":"172.21.15.19:6834","nonce":2130552083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-03-28T13:14:48.426 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 50 2024-03-28T13:14:48.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.428+0000 7fca64a00640 1 -- 172.21.15.19:0/2032864208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca3c06c530 msgr2=0x7fca3c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:48.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.428+0000 7fca64a00640 1 --2- 172.21.15.19:0/2032864208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca3c06c530 0x7fca3c06e9f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fca50000f60 tx=0x7fca50007c00 comp rx=0 tx=0).stop 2024-03-28T13:14:48.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.428+0000 7fca64a00640 1 -- 172.21.15.19:0/2032864208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca6015eab0 msgr2=0x7fca601f3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:48.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.428+0000 7fca64a00640 1 --2- 172.21.15.19:0/2032864208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca6015eab0 0x7fca601f3b10 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fca58009eb0 tx=0x7fca58004a40 comp rx=0 tx=0).stop 2024-03-28T13:14:48.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.428+0000 7fca64a00640 1 -- 172.21.15.19:0/2032864208 shutdown_connections 2024-03-28T13:14:48.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.428+0000 7fca64a00640 1 --2- 172.21.15.19:0/2032864208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca3c06c530 0x7fca3c06e9f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:48.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.428+0000 7fca64a00640 1 --2- 172.21.15.19:0/2032864208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca6015eab0 0x7fca601f3b10 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:48.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.428+0000 7fca64a00640 1 --2- 172.21.15.19:0/2032864208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca600b5990 0x7fca601f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:48.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.428+0000 7fca64a00640 1 --2- 172.21.15.19:0/2032864208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca6006cc90 0x7fca600b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:48.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.428+0000 7fca64a00640 1 -- 172.21.15.19:0/2032864208 >> 172.21.15.19:0/2032864208 conn(0x7fca60071df0 msgr2=0x7fca6006e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:48.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.429+0000 7fca64a00640 1 -- 172.21.15.19:0/2032864208 shutdown_connections 2024-03-28T13:14:48.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:48.429+0000 7fca64a00640 1 -- 172.21.15.19:0/2032864208 wait complete. 2024-03-28T13:14:48.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:14:31.526577+0000', 'modified': '2024-03-28T13:14:47.039000+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 78, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6708}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6708': {'gid': 6708, 'name': 'c', 'rank': 0, 'incarnation': 48, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.19:6834/2130552083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 2130552083}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 2130552083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-28T13:14:48.853 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6708:up:rejoin 2024-03-28T13:14:49.854 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:14:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.085+0000 7feb4be00640 1 Processor -- start 2024-03-28T13:14:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.085+0000 7feb4be00640 1 -- start start 2024-03-28T13:14:50.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.087+0000 7feb4be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb4c070250 0x7feb4c0d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:50.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.087+0000 7feb4be00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb4c0d2ed0 con 0x7feb4c13f100 2024-03-28T13:14:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.087+0000 7feb4be00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb4c13f4e0 con 0x7feb4c0d26e0 2024-03-28T13:14:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.087+0000 7feb4be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feb4c142270 con 0x7feb4c070250 2024-03-28T13:14:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.087+0000 7feb4aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7feb4c13f100 0x7feb4c06fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51534/0 (socket says 172.21.15.19:51534) 2024-03-28T13:14:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.087+0000 7feb4aa00640 1 -- 172.21.15.19:0/3225491274 learned_addr learned my addr 172.21.15.19:0/3225491274 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.088+0000 7feb43e00640 1 --2- 172.21.15.19:0/3225491274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb4c070250 0x7feb4c0d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.088+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2986883603 0 0) 0x7feb4c0d2ed0 con 0x7feb4c13f100 2024-03-28T13:14:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.088+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb30003520 con 0x7feb4c13f100 2024-03-28T13:14:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.088+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2004492906 0 0) 0x7feb4c13f4e0 con 0x7feb4c0d26e0 2024-03-28T13:14:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.088+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb4c0d2ed0 con 0x7feb4c0d26e0 2024-03-28T13:14:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3296877730 0 0) 0x7feb30003520 con 0x7feb4c13f100 2024-03-28T13:14:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb4c070250 msgr2=0x7feb4c0d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb4a000640 1 --2- 172.21.15.19:0/3225491274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb4c070250 0x7feb4c0d21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 >> v1:172.21.15.188:6789/0 conn(0x7feb4c0d26e0 legacy=0x7feb4c0d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4c141940 con 0x7feb4c13f100 2024-03-28T13:14:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7feb3c001670 con 0x7feb4c13f100 2024-03-28T13:14:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 >> v1:172.21.15.19:6789/0 conn(0x7feb4c13f100 legacy=0x7feb4c06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb4a000640 1 --2- 172.21.15.19:0/3225491274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb30003b70 0x7feb30003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb4a000640 1 --2- 172.21.15.19:0/3225491274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb30004800 0x7feb30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb4a000640 1 --2- 172.21.15.19:0/3225491274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb300051c0 0x7feb3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feb4c142270 con 0x7feb30004800 2024-03-28T13:14:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feb300103c0 con 0x7feb30003b70 2024-03-28T13:14:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feb300105c0 con 0x7feb300051c0 2024-03-28T13:14:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb43e00640 1 --2- 172.21.15.19:0/3225491274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb30004800 0x7feb30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb4b400640 1 --2- 172.21.15.19:0/3225491274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb300051c0 0x7feb3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.089+0000 7feb4aa00640 1 --2- 172.21.15.19:0/3225491274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb30003b70 0x7feb30003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.090+0000 7feb43e00640 1 -- 172.21.15.19:0/3225491274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb300051c0 msgr2=0x7feb3000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.090+0000 7feb43e00640 1 --2- 172.21.15.19:0/3225491274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb300051c0 0x7feb3000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.090+0000 7feb43e00640 1 -- 172.21.15.19:0/3225491274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb30003b70 msgr2=0x7feb30003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.090+0000 7feb43e00640 1 --2- 172.21.15.19:0/3225491274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb30003b70 0x7feb30003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.090+0000 7feb43e00640 1 -- 172.21.15.19:0/3225491274 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb30003520 con 0x7feb30004800 2024-03-28T13:14:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.090+0000 7feb4aa00640 1 --2- 172.21.15.19:0/3225491274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb30003b70 0x7feb30003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:14:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.092+0000 7feb43e00640 1 --2- 172.21.15.19:0/3225491274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb30004800 0x7feb30004c80 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7feb4c071d10 tx=0x7feb30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb83381439ba01a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.092+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34039070 con 0x7feb30004800 2024-03-28T13:14:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.092+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb3402c580 con 0x7feb30004800 2024-03-28T13:14:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.092+0000 7feb4a000640 1 -- 172.21.15.19:0/3225491274 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34009650 con 0x7feb30004800 2024-03-28T13:14:50.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.093+0000 7feb4be00640 1 -- 172.21.15.19:0/3225491274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb30004800 msgr2=0x7feb30004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:50.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.093+0000 7feb4be00640 1 --2- 172.21.15.19:0/3225491274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb30004800 0x7feb30004c80 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7feb4c071d10 tx=0x7feb30005830 comp rx=0 tx=0).stop 2024-03-28T13:14:50.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.093+0000 7feb4b400640 1 -- 172.21.15.19:0/3225491274 reap_dead start 2024-03-28T13:14:50.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.093+0000 7feb4be00640 1 -- 172.21.15.19:0/3225491274 shutdown_connections 2024-03-28T13:14:50.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.093+0000 7feb4be00640 1 -- 172.21.15.19:0/3225491274 >> 172.21.15.19:0/3225491274 conn(0x7feb4c067460 msgr2=0x7feb4c067840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:50.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.093+0000 7feb4be00640 1 -- 172.21.15.19:0/3225491274 shutdown_connections 2024-03-28T13:14:50.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.093+0000 7feb4be00640 1 -- 172.21.15.19:0/3225491274 wait complete. 2024-03-28T13:14:50.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.096+0000 7feb4be00640 1 Processor -- start 2024-03-28T13:14:50.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.096+0000 7feb4be00640 1 -- start start 2024-03-28T13:14:50.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.097+0000 7feb4be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb30004800 0x7feb4c152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.098+0000 7feb4be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb4c15ca20 0x7feb4c1f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.098+0000 7feb4be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb4c15cea0 0x7feb4c1f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.098+0000 7feb4be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feb4c142700 con 0x7feb30004800 2024-03-28T13:14:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.098+0000 7feb4be00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feb4c141f00 con 0x7feb4c15ca20 2024-03-28T13:14:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.098+0000 7feb4be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feb4c0d2b70 con 0x7feb4c15cea0 2024-03-28T13:14:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.098+0000 7feb4aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb30004800 0x7feb4c152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.098+0000 7feb4aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb30004800 0x7feb4c152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47238/0 (socket says 172.21.15.19:47238) 2024-03-28T13:14:50.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.098+0000 7feb4aa00640 1 -- 172.21.15.19:0/1754123822 learned_addr learned my addr 172.21.15.19:0/1754123822 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:50.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.098+0000 7feb4aa00640 1 -- 172.21.15.19:0/1754123822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb4c15cea0 msgr2=0x7feb4c1f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:50.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.098+0000 7feb4aa00640 1 --2- 172.21.15.19:0/1754123822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb4c15cea0 0x7feb4c1f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:50.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.098+0000 7feb4aa00640 1 -- 172.21.15.19:0/1754123822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb4c15ca20 msgr2=0x7feb4c1f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:50.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.098+0000 7feb4aa00640 1 --2- 172.21.15.19:0/1754123822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb4c15ca20 0x7feb4c1f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:50.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.098+0000 7feb4aa00640 1 -- 172.21.15.19:0/1754123822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4c1552d0 con 0x7feb30004800 2024-03-28T13:14:50.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.099+0000 7feb4aa00640 1 --2- 172.21.15.19:0/1754123822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb30004800 0x7feb4c152b50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00e780 tx=0x7feb3c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:50.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.099+0000 7feb42a00640 1 -- 172.21.15.19:0/1754123822 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c00c6c0 con 0x7feb30004800 2024-03-28T13:14:50.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.099+0000 7feb42a00640 1 -- 172.21.15.19:0/1754123822 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb3c018070 con 0x7feb30004800 2024-03-28T13:14:50.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.099+0000 7feb42a00640 1 -- 172.21.15.19:0/1754123822 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c00cac0 con 0x7feb30004800 2024-03-28T13:14:50.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.099+0000 7feb4be00640 1 -- 172.21.15.19:0/1754123822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb4c1555b0 con 0x7feb30004800 2024-03-28T13:14:50.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.100+0000 7feb4be00640 1 -- 172.21.15.19:0/1754123822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb4c155b70 con 0x7feb30004800 2024-03-28T13:14:50.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.101+0000 7feb4be00640 1 -- 172.21.15.19:0/1754123822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb4c075940 con 0x7feb30004800 2024-03-28T13:14:50.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.102+0000 7feb42a00640 1 -- 172.21.15.19:0/1754123822 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7feb3c00cc60 con 0x7feb30004800 2024-03-28T13:14:50.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.102+0000 7feb42a00640 1 --2- 172.21.15.19:0/1754123822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb2806c530 0x7feb2806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:50.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.103+0000 7feb42a00640 1 -- 172.21.15.19:0/1754123822 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7feb3c08b570 con 0x7feb30004800 2024-03-28T13:14:50.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.103+0000 7feb43e00640 1 --2- 172.21.15.19:0/1754123822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb2806c530 0x7feb2806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:50.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.104+0000 7feb43e00640 1 --2- 172.21.15.19:0/1754123822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb2806c530 0x7feb2806e9f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7feb34036000 tx=0x7feb3402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:50.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.107+0000 7feb42a00640 1 -- 172.21.15.19:0/1754123822 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7feb3c059b20 con 0x7feb30004800 2024-03-28T13:14:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.356+0000 7feb4be00640 1 -- 172.21.15.19:0/1754123822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb4c0b4b00 con 0x7feb30004800 2024-03-28T13:14:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.357+0000 7feb42a00640 1 -- 172.21.15.19:0/1754123822 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5187 (secure 0 0 0) 0x7feb3c059420 con 0x7feb30004800 2024-03-28T13:14:50.357 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:50.357 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/144726418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":144726418},{"type":"v1","addr":"172.21.15.188:6836","nonce":144726418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5363,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2642581607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2642581607},{"type":"v1","addr":"172.21.15.19:6835","nonce":2642581607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1721039756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1721039756},{"type":"v1","addr":"172.21.15.188:6837","nonce":1721039756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:14:31.526577+0000","modified":"2024-03-28T13:14:49.037897+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6834/2130552083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2130552083},{"type":"v1","addr":"172.21.15.19:6834","nonce":2130552083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6708,"qdb_cluster":[6708]},"id":5}]} 2024-03-28T13:14:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 51 2024-03-28T13:14:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.360+0000 7feb4be00640 1 -- 172.21.15.19:0/1754123822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb2806c530 msgr2=0x7feb2806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.360+0000 7feb4be00640 1 --2- 172.21.15.19:0/1754123822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb2806c530 0x7feb2806e9f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7feb34036000 tx=0x7feb3402e000 comp rx=0 tx=0).stop 2024-03-28T13:14:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.360+0000 7feb4be00640 1 -- 172.21.15.19:0/1754123822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb30004800 msgr2=0x7feb4c152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.360+0000 7feb4be00640 1 --2- 172.21.15.19:0/1754123822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb30004800 0x7feb4c152b50 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00e780 tx=0x7feb3c00ec40 comp rx=0 tx=0).stop 2024-03-28T13:14:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.360+0000 7feb4be00640 1 -- 172.21.15.19:0/1754123822 shutdown_connections 2024-03-28T13:14:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.360+0000 7feb4be00640 1 --2- 172.21.15.19:0/1754123822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb2806c530 0x7feb2806e9f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.360+0000 7feb4be00640 1 --2- 172.21.15.19:0/1754123822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb4c15cea0 0x7feb4c1f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.360+0000 7feb4be00640 1 --2- 172.21.15.19:0/1754123822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb4c15ca20 0x7feb4c1f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.360+0000 7feb4be00640 1 --2- 172.21.15.19:0/1754123822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb30004800 0x7feb4c152b50 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.360+0000 7feb4be00640 1 -- 172.21.15.19:0/1754123822 >> 172.21.15.19:0/1754123822 conn(0x7feb4c067460 msgr2=0x7feb4c06c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.360+0000 7feb4be00640 1 -- 172.21.15.19:0/1754123822 shutdown_connections 2024-03-28T13:14:50.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:50.360+0000 7feb4be00640 1 -- 172.21.15.19:0/1754123822 wait complete. 2024-03-28T13:14:50.783 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:14:31.526577+0000', 'modified': '2024-03-28T13:14:49.037897+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 78, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6708}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6708': {'gid': 6708, 'name': 'c', 'rank': 0, 'incarnation': 48, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.19:6834/2130552083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 2130552083}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 2130552083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6708, 'qdb_cluster': [6708]} 2024-03-28T13:14:50.783 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T13:14:50.783 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-28T13:14:51.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.002+0000 7f2614c00640 1 Processor -- start 2024-03-28T13:14:51.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.002+0000 7f2614c00640 1 -- start start 2024-03-28T13:14:51.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.004+0000 7f2614c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2610153580 0x7f26100d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.004+0000 7f2614c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f26100d07c0 0x7f26100aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.004+0000 7f2614c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26100afab0 con 0x7f26100af2e0 2024-03-28T13:14:51.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.004+0000 7f2614c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f26100d0c30 con 0x7f26100d07c0 2024-03-28T13:14:51.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.004+0000 7f2614c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f261015cc80 con 0x7f2610153580 2024-03-28T13:14:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.004+0000 7f2607e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f26100d07c0 0x7f26100aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.005+0000 7f2607e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f26100d07c0 0x7f26100aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47240/0 (socket says 172.21.15.19:47240) 2024-03-28T13:14:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.005+0000 7f2607e00640 1 -- 172.21.15.19:0/126929753 learned_addr learned my addr 172.21.15.19:0/126929753 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.005+0000 7f260de00640 1 --2- 172.21.15.19:0/126929753 >> v2:172.21.15.188:3300/0 conn(0x7f2610153580 0x7f26100d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.005+0000 7f2607e00640 1 -- 172.21.15.19:0/126929753 >> v2:172.21.15.188:3300/0 conn(0x7f2610153580 msgr2=0x7f26100d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.005+0000 7f2607e00640 1 --2- 172.21.15.19:0/126929753 >> v2:172.21.15.188:3300/0 conn(0x7f2610153580 0x7f26100d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.005+0000 7f2607e00640 1 -- 172.21.15.19:0/126929753 >> v1:172.21.15.188:6789/0 conn(0x7f26100af2e0 legacy=0x7f261006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:51.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.005+0000 7f2607e00640 1 -- 172.21.15.19:0/126929753 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261015da00 con 0x7f26100d07c0 2024-03-28T13:14:51.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.007+0000 7f2607e00640 1 --2- 172.21.15.19:0/126929753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26100d07c0 0x7f26100aeda0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f26100b5270 tx=0x7f25f802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eee3196143b1cdb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:51.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.007+0000 7f260d400640 1 -- 172.21.15.19:0/126929753 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f802e070 con 0x7f26100d07c0 2024-03-28T13:14:51.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.008+0000 7f260d400640 1 -- 172.21.15.19:0/126929753 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f8037070 con 0x7f26100d07c0 2024-03-28T13:14:51.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.008+0000 7f260d400640 1 -- 172.21.15.19:0/126929753 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8031a00 con 0x7f26100d07c0 2024-03-28T13:14:51.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.008+0000 7f2614c00640 1 -- 172.21.15.19:0/126929753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26100d07c0 msgr2=0x7f26100aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:51.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.008+0000 7f2614c00640 1 --2- 172.21.15.19:0/126929753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26100d07c0 0x7f26100aeda0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f26100b5270 tx=0x7f25f802b880 comp rx=0 tx=0).stop 2024-03-28T13:14:51.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.008+0000 7f2614c00640 1 -- 172.21.15.19:0/126929753 shutdown_connections 2024-03-28T13:14:51.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.008+0000 7f2614c00640 1 --2- 172.21.15.19:0/126929753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26100d07c0 0x7f26100aeda0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.008+0000 7f2614c00640 1 --2- 172.21.15.19:0/126929753 >> v2:172.21.15.188:3300/0 conn(0x7f2610153580 0x7f26100d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.008+0000 7f2614c00640 1 -- 172.21.15.19:0/126929753 >> 172.21.15.19:0/126929753 conn(0x7f261006b750 msgr2=0x7f261006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.008+0000 7f2614c00640 1 -- 172.21.15.19:0/126929753 shutdown_connections 2024-03-28T13:14:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.008+0000 7f2614c00640 1 -- 172.21.15.19:0/126929753 wait complete. 2024-03-28T13:14:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.010+0000 7f2614c00640 1 Processor -- start 2024-03-28T13:14:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.011+0000 7f2614c00640 1 -- start start 2024-03-28T13:14:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.012+0000 7f2614c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26100af2e0 0x7f261014b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.012+0000 7f2614c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26100d07c0 0x7f261015b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.012+0000 7f2614c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2610153580 0x7f261015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.012+0000 7f2614c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f261015d4e0 con 0x7f26100af2e0 2024-03-28T13:14:51.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.012+0000 7f2614c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f261015d200 con 0x7f2610153580 2024-03-28T13:14:51.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.012+0000 7f2614c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2610153960 con 0x7f26100d07c0 2024-03-28T13:14:51.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.012+0000 7f260de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26100af2e0 0x7f261014b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.012+0000 7f260de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26100af2e0 0x7f261014b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47246/0 (socket says 172.21.15.19:47246) 2024-03-28T13:14:51.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.012+0000 7f260de00640 1 -- 172.21.15.19:0/3007530111 learned_addr learned my addr 172.21.15.19:0/3007530111 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:51.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.012+0000 7f2607e00640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26100d07c0 0x7f261015b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.012+0000 7f260e800640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2610153580 0x7f261015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.012+0000 7f260de00640 1 -- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26100d07c0 msgr2=0x7f261015b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:51.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.013+0000 7f260de00640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26100d07c0 0x7f261015b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.013+0000 7f260de00640 1 -- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2610153580 msgr2=0x7f261015bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:51.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.013+0000 7f260de00640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2610153580 0x7f261015bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.013+0000 7f260de00640 1 -- 172.21.15.19:0/3007530111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26101f7970 con 0x7f26100af2e0 2024-03-28T13:14:51.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.013+0000 7f2607e00640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26100d07c0 0x7f261015b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:51.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.013+0000 7f260e800640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2610153580 0x7f261015bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:51.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.013+0000 7f260de00640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26100af2e0 0x7f261014b420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f260000d800 tx=0x7f260000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:51.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.013+0000 7f2606000640 1 -- 172.21.15.19:0/3007530111 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260000b6c0 con 0x7f26100af2e0 2024-03-28T13:14:51.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.013+0000 7f2606000640 1 -- 172.21.15.19:0/3007530111 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2600019070 con 0x7f26100af2e0 2024-03-28T13:14:51.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.013+0000 7f2606000640 1 -- 172.21.15.19:0/3007530111 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260000ba80 con 0x7f26100af2e0 2024-03-28T13:14:51.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.013+0000 7f2614c00640 1 -- 172.21.15.19:0/3007530111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26101f7c30 con 0x7f26100af2e0 2024-03-28T13:14:51.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.013+0000 7f2614c00640 1 -- 172.21.15.19:0/3007530111 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26101f81f0 con 0x7f26100af2e0 2024-03-28T13:14:51.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.016+0000 7f2606000640 1 -- 172.21.15.19:0/3007530111 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f260000bc20 con 0x7f26100af2e0 2024-03-28T13:14:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.017+0000 7f2606000640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f25ec06c530 0x7f25ec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.018+0000 7f2606000640 1 -- 172.21.15.19:0/3007530111 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7f260008bca0 con 0x7f26100af2e0 2024-03-28T13:14:51.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.018+0000 7f26177e2740 1 Processor -- start 2024-03-28T13:14:51.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.018+0000 7f2607e00640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f25ec06c530 0x7f25ec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.018+0000 7f26177e2740 1 -- start start 2024-03-28T13:14:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.019+0000 7f26177e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f040fd0 0x56241f013f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.019+0000 7f26177e2740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f0413b0 0x56241f043d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.019+0000 7f26177e2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f0144b0 0x56241f0442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.019+0000 7f26177e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56241efdb920 con 0x56241f040fd0 2024-03-28T13:14:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.019+0000 7f26177e2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56241ef2bfa0 con 0x56241f0144b0 2024-03-28T13:14:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.019+0000 7f26177e2740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x56241ef7df90 con 0x56241f0413b0 2024-03-28T13:14:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.019+0000 7f260de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f040fd0 0x56241f013f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.020+0000 7f260de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f040fd0 0x56241f013f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47258/0 (socket says 172.21.15.19:47258) 2024-03-28T13:14:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.020+0000 7f260de00640 1 -- 172.21.15.19:0/2136053416 learned_addr learned my addr 172.21.15.19:0/2136053416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.020+0000 7f2607e00640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f25ec06c530 0x7f25ec06e9f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f25f8036000 tx=0x7f25f8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.020+0000 7f260e800640 1 --2- 172.21.15.19:0/2136053416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f0144b0 0x56241f0442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.020+0000 7f260de00640 1 -- 172.21.15.19:0/2136053416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f0413b0 msgr2=0x56241f043d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.020+0000 7f260de00640 1 --2- 172.21.15.19:0/2136053416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f0413b0 0x56241f043d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.020+0000 7f260de00640 1 -- 172.21.15.19:0/2136053416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f0144b0 msgr2=0x56241f0442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.020+0000 7f260de00640 1 --2- 172.21.15.19:0/2136053416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f0144b0 0x56241f0442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.020+0000 7f260de00640 1 -- 172.21.15.19:0/2136053416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56241ec922d0 con 0x56241f040fd0 2024-03-28T13:14:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.020+0000 7f260de00640 1 --2- 172.21.15.19:0/2136053416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f040fd0 0x56241f013f70 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f260008a070 tx=0x7f260005a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a3a6fa9fc11dd1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.020+0000 7f25fe600640 1 -- 172.21.15.19:0/2136053416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260005de70 con 0x56241f040fd0 2024-03-28T13:14:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.020+0000 7f25fe600640 1 -- 172.21.15.19:0/2136053416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26000626a0 con 0x56241f040fd0 2024-03-28T13:14:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.020+0000 7f25fe600640 1 -- 172.21.15.19:0/2136053416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600057dc0 con 0x56241f040fd0 2024-03-28T13:14:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.021+0000 7f26177e2740 1 -- 172.21.15.19:0/2136053416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f040fd0 msgr2=0x56241f013f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.021+0000 7f26177e2740 1 --2- 172.21.15.19:0/2136053416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f040fd0 0x56241f013f70 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f260008a070 tx=0x7f260005a1c0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.021+0000 7f26177e2740 1 -- 172.21.15.19:0/2136053416 shutdown_connections 2024-03-28T13:14:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.021+0000 7f26177e2740 1 --2- 172.21.15.19:0/2136053416 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f0144b0 0x56241f0442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.021+0000 7f26177e2740 1 --2- 172.21.15.19:0/2136053416 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f0413b0 0x56241f043d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.021+0000 7f26177e2740 1 --2- 172.21.15.19:0/2136053416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f040fd0 0x56241f013f70 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.021+0000 7f26177e2740 1 -- 172.21.15.19:0/2136053416 >> 172.21.15.19:0/2136053416 conn(0x56241f0161e0 msgr2=0x56241f05ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.021+0000 7f26177e2740 1 -- 172.21.15.19:0/2136053416 shutdown_connections 2024-03-28T13:14:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.021+0000 7f26177e2740 1 -- 172.21.15.19:0/2136053416 wait complete. 2024-03-28T13:14:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.022+0000 7f26177e2740 1 -- start start 2024-03-28T13:14:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.023+0000 7f26177e2740 1 Processor -- start 2024-03-28T13:14:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.024+0000 7f26177e2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f040fd0 0x56241f03c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.024+0000 7f26177e2740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f0413b0 0x56241f06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.024+0000 7f26177e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f047c30 0x56241f046ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.024+0000 7f26177e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56241ef79f40 con 0x56241f047c30 2024-03-28T13:14:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.024+0000 7f26177e2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56241ef26650 con 0x56241f040fd0 2024-03-28T13:14:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.024+0000 7f26177e2740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x56241ef39a10 con 0x56241f0413b0 2024-03-28T13:14:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.024+0000 7f26177e2740 10 client.0 _ll_register_callbacks cb 0x56241ef78ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:14:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.025+0000 7f260e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f047c30 0x56241f046ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.025+0000 7f260e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f047c30 0x56241f046ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47274/0 (socket says 172.21.15.19:47274) 2024-03-28T13:14:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.025+0000 7f260e800640 1 -- 172.21.15.19:0/94503966 learned_addr learned my addr 172.21.15.19:0/94503966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:51.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.025+0000 7f260de00640 1 --2- 172.21.15.19:0/94503966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f040fd0 0x56241f03c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.025+0000 7f2607e00640 1 --2- 172.21.15.19:0/94503966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f0413b0 0x56241f06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.025+0000 7f260e800640 1 -- 172.21.15.19:0/94503966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f0413b0 msgr2=0x56241f06fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.025+0000 7f260e800640 1 --2- 172.21.15.19:0/94503966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f0413b0 0x56241f06fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.025+0000 7f260e800640 1 -- 172.21.15.19:0/94503966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f040fd0 msgr2=0x56241f03c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.025+0000 7f260e800640 1 --2- 172.21.15.19:0/94503966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f040fd0 0x56241f03c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.025+0000 7f260e800640 1 -- 172.21.15.19:0/94503966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56241ed5c6b0 con 0x56241f047c30 2024-03-28T13:14:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.026+0000 7f260e800640 1 --2- 172.21.15.19:0/94503966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f047c30 0x56241f046ce0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f260800d9e0 tx=0x7f260800dea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fad124278601356 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.026+0000 7f25f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:14:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.026+0000 7f25f2a00640 1 -- 172.21.15.19:0/94503966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260800bcb0 con 0x56241f047c30 2024-03-28T13:14:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.026+0000 7f25f2a00640 1 -- 172.21.15.19:0/94503966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7f2608017420 con 0x56241f047c30 2024-03-28T13:14:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.026+0000 7f26177e2740 1 -- 172.21.15.19:0/94503966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26000626a0 con 0x56241f047c30 2024-03-28T13:14:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.026+0000 7f25f2a00640 1 -- 172.21.15.19:0/94503966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f2608021050 con 0x56241f047c30 2024-03-28T13:14:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.026+0000 7f26177e2740 10 client.6756 fetch_fsmap learned FSMap version 51 2024-03-28T13:14:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.026+0000 7f26177e2740 1 -- 172.21.15.19:0/94503966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x56241f045f20 con 0x56241f047c30 2024-03-28T13:14:51.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.026+0000 7f25f2a00640 1 -- 172.21.15.19:0/94503966 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26000626a0 con 0x56241f047c30 2024-03-28T13:14:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.027+0000 7f25f2a00640 1 -- 172.21.15.19:0/94503966 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f2608024020 con 0x56241f047c30 2024-03-28T13:14:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.027+0000 7f26177e2740 10 client.6756 fetch_fsmap finished waiting for FSMap version 51 2024-03-28T13:14:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.027+0000 7f26177e2740 10 client.6756 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T13:14:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.027+0000 7f26177e2740 20 client.6756 populate_metadata read hostname 'smithi019' 2024-03-28T13:14:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.027+0000 7f26177e2740 1 --2- 172.21.15.19:0/94503966 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x56241f072e90 0x56241f07da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.027+0000 7f26177e2740 4 client.6756 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:14:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.027+0000 7f26177e2740 1 -- 172.21.15.19:0/94503966 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56241ef8b4d0 con 0x56241f072e90 2024-03-28T13:14:51.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.027+0000 7f260de00640 1 --2- 172.21.15.19:0/94503966 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x56241f072e90 0x56241f07da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.028+0000 7f260de00640 1 --2- 172.21.15.19:0/94503966 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x56241f072e90 0x56241f07da50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6622c535d0f912f9 server_cookie=aa60d030ca87bb79 in_seq=0 out_seq=0 2024-03-28T13:14:51.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.028+0000 7f25f2a00640 10 client.6756 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:14:51.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.029+0000 7f25f2a00640 1 -- 172.21.15.19:0/94503966 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56241ef8b4d0 con 0x56241f072e90 2024-03-28T13:14:51.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.029+0000 7f25f2a00640 10 client.6756 handle_command_reply: tid=0 2024-03-28T13:14:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.029+0000 7f26177e2740 1 client.6756 shutdown 2024-03-28T13:14:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.029+0000 7f26177e2740 1 -- 172.21.15.19:0/94503966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f047c30 msgr2=0x56241f046ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.029+0000 7f26177e2740 1 --2- 172.21.15.19:0/94503966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f047c30 0x56241f046ce0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f260800d9e0 tx=0x7f260800dea0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.029+0000 7f26177e2740 1 -- 172.21.15.19:0/94503966 shutdown_connections 2024-03-28T13:14:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.029+0000 7f26177e2740 1 --2- 172.21.15.19:0/94503966 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x56241f072e90 0x56241f07da50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.030+0000 7f26177e2740 1 --2- 172.21.15.19:0/94503966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f047c30 0x56241f046ce0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.030+0000 7f26177e2740 1 --2- 172.21.15.19:0/94503966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f0413b0 0x56241f06fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.030+0000 7f26177e2740 1 --2- 172.21.15.19:0/94503966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f040fd0 0x56241f03c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.030+0000 7f26177e2740 1 -- 172.21.15.19:0/94503966 >> 172.21.15.19:0/94503966 conn(0x56241f0144b0 msgr2=0x56241f013690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.030+0000 7f25f2a00640 0 client.6756 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:14:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.030+0000 7f26177e2740 1 -- 172.21.15.19:0/94503966 shutdown_connections 2024-03-28T13:14:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.030+0000 7f26177e2740 1 -- 172.21.15.19:0/94503966 wait complete. 2024-03-28T13:14:51.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.030+0000 7f26177e2740 20 client.6756 trim_cache size 0 max 16384 2024-03-28T13:14:51.034 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-03-28T13:14:51.034+0000 7fe33d200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:14:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.069+0000 7f26177e2740 1 Processor -- start 2024-03-28T13:14:51.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.070+0000 7f26177e2740 1 -- start start 2024-03-28T13:14:51.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.071+0000 7f26177e2740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f040fd0 0x56241f061fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.071+0000 7f26177e2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f0413b0 0x56241f062500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.071+0000 7f26177e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f047c30 0x56241f082a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.071+0000 7f26177e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26101ffd80 con 0x56241f047c30 2024-03-28T13:14:51.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.071+0000 7f26177e2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26100ce860 con 0x56241f0413b0 2024-03-28T13:14:51.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.071+0000 7f26177e2740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f26101f4e70 con 0x56241f040fd0 2024-03-28T13:14:51.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.071+0000 7f260e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f047c30 0x56241f082a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.071+0000 7f260e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f047c30 0x56241f082a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47288/0 (socket says 172.21.15.19:47288) 2024-03-28T13:14:51.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.071+0000 7f260e800640 1 -- 172.21.15.19:0/1836676792 learned_addr learned my addr 172.21.15.19:0/1836676792 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:51.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.072+0000 7f260e800640 1 -- 172.21.15.19:0/1836676792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f040fd0 msgr2=0x56241f061fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:51.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.072+0000 7f260e800640 1 --2- 172.21.15.19:0/1836676792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f040fd0 0x56241f061fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.072+0000 7f260e800640 1 -- 172.21.15.19:0/1836676792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f0413b0 msgr2=0x56241f062500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:51.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.072+0000 7f260e800640 1 --2- 172.21.15.19:0/1836676792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f0413b0 0x56241f062500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.072+0000 7f260e800640 1 -- 172.21.15.19:0/1836676792 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56241f048010 con 0x56241f047c30 2024-03-28T13:14:51.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.072+0000 7f260e800640 1 --2- 172.21.15.19:0/1836676792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f047c30 0x56241f082a80 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2608002230 tx=0x7f2608012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4a04a861ca6296 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:51.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.073+0000 7f25fdc00640 1 -- 172.21.15.19:0/1836676792 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260800bcb0 con 0x56241f047c30 2024-03-28T13:14:51.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.073+0000 7f25fdc00640 1 -- 172.21.15.19:0/1836676792 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2608003970 con 0x56241f047c30 2024-03-28T13:14:51.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.073+0000 7f25fdc00640 1 -- 172.21.15.19:0/1836676792 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2608003d40 con 0x56241f047c30 2024-03-28T13:14:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.073+0000 7f26177e2740 1 -- 172.21.15.19:0/1836676792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f047c30 msgr2=0x56241f082a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.073+0000 7f26177e2740 1 --2- 172.21.15.19:0/1836676792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f047c30 0x56241f082a80 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2608002230 tx=0x7f2608012000 comp rx=0 tx=0).stop 2024-03-28T13:14:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.073+0000 7f26177e2740 1 -- 172.21.15.19:0/1836676792 shutdown_connections 2024-03-28T13:14:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.073+0000 7f26177e2740 1 --2- 172.21.15.19:0/1836676792 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f047c30 0x56241f082a80 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.073+0000 7f26177e2740 1 --2- 172.21.15.19:0/1836676792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f0413b0 0x56241f062500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.073+0000 7f26177e2740 1 --2- 172.21.15.19:0/1836676792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f040fd0 0x56241f061fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.073+0000 7f26177e2740 1 -- 172.21.15.19:0/1836676792 >> 172.21.15.19:0/1836676792 conn(0x56241f0144b0 msgr2=0x56241f043f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.074+0000 7f26177e2740 1 -- 172.21.15.19:0/1836676792 shutdown_connections 2024-03-28T13:14:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.074+0000 7f26177e2740 1 -- 172.21.15.19:0/1836676792 wait complete. 2024-03-28T13:14:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.074+0000 7f26177e2740 1 -- start start 2024-03-28T13:14:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.074+0000 7f26177e2740 1 Processor -- start 2024-03-28T13:14:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.075+0000 7f26177e2740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f040fd0 0x56241f0d8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.075+0000 7f26177e2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f0413b0 0x56241f0193e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.075+0000 7f26177e2740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f047c30 0x56241f06d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.075+0000 7f26177e2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56241f03d920 con 0x56241f0413b0 2024-03-28T13:14:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.075+0000 7f26177e2740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56241f046200 con 0x56241f040fd0 2024-03-28T13:14:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.075+0000 7f26177e2740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x56241f0560e0 con 0x56241f047c30 2024-03-28T13:14:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.075+0000 7f26177e2740 10 client.0 _ll_register_callbacks cb 0x7f26080085e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:14:51.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.076+0000 7f2607e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f0413b0 0x56241f0193e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.076+0000 7f2607e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f0413b0 0x56241f0193e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47292/0 (socket says 172.21.15.19:47292) 2024-03-28T13:14:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.076+0000 7f2607e00640 1 -- 172.21.15.19:0/3035125600 learned_addr learned my addr 172.21.15.19:0/3035125600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.076+0000 7f260de00640 1 --2- 172.21.15.19:0/3035125600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f040fd0 0x56241f0d8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.076+0000 7f2607e00640 1 -- 172.21.15.19:0/3035125600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f047c30 msgr2=0x56241f06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.076+0000 7f2607e00640 1 --2- 172.21.15.19:0/3035125600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f047c30 0x56241f06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.076+0000 7f2607e00640 1 -- 172.21.15.19:0/3035125600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f040fd0 msgr2=0x56241f0d8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.076+0000 7f2607e00640 1 --2- 172.21.15.19:0/3035125600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f040fd0 0x56241f0d8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.076+0000 7f2607e00640 1 -- 172.21.15.19:0/3035125600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56241f056b50 con 0x56241f0413b0 2024-03-28T13:14:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.076+0000 7f260de00640 1 --2- 172.21.15.19:0/3035125600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f040fd0 0x56241f0d8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.076+0000 7f2607e00640 1 --2- 172.21.15.19:0/3035125600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f0413b0 0x56241f0193e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f25f80090b0 tx=0x7f25f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a63aecda35a1a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:51.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.076+0000 7f25f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:14:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.077+0000 7f25f2a00640 1 -- 172.21.15.19:0/3035125600 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8045600 con 0x56241f0413b0 2024-03-28T13:14:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.077+0000 7f25f2a00640 1 -- 172.21.15.19:0/3035125600 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7f25f8050b10 con 0x56241f0413b0 2024-03-28T13:14:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.077+0000 7f26177e2740 1 -- 172.21.15.19:0/3035125600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2608003970 con 0x56241f0413b0 2024-03-28T13:14:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.077+0000 7f25f2a00640 1 -- 172.21.15.19:0/3035125600 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f25f8050d10 con 0x56241f0413b0 2024-03-28T13:14:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.077+0000 7f26177e2740 10 client.6762 fetch_fsmap learned FSMap version 51 2024-03-28T13:14:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.077+0000 7f26177e2740 1 -- 172.21.15.19:0/3035125600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) -- 0x56241f018c40 con 0x56241f0413b0 2024-03-28T13:14:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.077+0000 7f25f2a00640 1 -- 172.21.15.19:0/3035125600 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2608003970 con 0x56241f0413b0 2024-03-28T13:14:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.078+0000 7f25f2a00640 1 -- 172.21.15.19:0/3035125600 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 51) ==== 3494+0+0 (secure 0 0 0) 0x7f25f80457a0 con 0x56241f0413b0 2024-03-28T13:14:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.078+0000 7f26177e2740 10 client.6762 fetch_fsmap finished waiting for FSMap version 51 2024-03-28T13:14:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.078+0000 7f26177e2740 10 client.6762 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T13:14:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.078+0000 7f26177e2740 20 client.6762 populate_metadata read hostname 'smithi019' 2024-03-28T13:14:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.078+0000 7f26177e2740 1 --2- 172.21.15.19:0/3035125600 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x56241f0822c0 0x56241f0e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.078+0000 7f26177e2740 4 client.6762 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T13:14:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.078+0000 7f26177e2740 1 -- 172.21.15.19:0/3035125600 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "status"}) -- 0x56241ef85820 con 0x56241f0822c0 2024-03-28T13:14:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.078+0000 7f260de00640 1 --2- 172.21.15.19:0/3035125600 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x56241f0822c0 0x56241f0e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.078+0000 7f260de00640 1 --2- 172.21.15.19:0/3035125600 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x56241f0822c0 0x56241f0e4800 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f82f6ffd0408a6d8 server_cookie=a0d15e139fabde49 in_seq=0 out_seq=0 2024-03-28T13:14:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.078+0000 7f25f2a00640 10 client.6762 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:14:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.079+0000 7f25f2a00640 1 -- 172.21.15.19:0/3035125600 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x56241ef85820 con 0x56241f0822c0 2024-03-28T13:14:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.079+0000 7f25f2a00640 10 client.6762 handle_command_reply: tid=0 2024-03-28T13:14:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.079+0000 7f26177e2740 1 client.6762 shutdown 2024-03-28T13:14:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.079+0000 7f26177e2740 1 -- 172.21.15.19:0/3035125600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f0413b0 msgr2=0x56241f0193e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.079+0000 7f26177e2740 1 --2- 172.21.15.19:0/3035125600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f0413b0 0x56241f0193e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f25f80090b0 tx=0x7f25f80046c0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.079+0000 7f26177e2740 1 -- 172.21.15.19:0/3035125600 shutdown_connections 2024-03-28T13:14:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.079+0000 7f26177e2740 1 --2- 172.21.15.19:0/3035125600 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x56241f0822c0 0x56241f0e4800 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.079+0000 7f26177e2740 1 --2- 172.21.15.19:0/3035125600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56241f047c30 0x56241f06d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.079+0000 7f26177e2740 1 --2- 172.21.15.19:0/3035125600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56241f0413b0 0x56241f0193e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.079+0000 7f26177e2740 1 --2- 172.21.15.19:0/3035125600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56241f040fd0 0x56241f0d8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.079+0000 7f26177e2740 1 -- 172.21.15.19:0/3035125600 >> 172.21.15.19:0/3035125600 conn(0x56241f0144b0 msgr2=0x56241f061fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:51.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.079+0000 7f25f2a00640 0 client.6762 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:14:51.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.079+0000 7f26177e2740 1 -- 172.21.15.19:0/3035125600 shutdown_connections 2024-03-28T13:14:51.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.079+0000 7f26177e2740 1 -- 172.21.15.19:0/3035125600 wait complete. 2024-03-28T13:14:51.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.080+0000 7f26177e2740 20 client.6762 trim_cache size 0 max 16384 2024-03-28T13:14:51.082 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:14:51.082 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "e6b569d9-f2ce-49dc-bbe7-c0ab33080203", 2024-03-28T13:14:51.082 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-03-28T13:14:51.082 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 6708, 2024-03-28T13:14:51.082 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-03-28T13:14:51.082 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-03-28T13:14:51.082 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-03-28T13:14:51.083 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 5.0482984020000004, 2024-03-28T13:14:51.083 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 51, 2024-03-28T13:14:51.083 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 78, 2024-03-28T13:14:51.083 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 78, 2024-03-28T13:14:51.083 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 5.968196657 2024-03-28T13:14:51.083 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:14:51.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.080+0000 7f2614c00640 1 -- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f25ec06c530 msgr2=0x7f25ec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.080+0000 7f2614c00640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f25ec06c530 0x7f25ec06e9f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f25f8036000 tx=0x7f25f8008030 comp rx=0 tx=0).stop 2024-03-28T13:14:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.080+0000 7f2614c00640 1 -- 172.21.15.19:0/3007530111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26100af2e0 msgr2=0x7f261014b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.080+0000 7f2614c00640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26100af2e0 0x7f261014b420 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f260000d800 tx=0x7f260000dcc0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.080+0000 7f2614c00640 1 -- 172.21.15.19:0/3007530111 shutdown_connections 2024-03-28T13:14:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.080+0000 7f2614c00640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f25ec06c530 0x7f25ec06e9f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.080+0000 7f2614c00640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2610153580 0x7f261015bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.080+0000 7f2614c00640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26100d07c0 0x7f261015b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.080+0000 7f2614c00640 1 --2- 172.21.15.19:0/3007530111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26100af2e0 0x7f261014b420 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.080+0000 7f2614c00640 1 -- 172.21.15.19:0/3007530111 >> 172.21.15.19:0/3007530111 conn(0x7f261006b750 msgr2=0x7f261006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.080+0000 7f2614c00640 1 -- 172.21.15.19:0/3007530111 shutdown_connections 2024-03-28T13:14:51.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.080+0000 7f2614c00640 1 -- 172.21.15.19:0/3007530111 wait complete. 2024-03-28T13:14:51.097 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:14:51.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.306+0000 7fc01b200640 1 Processor -- start 2024-03-28T13:14:51.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.306+0000 7fc01b200640 1 -- start start 2024-03-28T13:14:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.308+0000 7fc01b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc01c1451a0 0x7fc01c145c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.308+0000 7fc01b200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc01c0b4770 0x7fc01c0d1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.308+0000 7fc01b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc01c0d0bc0 con 0x7fc01c0b5ec0 2024-03-28T13:14:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.308+0000 7fc01b200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc01c0d1470 con 0x7fc01c0b4770 2024-03-28T13:14:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.308+0000 7fc01b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c145580 con 0x7fc01c1451a0 2024-03-28T13:14:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.308+0000 7fc013e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc01c0b5ec0 0x7fc01c0b4060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51544/0 (socket says 172.21.15.19:51544) 2024-03-28T13:14:51.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.308+0000 7fc013e00640 1 -- 172.21.15.19:0/2258876195 learned_addr learned my addr 172.21.15.19:0/2258876195 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:51.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.309+0000 7fc019e00640 1 --2- 172.21.15.19:0/2258876195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc01c1451a0 0x7fc01c145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.309+0000 7fc01a800640 1 --2- 172.21.15.19:0/2258876195 >> v2:172.21.15.188:3300/0 conn(0x7fc01c0b4770 0x7fc01c0d1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.309+0000 7fc019400640 1 -- 172.21.15.19:0/2258876195 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 742697963 0 0) 0x7fc004001110 con 0x7fc01c0b5ec0 2024-03-28T13:14:51.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.309+0000 7fc019400640 1 -- 172.21.15.19:0/2258876195 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc000003520 con 0x7fc01c0b5ec0 2024-03-28T13:14:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.309+0000 7fc019400640 1 -- 172.21.15.19:0/2258876195 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1668563826 0 0) 0x7fc000003520 con 0x7fc01c0b5ec0 2024-03-28T13:14:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.309+0000 7fc019400640 1 -- 172.21.15.19:0/2258876195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc01c1451a0 msgr2=0x7fc01c145c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.309+0000 7fc019400640 1 --2- 172.21.15.19:0/2258876195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc01c1451a0 0x7fc01c145c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.309+0000 7fc019400640 1 -- 172.21.15.19:0/2258876195 >> v2:172.21.15.188:3300/0 conn(0x7fc01c0b4770 msgr2=0x7fc01c0d1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.309+0000 7fc019400640 1 --2- 172.21.15.19:0/2258876195 >> v2:172.21.15.188:3300/0 conn(0x7fc01c0b4770 0x7fc01c0d1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.309+0000 7fc019400640 1 -- 172.21.15.19:0/2258876195 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c0d0e40 con 0x7fc01c0b5ec0 2024-03-28T13:14:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.309+0000 7fc019400640 1 -- 172.21.15.19:0/2258876195 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fc004001840 con 0x7fc01c0b5ec0 2024-03-28T13:14:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.309+0000 7fc019400640 1 -- 172.21.15.19:0/2258876195 >> v1:172.21.15.19:6789/0 conn(0x7fc01c0b5ec0 legacy=0x7fc01c0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.309+0000 7fc019400640 1 --2- 172.21.15.19:0/2258876195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0000039b0 0x7fc000003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.309+0000 7fc019400640 1 --2- 172.21.15.19:0/2258876195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc01c1451a0 0x7fc000004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.310+0000 7fc019400640 1 --2- 172.21.15.19:0/2258876195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc000004d40 0x7fc0000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.310+0000 7fc019400640 1 -- 172.21.15.19:0/2258876195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c0d1470 con 0x7fc0000039b0 2024-03-28T13:14:51.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.310+0000 7fc019400640 1 -- 172.21.15.19:0/2258876195 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c145580 con 0x7fc000004d40 2024-03-28T13:14:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.310+0000 7fc019400640 1 -- 172.21.15.19:0/2258876195 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc00000b9c0 con 0x7fc01c1451a0 2024-03-28T13:14:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.310+0000 7fc019e00640 1 --2- 172.21.15.19:0/2258876195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0000039b0 0x7fc000003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.310+0000 7fc013e00640 1 --2- 172.21.15.19:0/2258876195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc01c1451a0 0x7fc000004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.310+0000 7fc019e00640 1 -- 172.21.15.19:0/2258876195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc01c1451a0 msgr2=0x7fc000004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.310+0000 7fc01a800640 1 --2- 172.21.15.19:0/2258876195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc000004d40 0x7fc0000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.310+0000 7fc019e00640 1 --2- 172.21.15.19:0/2258876195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc01c1451a0 0x7fc000004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.310+0000 7fc019e00640 1 -- 172.21.15.19:0/2258876195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc000004d40 msgr2=0x7fc0000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.310+0000 7fc019e00640 1 --2- 172.21.15.19:0/2258876195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc000004d40 0x7fc0000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.310+0000 7fc019e00640 1 -- 172.21.15.19:0/2258876195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc000003520 con 0x7fc0000039b0 2024-03-28T13:14:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.310+0000 7fc013e00640 1 --2- 172.21.15.19:0/2258876195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc01c1451a0 0x7fc000004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:51.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.310+0000 7fc01a800640 1 --2- 172.21.15.19:0/2258876195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc000004d40 0x7fc0000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:51.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.311+0000 7fc019e00640 1 --2- 172.21.15.19:0/2258876195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0000039b0 0x7fc000003dd0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009ac0 tx=0x7fc000005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ebef427fa5871a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.312+0000 7fc019400640 1 -- 172.21.15.19:0/2258876195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c039070 con 0x7fc0000039b0 2024-03-28T13:14:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.312+0000 7fc019400640 1 -- 172.21.15.19:0/2258876195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00c02c6b0 con 0x7fc0000039b0 2024-03-28T13:14:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.312+0000 7fc019400640 1 -- 172.21.15.19:0/2258876195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c009690 con 0x7fc0000039b0 2024-03-28T13:14:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.312+0000 7fc01b200640 1 -- 172.21.15.19:0/2258876195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0000039b0 msgr2=0x7fc000003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.312+0000 7fc01b200640 1 --2- 172.21.15.19:0/2258876195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0000039b0 0x7fc000003dd0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009ac0 tx=0x7fc000005830 comp rx=0 tx=0).stop 2024-03-28T13:14:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.312+0000 7fc01a800640 1 -- 172.21.15.19:0/2258876195 reap_dead start 2024-03-28T13:14:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.312+0000 7fc01b200640 1 -- 172.21.15.19:0/2258876195 shutdown_connections 2024-03-28T13:14:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.312+0000 7fc01b200640 1 -- 172.21.15.19:0/2258876195 >> 172.21.15.19:0/2258876195 conn(0x7fc01c067460 msgr2=0x7fc01c067870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.312+0000 7fc01b200640 1 -- 172.21.15.19:0/2258876195 shutdown_connections 2024-03-28T13:14:51.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.313+0000 7fc01b200640 1 -- 172.21.15.19:0/2258876195 wait complete. 2024-03-28T13:14:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.314+0000 7fc01b200640 1 Processor -- start 2024-03-28T13:14:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.314+0000 7fc01b200640 1 -- start start 2024-03-28T13:14:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.315+0000 7fc01b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0000039b0 0x7fc01c0ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.315+0000 7fc01b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc01c1434f0 0x7fc01c0aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.315+0000 7fc01b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc01c143990 0x7fc01c0af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.315+0000 7fc01b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c0b6320 con 0x7fc0000039b0 2024-03-28T13:14:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.315+0000 7fc01b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c0d1100 con 0x7fc01c143990 2024-03-28T13:14:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.315+0000 7fc01b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c0b64a0 con 0x7fc01c1434f0 2024-03-28T13:14:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.316+0000 7fc019e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0000039b0 0x7fc01c0ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.316+0000 7fc019e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0000039b0 0x7fc01c0ae5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47310/0 (socket says 172.21.15.19:47310) 2024-03-28T13:14:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.316+0000 7fc019e00640 1 -- 172.21.15.19:0/3152421694 learned_addr learned my addr 172.21.15.19:0/3152421694 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.316+0000 7fc013e00640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc01c1434f0 0x7fc01c0aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.316+0000 7fc019e00640 1 -- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc01c1434f0 msgr2=0x7fc01c0aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:51.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.316+0000 7fc01a800640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc01c143990 0x7fc01c0af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.316+0000 7fc019e00640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc01c1434f0 0x7fc01c0aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.316+0000 7fc019e00640 1 -- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc01c143990 msgr2=0x7fc01c0af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.316+0000 7fc019e00640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc01c143990 0x7fc01c0af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.316+0000 7fc019e00640 1 -- 172.21.15.19:0/3152421694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c0ad050 con 0x7fc0000039b0 2024-03-28T13:14:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.316+0000 7fc013e00640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc01c1434f0 0x7fc01c0aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.316+0000 7fc01a800640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc01c143990 0x7fc01c0af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.316+0000 7fc019e00640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0000039b0 0x7fc01c0ae5b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009be0 tx=0x7fc00c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.317+0000 7fc012000640 1 -- 172.21.15.19:0/3152421694 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c042020 con 0x7fc0000039b0 2024-03-28T13:14:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.317+0000 7fc012000640 1 -- 172.21.15.19:0/3152421694 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc00c03a040 con 0x7fc0000039b0 2024-03-28T13:14:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.317+0000 7fc012000640 1 -- 172.21.15.19:0/3152421694 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c039070 con 0x7fc0000039b0 2024-03-28T13:14:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.317+0000 7fc01b200640 1 -- 172.21.15.19:0/3152421694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c0b0d20 con 0x7fc0000039b0 2024-03-28T13:14:51.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.317+0000 7fc01b200640 1 -- 172.21.15.19:0/3152421694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c0b12e0 con 0x7fc0000039b0 2024-03-28T13:14:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.318+0000 7fc01b200640 1 -- 172.21.15.19:0/3152421694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01c075940 con 0x7fc0000039b0 2024-03-28T13:14:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.319+0000 7fc012000640 1 -- 172.21.15.19:0/3152421694 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc00c045020 con 0x7fc0000039b0 2024-03-28T13:14:51.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.320+0000 7fc012000640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbff806c530 0x7fbff806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:51.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.320+0000 7fc012000640 1 -- 172.21.15.19:0/3152421694 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7fc00c0ad1c0 con 0x7fc0000039b0 2024-03-28T13:14:51.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.321+0000 7fc013e00640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbff806c530 0x7fbff806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:51.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.322+0000 7fc013e00640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbff806c530 0x7fbff806e9f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0b0510 tx=0x7fc004009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:51.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.329+0000 7fc012000640 1 -- 172.21.15.19:0/3152421694 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc00c04c020 con 0x7fc0000039b0 2024-03-28T13:14:51.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.594+0000 7fc01b200640 1 -- 172.21.15.19:0/3152421694 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc01c15cbd0 con 0x7fc0000039b0 2024-03-28T13:14:51.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.595+0000 7fc012000640 1 -- 172.21.15.19:0/3152421694 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v51) ==== 53+0+73 (secure 0 0 0) 0x7fc00c07b930 con 0x7fc0000039b0 2024-03-28T13:14:51.595 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:14:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.598+0000 7fc01b200640 1 -- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbff806c530 msgr2=0x7fbff806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.598+0000 7fc01b200640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbff806c530 0x7fbff806e9f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0b0510 tx=0x7fc004009040 comp rx=0 tx=0).stop 2024-03-28T13:14:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.598+0000 7fc01b200640 1 -- 172.21.15.19:0/3152421694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0000039b0 msgr2=0x7fc01c0ae5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.598+0000 7fc01b200640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0000039b0 0x7fc01c0ae5b0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009be0 tx=0x7fc00c004730 comp rx=0 tx=0).stop 2024-03-28T13:14:51.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.598+0000 7fc01b200640 1 -- 172.21.15.19:0/3152421694 shutdown_connections 2024-03-28T13:14:51.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.598+0000 7fc01b200640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbff806c530 0x7fbff806e9f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.598+0000 7fc01b200640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc01c143990 0x7fc01c0af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.598+0000 7fc01b200640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc01c1434f0 0x7fc01c0aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.598+0000 7fc01b200640 1 --2- 172.21.15.19:0/3152421694 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0000039b0 0x7fc01c0ae5b0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:51.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.598+0000 7fc01b200640 1 -- 172.21.15.19:0/3152421694 >> 172.21.15.19:0/3152421694 conn(0x7fc01c067460 msgr2=0x7fc01c15fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:51.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.598+0000 7fc01b200640 1 -- 172.21.15.19:0/3152421694 shutdown_connections 2024-03-28T13:14:51.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51.599+0000 7fc01b200640 1 -- 172.21.15.19:0/3152421694 wait complete. 2024-03-28T13:14:51.619 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-28T13:14:51.619 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:14:51.619 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:14:51.619 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:14:51.619 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:14:51.619 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-28T13:14:51.619 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-28T13:14:51.619 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:14:51.619 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:14:51.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:51.619 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:14:51.670 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:14:51.670 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:14:51.671 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:14:51.671 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:14:51.671 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:14:51.671 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:14:51.671 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:14:51.671 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:14:51.671 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:14:51.671 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:14:51.671 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:14:51.671 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:14:51.671 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:14:51.671 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:14:51.671 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:14:51.671 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:14:51.671 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42263sec preferred_lft 42263sec 2024-03-28T13:14:51.672 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:14:51.672 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:14:51.672 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:14:51.672 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:14:51.672 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:14:51.672 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:14:51.672 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:14:51.672 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:14:51.672 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:14:51.672 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:14:51.672 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:14:51.672 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:14:51.672 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:14:51.672 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:14:51.672 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:14:51.673 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:14:51.673 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether e6:17:c5:8e:2b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:14:51.673 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::e417:c5ff:fe8e:2bcf/64 scope link 2024-03-28T13:14:51.673 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:14:51.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:51.674 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:14:51.755 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:14:51.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:51.759 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:14:51.811 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:14:51.811 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:14:51.811 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:14:51.811 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:14:51.811 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:14:51.812 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:14:51.812 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:14:51.812 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:14:51.812 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:14:51.812 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:14:51.812 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:14:51.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:14:52.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:14:52.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:52.061 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:14:52.116 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:14:52.116 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:14:52.117 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:14:52.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:52.117 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:14:52.228 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:14:52.228 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:14:52.279 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:14:52.279 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:14:52.279 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:14:52.326 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:14:52.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:52.326 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:14:52.464 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:14:52.464 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:14:52.464 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:14:52.486 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:14:52.487 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:14:52.487 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:14:52.487 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:14:52.487 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:14:52.516 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:14:52.516 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:14:52.516 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:14:52.516 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:14:52.517 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:14:52.517 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:14:52.517 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:14:52.517 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:14:52.520 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:14:52.520 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:14:52.520 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:14:52.520 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:14:52.520 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:14:52.524 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:14:52.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:52.525 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:14:52.602 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:14:52.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:14:52.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:14:52.717 INFO:teuthology.orchestra.run.smithi019.stdout:client5447 2024-03-28T13:14:52.717 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:14:52.718 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:14:52.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:52.766 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-03-28T13:14:52.820 INFO:teuthology.orchestra.run.smithi019.stdout:file_undamaged 2024-03-28T13:14:52.821 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:14:52.821 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:14:52.822 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:14:52.822 DEBUG:teuthology.orchestra.run.smithi019:> import json 2024-03-28T13:14:52.822 DEBUG:teuthology.orchestra.run.smithi019:> import sys 2024-03-28T13:14:52.822 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:14:52.822 DEBUG:teuthology.orchestra.run.smithi019:> try: 2024-03-28T13:14:52.822 DEBUG:teuthology.orchestra.run.smithi019:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-03-28T13:14:52.822 DEBUG:teuthology.orchestra.run.smithi019:> except OSError as e: 2024-03-28T13:14:52.822 DEBUG:teuthology.orchestra.run.smithi019:> sys.exit(e.errno) 2024-03-28T13:14:52.822 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:14:52.822 DEBUG:teuthology.orchestra.run.smithi019:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-03-28T13:14:52.822 DEBUG:teuthology.orchestra.run.smithi019:> print(json.dumps( 2024-03-28T13:14:52.822 DEBUG:teuthology.orchestra.run.smithi019:> dict([(a, getattr(s, a)) for a in attrs]), 2024-03-28T13:14:52.822 DEBUG:teuthology.orchestra.run.smithi019:> indent=2)) 2024-03-28T13:14:52.822 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:14:52.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:52 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-03-28T13:14:52.979 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-03-28T13:14:52.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:52.980 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-03-28T13:14:53.075 INFO:teuthology.orchestra.run.smithi019.stdout:2 2024-03-28T13:14:53.075 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:14:53.075 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:14:53.123 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:14:53.123 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:14:53.223 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:14:53.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.453+0000 7f1a2ca00640 1 Processor -- start 2024-03-28T13:14:53.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.453+0000 7f1a2ca00640 1 -- start start 2024-03-28T13:14:53.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.454+0000 7f1a2ca00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280cf700 0x7f1a28168fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.454+0000 7f1a2ca00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a2815e880 con 0x7f1a2815e4a0 2024-03-28T13:14:53.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.455+0000 7f1a2ca00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a280d28d0 con 0x7f1a280d2310 2024-03-28T13:14:53.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.455+0000 7f1a2ca00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1a280d34e0 con 0x7f1a280cf700 2024-03-28T13:14:53.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.455+0000 7f1a26800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1a2815e4a0 0x7f1a280b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51550/0 (socket says 172.21.15.19:51550) 2024-03-28T13:14:53.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.455+0000 7f1a26800640 1 -- 172.21.15.19:0/3934184561 learned_addr learned my addr 172.21.15.19:0/3934184561 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:53.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.455+0000 7f1a25e00640 1 -- 172.21.15.19:0/3934184561 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3073858217 0 0) 0x7f1a2815e880 con 0x7f1a2815e4a0 2024-03-28T13:14:53.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.455+0000 7f1a25e00640 1 -- 172.21.15.19:0/3934184561 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1a0c003520 con 0x7f1a2815e4a0 2024-03-28T13:14:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.455+0000 7f1a27200640 1 --2- 172.21.15.19:0/3934184561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280cf700 0x7f1a28168fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.456+0000 7f1a25e00640 1 -- 172.21.15.19:0/3934184561 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1274125595 0 0) 0x7f1a0c003520 con 0x7f1a2815e4a0 2024-03-28T13:14:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.456+0000 7f1a25e00640 1 -- 172.21.15.19:0/3934184561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280cf700 msgr2=0x7f1a28168fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.456+0000 7f1a25e00640 1 --2- 172.21.15.19:0/3934184561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280cf700 0x7f1a28168fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.456+0000 7f1a25e00640 1 -- 172.21.15.19:0/3934184561 >> v1:172.21.15.188:6789/0 conn(0x7f1a280d2310 legacy=0x7f1a280ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.456+0000 7f1a25e00640 1 -- 172.21.15.19:0/3934184561 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a280d2bb0 con 0x7f1a2815e4a0 2024-03-28T13:14:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.456+0000 7f1a25e00640 1 -- 172.21.15.19:0/3934184561 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f1a18001670 con 0x7f1a2815e4a0 2024-03-28T13:14:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.456+0000 7f1a25e00640 1 -- 172.21.15.19:0/3934184561 >> v1:172.21.15.19:6789/0 conn(0x7f1a2815e4a0 legacy=0x7f1a280b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.456+0000 7f1a25e00640 1 --2- 172.21.15.19:0/3934184561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a0c0039e0 0x7f1a0c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.456+0000 7f1a25e00640 1 --2- 172.21.15.19:0/3934184561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280cf700 0x7f1a0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.456+0000 7f1a25e00640 1 --2- 172.21.15.19:0/3934184561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a0c004d40 0x7f1a0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.456+0000 7f1a25e00640 1 -- 172.21.15.19:0/3934184561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a280d34e0 con 0x7f1a0c0039e0 2024-03-28T13:14:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.456+0000 7f1a25e00640 1 -- 172.21.15.19:0/3934184561 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1a0c00bb80 con 0x7f1a0c004d40 2024-03-28T13:14:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.456+0000 7f1a25e00640 1 -- 172.21.15.19:0/3934184561 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1a0c00bd80 con 0x7f1a280cf700 2024-03-28T13:14:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.456+0000 7f1a26800640 1 --2- 172.21.15.19:0/3934184561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a0c0039e0 0x7f1a0c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.456+0000 7f1a27200640 1 --2- 172.21.15.19:0/3934184561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a0c004d40 0x7f1a0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.457+0000 7f1a1fe00640 1 --2- 172.21.15.19:0/3934184561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280cf700 0x7f1a0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.457+0000 7f1a26800640 1 -- 172.21.15.19:0/3934184561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280cf700 msgr2=0x7f1a0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.457+0000 7f1a26800640 1 --2- 172.21.15.19:0/3934184561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280cf700 0x7f1a0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.457+0000 7f1a26800640 1 -- 172.21.15.19:0/3934184561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a0c004d40 msgr2=0x7f1a0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.457+0000 7f1a26800640 1 --2- 172.21.15.19:0/3934184561 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a0c004d40 0x7f1a0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.457+0000 7f1a26800640 1 -- 172.21.15.19:0/3934184561 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0c003520 con 0x7f1a0c0039e0 2024-03-28T13:14:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.457+0000 7f1a1fe00640 1 --2- 172.21.15.19:0/3934184561 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a280cf700 0x7f1a0c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.458+0000 7f1a26800640 1 --2- 172.21.15.19:0/3934184561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a0c0039e0 0x7f1a0c003e00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800c0e0 tx=0x7f1a0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28eac2fc75a43fc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.459+0000 7f1a25e00640 1 -- 172.21.15.19:0/3934184561 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18007c20 con 0x7f1a0c0039e0 2024-03-28T13:14:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.459+0000 7f1a25e00640 1 -- 172.21.15.19:0/3934184561 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a18030070 con 0x7f1a0c0039e0 2024-03-28T13:14:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.459+0000 7f1a25e00640 1 -- 172.21.15.19:0/3934184561 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18005d30 con 0x7f1a0c0039e0 2024-03-28T13:14:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.459+0000 7f1a2ca00640 1 -- 172.21.15.19:0/3934184561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a0c0039e0 msgr2=0x7f1a0c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.459+0000 7f1a2ca00640 1 --2- 172.21.15.19:0/3934184561 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a0c0039e0 0x7f1a0c003e00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1a1800c0e0 tx=0x7f1a0c005830 comp rx=0 tx=0).stop 2024-03-28T13:14:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.459+0000 7f1a27200640 1 -- 172.21.15.19:0/3934184561 reap_dead start 2024-03-28T13:14:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.459+0000 7f1a2ca00640 1 -- 172.21.15.19:0/3934184561 shutdown_connections 2024-03-28T13:14:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.459+0000 7f1a2ca00640 1 -- 172.21.15.19:0/3934184561 >> 172.21.15.19:0/3934184561 conn(0x7f1a280676d0 msgr2=0x7f1a28067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.459+0000 7f1a2ca00640 1 -- 172.21.15.19:0/3934184561 shutdown_connections 2024-03-28T13:14:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.460+0000 7f1a2ca00640 1 -- 172.21.15.19:0/3934184561 wait complete. 2024-03-28T13:14:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.461+0000 7f1a2ca00640 1 Processor -- start 2024-03-28T13:14:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.462+0000 7f1a2ca00640 1 -- start start 2024-03-28T13:14:53.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.462+0000 7f1a2ca00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a0c0039e0 0x7f1a28156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.463+0000 7f1a2ca00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a281f7420 0x7f1a280c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.463+0000 7f1a2ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a280c9670 0x7f1a2815c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.463+0000 7f1a2ca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a280d3970 con 0x7f1a280c9670 2024-03-28T13:14:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.463+0000 7f1a2ca00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1a280d3170 con 0x7f1a281f7420 2024-03-28T13:14:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.463+0000 7f1a2ca00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1a280b83f0 con 0x7f1a0c0039e0 2024-03-28T13:14:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.463+0000 7f1a27200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a280c9670 0x7f1a2815c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.463+0000 7f1a27200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a280c9670 0x7f1a2815c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47322/0 (socket says 172.21.15.19:47322) 2024-03-28T13:14:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.463+0000 7f1a27200640 1 -- 172.21.15.19:0/1598345841 learned_addr learned my addr 172.21.15.19:0/1598345841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.463+0000 7f1a26800640 1 --2- 172.21.15.19:0/1598345841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a0c0039e0 0x7f1a28156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.463+0000 7f1a1fe00640 1 --2- 172.21.15.19:0/1598345841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a281f7420 0x7f1a280c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.463+0000 7f1a27200640 1 -- 172.21.15.19:0/1598345841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a0c0039e0 msgr2=0x7f1a28156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.463+0000 7f1a27200640 1 --2- 172.21.15.19:0/1598345841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a0c0039e0 0x7f1a28156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.463+0000 7f1a27200640 1 -- 172.21.15.19:0/1598345841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a281f7420 msgr2=0x7f1a280c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.463+0000 7f1a27200640 1 --2- 172.21.15.19:0/1598345841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a281f7420 0x7f1a280c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.463+0000 7f1a27200640 1 -- 172.21.15.19:0/1598345841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a280c8160 con 0x7f1a280c9670 2024-03-28T13:14:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.464+0000 7f1a1fe00640 1 --2- 172.21.15.19:0/1598345841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a281f7420 0x7f1a280c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.464+0000 7f1a27200640 1 --2- 172.21.15.19:0/1598345841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a280c9670 0x7f1a2815c580 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f1a280d20d0 tx=0x7f1a200115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.464+0000 7f1a1ea00640 1 -- 172.21.15.19:0/1598345841 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20011e40 con 0x7f1a280c9670 2024-03-28T13:14:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.464+0000 7f1a1ea00640 1 -- 172.21.15.19:0/1598345841 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2001b070 con 0x7f1a280c9670 2024-03-28T13:14:53.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.464+0000 7f1a2ca00640 1 -- 172.21.15.19:0/1598345841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a280c8440 con 0x7f1a280c9670 2024-03-28T13:14:53.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.464+0000 7f1a1ea00640 1 -- 172.21.15.19:0/1598345841 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a20003990 con 0x7f1a280c9670 2024-03-28T13:14:53.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.464+0000 7f1a2ca00640 1 -- 172.21.15.19:0/1598345841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a280c8a00 con 0x7f1a280c9670 2024-03-28T13:14:53.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.465+0000 7f1a2ca00640 1 -- 172.21.15.19:0/1598345841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a28075940 con 0x7f1a280c9670 2024-03-28T13:14:53.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.467+0000 7f1a1ea00640 1 -- 172.21.15.19:0/1598345841 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1a20005d10 con 0x7f1a280c9670 2024-03-28T13:14:53.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.467+0000 7f1a1ea00640 1 --2- 172.21.15.19:0/1598345841 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a0406c560 0x7f1a0406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.468+0000 7f1a1ea00640 1 -- 172.21.15.19:0/1598345841 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7f1a2008bc40 con 0x7f1a280c9670 2024-03-28T13:14:53.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.468+0000 7f1a26800640 1 --2- 172.21.15.19:0/1598345841 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a0406c560 0x7f1a0406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.469+0000 7f1a26800640 1 --2- 172.21.15.19:0/1598345841 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a0406c560 0x7f1a0406ea20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1a18034000 tx=0x7f1a18007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:53.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.473+0000 7f1a1ea00640 1 -- 172.21.15.19:0/1598345841 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1a20025020 con 0x7f1a280c9670 2024-03-28T13:14:53.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.676+0000 7f1a2ca00640 1 -- 172.21.15.19:0/1598345841 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a28157460 con 0x7f1a280c9670 2024-03-28T13:14:53.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.679+0000 7f1a1ea00640 1 -- 172.21.15.19:0/1598345841 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+16737 (secure 0 0 0) 0x7f1a20014350 con 0x7f1a280c9670 2024-03-28T13:14:53.680 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:14:53.680 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":78,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:14:45.007937+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:14:29.424080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-28T13:14:30.494667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/1189161682":"2024-03-29T13:14:45.007899+0000","172.21.15.19:6832/1189161682":"2024-03-29T13:14:45.007899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:14:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.682+0000 7f1a2ca00640 1 -- 172.21.15.19:0/1598345841 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a0406c560 msgr2=0x7f1a0406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.682+0000 7f1a2ca00640 1 --2- 172.21.15.19:0/1598345841 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a0406c560 0x7f1a0406ea20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1a18034000 tx=0x7f1a18007000 comp rx=0 tx=0).stop 2024-03-28T13:14:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.682+0000 7f1a2ca00640 1 -- 172.21.15.19:0/1598345841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a280c9670 msgr2=0x7f1a2815c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.682+0000 7f1a2ca00640 1 --2- 172.21.15.19:0/1598345841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a280c9670 0x7f1a2815c580 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f1a280d20d0 tx=0x7f1a200115a0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.682+0000 7f1a2ca00640 1 -- 172.21.15.19:0/1598345841 shutdown_connections 2024-03-28T13:14:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.682+0000 7f1a2ca00640 1 --2- 172.21.15.19:0/1598345841 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1a0406c560 0x7f1a0406ea20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.682+0000 7f1a2ca00640 1 --2- 172.21.15.19:0/1598345841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a280c9670 0x7f1a2815c580 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.682+0000 7f1a2ca00640 1 --2- 172.21.15.19:0/1598345841 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a281f7420 0x7f1a280c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.682+0000 7f1a2ca00640 1 --2- 172.21.15.19:0/1598345841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a0c0039e0 0x7f1a28156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.682+0000 7f1a2ca00640 1 -- 172.21.15.19:0/1598345841 >> 172.21.15.19:0/1598345841 conn(0x7f1a280676d0 msgr2=0x7f1a2815b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:53.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.682+0000 7f1a2ca00640 1 -- 172.21.15.19:0/1598345841 shutdown_connections 2024-03-28T13:14:53.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.682+0000 7f1a2ca00640 1 -- 172.21.15.19:0/1598345841 wait complete. 2024-03-28T13:14:53.710 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:14:29.424080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:14:30.494667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:14:53.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:14:53.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.931+0000 7fa7a3800640 1 Processor -- start 2024-03-28T13:14:53.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.932+0000 7fa7a3800640 1 -- start start 2024-03-28T13:14:53.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.933+0000 7fa7a3800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa79c0d1bf0 0x7fa79c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.933+0000 7fa7a3800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa79c0705f0 0x7fa79c06e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.933+0000 7fa7a3800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa79c15edc0 con 0x7fa79c070170 2024-03-28T13:14:53.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.934+0000 7fa7a3800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa79c0d1fd0 con 0x7fa79c0d1bf0 2024-03-28T13:14:53.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.934+0000 7fa7a3800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa79c15eb80 con 0x7fa79c0705f0 2024-03-28T13:14:53.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.934+0000 7fa79be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa79c070170 0x7fa79c06b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51560/0 (socket says 172.21.15.19:51560) 2024-03-28T13:14:53.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.934+0000 7fa79be00640 1 -- 172.21.15.19:0/3924006353 learned_addr learned my addr 172.21.15.19:0/3924006353 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:53.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.934+0000 7fa7a1800640 1 --2- 172.21.15.19:0/3924006353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa79c0705f0 0x7fa79c06e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.934+0000 7fa7a0e00640 1 --2- 172.21.15.19:0/3924006353 >> v2:172.21.15.188:3300/0 conn(0x7fa79c0d1bf0 0x7fa79c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.934+0000 7fa79b400640 1 -- 172.21.15.19:0/3924006353 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3426638995 0 0) 0x7fa79c15edc0 con 0x7fa79c070170 2024-03-28T13:14:53.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.934+0000 7fa79b400640 1 -- 172.21.15.19:0/3924006353 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa784003520 con 0x7fa79c070170 2024-03-28T13:14:53.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.935+0000 7fa79b400640 1 -- 172.21.15.19:0/3924006353 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 910038939 0 0) 0x7fa784003520 con 0x7fa79c070170 2024-03-28T13:14:53.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.935+0000 7fa79b400640 1 -- 172.21.15.19:0/3924006353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa79c0705f0 msgr2=0x7fa79c06e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:53.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.935+0000 7fa79b400640 1 --2- 172.21.15.19:0/3924006353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa79c0705f0 0x7fa79c06e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.935+0000 7fa79b400640 1 -- 172.21.15.19:0/3924006353 >> v2:172.21.15.188:3300/0 conn(0x7fa79c0d1bf0 msgr2=0x7fa79c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:53.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.935+0000 7fa79b400640 1 --2- 172.21.15.19:0/3924006353 >> v2:172.21.15.188:3300/0 conn(0x7fa79c0d1bf0 0x7fa79c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.935+0000 7fa79b400640 1 -- 172.21.15.19:0/3924006353 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c0b5e60 con 0x7fa79c070170 2024-03-28T13:14:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.935+0000 7fa79b400640 1 -- 172.21.15.19:0/3924006353 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fa788001670 con 0x7fa79c070170 2024-03-28T13:14:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.935+0000 7fa79b400640 1 -- 172.21.15.19:0/3924006353 >> v1:172.21.15.19:6789/0 conn(0x7fa79c070170 legacy=0x7fa79c06b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.935+0000 7fa79b400640 1 --2- 172.21.15.19:0/3924006353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7840039b0 0x7fa784003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.935+0000 7fa79b400640 1 --2- 172.21.15.19:0/3924006353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa784004800 0x7fa784004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.935+0000 7fa79b400640 1 --2- 172.21.15.19:0/3924006353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa79c0705f0 0x7fa7840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.935+0000 7fa79b400640 1 -- 172.21.15.19:0/3924006353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c15eb80 con 0x7fa7840039b0 2024-03-28T13:14:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.935+0000 7fa79b400640 1 -- 172.21.15.19:0/3924006353 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa78400b9c0 con 0x7fa784004800 2024-03-28T13:14:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.935+0000 7fa79b400640 1 -- 172.21.15.19:0/3924006353 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa78400bb40 con 0x7fa79c0705f0 2024-03-28T13:14:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.935+0000 7fa7a0e00640 1 --2- 172.21.15.19:0/3924006353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7840039b0 0x7fa784003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.936+0000 7fa79be00640 1 --2- 172.21.15.19:0/3924006353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa784004800 0x7fa784004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.936+0000 7fa7a1800640 1 --2- 172.21.15.19:0/3924006353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa79c0705f0 0x7fa7840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.936+0000 7fa7a0e00640 1 -- 172.21.15.19:0/3924006353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa79c0705f0 msgr2=0x7fa7840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.936+0000 7fa7a0e00640 1 --2- 172.21.15.19:0/3924006353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa79c0705f0 0x7fa7840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.936+0000 7fa7a0e00640 1 -- 172.21.15.19:0/3924006353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa784004800 msgr2=0x7fa784004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:53.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.936+0000 7fa7a0e00640 1 --2- 172.21.15.19:0/3924006353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa784004800 0x7fa784004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.936+0000 7fa7a0e00640 1 -- 172.21.15.19:0/3924006353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa784003520 con 0x7fa7840039b0 2024-03-28T13:14:53.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.936+0000 7fa7a1800640 1 --2- 172.21.15.19:0/3924006353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa79c0705f0 0x7fa7840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:53.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.937+0000 7fa7a0e00640 1 --2- 172.21.15.19:0/3924006353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7840039b0 0x7fa784003dd0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa790009ac0 tx=0x7fa784005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1315a7c59944e60 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:53.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.937+0000 7fa79b400640 1 -- 172.21.15.19:0/3924006353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790039070 con 0x7fa7840039b0 2024-03-28T13:14:53.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.937+0000 7fa79b400640 1 -- 172.21.15.19:0/3924006353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa79002c710 con 0x7fa7840039b0 2024-03-28T13:14:53.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.937+0000 7fa79b400640 1 -- 172.21.15.19:0/3924006353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790009690 con 0x7fa7840039b0 2024-03-28T13:14:53.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.938+0000 7fa7a3800640 1 -- 172.21.15.19:0/3924006353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7840039b0 msgr2=0x7fa784003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:53.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.938+0000 7fa7a3800640 1 --2- 172.21.15.19:0/3924006353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7840039b0 0x7fa784003dd0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa790009ac0 tx=0x7fa784005830 comp rx=0 tx=0).stop 2024-03-28T13:14:53.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.938+0000 7fa7a1800640 1 -- 172.21.15.19:0/3924006353 reap_dead start 2024-03-28T13:14:53.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.938+0000 7fa7a3800640 1 -- 172.21.15.19:0/3924006353 shutdown_connections 2024-03-28T13:14:53.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.938+0000 7fa7a3800640 1 -- 172.21.15.19:0/3924006353 >> 172.21.15.19:0/3924006353 conn(0x7fa79c0b6560 msgr2=0x7fa79c0b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:53.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.938+0000 7fa7a3800640 1 -- 172.21.15.19:0/3924006353 shutdown_connections 2024-03-28T13:14:53.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.939+0000 7fa7a3800640 1 -- 172.21.15.19:0/3924006353 wait complete. 2024-03-28T13:14:53.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.940+0000 7fa7a3800640 1 Processor -- start 2024-03-28T13:14:53.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.940+0000 7fa7a3800640 1 -- start start 2024-03-28T13:14:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.941+0000 7fa7a3800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa79c0b1f70 0x7fa79c154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.941+0000 7fa7a3800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa79c0b2350 0x7fa79c1491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.941+0000 7fa7a3800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa79c1f51c0 0x7fa79c15ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.941+0000 7fa7a3800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c15fae0 con 0x7fa79c0b2350 2024-03-28T13:14:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.941+0000 7fa7a3800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c15f080 con 0x7fa79c0b1f70 2024-03-28T13:14:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.941+0000 7fa7a3800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa79c0b5b10 con 0x7fa79c1f51c0 2024-03-28T13:14:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.942+0000 7fa79be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa79c0b2350 0x7fa79c1491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.942+0000 7fa79be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa79c0b2350 0x7fa79c1491b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47326/0 (socket says 172.21.15.19:47326) 2024-03-28T13:14:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.942+0000 7fa79be00640 1 -- 172.21.15.19:0/2457885986 learned_addr learned my addr 172.21.15.19:0/2457885986 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.942+0000 7fa7a0e00640 1 --2- 172.21.15.19:0/2457885986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa79c0b1f70 0x7fa79c154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.942+0000 7fa7a1800640 1 --2- 172.21.15.19:0/2457885986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa79c1f51c0 0x7fa79c15ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.942+0000 7fa79be00640 1 -- 172.21.15.19:0/2457885986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa79c1f51c0 msgr2=0x7fa79c15ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.942+0000 7fa79be00640 1 --2- 172.21.15.19:0/2457885986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa79c1f51c0 0x7fa79c15ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.942+0000 7fa79be00640 1 -- 172.21.15.19:0/2457885986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa79c0b1f70 msgr2=0x7fa79c154400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:53.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.942+0000 7fa79be00640 1 --2- 172.21.15.19:0/2457885986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa79c0b1f70 0x7fa79c154400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.942+0000 7fa79be00640 1 -- 172.21.15.19:0/2457885986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c154a90 con 0x7fa79c0b2350 2024-03-28T13:14:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.942+0000 7fa79be00640 1 --2- 172.21.15.19:0/2457885986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa79c0b2350 0x7fa79c1491b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa78800e700 tx=0x7fa78800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.943+0000 7fa798c00640 1 -- 172.21.15.19:0/2457885986 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78800c640 con 0x7fa79c0b2350 2024-03-28T13:14:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.943+0000 7fa798c00640 1 -- 172.21.15.19:0/2457885986 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa788018070 con 0x7fa79c0b2350 2024-03-28T13:14:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.943+0000 7fa798c00640 1 -- 172.21.15.19:0/2457885986 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78800ca40 con 0x7fa79c0b2350 2024-03-28T13:14:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.943+0000 7fa7a3800640 1 -- 172.21.15.19:0/2457885986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79c154d70 con 0x7fa79c0b2350 2024-03-28T13:14:53.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.943+0000 7fa7a3800640 1 -- 172.21.15.19:0/2457885986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa79c1f4ca0 con 0x7fa79c0b2350 2024-03-28T13:14:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.945+0000 7fa7a3800640 1 -- 172.21.15.19:0/2457885986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa79c1f4ea0 con 0x7fa79c0b2350 2024-03-28T13:14:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.945+0000 7fa798c00640 1 -- 172.21.15.19:0/2457885986 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa78800cbe0 con 0x7fa79c0b2350 2024-03-28T13:14:53.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.946+0000 7fa798c00640 1 --2- 172.21.15.19:0/2457885986 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa77c06c530 0x7fa77c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:53.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.946+0000 7fa798c00640 1 -- 172.21.15.19:0/2457885986 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7fa78808b4a0 con 0x7fa79c0b2350 2024-03-28T13:14:53.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.946+0000 7fa7a0e00640 1 --2- 172.21.15.19:0/2457885986 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa77c06c530 0x7fa77c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:53.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.948+0000 7fa7a0e00640 1 --2- 172.21.15.19:0/2457885986 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa77c06c530 0x7fa77c06e9f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa790036000 tx=0x7fa790030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:53.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:53.951+0000 7fa798c00640 1 -- 172.21.15.19:0/2457885986 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa79c1f4ea0 con 0x7fa79c0b2350 2024-03-28T13:14:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.217+0000 7fa7a3800640 1 -- 172.21.15.19:0/2457885986 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa79c0685c0 con 0x7fa79c0b2350 2024-03-28T13:14:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.218+0000 7fa798c00640 1 -- 172.21.15.19:0/2457885986 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7fa788059a60 con 0x7fa79c0b2350 2024-03-28T13:14:54.218 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:14:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.220+0000 7fa7a3800640 1 -- 172.21.15.19:0/2457885986 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa77c06c530 msgr2=0x7fa77c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.220+0000 7fa7a3800640 1 --2- 172.21.15.19:0/2457885986 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa77c06c530 0x7fa77c06e9f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa790036000 tx=0x7fa790030000 comp rx=0 tx=0).stop 2024-03-28T13:14:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.220+0000 7fa7a3800640 1 -- 172.21.15.19:0/2457885986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa79c0b2350 msgr2=0x7fa79c1491b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.220+0000 7fa7a3800640 1 --2- 172.21.15.19:0/2457885986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa79c0b2350 0x7fa79c1491b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fa78800e700 tx=0x7fa78800ebc0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.221+0000 7fa7a3800640 1 -- 172.21.15.19:0/2457885986 shutdown_connections 2024-03-28T13:14:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.221+0000 7fa7a3800640 1 --2- 172.21.15.19:0/2457885986 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa77c06c530 0x7fa77c06e9f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.221+0000 7fa7a3800640 1 --2- 172.21.15.19:0/2457885986 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa79c1f51c0 0x7fa79c15ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.221+0000 7fa7a3800640 1 --2- 172.21.15.19:0/2457885986 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa79c0b2350 0x7fa79c1491b0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.221+0000 7fa7a3800640 1 --2- 172.21.15.19:0/2457885986 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa79c0b1f70 0x7fa79c154400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.221+0000 7fa7a3800640 1 -- 172.21.15.19:0/2457885986 >> 172.21.15.19:0/2457885986 conn(0x7fa79c0b6560 msgr2=0x7fa79c0b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.221+0000 7fa7a3800640 1 -- 172.21.15.19:0/2457885986 shutdown_connections 2024-03-28T13:14:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.221+0000 7fa7a3800640 1 -- 172.21.15.19:0/2457885986 wait complete. 2024-03-28T13:14:54.239 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:14:54.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.447+0000 7fef82000640 1 Processor -- start 2024-03-28T13:14:54.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.447+0000 7fef82000640 1 -- start start 2024-03-28T13:14:54.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.449+0000 7fef82000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7c0d37b0 0x7fef7c0b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.449+0000 7fef82000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef7c0d2710 con 0x7fef7c0b67f0 2024-03-28T13:14:54.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.449+0000 7fef82000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef7c0d29f0 con 0x7fef7c15e4c0 2024-03-28T13:14:54.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.449+0000 7fef82000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fef7c15e8a0 con 0x7fef7c0d37b0 2024-03-28T13:14:54.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.449+0000 7fef81600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fef7c0b67f0 0x7fef7c0b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51572/0 (socket says 172.21.15.19:51572) 2024-03-28T13:14:54.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.449+0000 7fef81600640 1 -- 172.21.15.19:0/4062367073 learned_addr learned my addr 172.21.15.19:0/4062367073 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:54.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.450+0000 7fef7be00640 1 --2- 172.21.15.19:0/4062367073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7c0d37b0 0x7fef7c0b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:54.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.450+0000 7fef7b400640 1 -- 172.21.15.19:0/4062367073 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4170633663 0 0) 0x7fef7c0d2710 con 0x7fef7c0b67f0 2024-03-28T13:14:54.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.450+0000 7fef7b400640 1 -- 172.21.15.19:0/4062367073 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef64003520 con 0x7fef7c0b67f0 2024-03-28T13:14:54.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.450+0000 7fef7b400640 1 -- 172.21.15.19:0/4062367073 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 982895358 0 0) 0x7fef64003520 con 0x7fef7c0b67f0 2024-03-28T13:14:54.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.450+0000 7fef7b400640 1 -- 172.21.15.19:0/4062367073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7c0d37b0 msgr2=0x7fef7c0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:54.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.450+0000 7fef7b400640 1 --2- 172.21.15.19:0/4062367073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7c0d37b0 0x7fef7c0b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.450+0000 7fef7b400640 1 -- 172.21.15.19:0/4062367073 >> v1:172.21.15.188:6789/0 conn(0x7fef7c15e4c0 legacy=0x7fef7c0b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.450+0000 7fef7b400640 1 -- 172.21.15.19:0/4062367073 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c0d2cd0 con 0x7fef7c0b67f0 2024-03-28T13:14:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.450+0000 7fef7b400640 1 -- 172.21.15.19:0/4062367073 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fef740019f0 con 0x7fef7c0b67f0 2024-03-28T13:14:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.450+0000 7fef7b400640 1 -- 172.21.15.19:0/4062367073 >> v1:172.21.15.19:6789/0 conn(0x7fef7c0b67f0 legacy=0x7fef7c0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.451+0000 7fef7b400640 1 --2- 172.21.15.19:0/4062367073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef640039e0 0x7fef64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.451+0000 7fef7b400640 1 --2- 172.21.15.19:0/4062367073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7c0d37b0 0x7fef64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.451+0000 7fef7b400640 1 --2- 172.21.15.19:0/4062367073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef64004d40 0x7fef640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.451+0000 7fef7b400640 1 -- 172.21.15.19:0/4062367073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fef6400ba70 con 0x7fef640039e0 2024-03-28T13:14:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.451+0000 7fef7b400640 1 -- 172.21.15.19:0/4062367073 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fef6400bbf0 con 0x7fef64004d40 2024-03-28T13:14:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.451+0000 7fef7b400640 1 -- 172.21.15.19:0/4062367073 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fef6400bdf0 con 0x7fef7c0d37b0 2024-03-28T13:14:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.451+0000 7fef80c00640 1 --2- 172.21.15.19:0/4062367073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef640039e0 0x7fef64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.451+0000 7fef80c00640 1 -- 172.21.15.19:0/4062367073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7c0d37b0 msgr2=0x7fef64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.451+0000 7fef80c00640 1 --2- 172.21.15.19:0/4062367073 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7c0d37b0 0x7fef64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.451+0000 7fef80c00640 1 -- 172.21.15.19:0/4062367073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef64004d40 msgr2=0x7fef640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.451+0000 7fef80c00640 1 --2- 172.21.15.19:0/4062367073 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef64004d40 0x7fef640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.451+0000 7fef80c00640 1 -- 172.21.15.19:0/4062367073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef64003520 con 0x7fef640039e0 2024-03-28T13:14:54.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.452+0000 7fef80c00640 1 --2- 172.21.15.19:0/4062367073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef640039e0 0x7fef64003e00 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fef70001720 tx=0x7fef64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d90384db8b92c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:54.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.453+0000 7fef7b400640 1 -- 172.21.15.19:0/4062367073 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70001370 con 0x7fef640039e0 2024-03-28T13:14:54.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.453+0000 7fef7b400640 1 -- 172.21.15.19:0/4062367073 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef700052d0 con 0x7fef640039e0 2024-03-28T13:14:54.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.453+0000 7fef7b400640 1 -- 172.21.15.19:0/4062367073 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70003830 con 0x7fef640039e0 2024-03-28T13:14:54.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.453+0000 7fef82000640 1 -- 172.21.15.19:0/4062367073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef640039e0 msgr2=0x7fef64003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:54.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.453+0000 7fef82000640 1 --2- 172.21.15.19:0/4062367073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef640039e0 0x7fef64003e00 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fef70001720 tx=0x7fef64005830 comp rx=0 tx=0).stop 2024-03-28T13:14:54.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.453+0000 7fef81600640 1 -- 172.21.15.19:0/4062367073 reap_dead start 2024-03-28T13:14:54.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.453+0000 7fef82000640 1 -- 172.21.15.19:0/4062367073 shutdown_connections 2024-03-28T13:14:54.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.453+0000 7fef82000640 1 -- 172.21.15.19:0/4062367073 >> 172.21.15.19:0/4062367073 conn(0x7fef7c0676a0 msgr2=0x7fef7c067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:54.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.453+0000 7fef82000640 1 -- 172.21.15.19:0/4062367073 shutdown_connections 2024-03-28T13:14:54.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.453+0000 7fef82000640 1 -- 172.21.15.19:0/4062367073 wait complete. 2024-03-28T13:14:54.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.455+0000 7fef82000640 1 Processor -- start 2024-03-28T13:14:54.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.455+0000 7fef82000640 1 -- start start 2024-03-28T13:14:54.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.456+0000 7fef82000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef640039e0 0x7fef7c1f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.456+0000 7fef82000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef7c161150 0x7fef7c1540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.456+0000 7fef82000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7c0ca790 0x7fef7c0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.456+0000 7fef82000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c0d3320 con 0x7fef7c161150 2024-03-28T13:14:54.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.456+0000 7fef82000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c0d4040 con 0x7fef640039e0 2024-03-28T13:14:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.456+0000 7fef82000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fef7c0d2350 con 0x7fef7c0ca790 2024-03-28T13:14:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.456+0000 7fef7be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef7c161150 0x7fef7c1540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.456+0000 7fef7be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef7c161150 0x7fef7c1540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47336/0 (socket says 172.21.15.19:47336) 2024-03-28T13:14:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.456+0000 7fef7be00640 1 -- 172.21.15.19:0/2102330467 learned_addr learned my addr 172.21.15.19:0/2102330467 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.457+0000 7fef7be00640 1 -- 172.21.15.19:0/2102330467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7c0ca790 msgr2=0x7fef7c0c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.457+0000 7fef7be00640 1 --2- 172.21.15.19:0/2102330467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7c0ca790 0x7fef7c0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.457+0000 7fef7be00640 1 -- 172.21.15.19:0/2102330467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef640039e0 msgr2=0x7fef7c1f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.457+0000 7fef7be00640 1 --2- 172.21.15.19:0/2102330467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef640039e0 0x7fef7c1f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.457+0000 7fef7be00640 1 -- 172.21.15.19:0/2102330467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c1f4d30 con 0x7fef7c161150 2024-03-28T13:14:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.457+0000 7fef7be00640 1 --2- 172.21.15.19:0/2102330467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef7c161150 0x7fef7c1540f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fef6800e7c0 tx=0x7fef6800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.457+0000 7fef78c00640 1 -- 172.21.15.19:0/2102330467 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68017070 con 0x7fef7c161150 2024-03-28T13:14:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.457+0000 7fef78c00640 1 -- 172.21.15.19:0/2102330467 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef6800c940 con 0x7fef7c161150 2024-03-28T13:14:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.457+0000 7fef82000640 1 -- 172.21.15.19:0/2102330467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7c1616f0 con 0x7fef7c161150 2024-03-28T13:14:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.457+0000 7fef78c00640 1 -- 172.21.15.19:0/2102330467 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6800cd10 con 0x7fef7c161150 2024-03-28T13:14:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.457+0000 7fef82000640 1 -- 172.21.15.19:0/2102330467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7c0c9cb0 con 0x7fef7c161150 2024-03-28T13:14:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.458+0000 7fef82000640 1 -- 172.21.15.19:0/2102330467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef7c075940 con 0x7fef7c161150 2024-03-28T13:14:54.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.461+0000 7fef78c00640 1 -- 172.21.15.19:0/2102330467 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fef6801a070 con 0x7fef7c161150 2024-03-28T13:14:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.461+0000 7fef78c00640 1 --2- 172.21.15.19:0/2102330467 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fef5c06c530 0x7fef5c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.461+0000 7fef78c00640 1 -- 172.21.15.19:0/2102330467 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7fef6808b600 con 0x7fef7c161150 2024-03-28T13:14:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.462+0000 7fef80c00640 1 --2- 172.21.15.19:0/2102330467 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fef5c06c530 0x7fef5c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.463+0000 7fef80c00640 1 --2- 172.21.15.19:0/2102330467 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fef5c06c530 0x7fef5c06e9f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fef70033000 tx=0x7fef7002f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.466+0000 7fef78c00640 1 -- 172.21.15.19:0/2102330467 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fef7c1f4d30 con 0x7fef7c161150 2024-03-28T13:14:54.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.664+0000 7fef82000640 1 -- 172.21.15.19:0/2102330467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fef7c0b4e00 con 0x7fef7c161150 2024-03-28T13:14:54.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.665+0000 7fef78c00640 1 -- 172.21.15.19:0/2102330467 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) ==== 104+0+11 (secure 0 0 0) 0x7fef68059b20 con 0x7fef7c161150 2024-03-28T13:14:54.665 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:14:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.668+0000 7fef82000640 1 -- 172.21.15.19:0/2102330467 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fef5c06c530 msgr2=0x7fef5c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.668+0000 7fef82000640 1 --2- 172.21.15.19:0/2102330467 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fef5c06c530 0x7fef5c06e9f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fef70033000 tx=0x7fef7002f000 comp rx=0 tx=0).stop 2024-03-28T13:14:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.668+0000 7fef82000640 1 -- 172.21.15.19:0/2102330467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef7c161150 msgr2=0x7fef7c1540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.668+0000 7fef82000640 1 --2- 172.21.15.19:0/2102330467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef7c161150 0x7fef7c1540f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fef6800e7c0 tx=0x7fef6800ec80 comp rx=0 tx=0).stop 2024-03-28T13:14:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.668+0000 7fef82000640 1 -- 172.21.15.19:0/2102330467 shutdown_connections 2024-03-28T13:14:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.668+0000 7fef82000640 1 --2- 172.21.15.19:0/2102330467 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fef5c06c530 0x7fef5c06e9f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.668+0000 7fef82000640 1 --2- 172.21.15.19:0/2102330467 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7c0ca790 0x7fef7c0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.668+0000 7fef82000640 1 --2- 172.21.15.19:0/2102330467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef7c161150 0x7fef7c1540f0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.668+0000 7fef82000640 1 --2- 172.21.15.19:0/2102330467 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef640039e0 0x7fef7c1f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.668+0000 7fef82000640 1 -- 172.21.15.19:0/2102330467 >> 172.21.15.19:0/2102330467 conn(0x7fef7c0676a0 msgr2=0x7fef7c0b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:54.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.668+0000 7fef82000640 1 -- 172.21.15.19:0/2102330467 shutdown_connections 2024-03-28T13:14:54.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.668+0000 7fef82000640 1 -- 172.21.15.19:0/2102330467 wait complete. 2024-03-28T13:14:54.686 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:14:54.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.890+0000 7ff93d600640 1 Processor -- start 2024-03-28T13:14:54.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.890+0000 7ff93d600640 1 -- start start 2024-03-28T13:14:54.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.892+0000 7ff93d600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff93815e4c0 0x7ff9380b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.892+0000 7ff93d600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff9380d37b0 0x7ff9380b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.892+0000 7ff93d600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9380d2710 con 0x7ff9380b67f0 2024-03-28T13:14:54.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.892+0000 7ff93d600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff93815e8a0 con 0x7ff9380d37b0 2024-03-28T13:14:54.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.892+0000 7ff93d600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff9380d4040 con 0x7ff93815e4c0 2024-03-28T13:14:54.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.892+0000 7ff93cc00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff9380b67f0 0x7ff9380d4260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51586/0 (socket says 172.21.15.19:51586) 2024-03-28T13:14:54.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.892+0000 7ff93cc00640 1 -- 172.21.15.19:0/273787870 learned_addr learned my addr 172.21.15.19:0/273787870 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:54.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff92fe00640 1 --2- 172.21.15.19:0/273787870 >> v2:172.21.15.188:3300/0 conn(0x7ff9380d37b0 0x7ff9380b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:54.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937e00640 1 --2- 172.21.15.19:0/273787870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff93815e4c0 0x7ff9380b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:54.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 -- 172.21.15.19:0/273787870 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1573400138 0 0) 0x7ff9380d2710 con 0x7ff9380b67f0 2024-03-28T13:14:54.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 -- 172.21.15.19:0/273787870 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff91c003520 con 0x7ff9380b67f0 2024-03-28T13:14:54.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 -- 172.21.15.19:0/273787870 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3862357266 0 0) 0x7ff91c003520 con 0x7ff9380b67f0 2024-03-28T13:14:54.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 -- 172.21.15.19:0/273787870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff93815e4c0 msgr2=0x7ff9380b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:54.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 --2- 172.21.15.19:0/273787870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff93815e4c0 0x7ff9380b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 -- 172.21.15.19:0/273787870 >> v2:172.21.15.188:3300/0 conn(0x7ff9380d37b0 msgr2=0x7ff9380b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:54.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 --2- 172.21.15.19:0/273787870 >> v2:172.21.15.188:3300/0 conn(0x7ff9380d37b0 0x7ff9380b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 -- 172.21.15.19:0/273787870 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9380d29f0 con 0x7ff9380b67f0 2024-03-28T13:14:54.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 -- 172.21.15.19:0/273787870 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7ff9300019f0 con 0x7ff9380b67f0 2024-03-28T13:14:54.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 -- 172.21.15.19:0/273787870 >> v1:172.21.15.19:6789/0 conn(0x7ff9380b67f0 legacy=0x7ff9380d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:54.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 --2- 172.21.15.19:0/273787870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff91c0039b0 0x7ff91c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 --2- 172.21.15.19:0/273787870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff93815e4c0 0x7ff91c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 --2- 172.21.15.19:0/273787870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff91c004d40 0x7ff91c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 -- 172.21.15.19:0/273787870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9380d4040 con 0x7ff91c0039b0 2024-03-28T13:14:54.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 -- 172.21.15.19:0/273787870 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff91c00b9c0 con 0x7ff91c004d40 2024-03-28T13:14:54.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.893+0000 7ff937400640 1 -- 172.21.15.19:0/273787870 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff91c00bb40 con 0x7ff93815e4c0 2024-03-28T13:14:54.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.894+0000 7ff937e00640 1 --2- 172.21.15.19:0/273787870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff91c0039b0 0x7ff91c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:54.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.894+0000 7ff92fe00640 1 --2- 172.21.15.19:0/273787870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff93815e4c0 0x7ff91c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:54.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.894+0000 7ff93cc00640 1 --2- 172.21.15.19:0/273787870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff91c004d40 0x7ff91c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:54.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.894+0000 7ff937e00640 1 -- 172.21.15.19:0/273787870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff93815e4c0 msgr2=0x7ff91c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:54.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.894+0000 7ff937e00640 1 --2- 172.21.15.19:0/273787870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff93815e4c0 0x7ff91c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.894+0000 7ff937e00640 1 -- 172.21.15.19:0/273787870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff91c004d40 msgr2=0x7ff91c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:54.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.894+0000 7ff937e00640 1 --2- 172.21.15.19:0/273787870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff91c004d40 0x7ff91c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.894+0000 7ff937e00640 1 -- 172.21.15.19:0/273787870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c003520 con 0x7ff91c0039b0 2024-03-28T13:14:54.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.895+0000 7ff937e00640 1 --2- 172.21.15.19:0/273787870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff91c0039b0 0x7ff91c003dd0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff928009ac0 tx=0x7ff91c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cdff759d451e378 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:54.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.896+0000 7ff937400640 1 -- 172.21.15.19:0/273787870 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928039070 con 0x7ff91c0039b0 2024-03-28T13:14:54.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.896+0000 7ff937400640 1 -- 172.21.15.19:0/273787870 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff92802c6b0 con 0x7ff91c0039b0 2024-03-28T13:14:54.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.896+0000 7ff937400640 1 -- 172.21.15.19:0/273787870 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928009690 con 0x7ff91c0039b0 2024-03-28T13:14:54.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.896+0000 7ff93d600640 1 -- 172.21.15.19:0/273787870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff91c0039b0 msgr2=0x7ff91c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:54.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.896+0000 7ff93d600640 1 --2- 172.21.15.19:0/273787870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff91c0039b0 0x7ff91c003dd0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7ff928009ac0 tx=0x7ff91c005830 comp rx=0 tx=0).stop 2024-03-28T13:14:54.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.896+0000 7ff93cc00640 1 -- 172.21.15.19:0/273787870 reap_dead start 2024-03-28T13:14:54.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.896+0000 7ff93d600640 1 -- 172.21.15.19:0/273787870 shutdown_connections 2024-03-28T13:14:54.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.896+0000 7ff93d600640 1 -- 172.21.15.19:0/273787870 >> 172.21.15.19:0/273787870 conn(0x7ff9380676a0 msgr2=0x7ff938067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:54.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.896+0000 7ff93d600640 1 -- 172.21.15.19:0/273787870 shutdown_connections 2024-03-28T13:14:54.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.896+0000 7ff93d600640 1 -- 172.21.15.19:0/273787870 wait complete. 2024-03-28T13:14:54.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.898+0000 7ff93d600640 1 Processor -- start 2024-03-28T13:14:54.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.898+0000 7ff93d600640 1 -- start start 2024-03-28T13:14:54.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.899+0000 7ff93d600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff91c0039b0 0x7ff9381546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.899+0000 7ff93d600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff93815cb00 0x7ff93815cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.899+0000 7ff93d600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff938160930 0x7ff938160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.899+0000 7ff93d600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9380d2fb0 con 0x7ff93815cb00 2024-03-28T13:14:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.899+0000 7ff93d600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff9380d3320 con 0x7ff91c0039b0 2024-03-28T13:14:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.899+0000 7ff93d600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff9380d2350 con 0x7ff938160930 2024-03-28T13:14:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.899+0000 7ff92fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff93815cb00 0x7ff93815cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.899+0000 7ff92fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff93815cb00 0x7ff93815cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47354/0 (socket says 172.21.15.19:47354) 2024-03-28T13:14:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.899+0000 7ff92fe00640 1 -- 172.21.15.19:0/313462522 learned_addr learned my addr 172.21.15.19:0/313462522 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.900+0000 7ff93cc00640 1 --2- 172.21.15.19:0/313462522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff938160930 0x7ff938160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.900+0000 7ff937e00640 1 --2- 172.21.15.19:0/313462522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff91c0039b0 0x7ff9381546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.900+0000 7ff92fe00640 1 -- 172.21.15.19:0/313462522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff938160930 msgr2=0x7ff938160de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.900+0000 7ff92fe00640 1 --2- 172.21.15.19:0/313462522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff938160930 0x7ff938160de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.900+0000 7ff92fe00640 1 -- 172.21.15.19:0/313462522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff91c0039b0 msgr2=0x7ff9381546e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.900+0000 7ff92fe00640 1 --2- 172.21.15.19:0/313462522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff91c0039b0 0x7ff9381546e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:54.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.900+0000 7ff92fe00640 1 -- 172.21.15.19:0/313462522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9380cb520 con 0x7ff93815cb00 2024-03-28T13:14:54.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.900+0000 7ff937e00640 1 --2- 172.21.15.19:0/313462522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff91c0039b0 0x7ff9381546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:54.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.900+0000 7ff92fe00640 1 --2- 172.21.15.19:0/313462522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff93815cb00 0x7ff93815cf80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff9200116d0 tx=0x7ff920011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:54.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.901+0000 7ff934c00640 1 -- 172.21.15.19:0/313462522 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92000f960 con 0x7ff93815cb00 2024-03-28T13:14:54.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.901+0000 7ff934c00640 1 -- 172.21.15.19:0/313462522 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff92000fb00 con 0x7ff93815cb00 2024-03-28T13:14:54.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.901+0000 7ff93d600640 1 -- 172.21.15.19:0/313462522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9381616c0 con 0x7ff93815cb00 2024-03-28T13:14:54.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.901+0000 7ff934c00640 1 -- 172.21.15.19:0/313462522 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920015460 con 0x7ff93815cb00 2024-03-28T13:14:54.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.901+0000 7ff93d600640 1 -- 172.21.15.19:0/313462522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff938161320 con 0x7ff93815cb00 2024-03-28T13:14:54.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.902+0000 7ff93d600640 1 -- 172.21.15.19:0/313462522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9380d1e80 con 0x7ff93815cb00 2024-03-28T13:14:54.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.903+0000 7ff934c00640 1 -- 172.21.15.19:0/313462522 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff920015600 con 0x7ff93815cb00 2024-03-28T13:14:54.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.904+0000 7ff934c00640 1 --2- 172.21.15.19:0/313462522 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff91406c560 0x7ff91406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:54.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.904+0000 7ff934c00640 1 -- 172.21.15.19:0/313462522 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7ff92008bfd0 con 0x7ff93815cb00 2024-03-28T13:14:54.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.904+0000 7ff937e00640 1 --2- 172.21.15.19:0/313462522 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff91406c560 0x7ff91406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:54.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.906+0000 7ff937e00640 1 --2- 172.21.15.19:0/313462522 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff91406c560 0x7ff91406ea20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff928036000 tx=0x7ff928030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:54.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:54.908+0000 7ff934c00640 1 -- 172.21.15.19:0/313462522 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff9380d1e80 con 0x7ff93815cb00 2024-03-28T13:14:55.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.167+0000 7ff93d600640 1 -- 172.21.15.19:0/313462522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff938075940 con 0x7ff93815cb00 2024-03-28T13:14:55.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.168+0000 7ff934c00640 1 -- 172.21.15.19:0/313462522 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) ==== 100+0+11 (secure 0 0 0) 0x7ff920019030 con 0x7ff93815cb00 2024-03-28T13:14:55.168 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:14:55.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.170+0000 7ff93d600640 1 -- 172.21.15.19:0/313462522 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff91406c560 msgr2=0x7ff91406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:55.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.170+0000 7ff93d600640 1 --2- 172.21.15.19:0/313462522 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff91406c560 0x7ff91406ea20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff928036000 tx=0x7ff928030000 comp rx=0 tx=0).stop 2024-03-28T13:14:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.170+0000 7ff93d600640 1 -- 172.21.15.19:0/313462522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff93815cb00 msgr2=0x7ff93815cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.170+0000 7ff93d600640 1 --2- 172.21.15.19:0/313462522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff93815cb00 0x7ff93815cf80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff9200116d0 tx=0x7ff920011b90 comp rx=0 tx=0).stop 2024-03-28T13:14:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.171+0000 7ff93d600640 1 -- 172.21.15.19:0/313462522 shutdown_connections 2024-03-28T13:14:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.171+0000 7ff93d600640 1 --2- 172.21.15.19:0/313462522 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff91406c560 0x7ff91406ea20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.171+0000 7ff93d600640 1 --2- 172.21.15.19:0/313462522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff938160930 0x7ff938160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.171+0000 7ff93d600640 1 --2- 172.21.15.19:0/313462522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff93815cb00 0x7ff93815cf80 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:55.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.171+0000 7ff93d600640 1 --2- 172.21.15.19:0/313462522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff91c0039b0 0x7ff9381546e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.171+0000 7ff93d600640 1 -- 172.21.15.19:0/313462522 >> 172.21.15.19:0/313462522 conn(0x7ff9380676a0 msgr2=0x7ff9380d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.171+0000 7ff93d600640 1 -- 172.21.15.19:0/313462522 shutdown_connections 2024-03-28T13:14:55.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.171+0000 7ff93d600640 1 -- 172.21.15.19:0/313462522 wait complete. 2024-03-28T13:14:55.190 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:14:55.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.421+0000 7f2b9aa00640 1 Processor -- start 2024-03-28T13:14:55.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.421+0000 7f2b9aa00640 1 -- start start 2024-03-28T13:14:55.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.423+0000 7f2b9aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2b9415e4a0 0x7f2b940b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:55.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.423+0000 7f2b9aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b940cf700 0x7f2b940b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:55.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.423+0000 7f2b9aa00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b9415e880 con 0x7f2b940d2310 2024-03-28T13:14:55.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.423+0000 7f2b9aa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2b940d34e0 con 0x7f2b9415e4a0 2024-03-28T13:14:55.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.423+0000 7f2b9aa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2b940d28d0 con 0x7f2b940cf700 2024-03-28T13:14:55.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.424+0000 7f2b99600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2b9415e4a0 0x7f2b940b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:55.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.424+0000 7f2b98c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2b940d2310 0x7f2b940ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39460/0 (socket says 172.21.15.19:39460) 2024-03-28T13:14:55.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.424+0000 7f2b98c00640 1 -- 172.21.15.19:0/1843216346 learned_addr learned my addr 172.21.15.19:0/1843216346 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.424+0000 7f2b9a000640 1 --2- 172.21.15.19:0/1843216346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b940cf700 0x7f2b940b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.424+0000 7f2b99600640 1 -- 172.21.15.19:0/1843216346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b940cf700 msgr2=0x7f2b940b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.424+0000 7f2b99600640 1 --2- 172.21.15.19:0/1843216346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b940cf700 0x7f2b940b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.424+0000 7f2b99600640 1 -- 172.21.15.19:0/1843216346 >> v1:172.21.15.19:6789/0 conn(0x7f2b940d2310 legacy=0x7f2b940ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.424+0000 7f2b93e00640 1 -- 172.21.15.19:0/1843216346 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3560990439 0 0) 0x7f2b9415e880 con 0x7f2b940d2310 2024-03-28T13:14:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.424+0000 7f2b99600640 1 -- 172.21.15.19:0/1843216346 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b940d2bb0 con 0x7f2b9415e4a0 2024-03-28T13:14:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.424+0000 7f2b9a000640 1 --2- 172.21.15.19:0/1843216346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b940cf700 0x7f2b940b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:55.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.426+0000 7f2b99600640 1 --2- 172.21.15.19:0/1843216346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b9415e4a0 0x7f2b940b9c60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2b940d3660 tx=0x7f2b8802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e32ef728b41d358 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:55.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.427+0000 7f2b93e00640 1 -- 172.21.15.19:0/1843216346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88031780 con 0x7f2b9415e4a0 2024-03-28T13:14:55.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.427+0000 7f2b93e00640 1 -- 172.21.15.19:0/1843216346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b88037070 con 0x7f2b9415e4a0 2024-03-28T13:14:55.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.427+0000 7f2b93e00640 1 -- 172.21.15.19:0/1843216346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b88031b40 con 0x7f2b9415e4a0 2024-03-28T13:14:55.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.427+0000 7f2b9aa00640 1 -- 172.21.15.19:0/1843216346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b9415e4a0 msgr2=0x7f2b940b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:55.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.427+0000 7f2b9aa00640 1 --2- 172.21.15.19:0/1843216346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b9415e4a0 0x7f2b940b9c60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2b940d3660 tx=0x7f2b8802b820 comp rx=0 tx=0).stop 2024-03-28T13:14:55.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.427+0000 7f2b9aa00640 1 -- 172.21.15.19:0/1843216346 shutdown_connections 2024-03-28T13:14:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.427+0000 7f2b9aa00640 1 --2- 172.21.15.19:0/1843216346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b940cf700 0x7f2b940b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.427+0000 7f2b9aa00640 1 --2- 172.21.15.19:0/1843216346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b9415e4a0 0x7f2b940b9c60 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.427+0000 7f2b9aa00640 1 -- 172.21.15.19:0/1843216346 >> 172.21.15.19:0/1843216346 conn(0x7f2b940676d0 msgr2=0x7f2b94067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.427+0000 7f2b9aa00640 1 -- 172.21.15.19:0/1843216346 shutdown_connections 2024-03-28T13:14:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.428+0000 7f2b9aa00640 1 -- 172.21.15.19:0/1843216346 wait complete. 2024-03-28T13:14:55.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.429+0000 7f2b9aa00640 1 Processor -- start 2024-03-28T13:14:55.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.429+0000 7f2b9aa00640 1 -- start start 2024-03-28T13:14:55.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.430+0000 7f2b9aa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b940cf700 0x7f2b940c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:55.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.430+0000 7f2b9aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b940d2310 0x7f2b940c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:55.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.430+0000 7f2b9aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b9415e4a0 0x7f2b940cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:55.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.430+0000 7f2b9aa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2b940d3970 con 0x7f2b940d2310 2024-03-28T13:14:55.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.430+0000 7f2b9aa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2b940d3170 con 0x7f2b940cf700 2024-03-28T13:14:55.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.430+0000 7f2b9aa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2b940ba1a0 con 0x7f2b9415e4a0 2024-03-28T13:14:55.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.431+0000 7f2b98c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b940d2310 0x7f2b940c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:55.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.431+0000 7f2b98c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b940d2310 0x7f2b940c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35836/0 (socket says 172.21.15.19:35836) 2024-03-28T13:14:55.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.431+0000 7f2b98c00640 1 -- 172.21.15.19:0/893368956 learned_addr learned my addr 172.21.15.19:0/893368956 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:55.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.431+0000 7f2b9a000640 1 --2- 172.21.15.19:0/893368956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b9415e4a0 0x7f2b940cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:55.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.431+0000 7f2b98c00640 1 -- 172.21.15.19:0/893368956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b9415e4a0 msgr2=0x7f2b940cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:55.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.431+0000 7f2b98c00640 1 --2- 172.21.15.19:0/893368956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b9415e4a0 0x7f2b940cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:55.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.431+0000 7f2b98c00640 1 -- 172.21.15.19:0/893368956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b940cf700 msgr2=0x7f2b940c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:55.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.431+0000 7f2b98c00640 1 --2- 172.21.15.19:0/893368956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b940cf700 0x7f2b940c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:55.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.431+0000 7f2b98c00640 1 -- 172.21.15.19:0/893368956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b941f3aa0 con 0x7f2b940d2310 2024-03-28T13:14:55.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.431+0000 7f2b98c00640 1 --2- 172.21.15.19:0/893368956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b940d2310 0x7f2b940c8ce0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000c500 tx=0x7f2b8000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:55.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.432+0000 7f2b91600640 1 -- 172.21.15.19:0/893368956 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80018070 con 0x7f2b940d2310 2024-03-28T13:14:55.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.432+0000 7f2b91600640 1 -- 172.21.15.19:0/893368956 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b80007580 con 0x7f2b940d2310 2024-03-28T13:14:55.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.432+0000 7f2b91600640 1 -- 172.21.15.19:0/893368956 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80004a20 con 0x7f2b940d2310 2024-03-28T13:14:55.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.432+0000 7f2b9aa00640 1 -- 172.21.15.19:0/893368956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b941522e0 con 0x7f2b940d2310 2024-03-28T13:14:55.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.432+0000 7f2b9aa00640 1 -- 172.21.15.19:0/893368956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b940c9340 con 0x7f2b940d2310 2024-03-28T13:14:55.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.433+0000 7f2b9aa00640 1 -- 172.21.15.19:0/893368956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b94075940 con 0x7f2b940d2310 2024-03-28T13:14:55.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.434+0000 7f2b91600640 1 -- 172.21.15.19:0/893368956 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2b80004bc0 con 0x7f2b940d2310 2024-03-28T13:14:55.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.436+0000 7f2b91600640 1 --2- 172.21.15.19:0/893368956 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2b7406c530 0x7f2b7406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:55.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.436+0000 7f2b91600640 1 -- 172.21.15.19:0/893368956 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7f2b8008aa30 con 0x7f2b940d2310 2024-03-28T13:14:55.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.436+0000 7f2b99600640 1 --2- 172.21.15.19:0/893368956 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2b7406c530 0x7f2b7406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:55.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.438+0000 7f2b99600640 1 --2- 172.21.15.19:0/893368956 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2b7406c530 0x7f2b7406e9f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2b88036000 tx=0x7f2b88030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:55.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.439+0000 7f2b91600640 1 -- 172.21.15.19:0/893368956 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2b80058fe0 con 0x7f2b940d2310 2024-03-28T13:14:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.657+0000 7f2b9aa00640 1 -- 172.21.15.19:0/893368956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2b940b5850 con 0x7f2b940d2310 2024-03-28T13:14:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.658+0000 7f2b91600640 1 -- 172.21.15.19:0/893368956 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v78) ==== 80+0+122 (secure 0 0 0) 0x7f2b8005de20 con 0x7f2b940d2310 2024-03-28T13:14:55.657 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1189161682 2024-03-29T13:14:45.007899+0000 2024-03-28T13:14:55.658 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6832/1189161682 2024-03-29T13:14:45.007899+0000 2024-03-28T13:14:55.658 INFO:teuthology.orchestra.run.smithi019.stderr:listed 2 entries 2024-03-28T13:14:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.660+0000 7f2b9aa00640 1 -- 172.21.15.19:0/893368956 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2b7406c530 msgr2=0x7f2b7406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.660+0000 7f2b9aa00640 1 --2- 172.21.15.19:0/893368956 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2b7406c530 0x7f2b7406e9f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2b88036000 tx=0x7f2b88030000 comp rx=0 tx=0).stop 2024-03-28T13:14:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.660+0000 7f2b9aa00640 1 -- 172.21.15.19:0/893368956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b940d2310 msgr2=0x7f2b940c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.660+0000 7f2b9aa00640 1 --2- 172.21.15.19:0/893368956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b940d2310 0x7f2b940c8ce0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2b8000c500 tx=0x7f2b8000c9c0 comp rx=0 tx=0).stop 2024-03-28T13:14:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.661+0000 7f2b9aa00640 1 -- 172.21.15.19:0/893368956 shutdown_connections 2024-03-28T13:14:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.661+0000 7f2b9aa00640 1 --2- 172.21.15.19:0/893368956 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2b7406c530 0x7f2b7406e9f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.661+0000 7f2b9aa00640 1 --2- 172.21.15.19:0/893368956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b9415e4a0 0x7f2b940cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.661+0000 7f2b9aa00640 1 --2- 172.21.15.19:0/893368956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2b940d2310 0x7f2b940c8ce0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.661+0000 7f2b9aa00640 1 --2- 172.21.15.19:0/893368956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b940cf700 0x7f2b940c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.661+0000 7f2b9aa00640 1 -- 172.21.15.19:0/893368956 >> 172.21.15.19:0/893368956 conn(0x7f2b940676d0 msgr2=0x7f2b940ceba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.661+0000 7f2b9aa00640 1 -- 172.21.15.19:0/893368956 shutdown_connections 2024-03-28T13:14:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:55.661+0000 7f2b9aa00640 1 -- 172.21.15.19:0/893368956 wait complete. 2024-03-28T13:14:55.679 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:14:55.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:55.679 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T13:14:55.793 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:14:55.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:55.793 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:14:55.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:55.838 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-03-28T13:14:56.084 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:14:56.085+0000 7f6b77600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-28T13:14:56.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.109+0000 7f8351e00640 1 Processor -- start 2024-03-28T13:14:56.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.109+0000 7f8351e00640 1 -- start start 2024-03-28T13:14:56.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.110+0000 7f8351e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f834c141890 0x7f834c141d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.111+0000 7f8351e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f834c0cfbf0 con 0x7f834c13f190 2024-03-28T13:14:56.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.111+0000 7f8351e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f834c0cf350 con 0x7f834c06db90 2024-03-28T13:14:56.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.111+0000 7f8351e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f834c13f570 con 0x7f834c141890 2024-03-28T13:14:56.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.111+0000 7f8350a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f834c13f190 0x7f834c0ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39468/0 (socket says 172.21.15.19:39468) 2024-03-28T13:14:56.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.111+0000 7f8350a00640 1 -- 172.21.15.19:0/185327736 learned_addr learned my addr 172.21.15.19:0/185327736 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:56.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.111+0000 7f834be00640 1 --2- 172.21.15.19:0/185327736 >> v2:172.21.15.19:3300/0 conn(0x7f834c141890 0x7f834c141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.111+0000 7f834b400640 1 -- 172.21.15.19:0/185327736 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3111780891 0 0) 0x7f834c0cfbf0 con 0x7f834c13f190 2024-03-28T13:14:56.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.112+0000 7f834b400640 1 -- 172.21.15.19:0/185327736 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8334003520 con 0x7f834c13f190 2024-03-28T13:14:56.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.112+0000 7f834be00640 1 -- 172.21.15.19:0/185327736 >> v1:172.21.15.188:6789/0 conn(0x7f834c06db90 legacy=0x7f834c06dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.112+0000 7f834be00640 1 -- 172.21.15.19:0/185327736 >> v1:172.21.15.19:6789/0 conn(0x7f834c13f190 legacy=0x7f834c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.112+0000 7f834be00640 1 -- 172.21.15.19:0/185327736 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834c0cf630 con 0x7f834c141890 2024-03-28T13:14:56.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.113+0000 7f834be00640 1 --2- 172.21.15.19:0/185327736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f834c141890 0x7f834c141d20 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f83400097a0 tx=0x7f834002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accd9e182b58b961 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:56.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.113+0000 7f834b400640 1 -- 172.21.15.19:0/185327736 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8340034070 con 0x7f834c141890 2024-03-28T13:14:56.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.113+0000 7f834b400640 1 -- 172.21.15.19:0/185327736 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8340030610 con 0x7f834c141890 2024-03-28T13:14:56.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.113+0000 7f834b400640 1 -- 172.21.15.19:0/185327736 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83400309e0 con 0x7f834c141890 2024-03-28T13:14:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.114+0000 7f8351e00640 1 -- 172.21.15.19:0/185327736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f834c141890 msgr2=0x7f834c141d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.114+0000 7f8351e00640 1 --2- 172.21.15.19:0/185327736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f834c141890 0x7f834c141d20 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f83400097a0 tx=0x7f834002b960 comp rx=0 tx=0).stop 2024-03-28T13:14:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.114+0000 7f8351e00640 1 -- 172.21.15.19:0/185327736 shutdown_connections 2024-03-28T13:14:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.114+0000 7f8351e00640 1 --2- 172.21.15.19:0/185327736 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f834c141890 0x7f834c141d20 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.114+0000 7f8351e00640 1 -- 172.21.15.19:0/185327736 >> 172.21.15.19:0/185327736 conn(0x7f834c0b1d80 msgr2=0x7f834c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.114+0000 7f8351e00640 1 -- 172.21.15.19:0/185327736 shutdown_connections 2024-03-28T13:14:56.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.114+0000 7f8351e00640 1 -- 172.21.15.19:0/185327736 wait complete. 2024-03-28T13:14:56.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.116+0000 7f8351e00640 1 Processor -- start 2024-03-28T13:14:56.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.116+0000 7f8351e00640 1 -- start start 2024-03-28T13:14:56.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.117+0000 7f8351e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c06db90 0x7f834c15c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.117+0000 7f8351e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f834c13f190 0x7f834c13c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.117+0000 7f8351e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f834c141890 0x7f834c1f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.117+0000 7f8351e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f834c142550 con 0x7f834c141890 2024-03-28T13:14:56.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.117+0000 7f8351e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f834c06d670 con 0x7f834c13f190 2024-03-28T13:14:56.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.117+0000 7f8351e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f834c1426d0 con 0x7f834c06db90 2024-03-28T13:14:56.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.117+0000 7f8351400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f834c141890 0x7f834c1f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.118+0000 7f8351400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f834c141890 0x7f834c1f70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35860/0 (socket says 172.21.15.19:35860) 2024-03-28T13:14:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.118+0000 7f8351400640 1 -- 172.21.15.19:0/841393937 learned_addr learned my addr 172.21.15.19:0/841393937 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.118+0000 7f8350a00640 1 --2- 172.21.15.19:0/841393937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c06db90 0x7f834c15c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.118+0000 7f834be00640 1 --2- 172.21.15.19:0/841393937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f834c13f190 0x7f834c13c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.118+0000 7f8351400640 1 -- 172.21.15.19:0/841393937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c06db90 msgr2=0x7f834c15c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.118+0000 7f8351400640 1 --2- 172.21.15.19:0/841393937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c06db90 0x7f834c15c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.118+0000 7f8351400640 1 -- 172.21.15.19:0/841393937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f834c13f190 msgr2=0x7f834c13c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.118+0000 7f8351400640 1 --2- 172.21.15.19:0/841393937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f834c13f190 0x7f834c13c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.118+0000 7f8351400640 1 -- 172.21.15.19:0/841393937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834c1f7630 con 0x7f834c141890 2024-03-28T13:14:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.118+0000 7f834be00640 1 --2- 172.21.15.19:0/841393937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f834c13f190 0x7f834c13c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:56.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.118+0000 7f8351400640 1 --2- 172.21.15.19:0/841393937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f834c141890 0x7f834c1f70f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f834400c7d0 tx=0x7f834400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.119+0000 7f8348c00640 1 -- 172.21.15.19:0/841393937 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344003b70 con 0x7f834c141890 2024-03-28T13:14:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.119+0000 7f8348c00640 1 -- 172.21.15.19:0/841393937 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8344003d30 con 0x7f834c141890 2024-03-28T13:14:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.119+0000 7f8351e00640 1 -- 172.21.15.19:0/841393937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f834c15cda0 con 0x7f834c141890 2024-03-28T13:14:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.119+0000 7f8348c00640 1 -- 172.21.15.19:0/841393937 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834400aa90 con 0x7f834c141890 2024-03-28T13:14:56.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.119+0000 7f8351e00640 1 -- 172.21.15.19:0/841393937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f834c15d360 con 0x7f834c141890 2024-03-28T13:14:56.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.122+0000 7f8348c00640 1 -- 172.21.15.19:0/841393937 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f834400ac30 con 0x7f834c141890 2024-03-28T13:14:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.123+0000 7f8356186740 1 Processor -- start 2024-03-28T13:14:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.123+0000 7f8356186740 1 -- start start 2024-03-28T13:14:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.123+0000 7f8348c00640 1 --2- 172.21.15.19:0/841393937 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f832c06c530 0x7f832c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.123+0000 7f8348c00640 1 -- 172.21.15.19:0/841393937 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7f834408b750 con 0x7f834c141890 2024-03-28T13:14:56.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.124+0000 7f834be00640 1 --2- 172.21.15.19:0/841393937 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f832c06c530 0x7f832c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.124+0000 7f8356186740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ab9220 0x557846ab9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.124+0000 7f8356186740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846abd690 0x557846abf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.124+0000 7f8356186740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846ae93d0 0x557846ae9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.124+0000 7f8356186740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557846a23930 con 0x557846ae93d0 2024-03-28T13:14:56.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.124+0000 7f8356186740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5578469cb780 con 0x557846abd690 2024-03-28T13:14:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.124+0000 7f8356186740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x557846a1f8e0 con 0x557846ab9220 2024-03-28T13:14:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.124+0000 7f8351400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846ae93d0 0x557846ae9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.125+0000 7f8351400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846ae93d0 0x557846ae9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35870/0 (socket says 172.21.15.19:35870) 2024-03-28T13:14:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.125+0000 7f8351400640 1 -- 172.21.15.19:0/3748417692 learned_addr learned my addr 172.21.15.19:0/3748417692 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.125+0000 7f8350a00640 1 --2- 172.21.15.19:0/3748417692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ab9220 0x557846ab9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.125+0000 7f834be00640 1 --2- 172.21.15.19:0/3748417692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846abd690 0x557846abf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.125+0000 7f8351400640 1 -- 172.21.15.19:0/3748417692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ab9220 msgr2=0x557846ab9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.125+0000 7f8351400640 1 --2- 172.21.15.19:0/3748417692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ab9220 0x557846ab9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.125+0000 7f8351400640 1 -- 172.21.15.19:0/3748417692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846abd690 msgr2=0x557846abf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.125+0000 7f8351400640 1 --2- 172.21.15.19:0/3748417692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846abd690 0x557846abf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.125+0000 7f8351400640 1 -- 172.21.15.19:0/3748417692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557846803a20 con 0x557846ae93d0 2024-03-28T13:14:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.125+0000 7f834be00640 1 --2- 172.21.15.19:0/841393937 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f832c06c530 0x7f832c06e9f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8340036000 tx=0x7f8340030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.125+0000 7f8351400640 1 --2- 172.21.15.19:0/3748417692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846ae93d0 0x557846ae9850 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f8344089c40 tx=0x7f83440014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5abb333005840fe server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.125+0000 7f833d200640 1 -- 172.21.15.19:0/3748417692 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344059580 con 0x557846ae93d0 2024-03-28T13:14:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.126+0000 7f833d200640 1 -- 172.21.15.19:0/3748417692 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f834405d930 con 0x557846ae93d0 2024-03-28T13:14:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.126+0000 7f833d200640 1 -- 172.21.15.19:0/3748417692 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83440621f0 con 0x557846ae93d0 2024-03-28T13:14:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.126+0000 7f8356186740 1 -- 172.21.15.19:0/3748417692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846ae93d0 msgr2=0x557846ae9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.126+0000 7f8356186740 1 --2- 172.21.15.19:0/3748417692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846ae93d0 0x557846ae9850 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f8344089c40 tx=0x7f83440014a0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.126+0000 7f8356186740 1 -- 172.21.15.19:0/3748417692 shutdown_connections 2024-03-28T13:14:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.126+0000 7f8356186740 1 --2- 172.21.15.19:0/3748417692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846ae93d0 0x557846ae9850 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.126+0000 7f8356186740 1 --2- 172.21.15.19:0/3748417692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846abd690 0x557846abf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.126+0000 7f8356186740 1 --2- 172.21.15.19:0/3748417692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ab9220 0x557846ab9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.126+0000 7f8356186740 1 -- 172.21.15.19:0/3748417692 >> 172.21.15.19:0/3748417692 conn(0x557846abb6c0 msgr2=0x557846abcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.126+0000 7f8356186740 1 -- 172.21.15.19:0/3748417692 shutdown_connections 2024-03-28T13:14:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.126+0000 7f8356186740 1 -- 172.21.15.19:0/3748417692 wait complete. 2024-03-28T13:14:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.127+0000 7f8356186740 1 -- start start 2024-03-28T13:14:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.128+0000 7f8356186740 1 Processor -- start 2024-03-28T13:14:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.129+0000 7f8356186740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846ae93d0 0x557846b08aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.129+0000 7f8356186740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ae29a0 0x557846ae57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.129+0000 7f8356186740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846adc300 0x557846adc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.129+0000 7f8356186740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557846abde00 con 0x557846adc300 2024-03-28T13:14:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.129+0000 7f8356186740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5578469deca0 con 0x557846ae93d0 2024-03-28T13:14:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.129+0000 7f8356186740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x557846abdb20 con 0x557846ae29a0 2024-03-28T13:14:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.129+0000 7f8356186740 10 client.0 _ll_register_callbacks cb 0x5578469e3450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:14:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.129+0000 7f8351400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846adc300 0x557846adc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.129+0000 7f8351400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846adc300 0x557846adc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35874/0 (socket says 172.21.15.19:35874) 2024-03-28T13:14:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.129+0000 7f8351400640 1 -- 172.21.15.19:0/2506531037 learned_addr learned my addr 172.21.15.19:0/2506531037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.129+0000 7f8350a00640 1 --2- 172.21.15.19:0/2506531037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846ae93d0 0x557846b08aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.130+0000 7f834be00640 1 --2- 172.21.15.19:0/2506531037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ae29a0 0x557846ae57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.130+0000 7f8351400640 1 -- 172.21.15.19:0/2506531037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ae29a0 msgr2=0x557846ae57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.130+0000 7f8351400640 1 --2- 172.21.15.19:0/2506531037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ae29a0 0x557846ae57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.130+0000 7f8351400640 1 -- 172.21.15.19:0/2506531037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846ae93d0 msgr2=0x557846b08aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.130+0000 7f8351400640 1 --2- 172.21.15.19:0/2506531037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846ae93d0 0x557846b08aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.130+0000 7f8351400640 1 -- 172.21.15.19:0/2506531037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578466f7310 con 0x557846adc300 2024-03-28T13:14:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.130+0000 7f8351400640 1 --2- 172.21.15.19:0/2506531037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846adc300 0x557846adc780 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f834400a430 tx=0x7f8344089af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f1da9328c4c7b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.130+0000 7f8331600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:14:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.130+0000 7f8331600640 1 -- 172.21.15.19:0/2506531037 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344058bd0 con 0x557846adc300 2024-03-28T13:14:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.130+0000 7f8331600640 1 -- 172.21.15.19:0/2506531037 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7f83440552d0 con 0x557846adc300 2024-03-28T13:14:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.130+0000 7f8356186740 1 -- 172.21.15.19:0/2506531037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f834405d930 con 0x557846adc300 2024-03-28T13:14:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.131+0000 7f8331600640 1 -- 172.21.15.19:0/2506531037 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f83440554d0 con 0x557846adc300 2024-03-28T13:14:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.131+0000 7f8356186740 10 client.6816 fetch_fsmap learned FSMap version 52 2024-03-28T13:14:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.131+0000 7f8356186740 1 -- 172.21.15.19:0/2506531037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5578468641b0 con 0x557846adc300 2024-03-28T13:14:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.131+0000 7f8331600640 1 -- 172.21.15.19:0/2506531037 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f834405d930 con 0x557846adc300 2024-03-28T13:14:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.131+0000 7f8331600640 1 -- 172.21.15.19:0/2506531037 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f8344058440 con 0x557846adc300 2024-03-28T13:14:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.131+0000 7f8356186740 10 client.6816 fetch_fsmap finished waiting for FSMap version 52 2024-03-28T13:14:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.131+0000 7f8356186740 10 client.6816 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-28T13:14:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.131+0000 7f8356186740 20 client.6816 populate_metadata read hostname 'smithi019' 2024-03-28T13:14:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.131+0000 7f8356186740 1 --2- 172.21.15.19:0/2506531037 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x557846aeba60 0x557846b0c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.131+0000 7f8356186740 4 client.6816 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:14:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.131+0000 7f8356186740 1 -- 172.21.15.19:0/2506531037 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557846a30e70 con 0x557846aeba60 2024-03-28T13:14:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.131+0000 7f8350a00640 1 --2- 172.21.15.19:0/2506531037 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x557846aeba60 0x557846b0c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.132+0000 7f8350a00640 1 --2- 172.21.15.19:0/2506531037 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x557846aeba60 0x557846b0c020 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2804cf39076ebdd3 server_cookie=900d6fc084163bb in_seq=0 out_seq=0 2024-03-28T13:14:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.132+0000 7f8331600640 10 client.6816 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:14:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.133+0000 7f8331600640 1 -- 172.21.15.19:0/2506531037 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557846a30e70 con 0x557846aeba60 2024-03-28T13:14:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.133+0000 7f8331600640 10 client.6816 handle_command_reply: tid=0 2024-03-28T13:14:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.133+0000 7f8356186740 1 client.6816 shutdown 2024-03-28T13:14:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.133+0000 7f8356186740 1 -- 172.21.15.19:0/2506531037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846adc300 msgr2=0x557846adc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.133+0000 7f8356186740 1 --2- 172.21.15.19:0/2506531037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846adc300 0x557846adc780 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f834400a430 tx=0x7f8344089af0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.133+0000 7f8356186740 1 -- 172.21.15.19:0/2506531037 shutdown_connections 2024-03-28T13:14:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.133+0000 7f8356186740 1 --2- 172.21.15.19:0/2506531037 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x557846aeba60 0x557846b0c020 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.133+0000 7f8356186740 1 --2- 172.21.15.19:0/2506531037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846adc300 0x557846adc780 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.133+0000 7f8356186740 1 --2- 172.21.15.19:0/2506531037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ae29a0 0x557846ae57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.133+0000 7f8356186740 1 --2- 172.21.15.19:0/2506531037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846ae93d0 0x557846b08aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.133+0000 7f8356186740 1 -- 172.21.15.19:0/2506531037 >> 172.21.15.19:0/2506531037 conn(0x557846ab9220 msgr2=0x557846abed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.133+0000 7f8331600640 0 client.6816 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:14:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.133+0000 7f8356186740 1 -- 172.21.15.19:0/2506531037 shutdown_connections 2024-03-28T13:14:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.133+0000 7f8356186740 1 -- 172.21.15.19:0/2506531037 wait complete. 2024-03-28T13:14:56.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.134+0000 7f8356186740 20 client.6816 trim_cache size 0 max 16384 2024-03-28T13:14:56.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.153+0000 7f8356186740 1 Processor -- start 2024-03-28T13:14:56.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.153+0000 7f8356186740 1 -- start start 2024-03-28T13:14:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.154+0000 7f8356186740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846adc300 0x557846b31320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.154+0000 7f8356186740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846ae29a0 0x557846b04bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.154+0000 7f8356186740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ae93d0 0x557846ae97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.154+0000 7f8356186740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f834c158fa0 con 0x557846adc300 2024-03-28T13:14:56.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.154+0000 7f8356186740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f834c157fe0 con 0x557846ae29a0 2024-03-28T13:14:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.154+0000 7f8356186740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f834c071970 con 0x557846ae93d0 2024-03-28T13:14:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.154+0000 7f8350a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846adc300 0x557846b31320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.154+0000 7f8350a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846adc300 0x557846b31320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35876/0 (socket says 172.21.15.19:35876) 2024-03-28T13:14:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.154+0000 7f8350a00640 1 -- 172.21.15.19:0/3253434501 learned_addr learned my addr 172.21.15.19:0/3253434501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.154+0000 7f8350a00640 1 -- 172.21.15.19:0/3253434501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ae93d0 msgr2=0x557846ae97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.154+0000 7f8350a00640 1 --2- 172.21.15.19:0/3253434501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ae93d0 0x557846ae97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.154+0000 7f8350a00640 1 -- 172.21.15.19:0/3253434501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846ae29a0 msgr2=0x557846b04bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.154+0000 7f8350a00640 1 --2- 172.21.15.19:0/3253434501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846ae29a0 0x557846b04bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.154+0000 7f8350a00640 1 -- 172.21.15.19:0/3253434501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578466d0490 con 0x557846adc300 2024-03-28T13:14:56.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.155+0000 7f8350a00640 1 --2- 172.21.15.19:0/3253434501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846adc300 0x557846b31320 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8338000f60 tx=0x7f8338009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8587b3e50a809f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.155+0000 7f8333e00640 1 -- 172.21.15.19:0/3253434501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833801e020 con 0x557846adc300 2024-03-28T13:14:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.155+0000 7f8333e00640 1 -- 172.21.15.19:0/3253434501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f833801ae50 con 0x557846adc300 2024-03-28T13:14:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.155+0000 7f8333e00640 1 -- 172.21.15.19:0/3253434501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338007750 con 0x557846adc300 2024-03-28T13:14:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.156+0000 7f8356186740 1 -- 172.21.15.19:0/3253434501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846adc300 msgr2=0x557846b31320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.156+0000 7f8356186740 1 --2- 172.21.15.19:0/3253434501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846adc300 0x557846b31320 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8338000f60 tx=0x7f8338009040 comp rx=0 tx=0).stop 2024-03-28T13:14:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.156+0000 7f8356186740 1 -- 172.21.15.19:0/3253434501 shutdown_connections 2024-03-28T13:14:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.156+0000 7f8356186740 1 --2- 172.21.15.19:0/3253434501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ae93d0 0x557846ae97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.156+0000 7f8356186740 1 --2- 172.21.15.19:0/3253434501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846ae29a0 0x557846b04bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.156+0000 7f8356186740 1 --2- 172.21.15.19:0/3253434501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846adc300 0x557846b31320 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.156+0000 7f8356186740 1 -- 172.21.15.19:0/3253434501 >> 172.21.15.19:0/3253434501 conn(0x557846ab9220 msgr2=0x557846abcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.156+0000 7f8356186740 1 -- 172.21.15.19:0/3253434501 shutdown_connections 2024-03-28T13:14:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.156+0000 7f8356186740 1 -- 172.21.15.19:0/3253434501 wait complete. 2024-03-28T13:14:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.157+0000 7f8356186740 1 -- start start 2024-03-28T13:14:56.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.157+0000 7f8356186740 1 Processor -- start 2024-03-28T13:14:56.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.158+0000 7f8356186740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846adc300 0x557846abfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.158+0000 7f8356186740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ae29a0 0x557846b23b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.158+0000 7f8356186740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846ae93d0 0x557846b872d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.158+0000 7f8356186740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557846ac0720 con 0x557846ae93d0 2024-03-28T13:14:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.158+0000 7f8356186740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f834c071fc0 con 0x557846adc300 2024-03-28T13:14:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.158+0000 7f8356186740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x557846b1d2c0 con 0x557846ae29a0 2024-03-28T13:14:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.158+0000 7f8356186740 10 client.0 _ll_register_callbacks cb 0x7f83440593e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:14:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.158+0000 7f8351400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846ae93d0 0x557846b872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.158+0000 7f8351400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846ae93d0 0x557846b872d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35882/0 (socket says 172.21.15.19:35882) 2024-03-28T13:14:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.158+0000 7f8351400640 1 -- 172.21.15.19:0/1792956047 learned_addr learned my addr 172.21.15.19:0/1792956047 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.158+0000 7f8350a00640 1 --2- 172.21.15.19:0/1792956047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846adc300 0x557846abfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.159+0000 7f8351400640 1 -- 172.21.15.19:0/1792956047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ae29a0 msgr2=0x557846b23b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.159+0000 7f834be00640 1 --2- 172.21.15.19:0/1792956047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ae29a0 0x557846b23b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.159+0000 7f8351400640 1 --2- 172.21.15.19:0/1792956047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ae29a0 0x557846b23b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.159+0000 7f8351400640 1 -- 172.21.15.19:0/1792956047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846adc300 msgr2=0x557846abfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.159+0000 7f8351400640 1 --2- 172.21.15.19:0/1792956047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846adc300 0x557846abfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.159+0000 7f8351400640 1 -- 172.21.15.19:0/1792956047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557846b276e0 con 0x557846ae93d0 2024-03-28T13:14:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.159+0000 7f8350a00640 1 --2- 172.21.15.19:0/1792956047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846adc300 0x557846abfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.159+0000 7f8351400640 1 --2- 172.21.15.19:0/1792956047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846ae93d0 0x557846b872d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8344002230 tx=0x7f8344011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac86b1decc520e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.159+0000 7f8331600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:14:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.159+0000 7f8331600640 1 -- 172.21.15.19:0/1792956047 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8344055820 con 0x557846ae93d0 2024-03-28T13:14:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.159+0000 7f8331600640 1 -- 172.21.15.19:0/1792956047 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7f8344058be0 con 0x557846ae93d0 2024-03-28T13:14:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.159+0000 7f8356186740 1 -- 172.21.15.19:0/1792956047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f833801ae50 con 0x557846ae93d0 2024-03-28T13:14:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.159+0000 7f8331600640 1 -- 172.21.15.19:0/1792956047 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8344067050 con 0x557846ae93d0 2024-03-28T13:14:56.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.160+0000 7f8356186740 10 client.6822 fetch_fsmap learned FSMap version 52 2024-03-28T13:14:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.160+0000 7f8356186740 1 -- 172.21.15.19:0/1792956047 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x557846b276e0 con 0x557846ae93d0 2024-03-28T13:14:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.160+0000 7f8331600640 1 -- 172.21.15.19:0/1792956047 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f833801ae50 con 0x557846ae93d0 2024-03-28T13:14:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.160+0000 7f8331600640 1 -- 172.21.15.19:0/1792956047 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f8344083080 con 0x557846ae93d0 2024-03-28T13:14:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.160+0000 7f8356186740 10 client.6822 fetch_fsmap finished waiting for FSMap version 52 2024-03-28T13:14:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.160+0000 7f8356186740 10 client.6822 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-28T13:14:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.160+0000 7f8356186740 20 client.6822 populate_metadata read hostname 'smithi019' 2024-03-28T13:14:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.160+0000 7f8356186740 1 --2- 172.21.15.19:0/1792956047 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x557846abe940 0x557846abed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.160+0000 7f8356186740 4 client.6822 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-03-28T13:14:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.160+0000 7f8356186740 1 -- 172.21.15.19:0/1792956047 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x557846a2b1c0 con 0x557846abe940 2024-03-28T13:14:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.160+0000 7f8350a00640 1 --2- 172.21.15.19:0/1792956047 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x557846abe940 0x557846abed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.161+0000 7f8350a00640 1 --2- 172.21.15.19:0/1792956047 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x557846abe940 0x557846abed50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=668b254e585163f9 server_cookie=d774ee9b1de9b74c in_seq=0 out_seq=0 2024-03-28T13:14:56.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.161+0000 7f8331600640 10 client.6822 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:14:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.162+0000 7f8331600640 1 -- 172.21.15.19:0/1792956047 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557846a2b1c0 con 0x557846abe940 2024-03-28T13:14:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.162+0000 7f8331600640 10 client.6822 handle_command_reply: tid=0 2024-03-28T13:14:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.162+0000 7f8356186740 1 client.6822 shutdown 2024-03-28T13:14:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.162+0000 7f8356186740 1 -- 172.21.15.19:0/1792956047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846ae93d0 msgr2=0x557846b872d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.162+0000 7f8356186740 1 --2- 172.21.15.19:0/1792956047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846ae93d0 0x557846b872d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8344002230 tx=0x7f8344011000 comp rx=0 tx=0).stop 2024-03-28T13:14:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.162+0000 7f8356186740 1 -- 172.21.15.19:0/1792956047 shutdown_connections 2024-03-28T13:14:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.162+0000 7f8356186740 1 --2- 172.21.15.19:0/1792956047 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x557846abe940 0x557846abed50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.162+0000 7f8356186740 1 --2- 172.21.15.19:0/1792956047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557846ae93d0 0x557846b872d0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.162+0000 7f8356186740 1 --2- 172.21.15.19:0/1792956047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557846ae29a0 0x557846b23b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.162+0000 7f8356186740 1 --2- 172.21.15.19:0/1792956047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557846adc300 0x557846abfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.162+0000 7f8331600640 0 client.6822 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:14:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.162+0000 7f8356186740 1 -- 172.21.15.19:0/1792956047 >> 172.21.15.19:0/1792956047 conn(0x557846ab9220 msgr2=0x557846b09460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.162+0000 7f8356186740 1 -- 172.21.15.19:0/1792956047 shutdown_connections 2024-03-28T13:14:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.162+0000 7f8356186740 1 -- 172.21.15.19:0/1792956047 wait complete. 2024-03-28T13:14:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.162+0000 7f8356186740 20 client.6822 trim_cache size 0 max 16384 2024-03-28T13:14:56.166 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:14:56.166 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-03-28T13:14:56.166 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "0d5f4cdc-5ce6-4d4a-9a5f-04f110eb5c56", 2024-03-28T13:14:56.166 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-03-28T13:14:56.166 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:14:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.163+0000 7f8351e00640 1 -- 172.21.15.19:0/841393937 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f832c06c530 msgr2=0x7f832c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.163+0000 7f8351e00640 1 --2- 172.21.15.19:0/841393937 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f832c06c530 0x7f832c06e9f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f8340036000 tx=0x7f8340030000 comp rx=0 tx=0).stop 2024-03-28T13:14:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.163+0000 7f8351e00640 1 -- 172.21.15.19:0/841393937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f834c141890 msgr2=0x7f834c1f70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.163+0000 7f8351e00640 1 --2- 172.21.15.19:0/841393937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f834c141890 0x7f834c1f70f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f834400c7d0 tx=0x7f834400cc90 comp rx=0 tx=0).stop 2024-03-28T13:14:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.163+0000 7f8351e00640 1 -- 172.21.15.19:0/841393937 shutdown_connections 2024-03-28T13:14:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.163+0000 7f8351e00640 1 --2- 172.21.15.19:0/841393937 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f832c06c530 0x7f832c06e9f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.163+0000 7f8351e00640 1 --2- 172.21.15.19:0/841393937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f834c141890 0x7f834c1f70f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.163+0000 7f8351e00640 1 --2- 172.21.15.19:0/841393937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f834c13f190 0x7f834c13c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.163+0000 7f8351e00640 1 --2- 172.21.15.19:0/841393937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f834c06db90 0x7f834c15c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.163+0000 7f8351e00640 1 -- 172.21.15.19:0/841393937 >> 172.21.15.19:0/841393937 conn(0x7f834c0b1d80 msgr2=0x7f834c15b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.163+0000 7f8351e00640 1 -- 172.21.15.19:0/841393937 shutdown_connections 2024-03-28T13:14:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.163+0000 7f8351e00640 1 -- 172.21.15.19:0/841393937 wait complete. 2024-03-28T13:14:56.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:14:56.182 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-28T13:14:56.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.397+0000 7f5e89600640 1 Processor -- start 2024-03-28T13:14:56.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.397+0000 7f5e89600640 1 -- start start 2024-03-28T13:14:56.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.399+0000 7f5e89600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e840afd50 0x7f5e8406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.399+0000 7f5e89600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5e8406c8b0 0x7f5e840b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.399+0000 7f5e89600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e8415ed90 con 0x7f5e840b3760 2024-03-28T13:14:56.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.399+0000 7f5e89600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5e840b4100 con 0x7f5e8406c8b0 2024-03-28T13:14:56.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.399+0000 7f5e89600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5e8415f070 con 0x7f5e840afd50 2024-03-28T13:14:56.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.399+0000 7f5e83400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f5e840b3760 0x7f5e84161f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:39484/0 (socket says 172.21.15.19:39484) 2024-03-28T13:14:56.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.399+0000 7f5e83400640 1 -- 172.21.15.19:0/1630829993 learned_addr learned my addr 172.21.15.19:0/1630829993 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:14:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.400+0000 7f5e88c00640 1 --2- 172.21.15.19:0/1630829993 >> v2:172.21.15.188:3300/0 conn(0x7f5e8406c8b0 0x7f5e840b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.400+0000 7f5e83e00640 1 --2- 172.21.15.19:0/1630829993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e840afd50 0x7f5e8406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.400+0000 7f5e82a00640 1 -- 172.21.15.19:0/1630829993 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 700673421 0 0) 0x7f5e8415ed90 con 0x7f5e840b3760 2024-03-28T13:14:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.400+0000 7f5e82a00640 1 -- 172.21.15.19:0/1630829993 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e68003520 con 0x7f5e840b3760 2024-03-28T13:14:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e82a00640 1 -- 172.21.15.19:0/1630829993 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3414857983 0 0) 0x7f5e68003520 con 0x7f5e840b3760 2024-03-28T13:14:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e82a00640 1 -- 172.21.15.19:0/1630829993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e840afd50 msgr2=0x7f5e8406ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e82a00640 1 --2- 172.21.15.19:0/1630829993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e840afd50 0x7f5e8406ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e82a00640 1 -- 172.21.15.19:0/1630829993 >> v2:172.21.15.188:3300/0 conn(0x7f5e8406c8b0 msgr2=0x7f5e840b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e82a00640 1 --2- 172.21.15.19:0/1630829993 >> v2:172.21.15.188:3300/0 conn(0x7f5e8406c8b0 0x7f5e840b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e82a00640 1 -- 172.21.15.19:0/1630829993 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8415f350 con 0x7f5e840b3760 2024-03-28T13:14:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e82a00640 1 -- 172.21.15.19:0/1630829993 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f5e70001670 con 0x7f5e840b3760 2024-03-28T13:14:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e82a00640 1 -- 172.21.15.19:0/1630829993 >> v1:172.21.15.19:6789/0 conn(0x7f5e840b3760 legacy=0x7f5e84161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e82a00640 1 --2- 172.21.15.19:0/1630829993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e680039b0 0x7f5e68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e82a00640 1 --2- 172.21.15.19:0/1630829993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e840afd50 0x7f5e68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e82a00640 1 --2- 172.21.15.19:0/1630829993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e68004d40 0x7f5e680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e82a00640 1 -- 172.21.15.19:0/1630829993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8415f070 con 0x7f5e68004d40 2024-03-28T13:14:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e82a00640 1 -- 172.21.15.19:0/1630829993 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6800b9c0 con 0x7f5e680039b0 2024-03-28T13:14:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e82a00640 1 -- 172.21.15.19:0/1630829993 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5e6800bb40 con 0x7f5e840afd50 2024-03-28T13:14:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e88c00640 1 --2- 172.21.15.19:0/1630829993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e68004d40 0x7f5e680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e83e00640 1 --2- 172.21.15.19:0/1630829993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e680039b0 0x7f5e68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.401+0000 7f5e83400640 1 --2- 172.21.15.19:0/1630829993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e840afd50 0x7f5e68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.402+0000 7f5e88c00640 1 -- 172.21.15.19:0/1630829993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e840afd50 msgr2=0x7f5e68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.402+0000 7f5e88c00640 1 --2- 172.21.15.19:0/1630829993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e840afd50 0x7f5e68004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.402+0000 7f5e88c00640 1 -- 172.21.15.19:0/1630829993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e680039b0 msgr2=0x7f5e68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.402+0000 7f5e88c00640 1 --2- 172.21.15.19:0/1630829993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e680039b0 0x7f5e68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.402+0000 7f5e88c00640 1 -- 172.21.15.19:0/1630829993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e68003520 con 0x7f5e68004d40 2024-03-28T13:14:56.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.402+0000 7f5e83400640 1 --2- 172.21.15.19:0/1630829993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e840afd50 0x7f5e68004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:56.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.403+0000 7f5e88c00640 1 --2- 172.21.15.19:0/1630829993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e68004d40 0x7f5e680051c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009f90 tx=0x7f5e68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a2d28cc8d26e5c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:56.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.403+0000 7f5e82a00640 1 -- 172.21.15.19:0/1630829993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c039070 con 0x7f5e68004d40 2024-03-28T13:14:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.403+0000 7f5e82a00640 1 -- 172.21.15.19:0/1630829993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e7c02cc20 con 0x7f5e68004d40 2024-03-28T13:14:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.403+0000 7f5e82a00640 1 -- 172.21.15.19:0/1630829993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c009c10 con 0x7f5e68004d40 2024-03-28T13:14:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.404+0000 7f5e89600640 1 -- 172.21.15.19:0/1630829993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e68004d40 msgr2=0x7f5e680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.404+0000 7f5e89600640 1 --2- 172.21.15.19:0/1630829993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e68004d40 0x7f5e680051c0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009f90 tx=0x7f5e68005830 comp rx=0 tx=0).stop 2024-03-28T13:14:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.404+0000 7f5e88c00640 1 -- 172.21.15.19:0/1630829993 reap_dead start 2024-03-28T13:14:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.404+0000 7f5e89600640 1 -- 172.21.15.19:0/1630829993 shutdown_connections 2024-03-28T13:14:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.404+0000 7f5e89600640 1 -- 172.21.15.19:0/1630829993 >> 172.21.15.19:0/1630829993 conn(0x7f5e840d0770 msgr2=0x7f5e840d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.404+0000 7f5e89600640 1 -- 172.21.15.19:0/1630829993 shutdown_connections 2024-03-28T13:14:56.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.404+0000 7f5e89600640 1 -- 172.21.15.19:0/1630829993 wait complete. 2024-03-28T13:14:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.406+0000 7f5e89600640 1 Processor -- start 2024-03-28T13:14:56.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.406+0000 7f5e89600640 1 -- start start 2024-03-28T13:14:56.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.407+0000 7f5e89600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e68004d40 0x7f5e8414ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.407+0000 7f5e89600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8414adf0 0x7f5e840ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.407+0000 7f5e89600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e840addd0 0x7f5e84153640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.407+0000 7f5e89600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8406fa20 con 0x7f5e8414adf0 2024-03-28T13:14:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.407+0000 7f5e89600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5e840b0130 con 0x7f5e840addd0 2024-03-28T13:14:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.407+0000 7f5e89600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5e8406fba0 con 0x7f5e68004d40 2024-03-28T13:14:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.407+0000 7f5e83400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8414adf0 0x7f5e840ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.407+0000 7f5e83400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8414adf0 0x7f5e840ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35900/0 (socket says 172.21.15.19:35900) 2024-03-28T13:14:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.407+0000 7f5e83400640 1 -- 172.21.15.19:0/4234856979 learned_addr learned my addr 172.21.15.19:0/4234856979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.407+0000 7f5e83e00640 1 --2- 172.21.15.19:0/4234856979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e68004d40 0x7f5e8414ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.408+0000 7f5e83400640 1 -- 172.21.15.19:0/4234856979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e68004d40 msgr2=0x7f5e8414ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.408+0000 7f5e88c00640 1 --2- 172.21.15.19:0/4234856979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e840addd0 0x7f5e84153640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.408+0000 7f5e83400640 1 --2- 172.21.15.19:0/4234856979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e68004d40 0x7f5e8414ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.408+0000 7f5e83400640 1 -- 172.21.15.19:0/4234856979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e840addd0 msgr2=0x7f5e84153640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.408+0000 7f5e83400640 1 --2- 172.21.15.19:0/4234856979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e840addd0 0x7f5e84153640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.408+0000 7f5e83400640 1 -- 172.21.15.19:0/4234856979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e840af7a0 con 0x7f5e8414adf0 2024-03-28T13:14:56.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.408+0000 7f5e83400640 1 --2- 172.21.15.19:0/4234856979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8414adf0 0x7f5e840ad890 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000d910 tx=0x7f5e7000ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.408+0000 7f5e7be00640 1 -- 172.21.15.19:0/4234856979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70019070 con 0x7f5e8414adf0 2024-03-28T13:14:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.408+0000 7f5e7be00640 1 -- 172.21.15.19:0/4234856979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e700074e0 con 0x7f5e8414adf0 2024-03-28T13:14:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.408+0000 7f5e7be00640 1 -- 172.21.15.19:0/4234856979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70007690 con 0x7f5e8414adf0 2024-03-28T13:14:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.408+0000 7f5e89600640 1 -- 172.21.15.19:0/4234856979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e840ae400 con 0x7f5e8414adf0 2024-03-28T13:14:56.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.408+0000 7f5e89600640 1 -- 172.21.15.19:0/4234856979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e841591c0 con 0x7f5e8414adf0 2024-03-28T13:14:56.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.412+0000 7f5e7be00640 1 -- 172.21.15.19:0/4234856979 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5e7001c070 con 0x7f5e8414adf0 2024-03-28T13:14:56.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.412+0000 7f5e8d9d8740 1 Processor -- start 2024-03-28T13:14:56.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.412+0000 7f5e8d9d8740 1 -- start start 2024-03-28T13:14:56.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.413+0000 7f5e7be00640 1 --2- 172.21.15.19:0/4234856979 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5e6406c530 0x7f5e6406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.413+0000 7f5e7be00640 1 -- 172.21.15.19:0/4234856979 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7f5e7008b370 con 0x7f5e8414adf0 2024-03-28T13:14:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.413+0000 7f5e8d9d8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0573720 0x5587e05466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.413+0000 7f5e8d9d8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e05764b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.413+0000 7f5e8d9d8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0546c00 0x5587e05769f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.413+0000 7f5e8d9d8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5587e04b16b0 con 0x5587e0546c00 2024-03-28T13:14:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.413+0000 7f5e8d9d8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5587e0459750 con 0x5587e0573720 2024-03-28T13:14:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.413+0000 7f5e8d9d8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5587e04ad660 con 0x5587e0573b00 2024-03-28T13:14:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.414+0000 7f5e88c00640 1 --2- 172.21.15.19:0/4234856979 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5e6406c530 0x7f5e6406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.414+0000 7f5e88c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0546c00 0x5587e05769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.414+0000 7f5e88c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0546c00 0x5587e05769f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35902/0 (socket says 172.21.15.19:35902) 2024-03-28T13:14:56.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.414+0000 7f5e88c00640 1 -- 172.21.15.19:0/1215384056 learned_addr learned my addr 172.21.15.19:0/1215384056 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.414+0000 7f5e83400640 1 --2- 172.21.15.19:0/1215384056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e05764b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.414+0000 7f5e83e00640 1 --2- 172.21.15.19:0/1215384056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0573720 0x5587e05466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.414+0000 7f5e88c00640 1 -- 172.21.15.19:0/1215384056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 msgr2=0x5587e05764b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.414+0000 7f5e88c00640 1 --2- 172.21.15.19:0/1215384056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e05764b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.414+0000 7f5e88c00640 1 -- 172.21.15.19:0/1215384056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0573720 msgr2=0x5587e05466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.414+0000 7f5e88c00640 1 --2- 172.21.15.19:0/1215384056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0573720 0x5587e05466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.414+0000 7f5e88c00640 1 -- 172.21.15.19:0/1215384056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e028f7d0 con 0x5587e0546c00 2024-03-28T13:14:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.415+0000 7f5e88c00640 1 --2- 172.21.15.19:0/1215384056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0546c00 0x5587e05769f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00e230 tx=0x7f5e7c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7fd72ca813acad server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.415+0000 7f5e6fe00640 1 -- 172.21.15.19:0/1215384056 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c034a80 con 0x5587e0546c00 2024-03-28T13:14:56.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.415+0000 7f5e6fe00640 1 -- 172.21.15.19:0/1215384056 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e7c03a070 con 0x5587e0546c00 2024-03-28T13:14:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.415+0000 7f5e6fe00640 1 -- 172.21.15.19:0/1215384056 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c04d430 con 0x5587e0546c00 2024-03-28T13:14:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.416+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1215384056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0546c00 msgr2=0x5587e05769f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.416+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/1215384056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0546c00 0x5587e05769f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c00e230 tx=0x7f5e7c004ab0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.416+0000 7f5e88c00640 1 --2- 172.21.15.19:0/4234856979 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5e6406c530 0x7f5e6406e9f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c02c690 tx=0x7f5e7c03c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.416+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1215384056 shutdown_connections 2024-03-28T13:14:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.416+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/1215384056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0546c00 0x5587e05769f0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.416+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/1215384056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e05764b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.416+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/1215384056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0573720 0x5587e05466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.416+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1215384056 >> 172.21.15.19:0/1215384056 conn(0x5587e0548930 msgr2=0x5587e05914b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.416+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1215384056 shutdown_connections 2024-03-28T13:14:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.416+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1215384056 wait complete. 2024-03-28T13:14:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.417+0000 7f5e8d9d8740 1 -- start start 2024-03-28T13:14:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.417+0000 7f5e8d9d8740 1 Processor -- start 2024-03-28T13:14:56.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.418+0000 7f5e8d9d8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0573720 0x5587e05a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.418+0000 7f5e8d9d8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e058fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.418+0000 7f5e8d9d8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0590160 0x5587e0575660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.418+0000 7f5e8d9d8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5587e054afd0 con 0x5587e0573720 2024-03-28T13:14:56.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.418+0000 7f5e8d9d8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5587e046ca80 con 0x5587e0590160 2024-03-28T13:14:56.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.418+0000 7f5e8d9d8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5587e054d810 con 0x5587e0573b00 2024-03-28T13:14:56.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.418+0000 7f5e8d9d8740 10 client.0 _ll_register_callbacks cb 0x5587e046c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:14:56.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.418+0000 7f5e83e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0573720 0x5587e05a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.418+0000 7f5e83e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0573720 0x5587e05a13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35904/0 (socket says 172.21.15.19:35904) 2024-03-28T13:14:56.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.418+0000 7f5e83e00640 1 -- 172.21.15.19:0/1912348093 learned_addr learned my addr 172.21.15.19:0/1912348093 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:56.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.418+0000 7f5e88c00640 1 --2- 172.21.15.19:0/1912348093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0590160 0x5587e0575660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.418+0000 7f5e83400640 1 --2- 172.21.15.19:0/1912348093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e058fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.419+0000 7f5e83e00640 1 -- 172.21.15.19:0/1912348093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 msgr2=0x5587e058fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.419+0000 7f5e83e00640 1 --2- 172.21.15.19:0/1912348093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e058fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.419+0000 7f5e83e00640 1 -- 172.21.15.19:0/1912348093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0590160 msgr2=0x5587e0575660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.419+0000 7f5e83e00640 1 --2- 172.21.15.19:0/1912348093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0590160 0x5587e0575660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.419+0000 7f5e83e00640 1 -- 172.21.15.19:0/1912348093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e01c53f0 con 0x5587e0573720 2024-03-28T13:14:56.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.419+0000 7f5e88c00640 1 --2- 172.21.15.19:0/1912348093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0590160 0x5587e0575660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:56.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.419+0000 7f5e83e00640 1 --2- 172.21.15.19:0/1912348093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0573720 0x5587e05a13f0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009cf0 tx=0x7f5e74015730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f061b4197b80f26 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:56.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.419+0000 7f5e6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:14:56.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.419+0000 7f5e6cc00640 1 -- 172.21.15.19:0/1912348093 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7401b070 con 0x5587e0573720 2024-03-28T13:14:56.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.419+0000 7f5e6cc00640 1 -- 172.21.15.19:0/1912348093 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7f5e74020660 con 0x5587e0573720 2024-03-28T13:14:56.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.419+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1912348093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e7c03a070 con 0x5587e0573720 2024-03-28T13:14:56.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.420+0000 7f5e6cc00640 1 -- 172.21.15.19:0/1912348093 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5e74028050 con 0x5587e0573720 2024-03-28T13:14:56.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.420+0000 7f5e8d9d8740 10 client.6834 fetch_fsmap learned FSMap version 52 2024-03-28T13:14:56.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.420+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1912348093 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5587e01850f0 con 0x5587e0573720 2024-03-28T13:14:56.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.420+0000 7f5e6cc00640 1 -- 172.21.15.19:0/1912348093 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e7c03a070 con 0x5587e0573720 2024-03-28T13:14:56.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.420+0000 7f5e6cc00640 1 -- 172.21.15.19:0/1912348093 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f5e7402b020 con 0x5587e0573720 2024-03-28T13:14:56.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.420+0000 7f5e8d9d8740 10 client.6834 fetch_fsmap finished waiting for FSMap version 52 2024-03-28T13:14:56.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.420+0000 7f5e8d9d8740 10 client.6834 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-28T13:14:56.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.420+0000 7f5e8d9d8740 20 client.6834 populate_metadata read hostname 'smithi019' 2024-03-28T13:14:56.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.420+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/1912348093 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5587e0579040 0x5587e0579440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.420+0000 7f5e8d9d8740 4 client.6834 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:14:56.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.420+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1912348093 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587e04bebf0 con 0x5587e0579040 2024-03-28T13:14:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.421+0000 7f5e83400640 1 --2- 172.21.15.19:0/1912348093 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5587e0579040 0x5587e0579440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.421+0000 7f5e83400640 1 --2- 172.21.15.19:0/1912348093 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5587e0579040 0x5587e0579440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f31e4f64c9676db6 server_cookie=e99ea4b1b6263194 in_seq=0 out_seq=0 2024-03-28T13:14:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.421+0000 7f5e6cc00640 10 client.6834 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:14:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.422+0000 7f5e6cc00640 1 -- 172.21.15.19:0/1912348093 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5587e04bebf0 con 0x5587e0579040 2024-03-28T13:14:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.422+0000 7f5e6cc00640 10 client.6834 handle_command_reply: tid=0 2024-03-28T13:14:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.422+0000 7f5e8d9d8740 1 client.6834 shutdown 2024-03-28T13:14:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.422+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1912348093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0573720 msgr2=0x5587e05a13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.422+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/1912348093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0573720 0x5587e05a13f0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009cf0 tx=0x7f5e74015730 comp rx=0 tx=0).stop 2024-03-28T13:14:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.422+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1912348093 shutdown_connections 2024-03-28T13:14:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.422+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/1912348093 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5587e0579040 0x5587e0579440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.422+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/1912348093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0590160 0x5587e0575660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.422+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/1912348093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e058fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.422+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/1912348093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0573720 0x5587e05a13f0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.422+0000 7f5e6cc00640 0 client.6834 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:14:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.422+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1912348093 >> 172.21.15.19:0/1912348093 conn(0x5587e0546c00 msgr2=0x5587e0594930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:56.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.423+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1912348093 shutdown_connections 2024-03-28T13:14:56.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.423+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1912348093 wait complete. 2024-03-28T13:14:56.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.423+0000 7f5e8d9d8740 20 client.6834 trim_cache size 0 max 16384 2024-03-28T13:14:56.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.447+0000 7f5e8d9d8740 1 Processor -- start 2024-03-28T13:14:56.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.447+0000 7f5e8d9d8740 1 -- start start 2024-03-28T13:14:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.448+0000 7f5e8d9d8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0573720 0x5587e0573ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.448+0000 7f5e8d9d8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e05a1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.448+0000 7f5e8d9d8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0590160 0x5587e0594a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.448+0000 7f5e8d9d8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e84200d80 con 0x5587e0573720 2024-03-28T13:14:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.448+0000 7f5e8d9d8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5e841ffda0 con 0x5587e0590160 2024-03-28T13:14:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.448+0000 7f5e8d9d8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5e841ff1e0 con 0x5587e0573b00 2024-03-28T13:14:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.449+0000 7f5e83e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0573720 0x5587e0573ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.449+0000 7f5e83e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0573720 0x5587e0573ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35916/0 (socket says 172.21.15.19:35916) 2024-03-28T13:14:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.449+0000 7f5e83e00640 1 -- 172.21.15.19:0/1489293658 learned_addr learned my addr 172.21.15.19:0/1489293658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.449+0000 7f5e88c00640 1 --2- 172.21.15.19:0/1489293658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0590160 0x5587e0594a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.449+0000 7f5e83400640 1 --2- 172.21.15.19:0/1489293658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e05a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.449+0000 7f5e83e00640 1 -- 172.21.15.19:0/1489293658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 msgr2=0x5587e05a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.449+0000 7f5e83e00640 1 --2- 172.21.15.19:0/1489293658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e05a1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.449+0000 7f5e83e00640 1 -- 172.21.15.19:0/1489293658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0590160 msgr2=0x5587e0594a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.449+0000 7f5e83e00640 1 --2- 172.21.15.19:0/1489293658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0590160 0x5587e0594a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.449+0000 7f5e83e00640 1 -- 172.21.15.19:0/1489293658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e0574420 con 0x5587e0573720 2024-03-28T13:14:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.449+0000 7f5e83400640 1 --2- 172.21.15.19:0/1489293658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e05a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:14:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.450+0000 7f5e83e00640 1 --2- 172.21.15.19:0/1489293658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0573720 0x5587e0573ee0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5e74015cb0 tx=0x7f5e7400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5863a1c88897beb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.450+0000 7f5e6f400640 1 -- 172.21.15.19:0/1489293658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74030030 con 0x5587e0573720 2024-03-28T13:14:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.450+0000 7f5e6f400640 1 -- 172.21.15.19:0/1489293658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e74020460 con 0x5587e0573720 2024-03-28T13:14:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.450+0000 7f5e6f400640 1 -- 172.21.15.19:0/1489293658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7401b050 con 0x5587e0573720 2024-03-28T13:14:56.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.451+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1489293658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0573720 msgr2=0x5587e0573ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.451+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/1489293658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0573720 0x5587e0573ee0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5e74015cb0 tx=0x7f5e7400f000 comp rx=0 tx=0).stop 2024-03-28T13:14:56.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.451+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1489293658 shutdown_connections 2024-03-28T13:14:56.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.451+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/1489293658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0590160 0x5587e0594a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.451+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/1489293658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e05a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.451+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/1489293658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0573720 0x5587e0573ee0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.451+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1489293658 >> 172.21.15.19:0/1489293658 conn(0x5587e0546c00 msgr2=0x5587e0576790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:56.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.451+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1489293658 shutdown_connections 2024-03-28T13:14:56.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.451+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/1489293658 wait complete. 2024-03-28T13:14:56.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.452+0000 7f5e8d9d8740 1 -- start start 2024-03-28T13:14:56.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.452+0000 7f5e8d9d8740 1 Processor -- start 2024-03-28T13:14:56.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.453+0000 7f5e8d9d8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0573720 0x5587e05774b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.453+0000 7f5e8d9d8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e05fe230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.453+0000 7f5e8d9d8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0590160 0x5587e05b5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.453+0000 7f5e8d9d8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e841ff730 con 0x5587e0590160 2024-03-28T13:14:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.453+0000 7f5e8d9d8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5587e05755a0 con 0x5587e0573720 2024-03-28T13:14:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.453+0000 7f5e8d9d8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5587e0577ec0 con 0x5587e0573b00 2024-03-28T13:14:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.453+0000 7f5e8d9d8740 10 client.0 _ll_register_callbacks cb 0x7f5e740097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:14:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.454+0000 7f5e88c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0590160 0x5587e05b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.454+0000 7f5e88c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0590160 0x5587e05b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35920/0 (socket says 172.21.15.19:35920) 2024-03-28T13:14:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.454+0000 7f5e88c00640 1 -- 172.21.15.19:0/620645241 learned_addr learned my addr 172.21.15.19:0/620645241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:14:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.454+0000 7f5e83e00640 1 --2- 172.21.15.19:0/620645241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0573720 0x5587e05774b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.454+0000 7f5e83400640 1 --2- 172.21.15.19:0/620645241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e05fe230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.454+0000 7f5e88c00640 1 -- 172.21.15.19:0/620645241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 msgr2=0x5587e05fe230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.454+0000 7f5e88c00640 1 --2- 172.21.15.19:0/620645241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e05fe230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.454+0000 7f5e88c00640 1 -- 172.21.15.19:0/620645241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0573720 msgr2=0x5587e05774b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:14:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.454+0000 7f5e88c00640 1 --2- 172.21.15.19:0/620645241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0573720 0x5587e05774b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.454+0000 7f5e88c00640 1 -- 172.21.15.19:0/620645241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e05779f0 con 0x5587e0590160 2024-03-28T13:14:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.454+0000 7f5e88c00640 1 --2- 172.21.15.19:0/620645241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0590160 0x5587e05b5170 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009f90 tx=0x7f5e7c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14947902ecd7daa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:14:56.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.454+0000 7f5e6cc00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:14:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.455+0000 7f5e6cc00640 1 -- 172.21.15.19:0/620645241 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7c04d430 con 0x5587e0590160 2024-03-28T13:14:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.455+0000 7f5e6cc00640 1 -- 172.21.15.19:0/620645241 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(78..78 src has 1..78) ==== 5795+0+0 (secure 0 0 0) 0x7f5e7c061b20 con 0x5587e0590160 2024-03-28T13:14:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.455+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/620645241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e74020460 con 0x5587e0590160 2024-03-28T13:14:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.455+0000 7f5e6cc00640 1 -- 172.21.15.19:0/620645241 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5e7c061d20 con 0x5587e0590160 2024-03-28T13:14:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.455+0000 7f5e8d9d8740 10 client.6840 fetch_fsmap learned FSMap version 52 2024-03-28T13:14:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.455+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/620645241 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5587e0571e50 con 0x5587e0590160 2024-03-28T13:14:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.455+0000 7f5e6cc00640 1 -- 172.21.15.19:0/620645241 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e74020460 con 0x5587e0590160 2024-03-28T13:14:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.455+0000 7f5e6cc00640 1 -- 172.21.15.19:0/620645241 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f5e7c03f070 con 0x5587e0590160 2024-03-28T13:14:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.455+0000 7f5e8d9d8740 10 client.6840 fetch_fsmap finished waiting for FSMap version 52 2024-03-28T13:14:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.455+0000 7f5e8d9d8740 10 client.6840 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-28T13:14:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.455+0000 7f5e8d9d8740 20 client.6840 populate_metadata read hostname 'smithi019' 2024-03-28T13:14:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.455+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/620645241 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5587e0602ce0 0x5587e05643c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:14:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.456+0000 7f5e8d9d8740 4 client.6840 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:14:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.456+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/620645241 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5587e04b8f40 con 0x5587e0602ce0 2024-03-28T13:14:56.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.456+0000 7f5e83e00640 1 --2- 172.21.15.19:0/620645241 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5587e0602ce0 0x5587e05643c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:14:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.456+0000 7f5e83e00640 1 --2- 172.21.15.19:0/620645241 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5587e0602ce0 0x5587e05643c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c19ad20a3060825 server_cookie=ad7bcaecfea2aea6 in_seq=0 out_seq=0 2024-03-28T13:14:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.456+0000 7f5e6cc00640 10 client.6840 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:14:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.457+0000 7f5e6cc00640 1 -- 172.21.15.19:0/620645241 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5587e04b8f40 con 0x5587e0602ce0 2024-03-28T13:14:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.457+0000 7f5e6cc00640 10 client.6840 handle_command_reply: tid=0 2024-03-28T13:14:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.457+0000 7f5e8d9d8740 1 client.6840 shutdown 2024-03-28T13:14:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.457+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/620645241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0590160 msgr2=0x5587e05b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.457+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/620645241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0590160 0x5587e05b5170 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009f90 tx=0x7f5e7c00b040 comp rx=0 tx=0).stop 2024-03-28T13:14:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.457+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/620645241 shutdown_connections 2024-03-28T13:14:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.457+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/620645241 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5587e0602ce0 0x5587e05643c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.457+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/620645241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5587e0590160 0x5587e05b5170 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.457+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/620645241 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5587e0573b00 0x5587e05fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.457+0000 7f5e8d9d8740 1 --2- 172.21.15.19:0/620645241 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5587e0573720 0x5587e05774b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.457+0000 7f5e6cc00640 0 client.6840 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:14:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.457+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/620645241 >> 172.21.15.19:0/620645241 conn(0x5587e0546c00 msgr2=0x5587e05a4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.457+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/620645241 shutdown_connections 2024-03-28T13:14:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.457+0000 7f5e8d9d8740 1 -- 172.21.15.19:0/620645241 wait complete. 2024-03-28T13:14:56.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.457+0000 7f5e8d9d8740 20 client.6840 trim_cache size 0 max 16384 2024-03-28T13:14:56.460 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:14:56.460 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T13:14:56.460 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-03-28T13:14:56.460 INFO:teuthology.orchestra.run.smithi019.stdout: "0d5f4cdc-5ce6-4d4a-9a5f-04f110eb5c56": { 2024-03-28T13:14:56.460 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/subdir", 2024-03-28T13:14:56.461 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "0d5f4cdc-5ce6-4d4a-9a5f-04f110eb5c56", 2024-03-28T13:14:56.461 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive" 2024-03-28T13:14:56.461 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:56.461 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:14:56.461 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:14:56.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.458+0000 7f5e89600640 1 -- 172.21.15.19:0/4234856979 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5e6406c530 msgr2=0x7f5e6406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.458+0000 7f5e89600640 1 --2- 172.21.15.19:0/4234856979 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5e6406c530 0x7f5e6406e9f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c02c690 tx=0x7f5e7c03c000 comp rx=0 tx=0).stop 2024-03-28T13:14:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.458+0000 7f5e89600640 1 -- 172.21.15.19:0/4234856979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8414adf0 msgr2=0x7f5e840ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:14:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.458+0000 7f5e89600640 1 --2- 172.21.15.19:0/4234856979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8414adf0 0x7f5e840ad890 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000d910 tx=0x7f5e7000ddd0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.458+0000 7f5e89600640 1 -- 172.21.15.19:0/4234856979 shutdown_connections 2024-03-28T13:14:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.458+0000 7f5e89600640 1 --2- 172.21.15.19:0/4234856979 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5e6406c530 0x7f5e6406e9f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.458+0000 7f5e89600640 1 --2- 172.21.15.19:0/4234856979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e840addd0 0x7f5e84153640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.458+0000 7f5e89600640 1 --2- 172.21.15.19:0/4234856979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e8414adf0 0x7f5e840ad890 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.458+0000 7f5e89600640 1 --2- 172.21.15.19:0/4234856979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e68004d40 0x7f5e8414ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:14:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.458+0000 7f5e89600640 1 -- 172.21.15.19:0/4234856979 >> 172.21.15.19:0/4234856979 conn(0x7f5e840d0770 msgr2=0x7f5e840b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:14:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.458+0000 7f5e89600640 1 -- 172.21.15.19:0/4234856979 shutdown_connections 2024-03-28T13:14:56.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:14:56.458+0000 7f5e89600640 1 -- 172.21.15.19:0/4234856979 wait complete. 2024-03-28T13:14:56.477 INFO:tasks.cephfs.filesystem:scrub status for tag:0d5f4cdc-5ce6-4d4a-9a5f-04f110eb5c56 - {'path': '/subdir', 'tag': '0d5f4cdc-5ce6-4d4a-9a5f-04f110eb5c56', 'options': 'recursive'} 2024-03-28T13:15:02.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:15:02.604 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:15:26.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:15:26.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-28T13:15:26.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.705+0000 7f8752400640 1 Processor -- start 2024-03-28T13:15:26.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.705+0000 7f8752400640 1 -- start start 2024-03-28T13:15:26.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.707+0000 7f8752400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f874c15e860 0x7f874c06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.707+0000 7f8752400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f874c06dfa0 con 0x7f874c06da30 2024-03-28T13:15:26.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.707+0000 7f8752400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f874c06e240 con 0x7f874c0d4ae0 2024-03-28T13:15:26.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.707+0000 7f8752400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f874c0cf220 con 0x7f874c15e860 2024-03-28T13:15:26.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.707+0000 7f874b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f874c15e860 0x7f874c06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.707+0000 7f874aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f874c06da30 0x7f874c0d43d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43784/0 (socket says 172.21.15.19:43784) 2024-03-28T13:15:26.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.707+0000 7f874aa00640 1 -- 172.21.15.19:0/248244687 learned_addr learned my addr 172.21.15.19:0/248244687 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:15:26.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.708+0000 7f874b400640 1 -- 172.21.15.19:0/248244687 >> v1:172.21.15.188:6789/0 conn(0x7f874c0d4ae0 legacy=0x7f874c0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:26.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.708+0000 7f874b400640 1 -- 172.21.15.19:0/248244687 >> v1:172.21.15.19:6789/0 conn(0x7f874c06da30 legacy=0x7f874c0d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:26.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.708+0000 7f874b400640 1 -- 172.21.15.19:0/248244687 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874c06e520 con 0x7f874c15e860 2024-03-28T13:15:26.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.710+0000 7f874b400640 1 --2- 172.21.15.19:0/248244687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874c15e860 0x7f874c06d4f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f873c0097a0 tx=0x7f873c02b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4f4414d4c1c0eb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:26.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.710+0000 7f874a000640 1 -- 172.21.15.19:0/248244687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c037070 con 0x7f874c15e860 2024-03-28T13:15:26.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.710+0000 7f874a000640 1 -- 172.21.15.19:0/248244687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f873c030710 con 0x7f874c15e860 2024-03-28T13:15:26.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.710+0000 7f874a000640 1 -- 172.21.15.19:0/248244687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c030ae0 con 0x7f874c15e860 2024-03-28T13:15:26.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.711+0000 7f8752400640 1 -- 172.21.15.19:0/248244687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874c15e860 msgr2=0x7f874c06d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:26.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.711+0000 7f8752400640 1 --2- 172.21.15.19:0/248244687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874c15e860 0x7f874c06d4f0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f873c0097a0 tx=0x7f873c02b710 comp rx=0 tx=0).stop 2024-03-28T13:15:26.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.711+0000 7f8752400640 1 -- 172.21.15.19:0/248244687 shutdown_connections 2024-03-28T13:15:26.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.711+0000 7f8752400640 1 --2- 172.21.15.19:0/248244687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874c15e860 0x7f874c06d4f0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.711+0000 7f8752400640 1 -- 172.21.15.19:0/248244687 >> 172.21.15.19:0/248244687 conn(0x7f874c09d7d0 msgr2=0x7f874c08bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:26.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.711+0000 7f8752400640 1 -- 172.21.15.19:0/248244687 shutdown_connections 2024-03-28T13:15:26.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.711+0000 7f8752400640 1 -- 172.21.15.19:0/248244687 wait complete. 2024-03-28T13:15:26.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.713+0000 7f8752400640 1 Processor -- start 2024-03-28T13:15:26.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.713+0000 7f8752400640 1 -- start start 2024-03-28T13:15:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.714+0000 7f8752400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c06da30 0x7f874c157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.714+0000 7f8752400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c0d4ae0 0x7f874c0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.714+0000 7f8752400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874c15e860 0x7f874c0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.714+0000 7f8752400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f874c0cef30 con 0x7f874c15e860 2024-03-28T13:15:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.714+0000 7f8752400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f874c0cedb0 con 0x7f874c0d4ae0 2024-03-28T13:15:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.714+0000 7f8752400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f874c0cfdc0 con 0x7f874c06da30 2024-03-28T13:15:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.714+0000 7f874be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874c15e860 0x7f874c0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.714+0000 7f874be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874c15e860 0x7f874c0cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59112/0 (socket says 172.21.15.19:59112) 2024-03-28T13:15:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.714+0000 7f874be00640 1 -- 172.21.15.19:0/1940020143 learned_addr learned my addr 172.21.15.19:0/1940020143 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.715+0000 7f874b400640 1 --2- 172.21.15.19:0/1940020143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c06da30 0x7f874c157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.715+0000 7f874aa00640 1 --2- 172.21.15.19:0/1940020143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c0d4ae0 0x7f874c0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.715+0000 7f874be00640 1 -- 172.21.15.19:0/1940020143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c06da30 msgr2=0x7f874c157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:26.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.715+0000 7f874be00640 1 --2- 172.21.15.19:0/1940020143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c06da30 0x7f874c157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.715+0000 7f874be00640 1 -- 172.21.15.19:0/1940020143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c0d4ae0 msgr2=0x7f874c0cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:26.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.715+0000 7f874be00640 1 --2- 172.21.15.19:0/1940020143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c0d4ae0 0x7f874c0cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.715+0000 7f874be00640 1 -- 172.21.15.19:0/1940020143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874c0b7170 con 0x7f874c15e860 2024-03-28T13:15:26.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.715+0000 7f874be00640 1 --2- 172.21.15.19:0/1940020143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874c15e860 0x7f874c0cce50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f874400c780 tx=0x7f874400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:26.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.715+0000 7f8743400640 1 -- 172.21.15.19:0/1940020143 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744004b60 con 0x7f874c15e860 2024-03-28T13:15:26.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.716+0000 7f8743400640 1 -- 172.21.15.19:0/1940020143 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8744004d00 con 0x7f874c15e860 2024-03-28T13:15:26.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.716+0000 7f8752400640 1 -- 172.21.15.19:0/1940020143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874c0b7450 con 0x7f874c15e860 2024-03-28T13:15:26.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.716+0000 7f8743400640 1 -- 172.21.15.19:0/1940020143 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874400aa90 con 0x7f874c15e860 2024-03-28T13:15:26.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.716+0000 7f8752400640 1 -- 172.21.15.19:0/1940020143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f874c0b7a10 con 0x7f874c15e860 2024-03-28T13:15:26.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.718+0000 7f8743400640 1 -- 172.21.15.19:0/1940020143 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f874400ac30 con 0x7f874c15e860 2024-03-28T13:15:26.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.719+0000 7f8743400640 1 --2- 172.21.15.19:0/1940020143 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f872c06c530 0x7f872c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.719+0000 7f8743400640 1 -- 172.21.15.19:0/1940020143 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f874408ba60 con 0x7f874c15e860 2024-03-28T13:15:26.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.720+0000 7f874b400640 1 --2- 172.21.15.19:0/1940020143 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f872c06c530 0x7f872c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.721+0000 7f874b400640 1 --2- 172.21.15.19:0/1940020143 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f872c06c530 0x7f872c06e9f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f873c034000 tx=0x7f873c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:26.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.722+0000 7f8755182740 1 Processor -- start 2024-03-28T13:15:26.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.722+0000 7f8755182740 1 -- start start 2024-03-28T13:15:26.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.723+0000 7f8755182740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d3560 0x5567b82a6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.723+0000 7f8755182740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d3940 0x5567b82d62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.723+0000 7f8755182740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82a6a40 0x5567b82d6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.723+0000 7f8755182740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567b82116b0 con 0x5567b82d3560 2024-03-28T13:15:26.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.723+0000 7f8755182740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5567b81b9750 con 0x5567b82a6a40 2024-03-28T13:15:26.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.723+0000 7f8755182740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5567b820d660 con 0x5567b82d3940 2024-03-28T13:15:26.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.724+0000 7f874b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d3560 0x5567b82a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.724+0000 7f874b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d3560 0x5567b82a6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59120/0 (socket says 172.21.15.19:59120) 2024-03-28T13:15:26.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.724+0000 7f874b400640 1 -- 172.21.15.19:0/1371319921 learned_addr learned my addr 172.21.15.19:0/1371319921 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:26.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.724+0000 7f874aa00640 1 --2- 172.21.15.19:0/1371319921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d3940 0x5567b82d62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.724+0000 7f874be00640 1 --2- 172.21.15.19:0/1371319921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82a6a40 0x5567b82d6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.724+0000 7f874b400640 1 -- 172.21.15.19:0/1371319921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d3940 msgr2=0x5567b82d62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:26.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.724+0000 7f874b400640 1 --2- 172.21.15.19:0/1371319921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d3940 0x5567b82d62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.724+0000 7f874b400640 1 -- 172.21.15.19:0/1371319921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82a6a40 msgr2=0x5567b82d6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:26.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.724+0000 7f874b400640 1 --2- 172.21.15.19:0/1371319921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82a6a40 0x5567b82d6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.724+0000 7f874b400640 1 -- 172.21.15.19:0/1371319921 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b7fef7d0 con 0x5567b82d3560 2024-03-28T13:15:26.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.725+0000 7f874b400640 1 --2- 172.21.15.19:0/1371319921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d3560 0x5567b82a6500 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f873c02bd20 tx=0x7f873c02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bd613dc5833d1d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:26.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.725+0000 7f8737e00640 1 -- 172.21.15.19:0/1371319921 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c04c070 con 0x5567b82d3560 2024-03-28T13:15:26.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.725+0000 7f8737e00640 1 -- 172.21.15.19:0/1371319921 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f873c051460 con 0x5567b82d3560 2024-03-28T13:15:26.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.725+0000 7f8737e00640 1 -- 172.21.15.19:0/1371319921 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c037040 con 0x5567b82d3560 2024-03-28T13:15:26.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.726+0000 7f8755182740 1 -- 172.21.15.19:0/1371319921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d3560 msgr2=0x5567b82a6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:26.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.726+0000 7f8755182740 1 --2- 172.21.15.19:0/1371319921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d3560 0x5567b82a6500 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f873c02bd20 tx=0x7f873c02f000 comp rx=0 tx=0).stop 2024-03-28T13:15:26.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.726+0000 7f8755182740 1 -- 172.21.15.19:0/1371319921 shutdown_connections 2024-03-28T13:15:26.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.726+0000 7f8755182740 1 --2- 172.21.15.19:0/1371319921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82a6a40 0x5567b82d6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.726+0000 7f8755182740 1 --2- 172.21.15.19:0/1371319921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d3940 0x5567b82d62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.726+0000 7f8755182740 1 --2- 172.21.15.19:0/1371319921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d3560 0x5567b82a6500 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.726+0000 7f8755182740 1 -- 172.21.15.19:0/1371319921 >> 172.21.15.19:0/1371319921 conn(0x5567b82a8770 msgr2=0x5567b82f12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:26.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.726+0000 7f8755182740 1 -- 172.21.15.19:0/1371319921 shutdown_connections 2024-03-28T13:15:26.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.726+0000 7f8755182740 1 -- 172.21.15.19:0/1371319921 wait complete. 2024-03-28T13:15:26.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.727+0000 7f8755182740 1 -- start start 2024-03-28T13:15:26.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.727+0000 7f8755182740 1 Processor -- start 2024-03-28T13:15:26.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.728+0000 7f8755182740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82d3560 0x5567b82d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.728+0000 7f8755182740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d3940 0x5567b8309be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.728+0000 7f8755182740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d8b90 0x5567b82d9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.728+0000 7f8755182740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567b82aae10 con 0x5567b82d8b90 2024-03-28T13:15:26.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.728+0000 7f8755182740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5567b81cca80 con 0x5567b82d3560 2024-03-28T13:15:26.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.728+0000 7f8755182740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5567b82ad650 con 0x5567b82d3940 2024-03-28T13:15:26.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.728+0000 7f8755182740 10 client.0 _ll_register_callbacks cb 0x5567b81cc470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:15:26.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.728+0000 7f874be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d8b90 0x5567b82d9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.728+0000 7f874be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d8b90 0x5567b82d9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59128/0 (socket says 172.21.15.19:59128) 2024-03-28T13:15:26.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.728+0000 7f874be00640 1 -- 172.21.15.19:0/346170080 learned_addr learned my addr 172.21.15.19:0/346170080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:26.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.729+0000 7f874b400640 1 --2- 172.21.15.19:0/346170080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82d3560 0x5567b82d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.729+0000 7f874aa00640 1 --2- 172.21.15.19:0/346170080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d3940 0x5567b8309be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.729+0000 7f874be00640 1 -- 172.21.15.19:0/346170080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d3940 msgr2=0x5567b8309be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:26.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.729+0000 7f874be00640 1 --2- 172.21.15.19:0/346170080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d3940 0x5567b8309be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.729+0000 7f874be00640 1 -- 172.21.15.19:0/346170080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82d3560 msgr2=0x5567b82d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:26.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.729+0000 7f874be00640 1 --2- 172.21.15.19:0/346170080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82d3560 0x5567b82d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.729+0000 7f874be00640 1 -- 172.21.15.19:0/346170080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b7f253f0 con 0x5567b82d8b90 2024-03-28T13:15:26.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.729+0000 7f874aa00640 1 --2- 172.21.15.19:0/346170080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d3940 0x5567b8309be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:26.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.729+0000 7f874be00640 1 --2- 172.21.15.19:0/346170080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d8b90 0x5567b82d9010 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8744089e30 tx=0x7f874405a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa49dbdd24dfb966 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:26.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.729+0000 7f8734c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:15:26.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.729+0000 7f8734c00640 1 -- 172.21.15.19:0/346170080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87440624c0 con 0x5567b82d8b90 2024-03-28T13:15:26.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.729+0000 7f8734c00640 1 -- 172.21.15.19:0/346170080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f87440600b0 con 0x5567b82d8b90 2024-03-28T13:15:26.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.729+0000 7f8755182740 1 -- 172.21.15.19:0/346170080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f873c051460 con 0x5567b82d8b90 2024-03-28T13:15:26.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.730+0000 7f8734c00640 1 -- 172.21.15.19:0/346170080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8744078050 con 0x5567b82d8b90 2024-03-28T13:15:26.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.730+0000 7f8755182740 10 client.6855 fetch_fsmap learned FSMap version 52 2024-03-28T13:15:26.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.730+0000 7f8755182740 1 -- 172.21.15.19:0/346170080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5567b7ee50f0 con 0x5567b82d8b90 2024-03-28T13:15:26.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.730+0000 7f8734c00640 1 -- 172.21.15.19:0/346170080 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f873c051460 con 0x5567b82d8b90 2024-03-28T13:15:26.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.730+0000 7f8734c00640 1 -- 172.21.15.19:0/346170080 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f8744055f50 con 0x5567b82d8b90 2024-03-28T13:15:26.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.730+0000 7f8755182740 10 client.6855 fetch_fsmap finished waiting for FSMap version 52 2024-03-28T13:15:26.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.730+0000 7f8755182740 10 client.6855 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-28T13:15:26.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.730+0000 7f8755182740 20 client.6855 populate_metadata read hostname 'smithi019' 2024-03-28T13:15:26.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.730+0000 7f8755182740 1 --2- 172.21.15.19:0/346170080 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5567b830bc80 0x5567b830c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.730+0000 7f8755182740 4 client.6855 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:15:26.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.730+0000 7f8755182740 1 -- 172.21.15.19:0/346170080 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567b821ebf0 con 0x5567b830bc80 2024-03-28T13:15:26.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.730+0000 7f874b400640 1 --2- 172.21.15.19:0/346170080 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5567b830bc80 0x5567b830c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.731+0000 7f874b400640 1 --2- 172.21.15.19:0/346170080 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5567b830bc80 0x5567b830c090 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=831241743ac79985 server_cookie=efa5bbb814fb3324 in_seq=0 out_seq=0 2024-03-28T13:15:26.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.731+0000 7f8734c00640 10 client.6855 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:26.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.732+0000 7f8734c00640 1 -- 172.21.15.19:0/346170080 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567b821ebf0 con 0x5567b830bc80 2024-03-28T13:15:26.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.732+0000 7f8734c00640 10 client.6855 handle_command_reply: tid=0 2024-03-28T13:15:26.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.732+0000 7f8755182740 1 client.6855 shutdown 2024-03-28T13:15:26.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.732+0000 7f8755182740 1 -- 172.21.15.19:0/346170080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d8b90 msgr2=0x5567b82d9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:26.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.732+0000 7f8755182740 1 --2- 172.21.15.19:0/346170080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d8b90 0x5567b82d9010 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8744089e30 tx=0x7f874405a0a0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.732+0000 7f8755182740 1 -- 172.21.15.19:0/346170080 shutdown_connections 2024-03-28T13:15:26.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.732+0000 7f8755182740 1 --2- 172.21.15.19:0/346170080 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5567b830bc80 0x5567b830c090 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.732+0000 7f8755182740 1 --2- 172.21.15.19:0/346170080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d8b90 0x5567b82d9010 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.732+0000 7f8755182740 1 --2- 172.21.15.19:0/346170080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d3940 0x5567b8309be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.732+0000 7f8755182740 1 --2- 172.21.15.19:0/346170080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82d3560 0x5567b82d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.732+0000 7f8755182740 1 -- 172.21.15.19:0/346170080 >> 172.21.15.19:0/346170080 conn(0x5567b82a6a40 msgr2=0x5567b82aa4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.732+0000 7f8734c00640 0 client.6855 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.733+0000 7f8755182740 1 -- 172.21.15.19:0/346170080 shutdown_connections 2024-03-28T13:15:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.733+0000 7f8755182740 1 -- 172.21.15.19:0/346170080 wait complete. 2024-03-28T13:15:26.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.733+0000 7f8755182740 20 client.6855 trim_cache size 0 max 16384 2024-03-28T13:15:26.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.769+0000 7f8755182740 1 Processor -- start 2024-03-28T13:15:26.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.769+0000 7f8755182740 1 -- start start 2024-03-28T13:15:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.770+0000 7f8755182740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d3560 0x5567b82d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.770+0000 7f8755182740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82d3940 0x5567b82f5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.770+0000 7f8755182740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d8b90 0x5567b82f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.770+0000 7f8755182740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f874c070e70 con 0x5567b82d3560 2024-03-28T13:15:26.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.770+0000 7f8755182740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f874c06fed0 con 0x5567b82d3940 2024-03-28T13:15:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.770+0000 7f8755182740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f874c160640 con 0x5567b82d8b90 2024-03-28T13:15:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.770+0000 7f874b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d3560 0x5567b82d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.771+0000 7f874b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d3560 0x5567b82d1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59134/0 (socket says 172.21.15.19:59134) 2024-03-28T13:15:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.771+0000 7f874b400640 1 -- 172.21.15.19:0/69218842 learned_addr learned my addr 172.21.15.19:0/69218842 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:26.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.771+0000 7f874b400640 1 -- 172.21.15.19:0/69218842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d8b90 msgr2=0x5567b82f5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.771+0000 7f874b400640 1 --2- 172.21.15.19:0/69218842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d8b90 0x5567b82f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.771+0000 7f874b400640 1 -- 172.21.15.19:0/69218842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82d3940 msgr2=0x5567b82f5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.771+0000 7f874b400640 1 --2- 172.21.15.19:0/69218842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82d3940 0x5567b82f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.771+0000 7f874b400640 1 -- 172.21.15.19:0/69218842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b82f7ab0 con 0x5567b82d3560 2024-03-28T13:15:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.771+0000 7f874b400640 1 --2- 172.21.15.19:0/69218842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d3560 0x5567b82d1ec0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f873c02b750 tx=0x7f873c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e982eb89fe63758d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.771+0000 7f8737400640 1 -- 172.21.15.19:0/69218842 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c040030 con 0x5567b82d3560 2024-03-28T13:15:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.772+0000 7f8737400640 1 -- 172.21.15.19:0/69218842 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f873c004240 con 0x5567b82d3560 2024-03-28T13:15:26.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.772+0000 7f8737400640 1 -- 172.21.15.19:0/69218842 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c037070 con 0x5567b82d3560 2024-03-28T13:15:26.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.772+0000 7f8755182740 1 -- 172.21.15.19:0/69218842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d3560 msgr2=0x5567b82d1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:26.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.772+0000 7f8755182740 1 --2- 172.21.15.19:0/69218842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d3560 0x5567b82d1ec0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f873c02b750 tx=0x7f873c03d000 comp rx=0 tx=0).stop 2024-03-28T13:15:26.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.772+0000 7f8755182740 1 -- 172.21.15.19:0/69218842 shutdown_connections 2024-03-28T13:15:26.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.772+0000 7f8755182740 1 --2- 172.21.15.19:0/69218842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d8b90 0x5567b82f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.772+0000 7f8755182740 1 --2- 172.21.15.19:0/69218842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82d3940 0x5567b82f5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.772+0000 7f8755182740 1 --2- 172.21.15.19:0/69218842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b82d3560 0x5567b82d1ec0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.772+0000 7f8755182740 1 -- 172.21.15.19:0/69218842 >> 172.21.15.19:0/69218842 conn(0x5567b82a6a40 msgr2=0x5567b82d2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:26.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.772+0000 7f8755182740 1 -- 172.21.15.19:0/69218842 shutdown_connections 2024-03-28T13:15:26.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.772+0000 7f8755182740 1 -- 172.21.15.19:0/69218842 wait complete. 2024-03-28T13:15:26.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.773+0000 7f8755182740 1 -- start start 2024-03-28T13:15:26.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.773+0000 7f8755182740 1 Processor -- start 2024-03-28T13:15:26.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.774+0000 7f8755182740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d3940 0x5567b82f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.774+0000 7f8755182740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82d8b90 0x5567b82f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.774+0000 7f8755182740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8376620 0x5567b83115b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.774+0000 7f8755182740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567b82f7060 con 0x5567b8376620 2024-03-28T13:15:26.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.774+0000 7f8755182740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f874c1f8560 con 0x5567b82d8b90 2024-03-28T13:15:26.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.774+0000 7f8755182740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5567b82fd0f0 con 0x5567b82d3940 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.774+0000 7f8755182740 10 client.0 _ll_register_callbacks cb 0x7f87440594b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.775+0000 7f874be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8376620 0x5567b83115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.775+0000 7f874be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8376620 0x5567b83115b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59144/0 (socket says 172.21.15.19:59144) 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.775+0000 7f874be00640 1 -- 172.21.15.19:0/2313478275 learned_addr learned my addr 172.21.15.19:0/2313478275 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.775+0000 7f874aa00640 1 --2- 172.21.15.19:0/2313478275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82d8b90 0x5567b82f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.775+0000 7f874be00640 1 -- 172.21.15.19:0/2313478275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d3940 msgr2=0x5567b82f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.775+0000 7f874be00640 1 --2- 172.21.15.19:0/2313478275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d3940 0x5567b82f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.775+0000 7f874be00640 1 -- 172.21.15.19:0/2313478275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82d8b90 msgr2=0x5567b82f4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.775+0000 7f874be00640 1 --2- 172.21.15.19:0/2313478275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82d8b90 0x5567b82f4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.775+0000 7f874be00640 1 -- 172.21.15.19:0/2313478275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b82f1da0 con 0x5567b8376620 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.775+0000 7f874aa00640 1 --2- 172.21.15.19:0/2313478275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82d8b90 0x5567b82f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.775+0000 7f874be00640 1 --2- 172.21.15.19:0/2313478275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8376620 0x5567b83115b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8744060010 tx=0x7f8744007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfba1a2ba743502 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.775+0000 7f8734c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.775+0000 7f8734c00640 1 -- 172.21.15.19:0/2313478275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87440624c0 con 0x5567b8376620 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.776+0000 7f8734c00640 1 -- 172.21.15.19:0/2313478275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f8744006b50 con 0x5567b8376620 2024-03-28T13:15:26.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.776+0000 7f8755182740 1 -- 172.21.15.19:0/2313478275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f873c004240 con 0x5567b8376620 2024-03-28T13:15:26.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.776+0000 7f8734c00640 1 -- 172.21.15.19:0/2313478275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f8744068050 con 0x5567b8376620 2024-03-28T13:15:26.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.776+0000 7f8755182740 10 client.6861 fetch_fsmap learned FSMap version 52 2024-03-28T13:15:26.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.776+0000 7f8755182740 1 -- 172.21.15.19:0/2313478275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5567b82a6280 con 0x5567b8376620 2024-03-28T13:15:26.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.776+0000 7f8734c00640 1 -- 172.21.15.19:0/2313478275 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f873c004240 con 0x5567b8376620 2024-03-28T13:15:26.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.776+0000 7f8734c00640 1 -- 172.21.15.19:0/2313478275 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f874406b020 con 0x5567b8376620 2024-03-28T13:15:26.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.776+0000 7f8755182740 10 client.6861 fetch_fsmap finished waiting for FSMap version 52 2024-03-28T13:15:26.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.776+0000 7f8755182740 10 client.6861 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-28T13:15:26.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.776+0000 7f8755182740 20 client.6861 populate_metadata read hostname 'smithi019' 2024-03-28T13:15:26.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.776+0000 7f8755182740 1 --2- 172.21.15.19:0/2313478275 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5567b830c060 0x5567b8311af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:26.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.776+0000 7f8755182740 4 client.6861 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:15:26.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.776+0000 7f8755182740 1 -- 172.21.15.19:0/2313478275 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5567b8218f40 con 0x5567b830c060 2024-03-28T13:15:26.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.777+0000 7f874b400640 1 --2- 172.21.15.19:0/2313478275 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5567b830c060 0x5567b8311af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:26.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.777+0000 7f874b400640 1 --2- 172.21.15.19:0/2313478275 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5567b830c060 0x5567b8311af0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbbef361bbc0f239 server_cookie=772a24fd031dea9c in_seq=0 out_seq=0 2024-03-28T13:15:26.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.777+0000 7f8734c00640 10 client.6861 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:26.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.777+0000 7f8734c00640 1 -- 172.21.15.19:0/2313478275 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5567b8218f40 con 0x5567b830c060 2024-03-28T13:15:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.777+0000 7f8734c00640 10 client.6861 handle_command_reply: tid=0 2024-03-28T13:15:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.777+0000 7f8755182740 1 client.6861 shutdown 2024-03-28T13:15:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.778+0000 7f8755182740 1 -- 172.21.15.19:0/2313478275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8376620 msgr2=0x5567b83115b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.778+0000 7f8755182740 1 --2- 172.21.15.19:0/2313478275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8376620 0x5567b83115b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f8744060010 tx=0x7f8744007f50 comp rx=0 tx=0).stop 2024-03-28T13:15:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.778+0000 7f8755182740 1 -- 172.21.15.19:0/2313478275 shutdown_connections 2024-03-28T13:15:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.778+0000 7f8755182740 1 --2- 172.21.15.19:0/2313478275 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x5567b830c060 0x5567b8311af0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.778+0000 7f8755182740 1 --2- 172.21.15.19:0/2313478275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567b8376620 0x5567b83115b0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.778+0000 7f8755182740 1 --2- 172.21.15.19:0/2313478275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5567b82d8b90 0x5567b82f4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.778+0000 7f8755182740 1 --2- 172.21.15.19:0/2313478275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5567b82d3940 0x5567b82f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.778+0000 7f8734c00640 0 client.6861 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.778+0000 7f8755182740 1 -- 172.21.15.19:0/2313478275 >> 172.21.15.19:0/2313478275 conn(0x5567b82a6a40 msgr2=0x5567b82d6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.778+0000 7f8755182740 1 -- 172.21.15.19:0/2313478275 shutdown_connections 2024-03-28T13:15:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.778+0000 7f8755182740 1 -- 172.21.15.19:0/2313478275 wait complete. 2024-03-28T13:15:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.778+0000 7f8755182740 20 client.6861 trim_cache size 0 max 16384 2024-03-28T13:15:26.781 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:15:26.781 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-03-28T13:15:26.781 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-03-28T13:15:26.781 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:15:26.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.779+0000 7f8752400640 1 -- 172.21.15.19:0/1940020143 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f872c06c530 msgr2=0x7f872c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.779+0000 7f8752400640 1 --2- 172.21.15.19:0/1940020143 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f872c06c530 0x7f872c06e9f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f873c034000 tx=0x7f873c008030 comp rx=0 tx=0).stop 2024-03-28T13:15:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.779+0000 7f8752400640 1 -- 172.21.15.19:0/1940020143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874c15e860 msgr2=0x7f874c0cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.779+0000 7f8752400640 1 --2- 172.21.15.19:0/1940020143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874c15e860 0x7f874c0cce50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f874400c780 tx=0x7f874400cc40 comp rx=0 tx=0).stop 2024-03-28T13:15:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.779+0000 7f8752400640 1 -- 172.21.15.19:0/1940020143 shutdown_connections 2024-03-28T13:15:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.779+0000 7f8752400640 1 --2- 172.21.15.19:0/1940020143 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f872c06c530 0x7f872c06e9f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.779+0000 7f8752400640 1 --2- 172.21.15.19:0/1940020143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f874c15e860 0x7f874c0cce50 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.779+0000 7f8752400640 1 --2- 172.21.15.19:0/1940020143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f874c0d4ae0 0x7f874c0cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.779+0000 7f8752400640 1 --2- 172.21.15.19:0/1940020143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f874c06da30 0x7f874c157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.779+0000 7f8752400640 1 -- 172.21.15.19:0/1940020143 >> 172.21.15.19:0/1940020143 conn(0x7f874c09d7d0 msgr2=0x7f874c0cf6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.779+0000 7f8752400640 1 -- 172.21.15.19:0/1940020143 shutdown_connections 2024-03-28T13:15:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:26.779+0000 7f8752400640 1 -- 172.21.15.19:0/1940020143 wait complete. 2024-03-28T13:15:26.797 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T13:15:26.797 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:15:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.005+0000 7f0264c00640 1 Processor -- start 2024-03-28T13:15:27.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.006+0000 7f0264c00640 1 -- start start 2024-03-28T13:15:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.007+0000 7f0264c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015c390 0x7f0260073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.007+0000 7f0264c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f02600d0650 0x7f02600d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.007+0000 7f0264c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f026015fe80 con 0x7f02600af830 2024-03-28T13:15:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.007+0000 7f0264c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f02600d1000 con 0x7f02600d0650 2024-03-28T13:15:27.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.007+0000 7f0264c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f026015c770 con 0x7f026015c390 2024-03-28T13:15:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.008+0000 7f025de00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015c390 0x7f0260073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.008+0000 7f025e800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f02600d0650 0x7f02600d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.008+0000 7f025de00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015c390 0x7f0260073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:42422/0 (socket says 172.21.15.19:42422) 2024-03-28T13:15:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.008+0000 7f025de00640 1 -- 172.21.15.19:0/1668622218 learned_addr learned my addr 172.21.15.19:0/1668622218 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.008+0000 7f025e800640 1 -- 172.21.15.19:0/1668622218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015c390 msgr2=0x7f0260073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.009+0000 7f025e800640 1 --2- 172.21.15.19:0/1668622218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015c390 0x7f0260073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.009+0000 7f025e800640 1 -- 172.21.15.19:0/1668622218 >> v1:172.21.15.188:6789/0 conn(0x7f02600af830 legacy=0x7f0260160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.009+0000 7f025e800640 1 -- 172.21.15.19:0/1668622218 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02600ad120 con 0x7f02600d0650 2024-03-28T13:15:27.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.009+0000 7f025de00640 1 --2- 172.21.15.19:0/1668622218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015c390 0x7f0260073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:15:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.010+0000 7f025e800640 1 --2- 172.21.15.19:0/1668622218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02600d0650 0x7f02600d0ac0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0258009d90 tx=0x7f025802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=951eae144df533ee server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.011+0000 7f025d400640 1 -- 172.21.15.19:0/1668622218 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258039070 con 0x7f02600d0650 2024-03-28T13:15:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.011+0000 7f025d400640 1 -- 172.21.15.19:0/1668622218 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025802cc50 con 0x7f02600d0650 2024-03-28T13:15:27.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.011+0000 7f025d400640 1 -- 172.21.15.19:0/1668622218 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258034440 con 0x7f02600d0650 2024-03-28T13:15:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.011+0000 7f0264c00640 1 -- 172.21.15.19:0/1668622218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02600d0650 msgr2=0x7f02600d0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.011+0000 7f0264c00640 1 --2- 172.21.15.19:0/1668622218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02600d0650 0x7f02600d0ac0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0258009d90 tx=0x7f025802bea0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.012+0000 7f0264c00640 1 -- 172.21.15.19:0/1668622218 shutdown_connections 2024-03-28T13:15:27.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.012+0000 7f0264c00640 1 --2- 172.21.15.19:0/1668622218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02600d0650 0x7f02600d0ac0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.012+0000 7f0264c00640 1 --2- 172.21.15.19:0/1668622218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015c390 0x7f0260073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.012+0000 7f0264c00640 1 -- 172.21.15.19:0/1668622218 >> 172.21.15.19:0/1668622218 conn(0x7f026006b990 msgr2=0x7f026006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.012+0000 7f0264c00640 1 -- 172.21.15.19:0/1668622218 shutdown_connections 2024-03-28T13:15:27.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.012+0000 7f0264c00640 1 -- 172.21.15.19:0/1668622218 wait complete. 2024-03-28T13:15:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.014+0000 7f0264c00640 1 Processor -- start 2024-03-28T13:15:27.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.014+0000 7f0264c00640 1 -- start start 2024-03-28T13:15:27.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.016+0000 7f0264c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02600af830 0x7f026015f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.016+0000 7f0264c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02600d0650 0x7f02600b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.016+0000 7f0264c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015c390 0x7f02600b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.016+0000 7f0264c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0260160120 con 0x7f02600d0650 2024-03-28T13:15:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.016+0000 7f0264c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f02600ad6e0 con 0x7f02600af830 2024-03-28T13:15:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.016+0000 7f0264c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0260160d50 con 0x7f026015c390 2024-03-28T13:15:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.016+0000 7f0257e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02600d0650 0x7f02600b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.016+0000 7f0257e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02600d0650 0x7f02600b15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59148/0 (socket says 172.21.15.19:59148) 2024-03-28T13:15:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.016+0000 7f0257e00640 1 -- 172.21.15.19:0/4027937076 learned_addr learned my addr 172.21.15.19:0/4027937076 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.017+0000 7f025de00640 1 --2- 172.21.15.19:0/4027937076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02600af830 0x7f026015f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.017+0000 7f025e800640 1 --2- 172.21.15.19:0/4027937076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015c390 0x7f02600b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.017+0000 7f0257e00640 1 -- 172.21.15.19:0/4027937076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015c390 msgr2=0x7f02600b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.017+0000 7f0257e00640 1 --2- 172.21.15.19:0/4027937076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015c390 0x7f02600b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.017+0000 7f0257e00640 1 -- 172.21.15.19:0/4027937076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02600af830 msgr2=0x7f026015f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.017+0000 7f0257e00640 1 --2- 172.21.15.19:0/4027937076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02600af830 0x7f026015f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.017+0000 7f0257e00640 1 -- 172.21.15.19:0/4027937076 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0260155850 con 0x7f02600d0650 2024-03-28T13:15:27.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.017+0000 7f0257e00640 1 --2- 172.21.15.19:0/4027937076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02600d0650 0x7f02600b15a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f024800c3f0 tx=0x7f024800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:27.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.018+0000 7f0256000640 1 -- 172.21.15.19:0/4027937076 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02480052d0 con 0x7f02600d0650 2024-03-28T13:15:27.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.018+0000 7f0256000640 1 -- 172.21.15.19:0/4027937076 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0248005490 con 0x7f02600d0650 2024-03-28T13:15:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.018+0000 7f0256000640 1 -- 172.21.15.19:0/4027937076 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248007260 con 0x7f02600d0650 2024-03-28T13:15:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.018+0000 7f0264c00640 1 -- 172.21.15.19:0/4027937076 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f026015c900 con 0x7f02600d0650 2024-03-28T13:15:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.018+0000 7f0264c00640 1 -- 172.21.15.19:0/4027937076 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f026015cec0 con 0x7f02600d0650 2024-03-28T13:15:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.019+0000 7f0264c00640 1 -- 172.21.15.19:0/4027937076 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02600685c0 con 0x7f02600d0650 2024-03-28T13:15:27.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.020+0000 7f0256000640 1 -- 172.21.15.19:0/4027937076 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0248007400 con 0x7f02600d0650 2024-03-28T13:15:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.021+0000 7f0256000640 1 --2- 172.21.15.19:0/4027937076 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f023c06c530 0x7f023c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.021+0000 7f0256000640 1 -- 172.21.15.19:0/4027937076 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f024808ad20 con 0x7f02600d0650 2024-03-28T13:15:27.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.022+0000 7f025de00640 1 --2- 172.21.15.19:0/4027937076 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f023c06c530 0x7f023c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.023+0000 7f025de00640 1 --2- 172.21.15.19:0/4027937076 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f023c06c530 0x7f023c06e9f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0250009b70 tx=0x7f0250008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:27.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.026+0000 7f0256000640 1 -- 172.21.15.19:0/4027937076 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0248059210 con 0x7f02600d0650 2024-03-28T13:15:27.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.271+0000 7f0264c00640 1 -- 172.21.15.19:0/4027937076 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f026006f1b0 con 0x7f02600d0650 2024-03-28T13:15:27.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.272+0000 7f0256000640 1 -- 172.21.15.19:0/4027937076 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5187 (secure 0 0 0) 0x7f0248058b10 con 0x7f02600d0650 2024-03-28T13:15:27.272 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:15:27.272 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/144726418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":144726418},{"type":"v1","addr":"172.21.15.188:6836","nonce":144726418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5363,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2642581607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2642581607},{"type":"v1","addr":"172.21.15.19:6835","nonce":2642581607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1721039756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1721039756},{"type":"v1","addr":"172.21.15.188:6837","nonce":1721039756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:14:31.526577+0000","modified":"2024-03-28T13:14:56.038418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":6,"addr":"172.21.15.19:6834/2130552083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2130552083},{"type":"v1","addr":"172.21.15.19:6834","nonce":2130552083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6708,"qdb_cluster":[6708]},"id":5}]} 2024-03-28T13:15:27.272 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 52 2024-03-28T13:15:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.274+0000 7f0264c00640 1 -- 172.21.15.19:0/4027937076 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f023c06c530 msgr2=0x7f023c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.274+0000 7f0264c00640 1 --2- 172.21.15.19:0/4027937076 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f023c06c530 0x7f023c06e9f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0250009b70 tx=0x7f0250008040 comp rx=0 tx=0).stop 2024-03-28T13:15:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.274+0000 7f0264c00640 1 -- 172.21.15.19:0/4027937076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02600d0650 msgr2=0x7f02600b15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.274+0000 7f0264c00640 1 --2- 172.21.15.19:0/4027937076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02600d0650 0x7f02600b15a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f024800c3f0 tx=0x7f024800c8b0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.274+0000 7f0264c00640 1 -- 172.21.15.19:0/4027937076 shutdown_connections 2024-03-28T13:15:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.274+0000 7f0264c00640 1 --2- 172.21.15.19:0/4027937076 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f023c06c530 0x7f023c06e9f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.274+0000 7f0264c00640 1 --2- 172.21.15.19:0/4027937076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026015c390 0x7f02600b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.274+0000 7f0264c00640 1 --2- 172.21.15.19:0/4027937076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f02600d0650 0x7f02600b15a0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.274+0000 7f0264c00640 1 --2- 172.21.15.19:0/4027937076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02600af830 0x7f026015f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.274+0000 7f0264c00640 1 -- 172.21.15.19:0/4027937076 >> 172.21.15.19:0/4027937076 conn(0x7f026006b990 msgr2=0x7f0260159980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.275+0000 7f0264c00640 1 -- 172.21.15.19:0/4027937076 shutdown_connections 2024-03-28T13:15:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.275+0000 7f0264c00640 1 -- 172.21.15.19:0/4027937076 wait complete. 2024-03-28T13:15:27.293 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-28T13:15:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.522+0000 7f49ee200640 1 Processor -- start 2024-03-28T13:15:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.522+0000 7f49ee200640 1 -- start start 2024-03-28T13:15:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.524+0000 7f49ee200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8071af0 0x7f49e806d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.524+0000 7f49ee200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49e80b3920 con 0x7f49e80735c0 2024-03-28T13:15:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.524+0000 7f49ee200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49e80b3c00 con 0x7f49e8160360 2024-03-28T13:15:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.524+0000 7f49ee200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f49e80b36d0 con 0x7f49e8071af0 2024-03-28T13:15:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.524+0000 7f49ed800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f49e80735c0 0x7f49e806db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43788/0 (socket says 172.21.15.19:43788) 2024-03-28T13:15:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.524+0000 7f49ed800640 1 -- 172.21.15.19:0/2576678003 learned_addr learned my addr 172.21.15.19:0/2576678003 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:15:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.525+0000 7f49dfe00640 1 --2- 172.21.15.19:0/2576678003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8071af0 0x7f49e806d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.525+0000 7f49e7e00640 1 -- 172.21.15.19:0/2576678003 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1714536319 0 0) 0x7f49e80b3920 con 0x7f49e80735c0 2024-03-28T13:15:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.525+0000 7f49e7e00640 1 -- 172.21.15.19:0/2576678003 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49cc003520 con 0x7f49e80735c0 2024-03-28T13:15:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.525+0000 7f49e7e00640 1 -- 172.21.15.19:0/2576678003 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2810078407 0 0) 0x7f49e80b3c00 con 0x7f49e8160360 2024-03-28T13:15:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.525+0000 7f49dfe00640 1 -- 172.21.15.19:0/2576678003 >> v1:172.21.15.188:6789/0 conn(0x7f49e8160360 legacy=0x7f49e80713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.525+0000 7f49dfe00640 1 -- 172.21.15.19:0/2576678003 >> v1:172.21.15.19:6789/0 conn(0x7f49e80735c0 legacy=0x7f49e806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.525+0000 7f49dfe00640 1 -- 172.21.15.19:0/2576678003 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e80b3ee0 con 0x7f49e8071af0 2024-03-28T13:15:27.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.527+0000 7f49dfe00640 1 --2- 172.21.15.19:0/2576678003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8071af0 0x7f49e806d620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f49d00097d0 tx=0x7f49d002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=876b0ff06e1d0378 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.527+0000 7f49e7e00640 1 -- 172.21.15.19:0/2576678003 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0035020 con 0x7f49e8071af0 2024-03-28T13:15:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.527+0000 7f49e7e00640 1 -- 172.21.15.19:0/2576678003 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d0031890 con 0x7f49e8071af0 2024-03-28T13:15:27.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.527+0000 7f49e7e00640 1 -- 172.21.15.19:0/2576678003 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0031c60 con 0x7f49e8071af0 2024-03-28T13:15:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.528+0000 7f49ee200640 1 -- 172.21.15.19:0/2576678003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8071af0 msgr2=0x7f49e806d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.528+0000 7f49ee200640 1 --2- 172.21.15.19:0/2576678003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8071af0 0x7f49e806d620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f49d00097d0 tx=0x7f49d002b930 comp rx=0 tx=0).stop 2024-03-28T13:15:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.528+0000 7f49ee200640 1 -- 172.21.15.19:0/2576678003 shutdown_connections 2024-03-28T13:15:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.528+0000 7f49ee200640 1 --2- 172.21.15.19:0/2576678003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8071af0 0x7f49e806d620 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.528+0000 7f49ee200640 1 -- 172.21.15.19:0/2576678003 >> 172.21.15.19:0/2576678003 conn(0x7f49e80b1ba0 msgr2=0x7f49e80b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.528+0000 7f49ee200640 1 -- 172.21.15.19:0/2576678003 shutdown_connections 2024-03-28T13:15:27.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.528+0000 7f49ee200640 1 -- 172.21.15.19:0/2576678003 wait complete. 2024-03-28T13:15:27.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.530+0000 7f49ee200640 1 Processor -- start 2024-03-28T13:15:27.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.530+0000 7f49ee200640 1 -- start start 2024-03-28T13:15:27.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.531+0000 7f49ee200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8071af0 0x7f49e81580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.531+0000 7f49ee200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80735c0 0x7f49e8154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.531+0000 7f49ee200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e8160360 0x7f49e815cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.531+0000 7f49ee200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49e80b49c0 con 0x7f49e8160360 2024-03-28T13:15:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.531+0000 7f49ee200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49e80b4530 con 0x7f49e80735c0 2024-03-28T13:15:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.531+0000 7f49ee200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f49e8160740 con 0x7f49e8071af0 2024-03-28T13:15:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.531+0000 7f49ed800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e8160360 0x7f49e815cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.531+0000 7f49ed800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e8160360 0x7f49e815cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59152/0 (socket says 172.21.15.19:59152) 2024-03-28T13:15:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.531+0000 7f49ed800640 1 -- 172.21.15.19:0/677724291 learned_addr learned my addr 172.21.15.19:0/677724291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.531+0000 7f49ece00640 1 --2- 172.21.15.19:0/677724291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8071af0 0x7f49e81580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.532+0000 7f49dfe00640 1 --2- 172.21.15.19:0/677724291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80735c0 0x7f49e8154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.532+0000 7f49ed800640 1 -- 172.21.15.19:0/677724291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8071af0 msgr2=0x7f49e81580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.532+0000 7f49ed800640 1 --2- 172.21.15.19:0/677724291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8071af0 0x7f49e81580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.532+0000 7f49ed800640 1 -- 172.21.15.19:0/677724291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80735c0 msgr2=0x7f49e8154350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.532+0000 7f49ed800640 1 --2- 172.21.15.19:0/677724291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80735c0 0x7f49e8154350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.532+0000 7f49ed800640 1 -- 172.21.15.19:0/677724291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e8154890 con 0x7f49e8160360 2024-03-28T13:15:27.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.532+0000 7f49ed800640 1 --2- 172.21.15.19:0/677724291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e8160360 0x7f49e815cd30 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f49e000cb40 tx=0x7f49e000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.532+0000 7f49e5600640 1 -- 172.21.15.19:0/677724291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0017070 con 0x7f49e8160360 2024-03-28T13:15:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.532+0000 7f49e5600640 1 -- 172.21.15.19:0/677724291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e000aca0 con 0x7f49e8160360 2024-03-28T13:15:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.532+0000 7f49e5600640 1 -- 172.21.15.19:0/677724291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0012620 con 0x7f49e8160360 2024-03-28T13:15:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.532+0000 7f49ee200640 1 -- 172.21.15.19:0/677724291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e8158b40 con 0x7f49e8160360 2024-03-28T13:15:27.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.533+0000 7f49ee200640 1 -- 172.21.15.19:0/677724291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e8159100 con 0x7f49e8160360 2024-03-28T13:15:27.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.535+0000 7f49e5600640 1 -- 172.21.15.19:0/677724291 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f49e001a070 con 0x7f49e8160360 2024-03-28T13:15:27.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.536+0000 7f49e5600640 1 --2- 172.21.15.19:0/677724291 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49c406c530 0x7f49c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.536+0000 7f49e5600640 1 -- 172.21.15.19:0/677724291 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f49e008b620 con 0x7f49e8160360 2024-03-28T13:15:27.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.537+0000 7f49ece00640 1 --2- 172.21.15.19:0/677724291 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49c406c530 0x7f49c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.537+0000 7f49f2465740 1 Processor -- start 2024-03-28T13:15:27.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.537+0000 7f49f2465740 1 -- start start 2024-03-28T13:15:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.538+0000 7f49ece00640 1 --2- 172.21.15.19:0/677724291 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49c406c530 0x7f49c406e9f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f49d8000f60 tx=0x7f49d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.538+0000 7f49f2465740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2239758a0 0x55e223975c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.538+0000 7f49f2465740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2239458c0 0x55e223972b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.538+0000 7f49f2465740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e223949420 0x55e223973050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.538+0000 7f49f2465740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e2238af5a0 con 0x55e2239458c0 2024-03-28T13:15:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.538+0000 7f49f2465740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e223857390 con 0x55e2239758a0 2024-03-28T13:15:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.538+0000 7f49f2465740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55e2238ab550 con 0x55e223949420 2024-03-28T13:15:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.538+0000 7f49dfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2239458c0 0x55e223972b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.539+0000 7f49dfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2239458c0 0x55e223972b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59166/0 (socket says 172.21.15.19:59166) 2024-03-28T13:15:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.539+0000 7f49dfe00640 1 -- 172.21.15.19:0/3393468119 learned_addr learned my addr 172.21.15.19:0/3393468119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.539+0000 7f49ece00640 1 --2- 172.21.15.19:0/3393468119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2239758a0 0x55e223975c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.539+0000 7f49ed800640 1 --2- 172.21.15.19:0/3393468119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e223949420 0x55e223973050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.539+0000 7f49dfe00640 1 -- 172.21.15.19:0/3393468119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e223949420 msgr2=0x55e223973050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:27.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.539+0000 7f49dfe00640 1 --2- 172.21.15.19:0/3393468119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e223949420 0x55e223973050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.539+0000 7f49dfe00640 1 -- 172.21.15.19:0/3393468119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2239758a0 msgr2=0x55e223975c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:27.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.539+0000 7f49dfe00640 1 --2- 172.21.15.19:0/3393468119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2239758a0 0x55e223975c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.539+0000 7f49dfe00640 1 -- 172.21.15.19:0/3393468119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e22368f7c0 con 0x55e2239458c0 2024-03-28T13:15:27.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.539+0000 7f49ece00640 1 --2- 172.21.15.19:0/3393468119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2239758a0 0x55e223975c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:27.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.539+0000 7f49ed800640 1 --2- 172.21.15.19:0/3393468119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e223949420 0x55e223973050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:27.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.539+0000 7f49dfe00640 1 --2- 172.21.15.19:0/3393468119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2239458c0 0x55e223972b10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f49d0000c00 tx=0x7f49d0030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e498ea91543fbd88 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:27.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.540+0000 7f49dd600640 1 -- 172.21.15.19:0/3393468119 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d004a070 con 0x55e2239458c0 2024-03-28T13:15:27.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.540+0000 7f49dd600640 1 -- 172.21.15.19:0/3393468119 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d00317d0 con 0x55e2239458c0 2024-03-28T13:15:27.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.540+0000 7f49dd600640 1 -- 172.21.15.19:0/3393468119 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d0035040 con 0x55e2239458c0 2024-03-28T13:15:27.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.540+0000 7f49f2465740 1 -- 172.21.15.19:0/3393468119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2239458c0 msgr2=0x55e223972b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.540+0000 7f49f2465740 1 --2- 172.21.15.19:0/3393468119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2239458c0 0x55e223972b10 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f49d0000c00 tx=0x7f49d0030000 comp rx=0 tx=0).stop 2024-03-28T13:15:27.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.540+0000 7f49f2465740 1 -- 172.21.15.19:0/3393468119 shutdown_connections 2024-03-28T13:15:27.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.540+0000 7f49f2465740 1 --2- 172.21.15.19:0/3393468119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e223949420 0x55e223973050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.540+0000 7f49f2465740 1 --2- 172.21.15.19:0/3393468119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2239458c0 0x55e223972b10 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.540+0000 7f49f2465740 1 --2- 172.21.15.19:0/3393468119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e2239758a0 0x55e223975c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.540+0000 7f49f2465740 1 -- 172.21.15.19:0/3393468119 >> 172.21.15.19:0/3393468119 conn(0x55e223947b30 msgr2=0x55e2239906b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:27.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.540+0000 7f49f2465740 1 -- 172.21.15.19:0/3393468119 shutdown_connections 2024-03-28T13:15:27.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.540+0000 7f49f2465740 1 -- 172.21.15.19:0/3393468119 wait complete. 2024-03-28T13:15:27.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.541+0000 7f49f2465740 1 -- start start 2024-03-28T13:15:27.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.541+0000 7f49f2465740 1 Processor -- start 2024-03-28T13:15:27.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.542+0000 7f49f2465740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223949420 0x55e22396bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.542+0000 7f49f2465740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2239758a0 0x55e22396f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.542+0000 7f49f2465740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e223976600 0x55e22396f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.542+0000 7f49f2465740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e22394a5d0 con 0x55e223976600 2024-03-28T13:15:27.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.542+0000 7f49f2465740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e22386a780 con 0x55e223949420 2024-03-28T13:15:27.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.542+0000 7f49f2465740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55e2239499c0 con 0x55e2239758a0 2024-03-28T13:15:27.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.542+0000 7f49f2465740 10 client.0 _ll_register_callbacks cb 0x55e2238a6fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:15:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.543+0000 7f49ed800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e223976600 0x55e22396f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.543+0000 7f49ed800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e223976600 0x55e22396f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59180/0 (socket says 172.21.15.19:59180) 2024-03-28T13:15:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.543+0000 7f49ed800640 1 -- 172.21.15.19:0/3945129007 learned_addr learned my addr 172.21.15.19:0/3945129007 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.543+0000 7f49ece00640 1 --2- 172.21.15.19:0/3945129007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223949420 0x55e22396bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.543+0000 7f49dfe00640 1 --2- 172.21.15.19:0/3945129007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2239758a0 0x55e22396f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.543+0000 7f49ed800640 1 -- 172.21.15.19:0/3945129007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2239758a0 msgr2=0x55e22396f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.543+0000 7f49ed800640 1 --2- 172.21.15.19:0/3945129007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2239758a0 0x55e22396f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.543+0000 7f49ed800640 1 -- 172.21.15.19:0/3945129007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223949420 msgr2=0x55e22396bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.543+0000 7f49ed800640 1 --2- 172.21.15.19:0/3945129007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223949420 0x55e22396bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.543+0000 7f49ed800640 1 -- 172.21.15.19:0/3945129007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2236eff50 con 0x55e223976600 2024-03-28T13:15:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.543+0000 7f49dfe00640 1 --2- 172.21.15.19:0/3945129007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2239758a0 0x55e22396f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.543+0000 7f49ece00640 1 --2- 172.21.15.19:0/3945129007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223949420 0x55e22396bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.543+0000 7f49ed800640 1 --2- 172.21.15.19:0/3945129007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e223976600 0x55e22396f790 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f49e0089a20 tx=0x7f49e0059560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86749e548aee93f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.543+0000 7f49d6000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:15:27.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.544+0000 7f49d6000640 1 -- 172.21.15.19:0/3945129007 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0061d70 con 0x55e223976600 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.544+0000 7f49d6000640 1 -- 172.21.15.19:0/3945129007 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f49e0004b60 con 0x55e223976600 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.544+0000 7f49f2465740 1 -- 172.21.15.19:0/3945129007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49d00317d0 con 0x55e223976600 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.544+0000 7f49d6000640 1 -- 172.21.15.19:0/3945129007 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f49e0077170 con 0x55e223976600 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.544+0000 7f49f2465740 10 client.6876 fetch_fsmap learned FSMap version 52 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.544+0000 7f49f2465740 1 -- 172.21.15.19:0/3945129007 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55e2235830b0 con 0x55e223976600 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.544+0000 7f49d6000640 1 -- 172.21.15.19:0/3945129007 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d00317d0 con 0x55e223976600 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.544+0000 7f49d6000640 1 -- 172.21.15.19:0/3945129007 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f49e0058870 con 0x55e223976600 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.545+0000 7f49f2465740 10 client.6876 fetch_fsmap finished waiting for FSMap version 52 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.545+0000 7f49f2465740 10 client.6876 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.545+0000 7f49f2465740 20 client.6876 populate_metadata read hostname 'smithi019' 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.545+0000 7f49f2465740 1 --2- 172.21.15.19:0/3945129007 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55e22395ce00 0x55e2239a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.545+0000 7f49f2465740 4 client.6876 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.545+0000 7f49f2465740 1 -- 172.21.15.19:0/3945129007 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e223883870 con 0x55e22395ce00 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.545+0000 7f49ece00640 1 --2- 172.21.15.19:0/3945129007 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55e22395ce00 0x55e2239a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.545+0000 7f49ece00640 1 --2- 172.21.15.19:0/3945129007 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55e22395ce00 0x55e2239a3f30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=635741d6ddeb79a5 server_cookie=f0442177d0d5dd47 in_seq=0 out_seq=0 2024-03-28T13:15:27.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.545+0000 7f49d6000640 10 client.6876 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:27.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.547+0000 7f49d6000640 1 -- 172.21.15.19:0/3945129007 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e223883870 con 0x55e22395ce00 2024-03-28T13:15:27.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.547+0000 7f49d6000640 10 client.6876 handle_command_reply: tid=0 2024-03-28T13:15:27.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.547+0000 7f49f2465740 1 client.6876 shutdown 2024-03-28T13:15:27.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.547+0000 7f49f2465740 1 -- 172.21.15.19:0/3945129007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e223976600 msgr2=0x55e22396f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.547+0000 7f49f2465740 1 --2- 172.21.15.19:0/3945129007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e223976600 0x55e22396f790 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f49e0089a20 tx=0x7f49e0059560 comp rx=0 tx=0).stop 2024-03-28T13:15:27.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.547+0000 7f49f2465740 1 -- 172.21.15.19:0/3945129007 shutdown_connections 2024-03-28T13:15:27.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.547+0000 7f49f2465740 1 --2- 172.21.15.19:0/3945129007 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55e22395ce00 0x55e2239a3f30 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.547+0000 7f49f2465740 1 --2- 172.21.15.19:0/3945129007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e223976600 0x55e22396f790 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.547+0000 7f49f2465740 1 --2- 172.21.15.19:0/3945129007 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2239758a0 0x55e22396f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.547+0000 7f49f2465740 1 --2- 172.21.15.19:0/3945129007 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223949420 0x55e22396bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.547+0000 7f49d6000640 0 client.6876 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:27.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.547+0000 7f49f2465740 1 -- 172.21.15.19:0/3945129007 >> 172.21.15.19:0/3945129007 conn(0x55e2239458c0 msgr2=0x55e223993bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:27.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.547+0000 7f49f2465740 1 -- 172.21.15.19:0/3945129007 shutdown_connections 2024-03-28T13:15:27.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.547+0000 7f49f2465740 1 -- 172.21.15.19:0/3945129007 wait complete. 2024-03-28T13:15:27.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.547+0000 7f49f2465740 20 client.6876 trim_cache size 0 max 16384 2024-03-28T13:15:27.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.562+0000 7f49f2465740 1 Processor -- start 2024-03-28T13:15:27.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.562+0000 7f49f2465740 1 -- start start 2024-03-28T13:15:27.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.563+0000 7f49f2465740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e223949420 0x55e223972a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.563+0000 7f49f2465740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2239758a0 0x55e223977640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.563+0000 7f49f2465740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223976600 0x55e223993db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.563+0000 7f49f2465740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49e8204af0 con 0x55e223949420 2024-03-28T13:15:27.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.563+0000 7f49f2465740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49e8203b10 con 0x55e223976600 2024-03-28T13:15:27.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.563+0000 7f49f2465740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f49e815b130 con 0x55e2239758a0 2024-03-28T13:15:27.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.563+0000 7f49ece00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e223949420 0x55e223972a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.563+0000 7f49ece00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e223949420 0x55e223972a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59192/0 (socket says 172.21.15.19:59192) 2024-03-28T13:15:27.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.563+0000 7f49ece00640 1 -- 172.21.15.19:0/2364651512 learned_addr learned my addr 172.21.15.19:0/2364651512 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.564+0000 7f49ece00640 1 -- 172.21.15.19:0/2364651512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2239758a0 msgr2=0x55e223977640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.564+0000 7f49ece00640 1 --2- 172.21.15.19:0/2364651512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2239758a0 0x55e223977640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.564+0000 7f49ece00640 1 -- 172.21.15.19:0/2364651512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223976600 msgr2=0x55e223993db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:27.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.564+0000 7f49ece00640 1 --2- 172.21.15.19:0/2364651512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223976600 0x55e223993db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.564+0000 7f49ece00640 1 -- 172.21.15.19:0/2364651512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e223959a70 con 0x55e223949420 2024-03-28T13:15:27.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.564+0000 7f49ece00640 1 --2- 172.21.15.19:0/2364651512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e223949420 0x55e223972a70 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x55e22396d330 tx=0x7f49d8017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714a3d21bcc199e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:27.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.565+0000 7f49dcc00640 1 -- 172.21.15.19:0/2364651512 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d80244e0 con 0x55e223949420 2024-03-28T13:15:27.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.565+0000 7f49dcc00640 1 -- 172.21.15.19:0/2364651512 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d8029020 con 0x55e223949420 2024-03-28T13:15:27.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.565+0000 7f49dcc00640 1 -- 172.21.15.19:0/2364651512 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8024680 con 0x55e223949420 2024-03-28T13:15:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.565+0000 7f49f2465740 1 -- 172.21.15.19:0/2364651512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e223949420 msgr2=0x55e223972a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.565+0000 7f49f2465740 1 --2- 172.21.15.19:0/2364651512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e223949420 0x55e223972a70 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x55e22396d330 tx=0x7f49d8017040 comp rx=0 tx=0).stop 2024-03-28T13:15:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.565+0000 7f49f2465740 1 -- 172.21.15.19:0/2364651512 shutdown_connections 2024-03-28T13:15:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.565+0000 7f49f2465740 1 --2- 172.21.15.19:0/2364651512 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223976600 0x55e223993db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.565+0000 7f49f2465740 1 --2- 172.21.15.19:0/2364651512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e2239758a0 0x55e223977640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.565+0000 7f49f2465740 1 --2- 172.21.15.19:0/2364651512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e223949420 0x55e223972a70 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.565+0000 7f49f2465740 1 -- 172.21.15.19:0/2364651512 >> 172.21.15.19:0/2364651512 conn(0x55e2239458c0 msgr2=0x55e223976ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.565+0000 7f49f2465740 1 -- 172.21.15.19:0/2364651512 shutdown_connections 2024-03-28T13:15:27.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.565+0000 7f49f2465740 1 -- 172.21.15.19:0/2364651512 wait complete. 2024-03-28T13:15:27.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.566+0000 7f49f2465740 1 -- start start 2024-03-28T13:15:27.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.566+0000 7f49f2465740 1 Processor -- start 2024-03-28T13:15:27.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.567+0000 7f49f2465740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223949420 0x55e22396be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.567+0000 7f49f2465740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2239758a0 0x55e22396e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.567+0000 7f49f2465740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e223976600 0x55e22396edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.567+0000 7f49f2465740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e22396d1b0 con 0x55e2239758a0 2024-03-28T13:15:27.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.567+0000 7f49f2465740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49e815b780 con 0x55e223949420 2024-03-28T13:15:27.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.567+0000 7f49f2465740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55e22396c610 con 0x55e223976600 2024-03-28T13:15:27.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.567+0000 7f49f2465740 10 client.0 _ll_register_callbacks cb 0x7f49e0055740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:15:27.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.567+0000 7f49dfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2239758a0 0x55e22396e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.568+0000 7f49dfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2239758a0 0x55e22396e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59194/0 (socket says 172.21.15.19:59194) 2024-03-28T13:15:27.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.568+0000 7f49dfe00640 1 -- 172.21.15.19:0/102390432 learned_addr learned my addr 172.21.15.19:0/102390432 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:27.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.568+0000 7f49ece00640 1 --2- 172.21.15.19:0/102390432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223949420 0x55e22396be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.568+0000 7f49ed800640 1 --2- 172.21.15.19:0/102390432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e223976600 0x55e22396edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.568+0000 7f49dfe00640 1 -- 172.21.15.19:0/102390432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e223976600 msgr2=0x55e22396edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.568+0000 7f49dfe00640 1 --2- 172.21.15.19:0/102390432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e223976600 0x55e22396edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.568+0000 7f49dfe00640 1 -- 172.21.15.19:0/102390432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223949420 msgr2=0x55e22396be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.568+0000 7f49dfe00640 1 --2- 172.21.15.19:0/102390432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223949420 0x55e22396be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.568+0000 7f49dfe00640 1 -- 172.21.15.19:0/102390432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2239919b0 con 0x55e2239758a0 2024-03-28T13:15:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.568+0000 7f49ece00640 1 --2- 172.21.15.19:0/102390432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223949420 0x55e22396be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.568+0000 7f49dfe00640 1 --2- 172.21.15.19:0/102390432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2239758a0 0x55e22396e8b0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f49d00098c0 tx=0x7f49d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c854d83155d303 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.568+0000 7f49d6000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:15:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.569+0000 7f49d6000640 1 -- 172.21.15.19:0/102390432 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d004a070 con 0x55e2239758a0 2024-03-28T13:15:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.569+0000 7f49d6000640 1 -- 172.21.15.19:0/102390432 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f49d00319e0 con 0x55e2239758a0 2024-03-28T13:15:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.569+0000 7f49f2465740 1 -- 172.21.15.19:0/102390432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49d8029020 con 0x55e2239758a0 2024-03-28T13:15:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.569+0000 7f49d6000640 1 -- 172.21.15.19:0/102390432 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f49d0058050 con 0x55e2239758a0 2024-03-28T13:15:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.569+0000 7f49f2465740 10 client.6882 fetch_fsmap learned FSMap version 52 2024-03-28T13:15:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.569+0000 7f49f2465740 1 -- 172.21.15.19:0/102390432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55e2239777f0 con 0x55e2239758a0 2024-03-28T13:15:27.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.569+0000 7f49d6000640 1 -- 172.21.15.19:0/102390432 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d8029020 con 0x55e2239758a0 2024-03-28T13:15:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.569+0000 7f49d6000640 1 -- 172.21.15.19:0/102390432 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7f49d005b020 con 0x55e2239758a0 2024-03-28T13:15:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.569+0000 7f49f2465740 10 client.6882 fetch_fsmap finished waiting for FSMap version 52 2024-03-28T13:15:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.569+0000 7f49f2465740 10 client.6882 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T13:15:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.569+0000 7f49f2465740 20 client.6882 populate_metadata read hostname 'smithi019' 2024-03-28T13:15:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.569+0000 7f49f2465740 1 --2- 172.21.15.19:0/102390432 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55e223a0e0a0 0x55e2239b3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.569+0000 7f49f2465740 4 client.6882 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-28T13:15:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.569+0000 7f49f2465740 1 -- 172.21.15.19:0/102390432 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55e2238bcae0 con 0x55e223a0e0a0 2024-03-28T13:15:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.570+0000 7f49ece00640 1 --2- 172.21.15.19:0/102390432 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55e223a0e0a0 0x55e2239b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.570+0000 7f49ece00640 1 --2- 172.21.15.19:0/102390432 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55e223a0e0a0 0x55e2239b3320 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef636f5714bb7bd6 server_cookie=66831458b54edc7a in_seq=0 out_seq=0 2024-03-28T13:15:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.570+0000 7f49d6000640 10 client.6882 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.571+0000 7f49d6000640 1 -- 172.21.15.19:0/102390432 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x55e2238bcae0 con 0x55e223a0e0a0 2024-03-28T13:15:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.571+0000 7f49d6000640 10 client.6882 handle_command_reply: tid=0 2024-03-28T13:15:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.571+0000 7f49f2465740 1 client.6882 shutdown 2024-03-28T13:15:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.571+0000 7f49f2465740 1 -- 172.21.15.19:0/102390432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2239758a0 msgr2=0x55e22396e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.571+0000 7f49f2465740 1 --2- 172.21.15.19:0/102390432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2239758a0 0x55e22396e8b0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f49d00098c0 tx=0x7f49d00046c0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.571+0000 7f49f2465740 1 -- 172.21.15.19:0/102390432 shutdown_connections 2024-03-28T13:15:27.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.571+0000 7f49f2465740 1 --2- 172.21.15.19:0/102390432 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55e223a0e0a0 0x55e2239b3320 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.571+0000 7f49f2465740 1 --2- 172.21.15.19:0/102390432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e223976600 0x55e22396edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.571+0000 7f49f2465740 1 --2- 172.21.15.19:0/102390432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e2239758a0 0x55e22396e8b0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.571+0000 7f49f2465740 1 --2- 172.21.15.19:0/102390432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e223949420 0x55e22396be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.571+0000 7f49d6000640 0 client.6882 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:27.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.571+0000 7f49f2465740 1 -- 172.21.15.19:0/102390432 >> 172.21.15.19:0/102390432 conn(0x55e2239458c0 msgr2=0x55e223972a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:27.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.571+0000 7f49f2465740 1 -- 172.21.15.19:0/102390432 shutdown_connections 2024-03-28T13:15:27.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.571+0000 7f49f2465740 1 -- 172.21.15.19:0/102390432 wait complete. 2024-03-28T13:15:27.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.571+0000 7f49f2465740 20 client.6882 trim_cache size 0 max 16384 2024-03-28T13:15:27.576 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:15:27.577 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-03-28T13:15:27.577 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:15:27.577 INFO:teuthology.orchestra.run.smithi019.stdout: "damage_type": "dentry", 2024-03-28T13:15:27.577 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 1321930366, 2024-03-28T13:15:27.577 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511627776, 2024-03-28T13:15:27.577 INFO:teuthology.orchestra.run.smithi019.stdout: "frag": "*", 2024-03-28T13:15:27.577 INFO:teuthology.orchestra.run.smithi019.stdout: "dname": "file_to_be_damaged", 2024-03-28T13:15:27.577 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_id": "head", 2024-03-28T13:15:27.577 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/subdir/file_to_be_damaged" 2024-03-28T13:15:27.577 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:15:27.577 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-03-28T13:15:27.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.572+0000 7f49ee200640 1 -- 172.21.15.19:0/677724291 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49c406c530 msgr2=0x7f49c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.572+0000 7f49ee200640 1 --2- 172.21.15.19:0/677724291 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49c406c530 0x7f49c406e9f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f49d8000f60 tx=0x7f49d8009040 comp rx=0 tx=0).stop 2024-03-28T13:15:27.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.572+0000 7f49ee200640 1 -- 172.21.15.19:0/677724291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e8160360 msgr2=0x7f49e815cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.572+0000 7f49ee200640 1 --2- 172.21.15.19:0/677724291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e8160360 0x7f49e815cd30 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f49e000cb40 tx=0x7f49e000a060 comp rx=0 tx=0).stop 2024-03-28T13:15:27.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.572+0000 7f49ee200640 1 -- 172.21.15.19:0/677724291 shutdown_connections 2024-03-28T13:15:27.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.572+0000 7f49ee200640 1 --2- 172.21.15.19:0/677724291 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49c406c530 0x7f49c406e9f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.572+0000 7f49ee200640 1 --2- 172.21.15.19:0/677724291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49e8160360 0x7f49e815cd30 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.572+0000 7f49ee200640 1 --2- 172.21.15.19:0/677724291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49e80735c0 0x7f49e8154350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.572+0000 7f49ee200640 1 --2- 172.21.15.19:0/677724291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49e8071af0 0x7f49e81580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.572+0000 7f49ee200640 1 -- 172.21.15.19:0/677724291 >> 172.21.15.19:0/677724291 conn(0x7f49e80b1ba0 msgr2=0x7f49e81641b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:27.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.572+0000 7f49ee200640 1 -- 172.21.15.19:0/677724291 shutdown_connections 2024-03-28T13:15:27.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.572+0000 7f49ee200640 1 -- 172.21.15.19:0/677724291 wait complete. 2024-03-28T13:15:27.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-28T13:15:27.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.801+0000 7fae96e00640 1 Processor -- start 2024-03-28T13:15:27.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.801+0000 7fae96e00640 1 -- start start 2024-03-28T13:15:27.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.803+0000 7fae96e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fae900af4b0 0x7fae90167460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.803+0000 7fae96e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae900afab0 con 0x7fae900d0990 2024-03-28T13:15:27.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.803+0000 7fae96e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae9015cc80 con 0x7fae90153580 2024-03-28T13:15:27.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.803+0000 7fae96e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fae900af890 con 0x7fae900af4b0 2024-03-28T13:15:27.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.803+0000 7fae8f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fae900d0990 0x7fae900aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43792/0 (socket says 172.21.15.19:43792) 2024-03-28T13:15:27.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.803+0000 7fae8f400640 1 -- 172.21.15.19:0/2870353653 learned_addr learned my addr 172.21.15.19:0/2870353653 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:15:27.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.803+0000 7fae94e00640 1 --2- 172.21.15.19:0/2870353653 >> v2:172.21.15.19:3300/0 conn(0x7fae900af4b0 0x7fae90167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.804+0000 7fae94e00640 1 -- 172.21.15.19:0/2870353653 >> v1:172.21.15.188:6789/0 conn(0x7fae90153580 legacy=0x7fae900d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:27.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.804+0000 7fae8ea00640 1 -- 172.21.15.19:0/2870353653 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3112747930 0 0) 0x7fae900afab0 con 0x7fae900d0990 2024-03-28T13:15:27.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.804+0000 7fae94e00640 1 -- 172.21.15.19:0/2870353653 >> v1:172.21.15.19:6789/0 conn(0x7fae900d0990 legacy=0x7fae900aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.804+0000 7fae94e00640 1 -- 172.21.15.19:0/2870353653 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9015da00 con 0x7fae900af4b0 2024-03-28T13:15:27.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.805+0000 7fae94e00640 1 --2- 172.21.15.19:0/2870353653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae900af4b0 0x7fae90167460 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fae88009d90 tx=0x7fae8802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d2e59918750aef server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:27.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.805+0000 7fae8ea00640 1 -- 172.21.15.19:0/2870353653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88039070 con 0x7fae900af4b0 2024-03-28T13:15:27.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.805+0000 7fae8ea00640 1 -- 172.21.15.19:0/2870353653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae8802cc60 con 0x7fae900af4b0 2024-03-28T13:15:27.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.805+0000 7fae8ea00640 1 -- 172.21.15.19:0/2870353653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88034660 con 0x7fae900af4b0 2024-03-28T13:15:27.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.806+0000 7fae96e00640 1 -- 172.21.15.19:0/2870353653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae900af4b0 msgr2=0x7fae90167460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.806+0000 7fae96e00640 1 --2- 172.21.15.19:0/2870353653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae900af4b0 0x7fae90167460 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fae88009d90 tx=0x7fae8802beb0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.806+0000 7fae96e00640 1 -- 172.21.15.19:0/2870353653 shutdown_connections 2024-03-28T13:15:27.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.806+0000 7fae96e00640 1 --2- 172.21.15.19:0/2870353653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae900af4b0 0x7fae90167460 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.806+0000 7fae96e00640 1 -- 172.21.15.19:0/2870353653 >> 172.21.15.19:0/2870353653 conn(0x7fae9006b750 msgr2=0x7fae9006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:27.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.806+0000 7fae96e00640 1 -- 172.21.15.19:0/2870353653 shutdown_connections 2024-03-28T13:15:27.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.806+0000 7fae96e00640 1 -- 172.21.15.19:0/2870353653 wait complete. 2024-03-28T13:15:27.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.808+0000 7fae96e00640 1 Processor -- start 2024-03-28T13:15:27.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.808+0000 7fae96e00640 1 -- start start 2024-03-28T13:15:27.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.809+0000 7fae96e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae900af4b0 0x7fae9014f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.809+0000 7fae96e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae900d0990 0x7fae90159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.809+0000 7fae96e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae90153580 0x7fae90072660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.809+0000 7fae96e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae9015d4e0 con 0x7fae90153580 2024-03-28T13:15:27.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.809+0000 7fae96e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fae9015d200 con 0x7fae900af4b0 2024-03-28T13:15:27.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.809+0000 7fae96e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fae90153960 con 0x7fae900d0990 2024-03-28T13:15:27.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.810+0000 7fae94e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae90153580 0x7fae90072660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.810+0000 7fae94e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae90153580 0x7fae90072660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59214/0 (socket says 172.21.15.19:59214) 2024-03-28T13:15:27.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.810+0000 7fae94e00640 1 -- 172.21.15.19:0/4238422431 learned_addr learned my addr 172.21.15.19:0/4238422431 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:27.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.810+0000 7fae8fe00640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae900af4b0 0x7fae9014f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.810+0000 7fae8f400640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae900d0990 0x7fae90159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.810+0000 7fae94e00640 1 -- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae900d0990 msgr2=0x7fae90159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.810+0000 7fae94e00640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae900d0990 0x7fae90159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.810+0000 7fae94e00640 1 -- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae900af4b0 msgr2=0x7fae9014f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:27.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.810+0000 7fae94e00640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae900af4b0 0x7fae9014f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:27.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.810+0000 7fae94e00640 1 -- 172.21.15.19:0/4238422431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9014f8e0 con 0x7fae90153580 2024-03-28T13:15:27.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.810+0000 7fae8fe00640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae900af4b0 0x7fae9014f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:27.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.810+0000 7fae8f400640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae900d0990 0x7fae90159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:27.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.810+0000 7fae94e00640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae90153580 0x7fae90072660 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fae88037040 tx=0x7fae88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:27.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.811+0000 7fae87e00640 1 -- 172.21.15.19:0/4238422431 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88044030 con 0x7fae90153580 2024-03-28T13:15:27.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.811+0000 7fae87e00640 1 -- 172.21.15.19:0/4238422431 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae8802ca60 con 0x7fae90153580 2024-03-28T13:15:27.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.811+0000 7fae87e00640 1 -- 172.21.15.19:0/4238422431 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88039040 con 0x7fae90153580 2024-03-28T13:15:27.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.811+0000 7fae96e00640 1 -- 172.21.15.19:0/4238422431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae90150d90 con 0x7fae90153580 2024-03-28T13:15:27.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.811+0000 7fae96e00640 1 -- 172.21.15.19:0/4238422431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae90151350 con 0x7fae90153580 2024-03-28T13:15:27.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.812+0000 7fae96e00640 1 -- 172.21.15.19:0/4238422431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae900685c0 con 0x7fae90153580 2024-03-28T13:15:27.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.813+0000 7fae87e00640 1 -- 172.21.15.19:0/4238422431 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fae88047070 con 0x7fae90153580 2024-03-28T13:15:27.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.814+0000 7fae87e00640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fae7006c560 0x7fae7006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:27.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.815+0000 7fae87e00640 1 -- 172.21.15.19:0/4238422431 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7fae880adab0 con 0x7fae90153580 2024-03-28T13:15:27.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.815+0000 7fae8fe00640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fae7006c560 0x7fae7006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.816+0000 7fae8fe00640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fae7006c560 0x7fae7006ea20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fae900b5090 tx=0x7fae80009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:27.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:27.819+0000 7fae87e00640 1 -- 172.21.15.19:0/4238422431 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fae8807c060 con 0x7fae90153580 2024-03-28T13:15:28.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.185+0000 7fae96e00640 1 -- 172.21.15.19:0/4238422431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fae9006eca0 con 0x7fae90153580 2024-03-28T13:15:28.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.186+0000 7fae87e00640 1 -- 172.21.15.19:0/4238422431 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fae8807b960 con 0x7fae90153580 2024-03-28T13:15:28.185 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:15:28.185 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-28T13:15:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.188+0000 7fae96e00640 1 -- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fae7006c560 msgr2=0x7fae7006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.188+0000 7fae96e00640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fae7006c560 0x7fae7006ea20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fae900b5090 tx=0x7fae80009040 comp rx=0 tx=0).stop 2024-03-28T13:15:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.188+0000 7fae96e00640 1 -- 172.21.15.19:0/4238422431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae90153580 msgr2=0x7fae90072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.188+0000 7fae96e00640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae90153580 0x7fae90072660 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fae88037040 tx=0x7fae88004a40 comp rx=0 tx=0).stop 2024-03-28T13:15:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.188+0000 7fae96e00640 1 -- 172.21.15.19:0/4238422431 shutdown_connections 2024-03-28T13:15:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.188+0000 7fae96e00640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fae7006c560 0x7fae7006ea20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.188+0000 7fae96e00640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae90153580 0x7fae90072660 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.188+0000 7fae96e00640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae900d0990 0x7fae90159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.188+0000 7fae96e00640 1 --2- 172.21.15.19:0/4238422431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae900af4b0 0x7fae9014f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.188+0000 7fae96e00640 1 -- 172.21.15.19:0/4238422431 >> 172.21.15.19:0/4238422431 conn(0x7fae9006b750 msgr2=0x7fae9006bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.188+0000 7fae96e00640 1 -- 172.21.15.19:0/4238422431 shutdown_connections 2024-03-28T13:15:28.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.188+0000 7fae96e00640 1 -- 172.21.15.19:0/4238422431 wait complete. 2024-03-28T13:15:28.206 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-28T13:15:28.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:15:28.206 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-03-28T13:15:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.434+0000 7ff419400640 1 Processor -- start 2024-03-28T13:15:28.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.434+0000 7ff419400640 1 -- start start 2024-03-28T13:15:28.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.436+0000 7ff419400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff4140d4ee0 0x7ff4140cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.436+0000 7ff419400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4140d0bc0 0x7ff4140d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.436+0000 7ff419400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4140d0180 con 0x7ff41415e500 2024-03-28T13:15:28.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.436+0000 7ff419400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff4140d1dc0 con 0x7ff4140d4ee0 2024-03-28T13:15:28.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.436+0000 7ff419400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff4140d20a0 con 0x7ff4140d0bc0 2024-03-28T13:15:28.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.437+0000 7ff413e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff41415e500 0x7ff4140d2b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:43800/0 (socket says 172.21.15.19:43800) 2024-03-28T13:15:28.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.437+0000 7ff413e00640 1 -- 172.21.15.19:0/3263587702 learned_addr learned my addr 172.21.15.19:0/3263587702 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:15:28.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.437+0000 7ff40b400640 1 --2- 172.21.15.19:0/3263587702 >> v2:172.21.15.188:3300/0 conn(0x7ff4140d4ee0 0x7ff4140cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.437+0000 7ff418a00640 1 --2- 172.21.15.19:0/3263587702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4140d0bc0 0x7ff4140d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.437+0000 7ff413400640 1 -- 172.21.15.19:0/3263587702 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 607072616 0 0) 0x7ff4140d0180 con 0x7ff41415e500 2024-03-28T13:15:28.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.437+0000 7ff413400640 1 -- 172.21.15.19:0/3263587702 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3f8003520 con 0x7ff41415e500 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.437+0000 7ff413400640 1 -- 172.21.15.19:0/3263587702 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1279992648 0 0) 0x7ff3f8003520 con 0x7ff41415e500 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.437+0000 7ff413400640 1 -- 172.21.15.19:0/3263587702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4140d0bc0 msgr2=0x7ff4140d1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.437+0000 7ff413400640 1 --2- 172.21.15.19:0/3263587702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4140d0bc0 0x7ff4140d1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.437+0000 7ff413400640 1 -- 172.21.15.19:0/3263587702 >> v2:172.21.15.188:3300/0 conn(0x7ff4140d4ee0 msgr2=0x7ff4140cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.437+0000 7ff413400640 1 --2- 172.21.15.19:0/3263587702 >> v2:172.21.15.188:3300/0 conn(0x7ff4140d4ee0 0x7ff4140cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.437+0000 7ff413400640 1 -- 172.21.15.19:0/3263587702 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4140d2380 con 0x7ff41415e500 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.438+0000 7ff413400640 1 -- 172.21.15.19:0/3263587702 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7ff4040019f0 con 0x7ff41415e500 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.438+0000 7ff40b400640 1 --2- 172.21.15.19:0/3263587702 >> v2:172.21.15.188:3300/0 conn(0x7ff4140d4ee0 0x7ff4140cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.438+0000 7ff413400640 1 -- 172.21.15.19:0/3263587702 >> v1:172.21.15.19:6789/0 conn(0x7ff41415e500 legacy=0x7ff4140d2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.438+0000 7ff413400640 1 --2- 172.21.15.19:0/3263587702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3f80039b0 0x7ff3f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.438+0000 7ff413400640 1 --2- 172.21.15.19:0/3263587702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4140d0bc0 0x7ff3f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.438+0000 7ff413400640 1 --2- 172.21.15.19:0/3263587702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3f8004d40 0x7ff3f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.438+0000 7ff413400640 1 -- 172.21.15.19:0/3263587702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4140d1dc0 con 0x7ff3f8004d40 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.438+0000 7ff413400640 1 -- 172.21.15.19:0/3263587702 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff4140d20a0 con 0x7ff3f80039b0 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.438+0000 7ff413400640 1 -- 172.21.15.19:0/3263587702 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f800baa0 con 0x7ff4140d0bc0 2024-03-28T13:15:28.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.438+0000 7ff418a00640 1 --2- 172.21.15.19:0/3263587702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3f8004d40 0x7ff3f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.438+0000 7ff413e00640 1 --2- 172.21.15.19:0/3263587702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3f80039b0 0x7ff3f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.438+0000 7ff40b400640 1 --2- 172.21.15.19:0/3263587702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4140d0bc0 0x7ff3f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.439+0000 7ff418a00640 1 -- 172.21.15.19:0/3263587702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4140d0bc0 msgr2=0x7ff3f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.439+0000 7ff418a00640 1 --2- 172.21.15.19:0/3263587702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4140d0bc0 0x7ff3f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.439+0000 7ff418a00640 1 -- 172.21.15.19:0/3263587702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3f80039b0 msgr2=0x7ff3f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.439+0000 7ff418a00640 1 --2- 172.21.15.19:0/3263587702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3f80039b0 0x7ff3f8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.439+0000 7ff418a00640 1 -- 172.21.15.19:0/3263587702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f8003520 con 0x7ff3f8004d40 2024-03-28T13:15:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.440+0000 7ff418a00640 1 --2- 172.21.15.19:0/3263587702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3f8004d40 0x7ff3f80051c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009ac0 tx=0x7ff3f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac022de9d622c145 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.440+0000 7ff413400640 1 -- 172.21.15.19:0/3263587702 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c039070 con 0x7ff3f8004d40 2024-03-28T13:15:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.440+0000 7ff413400640 1 -- 172.21.15.19:0/3263587702 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40c02c6b0 con 0x7ff3f8004d40 2024-03-28T13:15:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.440+0000 7ff413400640 1 -- 172.21.15.19:0/3263587702 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c009690 con 0x7ff3f8004d40 2024-03-28T13:15:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.441+0000 7ff419400640 1 -- 172.21.15.19:0/3263587702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3f8004d40 msgr2=0x7ff3f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.441+0000 7ff419400640 1 --2- 172.21.15.19:0/3263587702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3f8004d40 0x7ff3f80051c0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009ac0 tx=0x7ff3f8005830 comp rx=0 tx=0).stop 2024-03-28T13:15:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.441+0000 7ff418a00640 1 -- 172.21.15.19:0/3263587702 reap_dead start 2024-03-28T13:15:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.441+0000 7ff419400640 1 -- 172.21.15.19:0/3263587702 shutdown_connections 2024-03-28T13:15:28.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.441+0000 7ff419400640 1 -- 172.21.15.19:0/3263587702 >> 172.21.15.19:0/3263587702 conn(0x7ff41409d530 msgr2=0x7ff4140d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.441+0000 7ff419400640 1 -- 172.21.15.19:0/3263587702 shutdown_connections 2024-03-28T13:15:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.441+0000 7ff419400640 1 -- 172.21.15.19:0/3263587702 wait complete. 2024-03-28T13:15:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.443+0000 7ff419400640 1 Processor -- start 2024-03-28T13:15:28.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.443+0000 7ff419400640 1 -- start start 2024-03-28T13:15:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff419400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff414154560 0x7ff414161620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff419400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff414154940 0x7ff4140c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff419400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff41415ed70 0x7ff41415f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff419400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff4140cfdc0 con 0x7ff414154940 2024-03-28T13:15:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff419400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff41415e8e0 con 0x7ff41415ed70 2024-03-28T13:15:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff419400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff4140cff40 con 0x7ff414154560 2024-03-28T13:15:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff40b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff414154940 0x7ff4140c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff40b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff414154940 0x7ff4140c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59234/0 (socket says 172.21.15.19:59234) 2024-03-28T13:15:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff40b400640 1 -- 172.21.15.19:0/1128298368 learned_addr learned my addr 172.21.15.19:0/1128298368 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff413e00640 1 --2- 172.21.15.19:0/1128298368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff414154560 0x7ff414161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff40b400640 1 -- 172.21.15.19:0/1128298368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff414154560 msgr2=0x7ff414161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff40b400640 1 --2- 172.21.15.19:0/1128298368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff414154560 0x7ff414161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff418a00640 1 --2- 172.21.15.19:0/1128298368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff41415ed70 0x7ff41415f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff40b400640 1 -- 172.21.15.19:0/1128298368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff41415ed70 msgr2=0x7ff41415f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff40b400640 1 --2- 172.21.15.19:0/1128298368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff41415ed70 0x7ff41415f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.444+0000 7ff40b400640 1 -- 172.21.15.19:0/1128298368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4140c8e20 con 0x7ff414154940 2024-03-28T13:15:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.445+0000 7ff418a00640 1 --2- 172.21.15.19:0/1128298368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff41415ed70 0x7ff41415f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.445+0000 7ff40b400640 1 --2- 172.21.15.19:0/1128298368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff414154940 0x7ff4140c95e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0116b0 tx=0x7ff3fc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:28.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.445+0000 7ff410c00640 1 -- 172.21.15.19:0/1128298368 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc019070 con 0x7ff414154940 2024-03-28T13:15:28.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.445+0000 7ff410c00640 1 -- 172.21.15.19:0/1128298368 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3fc004220 con 0x7ff414154940 2024-03-28T13:15:28.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.445+0000 7ff410c00640 1 -- 172.21.15.19:0/1128298368 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc00f040 con 0x7ff414154940 2024-03-28T13:15:28.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.445+0000 7ff419400640 1 -- 172.21.15.19:0/1128298368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4140cba20 con 0x7ff414154940 2024-03-28T13:15:28.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.445+0000 7ff419400640 1 -- 172.21.15.19:0/1128298368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4140cb740 con 0x7ff414154940 2024-03-28T13:15:28.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.447+0000 7ff410c00640 1 -- 172.21.15.19:0/1128298368 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff3fc00f7c0 con 0x7ff414154940 2024-03-28T13:15:28.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.449+0000 7ff410c00640 1 --2- 172.21.15.19:0/1128298368 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff3f006c530 0x7ff3f006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.449+0000 7ff410c00640 1 -- 172.21.15.19:0/1128298368 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7ff3fc08bb80 con 0x7ff414154940 2024-03-28T13:15:28.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.449+0000 7ff41d5ed740 1 Processor -- start 2024-03-28T13:15:28.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.449+0000 7ff41d5ed740 1 -- start start 2024-03-28T13:15:28.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.450+0000 7ff413e00640 1 --2- 172.21.15.19:0/1128298368 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff3f006c530 0x7ff3f006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.450+0000 7ff41d5ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee398300 0x55cdee3986e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.450+0000 7ff41d5ed740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee368320 0x55cdee395570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.450+0000 7ff41d5ed740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee36be80 0x55cdee395ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.450+0000 7ff41d5ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cdee2d2450 con 0x55cdee398300 2024-03-28T13:15:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.450+0000 7ff41d5ed740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cdee27a8c0 con 0x55cdee36be80 2024-03-28T13:15:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.450+0000 7ff41d5ed740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55cdee2ce400 con 0x55cdee368320 2024-03-28T13:15:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.451+0000 7ff413e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee398300 0x55cdee3986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.451+0000 7ff413e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee398300 0x55cdee3986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59246/0 (socket says 172.21.15.19:59246) 2024-03-28T13:15:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.451+0000 7ff413e00640 1 -- 172.21.15.19:0/473638966 learned_addr learned my addr 172.21.15.19:0/473638966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.451+0000 7ff40b400640 1 --2- 172.21.15.19:0/473638966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee368320 0x55cdee395570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.451+0000 7ff413e00640 1 -- 172.21.15.19:0/473638966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee368320 msgr2=0x55cdee395570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.451+0000 7ff413e00640 1 --2- 172.21.15.19:0/473638966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee368320 0x55cdee395570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.451+0000 7ff418a00640 1 --2- 172.21.15.19:0/473638966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee36be80 0x55cdee395ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.451+0000 7ff413e00640 1 -- 172.21.15.19:0/473638966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee36be80 msgr2=0x55cdee395ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.451+0000 7ff413e00640 1 --2- 172.21.15.19:0/473638966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee36be80 0x55cdee395ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.451+0000 7ff413e00640 1 -- 172.21.15.19:0/473638966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdee0aea50 con 0x55cdee398300 2024-03-28T13:15:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.451+0000 7ff413e00640 1 --2- 172.21.15.19:0/1128298368 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff3f006c530 0x7ff3f006e9f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff4040013d0 tx=0x7ff404006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.451+0000 7ff413e00640 1 --2- 172.21.15.19:0/473638966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee398300 0x55cdee3986e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff40400a760 tx=0x7ff40401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11c424188155de2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.452+0000 7ff403e00640 1 -- 172.21.15.19:0/473638966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff404025070 con 0x55cdee398300 2024-03-28T13:15:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.452+0000 7ff403e00640 1 -- 172.21.15.19:0/473638966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff404016a80 con 0x55cdee398300 2024-03-28T13:15:28.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.452+0000 7ff403e00640 1 -- 172.21.15.19:0/473638966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40400ae60 con 0x55cdee398300 2024-03-28T13:15:28.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.452+0000 7ff41d5ed740 1 -- 172.21.15.19:0/473638966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee398300 msgr2=0x55cdee3986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.452+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/473638966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee398300 0x55cdee3986e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff40400a760 tx=0x7ff40401c000 comp rx=0 tx=0).stop 2024-03-28T13:15:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.452+0000 7ff41d5ed740 1 -- 172.21.15.19:0/473638966 shutdown_connections 2024-03-28T13:15:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.452+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/473638966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee36be80 0x55cdee395ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.452+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/473638966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee368320 0x55cdee395570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.452+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/473638966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee398300 0x55cdee3986e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.452+0000 7ff41d5ed740 1 -- 172.21.15.19:0/473638966 >> 172.21.15.19:0/473638966 conn(0x55cdee36a590 msgr2=0x55cdee3b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.452+0000 7ff41d5ed740 1 -- 172.21.15.19:0/473638966 shutdown_connections 2024-03-28T13:15:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.453+0000 7ff41d5ed740 1 -- 172.21.15.19:0/473638966 wait complete. 2024-03-28T13:15:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.453+0000 7ff41d5ed740 1 -- start start 2024-03-28T13:15:28.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.453+0000 7ff41d5ed740 1 Processor -- start 2024-03-28T13:15:28.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.454+0000 7ff41d5ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee36be80 0x55cdee3b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.454+0000 7ff41d5ed740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee398300 0x55cdee39bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.455+0000 7ff41d5ed740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee3924a0 0x55cdee392920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.455+0000 7ff41d5ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cdee36d030 con 0x55cdee36be80 2024-03-28T13:15:28.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.455+0000 7ff41d5ed740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55cdee28dc70 con 0x55cdee3924a0 2024-03-28T13:15:28.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.455+0000 7ff41d5ed740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55cdee36c420 con 0x55cdee398300 2024-03-28T13:15:28.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.455+0000 7ff41d5ed740 10 client.0 _ll_register_callbacks cb 0x55cdee28d660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:15:28.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.455+0000 7ff413e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee36be80 0x55cdee3b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.455+0000 7ff413e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee36be80 0x55cdee3b3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59256/0 (socket says 172.21.15.19:59256) 2024-03-28T13:15:28.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.455+0000 7ff413e00640 1 -- 172.21.15.19:0/4144810687 learned_addr learned my addr 172.21.15.19:0/4144810687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:28.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.455+0000 7ff413e00640 1 -- 172.21.15.19:0/4144810687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee398300 msgr2=0x55cdee39bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.455+0000 7ff413e00640 1 --2- 172.21.15.19:0/4144810687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee398300 0x55cdee39bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.455+0000 7ff413e00640 1 -- 172.21.15.19:0/4144810687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee3924a0 msgr2=0x55cdee392920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.455+0000 7ff413e00640 1 --2- 172.21.15.19:0/4144810687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee3924a0 0x55cdee392920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.455+0000 7ff413e00640 1 -- 172.21.15.19:0/4144810687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdedfa63e0 con 0x55cdee36be80 2024-03-28T13:15:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.456+0000 7ff413e00640 1 --2- 172.21.15.19:0/4144810687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee36be80 0x55cdee3b3720 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff404018040 tx=0x7ff4140d0350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e8f81b764f7c41 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.456+0000 7ff400c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:15:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.456+0000 7ff400c00640 1 -- 172.21.15.19:0/4144810687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40402f020 con 0x55cdee36be80 2024-03-28T13:15:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.456+0000 7ff400c00640 1 -- 172.21.15.19:0/4144810687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7ff40402b620 con 0x55cdee36be80 2024-03-28T13:15:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.456+0000 7ff41d5ed740 1 -- 172.21.15.19:0/4144810687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff404016a80 con 0x55cdee36be80 2024-03-28T13:15:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.456+0000 7ff400c00640 1 -- 172.21.15.19:0/4144810687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff404039050 con 0x55cdee36be80 2024-03-28T13:15:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.456+0000 7ff41d5ed740 10 client.6903 fetch_fsmap learned FSMap version 52 2024-03-28T13:15:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.456+0000 7ff41d5ed740 1 -- 172.21.15.19:0/4144810687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55cdee10f1e0 con 0x55cdee36be80 2024-03-28T13:15:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.456+0000 7ff400c00640 1 -- 172.21.15.19:0/4144810687 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff404016a80 con 0x55cdee36be80 2024-03-28T13:15:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.457+0000 7ff400c00640 1 -- 172.21.15.19:0/4144810687 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7ff40403c020 con 0x55cdee36be80 2024-03-28T13:15:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.457+0000 7ff41d5ed740 10 client.6903 fetch_fsmap finished waiting for FSMap version 52 2024-03-28T13:15:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.457+0000 7ff41d5ed740 10 client.6903 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-28T13:15:28.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.457+0000 7ff41d5ed740 20 client.6903 populate_metadata read hostname 'smithi019' 2024-03-28T13:15:28.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.457+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/4144810687 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55cdee3c74e0 0x55cdee3c7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.457+0000 7ff41d5ed740 4 client.6903 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:15:28.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.457+0000 7ff41d5ed740 1 -- 172.21.15.19:0/4144810687 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdee2df990 con 0x55cdee3c74e0 2024-03-28T13:15:28.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.457+0000 7ff40b400640 1 --2- 172.21.15.19:0/4144810687 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55cdee3c74e0 0x55cdee3c7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.458+0000 7ff40b400640 1 --2- 172.21.15.19:0/4144810687 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55cdee3c74e0 0x55cdee3c7960 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12d5e58a664a8e82 server_cookie=ca25d725cc0bc12e in_seq=0 out_seq=0 2024-03-28T13:15:28.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.458+0000 7ff400c00640 10 client.6903 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:28.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.458+0000 7ff400c00640 1 -- 172.21.15.19:0/4144810687 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cdee2df990 con 0x55cdee3c74e0 2024-03-28T13:15:28.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.458+0000 7ff400c00640 10 client.6903 handle_command_reply: tid=0 2024-03-28T13:15:28.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.459+0000 7ff41d5ed740 1 client.6903 shutdown 2024-03-28T13:15:28.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.459+0000 7ff41d5ed740 1 -- 172.21.15.19:0/4144810687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee36be80 msgr2=0x55cdee3b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.459+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/4144810687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee36be80 0x55cdee3b3720 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff404018040 tx=0x7ff4140d0350 comp rx=0 tx=0).stop 2024-03-28T13:15:28.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.459+0000 7ff41d5ed740 1 -- 172.21.15.19:0/4144810687 shutdown_connections 2024-03-28T13:15:28.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.459+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/4144810687 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55cdee3c74e0 0x55cdee3c7960 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.459+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/4144810687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee3924a0 0x55cdee392920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.459+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/4144810687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee398300 0x55cdee39bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.459+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/4144810687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee36be80 0x55cdee3b3720 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.459+0000 7ff400c00640 0 client.6903 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:28.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.459+0000 7ff41d5ed740 1 -- 172.21.15.19:0/4144810687 >> 172.21.15.19:0/4144810687 conn(0x55cdee368320 msgr2=0x55cdee3b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:28.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.459+0000 7ff41d5ed740 1 -- 172.21.15.19:0/4144810687 shutdown_connections 2024-03-28T13:15:28.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.459+0000 7ff41d5ed740 1 -- 172.21.15.19:0/4144810687 wait complete. 2024-03-28T13:15:28.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.459+0000 7ff41d5ed740 20 client.6903 trim_cache size 0 max 16384 2024-03-28T13:15:28.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.479+0000 7ff41d5ed740 1 Processor -- start 2024-03-28T13:15:28.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.479+0000 7ff41d5ed740 1 -- start start 2024-03-28T13:15:28.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.479+0000 7ff41d5ed740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee36be80 0x55cdee395390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.479+0000 7ff41d5ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee3924a0 0x55cdee39b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.479+0000 7ff41d5ed740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee398300 0x55cdee390f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.480+0000 7ff41d5ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff414201800 con 0x55cdee3924a0 2024-03-28T13:15:28.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.480+0000 7ff41d5ed740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff4142007e0 con 0x55cdee398300 2024-03-28T13:15:28.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.480+0000 7ff41d5ed740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff4141ffb70 con 0x55cdee36be80 2024-03-28T13:15:28.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.480+0000 7ff40b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee3924a0 0x55cdee39b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.480+0000 7ff40b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee3924a0 0x55cdee39b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59264/0 (socket says 172.21.15.19:59264) 2024-03-28T13:15:28.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.480+0000 7ff40b400640 1 -- 172.21.15.19:0/101548727 learned_addr learned my addr 172.21.15.19:0/101548727 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:28.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.480+0000 7ff40b400640 1 -- 172.21.15.19:0/101548727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee36be80 msgr2=0x55cdee395390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.480+0000 7ff40b400640 1 --2- 172.21.15.19:0/101548727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee36be80 0x55cdee395390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.480+0000 7ff40b400640 1 -- 172.21.15.19:0/101548727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee398300 msgr2=0x55cdee390f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.480+0000 7ff418a00640 1 --2- 172.21.15.19:0/101548727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee398300 0x55cdee390f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.480+0000 7ff40b400640 1 --2- 172.21.15.19:0/101548727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee398300 0x55cdee390f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.480+0000 7ff40b400640 1 -- 172.21.15.19:0/101548727 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdedf7f4a0 con 0x55cdee3924a0 2024-03-28T13:15:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.481+0000 7ff40b400640 1 --2- 172.21.15.19:0/101548727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee3924a0 0x55cdee39b6c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc059900 tx=0x7ff3fc059400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f57f0138f742c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.481+0000 7ff403400640 1 -- 172.21.15.19:0/101548727 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc022040 con 0x55cdee3924a0 2024-03-28T13:15:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.481+0000 7ff403400640 1 -- 172.21.15.19:0/101548727 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3fc076070 con 0x55cdee3924a0 2024-03-28T13:15:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.481+0000 7ff403400640 1 -- 172.21.15.19:0/101548727 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc055fe0 con 0x55cdee3924a0 2024-03-28T13:15:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.481+0000 7ff41d5ed740 1 -- 172.21.15.19:0/101548727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee3924a0 msgr2=0x55cdee39b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.481+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/101548727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee3924a0 0x55cdee39b6c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc059900 tx=0x7ff3fc059400 comp rx=0 tx=0).stop 2024-03-28T13:15:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.481+0000 7ff41d5ed740 1 -- 172.21.15.19:0/101548727 shutdown_connections 2024-03-28T13:15:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.481+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/101548727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee398300 0x55cdee390f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.481+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/101548727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee3924a0 0x55cdee39b6c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.481+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/101548727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee36be80 0x55cdee395390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.481+0000 7ff41d5ed740 1 -- 172.21.15.19:0/101548727 >> 172.21.15.19:0/101548727 conn(0x55cdee368320 msgr2=0x55cdee3b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:28.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.482+0000 7ff41d5ed740 1 -- 172.21.15.19:0/101548727 shutdown_connections 2024-03-28T13:15:28.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.482+0000 7ff41d5ed740 1 -- 172.21.15.19:0/101548727 wait complete. 2024-03-28T13:15:28.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.482+0000 7ff41d5ed740 1 -- start start 2024-03-28T13:15:28.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.483+0000 7ff41d5ed740 1 Processor -- start 2024-03-28T13:15:28.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.483+0000 7ff41d5ed740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee36be80 0x55cdee42d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff41d5ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee3924a0 0x55cdee398910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff41d5ed740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee398300 0x55cdee38f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff41d5ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cdee2df990 con 0x55cdee3924a0 2024-03-28T13:15:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff41d5ed740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff414200190 con 0x55cdee398300 2024-03-28T13:15:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff41d5ed740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55cdee395b80 con 0x55cdee36be80 2024-03-28T13:15:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff41d5ed740 10 client.0 _ll_register_callbacks cb 0x7ff404001a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:15:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff40b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee3924a0 0x55cdee398910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff40b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee3924a0 0x55cdee398910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59280/0 (socket says 172.21.15.19:59280) 2024-03-28T13:15:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff40b400640 1 -- 172.21.15.19:0/2302359799 learned_addr learned my addr 172.21.15.19:0/2302359799 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff413e00640 1 --2- 172.21.15.19:0/2302359799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee36be80 0x55cdee42d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff418a00640 1 --2- 172.21.15.19:0/2302359799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee398300 0x55cdee38f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff40b400640 1 -- 172.21.15.19:0/2302359799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee36be80 msgr2=0x55cdee42d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff40b400640 1 --2- 172.21.15.19:0/2302359799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee36be80 0x55cdee42d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff40b400640 1 -- 172.21.15.19:0/2302359799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee398300 msgr2=0x55cdee38f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff40b400640 1 --2- 172.21.15.19:0/2302359799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee398300 0x55cdee38f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff40b400640 1 -- 172.21.15.19:0/2302359799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdee36f140 con 0x55cdee3924a0 2024-03-28T13:15:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.484+0000 7ff413e00640 1 --2- 172.21.15.19:0/2302359799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee36be80 0x55cdee42d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.485+0000 7ff40b400640 1 --2- 172.21.15.19:0/2302359799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee3924a0 0x55cdee398910 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc059900 tx=0x7ff3fc01f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ee5d9868ccce19 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.485+0000 7ff400c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:15:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.485+0000 7ff400c00640 1 -- 172.21.15.19:0/2302359799 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc022030 con 0x55cdee3924a0 2024-03-28T13:15:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.485+0000 7ff400c00640 1 -- 172.21.15.19:0/2302359799 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7ff3fc0620c0 con 0x55cdee3924a0 2024-03-28T13:15:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.485+0000 7ff41d5ed740 1 -- 172.21.15.19:0/2302359799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3fc076070 con 0x55cdee3924a0 2024-03-28T13:15:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.485+0000 7ff400c00640 1 -- 172.21.15.19:0/2302359799 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff3fc089070 con 0x55cdee3924a0 2024-03-28T13:15:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.485+0000 7ff41d5ed740 10 client.6909 fetch_fsmap learned FSMap version 52 2024-03-28T13:15:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.485+0000 7ff41d5ed740 1 -- 172.21.15.19:0/2302359799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55cdee3994a0 con 0x55cdee3924a0 2024-03-28T13:15:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.485+0000 7ff400c00640 1 -- 172.21.15.19:0/2302359799 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3fc076070 con 0x55cdee3924a0 2024-03-28T13:15:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.486+0000 7ff400c00640 1 -- 172.21.15.19:0/2302359799 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7ff3fc058dc0 con 0x55cdee3924a0 2024-03-28T13:15:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.486+0000 7ff41d5ed740 10 client.6909 fetch_fsmap finished waiting for FSMap version 52 2024-03-28T13:15:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.486+0000 7ff41d5ed740 10 client.6909 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-28T13:15:28.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.486+0000 7ff41d5ed740 20 client.6909 populate_metadata read hostname 'smithi019' 2024-03-28T13:15:28.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.486+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/2302359799 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55cdee392900 0x55cdee38fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.486+0000 7ff41d5ed740 4 client.6909 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-03-28T13:15:28.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.486+0000 7ff41d5ed740 1 -- 172.21.15.19:0/2302359799 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x55cdee2d9ce0 con 0x55cdee392900 2024-03-28T13:15:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.486+0000 7ff413e00640 1 --2- 172.21.15.19:0/2302359799 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55cdee392900 0x55cdee38fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.487+0000 7ff413e00640 1 --2- 172.21.15.19:0/2302359799 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55cdee392900 0x55cdee38fda0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d62df8e36fafae6 server_cookie=1ec3e579a30e247a in_seq=0 out_seq=0 2024-03-28T13:15:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.487+0000 7ff400c00640 10 client.6909 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.487+0000 7ff400c00640 1 -- 172.21.15.19:0/2302359799 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cdee2d9ce0 con 0x55cdee392900 2024-03-28T13:15:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.487+0000 7ff400c00640 10 client.6909 handle_command_reply: tid=0 2024-03-28T13:15:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.487+0000 7ff41d5ed740 1 client.6909 shutdown 2024-03-28T13:15:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.488+0000 7ff41d5ed740 1 -- 172.21.15.19:0/2302359799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee3924a0 msgr2=0x55cdee398910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.488+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/2302359799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee3924a0 0x55cdee398910 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc059900 tx=0x7ff3fc01f040 comp rx=0 tx=0).stop 2024-03-28T13:15:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.488+0000 7ff41d5ed740 1 -- 172.21.15.19:0/2302359799 shutdown_connections 2024-03-28T13:15:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.488+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/2302359799 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55cdee392900 0x55cdee38fda0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.488+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/2302359799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55cdee398300 0x55cdee38f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.488+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/2302359799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cdee3924a0 0x55cdee398910 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.488+0000 7ff41d5ed740 1 --2- 172.21.15.19:0/2302359799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55cdee36be80 0x55cdee42d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.488+0000 7ff400c00640 0 client.6909 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.488+0000 7ff41d5ed740 1 -- 172.21.15.19:0/2302359799 >> 172.21.15.19:0/2302359799 conn(0x55cdee368320 msgr2=0x55cdee3671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.488+0000 7ff41d5ed740 1 -- 172.21.15.19:0/2302359799 shutdown_connections 2024-03-28T13:15:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.488+0000 7ff41d5ed740 1 -- 172.21.15.19:0/2302359799 wait complete. 2024-03-28T13:15:28.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.488+0000 7ff41d5ed740 20 client.6909 trim_cache size 0 max 16384 2024-03-28T13:15:28.490 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:15:28.490 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-03-28T13:15:28.490 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "d8edb31f-bfa3-424c-8e03-5ef3a25182f9", 2024-03-28T13:15:28.490 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-03-28T13:15:28.490 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:15:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.489+0000 7ff419400640 1 -- 172.21.15.19:0/1128298368 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff3f006c530 msgr2=0x7ff3f006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.489+0000 7ff419400640 1 --2- 172.21.15.19:0/1128298368 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff3f006c530 0x7ff3f006e9f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff4040013d0 tx=0x7ff404006040 comp rx=0 tx=0).stop 2024-03-28T13:15:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.489+0000 7ff419400640 1 -- 172.21.15.19:0/1128298368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff414154940 msgr2=0x7ff4140c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.489+0000 7ff419400640 1 --2- 172.21.15.19:0/1128298368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff414154940 0x7ff4140c95e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0116b0 tx=0x7ff3fc011b70 comp rx=0 tx=0).stop 2024-03-28T13:15:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.489+0000 7ff419400640 1 -- 172.21.15.19:0/1128298368 shutdown_connections 2024-03-28T13:15:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.489+0000 7ff419400640 1 --2- 172.21.15.19:0/1128298368 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff3f006c530 0x7ff3f006e9f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.489+0000 7ff419400640 1 --2- 172.21.15.19:0/1128298368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff41415ed70 0x7ff41415f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.489+0000 7ff419400640 1 --2- 172.21.15.19:0/1128298368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff414154940 0x7ff4140c95e0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.489+0000 7ff419400640 1 --2- 172.21.15.19:0/1128298368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff414154560 0x7ff414161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.489+0000 7ff419400640 1 -- 172.21.15.19:0/1128298368 >> 172.21.15.19:0/1128298368 conn(0x7ff41409d530 msgr2=0x7ff4140d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:28.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.489+0000 7ff419400640 1 -- 172.21.15.19:0/1128298368 shutdown_connections 2024-03-28T13:15:28.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.489+0000 7ff419400640 1 -- 172.21.15.19:0/1128298368 wait complete. 2024-03-28T13:15:28.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:15:28.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-28T13:15:28.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.712+0000 7ff078c00640 1 Processor -- start 2024-03-28T13:15:28.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.713+0000 7ff078c00640 1 -- start start 2024-03-28T13:15:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.714+0000 7ff078c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff0740d43d0 0x7ff0740d47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.714+0000 7ff078c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0740cf6b0 0x7ff0740cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.714+0000 7ff078c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0740d4d20 con 0x7ff07415e860 2024-03-28T13:15:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.714+0000 7ff078c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff0740cf220 con 0x7ff0740d43d0 2024-03-28T13:15:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.714+0000 7ff078c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff07406dfa0 con 0x7ff0740cf6b0 2024-03-28T13:15:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.714+0000 7ff072800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0740cf6b0 0x7ff0740cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.714+0000 7ff071400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff0740d43d0 0x7ff0740d47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.715+0000 7ff072800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0740cf6b0 0x7ff0740cfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:42544/0 (socket says 172.21.15.19:42544) 2024-03-28T13:15:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.715+0000 7ff072800640 1 -- 172.21.15.19:0/2720498958 learned_addr learned my addr 172.21.15.19:0/2720498958 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:28.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.715+0000 7ff072800640 1 -- 172.21.15.19:0/2720498958 >> v2:172.21.15.188:3300/0 conn(0x7ff0740d43d0 msgr2=0x7ff0740d47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.715+0000 7ff072800640 1 --2- 172.21.15.19:0/2720498958 >> v2:172.21.15.188:3300/0 conn(0x7ff0740d43d0 0x7ff0740d47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.715+0000 7ff072800640 1 -- 172.21.15.19:0/2720498958 >> v1:172.21.15.188:6789/0 conn(0x7ff07415e860 legacy=0x7ff07406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.715+0000 7ff072800640 1 -- 172.21.15.19:0/2720498958 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07406e240 con 0x7ff0740cf6b0 2024-03-28T13:15:28.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.717+0000 7ff072800640 1 --2- 172.21.15.19:0/2720498958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0740cf6b0 0x7ff0740cfae0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009e50 tx=0x7ff06c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fdcbf712bc8dad5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:28.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.717+0000 7ff070a00640 1 -- 172.21.15.19:0/2720498958 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c02cb00 con 0x7ff0740cf6b0 2024-03-28T13:15:28.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.717+0000 7ff070a00640 1 -- 172.21.15.19:0/2720498958 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06c02cca0 con 0x7ff0740cf6b0 2024-03-28T13:15:28.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.717+0000 7ff070a00640 1 -- 172.21.15.19:0/2720498958 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c034610 con 0x7ff0740cf6b0 2024-03-28T13:15:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.718+0000 7ff078c00640 1 -- 172.21.15.19:0/2720498958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0740cf6b0 msgr2=0x7ff0740cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.718+0000 7ff078c00640 1 --2- 172.21.15.19:0/2720498958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0740cf6b0 0x7ff0740cfae0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009e50 tx=0x7ff06c02bf60 comp rx=0 tx=0).stop 2024-03-28T13:15:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.718+0000 7ff078c00640 1 -- 172.21.15.19:0/2720498958 shutdown_connections 2024-03-28T13:15:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.718+0000 7ff078c00640 1 --2- 172.21.15.19:0/2720498958 >> v2:172.21.15.188:3300/0 conn(0x7ff0740d43d0 0x7ff0740d47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.718+0000 7ff078c00640 1 --2- 172.21.15.19:0/2720498958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0740cf6b0 0x7ff0740cfae0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.718+0000 7ff078c00640 1 -- 172.21.15.19:0/2720498958 >> 172.21.15.19:0/2720498958 conn(0x7ff07409d7d0 msgr2=0x7ff07408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.718+0000 7ff078c00640 1 -- 172.21.15.19:0/2720498958 shutdown_connections 2024-03-28T13:15:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.718+0000 7ff078c00640 1 -- 172.21.15.19:0/2720498958 wait complete. 2024-03-28T13:15:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.720+0000 7ff078c00640 1 Processor -- start 2024-03-28T13:15:28.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.720+0000 7ff078c00640 1 -- start start 2024-03-28T13:15:28.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.721+0000 7ff078c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0740cf6b0 0x7ff07414db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.721+0000 7ff078c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0740d43d0 0x7ff0740cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.721+0000 7ff078c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff07415e860 0x7ff0740c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.721+0000 7ff078c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff07406dc00 con 0x7ff0740d43d0 2024-03-28T13:15:28.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.721+0000 7ff078c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff0740d2c50 con 0x7ff0740cf6b0 2024-03-28T13:15:28.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.721+0000 7ff078c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff07406dd80 con 0x7ff07415e860 2024-03-28T13:15:28.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.721+0000 7ff071400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0740d43d0 0x7ff0740cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.721+0000 7ff071400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0740d43d0 0x7ff0740cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59288/0 (socket says 172.21.15.19:59288) 2024-03-28T13:15:28.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.721+0000 7ff071400640 1 -- 172.21.15.19:0/2913624937 learned_addr learned my addr 172.21.15.19:0/2913624937 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:28.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.722+0000 7ff071e00640 1 --2- 172.21.15.19:0/2913624937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0740cf6b0 0x7ff07414db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.722+0000 7ff072800640 1 --2- 172.21.15.19:0/2913624937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff07415e860 0x7ff0740c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.722+0000 7ff071400640 1 -- 172.21.15.19:0/2913624937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff07415e860 msgr2=0x7ff0740c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.722+0000 7ff071400640 1 --2- 172.21.15.19:0/2913624937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff07415e860 0x7ff0740c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.722+0000 7ff071400640 1 -- 172.21.15.19:0/2913624937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0740cf6b0 msgr2=0x7ff07414db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.722+0000 7ff071400640 1 --2- 172.21.15.19:0/2913624937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0740cf6b0 0x7ff07414db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.722+0000 7ff071400640 1 -- 172.21.15.19:0/2913624937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0740cdb20 con 0x7ff0740d43d0 2024-03-28T13:15:28.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.722+0000 7ff072800640 1 --2- 172.21.15.19:0/2913624937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff07415e860 0x7ff0740c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:28.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.722+0000 7ff071400640 1 --2- 172.21.15.19:0/2913624937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0740d43d0 0x7ff0740cd0d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff06000e9b0 tx=0x7ff06000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:28.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.722+0000 7ff06a000640 1 -- 172.21.15.19:0/2913624937 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060017070 con 0x7ff0740d43d0 2024-03-28T13:15:28.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.723+0000 7ff06a000640 1 -- 172.21.15.19:0/2913624937 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06000c9a0 con 0x7ff0740d43d0 2024-03-28T13:15:28.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.723+0000 7ff06a000640 1 -- 172.21.15.19:0/2913624937 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06000cd90 con 0x7ff0740d43d0 2024-03-28T13:15:28.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.723+0000 7ff078c00640 1 -- 172.21.15.19:0/2913624937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0740cde00 con 0x7ff0740d43d0 2024-03-28T13:15:28.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.723+0000 7ff078c00640 1 -- 172.21.15.19:0/2913624937 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff074156fb0 con 0x7ff0740d43d0 2024-03-28T13:15:28.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.725+0000 7ff06a000640 1 -- 172.21.15.19:0/2913624937 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff06001a070 con 0x7ff0740d43d0 2024-03-28T13:15:28.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.726+0000 7ff06a000640 1 --2- 172.21.15.19:0/2913624937 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff05406c530 0x7ff05406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.726+0000 7ff06a000640 1 -- 172.21.15.19:0/2913624937 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7ff06008bb90 con 0x7ff0740d43d0 2024-03-28T13:15:28.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.726+0000 7ff07b992740 1 Processor -- start 2024-03-28T13:15:28.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.726+0000 7ff07b992740 1 -- start start 2024-03-28T13:15:28.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.727+0000 7ff071e00640 1 --2- 172.21.15.19:0/2913624937 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff05406c530 0x7ff05406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.727+0000 7ff07b992740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0560 0x555cfaca3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.727+0000 7ff07b992740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0940 0x555cfacd32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.727+0000 7ff07b992740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfaca3a40 0x555cfacd3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.727+0000 7ff07b992740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555cfac0e6b0 con 0x555cfaca3a40 2024-03-28T13:15:28.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.727+0000 7ff07b992740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555cfabb6750 con 0x555cfacd0940 2024-03-28T13:15:28.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.727+0000 7ff07b992740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x555cfac0a660 con 0x555cfacd0560 2024-03-28T13:15:28.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.728+0000 7ff072800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfaca3a40 0x555cfacd3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.728+0000 7ff072800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfaca3a40 0x555cfacd3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59294/0 (socket says 172.21.15.19:59294) 2024-03-28T13:15:28.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.728+0000 7ff072800640 1 -- 172.21.15.19:0/914046323 learned_addr learned my addr 172.21.15.19:0/914046323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:28.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.728+0000 7ff071e00640 1 --2- 172.21.15.19:0/2913624937 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff05406c530 0x7ff05406e9f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff064000f60 tx=0x7ff064009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:28.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.728+0000 7ff071400640 1 --2- 172.21.15.19:0/914046323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0940 0x555cfacd32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.728+0000 7ff071e00640 1 --2- 172.21.15.19:0/914046323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0560 0x555cfaca3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.728+0000 7ff072800640 1 -- 172.21.15.19:0/914046323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0560 msgr2=0x555cfaca3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.728+0000 7ff072800640 1 --2- 172.21.15.19:0/914046323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0560 0x555cfaca3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.728+0000 7ff072800640 1 -- 172.21.15.19:0/914046323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0940 msgr2=0x555cfacd32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.728+0000 7ff072800640 1 --2- 172.21.15.19:0/914046323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0940 0x555cfacd32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.728+0000 7ff072800640 1 -- 172.21.15.19:0/914046323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cfa9ec7d0 con 0x555cfaca3a40 2024-03-28T13:15:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.728+0000 7ff071400640 1 --2- 172.21.15.19:0/914046323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0940 0x555cfacd32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.729+0000 7ff072800640 1 --2- 172.21.15.19:0/914046323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfaca3a40 0x555cfacd3830 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff06c036040 tx=0x7ff06c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e0ee0018d39ef server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.729+0000 7ff05e600640 1 -- 172.21.15.19:0/914046323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c02cb00 con 0x555cfaca3a40 2024-03-28T13:15:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.729+0000 7ff05e600640 1 -- 172.21.15.19:0/914046323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06c02ce50 con 0x555cfaca3a40 2024-03-28T13:15:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.729+0000 7ff05e600640 1 -- 172.21.15.19:0/914046323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c040670 con 0x555cfaca3a40 2024-03-28T13:15:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.729+0000 7ff07b992740 1 -- 172.21.15.19:0/914046323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfaca3a40 msgr2=0x555cfacd3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.729+0000 7ff07b992740 1 --2- 172.21.15.19:0/914046323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfaca3a40 0x555cfacd3830 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff06c036040 tx=0x7ff06c02e000 comp rx=0 tx=0).stop 2024-03-28T13:15:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.730+0000 7ff07b992740 1 -- 172.21.15.19:0/914046323 shutdown_connections 2024-03-28T13:15:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.730+0000 7ff07b992740 1 --2- 172.21.15.19:0/914046323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfaca3a40 0x555cfacd3830 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.730+0000 7ff07b992740 1 --2- 172.21.15.19:0/914046323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0940 0x555cfacd32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.730+0000 7ff07b992740 1 --2- 172.21.15.19:0/914046323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0560 0x555cfaca3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.730+0000 7ff07b992740 1 -- 172.21.15.19:0/914046323 >> 172.21.15.19:0/914046323 conn(0x555cfaca5770 msgr2=0x555cfacee2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:28.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.730+0000 7ff07b992740 1 -- 172.21.15.19:0/914046323 shutdown_connections 2024-03-28T13:15:28.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.730+0000 7ff07b992740 1 -- 172.21.15.19:0/914046323 wait complete. 2024-03-28T13:15:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.730+0000 7ff07b992740 1 -- start start 2024-03-28T13:15:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.731+0000 7ff07b992740 1 Processor -- start 2024-03-28T13:15:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff07b992740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0560 0x555cfacff470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff07b992740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0940 0x555cfaccb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff07b992740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacc9ba0 0x555cfaccbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff07b992740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555cfaca7e10 con 0x555cfacc9ba0 2024-03-28T13:15:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff07b992740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x555cfabc9a80 con 0x555cfacd0560 2024-03-28T13:15:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff07b992740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x555cfacaa650 con 0x555cfacd0940 2024-03-28T13:15:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff07b992740 10 client.0 _ll_register_callbacks cb 0x555cfabc9470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:15:28.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff072800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacc9ba0 0x555cfaccbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff072800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacc9ba0 0x555cfaccbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59306/0 (socket says 172.21.15.19:59306) 2024-03-28T13:15:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff072800640 1 -- 172.21.15.19:0/3368824647 learned_addr learned my addr 172.21.15.19:0/3368824647 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff071e00640 1 --2- 172.21.15.19:0/3368824647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0560 0x555cfacff470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff071400640 1 --2- 172.21.15.19:0/3368824647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0940 0x555cfaccb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff072800640 1 -- 172.21.15.19:0/3368824647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0940 msgr2=0x555cfaccb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff072800640 1 --2- 172.21.15.19:0/3368824647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0940 0x555cfaccb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff072800640 1 -- 172.21.15.19:0/3368824647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0560 msgr2=0x555cfacff470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff072800640 1 --2- 172.21.15.19:0/3368824647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0560 0x555cfacff470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff072800640 1 -- 172.21.15.19:0/3368824647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cfa9223f0 con 0x555cfacc9ba0 2024-03-28T13:15:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.732+0000 7ff071400640 1 --2- 172.21.15.19:0/3368824647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0940 0x555cfaccb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.733+0000 7ff072800640 1 --2- 172.21.15.19:0/3368824647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacc9ba0 0x555cfaccbcb0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009f70 tx=0x7ff06c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adff9c2c18e0d1f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.733+0000 7ff052a00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:15:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.733+0000 7ff052a00640 1 -- 172.21.15.19:0/3368824647 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c040c30 con 0x555cfacc9ba0 2024-03-28T13:15:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.733+0000 7ff052a00640 1 -- 172.21.15.19:0/3368824647 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7ff06c040dd0 con 0x555cfacc9ba0 2024-03-28T13:15:28.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.733+0000 7ff07b992740 1 -- 172.21.15.19:0/3368824647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff06c02ce50 con 0x555cfacc9ba0 2024-03-28T13:15:28.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.733+0000 7ff052a00640 1 -- 172.21.15.19:0/3368824647 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff06c04a420 con 0x555cfacc9ba0 2024-03-28T13:15:28.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.733+0000 7ff07b992740 10 client.6918 fetch_fsmap learned FSMap version 52 2024-03-28T13:15:28.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.733+0000 7ff07b992740 1 -- 172.21.15.19:0/3368824647 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x555cfa8e20f0 con 0x555cfacc9ba0 2024-03-28T13:15:28.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.734+0000 7ff052a00640 1 -- 172.21.15.19:0/3368824647 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06c02ce50 con 0x555cfacc9ba0 2024-03-28T13:15:28.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.734+0000 7ff052a00640 1 -- 172.21.15.19:0/3368824647 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7ff06c008040 con 0x555cfacc9ba0 2024-03-28T13:15:28.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.734+0000 7ff07b992740 10 client.6918 fetch_fsmap finished waiting for FSMap version 52 2024-03-28T13:15:28.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.734+0000 7ff07b992740 10 client.6918 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-28T13:15:28.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.734+0000 7ff07b992740 20 client.6918 populate_metadata read hostname 'smithi019' 2024-03-28T13:15:28.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.734+0000 7ff07b992740 1 --2- 172.21.15.19:0/3368824647 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x555cfad024b0 0x555cfad028c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.734+0000 7ff07b992740 4 client.6918 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:15:28.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.734+0000 7ff07b992740 1 -- 172.21.15.19:0/3368824647 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cfac1bbf0 con 0x555cfad024b0 2024-03-28T13:15:28.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.734+0000 7ff071e00640 1 --2- 172.21.15.19:0/3368824647 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x555cfad024b0 0x555cfad028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.735+0000 7ff071e00640 1 --2- 172.21.15.19:0/3368824647 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x555cfad024b0 0x555cfad028c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5612843edb07d619 server_cookie=440c648d830ac5cd in_seq=0 out_seq=0 2024-03-28T13:15:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.735+0000 7ff052a00640 10 client.6918 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.736+0000 7ff052a00640 1 -- 172.21.15.19:0/3368824647 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555cfac1bbf0 con 0x555cfad024b0 2024-03-28T13:15:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.736+0000 7ff052a00640 10 client.6918 handle_command_reply: tid=0 2024-03-28T13:15:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.736+0000 7ff07b992740 1 client.6918 shutdown 2024-03-28T13:15:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.736+0000 7ff07b992740 1 -- 172.21.15.19:0/3368824647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacc9ba0 msgr2=0x555cfaccbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.736+0000 7ff07b992740 1 --2- 172.21.15.19:0/3368824647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacc9ba0 0x555cfaccbcb0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009f70 tx=0x7ff06c004ab0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.736+0000 7ff07b992740 1 -- 172.21.15.19:0/3368824647 shutdown_connections 2024-03-28T13:15:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.736+0000 7ff07b992740 1 --2- 172.21.15.19:0/3368824647 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x555cfad024b0 0x555cfad028c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.736+0000 7ff07b992740 1 --2- 172.21.15.19:0/3368824647 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacc9ba0 0x555cfaccbcb0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.736+0000 7ff07b992740 1 --2- 172.21.15.19:0/3368824647 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0940 0x555cfaccb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.736+0000 7ff07b992740 1 --2- 172.21.15.19:0/3368824647 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0560 0x555cfacff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.736+0000 7ff07b992740 1 -- 172.21.15.19:0/3368824647 >> 172.21.15.19:0/3368824647 conn(0x555cfaca3a40 msgr2=0x555cfaca9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.736+0000 7ff052a00640 0 client.6918 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:28.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.736+0000 7ff07b992740 1 -- 172.21.15.19:0/3368824647 shutdown_connections 2024-03-28T13:15:28.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.736+0000 7ff07b992740 1 -- 172.21.15.19:0/3368824647 wait complete. 2024-03-28T13:15:28.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.736+0000 7ff07b992740 20 client.6918 trim_cache size 0 max 16384 2024-03-28T13:15:28.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.755+0000 7ff07b992740 1 Processor -- start 2024-03-28T13:15:28.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.756+0000 7ff07b992740 1 -- start start 2024-03-28T13:15:28.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.756+0000 7ff07b992740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacc9ba0 0x555cfaca74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.756+0000 7ff07b992740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0560 0x555cfacfe290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.756+0000 7ff07b992740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0940 0x555cfaccb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.756+0000 7ff07b992740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555cfaca4b40 con 0x555cfacc9ba0 2024-03-28T13:15:28.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.756+0000 7ff07b992740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff074200e90 con 0x555cfacd0560 2024-03-28T13:15:28.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.756+0000 7ff07b992740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff0741fff00 con 0x555cfacd0940 2024-03-28T13:15:28.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.757+0000 7ff071e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacc9ba0 0x555cfaca74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.757+0000 7ff071e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacc9ba0 0x555cfaca74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59310/0 (socket says 172.21.15.19:59310) 2024-03-28T13:15:28.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.757+0000 7ff071e00640 1 -- 172.21.15.19:0/2799875081 learned_addr learned my addr 172.21.15.19:0/2799875081 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:28.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.757+0000 7ff071e00640 1 -- 172.21.15.19:0/2799875081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0940 msgr2=0x555cfaccb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.757+0000 7ff071e00640 1 --2- 172.21.15.19:0/2799875081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0940 0x555cfaccb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.757+0000 7ff071400640 1 --2- 172.21.15.19:0/2799875081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0560 0x555cfacfe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.757+0000 7ff071e00640 1 -- 172.21.15.19:0/2799875081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0560 msgr2=0x555cfacfe290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.757+0000 7ff071e00640 1 --2- 172.21.15.19:0/2799875081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0560 0x555cfacfe290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.757+0000 7ff071e00640 1 -- 172.21.15.19:0/2799875081 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cfaca5010 con 0x555cfacc9ba0 2024-03-28T13:15:28.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.758+0000 7ff071e00640 1 --2- 172.21.15.19:0/2799875081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacc9ba0 0x555cfaca74a0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff064005210 tx=0x7ff064012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b98f63160332ed4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:28.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.758+0000 7ff05dc00640 1 -- 172.21.15.19:0/2799875081 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff064026440 con 0x555cfacc9ba0 2024-03-28T13:15:28.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.758+0000 7ff05dc00640 1 -- 172.21.15.19:0/2799875081 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06402a020 con 0x555cfacc9ba0 2024-03-28T13:15:28.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.758+0000 7ff05dc00640 1 -- 172.21.15.19:0/2799875081 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0640265e0 con 0x555cfacc9ba0 2024-03-28T13:15:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.758+0000 7ff07b992740 1 -- 172.21.15.19:0/2799875081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacc9ba0 msgr2=0x555cfaca74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.758+0000 7ff07b992740 1 --2- 172.21.15.19:0/2799875081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacc9ba0 0x555cfaca74a0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff064005210 tx=0x7ff064012000 comp rx=0 tx=0).stop 2024-03-28T13:15:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.759+0000 7ff07b992740 1 -- 172.21.15.19:0/2799875081 shutdown_connections 2024-03-28T13:15:28.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.759+0000 7ff07b992740 1 --2- 172.21.15.19:0/2799875081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0940 0x555cfaccb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.759+0000 7ff07b992740 1 --2- 172.21.15.19:0/2799875081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacd0560 0x555cfacfe290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.759+0000 7ff07b992740 1 --2- 172.21.15.19:0/2799875081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacc9ba0 0x555cfaca74a0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.759+0000 7ff07b992740 1 -- 172.21.15.19:0/2799875081 >> 172.21.15.19:0/2799875081 conn(0x555cfaca3a40 msgr2=0x555cfacca630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.759+0000 7ff07b992740 1 -- 172.21.15.19:0/2799875081 shutdown_connections 2024-03-28T13:15:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.759+0000 7ff07b992740 1 -- 172.21.15.19:0/2799875081 wait complete. 2024-03-28T13:15:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.759+0000 7ff07b992740 1 -- start start 2024-03-28T13:15:28.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.760+0000 7ff07b992740 1 Processor -- start 2024-03-28T13:15:28.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff07b992740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacc9ba0 0x555cfacd0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff07b992740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacd0560 0x555cfacd1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff07b992740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0940 0x555cfad11870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff07b992740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0740b8a10 con 0x555cfacd0560 2024-03-28T13:15:28.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff07b992740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff0740b83c0 con 0x555cfacc9ba0 2024-03-28T13:15:28.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff07b992740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x555cfacfeb40 con 0x555cfacd0940 2024-03-28T13:15:28.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff07b992740 10 client.0 _ll_register_callbacks cb 0x7ff06c02cce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:15:28.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff071400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacd0560 0x555cfacd1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff071400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacd0560 0x555cfacd1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59322/0 (socket says 172.21.15.19:59322) 2024-03-28T13:15:28.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff071400640 1 -- 172.21.15.19:0/2472221456 learned_addr learned my addr 172.21.15.19:0/2472221456 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:28.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff071e00640 1 --2- 172.21.15.19:0/2472221456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacc9ba0 0x555cfacd0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff072800640 1 --2- 172.21.15.19:0/2472221456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0940 0x555cfad11870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff071400640 1 -- 172.21.15.19:0/2472221456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0940 msgr2=0x555cfad11870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff071400640 1 --2- 172.21.15.19:0/2472221456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0940 0x555cfad11870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff071400640 1 -- 172.21.15.19:0/2472221456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacc9ba0 msgr2=0x555cfacd0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:28.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff071400640 1 --2- 172.21.15.19:0/2472221456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacc9ba0 0x555cfacd0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.761+0000 7ff071400640 1 -- 172.21.15.19:0/2472221456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cfacfe670 con 0x555cfacd0560 2024-03-28T13:15:28.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.762+0000 7ff071400640 1 --2- 172.21.15.19:0/2472221456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacd0560 0x555cfacd1260 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff060059680 tx=0x7ff06005a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1b0b384a92fe9e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:28.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.762+0000 7ff052a00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:15:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.762+0000 7ff052a00640 1 -- 172.21.15.19:0/2472221456 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060021070 con 0x555cfacd0560 2024-03-28T13:15:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.762+0000 7ff052a00640 1 -- 172.21.15.19:0/2472221456 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7ff060058dd0 con 0x555cfacd0560 2024-03-28T13:15:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.762+0000 7ff07b992740 1 -- 172.21.15.19:0/2472221456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff06402a020 con 0x555cfacd0560 2024-03-28T13:15:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.762+0000 7ff052a00640 1 -- 172.21.15.19:0/2472221456 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff060062560 con 0x555cfacd0560 2024-03-28T13:15:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.762+0000 7ff07b992740 10 client.6924 fetch_fsmap learned FSMap version 52 2024-03-28T13:15:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.762+0000 7ff07b992740 1 -- 172.21.15.19:0/2472221456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x555cfaceded0 con 0x555cfacd0560 2024-03-28T13:15:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.763+0000 7ff052a00640 1 -- 172.21.15.19:0/2472221456 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff06402a020 con 0x555cfacd0560 2024-03-28T13:15:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.763+0000 7ff052a00640 1 -- 172.21.15.19:0/2472221456 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 3494+0+0 (secure 0 0 0) 0x7ff0600556d0 con 0x555cfacd0560 2024-03-28T13:15:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.763+0000 7ff07b992740 10 client.6924 fetch_fsmap finished waiting for FSMap version 52 2024-03-28T13:15:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.763+0000 7ff07b992740 10 client.6924 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-28T13:15:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.763+0000 7ff07b992740 20 client.6924 populate_metadata read hostname 'smithi019' 2024-03-28T13:15:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.763+0000 7ff07b992740 1 --2- 172.21.15.19:0/2472221456 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x555cfad4d910 0x555cfad11db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.763+0000 7ff07b992740 4 client.6924 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:15:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.763+0000 7ff07b992740 1 -- 172.21.15.19:0/2472221456 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555cfac15f40 con 0x555cfad4d910 2024-03-28T13:15:28.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.763+0000 7ff071e00640 1 --2- 172.21.15.19:0/2472221456 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x555cfad4d910 0x555cfad11db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.764+0000 7ff071e00640 1 --2- 172.21.15.19:0/2472221456 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x555cfad4d910 0x555cfad11db0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c91d66fdae36dc8f server_cookie=e81118a45490cb95 in_seq=0 out_seq=0 2024-03-28T13:15:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.764+0000 7ff052a00640 10 client.6924 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.764+0000 7ff052a00640 1 -- 172.21.15.19:0/2472221456 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x555cfac15f40 con 0x555cfad4d910 2024-03-28T13:15:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.764+0000 7ff052a00640 10 client.6924 handle_command_reply: tid=0 2024-03-28T13:15:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.764+0000 7ff07b992740 1 client.6924 shutdown 2024-03-28T13:15:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.764+0000 7ff07b992740 1 -- 172.21.15.19:0/2472221456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacd0560 msgr2=0x555cfacd1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.764+0000 7ff07b992740 1 --2- 172.21.15.19:0/2472221456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacd0560 0x555cfacd1260 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff060059680 tx=0x7ff06005a140 comp rx=0 tx=0).stop 2024-03-28T13:15:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.764+0000 7ff07b992740 1 -- 172.21.15.19:0/2472221456 shutdown_connections 2024-03-28T13:15:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.764+0000 7ff07b992740 1 --2- 172.21.15.19:0/2472221456 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x555cfad4d910 0x555cfad11db0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.764+0000 7ff07b992740 1 --2- 172.21.15.19:0/2472221456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555cfacd0940 0x555cfad11870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.764+0000 7ff07b992740 1 --2- 172.21.15.19:0/2472221456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555cfacd0560 0x555cfacd1260 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.764+0000 7ff052a00640 0 client.6924 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.764+0000 7ff07b992740 1 --2- 172.21.15.19:0/2472221456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555cfacc9ba0 0x555cfacd0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.764+0000 7ff07b992740 1 -- 172.21.15.19:0/2472221456 >> 172.21.15.19:0/2472221456 conn(0x555cfaca3a40 msgr2=0x555cfaca2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:28.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.764+0000 7ff07b992740 1 -- 172.21.15.19:0/2472221456 shutdown_connections 2024-03-28T13:15:28.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.765+0000 7ff07b992740 1 -- 172.21.15.19:0/2472221456 wait complete. 2024-03-28T13:15:28.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.765+0000 7ff07b992740 20 client.6924 trim_cache size 0 max 16384 2024-03-28T13:15:28.767 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:15:28.767 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T13:15:28.767 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-03-28T13:15:28.767 INFO:teuthology.orchestra.run.smithi019.stdout: "d8edb31f-bfa3-424c-8e03-5ef3a25182f9": { 2024-03-28T13:15:28.767 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/subdir", 2024-03-28T13:15:28.767 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "d8edb31f-bfa3-424c-8e03-5ef3a25182f9", 2024-03-28T13:15:28.767 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,repair" 2024-03-28T13:15:28.767 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:15:28.767 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:15:28.767 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:15:28.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.765+0000 7ff078c00640 1 -- 172.21.15.19:0/2913624937 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff05406c530 msgr2=0x7ff05406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.765+0000 7ff078c00640 1 --2- 172.21.15.19:0/2913624937 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff05406c530 0x7ff05406e9f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff064000f60 tx=0x7ff064009040 comp rx=0 tx=0).stop 2024-03-28T13:15:28.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.765+0000 7ff078c00640 1 -- 172.21.15.19:0/2913624937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0740d43d0 msgr2=0x7ff0740cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:28.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.765+0000 7ff078c00640 1 --2- 172.21.15.19:0/2913624937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0740d43d0 0x7ff0740cd0d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff06000e9b0 tx=0x7ff06000ee70 comp rx=0 tx=0).stop 2024-03-28T13:15:28.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.765+0000 7ff078c00640 1 -- 172.21.15.19:0/2913624937 shutdown_connections 2024-03-28T13:15:28.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.765+0000 7ff078c00640 1 --2- 172.21.15.19:0/2913624937 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff05406c530 0x7ff05406e9f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.765+0000 7ff078c00640 1 --2- 172.21.15.19:0/2913624937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff07415e860 0x7ff0740c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.765+0000 7ff078c00640 1 --2- 172.21.15.19:0/2913624937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0740d43d0 0x7ff0740cd0d0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.765+0000 7ff078c00640 1 --2- 172.21.15.19:0/2913624937 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0740cf6b0 0x7ff07414db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:28.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.765+0000 7ff078c00640 1 -- 172.21.15.19:0/2913624937 >> 172.21.15.19:0/2913624937 conn(0x7ff07409d7d0 msgr2=0x7ff0740b4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:28.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.766+0000 7ff078c00640 1 -- 172.21.15.19:0/2913624937 shutdown_connections 2024-03-28T13:15:28.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:28.766+0000 7ff078c00640 1 -- 172.21.15.19:0/2913624937 wait complete. 2024-03-28T13:15:28.781 INFO:tasks.cephfs.filesystem:scrub status for tag:d8edb31f-bfa3-424c-8e03-5ef3a25182f9 - {'path': '/subdir', 'tag': 'd8edb31f-bfa3-424c-8e03-5ef3a25182f9', 'options': 'recursive,repair'} 2024-03-28T13:15:32.768 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:15:32.776 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:15:58.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:15:58.782 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-28T13:15:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.008+0000 7fb981200640 1 Processor -- start 2024-03-28T13:15:59.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.008+0000 7fb981200640 1 -- start start 2024-03-28T13:15:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.010+0000 7fb981200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb97c13f190 0x7fb97c0cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.010+0000 7fb981200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb97c0cc880 0x7fb97c0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.010+0000 7fb981200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb97c0ce890 con 0x7fb97c06d620 2024-03-28T13:15:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.010+0000 7fb981200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb97c0cd200 con 0x7fb97c13f190 2024-03-28T13:15:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.010+0000 7fb981200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb97c0ceb50 con 0x7fb97c0cc880 2024-03-28T13:15:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.010+0000 7fb97be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb97c06d620 0x7fb97c06da70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38414/0 (socket says 172.21.15.19:38414) 2024-03-28T13:15:59.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.010+0000 7fb97be00640 1 -- 172.21.15.19:0/2217546247 learned_addr learned my addr 172.21.15.19:0/2217546247 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:15:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.011+0000 7fb97b400640 1 --2- 172.21.15.19:0/2217546247 >> v2:172.21.15.188:3300/0 conn(0x7fb97c13f190 0x7fb97c0cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.011+0000 7fb972a00640 1 --2- 172.21.15.19:0/2217546247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb97c0cc880 0x7fb97c0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.011+0000 7fb97aa00640 1 -- 172.21.15.19:0/2217546247 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1509478776 0 0) 0x7fb97c0ce890 con 0x7fb97c06d620 2024-03-28T13:15:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.011+0000 7fb97aa00640 1 -- 172.21.15.19:0/2217546247 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb960003520 con 0x7fb97c06d620 2024-03-28T13:15:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.011+0000 7fb97aa00640 1 -- 172.21.15.19:0/2217546247 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3581362900 0 0) 0x7fb960003520 con 0x7fb97c06d620 2024-03-28T13:15:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.011+0000 7fb97aa00640 1 -- 172.21.15.19:0/2217546247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb97c0cc880 msgr2=0x7fb97c0cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.011+0000 7fb97aa00640 1 --2- 172.21.15.19:0/2217546247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb97c0cc880 0x7fb97c0cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.011+0000 7fb97aa00640 1 -- 172.21.15.19:0/2217546247 >> v2:172.21.15.188:3300/0 conn(0x7fb97c13f190 msgr2=0x7fb97c0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.011+0000 7fb97aa00640 1 --2- 172.21.15.19:0/2217546247 >> v2:172.21.15.188:3300/0 conn(0x7fb97c13f190 0x7fb97c0cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.011+0000 7fb97aa00640 1 -- 172.21.15.19:0/2217546247 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c0cee30 con 0x7fb97c06d620 2024-03-28T13:15:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.011+0000 7fb97aa00640 1 -- 172.21.15.19:0/2217546247 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fb974001670 con 0x7fb97c06d620 2024-03-28T13:15:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.011+0000 7fb97aa00640 1 -- 172.21.15.19:0/2217546247 >> v1:172.21.15.19:6789/0 conn(0x7fb97c06d620 legacy=0x7fb97c06da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb97aa00640 1 --2- 172.21.15.19:0/2217546247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9600039b0 0x7fb960003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb97aa00640 1 --2- 172.21.15.19:0/2217546247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb960004800 0x7fb960004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb97aa00640 1 --2- 172.21.15.19:0/2217546247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb97c0cc880 0x7fb9600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb97aa00640 1 -- 172.21.15.19:0/2217546247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c0cd200 con 0x7fb960004800 2024-03-28T13:15:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb97aa00640 1 -- 172.21.15.19:0/2217546247 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c0ceb50 con 0x7fb9600039b0 2024-03-28T13:15:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb97aa00640 1 -- 172.21.15.19:0/2217546247 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb96000b9c0 con 0x7fb97c0cc880 2024-03-28T13:15:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb972a00640 1 --2- 172.21.15.19:0/2217546247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb960004800 0x7fb960004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb97be00640 1 --2- 172.21.15.19:0/2217546247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb97c0cc880 0x7fb9600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb972a00640 1 -- 172.21.15.19:0/2217546247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb97c0cc880 msgr2=0x7fb9600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb972a00640 1 --2- 172.21.15.19:0/2217546247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb97c0cc880 0x7fb9600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb972a00640 1 -- 172.21.15.19:0/2217546247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9600039b0 msgr2=0x7fb960003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb97b400640 1 --2- 172.21.15.19:0/2217546247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9600039b0 0x7fb960003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb972a00640 1 --2- 172.21.15.19:0/2217546247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9600039b0 0x7fb960003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb972a00640 1 -- 172.21.15.19:0/2217546247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb960003520 con 0x7fb960004800 2024-03-28T13:15:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb97be00640 1 --2- 172.21.15.19:0/2217546247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb97c0cc880 0x7fb9600051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:59.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.012+0000 7fb97b400640 1 --2- 172.21.15.19:0/2217546247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9600039b0 0x7fb960003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.013+0000 7fb972a00640 1 --2- 172.21.15.19:0/2217546247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb960004800 0x7fb960004c80 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb964009ac0 tx=0x7fb960005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270d8aef38032171 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.014+0000 7fb97aa00640 1 -- 172.21.15.19:0/2217546247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964039070 con 0x7fb960004800 2024-03-28T13:15:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.014+0000 7fb97aa00640 1 -- 172.21.15.19:0/2217546247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96402c6b0 con 0x7fb960004800 2024-03-28T13:15:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.014+0000 7fb97aa00640 1 -- 172.21.15.19:0/2217546247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964009690 con 0x7fb960004800 2024-03-28T13:15:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.014+0000 7fb981200640 1 -- 172.21.15.19:0/2217546247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb960004800 msgr2=0x7fb960004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.014+0000 7fb981200640 1 --2- 172.21.15.19:0/2217546247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb960004800 0x7fb960004c80 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb964009ac0 tx=0x7fb960005830 comp rx=0 tx=0).stop 2024-03-28T13:15:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.014+0000 7fb97be00640 1 -- 172.21.15.19:0/2217546247 reap_dead start 2024-03-28T13:15:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.014+0000 7fb981200640 1 -- 172.21.15.19:0/2217546247 shutdown_connections 2024-03-28T13:15:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.014+0000 7fb981200640 1 -- 172.21.15.19:0/2217546247 >> 172.21.15.19:0/2217546247 conn(0x7fb97c0b1ba0 msgr2=0x7fb97c0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.015+0000 7fb981200640 1 -- 172.21.15.19:0/2217546247 shutdown_connections 2024-03-28T13:15:59.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.015+0000 7fb981200640 1 -- 172.21.15.19:0/2217546247 wait complete. 2024-03-28T13:15:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.016+0000 7fb981200640 1 Processor -- start 2024-03-28T13:15:59.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.017+0000 7fb981200640 1 -- start start 2024-03-28T13:15:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.017+0000 7fb981200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb960004800 0x7fb97c154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.017+0000 7fb981200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb97c1508c0 0x7fb97c159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.017+0000 7fb981200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb97c15ded0 0x7fb97c15e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.017+0000 7fb981200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c13f570 con 0x7fb960004800 2024-03-28T13:15:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.017+0000 7fb981200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c160c70 con 0x7fb97c15ded0 2024-03-28T13:15:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.017+0000 7fb981200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb97c0cfb70 con 0x7fb97c1508c0 2024-03-28T13:15:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.018+0000 7fb97b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb960004800 0x7fb97c154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.018+0000 7fb97b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb960004800 0x7fb97c154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41870/0 (socket says 172.21.15.19:41870) 2024-03-28T13:15:59.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.018+0000 7fb97b400640 1 -- 172.21.15.19:0/2389595052 learned_addr learned my addr 172.21.15.19:0/2389595052 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:59.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.018+0000 7fb972a00640 1 --2- 172.21.15.19:0/2389595052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb97c1508c0 0x7fb97c159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.018+0000 7fb97b400640 1 -- 172.21.15.19:0/2389595052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb97c1508c0 msgr2=0x7fb97c159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.018+0000 7fb97b400640 1 --2- 172.21.15.19:0/2389595052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb97c1508c0 0x7fb97c159d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.018+0000 7fb97be00640 1 --2- 172.21.15.19:0/2389595052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb97c15ded0 0x7fb97c15e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.018+0000 7fb97b400640 1 -- 172.21.15.19:0/2389595052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb97c15ded0 msgr2=0x7fb97c15e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.018+0000 7fb97b400640 1 --2- 172.21.15.19:0/2389595052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb97c15ded0 0x7fb97c15e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.018+0000 7fb97b400640 1 -- 172.21.15.19:0/2389595052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c150d20 con 0x7fb960004800 2024-03-28T13:15:59.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.019+0000 7fb97b400640 1 --2- 172.21.15.19:0/2389595052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb960004800 0x7fb97c154020 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00f3d0 tx=0x7fb96c00f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.019+0000 7fb973e00640 1 -- 172.21.15.19:0/2389595052 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c003990 con 0x7fb960004800 2024-03-28T13:15:59.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.019+0000 7fb973e00640 1 -- 172.21.15.19:0/2389595052 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96c003b30 con 0x7fb960004800 2024-03-28T13:15:59.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.019+0000 7fb973e00640 1 -- 172.21.15.19:0/2389595052 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c004020 con 0x7fb960004800 2024-03-28T13:15:59.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.019+0000 7fb981200640 1 -- 172.21.15.19:0/2389595052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97c151960 con 0x7fb960004800 2024-03-28T13:15:59.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.019+0000 7fb981200640 1 -- 172.21.15.19:0/2389595052 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97c151f20 con 0x7fb960004800 2024-03-28T13:15:59.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.021+0000 7fb973e00640 1 -- 172.21.15.19:0/2389595052 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb96c005d10 con 0x7fb960004800 2024-03-28T13:15:59.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.023+0000 7fb973e00640 1 --2- 172.21.15.19:0/2389595052 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb95806c560 0x7fb95806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.023+0000 7fb973e00640 1 -- 172.21.15.19:0/2389595052 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7fb96c08c2a0 con 0x7fb960004800 2024-03-28T13:15:59.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.024+0000 7fb972a00640 1 --2- 172.21.15.19:0/2389595052 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb95806c560 0x7fb95806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.024+0000 7fb985468740 1 Processor -- start 2024-03-28T13:15:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.024+0000 7fb985468740 1 -- start start 2024-03-28T13:15:59.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.025+0000 7fb972a00640 1 --2- 172.21.15.19:0/2389595052 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb95806c560 0x7fb95806ea20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb964036000 tx=0x7fb964030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.025+0000 7fb985468740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c848681a0 0x559c84865e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.025+0000 7fb985468740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c84868580 0x559c8481f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.025+0000 7fb985468740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c84847ae0 0x559c8481fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.025+0000 7fb985468740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559c847856b0 con 0x559c84868580 2024-03-28T13:15:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.025+0000 7fb985468740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559c8472d750 con 0x559c848681a0 2024-03-28T13:15:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.025+0000 7fb985468740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x559c84781660 con 0x559c84847ae0 2024-03-28T13:15:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.025+0000 7fb972a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c84868580 0x559c8481f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.025+0000 7fb972a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c84868580 0x559c8481f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41874/0 (socket says 172.21.15.19:41874) 2024-03-28T13:15:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.026+0000 7fb972a00640 1 -- 172.21.15.19:0/281518754 learned_addr learned my addr 172.21.15.19:0/281518754 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:59.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.026+0000 7fb97b400640 1 --2- 172.21.15.19:0/281518754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c848681a0 0x559c84865e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.026+0000 7fb972a00640 1 -- 172.21.15.19:0/281518754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c84847ae0 msgr2=0x559c8481fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.026+0000 7fb97be00640 1 --2- 172.21.15.19:0/281518754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c84847ae0 0x559c8481fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.026+0000 7fb972a00640 1 --2- 172.21.15.19:0/281518754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c84847ae0 0x559c8481fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.026+0000 7fb972a00640 1 -- 172.21.15.19:0/281518754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c848681a0 msgr2=0x559c84865e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.026+0000 7fb972a00640 1 --2- 172.21.15.19:0/281518754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c848681a0 0x559c84865e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.026+0000 7fb972a00640 1 -- 172.21.15.19:0/281518754 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c845637d0 con 0x559c84868580 2024-03-28T13:15:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.026+0000 7fb97be00640 1 --2- 172.21.15.19:0/281518754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c84847ae0 0x559c8481fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:59.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.026+0000 7fb972a00640 1 --2- 172.21.15.19:0/281518754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c84868580 0x559c8481f7d0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb96400f200 tx=0x7fb96402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7f2c0226497516 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.027+0000 7fb96be00640 1 -- 172.21.15.19:0/281518754 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964039040 con 0x559c84868580 2024-03-28T13:15:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.027+0000 7fb96be00640 1 -- 172.21.15.19:0/281518754 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96402d840 con 0x559c84868580 2024-03-28T13:15:59.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.027+0000 7fb96be00640 1 -- 172.21.15.19:0/281518754 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96402dbf0 con 0x559c84868580 2024-03-28T13:15:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.027+0000 7fb985468740 1 -- 172.21.15.19:0/281518754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c84868580 msgr2=0x559c8481f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.027+0000 7fb985468740 1 --2- 172.21.15.19:0/281518754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c84868580 0x559c8481f7d0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb96400f200 tx=0x7fb96402e000 comp rx=0 tx=0).stop 2024-03-28T13:15:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.028+0000 7fb985468740 1 -- 172.21.15.19:0/281518754 shutdown_connections 2024-03-28T13:15:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.028+0000 7fb985468740 1 --2- 172.21.15.19:0/281518754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c84847ae0 0x559c8481fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.028+0000 7fb985468740 1 --2- 172.21.15.19:0/281518754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c84868580 0x559c8481f7d0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.028+0000 7fb985468740 1 --2- 172.21.15.19:0/281518754 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c848681a0 0x559c84865e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.028+0000 7fb985468740 1 -- 172.21.15.19:0/281518754 >> 172.21.15.19:0/281518754 conn(0x559c8481c930 msgr2=0x559c848654b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.028+0000 7fb985468740 1 -- 172.21.15.19:0/281518754 shutdown_connections 2024-03-28T13:15:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.028+0000 7fb985468740 1 -- 172.21.15.19:0/281518754 wait complete. 2024-03-28T13:15:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.028+0000 7fb985468740 1 -- start start 2024-03-28T13:15:59.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.029+0000 7fb985468740 1 Processor -- start 2024-03-28T13:15:59.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.030+0000 7fb985468740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c848681a0 0x559c8483af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.030+0000 7fb985468740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c84868580 0x559c84844c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.030+0000 7fb985468740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c8483b4c0 0x559c84845190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.030+0000 7fb985468740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559c8481af60 con 0x559c8483b4c0 2024-03-28T13:15:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.030+0000 7fb985468740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x559c84740a80 con 0x559c848681a0 2024-03-28T13:15:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.030+0000 7fb985468740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x559c8481ed60 con 0x559c84868580 2024-03-28T13:15:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.030+0000 7fb985468740 10 client.0 _ll_register_callbacks cb 0x559c84740470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:15:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.030+0000 7fb97be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c8483b4c0 0x559c84845190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.030+0000 7fb97be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c8483b4c0 0x559c84845190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41888/0 (socket says 172.21.15.19:41888) 2024-03-28T13:15:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.030+0000 7fb97be00640 1 -- 172.21.15.19:0/2286015627 learned_addr learned my addr 172.21.15.19:0/2286015627 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.030+0000 7fb97b400640 1 --2- 172.21.15.19:0/2286015627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c848681a0 0x559c8483af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.030+0000 7fb972a00640 1 --2- 172.21.15.19:0/2286015627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c84868580 0x559c84844c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.031+0000 7fb97be00640 1 -- 172.21.15.19:0/2286015627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c84868580 msgr2=0x559c84844c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.031+0000 7fb97be00640 1 --2- 172.21.15.19:0/2286015627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c84868580 0x559c84844c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.031+0000 7fb97be00640 1 -- 172.21.15.19:0/2286015627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c848681a0 msgr2=0x559c8483af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.031+0000 7fb97be00640 1 --2- 172.21.15.19:0/2286015627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c848681a0 0x559c8483af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.031+0000 7fb97be00640 1 -- 172.21.15.19:0/2286015627 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c844993f0 con 0x559c8483b4c0 2024-03-28T13:15:59.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.031+0000 7fb97b400640 1 --2- 172.21.15.19:0/2286015627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c848681a0 0x559c8483af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:15:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.031+0000 7fb97be00640 1 --2- 172.21.15.19:0/2286015627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c8483b4c0 0x559c84845190 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb97c0b2840 tx=0x7fb974007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef2477f2ca0b3d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.031+0000 7fb968c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:15:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.031+0000 7fb968c00640 1 -- 172.21.15.19:0/2286015627 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974019060 con 0x559c8483b4c0 2024-03-28T13:15:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.032+0000 7fb968c00640 1 -- 172.21.15.19:0/2286015627 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7fb974014b90 con 0x559c8483b4c0 2024-03-28T13:15:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.032+0000 7fb985468740 1 -- 172.21.15.19:0/2286015627 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb96402d840 con 0x559c8483b4c0 2024-03-28T13:15:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.032+0000 7fb968c00640 1 -- 172.21.15.19:0/2286015627 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fb974014d90 con 0x559c8483b4c0 2024-03-28T13:15:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.032+0000 7fb985468740 10 client.6936 fetch_fsmap learned FSMap version 53 2024-03-28T13:15:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.032+0000 7fb985468740 1 -- 172.21.15.19:0/2286015627 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x559c844590f0 con 0x559c8483b4c0 2024-03-28T13:15:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.032+0000 7fb968c00640 1 -- 172.21.15.19:0/2286015627 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96402d840 con 0x559c8483b4c0 2024-03-28T13:15:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.032+0000 7fb968c00640 1 -- 172.21.15.19:0/2286015627 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7fb9740072d0 con 0x559c8483b4c0 2024-03-28T13:15:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.032+0000 7fb985468740 10 client.6936 fetch_fsmap finished waiting for FSMap version 53 2024-03-28T13:15:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.032+0000 7fb985468740 10 client.6936 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-28T13:15:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.032+0000 7fb985468740 20 client.6936 populate_metadata read hostname 'smithi019' 2024-03-28T13:15:59.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.032+0000 7fb985468740 1 --2- 172.21.15.19:0/2286015627 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x559c8485fae0 0x559c848669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.032+0000 7fb985468740 4 client.6936 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:15:59.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.032+0000 7fb985468740 1 -- 172.21.15.19:0/2286015627 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c84792bf0 con 0x559c8485fae0 2024-03-28T13:15:59.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.033+0000 7fb97b400640 1 --2- 172.21.15.19:0/2286015627 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x559c8485fae0 0x559c848669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.033+0000 7fb97b400640 1 --2- 172.21.15.19:0/2286015627 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x559c8485fae0 0x559c848669e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a70113b9d36a589d server_cookie=d3941df8f42a7e7e in_seq=0 out_seq=0 2024-03-28T13:15:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.033+0000 7fb968c00640 10 client.6936 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.034+0000 7fb968c00640 1 -- 172.21.15.19:0/2286015627 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c84792bf0 con 0x559c8485fae0 2024-03-28T13:15:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.034+0000 7fb968c00640 10 client.6936 handle_command_reply: tid=0 2024-03-28T13:15:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.034+0000 7fb985468740 1 client.6936 shutdown 2024-03-28T13:15:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.034+0000 7fb985468740 1 -- 172.21.15.19:0/2286015627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c8483b4c0 msgr2=0x559c84845190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.034+0000 7fb985468740 1 --2- 172.21.15.19:0/2286015627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c8483b4c0 0x559c84845190 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb97c0b2840 tx=0x7fb974007f90 comp rx=0 tx=0).stop 2024-03-28T13:15:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.034+0000 7fb985468740 1 -- 172.21.15.19:0/2286015627 shutdown_connections 2024-03-28T13:15:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.034+0000 7fb985468740 1 --2- 172.21.15.19:0/2286015627 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x559c8485fae0 0x559c848669e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.034+0000 7fb985468740 1 --2- 172.21.15.19:0/2286015627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c8483b4c0 0x559c84845190 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.034+0000 7fb985468740 1 --2- 172.21.15.19:0/2286015627 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c84868580 0x559c84844c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.034+0000 7fb985468740 1 --2- 172.21.15.19:0/2286015627 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c848681a0 0x559c8483af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.034+0000 7fb985468740 1 -- 172.21.15.19:0/2286015627 >> 172.21.15.19:0/2286015627 conn(0x559c84847ae0 msgr2=0x559c848195b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:59.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.034+0000 7fb968c00640 0 client.6936 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:59.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.035+0000 7fb985468740 1 -- 172.21.15.19:0/2286015627 shutdown_connections 2024-03-28T13:15:59.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.035+0000 7fb985468740 1 -- 172.21.15.19:0/2286015627 wait complete. 2024-03-28T13:15:59.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.035+0000 7fb985468740 20 client.6936 trim_cache size 0 max 16384 2024-03-28T13:15:59.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.054+0000 7fb985468740 1 Processor -- start 2024-03-28T13:15:59.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.054+0000 7fb985468740 1 -- start start 2024-03-28T13:15:59.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.055+0000 7fb985468740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c8483b4c0 0x559c8481b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.055+0000 7fb985468740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c848681a0 0x559c84842770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.055+0000 7fb985468740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c84868580 0x559c8481ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.055+0000 7fb985468740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c1fcea0 con 0x559c848681a0 2024-03-28T13:15:59.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.055+0000 7fb985468740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c0b4800 con 0x559c84868580 2024-03-28T13:15:59.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.055+0000 7fb985468740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb97c0729c0 con 0x559c8483b4c0 2024-03-28T13:15:59.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.055+0000 7fb972a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c848681a0 0x559c84842770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.055+0000 7fb972a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c848681a0 0x559c84842770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41898/0 (socket says 172.21.15.19:41898) 2024-03-28T13:15:59.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.055+0000 7fb972a00640 1 -- 172.21.15.19:0/199670126 learned_addr learned my addr 172.21.15.19:0/199670126 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:59.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.056+0000 7fb972a00640 1 -- 172.21.15.19:0/199670126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c8483b4c0 msgr2=0x559c8481b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.056+0000 7fb972a00640 1 --2- 172.21.15.19:0/199670126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c8483b4c0 0x559c8481b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.056+0000 7fb972a00640 1 -- 172.21.15.19:0/199670126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c84868580 msgr2=0x559c8481ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.056+0000 7fb972a00640 1 --2- 172.21.15.19:0/199670126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c84868580 0x559c8481ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.056+0000 7fb972a00640 1 -- 172.21.15.19:0/199670126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c84865c20 con 0x559c848681a0 2024-03-28T13:15:59.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.056+0000 7fb972a00640 1 --2- 172.21.15.19:0/199670126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c848681a0 0x559c84842770 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb96402c320 tx=0x7fb96403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647acbd4e1ce583c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.056+0000 7fb96b400640 1 -- 172.21.15.19:0/199670126 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964039040 con 0x559c848681a0 2024-03-28T13:15:59.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.057+0000 7fb96b400640 1 -- 172.21.15.19:0/199670126 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb964059070 con 0x559c848681a0 2024-03-28T13:15:59.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.057+0000 7fb96b400640 1 -- 172.21.15.19:0/199670126 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96402da40 con 0x559c848681a0 2024-03-28T13:15:59.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.057+0000 7fb985468740 1 -- 172.21.15.19:0/199670126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c848681a0 msgr2=0x559c84842770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.057+0000 7fb985468740 1 --2- 172.21.15.19:0/199670126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c848681a0 0x559c84842770 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb96402c320 tx=0x7fb96403d000 comp rx=0 tx=0).stop 2024-03-28T13:15:59.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.057+0000 7fb985468740 1 -- 172.21.15.19:0/199670126 shutdown_connections 2024-03-28T13:15:59.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.057+0000 7fb985468740 1 --2- 172.21.15.19:0/199670126 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c84868580 0x559c8481ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.057+0000 7fb985468740 1 --2- 172.21.15.19:0/199670126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c848681a0 0x559c84842770 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.057+0000 7fb985468740 1 --2- 172.21.15.19:0/199670126 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c8483b4c0 0x559c8481b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.057+0000 7fb985468740 1 -- 172.21.15.19:0/199670126 >> 172.21.15.19:0/199670126 conn(0x559c84847ae0 msgr2=0x559c8483ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:59.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.057+0000 7fb985468740 1 -- 172.21.15.19:0/199670126 shutdown_connections 2024-03-28T13:15:59.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.057+0000 7fb985468740 1 -- 172.21.15.19:0/199670126 wait complete. 2024-03-28T13:15:59.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.058+0000 7fb985468740 1 -- start start 2024-03-28T13:15:59.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.058+0000 7fb985468740 1 Processor -- start 2024-03-28T13:15:59.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.059+0000 7fb985468740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c8483b4c0 0x559c84839930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.059+0000 7fb985468740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c848681a0 0x559c84839e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.059+0000 7fb985468740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c84868580 0x559c8488c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.059+0000 7fb985468740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559c84848030 con 0x559c84868580 2024-03-28T13:15:59.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.059+0000 7fb985468740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c0b41b0 con 0x559c8483b4c0 2024-03-28T13:15:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.059+0000 7fb985468740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x559c8481c3e0 con 0x559c848681a0 2024-03-28T13:15:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.059+0000 7fb985468740 10 client.0 _ll_register_callbacks cb 0x7fb974007b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:15:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.059+0000 7fb97be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c84868580 0x559c8488c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.059+0000 7fb97be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c84868580 0x559c8488c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41908/0 (socket says 172.21.15.19:41908) 2024-03-28T13:15:59.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.060+0000 7fb97be00640 1 -- 172.21.15.19:0/1348307501 learned_addr learned my addr 172.21.15.19:0/1348307501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:59.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.060+0000 7fb97b400640 1 --2- 172.21.15.19:0/1348307501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c8483b4c0 0x559c84839930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.060+0000 7fb97be00640 1 -- 172.21.15.19:0/1348307501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c848681a0 msgr2=0x559c84839e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.060+0000 7fb972a00640 1 --2- 172.21.15.19:0/1348307501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c848681a0 0x559c84839e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.060+0000 7fb97be00640 1 --2- 172.21.15.19:0/1348307501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c848681a0 0x559c84839e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.060+0000 7fb97be00640 1 -- 172.21.15.19:0/1348307501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c8483b4c0 msgr2=0x559c84839930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.060+0000 7fb97be00640 1 --2- 172.21.15.19:0/1348307501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c8483b4c0 0x559c84839930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.060+0000 7fb97be00640 1 -- 172.21.15.19:0/1348307501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c8488ddc0 con 0x559c84868580 2024-03-28T13:15:59.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.060+0000 7fb97b400640 1 --2- 172.21.15.19:0/1348307501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c8483b4c0 0x559c84839930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:59.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.060+0000 7fb97be00640 1 --2- 172.21.15.19:0/1348307501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c84868580 0x559c8488c610 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fb974000ea0 tx=0x7fb974001ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf045d8573212a37 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.060+0000 7fb968c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:15:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.061+0000 7fb968c00640 1 -- 172.21.15.19:0/1348307501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9740017d0 con 0x559c84868580 2024-03-28T13:15:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.061+0000 7fb968c00640 1 -- 172.21.15.19:0/1348307501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7fb974004830 con 0x559c84868580 2024-03-28T13:15:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.061+0000 7fb985468740 1 -- 172.21.15.19:0/1348307501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb964059070 con 0x559c84868580 2024-03-28T13:15:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.061+0000 7fb968c00640 1 -- 172.21.15.19:0/1348307501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fb974034050 con 0x559c84868580 2024-03-28T13:15:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.061+0000 7fb985468740 10 client.6942 fetch_fsmap learned FSMap version 53 2024-03-28T13:15:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.061+0000 7fb985468740 1 -- 172.21.15.19:0/1348307501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x559c84847810 con 0x559c84868580 2024-03-28T13:15:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.061+0000 7fb968c00640 1 -- 172.21.15.19:0/1348307501 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb964059070 con 0x559c84868580 2024-03-28T13:15:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.061+0000 7fb968c00640 1 -- 172.21.15.19:0/1348307501 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7fb974037020 con 0x559c84868580 2024-03-28T13:15:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.061+0000 7fb985468740 10 client.6942 fetch_fsmap finished waiting for FSMap version 53 2024-03-28T13:15:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.061+0000 7fb985468740 10 client.6942 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-28T13:15:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.061+0000 7fb985468740 20 client.6942 populate_metadata read hostname 'smithi019' 2024-03-28T13:15:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.061+0000 7fb985468740 1 --2- 172.21.15.19:0/1348307501 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x559c848e2fe0 0x559c8488cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.061+0000 7fb985468740 4 client.6942 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:15:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.061+0000 7fb985468740 1 -- 172.21.15.19:0/1348307501 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559c8478cf40 con 0x559c848e2fe0 2024-03-28T13:15:59.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.062+0000 7fb97b400640 1 --2- 172.21.15.19:0/1348307501 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x559c848e2fe0 0x559c8488cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.062+0000 7fb968c00640 10 client.6942 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:59.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.062+0000 7fb97b400640 1 --2- 172.21.15.19:0/1348307501 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x559c848e2fe0 0x559c8488cb50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d40179f80ca0ea server_cookie=2d24c644de17508d in_seq=0 out_seq=0 2024-03-28T13:15:59.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.063+0000 7fb968c00640 1 -- 172.21.15.19:0/1348307501 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559c8478cf40 con 0x559c848e2fe0 2024-03-28T13:15:59.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.063+0000 7fb968c00640 10 client.6942 handle_command_reply: tid=0 2024-03-28T13:15:59.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.063+0000 7fb985468740 1 client.6942 shutdown 2024-03-28T13:15:59.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.063+0000 7fb985468740 1 -- 172.21.15.19:0/1348307501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c84868580 msgr2=0x559c8488c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.063+0000 7fb985468740 1 --2- 172.21.15.19:0/1348307501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c84868580 0x559c8488c610 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fb974000ea0 tx=0x7fb974001ab0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.063+0000 7fb985468740 1 -- 172.21.15.19:0/1348307501 shutdown_connections 2024-03-28T13:15:59.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.063+0000 7fb985468740 1 --2- 172.21.15.19:0/1348307501 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x559c848e2fe0 0x559c8488cb50 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.063+0000 7fb985468740 1 --2- 172.21.15.19:0/1348307501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559c84868580 0x559c8488c610 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.063+0000 7fb985468740 1 --2- 172.21.15.19:0/1348307501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559c848681a0 0x559c84839e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.063+0000 7fb985468740 1 --2- 172.21.15.19:0/1348307501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559c8483b4c0 0x559c84839930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.063+0000 7fb985468740 1 -- 172.21.15.19:0/1348307501 >> 172.21.15.19:0/1348307501 conn(0x559c84847ae0 msgr2=0x559c8481ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:59.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.063+0000 7fb968c00640 0 client.6942 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:59.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.063+0000 7fb985468740 1 -- 172.21.15.19:0/1348307501 shutdown_connections 2024-03-28T13:15:59.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.063+0000 7fb985468740 1 -- 172.21.15.19:0/1348307501 wait complete. 2024-03-28T13:15:59.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.063+0000 7fb985468740 20 client.6942 trim_cache size 0 max 16384 2024-03-28T13:15:59.066 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:15:59.066 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-03-28T13:15:59.067 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-03-28T13:15:59.067 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:15:59.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.064+0000 7fb981200640 1 -- 172.21.15.19:0/2389595052 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb95806c560 msgr2=0x7fb95806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.064+0000 7fb981200640 1 --2- 172.21.15.19:0/2389595052 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb95806c560 0x7fb95806ea20 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb964036000 tx=0x7fb964030000 comp rx=0 tx=0).stop 2024-03-28T13:15:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.064+0000 7fb981200640 1 -- 172.21.15.19:0/2389595052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb960004800 msgr2=0x7fb97c154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.064+0000 7fb981200640 1 --2- 172.21.15.19:0/2389595052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb960004800 0x7fb97c154020 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00f3d0 tx=0x7fb96c00f890 comp rx=0 tx=0).stop 2024-03-28T13:15:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.064+0000 7fb981200640 1 -- 172.21.15.19:0/2389595052 shutdown_connections 2024-03-28T13:15:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.064+0000 7fb981200640 1 --2- 172.21.15.19:0/2389595052 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb95806c560 0x7fb95806ea20 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.064+0000 7fb981200640 1 --2- 172.21.15.19:0/2389595052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb97c15ded0 0x7fb97c15e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.064+0000 7fb981200640 1 --2- 172.21.15.19:0/2389595052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb97c1508c0 0x7fb97c159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.064+0000 7fb981200640 1 --2- 172.21.15.19:0/2389595052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb960004800 0x7fb97c154020 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.064+0000 7fb981200640 1 -- 172.21.15.19:0/2389595052 >> 172.21.15.19:0/2389595052 conn(0x7fb97c0b1ba0 msgr2=0x7fb97c0af7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.064+0000 7fb981200640 1 -- 172.21.15.19:0/2389595052 shutdown_connections 2024-03-28T13:15:59.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.064+0000 7fb981200640 1 -- 172.21.15.19:0/2389595052 wait complete. 2024-03-28T13:15:59.087 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T13:15:59.087 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:15:59.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.301+0000 7f5206e00640 1 Processor -- start 2024-03-28T13:15:59.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.302+0000 7f5206e00640 1 -- start start 2024-03-28T13:15:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.303+0000 7f5206e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f520015d120 0x7f5200162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.303+0000 7f5206e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52000af630 0x7f5200166b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.303+0000 7f5206e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52001633d0 con 0x7f5200167080 2024-03-28T13:15:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.303+0000 7f5206e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f52000723c0 con 0x7f520015d120 2024-03-28T13:15:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.303+0000 7f5206e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5200163650 con 0x7f52000af630 2024-03-28T13:15:59.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.304+0000 7f51ffe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f520015d120 0x7f5200162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.304+0000 7f51ffe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f520015d120 0x7f5200162e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:39882/0 (socket says 172.21.15.19:39882) 2024-03-28T13:15:59.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.304+0000 7f51ffe00640 1 -- 172.21.15.19:0/357770507 learned_addr learned my addr 172.21.15.19:0/357770507 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:59.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.304+0000 7f51f7400640 1 --2- 172.21.15.19:0/357770507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52000af630 0x7f5200166b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.305+0000 7f51ffe00640 1 -- 172.21.15.19:0/357770507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52000af630 msgr2=0x7f5200166b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.305+0000 7f51ffe00640 1 --2- 172.21.15.19:0/357770507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52000af630 0x7f5200166b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.305+0000 7f51ffe00640 1 -- 172.21.15.19:0/357770507 >> v1:172.21.15.188:6789/0 conn(0x7f5200167080 legacy=0x7f52000b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.305+0000 7f51ffe00640 1 -- 172.21.15.19:0/357770507 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5200163930 con 0x7f520015d120 2024-03-28T13:15:59.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.306+0000 7f51ffe00640 1 --2- 172.21.15.19:0/357770507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f520015d120 0x7f5200162e90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009e50 tx=0x7f51f002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad3341944b62b147 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.307+0000 7f51ff400640 1 -- 172.21.15.19:0/357770507 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f002cb00 con 0x7f520015d120 2024-03-28T13:15:59.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.307+0000 7f51ff400640 1 -- 172.21.15.19:0/357770507 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f002cca0 con 0x7f520015d120 2024-03-28T13:15:59.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.307+0000 7f51ff400640 1 -- 172.21.15.19:0/357770507 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0034610 con 0x7f520015d120 2024-03-28T13:15:59.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.307+0000 7f5206e00640 1 -- 172.21.15.19:0/357770507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f520015d120 msgr2=0x7f5200162e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.307+0000 7f5206e00640 1 --2- 172.21.15.19:0/357770507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f520015d120 0x7f5200162e90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009e50 tx=0x7f51f002bf60 comp rx=0 tx=0).stop 2024-03-28T13:15:59.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.308+0000 7f5206e00640 1 -- 172.21.15.19:0/357770507 shutdown_connections 2024-03-28T13:15:59.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.308+0000 7f5206e00640 1 --2- 172.21.15.19:0/357770507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52000af630 0x7f5200166b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.308+0000 7f5206e00640 1 --2- 172.21.15.19:0/357770507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f520015d120 0x7f5200162e90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.308+0000 7f5206e00640 1 -- 172.21.15.19:0/357770507 >> 172.21.15.19:0/357770507 conn(0x7f520006b990 msgr2=0x7f520006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:59.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.308+0000 7f5206e00640 1 -- 172.21.15.19:0/357770507 shutdown_connections 2024-03-28T13:15:59.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.308+0000 7f5206e00640 1 -- 172.21.15.19:0/357770507 wait complete. 2024-03-28T13:15:59.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.310+0000 7f5206e00640 1 Processor -- start 2024-03-28T13:15:59.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.310+0000 7f5206e00640 1 -- start start 2024-03-28T13:15:59.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.311+0000 7f5206e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52000af630 0x7f52001515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.311+0000 7f5206e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f520015d120 0x7f52000cee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.311+0000 7f5206e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5200167080 0x7f52000cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.311+0000 7f5206e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52000b3c10 con 0x7f52000af630 2024-03-28T13:15:59.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.311+0000 7f5206e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5200166620 con 0x7f520015d120 2024-03-28T13:15:59.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.311+0000 7f5206e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f52000b3d90 con 0x7f5200167080 2024-03-28T13:15:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.312+0000 7f51ffe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52000af630 0x7f52001515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.312+0000 7f51ffe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52000af630 0x7f52001515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41920/0 (socket says 172.21.15.19:41920) 2024-03-28T13:15:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.312+0000 7f51ffe00640 1 -- 172.21.15.19:0/2822370710 learned_addr learned my addr 172.21.15.19:0/2822370710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.312+0000 7f51f7400640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f520015d120 0x7f52000cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.312+0000 7f5204e00640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5200167080 0x7f52000cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.312+0000 7f51ffe00640 1 -- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5200167080 msgr2=0x7f52000cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.312+0000 7f51ffe00640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5200167080 0x7f52000cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.312+0000 7f51ffe00640 1 -- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f520015d120 msgr2=0x7f52000cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.312+0000 7f51ffe00640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f520015d120 0x7f52000cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.312+0000 7f51ffe00640 1 -- 172.21.15.19:0/2822370710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520009dad0 con 0x7f52000af630 2024-03-28T13:15:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.312+0000 7f51f7400640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f520015d120 0x7f52000cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:15:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.312+0000 7f5204e00640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5200167080 0x7f52000cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.313+0000 7f51ffe00640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52000af630 0x7f52001515b0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f51f0036040 tx=0x7f51f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.313+0000 7f51fcc00640 1 -- 172.21.15.19:0/2822370710 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0043020 con 0x7f52000af630 2024-03-28T13:15:59.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.313+0000 7f51fcc00640 1 -- 172.21.15.19:0/2822370710 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f002ce50 con 0x7f52000af630 2024-03-28T13:15:59.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.313+0000 7f5206e00640 1 -- 172.21.15.19:0/2822370710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520009ca30 con 0x7f52000af630 2024-03-28T13:15:59.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.313+0000 7f51fcc00640 1 -- 172.21.15.19:0/2822370710 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0009510 con 0x7f52000af630 2024-03-28T13:15:59.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.313+0000 7f5206e00640 1 -- 172.21.15.19:0/2822370710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520009cff0 con 0x7f52000af630 2024-03-28T13:15:59.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.314+0000 7f5206e00640 1 -- 172.21.15.19:0/2822370710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52000685c0 con 0x7f52000af630 2024-03-28T13:15:59.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.315+0000 7f51fcc00640 1 -- 172.21.15.19:0/2822370710 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f51f0046020 con 0x7f52000af630 2024-03-28T13:15:59.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.316+0000 7f51fcc00640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51dc06c580 0x7f51dc06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.316+0000 7f51fcc00640 1 -- 172.21.15.19:0/2822370710 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f51f00add60 con 0x7f52000af630 2024-03-28T13:15:59.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.317+0000 7f51f7400640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51dc06c580 0x7f51dc06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.318+0000 7f51f7400640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51dc06c580 0x7f51dc06ea40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f520009dde0 tx=0x7f51e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.329+0000 7f51fcc00640 1 -- 172.21.15.19:0/2822370710 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f51f004d030 con 0x7f52000af630 2024-03-28T13:15:59.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.579+0000 7f5206e00640 1 -- 172.21.15.19:0/2822370710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52000b3000 con 0x7f52000af630 2024-03-28T13:15:59.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.581+0000 7f51fcc00640 1 -- 172.21.15.19:0/2822370710 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5188 (secure 0 0 0) 0x7f51f003c350 con 0x7f52000af630 2024-03-28T13:15:59.581 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:15:59.581 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/144726418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":144726418},{"type":"v1","addr":"172.21.15.188:6836","nonce":144726418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5363,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2642581607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2642581607},{"type":"v1","addr":"172.21.15.19:6835","nonce":2642581607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1721039756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1721039756},{"type":"v1","addr":"172.21.15.188:6837","nonce":1721039756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:14:31.526577+0000","modified":"2024-03-28T13:15:32.038485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.19:6834/2130552083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2130552083},{"type":"v1","addr":"172.21.15.19:6834","nonce":2130552083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6708,"qdb_cluster":[6708]},"id":5}]} 2024-03-28T13:15:59.581 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 53 2024-03-28T13:15:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.583+0000 7f5206e00640 1 -- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51dc06c580 msgr2=0x7f51dc06ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.583+0000 7f5206e00640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51dc06c580 0x7f51dc06ea40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f520009dde0 tx=0x7f51e8008040 comp rx=0 tx=0).stop 2024-03-28T13:15:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.583+0000 7f5206e00640 1 -- 172.21.15.19:0/2822370710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52000af630 msgr2=0x7f52001515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.583+0000 7f5206e00640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52000af630 0x7f52001515b0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f51f0036040 tx=0x7f51f0004a40 comp rx=0 tx=0).stop 2024-03-28T13:15:59.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.583+0000 7f5206e00640 1 -- 172.21.15.19:0/2822370710 shutdown_connections 2024-03-28T13:15:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.583+0000 7f5206e00640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51dc06c580 0x7f51dc06ea40 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.583+0000 7f5206e00640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5200167080 0x7f52000cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.583+0000 7f5206e00640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f520015d120 0x7f52000cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.583+0000 7f5206e00640 1 --2- 172.21.15.19:0/2822370710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52000af630 0x7f52001515b0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.583+0000 7f5206e00640 1 -- 172.21.15.19:0/2822370710 >> 172.21.15.19:0/2822370710 conn(0x7f520006b990 msgr2=0x7f52000ad950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.583+0000 7f5206e00640 1 -- 172.21.15.19:0/2822370710 shutdown_connections 2024-03-28T13:15:59.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.584+0000 7f5206e00640 1 -- 172.21.15.19:0/2822370710 wait complete. 2024-03-28T13:15:59.606 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-28T13:15:59.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.823+0000 7f6a5c600640 1 Processor -- start 2024-03-28T13:15:59.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.824+0000 7f6a5c600640 1 -- start start 2024-03-28T13:15:59.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.825+0000 7f6a5c600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6a540d0280 0x7f6a540d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.826+0000 7f6a5c600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a5415ec80 con 0x7f6a54162160 2024-03-28T13:15:59.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.826+0000 7f6a5c600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a540d0ba0 con 0x7f6a54141c10 2024-03-28T13:15:59.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.826+0000 7f6a5c600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6a5415ef20 con 0x7f6a540d0280 2024-03-28T13:15:59.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.826+0000 7f6a59c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6a540d0280 0x7f6a540d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.826+0000 7f6a5a600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6a54162160 0x7f6a54162610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38424/0 (socket says 172.21.15.19:38424) 2024-03-28T13:15:59.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.826+0000 7f6a5a600640 1 -- 172.21.15.19:0/3558561426 learned_addr learned my addr 172.21.15.19:0/3558561426 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:15:59.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.826+0000 7f6a59c00640 1 -- 172.21.15.19:0/3558561426 >> v1:172.21.15.188:6789/0 conn(0x7f6a54141c10 legacy=0x7f6a54071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.826+0000 7f6a59c00640 1 -- 172.21.15.19:0/3558561426 >> v1:172.21.15.19:6789/0 conn(0x7f6a54162160 legacy=0x7f6a54162610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.826+0000 7f6a59c00640 1 -- 172.21.15.19:0/3558561426 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5415f200 con 0x7f6a540d0280 2024-03-28T13:15:59.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.828+0000 7f6a59c00640 1 --2- 172.21.15.19:0/3558561426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a540d0280 0x7f6a540d0660 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009d90 tx=0x7f6a4402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69202d24da686e20 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.828+0000 7f6a53e00640 1 -- 172.21.15.19:0/3558561426 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44039070 con 0x7f6a540d0280 2024-03-28T13:15:59.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.828+0000 7f6a53e00640 1 -- 172.21.15.19:0/3558561426 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a4402cc60 con 0x7f6a540d0280 2024-03-28T13:15:59.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.828+0000 7f6a53e00640 1 -- 172.21.15.19:0/3558561426 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44034660 con 0x7f6a540d0280 2024-03-28T13:15:59.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.829+0000 7f6a5c600640 1 -- 172.21.15.19:0/3558561426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a540d0280 msgr2=0x7f6a540d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.829+0000 7f6a5c600640 1 --2- 172.21.15.19:0/3558561426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a540d0280 0x7f6a540d0660 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009d90 tx=0x7f6a4402beb0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.829+0000 7f6a5c600640 1 -- 172.21.15.19:0/3558561426 shutdown_connections 2024-03-28T13:15:59.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.829+0000 7f6a5c600640 1 --2- 172.21.15.19:0/3558561426 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a540d0280 0x7f6a540d0660 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.829+0000 7f6a5c600640 1 -- 172.21.15.19:0/3558561426 >> 172.21.15.19:0/3558561426 conn(0x7f6a5406b750 msgr2=0x7f6a5406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:59.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.829+0000 7f6a5c600640 1 -- 172.21.15.19:0/3558561426 shutdown_connections 2024-03-28T13:15:59.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.829+0000 7f6a5c600640 1 -- 172.21.15.19:0/3558561426 wait complete. 2024-03-28T13:15:59.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.831+0000 7f6a5c600640 1 Processor -- start 2024-03-28T13:15:59.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.831+0000 7f6a5c600640 1 -- start start 2024-03-28T13:15:59.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.832+0000 7f6a5c600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a540d0280 0x7f6a54157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.832+0000 7f6a5c600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a54141c10 0x7f6a5409b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.832+0000 7f6a5c600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a54162160 0x7f6a541e8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.832+0000 7f6a5c600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a54158e90 con 0x7f6a54162160 2024-03-28T13:15:59.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.832+0000 7f6a5c600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5415f7c0 con 0x7f6a540d0280 2024-03-28T13:15:59.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.832+0000 7f6a5c600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6a5409c8b0 con 0x7f6a54141c10 2024-03-28T13:15:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.832+0000 7f6a5a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a54162160 0x7f6a541e8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.832+0000 7f6a5a600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a54162160 0x7f6a541e8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41942/0 (socket says 172.21.15.19:41942) 2024-03-28T13:15:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.832+0000 7f6a5a600640 1 -- 172.21.15.19:0/1190370885 learned_addr learned my addr 172.21.15.19:0/1190370885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.832+0000 7f6a59c00640 1 --2- 172.21.15.19:0/1190370885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a540d0280 0x7f6a54157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.832+0000 7f6a59200640 1 --2- 172.21.15.19:0/1190370885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a54141c10 0x7f6a5409b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.833+0000 7f6a5a600640 1 -- 172.21.15.19:0/1190370885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a54141c10 msgr2=0x7f6a5409b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.833+0000 7f6a5a600640 1 --2- 172.21.15.19:0/1190370885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a54141c10 0x7f6a5409b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.833+0000 7f6a5a600640 1 -- 172.21.15.19:0/1190370885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a540d0280 msgr2=0x7f6a54157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.833+0000 7f6a5a600640 1 --2- 172.21.15.19:0/1190370885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a540d0280 0x7f6a54157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.833+0000 7f6a5a600640 1 -- 172.21.15.19:0/1190370885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5409bca0 con 0x7f6a54162160 2024-03-28T13:15:59.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.833+0000 7f6a59200640 1 --2- 172.21.15.19:0/1190370885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a54141c10 0x7f6a5409b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:59.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.833+0000 7f6a5a600640 1 --2- 172.21.15.19:0/1190370885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a54162160 0x7f6a541e8ef0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00c3a0 tx=0x7f6a4c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.833+0000 7f6a51600640 1 -- 172.21.15.19:0/1190370885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c017070 con 0x7f6a54162160 2024-03-28T13:15:59.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.833+0000 7f6a51600640 1 -- 172.21.15.19:0/1190370885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a4c005ce0 con 0x7f6a54162160 2024-03-28T13:15:59.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.833+0000 7f6a51600640 1 -- 172.21.15.19:0/1190370885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0072d0 con 0x7f6a54162160 2024-03-28T13:15:59.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.833+0000 7f6a5c600640 1 -- 172.21.15.19:0/1190370885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5409bf80 con 0x7f6a54162160 2024-03-28T13:15:59.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.833+0000 7f6a5c600640 1 -- 172.21.15.19:0/1190370885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5409c540 con 0x7f6a54162160 2024-03-28T13:15:59.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.836+0000 7f6a51600640 1 -- 172.21.15.19:0/1190370885 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6a4c01a070 con 0x7f6a54162160 2024-03-28T13:15:59.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.837+0000 7f6a51600640 1 --2- 172.21.15.19:0/1190370885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a3406c5b0 0x7f6a3406ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.837+0000 7f6a51600640 1 -- 172.21.15.19:0/1190370885 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f6a4c08a9e0 con 0x7f6a54162160 2024-03-28T13:15:59.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.837+0000 7f6a59c00640 1 --2- 172.21.15.19:0/1190370885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a3406c5b0 0x7f6a3406ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.838+0000 7f6a59c00640 1 --2- 172.21.15.19:0/1190370885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a3406c5b0 0x7f6a3406ea70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6a44036000 tx=0x7f6a44030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.839+0000 7f6a5f236740 1 Processor -- start 2024-03-28T13:15:59.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.840+0000 7f6a5f236740 1 -- start start 2024-03-28T13:15:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.841+0000 7f6a5f236740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3ff16c0 0x55eba3ff1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.841+0000 7f6a5f236740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3fc1690 0x55eba3fee930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.841+0000 7f6a5f236740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 0x55eba3feee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.841+0000 7f6a5f236740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eba3f2b5a0 con 0x55eba3ff16c0 2024-03-28T13:15:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.841+0000 7f6a5f236740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eba3ed3390 con 0x55eba3fc5240 2024-03-28T13:15:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.841+0000 7f6a5f236740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55eba3f27550 con 0x55eba3fc1690 2024-03-28T13:15:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.841+0000 7f6a59c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3ff16c0 0x55eba3ff1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.841+0000 7f6a59c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3ff16c0 0x55eba3ff1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41948/0 (socket says 172.21.15.19:41948) 2024-03-28T13:15:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.841+0000 7f6a59c00640 1 -- 172.21.15.19:0/4113183114 learned_addr learned my addr 172.21.15.19:0/4113183114 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.842+0000 7f6a5a600640 1 --2- 172.21.15.19:0/4113183114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 0x55eba3feee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.842+0000 7f6a59c00640 1 -- 172.21.15.19:0/4113183114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3fc1690 msgr2=0x55eba3fee930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.842+0000 7f6a59200640 1 --2- 172.21.15.19:0/4113183114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3fc1690 0x55eba3fee930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.842+0000 7f6a59c00640 1 --2- 172.21.15.19:0/4113183114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3fc1690 0x55eba3fee930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.842+0000 7f6a59c00640 1 -- 172.21.15.19:0/4113183114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 msgr2=0x55eba3feee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.842+0000 7f6a59c00640 1 --2- 172.21.15.19:0/4113183114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 0x55eba3feee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.842+0000 7f6a59c00640 1 -- 172.21.15.19:0/4113183114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eba3d0b7c0 con 0x55eba3ff16c0 2024-03-28T13:15:59.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.842+0000 7f6a5a600640 1 --2- 172.21.15.19:0/4113183114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 0x55eba3feee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:59.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.842+0000 7f6a59c00640 1 --2- 172.21.15.19:0/4113183114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3ff16c0 0x55eba3ff1aa0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6a44037040 tx=0x7f6a4402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa0cf714ea670c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.843+0000 7f6a4a000640 1 -- 172.21.15.19:0/4113183114 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44051070 con 0x55eba3ff16c0 2024-03-28T13:15:59.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.843+0000 7f6a4a000640 1 -- 172.21.15.19:0/4113183114 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a44034dd0 con 0x55eba3ff16c0 2024-03-28T13:15:59.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.843+0000 7f6a4a000640 1 -- 172.21.15.19:0/4113183114 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44039040 con 0x55eba3ff16c0 2024-03-28T13:15:59.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.843+0000 7f6a5f236740 1 -- 172.21.15.19:0/4113183114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3ff16c0 msgr2=0x55eba3ff1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.843+0000 7f6a5f236740 1 --2- 172.21.15.19:0/4113183114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3ff16c0 0x55eba3ff1aa0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f6a44037040 tx=0x7f6a4402e000 comp rx=0 tx=0).stop 2024-03-28T13:15:59.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.844+0000 7f6a5f236740 1 -- 172.21.15.19:0/4113183114 shutdown_connections 2024-03-28T13:15:59.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.844+0000 7f6a5f236740 1 --2- 172.21.15.19:0/4113183114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 0x55eba3feee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.844+0000 7f6a5f236740 1 --2- 172.21.15.19:0/4113183114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3fc1690 0x55eba3fee930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.844+0000 7f6a5f236740 1 --2- 172.21.15.19:0/4113183114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3ff16c0 0x55eba3ff1aa0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.844+0000 7f6a5f236740 1 -- 172.21.15.19:0/4113183114 >> 172.21.15.19:0/4113183114 conn(0x55eba3fc3950 msgr2=0x55eba400c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.844+0000 7f6a5f236740 1 -- 172.21.15.19:0/4113183114 shutdown_connections 2024-03-28T13:15:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.844+0000 7f6a5f236740 1 -- 172.21.15.19:0/4113183114 wait complete. 2024-03-28T13:15:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.845+0000 7f6a5f236740 1 -- start start 2024-03-28T13:15:59.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.845+0000 7f6a5f236740 1 Processor -- start 2024-03-28T13:15:59.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.847+0000 7f6a5f236740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 0x55eba3ff3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.847+0000 7f6a5f236740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3ff16c0 0x55eba3ff8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.847+0000 7f6a5f236740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3fde3c0 0x55eba3fde800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.847+0000 7f6a5f236740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eba3fc63f0 con 0x55eba3ff16c0 2024-03-28T13:15:59.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.847+0000 7f6a5f236740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eba3ee6780 con 0x55eba3fc5240 2024-03-28T13:15:59.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.847+0000 7f6a5f236740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55eba3fc57e0 con 0x55eba3fde3c0 2024-03-28T13:15:59.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.847+0000 7f6a5f236740 10 client.0 _ll_register_callbacks cb 0x55eba3f22fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:15:59.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.847+0000 7f6a59200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3ff16c0 0x55eba3ff8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.847+0000 7f6a59200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3ff16c0 0x55eba3ff8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41954/0 (socket says 172.21.15.19:41954) 2024-03-28T13:15:59.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.847+0000 7f6a59200640 1 -- 172.21.15.19:0/3794733002 learned_addr learned my addr 172.21.15.19:0/3794733002 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:59.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.848+0000 7f6a59c00640 1 --2- 172.21.15.19:0/3794733002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 0x55eba3ff3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.848+0000 7f6a5a600640 1 --2- 172.21.15.19:0/3794733002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3fde3c0 0x55eba3fde800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.848+0000 7f6a59200640 1 -- 172.21.15.19:0/3794733002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3fde3c0 msgr2=0x55eba3fde800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.848+0000 7f6a59200640 1 --2- 172.21.15.19:0/3794733002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3fde3c0 0x55eba3fde800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.848+0000 7f6a59200640 1 -- 172.21.15.19:0/3794733002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 msgr2=0x55eba3ff3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.848+0000 7f6a59200640 1 --2- 172.21.15.19:0/3794733002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 0x55eba3ff3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.848+0000 7f6a59200640 1 -- 172.21.15.19:0/3794733002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eba3d6bf50 con 0x55eba3ff16c0 2024-03-28T13:15:59.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.848+0000 7f6a59200640 1 --2- 172.21.15.19:0/3794733002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3ff16c0 0x55eba3ff8920 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6a400054c0 tx=0x7f6a4000e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a50255b3e7c55c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.848+0000 7f6a3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:15:59.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.848+0000 7f6a3ea00640 1 -- 172.21.15.19:0/3794733002 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40003a60 con 0x55eba3ff16c0 2024-03-28T13:15:59.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.848+0000 7f6a3ea00640 1 -- 172.21.15.19:0/3794733002 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f6a40003c00 con 0x55eba3ff16c0 2024-03-28T13:15:59.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.848+0000 7f6a5f236740 1 -- 172.21.15.19:0/3794733002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a44034dd0 con 0x55eba3ff16c0 2024-03-28T13:15:59.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.849+0000 7f6a3ea00640 1 -- 172.21.15.19:0/3794733002 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f6a4001f050 con 0x55eba3ff16c0 2024-03-28T13:15:59.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.849+0000 7f6a5f236740 10 client.6960 fetch_fsmap learned FSMap version 53 2024-03-28T13:15:59.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.849+0000 7f6a5f236740 1 -- 172.21.15.19:0/3794733002 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55eba3bff0b0 con 0x55eba3ff16c0 2024-03-28T13:15:59.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.849+0000 7f6a3ea00640 1 -- 172.21.15.19:0/3794733002 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a44034dd0 con 0x55eba3ff16c0 2024-03-28T13:15:59.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.849+0000 7f6a3ea00640 1 -- 172.21.15.19:0/3794733002 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f6a40022020 con 0x55eba3ff16c0 2024-03-28T13:15:59.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.849+0000 7f6a5f236740 10 client.6960 fetch_fsmap finished waiting for FSMap version 53 2024-03-28T13:15:59.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.849+0000 7f6a5f236740 10 client.6960 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T13:15:59.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.849+0000 7f6a5f236740 20 client.6960 populate_metadata read hostname 'smithi019' 2024-03-28T13:15:59.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.849+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3794733002 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55eba3ff6860 0x55eba3ff6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.849+0000 7f6a5f236740 4 client.6960 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:15:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.849+0000 7f6a5f236740 1 -- 172.21.15.19:0/3794733002 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eba3eff870 con 0x55eba3ff6860 2024-03-28T13:15:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.850+0000 7f6a59c00640 1 --2- 172.21.15.19:0/3794733002 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55eba3ff6860 0x55eba3ff6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.850+0000 7f6a59c00640 1 --2- 172.21.15.19:0/3794733002 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55eba3ff6860 0x55eba3ff6c40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=654b8551ed0f2fda server_cookie=19dbcbacf2c2c32f in_seq=0 out_seq=0 2024-03-28T13:15:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.850+0000 7f6a3ea00640 10 client.6960 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.851+0000 7f6a3ea00640 1 -- 172.21.15.19:0/3794733002 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eba3eff870 con 0x55eba3ff6860 2024-03-28T13:15:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.851+0000 7f6a3ea00640 10 client.6960 handle_command_reply: tid=0 2024-03-28T13:15:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.851+0000 7f6a5f236740 1 client.6960 shutdown 2024-03-28T13:15:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.851+0000 7f6a5f236740 1 -- 172.21.15.19:0/3794733002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3ff16c0 msgr2=0x55eba3ff8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.851+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3794733002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3ff16c0 0x55eba3ff8920 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f6a400054c0 tx=0x7f6a4000e590 comp rx=0 tx=0).stop 2024-03-28T13:15:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.851+0000 7f6a5f236740 1 -- 172.21.15.19:0/3794733002 shutdown_connections 2024-03-28T13:15:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.851+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3794733002 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55eba3ff6860 0x55eba3ff6c40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.852+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3794733002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3fde3c0 0x55eba3fde800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.852+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3794733002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3ff16c0 0x55eba3ff8920 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.852+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3794733002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 0x55eba3ff3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.852+0000 7f6a3ea00640 0 client.6960 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:59.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.852+0000 7f6a5f236740 1 -- 172.21.15.19:0/3794733002 >> 172.21.15.19:0/3794733002 conn(0x55eba3fc1690 msgr2=0x55eba3fc6570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:59.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.852+0000 7f6a5f236740 1 -- 172.21.15.19:0/3794733002 shutdown_connections 2024-03-28T13:15:59.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.852+0000 7f6a5f236740 1 -- 172.21.15.19:0/3794733002 wait complete. 2024-03-28T13:15:59.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.852+0000 7f6a5f236740 20 client.6960 trim_cache size 0 max 16384 2024-03-28T13:15:59.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.882+0000 7f6a5f236740 1 Processor -- start 2024-03-28T13:15:59.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.882+0000 7f6a5f236740 1 -- start start 2024-03-28T13:15:59.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.883+0000 7f6a5f236740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3fc5240 0x55eba3ff8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.883+0000 7f6a5f236740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3fde3c0 0x55eba3fc6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.883+0000 7f6a5f236740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3ff16c0 0x55eba3fc1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.883+0000 7f6a5f236740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5415a440 con 0x55eba3fc5240 2024-03-28T13:15:59.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.883+0000 7f6a5f236740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6a541e8170 con 0x55eba3ff16c0 2024-03-28T13:15:59.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.883+0000 7f6a5f236740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6a54159df0 con 0x55eba3fde3c0 2024-03-28T13:15:59.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.884+0000 7f6a59c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3fc5240 0x55eba3ff8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.884+0000 7f6a59c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3fc5240 0x55eba3ff8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41962/0 (socket says 172.21.15.19:41962) 2024-03-28T13:15:59.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.884+0000 7f6a59c00640 1 -- 172.21.15.19:0/3383345658 learned_addr learned my addr 172.21.15.19:0/3383345658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:59.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.884+0000 7f6a59200640 1 --2- 172.21.15.19:0/3383345658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3fde3c0 0x55eba3fc6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.884+0000 7f6a5a600640 1 --2- 172.21.15.19:0/3383345658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3ff16c0 0x55eba3fc1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.884+0000 7f6a59c00640 1 -- 172.21.15.19:0/3383345658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3fde3c0 msgr2=0x55eba3fc6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.884+0000 7f6a59c00640 1 --2- 172.21.15.19:0/3383345658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3fde3c0 0x55eba3fc6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.884+0000 7f6a59c00640 1 -- 172.21.15.19:0/3383345658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3ff16c0 msgr2=0x55eba3fc1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.884+0000 7f6a59c00640 1 --2- 172.21.15.19:0/3383345658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3ff16c0 0x55eba3fc1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.884+0000 7f6a59c00640 1 -- 172.21.15.19:0/3383345658 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eba3fbd670 con 0x55eba3fc5240 2024-03-28T13:15:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.884+0000 7f6a5a600640 1 --2- 172.21.15.19:0/3383345658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3ff16c0 0x55eba3fc1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.885+0000 7f6a59c00640 1 --2- 172.21.15.19:0/3383345658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3fc5240 0x55eba3ff8e00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6a4402c4c0 tx=0x7f6a4403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b310d6d16359af server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.885+0000 7f6a49600640 1 -- 172.21.15.19:0/3383345658 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44051040 con 0x55eba3fc5240 2024-03-28T13:15:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.885+0000 7f6a49600640 1 -- 172.21.15.19:0/3383345658 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a44053040 con 0x55eba3fc5240 2024-03-28T13:15:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.885+0000 7f6a49600640 1 -- 172.21.15.19:0/3383345658 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a44039040 con 0x55eba3fc5240 2024-03-28T13:15:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.886+0000 7f6a5f236740 1 -- 172.21.15.19:0/3383345658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3fc5240 msgr2=0x55eba3ff8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.886+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3383345658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3fc5240 0x55eba3ff8e00 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6a4402c4c0 tx=0x7f6a4403f000 comp rx=0 tx=0).stop 2024-03-28T13:15:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.886+0000 7f6a5f236740 1 -- 172.21.15.19:0/3383345658 shutdown_connections 2024-03-28T13:15:59.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.886+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3383345658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3ff16c0 0x55eba3fc1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.886+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3383345658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3fde3c0 0x55eba3fc6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.886+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3383345658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3fc5240 0x55eba3ff8e00 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.886+0000 7f6a5f236740 1 -- 172.21.15.19:0/3383345658 >> 172.21.15.19:0/3383345658 conn(0x55eba3fc1690 msgr2=0x55eba3fc7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:59.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.886+0000 7f6a5f236740 1 -- 172.21.15.19:0/3383345658 shutdown_connections 2024-03-28T13:15:59.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.886+0000 7f6a5f236740 1 -- 172.21.15.19:0/3383345658 wait complete. 2024-03-28T13:15:59.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.887+0000 7f6a5f236740 1 -- start start 2024-03-28T13:15:59.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.887+0000 7f6a5f236740 1 Processor -- start 2024-03-28T13:15:59.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.889+0000 7f6a5f236740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 0x55eba406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.889+0000 7f6a5f236740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3fde3c0 0x55eba4030880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.889+0000 7f6a5f236740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3ff16c0 0x55eba3ff4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.889+0000 7f6a5f236740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eba3fdee80 con 0x55eba3fde3c0 2024-03-28T13:15:59.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.889+0000 7f6a5f236740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55eba3ff7720 con 0x55eba3fc5240 2024-03-28T13:15:59.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.889+0000 7f6a5f236740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55eba4020770 con 0x55eba3ff16c0 2024-03-28T13:15:59.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.889+0000 7f6a5f236740 10 client.0 _ll_register_callbacks cb 0x7f6a40007ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:15:59.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.889+0000 7f6a59200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3fde3c0 0x55eba4030880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.889+0000 7f6a59200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3fde3c0 0x55eba4030880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41966/0 (socket says 172.21.15.19:41966) 2024-03-28T13:15:59.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.889+0000 7f6a59200640 1 -- 172.21.15.19:0/3647711771 learned_addr learned my addr 172.21.15.19:0/3647711771 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:15:59.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.890+0000 7f6a59c00640 1 --2- 172.21.15.19:0/3647711771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 0x55eba406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.890+0000 7f6a59200640 1 -- 172.21.15.19:0/3647711771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3ff16c0 msgr2=0x55eba3ff4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.890+0000 7f6a5a600640 1 --2- 172.21.15.19:0/3647711771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3ff16c0 0x55eba3ff4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.890+0000 7f6a59200640 1 --2- 172.21.15.19:0/3647711771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3ff16c0 0x55eba3ff4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.890+0000 7f6a59200640 1 -- 172.21.15.19:0/3647711771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 msgr2=0x55eba406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:15:59.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.890+0000 7f6a59200640 1 --2- 172.21.15.19:0/3647711771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 0x55eba406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.890+0000 7f6a59200640 1 -- 172.21.15.19:0/3647711771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eba3fe1140 con 0x55eba3fde3c0 2024-03-28T13:15:59.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.890+0000 7f6a59c00640 1 --2- 172.21.15.19:0/3647711771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 0x55eba406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:15:59.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.890+0000 7f6a59200640 1 --2- 172.21.15.19:0/3647711771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3fde3c0 0x55eba4030880 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6a400054c0 tx=0x7f6a40003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a86c9e1e480bbd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:15:59.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.890+0000 7f6a3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:15:59.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.891+0000 7f6a3ea00640 1 -- 172.21.15.19:0/3647711771 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40004150 con 0x55eba3fde3c0 2024-03-28T13:15:59.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.891+0000 7f6a3ea00640 1 -- 172.21.15.19:0/3647711771 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f6a400042f0 con 0x55eba3fde3c0 2024-03-28T13:15:59.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.891+0000 7f6a5f236740 1 -- 172.21.15.19:0/3647711771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a44053040 con 0x55eba3fde3c0 2024-03-28T13:15:59.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.891+0000 7f6a3ea00640 1 -- 172.21.15.19:0/3647711771 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f6a40032050 con 0x55eba3fde3c0 2024-03-28T13:15:59.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.891+0000 7f6a5f236740 10 client.6966 fetch_fsmap learned FSMap version 53 2024-03-28T13:15:59.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.891+0000 7f6a5f236740 1 -- 172.21.15.19:0/3647711771 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55eba3ff2780 con 0x55eba3fde3c0 2024-03-28T13:15:59.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.891+0000 7f6a3ea00640 1 -- 172.21.15.19:0/3647711771 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a44053040 con 0x55eba3fde3c0 2024-03-28T13:15:59.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.891+0000 7f6a3ea00640 1 -- 172.21.15.19:0/3647711771 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 53) ==== 3494+0+0 (secure 0 0 0) 0x7f6a40035020 con 0x55eba3fde3c0 2024-03-28T13:15:59.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.891+0000 7f6a5f236740 10 client.6966 fetch_fsmap finished waiting for FSMap version 53 2024-03-28T13:15:59.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.891+0000 7f6a5f236740 10 client.6966 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T13:15:59.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.891+0000 7f6a5f236740 20 client.6966 populate_metadata read hostname 'smithi019' 2024-03-28T13:15:59.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.891+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3647711771 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55eba3ff54f0 0x55eba3fdcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:15:59.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.891+0000 7f6a5f236740 4 client.6966 mds_command: new command op to 6708 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-28T13:15:59.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.891+0000 7f6a5f236740 1 -- 172.21.15.19:0/3647711771 --> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55eba3f38ae0 con 0x55eba3ff54f0 2024-03-28T13:15:59.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.892+0000 7f6a59c00640 1 --2- 172.21.15.19:0/3647711771 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55eba3ff54f0 0x55eba3fdcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:15:59.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.892+0000 7f6a59c00640 1 --2- 172.21.15.19:0/3647711771 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55eba3ff54f0 0x55eba3fdcfc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=574318f60f2c7937 server_cookie=14b8abfb7d63c01b in_seq=0 out_seq=0 2024-03-28T13:15:59.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.892+0000 7f6a3ea00640 10 client.6966 ms_handle_connect on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:59.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.892+0000 7f6a3ea00640 1 -- 172.21.15.19:0/3647711771 <== mds.0 v2:172.21.15.19:6832/2130552083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eba3f38ae0 con 0x55eba3ff54f0 2024-03-28T13:15:59.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.892+0000 7f6a3ea00640 10 client.6966 handle_command_reply: tid=0 2024-03-28T13:15:59.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.892+0000 7f6a5f236740 1 client.6966 shutdown 2024-03-28T13:15:59.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.893+0000 7f6a5f236740 1 -- 172.21.15.19:0/3647711771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3fde3c0 msgr2=0x55eba4030880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.893+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3647711771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3fde3c0 0x55eba4030880 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6a400054c0 tx=0x7f6a40003760 comp rx=0 tx=0).stop 2024-03-28T13:15:59.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.893+0000 7f6a5f236740 1 -- 172.21.15.19:0/3647711771 shutdown_connections 2024-03-28T13:15:59.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.893+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3647711771 >> [v2:172.21.15.19:6832/2130552083,v1:172.21.15.19:6834/2130552083] conn(0x55eba3ff54f0 0x55eba3fdcfc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.893+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3647711771 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55eba3ff16c0 0x55eba3ff4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.893+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3647711771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eba3fde3c0 0x55eba4030880 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.893+0000 7f6a3ea00640 0 client.6966 ms_handle_reset on v2:172.21.15.19:6832/2130552083 2024-03-28T13:15:59.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.893+0000 7f6a5f236740 1 --2- 172.21.15.19:0/3647711771 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55eba3fc5240 0x55eba406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.893+0000 7f6a5f236740 1 -- 172.21.15.19:0/3647711771 >> 172.21.15.19:0/3647711771 conn(0x55eba3fc1690 msgr2=0x55eba3fc1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:59.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.893+0000 7f6a5f236740 1 -- 172.21.15.19:0/3647711771 shutdown_connections 2024-03-28T13:15:59.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.893+0000 7f6a5f236740 1 -- 172.21.15.19:0/3647711771 wait complete. 2024-03-28T13:15:59.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.893+0000 7f6a5f236740 20 client.6966 trim_cache size 0 max 16384 2024-03-28T13:15:59.896 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:15:59.896 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-03-28T13:15:59.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.894+0000 7f6a5c600640 1 -- 172.21.15.19:0/1190370885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a3406c5b0 msgr2=0x7f6a3406ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.894+0000 7f6a5c600640 1 --2- 172.21.15.19:0/1190370885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a3406c5b0 0x7f6a3406ea70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6a44036000 tx=0x7f6a44030000 comp rx=0 tx=0).stop 2024-03-28T13:15:59.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.894+0000 7f6a5c600640 1 -- 172.21.15.19:0/1190370885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a54162160 msgr2=0x7f6a541e8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:15:59.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.894+0000 7f6a5c600640 1 --2- 172.21.15.19:0/1190370885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a54162160 0x7f6a541e8ef0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00c3a0 tx=0x7f6a4c00c860 comp rx=0 tx=0).stop 2024-03-28T13:15:59.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.894+0000 7f6a5c600640 1 -- 172.21.15.19:0/1190370885 shutdown_connections 2024-03-28T13:15:59.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.894+0000 7f6a5c600640 1 --2- 172.21.15.19:0/1190370885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a3406c5b0 0x7f6a3406ea70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.894+0000 7f6a5c600640 1 --2- 172.21.15.19:0/1190370885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a54162160 0x7f6a541e8ef0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.894+0000 7f6a5c600640 1 --2- 172.21.15.19:0/1190370885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a54141c10 0x7f6a5409b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.894+0000 7f6a5c600640 1 --2- 172.21.15.19:0/1190370885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a540d0280 0x7f6a54157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:15:59.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.894+0000 7f6a5c600640 1 -- 172.21.15.19:0/1190370885 >> 172.21.15.19:0/1190370885 conn(0x7f6a5406b750 msgr2=0x7f6a54160fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:15:59.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.894+0000 7f6a5c600640 1 -- 172.21.15.19:0/1190370885 shutdown_connections 2024-03-28T13:15:59.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:15:59.894+0000 7f6a5c600640 1 -- 172.21.15.19:0/1190370885 wait complete. 2024-03-28T13:15:59.912 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-28T13:16:00.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.113+0000 7f3010e00640 1 Processor -- start 2024-03-28T13:16:00.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.113+0000 7f3010e00640 1 -- start start 2024-03-28T13:16:00.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.115+0000 7f3010e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f300c13f300 0x7f300c06e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:00.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.115+0000 7f3010e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f300c1419e0 0x7f300c15fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:00.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.115+0000 7f3010e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f300c142880 con 0x7f300c141e40 2024-03-28T13:16:00.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.115+0000 7f3010e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f300c0cee90 con 0x7f300c1419e0 2024-03-28T13:16:00.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.115+0000 7f3010e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f300c1422c0 con 0x7f300c13f300 2024-03-28T13:16:00.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.115+0000 7f300aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f300c1419e0 0x7f300c15fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:00.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.115+0000 7f300aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f300c1419e0 0x7f300c15fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41980/0 (socket says 172.21.15.19:41980) 2024-03-28T13:16:00.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.115+0000 7f300aa00640 1 -- 172.21.15.19:0/922065707 learned_addr learned my addr 172.21.15.19:0/922065707 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:00.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.115+0000 7f300b400640 1 --2- 172.21.15.19:0/922065707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f300c13f300 0x7f300c06e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:00.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.116+0000 7f300aa00640 1 -- 172.21.15.19:0/922065707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f300c13f300 msgr2=0x7f300c06e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:00.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.116+0000 7f300aa00640 1 --2- 172.21.15.19:0/922065707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f300c13f300 0x7f300c06e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.116+0000 7f300aa00640 1 -- 172.21.15.19:0/922065707 >> v1:172.21.15.188:6789/0 conn(0x7f300c141e40 legacy=0x7f300c15c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:00.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.116+0000 7f300aa00640 1 -- 172.21.15.19:0/922065707 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c1425a0 con 0x7f300c1419e0 2024-03-28T13:16:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.117+0000 7f300aa00640 1 --2- 172.21.15.19:0/922065707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c1419e0 0x7f300c15fd90 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009870 tx=0x7f2ff802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c5c18ffb395def server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.117+0000 7f300a000640 1 -- 172.21.15.19:0/922065707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8035020 con 0x7f300c1419e0 2024-03-28T13:16:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.117+0000 7f300a000640 1 -- 172.21.15.19:0/922065707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8031640 con 0x7f300c1419e0 2024-03-28T13:16:00.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.117+0000 7f300a000640 1 -- 172.21.15.19:0/922065707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff802d040 con 0x7f300c1419e0 2024-03-28T13:16:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.118+0000 7f3010e00640 1 -- 172.21.15.19:0/922065707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c1419e0 msgr2=0x7f300c15fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.118+0000 7f3010e00640 1 --2- 172.21.15.19:0/922065707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c1419e0 0x7f300c15fd90 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009870 tx=0x7f2ff802b8f0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.118+0000 7f3010e00640 1 -- 172.21.15.19:0/922065707 shutdown_connections 2024-03-28T13:16:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.118+0000 7f3010e00640 1 --2- 172.21.15.19:0/922065707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c1419e0 0x7f300c15fd90 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.118+0000 7f3010e00640 1 --2- 172.21.15.19:0/922065707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f300c13f300 0x7f300c06e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.118+0000 7f3010e00640 1 -- 172.21.15.19:0/922065707 >> 172.21.15.19:0/922065707 conn(0x7f300c06b570 msgr2=0x7f300c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:00.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.118+0000 7f3010e00640 1 -- 172.21.15.19:0/922065707 shutdown_connections 2024-03-28T13:16:00.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.118+0000 7f3010e00640 1 -- 172.21.15.19:0/922065707 wait complete. 2024-03-28T13:16:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.121+0000 7f3010e00640 1 Processor -- start 2024-03-28T13:16:00.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.121+0000 7f3010e00640 1 -- start start 2024-03-28T13:16:00.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.123+0000 7f3010e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f300c13f300 0x7f300c1f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:00.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.123+0000 7f3010e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c1419e0 0x7f300c0b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:00.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.123+0000 7f3010e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f300c141e40 0x7f300c0afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:00.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.123+0000 7f3010e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f300c0d04b0 con 0x7f300c1419e0 2024-03-28T13:16:00.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.123+0000 7f3010e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f300c0d0330 con 0x7f300c141e40 2024-03-28T13:16:00.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.123+0000 7f3010e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f300c0d0630 con 0x7f300c13f300 2024-03-28T13:16:00.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.123+0000 7f300aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c1419e0 0x7f300c0b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:00.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.123+0000 7f300aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c1419e0 0x7f300c0b1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41982/0 (socket says 172.21.15.19:41982) 2024-03-28T13:16:00.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.123+0000 7f300aa00640 1 -- 172.21.15.19:0/1765829013 learned_addr learned my addr 172.21.15.19:0/1765829013 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.124+0000 7f300b400640 1 --2- 172.21.15.19:0/1765829013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f300c13f300 0x7f300c1f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.124+0000 7f300be00640 1 --2- 172.21.15.19:0/1765829013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f300c141e40 0x7f300c0afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.124+0000 7f300aa00640 1 -- 172.21.15.19:0/1765829013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f300c13f300 msgr2=0x7f300c1f7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.124+0000 7f300aa00640 1 --2- 172.21.15.19:0/1765829013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f300c13f300 0x7f300c1f7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.124+0000 7f300aa00640 1 -- 172.21.15.19:0/1765829013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f300c141e40 msgr2=0x7f300c0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.124+0000 7f300aa00640 1 --2- 172.21.15.19:0/1765829013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f300c141e40 0x7f300c0afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.124+0000 7f300aa00640 1 -- 172.21.15.19:0/1765829013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c0b0760 con 0x7f300c1419e0 2024-03-28T13:16:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.124+0000 7f300aa00640 1 --2- 172.21.15.19:0/1765829013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c1419e0 0x7f300c0b1800 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2ff802b930 tx=0x7f2ff80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.124+0000 7f3003400640 1 -- 172.21.15.19:0/1765829013 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8040070 con 0x7f300c1419e0 2024-03-28T13:16:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.125+0000 7f3003400640 1 -- 172.21.15.19:0/1765829013 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff80399f0 con 0x7f300c1419e0 2024-03-28T13:16:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.125+0000 7f3010e00640 1 -- 172.21.15.19:0/1765829013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c0b0a40 con 0x7f300c1419e0 2024-03-28T13:16:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.125+0000 7f3003400640 1 -- 172.21.15.19:0/1765829013 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8035040 con 0x7f300c1419e0 2024-03-28T13:16:00.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.125+0000 7f3010e00640 1 -- 172.21.15.19:0/1765829013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c0b12e0 con 0x7f300c1419e0 2024-03-28T13:16:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.126+0000 7f3010e00640 1 -- 172.21.15.19:0/1765829013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f300c075940 con 0x7f300c1419e0 2024-03-28T13:16:00.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.127+0000 7f3003400640 1 -- 172.21.15.19:0/1765829013 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2ff804b060 con 0x7f300c1419e0 2024-03-28T13:16:00.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.128+0000 7f3003400640 1 --2- 172.21.15.19:0/1765829013 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2fec06c530 0x7f2fec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.128+0000 7f3003400640 1 -- 172.21.15.19:0/1765829013 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f2ff80ad170 con 0x7f300c1419e0 2024-03-28T13:16:00.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.129+0000 7f300b400640 1 --2- 172.21.15.19:0/1765829013 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2fec06c530 0x7f2fec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:00.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.130+0000 7f300b400640 1 --2- 172.21.15.19:0/1765829013 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2fec06c530 0x7f2fec06e9f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc006fd0 tx=0x7f2ffc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.132+0000 7f3003400640 1 -- 172.21.15.19:0/1765829013 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2ff807b660 con 0x7f300c1419e0 2024-03-28T13:16:00.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.380+0000 7f3010e00640 1 -- 172.21.15.19:0/1765829013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f300c13ee70 con 0x7f300c1419e0 2024-03-28T13:16:00.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.380+0000 7f3003400640 1 -- 172.21.15.19:0/1765829013 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2ff807af60 con 0x7f300c1419e0 2024-03-28T13:16:00.380 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:00.380 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-28T13:16:00.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.383+0000 7f3010e00640 1 -- 172.21.15.19:0/1765829013 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2fec06c530 msgr2=0x7f2fec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:00.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.383+0000 7f3010e00640 1 --2- 172.21.15.19:0/1765829013 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2fec06c530 0x7f2fec06e9f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc006fd0 tx=0x7f2ffc008040 comp rx=0 tx=0).stop 2024-03-28T13:16:00.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.383+0000 7f3010e00640 1 -- 172.21.15.19:0/1765829013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c1419e0 msgr2=0x7f300c0b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:00.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.383+0000 7f3010e00640 1 --2- 172.21.15.19:0/1765829013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c1419e0 0x7f300c0b1800 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2ff802b930 tx=0x7f2ff80046c0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.383+0000 7f3010e00640 1 -- 172.21.15.19:0/1765829013 shutdown_connections 2024-03-28T13:16:00.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.383+0000 7f3010e00640 1 --2- 172.21.15.19:0/1765829013 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2fec06c530 0x7f2fec06e9f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.383+0000 7f3010e00640 1 --2- 172.21.15.19:0/1765829013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f300c141e40 0x7f300c0afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.383+0000 7f3010e00640 1 --2- 172.21.15.19:0/1765829013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f300c1419e0 0x7f300c0b1800 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.383+0000 7f3010e00640 1 --2- 172.21.15.19:0/1765829013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f300c13f300 0x7f300c1f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.383+0000 7f3010e00640 1 -- 172.21.15.19:0/1765829013 >> 172.21.15.19:0/1765829013 conn(0x7f300c06b570 msgr2=0x7f300c13d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:00.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.383+0000 7f3010e00640 1 -- 172.21.15.19:0/1765829013 shutdown_connections 2024-03-28T13:16:00.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.383+0000 7f3010e00640 1 -- 172.21.15.19:0/1765829013 wait complete. 2024-03-28T13:16:00.411 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-28T13:16:00.411 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:16:00.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.617+0000 7f4be4a00640 1 Processor -- start 2024-03-28T13:16:00.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.617+0000 7f4be4a00640 1 -- start start 2024-03-28T13:16:00.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.619+0000 7f4be4a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4be015e420 0x7f4be00b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:00.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.619+0000 7f4be4a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4be00d2fd0 con 0x7f4be00b3460 2024-03-28T13:16:00.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.619+0000 7f4be4a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4be00cfa70 con 0x7f4be00d1a90 2024-03-28T13:16:00.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.619+0000 7f4be4a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f4be00cf790 con 0x7f4be015e420 2024-03-28T13:16:00.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.619+0000 7f4bd7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4be00b3460 0x7f4be00d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38426/0 (socket says 172.21.15.19:38426) 2024-03-28T13:16:00.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.619+0000 7f4bd7e00640 1 -- 172.21.15.19:0/1159731543 learned_addr learned my addr 172.21.15.19:0/1159731543 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.619+0000 7f4bde800640 1 --2- 172.21.15.19:0/1159731543 >> v2:172.21.15.188:3300/0 conn(0x7f4be015e420 0x7f4be00b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.620+0000 7f4bdde00640 1 -- 172.21.15.19:0/1159731543 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 943361183 0 0) 0x7f4be00d2fd0 con 0x7f4be00b3460 2024-03-28T13:16:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.620+0000 7f4bdde00640 1 -- 172.21.15.19:0/1159731543 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bc4003520 con 0x7f4be00b3460 2024-03-28T13:16:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.620+0000 7f4bdde00640 1 -- 172.21.15.19:0/1159731543 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2355427766 0 0) 0x7f4bc4003520 con 0x7f4be00b3460 2024-03-28T13:16:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.620+0000 7f4bdde00640 1 -- 172.21.15.19:0/1159731543 >> v2:172.21.15.188:3300/0 conn(0x7f4be015e420 msgr2=0x7f4be00b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.620+0000 7f4bdde00640 1 --2- 172.21.15.19:0/1159731543 >> v2:172.21.15.188:3300/0 conn(0x7f4be015e420 0x7f4be00b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.620+0000 7f4bdde00640 1 -- 172.21.15.19:0/1159731543 >> v1:172.21.15.188:6789/0 conn(0x7f4be00d1a90 legacy=0x7f4be00d3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.620+0000 7f4bdde00640 1 -- 172.21.15.19:0/1159731543 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be00d3270 con 0x7f4be00b3460 2024-03-28T13:16:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.620+0000 7f4bdde00640 1 -- 172.21.15.19:0/1159731543 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f4bc8001670 con 0x7f4be00b3460 2024-03-28T13:16:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.620+0000 7f4bdde00640 1 -- 172.21.15.19:0/1159731543 >> v1:172.21.15.19:6789/0 conn(0x7f4be00b3460 legacy=0x7f4be00d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:00.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.620+0000 7f4bdde00640 1 --2- 172.21.15.19:0/1159731543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4003a00 0x7f4bc4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:00.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.620+0000 7f4bdde00640 1 --2- 172.21.15.19:0/1159731543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc4004800 0x7f4bc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:00.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.620+0000 7f4bdde00640 1 --2- 172.21.15.19:0/1159731543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bc40051c0 0x7f4bc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:00.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.620+0000 7f4bdde00640 1 -- 172.21.15.19:0/1159731543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4be00cf790 con 0x7f4bc4003a00 2024-03-28T13:16:00.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.620+0000 7f4bdde00640 1 -- 172.21.15.19:0/1159731543 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc400c0a0 con 0x7f4bc4004800 2024-03-28T13:16:00.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.620+0000 7f4bdde00640 1 -- 172.21.15.19:0/1159731543 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc400c2a0 con 0x7f4bc40051c0 2024-03-28T13:16:00.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.621+0000 7f4bde800640 1 --2- 172.21.15.19:0/1159731543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4003a00 0x7f4bc4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:00.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.621+0000 7f4bde800640 1 -- 172.21.15.19:0/1159731543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bc40051c0 msgr2=0x7f4bc400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:00.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.621+0000 7f4bde800640 1 --2- 172.21.15.19:0/1159731543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bc40051c0 0x7f4bc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.621+0000 7f4bde800640 1 -- 172.21.15.19:0/1159731543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc4004800 msgr2=0x7f4bc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:00.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.621+0000 7f4bde800640 1 --2- 172.21.15.19:0/1159731543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc4004800 0x7f4bc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.621+0000 7f4bde800640 1 -- 172.21.15.19:0/1159731543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc4003520 con 0x7f4bc4003a00 2024-03-28T13:16:00.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.621+0000 7f4bdf200640 1 -- 172.21.15.19:0/1159731543 reap_dead start 2024-03-28T13:16:00.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.622+0000 7f4bde800640 1 --2- 172.21.15.19:0/1159731543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4003a00 0x7f4bc4003e20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009ac0 tx=0x7f4bc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560e1e18197c59ce server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:00.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.622+0000 7f4bdde00640 1 -- 172.21.15.19:0/1159731543 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0039070 con 0x7f4bc4003a00 2024-03-28T13:16:00.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.622+0000 7f4bdde00640 1 -- 172.21.15.19:0/1159731543 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd002c710 con 0x7f4bc4003a00 2024-03-28T13:16:00.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.622+0000 7f4bdde00640 1 -- 172.21.15.19:0/1159731543 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0009690 con 0x7f4bc4003a00 2024-03-28T13:16:00.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.623+0000 7f4be4a00640 1 -- 172.21.15.19:0/1159731543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4003a00 msgr2=0x7f4bc4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:00.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.623+0000 7f4be4a00640 1 --2- 172.21.15.19:0/1159731543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4003a00 0x7f4bc4003e20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009ac0 tx=0x7f4bc4005830 comp rx=0 tx=0).stop 2024-03-28T13:16:00.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.623+0000 7f4be4a00640 1 -- 172.21.15.19:0/1159731543 shutdown_connections 2024-03-28T13:16:00.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.623+0000 7f4be4a00640 1 --2- 172.21.15.19:0/1159731543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4bc4003a00 0x7f4bc4003e20 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.623+0000 7f4be4a00640 1 -- 172.21.15.19:0/1159731543 >> 172.21.15.19:0/1159731543 conn(0x7f4be0067670 msgr2=0x7f4be0067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:00.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.623+0000 7f4be4a00640 1 -- 172.21.15.19:0/1159731543 shutdown_connections 2024-03-28T13:16:00.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.623+0000 7f4be4a00640 1 -- 172.21.15.19:0/1159731543 wait complete. 2024-03-28T13:16:00.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.625+0000 7f4be4a00640 1 Processor -- start 2024-03-28T13:16:00.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.625+0000 7f4be4a00640 1 -- start start 2024-03-28T13:16:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.626+0000 7f4be4a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc4003a00 0x7f4be0152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.626+0000 7f4be4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4be01f7a30 0x7f4be01f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.626+0000 7f4be4a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4be01f83f0 0x7f4be00cc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.626+0000 7f4be4a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4be00cf420 con 0x7f4be01f7a30 2024-03-28T13:16:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.626+0000 7f4be4a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4be015e800 con 0x7f4bc4003a00 2024-03-28T13:16:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.626+0000 7f4be4a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4be00b2f20 con 0x7f4be01f83f0 2024-03-28T13:16:00.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.626+0000 7f4bd7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4be01f7a30 0x7f4be01f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.626+0000 7f4bd7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4be01f7a30 0x7f4be01f7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42006/0 (socket says 172.21.15.19:42006) 2024-03-28T13:16:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.626+0000 7f4bd7e00640 1 -- 172.21.15.19:0/1448748529 learned_addr learned my addr 172.21.15.19:0/1448748529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.627+0000 7f4bdf200640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4be01f83f0 0x7f4be00cc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.627+0000 7f4bde800640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc4003a00 0x7f4be0152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.627+0000 7f4bd7e00640 1 -- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4be01f83f0 msgr2=0x7f4be00cc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.627+0000 7f4bd7e00640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4be01f83f0 0x7f4be00cc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.627+0000 7f4bd7e00640 1 -- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc4003a00 msgr2=0x7f4be0152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.627+0000 7f4bd7e00640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc4003a00 0x7f4be0152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.627+0000 7f4bd7e00640 1 -- 172.21.15.19:0/1448748529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be01e8c90 con 0x7f4be01f7a30 2024-03-28T13:16:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.627+0000 7f4bdf200640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4be01f83f0 0x7f4be00cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.627+0000 7f4bde800640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc4003a00 0x7f4be0152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:00.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.627+0000 7f4bd7e00640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4be01f7a30 0x7f4be01f7eb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800c730 tx=0x7f4bc800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:00.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.627+0000 7f4bd6a00640 1 -- 172.21.15.19:0/1448748529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8018070 con 0x7f4be01f7a30 2024-03-28T13:16:00.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.628+0000 7f4bd6a00640 1 -- 172.21.15.19:0/1448748529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc80072f0 con 0x7f4be01f7a30 2024-03-28T13:16:00.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.628+0000 7f4bd6a00640 1 -- 172.21.15.19:0/1448748529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80047e0 con 0x7f4be01f7a30 2024-03-28T13:16:00.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.628+0000 7f4be4a00640 1 -- 172.21.15.19:0/1448748529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be01e8f70 con 0x7f4be01f7a30 2024-03-28T13:16:00.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.628+0000 7f4be4a00640 1 -- 172.21.15.19:0/1448748529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be01e9530 con 0x7f4be01f7a30 2024-03-28T13:16:00.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.629+0000 7f4be4a00640 1 -- 172.21.15.19:0/1448748529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be0075940 con 0x7f4be01f7a30 2024-03-28T13:16:00.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.630+0000 7f4bd6a00640 1 -- 172.21.15.19:0/1448748529 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4bc801b070 con 0x7f4be01f7a30 2024-03-28T13:16:00.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.630+0000 7f4bd6a00640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4bbc06c560 0x7f4bbc06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:00.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.631+0000 7f4bd6a00640 1 -- 172.21.15.19:0/1448748529 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f4bc808acc0 con 0x7f4be01f7a30 2024-03-28T13:16:00.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.631+0000 7f4bde800640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4bbc06c560 0x7f4bbc06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:00.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.632+0000 7f4bde800640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4bbc06c560 0x7f4bbc06ea20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0036000 tx=0x7f4bd0030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.640+0000 7f4bd6a00640 1 -- 172.21.15.19:0/1448748529 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4bc8091080 con 0x7f4be01f7a30 2024-03-28T13:16:00.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.886+0000 7f4be4a00640 1 -- 172.21.15.19:0/1448748529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4be00b49a0 con 0x7f4be01f7a30 2024-03-28T13:16:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.887+0000 7f4bd6a00640 1 -- 172.21.15.19:0/1448748529 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) ==== 53+0+73 (secure 0 0 0) 0x7f4bc8059120 con 0x7f4be01f7a30 2024-03-28T13:16:00.887 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:16:00.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.890+0000 7f4be4a00640 1 -- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4bbc06c560 msgr2=0x7f4bbc06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:00.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.890+0000 7f4be4a00640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4bbc06c560 0x7f4bbc06ea20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0036000 tx=0x7f4bd0030000 comp rx=0 tx=0).stop 2024-03-28T13:16:00.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.890+0000 7f4be4a00640 1 -- 172.21.15.19:0/1448748529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4be01f7a30 msgr2=0x7f4be01f7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:00.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.890+0000 7f4be4a00640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4be01f7a30 0x7f4be01f7eb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800c730 tx=0x7f4bc800cbf0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.890+0000 7f4be4a00640 1 -- 172.21.15.19:0/1448748529 shutdown_connections 2024-03-28T13:16:00.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.890+0000 7f4be4a00640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4bbc06c560 0x7f4bbc06ea20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.890+0000 7f4be4a00640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4be01f83f0 0x7f4be00cc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.890+0000 7f4be4a00640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4be01f7a30 0x7f4be01f7eb0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.890+0000 7f4be4a00640 1 --2- 172.21.15.19:0/1448748529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc4003a00 0x7f4be0152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.890+0000 7f4be4a00640 1 -- 172.21.15.19:0/1448748529 >> 172.21.15.19:0/1448748529 conn(0x7f4be0067670 msgr2=0x7f4be00b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.890+0000 7f4be4a00640 1 -- 172.21.15.19:0/1448748529 shutdown_connections 2024-03-28T13:16:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:00.891+0000 7f4be4a00640 1 -- 172.21.15.19:0/1448748529 wait complete. 2024-03-28T13:16:00.909 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-28T13:16:00.909 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:16:00.909 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:16:00.909 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:16:00.909 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:16:00.909 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-28T13:16:00.910 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-28T13:16:00.910 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:16:00.910 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:16:00.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:00.910 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:16:00.964 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:16:00.964 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42194sec preferred_lft 42194sec 2024-03-28T13:16:00.965 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:16:00.966 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:00.966 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:16:00.966 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:16:00.966 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:16:00.966 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:16:00.966 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:16:00.966 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:16:00.966 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:00.966 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:16:00.966 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:00.966 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:16:00.966 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:16:00.966 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:16:00.966 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:00.967 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:16:00.967 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether e6:17:c5:8e:2b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:16:00.967 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::e417:c5ff:fe8e:2bcf/64 scope link 2024-03-28T13:16:00.967 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:00.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:00.967 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:16:01.045 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:16:01.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:01.051 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:16:01.098 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:16:01.099 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:16:01.099 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:16:01.099 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:16:01.099 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:16:01.100 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:16:01.100 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:16:01.100 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:16:01.100 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:16:01.100 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:16:01.100 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:16:01.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:16:01.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:16:01.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:01.345 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:16:01.387 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:16:01.387 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:16:01.387 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:16:01.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:01.387 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:16:01.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:16:01.547 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:16:01.605 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:16:01.605 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:16:01.605 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:16:01.651 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:16:01.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:01.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:16:01.839 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:16:01.839 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:16:01.839 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:16:01.860 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:16:01.860 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:16:01.860 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:16:01.860 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:16:01.860 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:16:01.886 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:16:01.886 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:16:01.886 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:16:01.886 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:16:01.887 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:16:01.887 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:16:01.887 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:16:01.887 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:16:01.889 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:16:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:16:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:16:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:16:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:16:01.892 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:16:01.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:01.892 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:16:01.970 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:16:02.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:16:02.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:16:02.089 INFO:teuthology.orchestra.run.smithi019.stdout:client5549 2024-03-28T13:16:02.089 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:16:02.089 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:16:02.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:02.141 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-03-28T13:16:02.187 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:16:02.187 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-03-28T13:16:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.271+0000 7f162f644f40 1 Processor -- start 2024-03-28T13:16:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.271+0000 7f162f644f40 1 -- start start 2024-03-28T13:16:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.272+0000 7f162f644f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c50b649010 0x55c50b6d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.272+0000 7f162f644f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c50b64a4d0 0x55c50b6d99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.272+0000 7f162f644f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c50b6d8bd0 0x55c50b6d6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.272+0000 7f162f644f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c50b5a82c0 con 0x55c50b64a4d0 2024-03-28T13:16:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.272+0000 7f162f644f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c50b5ac310 con 0x55c50b6d8bd0 2024-03-28T13:16:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.272+0000 7f162f644f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55c50b641220 con 0x55c50b649010 2024-03-28T13:16:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.272+0000 7f162b600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c50b64a4d0 0x55c50b6d99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.273+0000 7f162b600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c50b64a4d0 0x55c50b6d99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42028/0 (socket says 172.21.15.19:42028) 2024-03-28T13:16:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.273+0000 7f162b600640 1 -- 172.21.15.19:0/2213927801 learned_addr learned my addr 172.21.15.19:0/2213927801 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.273+0000 7f162c000640 1 --2- 172.21.15.19:0/2213927801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c50b649010 0x55c50b6d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.273+0000 7f162b600640 1 -- 172.21.15.19:0/2213927801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c50b649010 msgr2=0x55c50b6d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.273+0000 7f162b600640 1 --2- 172.21.15.19:0/2213927801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c50b649010 0x55c50b6d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.273+0000 7f162b600640 1 -- 172.21.15.19:0/2213927801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c50b6d8bd0 msgr2=0x55c50b6d6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.273+0000 7f162b600640 1 --2- 172.21.15.19:0/2213927801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c50b6d8bd0 0x55c50b6d6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.273+0000 7f162b600640 1 -- 172.21.15.19:0/2213927801 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50b6d0a30 con 0x55c50b64a4d0 2024-03-28T13:16:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.273+0000 7f162b600640 1 --2- 172.21.15.19:0/2213927801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c50b64a4d0 0x55c50b6d99c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f161800b310 tx=0x7f161800b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda058ab5dbc33c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.274+0000 7f162a200640 1 -- 172.21.15.19:0/2213927801 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1618008070 con 0x55c50b64a4d0 2024-03-28T13:16:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.274+0000 7f162a200640 1 -- 172.21.15.19:0/2213927801 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161800d500 con 0x55c50b64a4d0 2024-03-28T13:16:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.274+0000 7f162a200640 1 -- 172.21.15.19:0/2213927801 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161800d8b0 con 0x55c50b64a4d0 2024-03-28T13:16:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.274+0000 7f162f644f40 1 -- 172.21.15.19:0/2213927801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c50b64a4d0 msgr2=0x55c50b6d99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.274+0000 7f162f644f40 1 --2- 172.21.15.19:0/2213927801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c50b64a4d0 0x55c50b6d99c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f161800b310 tx=0x7f161800b7d0 comp rx=0 tx=0).stop 2024-03-28T13:16:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.274+0000 7f162f644f40 1 -- 172.21.15.19:0/2213927801 shutdown_connections 2024-03-28T13:16:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.274+0000 7f162f644f40 1 --2- 172.21.15.19:0/2213927801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c50b6d8bd0 0x55c50b6d6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.274+0000 7f162f644f40 1 --2- 172.21.15.19:0/2213927801 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c50b64a4d0 0x55c50b6d99c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.274+0000 7f162f644f40 1 --2- 172.21.15.19:0/2213927801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c50b649010 0x55c50b6d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.274+0000 7f162f644f40 1 -- 172.21.15.19:0/2213927801 >> 172.21.15.19:0/2213927801 conn(0x55c50b62bbf0 msgr2=0x55c50b630f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.274+0000 7f162f644f40 1 -- 172.21.15.19:0/2213927801 shutdown_connections 2024-03-28T13:16:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.274+0000 7f162f644f40 1 -- 172.21.15.19:0/2213927801 wait complete. 2024-03-28T13:16:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.275+0000 7f162f644f40 1 Processor -- start 2024-03-28T13:16:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.275+0000 7f162f644f40 1 -- start start 2024-03-28T13:16:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.276+0000 7f162f644f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c50b649010 0x55c50b6d6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.276+0000 7f162f644f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c50b64a4d0 0x55c50b6d71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.276+0000 7f162f644f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c50b6d8bd0 0x55c50b6c7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.276+0000 7f162f644f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c50b604120 con 0x55c50b649010 2024-03-28T13:16:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.276+0000 7f162f644f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55c50b642950 con 0x55c50b6d8bd0 2024-03-28T13:16:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.276+0000 7f162f644f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55c50b642ba0 con 0x55c50b64a4d0 2024-03-28T13:16:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.277+0000 7f162c000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c50b649010 0x55c50b6d6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.277+0000 7f162c000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c50b649010 0x55c50b6d6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42036/0 (socket says 172.21.15.19:42036) 2024-03-28T13:16:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.277+0000 7f162c000640 1 -- 172.21.15.19:0/1954268983 learned_addr learned my addr 172.21.15.19:0/1954268983 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.277+0000 7f162ca00640 1 --2- 172.21.15.19:0/1954268983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c50b6d8bd0 0x55c50b6c7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.277+0000 7f162c000640 1 -- 172.21.15.19:0/1954268983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c50b64a4d0 msgr2=0x55c50b6d71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.277+0000 7f162b600640 1 --2- 172.21.15.19:0/1954268983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c50b64a4d0 0x55c50b6d71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.277+0000 7f162c000640 1 --2- 172.21.15.19:0/1954268983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c50b64a4d0 0x55c50b6d71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.277+0000 7f162c000640 1 -- 172.21.15.19:0/1954268983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c50b6d8bd0 msgr2=0x55c50b6c7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.277+0000 7f162c000640 1 --2- 172.21.15.19:0/1954268983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c50b6d8bd0 0x55c50b6c7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.277+0000 7f162c000640 1 -- 172.21.15.19:0/1954268983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c50b6c8960 con 0x55c50b649010 2024-03-28T13:16:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.277+0000 7f162ca00640 1 --2- 172.21.15.19:0/1954268983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c50b6d8bd0 0x55c50b6c7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.278+0000 7f162c000640 1 --2- 172.21.15.19:0/1954268983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c50b649010 0x55c50b6d6730 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f161c00a4f0 tx=0x7f161c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.278+0000 7f1628e00640 1 -- 172.21.15.19:0/1954268983 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c062070 con 0x55c50b649010 2024-03-28T13:16:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.278+0000 7f1628e00640 1 -- 172.21.15.19:0/1954268983 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f161c052880 con 0x55c50b649010 2024-03-28T13:16:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.278+0000 7f162f644f40 1 -- 172.21.15.19:0/1954268983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c50b6c86c0 con 0x55c50b649010 2024-03-28T13:16:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.278+0000 7f1628e00640 1 -- 172.21.15.19:0/1954268983 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c057040 con 0x55c50b649010 2024-03-28T13:16:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.278+0000 7f162f644f40 1 -- 172.21.15.19:0/1954268983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50b6cb8a0 con 0x55c50b649010 2024-03-28T13:16:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.280+0000 7f1628e00640 1 -- 172.21.15.19:0/1954268983 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f161c065070 con 0x55c50b649010 2024-03-28T13:16:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.281+0000 7f1628e00640 1 --2- 172.21.15.19:0/1954268983 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160c06af40 0x7f160c06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.281+0000 7f1628e00640 1 -- 172.21.15.19:0/1954268983 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f161c0cbe90 con 0x55c50b649010 2024-03-28T13:16:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.281+0000 7f162f644f40 1 --2- 172.21.15.19:0/1954268983 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x55c50b648bf0 0x55c50b6d3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.281+0000 7f162f644f40 1 -- 172.21.15.19:0/1954268983 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80) -- 0x55c50b6cc7b0 con 0x55c50b648bf0 2024-03-28T13:16:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.282+0000 7f162b600640 1 --2- 172.21.15.19:0/1954268983 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160c06af40 0x7f160c06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.282+0000 7f162ca00640 1 --2- 172.21.15.19:0/1954268983 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x55c50b648bf0 0x55c50b6d3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.282+0000 7f162ca00640 1 --2- 172.21.15.19:0/1954268983 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x55c50b648bf0 0x55c50b6d3890 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.284+0000 7f162b600640 1 --2- 172.21.15.19:0/1954268983 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160c06af40 0x7f160c06d400 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f161800b310 tx=0x7f1618008000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.284+0000 7f162ca00640 1 -- 172.21.15.19:0/1954268983 <== osd.1 v2:172.21.15.19:6801/1998544260 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v80'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f162400b500 con 0x55c50b648bf0 2024-03-28T13:16:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.284+0000 7f162f644f40 1 -- 172.21.15.19:0/1954268983 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x55c50b648bf0 msgr2=0x55c50b6d3890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.284+0000 7f162f644f40 1 --2- 172.21.15.19:0/1954268983 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x55c50b648bf0 0x55c50b6d3890 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.285+0000 7f162f644f40 1 -- 172.21.15.19:0/1954268983 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160c06af40 msgr2=0x7f160c06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.285+0000 7f162f644f40 1 --2- 172.21.15.19:0/1954268983 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f160c06af40 0x7f160c06d400 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f161800b310 tx=0x7f1618008000 comp rx=0 tx=0).stop 2024-03-28T13:16:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.285+0000 7f162f644f40 1 -- 172.21.15.19:0/1954268983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c50b649010 msgr2=0x55c50b6d6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.285+0000 7f162f644f40 1 --2- 172.21.15.19:0/1954268983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c50b649010 0x55c50b6d6730 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f161c00a4f0 tx=0x7f161c002fb0 comp rx=0 tx=0).stop 2024-03-28T13:16:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.285+0000 7f162ca00640 1 -- 172.21.15.19:0/1954268983 reap_dead start 2024-03-28T13:16:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.285+0000 7f162f644f40 1 -- 172.21.15.19:0/1954268983 shutdown_connections 2024-03-28T13:16:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.285+0000 7f162f644f40 1 -- 172.21.15.19:0/1954268983 >> 172.21.15.19:0/1954268983 conn(0x55c50b62bbf0 msgr2=0x55c50b6db480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.285+0000 7f162f644f40 1 -- 172.21.15.19:0/1954268983 shutdown_connections 2024-03-28T13:16:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.285+0000 7f162f644f40 1 -- 172.21.15.19:0/1954268983 wait complete. 2024-03-28T13:16:02.293 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-28T13:16:02.293 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:16:02.414 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:16:02.523 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-28T13:16:02.523 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:16:02.523 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:16:02.524 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:16:02.580 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:16:02.580 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:16:02.628 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:16:02.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.887+0000 7fb206400640 1 Processor -- start 2024-03-28T13:16:02.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.888+0000 7fb206400640 1 -- start start 2024-03-28T13:16:02.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.889+0000 7fb206400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb20015c390 0x7fb200073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.889+0000 7fb206400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb20015fe80 con 0x7fb2000d0650 2024-03-28T13:16:02.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.889+0000 7fb206400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2000ad120 con 0x7fb2000af830 2024-03-28T13:16:02.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.889+0000 7fb206400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb20015c770 con 0x7fb20015c390 2024-03-28T13:16:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.890+0000 7fb1ffe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb2000d0650 0x7fb200071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38442/0 (socket says 172.21.15.19:38442) 2024-03-28T13:16:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.890+0000 7fb1ffe00640 1 -- 172.21.15.19:0/839623218 learned_addr learned my addr 172.21.15.19:0/839623218 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.890+0000 7fb1ff400640 1 --2- 172.21.15.19:0/839623218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb20015c390 0x7fb200073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.890+0000 7fb1fea00640 1 -- 172.21.15.19:0/839623218 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 398545532 0 0) 0x7fb20015fe80 con 0x7fb2000d0650 2024-03-28T13:16:02.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.890+0000 7fb1fea00640 1 -- 172.21.15.19:0/839623218 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1e4003520 con 0x7fb2000d0650 2024-03-28T13:16:02.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.890+0000 7fb1fea00640 1 -- 172.21.15.19:0/839623218 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1915591044 0 0) 0x7fb1e4003520 con 0x7fb2000d0650 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1fea00640 1 -- 172.21.15.19:0/839623218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb20015c390 msgr2=0x7fb200073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1fea00640 1 --2- 172.21.15.19:0/839623218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb20015c390 0x7fb200073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1fea00640 1 -- 172.21.15.19:0/839623218 >> v1:172.21.15.188:6789/0 conn(0x7fb2000af830 legacy=0x7fb200160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1fea00640 1 -- 172.21.15.19:0/839623218 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2000ad400 con 0x7fb2000d0650 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1fea00640 1 -- 172.21.15.19:0/839623218 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fb1f80019f0 con 0x7fb2000d0650 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1fea00640 1 -- 172.21.15.19:0/839623218 >> v1:172.21.15.19:6789/0 conn(0x7fb2000d0650 legacy=0x7fb200071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1fea00640 1 --2- 172.21.15.19:0/839623218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e40039e0 0x7fb1e4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1fea00640 1 --2- 172.21.15.19:0/839623218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1e4004800 0x7fb1e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1fea00640 1 --2- 172.21.15.19:0/839623218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb20015c390 0x7fb1e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1fea00640 1 -- 172.21.15.19:0/839623218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e400ba70 con 0x7fb1e40039e0 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1fea00640 1 -- 172.21.15.19:0/839623218 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e400bbf0 con 0x7fb1e4004800 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1fea00640 1 -- 172.21.15.19:0/839623218 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e400bda0 con 0x7fb20015c390 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1ff400640 1 --2- 172.21.15.19:0/839623218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e40039e0 0x7fb1e4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1ffe00640 1 --2- 172.21.15.19:0/839623218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb20015c390 0x7fb1e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/839623218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1e4004800 0x7fb1e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:02.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.891+0000 7fb1ff400640 1 -- 172.21.15.19:0/839623218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb20015c390 msgr2=0x7fb1e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.892+0000 7fb1ff400640 1 --2- 172.21.15.19:0/839623218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb20015c390 0x7fb1e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.892+0000 7fb1ff400640 1 -- 172.21.15.19:0/839623218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1e4004800 msgr2=0x7fb1e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.892+0000 7fb1ff400640 1 --2- 172.21.15.19:0/839623218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1e4004800 0x7fb1e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.892+0000 7fb1ff400640 1 -- 172.21.15.19:0/839623218 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e4003520 con 0x7fb1e40039e0 2024-03-28T13:16:02.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.892+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/839623218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb1e4004800 0x7fb1e4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.893+0000 7fb1ff400640 1 --2- 172.21.15.19:0/839623218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e40039e0 0x7fb1e4003dc0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009a90 tx=0x7fb1e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697c4b99989f1b51 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.893+0000 7fb1fea00640 1 -- 172.21.15.19:0/839623218 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0039070 con 0x7fb1e40039e0 2024-03-28T13:16:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.893+0000 7fb1fea00640 1 -- 172.21.15.19:0/839623218 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f002c650 con 0x7fb1e40039e0 2024-03-28T13:16:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.893+0000 7fb1fea00640 1 -- 172.21.15.19:0/839623218 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0009690 con 0x7fb1e40039e0 2024-03-28T13:16:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.893+0000 7fb206400640 1 -- 172.21.15.19:0/839623218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e40039e0 msgr2=0x7fb1e4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.893+0000 7fb206400640 1 --2- 172.21.15.19:0/839623218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb1e40039e0 0x7fb1e4003dc0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009a90 tx=0x7fb1e4005830 comp rx=0 tx=0).stop 2024-03-28T13:16:02.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.894+0000 7fb1ffe00640 1 -- 172.21.15.19:0/839623218 reap_dead start 2024-03-28T13:16:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.894+0000 7fb206400640 1 -- 172.21.15.19:0/839623218 shutdown_connections 2024-03-28T13:16:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.894+0000 7fb206400640 1 -- 172.21.15.19:0/839623218 >> 172.21.15.19:0/839623218 conn(0x7fb20006b990 msgr2=0x7fb20006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.894+0000 7fb206400640 1 -- 172.21.15.19:0/839623218 shutdown_connections 2024-03-28T13:16:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.894+0000 7fb206400640 1 -- 172.21.15.19:0/839623218 wait complete. 2024-03-28T13:16:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.896+0000 7fb206400640 1 Processor -- start 2024-03-28T13:16:02.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.896+0000 7fb206400640 1 -- start start 2024-03-28T13:16:02.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.897+0000 7fb206400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1e40039e0 0x7fb20014dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.897+0000 7fb206400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2000b3f60 0x7fb2000b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.897+0000 7fb206400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb20014e820 0x7fb2000b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.897+0000 7fb206400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb200160d50 con 0x7fb2000b3f60 2024-03-28T13:16:02.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.897+0000 7fb206400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb200160120 con 0x7fb20014e820 2024-03-28T13:16:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.897+0000 7fb206400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb2000cf980 con 0x7fb1e40039e0 2024-03-28T13:16:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.897+0000 7fb1f7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2000b3f60 0x7fb2000b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.897+0000 7fb1f7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2000b3f60 0x7fb2000b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42042/0 (socket says 172.21.15.19:42042) 2024-03-28T13:16:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.897+0000 7fb1f7e00640 1 -- 172.21.15.19:0/1868489815 learned_addr learned my addr 172.21.15.19:0/1868489815 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.897+0000 7fb1ff400640 1 --2- 172.21.15.19:0/1868489815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1e40039e0 0x7fb20014dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.898+0000 7fb1ffe00640 1 --2- 172.21.15.19:0/1868489815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb20014e820 0x7fb2000b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.898+0000 7fb1f7e00640 1 -- 172.21.15.19:0/1868489815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1e40039e0 msgr2=0x7fb20014dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.898+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/1868489815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1e40039e0 0x7fb20014dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.898+0000 7fb1f7e00640 1 -- 172.21.15.19:0/1868489815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb20014e820 msgr2=0x7fb2000b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.898+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/1868489815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb20014e820 0x7fb2000b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.898+0000 7fb1f7e00640 1 -- 172.21.15.19:0/1868489815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20014e5b0 con 0x7fb2000b3f60 2024-03-28T13:16:02.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.898+0000 7fb1ff400640 1 --2- 172.21.15.19:0/1868489815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1e40039e0 0x7fb20014dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:02.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.898+0000 7fb1f7e00640 1 --2- 172.21.15.19:0/1868489815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2000b3f60 0x7fb2000b0160 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800d820 tx=0x7fb1e800dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:02.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.898+0000 7fb1f7400640 1 -- 172.21.15.19:0/1868489815 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e8005480 con 0x7fb2000b3f60 2024-03-28T13:16:02.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.898+0000 7fb1f7400640 1 -- 172.21.15.19:0/1868489815 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e8017070 con 0x7fb2000b3f60 2024-03-28T13:16:02.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.898+0000 7fb1f7400640 1 -- 172.21.15.19:0/1868489815 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e800b850 con 0x7fb2000b3f60 2024-03-28T13:16:02.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.898+0000 7fb206400640 1 -- 172.21.15.19:0/1868489815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb200158490 con 0x7fb2000b3f60 2024-03-28T13:16:02.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.898+0000 7fb206400640 1 -- 172.21.15.19:0/1868489815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20015efa0 con 0x7fb2000b3f60 2024-03-28T13:16:02.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.900+0000 7fb206400640 1 -- 172.21.15.19:0/1868489815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2000685c0 con 0x7fb2000b3f60 2024-03-28T13:16:02.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.902+0000 7fb1f7400640 1 -- 172.21.15.19:0/1868489815 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb1e800b9f0 con 0x7fb2000b3f60 2024-03-28T13:16:02.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.903+0000 7fb1f7400640 1 --2- 172.21.15.19:0/1868489815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb1dc06c560 0x7fb1dc06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:02.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.903+0000 7fb1f7400640 1 -- 172.21.15.19:0/1868489815 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7fb1e808b700 con 0x7fb2000b3f60 2024-03-28T13:16:02.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.903+0000 7fb1ff400640 1 --2- 172.21.15.19:0/1868489815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb1dc06c560 0x7fb1dc06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:02.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.905+0000 7fb1ff400640 1 --2- 172.21.15.19:0/1868489815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb1dc06c560 0x7fb1dc06ea20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0036000 tx=0x7fb1f0030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:02.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:02.908+0000 7fb1f7400640 1 -- 172.21.15.19:0/1868489815 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb1e8059b60 con 0x7fb2000b3f60 2024-03-28T13:16:02.930 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:16:02.935 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:16:03.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.203+0000 7fb206400640 1 -- 172.21.15.19:0/1868489815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb20015fac0 con 0x7fb2000b3f60 2024-03-28T13:16:03.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.206+0000 7fb1f7400640 1 -- 172.21.15.19:0/1868489815 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16835 (secure 0 0 0) 0x7fb1e8059460 con 0x7fb2000b3f60 2024-03-28T13:16:03.206 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:03.206 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:15:25.200670+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:14:29.424080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-28T13:14:30.494667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1d","mappings":[{"from":2,"to":7}]},{"pgid":"11.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/1189161682":"2024-03-29T13:14:45.007899+0000","172.21.15.19:6832/1189161682":"2024-03-29T13:14:45.007899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:16:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.209+0000 7fb206400640 1 -- 172.21.15.19:0/1868489815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb1dc06c560 msgr2=0x7fb1dc06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.209+0000 7fb206400640 1 --2- 172.21.15.19:0/1868489815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb1dc06c560 0x7fb1dc06ea20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0036000 tx=0x7fb1f0030000 comp rx=0 tx=0).stop 2024-03-28T13:16:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.209+0000 7fb206400640 1 -- 172.21.15.19:0/1868489815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2000b3f60 msgr2=0x7fb2000b0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.209+0000 7fb206400640 1 --2- 172.21.15.19:0/1868489815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2000b3f60 0x7fb2000b0160 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb1e800d820 tx=0x7fb1e800dce0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.209+0000 7fb206400640 1 -- 172.21.15.19:0/1868489815 shutdown_connections 2024-03-28T13:16:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.209+0000 7fb206400640 1 --2- 172.21.15.19:0/1868489815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb1dc06c560 0x7fb1dc06ea20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.209+0000 7fb206400640 1 --2- 172.21.15.19:0/1868489815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb20014e820 0x7fb2000b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.209+0000 7fb206400640 1 --2- 172.21.15.19:0/1868489815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2000b3f60 0x7fb2000b0160 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.209+0000 7fb206400640 1 --2- 172.21.15.19:0/1868489815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb1e40039e0 0x7fb20014dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.209+0000 7fb206400640 1 -- 172.21.15.19:0/1868489815 >> 172.21.15.19:0/1868489815 conn(0x7fb20006b990 msgr2=0x7fb2000b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.210+0000 7fb206400640 1 -- 172.21.15.19:0/1868489815 shutdown_connections 2024-03-28T13:16:03.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.210+0000 7fb206400640 1 -- 172.21.15.19:0/1868489815 wait complete. 2024-03-28T13:16:03.231 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:14:29.424080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:14:30.494667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:16:03.232 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:16:03.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.452+0000 7fcf0e200640 1 Processor -- start 2024-03-28T13:16:03.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.452+0000 7fcf0e200640 1 -- start start 2024-03-28T13:16:03.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.453+0000 7fcf0e200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf0815e4c0 0x7fcf080b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:03.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.453+0000 7fcf0e200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf080d37b0 0x7fcf080b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:03.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.453+0000 7fcf0e200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf080d2710 con 0x7fcf080b67f0 2024-03-28T13:16:03.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.453+0000 7fcf0e200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcf0815e8a0 con 0x7fcf080d37b0 2024-03-28T13:16:03.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.453+0000 7fcf0e200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcf080d4040 con 0x7fcf0815e4c0 2024-03-28T13:16:03.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.454+0000 7fcf07e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf080d37b0 0x7fcf080b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:03.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.454+0000 7fcf07e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcf080d37b0 0x7fcf080b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42046/0 (socket says 172.21.15.19:42046) 2024-03-28T13:16:03.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.454+0000 7fcf07e00640 1 -- 172.21.15.19:0/1169519170 learned_addr learned my addr 172.21.15.19:0/1169519170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:03.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.454+0000 7fcf0ce00640 1 --2- 172.21.15.19:0/1169519170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf0815e4c0 0x7fcf080b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:03.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.455+0000 7fcf07e00640 1 -- 172.21.15.19:0/1169519170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf0815e4c0 msgr2=0x7fcf080b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:03.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.455+0000 7fcf07e00640 1 --2- 172.21.15.19:0/1169519170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf0815e4c0 0x7fcf080b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.455+0000 7fcf07e00640 1 -- 172.21.15.19:0/1169519170 >> v1:172.21.15.188:6789/0 conn(0x7fcf080b67f0 legacy=0x7fcf080d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:03.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.455+0000 7fcf07e00640 1 -- 172.21.15.19:0/1169519170 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf080d29f0 con 0x7fcf080d37b0 2024-03-28T13:16:03.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.456+0000 7fcf07e00640 1 --2- 172.21.15.19:0/1169519170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf080d37b0 0x7fcf080b62b0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009870 tx=0x7fcef402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd31e71ab9b374a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:03.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.456+0000 7fcf07400640 1 -- 172.21.15.19:0/1169519170 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4034070 con 0x7fcf080d37b0 2024-03-28T13:16:03.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.457+0000 7fcf07400640 1 -- 172.21.15.19:0/1169519170 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcef4030840 con 0x7fcf080d37b0 2024-03-28T13:16:03.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.457+0000 7fcf07400640 1 -- 172.21.15.19:0/1169519170 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4030c10 con 0x7fcf080d37b0 2024-03-28T13:16:03.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.457+0000 7fcf0e200640 1 -- 172.21.15.19:0/1169519170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf080d37b0 msgr2=0x7fcf080b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:03.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.457+0000 7fcf0e200640 1 --2- 172.21.15.19:0/1169519170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf080d37b0 0x7fcf080b62b0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009870 tx=0x7fcef402b980 comp rx=0 tx=0).stop 2024-03-28T13:16:03.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.457+0000 7fcf0e200640 1 -- 172.21.15.19:0/1169519170 shutdown_connections 2024-03-28T13:16:03.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.457+0000 7fcf0e200640 1 --2- 172.21.15.19:0/1169519170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf080d37b0 0x7fcf080b62b0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.457+0000 7fcf0e200640 1 --2- 172.21.15.19:0/1169519170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf0815e4c0 0x7fcf080b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.457+0000 7fcf0e200640 1 -- 172.21.15.19:0/1169519170 >> 172.21.15.19:0/1169519170 conn(0x7fcf080676a0 msgr2=0x7fcf08067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:03.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.457+0000 7fcf0e200640 1 -- 172.21.15.19:0/1169519170 shutdown_connections 2024-03-28T13:16:03.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.457+0000 7fcf0e200640 1 -- 172.21.15.19:0/1169519170 wait complete. 2024-03-28T13:16:03.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.459+0000 7fcf0e200640 1 Processor -- start 2024-03-28T13:16:03.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.459+0000 7fcf0e200640 1 -- start start 2024-03-28T13:16:03.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.460+0000 7fcf0e200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf080b67f0 0x7fcf08157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:03.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.460+0000 7fcf0e200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf080d37b0 0x7fcf081f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:03.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.460+0000 7fcf0e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf0815e4c0 0x7fcf081f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:03.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.460+0000 7fcf0e200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcf080d2fb0 con 0x7fcf0815e4c0 2024-03-28T13:16:03.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.460+0000 7fcf0e200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf080d3320 con 0x7fcf080b67f0 2024-03-28T13:16:03.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.460+0000 7fcf0e200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcf080ba130 con 0x7fcf080d37b0 2024-03-28T13:16:03.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.460+0000 7fcf0d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf0815e4c0 0x7fcf081f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:03.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.461+0000 7fcf0d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf0815e4c0 0x7fcf081f7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42050/0 (socket says 172.21.15.19:42050) 2024-03-28T13:16:03.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.461+0000 7fcf0d800640 1 -- 172.21.15.19:0/1809752704 learned_addr learned my addr 172.21.15.19:0/1809752704 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:03.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.461+0000 7fcf0ce00640 1 --2- 172.21.15.19:0/1809752704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf080b67f0 0x7fcf08157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:03.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.461+0000 7fcf07e00640 1 --2- 172.21.15.19:0/1809752704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf080d37b0 0x7fcf081f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:03.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.461+0000 7fcf0d800640 1 -- 172.21.15.19:0/1809752704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf080d37b0 msgr2=0x7fcf081f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:03.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.461+0000 7fcf0d800640 1 --2- 172.21.15.19:0/1809752704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf080d37b0 0x7fcf081f7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.461+0000 7fcf0d800640 1 -- 172.21.15.19:0/1809752704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf080b67f0 msgr2=0x7fcf08157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:03.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.461+0000 7fcf0d800640 1 --2- 172.21.15.19:0/1809752704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf080b67f0 0x7fcf08157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.461+0000 7fcf0d800640 1 -- 172.21.15.19:0/1809752704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf081f8860 con 0x7fcf0815e4c0 2024-03-28T13:16:03.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.461+0000 7fcf0ce00640 1 --2- 172.21.15.19:0/1809752704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf080b67f0 0x7fcf08157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:16:03.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.462+0000 7fcf0d800640 1 --2- 172.21.15.19:0/1809752704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf0815e4c0 0x7fcf081f7e90 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000c7a0 tx=0x7fcf0000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:03.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.462+0000 7fcf04c00640 1 -- 172.21.15.19:0/1809752704 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf00003b70 con 0x7fcf0815e4c0 2024-03-28T13:16:03.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.462+0000 7fcf04c00640 1 -- 172.21.15.19:0/1809752704 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf00003d30 con 0x7fcf0815e4c0 2024-03-28T13:16:03.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.462+0000 7fcf0e200640 1 -- 172.21.15.19:0/1809752704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf080cdb80 con 0x7fcf0815e4c0 2024-03-28T13:16:03.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.462+0000 7fcf04c00640 1 -- 172.21.15.19:0/1809752704 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000aa90 con 0x7fcf0815e4c0 2024-03-28T13:16:03.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.462+0000 7fcf0e200640 1 -- 172.21.15.19:0/1809752704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf080ce140 con 0x7fcf0815e4c0 2024-03-28T13:16:03.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.463+0000 7fcf0e200640 1 -- 172.21.15.19:0/1809752704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf08075940 con 0x7fcf0815e4c0 2024-03-28T13:16:03.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.464+0000 7fcf04c00640 1 -- 172.21.15.19:0/1809752704 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcf0000ac30 con 0x7fcf0815e4c0 2024-03-28T13:16:03.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.465+0000 7fcf04c00640 1 --2- 172.21.15.19:0/1809752704 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcee806c530 0x7fcee806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:03.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.465+0000 7fcf04c00640 1 -- 172.21.15.19:0/1809752704 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7fcf0008b7b0 con 0x7fcf0815e4c0 2024-03-28T13:16:03.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.466+0000 7fcf0ce00640 1 --2- 172.21.15.19:0/1809752704 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcee806c530 0x7fcee806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:03.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.467+0000 7fcf0ce00640 1 --2- 172.21.15.19:0/1809752704 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcee806c530 0x7fcee806e9f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcefc006fd0 tx=0x7fcefc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:03.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.470+0000 7fcf04c00640 1 -- 172.21.15.19:0/1809752704 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcf00059ca0 con 0x7fcf0815e4c0 2024-03-28T13:16:03.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.673+0000 7fcf0e200640 1 -- 172.21.15.19:0/1809752704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf080ce310 con 0x7fcf0815e4c0 2024-03-28T13:16:03.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.673+0000 7fcf04c00640 1 -- 172.21.15.19:0/1809752704 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7fcf000595a0 con 0x7fcf0815e4c0 2024-03-28T13:16:03.673 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:16:03.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.676+0000 7fcf0e200640 1 -- 172.21.15.19:0/1809752704 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcee806c530 msgr2=0x7fcee806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:03.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.676+0000 7fcf0e200640 1 --2- 172.21.15.19:0/1809752704 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcee806c530 0x7fcee806e9f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcefc006fd0 tx=0x7fcefc008040 comp rx=0 tx=0).stop 2024-03-28T13:16:03.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.676+0000 7fcf0e200640 1 -- 172.21.15.19:0/1809752704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf0815e4c0 msgr2=0x7fcf081f7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:03.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.676+0000 7fcf0e200640 1 --2- 172.21.15.19:0/1809752704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf0815e4c0 0x7fcf081f7e90 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000c7a0 tx=0x7fcf0000cc60 comp rx=0 tx=0).stop 2024-03-28T13:16:03.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.676+0000 7fcf0e200640 1 -- 172.21.15.19:0/1809752704 shutdown_connections 2024-03-28T13:16:03.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.676+0000 7fcf0e200640 1 --2- 172.21.15.19:0/1809752704 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcee806c530 0x7fcee806e9f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.676+0000 7fcf0e200640 1 --2- 172.21.15.19:0/1809752704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf0815e4c0 0x7fcf081f7e90 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.676+0000 7fcf0e200640 1 --2- 172.21.15.19:0/1809752704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf080d37b0 0x7fcf081f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.676+0000 7fcf0e200640 1 --2- 172.21.15.19:0/1809752704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf080b67f0 0x7fcf08157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.676+0000 7fcf0e200640 1 -- 172.21.15.19:0/1809752704 >> 172.21.15.19:0/1809752704 conn(0x7fcf080676a0 msgr2=0x7fcf0815ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:03.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.676+0000 7fcf0e200640 1 -- 172.21.15.19:0/1809752704 shutdown_connections 2024-03-28T13:16:03.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.676+0000 7fcf0e200640 1 -- 172.21.15.19:0/1809752704 wait complete. 2024-03-28T13:16:03.693 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:16:03.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.901+0000 7f1075e00640 1 Processor -- start 2024-03-28T13:16:03.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.901+0000 7f1075e00640 1 -- start start 2024-03-28T13:16:03.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.903+0000 7f1075e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f107013f190 0x7f10700cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:03.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.903+0000 7f1075e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10700ce890 con 0x7f107006dd30 2024-03-28T13:16:03.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.903+0000 7f1075e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10700ceb50 con 0x7f10700cc880 2024-03-28T13:16:03.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.903+0000 7f1075e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f10700cee30 con 0x7f107013f190 2024-03-28T13:16:03.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.903+0000 7f1074a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f107013f190 0x7f10700cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:03.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.903+0000 7f1075400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f107006dd30 0x7f107015ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38450/0 (socket says 172.21.15.19:38450) 2024-03-28T13:16:03.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.903+0000 7f1074a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f107013f190 0x7f10700cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42060/0 (socket says 172.21.15.19:42060) 2024-03-28T13:16:03.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.903+0000 7f1075400640 1 -- 172.21.15.19:0/3685392507 learned_addr learned my addr 172.21.15.19:0/3685392507 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:03.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.904+0000 7f1074a00640 1 -- 172.21.15.19:0/3685392507 >> v1:172.21.15.188:6789/0 conn(0x7f10700cc880 legacy=0x7f107006d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:03.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.904+0000 7f1074a00640 1 -- 172.21.15.19:0/3685392507 >> v1:172.21.15.19:6789/0 conn(0x7f107006dd30 legacy=0x7f107015ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:03.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.904+0000 7f106f400640 1 -- 172.21.15.19:0/3685392507 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3379676873 0 0) 0x7f10700ce890 con 0x7f107006dd30 2024-03-28T13:16:03.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.904+0000 7f1074a00640 1 -- 172.21.15.19:0/3685392507 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10700cf110 con 0x7f107013f190 2024-03-28T13:16:03.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.905+0000 7f1074a00640 1 --2- 172.21.15.19:0/3685392507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f107013f190 0x7f10700cf630 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f10640097a0 tx=0x7f106402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfbc41ec0255e07 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.905+0000 7f106f400640 1 -- 172.21.15.19:0/3685392507 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064035020 con 0x7f107013f190 2024-03-28T13:16:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.905+0000 7f106f400640 1 -- 172.21.15.19:0/3685392507 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1064031830 con 0x7f107013f190 2024-03-28T13:16:03.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.905+0000 7f106f400640 1 -- 172.21.15.19:0/3685392507 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1064031c00 con 0x7f107013f190 2024-03-28T13:16:03.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.906+0000 7f1075e00640 1 -- 172.21.15.19:0/3685392507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f107013f190 msgr2=0x7f10700cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:03.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.906+0000 7f1075e00640 1 --2- 172.21.15.19:0/3685392507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f107013f190 0x7f10700cf630 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f10640097a0 tx=0x7f106402b8d0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.906+0000 7f1075e00640 1 -- 172.21.15.19:0/3685392507 shutdown_connections 2024-03-28T13:16:03.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.906+0000 7f1075e00640 1 --2- 172.21.15.19:0/3685392507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f107013f190 0x7f10700cf630 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.906+0000 7f1075e00640 1 -- 172.21.15.19:0/3685392507 >> 172.21.15.19:0/3685392507 conn(0x7f10700b1ba0 msgr2=0x7f10700b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:03.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.906+0000 7f1075e00640 1 -- 172.21.15.19:0/3685392507 shutdown_connections 2024-03-28T13:16:03.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.906+0000 7f1075e00640 1 -- 172.21.15.19:0/3685392507 wait complete. 2024-03-28T13:16:03.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.908+0000 7f1075e00640 1 Processor -- start 2024-03-28T13:16:03.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.908+0000 7f1075e00640 1 -- start start 2024-03-28T13:16:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.909+0000 7f1075e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f107006dd30 0x7f1070156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.909+0000 7f1075e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10700cc880 0x7f1070138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.909+0000 7f1075e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107013f190 0x7f107013df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.909+0000 7f1075e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10700cfb70 con 0x7f107006dd30 2024-03-28T13:16:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.909+0000 7f1075e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f107013f570 con 0x7f107013f190 2024-03-28T13:16:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.909+0000 7f1075e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1070160620 con 0x7f10700cc880 2024-03-28T13:16:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.909+0000 7f1074a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f107006dd30 0x7f1070156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.909+0000 7f1074a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f107006dd30 0x7f1070156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42064/0 (socket says 172.21.15.19:42064) 2024-03-28T13:16:03.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.909+0000 7f1074a00640 1 -- 172.21.15.19:0/590895585 learned_addr learned my addr 172.21.15.19:0/590895585 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.910+0000 7f1075400640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107013f190 0x7f107013df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.910+0000 7f1074a00640 1 -- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10700cc880 msgr2=0x7f1070138cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.910+0000 7f106fe00640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10700cc880 0x7f1070138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.910+0000 7f1074a00640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10700cc880 0x7f1070138cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.910+0000 7f1074a00640 1 -- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107013f190 msgr2=0x7f107013df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.910+0000 7f1074a00640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107013f190 0x7f107013df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.910+0000 7f1074a00640 1 -- 172.21.15.19:0/590895585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1070153920 con 0x7f107006dd30 2024-03-28T13:16:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.910+0000 7f106fe00640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10700cc880 0x7f1070138cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.910+0000 7f1075400640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107013f190 0x7f107013df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.910+0000 7f1074a00640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f107006dd30 0x7f1070156e50 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f10640098c0 tx=0x7f10640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.911+0000 7f106cc00640 1 -- 172.21.15.19:0/590895585 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f106403ba80 con 0x7f107006dd30 2024-03-28T13:16:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.911+0000 7f106cc00640 1 -- 172.21.15.19:0/590895585 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f106403bc20 con 0x7f107006dd30 2024-03-28T13:16:03.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.911+0000 7f106cc00640 1 -- 172.21.15.19:0/590895585 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10640453f0 con 0x7f107006dd30 2024-03-28T13:16:03.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.911+0000 7f1075e00640 1 -- 172.21.15.19:0/590895585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1070153c00 con 0x7f107006dd30 2024-03-28T13:16:03.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.911+0000 7f1075e00640 1 -- 172.21.15.19:0/590895585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10701541c0 con 0x7f107006dd30 2024-03-28T13:16:03.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.912+0000 7f1075e00640 1 -- 172.21.15.19:0/590895585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1070075940 con 0x7f107006dd30 2024-03-28T13:16:03.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.913+0000 7f106cc00640 1 -- 172.21.15.19:0/590895585 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1064038020 con 0x7f107006dd30 2024-03-28T13:16:03.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.914+0000 7f106cc00640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f105006c530 0x7f105006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:03.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.914+0000 7f106cc00640 1 -- 172.21.15.19:0/590895585 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f10640ad5d0 con 0x7f107006dd30 2024-03-28T13:16:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.914+0000 7f106fe00640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f105006c530 0x7f105006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.915+0000 7f106fe00640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f105006c530 0x7f105006e9f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f10700b2d90 tx=0x7f105c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:03.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:03.918+0000 7f106cc00640 1 -- 172.21.15.19:0/590895585 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1064040070 con 0x7f107006dd30 2024-03-28T13:16:04.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.120+0000 7f1075e00640 1 -- 172.21.15.19:0/590895585 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f10700b2870 con 0x7f107006dd30 2024-03-28T13:16:04.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.121+0000 7f106cc00640 1 -- 172.21.15.19:0/590895585 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7f106407bb50 con 0x7f107006dd30 2024-03-28T13:16:04.121 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:16:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.124+0000 7f1075e00640 1 -- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f105006c530 msgr2=0x7f105006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.124+0000 7f1075e00640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f105006c530 0x7f105006e9f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f10700b2d90 tx=0x7f105c009040 comp rx=0 tx=0).stop 2024-03-28T13:16:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.124+0000 7f1075e00640 1 -- 172.21.15.19:0/590895585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f107006dd30 msgr2=0x7f1070156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.124+0000 7f1075e00640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f107006dd30 0x7f1070156e50 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f10640098c0 tx=0x7f10640046c0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.124+0000 7f1075e00640 1 -- 172.21.15.19:0/590895585 shutdown_connections 2024-03-28T13:16:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.124+0000 7f1075e00640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f105006c530 0x7f105006e9f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.124+0000 7f1075e00640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f107013f190 0x7f107013df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.124+0000 7f1075e00640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10700cc880 0x7f1070138cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.124+0000 7f1075e00640 1 --2- 172.21.15.19:0/590895585 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f107006dd30 0x7f1070156e50 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.124+0000 7f1075e00640 1 -- 172.21.15.19:0/590895585 >> 172.21.15.19:0/590895585 conn(0x7f10700b1ba0 msgr2=0x7f107015ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:04.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.124+0000 7f1075e00640 1 -- 172.21.15.19:0/590895585 shutdown_connections 2024-03-28T13:16:04.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.124+0000 7f1075e00640 1 -- 172.21.15.19:0/590895585 wait complete. 2024-03-28T13:16:04.143 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:16:04.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.369+0000 7feea8800640 1 Processor -- start 2024-03-28T13:16:04.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.370+0000 7feea8800640 1 -- start start 2024-03-28T13:16:04.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.371+0000 7feea8800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feea015e6d0 0x7feea00d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:04.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.371+0000 7feea8800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feea00d4760 0x7feea00cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:04.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.371+0000 7feea8800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feea00ceef0 con 0x7feea00d0cb0 2024-03-28T13:16:04.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.371+0000 7feea8800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7feea015eab0 con 0x7feea00d4760 2024-03-28T13:16:04.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.371+0000 7feea8800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feea00d1220 con 0x7feea015e6d0 2024-03-28T13:16:04.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.372+0000 7feea6800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feea00d4760 0x7feea00cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:04.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.372+0000 7feea6800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feea00d4760 0x7feea00cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42080/0 (socket says 172.21.15.19:42080) 2024-03-28T13:16:04.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.372+0000 7feea6800640 1 -- 172.21.15.19:0/2239133611 learned_addr learned my addr 172.21.15.19:0/2239133611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:04.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.372+0000 7feea5e00640 1 --2- 172.21.15.19:0/2239133611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feea015e6d0 0x7feea00d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:04.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.372+0000 7feea6800640 1 -- 172.21.15.19:0/2239133611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feea015e6d0 msgr2=0x7feea00d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:04.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.372+0000 7feea6800640 1 --2- 172.21.15.19:0/2239133611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feea015e6d0 0x7feea00d21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.372+0000 7feea6800640 1 -- 172.21.15.19:0/2239133611 >> v1:172.21.15.188:6789/0 conn(0x7feea00d0cb0 legacy=0x7feea00d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.372+0000 7feea6800640 1 -- 172.21.15.19:0/2239133611 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea00d1500 con 0x7feea00d4760 2024-03-28T13:16:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.373+0000 7feea6800640 1 --2- 172.21.15.19:0/2239133611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea00d4760 0x7feea00cf3d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009e50 tx=0x7fee9c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e10326dfa7f4b79 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.374+0000 7feea4a00640 1 -- 172.21.15.19:0/2239133611 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c02ca70 con 0x7feea00d4760 2024-03-28T13:16:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.374+0000 7feea4a00640 1 -- 172.21.15.19:0/2239133611 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee9c02cc10 con 0x7feea00d4760 2024-03-28T13:16:04.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.374+0000 7feea4a00640 1 -- 172.21.15.19:0/2239133611 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c0343f0 con 0x7feea00d4760 2024-03-28T13:16:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.374+0000 7feea8800640 1 -- 172.21.15.19:0/2239133611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea00d4760 msgr2=0x7feea00cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.374+0000 7feea8800640 1 --2- 172.21.15.19:0/2239133611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea00d4760 0x7feea00cf3d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009e50 tx=0x7fee9c02bed0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.375+0000 7feea8800640 1 -- 172.21.15.19:0/2239133611 shutdown_connections 2024-03-28T13:16:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.375+0000 7feea8800640 1 --2- 172.21.15.19:0/2239133611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea00d4760 0x7feea00cf3d0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.375+0000 7feea8800640 1 --2- 172.21.15.19:0/2239133611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feea015e6d0 0x7feea00d21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.375+0000 7feea8800640 1 -- 172.21.15.19:0/2239133611 >> 172.21.15.19:0/2239133611 conn(0x7feea009d7b0 msgr2=0x7feea008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.375+0000 7feea8800640 1 -- 172.21.15.19:0/2239133611 shutdown_connections 2024-03-28T13:16:04.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.375+0000 7feea8800640 1 -- 172.21.15.19:0/2239133611 wait complete. 2024-03-28T13:16:04.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.377+0000 7feea8800640 1 Processor -- start 2024-03-28T13:16:04.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.378+0000 7feea8800640 1 -- start start 2024-03-28T13:16:04.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.379+0000 7feea8800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feea00d0cb0 0x7feea009ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:04.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.379+0000 7feea8800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feea00d4760 0x7feea015d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:04.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.379+0000 7feea8800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea015e6d0 0x7feea00b8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:04.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.379+0000 7feea8800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feea00d2720 con 0x7feea015e6d0 2024-03-28T13:16:04.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.379+0000 7feea8800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feea00d1ac0 con 0x7feea00d4760 2024-03-28T13:16:04.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.379+0000 7feea8800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feea006c890 con 0x7feea00d0cb0 2024-03-28T13:16:04.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.380+0000 7feea6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea015e6d0 0x7feea00b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:04.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.380+0000 7feea6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea015e6d0 0x7feea00b8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42096/0 (socket says 172.21.15.19:42096) 2024-03-28T13:16:04.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.380+0000 7feea6800640 1 -- 172.21.15.19:0/2145140070 learned_addr learned my addr 172.21.15.19:0/2145140070 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:04.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.380+0000 7feea5e00640 1 --2- 172.21.15.19:0/2145140070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feea00d0cb0 0x7feea009ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:04.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.380+0000 7feea5400640 1 --2- 172.21.15.19:0/2145140070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feea00d4760 0x7feea015d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:04.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.380+0000 7feea6800640 1 -- 172.21.15.19:0/2145140070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feea00d0cb0 msgr2=0x7feea009ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:04.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.380+0000 7feea6800640 1 --2- 172.21.15.19:0/2145140070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feea00d0cb0 0x7feea009ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.380+0000 7feea6800640 1 -- 172.21.15.19:0/2145140070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feea00d4760 msgr2=0x7feea015d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:04.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.380+0000 7feea6800640 1 --2- 172.21.15.19:0/2145140070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feea00d4760 0x7feea015d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.380+0000 7feea6800640 1 -- 172.21.15.19:0/2145140070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea01f40f0 con 0x7feea015e6d0 2024-03-28T13:16:04.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.380+0000 7feea5400640 1 --2- 172.21.15.19:0/2145140070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feea00d4760 0x7feea015d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:04.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.381+0000 7feea6800640 1 --2- 172.21.15.19:0/2145140070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea015e6d0 0x7feea00b8980 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009e50 tx=0x7fee9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:04.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.381+0000 7fee9a000640 1 -- 172.21.15.19:0/2145140070 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c043020 con 0x7feea015e6d0 2024-03-28T13:16:04.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.381+0000 7fee9a000640 1 -- 172.21.15.19:0/2145140070 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee9c03e450 con 0x7feea015e6d0 2024-03-28T13:16:04.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.381+0000 7fee9a000640 1 -- 172.21.15.19:0/2145140070 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c009510 con 0x7feea015e6d0 2024-03-28T13:16:04.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.381+0000 7feea8800640 1 -- 172.21.15.19:0/2145140070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea00b8ec0 con 0x7feea015e6d0 2024-03-28T13:16:04.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.381+0000 7feea8800640 1 -- 172.21.15.19:0/2145140070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea009f840 con 0x7feea015e6d0 2024-03-28T13:16:04.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.382+0000 7feea8800640 1 -- 172.21.15.19:0/2145140070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea00685c0 con 0x7feea015e6d0 2024-03-28T13:16:04.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.383+0000 7fee9a000640 1 -- 172.21.15.19:0/2145140070 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fee9c046020 con 0x7feea015e6d0 2024-03-28T13:16:04.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.384+0000 7fee9a000640 1 --2- 172.21.15.19:0/2145140070 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fee8406c530 0x7fee8406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:04.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.384+0000 7fee9a000640 1 -- 172.21.15.19:0/2145140070 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7fee9c0ae050 con 0x7feea015e6d0 2024-03-28T13:16:04.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.385+0000 7feea5e00640 1 --2- 172.21.15.19:0/2145140070 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fee8406c530 0x7fee8406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:04.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.386+0000 7feea5e00640 1 --2- 172.21.15.19:0/2145140070 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fee8406c530 0x7fee8406e9f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fee94006fd0 tx=0x7fee94008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:04.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.389+0000 7fee9a000640 1 -- 172.21.15.19:0/2145140070 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fee9c04d030 con 0x7feea015e6d0 2024-03-28T13:16:04.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.590+0000 7feea8800640 1 -- 172.21.15.19:0/2145140070 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feea006c670 con 0x7feea015e6d0 2024-03-28T13:16:04.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.591+0000 7fee9a000640 1 -- 172.21.15.19:0/2145140070 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7fee9c07c540 con 0x7feea015e6d0 2024-03-28T13:16:04.591 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:16:04.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.593+0000 7feea8800640 1 -- 172.21.15.19:0/2145140070 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fee8406c530 msgr2=0x7fee8406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.593+0000 7feea8800640 1 --2- 172.21.15.19:0/2145140070 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fee8406c530 0x7fee8406e9f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fee94006fd0 tx=0x7fee94008040 comp rx=0 tx=0).stop 2024-03-28T13:16:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.593+0000 7feea8800640 1 -- 172.21.15.19:0/2145140070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea015e6d0 msgr2=0x7feea00b8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.593+0000 7feea8800640 1 --2- 172.21.15.19:0/2145140070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea015e6d0 0x7feea00b8980 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fee9c009e50 tx=0x7fee9c004a40 comp rx=0 tx=0).stop 2024-03-28T13:16:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.593+0000 7feea8800640 1 -- 172.21.15.19:0/2145140070 shutdown_connections 2024-03-28T13:16:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.593+0000 7feea8800640 1 --2- 172.21.15.19:0/2145140070 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fee8406c530 0x7fee8406e9f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.594+0000 7feea8800640 1 --2- 172.21.15.19:0/2145140070 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feea015e6d0 0x7feea00b8980 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.594+0000 7feea8800640 1 --2- 172.21.15.19:0/2145140070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feea00d4760 0x7feea015d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.594+0000 7feea8800640 1 --2- 172.21.15.19:0/2145140070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feea00d0cb0 0x7feea009ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.594+0000 7feea8800640 1 -- 172.21.15.19:0/2145140070 >> 172.21.15.19:0/2145140070 conn(0x7feea009d7b0 msgr2=0x7feea006dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.594+0000 7feea8800640 1 -- 172.21.15.19:0/2145140070 shutdown_connections 2024-03-28T13:16:04.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.594+0000 7feea8800640 1 -- 172.21.15.19:0/2145140070 wait complete. 2024-03-28T13:16:04.611 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:16:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.835+0000 7ffbaac00640 1 Processor -- start 2024-03-28T13:16:04.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.835+0000 7ffbaac00640 1 -- start start 2024-03-28T13:16:04.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.837+0000 7ffbaac00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffba415a530 0x7ffba40b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:04.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.837+0000 7ffbaac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba406e2f0 0x7ffba40b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:04.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.837+0000 7ffbaac00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba406e780 con 0x7ffba415a910 2024-03-28T13:16:04.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.837+0000 7ffbaac00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ffba406ea60 con 0x7ffba415a530 2024-03-28T13:16:04.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.837+0000 7ffbaac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffba406ed40 con 0x7ffba406e2f0 2024-03-28T13:16:04.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.837+0000 7ffbaa200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ffba415a910 0x7ffba415e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38456/0 (socket says 172.21.15.19:38456) 2024-03-28T13:16:04.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.837+0000 7ffbaa200640 1 -- 172.21.15.19:0/2449534474 learned_addr learned my addr 172.21.15.19:0/2449534474 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:04.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.838+0000 7ffba8e00640 1 --2- 172.21.15.19:0/2449534474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba406e2f0 0x7ffba40b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:04.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.838+0000 7ffba9800640 1 --2- 172.21.15.19:0/2449534474 >> v2:172.21.15.188:3300/0 conn(0x7ffba415a530 0x7ffba40b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.838+0000 7ffba3e00640 1 -- 172.21.15.19:0/2449534474 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1459273401 0 0) 0x7ffba406e780 con 0x7ffba415a910 2024-03-28T13:16:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.838+0000 7ffba3e00640 1 -- 172.21.15.19:0/2449534474 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb8c003520 con 0x7ffba415a910 2024-03-28T13:16:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.838+0000 7ffba3e00640 1 -- 172.21.15.19:0/2449534474 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4166157565 0 0) 0x7ffb8c003520 con 0x7ffba415a910 2024-03-28T13:16:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.838+0000 7ffba3e00640 1 -- 172.21.15.19:0/2449534474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba406e2f0 msgr2=0x7ffba40b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.838+0000 7ffba3e00640 1 --2- 172.21.15.19:0/2449534474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba406e2f0 0x7ffba40b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.838+0000 7ffba3e00640 1 -- 172.21.15.19:0/2449534474 >> v2:172.21.15.188:3300/0 conn(0x7ffba415a530 msgr2=0x7ffba40b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.838+0000 7ffba3e00640 1 --2- 172.21.15.19:0/2449534474 >> v2:172.21.15.188:3300/0 conn(0x7ffba415a530 0x7ffba40b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.838+0000 7ffba3e00640 1 -- 172.21.15.19:0/2449534474 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba406f020 con 0x7ffba415a910 2024-03-28T13:16:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.838+0000 7ffba3e00640 1 -- 172.21.15.19:0/2449534474 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7ffb9c0019f0 con 0x7ffba415a910 2024-03-28T13:16:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.838+0000 7ffba3e00640 1 -- 172.21.15.19:0/2449534474 >> v1:172.21.15.19:6789/0 conn(0x7ffba415a910 legacy=0x7ffba415e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:04.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.839+0000 7ffba3e00640 1 --2- 172.21.15.19:0/2449534474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb8c0039b0 0x7ffb8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.839+0000 7ffba3e00640 1 --2- 172.21.15.19:0/2449534474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb8c004800 0x7ffb8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.839+0000 7ffba3e00640 1 --2- 172.21.15.19:0/2449534474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb8c0051c0 0x7ffb8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.839+0000 7ffba3e00640 1 -- 172.21.15.19:0/2449534474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffba406ea60 con 0x7ffb8c004800 2024-03-28T13:16:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.839+0000 7ffba3e00640 1 -- 172.21.15.19:0/2449534474 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffba406ed40 con 0x7ffb8c0039b0 2024-03-28T13:16:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.839+0000 7ffba3e00640 1 -- 172.21.15.19:0/2449534474 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8c0045a0 con 0x7ffb8c0051c0 2024-03-28T13:16:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.839+0000 7ffba8e00640 1 --2- 172.21.15.19:0/2449534474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb8c004800 0x7ffb8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.839+0000 7ffba8e00640 1 -- 172.21.15.19:0/2449534474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb8c0051c0 msgr2=0x7ffb8c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.839+0000 7ffba8e00640 1 --2- 172.21.15.19:0/2449534474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffb8c0051c0 0x7ffb8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.839+0000 7ffba8e00640 1 -- 172.21.15.19:0/2449534474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb8c0039b0 msgr2=0x7ffb8c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.839+0000 7ffba8e00640 1 --2- 172.21.15.19:0/2449534474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb8c0039b0 0x7ffb8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.839+0000 7ffba8e00640 1 -- 172.21.15.19:0/2449534474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c003520 con 0x7ffb8c004800 2024-03-28T13:16:04.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.840+0000 7ffba8e00640 1 --2- 172.21.15.19:0/2449534474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb8c004800 0x7ffb8c004c80 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009af0 tx=0x7ffb8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2081bab1c83443c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:04.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.841+0000 7ffba3e00640 1 -- 172.21.15.19:0/2449534474 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90039070 con 0x7ffb8c004800 2024-03-28T13:16:04.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.841+0000 7ffba3e00640 1 -- 172.21.15.19:0/2449534474 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9002c710 con 0x7ffb8c004800 2024-03-28T13:16:04.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.841+0000 7ffba3e00640 1 -- 172.21.15.19:0/2449534474 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90009690 con 0x7ffb8c004800 2024-03-28T13:16:04.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.841+0000 7ffbaac00640 1 -- 172.21.15.19:0/2449534474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb8c004800 msgr2=0x7ffb8c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:04.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.841+0000 7ffbaac00640 1 --2- 172.21.15.19:0/2449534474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb8c004800 0x7ffb8c004c80 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009af0 tx=0x7ffb8c005830 comp rx=0 tx=0).stop 2024-03-28T13:16:04.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.841+0000 7ffbaa200640 1 -- 172.21.15.19:0/2449534474 reap_dead start 2024-03-28T13:16:04.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.841+0000 7ffbaac00640 1 -- 172.21.15.19:0/2449534474 shutdown_connections 2024-03-28T13:16:04.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.841+0000 7ffbaac00640 1 -- 172.21.15.19:0/2449534474 >> 172.21.15.19:0/2449534474 conn(0x7ffba40d0750 msgr2=0x7ffba40d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:04.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.842+0000 7ffbaac00640 1 -- 172.21.15.19:0/2449534474 shutdown_connections 2024-03-28T13:16:04.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.842+0000 7ffbaac00640 1 -- 172.21.15.19:0/2449534474 wait complete. 2024-03-28T13:16:04.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.845+0000 7ffbaac00640 1 Processor -- start 2024-03-28T13:16:04.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.845+0000 7ffbaac00640 1 -- start start 2024-03-28T13:16:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.846+0000 7ffbaac00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb8c004800 0x7ffba414ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.846+0000 7ffbaac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba413f880 0x7ffba414c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.846+0000 7ffbaac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba414c680 0x7ffba414ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.846+0000 7ffbaac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffba40b3e70 con 0x7ffba414c680 2024-03-28T13:16:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.846+0000 7ffbaac00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffba40b3cf0 con 0x7ffb8c004800 2024-03-28T13:16:04.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.846+0000 7ffbaac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffba406c780 con 0x7ffba413f880 2024-03-28T13:16:04.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.847+0000 7ffbaa200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba414c680 0x7ffba414ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:04.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.847+0000 7ffbaa200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba414c680 0x7ffba414ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42110/0 (socket says 172.21.15.19:42110) 2024-03-28T13:16:04.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.847+0000 7ffbaa200640 1 -- 172.21.15.19:0/2502621589 learned_addr learned my addr 172.21.15.19:0/2502621589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:04.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.847+0000 7ffba8e00640 1 --2- 172.21.15.19:0/2502621589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba413f880 0x7ffba414c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:04.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.847+0000 7ffba9800640 1 --2- 172.21.15.19:0/2502621589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb8c004800 0x7ffba414ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:04.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.847+0000 7ffbaa200640 1 -- 172.21.15.19:0/2502621589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba413f880 msgr2=0x7ffba414c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:04.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.847+0000 7ffbaa200640 1 --2- 172.21.15.19:0/2502621589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba413f880 0x7ffba414c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.847+0000 7ffbaa200640 1 -- 172.21.15.19:0/2502621589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb8c004800 msgr2=0x7ffba414ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:04.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.847+0000 7ffbaa200640 1 --2- 172.21.15.19:0/2502621589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb8c004800 0x7ffba414ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:04.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.847+0000 7ffbaa200640 1 -- 172.21.15.19:0/2502621589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba414d9f0 con 0x7ffba414c680 2024-03-28T13:16:04.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.847+0000 7ffba9800640 1 --2- 172.21.15.19:0/2502621589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb8c004800 0x7ffba414ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:04.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.848+0000 7ffbaa200640 1 --2- 172.21.15.19:0/2502621589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba414c680 0x7ffba414ce10 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ffba40ccbe0 tx=0x7ffb9c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:04.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.848+0000 7ffba1600640 1 -- 172.21.15.19:0/2502621589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c007310 con 0x7ffba414c680 2024-03-28T13:16:04.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.848+0000 7ffba1600640 1 -- 172.21.15.19:0/2502621589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c0074b0 con 0x7ffba414c680 2024-03-28T13:16:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.848+0000 7ffba1600640 1 -- 172.21.15.19:0/2502621589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c007880 con 0x7ffba414c680 2024-03-28T13:16:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.848+0000 7ffbaac00640 1 -- 172.21.15.19:0/2502621589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba414dcd0 con 0x7ffba414c680 2024-03-28T13:16:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.848+0000 7ffbaac00640 1 -- 172.21.15.19:0/2502621589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba414e290 con 0x7ffba414c680 2024-03-28T13:16:04.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.849+0000 7ffbaac00640 1 -- 172.21.15.19:0/2502621589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba4075940 con 0x7ffba414c680 2024-03-28T13:16:04.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.852+0000 7ffba1600640 1 -- 172.21.15.19:0/2502621589 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ffb9c00a5b0 con 0x7ffba414c680 2024-03-28T13:16:04.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.853+0000 7ffba1600640 1 --2- 172.21.15.19:0/2502621589 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8406c560 0x7ffb8406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:04.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.853+0000 7ffba1600640 1 -- 172.21.15.19:0/2502621589 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7ffb9c08ae60 con 0x7ffba414c680 2024-03-28T13:16:04.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.853+0000 7ffba9800640 1 --2- 172.21.15.19:0/2502621589 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8406c560 0x7ffb8406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:04.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.854+0000 7ffba9800640 1 --2- 172.21.15.19:0/2502621589 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8406c560 0x7ffb8406ea20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009a40 tx=0x7ffb98008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:04.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:04.857+0000 7ffba1600640 1 -- 172.21.15.19:0/2502621589 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ffb9c01a030 con 0x7ffba414c680 2024-03-28T13:16:05.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.077+0000 7ffbaac00640 1 -- 172.21.15.19:0/2502621589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffba4150810 con 0x7ffba414c680 2024-03-28T13:16:05.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.077+0000 7ffba1600640 1 -- 172.21.15.19:0/2502621589 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v80) ==== 80+0+122 (secure 0 0 0) 0x7ffb9c059420 con 0x7ffba414c680 2024-03-28T13:16:05.077 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6834/1189161682 2024-03-29T13:14:45.007899+0000 2024-03-28T13:16:05.077 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6832/1189161682 2024-03-29T13:14:45.007899+0000 2024-03-28T13:16:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:listed 2 entries 2024-03-28T13:16:05.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.081+0000 7ffbaac00640 1 -- 172.21.15.19:0/2502621589 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8406c560 msgr2=0x7ffb8406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:05.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.081+0000 7ffbaac00640 1 --2- 172.21.15.19:0/2502621589 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8406c560 0x7ffb8406ea20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ffb98009a40 tx=0x7ffb98008040 comp rx=0 tx=0).stop 2024-03-28T13:16:05.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.081+0000 7ffbaac00640 1 -- 172.21.15.19:0/2502621589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba414c680 msgr2=0x7ffba414ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:05.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.081+0000 7ffbaac00640 1 --2- 172.21.15.19:0/2502621589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba414c680 0x7ffba414ce10 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ffba40ccbe0 tx=0x7ffb9c00cf00 comp rx=0 tx=0).stop 2024-03-28T13:16:05.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.081+0000 7ffbaac00640 1 -- 172.21.15.19:0/2502621589 shutdown_connections 2024-03-28T13:16:05.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.081+0000 7ffbaac00640 1 --2- 172.21.15.19:0/2502621589 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8406c560 0x7ffb8406ea20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:05.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.081+0000 7ffbaac00640 1 --2- 172.21.15.19:0/2502621589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba414c680 0x7ffba414ce10 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:05.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.081+0000 7ffbaac00640 1 --2- 172.21.15.19:0/2502621589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba413f880 0x7ffba414c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:05.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.081+0000 7ffbaac00640 1 --2- 172.21.15.19:0/2502621589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffb8c004800 0x7ffba414ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:05.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.081+0000 7ffbaac00640 1 -- 172.21.15.19:0/2502621589 >> 172.21.15.19:0/2502621589 conn(0x7ffba40d0750 msgr2=0x7ffba4150d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:05.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.081+0000 7ffbaac00640 1 -- 172.21.15.19:0/2502621589 shutdown_connections 2024-03-28T13:16:05.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.081+0000 7ffbaac00640 1 -- 172.21.15.19:0/2502621589 wait complete. 2024-03-28T13:16:05.106 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:16:05.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:05.107 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T13:16:05.208 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:16:05.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:05.209 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:16:05.255 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:16:05.255 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:16:05.255 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:16:05.312 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:16:05.312 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:16:05.312 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:16:05.312 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:16:05.369 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:16:05.369 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:16:05.369 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:16:05.369 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:16:05.423 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:16:05.424 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:16:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.639+0000 7fbab9a00640 1 Processor -- start 2024-03-28T13:16:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.639+0000 7fbab9a00640 1 -- start start 2024-03-28T13:16:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.640+0000 7fbab9a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbab40b3500 0x7fbab40b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.640+0000 7fbab9a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbab4163650 con 0x7fbab40af630 2024-03-28T13:16:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.640+0000 7fbab9a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbab4163930 con 0x7fbab415d120 2024-03-28T13:16:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.640+0000 7fbab9a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fbab40723c0 con 0x7fbab40b3500 2024-03-28T13:16:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.641+0000 7fbab2000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbab40af630 0x7fbab4166b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59724/0 (socket says 172.21.15.19:59724) 2024-03-28T13:16:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.641+0000 7fbab2000640 1 -- 172.21.15.19:0/381856713 learned_addr learned my addr 172.21.15.19:0/381856713 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.641+0000 7fbab3400640 1 --2- 172.21.15.19:0/381856713 >> v2:172.21.15.188:3300/0 conn(0x7fbab40b3500 0x7fbab40b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:05.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.641+0000 7fbab1600640 1 -- 172.21.15.19:0/381856713 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1786681455 0 0) 0x7fbab4163650 con 0x7fbab40af630 2024-03-28T13:16:05.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.641+0000 7fbab1600640 1 -- 172.21.15.19:0/381856713 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba98003520 con 0x7fbab40af630 2024-03-28T13:16:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.642+0000 7fbab1600640 1 -- 172.21.15.19:0/381856713 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 294537283 0 0) 0x7fba98003520 con 0x7fbab40af630 2024-03-28T13:16:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.642+0000 7fbab1600640 1 -- 172.21.15.19:0/381856713 >> v2:172.21.15.188:3300/0 conn(0x7fbab40b3500 msgr2=0x7fbab40b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.642+0000 7fbab1600640 1 --2- 172.21.15.19:0/381856713 >> v2:172.21.15.188:3300/0 conn(0x7fbab40b3500 0x7fbab40b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.642+0000 7fbab1600640 1 -- 172.21.15.19:0/381856713 >> v1:172.21.15.188:6789/0 conn(0x7fbab415d120 legacy=0x7fbab4162e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.642+0000 7fbab1600640 1 -- 172.21.15.19:0/381856713 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab4163c10 con 0x7fbab40af630 2024-03-28T13:16:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.642+0000 7fbab1600640 1 -- 172.21.15.19:0/381856713 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fba9c001670 con 0x7fbab40af630 2024-03-28T13:16:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.642+0000 7fbab1600640 1 -- 172.21.15.19:0/381856713 >> v1:172.21.15.19:6789/0 conn(0x7fbab40af630 legacy=0x7fbab4166b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.642+0000 7fbab1600640 1 --2- 172.21.15.19:0/381856713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba98003a00 0x7fba98003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.642+0000 7fbab1600640 1 --2- 172.21.15.19:0/381856713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba98004800 0x7fba98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.642+0000 7fbab1600640 1 --2- 172.21.15.19:0/381856713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba980051c0 0x7fba9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.642+0000 7fbab1600640 1 -- 172.21.15.19:0/381856713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbab40723c0 con 0x7fba98004800 2024-03-28T13:16:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.642+0000 7fbab1600640 1 -- 172.21.15.19:0/381856713 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fba9800c0c0 con 0x7fba980051c0 2024-03-28T13:16:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.642+0000 7fbab1600640 1 -- 172.21.15.19:0/381856713 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fba9800c2c0 con 0x7fba98003a00 2024-03-28T13:16:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.642+0000 7fbab2000640 1 --2- 172.21.15.19:0/381856713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba98004800 0x7fba98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.643+0000 7fbab2000640 1 -- 172.21.15.19:0/381856713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba98003a00 msgr2=0x7fba98003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.643+0000 7fbab3400640 1 --2- 172.21.15.19:0/381856713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba980051c0 0x7fba9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:05.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.643+0000 7fbab2a00640 1 --2- 172.21.15.19:0/381856713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba98003a00 0x7fba98003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:05.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.643+0000 7fbab2000640 1 --2- 172.21.15.19:0/381856713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba98003a00 0x7fba98003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:05.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.643+0000 7fbab2000640 1 -- 172.21.15.19:0/381856713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba980051c0 msgr2=0x7fba9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:05.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.643+0000 7fbab2000640 1 --2- 172.21.15.19:0/381856713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba980051c0 0x7fba9800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:05.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.643+0000 7fbab2000640 1 -- 172.21.15.19:0/381856713 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba98003520 con 0x7fba98004800 2024-03-28T13:16:05.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.643+0000 7fbab3400640 1 --2- 172.21.15.19:0/381856713 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba980051c0 0x7fba9800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:05.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.643+0000 7fbab3400640 1 -- 172.21.15.19:0/381856713 reap_dead start 2024-03-28T13:16:05.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.643+0000 7fbab2a00640 1 --2- 172.21.15.19:0/381856713 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba98003a00 0x7fba98003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:05.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.644+0000 7fbab2000640 1 --2- 172.21.15.19:0/381856713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba98004800 0x7fba98004c80 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00c0e0 tx=0x7fba98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f41105c34302d14 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:05.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.644+0000 7fbab1600640 1 -- 172.21.15.19:0/381856713 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c007c40 con 0x7fba98004800 2024-03-28T13:16:05.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.644+0000 7fbab1600640 1 -- 172.21.15.19:0/381856713 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9c030070 con 0x7fba98004800 2024-03-28T13:16:05.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.644+0000 7fbab1600640 1 -- 172.21.15.19:0/381856713 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c005d70 con 0x7fba98004800 2024-03-28T13:16:05.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.645+0000 7fbab9a00640 1 -- 172.21.15.19:0/381856713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba98004800 msgr2=0x7fba98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:05.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.645+0000 7fbab9a00640 1 --2- 172.21.15.19:0/381856713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba98004800 0x7fba98004c80 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fba9c00c0e0 tx=0x7fba98005830 comp rx=0 tx=0).stop 2024-03-28T13:16:05.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.645+0000 7fbab9a00640 1 -- 172.21.15.19:0/381856713 shutdown_connections 2024-03-28T13:16:05.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.645+0000 7fbab9a00640 1 --2- 172.21.15.19:0/381856713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba98004800 0x7fba98004c80 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:05.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.645+0000 7fbab9a00640 1 -- 172.21.15.19:0/381856713 >> 172.21.15.19:0/381856713 conn(0x7fbab406b990 msgr2=0x7fbab406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:05.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.645+0000 7fbab9a00640 1 -- 172.21.15.19:0/381856713 shutdown_connections 2024-03-28T13:16:05.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.645+0000 7fbab9a00640 1 -- 172.21.15.19:0/381856713 wait complete. 2024-03-28T13:16:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.647+0000 7fbab9a00640 1 Processor -- start 2024-03-28T13:16:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.647+0000 7fbab9a00640 1 -- start start 2024-03-28T13:16:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.648+0000 7fbab9a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba98004800 0x7fbab41516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.648+0000 7fbab9a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbab41fc670 0x7fbab415b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.648+0000 7fbab9a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab41fcaf0 0x7fbab415b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.648+0000 7fbab9a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbab40b3b30 con 0x7fbab41fcaf0 2024-03-28T13:16:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.648+0000 7fbab9a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbab40b39b0 con 0x7fbab41fc670 2024-03-28T13:16:05.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.648+0000 7fbab9a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbab40b3cb0 con 0x7fba98004800 2024-03-28T13:16:05.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.648+0000 7fbab3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab41fcaf0 0x7fbab415b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:05.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.648+0000 7fbab3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab41fcaf0 0x7fbab415b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40768/0 (socket says 172.21.15.19:40768) 2024-03-28T13:16:05.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.648+0000 7fbab3400640 1 -- 172.21.15.19:0/172285794 learned_addr learned my addr 172.21.15.19:0/172285794 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:05.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.648+0000 7fbab2a00640 1 --2- 172.21.15.19:0/172285794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba98004800 0x7fbab41516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:05.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.649+0000 7fbab2000640 1 --2- 172.21.15.19:0/172285794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbab41fc670 0x7fbab415b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:05.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.649+0000 7fbab3400640 1 -- 172.21.15.19:0/172285794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba98004800 msgr2=0x7fbab41516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.649+0000 7fbab3400640 1 --2- 172.21.15.19:0/172285794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba98004800 0x7fbab41516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.649+0000 7fbab3400640 1 -- 172.21.15.19:0/172285794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbab41fc670 msgr2=0x7fbab415b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.649+0000 7fbab3400640 1 --2- 172.21.15.19:0/172285794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbab41fc670 0x7fbab415b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.649+0000 7fbab3400640 1 -- 172.21.15.19:0/172285794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab4151c20 con 0x7fbab41fcaf0 2024-03-28T13:16:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.649+0000 7fbab3400640 1 --2- 172.21.15.19:0/172285794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab41fcaf0 0x7fbab415b790 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fbaac010790 tx=0x7fbaac010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.649+0000 7fbaaaa00640 1 -- 172.21.15.19:0/172285794 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac00f040 con 0x7fbab41fcaf0 2024-03-28T13:16:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.649+0000 7fbaaaa00640 1 -- 172.21.15.19:0/172285794 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbaac01b070 con 0x7fbab41fcaf0 2024-03-28T13:16:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.649+0000 7fbaaaa00640 1 -- 172.21.15.19:0/172285794 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac004430 con 0x7fbab41fcaf0 2024-03-28T13:16:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.649+0000 7fbab9a00640 1 -- 172.21.15.19:0/172285794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab40afe50 con 0x7fbab41fcaf0 2024-03-28T13:16:05.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.650+0000 7fbab9a00640 1 -- 172.21.15.19:0/172285794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab40b0410 con 0x7fbab41fcaf0 2024-03-28T13:16:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.651+0000 7fbab9a00640 1 -- 172.21.15.19:0/172285794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab40685c0 con 0x7fbab41fcaf0 2024-03-28T13:16:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.653+0000 7fbaaaa00640 1 -- 172.21.15.19:0/172285794 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbaac00e670 con 0x7fbab41fcaf0 2024-03-28T13:16:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.654+0000 7fbaaaa00640 1 --2- 172.21.15.19:0/172285794 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fba9006c560 0x7fba9006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.654+0000 7fbaaaa00640 1 -- 172.21.15.19:0/172285794 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7fbaac08b580 con 0x7fbab41fcaf0 2024-03-28T13:16:05.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.655+0000 7fbab2a00640 1 --2- 172.21.15.19:0/172285794 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fba9006c560 0x7fba9006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:05.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.656+0000 7fbab2a00640 1 --2- 172.21.15.19:0/172285794 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fba9006c560 0x7fba9006ea20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4007fd0 tx=0x7fbaa4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:05.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.659+0000 7fbaaaa00640 1 -- 172.21.15.19:0/172285794 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbab4151c20 con 0x7fbab41fcaf0 2024-03-28T13:16:05.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.903+0000 7fbab9a00640 1 -- 172.21.15.19:0/172285794 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbab40b2890 con 0x7fbab41fcaf0 2024-03-28T13:16:05.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.903+0000 7fbaaaa00640 1 -- 172.21.15.19:0/172285794 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5188 (secure 0 0 0) 0x7fbaac059a70 con 0x7fbab41fcaf0 2024-03-28T13:16:05.903 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:05.903 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/144726418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":144726418},{"type":"v1","addr":"172.21.15.188:6836","nonce":144726418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5363,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2642581607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2642581607},{"type":"v1","addr":"172.21.15.19:6835","nonce":2642581607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1721039756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1721039756},{"type":"v1","addr":"172.21.15.188:6837","nonce":1721039756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:14:31.526577+0000","modified":"2024-03-28T13:15:32.038485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.19:6834/2130552083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2130552083},{"type":"v1","addr":"172.21.15.19:6834","nonce":2130552083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6708,"qdb_cluster":[6708]},"id":5}]} 2024-03-28T13:16:05.904 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 53 2024-03-28T13:16:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.906+0000 7fbab9a00640 1 -- 172.21.15.19:0/172285794 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fba9006c560 msgr2=0x7fba9006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.906+0000 7fbab9a00640 1 --2- 172.21.15.19:0/172285794 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fba9006c560 0x7fba9006ea20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbaa4007fd0 tx=0x7fbaa4009040 comp rx=0 tx=0).stop 2024-03-28T13:16:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.906+0000 7fbab9a00640 1 -- 172.21.15.19:0/172285794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab41fcaf0 msgr2=0x7fbab415b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:05.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.906+0000 7fbab9a00640 1 --2- 172.21.15.19:0/172285794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab41fcaf0 0x7fbab415b790 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fbaac010790 tx=0x7fbaac010c50 comp rx=0 tx=0).stop 2024-03-28T13:16:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.906+0000 7fbab9a00640 1 -- 172.21.15.19:0/172285794 shutdown_connections 2024-03-28T13:16:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.906+0000 7fbab9a00640 1 --2- 172.21.15.19:0/172285794 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fba9006c560 0x7fba9006ea20 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.906+0000 7fbab9a00640 1 --2- 172.21.15.19:0/172285794 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbab41fcaf0 0x7fbab415b790 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.906+0000 7fbab9a00640 1 --2- 172.21.15.19:0/172285794 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbab41fc670 0x7fbab415b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.906+0000 7fbab9a00640 1 --2- 172.21.15.19:0/172285794 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba98004800 0x7fbab41516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.906+0000 7fbab9a00640 1 -- 172.21.15.19:0/172285794 >> 172.21.15.19:0/172285794 conn(0x7fbab406b990 msgr2=0x7fbab40b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.907+0000 7fbab9a00640 1 -- 172.21.15.19:0/172285794 shutdown_connections 2024-03-28T13:16:05.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:05.907+0000 7fbab9a00640 1 -- 172.21.15.19:0/172285794 wait complete. 2024-03-28T13:16:05.925 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:16:06.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.127+0000 7f9a96800640 1 Processor -- start 2024-03-28T13:16:06.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.127+0000 7f9a96800640 1 -- start start 2024-03-28T13:16:06.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.128+0000 7f9a96800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a9015e4a0 0x7f9a900b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:06.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.128+0000 7f9a96800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9a900d2310 0x7f9a900ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:06.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.128+0000 7f9a96800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9a900cf530 0x7f9a900d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:06.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.128+0000 7f9a96800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9a900d34e0 con 0x7f9a900d2310 2024-03-28T13:16:06.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.128+0000 7f9a96800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9a9015e880 con 0x7f9a900cf530 2024-03-28T13:16:06.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.128+0000 7f9a96800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9a900d28d0 con 0x7f9a9015e4a0 2024-03-28T13:16:06.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.129+0000 7f9a94a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9a900d2310 0x7f9a900ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:06.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.129+0000 7f9a94a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9a900d2310 0x7f9a900ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40782/0 (socket says 172.21.15.19:40782) 2024-03-28T13:16:06.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.129+0000 7f9a94a00640 1 -- 172.21.15.19:0/1267847330 learned_addr learned my addr 172.21.15.19:0/1267847330 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:06.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.129+0000 7f9a95400640 1 --2- 172.21.15.19:0/1267847330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a9015e4a0 0x7f9a900b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:06.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.129+0000 7f9a95e00640 1 --2- 172.21.15.19:0/1267847330 >> v2:172.21.15.188:3300/0 conn(0x7f9a900cf530 0x7f9a900d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:06.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.129+0000 7f9a94a00640 1 -- 172.21.15.19:0/1267847330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a9015e4a0 msgr2=0x7f9a900b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:06.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.129+0000 7f9a94a00640 1 --2- 172.21.15.19:0/1267847330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a9015e4a0 0x7f9a900b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.129+0000 7f9a94a00640 1 -- 172.21.15.19:0/1267847330 >> v2:172.21.15.188:3300/0 conn(0x7f9a900cf530 msgr2=0x7f9a900d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:06.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.129+0000 7f9a94a00640 1 --2- 172.21.15.19:0/1267847330 >> v2:172.21.15.188:3300/0 conn(0x7f9a900cf530 0x7f9a900d3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.129+0000 7f9a94a00640 1 -- 172.21.15.19:0/1267847330 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a900d2bb0 con 0x7f9a900d2310 2024-03-28T13:16:06.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.131+0000 7f9a94a00640 1 --2- 172.21.15.19:0/1267847330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a900d2310 0x7f9a900ceff0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9a900d50f0 tx=0x7f9a7c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a609050e9dd9ee server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:06.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.131+0000 7f9a8fe00640 1 -- 172.21.15.19:0/1267847330 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c035020 con 0x7f9a900d2310 2024-03-28T13:16:06.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.131+0000 7f9a8fe00640 1 -- 172.21.15.19:0/1267847330 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a7c031870 con 0x7f9a900d2310 2024-03-28T13:16:06.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.131+0000 7f9a8fe00640 1 -- 172.21.15.19:0/1267847330 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c031c40 con 0x7f9a900d2310 2024-03-28T13:16:06.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.132+0000 7f9a96800640 1 -- 172.21.15.19:0/1267847330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a900d2310 msgr2=0x7f9a900ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:06.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.132+0000 7f9a96800640 1 --2- 172.21.15.19:0/1267847330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a900d2310 0x7f9a900ceff0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9a900d50f0 tx=0x7f9a7c02b910 comp rx=0 tx=0).stop 2024-03-28T13:16:06.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.132+0000 7f9a96800640 1 -- 172.21.15.19:0/1267847330 shutdown_connections 2024-03-28T13:16:06.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.132+0000 7f9a96800640 1 --2- 172.21.15.19:0/1267847330 >> v2:172.21.15.188:3300/0 conn(0x7f9a900cf530 0x7f9a900d3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.132+0000 7f9a96800640 1 --2- 172.21.15.19:0/1267847330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a900d2310 0x7f9a900ceff0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.132+0000 7f9a96800640 1 --2- 172.21.15.19:0/1267847330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a9015e4a0 0x7f9a900b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.132+0000 7f9a96800640 1 -- 172.21.15.19:0/1267847330 >> 172.21.15.19:0/1267847330 conn(0x7f9a900676d0 msgr2=0x7f9a90067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:06.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.132+0000 7f9a96800640 1 -- 172.21.15.19:0/1267847330 shutdown_connections 2024-03-28T13:16:06.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.132+0000 7f9a96800640 1 -- 172.21.15.19:0/1267847330 wait complete. 2024-03-28T13:16:06.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.134+0000 7f9a96800640 1 Processor -- start 2024-03-28T13:16:06.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.134+0000 7f9a96800640 1 -- start start 2024-03-28T13:16:06.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.135+0000 7f9a96800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a900cf530 0x7f9a90143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:06.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.135+0000 7f9a96800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a900d2310 0x7f9a90150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:06.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.135+0000 7f9a96800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a9015e4a0 0x7f9a900c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:06.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.135+0000 7f9a96800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a900d3970 con 0x7f9a900cf530 2024-03-28T13:16:06.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.135+0000 7f9a96800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9a900d3170 con 0x7f9a9015e4a0 2024-03-28T13:16:06.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.135+0000 7f9a96800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9a900ba1a0 con 0x7f9a900d2310 2024-03-28T13:16:06.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.135+0000 7f9a95400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a900cf530 0x7f9a90143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:06.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.135+0000 7f9a95400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a900cf530 0x7f9a90143cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40796/0 (socket says 172.21.15.19:40796) 2024-03-28T13:16:06.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.135+0000 7f9a95400640 1 -- 172.21.15.19:0/2076840194 learned_addr learned my addr 172.21.15.19:0/2076840194 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:06.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.136+0000 7f9a95e00640 1 --2- 172.21.15.19:0/2076840194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a9015e4a0 0x7f9a900c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:06.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.136+0000 7f9a94a00640 1 --2- 172.21.15.19:0/2076840194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a900d2310 0x7f9a90150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:06.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.136+0000 7f9a95400640 1 -- 172.21.15.19:0/2076840194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a900d2310 msgr2=0x7f9a90150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:06.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.136+0000 7f9a95400640 1 --2- 172.21.15.19:0/2076840194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a900d2310 0x7f9a90150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.136+0000 7f9a95400640 1 -- 172.21.15.19:0/2076840194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a9015e4a0 msgr2=0x7f9a900c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:06.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.136+0000 7f9a95400640 1 --2- 172.21.15.19:0/2076840194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a9015e4a0 0x7f9a900c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.136+0000 7f9a95400640 1 -- 172.21.15.19:0/2076840194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a901f4080 con 0x7f9a900cf530 2024-03-28T13:16:06.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.136+0000 7f9a95400640 1 --2- 172.21.15.19:0/2076840194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a900cf530 0x7f9a90143cb0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400d5f0 tx=0x7f9a8400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:06.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.137+0000 7f9a8d600640 1 -- 172.21.15.19:0/2076840194 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a840125f0 con 0x7f9a900cf530 2024-03-28T13:16:06.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.137+0000 7f9a8d600640 1 -- 172.21.15.19:0/2076840194 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a84019070 con 0x7f9a900cf530 2024-03-28T13:16:06.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.137+0000 7f9a8d600640 1 -- 172.21.15.19:0/2076840194 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a840129d0 con 0x7f9a900cf530 2024-03-28T13:16:06.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.137+0000 7f9a96800640 1 -- 172.21.15.19:0/2076840194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a900b7f50 con 0x7f9a900cf530 2024-03-28T13:16:06.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.137+0000 7f9a96800640 1 -- 172.21.15.19:0/2076840194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a900b8510 con 0x7f9a900cf530 2024-03-28T13:16:06.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.138+0000 7f9a96800640 1 -- 172.21.15.19:0/2076840194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a90075940 con 0x7f9a900cf530 2024-03-28T13:16:06.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.139+0000 7f9a8d600640 1 -- 172.21.15.19:0/2076840194 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9a84012b70 con 0x7f9a900cf530 2024-03-28T13:16:06.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.140+0000 7f9a8d600640 1 --2- 172.21.15.19:0/2076840194 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9a7006c560 0x7f9a7006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:06.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.140+0000 7f9a8d600640 1 -- 172.21.15.19:0/2076840194 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f9a8408bac0 con 0x7f9a900cf530 2024-03-28T13:16:06.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.141+0000 7f9a94a00640 1 --2- 172.21.15.19:0/2076840194 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9a7006c560 0x7f9a7006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:06.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.142+0000 7f9a94a00640 1 --2- 172.21.15.19:0/2076840194 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9a7006c560 0x7f9a7006ea20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c036000 tx=0x7f9a7c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:06.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.144+0000 7f9a8d600640 1 -- 172.21.15.19:0/2076840194 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9a8405a040 con 0x7f9a900cf530 2024-03-28T13:16:06.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.379+0000 7f9a96800640 1 -- 172.21.15.19:0/2076840194 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a9015c080 con 0x7f9a900cf530 2024-03-28T13:16:06.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.380+0000 7f9a8d600640 1 -- 172.21.15.19:0/2076840194 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16835 (secure 0 0 0) 0x7f9a84059940 con 0x7f9a900cf530 2024-03-28T13:16:06.381 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:06.381 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:15:25.200670+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:14:29.424080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-28T13:14:30.494667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1d","mappings":[{"from":2,"to":7}]},{"pgid":"11.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/1189161682":"2024-03-29T13:14:45.007899+0000","172.21.15.19:6832/1189161682":"2024-03-29T13:14:45.007899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:16:06.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.384+0000 7f9a96800640 1 -- 172.21.15.19:0/2076840194 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9a7006c560 msgr2=0x7f9a7006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:06.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.384+0000 7f9a96800640 1 --2- 172.21.15.19:0/2076840194 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9a7006c560 0x7f9a7006ea20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c036000 tx=0x7f9a7c03c040 comp rx=0 tx=0).stop 2024-03-28T13:16:06.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.384+0000 7f9a96800640 1 -- 172.21.15.19:0/2076840194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a900cf530 msgr2=0x7f9a90143cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:06.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.384+0000 7f9a96800640 1 --2- 172.21.15.19:0/2076840194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a900cf530 0x7f9a90143cb0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400d5f0 tx=0x7f9a8400dab0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.384+0000 7f9a96800640 1 -- 172.21.15.19:0/2076840194 shutdown_connections 2024-03-28T13:16:06.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.384+0000 7f9a96800640 1 --2- 172.21.15.19:0/2076840194 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9a7006c560 0x7f9a7006ea20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.384+0000 7f9a96800640 1 --2- 172.21.15.19:0/2076840194 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a9015e4a0 0x7f9a900c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.384+0000 7f9a96800640 1 --2- 172.21.15.19:0/2076840194 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a900d2310 0x7f9a90150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.384+0000 7f9a96800640 1 --2- 172.21.15.19:0/2076840194 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a900cf530 0x7f9a90143cb0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.384+0000 7f9a96800640 1 -- 172.21.15.19:0/2076840194 >> 172.21.15.19:0/2076840194 conn(0x7f9a900676d0 msgr2=0x7f9a9015cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:06.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.384+0000 7f9a96800640 1 -- 172.21.15.19:0/2076840194 shutdown_connections 2024-03-28T13:16:06.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.384+0000 7f9a96800640 1 -- 172.21.15.19:0/2076840194 wait complete. 2024-03-28T13:16:06.407 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:14:29.424080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:14:30.494667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:16:06.408 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:16:06.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.622+0000 7fe596000640 1 Processor -- start 2024-03-28T13:16:06.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.623+0000 7fe596000640 1 -- start start 2024-03-28T13:16:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.624+0000 7fe596000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe590162920 0x7fe5901670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.624+0000 7fe596000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5900d0990 0x7fe590071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.624+0000 7fe596000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe590071d60 con 0x7fe590163ac0 2024-03-28T13:16:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.624+0000 7fe596000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fe590163f20 con 0x7fe590162920 2024-03-28T13:16:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.624+0000 7fe596000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe5901665d0 con 0x7fe5900d0990 2024-03-28T13:16:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.625+0000 7fe58f400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe590162920 0x7fe5901670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.625+0000 7fe58f400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe590162920 0x7fe5901670b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44112/0 (socket says 172.21.15.19:44112) 2024-03-28T13:16:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.625+0000 7fe58f400640 1 -- 172.21.15.19:0/2651221852 learned_addr learned my addr 172.21.15.19:0/2651221852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.625+0000 7fe58fe00640 1 --2- 172.21.15.19:0/2651221852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5900d0990 0x7fe590071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.626+0000 7fe58fe00640 1 -- 172.21.15.19:0/2651221852 >> v2:172.21.15.188:3300/0 conn(0x7fe590162920 msgr2=0x7fe5901670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.626+0000 7fe58fe00640 1 --2- 172.21.15.19:0/2651221852 >> v2:172.21.15.188:3300/0 conn(0x7fe590162920 0x7fe5901670b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.626+0000 7fe58fe00640 1 -- 172.21.15.19:0/2651221852 >> v1:172.21.15.188:6789/0 conn(0x7fe590163ac0 legacy=0x7fe5900d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.626+0000 7fe58fe00640 1 -- 172.21.15.19:0/2651221852 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5901668b0 con 0x7fe5900d0990 2024-03-28T13:16:06.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.626+0000 7fe58f400640 1 --2- 172.21.15.19:0/2651221852 >> v2:172.21.15.188:3300/0 conn(0x7fe590162920 0x7fe5901670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:16:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.628+0000 7fe58fe00640 1 --2- 172.21.15.19:0/2651221852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5900d0990 0x7fe590071610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe588009d90 tx=0x7fe58802be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfc5a5aa52eb98cf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.628+0000 7fe58ea00640 1 -- 172.21.15.19:0/2651221852 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588039070 con 0x7fe5900d0990 2024-03-28T13:16:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.628+0000 7fe58ea00640 1 -- 172.21.15.19:0/2651221852 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe58802cbd0 con 0x7fe5900d0990 2024-03-28T13:16:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.628+0000 7fe58ea00640 1 -- 172.21.15.19:0/2651221852 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588034440 con 0x7fe5900d0990 2024-03-28T13:16:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.629+0000 7fe596000640 1 -- 172.21.15.19:0/2651221852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5900d0990 msgr2=0x7fe590071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.629+0000 7fe596000640 1 --2- 172.21.15.19:0/2651221852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5900d0990 0x7fe590071610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe588009d90 tx=0x7fe58802be20 comp rx=0 tx=0).stop 2024-03-28T13:16:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.629+0000 7fe596000640 1 -- 172.21.15.19:0/2651221852 shutdown_connections 2024-03-28T13:16:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.629+0000 7fe596000640 1 --2- 172.21.15.19:0/2651221852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5900d0990 0x7fe590071610 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.629+0000 7fe596000640 1 --2- 172.21.15.19:0/2651221852 >> v2:172.21.15.188:3300/0 conn(0x7fe590162920 0x7fe5901670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.629+0000 7fe596000640 1 -- 172.21.15.19:0/2651221852 >> 172.21.15.19:0/2651221852 conn(0x7fe59006b750 msgr2=0x7fe59006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.629+0000 7fe596000640 1 -- 172.21.15.19:0/2651221852 shutdown_connections 2024-03-28T13:16:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.629+0000 7fe596000640 1 -- 172.21.15.19:0/2651221852 wait complete. 2024-03-28T13:16:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.631+0000 7fe596000640 1 Processor -- start 2024-03-28T13:16:06.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.631+0000 7fe596000640 1 -- start start 2024-03-28T13:16:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.632+0000 7fe596000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5900d0990 0x7fe5900b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.632+0000 7fe596000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe590162920 0x7fe5900b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.632+0000 7fe596000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe590163ac0 0x7fe590160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.632+0000 7fe596000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe590072040 con 0x7fe590162920 2024-03-28T13:16:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.632+0000 7fe596000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe590162d00 con 0x7fe590163ac0 2024-03-28T13:16:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.632+0000 7fe596000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe5900721c0 con 0x7fe5900d0990 2024-03-28T13:16:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.632+0000 7fe586a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe590162920 0x7fe5900b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.632+0000 7fe586a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe590162920 0x7fe5900b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40804/0 (socket says 172.21.15.19:40804) 2024-03-28T13:16:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.632+0000 7fe586a00640 1 -- 172.21.15.19:0/2962054601 learned_addr learned my addr 172.21.15.19:0/2962054601 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.633+0000 7fe58f400640 1 --2- 172.21.15.19:0/2962054601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5900d0990 0x7fe5900b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.633+0000 7fe58fe00640 1 --2- 172.21.15.19:0/2962054601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe590163ac0 0x7fe590160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.633+0000 7fe586a00640 1 -- 172.21.15.19:0/2962054601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5900d0990 msgr2=0x7fe5900b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.633+0000 7fe586a00640 1 --2- 172.21.15.19:0/2962054601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5900d0990 0x7fe5900b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.633+0000 7fe586a00640 1 -- 172.21.15.19:0/2962054601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe590163ac0 msgr2=0x7fe590160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.633+0000 7fe586a00640 1 --2- 172.21.15.19:0/2962054601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe590163ac0 0x7fe590160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.633+0000 7fe586a00640 1 -- 172.21.15.19:0/2962054601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5900b4600 con 0x7fe590162920 2024-03-28T13:16:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.633+0000 7fe586a00640 1 --2- 172.21.15.19:0/2962054601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe590162920 0x7fe5900b3b40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe57800c340 tx=0x7fe57800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.633+0000 7fe587e00640 1 -- 172.21.15.19:0/2962054601 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578017070 con 0x7fe590162920 2024-03-28T13:16:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.633+0000 7fe587e00640 1 -- 172.21.15.19:0/2962054601 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5780052d0 con 0x7fe590162920 2024-03-28T13:16:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.633+0000 7fe587e00640 1 -- 172.21.15.19:0/2962054601 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe578003850 con 0x7fe590162920 2024-03-28T13:16:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.633+0000 7fe596000640 1 -- 172.21.15.19:0/2962054601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5900b48e0 con 0x7fe590162920 2024-03-28T13:16:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.634+0000 7fe596000640 1 -- 172.21.15.19:0/2962054601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5900b4ea0 con 0x7fe590162920 2024-03-28T13:16:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.635+0000 7fe596000640 1 -- 172.21.15.19:0/2962054601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5900685c0 con 0x7fe590162920 2024-03-28T13:16:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.636+0000 7fe587e00640 1 -- 172.21.15.19:0/2962054601 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe57801a070 con 0x7fe590162920 2024-03-28T13:16:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.637+0000 7fe587e00640 1 --2- 172.21.15.19:0/2962054601 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe56c06c530 0x7fe56c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.637+0000 7fe587e00640 1 -- 172.21.15.19:0/2962054601 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7fe57808ab30 con 0x7fe590162920 2024-03-28T13:16:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.638+0000 7fe58f400640 1 --2- 172.21.15.19:0/2962054601 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe56c06c530 0x7fe56c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:06.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.639+0000 7fe58f400640 1 --2- 172.21.15.19:0/2962054601 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe56c06c530 0x7fe56c06e9f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe580009b70 tx=0x7fe580008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:06.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:06.641+0000 7fe587e00640 1 -- 172.21.15.19:0/2962054601 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe578059020 con 0x7fe590162920 2024-03-28T13:16:07.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.009+0000 7fe596000640 1 -- 172.21.15.19:0/2962054601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe59006ef60 con 0x7fe590162920 2024-03-28T13:16:07.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.010+0000 7fe587e00640 1 -- 172.21.15.19:0/2962054601 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) ==== 93+0+10 (secure 0 0 0) 0x7fe578058920 con 0x7fe590162920 2024-03-28T13:16:07.010 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:16:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.013+0000 7fe596000640 1 -- 172.21.15.19:0/2962054601 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe56c06c530 msgr2=0x7fe56c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.014+0000 7fe596000640 1 --2- 172.21.15.19:0/2962054601 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe56c06c530 0x7fe56c06e9f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe580009b70 tx=0x7fe580008040 comp rx=0 tx=0).stop 2024-03-28T13:16:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.014+0000 7fe596000640 1 -- 172.21.15.19:0/2962054601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe590162920 msgr2=0x7fe5900b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.014+0000 7fe596000640 1 --2- 172.21.15.19:0/2962054601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe590162920 0x7fe5900b3b40 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe57800c340 tx=0x7fe57800c800 comp rx=0 tx=0).stop 2024-03-28T13:16:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.014+0000 7fe596000640 1 -- 172.21.15.19:0/2962054601 shutdown_connections 2024-03-28T13:16:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.014+0000 7fe596000640 1 --2- 172.21.15.19:0/2962054601 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe56c06c530 0x7fe56c06e9f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.014+0000 7fe596000640 1 --2- 172.21.15.19:0/2962054601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe590163ac0 0x7fe590160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.014+0000 7fe596000640 1 --2- 172.21.15.19:0/2962054601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe590162920 0x7fe5900b3b40 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.014+0000 7fe596000640 1 --2- 172.21.15.19:0/2962054601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5900d0990 0x7fe5900b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.014+0000 7fe596000640 1 -- 172.21.15.19:0/2962054601 >> 172.21.15.19:0/2962054601 conn(0x7fe59006b750 msgr2=0x7fe5900d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:07.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.014+0000 7fe596000640 1 -- 172.21.15.19:0/2962054601 shutdown_connections 2024-03-28T13:16:07.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.014+0000 7fe596000640 1 -- 172.21.15.19:0/2962054601 wait complete. 2024-03-28T13:16:07.039 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:16:07.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.258+0000 7f274c600640 1 Processor -- start 2024-03-28T13:16:07.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.258+0000 7f274c600640 1 -- start start 2024-03-28T13:16:07.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.259+0000 7f274c600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27440d0990 0x7f27440aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:07.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.259+0000 7f274c600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f27440af2e0 0x7f27441600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:07.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.260+0000 7f274c600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27440afab0 con 0x7f2744153580 2024-03-28T13:16:07.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.260+0000 7f274c600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f274415cc80 con 0x7f27440af2e0 2024-03-28T13:16:07.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.260+0000 7f274c600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f274415da00 con 0x7f27440d0990 2024-03-28T13:16:07.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.260+0000 7f2749c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2744153580 0x7f27440d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59732/0 (socket says 172.21.15.19:59732) 2024-03-28T13:16:07.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.260+0000 7f2749c00640 1 -- 172.21.15.19:0/189015299 learned_addr learned my addr 172.21.15.19:0/189015299 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:07.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.260+0000 7f274a600640 1 --2- 172.21.15.19:0/189015299 >> v2:172.21.15.19:3300/0 conn(0x7f27440af2e0 0x7f27441600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:07.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.260+0000 7f2749200640 1 --2- 172.21.15.19:0/189015299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27440d0990 0x7f27440aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.260+0000 7f2743e00640 1 -- 172.21.15.19:0/189015299 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1053873824 0 0) 0x7f27440afab0 con 0x7f2744153580 2024-03-28T13:16:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.260+0000 7f2743e00640 1 -- 172.21.15.19:0/189015299 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f272c003520 con 0x7f2744153580 2024-03-28T13:16:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.261+0000 7f274a600640 1 -- 172.21.15.19:0/189015299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27440d0990 msgr2=0x7f27440aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.261+0000 7f274a600640 1 --2- 172.21.15.19:0/189015299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27440d0990 0x7f27440aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.261+0000 7f274a600640 1 -- 172.21.15.19:0/189015299 >> v1:172.21.15.19:6789/0 conn(0x7f2744153580 legacy=0x7f27440d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:07.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.261+0000 7f274a600640 1 -- 172.21.15.19:0/189015299 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274415cf20 con 0x7f27440af2e0 2024-03-28T13:16:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.262+0000 7f274a600640 1 --2- 172.21.15.19:0/189015299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440af2e0 0x7f27441600d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f273c009d90 tx=0x7f273c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2796fe4a35c60b4c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.263+0000 7f2743e00640 1 -- 172.21.15.19:0/189015299 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c039070 con 0x7f27440af2e0 2024-03-28T13:16:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.263+0000 7f2743e00640 1 -- 172.21.15.19:0/189015299 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f273c02cbc0 con 0x7f27440af2e0 2024-03-28T13:16:07.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.263+0000 7f2743e00640 1 -- 172.21.15.19:0/189015299 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f273c034440 con 0x7f27440af2e0 2024-03-28T13:16:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.263+0000 7f274c600640 1 -- 172.21.15.19:0/189015299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440af2e0 msgr2=0x7f27441600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.263+0000 7f274c600640 1 --2- 172.21.15.19:0/189015299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440af2e0 0x7f27441600d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f273c009d90 tx=0x7f273c02be10 comp rx=0 tx=0).stop 2024-03-28T13:16:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.263+0000 7f274c600640 1 -- 172.21.15.19:0/189015299 shutdown_connections 2024-03-28T13:16:07.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.263+0000 7f274c600640 1 --2- 172.21.15.19:0/189015299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440af2e0 0x7f27441600d0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.263+0000 7f274c600640 1 --2- 172.21.15.19:0/189015299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27440d0990 0x7f27440aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.264+0000 7f274c600640 1 -- 172.21.15.19:0/189015299 >> 172.21.15.19:0/189015299 conn(0x7f274406b750 msgr2=0x7f274406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:07.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.264+0000 7f274c600640 1 -- 172.21.15.19:0/189015299 shutdown_connections 2024-03-28T13:16:07.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.264+0000 7f274c600640 1 -- 172.21.15.19:0/189015299 wait complete. 2024-03-28T13:16:07.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.265+0000 7f274c600640 1 Processor -- start 2024-03-28T13:16:07.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.265+0000 7f274c600640 1 -- start start 2024-03-28T13:16:07.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.266+0000 7f274c600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27440af2e0 0x7f27441589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:07.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.266+0000 7f274c600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440d0990 0x7f27440cea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:07.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.266+0000 7f274c600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2744153580 0x7f27440cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:07.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.266+0000 7f274c600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2744153960 con 0x7f27440d0990 2024-03-28T13:16:07.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.266+0000 7f274c600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f274415d4e0 con 0x7f27440af2e0 2024-03-28T13:16:07.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.266+0000 7f274c600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f27440af710 con 0x7f2744153580 2024-03-28T13:16:07.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.267+0000 7f2749200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440d0990 0x7f27440cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:07.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.267+0000 7f2749200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440d0990 0x7f27440cea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40832/0 (socket says 172.21.15.19:40832) 2024-03-28T13:16:07.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.267+0000 7f2749200640 1 -- 172.21.15.19:0/1971075706 learned_addr learned my addr 172.21.15.19:0/1971075706 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:07.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.267+0000 7f2749c00640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27440af2e0 0x7f27441589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:07.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.267+0000 7f274a600640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2744153580 0x7f27440cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:07.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.267+0000 7f2749200640 1 -- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2744153580 msgr2=0x7f27440cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:07.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.267+0000 7f2749200640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2744153580 0x7f27440cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.267+0000 7f2749200640 1 -- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27440af2e0 msgr2=0x7f27441589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:07.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.267+0000 7f2749200640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27440af2e0 0x7f27441589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.267+0000 7f2749200640 1 -- 172.21.15.19:0/1971075706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2744159890 con 0x7f27440d0990 2024-03-28T13:16:07.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.267+0000 7f274a600640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2744153580 0x7f27440cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:07.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.267+0000 7f2749c00640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27440af2e0 0x7f27441589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:07.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.268+0000 7f2749200640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440d0990 0x7f27440cea60 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f273000d5f0 tx=0x7f273000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:07.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.268+0000 7f2741600640 1 -- 172.21.15.19:0/1971075706 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27300125f0 con 0x7f27440d0990 2024-03-28T13:16:07.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.268+0000 7f2741600640 1 -- 172.21.15.19:0/1971075706 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2730019070 con 0x7f27440d0990 2024-03-28T13:16:07.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.268+0000 7f2741600640 1 -- 172.21.15.19:0/1971075706 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27300129d0 con 0x7f27440d0990 2024-03-28T13:16:07.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.268+0000 7f274c600640 1 -- 172.21.15.19:0/1971075706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2744158f10 con 0x7f27440d0990 2024-03-28T13:16:07.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.268+0000 7f274c600640 1 -- 172.21.15.19:0/1971075706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27441f4f80 con 0x7f27440d0990 2024-03-28T13:16:07.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.269+0000 7f274c600640 1 -- 172.21.15.19:0/1971075706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27440ae3c0 con 0x7f27440d0990 2024-03-28T13:16:07.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.270+0000 7f2741600640 1 -- 172.21.15.19:0/1971075706 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2730012b70 con 0x7f27440d0990 2024-03-28T13:16:07.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.271+0000 7f2741600640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f272406c530 0x7f272406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:07.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.271+0000 7f2741600640 1 -- 172.21.15.19:0/1971075706 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f273008bb80 con 0x7f27440d0990 2024-03-28T13:16:07.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.272+0000 7f2749c00640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f272406c530 0x7f272406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:07.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.273+0000 7f2749c00640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f272406c530 0x7f272406e9f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2738005880 tx=0x7f2738009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:07.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.281+0000 7f2741600640 1 -- 172.21.15.19:0/1971075706 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f27440ae3c0 con 0x7f27440d0990 2024-03-28T13:16:07.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.523+0000 7f274c600640 1 -- 172.21.15.19:0/1971075706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2744159140 con 0x7f27440d0990 2024-03-28T13:16:07.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.524+0000 7f2741600640 1 -- 172.21.15.19:0/1971075706 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) ==== 104+0+11 (secure 0 0 0) 0x7f273005a100 con 0x7f27440d0990 2024-03-28T13:16:07.524 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:16:07.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.528+0000 7f274c600640 1 -- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f272406c530 msgr2=0x7f272406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:07.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.528+0000 7f274c600640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f272406c530 0x7f272406e9f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2738005880 tx=0x7f2738009040 comp rx=0 tx=0).stop 2024-03-28T13:16:07.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.528+0000 7f274c600640 1 -- 172.21.15.19:0/1971075706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440d0990 msgr2=0x7f27440cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:07.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.528+0000 7f274c600640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440d0990 0x7f27440cea60 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f273000d5f0 tx=0x7f273000dab0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.528+0000 7f274c600640 1 -- 172.21.15.19:0/1971075706 shutdown_connections 2024-03-28T13:16:07.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.528+0000 7f274c600640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f272406c530 0x7f272406e9f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.528+0000 7f274c600640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2744153580 0x7f27440cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.528+0000 7f274c600640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27440d0990 0x7f27440cea60 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.528+0000 7f274c600640 1 --2- 172.21.15.19:0/1971075706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27440af2e0 0x7f27441589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.528+0000 7f274c600640 1 -- 172.21.15.19:0/1971075706 >> 172.21.15.19:0/1971075706 conn(0x7f274406b750 msgr2=0x7f274415dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:07.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.528+0000 7f274c600640 1 -- 172.21.15.19:0/1971075706 shutdown_connections 2024-03-28T13:16:07.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.528+0000 7f274c600640 1 -- 172.21.15.19:0/1971075706 wait complete. 2024-03-28T13:16:07.547 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:16:07.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.771+0000 7f821aa00640 1 Processor -- start 2024-03-28T13:16:07.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.771+0000 7f821aa00640 1 -- start start 2024-03-28T13:16:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.773+0000 7f821aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f821413f190 0x7f82140cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.773+0000 7f821aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82140cc880 0x7f82140cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.773+0000 7f821aa00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82140ce890 con 0x7f821406d620 2024-03-28T13:16:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.773+0000 7f821aa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f82140cd200 con 0x7f821413f190 2024-03-28T13:16:07.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.773+0000 7f821aa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f82140ceb50 con 0x7f82140cc880 2024-03-28T13:16:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.773+0000 7f8219600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f821413f190 0x7f82140cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.773+0000 7f8219600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f821413f190 0x7f82140cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40848/0 (socket says 172.21.15.19:40848) 2024-03-28T13:16:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.774+0000 7f8219600640 1 -- 172.21.15.19:0/3180466977 learned_addr learned my addr 172.21.15.19:0/3180466977 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.773+0000 7f821a000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f821406d620 0x7f821406da70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59738/0 (socket says 172.21.15.19:59738) 2024-03-28T13:16:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.774+0000 7f8218c00640 1 --2- 172.21.15.19:0/3180466977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82140cc880 0x7f82140cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.774+0000 7f8219600640 1 -- 172.21.15.19:0/3180466977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82140cc880 msgr2=0x7f82140cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.774+0000 7f8219600640 1 --2- 172.21.15.19:0/3180466977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82140cc880 0x7f82140cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.774+0000 7f8219600640 1 -- 172.21.15.19:0/3180466977 >> v1:172.21.15.19:6789/0 conn(0x7f821406d620 legacy=0x7f821406da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.774+0000 7f8219600640 1 -- 172.21.15.19:0/3180466977 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82140cee30 con 0x7f821413f190 2024-03-28T13:16:07.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.774+0000 7f8218c00640 1 --2- 172.21.15.19:0/3180466977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82140cc880 0x7f82140cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:07.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.775+0000 7f8219600640 1 --2- 172.21.15.19:0/3180466977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f821413f190 0x7f82140cf630 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8208009870 tx=0x7f820802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e195ef6cef2e71 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:07.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.776+0000 7f8213e00640 1 -- 172.21.15.19:0/3180466977 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208034070 con 0x7f821413f190 2024-03-28T13:16:07.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.776+0000 7f8213e00640 1 -- 172.21.15.19:0/3180466977 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8208030840 con 0x7f821413f190 2024-03-28T13:16:07.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.776+0000 7f8213e00640 1 -- 172.21.15.19:0/3180466977 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208030c10 con 0x7f821413f190 2024-03-28T13:16:07.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.776+0000 7f821aa00640 1 -- 172.21.15.19:0/3180466977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f821413f190 msgr2=0x7f82140cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:07.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.776+0000 7f821aa00640 1 --2- 172.21.15.19:0/3180466977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f821413f190 0x7f82140cf630 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f8208009870 tx=0x7f820802b980 comp rx=0 tx=0).stop 2024-03-28T13:16:07.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.776+0000 7f821aa00640 1 -- 172.21.15.19:0/3180466977 shutdown_connections 2024-03-28T13:16:07.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.776+0000 7f821aa00640 1 --2- 172.21.15.19:0/3180466977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82140cc880 0x7f82140cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.776+0000 7f821aa00640 1 --2- 172.21.15.19:0/3180466977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f821413f190 0x7f82140cf630 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.776+0000 7f821aa00640 1 -- 172.21.15.19:0/3180466977 >> 172.21.15.19:0/3180466977 conn(0x7f82140b1ba0 msgr2=0x7f82140b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:07.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.777+0000 7f821aa00640 1 -- 172.21.15.19:0/3180466977 shutdown_connections 2024-03-28T13:16:07.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.777+0000 7f821aa00640 1 -- 172.21.15.19:0/3180466977 wait complete. 2024-03-28T13:16:07.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.779+0000 7f821aa00640 1 Processor -- start 2024-03-28T13:16:07.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.780+0000 7f821aa00640 1 -- start start 2024-03-28T13:16:07.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.781+0000 7f821aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f821406d620 0x7f8214158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:07.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.781+0000 7f821aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82140cc880 0x7f821413d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:07.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.781+0000 7f821aa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f821413f190 0x7f821413dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:07.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.781+0000 7f821aa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f821413f570 con 0x7f821406d620 2024-03-28T13:16:07.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.781+0000 7f821aa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8214160c70 con 0x7f821413f190 2024-03-28T13:16:07.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.781+0000 7f821aa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f82140cfb70 con 0x7f82140cc880 2024-03-28T13:16:07.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.782+0000 7f8219600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f821406d620 0x7f8214158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:07.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.782+0000 7f8219600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f821406d620 0x7f8214158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40864/0 (socket says 172.21.15.19:40864) 2024-03-28T13:16:07.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.782+0000 7f8219600640 1 -- 172.21.15.19:0/3382479578 learned_addr learned my addr 172.21.15.19:0/3382479578 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:07.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.782+0000 7f8219600640 1 -- 172.21.15.19:0/3382479578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82140cc880 msgr2=0x7f821413d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:07.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.782+0000 7f8219600640 1 --2- 172.21.15.19:0/3382479578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82140cc880 0x7f821413d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.782+0000 7f8219600640 1 -- 172.21.15.19:0/3382479578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f821413f190 msgr2=0x7f821413dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:07.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.782+0000 7f8219600640 1 --2- 172.21.15.19:0/3382479578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f821413f190 0x7f821413dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:07.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.782+0000 7f8219600640 1 -- 172.21.15.19:0/3382479578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8214158a90 con 0x7f821406d620 2024-03-28T13:16:07.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.782+0000 7f8219600640 1 --2- 172.21.15.19:0/3382479578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f821406d620 0x7f8214158550 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f820802b9c0 tx=0x7f8208004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:07.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.783+0000 7f8211600640 1 -- 172.21.15.19:0/3382479578 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208048070 con 0x7f821406d620 2024-03-28T13:16:07.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.783+0000 7f8211600640 1 -- 172.21.15.19:0/3382479578 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8208030640 con 0x7f821406d620 2024-03-28T13:16:07.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.783+0000 7f821aa00640 1 -- 172.21.15.19:0/3382479578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82141f2150 con 0x7f821406d620 2024-03-28T13:16:07.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.783+0000 7f8211600640 1 -- 172.21.15.19:0/3382479578 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8208034040 con 0x7f821406d620 2024-03-28T13:16:07.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.783+0000 7f821aa00640 1 -- 172.21.15.19:0/3382479578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82141f2710 con 0x7f821406d620 2024-03-28T13:16:07.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.785+0000 7f821aa00640 1 -- 172.21.15.19:0/3382479578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8214075940 con 0x7f821406d620 2024-03-28T13:16:07.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.785+0000 7f8211600640 1 -- 172.21.15.19:0/3382479578 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f820804b070 con 0x7f821406d620 2024-03-28T13:16:07.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.787+0000 7f8211600640 1 --2- 172.21.15.19:0/3382479578 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f81f406c530 0x7f81f406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:07.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.787+0000 7f8211600640 1 -- 172.21.15.19:0/3382479578 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f82080ad980 con 0x7f821406d620 2024-03-28T13:16:07.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.787+0000 7f8218c00640 1 --2- 172.21.15.19:0/3382479578 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f81f406c530 0x7f81f406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:07.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.788+0000 7f8218c00640 1 --2- 172.21.15.19:0/3382479578 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f81f406c530 0x7f81f406e9f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8200006fd0 tx=0x7f8200008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:07.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:07.790+0000 7f8211600640 1 -- 172.21.15.19:0/3382479578 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8208030810 con 0x7f821406d620 2024-03-28T13:16:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.031+0000 7f821aa00640 1 -- 172.21.15.19:0/3382479578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f82140b2870 con 0x7f821406d620 2024-03-28T13:16:08.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.032+0000 7f8211600640 1 -- 172.21.15.19:0/3382479578 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) ==== 100+0+11 (secure 0 0 0) 0x7f820807be70 con 0x7f821406d620 2024-03-28T13:16:08.032 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:16:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.035+0000 7f821aa00640 1 -- 172.21.15.19:0/3382479578 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f81f406c530 msgr2=0x7f81f406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.035+0000 7f821aa00640 1 --2- 172.21.15.19:0/3382479578 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f81f406c530 0x7f81f406e9f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8200006fd0 tx=0x7f8200008040 comp rx=0 tx=0).stop 2024-03-28T13:16:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.035+0000 7f821aa00640 1 -- 172.21.15.19:0/3382479578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f821406d620 msgr2=0x7f8214158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.035+0000 7f821aa00640 1 --2- 172.21.15.19:0/3382479578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f821406d620 0x7f8214158550 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f820802b9c0 tx=0x7f8208004730 comp rx=0 tx=0).stop 2024-03-28T13:16:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.035+0000 7f821aa00640 1 -- 172.21.15.19:0/3382479578 shutdown_connections 2024-03-28T13:16:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.035+0000 7f821aa00640 1 --2- 172.21.15.19:0/3382479578 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f81f406c530 0x7f81f406e9f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.035+0000 7f821aa00640 1 --2- 172.21.15.19:0/3382479578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f821413f190 0x7f821413dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.035+0000 7f821aa00640 1 --2- 172.21.15.19:0/3382479578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82140cc880 0x7f821413d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.035+0000 7f821aa00640 1 --2- 172.21.15.19:0/3382479578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f821406d620 0x7f8214158550 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.035+0000 7f821aa00640 1 -- 172.21.15.19:0/3382479578 >> 172.21.15.19:0/3382479578 conn(0x7f82140b1ba0 msgr2=0x7f821415eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.035+0000 7f821aa00640 1 -- 172.21.15.19:0/3382479578 shutdown_connections 2024-03-28T13:16:08.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.035+0000 7f821aa00640 1 -- 172.21.15.19:0/3382479578 wait complete. 2024-03-28T13:16:08.055 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:16:08.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.262+0000 7f32e7200640 1 Processor -- start 2024-03-28T13:16:08.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.262+0000 7f32e7200640 1 -- start start 2024-03-28T13:16:08.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.263+0000 7f32e7200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f32e015e700 0x7f32e00b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:08.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.263+0000 7f32e7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e00d1610 0x7f32e00b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:08.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.263+0000 7f32e7200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f32e00d1cf0 0x7f32e0070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:08.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.263+0000 7f32e7200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f32e015eae0 con 0x7f32e00d1cf0 2024-03-28T13:16:08.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.263+0000 7f32e7200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f32e00cfeb0 con 0x7f32e015e700 2024-03-28T13:16:08.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.263+0000 7f32e7200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f32e00d0190 con 0x7f32e00d1610 2024-03-28T13:16:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.264+0000 7f32e5200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f32e00d1cf0 0x7f32e0070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.264+0000 7f32e5200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f32e00d1cf0 0x7f32e0070fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40872/0 (socket says 172.21.15.19:40872) 2024-03-28T13:16:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.264+0000 7f32e5200640 1 -- 172.21.15.19:0/298485693 learned_addr learned my addr 172.21.15.19:0/298485693 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.264+0000 7f32dfe00640 1 --2- 172.21.15.19:0/298485693 >> v2:172.21.15.188:3300/0 conn(0x7f32e015e700 0x7f32e00b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.264+0000 7f32d7e00640 1 --2- 172.21.15.19:0/298485693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e00d1610 0x7f32e00b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.264+0000 7f32e5200640 1 -- 172.21.15.19:0/298485693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e00d1610 msgr2=0x7f32e00b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.264+0000 7f32e5200640 1 --2- 172.21.15.19:0/298485693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e00d1610 0x7f32e00b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.264+0000 7f32e5200640 1 -- 172.21.15.19:0/298485693 >> v2:172.21.15.188:3300/0 conn(0x7f32e015e700 msgr2=0x7f32e00b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:08.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.264+0000 7f32e5200640 1 --2- 172.21.15.19:0/298485693 >> v2:172.21.15.188:3300/0 conn(0x7f32e015e700 0x7f32e00b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.264+0000 7f32e5200640 1 -- 172.21.15.19:0/298485693 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e00d0470 con 0x7f32e00d1cf0 2024-03-28T13:16:08.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.265+0000 7f32d7e00640 1 --2- 172.21.15.19:0/298485693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e00d1610 0x7f32e00b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:08.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.266+0000 7f32e5200640 1 --2- 172.21.15.19:0/298485693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32e00d1cf0 0x7f32e0070fe0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f32e00723f0 tx=0x7f32d802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f8980afba9ea51 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.266+0000 7f32df400640 1 -- 172.21.15.19:0/298485693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d802c9f0 con 0x7f32e00d1cf0 2024-03-28T13:16:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.266+0000 7f32df400640 1 -- 172.21.15.19:0/298485693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d802cb90 con 0x7f32e00d1cf0 2024-03-28T13:16:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.266+0000 7f32df400640 1 -- 172.21.15.19:0/298485693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d80343f0 con 0x7f32e00d1cf0 2024-03-28T13:16:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.267+0000 7f32e7200640 1 -- 172.21.15.19:0/298485693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32e00d1cf0 msgr2=0x7f32e0070fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.267+0000 7f32e7200640 1 --2- 172.21.15.19:0/298485693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32e00d1cf0 0x7f32e0070fe0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f32e00723f0 tx=0x7f32d802be50 comp rx=0 tx=0).stop 2024-03-28T13:16:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.267+0000 7f32e7200640 1 -- 172.21.15.19:0/298485693 shutdown_connections 2024-03-28T13:16:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.267+0000 7f32e7200640 1 --2- 172.21.15.19:0/298485693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32e00d1cf0 0x7f32e0070fe0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.267+0000 7f32e7200640 1 --2- 172.21.15.19:0/298485693 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e00d1610 0x7f32e00b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.267+0000 7f32e7200640 1 --2- 172.21.15.19:0/298485693 >> v2:172.21.15.188:3300/0 conn(0x7f32e015e700 0x7f32e00b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.267+0000 7f32e7200640 1 -- 172.21.15.19:0/298485693 >> 172.21.15.19:0/298485693 conn(0x7f32e00d2240 msgr2=0x7f32e00d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:08.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.267+0000 7f32e7200640 1 -- 172.21.15.19:0/298485693 shutdown_connections 2024-03-28T13:16:08.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.267+0000 7f32e7200640 1 -- 172.21.15.19:0/298485693 wait complete. 2024-03-28T13:16:08.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.269+0000 7f32e7200640 1 Processor -- start 2024-03-28T13:16:08.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.269+0000 7f32e7200640 1 -- start start 2024-03-28T13:16:08.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.270+0000 7f32e7200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32e00d1610 0x7f32e00cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:08.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.270+0000 7f32e7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32e00d1cf0 0x7f32e00ca310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:08.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.270+0000 7f32e7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e015e700 0x7f32e0160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:08.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.270+0000 7f32e7200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32e006d1b0 con 0x7f32e00d1cf0 2024-03-28T13:16:08.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.270+0000 7f32e7200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f32e00d0a30 con 0x7f32e00d1610 2024-03-28T13:16:08.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.270+0000 7f32e7200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f32e006d330 con 0x7f32e015e700 2024-03-28T13:16:08.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.270+0000 7f32d7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32e00d1cf0 0x7f32e00ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:08.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.270+0000 7f32d7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32e00d1cf0 0x7f32e00ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40876/0 (socket says 172.21.15.19:40876) 2024-03-28T13:16:08.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.270+0000 7f32d7e00640 1 -- 172.21.15.19:0/3642649482 learned_addr learned my addr 172.21.15.19:0/3642649482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:08.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.271+0000 7f32dfe00640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32e00d1610 0x7f32e00cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:08.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.271+0000 7f32e5200640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e015e700 0x7f32e0160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:08.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.271+0000 7f32d7e00640 1 -- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e015e700 msgr2=0x7f32e0160fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:08.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.271+0000 7f32d7e00640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e015e700 0x7f32e0160fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.271+0000 7f32d7e00640 1 -- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32e00d1610 msgr2=0x7f32e00cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:08.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.271+0000 7f32d7e00640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32e00d1610 0x7f32e00cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.271+0000 7f32d7e00640 1 -- 172.21.15.19:0/3642649482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e00ca850 con 0x7f32e00d1cf0 2024-03-28T13:16:08.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.271+0000 7f32dfe00640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32e00d1610 0x7f32e00cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:08.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.271+0000 7f32e5200640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e015e700 0x7f32e0160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:08.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.271+0000 7f32d7e00640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32e00d1cf0 0x7f32e00ca310 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f32c800c7f0 tx=0x7f32c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:08.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.271+0000 7f32dcc00640 1 -- 172.21.15.19:0/3642649482 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8015070 con 0x7f32e00d1cf0 2024-03-28T13:16:08.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.271+0000 7f32dcc00640 1 -- 172.21.15.19:0/3642649482 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c8007820 con 0x7f32e00d1cf0 2024-03-28T13:16:08.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.271+0000 7f32dcc00640 1 -- 172.21.15.19:0/3642649482 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8007bd0 con 0x7f32e00d1cf0 2024-03-28T13:16:08.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.271+0000 7f32e7200640 1 -- 172.21.15.19:0/3642649482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e00caef0 con 0x7f32e00d1cf0 2024-03-28T13:16:08.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.272+0000 7f32e7200640 1 -- 172.21.15.19:0/3642649482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e00cb4b0 con 0x7f32e00d1cf0 2024-03-28T13:16:08.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.273+0000 7f32e7200640 1 -- 172.21.15.19:0/3642649482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32e00685c0 con 0x7f32e00d1cf0 2024-03-28T13:16:08.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.274+0000 7f32dcc00640 1 -- 172.21.15.19:0/3642649482 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f32c8018070 con 0x7f32e00d1cf0 2024-03-28T13:16:08.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.274+0000 7f32dcc00640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f32bc06c530 0x7f32bc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:08.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.275+0000 7f32dcc00640 1 -- 172.21.15.19:0/3642649482 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f32c808b920 con 0x7f32e00d1cf0 2024-03-28T13:16:08.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.275+0000 7f32dfe00640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f32bc06c530 0x7f32bc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:08.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.276+0000 7f32dfe00640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f32bc06c530 0x7f32bc06e9f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f32d0006fd0 tx=0x7f32d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:08.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.280+0000 7f32dcc00640 1 -- 172.21.15.19:0/3642649482 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f32c8059e10 con 0x7f32e00d1cf0 2024-03-28T13:16:08.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.527+0000 7f32e7200640 1 -- 172.21.15.19:0/3642649482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32e00c8100 con 0x7f32e00d1cf0 2024-03-28T13:16:08.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.528+0000 7f32dcc00640 1 -- 172.21.15.19:0/3642649482 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5188 (secure 0 0 0) 0x7f32c8059710 con 0x7f32e00d1cf0 2024-03-28T13:16:08.529 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:08.529 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/144726418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":144726418},{"type":"v1","addr":"172.21.15.188:6836","nonce":144726418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5363,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2642581607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2642581607},{"type":"v1","addr":"172.21.15.19:6835","nonce":2642581607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1721039756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1721039756},{"type":"v1","addr":"172.21.15.188:6837","nonce":1721039756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:14:31.526577+0000","modified":"2024-03-28T13:15:32.038485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.19:6834/2130552083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2130552083},{"type":"v1","addr":"172.21.15.19:6834","nonce":2130552083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6708,"qdb_cluster":[6708]},"id":5}]} 2024-03-28T13:16:08.529 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 53 2024-03-28T13:16:08.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.531+0000 7f32e7200640 1 -- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f32bc06c530 msgr2=0x7f32bc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:08.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.531+0000 7f32e7200640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f32bc06c530 0x7f32bc06e9f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f32d0006fd0 tx=0x7f32d0008040 comp rx=0 tx=0).stop 2024-03-28T13:16:08.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.531+0000 7f32e7200640 1 -- 172.21.15.19:0/3642649482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32e00d1cf0 msgr2=0x7f32e00ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:08.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.531+0000 7f32e7200640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32e00d1cf0 0x7f32e00ca310 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f32c800c7f0 tx=0x7f32c800ccb0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.531+0000 7f32e7200640 1 -- 172.21.15.19:0/3642649482 shutdown_connections 2024-03-28T13:16:08.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.531+0000 7f32e7200640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f32bc06c530 0x7f32bc06e9f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.531+0000 7f32e7200640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32e015e700 0x7f32e0160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.531+0000 7f32e7200640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32e00d1cf0 0x7f32e00ca310 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.531+0000 7f32e7200640 1 --2- 172.21.15.19:0/3642649482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f32e00d1610 0x7f32e00cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.531+0000 7f32e7200640 1 -- 172.21.15.19:0/3642649482 >> 172.21.15.19:0/3642649482 conn(0x7f32e00d2240 msgr2=0x7f32e00c7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:08.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.531+0000 7f32e7200640 1 -- 172.21.15.19:0/3642649482 shutdown_connections 2024-03-28T13:16:08.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.531+0000 7f32e7200640 1 -- 172.21.15.19:0/3642649482 wait complete. 2024-03-28T13:16:08.550 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:16:08.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.757+0000 7fca99e00640 1 Processor -- start 2024-03-28T13:16:08.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.757+0000 7fca99e00640 1 -- start start 2024-03-28T13:16:08.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.759+0000 7fca99e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9415c390 0x7fca9406b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:08.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.759+0000 7fca99e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fca940cdcc0 0x7fca940cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:08.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.759+0000 7fca99e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca940ce1c0 con 0x7fca9406bab0 2024-03-28T13:16:08.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.759+0000 7fca99e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fca9406bf10 con 0x7fca940cdcc0 2024-03-28T13:16:08.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.759+0000 7fca99e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fca940cea60 con 0x7fca9415c390 2024-03-28T13:16:08.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.760+0000 7fca99400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fca940cdcc0 0x7fca940cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.760+0000 7fca98a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9415c390 0x7fca9406b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.760+0000 7fca99400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fca940cdcc0 0x7fca940cf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44168/0 (socket says 172.21.15.19:44168) 2024-03-28T13:16:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.760+0000 7fca98a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9415c390 0x7fca9406b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:39628/0 (socket says 172.21.15.19:39628) 2024-03-28T13:16:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.760+0000 7fca99400640 1 -- 172.21.15.19:0/4255337641 learned_addr learned my addr 172.21.15.19:0/4255337641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.761+0000 7fca98a00640 1 -- 172.21.15.19:0/4255337641 >> v2:172.21.15.188:3300/0 conn(0x7fca940cdcc0 msgr2=0x7fca940cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:08.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.761+0000 7fca98a00640 1 --2- 172.21.15.19:0/4255337641 >> v2:172.21.15.188:3300/0 conn(0x7fca940cdcc0 0x7fca940cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.761+0000 7fca98a00640 1 -- 172.21.15.19:0/4255337641 >> v1:172.21.15.188:6789/0 conn(0x7fca9406bab0 legacy=0x7fca940cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:08.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.761+0000 7fca98a00640 1 -- 172.21.15.19:0/4255337641 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca940ce4a0 con 0x7fca9415c390 2024-03-28T13:16:08.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.761+0000 7fca99400640 1 --2- 172.21.15.19:0/4255337641 >> v2:172.21.15.188:3300/0 conn(0x7fca940cdcc0 0x7fca940cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:16:08.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.762+0000 7fca98a00640 1 --2- 172.21.15.19:0/4255337641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9415c390 0x7fca9406b570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fca880097a0 tx=0x7fca8802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf7bd5d6583bf8d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:08.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.762+0000 7fca93400640 1 -- 172.21.15.19:0/4255337641 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88035020 con 0x7fca9415c390 2024-03-28T13:16:08.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.763+0000 7fca93400640 1 -- 172.21.15.19:0/4255337641 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca88031810 con 0x7fca9415c390 2024-03-28T13:16:08.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.763+0000 7fca93400640 1 -- 172.21.15.19:0/4255337641 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88031be0 con 0x7fca9415c390 2024-03-28T13:16:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.763+0000 7fca99e00640 1 -- 172.21.15.19:0/4255337641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9415c390 msgr2=0x7fca9406b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.763+0000 7fca99e00640 1 --2- 172.21.15.19:0/4255337641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9415c390 0x7fca9406b570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fca880097a0 tx=0x7fca8802b8b0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.763+0000 7fca99e00640 1 -- 172.21.15.19:0/4255337641 shutdown_connections 2024-03-28T13:16:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.763+0000 7fca99e00640 1 --2- 172.21.15.19:0/4255337641 >> v2:172.21.15.188:3300/0 conn(0x7fca940cdcc0 0x7fca940cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.763+0000 7fca99e00640 1 --2- 172.21.15.19:0/4255337641 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9415c390 0x7fca9406b570 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.763+0000 7fca99e00640 1 -- 172.21.15.19:0/4255337641 >> 172.21.15.19:0/4255337641 conn(0x7fca9409aa20 msgr2=0x7fca940b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:08.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.763+0000 7fca99e00640 1 -- 172.21.15.19:0/4255337641 shutdown_connections 2024-03-28T13:16:08.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.763+0000 7fca99e00640 1 -- 172.21.15.19:0/4255337641 wait complete. 2024-03-28T13:16:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.765+0000 7fca99e00640 1 Processor -- start 2024-03-28T13:16:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.765+0000 7fca99e00640 1 -- start start 2024-03-28T13:16:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.766+0000 7fca99e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca9406bab0 0x7fca941531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.766+0000 7fca99e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca940cdcc0 0x7fca94157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.766+0000 7fca99e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9415c390 0x7fca941599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.766+0000 7fca99e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca940cf940 con 0x7fca940cdcc0 2024-03-28T13:16:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.766+0000 7fca99e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fca940cef70 con 0x7fca9406bab0 2024-03-28T13:16:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.766+0000 7fca99e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fca940cfac0 con 0x7fca9415c390 2024-03-28T13:16:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.766+0000 7fca93e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca940cdcc0 0x7fca94157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.766+0000 7fca93e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca940cdcc0 0x7fca94157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40888/0 (socket says 172.21.15.19:40888) 2024-03-28T13:16:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.767+0000 7fca93e00640 1 -- 172.21.15.19:0/2644873202 learned_addr learned my addr 172.21.15.19:0/2644873202 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.767+0000 7fca98a00640 1 --2- 172.21.15.19:0/2644873202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca9406bab0 0x7fca941531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.767+0000 7fca99400640 1 --2- 172.21.15.19:0/2644873202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9415c390 0x7fca941599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:08.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.767+0000 7fca93e00640 1 -- 172.21.15.19:0/2644873202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9415c390 msgr2=0x7fca941599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:08.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.767+0000 7fca93e00640 1 --2- 172.21.15.19:0/2644873202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9415c390 0x7fca941599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.767+0000 7fca93e00640 1 -- 172.21.15.19:0/2644873202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca9406bab0 msgr2=0x7fca941531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:08.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.767+0000 7fca93e00640 1 --2- 172.21.15.19:0/2644873202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca9406bab0 0x7fca941531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:08.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.767+0000 7fca93e00640 1 -- 172.21.15.19:0/2644873202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca940c7640 con 0x7fca940cdcc0 2024-03-28T13:16:08.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.767+0000 7fca98a00640 1 --2- 172.21.15.19:0/2644873202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca9406bab0 0x7fca941531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:16:08.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.767+0000 7fca93e00640 1 --2- 172.21.15.19:0/2644873202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca940cdcc0 0x7fca94157e60 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fca8000c400 tx=0x7fca8000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:08.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.768+0000 7fca90c00640 1 -- 172.21.15.19:0/2644873202 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca800052d0 con 0x7fca940cdcc0 2024-03-28T13:16:08.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.768+0000 7fca90c00640 1 -- 172.21.15.19:0/2644873202 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca80005490 con 0x7fca940cdcc0 2024-03-28T13:16:08.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.768+0000 7fca99e00640 1 -- 172.21.15.19:0/2644873202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca940c6da0 con 0x7fca940cdcc0 2024-03-28T13:16:08.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.768+0000 7fca90c00640 1 -- 172.21.15.19:0/2644873202 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca80007260 con 0x7fca940cdcc0 2024-03-28T13:16:08.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.768+0000 7fca99e00640 1 -- 172.21.15.19:0/2644873202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca940c7360 con 0x7fca940cdcc0 2024-03-28T13:16:08.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.769+0000 7fca99e00640 1 -- 172.21.15.19:0/2644873202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca94075940 con 0x7fca940cdcc0 2024-03-28T13:16:08.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.770+0000 7fca90c00640 1 -- 172.21.15.19:0/2644873202 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fca80007400 con 0x7fca940cdcc0 2024-03-28T13:16:08.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.771+0000 7fca90c00640 1 --2- 172.21.15.19:0/2644873202 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca7406c530 0x7fca7406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:08.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.771+0000 7fca90c00640 1 -- 172.21.15.19:0/2644873202 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7fca8008ad80 con 0x7fca940cdcc0 2024-03-28T13:16:08.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.771+0000 7fca98a00640 1 --2- 172.21.15.19:0/2644873202 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca7406c530 0x7fca7406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:08.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.772+0000 7fca98a00640 1 --2- 172.21.15.19:0/2644873202 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca7406c530 0x7fca7406e9f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fca88036000 tx=0x7fca8803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:08.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:08.775+0000 7fca90c00640 1 -- 172.21.15.19:0/2644873202 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fca80059280 con 0x7fca940cdcc0 2024-03-28T13:16:09.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.089+0000 7fca99e00640 1 -- 172.21.15.19:0/2644873202 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca940ccc00 con 0x7fca940cdcc0 2024-03-28T13:16:09.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.092+0000 7fca90c00640 1 -- 172.21.15.19:0/2644873202 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16835 (secure 0 0 0) 0x7fca80058b80 con 0x7fca940cdcc0 2024-03-28T13:16:09.093 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:09.093 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:15:25.200670+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:14:29.424080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-28T13:14:30.494667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1d","mappings":[{"from":2,"to":7}]},{"pgid":"11.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/1189161682":"2024-03-29T13:14:45.007899+0000","172.21.15.19:6832/1189161682":"2024-03-29T13:14:45.007899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:16:09.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.095+0000 7fca99e00640 1 -- 172.21.15.19:0/2644873202 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca7406c530 msgr2=0x7fca7406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:09.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.095+0000 7fca99e00640 1 --2- 172.21.15.19:0/2644873202 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca7406c530 0x7fca7406e9f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fca88036000 tx=0x7fca8803c040 comp rx=0 tx=0).stop 2024-03-28T13:16:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.095+0000 7fca99e00640 1 -- 172.21.15.19:0/2644873202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca940cdcc0 msgr2=0x7fca94157e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.095+0000 7fca99e00640 1 --2- 172.21.15.19:0/2644873202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca940cdcc0 0x7fca94157e60 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fca8000c400 tx=0x7fca8000c8c0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.096+0000 7fca99e00640 1 -- 172.21.15.19:0/2644873202 shutdown_connections 2024-03-28T13:16:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.096+0000 7fca99e00640 1 --2- 172.21.15.19:0/2644873202 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fca7406c530 0x7fca7406e9f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.096+0000 7fca99e00640 1 --2- 172.21.15.19:0/2644873202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca9415c390 0x7fca941599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.096+0000 7fca99e00640 1 --2- 172.21.15.19:0/2644873202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca940cdcc0 0x7fca94157e60 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.096+0000 7fca99e00640 1 --2- 172.21.15.19:0/2644873202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca9406bab0 0x7fca941531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.096+0000 7fca99e00640 1 -- 172.21.15.19:0/2644873202 >> 172.21.15.19:0/2644873202 conn(0x7fca9409aa20 msgr2=0x7fca940d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.096+0000 7fca99e00640 1 -- 172.21.15.19:0/2644873202 shutdown_connections 2024-03-28T13:16:09.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.096+0000 7fca99e00640 1 -- 172.21.15.19:0/2644873202 wait complete. 2024-03-28T13:16:09.113 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-28T13:16:09.114 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:16:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.328+0000 7f3ed0600640 1 Processor -- start 2024-03-28T13:16:09.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.328+0000 7f3ed0600640 1 -- start start 2024-03-28T13:16:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.330+0000 7f3ed0600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3ec80d3130 0x7f3ec80cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.330+0000 7f3ed0600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ec80d53a0 con 0x7f3ec80d4f10 2024-03-28T13:16:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.330+0000 7f3ed0600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ec80d2530 con 0x7f3ec815e980 2024-03-28T13:16:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.330+0000 7f3ed0600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3ec80cf630 con 0x7f3ec80d3130 2024-03-28T13:16:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.330+0000 7f3ece600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3ec80d4f10 0x7f3ec80b39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59740/0 (socket says 172.21.15.19:59740) 2024-03-28T13:16:09.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.330+0000 7f3ece600640 1 -- 172.21.15.19:0/3874761703 learned_addr learned my addr 172.21.15.19:0/3874761703 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ecd200640 1 --2- 172.21.15.19:0/3874761703 >> v2:172.21.15.188:3300/0 conn(0x7f3ec80d3130 0x7f3ec80cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ec7e00640 1 -- 172.21.15.19:0/3874761703 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2407788155 0 0) 0x7f3ec80d53a0 con 0x7f3ec80d4f10 2024-03-28T13:16:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ec7e00640 1 -- 172.21.15.19:0/3874761703 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3eb0003520 con 0x7f3ec80d4f10 2024-03-28T13:16:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ec7e00640 1 -- 172.21.15.19:0/3874761703 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2835022078 0 0) 0x7f3eb0003520 con 0x7f3ec80d4f10 2024-03-28T13:16:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ec7e00640 1 -- 172.21.15.19:0/3874761703 >> v2:172.21.15.188:3300/0 conn(0x7f3ec80d3130 msgr2=0x7f3ec80cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ec7e00640 1 --2- 172.21.15.19:0/3874761703 >> v2:172.21.15.188:3300/0 conn(0x7f3ec80d3130 0x7f3ec80cefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ec7e00640 1 -- 172.21.15.19:0/3874761703 >> v1:172.21.15.188:6789/0 conn(0x7f3ec815e980 legacy=0x7f3ec80d2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ec7e00640 1 -- 172.21.15.19:0/3874761703 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec80cf910 con 0x7f3ec80d4f10 2024-03-28T13:16:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ec7e00640 1 -- 172.21.15.19:0/3874761703 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f3ec00019f0 con 0x7f3ec80d4f10 2024-03-28T13:16:09.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ec7e00640 1 -- 172.21.15.19:0/3874761703 >> v1:172.21.15.19:6789/0 conn(0x7f3ec80d4f10 legacy=0x7f3ec80b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ec7e00640 1 --2- 172.21.15.19:0/3874761703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb00039b0 0x7f3eb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ec7e00640 1 --2- 172.21.15.19:0/3874761703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb0004800 0x7f3eb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ec7e00640 1 --2- 172.21.15.19:0/3874761703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3eb00051c0 0x7f3eb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ec7e00640 1 -- 172.21.15.19:0/3874761703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec80cf630 con 0x7f3eb0004800 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ec7e00640 1 -- 172.21.15.19:0/3874761703 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb000c060 con 0x7f3eb00039b0 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.331+0000 7f3ec7e00640 1 -- 172.21.15.19:0/3874761703 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb000c260 con 0x7f3eb00051c0 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.332+0000 7f3ecd200640 1 --2- 172.21.15.19:0/3874761703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb0004800 0x7f3eb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.332+0000 7f3ecdc00640 1 --2- 172.21.15.19:0/3874761703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb00039b0 0x7f3eb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.332+0000 7f3ece600640 1 --2- 172.21.15.19:0/3874761703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3eb00051c0 0x7f3eb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.332+0000 7f3ecd200640 1 -- 172.21.15.19:0/3874761703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3eb00051c0 msgr2=0x7f3eb000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.332+0000 7f3ecd200640 1 --2- 172.21.15.19:0/3874761703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3eb00051c0 0x7f3eb000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.332+0000 7f3ecd200640 1 -- 172.21.15.19:0/3874761703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb00039b0 msgr2=0x7f3eb0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.332+0000 7f3ecd200640 1 --2- 172.21.15.19:0/3874761703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eb00039b0 0x7f3eb0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.332+0000 7f3ecd200640 1 -- 172.21.15.19:0/3874761703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb0003520 con 0x7f3eb0004800 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.332+0000 7f3ece600640 1 --2- 172.21.15.19:0/3874761703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3eb00051c0 0x7f3eb000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:09.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.332+0000 7f3ece600640 1 -- 172.21.15.19:0/3874761703 reap_dead start 2024-03-28T13:16:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.333+0000 7f3ecd200640 1 --2- 172.21.15.19:0/3874761703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb0004800 0x7f3eb0004c80 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009ac0 tx=0x7f3eb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e23b048f9a0973 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.334+0000 7f3ec7e00640 1 -- 172.21.15.19:0/3874761703 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4039070 con 0x7f3eb0004800 2024-03-28T13:16:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.334+0000 7f3ec7e00640 1 -- 172.21.15.19:0/3874761703 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb402c6d0 con 0x7f3eb0004800 2024-03-28T13:16:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.334+0000 7f3ec7e00640 1 -- 172.21.15.19:0/3874761703 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4009690 con 0x7f3eb0004800 2024-03-28T13:16:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.334+0000 7f3ed0600640 1 -- 172.21.15.19:0/3874761703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb0004800 msgr2=0x7f3eb0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.335+0000 7f3ed0600640 1 --2- 172.21.15.19:0/3874761703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb0004800 0x7f3eb0004c80 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009ac0 tx=0x7f3eb0005830 comp rx=0 tx=0).stop 2024-03-28T13:16:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.335+0000 7f3ed0600640 1 -- 172.21.15.19:0/3874761703 shutdown_connections 2024-03-28T13:16:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.335+0000 7f3ed0600640 1 --2- 172.21.15.19:0/3874761703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3eb0004800 0x7f3eb0004c80 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.335+0000 7f3ed0600640 1 -- 172.21.15.19:0/3874761703 >> 172.21.15.19:0/3874761703 conn(0x7f3ec808bfb0 msgr2=0x7f3ec80d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.335+0000 7f3ed0600640 1 -- 172.21.15.19:0/3874761703 shutdown_connections 2024-03-28T13:16:09.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.335+0000 7f3ed0600640 1 -- 172.21.15.19:0/3874761703 wait complete. 2024-03-28T13:16:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.337+0000 7f3ed0600640 1 Processor -- start 2024-03-28T13:16:09.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.337+0000 7f3ed0600640 1 -- start start 2024-03-28T13:16:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.338+0000 7f3ed0600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3eb0004800 0x7f3ec815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.338+0000 7f3ed0600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec815d310 0x7f3ec8160ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.338+0000 7f3ed0600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ec81611e0 0x7f3ec8161660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.338+0000 7f3ed0600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec80d01b0 con 0x7f3ec815d310 2024-03-28T13:16:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.338+0000 7f3ed0600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec80cfed0 con 0x7f3ec81611e0 2024-03-28T13:16:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.338+0000 7f3ed0600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec80cefe0 con 0x7f3eb0004800 2024-03-28T13:16:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.338+0000 7f3ecd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec815d310 0x7f3ec8160ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.338+0000 7f3ecd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec815d310 0x7f3ec8160ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40908/0 (socket says 172.21.15.19:40908) 2024-03-28T13:16:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.338+0000 7f3ecd200640 1 -- 172.21.15.19:0/4250832364 learned_addr learned my addr 172.21.15.19:0/4250832364 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.338+0000 7f3ecdc00640 1 --2- 172.21.15.19:0/4250832364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3eb0004800 0x7f3ec815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:09.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.338+0000 7f3ece600640 1 --2- 172.21.15.19:0/4250832364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ec81611e0 0x7f3ec8161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:09.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.339+0000 7f3ecd200640 1 -- 172.21.15.19:0/4250832364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3eb0004800 msgr2=0x7f3ec815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.339+0000 7f3ecd200640 1 --2- 172.21.15.19:0/4250832364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3eb0004800 0x7f3ec815cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.339+0000 7f3ecd200640 1 -- 172.21.15.19:0/4250832364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ec81611e0 msgr2=0x7f3ec8161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.339+0000 7f3ecd200640 1 --2- 172.21.15.19:0/4250832364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ec81611e0 0x7f3ec8161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.339+0000 7f3ecd200640 1 -- 172.21.15.19:0/4250832364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec80cce80 con 0x7f3ec815d310 2024-03-28T13:16:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.339+0000 7f3ecd200640 1 --2- 172.21.15.19:0/4250832364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec815d310 0x7f3ec8160ca0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4037040 tx=0x7f3eb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.339+0000 7f3ec5600640 1 -- 172.21.15.19:0/4250832364 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4039050 con 0x7f3ec815d310 2024-03-28T13:16:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.339+0000 7f3ec5600640 1 -- 172.21.15.19:0/4250832364 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb4045070 con 0x7f3ec815d310 2024-03-28T13:16:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.339+0000 7f3ec5600640 1 -- 172.21.15.19:0/4250832364 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb40044e0 con 0x7f3ec815d310 2024-03-28T13:16:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.339+0000 7f3ed0600640 1 -- 172.21.15.19:0/4250832364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec80cc0c0 con 0x7f3ec815d310 2024-03-28T13:16:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.340+0000 7f3ed0600640 1 -- 172.21.15.19:0/4250832364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec80cc680 con 0x7f3ec815d310 2024-03-28T13:16:09.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.341+0000 7f3ed0600640 1 -- 172.21.15.19:0/4250832364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ec80685c0 con 0x7f3ec815d310 2024-03-28T13:16:09.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.342+0000 7f3ec5600640 1 -- 172.21.15.19:0/4250832364 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3eb4003a50 con 0x7f3ec815d310 2024-03-28T13:16:09.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.343+0000 7f3ec5600640 1 --2- 172.21.15.19:0/4250832364 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ea806c530 0x7f3ea806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:09.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.343+0000 7f3ec5600640 1 -- 172.21.15.19:0/4250832364 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f3eb40ad990 con 0x7f3ec815d310 2024-03-28T13:16:09.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.344+0000 7f3ecdc00640 1 --2- 172.21.15.19:0/4250832364 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ea806c530 0x7f3ea806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.345+0000 7f3ecdc00640 1 --2- 172.21.15.19:0/4250832364 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ea806c530 0x7f3ea806e9f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc007fd0 tx=0x7f3ebc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:09.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.348+0000 7f3ec5600640 1 -- 172.21.15.19:0/4250832364 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3eb407bf30 con 0x7f3ec815d310 2024-03-28T13:16:09.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.590+0000 7f3ed0600640 1 -- 172.21.15.19:0/4250832364 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ec80d2800 con 0x7f3ec815d310 2024-03-28T13:16:09.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.591+0000 7f3ec5600640 1 -- 172.21.15.19:0/4250832364 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5188 (secure 0 0 0) 0x7f3eb407b830 con 0x7f3ec815d310 2024-03-28T13:16:09.591 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:09.591 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/144726418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":144726418},{"type":"v1","addr":"172.21.15.188:6836","nonce":144726418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5363,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2642581607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2642581607},{"type":"v1","addr":"172.21.15.19:6835","nonce":2642581607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1721039756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1721039756},{"type":"v1","addr":"172.21.15.188:6837","nonce":1721039756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:14:31.526577+0000","modified":"2024-03-28T13:15:32.038485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.19:6834/2130552083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2130552083},{"type":"v1","addr":"172.21.15.19:6834","nonce":2130552083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6708,"qdb_cluster":[6708]},"id":5}]} 2024-03-28T13:16:09.591 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 53 2024-03-28T13:16:09.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.593+0000 7f3ed0600640 1 -- 172.21.15.19:0/4250832364 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ea806c530 msgr2=0x7f3ea806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:09.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.593+0000 7f3ed0600640 1 --2- 172.21.15.19:0/4250832364 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ea806c530 0x7f3ea806e9f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc007fd0 tx=0x7f3ebc009040 comp rx=0 tx=0).stop 2024-03-28T13:16:09.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.593+0000 7f3ed0600640 1 -- 172.21.15.19:0/4250832364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec815d310 msgr2=0x7f3ec8160ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:09.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.593+0000 7f3ed0600640 1 --2- 172.21.15.19:0/4250832364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec815d310 0x7f3ec8160ca0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4037040 tx=0x7f3eb4004730 comp rx=0 tx=0).stop 2024-03-28T13:16:09.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.593+0000 7f3ed0600640 1 -- 172.21.15.19:0/4250832364 shutdown_connections 2024-03-28T13:16:09.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.593+0000 7f3ed0600640 1 --2- 172.21.15.19:0/4250832364 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ea806c530 0x7f3ea806e9f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.593+0000 7f3ed0600640 1 --2- 172.21.15.19:0/4250832364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ec81611e0 0x7f3ec8161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.593+0000 7f3ed0600640 1 --2- 172.21.15.19:0/4250832364 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ec815d310 0x7f3ec8160ca0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.593+0000 7f3ed0600640 1 --2- 172.21.15.19:0/4250832364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3eb0004800 0x7f3ec815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.593+0000 7f3ed0600640 1 -- 172.21.15.19:0/4250832364 >> 172.21.15.19:0/4250832364 conn(0x7f3ec808bfb0 msgr2=0x7f3ec80d0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:09.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.594+0000 7f3ed0600640 1 -- 172.21.15.19:0/4250832364 shutdown_connections 2024-03-28T13:16:09.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.594+0000 7f3ed0600640 1 -- 172.21.15.19:0/4250832364 wait complete. 2024-03-28T13:16:09.613 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:16:09.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.808+0000 7f27d9c00640 1 Processor -- start 2024-03-28T13:16:09.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.808+0000 7f27d9c00640 1 -- start start 2024-03-28T13:16:09.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.810+0000 7f27d9c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27d40d04c0 0x7f27d40700b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:09.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.810+0000 7f27d9c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f27d40705f0 0x7f27d40b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:09.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.810+0000 7f27d9c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27d40afea0 con 0x7f27d406c7f0 2024-03-28T13:16:09.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.810+0000 7f27d9c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f27d40b0180 con 0x7f27d40705f0 2024-03-28T13:16:09.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.810+0000 7f27d9c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f27d40b0460 con 0x7f27d40d04c0 2024-03-28T13:16:09.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.810+0000 7f27d9200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f27d40705f0 0x7f27d40b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:09.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.810+0000 7f27d3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f27d406c7f0 0x7f27d4072990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59744/0 (socket says 172.21.15.19:59744) 2024-03-28T13:16:09.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.810+0000 7f27d3e00640 1 -- 172.21.15.19:0/1854279903 learned_addr learned my addr 172.21.15.19:0/1854279903 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:09.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.811+0000 7f27d3400640 1 --2- 172.21.15.19:0/1854279903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27d40d04c0 0x7f27d40700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:09.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.811+0000 7f27d9200640 1 -- 172.21.15.19:0/1854279903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27d40d04c0 msgr2=0x7f27d40700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:09.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.811+0000 7f27d9200640 1 --2- 172.21.15.19:0/1854279903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27d40d04c0 0x7f27d40700b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.811+0000 7f27d9200640 1 -- 172.21.15.19:0/1854279903 >> v1:172.21.15.19:6789/0 conn(0x7f27d406c7f0 legacy=0x7f27d4072990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:09.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.811+0000 7f27d2a00640 1 -- 172.21.15.19:0/1854279903 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 837833414 0 0) 0x7f27d40afea0 con 0x7f27d406c7f0 2024-03-28T13:16:09.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.811+0000 7f27d9200640 1 -- 172.21.15.19:0/1854279903 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d406f8b0 con 0x7f27d40705f0 2024-03-28T13:16:09.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.812+0000 7f27d9200640 1 --2- 172.21.15.19:0/1854279903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d40705f0 0x7f27d40b3ac0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009d90 tx=0x7f27cc02bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a41e6c5b025185b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:09.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.813+0000 7f27d2a00640 1 -- 172.21.15.19:0/1854279903 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc039070 con 0x7f27d40705f0 2024-03-28T13:16:09.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.813+0000 7f27d2a00640 1 -- 172.21.15.19:0/1854279903 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27cc02cce0 con 0x7f27d40705f0 2024-03-28T13:16:09.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.813+0000 7f27d2a00640 1 -- 172.21.15.19:0/1854279903 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc034660 con 0x7f27d40705f0 2024-03-28T13:16:09.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.813+0000 7f27d9c00640 1 -- 172.21.15.19:0/1854279903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d40705f0 msgr2=0x7f27d40b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:09.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.813+0000 7f27d9c00640 1 --2- 172.21.15.19:0/1854279903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d40705f0 0x7f27d40b3ac0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009d90 tx=0x7f27cc02bf30 comp rx=0 tx=0).stop 2024-03-28T13:16:09.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.813+0000 7f27d9c00640 1 -- 172.21.15.19:0/1854279903 shutdown_connections 2024-03-28T13:16:09.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.813+0000 7f27d9c00640 1 --2- 172.21.15.19:0/1854279903 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d40705f0 0x7f27d40b3ac0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.813+0000 7f27d9c00640 1 --2- 172.21.15.19:0/1854279903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27d40d04c0 0x7f27d40700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.813+0000 7f27d9c00640 1 -- 172.21.15.19:0/1854279903 >> 172.21.15.19:0/1854279903 conn(0x7f27d4071e70 msgr2=0x7f27d4072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:09.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.814+0000 7f27d9c00640 1 -- 172.21.15.19:0/1854279903 shutdown_connections 2024-03-28T13:16:09.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.814+0000 7f27d9c00640 1 -- 172.21.15.19:0/1854279903 wait complete. 2024-03-28T13:16:09.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.815+0000 7f27d9c00640 1 Processor -- start 2024-03-28T13:16:09.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.815+0000 7f27d9c00640 1 -- start start 2024-03-28T13:16:09.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.816+0000 7f27d9c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27d406c7f0 0x7f27d40b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:09.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.816+0000 7f27d9c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d40705f0 0x7f27d4141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:09.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.816+0000 7f27d9c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27d40d04c0 0x7f27d4142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:09.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.816+0000 7f27d9c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27d406cd50 con 0x7f27d40705f0 2024-03-28T13:16:09.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.816+0000 7f27d9c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f27d406cbd0 con 0x7f27d406c7f0 2024-03-28T13:16:09.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.816+0000 7f27d9c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f27d406ced0 con 0x7f27d40d04c0 2024-03-28T13:16:09.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.817+0000 7f27d3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d40705f0 0x7f27d4141c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:09.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.817+0000 7f27d3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d40705f0 0x7f27d4141c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40938/0 (socket says 172.21.15.19:40938) 2024-03-28T13:16:09.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.817+0000 7f27d3400640 1 -- 172.21.15.19:0/2747219991 learned_addr learned my addr 172.21.15.19:0/2747219991 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:09.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.817+0000 7f27d3400640 1 -- 172.21.15.19:0/2747219991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27d40d04c0 msgr2=0x7f27d4142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:09.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.817+0000 7f27d3400640 1 --2- 172.21.15.19:0/2747219991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27d40d04c0 0x7f27d4142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.817+0000 7f27d3400640 1 -- 172.21.15.19:0/2747219991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27d406c7f0 msgr2=0x7f27d40b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:09.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.817+0000 7f27d3e00640 1 --2- 172.21.15.19:0/2747219991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27d406c7f0 0x7f27d40b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:09.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.817+0000 7f27d3400640 1 --2- 172.21.15.19:0/2747219991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27d406c7f0 0x7f27d40b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:09.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.817+0000 7f27d3400640 1 -- 172.21.15.19:0/2747219991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d40b4220 con 0x7f27d40705f0 2024-03-28T13:16:09.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.817+0000 7f27d3e00640 1 --2- 172.21.15.19:0/2747219991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27d406c7f0 0x7f27d40b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:09.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.817+0000 7f27d3400640 1 --2- 172.21.15.19:0/2747219991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d40705f0 0x7f27d4141c00 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f27c000d5d0 tx=0x7f27c000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:09.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.818+0000 7f27cbe00640 1 -- 172.21.15.19:0/2747219991 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0004240 con 0x7f27d40705f0 2024-03-28T13:16:09.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.818+0000 7f27cbe00640 1 -- 172.21.15.19:0/2747219991 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27c00043e0 con 0x7f27d40705f0 2024-03-28T13:16:09.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.818+0000 7f27cbe00640 1 -- 172.21.15.19:0/2747219991 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c0003830 con 0x7f27d40705f0 2024-03-28T13:16:09.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.818+0000 7f27d9c00640 1 -- 172.21.15.19:0/2747219991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d40b4500 con 0x7f27d40705f0 2024-03-28T13:16:09.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.818+0000 7f27d9c00640 1 -- 172.21.15.19:0/2747219991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d40b4ac0 con 0x7f27d40705f0 2024-03-28T13:16:09.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.819+0000 7f27d9c00640 1 -- 172.21.15.19:0/2747219991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d4075940 con 0x7f27d40705f0 2024-03-28T13:16:09.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.820+0000 7f27cbe00640 1 -- 172.21.15.19:0/2747219991 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f27c00039d0 con 0x7f27d40705f0 2024-03-28T13:16:09.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.821+0000 7f27cbe00640 1 --2- 172.21.15.19:0/2747219991 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f27b406c530 0x7f27b406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:09.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.821+0000 7f27cbe00640 1 -- 172.21.15.19:0/2747219991 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f27c008bb60 con 0x7f27d40705f0 2024-03-28T13:16:09.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.822+0000 7f27d3e00640 1 --2- 172.21.15.19:0/2747219991 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f27b406c530 0x7f27b406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:09.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.823+0000 7f27d3e00640 1 --2- 172.21.15.19:0/2747219991 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f27b406c530 0x7f27b406e9f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f27c4000f60 tx=0x7f27c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:09.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:09.825+0000 7f27cbe00640 1 -- 172.21.15.19:0/2747219991 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f27c0059a00 con 0x7f27d40705f0 2024-03-28T13:16:10.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.068+0000 7f27d9c00640 1 -- 172.21.15.19:0/2747219991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27d4067860 con 0x7f27d40705f0 2024-03-28T13:16:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.069+0000 7f27cbe00640 1 -- 172.21.15.19:0/2747219991 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5188 (secure 0 0 0) 0x7f27c005dd00 con 0x7f27d40705f0 2024-03-28T13:16:10.069 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:10.069 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":53,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/144726418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":144726418},{"type":"v1","addr":"172.21.15.188:6836","nonce":144726418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5363,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2642581607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2642581607},{"type":"v1","addr":"172.21.15.19:6835","nonce":2642581607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1721039756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1721039756},{"type":"v1","addr":"172.21.15.188:6837","nonce":1721039756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:14:31.526577+0000","modified":"2024-03-28T13:15:32.038485+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"c","rank":0,"incarnation":48,"state":"up:active","state_seq":15,"addr":"172.21.15.19:6834/2130552083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2130552083},{"type":"v1","addr":"172.21.15.19:6834","nonce":2130552083}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6708,"qdb_cluster":[6708]},"id":5}]} 2024-03-28T13:16:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 53 2024-03-28T13:16:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.072+0000 7f27d9c00640 1 -- 172.21.15.19:0/2747219991 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f27b406c530 msgr2=0x7f27b406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.072+0000 7f27d9c00640 1 --2- 172.21.15.19:0/2747219991 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f27b406c530 0x7f27b406e9f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f27c4000f60 tx=0x7f27c4009040 comp rx=0 tx=0).stop 2024-03-28T13:16:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.072+0000 7f27d9c00640 1 -- 172.21.15.19:0/2747219991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d40705f0 msgr2=0x7f27d4141c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.072+0000 7f27d9c00640 1 --2- 172.21.15.19:0/2747219991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d40705f0 0x7f27d4141c00 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f27c000d5d0 tx=0x7f27c000da90 comp rx=0 tx=0).stop 2024-03-28T13:16:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.072+0000 7f27d9c00640 1 -- 172.21.15.19:0/2747219991 shutdown_connections 2024-03-28T13:16:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.072+0000 7f27d9c00640 1 --2- 172.21.15.19:0/2747219991 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f27b406c530 0x7f27b406e9f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.072+0000 7f27d9c00640 1 --2- 172.21.15.19:0/2747219991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27d40d04c0 0x7f27d4142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.072+0000 7f27d9c00640 1 --2- 172.21.15.19:0/2747219991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d40705f0 0x7f27d4141c00 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.072+0000 7f27d9c00640 1 --2- 172.21.15.19:0/2747219991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27d406c7f0 0x7f27d40b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.073+0000 7f27d9c00640 1 -- 172.21.15.19:0/2747219991 >> 172.21.15.19:0/2747219991 conn(0x7f27d4071e70 msgr2=0x7f27d40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.073+0000 7f27d9c00640 1 -- 172.21.15.19:0/2747219991 shutdown_connections 2024-03-28T13:16:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.073+0000 7f27d9c00640 1 -- 172.21.15.19:0/2747219991 wait complete. 2024-03-28T13:16:10.091 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:16:10.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.289+0000 7f2546600640 1 Processor -- start 2024-03-28T13:16:10.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.290+0000 7f2546600640 1 -- start start 2024-03-28T13:16:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.291+0000 7f2546600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f254015e4a0 0x7f25400b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.291+0000 7f2546600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f25400d2310 0x7f25400ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.291+0000 7f2546600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f254015e880 con 0x7f25400cf530 2024-03-28T13:16:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.291+0000 7f2546600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f25400cf9a0 con 0x7f25400d2310 2024-03-28T13:16:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.291+0000 7f2546600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f25400d34e0 con 0x7f254015e4a0 2024-03-28T13:16:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.292+0000 7f253fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f25400d2310 0x7f25400ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.292+0000 7f253fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f25400d2310 0x7f25400ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40952/0 (socket says 172.21.15.19:40952) 2024-03-28T13:16:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.292+0000 7f253fe00640 1 -- 172.21.15.19:0/3179502569 learned_addr learned my addr 172.21.15.19:0/3179502569 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.292+0000 7f2545200640 1 --2- 172.21.15.19:0/3179502569 >> v2:172.21.15.188:3300/0 conn(0x7f254015e4a0 0x7f25400b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:10.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.292+0000 7f253fe00640 1 -- 172.21.15.19:0/3179502569 >> v2:172.21.15.188:3300/0 conn(0x7f254015e4a0 msgr2=0x7f25400b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:10.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.292+0000 7f253fe00640 1 --2- 172.21.15.19:0/3179502569 >> v2:172.21.15.188:3300/0 conn(0x7f254015e4a0 0x7f25400b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.292+0000 7f253fe00640 1 -- 172.21.15.19:0/3179502569 >> v1:172.21.15.188:6789/0 conn(0x7f25400cf530 legacy=0x7f25400d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:10.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.292+0000 7f253fe00640 1 -- 172.21.15.19:0/3179502569 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25400d28d0 con 0x7f25400d2310 2024-03-28T13:16:10.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.293+0000 7f253fe00640 1 --2- 172.21.15.19:0/3179502569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25400d2310 0x7f25400ceff0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f25400d3660 tx=0x7f252c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d453397ff83d41 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:10.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.294+0000 7f253f400640 1 -- 172.21.15.19:0/3179502569 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c02e070 con 0x7f25400d2310 2024-03-28T13:16:10.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.294+0000 7f253f400640 1 -- 172.21.15.19:0/3179502569 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f252c037070 con 0x7f25400d2310 2024-03-28T13:16:10.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.294+0000 7f253f400640 1 -- 172.21.15.19:0/3179502569 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c031a00 con 0x7f25400d2310 2024-03-28T13:16:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.294+0000 7f2546600640 1 -- 172.21.15.19:0/3179502569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25400d2310 msgr2=0x7f25400ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.294+0000 7f2546600640 1 --2- 172.21.15.19:0/3179502569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25400d2310 0x7f25400ceff0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f25400d3660 tx=0x7f252c02b880 comp rx=0 tx=0).stop 2024-03-28T13:16:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.295+0000 7f2546600640 1 -- 172.21.15.19:0/3179502569 shutdown_connections 2024-03-28T13:16:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.295+0000 7f2546600640 1 --2- 172.21.15.19:0/3179502569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25400d2310 0x7f25400ceff0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.295+0000 7f2546600640 1 --2- 172.21.15.19:0/3179502569 >> v2:172.21.15.188:3300/0 conn(0x7f254015e4a0 0x7f25400b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.295+0000 7f2546600640 1 -- 172.21.15.19:0/3179502569 >> 172.21.15.19:0/3179502569 conn(0x7f25400676d0 msgr2=0x7f2540067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.295+0000 7f2546600640 1 -- 172.21.15.19:0/3179502569 shutdown_connections 2024-03-28T13:16:10.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.295+0000 7f2546600640 1 -- 172.21.15.19:0/3179502569 wait complete. 2024-03-28T13:16:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.296+0000 7f2546600640 1 Processor -- start 2024-03-28T13:16:10.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.296+0000 7f2546600640 1 -- start start 2024-03-28T13:16:10.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.297+0000 7f2546600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25400cf530 0x7f25400c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:10.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.297+0000 7f2546600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25400d2310 0x7f25400d3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:10.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.297+0000 7f2546600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f254015e4a0 0x7f25400d4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:10.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.297+0000 7f2546600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f25400d3170 con 0x7f25400d2310 2024-03-28T13:16:10.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.297+0000 7f2546600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f25400d2e90 con 0x7f25400cf530 2024-03-28T13:16:10.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.297+0000 7f2546600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f25400d3970 con 0x7f254015e4a0 2024-03-28T13:16:10.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.298+0000 7f253fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25400d2310 0x7f25400d3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:10.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.298+0000 7f253fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25400d2310 0x7f25400d3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40956/0 (socket says 172.21.15.19:40956) 2024-03-28T13:16:10.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.298+0000 7f253fe00640 1 -- 172.21.15.19:0/2199749728 learned_addr learned my addr 172.21.15.19:0/2199749728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:10.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.298+0000 7f253fe00640 1 -- 172.21.15.19:0/2199749728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f254015e4a0 msgr2=0x7f25400d4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:10.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.298+0000 7f253fe00640 1 --2- 172.21.15.19:0/2199749728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f254015e4a0 0x7f25400d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.298+0000 7f253fe00640 1 -- 172.21.15.19:0/2199749728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25400cf530 msgr2=0x7f25400c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:10.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.298+0000 7f2545200640 1 --2- 172.21.15.19:0/2199749728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25400cf530 0x7f25400c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:10.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.298+0000 7f253fe00640 1 --2- 172.21.15.19:0/2199749728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25400cf530 0x7f25400c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.298+0000 7f253fe00640 1 -- 172.21.15.19:0/2199749728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25400c7300 con 0x7f25400d2310 2024-03-28T13:16:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.298+0000 7f253fe00640 1 --2- 172.21.15.19:0/2199749728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25400d2310 0x7f25400d3c10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f252c000c00 tx=0x7f252c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.299+0000 7f253cc00640 1 -- 172.21.15.19:0/2199749728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c032880 con 0x7f25400d2310 2024-03-28T13:16:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.299+0000 7f253cc00640 1 -- 172.21.15.19:0/2199749728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f252c032a20 con 0x7f25400d2310 2024-03-28T13:16:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.299+0000 7f2546600640 1 -- 172.21.15.19:0/2199749728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254015ebb0 con 0x7f25400d2310 2024-03-28T13:16:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.299+0000 7f253cc00640 1 -- 172.21.15.19:0/2199749728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c032e10 con 0x7f25400d2310 2024-03-28T13:16:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.299+0000 7f2546600640 1 -- 172.21.15.19:0/2199749728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254015f130 con 0x7f25400d2310 2024-03-28T13:16:10.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.300+0000 7f2546600640 1 -- 172.21.15.19:0/2199749728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2540075940 con 0x7f25400d2310 2024-03-28T13:16:10.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.301+0000 7f253cc00640 1 -- 172.21.15.19:0/2199749728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f252c04c060 con 0x7f25400d2310 2024-03-28T13:16:10.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.303+0000 7f253cc00640 1 --2- 172.21.15.19:0/2199749728 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f252006c560 0x7f252006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:10.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.303+0000 7f253cc00640 1 -- 172.21.15.19:0/2199749728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f252c0ad270 con 0x7f25400d2310 2024-03-28T13:16:10.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.303+0000 7f2545200640 1 --2- 172.21.15.19:0/2199749728 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f252006c560 0x7f252006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:10.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.305+0000 7f2545200640 1 --2- 172.21.15.19:0/2199749728 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f252006c560 0x7f252006ea20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2534006fd0 tx=0x7f2534008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:10.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.307+0000 7f253cc00640 1 -- 172.21.15.19:0/2199749728 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f25400c7300 con 0x7f25400d2310 2024-03-28T13:16:10.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.503+0000 7f2546600640 1 -- 172.21.15.19:0/2199749728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25400b4870 con 0x7f25400d2310 2024-03-28T13:16:10.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.504+0000 7f253cc00640 1 -- 172.21.15.19:0/2199749728 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+16835 (secure 0 0 0) 0x7f252c040070 con 0x7f25400d2310 2024-03-28T13:16:10.505 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:10.505 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:15:25.200670+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:14:29.424080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-28T13:14:30.494667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1d","mappings":[{"from":2,"to":7}]},{"pgid":"11.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/1189161682":"2024-03-29T13:14:45.007899+0000","172.21.15.19:6832/1189161682":"2024-03-29T13:14:45.007899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:16:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.507+0000 7f2546600640 1 -- 172.21.15.19:0/2199749728 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f252006c560 msgr2=0x7f252006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.507+0000 7f2546600640 1 --2- 172.21.15.19:0/2199749728 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f252006c560 0x7f252006ea20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2534006fd0 tx=0x7f2534008040 comp rx=0 tx=0).stop 2024-03-28T13:16:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.507+0000 7f2546600640 1 -- 172.21.15.19:0/2199749728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25400d2310 msgr2=0x7f25400d3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.507+0000 7f2546600640 1 --2- 172.21.15.19:0/2199749728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25400d2310 0x7f25400d3c10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f252c000c00 tx=0x7f252c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.507+0000 7f2546600640 1 -- 172.21.15.19:0/2199749728 shutdown_connections 2024-03-28T13:16:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.507+0000 7f2546600640 1 --2- 172.21.15.19:0/2199749728 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f252006c560 0x7f252006ea20 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.507+0000 7f2546600640 1 --2- 172.21.15.19:0/2199749728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f254015e4a0 0x7f25400d4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.507+0000 7f2546600640 1 --2- 172.21.15.19:0/2199749728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f25400d2310 0x7f25400d3c10 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.507+0000 7f2546600640 1 --2- 172.21.15.19:0/2199749728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25400cf530 0x7f25400c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.507+0000 7f2546600640 1 -- 172.21.15.19:0/2199749728 >> 172.21.15.19:0/2199749728 conn(0x7f25400676d0 msgr2=0x7f254015cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.507+0000 7f2546600640 1 -- 172.21.15.19:0/2199749728 shutdown_connections 2024-03-28T13:16:10.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.507+0000 7f2546600640 1 -- 172.21.15.19:0/2199749728 wait complete. 2024-03-28T13:16:10.526 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T13:16:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.736+0000 7f8692e00640 1 Processor -- start 2024-03-28T13:16:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.736+0000 7f8692e00640 1 -- start start 2024-03-28T13:16:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.737+0000 7f8692e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f868c0d1bf0 0x7f868c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.737+0000 7f8692e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f868c070170 0x7f868c06b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.737+0000 7f8692e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f868c0705f0 0x7f868c06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.737+0000 7f8692e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f868c0d1fd0 con 0x7f868c070170 2024-03-28T13:16:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.737+0000 7f8692e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f868c15eb80 con 0x7f868c0d1bf0 2024-03-28T13:16:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.737+0000 7f8692e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f868c15edc0 con 0x7f868c0705f0 2024-03-28T13:16:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.738+0000 7f8683e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f868c070170 0x7f868c06b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.738+0000 7f8683e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f868c070170 0x7f868c06b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40970/0 (socket says 172.21.15.19:40970) 2024-03-28T13:16:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.738+0000 7f8683e00640 1 -- 172.21.15.19:0/1281283315 learned_addr learned my addr 172.21.15.19:0/1281283315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.738+0000 7f8690e00640 1 --2- 172.21.15.19:0/1281283315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f868c0705f0 0x7f868c06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.738+0000 7f868be00640 1 --2- 172.21.15.19:0/1281283315 >> v2:172.21.15.188:3300/0 conn(0x7f868c0d1bf0 0x7f868c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.738+0000 7f8683e00640 1 -- 172.21.15.19:0/1281283315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f868c0705f0 msgr2=0x7f868c06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.738+0000 7f8683e00640 1 --2- 172.21.15.19:0/1281283315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f868c0705f0 0x7f868c06bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.738+0000 7f8683e00640 1 -- 172.21.15.19:0/1281283315 >> v2:172.21.15.188:3300/0 conn(0x7f868c0d1bf0 msgr2=0x7f868c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.738+0000 7f8683e00640 1 --2- 172.21.15.19:0/1281283315 >> v2:172.21.15.188:3300/0 conn(0x7f868c0d1bf0 0x7f868c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.738+0000 7f8683e00640 1 -- 172.21.15.19:0/1281283315 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c0b5e60 con 0x7f868c070170 2024-03-28T13:16:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.739+0000 7f8683e00640 1 --2- 172.21.15.19:0/1281283315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f868c070170 0x7f868c06b570 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f868c0b3d50 tx=0x7f867402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0be51a1baf4635 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.740+0000 7f868b400640 1 -- 172.21.15.19:0/1281283315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674035020 con 0x7f868c070170 2024-03-28T13:16:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.740+0000 7f868b400640 1 -- 172.21.15.19:0/1281283315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8674031870 con 0x7f868c070170 2024-03-28T13:16:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.740+0000 7f868b400640 1 -- 172.21.15.19:0/1281283315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674031c40 con 0x7f868c070170 2024-03-28T13:16:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.740+0000 7f8692e00640 1 -- 172.21.15.19:0/1281283315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f868c070170 msgr2=0x7f868c06b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.740+0000 7f8692e00640 1 --2- 172.21.15.19:0/1281283315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f868c070170 0x7f868c06b570 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f868c0b3d50 tx=0x7f867402b910 comp rx=0 tx=0).stop 2024-03-28T13:16:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.740+0000 7f8692e00640 1 -- 172.21.15.19:0/1281283315 shutdown_connections 2024-03-28T13:16:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.741+0000 7f8692e00640 1 --2- 172.21.15.19:0/1281283315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f868c0705f0 0x7f868c06bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.741+0000 7f8692e00640 1 --2- 172.21.15.19:0/1281283315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f868c070170 0x7f868c06b570 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.741+0000 7f8692e00640 1 --2- 172.21.15.19:0/1281283315 >> v2:172.21.15.188:3300/0 conn(0x7f868c0d1bf0 0x7f868c15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.741+0000 7f8692e00640 1 -- 172.21.15.19:0/1281283315 >> 172.21.15.19:0/1281283315 conn(0x7f868c0b6560 msgr2=0x7f868c0b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.741+0000 7f8692e00640 1 -- 172.21.15.19:0/1281283315 shutdown_connections 2024-03-28T13:16:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.741+0000 7f8692e00640 1 -- 172.21.15.19:0/1281283315 wait complete. 2024-03-28T13:16:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.742+0000 7f8692e00640 1 Processor -- start 2024-03-28T13:16:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.742+0000 7f8692e00640 1 -- start start 2024-03-28T13:16:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.743+0000 7f8692e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f868c070170 0x7f868c158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.743+0000 7f8692e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f868c0705f0 0x7f868c1f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.743+0000 7f8692e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f868c0d1bf0 0x7f868c146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.743+0000 7f8692e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f868c15fae0 con 0x7f868c0705f0 2024-03-28T13:16:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.743+0000 7f8692e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f868c15f080 con 0x7f868c070170 2024-03-28T13:16:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.743+0000 7f8692e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f868c0b5b10 con 0x7f868c0d1bf0 2024-03-28T13:16:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.744+0000 7f8683e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f868c0705f0 0x7f868c1f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.744+0000 7f8683e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f868c0705f0 0x7f868c1f83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40982/0 (socket says 172.21.15.19:40982) 2024-03-28T13:16:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.744+0000 7f8683e00640 1 -- 172.21.15.19:0/2739477555 learned_addr learned my addr 172.21.15.19:0/2739477555 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:10.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.744+0000 7f868be00640 1 --2- 172.21.15.19:0/2739477555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f868c070170 0x7f868c158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.744+0000 7f8690e00640 1 --2- 172.21.15.19:0/2739477555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f868c0d1bf0 0x7f868c146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.744+0000 7f8683e00640 1 -- 172.21.15.19:0/2739477555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f868c0d1bf0 msgr2=0x7f868c146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.744+0000 7f8683e00640 1 --2- 172.21.15.19:0/2739477555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f868c0d1bf0 0x7f868c146b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.744+0000 7f8683e00640 1 -- 172.21.15.19:0/2739477555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f868c070170 msgr2=0x7f868c158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.744+0000 7f8683e00640 1 --2- 172.21.15.19:0/2739477555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f868c070170 0x7f868c158580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.744+0000 7f8683e00640 1 -- 172.21.15.19:0/2739477555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868c158c10 con 0x7f868c0705f0 2024-03-28T13:16:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.744+0000 7f868be00640 1 --2- 172.21.15.19:0/2739477555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f868c070170 0x7f868c158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:10.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.745+0000 7f8683e00640 1 --2- 172.21.15.19:0/2739477555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f868c0705f0 0x7f868c1f83e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f867402b950 tx=0x7f86740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.745+0000 7f8688c00640 1 -- 172.21.15.19:0/2739477555 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674040070 con 0x7f868c0705f0 2024-03-28T13:16:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.745+0000 7f8688c00640 1 -- 172.21.15.19:0/2739477555 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8674030d00 con 0x7f868c0705f0 2024-03-28T13:16:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.745+0000 7f8688c00640 1 -- 172.21.15.19:0/2739477555 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674035040 con 0x7f868c0705f0 2024-03-28T13:16:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.745+0000 7f8692e00640 1 -- 172.21.15.19:0/2739477555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868c158ef0 con 0x7f868c0705f0 2024-03-28T13:16:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.745+0000 7f8692e00640 1 -- 172.21.15.19:0/2739477555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f868c1594b0 con 0x7f868c0705f0 2024-03-28T13:16:10.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.746+0000 7f8692e00640 1 -- 172.21.15.19:0/2739477555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f868c0685c0 con 0x7f868c0705f0 2024-03-28T13:16:10.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.747+0000 7f8688c00640 1 -- 172.21.15.19:0/2739477555 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8674038020 con 0x7f868c0705f0 2024-03-28T13:16:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.748+0000 7f8688c00640 1 --2- 172.21.15.19:0/2739477555 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f866806c530 0x7f866806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.748+0000 7f8688c00640 1 -- 172.21.15.19:0/2739477555 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5853+0+0 (secure 0 0 0) 0x7f86740ad530 con 0x7f868c0705f0 2024-03-28T13:16:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.749+0000 7f868be00640 1 --2- 172.21.15.19:0/2739477555 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f866806c530 0x7f866806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:10.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.750+0000 7f868be00640 1 --2- 172.21.15.19:0/2739477555 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f866806c530 0x7f866806e9f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f867c006fd0 tx=0x7f867c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:10.757+0000 7f8688c00640 1 -- 172.21.15.19:0/2739477555 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f867407bac0 con 0x7f868c0705f0 2024-03-28T13:16:11.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.004+0000 7f8692e00640 1 -- 172.21.15.19:0/2739477555 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f868c0afad0 con 0x7f868c0705f0 2024-03-28T13:16:11.004 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:16:11.004+0000 7f6b74400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T13:16:11.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.007+0000 7f8688c00640 1 -- 172.21.15.19:0/2739477555 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v54) ==== 161+0+0 (secure 0 0 0) 0x7f867407b3c0 con 0x7f868c0705f0 2024-03-28T13:16:11.007 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T13:16:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.011+0000 7f8692e00640 1 -- 172.21.15.19:0/2739477555 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f866806c530 msgr2=0x7f866806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.011+0000 7f8692e00640 1 --2- 172.21.15.19:0/2739477555 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f866806c530 0x7f866806e9f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f867c006fd0 tx=0x7f867c008040 comp rx=0 tx=0).stop 2024-03-28T13:16:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.011+0000 7f8692e00640 1 -- 172.21.15.19:0/2739477555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f868c0705f0 msgr2=0x7f868c1f83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:11.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.011+0000 7f8692e00640 1 --2- 172.21.15.19:0/2739477555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f868c0705f0 0x7f868c1f83e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f867402b950 tx=0x7f86740046c0 comp rx=0 tx=0).stop 2024-03-28T13:16:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.011+0000 7f8692e00640 1 -- 172.21.15.19:0/2739477555 shutdown_connections 2024-03-28T13:16:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.011+0000 7f8692e00640 1 --2- 172.21.15.19:0/2739477555 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f866806c530 0x7f866806e9f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.011+0000 7f8692e00640 1 --2- 172.21.15.19:0/2739477555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f868c0d1bf0 0x7f868c146b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.011+0000 7f8692e00640 1 --2- 172.21.15.19:0/2739477555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f868c0705f0 0x7f868c1f83e0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.011+0000 7f8692e00640 1 --2- 172.21.15.19:0/2739477555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f868c070170 0x7f868c158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.011+0000 7f8692e00640 1 -- 172.21.15.19:0/2739477555 >> 172.21.15.19:0/2739477555 conn(0x7f868c0b6560 msgr2=0x7f868c160850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.011+0000 7f8692e00640 1 -- 172.21.15.19:0/2739477555 shutdown_connections 2024-03-28T13:16:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.012+0000 7f8692e00640 1 -- 172.21.15.19:0/2739477555 wait complete. 2024-03-28T13:16:11.026 INFO:tasks.ceph.mds.c.smithi019.stderr: -4471> 2024-03-28T13:14:51.034+0000 7fe33d200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:16:11.033 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-28T13:16:11.093 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-03-28T13:16:11.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.231+0000 7f36c0c00640 1 Processor -- start 2024-03-28T13:16:11.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.232+0000 7f36c0c00640 1 -- start start 2024-03-28T13:16:11.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.233+0000 7f36c0c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f36bc15e4c0 0x7f36bc0b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:11.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.234+0000 7f36c0c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36bc0d2710 con 0x7f36bc0d37b0 2024-03-28T13:16:11.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.234+0000 7f36c0c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36bc0d29f0 con 0x7f36bc0b69c0 2024-03-28T13:16:11.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.234+0000 7f36c0c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f36bc15e8a0 con 0x7f36bc15e4c0 2024-03-28T13:16:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.234+0000 7f36baa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f36bc0d37b0 0x7f36bc0b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59764/0 (socket says 172.21.15.19:59764) 2024-03-28T13:16:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.234+0000 7f36baa00640 1 -- 172.21.15.19:0/3057904078 learned_addr learned my addr 172.21.15.19:0/3057904078 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.234+0000 7f36bb400640 1 --2- 172.21.15.19:0/3057904078 >> v2:172.21.15.188:3300/0 conn(0x7f36bc15e4c0 0x7f36bc0b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.235+0000 7f36ba000640 1 -- 172.21.15.19:0/3057904078 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3321539638 0 0) 0x7f36bc0d2710 con 0x7f36bc0d37b0 2024-03-28T13:16:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.235+0000 7f36ba000640 1 -- 172.21.15.19:0/3057904078 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36a0003520 con 0x7f36bc0d37b0 2024-03-28T13:16:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.235+0000 7f36bb400640 1 -- 172.21.15.19:0/3057904078 >> v1:172.21.15.188:6789/0 conn(0x7f36bc0b69c0 legacy=0x7f36bc0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.235+0000 7f36bb400640 1 -- 172.21.15.19:0/3057904078 >> v1:172.21.15.19:6789/0 conn(0x7f36bc0d37b0 legacy=0x7f36bc0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:11.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.235+0000 7f36bb400640 1 -- 172.21.15.19:0/3057904078 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc0d2cd0 con 0x7f36bc15e4c0 2024-03-28T13:16:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.236+0000 7f36bb400640 1 --2- 172.21.15.19:0/3057904078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36bc15e4c0 0x7f36bc0b9bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009d90 tx=0x7f36ac02bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=780ca09929d8c98 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.237+0000 7f36ba000640 1 -- 172.21.15.19:0/3057904078 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac039070 con 0x7f36bc15e4c0 2024-03-28T13:16:11.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.237+0000 7f36ba000640 1 -- 172.21.15.19:0/3057904078 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36ac02cd00 con 0x7f36bc15e4c0 2024-03-28T13:16:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.237+0000 7f36ba000640 1 -- 172.21.15.19:0/3057904078 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac034660 con 0x7f36bc15e4c0 2024-03-28T13:16:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.237+0000 7f36c0c00640 1 -- 172.21.15.19:0/3057904078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36bc15e4c0 msgr2=0x7f36bc0b9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.237+0000 7f36c0c00640 1 --2- 172.21.15.19:0/3057904078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36bc15e4c0 0x7f36bc0b9bf0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f36ac009d90 tx=0x7f36ac02bf50 comp rx=0 tx=0).stop 2024-03-28T13:16:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.238+0000 7f36c0c00640 1 -- 172.21.15.19:0/3057904078 shutdown_connections 2024-03-28T13:16:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.238+0000 7f36c0c00640 1 --2- 172.21.15.19:0/3057904078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36bc15e4c0 0x7f36bc0b9bf0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.238+0000 7f36c0c00640 1 -- 172.21.15.19:0/3057904078 >> 172.21.15.19:0/3057904078 conn(0x7f36bc0676a0 msgr2=0x7f36bc067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:11.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.238+0000 7f36c0c00640 1 -- 172.21.15.19:0/3057904078 shutdown_connections 2024-03-28T13:16:11.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.238+0000 7f36c0c00640 1 -- 172.21.15.19:0/3057904078 wait complete. 2024-03-28T13:16:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.239+0000 7f36c0c00640 1 Processor -- start 2024-03-28T13:16:11.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.239+0000 7f36c0c00640 1 -- start start 2024-03-28T13:16:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.240+0000 7f36c0c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36bc0b69c0 0x7f36bc14f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.240+0000 7f36c0c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36bc0d37b0 0x7f36bc0ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.240+0000 7f36c0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36bc15e4c0 0x7f36bc0c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.240+0000 7f36c0c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc0d3320 con 0x7f36bc15e4c0 2024-03-28T13:16:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.240+0000 7f36c0c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc0d4040 con 0x7f36bc0b69c0 2024-03-28T13:16:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.240+0000 7f36c0c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f36bc0ba130 con 0x7f36bc0d37b0 2024-03-28T13:16:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.241+0000 7f36bbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36bc15e4c0 0x7f36bc0c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.241+0000 7f36bbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36bc15e4c0 0x7f36bc0c7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41012/0 (socket says 172.21.15.19:41012) 2024-03-28T13:16:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.241+0000 7f36bbe00640 1 -- 172.21.15.19:0/3674050327 learned_addr learned my addr 172.21.15.19:0/3674050327 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.241+0000 7f36bb400640 1 --2- 172.21.15.19:0/3674050327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36bc0b69c0 0x7f36bc14f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.241+0000 7f36baa00640 1 --2- 172.21.15.19:0/3674050327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36bc0d37b0 0x7f36bc0ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.241+0000 7f36bbe00640 1 -- 172.21.15.19:0/3674050327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36bc0d37b0 msgr2=0x7f36bc0ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.241+0000 7f36bbe00640 1 --2- 172.21.15.19:0/3674050327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36bc0d37b0 0x7f36bc0ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.241+0000 7f36bbe00640 1 -- 172.21.15.19:0/3674050327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36bc0b69c0 msgr2=0x7f36bc14f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.241+0000 7f36bbe00640 1 --2- 172.21.15.19:0/3674050327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36bc0b69c0 0x7f36bc14f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.241+0000 7f36bbe00640 1 -- 172.21.15.19:0/3674050327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc0ca4b0 con 0x7f36bc15e4c0 2024-03-28T13:16:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.241+0000 7f36baa00640 1 --2- 172.21.15.19:0/3674050327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36bc0d37b0 0x7f36bc0ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:11.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.242+0000 7f36bbe00640 1 --2- 172.21.15.19:0/3674050327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36bc15e4c0 0x7f36bc0c7f30 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c3c0 tx=0x7f36b400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.242+0000 7f36b3400640 1 -- 172.21.15.19:0/3674050327 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4017070 con 0x7f36bc15e4c0 2024-03-28T13:16:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.242+0000 7f36b3400640 1 -- 172.21.15.19:0/3674050327 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b40052d0 con 0x7f36bc15e4c0 2024-03-28T13:16:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.242+0000 7f36b3400640 1 -- 172.21.15.19:0/3674050327 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4003830 con 0x7f36bc15e4c0 2024-03-28T13:16:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.242+0000 7f36c0c00640 1 -- 172.21.15.19:0/3674050327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36bc0ca790 con 0x7f36bc15e4c0 2024-03-28T13:16:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.242+0000 7f36c0c00640 1 -- 172.21.15.19:0/3674050327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36bc0b7560 con 0x7f36bc15e4c0 2024-03-28T13:16:11.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.243+0000 7f36c0c00640 1 -- 172.21.15.19:0/3674050327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36bc0cac50 con 0x7f36bc15e4c0 2024-03-28T13:16:11.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.244+0000 7f36b3400640 1 -- 172.21.15.19:0/3674050327 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f36b401a070 con 0x7f36bc15e4c0 2024-03-28T13:16:11.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.245+0000 7f36b3400640 1 --2- 172.21.15.19:0/3674050327 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f369806c530 0x7f369806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:11.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.245+0000 7f36b3400640 1 -- 172.21.15.19:0/3674050327 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5939+0+0 (secure 0 0 0) 0x7f36b408aa00 con 0x7f36bc15e4c0 2024-03-28T13:16:11.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.246+0000 7f36bb400640 1 --2- 172.21.15.19:0/3674050327 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f369806c530 0x7f369806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:11.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.247+0000 7f36bb400640 1 --2- 172.21.15.19:0/3674050327 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f369806c530 0x7f369806e9f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f36ac036000 tx=0x7f36ac030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:11.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.250+0000 7f36b3400640 1 -- 172.21.15.19:0/3674050327 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f36b4058fe0 con 0x7f36bc15e4c0 2024-03-28T13:16:11.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:11.499+0000 7f36c0c00640 1 -- 172.21.15.19:0/3674050327 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f36bc075940 con 0x7f36bc15e4c0 2024-03-28T13:16:12.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.011+0000 7f36b3400640 1 -- 172.21.15.19:0/3674050327 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v55) ==== 104+0+0 (secure 0 0 0) 0x7f36b405de20 con 0x7f36bc15e4c0 2024-03-28T13:16:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.015+0000 7f36c0c00640 1 -- 172.21.15.19:0/3674050327 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f369806c530 msgr2=0x7f369806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.015+0000 7f36c0c00640 1 --2- 172.21.15.19:0/3674050327 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f369806c530 0x7f369806e9f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f36ac036000 tx=0x7f36ac030000 comp rx=0 tx=0).stop 2024-03-28T13:16:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.015+0000 7f36c0c00640 1 -- 172.21.15.19:0/3674050327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36bc15e4c0 msgr2=0x7f36bc0c7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.015+0000 7f36c0c00640 1 --2- 172.21.15.19:0/3674050327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36bc15e4c0 0x7f36bc0c7f30 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c3c0 tx=0x7f36b400c7c0 comp rx=0 tx=0).stop 2024-03-28T13:16:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.015+0000 7f36c0c00640 1 -- 172.21.15.19:0/3674050327 shutdown_connections 2024-03-28T13:16:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.015+0000 7f36c0c00640 1 --2- 172.21.15.19:0/3674050327 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f369806c530 0x7f369806e9f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.015+0000 7f36c0c00640 1 --2- 172.21.15.19:0/3674050327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f36bc15e4c0 0x7f36bc0c7f30 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:12.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.015+0000 7f36c0c00640 1 --2- 172.21.15.19:0/3674050327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f36bc0d37b0 0x7f36bc0ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:12.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.015+0000 7f36c0c00640 1 --2- 172.21.15.19:0/3674050327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f36bc0b69c0 0x7f36bc14f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:12.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.015+0000 7f36c0c00640 1 -- 172.21.15.19:0/3674050327 >> 172.21.15.19:0/3674050327 conn(0x7f36bc0676a0 msgr2=0x7f36bc067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:12.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.015+0000 7f36c0c00640 1 -- 172.21.15.19:0/3674050327 shutdown_connections 2024-03-28T13:16:12.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.015+0000 7f36c0c00640 1 -- 172.21.15.19:0/3674050327 wait complete. 2024-03-28T13:16:12.037 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-28T13:16:12.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.255+0000 7f3833600640 1 Processor -- start 2024-03-28T13:16:12.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.255+0000 7f3833600640 1 -- start start 2024-03-28T13:16:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.256+0000 7f3833600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f382c09dcf0 0x7f382c1453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.256+0000 7f3833600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f382c071710 0x7f382c071b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.256+0000 7f3833600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f382c15cb90 con 0x7f382c0acf10 2024-03-28T13:16:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.256+0000 7f3833600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f382c145f60 con 0x7f382c071710 2024-03-28T13:16:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.256+0000 7f3833600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f382c15c950 con 0x7f382c09dcf0 2024-03-28T13:16:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.256+0000 7f3831600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f382c071710 0x7f382c071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.257+0000 7f3831600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f382c071710 0x7f382c071b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41024/0 (socket says 172.21.15.19:41024) 2024-03-28T13:16:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.257+0000 7f3831600640 1 -- 172.21.15.19:0/1562929047 learned_addr learned my addr 172.21.15.19:0/1562929047 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.257+0000 7f382be00640 1 --2- 172.21.15.19:0/1562929047 >> v2:172.21.15.188:3300/0 conn(0x7f382c09dcf0 0x7f382c1453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.257+0000 7f3831600640 1 -- 172.21.15.19:0/1562929047 >> v2:172.21.15.188:3300/0 conn(0x7f382c09dcf0 msgr2=0x7f382c1453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.257+0000 7f3831600640 1 --2- 172.21.15.19:0/1562929047 >> v2:172.21.15.188:3300/0 conn(0x7f382c09dcf0 0x7f382c1453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.257+0000 7f3831600640 1 -- 172.21.15.19:0/1562929047 >> v1:172.21.15.188:6789/0 conn(0x7f382c0acf10 legacy=0x7f382c0ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.257+0000 7f3831600640 1 -- 172.21.15.19:0/1562929047 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382c15ce70 con 0x7f382c071710 2024-03-28T13:16:12.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.257+0000 7f382be00640 1 --2- 172.21.15.19:0/1562929047 >> v2:172.21.15.188:3300/0 conn(0x7f382c09dcf0 0x7f382c1453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:12.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.258+0000 7f3831600640 1 --2- 172.21.15.19:0/1562929047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f382c071710 0x7f382c071b40 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f382c0cf8c0 tx=0x7f382402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832ecfe6dff0721 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:12.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.259+0000 7f382b400640 1 -- 172.21.15.19:0/1562929047 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382402c9f0 con 0x7f382c071710 2024-03-28T13:16:12.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.259+0000 7f382b400640 1 -- 172.21.15.19:0/1562929047 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f382402cb90 con 0x7f382c071710 2024-03-28T13:16:12.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.259+0000 7f382b400640 1 -- 172.21.15.19:0/1562929047 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38240343f0 con 0x7f382c071710 2024-03-28T13:16:12.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.259+0000 7f3833600640 1 -- 172.21.15.19:0/1562929047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f382c071710 msgr2=0x7f382c071b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:12.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.259+0000 7f3833600640 1 --2- 172.21.15.19:0/1562929047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f382c071710 0x7f382c071b40 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f382c0cf8c0 tx=0x7f382402be50 comp rx=0 tx=0).stop 2024-03-28T13:16:12.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.259+0000 7f3833600640 1 -- 172.21.15.19:0/1562929047 shutdown_connections 2024-03-28T13:16:12.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.259+0000 7f3833600640 1 --2- 172.21.15.19:0/1562929047 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f382c071710 0x7f382c071b40 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:12.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.259+0000 7f3833600640 1 --2- 172.21.15.19:0/1562929047 >> v2:172.21.15.188:3300/0 conn(0x7f382c09dcf0 0x7f382c1453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:12.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.259+0000 7f3833600640 1 -- 172.21.15.19:0/1562929047 >> 172.21.15.19:0/1562929047 conn(0x7f382c06bba0 msgr2=0x7f382c0b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:12.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.259+0000 7f3833600640 1 -- 172.21.15.19:0/1562929047 shutdown_connections 2024-03-28T13:16:12.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.259+0000 7f3833600640 1 -- 172.21.15.19:0/1562929047 wait complete. 2024-03-28T13:16:12.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.261+0000 7f3833600640 1 Processor -- start 2024-03-28T13:16:12.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.261+0000 7f3833600640 1 -- start start 2024-03-28T13:16:12.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.262+0000 7f3833600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382c071710 0x7f382c13f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:12.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.262+0000 7f3833600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f382c09dcf0 0x7f382c144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:12.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.262+0000 7f3833600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f382c0acf10 0x7f382c1fc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:12.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.262+0000 7f3833600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f382c15d430 con 0x7f382c09dcf0 2024-03-28T13:16:12.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.262+0000 7f3833600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f382c15d7a0 con 0x7f382c0acf10 2024-03-28T13:16:12.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.262+0000 7f3833600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f382c145930 con 0x7f382c071710 2024-03-28T13:16:12.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.263+0000 7f382be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f382c09dcf0 0x7f382c144340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:12.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.263+0000 7f382be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f382c09dcf0 0x7f382c144340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41040/0 (socket says 172.21.15.19:41040) 2024-03-28T13:16:12.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.263+0000 7f382be00640 1 -- 172.21.15.19:0/2847484579 learned_addr learned my addr 172.21.15.19:0/2847484579 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:12.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.263+0000 7f3830c00640 1 --2- 172.21.15.19:0/2847484579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382c071710 0x7f382c13f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:12.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.263+0000 7f382be00640 1 -- 172.21.15.19:0/2847484579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382c071710 msgr2=0x7f382c13f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:12.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.263+0000 7f3831600640 1 --2- 172.21.15.19:0/2847484579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f382c0acf10 0x7f382c1fc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:12.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.263+0000 7f382be00640 1 --2- 172.21.15.19:0/2847484579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382c071710 0x7f382c13f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:12.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.263+0000 7f382be00640 1 -- 172.21.15.19:0/2847484579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f382c0acf10 msgr2=0x7f382c1fc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:12.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.263+0000 7f382be00640 1 --2- 172.21.15.19:0/2847484579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f382c0acf10 0x7f382c1fc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:12.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.263+0000 7f382be00640 1 -- 172.21.15.19:0/2847484579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382c160dd0 con 0x7f382c09dcf0 2024-03-28T13:16:12.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.263+0000 7f3830c00640 1 --2- 172.21.15.19:0/2847484579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382c071710 0x7f382c13f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.263+0000 7f382be00640 1 --2- 172.21.15.19:0/2847484579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f382c09dcf0 0x7f382c144340 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f381800c7f0 tx=0x7f381800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.264+0000 7f3828c00640 1 -- 172.21.15.19:0/2847484579 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818015070 con 0x7f382c09dcf0 2024-03-28T13:16:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.264+0000 7f3828c00640 1 -- 172.21.15.19:0/2847484579 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3818007a30 con 0x7f382c09dcf0 2024-03-28T13:16:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.264+0000 7f3828c00640 1 -- 172.21.15.19:0/2847484579 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3818007e00 con 0x7f382c09dcf0 2024-03-28T13:16:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.264+0000 7f3833600640 1 -- 172.21.15.19:0/2847484579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382c160010 con 0x7f382c09dcf0 2024-03-28T13:16:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.264+0000 7f3833600640 1 -- 172.21.15.19:0/2847484579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f382c1605d0 con 0x7f382c09dcf0 2024-03-28T13:16:12.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.265+0000 7f3833600640 1 -- 172.21.15.19:0/2847484579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f382c0cfdb0 con 0x7f382c09dcf0 2024-03-28T13:16:12.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.266+0000 7f3828c00640 1 -- 172.21.15.19:0/2847484579 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3818018070 con 0x7f382c09dcf0 2024-03-28T13:16:12.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.267+0000 7f3828c00640 1 --2- 172.21.15.19:0/2847484579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f380c06c530 0x7f380c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:12.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.267+0000 7f3828c00640 1 -- 172.21.15.19:0/2847484579 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5939+0+0 (secure 0 0 0) 0x7f381808b840 con 0x7f382c09dcf0 2024-03-28T13:16:12.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.268+0000 7f3830c00640 1 --2- 172.21.15.19:0/2847484579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f380c06c530 0x7f380c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:12.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.269+0000 7f3830c00640 1 --2- 172.21.15.19:0/2847484579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f380c06c530 0x7f380c06e9f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f381c003eb0 tx=0x7f381c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:12.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.272+0000 7f3828c00640 1 -- 172.21.15.19:0/2847484579 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f382c0cfdb0 con 0x7f382c09dcf0 2024-03-28T13:16:12.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:12.478+0000 7f3833600640 1 -- 172.21.15.19:0/2847484579 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f382c0685c0 con 0x7f382c09dcf0 2024-03-28T13:16:13.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.412+0000 7f3828c00640 1 -- 172.21.15.19:0/2847484579 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) ==== 181+0+0 (secure 0 0 0) 0x7f3818059cd0 con 0x7f382c09dcf0 2024-03-28T13:16:13.412 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-03-28T13:16:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.417+0000 7f3833600640 1 -- 172.21.15.19:0/2847484579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f380c06c530 msgr2=0x7f380c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.417+0000 7f3833600640 1 --2- 172.21.15.19:0/2847484579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f380c06c530 0x7f380c06e9f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f381c003eb0 tx=0x7f381c009040 comp rx=0 tx=0).stop 2024-03-28T13:16:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.417+0000 7f3833600640 1 -- 172.21.15.19:0/2847484579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f382c09dcf0 msgr2=0x7f382c144340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.417+0000 7f3833600640 1 --2- 172.21.15.19:0/2847484579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f382c09dcf0 0x7f382c144340 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f381800c7f0 tx=0x7f381800cbf0 comp rx=0 tx=0).stop 2024-03-28T13:16:13.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.417+0000 7f3833600640 1 -- 172.21.15.19:0/2847484579 shutdown_connections 2024-03-28T13:16:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.417+0000 7f3833600640 1 --2- 172.21.15.19:0/2847484579 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f380c06c530 0x7f380c06e9f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.417+0000 7f3833600640 1 --2- 172.21.15.19:0/2847484579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f382c0acf10 0x7f382c1fc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.417+0000 7f3833600640 1 --2- 172.21.15.19:0/2847484579 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f382c09dcf0 0x7f382c144340 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.417+0000 7f3833600640 1 --2- 172.21.15.19:0/2847484579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f382c071710 0x7f382c13f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.417+0000 7f3833600640 1 -- 172.21.15.19:0/2847484579 >> 172.21.15.19:0/2847484579 conn(0x7f382c06bba0 msgr2=0x7f382c0b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.418+0000 7f3833600640 1 -- 172.21.15.19:0/2847484579 shutdown_connections 2024-03-28T13:16:13.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.418+0000 7f3833600640 1 -- 172.21.15.19:0/2847484579 wait complete. 2024-03-28T13:16:13.441 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-28T13:16:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.662+0000 7fa0e6000640 1 Processor -- start 2024-03-28T13:16:13.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.662+0000 7fa0e6000640 1 -- start start 2024-03-28T13:16:13.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.664+0000 7fa0e6000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e00b3940 0x7fa0e00adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:13.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.664+0000 7fa0e6000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0e00ae4f0 0x7fa0e00ae930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:13.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.664+0000 7fa0e6000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0e006ea10 con 0x7fa0e0072720 2024-03-28T13:16:13.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.664+0000 7fa0e6000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e006ecf0 con 0x7fa0e00ae4f0 2024-03-28T13:16:13.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.664+0000 7fa0e6000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e006efd0 con 0x7fa0e00b3940 2024-03-28T13:16:13.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.664+0000 7fa0df400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa0e0072720 0x7fa0e0072b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59776/0 (socket says 172.21.15.19:59776) 2024-03-28T13:16:13.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.664+0000 7fa0df400640 1 -- 172.21.15.19:0/2773431911 learned_addr learned my addr 172.21.15.19:0/2773431911 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:13.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.664+0000 7fa0dfe00640 1 --2- 172.21.15.19:0/2773431911 >> v2:172.21.15.19:3300/0 conn(0x7fa0e00ae4f0 0x7fa0e00ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:13.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.665+0000 7fa0dea00640 1 --2- 172.21.15.19:0/2773431911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e00b3940 0x7fa0e00adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:13.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.665+0000 7fa0de000640 1 -- 172.21.15.19:0/2773431911 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1847082371 0 0) 0x7fa0e006ea10 con 0x7fa0e0072720 2024-03-28T13:16:13.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.665+0000 7fa0de000640 1 -- 172.21.15.19:0/2773431911 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0c4005730 con 0x7fa0e0072720 2024-03-28T13:16:13.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.665+0000 7fa0dfe00640 1 -- 172.21.15.19:0/2773431911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e00b3940 msgr2=0x7fa0e00adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:13.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.665+0000 7fa0dfe00640 1 --2- 172.21.15.19:0/2773431911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e00b3940 0x7fa0e00adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:13.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.665+0000 7fa0dfe00640 1 -- 172.21.15.19:0/2773431911 >> v1:172.21.15.19:6789/0 conn(0x7fa0e0072720 legacy=0x7fa0e0072b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.665+0000 7fa0dfe00640 1 -- 172.21.15.19:0/2773431911 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e006f2b0 con 0x7fa0e00ae4f0 2024-03-28T13:16:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.666+0000 7fa0dfe00640 1 --2- 172.21.15.19:0/2773431911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00ae4f0 0x7fa0e00ae930 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009d90 tx=0x7fa0d802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d3a4821a5cc0c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.666+0000 7fa0de000640 1 -- 172.21.15.19:0/2773431911 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8039070 con 0x7fa0e00ae4f0 2024-03-28T13:16:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.666+0000 7fa0de000640 1 -- 172.21.15.19:0/2773431911 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d802cc50 con 0x7fa0e00ae4f0 2024-03-28T13:16:13.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.666+0000 7fa0de000640 1 -- 172.21.15.19:0/2773431911 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8034440 con 0x7fa0e00ae4f0 2024-03-28T13:16:13.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.667+0000 7fa0e6000640 1 -- 172.21.15.19:0/2773431911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00ae4f0 msgr2=0x7fa0e00ae930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:13.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.667+0000 7fa0e6000640 1 --2- 172.21.15.19:0/2773431911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00ae4f0 0x7fa0e00ae930 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009d90 tx=0x7fa0d802bea0 comp rx=0 tx=0).stop 2024-03-28T13:16:13.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.667+0000 7fa0e6000640 1 -- 172.21.15.19:0/2773431911 shutdown_connections 2024-03-28T13:16:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.667+0000 7fa0e6000640 1 --2- 172.21.15.19:0/2773431911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00ae4f0 0x7fa0e00ae930 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.667+0000 7fa0e6000640 1 --2- 172.21.15.19:0/2773431911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e00b3940 0x7fa0e00adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.667+0000 7fa0e6000640 1 -- 172.21.15.19:0/2773431911 >> 172.21.15.19:0/2773431911 conn(0x7fa0e006bba0 msgr2=0x7fa0e00b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.667+0000 7fa0e6000640 1 -- 172.21.15.19:0/2773431911 shutdown_connections 2024-03-28T13:16:13.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.667+0000 7fa0e6000640 1 -- 172.21.15.19:0/2773431911 wait complete. 2024-03-28T13:16:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.669+0000 7fa0e6000640 1 Processor -- start 2024-03-28T13:16:13.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.669+0000 7fa0e6000640 1 -- start start 2024-03-28T13:16:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.670+0000 7fa0e6000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e0072720 0x7fa0e00afc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.670+0000 7fa0e6000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e00ae4f0 0x7fa0e01f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.670+0000 7fa0e6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00b3940 0x7fa0e01f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.670+0000 7fa0e6000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e006e690 con 0x7fa0e00b3940 2024-03-28T13:16:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.670+0000 7fa0e6000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e00b3d80 con 0x7fa0e0072720 2024-03-28T13:16:13.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.670+0000 7fa0e6000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e015eab0 con 0x7fa0e00ae4f0 2024-03-28T13:16:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.671+0000 7fa0dfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00b3940 0x7fa0e01f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.671+0000 7fa0dfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00b3940 0x7fa0e01f81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41050/0 (socket says 172.21.15.19:41050) 2024-03-28T13:16:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.671+0000 7fa0dfe00640 1 -- 172.21.15.19:0/3071601175 learned_addr learned my addr 172.21.15.19:0/3071601175 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.671+0000 7fa0dea00640 1 --2- 172.21.15.19:0/3071601175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e00ae4f0 0x7fa0e01f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.671+0000 7fa0df400640 1 --2- 172.21.15.19:0/3071601175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e0072720 0x7fa0e00afc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.671+0000 7fa0dfe00640 1 -- 172.21.15.19:0/3071601175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e00ae4f0 msgr2=0x7fa0e01f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.671+0000 7fa0dfe00640 1 --2- 172.21.15.19:0/3071601175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e00ae4f0 0x7fa0e01f7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.671+0000 7fa0dfe00640 1 -- 172.21.15.19:0/3071601175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e0072720 msgr2=0x7fa0e00afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.671+0000 7fa0dfe00640 1 --2- 172.21.15.19:0/3071601175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e0072720 0x7fa0e00afc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.671+0000 7fa0dfe00640 1 -- 172.21.15.19:0/3071601175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e01f8920 con 0x7fa0e00b3940 2024-03-28T13:16:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.671+0000 7fa0dea00640 1 --2- 172.21.15.19:0/3071601175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e00ae4f0 0x7fa0e01f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:16:13.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.672+0000 7fa0dfe00640 1 --2- 172.21.15.19:0/3071601175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00b3940 0x7fa0e01f81a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009eb0 tx=0x7fa0d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.672+0000 7fa0d7400640 1 -- 172.21.15.19:0/3071601175 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8039050 con 0x7fa0e00b3940 2024-03-28T13:16:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.672+0000 7fa0d7400640 1 -- 172.21.15.19:0/3071601175 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0d8045070 con 0x7fa0e00b3940 2024-03-28T13:16:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.672+0000 7fa0e6000640 1 -- 172.21.15.19:0/3071601175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e01f4880 con 0x7fa0e00b3940 2024-03-28T13:16:13.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.672+0000 7fa0d7400640 1 -- 172.21.15.19:0/3071601175 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8009450 con 0x7fa0e00b3940 2024-03-28T13:16:13.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.672+0000 7fa0e6000640 1 -- 172.21.15.19:0/3071601175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e00b0190 con 0x7fa0e00b3940 2024-03-28T13:16:13.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.673+0000 7fa0e6000640 1 -- 172.21.15.19:0/3071601175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e01f4670 con 0x7fa0e00b3940 2024-03-28T13:16:13.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.674+0000 7fa0d7400640 1 -- 172.21.15.19:0/3071601175 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa0d803e610 con 0x7fa0e00b3940 2024-03-28T13:16:13.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.675+0000 7fa0d7400640 1 --2- 172.21.15.19:0/3071601175 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c530 0x7fa0c006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:13.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.675+0000 7fa0d7400640 1 -- 172.21.15.19:0/3071601175 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5472+0+0 (secure 0 0 0) 0x7fa0d80ae050 con 0x7fa0e00b3940 2024-03-28T13:16:13.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.676+0000 7fa0df400640 1 --2- 172.21.15.19:0/3071601175 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c530 0x7fa0c006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:13.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.677+0000 7fa0df400640 1 --2- 172.21.15.19:0/3071601175 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c530 0x7fa0c006e9f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0007fd0 tx=0x7fa0d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:13.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.679+0000 7fa0d7400640 1 -- 172.21.15.19:0/3071601175 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa0d807c770 con 0x7fa0e00b3940 2024-03-28T13:16:13.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:13.884+0000 7fa0e6000640 1 -- 172.21.15.19:0/3071601175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa0e00685c0 con 0x7fa0e00b3940 2024-03-28T13:16:14.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.416+0000 7fa0d7400640 1 -- 172.21.15.19:0/3071601175 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) ==== 169+0+0 (secure 0 0 0) 0x7fa0d807c070 con 0x7fa0e00b3940 2024-03-28T13:16:14.416 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-03-28T13:16:14.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.419+0000 7fa0e6000640 1 -- 172.21.15.19:0/3071601175 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c530 msgr2=0x7fa0c006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:14.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.419+0000 7fa0e6000640 1 --2- 172.21.15.19:0/3071601175 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c530 0x7fa0c006e9f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0007fd0 tx=0x7fa0d0009040 comp rx=0 tx=0).stop 2024-03-28T13:16:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.419+0000 7fa0e6000640 1 -- 172.21.15.19:0/3071601175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00b3940 msgr2=0x7fa0e01f81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.419+0000 7fa0e6000640 1 --2- 172.21.15.19:0/3071601175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00b3940 0x7fa0e01f81a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8009eb0 tx=0x7fa0d8004a40 comp rx=0 tx=0).stop 2024-03-28T13:16:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.420+0000 7fa0e6000640 1 -- 172.21.15.19:0/3071601175 shutdown_connections 2024-03-28T13:16:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.420+0000 7fa0e6000640 1 --2- 172.21.15.19:0/3071601175 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c530 0x7fa0c006e9f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.420+0000 7fa0e6000640 1 --2- 172.21.15.19:0/3071601175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0e00b3940 0x7fa0e01f81a0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.420+0000 7fa0e6000640 1 --2- 172.21.15.19:0/3071601175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e00ae4f0 0x7fa0e01f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.420+0000 7fa0e6000640 1 --2- 172.21.15.19:0/3071601175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e0072720 0x7fa0e00afc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.420+0000 7fa0e6000640 1 -- 172.21.15.19:0/3071601175 >> 172.21.15.19:0/3071601175 conn(0x7fa0e006bba0 msgr2=0x7fa0e00b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.420+0000 7fa0e6000640 1 -- 172.21.15.19:0/3071601175 shutdown_connections 2024-03-28T13:16:14.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.420+0000 7fa0e6000640 1 -- 172.21.15.19:0/3071601175 wait complete. 2024-03-28T13:16:14.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-28T13:16:14.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.672+0000 7f01dd400640 1 Processor -- start 2024-03-28T13:16:14.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.673+0000 7f01dd400640 1 -- start start 2024-03-28T13:16:14.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.674+0000 7f01dd400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01d8141c10 0x7f01d8071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:14.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.674+0000 7f01dd400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f01d8071eb0 0x7f01d8162160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:14.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.674+0000 7f01dd400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01d815ec80 con 0x7f01d80d0280 2024-03-28T13:16:14.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.674+0000 7f01dd400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f01d815ef20 con 0x7f01d8071eb0 2024-03-28T13:16:14.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.674+0000 7f01dd400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f01d815f200 con 0x7f01d8141c10 2024-03-28T13:16:14.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.675+0000 7f01d6800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f01d80d0280 0x7f01d80d0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:59778/0 (socket says 172.21.15.19:59778) 2024-03-28T13:16:14.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.675+0000 7f01d6800640 1 -- 172.21.15.19:0/2364626844 learned_addr learned my addr 172.21.15.19:0/2364626844 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:14.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.675+0000 7f01d7200640 1 --2- 172.21.15.19:0/2364626844 >> v2:172.21.15.19:3300/0 conn(0x7f01d8071eb0 0x7f01d8162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:14.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.675+0000 7f01d5e00640 1 --2- 172.21.15.19:0/2364626844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01d8141c10 0x7f01d8071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.675+0000 7f01d5400640 1 -- 172.21.15.19:0/2364626844 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1523948153 0 0) 0x7f01d815ec80 con 0x7f01d80d0280 2024-03-28T13:16:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.675+0000 7f01d5400640 1 -- 172.21.15.19:0/2364626844 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01bc003520 con 0x7f01d80d0280 2024-03-28T13:16:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.675+0000 7f01d7200640 1 -- 172.21.15.19:0/2364626844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01d8141c10 msgr2=0x7f01d8071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.675+0000 7f01d7200640 1 --2- 172.21.15.19:0/2364626844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01d8141c10 0x7f01d8071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.675+0000 7f01d7200640 1 -- 172.21.15.19:0/2364626844 >> v1:172.21.15.19:6789/0 conn(0x7f01d80d0280 legacy=0x7f01d80d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:14.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.676+0000 7f01d7200640 1 -- 172.21.15.19:0/2364626844 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d815f4e0 con 0x7f01d8071eb0 2024-03-28T13:16:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.677+0000 7f01d7200640 1 --2- 172.21.15.19:0/2364626844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d8071eb0 0x7f01d8162160 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009d90 tx=0x7f01d002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf02d175a826a61 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.678+0000 7f01d5400640 1 -- 172.21.15.19:0/2364626844 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0039070 con 0x7f01d8071eb0 2024-03-28T13:16:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.678+0000 7f01d5400640 1 -- 172.21.15.19:0/2364626844 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d002cc50 con 0x7f01d8071eb0 2024-03-28T13:16:14.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.678+0000 7f01d5400640 1 -- 172.21.15.19:0/2364626844 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0034440 con 0x7f01d8071eb0 2024-03-28T13:16:14.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.678+0000 7f01dd400640 1 -- 172.21.15.19:0/2364626844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d8071eb0 msgr2=0x7f01d8162160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:14.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.678+0000 7f01dd400640 1 --2- 172.21.15.19:0/2364626844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d8071eb0 0x7f01d8162160 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009d90 tx=0x7f01d002bea0 comp rx=0 tx=0).stop 2024-03-28T13:16:14.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.678+0000 7f01dd400640 1 -- 172.21.15.19:0/2364626844 shutdown_connections 2024-03-28T13:16:14.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.678+0000 7f01dd400640 1 --2- 172.21.15.19:0/2364626844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d8071eb0 0x7f01d8162160 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:14.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.678+0000 7f01dd400640 1 --2- 172.21.15.19:0/2364626844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01d8141c10 0x7f01d8071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:14.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.678+0000 7f01dd400640 1 -- 172.21.15.19:0/2364626844 >> 172.21.15.19:0/2364626844 conn(0x7f01d806b750 msgr2=0x7f01d806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:14.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.678+0000 7f01dd400640 1 -- 172.21.15.19:0/2364626844 shutdown_connections 2024-03-28T13:16:14.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.678+0000 7f01dd400640 1 -- 172.21.15.19:0/2364626844 wait complete. 2024-03-28T13:16:14.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.680+0000 7f01dd400640 1 Processor -- start 2024-03-28T13:16:14.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.680+0000 7f01dd400640 1 -- start start 2024-03-28T13:16:14.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.681+0000 7f01dd400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01d8071eb0 0x7f01d8099900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:14.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.681+0000 7f01dd400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d80d0280 0x7f01d80ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:14.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.681+0000 7f01dd400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01d8141c10 0x7f01d80ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:14.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.681+0000 7f01dd400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f01d8158e90 con 0x7f01d80d0280 2024-03-28T13:16:14.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.681+0000 7f01dd400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f01d806f340 con 0x7f01d8141c10 2024-03-28T13:16:14.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.681+0000 7f01dd400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f01d81460e0 con 0x7f01d8071eb0 2024-03-28T13:16:14.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.681+0000 7f01d5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d80d0280 0x7f01d80ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:14.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.681+0000 7f01d5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d80d0280 0x7f01d80ae060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41064/0 (socket says 172.21.15.19:41064) 2024-03-28T13:16:14.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.681+0000 7f01d5e00640 1 -- 172.21.15.19:0/1617420405 learned_addr learned my addr 172.21.15.19:0/1617420405 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:14.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.682+0000 7f01d6800640 1 --2- 172.21.15.19:0/1617420405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01d8071eb0 0x7f01d8099900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:14.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.682+0000 7f01d5e00640 1 -- 172.21.15.19:0/1617420405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01d8071eb0 msgr2=0x7f01d8099900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:14.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.682+0000 7f01d5e00640 1 --2- 172.21.15.19:0/1617420405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01d8071eb0 0x7f01d8099900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.682+0000 7f01d5e00640 1 -- 172.21.15.19:0/1617420405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01d8141c10 msgr2=0x7f01d80ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.682+0000 7f01d5e00640 1 --2- 172.21.15.19:0/1617420405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01d8141c10 0x7f01d80ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.682+0000 7f01d5e00640 1 -- 172.21.15.19:0/1617420405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d809daf0 con 0x7f01d80d0280 2024-03-28T13:16:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.682+0000 7f01d5e00640 1 --2- 172.21.15.19:0/1617420405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d80d0280 0x7f01d80ae060 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f01c000e870 tx=0x7f01c000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:14.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.682+0000 7f01cea00640 1 -- 172.21.15.19:0/1617420405 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c0017070 con 0x7f01d80d0280 2024-03-28T13:16:14.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.683+0000 7f01cea00640 1 -- 172.21.15.19:0/1617420405 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c000ee60 con 0x7f01d80d0280 2024-03-28T13:16:14.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.683+0000 7f01cea00640 1 -- 172.21.15.19:0/1617420405 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c000cb60 con 0x7f01d80d0280 2024-03-28T13:16:14.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.683+0000 7f01dd400640 1 -- 172.21.15.19:0/1617420405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d809ddd0 con 0x7f01d80d0280 2024-03-28T13:16:14.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.683+0000 7f01dd400640 1 -- 172.21.15.19:0/1617420405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d8099e40 con 0x7f01d80d0280 2024-03-28T13:16:14.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.684+0000 7f01dd400640 1 -- 172.21.15.19:0/1617420405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d80685c0 con 0x7f01d80d0280 2024-03-28T13:16:14.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.685+0000 7f01cea00640 1 -- 172.21.15.19:0/1617420405 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f01c001a070 con 0x7f01d80d0280 2024-03-28T13:16:14.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.686+0000 7f01cea00640 1 --2- 172.21.15.19:0/1617420405 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f01b406c530 0x7f01b406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:14.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.686+0000 7f01cea00640 1 -- 172.21.15.19:0/1617420405 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7f01c008b6f0 con 0x7f01d80d0280 2024-03-28T13:16:14.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.687+0000 7f01d6800640 1 --2- 172.21.15.19:0/1617420405 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f01b406c530 0x7f01b406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:14.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.688+0000 7f01d6800640 1 --2- 172.21.15.19:0/1617420405 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f01b406c530 0x7f01b406e9f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f01c8005880 tx=0x7f01c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:14.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.691+0000 7f01cea00640 1 -- 172.21.15.19:0/1617420405 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f01c0059f10 con 0x7f01d80d0280 2024-03-28T13:16:14.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.892+0000 7f01dd400640 1 -- 172.21.15.19:0/1617420405 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f01d80cfdb0 con 0x7f01d80d0280 2024-03-28T13:16:14.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.900+0000 7f01cea00640 1 -- 172.21.15.19:0/1617420405 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f01c0059810 con 0x7f01d80d0280 2024-03-28T13:16:14.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.902+0000 7f01dd400640 1 -- 172.21.15.19:0/1617420405 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f01b406c530 msgr2=0x7f01b406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:14.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.902+0000 7f01dd400640 1 --2- 172.21.15.19:0/1617420405 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f01b406c530 0x7f01b406e9f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f01c8005880 tx=0x7f01c8009040 comp rx=0 tx=0).stop 2024-03-28T13:16:14.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.902+0000 7f01dd400640 1 -- 172.21.15.19:0/1617420405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d80d0280 msgr2=0x7f01d80ae060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:14.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.902+0000 7f01dd400640 1 --2- 172.21.15.19:0/1617420405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d80d0280 0x7f01d80ae060 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f01c000e870 tx=0x7f01c000ed30 comp rx=0 tx=0).stop 2024-03-28T13:16:14.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.903+0000 7f01dd400640 1 -- 172.21.15.19:0/1617420405 shutdown_connections 2024-03-28T13:16:14.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.903+0000 7f01dd400640 1 --2- 172.21.15.19:0/1617420405 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f01b406c530 0x7f01b406e9f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:14.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.903+0000 7f01dd400640 1 --2- 172.21.15.19:0/1617420405 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01d8141c10 0x7f01d80ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:14.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.903+0000 7f01dd400640 1 --2- 172.21.15.19:0/1617420405 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01d80d0280 0x7f01d80ae060 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:14.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.903+0000 7f01dd400640 1 --2- 172.21.15.19:0/1617420405 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01d8071eb0 0x7f01d8099900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:14.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.903+0000 7f01dd400640 1 -- 172.21.15.19:0/1617420405 >> 172.21.15.19:0/1617420405 conn(0x7f01d806b750 msgr2=0x7f01d81605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:14.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.903+0000 7f01dd400640 1 -- 172.21.15.19:0/1617420405 shutdown_connections 2024-03-28T13:16:14.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:14.903+0000 7f01dd400640 1 -- 172.21.15.19:0/1617420405 wait complete. 2024-03-28T13:16:14.919 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-03-28T13:16:15.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.135+0000 7f5fe6400640 1 Processor -- start 2024-03-28T13:16:15.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.136+0000 7f5fe6400640 1 -- start start 2024-03-28T13:16:15.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.137+0000 7f5fe6400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5fe013f300 0x7f5fe006e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:15.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.137+0000 7f5fe6400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe0141e40 0x7f5fe00cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:15.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.137+0000 7f5fe6400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fe0142880 con 0x7f5fe01419e0 2024-03-28T13:16:15.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.137+0000 7f5fe6400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5fe00cee90 con 0x7f5fe013f300 2024-03-28T13:16:15.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.137+0000 7f5fe6400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe01422c0 con 0x7f5fe0141e40 2024-03-28T13:16:15.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.138+0000 7f5fe5000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5fe013f300 0x7f5fe006e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:15.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.138+0000 7f5fe5a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe0141e40 0x7f5fe00cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:15.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.138+0000 7f5fe5000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5fe013f300 0x7f5fe006e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:44352/0 (socket says 172.21.15.19:44352) 2024-03-28T13:16:15.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.138+0000 7f5fe5000640 1 -- 172.21.15.19:0/3959751542 learned_addr learned my addr 172.21.15.19:0/3959751542 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:15.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.139+0000 7f5fdfe00640 1 -- 172.21.15.19:0/3959751542 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 416535592 0 0) 0x7f5fe0142880 con 0x7f5fe01419e0 2024-03-28T13:16:15.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.139+0000 7f5fdfe00640 1 -- 172.21.15.19:0/3959751542 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fc4003520 con 0x7f5fe01419e0 2024-03-28T13:16:15.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.139+0000 7f5fe5000640 1 -- 172.21.15.19:0/3959751542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe0141e40 msgr2=0x7f5fe00cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:15.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.139+0000 7f5fe5000640 1 --2- 172.21.15.19:0/3959751542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe0141e40 0x7f5fe00cf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:15.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.139+0000 7f5fe5000640 1 -- 172.21.15.19:0/3959751542 >> v1:172.21.15.188:6789/0 conn(0x7f5fe01419e0 legacy=0x7f5fe015fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:15.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.139+0000 7f5fe5000640 1 -- 172.21.15.19:0/3959751542 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe01425a0 con 0x7f5fe013f300 2024-03-28T13:16:15.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.139+0000 7f5fe5a00640 1 --2- 172.21.15.19:0/3959751542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe0141e40 0x7f5fe00cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:16:15.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.140+0000 7f5fe5000640 1 --2- 172.21.15.19:0/3959751542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe013f300 0x7f5fe006e030 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00cf010 tx=0x7f5fd002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=832d3e6d92086640 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:15.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.141+0000 7f5fdfe00640 1 -- 172.21.15.19:0/3959751542 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0034070 con 0x7f5fe013f300 2024-03-28T13:16:15.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.141+0000 7f5fdfe00640 1 -- 172.21.15.19:0/3959751542 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd0030840 con 0x7f5fe013f300 2024-03-28T13:16:15.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.141+0000 7f5fdfe00640 1 -- 172.21.15.19:0/3959751542 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0030bf0 con 0x7f5fe013f300 2024-03-28T13:16:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.142+0000 7f5fe6400640 1 -- 172.21.15.19:0/3959751542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe013f300 msgr2=0x7f5fe006e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.142+0000 7f5fe6400640 1 --2- 172.21.15.19:0/3959751542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe013f300 0x7f5fe006e030 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5fe00cf010 tx=0x7f5fd002b980 comp rx=0 tx=0).stop 2024-03-28T13:16:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.142+0000 7f5fe6400640 1 -- 172.21.15.19:0/3959751542 shutdown_connections 2024-03-28T13:16:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.142+0000 7f5fe6400640 1 --2- 172.21.15.19:0/3959751542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe0141e40 0x7f5fe00cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.142+0000 7f5fe6400640 1 --2- 172.21.15.19:0/3959751542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe013f300 0x7f5fe006e030 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.142+0000 7f5fe6400640 1 -- 172.21.15.19:0/3959751542 >> 172.21.15.19:0/3959751542 conn(0x7f5fe006b570 msgr2=0x7f5fe006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.142+0000 7f5fe6400640 1 -- 172.21.15.19:0/3959751542 shutdown_connections 2024-03-28T13:16:15.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.142+0000 7f5fe6400640 1 -- 172.21.15.19:0/3959751542 wait complete. 2024-03-28T13:16:15.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.144+0000 7f5fe6400640 1 Processor -- start 2024-03-28T13:16:15.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.144+0000 7f5fe6400640 1 -- start start 2024-03-28T13:16:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.145+0000 7f5fe6400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe013f300 0x7f5fe0156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fe6400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe01419e0 0x7f5fe01f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fe6400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe0141e40 0x7f5fe01565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fe6400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe0160620 con 0x7f5fe0141e40 2024-03-28T13:16:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fe6400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe01604a0 con 0x7f5fe01419e0 2024-03-28T13:16:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fe6400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5fe00cf860 con 0x7f5fe013f300 2024-03-28T13:16:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fe5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe0141e40 0x7f5fe01565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fe5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe0141e40 0x7f5fe01565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41070/0 (socket says 172.21.15.19:41070) 2024-03-28T13:16:15.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fe5a00640 1 -- 172.21.15.19:0/2992585267 learned_addr learned my addr 172.21.15.19:0/2992585267 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fe5000640 1 --2- 172.21.15.19:0/2992585267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe013f300 0x7f5fe0156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fd7e00640 1 --2- 172.21.15.19:0/2992585267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe01419e0 0x7f5fe01f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fe5a00640 1 -- 172.21.15.19:0/2992585267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe013f300 msgr2=0x7f5fe0156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fe5a00640 1 --2- 172.21.15.19:0/2992585267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe013f300 0x7f5fe0156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fe5a00640 1 -- 172.21.15.19:0/2992585267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe01419e0 msgr2=0x7f5fe01f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fe5a00640 1 --2- 172.21.15.19:0/2992585267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe01419e0 0x7f5fe01f2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fe5a00640 1 -- 172.21.15.19:0/2992585267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe0157460 con 0x7f5fe0141e40 2024-03-28T13:16:15.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.146+0000 7f5fd7e00640 1 --2- 172.21.15.19:0/2992585267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe01419e0 0x7f5fe01f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.147+0000 7f5fe5a00640 1 --2- 172.21.15.19:0/2992585267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe0141e40 0x7f5fe01565d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800ef90 tx=0x7f5fd800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.147+0000 7f5fdd600640 1 -- 172.21.15.19:0/2992585267 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8018020 con 0x7f5fe0141e40 2024-03-28T13:16:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.147+0000 7f5fdd600640 1 -- 172.21.15.19:0/2992585267 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fd8014480 con 0x7f5fe0141e40 2024-03-28T13:16:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.147+0000 7f5fdd600640 1 -- 172.21.15.19:0/2992585267 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8014870 con 0x7f5fe0141e40 2024-03-28T13:16:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.147+0000 7f5fe6400640 1 -- 172.21.15.19:0/2992585267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe0157740 con 0x7f5fe0141e40 2024-03-28T13:16:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.147+0000 7f5fe6400640 1 -- 172.21.15.19:0/2992585267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe0157d00 con 0x7f5fe0141e40 2024-03-28T13:16:15.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.148+0000 7f5fe6400640 1 -- 172.21.15.19:0/2992585267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fe0075940 con 0x7f5fe0141e40 2024-03-28T13:16:15.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.149+0000 7f5fdd600640 1 -- 172.21.15.19:0/2992585267 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5fd8005d10 con 0x7f5fe0141e40 2024-03-28T13:16:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.150+0000 7f5fdd600640 1 --2- 172.21.15.19:0/2992585267 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5fbc06c530 0x7f5fbc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.150+0000 7f5fdd600640 1 -- 172.21.15.19:0/2992585267 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7f5fd808c1c0 con 0x7f5fe0141e40 2024-03-28T13:16:15.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.151+0000 7f5fe5000640 1 --2- 172.21.15.19:0/2992585267 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5fbc06c530 0x7f5fbc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:15.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.152+0000 7f5fe5000640 1 --2- 172.21.15.19:0/2992585267 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5fbc06c530 0x7f5fbc06e9f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0036000 tx=0x7f5fd003b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.154+0000 7f5fdd600640 1 -- 172.21.15.19:0/2992585267 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5fd8022030 con 0x7f5fe0141e40 2024-03-28T13:16:15.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:15.430+0000 7f5fe6400640 1 -- 172.21.15.19:0/2992585267 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f5fe01600f0 con 0x7f5fe0141e40 2024-03-28T13:16:16.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.443+0000 7f5fdd600640 1 -- 172.21.15.19:0/2992585267 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v334) ==== 166+0+0 (secure 0 0 0) 0x7f5fd8010040 con 0x7f5fe0141e40 2024-03-28T13:16:16.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.447+0000 7f5fe6400640 1 -- 172.21.15.19:0/2992585267 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5fbc06c530 msgr2=0x7f5fbc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:16.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.447+0000 7f5fe6400640 1 --2- 172.21.15.19:0/2992585267 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5fbc06c530 0x7f5fbc06e9f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0036000 tx=0x7f5fd003b040 comp rx=0 tx=0).stop 2024-03-28T13:16:16.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.447+0000 7f5fe6400640 1 -- 172.21.15.19:0/2992585267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe0141e40 msgr2=0x7f5fe01565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.447+0000 7f5fe6400640 1 --2- 172.21.15.19:0/2992585267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe0141e40 0x7f5fe01565d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800ef90 tx=0x7f5fd800c490 comp rx=0 tx=0).stop 2024-03-28T13:16:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.447+0000 7f5fe6400640 1 -- 172.21.15.19:0/2992585267 shutdown_connections 2024-03-28T13:16:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.447+0000 7f5fe6400640 1 --2- 172.21.15.19:0/2992585267 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5fbc06c530 0x7f5fbc06e9f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.447+0000 7f5fe6400640 1 --2- 172.21.15.19:0/2992585267 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fe0141e40 0x7f5fe01565d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.447+0000 7f5fe6400640 1 --2- 172.21.15.19:0/2992585267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fe01419e0 0x7f5fe01f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.447+0000 7f5fe6400640 1 --2- 172.21.15.19:0/2992585267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fe013f300 0x7f5fe0156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.448+0000 7f5fe6400640 1 -- 172.21.15.19:0/2992585267 >> 172.21.15.19:0/2992585267 conn(0x7f5fe006b570 msgr2=0x7f5fe00cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.448+0000 7f5fe6400640 1 -- 172.21.15.19:0/2992585267 shutdown_connections 2024-03-28T13:16:16.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.448+0000 7f5fe6400640 1 -- 172.21.15.19:0/2992585267 wait complete. 2024-03-28T13:16:16.471 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-03-28T13:16:16.471 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-03-28T13:16:16.471 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-28T13:16:16.472 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:16:16.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.688+0000 7fd014800640 1 Processor -- start 2024-03-28T13:16:16.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.689+0000 7fd014800640 1 -- start start 2024-03-28T13:16:16.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.690+0000 7fd014800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd00c0af830 0x7fd00c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:16.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.690+0000 7fd014800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd00c0d0650 0x7fd00c0d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:16.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.690+0000 7fd014800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd00c15fe80 con 0x7fd00c15c390 2024-03-28T13:16:16.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.690+0000 7fd014800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd00c15c770 con 0x7fd00c0d0650 2024-03-28T13:16:16.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.690+0000 7fd014800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd00c0ad120 con 0x7fd00c0af830 2024-03-28T13:16:16.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.690+0000 7fd012800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd00c0d0650 0x7fd00c0d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:16.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.690+0000 7fd012800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd00c0d0650 0x7fd00c0d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49246/0 (socket says 172.21.15.19:49246) 2024-03-28T13:16:16.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.690+0000 7fd012800640 1 -- 172.21.15.19:0/2192251789 learned_addr learned my addr 172.21.15.19:0/2192251789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:16.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.691+0000 7fd012800640 1 -- 172.21.15.19:0/2192251789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd00c0af830 msgr2=0x7fd00c073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:16.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.691+0000 7fd011400640 1 --2- 172.21.15.19:0/2192251789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd00c0af830 0x7fd00c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:16.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.691+0000 7fd012800640 1 --2- 172.21.15.19:0/2192251789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd00c0af830 0x7fd00c073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:16.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.691+0000 7fd012800640 1 -- 172.21.15.19:0/2192251789 >> v1:172.21.15.188:6789/0 conn(0x7fd00c15c390 legacy=0x7fd00c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:16.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.691+0000 7fd012800640 1 -- 172.21.15.19:0/2192251789 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00c0ad400 con 0x7fd00c0d0650 2024-03-28T13:16:16.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.691+0000 7fd011400640 1 --2- 172.21.15.19:0/2192251789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd00c0af830 0x7fd00c073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:16.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.692+0000 7fd012800640 1 --2- 172.21.15.19:0/2192251789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00c0d0650 0x7fd00c0d0b00 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd008009e50 tx=0x7fd00802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a9a4c2405c9ea9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.692+0000 7fd010a00640 1 -- 172.21.15.19:0/2192251789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd00802ca70 con 0x7fd00c0d0650 2024-03-28T13:16:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.692+0000 7fd010a00640 1 -- 172.21.15.19:0/2192251789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd00802cc10 con 0x7fd00c0d0650 2024-03-28T13:16:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.692+0000 7fd010a00640 1 -- 172.21.15.19:0/2192251789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0080343f0 con 0x7fd00c0d0650 2024-03-28T13:16:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.693+0000 7fd014800640 1 -- 172.21.15.19:0/2192251789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00c0d0650 msgr2=0x7fd00c0d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.693+0000 7fd014800640 1 --2- 172.21.15.19:0/2192251789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00c0d0650 0x7fd00c0d0b00 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd008009e50 tx=0x7fd00802bed0 comp rx=0 tx=0).stop 2024-03-28T13:16:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.693+0000 7fd014800640 1 -- 172.21.15.19:0/2192251789 shutdown_connections 2024-03-28T13:16:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.693+0000 7fd014800640 1 --2- 172.21.15.19:0/2192251789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00c0d0650 0x7fd00c0d0b00 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:16.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.693+0000 7fd014800640 1 --2- 172.21.15.19:0/2192251789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd00c0af830 0x7fd00c073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.693+0000 7fd014800640 1 -- 172.21.15.19:0/2192251789 >> 172.21.15.19:0/2192251789 conn(0x7fd00c06b990 msgr2=0x7fd00c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.693+0000 7fd014800640 1 -- 172.21.15.19:0/2192251789 shutdown_connections 2024-03-28T13:16:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.693+0000 7fd014800640 1 -- 172.21.15.19:0/2192251789 wait complete. 2024-03-28T13:16:16.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.695+0000 7fd014800640 1 Processor -- start 2024-03-28T13:16:16.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.695+0000 7fd014800640 1 -- start start 2024-03-28T13:16:16.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.696+0000 7fd014800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd00c0af830 0x7fd00c15f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:16.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.696+0000 7fd014800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00c0d0650 0x7fd00c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:16.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.696+0000 7fd014800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd00c15c390 0x7fd00c1f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:16.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.696+0000 7fd014800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c160d50 con 0x7fd00c0d0650 2024-03-28T13:16:16.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.696+0000 7fd014800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c160120 con 0x7fd00c0af830 2024-03-28T13:16:16.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.696+0000 7fd014800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd00c0cf980 con 0x7fd00c15c390 2024-03-28T13:16:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.696+0000 7fd011400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00c0d0650 0x7fd00c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.696+0000 7fd011400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00c0d0650 0x7fd00c15d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49262/0 (socket says 172.21.15.19:49262) 2024-03-28T13:16:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.696+0000 7fd011400640 1 -- 172.21.15.19:0/1886938865 learned_addr learned my addr 172.21.15.19:0/1886938865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:16.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.697+0000 7fd011e00640 1 --2- 172.21.15.19:0/1886938865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd00c0af830 0x7fd00c15f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.697+0000 7fd011400640 1 -- 172.21.15.19:0/1886938865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd00c15c390 msgr2=0x7fd00c1f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.697+0000 7fd011400640 1 --2- 172.21.15.19:0/1886938865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd00c15c390 0x7fd00c1f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.697+0000 7fd011400640 1 -- 172.21.15.19:0/1886938865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd00c0af830 msgr2=0x7fd00c15f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.697+0000 7fd011400640 1 --2- 172.21.15.19:0/1886938865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd00c0af830 0x7fd00c15f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.697+0000 7fd011400640 1 -- 172.21.15.19:0/1886938865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00c15de90 con 0x7fd00c0d0650 2024-03-28T13:16:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.697+0000 7fd011e00640 1 --2- 172.21.15.19:0/1886938865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd00c0af830 0x7fd00c15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.697+0000 7fd011400640 1 --2- 172.21.15.19:0/1886938865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00c0d0650 0x7fd00c15d950 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00c820 tx=0x7fcffc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.697+0000 7fd006000640 1 -- 172.21.15.19:0/1886938865 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc015070 con 0x7fd00c0d0650 2024-03-28T13:16:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.697+0000 7fd006000640 1 -- 172.21.15.19:0/1886938865 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcffc007a30 con 0x7fd00c0d0650 2024-03-28T13:16:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.698+0000 7fd006000640 1 -- 172.21.15.19:0/1886938865 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcffc007e00 con 0x7fd00c0d0650 2024-03-28T13:16:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.698+0000 7fd014800640 1 -- 172.21.15.19:0/1886938865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00c15e130 con 0x7fd00c0d0650 2024-03-28T13:16:16.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.698+0000 7fd014800640 1 -- 172.21.15.19:0/1886938865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd00c15e6f0 con 0x7fd00c0d0650 2024-03-28T13:16:16.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.699+0000 7fd014800640 1 -- 172.21.15.19:0/1886938865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00c0685c0 con 0x7fd00c0d0650 2024-03-28T13:16:16.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.700+0000 7fd006000640 1 -- 172.21.15.19:0/1886938865 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcffc018070 con 0x7fd00c0d0650 2024-03-28T13:16:16.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.701+0000 7fd006000640 1 --2- 172.21.15.19:0/1886938865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcff006c530 0x7fcff006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:16.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.701+0000 7fd006000640 1 -- 172.21.15.19:0/1886938865 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7fcffc08b420 con 0x7fd00c0d0650 2024-03-28T13:16:16.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.702+0000 7fd011e00640 1 --2- 172.21.15.19:0/1886938865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcff006c530 0x7fcff006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:16.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.703+0000 7fd011e00640 1 --2- 172.21.15.19:0/1886938865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcff006c530 0x7fcff006e9f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd00000a830 tx=0x7fd000008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:16.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.706+0000 7fd006000640 1 -- 172.21.15.19:0/1886938865 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcffc059cd0 con 0x7fd00c0d0650 2024-03-28T13:16:16.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.909+0000 7fd014800640 1 -- 172.21.15.19:0/1886938865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd00c071e40 con 0x7fd00c0d0650 2024-03-28T13:16:16.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.910+0000 7fd006000640 1 -- 172.21.15.19:0/1886938865 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+13151 (secure 0 0 0) 0x7fcffc0595d0 con 0x7fd00c0d0650 2024-03-28T13:16:16.911 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:16.911 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:16:14.413081+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/2130552083":"2024-03-29T13:16:11.005818+0000","172.21.15.19:6832/2130552083":"2024-03-29T13:16:11.005818+0000","172.21.15.19:6834/1189161682":"2024-03-29T13:14:45.007899+0000","172.21.15.19:6832/1189161682":"2024-03-29T13:14:45.007899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:16:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.913+0000 7fd014800640 1 -- 172.21.15.19:0/1886938865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcff006c530 msgr2=0x7fcff006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.913+0000 7fd014800640 1 --2- 172.21.15.19:0/1886938865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcff006c530 0x7fcff006e9f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd00000a830 tx=0x7fd000008040 comp rx=0 tx=0).stop 2024-03-28T13:16:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.913+0000 7fd014800640 1 -- 172.21.15.19:0/1886938865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00c0d0650 msgr2=0x7fd00c15d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.913+0000 7fd014800640 1 --2- 172.21.15.19:0/1886938865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00c0d0650 0x7fd00c15d950 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fcffc00c820 tx=0x7fcffc00cc20 comp rx=0 tx=0).stop 2024-03-28T13:16:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.914+0000 7fd014800640 1 -- 172.21.15.19:0/1886938865 shutdown_connections 2024-03-28T13:16:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.914+0000 7fd014800640 1 --2- 172.21.15.19:0/1886938865 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcff006c530 0x7fcff006e9f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.914+0000 7fd014800640 1 --2- 172.21.15.19:0/1886938865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd00c15c390 0x7fd00c1f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.914+0000 7fd014800640 1 --2- 172.21.15.19:0/1886938865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd00c0d0650 0x7fd00c15d950 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.914+0000 7fd014800640 1 --2- 172.21.15.19:0/1886938865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd00c0af830 0x7fd00c15f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.914+0000 7fd014800640 1 -- 172.21.15.19:0/1886938865 >> 172.21.15.19:0/1886938865 conn(0x7fd00c06b990 msgr2=0x7fd00c154a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.914+0000 7fd014800640 1 -- 172.21.15.19:0/1886938865 shutdown_connections 2024-03-28T13:16:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:16.914+0000 7fd014800640 1 -- 172.21.15.19:0/1886938865 wait complete. 2024-03-28T13:16:16.934 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:16:16.934 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:16:17.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.162+0000 7febf7200640 1 Processor -- start 2024-03-28T13:16:17.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.162+0000 7febf7200640 1 -- start start 2024-03-28T13:16:17.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.163+0000 7febf7200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7febf815e460 0x7febf80b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:17.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.163+0000 7febf7200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febf80d2da0 con 0x7febf80cef20 2024-03-28T13:16:17.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.163+0000 7febf7200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febf80d3080 con 0x7febf80d3e40 2024-03-28T13:16:17.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.163+0000 7febf7200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7febf80d3360 con 0x7febf815e460 2024-03-28T13:16:17.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.164+0000 7febf6800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7febf80cef20 0x7febf80b7310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36290/0 (socket says 172.21.15.19:36290) 2024-03-28T13:16:17.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.164+0000 7febf6800640 1 -- 172.21.15.19:0/4073145597 learned_addr learned my addr 172.21.15.19:0/4073145597 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:17.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.164+0000 7febf5e00640 1 --2- 172.21.15.19:0/4073145597 >> v2:172.21.15.188:3300/0 conn(0x7febf815e460 0x7febf80b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:17.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.164+0000 7febf5400640 1 -- 172.21.15.19:0/4073145597 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3565010450 0 0) 0x7febf80d2da0 con 0x7febf80cef20 2024-03-28T13:16:17.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.164+0000 7febf5400640 1 -- 172.21.15.19:0/4073145597 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febdc003520 con 0x7febf80cef20 2024-03-28T13:16:17.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.165+0000 7febf5400640 1 -- 172.21.15.19:0/4073145597 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1237271522 0 0) 0x7febdc003520 con 0x7febf80cef20 2024-03-28T13:16:17.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.165+0000 7febf5400640 1 -- 172.21.15.19:0/4073145597 >> v2:172.21.15.188:3300/0 conn(0x7febf815e460 msgr2=0x7febf80b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:17.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.165+0000 7febf5400640 1 --2- 172.21.15.19:0/4073145597 >> v2:172.21.15.188:3300/0 conn(0x7febf815e460 0x7febf80b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.165+0000 7febf5400640 1 -- 172.21.15.19:0/4073145597 >> v1:172.21.15.188:6789/0 conn(0x7febf80d3e40 legacy=0x7febf80d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:17.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.165+0000 7febf5400640 1 -- 172.21.15.19:0/4073145597 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf80d3640 con 0x7febf80cef20 2024-03-28T13:16:17.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.165+0000 7febf5400640 1 -- 172.21.15.19:0/4073145597 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7febf00019f0 con 0x7febf80cef20 2024-03-28T13:16:17.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.165+0000 7febf5400640 1 -- 172.21.15.19:0/4073145597 >> v1:172.21.15.19:6789/0 conn(0x7febf80cef20 legacy=0x7febf80b7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:17.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.165+0000 7febf5400640 1 --2- 172.21.15.19:0/4073145597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febdc003a00 0x7febdc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:17.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.165+0000 7febf5400640 1 --2- 172.21.15.19:0/4073145597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febdc004800 0x7febdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:17.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.165+0000 7febf5400640 1 --2- 172.21.15.19:0/4073145597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febdc0051c0 0x7febdc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:17.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.165+0000 7febf5400640 1 -- 172.21.15.19:0/4073145597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7febf80d3360 con 0x7febdc0051c0 2024-03-28T13:16:17.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.165+0000 7febf5400640 1 -- 172.21.15.19:0/4073145597 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7febdc00c0c0 con 0x7febdc004800 2024-03-28T13:16:17.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.165+0000 7febf5400640 1 -- 172.21.15.19:0/4073145597 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7febdc00c2c0 con 0x7febdc003a00 2024-03-28T13:16:17.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.165+0000 7febf6800640 1 --2- 172.21.15.19:0/4073145597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febdc0051c0 0x7febdc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:17.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.166+0000 7febf5e00640 1 --2- 172.21.15.19:0/4073145597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febdc003a00 0x7febdc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:17.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.166+0000 7febefe00640 1 --2- 172.21.15.19:0/4073145597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febdc004800 0x7febdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:17.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.166+0000 7febf6800640 1 -- 172.21.15.19:0/4073145597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febdc003a00 msgr2=0x7febdc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:17.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.166+0000 7febf6800640 1 --2- 172.21.15.19:0/4073145597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febdc003a00 0x7febdc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.166+0000 7febf6800640 1 -- 172.21.15.19:0/4073145597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febdc004800 msgr2=0x7febdc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:17.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.166+0000 7febf6800640 1 --2- 172.21.15.19:0/4073145597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febdc004800 0x7febdc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.166+0000 7febf6800640 1 -- 172.21.15.19:0/4073145597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febdc003520 con 0x7febdc0051c0 2024-03-28T13:16:17.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.167+0000 7febf6800640 1 -- 172.21.15.19:0/4073145597 reap_dead start 2024-03-28T13:16:17.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.167+0000 7febf6800640 1 --2- 172.21.15.19:0/4073145597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febdc0051c0 0x7febdc00b9c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7febf000a790 tx=0x7febdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4047728724784c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:17.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.167+0000 7febf5400640 1 -- 172.21.15.19:0/4073145597 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf000a3b0 con 0x7febdc0051c0 2024-03-28T13:16:17.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.167+0000 7febf5400640 1 -- 172.21.15.19:0/4073145597 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febf0034670 con 0x7febdc0051c0 2024-03-28T13:16:17.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.167+0000 7febf5400640 1 -- 172.21.15.19:0/4073145597 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febf0034a60 con 0x7febdc0051c0 2024-03-28T13:16:17.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.168+0000 7febf7200640 1 -- 172.21.15.19:0/4073145597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febdc0051c0 msgr2=0x7febdc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:17.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.168+0000 7febf7200640 1 --2- 172.21.15.19:0/4073145597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febdc0051c0 0x7febdc00b9c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7febf000a790 tx=0x7febdc005830 comp rx=0 tx=0).stop 2024-03-28T13:16:17.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.168+0000 7febf7200640 1 -- 172.21.15.19:0/4073145597 shutdown_connections 2024-03-28T13:16:17.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.168+0000 7febf7200640 1 --2- 172.21.15.19:0/4073145597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febdc0051c0 0x7febdc00b9c0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.168+0000 7febf7200640 1 -- 172.21.15.19:0/4073145597 >> 172.21.15.19:0/4073145597 conn(0x7febf80678b0 msgr2=0x7febf80d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:17.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.168+0000 7febf7200640 1 -- 172.21.15.19:0/4073145597 shutdown_connections 2024-03-28T13:16:17.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.168+0000 7febf7200640 1 -- 172.21.15.19:0/4073145597 wait complete. 2024-03-28T13:16:17.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.170+0000 7febf7200640 1 Processor -- start 2024-03-28T13:16:17.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.170+0000 7febf7200640 1 -- start start 2024-03-28T13:16:17.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.171+0000 7febf7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febdc0051c0 0x7febf80aac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.171+0000 7febf7200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febf81535b0 0x7febf81f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.171+0000 7febf7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febf8153a10 0x7febf8160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.171+0000 7febf7200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7febf80b9fa0 con 0x7febdc0051c0 2024-03-28T13:16:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.171+0000 7febf7200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7febf80b9e20 con 0x7febf81535b0 2024-03-28T13:16:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.171+0000 7febf7200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7febf80ba120 con 0x7febf8153a10 2024-03-28T13:16:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.171+0000 7febf5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febdc0051c0 0x7febf80aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.171+0000 7febf5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febdc0051c0 0x7febf80aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49282/0 (socket says 172.21.15.19:49282) 2024-03-28T13:16:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.171+0000 7febf5e00640 1 -- 172.21.15.19:0/2316664150 learned_addr learned my addr 172.21.15.19:0/2316664150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.172+0000 7febf6800640 1 --2- 172.21.15.19:0/2316664150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febf8153a10 0x7febf8160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.172+0000 7febefe00640 1 --2- 172.21.15.19:0/2316664150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febf81535b0 0x7febf81f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.172+0000 7febf5e00640 1 -- 172.21.15.19:0/2316664150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febf8153a10 msgr2=0x7febf8160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.172+0000 7febf5e00640 1 --2- 172.21.15.19:0/2316664150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febf8153a10 0x7febf8160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.172+0000 7febf5e00640 1 -- 172.21.15.19:0/2316664150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febf81535b0 msgr2=0x7febf81f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.172+0000 7febf5e00640 1 --2- 172.21.15.19:0/2316664150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febf81535b0 0x7febf81f4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.172+0000 7febf5e00640 1 -- 172.21.15.19:0/2316664150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febf815f560 con 0x7febdc0051c0 2024-03-28T13:16:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.172+0000 7febefe00640 1 --2- 172.21.15.19:0/2316664150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febf81535b0 0x7febf81f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.172+0000 7febf5e00640 1 --2- 172.21.15.19:0/2316664150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febdc0051c0 0x7febf80aac40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7febe8011540 tx=0x7febe8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.173+0000 7febee000640 1 -- 172.21.15.19:0/2316664150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe8015930 con 0x7febdc0051c0 2024-03-28T13:16:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.173+0000 7febee000640 1 -- 172.21.15.19:0/2316664150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febe8015ad0 con 0x7febdc0051c0 2024-03-28T13:16:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.173+0000 7febee000640 1 -- 172.21.15.19:0/2316664150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe800f460 con 0x7febdc0051c0 2024-03-28T13:16:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.173+0000 7febf7200640 1 -- 172.21.15.19:0/2316664150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febf80ab180 con 0x7febdc0051c0 2024-03-28T13:16:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.173+0000 7febf7200640 1 -- 172.21.15.19:0/2316664150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febf815f800 con 0x7febdc0051c0 2024-03-28T13:16:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.174+0000 7febf7200640 1 -- 172.21.15.19:0/2316664150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febf8075940 con 0x7febdc0051c0 2024-03-28T13:16:17.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.175+0000 7febee000640 1 -- 172.21.15.19:0/2316664150 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7febe800f600 con 0x7febdc0051c0 2024-03-28T13:16:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.176+0000 7febee000640 1 --2- 172.21.15.19:0/2316664150 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7febd406c560 0x7febd406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.176+0000 7febee000640 1 -- 172.21.15.19:0/2316664150 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7febe808bc90 con 0x7febdc0051c0 2024-03-28T13:16:17.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.177+0000 7febefe00640 1 --2- 172.21.15.19:0/2316664150 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7febd406c560 0x7febd406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:17.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.178+0000 7febefe00640 1 --2- 172.21.15.19:0/2316664150 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7febd406c560 0x7febd406ea20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7febe0000c00 tx=0x7febe0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:17.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.180+0000 7febee000640 1 -- 172.21.15.19:0/2316664150 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7febe8019030 con 0x7febdc0051c0 2024-03-28T13:16:17.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.381+0000 7febf7200640 1 -- 172.21.15.19:0/2316664150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7febf80d0dd0 con 0x7febdc0051c0 2024-03-28T13:16:17.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.382+0000 7febee000640 1 -- 172.21.15.19:0/2316664150 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7febe805a550 con 0x7febdc0051c0 2024-03-28T13:16:17.382 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:16:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.384+0000 7febf7200640 1 -- 172.21.15.19:0/2316664150 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7febd406c560 msgr2=0x7febd406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.384+0000 7febf7200640 1 --2- 172.21.15.19:0/2316664150 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7febd406c560 0x7febd406ea20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7febe0000c00 tx=0x7febe0009040 comp rx=0 tx=0).stop 2024-03-28T13:16:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.385+0000 7febf7200640 1 -- 172.21.15.19:0/2316664150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febdc0051c0 msgr2=0x7febf80aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.385+0000 7febf7200640 1 --2- 172.21.15.19:0/2316664150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febdc0051c0 0x7febf80aac40 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7febe8011540 tx=0x7febe8011a00 comp rx=0 tx=0).stop 2024-03-28T13:16:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.385+0000 7febf7200640 1 -- 172.21.15.19:0/2316664150 shutdown_connections 2024-03-28T13:16:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.385+0000 7febf7200640 1 --2- 172.21.15.19:0/2316664150 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7febd406c560 0x7febd406ea20 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.385+0000 7febf7200640 1 --2- 172.21.15.19:0/2316664150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7febf8153a10 0x7febf8160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.385+0000 7febf7200640 1 --2- 172.21.15.19:0/2316664150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7febf81535b0 0x7febf81f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.385+0000 7febf7200640 1 --2- 172.21.15.19:0/2316664150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febdc0051c0 0x7febf80aac40 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.385+0000 7febf7200640 1 -- 172.21.15.19:0/2316664150 >> 172.21.15.19:0/2316664150 conn(0x7febf80678b0 msgr2=0x7febf80b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.385+0000 7febf7200640 1 -- 172.21.15.19:0/2316664150 shutdown_connections 2024-03-28T13:16:17.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.385+0000 7febf7200640 1 -- 172.21.15.19:0/2316664150 wait complete. 2024-03-28T13:16:17.405 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-03-28T13:16:17.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.624+0000 7fc95b000640 1 Processor -- start 2024-03-28T13:16:17.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.624+0000 7fc95b000640 1 -- start start 2024-03-28T13:16:17.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.626+0000 7fc95b000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc9540736f0 0x7fc95415f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:17.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.626+0000 7fc95b000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc9540cfc80 0x7fc9540d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:17.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.626+0000 7fc95b000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9540717c0 con 0x7fc95415e4d0 2024-03-28T13:16:17.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.626+0000 7fc95b000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc9540b3ab0 con 0x7fc9540cfc80 2024-03-28T13:16:17.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.626+0000 7fc95b000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc954071dd0 con 0x7fc9540736f0 2024-03-28T13:16:17.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.626+0000 7fc95a600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc9540cfc80 0x7fc9540d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:17.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.626+0000 7fc95a600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc9540cfc80 0x7fc9540d0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49298/0 (socket says 172.21.15.19:49298) 2024-03-28T13:16:17.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.626+0000 7fc95a600640 1 -- 172.21.15.19:0/2664982115 learned_addr learned my addr 172.21.15.19:0/2664982115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:17.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.626+0000 7fc959200640 1 --2- 172.21.15.19:0/2664982115 >> v2:172.21.15.188:3300/0 conn(0x7fc9540736f0 0x7fc95415f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:17.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.627+0000 7fc95a600640 1 -- 172.21.15.19:0/2664982115 >> v2:172.21.15.188:3300/0 conn(0x7fc9540736f0 msgr2=0x7fc95415f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:17.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.627+0000 7fc95a600640 1 --2- 172.21.15.19:0/2664982115 >> v2:172.21.15.188:3300/0 conn(0x7fc9540736f0 0x7fc95415f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.627+0000 7fc95a600640 1 -- 172.21.15.19:0/2664982115 >> v1:172.21.15.188:6789/0 conn(0x7fc95415e4d0 legacy=0x7fc9540ade40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:17.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.627+0000 7fc95a600640 1 -- 172.21.15.19:0/2664982115 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95415e8b0 con 0x7fc9540cfc80 2024-03-28T13:16:17.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.628+0000 7fc95a600640 1 --2- 172.21.15.19:0/2664982115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9540cfc80 0x7fc9540d0130 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc9540b3c30 tx=0x7fc94c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61273a7d287d2a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:17.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.628+0000 7fc953e00640 1 -- 172.21.15.19:0/2664982115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c02ca70 con 0x7fc9540cfc80 2024-03-28T13:16:17.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.628+0000 7fc953e00640 1 -- 172.21.15.19:0/2664982115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc94c02cc10 con 0x7fc9540cfc80 2024-03-28T13:16:17.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.628+0000 7fc953e00640 1 -- 172.21.15.19:0/2664982115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c0343f0 con 0x7fc9540cfc80 2024-03-28T13:16:17.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.629+0000 7fc95b000640 1 -- 172.21.15.19:0/2664982115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9540cfc80 msgr2=0x7fc9540d0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:17.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.629+0000 7fc95b000640 1 --2- 172.21.15.19:0/2664982115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9540cfc80 0x7fc9540d0130 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc9540b3c30 tx=0x7fc94c02bed0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.629+0000 7fc95b000640 1 -- 172.21.15.19:0/2664982115 shutdown_connections 2024-03-28T13:16:17.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.629+0000 7fc95b000640 1 --2- 172.21.15.19:0/2664982115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9540cfc80 0x7fc9540d0130 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.629+0000 7fc95b000640 1 --2- 172.21.15.19:0/2664982115 >> v2:172.21.15.188:3300/0 conn(0x7fc9540736f0 0x7fc95415f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.629+0000 7fc95b000640 1 -- 172.21.15.19:0/2664982115 >> 172.21.15.19:0/2664982115 conn(0x7fc95406b570 msgr2=0x7fc95406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:17.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.629+0000 7fc95b000640 1 -- 172.21.15.19:0/2664982115 shutdown_connections 2024-03-28T13:16:17.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.629+0000 7fc95b000640 1 -- 172.21.15.19:0/2664982115 wait complete. 2024-03-28T13:16:17.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.631+0000 7fc95b000640 1 Processor -- start 2024-03-28T13:16:17.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.631+0000 7fc95b000640 1 -- start start 2024-03-28T13:16:17.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.632+0000 7fc95b000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9540736f0 0x7fc954144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:17.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.632+0000 7fc95b000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9540cfc80 0x7fc9541f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:17.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.632+0000 7fc95b000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc95415e4d0 0x7fc954156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:17.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.632+0000 7fc95b000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc954071a60 con 0x7fc9540736f0 2024-03-28T13:16:17.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.632+0000 7fc95b000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc9540b3fe0 con 0x7fc95415e4d0 2024-03-28T13:16:17.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.632+0000 7fc95b000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc954072260 con 0x7fc9540cfc80 2024-03-28T13:16:17.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.632+0000 7fc959c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9540736f0 0x7fc954144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:17.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.632+0000 7fc959c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9540736f0 0x7fc954144b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49312/0 (socket says 172.21.15.19:49312) 2024-03-28T13:16:17.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.632+0000 7fc959c00640 1 -- 172.21.15.19:0/364817316 learned_addr learned my addr 172.21.15.19:0/364817316 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:17.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.633+0000 7fc95a600640 1 --2- 172.21.15.19:0/364817316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc95415e4d0 0x7fc954156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:17.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.633+0000 7fc959c00640 1 -- 172.21.15.19:0/364817316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9540cfc80 msgr2=0x7fc9541f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:17.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.633+0000 7fc959c00640 1 --2- 172.21.15.19:0/364817316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9540cfc80 0x7fc9541f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.633+0000 7fc959c00640 1 -- 172.21.15.19:0/364817316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc95415e4d0 msgr2=0x7fc954156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:17.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.633+0000 7fc959c00640 1 --2- 172.21.15.19:0/364817316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc95415e4d0 0x7fc954156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:17.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.633+0000 7fc959c00640 1 -- 172.21.15.19:0/364817316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9541f8510 con 0x7fc9540736f0 2024-03-28T13:16:17.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.633+0000 7fc959c00640 1 --2- 172.21.15.19:0/364817316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9540736f0 0x7fc954144b60 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc94800b420 tx=0x7fc94800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:17.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.633+0000 7fc951600640 1 -- 172.21.15.19:0/364817316 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948011830 con 0x7fc9540736f0 2024-03-28T13:16:17.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.634+0000 7fc951600640 1 -- 172.21.15.19:0/364817316 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc948016070 con 0x7fc9540736f0 2024-03-28T13:16:17.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.634+0000 7fc951600640 1 -- 172.21.15.19:0/364817316 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc948011c10 con 0x7fc9540736f0 2024-03-28T13:16:17.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.634+0000 7fc95b000640 1 -- 172.21.15.19:0/364817316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9541f7f50 con 0x7fc9540736f0 2024-03-28T13:16:17.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.634+0000 7fc95b000640 1 -- 172.21.15.19:0/364817316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9541564a0 con 0x7fc9540736f0 2024-03-28T13:16:17.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.635+0000 7fc95b000640 1 -- 172.21.15.19:0/364817316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc954075940 con 0x7fc9540736f0 2024-03-28T13:16:17.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.636+0000 7fc951600640 1 -- 172.21.15.19:0/364817316 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc94801b460 con 0x7fc9540736f0 2024-03-28T13:16:17.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.637+0000 7fc951600640 1 --2- 172.21.15.19:0/364817316 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc93406c530 0x7fc93406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:17.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.637+0000 7fc951600640 1 -- 172.21.15.19:0/364817316 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7fc94808af90 con 0x7fc9540736f0 2024-03-28T13:16:17.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.638+0000 7fc959200640 1 --2- 172.21.15.19:0/364817316 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc93406c530 0x7fc93406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:17.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.639+0000 7fc959200640 1 --2- 172.21.15.19:0/364817316 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc93406c530 0x7fc93406e9f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc9540b4560 tx=0x7fc940008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:17.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.642+0000 7fc951600640 1 -- 172.21.15.19:0/364817316 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc948020020 con 0x7fc9540736f0 2024-03-28T13:16:17.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:17.921+0000 7fc95b000640 1 -- 172.21.15.19:0/364817316 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7fc95415c410 con 0x7fc9540736f0 2024-03-28T13:16:18.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.458+0000 7fc951600640 1 -- 172.21.15.19:0/364817316 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v336) ==== 170+0+0 (secure 0 0 0) 0x7fc9480597b0 con 0x7fc9540736f0 2024-03-28T13:16:18.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.461+0000 7fc95b000640 1 -- 172.21.15.19:0/364817316 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc93406c530 msgr2=0x7fc93406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:18.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.461+0000 7fc95b000640 1 --2- 172.21.15.19:0/364817316 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc93406c530 0x7fc93406e9f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc9540b4560 tx=0x7fc940008040 comp rx=0 tx=0).stop 2024-03-28T13:16:18.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.461+0000 7fc95b000640 1 -- 172.21.15.19:0/364817316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9540736f0 msgr2=0x7fc954144b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:18.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.461+0000 7fc95b000640 1 --2- 172.21.15.19:0/364817316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9540736f0 0x7fc954144b60 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc94800b420 tx=0x7fc94800b8e0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.461+0000 7fc95b000640 1 -- 172.21.15.19:0/364817316 shutdown_connections 2024-03-28T13:16:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.461+0000 7fc95b000640 1 --2- 172.21.15.19:0/364817316 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc93406c530 0x7fc93406e9f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.461+0000 7fc95b000640 1 --2- 172.21.15.19:0/364817316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc95415e4d0 0x7fc954156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.461+0000 7fc95b000640 1 --2- 172.21.15.19:0/364817316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9540cfc80 0x7fc9541f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.461+0000 7fc95b000640 1 --2- 172.21.15.19:0/364817316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9540736f0 0x7fc954144b60 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.461+0000 7fc95b000640 1 -- 172.21.15.19:0/364817316 >> 172.21.15.19:0/364817316 conn(0x7fc95406b570 msgr2=0x7fc9540cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.462+0000 7fc95b000640 1 -- 172.21.15.19:0/364817316 shutdown_connections 2024-03-28T13:16:18.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.462+0000 7fc95b000640 1 -- 172.21.15.19:0/364817316 wait complete. 2024-03-28T13:16:18.486 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-28T13:16:18.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.714+0000 7fbc81000640 1 Processor -- start 2024-03-28T13:16:18.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.714+0000 7fbc81000640 1 -- start start 2024-03-28T13:16:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.716+0000 7fbc81000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc7c0d0280 0x7fbc7c0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.716+0000 7fbc81000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbc7c162160 0x7fbc7c1625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.716+0000 7fbc81000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc7c15ec80 con 0x7fbc7c141c10 2024-03-28T13:16:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.716+0000 7fbc81000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbc7c0d0ba0 con 0x7fbc7c162160 2024-03-28T13:16:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.716+0000 7fbc81000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbc7c15ef20 con 0x7fbc7c0d0280 2024-03-28T13:16:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.716+0000 7fbc7a800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbc7c162160 0x7fbc7c1625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.716+0000 7fbc73e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbc7c141c10 0x7fbc7c071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36300/0 (socket says 172.21.15.19:36300) 2024-03-28T13:16:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.716+0000 7fbc73e00640 1 -- 172.21.15.19:0/4010042435 learned_addr learned my addr 172.21.15.19:0/4010042435 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:18.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.716+0000 7fbc79e00640 1 --2- 172.21.15.19:0/4010042435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc7c0d0280 0x7fbc7c0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.717+0000 7fbc79400640 1 -- 172.21.15.19:0/4010042435 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3799054058 0 0) 0x7fbc7c15ec80 con 0x7fbc7c141c10 2024-03-28T13:16:18.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.717+0000 7fbc79400640 1 -- 172.21.15.19:0/4010042435 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc60003520 con 0x7fbc7c141c10 2024-03-28T13:16:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.717+0000 7fbc7a800640 1 -- 172.21.15.19:0/4010042435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc7c0d0280 msgr2=0x7fbc7c0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.717+0000 7fbc7a800640 1 --2- 172.21.15.19:0/4010042435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc7c0d0280 0x7fbc7c0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.717+0000 7fbc7a800640 1 -- 172.21.15.19:0/4010042435 >> v1:172.21.15.19:6789/0 conn(0x7fbc7c141c10 legacy=0x7fbc7c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.717+0000 7fbc7a800640 1 -- 172.21.15.19:0/4010042435 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7c15f200 con 0x7fbc7c162160 2024-03-28T13:16:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.717+0000 7fbc79e00640 1 --2- 172.21.15.19:0/4010042435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc7c0d0280 0x7fbc7c0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:16:18.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.718+0000 7fbc7a800640 1 --2- 172.21.15.19:0/4010042435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7c162160 0x7fbc7c1625e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fbc740097a0 tx=0x7fbc7402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03d79a3e706d760 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:18.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.718+0000 7fbc79400640 1 -- 172.21.15.19:0/4010042435 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74035020 con 0x7fbc7c162160 2024-03-28T13:16:18.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.718+0000 7fbc79400640 1 -- 172.21.15.19:0/4010042435 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc74031810 con 0x7fbc7c162160 2024-03-28T13:16:18.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.718+0000 7fbc79400640 1 -- 172.21.15.19:0/4010042435 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74031be0 con 0x7fbc7c162160 2024-03-28T13:16:18.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.719+0000 7fbc81000640 1 -- 172.21.15.19:0/4010042435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7c162160 msgr2=0x7fbc7c1625e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:18.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.719+0000 7fbc81000640 1 --2- 172.21.15.19:0/4010042435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7c162160 0x7fbc7c1625e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fbc740097a0 tx=0x7fbc7402b8b0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.719+0000 7fbc81000640 1 -- 172.21.15.19:0/4010042435 shutdown_connections 2024-03-28T13:16:18.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.719+0000 7fbc81000640 1 --2- 172.21.15.19:0/4010042435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7c162160 0x7fbc7c1625e0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.719+0000 7fbc81000640 1 --2- 172.21.15.19:0/4010042435 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc7c0d0280 0x7fbc7c0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.719+0000 7fbc81000640 1 -- 172.21.15.19:0/4010042435 >> 172.21.15.19:0/4010042435 conn(0x7fbc7c06b750 msgr2=0x7fbc7c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.719+0000 7fbc81000640 1 -- 172.21.15.19:0/4010042435 shutdown_connections 2024-03-28T13:16:18.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.719+0000 7fbc81000640 1 -- 172.21.15.19:0/4010042435 wait complete. 2024-03-28T13:16:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.722+0000 7fbc81000640 1 Processor -- start 2024-03-28T13:16:18.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.722+0000 7fbc81000640 1 -- start start 2024-03-28T13:16:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.723+0000 7fbc81000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc7c0d0280 0x7fbc7c09dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.723+0000 7fbc81000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7c141c10 0x7fbc7c0ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.723+0000 7fbc81000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc7c162160 0x7fbc7c1ea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.723+0000 7fbc81000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c158e90 con 0x7fbc7c141c10 2024-03-28T13:16:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.723+0000 7fbc81000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c15f7c0 con 0x7fbc7c162160 2024-03-28T13:16:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.723+0000 7fbc81000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbc7c0aee80 con 0x7fbc7c0d0280 2024-03-28T13:16:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.723+0000 7fbc73e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7c141c10 0x7fbc7c0ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:18.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.724+0000 7fbc73e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7c141c10 0x7fbc7c0ae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49336/0 (socket says 172.21.15.19:49336) 2024-03-28T13:16:18.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.724+0000 7fbc73e00640 1 -- 172.21.15.19:0/3870349744 learned_addr learned my addr 172.21.15.19:0/3870349744 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.724+0000 7fbc7a800640 1 --2- 172.21.15.19:0/3870349744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc7c162160 0x7fbc7c1ea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.724+0000 7fbc79e00640 1 --2- 172.21.15.19:0/3870349744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc7c0d0280 0x7fbc7c09dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.724+0000 7fbc73e00640 1 -- 172.21.15.19:0/3870349744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc7c0d0280 msgr2=0x7fbc7c09dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.724+0000 7fbc73e00640 1 --2- 172.21.15.19:0/3870349744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc7c0d0280 0x7fbc7c09dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.724+0000 7fbc73e00640 1 -- 172.21.15.19:0/3870349744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc7c162160 msgr2=0x7fbc7c1ea3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.724+0000 7fbc73e00640 1 --2- 172.21.15.19:0/3870349744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc7c162160 0x7fbc7c1ea3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.724+0000 7fbc73e00640 1 -- 172.21.15.19:0/3870349744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7c0ad1b0 con 0x7fbc7c141c10 2024-03-28T13:16:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.725+0000 7fbc73e00640 1 --2- 172.21.15.19:0/3870349744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7c141c10 0x7fbc7c0ae4a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400c430 tx=0x7fbc6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:18.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.725+0000 7fbc72000640 1 -- 172.21.15.19:0/3870349744 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64017070 con 0x7fbc7c141c10 2024-03-28T13:16:18.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.725+0000 7fbc72000640 1 -- 172.21.15.19:0/3870349744 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc64005ce0 con 0x7fbc7c141c10 2024-03-28T13:16:18.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.725+0000 7fbc72000640 1 -- 172.21.15.19:0/3870349744 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64007310 con 0x7fbc7c141c10 2024-03-28T13:16:18.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.725+0000 7fbc81000640 1 -- 172.21.15.19:0/3870349744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7c0ad490 con 0x7fbc7c141c10 2024-03-28T13:16:18.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.725+0000 7fbc81000640 1 -- 172.21.15.19:0/3870349744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc7c0adc90 con 0x7fbc7c141c10 2024-03-28T13:16:18.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.727+0000 7fbc81000640 1 -- 172.21.15.19:0/3870349744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc7c067db0 con 0x7fbc7c141c10 2024-03-28T13:16:18.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.727+0000 7fbc72000640 1 -- 172.21.15.19:0/3870349744 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbc6401a070 con 0x7fbc7c141c10 2024-03-28T13:16:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.728+0000 7fbc72000640 1 --2- 172.21.15.19:0/3870349744 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbc5806c560 0x7fbc5806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.728+0000 7fbc72000640 1 -- 172.21.15.19:0/3870349744 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7fbc6408a8c0 con 0x7fbc7c141c10 2024-03-28T13:16:18.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.729+0000 7fbc79e00640 1 --2- 172.21.15.19:0/3870349744 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbc5806c560 0x7fbc5806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:18.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.730+0000 7fbc79e00640 1 --2- 172.21.15.19:0/3870349744 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbc5806c560 0x7fbc5806ea20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009f70 tx=0x7fbc6c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:18.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.733+0000 7fbc72000640 1 -- 172.21.15.19:0/3870349744 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbc7c067db0 con 0x7fbc7c141c10 2024-03-28T13:16:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.937+0000 7fbc81000640 1 -- 172.21.15.19:0/3870349744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbc7c0685c0 con 0x7fbc7c141c10 2024-03-28T13:16:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.943+0000 7fbc72000640 1 -- 172.21.15.19:0/3870349744 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fbc640590e0 con 0x7fbc7c141c10 2024-03-28T13:16:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.946+0000 7fbc81000640 1 -- 172.21.15.19:0/3870349744 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbc5806c560 msgr2=0x7fbc5806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.946+0000 7fbc81000640 1 --2- 172.21.15.19:0/3870349744 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbc5806c560 0x7fbc5806ea20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009f70 tx=0x7fbc6c00b040 comp rx=0 tx=0).stop 2024-03-28T13:16:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.946+0000 7fbc81000640 1 -- 172.21.15.19:0/3870349744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7c141c10 msgr2=0x7fbc7c0ae4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.946+0000 7fbc81000640 1 --2- 172.21.15.19:0/3870349744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7c141c10 0x7fbc7c0ae4a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400c430 tx=0x7fbc6400c8f0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.946+0000 7fbc81000640 1 -- 172.21.15.19:0/3870349744 shutdown_connections 2024-03-28T13:16:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.946+0000 7fbc81000640 1 --2- 172.21.15.19:0/3870349744 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbc5806c560 0x7fbc5806ea20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.946+0000 7fbc81000640 1 --2- 172.21.15.19:0/3870349744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc7c162160 0x7fbc7c1ea3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.946+0000 7fbc81000640 1 --2- 172.21.15.19:0/3870349744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbc7c141c10 0x7fbc7c0ae4a0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.946+0000 7fbc81000640 1 --2- 172.21.15.19:0/3870349744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc7c0d0280 0x7fbc7c09dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.946+0000 7fbc81000640 1 -- 172.21.15.19:0/3870349744 >> 172.21.15.19:0/3870349744 conn(0x7fbc7c06b750 msgr2=0x7fbc7c160e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.946+0000 7fbc81000640 1 -- 172.21.15.19:0/3870349744 shutdown_connections 2024-03-28T13:16:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:18.946+0000 7fbc81000640 1 -- 172.21.15.19:0/3870349744 wait complete. 2024-03-28T13:16:18.967 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-28T13:16:18.967 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:16:19.085 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:16:19.207 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-28T13:16:19.208 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:16:19.208 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:16:19.269 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:16:19.270 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:16:19.270 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:16:19.327 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:16:19.327 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:16:19.327 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:16:19.387 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:16:19.387 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:16:19.387 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:16:19.445 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:16:19.445 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-28T13:16:19.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:16:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.673+0000 7fc16b200640 1 Processor -- start 2024-03-28T13:16:19.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.673+0000 7fc16b200640 1 -- start start 2024-03-28T13:16:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.675+0000 7fc16b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0d04d0 0x7fc16c0b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.675+0000 7fc16b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc16c0b3150 con 0x7fc16c0cc0b0 2024-03-28T13:16:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.675+0000 7fc16b200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc16c1600e0 con 0x7fc16c15fc50 2024-03-28T13:16:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.675+0000 7fc16b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc16c15c610 con 0x7fc16c0d04d0 2024-03-28T13:16:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.675+0000 7fc169e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc16c0cc0b0 0x7fc16c06dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36304/0 (socket says 172.21.15.19:36304) 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.675+0000 7fc169e00640 1 -- 172.21.15.19:0/1854859043 learned_addr learned my addr 172.21.15.19:0/1854859043 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.675+0000 7fc168a00640 1 -- 172.21.15.19:0/1854859043 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1013580855 0 0) 0x7fc16c0b3150 con 0x7fc16c0cc0b0 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.675+0000 7fc168a00640 1 -- 172.21.15.19:0/1854859043 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc150003520 con 0x7fc16c0cc0b0 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.675+0000 7fc16a800640 1 --2- 172.21.15.19:0/1854859043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0d04d0 0x7fc16c0b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.676+0000 7fc168a00640 1 -- 172.21.15.19:0/1854859043 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2372674259 0 0) 0x7fc150003520 con 0x7fc16c0cc0b0 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.676+0000 7fc168a00640 1 -- 172.21.15.19:0/1854859043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0d04d0 msgr2=0x7fc16c0b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.676+0000 7fc168a00640 1 --2- 172.21.15.19:0/1854859043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0d04d0 0x7fc16c0b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.676+0000 7fc168a00640 1 -- 172.21.15.19:0/1854859043 >> v1:172.21.15.188:6789/0 conn(0x7fc16c15fc50 legacy=0x7fc16c0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.676+0000 7fc168a00640 1 -- 172.21.15.19:0/1854859043 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16c1603c0 con 0x7fc16c0cc0b0 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.676+0000 7fc168a00640 1 -- 172.21.15.19:0/1854859043 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fc15c001670 con 0x7fc16c0cc0b0 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.676+0000 7fc168a00640 1 -- 172.21.15.19:0/1854859043 >> v1:172.21.15.19:6789/0 conn(0x7fc16c0cc0b0 legacy=0x7fc16c06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.676+0000 7fc168a00640 1 --2- 172.21.15.19:0/1854859043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1500039e0 0x7fc150003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.676+0000 7fc168a00640 1 --2- 172.21.15.19:0/1854859043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc150004800 0x7fc150004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.676+0000 7fc168a00640 1 --2- 172.21.15.19:0/1854859043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0d04d0 0x7fc150005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.676+0000 7fc168a00640 1 -- 172.21.15.19:0/1854859043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c15c610 con 0x7fc150004800 2024-03-28T13:16:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.676+0000 7fc168a00640 1 -- 172.21.15.19:0/1854859043 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc15000bb80 con 0x7fc1500039e0 2024-03-28T13:16:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.676+0000 7fc168a00640 1 -- 172.21.15.19:0/1854859043 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc15000bd80 con 0x7fc16c0d04d0 2024-03-28T13:16:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.676+0000 7fc169400640 1 --2- 172.21.15.19:0/1854859043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc150004800 0x7fc150004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.677+0000 7fc169400640 1 -- 172.21.15.19:0/1854859043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0d04d0 msgr2=0x7fc150005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.677+0000 7fc169400640 1 --2- 172.21.15.19:0/1854859043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c0d04d0 0x7fc150005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.677+0000 7fc169e00640 1 --2- 172.21.15.19:0/1854859043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1500039e0 0x7fc150003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.677+0000 7fc169400640 1 -- 172.21.15.19:0/1854859043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1500039e0 msgr2=0x7fc150003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.677+0000 7fc169400640 1 --2- 172.21.15.19:0/1854859043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1500039e0 0x7fc150003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.677+0000 7fc169400640 1 -- 172.21.15.19:0/1854859043 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc150003520 con 0x7fc150004800 2024-03-28T13:16:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.678+0000 7fc169400640 1 --2- 172.21.15.19:0/1854859043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc150004800 0x7fc150004c80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc158001750 tx=0x7fc150005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708bf459a6802a0f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.678+0000 7fc168a00640 1 -- 172.21.15.19:0/1854859043 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158001370 con 0x7fc150004800 2024-03-28T13:16:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.678+0000 7fc168a00640 1 -- 172.21.15.19:0/1854859043 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1580052d0 con 0x7fc150004800 2024-03-28T13:16:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.678+0000 7fc168a00640 1 -- 172.21.15.19:0/1854859043 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc158003830 con 0x7fc150004800 2024-03-28T13:16:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.679+0000 7fc16b200640 1 -- 172.21.15.19:0/1854859043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc150004800 msgr2=0x7fc150004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.679+0000 7fc16b200640 1 --2- 172.21.15.19:0/1854859043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc150004800 0x7fc150004c80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc158001750 tx=0x7fc150005830 comp rx=0 tx=0).stop 2024-03-28T13:16:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.679+0000 7fc16a800640 1 -- 172.21.15.19:0/1854859043 reap_dead start 2024-03-28T13:16:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.679+0000 7fc16b200640 1 -- 172.21.15.19:0/1854859043 shutdown_connections 2024-03-28T13:16:19.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.679+0000 7fc16b200640 1 -- 172.21.15.19:0/1854859043 >> 172.21.15.19:0/1854859043 conn(0x7fc16c0ccad0 msgr2=0x7fc16c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:19.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.679+0000 7fc16b200640 1 -- 172.21.15.19:0/1854859043 shutdown_connections 2024-03-28T13:16:19.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.679+0000 7fc16b200640 1 -- 172.21.15.19:0/1854859043 wait complete. 2024-03-28T13:16:19.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.681+0000 7fc16b200640 1 Processor -- start 2024-03-28T13:16:19.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.681+0000 7fc16b200640 1 -- start start 2024-03-28T13:16:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.682+0000 7fc16b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc150004800 0x7fc16c1f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.682+0000 7fc16b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c1e60d0 0x7fc16c1e6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.682+0000 7fc16b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c1310b0 0x7fc16c131560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.682+0000 7fc16b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c0cc490 con 0x7fc150004800 2024-03-28T13:16:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.682+0000 7fc16b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc16c160980 con 0x7fc16c1310b0 2024-03-28T13:16:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.682+0000 7fc16b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc16c0cc610 con 0x7fc16c1e60d0 2024-03-28T13:16:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.682+0000 7fc169e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc150004800 0x7fc16c1f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.682+0000 7fc169e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc150004800 0x7fc16c1f46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49354/0 (socket says 172.21.15.19:49354) 2024-03-28T13:16:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.682+0000 7fc169e00640 1 -- 172.21.15.19:0/574774836 learned_addr learned my addr 172.21.15.19:0/574774836 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:19.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.682+0000 7fc169400640 1 --2- 172.21.15.19:0/574774836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c1e60d0 0x7fc16c1e6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:19.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.682+0000 7fc169e00640 1 -- 172.21.15.19:0/574774836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c1e60d0 msgr2=0x7fc16c1e6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:19.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.682+0000 7fc16a800640 1 --2- 172.21.15.19:0/574774836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c1310b0 0x7fc16c131560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:19.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.682+0000 7fc169e00640 1 --2- 172.21.15.19:0/574774836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c1e60d0 0x7fc16c1e6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.682+0000 7fc169e00640 1 -- 172.21.15.19:0/574774836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c1310b0 msgr2=0x7fc16c131560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.682+0000 7fc169e00640 1 --2- 172.21.15.19:0/574774836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c1310b0 0x7fc16c131560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.683+0000 7fc169e00640 1 -- 172.21.15.19:0/574774836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc16c1e6cb0 con 0x7fc150004800 2024-03-28T13:16:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.683+0000 7fc169e00640 1 --2- 172.21.15.19:0/574774836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc150004800 0x7fc16c1f46f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00d7a0 tx=0x7fc15c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.683+0000 7fc162000640 1 -- 172.21.15.19:0/574774836 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c019070 con 0x7fc150004800 2024-03-28T13:16:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.683+0000 7fc162000640 1 -- 172.21.15.19:0/574774836 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc15c0074e0 con 0x7fc150004800 2024-03-28T13:16:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.683+0000 7fc162000640 1 -- 172.21.15.19:0/574774836 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc15c004a00 con 0x7fc150004800 2024-03-28T13:16:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.683+0000 7fc16b200640 1 -- 172.21.15.19:0/574774836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16c1e6f90 con 0x7fc150004800 2024-03-28T13:16:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.683+0000 7fc16b200640 1 -- 172.21.15.19:0/574774836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16c1e7550 con 0x7fc150004800 2024-03-28T13:16:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.684+0000 7fc16b200640 1 -- 172.21.15.19:0/574774836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc16c075940 con 0x7fc150004800 2024-03-28T13:16:19.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.685+0000 7fc162000640 1 -- 172.21.15.19:0/574774836 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc15c01c070 con 0x7fc150004800 2024-03-28T13:16:19.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.686+0000 7fc162000640 1 --2- 172.21.15.19:0/574774836 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc14c06c530 0x7fc14c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:19.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.687+0000 7fc162000640 1 -- 172.21.15.19:0/574774836 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7fc15c08aea0 con 0x7fc150004800 2024-03-28T13:16:19.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.687+0000 7fc169400640 1 --2- 172.21.15.19:0/574774836 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc14c06c530 0x7fc14c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:19.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.688+0000 7fc169400640 1 --2- 172.21.15.19:0/574774836 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc14c06c530 0x7fc14c06e9f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc158033000 tx=0x7fc158009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:19.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.692+0000 7fc162000640 1 -- 172.21.15.19:0/574774836 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc15c0597f0 con 0x7fc150004800 2024-03-28T13:16:19.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.938+0000 7fc16b200640 1 -- 172.21.15.19:0/574774836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc16c073830 con 0x7fc150004800 2024-03-28T13:16:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.939+0000 7fc162000640 1 -- 172.21.15.19:0/574774836 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+3872 (secure 0 0 0) 0x7fc15c0590f0 con 0x7fc150004800 2024-03-28T13:16:19.939 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:19.939 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":55,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5351,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/144726418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":144726418},{"type":"v1","addr":"172.21.15.188:6836","nonce":144726418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5363,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/2642581607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":2642581607},{"type":"v1","addr":"172.21.15.19:6835","nonce":2642581607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5677,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/1721039756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":1721039756},{"type":"v1","addr":"172.21.15.188:6837","nonce":1721039756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":7101,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3785087985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3785087985},{"type":"v1","addr":"172.21.15.19:6834","nonce":3785087985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":55}],"filesystems":[]} 2024-03-28T13:16:19.939 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 55 2024-03-28T13:16:19.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.942+0000 7fc16b200640 1 -- 172.21.15.19:0/574774836 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc14c06c530 msgr2=0x7fc14c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.942+0000 7fc16b200640 1 --2- 172.21.15.19:0/574774836 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc14c06c530 0x7fc14c06e9f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc158033000 tx=0x7fc158009030 comp rx=0 tx=0).stop 2024-03-28T13:16:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.942+0000 7fc16b200640 1 -- 172.21.15.19:0/574774836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc150004800 msgr2=0x7fc16c1f46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.942+0000 7fc16b200640 1 --2- 172.21.15.19:0/574774836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc150004800 0x7fc16c1f46f0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00d7a0 tx=0x7fc15c00dc60 comp rx=0 tx=0).stop 2024-03-28T13:16:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.942+0000 7fc16b200640 1 -- 172.21.15.19:0/574774836 shutdown_connections 2024-03-28T13:16:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.942+0000 7fc16b200640 1 --2- 172.21.15.19:0/574774836 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc14c06c530 0x7fc14c06e9f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.942+0000 7fc16b200640 1 --2- 172.21.15.19:0/574774836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc16c1310b0 0x7fc16c131560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.942+0000 7fc16b200640 1 --2- 172.21.15.19:0/574774836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc16c1e60d0 0x7fc16c1e6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.942+0000 7fc16b200640 1 --2- 172.21.15.19:0/574774836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc150004800 0x7fc16c1f46f0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.942+0000 7fc16b200640 1 -- 172.21.15.19:0/574774836 >> 172.21.15.19:0/574774836 conn(0x7fc16c0ccad0 msgr2=0x7fc16c0ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.942+0000 7fc16b200640 1 -- 172.21.15.19:0/574774836 shutdown_connections 2024-03-28T13:16:19.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:19.942+0000 7fc16b200640 1 -- 172.21.15.19:0/574774836 wait complete. 2024-03-28T13:16:19.963 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-28T13:16:19.963 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-28T13:16:19.963 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-28T13:16:19.963 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:16:19.963 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-28T13:16:19.964 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-28T13:16:19.964 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-28T13:16:19.964 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:16:19.964 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-28T13:16:19.964 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-28T13:16:19.964 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-28T13:16:19.964 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:16:19.965 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-28T13:16:19.965 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-28T13:16:19.965 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-28T13:16:19.965 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:16:19.989 INFO:tasks.ceph.mds.b:Stopped 2024-03-28T13:16:19.989 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-28T13:16:19.991 INFO:tasks.ceph.mds.a:Stopped 2024-03-28T13:16:19.991 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-28T13:16:19.992 INFO:tasks.ceph.mds.d:Stopped 2024-03-28T13:16:19.992 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-28T13:16:19.994 INFO:tasks.ceph.mds.c:Stopped 2024-03-28T13:16:19.994 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-28T13:16:20.039 INFO:tasks.ceph.mds.b:Started 2024-03-28T13:16:20.040 DEBUG:teuthology.parallel:result is None 2024-03-28T13:16:20.041 INFO:tasks.ceph.mds.a:Started 2024-03-28T13:16:20.041 DEBUG:teuthology.parallel:result is None 2024-03-28T13:16:20.046 INFO:tasks.ceph.mds.d:Started 2024-03-28T13:16:20.047 DEBUG:teuthology.parallel:result is None 2024-03-28T13:16:20.047 INFO:tasks.ceph.mds.c:Started 2024-03-28T13:16:20.048 DEBUG:teuthology.parallel:result is None 2024-03-28T13:16:20.048 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-28T13:16:20.203 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-03-28T13:16:20.204 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-03-28T13:16:20.207 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-03-28T13:16:20.208 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-03-28T13:16:20.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.247+0000 7f4679000640 1 Processor -- start 2024-03-28T13:16:20.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.248+0000 7f4679000640 1 -- start start 2024-03-28T13:16:20.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.249+0000 7f4679000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4674147570 0x7f46740ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.249+0000 7f4679000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4674147340 con 0x7f467406bc80 2024-03-28T13:16:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.249+0000 7f4679000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46740cf260 con 0x7f4674146f60 2024-03-28T13:16:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.249+0000 7f4679000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f46740cf030 con 0x7f4674147570 2024-03-28T13:16:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.249+0000 7f4673e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4674147570 0x7f46740ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:20.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.249+0000 7f466aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f467406bc80 0x7f4674161ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36332/0 (socket says 172.21.15.19:36332) 2024-03-28T13:16:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.249+0000 7f466aa00640 1 -- 172.21.15.19:0/1652158178 learned_addr learned my addr 172.21.15.19:0/1652158178 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.250+0000 7f4672a00640 1 -- 172.21.15.19:0/1652158178 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3705331126 0 0) 0x7f4674147340 con 0x7f467406bc80 2024-03-28T13:16:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.250+0000 7f4672a00640 1 -- 172.21.15.19:0/1652158178 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4658003520 con 0x7f467406bc80 2024-03-28T13:16:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.250+0000 7f4672a00640 1 -- 172.21.15.19:0/1652158178 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3448202558 0 0) 0x7f46740cf260 con 0x7f4674146f60 2024-03-28T13:16:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.250+0000 7f4672a00640 1 -- 172.21.15.19:0/1652158178 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4674147340 con 0x7f4674146f60 2024-03-28T13:16:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.250+0000 7f4673e00640 1 -- 172.21.15.19:0/1652158178 >> v1:172.21.15.188:6789/0 conn(0x7f4674146f60 legacy=0x7f467406b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.250+0000 7f4673e00640 1 -- 172.21.15.19:0/1652158178 >> v1:172.21.15.19:6789/0 conn(0x7f467406bc80 legacy=0x7f4674161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:20.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.250+0000 7f4673e00640 1 -- 172.21.15.19:0/1652158178 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467406e570 con 0x7f4674147570 2024-03-28T13:16:20.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.251+0000 7f4673e00640 1 --2- 172.21.15.19:0/1652158178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4674147570 0x7f46740ceaf0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f466c0097a0 tx=0x7f466c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9757ccbbad648fd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:20.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.252+0000 7f4672a00640 1 -- 172.21.15.19:0/1652158178 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c034070 con 0x7f4674147570 2024-03-28T13:16:20.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.252+0000 7f4672a00640 1 -- 172.21.15.19:0/1652158178 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f466c030610 con 0x7f4674147570 2024-03-28T13:16:20.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.252+0000 7f4672a00640 1 -- 172.21.15.19:0/1652158178 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c0309c0 con 0x7f4674147570 2024-03-28T13:16:20.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.252+0000 7f4679000640 1 -- 172.21.15.19:0/1652158178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4674147570 msgr2=0x7f46740ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:20.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.252+0000 7f4679000640 1 --2- 172.21.15.19:0/1652158178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4674147570 0x7f46740ceaf0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f466c0097a0 tx=0x7f466c02b960 comp rx=0 tx=0).stop 2024-03-28T13:16:20.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.252+0000 7f4679000640 1 -- 172.21.15.19:0/1652158178 shutdown_connections 2024-03-28T13:16:20.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.252+0000 7f4679000640 1 --2- 172.21.15.19:0/1652158178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4674147570 0x7f46740ceaf0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:20.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.252+0000 7f4679000640 1 -- 172.21.15.19:0/1652158178 >> 172.21.15.19:0/1652158178 conn(0x7f46740b2c90 msgr2=0x7f46740b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:20.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.252+0000 7f4679000640 1 -- 172.21.15.19:0/1652158178 shutdown_connections 2024-03-28T13:16:20.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.252+0000 7f4679000640 1 -- 172.21.15.19:0/1652158178 wait complete. 2024-03-28T13:16:20.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.254+0000 7f4679000640 1 Processor -- start 2024-03-28T13:16:20.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.254+0000 7f4679000640 1 -- start start 2024-03-28T13:16:20.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.255+0000 7f4679000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f467406bc80 0x7f467413f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:20.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.255+0000 7f4679000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4674146f60 0x7f46740afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:20.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.255+0000 7f4679000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4674147570 0x7f46740b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:20.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.255+0000 7f4679000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f467406ebc0 con 0x7f467406bc80 2024-03-28T13:16:20.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.255+0000 7f4679000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f467406e320 con 0x7f4674146f60 2024-03-28T13:16:20.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.255+0000 7f4679000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f467406f050 con 0x7f4674147570 2024-03-28T13:16:20.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.256+0000 7f4673400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f467406bc80 0x7f467413f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:20.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.256+0000 7f4673400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f467406bc80 0x7f467413f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49402/0 (socket says 172.21.15.19:49402) 2024-03-28T13:16:20.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.256+0000 7f4673400640 1 -- 172.21.15.19:0/2299459402 learned_addr learned my addr 172.21.15.19:0/2299459402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:20.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.256+0000 7f466aa00640 1 --2- 172.21.15.19:0/2299459402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4674146f60 0x7f46740afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:20.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.256+0000 7f4673400640 1 -- 172.21.15.19:0/2299459402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4674147570 msgr2=0x7f46740b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:20.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.256+0000 7f4673400640 1 --2- 172.21.15.19:0/2299459402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4674147570 0x7f46740b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:20.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.256+0000 7f4673400640 1 -- 172.21.15.19:0/2299459402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4674146f60 msgr2=0x7f46740afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:20.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.256+0000 7f4673400640 1 --2- 172.21.15.19:0/2299459402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4674146f60 0x7f46740afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:20.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.256+0000 7f4673400640 1 -- 172.21.15.19:0/2299459402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46741e3b10 con 0x7f467406bc80 2024-03-28T13:16:20.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.256+0000 7f466aa00640 1 --2- 172.21.15.19:0/2299459402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4674146f60 0x7f46740afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:20.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.256+0000 7f4673400640 1 --2- 172.21.15.19:0/2299459402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f467406bc80 0x7f467413f9c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f466400c8a0 tx=0x7f466400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.257+0000 7f466be00640 1 -- 172.21.15.19:0/2299459402 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664003b70 con 0x7f467406bc80 2024-03-28T13:16:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.257+0000 7f466be00640 1 -- 172.21.15.19:0/2299459402 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4664003d30 con 0x7f467406bc80 2024-03-28T13:16:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.257+0000 7f4679000640 1 -- 172.21.15.19:0/2299459402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46740ae480 con 0x7f467406bc80 2024-03-28T13:16:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.257+0000 7f466be00640 1 -- 172.21.15.19:0/2299459402 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466400aa90 con 0x7f467406bc80 2024-03-28T13:16:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.257+0000 7f4679000640 1 -- 172.21.15.19:0/2299459402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f467413ff00 con 0x7f467406bc80 2024-03-28T13:16:20.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.258+0000 7f4679000640 1 -- 172.21.15.19:0/2299459402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4674075940 con 0x7f467406bc80 2024-03-28T13:16:20.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.259+0000 7f466be00640 1 -- 172.21.15.19:0/2299459402 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f466400ac30 con 0x7f467406bc80 2024-03-28T13:16:20.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.260+0000 7f466be00640 1 --2- 172.21.15.19:0/2299459402 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f465006c530 0x7f465006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:20.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.260+0000 7f466be00640 1 -- 172.21.15.19:0/2299459402 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5029+0+0 (secure 0 0 0) 0x7f466408b400 con 0x7f467406bc80 2024-03-28T13:16:20.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.260+0000 7f466aa00640 1 --2- 172.21.15.19:0/2299459402 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f465006c530 0x7f465006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:20.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.261+0000 7f466aa00640 1 --2- 172.21.15.19:0/2299459402 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f465006c530 0x7f465006e9f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f46740b1f50 tx=0x7f465c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:20.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.264+0000 7f466be00640 1 -- 172.21.15.19:0/2299459402 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4664059cc0 con 0x7f467406bc80 2024-03-28T13:16:20.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:20.480+0000 7f4679000640 1 -- 172.21.15.19:0/2299459402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f4674072770 con 0x7f467406bc80 2024-03-28T13:16:21.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.505+0000 7f466be00640 1 -- 172.21.15.19:0/2299459402 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v88) ==== 97+0+0 (secure 0 0 0) 0x7f46640595c0 con 0x7f467406bc80 2024-03-28T13:16:21.506 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-03-28T13:16:21.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.509+0000 7f4679000640 1 -- 172.21.15.19:0/2299459402 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f465006c530 msgr2=0x7f465006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:21.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.509+0000 7f4679000640 1 --2- 172.21.15.19:0/2299459402 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f465006c530 0x7f465006e9f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f46740b1f50 tx=0x7f465c009040 comp rx=0 tx=0).stop 2024-03-28T13:16:21.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.509+0000 7f4679000640 1 -- 172.21.15.19:0/2299459402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f467406bc80 msgr2=0x7f467413f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:21.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.509+0000 7f4679000640 1 --2- 172.21.15.19:0/2299459402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f467406bc80 0x7f467413f9c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f466400c8a0 tx=0x7f466400cd60 comp rx=0 tx=0).stop 2024-03-28T13:16:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.510+0000 7f4679000640 1 -- 172.21.15.19:0/2299459402 shutdown_connections 2024-03-28T13:16:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.510+0000 7f4679000640 1 --2- 172.21.15.19:0/2299459402 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f465006c530 0x7f465006e9f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.510+0000 7f4679000640 1 --2- 172.21.15.19:0/2299459402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4674147570 0x7f46740b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.510+0000 7f4679000640 1 --2- 172.21.15.19:0/2299459402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4674146f60 0x7f46740afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.510+0000 7f4679000640 1 --2- 172.21.15.19:0/2299459402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f467406bc80 0x7f467413f9c0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.510+0000 7f4679000640 1 -- 172.21.15.19:0/2299459402 >> 172.21.15.19:0/2299459402 conn(0x7f46740b2c90 msgr2=0x7f46741402e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.510+0000 7f4679000640 1 -- 172.21.15.19:0/2299459402 shutdown_connections 2024-03-28T13:16:21.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.510+0000 7f4679000640 1 -- 172.21.15.19:0/2299459402 wait complete. 2024-03-28T13:16:21.531 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-28T13:16:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.745+0000 7fa685400640 1 Processor -- start 2024-03-28T13:16:21.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.745+0000 7fa685400640 1 -- start start 2024-03-28T13:16:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.747+0000 7fa685400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa680072f50 0x7fa6800b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.747+0000 7fa685400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa680144eb0 con 0x7fa680144ad0 2024-03-28T13:16:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.747+0000 7fa685400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6800b36f0 con 0x7fa68015f710 2024-03-28T13:16:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.747+0000 7fa685400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa6800cdad0 con 0x7fa680072f50 2024-03-28T13:16:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.747+0000 7fa67fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa680144ad0 0x7fa680072840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36340/0 (socket says 172.21.15.19:36340) 2024-03-28T13:16:21.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.747+0000 7fa67fe00640 1 -- 172.21.15.19:0/1834414683 learned_addr learned my addr 172.21.15.19:0/1834414683 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67f400640 1 --2- 172.21.15.19:0/1834414683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa680072f50 0x7fa6800b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67ea00640 1 -- 172.21.15.19:0/1834414683 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 314255454 0 0) 0x7fa680144eb0 con 0x7fa680144ad0 2024-03-28T13:16:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67ea00640 1 -- 172.21.15.19:0/1834414683 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa664005730 con 0x7fa680144ad0 2024-03-28T13:16:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67ea00640 1 -- 172.21.15.19:0/1834414683 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 5761423 0 0) 0x7fa664005730 con 0x7fa680144ad0 2024-03-28T13:16:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67ea00640 1 -- 172.21.15.19:0/1834414683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa680072f50 msgr2=0x7fa6800b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67ea00640 1 --2- 172.21.15.19:0/1834414683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa680072f50 0x7fa6800b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67ea00640 1 -- 172.21.15.19:0/1834414683 >> v1:172.21.15.188:6789/0 conn(0x7fa68015f710 legacy=0x7fa6800cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67ea00640 1 -- 172.21.15.19:0/1834414683 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6800b39d0 con 0x7fa680144ad0 2024-03-28T13:16:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67ea00640 1 -- 172.21.15.19:0/1834414683 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fa670001670 con 0x7fa680144ad0 2024-03-28T13:16:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67ea00640 1 -- 172.21.15.19:0/1834414683 >> v1:172.21.15.19:6789/0 conn(0x7fa680144ad0 legacy=0x7fa680072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67ea00640 1 --2- 172.21.15.19:0/1834414683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa680072f50 0x7fa664005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67ea00640 1 --2- 172.21.15.19:0/1834414683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664006170 0x7fa664006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67ea00640 1 --2- 172.21.15.19:0/1834414683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa664006f50 0x7fa6640073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67ea00640 1 -- 172.21.15.19:0/1834414683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6800cdad0 con 0x7fa664006170 2024-03-28T13:16:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67ea00640 1 -- 172.21.15.19:0/1834414683 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa66400dcb0 con 0x7fa664006f50 2024-03-28T13:16:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.748+0000 7fa67ea00640 1 -- 172.21.15.19:0/1834414683 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa66400deb0 con 0x7fa680072f50 2024-03-28T13:16:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.749+0000 7fa67f400640 1 --2- 172.21.15.19:0/1834414683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664006170 0x7fa664006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.749+0000 7fa684a00640 1 --2- 172.21.15.19:0/1834414683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa664006f50 0x7fa6640073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.749+0000 7fa67fe00640 1 --2- 172.21.15.19:0/1834414683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa680072f50 0x7fa664005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.749+0000 7fa67f400640 1 -- 172.21.15.19:0/1834414683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa680072f50 msgr2=0x7fa664005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.749+0000 7fa67f400640 1 --2- 172.21.15.19:0/1834414683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa680072f50 0x7fa664005bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.749+0000 7fa67f400640 1 -- 172.21.15.19:0/1834414683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa664006f50 msgr2=0x7fa6640073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.749+0000 7fa67f400640 1 --2- 172.21.15.19:0/1834414683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa664006f50 0x7fa6640073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.749+0000 7fa67f400640 1 -- 172.21.15.19:0/1834414683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa664005730 con 0x7fa664006170 2024-03-28T13:16:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.749+0000 7fa684a00640 1 --2- 172.21.15.19:0/1834414683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa664006f50 0x7fa6640073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:21.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.750+0000 7fa67f400640 1 --2- 172.21.15.19:0/1834414683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664006170 0x7fa664006a10 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009ac0 tx=0x7fa664007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb9175a6e77d5b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:21.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.751+0000 7fa67ea00640 1 -- 172.21.15.19:0/1834414683 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c039070 con 0x7fa664006170 2024-03-28T13:16:21.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.751+0000 7fa67ea00640 1 -- 172.21.15.19:0/1834414683 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa66c02c6b0 con 0x7fa664006170 2024-03-28T13:16:21.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.751+0000 7fa67ea00640 1 -- 172.21.15.19:0/1834414683 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c009690 con 0x7fa664006170 2024-03-28T13:16:21.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.751+0000 7fa685400640 1 -- 172.21.15.19:0/1834414683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664006170 msgr2=0x7fa664006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:21.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.751+0000 7fa685400640 1 --2- 172.21.15.19:0/1834414683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664006170 0x7fa664006a10 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009ac0 tx=0x7fa664007a40 comp rx=0 tx=0).stop 2024-03-28T13:16:21.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.751+0000 7fa684a00640 1 -- 172.21.15.19:0/1834414683 reap_dead start 2024-03-28T13:16:21.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.751+0000 7fa685400640 1 -- 172.21.15.19:0/1834414683 shutdown_connections 2024-03-28T13:16:21.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.751+0000 7fa685400640 1 -- 172.21.15.19:0/1834414683 >> 172.21.15.19:0/1834414683 conn(0x7fa68006b570 msgr2=0x7fa68006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:21.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.751+0000 7fa685400640 1 -- 172.21.15.19:0/1834414683 shutdown_connections 2024-03-28T13:16:21.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.751+0000 7fa685400640 1 -- 172.21.15.19:0/1834414683 wait complete. 2024-03-28T13:16:21.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.753+0000 7fa685400640 1 Processor -- start 2024-03-28T13:16:21.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.753+0000 7fa685400640 1 -- start start 2024-03-28T13:16:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.754+0000 7fa685400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664006170 0x7fa680141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.754+0000 7fa685400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa68015a4c0 0x7fa68015a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.754+0000 7fa685400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6800ae590 0x7fa6800aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.754+0000 7fa685400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6800b3f90 con 0x7fa664006170 2024-03-28T13:16:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.754+0000 7fa685400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa68015fb40 con 0x7fa68015a4c0 2024-03-28T13:16:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.754+0000 7fa685400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa6800b2ff0 con 0x7fa6800ae590 2024-03-28T13:16:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.755+0000 7fa67fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664006170 0x7fa680141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.755+0000 7fa67fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664006170 0x7fa680141a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49414/0 (socket says 172.21.15.19:49414) 2024-03-28T13:16:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.755+0000 7fa67fe00640 1 -- 172.21.15.19:0/1690213183 learned_addr learned my addr 172.21.15.19:0/1690213183 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.755+0000 7fa67f400640 1 --2- 172.21.15.19:0/1690213183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa68015a4c0 0x7fa68015a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.755+0000 7fa684a00640 1 --2- 172.21.15.19:0/1690213183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6800ae590 0x7fa6800aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.755+0000 7fa67fe00640 1 -- 172.21.15.19:0/1690213183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6800ae590 msgr2=0x7fa6800aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.755+0000 7fa67fe00640 1 --2- 172.21.15.19:0/1690213183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6800ae590 0x7fa6800aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.755+0000 7fa67fe00640 1 -- 172.21.15.19:0/1690213183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa68015a4c0 msgr2=0x7fa68015a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.755+0000 7fa67fe00640 1 --2- 172.21.15.19:0/1690213183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa68015a4c0 0x7fa68015a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.755+0000 7fa67fe00640 1 -- 172.21.15.19:0/1690213183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6801f8560 con 0x7fa664006170 2024-03-28T13:16:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.755+0000 7fa684a00640 1 --2- 172.21.15.19:0/1690213183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6800ae590 0x7fa6800aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.755+0000 7fa67fe00640 1 --2- 172.21.15.19:0/1690213183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664006170 0x7fa680141a00 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa67000d800 tx=0x7fa67000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.756+0000 7fa677e00640 1 -- 172.21.15.19:0/1690213183 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670019070 con 0x7fa664006170 2024-03-28T13:16:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.756+0000 7fa677e00640 1 -- 172.21.15.19:0/1690213183 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6700074e0 con 0x7fa664006170 2024-03-28T13:16:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.756+0000 7fa677e00640 1 -- 172.21.15.19:0/1690213183 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6700049e0 con 0x7fa664006170 2024-03-28T13:16:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.756+0000 7fa685400640 1 -- 172.21.15.19:0/1690213183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6801f77a0 con 0x7fa664006170 2024-03-28T13:16:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.756+0000 7fa685400640 1 -- 172.21.15.19:0/1690213183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6801f7d60 con 0x7fa664006170 2024-03-28T13:16:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.757+0000 7fa685400640 1 -- 172.21.15.19:0/1690213183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa680075940 con 0x7fa664006170 2024-03-28T13:16:21.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.758+0000 7fa677e00640 1 -- 172.21.15.19:0/1690213183 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa67001c070 con 0x7fa664006170 2024-03-28T13:16:21.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.759+0000 7fa677e00640 1 --2- 172.21.15.19:0/1690213183 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa66006c530 0x7fa66006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:21.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.759+0000 7fa677e00640 1 -- 172.21.15.19:0/1690213183 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7fa67008acf0 con 0x7fa664006170 2024-03-28T13:16:21.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.759+0000 7fa67f400640 1 --2- 172.21.15.19:0/1690213183 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa66006c530 0x7fa66006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:21.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.761+0000 7fa67f400640 1 --2- 172.21.15.19:0/1690213183 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa66006c530 0x7fa66006e9f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa66c036000 tx=0x7fa66c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:21.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:21.764+0000 7fa677e00640 1 -- 172.21.15.19:0/1690213183 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa6700590c0 con 0x7fa664006170 2024-03-28T13:16:22.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.039+0000 7fa685400640 1 -- 172.21.15.19:0/1690213183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa6800b4240 con 0x7fa664006170 2024-03-28T13:16:22.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.040+0000 7fa677e00640 1 -- 172.21.15.19:0/1690213183 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7fa67005d3c0 con 0x7fa664006170 2024-03-28T13:16:22.040 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:22.040 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:16:22.040 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-03-28T13:16:22.040 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.040 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-03-28T13:16:22.040 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmNRIkABAAq/nJioDZjp1zFWAYOLPBYw==", 2024-03-28T13:16:22.040 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.040 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:16:22.040 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:16:22.040 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:16:22.040 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:16:22.040 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-03-28T13:16:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmc10CIxAAQWCSjH0hW3KUk8/qQm91QA==", 2024-03-28T13:16:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:16:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:16:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:16:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:16:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-03-28T13:16:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmmlUoEhAA4FUGXoZ/h/ve5fThJNdE2Q==", 2024-03-28T13:16:22.041 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.042 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:16:22.042 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:16:22.042 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:16:22.042 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:16:22.042 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.042 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-03-28T13:16:22.042 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmWDRnMxAA34LiLI79N5fVJnbcPacbQA==", 2024-03-28T13:16:22.042 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.042 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:16:22.042 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:16:22.043 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:16:22.043 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:16:22.043 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.043 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.043 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-03-28T13:16:22.043 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDYawVmxhwWLBAAm7C56UGConFx2D7Lbq2SSA==", 2024-03-28T13:16:22.043 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.043 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:16:22.043 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:16:22.043 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:16:22.043 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.043 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.043 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-03-28T13:16:22.044 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDbawVmC2BiNxAALQrAcXU44m2dTCjks5yy/Q==", 2024-03-28T13:16:22.044 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.044 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:16:22.044 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:16:22.044 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:16:22.044 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.044 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.044 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-03-28T13:16:22.044 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfawVmozrJFRAA7YV1uMucIW7H5SI4KY8aDw==", 2024-03-28T13:16:22.044 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.044 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:16:22.044 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:16:22.044 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:16:22.044 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.045 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.045 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-03-28T13:16:22.045 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiawVmccbTMBAATxk24Bf4LuQSqU92HdwgUg==", 2024-03-28T13:16:22.045 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.045 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:16:22.045 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:16:22.045 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:16:22.045 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.045 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.045 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-03-28T13:16:22.045 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDpawVmdwnILRAAjUEBwnBEeSltu2CLRGhmDQ==", 2024-03-28T13:16:22.046 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.046 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:16:22.046 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:16:22.046 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:16:22.046 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.046 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.046 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-03-28T13:16:22.046 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDtawVmkwW5ARAAZJ/Nd/Wtq7LcbUjeqPNUag==", 2024-03-28T13:16:22.046 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.046 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:16:22.046 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:16:22.046 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:16:22.046 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.047 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.047 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-03-28T13:16:22.047 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDwawVm254wKRAAmCYODqSd2Whn4YKQzywrng==", 2024-03-28T13:16:22.047 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.047 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:16:22.047 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:16:22.047 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:16:22.047 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.047 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.047 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-03-28T13:16:22.047 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDzawVmV8z+MhAA9JeF55K7ph1mQRWalT7XhA==", 2024-03-28T13:16:22.047 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.048 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:16:22.048 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:16:22.048 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:16:22.048 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.048 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.048 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-03-28T13:16:22.048 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q==", 2024-03-28T13:16:22.048 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.048 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:16:22.048 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-03-28T13:16:22.048 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-28T13:16:22.048 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-03-28T13:16:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmJYGjHBAAAx2TtR0Gb3fafnZU26FCMA==", 2024-03-28T13:16:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:16:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:16:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:16:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:16:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-03-28T13:16:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmE7V/ERAAYcdvolIMDd5ff+pCn7hN1w==", 2024-03-28T13:16:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.049 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:16:22.050 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:16:22.050 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:16:22.050 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:16:22.050 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.050 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.050 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-03-28T13:16:22.050 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmk6q9JhAAeC3JHQDnK8IXGVoK6qGEFA==", 2024-03-28T13:16:22.050 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.050 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:16:22.050 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:16:22.050 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:16:22.050 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:16:22.050 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.051 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.051 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-03-28T13:16:22.051 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDRawVm9IzFLRAA5i1Yl7LQ7eNNHp7v4H2kog==", 2024-03-28T13:16:22.051 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.051 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:16:22.051 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:16:22.051 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:16:22.051 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:16:22.051 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.051 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.052 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-03-28T13:16:22.052 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDSawVm5H1FLxAAtyjOPcCEUQyvTpBdu3Fn3g==", 2024-03-28T13:16:22.052 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.052 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:16:22.052 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-03-28T13:16:22.052 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:16:22.052 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:16:22.052 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:22.052 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-03-28T13:16:22.052 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDSawVmebzfIRAAA0+qIIaprr4SfEZT6qRooA==", 2024-03-28T13:16:22.052 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:16:22.052 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:16:22.053 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-03-28T13:16:22.053 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:16:22.053 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.053 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:22.053 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-03-28T13:16:22.053 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:16:22.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.042+0000 7fa685400640 1 -- 172.21.15.19:0/1690213183 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa66006c530 msgr2=0x7fa66006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:22.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.042+0000 7fa685400640 1 --2- 172.21.15.19:0/1690213183 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa66006c530 0x7fa66006e9f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa66c036000 tx=0x7fa66c030000 comp rx=0 tx=0).stop 2024-03-28T13:16:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.042+0000 7fa685400640 1 -- 172.21.15.19:0/1690213183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664006170 msgr2=0x7fa680141a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.042+0000 7fa685400640 1 --2- 172.21.15.19:0/1690213183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664006170 0x7fa680141a00 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa67000d800 tx=0x7fa67000dcc0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.042+0000 7fa685400640 1 -- 172.21.15.19:0/1690213183 shutdown_connections 2024-03-28T13:16:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.042+0000 7fa685400640 1 --2- 172.21.15.19:0/1690213183 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa66006c530 0x7fa66006e9f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.042+0000 7fa685400640 1 --2- 172.21.15.19:0/1690213183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6800ae590 0x7fa6800aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.042+0000 7fa685400640 1 --2- 172.21.15.19:0/1690213183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa68015a4c0 0x7fa68015a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.042+0000 7fa685400640 1 --2- 172.21.15.19:0/1690213183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa664006170 0x7fa680141a00 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.042+0000 7fa685400640 1 -- 172.21.15.19:0/1690213183 >> 172.21.15.19:0/1690213183 conn(0x7fa68006b570 msgr2=0x7fa6800cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.043+0000 7fa685400640 1 -- 172.21.15.19:0/1690213183 shutdown_connections 2024-03-28T13:16:22.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.043+0000 7fa685400640 1 -- 172.21.15.19:0/1690213183 wait complete. 2024-03-28T13:16:22.062 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:16:22.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.275+0000 7f9cd3200640 1 Processor -- start 2024-03-28T13:16:22.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.276+0000 7f9cd3200640 1 -- start start 2024-03-28T13:16:22.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.277+0000 7f9cd3200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40d2310 0x7f9cd40ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:22.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.277+0000 7f9cd3200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9cd40cf530 0x7f9cd40b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:22.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.277+0000 7f9cd3200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cd415e880 con 0x7f9cd415e4a0 2024-03-28T13:16:22.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.277+0000 7f9cd3200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9cd40d34e0 con 0x7f9cd40cf530 2024-03-28T13:16:22.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.277+0000 7f9cd3200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd40d28d0 con 0x7f9cd40d2310 2024-03-28T13:16:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.278+0000 7f9cd1400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40d2310 0x7f9cd40ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.278+0000 7f9cd2800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9cd40cf530 0x7f9cd40b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.278+0000 7f9cd1400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40d2310 0x7f9cd40ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:40160/0 (socket says 172.21.15.19:40160) 2024-03-28T13:16:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.278+0000 7f9cd1400640 1 -- 172.21.15.19:0/3381433871 learned_addr learned my addr 172.21.15.19:0/3381433871 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:22.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.278+0000 7f9cd1e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f9cd415e4a0 0x7f9cd40b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:57516/0 (socket says 172.21.15.19:57516) 2024-03-28T13:16:22.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.279+0000 7f9cd1400640 1 -- 172.21.15.19:0/3381433871 >> v2:172.21.15.188:3300/0 conn(0x7f9cd40cf530 msgr2=0x7f9cd40b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:22.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.279+0000 7f9cd1400640 1 --2- 172.21.15.19:0/3381433871 >> v2:172.21.15.188:3300/0 conn(0x7f9cd40cf530 0x7f9cd40b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.279+0000 7f9cd1400640 1 -- 172.21.15.19:0/3381433871 >> v1:172.21.15.188:6789/0 conn(0x7f9cd415e4a0 legacy=0x7f9cd40b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:22.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.279+0000 7f9cd1400640 1 -- 172.21.15.19:0/3381433871 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd40d2bb0 con 0x7f9cd40d2310 2024-03-28T13:16:22.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.280+0000 7f9cd1400640 1 --2- 172.21.15.19:0/3381433871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40d2310 0x7f9cd40ceff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009870 tx=0x7f9cbc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8de3078fb080fb10 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:22.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.281+0000 7f9cd0a00640 1 -- 172.21.15.19:0/3381433871 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc034070 con 0x7f9cd40d2310 2024-03-28T13:16:22.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.281+0000 7f9cd0a00640 1 -- 172.21.15.19:0/3381433871 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cbc030840 con 0x7f9cd40d2310 2024-03-28T13:16:22.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.281+0000 7f9cd0a00640 1 -- 172.21.15.19:0/3381433871 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc030c10 con 0x7f9cd40d2310 2024-03-28T13:16:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.281+0000 7f9cd3200640 1 -- 172.21.15.19:0/3381433871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40d2310 msgr2=0x7f9cd40ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.281+0000 7f9cd3200640 1 --2- 172.21.15.19:0/3381433871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40d2310 0x7f9cd40ceff0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009870 tx=0x7f9cbc02b980 comp rx=0 tx=0).stop 2024-03-28T13:16:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.282+0000 7f9cd3200640 1 -- 172.21.15.19:0/3381433871 shutdown_connections 2024-03-28T13:16:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.282+0000 7f9cd3200640 1 --2- 172.21.15.19:0/3381433871 >> v2:172.21.15.188:3300/0 conn(0x7f9cd40cf530 0x7f9cd40b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.282+0000 7f9cd3200640 1 --2- 172.21.15.19:0/3381433871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40d2310 0x7f9cd40ceff0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.282+0000 7f9cd3200640 1 -- 172.21.15.19:0/3381433871 >> 172.21.15.19:0/3381433871 conn(0x7f9cd40676d0 msgr2=0x7f9cd4067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:22.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.282+0000 7f9cd3200640 1 -- 172.21.15.19:0/3381433871 shutdown_connections 2024-03-28T13:16:22.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.282+0000 7f9cd3200640 1 -- 172.21.15.19:0/3381433871 wait complete. 2024-03-28T13:16:22.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.283+0000 7f9cd3200640 1 Processor -- start 2024-03-28T13:16:22.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.284+0000 7f9cd3200640 1 -- start start 2024-03-28T13:16:22.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.284+0000 7f9cd3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cd40cf530 0x7f9cd4148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:22.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.284+0000 7f9cd3200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40d2310 0x7f9cd415ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:22.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.284+0000 7f9cd3200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cd415e4a0 0x7f9cd40c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:22.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.284+0000 7f9cd3200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd40d3970 con 0x7f9cd40cf530 2024-03-28T13:16:22.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.284+0000 7f9cd3200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd40d3170 con 0x7f9cd415e4a0 2024-03-28T13:16:22.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.284+0000 7f9cd3200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd40b7920 con 0x7f9cd40d2310 2024-03-28T13:16:22.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.285+0000 7f9cd1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cd40cf530 0x7f9cd4148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:22.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.285+0000 7f9cd1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cd40cf530 0x7f9cd4148fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49422/0 (socket says 172.21.15.19:49422) 2024-03-28T13:16:22.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.285+0000 7f9cd1e00640 1 -- 172.21.15.19:0/2671627704 learned_addr learned my addr 172.21.15.19:0/2671627704 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.285+0000 7f9cd2800640 1 --2- 172.21.15.19:0/2671627704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cd415e4a0 0x7f9cd40c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.285+0000 7f9cd1400640 1 --2- 172.21.15.19:0/2671627704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40d2310 0x7f9cd415ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.285+0000 7f9cd1e00640 1 -- 172.21.15.19:0/2671627704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40d2310 msgr2=0x7f9cd415ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.285+0000 7f9cd1e00640 1 --2- 172.21.15.19:0/2671627704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40d2310 0x7f9cd415ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.285+0000 7f9cd1e00640 1 -- 172.21.15.19:0/2671627704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cd415e4a0 msgr2=0x7f9cd40c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.285+0000 7f9cd1e00640 1 --2- 172.21.15.19:0/2671627704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cd415e4a0 0x7f9cd40c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.285+0000 7f9cd1e00640 1 -- 172.21.15.19:0/2671627704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd4156830 con 0x7f9cd40cf530 2024-03-28T13:16:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.285+0000 7f9cd1400640 1 --2- 172.21.15.19:0/2671627704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40d2310 0x7f9cd415ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.286+0000 7f9cd1e00640 1 --2- 172.21.15.19:0/2671627704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cd40cf530 0x7f9cd4148fc0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400c3c0 tx=0x7f9cc400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.286+0000 7f9cca000640 1 -- 172.21.15.19:0/2671627704 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40074e0 con 0x7f9cd40cf530 2024-03-28T13:16:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.286+0000 7f9cca000640 1 -- 172.21.15.19:0/2671627704 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc4007680 con 0x7f9cd40cf530 2024-03-28T13:16:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.286+0000 7f9cd3200640 1 -- 172.21.15.19:0/2671627704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd4156b10 con 0x7f9cd40cf530 2024-03-28T13:16:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.286+0000 7f9cca000640 1 -- 172.21.15.19:0/2671627704 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4004a00 con 0x7f9cd40cf530 2024-03-28T13:16:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.286+0000 7f9cd3200640 1 -- 172.21.15.19:0/2671627704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd41570d0 con 0x7f9cd40cf530 2024-03-28T13:16:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.287+0000 7f9cd3200640 1 -- 172.21.15.19:0/2671627704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd4075940 con 0x7f9cd40cf530 2024-03-28T13:16:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.288+0000 7f9cca000640 1 -- 172.21.15.19:0/2671627704 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9cc401a020 con 0x7f9cd40cf530 2024-03-28T13:16:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.289+0000 7f9cca000640 1 --2- 172.21.15.19:0/2671627704 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9cb006c530 0x7f9cb006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.289+0000 7f9cca000640 1 -- 172.21.15.19:0/2671627704 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7f9cc408abc0 con 0x7f9cd40cf530 2024-03-28T13:16:22.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.289+0000 7f9cd1400640 1 --2- 172.21.15.19:0/2671627704 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9cb006c530 0x7f9cb006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:22.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.290+0000 7f9cd1400640 1 --2- 172.21.15.19:0/2671627704 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9cb006c530 0x7f9cb006e9f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc036000 tx=0x7f9cbc03b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:22.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.293+0000 7f9cca000640 1 -- 172.21.15.19:0/2671627704 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9cc4059490 con 0x7f9cd40cf530 2024-03-28T13:16:22.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.490+0000 7f9cd3200640 1 -- 172.21.15.19:0/2671627704 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cd40b3dd0 con 0x7f9cd40cf530 2024-03-28T13:16:22.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.495+0000 7f9cca000640 1 -- 172.21.15.19:0/2671627704 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+12892 (secure 0 0 0) 0x7f9cc4058d90 con 0x7f9cd40cf530 2024-03-28T13:16:22.495 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:22.495 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:16:21.503228+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:16:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.498+0000 7f9cd3200640 1 -- 172.21.15.19:0/2671627704 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9cb006c530 msgr2=0x7f9cb006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.498+0000 7f9cd3200640 1 --2- 172.21.15.19:0/2671627704 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9cb006c530 0x7f9cb006e9f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc036000 tx=0x7f9cbc03b040 comp rx=0 tx=0).stop 2024-03-28T13:16:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.498+0000 7f9cd3200640 1 -- 172.21.15.19:0/2671627704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cd40cf530 msgr2=0x7f9cd4148fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.498+0000 7f9cd3200640 1 --2- 172.21.15.19:0/2671627704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cd40cf530 0x7f9cd4148fc0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f9cc400c3c0 tx=0x7f9cc400c880 comp rx=0 tx=0).stop 2024-03-28T13:16:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.498+0000 7f9cd3200640 1 -- 172.21.15.19:0/2671627704 shutdown_connections 2024-03-28T13:16:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.498+0000 7f9cd3200640 1 --2- 172.21.15.19:0/2671627704 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9cb006c530 0x7f9cb006e9f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.498+0000 7f9cd3200640 1 --2- 172.21.15.19:0/2671627704 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cd415e4a0 0x7f9cd40c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.498+0000 7f9cd3200640 1 --2- 172.21.15.19:0/2671627704 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cd40d2310 0x7f9cd415ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.498+0000 7f9cd3200640 1 --2- 172.21.15.19:0/2671627704 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cd40cf530 0x7f9cd4148fc0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.498+0000 7f9cd3200640 1 -- 172.21.15.19:0/2671627704 >> 172.21.15.19:0/2671627704 conn(0x7f9cd40676d0 msgr2=0x7f9cd40b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.498+0000 7f9cd3200640 1 -- 172.21.15.19:0/2671627704 shutdown_connections 2024-03-28T13:16:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.498+0000 7f9cd3200640 1 -- 172.21.15.19:0/2671627704 wait complete. 2024-03-28T13:16:22.518 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:16:22.518 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:16:22.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.738+0000 7f2cea000640 1 Processor -- start 2024-03-28T13:16:22.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.738+0000 7f2cea000640 1 -- start start 2024-03-28T13:16:22.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.739+0000 7f2cea000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ce415a950 0x7f2ce40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:22.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.739+0000 7f2cea000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ce40cd2d0 con 0x7f2ce406e5c0 2024-03-28T13:16:22.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.739+0000 7f2cea000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ce40cd5b0 con 0x7f2ce40b2ea0 2024-03-28T13:16:22.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.739+0000 7f2cea000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2ce40cd890 con 0x7f2ce415a950 2024-03-28T13:16:22.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.739+0000 7f2ce8c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ce415a950 0x7f2ce40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:22.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.740+0000 7f2ce9600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2ce406e5c0 0x7f2ce40ce370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36354/0 (socket says 172.21.15.19:36354) 2024-03-28T13:16:22.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.740+0000 7f2ce8c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ce415a950 0x7f2ce40ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49426/0 (socket says 172.21.15.19:49426) 2024-03-28T13:16:22.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.740+0000 7f2ce9600640 1 -- 172.21.15.19:0/3062195345 learned_addr learned my addr 172.21.15.19:0/3062195345 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:22.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.740+0000 7f2ce8c00640 1 -- 172.21.15.19:0/3062195345 >> v1:172.21.15.188:6789/0 conn(0x7f2ce40b2ea0 legacy=0x7f2ce406deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:22.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.740+0000 7f2ce8c00640 1 -- 172.21.15.19:0/3062195345 >> v1:172.21.15.19:6789/0 conn(0x7f2ce406e5c0 legacy=0x7f2ce40ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:22.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.740+0000 7f2ce8c00640 1 -- 172.21.15.19:0/3062195345 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce40cdb70 con 0x7f2ce415a950 2024-03-28T13:16:22.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.741+0000 7f2ce8c00640 1 --2- 172.21.15.19:0/3062195345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce415a950 0x7f2ce40ccc60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80097a0 tx=0x7f2cd802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83cc41c89eb6f031 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:22.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.741+0000 7f2ce3400640 1 -- 172.21.15.19:0/3062195345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd80315a0 con 0x7f2ce415a950 2024-03-28T13:16:22.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.742+0000 7f2ce3400640 1 -- 172.21.15.19:0/3062195345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd8037070 con 0x7f2ce415a950 2024-03-28T13:16:22.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.742+0000 7f2ce3400640 1 -- 172.21.15.19:0/3062195345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8031960 con 0x7f2ce415a950 2024-03-28T13:16:22.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.742+0000 7f2cea000640 1 -- 172.21.15.19:0/3062195345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce415a950 msgr2=0x7f2ce40ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:22.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.742+0000 7f2cea000640 1 --2- 172.21.15.19:0/3062195345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce415a950 0x7f2ce40ccc60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80097a0 tx=0x7f2cd802b840 comp rx=0 tx=0).stop 2024-03-28T13:16:22.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.742+0000 7f2cea000640 1 -- 172.21.15.19:0/3062195345 shutdown_connections 2024-03-28T13:16:22.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.742+0000 7f2cea000640 1 --2- 172.21.15.19:0/3062195345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce415a950 0x7f2ce40ccc60 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.742+0000 7f2cea000640 1 -- 172.21.15.19:0/3062195345 >> 172.21.15.19:0/3062195345 conn(0x7f2ce40b1ba0 msgr2=0x7f2ce40b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:22.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.742+0000 7f2cea000640 1 -- 172.21.15.19:0/3062195345 shutdown_connections 2024-03-28T13:16:22.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.742+0000 7f2cea000640 1 -- 172.21.15.19:0/3062195345 wait complete. 2024-03-28T13:16:22.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.744+0000 7f2cea000640 1 Processor -- start 2024-03-28T13:16:22.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.744+0000 7f2cea000640 1 -- start start 2024-03-28T13:16:22.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.745+0000 7f2cea000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce406e5c0 0x7f2ce4150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:22.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.745+0000 7f2cea000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ce40b2ea0 0x7f2ce414a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:22.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.745+0000 7f2cea000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ce415a950 0x7f2ce415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:22.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.745+0000 7f2cea000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce40715d0 con 0x7f2ce406e5c0 2024-03-28T13:16:22.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.745+0000 7f2cea000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce4071450 con 0x7f2ce415a950 2024-03-28T13:16:22.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.745+0000 7f2cea000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2ce40cf600 con 0x7f2ce40b2ea0 2024-03-28T13:16:22.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.745+0000 7f2ce8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce406e5c0 0x7f2ce4150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:22.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.745+0000 7f2ce8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce406e5c0 0x7f2ce4150d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49436/0 (socket says 172.21.15.19:49436) 2024-03-28T13:16:22.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.745+0000 7f2ce8c00640 1 -- 172.21.15.19:0/2085630515 learned_addr learned my addr 172.21.15.19:0/2085630515 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.746+0000 7f2ce8c00640 1 -- 172.21.15.19:0/2085630515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ce40b2ea0 msgr2=0x7f2ce414a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.746+0000 7f2ce8c00640 1 --2- 172.21.15.19:0/2085630515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ce40b2ea0 0x7f2ce414a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.746+0000 7f2ce8c00640 1 -- 172.21.15.19:0/2085630515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ce415a950 msgr2=0x7f2ce415ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.746+0000 7f2ce9600640 1 --2- 172.21.15.19:0/2085630515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ce415a950 0x7f2ce415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.746+0000 7f2ce8c00640 1 --2- 172.21.15.19:0/2085630515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ce415a950 0x7f2ce415ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.746+0000 7f2ce8c00640 1 -- 172.21.15.19:0/2085630515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce4151240 con 0x7f2ce406e5c0 2024-03-28T13:16:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.746+0000 7f2ce9600640 1 --2- 172.21.15.19:0/2085630515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ce415a950 0x7f2ce415ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.746+0000 7f2ce8c00640 1 --2- 172.21.15.19:0/2085630515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce406e5c0 0x7f2ce4150d00 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8034040 tx=0x7f2cd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.747+0000 7f2ce0c00640 1 -- 172.21.15.19:0/2085630515 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8032840 con 0x7f2ce406e5c0 2024-03-28T13:16:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.747+0000 7f2ce0c00640 1 -- 172.21.15.19:0/2085630515 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cd8032a00 con 0x7f2ce406e5c0 2024-03-28T13:16:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.747+0000 7f2ce0c00640 1 -- 172.21.15.19:0/2085630515 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8032df0 con 0x7f2ce406e5c0 2024-03-28T13:16:22.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.747+0000 7f2cea000640 1 -- 172.21.15.19:0/2085630515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce41402d0 con 0x7f2ce406e5c0 2024-03-28T13:16:22.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.747+0000 7f2cea000640 1 -- 172.21.15.19:0/2085630515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce4140890 con 0x7f2ce406e5c0 2024-03-28T13:16:22.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.748+0000 7f2cea000640 1 -- 172.21.15.19:0/2085630515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce4147de0 con 0x7f2ce406e5c0 2024-03-28T13:16:22.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.749+0000 7f2ce0c00640 1 -- 172.21.15.19:0/2085630515 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2cd803b3f0 con 0x7f2ce406e5c0 2024-03-28T13:16:22.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.750+0000 7f2ce0c00640 1 --2- 172.21.15.19:0/2085630515 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2cc4070940 0x7f2cc4072e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:22.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.750+0000 7f2ce0c00640 1 -- 172.21.15.19:0/2085630515 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7f2cd802f030 con 0x7f2ce406e5c0 2024-03-28T13:16:22.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.750+0000 7f2ce3e00640 1 --2- 172.21.15.19:0/2085630515 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2cc4070940 0x7f2cc4072e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:22.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.752+0000 7f2ce3e00640 1 --2- 172.21.15.19:0/2085630515 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2cc4070940 0x7f2cc4072e00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2ce415be60 tx=0x7f2cd00079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:22.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.754+0000 7f2ce0c00640 1 -- 172.21.15.19:0/2085630515 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2ce4151240 con 0x7f2ce406e5c0 2024-03-28T13:16:22.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.958+0000 7f2cea000640 1 -- 172.21.15.19:0/2085630515 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2ce4075940 con 0x7f2ce406e5c0 2024-03-28T13:16:22.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.959+0000 7f2ce0c00640 1 -- 172.21.15.19:0/2085630515 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f2cd80ae020 con 0x7f2ce406e5c0 2024-03-28T13:16:22.959 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:16:22.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.962+0000 7f2cea000640 1 -- 172.21.15.19:0/2085630515 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2cc4070940 msgr2=0x7f2cc4072e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:22.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.962+0000 7f2cea000640 1 --2- 172.21.15.19:0/2085630515 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2cc4070940 0x7f2cc4072e00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f2ce415be60 tx=0x7f2cd00079b0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.962+0000 7f2cea000640 1 -- 172.21.15.19:0/2085630515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce406e5c0 msgr2=0x7f2ce4150d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:22.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.962+0000 7f2cea000640 1 --2- 172.21.15.19:0/2085630515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce406e5c0 0x7f2ce4150d00 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8034040 tx=0x7f2cd80046c0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.962+0000 7f2cea000640 1 -- 172.21.15.19:0/2085630515 shutdown_connections 2024-03-28T13:16:22.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.962+0000 7f2cea000640 1 --2- 172.21.15.19:0/2085630515 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2cc4070940 0x7f2cc4072e00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.962+0000 7f2cea000640 1 --2- 172.21.15.19:0/2085630515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ce415a950 0x7f2ce415ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.962+0000 7f2cea000640 1 --2- 172.21.15.19:0/2085630515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ce40b2ea0 0x7f2ce414a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.962+0000 7f2cea000640 1 --2- 172.21.15.19:0/2085630515 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ce406e5c0 0x7f2ce4150d00 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:22.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.962+0000 7f2cea000640 1 -- 172.21.15.19:0/2085630515 >> 172.21.15.19:0/2085630515 conn(0x7f2ce40b1ba0 msgr2=0x7f2ce40b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.962+0000 7f2cea000640 1 -- 172.21.15.19:0/2085630515 shutdown_connections 2024-03-28T13:16:22.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:22.962+0000 7f2cea000640 1 -- 172.21.15.19:0/2085630515 wait complete. 2024-03-28T13:16:22.982 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-28T13:16:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.199+0000 7fe4daa00640 1 Processor -- start 2024-03-28T13:16:23.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.199+0000 7fe4daa00640 1 -- start start 2024-03-28T13:16:23.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.201+0000 7fe4daa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe4d406e030 0x7fe4d415fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:23.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.201+0000 7fe4daa00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d4142880 con 0x7fe4d413f300 2024-03-28T13:16:23.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.201+0000 7fe4daa00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d41422c0 con 0x7fe4d41602d0 2024-03-28T13:16:23.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.201+0000 7fe4daa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe4d4160720 con 0x7fe4d406e030 2024-03-28T13:16:23.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.201+0000 7fe4d8c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe4d406e030 0x7fe4d415fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:23.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.201+0000 7fe4d8c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe4d406e030 0x7fe4d415fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49438/0 (socket says 172.21.15.19:49438) 2024-03-28T13:16:23.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.201+0000 7fe4d8c00640 1 -- 172.21.15.19:0/4228130490 learned_addr learned my addr 172.21.15.19:0/4228130490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:23.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.201+0000 7fe4d8c00640 1 -- 172.21.15.19:0/4228130490 >> v1:172.21.15.188:6789/0 conn(0x7fe4d41602d0 legacy=0x7fe4d40cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:23.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.202+0000 7fe4d8c00640 1 -- 172.21.15.19:0/4228130490 >> v1:172.21.15.19:6789/0 conn(0x7fe4d413f300 legacy=0x7fe4d41419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:23.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.202+0000 7fe4d8c00640 1 -- 172.21.15.19:0/4228130490 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d41425a0 con 0x7fe4d406e030 2024-03-28T13:16:23.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.203+0000 7fe4d8c00640 1 --2- 172.21.15.19:0/4228130490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d406e030 0x7fe4d415fd90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009870 tx=0x7fe4c002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6daf0d064871686 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:23.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.203+0000 7fe4d3e00640 1 -- 172.21.15.19:0/4228130490 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0034070 con 0x7fe4d406e030 2024-03-28T13:16:23.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.203+0000 7fe4d3e00640 1 -- 172.21.15.19:0/4228130490 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c0030860 con 0x7fe4d406e030 2024-03-28T13:16:23.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.203+0000 7fe4d3e00640 1 -- 172.21.15.19:0/4228130490 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0030c30 con 0x7fe4d406e030 2024-03-28T13:16:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.204+0000 7fe4daa00640 1 -- 172.21.15.19:0/4228130490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d406e030 msgr2=0x7fe4d415fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.204+0000 7fe4daa00640 1 --2- 172.21.15.19:0/4228130490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d406e030 0x7fe4d415fd90 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009870 tx=0x7fe4c002b9a0 comp rx=0 tx=0).stop 2024-03-28T13:16:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.204+0000 7fe4daa00640 1 -- 172.21.15.19:0/4228130490 shutdown_connections 2024-03-28T13:16:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.204+0000 7fe4daa00640 1 --2- 172.21.15.19:0/4228130490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d406e030 0x7fe4d415fd90 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.204+0000 7fe4daa00640 1 -- 172.21.15.19:0/4228130490 >> 172.21.15.19:0/4228130490 conn(0x7fe4d406b570 msgr2=0x7fe4d406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.204+0000 7fe4daa00640 1 -- 172.21.15.19:0/4228130490 shutdown_connections 2024-03-28T13:16:23.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.204+0000 7fe4daa00640 1 -- 172.21.15.19:0/4228130490 wait complete. 2024-03-28T13:16:23.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.205+0000 7fe4daa00640 1 Processor -- start 2024-03-28T13:16:23.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.206+0000 7fe4daa00640 1 -- start start 2024-03-28T13:16:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4daa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4d406e030 0x7fe4d40aa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4daa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d413f300 0x7fe4d413deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4daa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4d41602d0 0x7fe4d4071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4daa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d40cfa30 con 0x7fe4d413f300 2024-03-28T13:16:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4daa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d40cee90 con 0x7fe4d406e030 2024-03-28T13:16:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4daa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d40cfbb0 con 0x7fe4d41602d0 2024-03-28T13:16:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4d8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d413f300 0x7fe4d413deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4d8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d413f300 0x7fe4d413deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49440/0 (socket says 172.21.15.19:49440) 2024-03-28T13:16:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4d8c00640 1 -- 172.21.15.19:0/4106585208 learned_addr learned my addr 172.21.15.19:0/4106585208 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4d9600640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4d406e030 0x7fe4d40aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4d8c00640 1 -- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4d41602d0 msgr2=0x7fe4d4071dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:23.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4da000640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4d41602d0 0x7fe4d4071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:23.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4d8c00640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4d41602d0 0x7fe4d4071dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:23.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4d8c00640 1 -- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4d406e030 msgr2=0x7fe4d40aa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:23.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4d8c00640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4d406e030 0x7fe4d40aa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:23.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4d8c00640 1 -- 172.21.15.19:0/4106585208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d40aabb0 con 0x7fe4d413f300 2024-03-28T13:16:23.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4d9600640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4d406e030 0x7fe4d40aa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:23.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.207+0000 7fe4da000640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4d41602d0 0x7fe4d4071dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:23.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.208+0000 7fe4d8c00640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d413f300 0x7fe4d413deb0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0000c00 tx=0x7fe4c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:23.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.208+0000 7fe4d1600640 1 -- 172.21.15.19:0/4106585208 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0048070 con 0x7fe4d413f300 2024-03-28T13:16:23.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.208+0000 7fe4d1600640 1 -- 172.21.15.19:0/4106585208 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4c0039d90 con 0x7fe4d413f300 2024-03-28T13:16:23.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.208+0000 7fe4d1600640 1 -- 172.21.15.19:0/4106585208 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0034040 con 0x7fe4d413f300 2024-03-28T13:16:23.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.208+0000 7fe4daa00640 1 -- 172.21.15.19:0/4106585208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d40aae90 con 0x7fe4d413f300 2024-03-28T13:16:23.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.208+0000 7fe4daa00640 1 -- 172.21.15.19:0/4106585208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d40ab450 con 0x7fe4d413f300 2024-03-28T13:16:23.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.210+0000 7fe4daa00640 1 -- 172.21.15.19:0/4106585208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d4075940 con 0x7fe4d413f300 2024-03-28T13:16:23.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.210+0000 7fe4d1600640 1 -- 172.21.15.19:0/4106585208 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe4c004b070 con 0x7fe4d413f300 2024-03-28T13:16:23.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.212+0000 7fe4d1600640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe4b406c560 0x7fe4b406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:23.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.212+0000 7fe4d1600640 1 -- 172.21.15.19:0/4106585208 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7fe4c00ad610 con 0x7fe4d413f300 2024-03-28T13:16:23.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.212+0000 7fe4d9600640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe4b406c560 0x7fe4b406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:23.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.213+0000 7fe4d9600640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe4b406c560 0x7fe4b406ea20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8001680 tx=0x7fe4c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:23.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.216+0000 7fe4d1600640 1 -- 172.21.15.19:0/4106585208 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe4c007bee0 con 0x7fe4d413f300 2024-03-28T13:16:23.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.425+0000 7fe4daa00640 1 -- 172.21.15.19:0/4106585208 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe4d41416f0 con 0x7fe4d413f300 2024-03-28T13:16:23.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.426+0000 7fe4d1600640 1 -- 172.21.15.19:0/4106585208 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v88) ==== 84+0+65 (secure 0 0 0) 0x7fe4c007b7e0 con 0x7fe4d413f300 2024-03-28T13:16:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:23.426 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-03-28T13:16:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:16:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-03-28T13:16:23.426 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-03-28T13:16:23.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:23.427 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-03-28T13:16:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.429+0000 7fe4daa00640 1 -- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe4b406c560 msgr2=0x7fe4b406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.429+0000 7fe4daa00640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe4b406c560 0x7fe4b406ea20 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8001680 tx=0x7fe4c8009040 comp rx=0 tx=0).stop 2024-03-28T13:16:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.429+0000 7fe4daa00640 1 -- 172.21.15.19:0/4106585208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d413f300 msgr2=0x7fe4d413deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.429+0000 7fe4daa00640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d413f300 0x7fe4d413deb0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0000c00 tx=0x7fe4c0004730 comp rx=0 tx=0).stop 2024-03-28T13:16:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.429+0000 7fe4daa00640 1 -- 172.21.15.19:0/4106585208 shutdown_connections 2024-03-28T13:16:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.429+0000 7fe4daa00640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe4b406c560 0x7fe4b406ea20 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.429+0000 7fe4daa00640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4d41602d0 0x7fe4d4071dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:23.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.429+0000 7fe4daa00640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4d413f300 0x7fe4d413deb0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:23.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.429+0000 7fe4daa00640 1 --2- 172.21.15.19:0/4106585208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4d406e030 0x7fe4d40aa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:23.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.429+0000 7fe4daa00640 1 -- 172.21.15.19:0/4106585208 >> 172.21.15.19:0/4106585208 conn(0x7fe4d406b570 msgr2=0x7fe4d40b4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:23.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.429+0000 7fe4daa00640 1 -- 172.21.15.19:0/4106585208 shutdown_connections 2024-03-28T13:16:23.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.429+0000 7fe4daa00640 1 -- 172.21.15.19:0/4106585208 wait complete. 2024-03-28T13:16:23.448 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-28T13:16:23.449 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-28T13:16:23.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.668+0000 7fa1a8800640 1 Processor -- start 2024-03-28T13:16:23.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.668+0000 7fa1a8800640 1 -- start start 2024-03-28T13:16:23.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.670+0000 7fa1a8800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa1a009dcf0 0x7fa1a01453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:23.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.670+0000 7fa1a8800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a015cb90 con 0x7fa1a00acf10 2024-03-28T13:16:23.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.670+0000 7fa1a8800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a015ce70 con 0x7fa1a0071710 2024-03-28T13:16:23.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.670+0000 7fa1a8800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a0145f60 con 0x7fa1a009dcf0 2024-03-28T13:16:23.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.670+0000 7fa1a5e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa1a00acf10 0x7fa1a00ad2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36382/0 (socket says 172.21.15.19:36382) 2024-03-28T13:16:23.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.671+0000 7fa1a5e00640 1 -- 172.21.15.19:0/2150523567 learned_addr learned my addr 172.21.15.19:0/2150523567 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:23.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.671+0000 7fa1a5400640 1 --2- 172.21.15.19:0/2150523567 >> v2:172.21.15.188:3300/0 conn(0x7fa1a009dcf0 0x7fa1a01453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:23.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.671+0000 7fa1a4a00640 1 -- 172.21.15.19:0/2150523567 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 283407977 0 0) 0x7fa1a015cb90 con 0x7fa1a00acf10 2024-03-28T13:16:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.671+0000 7fa1a4a00640 1 -- 172.21.15.19:0/2150523567 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa188003520 con 0x7fa1a00acf10 2024-03-28T13:16:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a4a00640 1 -- 172.21.15.19:0/2150523567 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3333598678 0 0) 0x7fa188003520 con 0x7fa1a00acf10 2024-03-28T13:16:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a4a00640 1 -- 172.21.15.19:0/2150523567 >> v2:172.21.15.188:3300/0 conn(0x7fa1a009dcf0 msgr2=0x7fa1a01453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a4a00640 1 --2- 172.21.15.19:0/2150523567 >> v2:172.21.15.188:3300/0 conn(0x7fa1a009dcf0 0x7fa1a01453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a4a00640 1 -- 172.21.15.19:0/2150523567 >> v1:172.21.15.188:6789/0 conn(0x7fa1a0071710 legacy=0x7fa1a0071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a4a00640 1 -- 172.21.15.19:0/2150523567 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a015d150 con 0x7fa1a00acf10 2024-03-28T13:16:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a4a00640 1 -- 172.21.15.19:0/2150523567 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fa194001670 con 0x7fa1a00acf10 2024-03-28T13:16:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a5400640 1 --2- 172.21.15.19:0/2150523567 >> v2:172.21.15.188:3300/0 conn(0x7fa1a009dcf0 0x7fa1a01453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:16:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a4a00640 1 -- 172.21.15.19:0/2150523567 >> v1:172.21.15.19:6789/0 conn(0x7fa1a00acf10 legacy=0x7fa1a00ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a4a00640 1 --2- 172.21.15.19:0/2150523567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188003a00 0x7fa188003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:23.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a4a00640 1 --2- 172.21.15.19:0/2150523567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa188004800 0x7fa188004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a4a00640 1 --2- 172.21.15.19:0/2150523567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa188005210 0x7fa18800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a4a00640 1 -- 172.21.15.19:0/2150523567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a0145f60 con 0x7fa188003a00 2024-03-28T13:16:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a4a00640 1 -- 172.21.15.19:0/2150523567 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa18800c160 con 0x7fa188005210 2024-03-28T13:16:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a4a00640 1 -- 172.21.15.19:0/2150523567 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa18800c360 con 0x7fa188004800 2024-03-28T13:16:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a5e00640 1 --2- 172.21.15.19:0/2150523567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188003a00 0x7fa188003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.672+0000 7fa1a5400640 1 --2- 172.21.15.19:0/2150523567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa188004800 0x7fa188004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.673+0000 7fa1a6800640 1 --2- 172.21.15.19:0/2150523567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa188005210 0x7fa18800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.673+0000 7fa1a5e00640 1 -- 172.21.15.19:0/2150523567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa188004800 msgr2=0x7fa188004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.673+0000 7fa1a5e00640 1 --2- 172.21.15.19:0/2150523567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa188004800 0x7fa188004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.673+0000 7fa1a5e00640 1 -- 172.21.15.19:0/2150523567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa188005210 msgr2=0x7fa18800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:23.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.673+0000 7fa1a5e00640 1 --2- 172.21.15.19:0/2150523567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa188005210 0x7fa18800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:23.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.673+0000 7fa1a6800640 1 -- 172.21.15.19:0/2150523567 reap_dead start 2024-03-28T13:16:23.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.673+0000 7fa1a5e00640 1 -- 172.21.15.19:0/2150523567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa188003520 con 0x7fa188003a00 2024-03-28T13:16:23.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.674+0000 7fa1a5e00640 1 --2- 172.21.15.19:0/2150523567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188003a00 0x7fa188003e20 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fa194007fd0 tx=0x7fa188005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3677e559193ae8d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:23.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.674+0000 7fa1a4a00640 1 -- 172.21.15.19:0/2150523567 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194007c20 con 0x7fa188003a00 2024-03-28T13:16:23.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.674+0000 7fa1a4a00640 1 -- 172.21.15.19:0/2150523567 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa194030070 con 0x7fa188003a00 2024-03-28T13:16:23.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.674+0000 7fa1a4a00640 1 -- 172.21.15.19:0/2150523567 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194005d70 con 0x7fa188003a00 2024-03-28T13:16:23.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.675+0000 7fa1a8800640 1 -- 172.21.15.19:0/2150523567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188003a00 msgr2=0x7fa188003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:23.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.675+0000 7fa1a8800640 1 --2- 172.21.15.19:0/2150523567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188003a00 0x7fa188003e20 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fa194007fd0 tx=0x7fa188005830 comp rx=0 tx=0).stop 2024-03-28T13:16:23.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.675+0000 7fa1a8800640 1 -- 172.21.15.19:0/2150523567 shutdown_connections 2024-03-28T13:16:23.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.675+0000 7fa1a8800640 1 --2- 172.21.15.19:0/2150523567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188003a00 0x7fa188003e20 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:23.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.675+0000 7fa1a8800640 1 -- 172.21.15.19:0/2150523567 >> 172.21.15.19:0/2150523567 conn(0x7fa1a006bba0 msgr2=0x7fa1a00b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:23.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.675+0000 7fa1a8800640 1 -- 172.21.15.19:0/2150523567 shutdown_connections 2024-03-28T13:16:23.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.675+0000 7fa1a8800640 1 -- 172.21.15.19:0/2150523567 wait complete. 2024-03-28T13:16:23.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.677+0000 7fa1a8800640 1 Processor -- start 2024-03-28T13:16:23.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.677+0000 7fa1a8800640 1 -- start start 2024-03-28T13:16:23.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.678+0000 7fa1a8800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188003a00 0x7fa1a0144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:23.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.678+0000 7fa1a8800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a015db40 0x7fa1a015f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.678+0000 7fa1a8800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a00ce6c0 0x7fa1a00ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.678+0000 7fa1a8800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a015d7a0 con 0x7fa188003a00 2024-03-28T13:16:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.678+0000 7fa1a8800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a015c950 con 0x7fa1a00ce6c0 2024-03-28T13:16:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.678+0000 7fa1a8800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a00ad2f0 con 0x7fa1a015db40 2024-03-28T13:16:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.678+0000 7fa1a5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188003a00 0x7fa1a0144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.678+0000 7fa1a5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188003a00 0x7fa1a0144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49462/0 (socket says 172.21.15.19:49462) 2024-03-28T13:16:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.678+0000 7fa1a5e00640 1 -- 172.21.15.19:0/2256969546 learned_addr learned my addr 172.21.15.19:0/2256969546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.678+0000 7fa1a5400640 1 --2- 172.21.15.19:0/2256969546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a015db40 0x7fa1a015f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.678+0000 7fa1a5e00640 1 -- 172.21.15.19:0/2256969546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a015db40 msgr2=0x7fa1a015f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:23.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.678+0000 7fa1a5e00640 1 --2- 172.21.15.19:0/2256969546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a015db40 0x7fa1a015f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:23.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.678+0000 7fa1a5e00640 1 -- 172.21.15.19:0/2256969546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a00ce6c0 msgr2=0x7fa1a00ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:23.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.678+0000 7fa1a5e00640 1 --2- 172.21.15.19:0/2256969546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a00ce6c0 0x7fa1a00ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:23.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.678+0000 7fa1a5e00640 1 -- 172.21.15.19:0/2256969546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a015e320 con 0x7fa188003a00 2024-03-28T13:16:23.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.679+0000 7fa1a5e00640 1 --2- 172.21.15.19:0/2256969546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188003a00 0x7fa1a0144850 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fa19400c1d0 tx=0x7fa1a015d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:23.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.679+0000 7fa19a000640 1 -- 172.21.15.19:0/2256969546 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194001b60 con 0x7fa188003a00 2024-03-28T13:16:23.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.679+0000 7fa19a000640 1 -- 172.21.15.19:0/2256969546 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa194000ed0 con 0x7fa188003a00 2024-03-28T13:16:23.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.679+0000 7fa19a000640 1 -- 172.21.15.19:0/2256969546 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19402ce10 con 0x7fa188003a00 2024-03-28T13:16:23.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.679+0000 7fa1a8800640 1 -- 172.21.15.19:0/2256969546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a015e600 con 0x7fa188003a00 2024-03-28T13:16:23.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.679+0000 7fa1a8800640 1 -- 172.21.15.19:0/2256969546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a013e120 con 0x7fa188003a00 2024-03-28T13:16:23.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.680+0000 7fa1a8800640 1 -- 172.21.15.19:0/2256969546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a00685c0 con 0x7fa188003a00 2024-03-28T13:16:23.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.681+0000 7fa19a000640 1 -- 172.21.15.19:0/2256969546 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa194007070 con 0x7fa188003a00 2024-03-28T13:16:23.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.682+0000 7fa19a000640 1 --2- 172.21.15.19:0/2256969546 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa18006c560 0x7fa18006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:23.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.682+0000 7fa19a000640 1 -- 172.21.15.19:0/2256969546 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 4857+0+0 (secure 0 0 0) 0x7fa1940ac7e0 con 0x7fa188003a00 2024-03-28T13:16:23.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.683+0000 7fa1a5400640 1 --2- 172.21.15.19:0/2256969546 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa18006c560 0x7fa18006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:23.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.684+0000 7fa1a5400640 1 --2- 172.21.15.19:0/2256969546 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa18006c560 0x7fa18006ea20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa1a006f7b0 tx=0x7fa18c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:23.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.687+0000 7fa19a000640 1 -- 172.21.15.19:0/2256969546 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa19407b140 con 0x7fa188003a00 2024-03-28T13:16:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:23.885+0000 7fa1a8800640 1 -- 172.21.15.19:0/2256969546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa1a00b2900 con 0x7fa188003a00 2024-03-28T13:16:24.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.505+0000 7fa19a000640 1 -- 172.21.15.19:0/2256969546 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v89) ==== 138+0+0 (secure 0 0 0) 0x7fa19407aa40 con 0x7fa188003a00 2024-03-28T13:16:24.505 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-03-28T13:16:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.509+0000 7fa1a8800640 1 -- 172.21.15.19:0/2256969546 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa18006c560 msgr2=0x7fa18006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.509+0000 7fa1a8800640 1 --2- 172.21.15.19:0/2256969546 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa18006c560 0x7fa18006ea20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa1a006f7b0 tx=0x7fa18c008040 comp rx=0 tx=0).stop 2024-03-28T13:16:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.509+0000 7fa1a8800640 1 -- 172.21.15.19:0/2256969546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188003a00 msgr2=0x7fa1a0144850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.509+0000 7fa1a8800640 1 --2- 172.21.15.19:0/2256969546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188003a00 0x7fa1a0144850 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fa19400c1d0 tx=0x7fa1a015d710 comp rx=0 tx=0).stop 2024-03-28T13:16:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.509+0000 7fa1a8800640 1 -- 172.21.15.19:0/2256969546 shutdown_connections 2024-03-28T13:16:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.509+0000 7fa1a8800640 1 --2- 172.21.15.19:0/2256969546 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa18006c560 0x7fa18006ea20 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:24.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.509+0000 7fa1a8800640 1 --2- 172.21.15.19:0/2256969546 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a00ce6c0 0x7fa1a00ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:24.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.509+0000 7fa1a8800640 1 --2- 172.21.15.19:0/2256969546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a015db40 0x7fa1a015f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:24.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.509+0000 7fa1a8800640 1 --2- 172.21.15.19:0/2256969546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa188003a00 0x7fa1a0144850 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:24.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.509+0000 7fa1a8800640 1 -- 172.21.15.19:0/2256969546 >> 172.21.15.19:0/2256969546 conn(0x7fa1a006bba0 msgr2=0x7fa1a013fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:24.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.509+0000 7fa1a8800640 1 -- 172.21.15.19:0/2256969546 shutdown_connections 2024-03-28T13:16:24.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.509+0000 7fa1a8800640 1 -- 172.21.15.19:0/2256969546 wait complete. 2024-03-28T13:16:24.535 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-28T13:16:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.749+0000 7fe3d0400640 1 Processor -- start 2024-03-28T13:16:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.750+0000 7fe3d0400640 1 -- start start 2024-03-28T13:16:24.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.752+0000 7fe3d0400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe3c80d0280 0x7fe3c80d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.752+0000 7fe3d0400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3c8141c10 0x7fe3c8071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.752+0000 7fe3d0400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3c815ec80 con 0x7fe3c8071eb0 2024-03-28T13:16:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.752+0000 7fe3d0400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe3c80d0ba0 con 0x7fe3c80d0280 2024-03-28T13:16:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.752+0000 7fe3d0400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c815ef20 con 0x7fe3c8141c10 2024-03-28T13:16:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.752+0000 7fe3cda00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe3c80d0280 0x7fe3c80d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.752+0000 7fe3cda00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe3c80d0280 0x7fe3c80d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49476/0 (socket says 172.21.15.19:49476) 2024-03-28T13:16:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.752+0000 7fe3cda00640 1 -- 172.21.15.19:0/1289220469 learned_addr learned my addr 172.21.15.19:0/1289220469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.752+0000 7fe3cd000640 1 --2- 172.21.15.19:0/1289220469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3c8141c10 0x7fe3c8071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.753+0000 7fe3cda00640 1 -- 172.21.15.19:0/1289220469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3c8141c10 msgr2=0x7fe3c8071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.753+0000 7fe3cda00640 1 --2- 172.21.15.19:0/1289220469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3c8141c10 0x7fe3c8071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.753+0000 7fe3cda00640 1 -- 172.21.15.19:0/1289220469 >> v1:172.21.15.188:6789/0 conn(0x7fe3c8071eb0 legacy=0x7fe3c806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.753+0000 7fe3cda00640 1 -- 172.21.15.19:0/1289220469 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c815f200 con 0x7fe3c80d0280 2024-03-28T13:16:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.755+0000 7fe3cda00640 1 --2- 172.21.15.19:0/1289220469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c80d0280 0x7fe3c80d0660 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009e50 tx=0x7fe3bc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7de6711552d9ad server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:24.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.755+0000 7fe3c7e00640 1 -- 172.21.15.19:0/1289220469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc02cb00 con 0x7fe3c80d0280 2024-03-28T13:16:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.755+0000 7fe3c7e00640 1 -- 172.21.15.19:0/1289220469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3bc02cca0 con 0x7fe3c80d0280 2024-03-28T13:16:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.755+0000 7fe3c7e00640 1 -- 172.21.15.19:0/1289220469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc034610 con 0x7fe3c80d0280 2024-03-28T13:16:24.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.756+0000 7fe3d0400640 1 -- 172.21.15.19:0/1289220469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c80d0280 msgr2=0x7fe3c80d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.756+0000 7fe3d0400640 1 --2- 172.21.15.19:0/1289220469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c80d0280 0x7fe3c80d0660 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009e50 tx=0x7fe3bc02bf60 comp rx=0 tx=0).stop 2024-03-28T13:16:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.756+0000 7fe3d0400640 1 -- 172.21.15.19:0/1289220469 shutdown_connections 2024-03-28T13:16:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.756+0000 7fe3d0400640 1 --2- 172.21.15.19:0/1289220469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3c8141c10 0x7fe3c8071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.756+0000 7fe3d0400640 1 --2- 172.21.15.19:0/1289220469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c80d0280 0x7fe3c80d0660 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.756+0000 7fe3d0400640 1 -- 172.21.15.19:0/1289220469 >> 172.21.15.19:0/1289220469 conn(0x7fe3c806b750 msgr2=0x7fe3c806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.756+0000 7fe3d0400640 1 -- 172.21.15.19:0/1289220469 shutdown_connections 2024-03-28T13:16:24.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.756+0000 7fe3d0400640 1 -- 172.21.15.19:0/1289220469 wait complete. 2024-03-28T13:16:24.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.758+0000 7fe3d0400640 1 Processor -- start 2024-03-28T13:16:24.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.758+0000 7fe3d0400640 1 -- start start 2024-03-28T13:16:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.759+0000 7fe3d0400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c8071eb0 0x7fe3c81ea8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.759+0000 7fe3d0400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3c80d0280 0x7fe3c809be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.759+0000 7fe3d0400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3c8141c10 0x7fe3c809a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.759+0000 7fe3d0400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c8158e90 con 0x7fe3c8071eb0 2024-03-28T13:16:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.759+0000 7fe3d0400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c815f7c0 con 0x7fe3c8141c10 2024-03-28T13:16:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.759+0000 7fe3d0400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c809c370 con 0x7fe3c80d0280 2024-03-28T13:16:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.759+0000 7fe3cda00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c8071eb0 0x7fe3c81ea8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.759+0000 7fe3cda00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c8071eb0 0x7fe3c81ea8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49480/0 (socket says 172.21.15.19:49480) 2024-03-28T13:16:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.759+0000 7fe3cda00640 1 -- 172.21.15.19:0/2369901813 learned_addr learned my addr 172.21.15.19:0/2369901813 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.759+0000 7fe3ce400640 1 --2- 172.21.15.19:0/2369901813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3c8141c10 0x7fe3c809a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.759+0000 7fe3cd000640 1 --2- 172.21.15.19:0/2369901813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3c80d0280 0x7fe3c809be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.760+0000 7fe3cda00640 1 -- 172.21.15.19:0/2369901813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3c80d0280 msgr2=0x7fe3c809be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.760+0000 7fe3cda00640 1 --2- 172.21.15.19:0/2369901813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3c80d0280 0x7fe3c809be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.760+0000 7fe3cda00640 1 -- 172.21.15.19:0/2369901813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3c8141c10 msgr2=0x7fe3c809a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.760+0000 7fe3cda00640 1 --2- 172.21.15.19:0/2369901813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3c8141c10 0x7fe3c809a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.760+0000 7fe3cda00640 1 -- 172.21.15.19:0/2369901813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c81e7fb0 con 0x7fe3c8071eb0 2024-03-28T13:16:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.760+0000 7fe3cda00640 1 --2- 172.21.15.19:0/2369901813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c8071eb0 0x7fe3c81ea8c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009e50 tx=0x7fe3bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.760+0000 7fe3c5600640 1 -- 172.21.15.19:0/2369901813 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc043020 con 0x7fe3c8071eb0 2024-03-28T13:16:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.760+0000 7fe3c5600640 1 -- 172.21.15.19:0/2369901813 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3bc02ce50 con 0x7fe3c8071eb0 2024-03-28T13:16:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.760+0000 7fe3c5600640 1 -- 172.21.15.19:0/2369901813 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc009510 con 0x7fe3c8071eb0 2024-03-28T13:16:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.760+0000 7fe3d0400640 1 -- 172.21.15.19:0/2369901813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c81f59e0 con 0x7fe3c8071eb0 2024-03-28T13:16:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.761+0000 7fe3d0400640 1 -- 172.21.15.19:0/2369901813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c81e8220 con 0x7fe3c8071eb0 2024-03-28T13:16:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.762+0000 7fe3d0400640 1 -- 172.21.15.19:0/2369901813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c80685c0 con 0x7fe3c8071eb0 2024-03-28T13:16:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.763+0000 7fe3c5600640 1 -- 172.21.15.19:0/2369901813 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe3bc046020 con 0x7fe3c8071eb0 2024-03-28T13:16:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.764+0000 7fe3c5600640 1 --2- 172.21.15.19:0/2369901813 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe3a806c560 0x7fe3a806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.764+0000 7fe3c5600640 1 -- 172.21.15.19:0/2369901813 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5227+0+0 (secure 0 0 0) 0x7fe3bc0ad930 con 0x7fe3c8071eb0 2024-03-28T13:16:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.765+0000 7fe3cd000640 1 --2- 172.21.15.19:0/2369901813 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe3a806c560 0x7fe3a806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:24.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.766+0000 7fe3cd000640 1 --2- 172.21.15.19:0/2369901813 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe3a806c560 0x7fe3a806ea20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8141ff0 tx=0x7fe3b4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:24.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.770+0000 7fe3c5600640 1 -- 172.21.15.19:0/2369901813 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe3bc04d070 con 0x7fe3c8071eb0 2024-03-28T13:16:24.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:24.973+0000 7fe3d0400640 1 -- 172.21.15.19:0/2369901813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe3c81eae70 con 0x7fe3c8071eb0 2024-03-28T13:16:25.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.512+0000 7fe3c5600640 1 -- 172.21.15.19:0/2369901813 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v90) ==== 170+0+0 (secure 0 0 0) 0x7fe3bc07c090 con 0x7fe3c8071eb0 2024-03-28T13:16:25.513 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-03-28T13:16:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.517+0000 7fe3d0400640 1 -- 172.21.15.19:0/2369901813 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe3a806c560 msgr2=0x7fe3a806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.517+0000 7fe3d0400640 1 --2- 172.21.15.19:0/2369901813 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe3a806c560 0x7fe3a806ea20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8141ff0 tx=0x7fe3b4008040 comp rx=0 tx=0).stop 2024-03-28T13:16:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.517+0000 7fe3d0400640 1 -- 172.21.15.19:0/2369901813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c8071eb0 msgr2=0x7fe3c81ea8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:25.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.517+0000 7fe3d0400640 1 --2- 172.21.15.19:0/2369901813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c8071eb0 0x7fe3c81ea8c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc009e50 tx=0x7fe3bc004a40 comp rx=0 tx=0).stop 2024-03-28T13:16:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.518+0000 7fe3d0400640 1 -- 172.21.15.19:0/2369901813 shutdown_connections 2024-03-28T13:16:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.518+0000 7fe3d0400640 1 --2- 172.21.15.19:0/2369901813 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe3a806c560 0x7fe3a806ea20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.518+0000 7fe3d0400640 1 --2- 172.21.15.19:0/2369901813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3c8141c10 0x7fe3c809a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.518+0000 7fe3d0400640 1 --2- 172.21.15.19:0/2369901813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3c80d0280 0x7fe3c809be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.518+0000 7fe3d0400640 1 --2- 172.21.15.19:0/2369901813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3c8071eb0 0x7fe3c81ea8c0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:25.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.518+0000 7fe3d0400640 1 -- 172.21.15.19:0/2369901813 >> 172.21.15.19:0/2369901813 conn(0x7fe3c806b750 msgr2=0x7fe3c8160ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:25.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.518+0000 7fe3d0400640 1 -- 172.21.15.19:0/2369901813 shutdown_connections 2024-03-28T13:16:25.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.519+0000 7fe3d0400640 1 -- 172.21.15.19:0/2369901813 wait complete. 2024-03-28T13:16:25.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-28T13:16:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.739+0000 7f7001c00640 1 Processor -- start 2024-03-28T13:16:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.739+0000 7f7001c00640 1 -- start start 2024-03-28T13:16:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.741+0000 7f7001c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6ffc0d0280 0x7f6ffc0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.741+0000 7f7001c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ffc141c10 0x7f6ffc071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.741+0000 7f7001c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ffc071eb0 0x7f6ffc06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.741+0000 7f7001c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6ffc15ec80 con 0x7f6ffc141c10 2024-03-28T13:16:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.741+0000 7f7001c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f6ffc0d0ba0 con 0x7f6ffc0d0280 2024-03-28T13:16:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.741+0000 7f7001c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6ffc15ef20 con 0x7f6ffc071eb0 2024-03-28T13:16:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.741+0000 7f6ff3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ffc141c10 0x7f6ffc071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.742+0000 7f6ff3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6ffc141c10 0x7f6ffc071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39340/0 (socket says 172.21.15.19:39340) 2024-03-28T13:16:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.742+0000 7f6ff3e00640 1 -- 172.21.15.19:0/2024856372 learned_addr learned my addr 172.21.15.19:0/2024856372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.742+0000 7f6ffaa00640 1 --2- 172.21.15.19:0/2024856372 >> v2:172.21.15.188:3300/0 conn(0x7f6ffc0d0280 0x7f6ffc0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.742+0000 7f6ffb400640 1 --2- 172.21.15.19:0/2024856372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ffc071eb0 0x7f6ffc06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.742+0000 7f6ff3e00640 1 -- 172.21.15.19:0/2024856372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ffc071eb0 msgr2=0x7f6ffc06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.742+0000 7f6ff3e00640 1 --2- 172.21.15.19:0/2024856372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ffc071eb0 0x7f6ffc06d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.742+0000 7f6ff3e00640 1 -- 172.21.15.19:0/2024856372 >> v2:172.21.15.188:3300/0 conn(0x7f6ffc0d0280 msgr2=0x7f6ffc0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.742+0000 7f6ff3e00640 1 --2- 172.21.15.19:0/2024856372 >> v2:172.21.15.188:3300/0 conn(0x7f6ffc0d0280 0x7f6ffc0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.742+0000 7f6ff3e00640 1 -- 172.21.15.19:0/2024856372 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ffc15f200 con 0x7f6ffc141c10 2024-03-28T13:16:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.742+0000 7f6ffb400640 1 --2- 172.21.15.19:0/2024856372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ffc071eb0 0x7f6ffc06d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.743+0000 7f6ff3e00640 1 --2- 172.21.15.19:0/2024856372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ffc141c10 0x7f6ffc071970 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0d1050 tx=0x7f6fe402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca8682af5ff9042 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.743+0000 7f6ffa000640 1 -- 172.21.15.19:0/2024856372 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4035020 con 0x7f6ffc141c10 2024-03-28T13:16:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.743+0000 7f6ffa000640 1 -- 172.21.15.19:0/2024856372 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fe4031880 con 0x7f6ffc141c10 2024-03-28T13:16:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.743+0000 7f6ffa000640 1 -- 172.21.15.19:0/2024856372 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4031c50 con 0x7f6ffc141c10 2024-03-28T13:16:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.744+0000 7f7001c00640 1 -- 172.21.15.19:0/2024856372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ffc141c10 msgr2=0x7f6ffc071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.744+0000 7f7001c00640 1 --2- 172.21.15.19:0/2024856372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ffc141c10 0x7f6ffc071970 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0d1050 tx=0x7f6fe402b920 comp rx=0 tx=0).stop 2024-03-28T13:16:25.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.744+0000 7f7001c00640 1 -- 172.21.15.19:0/2024856372 shutdown_connections 2024-03-28T13:16:25.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.744+0000 7f7001c00640 1 --2- 172.21.15.19:0/2024856372 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ffc071eb0 0x7f6ffc06d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:25.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.744+0000 7f7001c00640 1 --2- 172.21.15.19:0/2024856372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ffc141c10 0x7f6ffc071970 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:25.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.744+0000 7f7001c00640 1 --2- 172.21.15.19:0/2024856372 >> v2:172.21.15.188:3300/0 conn(0x7f6ffc0d0280 0x7f6ffc0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:25.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.744+0000 7f7001c00640 1 -- 172.21.15.19:0/2024856372 >> 172.21.15.19:0/2024856372 conn(0x7f6ffc06b750 msgr2=0x7f6ffc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:25.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.744+0000 7f7001c00640 1 -- 172.21.15.19:0/2024856372 shutdown_connections 2024-03-28T13:16:25.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.744+0000 7f7001c00640 1 -- 172.21.15.19:0/2024856372 wait complete. 2024-03-28T13:16:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.746+0000 7f7001c00640 1 Processor -- start 2024-03-28T13:16:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.746+0000 7f7001c00640 1 -- start start 2024-03-28T13:16:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.747+0000 7f7001c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ffc071eb0 0x7f6ffc0999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.747+0000 7f7001c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ffc0d0280 0x7f6ffc0ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.747+0000 7f7001c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ffc141c10 0x7f6ffc0ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.747+0000 7f7001c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc158e90 con 0x7f6ffc141c10 2024-03-28T13:16:25.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.747+0000 7f7001c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6ffc15f7c0 con 0x7f6ffc071eb0 2024-03-28T13:16:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.747+0000 7f7001c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6ffc09e160 con 0x7f6ffc0d0280 2024-03-28T13:16:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.747+0000 7f6ffb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ffc141c10 0x7f6ffc0ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.748+0000 7f6ffb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ffc141c10 0x7f6ffc0ae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39346/0 (socket says 172.21.15.19:39346) 2024-03-28T13:16:25.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.748+0000 7f6ffb400640 1 -- 172.21.15.19:0/3930097215 learned_addr learned my addr 172.21.15.19:0/3930097215 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.748+0000 7f6ffb400640 1 -- 172.21.15.19:0/3930097215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ffc0d0280 msgr2=0x7f6ffc0ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.748+0000 7f6ffb400640 1 --2- 172.21.15.19:0/3930097215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ffc0d0280 0x7f6ffc0ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.748+0000 7f6ffb400640 1 -- 172.21.15.19:0/3930097215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ffc071eb0 msgr2=0x7f6ffc0999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.748+0000 7f6ffaa00640 1 --2- 172.21.15.19:0/3930097215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ffc071eb0 0x7f6ffc0999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.748+0000 7f6ffb400640 1 --2- 172.21.15.19:0/3930097215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ffc071eb0 0x7f6ffc0999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.748+0000 7f6ffb400640 1 -- 172.21.15.19:0/3930097215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ffc1e6ea0 con 0x7f6ffc141c10 2024-03-28T13:16:25.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.748+0000 7f6ffaa00640 1 --2- 172.21.15.19:0/3930097215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ffc071eb0 0x7f6ffc0999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.748+0000 7f6ffb400640 1 --2- 172.21.15.19:0/3930097215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ffc141c10 0x7f6ffc0ae5a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400cef0 tx=0x7f6ff400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.749+0000 7f6ff2a00640 1 -- 172.21.15.19:0/3930097215 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff401a070 con 0x7f6ffc141c10 2024-03-28T13:16:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.749+0000 7f6ff2a00640 1 -- 172.21.15.19:0/3930097215 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ff400d840 con 0x7f6ffc141c10 2024-03-28T13:16:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.749+0000 7f6ff2a00640 1 -- 172.21.15.19:0/3930097215 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff400dc30 con 0x7f6ffc141c10 2024-03-28T13:16:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.749+0000 7f7001c00640 1 -- 172.21.15.19:0/3930097215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ffc1e7180 con 0x7f6ffc141c10 2024-03-28T13:16:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.749+0000 7f7001c00640 1 -- 172.21.15.19:0/3930097215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ffc1e7740 con 0x7f6ffc141c10 2024-03-28T13:16:25.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.750+0000 7f7001c00640 1 -- 172.21.15.19:0/3930097215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ffc0685c0 con 0x7f6ffc141c10 2024-03-28T13:16:25.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.751+0000 7f6ff2a00640 1 -- 172.21.15.19:0/3930097215 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6ff401d070 con 0x7f6ffc141c10 2024-03-28T13:16:25.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.753+0000 7f6ff2a00640 1 --2- 172.21.15.19:0/3930097215 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6fd806c530 0x7f6fd806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:25.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.753+0000 7f6ff2a00640 1 -- 172.21.15.19:0/3930097215 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5609+0+0 (secure 0 0 0) 0x7f6ff408bf50 con 0x7f6ffc141c10 2024-03-28T13:16:25.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.754+0000 7f6ffaa00640 1 --2- 172.21.15.19:0/3930097215 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6fd806c530 0x7f6fd806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:25.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.755+0000 7f6ffaa00640 1 --2- 172.21.15.19:0/3930097215 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6fd806c530 0x7f6fd806e9f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6fec006fd0 tx=0x7f6fec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:25.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.758+0000 7f6ff2a00640 1 -- 172.21.15.19:0/3930097215 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6ff405a530 con 0x7f6ffc141c10 2024-03-28T13:16:25.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:25.992+0000 7f7001c00640 1 -- 172.21.15.19:0/3930097215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6ffc0cfdb0 con 0x7f6ffc141c10 2024-03-28T13:16:26.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.561+0000 7f6ff2a00640 1 -- 172.21.15.19:0/3930097215 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v58) ==== 174+0+0 (secure 0 0 0) 0x7f6ff4059e30 con 0x7f6ffc141c10 2024-03-28T13:16:26.563 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 12 and data pool 13 2024-03-28T13:16:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.576+0000 7f7001c00640 1 -- 172.21.15.19:0/3930097215 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6fd806c530 msgr2=0x7f6fd806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.576+0000 7f7001c00640 1 --2- 172.21.15.19:0/3930097215 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6fd806c530 0x7f6fd806e9f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6fec006fd0 tx=0x7f6fec008040 comp rx=0 tx=0).stop 2024-03-28T13:16:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.576+0000 7f7001c00640 1 -- 172.21.15.19:0/3930097215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ffc141c10 msgr2=0x7f6ffc0ae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.576+0000 7f7001c00640 1 --2- 172.21.15.19:0/3930097215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ffc141c10 0x7f6ffc0ae5a0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400cef0 tx=0x7f6ff400a710 comp rx=0 tx=0).stop 2024-03-28T13:16:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.576+0000 7f7001c00640 1 -- 172.21.15.19:0/3930097215 shutdown_connections 2024-03-28T13:16:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.576+0000 7f7001c00640 1 --2- 172.21.15.19:0/3930097215 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6fd806c530 0x7f6fd806e9f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.576+0000 7f7001c00640 1 --2- 172.21.15.19:0/3930097215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6ffc141c10 0x7f6ffc0ae5a0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.576+0000 7f7001c00640 1 --2- 172.21.15.19:0/3930097215 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ffc0d0280 0x7f6ffc0ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.576+0000 7f7001c00640 1 --2- 172.21.15.19:0/3930097215 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ffc071eb0 0x7f6ffc0999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:26.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.576+0000 7f7001c00640 1 -- 172.21.15.19:0/3930097215 >> 172.21.15.19:0/3930097215 conn(0x7f6ffc06b750 msgr2=0x7f6ffc0b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:26.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.577+0000 7f7001c00640 1 -- 172.21.15.19:0/3930097215 shutdown_connections 2024-03-28T13:16:26.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.578+0000 7f7001c00640 1 -- 172.21.15.19:0/3930097215 wait complete. 2024-03-28T13:16:26.604 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:16:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.780+0000 7f49bb000640 1 Processor -- start 2024-03-28T13:16:26.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.781+0000 7f49bb000640 1 -- start start 2024-03-28T13:16:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.782+0000 7f49bb000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49b415b310 0x7f49b40727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.782+0000 7f49bb000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49b40d0780 0x7f49b40d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.782+0000 7f49bb000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49b415b6f0 con 0x7f49b40b4f90 2024-03-28T13:16:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.782+0000 7f49bb000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f49b4072e80 con 0x7f49b415b310 2024-03-28T13:16:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.782+0000 7f49bb000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f49b4073160 con 0x7f49b40d0780 2024-03-28T13:16:26.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.782+0000 7f49b3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49b415b310 0x7f49b40727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.782+0000 7f49b3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49b415b310 0x7f49b40727f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39362/0 (socket says 172.21.15.19:39362) 2024-03-28T13:16:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.782+0000 7f49b3e00640 1 -- 172.21.15.19:0/2147651942 learned_addr learned my addr 172.21.15.19:0/2147651942 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.782+0000 7f49b9000640 1 --2- 172.21.15.19:0/2147651942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49b40d0780 0x7f49b40d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.782+0000 7f49b3e00640 1 -- 172.21.15.19:0/2147651942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49b40d0780 msgr2=0x7f49b40d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.782+0000 7f49b3e00640 1 --2- 172.21.15.19:0/2147651942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49b40d0780 0x7f49b40d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.782+0000 7f49b3e00640 1 -- 172.21.15.19:0/2147651942 >> v1:172.21.15.188:6789/0 conn(0x7f49b40b4f90 legacy=0x7f49b415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.782+0000 7f49b3e00640 1 -- 172.21.15.19:0/2147651942 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b415efe0 con 0x7f49b415b310 2024-03-28T13:16:26.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.783+0000 7f49b9000640 1 --2- 172.21.15.19:0/2147651942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49b40d0780 0x7f49b40d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:26.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.784+0000 7f49b3e00640 1 --2- 172.21.15.19:0/2147651942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49b415b310 0x7f49b40727f0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009870 tx=0x7f49a402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad282ca59c8c162 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:26.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.784+0000 7f49b3400640 1 -- 172.21.15.19:0/2147651942 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a4038070 con 0x7f49b415b310 2024-03-28T13:16:26.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.784+0000 7f49b3400640 1 -- 172.21.15.19:0/2147651942 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49a4031740 con 0x7f49b415b310 2024-03-28T13:16:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.784+0000 7f49b3400640 1 -- 172.21.15.19:0/2147651942 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49a402f050 con 0x7f49b415b310 2024-03-28T13:16:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.784+0000 7f49bb000640 1 -- 172.21.15.19:0/2147651942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49b415b310 msgr2=0x7f49b40727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.784+0000 7f49bb000640 1 --2- 172.21.15.19:0/2147651942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49b415b310 0x7f49b40727f0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009870 tx=0x7f49a402b7e0 comp rx=0 tx=0).stop 2024-03-28T13:16:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.785+0000 7f49bb000640 1 -- 172.21.15.19:0/2147651942 shutdown_connections 2024-03-28T13:16:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.785+0000 7f49bb000640 1 --2- 172.21.15.19:0/2147651942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49b40d0780 0x7f49b40d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.785+0000 7f49bb000640 1 --2- 172.21.15.19:0/2147651942 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49b415b310 0x7f49b40727f0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.785+0000 7f49bb000640 1 -- 172.21.15.19:0/2147651942 >> 172.21.15.19:0/2147651942 conn(0x7f49b406b990 msgr2=0x7f49b406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.785+0000 7f49bb000640 1 -- 172.21.15.19:0/2147651942 shutdown_connections 2024-03-28T13:16:26.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.785+0000 7f49bb000640 1 -- 172.21.15.19:0/2147651942 wait complete. 2024-03-28T13:16:26.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.786+0000 7f49bb000640 1 Processor -- start 2024-03-28T13:16:26.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.786+0000 7f49bb000640 1 -- start start 2024-03-28T13:16:26.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.787+0000 7f49bb000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49b40b4f90 0x7f49b4151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:26.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.787+0000 7f49bb000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49b40d0780 0x7f49b41510e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:26.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.787+0000 7f49bb000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49b415b310 0x7f49b415ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:26.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.787+0000 7f49bb000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49b4073440 con 0x7f49b415b310 2024-03-28T13:16:26.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.787+0000 7f49bb000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49b40b53f0 con 0x7f49b40b4f90 2024-03-28T13:16:26.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.787+0000 7f49bb000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f49b40adc80 con 0x7f49b40d0780 2024-03-28T13:16:26.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.788+0000 7f49b9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49b415b310 0x7f49b415ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:26.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.788+0000 7f49b9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49b415b310 0x7f49b415ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39366/0 (socket says 172.21.15.19:39366) 2024-03-28T13:16:26.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.788+0000 7f49b9000640 1 -- 172.21.15.19:0/2809899806 learned_addr learned my addr 172.21.15.19:0/2809899806 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:26.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.788+0000 7f49b3e00640 1 --2- 172.21.15.19:0/2809899806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49b40b4f90 0x7f49b4151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:26.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.788+0000 7f49ab400640 1 --2- 172.21.15.19:0/2809899806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49b40d0780 0x7f49b41510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:26.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.788+0000 7f49b9000640 1 -- 172.21.15.19:0/2809899806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49b40d0780 msgr2=0x7f49b41510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:26.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.788+0000 7f49b9000640 1 --2- 172.21.15.19:0/2809899806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49b40d0780 0x7f49b41510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:26.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.788+0000 7f49b9000640 1 -- 172.21.15.19:0/2809899806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49b40b4f90 msgr2=0x7f49b4151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:26.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.788+0000 7f49b9000640 1 --2- 172.21.15.19:0/2809899806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49b40b4f90 0x7f49b4151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:26.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.788+0000 7f49b9000640 1 -- 172.21.15.19:0/2809899806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49b415bfc0 con 0x7f49b415b310 2024-03-28T13:16:26.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.788+0000 7f49ab400640 1 --2- 172.21.15.19:0/2809899806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49b40d0780 0x7f49b41510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:26.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.788+0000 7f49b9000640 1 --2- 172.21.15.19:0/2809899806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49b415b310 0x7f49b415ba80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00c7f0 tx=0x7f49ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:26.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.789+0000 7f49b0c00640 1 -- 172.21.15.19:0/2809899806 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac015070 con 0x7f49b415b310 2024-03-28T13:16:26.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.789+0000 7f49b0c00640 1 -- 172.21.15.19:0/2809899806 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ac007820 con 0x7f49b415b310 2024-03-28T13:16:26.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.789+0000 7f49b0c00640 1 -- 172.21.15.19:0/2809899806 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49ac007c50 con 0x7f49b415b310 2024-03-28T13:16:26.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.789+0000 7f49bb000640 1 -- 172.21.15.19:0/2809899806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49b4152120 con 0x7f49b415b310 2024-03-28T13:16:26.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.789+0000 7f49bb000640 1 -- 172.21.15.19:0/2809899806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49b41526c0 con 0x7f49b415b310 2024-03-28T13:16:26.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.790+0000 7f49bb000640 1 -- 172.21.15.19:0/2809899806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49b40685c0 con 0x7f49b415b310 2024-03-28T13:16:26.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.791+0000 7f49b0c00640 1 -- 172.21.15.19:0/2809899806 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f49ac018070 con 0x7f49b415b310 2024-03-28T13:16:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.792+0000 7f49b0c00640 1 --2- 172.21.15.19:0/2809899806 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f499006c530 0x7f499006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.792+0000 7f49b0c00640 1 -- 172.21.15.19:0/2809899806 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5709+0+0 (secure 0 0 0) 0x7f49ac08b920 con 0x7f49b415b310 2024-03-28T13:16:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.793+0000 7f49b3e00640 1 --2- 172.21.15.19:0/2809899806 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f499006c530 0x7f499006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:26.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.794+0000 7f49b3e00640 1 --2- 172.21.15.19:0/2809899806 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f499006c530 0x7f499006e9f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f49a4035000 tx=0x7f49a4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:26.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.797+0000 7f49b0c00640 1 -- 172.21.15.19:0/2809899806 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f49b415bfc0 con 0x7f49b415b310 2024-03-28T13:16:26.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.995+0000 7f49bb000640 1 -- 172.21.15.19:0/2809899806 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49b40cfa80 con 0x7f49b415b310 2024-03-28T13:16:26.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.997+0000 7f49b0c00640 1 -- 172.21.15.19:0/2809899806 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16685 (secure 0 0 0) 0x7f49ac059ea0 con 0x7f49b415b310 2024-03-28T13:16:26.997 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:26.998 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":91,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:16:25.994580+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:16:23.901120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-28T13:16:24.990904+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:16:27.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:26.999+0000 7f49bb000640 1 -- 172.21.15.19:0/2809899806 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f499006c530 msgr2=0x7f499006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:27.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.000+0000 7f49bb000640 1 --2- 172.21.15.19:0/2809899806 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f499006c530 0x7f499006e9f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f49a4035000 tx=0x7f49a4008030 comp rx=0 tx=0).stop 2024-03-28T13:16:27.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.000+0000 7f49bb000640 1 -- 172.21.15.19:0/2809899806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49b415b310 msgr2=0x7f49b415ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.000+0000 7f49bb000640 1 --2- 172.21.15.19:0/2809899806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49b415b310 0x7f49b415ba80 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f49ac00c7f0 tx=0x7f49ac00ccb0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.000+0000 7f49bb000640 1 -- 172.21.15.19:0/2809899806 shutdown_connections 2024-03-28T13:16:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.000+0000 7f49bb000640 1 --2- 172.21.15.19:0/2809899806 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f499006c530 0x7f499006e9f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.000+0000 7f49bb000640 1 --2- 172.21.15.19:0/2809899806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49b415b310 0x7f49b415ba80 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.000+0000 7f49bb000640 1 --2- 172.21.15.19:0/2809899806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49b40d0780 0x7f49b41510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.000+0000 7f49bb000640 1 --2- 172.21.15.19:0/2809899806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49b40b4f90 0x7f49b4151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.000+0000 7f49bb000640 1 -- 172.21.15.19:0/2809899806 >> 172.21.15.19:0/2809899806 conn(0x7f49b406b990 msgr2=0x7f49b4157150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.000+0000 7f49bb000640 1 -- 172.21.15.19:0/2809899806 shutdown_connections 2024-03-28T13:16:27.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.000+0000 7f49bb000640 1 -- 172.21.15.19:0/2809899806 wait complete. 2024-03-28T13:16:27.018 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:16:27.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.233+0000 7f99bcc00640 1 Processor -- start 2024-03-28T13:16:27.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.234+0000 7f99bcc00640 1 -- start start 2024-03-28T13:16:27.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.235+0000 7f99bcc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b815c600 0x7f99b80cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:27.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.235+0000 7f99bcc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99b815c9e0 con 0x7f99b80cf790 2024-03-28T13:16:27.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.235+0000 7f99bcc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99b806f6a0 con 0x7f99b806ef30 2024-03-28T13:16:27.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.235+0000 7f99bcc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f99b80cfbf0 con 0x7f99b815c600 2024-03-28T13:16:27.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.236+0000 7f99b5400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f99b80cf790 0x7f99b806e820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57260/0 (socket says 172.21.15.19:57260) 2024-03-28T13:16:27.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.236+0000 7f99b5400640 1 -- 172.21.15.19:0/3531803114 learned_addr learned my addr 172.21.15.19:0/3531803114 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:27.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.236+0000 7f99b5e00640 1 --2- 172.21.15.19:0/3531803114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b815c600 0x7f99b80cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:27.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.236+0000 7f99b4a00640 1 -- 172.21.15.19:0/3531803114 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4213815709 0 0) 0x7f99b815c9e0 con 0x7f99b80cf790 2024-03-28T13:16:27.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.236+0000 7f99b4a00640 1 -- 172.21.15.19:0/3531803114 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f999c003520 con 0x7f99b80cf790 2024-03-28T13:16:27.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.236+0000 7f99b5e00640 1 -- 172.21.15.19:0/3531803114 >> v1:172.21.15.188:6789/0 conn(0x7f99b806ef30 legacy=0x7f99b80b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:27.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.236+0000 7f99b5e00640 1 -- 172.21.15.19:0/3531803114 >> v1:172.21.15.19:6789/0 conn(0x7f99b80cf790 legacy=0x7f99b806e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:27.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.236+0000 7f99b5e00640 1 -- 172.21.15.19:0/3531803114 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b806f980 con 0x7f99b815c600 2024-03-28T13:16:27.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.238+0000 7f99b5e00640 1 --2- 172.21.15.19:0/3531803114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b815c600 0x7f99b80cf250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99a00097d0 tx=0x7f99a002b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=288c754f98a2445f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.238+0000 7f99b4a00640 1 -- 172.21.15.19:0/3531803114 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0034070 con 0x7f99b815c600 2024-03-28T13:16:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.238+0000 7f99b4a00640 1 -- 172.21.15.19:0/3531803114 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99a0030890 con 0x7f99b815c600 2024-03-28T13:16:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.238+0000 7f99b4a00640 1 -- 172.21.15.19:0/3531803114 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0030c60 con 0x7f99b815c600 2024-03-28T13:16:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.239+0000 7f99bcc00640 1 -- 172.21.15.19:0/3531803114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b815c600 msgr2=0x7f99b80cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.239+0000 7f99bcc00640 1 --2- 172.21.15.19:0/3531803114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b815c600 0x7f99b80cf250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f99a00097d0 tx=0x7f99a002b9d0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.239+0000 7f99bcc00640 1 -- 172.21.15.19:0/3531803114 shutdown_connections 2024-03-28T13:16:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.239+0000 7f99bcc00640 1 --2- 172.21.15.19:0/3531803114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b815c600 0x7f99b80cf250 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.239+0000 7f99bcc00640 1 -- 172.21.15.19:0/3531803114 >> 172.21.15.19:0/3531803114 conn(0x7f99b80b2cb0 msgr2=0x7f99b80b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.239+0000 7f99bcc00640 1 -- 172.21.15.19:0/3531803114 shutdown_connections 2024-03-28T13:16:27.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.239+0000 7f99bcc00640 1 -- 172.21.15.19:0/3531803114 wait complete. 2024-03-28T13:16:27.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.241+0000 7f99bcc00640 1 Processor -- start 2024-03-28T13:16:27.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.241+0000 7f99bcc00640 1 -- start start 2024-03-28T13:16:27.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.242+0000 7f99bcc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b806ef30 0x7f99b80ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:27.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.242+0000 7f99bcc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99b80cf790 0x7f99b814db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:27.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.242+0000 7f99bcc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b815c600 0x7f99b814f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.242+0000 7f99bcc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99b8070220 con 0x7f99b80cf790 2024-03-28T13:16:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.242+0000 7f99bcc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f99b806ff40 con 0x7f99b815c600 2024-03-28T13:16:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.242+0000 7f99bcc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f99b806f3a0 con 0x7f99b806ef30 2024-03-28T13:16:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.242+0000 7f99b5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99b80cf790 0x7f99b814db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.242+0000 7f99b5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99b80cf790 0x7f99b814db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39380/0 (socket says 172.21.15.19:39380) 2024-03-28T13:16:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.242+0000 7f99b5400640 1 -- 172.21.15.19:0/3375271182 learned_addr learned my addr 172.21.15.19:0/3375271182 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.242+0000 7f99b6800640 1 --2- 172.21.15.19:0/3375271182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b815c600 0x7f99b814f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.243+0000 7f99b5e00640 1 --2- 172.21.15.19:0/3375271182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b806ef30 0x7f99b80ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.243+0000 7f99b5400640 1 -- 172.21.15.19:0/3375271182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b806ef30 msgr2=0x7f99b80ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.243+0000 7f99b5400640 1 --2- 172.21.15.19:0/3375271182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b806ef30 0x7f99b80ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.243+0000 7f99b5400640 1 -- 172.21.15.19:0/3375271182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b815c600 msgr2=0x7f99b814f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.243+0000 7f99b5400640 1 --2- 172.21.15.19:0/3375271182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b815c600 0x7f99b814f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.243+0000 7f99b5400640 1 -- 172.21.15.19:0/3375271182 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b80af0d0 con 0x7f99b80cf790 2024-03-28T13:16:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.243+0000 7f99b5e00640 1 --2- 172.21.15.19:0/3375271182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b806ef30 0x7f99b80ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:27.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.243+0000 7f99b5400640 1 --2- 172.21.15.19:0/3375271182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99b80cf790 0x7f99b814db90 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f99a800cbf0 tx=0x7f99a800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:27.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.243+0000 7f99ae000640 1 -- 172.21.15.19:0/3375271182 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8017070 con 0x7f99b80cf790 2024-03-28T13:16:27.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.243+0000 7f99ae000640 1 -- 172.21.15.19:0/3375271182 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99a800ad50 con 0x7f99b80cf790 2024-03-28T13:16:27.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.243+0000 7f99bcc00640 1 -- 172.21.15.19:0/3375271182 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b80af3b0 con 0x7f99b80cf790 2024-03-28T13:16:27.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.243+0000 7f99ae000640 1 -- 172.21.15.19:0/3375271182 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8012620 con 0x7f99b80cf790 2024-03-28T13:16:27.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.243+0000 7f99bcc00640 1 -- 172.21.15.19:0/3375271182 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b815d320 con 0x7f99b80cf790 2024-03-28T13:16:27.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.245+0000 7f99bcc00640 1 -- 172.21.15.19:0/3375271182 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b80685c0 con 0x7f99b80cf790 2024-03-28T13:16:27.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.245+0000 7f99ae000640 1 -- 172.21.15.19:0/3375271182 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f99a801a070 con 0x7f99b80cf790 2024-03-28T13:16:27.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.247+0000 7f99ae000640 1 --2- 172.21.15.19:0/3375271182 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f999406c530 0x7f999406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:27.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.247+0000 7f99ae000640 1 -- 172.21.15.19:0/3375271182 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5709+0+0 (secure 0 0 0) 0x7f99a808b5a0 con 0x7f99b80cf790 2024-03-28T13:16:27.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.247+0000 7f99b5e00640 1 --2- 172.21.15.19:0/3375271182 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f999406c530 0x7f999406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:27.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.248+0000 7f99b5e00640 1 --2- 172.21.15.19:0/3375271182 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f999406c530 0x7f999406e9f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f99a0036000 tx=0x7f99a003b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:27.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.252+0000 7f99ae000640 1 -- 172.21.15.19:0/3375271182 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f99a8059c30 con 0x7f99b80cf790 2024-03-28T13:16:27.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.458+0000 7f99bcc00640 1 -- 172.21.15.19:0/3375271182 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99b80b0340 con 0x7f99b80cf790 2024-03-28T13:16:27.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.461+0000 7f99ae000640 1 -- 172.21.15.19:0/3375271182 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16685 (secure 0 0 0) 0x7f99a8059530 con 0x7f99b80cf790 2024-03-28T13:16:27.462 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:27.462 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":91,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:16:25.994580+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:16:23.901120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-28T13:16:24.990904+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:16:27.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.464+0000 7f99bcc00640 1 -- 172.21.15.19:0/3375271182 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f999406c530 msgr2=0x7f999406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:27.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.464+0000 7f99bcc00640 1 --2- 172.21.15.19:0/3375271182 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f999406c530 0x7f999406e9f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f99a0036000 tx=0x7f99a003b040 comp rx=0 tx=0).stop 2024-03-28T13:16:27.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.464+0000 7f99bcc00640 1 -- 172.21.15.19:0/3375271182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99b80cf790 msgr2=0x7f99b814db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:27.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.464+0000 7f99bcc00640 1 --2- 172.21.15.19:0/3375271182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99b80cf790 0x7f99b814db90 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f99a800cbf0 tx=0x7f99a800a430 comp rx=0 tx=0).stop 2024-03-28T13:16:27.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.464+0000 7f99bcc00640 1 -- 172.21.15.19:0/3375271182 shutdown_connections 2024-03-28T13:16:27.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.464+0000 7f99bcc00640 1 --2- 172.21.15.19:0/3375271182 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f999406c530 0x7f999406e9f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.464+0000 7f99bcc00640 1 --2- 172.21.15.19:0/3375271182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99b815c600 0x7f99b814f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.464+0000 7f99bcc00640 1 --2- 172.21.15.19:0/3375271182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99b80cf790 0x7f99b814db90 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.464+0000 7f99bcc00640 1 --2- 172.21.15.19:0/3375271182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99b806ef30 0x7f99b80ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.464+0000 7f99bcc00640 1 -- 172.21.15.19:0/3375271182 >> 172.21.15.19:0/3375271182 conn(0x7f99b80b2cb0 msgr2=0x7f99b80b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:27.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.464+0000 7f99bcc00640 1 -- 172.21.15.19:0/3375271182 shutdown_connections 2024-03-28T13:16:27.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.465+0000 7f99bcc00640 1 -- 172.21.15.19:0/3375271182 wait complete. 2024-03-28T13:16:27.484 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-28T13:16:27.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.714+0000 7f0b19e00640 1 Processor -- start 2024-03-28T13:16:27.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.714+0000 7f0b19e00640 1 -- start start 2024-03-28T13:16:27.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.715+0000 7f0b19e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b140d0990 0x7f0b140aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:27.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.716+0000 7f0b19e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0b140af2e0 0x7f0b141600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:27.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.716+0000 7f0b19e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b140afab0 con 0x7f0b14153580 2024-03-28T13:16:27.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.716+0000 7f0b19e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f0b1415cc80 con 0x7f0b140af2e0 2024-03-28T13:16:27.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.716+0000 7f0b19e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1415da00 con 0x7f0b140d0990 2024-03-28T13:16:27.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.716+0000 7f0b12a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0b14153580 0x7f0b140d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57274/0 (socket says 172.21.15.19:57274) 2024-03-28T13:16:27.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.716+0000 7f0b12a00640 1 -- 172.21.15.19:0/2371676615 learned_addr learned my addr 172.21.15.19:0/2371676615 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:27.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.716+0000 7f0b12000640 1 --2- 172.21.15.19:0/2371676615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b140d0990 0x7f0b140aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:27.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.716+0000 7f0b13400640 1 --2- 172.21.15.19:0/2371676615 >> v2:172.21.15.188:3300/0 conn(0x7f0b140af2e0 0x7f0b141600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:27.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.716+0000 7f0b11600640 1 -- 172.21.15.19:0/2371676615 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 272173594 0 0) 0x7f0b140afab0 con 0x7f0b14153580 2024-03-28T13:16:27.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b11600640 1 -- 172.21.15.19:0/2371676615 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0af8003520 con 0x7f0b14153580 2024-03-28T13:16:27.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b11600640 1 -- 172.21.15.19:0/2371676615 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 993953235 0 0) 0x7f0af8003520 con 0x7f0b14153580 2024-03-28T13:16:27.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b11600640 1 -- 172.21.15.19:0/2371676615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b140d0990 msgr2=0x7f0b140aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b11600640 1 --2- 172.21.15.19:0/2371676615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b140d0990 0x7f0b140aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b11600640 1 -- 172.21.15.19:0/2371676615 >> v2:172.21.15.188:3300/0 conn(0x7f0b140af2e0 msgr2=0x7f0b141600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b11600640 1 --2- 172.21.15.19:0/2371676615 >> v2:172.21.15.188:3300/0 conn(0x7f0b140af2e0 0x7f0b141600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b11600640 1 -- 172.21.15.19:0/2371676615 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1415cf20 con 0x7f0b14153580 2024-03-28T13:16:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b11600640 1 -- 172.21.15.19:0/2371676615 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f0b04001670 con 0x7f0b14153580 2024-03-28T13:16:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b11600640 1 -- 172.21.15.19:0/2371676615 >> v1:172.21.15.19:6789/0 conn(0x7f0b14153580 legacy=0x7f0b140d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b11600640 1 --2- 172.21.15.19:0/2371676615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af80039b0 0x7f0af8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b11600640 1 --2- 172.21.15.19:0/2371676615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8004800 0x7f0af8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b11600640 1 --2- 172.21.15.19:0/2371676615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b140d0990 0x7f0af80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b11600640 1 -- 172.21.15.19:0/2371676615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1415cc80 con 0x7f0af8004800 2024-03-28T13:16:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b11600640 1 -- 172.21.15.19:0/2371676615 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1415da00 con 0x7f0af80039b0 2024-03-28T13:16:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b11600640 1 -- 172.21.15.19:0/2371676615 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0af800ba10 con 0x7f0b140d0990 2024-03-28T13:16:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.717+0000 7f0b12000640 1 --2- 172.21.15.19:0/2371676615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8004800 0x7f0af8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.718+0000 7f0b12a00640 1 --2- 172.21.15.19:0/2371676615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af80039b0 0x7f0af8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.718+0000 7f0b13400640 1 --2- 172.21.15.19:0/2371676615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b140d0990 0x7f0af80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:27.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.718+0000 7f0b12000640 1 -- 172.21.15.19:0/2371676615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b140d0990 msgr2=0x7f0af80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:27.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.718+0000 7f0b12000640 1 --2- 172.21.15.19:0/2371676615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b140d0990 0x7f0af80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.718+0000 7f0b12000640 1 -- 172.21.15.19:0/2371676615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af80039b0 msgr2=0x7f0af8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:27.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.718+0000 7f0b12000640 1 --2- 172.21.15.19:0/2371676615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af80039b0 0x7f0af8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.718+0000 7f0b12000640 1 -- 172.21.15.19:0/2371676615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af8003520 con 0x7f0af8004800 2024-03-28T13:16:27.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.718+0000 7f0b12a00640 1 --2- 172.21.15.19:0/2371676615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af80039b0 0x7f0af8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:27.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.718+0000 7f0b13400640 1 --2- 172.21.15.19:0/2371676615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b140d0990 0x7f0af80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:27.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.719+0000 7f0b12000640 1 --2- 172.21.15.19:0/2371676615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8004800 0x7f0af8004c80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009af0 tx=0x7f0af8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e41d40cceace1a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:27.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.719+0000 7f0b11600640 1 -- 172.21.15.19:0/2371676615 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00039070 con 0x7f0af8004800 2024-03-28T13:16:27.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.719+0000 7f0b11600640 1 -- 172.21.15.19:0/2371676615 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0002c6b0 con 0x7f0af8004800 2024-03-28T13:16:27.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.719+0000 7f0b11600640 1 -- 172.21.15.19:0/2371676615 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00009690 con 0x7f0af8004800 2024-03-28T13:16:27.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.720+0000 7f0b19e00640 1 -- 172.21.15.19:0/2371676615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8004800 msgr2=0x7f0af8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:27.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.720+0000 7f0b19e00640 1 --2- 172.21.15.19:0/2371676615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8004800 0x7f0af8004c80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009af0 tx=0x7f0af8005830 comp rx=0 tx=0).stop 2024-03-28T13:16:27.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.720+0000 7f0b13400640 1 -- 172.21.15.19:0/2371676615 reap_dead start 2024-03-28T13:16:27.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.720+0000 7f0b19e00640 1 -- 172.21.15.19:0/2371676615 shutdown_connections 2024-03-28T13:16:27.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.720+0000 7f0b19e00640 1 -- 172.21.15.19:0/2371676615 >> 172.21.15.19:0/2371676615 conn(0x7f0b1406b750 msgr2=0x7f0b1406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:27.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.720+0000 7f0b19e00640 1 -- 172.21.15.19:0/2371676615 shutdown_connections 2024-03-28T13:16:27.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.720+0000 7f0b19e00640 1 -- 172.21.15.19:0/2371676615 wait complete. 2024-03-28T13:16:27.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.722+0000 7f0b19e00640 1 Processor -- start 2024-03-28T13:16:27.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.722+0000 7f0b19e00640 1 -- start start 2024-03-28T13:16:27.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.723+0000 7f0b19e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af8004800 0x7f0b141f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.723+0000 7f0b19e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b14158900 0x7f0b14158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.723+0000 7f0b19e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b140cd0d0 0x7f0b140cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.723+0000 7f0b19e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b14153960 con 0x7f0b14158900 2024-03-28T13:16:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.723+0000 7f0b19e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1415d4e0 con 0x7f0af8004800 2024-03-28T13:16:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.723+0000 7f0b19e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0b140af710 con 0x7f0b140cd0d0 2024-03-28T13:16:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.723+0000 7f0b12000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b14158900 0x7f0b14158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.723+0000 7f0b12000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b14158900 0x7f0b14158d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39398/0 (socket says 172.21.15.19:39398) 2024-03-28T13:16:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.723+0000 7f0b12000640 1 -- 172.21.15.19:0/3286215892 learned_addr learned my addr 172.21.15.19:0/3286215892 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.723+0000 7f0b12a00640 1 --2- 172.21.15.19:0/3286215892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af8004800 0x7f0b141f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.724+0000 7f0b13400640 1 --2- 172.21.15.19:0/3286215892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b140cd0d0 0x7f0b140cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.724+0000 7f0b12000640 1 -- 172.21.15.19:0/3286215892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b140cd0d0 msgr2=0x7f0b140cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.724+0000 7f0b12000640 1 --2- 172.21.15.19:0/3286215892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b140cd0d0 0x7f0b140cd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.724+0000 7f0b12000640 1 -- 172.21.15.19:0/3286215892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af8004800 msgr2=0x7f0b141f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.724+0000 7f0b12000640 1 --2- 172.21.15.19:0/3286215892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af8004800 0x7f0b141f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:27.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.724+0000 7f0b12000640 1 -- 172.21.15.19:0/3286215892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b141593c0 con 0x7f0b14158900 2024-03-28T13:16:27.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.724+0000 7f0b12000640 1 --2- 172.21.15.19:0/3286215892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b14158900 0x7f0b14158d80 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f0b00037040 tx=0x7f0b00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:27.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.724+0000 7f0b0b400640 1 -- 172.21.15.19:0/3286215892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00042020 con 0x7f0b14158900 2024-03-28T13:16:27.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.724+0000 7f0b0b400640 1 -- 172.21.15.19:0/3286215892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0003a040 con 0x7f0b14158900 2024-03-28T13:16:27.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.724+0000 7f0b0b400640 1 -- 172.21.15.19:0/3286215892 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00039070 con 0x7f0b14158900 2024-03-28T13:16:27.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.724+0000 7f0b19e00640 1 -- 172.21.15.19:0/3286215892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b141596a0 con 0x7f0b14158900 2024-03-28T13:16:27.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.724+0000 7f0b19e00640 1 -- 172.21.15.19:0/3286215892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b14071cd0 con 0x7f0b14158900 2024-03-28T13:16:27.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.726+0000 7f0b19e00640 1 -- 172.21.15.19:0/3286215892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b14159870 con 0x7f0b14158900 2024-03-28T13:16:27.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.727+0000 7f0b0b400640 1 -- 172.21.15.19:0/3286215892 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0b00045020 con 0x7f0b14158900 2024-03-28T13:16:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.728+0000 7f0b0b400640 1 --2- 172.21.15.19:0/3286215892 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0af406c530 0x7f0af406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.728+0000 7f0b0b400640 1 -- 172.21.15.19:0/3286215892 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f0b000acbb0 con 0x7f0b14158900 2024-03-28T13:16:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.728+0000 7f0b12a00640 1 --2- 172.21.15.19:0/3286215892 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0af406c530 0x7f0af406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.729+0000 7f0b12a00640 1 --2- 172.21.15.19:0/3286215892 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0af406c530 0x7f0af406e9f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0b040018a0 tx=0x7f0b04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:27.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.732+0000 7f0b0b400640 1 -- 172.21.15.19:0/3286215892 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0b0004c070 con 0x7f0b14158900 2024-03-28T13:16:27.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:27.979+0000 7f0b19e00640 1 -- 172.21.15.19:0/3286215892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0b140685c0 con 0x7f0b14158900 2024-03-28T13:16:28.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.562+0000 7f0b0b400640 1 -- 172.21.15.19:0/3286215892 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) ==== 118+0+0 (secure 0 0 0) 0x7f0b0007b130 con 0x7f0b14158900 2024-03-28T13:16:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.565+0000 7f0b19e00640 1 -- 172.21.15.19:0/3286215892 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0af406c530 msgr2=0x7f0af406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.565+0000 7f0b19e00640 1 --2- 172.21.15.19:0/3286215892 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0af406c530 0x7f0af406e9f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0b040018a0 tx=0x7f0b04009040 comp rx=0 tx=0).stop 2024-03-28T13:16:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.565+0000 7f0b19e00640 1 -- 172.21.15.19:0/3286215892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b14158900 msgr2=0x7f0b14158d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.565+0000 7f0b19e00640 1 --2- 172.21.15.19:0/3286215892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b14158900 0x7f0b14158d80 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f0b00037040 tx=0x7f0b00004730 comp rx=0 tx=0).stop 2024-03-28T13:16:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.566+0000 7f0b19e00640 1 -- 172.21.15.19:0/3286215892 shutdown_connections 2024-03-28T13:16:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.566+0000 7f0b19e00640 1 --2- 172.21.15.19:0/3286215892 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0af406c530 0x7f0af406e9f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.566+0000 7f0b19e00640 1 --2- 172.21.15.19:0/3286215892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b140cd0d0 0x7f0b140cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.566+0000 7f0b19e00640 1 --2- 172.21.15.19:0/3286215892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b14158900 0x7f0b14158d80 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.566+0000 7f0b19e00640 1 --2- 172.21.15.19:0/3286215892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af8004800 0x7f0b141f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:28.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.566+0000 7f0b19e00640 1 -- 172.21.15.19:0/3286215892 >> 172.21.15.19:0/3286215892 conn(0x7f0b1406b750 msgr2=0x7f0b140b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:28.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.566+0000 7f0b19e00640 1 -- 172.21.15.19:0/3286215892 shutdown_connections 2024-03-28T13:16:28.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.566+0000 7f0b19e00640 1 -- 172.21.15.19:0/3286215892 wait complete. 2024-03-28T13:16:28.588 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-28T13:16:28.588 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-28T13:16:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.820+0000 7f6a03c00640 1 Processor -- start 2024-03-28T13:16:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.820+0000 7f6a03c00640 1 -- start start 2024-03-28T13:16:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.822+0000 7f6a03c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69fc153580 0x7f69fc0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.822+0000 7f6a03c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69fc0d07c0 0x7f69fc0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.822+0000 7f6a03c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f69fc0af2e0 0x7f69fc06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.822+0000 7f6a03c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f69fc0d0c00 con 0x7f69fc0af2e0 2024-03-28T13:16:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.822+0000 7f6a03c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f69fc0afab0 con 0x7f69fc153580 2024-03-28T13:16:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.822+0000 7f6a03c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f69fc15cc80 con 0x7f69fc0d07c0 2024-03-28T13:16:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.822+0000 7f6a01c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f69fc0af2e0 0x7f69fc06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.822+0000 7f6a01c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f69fc0af2e0 0x7f69fc06d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39400/0 (socket says 172.21.15.19:39400) 2024-03-28T13:16:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.822+0000 7f6a01c00640 1 -- 172.21.15.19:0/759878490 learned_addr learned my addr 172.21.15.19:0/759878490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.823+0000 7f69fbe00640 1 --2- 172.21.15.19:0/759878490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69fc0d07c0 0x7f69fc0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.823+0000 7f6a01200640 1 --2- 172.21.15.19:0/759878490 >> v2:172.21.15.188:3300/0 conn(0x7f69fc153580 0x7f69fc0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.823+0000 7f6a01c00640 1 -- 172.21.15.19:0/759878490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69fc0d07c0 msgr2=0x7f69fc0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.823+0000 7f6a01c00640 1 --2- 172.21.15.19:0/759878490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69fc0d07c0 0x7f69fc0aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.823+0000 7f6a01c00640 1 -- 172.21.15.19:0/759878490 >> v2:172.21.15.188:3300/0 conn(0x7f69fc153580 msgr2=0x7f69fc0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.823+0000 7f6a01c00640 1 --2- 172.21.15.19:0/759878490 >> v2:172.21.15.188:3300/0 conn(0x7f69fc153580 0x7f69fc0d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.823+0000 7f6a01c00640 1 -- 172.21.15.19:0/759878490 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc15da00 con 0x7f69fc0af2e0 2024-03-28T13:16:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.824+0000 7f6a01c00640 1 --2- 172.21.15.19:0/759878490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69fc0af2e0 0x7f69fc06d5b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0b5270 tx=0x7f69f402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd71f23351e8b06 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.824+0000 7f69fb400640 1 -- 172.21.15.19:0/759878490 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f402c9e0 con 0x7f69fc0af2e0 2024-03-28T13:16:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.824+0000 7f69fb400640 1 -- 172.21.15.19:0/759878490 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69f402cb80 con 0x7f69fc0af2e0 2024-03-28T13:16:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.824+0000 7f69fb400640 1 -- 172.21.15.19:0/759878490 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f40343f0 con 0x7f69fc0af2e0 2024-03-28T13:16:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.825+0000 7f6a03c00640 1 -- 172.21.15.19:0/759878490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69fc0af2e0 msgr2=0x7f69fc06d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.825+0000 7f6a03c00640 1 --2- 172.21.15.19:0/759878490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69fc0af2e0 0x7f69fc06d5b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0b5270 tx=0x7f69f402be40 comp rx=0 tx=0).stop 2024-03-28T13:16:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.825+0000 7f6a03c00640 1 -- 172.21.15.19:0/759878490 shutdown_connections 2024-03-28T13:16:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.825+0000 7f6a03c00640 1 --2- 172.21.15.19:0/759878490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69fc0af2e0 0x7f69fc06d5b0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.825+0000 7f6a03c00640 1 --2- 172.21.15.19:0/759878490 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69fc0d07c0 0x7f69fc0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.825+0000 7f6a03c00640 1 --2- 172.21.15.19:0/759878490 >> v2:172.21.15.188:3300/0 conn(0x7f69fc153580 0x7f69fc0d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.825+0000 7f6a03c00640 1 -- 172.21.15.19:0/759878490 >> 172.21.15.19:0/759878490 conn(0x7f69fc06b750 msgr2=0x7f69fc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.825+0000 7f6a03c00640 1 -- 172.21.15.19:0/759878490 shutdown_connections 2024-03-28T13:16:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.825+0000 7f6a03c00640 1 -- 172.21.15.19:0/759878490 wait complete. 2024-03-28T13:16:28.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.828+0000 7f6a03c00640 1 Processor -- start 2024-03-28T13:16:28.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.828+0000 7f6a03c00640 1 -- start start 2024-03-28T13:16:28.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.829+0000 7f6a03c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69fc0af2e0 0x7f69fc151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:28.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.829+0000 7f6a03c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69fc0d07c0 0x7f69fc0b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:28.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.829+0000 7f6a03c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69fc153580 0x7f69fc151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:28.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.829+0000 7f6a03c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc15d4e0 con 0x7f69fc153580 2024-03-28T13:16:28.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.829+0000 7f6a03c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc15d200 con 0x7f69fc0d07c0 2024-03-28T13:16:28.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.829+0000 7f6a03c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f69fc153960 con 0x7f69fc0af2e0 2024-03-28T13:16:28.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.829+0000 7f6a01c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69fc153580 0x7f69fc151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:28.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.829+0000 7f6a01c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69fc153580 0x7f69fc151ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39404/0 (socket says 172.21.15.19:39404) 2024-03-28T13:16:28.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.829+0000 7f6a01c00640 1 -- 172.21.15.19:0/3146229835 learned_addr learned my addr 172.21.15.19:0/3146229835 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:28.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.830+0000 7f69fbe00640 1 --2- 172.21.15.19:0/3146229835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69fc0d07c0 0x7f69fc0b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:28.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.830+0000 7f6a01200640 1 --2- 172.21.15.19:0/3146229835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69fc0af2e0 0x7f69fc151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:28.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.830+0000 7f6a01c00640 1 -- 172.21.15.19:0/3146229835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69fc0af2e0 msgr2=0x7f69fc151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:28.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.830+0000 7f6a01c00640 1 --2- 172.21.15.19:0/3146229835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69fc0af2e0 0x7f69fc151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:28.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.830+0000 7f6a01c00640 1 -- 172.21.15.19:0/3146229835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69fc0d07c0 msgr2=0x7f69fc0b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:28.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.830+0000 7f6a01c00640 1 --2- 172.21.15.19:0/3146229835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69fc0d07c0 0x7f69fc0b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:28.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.830+0000 7f6a01c00640 1 -- 172.21.15.19:0/3146229835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc0b2c20 con 0x7f69fc153580 2024-03-28T13:16:28.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.830+0000 7f6a01200640 1 --2- 172.21.15.19:0/3146229835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69fc0af2e0 0x7f69fc151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:28.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.830+0000 7f6a01c00640 1 --2- 172.21.15.19:0/3146229835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69fc153580 0x7f69fc151ff0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f69f4036040 tx=0x7f69f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:28.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.830+0000 7f69f8c00640 1 -- 172.21.15.19:0/3146229835 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4043020 con 0x7f69fc153580 2024-03-28T13:16:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.831+0000 7f69f8c00640 1 -- 172.21.15.19:0/3146229835 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69f403e450 con 0x7f69fc153580 2024-03-28T13:16:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.831+0000 7f69f8c00640 1 -- 172.21.15.19:0/3146229835 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4009510 con 0x7f69fc153580 2024-03-28T13:16:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.831+0000 7f6a03c00640 1 -- 172.21.15.19:0/3146229835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69fc0b2f00 con 0x7f69fc153580 2024-03-28T13:16:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.831+0000 7f6a03c00640 1 -- 172.21.15.19:0/3146229835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69fc0b4940 con 0x7f69fc153580 2024-03-28T13:16:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.832+0000 7f6a03c00640 1 -- 172.21.15.19:0/3146229835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69fc0ae3c0 con 0x7f69fc153580 2024-03-28T13:16:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.833+0000 7f69f8c00640 1 -- 172.21.15.19:0/3146229835 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f69f4046020 con 0x7f69fc153580 2024-03-28T13:16:28.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.834+0000 7f69f8c00640 1 --2- 172.21.15.19:0/3146229835 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f69dc06c530 0x7f69dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:28.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.834+0000 7f69f8c00640 1 -- 172.21.15.19:0/3146229835 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f69f40adf50 con 0x7f69fc153580 2024-03-28T13:16:28.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.835+0000 7f6a01200640 1 --2- 172.21.15.19:0/3146229835 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f69dc06c530 0x7f69dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:28.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.836+0000 7f6a01200640 1 --2- 172.21.15.19:0/3146229835 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f69dc06c530 0x7f69dc06e9f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0ae850 tx=0x7f69f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:28.839+0000 7f69f8c00640 1 -- 172.21.15.19:0/3146229835 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f69fc0ae3c0 con 0x7f69fc153580 2024-03-28T13:16:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.081+0000 7f6a03c00640 1 -- 172.21.15.19:0/3146229835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f69fc0685c0 con 0x7f69fc153580 2024-03-28T13:16:29.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.565+0000 7f69f8c00640 1 -- 172.21.15.19:0/3146229835 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v61) ==== 122+0+0 (secure 0 0 0) 0x7f69f404d030 con 0x7f69fc153580 2024-03-28T13:16:29.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.569+0000 7f6a03c00640 1 -- 172.21.15.19:0/3146229835 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f69dc06c530 msgr2=0x7f69dc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:29.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.569+0000 7f6a03c00640 1 --2- 172.21.15.19:0/3146229835 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f69dc06c530 0x7f69dc06e9f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0ae850 tx=0x7f69f0008040 comp rx=0 tx=0).stop 2024-03-28T13:16:29.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.569+0000 7f6a03c00640 1 -- 172.21.15.19:0/3146229835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69fc153580 msgr2=0x7f69fc151ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:29.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.569+0000 7f6a03c00640 1 --2- 172.21.15.19:0/3146229835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69fc153580 0x7f69fc151ff0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f69f4036040 tx=0x7f69f4004a40 comp rx=0 tx=0).stop 2024-03-28T13:16:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.569+0000 7f6a03c00640 1 -- 172.21.15.19:0/3146229835 shutdown_connections 2024-03-28T13:16:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.569+0000 7f6a03c00640 1 --2- 172.21.15.19:0/3146229835 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f69dc06c530 0x7f69dc06e9f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.569+0000 7f6a03c00640 1 --2- 172.21.15.19:0/3146229835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69fc153580 0x7f69fc151ff0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.569+0000 7f6a03c00640 1 --2- 172.21.15.19:0/3146229835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69fc0d07c0 0x7f69fc0b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.569+0000 7f6a03c00640 1 --2- 172.21.15.19:0/3146229835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69fc0af2e0 0x7f69fc151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.569+0000 7f6a03c00640 1 -- 172.21.15.19:0/3146229835 >> 172.21.15.19:0/3146229835 conn(0x7f69fc06b750 msgr2=0x7f69fc15f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.570+0000 7f6a03c00640 1 -- 172.21.15.19:0/3146229835 shutdown_connections 2024-03-28T13:16:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.570+0000 7f6a03c00640 1 -- 172.21.15.19:0/3146229835 wait complete. 2024-03-28T13:16:29.594 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-28T13:16:29.594 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:16:29.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.817+0000 7fc843000640 1 Processor -- start 2024-03-28T13:16:29.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.818+0000 7fc843000640 1 -- start start 2024-03-28T13:16:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.819+0000 7fc843000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc83c0cd270 0x7fc83c06b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.819+0000 7fc843000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc83c0d0130 con 0x7fc83c06bab0 2024-03-28T13:16:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.819+0000 7fc843000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc83c0cd830 con 0x7fc83c15e750 2024-03-28T13:16:29.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.819+0000 7fc843000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc83c06bf20 con 0x7fc83c0cd270 2024-03-28T13:16:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.820+0000 7fc841000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc83c06bab0 0x7fc83c0726b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57282/0 (socket says 172.21.15.19:57282) 2024-03-28T13:16:29.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.820+0000 7fc841000640 1 -- 172.21.15.19:0/632328854 learned_addr learned my addr 172.21.15.19:0/632328854 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.820+0000 7fc83b400640 1 --2- 172.21.15.19:0/632328854 >> v2:172.21.15.188:3300/0 conn(0x7fc83c0cd270 0x7fc83c06b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.820+0000 7fc83aa00640 1 -- 172.21.15.19:0/632328854 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4058567076 0 0) 0x7fc83c0d0130 con 0x7fc83c06bab0 2024-03-28T13:16:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.820+0000 7fc83aa00640 1 -- 172.21.15.19:0/632328854 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc820003520 con 0x7fc83c06bab0 2024-03-28T13:16:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.821+0000 7fc83aa00640 1 -- 172.21.15.19:0/632328854 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3384958083 0 0) 0x7fc820003520 con 0x7fc83c06bab0 2024-03-28T13:16:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.821+0000 7fc83aa00640 1 -- 172.21.15.19:0/632328854 >> v2:172.21.15.188:3300/0 conn(0x7fc83c0cd270 msgr2=0x7fc83c06b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.821+0000 7fc83aa00640 1 --2- 172.21.15.19:0/632328854 >> v2:172.21.15.188:3300/0 conn(0x7fc83c0cd270 0x7fc83c06b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.821+0000 7fc83aa00640 1 -- 172.21.15.19:0/632328854 >> v1:172.21.15.188:6789/0 conn(0x7fc83c15e750 legacy=0x7fc83c0cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.821+0000 7fc83aa00640 1 -- 172.21.15.19:0/632328854 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83c0cdb10 con 0x7fc83c06bab0 2024-03-28T13:16:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.821+0000 7fc83aa00640 1 -- 172.21.15.19:0/632328854 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fc8340019f0 con 0x7fc83c06bab0 2024-03-28T13:16:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.821+0000 7fc83aa00640 1 -- 172.21.15.19:0/632328854 >> v1:172.21.15.19:6789/0 conn(0x7fc83c06bab0 legacy=0x7fc83c0726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.821+0000 7fc83aa00640 1 --2- 172.21.15.19:0/632328854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc820003a00 0x7fc820003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.821+0000 7fc83aa00640 1 --2- 172.21.15.19:0/632328854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc820004800 0x7fc820004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.821+0000 7fc83aa00640 1 --2- 172.21.15.19:0/632328854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8200051c0 0x7fc82000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:29.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.821+0000 7fc83aa00640 1 -- 172.21.15.19:0/632328854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c06bf20 con 0x7fc820003a00 2024-03-28T13:16:29.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.821+0000 7fc83aa00640 1 -- 172.21.15.19:0/632328854 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc82000c0c0 con 0x7fc820004800 2024-03-28T13:16:29.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.821+0000 7fc83aa00640 1 -- 172.21.15.19:0/632328854 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc82000c2c0 con 0x7fc8200051c0 2024-03-28T13:16:29.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.821+0000 7fc83be00640 1 --2- 172.21.15.19:0/632328854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc820003a00 0x7fc820003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:29.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.821+0000 7fc83b400640 1 --2- 172.21.15.19:0/632328854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc820004800 0x7fc820004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:29.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.822+0000 7fc841000640 1 --2- 172.21.15.19:0/632328854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8200051c0 0x7fc82000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:29.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.822+0000 7fc83be00640 1 -- 172.21.15.19:0/632328854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8200051c0 msgr2=0x7fc82000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:29.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.822+0000 7fc83be00640 1 --2- 172.21.15.19:0/632328854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8200051c0 0x7fc82000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:29.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.822+0000 7fc83be00640 1 -- 172.21.15.19:0/632328854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc820004800 msgr2=0x7fc820004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:29.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.822+0000 7fc83be00640 1 --2- 172.21.15.19:0/632328854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc820004800 0x7fc820004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:29.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.822+0000 7fc83be00640 1 -- 172.21.15.19:0/632328854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc820003520 con 0x7fc820003a00 2024-03-28T13:16:29.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.822+0000 7fc841000640 1 -- 172.21.15.19:0/632328854 reap_dead start 2024-03-28T13:16:29.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.823+0000 7fc83be00640 1 --2- 172.21.15.19:0/632328854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc820003a00 0x7fc820003e20 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc82c001750 tx=0x7fc820005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1bbc7e657a9f32 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:29.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.823+0000 7fc83aa00640 1 -- 172.21.15.19:0/632328854 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c001370 con 0x7fc820003a00 2024-03-28T13:16:29.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.823+0000 7fc83aa00640 1 -- 172.21.15.19:0/632328854 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82c0052d0 con 0x7fc820003a00 2024-03-28T13:16:29.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.823+0000 7fc83aa00640 1 -- 172.21.15.19:0/632328854 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c003830 con 0x7fc820003a00 2024-03-28T13:16:29.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.824+0000 7fc843000640 1 -- 172.21.15.19:0/632328854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc820003a00 msgr2=0x7fc820003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:29.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.824+0000 7fc843000640 1 --2- 172.21.15.19:0/632328854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc820003a00 0x7fc820003e20 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fc82c001750 tx=0x7fc820005830 comp rx=0 tx=0).stop 2024-03-28T13:16:29.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.824+0000 7fc843000640 1 -- 172.21.15.19:0/632328854 shutdown_connections 2024-03-28T13:16:29.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.824+0000 7fc843000640 1 --2- 172.21.15.19:0/632328854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc820003a00 0x7fc820003e20 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:29.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.824+0000 7fc843000640 1 -- 172.21.15.19:0/632328854 >> 172.21.15.19:0/632328854 conn(0x7fc83c0b2cb0 msgr2=0x7fc83c0b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:29.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.824+0000 7fc843000640 1 -- 172.21.15.19:0/632328854 shutdown_connections 2024-03-28T13:16:29.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.824+0000 7fc843000640 1 -- 172.21.15.19:0/632328854 wait complete. 2024-03-28T13:16:29.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.826+0000 7fc843000640 1 Processor -- start 2024-03-28T13:16:29.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.826+0000 7fc843000640 1 -- start start 2024-03-28T13:16:29.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.827+0000 7fc843000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc820003a00 0x7fc83c159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:29.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.827+0000 7fc843000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c1f51d0 0x7fc83c15a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:29.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.827+0000 7fc843000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc83c15acd0 0x7fc83c0b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:29.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.827+0000 7fc843000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c0ce3b0 con 0x7fc83c1f51d0 2024-03-28T13:16:29.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.827+0000 7fc843000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c0ce0d0 con 0x7fc820003a00 2024-03-28T13:16:29.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.827+0000 7fc843000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc83c15eb30 con 0x7fc83c15acd0 2024-03-28T13:16:29.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.827+0000 7fc83b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c1f51d0 0x7fc83c15a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:29.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.827+0000 7fc83b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c1f51d0 0x7fc83c15a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39424/0 (socket says 172.21.15.19:39424) 2024-03-28T13:16:29.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.827+0000 7fc83b400640 1 -- 172.21.15.19:0/1367983460 learned_addr learned my addr 172.21.15.19:0/1367983460 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.827+0000 7fc83be00640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc820003a00 0x7fc83c159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.828+0000 7fc83b400640 1 -- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc83c15acd0 msgr2=0x7fc83c0b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.828+0000 7fc841000640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc83c15acd0 0x7fc83c0b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.828+0000 7fc83b400640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc83c15acd0 0x7fc83c0b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.828+0000 7fc83b400640 1 -- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc820003a00 msgr2=0x7fc83c159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.828+0000 7fc83b400640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc820003a00 0x7fc83c159080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.828+0000 7fc83b400640 1 -- 172.21.15.19:0/1367983460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83c15eeb0 con 0x7fc83c1f51d0 2024-03-28T13:16:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.828+0000 7fc83be00640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc820003a00 0x7fc83c159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.828+0000 7fc841000640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc83c15acd0 0x7fc83c0b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.828+0000 7fc83b400640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c1f51d0 0x7fc83c15a790 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc8240115e0 tx=0x7fc824011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.828+0000 7fc833e00640 1 -- 172.21.15.19:0/1367983460 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824004220 con 0x7fc83c1f51d0 2024-03-28T13:16:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.828+0000 7fc833e00640 1 -- 172.21.15.19:0/1367983460 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8240043c0 con 0x7fc83c1f51d0 2024-03-28T13:16:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.828+0000 7fc833e00640 1 -- 172.21.15.19:0/1367983460 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82400f930 con 0x7fc83c1f51d0 2024-03-28T13:16:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.828+0000 7fc843000640 1 -- 172.21.15.19:0/1367983460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83c161210 con 0x7fc83c1f51d0 2024-03-28T13:16:29.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.828+0000 7fc843000640 1 -- 172.21.15.19:0/1367983460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc83c15fa90 con 0x7fc83c1f51d0 2024-03-28T13:16:29.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.829+0000 7fc843000640 1 -- 172.21.15.19:0/1367983460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc83c0d0a60 con 0x7fc83c1f51d0 2024-03-28T13:16:29.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.831+0000 7fc833e00640 1 -- 172.21.15.19:0/1367983460 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc82400fad0 con 0x7fc83c1f51d0 2024-03-28T13:16:29.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.831+0000 7fc833e00640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc81806c530 0x7fc81806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:29.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.831+0000 7fc833e00640 1 -- 172.21.15.19:0/1367983460 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fc82408b9b0 con 0x7fc83c1f51d0 2024-03-28T13:16:29.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.832+0000 7fc83be00640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc81806c530 0x7fc81806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:29.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.833+0000 7fc83be00640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc81806c530 0x7fc81806e9f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc82c033000 tx=0x7fc82c009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:29.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:29.836+0000 7fc833e00640 1 -- 172.21.15.19:0/1367983460 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc83c0d0a60 con 0x7fc83c1f51d0 2024-03-28T13:16:30.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.073+0000 7fc843000640 1 -- 172.21.15.19:0/1367983460 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc83c0685c0 con 0x7fc83c1f51d0 2024-03-28T13:16:30.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.075+0000 7fc833e00640 1 -- 172.21.15.19:0/1367983460 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5186 (secure 0 0 0) 0x7fc824019030 con 0x7fc83c1f51d0 2024-03-28T13:16:30.075 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:30.075 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:16:29.083492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6043":{"gid":6043,"name":"b","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2260087679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2260087679},{"type":"v1","addr":"172.21.15.188:6836","nonce":2260087679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6043,"qdb_cluster":[6043]},"id":6}]} 2024-03-28T13:16:30.075 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 61 2024-03-28T13:16:30.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.077+0000 7fc843000640 1 -- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc81806c530 msgr2=0x7fc81806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:30.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.077+0000 7fc843000640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc81806c530 0x7fc81806e9f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc82c033000 tx=0x7fc82c009030 comp rx=0 tx=0).stop 2024-03-28T13:16:30.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.077+0000 7fc843000640 1 -- 172.21.15.19:0/1367983460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c1f51d0 msgr2=0x7fc83c15a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:30.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.077+0000 7fc843000640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c1f51d0 0x7fc83c15a790 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fc8240115e0 tx=0x7fc824011aa0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.077+0000 7fc843000640 1 -- 172.21.15.19:0/1367983460 shutdown_connections 2024-03-28T13:16:30.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.077+0000 7fc843000640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc81806c530 0x7fc81806e9f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.077+0000 7fc843000640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc83c15acd0 0x7fc83c0b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.077+0000 7fc843000640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc83c1f51d0 0x7fc83c15a790 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.077+0000 7fc843000640 1 --2- 172.21.15.19:0/1367983460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc820003a00 0x7fc83c159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.077+0000 7fc843000640 1 -- 172.21.15.19:0/1367983460 >> 172.21.15.19:0/1367983460 conn(0x7fc83c0b2cb0 msgr2=0x7fc83c0cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:30.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.077+0000 7fc843000640 1 -- 172.21.15.19:0/1367983460 shutdown_connections 2024-03-28T13:16:30.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.077+0000 7fc843000640 1 -- 172.21.15.19:0/1367983460 wait complete. 2024-03-28T13:16:30.096 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:16:30.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.298+0000 7f5322400640 1 Processor -- start 2024-03-28T13:16:30.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.299+0000 7f5322400640 1 -- start start 2024-03-28T13:16:30.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.300+0000 7f5322400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f531c0d2310 0x7f531c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:30.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.300+0000 7f5322400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f531c15e880 con 0x7f531c15e4a0 2024-03-28T13:16:30.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.300+0000 7f5322400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f531c0d28d0 con 0x7f531c0cf530 2024-03-28T13:16:30.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.300+0000 7f5322400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f531c0d34e0 con 0x7f531c0d2310 2024-03-28T13:16:30.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.300+0000 7f5321000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f531c15e4a0 0x7f531c0b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57294/0 (socket says 172.21.15.19:57294) 2024-03-28T13:16:30.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.300+0000 7f5321000640 1 -- 172.21.15.19:0/427260381 learned_addr learned my addr 172.21.15.19:0/427260381 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:30.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.301+0000 7f531be00640 1 --2- 172.21.15.19:0/427260381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f531c0d2310 0x7f531c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:30.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.301+0000 7f531b400640 1 -- 172.21.15.19:0/427260381 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3547456326 0 0) 0x7f531c15e880 con 0x7f531c15e4a0 2024-03-28T13:16:30.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.301+0000 7f531b400640 1 -- 172.21.15.19:0/427260381 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5300003520 con 0x7f531c15e4a0 2024-03-28T13:16:30.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.301+0000 7f531b400640 1 -- 172.21.15.19:0/427260381 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 425442682 0 0) 0x7f5300003520 con 0x7f531c15e4a0 2024-03-28T13:16:30.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.301+0000 7f531b400640 1 -- 172.21.15.19:0/427260381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f531c0d2310 msgr2=0x7f531c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:30.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.301+0000 7f531b400640 1 --2- 172.21.15.19:0/427260381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f531c0d2310 0x7f531c0ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.301+0000 7f531b400640 1 -- 172.21.15.19:0/427260381 >> v1:172.21.15.188:6789/0 conn(0x7f531c0cf530 legacy=0x7f531c0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:30.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.301+0000 7f531b400640 1 -- 172.21.15.19:0/427260381 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531c0d2bb0 con 0x7f531c15e4a0 2024-03-28T13:16:30.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.301+0000 7f531b400640 1 -- 172.21.15.19:0/427260381 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f530c001670 con 0x7f531c15e4a0 2024-03-28T13:16:30.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.302+0000 7f531b400640 1 -- 172.21.15.19:0/427260381 >> v1:172.21.15.19:6789/0 conn(0x7f531c15e4a0 legacy=0x7f531c0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:30.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.302+0000 7f531b400640 1 --2- 172.21.15.19:0/427260381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53000039e0 0x7f5300003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:30.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.302+0000 7f531b400640 1 --2- 172.21.15.19:0/427260381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f531c0d2310 0x7f5300004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:30.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.302+0000 7f531b400640 1 --2- 172.21.15.19:0/427260381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5300004d40 0x7f53000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:30.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.302+0000 7f531b400640 1 -- 172.21.15.19:0/427260381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f531c0d34e0 con 0x7f5300004d40 2024-03-28T13:16:30.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.302+0000 7f531b400640 1 -- 172.21.15.19:0/427260381 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f530000bad0 con 0x7f53000039e0 2024-03-28T13:16:30.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.302+0000 7f531b400640 1 -- 172.21.15.19:0/427260381 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f530000bcd0 con 0x7f531c0d2310 2024-03-28T13:16:30.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.302+0000 7f5321a00640 1 --2- 172.21.15.19:0/427260381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5300004d40 0x7f53000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:30.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.302+0000 7f5321000640 1 --2- 172.21.15.19:0/427260381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53000039e0 0x7f5300003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:30.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.302+0000 7f531be00640 1 --2- 172.21.15.19:0/427260381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f531c0d2310 0x7f5300004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:30.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.302+0000 7f5321a00640 1 -- 172.21.15.19:0/427260381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f531c0d2310 msgr2=0x7f5300004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:30.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.302+0000 7f5321a00640 1 --2- 172.21.15.19:0/427260381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f531c0d2310 0x7f5300004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.302+0000 7f5321a00640 1 -- 172.21.15.19:0/427260381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53000039e0 msgr2=0x7f5300003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:30.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.302+0000 7f5321a00640 1 --2- 172.21.15.19:0/427260381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53000039e0 0x7f5300003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.303+0000 7f5321a00640 1 -- 172.21.15.19:0/427260381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5300003520 con 0x7f5300004d40 2024-03-28T13:16:30.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.304+0000 7f5321a00640 1 --2- 172.21.15.19:0/427260381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5300004d40 0x7f53000051c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f5314001720 tx=0x7f5300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbe50ccf26017a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:30.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.304+0000 7f531b400640 1 -- 172.21.15.19:0/427260381 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314001370 con 0x7f5300004d40 2024-03-28T13:16:30.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.304+0000 7f531b400640 1 -- 172.21.15.19:0/427260381 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53140052d0 con 0x7f5300004d40 2024-03-28T13:16:30.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.304+0000 7f531b400640 1 -- 172.21.15.19:0/427260381 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314003830 con 0x7f5300004d40 2024-03-28T13:16:30.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.305+0000 7f5322400640 1 -- 172.21.15.19:0/427260381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5300004d40 msgr2=0x7f53000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:30.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.305+0000 7f5322400640 1 --2- 172.21.15.19:0/427260381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5300004d40 0x7f53000051c0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f5314001720 tx=0x7f5300005830 comp rx=0 tx=0).stop 2024-03-28T13:16:30.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.305+0000 7f5321a00640 1 -- 172.21.15.19:0/427260381 reap_dead start 2024-03-28T13:16:30.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.305+0000 7f5322400640 1 -- 172.21.15.19:0/427260381 shutdown_connections 2024-03-28T13:16:30.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.305+0000 7f5322400640 1 -- 172.21.15.19:0/427260381 >> 172.21.15.19:0/427260381 conn(0x7f531c0676d0 msgr2=0x7f531c067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:30.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.305+0000 7f5322400640 1 -- 172.21.15.19:0/427260381 shutdown_connections 2024-03-28T13:16:30.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.305+0000 7f5322400640 1 -- 172.21.15.19:0/427260381 wait complete. 2024-03-28T13:16:30.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.307+0000 7f5322400640 1 Processor -- start 2024-03-28T13:16:30.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.307+0000 7f5322400640 1 -- start start 2024-03-28T13:16:30.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.308+0000 7f5322400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5300004d40 0x7f531c152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:30.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.308+0000 7f5322400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f531c14f510 0x7f531c15f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:30.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.308+0000 7f5322400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c161460 0x7f531c154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:30.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.308+0000 7f5322400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f531c0d3970 con 0x7f531c161460 2024-03-28T13:16:30.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.308+0000 7f5322400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f531c0d3170 con 0x7f531c14f510 2024-03-28T13:16:30.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.309+0000 7f5322400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f531c0ceff0 con 0x7f5300004d40 2024-03-28T13:16:30.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.309+0000 7f5321a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c161460 0x7f531c154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:30.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.309+0000 7f5321a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c161460 0x7f531c154270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39450/0 (socket says 172.21.15.19:39450) 2024-03-28T13:16:30.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.309+0000 7f5321a00640 1 -- 172.21.15.19:0/1197094140 learned_addr learned my addr 172.21.15.19:0/1197094140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:30.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.309+0000 7f531be00640 1 --2- 172.21.15.19:0/1197094140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f531c14f510 0x7f531c15f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:30.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.309+0000 7f5321000640 1 --2- 172.21.15.19:0/1197094140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5300004d40 0x7f531c152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:30.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.309+0000 7f5321a00640 1 -- 172.21.15.19:0/1197094140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5300004d40 msgr2=0x7f531c152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:30.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.309+0000 7f5321a00640 1 --2- 172.21.15.19:0/1197094140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5300004d40 0x7f531c152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.309+0000 7f5321a00640 1 -- 172.21.15.19:0/1197094140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f531c14f510 msgr2=0x7f531c15f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:30.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.309+0000 7f5321a00640 1 --2- 172.21.15.19:0/1197094140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f531c14f510 0x7f531c15f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.309+0000 7f5321a00640 1 -- 172.21.15.19:0/1197094140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531c15fda0 con 0x7f531c161460 2024-03-28T13:16:30.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.309+0000 7f5321000640 1 --2- 172.21.15.19:0/1197094140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5300004d40 0x7f531c152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:30.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.310+0000 7f5321a00640 1 --2- 172.21.15.19:0/1197094140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c161460 0x7f531c154270 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f5314034040 tx=0x7f5314001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:30.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.310+0000 7f5318c00640 1 -- 172.21.15.19:0/1197094140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314001370 con 0x7f531c161460 2024-03-28T13:16:30.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.310+0000 7f5318c00640 1 -- 172.21.15.19:0/1197094140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5314005480 con 0x7f531c161460 2024-03-28T13:16:30.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.310+0000 7f5322400640 1 -- 172.21.15.19:0/1197094140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531c152d00 con 0x7f531c161460 2024-03-28T13:16:30.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.310+0000 7f5318c00640 1 -- 172.21.15.19:0/1197094140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314007a30 con 0x7f531c161460 2024-03-28T13:16:30.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.310+0000 7f5322400640 1 -- 172.21.15.19:0/1197094140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f531c158eb0 con 0x7f531c161460 2024-03-28T13:16:30.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.312+0000 7f5322400640 1 -- 172.21.15.19:0/1197094140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f531c075940 con 0x7f531c161460 2024-03-28T13:16:30.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.313+0000 7f5318c00640 1 -- 172.21.15.19:0/1197094140 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5314046020 con 0x7f531c161460 2024-03-28T13:16:30.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.314+0000 7f5318c00640 1 --2- 172.21.15.19:0/1197094140 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52fc06c560 0x7f52fc06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:30.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.314+0000 7f5318c00640 1 -- 172.21.15.19:0/1197094140 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f53140ac830 con 0x7f531c161460 2024-03-28T13:16:30.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.314+0000 7f5321000640 1 --2- 172.21.15.19:0/1197094140 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52fc06c560 0x7f52fc06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:30.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.315+0000 7f5321000640 1 --2- 172.21.15.19:0/1197094140 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52fc06c560 0x7f52fc06ea20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f530c001eb0 tx=0x7f530c000ed0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:30.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.318+0000 7f5318c00640 1 -- 172.21.15.19:0/1197094140 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f531407aee0 con 0x7f531c161460 2024-03-28T13:16:30.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.521+0000 7f5322400640 1 -- 172.21.15.19:0/1197094140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f531c15d2b0 con 0x7f531c161460 2024-03-28T13:16:30.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.524+0000 7f5318c00640 1 -- 172.21.15.19:0/1197094140 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16672 (secure 0 0 0) 0x7f531407a7e0 con 0x7f531c161460 2024-03-28T13:16:30.524 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:30.524 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":92,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:16:27.532934+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:16:23.901120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-28T13:16:24.990904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:16:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.526+0000 7f5322400640 1 -- 172.21.15.19:0/1197094140 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52fc06c560 msgr2=0x7f52fc06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.526+0000 7f5322400640 1 --2- 172.21.15.19:0/1197094140 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52fc06c560 0x7f52fc06ea20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f530c001eb0 tx=0x7f530c000ed0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.526+0000 7f5322400640 1 -- 172.21.15.19:0/1197094140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c161460 msgr2=0x7f531c154270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.526+0000 7f5322400640 1 --2- 172.21.15.19:0/1197094140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c161460 0x7f531c154270 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f5314034040 tx=0x7f5314001120 comp rx=0 tx=0).stop 2024-03-28T13:16:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.526+0000 7f5322400640 1 -- 172.21.15.19:0/1197094140 shutdown_connections 2024-03-28T13:16:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.526+0000 7f5322400640 1 --2- 172.21.15.19:0/1197094140 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52fc06c560 0x7f52fc06ea20 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.526+0000 7f5322400640 1 --2- 172.21.15.19:0/1197094140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f531c161460 0x7f531c154270 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.526+0000 7f5322400640 1 --2- 172.21.15.19:0/1197094140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f531c14f510 0x7f531c15f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.526+0000 7f5322400640 1 --2- 172.21.15.19:0/1197094140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5300004d40 0x7f531c152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.526+0000 7f5322400640 1 -- 172.21.15.19:0/1197094140 >> 172.21.15.19:0/1197094140 conn(0x7f531c0676d0 msgr2=0x7f531c067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.527+0000 7f5322400640 1 -- 172.21.15.19:0/1197094140 shutdown_connections 2024-03-28T13:16:30.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.527+0000 7f5322400640 1 -- 172.21.15.19:0/1197094140 wait complete. 2024-03-28T13:16:30.547 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-28T13:16:30.547 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:16:30.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.761+0000 7ff2df000640 1 Processor -- start 2024-03-28T13:16:30.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.762+0000 7ff2df000640 1 -- start start 2024-03-28T13:16:30.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.763+0000 7ff2df000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d814ba20 0x7ff2d80b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:30.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.763+0000 7ff2df000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2d80cfda0 0x7ff2d80d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:30.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.763+0000 7ff2df000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2d80b33e0 con 0x7ff2d814c1b0 2024-03-28T13:16:30.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.763+0000 7ff2df000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff2d80d0710 con 0x7ff2d80cfda0 2024-03-28T13:16:30.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.763+0000 7ff2df000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff2d80cde20 con 0x7ff2d814ba20 2024-03-28T13:16:30.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.764+0000 7ff2de600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2d80cfda0 0x7ff2d80d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:30.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.764+0000 7ff2dd200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff2d814c1b0 0x7ff2d8166290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57304/0 (socket says 172.21.15.19:57304) 2024-03-28T13:16:30.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.764+0000 7ff2dd200640 1 -- 172.21.15.19:0/3016405248 learned_addr learned my addr 172.21.15.19:0/3016405248 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:30.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.764+0000 7ff2de600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2d80cfda0 0x7ff2d80d01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39456/0 (socket says 172.21.15.19:39456) 2024-03-28T13:16:30.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.764+0000 7ff2ddc00640 1 --2- 172.21.15.19:0/3016405248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d814ba20 0x7ff2d80b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:30.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.764+0000 7ff2de600640 1 -- 172.21.15.19:0/3016405248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d814ba20 msgr2=0x7ff2d80b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:30.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.764+0000 7ff2de600640 1 --2- 172.21.15.19:0/3016405248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d814ba20 0x7ff2d80b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.764+0000 7ff2de600640 1 -- 172.21.15.19:0/3016405248 >> v1:172.21.15.19:6789/0 conn(0x7ff2d814c1b0 legacy=0x7ff2d8166290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:30.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.764+0000 7ff2de600640 1 -- 172.21.15.19:0/3016405248 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d8166ae0 con 0x7ff2d80cfda0 2024-03-28T13:16:30.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.764+0000 7ff2d7e00640 1 -- 172.21.15.19:0/3016405248 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 925057484 0 0) 0x7ff2d80b33e0 con 0x7ff2d814c1b0 2024-03-28T13:16:30.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.766+0000 7ff2de600640 1 --2- 172.21.15.19:0/3016405248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d80cfda0 0x7ff2d80d01d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009d90 tx=0x7ff2d002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15525f4ed095515c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:30.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.766+0000 7ff2d7e00640 1 -- 172.21.15.19:0/3016405248 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0039070 con 0x7ff2d80cfda0 2024-03-28T13:16:30.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.766+0000 7ff2d7e00640 1 -- 172.21.15.19:0/3016405248 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2d002cc50 con 0x7ff2d80cfda0 2024-03-28T13:16:30.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.766+0000 7ff2d7e00640 1 -- 172.21.15.19:0/3016405248 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0034440 con 0x7ff2d80cfda0 2024-03-28T13:16:30.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.767+0000 7ff2df000640 1 -- 172.21.15.19:0/3016405248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d80cfda0 msgr2=0x7ff2d80d01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:30.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.767+0000 7ff2df000640 1 --2- 172.21.15.19:0/3016405248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d80cfda0 0x7ff2d80d01d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009d90 tx=0x7ff2d002bea0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.767+0000 7ff2df000640 1 -- 172.21.15.19:0/3016405248 shutdown_connections 2024-03-28T13:16:30.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.767+0000 7ff2df000640 1 --2- 172.21.15.19:0/3016405248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d80cfda0 0x7ff2d80d01d0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.767+0000 7ff2df000640 1 --2- 172.21.15.19:0/3016405248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d814ba20 0x7ff2d80b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.767+0000 7ff2df000640 1 -- 172.21.15.19:0/3016405248 >> 172.21.15.19:0/3016405248 conn(0x7ff2d80ccad0 msgr2=0x7ff2d80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:30.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.767+0000 7ff2df000640 1 -- 172.21.15.19:0/3016405248 shutdown_connections 2024-03-28T13:16:30.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.767+0000 7ff2df000640 1 -- 172.21.15.19:0/3016405248 wait complete. 2024-03-28T13:16:30.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.769+0000 7ff2df000640 1 Processor -- start 2024-03-28T13:16:30.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.769+0000 7ff2df000640 1 -- start start 2024-03-28T13:16:30.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.770+0000 7ff2df000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d80cfda0 0x7ff2d80cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:30.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.770+0000 7ff2df000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d814ba20 0x7ff2d80c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:30.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.770+0000 7ff2df000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d814c1b0 0x7ff2d80c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:30.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.770+0000 7ff2df000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d80ce0c0 con 0x7ff2d80cfda0 2024-03-28T13:16:30.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.770+0000 7ff2df000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d81670a0 con 0x7ff2d814ba20 2024-03-28T13:16:30.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.770+0000 7ff2df000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff2d814be00 con 0x7ff2d814c1b0 2024-03-28T13:16:30.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.770+0000 7ff2ddc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d80cfda0 0x7ff2d80cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:30.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.770+0000 7ff2ddc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d80cfda0 0x7ff2d80cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39468/0 (socket says 172.21.15.19:39468) 2024-03-28T13:16:30.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.770+0000 7ff2ddc00640 1 -- 172.21.15.19:0/2043866010 learned_addr learned my addr 172.21.15.19:0/2043866010 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:30.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.771+0000 7ff2dd200640 1 --2- 172.21.15.19:0/2043866010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d814ba20 0x7ff2d80c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:30.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.771+0000 7ff2ddc00640 1 -- 172.21.15.19:0/2043866010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d814c1b0 msgr2=0x7ff2d80c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:30.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.771+0000 7ff2ddc00640 1 --2- 172.21.15.19:0/2043866010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d814c1b0 0x7ff2d80c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.771+0000 7ff2ddc00640 1 -- 172.21.15.19:0/2043866010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d814ba20 msgr2=0x7ff2d80c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:30.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.771+0000 7ff2ddc00640 1 --2- 172.21.15.19:0/2043866010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d814ba20 0x7ff2d80c7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:30.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.771+0000 7ff2ddc00640 1 -- 172.21.15.19:0/2043866010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d8165760 con 0x7ff2d80cfda0 2024-03-28T13:16:30.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.771+0000 7ff2ddc00640 1 --2- 172.21.15.19:0/2043866010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d80cfda0 0x7ff2d80cbfc0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00d5f0 tx=0x7ff2cc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:30.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.771+0000 7ff2d5600640 1 -- 172.21.15.19:0/2043866010 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0125f0 con 0x7ff2d80cfda0 2024-03-28T13:16:30.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.771+0000 7ff2d5600640 1 -- 172.21.15.19:0/2043866010 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2cc019070 con 0x7ff2d80cfda0 2024-03-28T13:16:30.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.771+0000 7ff2d5600640 1 -- 172.21.15.19:0/2043866010 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0129d0 con 0x7ff2d80cfda0 2024-03-28T13:16:30.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.771+0000 7ff2df000640 1 -- 172.21.15.19:0/2043866010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d8165480 con 0x7ff2d80cfda0 2024-03-28T13:16:30.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.772+0000 7ff2df000640 1 -- 172.21.15.19:0/2043866010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d8136200 con 0x7ff2d80cfda0 2024-03-28T13:16:30.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.773+0000 7ff2df000640 1 -- 172.21.15.19:0/2043866010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d8075940 con 0x7ff2d80cfda0 2024-03-28T13:16:30.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.774+0000 7ff2d5600640 1 -- 172.21.15.19:0/2043866010 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff2cc012b70 con 0x7ff2d80cfda0 2024-03-28T13:16:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.775+0000 7ff2d5600640 1 --2- 172.21.15.19:0/2043866010 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff2b806c530 0x7ff2b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:30.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.775+0000 7ff2d5600640 1 -- 172.21.15.19:0/2043866010 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7ff2cc08ba50 con 0x7ff2d80cfda0 2024-03-28T13:16:30.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.775+0000 7ff2dd200640 1 --2- 172.21.15.19:0/2043866010 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff2b806c530 0x7ff2b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:30.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.777+0000 7ff2dd200640 1 --2- 172.21.15.19:0/2043866010 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff2b806c530 0x7ff2b806e9f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8135cc0 tx=0x7ff2c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:30.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.780+0000 7ff2d5600640 1 -- 172.21.15.19:0/2043866010 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff2cc05a100 con 0x7ff2d80cfda0 2024-03-28T13:16:30.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:30.984+0000 7ff2df000640 1 -- 172.21.15.19:0/2043866010 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff2d806f430 con 0x7ff2b806c530 2024-03-28T13:16:31.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.009+0000 7ff2d5600640 1 -- 172.21.15.19:0/2043866010 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252083 (secure 0 0 0) 0x7ff2d806f430 con 0x7ff2b806c530 2024-03-28T13:16:31.018 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:31.020 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:16:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.013+0000 7ff2df000640 1 -- 172.21.15.19:0/2043866010 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff2b806c530 msgr2=0x7ff2b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.013+0000 7ff2df000640 1 --2- 172.21.15.19:0/2043866010 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff2b806c530 0x7ff2b806e9f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8135cc0 tx=0x7ff2c4009040 comp rx=0 tx=0).stop 2024-03-28T13:16:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.013+0000 7ff2df000640 1 -- 172.21.15.19:0/2043866010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d80cfda0 msgr2=0x7ff2d80cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.013+0000 7ff2df000640 1 --2- 172.21.15.19:0/2043866010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d80cfda0 0x7ff2d80cbfc0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc00d5f0 tx=0x7ff2cc00dab0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.014+0000 7ff2df000640 1 -- 172.21.15.19:0/2043866010 shutdown_connections 2024-03-28T13:16:31.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.014+0000 7ff2df000640 1 --2- 172.21.15.19:0/2043866010 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff2b806c530 0x7ff2b806e9f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.014+0000 7ff2df000640 1 --2- 172.21.15.19:0/2043866010 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2d814c1b0 0x7ff2d80c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.014+0000 7ff2df000640 1 --2- 172.21.15.19:0/2043866010 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2d814ba20 0x7ff2d80c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.014+0000 7ff2df000640 1 --2- 172.21.15.19:0/2043866010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2d80cfda0 0x7ff2d80cbfc0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.014+0000 7ff2df000640 1 -- 172.21.15.19:0/2043866010 >> 172.21.15.19:0/2043866010 conn(0x7ff2d80ccad0 msgr2=0x7ff2d80b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.014+0000 7ff2df000640 1 -- 172.21.15.19:0/2043866010 shutdown_connections 2024-03-28T13:16:31.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.014+0000 7ff2df000640 1 -- 172.21.15.19:0/2043866010 wait complete. 2024-03-28T13:16:31.033 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-03-28T13:16:30.137303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259420,"kb_used_data":9948,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715300,"statfs":{"total":773094113280,"available":772828467200,"internally_reserved":0,"allocated":10186752,"data_stored":5021464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.628498"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.950026+0000","last_change":"2024-03-28T13:16:26.571803+0000","last_active":"2024-03-28T13:16:27.950026+0000","last_peered":"2024-03-28T13:16:27.950026+0000","last_clean":"2024-03-28T13:16:27.950026+0000","last_became_active":"2024-03-28T13:16:26.570162+0000","last_became_peered":"2024-03-28T13:16:26.570162+0000","last_unstale":"2024-03-28T13:16:27.950026+0000","last_undegraded":"2024-03-28T13:16:27.950026+0000","last_fullsized":"2024-03-28T13:16:27.950026+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:29.218444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.599278+0000","last_change":"2024-03-28T13:16:26.576354+0000","last_active":"2024-03-28T13:16:27.599278+0000","last_peered":"2024-03-28T13:16:27.599278+0000","last_clean":"2024-03-28T13:16:27.599278+0000","last_became_active":"2024-03-28T13:16:26.576026+0000","last_became_peered":"2024-03-28T13:16:26.576026+0000","last_unstale":"2024-03-28T13:16:27.599278+0000","last_undegraded":"2024-03-28T13:16:27.599278+0000","last_fullsized":"2024-03-28T13:16:27.599278+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:33:31.282726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948872+0000","last_change":"2024-03-28T13:16:26.574621+0000","last_active":"2024-03-28T13:16:27.948872+0000","last_peered":"2024-03-28T13:16:27.948872+0000","last_clean":"2024-03-28T13:16:27.948872+0000","last_became_active":"2024-03-28T13:16:26.574325+0000","last_became_peered":"2024-03-28T13:16:26.574325+0000","last_unstale":"2024-03-28T13:16:27.948872+0000","last_undegraded":"2024-03-28T13:16:27.948872+0000","last_fullsized":"2024-03-28T13:16:27.948872+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:04:13.711061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948508+0000","last_change":"2024-03-28T13:16:26.570230+0000","last_active":"2024-03-28T13:16:27.948508+0000","last_peered":"2024-03-28T13:16:27.948508+0000","last_clean":"2024-03-28T13:16:27.948508+0000","last_became_active":"2024-03-28T13:16:26.568040+0000","last_became_peered":"2024-03-28T13:16:26.568040+0000","last_unstale":"2024-03-28T13:16:27.948508+0000","last_undegraded":"2024-03-28T13:16:27.948508+0000","last_fullsized":"2024-03-28T13:16:27.948508+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:53:23.853768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.542537+0000","last_change":"2024-03-28T13:16:26.566391+0000","last_active":"2024-03-28T13:16:27.542537+0000","last_peered":"2024-03-28T13:16:27.542537+0000","last_clean":"2024-03-28T13:16:27.542537+0000","last_became_active":"2024-03-28T13:16:26.566034+0000","last_became_peered":"2024-03-28T13:16:26.566034+0000","last_unstale":"2024-03-28T13:16:27.542537+0000","last_undegraded":"2024-03-28T13:16:27.542537+0000","last_fullsized":"2024-03-28T13:16:27.542537+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:26:17.121630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.949953+0000","last_change":"2024-03-28T13:16:26.565127+0000","last_active":"2024-03-28T13:16:27.949953+0000","last_peered":"2024-03-28T13:16:27.949953+0000","last_clean":"2024-03-28T13:16:27.949953+0000","last_became_active":"2024-03-28T13:16:26.564824+0000","last_became_peered":"2024-03-28T13:16:26.564824+0000","last_unstale":"2024-03-28T13:16:27.949953+0000","last_undegraded":"2024-03-28T13:16:27.949953+0000","last_fullsized":"2024-03-28T13:16:27.949953+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:09:08.501208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.599029+0000","last_change":"2024-03-28T13:16:26.571305+0000","last_active":"2024-03-28T13:16:27.599029+0000","last_peered":"2024-03-28T13:16:27.599029+0000","last_clean":"2024-03-28T13:16:27.599029+0000","last_became_active":"2024-03-28T13:16:26.570983+0000","last_became_peered":"2024-03-28T13:16:26.570983+0000","last_unstale":"2024-03-28T13:16:27.599029+0000","last_undegraded":"2024-03-28T13:16:27.599029+0000","last_fullsized":"2024-03-28T13:16:27.599029+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:05.539379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.538699+0000","last_change":"2024-03-28T13:16:26.582763+0000","last_active":"2024-03-28T13:16:27.538699+0000","last_peered":"2024-03-28T13:16:27.538699+0000","last_clean":"2024-03-28T13:16:27.538699+0000","last_became_active":"2024-03-28T13:16:26.582439+0000","last_became_peered":"2024-03-28T13:16:26.582439+0000","last_unstale":"2024-03-28T13:16:27.538699+0000","last_undegraded":"2024-03-28T13:16:27.538699+0000","last_fullsized":"2024-03-28T13:16:27.538699+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:46:29.376263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.950157+0000","last_change":"2024-03-28T13:16:26.570657+0000","last_active":"2024-03-28T13:16:27.950157+0000","last_peered":"2024-03-28T13:16:27.950157+0000","last_clean":"2024-03-28T13:16:27.950157+0000","last_became_active":"2024-03-28T13:16:26.570377+0000","last_became_peered":"2024-03-28T13:16:26.570377+0000","last_unstale":"2024-03-28T13:16:27.950157+0000","last_undegraded":"2024-03-28T13:16:27.950157+0000","last_fullsized":"2024-03-28T13:16:27.950157+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:17:03.381917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948390+0000","last_change":"2024-03-28T13:16:26.576814+0000","last_active":"2024-03-28T13:16:27.948390+0000","last_peered":"2024-03-28T13:16:27.948390+0000","last_clean":"2024-03-28T13:16:27.948390+0000","last_became_active":"2024-03-28T13:16:26.576506+0000","last_became_peered":"2024-03-28T13:16:26.576506+0000","last_unstale":"2024-03-28T13:16:27.948390+0000","last_undegraded":"2024-03-28T13:16:27.948390+0000","last_fullsized":"2024-03-28T13:16:27.948390+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:16:55.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594819+0000","last_change":"2024-03-28T13:16:26.585084+0000","last_active":"2024-03-28T13:16:27.594819+0000","last_peered":"2024-03-28T13:16:27.594819+0000","last_clean":"2024-03-28T13:16:27.594819+0000","last_became_active":"2024-03-28T13:16:26.584816+0000","last_became_peered":"2024-03-28T13:16:26.584816+0000","last_unstale":"2024-03-28T13:16:27.594819+0000","last_undegraded":"2024-03-28T13:16:27.594819+0000","last_fullsized":"2024-03-28T13:16:27.594819+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:16:55.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.538930+0000","last_change":"2024-03-28T13:16:26.572546+0000","last_active":"2024-03-28T13:16:27.538930+0000","last_peered":"2024-03-28T13:16:27.538930+0000","last_clean":"2024-03-28T13:16:27.538930+0000","last_became_active":"2024-03-28T13:16:26.572254+0000","last_became_peered":"2024-03-28T13:16:26.572254+0000","last_unstale":"2024-03-28T13:16:27.538930+0000","last_undegraded":"2024-03-28T13:16:27.538930+0000","last_fullsized":"2024-03-28T13:16:27.538930+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:16:55.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.542674+0000","last_change":"2024-03-28T13:16:26.566352+0000","last_active":"2024-03-28T13:16:27.542674+0000","last_peered":"2024-03-28T13:16:27.542674+0000","last_clean":"2024-03-28T13:16:27.542674+0000","last_became_active":"2024-03-28T13:16:26.565936+0000","last_became_peered":"2024-03-28T13:16:26.565936+0000","last_unstale":"2024-03-28T13:16:27.542674+0000","last_undegraded":"2024-03-28T13:16:27.542674+0000","last_fullsized":"2024-03-28T13:16:27.542674+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:13:26.841255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543661+0000","last_change":"2024-03-28T13:16:26.574959+0000","last_active":"2024-03-28T13:16:27.543661+0000","last_peered":"2024-03-28T13:16:27.543661+0000","last_clean":"2024-03-28T13:16:27.543661+0000","last_became_active":"2024-03-28T13:16:26.573832+0000","last_became_peered":"2024-03-28T13:16:26.573832+0000","last_unstale":"2024-03-28T13:16:27.543661+0000","last_undegraded":"2024-03-28T13:16:27.543661+0000","last_fullsized":"2024-03-28T13:16:27.543661+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:21:31.251085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.599204+0000","last_change":"2024-03-28T13:16:26.573795+0000","last_active":"2024-03-28T13:16:27.599204+0000","last_peered":"2024-03-28T13:16:27.599204+0000","last_clean":"2024-03-28T13:16:27.599204+0000","last_became_active":"2024-03-28T13:16:26.573510+0000","last_became_peered":"2024-03-28T13:16:26.573510+0000","last_unstale":"2024-03-28T13:16:27.599204+0000","last_undegraded":"2024-03-28T13:16:27.599204+0000","last_fullsized":"2024-03-28T13:16:27.599204+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:28:57.681141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.947933+0000","last_change":"2024-03-28T13:16:26.560865+0000","last_active":"2024-03-28T13:16:27.947933+0000","last_peered":"2024-03-28T13:16:27.947933+0000","last_clean":"2024-03-28T13:16:27.947933+0000","last_became_active":"2024-03-28T13:16:26.560549+0000","last_became_peered":"2024-03-28T13:16:26.560549+0000","last_unstale":"2024-03-28T13:16:27.947933+0000","last_undegraded":"2024-03-28T13:16:27.947933+0000","last_fullsized":"2024-03-28T13:16:27.947933+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:03.653829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.539630+0000","last_change":"2024-03-28T13:16:26.574844+0000","last_active":"2024-03-28T13:16:27.539630+0000","last_peered":"2024-03-28T13:16:27.539630+0000","last_clean":"2024-03-28T13:16:27.539630+0000","last_became_active":"2024-03-28T13:16:26.571253+0000","last_became_peered":"2024-03-28T13:16:26.571253+0000","last_unstale":"2024-03-28T13:16:27.539630+0000","last_undegraded":"2024-03-28T13:16:27.539630+0000","last_fullsized":"2024-03-28T13:16:27.539630+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:52:29.017485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594259+0000","last_change":"2024-03-28T13:16:26.574039+0000","last_active":"2024-03-28T13:16:27.594259+0000","last_peered":"2024-03-28T13:16:27.594259+0000","last_clean":"2024-03-28T13:16:27.594259+0000","last_became_active":"2024-03-28T13:16:26.573698+0000","last_became_peered":"2024-03-28T13:16:26.573698+0000","last_unstale":"2024-03-28T13:16:27.594259+0000","last_undegraded":"2024-03-28T13:16:27.594259+0000","last_fullsized":"2024-03-28T13:16:27.594259+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:18:32.509652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.544973+0000","last_change":"2024-03-28T13:16:26.573914+0000","last_active":"2024-03-28T13:16:27.544973+0000","last_peered":"2024-03-28T13:16:27.544973+0000","last_clean":"2024-03-28T13:16:27.544973+0000","last_became_active":"2024-03-28T13:16:26.573587+0000","last_became_peered":"2024-03-28T13:16:26.573587+0000","last_unstale":"2024-03-28T13:16:27.544973+0000","last_undegraded":"2024-03-28T13:16:27.544973+0000","last_fullsized":"2024-03-28T13:16:27.544973+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:04:32.980887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.545063+0000","last_change":"2024-03-28T13:16:26.576207+0000","last_active":"2024-03-28T13:16:27.545063+0000","last_peered":"2024-03-28T13:16:27.545063+0000","last_clean":"2024-03-28T13:16:27.545063+0000","last_became_active":"2024-03-28T13:16:26.574161+0000","last_became_peered":"2024-03-28T13:16:26.574161+0000","last_unstale":"2024-03-28T13:16:27.545063+0000","last_undegraded":"2024-03-28T13:16:27.545063+0000","last_fullsized":"2024-03-28T13:16:27.545063+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:14:39.835203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948483+0000","last_change":"2024-03-28T13:16:26.571202+0000","last_active":"2024-03-28T13:16:27.948483+0000","last_peered":"2024-03-28T13:16:27.948483+0000","last_clean":"2024-03-28T13:16:27.948483+0000","last_became_active":"2024-03-28T13:16:26.570576+0000","last_became_peered":"2024-03-28T13:16:26.570576+0000","last_unstale":"2024-03-28T13:16:27.948483+0000","last_undegraded":"2024-03-28T13:16:27.948483+0000","last_fullsized":"2024-03-28T13:16:27.948483+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:55:08.680969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.544177+0000","last_change":"2024-03-28T13:16:26.575971+0000","last_active":"2024-03-28T13:16:27.544177+0000","last_peered":"2024-03-28T13:16:27.544177+0000","last_clean":"2024-03-28T13:16:27.544177+0000","last_became_active":"2024-03-28T13:16:26.575033+0000","last_became_peered":"2024-03-28T13:16:26.575033+0000","last_unstale":"2024-03-28T13:16:27.544177+0000","last_undegraded":"2024-03-28T13:16:27.544177+0000","last_fullsized":"2024-03-28T13:16:27.544177+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:44:58.961862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.538835+0000","last_change":"2024-03-28T13:16:26.572925+0000","last_active":"2024-03-28T13:16:27.538835+0000","last_peered":"2024-03-28T13:16:27.538835+0000","last_clean":"2024-03-28T13:16:27.538835+0000","last_became_active":"2024-03-28T13:16:26.572639+0000","last_became_peered":"2024-03-28T13:16:26.572639+0000","last_unstale":"2024-03-28T13:16:27.538835+0000","last_undegraded":"2024-03-28T13:16:27.538835+0000","last_fullsized":"2024-03-28T13:16:27.538835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:03.653829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.593878+0000","last_change":"2024-03-28T13:16:26.581047+0000","last_active":"2024-03-28T13:16:27.593878+0000","last_peered":"2024-03-28T13:16:27.593878+0000","last_clean":"2024-03-28T13:16:27.593878+0000","last_became_active":"2024-03-28T13:16:26.580721+0000","last_became_peered":"2024-03-28T13:16:26.580721+0000","last_unstale":"2024-03-28T13:16:27.593878+0000","last_undegraded":"2024-03-28T13:16:27.593878+0000","last_fullsized":"2024-03-28T13:16:27.593878+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:20.870581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594177+0000","last_change":"2024-03-28T13:16:26.578020+0000","last_active":"2024-03-28T13:16:27.594177+0000","last_peered":"2024-03-28T13:16:27.594177+0000","last_clean":"2024-03-28T13:16:27.594177+0000","last_became_active":"2024-03-28T13:16:26.577134+0000","last_became_peered":"2024-03-28T13:16:26.577134+0000","last_unstale":"2024-03-28T13:16:27.594177+0000","last_undegraded":"2024-03-28T13:16:27.594177+0000","last_fullsized":"2024-03-28T13:16:27.594177+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:46:29.376263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543203+0000","last_change":"2024-03-28T13:16:26.576452+0000","last_active":"2024-03-28T13:16:27.543203+0000","last_peered":"2024-03-28T13:16:27.543203+0000","last_clean":"2024-03-28T13:16:27.543203+0000","last_became_active":"2024-03-28T13:16:26.576110+0000","last_became_peered":"2024-03-28T13:16:26.576110+0000","last_unstale":"2024-03-28T13:16:27.543203+0000","last_undegraded":"2024-03-28T13:16:27.543203+0000","last_fullsized":"2024-03-28T13:16:27.543203+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:14:46.317120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543028+0000","last_change":"2024-03-28T13:16:26.553909+0000","last_active":"2024-03-28T13:16:27.543028+0000","last_peered":"2024-03-28T13:16:27.543028+0000","last_clean":"2024-03-28T13:16:27.543028+0000","last_became_active":"2024-03-28T13:16:26.553205+0000","last_became_peered":"2024-03-28T13:16:26.553205+0000","last_unstale":"2024-03-28T13:16:27.543028+0000","last_undegraded":"2024-03-28T13:16:27.543028+0000","last_fullsized":"2024-03-28T13:16:27.543028+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:35:13.128575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.949005+0000","last_change":"2024-03-28T13:16:26.580587+0000","last_active":"2024-03-28T13:16:27.949005+0000","last_peered":"2024-03-28T13:16:27.949005+0000","last_clean":"2024-03-28T13:16:27.949005+0000","last_became_active":"2024-03-28T13:16:26.580280+0000","last_became_peered":"2024-03-28T13:16:26.580280+0000","last_unstale":"2024-03-28T13:16:27.949005+0000","last_undegraded":"2024-03-28T13:16:27.949005+0000","last_fullsized":"2024-03-28T13:16:27.949005+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:51:33.798003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948523+0000","last_change":"2024-03-28T13:16:26.563986+0000","last_active":"2024-03-28T13:16:27.948523+0000","last_peered":"2024-03-28T13:16:27.948523+0000","last_clean":"2024-03-28T13:16:27.948523+0000","last_became_active":"2024-03-28T13:16:26.561868+0000","last_became_peered":"2024-03-28T13:16:26.561868+0000","last_unstale":"2024-03-28T13:16:27.948523+0000","last_undegraded":"2024-03-28T13:16:27.948523+0000","last_fullsized":"2024-03-28T13:16:27.948523+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:37:01.791093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.544412+0000","last_change":"2024-03-28T13:16:26.575224+0000","last_active":"2024-03-28T13:16:27.544412+0000","last_peered":"2024-03-28T13:16:27.544412+0000","last_clean":"2024-03-28T13:16:27.544412+0000","last_became_active":"2024-03-28T13:16:26.574702+0000","last_became_peered":"2024-03-28T13:16:26.574702+0000","last_unstale":"2024-03-28T13:16:27.544412+0000","last_undegraded":"2024-03-28T13:16:27.544412+0000","last_fullsized":"2024-03-28T13:16:27.544412+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:48.692843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948582+0000","last_change":"2024-03-28T13:16:26.552978+0000","last_active":"2024-03-28T13:16:27.948582+0000","last_peered":"2024-03-28T13:16:27.948582+0000","last_clean":"2024-03-28T13:16:27.948582+0000","last_became_active":"2024-03-28T13:16:26.552424+0000","last_became_peered":"2024-03-28T13:16:26.552424+0000","last_unstale":"2024-03-28T13:16:27.948582+0000","last_undegraded":"2024-03-28T13:16:27.948582+0000","last_fullsized":"2024-03-28T13:16:27.948582+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:25:59.100715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948091+0000","last_change":"2024-03-28T13:16:26.565426+0000","last_active":"2024-03-28T13:16:27.948091+0000","last_peered":"2024-03-28T13:16:27.948091+0000","last_clean":"2024-03-28T13:16:27.948091+0000","last_became_active":"2024-03-28T13:16:26.564687+0000","last_became_peered":"2024-03-28T13:16:26.564687+0000","last_unstale":"2024-03-28T13:16:27.948091+0000","last_undegraded":"2024-03-28T13:16:27.948091+0000","last_fullsized":"2024-03-28T13:16:27.948091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:18:32.509652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.539594+0000","last_change":"2024-03-28T13:16:26.577450+0000","last_active":"2024-03-28T13:16:27.539594+0000","last_peered":"2024-03-28T13:16:27.539594+0000","last_clean":"2024-03-28T13:16:27.539594+0000","last_became_active":"2024-03-28T13:16:26.572196+0000","last_became_peered":"2024-03-28T13:16:26.572196+0000","last_unstale":"2024-03-28T13:16:27.539594+0000","last_undegraded":"2024-03-28T13:16:27.539594+0000","last_fullsized":"2024-03-28T13:16:27.539594+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:58.786409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"91'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543122+0000","last_change":"2024-03-28T13:16:25.550901+0000","last_active":"2024-03-28T13:16:27.543122+0000","last_peered":"2024-03-28T13:16:27.543122+0000","last_clean":"2024-03-28T13:16:27.543122+0000","last_became_active":"2024-03-28T13:16:25.550246+0000","last_became_peered":"2024-03-28T13:16:25.550246+0000","last_unstale":"2024-03-28T13:16:27.543122+0000","last_undegraded":"2024-03-28T13:16:27.543122+0000","last_fullsized":"2024-03-28T13:16:27.543122+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:01:03.775777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.950653+0000","last_change":"2024-03-28T13:16:26.568503+0000","last_active":"2024-03-28T13:16:27.950653+0000","last_peered":"2024-03-28T13:16:27.950653+0000","last_clean":"2024-03-28T13:16:27.950653+0000","last_became_active":"2024-03-28T13:16:26.568189+0000","last_became_peered":"2024-03-28T13:16:26.568189+0000","last_unstale":"2024-03-28T13:16:27.950653+0000","last_undegraded":"2024-03-28T13:16:27.950653+0000","last_fullsized":"2024-03-28T13:16:27.950653+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:17:08.352742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948445+0000","last_change":"2024-03-28T13:16:25.554343+0000","last_active":"2024-03-28T13:16:27.948445+0000","last_peered":"2024-03-28T13:16:27.948445+0000","last_clean":"2024-03-28T13:16:27.948445+0000","last_became_active":"2024-03-28T13:16:25.554012+0000","last_became_peered":"2024-03-28T13:16:25.554012+0000","last_unstale":"2024-03-28T13:16:27.948445+0000","last_undegraded":"2024-03-28T13:16:27.948445+0000","last_fullsized":"2024-03-28T13:16:27.948445+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:04.525529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.539056+0000","last_change":"2024-03-28T13:16:26.574653+0000","last_active":"2024-03-28T13:16:27.539056+0000","last_peered":"2024-03-28T13:16:27.539056+0000","last_clean":"2024-03-28T13:16:27.539056+0000","last_became_active":"2024-03-28T13:16:26.574373+0000","last_became_peered":"2024-03-28T13:16:26.574373+0000","last_unstale":"2024-03-28T13:16:27.539056+0000","last_undegraded":"2024-03-28T13:16:27.539056+0000","last_fullsized":"2024-03-28T13:16:27.539056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:25:59.100715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594711+0000","last_change":"2024-03-28T13:16:25.544843+0000","last_active":"2024-03-28T13:16:27.594711+0000","last_peered":"2024-03-28T13:16:27.594711+0000","last_clean":"2024-03-28T13:16:27.594711+0000","last_became_active":"2024-03-28T13:16:25.544565+0000","last_became_peered":"2024-03-28T13:16:25.544565+0000","last_unstale":"2024-03-28T13:16:27.594711+0000","last_undegraded":"2024-03-28T13:16:27.594711+0000","last_fullsized":"2024-03-28T13:16:27.594711+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:10:59.152597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948971+0000","last_change":"2024-03-28T13:16:26.588223+0000","last_active":"2024-03-28T13:16:27.948971+0000","last_peered":"2024-03-28T13:16:27.948971+0000","last_clean":"2024-03-28T13:16:27.948971+0000","last_became_active":"2024-03-28T13:16:26.587934+0000","last_became_peered":"2024-03-28T13:16:26.587934+0000","last_unstale":"2024-03-28T13:16:27.948971+0000","last_undegraded":"2024-03-28T13:16:27.948971+0000","last_fullsized":"2024-03-28T13:16:27.948971+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:33:31.282726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948520+0000","last_change":"2024-03-28T13:16:25.544704+0000","last_active":"2024-03-28T13:16:27.948520+0000","last_peered":"2024-03-28T13:16:27.948520+0000","last_clean":"2024-03-28T13:16:27.948520+0000","last_became_active":"2024-03-28T13:16:25.544332+0000","last_became_peered":"2024-03-28T13:16:25.544332+0000","last_unstale":"2024-03-28T13:16:27.948520+0000","last_undegraded":"2024-03-28T13:16:27.948520+0000","last_fullsized":"2024-03-28T13:16:27.948520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:39:12.185447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948683+0000","last_change":"2024-03-28T13:16:26.579300+0000","last_active":"2024-03-28T13:16:27.948683+0000","last_peered":"2024-03-28T13:16:27.948683+0000","last_clean":"2024-03-28T13:16:27.948683+0000","last_became_active":"2024-03-28T13:16:26.578964+0000","last_became_peered":"2024-03-28T13:16:26.578964+0000","last_unstale":"2024-03-28T13:16:27.948683+0000","last_undegraded":"2024-03-28T13:16:27.948683+0000","last_fullsized":"2024-03-28T13:16:27.948683+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:52:18.845220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.538937+0000","last_change":"2024-03-28T13:16:25.535115+0000","last_active":"2024-03-28T13:16:27.538937+0000","last_peered":"2024-03-28T13:16:27.538937+0000","last_clean":"2024-03-28T13:16:27.538937+0000","last_became_active":"2024-03-28T13:16:25.534798+0000","last_became_peered":"2024-03-28T13:16:25.534798+0000","last_unstale":"2024-03-28T13:16:27.538937+0000","last_undegraded":"2024-03-28T13:16:27.538937+0000","last_fullsized":"2024-03-28T13:16:27.538937+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:13:34.656708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.949939+0000","last_change":"2024-03-28T13:16:26.575847+0000","last_active":"2024-03-28T13:16:27.949939+0000","last_peered":"2024-03-28T13:16:27.949939+0000","last_clean":"2024-03-28T13:16:27.949939+0000","last_became_active":"2024-03-28T13:16:26.575565+0000","last_became_peered":"2024-03-28T13:16:26.575565+0000","last_unstale":"2024-03-28T13:16:27.949939+0000","last_undegraded":"2024-03-28T13:16:27.949939+0000","last_fullsized":"2024-03-28T13:16:27.949939+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:39:13.199297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.950172+0000","last_change":"2024-03-28T13:16:25.546604+0000","last_active":"2024-03-28T13:16:27.950172+0000","last_peered":"2024-03-28T13:16:27.950172+0000","last_clean":"2024-03-28T13:16:27.950172+0000","last_became_active":"2024-03-28T13:16:25.546322+0000","last_became_peered":"2024-03-28T13:16:25.546322+0000","last_unstale":"2024-03-28T13:16:27.950172+0000","last_undegraded":"2024-03-28T13:16:27.950172+0000","last_fullsized":"2024-03-28T13:16:27.950172+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:13:34.656708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.947500+0000","last_change":"2024-03-28T13:16:26.582773+0000","last_active":"2024-03-28T13:16:27.947500+0000","last_peered":"2024-03-28T13:16:27.947500+0000","last_clean":"2024-03-28T13:16:27.947500+0000","last_became_active":"2024-03-28T13:16:26.582460+0000","last_became_peered":"2024-03-28T13:16:26.582460+0000","last_unstale":"2024-03-28T13:16:27.947500+0000","last_undegraded":"2024-03-28T13:16:27.947500+0000","last_fullsized":"2024-03-28T13:16:27.947500+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:13:35.670558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543799+0000","last_change":"2024-03-28T13:16:25.559007+0000","last_active":"2024-03-28T13:16:27.543799+0000","last_peered":"2024-03-28T13:16:27.543799+0000","last_clean":"2024-03-28T13:16:27.543799+0000","last_became_active":"2024-03-28T13:16:25.558718+0000","last_became_peered":"2024-03-28T13:16:25.558718+0000","last_unstale":"2024-03-28T13:16:27.543799+0000","last_undegraded":"2024-03-28T13:16:27.543799+0000","last_fullsized":"2024-03-28T13:16:27.543799+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:30.655785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543852+0000","last_change":"2024-03-28T13:16:26.570137+0000","last_active":"2024-03-28T13:16:27.543852+0000","last_peered":"2024-03-28T13:16:27.543852+0000","last_clean":"2024-03-28T13:16:27.543852+0000","last_became_active":"2024-03-28T13:16:26.569880+0000","last_became_peered":"2024-03-28T13:16:26.569880+0000","last_unstale":"2024-03-28T13:16:27.543852+0000","last_undegraded":"2024-03-28T13:16:27.543852+0000","last_fullsized":"2024-03-28T13:16:27.543852+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:42:51.863499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543541+0000","last_change":"2024-03-28T13:16:25.554767+0000","last_active":"2024-03-28T13:16:27.543541+0000","last_peered":"2024-03-28T13:16:27.543541+0000","last_clean":"2024-03-28T13:16:27.543541+0000","last_became_active":"2024-03-28T13:16:25.554244+0000","last_became_peered":"2024-03-28T13:16:25.554244+0000","last_unstale":"2024-03-28T13:16:27.543541+0000","last_undegraded":"2024-03-28T13:16:27.543541+0000","last_fullsized":"2024-03-28T13:16:27.543541+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:20.106923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.545512+0000","last_change":"2024-03-28T13:16:26.569168+0000","last_active":"2024-03-28T13:16:27.545512+0000","last_peered":"2024-03-28T13:16:27.545512+0000","last_clean":"2024-03-28T13:16:27.545512+0000","last_became_active":"2024-03-28T13:16:26.568771+0000","last_became_peered":"2024-03-28T13:16:26.568771+0000","last_unstale":"2024-03-28T13:16:27.545512+0000","last_undegraded":"2024-03-28T13:16:27.545512+0000","last_fullsized":"2024-03-28T13:16:27.545512+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:26:29.748821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.950731+0000","last_change":"2024-03-28T13:16:25.541863+0000","last_active":"2024-03-28T13:16:27.950731+0000","last_peered":"2024-03-28T13:16:27.950731+0000","last_clean":"2024-03-28T13:16:27.950731+0000","last_became_active":"2024-03-28T13:16:25.541381+0000","last_became_peered":"2024-03-28T13:16:25.541381+0000","last_unstale":"2024-03-28T13:16:27.950731+0000","last_undegraded":"2024-03-28T13:16:27.950731+0000","last_fullsized":"2024-03-28T13:16:27.950731+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:02:05.693865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.950704+0000","last_change":"2024-03-28T13:16:26.581749+0000","last_active":"2024-03-28T13:16:27.950704+0000","last_peered":"2024-03-28T13:16:27.950704+0000","last_clean":"2024-03-28T13:16:27.950704+0000","last_became_active":"2024-03-28T13:16:26.581452+0000","last_became_peered":"2024-03-28T13:16:26.581452+0000","last_unstale":"2024-03-28T13:16:27.950704+0000","last_undegraded":"2024-03-28T13:16:27.950704+0000","last_fullsized":"2024-03-28T13:16:27.950704+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:14:27.227998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948555+0000","last_change":"2024-03-28T13:16:25.541016+0000","last_active":"2024-03-28T13:16:27.948555+0000","last_peered":"2024-03-28T13:16:27.948555+0000","last_clean":"2024-03-28T13:16:27.948555+0000","last_became_active":"2024-03-28T13:16:25.540685+0000","last_became_peered":"2024-03-28T13:16:25.540685+0000","last_unstale":"2024-03-28T13:16:27.948555+0000","last_undegraded":"2024-03-28T13:16:27.948555+0000","last_fullsized":"2024-03-28T13:16:27.948555+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:33:30.268877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594109+0000","last_change":"2024-03-28T13:16:26.579141+0000","last_active":"2024-03-28T13:16:27.594109+0000","last_peered":"2024-03-28T13:16:27.594109+0000","last_clean":"2024-03-28T13:16:27.594109+0000","last_became_active":"2024-03-28T13:16:26.578791+0000","last_became_peered":"2024-03-28T13:16:26.578791+0000","last_unstale":"2024-03-28T13:16:27.594109+0000","last_undegraded":"2024-03-28T13:16:27.594109+0000","last_fullsized":"2024-03-28T13:16:27.594109+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:13:38.633845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"91'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.949116+0000","last_change":"2024-03-28T13:16:25.550301+0000","last_active":"2024-03-28T13:16:27.949116+0000","last_peered":"2024-03-28T13:16:27.949116+0000","last_clean":"2024-03-28T13:16:27.949116+0000","last_became_active":"2024-03-28T13:16:25.549994+0000","last_became_peered":"2024-03-28T13:16:25.549994+0000","last_unstale":"2024-03-28T13:16:27.949116+0000","last_undegraded":"2024-03-28T13:16:27.949116+0000","last_fullsized":"2024-03-28T13:16:27.949116+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:06:19.939323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.542963+0000","last_change":"2024-03-28T13:16:26.585091+0000","last_active":"2024-03-28T13:16:27.542963+0000","last_peered":"2024-03-28T13:16:27.542963+0000","last_clean":"2024-03-28T13:16:27.542963+0000","last_became_active":"2024-03-28T13:16:26.584637+0000","last_became_peered":"2024-03-28T13:16:26.584637+0000","last_unstale":"2024-03-28T13:16:27.542963+0000","last_undegraded":"2024-03-28T13:16:27.542963+0000","last_fullsized":"2024-03-28T13:16:27.542963+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:31:43.159136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"91'6","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.539158+0000","last_change":"2024-03-28T13:16:25.533821+0000","last_active":"2024-03-28T13:16:27.539158+0000","last_peered":"2024-03-28T13:16:27.539158+0000","last_clean":"2024-03-28T13:16:27.539158+0000","last_became_active":"2024-03-28T13:16:25.533446+0000","last_became_peered":"2024-03-28T13:16:25.533446+0000","last_unstale":"2024-03-28T13:16:27.539158+0000","last_undegraded":"2024-03-28T13:16:27.539158+0000","last_fullsized":"2024-03-28T13:16:27.539158+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:25:07.854479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.593727+0000","last_change":"2024-03-28T13:16:26.575893+0000","last_active":"2024-03-28T13:16:27.593727+0000","last_peered":"2024-03-28T13:16:27.593727+0000","last_clean":"2024-03-28T13:16:27.593727+0000","last_became_active":"2024-03-28T13:16:26.575451+0000","last_became_peered":"2024-03-28T13:16:26.575451+0000","last_unstale":"2024-03-28T13:16:27.593727+0000","last_undegraded":"2024-03-28T13:16:27.593727+0000","last_fullsized":"2024-03-28T13:16:27.593727+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:14:27.227998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948707+0000","last_change":"2024-03-28T13:16:25.549133+0000","last_active":"2024-03-28T13:16:27.948707+0000","last_peered":"2024-03-28T13:16:27.948707+0000","last_clean":"2024-03-28T13:16:27.948707+0000","last_became_active":"2024-03-28T13:16:25.548803+0000","last_became_peered":"2024-03-28T13:16:25.548803+0000","last_unstale":"2024-03-28T13:16:27.948707+0000","last_undegraded":"2024-03-28T13:16:27.948707+0000","last_fullsized":"2024-03-28T13:16:27.948707+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:52:28.003635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948169+0000","last_change":"2024-03-28T13:16:26.585179+0000","last_active":"2024-03-28T13:16:27.948169+0000","last_peered":"2024-03-28T13:16:27.948169+0000","last_clean":"2024-03-28T13:16:27.948169+0000","last_became_active":"2024-03-28T13:16:26.584802+0000","last_became_peered":"2024-03-28T13:16:26.584802+0000","last_unstale":"2024-03-28T13:16:27.948169+0000","last_undegraded":"2024-03-28T13:16:27.948169+0000","last_fullsized":"2024-03-28T13:16:27.948169+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:28:57.681141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.598822+0000","last_change":"2024-03-28T13:16:25.547021+0000","last_active":"2024-03-28T13:16:27.598822+0000","last_peered":"2024-03-28T13:16:27.598822+0000","last_clean":"2024-03-28T13:16:27.598822+0000","last_became_active":"2024-03-28T13:16:25.546734+0000","last_became_peered":"2024-03-28T13:16:25.546734+0000","last_unstale":"2024-03-28T13:16:27.598822+0000","last_undegraded":"2024-03-28T13:16:27.598822+0000","last_fullsized":"2024-03-28T13:16:27.598822+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:02:15.752917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543736+0000","last_change":"2024-03-28T13:16:26.581984+0000","last_active":"2024-03-28T13:16:27.543736+0000","last_peered":"2024-03-28T13:16:27.543736+0000","last_clean":"2024-03-28T13:16:27.543736+0000","last_became_active":"2024-03-28T13:16:26.581697+0000","last_became_peered":"2024-03-28T13:16:26.581697+0000","last_unstale":"2024-03-28T13:16:27.543736+0000","last_undegraded":"2024-03-28T13:16:27.543736+0000","last_fullsized":"2024-03-28T13:16:27.543736+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:38.777163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948724+0000","last_change":"2024-03-28T13:16:25.543304+0000","last_active":"2024-03-28T13:16:27.948724+0000","last_peered":"2024-03-28T13:16:27.948724+0000","last_clean":"2024-03-28T13:16:27.948724+0000","last_became_active":"2024-03-28T13:16:25.542973+0000","last_became_peered":"2024-03-28T13:16:25.542973+0000","last_unstale":"2024-03-28T13:16:27.948724+0000","last_undegraded":"2024-03-28T13:16:27.948724+0000","last_fullsized":"2024-03-28T13:16:27.948724+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:53:31.570839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.593381+0000","last_change":"2024-03-28T13:16:26.573989+0000","last_active":"2024-03-28T13:16:27.593381+0000","last_peered":"2024-03-28T13:16:27.593381+0000","last_clean":"2024-03-28T13:16:27.593381+0000","last_became_active":"2024-03-28T13:16:26.573669+0000","last_became_peered":"2024-03-28T13:16:26.573669+0000","last_unstale":"2024-03-28T13:16:27.593381+0000","last_undegraded":"2024-03-28T13:16:27.593381+0000","last_fullsized":"2024-03-28T13:16:27.593381+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:17:08.352742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594427+0000","last_change":"2024-03-28T13:16:25.549703+0000","last_active":"2024-03-28T13:16:27.594427+0000","last_peered":"2024-03-28T13:16:27.594427+0000","last_clean":"2024-03-28T13:16:27.594427+0000","last_became_active":"2024-03-28T13:16:25.549285+0000","last_became_peered":"2024-03-28T13:16:25.549285+0000","last_unstale":"2024-03-28T13:16:27.594427+0000","last_undegraded":"2024-03-28T13:16:27.594427+0000","last_fullsized":"2024-03-28T13:16:27.594427+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:16:33.411736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.539697+0000","last_change":"2024-03-28T13:16:26.569939+0000","last_active":"2024-03-28T13:16:27.539697+0000","last_peered":"2024-03-28T13:16:27.539697+0000","last_clean":"2024-03-28T13:16:27.539697+0000","last_became_active":"2024-03-28T13:16:26.569621+0000","last_became_peered":"2024-03-28T13:16:26.569621+0000","last_unstale":"2024-03-28T13:16:27.539697+0000","last_undegraded":"2024-03-28T13:16:27.539697+0000","last_fullsized":"2024-03-28T13:16:27.539697+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:39:13.199297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543313+0000","last_change":"2024-03-28T13:16:25.554065+0000","last_active":"2024-03-28T13:16:27.543313+0000","last_peered":"2024-03-28T13:16:27.543313+0000","last_clean":"2024-03-28T13:16:27.543313+0000","last_became_active":"2024-03-28T13:16:25.553751+0000","last_became_peered":"2024-03-28T13:16:25.553751+0000","last_unstale":"2024-03-28T13:16:27.543313+0000","last_undegraded":"2024-03-28T13:16:27.543313+0000","last_fullsized":"2024-03-28T13:16:27.543313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:52.836993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.949423+0000","last_change":"2024-03-28T13:16:26.576404+0000","last_active":"2024-03-28T13:16:27.949423+0000","last_peered":"2024-03-28T13:16:27.949423+0000","last_clean":"2024-03-28T13:16:27.949423+0000","last_became_active":"2024-03-28T13:16:26.575956+0000","last_became_peered":"2024-03-28T13:16:26.575956+0000","last_unstale":"2024-03-28T13:16:27.949423+0000","last_undegraded":"2024-03-28T13:16:27.949423+0000","last_fullsized":"2024-03-28T13:16:27.949423+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:52:29.017485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.545603+0000","last_change":"2024-03-28T13:16:25.557497+0000","last_active":"2024-03-28T13:16:27.545603+0000","last_peered":"2024-03-28T13:16:27.545603+0000","last_clean":"2024-03-28T13:16:27.545603+0000","last_became_active":"2024-03-28T13:16:25.557086+0000","last_became_peered":"2024-03-28T13:16:25.557086+0000","last_unstale":"2024-03-28T13:16:27.545603+0000","last_undegraded":"2024-03-28T13:16:27.545603+0000","last_fullsized":"2024-03-28T13:16:27.545603+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:16:42.842395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.539409+0000","last_change":"2024-03-28T13:16:26.570099+0000","last_active":"2024-03-28T13:16:27.539409+0000","last_peered":"2024-03-28T13:16:27.539409+0000","last_clean":"2024-03-28T13:16:27.539409+0000","last_became_active":"2024-03-28T13:16:26.569798+0000","last_became_peered":"2024-03-28T13:16:26.569798+0000","last_unstale":"2024-03-28T13:16:27.539409+0000","last_undegraded":"2024-03-28T13:16:27.539409+0000","last_fullsized":"2024-03-28T13:16:27.539409+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:04:13.711061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.545149+0000","last_change":"2024-03-28T13:16:25.555200+0000","last_active":"2024-03-28T13:16:27.545149+0000","last_peered":"2024-03-28T13:16:27.545149+0000","last_clean":"2024-03-28T13:16:27.545149+0000","last_became_active":"2024-03-28T13:16:25.554835+0000","last_became_peered":"2024-03-28T13:16:25.554835+0000","last_unstale":"2024-03-28T13:16:27.545149+0000","last_undegraded":"2024-03-28T13:16:27.545149+0000","last_fullsized":"2024-03-28T13:16:27.545149+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:52:43.906362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.598820+0000","last_change":"2024-03-28T13:16:26.584849+0000","last_active":"2024-03-28T13:16:27.598820+0000","last_peered":"2024-03-28T13:16:27.598820+0000","last_clean":"2024-03-28T13:16:27.598820+0000","last_became_active":"2024-03-28T13:16:26.584510+0000","last_became_peered":"2024-03-28T13:16:26.584510+0000","last_unstale":"2024-03-28T13:16:27.598820+0000","last_undegraded":"2024-03-28T13:16:27.598820+0000","last_fullsized":"2024-03-28T13:16:27.598820+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:25:08.868328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.544797+0000","last_change":"2024-03-28T13:16:25.555898+0000","last_active":"2024-03-28T13:16:27.544797+0000","last_peered":"2024-03-28T13:16:27.544797+0000","last_clean":"2024-03-28T13:16:27.544797+0000","last_became_active":"2024-03-28T13:16:25.555609+0000","last_became_peered":"2024-03-28T13:16:25.555609+0000","last_unstale":"2024-03-28T13:16:27.544797+0000","last_undegraded":"2024-03-28T13:16:27.544797+0000","last_fullsized":"2024-03-28T13:16:27.544797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:11:15.489678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.542790+0000","last_change":"2024-03-28T13:16:26.578505+0000","last_active":"2024-03-28T13:16:27.542790+0000","last_peered":"2024-03-28T13:16:27.542790+0000","last_clean":"2024-03-28T13:16:27.542790+0000","last_became_active":"2024-03-28T13:16:26.578166+0000","last_became_peered":"2024-03-28T13:16:26.578166+0000","last_unstale":"2024-03-28T13:16:27.542790+0000","last_undegraded":"2024-03-28T13:16:27.542790+0000","last_fullsized":"2024-03-28T13:16:27.542790+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:46:47.614785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.598966+0000","last_change":"2024-03-28T13:16:25.538552+0000","last_active":"2024-03-28T13:16:27.598966+0000","last_peered":"2024-03-28T13:16:27.598966+0000","last_clean":"2024-03-28T13:16:27.598966+0000","last_became_active":"2024-03-28T13:16:25.538187+0000","last_became_peered":"2024-03-28T13:16:25.538187+0000","last_unstale":"2024-03-28T13:16:27.598966+0000","last_undegraded":"2024-03-28T13:16:27.598966+0000","last_fullsized":"2024-03-28T13:16:27.598966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:55.502492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.947948+0000","last_change":"2024-03-28T13:16:26.577569+0000","last_active":"2024-03-28T13:16:27.947948+0000","last_peered":"2024-03-28T13:16:27.947948+0000","last_clean":"2024-03-28T13:16:27.947948+0000","last_became_active":"2024-03-28T13:16:26.576970+0000","last_became_peered":"2024-03-28T13:16:26.576970+0000","last_unstale":"2024-03-28T13:16:27.947948+0000","last_undegraded":"2024-03-28T13:16:27.947948+0000","last_fullsized":"2024-03-28T13:16:27.947948+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:59:54.712670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594272+0000","last_change":"2024-03-28T13:16:25.547754+0000","last_active":"2024-03-28T13:16:27.594272+0000","last_peered":"2024-03-28T13:16:27.594272+0000","last_clean":"2024-03-28T13:16:27.594272+0000","last_became_active":"2024-03-28T13:16:25.547481+0000","last_became_peered":"2024-03-28T13:16:25.547481+0000","last_unstale":"2024-03-28T13:16:27.594272+0000","last_undegraded":"2024-03-28T13:16:27.594272+0000","last_fullsized":"2024-03-28T13:16:27.594272+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:55.502492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.538859+0000","last_change":"2024-03-28T13:16:26.575008+0000","last_active":"2024-03-28T13:16:27.538859+0000","last_peered":"2024-03-28T13:16:27.538859+0000","last_clean":"2024-03-28T13:16:27.538859+0000","last_became_active":"2024-03-28T13:16:26.574434+0000","last_became_peered":"2024-03-28T13:16:26.574434+0000","last_unstale":"2024-03-28T13:16:27.538859+0000","last_undegraded":"2024-03-28T13:16:27.538859+0000","last_fullsized":"2024-03-28T13:16:27.538859+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:18:32.509652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.544005+0000","last_change":"2024-03-28T13:16:25.555119+0000","last_active":"2024-03-28T13:16:27.544005+0000","last_peered":"2024-03-28T13:16:27.544005+0000","last_clean":"2024-03-28T13:16:27.544005+0000","last_became_active":"2024-03-28T13:16:25.554798+0000","last_became_peered":"2024-03-28T13:16:25.554798+0000","last_unstale":"2024-03-28T13:16:27.544005+0000","last_undegraded":"2024-03-28T13:16:27.544005+0000","last_fullsized":"2024-03-28T13:16:27.544005+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:37:12.073635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543258+0000","last_change":"2024-03-28T13:16:26.573241+0000","last_active":"2024-03-28T13:16:27.543258+0000","last_peered":"2024-03-28T13:16:27.543258+0000","last_clean":"2024-03-28T13:16:27.543258+0000","last_became_active":"2024-03-28T13:16:26.570418+0000","last_became_peered":"2024-03-28T13:16:26.570418+0000","last_unstale":"2024-03-28T13:16:27.543258+0000","last_undegraded":"2024-03-28T13:16:27.543258+0000","last_fullsized":"2024-03-28T13:16:27.543258+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:42.354543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.545504+0000","last_change":"2024-03-28T13:16:25.557467+0000","last_active":"2024-03-28T13:16:27.545504+0000","last_peered":"2024-03-28T13:16:27.545504+0000","last_clean":"2024-03-28T13:16:27.545504+0000","last_became_active":"2024-03-28T13:16:25.557053+0000","last_became_peered":"2024-03-28T13:16:25.557053+0000","last_unstale":"2024-03-28T13:16:27.545504+0000","last_undegraded":"2024-03-28T13:16:27.545504+0000","last_fullsized":"2024-03-28T13:16:27.545504+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:03:28.580115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543227+0000","last_change":"2024-03-28T13:16:26.572470+0000","last_active":"2024-03-28T13:16:27.543227+0000","last_peered":"2024-03-28T13:16:27.543227+0000","last_clean":"2024-03-28T13:16:27.543227+0000","last_became_active":"2024-03-28T13:16:26.572130+0000","last_became_peered":"2024-03-28T13:16:26.572130+0000","last_unstale":"2024-03-28T13:16:27.543227+0000","last_undegraded":"2024-03-28T13:16:27.543227+0000","last_fullsized":"2024-03-28T13:16:27.543227+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:25:07.768733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"92'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.597828+0000","last_change":"2024-03-28T13:16:25.552029+0000","last_active":"2024-03-28T13:16:27.597828+0000","last_peered":"2024-03-28T13:16:27.597828+0000","last_clean":"2024-03-28T13:16:27.597828+0000","last_became_active":"2024-03-28T13:16:25.551733+0000","last_became_peered":"2024-03-28T13:16:25.551733+0000","last_unstale":"2024-03-28T13:16:27.597828+0000","last_undegraded":"2024-03-28T13:16:27.597828+0000","last_fullsized":"2024-03-28T13:16:27.597828+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:14:24.831395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.542914+0000","last_change":"2024-03-28T13:16:26.567250+0000","last_active":"2024-03-28T13:16:27.542914+0000","last_peered":"2024-03-28T13:16:27.542914+0000","last_clean":"2024-03-28T13:16:27.542914+0000","last_became_active":"2024-03-28T13:16:26.566906+0000","last_became_peered":"2024-03-28T13:16:26.566906+0000","last_unstale":"2024-03-28T13:16:27.542914+0000","last_undegraded":"2024-03-28T13:16:27.542914+0000","last_fullsized":"2024-03-28T13:16:27.542914+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:58.831175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.539266+0000","last_change":"2024-03-28T13:16:25.550476+0000","last_active":"2024-03-28T13:16:27.539266+0000","last_peered":"2024-03-28T13:16:27.539266+0000","last_clean":"2024-03-28T13:16:27.539266+0000","last_became_active":"2024-03-28T13:16:25.549893+0000","last_became_peered":"2024-03-28T13:16:25.549893+0000","last_unstale":"2024-03-28T13:16:27.539266+0000","last_undegraded":"2024-03-28T13:16:27.539266+0000","last_fullsized":"2024-03-28T13:16:27.539266+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:04.525529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.949280+0000","last_change":"2024-03-28T13:16:26.571564+0000","last_active":"2024-03-28T13:16:27.949280+0000","last_peered":"2024-03-28T13:16:27.949280+0000","last_clean":"2024-03-28T13:16:27.949280+0000","last_became_active":"2024-03-28T13:16:26.571224+0000","last_became_peered":"2024-03-28T13:16:26.571224+0000","last_unstale":"2024-03-28T13:16:27.949280+0000","last_undegraded":"2024-03-28T13:16:27.949280+0000","last_fullsized":"2024-03-28T13:16:27.949280+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:58.786409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948365+0000","last_change":"2024-03-28T13:16:25.554236+0000","last_active":"2024-03-28T13:16:27.948365+0000","last_peered":"2024-03-28T13:16:27.948365+0000","last_clean":"2024-03-28T13:16:27.948365+0000","last_became_active":"2024-03-28T13:16:25.553943+0000","last_became_peered":"2024-03-28T13:16:25.553943+0000","last_unstale":"2024-03-28T13:16:27.948365+0000","last_undegraded":"2024-03-28T13:16:27.948365+0000","last_fullsized":"2024-03-28T13:16:27.948365+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:28:56.667291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948119+0000","last_change":"2024-03-28T13:16:26.576725+0000","last_active":"2024-03-28T13:16:27.948119+0000","last_peered":"2024-03-28T13:16:27.948119+0000","last_clean":"2024-03-28T13:16:27.948119+0000","last_became_active":"2024-03-28T13:16:26.576434+0000","last_became_peered":"2024-03-28T13:16:26.576434+0000","last_unstale":"2024-03-28T13:16:27.948119+0000","last_undegraded":"2024-03-28T13:16:27.948119+0000","last_fullsized":"2024-03-28T13:16:27.948119+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:05.539379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.599013+0000","last_change":"2024-03-28T13:16:25.548513+0000","last_active":"2024-03-28T13:16:27.599013+0000","last_peered":"2024-03-28T13:16:27.599013+0000","last_clean":"2024-03-28T13:16:27.599013+0000","last_became_active":"2024-03-28T13:16:25.547239+0000","last_became_peered":"2024-03-28T13:16:25.547239+0000","last_unstale":"2024-03-28T13:16:27.599013+0000","last_undegraded":"2024-03-28T13:16:27.599013+0000","last_fullsized":"2024-03-28T13:16:27.599013+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:14:24.831395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594216+0000","last_change":"2024-03-28T13:16:26.577794+0000","last_active":"2024-03-28T13:16:27.594216+0000","last_peered":"2024-03-28T13:16:27.594216+0000","last_clean":"2024-03-28T13:16:27.594216+0000","last_became_active":"2024-03-28T13:16:26.577506+0000","last_became_peered":"2024-03-28T13:16:26.577506+0000","last_unstale":"2024-03-28T13:16:27.594216+0000","last_undegraded":"2024-03-28T13:16:27.594216+0000","last_fullsized":"2024-03-28T13:16:27.594216+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:03.653829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.950171+0000","last_change":"2024-03-28T13:16:25.546354+0000","last_active":"2024-03-28T13:16:27.950171+0000","last_peered":"2024-03-28T13:16:27.950171+0000","last_clean":"2024-03-28T13:16:27.950171+0000","last_became_active":"2024-03-28T13:16:25.545904+0000","last_became_peered":"2024-03-28T13:16:25.545904+0000","last_unstale":"2024-03-28T13:16:27.950171+0000","last_undegraded":"2024-03-28T13:16:27.950171+0000","last_fullsized":"2024-03-28T13:16:27.950171+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:25:07.854479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.947497+0000","last_change":"2024-03-28T13:16:26.571384+0000","last_active":"2024-03-28T13:16:27.947497+0000","last_peered":"2024-03-28T13:16:27.947497+0000","last_clean":"2024-03-28T13:16:27.947497+0000","last_became_active":"2024-03-28T13:16:26.571053+0000","last_became_peered":"2024-03-28T13:16:26.571053+0000","last_unstale":"2024-03-28T13:16:27.947497+0000","last_undegraded":"2024-03-28T13:16:27.947497+0000","last_fullsized":"2024-03-28T13:16:27.947497+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:53:32.584688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.947379+0000","last_change":"2024-03-28T13:16:25.552038+0000","last_active":"2024-03-28T13:16:27.947379+0000","last_peered":"2024-03-28T13:16:27.947379+0000","last_clean":"2024-03-28T13:16:27.947379+0000","last_became_active":"2024-03-28T13:16:25.550418+0000","last_became_peered":"2024-03-28T13:16:25.550418+0000","last_unstale":"2024-03-28T13:16:27.947379+0000","last_undegraded":"2024-03-28T13:16:27.947379+0000","last_fullsized":"2024-03-28T13:16:27.947379+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:46:51.091024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.947535+0000","last_change":"2024-03-28T13:16:26.573312+0000","last_active":"2024-03-28T13:16:27.947535+0000","last_peered":"2024-03-28T13:16:27.947535+0000","last_clean":"2024-03-28T13:16:27.947535+0000","last_became_active":"2024-03-28T13:16:26.573032+0000","last_became_peered":"2024-03-28T13:16:26.573032+0000","last_unstale":"2024-03-28T13:16:27.947535+0000","last_undegraded":"2024-03-28T13:16:27.947535+0000","last_fullsized":"2024-03-28T13:16:27.947535+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:25:08.868328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.538672+0000","last_change":"2024-03-28T13:16:25.544118+0000","last_active":"2024-03-28T13:16:27.538672+0000","last_peered":"2024-03-28T13:16:27.538672+0000","last_clean":"2024-03-28T13:16:27.538672+0000","last_became_active":"2024-03-28T13:16:25.543788+0000","last_became_peered":"2024-03-28T13:16:25.543788+0000","last_unstale":"2024-03-28T13:16:27.538672+0000","last_undegraded":"2024-03-28T13:16:27.538672+0000","last_fullsized":"2024-03-28T13:16:27.538672+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:53:31.570839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.593972+0000","last_change":"2024-03-28T13:16:26.574867+0000","last_active":"2024-03-28T13:16:27.593972+0000","last_peered":"2024-03-28T13:16:27.593972+0000","last_clean":"2024-03-28T13:16:27.593972+0000","last_became_active":"2024-03-28T13:16:26.573997+0000","last_became_peered":"2024-03-28T13:16:26.573997+0000","last_unstale":"2024-03-28T13:16:27.593972+0000","last_undegraded":"2024-03-28T13:16:27.593972+0000","last_fullsized":"2024-03-28T13:16:27.593972+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:44:39.700316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543053+0000","last_change":"2024-03-28T13:16:25.552367+0000","last_active":"2024-03-28T13:16:27.543053+0000","last_peered":"2024-03-28T13:16:27.543053+0000","last_clean":"2024-03-28T13:16:27.543053+0000","last_became_active":"2024-03-28T13:16:25.551693+0000","last_became_peered":"2024-03-28T13:16:25.551693+0000","last_unstale":"2024-03-28T13:16:27.543053+0000","last_undegraded":"2024-03-28T13:16:27.543053+0000","last_fullsized":"2024-03-28T13:16:27.543053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:57.884493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":298,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.947654+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:16:27.947654+0000","last_peered":"2024-03-28T13:16:27.947654+0000","last_clean":"2024-03-28T13:16:27.947654+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:16:27.947654+0000","last_undegraded":"2024-03-28T13:16:27.947654+0000","last_fullsized":"2024-03-28T13:16:27.947654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476826,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":480950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476824,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1118208,"data_stored":480276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476824,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1716224,"data_stored":1070734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476824,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1134592,"data_stored":481910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509527,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":480948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509528,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":1072290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509528,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":480310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542232,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1110016,"data_stored":474046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:16:31.041 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:16:31.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.300+0000 7f3bdfe00640 1 Processor -- start 2024-03-28T13:16:31.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.300+0000 7f3bdfe00640 1 -- start start 2024-03-28T13:16:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.301+0000 7f3bdfe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3be01442c0 0x7f3be00ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.301+0000 7f3bdfe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3be00d02a0 0x7f3be00b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.301+0000 7f3bdfe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3be0071460 con 0x7f3be00cefe0 2024-03-28T13:16:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.301+0000 7f3bdfe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3be0071d20 con 0x7f3be01442c0 2024-03-28T13:16:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.301+0000 7f3bdfe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3be00cf420 con 0x7f3be00d02a0 2024-03-28T13:16:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.302+0000 7f3bdea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3be01442c0 0x7f3be00ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.302+0000 7f3bdea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3be01442c0 0x7f3be00ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39476/0 (socket says 172.21.15.19:39476) 2024-03-28T13:16:31.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.302+0000 7f3bdea00640 1 -- 172.21.15.19:0/1418477299 learned_addr learned my addr 172.21.15.19:0/1418477299 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:31.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.302+0000 7f3bdea00640 1 -- 172.21.15.19:0/1418477299 >> v2:172.21.15.188:3300/0 conn(0x7f3be00d02a0 msgr2=0x7f3be00b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:31.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.302+0000 7f3bdea00640 1 --2- 172.21.15.19:0/1418477299 >> v2:172.21.15.188:3300/0 conn(0x7f3be00d02a0 0x7f3be00b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.302+0000 7f3bdea00640 1 -- 172.21.15.19:0/1418477299 >> v1:172.21.15.188:6789/0 conn(0x7f3be00cefe0 legacy=0x7f3be00cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:31.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.302+0000 7f3bdea00640 1 -- 172.21.15.19:0/1418477299 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be00721b0 con 0x7f3be01442c0 2024-03-28T13:16:31.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.303+0000 7f3bdea00640 1 --2- 172.21.15.19:0/1418477299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be01442c0 0x7f3be00ceaa0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f3be00b3a70 tx=0x7f3bd002b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ae7a4ec309327c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:31.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.304+0000 7f3bde000640 1 -- 172.21.15.19:0/1418477299 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0034070 con 0x7f3be01442c0 2024-03-28T13:16:31.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.304+0000 7f3bde000640 1 -- 172.21.15.19:0/1418477299 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bd0030880 con 0x7f3be01442c0 2024-03-28T13:16:31.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.304+0000 7f3bde000640 1 -- 172.21.15.19:0/1418477299 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0030c50 con 0x7f3be01442c0 2024-03-28T13:16:31.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.304+0000 7f3bdfe00640 1 -- 172.21.15.19:0/1418477299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be01442c0 msgr2=0x7f3be00ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:31.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.304+0000 7f3bdfe00640 1 --2- 172.21.15.19:0/1418477299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be01442c0 0x7f3be00ceaa0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f3be00b3a70 tx=0x7f3bd002b9c0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.304+0000 7f3bdfe00640 1 -- 172.21.15.19:0/1418477299 shutdown_connections 2024-03-28T13:16:31.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.304+0000 7f3bdfe00640 1 --2- 172.21.15.19:0/1418477299 >> v2:172.21.15.188:3300/0 conn(0x7f3be00d02a0 0x7f3be00b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.304+0000 7f3bdfe00640 1 --2- 172.21.15.19:0/1418477299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be01442c0 0x7f3be00ceaa0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.305+0000 7f3bdfe00640 1 -- 172.21.15.19:0/1418477299 >> 172.21.15.19:0/1418477299 conn(0x7f3be00ccad0 msgr2=0x7f3be00cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:31.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.305+0000 7f3bdfe00640 1 -- 172.21.15.19:0/1418477299 shutdown_connections 2024-03-28T13:16:31.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.305+0000 7f3bdfe00640 1 -- 172.21.15.19:0/1418477299 wait complete. 2024-03-28T13:16:31.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.306+0000 7f3bdfe00640 1 Processor -- start 2024-03-28T13:16:31.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.307+0000 7f3bdfe00640 1 -- start start 2024-03-28T13:16:31.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.307+0000 7f3bdfe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3be00cefe0 0x7f3be015acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:31.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.307+0000 7f3bdfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be00d02a0 0x7f3be0136ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:31.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.308+0000 7f3bdfe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3be01442c0 0x7f3be01373e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:31.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.308+0000 7f3bdfe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3be00d0770 con 0x7f3be00d02a0 2024-03-28T13:16:31.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.308+0000 7f3bdfe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3be00719b0 con 0x7f3be00cefe0 2024-03-28T13:16:31.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.308+0000 7f3bdfe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3be01446a0 con 0x7f3be01442c0 2024-03-28T13:16:31.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.308+0000 7f3bd6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be00d02a0 0x7f3be0136ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:31.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.308+0000 7f3bd6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be00d02a0 0x7f3be0136ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39478/0 (socket says 172.21.15.19:39478) 2024-03-28T13:16:31.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.308+0000 7f3bd6000640 1 -- 172.21.15.19:0/532572153 learned_addr learned my addr 172.21.15.19:0/532572153 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:31.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.308+0000 7f3bdea00640 1 --2- 172.21.15.19:0/532572153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3be00cefe0 0x7f3be015acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:31.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.308+0000 7f3bdf400640 1 --2- 172.21.15.19:0/532572153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3be01442c0 0x7f3be01373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:31.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.308+0000 7f3bd6000640 1 -- 172.21.15.19:0/532572153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3be01442c0 msgr2=0x7f3be01373e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:31.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.308+0000 7f3bd6000640 1 --2- 172.21.15.19:0/532572153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3be01442c0 0x7f3be01373e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.308+0000 7f3bd6000640 1 -- 172.21.15.19:0/532572153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3be00cefe0 msgr2=0x7f3be015acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:31.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.308+0000 7f3bd6000640 1 --2- 172.21.15.19:0/532572153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3be00cefe0 0x7f3be015acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.308+0000 7f3bd6000640 1 -- 172.21.15.19:0/532572153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be0137af0 con 0x7f3be00d02a0 2024-03-28T13:16:31.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.308+0000 7f3bdf400640 1 --2- 172.21.15.19:0/532572153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3be01442c0 0x7f3be01373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:31.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.309+0000 7f3bd6000640 1 --2- 172.21.15.19:0/532572153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be00d02a0 0x7f3be0136ea0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800c430 tx=0x7f3bc800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:31.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.309+0000 7f3bd7e00640 1 -- 172.21.15.19:0/532572153 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8011790 con 0x7f3be00d02a0 2024-03-28T13:16:31.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.309+0000 7f3bd7e00640 1 -- 172.21.15.19:0/532572153 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bc8017070 con 0x7f3be00d02a0 2024-03-28T13:16:31.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.309+0000 7f3bd7e00640 1 -- 172.21.15.19:0/532572153 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8011bb0 con 0x7f3be00d02a0 2024-03-28T13:16:31.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.309+0000 7f3bdfe00640 1 -- 172.21.15.19:0/532572153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be0137dd0 con 0x7f3be00d02a0 2024-03-28T13:16:31.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.309+0000 7f3bdfe00640 1 -- 172.21.15.19:0/532572153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be0138390 con 0x7f3be00d02a0 2024-03-28T13:16:31.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.310+0000 7f3bdfe00640 1 -- 172.21.15.19:0/532572153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be0075940 con 0x7f3be00d02a0 2024-03-28T13:16:31.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.311+0000 7f3bd7e00640 1 -- 172.21.15.19:0/532572153 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3bc801c460 con 0x7f3be00d02a0 2024-03-28T13:16:31.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.312+0000 7f3bd7e00640 1 --2- 172.21.15.19:0/532572153 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3bbc06c530 0x7f3bbc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:31.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.312+0000 7f3bd7e00640 1 -- 172.21.15.19:0/532572153 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f3bc808b680 con 0x7f3be00d02a0 2024-03-28T13:16:31.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.313+0000 7f3bdea00640 1 --2- 172.21.15.19:0/532572153 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3bbc06c530 0x7f3bbc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:31.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.314+0000 7f3bdea00640 1 --2- 172.21.15.19:0/532572153 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3bbc06c530 0x7f3bbc06e9f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0036000 tx=0x7f3bd003b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:31.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.317+0000 7f3bd7e00640 1 -- 172.21.15.19:0/532572153 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3bc8059c00 con 0x7f3be00d02a0 2024-03-28T13:16:31.498 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-03-28T13:16:31.498+0000 7fb54e600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:16:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.517+0000 7f3bdfe00640 1 -- 172.21.15.19:0/532572153 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3be0138590 con 0x7f3bbc06c530 2024-03-28T13:16:31.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.547+0000 7f3bd7e00640 1 -- 172.21.15.19:0/532572153 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252083 (secure 0 0 0) 0x7f3be0138590 con 0x7f3bbc06c530 2024-03-28T13:16:31.551 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:31.553 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:16:31.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.553+0000 7f3bdfe00640 1 -- 172.21.15.19:0/532572153 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3bbc06c530 msgr2=0x7f3bbc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:31.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.553+0000 7f3bdfe00640 1 --2- 172.21.15.19:0/532572153 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3bbc06c530 0x7f3bbc06e9f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0036000 tx=0x7f3bd003b040 comp rx=0 tx=0).stop 2024-03-28T13:16:31.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.553+0000 7f3bdfe00640 1 -- 172.21.15.19:0/532572153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be00d02a0 msgr2=0x7f3be0136ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:31.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.553+0000 7f3bdfe00640 1 --2- 172.21.15.19:0/532572153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be00d02a0 0x7f3be0136ea0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800c430 tx=0x7f3bc800c8f0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.553+0000 7f3bdfe00640 1 -- 172.21.15.19:0/532572153 shutdown_connections 2024-03-28T13:16:31.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.553+0000 7f3bdfe00640 1 --2- 172.21.15.19:0/532572153 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3bbc06c530 0x7f3bbc06e9f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.553+0000 7f3bdfe00640 1 --2- 172.21.15.19:0/532572153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3be01442c0 0x7f3be01373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.553+0000 7f3bdfe00640 1 --2- 172.21.15.19:0/532572153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3be00d02a0 0x7f3be0136ea0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.553+0000 7f3bdfe00640 1 --2- 172.21.15.19:0/532572153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3be00cefe0 0x7f3be015acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.553+0000 7f3bdfe00640 1 -- 172.21.15.19:0/532572153 >> 172.21.15.19:0/532572153 conn(0x7f3be00ccad0 msgr2=0x7f3be00cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:31.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.553+0000 7f3bdfe00640 1 -- 172.21.15.19:0/532572153 shutdown_connections 2024-03-28T13:16:31.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.553+0000 7f3bdfe00640 1 -- 172.21.15.19:0/532572153 wait complete. 2024-03-28T13:16:31.573 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-03-28T13:16:30.137303+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259420,"kb_used_data":9948,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715300,"statfs":{"total":773094113280,"available":772828467200,"internally_reserved":0,"allocated":10186752,"data_stored":5021464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.628498"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.950026+0000","last_change":"2024-03-28T13:16:26.571803+0000","last_active":"2024-03-28T13:16:27.950026+0000","last_peered":"2024-03-28T13:16:27.950026+0000","last_clean":"2024-03-28T13:16:27.950026+0000","last_became_active":"2024-03-28T13:16:26.570162+0000","last_became_peered":"2024-03-28T13:16:26.570162+0000","last_unstale":"2024-03-28T13:16:27.950026+0000","last_undegraded":"2024-03-28T13:16:27.950026+0000","last_fullsized":"2024-03-28T13:16:27.950026+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:29.218444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.599278+0000","last_change":"2024-03-28T13:16:26.576354+0000","last_active":"2024-03-28T13:16:27.599278+0000","last_peered":"2024-03-28T13:16:27.599278+0000","last_clean":"2024-03-28T13:16:27.599278+0000","last_became_active":"2024-03-28T13:16:26.576026+0000","last_became_peered":"2024-03-28T13:16:26.576026+0000","last_unstale":"2024-03-28T13:16:27.599278+0000","last_undegraded":"2024-03-28T13:16:27.599278+0000","last_fullsized":"2024-03-28T13:16:27.599278+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:33:31.282726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948872+0000","last_change":"2024-03-28T13:16:26.574621+0000","last_active":"2024-03-28T13:16:27.948872+0000","last_peered":"2024-03-28T13:16:27.948872+0000","last_clean":"2024-03-28T13:16:27.948872+0000","last_became_active":"2024-03-28T13:16:26.574325+0000","last_became_peered":"2024-03-28T13:16:26.574325+0000","last_unstale":"2024-03-28T13:16:27.948872+0000","last_undegraded":"2024-03-28T13:16:27.948872+0000","last_fullsized":"2024-03-28T13:16:27.948872+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:04:13.711061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948508+0000","last_change":"2024-03-28T13:16:26.570230+0000","last_active":"2024-03-28T13:16:27.948508+0000","last_peered":"2024-03-28T13:16:27.948508+0000","last_clean":"2024-03-28T13:16:27.948508+0000","last_became_active":"2024-03-28T13:16:26.568040+0000","last_became_peered":"2024-03-28T13:16:26.568040+0000","last_unstale":"2024-03-28T13:16:27.948508+0000","last_undegraded":"2024-03-28T13:16:27.948508+0000","last_fullsized":"2024-03-28T13:16:27.948508+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:53:23.853768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.542537+0000","last_change":"2024-03-28T13:16:26.566391+0000","last_active":"2024-03-28T13:16:27.542537+0000","last_peered":"2024-03-28T13:16:27.542537+0000","last_clean":"2024-03-28T13:16:27.542537+0000","last_became_active":"2024-03-28T13:16:26.566034+0000","last_became_peered":"2024-03-28T13:16:26.566034+0000","last_unstale":"2024-03-28T13:16:27.542537+0000","last_undegraded":"2024-03-28T13:16:27.542537+0000","last_fullsized":"2024-03-28T13:16:27.542537+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:26:17.121630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.949953+0000","last_change":"2024-03-28T13:16:26.565127+0000","last_active":"2024-03-28T13:16:27.949953+0000","last_peered":"2024-03-28T13:16:27.949953+0000","last_clean":"2024-03-28T13:16:27.949953+0000","last_became_active":"2024-03-28T13:16:26.564824+0000","last_became_peered":"2024-03-28T13:16:26.564824+0000","last_unstale":"2024-03-28T13:16:27.949953+0000","last_undegraded":"2024-03-28T13:16:27.949953+0000","last_fullsized":"2024-03-28T13:16:27.949953+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:09:08.501208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.599029+0000","last_change":"2024-03-28T13:16:26.571305+0000","last_active":"2024-03-28T13:16:27.599029+0000","last_peered":"2024-03-28T13:16:27.599029+0000","last_clean":"2024-03-28T13:16:27.599029+0000","last_became_active":"2024-03-28T13:16:26.570983+0000","last_became_peered":"2024-03-28T13:16:26.570983+0000","last_unstale":"2024-03-28T13:16:27.599029+0000","last_undegraded":"2024-03-28T13:16:27.599029+0000","last_fullsized":"2024-03-28T13:16:27.599029+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:05.539379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.538699+0000","last_change":"2024-03-28T13:16:26.582763+0000","last_active":"2024-03-28T13:16:27.538699+0000","last_peered":"2024-03-28T13:16:27.538699+0000","last_clean":"2024-03-28T13:16:27.538699+0000","last_became_active":"2024-03-28T13:16:26.582439+0000","last_became_peered":"2024-03-28T13:16:26.582439+0000","last_unstale":"2024-03-28T13:16:27.538699+0000","last_undegraded":"2024-03-28T13:16:27.538699+0000","last_fullsized":"2024-03-28T13:16:27.538699+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:46:29.376263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.950157+0000","last_change":"2024-03-28T13:16:26.570657+0000","last_active":"2024-03-28T13:16:27.950157+0000","last_peered":"2024-03-28T13:16:27.950157+0000","last_clean":"2024-03-28T13:16:27.950157+0000","last_became_active":"2024-03-28T13:16:26.570377+0000","last_became_peered":"2024-03-28T13:16:26.570377+0000","last_unstale":"2024-03-28T13:16:27.950157+0000","last_undegraded":"2024-03-28T13:16:27.950157+0000","last_fullsized":"2024-03-28T13:16:27.950157+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:17:03.381917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948390+0000","last_change":"2024-03-28T13:16:26.576814+0000","last_active":"2024-03-28T13:16:27.948390+0000","last_peered":"2024-03-28T13:16:27.948390+0000","last_clean":"2024-03-28T13:16:27.948390+0000","last_became_active":"2024-03-28T13:16:26.576506+0000","last_became_peered":"2024-03-28T13:16:26.576506+0000","last_unstale":"2024-03-28T13:16:27.948390+0000","last_undegraded":"2024-03-28T13:16:27.948390+0000","last_fullsized":"2024-03-28T13:16:27.948390+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:16:55.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594819+0000","last_change":"2024-03-28T13:16:26.585084+0000","last_active":"2024-03-28T13:16:27.594819+0000","last_peered":"2024-03-28T13:16:27.594819+0000","last_clean":"2024-03-28T13:16:27.594819+0000","last_became_active":"2024-03-28T13:16:26.584816+0000","last_became_peered":"2024-03-28T13:16:26.584816+0000","last_unstale":"2024-03-28T13:16:27.594819+0000","last_undegraded":"2024-03-28T13:16:27.594819+0000","last_fullsized":"2024-03-28T13:16:27.594819+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:16:55.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.538930+0000","last_change":"2024-03-28T13:16:26.572546+0000","last_active":"2024-03-28T13:16:27.538930+0000","last_peered":"2024-03-28T13:16:27.538930+0000","last_clean":"2024-03-28T13:16:27.538930+0000","last_became_active":"2024-03-28T13:16:26.572254+0000","last_became_peered":"2024-03-28T13:16:26.572254+0000","last_unstale":"2024-03-28T13:16:27.538930+0000","last_undegraded":"2024-03-28T13:16:27.538930+0000","last_fullsized":"2024-03-28T13:16:27.538930+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:16:55.974851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.542674+0000","last_change":"2024-03-28T13:16:26.566352+0000","last_active":"2024-03-28T13:16:27.542674+0000","last_peered":"2024-03-28T13:16:27.542674+0000","last_clean":"2024-03-28T13:16:27.542674+0000","last_became_active":"2024-03-28T13:16:26.565936+0000","last_became_peered":"2024-03-28T13:16:26.565936+0000","last_unstale":"2024-03-28T13:16:27.542674+0000","last_undegraded":"2024-03-28T13:16:27.542674+0000","last_fullsized":"2024-03-28T13:16:27.542674+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:13:26.841255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543661+0000","last_change":"2024-03-28T13:16:26.574959+0000","last_active":"2024-03-28T13:16:27.543661+0000","last_peered":"2024-03-28T13:16:27.543661+0000","last_clean":"2024-03-28T13:16:27.543661+0000","last_became_active":"2024-03-28T13:16:26.573832+0000","last_became_peered":"2024-03-28T13:16:26.573832+0000","last_unstale":"2024-03-28T13:16:27.543661+0000","last_undegraded":"2024-03-28T13:16:27.543661+0000","last_fullsized":"2024-03-28T13:16:27.543661+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:21:31.251085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.599204+0000","last_change":"2024-03-28T13:16:26.573795+0000","last_active":"2024-03-28T13:16:27.599204+0000","last_peered":"2024-03-28T13:16:27.599204+0000","last_clean":"2024-03-28T13:16:27.599204+0000","last_became_active":"2024-03-28T13:16:26.573510+0000","last_became_peered":"2024-03-28T13:16:26.573510+0000","last_unstale":"2024-03-28T13:16:27.599204+0000","last_undegraded":"2024-03-28T13:16:27.599204+0000","last_fullsized":"2024-03-28T13:16:27.599204+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:28:57.681141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.947933+0000","last_change":"2024-03-28T13:16:26.560865+0000","last_active":"2024-03-28T13:16:27.947933+0000","last_peered":"2024-03-28T13:16:27.947933+0000","last_clean":"2024-03-28T13:16:27.947933+0000","last_became_active":"2024-03-28T13:16:26.560549+0000","last_became_peered":"2024-03-28T13:16:26.560549+0000","last_unstale":"2024-03-28T13:16:27.947933+0000","last_undegraded":"2024-03-28T13:16:27.947933+0000","last_fullsized":"2024-03-28T13:16:27.947933+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:03.653829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.539630+0000","last_change":"2024-03-28T13:16:26.574844+0000","last_active":"2024-03-28T13:16:27.539630+0000","last_peered":"2024-03-28T13:16:27.539630+0000","last_clean":"2024-03-28T13:16:27.539630+0000","last_became_active":"2024-03-28T13:16:26.571253+0000","last_became_peered":"2024-03-28T13:16:26.571253+0000","last_unstale":"2024-03-28T13:16:27.539630+0000","last_undegraded":"2024-03-28T13:16:27.539630+0000","last_fullsized":"2024-03-28T13:16:27.539630+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:52:29.017485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594259+0000","last_change":"2024-03-28T13:16:26.574039+0000","last_active":"2024-03-28T13:16:27.594259+0000","last_peered":"2024-03-28T13:16:27.594259+0000","last_clean":"2024-03-28T13:16:27.594259+0000","last_became_active":"2024-03-28T13:16:26.573698+0000","last_became_peered":"2024-03-28T13:16:26.573698+0000","last_unstale":"2024-03-28T13:16:27.594259+0000","last_undegraded":"2024-03-28T13:16:27.594259+0000","last_fullsized":"2024-03-28T13:16:27.594259+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:18:32.509652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.544973+0000","last_change":"2024-03-28T13:16:26.573914+0000","last_active":"2024-03-28T13:16:27.544973+0000","last_peered":"2024-03-28T13:16:27.544973+0000","last_clean":"2024-03-28T13:16:27.544973+0000","last_became_active":"2024-03-28T13:16:26.573587+0000","last_became_peered":"2024-03-28T13:16:26.573587+0000","last_unstale":"2024-03-28T13:16:27.544973+0000","last_undegraded":"2024-03-28T13:16:27.544973+0000","last_fullsized":"2024-03-28T13:16:27.544973+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:04:32.980887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.545063+0000","last_change":"2024-03-28T13:16:26.576207+0000","last_active":"2024-03-28T13:16:27.545063+0000","last_peered":"2024-03-28T13:16:27.545063+0000","last_clean":"2024-03-28T13:16:27.545063+0000","last_became_active":"2024-03-28T13:16:26.574161+0000","last_became_peered":"2024-03-28T13:16:26.574161+0000","last_unstale":"2024-03-28T13:16:27.545063+0000","last_undegraded":"2024-03-28T13:16:27.545063+0000","last_fullsized":"2024-03-28T13:16:27.545063+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:14:39.835203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948483+0000","last_change":"2024-03-28T13:16:26.571202+0000","last_active":"2024-03-28T13:16:27.948483+0000","last_peered":"2024-03-28T13:16:27.948483+0000","last_clean":"2024-03-28T13:16:27.948483+0000","last_became_active":"2024-03-28T13:16:26.570576+0000","last_became_peered":"2024-03-28T13:16:26.570576+0000","last_unstale":"2024-03-28T13:16:27.948483+0000","last_undegraded":"2024-03-28T13:16:27.948483+0000","last_fullsized":"2024-03-28T13:16:27.948483+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:55:08.680969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.544177+0000","last_change":"2024-03-28T13:16:26.575971+0000","last_active":"2024-03-28T13:16:27.544177+0000","last_peered":"2024-03-28T13:16:27.544177+0000","last_clean":"2024-03-28T13:16:27.544177+0000","last_became_active":"2024-03-28T13:16:26.575033+0000","last_became_peered":"2024-03-28T13:16:26.575033+0000","last_unstale":"2024-03-28T13:16:27.544177+0000","last_undegraded":"2024-03-28T13:16:27.544177+0000","last_fullsized":"2024-03-28T13:16:27.544177+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:44:58.961862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.538835+0000","last_change":"2024-03-28T13:16:26.572925+0000","last_active":"2024-03-28T13:16:27.538835+0000","last_peered":"2024-03-28T13:16:27.538835+0000","last_clean":"2024-03-28T13:16:27.538835+0000","last_became_active":"2024-03-28T13:16:26.572639+0000","last_became_peered":"2024-03-28T13:16:26.572639+0000","last_unstale":"2024-03-28T13:16:27.538835+0000","last_undegraded":"2024-03-28T13:16:27.538835+0000","last_fullsized":"2024-03-28T13:16:27.538835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:03.653829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.593878+0000","last_change":"2024-03-28T13:16:26.581047+0000","last_active":"2024-03-28T13:16:27.593878+0000","last_peered":"2024-03-28T13:16:27.593878+0000","last_clean":"2024-03-28T13:16:27.593878+0000","last_became_active":"2024-03-28T13:16:26.580721+0000","last_became_peered":"2024-03-28T13:16:26.580721+0000","last_unstale":"2024-03-28T13:16:27.593878+0000","last_undegraded":"2024-03-28T13:16:27.593878+0000","last_fullsized":"2024-03-28T13:16:27.593878+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:20.870581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594177+0000","last_change":"2024-03-28T13:16:26.578020+0000","last_active":"2024-03-28T13:16:27.594177+0000","last_peered":"2024-03-28T13:16:27.594177+0000","last_clean":"2024-03-28T13:16:27.594177+0000","last_became_active":"2024-03-28T13:16:26.577134+0000","last_became_peered":"2024-03-28T13:16:26.577134+0000","last_unstale":"2024-03-28T13:16:27.594177+0000","last_undegraded":"2024-03-28T13:16:27.594177+0000","last_fullsized":"2024-03-28T13:16:27.594177+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:46:29.376263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543203+0000","last_change":"2024-03-28T13:16:26.576452+0000","last_active":"2024-03-28T13:16:27.543203+0000","last_peered":"2024-03-28T13:16:27.543203+0000","last_clean":"2024-03-28T13:16:27.543203+0000","last_became_active":"2024-03-28T13:16:26.576110+0000","last_became_peered":"2024-03-28T13:16:26.576110+0000","last_unstale":"2024-03-28T13:16:27.543203+0000","last_undegraded":"2024-03-28T13:16:27.543203+0000","last_fullsized":"2024-03-28T13:16:27.543203+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:14:46.317120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543028+0000","last_change":"2024-03-28T13:16:26.553909+0000","last_active":"2024-03-28T13:16:27.543028+0000","last_peered":"2024-03-28T13:16:27.543028+0000","last_clean":"2024-03-28T13:16:27.543028+0000","last_became_active":"2024-03-28T13:16:26.553205+0000","last_became_peered":"2024-03-28T13:16:26.553205+0000","last_unstale":"2024-03-28T13:16:27.543028+0000","last_undegraded":"2024-03-28T13:16:27.543028+0000","last_fullsized":"2024-03-28T13:16:27.543028+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:35:13.128575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.949005+0000","last_change":"2024-03-28T13:16:26.580587+0000","last_active":"2024-03-28T13:16:27.949005+0000","last_peered":"2024-03-28T13:16:27.949005+0000","last_clean":"2024-03-28T13:16:27.949005+0000","last_became_active":"2024-03-28T13:16:26.580280+0000","last_became_peered":"2024-03-28T13:16:26.580280+0000","last_unstale":"2024-03-28T13:16:27.949005+0000","last_undegraded":"2024-03-28T13:16:27.949005+0000","last_fullsized":"2024-03-28T13:16:27.949005+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:51:33.798003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948523+0000","last_change":"2024-03-28T13:16:26.563986+0000","last_active":"2024-03-28T13:16:27.948523+0000","last_peered":"2024-03-28T13:16:27.948523+0000","last_clean":"2024-03-28T13:16:27.948523+0000","last_became_active":"2024-03-28T13:16:26.561868+0000","last_became_peered":"2024-03-28T13:16:26.561868+0000","last_unstale":"2024-03-28T13:16:27.948523+0000","last_undegraded":"2024-03-28T13:16:27.948523+0000","last_fullsized":"2024-03-28T13:16:27.948523+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:37:01.791093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.544412+0000","last_change":"2024-03-28T13:16:26.575224+0000","last_active":"2024-03-28T13:16:27.544412+0000","last_peered":"2024-03-28T13:16:27.544412+0000","last_clean":"2024-03-28T13:16:27.544412+0000","last_became_active":"2024-03-28T13:16:26.574702+0000","last_became_peered":"2024-03-28T13:16:26.574702+0000","last_unstale":"2024-03-28T13:16:27.544412+0000","last_undegraded":"2024-03-28T13:16:27.544412+0000","last_fullsized":"2024-03-28T13:16:27.544412+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:05:48.692843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948582+0000","last_change":"2024-03-28T13:16:26.552978+0000","last_active":"2024-03-28T13:16:27.948582+0000","last_peered":"2024-03-28T13:16:27.948582+0000","last_clean":"2024-03-28T13:16:27.948582+0000","last_became_active":"2024-03-28T13:16:26.552424+0000","last_became_peered":"2024-03-28T13:16:26.552424+0000","last_unstale":"2024-03-28T13:16:27.948582+0000","last_undegraded":"2024-03-28T13:16:27.948582+0000","last_fullsized":"2024-03-28T13:16:27.948582+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:25:59.100715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948091+0000","last_change":"2024-03-28T13:16:26.565426+0000","last_active":"2024-03-28T13:16:27.948091+0000","last_peered":"2024-03-28T13:16:27.948091+0000","last_clean":"2024-03-28T13:16:27.948091+0000","last_became_active":"2024-03-28T13:16:26.564687+0000","last_became_peered":"2024-03-28T13:16:26.564687+0000","last_unstale":"2024-03-28T13:16:27.948091+0000","last_undegraded":"2024-03-28T13:16:27.948091+0000","last_fullsized":"2024-03-28T13:16:27.948091+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:18:32.509652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.539594+0000","last_change":"2024-03-28T13:16:26.577450+0000","last_active":"2024-03-28T13:16:27.539594+0000","last_peered":"2024-03-28T13:16:27.539594+0000","last_clean":"2024-03-28T13:16:27.539594+0000","last_became_active":"2024-03-28T13:16:26.572196+0000","last_became_peered":"2024-03-28T13:16:26.572196+0000","last_unstale":"2024-03-28T13:16:27.539594+0000","last_undegraded":"2024-03-28T13:16:27.539594+0000","last_fullsized":"2024-03-28T13:16:27.539594+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:58.786409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"91'4","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543122+0000","last_change":"2024-03-28T13:16:25.550901+0000","last_active":"2024-03-28T13:16:27.543122+0000","last_peered":"2024-03-28T13:16:27.543122+0000","last_clean":"2024-03-28T13:16:27.543122+0000","last_became_active":"2024-03-28T13:16:25.550246+0000","last_became_peered":"2024-03-28T13:16:25.550246+0000","last_unstale":"2024-03-28T13:16:27.543122+0000","last_undegraded":"2024-03-28T13:16:27.543122+0000","last_fullsized":"2024-03-28T13:16:27.543122+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:01:03.775777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.950653+0000","last_change":"2024-03-28T13:16:26.568503+0000","last_active":"2024-03-28T13:16:27.950653+0000","last_peered":"2024-03-28T13:16:27.950653+0000","last_clean":"2024-03-28T13:16:27.950653+0000","last_became_active":"2024-03-28T13:16:26.568189+0000","last_became_peered":"2024-03-28T13:16:26.568189+0000","last_unstale":"2024-03-28T13:16:27.950653+0000","last_undegraded":"2024-03-28T13:16:27.950653+0000","last_fullsized":"2024-03-28T13:16:27.950653+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:17:08.352742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948445+0000","last_change":"2024-03-28T13:16:25.554343+0000","last_active":"2024-03-28T13:16:27.948445+0000","last_peered":"2024-03-28T13:16:27.948445+0000","last_clean":"2024-03-28T13:16:27.948445+0000","last_became_active":"2024-03-28T13:16:25.554012+0000","last_became_peered":"2024-03-28T13:16:25.554012+0000","last_unstale":"2024-03-28T13:16:27.948445+0000","last_undegraded":"2024-03-28T13:16:27.948445+0000","last_fullsized":"2024-03-28T13:16:27.948445+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:04.525529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.539056+0000","last_change":"2024-03-28T13:16:26.574653+0000","last_active":"2024-03-28T13:16:27.539056+0000","last_peered":"2024-03-28T13:16:27.539056+0000","last_clean":"2024-03-28T13:16:27.539056+0000","last_became_active":"2024-03-28T13:16:26.574373+0000","last_became_peered":"2024-03-28T13:16:26.574373+0000","last_unstale":"2024-03-28T13:16:27.539056+0000","last_undegraded":"2024-03-28T13:16:27.539056+0000","last_fullsized":"2024-03-28T13:16:27.539056+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:25:59.100715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594711+0000","last_change":"2024-03-28T13:16:25.544843+0000","last_active":"2024-03-28T13:16:27.594711+0000","last_peered":"2024-03-28T13:16:27.594711+0000","last_clean":"2024-03-28T13:16:27.594711+0000","last_became_active":"2024-03-28T13:16:25.544565+0000","last_became_peered":"2024-03-28T13:16:25.544565+0000","last_unstale":"2024-03-28T13:16:27.594711+0000","last_undegraded":"2024-03-28T13:16:27.594711+0000","last_fullsized":"2024-03-28T13:16:27.594711+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:10:59.152597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948971+0000","last_change":"2024-03-28T13:16:26.588223+0000","last_active":"2024-03-28T13:16:27.948971+0000","last_peered":"2024-03-28T13:16:27.948971+0000","last_clean":"2024-03-28T13:16:27.948971+0000","last_became_active":"2024-03-28T13:16:26.587934+0000","last_became_peered":"2024-03-28T13:16:26.587934+0000","last_unstale":"2024-03-28T13:16:27.948971+0000","last_undegraded":"2024-03-28T13:16:27.948971+0000","last_fullsized":"2024-03-28T13:16:27.948971+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:33:31.282726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948520+0000","last_change":"2024-03-28T13:16:25.544704+0000","last_active":"2024-03-28T13:16:27.948520+0000","last_peered":"2024-03-28T13:16:27.948520+0000","last_clean":"2024-03-28T13:16:27.948520+0000","last_became_active":"2024-03-28T13:16:25.544332+0000","last_became_peered":"2024-03-28T13:16:25.544332+0000","last_unstale":"2024-03-28T13:16:27.948520+0000","last_undegraded":"2024-03-28T13:16:27.948520+0000","last_fullsized":"2024-03-28T13:16:27.948520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:39:12.185447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948683+0000","last_change":"2024-03-28T13:16:26.579300+0000","last_active":"2024-03-28T13:16:27.948683+0000","last_peered":"2024-03-28T13:16:27.948683+0000","last_clean":"2024-03-28T13:16:27.948683+0000","last_became_active":"2024-03-28T13:16:26.578964+0000","last_became_peered":"2024-03-28T13:16:26.578964+0000","last_unstale":"2024-03-28T13:16:27.948683+0000","last_undegraded":"2024-03-28T13:16:27.948683+0000","last_fullsized":"2024-03-28T13:16:27.948683+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:52:18.845220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.538937+0000","last_change":"2024-03-28T13:16:25.535115+0000","last_active":"2024-03-28T13:16:27.538937+0000","last_peered":"2024-03-28T13:16:27.538937+0000","last_clean":"2024-03-28T13:16:27.538937+0000","last_became_active":"2024-03-28T13:16:25.534798+0000","last_became_peered":"2024-03-28T13:16:25.534798+0000","last_unstale":"2024-03-28T13:16:27.538937+0000","last_undegraded":"2024-03-28T13:16:27.538937+0000","last_fullsized":"2024-03-28T13:16:27.538937+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:13:34.656708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.949939+0000","last_change":"2024-03-28T13:16:26.575847+0000","last_active":"2024-03-28T13:16:27.949939+0000","last_peered":"2024-03-28T13:16:27.949939+0000","last_clean":"2024-03-28T13:16:27.949939+0000","last_became_active":"2024-03-28T13:16:26.575565+0000","last_became_peered":"2024-03-28T13:16:26.575565+0000","last_unstale":"2024-03-28T13:16:27.949939+0000","last_undegraded":"2024-03-28T13:16:27.949939+0000","last_fullsized":"2024-03-28T13:16:27.949939+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:39:13.199297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.950172+0000","last_change":"2024-03-28T13:16:25.546604+0000","last_active":"2024-03-28T13:16:27.950172+0000","last_peered":"2024-03-28T13:16:27.950172+0000","last_clean":"2024-03-28T13:16:27.950172+0000","last_became_active":"2024-03-28T13:16:25.546322+0000","last_became_peered":"2024-03-28T13:16:25.546322+0000","last_unstale":"2024-03-28T13:16:27.950172+0000","last_undegraded":"2024-03-28T13:16:27.950172+0000","last_fullsized":"2024-03-28T13:16:27.950172+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:13:34.656708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.947500+0000","last_change":"2024-03-28T13:16:26.582773+0000","last_active":"2024-03-28T13:16:27.947500+0000","last_peered":"2024-03-28T13:16:27.947500+0000","last_clean":"2024-03-28T13:16:27.947500+0000","last_became_active":"2024-03-28T13:16:26.582460+0000","last_became_peered":"2024-03-28T13:16:26.582460+0000","last_unstale":"2024-03-28T13:16:27.947500+0000","last_undegraded":"2024-03-28T13:16:27.947500+0000","last_fullsized":"2024-03-28T13:16:27.947500+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:13:35.670558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543799+0000","last_change":"2024-03-28T13:16:25.559007+0000","last_active":"2024-03-28T13:16:27.543799+0000","last_peered":"2024-03-28T13:16:27.543799+0000","last_clean":"2024-03-28T13:16:27.543799+0000","last_became_active":"2024-03-28T13:16:25.558718+0000","last_became_peered":"2024-03-28T13:16:25.558718+0000","last_unstale":"2024-03-28T13:16:27.543799+0000","last_undegraded":"2024-03-28T13:16:27.543799+0000","last_fullsized":"2024-03-28T13:16:27.543799+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:59:30.655785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543852+0000","last_change":"2024-03-28T13:16:26.570137+0000","last_active":"2024-03-28T13:16:27.543852+0000","last_peered":"2024-03-28T13:16:27.543852+0000","last_clean":"2024-03-28T13:16:27.543852+0000","last_became_active":"2024-03-28T13:16:26.569880+0000","last_became_peered":"2024-03-28T13:16:26.569880+0000","last_unstale":"2024-03-28T13:16:27.543852+0000","last_undegraded":"2024-03-28T13:16:27.543852+0000","last_fullsized":"2024-03-28T13:16:27.543852+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:42:51.863499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543541+0000","last_change":"2024-03-28T13:16:25.554767+0000","last_active":"2024-03-28T13:16:27.543541+0000","last_peered":"2024-03-28T13:16:27.543541+0000","last_clean":"2024-03-28T13:16:27.543541+0000","last_became_active":"2024-03-28T13:16:25.554244+0000","last_became_peered":"2024-03-28T13:16:25.554244+0000","last_unstale":"2024-03-28T13:16:27.543541+0000","last_undegraded":"2024-03-28T13:16:27.543541+0000","last_fullsized":"2024-03-28T13:16:27.543541+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:52:20.106923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.545512+0000","last_change":"2024-03-28T13:16:26.569168+0000","last_active":"2024-03-28T13:16:27.545512+0000","last_peered":"2024-03-28T13:16:27.545512+0000","last_clean":"2024-03-28T13:16:27.545512+0000","last_became_active":"2024-03-28T13:16:26.568771+0000","last_became_peered":"2024-03-28T13:16:26.568771+0000","last_unstale":"2024-03-28T13:16:27.545512+0000","last_undegraded":"2024-03-28T13:16:27.545512+0000","last_fullsized":"2024-03-28T13:16:27.545512+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:26:29.748821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.950731+0000","last_change":"2024-03-28T13:16:25.541863+0000","last_active":"2024-03-28T13:16:27.950731+0000","last_peered":"2024-03-28T13:16:27.950731+0000","last_clean":"2024-03-28T13:16:27.950731+0000","last_became_active":"2024-03-28T13:16:25.541381+0000","last_became_peered":"2024-03-28T13:16:25.541381+0000","last_unstale":"2024-03-28T13:16:27.950731+0000","last_undegraded":"2024-03-28T13:16:27.950731+0000","last_fullsized":"2024-03-28T13:16:27.950731+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:02:05.693865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.950704+0000","last_change":"2024-03-28T13:16:26.581749+0000","last_active":"2024-03-28T13:16:27.950704+0000","last_peered":"2024-03-28T13:16:27.950704+0000","last_clean":"2024-03-28T13:16:27.950704+0000","last_became_active":"2024-03-28T13:16:26.581452+0000","last_became_peered":"2024-03-28T13:16:26.581452+0000","last_unstale":"2024-03-28T13:16:27.950704+0000","last_undegraded":"2024-03-28T13:16:27.950704+0000","last_fullsized":"2024-03-28T13:16:27.950704+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:14:27.227998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948555+0000","last_change":"2024-03-28T13:16:25.541016+0000","last_active":"2024-03-28T13:16:27.948555+0000","last_peered":"2024-03-28T13:16:27.948555+0000","last_clean":"2024-03-28T13:16:27.948555+0000","last_became_active":"2024-03-28T13:16:25.540685+0000","last_became_peered":"2024-03-28T13:16:25.540685+0000","last_unstale":"2024-03-28T13:16:27.948555+0000","last_undegraded":"2024-03-28T13:16:27.948555+0000","last_fullsized":"2024-03-28T13:16:27.948555+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:33:30.268877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594109+0000","last_change":"2024-03-28T13:16:26.579141+0000","last_active":"2024-03-28T13:16:27.594109+0000","last_peered":"2024-03-28T13:16:27.594109+0000","last_clean":"2024-03-28T13:16:27.594109+0000","last_became_active":"2024-03-28T13:16:26.578791+0000","last_became_peered":"2024-03-28T13:16:26.578791+0000","last_unstale":"2024-03-28T13:16:27.594109+0000","last_undegraded":"2024-03-28T13:16:27.594109+0000","last_fullsized":"2024-03-28T13:16:27.594109+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:13:38.633845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"91'3","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.949116+0000","last_change":"2024-03-28T13:16:25.550301+0000","last_active":"2024-03-28T13:16:27.949116+0000","last_peered":"2024-03-28T13:16:27.949116+0000","last_clean":"2024-03-28T13:16:27.949116+0000","last_became_active":"2024-03-28T13:16:25.549994+0000","last_became_peered":"2024-03-28T13:16:25.549994+0000","last_unstale":"2024-03-28T13:16:27.949116+0000","last_undegraded":"2024-03-28T13:16:27.949116+0000","last_fullsized":"2024-03-28T13:16:27.949116+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:06:19.939323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.542963+0000","last_change":"2024-03-28T13:16:26.585091+0000","last_active":"2024-03-28T13:16:27.542963+0000","last_peered":"2024-03-28T13:16:27.542963+0000","last_clean":"2024-03-28T13:16:27.542963+0000","last_became_active":"2024-03-28T13:16:26.584637+0000","last_became_peered":"2024-03-28T13:16:26.584637+0000","last_unstale":"2024-03-28T13:16:27.542963+0000","last_undegraded":"2024-03-28T13:16:27.542963+0000","last_fullsized":"2024-03-28T13:16:27.542963+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:31:43.159136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"91'6","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.539158+0000","last_change":"2024-03-28T13:16:25.533821+0000","last_active":"2024-03-28T13:16:27.539158+0000","last_peered":"2024-03-28T13:16:27.539158+0000","last_clean":"2024-03-28T13:16:27.539158+0000","last_became_active":"2024-03-28T13:16:25.533446+0000","last_became_peered":"2024-03-28T13:16:25.533446+0000","last_unstale":"2024-03-28T13:16:27.539158+0000","last_undegraded":"2024-03-28T13:16:27.539158+0000","last_fullsized":"2024-03-28T13:16:27.539158+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:25:07.854479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.593727+0000","last_change":"2024-03-28T13:16:26.575893+0000","last_active":"2024-03-28T13:16:27.593727+0000","last_peered":"2024-03-28T13:16:27.593727+0000","last_clean":"2024-03-28T13:16:27.593727+0000","last_became_active":"2024-03-28T13:16:26.575451+0000","last_became_peered":"2024-03-28T13:16:26.575451+0000","last_unstale":"2024-03-28T13:16:27.593727+0000","last_undegraded":"2024-03-28T13:16:27.593727+0000","last_fullsized":"2024-03-28T13:16:27.593727+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:14:27.227998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948707+0000","last_change":"2024-03-28T13:16:25.549133+0000","last_active":"2024-03-28T13:16:27.948707+0000","last_peered":"2024-03-28T13:16:27.948707+0000","last_clean":"2024-03-28T13:16:27.948707+0000","last_became_active":"2024-03-28T13:16:25.548803+0000","last_became_peered":"2024-03-28T13:16:25.548803+0000","last_unstale":"2024-03-28T13:16:27.948707+0000","last_undegraded":"2024-03-28T13:16:27.948707+0000","last_fullsized":"2024-03-28T13:16:27.948707+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:52:28.003635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948169+0000","last_change":"2024-03-28T13:16:26.585179+0000","last_active":"2024-03-28T13:16:27.948169+0000","last_peered":"2024-03-28T13:16:27.948169+0000","last_clean":"2024-03-28T13:16:27.948169+0000","last_became_active":"2024-03-28T13:16:26.584802+0000","last_became_peered":"2024-03-28T13:16:26.584802+0000","last_unstale":"2024-03-28T13:16:27.948169+0000","last_undegraded":"2024-03-28T13:16:27.948169+0000","last_fullsized":"2024-03-28T13:16:27.948169+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:28:57.681141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.598822+0000","last_change":"2024-03-28T13:16:25.547021+0000","last_active":"2024-03-28T13:16:27.598822+0000","last_peered":"2024-03-28T13:16:27.598822+0000","last_clean":"2024-03-28T13:16:27.598822+0000","last_became_active":"2024-03-28T13:16:25.546734+0000","last_became_peered":"2024-03-28T13:16:25.546734+0000","last_unstale":"2024-03-28T13:16:27.598822+0000","last_undegraded":"2024-03-28T13:16:27.598822+0000","last_fullsized":"2024-03-28T13:16:27.598822+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:02:15.752917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543736+0000","last_change":"2024-03-28T13:16:26.581984+0000","last_active":"2024-03-28T13:16:27.543736+0000","last_peered":"2024-03-28T13:16:27.543736+0000","last_clean":"2024-03-28T13:16:27.543736+0000","last_became_active":"2024-03-28T13:16:26.581697+0000","last_became_peered":"2024-03-28T13:16:26.581697+0000","last_unstale":"2024-03-28T13:16:27.543736+0000","last_undegraded":"2024-03-28T13:16:27.543736+0000","last_fullsized":"2024-03-28T13:16:27.543736+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:38.777163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948724+0000","last_change":"2024-03-28T13:16:25.543304+0000","last_active":"2024-03-28T13:16:27.948724+0000","last_peered":"2024-03-28T13:16:27.948724+0000","last_clean":"2024-03-28T13:16:27.948724+0000","last_became_active":"2024-03-28T13:16:25.542973+0000","last_became_peered":"2024-03-28T13:16:25.542973+0000","last_unstale":"2024-03-28T13:16:27.948724+0000","last_undegraded":"2024-03-28T13:16:27.948724+0000","last_fullsized":"2024-03-28T13:16:27.948724+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:53:31.570839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.593381+0000","last_change":"2024-03-28T13:16:26.573989+0000","last_active":"2024-03-28T13:16:27.593381+0000","last_peered":"2024-03-28T13:16:27.593381+0000","last_clean":"2024-03-28T13:16:27.593381+0000","last_became_active":"2024-03-28T13:16:26.573669+0000","last_became_peered":"2024-03-28T13:16:26.573669+0000","last_unstale":"2024-03-28T13:16:27.593381+0000","last_undegraded":"2024-03-28T13:16:27.593381+0000","last_fullsized":"2024-03-28T13:16:27.593381+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:17:08.352742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594427+0000","last_change":"2024-03-28T13:16:25.549703+0000","last_active":"2024-03-28T13:16:27.594427+0000","last_peered":"2024-03-28T13:16:27.594427+0000","last_clean":"2024-03-28T13:16:27.594427+0000","last_became_active":"2024-03-28T13:16:25.549285+0000","last_became_peered":"2024-03-28T13:16:25.549285+0000","last_unstale":"2024-03-28T13:16:27.594427+0000","last_undegraded":"2024-03-28T13:16:27.594427+0000","last_fullsized":"2024-03-28T13:16:27.594427+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:16:33.411736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.539697+0000","last_change":"2024-03-28T13:16:26.569939+0000","last_active":"2024-03-28T13:16:27.539697+0000","last_peered":"2024-03-28T13:16:27.539697+0000","last_clean":"2024-03-28T13:16:27.539697+0000","last_became_active":"2024-03-28T13:16:26.569621+0000","last_became_peered":"2024-03-28T13:16:26.569621+0000","last_unstale":"2024-03-28T13:16:27.539697+0000","last_undegraded":"2024-03-28T13:16:27.539697+0000","last_fullsized":"2024-03-28T13:16:27.539697+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:39:13.199297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543313+0000","last_change":"2024-03-28T13:16:25.554065+0000","last_active":"2024-03-28T13:16:27.543313+0000","last_peered":"2024-03-28T13:16:27.543313+0000","last_clean":"2024-03-28T13:16:27.543313+0000","last_became_active":"2024-03-28T13:16:25.553751+0000","last_became_peered":"2024-03-28T13:16:25.553751+0000","last_unstale":"2024-03-28T13:16:27.543313+0000","last_undegraded":"2024-03-28T13:16:27.543313+0000","last_fullsized":"2024-03-28T13:16:27.543313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:00:52.836993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.949423+0000","last_change":"2024-03-28T13:16:26.576404+0000","last_active":"2024-03-28T13:16:27.949423+0000","last_peered":"2024-03-28T13:16:27.949423+0000","last_clean":"2024-03-28T13:16:27.949423+0000","last_became_active":"2024-03-28T13:16:26.575956+0000","last_became_peered":"2024-03-28T13:16:26.575956+0000","last_unstale":"2024-03-28T13:16:27.949423+0000","last_undegraded":"2024-03-28T13:16:27.949423+0000","last_fullsized":"2024-03-28T13:16:27.949423+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:52:29.017485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.545603+0000","last_change":"2024-03-28T13:16:25.557497+0000","last_active":"2024-03-28T13:16:27.545603+0000","last_peered":"2024-03-28T13:16:27.545603+0000","last_clean":"2024-03-28T13:16:27.545603+0000","last_became_active":"2024-03-28T13:16:25.557086+0000","last_became_peered":"2024-03-28T13:16:25.557086+0000","last_unstale":"2024-03-28T13:16:27.545603+0000","last_undegraded":"2024-03-28T13:16:27.545603+0000","last_fullsized":"2024-03-28T13:16:27.545603+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:16:42.842395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.539409+0000","last_change":"2024-03-28T13:16:26.570099+0000","last_active":"2024-03-28T13:16:27.539409+0000","last_peered":"2024-03-28T13:16:27.539409+0000","last_clean":"2024-03-28T13:16:27.539409+0000","last_became_active":"2024-03-28T13:16:26.569798+0000","last_became_peered":"2024-03-28T13:16:26.569798+0000","last_unstale":"2024-03-28T13:16:27.539409+0000","last_undegraded":"2024-03-28T13:16:27.539409+0000","last_fullsized":"2024-03-28T13:16:27.539409+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:04:13.711061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.545149+0000","last_change":"2024-03-28T13:16:25.555200+0000","last_active":"2024-03-28T13:16:27.545149+0000","last_peered":"2024-03-28T13:16:27.545149+0000","last_clean":"2024-03-28T13:16:27.545149+0000","last_became_active":"2024-03-28T13:16:25.554835+0000","last_became_peered":"2024-03-28T13:16:25.554835+0000","last_unstale":"2024-03-28T13:16:27.545149+0000","last_undegraded":"2024-03-28T13:16:27.545149+0000","last_fullsized":"2024-03-28T13:16:27.545149+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:52:43.906362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.598820+0000","last_change":"2024-03-28T13:16:26.584849+0000","last_active":"2024-03-28T13:16:27.598820+0000","last_peered":"2024-03-28T13:16:27.598820+0000","last_clean":"2024-03-28T13:16:27.598820+0000","last_became_active":"2024-03-28T13:16:26.584510+0000","last_became_peered":"2024-03-28T13:16:26.584510+0000","last_unstale":"2024-03-28T13:16:27.598820+0000","last_undegraded":"2024-03-28T13:16:27.598820+0000","last_fullsized":"2024-03-28T13:16:27.598820+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:25:08.868328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.544797+0000","last_change":"2024-03-28T13:16:25.555898+0000","last_active":"2024-03-28T13:16:27.544797+0000","last_peered":"2024-03-28T13:16:27.544797+0000","last_clean":"2024-03-28T13:16:27.544797+0000","last_became_active":"2024-03-28T13:16:25.555609+0000","last_became_peered":"2024-03-28T13:16:25.555609+0000","last_unstale":"2024-03-28T13:16:27.544797+0000","last_undegraded":"2024-03-28T13:16:27.544797+0000","last_fullsized":"2024-03-28T13:16:27.544797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:11:15.489678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.542790+0000","last_change":"2024-03-28T13:16:26.578505+0000","last_active":"2024-03-28T13:16:27.542790+0000","last_peered":"2024-03-28T13:16:27.542790+0000","last_clean":"2024-03-28T13:16:27.542790+0000","last_became_active":"2024-03-28T13:16:26.578166+0000","last_became_peered":"2024-03-28T13:16:26.578166+0000","last_unstale":"2024-03-28T13:16:27.542790+0000","last_undegraded":"2024-03-28T13:16:27.542790+0000","last_fullsized":"2024-03-28T13:16:27.542790+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:46:47.614785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.598966+0000","last_change":"2024-03-28T13:16:25.538552+0000","last_active":"2024-03-28T13:16:27.598966+0000","last_peered":"2024-03-28T13:16:27.598966+0000","last_clean":"2024-03-28T13:16:27.598966+0000","last_became_active":"2024-03-28T13:16:25.538187+0000","last_became_peered":"2024-03-28T13:16:25.538187+0000","last_unstale":"2024-03-28T13:16:27.598966+0000","last_undegraded":"2024-03-28T13:16:27.598966+0000","last_fullsized":"2024-03-28T13:16:27.598966+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:55.502492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.947948+0000","last_change":"2024-03-28T13:16:26.577569+0000","last_active":"2024-03-28T13:16:27.947948+0000","last_peered":"2024-03-28T13:16:27.947948+0000","last_clean":"2024-03-28T13:16:27.947948+0000","last_became_active":"2024-03-28T13:16:26.576970+0000","last_became_peered":"2024-03-28T13:16:26.576970+0000","last_unstale":"2024-03-28T13:16:27.947948+0000","last_undegraded":"2024-03-28T13:16:27.947948+0000","last_fullsized":"2024-03-28T13:16:27.947948+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:59:54.712670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594272+0000","last_change":"2024-03-28T13:16:25.547754+0000","last_active":"2024-03-28T13:16:27.594272+0000","last_peered":"2024-03-28T13:16:27.594272+0000","last_clean":"2024-03-28T13:16:27.594272+0000","last_became_active":"2024-03-28T13:16:25.547481+0000","last_became_peered":"2024-03-28T13:16:25.547481+0000","last_unstale":"2024-03-28T13:16:27.594272+0000","last_undegraded":"2024-03-28T13:16:27.594272+0000","last_fullsized":"2024-03-28T13:16:27.594272+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:35:55.502492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.538859+0000","last_change":"2024-03-28T13:16:26.575008+0000","last_active":"2024-03-28T13:16:27.538859+0000","last_peered":"2024-03-28T13:16:27.538859+0000","last_clean":"2024-03-28T13:16:27.538859+0000","last_became_active":"2024-03-28T13:16:26.574434+0000","last_became_peered":"2024-03-28T13:16:26.574434+0000","last_unstale":"2024-03-28T13:16:27.538859+0000","last_undegraded":"2024-03-28T13:16:27.538859+0000","last_fullsized":"2024-03-28T13:16:27.538859+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:18:32.509652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.544005+0000","last_change":"2024-03-28T13:16:25.555119+0000","last_active":"2024-03-28T13:16:27.544005+0000","last_peered":"2024-03-28T13:16:27.544005+0000","last_clean":"2024-03-28T13:16:27.544005+0000","last_became_active":"2024-03-28T13:16:25.554798+0000","last_became_peered":"2024-03-28T13:16:25.554798+0000","last_unstale":"2024-03-28T13:16:27.544005+0000","last_undegraded":"2024-03-28T13:16:27.544005+0000","last_fullsized":"2024-03-28T13:16:27.544005+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:37:12.073635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543258+0000","last_change":"2024-03-28T13:16:26.573241+0000","last_active":"2024-03-28T13:16:27.543258+0000","last_peered":"2024-03-28T13:16:27.543258+0000","last_clean":"2024-03-28T13:16:27.543258+0000","last_became_active":"2024-03-28T13:16:26.570418+0000","last_became_peered":"2024-03-28T13:16:26.570418+0000","last_unstale":"2024-03-28T13:16:27.543258+0000","last_undegraded":"2024-03-28T13:16:27.543258+0000","last_fullsized":"2024-03-28T13:16:27.543258+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:42.354543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"91'2","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.545504+0000","last_change":"2024-03-28T13:16:25.557467+0000","last_active":"2024-03-28T13:16:27.545504+0000","last_peered":"2024-03-28T13:16:27.545504+0000","last_clean":"2024-03-28T13:16:27.545504+0000","last_became_active":"2024-03-28T13:16:25.557053+0000","last_became_peered":"2024-03-28T13:16:25.557053+0000","last_unstale":"2024-03-28T13:16:27.545504+0000","last_undegraded":"2024-03-28T13:16:27.545504+0000","last_fullsized":"2024-03-28T13:16:27.545504+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:03:28.580115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543227+0000","last_change":"2024-03-28T13:16:26.572470+0000","last_active":"2024-03-28T13:16:27.543227+0000","last_peered":"2024-03-28T13:16:27.543227+0000","last_clean":"2024-03-28T13:16:27.543227+0000","last_became_active":"2024-03-28T13:16:26.572130+0000","last_became_peered":"2024-03-28T13:16:26.572130+0000","last_unstale":"2024-03-28T13:16:27.543227+0000","last_undegraded":"2024-03-28T13:16:27.543227+0000","last_fullsized":"2024-03-28T13:16:27.543227+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:25:07.768733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"92'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.597828+0000","last_change":"2024-03-28T13:16:25.552029+0000","last_active":"2024-03-28T13:16:27.597828+0000","last_peered":"2024-03-28T13:16:27.597828+0000","last_clean":"2024-03-28T13:16:27.597828+0000","last_became_active":"2024-03-28T13:16:25.551733+0000","last_became_peered":"2024-03-28T13:16:25.551733+0000","last_unstale":"2024-03-28T13:16:27.597828+0000","last_undegraded":"2024-03-28T13:16:27.597828+0000","last_fullsized":"2024-03-28T13:16:27.597828+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:14:24.831395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.542914+0000","last_change":"2024-03-28T13:16:26.567250+0000","last_active":"2024-03-28T13:16:27.542914+0000","last_peered":"2024-03-28T13:16:27.542914+0000","last_clean":"2024-03-28T13:16:27.542914+0000","last_became_active":"2024-03-28T13:16:26.566906+0000","last_became_peered":"2024-03-28T13:16:26.566906+0000","last_unstale":"2024-03-28T13:16:27.542914+0000","last_undegraded":"2024-03-28T13:16:27.542914+0000","last_fullsized":"2024-03-28T13:16:27.542914+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:11:58.831175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.539266+0000","last_change":"2024-03-28T13:16:25.550476+0000","last_active":"2024-03-28T13:16:27.539266+0000","last_peered":"2024-03-28T13:16:27.539266+0000","last_clean":"2024-03-28T13:16:27.539266+0000","last_became_active":"2024-03-28T13:16:25.549893+0000","last_became_peered":"2024-03-28T13:16:25.549893+0000","last_unstale":"2024-03-28T13:16:27.539266+0000","last_undegraded":"2024-03-28T13:16:27.539266+0000","last_fullsized":"2024-03-28T13:16:27.539266+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:04.525529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.949280+0000","last_change":"2024-03-28T13:16:26.571564+0000","last_active":"2024-03-28T13:16:27.949280+0000","last_peered":"2024-03-28T13:16:27.949280+0000","last_clean":"2024-03-28T13:16:27.949280+0000","last_became_active":"2024-03-28T13:16:26.571224+0000","last_became_peered":"2024-03-28T13:16:26.571224+0000","last_unstale":"2024-03-28T13:16:27.949280+0000","last_undegraded":"2024-03-28T13:16:27.949280+0000","last_fullsized":"2024-03-28T13:16:27.949280+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:58.786409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948365+0000","last_change":"2024-03-28T13:16:25.554236+0000","last_active":"2024-03-28T13:16:27.948365+0000","last_peered":"2024-03-28T13:16:27.948365+0000","last_clean":"2024-03-28T13:16:27.948365+0000","last_became_active":"2024-03-28T13:16:25.553943+0000","last_became_peered":"2024-03-28T13:16:25.553943+0000","last_unstale":"2024-03-28T13:16:27.948365+0000","last_undegraded":"2024-03-28T13:16:27.948365+0000","last_fullsized":"2024-03-28T13:16:27.948365+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:28:56.667291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.948119+0000","last_change":"2024-03-28T13:16:26.576725+0000","last_active":"2024-03-28T13:16:27.948119+0000","last_peered":"2024-03-28T13:16:27.948119+0000","last_clean":"2024-03-28T13:16:27.948119+0000","last_became_active":"2024-03-28T13:16:26.576434+0000","last_became_peered":"2024-03-28T13:16:26.576434+0000","last_unstale":"2024-03-28T13:16:27.948119+0000","last_undegraded":"2024-03-28T13:16:27.948119+0000","last_fullsized":"2024-03-28T13:16:27.948119+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:05.539379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.599013+0000","last_change":"2024-03-28T13:16:25.548513+0000","last_active":"2024-03-28T13:16:27.599013+0000","last_peered":"2024-03-28T13:16:27.599013+0000","last_clean":"2024-03-28T13:16:27.599013+0000","last_became_active":"2024-03-28T13:16:25.547239+0000","last_became_peered":"2024-03-28T13:16:25.547239+0000","last_unstale":"2024-03-28T13:16:27.599013+0000","last_undegraded":"2024-03-28T13:16:27.599013+0000","last_fullsized":"2024-03-28T13:16:27.599013+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:14:24.831395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.594216+0000","last_change":"2024-03-28T13:16:26.577794+0000","last_active":"2024-03-28T13:16:27.594216+0000","last_peered":"2024-03-28T13:16:27.594216+0000","last_clean":"2024-03-28T13:16:27.594216+0000","last_became_active":"2024-03-28T13:16:26.577506+0000","last_became_peered":"2024-03-28T13:16:26.577506+0000","last_unstale":"2024-03-28T13:16:27.594216+0000","last_undegraded":"2024-03-28T13:16:27.594216+0000","last_fullsized":"2024-03-28T13:16:27.594216+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:03.653829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.950171+0000","last_change":"2024-03-28T13:16:25.546354+0000","last_active":"2024-03-28T13:16:27.950171+0000","last_peered":"2024-03-28T13:16:27.950171+0000","last_clean":"2024-03-28T13:16:27.950171+0000","last_became_active":"2024-03-28T13:16:25.545904+0000","last_became_peered":"2024-03-28T13:16:25.545904+0000","last_unstale":"2024-03-28T13:16:27.950171+0000","last_undegraded":"2024-03-28T13:16:27.950171+0000","last_fullsized":"2024-03-28T13:16:27.950171+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:25:07.854479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.947497+0000","last_change":"2024-03-28T13:16:26.571384+0000","last_active":"2024-03-28T13:16:27.947497+0000","last_peered":"2024-03-28T13:16:27.947497+0000","last_clean":"2024-03-28T13:16:27.947497+0000","last_became_active":"2024-03-28T13:16:26.571053+0000","last_became_peered":"2024-03-28T13:16:26.571053+0000","last_unstale":"2024-03-28T13:16:27.947497+0000","last_undegraded":"2024-03-28T13:16:27.947497+0000","last_fullsized":"2024-03-28T13:16:27.947497+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:53:32.584688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.947379+0000","last_change":"2024-03-28T13:16:25.552038+0000","last_active":"2024-03-28T13:16:27.947379+0000","last_peered":"2024-03-28T13:16:27.947379+0000","last_clean":"2024-03-28T13:16:27.947379+0000","last_became_active":"2024-03-28T13:16:25.550418+0000","last_became_peered":"2024-03-28T13:16:25.550418+0000","last_unstale":"2024-03-28T13:16:27.947379+0000","last_undegraded":"2024-03-28T13:16:27.947379+0000","last_fullsized":"2024-03-28T13:16:27.947379+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:46:51.091024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.947535+0000","last_change":"2024-03-28T13:16:26.573312+0000","last_active":"2024-03-28T13:16:27.947535+0000","last_peered":"2024-03-28T13:16:27.947535+0000","last_clean":"2024-03-28T13:16:27.947535+0000","last_became_active":"2024-03-28T13:16:26.573032+0000","last_became_peered":"2024-03-28T13:16:26.573032+0000","last_unstale":"2024-03-28T13:16:27.947535+0000","last_undegraded":"2024-03-28T13:16:27.947535+0000","last_fullsized":"2024-03-28T13:16:27.947535+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:25:08.868328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.538672+0000","last_change":"2024-03-28T13:16:25.544118+0000","last_active":"2024-03-28T13:16:27.538672+0000","last_peered":"2024-03-28T13:16:27.538672+0000","last_clean":"2024-03-28T13:16:27.538672+0000","last_became_active":"2024-03-28T13:16:25.543788+0000","last_became_peered":"2024-03-28T13:16:25.543788+0000","last_unstale":"2024-03-28T13:16:27.538672+0000","last_undegraded":"2024-03-28T13:16:27.538672+0000","last_fullsized":"2024-03-28T13:16:27.538672+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:53:31.570839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.593972+0000","last_change":"2024-03-28T13:16:26.574867+0000","last_active":"2024-03-28T13:16:27.593972+0000","last_peered":"2024-03-28T13:16:27.593972+0000","last_clean":"2024-03-28T13:16:27.593972+0000","last_became_active":"2024-03-28T13:16:26.573997+0000","last_became_peered":"2024-03-28T13:16:26.573997+0000","last_unstale":"2024-03-28T13:16:27.593972+0000","last_undegraded":"2024-03-28T13:16:27.593972+0000","last_fullsized":"2024-03-28T13:16:27.593972+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:25.506286+0000","last_clean_scrub_stamp":"2024-03-28T13:16:25.506286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:44:39.700316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"91'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.543053+0000","last_change":"2024-03-28T13:16:25.552367+0000","last_active":"2024-03-28T13:16:27.543053+0000","last_peered":"2024-03-28T13:16:27.543053+0000","last_clean":"2024-03-28T13:16:27.543053+0000","last_became_active":"2024-03-28T13:16:25.551693+0000","last_became_peered":"2024-03-28T13:16:25.551693+0000","last_unstale":"2024-03-28T13:16:27.543053+0000","last_undegraded":"2024-03-28T13:16:27.543053+0000","last_fullsized":"2024-03-28T13:16:27.543053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:16:24.492436+0000","last_clean_scrub_stamp":"2024-03-28T13:16:24.492436+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:07:57.884493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":298,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-28T13:16:27.947654+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:16:27.947654+0000","last_peered":"2024-03-28T13:16:27.947654+0000","last_clean":"2024-03-28T13:16:27.947654+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:16:27.947654+0000","last_undegraded":"2024-03-28T13:16:27.947654+0000","last_fullsized":"2024-03-28T13:16:27.947654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476826,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":480950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476824,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1118208,"data_stored":480276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476824,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1716224,"data_stored":1070734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476824,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1134592,"data_stored":481910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509527,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":480948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509528,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":1072290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509528,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":480310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542232,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1110016,"data_stored":474046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:16:31.577 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-28T13:16:31.578 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-28T13:16:31.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.850+0000 7f2215a00640 1 Processor -- start 2024-03-28T13:16:31.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.850+0000 7f2215a00640 1 -- start start 2024-03-28T13:16:31.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.851+0000 7f2215a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2210072430 0x7f221006b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:31.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.852+0000 7f2215a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f221006f300 con 0x7f221015c9c0 2024-03-28T13:16:31.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.852+0000 7f2215a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f221006f5e0 con 0x7f221015a190 2024-03-28T13:16:31.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.852+0000 7f2215a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f221006f0c0 con 0x7f2210072430 2024-03-28T13:16:31.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.852+0000 7f2215000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2210072430 0x7f221006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:31.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.852+0000 7f2215000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2210072430 0x7f221006b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39494/0 (socket says 172.21.15.19:39494) 2024-03-28T13:16:31.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.852+0000 7f2215000640 1 -- 172.21.15.19:0/4161152236 learned_addr learned my addr 172.21.15.19:0/4161152236 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:31.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.852+0000 7f220f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f221015c9c0 0x7f2210071d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57314/0 (socket says 172.21.15.19:57314) 2024-03-28T13:16:31.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.852+0000 7f2215000640 1 -- 172.21.15.19:0/4161152236 >> v1:172.21.15.188:6789/0 conn(0x7f221015a190 legacy=0x7f221006e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:31.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.852+0000 7f2215000640 1 -- 172.21.15.19:0/4161152236 >> v1:172.21.15.19:6789/0 conn(0x7f221015c9c0 legacy=0x7f2210071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:31.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.852+0000 7f2215000640 1 -- 172.21.15.19:0/4161152236 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221006f8c0 con 0x7f2210072430 2024-03-28T13:16:31.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.854+0000 7f2215000640 1 --2- 172.21.15.19:0/4161152236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2210072430 0x7f221006b570 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f2208009870 tx=0x7f220802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e1510e959c933b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:31.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.854+0000 7f220ea00640 1 -- 172.21.15.19:0/4161152236 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220802e070 con 0x7f2210072430 2024-03-28T13:16:31.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.854+0000 7f220ea00640 1 -- 172.21.15.19:0/4161152236 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2208037070 con 0x7f2210072430 2024-03-28T13:16:31.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.854+0000 7f220ea00640 1 -- 172.21.15.19:0/4161152236 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208031a00 con 0x7f2210072430 2024-03-28T13:16:31.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.854+0000 7f2215a00640 1 -- 172.21.15.19:0/4161152236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2210072430 msgr2=0x7f221006b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.854+0000 7f2215a00640 1 --2- 172.21.15.19:0/4161152236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2210072430 0x7f221006b570 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f2208009870 tx=0x7f220802b880 comp rx=0 tx=0).stop 2024-03-28T13:16:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.855+0000 7f2215a00640 1 -- 172.21.15.19:0/4161152236 shutdown_connections 2024-03-28T13:16:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.855+0000 7f2215a00640 1 --2- 172.21.15.19:0/4161152236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2210072430 0x7f221006b570 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.855+0000 7f2215a00640 1 -- 172.21.15.19:0/4161152236 >> 172.21.15.19:0/4161152236 conn(0x7f22100d0800 msgr2=0x7f22100d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.855+0000 7f2215a00640 1 -- 172.21.15.19:0/4161152236 shutdown_connections 2024-03-28T13:16:31.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.855+0000 7f2215a00640 1 -- 172.21.15.19:0/4161152236 wait complete. 2024-03-28T13:16:31.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.856+0000 7f2215a00640 1 Processor -- start 2024-03-28T13:16:31.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.857+0000 7f2215a00640 1 -- start start 2024-03-28T13:16:31.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.857+0000 7f2215a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2210072430 0x7f2210156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:31.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.857+0000 7f2215a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f221015a190 0x7f221014b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.858+0000 7f2215a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221015c9c0 0x7f2210154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.858+0000 7f2215a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22100cf280 con 0x7f221015a190 2024-03-28T13:16:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.858+0000 7f2215a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f221006fba0 con 0x7f2210072430 2024-03-28T13:16:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.858+0000 7f2215a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f22100cf400 con 0x7f221015c9c0 2024-03-28T13:16:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.858+0000 7f220f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f221015a190 0x7f221014b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.858+0000 7f220f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f221015a190 0x7f221014b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39508/0 (socket says 172.21.15.19:39508) 2024-03-28T13:16:31.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.858+0000 7f220f400640 1 -- 172.21.15.19:0/1923453409 learned_addr learned my addr 172.21.15.19:0/1923453409 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.858+0000 7f220f400640 1 -- 172.21.15.19:0/1923453409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221015c9c0 msgr2=0x7f2210154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.858+0000 7f220f400640 1 --2- 172.21.15.19:0/1923453409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221015c9c0 0x7f2210154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.858+0000 7f220fe00640 1 --2- 172.21.15.19:0/1923453409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2210072430 0x7f2210156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.858+0000 7f220f400640 1 -- 172.21.15.19:0/1923453409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2210072430 msgr2=0x7f2210156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:31.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.858+0000 7f220f400640 1 --2- 172.21.15.19:0/1923453409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2210072430 0x7f2210156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.858+0000 7f220f400640 1 -- 172.21.15.19:0/1923453409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22100b4f70 con 0x7f221015a190 2024-03-28T13:16:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.858+0000 7f220fe00640 1 --2- 172.21.15.19:0/1923453409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2210072430 0x7f2210156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.859+0000 7f220f400640 1 --2- 172.21.15.19:0/1923453409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f221015a190 0x7f221014b5f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f21f800c3c0 tx=0x7f21f800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.859+0000 7f2207e00640 1 -- 172.21.15.19:0/1923453409 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f80074e0 con 0x7f221015a190 2024-03-28T13:16:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.859+0000 7f2207e00640 1 -- 172.21.15.19:0/1923453409 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f8007680 con 0x7f221015a190 2024-03-28T13:16:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.859+0000 7f2215a00640 1 -- 172.21.15.19:0/1923453409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22100b5250 con 0x7f221015a190 2024-03-28T13:16:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.859+0000 7f2207e00640 1 -- 172.21.15.19:0/1923453409 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f8004a00 con 0x7f221015a190 2024-03-28T13:16:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.859+0000 7f2215a00640 1 -- 172.21.15.19:0/1923453409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22100b5810 con 0x7f221015a190 2024-03-28T13:16:31.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.860+0000 7f2215a00640 1 -- 172.21.15.19:0/1923453409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2210075940 con 0x7f221015a190 2024-03-28T13:16:31.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.861+0000 7f2207e00640 1 -- 172.21.15.19:0/1923453409 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f21f801a020 con 0x7f221015a190 2024-03-28T13:16:31.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.863+0000 7f2207e00640 1 --2- 172.21.15.19:0/1923453409 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21ec06c560 0x7f21ec06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:31.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.863+0000 7f2207e00640 1 -- 172.21.15.19:0/1923453409 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f21f808aec0 con 0x7f221015a190 2024-03-28T13:16:31.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.863+0000 7f220fe00640 1 --2- 172.21.15.19:0/1923453409 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21ec06c560 0x7f21ec06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:31.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.864+0000 7f220fe00640 1 --2- 172.21.15.19:0/1923453409 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21ec06c560 0x7f21ec06ea20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2200004520 tx=0x7f2200009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:31.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:31.867+0000 7f2207e00640 1 -- 172.21.15.19:0/1923453409 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f21f8059440 con 0x7f221015a190 2024-03-28T13:16:32.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.150+0000 7f2215a00640 1 -- 172.21.15.19:0/1923453409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f22100720a0 con 0x7f221015a190 2024-03-28T13:16:32.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.155+0000 7f2207e00640 1 -- 172.21.15.19:0/1923453409 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f21f8058d40 con 0x7f221015a190 2024-03-28T13:16:32.155 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-03-28T13:16:32.155 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q== 2024-03-28T13:16:32.155 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-03-28T13:16:32.155 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-03-28T13:16:32.155 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-28T13:16:32.156 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-03-28T13:16:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.158+0000 7f2215a00640 1 -- 172.21.15.19:0/1923453409 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21ec06c560 msgr2=0x7f21ec06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.158+0000 7f2215a00640 1 --2- 172.21.15.19:0/1923453409 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21ec06c560 0x7f21ec06ea20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2200004520 tx=0x7f2200009040 comp rx=0 tx=0).stop 2024-03-28T13:16:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.158+0000 7f2215a00640 1 -- 172.21.15.19:0/1923453409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f221015a190 msgr2=0x7f221014b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:32.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.158+0000 7f2215a00640 1 --2- 172.21.15.19:0/1923453409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f221015a190 0x7f221014b5f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f21f800c3c0 tx=0x7f21f800c880 comp rx=0 tx=0).stop 2024-03-28T13:16:32.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.158+0000 7f2215a00640 1 -- 172.21.15.19:0/1923453409 shutdown_connections 2024-03-28T13:16:32.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.158+0000 7f2215a00640 1 --2- 172.21.15.19:0/1923453409 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21ec06c560 0x7f21ec06ea20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:32.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.158+0000 7f2215a00640 1 --2- 172.21.15.19:0/1923453409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f221015c9c0 0x7f2210154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:32.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.158+0000 7f2215a00640 1 --2- 172.21.15.19:0/1923453409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f221015a190 0x7f221014b5f0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:32.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.158+0000 7f2215a00640 1 --2- 172.21.15.19:0/1923453409 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2210072430 0x7f2210156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:32.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.158+0000 7f2215a00640 1 -- 172.21.15.19:0/1923453409 >> 172.21.15.19:0/1923453409 conn(0x7f22100d0800 msgr2=0x7f22101593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:32.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.158+0000 7f2215a00640 1 -- 172.21.15.19:0/1923453409 shutdown_connections 2024-03-28T13:16:32.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.158+0000 7f2215a00640 1 -- 172.21.15.19:0/1923453409 wait complete. 2024-03-28T13:16:32.176 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:16:32.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.393+0000 7f81ae800640 1 Processor -- start 2024-03-28T13:16:32.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.393+0000 7f81ae800640 1 -- start start 2024-03-28T13:16:32.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.395+0000 7f81ae800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a80d0e90 0x7f81a80b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:32.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.395+0000 7f81ae800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81a815b6f0 con 0x7f81a80727f0 2024-03-28T13:16:32.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.395+0000 7f81ae800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81a8072e80 con 0x7f81a815b310 2024-03-28T13:16:32.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.395+0000 7f81ae800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f81a8072c50 con 0x7f81a80d0e90 2024-03-28T13:16:32.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.395+0000 7f81a6a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f81a80727f0 0x7f81a80d0780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57328/0 (socket says 172.21.15.19:57328) 2024-03-28T13:16:32.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.395+0000 7f81a6a00640 1 -- 172.21.15.19:0/2073163758 learned_addr learned my addr 172.21.15.19:0/2073163758 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:32.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.395+0000 7f81a7e00640 1 --2- 172.21.15.19:0/2073163758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a80d0e90 0x7f81a80b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:32.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.395+0000 7f81a6000640 1 -- 172.21.15.19:0/2073163758 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1431615728 0 0) 0x7f81a815b6f0 con 0x7f81a80727f0 2024-03-28T13:16:32.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.395+0000 7f81a6000640 1 -- 172.21.15.19:0/2073163758 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f818c005730 con 0x7f81a80727f0 2024-03-28T13:16:32.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.396+0000 7f81a6000640 1 -- 172.21.15.19:0/2073163758 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2970993870 0 0) 0x7f818c005730 con 0x7f81a80727f0 2024-03-28T13:16:32.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.396+0000 7f81a6000640 1 -- 172.21.15.19:0/2073163758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a80d0e90 msgr2=0x7f81a80b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:32.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.396+0000 7f81a6000640 1 --2- 172.21.15.19:0/2073163758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a80d0e90 0x7f81a80b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:32.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.396+0000 7f81a6000640 1 -- 172.21.15.19:0/2073163758 >> v1:172.21.15.188:6789/0 conn(0x7f81a815b310 legacy=0x7f81a815f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:32.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.396+0000 7f81a6000640 1 -- 172.21.15.19:0/2073163758 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a8073160 con 0x7f81a80727f0 2024-03-28T13:16:32.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.396+0000 7f81a6000640 1 -- 172.21.15.19:0/2073163758 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f8194001670 con 0x7f81a80727f0 2024-03-28T13:16:32.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.396+0000 7f81a6000640 1 -- 172.21.15.19:0/2073163758 >> v1:172.21.15.19:6789/0 conn(0x7f81a80727f0 legacy=0x7f81a80d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:32.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.396+0000 7f81a6000640 1 --2- 172.21.15.19:0/2073163758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f818c005bf0 0x7f818c006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:32.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.396+0000 7f81a6000640 1 --2- 172.21.15.19:0/2073163758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a80d0e90 0x7f818c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:32.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.396+0000 7f81a6000640 1 --2- 172.21.15.19:0/2073163758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f818c006f50 0x7f818c0073b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:32.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.396+0000 7f81a6000640 1 -- 172.21.15.19:0/2073163758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f818c00dde0 con 0x7f818c006f50 2024-03-28T13:16:32.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.396+0000 7f81a6000640 1 -- 172.21.15.19:0/2073163758 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f818c00df90 con 0x7f818c005bf0 2024-03-28T13:16:32.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.396+0000 7f81a6000640 1 -- 172.21.15.19:0/2073163758 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f818c00e190 con 0x7f81a80d0e90 2024-03-28T13:16:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.396+0000 7f81a7e00640 1 --2- 172.21.15.19:0/2073163758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f818c006f50 0x7f818c0073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.397+0000 7f81a6a00640 1 --2- 172.21.15.19:0/2073163758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a80d0e90 0x7f818c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.397+0000 7f81a7400640 1 --2- 172.21.15.19:0/2073163758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f818c005bf0 0x7f818c006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.397+0000 7f81a7e00640 1 -- 172.21.15.19:0/2073163758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a80d0e90 msgr2=0x7f818c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.397+0000 7f81a7e00640 1 --2- 172.21.15.19:0/2073163758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a80d0e90 0x7f818c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.397+0000 7f81a7e00640 1 -- 172.21.15.19:0/2073163758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f818c005bf0 msgr2=0x7f818c006010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.397+0000 7f81a7e00640 1 --2- 172.21.15.19:0/2073163758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f818c005bf0 0x7f818c006010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.397+0000 7f81a7e00640 1 -- 172.21.15.19:0/2073163758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818c005730 con 0x7f818c006f50 2024-03-28T13:16:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.397+0000 7f81a7400640 1 --2- 172.21.15.19:0/2073163758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f818c005bf0 0x7f818c006010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.397+0000 7f81a6a00640 1 --2- 172.21.15.19:0/2073163758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a80d0e90 0x7f818c006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.398+0000 7f81a7e00640 1 --2- 172.21.15.19:0/2073163758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f818c006f50 0x7f818c0073b0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009fc0 tx=0x7f818c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183efc06bd2cc98c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.398+0000 7f81a6000640 1 -- 172.21.15.19:0/2073163758 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0039070 con 0x7f818c006f50 2024-03-28T13:16:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.398+0000 7f81a6000640 1 -- 172.21.15.19:0/2073163758 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a002cbf0 con 0x7f818c006f50 2024-03-28T13:16:32.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.398+0000 7f81a6000640 1 -- 172.21.15.19:0/2073163758 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0009c10 con 0x7f818c006f50 2024-03-28T13:16:32.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.399+0000 7f81ae800640 1 -- 172.21.15.19:0/2073163758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f818c006f50 msgr2=0x7f818c0073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:32.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.399+0000 7f81ae800640 1 --2- 172.21.15.19:0/2073163758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f818c006f50 0x7f818c0073b0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009fc0 tx=0x7f818c007a40 comp rx=0 tx=0).stop 2024-03-28T13:16:32.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.399+0000 7f81a7e00640 1 -- 172.21.15.19:0/2073163758 reap_dead start 2024-03-28T13:16:32.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.399+0000 7f81ae800640 1 -- 172.21.15.19:0/2073163758 shutdown_connections 2024-03-28T13:16:32.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.399+0000 7f81ae800640 1 -- 172.21.15.19:0/2073163758 >> 172.21.15.19:0/2073163758 conn(0x7f81a806b990 msgr2=0x7f81a806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:32.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.399+0000 7f81ae800640 1 -- 172.21.15.19:0/2073163758 shutdown_connections 2024-03-28T13:16:32.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.399+0000 7f81ae800640 1 -- 172.21.15.19:0/2073163758 wait complete. 2024-03-28T13:16:32.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.401+0000 7f81ae800640 1 Processor -- start 2024-03-28T13:16:32.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.401+0000 7f81ae800640 1 -- start start 2024-03-28T13:16:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.402+0000 7f81ae800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f818c006f50 0x7f81a815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.402+0000 7f81ae800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a806f1d0 0x7f81a8151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.402+0000 7f81ae800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a81f3e80 0x7f81a81f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.402+0000 7f81ae800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81a8073440 con 0x7f81a81f3e80 2024-03-28T13:16:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.402+0000 7f81ae800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f81a815f2c0 con 0x7f818c006f50 2024-03-28T13:16:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.402+0000 7f81ae800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f81a80b4f90 con 0x7f81a806f1d0 2024-03-28T13:16:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.402+0000 7f81a7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a81f3e80 0x7f81a81f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:32.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.402+0000 7f81a7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a81f3e80 0x7f81a81f7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39538/0 (socket says 172.21.15.19:39538) 2024-03-28T13:16:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.402+0000 7f81a7e00640 1 -- 172.21.15.19:0/3754190939 learned_addr learned my addr 172.21.15.19:0/3754190939 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.402+0000 7f81a7400640 1 --2- 172.21.15.19:0/3754190939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f818c006f50 0x7f81a815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.402+0000 7f81a6a00640 1 --2- 172.21.15.19:0/3754190939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a806f1d0 0x7f81a8151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.403+0000 7f81a7e00640 1 -- 172.21.15.19:0/3754190939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a806f1d0 msgr2=0x7f81a8151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.403+0000 7f81a7e00640 1 --2- 172.21.15.19:0/3754190939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a806f1d0 0x7f81a8151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.403+0000 7f81a7e00640 1 -- 172.21.15.19:0/3754190939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f818c006f50 msgr2=0x7f81a815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.403+0000 7f81a7e00640 1 --2- 172.21.15.19:0/3754190939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f818c006f50 0x7f81a815e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.403+0000 7f81a7e00640 1 -- 172.21.15.19:0/3754190939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a81509b0 con 0x7f81a81f3e80 2024-03-28T13:16:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.403+0000 7f81a7e00640 1 --2- 172.21.15.19:0/3754190939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a81f3e80 0x7f81a81f7c70 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f81a000e230 tx=0x7f81a0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.403+0000 7f819f400640 1 -- 172.21.15.19:0/3754190939 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0039030 con 0x7f81a81f3e80 2024-03-28T13:16:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.403+0000 7f819f400640 1 -- 172.21.15.19:0/3754190939 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81a003a040 con 0x7f81a81f3e80 2024-03-28T13:16:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.403+0000 7f819f400640 1 -- 172.21.15.19:0/3754190939 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0009c10 con 0x7f81a81f3e80 2024-03-28T13:16:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.403+0000 7f81ae800640 1 -- 172.21.15.19:0/3754190939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a8150c50 con 0x7f81a81f3e80 2024-03-28T13:16:32.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.403+0000 7f81ae800640 1 -- 172.21.15.19:0/3754190939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a8151210 con 0x7f81a81f3e80 2024-03-28T13:16:32.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.405+0000 7f81ae800640 1 -- 172.21.15.19:0/3754190939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a8151410 con 0x7f81a81f3e80 2024-03-28T13:16:32.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.405+0000 7f819f400640 1 -- 172.21.15.19:0/3754190939 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f81a0045030 con 0x7f81a81f3e80 2024-03-28T13:16:32.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.406+0000 7f819f400640 1 --2- 172.21.15.19:0/3754190939 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f818806c530 0x7f818806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:32.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.406+0000 7f819f400640 1 -- 172.21.15.19:0/3754190939 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f81a00ad390 con 0x7f81a81f3e80 2024-03-28T13:16:32.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.407+0000 7f81a7400640 1 --2- 172.21.15.19:0/3754190939 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f818806c530 0x7f818806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:32.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.408+0000 7f81a7400640 1 --2- 172.21.15.19:0/3754190939 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f818806c530 0x7f818806e9f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8198001750 tx=0x7f8198009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:32.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.411+0000 7f819f400640 1 -- 172.21.15.19:0/3754190939 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f81a8151410 con 0x7f81a81f3e80 2024-03-28T13:16:32.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.655+0000 7f81ae800640 1 -- 172.21.15.19:0/3754190939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81a80685c0 con 0x7f81a81f3e80 2024-03-28T13:16:32.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.656+0000 7f819f400640 1 -- 172.21.15.19:0/3754190939 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5186 (secure 0 0 0) 0x7f81a004c060 con 0x7f81a81f3e80 2024-03-28T13:16:32.656 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:32.656 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:16:29.083492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6043":{"gid":6043,"name":"b","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2260087679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2260087679},{"type":"v1","addr":"172.21.15.188:6836","nonce":2260087679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6043,"qdb_cluster":[6043]},"id":6}]} 2024-03-28T13:16:32.657 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 61 2024-03-28T13:16:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.659+0000 7f81ae800640 1 -- 172.21.15.19:0/3754190939 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f818806c530 msgr2=0x7f818806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:32.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.659+0000 7f81ae800640 1 --2- 172.21.15.19:0/3754190939 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f818806c530 0x7f818806e9f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8198001750 tx=0x7f8198009040 comp rx=0 tx=0).stop 2024-03-28T13:16:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.659+0000 7f81ae800640 1 -- 172.21.15.19:0/3754190939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a81f3e80 msgr2=0x7f81a81f7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.659+0000 7f81ae800640 1 --2- 172.21.15.19:0/3754190939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a81f3e80 0x7f81a81f7c70 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f81a000e230 tx=0x7f81a0009750 comp rx=0 tx=0).stop 2024-03-28T13:16:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.659+0000 7f81ae800640 1 -- 172.21.15.19:0/3754190939 shutdown_connections 2024-03-28T13:16:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.659+0000 7f81ae800640 1 --2- 172.21.15.19:0/3754190939 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f818806c530 0x7f818806e9f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.659+0000 7f81ae800640 1 --2- 172.21.15.19:0/3754190939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f81a81f3e80 0x7f81a81f7c70 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.659+0000 7f81ae800640 1 --2- 172.21.15.19:0/3754190939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81a806f1d0 0x7f81a8151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.659+0000 7f81ae800640 1 --2- 172.21.15.19:0/3754190939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f818c006f50 0x7f81a815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:32.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.659+0000 7f81ae800640 1 -- 172.21.15.19:0/3754190939 >> 172.21.15.19:0/3754190939 conn(0x7f81a806b990 msgr2=0x7f81a8153580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:32.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.659+0000 7f81ae800640 1 -- 172.21.15.19:0/3754190939 shutdown_connections 2024-03-28T13:16:32.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:32.659+0000 7f81ae800640 1 -- 172.21.15.19:0/3754190939 wait complete. 2024-03-28T13:16:33.081 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:16:25.995386+0000', 'modified': '2024-03-28T13:16:29.083492+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6043}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6043': {'gid': 6043, 'name': 'b', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6836/2260087679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 2260087679}, {'type': 'v1', 'addr': '172.21.15.188:6836', 'nonce': 2260087679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6043, 'qdb_cluster': [6043]} 2024-03-28T13:16:33.081 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T13:16:33.082 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-28T13:16:33.105 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:16:33.135 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:16:33.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.306+0000 7f8c7e800640 1 Processor -- start 2024-03-28T13:16:33.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.306+0000 7f8c7e800640 1 -- start start 2024-03-28T13:16:33.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.308+0000 7f8c7e800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8c78162920 0x7f8c781670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.308+0000 7f8c7e800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8c78163ac0 0x7f8c78071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.308+0000 7f8c7e800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c78071d60 con 0x7f8c780d0280 2024-03-28T13:16:33.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.308+0000 7f8c7e800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8c78163f20 con 0x7f8c78162920 2024-03-28T13:16:33.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.308+0000 7f8c7e800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f8c781665d0 con 0x7f8c78163ac0 2024-03-28T13:16:33.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.309+0000 7f8c77e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8c780d0280 0x7f8c780ad010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57344/0 (socket says 172.21.15.19:57344) 2024-03-28T13:16:33.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.309+0000 7f8c77e00640 1 -- 172.21.15.19:0/4142962811 learned_addr learned my addr 172.21.15.19:0/4142962811 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:33.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.309+0000 7f8c77400640 1 --2- 172.21.15.19:0/4142962811 >> v2:172.21.15.19:3300/0 conn(0x7f8c78162920 0x7f8c781670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.309+0000 7f8c6fe00640 1 --2- 172.21.15.19:0/4142962811 >> v2:172.21.15.188:3300/0 conn(0x7f8c78163ac0 0x7f8c78071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.309+0000 7f8c76a00640 1 -- 172.21.15.19:0/4142962811 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4184069058 0 0) 0x7f8c78071d60 con 0x7f8c780d0280 2024-03-28T13:16:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.309+0000 7f8c76a00640 1 -- 172.21.15.19:0/4142962811 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c5c003520 con 0x7f8c780d0280 2024-03-28T13:16:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.309+0000 7f8c77400640 1 -- 172.21.15.19:0/4142962811 >> v2:172.21.15.188:3300/0 conn(0x7f8c78163ac0 msgr2=0x7f8c78071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:33.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.309+0000 7f8c77400640 1 --2- 172.21.15.19:0/4142962811 >> v2:172.21.15.188:3300/0 conn(0x7f8c78163ac0 0x7f8c78071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.309+0000 7f8c77400640 1 -- 172.21.15.19:0/4142962811 >> v1:172.21.15.19:6789/0 conn(0x7f8c780d0280 legacy=0x7f8c780ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.309+0000 7f8c77400640 1 -- 172.21.15.19:0/4142962811 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c781668b0 con 0x7f8c78162920 2024-03-28T13:16:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.310+0000 7f8c77400640 1 --2- 172.21.15.19:0/4142962811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c78162920 0x7f8c781670b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f8c780d1050 tx=0x7f8c6802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ee62be4775716 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.311+0000 7f8c76a00640 1 -- 172.21.15.19:0/4142962811 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c680315a0 con 0x7f8c78162920 2024-03-28T13:16:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.311+0000 7f8c76a00640 1 -- 172.21.15.19:0/4142962811 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c68037070 con 0x7f8c78162920 2024-03-28T13:16:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.311+0000 7f8c76a00640 1 -- 172.21.15.19:0/4142962811 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68031940 con 0x7f8c78162920 2024-03-28T13:16:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.311+0000 7f8c7e800640 1 -- 172.21.15.19:0/4142962811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c78162920 msgr2=0x7f8c781670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.311+0000 7f8c7e800640 1 --2- 172.21.15.19:0/4142962811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c78162920 0x7f8c781670b0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f8c780d1050 tx=0x7f8c6802b840 comp rx=0 tx=0).stop 2024-03-28T13:16:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.312+0000 7f8c7e800640 1 -- 172.21.15.19:0/4142962811 shutdown_connections 2024-03-28T13:16:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.312+0000 7f8c7e800640 1 --2- 172.21.15.19:0/4142962811 >> v2:172.21.15.188:3300/0 conn(0x7f8c78163ac0 0x7f8c78071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.312+0000 7f8c7e800640 1 --2- 172.21.15.19:0/4142962811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c78162920 0x7f8c781670b0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.312+0000 7f8c7e800640 1 -- 172.21.15.19:0/4142962811 >> 172.21.15.19:0/4142962811 conn(0x7f8c7806b750 msgr2=0x7f8c7806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:33.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.312+0000 7f8c7e800640 1 -- 172.21.15.19:0/4142962811 shutdown_connections 2024-03-28T13:16:33.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.312+0000 7f8c7e800640 1 -- 172.21.15.19:0/4142962811 wait complete. 2024-03-28T13:16:33.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.314+0000 7f8c7e800640 1 Processor -- start 2024-03-28T13:16:33.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.314+0000 7f8c7e800640 1 -- start start 2024-03-28T13:16:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c7e800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c780d0280 0x7f8c7809a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c7e800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c78162920 0x7f8c7809c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c7e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c78163ac0 0x7f8c78160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c7e800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78072040 con 0x7f8c78163ac0 2024-03-28T13:16:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c7e800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78162d00 con 0x7f8c780d0280 2024-03-28T13:16:33.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c7e800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8c780721c0 con 0x7f8c78162920 2024-03-28T13:16:33.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c77e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c78163ac0 0x7f8c78160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c77e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c78163ac0 0x7f8c78160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39562/0 (socket says 172.21.15.19:39562) 2024-03-28T13:16:33.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c77e00640 1 -- 172.21.15.19:0/721633974 learned_addr learned my addr 172.21.15.19:0/721633974 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:33.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c77400640 1 --2- 172.21.15.19:0/721633974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c780d0280 0x7f8c7809a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c6fe00640 1 --2- 172.21.15.19:0/721633974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c78162920 0x7f8c7809c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c77e00640 1 -- 172.21.15.19:0/721633974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c78162920 msgr2=0x7f8c7809c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:33.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c77e00640 1 --2- 172.21.15.19:0/721633974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c78162920 0x7f8c7809c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c77e00640 1 -- 172.21.15.19:0/721633974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c780d0280 msgr2=0x7f8c7809a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:33.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c77e00640 1 --2- 172.21.15.19:0/721633974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c780d0280 0x7f8c7809a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.315+0000 7f8c77e00640 1 -- 172.21.15.19:0/721633974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7815f650 con 0x7f8c78163ac0 2024-03-28T13:16:33.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.316+0000 7f8c6fe00640 1 --2- 172.21.15.19:0/721633974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c78162920 0x7f8c7809c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:33.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.316+0000 7f8c77e00640 1 --2- 172.21.15.19:0/721633974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c78163ac0 0x7f8c78160240 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000caa0 tx=0x7f8c7000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:33.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.316+0000 7f8c6f400640 1 -- 172.21.15.19:0/721633974 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70017070 con 0x7f8c78163ac0 2024-03-28T13:16:33.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.316+0000 7f8c6f400640 1 -- 172.21.15.19:0/721633974 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c7000ac00 con 0x7f8c78163ac0 2024-03-28T13:16:33.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.316+0000 7f8c6f400640 1 -- 172.21.15.19:0/721633974 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70012400 con 0x7f8c78163ac0 2024-03-28T13:16:33.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.316+0000 7f8c7e800640 1 -- 172.21.15.19:0/721633974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c781642e0 con 0x7f8c78163ac0 2024-03-28T13:16:33.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.316+0000 7f8c7e800640 1 -- 172.21.15.19:0/721633974 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c781648a0 con 0x7f8c78163ac0 2024-03-28T13:16:33.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.319+0000 7f8c6f400640 1 -- 172.21.15.19:0/721633974 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8c7001a070 con 0x7f8c78163ac0 2024-03-28T13:16:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.320+0000 7f8c6f400640 1 --2- 172.21.15.19:0/721633974 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8c5406c530 0x7f8c5406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.320+0000 7f8c6f400640 1 -- 172.21.15.19:0/721633974 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f8c7008b510 con 0x7f8c78163ac0 2024-03-28T13:16:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.321+0000 7f8c8145f740 1 Processor -- start 2024-03-28T13:16:33.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.321+0000 7f8c77400640 1 --2- 172.21.15.19:0/721633974 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8c5406c530 0x7f8c5406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.321+0000 7f8c8145f740 1 -- start start 2024-03-28T13:16:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c8145f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad8fd0 0x5578eaaabf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c8145f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad93b0 0x5578eaadbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c8145f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaaac4b0 0x5578eaadc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c8145f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5578eaa73920 con 0x5578eaad8fd0 2024-03-28T13:16:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c8145f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5578ea9c3fa0 con 0x5578eaad93b0 2024-03-28T13:16:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c8145f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5578eaa15f90 con 0x5578eaaac4b0 2024-03-28T13:16:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c77400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad8fd0 0x5578eaaabf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c77400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad8fd0 0x5578eaaabf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39564/0 (socket says 172.21.15.19:39564) 2024-03-28T13:16:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c77400640 1 -- 172.21.15.19:0/4195026558 learned_addr learned my addr 172.21.15.19:0/4195026558 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c6fe00640 1 --2- 172.21.15.19:0/4195026558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad93b0 0x5578eaadbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c77e00640 1 --2- 172.21.15.19:0/4195026558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaaac4b0 0x5578eaadc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c77400640 1 --2- 172.21.15.19:0/721633974 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8c5406c530 0x7f8c5406e9f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8c68036000 tx=0x7f8c6803d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:33.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c77400640 1 -- 172.21.15.19:0/4195026558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaaac4b0 msgr2=0x5578eaadc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c77400640 1 --2- 172.21.15.19:0/4195026558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaaac4b0 0x5578eaadc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c77400640 1 -- 172.21.15.19:0/4195026558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad93b0 msgr2=0x5578eaadbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c77400640 1 --2- 172.21.15.19:0/4195026558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad93b0 0x5578eaadbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.322+0000 7f8c77400640 1 -- 172.21.15.19:0/4195026558 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578ea72a2d0 con 0x5578eaad8fd0 2024-03-28T13:16:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.323+0000 7f8c77400640 1 --2- 172.21.15.19:0/4195026558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad8fd0 0x5578eaaabf70 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f8c6802be50 tx=0x7f8c68008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80cee8e51d3b144 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.323+0000 7f8c67e00640 1 -- 172.21.15.19:0/4195026558 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68053070 con 0x5578eaad8fd0 2024-03-28T13:16:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.323+0000 7f8c67e00640 1 -- 172.21.15.19:0/4195026558 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c68037040 con 0x5578eaad8fd0 2024-03-28T13:16:33.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.323+0000 7f8c67e00640 1 -- 172.21.15.19:0/4195026558 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68046db0 con 0x5578eaad8fd0 2024-03-28T13:16:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.324+0000 7f8c8145f740 1 -- 172.21.15.19:0/4195026558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad8fd0 msgr2=0x5578eaaabf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.324+0000 7f8c8145f740 1 --2- 172.21.15.19:0/4195026558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad8fd0 0x5578eaaabf70 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f8c6802be50 tx=0x7f8c68008030 comp rx=0 tx=0).stop 2024-03-28T13:16:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.324+0000 7f8c8145f740 1 -- 172.21.15.19:0/4195026558 shutdown_connections 2024-03-28T13:16:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.324+0000 7f8c8145f740 1 --2- 172.21.15.19:0/4195026558 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaaac4b0 0x5578eaadc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.324+0000 7f8c8145f740 1 --2- 172.21.15.19:0/4195026558 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad93b0 0x5578eaadbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.324+0000 7f8c8145f740 1 --2- 172.21.15.19:0/4195026558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad8fd0 0x5578eaaabf70 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.324+0000 7f8c8145f740 1 -- 172.21.15.19:0/4195026558 >> 172.21.15.19:0/4195026558 conn(0x5578eaaae1e0 msgr2=0x5578eaaf6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:33.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.324+0000 7f8c8145f740 1 -- 172.21.15.19:0/4195026558 shutdown_connections 2024-03-28T13:16:33.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.324+0000 7f8c8145f740 1 -- 172.21.15.19:0/4195026558 wait complete. 2024-03-28T13:16:33.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.325+0000 7f8c8145f740 1 -- start start 2024-03-28T13:16:33.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.325+0000 7f8c8145f740 1 Processor -- start 2024-03-28T13:16:33.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.326+0000 7f8c8145f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad8fd0 0x5578eaadf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.326+0000 7f8c8145f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad93b0 0x5578eab06d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.326+0000 7f8c8145f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad82d0 0x5578eab07f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.326+0000 7f8c8145f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5578eaa11f40 con 0x5578eaad82d0 2024-03-28T13:16:33.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.326+0000 7f8c8145f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5578ea9be650 con 0x5578eaad8fd0 2024-03-28T13:16:33.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.326+0000 7f8c8145f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5578ea9d1a10 con 0x5578eaad93b0 2024-03-28T13:16:33.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.326+0000 7f8c8145f740 10 client.0 _ll_register_callbacks cb 0x5578eaa10ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:16:33.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.326+0000 7f8c77e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad82d0 0x5578eab07f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.326+0000 7f8c77e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad82d0 0x5578eab07f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39578/0 (socket says 172.21.15.19:39578) 2024-03-28T13:16:33.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.326+0000 7f8c77e00640 1 -- 172.21.15.19:0/1996018764 learned_addr learned my addr 172.21.15.19:0/1996018764 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:33.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.327+0000 7f8c77400640 1 --2- 172.21.15.19:0/1996018764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad8fd0 0x5578eaadf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.327+0000 7f8c6fe00640 1 --2- 172.21.15.19:0/1996018764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad93b0 0x5578eab06d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.327+0000 7f8c77e00640 1 -- 172.21.15.19:0/1996018764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad93b0 msgr2=0x5578eab06d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.327+0000 7f8c77e00640 1 --2- 172.21.15.19:0/1996018764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad93b0 0x5578eab06d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.327+0000 7f8c77e00640 1 -- 172.21.15.19:0/1996018764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad8fd0 msgr2=0x5578eaadf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.327+0000 7f8c77e00640 1 --2- 172.21.15.19:0/1996018764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad8fd0 0x5578eaadf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.327+0000 7f8c77e00640 1 -- 172.21.15.19:0/1996018764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578ea7f46b0 con 0x5578eaad82d0 2024-03-28T13:16:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.327+0000 7f8c77e00640 1 --2- 172.21.15.19:0/1996018764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad82d0 0x5578eab07f60 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8c700557f0 tx=0x7f8c7005d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d32b82324f17c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.327+0000 7f8c64c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:16:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.327+0000 7f8c64c00640 1 -- 172.21.15.19:0/1996018764 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70057750 con 0x5578eaad82d0 2024-03-28T13:16:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.327+0000 7f8c64c00640 1 -- 172.21.15.19:0/1996018764 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f8c70061cf0 con 0x5578eaad82d0 2024-03-28T13:16:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.327+0000 7f8c8145f740 1 -- 172.21.15.19:0/1996018764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c68037040 con 0x5578eaad82d0 2024-03-28T13:16:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.328+0000 7f8c64c00640 1 -- 172.21.15.19:0/1996018764 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8c7006dbb0 con 0x5578eaad82d0 2024-03-28T13:16:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.328+0000 7f8c8145f740 10 client.7329 fetch_fsmap learned FSMap version 61 2024-03-28T13:16:33.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.328+0000 7f8c8145f740 1 -- 172.21.15.19:0/1996018764 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x5578eaadee80 con 0x5578eaad82d0 2024-03-28T13:16:33.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.328+0000 7f8c64c00640 1 -- 172.21.15.19:0/1996018764 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c68037040 con 0x5578eaad82d0 2024-03-28T13:16:33.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.328+0000 7f8c64c00640 1 -- 172.21.15.19:0/1996018764 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f8c7008b070 con 0x5578eaad82d0 2024-03-28T13:16:33.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.328+0000 7f8c8145f740 10 client.7329 fetch_fsmap finished waiting for FSMap version 61 2024-03-28T13:16:33.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.328+0000 7f8c8145f740 10 client.7329 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T13:16:33.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.328+0000 7f8c8145f740 20 client.7329 populate_metadata read hostname 'smithi019' 2024-03-28T13:16:33.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.328+0000 7f8c8145f740 1 --2- 172.21.15.19:0/1996018764 >> [v2:172.21.15.188:6834/2260087679,v1:172.21.15.188:6836/2260087679] conn(0x5578eab0abc0 0x5578eab15820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.328+0000 7f8c8145f740 4 client.7329 mds_command: new command op to 6043 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:16:33.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.328+0000 7f8c8145f740 1 -- 172.21.15.19:0/1996018764 --> [v2:172.21.15.188:6834/2260087679,v1:172.21.15.188:6836/2260087679] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578eaa234d0 con 0x5578eab0abc0 2024-03-28T13:16:33.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.329+0000 7f8c77400640 1 --2- 172.21.15.19:0/1996018764 >> [v2:172.21.15.188:6834/2260087679,v1:172.21.15.188:6836/2260087679] conn(0x5578eab0abc0 0x5578eab15820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.330+0000 7f8c77400640 1 --2- 172.21.15.19:0/1996018764 >> [v2:172.21.15.188:6834/2260087679,v1:172.21.15.188:6836/2260087679] conn(0x5578eab0abc0 0x5578eab15820 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a7b4fd601805525 server_cookie=1893588080e1b181 in_seq=0 out_seq=0 2024-03-28T13:16:33.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.330+0000 7f8c64c00640 10 client.7329 ms_handle_connect on v2:172.21.15.188:6834/2260087679 2024-03-28T13:16:33.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.332+0000 7f8c64c00640 1 -- 172.21.15.19:0/1996018764 <== mds.0 v2:172.21.15.188:6834/2260087679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5578eaa234d0 con 0x5578eab0abc0 2024-03-28T13:16:33.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.332+0000 7f8c64c00640 10 client.7329 handle_command_reply: tid=0 2024-03-28T13:16:33.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.332+0000 7f8c8145f740 1 client.7329 shutdown 2024-03-28T13:16:33.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.332+0000 7f8c8145f740 1 -- 172.21.15.19:0/1996018764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad82d0 msgr2=0x5578eab07f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:33.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.332+0000 7f8c8145f740 1 --2- 172.21.15.19:0/1996018764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad82d0 0x5578eab07f60 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f8c700557f0 tx=0x7f8c7005d740 comp rx=0 tx=0).stop 2024-03-28T13:16:33.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.332+0000 7f8c8145f740 1 -- 172.21.15.19:0/1996018764 shutdown_connections 2024-03-28T13:16:33.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.332+0000 7f8c8145f740 1 --2- 172.21.15.19:0/1996018764 >> [v2:172.21.15.188:6834/2260087679,v1:172.21.15.188:6836/2260087679] conn(0x5578eab0abc0 0x5578eab15820 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.332+0000 7f8c8145f740 1 --2- 172.21.15.19:0/1996018764 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad82d0 0x5578eab07f60 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.332+0000 7f8c8145f740 1 --2- 172.21.15.19:0/1996018764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad93b0 0x5578eab06d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.332+0000 7f8c8145f740 1 --2- 172.21.15.19:0/1996018764 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad8fd0 0x5578eaadf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.332+0000 7f8c8145f740 1 -- 172.21.15.19:0/1996018764 >> 172.21.15.19:0/1996018764 conn(0x5578eaaac4b0 msgr2=0x5578eaaab690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:33.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.332+0000 7f8c64c00640 0 client.7329 ms_handle_reset on v2:172.21.15.188:6834/2260087679 2024-03-28T13:16:33.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.332+0000 7f8c8145f740 1 -- 172.21.15.19:0/1996018764 shutdown_connections 2024-03-28T13:16:33.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.332+0000 7f8c8145f740 1 -- 172.21.15.19:0/1996018764 wait complete. 2024-03-28T13:16:33.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.332+0000 7f8c8145f740 20 client.7329 trim_cache size 0 max 16384 2024-03-28T13:16:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.352+0000 7f8c8145f740 1 Processor -- start 2024-03-28T13:16:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.352+0000 7f8c8145f740 1 -- start start 2024-03-28T13:16:33.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.353+0000 7f8c8145f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad82d0 0x5578eaadbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.353+0000 7f8c8145f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad8fd0 0x5578eaadc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.353+0000 7f8c8145f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad93b0 0x5578eaaec6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.353+0000 7f8c8145f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781fc900 con 0x5578eaad82d0 2024-03-28T13:16:33.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.353+0000 7f8c8145f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781fbc60 con 0x5578eaad93b0 2024-03-28T13:16:33.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.353+0000 7f8c8145f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8c781fc2b0 con 0x5578eaad8fd0 2024-03-28T13:16:33.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.354+0000 7f8c77400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad82d0 0x5578eaadbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.354+0000 7f8c77400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad82d0 0x5578eaadbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39592/0 (socket says 172.21.15.19:39592) 2024-03-28T13:16:33.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.354+0000 7f8c77400640 1 -- 172.21.15.19:0/259482292 learned_addr learned my addr 172.21.15.19:0/259482292 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:33.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.354+0000 7f8c6fe00640 1 --2- 172.21.15.19:0/259482292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad8fd0 0x5578eaadc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.354+0000 7f8c77e00640 1 --2- 172.21.15.19:0/259482292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad93b0 0x5578eaaec6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.354+0000 7f8c77400640 1 -- 172.21.15.19:0/259482292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad8fd0 msgr2=0x5578eaadc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:33.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.354+0000 7f8c77400640 1 --2- 172.21.15.19:0/259482292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad8fd0 0x5578eaadc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.354+0000 7f8c77400640 1 -- 172.21.15.19:0/259482292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad93b0 msgr2=0x5578eaaec6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.354+0000 7f8c77400640 1 --2- 172.21.15.19:0/259482292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad93b0 0x5578eaaec6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.354+0000 7f8c77400640 1 -- 172.21.15.19:0/259482292 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578eaaa6820 con 0x5578eaad82d0 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.354+0000 7f8c77e00640 1 --2- 172.21.15.19:0/259482292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad93b0 0x5578eaaec6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.354+0000 7f8c6fe00640 1 --2- 172.21.15.19:0/259482292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad8fd0 0x5578eaadc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.354+0000 7f8c77400640 1 --2- 172.21.15.19:0/259482292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad82d0 0x5578eaadbf60 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8c6802b880 tx=0x7f8c68030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92e451f14c192b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.355+0000 7f8c67400640 1 -- 172.21.15.19:0/259482292 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68053040 con 0x5578eaad82d0 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.355+0000 7f8c67400640 1 -- 172.21.15.19:0/259482292 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c68037040 con 0x5578eaad82d0 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.355+0000 7f8c67400640 1 -- 172.21.15.19:0/259482292 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68005990 con 0x5578eaad82d0 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.355+0000 7f8c8145f740 1 -- 172.21.15.19:0/259482292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad82d0 msgr2=0x5578eaadbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.355+0000 7f8c8145f740 1 --2- 172.21.15.19:0/259482292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad82d0 0x5578eaadbf60 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8c6802b880 tx=0x7f8c68030000 comp rx=0 tx=0).stop 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.355+0000 7f8c8145f740 1 -- 172.21.15.19:0/259482292 shutdown_connections 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.355+0000 7f8c8145f740 1 --2- 172.21.15.19:0/259482292 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad93b0 0x5578eaaec6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.355+0000 7f8c8145f740 1 --2- 172.21.15.19:0/259482292 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad8fd0 0x5578eaadc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.355+0000 7f8c8145f740 1 --2- 172.21.15.19:0/259482292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad82d0 0x5578eaadbf60 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.355+0000 7f8c8145f740 1 -- 172.21.15.19:0/259482292 >> 172.21.15.19:0/259482292 conn(0x5578eaaac4b0 msgr2=0x5578eaab1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:33.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.356+0000 7f8c8145f740 1 -- 172.21.15.19:0/259482292 shutdown_connections 2024-03-28T13:16:33.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.356+0000 7f8c8145f740 1 -- 172.21.15.19:0/259482292 wait complete. 2024-03-28T13:16:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.356+0000 7f8c8145f740 1 -- start start 2024-03-28T13:16:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.356+0000 7f8c8145f740 1 Processor -- start 2024-03-28T13:16:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.357+0000 7f8c8145f740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad82d0 0x5578eaaed100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.357+0000 7f8c8145f740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad8fd0 0x5578eab56f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.357+0000 7f8c8145f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad93b0 0x5578eaadbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.357+0000 7f8c8145f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5578eaade020 con 0x5578eaad93b0 2024-03-28T13:16:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.357+0000 7f8c8145f740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5578eab08aa0 con 0x5578eaad82d0 2024-03-28T13:16:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.357+0000 7f8c8145f740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5578eaaf1c60 con 0x5578eaad8fd0 2024-03-28T13:16:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.358+0000 7f8c8145f740 10 client.0 _ll_register_callbacks cb 0x7f8c700556b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:16:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.358+0000 7f8c77e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad93b0 0x5578eaadbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.358+0000 7f8c77e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad93b0 0x5578eaadbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39608/0 (socket says 172.21.15.19:39608) 2024-03-28T13:16:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.358+0000 7f8c77e00640 1 -- 172.21.15.19:0/107627150 learned_addr learned my addr 172.21.15.19:0/107627150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.358+0000 7f8c77400640 1 --2- 172.21.15.19:0/107627150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad82d0 0x5578eaaed100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.358+0000 7f8c6fe00640 1 --2- 172.21.15.19:0/107627150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad8fd0 0x5578eab56f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.358+0000 7f8c77e00640 1 -- 172.21.15.19:0/107627150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad8fd0 msgr2=0x5578eab56f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.358+0000 7f8c77e00640 1 --2- 172.21.15.19:0/107627150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad8fd0 0x5578eab56f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.358+0000 7f8c77e00640 1 -- 172.21.15.19:0/107627150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad82d0 msgr2=0x5578eaaed100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.358+0000 7f8c77e00640 1 --2- 172.21.15.19:0/107627150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad82d0 0x5578eaaed100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.358+0000 7f8c77e00640 1 -- 172.21.15.19:0/107627150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578eab713e0 con 0x5578eaad93b0 2024-03-28T13:16:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.358+0000 7f8c77400640 1 --2- 172.21.15.19:0/107627150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad82d0 0x5578eaaed100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.359+0000 7f8c77e00640 1 --2- 172.21.15.19:0/107627150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad93b0 0x5578eaadbd50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000ca70 tx=0x7f8c70089870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e356288acdaa52dc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.359+0000 7f8c64c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.359+0000 7f8c64c00640 1 -- 172.21.15.19:0/107627150 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70059160 con 0x5578eaad93b0 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.359+0000 7f8c64c00640 1 -- 172.21.15.19:0/107627150 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f8c70013c50 con 0x5578eaad93b0 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.359+0000 7f8c8145f740 1 -- 172.21.15.19:0/107627150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c68037040 con 0x5578eaad93b0 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.359+0000 7f8c64c00640 1 -- 172.21.15.19:0/107627150 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8c70057750 con 0x5578eaad93b0 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.359+0000 7f8c8145f740 10 client.7335 fetch_fsmap learned FSMap version 61 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.359+0000 7f8c8145f740 1 -- 172.21.15.19:0/107627150 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) -- 0x5578eaadcee0 con 0x5578eaad93b0 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.359+0000 7f8c64c00640 1 -- 172.21.15.19:0/107627150 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c68037040 con 0x5578eaad93b0 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.359+0000 7f8c64c00640 1 -- 172.21.15.19:0/107627150 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 61) ==== 3494+0+0 (secure 0 0 0) 0x7f8c700587e0 con 0x5578eaad93b0 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.359+0000 7f8c8145f740 10 client.7335 fetch_fsmap finished waiting for FSMap version 61 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.360+0000 7f8c8145f740 10 client.7335 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.360+0000 7f8c8145f740 20 client.7335 populate_metadata read hostname 'smithi019' 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.360+0000 7f8c8145f740 1 --2- 172.21.15.19:0/107627150 >> [v2:172.21.15.188:6834/2260087679,v1:172.21.15.188:6836/2260087679] conn(0x5578eab75000 0x5578eab19840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.360+0000 7f8c8145f740 4 client.7335 mds_command: new command op to 6043 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T13:16:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.360+0000 7f8c8145f740 1 -- 172.21.15.19:0/107627150 --> [v2:172.21.15.188:6834/2260087679,v1:172.21.15.188:6836/2260087679] -- command(tid 0: {"prefix": "status"}) -- 0x5578eaa1d820 con 0x5578eab75000 2024-03-28T13:16:33.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.360+0000 7f8c77400640 1 --2- 172.21.15.19:0/107627150 >> [v2:172.21.15.188:6834/2260087679,v1:172.21.15.188:6836/2260087679] conn(0x5578eab75000 0x5578eab19840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.361+0000 7f8c77400640 1 --2- 172.21.15.19:0/107627150 >> [v2:172.21.15.188:6834/2260087679,v1:172.21.15.188:6836/2260087679] conn(0x5578eab75000 0x5578eab19840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a86971f6a3e20a6 server_cookie=4b25778f650451a9 in_seq=0 out_seq=0 2024-03-28T13:16:33.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.361+0000 7f8c64c00640 10 client.7335 ms_handle_connect on v2:172.21.15.188:6834/2260087679 2024-03-28T13:16:33.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.362+0000 7f8c64c00640 1 -- 172.21.15.19:0/107627150 <== mds.0 v2:172.21.15.188:6834/2260087679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5578eaa1d820 con 0x5578eab75000 2024-03-28T13:16:33.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.362+0000 7f8c64c00640 10 client.7335 handle_command_reply: tid=0 2024-03-28T13:16:33.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.362+0000 7f8c8145f740 1 client.7335 shutdown 2024-03-28T13:16:33.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.362+0000 7f8c8145f740 1 -- 172.21.15.19:0/107627150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad93b0 msgr2=0x5578eaadbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:33.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.362+0000 7f8c8145f740 1 --2- 172.21.15.19:0/107627150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad93b0 0x5578eaadbd50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000ca70 tx=0x7f8c70089870 comp rx=0 tx=0).stop 2024-03-28T13:16:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.362+0000 7f8c8145f740 1 -- 172.21.15.19:0/107627150 shutdown_connections 2024-03-28T13:16:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.362+0000 7f8c8145f740 1 --2- 172.21.15.19:0/107627150 >> [v2:172.21.15.188:6834/2260087679,v1:172.21.15.188:6836/2260087679] conn(0x5578eab75000 0x5578eab19840 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.362+0000 7f8c8145f740 1 --2- 172.21.15.19:0/107627150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578eaad93b0 0x5578eaadbd50 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.362+0000 7f8c8145f740 1 --2- 172.21.15.19:0/107627150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5578eaad8fd0 0x5578eab56f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.362+0000 7f8c64c00640 0 client.7335 ms_handle_reset on v2:172.21.15.188:6834/2260087679 2024-03-28T13:16:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.362+0000 7f8c8145f740 1 --2- 172.21.15.19:0/107627150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5578eaad82d0 0x5578eaaed100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.362+0000 7f8c8145f740 1 -- 172.21.15.19:0/107627150 >> 172.21.15.19:0/107627150 conn(0x5578eaaac4b0 msgr2=0x5578eaab1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.362+0000 7f8c8145f740 1 -- 172.21.15.19:0/107627150 shutdown_connections 2024-03-28T13:16:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.362+0000 7f8c8145f740 1 -- 172.21.15.19:0/107627150 wait complete. 2024-03-28T13:16:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.362+0000 7f8c8145f740 20 client.7335 trim_cache size 0 max 16384 2024-03-28T13:16:33.364 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:16:33.365 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "e6b569d9-f2ce-49dc-bbe7-c0ab33080203", 2024-03-28T13:16:33.365 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-03-28T13:16:33.365 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 6043, 2024-03-28T13:16:33.365 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-03-28T13:16:33.365 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-03-28T13:16:33.365 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-03-28T13:16:33.365 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 6.7961193160000004, 2024-03-28T13:16:33.365 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 61, 2024-03-28T13:16:33.365 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 92, 2024-03-28T13:16:33.365 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 92, 2024-03-28T13:16:33.365 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 13.149970890000001 2024-03-28T13:16:33.365 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:16:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.363+0000 7f8c7e800640 1 -- 172.21.15.19:0/721633974 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8c5406c530 msgr2=0x7f8c5406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.363+0000 7f8c7e800640 1 --2- 172.21.15.19:0/721633974 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8c5406c530 0x7f8c5406e9f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8c68036000 tx=0x7f8c6803d040 comp rx=0 tx=0).stop 2024-03-28T13:16:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.363+0000 7f8c7e800640 1 -- 172.21.15.19:0/721633974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c78163ac0 msgr2=0x7f8c78160240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.363+0000 7f8c7e800640 1 --2- 172.21.15.19:0/721633974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c78163ac0 0x7f8c78160240 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000caa0 tx=0x7f8c7000cf60 comp rx=0 tx=0).stop 2024-03-28T13:16:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.363+0000 7f8c7e800640 1 -- 172.21.15.19:0/721633974 shutdown_connections 2024-03-28T13:16:33.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.363+0000 7f8c7e800640 1 --2- 172.21.15.19:0/721633974 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8c5406c530 0x7f8c5406e9f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.363+0000 7f8c7e800640 1 --2- 172.21.15.19:0/721633974 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c78163ac0 0x7f8c78160240 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.363+0000 7f8c7e800640 1 --2- 172.21.15.19:0/721633974 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c78162920 0x7f8c7809c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.363+0000 7f8c7e800640 1 --2- 172.21.15.19:0/721633974 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c780d0280 0x7f8c7809a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.363+0000 7f8c7e800640 1 -- 172.21.15.19:0/721633974 >> 172.21.15.19:0/721633974 conn(0x7f8c7806b750 msgr2=0x7f8c7815d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.363+0000 7f8c7e800640 1 -- 172.21.15.19:0/721633974 shutdown_connections 2024-03-28T13:16:33.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.363+0000 7f8c7e800640 1 -- 172.21.15.19:0/721633974 wait complete. 2024-03-28T13:16:33.381 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:16:33.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.604+0000 7f52e4800640 1 Processor -- start 2024-03-28T13:16:33.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.604+0000 7f52e4800640 1 -- start start 2024-03-28T13:16:33.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.606+0000 7f52e4800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f52dc153580 0x7f52dc0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.606+0000 7f52e4800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f52dc0af4b0 0x7f52dc1600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.606+0000 7f52e4800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52dc0afab0 con 0x7f52dc0d07c0 2024-03-28T13:16:33.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.606+0000 7f52e4800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f52dc0d0c30 con 0x7f52dc153580 2024-03-28T13:16:33.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.606+0000 7f52e4800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f52dc15cc80 con 0x7f52dc0af4b0 2024-03-28T13:16:33.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.606+0000 7f52e1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f52dc153580 0x7f52dc0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.606+0000 7f52e1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f52dc153580 0x7f52dc0d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39624/0 (socket says 172.21.15.19:39624) 2024-03-28T13:16:33.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.606+0000 7f52e1e00640 1 -- 172.21.15.19:0/1301293985 learned_addr learned my addr 172.21.15.19:0/1301293985 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:33.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.607+0000 7f52e2800640 1 --2- 172.21.15.19:0/1301293985 >> v2:172.21.15.188:3300/0 conn(0x7f52dc0af4b0 0x7f52dc1600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.607+0000 7f52e1e00640 1 -- 172.21.15.19:0/1301293985 >> v2:172.21.15.188:3300/0 conn(0x7f52dc0af4b0 msgr2=0x7f52dc1600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:33.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.607+0000 7f52e1e00640 1 --2- 172.21.15.19:0/1301293985 >> v2:172.21.15.188:3300/0 conn(0x7f52dc0af4b0 0x7f52dc1600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.607+0000 7f52e1e00640 1 -- 172.21.15.19:0/1301293985 >> v1:172.21.15.188:6789/0 conn(0x7f52dc0d07c0 legacy=0x7f52dc0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:33.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.607+0000 7f52e1e00640 1 -- 172.21.15.19:0/1301293985 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc15da00 con 0x7f52dc153580 2024-03-28T13:16:33.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.609+0000 7f52e1e00640 1 --2- 172.21.15.19:0/1301293985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52dc153580 0x7f52dc0d0280 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f52dc15a6b0 tx=0x7f52d002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3acebbdecdce48 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:33.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.609+0000 7f52e0a00640 1 -- 172.21.15.19:0/1301293985 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0035020 con 0x7f52dc153580 2024-03-28T13:16:33.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.609+0000 7f52e0a00640 1 -- 172.21.15.19:0/1301293985 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d0031870 con 0x7f52dc153580 2024-03-28T13:16:33.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.609+0000 7f52e0a00640 1 -- 172.21.15.19:0/1301293985 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0031c40 con 0x7f52dc153580 2024-03-28T13:16:33.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.610+0000 7f52e4800640 1 -- 172.21.15.19:0/1301293985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52dc153580 msgr2=0x7f52dc0d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:33.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.610+0000 7f52e4800640 1 --2- 172.21.15.19:0/1301293985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52dc153580 0x7f52dc0d0280 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f52dc15a6b0 tx=0x7f52d002b910 comp rx=0 tx=0).stop 2024-03-28T13:16:33.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.610+0000 7f52e4800640 1 -- 172.21.15.19:0/1301293985 shutdown_connections 2024-03-28T13:16:33.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.610+0000 7f52e4800640 1 --2- 172.21.15.19:0/1301293985 >> v2:172.21.15.188:3300/0 conn(0x7f52dc0af4b0 0x7f52dc1600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.610+0000 7f52e4800640 1 --2- 172.21.15.19:0/1301293985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52dc153580 0x7f52dc0d0280 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.610+0000 7f52e4800640 1 -- 172.21.15.19:0/1301293985 >> 172.21.15.19:0/1301293985 conn(0x7f52dc06b750 msgr2=0x7f52dc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:33.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.610+0000 7f52e4800640 1 -- 172.21.15.19:0/1301293985 shutdown_connections 2024-03-28T13:16:33.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.610+0000 7f52e4800640 1 -- 172.21.15.19:0/1301293985 wait complete. 2024-03-28T13:16:33.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.612+0000 7f52e4800640 1 Processor -- start 2024-03-28T13:16:33.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.612+0000 7f52e4800640 1 -- start start 2024-03-28T13:16:33.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.613+0000 7f52e4800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52dc0af4b0 0x7f52dc1f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.613+0000 7f52e4800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52dc0d07c0 0x7f52dc153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.613+0000 7f52e4800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52dc153580 0x7f52dc0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.613+0000 7f52e4800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc15d4e0 con 0x7f52dc0af4b0 2024-03-28T13:16:33.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.613+0000 7f52e4800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f52dc15d200 con 0x7f52dc153580 2024-03-28T13:16:33.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.613+0000 7f52e4800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f52dc153960 con 0x7f52dc0d07c0 2024-03-28T13:16:33.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.613+0000 7f52e1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52dc0af4b0 0x7f52dc1f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.613+0000 7f52e1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52dc0af4b0 0x7f52dc1f3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39626/0 (socket says 172.21.15.19:39626) 2024-03-28T13:16:33.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.613+0000 7f52e1e00640 1 -- 172.21.15.19:0/3306196622 learned_addr learned my addr 172.21.15.19:0/3306196622 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.614+0000 7f52e2800640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52dc153580 0x7f52dc0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.614+0000 7f52e1e00640 1 -- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52dc0d07c0 msgr2=0x7f52dc153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.614+0000 7f52e1400640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52dc0d07c0 0x7f52dc153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.614+0000 7f52e1e00640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52dc0d07c0 0x7f52dc153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.614+0000 7f52e1e00640 1 -- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52dc153580 msgr2=0x7f52dc0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.614+0000 7f52e1e00640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52dc153580 0x7f52dc0cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.614+0000 7f52e1e00640 1 -- 172.21.15.19:0/3306196622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52dc154650 con 0x7f52dc0af4b0 2024-03-28T13:16:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.614+0000 7f52e2800640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52dc153580 0x7f52dc0cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.614+0000 7f52e1400640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52dc0d07c0 0x7f52dc153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.614+0000 7f52e1e00640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52dc0af4b0 0x7f52dc1f3df0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f52d002b950 tx=0x7f52d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.614+0000 7f52d6000640 1 -- 172.21.15.19:0/3306196622 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0040070 con 0x7f52dc0af4b0 2024-03-28T13:16:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.614+0000 7f52d6000640 1 -- 172.21.15.19:0/3306196622 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d0030d00 con 0x7f52dc0af4b0 2024-03-28T13:16:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.614+0000 7f52d6000640 1 -- 172.21.15.19:0/3306196622 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0035040 con 0x7f52dc0af4b0 2024-03-28T13:16:33.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.614+0000 7f52e4800640 1 -- 172.21.15.19:0/3306196622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52dc154930 con 0x7f52dc0af4b0 2024-03-28T13:16:33.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.615+0000 7f52e4800640 1 -- 172.21.15.19:0/3306196622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52dc154ef0 con 0x7f52dc0af4b0 2024-03-28T13:16:33.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.616+0000 7f52e4800640 1 -- 172.21.15.19:0/3306196622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52dc0685c0 con 0x7f52dc0af4b0 2024-03-28T13:16:33.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.617+0000 7f52d6000640 1 -- 172.21.15.19:0/3306196622 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f52d004b060 con 0x7f52dc0af4b0 2024-03-28T13:16:33.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.618+0000 7f52d6000640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52c006c530 0x7f52c006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:33.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.618+0000 7f52d6000640 1 -- 172.21.15.19:0/3306196622 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f52d00ad530 con 0x7f52dc0af4b0 2024-03-28T13:16:33.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.619+0000 7f52e1400640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52c006c530 0x7f52c006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:33.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.620+0000 7f52e1400640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52c006c530 0x7f52c006e9f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f52dc147e60 tx=0x7f52cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:33.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.623+0000 7f52d6000640 1 -- 172.21.15.19:0/3306196622 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f52d007bab0 con 0x7f52dc0af4b0 2024-03-28T13:16:33.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.870+0000 7f52e4800640 1 -- 172.21.15.19:0/3306196622 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f52dc071990 con 0x7f52dc0af4b0 2024-03-28T13:16:33.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.872+0000 7f52d6000640 1 -- 172.21.15.19:0/3306196622 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) ==== 53+0+73 (secure 0 0 0) 0x7f52d007b3b0 con 0x7f52dc0af4b0 2024-03-28T13:16:33.872 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:16:33.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.875+0000 7f52e4800640 1 -- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52c006c530 msgr2=0x7f52c006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.875+0000 7f52e4800640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52c006c530 0x7f52c006e9f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f52dc147e60 tx=0x7f52cc009040 comp rx=0 tx=0).stop 2024-03-28T13:16:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.875+0000 7f52e4800640 1 -- 172.21.15.19:0/3306196622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52dc0af4b0 msgr2=0x7f52dc1f3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.875+0000 7f52e4800640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52dc0af4b0 0x7f52dc1f3df0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f52d002b950 tx=0x7f52d00046c0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.875+0000 7f52e4800640 1 -- 172.21.15.19:0/3306196622 shutdown_connections 2024-03-28T13:16:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.875+0000 7f52e4800640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f52c006c530 0x7f52c006e9f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.875+0000 7f52e4800640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52dc153580 0x7f52dc0cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.875+0000 7f52e4800640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52dc0d07c0 0x7f52dc153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.875+0000 7f52e4800640 1 --2- 172.21.15.19:0/3306196622 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52dc0af4b0 0x7f52dc1f3df0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.875+0000 7f52e4800640 1 -- 172.21.15.19:0/3306196622 >> 172.21.15.19:0/3306196622 conn(0x7f52dc06b750 msgr2=0x7f52dc15efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.875+0000 7f52e4800640 1 -- 172.21.15.19:0/3306196622 shutdown_connections 2024-03-28T13:16:33.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:33.875+0000 7f52e4800640 1 -- 172.21.15.19:0/3306196622 wait complete. 2024-03-28T13:16:33.893 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-28T13:16:33.893 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:16:33.893 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:16:33.893 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:16:33.893 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:16:33.893 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-28T13:16:33.893 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-28T13:16:33.893 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:16:33.894 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:16:33.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:33.894 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:16:33.949 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:16:33.949 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:16:33.949 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:16:33.949 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:33.949 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:16:33.949 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:33.950 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:16:33.950 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:16:33.950 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:16:33.950 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:16:33.950 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:16:33.950 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:16:33.950 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:16:33.950 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:16:33.950 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:16:33.950 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:16:33.950 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42161sec preferred_lft 42161sec 2024-03-28T13:16:33.950 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:16:33.950 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:33.950 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether e6:17:c5:8e:2b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::e417:c5ff:fe8e:2bcf/64 scope link 2024-03-28T13:16:33.951 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:33.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:33.953 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:16:34.047 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:16:34.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:34.051 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:16:34.106 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:16:34.107 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:16:34.107 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:16:34.107 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:16:34.107 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:16:34.108 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:16:34.108 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:16:34.108 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:16:34.108 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:16:34.108 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:16:34.108 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:16:34.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:16:34.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:16:34.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:34.374 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:16:34.428 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:16:34.428 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:16:34.429 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:16:34.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:34.429 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:16:34.529 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:16:34.529 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:16:34.587 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:16:34.587 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:16:34.587 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:16:34.642 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:16:34.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:34.642 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:16:34.775 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:16:34.775 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:16:34.776 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:16:34.798 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:16:34.798 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:16:34.798 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:16:34.798 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:16:34.798 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:16:34.827 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:16:34.827 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:16:34.828 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:16:34.828 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:16:34.828 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:16:34.828 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:16:34.828 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:16:34.828 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:16:34.831 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:16:34.831 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:16:34.831 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:16:34.831 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:16:34.831 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:16:34.836 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:16:34.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:34.836 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:16:34.927 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:16:35.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:16:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:16:35.069 INFO:teuthology.orchestra.run.smithi019.stdout:client5687 2024-03-28T13:16:35.070 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:16:35.070 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:16:35.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:35.123 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-03-28T13:16:35.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:35.190 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-03-28T13:16:35.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:35.251 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-03-28T13:16:35.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:35.311 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-03-28T13:16:35.379 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:16:35.379 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:16:35.379 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:16:35.379 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:16:35.379 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-03-28T13:16:35.379 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:16:35.531 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627776 2024-03-28T13:16:35.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:16:35.546 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:16:35.546 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:16:35.643 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:16:35.643 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:16:35.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:16:35.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.922+0000 7ff486400640 1 Processor -- start 2024-03-28T13:16:35.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.922+0000 7ff486400640 1 -- start start 2024-03-28T13:16:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.924+0000 7ff486400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff480070250 0x7ff4800d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.924+0000 7ff486400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff4800d26e0 0x7ff4800d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.924+0000 7ff486400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4800d2ed0 con 0x7ff48013f100 2024-03-28T13:16:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.924+0000 7ff486400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff480142270 con 0x7ff480070250 2024-03-28T13:16:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.924+0000 7ff486400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff48013f4e0 con 0x7ff4800d26e0 2024-03-28T13:16:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.924+0000 7ff47fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff480070250 0x7ff4800d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.924+0000 7ff47fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff480070250 0x7ff4800d21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60748/0 (socket says 172.21.15.19:60748) 2024-03-28T13:16:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.924+0000 7ff47fe00640 1 -- 172.21.15.19:0/1366302036 learned_addr learned my addr 172.21.15.19:0/1366302036 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:35.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.924+0000 7ff485a00640 1 --2- 172.21.15.19:0/1366302036 >> v2:172.21.15.188:3300/0 conn(0x7ff4800d26e0 0x7ff4800d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.925+0000 7ff47fe00640 1 -- 172.21.15.19:0/1366302036 >> v2:172.21.15.188:3300/0 conn(0x7ff4800d26e0 msgr2=0x7ff4800d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.925+0000 7ff47fe00640 1 --2- 172.21.15.19:0/1366302036 >> v2:172.21.15.188:3300/0 conn(0x7ff4800d26e0 0x7ff4800d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.925+0000 7ff47fe00640 1 -- 172.21.15.19:0/1366302036 >> v1:172.21.15.188:6789/0 conn(0x7ff48013f100 legacy=0x7ff48006fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:35.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.925+0000 7ff47fe00640 1 -- 172.21.15.19:0/1366302036 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff480141940 con 0x7ff480070250 2024-03-28T13:16:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.926+0000 7ff47fe00640 1 --2- 172.21.15.19:0/1366302036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff480070250 0x7ff4800d21a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff4801423f0 tx=0x7ff46c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52d92a9d13c7c3d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.927+0000 7ff47f400640 1 -- 172.21.15.19:0/1366302036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c034070 con 0x7ff480070250 2024-03-28T13:16:35.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.927+0000 7ff47f400640 1 -- 172.21.15.19:0/1366302036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff46c030860 con 0x7ff480070250 2024-03-28T13:16:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.927+0000 7ff47f400640 1 -- 172.21.15.19:0/1366302036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c030c30 con 0x7ff480070250 2024-03-28T13:16:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.927+0000 7ff486400640 1 -- 172.21.15.19:0/1366302036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff480070250 msgr2=0x7ff4800d21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.927+0000 7ff486400640 1 --2- 172.21.15.19:0/1366302036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff480070250 0x7ff4800d21a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7ff4801423f0 tx=0x7ff46c02b9a0 comp rx=0 tx=0).stop 2024-03-28T13:16:35.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.927+0000 7ff486400640 1 -- 172.21.15.19:0/1366302036 shutdown_connections 2024-03-28T13:16:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.927+0000 7ff486400640 1 --2- 172.21.15.19:0/1366302036 >> v2:172.21.15.188:3300/0 conn(0x7ff4800d26e0 0x7ff4800d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.927+0000 7ff486400640 1 --2- 172.21.15.19:0/1366302036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff480070250 0x7ff4800d21a0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.927+0000 7ff486400640 1 -- 172.21.15.19:0/1366302036 >> 172.21.15.19:0/1366302036 conn(0x7ff480067460 msgr2=0x7ff480067840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.928+0000 7ff486400640 1 -- 172.21.15.19:0/1366302036 shutdown_connections 2024-03-28T13:16:35.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.928+0000 7ff486400640 1 -- 172.21.15.19:0/1366302036 wait complete. 2024-03-28T13:16:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.929+0000 7ff486400640 1 Processor -- start 2024-03-28T13:16:35.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.929+0000 7ff486400640 1 -- start start 2024-03-28T13:16:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.930+0000 7ff486400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff480070250 0x7ff480155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.930+0000 7ff486400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4800d26e0 0x7ff48013fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.930+0000 7ff486400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48013f100 0x7ff480140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.930+0000 7ff486400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff480142700 con 0x7ff4800d26e0 2024-03-28T13:16:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.930+0000 7ff486400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff480141f00 con 0x7ff480070250 2024-03-28T13:16:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.930+0000 7ff486400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff4800d2b30 con 0x7ff48013f100 2024-03-28T13:16:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.931+0000 7ff47fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4800d26e0 0x7ff48013fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.931+0000 7ff47fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4800d26e0 0x7ff48013fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60758/0 (socket says 172.21.15.19:60758) 2024-03-28T13:16:35.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.931+0000 7ff47fe00640 1 -- 172.21.15.19:0/1412131445 learned_addr learned my addr 172.21.15.19:0/1412131445 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:35.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.931+0000 7ff485000640 1 --2- 172.21.15.19:0/1412131445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff480070250 0x7ff480155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:35.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.931+0000 7ff485a00640 1 --2- 172.21.15.19:0/1412131445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48013f100 0x7ff480140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:35.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.931+0000 7ff47fe00640 1 -- 172.21.15.19:0/1412131445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48013f100 msgr2=0x7ff480140150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:35.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.931+0000 7ff47fe00640 1 --2- 172.21.15.19:0/1412131445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48013f100 0x7ff480140150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:35.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.931+0000 7ff47fe00640 1 -- 172.21.15.19:0/1412131445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff480070250 msgr2=0x7ff480155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:35.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.931+0000 7ff47fe00640 1 --2- 172.21.15.19:0/1412131445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff480070250 0x7ff480155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:35.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.931+0000 7ff47fe00640 1 -- 172.21.15.19:0/1412131445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4800b3bd0 con 0x7ff4800d26e0 2024-03-28T13:16:35.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.931+0000 7ff485a00640 1 --2- 172.21.15.19:0/1412131445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48013f100 0x7ff480140150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.932+0000 7ff47fe00640 1 --2- 172.21.15.19:0/1412131445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4800d26e0 0x7ff48013fc10 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff46c02b9e0 tx=0x7ff46c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.932+0000 7ff47cc00640 1 -- 172.21.15.19:0/1412131445 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c048070 con 0x7ff4800d26e0 2024-03-28T13:16:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.932+0000 7ff47cc00640 1 -- 172.21.15.19:0/1412131445 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff46c030660 con 0x7ff4800d26e0 2024-03-28T13:16:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.932+0000 7ff47cc00640 1 -- 172.21.15.19:0/1412131445 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c034040 con 0x7ff4800d26e0 2024-03-28T13:16:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.932+0000 7ff486400640 1 -- 172.21.15.19:0/1412131445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4800b3eb0 con 0x7ff4800d26e0 2024-03-28T13:16:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.932+0000 7ff486400640 1 -- 172.21.15.19:0/1412131445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4800b4470 con 0x7ff4800d26e0 2024-03-28T13:16:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.933+0000 7ff486400640 1 -- 172.21.15.19:0/1412131445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff480075940 con 0x7ff4800d26e0 2024-03-28T13:16:35.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.934+0000 7ff47cc00640 1 -- 172.21.15.19:0/1412131445 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff46c037070 con 0x7ff4800d26e0 2024-03-28T13:16:35.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.935+0000 7ff47cc00640 1 --2- 172.21.15.19:0/1412131445 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff46006c530 0x7ff46006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:35.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.935+0000 7ff47cc00640 1 -- 172.21.15.19:0/1412131445 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7ff46c0ad770 con 0x7ff4800d26e0 2024-03-28T13:16:35.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.936+0000 7ff485000640 1 --2- 172.21.15.19:0/1412131445 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff46006c530 0x7ff46006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:35.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.937+0000 7ff485000640 1 --2- 172.21.15.19:0/1412131445 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff46006c530 0x7ff46006e9f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff470001ca0 tx=0x7ff470009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:35.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:35.940+0000 7ff47cc00640 1 -- 172.21.15.19:0/1412131445 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff46c07be10 con 0x7ff4800d26e0 2024-03-28T13:16:36.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.143+0000 7ff486400640 1 -- 172.21.15.19:0/1412131445 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff480071d40 con 0x7ff4800d26e0 2024-03-28T13:16:36.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.145+0000 7ff47cc00640 1 -- 172.21.15.19:0/1412131445 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+16672 (secure 0 0 0) 0x7ff46c07b710 con 0x7ff4800d26e0 2024-03-28T13:16:36.146 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:36.146 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":92,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:16:27.532934+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:16:23.901120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-28T13:16:24.990904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:16:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.148+0000 7ff486400640 1 -- 172.21.15.19:0/1412131445 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff46006c530 msgr2=0x7ff46006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.148+0000 7ff486400640 1 --2- 172.21.15.19:0/1412131445 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff46006c530 0x7ff46006e9f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff470001ca0 tx=0x7ff470009040 comp rx=0 tx=0).stop 2024-03-28T13:16:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.148+0000 7ff486400640 1 -- 172.21.15.19:0/1412131445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4800d26e0 msgr2=0x7ff48013fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.148+0000 7ff486400640 1 --2- 172.21.15.19:0/1412131445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4800d26e0 0x7ff48013fc10 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff46c02b9e0 tx=0x7ff46c004730 comp rx=0 tx=0).stop 2024-03-28T13:16:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.149+0000 7ff486400640 1 -- 172.21.15.19:0/1412131445 shutdown_connections 2024-03-28T13:16:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.149+0000 7ff486400640 1 --2- 172.21.15.19:0/1412131445 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff46006c530 0x7ff46006e9f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.149+0000 7ff486400640 1 --2- 172.21.15.19:0/1412131445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff48013f100 0x7ff480140150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.149+0000 7ff486400640 1 --2- 172.21.15.19:0/1412131445 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff4800d26e0 0x7ff48013fc10 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.149+0000 7ff486400640 1 --2- 172.21.15.19:0/1412131445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff480070250 0x7ff480155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.149+0000 7ff486400640 1 -- 172.21.15.19:0/1412131445 >> 172.21.15.19:0/1412131445 conn(0x7ff480067460 msgr2=0x7ff48006c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.149+0000 7ff486400640 1 -- 172.21.15.19:0/1412131445 shutdown_connections 2024-03-28T13:16:36.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.149+0000 7ff486400640 1 -- 172.21.15.19:0/1412131445 wait complete. 2024-03-28T13:16:36.169 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:16:23.901120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:16:24.990904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:16:36.170 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:16:36.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.395+0000 7fb103600640 1 Processor -- start 2024-03-28T13:16:36.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.395+0000 7fb103600640 1 -- start start 2024-03-28T13:16:36.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.397+0000 7fb103600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb0fc0d0280 0x7fb0fc0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:36.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.397+0000 7fb103600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0fc141c10 0x7fb0fc071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:36.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.397+0000 7fb103600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb0fc071eb0 0x7fb0fc06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:36.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.397+0000 7fb103600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb0fc15ec80 con 0x7fb0fc0d0280 2024-03-28T13:16:36.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.397+0000 7fb103600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb0fc0d0ba0 con 0x7fb0fc071eb0 2024-03-28T13:16:36.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.397+0000 7fb103600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb0fc15ef20 con 0x7fb0fc141c10 2024-03-28T13:16:36.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.397+0000 7fb100c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb0fc0d0280 0x7fb0fc0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:36.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.397+0000 7fb100c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb0fc0d0280 0x7fb0fc0d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60766/0 (socket says 172.21.15.19:60766) 2024-03-28T13:16:36.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.397+0000 7fb100c00640 1 -- 172.21.15.19:0/1812198206 learned_addr learned my addr 172.21.15.19:0/1812198206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:36.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.398+0000 7fb101600640 1 --2- 172.21.15.19:0/1812198206 >> v2:172.21.15.188:3300/0 conn(0x7fb0fc071eb0 0x7fb0fc06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:36.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.398+0000 7fb0fbe00640 1 --2- 172.21.15.19:0/1812198206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0fc141c10 0x7fb0fc071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:36.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.398+0000 7fb100c00640 1 -- 172.21.15.19:0/1812198206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0fc141c10 msgr2=0x7fb0fc071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:36.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.398+0000 7fb100c00640 1 --2- 172.21.15.19:0/1812198206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0fc141c10 0x7fb0fc071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.398+0000 7fb100c00640 1 -- 172.21.15.19:0/1812198206 >> v2:172.21.15.188:3300/0 conn(0x7fb0fc071eb0 msgr2=0x7fb0fc06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:36.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.398+0000 7fb100c00640 1 --2- 172.21.15.19:0/1812198206 >> v2:172.21.15.188:3300/0 conn(0x7fb0fc071eb0 0x7fb0fc06d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.398+0000 7fb100c00640 1 -- 172.21.15.19:0/1812198206 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc15f200 con 0x7fb0fc0d0280 2024-03-28T13:16:36.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.398+0000 7fb101600640 1 --2- 172.21.15.19:0/1812198206 >> v2:172.21.15.188:3300/0 conn(0x7fb0fc071eb0 0x7fb0fc06d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:36.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.398+0000 7fb0fbe00640 1 --2- 172.21.15.19:0/1812198206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0fc141c10 0x7fb0fc071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:36.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.399+0000 7fb100c00640 1 --2- 172.21.15.19:0/1812198206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc0d0280 0x7fb0fc0d0660 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc073ac0 tx=0x7fb0ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bf5a334a7978db server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:36.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.399+0000 7fb0fb400640 1 -- 172.21.15.19:0/1812198206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec035020 con 0x7fb0fc0d0280 2024-03-28T13:16:36.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.399+0000 7fb0fb400640 1 -- 172.21.15.19:0/1812198206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ec031870 con 0x7fb0fc0d0280 2024-03-28T13:16:36.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.399+0000 7fb0fb400640 1 -- 172.21.15.19:0/1812198206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec031c40 con 0x7fb0fc0d0280 2024-03-28T13:16:36.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.400+0000 7fb103600640 1 -- 172.21.15.19:0/1812198206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc0d0280 msgr2=0x7fb0fc0d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:36.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.400+0000 7fb103600640 1 --2- 172.21.15.19:0/1812198206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc0d0280 0x7fb0fc0d0660 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb0fc073ac0 tx=0x7fb0ec02b910 comp rx=0 tx=0).stop 2024-03-28T13:16:36.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.400+0000 7fb103600640 1 -- 172.21.15.19:0/1812198206 shutdown_connections 2024-03-28T13:16:36.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.400+0000 7fb103600640 1 --2- 172.21.15.19:0/1812198206 >> v2:172.21.15.188:3300/0 conn(0x7fb0fc071eb0 0x7fb0fc06d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.400+0000 7fb103600640 1 --2- 172.21.15.19:0/1812198206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0fc141c10 0x7fb0fc071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.400+0000 7fb103600640 1 --2- 172.21.15.19:0/1812198206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc0d0280 0x7fb0fc0d0660 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.400+0000 7fb103600640 1 -- 172.21.15.19:0/1812198206 >> 172.21.15.19:0/1812198206 conn(0x7fb0fc06b750 msgr2=0x7fb0fc06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:36.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.400+0000 7fb103600640 1 -- 172.21.15.19:0/1812198206 shutdown_connections 2024-03-28T13:16:36.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.400+0000 7fb103600640 1 -- 172.21.15.19:0/1812198206 wait complete. 2024-03-28T13:16:36.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.402+0000 7fb103600640 1 Processor -- start 2024-03-28T13:16:36.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.402+0000 7fb103600640 1 -- start start 2024-03-28T13:16:36.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.403+0000 7fb103600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0fc071eb0 0x7fb0fc099900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:36.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.403+0000 7fb103600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc0d0280 0x7fb0fc09b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:36.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.403+0000 7fb103600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0fc141c10 0x7fb0fc15a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:36.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.403+0000 7fb103600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc158e90 con 0x7fb0fc0d0280 2024-03-28T13:16:36.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.403+0000 7fb103600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc15f7c0 con 0x7fb0fc141c10 2024-03-28T13:16:36.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.403+0000 7fb103600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb0fc159710 con 0x7fb0fc071eb0 2024-03-28T13:16:36.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.403+0000 7fb0fbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc0d0280 0x7fb0fc09b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:36.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.403+0000 7fb0fbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc0d0280 0x7fb0fc09b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60776/0 (socket says 172.21.15.19:60776) 2024-03-28T13:16:36.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.403+0000 7fb0fbe00640 1 -- 172.21.15.19:0/1605065116 learned_addr learned my addr 172.21.15.19:0/1605065116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.404+0000 7fb100c00640 1 --2- 172.21.15.19:0/1605065116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0fc071eb0 0x7fb0fc099900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.404+0000 7fb0fbe00640 1 -- 172.21.15.19:0/1605065116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0fc071eb0 msgr2=0x7fb0fc099900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.404+0000 7fb0fbe00640 1 --2- 172.21.15.19:0/1605065116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0fc071eb0 0x7fb0fc099900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.404+0000 7fb0fbe00640 1 -- 172.21.15.19:0/1605065116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0fc141c10 msgr2=0x7fb0fc15a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.404+0000 7fb0fbe00640 1 --2- 172.21.15.19:0/1605065116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0fc141c10 0x7fb0fc15a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.404+0000 7fb0fbe00640 1 -- 172.21.15.19:0/1605065116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc1599f0 con 0x7fb0fc0d0280 2024-03-28T13:16:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.404+0000 7fb0fbe00640 1 --2- 172.21.15.19:0/1605065116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc0d0280 0x7fb0fc09b1e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400c7f0 tx=0x7fb0e400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.404+0000 7fb0f8c00640 1 -- 172.21.15.19:0/1605065116 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4015070 con 0x7fb0fc0d0280 2024-03-28T13:16:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.405+0000 7fb0f8c00640 1 -- 172.21.15.19:0/1605065116 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e4007820 con 0x7fb0fc0d0280 2024-03-28T13:16:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.405+0000 7fb0f8c00640 1 -- 172.21.15.19:0/1605065116 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4007bf0 con 0x7fb0fc0d0280 2024-03-28T13:16:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.405+0000 7fb103600640 1 -- 172.21.15.19:0/1605065116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0fc159cd0 con 0x7fb0fc0d0280 2024-03-28T13:16:36.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.405+0000 7fb103600640 1 -- 172.21.15.19:0/1605065116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0fc099e40 con 0x7fb0fc0d0280 2024-03-28T13:16:36.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.406+0000 7fb103600640 1 -- 172.21.15.19:0/1605065116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0fc09e150 con 0x7fb0fc0d0280 2024-03-28T13:16:36.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.407+0000 7fb0f8c00640 1 -- 172.21.15.19:0/1605065116 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb0e4018070 con 0x7fb0fc0d0280 2024-03-28T13:16:36.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.408+0000 7fb0f8c00640 1 --2- 172.21.15.19:0/1605065116 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb0d806c530 0x7fb0d806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:36.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.408+0000 7fb0f8c00640 1 -- 172.21.15.19:0/1605065116 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7fb0e408b8c0 con 0x7fb0fc0d0280 2024-03-28T13:16:36.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.408+0000 7fb100c00640 1 --2- 172.21.15.19:0/1605065116 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb0d806c530 0x7fb0d806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:36.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.409+0000 7fb100c00640 1 --2- 172.21.15.19:0/1605065116 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb0d806c530 0x7fb0d806e9f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec036000 tx=0x7fb0ec008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:36.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.412+0000 7fb0f8c00640 1 -- 172.21.15.19:0/1605065116 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb0e4059e40 con 0x7fb0fc0d0280 2024-03-28T13:16:36.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.619+0000 7fb103600640 1 -- 172.21.15.19:0/1605065116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb0fc0685c0 con 0x7fb0fc0d0280 2024-03-28T13:16:36.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.620+0000 7fb0f8c00640 1 -- 172.21.15.19:0/1605065116 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fb0e4059740 con 0x7fb0fc0d0280 2024-03-28T13:16:36.620 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:16:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.623+0000 7fb103600640 1 -- 172.21.15.19:0/1605065116 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb0d806c530 msgr2=0x7fb0d806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.623+0000 7fb103600640 1 --2- 172.21.15.19:0/1605065116 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb0d806c530 0x7fb0d806e9f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec036000 tx=0x7fb0ec008030 comp rx=0 tx=0).stop 2024-03-28T13:16:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.623+0000 7fb103600640 1 -- 172.21.15.19:0/1605065116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc0d0280 msgr2=0x7fb0fc09b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.623+0000 7fb103600640 1 --2- 172.21.15.19:0/1605065116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc0d0280 0x7fb0fc09b1e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400c7f0 tx=0x7fb0e400ccb0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.623+0000 7fb103600640 1 -- 172.21.15.19:0/1605065116 shutdown_connections 2024-03-28T13:16:36.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.623+0000 7fb103600640 1 --2- 172.21.15.19:0/1605065116 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb0d806c530 0x7fb0d806e9f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.623+0000 7fb103600640 1 --2- 172.21.15.19:0/1605065116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0fc141c10 0x7fb0fc15a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.623+0000 7fb103600640 1 --2- 172.21.15.19:0/1605065116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb0fc0d0280 0x7fb0fc09b1e0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.623+0000 7fb103600640 1 --2- 172.21.15.19:0/1605065116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0fc071eb0 0x7fb0fc099900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.623+0000 7fb103600640 1 -- 172.21.15.19:0/1605065116 >> 172.21.15.19:0/1605065116 conn(0x7fb0fc06b750 msgr2=0x7fb0fc0b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:36.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.623+0000 7fb103600640 1 -- 172.21.15.19:0/1605065116 shutdown_connections 2024-03-28T13:16:36.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.624+0000 7fb103600640 1 -- 172.21.15.19:0/1605065116 wait complete. 2024-03-28T13:16:36.643 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:16:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.865+0000 7f41d7e00640 1 Processor -- start 2024-03-28T13:16:36.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.865+0000 7f41d7e00640 1 -- start start 2024-03-28T13:16:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.867+0000 7f41d7e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f41d813f190 0x7f41d80cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.867+0000 7f41d7e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d80ce890 con 0x7f41d806dd30 2024-03-28T13:16:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.867+0000 7f41d7e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41d80ceb50 con 0x7f41d80cc880 2024-03-28T13:16:36.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.867+0000 7f41d7e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f41d80cee30 con 0x7f41d813f190 2024-03-28T13:16:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.868+0000 7f41d7400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f41d806dd30 0x7f41d815ff10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53628/0 (socket says 172.21.15.19:53628) 2024-03-28T13:16:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.868+0000 7f41d7400640 1 -- 172.21.15.19:0/2669555538 learned_addr learned my addr 172.21.15.19:0/2669555538 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.868+0000 7f41d6a00640 1 --2- 172.21.15.19:0/2669555538 >> v2:172.21.15.188:3300/0 conn(0x7f41d813f190 0x7f41d80cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.868+0000 7f41d6000640 1 -- 172.21.15.19:0/2669555538 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2345517768 0 0) 0x7f41d80ce890 con 0x7f41d806dd30 2024-03-28T13:16:36.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.868+0000 7f41d6000640 1 -- 172.21.15.19:0/2669555538 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41bc003520 con 0x7f41d806dd30 2024-03-28T13:16:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41d6000640 1 -- 172.21.15.19:0/2669555538 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1008171927 0 0) 0x7f41bc003520 con 0x7f41d806dd30 2024-03-28T13:16:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41d6000640 1 -- 172.21.15.19:0/2669555538 >> v2:172.21.15.188:3300/0 conn(0x7f41d813f190 msgr2=0x7f41d80cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41d6000640 1 --2- 172.21.15.19:0/2669555538 >> v2:172.21.15.188:3300/0 conn(0x7f41d813f190 0x7f41d80cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41d6000640 1 -- 172.21.15.19:0/2669555538 >> v1:172.21.15.188:6789/0 conn(0x7f41d80cc880 legacy=0x7f41d806d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41d6000640 1 -- 172.21.15.19:0/2669555538 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d80cf110 con 0x7f41d806dd30 2024-03-28T13:16:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41d6000640 1 -- 172.21.15.19:0/2669555538 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f41d0001670 con 0x7f41d806dd30 2024-03-28T13:16:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41d6000640 1 -- 172.21.15.19:0/2669555538 >> v1:172.21.15.19:6789/0 conn(0x7f41d806dd30 legacy=0x7f41d815ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41d6000640 1 --2- 172.21.15.19:0/2669555538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41bc0039b0 0x7f41bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41d6000640 1 --2- 172.21.15.19:0/2669555538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41bc004800 0x7f41bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:36.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41d6000640 1 --2- 172.21.15.19:0/2669555538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41bc0051c0 0x7f41bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41d6000640 1 -- 172.21.15.19:0/2669555538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41d80cee30 con 0x7f41bc0039b0 2024-03-28T13:16:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41d6000640 1 -- 172.21.15.19:0/2669555538 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc00c010 con 0x7f41bc0051c0 2024-03-28T13:16:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41d6000640 1 -- 172.21.15.19:0/2669555538 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f41bc00c210 con 0x7f41bc004800 2024-03-28T13:16:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41d6a00640 1 --2- 172.21.15.19:0/2669555538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41bc0039b0 0x7f41bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41d7400640 1 --2- 172.21.15.19:0/2669555538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41bc0051c0 0x7f41bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.869+0000 7f41ce000640 1 --2- 172.21.15.19:0/2669555538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41bc004800 0x7f41bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.870+0000 7f41d6a00640 1 -- 172.21.15.19:0/2669555538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41bc004800 msgr2=0x7f41bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.870+0000 7f41d6a00640 1 --2- 172.21.15.19:0/2669555538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41bc004800 0x7f41bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.870+0000 7f41d6a00640 1 -- 172.21.15.19:0/2669555538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41bc0051c0 msgr2=0x7f41bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.870+0000 7f41d6a00640 1 --2- 172.21.15.19:0/2669555538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41bc0051c0 0x7f41bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.870+0000 7f41d6a00640 1 -- 172.21.15.19:0/2669555538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc003520 con 0x7f41bc0039b0 2024-03-28T13:16:36.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.870+0000 7f41d7400640 1 -- 172.21.15.19:0/2669555538 reap_dead start 2024-03-28T13:16:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.871+0000 7f41d6a00640 1 --2- 172.21.15.19:0/2669555538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41bc0039b0 0x7f41bc003d90 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f41c8009fc0 tx=0x7f41bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c4785a7464d77b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.871+0000 7f41d6000640 1 -- 172.21.15.19:0/2669555538 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8039070 con 0x7f41bc0039b0 2024-03-28T13:16:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.871+0000 7f41d6000640 1 -- 172.21.15.19:0/2669555538 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c802cc20 con 0x7f41bc0039b0 2024-03-28T13:16:36.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.871+0000 7f41d6000640 1 -- 172.21.15.19:0/2669555538 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c8009c10 con 0x7f41bc0039b0 2024-03-28T13:16:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.872+0000 7f41d7e00640 1 -- 172.21.15.19:0/2669555538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41bc0039b0 msgr2=0x7f41bc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.872+0000 7f41d7e00640 1 --2- 172.21.15.19:0/2669555538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41bc0039b0 0x7f41bc003d90 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f41c8009fc0 tx=0x7f41bc005830 comp rx=0 tx=0).stop 2024-03-28T13:16:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.872+0000 7f41d7e00640 1 -- 172.21.15.19:0/2669555538 shutdown_connections 2024-03-28T13:16:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.872+0000 7f41d7e00640 1 --2- 172.21.15.19:0/2669555538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41bc0039b0 0x7f41bc003d90 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.872+0000 7f41d7e00640 1 -- 172.21.15.19:0/2669555538 >> 172.21.15.19:0/2669555538 conn(0x7f41d80b1ba0 msgr2=0x7f41d80b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.872+0000 7f41d7e00640 1 -- 172.21.15.19:0/2669555538 shutdown_connections 2024-03-28T13:16:36.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.872+0000 7f41d7e00640 1 -- 172.21.15.19:0/2669555538 wait complete. 2024-03-28T13:16:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.875+0000 7f41d7e00640 1 Processor -- start 2024-03-28T13:16:36.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.875+0000 7f41d7e00640 1 -- start start 2024-03-28T13:16:36.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.876+0000 7f41d7e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41d81559f0 0x7f41d815bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:36.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.876+0000 7f41d7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41d8155dd0 0x7f41d8157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.876+0000 7f41d7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41d8157da0 0x7f41d8158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.876+0000 7f41d7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41d80cccf0 con 0x7f41d8155dd0 2024-03-28T13:16:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.876+0000 7f41d7e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41d813f570 con 0x7f41d81559f0 2024-03-28T13:16:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.876+0000 7f41d7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f41d80cce70 con 0x7f41d8157da0 2024-03-28T13:16:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.877+0000 7f41ce000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41d8155dd0 0x7f41d8157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.877+0000 7f41ce000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41d8155dd0 0x7f41d8157860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60802/0 (socket says 172.21.15.19:60802) 2024-03-28T13:16:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.877+0000 7f41ce000640 1 -- 172.21.15.19:0/206945969 learned_addr learned my addr 172.21.15.19:0/206945969 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.877+0000 7f41d6a00640 1 --2- 172.21.15.19:0/206945969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41d81559f0 0x7f41d815bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.877+0000 7f41d7400640 1 --2- 172.21.15.19:0/206945969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41d8157da0 0x7f41d8158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.877+0000 7f41ce000640 1 -- 172.21.15.19:0/206945969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41d8157da0 msgr2=0x7f41d8158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.877+0000 7f41ce000640 1 --2- 172.21.15.19:0/206945969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41d8157da0 0x7f41d8158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.877+0000 7f41ce000640 1 -- 172.21.15.19:0/206945969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41d81559f0 msgr2=0x7f41d815bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.877+0000 7f41ce000640 1 --2- 172.21.15.19:0/206945969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41d81559f0 0x7f41d815bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.877+0000 7f41ce000640 1 -- 172.21.15.19:0/206945969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d815aee0 con 0x7f41d8155dd0 2024-03-28T13:16:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.877+0000 7f41d7400640 1 --2- 172.21.15.19:0/206945969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41d8157da0 0x7f41d8158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:16:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.877+0000 7f41ce000640 1 --2- 172.21.15.19:0/206945969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41d8155dd0 0x7f41d8157860 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f41c000e480 tx=0x7f41c000e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.878+0000 7f41cf400640 1 -- 172.21.15.19:0/206945969 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c00052e0 con 0x7f41d8155dd0 2024-03-28T13:16:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.878+0000 7f41cf400640 1 -- 172.21.15.19:0/206945969 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41c0018070 con 0x7f41d8155dd0 2024-03-28T13:16:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.878+0000 7f41cf400640 1 -- 172.21.15.19:0/206945969 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0003830 con 0x7f41d8155dd0 2024-03-28T13:16:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.878+0000 7f41d7e00640 1 -- 172.21.15.19:0/206945969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d815b1c0 con 0x7f41d8155dd0 2024-03-28T13:16:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.878+0000 7f41d7e00640 1 -- 172.21.15.19:0/206945969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d81e8060 con 0x7f41d8155dd0 2024-03-28T13:16:36.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.879+0000 7f41d7e00640 1 -- 172.21.15.19:0/206945969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d8075940 con 0x7f41d8155dd0 2024-03-28T13:16:36.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.881+0000 7f41cf400640 1 -- 172.21.15.19:0/206945969 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f41c00039d0 con 0x7f41d8155dd0 2024-03-28T13:16:36.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.881+0000 7f41cf400640 1 --2- 172.21.15.19:0/206945969 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41b406c560 0x7f41b406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:36.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.882+0000 7f41cf400640 1 -- 172.21.15.19:0/206945969 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f41c008b3a0 con 0x7f41d8155dd0 2024-03-28T13:16:36.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.882+0000 7f41d6a00640 1 --2- 172.21.15.19:0/206945969 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41b406c560 0x7f41b406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.883+0000 7f41d6a00640 1 --2- 172.21.15.19:0/206945969 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41b406c560 0x7f41b406ea20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f41c8036000 tx=0x7f41c802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:36.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:36.887+0000 7f41cf400640 1 -- 172.21.15.19:0/206945969 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f41c0059a50 con 0x7f41d8155dd0 2024-03-28T13:16:37.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.088+0000 7f41d7e00640 1 -- 172.21.15.19:0/206945969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f41d80b2870 con 0x7f41d8155dd0 2024-03-28T13:16:37.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.089+0000 7f41cf400640 1 -- 172.21.15.19:0/206945969 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f41c0059350 con 0x7f41d8155dd0 2024-03-28T13:16:37.089 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:16:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.092+0000 7f41d7e00640 1 -- 172.21.15.19:0/206945969 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41b406c560 msgr2=0x7f41b406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.092+0000 7f41d7e00640 1 --2- 172.21.15.19:0/206945969 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41b406c560 0x7f41b406ea20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f41c8036000 tx=0x7f41c802e000 comp rx=0 tx=0).stop 2024-03-28T13:16:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.092+0000 7f41d7e00640 1 -- 172.21.15.19:0/206945969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41d8155dd0 msgr2=0x7f41d8157860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.092+0000 7f41d7e00640 1 --2- 172.21.15.19:0/206945969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41d8155dd0 0x7f41d8157860 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f41c000e480 tx=0x7f41c000e940 comp rx=0 tx=0).stop 2024-03-28T13:16:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.092+0000 7f41d7e00640 1 -- 172.21.15.19:0/206945969 shutdown_connections 2024-03-28T13:16:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.092+0000 7f41d7e00640 1 --2- 172.21.15.19:0/206945969 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41b406c560 0x7f41b406ea20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.092+0000 7f41d7e00640 1 --2- 172.21.15.19:0/206945969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41d8157da0 0x7f41d8158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.092+0000 7f41d7e00640 1 --2- 172.21.15.19:0/206945969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41d8155dd0 0x7f41d8157860 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.092+0000 7f41d7e00640 1 --2- 172.21.15.19:0/206945969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41d81559f0 0x7f41d815bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.092+0000 7f41d7e00640 1 -- 172.21.15.19:0/206945969 >> 172.21.15.19:0/206945969 conn(0x7f41d80b1ba0 msgr2=0x7f41d806e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:37.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.092+0000 7f41d7e00640 1 -- 172.21.15.19:0/206945969 shutdown_connections 2024-03-28T13:16:37.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.092+0000 7f41d7e00640 1 -- 172.21.15.19:0/206945969 wait complete. 2024-03-28T13:16:37.112 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:16:37.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.337+0000 7ff20cc00640 1 Processor -- start 2024-03-28T13:16:37.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.337+0000 7ff20cc00640 1 -- start start 2024-03-28T13:16:37.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.339+0000 7ff20cc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff2080d0990 0x7ff2080aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.339+0000 7ff20cc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2080af2e0 0x7ff2081600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.339+0000 7ff20cc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2080afab0 con 0x7ff208153580 2024-03-28T13:16:37.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.339+0000 7ff20cc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff20815cc80 con 0x7ff2080d0990 2024-03-28T13:16:37.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.339+0000 7ff20cc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff20815da00 con 0x7ff2080af2e0 2024-03-28T13:16:37.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.339+0000 7ff205e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff208153580 0x7ff2080d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53640/0 (socket says 172.21.15.19:53640) 2024-03-28T13:16:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.339+0000 7ff205e00640 1 -- 172.21.15.19:0/3001812625 learned_addr learned my addr 172.21.15.19:0/3001812625 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.339+0000 7ff206800640 1 --2- 172.21.15.19:0/3001812625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2080af2e0 0x7ff2081600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.339+0000 7ff205400640 1 --2- 172.21.15.19:0/3001812625 >> v2:172.21.15.188:3300/0 conn(0x7ff2080d0990 0x7ff2080aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 -- 172.21.15.19:0/3001812625 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2565405427 0 0) 0x7ff2080afab0 con 0x7ff208153580 2024-03-28T13:16:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 -- 172.21.15.19:0/3001812625 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1ec003520 con 0x7ff208153580 2024-03-28T13:16:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 -- 172.21.15.19:0/3001812625 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3723589407 0 0) 0x7ff1ec003520 con 0x7ff208153580 2024-03-28T13:16:37.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 -- 172.21.15.19:0/3001812625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2080af2e0 msgr2=0x7ff2081600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 --2- 172.21.15.19:0/3001812625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2080af2e0 0x7ff2081600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 -- 172.21.15.19:0/3001812625 >> v2:172.21.15.188:3300/0 conn(0x7ff2080d0990 msgr2=0x7ff2080aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 --2- 172.21.15.19:0/3001812625 >> v2:172.21.15.188:3300/0 conn(0x7ff2080d0990 0x7ff2080aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 -- 172.21.15.19:0/3001812625 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20815cf20 con 0x7ff208153580 2024-03-28T13:16:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 -- 172.21.15.19:0/3001812625 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7ff1f80019f0 con 0x7ff208153580 2024-03-28T13:16:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 -- 172.21.15.19:0/3001812625 >> v1:172.21.15.19:6789/0 conn(0x7ff208153580 legacy=0x7ff2080d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 --2- 172.21.15.19:0/3001812625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1ec0039b0 0x7ff1ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 --2- 172.21.15.19:0/3001812625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec004800 0x7ff1ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 --2- 172.21.15.19:0/3001812625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2080af2e0 0x7ff1ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 -- 172.21.15.19:0/3001812625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff20815cc80 con 0x7ff1ec004800 2024-03-28T13:16:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 -- 172.21.15.19:0/3001812625 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff20815da00 con 0x7ff1ec0039b0 2024-03-28T13:16:37.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.340+0000 7ff204a00640 1 -- 172.21.15.19:0/3001812625 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ec00ba10 con 0x7ff2080af2e0 2024-03-28T13:16:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.341+0000 7ff205400640 1 --2- 172.21.15.19:0/3001812625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec004800 0x7ff1ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.341+0000 7ff205e00640 1 --2- 172.21.15.19:0/3001812625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1ec0039b0 0x7ff1ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.341+0000 7ff206800640 1 --2- 172.21.15.19:0/3001812625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2080af2e0 0x7ff1ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.341+0000 7ff205400640 1 -- 172.21.15.19:0/3001812625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2080af2e0 msgr2=0x7ff1ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.341+0000 7ff205400640 1 --2- 172.21.15.19:0/3001812625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2080af2e0 0x7ff1ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.341+0000 7ff205400640 1 -- 172.21.15.19:0/3001812625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1ec0039b0 msgr2=0x7ff1ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:37.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.341+0000 7ff205400640 1 --2- 172.21.15.19:0/3001812625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1ec0039b0 0x7ff1ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.342+0000 7ff205400640 1 -- 172.21.15.19:0/3001812625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ec003520 con 0x7ff1ec004800 2024-03-28T13:16:37.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.342+0000 7ff206800640 1 --2- 172.21.15.19:0/3001812625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2080af2e0 0x7ff1ec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:16:37.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.343+0000 7ff205400640 1 --2- 172.21.15.19:0/3001812625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec004800 0x7ff1ec004c80 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009a90 tx=0x7ff1ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccad2442439da383 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:37.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.344+0000 7ff204a00640 1 -- 172.21.15.19:0/3001812625 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0039070 con 0x7ff1ec004800 2024-03-28T13:16:37.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.344+0000 7ff204a00640 1 -- 172.21.15.19:0/3001812625 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f002c680 con 0x7ff1ec004800 2024-03-28T13:16:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.344+0000 7ff204a00640 1 -- 172.21.15.19:0/3001812625 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0009690 con 0x7ff1ec004800 2024-03-28T13:16:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.344+0000 7ff20cc00640 1 -- 172.21.15.19:0/3001812625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec004800 msgr2=0x7ff1ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.344+0000 7ff20cc00640 1 --2- 172.21.15.19:0/3001812625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec004800 0x7ff1ec004c80 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0009a90 tx=0x7ff1ec005830 comp rx=0 tx=0).stop 2024-03-28T13:16:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.344+0000 7ff206800640 1 -- 172.21.15.19:0/3001812625 reap_dead start 2024-03-28T13:16:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.345+0000 7ff20cc00640 1 -- 172.21.15.19:0/3001812625 shutdown_connections 2024-03-28T13:16:37.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.345+0000 7ff20cc00640 1 -- 172.21.15.19:0/3001812625 >> 172.21.15.19:0/3001812625 conn(0x7ff20806b750 msgr2=0x7ff20806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.345+0000 7ff20cc00640 1 -- 172.21.15.19:0/3001812625 shutdown_connections 2024-03-28T13:16:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.345+0000 7ff20cc00640 1 -- 172.21.15.19:0/3001812625 wait complete. 2024-03-28T13:16:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.346+0000 7ff20cc00640 1 Processor -- start 2024-03-28T13:16:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.347+0000 7ff20cc00640 1 -- start start 2024-03-28T13:16:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.347+0000 7ff20cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec004800 0x7ff20814fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.347+0000 7ff20cc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff208140ae0 0x7ff2080afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.348+0000 7ff20cc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2081e4d00 0x7ff2081e5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.348+0000 7ff20cc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff208153960 con 0x7ff1ec004800 2024-03-28T13:16:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.348+0000 7ff20cc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff20815d4e0 con 0x7ff208140ae0 2024-03-28T13:16:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.348+0000 7ff20cc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff2080af710 con 0x7ff2081e4d00 2024-03-28T13:16:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.348+0000 7ff205e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec004800 0x7ff20814fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.348+0000 7ff205e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec004800 0x7ff20814fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60830/0 (socket says 172.21.15.19:60830) 2024-03-28T13:16:37.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.348+0000 7ff205e00640 1 -- 172.21.15.19:0/2943308692 learned_addr learned my addr 172.21.15.19:0/2943308692 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.348+0000 7ff205e00640 1 -- 172.21.15.19:0/2943308692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2081e4d00 msgr2=0x7ff2081e5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.348+0000 7ff205400640 1 --2- 172.21.15.19:0/2943308692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff208140ae0 0x7ff2080afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.348+0000 7ff206800640 1 --2- 172.21.15.19:0/2943308692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2081e4d00 0x7ff2081e5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.348+0000 7ff205e00640 1 --2- 172.21.15.19:0/2943308692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2081e4d00 0x7ff2081e5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.348+0000 7ff205e00640 1 -- 172.21.15.19:0/2943308692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff208140ae0 msgr2=0x7ff2080afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:37.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.348+0000 7ff205e00640 1 --2- 172.21.15.19:0/2943308692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff208140ae0 0x7ff2080afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.348+0000 7ff205e00640 1 -- 172.21.15.19:0/2943308692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff208140f40 con 0x7ff1ec004800 2024-03-28T13:16:37.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.349+0000 7ff205e00640 1 --2- 172.21.15.19:0/2943308692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec004800 0x7ff20814fc00 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8007a20 tx=0x7ff1f800a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:37.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.349+0000 7ff1fe000640 1 -- 172.21.15.19:0/2943308692 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8017020 con 0x7ff1ec004800 2024-03-28T13:16:37.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.349+0000 7ff1fe000640 1 -- 172.21.15.19:0/2943308692 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f8004e30 con 0x7ff1ec004800 2024-03-28T13:16:37.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.349+0000 7ff1fe000640 1 -- 172.21.15.19:0/2943308692 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f8013610 con 0x7ff1ec004800 2024-03-28T13:16:37.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.349+0000 7ff20cc00640 1 -- 172.21.15.19:0/2943308692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff208150200 con 0x7ff1ec004800 2024-03-28T13:16:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.349+0000 7ff20cc00640 1 -- 172.21.15.19:0/2943308692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2081507c0 con 0x7ff1ec004800 2024-03-28T13:16:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.350+0000 7ff20cc00640 1 -- 172.21.15.19:0/2943308692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2080685c0 con 0x7ff1ec004800 2024-03-28T13:16:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.351+0000 7ff1fe000640 1 -- 172.21.15.19:0/2943308692 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff1f801a020 con 0x7ff1ec004800 2024-03-28T13:16:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.352+0000 7ff1fe000640 1 --2- 172.21.15.19:0/2943308692 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff1e406c530 0x7ff1e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.352+0000 7ff1fe000640 1 -- 172.21.15.19:0/2943308692 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7ff1f808c240 con 0x7ff1ec004800 2024-03-28T13:16:37.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.353+0000 7ff205400640 1 --2- 172.21.15.19:0/2943308692 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff1e406c530 0x7ff1e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.354+0000 7ff205400640 1 --2- 172.21.15.19:0/2943308692 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff1e406c530 0x7ff1e406e9f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0036000 tx=0x7ff1f0030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:37.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.357+0000 7ff1fe000640 1 -- 172.21.15.19:0/2943308692 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff1f8021020 con 0x7ff1ec004800 2024-03-28T13:16:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.560+0000 7ff20cc00640 1 -- 172.21.15.19:0/2943308692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff20806eca0 con 0x7ff1ec004800 2024-03-28T13:16:37.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.561+0000 7ff1fe000640 1 -- 172.21.15.19:0/2943308692 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7ff1f805a7c0 con 0x7ff1ec004800 2024-03-28T13:16:37.561 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:16:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.564+0000 7ff20cc00640 1 -- 172.21.15.19:0/2943308692 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff1e406c530 msgr2=0x7ff1e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.564+0000 7ff20cc00640 1 --2- 172.21.15.19:0/2943308692 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff1e406c530 0x7ff1e406e9f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff1f0036000 tx=0x7ff1f0030000 comp rx=0 tx=0).stop 2024-03-28T13:16:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.564+0000 7ff20cc00640 1 -- 172.21.15.19:0/2943308692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec004800 msgr2=0x7ff20814fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.564+0000 7ff20cc00640 1 --2- 172.21.15.19:0/2943308692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec004800 0x7ff20814fc00 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8007a20 tx=0x7ff1f800a6a0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.564+0000 7ff20cc00640 1 -- 172.21.15.19:0/2943308692 shutdown_connections 2024-03-28T13:16:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.564+0000 7ff20cc00640 1 --2- 172.21.15.19:0/2943308692 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff1e406c530 0x7ff1e406e9f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.564+0000 7ff20cc00640 1 --2- 172.21.15.19:0/2943308692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2081e4d00 0x7ff2081e5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.564+0000 7ff20cc00640 1 --2- 172.21.15.19:0/2943308692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff208140ae0 0x7ff2080afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.564+0000 7ff20cc00640 1 --2- 172.21.15.19:0/2943308692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff1ec004800 0x7ff20814fc00 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.564+0000 7ff20cc00640 1 -- 172.21.15.19:0/2943308692 >> 172.21.15.19:0/2943308692 conn(0x7ff20806b750 msgr2=0x7ff20806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.564+0000 7ff20cc00640 1 -- 172.21.15.19:0/2943308692 shutdown_connections 2024-03-28T13:16:37.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.564+0000 7ff20cc00640 1 -- 172.21.15.19:0/2943308692 wait complete. 2024-03-28T13:16:37.584 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:16:37.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.796+0000 7f656b200640 1 Processor -- start 2024-03-28T13:16:37.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.796+0000 7f656b200640 1 -- start start 2024-03-28T13:16:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.797+0000 7f656b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f656c15e4a0 0x7f656c0b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.797+0000 7f656b200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f656c0d2310 0x7f656c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.797+0000 7f656b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656c0cf530 0x7f656c0d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.797+0000 7f656b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f656c0d34e0 con 0x7f656c15e4a0 2024-03-28T13:16:37.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.797+0000 7f656b200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f656c15e880 con 0x7f656c0d2310 2024-03-28T13:16:37.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.797+0000 7f656b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f656c0d28d0 con 0x7f656c0cf530 2024-03-28T13:16:37.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.798+0000 7f6569e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f656c15e4a0 0x7f656c0b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.798+0000 7f6569e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f656c15e4a0 0x7f656c0b9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60844/0 (socket says 172.21.15.19:60844) 2024-03-28T13:16:37.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.798+0000 7f6569e00640 1 -- 172.21.15.19:0/3329015996 learned_addr learned my addr 172.21.15.19:0/3329015996 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:37.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.798+0000 7f6569400640 1 --2- 172.21.15.19:0/3329015996 >> v2:172.21.15.188:3300/0 conn(0x7f656c0d2310 0x7f656c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.798+0000 7f656a800640 1 --2- 172.21.15.19:0/3329015996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656c0cf530 0x7f656c0d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.798+0000 7f6569e00640 1 -- 172.21.15.19:0/3329015996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656c0cf530 msgr2=0x7f656c0d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:37.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.798+0000 7f6569e00640 1 --2- 172.21.15.19:0/3329015996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656c0cf530 0x7f656c0d3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.798+0000 7f6569e00640 1 -- 172.21.15.19:0/3329015996 >> v2:172.21.15.188:3300/0 conn(0x7f656c0d2310 msgr2=0x7f656c0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:37.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.798+0000 7f6569e00640 1 --2- 172.21.15.19:0/3329015996 >> v2:172.21.15.188:3300/0 conn(0x7f656c0d2310 0x7f656c0ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.798+0000 7f6569e00640 1 -- 172.21.15.19:0/3329015996 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656c0d2bb0 con 0x7f656c15e4a0 2024-03-28T13:16:37.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.800+0000 7f6569e00640 1 --2- 172.21.15.19:0/3329015996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656c15e4a0 0x7f656c0b9c60 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f656c0d1b90 tx=0x7f655c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed77fd970b77585e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:37.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.800+0000 7f6568a00640 1 -- 172.21.15.19:0/3329015996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c02e070 con 0x7f656c15e4a0 2024-03-28T13:16:37.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.800+0000 7f6568a00640 1 -- 172.21.15.19:0/3329015996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f655c037070 con 0x7f656c15e4a0 2024-03-28T13:16:37.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.800+0000 7f6568a00640 1 -- 172.21.15.19:0/3329015996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c031a00 con 0x7f656c15e4a0 2024-03-28T13:16:37.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.801+0000 7f656b200640 1 -- 172.21.15.19:0/3329015996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656c15e4a0 msgr2=0x7f656c0b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:37.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.801+0000 7f656b200640 1 --2- 172.21.15.19:0/3329015996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656c15e4a0 0x7f656c0b9c60 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f656c0d1b90 tx=0x7f655c02b880 comp rx=0 tx=0).stop 2024-03-28T13:16:37.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.801+0000 7f656b200640 1 -- 172.21.15.19:0/3329015996 shutdown_connections 2024-03-28T13:16:37.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.801+0000 7f656b200640 1 --2- 172.21.15.19:0/3329015996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656c0cf530 0x7f656c0d3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.801+0000 7f656b200640 1 --2- 172.21.15.19:0/3329015996 >> v2:172.21.15.188:3300/0 conn(0x7f656c0d2310 0x7f656c0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.801+0000 7f656b200640 1 --2- 172.21.15.19:0/3329015996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656c15e4a0 0x7f656c0b9c60 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.801+0000 7f656b200640 1 -- 172.21.15.19:0/3329015996 >> 172.21.15.19:0/3329015996 conn(0x7f656c0676d0 msgr2=0x7f656c067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:37.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.801+0000 7f656b200640 1 -- 172.21.15.19:0/3329015996 shutdown_connections 2024-03-28T13:16:37.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.801+0000 7f656b200640 1 -- 172.21.15.19:0/3329015996 wait complete. 2024-03-28T13:16:37.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.803+0000 7f656b200640 1 Processor -- start 2024-03-28T13:16:37.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.803+0000 7f656b200640 1 -- start start 2024-03-28T13:16:37.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.804+0000 7f656b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f656c0cf530 0x7f656c14fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.804+0000 7f656b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656c0d2310 0x7f656c0c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.804+0000 7f656b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656c15e4a0 0x7f656c159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.804+0000 7f656b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f656c0d3970 con 0x7f656c0d2310 2024-03-28T13:16:37.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.804+0000 7f656b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f656c0d3170 con 0x7f656c0cf530 2024-03-28T13:16:37.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.804+0000 7f656b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f656c0ba1a0 con 0x7f656c15e4a0 2024-03-28T13:16:37.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.804+0000 7f6569400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656c0d2310 0x7f656c0c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.804+0000 7f6569400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656c0d2310 0x7f656c0c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60848/0 (socket says 172.21.15.19:60848) 2024-03-28T13:16:37.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.804+0000 7f6569400640 1 -- 172.21.15.19:0/2111370448 learned_addr learned my addr 172.21.15.19:0/2111370448 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:37.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.804+0000 7f6569e00640 1 --2- 172.21.15.19:0/2111370448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f656c0cf530 0x7f656c14fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.804+0000 7f656a800640 1 --2- 172.21.15.19:0/2111370448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656c15e4a0 0x7f656c159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.805+0000 7f6569400640 1 -- 172.21.15.19:0/2111370448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656c15e4a0 msgr2=0x7f656c159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:37.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.805+0000 7f6569400640 1 --2- 172.21.15.19:0/2111370448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656c15e4a0 0x7f656c159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.805+0000 7f6569400640 1 -- 172.21.15.19:0/2111370448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f656c0cf530 msgr2=0x7f656c14fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:37.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.805+0000 7f6569400640 1 --2- 172.21.15.19:0/2111370448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f656c0cf530 0x7f656c14fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:37.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.805+0000 7f6569400640 1 -- 172.21.15.19:0/2111370448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f656c0c8b50 con 0x7f656c0d2310 2024-03-28T13:16:37.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.805+0000 7f656a800640 1 --2- 172.21.15.19:0/2111370448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656c15e4a0 0x7f656c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:37.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.805+0000 7f6569400640 1 --2- 172.21.15.19:0/2111370448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656c0d2310 0x7f656c0c9bf0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f655400d830 tx=0x7f655400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:37.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.805+0000 7f6562000640 1 -- 172.21.15.19:0/2111370448 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655400b6c0 con 0x7f656c0d2310 2024-03-28T13:16:37.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.805+0000 7f6562000640 1 -- 172.21.15.19:0/2111370448 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6554019070 con 0x7f656c0d2310 2024-03-28T13:16:37.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.805+0000 7f6562000640 1 -- 172.21.15.19:0/2111370448 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655400ba80 con 0x7f656c0d2310 2024-03-28T13:16:37.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.805+0000 7f656b200640 1 -- 172.21.15.19:0/2111370448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f656c0c8e30 con 0x7f656c0d2310 2024-03-28T13:16:37.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.805+0000 7f656b200640 1 -- 172.21.15.19:0/2111370448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f656c0c93f0 con 0x7f656c0d2310 2024-03-28T13:16:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.807+0000 7f656b200640 1 -- 172.21.15.19:0/2111370448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f656c075940 con 0x7f656c0d2310 2024-03-28T13:16:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.807+0000 7f6562000640 1 -- 172.21.15.19:0/2111370448 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f655401e460 con 0x7f656c0d2310 2024-03-28T13:16:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.808+0000 7f6562000640 1 --2- 172.21.15.19:0/2111370448 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f654806c560 0x7f654806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:37.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.808+0000 7f6562000640 1 -- 172.21.15.19:0/2111370448 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f655408be00 con 0x7f656c0d2310 2024-03-28T13:16:37.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.809+0000 7f6569e00640 1 --2- 172.21.15.19:0/2111370448 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f654806c560 0x7f654806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:37.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.810+0000 7f6569e00640 1 --2- 172.21.15.19:0/2111370448 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f654806c560 0x7f654806ea20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f655c036000 tx=0x7f655c03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:37.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:37.813+0000 7f6562000640 1 -- 172.21.15.19:0/2111370448 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f655405a380 con 0x7f656c0d2310 2024-03-28T13:16:38.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.031+0000 7f656b200640 1 -- 172.21.15.19:0/2111370448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f656c0b7910 con 0x7f656c0d2310 2024-03-28T13:16:38.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.032+0000 7f6562000640 1 -- 172.21.15.19:0/2111370448 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v92) ==== 80+0+0 (secure 0 0 0) 0x7f6554059c80 con 0x7f656c0d2310 2024-03-28T13:16:38.031 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-03-28T13:16:38.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.034+0000 7f656b200640 1 -- 172.21.15.19:0/2111370448 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f654806c560 msgr2=0x7f654806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:38.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.034+0000 7f656b200640 1 --2- 172.21.15.19:0/2111370448 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f654806c560 0x7f654806ea20 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f655c036000 tx=0x7f655c03d040 comp rx=0 tx=0).stop 2024-03-28T13:16:38.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.034+0000 7f656b200640 1 -- 172.21.15.19:0/2111370448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656c0d2310 msgr2=0x7f656c0c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:38.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.034+0000 7f656b200640 1 --2- 172.21.15.19:0/2111370448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656c0d2310 0x7f656c0c9bf0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f655400d830 tx=0x7f655400dcf0 comp rx=0 tx=0).stop 2024-03-28T13:16:38.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.034+0000 7f656b200640 1 -- 172.21.15.19:0/2111370448 shutdown_connections 2024-03-28T13:16:38.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.034+0000 7f656b200640 1 --2- 172.21.15.19:0/2111370448 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f654806c560 0x7f654806ea20 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:38.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.034+0000 7f656b200640 1 --2- 172.21.15.19:0/2111370448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656c15e4a0 0x7f656c159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:38.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.034+0000 7f656b200640 1 --2- 172.21.15.19:0/2111370448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f656c0d2310 0x7f656c0c9bf0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:38.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.034+0000 7f656b200640 1 --2- 172.21.15.19:0/2111370448 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f656c0cf530 0x7f656c14fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:38.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.034+0000 7f656b200640 1 -- 172.21.15.19:0/2111370448 >> 172.21.15.19:0/2111370448 conn(0x7f656c0676d0 msgr2=0x7f656c15c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:38.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.034+0000 7f656b200640 1 -- 172.21.15.19:0/2111370448 shutdown_connections 2024-03-28T13:16:38.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.035+0000 7f656b200640 1 -- 172.21.15.19:0/2111370448 wait complete. 2024-03-28T13:16:38.052 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:16:38.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:38.053 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T13:16:38.216 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:16:38.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:38.217 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:16:38.267 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:16:38.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.488+0000 7f2eeb200640 1 Processor -- start 2024-03-28T13:16:38.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.488+0000 7f2eeb200640 1 -- start start 2024-03-28T13:16:38.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.490+0000 7f2eeb200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eec0cc0b0 0x7f2eec06dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:38.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.490+0000 7f2eeb200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2eec0d04d0 0x7f2eec15ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:38.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.490+0000 7f2eeb200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eec06e4f0 con 0x7f2eec15fc50 2024-03-28T13:16:38.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.490+0000 7f2eeb200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f2eec15c610 con 0x7f2eec0d04d0 2024-03-28T13:16:38.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.490+0000 7f2eeb200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2eec0b3150 con 0x7f2eec0cc0b0 2024-03-28T13:16:38.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.491+0000 7f2eea800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2eec0d04d0 0x7f2eec15ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:38.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.491+0000 7f2ee9e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eec0cc0b0 0x7f2eec06dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:38.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.491+0000 7f2eea800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2eec0d04d0 0x7f2eec15ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:52906/0 (socket says 172.21.15.19:52906) 2024-03-28T13:16:38.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.491+0000 7f2eea800640 1 -- 172.21.15.19:0/1680533831 learned_addr learned my addr 172.21.15.19:0/1680533831 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:38.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.491+0000 7f2eea800640 1 -- 172.21.15.19:0/1680533831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eec0cc0b0 msgr2=0x7f2eec06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:38.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.491+0000 7f2eea800640 1 --2- 172.21.15.19:0/1680533831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eec0cc0b0 0x7f2eec06dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:38.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.491+0000 7f2eea800640 1 -- 172.21.15.19:0/1680533831 >> v1:172.21.15.188:6789/0 conn(0x7f2eec15fc50 legacy=0x7f2eec0cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:38.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.491+0000 7f2eea800640 1 -- 172.21.15.19:0/1680533831 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec1600e0 con 0x7f2eec0d04d0 2024-03-28T13:16:38.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.493+0000 7f2eea800640 1 --2- 172.21.15.19:0/1680533831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eec0d04d0 0x7f2eec15ca70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009e50 tx=0x7f2ee402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a588e70967cf1a96 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:38.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.493+0000 7f2ee8a00640 1 -- 172.21.15.19:0/1680533831 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee402cb00 con 0x7f2eec0d04d0 2024-03-28T13:16:38.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.493+0000 7f2ee8a00640 1 -- 172.21.15.19:0/1680533831 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee402cca0 con 0x7f2eec0d04d0 2024-03-28T13:16:38.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.493+0000 7f2ee8a00640 1 -- 172.21.15.19:0/1680533831 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4034610 con 0x7f2eec0d04d0 2024-03-28T13:16:38.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.494+0000 7f2eeb200640 1 -- 172.21.15.19:0/1680533831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eec0d04d0 msgr2=0x7f2eec15ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:38.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.494+0000 7f2eeb200640 1 --2- 172.21.15.19:0/1680533831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eec0d04d0 0x7f2eec15ca70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009e50 tx=0x7f2ee402bf60 comp rx=0 tx=0).stop 2024-03-28T13:16:38.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.494+0000 7f2eeb200640 1 -- 172.21.15.19:0/1680533831 shutdown_connections 2024-03-28T13:16:38.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.494+0000 7f2eeb200640 1 --2- 172.21.15.19:0/1680533831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eec0d04d0 0x7f2eec15ca70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:38.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.494+0000 7f2eeb200640 1 --2- 172.21.15.19:0/1680533831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eec0cc0b0 0x7f2eec06dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:38.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.494+0000 7f2eeb200640 1 -- 172.21.15.19:0/1680533831 >> 172.21.15.19:0/1680533831 conn(0x7f2eec0ccad0 msgr2=0x7f2eec0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:38.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.494+0000 7f2eeb200640 1 -- 172.21.15.19:0/1680533831 shutdown_connections 2024-03-28T13:16:38.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.494+0000 7f2eeb200640 1 -- 172.21.15.19:0/1680533831 wait complete. 2024-03-28T13:16:38.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.496+0000 7f2eeb200640 1 Processor -- start 2024-03-28T13:16:38.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.496+0000 7f2eeb200640 1 -- start start 2024-03-28T13:16:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.496+0000 7f2eeb200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eec0cc0b0 0x7f2eec1eca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.497+0000 7f2eeb200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eec0d04d0 0x7f2eec156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.497+0000 7f2eeb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eec0b1b70 0x7f2eec13e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.497+0000 7f2eeb200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec160980 con 0x7f2eec0b1b70 2024-03-28T13:16:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.497+0000 7f2eeb200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1606a0 con 0x7f2eec0cc0b0 2024-03-28T13:16:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.497+0000 7f2eeb200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2eec0cc490 con 0x7f2eec0d04d0 2024-03-28T13:16:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.497+0000 7f2eea800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eec0b1b70 0x7f2eec13e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.497+0000 7f2eea800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eec0b1b70 0x7f2eec13e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60856/0 (socket says 172.21.15.19:60856) 2024-03-28T13:16:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.497+0000 7f2eea800640 1 -- 172.21.15.19:0/3265785707 learned_addr learned my addr 172.21.15.19:0/3265785707 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:38.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.497+0000 7f2ee9e00640 1 --2- 172.21.15.19:0/3265785707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eec0cc0b0 0x7f2eec1eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.497+0000 7f2ee9400640 1 --2- 172.21.15.19:0/3265785707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eec0d04d0 0x7f2eec156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.497+0000 7f2eea800640 1 -- 172.21.15.19:0/3265785707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eec0d04d0 msgr2=0x7f2eec156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.497+0000 7f2eea800640 1 --2- 172.21.15.19:0/3265785707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eec0d04d0 0x7f2eec156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.497+0000 7f2eea800640 1 -- 172.21.15.19:0/3265785707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eec0cc0b0 msgr2=0x7f2eec1eca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.497+0000 7f2eea800640 1 --2- 172.21.15.19:0/3265785707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eec0cc0b0 0x7f2eec1eca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.498+0000 7f2eea800640 1 -- 172.21.15.19:0/3265785707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec158820 con 0x7f2eec0b1b70 2024-03-28T13:16:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.498+0000 7f2eea800640 1 --2- 172.21.15.19:0/3265785707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eec0b1b70 0x7f2eec13e410 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009e50 tx=0x7f2ee4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.498+0000 7f2ee2000640 1 -- 172.21.15.19:0/3265785707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4043020 con 0x7f2eec0b1b70 2024-03-28T13:16:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.498+0000 7f2ee2000640 1 -- 172.21.15.19:0/3265785707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee402ce50 con 0x7f2eec0b1b70 2024-03-28T13:16:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.498+0000 7f2ee2000640 1 -- 172.21.15.19:0/3265785707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4009510 con 0x7f2eec0b1b70 2024-03-28T13:16:38.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.498+0000 7f2eeb200640 1 -- 172.21.15.19:0/3265785707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eec157780 con 0x7f2eec0b1b70 2024-03-28T13:16:38.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.498+0000 7f2eeb200640 1 -- 172.21.15.19:0/3265785707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eec157d40 con 0x7f2eec0b1b70 2024-03-28T13:16:38.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.499+0000 7f2eeb200640 1 -- 172.21.15.19:0/3265785707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eec15af30 con 0x7f2eec0b1b70 2024-03-28T13:16:38.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.500+0000 7f2ee2000640 1 -- 172.21.15.19:0/3265785707 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2ee4046020 con 0x7f2eec0b1b70 2024-03-28T13:16:38.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.501+0000 7f2ee2000640 1 --2- 172.21.15.19:0/3265785707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ecc06c530 0x7f2ecc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:38.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.501+0000 7f2ee2000640 1 -- 172.21.15.19:0/3265785707 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f2ee40adc60 con 0x7f2eec0b1b70 2024-03-28T13:16:38.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.502+0000 7f2ee9e00640 1 --2- 172.21.15.19:0/3265785707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ecc06c530 0x7f2ecc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:38.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.503+0000 7f2ee9e00640 1 --2- 172.21.15.19:0/3265785707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ecc06c530 0x7f2ecc06e9f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0b26d0 tx=0x7f2edc009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:38.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.506+0000 7f2ee2000640 1 -- 172.21.15.19:0/3265785707 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2ee404d030 con 0x7f2eec0b1b70 2024-03-28T13:16:38.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.744+0000 7f2eeb200640 1 -- 172.21.15.19:0/3265785707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2eec075940 con 0x7f2eec0b1b70 2024-03-28T13:16:38.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.745+0000 7f2ee2000640 1 -- 172.21.15.19:0/3265785707 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5186 (secure 0 0 0) 0x7f2ee407c270 con 0x7f2eec0b1b70 2024-03-28T13:16:38.745 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:38.745 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:16:29.083492+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6043":{"gid":6043,"name":"b","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2260087679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2260087679},{"type":"v1","addr":"172.21.15.188:6836","nonce":2260087679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6043,"qdb_cluster":[6043]},"id":6}]} 2024-03-28T13:16:38.746 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 61 2024-03-28T13:16:38.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.748+0000 7f2eeb200640 1 -- 172.21.15.19:0/3265785707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ecc06c530 msgr2=0x7f2ecc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:38.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.748+0000 7f2eeb200640 1 --2- 172.21.15.19:0/3265785707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ecc06c530 0x7f2ecc06e9f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0b26d0 tx=0x7f2edc009190 comp rx=0 tx=0).stop 2024-03-28T13:16:38.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.748+0000 7f2eeb200640 1 -- 172.21.15.19:0/3265785707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eec0b1b70 msgr2=0x7f2eec13e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:38.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.748+0000 7f2eeb200640 1 --2- 172.21.15.19:0/3265785707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eec0b1b70 0x7f2eec13e410 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009e50 tx=0x7f2ee4004a40 comp rx=0 tx=0).stop 2024-03-28T13:16:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.748+0000 7f2eeb200640 1 -- 172.21.15.19:0/3265785707 shutdown_connections 2024-03-28T13:16:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.748+0000 7f2eeb200640 1 --2- 172.21.15.19:0/3265785707 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2ecc06c530 0x7f2ecc06e9f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.748+0000 7f2eeb200640 1 --2- 172.21.15.19:0/3265785707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2eec0b1b70 0x7f2eec13e410 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:38.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.748+0000 7f2eeb200640 1 --2- 172.21.15.19:0/3265785707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eec0d04d0 0x7f2eec156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.748+0000 7f2eeb200640 1 --2- 172.21.15.19:0/3265785707 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eec0cc0b0 0x7f2eec1eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.748+0000 7f2eeb200640 1 -- 172.21.15.19:0/3265785707 >> 172.21.15.19:0/3265785707 conn(0x7f2eec0ccad0 msgr2=0x7f2eec15ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.748+0000 7f2eeb200640 1 -- 172.21.15.19:0/3265785707 shutdown_connections 2024-03-28T13:16:38.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:38.748+0000 7f2eeb200640 1 -- 172.21.15.19:0/3265785707 wait complete. 2024-03-28T13:16:38.764 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-03-28T13:16:39.009 INFO:teuthology.orchestra.run.smithi188.stdout:{"message":"","return_code":0} 2024-03-28T13:16:39.010 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-28T13:16:39.010 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T13:16:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.233+0000 7fa00ca00640 1 Processor -- start 2024-03-28T13:16:39.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.233+0000 7fa00ca00640 1 -- start start 2024-03-28T13:16:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.233+0000 7fa00ca00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008153580 0x7fa0080d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.233+0000 7fa00ca00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0080afab0 con 0x7fa0080d07c0 2024-03-28T13:16:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.233+0000 7fa00ca00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa00815cc80 con 0x7fa0080af4b0 2024-03-28T13:16:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.233+0000 7fa00ca00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa0080d0c00 con 0x7fa008153580 2024-03-28T13:16:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.233+0000 7fa005400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa0080d07c0 0x7fa0080aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53642/0 (socket says 172.21.15.19:53642) 2024-03-28T13:16:39.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.234+0000 7fa005400640 1 -- 172.21.15.19:0/919341137 learned_addr learned my addr 172.21.15.19:0/919341137 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:39.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.234+0000 7fa005e00640 1 --2- 172.21.15.19:0/919341137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008153580 0x7fa0080d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.234+0000 7fa004a00640 1 -- 172.21.15.19:0/919341137 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2903667349 0 0) 0x7fa0080afab0 con 0x7fa0080d07c0 2024-03-28T13:16:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.234+0000 7fa004a00640 1 -- 172.21.15.19:0/919341137 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fec003520 con 0x7fa0080d07c0 2024-03-28T13:16:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.234+0000 7fa004a00640 1 -- 172.21.15.19:0/919341137 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2824546296 0 0) 0x7f9fec003520 con 0x7fa0080d07c0 2024-03-28T13:16:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.234+0000 7fa004a00640 1 -- 172.21.15.19:0/919341137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008153580 msgr2=0x7fa0080d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.234+0000 7fa004a00640 1 --2- 172.21.15.19:0/919341137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008153580 0x7fa0080d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.234+0000 7fa004a00640 1 -- 172.21.15.19:0/919341137 >> v1:172.21.15.188:6789/0 conn(0x7fa0080af4b0 legacy=0x7fa0081600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.234+0000 7fa004a00640 1 -- 172.21.15.19:0/919341137 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00815da00 con 0x7fa0080d07c0 2024-03-28T13:16:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.234+0000 7fa004a00640 1 -- 172.21.15.19:0/919341137 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f9ff0001670 con 0x7fa0080d07c0 2024-03-28T13:16:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.234+0000 7fa004a00640 1 -- 172.21.15.19:0/919341137 >> v1:172.21.15.19:6789/0 conn(0x7fa0080d07c0 legacy=0x7fa0080aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.234+0000 7fa004a00640 1 --2- 172.21.15.19:0/919341137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fec0039e0 0x7f9fec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.235+0000 7fa004a00640 1 --2- 172.21.15.19:0/919341137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008153580 0x7f9fec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.235+0000 7fa004a00640 1 --2- 172.21.15.19:0/919341137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fec004d40 0x7f9fec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.235+0000 7fa004a00640 1 -- 172.21.15.19:0/919341137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0080d0c00 con 0x7f9fec004d40 2024-03-28T13:16:39.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.235+0000 7fa004a00640 1 -- 172.21.15.19:0/919341137 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9fec00baf0 con 0x7f9fec0039e0 2024-03-28T13:16:39.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.235+0000 7fa004a00640 1 -- 172.21.15.19:0/919341137 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9fec00bcf0 con 0x7fa008153580 2024-03-28T13:16:39.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.235+0000 7fa006800640 1 --2- 172.21.15.19:0/919341137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fec004d40 0x7f9fec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.235+0000 7fa005e00640 1 --2- 172.21.15.19:0/919341137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fec0039e0 0x7f9fec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.235+0000 7fa005400640 1 --2- 172.21.15.19:0/919341137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008153580 0x7f9fec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.235+0000 7fa006800640 1 -- 172.21.15.19:0/919341137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008153580 msgr2=0x7f9fec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:39.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.235+0000 7fa006800640 1 --2- 172.21.15.19:0/919341137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008153580 0x7f9fec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.235+0000 7fa006800640 1 -- 172.21.15.19:0/919341137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fec0039e0 msgr2=0x7f9fec003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:39.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.235+0000 7fa006800640 1 --2- 172.21.15.19:0/919341137 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fec0039e0 0x7f9fec003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.235+0000 7fa006800640 1 -- 172.21.15.19:0/919341137 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fec003520 con 0x7f9fec004d40 2024-03-28T13:16:39.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.235+0000 7fa005400640 1 --2- 172.21.15.19:0/919341137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa008153580 0x7f9fec004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.236+0000 7fa006800640 1 --2- 172.21.15.19:0/919341137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fec004d40 0x7f9fec0051c0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8001750 tx=0x7f9fec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dc040505ab89d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.237+0000 7fa004a00640 1 -- 172.21.15.19:0/919341137 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8001370 con 0x7f9fec004d40 2024-03-28T13:16:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.237+0000 7fa004a00640 1 -- 172.21.15.19:0/919341137 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff80052d0 con 0x7f9fec004d40 2024-03-28T13:16:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.237+0000 7fa004a00640 1 -- 172.21.15.19:0/919341137 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8003830 con 0x7f9fec004d40 2024-03-28T13:16:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.237+0000 7fa00ca00640 1 -- 172.21.15.19:0/919341137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fec004d40 msgr2=0x7f9fec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.237+0000 7fa00ca00640 1 --2- 172.21.15.19:0/919341137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fec004d40 0x7f9fec0051c0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8001750 tx=0x7f9fec005830 comp rx=0 tx=0).stop 2024-03-28T13:16:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.237+0000 7fa006800640 1 -- 172.21.15.19:0/919341137 reap_dead start 2024-03-28T13:16:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.237+0000 7fa00ca00640 1 -- 172.21.15.19:0/919341137 shutdown_connections 2024-03-28T13:16:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.237+0000 7fa00ca00640 1 -- 172.21.15.19:0/919341137 >> 172.21.15.19:0/919341137 conn(0x7fa00806b750 msgr2=0x7fa00806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.237+0000 7fa00ca00640 1 -- 172.21.15.19:0/919341137 shutdown_connections 2024-03-28T13:16:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.237+0000 7fa00ca00640 1 -- 172.21.15.19:0/919341137 wait complete. 2024-03-28T13:16:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.239+0000 7fa00ca00640 1 Processor -- start 2024-03-28T13:16:39.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.239+0000 7fa00ca00640 1 -- start start 2024-03-28T13:16:39.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.240+0000 7fa00ca00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fec004d40 0x7fa00815b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.240+0000 7fa00ca00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa00814d050 0x7fa008159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.240+0000 7fa00ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa008153bc0 0x7fa008154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.240+0000 7fa00ca00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa00815d4e0 con 0x7fa008153bc0 2024-03-28T13:16:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.240+0000 7fa00ca00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa00815d200 con 0x7fa00814d050 2024-03-28T13:16:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.240+0000 7fa00ca00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa008153960 con 0x7f9fec004d40 2024-03-28T13:16:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.240+0000 7fa006800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa008153bc0 0x7fa008154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.240+0000 7fa006800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa008153bc0 0x7fa008154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60874/0 (socket says 172.21.15.19:60874) 2024-03-28T13:16:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.240+0000 7fa006800640 1 -- 172.21.15.19:0/969714875 learned_addr learned my addr 172.21.15.19:0/969714875 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.241+0000 7fa005400640 1 --2- 172.21.15.19:0/969714875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa00814d050 0x7fa008159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.241+0000 7fa005e00640 1 --2- 172.21.15.19:0/969714875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fec004d40 0x7fa00815b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.241+0000 7fa006800640 1 -- 172.21.15.19:0/969714875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fec004d40 msgr2=0x7fa00815b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.241+0000 7fa006800640 1 --2- 172.21.15.19:0/969714875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fec004d40 0x7fa00815b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.241+0000 7fa006800640 1 -- 172.21.15.19:0/969714875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa00814d050 msgr2=0x7fa008159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.241+0000 7fa006800640 1 --2- 172.21.15.19:0/969714875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa00814d050 0x7fa008159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.241+0000 7fa006800640 1 -- 172.21.15.19:0/969714875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa00814d860 con 0x7fa008153bc0 2024-03-28T13:16:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.241+0000 7fa005e00640 1 --2- 172.21.15.19:0/969714875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fec004d40 0x7fa00815b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.241+0000 7fa006800640 1 --2- 172.21.15.19:0/969714875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa008153bc0 0x7fa008154040 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8001870 tx=0x7f9ff8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.242+0000 7f9ffe000640 1 -- 172.21.15.19:0/969714875 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8041070 con 0x7fa008153bc0 2024-03-28T13:16:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.242+0000 7f9ffe000640 1 -- 172.21.15.19:0/969714875 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff8007680 con 0x7fa008153bc0 2024-03-28T13:16:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.242+0000 7fa00ca00640 1 -- 172.21.15.19:0/969714875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa00815c040 con 0x7fa008153bc0 2024-03-28T13:16:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.242+0000 7f9ffe000640 1 -- 172.21.15.19:0/969714875 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8001370 con 0x7fa008153bc0 2024-03-28T13:16:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.242+0000 7fa00ca00640 1 -- 172.21.15.19:0/969714875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa00815ba80 con 0x7fa008153bc0 2024-03-28T13:16:39.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.243+0000 7fa00ca00640 1 -- 172.21.15.19:0/969714875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0080685c0 con 0x7fa008153bc0 2024-03-28T13:16:39.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.244+0000 7f9ffe000640 1 -- 172.21.15.19:0/969714875 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9ff8044070 con 0x7fa008153bc0 2024-03-28T13:16:39.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.245+0000 7f9ffe000640 1 --2- 172.21.15.19:0/969714875 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9fe406c560 0x7f9fe406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.245+0000 7f9ffe000640 1 -- 172.21.15.19:0/969714875 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5709+0+0 (secure 0 0 0) 0x7f9ff80acde0 con 0x7fa008153bc0 2024-03-28T13:16:39.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.245+0000 7fa005e00640 1 --2- 172.21.15.19:0/969714875 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9fe406c560 0x7f9fe406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.247+0000 7fa005e00640 1 --2- 172.21.15.19:0/969714875 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9fe406c560 0x7f9fe406ea20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa000007440 tx=0x7fa00000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.250+0000 7f9ffe000640 1 -- 172.21.15.19:0/969714875 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9ff807b510 con 0x7fa008153bc0 2024-03-28T13:16:39.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.491+0000 7fa00ca00640 1 -- 172.21.15.19:0/969714875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa00806eca0 con 0x7fa008153bc0 2024-03-28T13:16:39.491 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:16:39.492+0000 7f6b74400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T13:16:39.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.495+0000 7f9ffe000640 1 -- 172.21.15.19:0/969714875 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) ==== 161+0+0 (secure 0 0 0) 0x7f9ff807ae10 con 0x7fa008153bc0 2024-03-28T13:16:39.495 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T13:16:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.497+0000 7fa00ca00640 1 -- 172.21.15.19:0/969714875 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9fe406c560 msgr2=0x7f9fe406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.497+0000 7fa00ca00640 1 --2- 172.21.15.19:0/969714875 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9fe406c560 0x7f9fe406ea20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa000007440 tx=0x7fa00000b040 comp rx=0 tx=0).stop 2024-03-28T13:16:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.497+0000 7fa00ca00640 1 -- 172.21.15.19:0/969714875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa008153bc0 msgr2=0x7fa008154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.497+0000 7fa00ca00640 1 --2- 172.21.15.19:0/969714875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa008153bc0 0x7fa008154040 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8001870 tx=0x7f9ff8001120 comp rx=0 tx=0).stop 2024-03-28T13:16:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.498+0000 7fa00ca00640 1 -- 172.21.15.19:0/969714875 shutdown_connections 2024-03-28T13:16:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.498+0000 7fa00ca00640 1 --2- 172.21.15.19:0/969714875 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9fe406c560 0x7f9fe406ea20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.498+0000 7fa00ca00640 1 --2- 172.21.15.19:0/969714875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa008153bc0 0x7fa008154040 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.498+0000 7fa00ca00640 1 --2- 172.21.15.19:0/969714875 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa00814d050 0x7fa008159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.498+0000 7fa00ca00640 1 --2- 172.21.15.19:0/969714875 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fec004d40 0x7fa00815b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.498+0000 7fa00ca00640 1 -- 172.21.15.19:0/969714875 >> 172.21.15.19:0/969714875 conn(0x7fa00806b750 msgr2=0x7fa008072b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.498+0000 7fa00ca00640 1 -- 172.21.15.19:0/969714875 shutdown_connections 2024-03-28T13:16:39.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.498+0000 7fa00ca00640 1 -- 172.21.15.19:0/969714875 wait complete. 2024-03-28T13:16:39.517 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-03-28T13:16:39.530 INFO:tasks.ceph.mds.b.smithi188.stderr: -5034> 2024-03-28T13:16:31.498+0000 7fb54e600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:16:39.610 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-03-28T13:16:39.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.622+0000 7fc99943ef40 1 Processor -- start 2024-03-28T13:16:39.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.622+0000 7fc99943ef40 1 -- start start 2024-03-28T13:16:39.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.623+0000 7fc99943ef40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faa706bcd0 0x55faa71485c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.623+0000 7fc99943ef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa7146990 0x55faa70b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.623+0000 7fc99943ef40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faa7146d70 0x55faa714a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.623+0000 7fc99943ef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55faa714cc60 con 0x55faa7146990 2024-03-28T13:16:39.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.623+0000 7fc99943ef40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55faa706c0e0 con 0x55faa706bcd0 2024-03-28T13:16:39.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.623+0000 7fc99943ef40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55faa714d4a0 con 0x55faa7146d70 2024-03-28T13:16:39.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.623+0000 7fc98d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa7146990 0x55faa70b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.623+0000 7fc98d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa7146990 0x55faa70b61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60890/0 (socket says 172.21.15.19:60890) 2024-03-28T13:16:39.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.623+0000 7fc98d400640 1 -- 172.21.15.19:0/33700355 learned_addr learned my addr 172.21.15.19:0/33700355 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:39.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.624+0000 7fc995e00640 1 --2- 172.21.15.19:0/33700355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faa706bcd0 0x55faa71485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.624+0000 7fc996800640 1 --2- 172.21.15.19:0/33700355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faa7146d70 0x55faa714a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.624+0000 7fc98d400640 1 -- 172.21.15.19:0/33700355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faa7146d70 msgr2=0x55faa714a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:39.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.624+0000 7fc98d400640 1 --2- 172.21.15.19:0/33700355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faa7146d70 0x55faa714a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.624+0000 7fc98d400640 1 -- 172.21.15.19:0/33700355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faa706bcd0 msgr2=0x55faa71485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:39.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.624+0000 7fc98d400640 1 --2- 172.21.15.19:0/33700355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faa706bcd0 0x55faa71485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.624+0000 7fc98d400640 1 -- 172.21.15.19:0/33700355 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa6fb04b0 con 0x55faa7146990 2024-03-28T13:16:39.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.624+0000 7fc98d400640 1 --2- 172.21.15.19:0/33700355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa7146990 0x55faa70b61b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc98000a7a0 tx=0x7fc98000a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95bbeb1b71a36d95 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.625+0000 7fc994a00640 1 -- 172.21.15.19:0/33700355 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc980019070 con 0x55faa7146990 2024-03-28T13:16:39.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.625+0000 7fc994a00640 1 -- 172.21.15.19:0/33700355 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc980007600 con 0x55faa7146990 2024-03-28T13:16:39.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.625+0000 7fc994a00640 1 -- 172.21.15.19:0/33700355 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9800079d0 con 0x55faa7146990 2024-03-28T13:16:39.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.625+0000 7fc99943ef40 1 -- 172.21.15.19:0/33700355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa7146990 msgr2=0x55faa70b61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.625+0000 7fc99943ef40 1 --2- 172.21.15.19:0/33700355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa7146990 0x55faa70b61b0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc98000a7a0 tx=0x7fc98000a890 comp rx=0 tx=0).stop 2024-03-28T13:16:39.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.625+0000 7fc99943ef40 1 -- 172.21.15.19:0/33700355 shutdown_connections 2024-03-28T13:16:39.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.625+0000 7fc99943ef40 1 --2- 172.21.15.19:0/33700355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faa7146d70 0x55faa714a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.625+0000 7fc99943ef40 1 --2- 172.21.15.19:0/33700355 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa7146990 0x55faa70b61b0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.625+0000 7fc99943ef40 1 --2- 172.21.15.19:0/33700355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faa706bcd0 0x55faa71485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.625+0000 7fc99943ef40 1 -- 172.21.15.19:0/33700355 >> 172.21.15.19:0/33700355 conn(0x55faa706c8a0 msgr2=0x55faa70b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:39.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.625+0000 7fc99943ef40 1 -- 172.21.15.19:0/33700355 shutdown_connections 2024-03-28T13:16:39.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.625+0000 7fc99943ef40 1 -- 172.21.15.19:0/33700355 wait complete. 2024-03-28T13:16:39.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.626+0000 7fc99943ef40 1 Processor -- start 2024-03-28T13:16:39.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.626+0000 7fc99943ef40 1 -- start start 2024-03-28T13:16:39.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.627+0000 7fc99943ef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa706bcd0 0x55faa7121ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.627+0000 7fc99943ef40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faa7146990 0x55faa709ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.627+0000 7fc99943ef40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faa7146d70 0x55faa7140f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.627+0000 7fc99943ef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55faa714acb0 con 0x55faa706bcd0 2024-03-28T13:16:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.627+0000 7fc99943ef40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55faa70b66f0 con 0x55faa7146d70 2024-03-28T13:16:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.628+0000 7fc99943ef40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55faa70b6970 con 0x55faa7146990 2024-03-28T13:16:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.628+0000 7fc995e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa706bcd0 0x55faa7121ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.628+0000 7fc995e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa706bcd0 0x55faa7121ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60906/0 (socket says 172.21.15.19:60906) 2024-03-28T13:16:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.628+0000 7fc995e00640 1 -- 172.21.15.19:0/1760583092 learned_addr learned my addr 172.21.15.19:0/1760583092 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.628+0000 7fc996800640 1 --2- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faa7146d70 0x55faa7140f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.628+0000 7fc98d400640 1 --2- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faa7146990 0x55faa709ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.628+0000 7fc995e00640 1 -- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faa7146990 msgr2=0x55faa709ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.628+0000 7fc995e00640 1 --2- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faa7146990 0x55faa709ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.628+0000 7fc995e00640 1 -- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faa7146d70 msgr2=0x55faa7140f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.628+0000 7fc995e00640 1 --2- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faa7146d70 0x55faa7140f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.628+0000 7fc995e00640 1 -- 172.21.15.19:0/1760583092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa7127a80 con 0x55faa706bcd0 2024-03-28T13:16:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.629+0000 7fc995e00640 1 --2- 172.21.15.19:0/1760583092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa706bcd0 0x55faa7121ef0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc98801ccd0 tx=0x7fc988003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.629+0000 7fc98f400640 1 -- 172.21.15.19:0/1760583092 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988062020 con 0x55faa706bcd0 2024-03-28T13:16:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.629+0000 7fc98f400640 1 -- 172.21.15.19:0/1760583092 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc98805e660 con 0x55faa706bcd0 2024-03-28T13:16:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.629+0000 7fc99943ef40 1 -- 172.21.15.19:0/1760583092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55faa70b78f0 con 0x55faa706bcd0 2024-03-28T13:16:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.629+0000 7fc98f400640 1 -- 172.21.15.19:0/1760583092 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988057040 con 0x55faa706bcd0 2024-03-28T13:16:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.629+0000 7fc99943ef40 1 -- 172.21.15.19:0/1760583092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa7127280 con 0x55faa706bcd0 2024-03-28T13:16:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.631+0000 7fc98f400640 1 -- 172.21.15.19:0/1760583092 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc988065020 con 0x55faa706bcd0 2024-03-28T13:16:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.633+0000 7fc98f400640 1 --2- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc97406af10 0x7fc97406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.633+0000 7fc98f400640 1 -- 172.21.15.19:0/1760583092 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fc9880cc650 con 0x55faa706bcd0 2024-03-28T13:16:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.633+0000 7fc99943ef40 1 --2- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x55faa7148430 0x55faa7123660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.633+0000 7fc99943ef40 1 -- 172.21.15.19:0/1760583092 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e93) -- 0x55faa7128ed0 con 0x55faa7148430 2024-03-28T13:16:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.633+0000 7fc98d400640 1 --2- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc97406af10 0x7fc97406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.634+0000 7fc996800640 1 --2- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x55faa7148430 0x55faa7123660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.635+0000 7fc98d400640 1 --2- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc97406af10 0x7fc97406d3d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55faa70b6c50 tx=0x7fc98000c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.635+0000 7fc996800640 1 --2- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x55faa7148430 0x55faa7123660 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.641+0000 7fc996800640 1 -- 172.21.15.19:0/1760583092 <== osd.6 v2:172.21.15.188:6816/3597358155 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v93'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc990009790 con 0x55faa7148430 2024-03-28T13:16:39.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.641+0000 7fc99943ef40 1 -- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x55faa7148430 msgr2=0x55faa7123660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.641+0000 7fc99943ef40 1 --2- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x55faa7148430 0x55faa7123660 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.641+0000 7fc99943ef40 1 -- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc97406af10 msgr2=0x7fc97406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.641+0000 7fc99943ef40 1 --2- 172.21.15.19:0/1760583092 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc97406af10 0x7fc97406d3d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55faa70b6c50 tx=0x7fc98000c000 comp rx=0 tx=0).stop 2024-03-28T13:16:39.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.641+0000 7fc99943ef40 1 -- 172.21.15.19:0/1760583092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa706bcd0 msgr2=0x55faa7121ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.641+0000 7fc99943ef40 1 --2- 172.21.15.19:0/1760583092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55faa706bcd0 0x55faa7121ef0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc98801ccd0 tx=0x7fc988003330 comp rx=0 tx=0).stop 2024-03-28T13:16:39.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.641+0000 7fc996800640 1 -- 172.21.15.19:0/1760583092 reap_dead start 2024-03-28T13:16:39.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.641+0000 7fc99943ef40 1 -- 172.21.15.19:0/1760583092 shutdown_connections 2024-03-28T13:16:39.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.641+0000 7fc99943ef40 1 -- 172.21.15.19:0/1760583092 >> 172.21.15.19:0/1760583092 conn(0x55faa706c8a0 msgr2=0x55faa70b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:39.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.642+0000 7fc99943ef40 1 -- 172.21.15.19:0/1760583092 shutdown_connections 2024-03-28T13:16:39.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.642+0000 7fc99943ef40 1 -- 172.21.15.19:0/1760583092 wait complete. 2024-03-28T13:16:39.646 DEBUG:teuthology.orchestra.run.smithi019:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-03-28T13:16:39.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.736+0000 7ff75142e280 1 -- start start 2024-03-28T13:16:39.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.736+0000 7ff75142e280 1 Processor -- start 2024-03-28T13:16:39.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.737+0000 7ff75142e280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561193452c00 0x56119421a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.737+0000 7ff75142e280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561193453800 0x56119421a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.737+0000 7ff75142e280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561193453c00 0x56119421ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.737+0000 7ff75142e280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5611941b6600 con 0x561193452c00 2024-03-28T13:16:39.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.737+0000 7ff75142e280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5611941b6480 con 0x561193453c00 2024-03-28T13:16:39.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.737+0000 7ff75142e280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5611941b6300 con 0x561193453800 2024-03-28T13:16:39.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.737+0000 7ff74e400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561193452c00 0x56119421a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.737+0000 7ff74e400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561193452c00 0x56119421a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60916/0 (socket says 172.21.15.19:60916) 2024-03-28T13:16:39.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.737+0000 7ff74e400640 1 -- 172.21.15.19:0/105837538 learned_addr learned my addr 172.21.15.19:0/105837538 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:39.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.738+0000 7ff74ee00640 1 --2- 172.21.15.19:0/105837538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561193453c00 0x56119421ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.738+0000 7ff74da00640 1 --2- 172.21.15.19:0/105837538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561193453800 0x56119421a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.738+0000 7ff74e400640 1 -- 172.21.15.19:0/105837538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561193453800 msgr2=0x56119421a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:39.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.738+0000 7ff74e400640 1 --2- 172.21.15.19:0/105837538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561193453800 0x56119421a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.738+0000 7ff74e400640 1 -- 172.21.15.19:0/105837538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561193453c00 msgr2=0x56119421ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:39.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.738+0000 7ff74e400640 1 --2- 172.21.15.19:0/105837538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561193453c00 0x56119421ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.738+0000 7ff74e400640 1 -- 172.21.15.19:0/105837538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561193486780 con 0x561193452c00 2024-03-28T13:16:39.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.738+0000 7ff74ee00640 1 --2- 172.21.15.19:0/105837538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561193453c00 0x56119421ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:39.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.738+0000 7ff74e400640 1 --2- 172.21.15.19:0/105837538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561193452c00 0x56119421a000 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x56119426b470 tx=0x5611941e61e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fbd7c0a235e629 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.738+0000 7ff74b200640 1 -- 172.21.15.19:0/105837538 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561193478680 con 0x561193452c00 2024-03-28T13:16:39.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.738+0000 7ff74b200640 1 -- 172.21.15.19:0/105837538 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56119347e700 con 0x561193452c00 2024-03-28T13:16:39.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.738+0000 7ff74b200640 1 -- 172.21.15.19:0/105837538 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561193478b60 con 0x561193452c00 2024-03-28T13:16:39.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.739+0000 7ff75142e280 1 -- 172.21.15.19:0/105837538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611934863c0 con 0x561193452c00 2024-03-28T13:16:39.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.739+0000 7ff75142e280 1 -- 172.21.15.19:0/105837538 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=0}) -- 0x561193487680 con 0x561193452c00 2024-03-28T13:16:39.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.739+0000 7ff75142e280 4 waiting for MDS map... 2024-03-28T13:16:39.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.739+0000 7ff74b200640 1 -- 172.21.15.19:0/105837538 <== mon.0 v2:172.21.15.19:3300/0 4 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x56119431c600 con 0x561193452c00 2024-03-28T13:16:39.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.739+0000 7ff74b200640 1 -- 172.21.15.19:0/105837538 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 62) ==== 2885+0+0 (secure 0 0 0) 0x5611934ad180 con 0x561193452c00 2024-03-28T13:16:39.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.739+0000 7ff75142e280 4 Got MDS map 62 2024-03-28T13:16:39.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.739+0000 7ff75142e280 10 main: JournalTool::main 2024-03-28T13:16:39.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.739+0000 7ff75142e280 4 main: JournalTool: connecting to RADOS... 2024-03-28T13:16:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.740+0000 7ff75142e280 1 Processor -- start 2024-03-28T13:16:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.740+0000 7ff75142e280 1 -- start start 2024-03-28T13:16:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.741+0000 7ff75142e280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119431f400 0x56119421b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.741+0000 7ff75142e280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56119431f800 0x56119421b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.741+0000 7ff75142e280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56119431fc00 0x56119421bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.741+0000 7ff75142e280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5611941b6180 con 0x56119431f400 2024-03-28T13:16:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.741+0000 7ff75142e280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5611934b4480 con 0x56119431fc00 2024-03-28T13:16:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.741+0000 7ff75142e280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5611934b4900 con 0x56119431f800 2024-03-28T13:16:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.741+0000 7ff74ee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119431f400 0x56119421b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.741+0000 7ff74ee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119431f400 0x56119421b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60924/0 (socket says 172.21.15.19:60924) 2024-03-28T13:16:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.741+0000 7ff74ee00640 1 -- 172.21.15.19:0/955127393 learned_addr learned my addr 172.21.15.19:0/955127393 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.742+0000 7ff74e400640 1 --2- 172.21.15.19:0/955127393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56119431f800 0x56119421b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.742+0000 7ff74da00640 1 --2- 172.21.15.19:0/955127393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56119431fc00 0x56119421bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.742+0000 7ff74ee00640 1 -- 172.21.15.19:0/955127393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56119431f800 msgr2=0x56119421b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.742+0000 7ff74ee00640 1 --2- 172.21.15.19:0/955127393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56119431f800 0x56119421b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.742+0000 7ff74ee00640 1 -- 172.21.15.19:0/955127393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56119431fc00 msgr2=0x56119421bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.742+0000 7ff74ee00640 1 --2- 172.21.15.19:0/955127393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56119431fc00 0x56119421bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.742+0000 7ff74ee00640 1 -- 172.21.15.19:0/955127393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611934874a0 con 0x56119431f400 2024-03-28T13:16:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.742+0000 7ff74ee00640 1 --2- 172.21.15.19:0/955127393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119431f400 0x56119421b080 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x56119432e300 tx=0x561194332cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcabd5b138090ca7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.742+0000 7ff748a00640 1 -- 172.21.15.19:0/955127393 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561193478680 con 0x56119431f400 2024-03-28T13:16:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.742+0000 7ff748a00640 1 -- 172.21.15.19:0/955127393 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56119347ea80 con 0x56119431f400 2024-03-28T13:16:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.742+0000 7ff748a00640 1 -- 172.21.15.19:0/955127393 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561193478b60 con 0x56119431f400 2024-03-28T13:16:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.743+0000 7ff75142e280 1 -- 172.21.15.19:0/955127393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119431f400 msgr2=0x56119421b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.743+0000 7ff75142e280 1 --2- 172.21.15.19:0/955127393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119431f400 0x56119421b080 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x56119432e300 tx=0x561194332cc0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.743+0000 7ff75142e280 1 -- 172.21.15.19:0/955127393 shutdown_connections 2024-03-28T13:16:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.743+0000 7ff75142e280 1 --2- 172.21.15.19:0/955127393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56119431fc00 0x56119421bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.743+0000 7ff75142e280 1 --2- 172.21.15.19:0/955127393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56119431f800 0x56119421b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.743+0000 7ff75142e280 1 --2- 172.21.15.19:0/955127393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119431f400 0x56119421b080 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.743+0000 7ff75142e280 1 -- 172.21.15.19:0/955127393 >> 172.21.15.19:0/955127393 conn(0x56119431f000 msgr2=0x5611941ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.743+0000 7ff75142e280 1 -- 172.21.15.19:0/955127393 shutdown_connections 2024-03-28T13:16:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.743+0000 7ff75142e280 1 -- 172.21.15.19:0/955127393 wait complete. 2024-03-28T13:16:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.744+0000 7ff75142e280 1 Processor -- start 2024-03-28T13:16:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.744+0000 7ff75142e280 1 -- start start 2024-03-28T13:16:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.744+0000 7ff75142e280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56119431f400 0x56119421b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.745+0000 7ff75142e280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56119431f800 0x56119421b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.745+0000 7ff75142e280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119431fc00 0x56119421bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.745+0000 7ff75142e280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5611941b7380 con 0x56119431fc00 2024-03-28T13:16:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.745+0000 7ff75142e280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5611941b6180 con 0x56119431f800 2024-03-28T13:16:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.745+0000 7ff75142e280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5611941b7500 con 0x56119431f400 2024-03-28T13:16:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.745+0000 7ff74da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119431fc00 0x56119421bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.745+0000 7ff74da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119431fc00 0x56119421bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60932/0 (socket says 172.21.15.19:60932) 2024-03-28T13:16:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.745+0000 7ff74da00640 1 -- 172.21.15.19:0/29756573 learned_addr learned my addr 172.21.15.19:0/29756573 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.745+0000 7ff74e400640 1 --2- 172.21.15.19:0/29756573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56119431f800 0x56119421b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.745+0000 7ff74da00640 1 -- 172.21.15.19:0/29756573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56119431f400 msgr2=0x56119421b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.745+0000 7ff74da00640 1 --2- 172.21.15.19:0/29756573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56119431f400 0x56119421b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.745+0000 7ff74da00640 1 -- 172.21.15.19:0/29756573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56119431f800 msgr2=0x56119421b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.745+0000 7ff74da00640 1 --2- 172.21.15.19:0/29756573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56119431f800 0x56119421b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.745+0000 7ff74da00640 1 -- 172.21.15.19:0/29756573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561193486780 con 0x56119431fc00 2024-03-28T13:16:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.745+0000 7ff74e400640 1 --2- 172.21.15.19:0/29756573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56119431f800 0x56119421b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.746+0000 7ff74da00640 1 --2- 172.21.15.19:0/29756573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119431fc00 0x56119421bb80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x56119426aa50 tx=0x5611941e6180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.746+0000 7ff746c00640 1 -- 172.21.15.19:0/29756573 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5611934789c0 con 0x56119431fc00 2024-03-28T13:16:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.746+0000 7ff746c00640 1 -- 172.21.15.19:0/29756573 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56119347ec40 con 0x56119431fc00 2024-03-28T13:16:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.746+0000 7ff75142e280 1 -- 172.21.15.19:0/29756573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5611934863c0 con 0x56119431fc00 2024-03-28T13:16:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.746+0000 7ff746c00640 1 -- 172.21.15.19:0/29756573 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561193478680 con 0x56119431fc00 2024-03-28T13:16:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.746+0000 7ff75142e280 1 -- 172.21.15.19:0/29756573 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561193486000 con 0x56119431fc00 2024-03-28T13:16:39.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.746+0000 7ff75142e280 4 main: JournalTool: resolving pool 12 2024-03-28T13:16:39.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.748+0000 7ff746c00640 1 -- 172.21.15.19:0/29756573 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x561194398c00 con 0x56119431fc00 2024-03-28T13:16:39.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.748+0000 7ff746c00640 1 --2- 172.21.15.19:0/29756573 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x561194390c00 0x56119421c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.749+0000 7ff746c00640 1 -- 172.21.15.19:0/29756573 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x561194392c00 con 0x56119431fc00 2024-03-28T13:16:39.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.749+0000 7ff75142e280 4 main: JournalTool: creating IoCtx.. 2024-03-28T13:16:39.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.749+0000 7ff75142e280 4 main: Executing for rank 0 2024-03-28T13:16:39.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.749+0000 7ff74ee00640 1 --2- 172.21.15.19:0/29756573 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x561194390c00 0x56119421c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.750+0000 7ff75142e280 1 -- start start 2024-03-28T13:16:39.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.750+0000 7ff75142e280 1 Processor -- start 2024-03-28T13:16:39.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.750+0000 7ff74ee00640 1 --2- 172.21.15.19:0/29756573 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x561194390c00 0x56119421c100 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x56119432e300 tx=0x561194332c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff75142e280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561194391400 0x56119421c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff75142e280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561194391800 0x56119421cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff75142e280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561194391c00 0x56119421d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff75142e280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5611941b7800 con 0x561194391800 2024-03-28T13:16:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff75142e280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5611941b7980 con 0x561194391c00 2024-03-28T13:16:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff75142e280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5611934b4900 con 0x561194391400 2024-03-28T13:16:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff74e400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561194391800 0x56119421cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff74e400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561194391800 0x56119421cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60940/0 (socket says 172.21.15.19:60940) 2024-03-28T13:16:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff74e400640 1 -- 172.21.15.19:0/801079703 learned_addr learned my addr 172.21.15.19:0/801079703 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff74ee00640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561194391400 0x56119421c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff74da00640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561194391c00 0x56119421d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff74e400640 1 -- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561194391400 msgr2=0x56119421c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff74e400640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561194391400 0x56119421c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff74e400640 1 -- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561194391c00 msgr2=0x56119421d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff74e400640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561194391c00 0x56119421d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff74e400640 1 -- 172.21.15.19:0/801079703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561193486960 con 0x561194391800 2024-03-28T13:16:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.751+0000 7ff74da00640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561194391c00 0x56119421d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.752+0000 7ff74e400640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561194391800 0x56119421cc00 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x5611937655f0 tx=0x5611941e4a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e426ee7fdf4feb84 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.752+0000 7ff743000640 1 -- 172.21.15.19:0/801079703 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561193478820 con 0x561194391800 2024-03-28T13:16:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.752+0000 7ff743000640 1 -- 172.21.15.19:0/801079703 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56119347e8c0 con 0x561194391800 2024-03-28T13:16:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.752+0000 7ff743000640 1 -- 172.21.15.19:0/801079703 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5611943c41a0 con 0x561194391800 2024-03-28T13:16:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.752+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561193487860 con 0x561194391800 2024-03-28T13:16:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.752+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=0}) -- 0x561193487a40 con 0x561194391800 2024-03-28T13:16:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.752+0000 7ff75142e280 4 waiting for MDS map... 2024-03-28T13:16:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.752+0000 7ff743000640 1 -- 172.21.15.19:0/801079703 <== mon.0 v2:172.21.15.19:3300/0 4 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x56119381ac00 con 0x561194391800 2024-03-28T13:16:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.752+0000 7ff743000640 1 -- 172.21.15.19:0/801079703 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 62) ==== 2885+0+0 (secure 0 0 0) 0x5611934ad500 con 0x561194391800 2024-03-28T13:16:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.752+0000 7ff75142e280 4 Got MDS map 62 2024-03-28T13:16:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.753+0000 7ff75142e280 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x56119381d800 0x56119421d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.753+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x56119381dc00 con 0x56119381d800 2024-03-28T13:16:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.753+0000 7ff74ee00640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x56119381d800 0x56119421d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.753+0000 7ff74ee00640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x56119381d800 0x56119421d700 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=5d71cda23f20feec server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.754+0000 7ff74ee00640 1 -- 172.21.15.19:0/801079703 <== osd.3 v2:172.21.15.19:6819/3265515865 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x56119352e000 con 0x56119381d800 2024-03-28T13:16:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.754+0000 7ff75142e280 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x561193832000 0x561193838000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.754+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:2 12.14 12:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) -- 0x561193832400 con 0x561193832000 2024-03-28T13:16:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.755+0000 7ff74da00640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x561193832000 0x561193838000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.755+0000 7ff74da00640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x561193832000 0x561193838000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=2d98e799ecb77296 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.755+0000 7ff74da00640 1 -- 172.21.15.19:0/801079703 <== osd.0 v2:172.21.15.19:6800/4106483280 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x56119352e500 con 0x561193832000 2024-03-28T13:16:39.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.756+0000 7ff741200640 1 -- 172.21.15.19:0/801079703 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:3 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e93) -- 0x561193833c00 con 0x561193832000 2024-03-28T13:16:39.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.756+0000 7ff741200640 1 -- 172.21.15.19:0/801079703 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:4 12.1b 12:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e93) -- 0x56119383c000 con 0x561193832000 2024-03-28T13:16:39.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.756+0000 7ff74da00640 1 -- 172.21.15.19:0/801079703 <== osd.0 v2:172.21.15.19:6800/4106483280 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x56119352e500 con 0x561193832000 2024-03-28T13:16:39.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.756+0000 7ff74da00640 1 -- 172.21.15.19:0/801079703 <== osd.0 v2:172.21.15.19:6800/4106483280 3 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56119352e500 con 0x561193832000 2024-03-28T13:16:39.761 INFO:teuthology.orchestra.run.smithi019.stdout:old journal was 4212746~2605 2024-03-28T13:16:39.761 INFO:teuthology.orchestra.run.smithi019.stdout:new journal start will be 8388608 (4173257 bytes past old end) 2024-03-28T13:16:39.761 INFO:teuthology.orchestra.run.smithi019.stdout:writing journal head 2024-03-28T13:16:39.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.756+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:5 12.14 12:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x561193832800 con 0x561193832000 2024-03-28T13:16:39.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.759+0000 7ff74da00640 1 -- 172.21.15.19:0/801079703 <== osd.0 v2:172.21.15.19:6800/4106483280 4 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v93'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x56119352e500 con 0x561193832000 2024-03-28T13:16:39.762 INFO:teuthology.orchestra.run.smithi019.stdout:writing EResetJournal entry 2024-03-28T13:16:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.760+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:6 12.1b 12:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x561193832c00 con 0x561193832000 2024-03-28T13:16:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.760+0000 7ff75142e280 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x561193833000 0x561193838580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.760+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:7 12.b 12:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x561193833400 con 0x561193833000 2024-03-28T13:16:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.760+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:8 12.1f 12:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x561193833800 con 0x56119381d800 2024-03-28T13:16:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.760+0000 7ff75142e280 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x56119383c000 0x561193838b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.760+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:9 12.1a 12:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x56119383c800 con 0x56119383c000 2024-03-28T13:16:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.760+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:10 12.1e 12:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x56119383cc00 con 0x56119383c000 2024-03-28T13:16:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.760+0000 7ff75142e280 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x56119383d000 0x561193839080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.760+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:11 12.1d 12:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x561193832400 con 0x56119383d000 2024-03-28T13:16:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.760+0000 7ff74e400640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x56119383c000 0x561193838b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.761+0000 7ff74ee00640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x561193833000 0x561193838580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.761+0000 7ff74da00640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x56119383d000 0x561193839080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.761+0000 7ff74e400640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x56119383c000 0x561193838b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=dab9215b7598e7cb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.762+0000 7ff74da00640 1 -- 172.21.15.19:0/801079703 <== osd.0 v2:172.21.15.19:6800/4106483280 5 ==== osd_op_reply(6 200.00000002 [delete] v93'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56119352e500 con 0x561193832000 2024-03-28T13:16:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.763+0000 7ff74ee00640 1 -- 172.21.15.19:0/801079703 <== osd.3 v2:172.21.15.19:6819/3265515865 2 ==== osd_op_reply(8 200.00000004 [delete] v93'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56119352e000 con 0x56119381d800 2024-03-28T13:16:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.763+0000 7ff74da00640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x56119383d000 0x561193839080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=b479923fe0fdccd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.763+0000 7ff74ee00640 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x561193833000 0x561193838580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=5bb2a626ecab60ee server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.766+0000 7ff74da00640 1 -- 172.21.15.19:0/801079703 <== osd.6 v2:172.21.15.188:6816/3597358155 1 ==== osd_op_reply(11 200.00000007 [delete] v93'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56119352e500 con 0x56119383d000 2024-03-28T13:16:39.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.767+0000 7ff74ee00640 1 -- 172.21.15.19:0/801079703 <== osd.7 v2:172.21.15.188:6824/861796513 1 ==== osd_op_reply(7 200.00000003 [delete] v93'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56119352e000 con 0x561193833000 2024-03-28T13:16:39.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.767+0000 7ff74e400640 1 -- 172.21.15.19:0/801079703 <== osd.2 v2:172.21.15.19:6816/1693606238 1 ==== osd_op_reply(9 200.00000005 [delete] v93'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56119352e780 con 0x56119383c000 2024-03-28T13:16:39.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.768+0000 7ff741200640 1 -- 172.21.15.19:0/801079703 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:12 12.1b 12:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x56119383c400 con 0x561193832000 2024-03-28T13:16:39.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.768+0000 7ff74e400640 1 -- 172.21.15.19:0/801079703 <== osd.2 v2:172.21.15.19:6816/1693606238 2 ==== osd_op_reply(10 200.00000006 [delete] v93'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x56119352e780 con 0x56119383c000 2024-03-28T13:16:39.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.769+0000 7ff74da00640 1 -- 172.21.15.19:0/801079703 <== osd.0 v2:172.21.15.19:6800/4106483280 6 ==== osd_op_reply(12 200.00000002 [write 0~48] v93'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x56119352e500 con 0x561193832000 2024-03-28T13:16:39.769 INFO:teuthology.orchestra.run.smithi019.stdout:done 2024-03-28T13:16:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x561193832000 msgr2=0x561193838000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x561193832000 0x561193838000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x56119383c000 msgr2=0x561193838b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x56119383c000 0x561193838b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x56119381d800 msgr2=0x56119421d700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x56119381d800 0x56119421d700 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x56119383d000 msgr2=0x561193839080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x56119383d000 0x561193839080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x561193833000 msgr2=0x561193838580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x561193833000 0x561193838580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff74e400640 1 -- 172.21.15.19:0/801079703 reap_dead start 2024-03-28T13:16:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561194391800 msgr2=0x56119421cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561194391800 0x56119421cc00 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x5611937655f0 tx=0x5611941e4a80 comp rx=0 tx=0).stop 2024-03-28T13:16:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 shutdown_connections 2024-03-28T13:16:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 --2- 172.21.15.19:0/801079703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561194391800 0x56119421cc00 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 >> 172.21.15.19:0/801079703 conn(0x561194391000 msgr2=0x5611941ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 shutdown_connections 2024-03-28T13:16:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.770+0000 7ff75142e280 1 -- 172.21.15.19:0/801079703 wait complete. 2024-03-28T13:16:39.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 -- 172.21.15.19:0/29756573 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x561194390c00 msgr2=0x56119421c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 --2- 172.21.15.19:0/29756573 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x561194390c00 0x56119421c100 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x56119432e300 tx=0x561194332c00 comp rx=0 tx=0).stop 2024-03-28T13:16:39.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 -- 172.21.15.19:0/29756573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119431fc00 msgr2=0x56119421bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 --2- 172.21.15.19:0/29756573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119431fc00 0x56119421bb80 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x56119426aa50 tx=0x5611941e6180 comp rx=0 tx=0).stop 2024-03-28T13:16:39.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 -- 172.21.15.19:0/29756573 shutdown_connections 2024-03-28T13:16:39.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 --2- 172.21.15.19:0/29756573 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x561194390c00 0x56119421c100 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 --2- 172.21.15.19:0/29756573 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56119431fc00 0x56119421bb80 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 --2- 172.21.15.19:0/29756573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56119431f800 0x56119421b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 --2- 172.21.15.19:0/29756573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56119431f400 0x56119421b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 -- 172.21.15.19:0/29756573 >> 172.21.15.19:0/29756573 conn(0x56119431f000 msgr2=0x5611941ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:39.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 -- 172.21.15.19:0/29756573 shutdown_connections 2024-03-28T13:16:39.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 -- 172.21.15.19:0/29756573 wait complete. 2024-03-28T13:16:39.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 -- 172.21.15.19:0/105837538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561193452c00 msgr2=0x56119421a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 --2- 172.21.15.19:0/105837538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561193452c00 0x56119421a000 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x56119426b470 tx=0x5611941e61e0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 -- 172.21.15.19:0/105837538 shutdown_connections 2024-03-28T13:16:39.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 --2- 172.21.15.19:0/105837538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561193453c00 0x56119421ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.771+0000 7ff75142e280 1 --2- 172.21.15.19:0/105837538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561193453800 0x56119421a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.772+0000 7ff75142e280 1 --2- 172.21.15.19:0/105837538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561193452c00 0x56119421a000 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.772+0000 7ff75142e280 1 -- 172.21.15.19:0/105837538 >> 172.21.15.19:0/105837538 conn(0x561193453000 msgr2=0x5611941cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:39.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.772+0000 7ff75142e280 1 -- 172.21.15.19:0/105837538 shutdown_connections 2024-03-28T13:16:39.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.772+0000 7ff75142e280 1 -- 172.21.15.19:0/105837538 wait complete. 2024-03-28T13:16:39.776 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.129988, result: old journal was 4212746~2605 new journal start will be 8388608 (4173257 bytes past old end) writing journal head writing EResetJournal entry done 2024-03-28T13:16:39.777 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-28T13:16:39.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.979+0000 7f198b000640 1 Processor -- start 2024-03-28T13:16:39.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.979+0000 7f198b000640 1 -- start start 2024-03-28T13:16:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.980+0000 7f198b000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f19840b3ec0 0x7f198406d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.980+0000 7f198b000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f198406db60 0x7f19840add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.980+0000 7f198b000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f198415c330 con 0x7f198415bf50 2024-03-28T13:16:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.980+0000 7f198b000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f198406ee40 con 0x7f198406db60 2024-03-28T13:16:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.980+0000 7f198b000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f198415ed30 con 0x7f19840b3ec0 2024-03-28T13:16:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.981+0000 7f198a600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f198406db60 0x7f19840add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.981+0000 7f198a600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f198406db60 0x7f19840add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60950/0 (socket says 172.21.15.19:60950) 2024-03-28T13:16:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.981+0000 7f198a600640 1 -- 172.21.15.19:0/3846992102 learned_addr learned my addr 172.21.15.19:0/3846992102 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.981+0000 7f1989200640 1 --2- 172.21.15.19:0/3846992102 >> v2:172.21.15.188:3300/0 conn(0x7f19840b3ec0 0x7f198406d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.981+0000 7f198a600640 1 -- 172.21.15.19:0/3846992102 >> v2:172.21.15.188:3300/0 conn(0x7f19840b3ec0 msgr2=0x7f198406d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.981+0000 7f198a600640 1 --2- 172.21.15.19:0/3846992102 >> v2:172.21.15.188:3300/0 conn(0x7f19840b3ec0 0x7f198406d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.981+0000 7f198a600640 1 -- 172.21.15.19:0/3846992102 >> v1:172.21.15.188:6789/0 conn(0x7f198415bf50 legacy=0x7f19840b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.981+0000 7f198a600640 1 -- 172.21.15.19:0/3846992102 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198406dfb0 con 0x7f198406db60 2024-03-28T13:16:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.981+0000 7f1989200640 1 --2- 172.21.15.19:0/3846992102 >> v2:172.21.15.188:3300/0 conn(0x7f19840b3ec0 0x7f198406d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.982+0000 7f198a600640 1 --2- 172.21.15.19:0/3846992102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f198406db60 0x7f19840add30 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1984071030 tx=0x7f197c02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ada817e3374429 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.983+0000 7f1983e00640 1 -- 172.21.15.19:0/3846992102 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c02cbb0 con 0x7f198406db60 2024-03-28T13:16:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.983+0000 7f1983e00640 1 -- 172.21.15.19:0/3846992102 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f197c02cd50 con 0x7f198406db60 2024-03-28T13:16:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.983+0000 7f1983e00640 1 -- 172.21.15.19:0/3846992102 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197c034610 con 0x7f198406db60 2024-03-28T13:16:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.983+0000 7f198b000640 1 -- 172.21.15.19:0/3846992102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f198406db60 msgr2=0x7f19840add30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.983+0000 7f198b000640 1 --2- 172.21.15.19:0/3846992102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f198406db60 0x7f19840add30 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1984071030 tx=0x7f197c02c010 comp rx=0 tx=0).stop 2024-03-28T13:16:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.983+0000 7f198b000640 1 -- 172.21.15.19:0/3846992102 shutdown_connections 2024-03-28T13:16:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.983+0000 7f198b000640 1 --2- 172.21.15.19:0/3846992102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f198406db60 0x7f19840add30 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.983+0000 7f198b000640 1 --2- 172.21.15.19:0/3846992102 >> v2:172.21.15.188:3300/0 conn(0x7f19840b3ec0 0x7f198406d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.983+0000 7f198b000640 1 -- 172.21.15.19:0/3846992102 >> 172.21.15.19:0/3846992102 conn(0x7f19840b1ba0 msgr2=0x7f19840b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.984+0000 7f198b000640 1 -- 172.21.15.19:0/3846992102 shutdown_connections 2024-03-28T13:16:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.984+0000 7f198b000640 1 -- 172.21.15.19:0/3846992102 wait complete. 2024-03-28T13:16:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.985+0000 7f198b000640 1 Processor -- start 2024-03-28T13:16:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.985+0000 7f198b000640 1 -- start start 2024-03-28T13:16:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.986+0000 7f198b000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f198406db60 0x7f19840c4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.986+0000 7f198b000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19840b3ec0 0x7f19841564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.986+0000 7f198b000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198415bf50 0x7f19840b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.986+0000 7f198b000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f198406e7f0 con 0x7f19840b3ec0 2024-03-28T13:16:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.986+0000 7f198b000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f198406e510 con 0x7f198406db60 2024-03-28T13:16:39.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.986+0000 7f198b000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f198406ead0 con 0x7f198415bf50 2024-03-28T13:16:39.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.987+0000 7f1989200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19840b3ec0 0x7f19841564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.987+0000 7f1989200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19840b3ec0 0x7f19841564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60960/0 (socket says 172.21.15.19:60960) 2024-03-28T13:16:39.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.987+0000 7f1989200640 1 -- 172.21.15.19:0/959156750 learned_addr learned my addr 172.21.15.19:0/959156750 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:39.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.987+0000 7f1989c00640 1 --2- 172.21.15.19:0/959156750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f198406db60 0x7f19840c4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.987+0000 7f1989200640 1 -- 172.21.15.19:0/959156750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198415bf50 msgr2=0x7f19840b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.987+0000 7f1989200640 1 --2- 172.21.15.19:0/959156750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198415bf50 0x7f19840b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.987+0000 7f1989200640 1 -- 172.21.15.19:0/959156750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f198406db60 msgr2=0x7f19840c4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:39.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.987+0000 7f1989200640 1 --2- 172.21.15.19:0/959156750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f198406db60 0x7f19840c4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:39.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.987+0000 7f1989200640 1 -- 172.21.15.19:0/959156750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19840c5560 con 0x7f19840b3ec0 2024-03-28T13:16:39.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.988+0000 7f1989200640 1 --2- 172.21.15.19:0/959156750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19840b3ec0 0x7f19841564c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f197000c7f0 tx=0x7f197000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.988+0000 7f1981600640 1 -- 172.21.15.19:0/959156750 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970015070 con 0x7f19840b3ec0 2024-03-28T13:16:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.988+0000 7f1981600640 1 -- 172.21.15.19:0/959156750 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1970007a30 con 0x7f19840b3ec0 2024-03-28T13:16:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.988+0000 7f1981600640 1 -- 172.21.15.19:0/959156750 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970007e00 con 0x7f19840b3ec0 2024-03-28T13:16:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.988+0000 7f198b000640 1 -- 172.21.15.19:0/959156750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19840af860 con 0x7f19840b3ec0 2024-03-28T13:16:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.988+0000 7f198b000640 1 -- 172.21.15.19:0/959156750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19840afe20 con 0x7f19840b3ec0 2024-03-28T13:16:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.989+0000 7f198b000640 1 -- 172.21.15.19:0/959156750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1984075940 con 0x7f19840b3ec0 2024-03-28T13:16:39.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.990+0000 7f1981600640 1 -- 172.21.15.19:0/959156750 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1970018070 con 0x7f19840b3ec0 2024-03-28T13:16:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.991+0000 7f1981600640 1 --2- 172.21.15.19:0/959156750 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f196406c530 0x7f196406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.991+0000 7f1981600640 1 -- 172.21.15.19:0/959156750 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f197008b7b0 con 0x7f19840b3ec0 2024-03-28T13:16:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.993+0000 7f1989c00640 1 --2- 172.21.15.19:0/959156750 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f196406c530 0x7f196406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.994+0000 7f1989c00640 1 --2- 172.21.15.19:0/959156750 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f196406c530 0x7f196406e9f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f198415aa90 tx=0x7f1974009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:39.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:39.996+0000 7f1981600640 1 -- 172.21.15.19:0/959156750 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1970059cd0 con 0x7f19840b3ec0 2024-03-28T13:16:40.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.250+0000 7f198b000640 1 -- 172.21.15.19:0/959156750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f198415f050 con 0x7f19840b3ec0 2024-03-28T13:16:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.499+0000 7f1981600640 1 -- 172.21.15.19:0/959156750 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v63) ==== 162+0+0 (secure 0 0 0) 0x7f19700595d0 con 0x7f19840b3ec0 2024-03-28T13:16:40.499 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-28T13:16:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.502+0000 7f198b000640 1 -- 172.21.15.19:0/959156750 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f196406c530 msgr2=0x7f196406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:40.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.502+0000 7f198b000640 1 --2- 172.21.15.19:0/959156750 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f196406c530 0x7f196406e9f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f198415aa90 tx=0x7f1974009040 comp rx=0 tx=0).stop 2024-03-28T13:16:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.502+0000 7f198b000640 1 -- 172.21.15.19:0/959156750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19840b3ec0 msgr2=0x7f19841564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.502+0000 7f198b000640 1 --2- 172.21.15.19:0/959156750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19840b3ec0 0x7f19841564c0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f197000c7f0 tx=0x7f197000cbf0 comp rx=0 tx=0).stop 2024-03-28T13:16:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.503+0000 7f198b000640 1 -- 172.21.15.19:0/959156750 shutdown_connections 2024-03-28T13:16:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.503+0000 7f198b000640 1 --2- 172.21.15.19:0/959156750 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f196406c530 0x7f196406e9f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.503+0000 7f198b000640 1 --2- 172.21.15.19:0/959156750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198415bf50 0x7f19840b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.503+0000 7f198b000640 1 --2- 172.21.15.19:0/959156750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f19840b3ec0 0x7f19841564c0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.503+0000 7f198b000640 1 --2- 172.21.15.19:0/959156750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f198406db60 0x7f19840c4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.503+0000 7f198b000640 1 -- 172.21.15.19:0/959156750 >> 172.21.15.19:0/959156750 conn(0x7f19840b1ba0 msgr2=0x7f198415cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:40.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.503+0000 7f198b000640 1 -- 172.21.15.19:0/959156750 shutdown_connections 2024-03-28T13:16:40.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.503+0000 7f198b000640 1 -- 172.21.15.19:0/959156750 wait complete. 2024-03-28T13:16:40.530 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:16:40.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.755+0000 7fb4e0800640 1 Processor -- start 2024-03-28T13:16:40.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.755+0000 7fb4e0800640 1 -- start start 2024-03-28T13:16:40.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.756+0000 7fb4e0800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb4d80ad120 0x7fb4d80d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:40.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.757+0000 7fb4e0800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d80ad580 0x7fb4d80d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:40.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.757+0000 7fb4e0800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4d8071b60 con 0x7fb4d8161b00 2024-03-28T13:16:40.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.757+0000 7fb4e0800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb4d8161ee0 con 0x7fb4d80ad120 2024-03-28T13:16:40.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.757+0000 7fb4e0800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d80b3450 con 0x7fb4d80ad580 2024-03-28T13:16:40.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.757+0000 7fb4dd400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb4d80ad120 0x7fb4d80d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:40.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.757+0000 7fb4de800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d80ad580 0x7fb4d80d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:40.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.757+0000 7fb4dd400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb4d80ad120 0x7fb4d80d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:53034/0 (socket says 172.21.15.19:53034) 2024-03-28T13:16:40.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.757+0000 7fb4dd400640 1 -- 172.21.15.19:0/486053891 learned_addr learned my addr 172.21.15.19:0/486053891 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:40.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.758+0000 7fb4dd400640 1 -- 172.21.15.19:0/486053891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d80ad580 msgr2=0x7fb4d80d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:40.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.758+0000 7fb4dd400640 1 --2- 172.21.15.19:0/486053891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d80ad580 0x7fb4d80d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:40.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.758+0000 7fb4dd400640 1 -- 172.21.15.19:0/486053891 >> v1:172.21.15.188:6789/0 conn(0x7fb4d8161b00 legacy=0x7fb4d8162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:40.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.758+0000 7fb4dd400640 1 -- 172.21.15.19:0/486053891 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d8071e40 con 0x7fb4d80ad120 2024-03-28T13:16:40.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.760+0000 7fb4dd400640 1 --2- 172.21.15.19:0/486053891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4d80ad120 0x7fb4d80d0650 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009870 tx=0x7fb4c802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74703439932a44cd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:40.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.760+0000 7fb4dca00640 1 -- 172.21.15.19:0/486053891 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8035020 con 0x7fb4d80ad120 2024-03-28T13:16:40.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.760+0000 7fb4dca00640 1 -- 172.21.15.19:0/486053891 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4c8031640 con 0x7fb4d80ad120 2024-03-28T13:16:40.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.760+0000 7fb4dca00640 1 -- 172.21.15.19:0/486053891 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c802d040 con 0x7fb4d80ad120 2024-03-28T13:16:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.761+0000 7fb4e0800640 1 -- 172.21.15.19:0/486053891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4d80ad120 msgr2=0x7fb4d80d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.761+0000 7fb4e0800640 1 --2- 172.21.15.19:0/486053891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4d80ad120 0x7fb4d80d0650 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009870 tx=0x7fb4c802b8f0 comp rx=0 tx=0).stop 2024-03-28T13:16:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.761+0000 7fb4e0800640 1 -- 172.21.15.19:0/486053891 shutdown_connections 2024-03-28T13:16:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.761+0000 7fb4e0800640 1 --2- 172.21.15.19:0/486053891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4d80ad120 0x7fb4d80d0650 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.761+0000 7fb4e0800640 1 --2- 172.21.15.19:0/486053891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d80ad580 0x7fb4d80d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.761+0000 7fb4e0800640 1 -- 172.21.15.19:0/486053891 >> 172.21.15.19:0/486053891 conn(0x7fb4d806b990 msgr2=0x7fb4d806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.761+0000 7fb4e0800640 1 -- 172.21.15.19:0/486053891 shutdown_connections 2024-03-28T13:16:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.761+0000 7fb4e0800640 1 -- 172.21.15.19:0/486053891 wait complete. 2024-03-28T13:16:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.764+0000 7fb4e0800640 1 Processor -- start 2024-03-28T13:16:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.764+0000 7fb4e0800640 1 -- start start 2024-03-28T13:16:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.764+0000 7fb4e0800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d80ad120 0x7fb4d8154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.764+0000 7fb4e0800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4d80ad580 0x7fb4d815ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.764+0000 7fb4e0800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8161b00 0x7fb4d81f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.764+0000 7fb4e0800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d80717f0 con 0x7fb4d8161b00 2024-03-28T13:16:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.764+0000 7fb4e0800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d80b3730 con 0x7fb4d80ad580 2024-03-28T13:16:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.764+0000 7fb4e0800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb4d80b5020 con 0x7fb4d80ad120 2024-03-28T13:16:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.765+0000 7fb4de800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8161b00 0x7fb4d81f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.765+0000 7fb4de800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8161b00 0x7fb4d81f5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60970/0 (socket says 172.21.15.19:60970) 2024-03-28T13:16:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.765+0000 7fb4de800640 1 -- 172.21.15.19:0/2914460313 learned_addr learned my addr 172.21.15.19:0/2914460313 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.765+0000 7fb4dd400640 1 --2- 172.21.15.19:0/2914460313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4d80ad580 0x7fb4d815ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.765+0000 7fb4dde00640 1 --2- 172.21.15.19:0/2914460313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d80ad120 0x7fb4d8154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.765+0000 7fb4de800640 1 -- 172.21.15.19:0/2914460313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d80ad120 msgr2=0x7fb4d8154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.765+0000 7fb4de800640 1 --2- 172.21.15.19:0/2914460313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d80ad120 0x7fb4d8154fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.765+0000 7fb4de800640 1 -- 172.21.15.19:0/2914460313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4d80ad580 msgr2=0x7fb4d815ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.765+0000 7fb4de800640 1 --2- 172.21.15.19:0/2914460313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4d80ad580 0x7fb4d815ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.765+0000 7fb4de800640 1 -- 172.21.15.19:0/2914460313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d81f5950 con 0x7fb4d8161b00 2024-03-28T13:16:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.766+0000 7fb4de800640 1 --2- 172.21.15.19:0/2914460313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8161b00 0x7fb4d81f5410 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400efc0 tx=0x7fb4d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.766+0000 7fb4d2000640 1 -- 172.21.15.19:0/2914460313 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4018020 con 0x7fb4d8161b00 2024-03-28T13:16:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.766+0000 7fb4d2000640 1 -- 172.21.15.19:0/2914460313 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4d4014480 con 0x7fb4d8161b00 2024-03-28T13:16:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.766+0000 7fb4e0800640 1 -- 172.21.15.19:0/2914460313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d8153f00 con 0x7fb4d8161b00 2024-03-28T13:16:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.766+0000 7fb4d2000640 1 -- 172.21.15.19:0/2914460313 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4014870 con 0x7fb4d8161b00 2024-03-28T13:16:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.766+0000 7fb4e0800640 1 -- 172.21.15.19:0/2914460313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d81544c0 con 0x7fb4d8161b00 2024-03-28T13:16:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.767+0000 7fb4e0800640 1 -- 172.21.15.19:0/2914460313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4d80685c0 con 0x7fb4d8161b00 2024-03-28T13:16:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.770+0000 7fb4d2000640 1 -- 172.21.15.19:0/2914460313 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb4d4005d10 con 0x7fb4d8161b00 2024-03-28T13:16:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.771+0000 7fb4d2000640 1 --2- 172.21.15.19:0/2914460313 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb4bc06c530 0x7fb4bc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.771+0000 7fb4d2000640 1 -- 172.21.15.19:0/2914460313 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fb4d408c100 con 0x7fb4d8161b00 2024-03-28T13:16:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.771+0000 7fb4dde00640 1 --2- 172.21.15.19:0/2914460313 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb4bc06c530 0x7fb4bc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.772+0000 7fb4dde00640 1 --2- 172.21.15.19:0/2914460313 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb4bc06c530 0x7fb4bc06e9f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc007fd0 tx=0x7fb4cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:40.776+0000 7fb4d2000640 1 -- 172.21.15.19:0/2914460313 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb4d4022070 con 0x7fb4d8161b00 2024-03-28T13:16:41.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:41.019+0000 7fb4e0800640 1 -- 172.21.15.19:0/2914460313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4d806f3a0 con 0x7fb4d8161b00 2024-03-28T13:16:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:41.020+0000 7fb4d2000640 1 -- 172.21.15.19:0/2914460313 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5180 (secure 0 0 0) 0x7fb4d4010040 con 0x7fb4d8161b00 2024-03-28T13:16:41.020 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:41.021 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":64,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:16:40.504790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6175":{"gid":6175,"name":"b","rank":0,"incarnation":64,"state":"up:replay","state_seq":1,"addr":"172.21.15.188:6836/1749969053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1749969053},{"type":"v1","addr":"172.21.15.188:6836","nonce":1749969053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-03-28T13:16:41.021 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 64 2024-03-28T13:16:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:41.023+0000 7fb4e0800640 1 -- 172.21.15.19:0/2914460313 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb4bc06c530 msgr2=0x7fb4bc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:41.023+0000 7fb4e0800640 1 --2- 172.21.15.19:0/2914460313 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb4bc06c530 0x7fb4bc06e9f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc007fd0 tx=0x7fb4cc009040 comp rx=0 tx=0).stop 2024-03-28T13:16:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:41.023+0000 7fb4e0800640 1 -- 172.21.15.19:0/2914460313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8161b00 msgr2=0x7fb4d81f5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:41.023+0000 7fb4e0800640 1 --2- 172.21.15.19:0/2914460313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8161b00 0x7fb4d81f5410 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400efc0 tx=0x7fb4d400c490 comp rx=0 tx=0).stop 2024-03-28T13:16:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:41.023+0000 7fb4e0800640 1 -- 172.21.15.19:0/2914460313 shutdown_connections 2024-03-28T13:16:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:41.023+0000 7fb4e0800640 1 --2- 172.21.15.19:0/2914460313 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb4bc06c530 0x7fb4bc06e9f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:41.023+0000 7fb4e0800640 1 --2- 172.21.15.19:0/2914460313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8161b00 0x7fb4d81f5410 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:41.023+0000 7fb4e0800640 1 --2- 172.21.15.19:0/2914460313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4d80ad580 0x7fb4d815ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:41.024+0000 7fb4e0800640 1 --2- 172.21.15.19:0/2914460313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4d80ad120 0x7fb4d8154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:41.024+0000 7fb4e0800640 1 -- 172.21.15.19:0/2914460313 >> 172.21.15.19:0/2914460313 conn(0x7fb4d806b990 msgr2=0x7fb4d806e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:41.024+0000 7fb4e0800640 1 -- 172.21.15.19:0/2914460313 shutdown_connections 2024-03-28T13:16:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:41.024+0000 7fb4e0800640 1 -- 172.21.15.19:0/2914460313 wait complete. 2024-03-28T13:16:41.444 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:16:25.995386+0000', 'modified': '2024-03-28T13:16:40.504790+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6175}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6175': {'gid': 6175, 'name': 'b', 'rank': 0, 'incarnation': 64, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.188:6836/1749969053', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 1749969053}, {'type': 'v1', 'addr': '172.21.15.188:6836', 'nonce': 1749969053}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-28T13:16:41.444 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6175:up:replay 2024-03-28T13:16:42.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:16:42.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.678+0000 7fc3b9800640 1 Processor -- start 2024-03-28T13:16:42.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.678+0000 7fc3b9800640 1 -- start start 2024-03-28T13:16:42.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.680+0000 7fc3b9800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc3b40d0d60 0x7fc3b40b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:42.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.680+0000 7fc3b9800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3b4071b60 con 0x7fc3b40ad120 2024-03-28T13:16:42.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.680+0000 7fc3b9800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3b4071e40 con 0x7fc3b4161b00 2024-03-28T13:16:42.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.680+0000 7fc3b9800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc3b4161ee0 con 0x7fc3b40d0d60 2024-03-28T13:16:42.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.680+0000 7fc3b2000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc3b40ad120 0x7fc3b40d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53666/0 (socket says 172.21.15.19:53666) 2024-03-28T13:16:42.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.680+0000 7fc3b2000640 1 -- 172.21.15.19:0/1295628140 learned_addr learned my addr 172.21.15.19:0/1295628140 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b3400640 1 --2- 172.21.15.19:0/1295628140 >> v2:172.21.15.188:3300/0 conn(0x7fc3b40d0d60 0x7fc3b40b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b1600640 1 -- 172.21.15.19:0/1295628140 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2486095381 0 0) 0x7fc3b4071b60 con 0x7fc3b40ad120 2024-03-28T13:16:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b1600640 1 -- 172.21.15.19:0/1295628140 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc398003520 con 0x7fc3b40ad120 2024-03-28T13:16:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b1600640 1 -- 172.21.15.19:0/1295628140 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3561206916 0 0) 0x7fc398003520 con 0x7fc3b40ad120 2024-03-28T13:16:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b1600640 1 -- 172.21.15.19:0/1295628140 >> v2:172.21.15.188:3300/0 conn(0x7fc3b40d0d60 msgr2=0x7fc3b40b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b1600640 1 --2- 172.21.15.19:0/1295628140 >> v2:172.21.15.188:3300/0 conn(0x7fc3b40d0d60 0x7fc3b40b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b1600640 1 -- 172.21.15.19:0/1295628140 >> v1:172.21.15.188:6789/0 conn(0x7fc3b4161b00 legacy=0x7fc3b4162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b1600640 1 -- 172.21.15.19:0/1295628140 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b4072120 con 0x7fc3b40ad120 2024-03-28T13:16:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b1600640 1 -- 172.21.15.19:0/1295628140 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fc3a0001690 con 0x7fc3b40ad120 2024-03-28T13:16:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b1600640 1 -- 172.21.15.19:0/1295628140 >> v1:172.21.15.19:6789/0 conn(0x7fc3b40ad120 legacy=0x7fc3b40d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:42.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b1600640 1 --2- 172.21.15.19:0/1295628140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3980039b0 0x7fc398003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b1600640 1 --2- 172.21.15.19:0/1295628140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc398004800 0x7fc398004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b1600640 1 --2- 172.21.15.19:0/1295628140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3980051c0 0x7fc39800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b1600640 1 -- 172.21.15.19:0/1295628140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b4161ee0 con 0x7fc3980051c0 2024-03-28T13:16:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b1600640 1 -- 172.21.15.19:0/1295628140 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc39800c160 con 0x7fc398004800 2024-03-28T13:16:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.681+0000 7fc3b1600640 1 -- 172.21.15.19:0/1295628140 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc39800c360 con 0x7fc3980039b0 2024-03-28T13:16:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.682+0000 7fc3b3400640 1 --2- 172.21.15.19:0/1295628140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3980051c0 0x7fc39800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.682+0000 7fc3b2000640 1 --2- 172.21.15.19:0/1295628140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc398004800 0x7fc398004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.682+0000 7fc3b3400640 1 -- 172.21.15.19:0/1295628140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3980039b0 msgr2=0x7fc398003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.682+0000 7fc3b3400640 1 --2- 172.21.15.19:0/1295628140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3980039b0 0x7fc398003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.682+0000 7fc3b3400640 1 -- 172.21.15.19:0/1295628140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc398004800 msgr2=0x7fc398004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.682+0000 7fc3b3400640 1 --2- 172.21.15.19:0/1295628140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc398004800 0x7fc398004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:42.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.682+0000 7fc3b3400640 1 -- 172.21.15.19:0/1295628140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc398003520 con 0x7fc3980051c0 2024-03-28T13:16:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.683+0000 7fc3b3400640 1 -- 172.21.15.19:0/1295628140 reap_dead start 2024-03-28T13:16:42.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.683+0000 7fc3b3400640 1 --2- 172.21.15.19:0/1295628140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3980051c0 0x7fc39800b9c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009f90 tx=0x7fc398005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a50ab4ba2fa2716 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.684+0000 7fc3b1600640 1 -- 172.21.15.19:0/1295628140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac039070 con 0x7fc3980051c0 2024-03-28T13:16:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.684+0000 7fc3b1600640 1 -- 172.21.15.19:0/1295628140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3ac02cc20 con 0x7fc3980051c0 2024-03-28T13:16:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.684+0000 7fc3b1600640 1 -- 172.21.15.19:0/1295628140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac009c10 con 0x7fc3980051c0 2024-03-28T13:16:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.684+0000 7fc3b9800640 1 -- 172.21.15.19:0/1295628140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3980051c0 msgr2=0x7fc39800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.684+0000 7fc3b9800640 1 --2- 172.21.15.19:0/1295628140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3980051c0 0x7fc39800b9c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009f90 tx=0x7fc398005830 comp rx=0 tx=0).stop 2024-03-28T13:16:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.684+0000 7fc3b9800640 1 -- 172.21.15.19:0/1295628140 shutdown_connections 2024-03-28T13:16:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.684+0000 7fc3b9800640 1 --2- 172.21.15.19:0/1295628140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3980051c0 0x7fc39800b9c0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.684+0000 7fc3b9800640 1 -- 172.21.15.19:0/1295628140 >> 172.21.15.19:0/1295628140 conn(0x7fc3b406b990 msgr2=0x7fc3b406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.684+0000 7fc3b9800640 1 -- 172.21.15.19:0/1295628140 shutdown_connections 2024-03-28T13:16:42.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.685+0000 7fc3b9800640 1 -- 172.21.15.19:0/1295628140 wait complete. 2024-03-28T13:16:42.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.687+0000 7fc3b9800640 1 Processor -- start 2024-03-28T13:16:42.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.687+0000 7fc3b9800640 1 -- start start 2024-03-28T13:16:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b9800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3980051c0 0x7fc3b414da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b9800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3b409bf10 0x7fc3b409d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b9800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3b40c94c0 0x7fc3b415f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b9800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b40b3730 con 0x7fc3980051c0 2024-03-28T13:16:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b9800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b40b3450 con 0x7fc3b40c94c0 2024-03-28T13:16:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b9800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b40b5020 con 0x7fc3b409bf10 2024-03-28T13:16:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3980051c0 0x7fc3b414da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3980051c0 0x7fc3b414da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60986/0 (socket says 172.21.15.19:60986) 2024-03-28T13:16:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b2a00640 1 -- 172.21.15.19:0/744709609 learned_addr learned my addr 172.21.15.19:0/744709609 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b3400640 1 --2- 172.21.15.19:0/744709609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3b40c94c0 0x7fc3b415f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b2a00640 1 -- 172.21.15.19:0/744709609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3b409bf10 msgr2=0x7fc3b409d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:42.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b2000640 1 --2- 172.21.15.19:0/744709609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3b409bf10 0x7fc3b409d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:42.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b2a00640 1 --2- 172.21.15.19:0/744709609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3b409bf10 0x7fc3b409d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:42.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b2a00640 1 -- 172.21.15.19:0/744709609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3b40c94c0 msgr2=0x7fc3b415f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:42.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b2a00640 1 --2- 172.21.15.19:0/744709609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3b40c94c0 0x7fc3b415f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:42.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b2a00640 1 -- 172.21.15.19:0/744709609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b409c3f0 con 0x7fc3980051c0 2024-03-28T13:16:42.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.689+0000 7fc3b3400640 1 --2- 172.21.15.19:0/744709609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3b40c94c0 0x7fc3b415f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:42.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.690+0000 7fc3b2a00640 1 --2- 172.21.15.19:0/744709609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3980051c0 0x7fc3b414da30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400c400 tx=0x7fc3a400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:42.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.690+0000 7fc3aaa00640 1 -- 172.21.15.19:0/744709609 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4017070 con 0x7fc3980051c0 2024-03-28T13:16:42.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.690+0000 7fc3aaa00640 1 -- 172.21.15.19:0/744709609 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a4005310 con 0x7fc3980051c0 2024-03-28T13:16:42.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.690+0000 7fc3aaa00640 1 -- 172.21.15.19:0/744709609 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4003830 con 0x7fc3980051c0 2024-03-28T13:16:42.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.690+0000 7fc3b9800640 1 -- 172.21.15.19:0/744709609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b409c6d0 con 0x7fc3980051c0 2024-03-28T13:16:42.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.690+0000 7fc3b9800640 1 -- 172.21.15.19:0/744709609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b409cc90 con 0x7fc3980051c0 2024-03-28T13:16:42.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.691+0000 7fc3b9800640 1 -- 172.21.15.19:0/744709609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b40685c0 con 0x7fc3980051c0 2024-03-28T13:16:42.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.692+0000 7fc3aaa00640 1 -- 172.21.15.19:0/744709609 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc3a401a070 con 0x7fc3980051c0 2024-03-28T13:16:42.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.693+0000 7fc3aaa00640 1 --2- 172.21.15.19:0/744709609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc39406c560 0x7fc39406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:42.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.693+0000 7fc3aaa00640 1 -- 172.21.15.19:0/744709609 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fc3a408a970 con 0x7fc3980051c0 2024-03-28T13:16:42.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.694+0000 7fc3b2000640 1 --2- 172.21.15.19:0/744709609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc39406c560 0x7fc39406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:42.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.695+0000 7fc3b2000640 1 --2- 172.21.15.19:0/744709609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc39406c560 0x7fc39406ea20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0001eb0 tx=0x7fc3a0000ed0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:42.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.698+0000 7fc3aaa00640 1 -- 172.21.15.19:0/744709609 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc3a40588e0 con 0x7fc3980051c0 2024-03-28T13:16:42.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.943+0000 7fc3b9800640 1 -- 172.21.15.19:0/744709609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3b406f3a0 con 0x7fc3980051c0 2024-03-28T13:16:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.944+0000 7fc3aaa00640 1 -- 172.21.15.19:0/744709609 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5180 (secure 0 0 0) 0x7fc3a405cbe0 con 0x7fc3980051c0 2024-03-28T13:16:42.943 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:42.943 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:16:41.519088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6175":{"gid":6175,"name":"b","rank":0,"incarnation":64,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.188:6836/1749969053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1749969053},{"type":"v1","addr":"172.21.15.188:6836","nonce":1749969053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-03-28T13:16:42.943 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 66 2024-03-28T13:16:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.946+0000 7fc3b9800640 1 -- 172.21.15.19:0/744709609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc39406c560 msgr2=0x7fc39406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.946+0000 7fc3b9800640 1 --2- 172.21.15.19:0/744709609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc39406c560 0x7fc39406ea20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0001eb0 tx=0x7fc3a0000ed0 comp rx=0 tx=0).stop 2024-03-28T13:16:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.946+0000 7fc3b9800640 1 -- 172.21.15.19:0/744709609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3980051c0 msgr2=0x7fc3b414da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.946+0000 7fc3b9800640 1 --2- 172.21.15.19:0/744709609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3980051c0 0x7fc3b414da30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400c400 tx=0x7fc3a400c8c0 comp rx=0 tx=0).stop 2024-03-28T13:16:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.947+0000 7fc3b9800640 1 -- 172.21.15.19:0/744709609 shutdown_connections 2024-03-28T13:16:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.947+0000 7fc3b9800640 1 --2- 172.21.15.19:0/744709609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc39406c560 0x7fc39406ea20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.947+0000 7fc3b9800640 1 --2- 172.21.15.19:0/744709609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3b40c94c0 0x7fc3b415f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.947+0000 7fc3b9800640 1 --2- 172.21.15.19:0/744709609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3b409bf10 0x7fc3b409d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.947+0000 7fc3b9800640 1 --2- 172.21.15.19:0/744709609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3980051c0 0x7fc3b414da30 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.947+0000 7fc3b9800640 1 -- 172.21.15.19:0/744709609 >> 172.21.15.19:0/744709609 conn(0x7fc3b406b990 msgr2=0x7fc3b406bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.947+0000 7fc3b9800640 1 -- 172.21.15.19:0/744709609 shutdown_connections 2024-03-28T13:16:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:42.947+0000 7fc3b9800640 1 -- 172.21.15.19:0/744709609 wait complete. 2024-03-28T13:16:43.369 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:16:25.995386+0000', 'modified': '2024-03-28T13:16:41.519088+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6175}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6175': {'gid': 6175, 'name': 'b', 'rank': 0, 'incarnation': 64, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.188:6836/1749969053', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 1749969053}, {'type': 'v1', 'addr': '172.21.15.188:6836', 'nonce': 1749969053}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-28T13:16:43.369 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6175:up:rejoin 2024-03-28T13:16:44.371 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:16:44.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.605+0000 7f211fe00640 1 Processor -- start 2024-03-28T13:16:44.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.606+0000 7f211fe00640 1 -- start start 2024-03-28T13:16:44.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.607+0000 7f211fe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2118161b00 0x7f2118162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:44.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.607+0000 7f211fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21180d0d60 0x7f21180b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:44.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.607+0000 7f211fe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2118071b60 con 0x7f21180ad120 2024-03-28T13:16:44.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.607+0000 7f211fe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2118161ee0 con 0x7f21180d0d60 2024-03-28T13:16:44.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.607+0000 7f211fe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f21180b3450 con 0x7f2118161b00 2024-03-28T13:16:44.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.608+0000 7f211de00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21180d0d60 0x7f21180b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:44.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.608+0000 7f211de00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21180d0d60 0x7f21180b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60988/0 (socket says 172.21.15.19:60988) 2024-03-28T13:16:44.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.608+0000 7f211de00640 1 -- 172.21.15.19:0/1663383133 learned_addr learned my addr 172.21.15.19:0/1663383133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:44.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.608+0000 7f211ca00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f21180ad120 0x7f21180d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53680/0 (socket says 172.21.15.19:53680) 2024-03-28T13:16:44.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.608+0000 7f211d400640 1 --2- 172.21.15.19:0/1663383133 >> v2:172.21.15.188:3300/0 conn(0x7f2118161b00 0x7f2118162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:44.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.608+0000 7f211de00640 1 -- 172.21.15.19:0/1663383133 >> v2:172.21.15.188:3300/0 conn(0x7f2118161b00 msgr2=0x7f2118162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:44.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.608+0000 7f211de00640 1 --2- 172.21.15.19:0/1663383133 >> v2:172.21.15.188:3300/0 conn(0x7f2118161b00 0x7f2118162ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:44.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.608+0000 7f211de00640 1 -- 172.21.15.19:0/1663383133 >> v1:172.21.15.19:6789/0 conn(0x7f21180ad120 legacy=0x7f21180d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:44.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.608+0000 7f211de00640 1 -- 172.21.15.19:0/1663383133 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2118071e40 con 0x7f21180d0d60 2024-03-28T13:16:44.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.608+0000 7f2117e00640 1 -- 172.21.15.19:0/1663383133 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2127530022 0 0) 0x7f2118071b60 con 0x7f21180ad120 2024-03-28T13:16:44.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.610+0000 7f211de00640 1 --2- 172.21.15.19:0/1663383133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21180d0d60 0x7f21180b3bc0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f21180b63b0 tx=0x7f211002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314b65df54ed86f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:44.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.610+0000 7f2117e00640 1 -- 172.21.15.19:0/1663383133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211002cb20 con 0x7f21180d0d60 2024-03-28T13:16:44.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.610+0000 7f2117e00640 1 -- 172.21.15.19:0/1663383133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211002ccc0 con 0x7f21180d0d60 2024-03-28T13:16:44.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.610+0000 7f2117e00640 1 -- 172.21.15.19:0/1663383133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110034610 con 0x7f21180d0d60 2024-03-28T13:16:44.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.610+0000 7f211fe00640 1 -- 172.21.15.19:0/1663383133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21180d0d60 msgr2=0x7f21180b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:44.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.610+0000 7f211fe00640 1 --2- 172.21.15.19:0/1663383133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21180d0d60 0x7f21180b3bc0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f21180b63b0 tx=0x7f211002bf80 comp rx=0 tx=0).stop 2024-03-28T13:16:44.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.611+0000 7f211fe00640 1 -- 172.21.15.19:0/1663383133 shutdown_connections 2024-03-28T13:16:44.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.611+0000 7f211fe00640 1 --2- 172.21.15.19:0/1663383133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21180d0d60 0x7f21180b3bc0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:44.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.611+0000 7f211fe00640 1 --2- 172.21.15.19:0/1663383133 >> v2:172.21.15.188:3300/0 conn(0x7f2118161b00 0x7f2118162ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:44.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.611+0000 7f211fe00640 1 -- 172.21.15.19:0/1663383133 >> 172.21.15.19:0/1663383133 conn(0x7f211806b990 msgr2=0x7f211806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:44.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.611+0000 7f211fe00640 1 -- 172.21.15.19:0/1663383133 shutdown_connections 2024-03-28T13:16:44.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.611+0000 7f211fe00640 1 -- 172.21.15.19:0/1663383133 wait complete. 2024-03-28T13:16:44.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.612+0000 7f211fe00640 1 Processor -- start 2024-03-28T13:16:44.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.613+0000 7f211fe00640 1 -- start start 2024-03-28T13:16:44.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.613+0000 7f211fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21180ad120 0x7f21181f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:44.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211fe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21180d0d60 0x7f211814b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:44.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2118161b00 0x7f211815f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:44.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2118162ff0 con 0x7f2118161b00 2024-03-28T13:16:44.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211fe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f21180b3730 con 0x7f21180d0d60 2024-03-28T13:16:44.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f21180717f0 con 0x7f21180ad120 2024-03-28T13:16:44.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2118161b00 0x7f211815f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:44.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2118161b00 0x7f211815f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32770/0 (socket says 172.21.15.19:32770) 2024-03-28T13:16:44.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211de00640 1 -- 172.21.15.19:0/1311289765 learned_addr learned my addr 172.21.15.19:0/1311289765 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:44.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211ca00640 1 --2- 172.21.15.19:0/1311289765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21180d0d60 0x7f211814b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:44.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211d400640 1 --2- 172.21.15.19:0/1311289765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21180ad120 0x7f21181f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:44.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211de00640 1 -- 172.21.15.19:0/1311289765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21180ad120 msgr2=0x7f21181f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:44.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211de00640 1 --2- 172.21.15.19:0/1311289765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21180ad120 0x7f21181f5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:44.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211de00640 1 -- 172.21.15.19:0/1311289765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21180d0d60 msgr2=0x7f211814b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:44.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211de00640 1 --2- 172.21.15.19:0/1311289765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21180d0d60 0x7f211814b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:44.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211de00640 1 -- 172.21.15.19:0/1311289765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21181f46b0 con 0x7f2118161b00 2024-03-28T13:16:44.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.614+0000 7f211ca00640 1 --2- 172.21.15.19:0/1311289765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21180d0d60 0x7f211814b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:44.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.615+0000 7f211de00640 1 --2- 172.21.15.19:0/1311289765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2118161b00 0x7f211815f690 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f2110036040 tx=0x7f2110004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:44.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.615+0000 7f2115600640 1 -- 172.21.15.19:0/1311289765 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110043020 con 0x7f2118161b00 2024-03-28T13:16:44.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.615+0000 7f2115600640 1 -- 172.21.15.19:0/1311289765 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211003e450 con 0x7f2118161b00 2024-03-28T13:16:44.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.615+0000 7f2115600640 1 -- 172.21.15.19:0/1311289765 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110009510 con 0x7f2118161b00 2024-03-28T13:16:44.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.615+0000 7f211fe00640 1 -- 172.21.15.19:0/1311289765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f211815fbd0 con 0x7f2118161b00 2024-03-28T13:16:44.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.615+0000 7f211fe00640 1 -- 172.21.15.19:0/1311289765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f211809dff0 con 0x7f2118161b00 2024-03-28T13:16:44.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.616+0000 7f211fe00640 1 -- 172.21.15.19:0/1311289765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21180685c0 con 0x7f2118161b00 2024-03-28T13:16:44.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.617+0000 7f2115600640 1 -- 172.21.15.19:0/1311289765 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2110046020 con 0x7f2118161b00 2024-03-28T13:16:44.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.618+0000 7f2115600640 1 --2- 172.21.15.19:0/1311289765 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f20f806c530 0x7f20f806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:44.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.618+0000 7f2115600640 1 -- 172.21.15.19:0/1311289765 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f21100adf70 con 0x7f2118161b00 2024-03-28T13:16:44.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.619+0000 7f211d400640 1 --2- 172.21.15.19:0/1311289765 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f20f806c530 0x7f20f806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:44.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.620+0000 7f211d400640 1 --2- 172.21.15.19:0/1311289765 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f20f806c530 0x7f20f806e9f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f210c006fd0 tx=0x7f210c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:44.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.623+0000 7f2115600640 1 -- 172.21.15.19:0/1311289765 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f211004d030 con 0x7f2118161b00 2024-03-28T13:16:44.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.872+0000 7f211fe00640 1 -- 172.21.15.19:0/1311289765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f211806f3a0 con 0x7f2118161b00 2024-03-28T13:16:44.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.873+0000 7f2115600640 1 -- 172.21.15.19:0/1311289765 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5187 (secure 0 0 0) 0x7f211007c520 con 0x7f2118161b00 2024-03-28T13:16:44.872 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:16:44.872 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:16:43.516696+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6175":{"gid":6175,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6836/1749969053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1749969053},{"type":"v1","addr":"172.21.15.188:6836","nonce":1749969053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6175,"qdb_cluster":[6175]},"id":6}]} 2024-03-28T13:16:44.873 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 67 2024-03-28T13:16:44.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.876+0000 7f211fe00640 1 -- 172.21.15.19:0/1311289765 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f20f806c530 msgr2=0x7f20f806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:44.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.876+0000 7f211fe00640 1 --2- 172.21.15.19:0/1311289765 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f20f806c530 0x7f20f806e9f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f210c006fd0 tx=0x7f210c008040 comp rx=0 tx=0).stop 2024-03-28T13:16:44.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.876+0000 7f211fe00640 1 -- 172.21.15.19:0/1311289765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2118161b00 msgr2=0x7f211815f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:44.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.876+0000 7f211fe00640 1 --2- 172.21.15.19:0/1311289765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2118161b00 0x7f211815f690 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f2110036040 tx=0x7f2110004a40 comp rx=0 tx=0).stop 2024-03-28T13:16:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.876+0000 7f211fe00640 1 -- 172.21.15.19:0/1311289765 shutdown_connections 2024-03-28T13:16:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.876+0000 7f211fe00640 1 --2- 172.21.15.19:0/1311289765 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f20f806c530 0x7f20f806e9f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.876+0000 7f211fe00640 1 --2- 172.21.15.19:0/1311289765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2118161b00 0x7f211815f690 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.877+0000 7f211fe00640 1 --2- 172.21.15.19:0/1311289765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21180d0d60 0x7f211814b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.877+0000 7f211fe00640 1 --2- 172.21.15.19:0/1311289765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21180ad120 0x7f21181f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.877+0000 7f211fe00640 1 -- 172.21.15.19:0/1311289765 >> 172.21.15.19:0/1311289765 conn(0x7f211806b990 msgr2=0x7f21180af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.877+0000 7f211fe00640 1 -- 172.21.15.19:0/1311289765 shutdown_connections 2024-03-28T13:16:44.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:44.877+0000 7f211fe00640 1 -- 172.21.15.19:0/1311289765 wait complete. 2024-03-28T13:16:45.299 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:16:25.995386+0000', 'modified': '2024-03-28T13:16:43.516696+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6175}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6175': {'gid': 6175, 'name': 'b', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.188:6836/1749969053', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 1749969053}, {'type': 'v1', 'addr': '172.21.15.188:6836', 'nonce': 1749969053}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6175, 'qdb_cluster': [6175]} 2024-03-28T13:16:45.299 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T13:16:45.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-28T13:16:45.515 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-03-28T13:16:45.515+0000 7f809c200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:16:45.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.532+0000 7f51e1200640 1 Processor -- start 2024-03-28T13:16:45.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.533+0000 7f51e1200640 1 -- start start 2024-03-28T13:16:45.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.535+0000 7f51e1200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f51dc13f080 0x7f51dc1607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.536+0000 7f51e1200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51dc160cf0 con 0x7f51dc0702d0 2024-03-28T13:16:45.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.536+0000 7f51e1200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51dc15d2c0 con 0x7f51dc15c7b0 2024-03-28T13:16:45.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.536+0000 7f51e1200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f51dc15d5a0 con 0x7f51dc13f080 2024-03-28T13:16:45.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.536+0000 7f51dbe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f51dc0702d0 0x7f51dc0d0a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48466/0 (socket says 172.21.15.19:48466) 2024-03-28T13:16:45.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.536+0000 7f51dbe00640 1 -- 172.21.15.19:0/1858347066 learned_addr learned my addr 172.21.15.19:0/1858347066 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:45.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.536+0000 7f51db400640 1 --2- 172.21.15.19:0/1858347066 >> v2:172.21.15.188:3300/0 conn(0x7f51dc13f080 0x7f51dc1607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.536+0000 7f51da000640 1 -- 172.21.15.19:0/1858347066 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1191657641 0 0) 0x7f51dc160cf0 con 0x7f51dc0702d0 2024-03-28T13:16:45.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.537+0000 7f51da000640 1 -- 172.21.15.19:0/1858347066 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51c0003520 con 0x7f51dc0702d0 2024-03-28T13:16:45.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.537+0000 7f51da000640 1 -- 172.21.15.19:0/1858347066 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 736412429 0 0) 0x7f51c0003520 con 0x7f51dc0702d0 2024-03-28T13:16:45.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.537+0000 7f51da000640 1 -- 172.21.15.19:0/1858347066 >> v2:172.21.15.188:3300/0 conn(0x7f51dc13f080 msgr2=0x7f51dc1607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:45.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.537+0000 7f51da000640 1 --2- 172.21.15.19:0/1858347066 >> v2:172.21.15.188:3300/0 conn(0x7f51dc13f080 0x7f51dc1607b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.537+0000 7f51da000640 1 -- 172.21.15.19:0/1858347066 >> v1:172.21.15.188:6789/0 conn(0x7f51dc15c7b0 legacy=0x7f51dc06fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:45.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.537+0000 7f51da000640 1 -- 172.21.15.19:0/1858347066 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51dc15ffb0 con 0x7f51dc0702d0 2024-03-28T13:16:45.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.537+0000 7f51da000640 1 -- 172.21.15.19:0/1858347066 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f51d4001a10 con 0x7f51dc0702d0 2024-03-28T13:16:45.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.537+0000 7f51da000640 1 -- 172.21.15.19:0/1858347066 >> v1:172.21.15.19:6789/0 conn(0x7f51dc0702d0 legacy=0x7f51dc0d0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:45.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.537+0000 7f51da000640 1 --2- 172.21.15.19:0/1858347066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51c00039b0 0x7f51c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.537+0000 7f51da000640 1 --2- 172.21.15.19:0/1858347066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51c0004800 0x7f51c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.537+0000 7f51da000640 1 --2- 172.21.15.19:0/1858347066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51c00051c0 0x7f51c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.537+0000 7f51da000640 1 -- 172.21.15.19:0/1858347066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc15d5a0 con 0x7f51c0004800 2024-03-28T13:16:45.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.537+0000 7f51da000640 1 -- 172.21.15.19:0/1858347066 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51c000c060 con 0x7f51c00039b0 2024-03-28T13:16:45.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.537+0000 7f51da000640 1 -- 172.21.15.19:0/1858347066 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f51c000c260 con 0x7f51c00051c0 2024-03-28T13:16:45.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.537+0000 7f51daa00640 1 --2- 172.21.15.19:0/1858347066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51c0004800 0x7f51c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.538+0000 7f51daa00640 1 -- 172.21.15.19:0/1858347066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51c00051c0 msgr2=0x7f51c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:45.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.538+0000 7f51daa00640 1 --2- 172.21.15.19:0/1858347066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51c00051c0 0x7f51c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.538+0000 7f51daa00640 1 -- 172.21.15.19:0/1858347066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51c00039b0 msgr2=0x7f51c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:45.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.538+0000 7f51daa00640 1 --2- 172.21.15.19:0/1858347066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51c00039b0 0x7f51c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.538+0000 7f51daa00640 1 -- 172.21.15.19:0/1858347066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c0003520 con 0x7f51c0004800 2024-03-28T13:16:45.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.538+0000 7f51dbe00640 1 -- 172.21.15.19:0/1858347066 reap_dead start 2024-03-28T13:16:45.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.539+0000 7f51daa00640 1 --2- 172.21.15.19:0/1858347066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51c0004800 0x7f51c0004c80 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f51c8001750 tx=0x7f51c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebac0d5aaa94ef89 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:45.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.539+0000 7f51da000640 1 -- 172.21.15.19:0/1858347066 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c8001370 con 0x7f51c0004800 2024-03-28T13:16:45.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.539+0000 7f51da000640 1 -- 172.21.15.19:0/1858347066 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c80052d0 con 0x7f51c0004800 2024-03-28T13:16:45.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.539+0000 7f51da000640 1 -- 172.21.15.19:0/1858347066 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c8003830 con 0x7f51c0004800 2024-03-28T13:16:45.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.540+0000 7f51e1200640 1 -- 172.21.15.19:0/1858347066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51c0004800 msgr2=0x7f51c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:45.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.540+0000 7f51e1200640 1 --2- 172.21.15.19:0/1858347066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51c0004800 0x7f51c0004c80 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f51c8001750 tx=0x7f51c0005830 comp rx=0 tx=0).stop 2024-03-28T13:16:45.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.540+0000 7f51e1200640 1 -- 172.21.15.19:0/1858347066 shutdown_connections 2024-03-28T13:16:45.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.540+0000 7f51e1200640 1 --2- 172.21.15.19:0/1858347066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51c0004800 0x7f51c0004c80 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.540+0000 7f51e1200640 1 -- 172.21.15.19:0/1858347066 >> 172.21.15.19:0/1858347066 conn(0x7f51dc067460 msgr2=0x7f51dc067840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:45.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.540+0000 7f51e1200640 1 -- 172.21.15.19:0/1858347066 shutdown_connections 2024-03-28T13:16:45.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.540+0000 7f51e1200640 1 -- 172.21.15.19:0/1858347066 wait complete. 2024-03-28T13:16:45.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.542+0000 7f51e1200640 1 Processor -- start 2024-03-28T13:16:45.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.542+0000 7f51e1200640 1 -- start start 2024-03-28T13:16:45.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.543+0000 7f51e1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51c0004800 0x7f51dc152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.543+0000 7f51e1200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51dc158b30 0x7f51dc0b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.543+0000 7f51e1200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51dc158fb0 0x7f51dc0b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.543+0000 7f51e1200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc13f460 con 0x7f51c0004800 2024-03-28T13:16:45.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.543+0000 7f51e1200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc0b1940 con 0x7f51dc158fb0 2024-03-28T13:16:45.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.543+0000 7f51e1200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f51dc1607b0 con 0x7f51dc158b30 2024-03-28T13:16:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.543+0000 7f51db400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51c0004800 0x7f51dc152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.543+0000 7f51db400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51c0004800 0x7f51dc152db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52518/0 (socket says 172.21.15.19:52518) 2024-03-28T13:16:45.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.543+0000 7f51db400640 1 -- 172.21.15.19:0/2241788330 learned_addr learned my addr 172.21.15.19:0/2241788330 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:45.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.544+0000 7f51dbe00640 1 --2- 172.21.15.19:0/2241788330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51dc158fb0 0x7f51dc0b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.544+0000 7f51daa00640 1 --2- 172.21.15.19:0/2241788330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51dc158b30 0x7f51dc0b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.544+0000 7f51db400640 1 -- 172.21.15.19:0/2241788330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51dc158b30 msgr2=0x7f51dc0b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:45.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.544+0000 7f51db400640 1 --2- 172.21.15.19:0/2241788330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51dc158b30 0x7f51dc0b6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.544+0000 7f51db400640 1 -- 172.21.15.19:0/2241788330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51dc158fb0 msgr2=0x7f51dc0b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:45.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.544+0000 7f51db400640 1 --2- 172.21.15.19:0/2241788330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51dc158fb0 0x7f51dc0b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.544+0000 7f51db400640 1 -- 172.21.15.19:0/2241788330 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51dc157a90 con 0x7f51c0004800 2024-03-28T13:16:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.544+0000 7f51dbe00640 1 --2- 172.21.15.19:0/2241788330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51dc158fb0 0x7f51dc0b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.544+0000 7f51db400640 1 --2- 172.21.15.19:0/2241788330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51c0004800 0x7f51dc152db0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00e770 tx=0x7f51cc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.544+0000 7f51d3400640 1 -- 172.21.15.19:0/2241788330 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc017070 con 0x7f51c0004800 2024-03-28T13:16:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.544+0000 7f51d3400640 1 -- 172.21.15.19:0/2241788330 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51cc00c730 con 0x7f51c0004800 2024-03-28T13:16:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.544+0000 7f51e1200640 1 -- 172.21.15.19:0/2241788330 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51dc157d70 con 0x7f51c0004800 2024-03-28T13:16:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.544+0000 7f51d3400640 1 -- 172.21.15.19:0/2241788330 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc00cb20 con 0x7f51c0004800 2024-03-28T13:16:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.545+0000 7f51e1200640 1 -- 172.21.15.19:0/2241788330 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51dc158330 con 0x7f51c0004800 2024-03-28T13:16:45.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.547+0000 7f51d3400640 1 -- 172.21.15.19:0/2241788330 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f51cc01a070 con 0x7f51c0004800 2024-03-28T13:16:45.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.548+0000 7f51d3400640 1 --2- 172.21.15.19:0/2241788330 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51bc06c560 0x7f51bc06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.548+0000 7f51d3400640 1 -- 172.21.15.19:0/2241788330 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f51cc08b800 con 0x7f51c0004800 2024-03-28T13:16:45.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.549+0000 7f51daa00640 1 --2- 172.21.15.19:0/2241788330 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51bc06c560 0x7f51bc06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.549+0000 7f51e5602740 1 Processor -- start 2024-03-28T13:16:45.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.549+0000 7f51e5602740 1 -- start start 2024-03-28T13:16:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.550+0000 7f51daa00640 1 --2- 172.21.15.19:0/2241788330 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51bc06c560 0x7f51bc06ea20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f51c8033000 tx=0x7f51c8009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.550+0000 7f51e5602740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565401791d0 0x55654014c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.550+0000 7f51e5602740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401795b0 0x55654017bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.550+0000 7f51e5602740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55654014c6b0 0x55654017c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.550+0000 7f51e5602740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556540113920 con 0x5565401795b0 2024-03-28T13:16:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.550+0000 7f51e5602740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556540063fa0 con 0x5565401791d0 2024-03-28T13:16:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.550+0000 7f51e5602740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5565400b5f90 con 0x55654014c6b0 2024-03-28T13:16:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.551+0000 7f51daa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401795b0 0x55654017bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.551+0000 7f51daa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401795b0 0x55654017bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52534/0 (socket says 172.21.15.19:52534) 2024-03-28T13:16:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.551+0000 7f51daa00640 1 -- 172.21.15.19:0/3372721885 learned_addr learned my addr 172.21.15.19:0/3372721885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.551+0000 7f51db400640 1 --2- 172.21.15.19:0/3372721885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565401791d0 0x55654014c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.551+0000 7f51daa00640 1 -- 172.21.15.19:0/3372721885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55654014c6b0 msgr2=0x55654017c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.551+0000 7f51dbe00640 1 --2- 172.21.15.19:0/3372721885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55654014c6b0 0x55654017c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.551+0000 7f51daa00640 1 --2- 172.21.15.19:0/3372721885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55654014c6b0 0x55654017c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.551+0000 7f51daa00640 1 -- 172.21.15.19:0/3372721885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565401791d0 msgr2=0x55654014c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.551+0000 7f51daa00640 1 --2- 172.21.15.19:0/3372721885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565401791d0 0x55654014c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.551+0000 7f51daa00640 1 -- 172.21.15.19:0/3372721885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55653fdca2d0 con 0x5565401795b0 2024-03-28T13:16:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.551+0000 7f51db400640 1 --2- 172.21.15.19:0/3372721885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565401791d0 0x55654014c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.551+0000 7f51dbe00640 1 --2- 172.21.15.19:0/3372721885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55654014c6b0 0x55654017c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.552+0000 7f51daa00640 1 --2- 172.21.15.19:0/3372721885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401795b0 0x55654017bf60 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f51c8005510 tx=0x7f51c802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780af26fcd3b27c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.552+0000 7f51c7e00640 1 -- 172.21.15.19:0/3372721885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c80067d0 con 0x5565401795b0 2024-03-28T13:16:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.552+0000 7f51c7e00640 1 -- 172.21.15.19:0/3372721885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c804d070 con 0x5565401795b0 2024-03-28T13:16:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.552+0000 7f51c7e00640 1 -- 172.21.15.19:0/3372721885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c8006b70 con 0x5565401795b0 2024-03-28T13:16:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.553+0000 7f51e5602740 1 -- 172.21.15.19:0/3372721885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401795b0 msgr2=0x55654017bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.553+0000 7f51e5602740 1 --2- 172.21.15.19:0/3372721885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401795b0 0x55654017bf60 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f51c8005510 tx=0x7f51c802f000 comp rx=0 tx=0).stop 2024-03-28T13:16:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.553+0000 7f51e5602740 1 -- 172.21.15.19:0/3372721885 shutdown_connections 2024-03-28T13:16:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.553+0000 7f51e5602740 1 --2- 172.21.15.19:0/3372721885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55654014c6b0 0x55654017c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.553+0000 7f51e5602740 1 --2- 172.21.15.19:0/3372721885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401795b0 0x55654017bf60 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.553+0000 7f51e5602740 1 --2- 172.21.15.19:0/3372721885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565401791d0 0x55654014c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.553+0000 7f51e5602740 1 -- 172.21.15.19:0/3372721885 >> 172.21.15.19:0/3372721885 conn(0x55654014e3e0 msgr2=0x556540196f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.553+0000 7f51e5602740 1 -- 172.21.15.19:0/3372721885 shutdown_connections 2024-03-28T13:16:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.553+0000 7f51e5602740 1 -- 172.21.15.19:0/3372721885 wait complete. 2024-03-28T13:16:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.554+0000 7f51e5602740 1 -- start start 2024-03-28T13:16:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.554+0000 7f51e5602740 1 Processor -- start 2024-03-28T13:16:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.555+0000 7f51e5602740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401791d0 0x5565401819d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.555+0000 7f51e5602740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565401795b0 0x5565401ab810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.555+0000 7f51e5602740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556540178750 0x556540196b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.555+0000 7f51e5602740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5565400b1f40 con 0x5565401791d0 2024-03-28T13:16:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.555+0000 7f51e5602740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55654005e650 con 0x5565401795b0 2024-03-28T13:16:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.555+0000 7f51e5602740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x556540071a10 con 0x556540178750 2024-03-28T13:16:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.555+0000 7f51e5602740 10 client.0 _ll_register_callbacks cb 0x5565400b0ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:16:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.555+0000 7f51db400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401791d0 0x5565401819d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.555+0000 7f51db400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401791d0 0x5565401819d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52546/0 (socket says 172.21.15.19:52546) 2024-03-28T13:16:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.555+0000 7f51db400640 1 -- 172.21.15.19:0/120329687 learned_addr learned my addr 172.21.15.19:0/120329687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.555+0000 7f51dbe00640 1 --2- 172.21.15.19:0/120329687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556540178750 0x556540196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.555+0000 7f51daa00640 1 --2- 172.21.15.19:0/120329687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565401795b0 0x5565401ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.556+0000 7f51db400640 1 -- 172.21.15.19:0/120329687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556540178750 msgr2=0x556540196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.556+0000 7f51db400640 1 --2- 172.21.15.19:0/120329687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556540178750 0x556540196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.556+0000 7f51db400640 1 -- 172.21.15.19:0/120329687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565401795b0 msgr2=0x5565401ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.556+0000 7f51db400640 1 --2- 172.21.15.19:0/120329687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565401795b0 0x5565401ab810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.556+0000 7f51db400640 1 -- 172.21.15.19:0/120329687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55653fe946b0 con 0x5565401791d0 2024-03-28T13:16:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.556+0000 7f51daa00640 1 --2- 172.21.15.19:0/120329687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565401795b0 0x5565401ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.556+0000 7f51db400640 1 --2- 172.21.15.19:0/120329687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401791d0 0x5565401819d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f51cc089bd0 tx=0x7f51cc059620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c43bc3334e3d491 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.556+0000 7f51c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:16:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.556+0000 7f51c4c00640 1 -- 172.21.15.19:0/120329687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc021040 con 0x5565401791d0 2024-03-28T13:16:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.556+0000 7f51c4c00640 1 -- 172.21.15.19:0/120329687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f51cc05ed60 con 0x5565401791d0 2024-03-28T13:16:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.556+0000 7f51e5602740 1 -- 172.21.15.19:0/120329687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51c804d070 con 0x5565401791d0 2024-03-28T13:16:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.556+0000 7f51c4c00640 1 -- 172.21.15.19:0/120329687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f51cc062200 con 0x5565401791d0 2024-03-28T13:16:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.556+0000 7f51e5602740 10 client.7446 fetch_fsmap learned FSMap version 67 2024-03-28T13:16:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.556+0000 7f51e5602740 1 -- 172.21.15.19:0/120329687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x5565401814b0 con 0x5565401791d0 2024-03-28T13:16:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.557+0000 7f51c4c00640 1 -- 172.21.15.19:0/120329687 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c804d070 con 0x5565401791d0 2024-03-28T13:16:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.557+0000 7f51c4c00640 1 -- 172.21.15.19:0/120329687 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f51cc024060 con 0x5565401791d0 2024-03-28T13:16:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.557+0000 7f51e5602740 10 client.7446 fetch_fsmap finished waiting for FSMap version 67 2024-03-28T13:16:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.557+0000 7f51e5602740 10 client.7446 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T13:16:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.557+0000 7f51e5602740 20 client.7446 populate_metadata read hostname 'smithi019' 2024-03-28T13:16:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.557+0000 7f51e5602740 1 --2- 172.21.15.19:0/120329687 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55654017f9c0 0x5565401ba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.557+0000 7f51e5602740 4 client.7446 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:16:45.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.557+0000 7f51e5602740 1 -- 172.21.15.19:0/120329687 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565400c34d0 con 0x55654017f9c0 2024-03-28T13:16:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.558+0000 7f51daa00640 1 --2- 172.21.15.19:0/120329687 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55654017f9c0 0x5565401ba0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.559+0000 7f51daa00640 1 --2- 172.21.15.19:0/120329687 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55654017f9c0 0x5565401ba0f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6ba68a0356f2885 server_cookie=5b989e1193024bc2 in_seq=0 out_seq=0 2024-03-28T13:16:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.559+0000 7f51c4c00640 10 client.7446 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:16:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.561+0000 7f51c4c00640 1 -- 172.21.15.19:0/120329687 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5565400c34d0 con 0x55654017f9c0 2024-03-28T13:16:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.561+0000 7f51c4c00640 10 client.7446 handle_command_reply: tid=0 2024-03-28T13:16:45.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.561+0000 7f51e5602740 1 client.7446 shutdown 2024-03-28T13:16:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.561+0000 7f51e5602740 1 -- 172.21.15.19:0/120329687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401791d0 msgr2=0x5565401819d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.561+0000 7f51e5602740 1 --2- 172.21.15.19:0/120329687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401791d0 0x5565401819d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f51cc089bd0 tx=0x7f51cc059620 comp rx=0 tx=0).stop 2024-03-28T13:16:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.561+0000 7f51e5602740 1 -- 172.21.15.19:0/120329687 shutdown_connections 2024-03-28T13:16:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.561+0000 7f51e5602740 1 --2- 172.21.15.19:0/120329687 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55654017f9c0 0x5565401ba0f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.561+0000 7f51e5602740 1 --2- 172.21.15.19:0/120329687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556540178750 0x556540196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.561+0000 7f51e5602740 1 --2- 172.21.15.19:0/120329687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5565401795b0 0x5565401ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.561+0000 7f51e5602740 1 --2- 172.21.15.19:0/120329687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401791d0 0x5565401819d0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.561+0000 7f51c4c00640 0 client.7446 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:16:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.561+0000 7f51e5602740 1 -- 172.21.15.19:0/120329687 >> 172.21.15.19:0/120329687 conn(0x55654014c6b0 msgr2=0x55654014b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.562+0000 7f51e5602740 1 -- 172.21.15.19:0/120329687 shutdown_connections 2024-03-28T13:16:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.562+0000 7f51e5602740 1 -- 172.21.15.19:0/120329687 wait complete. 2024-03-28T13:16:45.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.562+0000 7f51e5602740 20 client.7446 trim_cache size 0 max 16384 2024-03-28T13:16:45.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.582+0000 7f51e5602740 1 Processor -- start 2024-03-28T13:16:45.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.582+0000 7f51e5602740 1 -- start start 2024-03-28T13:16:45.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.583+0000 7f51e5602740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556540178750 0x556540196510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.583+0000 7f51e5602740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401791d0 0x5565401bf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.583+0000 7f51e5602740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5565401795b0 0x5565401906c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.583+0000 7f51e5602740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc15e260 con 0x5565401791d0 2024-03-28T13:16:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.583+0000 7f51e5602740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f51dc146380 con 0x556540178750 2024-03-28T13:16:45.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.583+0000 7f51e5602740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f51dc15dc10 con 0x5565401795b0 2024-03-28T13:16:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.583+0000 7f51daa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401791d0 0x5565401bf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.583+0000 7f51daa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401791d0 0x5565401bf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52562/0 (socket says 172.21.15.19:52562) 2024-03-28T13:16:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.583+0000 7f51daa00640 1 -- 172.21.15.19:0/2870730351 learned_addr learned my addr 172.21.15.19:0/2870730351 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:45.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.583+0000 7f51db400640 1 --2- 172.21.15.19:0/2870730351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556540178750 0x556540196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.583+0000 7f51dbe00640 1 --2- 172.21.15.19:0/2870730351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5565401795b0 0x5565401906c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.584+0000 7f51daa00640 1 -- 172.21.15.19:0/2870730351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5565401795b0 msgr2=0x5565401906c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.584+0000 7f51daa00640 1 --2- 172.21.15.19:0/2870730351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5565401795b0 0x5565401906c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.584+0000 7f51daa00640 1 -- 172.21.15.19:0/2870730351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556540178750 msgr2=0x556540196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.584+0000 7f51daa00640 1 --2- 172.21.15.19:0/2870730351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556540178750 0x556540196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.584+0000 7f51daa00640 1 -- 172.21.15.19:0/2870730351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565401836f0 con 0x5565401791d0 2024-03-28T13:16:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.584+0000 7f51daa00640 1 --2- 172.21.15.19:0/2870730351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401791d0 0x5565401bf890 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f51c8001720 tx=0x7f51c803d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989b657321d7d5f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.584+0000 7f51c7400640 1 -- 172.21.15.19:0/2870730351 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c8061070 con 0x5565401791d0 2024-03-28T13:16:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.584+0000 7f51c7400640 1 -- 172.21.15.19:0/2870730351 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c804d030 con 0x5565401791d0 2024-03-28T13:16:45.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.584+0000 7f51c7400640 1 -- 172.21.15.19:0/2870730351 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c805b550 con 0x5565401791d0 2024-03-28T13:16:45.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.585+0000 7f51e5602740 1 -- 172.21.15.19:0/2870730351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401791d0 msgr2=0x5565401bf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:45.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.585+0000 7f51e5602740 1 --2- 172.21.15.19:0/2870730351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401791d0 0x5565401bf890 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f51c8001720 tx=0x7f51c803d000 comp rx=0 tx=0).stop 2024-03-28T13:16:45.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.585+0000 7f51e5602740 1 -- 172.21.15.19:0/2870730351 shutdown_connections 2024-03-28T13:16:45.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.585+0000 7f51e5602740 1 --2- 172.21.15.19:0/2870730351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5565401795b0 0x5565401906c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.585+0000 7f51e5602740 1 --2- 172.21.15.19:0/2870730351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565401791d0 0x5565401bf890 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.585+0000 7f51e5602740 1 --2- 172.21.15.19:0/2870730351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556540178750 0x556540196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.585+0000 7f51e5602740 1 -- 172.21.15.19:0/2870730351 >> 172.21.15.19:0/2870730351 conn(0x55654014c6b0 msgr2=0x556540199b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.585+0000 7f51e5602740 1 -- 172.21.15.19:0/2870730351 shutdown_connections 2024-03-28T13:16:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.585+0000 7f51e5602740 1 -- 172.21.15.19:0/2870730351 wait complete. 2024-03-28T13:16:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.586+0000 7f51e5602740 1 -- start start 2024-03-28T13:16:45.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.586+0000 7f51e5602740 1 Processor -- start 2024-03-28T13:16:45.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.587+0000 7f51e5602740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556540178750 0x5565401720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.587+0000 7f51e5602740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5565401795b0 0x556540172610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.587+0000 7f51e5602740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556540182060 0x5565401741a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.587+0000 7f51e5602740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5565401970b0 con 0x556540178750 2024-03-28T13:16:45.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.587+0000 7f51e5602740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5565401ac180 con 0x556540182060 2024-03-28T13:16:45.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.587+0000 7f51e5602740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x556540199240 con 0x5565401795b0 2024-03-28T13:16:45.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.587+0000 7f51e5602740 10 client.0 _ll_register_callbacks cb 0x7f51cc0558c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:16:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.587+0000 7f51db400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556540178750 0x5565401720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.587+0000 7f51db400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556540178750 0x5565401720d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52574/0 (socket says 172.21.15.19:52574) 2024-03-28T13:16:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.587+0000 7f51db400640 1 -- 172.21.15.19:0/2772842020 learned_addr learned my addr 172.21.15.19:0/2772842020 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.588+0000 7f51daa00640 1 --2- 172.21.15.19:0/2772842020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5565401795b0 0x556540172610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.588+0000 7f51db400640 1 -- 172.21.15.19:0/2772842020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5565401795b0 msgr2=0x556540172610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.588+0000 7f51db400640 1 --2- 172.21.15.19:0/2772842020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5565401795b0 0x556540172610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.588+0000 7f51db400640 1 -- 172.21.15.19:0/2772842020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556540182060 msgr2=0x5565401741a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.588+0000 7f51db400640 1 --2- 172.21.15.19:0/2772842020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556540182060 0x5565401741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.588+0000 7f51db400640 1 -- 172.21.15.19:0/2772842020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55654017bb40 con 0x556540178750 2024-03-28T13:16:45.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.588+0000 7f51db400640 1 --2- 172.21.15.19:0/2772842020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556540178750 0x5565401720d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f51cc061cd0 tx=0x7f51cc089b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92112148fd31639 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.588+0000 7f51c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:16:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.588+0000 7f51c4c00640 1 -- 172.21.15.19:0/2772842020 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc021040 con 0x556540178750 2024-03-28T13:16:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.588+0000 7f51c4c00640 1 -- 172.21.15.19:0/2772842020 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f51cc0584e0 con 0x556540178750 2024-03-28T13:16:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.588+0000 7f51e5602740 1 -- 172.21.15.19:0/2772842020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51c804d030 con 0x556540178750 2024-03-28T13:16:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.589+0000 7f51c4c00640 1 -- 172.21.15.19:0/2772842020 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f51cc081050 con 0x556540178750 2024-03-28T13:16:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.589+0000 7f51e5602740 10 client.7452 fetch_fsmap learned FSMap version 67 2024-03-28T13:16:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.589+0000 7f51e5602740 1 -- 172.21.15.19:0/2772842020 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x5565401ab500 con 0x556540178750 2024-03-28T13:16:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.589+0000 7f51c4c00640 1 -- 172.21.15.19:0/2772842020 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c804d030 con 0x556540178750 2024-03-28T13:16:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.589+0000 7f51c4c00640 1 -- 172.21.15.19:0/2772842020 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f51cc058700 con 0x556540178750 2024-03-28T13:16:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.589+0000 7f51e5602740 10 client.7452 fetch_fsmap finished waiting for FSMap version 67 2024-03-28T13:16:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.589+0000 7f51e5602740 10 client.7452 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T13:16:45.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.589+0000 7f51e5602740 20 client.7452 populate_metadata read hostname 'smithi019' 2024-03-28T13:16:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.589+0000 7f51e5602740 1 --2- 172.21.15.19:0/2772842020 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55654014b000 0x55654014b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.589+0000 7f51e5602740 4 client.7452 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T13:16:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.589+0000 7f51e5602740 1 -- 172.21.15.19:0/2772842020 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "status"}) -- 0x5565400bd820 con 0x55654014b000 2024-03-28T13:16:45.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.590+0000 7f51daa00640 1 --2- 172.21.15.19:0/2772842020 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55654014b000 0x55654014b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.591+0000 7f51daa00640 1 --2- 172.21.15.19:0/2772842020 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55654014b000 0x55654014b3e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d866b9a71ebdc28 server_cookie=9f2139dc0038bc5d in_seq=0 out_seq=0 2024-03-28T13:16:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.591+0000 7f51c4c00640 10 client.7452 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:16:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.591+0000 7f51c4c00640 1 -- 172.21.15.19:0/2772842020 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5565400bd820 con 0x55654014b000 2024-03-28T13:16:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.591+0000 7f51c4c00640 10 client.7452 handle_command_reply: tid=0 2024-03-28T13:16:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.592+0000 7f51e5602740 1 client.7452 shutdown 2024-03-28T13:16:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.592+0000 7f51e5602740 1 -- 172.21.15.19:0/2772842020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556540178750 msgr2=0x5565401720d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:45.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.592+0000 7f51e5602740 1 --2- 172.21.15.19:0/2772842020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556540178750 0x5565401720d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f51cc061cd0 tx=0x7f51cc089b00 comp rx=0 tx=0).stop 2024-03-28T13:16:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.592+0000 7f51e5602740 1 -- 172.21.15.19:0/2772842020 shutdown_connections 2024-03-28T13:16:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.592+0000 7f51e5602740 1 --2- 172.21.15.19:0/2772842020 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55654014b000 0x55654014b3e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.592+0000 7f51e5602740 1 --2- 172.21.15.19:0/2772842020 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556540182060 0x5565401741a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.592+0000 7f51e5602740 1 --2- 172.21.15.19:0/2772842020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5565401795b0 0x556540172610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.592+0000 7f51e5602740 1 --2- 172.21.15.19:0/2772842020 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556540178750 0x5565401720d0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.592+0000 7f51c4c00640 0 client.7452 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:16:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.592+0000 7f51e5602740 1 -- 172.21.15.19:0/2772842020 >> 172.21.15.19:0/2772842020 conn(0x55654014c6b0 msgr2=0x55654019a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.592+0000 7f51e5602740 1 -- 172.21.15.19:0/2772842020 shutdown_connections 2024-03-28T13:16:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.592+0000 7f51e5602740 1 -- 172.21.15.19:0/2772842020 wait complete. 2024-03-28T13:16:45.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.592+0000 7f51e5602740 20 client.7452 trim_cache size 0 max 16384 2024-03-28T13:16:45.595 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:16:45.595 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "e6b569d9-f2ce-49dc-bbe7-c0ab33080203", 2024-03-28T13:16:45.595 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-03-28T13:16:45.595 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 6175, 2024-03-28T13:16:45.595 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-03-28T13:16:45.595 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-03-28T13:16:45.595 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-03-28T13:16:45.595 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 5.0777414869999999, 2024-03-28T13:16:45.595 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 67, 2024-03-28T13:16:45.595 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 93, 2024-03-28T13:16:45.595 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 93, 2024-03-28T13:16:45.595 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 5.9759214099999998 2024-03-28T13:16:45.596 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:16:45.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.593+0000 7f51e1200640 1 -- 172.21.15.19:0/2241788330 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51bc06c560 msgr2=0x7f51bc06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:45.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.593+0000 7f51e1200640 1 --2- 172.21.15.19:0/2241788330 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51bc06c560 0x7f51bc06ea20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f51c8033000 tx=0x7f51c8009030 comp rx=0 tx=0).stop 2024-03-28T13:16:45.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.593+0000 7f51e1200640 1 -- 172.21.15.19:0/2241788330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51c0004800 msgr2=0x7f51dc152db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:45.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.593+0000 7f51e1200640 1 --2- 172.21.15.19:0/2241788330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51c0004800 0x7f51dc152db0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00e770 tx=0x7f51cc00ec30 comp rx=0 tx=0).stop 2024-03-28T13:16:45.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.593+0000 7f51e1200640 1 -- 172.21.15.19:0/2241788330 shutdown_connections 2024-03-28T13:16:45.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.593+0000 7f51e1200640 1 --2- 172.21.15.19:0/2241788330 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51bc06c560 0x7f51bc06ea20 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.593+0000 7f51e1200640 1 --2- 172.21.15.19:0/2241788330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51dc158fb0 0x7f51dc0b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.593+0000 7f51e1200640 1 --2- 172.21.15.19:0/2241788330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51dc158b30 0x7f51dc0b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.593+0000 7f51e1200640 1 --2- 172.21.15.19:0/2241788330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51c0004800 0x7f51dc152db0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.593+0000 7f51e1200640 1 -- 172.21.15.19:0/2241788330 >> 172.21.15.19:0/2241788330 conn(0x7f51dc067460 msgr2=0x7f51dc0d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:45.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.593+0000 7f51e1200640 1 -- 172.21.15.19:0/2241788330 shutdown_connections 2024-03-28T13:16:45.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.593+0000 7f51e1200640 1 -- 172.21.15.19:0/2241788330 wait complete. 2024-03-28T13:16:45.612 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:16:45.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.831+0000 7fbb16400640 1 Processor -- start 2024-03-28T13:16:45.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.831+0000 7fbb16400640 1 -- start start 2024-03-28T13:16:45.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.832+0000 7fbb16400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbb100d1bf0 0x7fbb1015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.832+0000 7fbb16400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb100705f0 0x7fbb1006e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.832+0000 7fbb16400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb1015edc0 con 0x7fbb10070170 2024-03-28T13:16:45.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.832+0000 7fbb16400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fbb100d1fd0 con 0x7fbb100d1bf0 2024-03-28T13:16:45.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.832+0000 7fbb16400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbb1015eb80 con 0x7fbb100705f0 2024-03-28T13:16:45.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.833+0000 7fbb0ea00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbb10070170 0x7fbb1006b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48480/0 (socket says 172.21.15.19:48480) 2024-03-28T13:16:45.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.833+0000 7fbb0ea00640 1 -- 172.21.15.19:0/3243983524 learned_addr learned my addr 172.21.15.19:0/3243983524 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:45.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.833+0000 7fbb0f400640 1 --2- 172.21.15.19:0/3243983524 >> v2:172.21.15.188:3300/0 conn(0x7fbb100d1bf0 0x7fbb1015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.833+0000 7fbb0fe00640 1 --2- 172.21.15.19:0/3243983524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb100705f0 0x7fbb1006e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.833+0000 7fbb0e000640 1 -- 172.21.15.19:0/3243983524 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1072767083 0 0) 0x7fbb1015edc0 con 0x7fbb10070170 2024-03-28T13:16:45.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.833+0000 7fbb0e000640 1 -- 172.21.15.19:0/3243983524 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbaf4003520 con 0x7fbb10070170 2024-03-28T13:16:45.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.834+0000 7fbb0e000640 1 -- 172.21.15.19:0/3243983524 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1199860324 0 0) 0x7fbaf4003520 con 0x7fbb10070170 2024-03-28T13:16:45.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.834+0000 7fbb0e000640 1 -- 172.21.15.19:0/3243983524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb100705f0 msgr2=0x7fbb1006e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:45.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.834+0000 7fbb0e000640 1 --2- 172.21.15.19:0/3243983524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb100705f0 0x7fbb1006e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.834+0000 7fbb0e000640 1 -- 172.21.15.19:0/3243983524 >> v2:172.21.15.188:3300/0 conn(0x7fbb100d1bf0 msgr2=0x7fbb1015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:45.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.834+0000 7fbb0e000640 1 --2- 172.21.15.19:0/3243983524 >> v2:172.21.15.188:3300/0 conn(0x7fbb100d1bf0 0x7fbb1015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.834+0000 7fbb0e000640 1 -- 172.21.15.19:0/3243983524 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb100b5e60 con 0x7fbb10070170 2024-03-28T13:16:45.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.834+0000 7fbb0e000640 1 -- 172.21.15.19:0/3243983524 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fbafc001670 con 0x7fbb10070170 2024-03-28T13:16:45.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.834+0000 7fbb0e000640 1 -- 172.21.15.19:0/3243983524 >> v1:172.21.15.19:6789/0 conn(0x7fbb10070170 legacy=0x7fbb1006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:45.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.834+0000 7fbb0e000640 1 --2- 172.21.15.19:0/3243983524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb100705f0 0x7fbaf40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.834+0000 7fbb0e000640 1 --2- 172.21.15.19:0/3243983524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaf4003f30 0x7fbaf4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.834+0000 7fbb0e000640 1 --2- 172.21.15.19:0/3243983524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf4004d40 0x7fbaf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.834+0000 7fbb0e000640 1 -- 172.21.15.19:0/3243983524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1015eb80 con 0x7fbaf4003f30 2024-03-28T13:16:45.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.834+0000 7fbb0e000640 1 -- 172.21.15.19:0/3243983524 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf400b9c0 con 0x7fbaf4004d40 2024-03-28T13:16:45.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.834+0000 7fbb0e000640 1 -- 172.21.15.19:0/3243983524 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf400bb40 con 0x7fbb100705f0 2024-03-28T13:16:45.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.834+0000 7fbb0ea00640 1 --2- 172.21.15.19:0/3243983524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaf4003f30 0x7fbaf4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.835+0000 7fbb0fe00640 1 --2- 172.21.15.19:0/3243983524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf4004d40 0x7fbaf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.835+0000 7fbb0f400640 1 --2- 172.21.15.19:0/3243983524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb100705f0 0x7fbaf40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.835+0000 7fbb0ea00640 1 -- 172.21.15.19:0/3243983524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb100705f0 msgr2=0x7fbaf40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:45.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.835+0000 7fbb0ea00640 1 --2- 172.21.15.19:0/3243983524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb100705f0 0x7fbaf40039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.835+0000 7fbb0ea00640 1 -- 172.21.15.19:0/3243983524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf4004d40 msgr2=0x7fbaf40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:45.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.835+0000 7fbb0ea00640 1 --2- 172.21.15.19:0/3243983524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf4004d40 0x7fbaf40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.835+0000 7fbb0ea00640 1 -- 172.21.15.19:0/3243983524 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf4003520 con 0x7fbaf4003f30 2024-03-28T13:16:45.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.835+0000 7fbb0fe00640 1 --2- 172.21.15.19:0/3243983524 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf4004d40 0x7fbaf40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:45.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.836+0000 7fbb0ea00640 1 --2- 172.21.15.19:0/3243983524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaf4003f30 0x7fbaf4004800 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c210 tx=0x7fbaf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398ee68443452e24 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:45.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.836+0000 7fbb0e000640 1 -- 172.21.15.19:0/3243983524 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc007d50 con 0x7fbaf4003f30 2024-03-28T13:16:45.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.836+0000 7fbb0e000640 1 -- 172.21.15.19:0/3243983524 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc030070 con 0x7fbaf4003f30 2024-03-28T13:16:45.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.836+0000 7fbb0e000640 1 -- 172.21.15.19:0/3243983524 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc02cdb0 con 0x7fbaf4003f30 2024-03-28T13:16:45.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.837+0000 7fbb16400640 1 -- 172.21.15.19:0/3243983524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaf4003f30 msgr2=0x7fbaf4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:45.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.837+0000 7fbb16400640 1 --2- 172.21.15.19:0/3243983524 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaf4003f30 0x7fbaf4004800 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c210 tx=0x7fbaf4005830 comp rx=0 tx=0).stop 2024-03-28T13:16:45.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.837+0000 7fbb0fe00640 1 -- 172.21.15.19:0/3243983524 reap_dead start 2024-03-28T13:16:45.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.837+0000 7fbb16400640 1 -- 172.21.15.19:0/3243983524 shutdown_connections 2024-03-28T13:16:45.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.837+0000 7fbb16400640 1 -- 172.21.15.19:0/3243983524 >> 172.21.15.19:0/3243983524 conn(0x7fbb100b6560 msgr2=0x7fbb100b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:45.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.837+0000 7fbb16400640 1 -- 172.21.15.19:0/3243983524 shutdown_connections 2024-03-28T13:16:45.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.837+0000 7fbb16400640 1 -- 172.21.15.19:0/3243983524 wait complete. 2024-03-28T13:16:45.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.839+0000 7fbb16400640 1 Processor -- start 2024-03-28T13:16:45.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.839+0000 7fbb16400640 1 -- start start 2024-03-28T13:16:45.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.840+0000 7fbb16400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf4003f30 0x7fbb1009dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.840+0000 7fbb16400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb1014e5c0 0x7fbb101e88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.840+0000 7fbb16400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb100afcf0 0x7fbb100b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.840+0000 7fbb16400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1015fae0 con 0x7fbb1014e5c0 2024-03-28T13:16:45.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.840+0000 7fbb16400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1015f080 con 0x7fbaf4003f30 2024-03-28T13:16:45.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.840+0000 7fbb16400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbb100b5b10 con 0x7fbb100afcf0 2024-03-28T13:16:45.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.840+0000 7fbb0ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb1014e5c0 0x7fbb101e88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.840+0000 7fbb0ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb1014e5c0 0x7fbb101e88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52600/0 (socket says 172.21.15.19:52600) 2024-03-28T13:16:45.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.840+0000 7fbb0ea00640 1 -- 172.21.15.19:0/3822811753 learned_addr learned my addr 172.21.15.19:0/3822811753 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:45.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.841+0000 7fbb0fe00640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb100afcf0 0x7fbb100b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.841+0000 7fbb0f400640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf4003f30 0x7fbb1009dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.841+0000 7fbb0ea00640 1 -- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb100afcf0 msgr2=0x7fbb100b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:45.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.841+0000 7fbb0ea00640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb100afcf0 0x7fbb100b0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.841+0000 7fbb0ea00640 1 -- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf4003f30 msgr2=0x7fbb1009dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:45.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.841+0000 7fbb0ea00640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf4003f30 0x7fbb1009dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:45.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.841+0000 7fbb0ea00640 1 -- 172.21.15.19:0/3822811753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1009e610 con 0x7fbb1014e5c0 2024-03-28T13:16:45.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.841+0000 7fbb0f400640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf4003f30 0x7fbb1009dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:45.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.841+0000 7fbb0fe00640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb100afcf0 0x7fbb100b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:45.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.841+0000 7fbb0ea00640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb1014e5c0 0x7fbb101e88e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c210 tx=0x7fbafc001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:45.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.841+0000 7fbb07400640 1 -- 172.21.15.19:0/3822811753 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc001c90 con 0x7fbb1014e5c0 2024-03-28T13:16:45.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.841+0000 7fbb07400640 1 -- 172.21.15.19:0/3822811753 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbafc004bf0 con 0x7fbb1014e5c0 2024-03-28T13:16:45.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.841+0000 7fbb07400640 1 -- 172.21.15.19:0/3822811753 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbafc007d50 con 0x7fbb1014e5c0 2024-03-28T13:16:45.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.842+0000 7fbb16400640 1 -- 172.21.15.19:0/3822811753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1009e8f0 con 0x7fbb1014e5c0 2024-03-28T13:16:45.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.842+0000 7fbb16400640 1 -- 172.21.15.19:0/3822811753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1009eeb0 con 0x7fbb1014e5c0 2024-03-28T13:16:45.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.843+0000 7fbb16400640 1 -- 172.21.15.19:0/3822811753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb100685c0 con 0x7fbb1014e5c0 2024-03-28T13:16:45.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.844+0000 7fbb07400640 1 -- 172.21.15.19:0/3822811753 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbafc007070 con 0x7fbb1014e5c0 2024-03-28T13:16:45.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.846+0000 7fbb07400640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbaf006c530 0x7fbaf006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:45.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.846+0000 7fbb07400640 1 -- 172.21.15.19:0/3822811753 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fbafc0ac990 con 0x7fbb1014e5c0 2024-03-28T13:16:45.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.846+0000 7fbb0f400640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbaf006c530 0x7fbaf006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:45.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.847+0000 7fbb0f400640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbaf006c530 0x7fbaf006e9f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbb00007460 tx=0x7fbb0000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:45.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:45.852+0000 7fbb07400640 1 -- 172.21.15.19:0/3822811753 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbafc07af40 con 0x7fbb1014e5c0 2024-03-28T13:16:46.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46.095+0000 7fbb16400640 1 -- 172.21.15.19:0/3822811753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbb100afad0 con 0x7fbb1014e5c0 2024-03-28T13:16:46.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46.096+0000 7fbb07400640 1 -- 172.21.15.19:0/3822811753 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7fbafc07a840 con 0x7fbb1014e5c0 2024-03-28T13:16:46.095 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:16:46.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46.099+0000 7fbb16400640 1 -- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbaf006c530 msgr2=0x7fbaf006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:46.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46.099+0000 7fbb16400640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbaf006c530 0x7fbaf006e9f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbb00007460 tx=0x7fbb0000b040 comp rx=0 tx=0).stop 2024-03-28T13:16:46.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46.099+0000 7fbb16400640 1 -- 172.21.15.19:0/3822811753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb1014e5c0 msgr2=0x7fbb101e88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:46.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46.099+0000 7fbb16400640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb1014e5c0 0x7fbb101e88e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00c210 tx=0x7fbafc001a40 comp rx=0 tx=0).stop 2024-03-28T13:16:46.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46.099+0000 7fbb16400640 1 -- 172.21.15.19:0/3822811753 shutdown_connections 2024-03-28T13:16:46.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46.099+0000 7fbb16400640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbaf006c530 0x7fbaf006e9f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46.099+0000 7fbb16400640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb100afcf0 0x7fbb100b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46.099+0000 7fbb16400640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbb1014e5c0 0x7fbb101e88e0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46.099+0000 7fbb16400640 1 --2- 172.21.15.19:0/3822811753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbaf4003f30 0x7fbb1009dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46.099+0000 7fbb16400640 1 -- 172.21.15.19:0/3822811753 >> 172.21.15.19:0/3822811753 conn(0x7fbb100b6560 msgr2=0x7fbb100b2cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46.099+0000 7fbb16400640 1 -- 172.21.15.19:0/3822811753 shutdown_connections 2024-03-28T13:16:46.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46.099+0000 7fbb16400640 1 -- 172.21.15.19:0/3822811753 wait complete. 2024-03-28T13:16:46.117 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-28T13:16:46.118 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:16:46.118 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:16:46.118 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:16:46.118 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:16:46.118 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-28T13:16:46.118 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-28T13:16:46.118 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:16:46.118 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:16:46.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:46.119 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:16:46.171 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:16:46.171 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:16:46.171 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:16:46.172 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:46.172 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:16:46.172 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:46.172 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:16:46.172 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:16:46.172 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:16:46.172 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:16:46.172 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:16:46.172 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:16:46.172 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:16:46.172 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:16:46.172 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:16:46.172 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:16:46.172 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42148sec preferred_lft 42148sec 2024-03-28T13:16:46.172 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:16:46.173 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:46.173 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:16:46.173 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:16:46.173 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:16:46.173 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:16:46.173 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:16:46.173 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:16:46.173 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:46.173 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:16:46.173 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:46.173 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:16:46.173 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:16:46.173 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:16:46.173 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:46.174 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:16:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether e6:17:c5:8e:2b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:16:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::e417:c5ff:fe8e:2bcf/64 scope link 2024-03-28T13:16:46.174 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:16:46.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:46.175 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:16:46.263 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:16:46.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:46.268 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:16:46.323 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:16:46.323 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:16:46.323 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:16:46.323 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:16:46.323 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:16:46.324 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:16:46.325 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:16:46.325 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:16:46.325 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:16:46.325 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:16:46.325 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:16:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:16:46.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:16:46.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:46.584 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:16:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:16:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:16:46.634 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:16:46.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:46.634 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:16:46.733 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:16:46.733 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:16:46.790 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:16:46.790 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:16:46.790 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:16:46.844 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:16:46.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:46.844 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:16:46.985 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:16:46.985 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:16:46.985 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:16:47.008 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:16:47.008 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:16:47.008 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:16:47.008 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:16:47.009 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:16:47.036 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:16:47.036 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:16:47.036 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:16:47.036 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:16:47.036 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:16:47.036 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:16:47.036 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:16:47.036 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:16:47.039 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:16:47.039 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:16:47.039 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:16:47.040 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:16:47.040 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:16:47.045 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:16:47.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:47.045 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:16:47.132 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:16:47.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:16:47.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:16:47.263 INFO:teuthology.orchestra.run.smithi019.stdout:client7461 2024-03-28T13:16:47.263 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:16:47.263 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:16:47.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:47.320 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-03-28T13:16:47.380 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-03-28T13:16:47.380+0000 7f8097200640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-03-28T13:16:47.382 INFO:teuthology.orchestra.run.smithi019.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-03-28T13:16:47.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T13:16:47.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:47.384 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-03-28T13:16:47.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.617+0000 7f93f6600640 1 Processor -- start 2024-03-28T13:16:47.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.617+0000 7f93f6600640 1 -- start start 2024-03-28T13:16:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.618+0000 7f93f6600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f0144980 0x7f93f015ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.618+0000 7f93f6600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f93f00b35b0 0x7f93f0071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.619+0000 7f93f6600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93f0071540 con 0x7f93f015f150 2024-03-28T13:16:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.619+0000 7f93f6600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f93f015f5b0 con 0x7f93f00b35b0 2024-03-28T13:16:47.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.619+0000 7f93f6600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f93f0144d60 con 0x7f93f0144980 2024-03-28T13:16:47.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.619+0000 7f93f5200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f0144980 0x7f93f015ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.619+0000 7f93f5c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f93f00b35b0 0x7f93f0071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.619+0000 7f93f5200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f0144980 0x7f93f015ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:39384/0 (socket says 172.21.15.19:39384) 2024-03-28T13:16:47.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.619+0000 7f93f5200640 1 -- 172.21.15.19:0/4109299774 learned_addr learned my addr 172.21.15.19:0/4109299774 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:47.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.619+0000 7f93efe00640 1 --1- 172.21.15.19:0/4109299774 >> v1:172.21.15.188:6789/0 conn(0x7f93f015f150 0x7f93f00b2ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:55834/0 (socket says 172.21.15.19:55834) 2024-03-28T13:16:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.620+0000 7f93f5c00640 1 -- 172.21.15.19:0/4109299774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f0144980 msgr2=0x7f93f015ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.620+0000 7f93f5c00640 1 --2- 172.21.15.19:0/4109299774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f0144980 0x7f93f015ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.620+0000 7f93f5c00640 1 -- 172.21.15.19:0/4109299774 >> v1:172.21.15.188:6789/0 conn(0x7f93f015f150 legacy=0x7f93f00b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.620+0000 7f93f5c00640 1 -- 172.21.15.19:0/4109299774 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f00b3b60 con 0x7f93f00b35b0 2024-03-28T13:16:47.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.622+0000 7f93f5c00640 1 --2- 172.21.15.19:0/4109299774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f00b35b0 0x7f93f0071a40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009d90 tx=0x7f93e802e020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffbbe05c90224a57 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.623+0000 7f93ef400640 1 -- 172.21.15.19:0/4109299774 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e803b070 con 0x7f93f00b35b0 2024-03-28T13:16:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.623+0000 7f93ef400640 1 -- 172.21.15.19:0/4109299774 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e802edd0 con 0x7f93f00b35b0 2024-03-28T13:16:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.623+0000 7f93ef400640 1 -- 172.21.15.19:0/4109299774 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e8036660 con 0x7f93f00b35b0 2024-03-28T13:16:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.624+0000 7f93f6600640 1 -- 172.21.15.19:0/4109299774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f00b35b0 msgr2=0x7f93f0071a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.624+0000 7f93f6600640 1 --2- 172.21.15.19:0/4109299774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f00b35b0 0x7f93f0071a40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009d90 tx=0x7f93e802e020 comp rx=0 tx=0).stop 2024-03-28T13:16:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.624+0000 7f93f6600640 1 -- 172.21.15.19:0/4109299774 shutdown_connections 2024-03-28T13:16:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.624+0000 7f93f6600640 1 --2- 172.21.15.19:0/4109299774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f00b35b0 0x7f93f0071a40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.624+0000 7f93f6600640 1 --2- 172.21.15.19:0/4109299774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f0144980 0x7f93f015ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.624+0000 7f93f6600640 1 -- 172.21.15.19:0/4109299774 >> 172.21.15.19:0/4109299774 conn(0x7f93f00b1d80 msgr2=0x7f93f00b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.624+0000 7f93f6600640 1 -- 172.21.15.19:0/4109299774 shutdown_connections 2024-03-28T13:16:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.624+0000 7f93f6600640 1 -- 172.21.15.19:0/4109299774 wait complete. 2024-03-28T13:16:47.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.626+0000 7f93f6600640 1 Processor -- start 2024-03-28T13:16:47.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.626+0000 7f93f6600640 1 -- start start 2024-03-28T13:16:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.627+0000 7f93f6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f00b35b0 0x7f93f0155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.627+0000 7f93f6600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f0144980 0x7f93f015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.627+0000 7f93f6600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f015f150 0x7f93f015a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.627+0000 7f93f6600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f93f0071f80 con 0x7f93f00b35b0 2024-03-28T13:16:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.627+0000 7f93f6600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f93f00b4120 con 0x7f93f0144980 2024-03-28T13:16:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.627+0000 7f93f6600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f93f0072100 con 0x7f93f015f150 2024-03-28T13:16:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.628+0000 7f93f5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f00b35b0 0x7f93f0155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.628+0000 7f93f5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f00b35b0 0x7f93f0155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52604/0 (socket says 172.21.15.19:52604) 2024-03-28T13:16:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.628+0000 7f93f5200640 1 -- 172.21.15.19:0/3044248513 learned_addr learned my addr 172.21.15.19:0/3044248513 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.628+0000 7f93efe00640 1 --2- 172.21.15.19:0/3044248513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f0144980 0x7f93f015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.628+0000 7f93f5c00640 1 --2- 172.21.15.19:0/3044248513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f015f150 0x7f93f015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.628+0000 7f93f5200640 1 -- 172.21.15.19:0/3044248513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f015f150 msgr2=0x7f93f015a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.628+0000 7f93f5200640 1 --2- 172.21.15.19:0/3044248513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f015f150 0x7f93f015a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.628+0000 7f93f5200640 1 -- 172.21.15.19:0/3044248513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f0144980 msgr2=0x7f93f015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.628+0000 7f93f5200640 1 --2- 172.21.15.19:0/3044248513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f0144980 0x7f93f015a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.628+0000 7f93f5200640 1 -- 172.21.15.19:0/3044248513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f00a9480 con 0x7f93f00b35b0 2024-03-28T13:16:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.628+0000 7f93efe00640 1 --2- 172.21.15.19:0/3044248513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f0144980 0x7f93f015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:16:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.629+0000 7f93f5200640 1 --2- 172.21.15.19:0/3044248513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f00b35b0 0x7f93f0155130 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009fd0 tx=0x7f93e400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.629+0000 7f93ecc00640 1 -- 172.21.15.19:0/3044248513 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4017070 con 0x7f93f00b35b0 2024-03-28T13:16:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.629+0000 7f93ecc00640 1 -- 172.21.15.19:0/3044248513 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e400c940 con 0x7f93f00b35b0 2024-03-28T13:16:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.629+0000 7f93ecc00640 1 -- 172.21.15.19:0/3044248513 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e400cd10 con 0x7f93f00b35b0 2024-03-28T13:16:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.629+0000 7f93f6600640 1 -- 172.21.15.19:0/3044248513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f015e270 con 0x7f93f00b35b0 2024-03-28T13:16:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.629+0000 7f93f6600640 1 -- 172.21.15.19:0/3044248513 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f00a9a20 con 0x7f93f00b35b0 2024-03-28T13:16:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.631+0000 7f93ecc00640 1 -- 172.21.15.19:0/3044248513 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f93e401a070 con 0x7f93f00b35b0 2024-03-28T13:16:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.632+0000 7f93ecc00640 1 --2- 172.21.15.19:0/3044248513 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f93d006c560 0x7f93d006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.632+0000 7f93ecc00640 1 -- 172.21.15.19:0/3044248513 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f93e408b6d0 con 0x7f93f00b35b0 2024-03-28T13:16:47.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.633+0000 7f93fa98b740 1 Processor -- start 2024-03-28T13:16:47.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.633+0000 7f93efe00640 1 --2- 172.21.15.19:0/3044248513 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f93d006c560 0x7f93d006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.633+0000 7f93fa98b740 1 -- start start 2024-03-28T13:16:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.634+0000 7f93fa98b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17119140 0x557e170ec0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.634+0000 7f93fa98b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119520 0x557e1711bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.634+0000 7f93fa98b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e170ec620 0x557e1711c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.634+0000 7f93fa98b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557e17055360 con 0x557e17119520 2024-03-28T13:16:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.634+0000 7f93fa98b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557e16ffd750 con 0x557e17119140 2024-03-28T13:16:47.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.634+0000 7f93fa98b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x557e17051310 con 0x557e170ec620 2024-03-28T13:16:47.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.634+0000 7f93efe00640 1 --2- 172.21.15.19:0/3044248513 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f93d006c560 0x7f93d006ea20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f93dc007fd0 tx=0x7f93dc007d60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:47.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.634+0000 7f93efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119520 0x557e1711bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.634+0000 7f93efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119520 0x557e1711bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52612/0 (socket says 172.21.15.19:52612) 2024-03-28T13:16:47.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.635+0000 7f93efe00640 1 -- 172.21.15.19:0/2798753655 learned_addr learned my addr 172.21.15.19:0/2798753655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:47.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.635+0000 7f93f5200640 1 --2- 172.21.15.19:0/2798753655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17119140 0x557e170ec0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.635+0000 7f93f5c00640 1 --2- 172.21.15.19:0/2798753655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e170ec620 0x557e1711c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.635+0000 7f93efe00640 1 -- 172.21.15.19:0/2798753655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e170ec620 msgr2=0x557e1711c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.635+0000 7f93efe00640 1 --2- 172.21.15.19:0/2798753655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e170ec620 0x557e1711c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.635+0000 7f93efe00640 1 -- 172.21.15.19:0/2798753655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17119140 msgr2=0x557e170ec0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.635+0000 7f93efe00640 1 --2- 172.21.15.19:0/2798753655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17119140 0x557e170ec0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.635+0000 7f93efe00640 1 -- 172.21.15.19:0/2798753655 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e1701eda0 con 0x557e17119520 2024-03-28T13:16:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.635+0000 7f93efe00640 1 --2- 172.21.15.19:0/2798753655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119520 0x557e1711bed0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0076a0 tx=0x7f93dc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fba80ca7bd33902 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.636+0000 7f93e1600640 1 -- 172.21.15.19:0/2798753655 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc001b60 con 0x557e17119520 2024-03-28T13:16:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.636+0000 7f93e1600640 1 -- 172.21.15.19:0/2798753655 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93dc001d00 con 0x557e17119520 2024-03-28T13:16:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.636+0000 7f93e1600640 1 -- 172.21.15.19:0/2798753655 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc015bc0 con 0x557e17119520 2024-03-28T13:16:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.636+0000 7f93fa98b740 1 -- 172.21.15.19:0/2798753655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119520 msgr2=0x557e1711bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.636+0000 7f93fa98b740 1 --2- 172.21.15.19:0/2798753655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119520 0x557e1711bed0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0076a0 tx=0x7f93dc012000 comp rx=0 tx=0).stop 2024-03-28T13:16:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.636+0000 7f93fa98b740 1 -- 172.21.15.19:0/2798753655 shutdown_connections 2024-03-28T13:16:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.636+0000 7f93fa98b740 1 --2- 172.21.15.19:0/2798753655 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e170ec620 0x557e1711c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.636+0000 7f93fa98b740 1 --2- 172.21.15.19:0/2798753655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119520 0x557e1711bed0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.636+0000 7f93fa98b740 1 --2- 172.21.15.19:0/2798753655 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17119140 0x557e170ec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.636+0000 7f93fa98b740 1 -- 172.21.15.19:0/2798753655 >> 172.21.15.19:0/2798753655 conn(0x557e170ee350 msgr2=0x557e17136ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:47.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.637+0000 7f93fa98b740 1 -- 172.21.15.19:0/2798753655 shutdown_connections 2024-03-28T13:16:47.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.637+0000 7f93fa98b740 1 -- 172.21.15.19:0/2798753655 wait complete. 2024-03-28T13:16:47.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.637+0000 7f93fa98b740 1 -- start start 2024-03-28T13:16:47.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.638+0000 7f93fa98b740 1 Processor -- start 2024-03-28T13:16:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.639+0000 7f93fa98b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17119140 0x557e17111650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.639+0000 7f93fa98b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119520 0x557e17111f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.639+0000 7f93fa98b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e17135b30 0x557e17114580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.639+0000 7f93fa98b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557e170f09f0 con 0x557e17119520 2024-03-28T13:16:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.639+0000 7f93fa98b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557e17010b10 con 0x557e17119140 2024-03-28T13:16:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.639+0000 7f93fa98b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x557e170f3230 con 0x557e17135b30 2024-03-28T13:16:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.640+0000 7f93fa98b740 10 client.0 _ll_register_callbacks cb 0x557e1704bed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:16:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.640+0000 7f93efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119520 0x557e17111f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.640+0000 7f93efe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119520 0x557e17111f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52620/0 (socket says 172.21.15.19:52620) 2024-03-28T13:16:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.640+0000 7f93efe00640 1 -- 172.21.15.19:0/4252130994 learned_addr learned my addr 172.21.15.19:0/4252130994 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.640+0000 7f93f5200640 1 --2- 172.21.15.19:0/4252130994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17119140 0x557e17111650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.640+0000 7f93f5c00640 1 --2- 172.21.15.19:0/4252130994 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e17135b30 0x557e17114580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.640+0000 7f93efe00640 1 -- 172.21.15.19:0/4252130994 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e17135b30 msgr2=0x557e17114580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.640+0000 7f93efe00640 1 --2- 172.21.15.19:0/4252130994 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e17135b30 0x557e17114580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.640+0000 7f93efe00640 1 -- 172.21.15.19:0/4252130994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17119140 msgr2=0x557e17111650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.640+0000 7f93efe00640 1 --2- 172.21.15.19:0/4252130994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17119140 0x557e17111650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.640+0000 7f93efe00640 1 -- 172.21.15.19:0/4252130994 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e16e357b0 con 0x557e17119520 2024-03-28T13:16:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.641+0000 7f93efe00640 1 --2- 172.21.15.19:0/4252130994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119520 0x557e17111f90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f93dc015810 tx=0x7f93dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776b87ce8b95afee server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.641+0000 7f93d6000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:16:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.641+0000 7f93d6000640 1 -- 172.21.15.19:0/4252130994 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc030060 con 0x557e17119520 2024-03-28T13:16:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.641+0000 7f93d6000640 1 -- 172.21.15.19:0/4252130994 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f93dc010070 con 0x557e17119520 2024-03-28T13:16:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.641+0000 7f93fa98b740 1 -- 172.21.15.19:0/4252130994 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93dc001d00 con 0x557e17119520 2024-03-28T13:16:47.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.641+0000 7f93d6000640 1 -- 172.21.15.19:0/4252130994 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f93dc001b60 con 0x557e17119520 2024-03-28T13:16:47.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.641+0000 7f93fa98b740 10 client.7470 fetch_fsmap learned FSMap version 67 2024-03-28T13:16:47.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.641+0000 7f93fa98b740 1 -- 172.21.15.19:0/4252130994 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x557e16e95f40 con 0x557e17119520 2024-03-28T13:16:47.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.642+0000 7f93d6000640 1 -- 172.21.15.19:0/4252130994 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93dc001d00 con 0x557e17119520 2024-03-28T13:16:47.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.642+0000 7f93d6000640 1 -- 172.21.15.19:0/4252130994 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f93dc021450 con 0x557e17119520 2024-03-28T13:16:47.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.642+0000 7f93fa98b740 10 client.7470 fetch_fsmap finished waiting for FSMap version 67 2024-03-28T13:16:47.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.642+0000 7f93fa98b740 10 client.7470 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-28T13:16:47.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.642+0000 7f93fa98b740 20 client.7470 populate_metadata read hostname 'smithi019' 2024-03-28T13:16:47.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.642+0000 7f93fa98b740 1 --2- 172.21.15.19:0/4252130994 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x557e1711f590 0x557e1711f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.642+0000 7f93fa98b740 4 client.7470 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:16:47.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.642+0000 7f93fa98b740 1 -- 172.21.15.19:0/4252130994 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e170628a0 con 0x557e1711f590 2024-03-28T13:16:47.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.643+0000 7f93f5200640 1 --2- 172.21.15.19:0/4252130994 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x557e1711f590 0x557e1711f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.644+0000 7f93f5200640 1 --2- 172.21.15.19:0/4252130994 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x557e1711f590 0x557e1711f9b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8f1c6b92ea8dad server_cookie=a8ae3d1f6ba783d2 in_seq=0 out_seq=0 2024-03-28T13:16:47.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.644+0000 7f93d6000640 10 client.7470 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:16:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.646+0000 7f93d6000640 1 -- 172.21.15.19:0/4252130994 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e170628a0 con 0x557e1711f590 2024-03-28T13:16:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.646+0000 7f93d6000640 10 client.7470 handle_command_reply: tid=0 2024-03-28T13:16:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.646+0000 7f93fa98b740 1 client.7470 shutdown 2024-03-28T13:16:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.646+0000 7f93fa98b740 1 -- 172.21.15.19:0/4252130994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119520 msgr2=0x557e17111f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.646+0000 7f93fa98b740 1 --2- 172.21.15.19:0/4252130994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119520 0x557e17111f90 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f93dc015810 tx=0x7f93dc000bc0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.646+0000 7f93fa98b740 1 -- 172.21.15.19:0/4252130994 shutdown_connections 2024-03-28T13:16:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.646+0000 7f93fa98b740 1 --2- 172.21.15.19:0/4252130994 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x557e1711f590 0x557e1711f9b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.646+0000 7f93fa98b740 1 --2- 172.21.15.19:0/4252130994 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e17135b30 0x557e17114580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.646+0000 7f93fa98b740 1 --2- 172.21.15.19:0/4252130994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119520 0x557e17111f90 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.646+0000 7f93fa98b740 1 --2- 172.21.15.19:0/4252130994 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17119140 0x557e17111650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.646+0000 7f93d6000640 0 client.7470 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:16:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.646+0000 7f93fa98b740 1 -- 172.21.15.19:0/4252130994 >> 172.21.15.19:0/4252130994 conn(0x557e170ec620 msgr2=0x557e1711b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.646+0000 7f93fa98b740 1 -- 172.21.15.19:0/4252130994 shutdown_connections 2024-03-28T13:16:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.646+0000 7f93fa98b740 1 -- 172.21.15.19:0/4252130994 wait complete. 2024-03-28T13:16:47.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.647+0000 7f93fa98b740 20 client.7470 trim_cache size 0 max 16384 2024-03-28T13:16:47.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.667+0000 7f93fa98b740 1 Processor -- start 2024-03-28T13:16:47.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.667+0000 7f93fa98b740 1 -- start start 2024-03-28T13:16:47.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.668+0000 7f93fa98b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119140 0x557e1713a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.668+0000 7f93fa98b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e17119520 0x557e1711e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.668+0000 7f93fa98b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17135b30 0x557e171aace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.668+0000 7f93fa98b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f93f0200bd0 con 0x557e17119140 2024-03-28T13:16:47.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.668+0000 7f93fa98b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f93f01ffbb0 con 0x557e17135b30 2024-03-28T13:16:47.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.668+0000 7f93fa98b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f93f01fef20 con 0x557e17119520 2024-03-28T13:16:47.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.668+0000 7f93f5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119140 0x557e1713a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.668+0000 7f93f5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119140 0x557e1713a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52632/0 (socket says 172.21.15.19:52632) 2024-03-28T13:16:47.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.668+0000 7f93f5200640 1 -- 172.21.15.19:0/864957172 learned_addr learned my addr 172.21.15.19:0/864957172 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:47.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.669+0000 7f93f5200640 1 -- 172.21.15.19:0/864957172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e17119520 msgr2=0x557e1711e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.669+0000 7f93f5200640 1 --2- 172.21.15.19:0/864957172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e17119520 0x557e1711e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.669+0000 7f93f5200640 1 -- 172.21.15.19:0/864957172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17135b30 msgr2=0x557e171aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.669+0000 7f93f5c00640 1 --2- 172.21.15.19:0/864957172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17135b30 0x557e171aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.669+0000 7f93f5200640 1 --2- 172.21.15.19:0/864957172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17135b30 0x557e171aace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.669+0000 7f93f5200640 1 -- 172.21.15.19:0/864957172 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e16d290a0 con 0x557e17119140 2024-03-28T13:16:47.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.669+0000 7f93f5c00640 1 --2- 172.21.15.19:0/864957172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17135b30 0x557e171aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:47.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.669+0000 7f93f5200640 1 --2- 172.21.15.19:0/864957172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119140 0x557e1713a3e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f93e4059480 tx=0x7f93e408b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3ce92c60eddafd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:47.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.669+0000 7f93e0c00640 1 -- 172.21.15.19:0/864957172 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4087070 con 0x557e17119140 2024-03-28T13:16:47.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.669+0000 7f93e0c00640 1 -- 172.21.15.19:0/864957172 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e40620d0 con 0x557e17119140 2024-03-28T13:16:47.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.669+0000 7f93e0c00640 1 -- 172.21.15.19:0/864957172 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4021040 con 0x557e17119140 2024-03-28T13:16:47.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.670+0000 7f93fa98b740 1 -- 172.21.15.19:0/864957172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119140 msgr2=0x557e1713a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.670+0000 7f93fa98b740 1 --2- 172.21.15.19:0/864957172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119140 0x557e1713a3e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f93e4059480 tx=0x7f93e408b040 comp rx=0 tx=0).stop 2024-03-28T13:16:47.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.670+0000 7f93fa98b740 1 -- 172.21.15.19:0/864957172 shutdown_connections 2024-03-28T13:16:47.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.670+0000 7f93fa98b740 1 --2- 172.21.15.19:0/864957172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17135b30 0x557e171aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.670+0000 7f93fa98b740 1 --2- 172.21.15.19:0/864957172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e17119520 0x557e1711e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.670+0000 7f93fa98b740 1 --2- 172.21.15.19:0/864957172 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119140 0x557e1713a3e0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.670+0000 7f93fa98b740 1 -- 172.21.15.19:0/864957172 >> 172.21.15.19:0/864957172 conn(0x557e170ec620 msgr2=0x557e17111460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:47.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.670+0000 7f93fa98b740 1 -- 172.21.15.19:0/864957172 shutdown_connections 2024-03-28T13:16:47.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.670+0000 7f93fa98b740 1 -- 172.21.15.19:0/864957172 wait complete. 2024-03-28T13:16:47.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.671+0000 7f93fa98b740 1 -- start start 2024-03-28T13:16:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.671+0000 7f93fa98b740 1 Processor -- start 2024-03-28T13:16:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.672+0000 7f93fa98b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119140 0x557e1711e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.672+0000 7f93fa98b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17119520 0x557e1711ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.672+0000 7f93fa98b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e17135b30 0x557e17157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.672+0000 7f93fa98b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557e1715b540 con 0x557e17119140 2024-03-28T13:16:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.672+0000 7f93fa98b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f93f01ff540 con 0x557e17119520 2024-03-28T13:16:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.672+0000 7f93fa98b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x557e1711ced0 con 0x557e17135b30 2024-03-28T13:16:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.672+0000 7f93fa98b740 10 client.0 _ll_register_callbacks cb 0x7f93e400ceb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:16:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.673+0000 7f93f5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119140 0x557e1711e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.673+0000 7f93f5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119140 0x557e1711e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52642/0 (socket says 172.21.15.19:52642) 2024-03-28T13:16:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.673+0000 7f93f5200640 1 -- 172.21.15.19:0/2839631016 learned_addr learned my addr 172.21.15.19:0/2839631016 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:47.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.673+0000 7f93f5c00640 1 --2- 172.21.15.19:0/2839631016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e17135b30 0x557e17157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.673+0000 7f93efe00640 1 --2- 172.21.15.19:0/2839631016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17119520 0x557e1711ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.673+0000 7f93f5200640 1 -- 172.21.15.19:0/2839631016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e17135b30 msgr2=0x557e17157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.673+0000 7f93f5200640 1 --2- 172.21.15.19:0/2839631016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e17135b30 0x557e17157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.673+0000 7f93f5200640 1 -- 172.21.15.19:0/2839631016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17119520 msgr2=0x557e1711ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.673+0000 7f93f5200640 1 --2- 172.21.15.19:0/2839631016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17119520 0x557e1711ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.673+0000 7f93f5200640 1 -- 172.21.15.19:0/2839631016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e17135920 con 0x557e17119140 2024-03-28T13:16:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93f5200640 1 --2- 172.21.15.19:0/2839631016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119140 0x557e1711e810 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f93e4059480 tx=0x7f93e405c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18778099b95a5544 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93d6000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:16:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93d6000640 1 -- 172.21.15.19:0/2839631016 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4021070 con 0x557e17119140 2024-03-28T13:16:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93d6000640 1 -- 172.21.15.19:0/2839631016 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f93e4058940 con 0x557e17119140 2024-03-28T13:16:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93fa98b740 1 -- 172.21.15.19:0/2839631016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93e40620d0 con 0x557e17119140 2024-03-28T13:16:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93d6000640 1 -- 172.21.15.19:0/2839631016 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f93e4087070 con 0x557e17119140 2024-03-28T13:16:47.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93fa98b740 10 client.7476 fetch_fsmap learned FSMap version 67 2024-03-28T13:16:47.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93fa98b740 1 -- 172.21.15.19:0/2839631016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x557e17135400 con 0x557e17119140 2024-03-28T13:16:47.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93d6000640 1 -- 172.21.15.19:0/2839631016 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e40620d0 con 0x557e17119140 2024-03-28T13:16:47.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93d6000640 1 -- 172.21.15.19:0/2839631016 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f93e4024060 con 0x557e17119140 2024-03-28T13:16:47.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93fa98b740 10 client.7476 fetch_fsmap finished waiting for FSMap version 67 2024-03-28T13:16:47.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93fa98b740 10 client.7476 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-28T13:16:47.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93fa98b740 20 client.7476 populate_metadata read hostname 'smithi019' 2024-03-28T13:16:47.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93fa98b740 1 --2- 172.21.15.19:0/2839631016 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x557e171a42a0 0x557e17158420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93fa98b740 4 client.7476 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-03-28T13:16:47.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.674+0000 7f93fa98b740 1 -- 172.21.15.19:0/2839631016 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x557e1705cbf0 con 0x557e171a42a0 2024-03-28T13:16:47.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.675+0000 7f93efe00640 1 --2- 172.21.15.19:0/2839631016 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x557e171a42a0 0x557e17158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.677 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-03-28T13:16:47.677+0000 7f8097200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-03-28T13:16:35.247197+0000 1=1+0) n(v0 rc2024-03-28T13:16:35.374196+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x5622e97f0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-28T13:16:35.247197+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-28T13:16:35.374196+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-03-28T13:16:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.676+0000 7f93efe00640 1 --2- 172.21.15.19:0/2839631016 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x557e171a42a0 0x557e17158420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca098eff4af33270 server_cookie=d09ae7d01169d897 in_seq=0 out_seq=0 2024-03-28T13:16:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.676+0000 7f93d6000640 10 client.7476 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:16:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.677+0000 7f93d6000640 1 -- 172.21.15.19:0/2839631016 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557e1705cbf0 con 0x557e171a42a0 2024-03-28T13:16:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.677+0000 7f93d6000640 10 client.7476 handle_command_reply: tid=0 2024-03-28T13:16:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.677+0000 7f93fa98b740 1 client.7476 shutdown 2024-03-28T13:16:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.677+0000 7f93fa98b740 1 -- 172.21.15.19:0/2839631016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119140 msgr2=0x557e1711e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.677+0000 7f93fa98b740 1 --2- 172.21.15.19:0/2839631016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119140 0x557e1711e810 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f93e4059480 tx=0x7f93e405c040 comp rx=0 tx=0).stop 2024-03-28T13:16:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.677+0000 7f93fa98b740 1 -- 172.21.15.19:0/2839631016 shutdown_connections 2024-03-28T13:16:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.677+0000 7f93fa98b740 1 --2- 172.21.15.19:0/2839631016 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x557e171a42a0 0x557e17158420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.677+0000 7f93fa98b740 1 --2- 172.21.15.19:0/2839631016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557e17135b30 0x557e17157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.677+0000 7f93fa98b740 1 --2- 172.21.15.19:0/2839631016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557e17119520 0x557e1711ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.677+0000 7f93fa98b740 1 --2- 172.21.15.19:0/2839631016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557e17119140 0x557e1711e810 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.677+0000 7f93d6000640 0 client.7476 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:16:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.677+0000 7f93fa98b740 1 -- 172.21.15.19:0/2839631016 >> 172.21.15.19:0/2839631016 conn(0x557e170ec620 msgr2=0x557e170e7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.677+0000 7f93fa98b740 1 -- 172.21.15.19:0/2839631016 shutdown_connections 2024-03-28T13:16:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.678+0000 7f93fa98b740 1 -- 172.21.15.19:0/2839631016 wait complete. 2024-03-28T13:16:47.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.678+0000 7f93fa98b740 20 client.7476 trim_cache size 0 max 16384 2024-03-28T13:16:47.679 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:16:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-03-28T13:16:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "b7ec899d-e435-4456-a1da-7c3dc82479e4", 2024-03-28T13:16:47.680 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-03-28T13:16:47.680 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:16:47.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.678+0000 7f93f6600640 1 -- 172.21.15.19:0/3044248513 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f93d006c560 msgr2=0x7f93d006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.678+0000 7f93f6600640 1 --2- 172.21.15.19:0/3044248513 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f93d006c560 0x7f93d006ea20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f93dc007fd0 tx=0x7f93dc007d60 comp rx=0 tx=0).stop 2024-03-28T13:16:47.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.678+0000 7f93f6600640 1 -- 172.21.15.19:0/3044248513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f00b35b0 msgr2=0x7f93f0155130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.678+0000 7f93f6600640 1 --2- 172.21.15.19:0/3044248513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f00b35b0 0x7f93f0155130 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f93e4009fd0 tx=0x7f93e400ed40 comp rx=0 tx=0).stop 2024-03-28T13:16:47.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.678+0000 7f93f6600640 1 -- 172.21.15.19:0/3044248513 shutdown_connections 2024-03-28T13:16:47.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.678+0000 7f93f6600640 1 --2- 172.21.15.19:0/3044248513 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f93d006c560 0x7f93d006ea20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.678+0000 7f93f6600640 1 --2- 172.21.15.19:0/3044248513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93f015f150 0x7f93f015a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.678+0000 7f93f6600640 1 --2- 172.21.15.19:0/3044248513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93f0144980 0x7f93f015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.678+0000 7f93f6600640 1 --2- 172.21.15.19:0/3044248513 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93f00b35b0 0x7f93f0155130 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.678+0000 7f93f6600640 1 -- 172.21.15.19:0/3044248513 >> 172.21.15.19:0/3044248513 conn(0x7f93f00b1d80 msgr2=0x7f93f00cf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.679+0000 7f93f6600640 1 -- 172.21.15.19:0/3044248513 shutdown_connections 2024-03-28T13:16:47.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.679+0000 7f93f6600640 1 -- 172.21.15.19:0/3044248513 wait complete. 2024-03-28T13:16:47.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:16:47.695 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-28T13:16:47.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.911+0000 7f4198e00640 1 Processor -- start 2024-03-28T13:16:47.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.912+0000 7f4198e00640 1 -- start start 2024-03-28T13:16:47.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.913+0000 7f4198e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f41940cc880 0x7f41940cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.913+0000 7f4198e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41940ce890 con 0x7f419413f190 2024-03-28T13:16:47.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.913+0000 7f4198e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41940ceb50 con 0x7f419406d620 2024-03-28T13:16:47.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.913+0000 7f4198e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f41940cd230 con 0x7f41940cc880 2024-03-28T13:16:47.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.914+0000 7f4193400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f419413f190 0x7f41940cf630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:48494/0 (socket says 172.21.15.19:48494) 2024-03-28T13:16:47.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.914+0000 7f4193400640 1 -- 172.21.15.19:0/2913513361 learned_addr learned my addr 172.21.15.19:0/2913513361 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:16:47.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.914+0000 7f418be00640 1 --2- 172.21.15.19:0/2913513361 >> v2:172.21.15.19:3300/0 conn(0x7f41940cc880 0x7f41940cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.914+0000 7f4192a00640 1 -- 172.21.15.19:0/2913513361 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4173299471 0 0) 0x7f41940ce890 con 0x7f419413f190 2024-03-28T13:16:47.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.914+0000 7f4192a00640 1 -- 172.21.15.19:0/2913513361 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4178003520 con 0x7f419413f190 2024-03-28T13:16:47.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.915+0000 7f418be00640 1 -- 172.21.15.19:0/2913513361 >> v1:172.21.15.188:6789/0 conn(0x7f419406d620 legacy=0x7f419406da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.915+0000 7f418be00640 1 -- 172.21.15.19:0/2913513361 >> v1:172.21.15.19:6789/0 conn(0x7f419413f190 legacy=0x7f41940cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.915+0000 7f418be00640 1 -- 172.21.15.19:0/2913513361 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41940cee30 con 0x7f41940cc880 2024-03-28T13:16:47.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.916+0000 7f418be00640 1 --2- 172.21.15.19:0/2913513361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41940cc880 0x7f41940cccf0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f417c0097a0 tx=0x7f417c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cca61730281d76 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:47.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.916+0000 7f4192a00640 1 -- 172.21.15.19:0/2913513361 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c034070 con 0x7f41940cc880 2024-03-28T13:16:47.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.916+0000 7f4192a00640 1 -- 172.21.15.19:0/2913513361 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f417c030610 con 0x7f41940cc880 2024-03-28T13:16:47.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.916+0000 7f4192a00640 1 -- 172.21.15.19:0/2913513361 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c0309e0 con 0x7f41940cc880 2024-03-28T13:16:47.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.917+0000 7f4198e00640 1 -- 172.21.15.19:0/2913513361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41940cc880 msgr2=0x7f41940cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.917+0000 7f4198e00640 1 --2- 172.21.15.19:0/2913513361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41940cc880 0x7f41940cccf0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f417c0097a0 tx=0x7f417c02b960 comp rx=0 tx=0).stop 2024-03-28T13:16:47.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.917+0000 7f4198e00640 1 -- 172.21.15.19:0/2913513361 shutdown_connections 2024-03-28T13:16:47.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.917+0000 7f4198e00640 1 --2- 172.21.15.19:0/2913513361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41940cc880 0x7f41940cccf0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.917+0000 7f4198e00640 1 -- 172.21.15.19:0/2913513361 >> 172.21.15.19:0/2913513361 conn(0x7f41940b1ba0 msgr2=0x7f41940b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:47.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.917+0000 7f4198e00640 1 -- 172.21.15.19:0/2913513361 shutdown_connections 2024-03-28T13:16:47.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.917+0000 7f4198e00640 1 -- 172.21.15.19:0/2913513361 wait complete. 2024-03-28T13:16:47.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.919+0000 7f4198e00640 1 Processor -- start 2024-03-28T13:16:47.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.919+0000 7f4198e00640 1 -- start start 2024-03-28T13:16:47.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.920+0000 7f4198e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f419406d620 0x7f41940725f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.920+0000 7f4198e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41940cc880 0x7f4194072b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.920+0000 7f4198e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419413f190 0x7f419412d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.920+0000 7f4198e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f419413f570 con 0x7f419413f190 2024-03-28T13:16:47.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.920+0000 7f4198e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4194160c70 con 0x7f419406d620 2024-03-28T13:16:47.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.920+0000 7f4198e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f419415ff10 con 0x7f41940cc880 2024-03-28T13:16:47.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.920+0000 7f4193e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419413f190 0x7f419412d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.920+0000 7f4193e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419413f190 0x7f419412d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52670/0 (socket says 172.21.15.19:52670) 2024-03-28T13:16:47.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.920+0000 7f4193e00640 1 -- 172.21.15.19:0/3320612338 learned_addr learned my addr 172.21.15.19:0/3320612338 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:47.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.920+0000 7f4193400640 1 --2- 172.21.15.19:0/3320612338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f419406d620 0x7f41940725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.920+0000 7f418be00640 1 --2- 172.21.15.19:0/3320612338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41940cc880 0x7f4194072b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.921+0000 7f4193e00640 1 -- 172.21.15.19:0/3320612338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41940cc880 msgr2=0x7f4194072b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.921+0000 7f4193e00640 1 --2- 172.21.15.19:0/3320612338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41940cc880 0x7f4194072b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.921+0000 7f4193e00640 1 -- 172.21.15.19:0/3320612338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f419406d620 msgr2=0x7f41940725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.921+0000 7f4193e00640 1 --2- 172.21.15.19:0/3320612338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f419406d620 0x7f41940725f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.921+0000 7f4193e00640 1 -- 172.21.15.19:0/3320612338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f419412de10 con 0x7f419413f190 2024-03-28T13:16:47.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.921+0000 7f4193400640 1 --2- 172.21.15.19:0/3320612338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f419406d620 0x7f41940725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:47.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.921+0000 7f4193e00640 1 --2- 172.21.15.19:0/3320612338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419413f190 0x7f419412d0e0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f418c00c6b0 tx=0x7f418c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:47.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.921+0000 7f418b400640 1 -- 172.21.15.19:0/3320612338 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c003b70 con 0x7f419413f190 2024-03-28T13:16:47.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.921+0000 7f418b400640 1 -- 172.21.15.19:0/3320612338 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f418c003d30 con 0x7f419413f190 2024-03-28T13:16:47.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.921+0000 7f418b400640 1 -- 172.21.15.19:0/3320612338 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c005310 con 0x7f419413f190 2024-03-28T13:16:47.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.921+0000 7f4198e00640 1 -- 172.21.15.19:0/3320612338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41941e65c0 con 0x7f419413f190 2024-03-28T13:16:47.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.921+0000 7f4198e00640 1 -- 172.21.15.19:0/3320612338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41941e6b80 con 0x7f419413f190 2024-03-28T13:16:47.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.923+0000 7f418b400640 1 -- 172.21.15.19:0/3320612338 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f418c0054b0 con 0x7f419413f190 2024-03-28T13:16:47.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.924+0000 7f418b400640 1 --2- 172.21.15.19:0/3320612338 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f417006c530 0x7f417006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.924+0000 7f418b400640 1 -- 172.21.15.19:0/3320612338 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f418c08b710 con 0x7f419413f190 2024-03-28T13:16:47.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.925+0000 7f4193400640 1 --2- 172.21.15.19:0/3320612338 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f417006c530 0x7f417006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.925+0000 7f419d1d8740 1 Processor -- start 2024-03-28T13:16:47.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.925+0000 7f419d1d8740 1 -- start start 2024-03-28T13:16:47.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.926+0000 7f4193400640 1 --2- 172.21.15.19:0/3320612338 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f417006c530 0x7f417006e9f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f41840011e0 tx=0x7f4184009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:47.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.926+0000 7f419d1d8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b29720 0x5644d3afc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.926+0000 7f419d1d8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b2c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.926+0000 7f419d1d8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3afcc00 0x5644d3b2c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.926+0000 7f419d1d8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5644d3a676b0 con 0x5644d3b29b00 2024-03-28T13:16:47.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.926+0000 7f419d1d8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5644d3a0f750 con 0x5644d3b29720 2024-03-28T13:16:47.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.926+0000 7f419d1d8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5644d3a63660 con 0x5644d3afcc00 2024-03-28T13:16:47.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.927+0000 7f418be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b2c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.927+0000 7f418be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b2c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52674/0 (socket says 172.21.15.19:52674) 2024-03-28T13:16:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.927+0000 7f418be00640 1 -- 172.21.15.19:0/18655337 learned_addr learned my addr 172.21.15.19:0/18655337 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.927+0000 7f4193400640 1 --2- 172.21.15.19:0/18655337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b29720 0x5644d3afc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.927+0000 7f418be00640 1 -- 172.21.15.19:0/18655337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3afcc00 msgr2=0x5644d3b2c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.927+0000 7f4193e00640 1 --2- 172.21.15.19:0/18655337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3afcc00 0x5644d3b2c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.927+0000 7f418be00640 1 --2- 172.21.15.19:0/18655337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3afcc00 0x5644d3b2c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.927+0000 7f418be00640 1 -- 172.21.15.19:0/18655337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b29720 msgr2=0x5644d3afc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.927+0000 7f418be00640 1 --2- 172.21.15.19:0/18655337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b29720 0x5644d3afc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.927+0000 7f418be00640 1 -- 172.21.15.19:0/18655337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644d38457d0 con 0x5644d3b29b00 2024-03-28T13:16:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.928+0000 7f418be00640 1 --2- 172.21.15.19:0/18655337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b2c4b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f417c02bf70 tx=0x7f417c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd77520353784f0c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.928+0000 7f4183e00640 1 -- 172.21.15.19:0/18655337 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c034040 con 0x5644d3b29b00 2024-03-28T13:16:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.928+0000 7f4183e00640 1 -- 172.21.15.19:0/18655337 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f417c039690 con 0x5644d3b29b00 2024-03-28T13:16:47.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.928+0000 7f4183e00640 1 -- 172.21.15.19:0/18655337 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c039a40 con 0x5644d3b29b00 2024-03-28T13:16:47.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.928+0000 7f419d1d8740 1 -- 172.21.15.19:0/18655337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 msgr2=0x5644d3b2c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.928+0000 7f419d1d8740 1 --2- 172.21.15.19:0/18655337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b2c4b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f417c02bf70 tx=0x7f417c008030 comp rx=0 tx=0).stop 2024-03-28T13:16:47.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.929+0000 7f419d1d8740 1 -- 172.21.15.19:0/18655337 shutdown_connections 2024-03-28T13:16:47.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.929+0000 7f419d1d8740 1 --2- 172.21.15.19:0/18655337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3afcc00 0x5644d3b2c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.929+0000 7f419d1d8740 1 --2- 172.21.15.19:0/18655337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b2c4b0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.929+0000 7f419d1d8740 1 --2- 172.21.15.19:0/18655337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b29720 0x5644d3afc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.929+0000 7f419d1d8740 1 -- 172.21.15.19:0/18655337 >> 172.21.15.19:0/18655337 conn(0x5644d3afe930 msgr2=0x5644d3b474b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:47.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.929+0000 7f419d1d8740 1 -- 172.21.15.19:0/18655337 shutdown_connections 2024-03-28T13:16:47.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.929+0000 7f419d1d8740 1 -- 172.21.15.19:0/18655337 wait complete. 2024-03-28T13:16:47.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.929+0000 7f419d1d8740 1 -- start start 2024-03-28T13:16:47.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.930+0000 7f419d1d8740 1 Processor -- start 2024-03-28T13:16:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.931+0000 7f419d1d8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29720 0x5644d3b2ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.931+0000 7f419d1d8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29b00 0x5644d3b452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.931+0000 7f419d1d8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b1ce20 0x5644d3b45810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.931+0000 7f419d1d8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5644d3b00fd0 con 0x5644d3b29720 2024-03-28T13:16:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.931+0000 7f419d1d8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5644d3a22a80 con 0x5644d3b1ce20 2024-03-28T13:16:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.931+0000 7f419d1d8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5644d3b03810 con 0x5644d3b29b00 2024-03-28T13:16:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.931+0000 7f419d1d8740 10 client.0 _ll_register_callbacks cb 0x5644d3a22470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:16:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.931+0000 7f4193400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29720 0x5644d3b2ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.931+0000 7f4193400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29720 0x5644d3b2ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52682/0 (socket says 172.21.15.19:52682) 2024-03-28T13:16:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.931+0000 7f4193400640 1 -- 172.21.15.19:0/218340551 learned_addr learned my addr 172.21.15.19:0/218340551 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.931+0000 7f418be00640 1 --2- 172.21.15.19:0/218340551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29b00 0x5644d3b452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.931+0000 7f4193e00640 1 --2- 172.21.15.19:0/218340551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b1ce20 0x5644d3b45810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.932+0000 7f4193400640 1 -- 172.21.15.19:0/218340551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29b00 msgr2=0x5644d3b452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.932+0000 7f4193400640 1 --2- 172.21.15.19:0/218340551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29b00 0x5644d3b452d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.932+0000 7f4193400640 1 -- 172.21.15.19:0/218340551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b1ce20 msgr2=0x5644d3b45810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.932+0000 7f4193400640 1 --2- 172.21.15.19:0/218340551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b1ce20 0x5644d3b45810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.932+0000 7f4193400640 1 -- 172.21.15.19:0/218340551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644d377b3f0 con 0x5644d3b29720 2024-03-28T13:16:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.932+0000 7f418be00640 1 --2- 172.21.15.19:0/218340551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29b00 0x5644d3b452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.932+0000 7f4193400640 1 --2- 172.21.15.19:0/218340551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29720 0x5644d3b2ff30 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4184007360 tx=0x7f4184000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e2a397573c9914 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.932+0000 7f4180c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:16:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.932+0000 7f4180c00640 1 -- 172.21.15.19:0/218340551 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4184026060 con 0x5644d3b29720 2024-03-28T13:16:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.932+0000 7f4180c00640 1 -- 172.21.15.19:0/218340551 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f4184021af0 con 0x5644d3b29720 2024-03-28T13:16:47.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.932+0000 7f419d1d8740 1 -- 172.21.15.19:0/218340551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f417c039690 con 0x5644d3b29720 2024-03-28T13:16:47.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.933+0000 7f4180c00640 1 -- 172.21.15.19:0/218340551 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f4184012410 con 0x5644d3b29720 2024-03-28T13:16:47.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.933+0000 7f419d1d8740 10 client.7491 fetch_fsmap learned FSMap version 67 2024-03-28T13:16:47.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.933+0000 7f419d1d8740 1 -- 172.21.15.19:0/218340551 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x5644d373b0f0 con 0x5644d3b29720 2024-03-28T13:16:47.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.933+0000 7f4180c00640 1 -- 172.21.15.19:0/218340551 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f417c039690 con 0x5644d3b29720 2024-03-28T13:16:47.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.933+0000 7f4180c00640 1 -- 172.21.15.19:0/218340551 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f41840125b0 con 0x5644d3b29720 2024-03-28T13:16:47.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.933+0000 7f419d1d8740 10 client.7491 fetch_fsmap finished waiting for FSMap version 67 2024-03-28T13:16:47.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.933+0000 7f419d1d8740 10 client.7491 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-28T13:16:47.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.933+0000 7f419d1d8740 20 client.7491 populate_metadata read hostname 'smithi019' 2024-03-28T13:16:47.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.933+0000 7f419d1d8740 1 --2- 172.21.15.19:0/218340551 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5644d3b2ed30 0x5644d3b2f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.933+0000 7f419d1d8740 4 client.7491 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:16:47.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.933+0000 7f419d1d8740 1 -- 172.21.15.19:0/218340551 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644d3a74bf0 con 0x5644d3b2ed30 2024-03-28T13:16:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.934+0000 7f418be00640 1 --2- 172.21.15.19:0/218340551 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5644d3b2ed30 0x5644d3b2f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.935+0000 7f418be00640 1 --2- 172.21.15.19:0/218340551 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5644d3b2ed30 0x5644d3b2f1b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c77052ce20f8fa server_cookie=ddf42490174bf76e in_seq=0 out_seq=0 2024-03-28T13:16:47.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.935+0000 7f4180c00640 10 client.7491 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:16:47.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.937+0000 7f4180c00640 1 -- 172.21.15.19:0/218340551 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5644d3a74bf0 con 0x5644d3b2ed30 2024-03-28T13:16:47.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.937+0000 7f4180c00640 10 client.7491 handle_command_reply: tid=0 2024-03-28T13:16:47.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.937+0000 7f419d1d8740 1 client.7491 shutdown 2024-03-28T13:16:47.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.937+0000 7f419d1d8740 1 -- 172.21.15.19:0/218340551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29720 msgr2=0x5644d3b2ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.937+0000 7f419d1d8740 1 --2- 172.21.15.19:0/218340551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29720 0x5644d3b2ff30 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f4184007360 tx=0x7f4184000ed0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.937+0000 7f419d1d8740 1 -- 172.21.15.19:0/218340551 shutdown_connections 2024-03-28T13:16:47.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.937+0000 7f419d1d8740 1 --2- 172.21.15.19:0/218340551 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5644d3b2ed30 0x5644d3b2f1b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.937+0000 7f419d1d8740 1 --2- 172.21.15.19:0/218340551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b1ce20 0x5644d3b45810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.937+0000 7f419d1d8740 1 --2- 172.21.15.19:0/218340551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29b00 0x5644d3b452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.937+0000 7f419d1d8740 1 --2- 172.21.15.19:0/218340551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29720 0x5644d3b2ff30 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.937+0000 7f4180c00640 0 client.7491 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:16:47.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.937+0000 7f419d1d8740 1 -- 172.21.15.19:0/218340551 >> 172.21.15.19:0/218340551 conn(0x5644d3afcc00 msgr2=0x5644d3b4a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:47.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.937+0000 7f419d1d8740 1 -- 172.21.15.19:0/218340551 shutdown_connections 2024-03-28T13:16:47.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.938+0000 7f419d1d8740 1 -- 172.21.15.19:0/218340551 wait complete. 2024-03-28T13:16:47.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.938+0000 7f419d1d8740 20 client.7491 trim_cache size 0 max 16384 2024-03-28T13:16:47.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.957+0000 7f419d1d8740 1 Processor -- start 2024-03-28T13:16:47.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.957+0000 7f419d1d8740 1 -- start start 2024-03-28T13:16:47.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.958+0000 7f419d1d8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b1ce20 0x5644d3b47db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.958+0000 7f419d1d8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29720 0x5644d3b27350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.958+0000 7f419d1d8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b4ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.958+0000 7f419d1d8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41942009f0 con 0x5644d3b29b00 2024-03-28T13:16:47.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.958+0000 7f419d1d8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f419415f6c0 con 0x5644d3b1ce20 2024-03-28T13:16:47.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.958+0000 7f419d1d8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f419415ea20 con 0x5644d3b29720 2024-03-28T13:16:47.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.958+0000 7f4193e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b4ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.958+0000 7f4193e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b4ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52698/0 (socket says 172.21.15.19:52698) 2024-03-28T13:16:47.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.958+0000 7f4193e00640 1 -- 172.21.15.19:0/1210594297 learned_addr learned my addr 172.21.15.19:0/1210594297 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:47.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.959+0000 7f418be00640 1 --2- 172.21.15.19:0/1210594297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29720 0x5644d3b27350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.959+0000 7f4193e00640 1 -- 172.21.15.19:0/1210594297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29720 msgr2=0x5644d3b27350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.959+0000 7f4193e00640 1 --2- 172.21.15.19:0/1210594297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29720 0x5644d3b27350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.959+0000 7f4193e00640 1 -- 172.21.15.19:0/1210594297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b1ce20 msgr2=0x5644d3b47db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.959+0000 7f4193e00640 1 --2- 172.21.15.19:0/1210594297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b1ce20 0x5644d3b47db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.959+0000 7f4193e00640 1 -- 172.21.15.19:0/1210594297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644d3b2f9e0 con 0x5644d3b29b00 2024-03-28T13:16:47.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.959+0000 7f4193e00640 1 --2- 172.21.15.19:0/1210594297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b4ae70 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f418c00c680 tx=0x7f418c058ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6408a12054f3a21a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:47.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.959+0000 7f4183400640 1 -- 172.21.15.19:0/1210594297 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c05d850 con 0x5644d3b29b00 2024-03-28T13:16:47.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.959+0000 7f4183400640 1 -- 172.21.15.19:0/1210594297 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f418c088070 con 0x5644d3b29b00 2024-03-28T13:16:47.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.959+0000 7f4183400640 1 -- 172.21.15.19:0/1210594297 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c062110 con 0x5644d3b29b00 2024-03-28T13:16:47.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.960+0000 7f419d1d8740 1 -- 172.21.15.19:0/1210594297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 msgr2=0x5644d3b4ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.960+0000 7f419d1d8740 1 --2- 172.21.15.19:0/1210594297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b4ae70 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f418c00c680 tx=0x7f418c058ec0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.960+0000 7f419d1d8740 1 -- 172.21.15.19:0/1210594297 shutdown_connections 2024-03-28T13:16:47.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.960+0000 7f419d1d8740 1 --2- 172.21.15.19:0/1210594297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b4ae70 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.960+0000 7f419d1d8740 1 --2- 172.21.15.19:0/1210594297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29720 0x5644d3b27350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.960+0000 7f419d1d8740 1 --2- 172.21.15.19:0/1210594297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b1ce20 0x5644d3b47db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.960+0000 7f419d1d8740 1 -- 172.21.15.19:0/1210594297 >> 172.21.15.19:0/1210594297 conn(0x5644d3afcc00 msgr2=0x5644d3b56cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:47.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.960+0000 7f419d1d8740 1 -- 172.21.15.19:0/1210594297 shutdown_connections 2024-03-28T13:16:47.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.960+0000 7f419d1d8740 1 -- 172.21.15.19:0/1210594297 wait complete. 2024-03-28T13:16:47.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.961+0000 7f419d1d8740 1 -- start start 2024-03-28T13:16:47.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.961+0000 7f419d1d8740 1 Processor -- start 2024-03-28T13:16:47.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.962+0000 7f419d1d8740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b1ce20 0x5644d3bc09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.962+0000 7f419d1d8740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29720 0x5644d3b1c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.962+0000 7f419d1d8740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b1c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.962+0000 7f419d1d8740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f419415f070 con 0x5644d3b29b00 2024-03-28T13:16:47.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.962+0000 7f419d1d8740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5644d3a637e0 con 0x5644d3b1ce20 2024-03-28T13:16:47.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.962+0000 7f419d1d8740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5644d3bc10d0 con 0x5644d3b29720 2024-03-28T13:16:47.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.962+0000 7f419d1d8740 10 client.0 _ll_register_callbacks cb 0x7f4184005ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:16:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.962+0000 7f4193e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b1c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.962+0000 7f4193e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b1c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52714/0 (socket says 172.21.15.19:52714) 2024-03-28T13:16:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.962+0000 7f4193e00640 1 -- 172.21.15.19:0/3313390084 learned_addr learned my addr 172.21.15.19:0/3313390084 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:16:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.963+0000 7f418be00640 1 --2- 172.21.15.19:0/3313390084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29720 0x5644d3b1c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.963+0000 7f4193e00640 1 -- 172.21.15.19:0/3313390084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29720 msgr2=0x5644d3b1c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.963+0000 7f4193400640 1 --2- 172.21.15.19:0/3313390084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b1ce20 0x5644d3bc09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.963+0000 7f4193e00640 1 --2- 172.21.15.19:0/3313390084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29720 0x5644d3b1c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.963+0000 7f4193e00640 1 -- 172.21.15.19:0/3313390084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b1ce20 msgr2=0x5644d3bc09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:16:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.963+0000 7f4193e00640 1 --2- 172.21.15.19:0/3313390084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b1ce20 0x5644d3bc09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.963+0000 7f4193e00640 1 -- 172.21.15.19:0/3313390084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644d3b280c0 con 0x5644d3b29b00 2024-03-28T13:16:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.963+0000 7f418be00640 1 --2- 172.21.15.19:0/3313390084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29720 0x5644d3b1c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:16:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.963+0000 7f4193e00640 1 --2- 172.21.15.19:0/3313390084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b1c830 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f418c055900 tx=0x7f418c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ab828f4966a4fa server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:16:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.963+0000 7f4180c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:16:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.963+0000 7f4180c00640 1 -- 172.21.15.19:0/3313390084 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c07d060 con 0x5644d3b29b00 2024-03-28T13:16:47.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.963+0000 7f4180c00640 1 -- 172.21.15.19:0/3313390084 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f418c055b10 con 0x5644d3b29b00 2024-03-28T13:16:47.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.963+0000 7f419d1d8740 1 -- 172.21.15.19:0/3313390084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f418c088070 con 0x5644d3b29b00 2024-03-28T13:16:47.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.964+0000 7f4180c00640 1 -- 172.21.15.19:0/3313390084 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f418c083050 con 0x5644d3b29b00 2024-03-28T13:16:47.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.964+0000 7f419d1d8740 10 client.7497 fetch_fsmap learned FSMap version 67 2024-03-28T13:16:47.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.964+0000 7f419d1d8740 1 -- 172.21.15.19:0/3313390084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x5644d3b47db0 con 0x5644d3b29b00 2024-03-28T13:16:47.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.964+0000 7f4180c00640 1 -- 172.21.15.19:0/3313390084 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f418c088070 con 0x5644d3b29b00 2024-03-28T13:16:47.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.964+0000 7f4180c00640 1 -- 172.21.15.19:0/3313390084 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f418c061c80 con 0x5644d3b29b00 2024-03-28T13:16:47.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.964+0000 7f419d1d8740 10 client.7497 fetch_fsmap finished waiting for FSMap version 67 2024-03-28T13:16:47.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.964+0000 7f419d1d8740 10 client.7497 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-28T13:16:47.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.964+0000 7f419d1d8740 20 client.7497 populate_metadata read hostname 'smithi019' 2024-03-28T13:16:47.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.964+0000 7f419d1d8740 1 --2- 172.21.15.19:0/3313390084 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5644d3bc4520 0x5644d3bc4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:16:47.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.964+0000 7f419d1d8740 4 client.7497 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:16:47.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.964+0000 7f419d1d8740 1 -- 172.21.15.19:0/3313390084 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5644d3a6ef40 con 0x5644d3bc4520 2024-03-28T13:16:47.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.965+0000 7f4193400640 1 --2- 172.21.15.19:0/3313390084 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5644d3bc4520 0x5644d3bc4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:16:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.965+0000 7f4193400640 1 --2- 172.21.15.19:0/3313390084 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5644d3bc4520 0x5644d3bc4930 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a62127ecfd1b905 server_cookie=4b1b8eed501beb25 in_seq=0 out_seq=0 2024-03-28T13:16:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.965+0000 7f4180c00640 10 client.7497 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:16:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.966+0000 7f4180c00640 1 -- 172.21.15.19:0/3313390084 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+256 (crc 0 0 0) 0x5644d3a6ef40 con 0x5644d3bc4520 2024-03-28T13:16:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.966+0000 7f4180c00640 10 client.7497 handle_command_reply: tid=0 2024-03-28T13:16:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.966+0000 7f419d1d8740 1 client.7497 shutdown 2024-03-28T13:16:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.966+0000 7f419d1d8740 1 -- 172.21.15.19:0/3313390084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 msgr2=0x5644d3b1c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.966+0000 7f419d1d8740 1 --2- 172.21.15.19:0/3313390084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b1c830 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f418c055900 tx=0x7f418c001250 comp rx=0 tx=0).stop 2024-03-28T13:16:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.966+0000 7f419d1d8740 1 -- 172.21.15.19:0/3313390084 shutdown_connections 2024-03-28T13:16:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.966+0000 7f419d1d8740 1 --2- 172.21.15.19:0/3313390084 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5644d3bc4520 0x5644d3bc4930 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.966+0000 7f419d1d8740 1 --2- 172.21.15.19:0/3313390084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5644d3b29b00 0x5644d3b1c830 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.966+0000 7f419d1d8740 1 --2- 172.21.15.19:0/3313390084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5644d3b29720 0x5644d3b1c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.966+0000 7f419d1d8740 1 --2- 172.21.15.19:0/3313390084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5644d3b1ce20 0x5644d3bc09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.966+0000 7f4180c00640 0 client.7497 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:16:47.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.966+0000 7f419d1d8740 1 -- 172.21.15.19:0/3313390084 >> 172.21.15.19:0/3313390084 conn(0x5644d3afcc00 msgr2=0x5644d3b4ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:47.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.966+0000 7f419d1d8740 1 -- 172.21.15.19:0/3313390084 shutdown_connections 2024-03-28T13:16:47.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.966+0000 7f419d1d8740 1 -- 172.21.15.19:0/3313390084 wait complete. 2024-03-28T13:16:47.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.967+0000 7f419d1d8740 20 client.7497 trim_cache size 0 max 16384 2024-03-28T13:16:47.968 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:16:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T13:16:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-03-28T13:16:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "b7ec899d-e435-4456-a1da-7c3dc82479e4": { 2024-03-28T13:16:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/dir", 2024-03-28T13:16:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "b7ec899d-e435-4456-a1da-7c3dc82479e4", 2024-03-28T13:16:47.968 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive" 2024-03-28T13:16:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:47.969 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:16:47.969 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:16:47.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.967+0000 7f4198e00640 1 -- 172.21.15.19:0/3320612338 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f417006c530 msgr2=0x7f417006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.967+0000 7f4198e00640 1 --2- 172.21.15.19:0/3320612338 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f417006c530 0x7f417006e9f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f41840011e0 tx=0x7f4184009040 comp rx=0 tx=0).stop 2024-03-28T13:16:47.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.967+0000 7f4198e00640 1 -- 172.21.15.19:0/3320612338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419413f190 msgr2=0x7f419412d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:16:47.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.967+0000 7f4198e00640 1 --2- 172.21.15.19:0/3320612338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419413f190 0x7f419412d0e0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f418c00c6b0 tx=0x7f418c00cb70 comp rx=0 tx=0).stop 2024-03-28T13:16:47.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.967+0000 7f4198e00640 1 -- 172.21.15.19:0/3320612338 shutdown_connections 2024-03-28T13:16:47.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.967+0000 7f4198e00640 1 --2- 172.21.15.19:0/3320612338 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f417006c530 0x7f417006e9f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.967+0000 7f4198e00640 1 --2- 172.21.15.19:0/3320612338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f419413f190 0x7f419412d0e0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.967+0000 7f4198e00640 1 --2- 172.21.15.19:0/3320612338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41940cc880 0x7f4194072b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.967+0000 7f4198e00640 1 --2- 172.21.15.19:0/3320612338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f419406d620 0x7f41940725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:16:47.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.967+0000 7f4198e00640 1 -- 172.21.15.19:0/3320612338 >> 172.21.15.19:0/3320612338 conn(0x7f41940b1ba0 msgr2=0x7f41940b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:16:47.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.967+0000 7f4198e00640 1 -- 172.21.15.19:0/3320612338 shutdown_connections 2024-03-28T13:16:47.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:16:47.968+0000 7f4198e00640 1 -- 172.21.15.19:0/3320612338 wait complete. 2024-03-28T13:16:47.983 INFO:tasks.cephfs.filesystem:scrub status for tag:b7ec899d-e435-4456-a1da-7c3dc82479e4 - {'path': '/dir', 'tag': 'b7ec899d-e435-4456-a1da-7c3dc82479e4', 'options': 'recursive'} 2024-03-28T13:16:50.558 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:16:50.559+0000 7f6b77600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-28T13:17:03.284 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:17:03.292 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:17:17.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:17:17.984 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-28T13:17:18.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.221+0000 7f070ba00640 1 Processor -- start 2024-03-28T13:17:18.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.222+0000 7f070ba00640 1 -- start start 2024-03-28T13:17:18.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.223+0000 7f070ba00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0704153580 0x7f07040d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.223+0000 7f070ba00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040af4b0 0x7f07041600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.223+0000 7f070ba00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07040afab0 con 0x7f07040d07c0 2024-03-28T13:17:18.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.223+0000 7f070ba00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f07040d0c30 con 0x7f0704153580 2024-03-28T13:17:18.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.223+0000 7f070ba00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f070415cc80 con 0x7f07040af4b0 2024-03-28T13:17:18.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.224+0000 7f0709000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0704153580 0x7f07040d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.224+0000 7f0709000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0704153580 0x7f07040d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:32788/0 (socket says 172.21.15.19:32788) 2024-03-28T13:17:18.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.224+0000 7f0709000640 1 -- 172.21.15.19:0/1306159332 learned_addr learned my addr 172.21.15.19:0/1306159332 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:18.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.225+0000 7f0709a00640 1 --2- 172.21.15.19:0/1306159332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040af4b0 0x7f07041600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.225+0000 7f0709000640 1 -- 172.21.15.19:0/1306159332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040af4b0 msgr2=0x7f07041600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:18.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.225+0000 7f0709000640 1 --2- 172.21.15.19:0/1306159332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040af4b0 0x7f07041600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.225+0000 7f0709000640 1 -- 172.21.15.19:0/1306159332 >> v1:172.21.15.188:6789/0 conn(0x7f07040d07c0 legacy=0x7f07040aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.225+0000 7f0709000640 1 -- 172.21.15.19:0/1306159332 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070415da00 con 0x7f0704153580 2024-03-28T13:17:18.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.226+0000 7f0709000640 1 --2- 172.21.15.19:0/1306159332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0704153580 0x7f07040d0280 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009870 tx=0x7f06f402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a497af05327080c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:18.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.227+0000 7f0703400640 1 -- 172.21.15.19:0/1306159332 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4035020 con 0x7f0704153580 2024-03-28T13:17:18.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.227+0000 7f0703400640 1 -- 172.21.15.19:0/1306159332 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f4031640 con 0x7f0704153580 2024-03-28T13:17:18.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.227+0000 7f0703400640 1 -- 172.21.15.19:0/1306159332 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f402d040 con 0x7f0704153580 2024-03-28T13:17:18.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.228+0000 7f070ba00640 1 -- 172.21.15.19:0/1306159332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0704153580 msgr2=0x7f07040d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.228+0000 7f070ba00640 1 --2- 172.21.15.19:0/1306159332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0704153580 0x7f07040d0280 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009870 tx=0x7f06f402b8f0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.228+0000 7f070ba00640 1 -- 172.21.15.19:0/1306159332 shutdown_connections 2024-03-28T13:17:18.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.228+0000 7f070ba00640 1 --2- 172.21.15.19:0/1306159332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040af4b0 0x7f07041600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.228+0000 7f070ba00640 1 --2- 172.21.15.19:0/1306159332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0704153580 0x7f07040d0280 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.228+0000 7f070ba00640 1 -- 172.21.15.19:0/1306159332 >> 172.21.15.19:0/1306159332 conn(0x7f070406b750 msgr2=0x7f070406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:18.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.228+0000 7f070ba00640 1 -- 172.21.15.19:0/1306159332 shutdown_connections 2024-03-28T13:17:18.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.228+0000 7f070ba00640 1 -- 172.21.15.19:0/1306159332 wait complete. 2024-03-28T13:17:18.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.230+0000 7f070ba00640 1 Processor -- start 2024-03-28T13:17:18.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.230+0000 7f070ba00640 1 -- start start 2024-03-28T13:17:18.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.231+0000 7f070ba00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040af4b0 0x7f07040b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.231+0000 7f070ba00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07040d07c0 0x7f0704071e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.231+0000 7f070ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07040b2930 0x7f0704072390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.231+0000 7f070ba00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f070415d4e0 con 0x7f07040b2930 2024-03-28T13:17:18.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.231+0000 7f070ba00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f070415d200 con 0x7f07040d07c0 2024-03-28T13:17:18.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.231+0000 7f070ba00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0704153960 con 0x7f07040af4b0 2024-03-28T13:17:18.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.231+0000 7f0709a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07040b2930 0x7f0704072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.231+0000 7f0709a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07040b2930 0x7f0704072390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42402/0 (socket says 172.21.15.19:42402) 2024-03-28T13:17:18.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.231+0000 7f0709a00640 1 -- 172.21.15.19:0/2921173563 learned_addr learned my addr 172.21.15.19:0/2921173563 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:18.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.232+0000 7f0709000640 1 --2- 172.21.15.19:0/2921173563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040af4b0 0x7f07040b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.232+0000 7f0703e00640 1 --2- 172.21.15.19:0/2921173563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07040d07c0 0x7f0704071e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.232+0000 7f0709a00640 1 -- 172.21.15.19:0/2921173563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040af4b0 msgr2=0x7f07040b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.232+0000 7f0709a00640 1 --2- 172.21.15.19:0/2921173563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040af4b0 0x7f07040b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.232+0000 7f0709a00640 1 -- 172.21.15.19:0/2921173563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07040d07c0 msgr2=0x7f0704071e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.232+0000 7f0709a00640 1 --2- 172.21.15.19:0/2921173563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07040d07c0 0x7f0704071e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.232+0000 7f0709a00640 1 -- 172.21.15.19:0/2921173563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0704158b70 con 0x7f07040b2930 2024-03-28T13:17:18.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.232+0000 7f0709a00640 1 --2- 172.21.15.19:0/2921173563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07040b2930 0x7f0704072390 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00efc0 tx=0x7f06fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:18.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.233+0000 7f0700c00640 1 -- 172.21.15.19:0/2921173563 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc018020 con 0x7f07040b2930 2024-03-28T13:17:18.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.233+0000 7f0700c00640 1 -- 172.21.15.19:0/2921173563 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06fc014480 con 0x7f07040b2930 2024-03-28T13:17:18.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.233+0000 7f0700c00640 1 -- 172.21.15.19:0/2921173563 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc014870 con 0x7f07040b2930 2024-03-28T13:17:18.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.233+0000 7f070ba00640 1 -- 172.21.15.19:0/2921173563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0704158e50 con 0x7f07040b2930 2024-03-28T13:17:18.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.233+0000 7f070ba00640 1 -- 172.21.15.19:0/2921173563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0704159410 con 0x7f07040b2930 2024-03-28T13:17:18.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.235+0000 7f0700c00640 1 -- 172.21.15.19:0/2921173563 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f06fc005d10 con 0x7f07040b2930 2024-03-28T13:17:18.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.236+0000 7f0700c00640 1 --2- 172.21.15.19:0/2921173563 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f06e406c530 0x7f06e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.236+0000 7f0700c00640 1 -- 172.21.15.19:0/2921173563 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f06fc08c240 con 0x7f07040b2930 2024-03-28T13:17:18.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.237+0000 7f0709000640 1 --2- 172.21.15.19:0/2921173563 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f06e406c530 0x7f06e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.237+0000 7f070e76c740 1 Processor -- start 2024-03-28T13:17:18.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.237+0000 7f070e76c740 1 -- start start 2024-03-28T13:17:18.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.238+0000 7f0709000640 1 --2- 172.21.15.19:0/2921173563 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f06e406c530 0x7f06e406e9f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f06f4036000 tx=0x7f06f403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:18.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.238+0000 7f070e76c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9560 0x562b8dccc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.238+0000 7f070e76c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9940 0x562b8dcfc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.238+0000 7f070e76c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dccca40 0x562b8dcfc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.238+0000 7f070e76c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b8dc376b0 con 0x562b8dccca40 2024-03-28T13:17:18.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.238+0000 7f070e76c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562b8dbdf750 con 0x562b8dcf9560 2024-03-28T13:17:18.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.238+0000 7f070e76c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x562b8dc33660 con 0x562b8dcf9940 2024-03-28T13:17:18.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.238+0000 7f0709a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dccca40 0x562b8dcfc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.238+0000 7f0709a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dccca40 0x562b8dcfc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42418/0 (socket says 172.21.15.19:42418) 2024-03-28T13:17:18.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.238+0000 7f0709a00640 1 -- 172.21.15.19:0/724112971 learned_addr learned my addr 172.21.15.19:0/724112971 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:18.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.239+0000 7f0709000640 1 --2- 172.21.15.19:0/724112971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9560 0x562b8dccc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.239+0000 7f0703e00640 1 --2- 172.21.15.19:0/724112971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9940 0x562b8dcfc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.239+0000 7f0709a00640 1 -- 172.21.15.19:0/724112971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9940 msgr2=0x562b8dcfc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:18.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.239+0000 7f0709a00640 1 --2- 172.21.15.19:0/724112971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9940 0x562b8dcfc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.239+0000 7f0709a00640 1 -- 172.21.15.19:0/724112971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9560 msgr2=0x562b8dccc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:18.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.239+0000 7f0709a00640 1 --2- 172.21.15.19:0/724112971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9560 0x562b8dccc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.239+0000 7f0709a00640 1 -- 172.21.15.19:0/724112971 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b8da157d0 con 0x562b8dccca40 2024-03-28T13:17:18.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.239+0000 7f0709a00640 1 --2- 172.21.15.19:0/724112971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dccca40 0x562b8dcfc830 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f06fc062710 tx=0x7f06fc05a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307c84bdbbec0c4f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:18.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.239+0000 7f06f9600640 1 -- 172.21.15.19:0/724112971 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc062c40 con 0x562b8dccca40 2024-03-28T13:17:18.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.239+0000 7f06f9600640 1 -- 172.21.15.19:0/724112971 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06fc058360 con 0x562b8dccca40 2024-03-28T13:17:18.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.239+0000 7f06f9600640 1 -- 172.21.15.19:0/724112971 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc05e070 con 0x562b8dccca40 2024-03-28T13:17:18.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.240+0000 7f070e76c740 1 -- 172.21.15.19:0/724112971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dccca40 msgr2=0x562b8dcfc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.240+0000 7f070e76c740 1 --2- 172.21.15.19:0/724112971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dccca40 0x562b8dcfc830 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f06fc062710 tx=0x7f06fc05a060 comp rx=0 tx=0).stop 2024-03-28T13:17:18.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.240+0000 7f070e76c740 1 -- 172.21.15.19:0/724112971 shutdown_connections 2024-03-28T13:17:18.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.240+0000 7f070e76c740 1 --2- 172.21.15.19:0/724112971 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dccca40 0x562b8dcfc830 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.240+0000 7f070e76c740 1 --2- 172.21.15.19:0/724112971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9940 0x562b8dcfc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.240+0000 7f070e76c740 1 --2- 172.21.15.19:0/724112971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9560 0x562b8dccc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.240+0000 7f070e76c740 1 -- 172.21.15.19:0/724112971 >> 172.21.15.19:0/724112971 conn(0x562b8dcce770 msgr2=0x562b8dd172f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:18.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.240+0000 7f070e76c740 1 -- 172.21.15.19:0/724112971 shutdown_connections 2024-03-28T13:17:18.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.240+0000 7f070e76c740 1 -- 172.21.15.19:0/724112971 wait complete. 2024-03-28T13:17:18.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.241+0000 7f070e76c740 1 -- start start 2024-03-28T13:17:18.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.241+0000 7f070e76c740 1 Processor -- start 2024-03-28T13:17:18.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.242+0000 7f070e76c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf9560 0x562b8dd00c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.242+0000 7f070e76c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9940 0x562b8dcfb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.242+0000 7f070e76c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf7520 0x562b8dd173d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.242+0000 7f070e76c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b8dcd0e10 con 0x562b8dcf9560 2024-03-28T13:17:18.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.242+0000 7f070e76c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x562b8dbf2a80 con 0x562b8dcf9940 2024-03-28T13:17:18.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.242+0000 7f070e76c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x562b8dcd3650 con 0x562b8dcf7520 2024-03-28T13:17:18.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.242+0000 7f070e76c740 10 client.0 _ll_register_callbacks cb 0x562b8dbf2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:17:18.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.242+0000 7f0709000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf9560 0x562b8dd00c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.243+0000 7f0709000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf9560 0x562b8dd00c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42432/0 (socket says 172.21.15.19:42432) 2024-03-28T13:17:18.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.243+0000 7f0709000640 1 -- 172.21.15.19:0/481950786 learned_addr learned my addr 172.21.15.19:0/481950786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:18.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.243+0000 7f0703e00640 1 --2- 172.21.15.19:0/481950786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9940 0x562b8dcfb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.243+0000 7f0709a00640 1 --2- 172.21.15.19:0/481950786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf7520 0x562b8dd173d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.243+0000 7f0709000640 1 -- 172.21.15.19:0/481950786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf7520 msgr2=0x562b8dd173d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:18.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.243+0000 7f0709000640 1 --2- 172.21.15.19:0/481950786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf7520 0x562b8dd173d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.243+0000 7f0709000640 1 -- 172.21.15.19:0/481950786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9940 msgr2=0x562b8dcfb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:18.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.243+0000 7f0709000640 1 --2- 172.21.15.19:0/481950786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9940 0x562b8dcfb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.243+0000 7f0709000640 1 -- 172.21.15.19:0/481950786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b8d94b3f0 con 0x562b8dcf9560 2024-03-28T13:17:18.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.243+0000 7f0709000640 1 --2- 172.21.15.19:0/481950786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf9560 0x562b8dd00c20 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009990 tx=0x7f06f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3019ce0f1c0eec9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:18.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.243+0000 7f06ee000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:17:18.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.244+0000 7f06ee000640 1 -- 172.21.15.19:0/481950786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4052060 con 0x562b8dcf9560 2024-03-28T13:17:18.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.244+0000 7f06ee000640 1 -- 172.21.15.19:0/481950786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f06f4041070 con 0x562b8dcf9560 2024-03-28T13:17:18.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.244+0000 7f070e76c740 1 -- 172.21.15.19:0/481950786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06fc058360 con 0x562b8dcf9560 2024-03-28T13:17:18.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.244+0000 7f06ee000640 1 -- 172.21.15.19:0/481950786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f06f4035040 con 0x562b8dcf9560 2024-03-28T13:17:18.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.244+0000 7f070e76c740 10 client.7506 fetch_fsmap learned FSMap version 68 2024-03-28T13:17:18.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.244+0000 7f070e76c740 1 -- 172.21.15.19:0/481950786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x562b8d90b0f0 con 0x562b8dcf9560 2024-03-28T13:17:18.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.244+0000 7f06ee000640 1 -- 172.21.15.19:0/481950786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06fc058360 con 0x562b8dcf9560 2024-03-28T13:17:18.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.244+0000 7f06ee000640 1 -- 172.21.15.19:0/481950786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f06f404c070 con 0x562b8dcf9560 2024-03-28T13:17:18.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.244+0000 7f070e76c740 10 client.7506 fetch_fsmap finished waiting for FSMap version 68 2024-03-28T13:17:18.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.244+0000 7f070e76c740 10 client.7506 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-28T13:17:18.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.244+0000 7f070e76c740 20 client.7506 populate_metadata read hostname 'smithi019' 2024-03-28T13:17:18.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.244+0000 7f070e76c740 1 --2- 172.21.15.19:0/481950786 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x562b8dcffba0 0x562b8dcfffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.244+0000 7f070e76c740 4 client.7506 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:17:18.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.244+0000 7f070e76c740 1 -- 172.21.15.19:0/481950786 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b8dc44bf0 con 0x562b8dcffba0 2024-03-28T13:17:18.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.245+0000 7f0703e00640 1 --2- 172.21.15.19:0/481950786 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x562b8dcffba0 0x562b8dcfffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.246+0000 7f0703e00640 1 --2- 172.21.15.19:0/481950786 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x562b8dcffba0 0x562b8dcfffd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba7a855a99aae9c7 server_cookie=f3fe2e96195405c5 in_seq=0 out_seq=0 2024-03-28T13:17:18.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.246+0000 7f06ee000640 10 client.7506 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:18.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.248+0000 7f06ee000640 1 -- 172.21.15.19:0/481950786 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b8dc44bf0 con 0x562b8dcffba0 2024-03-28T13:17:18.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.248+0000 7f06ee000640 10 client.7506 handle_command_reply: tid=0 2024-03-28T13:17:18.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.248+0000 7f070e76c740 1 client.7506 shutdown 2024-03-28T13:17:18.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.248+0000 7f070e76c740 1 -- 172.21.15.19:0/481950786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf9560 msgr2=0x562b8dd00c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.248+0000 7f070e76c740 1 --2- 172.21.15.19:0/481950786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf9560 0x562b8dd00c20 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009990 tx=0x7f06f4004730 comp rx=0 tx=0).stop 2024-03-28T13:17:18.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.249+0000 7f070e76c740 1 -- 172.21.15.19:0/481950786 shutdown_connections 2024-03-28T13:17:18.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.249+0000 7f070e76c740 1 --2- 172.21.15.19:0/481950786 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x562b8dcffba0 0x562b8dcfffd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.249+0000 7f070e76c740 1 --2- 172.21.15.19:0/481950786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf7520 0x562b8dd173d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.249+0000 7f070e76c740 1 --2- 172.21.15.19:0/481950786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9940 0x562b8dcfb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.249+0000 7f070e76c740 1 --2- 172.21.15.19:0/481950786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf9560 0x562b8dd00c20 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.249+0000 7f06ee000640 0 client.7506 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:18.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.249+0000 7f070e76c740 1 -- 172.21.15.19:0/481950786 >> 172.21.15.19:0/481950786 conn(0x562b8dccca40 msgr2=0x562b8dd1a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:18.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.249+0000 7f070e76c740 1 -- 172.21.15.19:0/481950786 shutdown_connections 2024-03-28T13:17:18.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.249+0000 7f070e76c740 1 -- 172.21.15.19:0/481950786 wait complete. 2024-03-28T13:17:18.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.249+0000 7f070e76c740 20 client.7506 trim_cache size 0 max 16384 2024-03-28T13:17:18.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.268+0000 7f070e76c740 1 Processor -- start 2024-03-28T13:17:18.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.268+0000 7f070e76c740 1 -- start start 2024-03-28T13:17:18.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.269+0000 7f070e76c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf7520 0x562b8dcfc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.269+0000 7f070e76c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9560 0x562b8dd1a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.269+0000 7f070e76c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf9940 0x562b8dd15c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.269+0000 7f070e76c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0704201240 con 0x562b8dcf9940 2024-03-28T13:17:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.269+0000 7f070e76c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0704200400 con 0x562b8dcf7520 2024-03-28T13:17:18.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.269+0000 7f070e76c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0704157fd0 con 0x562b8dcf9560 2024-03-28T13:17:18.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.269+0000 7f0709a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf9940 0x562b8dd15c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.269+0000 7f0709a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf9940 0x562b8dd15c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42438/0 (socket says 172.21.15.19:42438) 2024-03-28T13:17:18.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.269+0000 7f0709a00640 1 -- 172.21.15.19:0/1344369791 learned_addr learned my addr 172.21.15.19:0/1344369791 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:18.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.270+0000 7f0709a00640 1 -- 172.21.15.19:0/1344369791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9560 msgr2=0x562b8dd1a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:18.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.270+0000 7f0709000640 1 --2- 172.21.15.19:0/1344369791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf7520 0x562b8dcfc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.270+0000 7f0703e00640 1 --2- 172.21.15.19:0/1344369791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9560 0x562b8dd1a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.270+0000 7f0709a00640 1 --2- 172.21.15.19:0/1344369791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9560 0x562b8dd1a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.270+0000 7f0709a00640 1 -- 172.21.15.19:0/1344369791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf7520 msgr2=0x562b8dcfc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:18.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.270+0000 7f0709a00640 1 --2- 172.21.15.19:0/1344369791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf7520 0x562b8dcfc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.270+0000 7f0709a00640 1 -- 172.21.15.19:0/1344369791 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b8dd3ba70 con 0x562b8dcf9940 2024-03-28T13:17:18.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.270+0000 7f0709000640 1 --2- 172.21.15.19:0/1344369791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf7520 0x562b8dcfc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:18.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.270+0000 7f0709a00640 1 --2- 172.21.15.19:0/1344369791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf9940 0x562b8dd15c40 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f06fc05a910 tx=0x7f06fc08b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a895340f14cb0a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:18.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.270+0000 7f06f8c00640 1 -- 172.21.15.19:0/1344369791 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc062c40 con 0x562b8dcf9940 2024-03-28T13:17:18.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.270+0000 7f06f8c00640 1 -- 172.21.15.19:0/1344369791 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06fc058360 con 0x562b8dcf9940 2024-03-28T13:17:18.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.270+0000 7f06f8c00640 1 -- 172.21.15.19:0/1344369791 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc0566d0 con 0x562b8dcf9940 2024-03-28T13:17:18.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.271+0000 7f070e76c740 1 -- 172.21.15.19:0/1344369791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf9940 msgr2=0x562b8dd15c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.271+0000 7f070e76c740 1 --2- 172.21.15.19:0/1344369791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf9940 0x562b8dd15c40 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f06fc05a910 tx=0x7f06fc08b000 comp rx=0 tx=0).stop 2024-03-28T13:17:18.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.271+0000 7f070e76c740 1 -- 172.21.15.19:0/1344369791 shutdown_connections 2024-03-28T13:17:18.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.271+0000 7f070e76c740 1 --2- 172.21.15.19:0/1344369791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf9940 0x562b8dd15c40 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.271+0000 7f070e76c740 1 --2- 172.21.15.19:0/1344369791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9560 0x562b8dd1a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.271+0000 7f070e76c740 1 --2- 172.21.15.19:0/1344369791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf7520 0x562b8dcfc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.271+0000 7f070e76c740 1 -- 172.21.15.19:0/1344369791 >> 172.21.15.19:0/1344369791 conn(0x562b8dccca40 msgr2=0x562b8dcfff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:18.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.271+0000 7f070e76c740 1 -- 172.21.15.19:0/1344369791 shutdown_connections 2024-03-28T13:17:18.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.271+0000 7f070e76c740 1 -- 172.21.15.19:0/1344369791 wait complete. 2024-03-28T13:17:18.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.272+0000 7f070e76c740 1 -- start start 2024-03-28T13:17:18.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.272+0000 7f070e76c740 1 Processor -- start 2024-03-28T13:17:18.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.273+0000 7f070e76c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf7520 0x562b8dd87d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.273+0000 7f070e76c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9560 0x562b8dd2cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.273+0000 7f070e76c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9940 0x562b8dd2ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.273+0000 7f070e76c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b8dcfc2e0 con 0x562b8dcf7520 2024-03-28T13:17:18.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.273+0000 7f070e76c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07040ce5a0 con 0x562b8dcf9560 2024-03-28T13:17:18.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.273+0000 7f070e76c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x562b8dd2f670 con 0x562b8dcf9940 2024-03-28T13:17:18.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.273+0000 7f070e76c740 10 client.0 _ll_register_callbacks cb 0x7f06f40319e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:17:18.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.273+0000 7f0709000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf7520 0x562b8dd87d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.274+0000 7f0709000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf7520 0x562b8dd87d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42440/0 (socket says 172.21.15.19:42440) 2024-03-28T13:17:18.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.274+0000 7f0709000640 1 -- 172.21.15.19:0/1168644611 learned_addr learned my addr 172.21.15.19:0/1168644611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:18.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.274+0000 7f0703e00640 1 --2- 172.21.15.19:0/1168644611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9560 0x562b8dd2cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.274+0000 7f0709a00640 1 --2- 172.21.15.19:0/1168644611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9940 0x562b8dd2ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.274+0000 7f0709000640 1 -- 172.21.15.19:0/1168644611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9940 msgr2=0x562b8dd2ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:18.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.274+0000 7f0709000640 1 --2- 172.21.15.19:0/1168644611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9940 0x562b8dd2ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.274+0000 7f0709000640 1 -- 172.21.15.19:0/1168644611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9560 msgr2=0x562b8dd2cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:18.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.274+0000 7f0709000640 1 --2- 172.21.15.19:0/1168644611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9560 0x562b8dd2cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.274+0000 7f0709000640 1 -- 172.21.15.19:0/1168644611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b8dce4b90 con 0x562b8dcf7520 2024-03-28T13:17:18.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.274+0000 7f0709a00640 1 --2- 172.21.15.19:0/1168644611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9940 0x562b8dd2ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:18.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.274+0000 7f0703e00640 1 --2- 172.21.15.19:0/1168644611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9560 0x562b8dd2cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:18.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.274+0000 7f0709000640 1 --2- 172.21.15.19:0/1168644611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf7520 0x562b8dd87d10 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f06f402b930 tx=0x7f06f4031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942cf638d700bb03 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:18.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.274+0000 7f06ee000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:17:18.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.274+0000 7f06ee000640 1 -- 172.21.15.19:0/1168644611 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f404db40 con 0x562b8dcf7520 2024-03-28T13:17:18.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.275+0000 7f06ee000640 1 -- 172.21.15.19:0/1168644611 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f06f404dce0 con 0x562b8dcf7520 2024-03-28T13:17:18.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.275+0000 7f070e76c740 1 -- 172.21.15.19:0/1168644611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06fc058360 con 0x562b8dcf7520 2024-03-28T13:17:18.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.275+0000 7f06ee000640 1 -- 172.21.15.19:0/1168644611 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f06f403a7f0 con 0x562b8dcf7520 2024-03-28T13:17:18.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.275+0000 7f070e76c740 10 client.7512 fetch_fsmap learned FSMap version 68 2024-03-28T13:17:18.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.275+0000 7f070e76c740 1 -- 172.21.15.19:0/1168644611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x562b8dd01c50 con 0x562b8dcf7520 2024-03-28T13:17:18.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.275+0000 7f06ee000640 1 -- 172.21.15.19:0/1168644611 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06fc058360 con 0x562b8dcf7520 2024-03-28T13:17:18.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.275+0000 7f06ee000640 1 -- 172.21.15.19:0/1168644611 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f06f404c040 con 0x562b8dcf7520 2024-03-28T13:17:18.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.275+0000 7f070e76c740 10 client.7512 fetch_fsmap finished waiting for FSMap version 68 2024-03-28T13:17:18.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.275+0000 7f070e76c740 10 client.7512 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-28T13:17:18.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.275+0000 7f070e76c740 20 client.7512 populate_metadata read hostname 'smithi019' 2024-03-28T13:17:18.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.275+0000 7f070e76c740 1 --2- 172.21.15.19:0/1168644611 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x562b8dd2f1c0 0x562b8dce3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.275+0000 7f070e76c740 4 client.7512 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:17:18.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.275+0000 7f070e76c740 1 -- 172.21.15.19:0/1168644611 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562b8dc3ef40 con 0x562b8dd2f1c0 2024-03-28T13:17:18.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.276+0000 7f0703e00640 1 --2- 172.21.15.19:0/1168644611 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x562b8dd2f1c0 0x562b8dce3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.277+0000 7f0703e00640 1 --2- 172.21.15.19:0/1168644611 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x562b8dd2f1c0 0x562b8dce3dc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67475c87defbc7d9 server_cookie=734b3da3d44c7bea in_seq=0 out_seq=0 2024-03-28T13:17:18.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.277+0000 7f06ee000640 10 client.7512 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:18.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.277+0000 7f06ee000640 1 -- 172.21.15.19:0/1168644611 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562b8dc3ef40 con 0x562b8dd2f1c0 2024-03-28T13:17:18.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.277+0000 7f06ee000640 10 client.7512 handle_command_reply: tid=0 2024-03-28T13:17:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.277+0000 7f070e76c740 1 client.7512 shutdown 2024-03-28T13:17:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.278+0000 7f070e76c740 1 -- 172.21.15.19:0/1168644611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf7520 msgr2=0x562b8dd87d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.278+0000 7f070e76c740 1 --2- 172.21.15.19:0/1168644611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf7520 0x562b8dd87d10 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f06f402b930 tx=0x7f06f4031000 comp rx=0 tx=0).stop 2024-03-28T13:17:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.278+0000 7f070e76c740 1 -- 172.21.15.19:0/1168644611 shutdown_connections 2024-03-28T13:17:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.278+0000 7f070e76c740 1 --2- 172.21.15.19:0/1168644611 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x562b8dd2f1c0 0x562b8dce3dc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.278+0000 7f070e76c740 1 --2- 172.21.15.19:0/1168644611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562b8dcf9940 0x562b8dd2ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.278+0000 7f070e76c740 1 --2- 172.21.15.19:0/1168644611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562b8dcf9560 0x562b8dd2cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.278+0000 7f070e76c740 1 --2- 172.21.15.19:0/1168644611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b8dcf7520 0x562b8dd87d10 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.278+0000 7f06ee000640 0 client.7512 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.278+0000 7f070e76c740 1 -- 172.21.15.19:0/1168644611 >> 172.21.15.19:0/1168644611 conn(0x562b8dccca40 msgr2=0x562b8dcfff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.278+0000 7f070e76c740 1 -- 172.21.15.19:0/1168644611 shutdown_connections 2024-03-28T13:17:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.278+0000 7f070e76c740 1 -- 172.21.15.19:0/1168644611 wait complete. 2024-03-28T13:17:18.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.278+0000 7f070e76c740 20 client.7512 trim_cache size 0 max 16384 2024-03-28T13:17:18.282 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:17:18.282 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-03-28T13:17:18.282 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-03-28T13:17:18.282 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:17:18.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.279+0000 7f070ba00640 1 -- 172.21.15.19:0/2921173563 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f06e406c530 msgr2=0x7f06e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.279+0000 7f070ba00640 1 --2- 172.21.15.19:0/2921173563 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f06e406c530 0x7f06e406e9f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f06f4036000 tx=0x7f06f403c040 comp rx=0 tx=0).stop 2024-03-28T13:17:18.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.279+0000 7f070ba00640 1 -- 172.21.15.19:0/2921173563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07040b2930 msgr2=0x7f0704072390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.279+0000 7f070ba00640 1 --2- 172.21.15.19:0/2921173563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07040b2930 0x7f0704072390 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00efc0 tx=0x7f06fc00c490 comp rx=0 tx=0).stop 2024-03-28T13:17:18.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.279+0000 7f070ba00640 1 -- 172.21.15.19:0/2921173563 shutdown_connections 2024-03-28T13:17:18.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.279+0000 7f070ba00640 1 --2- 172.21.15.19:0/2921173563 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f06e406c530 0x7f06e406e9f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.279+0000 7f070ba00640 1 --2- 172.21.15.19:0/2921173563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07040b2930 0x7f0704072390 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.279+0000 7f070ba00640 1 --2- 172.21.15.19:0/2921173563 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07040d07c0 0x7f0704071e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.279+0000 7f070ba00640 1 --2- 172.21.15.19:0/2921173563 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040af4b0 0x7f07040b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.279+0000 7f070ba00640 1 -- 172.21.15.19:0/2921173563 >> 172.21.15.19:0/2921173563 conn(0x7f070406b750 msgr2=0x7f070415b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:18.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.279+0000 7f070ba00640 1 -- 172.21.15.19:0/2921173563 shutdown_connections 2024-03-28T13:17:18.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.279+0000 7f070ba00640 1 -- 172.21.15.19:0/2921173563 wait complete. 2024-03-28T13:17:18.296 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T13:17:18.296 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:17:18.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.505+0000 7f640f200640 1 Processor -- start 2024-03-28T13:17:18.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.505+0000 7f640f200640 1 -- start start 2024-03-28T13:17:18.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.506+0000 7f640f200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f640815fa10 0x7f64080b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.506+0000 7f640f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64080d2bf0 0x7f64080cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.506+0000 7f640f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64080d3140 con 0x7f64080d0c30 2024-03-28T13:17:18.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.507+0000 7f640f200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f64080d3420 con 0x7f640815fa10 2024-03-28T13:17:18.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.507+0000 7f640f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f64080cf2b0 con 0x7f64080d2bf0 2024-03-28T13:17:18.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.507+0000 7f640d200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f64080d0c30 0x7f64081603b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37100/0 (socket says 172.21.15.19:37100) 2024-03-28T13:17:18.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.507+0000 7f640d200640 1 -- 172.21.15.19:0/754844338 learned_addr learned my addr 172.21.15.19:0/754844338 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:17:18.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.507+0000 7f6407e00640 1 --2- 172.21.15.19:0/754844338 >> v2:172.21.15.188:3300/0 conn(0x7f640815fa10 0x7f64080b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.507+0000 7f6407400640 1 --2- 172.21.15.19:0/754844338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64080d2bf0 0x7f64080cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.507+0000 7f6406a00640 1 -- 172.21.15.19:0/754844338 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1644122515 0 0) 0x7f64080d3140 con 0x7f64080d0c30 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.507+0000 7f6406a00640 1 -- 172.21.15.19:0/754844338 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63ec003520 con 0x7f64080d0c30 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.508+0000 7f6406a00640 1 -- 172.21.15.19:0/754844338 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 366605026 0 0) 0x7f63ec003520 con 0x7f64080d0c30 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.508+0000 7f6406a00640 1 -- 172.21.15.19:0/754844338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64080d2bf0 msgr2=0x7f64080cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.508+0000 7f6406a00640 1 --2- 172.21.15.19:0/754844338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64080d2bf0 0x7f64080cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.508+0000 7f6406a00640 1 -- 172.21.15.19:0/754844338 >> v2:172.21.15.188:3300/0 conn(0x7f640815fa10 msgr2=0x7f64080b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.508+0000 7f6406a00640 1 --2- 172.21.15.19:0/754844338 >> v2:172.21.15.188:3300/0 conn(0x7f640815fa10 0x7f64080b55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.508+0000 7f6406a00640 1 -- 172.21.15.19:0/754844338 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64080cf590 con 0x7f64080d0c30 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.508+0000 7f6406a00640 1 -- 172.21.15.19:0/754844338 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f64000019f0 con 0x7f64080d0c30 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.508+0000 7f6406a00640 1 -- 172.21.15.19:0/754844338 >> v1:172.21.15.19:6789/0 conn(0x7f64080d0c30 legacy=0x7f64081603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.508+0000 7f6406a00640 1 --2- 172.21.15.19:0/754844338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63ec0039b0 0x7f63ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.508+0000 7f6406a00640 1 --2- 172.21.15.19:0/754844338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63ec004800 0x7f63ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.508+0000 7f6406a00640 1 --2- 172.21.15.19:0/754844338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64080d2bf0 0x7f63ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.508+0000 7f6406a00640 1 -- 172.21.15.19:0/754844338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f64080d3420 con 0x7f63ec004800 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.508+0000 7f6406a00640 1 -- 172.21.15.19:0/754844338 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f64080cf2b0 con 0x7f63ec0039b0 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.508+0000 7f6406a00640 1 -- 172.21.15.19:0/754844338 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f63ec00b9f0 con 0x7f64080d2bf0 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.508+0000 7f6407400640 1 --2- 172.21.15.19:0/754844338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63ec004800 0x7f63ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.509+0000 7f6407e00640 1 --2- 172.21.15.19:0/754844338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63ec0039b0 0x7f63ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.509+0000 7f6407400640 1 -- 172.21.15.19:0/754844338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64080d2bf0 msgr2=0x7f63ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:18.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.509+0000 7f6407400640 1 --2- 172.21.15.19:0/754844338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64080d2bf0 0x7f63ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.509+0000 7f6407400640 1 -- 172.21.15.19:0/754844338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63ec0039b0 msgr2=0x7f63ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:18.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.509+0000 7f6407400640 1 --2- 172.21.15.19:0/754844338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63ec0039b0 0x7f63ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.509+0000 7f6407400640 1 -- 172.21.15.19:0/754844338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec003520 con 0x7f63ec004800 2024-03-28T13:17:18.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.510+0000 7f6407400640 1 --2- 172.21.15.19:0/754844338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63ec004800 0x7f63ec004c80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009ab0 tx=0x7f63ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1f0511de93228f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:18.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.510+0000 7f6406a00640 1 -- 172.21.15.19:0/754844338 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4009680 con 0x7f63ec004800 2024-03-28T13:17:18.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.510+0000 7f6406a00640 1 -- 172.21.15.19:0/754844338 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f4005990 con 0x7f63ec004800 2024-03-28T13:17:18.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.510+0000 7f6406a00640 1 -- 172.21.15.19:0/754844338 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4005d80 con 0x7f63ec004800 2024-03-28T13:17:18.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.511+0000 7f640f200640 1 -- 172.21.15.19:0/754844338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63ec004800 msgr2=0x7f63ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.511+0000 7f640f200640 1 --2- 172.21.15.19:0/754844338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63ec004800 0x7f63ec004c80 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009ab0 tx=0x7f63ec005830 comp rx=0 tx=0).stop 2024-03-28T13:17:18.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.511+0000 7f640d200640 1 -- 172.21.15.19:0/754844338 reap_dead start 2024-03-28T13:17:18.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.511+0000 7f640f200640 1 -- 172.21.15.19:0/754844338 shutdown_connections 2024-03-28T13:17:18.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.511+0000 7f640f200640 1 -- 172.21.15.19:0/754844338 >> 172.21.15.19:0/754844338 conn(0x7f640808bfb0 msgr2=0x7f64080d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:18.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.511+0000 7f640f200640 1 -- 172.21.15.19:0/754844338 shutdown_connections 2024-03-28T13:17:18.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.511+0000 7f640f200640 1 -- 172.21.15.19:0/754844338 wait complete. 2024-03-28T13:17:18.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.513+0000 7f640f200640 1 Processor -- start 2024-03-28T13:17:18.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.513+0000 7f640f200640 1 -- start start 2024-03-28T13:17:18.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.514+0000 7f640f200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63ec004800 0x7f64080cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.514+0000 7f640f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64080ccd80 0x7f640806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.514+0000 7f640f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64080c9e20 0x7f64080ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.514+0000 7f640f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f64080b5c60 con 0x7f64080c9e20 2024-03-28T13:17:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.514+0000 7f640f200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f64080b5ae0 con 0x7f63ec004800 2024-03-28T13:17:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.514+0000 7f640f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f640815fdf0 con 0x7f64080ccd80 2024-03-28T13:17:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.514+0000 7f640d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64080c9e20 0x7f64080ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.514+0000 7f640d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64080c9e20 0x7f64080ca2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42454/0 (socket says 172.21.15.19:42454) 2024-03-28T13:17:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.514+0000 7f640d200640 1 -- 172.21.15.19:0/3051872051 learned_addr learned my addr 172.21.15.19:0/3051872051 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.515+0000 7f6407e00640 1 --2- 172.21.15.19:0/3051872051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63ec004800 0x7f64080cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.515+0000 7f6407400640 1 --2- 172.21.15.19:0/3051872051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64080ccd80 0x7f640806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.515+0000 7f640d200640 1 -- 172.21.15.19:0/3051872051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64080ccd80 msgr2=0x7f640806da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.515+0000 7f640d200640 1 --2- 172.21.15.19:0/3051872051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64080ccd80 0x7f640806da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.515+0000 7f640d200640 1 -- 172.21.15.19:0/3051872051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63ec004800 msgr2=0x7f64080cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.515+0000 7f640d200640 1 --2- 172.21.15.19:0/3051872051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63ec004800 0x7f64080cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.515+0000 7f640d200640 1 -- 172.21.15.19:0/3051872051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64080ca7e0 con 0x7f64080c9e20 2024-03-28T13:17:18.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.515+0000 7f640d200640 1 --2- 172.21.15.19:0/3051872051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64080c9e20 0x7f64080ca2a0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f64080cb4d0 tx=0x7f640000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:18.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.515+0000 7f63ffe00640 1 -- 172.21.15.19:0/3051872051 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640000a610 con 0x7f64080c9e20 2024-03-28T13:17:18.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.515+0000 7f63ffe00640 1 -- 172.21.15.19:0/3051872051 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f640000a7b0 con 0x7f64080c9e20 2024-03-28T13:17:18.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.515+0000 7f63ffe00640 1 -- 172.21.15.19:0/3051872051 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640000ab80 con 0x7f64080c9e20 2024-03-28T13:17:18.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.516+0000 7f640f200640 1 -- 172.21.15.19:0/3051872051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64080cd2f0 con 0x7f64080c9e20 2024-03-28T13:17:18.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.516+0000 7f640f200640 1 -- 172.21.15.19:0/3051872051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64080cd8b0 con 0x7f64080c9e20 2024-03-28T13:17:18.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.517+0000 7f640f200640 1 -- 172.21.15.19:0/3051872051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6408068cd0 con 0x7f64080c9e20 2024-03-28T13:17:18.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.518+0000 7f63ffe00640 1 -- 172.21.15.19:0/3051872051 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f640000ad20 con 0x7f64080c9e20 2024-03-28T13:17:18.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.519+0000 7f63ffe00640 1 --2- 172.21.15.19:0/3051872051 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f63e806c560 0x7f63e806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:18.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.519+0000 7f63ffe00640 1 -- 172.21.15.19:0/3051872051 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f640008acd0 con 0x7f64080c9e20 2024-03-28T13:17:18.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.520+0000 7f6407e00640 1 --2- 172.21.15.19:0/3051872051 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f63e806c560 0x7f63e806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:18.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.521+0000 7f6407e00640 1 --2- 172.21.15.19:0/3051872051 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f63e806c560 0x7f63e806ea20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009a90 tx=0x7f63f8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:18.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.524+0000 7f63ffe00640 1 -- 172.21.15.19:0/3051872051 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f640001a030 con 0x7f64080c9e20 2024-03-28T13:17:18.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.771+0000 7f640f200640 1 -- 172.21.15.19:0/3051872051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f640806d2d0 con 0x7f64080c9e20 2024-03-28T13:17:18.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.772+0000 7f63ffe00640 1 -- 172.21.15.19:0/3051872051 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5187 (secure 0 0 0) 0x7f6400059290 con 0x7f64080c9e20 2024-03-28T13:17:18.772 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:17:18.772 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:16:50.520332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6175":{"gid":6175,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6836/1749969053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1749969053},{"type":"v1","addr":"172.21.15.188:6836","nonce":1749969053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6175,"qdb_cluster":[6175]},"id":6}]} 2024-03-28T13:17:18.773 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 68 2024-03-28T13:17:18.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.775+0000 7f640f200640 1 -- 172.21.15.19:0/3051872051 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f63e806c560 msgr2=0x7f63e806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.775+0000 7f640f200640 1 --2- 172.21.15.19:0/3051872051 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f63e806c560 0x7f63e806ea20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009a90 tx=0x7f63f8008040 comp rx=0 tx=0).stop 2024-03-28T13:17:18.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.775+0000 7f640f200640 1 -- 172.21.15.19:0/3051872051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64080c9e20 msgr2=0x7f64080ca2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:18.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.775+0000 7f640f200640 1 --2- 172.21.15.19:0/3051872051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64080c9e20 0x7f64080ca2a0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f64080cb4d0 tx=0x7f640000ccd0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.775+0000 7f640f200640 1 -- 172.21.15.19:0/3051872051 shutdown_connections 2024-03-28T13:17:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.775+0000 7f640f200640 1 --2- 172.21.15.19:0/3051872051 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f63e806c560 0x7f63e806ea20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.775+0000 7f640f200640 1 --2- 172.21.15.19:0/3051872051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64080c9e20 0x7f64080ca2a0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.775+0000 7f640f200640 1 --2- 172.21.15.19:0/3051872051 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64080ccd80 0x7f640806da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.775+0000 7f640f200640 1 --2- 172.21.15.19:0/3051872051 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63ec004800 0x7f64080cc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:18.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.775+0000 7f640f200640 1 -- 172.21.15.19:0/3051872051 >> 172.21.15.19:0/3051872051 conn(0x7f640808bfb0 msgr2=0x7f64080b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:18.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.775+0000 7f640f200640 1 -- 172.21.15.19:0/3051872051 shutdown_connections 2024-03-28T13:17:18.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:18.775+0000 7f640f200640 1 -- 172.21.15.19:0/3051872051 wait complete. 2024-03-28T13:17:18.794 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-03-28T13:17:19.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.019+0000 7f939ec00640 1 Processor -- start 2024-03-28T13:17:19.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.019+0000 7f939ec00640 1 -- start start 2024-03-28T13:17:19.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.021+0000 7f939ec00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9398160360 0x7f93980713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.021+0000 7f939ec00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93980735c0 0x7f9398071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.021+0000 7f939ec00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9398162de0 0x7f93981663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.021+0000 7f939ec00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f93980b36d0 con 0x7f9398160360 2024-03-28T13:17:19.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.021+0000 7f939ec00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f93980b4530 con 0x7f9398162de0 2024-03-28T13:17:19.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.021+0000 7f939ec00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f93980b3920 con 0x7f93980735c0 2024-03-28T13:17:19.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.021+0000 7f939d800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9398160360 0x7f93980713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.021+0000 7f939d800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9398160360 0x7f93980713e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42462/0 (socket says 172.21.15.19:42462) 2024-03-28T13:17:19.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.021+0000 7f939d800640 1 -- 172.21.15.19:0/183457331 learned_addr learned my addr 172.21.15.19:0/183457331 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:19.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.021+0000 7f939e200640 1 --2- 172.21.15.19:0/183457331 >> v2:172.21.15.188:3300/0 conn(0x7f9398162de0 0x7f93981663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.021+0000 7f939ce00640 1 --2- 172.21.15.19:0/183457331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93980735c0 0x7f9398071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.022+0000 7f939d800640 1 -- 172.21.15.19:0/183457331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93980735c0 msgr2=0x7f9398071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.022+0000 7f939d800640 1 --2- 172.21.15.19:0/183457331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93980735c0 0x7f9398071920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.022+0000 7f939d800640 1 -- 172.21.15.19:0/183457331 >> v2:172.21.15.188:3300/0 conn(0x7f9398162de0 msgr2=0x7f93981663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.022+0000 7f939d800640 1 --2- 172.21.15.19:0/183457331 >> v2:172.21.15.188:3300/0 conn(0x7f9398162de0 0x7f93981663d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.022+0000 7f939d800640 1 -- 172.21.15.19:0/183457331 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93980b3c00 con 0x7f9398160360 2024-03-28T13:17:19.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.023+0000 7f939d800640 1 --2- 172.21.15.19:0/183457331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9398160360 0x7f93980713e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f93980b3600 tx=0x7f938c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb91386f67f115f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.023+0000 7f9397e00640 1 -- 172.21.15.19:0/183457331 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c035020 con 0x7f9398160360 2024-03-28T13:17:19.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.023+0000 7f9397e00640 1 -- 172.21.15.19:0/183457331 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f938c02d040 con 0x7f9398160360 2024-03-28T13:17:19.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.023+0000 7f9397e00640 1 -- 172.21.15.19:0/183457331 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c031a80 con 0x7f9398160360 2024-03-28T13:17:19.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.024+0000 7f939ec00640 1 -- 172.21.15.19:0/183457331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9398160360 msgr2=0x7f93980713e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.024+0000 7f939ec00640 1 --2- 172.21.15.19:0/183457331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9398160360 0x7f93980713e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f93980b3600 tx=0x7f938c02b900 comp rx=0 tx=0).stop 2024-03-28T13:17:19.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.024+0000 7f939ec00640 1 -- 172.21.15.19:0/183457331 shutdown_connections 2024-03-28T13:17:19.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.024+0000 7f939ec00640 1 --2- 172.21.15.19:0/183457331 >> v2:172.21.15.188:3300/0 conn(0x7f9398162de0 0x7f93981663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.024+0000 7f939ec00640 1 --2- 172.21.15.19:0/183457331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93980735c0 0x7f9398071920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.024+0000 7f939ec00640 1 --2- 172.21.15.19:0/183457331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9398160360 0x7f93980713e0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.024+0000 7f939ec00640 1 -- 172.21.15.19:0/183457331 >> 172.21.15.19:0/183457331 conn(0x7f93980b1ba0 msgr2=0x7f93980b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:19.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.024+0000 7f939ec00640 1 -- 172.21.15.19:0/183457331 shutdown_connections 2024-03-28T13:17:19.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.024+0000 7f939ec00640 1 -- 172.21.15.19:0/183457331 wait complete. 2024-03-28T13:17:19.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.026+0000 7f939ec00640 1 Processor -- start 2024-03-28T13:17:19.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.026+0000 7f939ec00640 1 -- start start 2024-03-28T13:17:19.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.027+0000 7f939ec00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93980735c0 0x7f9398158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.027+0000 7f939ec00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9398160360 0x7f93981598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.027+0000 7f939ec00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9398162de0 0x7f93981661f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.027+0000 7f939ec00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f93980b49c0 con 0x7f9398160360 2024-03-28T13:17:19.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.027+0000 7f939ec00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f93980b41c0 con 0x7f93980735c0 2024-03-28T13:17:19.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.027+0000 7f939ec00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9398160740 con 0x7f9398162de0 2024-03-28T13:17:19.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.027+0000 7f939ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9398160360 0x7f93981598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.027+0000 7f939ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9398160360 0x7f93981598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42476/0 (socket says 172.21.15.19:42476) 2024-03-28T13:17:19.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.027+0000 7f939ce00640 1 -- 172.21.15.19:0/403909928 learned_addr learned my addr 172.21.15.19:0/403909928 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:19.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.027+0000 7f939d800640 1 --2- 172.21.15.19:0/403909928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93980735c0 0x7f9398158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.028+0000 7f939e200640 1 --2- 172.21.15.19:0/403909928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9398162de0 0x7f93981661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.028+0000 7f939ce00640 1 -- 172.21.15.19:0/403909928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9398162de0 msgr2=0x7f93981661f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.028+0000 7f939ce00640 1 --2- 172.21.15.19:0/403909928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9398162de0 0x7f93981661f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.028+0000 7f939ce00640 1 -- 172.21.15.19:0/403909928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93980735c0 msgr2=0x7f9398158110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.028+0000 7f939ce00640 1 --2- 172.21.15.19:0/403909928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93980735c0 0x7f9398158110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.028+0000 7f939ce00640 1 -- 172.21.15.19:0/403909928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9398158840 con 0x7f9398160360 2024-03-28T13:17:19.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.028+0000 7f939ce00640 1 --2- 172.21.15.19:0/403909928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9398160360 0x7f93981598e0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f938400d5d0 tx=0x7f938400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.029+0000 7f9395600640 1 -- 172.21.15.19:0/403909928 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384004240 con 0x7f9398160360 2024-03-28T13:17:19.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.029+0000 7f9395600640 1 -- 172.21.15.19:0/403909928 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93840043e0 con 0x7f9398160360 2024-03-28T13:17:19.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.029+0000 7f9395600640 1 -- 172.21.15.19:0/403909928 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384003830 con 0x7f9398160360 2024-03-28T13:17:19.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.029+0000 7f939ec00640 1 -- 172.21.15.19:0/403909928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9398158b20 con 0x7f9398160360 2024-03-28T13:17:19.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.029+0000 7f939ec00640 1 -- 172.21.15.19:0/403909928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93981590e0 con 0x7f9398160360 2024-03-28T13:17:19.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.031+0000 7f9395600640 1 -- 172.21.15.19:0/403909928 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f93840039d0 con 0x7f9398160360 2024-03-28T13:17:19.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.032+0000 7f9395600640 1 --2- 172.21.15.19:0/403909928 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f937806c530 0x7f937806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.032+0000 7f9395600640 1 -- 172.21.15.19:0/403909928 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f938408ba60 con 0x7f9398160360 2024-03-28T13:17:19.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.032+0000 7f939d800640 1 --2- 172.21.15.19:0/403909928 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f937806c530 0x7f937806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.033+0000 7f93a2fd6740 1 Processor -- start 2024-03-28T13:17:19.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.033+0000 7f93a2fd6740 1 -- start start 2024-03-28T13:17:19.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.033+0000 7f939d800640 1 --2- 172.21.15.19:0/403909928 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f937806c530 0x7f937806e9f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f938c036000 tx=0x7f938c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.034+0000 7f93a2fd6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f378a0 0x558c23f37c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.034+0000 7f93a2fd6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f078c0 0x558c23f34b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.034+0000 7f93a2fd6740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 0x558c23f35050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.034+0000 7f93a2fd6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558c23e715a0 con 0x558c23f078c0 2024-03-28T13:17:19.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.034+0000 7f93a2fd6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558c23e19390 con 0x558c23f378a0 2024-03-28T13:17:19.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.034+0000 7f93a2fd6740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x558c23e6d550 con 0x558c23f0b420 2024-03-28T13:17:19.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.034+0000 7f939ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f078c0 0x558c23f34b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.034+0000 7f939ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f078c0 0x558c23f34b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42482/0 (socket says 172.21.15.19:42482) 2024-03-28T13:17:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.034+0000 7f939ce00640 1 -- 172.21.15.19:0/1272738592 learned_addr learned my addr 172.21.15.19:0/1272738592 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.034+0000 7f939d800640 1 --2- 172.21.15.19:0/1272738592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f378a0 0x558c23f37c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.035+0000 7f939ce00640 1 -- 172.21.15.19:0/1272738592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 msgr2=0x558c23f35050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.035+0000 7f939e200640 1 --2- 172.21.15.19:0/1272738592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 0x558c23f35050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.035+0000 7f939ce00640 1 --2- 172.21.15.19:0/1272738592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 0x558c23f35050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.035+0000 7f939ce00640 1 -- 172.21.15.19:0/1272738592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f378a0 msgr2=0x558c23f37c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.035+0000 7f939ce00640 1 --2- 172.21.15.19:0/1272738592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f378a0 0x558c23f37c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.035+0000 7f939ce00640 1 -- 172.21.15.19:0/1272738592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c23c517c0 con 0x558c23f078c0 2024-03-28T13:17:19.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.035+0000 7f939d800640 1 --2- 172.21.15.19:0/1272738592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f378a0 0x558c23f37c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:17:19.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.035+0000 7f939ce00640 1 --2- 172.21.15.19:0/1272738592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f078c0 0x558c23f34b10 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f9384055ce0 tx=0x7f93840594f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f502f5d95a397a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.035+0000 7f938a000640 1 -- 172.21.15.19:0/1272738592 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384088070 con 0x558c23f078c0 2024-03-28T13:17:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.035+0000 7f938a000640 1 -- 172.21.15.19:0/1272738592 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9384062460 con 0x558c23f078c0 2024-03-28T13:17:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.035+0000 7f938a000640 1 -- 172.21.15.19:0/1272738592 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384060070 con 0x558c23f078c0 2024-03-28T13:17:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.036+0000 7f93a2fd6740 1 -- 172.21.15.19:0/1272738592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f078c0 msgr2=0x558c23f34b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.036+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/1272738592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f078c0 0x558c23f34b10 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f9384055ce0 tx=0x7f93840594f0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.036+0000 7f93a2fd6740 1 -- 172.21.15.19:0/1272738592 shutdown_connections 2024-03-28T13:17:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.036+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/1272738592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 0x558c23f35050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.036+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/1272738592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f078c0 0x558c23f34b10 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.036+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/1272738592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f378a0 0x558c23f37c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.036+0000 7f93a2fd6740 1 -- 172.21.15.19:0/1272738592 >> 172.21.15.19:0/1272738592 conn(0x558c23f09b30 msgr2=0x558c23f526b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.036+0000 7f93a2fd6740 1 -- 172.21.15.19:0/1272738592 shutdown_connections 2024-03-28T13:17:19.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.036+0000 7f93a2fd6740 1 -- 172.21.15.19:0/1272738592 wait complete. 2024-03-28T13:17:19.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.037+0000 7f93a2fd6740 1 -- start start 2024-03-28T13:17:19.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.037+0000 7f93a2fd6740 1 Processor -- start 2024-03-28T13:17:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.038+0000 7f93a2fd6740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 0x558c23f3b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.038+0000 7f93a2fd6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f378a0 0x558c23f33490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.038+0000 7f93a2fd6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f629b0 0x558c23f381e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.038+0000 7f93a2fd6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558c23f0c5d0 con 0x558c23f629b0 2024-03-28T13:17:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.038+0000 7f93a2fd6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558c23e2c780 con 0x558c23f378a0 2024-03-28T13:17:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.038+0000 7f93a2fd6740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x558c23f0b9c0 con 0x558c23f0b420 2024-03-28T13:17:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.038+0000 7f93a2fd6740 10 client.0 _ll_register_callbacks cb 0x558c23e68fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:17:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.038+0000 7f939e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f629b0 0x558c23f381e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.038+0000 7f939e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f629b0 0x558c23f381e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42498/0 (socket says 172.21.15.19:42498) 2024-03-28T13:17:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.038+0000 7f939e200640 1 -- 172.21.15.19:0/4174402821 learned_addr learned my addr 172.21.15.19:0/4174402821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:19.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.039+0000 7f939ce00640 1 --2- 172.21.15.19:0/4174402821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f378a0 0x558c23f33490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.039+0000 7f939d800640 1 --2- 172.21.15.19:0/4174402821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 0x558c23f3b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.039+0000 7f939e200640 1 -- 172.21.15.19:0/4174402821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 msgr2=0x558c23f3b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.039+0000 7f939e200640 1 --2- 172.21.15.19:0/4174402821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 0x558c23f3b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.039+0000 7f939e200640 1 -- 172.21.15.19:0/4174402821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f378a0 msgr2=0x558c23f33490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.039+0000 7f939e200640 1 --2- 172.21.15.19:0/4174402821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f378a0 0x558c23f33490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.039+0000 7f939e200640 1 -- 172.21.15.19:0/4174402821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c23cb1f50 con 0x558c23f629b0 2024-03-28T13:17:19.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.039+0000 7f939d800640 1 --2- 172.21.15.19:0/4174402821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 0x558c23f3b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:19.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.039+0000 7f939e200640 1 --2- 172.21.15.19:0/4174402821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f629b0 0x558c23f381e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f9390007dc0 tx=0x7f9390012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31e812e62eee7dd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.039+0000 7f937ea00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:17:19.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.039+0000 7f937ea00640 1 -- 172.21.15.19:0/4174402821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939001a070 con 0x558c23f629b0 2024-03-28T13:17:19.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.040+0000 7f937ea00640 1 -- 172.21.15.19:0/4174402821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f9390012e40 con 0x558c23f629b0 2024-03-28T13:17:19.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.040+0000 7f93a2fd6740 1 -- 172.21.15.19:0/4174402821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9384062460 con 0x558c23f629b0 2024-03-28T13:17:19.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.040+0000 7f937ea00640 1 -- 172.21.15.19:0/4174402821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f9390025050 con 0x558c23f629b0 2024-03-28T13:17:19.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.040+0000 7f93a2fd6740 10 client.7530 fetch_fsmap learned FSMap version 68 2024-03-28T13:17:19.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.040+0000 7f93a2fd6740 1 -- 172.21.15.19:0/4174402821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x558c23b450b0 con 0x558c23f629b0 2024-03-28T13:17:19.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.040+0000 7f937ea00640 1 -- 172.21.15.19:0/4174402821 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9384062460 con 0x558c23f629b0 2024-03-28T13:17:19.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.040+0000 7f937ea00640 1 -- 172.21.15.19:0/4174402821 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f9390028020 con 0x558c23f629b0 2024-03-28T13:17:19.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.040+0000 7f93a2fd6740 10 client.7530 fetch_fsmap finished waiting for FSMap version 68 2024-03-28T13:17:19.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.040+0000 7f93a2fd6740 10 client.7530 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T13:17:19.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.040+0000 7f93a2fd6740 20 client.7530 populate_metadata read hostname 'smithi019' 2024-03-28T13:17:19.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.040+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/4174402821 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x558c23f66250 0x558c23f66630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.040+0000 7f93a2fd6740 4 client.7530 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:17:19.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.040+0000 7f93a2fd6740 1 -- 172.21.15.19:0/4174402821 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c23e45870 con 0x558c23f66250 2024-03-28T13:17:19.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.041+0000 7f939d800640 1 --2- 172.21.15.19:0/4174402821 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x558c23f66250 0x558c23f66630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.042+0000 7f939d800640 1 --2- 172.21.15.19:0/4174402821 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x558c23f66250 0x558c23f66630 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d24f6949edd8b59b server_cookie=61567f9fe6512eb1 in_seq=0 out_seq=0 2024-03-28T13:17:19.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.042+0000 7f937ea00640 10 client.7530 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:19.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.044+0000 7f937ea00640 1 -- 172.21.15.19:0/4174402821 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c23e45870 con 0x558c23f66250 2024-03-28T13:17:19.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.044+0000 7f937ea00640 10 client.7530 handle_command_reply: tid=0 2024-03-28T13:17:19.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.044+0000 7f93a2fd6740 1 client.7530 shutdown 2024-03-28T13:17:19.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.044+0000 7f93a2fd6740 1 -- 172.21.15.19:0/4174402821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f629b0 msgr2=0x558c23f381e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.044+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/4174402821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f629b0 0x558c23f381e0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f9390007dc0 tx=0x7f9390012610 comp rx=0 tx=0).stop 2024-03-28T13:17:19.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.044+0000 7f93a2fd6740 1 -- 172.21.15.19:0/4174402821 shutdown_connections 2024-03-28T13:17:19.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.044+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/4174402821 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x558c23f66250 0x558c23f66630 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.044+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/4174402821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f629b0 0x558c23f381e0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.044+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/4174402821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f378a0 0x558c23f33490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.044+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/4174402821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 0x558c23f3b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.044+0000 7f937ea00640 0 client.7530 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:19.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.044+0000 7f93a2fd6740 1 -- 172.21.15.19:0/4174402821 >> 172.21.15.19:0/4174402821 conn(0x558c23f078c0 msgr2=0x558c23f0d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:19.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.044+0000 7f93a2fd6740 1 -- 172.21.15.19:0/4174402821 shutdown_connections 2024-03-28T13:17:19.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.044+0000 7f93a2fd6740 1 -- 172.21.15.19:0/4174402821 wait complete. 2024-03-28T13:17:19.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.045+0000 7f93a2fd6740 20 client.7530 trim_cache size 0 max 16384 2024-03-28T13:17:19.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.059+0000 7f93a2fd6740 1 Processor -- start 2024-03-28T13:17:19.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.059+0000 7f93a2fd6740 1 -- start start 2024-03-28T13:17:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.060+0000 7f93a2fd6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f0b420 0x558c23f3a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.060+0000 7f93a2fd6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f378a0 0x558c23f6c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.060+0000 7f93a2fd6740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f629b0 0x558c23f55bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.060+0000 7f93a2fd6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9398203f70 con 0x558c23f378a0 2024-03-28T13:17:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.060+0000 7f93a2fd6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f93981440b0 con 0x558c23f0b420 2024-03-28T13:17:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.060+0000 7f93a2fd6740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9398144700 con 0x558c23f629b0 2024-03-28T13:17:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.061+0000 7f939ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f378a0 0x558c23f6c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.061+0000 7f939ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f378a0 0x558c23f6c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42510/0 (socket says 172.21.15.19:42510) 2024-03-28T13:17:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.061+0000 7f939ce00640 1 -- 172.21.15.19:0/1097663361 learned_addr learned my addr 172.21.15.19:0/1097663361 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.061+0000 7f939d800640 1 --2- 172.21.15.19:0/1097663361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f0b420 0x558c23f3a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.061+0000 7f939e200640 1 --2- 172.21.15.19:0/1097663361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f629b0 0x558c23f55bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.061+0000 7f939ce00640 1 -- 172.21.15.19:0/1097663361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f629b0 msgr2=0x558c23f55bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.061+0000 7f939ce00640 1 --2- 172.21.15.19:0/1097663361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f629b0 0x558c23f55bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.061+0000 7f939ce00640 1 -- 172.21.15.19:0/1097663361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f0b420 msgr2=0x558c23f3a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.061+0000 7f939ce00640 1 --2- 172.21.15.19:0/1097663361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f0b420 0x558c23f3a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.061+0000 7f939ce00640 1 -- 172.21.15.19:0/1097663361 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c23f536d0 con 0x558c23f378a0 2024-03-28T13:17:19.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.062+0000 7f939ce00640 1 --2- 172.21.15.19:0/1097663361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f378a0 0x558c23f6c660 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f9384059530 tx=0x7f9384060000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842ffb43c3747f25 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.062+0000 7f9389600640 1 -- 172.21.15.19:0/1097663361 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384077070 con 0x558c23f378a0 2024-03-28T13:17:19.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.062+0000 7f9389600640 1 -- 172.21.15.19:0/1097663361 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9384062460 con 0x558c23f378a0 2024-03-28T13:17:19.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.062+0000 7f9389600640 1 -- 172.21.15.19:0/1097663361 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384088040 con 0x558c23f378a0 2024-03-28T13:17:19.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.062+0000 7f93a2fd6740 1 -- 172.21.15.19:0/1097663361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f378a0 msgr2=0x558c23f6c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.062+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/1097663361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f378a0 0x558c23f6c660 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f9384059530 tx=0x7f9384060000 comp rx=0 tx=0).stop 2024-03-28T13:17:19.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.062+0000 7f93a2fd6740 1 -- 172.21.15.19:0/1097663361 shutdown_connections 2024-03-28T13:17:19.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.062+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/1097663361 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f629b0 0x558c23f55bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.062+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/1097663361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f378a0 0x558c23f6c660 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.062+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/1097663361 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f0b420 0x558c23f3a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.062+0000 7f93a2fd6740 1 -- 172.21.15.19:0/1097663361 >> 172.21.15.19:0/1097663361 conn(0x558c23f078c0 msgr2=0x558c23f37c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:19.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.063+0000 7f93a2fd6740 1 -- 172.21.15.19:0/1097663361 shutdown_connections 2024-03-28T13:17:19.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.063+0000 7f93a2fd6740 1 -- 172.21.15.19:0/1097663361 wait complete. 2024-03-28T13:17:19.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.063+0000 7f93a2fd6740 1 -- start start 2024-03-28T13:17:19.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.063+0000 7f93a2fd6740 1 Processor -- start 2024-03-28T13:17:19.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.064+0000 7f93a2fd6740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 0x558c23fcbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.064+0000 7f93a2fd6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f378a0 0x558c23f55bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.064+0000 7f93a2fd6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f629b0 0x558c23f56100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.064+0000 7f93a2fd6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558c23f2e370 con 0x558c23f378a0 2024-03-28T13:17:19.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.064+0000 7f93a2fd6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558c23f38720 con 0x558c23f629b0 2024-03-28T13:17:19.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.064+0000 7f93a2fd6740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x558c23f390a0 con 0x558c23f0b420 2024-03-28T13:17:19.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.064+0000 7f93a2fd6740 10 client.0 _ll_register_callbacks cb 0x7f9390009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:17:19.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.065+0000 7f939ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f378a0 0x558c23f55bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.065+0000 7f939ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f378a0 0x558c23f55bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42520/0 (socket says 172.21.15.19:42520) 2024-03-28T13:17:19.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.065+0000 7f939ce00640 1 -- 172.21.15.19:0/2611357814 learned_addr learned my addr 172.21.15.19:0/2611357814 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:19.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.065+0000 7f939e200640 1 --2- 172.21.15.19:0/2611357814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f629b0 0x558c23f56100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.065+0000 7f939d800640 1 --2- 172.21.15.19:0/2611357814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 0x558c23fcbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.065+0000 7f939ce00640 1 -- 172.21.15.19:0/2611357814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 msgr2=0x558c23fcbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.065+0000 7f939ce00640 1 --2- 172.21.15.19:0/2611357814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 0x558c23fcbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.065+0000 7f939ce00640 1 -- 172.21.15.19:0/2611357814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f629b0 msgr2=0x558c23f56100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.065+0000 7f939ce00640 1 --2- 172.21.15.19:0/2611357814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f629b0 0x558c23f56100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.065+0000 7f939ce00640 1 -- 172.21.15.19:0/2611357814 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c23f03460 con 0x558c23f378a0 2024-03-28T13:17:19.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.065+0000 7f939e200640 1 --2- 172.21.15.19:0/2611357814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f629b0 0x558c23f56100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:19.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.065+0000 7f939ce00640 1 --2- 172.21.15.19:0/2611357814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f378a0 0x558c23f55bc0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f9384075040 tx=0x7f938405c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6125898e4a763b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.065+0000 7f937ea00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:17:19.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.066+0000 7f937ea00640 1 -- 172.21.15.19:0/2611357814 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384077070 con 0x558c23f378a0 2024-03-28T13:17:19.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.066+0000 7f937ea00640 1 -- 172.21.15.19:0/2611357814 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f93840600b0 con 0x558c23f378a0 2024-03-28T13:17:19.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.066+0000 7f93a2fd6740 1 -- 172.21.15.19:0/2611357814 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9384062460 con 0x558c23f378a0 2024-03-28T13:17:19.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.066+0000 7f937ea00640 1 -- 172.21.15.19:0/2611357814 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f9384080050 con 0x558c23f378a0 2024-03-28T13:17:19.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.066+0000 7f93a2fd6740 10 client.7536 fetch_fsmap learned FSMap version 68 2024-03-28T13:17:19.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.066+0000 7f93a2fd6740 1 -- 172.21.15.19:0/2611357814 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x558c23fcc3d0 con 0x558c23f378a0 2024-03-28T13:17:19.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.066+0000 7f937ea00640 1 -- 172.21.15.19:0/2611357814 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9384062460 con 0x558c23f378a0 2024-03-28T13:17:19.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.066+0000 7f937ea00640 1 -- 172.21.15.19:0/2611357814 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f9384083020 con 0x558c23f378a0 2024-03-28T13:17:19.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.066+0000 7f93a2fd6740 10 client.7536 fetch_fsmap finished waiting for FSMap version 68 2024-03-28T13:17:19.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.066+0000 7f93a2fd6740 10 client.7536 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T13:17:19.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.066+0000 7f93a2fd6740 20 client.7536 populate_metadata read hostname 'smithi019' 2024-03-28T13:17:19.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.066+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/2611357814 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x558c23f5aaf0 0x558c23f544b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.066+0000 7f93a2fd6740 4 client.7536 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-28T13:17:19.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.066+0000 7f93a2fd6740 1 -- 172.21.15.19:0/2611357814 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x558c23f080c0 con 0x558c23f5aaf0 2024-03-28T13:17:19.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.067+0000 7f939d800640 1 --2- 172.21.15.19:0/2611357814 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x558c23f5aaf0 0x558c23f544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.068+0000 7f939d800640 1 --2- 172.21.15.19:0/2611357814 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x558c23f5aaf0 0x558c23f544b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6673b07850c01c05 server_cookie=666d9524af007b5c in_seq=0 out_seq=0 2024-03-28T13:17:19.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.068+0000 7f937ea00640 10 client.7536 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:19.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.068+0000 7f937ea00640 1 -- 172.21.15.19:0/2611357814 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+419 (crc 0 0 0) 0x558c23f080c0 con 0x558c23f5aaf0 2024-03-28T13:17:19.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.068+0000 7f937ea00640 10 client.7536 handle_command_reply: tid=0 2024-03-28T13:17:19.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.068+0000 7f93a2fd6740 1 client.7536 shutdown 2024-03-28T13:17:19.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.069+0000 7f93a2fd6740 1 -- 172.21.15.19:0/2611357814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f378a0 msgr2=0x558c23f55bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.069+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/2611357814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f378a0 0x558c23f55bc0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f9384075040 tx=0x7f938405c000 comp rx=0 tx=0).stop 2024-03-28T13:17:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.069+0000 7f93a2fd6740 1 -- 172.21.15.19:0/2611357814 shutdown_connections 2024-03-28T13:17:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.069+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/2611357814 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x558c23f5aaf0 0x558c23f544b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.069+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/2611357814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558c23f629b0 0x558c23f56100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.069+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/2611357814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558c23f378a0 0x558c23f55bc0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.069+0000 7f93a2fd6740 1 --2- 172.21.15.19:0/2611357814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558c23f0b420 0x558c23fcbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.069+0000 7f937ea00640 0 client.7536 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.069+0000 7f93a2fd6740 1 -- 172.21.15.19:0/2611357814 >> 172.21.15.19:0/2611357814 conn(0x558c23f078c0 msgr2=0x558c23f765f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.069+0000 7f93a2fd6740 1 -- 172.21.15.19:0/2611357814 shutdown_connections 2024-03-28T13:17:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.069+0000 7f93a2fd6740 1 -- 172.21.15.19:0/2611357814 wait complete. 2024-03-28T13:17:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.069+0000 7f93a2fd6740 20 client.7536 trim_cache size 0 max 16384 2024-03-28T13:17:19.072 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:17:19.073 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-03-28T13:17:19.073 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:17:19.073 INFO:teuthology.orchestra.run.smithi019.stdout: "damage_type": "dir_frag", 2024-03-28T13:17:19.073 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 427240806, 2024-03-28T13:17:19.073 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511627776, 2024-03-28T13:17:19.073 INFO:teuthology.orchestra.run.smithi019.stdout: "frag": "*", 2024-03-28T13:17:19.073 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/dir" 2024-03-28T13:17:19.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:17:19.073 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:17:19.073 INFO:teuthology.orchestra.run.smithi019.stdout: "damage_type": "backtrace", 2024-03-28T13:17:19.073 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 3150093696, 2024-03-28T13:17:19.073 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511627776, 2024-03-28T13:17:19.073 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/dir" 2024-03-28T13:17:19.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:17:19.074 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:17:19.074 INFO:teuthology.orchestra.run.smithi019.stdout: "damage_type": "backtrace", 2024-03-28T13:17:19.074 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 3207548007, 2024-03-28T13:17:19.074 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511627777, 2024-03-28T13:17:19.074 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/testdir/hardlink" 2024-03-28T13:17:19.074 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:17:19.074 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-03-28T13:17:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.070+0000 7f939ec00640 1 -- 172.21.15.19:0/403909928 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f937806c530 msgr2=0x7f937806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.070+0000 7f939ec00640 1 --2- 172.21.15.19:0/403909928 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f937806c530 0x7f937806e9f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f938c036000 tx=0x7f938c03c040 comp rx=0 tx=0).stop 2024-03-28T13:17:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.070+0000 7f939ec00640 1 -- 172.21.15.19:0/403909928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9398160360 msgr2=0x7f93981598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.070+0000 7f939ec00640 1 --2- 172.21.15.19:0/403909928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9398160360 0x7f93981598e0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f938400d5d0 tx=0x7f938400da90 comp rx=0 tx=0).stop 2024-03-28T13:17:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.070+0000 7f939ec00640 1 -- 172.21.15.19:0/403909928 shutdown_connections 2024-03-28T13:17:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.070+0000 7f939ec00640 1 --2- 172.21.15.19:0/403909928 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f937806c530 0x7f937806e9f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.070+0000 7f939ec00640 1 --2- 172.21.15.19:0/403909928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9398162de0 0x7f93981661f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.070+0000 7f939ec00640 1 --2- 172.21.15.19:0/403909928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9398160360 0x7f93981598e0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.070+0000 7f939ec00640 1 --2- 172.21.15.19:0/403909928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93980735c0 0x7f9398158110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.070+0000 7f939ec00640 1 -- 172.21.15.19:0/403909928 >> 172.21.15.19:0/403909928 conn(0x7f93980b1ba0 msgr2=0x7f93981641b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.070+0000 7f939ec00640 1 -- 172.21.15.19:0/403909928 shutdown_connections 2024-03-28T13:17:19.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.070+0000 7f939ec00640 1 -- 172.21.15.19:0/403909928 wait complete. 2024-03-28T13:17:19.086 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-28T13:17:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.292+0000 7efe36a00640 1 Processor -- start 2024-03-28T13:17:19.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.292+0000 7efe36a00640 1 -- start start 2024-03-28T13:17:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.294+0000 7efe36a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe300d0280 0x7efe300d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.294+0000 7efe36a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe3015ec80 con 0x7efe30141c10 2024-03-28T13:17:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.294+0000 7efe36a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe300d0ba0 con 0x7efe30162160 2024-03-28T13:17:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.294+0000 7efe36a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efe3015ef20 con 0x7efe300d0280 2024-03-28T13:17:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.294+0000 7efe27400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efe30141c10 0x7efe30071970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37108/0 (socket says 172.21.15.19:37108) 2024-03-28T13:17:19.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.294+0000 7efe27400640 1 -- 172.21.15.19:0/447757351 learned_addr learned my addr 172.21.15.19:0/447757351 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:17:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.295+0000 7efe2fe00640 1 --2- 172.21.15.19:0/447757351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe300d0280 0x7efe300d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.295+0000 7efe2f400640 1 -- 172.21.15.19:0/447757351 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2996308842 0 0) 0x7efe3015ec80 con 0x7efe30141c10 2024-03-28T13:17:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.295+0000 7efe2f400640 1 -- 172.21.15.19:0/447757351 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe14003520 con 0x7efe30141c10 2024-03-28T13:17:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.295+0000 7efe2f400640 1 -- 172.21.15.19:0/447757351 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4201461210 0 0) 0x7efe14003520 con 0x7efe30141c10 2024-03-28T13:17:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.295+0000 7efe2f400640 1 -- 172.21.15.19:0/447757351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe300d0280 msgr2=0x7efe300d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.295+0000 7efe2f400640 1 --2- 172.21.15.19:0/447757351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe300d0280 0x7efe300d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.295+0000 7efe2f400640 1 -- 172.21.15.19:0/447757351 >> v1:172.21.15.188:6789/0 conn(0x7efe30162160 legacy=0x7efe301625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.295+0000 7efe2f400640 1 -- 172.21.15.19:0/447757351 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3015f200 con 0x7efe30141c10 2024-03-28T13:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.295+0000 7efe2f400640 1 -- 172.21.15.19:0/447757351 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7efe18001670 con 0x7efe30141c10 2024-03-28T13:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.296+0000 7efe2f400640 1 -- 172.21.15.19:0/447757351 >> v1:172.21.15.19:6789/0 conn(0x7efe30141c10 legacy=0x7efe30071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.296+0000 7efe2f400640 1 --2- 172.21.15.19:0/447757351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe140039e0 0x7efe14003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.296+0000 7efe2f400640 1 --2- 172.21.15.19:0/447757351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe300d0280 0x7efe14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.296+0000 7efe2f400640 1 --2- 172.21.15.19:0/447757351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe14004d40 0x7efe140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.296+0000 7efe2f400640 1 -- 172.21.15.19:0/447757351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe3015ef20 con 0x7efe14004d40 2024-03-28T13:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.296+0000 7efe2f400640 1 -- 172.21.15.19:0/447757351 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efe1400ba70 con 0x7efe140039e0 2024-03-28T13:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.296+0000 7efe2f400640 1 -- 172.21.15.19:0/447757351 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efe1400bc20 con 0x7efe300d0280 2024-03-28T13:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.296+0000 7efe34a00640 1 --2- 172.21.15.19:0/447757351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe14004d40 0x7efe140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.296+0000 7efe2fe00640 1 --2- 172.21.15.19:0/447757351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe140039e0 0x7efe14003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.296+0000 7efe27400640 1 --2- 172.21.15.19:0/447757351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe300d0280 0x7efe14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.296+0000 7efe34a00640 1 -- 172.21.15.19:0/447757351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe300d0280 msgr2=0x7efe14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.296+0000 7efe34a00640 1 --2- 172.21.15.19:0/447757351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe300d0280 0x7efe14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.296+0000 7efe34a00640 1 -- 172.21.15.19:0/447757351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe140039e0 msgr2=0x7efe14003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.296+0000 7efe34a00640 1 --2- 172.21.15.19:0/447757351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe140039e0 0x7efe14003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.296+0000 7efe34a00640 1 -- 172.21.15.19:0/447757351 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe14003520 con 0x7efe14004d40 2024-03-28T13:17:19.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.298+0000 7efe34a00640 1 --2- 172.21.15.19:0/447757351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe14004d40 0x7efe140051c0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7efe28001f60 tx=0x7efe14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77bb5179770ccd0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.298+0000 7efe2f400640 1 -- 172.21.15.19:0/447757351 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28001b30 con 0x7efe14004d40 2024-03-28T13:17:19.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.298+0000 7efe2f400640 1 -- 172.21.15.19:0/447757351 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2802d710 con 0x7efe14004d40 2024-03-28T13:17:19.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.298+0000 7efe2f400640 1 -- 172.21.15.19:0/447757351 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2802db00 con 0x7efe14004d40 2024-03-28T13:17:19.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.298+0000 7efe36a00640 1 -- 172.21.15.19:0/447757351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe14004d40 msgr2=0x7efe140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.298+0000 7efe36a00640 1 --2- 172.21.15.19:0/447757351 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe14004d40 0x7efe140051c0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7efe28001f60 tx=0x7efe14005830 comp rx=0 tx=0).stop 2024-03-28T13:17:19.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.298+0000 7efe34a00640 1 -- 172.21.15.19:0/447757351 reap_dead start 2024-03-28T13:17:19.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.299+0000 7efe36a00640 1 -- 172.21.15.19:0/447757351 shutdown_connections 2024-03-28T13:17:19.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.299+0000 7efe36a00640 1 -- 172.21.15.19:0/447757351 >> 172.21.15.19:0/447757351 conn(0x7efe3006b750 msgr2=0x7efe3006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:19.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.299+0000 7efe36a00640 1 -- 172.21.15.19:0/447757351 shutdown_connections 2024-03-28T13:17:19.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.299+0000 7efe36a00640 1 -- 172.21.15.19:0/447757351 wait complete. 2024-03-28T13:17:19.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.301+0000 7efe36a00640 1 Processor -- start 2024-03-28T13:17:19.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.301+0000 7efe36a00640 1 -- start start 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.303+0000 7efe36a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe14004d40 0x7efe300ad820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.303+0000 7efe36a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe301f8990 0x7efe30143260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.303+0000 7efe36a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe301437a0 0x7efe30155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.303+0000 7efe36a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe30158e90 con 0x7efe301437a0 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.303+0000 7efe36a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efe3015f7c0 con 0x7efe14004d40 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.303+0000 7efe36a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efe3009dd70 con 0x7efe301f8990 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.303+0000 7efe34a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe301437a0 0x7efe30155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.303+0000 7efe34a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe301437a0 0x7efe30155c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42546/0 (socket says 172.21.15.19:42546) 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.303+0000 7efe34a00640 1 -- 172.21.15.19:0/4153056543 learned_addr learned my addr 172.21.15.19:0/4153056543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.303+0000 7efe2fe00640 1 --2- 172.21.15.19:0/4153056543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe14004d40 0x7efe300ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.303+0000 7efe27400640 1 --2- 172.21.15.19:0/4153056543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe301f8990 0x7efe30143260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.303+0000 7efe34a00640 1 -- 172.21.15.19:0/4153056543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe301f8990 msgr2=0x7efe30143260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.304+0000 7efe34a00640 1 --2- 172.21.15.19:0/4153056543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe301f8990 0x7efe30143260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.304+0000 7efe34a00640 1 -- 172.21.15.19:0/4153056543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe14004d40 msgr2=0x7efe300ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.304+0000 7efe34a00640 1 --2- 172.21.15.19:0/4153056543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe14004d40 0x7efe300ad820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.304+0000 7efe34a00640 1 -- 172.21.15.19:0/4153056543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe301440e0 con 0x7efe301437a0 2024-03-28T13:17:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.304+0000 7efe34a00640 1 --2- 172.21.15.19:0/4153056543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe301437a0 0x7efe30155c00 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7efe28002080 tx=0x7efe280015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.304+0000 7efe2cc00640 1 -- 172.21.15.19:0/4153056543 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28001b30 con 0x7efe301437a0 2024-03-28T13:17:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.304+0000 7efe2cc00640 1 -- 172.21.15.19:0/4153056543 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe28043070 con 0x7efe301437a0 2024-03-28T13:17:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.304+0000 7efe2cc00640 1 -- 172.21.15.19:0/4153056543 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28003b90 con 0x7efe301437a0 2024-03-28T13:17:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.304+0000 7efe36a00640 1 -- 172.21.15.19:0/4153056543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe30143e00 con 0x7efe301437a0 2024-03-28T13:17:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.304+0000 7efe36a00640 1 -- 172.21.15.19:0/4153056543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe301f8190 con 0x7efe301437a0 2024-03-28T13:17:19.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.306+0000 7efe36a00640 1 -- 172.21.15.19:0/4153056543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe300685c0 con 0x7efe301437a0 2024-03-28T13:17:19.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.307+0000 7efe2cc00640 1 -- 172.21.15.19:0/4153056543 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efe28005070 con 0x7efe301437a0 2024-03-28T13:17:19.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.307+0000 7efe2cc00640 1 --2- 172.21.15.19:0/4153056543 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efe0c06c530 0x7efe0c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.307+0000 7efe2cc00640 1 -- 172.21.15.19:0/4153056543 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7efe280ad570 con 0x7efe301437a0 2024-03-28T13:17:19.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.308+0000 7efe2fe00640 1 --2- 172.21.15.19:0/4153056543 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efe0c06c530 0x7efe0c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.309+0000 7efe2fe00640 1 --2- 172.21.15.19:0/4153056543 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efe0c06c530 0x7efe0c06e9f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efe20009a10 tx=0x7efe20008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.312+0000 7efe2cc00640 1 -- 172.21.15.19:0/4153056543 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efe2807bb30 con 0x7efe301437a0 2024-03-28T13:17:19.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.557+0000 7efe36a00640 1 -- 172.21.15.19:0/4153056543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efe300cfdb0 con 0x7efe301437a0 2024-03-28T13:17:19.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.558+0000 7efe2cc00640 1 -- 172.21.15.19:0/4153056543 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7efe2807b430 con 0x7efe301437a0 2024-03-28T13:17:19.558 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:17:19.558 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-28T13:17:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.560+0000 7efe36a00640 1 -- 172.21.15.19:0/4153056543 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efe0c06c530 msgr2=0x7efe0c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.560+0000 7efe36a00640 1 --2- 172.21.15.19:0/4153056543 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efe0c06c530 0x7efe0c06e9f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7efe20009a10 tx=0x7efe20008040 comp rx=0 tx=0).stop 2024-03-28T13:17:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.560+0000 7efe36a00640 1 -- 172.21.15.19:0/4153056543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe301437a0 msgr2=0x7efe30155c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.560+0000 7efe36a00640 1 --2- 172.21.15.19:0/4153056543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe301437a0 0x7efe30155c00 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7efe28002080 tx=0x7efe280015b0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.561+0000 7efe36a00640 1 -- 172.21.15.19:0/4153056543 shutdown_connections 2024-03-28T13:17:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.561+0000 7efe36a00640 1 --2- 172.21.15.19:0/4153056543 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efe0c06c530 0x7efe0c06e9f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.561+0000 7efe36a00640 1 --2- 172.21.15.19:0/4153056543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe301437a0 0x7efe30155c00 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.561+0000 7efe36a00640 1 --2- 172.21.15.19:0/4153056543 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe301f8990 0x7efe30143260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.561+0000 7efe36a00640 1 --2- 172.21.15.19:0/4153056543 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe14004d40 0x7efe300ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.561+0000 7efe36a00640 1 -- 172.21.15.19:0/4153056543 >> 172.21.15.19:0/4153056543 conn(0x7efe3006b750 msgr2=0x7efe301615d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.561+0000 7efe36a00640 1 -- 172.21.15.19:0/4153056543 shutdown_connections 2024-03-28T13:17:19.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.561+0000 7efe36a00640 1 -- 172.21.15.19:0/4153056543 wait complete. 2024-03-28T13:17:19.580 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-28T13:17:19.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:17:19.581 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-03-28T13:17:19.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.807+0000 7f220a400640 1 Processor -- start 2024-03-28T13:17:19.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.808+0000 7f220a400640 1 -- start start 2024-03-28T13:17:19.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.809+0000 7f220a400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220415e4d0 0x7f220406e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.809+0000 7f220a400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2204071610 0x7f220406efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.810+0000 7f220a400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22040adf20 con 0x7f220406f510 2024-03-28T13:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.810+0000 7f220a400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f220406f960 con 0x7f2204071610 2024-03-28T13:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.810+0000 7f220a400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2204071e40 con 0x7f220415e4d0 2024-03-28T13:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.810+0000 7f2203e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f220406f510 0x7f22040b2f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37116/0 (socket says 172.21.15.19:37116) 2024-03-28T13:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.810+0000 7f2203e00640 1 -- 172.21.15.19:0/423676879 learned_addr learned my addr 172.21.15.19:0/423676879 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:17:19.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.810+0000 7f2203400640 1 --2- 172.21.15.19:0/423676879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220415e4d0 0x7f220406e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.810+0000 7f2202a00640 1 --2- 172.21.15.19:0/423676879 >> v2:172.21.15.188:3300/0 conn(0x7f2204071610 0x7f220406efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 -- 172.21.15.19:0/423676879 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3585822761 0 0) 0x7f22040adf20 con 0x7f220406f510 2024-03-28T13:17:19.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 -- 172.21.15.19:0/423676879 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f21e8003520 con 0x7f220406f510 2024-03-28T13:17:19.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 -- 172.21.15.19:0/423676879 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 400833725 0 0) 0x7f21e8003520 con 0x7f220406f510 2024-03-28T13:17:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 -- 172.21.15.19:0/423676879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220415e4d0 msgr2=0x7f220406e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 --2- 172.21.15.19:0/423676879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220415e4d0 0x7f220406e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 -- 172.21.15.19:0/423676879 >> v2:172.21.15.188:3300/0 conn(0x7f2204071610 msgr2=0x7f220406efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 --2- 172.21.15.19:0/423676879 >> v2:172.21.15.188:3300/0 conn(0x7f2204071610 0x7f220406efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 -- 172.21.15.19:0/423676879 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2204072120 con 0x7f220406f510 2024-03-28T13:17:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 -- 172.21.15.19:0/423676879 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f21fc001670 con 0x7f220406f510 2024-03-28T13:17:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 -- 172.21.15.19:0/423676879 >> v1:172.21.15.19:6789/0 conn(0x7f220406f510 legacy=0x7f22040b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 --2- 172.21.15.19:0/423676879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21e80039b0 0x7f21e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 --2- 172.21.15.19:0/423676879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21e8004800 0x7f21e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 --2- 172.21.15.19:0/423676879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220415e4d0 0x7f21e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 -- 172.21.15.19:0/423676879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f220406f960 con 0x7f21e8004800 2024-03-28T13:17:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 -- 172.21.15.19:0/423676879 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2204071e40 con 0x7f21e80039b0 2024-03-28T13:17:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.811+0000 7f2202000640 1 -- 172.21.15.19:0/423676879 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f21e800b9c0 con 0x7f220415e4d0 2024-03-28T13:17:19.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.812+0000 7f2202a00640 1 --2- 172.21.15.19:0/423676879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21e8004800 0x7f21e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.812+0000 7f2203400640 1 --2- 172.21.15.19:0/423676879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21e80039b0 0x7f21e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.812+0000 7f2203e00640 1 --2- 172.21.15.19:0/423676879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220415e4d0 0x7f21e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.812+0000 7f2202a00640 1 -- 172.21.15.19:0/423676879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220415e4d0 msgr2=0x7f21e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.812+0000 7f2202a00640 1 --2- 172.21.15.19:0/423676879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220415e4d0 0x7f21e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.812+0000 7f2202a00640 1 -- 172.21.15.19:0/423676879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21e80039b0 msgr2=0x7f21e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.812+0000 7f2202a00640 1 --2- 172.21.15.19:0/423676879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21e80039b0 0x7f21e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.812+0000 7f2202a00640 1 -- 172.21.15.19:0/423676879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e8003520 con 0x7f21e8004800 2024-03-28T13:17:19.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.812+0000 7f2203e00640 1 --2- 172.21.15.19:0/423676879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220415e4d0 0x7f21e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:19.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.813+0000 7f2202a00640 1 --2- 172.21.15.19:0/423676879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21e8004800 0x7f21e8004c80 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009af0 tx=0x7f21e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f98d6f0b867032 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.814+0000 7f2202000640 1 -- 172.21.15.19:0/423676879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec039070 con 0x7f21e8004800 2024-03-28T13:17:19.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.814+0000 7f2202000640 1 -- 172.21.15.19:0/423676879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21ec02c710 con 0x7f21e8004800 2024-03-28T13:17:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.814+0000 7f2202000640 1 -- 172.21.15.19:0/423676879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ec009690 con 0x7f21e8004800 2024-03-28T13:17:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.814+0000 7f220a400640 1 -- 172.21.15.19:0/423676879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21e8004800 msgr2=0x7f21e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.814+0000 7f220a400640 1 --2- 172.21.15.19:0/423676879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21e8004800 0x7f21e8004c80 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009af0 tx=0x7f21e8005830 comp rx=0 tx=0).stop 2024-03-28T13:17:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.814+0000 7f2203e00640 1 -- 172.21.15.19:0/423676879 reap_dead start 2024-03-28T13:17:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.815+0000 7f220a400640 1 -- 172.21.15.19:0/423676879 shutdown_connections 2024-03-28T13:17:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.815+0000 7f220a400640 1 -- 172.21.15.19:0/423676879 >> 172.21.15.19:0/423676879 conn(0x7f220406bba0 msgr2=0x7f22040b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.815+0000 7f220a400640 1 -- 172.21.15.19:0/423676879 shutdown_connections 2024-03-28T13:17:19.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.815+0000 7f220a400640 1 -- 172.21.15.19:0/423676879 wait complete. 2024-03-28T13:17:19.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.816+0000 7f220a400640 1 Processor -- start 2024-03-28T13:17:19.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.817+0000 7f220a400640 1 -- start start 2024-03-28T13:17:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.817+0000 7f220a400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e8004800 0x7f22041f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f220a400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f220409dd30 0x7f22041f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f220a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22041f80b0 0x7f22041f8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f220a400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f22040729c0 con 0x7f22041f80b0 2024-03-28T13:17:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f220a400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f22040726e0 con 0x7f220409dd30 2024-03-28T13:17:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f220a400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f220415e8b0 con 0x7f21e8004800 2024-03-28T13:17:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f2203e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22041f80b0 0x7f22041f8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f2203e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22041f80b0 0x7f22041f8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42554/0 (socket says 172.21.15.19:42554) 2024-03-28T13:17:19.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f2203e00640 1 -- 172.21.15.19:0/1853134953 learned_addr learned my addr 172.21.15.19:0/1853134953 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f2202a00640 1 --2- 172.21.15.19:0/1853134953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f220409dd30 0x7f22041f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f2203400640 1 --2- 172.21.15.19:0/1853134953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e8004800 0x7f22041f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f2203e00640 1 -- 172.21.15.19:0/1853134953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e8004800 msgr2=0x7f22041f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f2203e00640 1 --2- 172.21.15.19:0/1853134953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e8004800 0x7f22041f3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f2203e00640 1 -- 172.21.15.19:0/1853134953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f220409dd30 msgr2=0x7f22041f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f2203e00640 1 --2- 172.21.15.19:0/1853134953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f220409dd30 0x7f22041f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f2203e00640 1 -- 172.21.15.19:0/1853134953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2204149860 con 0x7f22041f80b0 2024-03-28T13:17:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.818+0000 7f2202a00640 1 --2- 172.21.15.19:0/1853134953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f220409dd30 0x7f22041f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.819+0000 7f2203e00640 1 --2- 172.21.15.19:0/1853134953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22041f80b0 0x7f22041f8510 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f21fc000c00 tx=0x7f21fc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.819+0000 7f21fb400640 1 -- 172.21.15.19:0/1853134953 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0072d0 con 0x7f22041f80b0 2024-03-28T13:17:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.819+0000 7f21fb400640 1 -- 172.21.15.19:0/1853134953 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21fc007470 con 0x7f22041f80b0 2024-03-28T13:17:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.819+0000 7f220a400640 1 -- 172.21.15.19:0/1853134953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22040acf10 con 0x7f22041f80b0 2024-03-28T13:17:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.819+0000 7f21fb400640 1 -- 172.21.15.19:0/1853134953 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0049e0 con 0x7f22041f80b0 2024-03-28T13:17:19.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.819+0000 7f220a400640 1 -- 172.21.15.19:0/1853134953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2204149b00 con 0x7f22041f80b0 2024-03-28T13:17:19.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.822+0000 7f21fb400640 1 -- 172.21.15.19:0/1853134953 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f21fc01b020 con 0x7f22041f80b0 2024-03-28T13:17:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.823+0000 7f21fb400640 1 --2- 172.21.15.19:0/1853134953 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21e006c560 0x7f21e006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.823+0000 7f21fb400640 1 -- 172.21.15.19:0/1853134953 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f21fc08b310 con 0x7f22041f80b0 2024-03-28T13:17:19.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.824+0000 7f220d037740 1 Processor -- start 2024-03-28T13:17:19.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.824+0000 7f220d037740 1 -- start start 2024-03-28T13:17:19.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.824+0000 7f2203400640 1 --2- 172.21.15.19:0/1853134953 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21e006c560 0x7f21e006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.825+0000 7f220d037740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563475fc4fc0 0x563475fc53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.825+0000 7f220d037740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563475fcade0 0x563475fc9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.825+0000 7f220d037740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634760126d0 0x563476012b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.825+0000 7f220d037740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563475f2f280 con 0x5634760126d0 2024-03-28T13:17:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.825+0000 7f220d037740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563475ed76f0 con 0x563475fcade0 2024-03-28T13:17:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.825+0000 7f220d037740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x563475f2b230 con 0x563475fc4fc0 2024-03-28T13:17:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.825+0000 7f2203e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634760126d0 0x563476012b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.825+0000 7f2203e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634760126d0 0x563476012b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42564/0 (socket says 172.21.15.19:42564) 2024-03-28T13:17:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.825+0000 7f2203e00640 1 -- 172.21.15.19:0/1815314119 learned_addr learned my addr 172.21.15.19:0/1815314119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:19.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.825+0000 7f2203400640 1 --2- 172.21.15.19:0/1853134953 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21e006c560 0x7f21e006ea20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f21f4012350 tx=0x7f21f400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.826+0000 7f2203e00640 1 -- 172.21.15.19:0/1815314119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563475fc4fc0 msgr2=0x563475fc53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.826+0000 7f2203e00640 1 --2- 172.21.15.19:0/1815314119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563475fc4fc0 0x563475fc53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.826+0000 7f2203e00640 1 -- 172.21.15.19:0/1815314119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563475fcade0 msgr2=0x563475fc9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.826+0000 7f2202a00640 1 --2- 172.21.15.19:0/1815314119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563475fcade0 0x563475fc9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.826+0000 7f2203e00640 1 --2- 172.21.15.19:0/1815314119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563475fcade0 0x563475fc9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.826+0000 7f2203e00640 1 -- 172.21.15.19:0/1815314119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563475d0b7e0 con 0x5634760126d0 2024-03-28T13:17:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.826+0000 7f2202a00640 1 --2- 172.21.15.19:0/1815314119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563475fcade0 0x563475fc9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.826+0000 7f2203e00640 1 --2- 172.21.15.19:0/1815314119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634760126d0 0x563476012b50 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f21fc089710 tx=0x7f21fc059980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77db9c90069ab4d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.826+0000 7f21f3e00640 1 -- 172.21.15.19:0/1815314119 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc061da0 con 0x5634760126d0 2024-03-28T13:17:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.826+0000 7f21f3e00640 1 -- 172.21.15.19:0/1815314119 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21fc007620 con 0x5634760126d0 2024-03-28T13:17:19.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.826+0000 7f21f3e00640 1 -- 172.21.15.19:0/1815314119 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0574c0 con 0x5634760126d0 2024-03-28T13:17:19.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.827+0000 7f220d037740 1 -- 172.21.15.19:0/1815314119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634760126d0 msgr2=0x563476012b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.827+0000 7f220d037740 1 --2- 172.21.15.19:0/1815314119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634760126d0 0x563476012b50 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f21fc089710 tx=0x7f21fc059980 comp rx=0 tx=0).stop 2024-03-28T13:17:19.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.827+0000 7f220d037740 1 -- 172.21.15.19:0/1815314119 shutdown_connections 2024-03-28T13:17:19.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.827+0000 7f220d037740 1 --2- 172.21.15.19:0/1815314119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5634760126d0 0x563476012b50 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.827+0000 7f220d037740 1 --2- 172.21.15.19:0/1815314119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563475fcade0 0x563475fc9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.827+0000 7f220d037740 1 --2- 172.21.15.19:0/1815314119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563475fc4fc0 0x563475fc53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.827+0000 7f220d037740 1 -- 172.21.15.19:0/1815314119 >> 172.21.15.19:0/1815314119 conn(0x563475fc7040 msgr2=0x563475fc8900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.827+0000 7f220d037740 1 -- 172.21.15.19:0/1815314119 shutdown_connections 2024-03-28T13:17:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.827+0000 7f220d037740 1 -- 172.21.15.19:0/1815314119 wait complete. 2024-03-28T13:17:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.828+0000 7f220d037740 1 -- start start 2024-03-28T13:17:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.828+0000 7f220d037740 1 Processor -- start 2024-03-28T13:17:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.829+0000 7f220d037740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fcade0 0x563476020980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.829+0000 7f220d037740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634760126d0 0x56347600b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.829+0000 7f220d037740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563475fe9670 0x563475ff8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.829+0000 7f220d037740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563475ff1730 con 0x563475fcade0 2024-03-28T13:17:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.829+0000 7f220d037740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563475eeaaa0 con 0x5634760126d0 2024-03-28T13:17:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.829+0000 7f220d037740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x563475ff1980 con 0x563475fe9670 2024-03-28T13:17:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.829+0000 7f220d037740 10 client.0 _ll_register_callbacks cb 0x563475eea490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:17:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.829+0000 7f2203400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fcade0 0x563476020980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.829+0000 7f2203400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fcade0 0x563476020980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42570/0 (socket says 172.21.15.19:42570) 2024-03-28T13:17:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.829+0000 7f2203400640 1 -- 172.21.15.19:0/2707928026 learned_addr learned my addr 172.21.15.19:0/2707928026 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.830+0000 7f2202a00640 1 --2- 172.21.15.19:0/2707928026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634760126d0 0x56347600b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.830+0000 7f2203400640 1 -- 172.21.15.19:0/2707928026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563475fe9670 msgr2=0x563475ff8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.830+0000 7f2203e00640 1 --2- 172.21.15.19:0/2707928026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563475fe9670 0x563475ff8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.830+0000 7f2203400640 1 --2- 172.21.15.19:0/2707928026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563475fe9670 0x563475ff8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.830+0000 7f2203400640 1 -- 172.21.15.19:0/2707928026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634760126d0 msgr2=0x56347600b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.830+0000 7f2203400640 1 --2- 172.21.15.19:0/2707928026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634760126d0 0x56347600b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.830+0000 7f2203400640 1 -- 172.21.15.19:0/2707928026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563475d6bf70 con 0x563475fcade0 2024-03-28T13:17:19.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.830+0000 7f2202a00640 1 --2- 172.21.15.19:0/2707928026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634760126d0 0x56347600b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:19.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.830+0000 7f2203400640 1 --2- 172.21.15.19:0/2707928026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fcade0 0x563476020980 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f21f40094c0 tx=0x7f21f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4238eb3ec4d64617 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.830+0000 7f21f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.830+0000 7f21f0c00640 1 -- 172.21.15.19:0/2707928026 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4028060 con 0x563475fcade0 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.830+0000 7f21f0c00640 1 -- 172.21.15.19:0/2707928026 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f21f401a0d0 con 0x563475fcade0 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.830+0000 7f220d037740 1 -- 172.21.15.19:0/2707928026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21fc007620 con 0x563475fcade0 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.831+0000 7f21f0c00640 1 -- 172.21.15.19:0/2707928026 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f21f401a2d0 con 0x563475fcade0 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.831+0000 7f220d037740 10 client.7554 fetch_fsmap learned FSMap version 68 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.831+0000 7f220d037740 1 -- 172.21.15.19:0/2707928026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x563475c03170 con 0x563475fcade0 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.831+0000 7f21f0c00640 1 -- 172.21.15.19:0/2707928026 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f402e070 con 0x563475fcade0 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.831+0000 7f21f0c00640 1 -- 172.21.15.19:0/2707928026 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f21f4018470 con 0x563475fcade0 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.831+0000 7f220d037740 10 client.7554 fetch_fsmap finished waiting for FSMap version 68 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.831+0000 7f220d037740 10 client.7554 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.831+0000 7f220d037740 20 client.7554 populate_metadata read hostname 'smithi019' 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.831+0000 7f220d037740 1 --2- 172.21.15.19:0/2707928026 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x563475ff4b40 0x563475ff5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.831+0000 7f220d037740 4 client.7554 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.831+0000 7f220d037740 1 -- 172.21.15.19:0/2707928026 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563475f3c7c0 con 0x563475ff4b40 2024-03-28T13:17:19.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.832+0000 7f2202a00640 1 --2- 172.21.15.19:0/2707928026 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x563475ff4b40 0x563475ff5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.833+0000 7f2202a00640 1 --2- 172.21.15.19:0/2707928026 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x563475ff4b40 0x563475ff5ef0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24e4273aea06ef8c server_cookie=ff5fcce1578c75c5 in_seq=0 out_seq=0 2024-03-28T13:17:19.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.833+0000 7f21f0c00640 10 client.7554 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.835+0000 7f21f0c00640 1 -- 172.21.15.19:0/2707928026 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563475f3c7c0 con 0x563475ff4b40 2024-03-28T13:17:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.835+0000 7f21f0c00640 10 client.7554 handle_command_reply: tid=0 2024-03-28T13:17:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.835+0000 7f220d037740 1 client.7554 shutdown 2024-03-28T13:17:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.835+0000 7f220d037740 1 -- 172.21.15.19:0/2707928026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fcade0 msgr2=0x563476020980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.835+0000 7f220d037740 1 --2- 172.21.15.19:0/2707928026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fcade0 0x563476020980 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f21f40094c0 tx=0x7f21f4009750 comp rx=0 tx=0).stop 2024-03-28T13:17:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.835+0000 7f220d037740 1 -- 172.21.15.19:0/2707928026 shutdown_connections 2024-03-28T13:17:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.835+0000 7f220d037740 1 --2- 172.21.15.19:0/2707928026 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x563475ff4b40 0x563475ff5ef0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.835+0000 7f220d037740 1 --2- 172.21.15.19:0/2707928026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563475fe9670 0x563475ff8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.835+0000 7f220d037740 1 --2- 172.21.15.19:0/2707928026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634760126d0 0x56347600b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.835+0000 7f220d037740 1 --2- 172.21.15.19:0/2707928026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fcade0 0x563476020980 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.835+0000 7f21f0c00640 0 client.7554 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.835+0000 7f220d037740 1 -- 172.21.15.19:0/2707928026 >> 172.21.15.19:0/2707928026 conn(0x563475fc4fc0 msgr2=0x563476011140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.835+0000 7f220d037740 1 -- 172.21.15.19:0/2707928026 shutdown_connections 2024-03-28T13:17:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.835+0000 7f220d037740 1 -- 172.21.15.19:0/2707928026 wait complete. 2024-03-28T13:17:19.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.835+0000 7f220d037740 20 client.7554 trim_cache size 0 max 16384 2024-03-28T13:17:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.855+0000 7f220d037740 1 Processor -- start 2024-03-28T13:17:19.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.855+0000 7f220d037740 1 -- start start 2024-03-28T13:17:19.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f220d037740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563475fe9670 0x563476034810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f220d037740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5634760126d0 0x5634760106a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f220d037740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fc4b90 0x56347600c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f220d037740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2204201250 con 0x563475fc4b90 2024-03-28T13:17:19.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f220d037740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2204200270 con 0x563475fe9670 2024-03-28T13:17:19.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f220d037740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f220414abc0 con 0x5634760126d0 2024-03-28T13:17:19.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f2203e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fc4b90 0x56347600c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f2203e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fc4b90 0x56347600c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42586/0 (socket says 172.21.15.19:42586) 2024-03-28T13:17:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f2203e00640 1 -- 172.21.15.19:0/4269660927 learned_addr learned my addr 172.21.15.19:0/4269660927 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f2203400640 1 --2- 172.21.15.19:0/4269660927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563475fe9670 0x563476034810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f2202a00640 1 --2- 172.21.15.19:0/4269660927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5634760126d0 0x5634760106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f2203e00640 1 -- 172.21.15.19:0/4269660927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5634760126d0 msgr2=0x5634760106a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f2203e00640 1 --2- 172.21.15.19:0/4269660927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5634760126d0 0x5634760106a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f2203e00640 1 -- 172.21.15.19:0/4269660927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563475fe9670 msgr2=0x563476034810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f2203e00640 1 --2- 172.21.15.19:0/4269660927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563475fe9670 0x563476034810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f2203e00640 1 -- 172.21.15.19:0/4269660927 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563475fc3e90 con 0x563475fc4b90 2024-03-28T13:17:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.856+0000 7f2203400640 1 --2- 172.21.15.19:0/4269660927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563475fe9670 0x563476034810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.857+0000 7f2203e00640 1 --2- 172.21.15.19:0/4269660927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fc4b90 0x56347600c1c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f21fc058e70 tx=0x7f21fc08b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e22306ad4bcefe server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.857+0000 7f21f3400640 1 -- 172.21.15.19:0/4269660927 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc057260 con 0x563475fc4b90 2024-03-28T13:17:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.857+0000 7f21f3400640 1 -- 172.21.15.19:0/4269660927 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21fc0574c0 con 0x563475fc4b90 2024-03-28T13:17:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.857+0000 7f21f3400640 1 -- 172.21.15.19:0/4269660927 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc05d120 con 0x563475fc4b90 2024-03-28T13:17:19.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.858+0000 7f220d037740 1 -- 172.21.15.19:0/4269660927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fc4b90 msgr2=0x56347600c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.858+0000 7f220d037740 1 --2- 172.21.15.19:0/4269660927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fc4b90 0x56347600c1c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f21fc058e70 tx=0x7f21fc08b050 comp rx=0 tx=0).stop 2024-03-28T13:17:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.858+0000 7f220d037740 1 -- 172.21.15.19:0/4269660927 shutdown_connections 2024-03-28T13:17:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.858+0000 7f220d037740 1 --2- 172.21.15.19:0/4269660927 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fc4b90 0x56347600c1c0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.858+0000 7f220d037740 1 --2- 172.21.15.19:0/4269660927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5634760126d0 0x5634760106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.858+0000 7f220d037740 1 --2- 172.21.15.19:0/4269660927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563475fe9670 0x563476034810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.858+0000 7f220d037740 1 -- 172.21.15.19:0/4269660927 >> 172.21.15.19:0/4269660927 conn(0x563475fc4fc0 msgr2=0x563475ff6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.858+0000 7f220d037740 1 -- 172.21.15.19:0/4269660927 shutdown_connections 2024-03-28T13:17:19.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.858+0000 7f220d037740 1 -- 172.21.15.19:0/4269660927 wait complete. 2024-03-28T13:17:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.858+0000 7f220d037740 1 -- start start 2024-03-28T13:17:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.859+0000 7f220d037740 1 Processor -- start 2024-03-28T13:17:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f220d037740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fc4b90 0x563475fe6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f220d037740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563475fe9670 0x563475fe7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f220d037740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634760126d0 0x56347600f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f220d037740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563475ff1570 con 0x563475fc4b90 2024-03-28T13:17:19.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f220d037740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f220414b210 con 0x5634760126d0 2024-03-28T13:17:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f220d037740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x563475fc0d50 con 0x563475fe9670 2024-03-28T13:17:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f220d037740 10 client.0 _ll_register_callbacks cb 0x7f21f4009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:17:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f2203400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fc4b90 0x563475fe6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f2203400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fc4b90 0x563475fe6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42588/0 (socket says 172.21.15.19:42588) 2024-03-28T13:17:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f2203400640 1 -- 172.21.15.19:0/2504330597 learned_addr learned my addr 172.21.15.19:0/2504330597 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f2203e00640 1 --2- 172.21.15.19:0/2504330597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634760126d0 0x56347600f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f2202a00640 1 --2- 172.21.15.19:0/2504330597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563475fe9670 0x563475fe7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f2203400640 1 -- 172.21.15.19:0/2504330597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563475fe9670 msgr2=0x563475fe7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f2203400640 1 --2- 172.21.15.19:0/2504330597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563475fe9670 0x563475fe7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f2203400640 1 -- 172.21.15.19:0/2504330597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634760126d0 msgr2=0x56347600f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.860+0000 7f2203400640 1 --2- 172.21.15.19:0/2504330597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634760126d0 0x56347600f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.861+0000 7f2203400640 1 -- 172.21.15.19:0/2504330597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563475ff1e90 con 0x563475fc4b90 2024-03-28T13:17:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.861+0000 7f2203400640 1 --2- 172.21.15.19:0/2504330597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fc4b90 0x563475fe6ae0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f21f400b010 tx=0x7f21f4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d65eb8b670041b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.861+0000 7f21f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:17:19.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.861+0000 7f21f0c00640 1 -- 172.21.15.19:0/2504330597 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4018470 con 0x563475fc4b90 2024-03-28T13:17:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.861+0000 7f220d037740 1 -- 172.21.15.19:0/2504330597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21fc0574c0 con 0x563475fc4b90 2024-03-28T13:17:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.861+0000 7f21f0c00640 1 -- 172.21.15.19:0/2504330597 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f21f4018610 con 0x563475fc4b90 2024-03-28T13:17:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.861+0000 7f21f0c00640 1 -- 172.21.15.19:0/2504330597 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f21f403f050 con 0x563475fc4b90 2024-03-28T13:17:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.862+0000 7f220d037740 10 client.7560 fetch_fsmap learned FSMap version 68 2024-03-28T13:17:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.862+0000 7f220d037740 1 -- 172.21.15.19:0/2504330597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x563475fc8e40 con 0x563475fc4b90 2024-03-28T13:17:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.862+0000 7f21f0c00640 1 -- 172.21.15.19:0/2504330597 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21fc0574c0 con 0x563475fc4b90 2024-03-28T13:17:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.862+0000 7f21f0c00640 1 -- 172.21.15.19:0/2504330597 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f21f4042020 con 0x563475fc4b90 2024-03-28T13:17:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.862+0000 7f220d037740 10 client.7560 fetch_fsmap finished waiting for FSMap version 68 2024-03-28T13:17:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.862+0000 7f220d037740 10 client.7560 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-28T13:17:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.862+0000 7f220d037740 20 client.7560 populate_metadata read hostname 'smithi019' 2024-03-28T13:17:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.862+0000 7f220d037740 1 --2- 172.21.15.19:0/2504330597 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x563475ff0190 0x56347600f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.862+0000 7f220d037740 4 client.7560 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-03-28T13:17:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.862+0000 7f220d037740 1 -- 172.21.15.19:0/2504330597 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x563475f36b10 con 0x563475ff0190 2024-03-28T13:17:19.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.863+0000 7f2202a00640 1 --2- 172.21.15.19:0/2504330597 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x563475ff0190 0x56347600f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.864+0000 7f2202a00640 1 --2- 172.21.15.19:0/2504330597 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x563475ff0190 0x56347600f590 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37f27dc7c2767080 server_cookie=466443f227c1c8c1 in_seq=0 out_seq=0 2024-03-28T13:17:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.864+0000 7f21f0c00640 10 client.7560 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.865+0000 7f21f0c00640 1 -- 172.21.15.19:0/2504330597 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563475f36b10 con 0x563475ff0190 2024-03-28T13:17:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.865+0000 7f21f0c00640 10 client.7560 handle_command_reply: tid=0 2024-03-28T13:17:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.865+0000 7f220d037740 1 client.7560 shutdown 2024-03-28T13:17:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.865+0000 7f220d037740 1 -- 172.21.15.19:0/2504330597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fc4b90 msgr2=0x563475fe6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.865+0000 7f220d037740 1 --2- 172.21.15.19:0/2504330597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fc4b90 0x563475fe6ae0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f21f400b010 tx=0x7f21f4009f90 comp rx=0 tx=0).stop 2024-03-28T13:17:19.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.865+0000 7f220d037740 1 -- 172.21.15.19:0/2504330597 shutdown_connections 2024-03-28T13:17:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.865+0000 7f220d037740 1 --2- 172.21.15.19:0/2504330597 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x563475ff0190 0x56347600f590 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.865+0000 7f220d037740 1 --2- 172.21.15.19:0/2504330597 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5634760126d0 0x56347600f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.865+0000 7f220d037740 1 --2- 172.21.15.19:0/2504330597 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563475fe9670 0x563475fe7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.865+0000 7f220d037740 1 --2- 172.21.15.19:0/2504330597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563475fc4b90 0x563475fe6ae0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.865+0000 7f21f0c00640 0 client.7560 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.865+0000 7f220d037740 1 -- 172.21.15.19:0/2504330597 >> 172.21.15.19:0/2504330597 conn(0x563475fc4fc0 msgr2=0x56347600bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.865+0000 7f220d037740 1 -- 172.21.15.19:0/2504330597 shutdown_connections 2024-03-28T13:17:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.865+0000 7f220d037740 1 -- 172.21.15.19:0/2504330597 wait complete. 2024-03-28T13:17:19.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.865+0000 7f220d037740 20 client.7560 trim_cache size 0 max 16384 2024-03-28T13:17:19.867 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:17:19.867 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-03-28T13:17:19.868 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "cb2b574d-52ab-4b7b-8395-505791b23034", 2024-03-28T13:17:19.868 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-03-28T13:17:19.868 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:17:19.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.866+0000 7f220a400640 1 -- 172.21.15.19:0/1853134953 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21e006c560 msgr2=0x7f21e006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.866+0000 7f220a400640 1 --2- 172.21.15.19:0/1853134953 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21e006c560 0x7f21e006ea20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f21f4012350 tx=0x7f21f400b040 comp rx=0 tx=0).stop 2024-03-28T13:17:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.866+0000 7f220a400640 1 -- 172.21.15.19:0/1853134953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22041f80b0 msgr2=0x7f22041f8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.866+0000 7f220a400640 1 --2- 172.21.15.19:0/1853134953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22041f80b0 0x7f22041f8510 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f21fc000c00 tx=0x7f21fc00dc00 comp rx=0 tx=0).stop 2024-03-28T13:17:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.866+0000 7f220a400640 1 -- 172.21.15.19:0/1853134953 shutdown_connections 2024-03-28T13:17:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.866+0000 7f220a400640 1 --2- 172.21.15.19:0/1853134953 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21e006c560 0x7f21e006ea20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.866+0000 7f220a400640 1 --2- 172.21.15.19:0/1853134953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22041f80b0 0x7f22041f8510 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.866+0000 7f220a400640 1 --2- 172.21.15.19:0/1853134953 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f220409dd30 0x7f22041f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.866+0000 7f220a400640 1 --2- 172.21.15.19:0/1853134953 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e8004800 0x7f22041f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.866+0000 7f220a400640 1 -- 172.21.15.19:0/1853134953 >> 172.21.15.19:0/1853134953 conn(0x7f220406bba0 msgr2=0x7f220415eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.866+0000 7f220a400640 1 -- 172.21.15.19:0/1853134953 shutdown_connections 2024-03-28T13:17:19.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:19.866+0000 7f220a400640 1 -- 172.21.15.19:0/1853134953 wait complete. 2024-03-28T13:17:19.869 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-03-28T13:17:19.868+0000 7f8097200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-03-28T13:16:35.247197+0000 1=1+0)->f(v1 m2024-03-28T13:16:35.247197+0000) n(v0 rc2024-03-28T13:16:35.374196+0000 2=1+1)->n(v0 rc2024-03-28T13:16:35.374196+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5622e97f0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-28T13:16:35.247197+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-28T13:16:35.374196+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-03-28T13:17:19.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:17:19.886 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-28T13:17:20.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.102+0000 7f0fa2800640 1 Processor -- start 2024-03-28T13:17:20.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.102+0000 7f0fa2800640 1 -- start start 2024-03-28T13:17:20.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.104+0000 7f0fa2800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f9c06dd30 0x7f0f9c0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.104+0000 7f0fa2800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f9c0ce890 con 0x7f0f9c0cc880 2024-03-28T13:17:20.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.104+0000 7f0fa2800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f9c0ceb50 con 0x7f0f9c13f190 2024-03-28T13:17:20.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.104+0000 7f0fa2800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9c0cd230 con 0x7f0f9c06dd30 2024-03-28T13:17:20.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.104+0000 7f0fa0a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0f9c0cc880 0x7f0f9c06d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37122/0 (socket says 172.21.15.19:37122) 2024-03-28T13:17:20.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.104+0000 7f0fa0a00640 1 -- 172.21.15.19:0/824451457 learned_addr learned my addr 172.21.15.19:0/824451457 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:17:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.105+0000 7f0fa1e00640 1 --2- 172.21.15.19:0/824451457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f9c06dd30 0x7f0f9c0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.105+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1172817413 0 0) 0x7f0f9c0ce890 con 0x7f0f9c0cc880 2024-03-28T13:17:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.105+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f84003520 con 0x7f0f9c0cc880 2024-03-28T13:17:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.105+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3196964036 0 0) 0x7f0f9c0ceb50 con 0x7f0f9c13f190 2024-03-28T13:17:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.105+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f9c0ce890 con 0x7f0f9c13f190 2024-03-28T13:17:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.105+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1313064033 0 0) 0x7f0f84003520 con 0x7f0f9c0cc880 2024-03-28T13:17:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.105+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f9c06dd30 msgr2=0x7f0f9c0cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.105+0000 7f0f9be00640 1 --2- 172.21.15.19:0/824451457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f9c06dd30 0x7f0f9c0cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.105+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 >> v1:172.21.15.188:6789/0 conn(0x7f0f9c13f190 legacy=0x7f0f9c0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.105+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9c0cee30 con 0x7f0f9c0cc880 2024-03-28T13:17:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.105+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f0f88001690 con 0x7f0f9c0cc880 2024-03-28T13:17:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.105+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 >> v1:172.21.15.19:6789/0 conn(0x7f0f9c0cc880 legacy=0x7f0f9c06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.105+0000 7f0f9be00640 1 --2- 172.21.15.19:0/824451457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84003b70 0x7f0f84003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.105+0000 7f0f9be00640 1 --2- 172.21.15.19:0/824451457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f84004800 0x7f0f84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.106+0000 7f0f9be00640 1 --2- 172.21.15.19:0/824451457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f9c06dd30 0x7f0f840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.106+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c0cd230 con 0x7f0f84003b70 2024-03-28T13:17:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.106+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8400baa0 con 0x7f0f84004800 2024-03-28T13:17:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.106+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8400bca0 con 0x7f0f9c06dd30 2024-03-28T13:17:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.106+0000 7f0fa1400640 1 --2- 172.21.15.19:0/824451457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84003b70 0x7f0f84003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.106+0000 7f0fa0a00640 1 --2- 172.21.15.19:0/824451457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f84004800 0x7f0f84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.106+0000 7f0fa1400640 1 -- 172.21.15.19:0/824451457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f9c06dd30 msgr2=0x7f0f840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.106+0000 7f0fa1e00640 1 --2- 172.21.15.19:0/824451457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f9c06dd30 0x7f0f840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.106+0000 7f0fa1400640 1 --2- 172.21.15.19:0/824451457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f9c06dd30 0x7f0f840051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.106+0000 7f0fa1400640 1 -- 172.21.15.19:0/824451457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f84004800 msgr2=0x7f0f84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.106+0000 7f0fa1400640 1 --2- 172.21.15.19:0/824451457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f84004800 0x7f0f84004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.106+0000 7f0fa1400640 1 -- 172.21.15.19:0/824451457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f84003520 con 0x7f0f84003b70 2024-03-28T13:17:20.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.106+0000 7f0fa0a00640 1 --2- 172.21.15.19:0/824451457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f84004800 0x7f0f84004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:17:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.108+0000 7f0fa1400640 1 --2- 172.21.15.19:0/824451457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84003b70 0x7f0f84003f70 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f0f900018d0 tx=0x7f0f84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe62cfce18eca18 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:20.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.108+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f90037070 con 0x7f0f84003b70 2024-03-28T13:17:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.108+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f900052d0 con 0x7f0f84003b70 2024-03-28T13:17:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.108+0000 7f0f9be00640 1 -- 172.21.15.19:0/824451457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f900014f0 con 0x7f0f84003b70 2024-03-28T13:17:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.108+0000 7f0fa2800640 1 -- 172.21.15.19:0/824451457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84003b70 msgr2=0x7f0f84003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.108+0000 7f0fa2800640 1 --2- 172.21.15.19:0/824451457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84003b70 0x7f0f84003f70 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f0f900018d0 tx=0x7f0f84005830 comp rx=0 tx=0).stop 2024-03-28T13:17:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.108+0000 7f0fa1e00640 1 -- 172.21.15.19:0/824451457 reap_dead start 2024-03-28T13:17:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.109+0000 7f0fa2800640 1 -- 172.21.15.19:0/824451457 shutdown_connections 2024-03-28T13:17:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.109+0000 7f0fa2800640 1 -- 172.21.15.19:0/824451457 >> 172.21.15.19:0/824451457 conn(0x7f0f9c0b1ba0 msgr2=0x7f0f9c0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.109+0000 7f0fa2800640 1 -- 172.21.15.19:0/824451457 shutdown_connections 2024-03-28T13:17:20.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.109+0000 7f0fa2800640 1 -- 172.21.15.19:0/824451457 wait complete. 2024-03-28T13:17:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.111+0000 7f0fa2800640 1 Processor -- start 2024-03-28T13:17:20.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.111+0000 7f0fa2800640 1 -- start start 2024-03-28T13:17:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa2800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f84003b70 0x7f0f9c13e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa2800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f9c13ec50 0x7f0f9c1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c0a9480 0x7f0f9c0a9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa2800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c13f570 con 0x7f0f9c0a9480 2024-03-28T13:17:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa2800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c160c70 con 0x7f0f84003b70 2024-03-28T13:17:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa2800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9c0cccf0 con 0x7f0f9c13ec50 2024-03-28T13:17:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c0a9480 0x7f0f9c0a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c0a9480 0x7f0f9c0a9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42598/0 (socket says 172.21.15.19:42598) 2024-03-28T13:17:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa1e00640 1 -- 172.21.15.19:0/573416042 learned_addr learned my addr 172.21.15.19:0/573416042 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa1400640 1 --2- 172.21.15.19:0/573416042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f84003b70 0x7f0f9c13e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa0a00640 1 --2- 172.21.15.19:0/573416042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f9c13ec50 0x7f0f9c1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa1e00640 1 -- 172.21.15.19:0/573416042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f9c13ec50 msgr2=0x7f0f9c1592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:20.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa1e00640 1 --2- 172.21.15.19:0/573416042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f9c13ec50 0x7f0f9c1592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa1e00640 1 -- 172.21.15.19:0/573416042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f84003b70 msgr2=0x7f0f9c13e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa1e00640 1 --2- 172.21.15.19:0/573416042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f84003b70 0x7f0f9c13e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.112+0000 7f0fa1e00640 1 -- 172.21.15.19:0/573416042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9c159bf0 con 0x7f0f9c0a9480 2024-03-28T13:17:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.113+0000 7f0fa0a00640 1 --2- 172.21.15.19:0/573416042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f9c13ec50 0x7f0f9c1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.113+0000 7f0fa1e00640 1 --2- 172.21.15.19:0/573416042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c0a9480 0x7f0f9c0a9900 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0b2c40 tx=0x7f0f9400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.113+0000 7f0f99600640 1 -- 172.21.15.19:0/573416042 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9401c070 con 0x7f0f9c0a9480 2024-03-28T13:17:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.113+0000 7f0f99600640 1 -- 172.21.15.19:0/573416042 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f94017400 con 0x7f0f9c0a9480 2024-03-28T13:17:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.113+0000 7f0f99600640 1 -- 172.21.15.19:0/573416042 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f940177f0 con 0x7f0f9c0a9480 2024-03-28T13:17:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.113+0000 7f0fa2800640 1 -- 172.21.15.19:0/573416042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f9c159ed0 con 0x7f0f9c0a9480 2024-03-28T13:17:20.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.113+0000 7f0fa2800640 1 -- 172.21.15.19:0/573416042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f9c15a490 con 0x7f0f9c0a9480 2024-03-28T13:17:20.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.116+0000 7f0f99600640 1 -- 172.21.15.19:0/573416042 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0f94005d10 con 0x7f0f9c0a9480 2024-03-28T13:17:20.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.116+0000 7f0f99600640 1 --2- 172.21.15.19:0/573416042 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f7c06c560 0x7f0f7c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.116+0000 7f0f99600640 1 -- 172.21.15.19:0/573416042 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f0f9408b8b0 con 0x7f0f9c0a9480 2024-03-28T13:17:20.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.117+0000 7f0fa1400640 1 --2- 172.21.15.19:0/573416042 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f7c06c560 0x7f0f7c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.117+0000 7f0fa6b87740 1 Processor -- start 2024-03-28T13:17:20.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.117+0000 7f0fa6b87740 1 -- start start 2024-03-28T13:17:20.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.118+0000 7f0fa1400640 1 --2- 172.21.15.19:0/573416042 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f7c06c560 0x7f0f7c06ea20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0f90034000 tx=0x7f0f90031000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:20.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.118+0000 7f0fa6b87740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x5586317926c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.118+0000 7f0fa6b87740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317bfb00 0x5586317c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.118+0000 7f0fa6b87740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558631792c00 0x5586317c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.118+0000 7f0fa6b87740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5586316fd6b0 con 0x5586317bfb00 2024-03-28T13:17:20.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.118+0000 7f0fa6b87740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5586316a5750 con 0x5586317bf720 2024-03-28T13:17:20.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.118+0000 7f0fa6b87740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5586316f9660 con 0x558631792c00 2024-03-28T13:17:20.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.119+0000 7f0fa0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317bfb00 0x5586317c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.119+0000 7f0fa0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317bfb00 0x5586317c24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42608/0 (socket says 172.21.15.19:42608) 2024-03-28T13:17:20.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.119+0000 7f0fa0a00640 1 -- 172.21.15.19:0/1109493945 learned_addr learned my addr 172.21.15.19:0/1109493945 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.119+0000 7f0fa1400640 1 --2- 172.21.15.19:0/1109493945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x5586317926c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.119+0000 7f0fa0a00640 1 -- 172.21.15.19:0/1109493945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558631792c00 msgr2=0x5586317c29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.119+0000 7f0fa1e00640 1 --2- 172.21.15.19:0/1109493945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558631792c00 0x5586317c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.119+0000 7f0fa0a00640 1 --2- 172.21.15.19:0/1109493945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558631792c00 0x5586317c29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.119+0000 7f0fa0a00640 1 -- 172.21.15.19:0/1109493945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 msgr2=0x5586317926c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.119+0000 7f0fa0a00640 1 --2- 172.21.15.19:0/1109493945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x5586317926c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.119+0000 7f0fa0a00640 1 -- 172.21.15.19:0/1109493945 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586314db7d0 con 0x5586317bfb00 2024-03-28T13:17:20.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.119+0000 7f0fa0a00640 1 --2- 172.21.15.19:0/1109493945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317bfb00 0x5586317c24b0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f0f880058d0 tx=0x7f0f88005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58388f8a71cebea0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.120+0000 7f0f8dc00640 1 -- 172.21.15.19:0/1109493945 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88019070 con 0x5586317bfb00 2024-03-28T13:17:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.120+0000 7f0f8dc00640 1 -- 172.21.15.19:0/1109493945 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f8800f8f0 con 0x5586317bfb00 2024-03-28T13:17:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.120+0000 7f0f8dc00640 1 -- 172.21.15.19:0/1109493945 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88010040 con 0x5586317bfb00 2024-03-28T13:17:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.120+0000 7f0fa6b87740 1 -- 172.21.15.19:0/1109493945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317bfb00 msgr2=0x5586317c24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:20.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.120+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/1109493945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317bfb00 0x5586317c24b0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f0f880058d0 tx=0x7f0f88005d90 comp rx=0 tx=0).stop 2024-03-28T13:17:20.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.121+0000 7f0fa6b87740 1 -- 172.21.15.19:0/1109493945 shutdown_connections 2024-03-28T13:17:20.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.121+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/1109493945 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558631792c00 0x5586317c29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.121+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/1109493945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317bfb00 0x5586317c24b0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.121+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/1109493945 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x5586317926c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.121+0000 7f0fa6b87740 1 -- 172.21.15.19:0/1109493945 >> 172.21.15.19:0/1109493945 conn(0x558631794930 msgr2=0x5586317dd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:20.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.121+0000 7f0fa6b87740 1 -- 172.21.15.19:0/1109493945 shutdown_connections 2024-03-28T13:17:20.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.121+0000 7f0fa6b87740 1 -- 172.21.15.19:0/1109493945 wait complete. 2024-03-28T13:17:20.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.121+0000 7f0fa6b87740 1 -- start start 2024-03-28T13:17:20.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.122+0000 7f0fa6b87740 1 Processor -- start 2024-03-28T13:17:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.123+0000 7f0fa6b87740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x5586317c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.123+0000 7f0fa6b87740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317bfb00 0x5586317c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.123+0000 7f0fa6b87740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5586317ee410 0x5586317dd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.123+0000 7f0fa6b87740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558631796fd0 con 0x5586317bfb00 2024-03-28T13:17:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.123+0000 7f0fa6b87740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5586316b8a80 con 0x5586317bf720 2024-03-28T13:17:20.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.123+0000 7f0fa6b87740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x558631799810 con 0x5586317ee410 2024-03-28T13:17:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.123+0000 7f0fa6b87740 10 client.0 _ll_register_callbacks cb 0x5586316b8470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:17:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.123+0000 7f0fa0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317bfb00 0x5586317c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.123+0000 7f0fa0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317bfb00 0x5586317c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42612/0 (socket says 172.21.15.19:42612) 2024-03-28T13:17:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.123+0000 7f0fa0a00640 1 -- 172.21.15.19:0/15326303 learned_addr learned my addr 172.21.15.19:0/15326303 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.123+0000 7f0fa1400640 1 --2- 172.21.15.19:0/15326303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x5586317c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.123+0000 7f0fa1e00640 1 --2- 172.21.15.19:0/15326303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5586317ee410 0x5586317dd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.123+0000 7f0fa0a00640 1 -- 172.21.15.19:0/15326303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5586317ee410 msgr2=0x5586317dd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.123+0000 7f0fa0a00640 1 --2- 172.21.15.19:0/15326303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5586317ee410 0x5586317dd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.124+0000 7f0fa0a00640 1 -- 172.21.15.19:0/15326303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 msgr2=0x5586317c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.124+0000 7f0fa0a00640 1 --2- 172.21.15.19:0/15326303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x5586317c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.124+0000 7f0fa0a00640 1 -- 172.21.15.19:0/15326303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586314113f0 con 0x5586317bfb00 2024-03-28T13:17:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.124+0000 7f0fa1e00640 1 --2- 172.21.15.19:0/15326303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5586317ee410 0x5586317dd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.124+0000 7f0fa0a00640 1 --2- 172.21.15.19:0/15326303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317bfb00 0x5586317c5fc0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f0f88014000 tx=0x7f0f88001990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbcba66e1f409af server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:20.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.124+0000 7f0f82000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:17:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.124+0000 7f0f82000640 1 -- 172.21.15.19:0/15326303 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88019050 con 0x5586317bfb00 2024-03-28T13:17:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.124+0000 7f0f82000640 1 -- 172.21.15.19:0/15326303 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f0f88015940 con 0x5586317bfb00 2024-03-28T13:17:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.124+0000 7f0fa6b87740 1 -- 172.21.15.19:0/15326303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f8800f8f0 con 0x5586317bfb00 2024-03-28T13:17:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.124+0000 7f0f82000640 1 -- 172.21.15.19:0/15326303 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f0f88015b40 con 0x5586317bfb00 2024-03-28T13:17:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.124+0000 7f0fa6b87740 10 client.7572 fetch_fsmap learned FSMap version 68 2024-03-28T13:17:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.124+0000 7f0fa6b87740 1 -- 172.21.15.19:0/15326303 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x5586313d10f0 con 0x5586317bfb00 2024-03-28T13:17:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.125+0000 7f0f82000640 1 -- 172.21.15.19:0/15326303 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f8800f8f0 con 0x5586317bfb00 2024-03-28T13:17:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.125+0000 7f0f82000640 1 -- 172.21.15.19:0/15326303 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f0f88030020 con 0x5586317bfb00 2024-03-28T13:17:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.125+0000 7f0fa6b87740 10 client.7572 fetch_fsmap finished waiting for FSMap version 68 2024-03-28T13:17:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.125+0000 7f0fa6b87740 10 client.7572 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-28T13:17:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.125+0000 7f0fa6b87740 20 client.7572 populate_metadata read hostname 'smithi019' 2024-03-28T13:17:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.125+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/15326303 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5586317f12c0 0x5586317f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.125+0000 7f0fa6b87740 4 client.7572 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:17:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.125+0000 7f0fa6b87740 1 -- 172.21.15.19:0/15326303 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55863170abf0 con 0x5586317f12c0 2024-03-28T13:17:20.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.126+0000 7f0fa1400640 1 --2- 172.21.15.19:0/15326303 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5586317f12c0 0x5586317f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.127+0000 7f0fa1400640 1 --2- 172.21.15.19:0/15326303 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5586317f12c0 0x5586317f16d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=798f38d6ed7e3815 server_cookie=5141bf3021d82dd2 in_seq=0 out_seq=0 2024-03-28T13:17:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.127+0000 7f0f82000640 10 client.7572 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.128+0000 7f0f82000640 1 -- 172.21.15.19:0/15326303 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55863170abf0 con 0x5586317f12c0 2024-03-28T13:17:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.129+0000 7f0f82000640 10 client.7572 handle_command_reply: tid=0 2024-03-28T13:17:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.129+0000 7f0fa6b87740 1 client.7572 shutdown 2024-03-28T13:17:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.129+0000 7f0fa6b87740 1 -- 172.21.15.19:0/15326303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317bfb00 msgr2=0x5586317c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.129+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/15326303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317bfb00 0x5586317c5fc0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f0f88014000 tx=0x7f0f88001990 comp rx=0 tx=0).stop 2024-03-28T13:17:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.129+0000 7f0fa6b87740 1 -- 172.21.15.19:0/15326303 shutdown_connections 2024-03-28T13:17:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.129+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/15326303 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5586317f12c0 0x5586317f16d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.129+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/15326303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5586317ee410 0x5586317dd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.129+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/15326303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317bfb00 0x5586317c5fc0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.129+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/15326303 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x5586317c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.129+0000 7f0f82000640 0 client.7572 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.129+0000 7f0fa6b87740 1 -- 172.21.15.19:0/15326303 >> 172.21.15.19:0/15326303 conn(0x558631792c00 msgr2=0x5586317966a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.129+0000 7f0fa6b87740 1 -- 172.21.15.19:0/15326303 shutdown_connections 2024-03-28T13:17:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.129+0000 7f0fa6b87740 1 -- 172.21.15.19:0/15326303 wait complete. 2024-03-28T13:17:20.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.129+0000 7f0fa6b87740 20 client.7572 trim_cache size 0 max 16384 2024-03-28T13:17:20.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.148+0000 7f0fa6b87740 1 Processor -- start 2024-03-28T13:17:20.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.149+0000 7f0fa6b87740 1 -- start start 2024-03-28T13:17:20.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.149+0000 7f0fa6b87740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x5586317ed330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.149+0000 7f0fa6b87740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5586317bfb00 0x5586317df800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.149+0000 7f0fa6b87740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317ee410 0x5586317bea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.149+0000 7f0fa6b87740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1fcc90 con 0x5586317ee410 2024-03-28T13:17:20.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.149+0000 7f0fa6b87740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1fbcb0 con 0x5586317bf720 2024-03-28T13:17:20.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.149+0000 7f0fa6b87740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9c15b830 con 0x5586317bfb00 2024-03-28T13:17:20.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.150+0000 7f0fa1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317ee410 0x5586317bea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.150+0000 7f0fa1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317ee410 0x5586317bea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42626/0 (socket says 172.21.15.19:42626) 2024-03-28T13:17:20.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.150+0000 7f0fa1e00640 1 -- 172.21.15.19:0/726594864 learned_addr learned my addr 172.21.15.19:0/726594864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:20.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.150+0000 7f0fa0a00640 1 --2- 172.21.15.19:0/726594864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5586317bfb00 0x5586317df800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.150+0000 7f0fa1400640 1 --2- 172.21.15.19:0/726594864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x5586317ed330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.150+0000 7f0fa1e00640 1 -- 172.21.15.19:0/726594864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5586317bfb00 msgr2=0x5586317df800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:20.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.150+0000 7f0fa1e00640 1 --2- 172.21.15.19:0/726594864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5586317bfb00 0x5586317df800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.150+0000 7f0fa1e00640 1 -- 172.21.15.19:0/726594864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 msgr2=0x5586317ed330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:20.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.150+0000 7f0fa1e00640 1 --2- 172.21.15.19:0/726594864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x5586317ed330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.150+0000 7f0fa1e00640 1 -- 172.21.15.19:0/726594864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558631793410 con 0x5586317ee410 2024-03-28T13:17:20.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.151+0000 7f0fa1e00640 1 --2- 172.21.15.19:0/726594864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317ee410 0x5586317bea40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0f94059480 tx=0x7f0f94060040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381fc84efcb110b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:20.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.151+0000 7f0f8d200640 1 -- 172.21.15.19:0/726594864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94057a80 con 0x5586317ee410 2024-03-28T13:17:20.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.151+0000 7f0f8d200640 1 -- 172.21.15.19:0/726594864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f94055df0 con 0x5586317ee410 2024-03-28T13:17:20.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.151+0000 7f0f8d200640 1 -- 172.21.15.19:0/726594864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94059c00 con 0x5586317ee410 2024-03-28T13:17:20.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.151+0000 7f0fa6b87740 1 -- 172.21.15.19:0/726594864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317ee410 msgr2=0x5586317bea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.151+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/726594864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317ee410 0x5586317bea40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0f94059480 tx=0x7f0f94060040 comp rx=0 tx=0).stop 2024-03-28T13:17:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.151+0000 7f0fa6b87740 1 -- 172.21.15.19:0/726594864 shutdown_connections 2024-03-28T13:17:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.151+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/726594864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317ee410 0x5586317bea40 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.151+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/726594864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5586317bfb00 0x5586317df800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.151+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/726594864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x5586317ed330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.151+0000 7f0fa6b87740 1 -- 172.21.15.19:0/726594864 >> 172.21.15.19:0/726594864 conn(0x558631792c00 msgr2=0x5586317e0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.152+0000 7f0fa6b87740 1 -- 172.21.15.19:0/726594864 shutdown_connections 2024-03-28T13:17:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.152+0000 7f0fa6b87740 1 -- 172.21.15.19:0/726594864 wait complete. 2024-03-28T13:17:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.152+0000 7f0fa6b87740 1 -- start start 2024-03-28T13:17:20.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.152+0000 7f0fa6b87740 1 Processor -- start 2024-03-28T13:17:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.153+0000 7f0fa6b87740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x55863183dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.153+0000 7f0fa6b87740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5586317bfb00 0x5586317df790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.153+0000 7f0fa6b87740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317ee410 0x5586317de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.153+0000 7f0fa6b87740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5586317c50e0 con 0x5586317ee410 2024-03-28T13:17:20.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.153+0000 7f0fa6b87740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9c1fb6e0 con 0x5586317bf720 2024-03-28T13:17:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.153+0000 7f0fa6b87740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55863183e500 con 0x5586317bfb00 2024-03-28T13:17:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.154+0000 7f0fa6b87740 10 client.0 _ll_register_callbacks cb 0x7f0f88017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:17:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.154+0000 7f0fa1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317ee410 0x5586317de9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.154+0000 7f0fa1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317ee410 0x5586317de9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42628/0 (socket says 172.21.15.19:42628) 2024-03-28T13:17:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.154+0000 7f0fa1e00640 1 -- 172.21.15.19:0/1588068115 learned_addr learned my addr 172.21.15.19:0/1588068115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.154+0000 7f0fa1400640 1 --2- 172.21.15.19:0/1588068115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x55863183dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.154+0000 7f0fa1e00640 1 -- 172.21.15.19:0/1588068115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5586317bfb00 msgr2=0x5586317df790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.154+0000 7f0fa1e00640 1 --2- 172.21.15.19:0/1588068115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5586317bfb00 0x5586317df790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.154+0000 7f0fa1e00640 1 -- 172.21.15.19:0/1588068115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 msgr2=0x55863183dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.154+0000 7f0fa1e00640 1 --2- 172.21.15.19:0/1588068115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x55863183dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.154+0000 7f0fa1e00640 1 -- 172.21.15.19:0/1588068115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586317c3030 con 0x5586317ee410 2024-03-28T13:17:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0fa1e00640 1 --2- 172.21.15.19:0/1588068115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317ee410 0x5586317de9e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009fd0 tx=0x7f0f94000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79293b571f852a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0f82000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:17:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0f82000640 1 -- 172.21.15.19:0/1588068115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94061e20 con 0x5586317ee410 2024-03-28T13:17:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0f82000640 1 -- 172.21.15.19:0/1588068115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f0f94061fc0 con 0x5586317ee410 2024-03-28T13:17:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0fa6b87740 1 -- 172.21.15.19:0/1588068115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f94055df0 con 0x5586317ee410 2024-03-28T13:17:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0f82000640 1 -- 172.21.15.19:0/1588068115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f0f9407f050 con 0x5586317ee410 2024-03-28T13:17:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0fa6b87740 10 client.7578 fetch_fsmap learned FSMap version 68 2024-03-28T13:17:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0fa6b87740 1 -- 172.21.15.19:0/1588068115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55863183e2a0 con 0x5586317ee410 2024-03-28T13:17:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0f82000640 1 -- 172.21.15.19:0/1588068115 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f94055df0 con 0x5586317ee410 2024-03-28T13:17:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0f82000640 1 -- 172.21.15.19:0/1588068115 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f0f94058bd0 con 0x5586317ee410 2024-03-28T13:17:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0fa6b87740 10 client.7578 fetch_fsmap finished waiting for FSMap version 68 2024-03-28T13:17:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0fa6b87740 10 client.7578 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-28T13:17:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0fa6b87740 20 client.7578 populate_metadata read hostname 'smithi019' 2024-03-28T13:17:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/1588068115 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x558631800910 0x558631800d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0fa6b87740 4 client.7578 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:17:20.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.155+0000 7f0fa6b87740 1 -- 172.21.15.19:0/1588068115 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558631704f40 con 0x558631800910 2024-03-28T13:17:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.156+0000 7f0fa1400640 1 --2- 172.21.15.19:0/1588068115 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x558631800910 0x558631800d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.157+0000 7f0fa1400640 1 --2- 172.21.15.19:0/1588068115 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x558631800910 0x558631800d20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53edabe861ad0765 server_cookie=b754927bc56a9516 in_seq=0 out_seq=0 2024-03-28T13:17:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.157+0000 7f0f82000640 10 client.7578 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.157+0000 7f0f82000640 1 -- 172.21.15.19:0/1588068115 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x558631704f40 con 0x558631800910 2024-03-28T13:17:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.157+0000 7f0f82000640 10 client.7578 handle_command_reply: tid=0 2024-03-28T13:17:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.158+0000 7f0fa6b87740 1 client.7578 shutdown 2024-03-28T13:17:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.158+0000 7f0fa6b87740 1 -- 172.21.15.19:0/1588068115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317ee410 msgr2=0x5586317de9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.158+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/1588068115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317ee410 0x5586317de9e0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f0f94009fd0 tx=0x7f0f94000f40 comp rx=0 tx=0).stop 2024-03-28T13:17:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.158+0000 7f0fa6b87740 1 -- 172.21.15.19:0/1588068115 shutdown_connections 2024-03-28T13:17:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.158+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/1588068115 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x558631800910 0x558631800d20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.158+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/1588068115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5586317ee410 0x5586317de9e0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.158+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/1588068115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5586317bfb00 0x5586317df790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.158+0000 7f0fa6b87740 1 --2- 172.21.15.19:0/1588068115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5586317bf720 0x55863183dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.158+0000 7f0f82000640 0 client.7578 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.158+0000 7f0fa6b87740 1 -- 172.21.15.19:0/1588068115 >> 172.21.15.19:0/1588068115 conn(0x558631792c00 msgr2=0x5586317c3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.158+0000 7f0fa6b87740 1 -- 172.21.15.19:0/1588068115 shutdown_connections 2024-03-28T13:17:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.158+0000 7f0fa6b87740 1 -- 172.21.15.19:0/1588068115 wait complete. 2024-03-28T13:17:20.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.158+0000 7f0fa6b87740 20 client.7578 trim_cache size 0 max 16384 2024-03-28T13:17:20.160 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:17:20.160 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T13:17:20.160 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-03-28T13:17:20.161 INFO:teuthology.orchestra.run.smithi019.stdout: "cb2b574d-52ab-4b7b-8395-505791b23034": { 2024-03-28T13:17:20.161 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/dir", 2024-03-28T13:17:20.161 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "cb2b574d-52ab-4b7b-8395-505791b23034", 2024-03-28T13:17:20.161 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,repair" 2024-03-28T13:17:20.161 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:17:20.161 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:17:20.161 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:17:20.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.159+0000 7f0fa2800640 1 -- 172.21.15.19:0/573416042 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f7c06c560 msgr2=0x7f0f7c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:20.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.159+0000 7f0fa2800640 1 --2- 172.21.15.19:0/573416042 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f7c06c560 0x7f0f7c06ea20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0f90034000 tx=0x7f0f90031000 comp rx=0 tx=0).stop 2024-03-28T13:17:20.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.159+0000 7f0fa2800640 1 -- 172.21.15.19:0/573416042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c0a9480 msgr2=0x7f0f9c0a9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:20.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.159+0000 7f0fa2800640 1 --2- 172.21.15.19:0/573416042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c0a9480 0x7f0f9c0a9900 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0b2c40 tx=0x7f0f9400f4a0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.159+0000 7f0fa2800640 1 -- 172.21.15.19:0/573416042 shutdown_connections 2024-03-28T13:17:20.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.159+0000 7f0fa2800640 1 --2- 172.21.15.19:0/573416042 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f7c06c560 0x7f0f7c06ea20 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.159+0000 7f0fa2800640 1 --2- 172.21.15.19:0/573416042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9c0a9480 0x7f0f9c0a9900 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.159+0000 7f0fa2800640 1 --2- 172.21.15.19:0/573416042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f9c13ec50 0x7f0f9c1592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.159+0000 7f0fa2800640 1 --2- 172.21.15.19:0/573416042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f84003b70 0x7f0f9c13e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:20.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.159+0000 7f0fa2800640 1 -- 172.21.15.19:0/573416042 >> 172.21.15.19:0/573416042 conn(0x7f0f9c0b1ba0 msgr2=0x7f0f9c0b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:20.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.159+0000 7f0fa2800640 1 -- 172.21.15.19:0/573416042 shutdown_connections 2024-03-28T13:17:20.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:20.159+0000 7f0fa2800640 1 -- 172.21.15.19:0/573416042 wait complete. 2024-03-28T13:17:20.176 INFO:tasks.cephfs.filesystem:scrub status for tag:cb2b574d-52ab-4b7b-8395-505791b23034 - {'path': '/dir', 'tag': 'cb2b574d-52ab-4b7b-8395-505791b23034', 'options': 'recursive,repair'} 2024-03-28T13:17:33.443 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:17:33.451 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:17:50.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:17:50.177 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-28T13:17:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.411+0000 7f3c8bc00640 1 Processor -- start 2024-03-28T13:17:50.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.411+0000 7f3c8bc00640 1 -- start start 2024-03-28T13:17:50.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.412+0000 7f3c8bc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c84141c10 0x7f3c84071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.412+0000 7f3c8bc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3c84071eb0 0x7f3c84162160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.412+0000 7f3c8bc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c8415ec80 con 0x7f3c840d0280 2024-03-28T13:17:50.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.412+0000 7f3c8bc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3c8415ef20 con 0x7f3c84071eb0 2024-03-28T13:17:50.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.412+0000 7f3c8bc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3c8415f200 con 0x7f3c84141c10 2024-03-28T13:17:50.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.413+0000 7f3c89200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3c840d0280 0x7f3c840d0660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34756/0 (socket says 172.21.15.19:34756) 2024-03-28T13:17:50.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.413+0000 7f3c89200640 1 -- 172.21.15.19:0/496116462 learned_addr learned my addr 172.21.15.19:0/496116462 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:17:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.413+0000 7f3c89c00640 1 --2- 172.21.15.19:0/496116462 >> v2:172.21.15.188:3300/0 conn(0x7f3c84071eb0 0x7f3c84162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.413+0000 7f3c83e00640 1 --2- 172.21.15.19:0/496116462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c84141c10 0x7f3c84071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.413+0000 7f3c83400640 1 -- 172.21.15.19:0/496116462 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2433323705 0 0) 0x7f3c8415ec80 con 0x7f3c840d0280 2024-03-28T13:17:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.413+0000 7f3c83400640 1 -- 172.21.15.19:0/496116462 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c68003520 con 0x7f3c840d0280 2024-03-28T13:17:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.414+0000 7f3c83400640 1 -- 172.21.15.19:0/496116462 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 97029150 0 0) 0x7f3c68003520 con 0x7f3c840d0280 2024-03-28T13:17:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.414+0000 7f3c83400640 1 -- 172.21.15.19:0/496116462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c84141c10 msgr2=0x7f3c84071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.414+0000 7f3c83400640 1 --2- 172.21.15.19:0/496116462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c84141c10 0x7f3c84071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.414+0000 7f3c83400640 1 -- 172.21.15.19:0/496116462 >> v2:172.21.15.188:3300/0 conn(0x7f3c84071eb0 msgr2=0x7f3c84162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.414+0000 7f3c83400640 1 --2- 172.21.15.19:0/496116462 >> v2:172.21.15.188:3300/0 conn(0x7f3c84071eb0 0x7f3c84162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.414+0000 7f3c83400640 1 -- 172.21.15.19:0/496116462 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8415f4e0 con 0x7f3c840d0280 2024-03-28T13:17:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.414+0000 7f3c83400640 1 -- 172.21.15.19:0/496116462 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f3c6c001670 con 0x7f3c840d0280 2024-03-28T13:17:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.414+0000 7f3c83400640 1 -- 172.21.15.19:0/496116462 >> v1:172.21.15.19:6789/0 conn(0x7f3c840d0280 legacy=0x7f3c840d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.414+0000 7f3c83400640 1 --2- 172.21.15.19:0/496116462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c84141c10 0x7f3c680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.414+0000 7f3c83400640 1 --2- 172.21.15.19:0/496116462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c68003f30 0x7f3c68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.414+0000 7f3c83400640 1 --2- 172.21.15.19:0/496116462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c68004d40 0x7f3c680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.414+0000 7f3c83400640 1 -- 172.21.15.19:0/496116462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8415ef20 con 0x7f3c68004d40 2024-03-28T13:17:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.414+0000 7f3c83400640 1 -- 172.21.15.19:0/496116462 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8415f200 con 0x7f3c68003f30 2024-03-28T13:17:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.414+0000 7f3c83400640 1 -- 172.21.15.19:0/496116462 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3c6800b9c0 con 0x7f3c84141c10 2024-03-28T13:17:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.414+0000 7f3c89c00640 1 --2- 172.21.15.19:0/496116462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c68004d40 0x7f3c680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.415+0000 7f3c83e00640 1 --2- 172.21.15.19:0/496116462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c68003f30 0x7f3c68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.415+0000 7f3c89200640 1 --2- 172.21.15.19:0/496116462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c84141c10 0x7f3c680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.415+0000 7f3c89c00640 1 -- 172.21.15.19:0/496116462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c84141c10 msgr2=0x7f3c680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.415+0000 7f3c89c00640 1 --2- 172.21.15.19:0/496116462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c84141c10 0x7f3c680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.415+0000 7f3c89c00640 1 -- 172.21.15.19:0/496116462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c68003f30 msgr2=0x7f3c68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:50.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.415+0000 7f3c89c00640 1 --2- 172.21.15.19:0/496116462 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c68003f30 0x7f3c68004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.415+0000 7f3c89c00640 1 -- 172.21.15.19:0/496116462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c68003520 con 0x7f3c68004d40 2024-03-28T13:17:50.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.415+0000 7f3c89200640 1 --2- 172.21.15.19:0/496116462 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c84141c10 0x7f3c680039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:50.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.416+0000 7f3c89c00640 1 --2- 172.21.15.19:0/496116462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c68004d40 0x7f3c680051c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009fc0 tx=0x7f3c68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c50174f6b16333d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:50.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.416+0000 7f3c83400640 1 -- 172.21.15.19:0/496116462 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c039070 con 0x7f3c68004d40 2024-03-28T13:17:50.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.416+0000 7f3c83400640 1 -- 172.21.15.19:0/496116462 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c7c02cc40 con 0x7f3c68004d40 2024-03-28T13:17:50.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.416+0000 7f3c83400640 1 -- 172.21.15.19:0/496116462 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c009c10 con 0x7f3c68004d40 2024-03-28T13:17:50.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.417+0000 7f3c8bc00640 1 -- 172.21.15.19:0/496116462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c68004d40 msgr2=0x7f3c680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:50.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.417+0000 7f3c8bc00640 1 --2- 172.21.15.19:0/496116462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c68004d40 0x7f3c680051c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009fc0 tx=0x7f3c68005830 comp rx=0 tx=0).stop 2024-03-28T13:17:50.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.417+0000 7f3c89c00640 1 -- 172.21.15.19:0/496116462 reap_dead start 2024-03-28T13:17:50.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.417+0000 7f3c8bc00640 1 -- 172.21.15.19:0/496116462 shutdown_connections 2024-03-28T13:17:50.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.417+0000 7f3c8bc00640 1 -- 172.21.15.19:0/496116462 >> 172.21.15.19:0/496116462 conn(0x7f3c8406b750 msgr2=0x7f3c8406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:50.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.417+0000 7f3c8bc00640 1 -- 172.21.15.19:0/496116462 shutdown_connections 2024-03-28T13:17:50.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.417+0000 7f3c8bc00640 1 -- 172.21.15.19:0/496116462 wait complete. 2024-03-28T13:17:50.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.419+0000 7f3c8bc00640 1 Processor -- start 2024-03-28T13:17:50.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.419+0000 7f3c8bc00640 1 -- start start 2024-03-28T13:17:50.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.420+0000 7f3c8bc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c68004d40 0x7f3c8409b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.420+0000 7f3c8bc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c8409c8f0 0x7f3c8414ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.420+0000 7f3c8bc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c841e79c0 0x7f3c8409d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.420+0000 7f3c8bc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c84158e90 con 0x7f3c68004d40 2024-03-28T13:17:50.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.420+0000 7f3c8bc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8406f340 con 0x7f3c8409c8f0 2024-03-28T13:17:50.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.420+0000 7f3c8bc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3c8409b850 con 0x7f3c841e79c0 2024-03-28T13:17:50.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.420+0000 7f3c89200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c68004d40 0x7f3c8409b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.420+0000 7f3c89200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c68004d40 0x7f3c8409b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53590/0 (socket says 172.21.15.19:53590) 2024-03-28T13:17:50.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.420+0000 7f3c89200640 1 -- 172.21.15.19:0/628999628 learned_addr learned my addr 172.21.15.19:0/628999628 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.421+0000 7f3c83e00640 1 --2- 172.21.15.19:0/628999628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c8409c8f0 0x7f3c8414ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.421+0000 7f3c89c00640 1 --2- 172.21.15.19:0/628999628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c841e79c0 0x7f3c8409d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.421+0000 7f3c89200640 1 -- 172.21.15.19:0/628999628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c841e79c0 msgr2=0x7f3c8409d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.421+0000 7f3c89200640 1 --2- 172.21.15.19:0/628999628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c841e79c0 0x7f3c8409d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.421+0000 7f3c89200640 1 -- 172.21.15.19:0/628999628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c8409c8f0 msgr2=0x7f3c8414ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.421+0000 7f3c89200640 1 --2- 172.21.15.19:0/628999628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c8409c8f0 0x7f3c8414ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.421+0000 7f3c89200640 1 -- 172.21.15.19:0/628999628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8409bb30 con 0x7f3c68004d40 2024-03-28T13:17:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.421+0000 7f3c89200640 1 --2- 172.21.15.19:0/628999628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c68004d40 0x7f3c8409b1e0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00d8b0 tx=0x7f3c6c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.421+0000 7f3c80c00640 1 -- 172.21.15.19:0/628999628 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c019070 con 0x7f3c68004d40 2024-03-28T13:17:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.422+0000 7f3c80c00640 1 -- 172.21.15.19:0/628999628 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6c0074e0 con 0x7f3c68004d40 2024-03-28T13:17:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.422+0000 7f3c8bc00640 1 -- 172.21.15.19:0/628999628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c8409be10 con 0x7f3c68004d40 2024-03-28T13:17:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.422+0000 7f3c80c00640 1 -- 172.21.15.19:0/628999628 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c004a00 con 0x7f3c68004d40 2024-03-28T13:17:50.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.422+0000 7f3c8bc00640 1 -- 172.21.15.19:0/628999628 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c8409c3d0 con 0x7f3c68004d40 2024-03-28T13:17:50.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.424+0000 7f3c80c00640 1 -- 172.21.15.19:0/628999628 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3c6c01c070 con 0x7f3c68004d40 2024-03-28T13:17:50.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.425+0000 7f3c80c00640 1 --2- 172.21.15.19:0/628999628 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c6006c530 0x7f3c6006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.425+0000 7f3c80c00640 1 -- 172.21.15.19:0/628999628 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f3c6c08b330 con 0x7f3c68004d40 2024-03-28T13:17:50.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.425+0000 7f3c83e00640 1 --2- 172.21.15.19:0/628999628 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c6006c530 0x7f3c6006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.425+0000 7f3c8e9d5740 1 Processor -- start 2024-03-28T13:17:50.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.426+0000 7f3c8e9d5740 1 -- start start 2024-03-28T13:17:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.426+0000 7f3c8e9d5740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555680374560 0x555680347500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.426+0000 7f3c8e9d5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555680374940 0x5556803772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.426+0000 7f3c8e9d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680347a40 0x555680377830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.426+0000 7f3c8e9d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5556802b26b0 con 0x555680347a40 2024-03-28T13:17:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.426+0000 7f3c8e9d5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55568025a750 con 0x555680374940 2024-03-28T13:17:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.426+0000 7f3c8e9d5740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5556802ae660 con 0x555680374560 2024-03-28T13:17:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.427+0000 7f3c83e00640 1 --2- 172.21.15.19:0/628999628 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c6006c530 0x7f3c6006e9f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3c840cf4c0 tx=0x7f3c74008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.427+0000 7f3c89c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680347a40 0x555680377830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.427+0000 7f3c89c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680347a40 0x555680377830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53594/0 (socket says 172.21.15.19:53594) 2024-03-28T13:17:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.427+0000 7f3c89c00640 1 -- 172.21.15.19:0/3039428671 learned_addr learned my addr 172.21.15.19:0/3039428671 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.427+0000 7f3c83e00640 1 --2- 172.21.15.19:0/3039428671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555680374940 0x5556803772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.427+0000 7f3c89200640 1 --2- 172.21.15.19:0/3039428671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555680374560 0x555680347500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.427+0000 7f3c89c00640 1 -- 172.21.15.19:0/3039428671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555680374560 msgr2=0x555680347500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.427+0000 7f3c89c00640 1 --2- 172.21.15.19:0/3039428671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555680374560 0x555680347500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.427+0000 7f3c89c00640 1 -- 172.21.15.19:0/3039428671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555680374940 msgr2=0x5556803772f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.427+0000 7f3c89c00640 1 --2- 172.21.15.19:0/3039428671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555680374940 0x5556803772f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.427+0000 7f3c89c00640 1 -- 172.21.15.19:0/3039428671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556800907d0 con 0x555680347a40 2024-03-28T13:17:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.428+0000 7f3c89c00640 1 --2- 172.21.15.19:0/3039428671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680347a40 0x555680377830 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c02c710 tx=0x7f3c7c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1923960a7a3baba server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.428+0000 7f3c79600640 1 -- 172.21.15.19:0/3039428671 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c044070 con 0x555680347a40 2024-03-28T13:17:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.428+0000 7f3c79600640 1 -- 172.21.15.19:0/3039428671 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c7c03a040 con 0x555680347a40 2024-03-28T13:17:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.428+0000 7f3c79600640 1 -- 172.21.15.19:0/3039428671 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c039070 con 0x555680347a40 2024-03-28T13:17:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.428+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/3039428671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680347a40 msgr2=0x555680377830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:50.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.428+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/3039428671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680347a40 0x555680377830 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c02c710 tx=0x7f3c7c02e070 comp rx=0 tx=0).stop 2024-03-28T13:17:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.429+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/3039428671 shutdown_connections 2024-03-28T13:17:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.429+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/3039428671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680347a40 0x555680377830 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.429+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/3039428671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555680374940 0x5556803772f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.429+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/3039428671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555680374560 0x555680347500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.429+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/3039428671 >> 172.21.15.19:0/3039428671 conn(0x555680349770 msgr2=0x5556803922f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.429+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/3039428671 shutdown_connections 2024-03-28T13:17:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.429+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/3039428671 wait complete. 2024-03-28T13:17:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.429+0000 7f3c8e9d5740 1 -- start start 2024-03-28T13:17:50.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.430+0000 7f3c8e9d5740 1 Processor -- start 2024-03-28T13:17:50.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.431+0000 7f3c8e9d5740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555680374560 0x5556803a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.431+0000 7f3c8e9d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374940 0x55568037f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.431+0000 7f3c8e9d5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556803a5520 0x55568037fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.431+0000 7f3c8e9d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55568034be10 con 0x555680374940 2024-03-28T13:17:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.431+0000 7f3c8e9d5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55568026da80 con 0x5556803a5520 2024-03-28T13:17:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.431+0000 7f3c8e9d5740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55568034e650 con 0x555680374560 2024-03-28T13:17:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.431+0000 7f3c8e9d5740 10 client.0 _ll_register_callbacks cb 0x55568026d470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:17:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.431+0000 7f3c83e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374940 0x55568037f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.431+0000 7f3c83e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374940 0x55568037f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53610/0 (socket says 172.21.15.19:53610) 2024-03-28T13:17:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.431+0000 7f3c83e00640 1 -- 172.21.15.19:0/2041267800 learned_addr learned my addr 172.21.15.19:0/2041267800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.431+0000 7f3c89c00640 1 --2- 172.21.15.19:0/2041267800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556803a5520 0x55568037fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.431+0000 7f3c89200640 1 --2- 172.21.15.19:0/2041267800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555680374560 0x5556803a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.432+0000 7f3c83e00640 1 -- 172.21.15.19:0/2041267800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555680374560 msgr2=0x5556803a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.432+0000 7f3c83e00640 1 --2- 172.21.15.19:0/2041267800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555680374560 0x5556803a4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.432+0000 7f3c83e00640 1 -- 172.21.15.19:0/2041267800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556803a5520 msgr2=0x55568037fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.432+0000 7f3c83e00640 1 --2- 172.21.15.19:0/2041267800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556803a5520 0x55568037fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.432+0000 7f3c83e00640 1 -- 172.21.15.19:0/2041267800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55567ffc63f0 con 0x555680374940 2024-03-28T13:17:50.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.432+0000 7f3c83e00640 1 --2- 172.21.15.19:0/2041267800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374940 0x55568037f500 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f3c74004540 tx=0x7f3c74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a9cd8255419f91 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.432+0000 7f3c72000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:17:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.432+0000 7f3c72000640 1 -- 172.21.15.19:0/2041267800 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74003d90 con 0x555680374940 2024-03-28T13:17:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.432+0000 7f3c72000640 1 -- 172.21.15.19:0/2041267800 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f3c74016460 con 0x555680374940 2024-03-28T13:17:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.432+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2041267800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c7c03a040 con 0x555680374940 2024-03-28T13:17:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.433+0000 7f3c72000640 1 -- 172.21.15.19:0/2041267800 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f3c74032050 con 0x555680374940 2024-03-28T13:17:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.433+0000 7f3c8e9d5740 10 client.7590 fetch_fsmap learned FSMap version 68 2024-03-28T13:17:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.433+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2041267800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55567ff860f0 con 0x555680374940 2024-03-28T13:17:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.433+0000 7f3c72000640 1 -- 172.21.15.19:0/2041267800 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c7c03a040 con 0x555680374940 2024-03-28T13:17:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.433+0000 7f3c72000640 1 -- 172.21.15.19:0/2041267800 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f3c74035020 con 0x555680374940 2024-03-28T13:17:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.433+0000 7f3c8e9d5740 10 client.7590 fetch_fsmap finished waiting for FSMap version 68 2024-03-28T13:17:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.433+0000 7f3c8e9d5740 10 client.7590 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-28T13:17:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.433+0000 7f3c8e9d5740 20 client.7590 populate_metadata read hostname 'smithi019' 2024-03-28T13:17:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.433+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/2041267800 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5556803a3fe0 0x5556803a4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.433+0000 7f3c8e9d5740 4 client.7590 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:17:50.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.433+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2041267800 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556802bfbf0 con 0x5556803a3fe0 2024-03-28T13:17:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.434+0000 7f3c89200640 1 --2- 172.21.15.19:0/2041267800 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5556803a3fe0 0x5556803a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.435+0000 7f3c89200640 1 --2- 172.21.15.19:0/2041267800 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5556803a3fe0 0x5556803a4410 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c66a65bd4cd47ba5 server_cookie=7d24aed772149967 in_seq=0 out_seq=0 2024-03-28T13:17:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.435+0000 7f3c72000640 10 client.7590 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.437+0000 7f3c72000640 1 -- 172.21.15.19:0/2041267800 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556802bfbf0 con 0x5556803a3fe0 2024-03-28T13:17:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.437+0000 7f3c72000640 10 client.7590 handle_command_reply: tid=0 2024-03-28T13:17:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.437+0000 7f3c8e9d5740 1 client.7590 shutdown 2024-03-28T13:17:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.437+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2041267800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374940 msgr2=0x55568037f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.437+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/2041267800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374940 0x55568037f500 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f3c74004540 tx=0x7f3c74004730 comp rx=0 tx=0).stop 2024-03-28T13:17:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.437+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2041267800 shutdown_connections 2024-03-28T13:17:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.437+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/2041267800 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x5556803a3fe0 0x5556803a4410 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.437+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/2041267800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556803a5520 0x55568037fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.437+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/2041267800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374940 0x55568037f500 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.437+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/2041267800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555680374560 0x5556803a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.437+0000 7f3c72000640 0 client.7590 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.437+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2041267800 >> 172.21.15.19:0/2041267800 conn(0x555680347a40 msgr2=0x555680376df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.437+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2041267800 shutdown_connections 2024-03-28T13:17:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.437+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2041267800 wait complete. 2024-03-28T13:17:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.438+0000 7f3c8e9d5740 20 client.7590 trim_cache size 0 max 16384 2024-03-28T13:17:50.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.457+0000 7f3c8e9d5740 1 Processor -- start 2024-03-28T13:17:50.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.457+0000 7f3c8e9d5740 1 -- start start 2024-03-28T13:17:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.458+0000 7f3c8e9d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374560 0x55568040ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.458+0000 7f3c8e9d5740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555680374940 0x555680376df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.458+0000 7f3c8e9d5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556803a5520 0x555680377330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.458+0000 7f3c8e9d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c84201180 con 0x555680374560 2024-03-28T13:17:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.458+0000 7f3c8e9d5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3c842001a0 con 0x5556803a5520 2024-03-28T13:17:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.458+0000 7f3c8e9d5740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3c841e9520 con 0x555680374940 2024-03-28T13:17:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.458+0000 7f3c89200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374560 0x55568040ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.458+0000 7f3c89200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374560 0x55568040ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53622/0 (socket says 172.21.15.19:53622) 2024-03-28T13:17:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.458+0000 7f3c89200640 1 -- 172.21.15.19:0/368754541 learned_addr learned my addr 172.21.15.19:0/368754541 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.458+0000 7f3c83e00640 1 --2- 172.21.15.19:0/368754541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555680374940 0x555680376df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.458+0000 7f3c89c00640 1 --2- 172.21.15.19:0/368754541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556803a5520 0x555680377330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.459+0000 7f3c89200640 1 -- 172.21.15.19:0/368754541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555680374940 msgr2=0x555680376df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:50.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.459+0000 7f3c89200640 1 --2- 172.21.15.19:0/368754541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555680374940 0x555680376df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.459+0000 7f3c89200640 1 -- 172.21.15.19:0/368754541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556803a5520 msgr2=0x555680377330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:50.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.459+0000 7f3c89200640 1 --2- 172.21.15.19:0/368754541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556803a5520 0x555680377330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.459+0000 7f3c89200640 1 -- 172.21.15.19:0/368754541 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556802ae7e0 con 0x555680374560 2024-03-28T13:17:50.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.459+0000 7f3c89c00640 1 --2- 172.21.15.19:0/368754541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556803a5520 0x555680377330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:17:50.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.459+0000 7f3c89200640 1 --2- 172.21.15.19:0/368754541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374560 0x55568040ebc0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0555b0 tx=0x7f3c6c089630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14614eb1af662c03 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:50.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.459+0000 7f3c78c00640 1 -- 172.21.15.19:0/368754541 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c061d30 con 0x555680374560 2024-03-28T13:17:50.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.459+0000 7f3c78c00640 1 -- 172.21.15.19:0/368754541 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6c089870 con 0x555680374560 2024-03-28T13:17:50.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.459+0000 7f3c78c00640 1 -- 172.21.15.19:0/368754541 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c057450 con 0x555680374560 2024-03-28T13:17:50.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.460+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/368754541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374560 msgr2=0x55568040ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:50.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.460+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/368754541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374560 0x55568040ebc0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c0555b0 tx=0x7f3c6c089630 comp rx=0 tx=0).stop 2024-03-28T13:17:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.460+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/368754541 shutdown_connections 2024-03-28T13:17:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.460+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/368754541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5556803a5520 0x555680377330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.460+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/368754541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555680374940 0x555680376df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.460+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/368754541 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374560 0x55568040ebc0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.460+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/368754541 >> 172.21.15.19:0/368754541 conn(0x555680347a40 msgr2=0x5556803a2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.460+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/368754541 shutdown_connections 2024-03-28T13:17:50.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.460+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/368754541 wait complete. 2024-03-28T13:17:50.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.461+0000 7f3c8e9d5740 1 -- start start 2024-03-28T13:17:50.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.461+0000 7f3c8e9d5740 1 Processor -- start 2024-03-28T13:17:50.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.462+0000 7f3c8e9d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374560 0x55568040b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.462+0000 7f3c8e9d5740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555680374940 0x555680378620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.462+0000 7f3c8e9d5740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5556803a5520 0x555680378b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.462+0000 7f3c8e9d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555680349070 con 0x555680374560 2024-03-28T13:17:50.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.462+0000 7f3c8e9d5740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3c84149020 con 0x555680374940 2024-03-28T13:17:50.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.462+0000 7f3c8e9d5740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x555680379f90 con 0x5556803a5520 2024-03-28T13:17:50.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.462+0000 7f3c8e9d5740 10 client.0 _ll_register_callbacks cb 0x7f3c740041f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:17:50.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.462+0000 7f3c89200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374560 0x55568040b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.462+0000 7f3c89200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374560 0x55568040b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53624/0 (socket says 172.21.15.19:53624) 2024-03-28T13:17:50.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.462+0000 7f3c89200640 1 -- 172.21.15.19:0/2688597880 learned_addr learned my addr 172.21.15.19:0/2688597880 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:50.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.463+0000 7f3c89200640 1 -- 172.21.15.19:0/2688597880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5556803a5520 msgr2=0x555680378b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:50.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.463+0000 7f3c89200640 1 --2- 172.21.15.19:0/2688597880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5556803a5520 0x555680378b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.463+0000 7f3c83e00640 1 --2- 172.21.15.19:0/2688597880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555680374940 0x555680378620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.463+0000 7f3c89200640 1 -- 172.21.15.19:0/2688597880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555680374940 msgr2=0x555680378620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:50.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.463+0000 7f3c89200640 1 --2- 172.21.15.19:0/2688597880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555680374940 0x555680378620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.463+0000 7f3c89200640 1 -- 172.21.15.19:0/2688597880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556803a0e90 con 0x555680374560 2024-03-28T13:17:50.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.463+0000 7f3c89200640 1 --2- 172.21.15.19:0/2688597880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374560 0x55568040b370 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c089700 tx=0x7f3c6c05c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebaedd091bab437d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:50.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.463+0000 7f3c72000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:17:50.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.463+0000 7f3c72000640 1 -- 172.21.15.19:0/2688597880 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c061d30 con 0x555680374560 2024-03-28T13:17:50.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.463+0000 7f3c72000640 1 -- 172.21.15.19:0/2688597880 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f3c6c0893b0 con 0x555680374560 2024-03-28T13:17:50.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.463+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2688597880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c6c089870 con 0x555680374560 2024-03-28T13:17:50.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.463+0000 7f3c72000640 1 -- 172.21.15.19:0/2688597880 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f3c6c0557c0 con 0x555680374560 2024-03-28T13:17:50.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.463+0000 7f3c8e9d5740 10 client.7596 fetch_fsmap learned FSMap version 68 2024-03-28T13:17:50.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.463+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2688597880 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x555680346f40 con 0x555680374560 2024-03-28T13:17:50.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.464+0000 7f3c72000640 1 -- 172.21.15.19:0/2688597880 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c6c089870 con 0x555680374560 2024-03-28T13:17:50.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.464+0000 7f3c72000640 1 -- 172.21.15.19:0/2688597880 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f3c6c05d120 con 0x555680374560 2024-03-28T13:17:50.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.464+0000 7f3c8e9d5740 10 client.7596 fetch_fsmap finished waiting for FSMap version 68 2024-03-28T13:17:50.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.464+0000 7f3c8e9d5740 10 client.7596 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.b 2024-03-28T13:17:50.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.464+0000 7f3c8e9d5740 20 client.7596 populate_metadata read hostname 'smithi019' 2024-03-28T13:17:50.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.464+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/2688597880 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55568037c060 0x555680414e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.464+0000 7f3c8e9d5740 4 client.7596 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:17:50.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.464+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2688597880 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5556802b9f40 con 0x55568037c060 2024-03-28T13:17:50.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.464+0000 7f3c83e00640 1 --2- 172.21.15.19:0/2688597880 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55568037c060 0x555680414e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.465+0000 7f3c83e00640 1 --2- 172.21.15.19:0/2688597880 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55568037c060 0x555680414e60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa33357d676a241 server_cookie=7c159b356cad057d in_seq=0 out_seq=0 2024-03-28T13:17:50.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.465+0000 7f3c72000640 10 client.7596 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:50.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.466+0000 7f3c72000640 1 -- 172.21.15.19:0/2688597880 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5556802b9f40 con 0x55568037c060 2024-03-28T13:17:50.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.466+0000 7f3c72000640 10 client.7596 handle_command_reply: tid=0 2024-03-28T13:17:50.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.466+0000 7f3c8e9d5740 1 client.7596 shutdown 2024-03-28T13:17:50.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.466+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2688597880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374560 msgr2=0x55568040b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:50.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.466+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/2688597880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374560 0x55568040b370 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c089700 tx=0x7f3c6c05c040 comp rx=0 tx=0).stop 2024-03-28T13:17:50.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.466+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2688597880 shutdown_connections 2024-03-28T13:17:50.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.466+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/2688597880 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55568037c060 0x555680414e60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.466+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/2688597880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5556803a5520 0x555680378b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.466+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/2688597880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555680374940 0x555680378620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.466+0000 7f3c8e9d5740 1 --2- 172.21.15.19:0/2688597880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555680374560 0x55568040b370 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.466+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2688597880 >> 172.21.15.19:0/2688597880 conn(0x555680347a40 msgr2=0x55568037f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:50.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.466+0000 7f3c72000640 0 client.7596 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:50.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.467+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2688597880 shutdown_connections 2024-03-28T13:17:50.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.467+0000 7f3c8e9d5740 1 -- 172.21.15.19:0/2688597880 wait complete. 2024-03-28T13:17:50.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.467+0000 7f3c8e9d5740 20 client.7596 trim_cache size 0 max 16384 2024-03-28T13:17:50.469 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:17:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-03-28T13:17:50.469 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-03-28T13:17:50.469 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:17:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.467+0000 7f3c8bc00640 1 -- 172.21.15.19:0/628999628 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c6006c530 msgr2=0x7f3c6006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.467+0000 7f3c8bc00640 1 --2- 172.21.15.19:0/628999628 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c6006c530 0x7f3c6006e9f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3c840cf4c0 tx=0x7f3c74008040 comp rx=0 tx=0).stop 2024-03-28T13:17:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.467+0000 7f3c8bc00640 1 -- 172.21.15.19:0/628999628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c68004d40 msgr2=0x7f3c8409b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.467+0000 7f3c8bc00640 1 --2- 172.21.15.19:0/628999628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c68004d40 0x7f3c8409b1e0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00d8b0 tx=0x7f3c6c00dd70 comp rx=0 tx=0).stop 2024-03-28T13:17:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.467+0000 7f3c8bc00640 1 -- 172.21.15.19:0/628999628 shutdown_connections 2024-03-28T13:17:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.468+0000 7f3c8bc00640 1 --2- 172.21.15.19:0/628999628 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3c6006c530 0x7f3c6006e9f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.468+0000 7f3c8bc00640 1 --2- 172.21.15.19:0/628999628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c841e79c0 0x7f3c8409d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.468+0000 7f3c8bc00640 1 --2- 172.21.15.19:0/628999628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c8409c8f0 0x7f3c8414ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.468+0000 7f3c8bc00640 1 --2- 172.21.15.19:0/628999628 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c68004d40 0x7f3c8409b1e0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.468+0000 7f3c8bc00640 1 -- 172.21.15.19:0/628999628 >> 172.21.15.19:0/628999628 conn(0x7f3c8406b750 msgr2=0x7f3c84161290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:50.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.468+0000 7f3c8bc00640 1 -- 172.21.15.19:0/628999628 shutdown_connections 2024-03-28T13:17:50.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.468+0000 7f3c8bc00640 1 -- 172.21.15.19:0/628999628 wait complete. 2024-03-28T13:17:50.485 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T13:17:50.485 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:17:50.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.701+0000 7fa0f7e00640 1 Processor -- start 2024-03-28T13:17:50.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.701+0000 7fa0f7e00640 1 -- start start 2024-03-28T13:17:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.703+0000 7fa0f7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f81442c0 0x7fa0f80ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.703+0000 7fa0f7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0f80cefe0 0x7fa0f80cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.703+0000 7fa0f7e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0f8071460 con 0x7fa0f80d00d0 2024-03-28T13:17:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.703+0000 7fa0f7e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa0f8071d20 con 0x7fa0f80cefe0 2024-03-28T13:17:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.703+0000 7fa0f7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f80cf420 con 0x7fa0f81442c0 2024-03-28T13:17:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.704+0000 7fa0ee000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0f80cefe0 0x7fa0f80cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.704+0000 7fa0ee000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0f80cefe0 0x7fa0f80cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53640/0 (socket says 172.21.15.19:53640) 2024-03-28T13:17:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.704+0000 7fa0ee000640 1 -- 172.21.15.19:0/944078920 learned_addr learned my addr 172.21.15.19:0/944078920 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.704+0000 7fa0f6a00640 1 --2- 172.21.15.19:0/944078920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f81442c0 0x7fa0f80ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.704+0000 7fa0ee000640 1 -- 172.21.15.19:0/944078920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f81442c0 msgr2=0x7fa0f80ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:50.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.704+0000 7fa0ee000640 1 --2- 172.21.15.19:0/944078920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f81442c0 0x7fa0f80ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.704+0000 7fa0ee000640 1 -- 172.21.15.19:0/944078920 >> v1:172.21.15.188:6789/0 conn(0x7fa0f80d00d0 legacy=0x7fa0f806e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.704+0000 7fa0ee000640 1 -- 172.21.15.19:0/944078920 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f80721b0 con 0x7fa0f80cefe0 2024-03-28T13:17:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.704+0000 7fa0f6a00640 1 --2- 172.21.15.19:0/944078920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f81442c0 0x7fa0f80ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.705+0000 7fa0ee000640 1 --2- 172.21.15.19:0/944078920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80cefe0 0x7fa0f80cfb90 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009870 tx=0x7fa0e002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e980a881d56ebb95 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.706+0000 7fa0f6000640 1 -- 172.21.15.19:0/944078920 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0034070 con 0x7fa0f80cefe0 2024-03-28T13:17:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.706+0000 7fa0f6000640 1 -- 172.21.15.19:0/944078920 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e0030840 con 0x7fa0f80cefe0 2024-03-28T13:17:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.706+0000 7fa0f6000640 1 -- 172.21.15.19:0/944078920 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0030c10 con 0x7fa0f80cefe0 2024-03-28T13:17:50.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.706+0000 7fa0f7e00640 1 -- 172.21.15.19:0/944078920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80cefe0 msgr2=0x7fa0f80cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:50.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.706+0000 7fa0f7e00640 1 --2- 172.21.15.19:0/944078920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80cefe0 0x7fa0f80cfb90 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009870 tx=0x7fa0e002b980 comp rx=0 tx=0).stop 2024-03-28T13:17:50.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.706+0000 7fa0f7e00640 1 -- 172.21.15.19:0/944078920 shutdown_connections 2024-03-28T13:17:50.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.706+0000 7fa0f7e00640 1 --2- 172.21.15.19:0/944078920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80cefe0 0x7fa0f80cfb90 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.706+0000 7fa0f7e00640 1 --2- 172.21.15.19:0/944078920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f81442c0 0x7fa0f80ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.706+0000 7fa0f7e00640 1 -- 172.21.15.19:0/944078920 >> 172.21.15.19:0/944078920 conn(0x7fa0f80ccad0 msgr2=0x7fa0f80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.707+0000 7fa0f7e00640 1 -- 172.21.15.19:0/944078920 shutdown_connections 2024-03-28T13:17:50.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.707+0000 7fa0f7e00640 1 -- 172.21.15.19:0/944078920 wait complete. 2024-03-28T13:17:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.708+0000 7fa0f7e00640 1 Processor -- start 2024-03-28T13:17:50.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.708+0000 7fa0f7e00640 1 -- start start 2024-03-28T13:17:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.709+0000 7fa0f7e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f80cefe0 0x7fa0f81ede00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.709+0000 7fa0f7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80d00d0 0x7fa0f80aead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.709+0000 7fa0f7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f81442c0 0x7fa0f813e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.709+0000 7fa0f7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f80d0770 con 0x7fa0f80d00d0 2024-03-28T13:17:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.709+0000 7fa0f7e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f80719b0 con 0x7fa0f80cefe0 2024-03-28T13:17:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.709+0000 7fa0f7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f81446a0 con 0x7fa0f81442c0 2024-03-28T13:17:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.710+0000 7fa0ee000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80d00d0 0x7fa0f80aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.710+0000 7fa0ee000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80d00d0 0x7fa0f80aead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53648/0 (socket says 172.21.15.19:53648) 2024-03-28T13:17:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.710+0000 7fa0ee000640 1 -- 172.21.15.19:0/3701598451 learned_addr learned my addr 172.21.15.19:0/3701598451 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.710+0000 7fa0f7400640 1 --2- 172.21.15.19:0/3701598451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f81442c0 0x7fa0f813e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.710+0000 7fa0f6a00640 1 --2- 172.21.15.19:0/3701598451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f80cefe0 0x7fa0f81ede00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.710+0000 7fa0ee000640 1 -- 172.21.15.19:0/3701598451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f81442c0 msgr2=0x7fa0f813e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.710+0000 7fa0ee000640 1 --2- 172.21.15.19:0/3701598451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f81442c0 0x7fa0f813e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.710+0000 7fa0ee000640 1 -- 172.21.15.19:0/3701598451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f80cefe0 msgr2=0x7fa0f81ede00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.710+0000 7fa0ee000640 1 --2- 172.21.15.19:0/3701598451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f80cefe0 0x7fa0f81ede00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.710+0000 7fa0ee000640 1 -- 172.21.15.19:0/3701598451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f80ada30 con 0x7fa0f80d00d0 2024-03-28T13:17:50.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.711+0000 7fa0ee000640 1 --2- 172.21.15.19:0/3701598451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80d00d0 0x7fa0f80aead0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009870 tx=0x7fa0e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.711+0000 7fa0ef400640 1 -- 172.21.15.19:0/3701598451 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0048070 con 0x7fa0f80d00d0 2024-03-28T13:17:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.711+0000 7fa0ef400640 1 -- 172.21.15.19:0/3701598451 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e0030640 con 0x7fa0f80d00d0 2024-03-28T13:17:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.711+0000 7fa0f7e00640 1 -- 172.21.15.19:0/3701598451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f80add10 con 0x7fa0f80d00d0 2024-03-28T13:17:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.711+0000 7fa0ef400640 1 -- 172.21.15.19:0/3701598451 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0034040 con 0x7fa0f80d00d0 2024-03-28T13:17:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.711+0000 7fa0f7e00640 1 -- 172.21.15.19:0/3701598451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f80ae2d0 con 0x7fa0f80d00d0 2024-03-28T13:17:50.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.712+0000 7fa0f7e00640 1 -- 172.21.15.19:0/3701598451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f8075940 con 0x7fa0f80d00d0 2024-03-28T13:17:50.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.713+0000 7fa0ef400640 1 -- 172.21.15.19:0/3701598451 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa0e004b070 con 0x7fa0f80d00d0 2024-03-28T13:17:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.714+0000 7fa0ef400640 1 --2- 172.21.15.19:0/3701598451 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0d406c560 0x7fa0d406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.714+0000 7fa0ef400640 1 -- 172.21.15.19:0/3701598451 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fa0e00ad910 con 0x7fa0f80d00d0 2024-03-28T13:17:50.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.715+0000 7fa0f6a00640 1 --2- 172.21.15.19:0/3701598451 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0d406c560 0x7fa0d406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:50.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.716+0000 7fa0f6a00640 1 --2- 172.21.15.19:0/3701598451 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0d406c560 0x7fa0d406ea20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa0f815d4a0 tx=0x7fa0e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:50.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.719+0000 7fa0ef400640 1 -- 172.21.15.19:0/3701598451 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa0e007be30 con 0x7fa0f80d00d0 2024-03-28T13:17:50.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.969+0000 7fa0f7e00640 1 -- 172.21.15.19:0/3701598451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0f80cf5d0 con 0x7fa0f80d00d0 2024-03-28T13:17:50.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.971+0000 7fa0ef400640 1 -- 172.21.15.19:0/3701598451 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5187 (secure 0 0 0) 0x7fa0e007b730 con 0x7fa0f80d00d0 2024-03-28T13:17:50.971 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:17:50.971 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:16:50.520332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6175":{"gid":6175,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6836/1749969053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1749969053},{"type":"v1","addr":"172.21.15.188:6836","nonce":1749969053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6175,"qdb_cluster":[6175]},"id":6}]} 2024-03-28T13:17:50.971 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 68 2024-03-28T13:17:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.974+0000 7fa0f7e00640 1 -- 172.21.15.19:0/3701598451 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0d406c560 msgr2=0x7fa0d406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.974+0000 7fa0f7e00640 1 --2- 172.21.15.19:0/3701598451 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0d406c560 0x7fa0d406ea20 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa0f815d4a0 tx=0x7fa0e8008040 comp rx=0 tx=0).stop 2024-03-28T13:17:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.974+0000 7fa0f7e00640 1 -- 172.21.15.19:0/3701598451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80d00d0 msgr2=0x7fa0f80aead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.974+0000 7fa0f7e00640 1 --2- 172.21.15.19:0/3701598451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80d00d0 0x7fa0f80aead0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0009870 tx=0x7fa0e0004730 comp rx=0 tx=0).stop 2024-03-28T13:17:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.974+0000 7fa0f7e00640 1 -- 172.21.15.19:0/3701598451 shutdown_connections 2024-03-28T13:17:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.974+0000 7fa0f7e00640 1 --2- 172.21.15.19:0/3701598451 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0d406c560 0x7fa0d406ea20 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.974+0000 7fa0f7e00640 1 --2- 172.21.15.19:0/3701598451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0f81442c0 0x7fa0f813e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.974+0000 7fa0f7e00640 1 --2- 172.21.15.19:0/3701598451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0f80d00d0 0x7fa0f80aead0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.974+0000 7fa0f7e00640 1 --2- 172.21.15.19:0/3701598451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0f80cefe0 0x7fa0f81ede00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.974+0000 7fa0f7e00640 1 -- 172.21.15.19:0/3701598451 >> 172.21.15.19:0/3701598451 conn(0x7fa0f80ccad0 msgr2=0x7fa0f806e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.974+0000 7fa0f7e00640 1 -- 172.21.15.19:0/3701598451 shutdown_connections 2024-03-28T13:17:50.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:50.974+0000 7fa0f7e00640 1 -- 172.21.15.19:0/3701598451 wait complete. 2024-03-28T13:17:50.994 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-03-28T13:17:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.214+0000 7f0968e00640 1 Processor -- start 2024-03-28T13:17:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.215+0000 7f0968e00640 1 -- start start 2024-03-28T13:17:51.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.216+0000 7f0968e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f09640705f0 0x7f096406e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.216+0000 7f0968e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f096415edc0 con 0x7f09640d1bf0 2024-03-28T13:17:51.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.216+0000 7f0968e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09640b5e60 con 0x7f0964070170 2024-03-28T13:17:51.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.216+0000 7f0968e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f09640d1fd0 con 0x7f09640705f0 2024-03-28T13:17:51.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.217+0000 7f0961e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f09640d1bf0 0x7f096415f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34762/0 (socket says 172.21.15.19:34762) 2024-03-28T13:17:51.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.217+0000 7f0961e00640 1 -- 172.21.15.19:0/2700864228 learned_addr learned my addr 172.21.15.19:0/2700864228 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:17:51.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.217+0000 7f0961400640 1 -- 172.21.15.19:0/2700864228 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4291005875 0 0) 0x7f096415edc0 con 0x7f09640d1bf0 2024-03-28T13:17:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.217+0000 7f0961400640 1 -- 172.21.15.19:0/2700864228 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0948003520 con 0x7f09640d1bf0 2024-03-28T13:17:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.217+0000 7f0962800640 1 --2- 172.21.15.19:0/2700864228 >> v2:172.21.15.188:3300/0 conn(0x7f09640705f0 0x7f096406e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.218+0000 7f0961400640 1 -- 172.21.15.19:0/2700864228 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1557963086 0 0) 0x7f0948003520 con 0x7f09640d1bf0 2024-03-28T13:17:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.218+0000 7f0961400640 1 -- 172.21.15.19:0/2700864228 >> v2:172.21.15.188:3300/0 conn(0x7f09640705f0 msgr2=0x7f096406e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.218+0000 7f0961400640 1 --2- 172.21.15.19:0/2700864228 >> v2:172.21.15.188:3300/0 conn(0x7f09640705f0 0x7f096406e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.218+0000 7f0961400640 1 -- 172.21.15.19:0/2700864228 >> v1:172.21.15.188:6789/0 conn(0x7f0964070170 legacy=0x7f096406b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.218+0000 7f0961400640 1 -- 172.21.15.19:0/2700864228 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09640b6140 con 0x7f09640d1bf0 2024-03-28T13:17:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.218+0000 7f0961400640 1 -- 172.21.15.19:0/2700864228 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f09540019f0 con 0x7f09640d1bf0 2024-03-28T13:17:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.218+0000 7f0961400640 1 -- 172.21.15.19:0/2700864228 >> v1:172.21.15.19:6789/0 conn(0x7f09640d1bf0 legacy=0x7f096415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.218+0000 7f0961400640 1 --2- 172.21.15.19:0/2700864228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948003a00 0x7f0948003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.218+0000 7f0961400640 1 --2- 172.21.15.19:0/2700864228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0948004800 0x7f0948004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.218+0000 7f0961400640 1 --2- 172.21.15.19:0/2700864228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0948005290 0x7f094800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.218+0000 7f0961400640 1 -- 172.21.15.19:0/2700864228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f094800c1b0 con 0x7f0948005290 2024-03-28T13:17:51.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.218+0000 7f0961400640 1 -- 172.21.15.19:0/2700864228 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f094800c360 con 0x7f0948003a00 2024-03-28T13:17:51.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.218+0000 7f0961400640 1 -- 172.21.15.19:0/2700864228 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f094800c560 con 0x7f0948004800 2024-03-28T13:17:51.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.218+0000 7f0962800640 1 --2- 172.21.15.19:0/2700864228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0948005290 0x7f094800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.218+0000 7f0961e00640 1 --2- 172.21.15.19:0/2700864228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948003a00 0x7f0948003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.219+0000 7f0959400640 1 --2- 172.21.15.19:0/2700864228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0948004800 0x7f0948004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.219+0000 7f0962800640 1 -- 172.21.15.19:0/2700864228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0948004800 msgr2=0x7f0948004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:51.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.219+0000 7f0962800640 1 --2- 172.21.15.19:0/2700864228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0948004800 0x7f0948004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.219+0000 7f0962800640 1 -- 172.21.15.19:0/2700864228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948003a00 msgr2=0x7f0948003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:51.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.219+0000 7f0962800640 1 --2- 172.21.15.19:0/2700864228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948003a00 0x7f0948003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.219+0000 7f0962800640 1 -- 172.21.15.19:0/2700864228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0948003520 con 0x7f0948005290 2024-03-28T13:17:51.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.219+0000 7f0959400640 1 --2- 172.21.15.19:0/2700864228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0948004800 0x7f0948004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:51.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.220+0000 7f0962800640 1 -- 172.21.15.19:0/2700864228 reap_dead start 2024-03-28T13:17:51.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.220+0000 7f0962800640 1 --2- 172.21.15.19:0/2700864228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0948005290 0x7f094800b9c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f095c00ba30 tx=0x7f0948005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e0cb712717f47b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:51.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.220+0000 7f0961400640 1 -- 172.21.15.19:0/2700864228 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c036030 con 0x7f0948005290 2024-03-28T13:17:51.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.220+0000 7f0961400640 1 -- 172.21.15.19:0/2700864228 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c004020 con 0x7f0948005290 2024-03-28T13:17:51.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.220+0000 7f0961400640 1 -- 172.21.15.19:0/2700864228 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c00b600 con 0x7f0948005290 2024-03-28T13:17:51.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.221+0000 7f0968e00640 1 -- 172.21.15.19:0/2700864228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0948005290 msgr2=0x7f094800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.221+0000 7f0968e00640 1 --2- 172.21.15.19:0/2700864228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0948005290 0x7f094800b9c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f095c00ba30 tx=0x7f0948005830 comp rx=0 tx=0).stop 2024-03-28T13:17:51.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.221+0000 7f0968e00640 1 -- 172.21.15.19:0/2700864228 shutdown_connections 2024-03-28T13:17:51.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.221+0000 7f0968e00640 1 --2- 172.21.15.19:0/2700864228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0948005290 0x7f094800b9c0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.221+0000 7f0968e00640 1 -- 172.21.15.19:0/2700864228 >> 172.21.15.19:0/2700864228 conn(0x7f09640b6560 msgr2=0x7f09640b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:51.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.221+0000 7f0968e00640 1 -- 172.21.15.19:0/2700864228 shutdown_connections 2024-03-28T13:17:51.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.221+0000 7f0968e00640 1 -- 172.21.15.19:0/2700864228 wait complete. 2024-03-28T13:17:51.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.223+0000 7f0968e00640 1 Processor -- start 2024-03-28T13:17:51.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.223+0000 7f0968e00640 1 -- start start 2024-03-28T13:17:51.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.224+0000 7f0968e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948005290 0x7f096414d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.224+0000 7f0968e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0964142b80 0x7f096414db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.224+0000 7f0968e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09641f7990 0x7f09641f7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.224+0000 7f0968e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f09640b5b10 con 0x7f0964142b80 2024-03-28T13:17:51.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.224+0000 7f0968e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f096415eb80 con 0x7f0948005290 2024-03-28T13:17:51.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.224+0000 7f0968e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f09640b4220 con 0x7f09641f7990 2024-03-28T13:17:51.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.224+0000 7f0959400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0964142b80 0x7f096414db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.224+0000 7f0959400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0964142b80 0x7f096414db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53658/0 (socket says 172.21.15.19:53658) 2024-03-28T13:17:51.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.224+0000 7f0959400640 1 -- 172.21.15.19:0/1437413352 learned_addr learned my addr 172.21.15.19:0/1437413352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:51.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.224+0000 7f0961e00640 1 --2- 172.21.15.19:0/1437413352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948005290 0x7f096414d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.225+0000 7f0962800640 1 --2- 172.21.15.19:0/1437413352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09641f7990 0x7f09641f7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.225+0000 7f0959400640 1 -- 172.21.15.19:0/1437413352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09641f7990 msgr2=0x7f09641f7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.225+0000 7f0959400640 1 --2- 172.21.15.19:0/1437413352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09641f7990 0x7f09641f7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.225+0000 7f0959400640 1 -- 172.21.15.19:0/1437413352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948005290 msgr2=0x7f096414d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.225+0000 7f0959400640 1 --2- 172.21.15.19:0/1437413352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948005290 0x7f096414d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.225+0000 7f0959400640 1 -- 172.21.15.19:0/1437413352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096414f7c0 con 0x7f0964142b80 2024-03-28T13:17:51.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.225+0000 7f0962800640 1 --2- 172.21.15.19:0/1437413352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09641f7990 0x7f09641f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:51.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.225+0000 7f0959400640 1 --2- 172.21.15.19:0/1437413352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0964142b80 0x7f096414db10 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f094c00d5f0 tx=0x7f094c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:51.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.225+0000 7f095aa00640 1 -- 172.21.15.19:0/1437413352 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c016070 con 0x7f0964142b80 2024-03-28T13:17:51.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.225+0000 7f095aa00640 1 -- 172.21.15.19:0/1437413352 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f094c007240 con 0x7f0964142b80 2024-03-28T13:17:51.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.225+0000 7f095aa00640 1 -- 172.21.15.19:0/1437413352 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c007630 con 0x7f0964142b80 2024-03-28T13:17:51.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.225+0000 7f0968e00640 1 -- 172.21.15.19:0/1437413352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096414fa60 con 0x7f0964142b80 2024-03-28T13:17:51.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.226+0000 7f0968e00640 1 -- 172.21.15.19:0/1437413352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0964150020 con 0x7f0964142b80 2024-03-28T13:17:51.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.228+0000 7f095aa00640 1 -- 172.21.15.19:0/1437413352 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f094c0067d0 con 0x7f0964142b80 2024-03-28T13:17:51.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.229+0000 7f095aa00640 1 --2- 172.21.15.19:0/1437413352 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f094006c530 0x7f094006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.229+0000 7f095aa00640 1 -- 172.21.15.19:0/1437413352 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f094c08b670 con 0x7f0964142b80 2024-03-28T13:17:51.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.229+0000 7f096bc2e740 1 Processor -- start 2024-03-28T13:17:51.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.229+0000 7f0961e00640 1 --2- 172.21.15.19:0/1437413352 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f094006c530 0x7f094006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.230+0000 7f096bc2e740 1 -- start start 2024-03-28T13:17:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.230+0000 7f096bc2e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bbfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f096bc2e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b8f690 0x55f4a7bbc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f096bc2e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7b93240 0x55f4a7bbce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f096bc2e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7af95a0 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f096bc2e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7aa1390 con 0x55f4a7b8f690 2024-03-28T13:17:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f096bc2e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55f4a7af5550 con 0x55f4a7b93240 2024-03-28T13:17:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f0961e00640 1 --2- 172.21.15.19:0/1437413352 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f094006c530 0x7f094006e9f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0954002230 tx=0x7f0954009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:51.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f0961e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bbfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f0961e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bbfaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53668/0 (socket says 172.21.15.19:53668) 2024-03-28T13:17:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f0961e00640 1 -- 172.21.15.19:0/2129131723 learned_addr learned my addr 172.21.15.19:0/2129131723 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f0959400640 1 --2- 172.21.15.19:0/2129131723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b8f690 0x55f4a7bbc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f0962800640 1 --2- 172.21.15.19:0/2129131723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7b93240 0x55f4a7bbce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f0961e00640 1 -- 172.21.15.19:0/2129131723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7b93240 msgr2=0x55f4a7bbce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f0961e00640 1 --2- 172.21.15.19:0/2129131723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7b93240 0x55f4a7bbce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f0961e00640 1 -- 172.21.15.19:0/2129131723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b8f690 msgr2=0x55f4a7bbc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f0961e00640 1 --2- 172.21.15.19:0/2129131723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b8f690 0x55f4a7bbc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f0961e00640 1 -- 172.21.15.19:0/2129131723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a78d97c0 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.231+0000 7f0959400640 1 --2- 172.21.15.19:0/2129131723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b8f690 0x55f4a7bbc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.232+0000 7f0961e00640 1 --2- 172.21.15.19:0/2129131723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bbfaa0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f0954005e50 tx=0x7f0954013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff22b35c81dd417 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.232+0000 7f0952a00640 1 -- 172.21.15.19:0/2129131723 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954026070 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.232+0000 7f0952a00640 1 -- 172.21.15.19:0/2129131723 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095401c050 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.232+0000 7f0952a00640 1 -- 172.21.15.19:0/2129131723 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954021810 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.233+0000 7f096bc2e740 1 -- 172.21.15.19:0/2129131723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 msgr2=0x55f4a7bbfaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.233+0000 7f096bc2e740 1 --2- 172.21.15.19:0/2129131723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bbfaa0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f0954005e50 tx=0x7f0954013000 comp rx=0 tx=0).stop 2024-03-28T13:17:51.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.233+0000 7f096bc2e740 1 -- 172.21.15.19:0/2129131723 shutdown_connections 2024-03-28T13:17:51.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.233+0000 7f096bc2e740 1 --2- 172.21.15.19:0/2129131723 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7b93240 0x55f4a7bbce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.233+0000 7f096bc2e740 1 --2- 172.21.15.19:0/2129131723 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b8f690 0x55f4a7bbc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.233+0000 7f096bc2e740 1 --2- 172.21.15.19:0/2129131723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bbfaa0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.233+0000 7f096bc2e740 1 -- 172.21.15.19:0/2129131723 >> 172.21.15.19:0/2129131723 conn(0x55f4a7b91950 msgr2=0x55f4a7bda4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:51.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.233+0000 7f096bc2e740 1 -- 172.21.15.19:0/2129131723 shutdown_connections 2024-03-28T13:17:51.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.233+0000 7f096bc2e740 1 -- 172.21.15.19:0/2129131723 wait complete. 2024-03-28T13:17:51.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.234+0000 7f096bc2e740 1 -- start start 2024-03-28T13:17:51.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.234+0000 7f096bc2e740 1 Processor -- start 2024-03-28T13:17:51.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.235+0000 7f096bc2e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7b93240 0x55f4a7bb4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.235+0000 7f096bc2e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bb61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.235+0000 7f096bc2e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7bb9990 0x55f4a7bb7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.235+0000 7f096bc2e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7b943f0 con 0x55f4a7b93240 2024-03-28T13:17:51.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.235+0000 7f096bc2e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7ab4780 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.235+0000 7f096bc2e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55f4a7b937e0 con 0x55f4a7bb9990 2024-03-28T13:17:51.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.235+0000 7f096bc2e740 10 client.0 _ll_register_callbacks cb 0x55f4a7af0fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:17:51.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.235+0000 7f0961e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7b93240 0x55f4a7bb4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.235+0000 7f0961e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7b93240 0x55f4a7bb4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53674/0 (socket says 172.21.15.19:53674) 2024-03-28T13:17:51.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.235+0000 7f0961e00640 1 -- 172.21.15.19:0/89916415 learned_addr learned my addr 172.21.15.19:0/89916415 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.236+0000 7f0959400640 1 --2- 172.21.15.19:0/89916415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bb61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.236+0000 7f0961e00640 1 -- 172.21.15.19:0/89916415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7bb9990 msgr2=0x55f4a7bb7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.236+0000 7f0961e00640 1 --2- 172.21.15.19:0/89916415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7bb9990 0x55f4a7bb7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.236+0000 7f0961e00640 1 -- 172.21.15.19:0/89916415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7bbf6c0 msgr2=0x55f4a7bb61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.236+0000 7f0961e00640 1 --2- 172.21.15.19:0/89916415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bb61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.236+0000 7f0961e00640 1 -- 172.21.15.19:0/89916415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a7939f50 con 0x55f4a7b93240 2024-03-28T13:17:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.236+0000 7f0959400640 1 --2- 172.21.15.19:0/89916415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bb61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.236+0000 7f0961e00640 1 --2- 172.21.15.19:0/89916415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7b93240 0x55f4a7bb4e90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f0954001aa0 tx=0x7f0954001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc823b1da223ff5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.236+0000 7f0947400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:17:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.236+0000 7f0947400640 1 -- 172.21.15.19:0/89916415 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954031020 con 0x55f4a7b93240 2024-03-28T13:17:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.236+0000 7f0947400640 1 -- 172.21.15.19:0/89916415 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f0954021dd0 con 0x55f4a7b93240 2024-03-28T13:17:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.236+0000 7f096bc2e740 1 -- 172.21.15.19:0/89916415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f095401c050 con 0x55f4a7b93240 2024-03-28T13:17:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.237+0000 7f0947400640 1 -- 172.21.15.19:0/89916415 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f095403b050 con 0x55f4a7b93240 2024-03-28T13:17:51.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.237+0000 7f096bc2e740 10 client.7614 fetch_fsmap learned FSMap version 68 2024-03-28T13:17:51.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.237+0000 7f096bc2e740 1 -- 172.21.15.19:0/89916415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55f4a77cd0b0 con 0x55f4a7b93240 2024-03-28T13:17:51.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.237+0000 7f0947400640 1 -- 172.21.15.19:0/89916415 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095401c050 con 0x55f4a7b93240 2024-03-28T13:17:51.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.237+0000 7f0947400640 1 -- 172.21.15.19:0/89916415 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f095403e020 con 0x55f4a7b93240 2024-03-28T13:17:51.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.237+0000 7f096bc2e740 10 client.7614 fetch_fsmap finished waiting for FSMap version 68 2024-03-28T13:17:51.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.237+0000 7f096bc2e740 10 client.7614 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T13:17:51.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.237+0000 7f096bc2e740 20 client.7614 populate_metadata read hostname 'smithi019' 2024-03-28T13:17:51.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.237+0000 7f096bc2e740 1 --2- 172.21.15.19:0/89916415 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55f4a7bc1250 0x55f4a7bc1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.237+0000 7f096bc2e740 4 client.7614 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:17:51.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.237+0000 7f096bc2e740 1 -- 172.21.15.19:0/89916415 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4a7acd870 con 0x55f4a7bc1250 2024-03-28T13:17:51.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.238+0000 7f0959400640 1 --2- 172.21.15.19:0/89916415 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55f4a7bc1250 0x55f4a7bc1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.239+0000 7f0959400640 1 --2- 172.21.15.19:0/89916415 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55f4a7bc1250 0x55f4a7bc1630 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6496e25cc97473a9 server_cookie=56475557714f266 in_seq=0 out_seq=0 2024-03-28T13:17:51.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.239+0000 7f0947400640 10 client.7614 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.240+0000 7f0947400640 1 -- 172.21.15.19:0/89916415 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f4a7acd870 con 0x55f4a7bc1250 2024-03-28T13:17:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.240+0000 7f0947400640 10 client.7614 handle_command_reply: tid=0 2024-03-28T13:17:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.241+0000 7f096bc2e740 1 client.7614 shutdown 2024-03-28T13:17:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.241+0000 7f096bc2e740 1 -- 172.21.15.19:0/89916415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7b93240 msgr2=0x55f4a7bb4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.241+0000 7f096bc2e740 1 --2- 172.21.15.19:0/89916415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7b93240 0x55f4a7bb4e90 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f0954001aa0 tx=0x7f0954001cf0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.241+0000 7f096bc2e740 1 -- 172.21.15.19:0/89916415 shutdown_connections 2024-03-28T13:17:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.241+0000 7f096bc2e740 1 --2- 172.21.15.19:0/89916415 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55f4a7bc1250 0x55f4a7bc1630 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.241+0000 7f096bc2e740 1 --2- 172.21.15.19:0/89916415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7bb9990 0x55f4a7bb7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.241+0000 7f096bc2e740 1 --2- 172.21.15.19:0/89916415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bb61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.241+0000 7f096bc2e740 1 --2- 172.21.15.19:0/89916415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7b93240 0x55f4a7bb4e90 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.241+0000 7f096bc2e740 1 -- 172.21.15.19:0/89916415 >> 172.21.15.19:0/89916415 conn(0x55f4a7b8f690 msgr2=0x55f4a7bdb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:51.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.241+0000 7f0947400640 0 client.7614 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:51.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.241+0000 7f096bc2e740 1 -- 172.21.15.19:0/89916415 shutdown_connections 2024-03-28T13:17:51.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.241+0000 7f096bc2e740 1 -- 172.21.15.19:0/89916415 wait complete. 2024-03-28T13:17:51.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.241+0000 7f096bc2e740 20 client.7614 trim_cache size 0 max 16384 2024-03-28T13:17:51.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.256+0000 7f096bc2e740 1 Processor -- start 2024-03-28T13:17:51.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.256+0000 7f096bc2e740 1 -- start start 2024-03-28T13:17:51.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.257+0000 7f096bc2e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b93240 0x55f4a7be7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.257+0000 7f096bc2e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7bb9990 0x55f4a7bb44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.257+0000 7f096bc2e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bb6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.257+0000 7f096bc2e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f09642000b0 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.257+0000 7f096bc2e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f09641519b0 con 0x55f4a7b93240 2024-03-28T13:17:51.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.257+0000 7f096bc2e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f09641ffaf0 con 0x55f4a7bb9990 2024-03-28T13:17:51.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.257+0000 7f0962800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bb6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.257+0000 7f0962800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bb6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53686/0 (socket says 172.21.15.19:53686) 2024-03-28T13:17:51.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.257+0000 7f0962800640 1 -- 172.21.15.19:0/2610237759 learned_addr learned my addr 172.21.15.19:0/2610237759 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:51.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.258+0000 7f0961e00640 1 --2- 172.21.15.19:0/2610237759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b93240 0x55f4a7be7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.258+0000 7f0959400640 1 --2- 172.21.15.19:0/2610237759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7bb9990 0x55f4a7bb44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.258+0000 7f0962800640 1 -- 172.21.15.19:0/2610237759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7bb9990 msgr2=0x55f4a7bb44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:51.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.258+0000 7f0962800640 1 --2- 172.21.15.19:0/2610237759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7bb9990 0x55f4a7bb44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.258+0000 7f0962800640 1 -- 172.21.15.19:0/2610237759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b93240 msgr2=0x55f4a7be7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.258+0000 7f0962800640 1 --2- 172.21.15.19:0/2610237759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b93240 0x55f4a7be7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.258+0000 7f0962800640 1 -- 172.21.15.19:0/2610237759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4a7b8f4c0 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.258+0000 7f0961e00640 1 --2- 172.21.15.19:0/2610237759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b93240 0x55f4a7be7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.258+0000 7f0959400640 1 --2- 172.21.15.19:0/2610237759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7bb9990 0x55f4a7bb44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.258+0000 7f0962800640 1 --2- 172.21.15.19:0/2610237759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bb6810 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f095c006ab0 tx=0x7f095c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547b6475901487b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.259+0000 7f0952000640 1 -- 172.21.15.19:0/2610237759 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c040070 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.259+0000 7f0952000640 1 -- 172.21.15.19:0/2610237759 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c004240 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.259+0000 7f0952000640 1 -- 172.21.15.19:0/2610237759 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c036040 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.259+0000 7f096bc2e740 1 -- 172.21.15.19:0/2610237759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 msgr2=0x55f4a7bb6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.259+0000 7f096bc2e740 1 --2- 172.21.15.19:0/2610237759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bb6810 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f095c006ab0 tx=0x7f095c02e000 comp rx=0 tx=0).stop 2024-03-28T13:17:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.259+0000 7f096bc2e740 1 -- 172.21.15.19:0/2610237759 shutdown_connections 2024-03-28T13:17:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.259+0000 7f096bc2e740 1 --2- 172.21.15.19:0/2610237759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bb6810 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.259+0000 7f096bc2e740 1 --2- 172.21.15.19:0/2610237759 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7bb9990 0x55f4a7bb44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.259+0000 7f096bc2e740 1 --2- 172.21.15.19:0/2610237759 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b93240 0x55f4a7be7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.259+0000 7f096bc2e740 1 -- 172.21.15.19:0/2610237759 >> 172.21.15.19:0/2610237759 conn(0x55f4a7b8f690 msgr2=0x55f4a7bc3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:51.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.260+0000 7f096bc2e740 1 -- 172.21.15.19:0/2610237759 shutdown_connections 2024-03-28T13:17:51.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.260+0000 7f096bc2e740 1 -- 172.21.15.19:0/2610237759 wait complete. 2024-03-28T13:17:51.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.260+0000 7f096bc2e740 1 -- start start 2024-03-28T13:17:51.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.260+0000 7f096bc2e740 1 Processor -- start 2024-03-28T13:17:51.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.261+0000 7f096bc2e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b93240 0x55f4a7be6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.261+0000 7f096bc2e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7bb9990 0x55f4a7be7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.261+0000 7f096bc2e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.261+0000 7f096bc2e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7bc22d0 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.261+0000 7f096bc2e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f4a7bba6a0 con 0x55f4a7b93240 2024-03-28T13:17:51.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.261+0000 7f096bc2e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55f4a7bee450 con 0x55f4a7bb9990 2024-03-28T13:17:51.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.261+0000 7f096bc2e740 10 client.0 _ll_register_callbacks cb 0x7f0954024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:17:51.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.262+0000 7f0962800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.262+0000 7f0962800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bea830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53690/0 (socket says 172.21.15.19:53690) 2024-03-28T13:17:51.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.262+0000 7f0962800640 1 -- 172.21.15.19:0/2690925504 learned_addr learned my addr 172.21.15.19:0/2690925504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:51.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.262+0000 7f0961e00640 1 --2- 172.21.15.19:0/2690925504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b93240 0x55f4a7be6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.262+0000 7f0959400640 1 --2- 172.21.15.19:0/2690925504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7bb9990 0x55f4a7be7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.262+0000 7f0962800640 1 -- 172.21.15.19:0/2690925504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7bb9990 msgr2=0x55f4a7be7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:51.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.262+0000 7f0962800640 1 --2- 172.21.15.19:0/2690925504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7bb9990 0x55f4a7be7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.262+0000 7f0962800640 1 -- 172.21.15.19:0/2690925504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b93240 msgr2=0x55f4a7be6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:51.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.262+0000 7f0962800640 1 --2- 172.21.15.19:0/2690925504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b93240 0x55f4a7be6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.262+0000 7f0962800640 1 -- 172.21.15.19:0/2690925504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4a7b93bf0 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.263+0000 7f0962800640 1 --2- 172.21.15.19:0/2690925504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bea830 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f095c00bb50 tx=0x7f095c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263574eae64875c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:51.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.263+0000 7f0947400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:17:51.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.263+0000 7f0947400640 1 -- 172.21.15.19:0/2690925504 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c036040 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.263+0000 7f0947400640 1 -- 172.21.15.19:0/2690925504 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f095c003ae0 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.263+0000 7f096bc2e740 1 -- 172.21.15.19:0/2690925504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f095c004240 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.263+0000 7f0947400640 1 -- 172.21.15.19:0/2690925504 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f095c053050 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.263+0000 7f096bc2e740 10 client.7620 fetch_fsmap learned FSMap version 68 2024-03-28T13:17:51.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.263+0000 7f096bc2e740 1 -- 172.21.15.19:0/2690925504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=2+}) -- 0x55f4a7b94770 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.264+0000 7f0947400640 1 -- 172.21.15.19:0/2690925504 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c004240 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.264+0000 7f0947400640 1 -- 172.21.15.19:0/2690925504 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 68) ==== 3494+0+0 (secure 0 0 0) 0x7f095c056020 con 0x55f4a7bbf6c0 2024-03-28T13:17:51.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.264+0000 7f096bc2e740 10 client.7620 fetch_fsmap finished waiting for FSMap version 68 2024-03-28T13:17:51.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.264+0000 7f096bc2e740 10 client.7620 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-28T13:17:51.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.264+0000 7f096bc2e740 20 client.7620 populate_metadata read hostname 'smithi019' 2024-03-28T13:17:51.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.264+0000 7f096bc2e740 1 --2- 172.21.15.19:0/2690925504 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55f4a7beb0e0 0x55f4a7bda2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.264+0000 7f096bc2e740 4 client.7620 mds_command: new command op to 6175 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-28T13:17:51.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.264+0000 7f096bc2e740 1 -- 172.21.15.19:0/2690925504 --> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55f4a7b06ae0 con 0x55f4a7beb0e0 2024-03-28T13:17:51.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.264+0000 7f0961e00640 1 --2- 172.21.15.19:0/2690925504 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55f4a7beb0e0 0x55f4a7bda2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.265+0000 7f0961e00640 1 --2- 172.21.15.19:0/2690925504 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55f4a7beb0e0 0x55f4a7bda2b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=621734dd3ac40bbe server_cookie=358a52ccc25b907a in_seq=0 out_seq=0 2024-03-28T13:17:51.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.265+0000 7f0947400640 10 client.7620 ms_handle_connect on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:51.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.266+0000 7f0947400640 1 -- 172.21.15.19:0/2690925504 <== mds.0 v2:172.21.15.188:6834/1749969053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+144 (crc 0 0 0) 0x55f4a7b06ae0 con 0x55f4a7beb0e0 2024-03-28T13:17:51.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.266+0000 7f0947400640 10 client.7620 handle_command_reply: tid=0 2024-03-28T13:17:51.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.266+0000 7f096bc2e740 1 client.7620 shutdown 2024-03-28T13:17:51.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.266+0000 7f096bc2e740 1 -- 172.21.15.19:0/2690925504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 msgr2=0x55f4a7bea830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.266+0000 7f096bc2e740 1 --2- 172.21.15.19:0/2690925504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bea830 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f095c00bb50 tx=0x7f095c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.266+0000 7f096bc2e740 1 -- 172.21.15.19:0/2690925504 shutdown_connections 2024-03-28T13:17:51.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.266+0000 7f096bc2e740 1 --2- 172.21.15.19:0/2690925504 >> [v2:172.21.15.188:6834/1749969053,v1:172.21.15.188:6836/1749969053] conn(0x55f4a7beb0e0 0x55f4a7bda2b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.266+0000 7f096bc2e740 1 --2- 172.21.15.19:0/2690925504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4a7bbf6c0 0x55f4a7bea830 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.266+0000 7f096bc2e740 1 --2- 172.21.15.19:0/2690925504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f4a7bb9990 0x55f4a7be7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.266+0000 7f096bc2e740 1 --2- 172.21.15.19:0/2690925504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f4a7b93240 0x55f4a7be6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.266+0000 7f096bc2e740 1 -- 172.21.15.19:0/2690925504 >> 172.21.15.19:0/2690925504 conn(0x55f4a7b8f690 msgr2=0x55f4a7b8e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:51.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.266+0000 7f0947400640 0 client.7620 ms_handle_reset on v2:172.21.15.188:6834/1749969053 2024-03-28T13:17:51.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.266+0000 7f096bc2e740 1 -- 172.21.15.19:0/2690925504 shutdown_connections 2024-03-28T13:17:51.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.266+0000 7f096bc2e740 1 -- 172.21.15.19:0/2690925504 wait complete. 2024-03-28T13:17:51.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.266+0000 7f096bc2e740 20 client.7620 trim_cache size 0 max 16384 2024-03-28T13:17:51.268 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:17:51.268 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-03-28T13:17:51.268 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:17:51.269 INFO:teuthology.orchestra.run.smithi019.stdout: "damage_type": "backtrace", 2024-03-28T13:17:51.269 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 3207548007, 2024-03-28T13:17:51.269 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 1099511627777, 2024-03-28T13:17:51.269 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/testdir/hardlink" 2024-03-28T13:17:51.269 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:17:51.269 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-03-28T13:17:51.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.267+0000 7f0968e00640 1 -- 172.21.15.19:0/1437413352 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f094006c530 msgr2=0x7f094006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.267+0000 7f0968e00640 1 --2- 172.21.15.19:0/1437413352 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f094006c530 0x7f094006e9f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0954002230 tx=0x7f0954009040 comp rx=0 tx=0).stop 2024-03-28T13:17:51.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.267+0000 7f0968e00640 1 -- 172.21.15.19:0/1437413352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0964142b80 msgr2=0x7f096414db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.267+0000 7f0968e00640 1 --2- 172.21.15.19:0/1437413352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0964142b80 0x7f096414db10 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f094c00d5f0 tx=0x7f094c00dab0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.267+0000 7f0968e00640 1 -- 172.21.15.19:0/1437413352 shutdown_connections 2024-03-28T13:17:51.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.267+0000 7f0968e00640 1 --2- 172.21.15.19:0/1437413352 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f094006c530 0x7f094006e9f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.267+0000 7f0968e00640 1 --2- 172.21.15.19:0/1437413352 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09641f7990 0x7f09641f7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.267+0000 7f0968e00640 1 --2- 172.21.15.19:0/1437413352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0964142b80 0x7f096414db10 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.267+0000 7f0968e00640 1 --2- 172.21.15.19:0/1437413352 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0948005290 0x7f096414d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.267+0000 7f0968e00640 1 -- 172.21.15.19:0/1437413352 >> 172.21.15.19:0/1437413352 conn(0x7f09640b6560 msgr2=0x7f09640b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:51.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.267+0000 7f0968e00640 1 -- 172.21.15.19:0/1437413352 shutdown_connections 2024-03-28T13:17:51.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.267+0000 7f0968e00640 1 -- 172.21.15.19:0/1437413352 wait complete. 2024-03-28T13:17:51.284 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:17:51.285 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:17:51.337 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:17:51.337 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:17:51.391 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:17:51.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.617+0000 7f57cb000640 1 Processor -- start 2024-03-28T13:17:51.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.617+0000 7f57cb000640 1 -- start start 2024-03-28T13:17:51.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.618+0000 7f57cb000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f57c40cf960 0x7f57c406e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.618+0000 7f57cb000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57c415c9e0 con 0x7f57c415c600 2024-03-28T13:17:51.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.618+0000 7f57cb000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57c406f6a0 con 0x7f57c406ed60 2024-03-28T13:17:51.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.618+0000 7f57cb000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f57c406f980 con 0x7f57c40cf960 2024-03-28T13:17:51.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.619+0000 7f57c3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f57c415c600 0x7f57c40cf250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34772/0 (socket says 172.21.15.19:34772) 2024-03-28T13:17:51.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.619+0000 7f57c3e00640 1 -- 172.21.15.19:0/1872172935 learned_addr learned my addr 172.21.15.19:0/1872172935 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:17:51.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.619+0000 7f57c3400640 1 --2- 172.21.15.19:0/1872172935 >> v2:172.21.15.19:3300/0 conn(0x7f57c40cf960 0x7f57c406e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.619+0000 7f57c3400640 1 -- 172.21.15.19:0/1872172935 >> v1:172.21.15.188:6789/0 conn(0x7f57c406ed60 legacy=0x7f57c40b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:51.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.620+0000 7f57c3400640 1 -- 172.21.15.19:0/1872172935 >> v1:172.21.15.19:6789/0 conn(0x7f57c415c600 legacy=0x7f57c40cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.620+0000 7f57c2a00640 1 -- 172.21.15.19:0/1872172935 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3361064084 0 0) 0x7f57c415c9e0 con 0x7f57c415c600 2024-03-28T13:17:51.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.620+0000 7f57c3400640 1 -- 172.21.15.19:0/1872172935 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c406fc60 con 0x7f57c40cf960 2024-03-28T13:17:51.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.621+0000 7f57c3400640 1 --2- 172.21.15.19:0/1872172935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c40cf960 0x7f57c406e820 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0097a0 tx=0x7f57ac02b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8fef2d6902de62 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:51.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.621+0000 7f57c2a00640 1 -- 172.21.15.19:0/1872172935 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac02f810 con 0x7f57c40cf960 2024-03-28T13:17:51.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.621+0000 7f57c2a00640 1 -- 172.21.15.19:0/1872172935 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ac02f9b0 con 0x7f57c40cf960 2024-03-28T13:17:51.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.621+0000 7f57c2a00640 1 -- 172.21.15.19:0/1872172935 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac02fd80 con 0x7f57c40cf960 2024-03-28T13:17:51.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.622+0000 7f57cb000640 1 -- 172.21.15.19:0/1872172935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c40cf960 msgr2=0x7f57c406e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.622+0000 7f57cb000640 1 --2- 172.21.15.19:0/1872172935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c40cf960 0x7f57c406e820 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f57ac0097a0 tx=0x7f57ac02b9f0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.622+0000 7f57cb000640 1 -- 172.21.15.19:0/1872172935 shutdown_connections 2024-03-28T13:17:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.622+0000 7f57cb000640 1 --2- 172.21.15.19:0/1872172935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c40cf960 0x7f57c406e820 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.622+0000 7f57cb000640 1 -- 172.21.15.19:0/1872172935 >> 172.21.15.19:0/1872172935 conn(0x7f57c40b2cb0 msgr2=0x7f57c40b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.622+0000 7f57cb000640 1 -- 172.21.15.19:0/1872172935 shutdown_connections 2024-03-28T13:17:51.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.622+0000 7f57cb000640 1 -- 172.21.15.19:0/1872172935 wait complete. 2024-03-28T13:17:51.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.624+0000 7f57cb000640 1 Processor -- start 2024-03-28T13:17:51.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.624+0000 7f57cb000640 1 -- start start 2024-03-28T13:17:51.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.625+0000 7f57cb000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57c406ed60 0x7f57c414ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.625+0000 7f57cb000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57c40cf960 0x7f57c4156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.625+0000 7f57cb000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c415c600 0x7f57c415ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.625+0000 7f57cb000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f57c406f140 con 0x7f57c415c600 2024-03-28T13:17:51.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.625+0000 7f57cb000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f57c4070220 con 0x7f57c406ed60 2024-03-28T13:17:51.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.625+0000 7f57cb000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f57c406f2c0 con 0x7f57c40cf960 2024-03-28T13:17:51.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.625+0000 7f57c9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c415c600 0x7f57c415ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.625+0000 7f57c9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c415c600 0x7f57c415ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53712/0 (socket says 172.21.15.19:53712) 2024-03-28T13:17:51.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.625+0000 7f57c9000640 1 -- 172.21.15.19:0/2317223615 learned_addr learned my addr 172.21.15.19:0/2317223615 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:51.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.625+0000 7f57c3e00640 1 --2- 172.21.15.19:0/2317223615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57c406ed60 0x7f57c414ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.625+0000 7f57c3400640 1 --2- 172.21.15.19:0/2317223615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57c40cf960 0x7f57c4156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.626+0000 7f57c9000640 1 -- 172.21.15.19:0/2317223615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57c40cf960 msgr2=0x7f57c4156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.626+0000 7f57c9000640 1 --2- 172.21.15.19:0/2317223615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57c40cf960 0x7f57c4156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.626+0000 7f57c9000640 1 -- 172.21.15.19:0/2317223615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57c406ed60 msgr2=0x7f57c414ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.626+0000 7f57c9000640 1 --2- 172.21.15.19:0/2317223615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57c406ed60 0x7f57c414ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.626+0000 7f57c9000640 1 -- 172.21.15.19:0/2317223615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c4156b40 con 0x7f57c415c600 2024-03-28T13:17:51.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.626+0000 7f57c9000640 1 --2- 172.21.15.19:0/2317223615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c415c600 0x7f57c415ed40 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00c400 tx=0x7f57bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:51.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.626+0000 7f57bbe00640 1 -- 172.21.15.19:0/2317223615 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc017070 con 0x7f57c415c600 2024-03-28T13:17:51.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.626+0000 7f57bbe00640 1 -- 172.21.15.19:0/2317223615 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57bc0052d0 con 0x7f57c415c600 2024-03-28T13:17:51.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.626+0000 7f57bbe00640 1 -- 172.21.15.19:0/2317223615 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc003810 con 0x7f57c415c600 2024-03-28T13:17:51.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.626+0000 7f57cb000640 1 -- 172.21.15.19:0/2317223615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c4151780 con 0x7f57c415c600 2024-03-28T13:17:51.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.627+0000 7f57cb000640 1 -- 172.21.15.19:0/2317223615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c414f490 con 0x7f57c415c600 2024-03-28T13:17:51.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.628+0000 7f57cb000640 1 -- 172.21.15.19:0/2317223615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c41515b0 con 0x7f57c415c600 2024-03-28T13:17:51.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.629+0000 7f57bbe00640 1 -- 172.21.15.19:0/2317223615 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f57bc01a070 con 0x7f57c415c600 2024-03-28T13:17:51.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.630+0000 7f57bbe00640 1 --2- 172.21.15.19:0/2317223615 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f57a006c560 0x7f57a006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:51.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.630+0000 7f57bbe00640 1 -- 172.21.15.19:0/2317223615 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f57bc08abb0 con 0x7f57c415c600 2024-03-28T13:17:51.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.630+0000 7f57c3e00640 1 --2- 172.21.15.19:0/2317223615 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f57a006c560 0x7f57a006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:51.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.631+0000 7f57c3e00640 1 --2- 172.21.15.19:0/2317223615 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f57a006c560 0x7f57a006ea20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f57b40012e0 tx=0x7f57b4005bf0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:51.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.634+0000 7f57bbe00640 1 -- 172.21.15.19:0/2317223615 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f57bc0590d0 con 0x7f57c415c600 2024-03-28T13:17:51.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.843+0000 7f57cb000640 1 -- 172.21.15.19:0/2317223615 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57c40685c0 con 0x7f57c415c600 2024-03-28T13:17:51.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.846+0000 7f57bbe00640 1 -- 172.21.15.19:0/2317223615 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16803 (secure 0 0 0) 0x7f57bc0589d0 con 0x7f57c415c600 2024-03-28T13:17:51.846 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:17:51.846 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":93,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:16:39.493005+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:16:23.901120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-28T13:16:24.990904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6836/2260087679":"2024-03-29T13:16:39.492982+0000","172.21.15.188:6834/2260087679":"2024-03-29T13:16:39.492982+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:17:51.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.848+0000 7f57cb000640 1 -- 172.21.15.19:0/2317223615 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f57a006c560 msgr2=0x7f57a006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.849+0000 7f57cb000640 1 --2- 172.21.15.19:0/2317223615 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f57a006c560 0x7f57a006ea20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f57b40012e0 tx=0x7f57b4005bf0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.849+0000 7f57cb000640 1 -- 172.21.15.19:0/2317223615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c415c600 msgr2=0x7f57c415ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:51.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.849+0000 7f57cb000640 1 --2- 172.21.15.19:0/2317223615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c415c600 0x7f57c415ed40 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00c400 tx=0x7f57bc00c8c0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.849+0000 7f57cb000640 1 -- 172.21.15.19:0/2317223615 shutdown_connections 2024-03-28T13:17:51.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.849+0000 7f57cb000640 1 --2- 172.21.15.19:0/2317223615 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f57a006c560 0x7f57a006ea20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.849+0000 7f57cb000640 1 --2- 172.21.15.19:0/2317223615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57c415c600 0x7f57c415ed40 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.849+0000 7f57cb000640 1 --2- 172.21.15.19:0/2317223615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57c40cf960 0x7f57c4156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.849+0000 7f57cb000640 1 --2- 172.21.15.19:0/2317223615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57c406ed60 0x7f57c414ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:51.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.849+0000 7f57cb000640 1 -- 172.21.15.19:0/2317223615 >> 172.21.15.19:0/2317223615 conn(0x7f57c40b2cb0 msgr2=0x7f57c40d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:51.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.849+0000 7f57cb000640 1 -- 172.21.15.19:0/2317223615 shutdown_connections 2024-03-28T13:17:51.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:51.849+0000 7f57cb000640 1 -- 172.21.15.19:0/2317223615 wait complete. 2024-03-28T13:17:51.868 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:16:23.901120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:16:24.990904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:17:51.869 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:17:52.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.099+0000 7fafe0e00640 1 Processor -- start 2024-03-28T13:17:52.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.099+0000 7fafe0e00640 1 -- start start 2024-03-28T13:17:52.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.101+0000 7fafe0e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafdc0735c0 0x7fafdc06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.101+0000 7fafe0e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafdc0b3920 con 0x7fafdc160360 2024-03-28T13:17:52.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.101+0000 7fafe0e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafdc0b3c00 con 0x7fafdc071af0 2024-03-28T13:17:52.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.101+0000 7fafe0e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fafdc0b36d0 con 0x7fafdc0735c0 2024-03-28T13:17:52.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.101+0000 7fafdb400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fafdc160360 0x7fafdc0713e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34788/0 (socket says 172.21.15.19:34788) 2024-03-28T13:17:52.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.101+0000 7fafdb400640 1 -- 172.21.15.19:0/3453750154 learned_addr learned my addr 172.21.15.19:0/3453750154 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:17:52.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafdbe00640 1 --2- 172.21.15.19:0/3453750154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafdc0735c0 0x7fafdc06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafda000640 1 -- 172.21.15.19:0/3453750154 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3818962800 0 0) 0x7fafdc0b3920 con 0x7fafdc160360 2024-03-28T13:17:52.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafda000640 1 -- 172.21.15.19:0/3453750154 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafc0003520 con 0x7fafdc160360 2024-03-28T13:17:52.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafda000640 1 -- 172.21.15.19:0/3453750154 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 621978422 0 0) 0x7fafc0003520 con 0x7fafdc160360 2024-03-28T13:17:52.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafda000640 1 -- 172.21.15.19:0/3453750154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafdc0735c0 msgr2=0x7fafdc06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:52.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafda000640 1 --2- 172.21.15.19:0/3453750154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafdc0735c0 0x7fafdc06dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafda000640 1 -- 172.21.15.19:0/3453750154 >> v1:172.21.15.188:6789/0 conn(0x7fafdc071af0 legacy=0x7fafdc06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:52.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafda000640 1 -- 172.21.15.19:0/3453750154 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc0b3ee0 con 0x7fafdc160360 2024-03-28T13:17:52.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafda000640 1 -- 172.21.15.19:0/3453750154 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fafcc0019f0 con 0x7fafdc160360 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafda000640 1 -- 172.21.15.19:0/3453750154 >> v1:172.21.15.19:6789/0 conn(0x7fafdc160360 legacy=0x7fafdc0713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafda000640 1 --2- 172.21.15.19:0/3453750154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafc00039e0 0x7fafc0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafda000640 1 --2- 172.21.15.19:0/3453750154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc0004800 0x7fafc0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafda000640 1 --2- 172.21.15.19:0/3453750154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafdc0735c0 0x7fafc0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafda000640 1 -- 172.21.15.19:0/3453750154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc0b36d0 con 0x7fafc0004800 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafda000640 1 -- 172.21.15.19:0/3453750154 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fafc000baf0 con 0x7fafc00039e0 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.102+0000 7fafda000640 1 -- 172.21.15.19:0/3453750154 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fafc000bcf0 con 0x7fafdc0735c0 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.103+0000 7fafdaa00640 1 --2- 172.21.15.19:0/3453750154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc0004800 0x7fafc0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.103+0000 7fafdbe00640 1 --2- 172.21.15.19:0/3453750154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafdc0735c0 0x7fafc0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.103+0000 7fafdb400640 1 --2- 172.21.15.19:0/3453750154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafc00039e0 0x7fafc0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.103+0000 7fafdaa00640 1 -- 172.21.15.19:0/3453750154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafdc0735c0 msgr2=0x7fafc0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.103+0000 7fafdaa00640 1 --2- 172.21.15.19:0/3453750154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafdc0735c0 0x7fafc0005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.103+0000 7fafdaa00640 1 -- 172.21.15.19:0/3453750154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafc00039e0 msgr2=0x7fafc0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.103+0000 7fafdaa00640 1 --2- 172.21.15.19:0/3453750154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafc00039e0 0x7fafc0003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.103+0000 7fafdaa00640 1 -- 172.21.15.19:0/3453750154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc0003520 con 0x7fafc0004800 2024-03-28T13:17:52.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.104+0000 7fafdaa00640 1 --2- 172.21.15.19:0/3453750154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc0004800 0x7fafc0004c80 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fafc4001750 tx=0x7fafc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec9b40e3d72efee server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:52.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.104+0000 7fafda000640 1 -- 172.21.15.19:0/3453750154 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4001370 con 0x7fafc0004800 2024-03-28T13:17:52.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.104+0000 7fafda000640 1 -- 172.21.15.19:0/3453750154 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc40052d0 con 0x7fafc0004800 2024-03-28T13:17:52.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.104+0000 7fafda000640 1 -- 172.21.15.19:0/3453750154 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4003830 con 0x7fafc0004800 2024-03-28T13:17:52.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.105+0000 7fafe0e00640 1 -- 172.21.15.19:0/3453750154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc0004800 msgr2=0x7fafc0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:52.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.105+0000 7fafe0e00640 1 --2- 172.21.15.19:0/3453750154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc0004800 0x7fafc0004c80 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fafc4001750 tx=0x7fafc0005830 comp rx=0 tx=0).stop 2024-03-28T13:17:52.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.105+0000 7fafdbe00640 1 -- 172.21.15.19:0/3453750154 reap_dead start 2024-03-28T13:17:52.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.105+0000 7fafe0e00640 1 -- 172.21.15.19:0/3453750154 shutdown_connections 2024-03-28T13:17:52.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.105+0000 7fafe0e00640 1 -- 172.21.15.19:0/3453750154 >> 172.21.15.19:0/3453750154 conn(0x7fafdc0b1ba0 msgr2=0x7fafdc0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:52.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.105+0000 7fafe0e00640 1 -- 172.21.15.19:0/3453750154 shutdown_connections 2024-03-28T13:17:52.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.105+0000 7fafe0e00640 1 -- 172.21.15.19:0/3453750154 wait complete. 2024-03-28T13:17:52.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.107+0000 7fafe0e00640 1 Processor -- start 2024-03-28T13:17:52.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.107+0000 7fafe0e00640 1 -- start start 2024-03-28T13:17:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.108+0000 7fafe0e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc0004800 0x7fafdc158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.108+0000 7fafe0e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafdc1fc4e0 0x7fafdc159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.108+0000 7fafe0e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafdc154c40 0x7fafdc153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.108+0000 7fafe0e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc0b49c0 con 0x7fafc0004800 2024-03-28T13:17:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.108+0000 7fafe0e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc0b4530 con 0x7fafdc1fc4e0 2024-03-28T13:17:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.108+0000 7fafe0e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fafdc160740 con 0x7fafdc154c40 2024-03-28T13:17:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.109+0000 7fafdb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc0004800 0x7fafdc158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.109+0000 7fafdb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc0004800 0x7fafdc158ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53718/0 (socket says 172.21.15.19:53718) 2024-03-28T13:17:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.109+0000 7fafdb400640 1 -- 172.21.15.19:0/2970195333 learned_addr learned my addr 172.21.15.19:0/2970195333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.109+0000 7fafdbe00640 1 --2- 172.21.15.19:0/2970195333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafdc154c40 0x7fafdc153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.109+0000 7fafdaa00640 1 --2- 172.21.15.19:0/2970195333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafdc1fc4e0 0x7fafdc159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.109+0000 7fafdb400640 1 -- 172.21.15.19:0/2970195333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafdc154c40 msgr2=0x7fafdc153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:52.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.109+0000 7fafdb400640 1 --2- 172.21.15.19:0/2970195333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafdc154c40 0x7fafdc153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.109+0000 7fafdb400640 1 -- 172.21.15.19:0/2970195333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafdc1fc4e0 msgr2=0x7fafdc159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:52.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.109+0000 7fafdb400640 1 --2- 172.21.15.19:0/2970195333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafdc1fc4e0 0x7fafdc159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.109+0000 7fafdb400640 1 -- 172.21.15.19:0/2970195333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc1fc960 con 0x7fafc0004800 2024-03-28T13:17:52.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.109+0000 7fafdb400640 1 --2- 172.21.15.19:0/2970195333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc0004800 0x7fafdc158ba0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fafcc002130 tx=0x7fafcc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:52.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.110+0000 7fafd3400640 1 -- 172.21.15.19:0/2970195333 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc019070 con 0x7fafc0004800 2024-03-28T13:17:52.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.110+0000 7fafd3400640 1 -- 172.21.15.19:0/2970195333 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafcc004d60 con 0x7fafc0004800 2024-03-28T13:17:52.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.110+0000 7fafe0e00640 1 -- 172.21.15.19:0/2970195333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafdc15ba70 con 0x7fafc0004800 2024-03-28T13:17:52.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.110+0000 7fafd3400640 1 -- 172.21.15.19:0/2970195333 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0072d0 con 0x7fafc0004800 2024-03-28T13:17:52.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.110+0000 7fafe0e00640 1 -- 172.21.15.19:0/2970195333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafdc15c030 con 0x7fafc0004800 2024-03-28T13:17:52.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.111+0000 7fafe0e00640 1 -- 172.21.15.19:0/2970195333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafdc075940 con 0x7fafc0004800 2024-03-28T13:17:52.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.112+0000 7fafd3400640 1 -- 172.21.15.19:0/2970195333 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fafcc01c070 con 0x7fafc0004800 2024-03-28T13:17:52.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.113+0000 7fafd3400640 1 --2- 172.21.15.19:0/2970195333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fafb806c530 0x7fafb806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.113+0000 7fafd3400640 1 -- 172.21.15.19:0/2970195333 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fafcc08b8f0 con 0x7fafc0004800 2024-03-28T13:17:52.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.114+0000 7fafdaa00640 1 --2- 172.21.15.19:0/2970195333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fafb806c530 0x7fafb806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.115+0000 7fafdaa00640 1 --2- 172.21.15.19:0/2970195333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fafb806c530 0x7fafb806e9f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fafc4033000 tx=0x7fafc4009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:52.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.118+0000 7fafd3400640 1 -- 172.21.15.19:0/2970195333 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fafcc059ea0 con 0x7fafc0004800 2024-03-28T13:17:52.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.320+0000 7fafe0e00640 1 -- 172.21.15.19:0/2970195333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fafdc0cf610 con 0x7fafc0004800 2024-03-28T13:17:52.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.321+0000 7fafd3400640 1 -- 172.21.15.19:0/2970195333 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fafcc0597a0 con 0x7fafc0004800 2024-03-28T13:17:52.321 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:17:52.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.324+0000 7fafe0e00640 1 -- 172.21.15.19:0/2970195333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fafb806c530 msgr2=0x7fafb806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:52.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.324+0000 7fafe0e00640 1 --2- 172.21.15.19:0/2970195333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fafb806c530 0x7fafb806e9f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fafc4033000 tx=0x7fafc4009030 comp rx=0 tx=0).stop 2024-03-28T13:17:52.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.324+0000 7fafe0e00640 1 -- 172.21.15.19:0/2970195333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc0004800 msgr2=0x7fafdc158ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:52.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.324+0000 7fafe0e00640 1 --2- 172.21.15.19:0/2970195333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc0004800 0x7fafdc158ba0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fafcc002130 tx=0x7fafcc00b4a0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.324+0000 7fafe0e00640 1 -- 172.21.15.19:0/2970195333 shutdown_connections 2024-03-28T13:17:52.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.324+0000 7fafe0e00640 1 --2- 172.21.15.19:0/2970195333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fafb806c530 0x7fafb806e9f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.324+0000 7fafe0e00640 1 --2- 172.21.15.19:0/2970195333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafdc154c40 0x7fafdc153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.324+0000 7fafe0e00640 1 --2- 172.21.15.19:0/2970195333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafdc1fc4e0 0x7fafdc159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.324+0000 7fafe0e00640 1 --2- 172.21.15.19:0/2970195333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fafc0004800 0x7fafdc158ba0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.324+0000 7fafe0e00640 1 -- 172.21.15.19:0/2970195333 >> 172.21.15.19:0/2970195333 conn(0x7fafdc0b1ba0 msgr2=0x7fafdc15f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:52.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.324+0000 7fafe0e00640 1 -- 172.21.15.19:0/2970195333 shutdown_connections 2024-03-28T13:17:52.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.324+0000 7fafe0e00640 1 -- 172.21.15.19:0/2970195333 wait complete. 2024-03-28T13:17:52.345 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:17:52.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.574+0000 7f97c4e00640 1 Processor -- start 2024-03-28T13:17:52.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.574+0000 7f97c4e00640 1 -- start start 2024-03-28T13:17:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.575+0000 7f97c4e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c00af4b0 0x7f97c0167460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.575+0000 7f97c4e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97c00afab0 con 0x7f97c0153580 2024-03-28T13:17:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.575+0000 7f97c4e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97c015cc80 con 0x7f97c00d0990 2024-03-28T13:17:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.575+0000 7f97c4e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f97c015da00 con 0x7f97c00af4b0 2024-03-28T13:17:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.575+0000 7f97bde00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f97c0153580 0x7f97c00d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34790/0 (socket says 172.21.15.19:34790) 2024-03-28T13:17:52.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.575+0000 7f97bde00640 1 -- 172.21.15.19:0/1784106407 learned_addr learned my addr 172.21.15.19:0/1784106407 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:17:52.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97be800640 1 --2- 172.21.15.19:0/1784106407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c00af4b0 0x7f97c0167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97bca00640 1 -- 172.21.15.19:0/1784106407 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3071353846 0 0) 0x7f97c00afab0 con 0x7f97c0153580 2024-03-28T13:17:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97bca00640 1 -- 172.21.15.19:0/1784106407 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97a0003520 con 0x7f97c0153580 2024-03-28T13:17:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97bca00640 1 -- 172.21.15.19:0/1784106407 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1289351402 0 0) 0x7f97a0003520 con 0x7f97c0153580 2024-03-28T13:17:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97bca00640 1 -- 172.21.15.19:0/1784106407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c00af4b0 msgr2=0x7f97c0167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97bca00640 1 --2- 172.21.15.19:0/1784106407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c00af4b0 0x7f97c0167460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97bca00640 1 -- 172.21.15.19:0/1784106407 >> v1:172.21.15.188:6789/0 conn(0x7f97c00d0990 legacy=0x7f97c00aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97bca00640 1 -- 172.21.15.19:0/1784106407 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c015cf20 con 0x7f97c0153580 2024-03-28T13:17:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97bca00640 1 -- 172.21.15.19:0/1784106407 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f97b0001670 con 0x7f97c0153580 2024-03-28T13:17:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97bca00640 1 -- 172.21.15.19:0/1784106407 >> v1:172.21.15.19:6789/0 conn(0x7f97c0153580 legacy=0x7f97c00d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97bca00640 1 --2- 172.21.15.19:0/1784106407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a00039e0 0x7f97a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97bca00640 1 --2- 172.21.15.19:0/1784106407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0004800 0x7f97a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97bca00640 1 --2- 172.21.15.19:0/1784106407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c00af4b0 0x7f97a0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97bca00640 1 -- 172.21.15.19:0/1784106407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97c015da00 con 0x7f97a0004800 2024-03-28T13:17:52.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97bca00640 1 -- 172.21.15.19:0/1784106407 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97a000baf0 con 0x7f97a00039e0 2024-03-28T13:17:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.576+0000 7f97bca00640 1 -- 172.21.15.19:0/1784106407 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f97a000bcf0 con 0x7f97c00af4b0 2024-03-28T13:17:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.577+0000 7f97bd400640 1 --2- 172.21.15.19:0/1784106407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0004800 0x7f97a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.577+0000 7f97bde00640 1 --2- 172.21.15.19:0/1784106407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a00039e0 0x7f97a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.577+0000 7f97be800640 1 --2- 172.21.15.19:0/1784106407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c00af4b0 0x7f97a0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.577+0000 7f97bd400640 1 -- 172.21.15.19:0/1784106407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c00af4b0 msgr2=0x7f97a0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.577+0000 7f97bd400640 1 --2- 172.21.15.19:0/1784106407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c00af4b0 0x7f97a0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.577+0000 7f97bd400640 1 -- 172.21.15.19:0/1784106407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a00039e0 msgr2=0x7f97a0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.577+0000 7f97bd400640 1 --2- 172.21.15.19:0/1784106407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a00039e0 0x7f97a0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.577+0000 7f97bd400640 1 -- 172.21.15.19:0/1784106407 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a0003520 con 0x7f97a0004800 2024-03-28T13:17:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.578+0000 7f97bd400640 1 --2- 172.21.15.19:0/1784106407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0004800 0x7f97a0004c80 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f97c006f820 tx=0x7f97a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471c43b004cff06a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.579+0000 7f97bca00640 1 -- 172.21.15.19:0/1784106407 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a8001a70 con 0x7f97a0004800 2024-03-28T13:17:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.579+0000 7f97bca00640 1 -- 172.21.15.19:0/1784106407 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97a8003b70 con 0x7f97a0004800 2024-03-28T13:17:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.579+0000 7f97bca00640 1 -- 172.21.15.19:0/1784106407 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a80050d0 con 0x7f97a0004800 2024-03-28T13:17:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.579+0000 7f97c4e00640 1 -- 172.21.15.19:0/1784106407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0004800 msgr2=0x7f97a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.579+0000 7f97c4e00640 1 --2- 172.21.15.19:0/1784106407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0004800 0x7f97a0004c80 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f97c006f820 tx=0x7f97a0005830 comp rx=0 tx=0).stop 2024-03-28T13:17:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.579+0000 7f97be800640 1 -- 172.21.15.19:0/1784106407 reap_dead start 2024-03-28T13:17:52.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.579+0000 7f97c4e00640 1 -- 172.21.15.19:0/1784106407 shutdown_connections 2024-03-28T13:17:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.579+0000 7f97c4e00640 1 -- 172.21.15.19:0/1784106407 >> 172.21.15.19:0/1784106407 conn(0x7f97c006b750 msgr2=0x7f97c006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.580+0000 7f97c4e00640 1 -- 172.21.15.19:0/1784106407 shutdown_connections 2024-03-28T13:17:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.580+0000 7f97c4e00640 1 -- 172.21.15.19:0/1784106407 wait complete. 2024-03-28T13:17:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.581+0000 7f97c4e00640 1 Processor -- start 2024-03-28T13:17:52.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.581+0000 7f97c4e00640 1 -- start start 2024-03-28T13:17:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.582+0000 7f97c4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0004800 0x7f97c00ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.582+0000 7f97c4e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c0148560 0x7f97c00cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.582+0000 7f97c4e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97c014ee10 0x7f97c01478c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.582+0000 7f97c4e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97c0153960 con 0x7f97a0004800 2024-03-28T13:17:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.582+0000 7f97c4e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f97c015d4e0 con 0x7f97c014ee10 2024-03-28T13:17:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.582+0000 7f97c4e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f97c0165450 con 0x7f97c0148560 2024-03-28T13:17:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.583+0000 7f97bde00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0004800 0x7f97c00ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.583+0000 7f97bde00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0004800 0x7f97c00ced60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53744/0 (socket says 172.21.15.19:53744) 2024-03-28T13:17:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.583+0000 7f97bde00640 1 -- 172.21.15.19:0/2485450879 learned_addr learned my addr 172.21.15.19:0/2485450879 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.583+0000 7f97be800640 1 --2- 172.21.15.19:0/2485450879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97c014ee10 0x7f97c01478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.583+0000 7f97bd400640 1 --2- 172.21.15.19:0/2485450879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c0148560 0x7f97c00cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.583+0000 7f97bde00640 1 -- 172.21.15.19:0/2485450879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c0148560 msgr2=0x7f97c00cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:52.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.583+0000 7f97bde00640 1 --2- 172.21.15.19:0/2485450879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c0148560 0x7f97c00cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.583+0000 7f97bde00640 1 -- 172.21.15.19:0/2485450879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97c014ee10 msgr2=0x7f97c01478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:52.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.583+0000 7f97bde00640 1 --2- 172.21.15.19:0/2485450879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97c014ee10 0x7f97c01478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.583+0000 7f97bde00640 1 -- 172.21.15.19:0/2485450879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c014dd70 con 0x7f97a0004800 2024-03-28T13:17:52.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.583+0000 7f97be800640 1 --2- 172.21.15.19:0/2485450879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97c014ee10 0x7f97c01478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:52.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.584+0000 7f97bde00640 1 --2- 172.21.15.19:0/2485450879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0004800 0x7f97c00ced60 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f97b000e7f0 tx=0x7f97b000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:52.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.584+0000 7f97b6000640 1 -- 172.21.15.19:0/2485450879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b000c870 con 0x7f97a0004800 2024-03-28T13:17:52.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.584+0000 7f97b6000640 1 -- 172.21.15.19:0/2485450879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97b0018070 con 0x7f97a0004800 2024-03-28T13:17:52.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.584+0000 7f97b6000640 1 -- 172.21.15.19:0/2485450879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b000cc30 con 0x7f97a0004800 2024-03-28T13:17:52.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.584+0000 7f97c4e00640 1 -- 172.21.15.19:0/2485450879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c014e050 con 0x7f97a0004800 2024-03-28T13:17:52.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.584+0000 7f97c4e00640 1 -- 172.21.15.19:0/2485450879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c014e610 con 0x7f97a0004800 2024-03-28T13:17:52.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.585+0000 7f97c4e00640 1 -- 172.21.15.19:0/2485450879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c00685c0 con 0x7f97a0004800 2024-03-28T13:17:52.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.586+0000 7f97b6000640 1 -- 172.21.15.19:0/2485450879 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f97b001d460 con 0x7f97a0004800 2024-03-28T13:17:52.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.587+0000 7f97b6000640 1 --2- 172.21.15.19:0/2485450879 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f979c06c530 0x7f979c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:52.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.587+0000 7f97b6000640 1 -- 172.21.15.19:0/2485450879 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f97b008b600 con 0x7f97a0004800 2024-03-28T13:17:52.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.588+0000 7f97bd400640 1 --2- 172.21.15.19:0/2485450879 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f979c06c530 0x7f979c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:52.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.589+0000 7f97bd400640 1 --2- 172.21.15.19:0/2485450879 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f979c06c530 0x7f979c06e9f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f97a8034000 tx=0x7f97a8009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:52.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.592+0000 7f97b6000640 1 -- 172.21.15.19:0/2485450879 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f97b0059bb0 con 0x7f97a0004800 2024-03-28T13:17:52.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.793+0000 7f97c4e00640 1 -- 172.21.15.19:0/2485450879 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f97c00aeff0 con 0x7f97a0004800 2024-03-28T13:17:52.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.794+0000 7f97b6000640 1 -- 172.21.15.19:0/2485450879 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f97b00594b0 con 0x7f97a0004800 2024-03-28T13:17:52.794 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:17:52.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.796+0000 7f97c4e00640 1 -- 172.21.15.19:0/2485450879 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f979c06c530 msgr2=0x7f979c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.796+0000 7f97c4e00640 1 --2- 172.21.15.19:0/2485450879 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f979c06c530 0x7f979c06e9f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f97a8034000 tx=0x7f97a8009030 comp rx=0 tx=0).stop 2024-03-28T13:17:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.797+0000 7f97c4e00640 1 -- 172.21.15.19:0/2485450879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0004800 msgr2=0x7f97c00ced60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.797+0000 7f97c4e00640 1 --2- 172.21.15.19:0/2485450879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0004800 0x7f97c00ced60 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f97b000e7f0 tx=0x7f97b000ecb0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.797+0000 7f97c4e00640 1 -- 172.21.15.19:0/2485450879 shutdown_connections 2024-03-28T13:17:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.797+0000 7f97c4e00640 1 --2- 172.21.15.19:0/2485450879 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f979c06c530 0x7f979c06e9f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.797+0000 7f97c4e00640 1 --2- 172.21.15.19:0/2485450879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97c014ee10 0x7f97c01478c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.797+0000 7f97c4e00640 1 --2- 172.21.15.19:0/2485450879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97c0148560 0x7f97c00cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.797+0000 7f97c4e00640 1 --2- 172.21.15.19:0/2485450879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97a0004800 0x7f97c00ced60 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.797+0000 7f97c4e00640 1 -- 172.21.15.19:0/2485450879 >> 172.21.15.19:0/2485450879 conn(0x7f97c006b750 msgr2=0x7f97c00b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.797+0000 7f97c4e00640 1 -- 172.21.15.19:0/2485450879 shutdown_connections 2024-03-28T13:17:52.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:52.797+0000 7f97c4e00640 1 -- 172.21.15.19:0/2485450879 wait complete. 2024-03-28T13:17:52.818 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:17:53.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.053+0000 7f47d6e00640 1 Processor -- start 2024-03-28T13:17:53.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.053+0000 7f47d6e00640 1 -- start start 2024-03-28T13:17:53.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.054+0000 7f47d6e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f47d00cf050 0x7f47d015f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:53.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.054+0000 7f47d6e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f47d00d2110 0x7f47d00d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:53.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.054+0000 7f47d6e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47d00cf6c0 con 0x7f47d00d0e70 2024-03-28T13:17:53.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.054+0000 7f47d6e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f47d015fc20 con 0x7f47d00cf050 2024-03-28T13:17:53.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.054+0000 7f47d6e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f47d00cf9a0 con 0x7f47d00d2110 2024-03-28T13:17:53.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.055+0000 7f47cf400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f47d00cf050 0x7f47d015f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:53.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.055+0000 7f47cfe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f47d00d0e70 0x7f47d00d1250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34804/0 (socket says 172.21.15.19:34804) 2024-03-28T13:17:53.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.055+0000 7f47cfe00640 1 -- 172.21.15.19:0/3151715110 learned_addr learned my addr 172.21.15.19:0/3151715110 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:17:53.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.055+0000 7f47d4e00640 1 --2- 172.21.15.19:0/3151715110 >> v2:172.21.15.188:3300/0 conn(0x7f47d00d2110 0x7f47d00d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:53.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.055+0000 7f47cf400640 1 -- 172.21.15.19:0/3151715110 >> v2:172.21.15.188:3300/0 conn(0x7f47d00d2110 msgr2=0x7f47d00d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.055+0000 7f47cf400640 1 --2- 172.21.15.19:0/3151715110 >> v2:172.21.15.188:3300/0 conn(0x7f47d00d2110 0x7f47d00d25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.055+0000 7f47cf400640 1 -- 172.21.15.19:0/3151715110 >> v1:172.21.15.19:6789/0 conn(0x7f47d00d0e70 legacy=0x7f47d00d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.055+0000 7f47cf400640 1 -- 172.21.15.19:0/3151715110 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d00cfc80 con 0x7f47d00cf050 2024-03-28T13:17:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.055+0000 7f47cea00640 1 -- 172.21.15.19:0/3151715110 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4165857761 0 0) 0x7f47d00cf6c0 con 0x7f47d00d0e70 2024-03-28T13:17:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.055+0000 7f47d4e00640 1 --2- 172.21.15.19:0/3151715110 >> v2:172.21.15.188:3300/0 conn(0x7f47d00d2110 0x7f47d00d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.056+0000 7f47cf400640 1 --2- 172.21.15.19:0/3151715110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47d00cf050 0x7f47d015f6e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f47d00d4380 tx=0x7f47b802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6fa4526d5c1838 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:53.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.057+0000 7f47cea00640 1 -- 172.21.15.19:0/3151715110 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8035020 con 0x7f47d00cf050 2024-03-28T13:17:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.057+0000 7f47cea00640 1 -- 172.21.15.19:0/3151715110 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b8031830 con 0x7f47d00cf050 2024-03-28T13:17:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.057+0000 7f47cea00640 1 -- 172.21.15.19:0/3151715110 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8031c00 con 0x7f47d00cf050 2024-03-28T13:17:53.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.057+0000 7f47d6e00640 1 -- 172.21.15.19:0/3151715110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47d00cf050 msgr2=0x7f47d015f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.057+0000 7f47d6e00640 1 --2- 172.21.15.19:0/3151715110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47d00cf050 0x7f47d015f6e0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f47d00d4380 tx=0x7f47b802b8d0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.058+0000 7f47d6e00640 1 -- 172.21.15.19:0/3151715110 shutdown_connections 2024-03-28T13:17:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.058+0000 7f47d6e00640 1 --2- 172.21.15.19:0/3151715110 >> v2:172.21.15.188:3300/0 conn(0x7f47d00d2110 0x7f47d00d25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.058+0000 7f47d6e00640 1 --2- 172.21.15.19:0/3151715110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47d00cf050 0x7f47d015f6e0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.058+0000 7f47d6e00640 1 -- 172.21.15.19:0/3151715110 >> 172.21.15.19:0/3151715110 conn(0x7f47d008bfb0 msgr2=0x7f47d00d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.058+0000 7f47d6e00640 1 -- 172.21.15.19:0/3151715110 shutdown_connections 2024-03-28T13:17:53.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.058+0000 7f47d6e00640 1 -- 172.21.15.19:0/3151715110 wait complete. 2024-03-28T13:17:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.059+0000 7f47d6e00640 1 Processor -- start 2024-03-28T13:17:53.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.060+0000 7f47d6e00640 1 -- start start 2024-03-28T13:17:53.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.060+0000 7f47d6e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47d00cf050 0x7f47d01554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:53.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.061+0000 7f47d6e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47d00d0e70 0x7f47d00b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:53.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.061+0000 7f47d6e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47d00d2110 0x7f47d01f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:53.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.061+0000 7f47d6e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f47d015e910 con 0x7f47d00d2110 2024-03-28T13:17:53.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.061+0000 7f47d6e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00d0240 con 0x7f47d00d0e70 2024-03-28T13:17:53.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.061+0000 7f47d6e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f47d015ea90 con 0x7f47d00cf050 2024-03-28T13:17:53.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.061+0000 7f47d4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47d00d2110 0x7f47d01f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:53.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.061+0000 7f47d4e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47d00d2110 0x7f47d01f7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53774/0 (socket says 172.21.15.19:53774) 2024-03-28T13:17:53.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.061+0000 7f47d4e00640 1 -- 172.21.15.19:0/1390053176 learned_addr learned my addr 172.21.15.19:0/1390053176 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:53.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.061+0000 7f47cf400640 1 --2- 172.21.15.19:0/1390053176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47d00d0e70 0x7f47d00b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:53.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.061+0000 7f47cfe00640 1 --2- 172.21.15.19:0/1390053176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47d00cf050 0x7f47d01554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:53.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.062+0000 7f47d4e00640 1 -- 172.21.15.19:0/1390053176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47d00cf050 msgr2=0x7f47d01554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:53.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.062+0000 7f47d4e00640 1 --2- 172.21.15.19:0/1390053176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47d00cf050 0x7f47d01554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.062+0000 7f47d4e00640 1 -- 172.21.15.19:0/1390053176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47d00d0e70 msgr2=0x7f47d00b97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:53.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.062+0000 7f47d4e00640 1 --2- 172.21.15.19:0/1390053176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47d00d0e70 0x7f47d00b97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.062+0000 7f47d4e00640 1 -- 172.21.15.19:0/1390053176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d00c89e0 con 0x7f47d00d2110 2024-03-28T13:17:53.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.062+0000 7f47d4e00640 1 --2- 172.21.15.19:0/1390053176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47d00d2110 0x7f47d01f7b00 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f47c800cef0 tx=0x7f47c800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:53.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.062+0000 7f47c7e00640 1 -- 172.21.15.19:0/1390053176 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c801a070 con 0x7f47d00d2110 2024-03-28T13:17:53.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.062+0000 7f47c7e00640 1 -- 172.21.15.19:0/1390053176 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c800d860 con 0x7f47d00d2110 2024-03-28T13:17:53.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.062+0000 7f47c7e00640 1 -- 172.21.15.19:0/1390053176 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c800dc50 con 0x7f47d00d2110 2024-03-28T13:17:53.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.062+0000 7f47d6e00640 1 -- 172.21.15.19:0/1390053176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d0155a00 con 0x7f47d00d2110 2024-03-28T13:17:53.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.062+0000 7f47d6e00640 1 -- 172.21.15.19:0/1390053176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d00c8f30 con 0x7f47d00d2110 2024-03-28T13:17:53.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.063+0000 7f47d6e00640 1 -- 172.21.15.19:0/1390053176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d0068cd0 con 0x7f47d00d2110 2024-03-28T13:17:53.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.064+0000 7f47c7e00640 1 -- 172.21.15.19:0/1390053176 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f47c801d070 con 0x7f47d00d2110 2024-03-28T13:17:53.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.065+0000 7f47c7e00640 1 --2- 172.21.15.19:0/1390053176 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f47ac06c530 0x7f47ac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:53.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.065+0000 7f47c7e00640 1 -- 172.21.15.19:0/1390053176 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f47c808c0a0 con 0x7f47d00d2110 2024-03-28T13:17:53.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.066+0000 7f47cfe00640 1 --2- 172.21.15.19:0/1390053176 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f47ac06c530 0x7f47ac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:53.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.068+0000 7f47cfe00640 1 --2- 172.21.15.19:0/1390053176 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f47ac06c530 0x7f47ac06e9f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f47c0000f60 tx=0x7f47c0007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:53.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.071+0000 7f47c7e00640 1 -- 172.21.15.19:0/1390053176 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f47c805a680 con 0x7f47d00d2110 2024-03-28T13:17:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.274+0000 7f47d6e00640 1 -- 172.21.15.19:0/1390053176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f47d00d1b80 con 0x7f47d00d2110 2024-03-28T13:17:53.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.275+0000 7f47c7e00640 1 -- 172.21.15.19:0/1390053176 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f47c8059f80 con 0x7f47d00d2110 2024-03-28T13:17:53.275 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:17:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.277+0000 7f47d6e00640 1 -- 172.21.15.19:0/1390053176 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f47ac06c530 msgr2=0x7f47ac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.277+0000 7f47d6e00640 1 --2- 172.21.15.19:0/1390053176 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f47ac06c530 0x7f47ac06e9f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f47c0000f60 tx=0x7f47c0007b50 comp rx=0 tx=0).stop 2024-03-28T13:17:53.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.277+0000 7f47d6e00640 1 -- 172.21.15.19:0/1390053176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47d00d2110 msgr2=0x7f47d01f7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.277+0000 7f47d6e00640 1 --2- 172.21.15.19:0/1390053176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47d00d2110 0x7f47d01f7b00 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f47c800cef0 tx=0x7f47c800a710 comp rx=0 tx=0).stop 2024-03-28T13:17:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.277+0000 7f47d6e00640 1 -- 172.21.15.19:0/1390053176 shutdown_connections 2024-03-28T13:17:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.277+0000 7f47d6e00640 1 --2- 172.21.15.19:0/1390053176 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f47ac06c530 0x7f47ac06e9f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.277+0000 7f47d6e00640 1 --2- 172.21.15.19:0/1390053176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47d00d2110 0x7f47d01f7b00 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.277+0000 7f47d6e00640 1 --2- 172.21.15.19:0/1390053176 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47d00d0e70 0x7f47d00b97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.277+0000 7f47d6e00640 1 --2- 172.21.15.19:0/1390053176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47d00cf050 0x7f47d01554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.277+0000 7f47d6e00640 1 -- 172.21.15.19:0/1390053176 >> 172.21.15.19:0/1390053176 conn(0x7f47d008bfb0 msgr2=0x7f47d0160290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.278+0000 7f47d6e00640 1 -- 172.21.15.19:0/1390053176 shutdown_connections 2024-03-28T13:17:53.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.278+0000 7f47d6e00640 1 -- 172.21.15.19:0/1390053176 wait complete. 2024-03-28T13:17:53.298 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:17:53.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.516+0000 7fe2a6800640 1 Processor -- start 2024-03-28T13:17:53.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.517+0000 7fe2a6800640 1 -- start start 2024-03-28T13:17:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.518+0000 7fe2a6800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a015e4a0 0x7fe2a00b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:53.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.519+0000 7fe2a6800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a015e880 con 0x7fe2a00cf700 2024-03-28T13:17:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.519+0000 7fe2a6800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2a00d28d0 con 0x7fe2a00d2310 2024-03-28T13:17:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.519+0000 7fe2a6800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a00d34e0 con 0x7fe2a015e4a0 2024-03-28T13:17:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.519+0000 7fe2a5e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe2a00cf700 0x7fe2a00b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34814/0 (socket says 172.21.15.19:34814) 2024-03-28T13:17:53.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.519+0000 7fe2a5e00640 1 -- 172.21.15.19:0/1114543944 learned_addr learned my addr 172.21.15.19:0/1114543944 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:17:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.519+0000 7fe2a5400640 1 --2- 172.21.15.19:0/1114543944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a015e4a0 0x7fe2a00b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe29fe00640 1 -- 172.21.15.19:0/1114543944 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 973306392 0 0) 0x7fe2a015e880 con 0x7fe2a00cf700 2024-03-28T13:17:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe29fe00640 1 -- 172.21.15.19:0/1114543944 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe288003520 con 0x7fe2a00cf700 2024-03-28T13:17:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe29fe00640 1 -- 172.21.15.19:0/1114543944 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3469452446 0 0) 0x7fe288003520 con 0x7fe2a00cf700 2024-03-28T13:17:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe29fe00640 1 -- 172.21.15.19:0/1114543944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a015e4a0 msgr2=0x7fe2a00b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe29fe00640 1 --2- 172.21.15.19:0/1114543944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a015e4a0 0x7fe2a00b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe29fe00640 1 -- 172.21.15.19:0/1114543944 >> v1:172.21.15.188:6789/0 conn(0x7fe2a00d2310 legacy=0x7fe2a00ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe29fe00640 1 -- 172.21.15.19:0/1114543944 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a00d2bb0 con 0x7fe2a00cf700 2024-03-28T13:17:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe29fe00640 1 -- 172.21.15.19:0/1114543944 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fe2980019f0 con 0x7fe2a00cf700 2024-03-28T13:17:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe29fe00640 1 -- 172.21.15.19:0/1114543944 >> v1:172.21.15.19:6789/0 conn(0x7fe2a00cf700 legacy=0x7fe2a00b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe29fe00640 1 --2- 172.21.15.19:0/1114543944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2880039e0 0x7fe288003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe29fe00640 1 --2- 172.21.15.19:0/1114543944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe288004800 0x7fe288004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:53.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe29fe00640 1 --2- 172.21.15.19:0/1114543944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a015e4a0 0x7fe2880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe29fe00640 1 -- 172.21.15.19:0/1114543944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a00d34e0 con 0x7fe288004800 2024-03-28T13:17:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe29fe00640 1 -- 172.21.15.19:0/1114543944 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe28800ba70 con 0x7fe2880039e0 2024-03-28T13:17:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe29fe00640 1 -- 172.21.15.19:0/1114543944 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe28800bc70 con 0x7fe2a015e4a0 2024-03-28T13:17:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.520+0000 7fe2a4a00640 1 --2- 172.21.15.19:0/1114543944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe288004800 0x7fe288004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.521+0000 7fe2a5400640 1 --2- 172.21.15.19:0/1114543944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2880039e0 0x7fe288003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.521+0000 7fe2a4a00640 1 -- 172.21.15.19:0/1114543944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a015e4a0 msgr2=0x7fe2880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.521+0000 7fe2a4a00640 1 --2- 172.21.15.19:0/1114543944 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a015e4a0 0x7fe2880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.521+0000 7fe2a4a00640 1 -- 172.21.15.19:0/1114543944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2880039e0 msgr2=0x7fe288003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.521+0000 7fe2a4a00640 1 --2- 172.21.15.19:0/1114543944 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2880039e0 0x7fe288003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.521+0000 7fe2a4a00640 1 -- 172.21.15.19:0/1114543944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe288003520 con 0x7fe288004800 2024-03-28T13:17:53.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.522+0000 7fe2a4a00640 1 --2- 172.21.15.19:0/1114543944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe288004800 0x7fe288004c80 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fe28c001720 tx=0x7fe288005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99aef3e9f365a8d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:53.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.522+0000 7fe29fe00640 1 -- 172.21.15.19:0/1114543944 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c001370 con 0x7fe288004800 2024-03-28T13:17:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.522+0000 7fe29fe00640 1 -- 172.21.15.19:0/1114543944 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe28c0052d0 con 0x7fe288004800 2024-03-28T13:17:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.522+0000 7fe29fe00640 1 -- 172.21.15.19:0/1114543944 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c003830 con 0x7fe288004800 2024-03-28T13:17:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.523+0000 7fe2a6800640 1 -- 172.21.15.19:0/1114543944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe288004800 msgr2=0x7fe288004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.523+0000 7fe2a6800640 1 --2- 172.21.15.19:0/1114543944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe288004800 0x7fe288004c80 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fe28c001720 tx=0x7fe288005830 comp rx=0 tx=0).stop 2024-03-28T13:17:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.523+0000 7fe2a5e00640 1 -- 172.21.15.19:0/1114543944 reap_dead start 2024-03-28T13:17:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.523+0000 7fe2a6800640 1 -- 172.21.15.19:0/1114543944 shutdown_connections 2024-03-28T13:17:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.523+0000 7fe2a6800640 1 -- 172.21.15.19:0/1114543944 >> 172.21.15.19:0/1114543944 conn(0x7fe2a00676d0 msgr2=0x7fe2a0067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.523+0000 7fe2a6800640 1 -- 172.21.15.19:0/1114543944 shutdown_connections 2024-03-28T13:17:53.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.523+0000 7fe2a6800640 1 -- 172.21.15.19:0/1114543944 wait complete. 2024-03-28T13:17:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.525+0000 7fe2a6800640 1 Processor -- start 2024-03-28T13:17:53.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.525+0000 7fe2a6800640 1 -- start start 2024-03-28T13:17:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.526+0000 7fe2a6800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe288004800 0x7fe2a014f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.526+0000 7fe2a6800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a0155c20 0x7fe2a0161420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.526+0000 7fe2a6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2a0151c90 0x7fe2a00b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.526+0000 7fe2a6800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a00d3970 con 0x7fe2a0151c90 2024-03-28T13:17:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.526+0000 7fe2a6800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a00d3170 con 0x7fe288004800 2024-03-28T13:17:53.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.526+0000 7fe2a6800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a00b9c60 con 0x7fe2a0155c20 2024-03-28T13:17:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.526+0000 7fe2a5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2a0151c90 0x7fe2a00b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.526+0000 7fe2a5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2a0151c90 0x7fe2a00b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53796/0 (socket says 172.21.15.19:53796) 2024-03-28T13:17:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.526+0000 7fe2a5e00640 1 -- 172.21.15.19:0/2770195455 learned_addr learned my addr 172.21.15.19:0/2770195455 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.526+0000 7fe2a5400640 1 --2- 172.21.15.19:0/2770195455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe288004800 0x7fe2a014f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.526+0000 7fe2a4a00640 1 --2- 172.21.15.19:0/2770195455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a0155c20 0x7fe2a0161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.527+0000 7fe2a5e00640 1 -- 172.21.15.19:0/2770195455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a0155c20 msgr2=0x7fe2a0161420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.527+0000 7fe2a5e00640 1 --2- 172.21.15.19:0/2770195455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a0155c20 0x7fe2a0161420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.527+0000 7fe2a5e00640 1 -- 172.21.15.19:0/2770195455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe288004800 msgr2=0x7fe2a014f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.527+0000 7fe2a5e00640 1 --2- 172.21.15.19:0/2770195455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe288004800 0x7fe2a014f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.527+0000 7fe2a5e00640 1 -- 172.21.15.19:0/2770195455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a0156150 con 0x7fe2a0151c90 2024-03-28T13:17:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.527+0000 7fe2a5400640 1 --2- 172.21.15.19:0/2770195455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe288004800 0x7fe2a014f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.527+0000 7fe2a5e00640 1 --2- 172.21.15.19:0/2770195455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2a0151c90 0x7fe2a00b3d30 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00cfb20 tx=0x7fe29800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.527+0000 7fe29d600640 1 -- 172.21.15.19:0/2770195455 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29800a5b0 con 0x7fe2a0151c90 2024-03-28T13:17:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.527+0000 7fe29d600640 1 -- 172.21.15.19:0/2770195455 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe29800a750 con 0x7fe2a0151c90 2024-03-28T13:17:53.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.527+0000 7fe29d600640 1 -- 172.21.15.19:0/2770195455 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29800ab00 con 0x7fe2a0151c90 2024-03-28T13:17:53.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.527+0000 7fe2a6800640 1 -- 172.21.15.19:0/2770195455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a0156430 con 0x7fe2a0151c90 2024-03-28T13:17:53.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.528+0000 7fe2a6800640 1 -- 172.21.15.19:0/2770195455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a01569f0 con 0x7fe2a0151c90 2024-03-28T13:17:53.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.529+0000 7fe2a6800640 1 -- 172.21.15.19:0/2770195455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a0075940 con 0x7fe2a0151c90 2024-03-28T13:17:53.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.530+0000 7fe29d600640 1 -- 172.21.15.19:0/2770195455 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe298011070 con 0x7fe2a0151c90 2024-03-28T13:17:53.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.531+0000 7fe29d600640 1 --2- 172.21.15.19:0/2770195455 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe28006c560 0x7fe28006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:53.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.531+0000 7fe29d600640 1 -- 172.21.15.19:0/2770195455 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fe29808ae30 con 0x7fe2a0151c90 2024-03-28T13:17:53.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.532+0000 7fe2a5400640 1 --2- 172.21.15.19:0/2770195455 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe28006c560 0x7fe28006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.533+0000 7fe2a5400640 1 --2- 172.21.15.19:0/2770195455 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe28006c560 0x7fe28006ea20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0157f20 tx=0x7fe294008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:53.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.536+0000 7fe29d600640 1 -- 172.21.15.19:0/2770195455 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe298019060 con 0x7fe2a0151c90 2024-03-28T13:17:53.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.753+0000 7fe2a6800640 1 -- 172.21.15.19:0/2770195455 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe2a00b77d0 con 0x7fe2a0151c90 2024-03-28T13:17:53.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.754+0000 7fe29d600640 1 -- 172.21.15.19:0/2770195455 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v93) ==== 80+0+124 (secure 0 0 0) 0x7fe298059350 con 0x7fe2a0151c90 2024-03-28T13:17:53.754 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.188:6836/2260087679 2024-03-29T13:16:39.492982+0000 2024-03-28T13:17:53.754 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.188:6834/2260087679 2024-03-29T13:16:39.492982+0000 2024-03-28T13:17:53.754 INFO:teuthology.orchestra.run.smithi019.stderr:listed 2 entries 2024-03-28T13:17:53.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.757+0000 7fe2a6800640 1 -- 172.21.15.19:0/2770195455 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe28006c560 msgr2=0x7fe28006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:53.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.757+0000 7fe2a6800640 1 --2- 172.21.15.19:0/2770195455 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe28006c560 0x7fe28006ea20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fe2a0157f20 tx=0x7fe294008040 comp rx=0 tx=0).stop 2024-03-28T13:17:53.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.757+0000 7fe2a6800640 1 -- 172.21.15.19:0/2770195455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2a0151c90 msgr2=0x7fe2a00b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:53.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.757+0000 7fe2a6800640 1 --2- 172.21.15.19:0/2770195455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2a0151c90 0x7fe2a00b3d30 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fe2a00cfb20 tx=0x7fe29800ccd0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.757+0000 7fe2a6800640 1 -- 172.21.15.19:0/2770195455 shutdown_connections 2024-03-28T13:17:53.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.757+0000 7fe2a6800640 1 --2- 172.21.15.19:0/2770195455 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe28006c560 0x7fe28006ea20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.757+0000 7fe2a6800640 1 --2- 172.21.15.19:0/2770195455 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2a0151c90 0x7fe2a00b3d30 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.757+0000 7fe2a6800640 1 --2- 172.21.15.19:0/2770195455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2a0155c20 0x7fe2a0161420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.757+0000 7fe2a6800640 1 --2- 172.21.15.19:0/2770195455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe288004800 0x7fe2a014f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:53.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.757+0000 7fe2a6800640 1 -- 172.21.15.19:0/2770195455 >> 172.21.15.19:0/2770195455 conn(0x7fe2a00676d0 msgr2=0x7fe2a015d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:53.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.757+0000 7fe2a6800640 1 -- 172.21.15.19:0/2770195455 shutdown_connections 2024-03-28T13:17:53.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:53.757+0000 7fe2a6800640 1 -- 172.21.15.19:0/2770195455 wait complete. 2024-03-28T13:17:53.777 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:17:53.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:17:53.777 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T13:17:53.886 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:17:53.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:17:53.887 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:17:53.942 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:17:54.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.151+0000 7feba7e00640 1 Processor -- start 2024-03-28T13:17:54.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.151+0000 7feba7e00640 1 -- start start 2024-03-28T13:17:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.153+0000 7feba7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba80cf700 0x7feba8168fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.153+0000 7feba7e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba815e880 con 0x7feba815e4a0 2024-03-28T13:17:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.153+0000 7feba7e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba80d28d0 con 0x7feba80d2310 2024-03-28T13:17:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.153+0000 7feba7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feba80d34e0 con 0x7feba80cf700 2024-03-28T13:17:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.153+0000 7feba6a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7feba815e4a0 0x7feba80b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34830/0 (socket says 172.21.15.19:34830) 2024-03-28T13:17:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.153+0000 7feba6a00640 1 -- 172.21.15.19:0/3095256878 learned_addr learned my addr 172.21.15.19:0/3095256878 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:17:54.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.154+0000 7feba7400640 1 --2- 172.21.15.19:0/3095256878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba80cf700 0x7feba8168fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:54.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.154+0000 7feba6000640 1 -- 172.21.15.19:0/3095256878 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3632866740 0 0) 0x7feba815e880 con 0x7feba815e4a0 2024-03-28T13:17:54.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.154+0000 7feba6000640 1 -- 172.21.15.19:0/3095256878 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb8c003520 con 0x7feba815e4a0 2024-03-28T13:17:54.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.154+0000 7feba6000640 1 -- 172.21.15.19:0/3095256878 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3691333135 0 0) 0x7feb8c003520 con 0x7feba815e4a0 2024-03-28T13:17:54.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.154+0000 7feba6000640 1 -- 172.21.15.19:0/3095256878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba80cf700 msgr2=0x7feba8168fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:54.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.154+0000 7feba6000640 1 --2- 172.21.15.19:0/3095256878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba80cf700 0x7feba8168fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:54.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.154+0000 7feba6000640 1 -- 172.21.15.19:0/3095256878 >> v1:172.21.15.188:6789/0 conn(0x7feba80d2310 legacy=0x7feba80ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:54.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.154+0000 7feba6000640 1 -- 172.21.15.19:0/3095256878 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba80d2bb0 con 0x7feba815e4a0 2024-03-28T13:17:54.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.154+0000 7feba6000640 1 -- 172.21.15.19:0/3095256878 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7feb98001670 con 0x7feba815e4a0 2024-03-28T13:17:54.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.154+0000 7feba7400640 1 --2- 172.21.15.19:0/3095256878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba80cf700 0x7feba8168fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:17:54.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.154+0000 7feba6000640 1 -- 172.21.15.19:0/3095256878 >> v1:172.21.15.19:6789/0 conn(0x7feba815e4a0 legacy=0x7feba80b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:54.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.154+0000 7feba6000640 1 --2- 172.21.15.19:0/3095256878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c0039e0 0x7feb8c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:54.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.155+0000 7feba6000640 1 --2- 172.21.15.19:0/3095256878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb8c004800 0x7feb8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:54.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.155+0000 7feba6000640 1 --2- 172.21.15.19:0/3095256878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba80cf700 0x7feb8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:54.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.155+0000 7feba6000640 1 -- 172.21.15.19:0/3095256878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feba80d34e0 con 0x7feb8c0039e0 2024-03-28T13:17:54.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.155+0000 7feba6000640 1 -- 172.21.15.19:0/3095256878 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c005510 con 0x7feb8c004800 2024-03-28T13:17:54.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.155+0000 7feba6000640 1 -- 172.21.15.19:0/3095256878 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feb8c00bf00 con 0x7feba80cf700 2024-03-28T13:17:54.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.155+0000 7feba6a00640 1 --2- 172.21.15.19:0/3095256878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c0039e0 0x7feb8c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:54.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.155+0000 7feb9fe00640 1 --2- 172.21.15.19:0/3095256878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb8c004800 0x7feb8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:54.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.155+0000 7feba6a00640 1 -- 172.21.15.19:0/3095256878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba80cf700 msgr2=0x7feb8c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:54.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.155+0000 7feba6a00640 1 --2- 172.21.15.19:0/3095256878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feba80cf700 0x7feb8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:54.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.155+0000 7feba6a00640 1 -- 172.21.15.19:0/3095256878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb8c004800 msgr2=0x7feb8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:54.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.155+0000 7feba6a00640 1 --2- 172.21.15.19:0/3095256878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb8c004800 0x7feb8c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:54.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.155+0000 7feba6a00640 1 -- 172.21.15.19:0/3095256878 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8c003520 con 0x7feb8c0039e0 2024-03-28T13:17:54.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.155+0000 7feb9fe00640 1 --2- 172.21.15.19:0/3095256878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb8c004800 0x7feb8c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:54.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.156+0000 7feba6a00640 1 --2- 172.21.15.19:0/3095256878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c0039e0 0x7feb8c003e00 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7feb9800c240 tx=0x7feb8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c1dbedd396d157 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:54.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.157+0000 7feba6000640 1 -- 172.21.15.19:0/3095256878 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98007cf0 con 0x7feb8c0039e0 2024-03-28T13:17:54.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.157+0000 7feba6000640 1 -- 172.21.15.19:0/3095256878 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb98030070 con 0x7feb8c0039e0 2024-03-28T13:17:54.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.157+0000 7feba6000640 1 -- 172.21.15.19:0/3095256878 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98005d30 con 0x7feb8c0039e0 2024-03-28T13:17:54.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.157+0000 7feba7e00640 1 -- 172.21.15.19:0/3095256878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c0039e0 msgr2=0x7feb8c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:54.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.157+0000 7feba7e00640 1 --2- 172.21.15.19:0/3095256878 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c0039e0 0x7feb8c003e00 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7feb9800c240 tx=0x7feb8c005830 comp rx=0 tx=0).stop 2024-03-28T13:17:54.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.157+0000 7feba7400640 1 -- 172.21.15.19:0/3095256878 reap_dead start 2024-03-28T13:17:54.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.157+0000 7feba7e00640 1 -- 172.21.15.19:0/3095256878 shutdown_connections 2024-03-28T13:17:54.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.157+0000 7feba7e00640 1 -- 172.21.15.19:0/3095256878 >> 172.21.15.19:0/3095256878 conn(0x7feba80676d0 msgr2=0x7feba8067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:54.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.157+0000 7feba7e00640 1 -- 172.21.15.19:0/3095256878 shutdown_connections 2024-03-28T13:17:54.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.157+0000 7feba7e00640 1 -- 172.21.15.19:0/3095256878 wait complete. 2024-03-28T13:17:54.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.159+0000 7feba7e00640 1 Processor -- start 2024-03-28T13:17:54.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.159+0000 7feba7e00640 1 -- start start 2024-03-28T13:17:54.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.160+0000 7feba7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb8c0039e0 0x7feba814ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:54.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.160+0000 7feba7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feba81580d0 0x7feba81545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:54.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.160+0000 7feba7e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feba8154ae0 0x7feba8160a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:54.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.160+0000 7feba7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feba80d3970 con 0x7feba81580d0 2024-03-28T13:17:54.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.160+0000 7feba7e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feba80d3170 con 0x7feba8154ae0 2024-03-28T13:17:54.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.160+0000 7feba7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feba80b83f0 con 0x7feb8c0039e0 2024-03-28T13:17:54.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.161+0000 7feb9fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feba81580d0 0x7feba81545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:54.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.161+0000 7feb9fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feba81580d0 0x7feba81545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53806/0 (socket says 172.21.15.19:53806) 2024-03-28T13:17:54.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.161+0000 7feb9fe00640 1 -- 172.21.15.19:0/2241867112 learned_addr learned my addr 172.21.15.19:0/2241867112 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:54.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.161+0000 7feba7400640 1 --2- 172.21.15.19:0/2241867112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feba8154ae0 0x7feba8160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:54.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.161+0000 7feba6a00640 1 --2- 172.21.15.19:0/2241867112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb8c0039e0 0x7feba814ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:54.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.161+0000 7feb9fe00640 1 -- 172.21.15.19:0/2241867112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb8c0039e0 msgr2=0x7feba814ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:54.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.161+0000 7feb9fe00640 1 --2- 172.21.15.19:0/2241867112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb8c0039e0 0x7feba814ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:54.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.161+0000 7feb9fe00640 1 -- 172.21.15.19:0/2241867112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feba8154ae0 msgr2=0x7feba8160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:54.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.161+0000 7feb9fe00640 1 --2- 172.21.15.19:0/2241867112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feba8154ae0 0x7feba8160a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:54.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.161+0000 7feb9fe00640 1 -- 172.21.15.19:0/2241867112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba814f370 con 0x7feba81580d0 2024-03-28T13:17:54.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.162+0000 7feb9fe00640 1 --2- 172.21.15.19:0/2241867112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feba81580d0 0x7feba81545a0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7feb9000d7a0 tx=0x7feb9000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:54.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.162+0000 7feb9ea00640 1 -- 172.21.15.19:0/2241867112 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90005070 con 0x7feba81580d0 2024-03-28T13:17:54.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.162+0000 7feb9ea00640 1 -- 172.21.15.19:0/2241867112 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb90005210 con 0x7feba81580d0 2024-03-28T13:17:54.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.162+0000 7feb9ea00640 1 -- 172.21.15.19:0/2241867112 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb90007030 con 0x7feba81580d0 2024-03-28T13:17:54.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.162+0000 7feba7e00640 1 -- 172.21.15.19:0/2241867112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba814f650 con 0x7feba81580d0 2024-03-28T13:17:54.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.162+0000 7feba7e00640 1 -- 172.21.15.19:0/2241867112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba8155010 con 0x7feba81580d0 2024-03-28T13:17:54.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.163+0000 7feba7e00640 1 -- 172.21.15.19:0/2241867112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba8075940 con 0x7feba81580d0 2024-03-28T13:17:54.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.164+0000 7feb9ea00640 1 -- 172.21.15.19:0/2241867112 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7feb9001a020 con 0x7feba81580d0 2024-03-28T13:17:54.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.165+0000 7feb9ea00640 1 --2- 172.21.15.19:0/2241867112 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb8406c530 0x7feb8406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:54.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.165+0000 7feb9ea00640 1 -- 172.21.15.19:0/2241867112 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7feb9008aff0 con 0x7feba81580d0 2024-03-28T13:17:54.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.166+0000 7feba6a00640 1 --2- 172.21.15.19:0/2241867112 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb8406c530 0x7feb8406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:54.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.167+0000 7feba6a00640 1 --2- 172.21.15.19:0/2241867112 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb8406c530 0x7feb8406e9f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7feb98034000 tx=0x7feb98007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:54.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.170+0000 7feb9ea00640 1 -- 172.21.15.19:0/2241867112 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7feb900595a0 con 0x7feba81580d0 2024-03-28T13:17:54.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.413+0000 7feba7e00640 1 -- 172.21.15.19:0/2241867112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feba80b4a20 con 0x7feba81580d0 2024-03-28T13:17:54.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.415+0000 7feb9ea00640 1 -- 172.21.15.19:0/2241867112 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5187 (secure 0 0 0) 0x7feb90058ea0 con 0x7feba81580d0 2024-03-28T13:17:54.415 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:17:54.415 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":6040,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:16:50.520332+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6175":{"gid":6175,"name":"b","rank":0,"incarnation":64,"state":"up:active","state_seq":6,"addr":"172.21.15.188:6836/1749969053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1749969053},{"type":"v1","addr":"172.21.15.188:6836","nonce":1749969053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6175,"qdb_cluster":[6175]},"id":6}]} 2024-03-28T13:17:54.415 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 68 2024-03-28T13:17:54.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.417+0000 7feba7e00640 1 -- 172.21.15.19:0/2241867112 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb8406c530 msgr2=0x7feb8406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:54.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.417+0000 7feba7e00640 1 --2- 172.21.15.19:0/2241867112 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb8406c530 0x7feb8406e9f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7feb98034000 tx=0x7feb98007000 comp rx=0 tx=0).stop 2024-03-28T13:17:54.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.417+0000 7feba7e00640 1 -- 172.21.15.19:0/2241867112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feba81580d0 msgr2=0x7feba81545a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:54.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.417+0000 7feba7e00640 1 --2- 172.21.15.19:0/2241867112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feba81580d0 0x7feba81545a0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7feb9000d7a0 tx=0x7feb9000dc60 comp rx=0 tx=0).stop 2024-03-28T13:17:54.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.417+0000 7feba7e00640 1 -- 172.21.15.19:0/2241867112 shutdown_connections 2024-03-28T13:17:54.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.417+0000 7feba7e00640 1 --2- 172.21.15.19:0/2241867112 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb8406c530 0x7feb8406e9f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:54.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.417+0000 7feba7e00640 1 --2- 172.21.15.19:0/2241867112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feba8154ae0 0x7feba8160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:54.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.417+0000 7feba7e00640 1 --2- 172.21.15.19:0/2241867112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feba81580d0 0x7feba81545a0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:54.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.417+0000 7feba7e00640 1 --2- 172.21.15.19:0/2241867112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb8c0039e0 0x7feba814ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:54.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.417+0000 7feba7e00640 1 -- 172.21.15.19:0/2241867112 >> 172.21.15.19:0/2241867112 conn(0x7feba80676d0 msgr2=0x7feba815be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:54.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.418+0000 7feba7e00640 1 -- 172.21.15.19:0/2241867112 shutdown_connections 2024-03-28T13:17:54.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.418+0000 7feba7e00640 1 -- 172.21.15.19:0/2241867112 wait complete. 2024-03-28T13:17:54.436 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-03-28T13:17:54.700 INFO:teuthology.orchestra.run.smithi188.stdout:{"message":"","return_code":0} 2024-03-28T13:17:54.701 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-28T13:17:54.701 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T13:17:54.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.932+0000 7f84e9e00640 1 Processor -- start 2024-03-28T13:17:54.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.932+0000 7f84e9e00640 1 -- start start 2024-03-28T13:17:54.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.933+0000 7f84e9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f84e4147a40 0x7f84e40cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:54.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.933+0000 7f84e9e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84e40d01c0 0x7f84e40cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:54.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.933+0000 7f84e9e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e4161f30 con 0x7f84e40ce1b0 2024-03-28T13:17:54.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.933+0000 7f84e9e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f84e4162b00 con 0x7f84e4147a40 2024-03-28T13:17:54.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.933+0000 7f84e9e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f84e41621d0 con 0x7f84e40d01c0 2024-03-28T13:17:54.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.934+0000 7f84e9400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f84e40ce1b0 0x7f84e40ad890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34836/0 (socket says 172.21.15.19:34836) 2024-03-28T13:17:54.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.934+0000 7f84e9400640 1 -- 172.21.15.19:0/1483856404 learned_addr learned my addr 172.21.15.19:0/1483856404 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:17:54.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.934+0000 7f84e8a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f84e4147a40 0x7f84e40cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:54.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.934+0000 7f84e3e00640 1 --2- 172.21.15.19:0/1483856404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84e40d01c0 0x7f84e40cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:54.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.934+0000 7f84e3400640 1 -- 172.21.15.19:0/1483856404 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3917572453 0 0) 0x7f84e4161f30 con 0x7f84e40ce1b0 2024-03-28T13:17:54.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.934+0000 7f84e3400640 1 -- 172.21.15.19:0/1483856404 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84cc003520 con 0x7f84e40ce1b0 2024-03-28T13:17:54.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.934+0000 7f84e8a00640 1 -- 172.21.15.19:0/1483856404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84e40d01c0 msgr2=0x7f84e40cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:54.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.934+0000 7f84e8a00640 1 --2- 172.21.15.19:0/1483856404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84e40d01c0 0x7f84e40cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:54.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.934+0000 7f84e8a00640 1 -- 172.21.15.19:0/1483856404 >> v1:172.21.15.19:6789/0 conn(0x7f84e40ce1b0 legacy=0x7f84e40ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:54.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.934+0000 7f84e8a00640 1 -- 172.21.15.19:0/1483856404 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e41624b0 con 0x7f84e4147a40 2024-03-28T13:17:54.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.935+0000 7f84e3e00640 1 --2- 172.21.15.19:0/1483856404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84e40d01c0 0x7f84e40cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:17:54.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.936+0000 7f84e8a00640 1 --2- 172.21.15.19:0/1483856404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e4147a40 0x7f84e40cfc80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f84d80097a0 tx=0x7f84d802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2366f0a9e9849e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:54.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.936+0000 7f84e3400640 1 -- 172.21.15.19:0/1483856404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8035020 con 0x7f84e4147a40 2024-03-28T13:17:54.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.936+0000 7f84e3400640 1 -- 172.21.15.19:0/1483856404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d8031810 con 0x7f84e4147a40 2024-03-28T13:17:54.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.936+0000 7f84e3400640 1 -- 172.21.15.19:0/1483856404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8031be0 con 0x7f84e4147a40 2024-03-28T13:17:54.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.937+0000 7f84e9e00640 1 -- 172.21.15.19:0/1483856404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e4147a40 msgr2=0x7f84e40cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:54.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.937+0000 7f84e9e00640 1 --2- 172.21.15.19:0/1483856404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e4147a40 0x7f84e40cfc80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f84d80097a0 tx=0x7f84d802b8b0 comp rx=0 tx=0).stop 2024-03-28T13:17:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.937+0000 7f84e9e00640 1 -- 172.21.15.19:0/1483856404 shutdown_connections 2024-03-28T13:17:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.937+0000 7f84e9e00640 1 --2- 172.21.15.19:0/1483856404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84e40d01c0 0x7f84e40cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.937+0000 7f84e9e00640 1 --2- 172.21.15.19:0/1483856404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e4147a40 0x7f84e40cfc80 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.937+0000 7f84e9e00640 1 -- 172.21.15.19:0/1483856404 >> 172.21.15.19:0/1483856404 conn(0x7f84e40ccad0 msgr2=0x7f84e40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.937+0000 7f84e9e00640 1 -- 172.21.15.19:0/1483856404 shutdown_connections 2024-03-28T13:17:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.937+0000 7f84e9e00640 1 -- 172.21.15.19:0/1483856404 wait complete. 2024-03-28T13:17:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.939+0000 7f84e9e00640 1 Processor -- start 2024-03-28T13:17:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.939+0000 7f84e9e00640 1 -- start start 2024-03-28T13:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.940+0000 7f84e9e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84e40ce1b0 0x7f84e413a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.940+0000 7f84e9e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84e40d01c0 0x7f84e415dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.940+0000 7f84e9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e4147a40 0x7f84e4145890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.940+0000 7f84e9e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f84e4147e20 con 0x7f84e4147a40 2024-03-28T13:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.940+0000 7f84e9e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f84e40d0620 con 0x7f84e40d01c0 2024-03-28T13:17:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.940+0000 7f84e9e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f84e4147fa0 con 0x7f84e40ce1b0 2024-03-28T13:17:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.941+0000 7f84e9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e4147a40 0x7f84e4145890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.941+0000 7f84e9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e4147a40 0x7f84e4145890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53816/0 (socket says 172.21.15.19:53816) 2024-03-28T13:17:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.941+0000 7f84e9400640 1 -- 172.21.15.19:0/3606763581 learned_addr learned my addr 172.21.15.19:0/3606763581 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:54.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.941+0000 7f84e8a00640 1 --2- 172.21.15.19:0/3606763581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84e40ce1b0 0x7f84e413a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.941+0000 7f84e9400640 1 -- 172.21.15.19:0/3606763581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84e40ce1b0 msgr2=0x7f84e413a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.941+0000 7f84e9400640 1 --2- 172.21.15.19:0/3606763581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84e40ce1b0 0x7f84e413a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.941+0000 7f84e9400640 1 -- 172.21.15.19:0/3606763581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84e40d01c0 msgr2=0x7f84e415dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.941+0000 7f84e3e00640 1 --2- 172.21.15.19:0/3606763581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84e40d01c0 0x7f84e415dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.941+0000 7f84e9400640 1 --2- 172.21.15.19:0/3606763581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84e40d01c0 0x7f84e415dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.941+0000 7f84e9400640 1 -- 172.21.15.19:0/3606763581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e413ad90 con 0x7f84e4147a40 2024-03-28T13:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.942+0000 7f84e9400640 1 --2- 172.21.15.19:0/3606763581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e4147a40 0x7f84e4145890 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00cb60 tx=0x7f84dc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.942+0000 7f84e0c00640 1 -- 172.21.15.19:0/3606763581 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc017070 con 0x7f84e4147a40 2024-03-28T13:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.942+0000 7f84e0c00640 1 -- 172.21.15.19:0/3606763581 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84dc00ad40 con 0x7f84e4147a40 2024-03-28T13:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.942+0000 7f84e0c00640 1 -- 172.21.15.19:0/3606763581 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc012600 con 0x7f84e4147a40 2024-03-28T13:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.942+0000 7f84e9e00640 1 -- 172.21.15.19:0/3606763581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e413b070 con 0x7f84e4147a40 2024-03-28T13:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.942+0000 7f84e9e00640 1 -- 172.21.15.19:0/3606763581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e413b630 con 0x7f84e4147a40 2024-03-28T13:17:54.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.943+0000 7f84e9e00640 1 -- 172.21.15.19:0/3606763581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e40ce900 con 0x7f84e4147a40 2024-03-28T13:17:54.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.944+0000 7f84e0c00640 1 -- 172.21.15.19:0/3606763581 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f84dc01a070 con 0x7f84e4147a40 2024-03-28T13:17:54.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.945+0000 7f84e0c00640 1 --2- 172.21.15.19:0/3606763581 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f84c406c530 0x7f84c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:54.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.946+0000 7f84e0c00640 1 -- 172.21.15.19:0/3606763581 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f84dc08b390 con 0x7f84e4147a40 2024-03-28T13:17:54.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.946+0000 7f84e8a00640 1 --2- 172.21.15.19:0/3606763581 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f84c406c530 0x7f84c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:54.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.947+0000 7f84e8a00640 1 --2- 172.21.15.19:0/3606763581 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f84c406c530 0x7f84c406e9f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f84d8036000 tx=0x7f84d803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:54.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:54.950+0000 7f84e0c00640 1 -- 172.21.15.19:0/3606763581 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f84e40ce900 con 0x7f84e4147a40 2024-03-28T13:17:55.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.193+0000 7f84e9e00640 1 -- 172.21.15.19:0/3606763581 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f84e4075940 con 0x7f84e4147a40 2024-03-28T13:17:55.194 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:17:55.194+0000 7f6b74400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T13:17:55.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.197+0000 7f84e0c00640 1 -- 172.21.15.19:0/3606763581 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) ==== 161+0+0 (secure 0 0 0) 0x7f84dc059390 con 0x7f84e4147a40 2024-03-28T13:17:55.197 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T13:17:55.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.200+0000 7f84e9e00640 1 -- 172.21.15.19:0/3606763581 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f84c406c530 msgr2=0x7f84c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.200+0000 7f84e9e00640 1 --2- 172.21.15.19:0/3606763581 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f84c406c530 0x7f84c406e9f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f84d8036000 tx=0x7f84d803c040 comp rx=0 tx=0).stop 2024-03-28T13:17:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.200+0000 7f84e9e00640 1 -- 172.21.15.19:0/3606763581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e4147a40 msgr2=0x7f84e4145890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.200+0000 7f84e9e00640 1 --2- 172.21.15.19:0/3606763581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e4147a40 0x7f84e4145890 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00cb60 tx=0x7f84dc00a430 comp rx=0 tx=0).stop 2024-03-28T13:17:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.201+0000 7f84e9e00640 1 -- 172.21.15.19:0/3606763581 shutdown_connections 2024-03-28T13:17:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.201+0000 7f84e9e00640 1 --2- 172.21.15.19:0/3606763581 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f84c406c530 0x7f84c406e9f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.201+0000 7f84e9e00640 1 --2- 172.21.15.19:0/3606763581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f84e4147a40 0x7f84e4145890 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.201+0000 7f84e9e00640 1 --2- 172.21.15.19:0/3606763581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84e40d01c0 0x7f84e415dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.201+0000 7f84e9e00640 1 --2- 172.21.15.19:0/3606763581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84e40ce1b0 0x7f84e413a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.201+0000 7f84e9e00640 1 -- 172.21.15.19:0/3606763581 >> 172.21.15.19:0/3606763581 conn(0x7f84e40ccad0 msgr2=0x7f84e40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.201+0000 7f84e9e00640 1 -- 172.21.15.19:0/3606763581 shutdown_connections 2024-03-28T13:17:55.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.201+0000 7f84e9e00640 1 -- 172.21.15.19:0/3606763581 wait complete. 2024-03-28T13:17:55.220 DEBUG:teuthology.orchestra.run.smithi019:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-03-28T13:17:55.221 INFO:tasks.ceph.mds.b.smithi188.stderr: -4835> 2024-03-28T13:16:45.515+0000 7f809c200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:17:55.223 INFO:tasks.ceph.mds.b.smithi188.stderr: -4367> 2024-03-28T13:16:47.380+0000 7f8097200640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-03-28T13:17:55.224 INFO:tasks.ceph.mds.b.smithi188.stderr: -4157> 2024-03-28T13:16:47.677+0000 7f8097200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-03-28T13:16:35.247197+0000 1=1+0) n(v0 rc2024-03-28T13:16:35.374196+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x5622e97f0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-28T13:16:35.247197+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-28T13:16:35.374196+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-03-28T13:17:55.230 INFO:tasks.ceph.mds.b.smithi188.stderr: -2278> 2024-03-28T13:17:19.868+0000 7f8097200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv20 ap=3 DIRTYPARENT f(v0 m2024-03-28T13:16:35.247197+0000 1=1+0)->f(v1 m2024-03-28T13:16:35.247197+0000) n(v0 rc2024-03-28T13:16:35.374196+0000 2=1+1)->n(v0 rc2024-03-28T13:16:35.374196+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5622e97f0100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-28T13:16:35.247197+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-28T13:16:35.374196+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-03-28T13:17:55.287 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-03-28T13:17:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.321+0000 7fae63862740 1 -- start start 2024-03-28T13:17:55.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.321+0000 7fae63862740 1 Processor -- start 2024-03-28T13:17:55.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.322+0000 7fae63862740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b758edcc00 0x55b759054000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.322+0000 7fae63862740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b758edd800 0x55b759054580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.322+0000 7fae63862740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b758eddc00 0x55b759054b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.322+0000 7fae63862740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b759058780 con 0x55b758eddc00 2024-03-28T13:17:55.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.322+0000 7fae63862740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b759058600 con 0x55b758edd800 2024-03-28T13:17:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.322+0000 7fae63862740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55b759058480 con 0x55b758edcc00 2024-03-28T13:17:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.322+0000 7fae61c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b758eddc00 0x55b759054b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.322+0000 7fae61c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b758eddc00 0x55b759054b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53834/0 (socket says 172.21.15.19:53834) 2024-03-28T13:17:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.322+0000 7fae61c00640 1 -- 172.21.15.19:0/3053995795 learned_addr learned my addr 172.21.15.19:0/3053995795 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.323+0000 7fae61200640 1 --2- 172.21.15.19:0/3053995795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b758edcc00 0x55b759054000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.323+0000 7fae60800640 1 --2- 172.21.15.19:0/3053995795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b758edd800 0x55b759054580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.323+0000 7fae61c00640 1 -- 172.21.15.19:0/3053995795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b758edcc00 msgr2=0x55b759054000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.323+0000 7fae61c00640 1 --2- 172.21.15.19:0/3053995795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b758edcc00 0x55b759054000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.323+0000 7fae61c00640 1 -- 172.21.15.19:0/3053995795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b758edd800 msgr2=0x55b759054580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.323+0000 7fae61c00640 1 --2- 172.21.15.19:0/3053995795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b758edd800 0x55b759054580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.323+0000 7fae61c00640 1 -- 172.21.15.19:0/3053995795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b758f09680 con 0x55b758eddc00 2024-03-28T13:17:55.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.323+0000 7fae60800640 1 --2- 172.21.15.19:0/3053995795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b758edd800 0x55b759054580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:55.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.323+0000 7fae61c00640 1 --2- 172.21.15.19:0/3053995795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b758eddc00 0x55b759054b00 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x55b759cf70b0 tx=0x55b759c74a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeeddd6cc9148fba server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.323+0000 7fae5e000640 1 -- 172.21.15.19:0/3053995795 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b758f029c0 con 0x55b758eddc00 2024-03-28T13:17:55.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.324+0000 7fae5e000640 1 -- 172.21.15.19:0/3053995795 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b758f26e00 con 0x55b758eddc00 2024-03-28T13:17:55.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.324+0000 7fae5e000640 1 -- 172.21.15.19:0/3053995795 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b758f02680 con 0x55b758eddc00 2024-03-28T13:17:55.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.324+0000 7fae63862740 1 -- 172.21.15.19:0/3053995795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b758f09860 con 0x55b758eddc00 2024-03-28T13:17:55.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.324+0000 7fae63862740 1 -- 172.21.15.19:0/3053995795 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55b758f092c0 con 0x55b758eddc00 2024-03-28T13:17:55.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.324+0000 7fae63862740 4 waiting for MDS map... 2024-03-28T13:17:55.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.324+0000 7fae5e000640 1 -- 172.21.15.19:0/3053995795 <== mon.0 v2:172.21.15.19:3300/0 4 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x55b759dac200 con 0x55b758eddc00 2024-03-28T13:17:55.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.324+0000 7fae5e000640 1 -- 172.21.15.19:0/3053995795 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 69) ==== 2885+0+0 (secure 0 0 0) 0x55b758f35180 con 0x55b758eddc00 2024-03-28T13:17:55.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.324+0000 7fae63862740 4 Got MDS map 69 2024-03-28T13:17:55.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.324+0000 7fae63862740 4 datascan.main: Using metadata pool output 2024-03-28T13:17:55.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.324+0000 7fae63862740 4 datascan.main: connecting to RADOS... 2024-03-28T13:17:55.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.325+0000 7fae63862740 1 Processor -- start 2024-03-28T13:17:55.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.325+0000 7fae63862740 1 -- start start 2024-03-28T13:17:55.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.326+0000 7fae63862740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b759e08000 0x55b759055080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.326+0000 7fae63862740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b759e08400 0x55b759055600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.326+0000 7fae63862740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b759e08800 0x55b759055b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.326+0000 7fae63862740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b759058300 con 0x55b759e08400 2024-03-28T13:17:55.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.326+0000 7fae63862740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b759058000 con 0x55b759e08000 2024-03-28T13:17:55.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.326+0000 7fae63862740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55b758f3c900 con 0x55b759e08800 2024-03-28T13:17:55.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.326+0000 7fae61200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b759e08400 0x55b759055600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.327+0000 7fae61200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b759e08400 0x55b759055600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53850/0 (socket says 172.21.15.19:53850) 2024-03-28T13:17:55.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.327+0000 7fae61200640 1 -- 172.21.15.19:0/2425839862 learned_addr learned my addr 172.21.15.19:0/2425839862 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:55.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.327+0000 7fae60800640 1 --2- 172.21.15.19:0/2425839862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b759e08000 0x55b759055080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.327+0000 7fae60800640 1 --2- 172.21.15.19:0/2425839862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b759e08800 0x55b759055b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.327+0000 7fae61200640 1 -- 172.21.15.19:0/2425839862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b759e08800 msgr2=0x55b759055b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:55.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.327+0000 7fae61200640 1 --2- 172.21.15.19:0/2425839862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b759e08800 0x55b759055b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.327+0000 7fae61200640 1 -- 172.21.15.19:0/2425839862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b759e08000 msgr2=0x55b759055080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:55.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.327+0000 7fae61200640 1 --2- 172.21.15.19:0/2425839862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b759e08000 0x55b759055080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.327+0000 7fae61200640 1 -- 172.21.15.19:0/2425839862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b758f094a0 con 0x55b759e08400 2024-03-28T13:17:55.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.327+0000 7fae61200640 1 --2- 172.21.15.19:0/2425839862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b759e08400 0x55b759055600 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x55b759dbcc90 tx=0x55b759dc4de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14092eb67d8fbdd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.328+0000 7fae5b800640 1 -- 172.21.15.19:0/2425839862 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b758f029c0 con 0x55b759e08400 2024-03-28T13:17:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.328+0000 7fae5b800640 1 -- 172.21.15.19:0/2425839862 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b758f26540 con 0x55b759e08400 2024-03-28T13:17:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.328+0000 7fae5b800640 1 -- 172.21.15.19:0/2425839862 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b758f02680 con 0x55b759e08400 2024-03-28T13:17:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.328+0000 7fae63862740 1 -- 172.21.15.19:0/2425839862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b759e08400 msgr2=0x55b759055600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.328+0000 7fae63862740 1 --2- 172.21.15.19:0/2425839862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b759e08400 0x55b759055600 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x55b759dbcc90 tx=0x55b759dc4de0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.328+0000 7fae63862740 1 -- 172.21.15.19:0/2425839862 shutdown_connections 2024-03-28T13:17:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.328+0000 7fae63862740 1 --2- 172.21.15.19:0/2425839862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b759e08800 0x55b759055b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.328+0000 7fae63862740 1 --2- 172.21.15.19:0/2425839862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b759e08400 0x55b759055600 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.328+0000 7fae63862740 1 --2- 172.21.15.19:0/2425839862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b759e08000 0x55b759055080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.328+0000 7fae63862740 1 -- 172.21.15.19:0/2425839862 >> 172.21.15.19:0/2425839862 conn(0x55b759c45c00 msgr2=0x55b759c7b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.328+0000 7fae63862740 1 -- 172.21.15.19:0/2425839862 shutdown_connections 2024-03-28T13:17:55.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.328+0000 7fae63862740 1 -- 172.21.15.19:0/2425839862 wait complete. 2024-03-28T13:17:55.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.329+0000 7fae63862740 1 Processor -- start 2024-03-28T13:17:55.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.329+0000 7fae63862740 1 -- start start 2024-03-28T13:17:55.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.330+0000 7fae63862740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b759e08000 0x55b759055080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.330+0000 7fae63862740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b759e08400 0x55b759055600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.330+0000 7fae63862740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b759e08800 0x55b759055b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.330+0000 7fae63862740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b759058300 con 0x55b759e08000 2024-03-28T13:17:55.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.330+0000 7fae63862740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b759058000 con 0x55b759e08400 2024-03-28T13:17:55.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.330+0000 7fae63862740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55b759059500 con 0x55b759e08800 2024-03-28T13:17:55.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.330+0000 7fae60800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b759e08000 0x55b759055080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.330+0000 7fae60800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b759e08000 0x55b759055080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53858/0 (socket says 172.21.15.19:53858) 2024-03-28T13:17:55.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.330+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 learned_addr learned my addr 172.21.15.19:0/2166498107 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:55.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.331+0000 7fae61200640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b759e08400 0x55b759055600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.331+0000 7fae60800640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b759e08800 0x55b759055b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.331+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b759e08800 msgr2=0x55b759055b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.331+0000 7fae60800640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b759e08800 0x55b759055b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.331+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b759e08400 msgr2=0x55b759055600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.331+0000 7fae60800640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b759e08400 0x55b759055600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.331+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b758f090e0 con 0x55b759e08000 2024-03-28T13:17:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.331+0000 7fae60800640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b759e08000 0x55b759055080 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x55b759dbd980 tx=0x55b759dc4d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.331+0000 7fae59a00640 1 -- 172.21.15.19:0/2166498107 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b758f029c0 con 0x55b759e08000 2024-03-28T13:17:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.331+0000 7fae59a00640 1 -- 172.21.15.19:0/2166498107 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55b758f26a80 con 0x55b759e08000 2024-03-28T13:17:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.331+0000 7fae59a00640 1 -- 172.21.15.19:0/2166498107 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55b758f02680 con 0x55b759e08000 2024-03-28T13:17:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.332+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b758f09a40 con 0x55b759e08000 2024-03-28T13:17:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.332+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b758f083c0 con 0x55b759e08000 2024-03-28T13:17:55.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.332+0000 7fae63862740 4 datascan.init: resolving metadata pool 12 2024-03-28T13:17:55.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.333+0000 7fae59a00640 1 -- 172.21.15.19:0/2166498107 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x55b759e16c00 con 0x55b759e08000 2024-03-28T13:17:55.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.334+0000 7fae59a00640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x55b759e18c00 0x55b759056100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.334+0000 7fae59a00640 1 -- 172.21.15.19:0/2166498107 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x55b759e1ac00 con 0x55b759e08000 2024-03-28T13:17:55.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.334+0000 7fae63862740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-28T13:17:55.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.334+0000 7fae63862740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-03-28T13:17:55.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.334+0000 7fae63862740 4 datascan.main: opening data pool 'cephfs_data' 2024-03-28T13:17:55.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.334+0000 7fae63862740 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x55b759e08c00 0x55b759056680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.334+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759e09000 con 0x55b759e08c00 2024-03-28T13:17:55.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.335+0000 7fae60800640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x55b759e08c00 0x55b759056680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.335+0000 7fae61200640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x55b759e18c00 0x55b759056100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.335+0000 7fae60800640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x55b759e08c00 0x55b759056680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.335+0000 7fae61200640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x55b759e18c00 0x55b759056100 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55b759dbcc90 tx=0x55b759c74a80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.338+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.1 v2:172.21.15.19:6801/1998544260 1 ==== osd_op_reply(1 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759e08c00 2024-03-28T13:17:55.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.338+0000 7fae5ae00640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x55b759e19000 0x55b759262000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.338+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759e19400 con 0x55b759e19000 2024-03-28T13:17:55.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.339+0000 7fae61c00640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x55b759e19000 0x55b759262000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.339+0000 7fae61c00640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x55b759e19000 0x55b759262000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.342+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.2 v2:172.21.15.19:6816/1693606238 1 ==== osd_op_reply(2 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759e19000 2024-03-28T13:17:55.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.342+0000 7fae5b800640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x55b759e19c00 0x55b759262580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.342+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759270000 con 0x55b759e19c00 2024-03-28T13:17:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.343+0000 7fae61200640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x55b759e19c00 0x55b759262580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.343+0000 7fae61200640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x55b759e19c00 0x55b759262580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.349+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.7 v2:172.21.15.188:6824/861796513 1 ==== osd_op_reply(3 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759e19c00 2024-03-28T13:17:55.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.349+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759e19800 con 0x55b759e19000 2024-03-28T13:17:55.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.349+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.2 v2:172.21.15.19:6816/1693606238 2 ==== osd_op_reply(4 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759e19000 2024-03-28T13:17:55.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.349+0000 7fae5b800640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x55b759270400 0x55b759262b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.349+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759270800 con 0x55b759270400 2024-03-28T13:17:55.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.350+0000 7fae60800640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x55b759270400 0x55b759262b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.351+0000 7fae60800640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x55b759270400 0x55b759262b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.352+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.5 v2:172.21.15.188:6800/3324247755 1 ==== osd_op_reply(5 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759270400 2024-03-28T13:17:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.352+0000 7fae5ae00640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x55b759271400 0x55b759263600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.352+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759271800 con 0x55b759271400 2024-03-28T13:17:55.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.353+0000 7fae61c00640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x55b759271400 0x55b759263600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.354+0000 7fae61c00640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x55b759271400 0x55b759263600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.355+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.6 v2:172.21.15.188:6816/3597358155 1 ==== osd_op_reply(6 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759271400 2024-03-28T13:17:55.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.355+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759270c00 con 0x55b759e08c00 2024-03-28T13:17:55.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.356+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.1 v2:172.21.15.19:6801/1998544260 2 ==== osd_op_reply(7 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759e08c00 2024-03-28T13:17:55.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.356+0000 7fae5ae00640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x55b759271c00 0x55b759263b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.356+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd8000 con 0x55b759271c00 2024-03-28T13:17:55.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.356+0000 7fae61200640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x55b759271c00 0x55b759263b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.357+0000 7fae61200640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x55b759271c00 0x55b759263b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.360+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.0 v2:172.21.15.19:6800/4106483280 1 ==== osd_op_reply(8 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759271c00 2024-03-28T13:17:55.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.360+0000 7fae5b800640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x55b759271000 0x55b759263080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.360+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd9000 con 0x55b759271000 2024-03-28T13:17:55.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.360+0000 7fae60800640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x55b759271000 0x55b759263080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.361+0000 7fae60800640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x55b759271000 0x55b759263080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.361+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.3 v2:172.21.15.19:6819/3265515865 1 ==== osd_op_reply(9 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759271000 2024-03-28T13:17:55.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.361+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd8400 con 0x55b759e08c00 2024-03-28T13:17:55.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.362+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.1 v2:172.21.15.19:6801/1998544260 3 ==== osd_op_reply(10 [pgnls start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759e08c00 2024-03-28T13:17:55.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.362+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd9400 con 0x55b759271400 2024-03-28T13:17:55.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.363+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.6 v2:172.21.15.188:6816/3597358155 2 ==== osd_op_reply(11 [pgnls start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759271400 2024-03-28T13:17:55.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.363+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd8800 con 0x55b759271c00 2024-03-28T13:17:55.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.363+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.0 v2:172.21.15.19:6800/4106483280 2 ==== osd_op_reply(12 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759271c00 2024-03-28T13:17:55.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.363+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd9800 con 0x55b759271400 2024-03-28T13:17:55.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.364+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.6 v2:172.21.15.188:6816/3597358155 3 ==== osd_op_reply(13 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759271400 2024-03-28T13:17:55.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.364+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd8c00 con 0x55b759e19c00 2024-03-28T13:17:55.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.365+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.7 v2:172.21.15.188:6824/861796513 2 ==== osd_op_reply(14 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759e19c00 2024-03-28T13:17:55.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.365+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd9c00 con 0x55b759271c00 2024-03-28T13:17:55.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.366+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.0 v2:172.21.15.19:6800/4106483280 3 ==== osd_op_reply(15 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759271c00 2024-03-28T13:17:55.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.366+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759271800 con 0x55b759270400 2024-03-28T13:17:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.367+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.5 v2:172.21.15.188:6800/3324247755 2 ==== osd_op_reply(16 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759270400 2024-03-28T13:17:55.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.367+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd9400 con 0x55b759e08c00 2024-03-28T13:17:55.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.368+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.1 v2:172.21.15.19:6801/1998544260 4 ==== osd_op_reply(17 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759e08c00 2024-03-28T13:17:55.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.368+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759e09c00 con 0x55b759e19c00 2024-03-28T13:17:55.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.369+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.7 v2:172.21.15.188:6824/861796513 3 ==== osd_op_reply(18 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759e19c00 2024-03-28T13:17:55.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.369+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd9800 con 0x55b759271400 2024-03-28T13:17:55.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.370+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.6 v2:172.21.15.188:6816/3597358155 4 ==== osd_op_reply(19 [pgnls start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759271400 2024-03-28T13:17:55.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.370+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759e09000 con 0x55b759e19c00 2024-03-28T13:17:55.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.372+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.7 v2:172.21.15.188:6824/861796513 4 ==== osd_op_reply(20 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759e19c00 2024-03-28T13:17:55.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.372+0000 7fae5b800640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x55b759270000 0x55b759264100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.372+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759270800 con 0x55b759270000 2024-03-28T13:17:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.372+0000 7fae61c00640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x55b759270000 0x55b759264100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.373+0000 7fae61c00640 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x55b759270000 0x55b759264100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.379+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.4 v2:172.21.15.188:6808/1845695132 1 ==== osd_op_reply(21 [pgnls start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759270000 2024-03-28T13:17:55.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.379+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759e19400 con 0x55b759e19c00 2024-03-28T13:17:55.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.380+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.7 v2:172.21.15.188:6824/861796513 5 ==== osd_op_reply(22 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759e19c00 2024-03-28T13:17:55.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.380+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759270c00 con 0x55b759270400 2024-03-28T13:17:55.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.381+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.5 v2:172.21.15.188:6800/3324247755 3 ==== osd_op_reply(23 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759270400 2024-03-28T13:17:55.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.381+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759e19800 con 0x55b759e19000 2024-03-28T13:17:55.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.382+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.2 v2:172.21.15.19:6816/1693606238 3 ==== osd_op_reply(24 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759e19000 2024-03-28T13:17:55.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.382+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd8c00 con 0x55b759271000 2024-03-28T13:17:55.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.383+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.3 v2:172.21.15.19:6819/3265515865 2 ==== osd_op_reply(25 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759271000 2024-03-28T13:17:55.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.383+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd8400 con 0x55b759271000 2024-03-28T13:17:55.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.384+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.3 v2:172.21.15.19:6819/3265515865 3 ==== osd_op_reply(26 [pgnls start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759271000 2024-03-28T13:17:55.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.384+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd9c00 con 0x55b759e19c00 2024-03-28T13:17:55.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.385+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.7 v2:172.21.15.188:6824/861796513 6 ==== osd_op_reply(27 [pgnls start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759e19c00 2024-03-28T13:17:55.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.385+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd9400 con 0x55b759e19000 2024-03-28T13:17:55.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.386+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.2 v2:172.21.15.19:6816/1693606238 4 ==== osd_op_reply(28 [pgnls start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759e19000 2024-03-28T13:17:55.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.386+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd8000 con 0x55b759271c00 2024-03-28T13:17:55.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.387+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.0 v2:172.21.15.19:6800/4106483280 4 ==== osd_op_reply(29 [pgnls start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759271c00 2024-03-28T13:17:55.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.387+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd8c00 con 0x55b759271400 2024-03-28T13:17:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.388+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.6 v2:172.21.15.188:6816/3597358155 5 ==== osd_op_reply(30 [pgnls start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759271400 2024-03-28T13:17:55.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.388+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759e19400 con 0x55b759271c00 2024-03-28T13:17:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.389+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.0 v2:172.21.15.19:6800/4106483280 5 ==== osd_op_reply(31 [pgnls start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759271c00 2024-03-28T13:17:55.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.389+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd8800 con 0x55b759270000 2024-03-28T13:17:55.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.390+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.4 v2:172.21.15.188:6808/1845695132 2 ==== osd_op_reply(32 [pgnls start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759270000 2024-03-28T13:17:55.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.390+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759e09c00 con 0x55b759e08c00 2024-03-28T13:17:55.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.391+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.1 v2:172.21.15.19:6801/1998544260 5 ==== osd_op_reply(33 [pgnls start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759e08c00 2024-03-28T13:17:55.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.391+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759271800 con 0x55b759e19000 2024-03-28T13:17:55.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.392+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.2 v2:172.21.15.19:6816/1693606238 5 ==== osd_op_reply(34 [pgnls start_epoch 94 out=49b] v92'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759e19000 2024-03-28T13:17:55.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.392+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759e09000 con 0x55b759271400 2024-03-28T13:17:55.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.394+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.6 v2:172.21.15.188:6816/3597358155 6 ==== osd_op_reply(35 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759271400 2024-03-28T13:17:55.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.394+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759270c00 con 0x55b759270000 2024-03-28T13:17:55.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.395+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.4 v2:172.21.15.188:6808/1845695132 3 ==== osd_op_reply(36 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759270000 2024-03-28T13:17:55.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.395+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd9800 con 0x55b759271c00 2024-03-28T13:17:55.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.396+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.0 v2:172.21.15.19:6800/4106483280 6 ==== osd_op_reply(37 [pgnls start_epoch 94 out=49b] v92'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759271c00 2024-03-28T13:17:55.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.396+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cde000 con 0x55b759271c00 2024-03-28T13:17:55.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.396+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.0 v2:172.21.15.19:6800/4106483280 7 ==== osd_op_reply(38 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759271c00 2024-03-28T13:17:55.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.397+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd9400 con 0x55b759e19c00 2024-03-28T13:17:55.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.398+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.7 v2:172.21.15.188:6824/861796513 7 ==== osd_op_reply(39 [pgnls start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759e19c00 2024-03-28T13:17:55.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.398+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cde400 con 0x55b759270000 2024-03-28T13:17:55.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.399+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.4 v2:172.21.15.188:6808/1845695132 4 ==== osd_op_reply(40 [pgnls start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759270000 2024-03-28T13:17:55.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.399+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd8c00 con 0x55b759e19000 2024-03-28T13:17:55.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.400+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.2 v2:172.21.15.19:6816/1693606238 6 ==== osd_op_reply(41 [pgnls start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759e19000 2024-03-28T13:17:55.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.400+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cde800 con 0x55b759271000 2024-03-28T13:17:55.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.400+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.3 v2:172.21.15.19:6819/3265515865 4 ==== osd_op_reply(42 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759271000 2024-03-28T13:17:55.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.401+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd8800 con 0x55b759271000 2024-03-28T13:17:55.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.401+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.3 v2:172.21.15.19:6819/3265515865 5 ==== osd_op_reply(43 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759271000 2024-03-28T13:17:55.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.402+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cdec00 con 0x55b759271400 2024-03-28T13:17:55.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.403+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.6 v2:172.21.15.188:6816/3597358155 7 ==== osd_op_reply(44 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759271400 2024-03-28T13:17:55.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.403+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd9000 con 0x55b759271c00 2024-03-28T13:17:55.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.404+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.0 v2:172.21.15.19:6800/4106483280 8 ==== osd_op_reply(45 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759271c00 2024-03-28T13:17:55.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.404+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cdf000 con 0x55b759e19c00 2024-03-28T13:17:55.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.405+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.7 v2:172.21.15.188:6824/861796513 8 ==== osd_op_reply(46 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759e19c00 2024-03-28T13:17:55.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.405+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759e19800 con 0x55b759e08c00 2024-03-28T13:17:55.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.406+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.1 v2:172.21.15.19:6801/1998544260 6 ==== osd_op_reply(47 [pgnls start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759e08c00 2024-03-28T13:17:55.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.406+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cdf400 con 0x55b759e19000 2024-03-28T13:17:55.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.406+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.2 v2:172.21.15.19:6816/1693606238 7 ==== osd_op_reply(48 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759e19000 2024-03-28T13:17:55.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.406+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759270800 con 0x55b759271400 2024-03-28T13:17:55.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.407+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.6 v2:172.21.15.188:6816/3597358155 8 ==== osd_op_reply(49 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759271400 2024-03-28T13:17:55.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.407+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cdf800 con 0x55b759270400 2024-03-28T13:17:55.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.409+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.5 v2:172.21.15.188:6800/3324247755 4 ==== osd_op_reply(50 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759270400 2024-03-28T13:17:55.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.409+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd8400 con 0x55b759e08c00 2024-03-28T13:17:55.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.410+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.1 v2:172.21.15.19:6801/1998544260 7 ==== osd_op_reply(51 [pgnls start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759e08c00 2024-03-28T13:17:55.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.410+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cde800 con 0x55b759271000 2024-03-28T13:17:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.410+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.3 v2:172.21.15.19:6819/3265515865 6 ==== osd_op_reply(52 [pgnls start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759271000 2024-03-28T13:17:55.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.410+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd8800 con 0x55b759271000 2024-03-28T13:17:55.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.411+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.3 v2:172.21.15.19:6819/3265515865 7 ==== osd_op_reply(53 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759271000 2024-03-28T13:17:55.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.411+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cdf800 con 0x55b759270400 2024-03-28T13:17:55.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.412+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.5 v2:172.21.15.188:6800/3324247755 5 ==== osd_op_reply(54 [pgnls start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759270400 2024-03-28T13:17:55.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.412+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759e19800 con 0x55b759e19000 2024-03-28T13:17:55.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.413+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.2 v2:172.21.15.19:6816/1693606238 8 ==== osd_op_reply(55 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759e19000 2024-03-28T13:17:55.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.413+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cde400 con 0x55b759271000 2024-03-28T13:17:55.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.414+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.3 v2:172.21.15.19:6819/3265515865 8 ==== osd_op_reply(56 [pgnls start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759271000 2024-03-28T13:17:55.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.414+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759e19400 con 0x55b759271000 2024-03-28T13:17:55.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.415+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.3 v2:172.21.15.19:6819/3265515865 9 ==== osd_op_reply(57 [pgnls start_epoch 94 out=81b] v92'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55b759c5e000 con 0x55b759271000 2024-03-28T13:17:55.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.415+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:58 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x55b759cdec00 con 0x55b759271000 2024-03-28T13:17:55.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.416+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.3 v2:172.21.15.19:6819/3265515865 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55b759c5e000 con 0x55b759271000 2024-03-28T13:17:55.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.416+0000 7fae63862740 10 datascan.operator(): handling object 1099511627777.0 2024-03-28T13:17:55.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.416+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:59 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x55b759cdf400 con 0x55b759271000 2024-03-28T13:17:55.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.420+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.3 v2:172.21.15.19:6819/3265515865 11 ==== osd_op_reply(59 10000000001.00000000 [call] v94'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55b759c5e000 con 0x55b759271000 2024-03-28T13:17:55.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.420+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:60 13.27 13:e4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cde000 con 0x55b759271400 2024-03-28T13:17:55.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.421+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.6 v2:172.21.15.188:6816/3597358155 9 ==== osd_op_reply(60 [pgnls start_epoch 94 out=49b] v92'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759271400 2024-03-28T13:17:55.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.421+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:61 13.17 13:e8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759271800 con 0x55b759e08c00 2024-03-28T13:17:55.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.422+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.1 v2:172.21.15.19:6801/1998544260 8 ==== osd_op_reply(61 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759e08c00 2024-03-28T13:17:55.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.422+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:62 13.37 13:ec000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd8c00 con 0x55b759e08c00 2024-03-28T13:17:55.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.423+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.1 v2:172.21.15.19:6801/1998544260 9 ==== osd_op_reply(62 [pgnls start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759e08c00 2024-03-28T13:17:55.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.423+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:63 13.f 13:f0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759270c00 con 0x55b759e19000 2024-03-28T13:17:55.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.424+0000 7fae61c00640 1 -- 172.21.15.19:0/2166498107 <== osd.2 v2:172.21.15.19:6816/1693606238 9 ==== osd_op_reply(63 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e500 con 0x55b759e19000 2024-03-28T13:17:55.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.424+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:64 13.2f 13:f4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd9c00 con 0x55b759271c00 2024-03-28T13:17:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.425+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 <== osd.0 v2:172.21.15.19:6800/4106483280 9 ==== osd_op_reply(64 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e780 con 0x55b759271c00 2024-03-28T13:17:55.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.425+0000 7fae5b800640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:65 13.1f 13:f8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759e09c00 con 0x55b759e08c00 2024-03-28T13:17:55.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.426+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.1 v2:172.21.15.19:6801/1998544260 10 ==== osd_op_reply(65 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759e08c00 2024-03-28T13:17:55.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.426+0000 7fae5ae00640 1 -- 172.21.15.19:0/2166498107 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:66 13.3f 13:fc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x55b759cd8000 con 0x55b759e08c00 2024-03-28T13:17:55.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae60800640 1 -- 172.21.15.19:0/2166498107 <== osd.1 v2:172.21.15.19:6801/1998544260 11 ==== osd_op_reply(66 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55b759c5e000 con 0x55b759e08c00 2024-03-28T13:17:55.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x55b759271c00 msgr2=0x55b759263b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x55b759271c00 0x55b759263b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x55b759e08c00 msgr2=0x55b759056680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x55b759e08c00 0x55b759056680 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x55b759e19000 msgr2=0x55b759262000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x55b759e19000 0x55b759262000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x55b759271000 msgr2=0x55b759263080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x55b759271000 0x55b759263080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x55b759270000 msgr2=0x55b759264100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x55b759270000 0x55b759264100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x55b759270400 msgr2=0x55b759262b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x55b759270400 0x55b759262b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x55b759271400 msgr2=0x55b759263600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x55b759271400 0x55b759263600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x55b759e19c00 msgr2=0x55b759262580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae61200640 1 -- 172.21.15.19:0/2166498107 reap_dead start 2024-03-28T13:17:55.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.427+0000 7fae63862740 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x55b759e19c00 0x55b759262580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.428+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x55b759e18c00 msgr2=0x55b759056100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.428+0000 7fae63862740 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x55b759e18c00 0x55b759056100 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55b759dbcc90 tx=0x55b759c74a80 comp rx=0 tx=0).stop 2024-03-28T13:17:55.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.428+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b759e08000 msgr2=0x55b759055080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.428+0000 7fae63862740 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b759e08000 0x55b759055080 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x55b759dbd980 tx=0x55b759dc4d20 comp rx=0 tx=0).stop 2024-03-28T13:17:55.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.428+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 shutdown_connections 2024-03-28T13:17:55.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.428+0000 7fae63862740 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x55b759e18c00 0x55b759056100 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.428+0000 7fae63862740 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x55b759e19c00 0x55b759262580 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.428+0000 7fae63862740 1 --2- 172.21.15.19:0/2166498107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b759e08000 0x55b759055080 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.428+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 >> 172.21.15.19:0/2166498107 conn(0x55b759c45c00 msgr2=0x55b759c7b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:55.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.429+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 shutdown_connections 2024-03-28T13:17:55.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.429+0000 7fae63862740 1 -- 172.21.15.19:0/2166498107 wait complete. 2024-03-28T13:17:55.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.430+0000 7fae63862740 1 -- 172.21.15.19:0/3053995795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b758eddc00 msgr2=0x55b759054b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.430+0000 7fae63862740 1 --2- 172.21.15.19:0/3053995795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b758eddc00 0x55b759054b00 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x55b759cf70b0 tx=0x55b759c74a20 comp rx=0 tx=0).stop 2024-03-28T13:17:55.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.430+0000 7fae63862740 1 -- 172.21.15.19:0/3053995795 shutdown_connections 2024-03-28T13:17:55.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.430+0000 7fae63862740 1 --2- 172.21.15.19:0/3053995795 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b758eddc00 0x55b759054b00 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.430+0000 7fae63862740 1 --2- 172.21.15.19:0/3053995795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b758edd800 0x55b759054580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.430+0000 7fae63862740 1 --2- 172.21.15.19:0/3053995795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b758edcc00 0x55b759054000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.430+0000 7fae63862740 1 -- 172.21.15.19:0/3053995795 >> 172.21.15.19:0/3053995795 conn(0x55b758edd000 msgr2=0x55b759c7a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:55.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.430+0000 7fae63862740 1 -- 172.21.15.19:0/3053995795 shutdown_connections 2024-03-28T13:17:55.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.430+0000 7fae63862740 1 -- 172.21.15.19:0/3053995795 wait complete. 2024-03-28T13:17:55.440 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.220336, result: 2024-03-28T13:17:55.441 DEBUG:teuthology.orchestra.run.smithi019:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-03-28T13:17:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.551+0000 7fe52244b740 1 -- start start 2024-03-28T13:17:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.551+0000 7fe52244b740 1 Processor -- start 2024-03-28T13:17:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.552+0000 7fe52244b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5597efccac00 0x5597f0a2a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.552+0000 7fe52244b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597efccb800 0x5597f0a2a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.552+0000 7fe52244b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597efccbc00 0x5597f0a2ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.552+0000 7fe52244b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5597f0a32180 con 0x5597efccbc00 2024-03-28T13:17:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.552+0000 7fe52244b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5597f0a32300 con 0x5597efccb800 2024-03-28T13:17:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.553+0000 7fe52244b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5597f0a32480 con 0x5597efccac00 2024-03-28T13:17:55.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.553+0000 7fe520a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597efccbc00 0x5597f0a2ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.553+0000 7fe520a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597efccbc00 0x5597f0a2ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45004/0 (socket says 172.21.15.19:45004) 2024-03-28T13:17:55.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.553+0000 7fe520a00640 1 -- 172.21.15.19:0/3311577009 learned_addr learned my addr 172.21.15.19:0/3311577009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:55.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.553+0000 7fe520000640 1 --2- 172.21.15.19:0/3311577009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5597efccac00 0x5597f0a2a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.553+0000 7fe51f600640 1 --2- 172.21.15.19:0/3311577009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597efccb800 0x5597f0a2a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.553+0000 7fe520a00640 1 -- 172.21.15.19:0/3311577009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5597efccac00 msgr2=0x5597f0a2a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:55.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.553+0000 7fe520a00640 1 --2- 172.21.15.19:0/3311577009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5597efccac00 0x5597f0a2a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.553+0000 7fe520a00640 1 -- 172.21.15.19:0/3311577009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597efccb800 msgr2=0x5597f0a2a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:55.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.553+0000 7fe520a00640 1 --2- 172.21.15.19:0/3311577009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597efccb800 0x5597f0a2a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.553+0000 7fe520a00640 1 -- 172.21.15.19:0/3311577009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597efcf7680 con 0x5597efccbc00 2024-03-28T13:17:55.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.554+0000 7fe520a00640 1 --2- 172.21.15.19:0/3311577009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597efccbc00 0x5597f0a2ab00 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x5597f0aefbc0 tx=0x5597f0a651a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d4dface00197c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.554+0000 7fe51ce00640 1 -- 172.21.15.19:0/3311577009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5597efcf0680 con 0x5597efccbc00 2024-03-28T13:17:55.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.554+0000 7fe51ce00640 1 -- 172.21.15.19:0/3311577009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5597efd141c0 con 0x5597efccbc00 2024-03-28T13:17:55.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.554+0000 7fe52244b740 1 -- 172.21.15.19:0/3311577009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597efcf7860 con 0x5597efccbc00 2024-03-28T13:17:55.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.554+0000 7fe51ce00640 1 -- 172.21.15.19:0/3311577009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5597efcf0b60 con 0x5597efccbc00 2024-03-28T13:17:55.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.554+0000 7fe52244b740 1 -- 172.21.15.19:0/3311577009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5597efcf72c0 con 0x5597efccbc00 2024-03-28T13:17:55.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.554+0000 7fe52244b740 4 waiting for MDS map... 2024-03-28T13:17:55.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.555+0000 7fe51ce00640 1 -- 172.21.15.19:0/3311577009 <== mon.0 v2:172.21.15.19:3300/0 4 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x5597f0ba2600 con 0x5597efccbc00 2024-03-28T13:17:55.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.555+0000 7fe51ce00640 1 -- 172.21.15.19:0/3311577009 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 69) ==== 2885+0+0 (secure 0 0 0) 0x5597efd23180 con 0x5597efccbc00 2024-03-28T13:17:55.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.555+0000 7fe52244b740 4 Got MDS map 69 2024-03-28T13:17:55.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.555+0000 7fe52244b740 4 datascan.main: Using metadata pool output 2024-03-28T13:17:55.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.555+0000 7fe52244b740 4 datascan.main: connecting to RADOS... 2024-03-28T13:17:55.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.556+0000 7fe52244b740 1 Processor -- start 2024-03-28T13:17:55.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.556+0000 7fe52244b740 1 -- start start 2024-03-28T13:17:55.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.557+0000 7fe52244b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5597f0ba5400 0x5597f0a2b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.557+0000 7fe52244b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597f0ba5800 0x5597f0a2b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.557+0000 7fe52244b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f0ba5c00 0x5597f0a2bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.557+0000 7fe52244b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5597f0a32600 con 0x5597f0ba5c00 2024-03-28T13:17:55.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.557+0000 7fe52244b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5597f0a32780 con 0x5597f0ba5800 2024-03-28T13:17:55.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.557+0000 7fe52244b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5597f0a32900 con 0x5597f0ba5400 2024-03-28T13:17:55.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.557+0000 7fe51f600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f0ba5c00 0x5597f0a2bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.557+0000 7fe51f600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f0ba5c00 0x5597f0a2bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45008/0 (socket says 172.21.15.19:45008) 2024-03-28T13:17:55.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.557+0000 7fe51f600640 1 -- 172.21.15.19:0/3419424222 learned_addr learned my addr 172.21.15.19:0/3419424222 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:55.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.557+0000 7fe51f600640 1 --2- 172.21.15.19:0/3419424222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5597f0ba5400 0x5597f0a2b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.557+0000 7fe520000640 1 --2- 172.21.15.19:0/3419424222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597f0ba5800 0x5597f0a2b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.558+0000 7fe51f600640 1 -- 172.21.15.19:0/3419424222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5597f0ba5400 msgr2=0x5597f0a2b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:55.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.558+0000 7fe51f600640 1 --2- 172.21.15.19:0/3419424222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5597f0ba5400 0x5597f0a2b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.558+0000 7fe51f600640 1 -- 172.21.15.19:0/3419424222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597f0ba5800 msgr2=0x5597f0a2b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:55.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.558+0000 7fe51f600640 1 --2- 172.21.15.19:0/3419424222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597f0ba5800 0x5597f0a2b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.558+0000 7fe51f600640 1 -- 172.21.15.19:0/3419424222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597efcf74a0 con 0x5597f0ba5c00 2024-03-28T13:17:55.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.558+0000 7fe520000640 1 --2- 172.21.15.19:0/3419424222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597f0ba5800 0x5597f0a2b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:17:55.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.558+0000 7fe51f600640 1 --2- 172.21.15.19:0/3419424222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f0ba5c00 0x5597f0a2bb80 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x5597f0bb0b10 tx=0x5597f0bb4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4123ab76154ac09 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.559+0000 7fe51a600640 1 -- 172.21.15.19:0/3419424222 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5597efcf0680 con 0x5597f0ba5c00 2024-03-28T13:17:55.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.559+0000 7fe51a600640 1 -- 172.21.15.19:0/3419424222 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5597efd14e00 con 0x5597f0ba5c00 2024-03-28T13:17:55.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.559+0000 7fe51a600640 1 -- 172.21.15.19:0/3419424222 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5597efcf0b60 con 0x5597f0ba5c00 2024-03-28T13:17:55.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.559+0000 7fe52244b740 1 -- 172.21.15.19:0/3419424222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f0ba5c00 msgr2=0x5597f0a2bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.559+0000 7fe52244b740 1 --2- 172.21.15.19:0/3419424222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f0ba5c00 0x5597f0a2bb80 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x5597f0bb0b10 tx=0x5597f0bb4c00 comp rx=0 tx=0).stop 2024-03-28T13:17:55.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.559+0000 7fe52244b740 1 -- 172.21.15.19:0/3419424222 shutdown_connections 2024-03-28T13:17:55.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.559+0000 7fe52244b740 1 --2- 172.21.15.19:0/3419424222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f0ba5c00 0x5597f0a2bb80 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.559+0000 7fe52244b740 1 --2- 172.21.15.19:0/3419424222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597f0ba5800 0x5597f0a2b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.559+0000 7fe52244b740 1 --2- 172.21.15.19:0/3419424222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5597f0ba5400 0x5597f0a2b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.559+0000 7fe52244b740 1 -- 172.21.15.19:0/3419424222 >> 172.21.15.19:0/3419424222 conn(0x5597f0ba5000 msgr2=0x5597efdb5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:55.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.559+0000 7fe52244b740 1 -- 172.21.15.19:0/3419424222 shutdown_connections 2024-03-28T13:17:55.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.559+0000 7fe52244b740 1 -- 172.21.15.19:0/3419424222 wait complete. 2024-03-28T13:17:55.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.560+0000 7fe52244b740 1 Processor -- start 2024-03-28T13:17:55.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.560+0000 7fe52244b740 1 -- start start 2024-03-28T13:17:55.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.561+0000 7fe52244b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f0ba5400 0x5597f0a2b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.561+0000 7fe52244b740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5597f0ba5800 0x5597f0a2b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.561+0000 7fe52244b740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597f0ba5c00 0x5597f0a2bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.561+0000 7fe52244b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5597f0a33c80 con 0x5597f0ba5400 2024-03-28T13:17:55.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.561+0000 7fe52244b740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5597f0a33b00 con 0x5597f0ba5c00 2024-03-28T13:17:55.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.561+0000 7fe52244b740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5597f0a33980 con 0x5597f0ba5800 2024-03-28T13:17:55.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.561+0000 7fe51f600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f0ba5400 0x5597f0a2b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.561+0000 7fe51f600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f0ba5400 0x5597f0a2b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45020/0 (socket says 172.21.15.19:45020) 2024-03-28T13:17:55.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.561+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 learned_addr learned my addr 172.21.15.19:0/3362584383 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.562+0000 7fe520000640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5597f0ba5800 0x5597f0a2b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.562+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5597f0ba5800 msgr2=0x5597f0a2b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.562+0000 7fe51f600640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5597f0ba5800 0x5597f0a2b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.562+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597f0ba5c00 msgr2=0x5597f0a2bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.562+0000 7fe51f600640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597f0ba5c00 0x5597f0a2bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.562+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597efcf7680 con 0x5597f0ba5400 2024-03-28T13:17:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.562+0000 7fe51f600640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f0ba5400 0x5597f0a2b080 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x5597f0bb1dd0 tx=0x5597f0a65140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.562+0000 7fe518800640 1 -- 172.21.15.19:0/3362584383 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5597efcf04e0 con 0x5597f0ba5400 2024-03-28T13:17:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.562+0000 7fe518800640 1 -- 172.21.15.19:0/3362584383 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5597efd14c40 con 0x5597f0ba5400 2024-03-28T13:17:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.562+0000 7fe518800640 1 -- 172.21.15.19:0/3362584383 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5597efcf0680 con 0x5597f0ba5400 2024-03-28T13:17:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.563+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5597efcf7860 con 0x5597f0ba5400 2024-03-28T13:17:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.563+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597efcf70e0 con 0x5597f0ba5400 2024-03-28T13:17:55.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.563+0000 7fe52244b740 4 datascan.init: resolving metadata pool 12 2024-03-28T13:17:55.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.564+0000 7fe518800640 1 -- 172.21.15.19:0/3362584383 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x5597f0bccc00 con 0x5597f0ba5400 2024-03-28T13:17:55.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.565+0000 7fe518800640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5597f0c1ac00 0x5597f0a2c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.565+0000 7fe518800640 1 -- 172.21.15.19:0/3362584383 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x5597f0c12a00 con 0x5597f0ba5400 2024-03-28T13:17:55.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.565+0000 7fe52244b740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-28T13:17:55.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.565+0000 7fe52244b740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-03-28T13:17:55.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.565+0000 7fe52244b740 4 datascan.main: opening data pool 'cephfs_data' 2024-03-28T13:17:55.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.565+0000 7fe52244b740 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x5597f0c1b000 0x5597f0a2c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.565+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5597f0c1b400 con 0x5597f0c1b000 2024-03-28T13:17:55.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.566+0000 7fe51f600640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x5597f0c1b000 0x5597f0a2c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.566+0000 7fe520000640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5597f0c1ac00 0x5597f0a2c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.566+0000 7fe51f600640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x5597f0c1b000 0x5597f0a2c680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.567+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.567+0000 7fe52244b740 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x5597f0c1b800 0x5597f0a2cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.567+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:2 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5597f0c1bc00 con 0x5597f0c1b800 2024-03-28T13:17:55.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.567+0000 7fe520a00640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x5597f0c1b800 0x5597f0a2cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.567+0000 7fe520000640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5597f0c1ac00 0x5597f0a2c100 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x5597f0a4a450 tx=0x5597f0a64ea0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.568+0000 7fe520a00640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x5597f0c1b800 0x5597f0a2cc00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.568+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.568+0000 7fe52244b740 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x5597f0062000 0x5597f0a2d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.568+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:3 13.0 13:00000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0062400 con 0x5597f0062000 2024-03-28T13:17:55.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.568+0000 7fe520000640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x5597f0062000 0x5597f0a2d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.569+0000 7fe520000640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x5597f0062000 0x5597f0a2d180 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.569+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.569+0000 7fe519c00640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x5597f0063800 0x5597f0068000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.570+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:4 13.20 13:04000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0063c00 con 0x5597f0063800 2024-03-28T13:17:55.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.570+0000 7fe51f600640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x5597f0063800 0x5597f0068000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.570+0000 7fe51f600640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x5597f0063800 0x5597f0068000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.571+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.571+0000 7fe51a600640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5597f006c000 0x5597f0068580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.571+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:5 13.10 13:08000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006c400 con 0x5597f006c000 2024-03-28T13:17:55.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.571+0000 7fe520a00640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5597f006c000 0x5597f0068580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.572+0000 7fe520a00640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5597f006c000 0x5597f0068580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.573+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.573+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:6 13.30 13:0c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006cc00 con 0x5597f0063800 2024-03-28T13:17:55.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.574+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.574+0000 7fe51a600640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5597f006c800 0x5597f0068b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.574+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:7 13.8 13:10000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006d800 con 0x5597f006c800 2024-03-28T13:17:55.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.574+0000 7fe520000640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5597f006c800 0x5597f0068b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.575+0000 7fe520000640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5597f006c800 0x5597f0068b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.576+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.5 v2:172.21.15.188:6800/3324247755 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f006c800 2024-03-28T13:17:55.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.576+0000 7fe519c00640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5597f006d000 0x5597f0069600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.576+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:8 13.28 13:14000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006d400 con 0x5597f006d000 2024-03-28T13:17:55.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.577+0000 7fe51f600640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5597f006d000 0x5597f0069600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.578+0000 7fe51f600640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5597f006d000 0x5597f0069600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.579+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.579+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:9 13.18 13:18000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006dc00 con 0x5597f0062000 2024-03-28T13:17:55.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.579+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.579+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:10 13.38 13:1c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0062400 con 0x5597f0c1b000 2024-03-28T13:17:55.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.580+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.580+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:11 13.4 13:20000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006d800 con 0x5597f0c1b800 2024-03-28T13:17:55.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.580+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.580+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:12 13.24 13:24000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0c1b400 con 0x5597f0062000 2024-03-28T13:17:55.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.581+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.581+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:13 13.14 13:28000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006dc00 con 0x5597f006d000 2024-03-28T13:17:55.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.581+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.581+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:14 13.34 13:2c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0abd400 con 0x5597f0c1b000 2024-03-28T13:17:55.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.582+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.582+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:15 13.c 13:30000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006cc00 con 0x5597f006d000 2024-03-28T13:17:55.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.583+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.583+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:16 13.2c 13:34000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0abd000 con 0x5597f006c000 2024-03-28T13:17:55.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.584+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 2 ==== osd_op_reply(16 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.584+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:17 13.1c 13:38000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0063c00 con 0x5597f0c1b000 2024-03-28T13:17:55.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.584+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 4 ==== osd_op_reply(17 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.584+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:18 13.3c 13:3c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0abd400 con 0x5597f006c800 2024-03-28T13:17:55.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.586+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.5 v2:172.21.15.188:6800/3324247755 2 ==== osd_op_reply(18 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f006c800 2024-03-28T13:17:55.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.586+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:19 13.2 13:40000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006dc00 con 0x5597f0062000 2024-03-28T13:17:55.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.586+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 4 ==== osd_op_reply(19 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.586+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:20 13.22 13:44000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0c1bc00 con 0x5597f006c000 2024-03-28T13:17:55.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.587+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.587+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:21 13.12 13:48000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006cc00 con 0x5597f006d000 2024-03-28T13:17:55.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.588+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 4 ==== osd_op_reply(21 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.588+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:22 13.32 13:4c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0ba4800 con 0x5597f006c000 2024-03-28T13:17:55.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.589+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 4 ==== osd_op_reply(22 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.590+0000 7fe51a600640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5597f006c400 0x5597f0069080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.590+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:23 13.a 13:50000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006d800 con 0x5597f006c400 2024-03-28T13:17:55.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.590+0000 7fe520a00640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5597f006c400 0x5597f0069080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.592+0000 7fe520a00640 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5597f006c400 0x5597f0069080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.592+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.4 v2:172.21.15.188:6808/1845695132 1 ==== osd_op_reply(23 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c400 2024-03-28T13:17:55.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.593+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:24 13.2a 13:54000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0062400 con 0x5597f006c000 2024-03-28T13:17:55.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.594+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 5 ==== osd_op_reply(24 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.594+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:25 13.1a 13:58000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0abd000 con 0x5597f006c800 2024-03-28T13:17:55.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.595+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.5 v2:172.21.15.188:6800/3324247755 3 ==== osd_op_reply(25 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f006c800 2024-03-28T13:17:55.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.595+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:26 13.3a 13:5c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0063c00 con 0x5597f0063800 2024-03-28T13:17:55.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.596+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 3 ==== osd_op_reply(26 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.596+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:27 13.6 13:60000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006d400 con 0x5597f0c1b800 2024-03-28T13:17:55.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.596+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 3 ==== osd_op_reply(27 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.597+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:28 13.26 13:64000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084000 con 0x5597f0c1b800 2024-03-28T13:17:55.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.597+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 4 ==== osd_op_reply(28 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.597+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:29 13.16 13:68000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084000 con 0x5597f006c000 2024-03-28T13:17:55.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.598+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 6 ==== osd_op_reply(29 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.598+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:30 13.36 13:6c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084400 con 0x5597f0063800 2024-03-28T13:17:55.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.599+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.599+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:31 13.e 13:70000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0085800 con 0x5597f0c1b000 2024-03-28T13:17:55.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.600+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 5 ==== osd_op_reply(31 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.600+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:32 13.2e 13:74000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084800 con 0x5597f006d000 2024-03-28T13:17:55.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.601+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.601+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:33 13.1e 13:78000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006d800 con 0x5597f0c1b000 2024-03-28T13:17:55.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.602+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 6 ==== osd_op_reply(33 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.602+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:34 13.3e 13:7c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084c00 con 0x5597f006c400 2024-03-28T13:17:55.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.603+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.4 v2:172.21.15.188:6808/1845695132 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c400 2024-03-28T13:17:55.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.603+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:35 13.1 13:80000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006d400 con 0x5597f0062000 2024-03-28T13:17:55.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.604+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 5 ==== osd_op_reply(35 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.604+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:36 13.21 13:84000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0085000 con 0x5597f0063800 2024-03-28T13:17:55.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.605+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 5 ==== osd_op_reply(36 [pgnls-filter start_epoch 94 out=49b] v92'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.605+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:37 13.11 13:88000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0062400 con 0x5597f006d000 2024-03-28T13:17:55.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.607+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 6 ==== osd_op_reply(37 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.607+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:38 13.31 13:8c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0085400 con 0x5597f006c400 2024-03-28T13:17:55.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.608+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.4 v2:172.21.15.188:6808/1845695132 3 ==== osd_op_reply(38 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c400 2024-03-28T13:17:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.608+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:39 13.9 13:90000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0ba4800 con 0x5597f0c1b000 2024-03-28T13:17:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.609+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 7 ==== osd_op_reply(39 [pgnls-filter start_epoch 94 out=49b] v92'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.609+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:40 13.29 13:94000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084400 con 0x5597f0c1b000 2024-03-28T13:17:55.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.609+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 8 ==== osd_op_reply(40 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.609+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:41 13.19 13:98000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0c1bc00 con 0x5597f006c000 2024-03-28T13:17:55.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.611+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 7 ==== osd_op_reply(41 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.611+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:42 13.39 13:9c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0085800 con 0x5597f006c400 2024-03-28T13:17:55.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.612+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.4 v2:172.21.15.188:6808/1845695132 4 ==== osd_op_reply(42 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c400 2024-03-28T13:17:55.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.612+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:43 13.5 13:a0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088000 con 0x5597f0063800 2024-03-28T13:17:55.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.613+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 6 ==== osd_op_reply(43 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.613+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:44 13.25 13:a4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084800 con 0x5597f0c1b800 2024-03-28T13:17:55.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.614+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.614+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:45 13.15 13:a8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088400 con 0x5597f0c1b800 2024-03-28T13:17:55.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.615+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 6 ==== osd_op_reply(45 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.615+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:46 13.35 13:ac000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0085c00 con 0x5597f006d000 2024-03-28T13:17:55.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.616+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 7 ==== osd_op_reply(46 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.616+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:47 13.d 13:b0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088800 con 0x5597f0c1b000 2024-03-28T13:17:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.617+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 9 ==== osd_op_reply(47 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.617+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:48 13.2d 13:b4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006cc00 con 0x5597f006c000 2024-03-28T13:17:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.618+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 8 ==== osd_op_reply(48 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.618+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:49 13.1d 13:b8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088c00 con 0x5597f0062000 2024-03-28T13:17:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.619+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 6 ==== osd_op_reply(49 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.619+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:50 13.3d 13:bc000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006d800 con 0x5597f0063800 2024-03-28T13:17:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.620+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 7 ==== osd_op_reply(50 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.620+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:51 13.3 13:c0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0089000 con 0x5597f006d000 2024-03-28T13:17:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.621+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 8 ==== osd_op_reply(51 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.621+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:52 13.23 13:c4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0063c00 con 0x5597f006c800 2024-03-28T13:17:55.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.623+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.5 v2:172.21.15.188:6800/3324247755 4 ==== osd_op_reply(52 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f006c800 2024-03-28T13:17:55.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.623+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:53 13.13 13:c8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0089400 con 0x5597f0062000 2024-03-28T13:17:55.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.624+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.624+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:54 13.33 13:cc000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0c1b400 con 0x5597f0c1b800 2024-03-28T13:17:55.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.625+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 7 ==== osd_op_reply(54 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.625+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:55 13.b 13:d0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0089800 con 0x5597f0c1b800 2024-03-28T13:17:55.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.625+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 8 ==== osd_op_reply(55 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.626+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:56 13.2b 13:d4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0c1bc00 con 0x5597f006c800 2024-03-28T13:17:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.627+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.5 v2:172.21.15.188:6800/3324247755 5 ==== osd_op_reply(56 [pgnls-filter start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f006c800 2024-03-28T13:17:55.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.627+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:57 13.1b 13:d8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0089c00 con 0x5597f0063800 2024-03-28T13:17:55.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.628+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 8 ==== osd_op_reply(57 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.628+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:58 13.3b 13:dc000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0ba4800 con 0x5597f0c1b800 2024-03-28T13:17:55.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.629+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 9 ==== osd_op_reply(58 [pgnls-filter start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.629+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:59 13.7 13:e0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088c00 con 0x5597f0c1b800 2024-03-28T13:17:55.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.630+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 10 ==== osd_op_reply(59 [pgnls-filter start_epoch 94 out=81b] v94'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.630+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:60 13.0 13:00000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0062800 con 0x5597f0062000 2024-03-28T13:17:55.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.631+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 8 ==== osd_op_reply(60 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.631+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:61 13.20 13:04000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088000 con 0x5597f0063800 2024-03-28T13:17:55.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.632+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 9 ==== osd_op_reply(61 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.632+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:62 13.10 13:08000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0062400 con 0x5597f006c000 2024-03-28T13:17:55.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.633+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.633+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:63 13.30 13:0c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088800 con 0x5597f0063800 2024-03-28T13:17:55.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.634+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 10 ==== osd_op_reply(63 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.634+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:64 13.8 13:10000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006dc00 con 0x5597f006c800 2024-03-28T13:17:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.635+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.5 v2:172.21.15.188:6800/3324247755 6 ==== osd_op_reply(64 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f006c800 2024-03-28T13:17:55.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.636+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:65 13.28 13:14000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088400 con 0x5597f006d000 2024-03-28T13:17:55.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.637+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 9 ==== osd_op_reply(65 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.637+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:66 13.18 13:18000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006d400 con 0x5597f0062000 2024-03-28T13:17:55.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.638+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 9 ==== osd_op_reply(66 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.638+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:67 13.38 13:1c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084000 con 0x5597f0c1b000 2024-03-28T13:17:55.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.639+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 10 ==== osd_op_reply(67 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.639+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:68 13.4 13:20000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0abd400 con 0x5597f0c1b800 2024-03-28T13:17:55.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.640+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.640+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:69 13.24 13:24000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084400 con 0x5597f0062000 2024-03-28T13:17:55.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.641+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 10 ==== osd_op_reply(69 [pgnls-filter start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.641+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:70 13.14 13:28000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0abd000 con 0x5597f006d000 2024-03-28T13:17:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.642+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 10 ==== osd_op_reply(70 [pgnls-filter start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.642+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:71 13.34 13:2c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084800 con 0x5597f0c1b000 2024-03-28T13:17:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.643+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 11 ==== osd_op_reply(71 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.643+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:72 13.c 13:30000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0063c00 con 0x5597f006d000 2024-03-28T13:17:55.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.644+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.644+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:73 13.2c 13:34000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084c00 con 0x5597f006c000 2024-03-28T13:17:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.645+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 10 ==== osd_op_reply(73 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.645+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0c1bc00 con 0x5597f0c1b000 2024-03-28T13:17:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.646+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 12 ==== osd_op_reply(74 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.646+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:75 13.3c 13:3c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0085000 con 0x5597f006c800 2024-03-28T13:17:55.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.647+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.5 v2:172.21.15.188:6800/3324247755 7 ==== osd_op_reply(75 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f006c800 2024-03-28T13:17:55.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.648+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:76 13.2 13:40000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0089000 con 0x5597f0062000 2024-03-28T13:17:55.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.648+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 11 ==== osd_op_reply(76 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.649+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:77 13.22 13:44000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0085400 con 0x5597f006c000 2024-03-28T13:17:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.650+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 11 ==== osd_op_reply(77 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.650+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:78 13.12 13:48000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0089c00 con 0x5597f006d000 2024-03-28T13:17:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.651+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 12 ==== osd_op_reply(78 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.651+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:79 13.32 13:4c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0085800 con 0x5597f006c000 2024-03-28T13:17:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.652+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 12 ==== osd_op_reply(79 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.652+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:80 13.a 13:50000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088000 con 0x5597f006c400 2024-03-28T13:17:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.653+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.4 v2:172.21.15.188:6808/1845695132 5 ==== osd_op_reply(80 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c400 2024-03-28T13:17:55.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.653+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:81 13.2a 13:54000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0085c00 con 0x5597f006c000 2024-03-28T13:17:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.655+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 13 ==== osd_op_reply(81 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.655+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:82 13.1a 13:58000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088800 con 0x5597f006c800 2024-03-28T13:17:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.657+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.5 v2:172.21.15.188:6800/3324247755 8 ==== osd_op_reply(82 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f006c800 2024-03-28T13:17:55.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.657+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:83 13.3a 13:5c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006cc00 con 0x5597f0063800 2024-03-28T13:17:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.657+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 11 ==== osd_op_reply(83 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.657+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:84 13.6 13:60000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088400 con 0x5597f0c1b800 2024-03-28T13:17:55.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.658+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.659+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:85 13.26 13:64000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006d800 con 0x5597f0c1b800 2024-03-28T13:17:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.660+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 13 ==== osd_op_reply(85 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.660+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:86 13.16 13:68000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0089400 con 0x5597f006c000 2024-03-28T13:17:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.661+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 14 ==== osd_op_reply(86 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.661+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:87 13.36 13:6c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006dc00 con 0x5597f0063800 2024-03-28T13:17:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.662+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 12 ==== osd_op_reply(87 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.662+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:88 13.e 13:70000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0abd000 con 0x5597f0c1b000 2024-03-28T13:17:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.663+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.663+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:89 13.2e 13:74000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006d400 con 0x5597f006d000 2024-03-28T13:17:55.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.665+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 13 ==== osd_op_reply(89 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.665+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:90 13.1e 13:78000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084c00 con 0x5597f0c1b000 2024-03-28T13:17:55.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.666+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 14 ==== osd_op_reply(90 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.666+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:91 13.3e 13:7c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0062400 con 0x5597f006c400 2024-03-28T13:17:55.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.667+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.4 v2:172.21.15.188:6808/1845695132 6 ==== osd_op_reply(91 [pgnls-filter start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c400 2024-03-28T13:17:55.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.667+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:92 13.1 13:80000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084000 con 0x5597f0062000 2024-03-28T13:17:55.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.668+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 12 ==== osd_op_reply(92 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.668+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:93 13.21 13:84000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0062800 con 0x5597f0063800 2024-03-28T13:17:55.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.669+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 13 ==== osd_op_reply(93 [pgnls-filter start_epoch 94 out=49b] v92'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.669+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:94 13.11 13:88000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084400 con 0x5597f006d000 2024-03-28T13:17:55.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.671+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 14 ==== osd_op_reply(94 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.671+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:95 13.31 13:8c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0abd400 con 0x5597f006c400 2024-03-28T13:17:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.672+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.4 v2:172.21.15.188:6808/1845695132 7 ==== osd_op_reply(95 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c400 2024-03-28T13:17:55.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.672+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:96 13.9 13:90000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006d800 con 0x5597f0c1b000 2024-03-28T13:17:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.673+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 15 ==== osd_op_reply(96 [pgnls-filter start_epoch 94 out=49b] v92'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.673+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:97 13.29 13:94000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0089800 con 0x5597f0c1b000 2024-03-28T13:17:55.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.674+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 16 ==== osd_op_reply(97 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.674+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:98 13.19 13:98000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006cc00 con 0x5597f006c000 2024-03-28T13:17:55.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.675+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 15 ==== osd_op_reply(98 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.675+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:99 13.39 13:9c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088c00 con 0x5597f006c400 2024-03-28T13:17:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.676+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.4 v2:172.21.15.188:6808/1845695132 8 ==== osd_op_reply(99 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c400 2024-03-28T13:17:55.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.677+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:100 13.5 13:a0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0abd400 con 0x5597f0063800 2024-03-28T13:17:55.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.678+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 14 ==== osd_op_reply(100 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.678+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:101 13.25 13:a4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0ba4800 con 0x5597f0c1b800 2024-03-28T13:17:55.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.679+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 14 ==== osd_op_reply(101 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.679+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:102 13.15 13:a8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0abd000 con 0x5597f0c1b800 2024-03-28T13:17:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.680+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 15 ==== osd_op_reply(102 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.680+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:103 13.35 13:ac000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0c1b400 con 0x5597f006d000 2024-03-28T13:17:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.681+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 15 ==== osd_op_reply(103 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.681+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:104 13.d 13:b0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088000 con 0x5597f0c1b000 2024-03-28T13:17:55.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.682+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 17 ==== osd_op_reply(104 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.682+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:105 13.2d 13:b4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0085400 con 0x5597f006c000 2024-03-28T13:17:55.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.683+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.7 v2:172.21.15.188:6824/861796513 16 ==== osd_op_reply(105 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f006c000 2024-03-28T13:17:55.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.683+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:106 13.1d 13:b8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088400 con 0x5597f0062000 2024-03-28T13:17:55.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.684+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 13 ==== osd_op_reply(106 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.684+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:107 13.3d 13:bc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0085800 con 0x5597f0063800 2024-03-28T13:17:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.685+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.685+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:108 13.3 13:c0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0089400 con 0x5597f006d000 2024-03-28T13:17:55.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.687+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 16 ==== osd_op_reply(108 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.687+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:109 13.23 13:c4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0085c00 con 0x5597f006c800 2024-03-28T13:17:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.688+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.5 v2:172.21.15.188:6800/3324247755 9 ==== osd_op_reply(109 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f006c800 2024-03-28T13:17:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.688+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:110 13.13 13:c8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0089c00 con 0x5597f0062000 2024-03-28T13:17:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.689+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 14 ==== osd_op_reply(110 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.689+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:111 13.33 13:cc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084800 con 0x5597f0c1b800 2024-03-28T13:17:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.690+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 16 ==== osd_op_reply(111 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.690+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:112 13.b 13:d0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0c1bc00 con 0x5597f0c1b800 2024-03-28T13:17:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.691+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 17 ==== osd_op_reply(112 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.691+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:113 13.2b 13:d4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0084c00 con 0x5597f006c800 2024-03-28T13:17:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.692+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.5 v2:172.21.15.188:6800/3324247755 10 ==== osd_op_reply(113 [pgnls-filter start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f006c800 2024-03-28T13:17:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.692+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:114 13.1b 13:d8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0062400 con 0x5597f0063800 2024-03-28T13:17:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.693+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 16 ==== osd_op_reply(114 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.693+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:115 13.3b 13:dc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006dc00 con 0x5597f0c1b800 2024-03-28T13:17:55.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.694+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 18 ==== osd_op_reply(115 [pgnls-filter start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.694+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:116 13.7 13:e0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0063c00 con 0x5597f0c1b800 2024-03-28T13:17:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.695+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 19 ==== osd_op_reply(116 [pgnls-filter start_epoch 94 out=81b] v94'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.695+0000 7fe52244b740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-03-28T13:17:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.695+0000 7fe52244b740 10 datascan.operator(): handling object 10000000001.0 2024-03-28T13:17:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.695+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:117 13.7 13:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5597f0063000 con 0x5597f0c1b800 2024-03-28T13:17:55.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.697+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 20 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.697+0000 7fe52244b740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-03-28T13:17:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.697+0000 7fe52244b740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-03-28T13:17:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.697+0000 7fe52244b740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-03-28T13:17:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.697+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:118 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5597f0063400 con 0x5597f006d000 2024-03-28T13:17:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.698+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 17 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.698+0000 7fe52244b740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-03-28T13:17:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.698+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5597f0c1bc00 con 0x5597f0c1b800 2024-03-28T13:17:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.699+0000 7fe520a00640 1 -- 172.21.15.19:0/3362584383 <== osd.3 v2:172.21.15.19:6819/3265515865 21 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=581b] v0'0 uv9 ondisk = 0) ==== 154+0+581 (crc 0 0 0) 0x5597f0a4c500 con 0x5597f0c1b800 2024-03-28T13:17:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.699+0000 7fe52244b740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-03-28T13:17:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.699+0000 7fe52244b740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-03-28T13:17:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.699+0000 7fe52244b740 20 datascan.get_frag_of: frag is * 2024-03-28T13:17:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.699+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:120 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5597f0abd000 con 0x5597f006d000 2024-03-28T13:17:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.700+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 18 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.700+0000 7fe52244b740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-03-28T13:17:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.701+0000 7fe52244b740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-03-28T13:17:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.701+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:121 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5597f0abd400 con 0x5597f006d000 2024-03-28T13:17:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.702+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 19 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.702+0000 7fe52244b740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-03-28T13:17:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.702+0000 7fe52244b740 20 datascan.read_dentry: file_head not found in result 2024-03-28T13:17:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.702+0000 7fe52244b740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-03-28T13:17:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.702+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:122 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x5597f006cc00 con 0x5597f006d000 2024-03-28T13:17:55.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.706+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 20 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v94'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.706+0000 7fe52244b740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-03-28T13:17:55.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.707+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:123 13.27 13:e4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006dc00 con 0x5597f006d000 2024-03-28T13:17:55.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.707+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.6 v2:172.21.15.188:6816/3597358155 21 ==== osd_op_reply(123 [pgnls-filter start_epoch 94 out=49b] v92'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f006d000 2024-03-28T13:17:55.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.708+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:124 13.17 13:e8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0063c00 con 0x5597f0062000 2024-03-28T13:17:55.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.709+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 15 ==== osd_op_reply(124 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.709+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:125 13.37 13:ec000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f006d400 con 0x5597f0062000 2024-03-28T13:17:55.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.709+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 16 ==== osd_op_reply(125 [pgnls-filter start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.710+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:126 13.f 13:f0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0063000 con 0x5597f0063800 2024-03-28T13:17:55.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.711+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.2 v2:172.21.15.19:6816/1693606238 17 ==== osd_op_reply(126 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0063800 2024-03-28T13:17:55.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.711+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:127 13.2f 13:f4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088000 con 0x5597f0c1b000 2024-03-28T13:17:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.712+0000 7fe51f600640 1 -- 172.21.15.19:0/3362584383 <== osd.0 v2:172.21.15.19:6800/4106483280 18 ==== osd_op_reply(127 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c000 con 0x5597f0c1b000 2024-03-28T13:17:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.712+0000 7fe519c00640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:128 13.1f 13:f8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0062800 con 0x5597f0062000 2024-03-28T13:17:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.712+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 17 ==== osd_op_reply(128 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.713+0000 7fe51a600640 1 -- 172.21.15.19:0/3362584383 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:129 13.3f 13:fc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5597f0088400 con 0x5597f0062000 2024-03-28T13:17:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.713+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 <== osd.1 v2:172.21.15.19:6801/1998544260 18 ==== osd_op_reply(129 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5597f0a4c780 con 0x5597f0062000 2024-03-28T13:17:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x5597f0c1b000 msgr2=0x5597f0a2c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x5597f0c1b000 0x5597f0a2c680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x5597f0062000 msgr2=0x5597f0a2d180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x5597f0062000 0x5597f0a2d180 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x5597f0063800 msgr2=0x5597f0068000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x5597f0063800 0x5597f0068000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x5597f0c1b800 msgr2=0x5597f0a2cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x5597f0c1b800 0x5597f0a2cc00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5597f006c400 msgr2=0x5597f0069080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5597f006c400 0x5597f0069080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5597f006c800 msgr2=0x5597f0068b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5597f006c800 0x5597f0068b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5597f006d000 msgr2=0x5597f0069600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5597f006d000 0x5597f0069600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5597f006c000 msgr2=0x5597f0068580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5597f006c000 0x5597f0068580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 reap_dead start 2024-03-28T13:17:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 reap_dead start 2024-03-28T13:17:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 reap_dead start 2024-03-28T13:17:55.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe520000640 1 -- 172.21.15.19:0/3362584383 reap_dead start 2024-03-28T13:17:55.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.714+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5597f0c1ac00 msgr2=0x5597f0a2c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.715+0000 7fe52244b740 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5597f0c1ac00 0x5597f0a2c100 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x5597f0a4a450 tx=0x5597f0a64ea0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.715+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f0ba5400 msgr2=0x5597f0a2b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.715+0000 7fe52244b740 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f0ba5400 0x5597f0a2b080 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x5597f0bb1dd0 tx=0x5597f0a65140 comp rx=0 tx=0).stop 2024-03-28T13:17:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.715+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 shutdown_connections 2024-03-28T13:17:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.715+0000 7fe52244b740 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5597f0c1ac00 0x5597f0a2c100 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.715+0000 7fe52244b740 1 --2- 172.21.15.19:0/3362584383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597f0ba5400 0x5597f0a2b080 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.715+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 >> 172.21.15.19:0/3362584383 conn(0x5597f0ba5000 msgr2=0x5597efdb5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:55.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.715+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 shutdown_connections 2024-03-28T13:17:55.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.715+0000 7fe52244b740 1 -- 172.21.15.19:0/3362584383 wait complete. 2024-03-28T13:17:55.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.716+0000 7fe52244b740 1 -- 172.21.15.19:0/3311577009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597efccbc00 msgr2=0x5597f0a2ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.716+0000 7fe52244b740 1 --2- 172.21.15.19:0/3311577009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597efccbc00 0x5597f0a2ab00 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x5597f0aefbc0 tx=0x5597f0a651a0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.717+0000 7fe52244b740 1 -- 172.21.15.19:0/3311577009 shutdown_connections 2024-03-28T13:17:55.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.717+0000 7fe52244b740 1 --2- 172.21.15.19:0/3311577009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5597efccbc00 0x5597f0a2ab00 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.717+0000 7fe52244b740 1 --2- 172.21.15.19:0/3311577009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5597efccb800 0x5597f0a2a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.717+0000 7fe52244b740 1 --2- 172.21.15.19:0/3311577009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5597efccac00 0x5597f0a2a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.717+0000 7fe52244b740 1 -- 172.21.15.19:0/3311577009 >> 172.21.15.19:0/3311577009 conn(0x5597efccb000 msgr2=0x5597efdb5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.717+0000 7fe52244b740 1 -- 172.21.15.19:0/3311577009 shutdown_connections 2024-03-28T13:17:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.717+0000 7fe52244b740 1 -- 172.21.15.19:0/3311577009 wait complete. 2024-03-28T13:17:55.731 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.290625, result: 2024-03-28T13:17:55.732 DEBUG:teuthology.orchestra.run.smithi019:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-03-28T13:17:55.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.842+0000 7f1d34445740 1 -- start start 2024-03-28T13:17:55.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.842+0000 7f1d34445740 1 Processor -- start 2024-03-28T13:17:55.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.843+0000 7f1d34445740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5589fc36d800 0x5589fd13e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.843+0000 7f1d34445740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fc36dc00 0x5589fd13eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.843+0000 7f1d34445740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15b400 0x5589fd13f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.843+0000 7f1d34445740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5589fd0d2180 con 0x5589fc36dc00 2024-03-28T13:17:55.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.843+0000 7f1d34445740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5589fd0d2300 con 0x5589fd15b400 2024-03-28T13:17:55.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.843+0000 7f1d34445740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5589fd0d2480 con 0x5589fc36d800 2024-03-28T13:17:55.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.843+0000 7f1d31600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fc36dc00 0x5589fd13eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.844+0000 7f1d31600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fc36dc00 0x5589fd13eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45036/0 (socket says 172.21.15.19:45036) 2024-03-28T13:17:55.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.844+0000 7f1d31600640 1 -- 172.21.15.19:0/3050914602 learned_addr learned my addr 172.21.15.19:0/3050914602 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:55.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.844+0000 7f1d32a00640 1 --2- 172.21.15.19:0/3050914602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15b400 0x5589fd13f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.844+0000 7f1d32000640 1 --2- 172.21.15.19:0/3050914602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5589fc36d800 0x5589fd13e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.844+0000 7f1d31600640 1 -- 172.21.15.19:0/3050914602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5589fc36d800 msgr2=0x5589fd13e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:55.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.844+0000 7f1d31600640 1 --2- 172.21.15.19:0/3050914602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5589fc36d800 0x5589fd13e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.844+0000 7f1d31600640 1 -- 172.21.15.19:0/3050914602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15b400 msgr2=0x5589fd13f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:55.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.844+0000 7f1d31600640 1 --2- 172.21.15.19:0/3050914602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15b400 0x5589fd13f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.844+0000 7f1d31600640 1 -- 172.21.15.19:0/3050914602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589fc399680 con 0x5589fc36dc00 2024-03-28T13:17:55.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.844+0000 7f1d32a00640 1 --2- 172.21.15.19:0/3050914602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15b400 0x5589fd13f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:17:55.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.844+0000 7f1d31600640 1 --2- 172.21.15.19:0/3050914602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fc36dc00 0x5589fd13eb00 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x5589fd18d8f0 tx=0x5589fd106120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a2dbd23793be63 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.845+0000 7f1d2ee00640 1 -- 172.21.15.19:0/3050914602 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5589fc392680 con 0x5589fc36dc00 2024-03-28T13:17:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.845+0000 7f1d2ee00640 1 -- 172.21.15.19:0/3050914602 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5589fc3b6540 con 0x5589fc36dc00 2024-03-28T13:17:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.845+0000 7f1d2ee00640 1 -- 172.21.15.19:0/3050914602 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5589fc392b60 con 0x5589fc36dc00 2024-03-28T13:17:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.845+0000 7f1d34445740 1 -- 172.21.15.19:0/3050914602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589fc399860 con 0x5589fc36dc00 2024-03-28T13:17:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.845+0000 7f1d34445740 1 -- 172.21.15.19:0/3050914602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5589fc3992c0 con 0x5589fc36dc00 2024-03-28T13:17:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.845+0000 7f1d34445740 4 waiting for MDS map... 2024-03-28T13:17:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.845+0000 7f1d2ee00640 1 -- 172.21.15.19:0/3050914602 <== mon.0 v2:172.21.15.19:3300/0 4 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x5589fd248600 con 0x5589fc36dc00 2024-03-28T13:17:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.845+0000 7f1d2ee00640 1 -- 172.21.15.19:0/3050914602 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 69) ==== 2885+0+0 (secure 0 0 0) 0x5589fc3c5180 con 0x5589fc36dc00 2024-03-28T13:17:55.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.845+0000 7f1d34445740 4 Got MDS map 69 2024-03-28T13:17:55.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.846+0000 7f1d34445740 4 datascan.main: Using metadata pool output 2024-03-28T13:17:55.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.846+0000 7f1d34445740 4 datascan.main: connecting to RADOS... 2024-03-28T13:17:55.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.846+0000 7f1d34445740 1 Processor -- start 2024-03-28T13:17:55.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.846+0000 7f1d34445740 1 -- start start 2024-03-28T13:17:55.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.847+0000 7f1d34445740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15a800 0x5589fd13f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.847+0000 7f1d34445740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5589fc36cc00 0x5589fd13fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.847+0000 7f1d34445740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fd251000 0x5589fd140100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.847+0000 7f1d34445740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5589fd0d2600 con 0x5589fd251000 2024-03-28T13:17:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.847+0000 7f1d34445740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5589fd0d2780 con 0x5589fd15a800 2024-03-28T13:17:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.847+0000 7f1d34445740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5589fd0d2900 con 0x5589fc36cc00 2024-03-28T13:17:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.848+0000 7f1d31600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fd251000 0x5589fd140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.848+0000 7f1d31600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fd251000 0x5589fd140100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45040/0 (socket says 172.21.15.19:45040) 2024-03-28T13:17:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.848+0000 7f1d31600640 1 -- 172.21.15.19:0/1742297027 learned_addr learned my addr 172.21.15.19:0/1742297027 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.848+0000 7f1d32000640 1 --2- 172.21.15.19:0/1742297027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5589fc36cc00 0x5589fd13fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.848+0000 7f1d31600640 1 -- 172.21.15.19:0/1742297027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5589fc36cc00 msgr2=0x5589fd13fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.848+0000 7f1d31600640 1 --2- 172.21.15.19:0/1742297027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5589fc36cc00 0x5589fd13fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.848+0000 7f1d32a00640 1 --2- 172.21.15.19:0/1742297027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15a800 0x5589fd13f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.848+0000 7f1d31600640 1 -- 172.21.15.19:0/1742297027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15a800 msgr2=0x5589fd13f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.848+0000 7f1d31600640 1 --2- 172.21.15.19:0/1742297027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15a800 0x5589fd13f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.848+0000 7f1d31600640 1 -- 172.21.15.19:0/1742297027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589fc3994a0 con 0x5589fd251000 2024-03-28T13:17:55.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.848+0000 7f1d32000640 1 --2- 172.21.15.19:0/1742297027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5589fc36cc00 0x5589fd13fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:55.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.848+0000 7f1d32a00640 1 --2- 172.21.15.19:0/1742297027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15a800 0x5589fd13f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:55.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.849+0000 7f1d31600640 1 --2- 172.21.15.19:0/1742297027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fd251000 0x5589fd140100 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x5589fd258c90 tx=0x5589fd104ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c62f93eda38533 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.851+0000 7f1d2c600640 1 -- 172.21.15.19:0/1742297027 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5589fc392820 con 0x5589fd251000 2024-03-28T13:17:55.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.851+0000 7f1d2c600640 1 -- 172.21.15.19:0/1742297027 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5589fc3b6e00 con 0x5589fd251000 2024-03-28T13:17:55.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.851+0000 7f1d2c600640 1 -- 172.21.15.19:0/1742297027 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5589fc392680 con 0x5589fd251000 2024-03-28T13:17:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.851+0000 7f1d34445740 1 -- 172.21.15.19:0/1742297027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fd251000 msgr2=0x5589fd140100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.851+0000 7f1d34445740 1 --2- 172.21.15.19:0/1742297027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fd251000 0x5589fd140100 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x5589fd258c90 tx=0x5589fd104ae0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.851+0000 7f1d34445740 1 -- 172.21.15.19:0/1742297027 shutdown_connections 2024-03-28T13:17:55.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.852+0000 7f1d34445740 1 --2- 172.21.15.19:0/1742297027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fd251000 0x5589fd140100 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.852+0000 7f1d34445740 1 --2- 172.21.15.19:0/1742297027 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5589fc36cc00 0x5589fd13fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.852+0000 7f1d34445740 1 --2- 172.21.15.19:0/1742297027 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15a800 0x5589fd13f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.852+0000 7f1d34445740 1 -- 172.21.15.19:0/1742297027 >> 172.21.15.19:0/1742297027 conn(0x5589fd15b000 msgr2=0x5589fc459000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.852+0000 7f1d34445740 1 -- 172.21.15.19:0/1742297027 shutdown_connections 2024-03-28T13:17:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.852+0000 7f1d34445740 1 -- 172.21.15.19:0/1742297027 wait complete. 2024-03-28T13:17:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.852+0000 7f1d34445740 1 Processor -- start 2024-03-28T13:17:55.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.853+0000 7f1d34445740 1 -- start start 2024-03-28T13:17:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.853+0000 7f1d34445740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15b000 0x5589fd141180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.853+0000 7f1d34445740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fc36cc00 0x5589fd140c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.853+0000 7f1d34445740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5589fd251c00 0x5589fd140680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.853+0000 7f1d34445740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5589fd0d3c80 con 0x5589fc36cc00 2024-03-28T13:17:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.853+0000 7f1d34445740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5589fd0d3b00 con 0x5589fd15b000 2024-03-28T13:17:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.853+0000 7f1d34445740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5589fd0d3980 con 0x5589fd251c00 2024-03-28T13:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.854+0000 7f1d32000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fc36cc00 0x5589fd140c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.854+0000 7f1d32000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fc36cc00 0x5589fd140c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45052/0 (socket says 172.21.15.19:45052) 2024-03-28T13:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.854+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 learned_addr learned my addr 172.21.15.19:0/4154463596 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.854+0000 7f1d31600640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5589fd251c00 0x5589fd140680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.854+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5589fd251c00 msgr2=0x5589fd140680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.854+0000 7f1d32000640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5589fd251c00 0x5589fd140680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.854+0000 7f1d32a00640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15b000 0x5589fd141180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.854+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15b000 msgr2=0x5589fd141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.854+0000 7f1d32000640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15b000 0x5589fd141180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.854+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589fc399680 con 0x5589fc36cc00 2024-03-28T13:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.855+0000 7f1d32000640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fc36cc00 0x5589fd140c00 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x5589fd18cff0 tx=0x5589fd1062a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.855+0000 7f1d2a800640 1 -- 172.21.15.19:0/4154463596 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5589fc3929c0 con 0x5589fc36cc00 2024-03-28T13:17:55.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.855+0000 7f1d2a800640 1 -- 172.21.15.19:0/4154463596 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5589fc3b6c40 con 0x5589fc36cc00 2024-03-28T13:17:55.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.855+0000 7f1d2a800640 1 -- 172.21.15.19:0/4154463596 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5589fc392820 con 0x5589fc36cc00 2024-03-28T13:17:55.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.855+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5589fc399860 con 0x5589fc36cc00 2024-03-28T13:17:55.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.855+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589fc3990e0 con 0x5589fc36cc00 2024-03-28T13:17:55.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.855+0000 7f1d34445740 4 datascan.init: resolving metadata pool 12 2024-03-28T13:17:55.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.857+0000 7f1d2a800640 1 -- 172.21.15.19:0/4154463596 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x5589fd2aac00 con 0x5589fc36cc00 2024-03-28T13:17:55.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.857+0000 7f1d2a800640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5589fd2b3400 0x5589fd141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.857+0000 7f1d2a800640 1 -- 172.21.15.19:0/4154463596 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x5589fd2b4600 con 0x5589fc36cc00 2024-03-28T13:17:55.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.858+0000 7f1d34445740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-28T13:17:55.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.858+0000 7f1d34445740 4 datascan.main: resolving metadata pool 12 2024-03-28T13:17:55.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.858+0000 7f1d34445740 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x5589fd251800 0x5589fd140100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.858+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fd251400 con 0x5589fd251800 2024-03-28T13:17:55.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.858+0000 7f1d32a00640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5589fd2b3400 0x5589fd141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.858+0000 7f1d31600640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x5589fd251800 0x5589fd140100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.859+0000 7f1d31600640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x5589fd251800 0x5589fd140100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.859+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.0 v2:172.21.15.19:6800/4106483280 1 ==== osd_op_reply(1 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd251800 2024-03-28T13:17:55.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.859+0000 7f1d31600640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5589fd251400 0x5589fc6e4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.859+0000 7f1d32a00640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5589fd2b3400 0x5589fd141700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x5589fd0ecdb0 tx=0x5589fd25b3e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.859+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fd250800 con 0x5589fd251400 2024-03-28T13:17:55.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.860+0000 7f1d32a00640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5589fd251400 0x5589fc6e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.861+0000 7f1d32a00640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5589fd251400 0x5589fc6e4000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.862+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 1 ==== osd_op_reply(2 [pgnls start_epoch 94 out=177b] v94'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.862+0000 7f1d32a00640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5589fd250800 0x5589fc6e4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.862+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ec000 con 0x5589fd250800 2024-03-28T13:17:55.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.863+0000 7f1d32000640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5589fd250800 0x5589fc6e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.864+0000 7f1d32000640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5589fd250800 0x5589fc6e4580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.865+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 1 ==== osd_op_reply(3 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.865+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ec000 con 0x5589fd250800 2024-03-28T13:17:55.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.865+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 2 ==== osd_op_reply(4 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.865+0000 7f1d32000640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x5589fc6ec000 0x5589fd13e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.865+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fd2b2c00 con 0x5589fc6ec000 2024-03-28T13:17:55.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.866+0000 7f1d31600640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x5589fc6ec000 0x5589fd13e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.866+0000 7f1d31600640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x5589fc6ec000 0x5589fd13e000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.867+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.2 v2:172.21.15.19:6816/1693606238 1 ==== osd_op_reply(5 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fc6ec000 2024-03-28T13:17:55.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.867+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fd2b2c00 con 0x5589fd251800 2024-03-28T13:17:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.867+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.0 v2:172.21.15.19:6800/4106483280 2 ==== osd_op_reply(6 [pgnls start_epoch 94 out=149b] v93'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd251800 2024-03-28T13:17:55.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.867+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fd2b2c00 con 0x5589fd250800 2024-03-28T13:17:55.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.868+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 3 ==== osd_op_reply(7 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.868+0000 7f1d32000640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5589fd2b2c00 0x5589fc6e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.868+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fd2b3000 con 0x5589fd2b2c00 2024-03-28T13:17:55.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.869+0000 7f1d32a00640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5589fd2b2c00 0x5589fc6e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.870+0000 7f1d32a00640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5589fd2b2c00 0x5589fc6e4b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.872+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.5 v2:172.21.15.188:6800/3324247755 1 ==== osd_op_reply(8 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd2b2c00 2024-03-28T13:17:55.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.872+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fd2b3000 con 0x5589fc6ec000 2024-03-28T13:17:55.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.872+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.2 v2:172.21.15.19:6816/1693606238 2 ==== osd_op_reply(9 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fc6ec000 2024-03-28T13:17:55.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.872+0000 7f1d31600640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5589fd2b3000 0x5589fc6e5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.872+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fd250400 con 0x5589fd2b3000 2024-03-28T13:17:55.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.873+0000 7f1d32000640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5589fd2b3000 0x5589fc6e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.874+0000 7f1d32000640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5589fd2b3000 0x5589fc6e5080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.875+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.4 v2:172.21.15.188:6808/1845695132 1 ==== osd_op_reply(10 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd2b3000 2024-03-28T13:17:55.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.875+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fd250400 con 0x5589fd2b3000 2024-03-28T13:17:55.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.876+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.4 v2:172.21.15.188:6808/1845695132 2 ==== osd_op_reply(11 [pgnls start_epoch 94 out=81b] v93'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd2b3000 2024-03-28T13:17:55.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.876+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fd250400 con 0x5589fc6ec000 2024-03-28T13:17:55.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.876+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.2 v2:172.21.15.19:6816/1693606238 3 ==== osd_op_reply(12 [pgnls start_epoch 94 out=49b] v93'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fc6ec000 2024-03-28T13:17:55.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.876+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fd250400 con 0x5589fd251400 2024-03-28T13:17:55.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.877+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 2 ==== osd_op_reply(13 [pgnls start_epoch 94 out=77b] v93'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.877+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fd250400 con 0x5589fc6ec000 2024-03-28T13:17:55.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.878+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.2 v2:172.21.15.19:6816/1693606238 4 ==== osd_op_reply(14 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fc6ec000 2024-03-28T13:17:55.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.878+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fd250400 con 0x5589fd250800 2024-03-28T13:17:55.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.879+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 4 ==== osd_op_reply(15 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.879+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fd250400 con 0x5589fc6ec000 2024-03-28T13:17:55.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.880+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.2 v2:172.21.15.19:6816/1693606238 5 ==== osd_op_reply(16 [pgnls start_epoch 94 out=49b] v93'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fc6ec000 2024-03-28T13:17:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.880+0000 7f1d31600640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x5589fd250400 0x5589fc6e5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.880+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ecc00 con 0x5589fd250400 2024-03-28T13:17:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.880+0000 7f1d31600640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x5589fd250400 0x5589fc6e5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.881+0000 7f1d31600640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x5589fd250400 0x5589fc6e5600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.881+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.1 v2:172.21.15.19:6801/1998544260 1 ==== osd_op_reply(17 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd250400 2024-03-28T13:17:55.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.881+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ecc00 con 0x5589fc6ec000 2024-03-28T13:17:55.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.882+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.2 v2:172.21.15.19:6816/1693606238 6 ==== osd_op_reply(18 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fc6ec000 2024-03-28T13:17:55.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.882+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ecc00 con 0x5589fd251400 2024-03-28T13:17:55.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.883+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 3 ==== osd_op_reply(19 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.883+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ecc00 con 0x5589fd250800 2024-03-28T13:17:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.884+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 5 ==== osd_op_reply(20 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.884+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ecc00 con 0x5589fd251800 2024-03-28T13:17:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.885+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.0 v2:172.21.15.19:6800/4106483280 3 ==== osd_op_reply(21 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd251800 2024-03-28T13:17:55.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.885+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ecc00 con 0x5589fd250400 2024-03-28T13:17:55.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.885+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.1 v2:172.21.15.19:6801/1998544260 2 ==== osd_op_reply(22 [pgnls start_epoch 94 out=97b] v93'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd250400 2024-03-28T13:17:55.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.885+0000 7f1d31600640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x5589fc6ecc00 0x5589fc6e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:55.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.885+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed000 con 0x5589fc6ecc00 2024-03-28T13:17:55.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.886+0000 7f1d32a00640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x5589fc6ecc00 0x5589fc6e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:55.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.886+0000 7f1d32a00640 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x5589fc6ecc00 0x5589fc6e5b80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:55.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.887+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.3 v2:172.21.15.19:6819/3265515865 1 ==== osd_op_reply(23 [pgnls start_epoch 94 out=74b] v91'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fc6ecc00 2024-03-28T13:17:55.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.887+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed000 con 0x5589fd251400 2024-03-28T13:17:55.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.888+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 4 ==== osd_op_reply(24 [pgnls start_epoch 94 out=73b] v93'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.888+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed000 con 0x5589fd2b3000 2024-03-28T13:17:55.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.889+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.4 v2:172.21.15.188:6808/1845695132 3 ==== osd_op_reply(25 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd2b3000 2024-03-28T13:17:55.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.889+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed000 con 0x5589fd2b2c00 2024-03-28T13:17:55.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.890+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.5 v2:172.21.15.188:6800/3324247755 2 ==== osd_op_reply(26 [pgnls start_epoch 94 out=98b] v93'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd2b2c00 2024-03-28T13:17:55.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.890+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed000 con 0x5589fd250800 2024-03-28T13:17:55.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.891+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 6 ==== osd_op_reply(27 [pgnls start_epoch 94 out=76b] v93'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.891+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed000 con 0x5589fd251800 2024-03-28T13:17:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.892+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.0 v2:172.21.15.19:6800/4106483280 4 ==== osd_op_reply(28 [pgnls start_epoch 94 out=73b] v93'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd251800 2024-03-28T13:17:55.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.892+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed000 con 0x5589fd250800 2024-03-28T13:17:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.893+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 7 ==== osd_op_reply(29 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.893+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed000 con 0x5589fd250400 2024-03-28T13:17:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.893+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.1 v2:172.21.15.19:6801/1998544260 3 ==== osd_op_reply(30 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd250400 2024-03-28T13:17:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.893+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed000 con 0x5589fc6ecc00 2024-03-28T13:17:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.894+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.3 v2:172.21.15.19:6819/3265515865 2 ==== osd_op_reply(31 [pgnls start_epoch 94 out=73b] v91'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fc6ecc00 2024-03-28T13:17:55.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.894+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed000 con 0x5589fc6ecc00 2024-03-28T13:17:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.894+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.3 v2:172.21.15.19:6819/3265515865 3 ==== osd_op_reply(32 [pgnls start_epoch 94 out=101b] v93'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fc6ecc00 2024-03-28T13:17:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.894+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-03-28T13:17:55.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.894+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fd251000 con 0x5589fd251400 2024-03-28T13:17:55.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.895+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.896+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-03-28T13:17:55.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.896+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:34 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6edc00 con 0x5589fd251400 2024-03-28T13:17:55.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.896+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.896+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-03-28T13:17:55.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.897+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:35 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ec800 con 0x5589fd251400 2024-03-28T13:17:55.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.897+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.898+0000 7f1d34445740 20 datascan.scan_links: injected ino detected 2024-03-28T13:17:55.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.898+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-03-28T13:17:55.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.898+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:36 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ec400 con 0x5589fd251400 2024-03-28T13:17:55.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.899+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.899+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-03-28T13:17:55.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.899+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:37 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed000 con 0x5589fd251400 2024-03-28T13:17:55.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.900+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.900+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-03-28T13:17:55.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.900+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-28T13:17:55.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.900+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-03-28T13:17:55.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.900+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-28T13:17:55.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.900+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-03-28T13:17:55.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.900+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-28T13:17:55.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.900+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-03-28T13:17:55.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.900+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:38 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6edc00 con 0x5589fd251800 2024-03-28T13:17:55.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.900+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.0 v2:172.21.15.19:6800/4106483280 5 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd251800 2024-03-28T13:17:55.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.900+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-03-28T13:17:55.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.900+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:39 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fd251000 con 0x5589fd2b2c00 2024-03-28T13:17:55.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.901+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.5 v2:172.21.15.188:6800/3324247755 3 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd2b2c00 2024-03-28T13:17:55.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.902+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-03-28T13:17:55.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.902+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:40 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc712000 con 0x5589fd2b3000 2024-03-28T13:17:55.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.903+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.4 v2:172.21.15.188:6808/1845695132 4 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd2b3000 2024-03-28T13:17:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.903+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-03-28T13:17:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.903+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-28T13:17:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.903+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-03-28T13:17:55.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.903+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:41 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc712400 con 0x5589fd250800 2024-03-28T13:17:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.904+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 8 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.904+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-03-28T13:17:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.904+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:42 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc712800 con 0x5589fd250400 2024-03-28T13:17:55.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.905+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.1 v2:172.21.15.19:6801/1998544260 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd250400 2024-03-28T13:17:55.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.905+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-03-28T13:17:55.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.905+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-28T13:17:55.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.905+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-03-28T13:17:55.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.905+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-28T13:17:55.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.905+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-03-28T13:17:55.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.905+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:43 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc712c00 con 0x5589fd251400 2024-03-28T13:17:55.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.906+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 10 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.906+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-03-28T13:17:55.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.906+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-28T13:17:55.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.906+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-03-28T13:17:55.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.906+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:44 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713000 con 0x5589fd2b2c00 2024-03-28T13:17:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.908+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.5 v2:172.21.15.188:6800/3324247755 4 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd2b2c00 2024-03-28T13:17:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.908+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-03-28T13:17:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.908+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-28T13:17:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.908+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-03-28T13:17:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.908+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-03-28T13:17:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.908+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-03-28T13:17:55.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.908+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:45 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713400 con 0x5589fd250800 2024-03-28T13:17:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.909+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 9 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.909+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-03-28T13:17:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.909+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-28T13:17:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.909+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-03-28T13:17:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.909+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-28T13:17:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.909+0000 7f1d34445740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-03-28T13:17:55.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.909+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:46 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713800 con 0x5589fc6ecc00 2024-03-28T13:17:55.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.910+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.3 v2:172.21.15.19:6819/3265515865 4 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fc6ecc00 2024-03-28T13:17:55.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.910+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:47 12.0 12:00000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd251800 2024-03-28T13:17:55.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.911+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.0 v2:172.21.15.19:6800/4106483280 6 ==== osd_op_reply(47 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd251800 2024-03-28T13:17:55.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.911+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:48 12.10 12:08000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd251400 2024-03-28T13:17:55.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.913+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 11 ==== osd_op_reply(48 [pgnls start_epoch 94 out=177b] v94'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.913+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:49 12.8 12:10000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd250800 2024-03-28T13:17:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.914+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 10 ==== osd_op_reply(49 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.914+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:50 12.18 12:18000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd250800 2024-03-28T13:17:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.915+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 11 ==== osd_op_reply(50 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.915+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:51 12.4 12:20000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fc6ec000 2024-03-28T13:17:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.915+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.2 v2:172.21.15.19:6816/1693606238 7 ==== osd_op_reply(51 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fc6ec000 2024-03-28T13:17:55.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.915+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:52 12.14 12:28000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd251800 2024-03-28T13:17:55.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.916+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.0 v2:172.21.15.19:6800/4106483280 7 ==== osd_op_reply(52 [pgnls start_epoch 94 out=149b] v93'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd251800 2024-03-28T13:17:55.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.916+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:53 12.c 12:30000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd250800 2024-03-28T13:17:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.917+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 12 ==== osd_op_reply(53 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.917+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:54 12.1c 12:38000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd2b2c00 2024-03-28T13:17:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.919+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.5 v2:172.21.15.188:6800/3324247755 5 ==== osd_op_reply(54 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd2b2c00 2024-03-28T13:17:55.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.919+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:55 12.2 12:40000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fc6ec000 2024-03-28T13:17:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.920+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.2 v2:172.21.15.19:6816/1693606238 8 ==== osd_op_reply(55 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fc6ec000 2024-03-28T13:17:55.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.920+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:56 12.12 12:48000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd2b3000 2024-03-28T13:17:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.922+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.4 v2:172.21.15.188:6808/1845695132 5 ==== osd_op_reply(56 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd2b3000 2024-03-28T13:17:55.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.922+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:57 12.a 12:50000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd2b3000 2024-03-28T13:17:55.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.923+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.4 v2:172.21.15.188:6808/1845695132 6 ==== osd_op_reply(57 [pgnls start_epoch 94 out=81b] v93'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd2b3000 2024-03-28T13:17:55.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.923+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:58 12.1a 12:58000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fc6ec000 2024-03-28T13:17:55.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.923+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.2 v2:172.21.15.19:6816/1693606238 9 ==== osd_op_reply(58 [pgnls start_epoch 94 out=49b] v93'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fc6ec000 2024-03-28T13:17:55.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.923+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:59 12.6 12:60000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd251400 2024-03-28T13:17:55.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.925+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 12 ==== osd_op_reply(59 [pgnls start_epoch 94 out=77b] v93'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.925+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:60 12.16 12:68000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fc6ec000 2024-03-28T13:17:55.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.925+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.2 v2:172.21.15.19:6816/1693606238 10 ==== osd_op_reply(60 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fc6ec000 2024-03-28T13:17:55.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.925+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:61 12.e 12:70000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd250800 2024-03-28T13:17:55.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.926+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 13 ==== osd_op_reply(61 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.926+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:62 12.1e 12:78000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fc6ec000 2024-03-28T13:17:55.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.927+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.2 v2:172.21.15.19:6816/1693606238 11 ==== osd_op_reply(62 [pgnls start_epoch 94 out=49b] v93'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fc6ec000 2024-03-28T13:17:55.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.927+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:63 12.1 12:80000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd250400 2024-03-28T13:17:55.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.928+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.1 v2:172.21.15.19:6801/1998544260 5 ==== osd_op_reply(63 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd250400 2024-03-28T13:17:55.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.928+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:64 12.11 12:88000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fc6ec000 2024-03-28T13:17:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.929+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.2 v2:172.21.15.19:6816/1693606238 12 ==== osd_op_reply(64 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fc6ec000 2024-03-28T13:17:55.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.929+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:65 12.9 12:90000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd251400 2024-03-28T13:17:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.930+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 13 ==== osd_op_reply(65 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.930+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:66 12.19 12:98000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd250800 2024-03-28T13:17:55.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.932+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 14 ==== osd_op_reply(66 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.932+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:67 12.5 12:a0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd251800 2024-03-28T13:17:55.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.933+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.0 v2:172.21.15.19:6800/4106483280 8 ==== osd_op_reply(67 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd251800 2024-03-28T13:17:55.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.933+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:68 12.15 12:a8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd250400 2024-03-28T13:17:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.934+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.1 v2:172.21.15.19:6801/1998544260 6 ==== osd_op_reply(68 [pgnls start_epoch 94 out=97b] v93'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd250400 2024-03-28T13:17:55.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.934+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:69 12.d 12:b0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fc6ecc00 2024-03-28T13:17:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.935+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.3 v2:172.21.15.19:6819/3265515865 5 ==== osd_op_reply(69 [pgnls start_epoch 94 out=74b] v91'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fc6ecc00 2024-03-28T13:17:55.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.935+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:70 12.1d 12:b8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd251400 2024-03-28T13:17:55.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.936+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 14 ==== osd_op_reply(70 [pgnls start_epoch 94 out=73b] v93'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.936+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:71 12.3 12:c0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd2b3000 2024-03-28T13:17:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.937+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.4 v2:172.21.15.188:6808/1845695132 7 ==== osd_op_reply(71 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd2b3000 2024-03-28T13:17:55.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.937+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:72 12.13 12:c8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd2b2c00 2024-03-28T13:17:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.938+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.5 v2:172.21.15.188:6800/3324247755 6 ==== osd_op_reply(72 [pgnls start_epoch 94 out=98b] v93'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd2b2c00 2024-03-28T13:17:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.939+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:73 12.b 12:d0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd250800 2024-03-28T13:17:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.940+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 15 ==== osd_op_reply(73 [pgnls start_epoch 94 out=76b] v93'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.940+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:74 12.1b 12:d8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd251800 2024-03-28T13:17:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.941+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.0 v2:172.21.15.19:6800/4106483280 9 ==== osd_op_reply(74 [pgnls start_epoch 94 out=73b] v93'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd251800 2024-03-28T13:17:55.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.941+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:75 12.7 12:e0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd250800 2024-03-28T13:17:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.942+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 16 ==== osd_op_reply(75 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.942+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:76 12.17 12:e8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd250400 2024-03-28T13:17:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.943+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.1 v2:172.21.15.19:6801/1998544260 7 ==== osd_op_reply(76 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd250400 2024-03-28T13:17:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.943+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:77 12.f 12:f0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fc6ecc00 2024-03-28T13:17:55.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.944+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.3 v2:172.21.15.19:6819/3265515865 6 ==== osd_op_reply(77 [pgnls start_epoch 94 out=73b] v91'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fc6ecc00 2024-03-28T13:17:55.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.944+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:78 12.1f 12:f8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fc6ecc00 2024-03-28T13:17:55.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.945+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.3 v2:172.21.15.19:6819/3265515865 7 ==== osd_op_reply(78 [pgnls start_epoch 94 out=101b] v93'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fc6ecc00 2024-03-28T13:17:55.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.945+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-03-28T13:17:55.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.945+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:79 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc712c00 con 0x5589fd251400 2024-03-28T13:17:55.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.947+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 15 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.947+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-03-28T13:17:55.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.947+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:80 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713000 con 0x5589fd251400 2024-03-28T13:17:55.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.948+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 16 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.948+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-03-28T13:17:55.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.948+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:81 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fd251000 con 0x5589fd251400 2024-03-28T13:17:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.949+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 17 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.949+0000 7f1d34445740 20 datascan.scan_links: injected ino detected 2024-03-28T13:17:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.949+0000 7f1d34445740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-03-28T13:17:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.949+0000 7f1d34445740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-03-28T13:17:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.949+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-03-28T13:17:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.949+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:82 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ec800 con 0x5589fd251400 2024-03-28T13:17:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.950+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 18 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.950+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-03-28T13:17:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.950+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:83 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ec400 con 0x5589fd251400 2024-03-28T13:17:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.951+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 19 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.952+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-03-28T13:17:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.952+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-28T13:17:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.952+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-03-28T13:17:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.952+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-28T13:17:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.952+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-03-28T13:17:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.952+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-28T13:17:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.952+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-03-28T13:17:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.952+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:84 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed000 con 0x5589fd251800 2024-03-28T13:17:55.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.953+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.0 v2:172.21.15.19:6800/4106483280 10 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd251800 2024-03-28T13:17:55.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.953+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-03-28T13:17:55.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.953+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:85 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ec800 con 0x5589fd2b2c00 2024-03-28T13:17:55.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.954+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.5 v2:172.21.15.188:6800/3324247755 7 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd2b2c00 2024-03-28T13:17:55.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.954+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-03-28T13:17:55.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.954+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:86 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ec400 con 0x5589fd2b3000 2024-03-28T13:17:55.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.955+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.4 v2:172.21.15.188:6808/1845695132 8 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd2b3000 2024-03-28T13:17:55.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.955+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-03-28T13:17:55.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.955+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-28T13:17:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.955+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-03-28T13:17:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.955+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:87 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fd251000 con 0x5589fd250800 2024-03-28T13:17:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.956+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.956+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-03-28T13:17:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.956+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:88 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713800 con 0x5589fd250400 2024-03-28T13:17:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.957+0000 7f1d31600640 1 -- 172.21.15.19:0/4154463596 <== osd.1 v2:172.21.15.19:6801/1998544260 8 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee000 con 0x5589fd250400 2024-03-28T13:17:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.958+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-03-28T13:17:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.958+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-28T13:17:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.958+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-03-28T13:17:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.958+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-28T13:17:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.958+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-03-28T13:17:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.958+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713c00 con 0x5589fd251400 2024-03-28T13:17:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.959+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 20 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.959+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-03-28T13:17:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.959+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-28T13:17:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.959+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-03-28T13:17:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.959+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:90 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc712c00 con 0x5589fd2b2c00 2024-03-28T13:17:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.961+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.5 v2:172.21.15.188:6800/3324247755 8 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd2b2c00 2024-03-28T13:17:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.961+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-03-28T13:17:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.961+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-28T13:17:55.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.961+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-03-28T13:17:55.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.961+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-03-28T13:17:55.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.961+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-03-28T13:17:55.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.961+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:91 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713000 con 0x5589fd250800 2024-03-28T13:17:55.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.962+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 <== osd.7 v2:172.21.15.188:6824/861796513 18 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5589fd0ee780 con 0x5589fd250800 2024-03-28T13:17:55.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.962+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-03-28T13:17:55.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.962+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-28T13:17:55.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.962+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-03-28T13:17:55.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.962+0000 7f1d34445740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-28T13:17:55.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.962+0000 7f1d34445740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-03-28T13:17:55.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.962+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:92 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc712800 con 0x5589fc6ecc00 2024-03-28T13:17:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.963+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.3 v2:172.21.15.19:6819/3265515865 8 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fc6ecc00 2024-03-28T13:17:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.963+0000 7f1d34445740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-03-28T13:17:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.963+0000 7f1d34445740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-03-28T13:17:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.963+0000 7f1d34445740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-03-28T13:17:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.963+0000 7f1d34445740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-03-28T13:17:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.963+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:93 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc713800 con 0x5589fd251400 2024-03-28T13:17:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.965+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 21 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.965+0000 7f1d34445740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-03-28T13:17:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.965+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:94 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed800 con 0x5589fd251400 2024-03-28T13:17:55.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.969+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 22 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v94'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.969+0000 7f1d34445740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-03-28T13:17:55.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.969+0000 7f1d34445740 10 datascan.scan_links: processing 1 injected_inos 2024-03-28T13:17:55.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.969+0000 7f1d34445740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-03-28T13:17:55.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.969+0000 7f1d34445740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-03-28T13:17:55.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.969+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:95 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed400 con 0x5589fd251400 2024-03-28T13:17:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.970+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 23 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.970+0000 7f1d34445740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-03-28T13:17:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.970+0000 7f1d34445740 20 datascan.scan_links: first is now 2 2024-03-28T13:17:55.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.970+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:96 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6edc00 con 0x5589fd251400 2024-03-28T13:17:55.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.974+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.6 v2:172.21.15.188:6816/3597358155 24 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v94'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5589fd0ee500 con 0x5589fd251400 2024-03-28T13:17:55.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.975+0000 7f1d34445740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-03-28T13:17:55.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.975+0000 7f1d34445740 10 datascan.scan_links: updating inotable 2024-03-28T13:17:55.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.975+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed000 con 0x5589fd2b2c00 2024-03-28T13:17:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.976+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.5 v2:172.21.15.188:6800/3324247755 9 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x5589fd0ee280 con 0x5589fd2b2c00 2024-03-28T13:17:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.976+0000 7f1d34445740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-03-28T13:17:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.976+0000 7f1d34445740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-03-28T13:17:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.976+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ec800 con 0x5589fd2b2c00 2024-03-28T13:17:55.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.981+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.5 v2:172.21.15.188:6800/3324247755 10 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v94'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x5589fd0ee280 con 0x5589fd2b2c00 2024-03-28T13:17:55.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.981+0000 7f1d34445740 10 datascan.scan_links: updating snaptable 2024-03-28T13:17:55.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.981+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:99 12.d 12:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) -- 0x5589fc6ed800 con 0x5589fc6ecc00 2024-03-28T13:17:55.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d32a00640 1 -- 172.21.15.19:0/4154463596 <== osd.3 v2:172.21.15.19:6819/3265515865 9 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x5589fd0ee280 con 0x5589fc6ecc00 2024-03-28T13:17:55.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x5589fd251800 msgr2=0x5589fd140100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x5589fd251800 0x5589fd140100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x5589fd250400 msgr2=0x5589fc6e5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x5589fd250400 0x5589fc6e5600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x5589fc6ec000 msgr2=0x5589fd13e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x5589fc6ec000 0x5589fd13e000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x5589fc6ecc00 msgr2=0x5589fc6e5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x5589fc6ecc00 0x5589fc6e5b80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5589fd2b3000 msgr2=0x5589fc6e5080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5589fd2b3000 0x5589fc6e5080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 reap_dead start 2024-03-28T13:17:55.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5589fd2b2c00 msgr2=0x5589fc6e4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5589fd2b2c00 0x5589fc6e4b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5589fd251400 msgr2=0x5589fc6e4000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5589fd251400 0x5589fc6e4000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5589fd250800 msgr2=0x5589fc6e4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.982+0000 7f1d34445740 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5589fd250800 0x5589fc6e4580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.983+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5589fd2b3400 msgr2=0x5589fd141700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.983+0000 7f1d34445740 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5589fd2b3400 0x5589fd141700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x5589fd0ecdb0 tx=0x5589fd25b3e0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.983+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fc36cc00 msgr2=0x5589fd140c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.983+0000 7f1d34445740 1 --2- 172.21.15.19:0/4154463596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fc36cc00 0x5589fd140c00 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x5589fd18cff0 tx=0x5589fd1062a0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.983+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 reap_dead start 2024-03-28T13:17:55.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.983+0000 7f1d32000640 1 -- 172.21.15.19:0/4154463596 reap_dead start 2024-03-28T13:17:55.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.983+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 shutdown_connections 2024-03-28T13:17:55.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.983+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 >> 172.21.15.19:0/4154463596 conn(0x5589fd15a800 msgr2=0x5589fc459000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:55.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.984+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 shutdown_connections 2024-03-28T13:17:55.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.984+0000 7f1d34445740 1 -- 172.21.15.19:0/4154463596 wait complete. 2024-03-28T13:17:55.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.985+0000 7f1d34445740 1 -- 172.21.15.19:0/3050914602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fc36dc00 msgr2=0x5589fd13eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:55.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.985+0000 7f1d34445740 1 --2- 172.21.15.19:0/3050914602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fc36dc00 0x5589fd13eb00 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x5589fd18d8f0 tx=0x5589fd106120 comp rx=0 tx=0).stop 2024-03-28T13:17:55.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.985+0000 7f1d34445740 1 -- 172.21.15.19:0/3050914602 shutdown_connections 2024-03-28T13:17:55.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.985+0000 7f1d34445740 1 --2- 172.21.15.19:0/3050914602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5589fd15b400 0x5589fd13f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.985+0000 7f1d34445740 1 --2- 172.21.15.19:0/3050914602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589fc36dc00 0x5589fd13eb00 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.985+0000 7f1d34445740 1 --2- 172.21.15.19:0/3050914602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5589fc36d800 0x5589fd13e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:55.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.985+0000 7f1d34445740 1 -- 172.21.15.19:0/3050914602 >> 172.21.15.19:0/3050914602 conn(0x5589fc36d000 msgr2=0x5589fc458800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:55.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.986+0000 7f1d34445740 1 -- 172.21.15.19:0/3050914602 shutdown_connections 2024-03-28T13:17:55.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:55.986+0000 7f1d34445740 1 -- 172.21.15.19:0/3050914602 wait complete. 2024-03-28T13:17:55.998 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.266986, result: 2024-03-28T13:17:55.999 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-28T13:17:56.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.237+0000 7f15aa000640 1 Processor -- start 2024-03-28T13:17:56.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.237+0000 7f15aa000640 1 -- start start 2024-03-28T13:17:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.239+0000 7f15aa000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f15a40d0990 0x7f15a40aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.239+0000 7f15aa000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f15a40af2e0 0x7f15a41600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.239+0000 7f15aa000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15a40afab0 con 0x7f15a4153580 2024-03-28T13:17:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.239+0000 7f15aa000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f15a415cc80 con 0x7f15a40af2e0 2024-03-28T13:17:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.239+0000 7f15aa000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f15a415da00 con 0x7f15a40d0990 2024-03-28T13:17:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.239+0000 7f15a3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f15a4153580 0x7f15a40d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37336/0 (socket says 172.21.15.19:37336) 2024-03-28T13:17:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.239+0000 7f15a3400640 1 -- 172.21.15.19:0/195864238 learned_addr learned my addr 172.21.15.19:0/195864238 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:17:56.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.239+0000 7f15a3e00640 1 --2- 172.21.15.19:0/195864238 >> v2:172.21.15.19:3300/0 conn(0x7f15a40af2e0 0x7f15a41600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.240+0000 7f159be00640 1 --2- 172.21.15.19:0/195864238 >> v2:172.21.15.188:3300/0 conn(0x7f15a40d0990 0x7f15a40aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.240+0000 7f15a2a00640 1 -- 172.21.15.19:0/195864238 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3189676550 0 0) 0x7f15a40afab0 con 0x7f15a4153580 2024-03-28T13:17:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.240+0000 7f15a3e00640 1 -- 172.21.15.19:0/195864238 >> v2:172.21.15.188:3300/0 conn(0x7f15a40d0990 msgr2=0x7f15a40aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.240+0000 7f15a3e00640 1 --2- 172.21.15.19:0/195864238 >> v2:172.21.15.188:3300/0 conn(0x7f15a40d0990 0x7f15a40aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:56.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.240+0000 7f15a3e00640 1 -- 172.21.15.19:0/195864238 >> v1:172.21.15.19:6789/0 conn(0x7f15a4153580 legacy=0x7f15a40d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:56.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.240+0000 7f15a3e00640 1 -- 172.21.15.19:0/195864238 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a415cf20 con 0x7f15a40af2e0 2024-03-28T13:17:56.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.240+0000 7f159be00640 1 --2- 172.21.15.19:0/195864238 >> v2:172.21.15.188:3300/0 conn(0x7f15a40d0990 0x7f15a40aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:56.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.241+0000 7f15a3e00640 1 --2- 172.21.15.19:0/195864238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40af2e0 0x7f15a41600d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f15a415a6b0 tx=0x7f159c02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb06c34a685c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:56.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.242+0000 7f15a2a00640 1 -- 172.21.15.19:0/195864238 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c039070 con 0x7f15a40af2e0 2024-03-28T13:17:56.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.242+0000 7f15a2a00640 1 -- 172.21.15.19:0/195864238 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159c02cbe0 con 0x7f15a40af2e0 2024-03-28T13:17:56.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.242+0000 7f15a2a00640 1 -- 172.21.15.19:0/195864238 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c034440 con 0x7f15a40af2e0 2024-03-28T13:17:56.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.242+0000 7f15aa000640 1 -- 172.21.15.19:0/195864238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40af2e0 msgr2=0x7f15a41600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:56.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.242+0000 7f15aa000640 1 --2- 172.21.15.19:0/195864238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40af2e0 0x7f15a41600d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f15a415a6b0 tx=0x7f159c02be30 comp rx=0 tx=0).stop 2024-03-28T13:17:56.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.242+0000 7f15aa000640 1 -- 172.21.15.19:0/195864238 shutdown_connections 2024-03-28T13:17:56.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.242+0000 7f15aa000640 1 --2- 172.21.15.19:0/195864238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40af2e0 0x7f15a41600d0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:56.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.242+0000 7f15aa000640 1 --2- 172.21.15.19:0/195864238 >> v2:172.21.15.188:3300/0 conn(0x7f15a40d0990 0x7f15a40aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:56.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.242+0000 7f15aa000640 1 -- 172.21.15.19:0/195864238 >> 172.21.15.19:0/195864238 conn(0x7f15a406b750 msgr2=0x7f15a406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:56.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.242+0000 7f15aa000640 1 -- 172.21.15.19:0/195864238 shutdown_connections 2024-03-28T13:17:56.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.243+0000 7f15aa000640 1 -- 172.21.15.19:0/195864238 wait complete. 2024-03-28T13:17:56.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.244+0000 7f15aa000640 1 Processor -- start 2024-03-28T13:17:56.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.244+0000 7f15aa000640 1 -- start start 2024-03-28T13:17:56.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.245+0000 7f15aa000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15a40af2e0 0x7f15a4073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:56.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.245+0000 7f15aa000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40d0990 0x7f15a414c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:56.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.245+0000 7f15aa000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a4153580 0x7f15a4154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:56.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.245+0000 7f15aa000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15a4153960 con 0x7f15a40d0990 2024-03-28T13:17:56.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.245+0000 7f15aa000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f15a415d4e0 con 0x7f15a4153580 2024-03-28T13:17:56.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.245+0000 7f15aa000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f15a40af6c0 con 0x7f15a40af2e0 2024-03-28T13:17:56.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.246+0000 7f159be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40d0990 0x7f15a414c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:56.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.246+0000 7f159be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40d0990 0x7f15a414c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45086/0 (socket says 172.21.15.19:45086) 2024-03-28T13:17:56.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.246+0000 7f159be00640 1 -- 172.21.15.19:0/2371727006 learned_addr learned my addr 172.21.15.19:0/2371727006 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:56.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.246+0000 7f15a3e00640 1 --2- 172.21.15.19:0/2371727006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a4153580 0x7f15a4154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:56.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.246+0000 7f15a3400640 1 --2- 172.21.15.19:0/2371727006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15a40af2e0 0x7f15a4073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:56.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.246+0000 7f159be00640 1 -- 172.21.15.19:0/2371727006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15a40af2e0 msgr2=0x7f15a4073210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:56.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.246+0000 7f159be00640 1 --2- 172.21.15.19:0/2371727006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15a40af2e0 0x7f15a4073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.246+0000 7f159be00640 1 -- 172.21.15.19:0/2371727006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a4153580 msgr2=0x7f15a4154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.246+0000 7f159be00640 1 --2- 172.21.15.19:0/2371727006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a4153580 0x7f15a4154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.246+0000 7f159be00640 1 -- 172.21.15.19:0/2371727006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a41552b0 con 0x7f15a40d0990 2024-03-28T13:17:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.247+0000 7f159be00640 1 --2- 172.21.15.19:0/2371727006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40d0990 0x7f15a414c810 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c7c0 tx=0x7f158c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.247+0000 7f159b400640 1 -- 172.21.15.19:0/2371727006 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c015070 con 0x7f15a40d0990 2024-03-28T13:17:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.247+0000 7f159b400640 1 -- 172.21.15.19:0/2371727006 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f158c007820 con 0x7f15a40d0990 2024-03-28T13:17:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.247+0000 7f159b400640 1 -- 172.21.15.19:0/2371727006 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158c007bf0 con 0x7f15a40d0990 2024-03-28T13:17:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.247+0000 7f15aa000640 1 -- 172.21.15.19:0/2371727006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a4155590 con 0x7f15a40d0990 2024-03-28T13:17:56.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.247+0000 7f15aa000640 1 -- 172.21.15.19:0/2371727006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a4155b50 con 0x7f15a40d0990 2024-03-28T13:17:56.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.248+0000 7f15aa000640 1 -- 172.21.15.19:0/2371727006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a40685c0 con 0x7f15a40d0990 2024-03-28T13:17:56.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.249+0000 7f159b400640 1 -- 172.21.15.19:0/2371727006 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f158c018070 con 0x7f15a40d0990 2024-03-28T13:17:56.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.250+0000 7f159b400640 1 --2- 172.21.15.19:0/2371727006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f158006c530 0x7f158006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:56.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.250+0000 7f159b400640 1 -- 172.21.15.19:0/2371727006 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f158c08b660 con 0x7f15a40d0990 2024-03-28T13:17:56.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.251+0000 7f15a3400640 1 --2- 172.21.15.19:0/2371727006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f158006c530 0x7f158006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:56.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.252+0000 7f15a3400640 1 --2- 172.21.15.19:0/2371727006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f158006c530 0x7f158006e9f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1594007fd0 tx=0x7f1594009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:56.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.254+0000 7f159b400640 1 -- 172.21.15.19:0/2371727006 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f158c059bc0 con 0x7f15a40d0990 2024-03-28T13:17:56.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.500+0000 7f15aa000640 1 -- 172.21.15.19:0/2371727006 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f15a406eca0 con 0x7f15a40d0990 2024-03-28T13:17:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.697+0000 7f159b400640 1 -- 172.21.15.19:0/2371727006 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v71) ==== 162+0+0 (secure 0 0 0) 0x7f158c0594c0 con 0x7f15a40d0990 2024-03-28T13:17:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-28T13:17:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.700+0000 7f15aa000640 1 -- 172.21.15.19:0/2371727006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f158006c530 msgr2=0x7f158006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.700+0000 7f15aa000640 1 --2- 172.21.15.19:0/2371727006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f158006c530 0x7f158006e9f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1594007fd0 tx=0x7f1594009040 comp rx=0 tx=0).stop 2024-03-28T13:17:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.700+0000 7f15aa000640 1 -- 172.21.15.19:0/2371727006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40d0990 msgr2=0x7f15a414c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:56.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.700+0000 7f15aa000640 1 --2- 172.21.15.19:0/2371727006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40d0990 0x7f15a414c810 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c7c0 tx=0x7f158c00cc80 comp rx=0 tx=0).stop 2024-03-28T13:17:56.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.701+0000 7f15aa000640 1 -- 172.21.15.19:0/2371727006 shutdown_connections 2024-03-28T13:17:56.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.701+0000 7f15aa000640 1 --2- 172.21.15.19:0/2371727006 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f158006c530 0x7f158006e9f0 secure :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1594007fd0 tx=0x7f1594009040 comp rx=0 tx=0).stop 2024-03-28T13:17:56.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.701+0000 7f15aa000640 1 --2- 172.21.15.19:0/2371727006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a4153580 0x7f15a4154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:56.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.701+0000 7f15aa000640 1 --2- 172.21.15.19:0/2371727006 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15a40d0990 0x7f15a414c810 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:56.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.701+0000 7f15aa000640 1 --2- 172.21.15.19:0/2371727006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15a40af2e0 0x7f15a4073210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:56.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.701+0000 7f15aa000640 1 -- 172.21.15.19:0/2371727006 >> 172.21.15.19:0/2371727006 conn(0x7f15a406b750 msgr2=0x7f15a40d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:56.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.701+0000 7f15aa000640 1 -- 172.21.15.19:0/2371727006 shutdown_connections 2024-03-28T13:17:56.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.701+0000 7f15aa000640 1 -- 172.21.15.19:0/2371727006 wait complete. 2024-03-28T13:17:56.724 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:17:56.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.921+0000 7f6a9ba00640 1 Processor -- start 2024-03-28T13:17:56.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.921+0000 7f6a9ba00640 1 -- start start 2024-03-28T13:17:56.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.922+0000 7f6a9ba00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a9415c390 0x7f6a94073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:56.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.922+0000 7f6a9ba00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a940af830 0x7f6a94160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:56.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.923+0000 7f6a9ba00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a9415fe80 con 0x7f6a94160b80 2024-03-28T13:17:56.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.923+0000 7f6a9ba00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f6a9415c770 con 0x7f6a9415c390 2024-03-28T13:17:56.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.923+0000 7f6a9ba00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6a940ad120 con 0x7f6a940af830 2024-03-28T13:17:56.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.923+0000 7f6a93e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a940af830 0x7f6a94160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:56.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.923+0000 7f6a99000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a9415c390 0x7f6a94073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:56.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.923+0000 7f6a93e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a940af830 0x7f6a94160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:32810/0 (socket says 172.21.15.19:32810) 2024-03-28T13:17:56.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.923+0000 7f6a93e00640 1 -- 172.21.15.19:0/3471555518 learned_addr learned my addr 172.21.15.19:0/3471555518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:56.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.924+0000 7f6a93e00640 1 -- 172.21.15.19:0/3471555518 >> v2:172.21.15.188:3300/0 conn(0x7f6a9415c390 msgr2=0x7f6a94073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:56.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.924+0000 7f6a93e00640 1 --2- 172.21.15.19:0/3471555518 >> v2:172.21.15.188:3300/0 conn(0x7f6a9415c390 0x7f6a94073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:56.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.924+0000 7f6a93e00640 1 -- 172.21.15.19:0/3471555518 >> v1:172.21.15.188:6789/0 conn(0x7f6a94160b80 legacy=0x7f6a940d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:56.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.924+0000 7f6a93e00640 1 -- 172.21.15.19:0/3471555518 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a940ad400 con 0x7f6a940af830 2024-03-28T13:17:56.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.926+0000 7f6a93e00640 1 --2- 172.21.15.19:0/3471555518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a940af830 0x7f6a94160640 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009870 tx=0x7f6a8002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a9404b0ad6c2c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.926+0000 7f6a93400640 1 -- 172.21.15.19:0/3471555518 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80034070 con 0x7f6a940af830 2024-03-28T13:17:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.926+0000 7f6a93400640 1 -- 172.21.15.19:0/3471555518 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a80030840 con 0x7f6a940af830 2024-03-28T13:17:56.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.926+0000 7f6a93400640 1 -- 172.21.15.19:0/3471555518 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80030c10 con 0x7f6a940af830 2024-03-28T13:17:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.927+0000 7f6a9ba00640 1 -- 172.21.15.19:0/3471555518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a940af830 msgr2=0x7f6a94160640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.927+0000 7f6a9ba00640 1 --2- 172.21.15.19:0/3471555518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a940af830 0x7f6a94160640 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6a80009870 tx=0x7f6a8002b980 comp rx=0 tx=0).stop 2024-03-28T13:17:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.927+0000 7f6a9ba00640 1 -- 172.21.15.19:0/3471555518 shutdown_connections 2024-03-28T13:17:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.927+0000 7f6a9ba00640 1 --2- 172.21.15.19:0/3471555518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a940af830 0x7f6a94160640 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.927+0000 7f6a9ba00640 1 --2- 172.21.15.19:0/3471555518 >> v2:172.21.15.188:3300/0 conn(0x7f6a9415c390 0x7f6a94073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.927+0000 7f6a9ba00640 1 -- 172.21.15.19:0/3471555518 >> 172.21.15.19:0/3471555518 conn(0x7f6a9406b990 msgr2=0x7f6a9406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.927+0000 7f6a9ba00640 1 -- 172.21.15.19:0/3471555518 shutdown_connections 2024-03-28T13:17:56.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.927+0000 7f6a9ba00640 1 -- 172.21.15.19:0/3471555518 wait complete. 2024-03-28T13:17:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.929+0000 7f6a9ba00640 1 Processor -- start 2024-03-28T13:17:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.929+0000 7f6a9ba00640 1 -- start start 2024-03-28T13:17:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.930+0000 7f6a9ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a940af830 0x7f6a940b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.930+0000 7f6a9ba00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9415c390 0x7f6a940b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.930+0000 7f6a9ba00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a94160b80 0x7f6a940b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.930+0000 7f6a9ba00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6a940cf980 con 0x7f6a940af830 2024-03-28T13:17:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.930+0000 7f6a9ba00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94160120 con 0x7f6a9415c390 2024-03-28T13:17:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.930+0000 7f6a9ba00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6a940d0d60 con 0x7f6a94160b80 2024-03-28T13:17:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.930+0000 7f6a99000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a940af830 0x7f6a940b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.930+0000 7f6a99000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a940af830 0x7f6a940b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45112/0 (socket says 172.21.15.19:45112) 2024-03-28T13:17:56.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.930+0000 7f6a99000640 1 -- 172.21.15.19:0/874236765 learned_addr learned my addr 172.21.15.19:0/874236765 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.931+0000 7f6a99a00640 1 --2- 172.21.15.19:0/874236765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a94160b80 0x7f6a940b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.931+0000 7f6a93e00640 1 --2- 172.21.15.19:0/874236765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9415c390 0x7f6a940b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.931+0000 7f6a99000640 1 -- 172.21.15.19:0/874236765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a94160b80 msgr2=0x7f6a940b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.931+0000 7f6a99000640 1 --2- 172.21.15.19:0/874236765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a94160b80 0x7f6a940b0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.931+0000 7f6a99000640 1 -- 172.21.15.19:0/874236765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9415c390 msgr2=0x7f6a940b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.931+0000 7f6a99000640 1 --2- 172.21.15.19:0/874236765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9415c390 0x7f6a940b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.931+0000 7f6a99000640 1 -- 172.21.15.19:0/874236765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a940b3ac0 con 0x7f6a940af830 2024-03-28T13:17:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.931+0000 7f6a93e00640 1 --2- 172.21.15.19:0/874236765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9415c390 0x7f6a940b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:17:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.931+0000 7f6a99000640 1 --2- 172.21.15.19:0/874236765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a940af830 0x7f6a940b2f10 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800e9b0 tx=0x7f6a8800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.931+0000 7f6a90c00640 1 -- 172.21.15.19:0/874236765 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88017070 con 0x7f6a940af830 2024-03-28T13:17:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.931+0000 7f6a90c00640 1 -- 172.21.15.19:0/874236765 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a8800c9a0 con 0x7f6a940af830 2024-03-28T13:17:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.931+0000 7f6a90c00640 1 -- 172.21.15.19:0/874236765 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8800cd90 con 0x7f6a940af830 2024-03-28T13:17:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.931+0000 7f6a9ba00640 1 -- 172.21.15.19:0/874236765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a940b3da0 con 0x7f6a940af830 2024-03-28T13:17:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.932+0000 7f6a9ba00640 1 -- 172.21.15.19:0/874236765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a940b4640 con 0x7f6a940af830 2024-03-28T13:17:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.933+0000 7f6a9ba00640 1 -- 172.21.15.19:0/874236765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a940b3f70 con 0x7f6a940af830 2024-03-28T13:17:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.934+0000 7f6a90c00640 1 -- 172.21.15.19:0/874236765 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6a8801a070 con 0x7f6a940af830 2024-03-28T13:17:56.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.935+0000 7f6a90c00640 1 --2- 172.21.15.19:0/874236765 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a7406c530 0x7f6a7406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:56.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.935+0000 7f6a90c00640 1 -- 172.21.15.19:0/874236765 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f6a8808baf0 con 0x7f6a940af830 2024-03-28T13:17:56.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.935+0000 7f6a93e00640 1 --2- 172.21.15.19:0/874236765 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a7406c530 0x7f6a7406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.937+0000 7f6a93e00640 1 --2- 172.21.15.19:0/874236765 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a7406c530 0x7f6a7406e9f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6a80036000 tx=0x7f6a8003b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:56.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:56.940+0000 7f6a90c00640 1 -- 172.21.15.19:0/874236765 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6a8805a0f0 con 0x7f6a940af830 2024-03-28T13:17:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:57.182+0000 7f6a9ba00640 1 -- 172.21.15.19:0/874236765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a940685c0 con 0x7f6a940af830 2024-03-28T13:17:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:57.184+0000 7f6a90c00640 1 -- 172.21.15.19:0/874236765 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5180 (secure 0 0 0) 0x7f6a880599f0 con 0x7f6a940af830 2024-03-28T13:17:57.184 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:17:57.184 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":71,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1144703580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1144703580},{"type":"v1","addr":"172.21.15.188:6836","nonce":1144703580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:17:56.694736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6040":{"gid":6040,"name":"a","rank":0,"incarnation":71,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-03-28T13:17:57.184 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 71 2024-03-28T13:17:57.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:57.186+0000 7f6a9ba00640 1 -- 172.21.15.19:0/874236765 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a7406c530 msgr2=0x7f6a7406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:57.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:57.186+0000 7f6a9ba00640 1 --2- 172.21.15.19:0/874236765 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a7406c530 0x7f6a7406e9f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6a80036000 tx=0x7f6a8003b040 comp rx=0 tx=0).stop 2024-03-28T13:17:57.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:57.186+0000 7f6a9ba00640 1 -- 172.21.15.19:0/874236765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a940af830 msgr2=0x7f6a940b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:57.186+0000 7f6a9ba00640 1 --2- 172.21.15.19:0/874236765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a940af830 0x7f6a940b2f10 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800e9b0 tx=0x7f6a8800ee70 comp rx=0 tx=0).stop 2024-03-28T13:17:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:57.186+0000 7f6a9ba00640 1 -- 172.21.15.19:0/874236765 shutdown_connections 2024-03-28T13:17:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:57.186+0000 7f6a9ba00640 1 --2- 172.21.15.19:0/874236765 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6a7406c530 0x7f6a7406e9f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:57.186+0000 7f6a9ba00640 1 --2- 172.21.15.19:0/874236765 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a94160b80 0x7f6a940b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:57.186+0000 7f6a9ba00640 1 --2- 172.21.15.19:0/874236765 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a9415c390 0x7f6a940b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:57.186+0000 7f6a9ba00640 1 --2- 172.21.15.19:0/874236765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6a940af830 0x7f6a940b2f10 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:57.186+0000 7f6a9ba00640 1 -- 172.21.15.19:0/874236765 >> 172.21.15.19:0/874236765 conn(0x7f6a9406b990 msgr2=0x7f6a94072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:57.187+0000 7f6a9ba00640 1 -- 172.21.15.19:0/874236765 shutdown_connections 2024-03-28T13:17:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:57.187+0000 7f6a9ba00640 1 -- 172.21.15.19:0/874236765 wait complete. 2024-03-28T13:17:57.610 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:16:25.995386+0000', 'modified': '2024-03-28T13:17:56.694736+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6040}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6040': {'gid': 6040, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6834/2209799371', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 2209799371}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 2209799371}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-28T13:17:57.610 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6040:up:replay 2024-03-28T13:17:58.611 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:17:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.842+0000 7efd46000640 1 Processor -- start 2024-03-28T13:17:58.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.842+0000 7efd46000640 1 -- start start 2024-03-28T13:17:58.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.844+0000 7efd46000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efd400af830 0x7efd40073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:58.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.844+0000 7efd46000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd400d0650 0x7efd400d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:58.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.844+0000 7efd46000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd4015fe80 con 0x7efd4015c390 2024-03-28T13:17:58.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.844+0000 7efd46000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7efd400d1000 con 0x7efd400af830 2024-03-28T13:17:58.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.844+0000 7efd46000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efd4015c770 con 0x7efd400d0650 2024-03-28T13:17:58.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.844+0000 7efd3ea00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efd400af830 0x7efd40073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:58.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.844+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd400d0650 0x7efd400d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:58.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.844+0000 7efd3fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd400d0650 0x7efd400d0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:32834/0 (socket says 172.21.15.19:32834) 2024-03-28T13:17:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.844+0000 7efd3fe00640 1 -- 172.21.15.19:0/2655587055 learned_addr learned my addr 172.21.15.19:0/2655587055 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.844+0000 7efd3ea00640 1 --2- 172.21.15.19:0/2655587055 >> v2:172.21.15.188:3300/0 conn(0x7efd400af830 0x7efd40073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:37990/0 (socket says 172.21.15.19:37990) 2024-03-28T13:17:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.845+0000 7efd3fe00640 1 -- 172.21.15.19:0/2655587055 >> v2:172.21.15.188:3300/0 conn(0x7efd400af830 msgr2=0x7efd40073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:17:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.845+0000 7efd3fe00640 1 --2- 172.21.15.19:0/2655587055 >> v2:172.21.15.188:3300/0 conn(0x7efd400af830 0x7efd40073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.845+0000 7efd3fe00640 1 -- 172.21.15.19:0/2655587055 >> v1:172.21.15.188:6789/0 conn(0x7efd4015c390 legacy=0x7efd40160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.845+0000 7efd3fe00640 1 -- 172.21.15.19:0/2655587055 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd400ad120 con 0x7efd400d0650 2024-03-28T13:17:58.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.845+0000 7efd3ea00640 1 --2- 172.21.15.19:0/2655587055 >> v2:172.21.15.188:3300/0 conn(0x7efd400af830 0x7efd40073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:17:58.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.847+0000 7efd3fe00640 1 --2- 172.21.15.19:0/2655587055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd400d0650 0x7efd400d0ac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efd38009870 tx=0x7efd3802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf49b49402b8c02f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.847+0000 7efd3e000640 1 -- 172.21.15.19:0/2655587055 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38034070 con 0x7efd400d0650 2024-03-28T13:17:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.847+0000 7efd3e000640 1 -- 172.21.15.19:0/2655587055 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd38030840 con 0x7efd400d0650 2024-03-28T13:17:58.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.847+0000 7efd3e000640 1 -- 172.21.15.19:0/2655587055 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38030c10 con 0x7efd400d0650 2024-03-28T13:17:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.848+0000 7efd46000640 1 -- 172.21.15.19:0/2655587055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd400d0650 msgr2=0x7efd400d0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.848+0000 7efd46000640 1 --2- 172.21.15.19:0/2655587055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd400d0650 0x7efd400d0ac0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efd38009870 tx=0x7efd3802b980 comp rx=0 tx=0).stop 2024-03-28T13:17:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.848+0000 7efd46000640 1 -- 172.21.15.19:0/2655587055 shutdown_connections 2024-03-28T13:17:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.848+0000 7efd46000640 1 --2- 172.21.15.19:0/2655587055 >> v2:172.21.15.188:3300/0 conn(0x7efd400af830 0x7efd40073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.848+0000 7efd46000640 1 --2- 172.21.15.19:0/2655587055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd400d0650 0x7efd400d0ac0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.848+0000 7efd46000640 1 -- 172.21.15.19:0/2655587055 >> 172.21.15.19:0/2655587055 conn(0x7efd4006b990 msgr2=0x7efd4006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.848+0000 7efd46000640 1 -- 172.21.15.19:0/2655587055 shutdown_connections 2024-03-28T13:17:58.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.848+0000 7efd46000640 1 -- 172.21.15.19:0/2655587055 wait complete. 2024-03-28T13:17:58.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.850+0000 7efd46000640 1 Processor -- start 2024-03-28T13:17:58.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.850+0000 7efd46000640 1 -- start start 2024-03-28T13:17:58.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.851+0000 7efd46000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd400af830 0x7efd401e8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.851+0000 7efd46000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd400d0650 0x7efd4014f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.851+0000 7efd46000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd4015c390 0x7efd40159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.851+0000 7efd46000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd40160120 con 0x7efd400af830 2024-03-28T13:17:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.851+0000 7efd46000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd400ad6e0 con 0x7efd400d0650 2024-03-28T13:17:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.851+0000 7efd46000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efd40160d50 con 0x7efd4015c390 2024-03-28T13:17:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.851+0000 7efd3f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd400af830 0x7efd401e8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.852+0000 7efd3f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd400af830 0x7efd401e8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45122/0 (socket says 172.21.15.19:45122) 2024-03-28T13:17:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.852+0000 7efd3f400640 1 -- 172.21.15.19:0/2240724083 learned_addr learned my addr 172.21.15.19:0/2240724083 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:17:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.852+0000 7efd3fe00640 1 --2- 172.21.15.19:0/2240724083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd4015c390 0x7efd40159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.852+0000 7efd3ea00640 1 --2- 172.21.15.19:0/2240724083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd400d0650 0x7efd4014f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.852+0000 7efd3f400640 1 -- 172.21.15.19:0/2240724083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd4015c390 msgr2=0x7efd40159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.852+0000 7efd3f400640 1 --2- 172.21.15.19:0/2240724083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd4015c390 0x7efd40159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.852+0000 7efd3f400640 1 -- 172.21.15.19:0/2240724083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd400d0650 msgr2=0x7efd4014f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:58.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.852+0000 7efd3f400640 1 --2- 172.21.15.19:0/2240724083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd400d0650 0x7efd4014f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.852+0000 7efd3f400640 1 -- 172.21.15.19:0/2240724083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd401e71c0 con 0x7efd400af830 2024-03-28T13:17:58.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.852+0000 7efd3f400640 1 --2- 172.21.15.19:0/2240724083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd400af830 0x7efd401e8260 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c790 tx=0x7efd3000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.853+0000 7efd37400640 1 -- 172.21.15.19:0/2240724083 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30016020 con 0x7efd400af830 2024-03-28T13:17:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.853+0000 7efd37400640 1 -- 172.21.15.19:0/2240724083 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd30006d50 con 0x7efd400af830 2024-03-28T13:17:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.853+0000 7efd37400640 1 -- 172.21.15.19:0/2240724083 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd30003bb0 con 0x7efd400af830 2024-03-28T13:17:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.853+0000 7efd46000640 1 -- 172.21.15.19:0/2240724083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd401e74a0 con 0x7efd400af830 2024-03-28T13:17:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.853+0000 7efd46000640 1 -- 172.21.15.19:0/2240724083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd401e7a60 con 0x7efd400af830 2024-03-28T13:17:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.854+0000 7efd46000640 1 -- 172.21.15.19:0/2240724083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd400685c0 con 0x7efd400af830 2024-03-28T13:17:58.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.855+0000 7efd37400640 1 -- 172.21.15.19:0/2240724083 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efd3000a850 con 0x7efd400af830 2024-03-28T13:17:58.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.856+0000 7efd37400640 1 --2- 172.21.15.19:0/2240724083 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd1c06c560 0x7efd1c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:17:58.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.857+0000 7efd37400640 1 -- 172.21.15.19:0/2240724083 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7efd3008b4c0 con 0x7efd400af830 2024-03-28T13:17:58.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.857+0000 7efd3ea00640 1 --2- 172.21.15.19:0/2240724083 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd1c06c560 0x7efd1c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:17:58.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.858+0000 7efd3ea00640 1 --2- 172.21.15.19:0/2240724083 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd1c06c560 0x7efd1c06ea20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7efd40155b60 tx=0x7efd28008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:17:58.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:58.861+0000 7efd37400640 1 -- 172.21.15.19:0/2240724083 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efd30020020 con 0x7efd400af830 2024-03-28T13:17:59.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:59.101+0000 7efd46000640 1 -- 172.21.15.19:0/2240724083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd4006e450 con 0x7efd400af830 2024-03-28T13:17:59.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:59.103+0000 7efd37400640 1 -- 172.21.15.19:0/2240724083 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5181 (secure 0 0 0) 0x7efd30019310 con 0x7efd400af830 2024-03-28T13:17:59.103 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:17:59.103 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1144703580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1144703580},{"type":"v1","addr":"172.21.15.188:6836","nonce":1144703580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:17:57.707691+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6040":{"gid":6040,"name":"a","rank":0,"incarnation":71,"state":"up:rejoin","state_seq":27,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-03-28T13:17:59.103 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 73 2024-03-28T13:17:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:59.105+0000 7efd46000640 1 -- 172.21.15.19:0/2240724083 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd1c06c560 msgr2=0x7efd1c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:59.105+0000 7efd46000640 1 --2- 172.21.15.19:0/2240724083 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd1c06c560 0x7efd1c06ea20 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7efd40155b60 tx=0x7efd28008040 comp rx=0 tx=0).stop 2024-03-28T13:17:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:59.105+0000 7efd46000640 1 -- 172.21.15.19:0/2240724083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd400af830 msgr2=0x7efd401e8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:17:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:59.105+0000 7efd46000640 1 --2- 172.21.15.19:0/2240724083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd400af830 0x7efd401e8260 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7efd3000c790 tx=0x7efd3000cc50 comp rx=0 tx=0).stop 2024-03-28T13:17:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:59.105+0000 7efd46000640 1 -- 172.21.15.19:0/2240724083 shutdown_connections 2024-03-28T13:17:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:59.105+0000 7efd46000640 1 --2- 172.21.15.19:0/2240724083 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd1c06c560 0x7efd1c06ea20 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:59.105+0000 7efd46000640 1 --2- 172.21.15.19:0/2240724083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd4015c390 0x7efd40159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:59.105+0000 7efd46000640 1 --2- 172.21.15.19:0/2240724083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd400d0650 0x7efd4014f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:59.105+0000 7efd46000640 1 --2- 172.21.15.19:0/2240724083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd400af830 0x7efd401e8260 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:17:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:59.105+0000 7efd46000640 1 -- 172.21.15.19:0/2240724083 >> 172.21.15.19:0/2240724083 conn(0x7efd4006b990 msgr2=0x7efd4006d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:17:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:59.105+0000 7efd46000640 1 -- 172.21.15.19:0/2240724083 shutdown_connections 2024-03-28T13:17:59.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:17:59.105+0000 7efd46000640 1 -- 172.21.15.19:0/2240724083 wait complete. 2024-03-28T13:17:59.527 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:16:25.995386+0000', 'modified': '2024-03-28T13:17:57.707691+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6040}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6040': {'gid': 6040, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:rejoin', 'state_seq': 27, 'addr': '172.21.15.19:6834/2209799371', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 2209799371}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 2209799371}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-28T13:17:59.527 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6040:up:rejoin 2024-03-28T13:18:00.488 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-03-28T13:18:00.488+0000 7f87b4a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:18:00.528 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:18:00.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.756+0000 7f138fa00640 1 Processor -- start 2024-03-28T13:18:00.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.757+0000 7f138fa00640 1 -- start start 2024-03-28T13:18:00.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.758+0000 7f138fa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f138815c390 0x7f1388073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:00.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.758+0000 7f138fa00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f13880af830 0x7f1388160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:00.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.758+0000 7f138fa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1388160b80 0x7f13880d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:00.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.758+0000 7f138fa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f138815c770 con 0x7f138815c390 2024-03-28T13:18:00.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.758+0000 7f138fa00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f138815fe80 con 0x7f13880af830 2024-03-28T13:18:00.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.758+0000 7f138fa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f13880ad120 con 0x7f1388160b80 2024-03-28T13:18:00.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.758+0000 7f138d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f138815c390 0x7f1388073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:00.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.759+0000 7f138d000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f138815c390 0x7f1388073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45126/0 (socket says 172.21.15.19:45126) 2024-03-28T13:18:00.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.759+0000 7f138d000640 1 -- 172.21.15.19:0/4217883907 learned_addr learned my addr 172.21.15.19:0/4217883907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:00.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.759+0000 7f1387e00640 1 --2- 172.21.15.19:0/4217883907 >> v2:172.21.15.188:3300/0 conn(0x7f13880af830 0x7f1388160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:00.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.759+0000 7f138da00640 1 --2- 172.21.15.19:0/4217883907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1388160b80 0x7f13880d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:00.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.759+0000 7f138d000640 1 -- 172.21.15.19:0/4217883907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1388160b80 msgr2=0x7f13880d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:00.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.759+0000 7f138d000640 1 --2- 172.21.15.19:0/4217883907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1388160b80 0x7f13880d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:00.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.759+0000 7f138d000640 1 -- 172.21.15.19:0/4217883907 >> v2:172.21.15.188:3300/0 conn(0x7f13880af830 msgr2=0x7f1388160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:00.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.759+0000 7f138d000640 1 --2- 172.21.15.19:0/4217883907 >> v2:172.21.15.188:3300/0 conn(0x7f13880af830 0x7f1388160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:00.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.759+0000 7f138d000640 1 -- 172.21.15.19:0/4217883907 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13880ad400 con 0x7f138815c390 2024-03-28T13:18:00.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.759+0000 7f1387e00640 1 --2- 172.21.15.19:0/4217883907 >> v2:172.21.15.188:3300/0 conn(0x7f13880af830 0x7f1388160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:00.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.760+0000 7f138d000640 1 --2- 172.21.15.19:0/4217883907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138815c390 0x7f1388073790 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f138806eaa0 tx=0x7f137c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a68ad691333fb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:00.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.761+0000 7f1387400640 1 -- 172.21.15.19:0/4217883907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c035020 con 0x7f138815c390 2024-03-28T13:18:00.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.761+0000 7f1387400640 1 -- 172.21.15.19:0/4217883907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137c02d040 con 0x7f138815c390 2024-03-28T13:18:00.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.761+0000 7f1387400640 1 -- 172.21.15.19:0/4217883907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c031a80 con 0x7f138815c390 2024-03-28T13:18:00.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.761+0000 7f138fa00640 1 -- 172.21.15.19:0/4217883907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138815c390 msgr2=0x7f1388073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:00.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.761+0000 7f138fa00640 1 --2- 172.21.15.19:0/4217883907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138815c390 0x7f1388073790 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f138806eaa0 tx=0x7f137c02b900 comp rx=0 tx=0).stop 2024-03-28T13:18:00.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.761+0000 7f138fa00640 1 -- 172.21.15.19:0/4217883907 shutdown_connections 2024-03-28T13:18:00.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.761+0000 7f138fa00640 1 --2- 172.21.15.19:0/4217883907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1388160b80 0x7f13880d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:00.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.761+0000 7f138fa00640 1 --2- 172.21.15.19:0/4217883907 >> v2:172.21.15.188:3300/0 conn(0x7f13880af830 0x7f1388160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:00.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.761+0000 7f138fa00640 1 --2- 172.21.15.19:0/4217883907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138815c390 0x7f1388073790 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:00.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.761+0000 7f138fa00640 1 -- 172.21.15.19:0/4217883907 >> 172.21.15.19:0/4217883907 conn(0x7f138806b990 msgr2=0x7f138806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:00.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.762+0000 7f138fa00640 1 -- 172.21.15.19:0/4217883907 shutdown_connections 2024-03-28T13:18:00.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.762+0000 7f138fa00640 1 -- 172.21.15.19:0/4217883907 wait complete. 2024-03-28T13:18:00.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.763+0000 7f138fa00640 1 Processor -- start 2024-03-28T13:18:00.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.763+0000 7f138fa00640 1 -- start start 2024-03-28T13:18:00.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.764+0000 7f138fa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13880af830 0x7f138814f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:00.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.764+0000 7f138fa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138815c390 0x7f13881f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:00.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.764+0000 7f138fa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1388160b80 0x7f13881f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:00.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.764+0000 7f138fa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13880cf980 con 0x7f138815c390 2024-03-28T13:18:00.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.764+0000 7f138fa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1388160120 con 0x7f1388160b80 2024-03-28T13:18:00.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.764+0000 7f138fa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f13880d0b90 con 0x7f13880af830 2024-03-28T13:18:00.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.765+0000 7f1387e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138815c390 0x7f13881f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:00.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.765+0000 7f1387e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138815c390 0x7f13881f7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45134/0 (socket says 172.21.15.19:45134) 2024-03-28T13:18:00.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.765+0000 7f1387e00640 1 -- 172.21.15.19:0/2422933266 learned_addr learned my addr 172.21.15.19:0/2422933266 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:00.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.765+0000 7f138d000640 1 --2- 172.21.15.19:0/2422933266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13880af830 0x7f138814f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:00.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.765+0000 7f138da00640 1 --2- 172.21.15.19:0/2422933266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1388160b80 0x7f13881f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:00.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.765+0000 7f1387e00640 1 -- 172.21.15.19:0/2422933266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13880af830 msgr2=0x7f138814f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:00.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.765+0000 7f1387e00640 1 --2- 172.21.15.19:0/2422933266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13880af830 0x7f138814f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:00.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.765+0000 7f1387e00640 1 -- 172.21.15.19:0/2422933266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1388160b80 msgr2=0x7f13881f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:00.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.765+0000 7f1387e00640 1 --2- 172.21.15.19:0/2422933266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1388160b80 0x7f13881f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:00.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.765+0000 7f1387e00640 1 -- 172.21.15.19:0/2422933266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1388151a50 con 0x7f138815c390 2024-03-28T13:18:00.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.765+0000 7f138d000640 1 --2- 172.21.15.19:0/2422933266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13880af830 0x7f138814f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:00.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.766+0000 7f1387e00640 1 --2- 172.21.15.19:0/2422933266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138815c390 0x7f13881f7d40 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f137400c7f0 tx=0x7f137400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:00.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.766+0000 7f1384c00640 1 -- 172.21.15.19:0/2422933266 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374015070 con 0x7f138815c390 2024-03-28T13:18:00.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.766+0000 7f1384c00640 1 -- 172.21.15.19:0/2422933266 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13740079d0 con 0x7f138815c390 2024-03-28T13:18:00.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.766+0000 7f1384c00640 1 -- 172.21.15.19:0/2422933266 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374007da0 con 0x7f138815c390 2024-03-28T13:18:00.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.766+0000 7f138fa00640 1 -- 172.21.15.19:0/2422933266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1388151cc0 con 0x7f138815c390 2024-03-28T13:18:00.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.766+0000 7f138fa00640 1 -- 172.21.15.19:0/2422933266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1388152280 con 0x7f138815c390 2024-03-28T13:18:00.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.767+0000 7f138fa00640 1 -- 172.21.15.19:0/2422933266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13880685c0 con 0x7f138815c390 2024-03-28T13:18:00.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.768+0000 7f1384c00640 1 -- 172.21.15.19:0/2422933266 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1374018070 con 0x7f138815c390 2024-03-28T13:18:00.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.769+0000 7f1384c00640 1 --2- 172.21.15.19:0/2422933266 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f136806c530 0x7f136806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:00.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.770+0000 7f1384c00640 1 -- 172.21.15.19:0/2422933266 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f137408b6b0 con 0x7f138815c390 2024-03-28T13:18:00.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.771+0000 7f138d000640 1 --2- 172.21.15.19:0/2422933266 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f136806c530 0x7f136806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:00.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.772+0000 7f138d000640 1 --2- 172.21.15.19:0/2422933266 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f136806c530 0x7f136806e9f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f137c036000 tx=0x7f137c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:00.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:00.774+0000 7f1384c00640 1 -- 172.21.15.19:0/2422933266 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1374059c10 con 0x7f138815c390 2024-03-28T13:18:01.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.016+0000 7f138fa00640 1 -- 172.21.15.19:0/2422933266 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1388152450 con 0x7f138815c390 2024-03-28T13:18:01.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.017+0000 7f1384c00640 1 -- 172.21.15.19:0/2422933266 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5188 (secure 0 0 0) 0x7f1374059510 con 0x7f138815c390 2024-03-28T13:18:01.017 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:01.017 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1144703580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1144703580},{"type":"v1","addr":"172.21.15.188:6836","nonce":1144703580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:17:59.710623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6040":{"gid":6040,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":28,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6040,"qdb_cluster":[6040]},"id":6}]} 2024-03-28T13:18:01.018 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 74 2024-03-28T13:18:01.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.019+0000 7f138fa00640 1 -- 172.21.15.19:0/2422933266 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f136806c530 msgr2=0x7f136806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.019+0000 7f138fa00640 1 --2- 172.21.15.19:0/2422933266 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f136806c530 0x7f136806e9f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f137c036000 tx=0x7f137c008030 comp rx=0 tx=0).stop 2024-03-28T13:18:01.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.019+0000 7f138fa00640 1 -- 172.21.15.19:0/2422933266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138815c390 msgr2=0x7f13881f7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.019+0000 7f138fa00640 1 --2- 172.21.15.19:0/2422933266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138815c390 0x7f13881f7d40 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f137400c7f0 tx=0x7f137400cbf0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.019+0000 7f138fa00640 1 -- 172.21.15.19:0/2422933266 shutdown_connections 2024-03-28T13:18:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.019+0000 7f138fa00640 1 --2- 172.21.15.19:0/2422933266 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f136806c530 0x7f136806e9f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.019+0000 7f138fa00640 1 --2- 172.21.15.19:0/2422933266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1388160b80 0x7f13881f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.019+0000 7f138fa00640 1 --2- 172.21.15.19:0/2422933266 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f138815c390 0x7f13881f7d40 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.019+0000 7f138fa00640 1 --2- 172.21.15.19:0/2422933266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13880af830 0x7f138814f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.019+0000 7f138fa00640 1 -- 172.21.15.19:0/2422933266 >> 172.21.15.19:0/2422933266 conn(0x7f138806b990 msgr2=0x7f13880afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.020+0000 7f138fa00640 1 -- 172.21.15.19:0/2422933266 shutdown_connections 2024-03-28T13:18:01.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.020+0000 7f138fa00640 1 -- 172.21.15.19:0/2422933266 wait complete. 2024-03-28T13:18:01.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:16:25.995386+0000', 'modified': '2024-03-28T13:17:59.710623+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6040}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6040': {'gid': 6040, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 28, 'addr': '172.21.15.19:6834/2209799371', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 2209799371}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 2209799371}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6040, 'qdb_cluster': [6040]} 2024-03-28T13:18:01.446 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T13:18:01.446 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-28T13:18:01.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.678+0000 7fa1e6600640 1 Processor -- start 2024-03-28T13:18:01.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.678+0000 7fa1e6600640 1 -- start start 2024-03-28T13:18:01.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.680+0000 7fa1e6600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e00d2310 0x7fa1e00ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.680+0000 7fa1e6600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1e015e880 con 0x7fa1e00cf530 2024-03-28T13:18:01.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.680+0000 7fa1e6600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1e00d28d0 con 0x7fa1e015e4a0 2024-03-28T13:18:01.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.680+0000 7fa1e6600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e00d34e0 con 0x7fa1e00d2310 2024-03-28T13:18:01.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.680+0000 7fa1e5c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa1e00cf530 0x7fa1e00b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37346/0 (socket says 172.21.15.19:37346) 2024-03-28T13:18:01.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.680+0000 7fa1e5c00640 1 -- 172.21.15.19:0/799399635 learned_addr learned my addr 172.21.15.19:0/799399635 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:01.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.681+0000 7fa1d7e00640 1 --2- 172.21.15.19:0/799399635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e00d2310 0x7fa1e00ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.681+0000 7fa1dfe00640 1 -- 172.21.15.19:0/799399635 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1187317342 0 0) 0x7fa1e015e880 con 0x7fa1e00cf530 2024-03-28T13:18:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.681+0000 7fa1dfe00640 1 -- 172.21.15.19:0/799399635 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1c4003520 con 0x7fa1e00cf530 2024-03-28T13:18:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.681+0000 7fa1dfe00640 1 -- 172.21.15.19:0/799399635 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2093700019 0 0) 0x7fa1c4003520 con 0x7fa1e00cf530 2024-03-28T13:18:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.681+0000 7fa1dfe00640 1 -- 172.21.15.19:0/799399635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e00d2310 msgr2=0x7fa1e00ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.681+0000 7fa1dfe00640 1 --2- 172.21.15.19:0/799399635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e00d2310 0x7fa1e00ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.681+0000 7fa1dfe00640 1 -- 172.21.15.19:0/799399635 >> v1:172.21.15.188:6789/0 conn(0x7fa1e015e4a0 legacy=0x7fa1e00b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.681+0000 7fa1dfe00640 1 -- 172.21.15.19:0/799399635 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e00d2bb0 con 0x7fa1e00cf530 2024-03-28T13:18:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.681+0000 7fa1dfe00640 1 -- 172.21.15.19:0/799399635 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fa1d80019f0 con 0x7fa1e00cf530 2024-03-28T13:18:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.682+0000 7fa1dfe00640 1 -- 172.21.15.19:0/799399635 >> v1:172.21.15.19:6789/0 conn(0x7fa1e00cf530 legacy=0x7fa1e00b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.682+0000 7fa1dfe00640 1 --2- 172.21.15.19:0/799399635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e00d2310 0x7fa1c40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.682+0000 7fa1dfe00640 1 --2- 172.21.15.19:0/799399635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1c4003fb0 0x7fa1c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.682+0000 7fa1dfe00640 1 --2- 172.21.15.19:0/799399635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1c4004d40 0x7fa1c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.682+0000 7fa1dfe00640 1 -- 172.21.15.19:0/799399635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e00d34e0 con 0x7fa1c4004d40 2024-03-28T13:18:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.682+0000 7fa1dfe00640 1 -- 172.21.15.19:0/799399635 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c400bb00 con 0x7fa1c4003fb0 2024-03-28T13:18:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.682+0000 7fa1dfe00640 1 -- 172.21.15.19:0/799399635 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c400bd00 con 0x7fa1e00d2310 2024-03-28T13:18:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.682+0000 7fa1e5c00640 1 --2- 172.21.15.19:0/799399635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1c4004d40 0x7fa1c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.682+0000 7fa1d7e00640 1 --2- 172.21.15.19:0/799399635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1c4003fb0 0x7fa1c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.682+0000 7fa1e5200640 1 --2- 172.21.15.19:0/799399635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e00d2310 0x7fa1c40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.683+0000 7fa1e5c00640 1 -- 172.21.15.19:0/799399635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e00d2310 msgr2=0x7fa1c40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.683+0000 7fa1e5c00640 1 --2- 172.21.15.19:0/799399635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1e00d2310 0x7fa1c40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.683+0000 7fa1e5c00640 1 -- 172.21.15.19:0/799399635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1c4003fb0 msgr2=0x7fa1c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:01.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.683+0000 7fa1e5c00640 1 --2- 172.21.15.19:0/799399635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1c4003fb0 0x7fa1c4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.683+0000 7fa1e5c00640 1 -- 172.21.15.19:0/799399635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c4003520 con 0x7fa1c4004d40 2024-03-28T13:18:01.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.684+0000 7fa1e5c00640 1 --2- 172.21.15.19:0/799399635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1c4004d40 0x7fa1c40051c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800a770 tx=0x7fa1c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469d320bda0da8da server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:01.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.684+0000 7fa1dfe00640 1 -- 172.21.15.19:0/799399635 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800a390 con 0x7fa1c4004d40 2024-03-28T13:18:01.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.684+0000 7fa1dfe00640 1 -- 172.21.15.19:0/799399635 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d8034650 con 0x7fa1c4004d40 2024-03-28T13:18:01.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.684+0000 7fa1dfe00640 1 -- 172.21.15.19:0/799399635 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8034a80 con 0x7fa1c4004d40 2024-03-28T13:18:01.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.684+0000 7fa1e6600640 1 -- 172.21.15.19:0/799399635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1c4004d40 msgr2=0x7fa1c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.684+0000 7fa1e6600640 1 --2- 172.21.15.19:0/799399635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1c4004d40 0x7fa1c40051c0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800a770 tx=0x7fa1c4005830 comp rx=0 tx=0).stop 2024-03-28T13:18:01.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.684+0000 7fa1e5c00640 1 -- 172.21.15.19:0/799399635 reap_dead start 2024-03-28T13:18:01.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.685+0000 7fa1e6600640 1 -- 172.21.15.19:0/799399635 shutdown_connections 2024-03-28T13:18:01.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.685+0000 7fa1e6600640 1 -- 172.21.15.19:0/799399635 >> 172.21.15.19:0/799399635 conn(0x7fa1e00676d0 msgr2=0x7fa1e0067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:01.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.685+0000 7fa1e6600640 1 -- 172.21.15.19:0/799399635 shutdown_connections 2024-03-28T13:18:01.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.685+0000 7fa1e6600640 1 -- 172.21.15.19:0/799399635 wait complete. 2024-03-28T13:18:01.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.686+0000 7fa1e6600640 1 Processor -- start 2024-03-28T13:18:01.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.687+0000 7fa1e6600640 1 -- start start 2024-03-28T13:18:01.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.687+0000 7fa1e6600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1c4004d40 0x7fa1e0153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.687+0000 7fa1e6600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e01604a0 0x7fa1e01535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.687+0000 7fa1e6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1e014e680 0x7fa1e0159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.687+0000 7fa1e6600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e00d3970 con 0x7fa1e014e680 2024-03-28T13:18:01.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.687+0000 7fa1e6600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e00d3170 con 0x7fa1e01604a0 2024-03-28T13:18:01.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.688+0000 7fa1e6600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e00ceff0 con 0x7fa1c4004d40 2024-03-28T13:18:01.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.688+0000 7fa1e5c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1e014e680 0x7fa1e0159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.688+0000 7fa1e5c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1e014e680 0x7fa1e0159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45142/0 (socket says 172.21.15.19:45142) 2024-03-28T13:18:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.688+0000 7fa1e5c00640 1 -- 172.21.15.19:0/2123303742 learned_addr learned my addr 172.21.15.19:0/2123303742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.688+0000 7fa1e5c00640 1 -- 172.21.15.19:0/2123303742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1c4004d40 msgr2=0x7fa1e0153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.688+0000 7fa1e5c00640 1 --2- 172.21.15.19:0/2123303742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1c4004d40 0x7fa1e0153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.688+0000 7fa1d7e00640 1 --2- 172.21.15.19:0/2123303742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e01604a0 0x7fa1e01535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.688+0000 7fa1e5c00640 1 -- 172.21.15.19:0/2123303742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e01604a0 msgr2=0x7fa1e01535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.688+0000 7fa1e5c00640 1 --2- 172.21.15.19:0/2123303742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e01604a0 0x7fa1e01535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.688+0000 7fa1e5c00640 1 -- 172.21.15.19:0/2123303742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e015f6e0 con 0x7fa1e014e680 2024-03-28T13:18:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.689+0000 7fa1e5c00640 1 --2- 172.21.15.19:0/2123303742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1e014e680 0x7fa1e0159700 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800a770 tx=0x7fa1e00d3450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.689+0000 7fa1dd600640 1 -- 172.21.15.19:0/2123303742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80019f0 con 0x7fa1e014e680 2024-03-28T13:18:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.689+0000 7fa1dd600640 1 -- 172.21.15.19:0/2123303742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d8044070 con 0x7fa1e014e680 2024-03-28T13:18:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.689+0000 7fa1dd600640 1 -- 172.21.15.19:0/2123303742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8034a00 con 0x7fa1e014e680 2024-03-28T13:18:01.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.689+0000 7fa1e6600640 1 -- 172.21.15.19:0/2123303742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e015f9c0 con 0x7fa1e014e680 2024-03-28T13:18:01.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.689+0000 7fa1e6600640 1 -- 172.21.15.19:0/2123303742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e015ff80 con 0x7fa1e014e680 2024-03-28T13:18:01.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.691+0000 7fa1dd600640 1 -- 172.21.15.19:0/2123303742 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa1d8034ba0 con 0x7fa1e014e680 2024-03-28T13:18:01.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.692+0000 7fa1dd600640 1 --2- 172.21.15.19:0/2123303742 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa1bc06c5b0 0x7fa1bc06ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.692+0000 7fa1dd600640 1 -- 172.21.15.19:0/2123303742 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7fa1d80adb10 con 0x7fa1e014e680 2024-03-28T13:18:01.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.693+0000 7fa1e5200640 1 --2- 172.21.15.19:0/2123303742 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa1bc06c5b0 0x7fa1bc06ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.693+0000 7fa1ea9aa740 1 Processor -- start 2024-03-28T13:18:01.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.693+0000 7fa1ea9aa740 1 -- start start 2024-03-28T13:18:01.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.694+0000 7fa1ea9aa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f0761d0 0x55a74f049170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.694+0000 7fa1ea9aa740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f0765b0 0x55a74f078f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.694+0000 7fa1ea9aa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0496b0 0x55a74f0794a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.694+0000 7fa1ea9aa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a74f010920 con 0x55a74f0496b0 2024-03-28T13:18:01.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.694+0000 7fa1ea9aa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a74ef60fa0 con 0x55a74f0761d0 2024-03-28T13:18:01.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.694+0000 7fa1ea9aa740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55a74efb2f90 con 0x55a74f0765b0 2024-03-28T13:18:01.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.694+0000 7fa1e5200640 1 --2- 172.21.15.19:0/2123303742 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa1bc06c5b0 0x7fa1bc06ea70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0007fd0 tx=0x7fa1d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:01.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.694+0000 7fa1e5c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0496b0 0x55a74f0794a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.695+0000 7fa1e5c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0496b0 0x55a74f0794a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45148/0 (socket says 172.21.15.19:45148) 2024-03-28T13:18:01.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.695+0000 7fa1e5c00640 1 -- 172.21.15.19:0/1348707748 learned_addr learned my addr 172.21.15.19:0/1348707748 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:01.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.695+0000 7fa1d7e00640 1 --2- 172.21.15.19:0/1348707748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f0765b0 0x55a74f078f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.695+0000 7fa1e5c00640 1 -- 172.21.15.19:0/1348707748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f0765b0 msgr2=0x55a74f078f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:01.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.695+0000 7fa1e5c00640 1 --2- 172.21.15.19:0/1348707748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f0765b0 0x55a74f078f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.695+0000 7fa1e5200640 1 --2- 172.21.15.19:0/1348707748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f0761d0 0x55a74f049170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.695+0000 7fa1e5c00640 1 -- 172.21.15.19:0/1348707748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f0761d0 msgr2=0x55a74f049170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:01.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.695+0000 7fa1e5c00640 1 --2- 172.21.15.19:0/1348707748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f0761d0 0x55a74f049170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.695+0000 7fa1e5c00640 1 -- 172.21.15.19:0/1348707748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a74ecc72d0 con 0x55a74f0496b0 2024-03-28T13:18:01.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.695+0000 7fa1e5200640 1 --2- 172.21.15.19:0/1348707748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f0761d0 0x55a74f049170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:01.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.695+0000 7fa1e5c00640 1 --2- 172.21.15.19:0/1348707748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0496b0 0x55a74f0794a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fa1d807b8f0 tx=0x7fa1d8002130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531cda26c9fa6020 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.698+0000 7fa1d5600640 1 -- 172.21.15.19:0/1348707748 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80aa070 con 0x55a74f0496b0 2024-03-28T13:18:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.698+0000 7fa1d5600640 1 -- 172.21.15.19:0/1348707748 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d8091070 con 0x55a74f0496b0 2024-03-28T13:18:01.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.698+0000 7fa1d5600640 1 -- 172.21.15.19:0/1348707748 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d807b960 con 0x55a74f0496b0 2024-03-28T13:18:01.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.698+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/1348707748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0496b0 msgr2=0x55a74f0794a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.698+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/1348707748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0496b0 0x55a74f0794a0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fa1d807b8f0 tx=0x7fa1d8002130 comp rx=0 tx=0).stop 2024-03-28T13:18:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.699+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/1348707748 shutdown_connections 2024-03-28T13:18:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.699+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/1348707748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0496b0 0x55a74f0794a0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.699+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/1348707748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f0765b0 0x55a74f078f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.699+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/1348707748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f0761d0 0x55a74f049170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.699+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/1348707748 >> 172.21.15.19:0/1348707748 conn(0x55a74f04b3e0 msgr2=0x55a74f093f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.699+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/1348707748 shutdown_connections 2024-03-28T13:18:01.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.699+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/1348707748 wait complete. 2024-03-28T13:18:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.700+0000 7fa1ea9aa740 1 -- start start 2024-03-28T13:18:01.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.700+0000 7fa1ea9aa740 1 Processor -- start 2024-03-28T13:18:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.702+0000 7fa1ea9aa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0761d0 0x55a74f08bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.702+0000 7fa1ea9aa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f0765b0 0x55a74f0651c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.702+0000 7fa1ea9aa740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f065700 0x55a74f08c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.702+0000 7fa1ea9aa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a74efaef40 con 0x55a74f0761d0 2024-03-28T13:18:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.702+0000 7fa1ea9aa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a74ef5b650 con 0x55a74f0765b0 2024-03-28T13:18:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.702+0000 7fa1ea9aa740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55a74ef6ea10 con 0x55a74f065700 2024-03-28T13:18:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.702+0000 7fa1ea9aa740 10 client.0 _ll_register_callbacks cb 0x55a74efadff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:18:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.702+0000 7fa1e5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0761d0 0x55a74f08bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.702+0000 7fa1e5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0761d0 0x55a74f08bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45154/0 (socket says 172.21.15.19:45154) 2024-03-28T13:18:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.702+0000 7fa1e5200640 1 -- 172.21.15.19:0/2593433786 learned_addr learned my addr 172.21.15.19:0/2593433786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:01.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.702+0000 7fa1d7e00640 1 --2- 172.21.15.19:0/2593433786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f0765b0 0x55a74f0651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.702+0000 7fa1e5c00640 1 --2- 172.21.15.19:0/2593433786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f065700 0x55a74f08c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.703+0000 7fa1e5200640 1 -- 172.21.15.19:0/2593433786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f065700 msgr2=0x55a74f08c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.703+0000 7fa1e5200640 1 --2- 172.21.15.19:0/2593433786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f065700 0x55a74f08c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.703+0000 7fa1e5200640 1 -- 172.21.15.19:0/2593433786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f0765b0 msgr2=0x55a74f0651c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.703+0000 7fa1e5200640 1 --2- 172.21.15.19:0/2593433786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f0765b0 0x55a74f0651c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.703+0000 7fa1e5200640 1 -- 172.21.15.19:0/2593433786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a74ed916b0 con 0x55a74f0761d0 2024-03-28T13:18:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.703+0000 7fa1e5c00640 1 --2- 172.21.15.19:0/2593433786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f065700 0x55a74f08c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.703+0000 7fa1e5200640 1 --2- 172.21.15.19:0/2593433786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0761d0 0x55a74f08bab0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0001400 tx=0x7fa1d0007a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b5602e8ea63ed1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.703+0000 7fa1ce000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:18:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.703+0000 7fa1ce000640 1 -- 172.21.15.19:0/2593433786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d000ebe0 con 0x55a74f0761d0 2024-03-28T13:18:01.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.703+0000 7fa1ce000640 1 -- 172.21.15.19:0/2593433786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7fa1d000ed80 con 0x55a74f0761d0 2024-03-28T13:18:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.703+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/2593433786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1d8091070 con 0x55a74f0761d0 2024-03-28T13:18:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.703+0000 7fa1ce000640 1 -- 172.21.15.19:0/2593433786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fa1d0022430 con 0x55a74f0761d0 2024-03-28T13:18:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.704+0000 7fa1ea9aa740 10 client.7746 fetch_fsmap learned FSMap version 74 2024-03-28T13:18:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.704+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/2593433786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55a74f08d000 con 0x55a74f0761d0 2024-03-28T13:18:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.704+0000 7fa1ce000640 1 -- 172.21.15.19:0/2593433786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d8091070 con 0x55a74f0761d0 2024-03-28T13:18:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.704+0000 7fa1ce000640 1 -- 172.21.15.19:0/2593433786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fa1d00225d0 con 0x55a74f0761d0 2024-03-28T13:18:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.704+0000 7fa1ea9aa740 10 client.7746 fetch_fsmap finished waiting for FSMap version 74 2024-03-28T13:18:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.704+0000 7fa1ea9aa740 10 client.7746 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:18:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.704+0000 7fa1ea9aa740 20 client.7746 populate_metadata read hostname 'smithi019' 2024-03-28T13:18:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.704+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/2593433786 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a74f07b1a0 0x55a74f0b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.704+0000 7fa1ea9aa740 4 client.7746 mds_command: new command op to 6040 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:18:01.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.704+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/2593433786 --> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a74efc04d0 con 0x55a74f07b1a0 2024-03-28T13:18:01.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.704+0000 7fa1d7e00640 1 --2- 172.21.15.19:0/2593433786 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a74f07b1a0 0x55a74f0b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.705+0000 7fa1d7e00640 1 --2- 172.21.15.19:0/2593433786 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a74f07b1a0 0x55a74f0b1e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e0aee52071ea33c server_cookie=6182ca921d7abc1 in_seq=0 out_seq=0 2024-03-28T13:18:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.705+0000 7fa1ce000640 10 client.7746 ms_handle_connect on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.706+0000 7fa1ce000640 1 -- 172.21.15.19:0/2593433786 <== mds.0 v2:172.21.15.19:6832/2209799371 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a74efc04d0 con 0x55a74f07b1a0 2024-03-28T13:18:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.706+0000 7fa1ce000640 10 client.7746 handle_command_reply: tid=0 2024-03-28T13:18:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.706+0000 7fa1ea9aa740 1 client.7746 shutdown 2024-03-28T13:18:01.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.706+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/2593433786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0761d0 msgr2=0x55a74f08bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.706+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/2593433786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0761d0 0x55a74f08bab0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0001400 tx=0x7fa1d0007a30 comp rx=0 tx=0).stop 2024-03-28T13:18:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.706+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/2593433786 shutdown_connections 2024-03-28T13:18:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.706+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/2593433786 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a74f07b1a0 0x55a74f0b1e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.706+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/2593433786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f065700 0x55a74f08c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.706+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/2593433786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f0765b0 0x55a74f0651c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.706+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/2593433786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0761d0 0x55a74f08bab0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.706+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/2593433786 >> 172.21.15.19:0/2593433786 conn(0x55a74f0496b0 msgr2=0x55a74f048890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.706+0000 7fa1ce000640 0 client.7746 ms_handle_reset on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.707+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/2593433786 shutdown_connections 2024-03-28T13:18:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.707+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/2593433786 wait complete. 2024-03-28T13:18:01.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.707+0000 7fa1ea9aa740 20 client.7746 trim_cache size 0 max 16384 2024-03-28T13:18:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.727+0000 7fa1ea9aa740 1 Processor -- start 2024-03-28T13:18:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.727+0000 7fa1ea9aa740 1 -- start start 2024-03-28T13:18:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.728+0000 7fa1ea9aa740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f065700 0x55a74f04cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.728+0000 7fa1ea9aa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0761d0 0x55a74f050190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.728+0000 7fa1ea9aa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f0765b0 0x55a74f064a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.728+0000 7fa1ea9aa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e0200090 con 0x55a74f0761d0 2024-03-28T13:18:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.728+0000 7fa1ea9aa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e0156010 con 0x55a74f0765b0 2024-03-28T13:18:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.728+0000 7fa1ea9aa740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e01ffa40 con 0x55a74f065700 2024-03-28T13:18:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.728+0000 7fa1d7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0761d0 0x55a74f050190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.728+0000 7fa1d7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0761d0 0x55a74f050190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45170/0 (socket says 172.21.15.19:45170) 2024-03-28T13:18:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.728+0000 7fa1d7e00640 1 -- 172.21.15.19:0/944020000 learned_addr learned my addr 172.21.15.19:0/944020000 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.729+0000 7fa1e5200640 1 --2- 172.21.15.19:0/944020000 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f065700 0x55a74f04cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.729+0000 7fa1d7e00640 1 -- 172.21.15.19:0/944020000 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f065700 msgr2=0x55a74f04cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.729+0000 7fa1d7e00640 1 --2- 172.21.15.19:0/944020000 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f065700 0x55a74f04cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.729+0000 7fa1d7e00640 1 -- 172.21.15.19:0/944020000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f0765b0 msgr2=0x55a74f064a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.729+0000 7fa1d7e00640 1 --2- 172.21.15.19:0/944020000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f0765b0 0x55a74f064a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.729+0000 7fa1d7e00640 1 -- 172.21.15.19:0/944020000 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a74f04f650 con 0x55a74f0761d0 2024-03-28T13:18:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.729+0000 7fa1d7e00640 1 --2- 172.21.15.19:0/944020000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0761d0 0x55a74f050190 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x55a74f07ca30 tx=0x7fa1c8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b60d1daf5611ea2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.729+0000 7fa1d4c00640 1 -- 172.21.15.19:0/944020000 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8004030 con 0x55a74f0761d0 2024-03-28T13:18:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.729+0000 7fa1d4c00640 1 -- 172.21.15.19:0/944020000 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c8024070 con 0x55a74f0761d0 2024-03-28T13:18:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.729+0000 7fa1d4c00640 1 -- 172.21.15.19:0/944020000 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c80043d0 con 0x55a74f0761d0 2024-03-28T13:18:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.730+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/944020000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0761d0 msgr2=0x55a74f050190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.730+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/944020000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0761d0 0x55a74f050190 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x55a74f07ca30 tx=0x7fa1c8008040 comp rx=0 tx=0).stop 2024-03-28T13:18:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.730+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/944020000 shutdown_connections 2024-03-28T13:18:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.730+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/944020000 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f0765b0 0x55a74f064a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.730+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/944020000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0761d0 0x55a74f050190 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.730+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/944020000 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f065700 0x55a74f04cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.730+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/944020000 >> 172.21.15.19:0/944020000 conn(0x55a74f0496b0 msgr2=0x55a74f094970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.730+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/944020000 shutdown_connections 2024-03-28T13:18:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.730+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/944020000 wait complete. 2024-03-28T13:18:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.731+0000 7fa1ea9aa740 1 -- start start 2024-03-28T13:18:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.731+0000 7fa1ea9aa740 1 Processor -- start 2024-03-28T13:18:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.732+0000 7fa1ea9aa740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f065700 0x55a74f100440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.732+0000 7fa1ea9aa740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f0761d0 0x55a74f073da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.732+0000 7fa1ea9aa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0765b0 0x55a74f0b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.732+0000 7fa1ea9aa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a74f07b020 con 0x55a74f0765b0 2024-03-28T13:18:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.732+0000 7fa1ea9aa740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a74f07ab50 con 0x55a74f065700 2024-03-28T13:18:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.732+0000 7fa1ea9aa740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55a74f07b690 con 0x55a74f0761d0 2024-03-28T13:18:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.732+0000 7fa1ea9aa740 10 client.0 _ll_register_callbacks cb 0x7fa1d00050d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:18:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.733+0000 7fa1e5c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0765b0 0x55a74f0b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.733+0000 7fa1e5c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0765b0 0x55a74f0b7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45180/0 (socket says 172.21.15.19:45180) 2024-03-28T13:18:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.733+0000 7fa1e5c00640 1 -- 172.21.15.19:0/3656095307 learned_addr learned my addr 172.21.15.19:0/3656095307 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.733+0000 7fa1e5200640 1 --2- 172.21.15.19:0/3656095307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f065700 0x55a74f100440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.733+0000 7fa1d7e00640 1 --2- 172.21.15.19:0/3656095307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f0761d0 0x55a74f073da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.733+0000 7fa1e5c00640 1 -- 172.21.15.19:0/3656095307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f0761d0 msgr2=0x55a74f073da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.733+0000 7fa1e5c00640 1 --2- 172.21.15.19:0/3656095307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f0761d0 0x55a74f073da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.733+0000 7fa1e5c00640 1 -- 172.21.15.19:0/3656095307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f065700 msgr2=0x55a74f100440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.733+0000 7fa1e5c00640 1 --2- 172.21.15.19:0/3656095307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f065700 0x55a74f100440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.733+0000 7fa1e5c00640 1 -- 172.21.15.19:0/3656095307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a74f08c480 con 0x55a74f0765b0 2024-03-28T13:18:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.733+0000 7fa1e5c00640 1 --2- 172.21.15.19:0/3656095307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0765b0 0x55a74f0b7180 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8005c20 tx=0x7fa1d80011c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7a1d230e3918cb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.733+0000 7fa1ce000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:18:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.734+0000 7fa1ce000640 1 -- 172.21.15.19:0/3656095307 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80aa070 con 0x55a74f0765b0 2024-03-28T13:18:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.734+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/3656095307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1c8024070 con 0x55a74f0765b0 2024-03-28T13:18:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.734+0000 7fa1ce000640 1 -- 172.21.15.19:0/3656095307 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7fa1d807adb0 con 0x55a74f0765b0 2024-03-28T13:18:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.734+0000 7fa1ce000640 1 -- 172.21.15.19:0/3656095307 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fa1d807fd10 con 0x55a74f0765b0 2024-03-28T13:18:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.734+0000 7fa1ea9aa740 10 client.7752 fetch_fsmap learned FSMap version 74 2024-03-28T13:18:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.734+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/3656095307 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55a74f0744b0 con 0x55a74f0765b0 2024-03-28T13:18:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.734+0000 7fa1ce000640 1 -- 172.21.15.19:0/3656095307 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c8024070 con 0x55a74f0765b0 2024-03-28T13:18:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.734+0000 7fa1ce000640 1 -- 172.21.15.19:0/3656095307 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7fa1d80776b0 con 0x55a74f0765b0 2024-03-28T13:18:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.734+0000 7fa1ea9aa740 10 client.7752 fetch_fsmap finished waiting for FSMap version 74 2024-03-28T13:18:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.734+0000 7fa1ea9aa740 10 client.7752 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:18:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.734+0000 7fa1ea9aa740 20 client.7752 populate_metadata read hostname 'smithi019' 2024-03-28T13:18:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.734+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/3656095307 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a74f064bf0 0x55a74f064fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.734+0000 7fa1ea9aa740 4 client.7752 mds_command: new command op to 6040 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T13:18:01.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.734+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/3656095307 --> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] -- command(tid 0: {"prefix": "status"}) -- 0x55a74efba820 con 0x55a74f064bf0 2024-03-28T13:18:01.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.735+0000 7fa1e5200640 1 --2- 172.21.15.19:0/3656095307 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a74f064bf0 0x55a74f064fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.735+0000 7fa1e5200640 1 --2- 172.21.15.19:0/3656095307 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a74f064bf0 0x55a74f064fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee9bf758c8d28a9 server_cookie=aa451df2165039cc in_seq=0 out_seq=0 2024-03-28T13:18:01.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.735+0000 7fa1ce000640 10 client.7752 ms_handle_connect on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:01.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.736+0000 7fa1ce000640 1 -- 172.21.15.19:0/3656095307 <== mds.0 v2:172.21.15.19:6832/2209799371 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x55a74efba820 con 0x55a74f064bf0 2024-03-28T13:18:01.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.736+0000 7fa1ce000640 10 client.7752 handle_command_reply: tid=0 2024-03-28T13:18:01.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.736+0000 7fa1ea9aa740 1 client.7752 shutdown 2024-03-28T13:18:01.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.736+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/3656095307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0765b0 msgr2=0x55a74f0b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.736+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/3656095307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0765b0 0x55a74f0b7180 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8005c20 tx=0x7fa1d80011c0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.736+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/3656095307 shutdown_connections 2024-03-28T13:18:01.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.736+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/3656095307 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a74f064bf0 0x55a74f064fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.736+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/3656095307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a74f0765b0 0x55a74f0b7180 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.736+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/3656095307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a74f0761d0 0x55a74f073da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.736+0000 7fa1ea9aa740 1 --2- 172.21.15.19:0/3656095307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a74f065700 0x55a74f100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.736+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/3656095307 >> 172.21.15.19:0/3656095307 conn(0x55a74f0496b0 msgr2=0x55a74f044460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:01.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.736+0000 7fa1ce000640 0 client.7752 ms_handle_reset on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:01.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.736+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/3656095307 shutdown_connections 2024-03-28T13:18:01.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.736+0000 7fa1ea9aa740 1 -- 172.21.15.19:0/3656095307 wait complete. 2024-03-28T13:18:01.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.736+0000 7fa1ea9aa740 20 client.7752 trim_cache size 0 max 16384 2024-03-28T13:18:01.743 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:18:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "e6b569d9-f2ce-49dc-bbe7-c0ab33080203", 2024-03-28T13:18:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-03-28T13:18:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 6040, 2024-03-28T13:18:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-03-28T13:18:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-03-28T13:18:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-03-28T13:18:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 5.036048783, 2024-03-28T13:18:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 74, 2024-03-28T13:18:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 94, 2024-03-28T13:18:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 94, 2024-03-28T13:18:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 101.528333954 2024-03-28T13:18:01.744 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:18:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.737+0000 7fa1e6600640 1 -- 172.21.15.19:0/2123303742 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa1bc06c5b0 msgr2=0x7fa1bc06ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.737+0000 7fa1e6600640 1 --2- 172.21.15.19:0/2123303742 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa1bc06c5b0 0x7fa1bc06ea70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0007fd0 tx=0x7fa1d0009040 comp rx=0 tx=0).stop 2024-03-28T13:18:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.737+0000 7fa1e6600640 1 -- 172.21.15.19:0/2123303742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1e014e680 msgr2=0x7fa1e0159700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.737+0000 7fa1e6600640 1 --2- 172.21.15.19:0/2123303742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1e014e680 0x7fa1e0159700 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800a770 tx=0x7fa1e00d3450 comp rx=0 tx=0).stop 2024-03-28T13:18:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.737+0000 7fa1e6600640 1 -- 172.21.15.19:0/2123303742 shutdown_connections 2024-03-28T13:18:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.737+0000 7fa1e6600640 1 --2- 172.21.15.19:0/2123303742 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa1bc06c5b0 0x7fa1bc06ea70 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.737+0000 7fa1e6600640 1 --2- 172.21.15.19:0/2123303742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1e014e680 0x7fa1e0159700 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.737+0000 7fa1e6600640 1 --2- 172.21.15.19:0/2123303742 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1e01604a0 0x7fa1e01535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.737+0000 7fa1e6600640 1 --2- 172.21.15.19:0/2123303742 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1c4004d40 0x7fa1e0153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.737+0000 7fa1e6600640 1 -- 172.21.15.19:0/2123303742 >> 172.21.15.19:0/2123303742 conn(0x7fa1e00676d0 msgr2=0x7fa1e015d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:01.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.737+0000 7fa1e6600640 1 -- 172.21.15.19:0/2123303742 shutdown_connections 2024-03-28T13:18:01.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.738+0000 7fa1e6600640 1 -- 172.21.15.19:0/2123303742 wait complete. 2024-03-28T13:18:01.761 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:18:01.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.979+0000 7f706dc00640 1 Processor -- start 2024-03-28T13:18:01.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.980+0000 7f706dc00640 1 -- start start 2024-03-28T13:18:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.981+0000 7f706dc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f70680d1380 0x7f70680d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.981+0000 7f706dc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f70680d3a70 0x7f70680d3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.981+0000 7f706dc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70680d2fd0 con 0x7f706815e420 2024-03-28T13:18:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.981+0000 7f706dc00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f70680cfa70 con 0x7f70680d3a70 2024-03-28T13:18:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.981+0000 7f706dc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f70680cf790 con 0x7f70680d1380 2024-03-28T13:18:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.981+0000 7f706d200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f70680d3a70 0x7f70680d3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.981+0000 7f706d200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f70680d3a70 0x7f70680d3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45196/0 (socket says 172.21.15.19:45196) 2024-03-28T13:18:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.981+0000 7f706d200640 1 -- 172.21.15.19:0/2961004701 learned_addr learned my addr 172.21.15.19:0/2961004701 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:01.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.981+0000 7f705f400640 1 --2- 172.21.15.19:0/2961004701 >> v2:172.21.15.188:3300/0 conn(0x7f70680d1380 0x7f70680d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.982+0000 7f706d200640 1 -- 172.21.15.19:0/2961004701 >> v2:172.21.15.188:3300/0 conn(0x7f70680d1380 msgr2=0x7f70680d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:01.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.982+0000 7f706d200640 1 --2- 172.21.15.19:0/2961004701 >> v2:172.21.15.188:3300/0 conn(0x7f70680d1380 0x7f70680d1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.982+0000 7f706d200640 1 -- 172.21.15.19:0/2961004701 >> v1:172.21.15.188:6789/0 conn(0x7f706815e420 legacy=0x7f70680b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:01.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.982+0000 7f706d200640 1 -- 172.21.15.19:0/2961004701 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70680d3270 con 0x7f70680d3a70 2024-03-28T13:18:01.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.983+0000 7f706d200640 1 --2- 172.21.15.19:0/2961004701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70680d3a70 0x7f70680d3e80 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f70680d31a0 tx=0x7f706002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91fb3c384272b31e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:01.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.983+0000 7f7067400640 1 -- 172.21.15.19:0/2961004701 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060034070 con 0x7f70680d3a70 2024-03-28T13:18:01.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.983+0000 7f7067400640 1 -- 172.21.15.19:0/2961004701 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7060030860 con 0x7f70680d3a70 2024-03-28T13:18:01.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.983+0000 7f7067400640 1 -- 172.21.15.19:0/2961004701 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060030c30 con 0x7f70680d3a70 2024-03-28T13:18:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.984+0000 7f706dc00640 1 -- 172.21.15.19:0/2961004701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70680d3a70 msgr2=0x7f70680d3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.984+0000 7f706dc00640 1 --2- 172.21.15.19:0/2961004701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70680d3a70 0x7f70680d3e80 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f70680d31a0 tx=0x7f706002b9a0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.984+0000 7f706dc00640 1 -- 172.21.15.19:0/2961004701 shutdown_connections 2024-03-28T13:18:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.984+0000 7f706dc00640 1 --2- 172.21.15.19:0/2961004701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70680d3a70 0x7f70680d3e80 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.984+0000 7f706dc00640 1 --2- 172.21.15.19:0/2961004701 >> v2:172.21.15.188:3300/0 conn(0x7f70680d1380 0x7f70680d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.984+0000 7f706dc00640 1 -- 172.21.15.19:0/2961004701 >> 172.21.15.19:0/2961004701 conn(0x7f7068067670 msgr2=0x7f7068067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:01.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.984+0000 7f706dc00640 1 -- 172.21.15.19:0/2961004701 shutdown_connections 2024-03-28T13:18:01.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.984+0000 7f706dc00640 1 -- 172.21.15.19:0/2961004701 wait complete. 2024-03-28T13:18:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.986+0000 7f706dc00640 1 Processor -- start 2024-03-28T13:18:01.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.986+0000 7f706dc00640 1 -- start start 2024-03-28T13:18:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.987+0000 7f706dc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70680d1380 0x7f706815b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.987+0000 7f706dc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70680d3a70 0x7f70681e8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.987+0000 7f706dc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f706815e420 0x7f7068156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.987+0000 7f706dc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70680cf420 con 0x7f70680d3a70 2024-03-28T13:18:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.987+0000 7f706dc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f706815e800 con 0x7f70680d1380 2024-03-28T13:18:01.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.987+0000 7f706dc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f70680b3630 con 0x7f706815e420 2024-03-28T13:18:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.987+0000 7f705f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70680d3a70 0x7f70681e8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.987+0000 7f705f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70680d3a70 0x7f70681e8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45210/0 (socket says 172.21.15.19:45210) 2024-03-28T13:18:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.987+0000 7f705f400640 1 -- 172.21.15.19:0/3931775782 learned_addr learned my addr 172.21.15.19:0/3931775782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.988+0000 7f7067e00640 1 --2- 172.21.15.19:0/3931775782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70680d1380 0x7f706815b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.988+0000 7f706d200640 1 --2- 172.21.15.19:0/3931775782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f706815e420 0x7f7068156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.988+0000 7f705f400640 1 -- 172.21.15.19:0/3931775782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f706815e420 msgr2=0x7f7068156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.988+0000 7f705f400640 1 --2- 172.21.15.19:0/3931775782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f706815e420 0x7f7068156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.988+0000 7f705f400640 1 -- 172.21.15.19:0/3931775782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70680d1380 msgr2=0x7f706815b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.988+0000 7f705f400640 1 --2- 172.21.15.19:0/3931775782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70680d1380 0x7f706815b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.988+0000 7f705f400640 1 -- 172.21.15.19:0/3931775782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70681e94c0 con 0x7f70680d3a70 2024-03-28T13:18:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.988+0000 7f7067e00640 1 --2- 172.21.15.19:0/3931775782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70680d1380 0x7f706815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.988+0000 7f705f400640 1 --2- 172.21.15.19:0/3931775782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70680d3a70 0x7f70681e8820 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f705000d800 tx=0x7f705000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.988+0000 7f7064c00640 1 -- 172.21.15.19:0/3931775782 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705000b6c0 con 0x7f70680d3a70 2024-03-28T13:18:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.988+0000 7f7064c00640 1 -- 172.21.15.19:0/3931775782 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7050019070 con 0x7f70680d3a70 2024-03-28T13:18:01.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.988+0000 7f7064c00640 1 -- 172.21.15.19:0/3931775782 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705000ba80 con 0x7f70680d3a70 2024-03-28T13:18:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.988+0000 7f706dc00640 1 -- 172.21.15.19:0/3931775782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70681e8f00 con 0x7f70680d3a70 2024-03-28T13:18:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.989+0000 7f706dc00640 1 -- 172.21.15.19:0/3931775782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f706815fe40 con 0x7f70680d3a70 2024-03-28T13:18:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.990+0000 7f706dc00640 1 -- 172.21.15.19:0/3931775782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7068075940 con 0x7f70680d3a70 2024-03-28T13:18:01.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.991+0000 7f7064c00640 1 -- 172.21.15.19:0/3931775782 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f705000bc20 con 0x7f70680d3a70 2024-03-28T13:18:01.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.991+0000 7f7064c00640 1 --2- 172.21.15.19:0/3931775782 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f704406c530 0x7f704406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:01.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.992+0000 7f7064c00640 1 -- 172.21.15.19:0/3931775782 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f705008bd80 con 0x7f70680d3a70 2024-03-28T13:18:01.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.992+0000 7f7067e00640 1 --2- 172.21.15.19:0/3931775782 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f704406c530 0x7f704406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:01.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.993+0000 7f7067e00640 1 --2- 172.21.15.19:0/3931775782 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f704406c530 0x7f704406e9f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7058007e90 tx=0x7f70580073a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:01.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:01.997+0000 7f7064c00640 1 -- 172.21.15.19:0/3931775782 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f705005a1c0 con 0x7f70680d3a70 2024-03-28T13:18:02.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02.241+0000 7f706dc00640 1 -- 172.21.15.19:0/3931775782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f706815d960 con 0x7f70680d3a70 2024-03-28T13:18:02.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02.242+0000 7f7064c00640 1 -- 172.21.15.19:0/3931775782 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7f7050059ac0 con 0x7f70680d3a70 2024-03-28T13:18:02.242 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:18:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02.244+0000 7f706dc00640 1 -- 172.21.15.19:0/3931775782 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f704406c530 msgr2=0x7f704406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02.244+0000 7f706dc00640 1 --2- 172.21.15.19:0/3931775782 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f704406c530 0x7f704406e9f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7058007e90 tx=0x7f70580073a0 comp rx=0 tx=0).stop 2024-03-28T13:18:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02.244+0000 7f706dc00640 1 -- 172.21.15.19:0/3931775782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70680d3a70 msgr2=0x7f70681e8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02.244+0000 7f706dc00640 1 --2- 172.21.15.19:0/3931775782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70680d3a70 0x7f70681e8820 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f705000d800 tx=0x7f705000dcc0 comp rx=0 tx=0).stop 2024-03-28T13:18:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02.244+0000 7f706dc00640 1 -- 172.21.15.19:0/3931775782 shutdown_connections 2024-03-28T13:18:02.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02.244+0000 7f706dc00640 1 --2- 172.21.15.19:0/3931775782 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f704406c530 0x7f704406e9f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02.244+0000 7f706dc00640 1 --2- 172.21.15.19:0/3931775782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f706815e420 0x7f7068156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02.244+0000 7f706dc00640 1 --2- 172.21.15.19:0/3931775782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70680d3a70 0x7f70681e8820 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02.244+0000 7f706dc00640 1 --2- 172.21.15.19:0/3931775782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70680d1380 0x7f706815b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02.244+0000 7f706dc00640 1 -- 172.21.15.19:0/3931775782 >> 172.21.15.19:0/3931775782 conn(0x7f7068067670 msgr2=0x7f7068067a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02.245+0000 7f706dc00640 1 -- 172.21.15.19:0/3931775782 shutdown_connections 2024-03-28T13:18:02.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02.245+0000 7f706dc00640 1 -- 172.21.15.19:0/3931775782 wait complete. 2024-03-28T13:18:02.273 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-28T13:18:02.273 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:18:02.274 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:18:02.274 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:18:02.274 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:18:02.274 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-28T13:18:02.274 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-28T13:18:02.274 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:18:02.274 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:18:02.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:18:02.274 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:18:02.331 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:18:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:18:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:18:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:18:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:18:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:18:02.332 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:18:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:18:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:18:02.332 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:18:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:18:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:18:02.332 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:18:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:18:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:18:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:18:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42072sec preferred_lft 42072sec 2024-03-28T13:18:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:18:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:18:02.333 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:18:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:18:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:18:02.333 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:18:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:18:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:18:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:18:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:18:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:18:02.333 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:18:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:18:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:18:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:18:02.334 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:18:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether e6:17:c5:8e:2b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:18:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::e417:c5ff:fe8e:2bcf/64 scope link 2024-03-28T13:18:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:18:02.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:18:02.334 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:18:02.420 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:18:02.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:18:02.424 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:18:02.478 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:18:02.478 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:18:02.478 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:18:02.478 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:18:02.478 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:18:02.479 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:18:02.479 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:18:02.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:18:02.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:18:02.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:18:02.479 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:18:02.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:18:02.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:18:02.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:18:02.728 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:18:02.780 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:18:02.781 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:18:02.782 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:18:02.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:18:02.782 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:18:02.884 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:18:02.884 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:18:02.947 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:18:02.948 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:18:02.948 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:18:03.010 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:18:03.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:18:03.010 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:18:03.148 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:18:03.148 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:18:03.148 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:18:03.169 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:18:03.170 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:18:03.170 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:18:03.170 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:18:03.170 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:18:03.196 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:18:03.196 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:18:03.196 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:18:03.196 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:18:03.196 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:18:03.196 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:18:03.197 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:18:03.197 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:18:03.199 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:18:03.199 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:18:03.199 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:18:03.199 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:18:03.199 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:18:03.203 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:18:03.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:18:03.204 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:18:03.285 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:18:03.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:18:03.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:18:03.400 INFO:teuthology.orchestra.run.smithi019.stdout:client6406 2024-03-28T13:18:03.400 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:18:03.400 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:18:03.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:18:03.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-03-28T13:18:03.589 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:18:03.596 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:18:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.748+0000 7f6e24200640 1 Processor -- start 2024-03-28T13:18:03.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.748+0000 7f6e24200640 1 -- start start 2024-03-28T13:18:03.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.749+0000 7f6e24200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6e1c161f10 0x7f6e1c0d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.750+0000 7f6e24200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e1c06d370 con 0x7f6e1c0b3270 2024-03-28T13:18:03.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.750+0000 7f6e24200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e1c0d0f00 con 0x7f6e1c06e110 2024-03-28T13:18:03.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.750+0000 7f6e24200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6e1c06d650 con 0x7f6e1c161f10 2024-03-28T13:18:03.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.750+0000 7f6e21800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6e1c161f10 0x7f6e1c0d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.750+0000 7f6e20e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6e1c0b3270 0x7f6e1c06da00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37354/0 (socket says 172.21.15.19:37354) 2024-03-28T13:18:03.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.750+0000 7f6e20e00640 1 -- 172.21.15.19:0/285784589 learned_addr learned my addr 172.21.15.19:0/285784589 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:03.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.750+0000 7f6e21800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6e1c161f10 0x7f6e1c0d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45216/0 (socket says 172.21.15.19:45216) 2024-03-28T13:18:03.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.751+0000 7f6e21800640 1 -- 172.21.15.19:0/285784589 >> v1:172.21.15.188:6789/0 conn(0x7f6e1c06e110 legacy=0x7f6e1c0d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.751+0000 7f6e21800640 1 -- 172.21.15.19:0/285784589 >> v1:172.21.15.19:6789/0 conn(0x7f6e1c0b3270 legacy=0x7f6e1c06da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.751+0000 7f6e21800640 1 -- 172.21.15.19:0/285784589 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1c0d11e0 con 0x7f6e1c161f10 2024-03-28T13:18:03.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.752+0000 7f6e21800640 1 --2- 172.21.15.19:0/285784589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c161f10 0x7f6e1c0d19e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f6e100097a0 tx=0x7f6e1002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f52291f38f0a2e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.752+0000 7f6e1be00640 1 -- 172.21.15.19:0/285784589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10035020 con 0x7f6e1c161f10 2024-03-28T13:18:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.752+0000 7f6e1be00640 1 -- 172.21.15.19:0/285784589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e10031830 con 0x7f6e1c161f10 2024-03-28T13:18:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.752+0000 7f6e1be00640 1 -- 172.21.15.19:0/285784589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e10031c00 con 0x7f6e1c161f10 2024-03-28T13:18:03.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.753+0000 7f6e24200640 1 -- 172.21.15.19:0/285784589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c161f10 msgr2=0x7f6e1c0d19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:03.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.753+0000 7f6e24200640 1 --2- 172.21.15.19:0/285784589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c161f10 0x7f6e1c0d19e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f6e100097a0 tx=0x7f6e1002b8d0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.753+0000 7f6e24200640 1 -- 172.21.15.19:0/285784589 shutdown_connections 2024-03-28T13:18:03.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.753+0000 7f6e24200640 1 --2- 172.21.15.19:0/285784589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c161f10 0x7f6e1c0d19e0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.753+0000 7f6e24200640 1 -- 172.21.15.19:0/285784589 >> 172.21.15.19:0/285784589 conn(0x7f6e1c08bfb0 msgr2=0x7f6e1c0d4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:03.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.753+0000 7f6e24200640 1 -- 172.21.15.19:0/285784589 shutdown_connections 2024-03-28T13:18:03.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.753+0000 7f6e24200640 1 -- 172.21.15.19:0/285784589 wait complete. 2024-03-28T13:18:03.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.755+0000 7f6e24200640 1 Processor -- start 2024-03-28T13:18:03.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.755+0000 7f6e24200640 1 -- start start 2024-03-28T13:18:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.756+0000 7f6e24200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e1c06e110 0x7f6e1c0b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.756+0000 7f6e24200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c0b3270 0x7f6e1c15a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.756+0000 7f6e24200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e1c161f10 0x7f6e1c0b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.756+0000 7f6e24200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c06cff0 con 0x7f6e1c0b3270 2024-03-28T13:18:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.756+0000 7f6e24200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c1622f0 con 0x7f6e1c06e110 2024-03-28T13:18:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.756+0000 7f6e24200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6e1c0d4b10 con 0x7f6e1c161f10 2024-03-28T13:18:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.756+0000 7f6e20e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c0b3270 0x7f6e1c15a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.756+0000 7f6e20e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c0b3270 0x7f6e1c15a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45230/0 (socket says 172.21.15.19:45230) 2024-03-28T13:18:03.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.756+0000 7f6e20e00640 1 -- 172.21.15.19:0/3638083890 learned_addr learned my addr 172.21.15.19:0/3638083890 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.757+0000 7f6e21800640 1 --2- 172.21.15.19:0/3638083890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e1c06e110 0x7f6e1c0b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.757+0000 7f6e22200640 1 --2- 172.21.15.19:0/3638083890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e1c161f10 0x7f6e1c0b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.757+0000 7f6e20e00640 1 -- 172.21.15.19:0/3638083890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e1c161f10 msgr2=0x7f6e1c0b7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.757+0000 7f6e20e00640 1 --2- 172.21.15.19:0/3638083890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e1c161f10 0x7f6e1c0b7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.757+0000 7f6e20e00640 1 -- 172.21.15.19:0/3638083890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e1c06e110 msgr2=0x7f6e1c0b4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.757+0000 7f6e20e00640 1 --2- 172.21.15.19:0/3638083890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e1c06e110 0x7f6e1c0b4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.757+0000 7f6e20e00640 1 -- 172.21.15.19:0/3638083890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1c0b5700 con 0x7f6e1c0b3270 2024-03-28T13:18:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.757+0000 7f6e21800640 1 --2- 172.21.15.19:0/3638083890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e1c06e110 0x7f6e1c0b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.757+0000 7f6e20e00640 1 --2- 172.21.15.19:0/3638083890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c0b3270 0x7f6e1c15a4a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c430 tx=0x7f6e0800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:03.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.758+0000 7f6e19600640 1 -- 172.21.15.19:0/3638083890 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08017070 con 0x7f6e1c0b3270 2024-03-28T13:18:03.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.758+0000 7f6e19600640 1 -- 172.21.15.19:0/3638083890 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e08005ce0 con 0x7f6e1c0b3270 2024-03-28T13:18:03.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.758+0000 7f6e19600640 1 -- 172.21.15.19:0/3638083890 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e080072f0 con 0x7f6e1c0b3270 2024-03-28T13:18:03.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.758+0000 7f6e24200640 1 -- 172.21.15.19:0/3638083890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1c0b59e0 con 0x7f6e1c0b3270 2024-03-28T13:18:03.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.758+0000 7f6e24200640 1 -- 172.21.15.19:0/3638083890 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e1c0cb4d0 con 0x7f6e1c0b3270 2024-03-28T13:18:03.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.761+0000 7f6e19600640 1 -- 172.21.15.19:0/3638083890 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6e0801a070 con 0x7f6e1c0b3270 2024-03-28T13:18:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.762+0000 7f6e19600640 1 --2- 172.21.15.19:0/3638083890 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6dfc06c530 0x7f6dfc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.762+0000 7f6e19600640 1 -- 172.21.15.19:0/3638083890 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f6e0808ab80 con 0x7f6e1c0b3270 2024-03-28T13:18:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.762+0000 7f6e21800640 1 --2- 172.21.15.19:0/3638083890 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6dfc06c530 0x7f6dfc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.763+0000 7f6e26fbd740 1 Processor -- start 2024-03-28T13:18:03.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.763+0000 7f6e26fbd740 1 -- start start 2024-03-28T13:18:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.764+0000 7f6e21800640 1 --2- 172.21.15.19:0/3638083890 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6dfc06c530 0x7f6dfc06e9f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6e10036000 tx=0x7f6e1003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.764+0000 7f6e26fbd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10631fc0 0x564a106323a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.764+0000 7f6e26fbd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10637de0 0x564a10636010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.764+0000 7f6e26fbd740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a1067f6d0 0x564a1067fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.764+0000 7f6e26fbd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564a1059c280 con 0x564a10631fc0 2024-03-28T13:18:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.764+0000 7f6e26fbd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564a105446f0 con 0x564a10637de0 2024-03-28T13:18:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.764+0000 7f6e26fbd740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x564a10598230 con 0x564a1067f6d0 2024-03-28T13:18:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.764+0000 7f6e21800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10631fc0 0x564a106323a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.764+0000 7f6e21800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10631fc0 0x564a106323a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45232/0 (socket says 172.21.15.19:45232) 2024-03-28T13:18:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.764+0000 7f6e21800640 1 -- 172.21.15.19:0/558931337 learned_addr learned my addr 172.21.15.19:0/558931337 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.764+0000 7f6e20e00640 1 --2- 172.21.15.19:0/558931337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10637de0 0x564a10636010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.764+0000 7f6e22200640 1 --2- 172.21.15.19:0/558931337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a1067f6d0 0x564a1067fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.764+0000 7f6e21800640 1 -- 172.21.15.19:0/558931337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a1067f6d0 msgr2=0x564a1067fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.765+0000 7f6e21800640 1 --2- 172.21.15.19:0/558931337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a1067f6d0 0x564a1067fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.765+0000 7f6e21800640 1 -- 172.21.15.19:0/558931337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10637de0 msgr2=0x564a10636010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.765+0000 7f6e21800640 1 --2- 172.21.15.19:0/558931337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10637de0 0x564a10636010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.765+0000 7f6e21800640 1 -- 172.21.15.19:0/558931337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a103787e0 con 0x564a10631fc0 2024-03-28T13:18:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.765+0000 7f6e22200640 1 --2- 172.21.15.19:0/558931337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a1067f6d0 0x564a1067fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.765+0000 7f6e21800640 1 --2- 172.21.15.19:0/558931337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10631fc0 0x564a106323a0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6e1002bee0 tx=0x7f6e10030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6042d81408133d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:03.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.765+0000 7f6e0e000640 1 -- 172.21.15.19:0/558931337 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1003a970 con 0x564a10631fc0 2024-03-28T13:18:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.765+0000 7f6e0e000640 1 -- 172.21.15.19:0/558931337 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1003ab10 con 0x564a10631fc0 2024-03-28T13:18:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.765+0000 7f6e0e000640 1 -- 172.21.15.19:0/558931337 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1004f430 con 0x564a10631fc0 2024-03-28T13:18:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.766+0000 7f6e26fbd740 1 -- 172.21.15.19:0/558931337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10631fc0 msgr2=0x564a106323a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.766+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/558931337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10631fc0 0x564a106323a0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f6e1002bee0 tx=0x7f6e10030000 comp rx=0 tx=0).stop 2024-03-28T13:18:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.766+0000 7f6e26fbd740 1 -- 172.21.15.19:0/558931337 shutdown_connections 2024-03-28T13:18:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.766+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/558931337 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a1067f6d0 0x564a1067fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.766+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/558931337 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10637de0 0x564a10636010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.766+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/558931337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10631fc0 0x564a106323a0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.766+0000 7f6e26fbd740 1 -- 172.21.15.19:0/558931337 >> 172.21.15.19:0/558931337 conn(0x564a10634040 msgr2=0x564a10635900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.766+0000 7f6e26fbd740 1 -- 172.21.15.19:0/558931337 shutdown_connections 2024-03-28T13:18:03.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.766+0000 7f6e26fbd740 1 -- 172.21.15.19:0/558931337 wait complete. 2024-03-28T13:18:03.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.767+0000 7f6e26fbd740 1 -- start start 2024-03-28T13:18:03.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.767+0000 7f6e26fbd740 1 Processor -- start 2024-03-28T13:18:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.768+0000 7f6e26fbd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10637de0 0x564a10665f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.768+0000 7f6e26fbd740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a1067f6d0 0x564a10659090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.768+0000 7f6e26fbd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10657980 0x564a1065aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.768+0000 7f6e26fbd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564a1065e730 con 0x564a10637de0 2024-03-28T13:18:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.768+0000 7f6e26fbd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x564a10557aa0 con 0x564a10657980 2024-03-28T13:18:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.768+0000 7f6e26fbd740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x564a1065e980 con 0x564a1067f6d0 2024-03-28T13:18:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.768+0000 7f6e26fbd740 10 client.0 _ll_register_callbacks cb 0x564a10557490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:18:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.768+0000 7f6e21800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10637de0 0x564a10665f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.769+0000 7f6e21800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10637de0 0x564a10665f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45236/0 (socket says 172.21.15.19:45236) 2024-03-28T13:18:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.769+0000 7f6e21800640 1 -- 172.21.15.19:0/2578610503 learned_addr learned my addr 172.21.15.19:0/2578610503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.769+0000 7f6e22200640 1 --2- 172.21.15.19:0/2578610503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10657980 0x564a1065aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.769+0000 7f6e20e00640 1 --2- 172.21.15.19:0/2578610503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a1067f6d0 0x564a10659090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.769+0000 7f6e21800640 1 -- 172.21.15.19:0/2578610503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a1067f6d0 msgr2=0x564a10659090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:03.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.769+0000 7f6e21800640 1 --2- 172.21.15.19:0/2578610503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a1067f6d0 0x564a10659090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.769+0000 7f6e21800640 1 -- 172.21.15.19:0/2578610503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10657980 msgr2=0x564a1065aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.769+0000 7f6e21800640 1 --2- 172.21.15.19:0/2578610503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10657980 0x564a1065aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.769+0000 7f6e21800640 1 -- 172.21.15.19:0/2578610503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a103d8f70 con 0x564a10637de0 2024-03-28T13:18:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.769+0000 7f6e22200640 1 --2- 172.21.15.19:0/2578610503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10657980 0x564a1065aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.769+0000 7f6e21800640 1 --2- 172.21.15.19:0/2578610503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10637de0 0x564a10665f10 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6e10034040 tx=0x7f6e10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabadc3438b5a378 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.769+0000 7f6e02a00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:18:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.770+0000 7f6e02a00640 1 -- 172.21.15.19:0/2578610503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1005c060 con 0x564a10637de0 2024-03-28T13:18:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.770+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2578610503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e1003ab10 con 0x564a10637de0 2024-03-28T13:18:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.770+0000 7f6e02a00640 1 -- 172.21.15.19:0/2578610503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f6e1004f840 con 0x564a10637de0 2024-03-28T13:18:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.770+0000 7f6e02a00640 1 -- 172.21.15.19:0/2578610503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f6e10062050 con 0x564a10637de0 2024-03-28T13:18:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.770+0000 7f6e26fbd740 10 client.7770 fetch_fsmap learned FSMap version 74 2024-03-28T13:18:03.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.770+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2578610503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x564a10270170 con 0x564a10637de0 2024-03-28T13:18:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.770+0000 7f6e02a00640 1 -- 172.21.15.19:0/2578610503 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e1003ab10 con 0x564a10637de0 2024-03-28T13:18:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.770+0000 7f6e02a00640 1 -- 172.21.15.19:0/2578610503 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f6e10065020 con 0x564a10637de0 2024-03-28T13:18:03.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.770+0000 7f6e26fbd740 10 client.7770 fetch_fsmap finished waiting for FSMap version 74 2024-03-28T13:18:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.770+0000 7f6e26fbd740 10 client.7770 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-28T13:18:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.771+0000 7f6e26fbd740 20 client.7770 populate_metadata read hostname 'smithi019' 2024-03-28T13:18:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.771+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/2578610503 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x564a10665110 0x564a10662ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.771+0000 7f6e26fbd740 4 client.7770 mds_command: new command op to 6040 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:18:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.771+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2578610503 --> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a105a97c0 con 0x564a10665110 2024-03-28T13:18:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.771+0000 7f6e20e00640 1 --2- 172.21.15.19:0/2578610503 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x564a10665110 0x564a10662ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.771+0000 7f6e20e00640 1 --2- 172.21.15.19:0/2578610503 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x564a10665110 0x564a10662ef0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c5f628442fd0907 server_cookie=f04555a1eee06f6c in_seq=0 out_seq=0 2024-03-28T13:18:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.771+0000 7f6e02a00640 10 client.7770 ms_handle_connect on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.772+0000 7f6e02a00640 1 -- 172.21.15.19:0/2578610503 <== mds.0 v2:172.21.15.19:6832/2209799371 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a105a97c0 con 0x564a10665110 2024-03-28T13:18:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.772+0000 7f6e02a00640 10 client.7770 handle_command_reply: tid=0 2024-03-28T13:18:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.772+0000 7f6e26fbd740 1 client.7770 shutdown 2024-03-28T13:18:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.772+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2578610503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10637de0 msgr2=0x564a10665f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.772+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/2578610503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10637de0 0x564a10665f10 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f6e10034040 tx=0x7f6e10004730 comp rx=0 tx=0).stop 2024-03-28T13:18:03.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.773+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2578610503 shutdown_connections 2024-03-28T13:18:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.773+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/2578610503 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x564a10665110 0x564a10662ef0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.773+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/2578610503 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10657980 0x564a1065aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.773+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/2578610503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a1067f6d0 0x564a10659090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.773+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/2578610503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10637de0 0x564a10665f10 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.773+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2578610503 >> 172.21.15.19:0/2578610503 conn(0x564a10631fc0 msgr2=0x564a1065dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.773+0000 7f6e02a00640 0 client.7770 ms_handle_reset on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.773+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2578610503 shutdown_connections 2024-03-28T13:18:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.773+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2578610503 wait complete. 2024-03-28T13:18:03.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.773+0000 7f6e26fbd740 20 client.7770 trim_cache size 0 max 16384 2024-03-28T13:18:03.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.813+0000 7f6e26fbd740 1 Processor -- start 2024-03-28T13:18:03.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.813+0000 7f6e26fbd740 1 -- start start 2024-03-28T13:18:03.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.814+0000 7f6e26fbd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10637de0 0x564a1065dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.814+0000 7f6e26fbd740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a10657980 0x564a106a1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.814+0000 7f6e26fbd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a1067f6d0 0x564a10661d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.814+0000 7f6e26fbd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564a10659c70 con 0x564a1067f6d0 2024-03-28T13:18:03.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.814+0000 7f6e26fbd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c200f30 con 0x564a10637de0 2024-03-28T13:18:03.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.814+0000 7f6e26fbd740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6e1c1ffff0 con 0x564a10657980 2024-03-28T13:18:03.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.815+0000 7f6e22200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a1067f6d0 0x564a10661d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.815+0000 7f6e22200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a1067f6d0 0x564a10661d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45242/0 (socket says 172.21.15.19:45242) 2024-03-28T13:18:03.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.815+0000 7f6e22200640 1 -- 172.21.15.19:0/1434509907 learned_addr learned my addr 172.21.15.19:0/1434509907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:03.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.815+0000 7f6e21800640 1 --2- 172.21.15.19:0/1434509907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10637de0 0x564a1065dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.815+0000 7f6e20e00640 1 --2- 172.21.15.19:0/1434509907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a10657980 0x564a106a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.815+0000 7f6e22200640 1 -- 172.21.15.19:0/1434509907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a10657980 msgr2=0x564a106a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.815+0000 7f6e22200640 1 --2- 172.21.15.19:0/1434509907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a10657980 0x564a106a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.815+0000 7f6e22200640 1 -- 172.21.15.19:0/1434509907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10637de0 msgr2=0x564a1065dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.815+0000 7f6e22200640 1 --2- 172.21.15.19:0/1434509907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10637de0 0x564a1065dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.815+0000 7f6e22200640 1 -- 172.21.15.19:0/1434509907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a1062db10 con 0x564a1067f6d0 2024-03-28T13:18:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.816+0000 7f6e22200640 1 --2- 172.21.15.19:0/1434509907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a1067f6d0 0x564a10661d60 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c1636e0 tx=0x7f6e140047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7814f1a1710695 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:03.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.816+0000 7f6e0d600640 1 -- 172.21.15.19:0/1434509907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e14017020 con 0x564a1067f6d0 2024-03-28T13:18:03.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.816+0000 7f6e0d600640 1 -- 172.21.15.19:0/1434509907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e14004d90 con 0x564a1067f6d0 2024-03-28T13:18:03.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.816+0000 7f6e0d600640 1 -- 172.21.15.19:0/1434509907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1400fae0 con 0x564a1067f6d0 2024-03-28T13:18:03.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.818+0000 7f6e26fbd740 1 -- 172.21.15.19:0/1434509907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a1067f6d0 msgr2=0x564a10661d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.818+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/1434509907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a1067f6d0 0x564a10661d60 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c1636e0 tx=0x7f6e140047d0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.818+0000 7f6e26fbd740 1 -- 172.21.15.19:0/1434509907 shutdown_connections 2024-03-28T13:18:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.818+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/1434509907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a1067f6d0 0x564a10661d60 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.818+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/1434509907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a10657980 0x564a106a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.818+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/1434509907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10637de0 0x564a1065dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.818+0000 7f6e26fbd740 1 -- 172.21.15.19:0/1434509907 >> 172.21.15.19:0/1434509907 conn(0x564a10631fc0 msgr2=0x564a10665110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.818+0000 7f6e26fbd740 1 -- 172.21.15.19:0/1434509907 shutdown_connections 2024-03-28T13:18:03.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.818+0000 7f6e26fbd740 1 -- 172.21.15.19:0/1434509907 wait complete. 2024-03-28T13:18:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.819+0000 7f6e26fbd740 1 -- start start 2024-03-28T13:18:03.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.819+0000 7f6e26fbd740 1 Processor -- start 2024-03-28T13:18:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.820+0000 7f6e26fbd740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10637de0 0x564a106f6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.820+0000 7f6e26fbd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10657980 0x564a1068f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.820+0000 7f6e26fbd740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a1067f6d0 0x564a1068fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.820+0000 7f6e26fbd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c1f8600 con 0x564a10657980 2024-03-28T13:18:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.820+0000 7f6e26fbd740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1c1f7fb0 con 0x564a10637de0 2024-03-28T13:18:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.820+0000 7f6e26fbd740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x564a10660390 con 0x564a1067f6d0 2024-03-28T13:18:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.820+0000 7f6e26fbd740 10 client.0 _ll_register_callbacks cb 0x7f6e10031c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:18:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.820+0000 7f6e20e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10657980 0x564a1068f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.820+0000 7f6e20e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10657980 0x564a1068f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45256/0 (socket says 172.21.15.19:45256) 2024-03-28T13:18:03.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.820+0000 7f6e20e00640 1 -- 172.21.15.19:0/2139268168 learned_addr learned my addr 172.21.15.19:0/2139268168 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.820+0000 7f6e20e00640 1 -- 172.21.15.19:0/2139268168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a1067f6d0 msgr2=0x564a1068fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.820+0000 7f6e20e00640 1 --2- 172.21.15.19:0/2139268168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a1067f6d0 0x564a1068fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.820+0000 7f6e20e00640 1 -- 172.21.15.19:0/2139268168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10637de0 msgr2=0x564a106f6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.820+0000 7f6e20e00640 1 --2- 172.21.15.19:0/2139268168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10637de0 0x564a106f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.820+0000 7f6e20e00640 1 -- 172.21.15.19:0/2139268168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a10661170 con 0x564a10657980 2024-03-28T13:18:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.821+0000 7f6e20e00640 1 --2- 172.21.15.19:0/2139268168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10657980 0x564a1068f820 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f6e08000c00 tx=0x7f6e08059170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac3c081e81888af server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.821+0000 7f6e02a00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:18:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.821+0000 7f6e02a00640 1 -- 172.21.15.19:0/2139268168 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e08055060 con 0x564a10657980 2024-03-28T13:18:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.821+0000 7f6e02a00640 1 -- 172.21.15.19:0/2139268168 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f6e08061140 con 0x564a10657980 2024-03-28T13:18:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.821+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2139268168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e14004d90 con 0x564a10657980 2024-03-28T13:18:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.821+0000 7f6e02a00640 1 -- 172.21.15.19:0/2139268168 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f6e08055200 con 0x564a10657980 2024-03-28T13:18:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.821+0000 7f6e26fbd740 10 client.7776 fetch_fsmap learned FSMap version 74 2024-03-28T13:18:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.821+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2139268168 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x564a106644c0 con 0x564a10657980 2024-03-28T13:18:03.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.822+0000 7f6e02a00640 1 -- 172.21.15.19:0/2139268168 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e14004d90 con 0x564a10657980 2024-03-28T13:18:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.822+0000 7f6e02a00640 1 -- 172.21.15.19:0/2139268168 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f6e0807d080 con 0x564a10657980 2024-03-28T13:18:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.822+0000 7f6e26fbd740 10 client.7776 fetch_fsmap finished waiting for FSMap version 74 2024-03-28T13:18:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.822+0000 7f6e26fbd740 10 client.7776 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-28T13:18:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.822+0000 7f6e26fbd740 20 client.7776 populate_metadata read hostname 'smithi019' 2024-03-28T13:18:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.822+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/2139268168 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x564a1065dc10 0x564a10663140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.822+0000 7f6e26fbd740 4 client.7776 mds_command: new command op to 6040 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-03-28T13:18:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.822+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2139268168 --> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x564a105a3b10 con 0x564a1065dc10 2024-03-28T13:18:03.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.822+0000 7f6e21800640 1 --2- 172.21.15.19:0/2139268168 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x564a1065dc10 0x564a10663140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.823+0000 7f6e21800640 1 --2- 172.21.15.19:0/2139268168 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x564a1065dc10 0x564a10663140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca49065fd76257bc server_cookie=409b6ccd359ba320 in_seq=0 out_seq=0 2024-03-28T13:18:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.823+0000 7f6e02a00640 10 client.7776 ms_handle_connect on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.823+0000 7f6e02a00640 1 -- 172.21.15.19:0/2139268168 <== mds.0 v2:172.21.15.19:6832/2209799371 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564a105a3b10 con 0x564a1065dc10 2024-03-28T13:18:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.823+0000 7f6e02a00640 10 client.7776 handle_command_reply: tid=0 2024-03-28T13:18:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.823+0000 7f6e26fbd740 1 client.7776 shutdown 2024-03-28T13:18:03.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.823+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2139268168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10657980 msgr2=0x564a1068f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.823+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/2139268168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10657980 0x564a1068f820 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f6e08000c00 tx=0x7f6e08059170 comp rx=0 tx=0).stop 2024-03-28T13:18:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.823+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2139268168 shutdown_connections 2024-03-28T13:18:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.823+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/2139268168 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x564a1065dc10 0x564a10663140 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.823+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/2139268168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x564a1067f6d0 0x564a1068fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.823+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/2139268168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a10657980 0x564a1068f820 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.823+0000 7f6e26fbd740 1 --2- 172.21.15.19:0/2139268168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x564a10637de0 0x564a106f6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.823+0000 7f6e02a00640 0 client.7776 ms_handle_reset on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.823+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2139268168 >> 172.21.15.19:0/2139268168 conn(0x564a10631fc0 msgr2=0x564a10665110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.824+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2139268168 shutdown_connections 2024-03-28T13:18:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.824+0000 7f6e26fbd740 1 -- 172.21.15.19:0/2139268168 wait complete. 2024-03-28T13:18:03.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.824+0000 7f6e26fbd740 20 client.7776 trim_cache size 0 max 16384 2024-03-28T13:18:03.825 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:18:03.825 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-03-28T13:18:03.826 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "cfc4eacd-a41d-4c2f-85ea-8231183b03d0", 2024-03-28T13:18:03.826 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-03-28T13:18:03.826 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:18:03.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.824+0000 7f6e24200640 1 -- 172.21.15.19:0/3638083890 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6dfc06c530 msgr2=0x7f6dfc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.824+0000 7f6e24200640 1 --2- 172.21.15.19:0/3638083890 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6dfc06c530 0x7f6dfc06e9f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6e10036000 tx=0x7f6e1003c040 comp rx=0 tx=0).stop 2024-03-28T13:18:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.824+0000 7f6e24200640 1 -- 172.21.15.19:0/3638083890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c0b3270 msgr2=0x7f6e1c15a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.824+0000 7f6e24200640 1 --2- 172.21.15.19:0/3638083890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c0b3270 0x7f6e1c15a4a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f6e0800c430 tx=0x7f6e0800c8f0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.825+0000 7f6e24200640 1 -- 172.21.15.19:0/3638083890 shutdown_connections 2024-03-28T13:18:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.825+0000 7f6e24200640 1 --2- 172.21.15.19:0/3638083890 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6dfc06c530 0x7f6dfc06e9f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.825+0000 7f6e24200640 1 --2- 172.21.15.19:0/3638083890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6e1c161f10 0x7f6e1c0b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.825+0000 7f6e24200640 1 --2- 172.21.15.19:0/3638083890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e1c0b3270 0x7f6e1c15a4a0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.825+0000 7f6e24200640 1 --2- 172.21.15.19:0/3638083890 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6e1c06e110 0x7f6e1c0b4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.825+0000 7f6e24200640 1 -- 172.21.15.19:0/3638083890 >> 172.21.15.19:0/3638083890 conn(0x7f6e1c08bfb0 msgr2=0x7f6e1c0d0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.825+0000 7f6e24200640 1 -- 172.21.15.19:0/3638083890 shutdown_connections 2024-03-28T13:18:03.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:03.825+0000 7f6e24200640 1 -- 172.21.15.19:0/3638083890 wait complete. 2024-03-28T13:18:03.828 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-03-28T13:18:03.827+0000 7f87afa00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x5578cd576680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-28T13:18:03.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:18:03.844 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-28T13:18:04.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.048+0000 7f4d24a00640 1 Processor -- start 2024-03-28T13:18:04.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.048+0000 7f4d24a00640 1 -- start start 2024-03-28T13:18:04.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.050+0000 7f4d24a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d200d0990 0x7f4d200aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.050+0000 7f4d24a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d200afab0 con 0x7f4d200af2e0 2024-03-28T13:18:04.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.050+0000 7f4d24a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d2015cc80 con 0x7f4d20153580 2024-03-28T13:18:04.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.050+0000 7f4d24a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4d2015da00 con 0x7f4d200d0990 2024-03-28T13:18:04.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.050+0000 7f4d1e800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4d200af2e0 0x7f4d201600d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:37362/0 (socket says 172.21.15.19:37362) 2024-03-28T13:18:04.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.050+0000 7f4d1e800640 1 -- 172.21.15.19:0/2018414112 learned_addr learned my addr 172.21.15.19:0/2018414112 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.051+0000 7f4d1d400640 1 --2- 172.21.15.19:0/2018414112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d200d0990 0x7f4d200aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.051+0000 7f4d1ca00640 1 -- 172.21.15.19:0/2018414112 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3243427506 0 0) 0x7f4d200afab0 con 0x7f4d200af2e0 2024-03-28T13:18:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.051+0000 7f4d1ca00640 1 -- 172.21.15.19:0/2018414112 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d04003520 con 0x7f4d200af2e0 2024-03-28T13:18:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.051+0000 7f4d1ca00640 1 -- 172.21.15.19:0/2018414112 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 42782994 0 0) 0x7f4d04003520 con 0x7f4d200af2e0 2024-03-28T13:18:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.051+0000 7f4d1ca00640 1 -- 172.21.15.19:0/2018414112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d200d0990 msgr2=0x7f4d200aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.051+0000 7f4d1ca00640 1 --2- 172.21.15.19:0/2018414112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d200d0990 0x7f4d200aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.051+0000 7f4d1ca00640 1 -- 172.21.15.19:0/2018414112 >> v1:172.21.15.188:6789/0 conn(0x7f4d20153580 legacy=0x7f4d200d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:04.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.051+0000 7f4d1ca00640 1 -- 172.21.15.19:0/2018414112 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2015cf20 con 0x7f4d200af2e0 2024-03-28T13:18:04.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.051+0000 7f4d1ca00640 1 -- 172.21.15.19:0/2018414112 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f4d180019f0 con 0x7f4d200af2e0 2024-03-28T13:18:04.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.051+0000 7f4d1ca00640 1 -- 172.21.15.19:0/2018414112 >> v1:172.21.15.19:6789/0 conn(0x7f4d200af2e0 legacy=0x7f4d201600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:04.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.052+0000 7f4d1ca00640 1 --2- 172.21.15.19:0/2018414112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d040039e0 0x7f4d04003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.052+0000 7f4d1ca00640 1 --2- 172.21.15.19:0/2018414112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d200d0990 0x7f4d04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.052+0000 7f4d1ca00640 1 --2- 172.21.15.19:0/2018414112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d04004d40 0x7f4d040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.052+0000 7f4d1ca00640 1 -- 172.21.15.19:0/2018414112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2015da00 con 0x7f4d04004d40 2024-03-28T13:18:04.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.052+0000 7f4d1ca00640 1 -- 172.21.15.19:0/2018414112 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d0400ba70 con 0x7f4d040039e0 2024-03-28T13:18:04.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.052+0000 7f4d1ca00640 1 -- 172.21.15.19:0/2018414112 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4d0400bc70 con 0x7f4d200d0990 2024-03-28T13:18:04.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.052+0000 7f4d1e800640 1 --2- 172.21.15.19:0/2018414112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d04004d40 0x7f4d040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.052+0000 7f4d1de00640 1 --2- 172.21.15.19:0/2018414112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d040039e0 0x7f4d04003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.052+0000 7f4d1d400640 1 --2- 172.21.15.19:0/2018414112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d200d0990 0x7f4d04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.053+0000 7f4d1e800640 1 -- 172.21.15.19:0/2018414112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d200d0990 msgr2=0x7f4d04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.053+0000 7f4d1e800640 1 --2- 172.21.15.19:0/2018414112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d200d0990 0x7f4d04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.053+0000 7f4d1e800640 1 -- 172.21.15.19:0/2018414112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d040039e0 msgr2=0x7f4d04003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.053+0000 7f4d1e800640 1 --2- 172.21.15.19:0/2018414112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d040039e0 0x7f4d04003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.053+0000 7f4d1e800640 1 -- 172.21.15.19:0/2018414112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d04003520 con 0x7f4d04004d40 2024-03-28T13:18:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.053+0000 7f4d1d400640 1 --2- 172.21.15.19:0/2018414112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d200d0990 0x7f4d04004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:04.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.053+0000 7f4d1de00640 1 --2- 172.21.15.19:0/2018414112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d040039e0 0x7f4d04003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:18:04.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.054+0000 7f4d1e800640 1 --2- 172.21.15.19:0/2018414112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d04004d40 0x7f4d040051c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800a810 tx=0x7f4d04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79deefe0cf333f4b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:04.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.054+0000 7f4d1ca00640 1 -- 172.21.15.19:0/2018414112 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1800a430 con 0x7f4d04004d40 2024-03-28T13:18:04.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.054+0000 7f4d1ca00640 1 -- 172.21.15.19:0/2018414112 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d18034630 con 0x7f4d04004d40 2024-03-28T13:18:04.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.054+0000 7f4d1ca00640 1 -- 172.21.15.19:0/2018414112 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18034a20 con 0x7f4d04004d40 2024-03-28T13:18:04.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.055+0000 7f4d24a00640 1 -- 172.21.15.19:0/2018414112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d04004d40 msgr2=0x7f4d040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:04.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.055+0000 7f4d24a00640 1 --2- 172.21.15.19:0/2018414112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d04004d40 0x7f4d040051c0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800a810 tx=0x7f4d04005830 comp rx=0 tx=0).stop 2024-03-28T13:18:04.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.055+0000 7f4d1e800640 1 -- 172.21.15.19:0/2018414112 reap_dead start 2024-03-28T13:18:04.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.055+0000 7f4d24a00640 1 -- 172.21.15.19:0/2018414112 shutdown_connections 2024-03-28T13:18:04.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.055+0000 7f4d24a00640 1 -- 172.21.15.19:0/2018414112 >> 172.21.15.19:0/2018414112 conn(0x7f4d2006b750 msgr2=0x7f4d2006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:04.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.055+0000 7f4d24a00640 1 -- 172.21.15.19:0/2018414112 shutdown_connections 2024-03-28T13:18:04.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.055+0000 7f4d24a00640 1 -- 172.21.15.19:0/2018414112 wait complete. 2024-03-28T13:18:04.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.058+0000 7f4d24a00640 1 Processor -- start 2024-03-28T13:18:04.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.058+0000 7f4d24a00640 1 -- start start 2024-03-28T13:18:04.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.060+0000 7f4d24a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d04004d40 0x7f4d200cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.060+0000 7f4d24a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d20151f90 0x7f4d20146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.060+0000 7f4d24a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d201471a0 0x7f4d201502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.060+0000 7f4d24a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d20153960 con 0x7f4d20151f90 2024-03-28T13:18:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.060+0000 7f4d24a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2015d4e0 con 0x7f4d201471a0 2024-03-28T13:18:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.060+0000 7f4d24a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4d200af710 con 0x7f4d04004d40 2024-03-28T13:18:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.060+0000 7f4d1d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d20151f90 0x7f4d20146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.060+0000 7f4d1d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d20151f90 0x7f4d20146c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45272/0 (socket says 172.21.15.19:45272) 2024-03-28T13:18:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.060+0000 7f4d1d400640 1 -- 172.21.15.19:0/2429930634 learned_addr learned my addr 172.21.15.19:0/2429930634 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.060+0000 7f4d1de00640 1 --2- 172.21.15.19:0/2429930634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d04004d40 0x7f4d200cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.060+0000 7f4d1e800640 1 --2- 172.21.15.19:0/2429930634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d201471a0 0x7f4d201502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.060+0000 7f4d1d400640 1 -- 172.21.15.19:0/2429930634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d04004d40 msgr2=0x7f4d200cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.061+0000 7f4d1d400640 1 --2- 172.21.15.19:0/2429930634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d04004d40 0x7f4d200cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.061+0000 7f4d1d400640 1 -- 172.21.15.19:0/2429930634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d201471a0 msgr2=0x7f4d201502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.061+0000 7f4d1d400640 1 --2- 172.21.15.19:0/2429930634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d201471a0 0x7f4d201502e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.061+0000 7f4d1d400640 1 -- 172.21.15.19:0/2429930634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d200ce080 con 0x7f4d20151f90 2024-03-28T13:18:04.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.061+0000 7f4d1d400640 1 --2- 172.21.15.19:0/2429930634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d20151f90 0x7f4d20146c60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0108b0 tx=0x7f4d0c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:04.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.061+0000 7f4d16000640 1 -- 172.21.15.19:0/2429930634 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c00f040 con 0x7f4d20151f90 2024-03-28T13:18:04.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.061+0000 7f4d16000640 1 -- 172.21.15.19:0/2429930634 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d0c00eb60 con 0x7f4d20151f90 2024-03-28T13:18:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.061+0000 7f4d24a00640 1 -- 172.21.15.19:0/2429930634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d200ce360 con 0x7f4d20151f90 2024-03-28T13:18:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.061+0000 7f4d16000640 1 -- 172.21.15.19:0/2429930634 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c01c400 con 0x7f4d20151f90 2024-03-28T13:18:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.061+0000 7f4d24a00640 1 -- 172.21.15.19:0/2429930634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d200ce920 con 0x7f4d20151f90 2024-03-28T13:18:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.064+0000 7f4d16000640 1 -- 172.21.15.19:0/2429930634 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4d0c01c5a0 con 0x7f4d20151f90 2024-03-28T13:18:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.065+0000 7f4d16000640 1 --2- 172.21.15.19:0/2429930634 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d0006c560 0x7f4d0006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.065+0000 7f4d16000640 1 -- 172.21.15.19:0/2429930634 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f4d0c08bf60 con 0x7f4d20151f90 2024-03-28T13:18:04.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.065+0000 7f4d1de00640 1 --2- 172.21.15.19:0/2429930634 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d0006c560 0x7f4d0006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.067+0000 7f4d1de00640 1 --2- 172.21.15.19:0/2429930634 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d0006c560 0x7f4d0006ea20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4d10007fd0 tx=0x7f4d10009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:04.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.067+0000 7f4d277bb740 1 Processor -- start 2024-03-28T13:18:04.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.067+0000 7f4d277bb740 1 -- start start 2024-03-28T13:18:04.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.068+0000 7f4d277bb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7d9560 0x55a7fa7ac500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.068+0000 7f4d277bb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9940 0x55a7fa7dc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.068+0000 7f4d277bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7aca40 0x55a7fa7dc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.068+0000 7f4d277bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a7fa7176b0 con 0x55a7fa7aca40 2024-03-28T13:18:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.068+0000 7f4d277bb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a7fa6bf750 con 0x55a7fa7d9940 2024-03-28T13:18:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.068+0000 7f4d277bb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55a7fa713660 con 0x55a7fa7d9560 2024-03-28T13:18:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.069+0000 7f4d1e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7aca40 0x55a7fa7dc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.069+0000 7f4d1e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7aca40 0x55a7fa7dc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45286/0 (socket says 172.21.15.19:45286) 2024-03-28T13:18:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.069+0000 7f4d1e800640 1 -- 172.21.15.19:0/2836352096 learned_addr learned my addr 172.21.15.19:0/2836352096 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.069+0000 7f4d1d400640 1 --2- 172.21.15.19:0/2836352096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9940 0x55a7fa7dc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.069+0000 7f4d1de00640 1 --2- 172.21.15.19:0/2836352096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7d9560 0x55a7fa7ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.069+0000 7f4d1e800640 1 -- 172.21.15.19:0/2836352096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7d9560 msgr2=0x55a7fa7ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.069+0000 7f4d1e800640 1 --2- 172.21.15.19:0/2836352096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7d9560 0x55a7fa7ac500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.069+0000 7f4d1e800640 1 -- 172.21.15.19:0/2836352096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9940 msgr2=0x55a7fa7dc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.069+0000 7f4d1e800640 1 --2- 172.21.15.19:0/2836352096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9940 0x55a7fa7dc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.069+0000 7f4d1e800640 1 -- 172.21.15.19:0/2836352096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7fa4f57d0 con 0x55a7fa7aca40 2024-03-28T13:18:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.070+0000 7f4d1e800640 1 --2- 172.21.15.19:0/2836352096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7aca40 0x55a7fa7dc830 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4d1802cc00 tx=0x7f4d18009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa776ff1e4d1ee00 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.070+0000 7f4d0a600640 1 -- 172.21.15.19:0/2836352096 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1800a430 con 0x55a7fa7aca40 2024-03-28T13:18:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.070+0000 7f4d0a600640 1 -- 172.21.15.19:0/2836352096 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d18046070 con 0x55a7fa7aca40 2024-03-28T13:18:04.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.070+0000 7f4d0a600640 1 -- 172.21.15.19:0/2836352096 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18001930 con 0x55a7fa7aca40 2024-03-28T13:18:04.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.071+0000 7f4d277bb740 1 -- 172.21.15.19:0/2836352096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7aca40 msgr2=0x55a7fa7dc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:04.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.071+0000 7f4d277bb740 1 --2- 172.21.15.19:0/2836352096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7aca40 0x55a7fa7dc830 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4d1802cc00 tx=0x7f4d18009030 comp rx=0 tx=0).stop 2024-03-28T13:18:04.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.071+0000 7f4d277bb740 1 -- 172.21.15.19:0/2836352096 shutdown_connections 2024-03-28T13:18:04.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.071+0000 7f4d277bb740 1 --2- 172.21.15.19:0/2836352096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7aca40 0x55a7fa7dc830 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.071+0000 7f4d277bb740 1 --2- 172.21.15.19:0/2836352096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9940 0x55a7fa7dc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.071+0000 7f4d277bb740 1 --2- 172.21.15.19:0/2836352096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7d9560 0x55a7fa7ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.071+0000 7f4d277bb740 1 -- 172.21.15.19:0/2836352096 >> 172.21.15.19:0/2836352096 conn(0x55a7fa7ae770 msgr2=0x55a7fa7f72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:04.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.071+0000 7f4d277bb740 1 -- 172.21.15.19:0/2836352096 shutdown_connections 2024-03-28T13:18:04.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.071+0000 7f4d277bb740 1 -- 172.21.15.19:0/2836352096 wait complete. 2024-03-28T13:18:04.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.072+0000 7f4d277bb740 1 -- start start 2024-03-28T13:18:04.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.073+0000 7f4d277bb740 1 Processor -- start 2024-03-28T13:18:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.074+0000 7f4d277bb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9560 0x55a7fa7dd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.074+0000 7f4d277bb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7d9940 0x55a7fa807230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.074+0000 7f4d277bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7ddea0 0x55a7fa7dfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.074+0000 7f4d277bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a7fa7b0e10 con 0x55a7fa7ddea0 2024-03-28T13:18:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.074+0000 7f4d277bb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55a7fa6d2a80 con 0x55a7fa7d9560 2024-03-28T13:18:04.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.074+0000 7f4d277bb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55a7fa7b3650 con 0x55a7fa7d9940 2024-03-28T13:18:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.074+0000 7f4d277bb740 10 client.0 _ll_register_callbacks cb 0x55a7fa6d2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:18:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.074+0000 7f4d1e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7ddea0 0x55a7fa7dfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.074+0000 7f4d1e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7ddea0 0x55a7fa7dfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45294/0 (socket says 172.21.15.19:45294) 2024-03-28T13:18:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.074+0000 7f4d1e800640 1 -- 172.21.15.19:0/3866725983 learned_addr learned my addr 172.21.15.19:0/3866725983 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.074+0000 7f4d1e800640 1 -- 172.21.15.19:0/3866725983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7d9940 msgr2=0x55a7fa807230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.074+0000 7f4d1e800640 1 --2- 172.21.15.19:0/3866725983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7d9940 0x55a7fa807230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.074+0000 7f4d1e800640 1 -- 172.21.15.19:0/3866725983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9560 msgr2=0x55a7fa7dd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.074+0000 7f4d1e800640 1 --2- 172.21.15.19:0/3866725983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9560 0x55a7fa7dd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.074+0000 7f4d1e800640 1 -- 172.21.15.19:0/3866725983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7fa42b3f0 con 0x55a7fa7ddea0 2024-03-28T13:18:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.075+0000 7f4d1e800640 1 --2- 172.21.15.19:0/3866725983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7ddea0 0x55a7fa7dfac0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005c20 tx=0x7f4d1802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbe35a2626f00de server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:04.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.075+0000 7f4cfea00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:18:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.075+0000 7f4cfea00640 1 -- 172.21.15.19:0/3866725983 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18050030 con 0x55a7fa7ddea0 2024-03-28T13:18:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.075+0000 7f4cfea00640 1 -- 172.21.15.19:0/3866725983 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f4d18001940 con 0x55a7fa7ddea0 2024-03-28T13:18:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.075+0000 7f4d277bb740 1 -- 172.21.15.19:0/3866725983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d18046070 con 0x55a7fa7ddea0 2024-03-28T13:18:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.075+0000 7f4cfea00640 1 -- 172.21.15.19:0/3866725983 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f4d18039040 con 0x55a7fa7ddea0 2024-03-28T13:18:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.075+0000 7f4d277bb740 10 client.7788 fetch_fsmap learned FSMap version 74 2024-03-28T13:18:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.075+0000 7f4d277bb740 1 -- 172.21.15.19:0/3866725983 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55a7fa3eb0f0 con 0x55a7fa7ddea0 2024-03-28T13:18:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.076+0000 7f4cfea00640 1 -- 172.21.15.19:0/3866725983 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d18046070 con 0x55a7fa7ddea0 2024-03-28T13:18:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.076+0000 7f4cfea00640 1 -- 172.21.15.19:0/3866725983 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f4d1805f080 con 0x55a7fa7ddea0 2024-03-28T13:18:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.076+0000 7f4d277bb740 10 client.7788 fetch_fsmap finished waiting for FSMap version 74 2024-03-28T13:18:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.076+0000 7f4d277bb740 10 client.7788 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-28T13:18:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.076+0000 7f4d277bb740 20 client.7788 populate_metadata read hostname 'smithi019' 2024-03-28T13:18:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.076+0000 7f4d277bb740 1 --2- 172.21.15.19:0/3866725983 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a7fa7f5f20 0x55a7fa7f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.076+0000 7f4d277bb740 4 client.7788 mds_command: new command op to 6040 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:18:04.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.076+0000 7f4d277bb740 1 -- 172.21.15.19:0/3866725983 --> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7fa724bf0 con 0x55a7fa7f5f20 2024-03-28T13:18:04.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.076+0000 7f4d1de00640 1 --2- 172.21.15.19:0/3866725983 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a7fa7f5f20 0x55a7fa7f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.077+0000 7f4cfea00640 10 client.7788 ms_handle_connect on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.077+0000 7f4d1de00640 1 --2- 172.21.15.19:0/3866725983 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a7fa7f5f20 0x55a7fa7f63b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae161d0823423c12 server_cookie=5bbfe408b729ea50 in_seq=0 out_seq=0 2024-03-28T13:18:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.078+0000 7f4cfea00640 1 -- 172.21.15.19:0/3866725983 <== mds.0 v2:172.21.15.19:6832/2209799371 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a7fa724bf0 con 0x55a7fa7f5f20 2024-03-28T13:18:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.078+0000 7f4cfea00640 10 client.7788 handle_command_reply: tid=0 2024-03-28T13:18:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.078+0000 7f4d277bb740 1 client.7788 shutdown 2024-03-28T13:18:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.078+0000 7f4d277bb740 1 -- 172.21.15.19:0/3866725983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7ddea0 msgr2=0x55a7fa7dfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.078+0000 7f4d277bb740 1 --2- 172.21.15.19:0/3866725983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7ddea0 0x55a7fa7dfac0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4d18005c20 tx=0x7f4d1802e000 comp rx=0 tx=0).stop 2024-03-28T13:18:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.078+0000 7f4d277bb740 1 -- 172.21.15.19:0/3866725983 shutdown_connections 2024-03-28T13:18:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.078+0000 7f4d277bb740 1 --2- 172.21.15.19:0/3866725983 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a7fa7f5f20 0x55a7fa7f63b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.078+0000 7f4d277bb740 1 --2- 172.21.15.19:0/3866725983 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7ddea0 0x55a7fa7dfac0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.078+0000 7f4d277bb740 1 --2- 172.21.15.19:0/3866725983 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7d9940 0x55a7fa807230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.078+0000 7f4d277bb740 1 --2- 172.21.15.19:0/3866725983 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9560 0x55a7fa7dd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.078+0000 7f4d277bb740 1 -- 172.21.15.19:0/3866725983 >> 172.21.15.19:0/3866725983 conn(0x55a7fa7aca40 msgr2=0x55a7fa7b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:04.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.078+0000 7f4cfea00640 0 client.7788 ms_handle_reset on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:04.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.078+0000 7f4d277bb740 1 -- 172.21.15.19:0/3866725983 shutdown_connections 2024-03-28T13:18:04.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.078+0000 7f4d277bb740 1 -- 172.21.15.19:0/3866725983 wait complete. 2024-03-28T13:18:04.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.079+0000 7f4d277bb740 20 client.7788 trim_cache size 0 max 16384 2024-03-28T13:18:04.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.098+0000 7f4d277bb740 1 Processor -- start 2024-03-28T13:18:04.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.098+0000 7f4d277bb740 1 -- start start 2024-03-28T13:18:04.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.099+0000 7f4d277bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7d9560 0x55a7fa811580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.099+0000 7f4d277bb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9940 0x55a7fa7f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.099+0000 7f4d277bb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7ddea0 0x55a7fa7fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.099+0000 7f4d277bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d20201150 con 0x55a7fa7d9560 2024-03-28T13:18:04.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.099+0000 7f4d277bb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d20200170 con 0x55a7fa7d9940 2024-03-28T13:18:04.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.099+0000 7f4d277bb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4d20158a90 con 0x55a7fa7ddea0 2024-03-28T13:18:04.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.099+0000 7f4d1de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7d9560 0x55a7fa811580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.099+0000 7f4d1de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7d9560 0x55a7fa811580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45302/0 (socket says 172.21.15.19:45302) 2024-03-28T13:18:04.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.099+0000 7f4d1de00640 1 -- 172.21.15.19:0/1912423831 learned_addr learned my addr 172.21.15.19:0/1912423831 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:04.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.099+0000 7f4d1de00640 1 -- 172.21.15.19:0/1912423831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7ddea0 msgr2=0x55a7fa7fa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:04.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.099+0000 7f4d1de00640 1 --2- 172.21.15.19:0/1912423831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7ddea0 0x55a7fa7fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.099+0000 7f4d1de00640 1 -- 172.21.15.19:0/1912423831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9940 msgr2=0x55a7fa7f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:04.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.099+0000 7f4d1d400640 1 --2- 172.21.15.19:0/1912423831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9940 0x55a7fa7f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.099+0000 7f4d1de00640 1 --2- 172.21.15.19:0/1912423831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9940 0x55a7fa7f4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.099+0000 7f4d1de00640 1 -- 172.21.15.19:0/1912423831 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7fa7dfe30 con 0x55a7fa7d9560 2024-03-28T13:18:04.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.100+0000 7f4d1de00640 1 --2- 172.21.15.19:0/1912423831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7d9560 0x55a7fa811580 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f4d10007ac0 tx=0x7f4d10001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a69124e26bf3c00 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:04.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.100+0000 7f4d09c00640 1 -- 172.21.15.19:0/1912423831 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1001a040 con 0x55a7fa7d9560 2024-03-28T13:18:04.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.100+0000 7f4d09c00640 1 -- 172.21.15.19:0/1912423831 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d10007050 con 0x55a7fa7d9560 2024-03-28T13:18:04.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.100+0000 7f4d09c00640 1 -- 172.21.15.19:0/1912423831 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10007420 con 0x55a7fa7d9560 2024-03-28T13:18:04.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.101+0000 7f4d277bb740 1 -- 172.21.15.19:0/1912423831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7d9560 msgr2=0x55a7fa811580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:04.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.101+0000 7f4d277bb740 1 --2- 172.21.15.19:0/1912423831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7d9560 0x55a7fa811580 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f4d10007ac0 tx=0x7f4d10001230 comp rx=0 tx=0).stop 2024-03-28T13:18:04.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.101+0000 7f4d277bb740 1 -- 172.21.15.19:0/1912423831 shutdown_connections 2024-03-28T13:18:04.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.101+0000 7f4d277bb740 1 --2- 172.21.15.19:0/1912423831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7ddea0 0x55a7fa7fa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.101+0000 7f4d277bb740 1 --2- 172.21.15.19:0/1912423831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9940 0x55a7fa7f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.101+0000 7f4d277bb740 1 --2- 172.21.15.19:0/1912423831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7d9560 0x55a7fa811580 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.101+0000 7f4d277bb740 1 -- 172.21.15.19:0/1912423831 >> 172.21.15.19:0/1912423831 conn(0x55a7fa7aca40 msgr2=0x55a7fa7d4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:04.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.101+0000 7f4d277bb740 1 -- 172.21.15.19:0/1912423831 shutdown_connections 2024-03-28T13:18:04.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.101+0000 7f4d277bb740 1 -- 172.21.15.19:0/1912423831 wait complete. 2024-03-28T13:18:04.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.102+0000 7f4d277bb740 1 -- start start 2024-03-28T13:18:04.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.102+0000 7f4d277bb740 1 Processor -- start 2024-03-28T13:18:04.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.103+0000 7f4d277bb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7d9560 0x55a7fa811560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.103+0000 7f4d277bb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9940 0x55a7fa7f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.103+0000 7f4d277bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7ddea0 0x55a7fa7f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.103+0000 7f4d277bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a7fa7d8e20 con 0x55a7fa7ddea0 2024-03-28T13:18:04.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.103+0000 7f4d277bb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4d201ffb20 con 0x55a7fa7d9940 2024-03-28T13:18:04.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.103+0000 7f4d277bb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55a7fa871a50 con 0x55a7fa7d9560 2024-03-28T13:18:04.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.103+0000 7f4d277bb740 10 client.0 _ll_register_callbacks cb 0x7f4d1800a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:18:04.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.103+0000 7f4d1e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7ddea0 0x55a7fa7f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.103+0000 7f4d1e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7ddea0 0x55a7fa7f53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45312/0 (socket says 172.21.15.19:45312) 2024-03-28T13:18:04.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.103+0000 7f4d1e800640 1 -- 172.21.15.19:0/1518222662 learned_addr learned my addr 172.21.15.19:0/1518222662 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:04.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.103+0000 7f4d1de00640 1 --2- 172.21.15.19:0/1518222662 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7d9560 0x55a7fa811560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.104+0000 7f4d1d400640 1 --2- 172.21.15.19:0/1518222662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9940 0x55a7fa7f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.104+0000 7f4d1e800640 1 -- 172.21.15.19:0/1518222662 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7d9560 msgr2=0x55a7fa811560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:04.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.104+0000 7f4d1e800640 1 --2- 172.21.15.19:0/1518222662 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7d9560 0x55a7fa811560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.104+0000 7f4d1e800640 1 -- 172.21.15.19:0/1518222662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9940 msgr2=0x55a7fa7f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:04.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.104+0000 7f4d1e800640 1 --2- 172.21.15.19:0/1518222662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9940 0x55a7fa7f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.104+0000 7f4d1e800640 1 -- 172.21.15.19:0/1518222662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7fa870c90 con 0x55a7fa7ddea0 2024-03-28T13:18:04.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.104+0000 7f4d1e800640 1 --2- 172.21.15.19:0/1518222662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7ddea0 0x55a7fa7f53f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800a590 tx=0x7f4d18001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5798de748c074ba0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:04.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.104+0000 7f4cfea00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:18:04.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.104+0000 7f4cfea00640 1 -- 172.21.15.19:0/1518222662 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18050070 con 0x55a7fa7ddea0 2024-03-28T13:18:04.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.104+0000 7f4cfea00640 1 -- 172.21.15.19:0/1518222662 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f4d18041bf0 con 0x55a7fa7ddea0 2024-03-28T13:18:04.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.104+0000 7f4d277bb740 1 -- 172.21.15.19:0/1518222662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d10007050 con 0x55a7fa7ddea0 2024-03-28T13:18:04.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.105+0000 7f4cfea00640 1 -- 172.21.15.19:0/1518222662 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f4d18041df0 con 0x55a7fa7ddea0 2024-03-28T13:18:04.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.105+0000 7f4d277bb740 10 client.7794 fetch_fsmap learned FSMap version 74 2024-03-28T13:18:04.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.105+0000 7f4d277bb740 1 -- 172.21.15.19:0/1518222662 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55a7fa8712b0 con 0x55a7fa7ddea0 2024-03-28T13:18:04.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.105+0000 7f4cfea00640 1 -- 172.21.15.19:0/1518222662 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d10007050 con 0x55a7fa7ddea0 2024-03-28T13:18:04.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.105+0000 7f4cfea00640 1 -- 172.21.15.19:0/1518222662 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f4d1803f040 con 0x55a7fa7ddea0 2024-03-28T13:18:04.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.105+0000 7f4d277bb740 10 client.7794 fetch_fsmap finished waiting for FSMap version 74 2024-03-28T13:18:04.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.105+0000 7f4d277bb740 10 client.7794 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-28T13:18:04.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.105+0000 7f4d277bb740 20 client.7794 populate_metadata read hostname 'smithi019' 2024-03-28T13:18:04.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.105+0000 7f4d277bb740 1 --2- 172.21.15.19:0/1518222662 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a7fa7d7920 0x55a7fa7d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:04.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.105+0000 7f4d277bb740 4 client.7794 mds_command: new command op to 6040 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:18:04.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.105+0000 7f4d277bb740 1 -- 172.21.15.19:0/1518222662 --> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a7fa7df630 con 0x55a7fa7d7920 2024-03-28T13:18:04.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.106+0000 7f4d1de00640 1 --2- 172.21.15.19:0/1518222662 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a7fa7d7920 0x55a7fa7d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:04.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.106+0000 7f4d1de00640 1 --2- 172.21.15.19:0/1518222662 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a7fa7d7920 0x55a7fa7d7d30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b919450176090 server_cookie=c6723eb4d31fd9b8 in_seq=0 out_seq=0 2024-03-28T13:18:04.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.106+0000 7f4cfea00640 10 client.7794 ms_handle_connect on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:04.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.106+0000 7f4cfea00640 1 -- 172.21.15.19:0/1518222662 <== mds.0 v2:172.21.15.19:6832/2209799371 1 ==== command_reply(tid 0: 0 ) ==== 8+0+263 (crc 0 0 0) 0x55a7fa7df630 con 0x55a7fa7d7920 2024-03-28T13:18:04.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.106+0000 7f4cfea00640 10 client.7794 handle_command_reply: tid=0 2024-03-28T13:18:04.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.106+0000 7f4d277bb740 1 client.7794 shutdown 2024-03-28T13:18:04.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.107+0000 7f4d277bb740 1 -- 172.21.15.19:0/1518222662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7ddea0 msgr2=0x55a7fa7f53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.107+0000 7f4d277bb740 1 --2- 172.21.15.19:0/1518222662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7ddea0 0x55a7fa7f53f0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800a590 tx=0x7f4d18001d40 comp rx=0 tx=0).stop 2024-03-28T13:18:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.107+0000 7f4d277bb740 1 -- 172.21.15.19:0/1518222662 shutdown_connections 2024-03-28T13:18:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.107+0000 7f4d277bb740 1 --2- 172.21.15.19:0/1518222662 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x55a7fa7d7920 0x55a7fa7d7d30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.107+0000 7f4d277bb740 1 --2- 172.21.15.19:0/1518222662 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a7fa7ddea0 0x55a7fa7f53f0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.107+0000 7f4d277bb740 1 --2- 172.21.15.19:0/1518222662 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a7fa7d9940 0x55a7fa7f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.107+0000 7f4d277bb740 1 --2- 172.21.15.19:0/1518222662 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a7fa7d9560 0x55a7fa811560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.107+0000 7f4d277bb740 1 -- 172.21.15.19:0/1518222662 >> 172.21.15.19:0/1518222662 conn(0x55a7fa7aca40 msgr2=0x55a7fa7d1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.107+0000 7f4cfea00640 0 client.7794 ms_handle_reset on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.107+0000 7f4d277bb740 1 -- 172.21.15.19:0/1518222662 shutdown_connections 2024-03-28T13:18:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.107+0000 7f4d277bb740 1 -- 172.21.15.19:0/1518222662 wait complete. 2024-03-28T13:18:04.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.107+0000 7f4d277bb740 20 client.7794 trim_cache size 0 max 16384 2024-03-28T13:18:04.113 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:18:04.113 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T13:18:04.113 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-03-28T13:18:04.113 INFO:teuthology.orchestra.run.smithi019.stdout: "cfc4eacd-a41d-4c2f-85ea-8231183b03d0": { 2024-03-28T13:18:04.113 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/dir", 2024-03-28T13:18:04.113 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "cfc4eacd-a41d-4c2f-85ea-8231183b03d0", 2024-03-28T13:18:04.113 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,repair" 2024-03-28T13:18:04.113 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:04.113 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:04.114 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:18:04.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.108+0000 7f4d24a00640 1 -- 172.21.15.19:0/2429930634 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d0006c560 msgr2=0x7f4d0006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.108+0000 7f4d24a00640 1 --2- 172.21.15.19:0/2429930634 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d0006c560 0x7f4d0006ea20 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f4d10007fd0 tx=0x7f4d10009040 comp rx=0 tx=0).stop 2024-03-28T13:18:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.108+0000 7f4d24a00640 1 -- 172.21.15.19:0/2429930634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d20151f90 msgr2=0x7f4d20146c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.108+0000 7f4d24a00640 1 --2- 172.21.15.19:0/2429930634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d20151f90 0x7f4d20146c60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c0108b0 tx=0x7f4d0c010d70 comp rx=0 tx=0).stop 2024-03-28T13:18:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.108+0000 7f4d24a00640 1 -- 172.21.15.19:0/2429930634 shutdown_connections 2024-03-28T13:18:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.108+0000 7f4d24a00640 1 --2- 172.21.15.19:0/2429930634 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4d0006c560 0x7f4d0006ea20 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.108+0000 7f4d24a00640 1 --2- 172.21.15.19:0/2429930634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d201471a0 0x7f4d201502e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.108+0000 7f4d24a00640 1 --2- 172.21.15.19:0/2429930634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d20151f90 0x7f4d20146c60 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.108+0000 7f4d24a00640 1 --2- 172.21.15.19:0/2429930634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d04004d40 0x7f4d200cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.108+0000 7f4d24a00640 1 -- 172.21.15.19:0/2429930634 >> 172.21.15.19:0/2429930634 conn(0x7f4d2006b750 msgr2=0x7f4d2006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.108+0000 7f4d24a00640 1 -- 172.21.15.19:0/2429930634 shutdown_connections 2024-03-28T13:18:04.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:04.108+0000 7f4d24a00640 1 -- 172.21.15.19:0/2429930634 wait complete. 2024-03-28T13:18:04.132 INFO:tasks.cephfs.filesystem:scrub status for tag:cfc4eacd-a41d-4c2f-85ea-8231183b03d0 - {'path': '/dir', 'tag': 'cfc4eacd-a41d-4c2f-85ea-8231183b03d0', 'options': 'recursive,repair'} 2024-03-28T13:18:33.718 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:18:33.727 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:18:34.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:18:34.134 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-28T13:18:34.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.374+0000 7f6cc0c00640 1 Processor -- start 2024-03-28T13:18:34.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.375+0000 7f6cc0c00640 1 -- start start 2024-03-28T13:18:34.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.376+0000 7f6cc0c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0d37b0 0x7f6cbc0b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.376+0000 7f6cc0c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6cbc0b67f0 0x7f6cbc0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.376+0000 7f6cc0c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cbc0d2710 con 0x7f6cbc15e4c0 2024-03-28T13:18:34.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.376+0000 7f6cc0c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6cbc15e8a0 con 0x7f6cbc0b67f0 2024-03-28T13:18:34.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.376+0000 7f6cc0c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6cbc0d4040 con 0x7f6cbc0d37b0 2024-03-28T13:18:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.377+0000 7f6cbbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6cbc0b67f0 0x7f6cbc0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.377+0000 7f6cbbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6cbc0b67f0 0x7f6cbc0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49356/0 (socket says 172.21.15.19:49356) 2024-03-28T13:18:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.377+0000 7f6cbbe00640 1 -- 172.21.15.19:0/2086776104 learned_addr learned my addr 172.21.15.19:0/2086776104 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.377+0000 7f6cbaa00640 1 --2- 172.21.15.19:0/2086776104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0d37b0 0x7f6cbc0b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.378+0000 7f6cbbe00640 1 -- 172.21.15.19:0/2086776104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0d37b0 msgr2=0x7f6cbc0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:34.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.378+0000 7f6cbbe00640 1 --2- 172.21.15.19:0/2086776104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0d37b0 0x7f6cbc0b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.378+0000 7f6cbbe00640 1 -- 172.21.15.19:0/2086776104 >> v1:172.21.15.188:6789/0 conn(0x7f6cbc15e4c0 legacy=0x7f6cbc0b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.378+0000 7f6cbbe00640 1 -- 172.21.15.19:0/2086776104 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc0d29f0 con 0x7f6cbc0b67f0 2024-03-28T13:18:34.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.380+0000 7f6cbbe00640 1 --2- 172.21.15.19:0/2086776104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cbc0b67f0 0x7f6cbc0b7400 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009e50 tx=0x7f6cb402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae2a2a63ec1a8a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:34.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.381+0000 7f6cba000640 1 -- 172.21.15.19:0/2086776104 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb402cb00 con 0x7f6cbc0b67f0 2024-03-28T13:18:34.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.381+0000 7f6cba000640 1 -- 172.21.15.19:0/2086776104 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb402cca0 con 0x7f6cbc0b67f0 2024-03-28T13:18:34.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.381+0000 7f6cba000640 1 -- 172.21.15.19:0/2086776104 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4034610 con 0x7f6cbc0b67f0 2024-03-28T13:18:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.382+0000 7f6cc0c00640 1 -- 172.21.15.19:0/2086776104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cbc0b67f0 msgr2=0x7f6cbc0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.382+0000 7f6cc0c00640 1 --2- 172.21.15.19:0/2086776104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cbc0b67f0 0x7f6cbc0b7400 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009e50 tx=0x7f6cb402bf60 comp rx=0 tx=0).stop 2024-03-28T13:18:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.382+0000 7f6cc0c00640 1 -- 172.21.15.19:0/2086776104 shutdown_connections 2024-03-28T13:18:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.382+0000 7f6cc0c00640 1 --2- 172.21.15.19:0/2086776104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cbc0b67f0 0x7f6cbc0b7400 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.382+0000 7f6cc0c00640 1 --2- 172.21.15.19:0/2086776104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0d37b0 0x7f6cbc0b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.382+0000 7f6cc0c00640 1 -- 172.21.15.19:0/2086776104 >> 172.21.15.19:0/2086776104 conn(0x7f6cbc0676a0 msgr2=0x7f6cbc067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.382+0000 7f6cc0c00640 1 -- 172.21.15.19:0/2086776104 shutdown_connections 2024-03-28T13:18:34.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.382+0000 7f6cc0c00640 1 -- 172.21.15.19:0/2086776104 wait complete. 2024-03-28T13:18:34.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.384+0000 7f6cc0c00640 1 Processor -- start 2024-03-28T13:18:34.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.384+0000 7f6cc0c00640 1 -- start start 2024-03-28T13:18:34.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.385+0000 7f6cc0c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0b67f0 0x7f6cbc146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.385+0000 7f6cc0c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cbc0d37b0 0x7f6cbc155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.385+0000 7f6cc0c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cbc15e4c0 0x7f6cbc1485a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.385+0000 7f6cc0c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc0d2fb0 con 0x7f6cbc0d37b0 2024-03-28T13:18:34.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.385+0000 7f6cc0c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc0d3320 con 0x7f6cbc15e4c0 2024-03-28T13:18:34.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.385+0000 7f6cc0c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6cbc0d2350 con 0x7f6cbc0b67f0 2024-03-28T13:18:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.385+0000 7f6cbaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cbc0d37b0 0x7f6cbc155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.385+0000 7f6cbaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cbc0d37b0 0x7f6cbc155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49364/0 (socket says 172.21.15.19:49364) 2024-03-28T13:18:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.385+0000 7f6cbaa00640 1 -- 172.21.15.19:0/3937071227 learned_addr learned my addr 172.21.15.19:0/3937071227 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.385+0000 7f6cbb400640 1 --2- 172.21.15.19:0/3937071227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0b67f0 0x7f6cbc146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.386+0000 7f6cbbe00640 1 --2- 172.21.15.19:0/3937071227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cbc15e4c0 0x7f6cbc1485a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.386+0000 7f6cbaa00640 1 -- 172.21.15.19:0/3937071227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0b67f0 msgr2=0x7f6cbc146590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.386+0000 7f6cbaa00640 1 --2- 172.21.15.19:0/3937071227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0b67f0 0x7f6cbc146590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.386+0000 7f6cbaa00640 1 -- 172.21.15.19:0/3937071227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cbc15e4c0 msgr2=0x7f6cbc1485a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.386+0000 7f6cbaa00640 1 --2- 172.21.15.19:0/3937071227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cbc15e4c0 0x7f6cbc1485a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.386+0000 7f6cbaa00640 1 -- 172.21.15.19:0/3937071227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc1579c0 con 0x7f6cbc0d37b0 2024-03-28T13:18:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.386+0000 7f6cbaa00640 1 --2- 172.21.15.19:0/3937071227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cbc0d37b0 0x7f6cbc155400 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800e9e0 tx=0x7f6ca800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.386+0000 7f6cb3400640 1 -- 172.21.15.19:0/3937071227 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8017070 con 0x7f6cbc0d37b0 2024-03-28T13:18:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.386+0000 7f6cb3400640 1 -- 172.21.15.19:0/3937071227 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ca800c9d0 con 0x7f6cbc0d37b0 2024-03-28T13:18:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.386+0000 7f6cb3400640 1 -- 172.21.15.19:0/3937071227 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca800cdc0 con 0x7f6cbc0d37b0 2024-03-28T13:18:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.386+0000 7f6cc0c00640 1 -- 172.21.15.19:0/3937071227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cbc157ca0 con 0x7f6cbc0d37b0 2024-03-28T13:18:34.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.387+0000 7f6cc0c00640 1 -- 172.21.15.19:0/3937071227 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cbc155940 con 0x7f6cbc0d37b0 2024-03-28T13:18:34.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.390+0000 7f6cb3400640 1 -- 172.21.15.19:0/3937071227 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6ca801a070 con 0x7f6cbc0d37b0 2024-03-28T13:18:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.391+0000 7f6cc4f81740 1 Processor -- start 2024-03-28T13:18:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.391+0000 7f6cb3400640 1 --2- 172.21.15.19:0/3937071227 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6c9c06c530 0x7f6c9c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.391+0000 7f6cc4f81740 1 -- start start 2024-03-28T13:18:34.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.392+0000 7f6cb3400640 1 -- 172.21.15.19:0/3937071227 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f6ca808bc00 con 0x7f6cbc0d37b0 2024-03-28T13:18:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.392+0000 7f6cbbe00640 1 --2- 172.21.15.19:0/3937071227 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6c9c06c530 0x7f6c9c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.392+0000 7f6cc4f81740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775720 0x5560c67486c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.392+0000 7f6cc4f81740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775b00 0x5560c67784b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.392+0000 7f6cc4f81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6748c00 0x5560c67789f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.392+0000 7f6cc4f81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5560c66b36b0 con 0x5560c6748c00 2024-03-28T13:18:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.392+0000 7f6cc4f81740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5560c665b750 con 0x5560c6775720 2024-03-28T13:18:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.392+0000 7f6cc4f81740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5560c66af660 con 0x5560c6775b00 2024-03-28T13:18:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.393+0000 7f6cbbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6748c00 0x5560c67789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.393+0000 7f6cbbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6748c00 0x5560c67789f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49370/0 (socket says 172.21.15.19:49370) 2024-03-28T13:18:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.393+0000 7f6cbbe00640 1 -- 172.21.15.19:0/2505284112 learned_addr learned my addr 172.21.15.19:0/2505284112 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:34.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.393+0000 7f6cbb400640 1 --2- 172.21.15.19:0/2505284112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775720 0x5560c67486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.393+0000 7f6cbaa00640 1 --2- 172.21.15.19:0/2505284112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775b00 0x5560c67784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.393+0000 7f6cbbe00640 1 -- 172.21.15.19:0/2505284112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775b00 msgr2=0x5560c67784b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.393+0000 7f6cbbe00640 1 --2- 172.21.15.19:0/2505284112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775b00 0x5560c67784b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.393+0000 7f6cbbe00640 1 -- 172.21.15.19:0/2505284112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775720 msgr2=0x5560c67486c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.393+0000 7f6cbbe00640 1 --2- 172.21.15.19:0/2505284112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775720 0x5560c67486c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.393+0000 7f6cbbe00640 1 -- 172.21.15.19:0/2505284112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c64917d0 con 0x5560c6748c00 2024-03-28T13:18:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.394+0000 7f6cbbe00640 1 --2- 172.21.15.19:0/2505284112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6748c00 0x5560c67789f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4036040 tx=0x7f6cb4046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9996dd7e18f5e982 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.394+0000 7f6cbbe00640 1 --2- 172.21.15.19:0/3937071227 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6c9c06c530 0x7f6c9c06e9f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6cb402bfa0 tx=0x7f6cb402e070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.394+0000 7f6ca7400640 1 -- 172.21.15.19:0/2505284112 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4052070 con 0x5560c6748c00 2024-03-28T13:18:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.394+0000 7f6ca7400640 1 -- 172.21.15.19:0/2505284112 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb402ce50 con 0x5560c6748c00 2024-03-28T13:18:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.394+0000 7f6ca7400640 1 -- 172.21.15.19:0/2505284112 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4043d70 con 0x5560c6748c00 2024-03-28T13:18:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.395+0000 7f6cc4f81740 1 -- 172.21.15.19:0/2505284112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6748c00 msgr2=0x5560c67789f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.395+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/2505284112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6748c00 0x5560c67789f0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4036040 tx=0x7f6cb4046000 comp rx=0 tx=0).stop 2024-03-28T13:18:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.395+0000 7f6cc4f81740 1 -- 172.21.15.19:0/2505284112 shutdown_connections 2024-03-28T13:18:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.395+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/2505284112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6748c00 0x5560c67789f0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.395+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/2505284112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775b00 0x5560c67784b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.395+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/2505284112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775720 0x5560c67486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.395+0000 7f6cc4f81740 1 -- 172.21.15.19:0/2505284112 >> 172.21.15.19:0/2505284112 conn(0x5560c674a930 msgr2=0x5560c67934b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.395+0000 7f6cc4f81740 1 -- 172.21.15.19:0/2505284112 shutdown_connections 2024-03-28T13:18:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.395+0000 7f6cc4f81740 1 -- 172.21.15.19:0/2505284112 wait complete. 2024-03-28T13:18:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.395+0000 7f6cc4f81740 1 -- start start 2024-03-28T13:18:34.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.396+0000 7f6cc4f81740 1 Processor -- start 2024-03-28T13:18:34.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.397+0000 7f6cc4f81740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775720 0x5560c6762ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.397+0000 7f6cc4f81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6775b00 0x5560c6793b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.397+0000 7f6cc4f81740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6765780 0x5560c67643d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.397+0000 7f6cc4f81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5560c674cfd0 con 0x5560c6775b00 2024-03-28T13:18:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.397+0000 7f6cc4f81740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5560c666ea80 con 0x5560c6775720 2024-03-28T13:18:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.397+0000 7f6cc4f81740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5560c674f810 con 0x5560c6765780 2024-03-28T13:18:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.398+0000 7f6cc4f81740 10 client.0 _ll_register_callbacks cb 0x5560c666e470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:18:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.398+0000 7f6cbaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6775b00 0x5560c6793b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.398+0000 7f6cbaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6775b00 0x5560c6793b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49386/0 (socket says 172.21.15.19:49386) 2024-03-28T13:18:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.398+0000 7f6cbaa00640 1 -- 172.21.15.19:0/175673813 learned_addr learned my addr 172.21.15.19:0/175673813 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.398+0000 7f6cbb400640 1 --2- 172.21.15.19:0/175673813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775720 0x5560c6762ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.398+0000 7f6cbaa00640 1 -- 172.21.15.19:0/175673813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6765780 msgr2=0x5560c67643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.398+0000 7f6cbaa00640 1 --2- 172.21.15.19:0/175673813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6765780 0x5560c67643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.398+0000 7f6cbaa00640 1 -- 172.21.15.19:0/175673813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775720 msgr2=0x5560c6762ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:34.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.398+0000 7f6cbaa00640 1 --2- 172.21.15.19:0/175673813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775720 0x5560c6762ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.398+0000 7f6cbaa00640 1 -- 172.21.15.19:0/175673813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c63c73f0 con 0x5560c6775b00 2024-03-28T13:18:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.398+0000 7f6cbaa00640 1 --2- 172.21.15.19:0/175673813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6775b00 0x5560c6793b10 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800e9b0 tx=0x7f6ca805a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a3769072eeec50 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.398+0000 7f6c9be00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:18:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.399+0000 7f6c9be00640 1 -- 172.21.15.19:0/175673813 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8021040 con 0x5560c6775b00 2024-03-28T13:18:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.399+0000 7f6cc4f81740 1 -- 172.21.15.19:0/175673813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cb402ce50 con 0x5560c6775b00 2024-03-28T13:18:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.399+0000 7f6c9be00640 1 -- 172.21.15.19:0/175673813 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f6ca8062320 con 0x5560c6775b00 2024-03-28T13:18:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.399+0000 7f6c9be00640 1 -- 172.21.15.19:0/175673813 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f6ca80625b0 con 0x5560c6775b00 2024-03-28T13:18:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.399+0000 7f6cc4f81740 10 client.7806 fetch_fsmap learned FSMap version 74 2024-03-28T13:18:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.399+0000 7f6cc4f81740 1 -- 172.21.15.19:0/175673813 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5560c63870f0 con 0x5560c6775b00 2024-03-28T13:18:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.399+0000 7f6c9be00640 1 -- 172.21.15.19:0/175673813 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb402ce50 con 0x5560c6775b00 2024-03-28T13:18:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.399+0000 7f6c9be00640 1 -- 172.21.15.19:0/175673813 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f6ca8071020 con 0x5560c6775b00 2024-03-28T13:18:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.399+0000 7f6cc4f81740 10 client.7806 fetch_fsmap finished waiting for FSMap version 74 2024-03-28T13:18:34.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.399+0000 7f6cc4f81740 10 client.7806 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-28T13:18:34.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.399+0000 7f6cc4f81740 20 client.7806 populate_metadata read hostname 'smithi019' 2024-03-28T13:18:34.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.399+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/175673813 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x5560c6765040 0x5560c6779650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.399+0000 7f6cc4f81740 4 client.7806 mds_command: new command op to 6040 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:18:34.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.399+0000 7f6cc4f81740 1 -- 172.21.15.19:0/175673813 --> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560c66c0bf0 con 0x5560c6765040 2024-03-28T13:18:34.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.400+0000 7f6cbb400640 1 --2- 172.21.15.19:0/175673813 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x5560c6765040 0x5560c6779650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.400+0000 7f6cbb400640 1 --2- 172.21.15.19:0/175673813 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x5560c6765040 0x5560c6779650 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e72d718e14423c50 server_cookie=7588bf95f392e42a in_seq=0 out_seq=0 2024-03-28T13:18:34.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.400+0000 7f6c9be00640 10 client.7806 ms_handle_connect on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:34.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.401+0000 7f6c9be00640 1 -- 172.21.15.19:0/175673813 <== mds.0 v2:172.21.15.19:6832/2209799371 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5560c66c0bf0 con 0x5560c6765040 2024-03-28T13:18:34.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.401+0000 7f6c9be00640 10 client.7806 handle_command_reply: tid=0 2024-03-28T13:18:34.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.401+0000 7f6cc4f81740 1 client.7806 shutdown 2024-03-28T13:18:34.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.401+0000 7f6cc4f81740 1 -- 172.21.15.19:0/175673813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6775b00 msgr2=0x5560c6793b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.401+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/175673813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6775b00 0x5560c6793b10 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800e9b0 tx=0x7f6ca805a210 comp rx=0 tx=0).stop 2024-03-28T13:18:34.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.401+0000 7f6cc4f81740 1 -- 172.21.15.19:0/175673813 shutdown_connections 2024-03-28T13:18:34.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.401+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/175673813 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x5560c6765040 0x5560c6779650 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.401+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/175673813 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6765780 0x5560c67643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.401+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/175673813 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6775b00 0x5560c6793b10 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.401+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/175673813 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775720 0x5560c6762ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.401+0000 7f6cc4f81740 1 -- 172.21.15.19:0/175673813 >> 172.21.15.19:0/175673813 conn(0x5560c6748c00 msgr2=0x5560c6796930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:34.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.402+0000 7f6c9be00640 0 client.7806 ms_handle_reset on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:34.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.402+0000 7f6cc4f81740 1 -- 172.21.15.19:0/175673813 shutdown_connections 2024-03-28T13:18:34.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.402+0000 7f6cc4f81740 1 -- 172.21.15.19:0/175673813 wait complete. 2024-03-28T13:18:34.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.402+0000 7f6cc4f81740 20 client.7806 trim_cache size 0 max 16384 2024-03-28T13:18:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.421+0000 7f6cc4f81740 1 Processor -- start 2024-03-28T13:18:34.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.421+0000 7f6cc4f81740 1 -- start start 2024-03-28T13:18:34.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.422+0000 7f6cc4f81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6765780 0x5560c67626d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.422+0000 7f6cc4f81740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775720 0x5560c67b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.422+0000 7f6cc4f81740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775b00 0x5560c67b6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.422+0000 7f6cc4f81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc200ad0 con 0x5560c6765780 2024-03-28T13:18:34.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.422+0000 7f6cc4f81740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc1ffb10 con 0x5560c6775b00 2024-03-28T13:18:34.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.422+0000 7f6cc4f81740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6cbc0cc750 con 0x5560c6775720 2024-03-28T13:18:34.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.423+0000 7f6cbb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6765780 0x5560c67626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.423+0000 7f6cbb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6765780 0x5560c67626d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49402/0 (socket says 172.21.15.19:49402) 2024-03-28T13:18:34.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.423+0000 7f6cbb400640 1 -- 172.21.15.19:0/3209673097 learned_addr learned my addr 172.21.15.19:0/3209673097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.423+0000 7f6cbbe00640 1 --2- 172.21.15.19:0/3209673097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775b00 0x5560c67b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.423+0000 7f6cbaa00640 1 --2- 172.21.15.19:0/3209673097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775720 0x5560c67b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.423+0000 7f6cbb400640 1 -- 172.21.15.19:0/3209673097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775720 msgr2=0x5560c67b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.423+0000 7f6cbb400640 1 --2- 172.21.15.19:0/3209673097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775720 0x5560c67b6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.423+0000 7f6cbb400640 1 -- 172.21.15.19:0/3209673097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775b00 msgr2=0x5560c67b6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.423+0000 7f6cbb400640 1 --2- 172.21.15.19:0/3209673097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775b00 0x5560c67b6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.423+0000 7f6cbb400640 1 -- 172.21.15.19:0/3209673097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c67a2e40 con 0x5560c6765780 2024-03-28T13:18:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.423+0000 7f6cbaa00640 1 --2- 172.21.15.19:0/3209673097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775720 0x5560c67b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:34.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.424+0000 7f6cbb400640 1 --2- 172.21.15.19:0/3209673097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6765780 0x5560c67626d0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f6cac005370 tx=0x7f6cac004b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6983f0f10470ad24 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:34.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.424+0000 7f6ca6a00640 1 -- 172.21.15.19:0/3209673097 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac010040 con 0x5560c6765780 2024-03-28T13:18:34.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.424+0000 7f6ca6a00640 1 -- 172.21.15.19:0/3209673097 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cac01d030 con 0x5560c6765780 2024-03-28T13:18:34.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.424+0000 7f6ca6a00640 1 -- 172.21.15.19:0/3209673097 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac007050 con 0x5560c6765780 2024-03-28T13:18:34.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.424+0000 7f6cc4f81740 1 -- 172.21.15.19:0/3209673097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6765780 msgr2=0x5560c67626d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.424+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/3209673097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6765780 0x5560c67626d0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f6cac005370 tx=0x7f6cac004b30 comp rx=0 tx=0).stop 2024-03-28T13:18:34.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.425+0000 7f6cc4f81740 1 -- 172.21.15.19:0/3209673097 shutdown_connections 2024-03-28T13:18:34.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.425+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/3209673097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775b00 0x5560c67b6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.425+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/3209673097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775720 0x5560c67b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.425+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/3209673097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6765780 0x5560c67626d0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.425+0000 7f6cc4f81740 1 -- 172.21.15.19:0/3209673097 >> 172.21.15.19:0/3209673097 conn(0x5560c6748c00 msgr2=0x5560c6765040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:34.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.425+0000 7f6cc4f81740 1 -- 172.21.15.19:0/3209673097 shutdown_connections 2024-03-28T13:18:34.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.425+0000 7f6cc4f81740 1 -- 172.21.15.19:0/3209673097 wait complete. 2024-03-28T13:18:34.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.426+0000 7f6cc4f81740 1 -- start start 2024-03-28T13:18:34.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.426+0000 7f6cc4f81740 1 Processor -- start 2024-03-28T13:18:34.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.428+0000 7f6cc4f81740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6765780 0x5560c6747fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.428+0000 7f6cc4f81740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775720 0x5560c6779c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.428+0000 7f6cc4f81740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775b00 0x5560c67b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.428+0000 7f6cc4f81740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5560c6762150 con 0x5560c6765780 2024-03-28T13:18:34.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.428+0000 7f6cc4f81740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc0ccda0 con 0x5560c6775b00 2024-03-28T13:18:34.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.428+0000 7f6cc4f81740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5560c6766820 con 0x5560c6775720 2024-03-28T13:18:34.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.428+0000 7f6cc4f81740 10 client.0 _ll_register_callbacks cb 0x7f6ca8059600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:18:34.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.428+0000 7f6cbb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6765780 0x5560c6747fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.428+0000 7f6cbb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6765780 0x5560c6747fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49406/0 (socket says 172.21.15.19:49406) 2024-03-28T13:18:34.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.428+0000 7f6cbb400640 1 -- 172.21.15.19:0/189417706 learned_addr learned my addr 172.21.15.19:0/189417706 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:34.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.428+0000 7f6cbaa00640 1 --2- 172.21.15.19:0/189417706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775720 0x5560c6779c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.429+0000 7f6cbbe00640 1 --2- 172.21.15.19:0/189417706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775b00 0x5560c67b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.429+0000 7f6cbb400640 1 -- 172.21.15.19:0/189417706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775720 msgr2=0x5560c6779c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:34.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.429+0000 7f6cbb400640 1 --2- 172.21.15.19:0/189417706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775720 0x5560c6779c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.429+0000 7f6cbb400640 1 -- 172.21.15.19:0/189417706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775b00 msgr2=0x5560c67b6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:34.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.429+0000 7f6cbb400640 1 --2- 172.21.15.19:0/189417706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775b00 0x5560c67b6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.429+0000 7f6cbb400640 1 -- 172.21.15.19:0/189417706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c6794c30 con 0x5560c6765780 2024-03-28T13:18:34.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.429+0000 7f6cbaa00640 1 --2- 172.21.15.19:0/189417706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775720 0x5560c6779c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:34.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.429+0000 7f6cbbe00640 1 --2- 172.21.15.19:0/189417706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775b00 0x5560c67b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:34.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.429+0000 7f6cbb400640 1 --2- 172.21.15.19:0/189417706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6765780 0x5560c6747fc0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f6cac01d000 tx=0x7f6cac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9131c732718d374 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:34.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.429+0000 7f6c9be00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:18:34.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.429+0000 7f6c9be00640 1 -- 172.21.15.19:0/189417706 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac019870 con 0x5560c6765780 2024-03-28T13:18:34.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.429+0000 7f6c9be00640 1 -- 172.21.15.19:0/189417706 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f6cac02db80 con 0x5560c6765780 2024-03-28T13:18:34.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.429+0000 7f6cc4f81740 1 -- 172.21.15.19:0/189417706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cac01d030 con 0x5560c6765780 2024-03-28T13:18:34.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.430+0000 7f6c9be00640 1 -- 172.21.15.19:0/189417706 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f6cac02dd80 con 0x5560c6765780 2024-03-28T13:18:34.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.430+0000 7f6cc4f81740 10 client.7812 fetch_fsmap learned FSMap version 74 2024-03-28T13:18:34.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.430+0000 7f6cc4f81740 1 -- 172.21.15.19:0/189417706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5560c677a990 con 0x5560c6765780 2024-03-28T13:18:34.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.430+0000 7f6c9be00640 1 -- 172.21.15.19:0/189417706 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cac01d030 con 0x5560c6765780 2024-03-28T13:18:34.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.430+0000 7f6c9be00640 1 -- 172.21.15.19:0/189417706 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f6cac016040 con 0x5560c6765780 2024-03-28T13:18:34.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.430+0000 7f6cc4f81740 10 client.7812 fetch_fsmap finished waiting for FSMap version 74 2024-03-28T13:18:34.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.430+0000 7f6cc4f81740 10 client.7812 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-03-28T13:18:34.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.430+0000 7f6cc4f81740 20 client.7812 populate_metadata read hostname 'smithi019' 2024-03-28T13:18:34.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.430+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/189417706 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x5560c674e9f0 0x5560c67b6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.430+0000 7f6cc4f81740 4 client.7812 mds_command: new command op to 6040 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:18:34.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.430+0000 7f6cc4f81740 1 -- 172.21.15.19:0/189417706 --> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5560c66baf40 con 0x5560c674e9f0 2024-03-28T13:18:34.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.431+0000 7f6cbaa00640 1 --2- 172.21.15.19:0/189417706 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x5560c674e9f0 0x5560c67b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.431+0000 7f6cbaa00640 1 --2- 172.21.15.19:0/189417706 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x5560c674e9f0 0x5560c67b6560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79083379aeb813a9 server_cookie=8077962c546c3c09 in_seq=0 out_seq=0 2024-03-28T13:18:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.431+0000 7f6c9be00640 10 client.7812 ms_handle_connect on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.431+0000 7f6c9be00640 1 -- 172.21.15.19:0/189417706 <== mds.0 v2:172.21.15.19:6832/2209799371 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5560c66baf40 con 0x5560c674e9f0 2024-03-28T13:18:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.431+0000 7f6c9be00640 10 client.7812 handle_command_reply: tid=0 2024-03-28T13:18:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.431+0000 7f6cc4f81740 1 client.7812 shutdown 2024-03-28T13:18:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.432+0000 7f6cc4f81740 1 -- 172.21.15.19:0/189417706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6765780 msgr2=0x5560c6747fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.432+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/189417706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6765780 0x5560c6747fc0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f6cac01d000 tx=0x7f6cac000bc0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.432+0000 7f6cc4f81740 1 -- 172.21.15.19:0/189417706 shutdown_connections 2024-03-28T13:18:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.432+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/189417706 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x5560c674e9f0 0x5560c67b6560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.432+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/189417706 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560c6775b00 0x5560c67b6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.432+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/189417706 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560c6775720 0x5560c6779c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.432+0000 7f6cc4f81740 1 --2- 172.21.15.19:0/189417706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5560c6765780 0x5560c6747fc0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.432+0000 7f6c9be00640 0 client.7812 ms_handle_reset on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.432+0000 7f6cc4f81740 1 -- 172.21.15.19:0/189417706 >> 172.21.15.19:0/189417706 conn(0x5560c6748c00 msgr2=0x5560c67626d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:34.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.432+0000 7f6cc4f81740 1 -- 172.21.15.19:0/189417706 shutdown_connections 2024-03-28T13:18:34.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.432+0000 7f6cc4f81740 1 -- 172.21.15.19:0/189417706 wait complete. 2024-03-28T13:18:34.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.432+0000 7f6cc4f81740 20 client.7812 trim_cache size 0 max 16384 2024-03-28T13:18:34.437 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:18:34.437 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-03-28T13:18:34.437 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-03-28T13:18:34.437 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:18:34.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.433+0000 7f6cc0c00640 1 -- 172.21.15.19:0/3937071227 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6c9c06c530 msgr2=0x7f6c9c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.433+0000 7f6cc0c00640 1 --2- 172.21.15.19:0/3937071227 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6c9c06c530 0x7f6c9c06e9f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6cb402bfa0 tx=0x7f6cb402e070 comp rx=0 tx=0).stop 2024-03-28T13:18:34.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.433+0000 7f6cc0c00640 1 -- 172.21.15.19:0/3937071227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cbc0d37b0 msgr2=0x7f6cbc155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.433+0000 7f6cc0c00640 1 --2- 172.21.15.19:0/3937071227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cbc0d37b0 0x7f6cbc155400 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800e9e0 tx=0x7f6ca800eea0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.433+0000 7f6cc0c00640 1 -- 172.21.15.19:0/3937071227 shutdown_connections 2024-03-28T13:18:34.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.433+0000 7f6cc0c00640 1 --2- 172.21.15.19:0/3937071227 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6c9c06c530 0x7f6c9c06e9f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.433+0000 7f6cc0c00640 1 --2- 172.21.15.19:0/3937071227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6cbc15e4c0 0x7f6cbc1485a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.433+0000 7f6cc0c00640 1 --2- 172.21.15.19:0/3937071227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6cbc0d37b0 0x7f6cbc155400 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.433+0000 7f6cc0c00640 1 --2- 172.21.15.19:0/3937071227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6cbc0b67f0 0x7f6cbc146590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.433+0000 7f6cc0c00640 1 -- 172.21.15.19:0/3937071227 >> 172.21.15.19:0/3937071227 conn(0x7f6cbc0676a0 msgr2=0x7f6cbc0c96e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:34.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.433+0000 7f6cc0c00640 1 -- 172.21.15.19:0/3937071227 shutdown_connections 2024-03-28T13:18:34.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.433+0000 7f6cc0c00640 1 -- 172.21.15.19:0/3937071227 wait complete. 2024-03-28T13:18:34.449 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T13:18:34.449 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:18:34.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.651+0000 7fd756600640 1 Processor -- start 2024-03-28T13:18:34.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.651+0000 7fd756600640 1 -- start start 2024-03-28T13:18:34.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.653+0000 7fd756600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7500b4530 0x7fd7500c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.653+0000 7fd756600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd7500d1610 0x7fd7500c7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.653+0000 7fd756600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd75015eae0 con 0x7fd75015e700 2024-03-28T13:18:34.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.653+0000 7fd756600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fd7500cfeb0 con 0x7fd7500d1610 2024-03-28T13:18:34.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.653+0000 7fd756600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd7500d0190 con 0x7fd7500b4530 2024-03-28T13:18:34.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.654+0000 7fd747e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7500b4530 0x7fd7500c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.654+0000 7fd74fe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd7500d1610 0x7fd7500c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.654+0000 7fd74f400640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd75015e700 0x7fd7500b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:44982/0 (socket says 172.21.15.19:44982) 2024-03-28T13:18:34.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.654+0000 7fd74fe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd7500d1610 0x7fd7500c7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:54590/0 (socket says 172.21.15.19:54590) 2024-03-28T13:18:34.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.654+0000 7fd74f400640 1 -- 172.21.15.19:0/2639029735 learned_addr learned my addr 172.21.15.19:0/2639029735 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:34.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.654+0000 7fd747e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7500b4530 0x7fd7500c7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:60530/0 (socket says 172.21.15.19:60530) 2024-03-28T13:18:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.655+0000 7fd74fe00640 1 -- 172.21.15.19:0/2639029735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7500b4530 msgr2=0x7fd7500c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.655+0000 7fd74fe00640 1 --2- 172.21.15.19:0/2639029735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7500b4530 0x7fd7500c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.655+0000 7fd74fe00640 1 -- 172.21.15.19:0/2639029735 >> v1:172.21.15.188:6789/0 conn(0x7fd75015e700 legacy=0x7fd7500b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.655+0000 7fd74fe00640 1 -- 172.21.15.19:0/2639029735 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7500d0470 con 0x7fd7500d1610 2024-03-28T13:18:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.655+0000 7fd747e00640 1 --2- 172.21.15.19:0/2639029735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7500b4530 0x7fd7500c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:18:34.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.656+0000 7fd74fe00640 1 --2- 172.21.15.19:0/2639029735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7500d1610 0x7fd7500c7640 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd748009d90 tx=0x7fd74802be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cbde23219a63a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:34.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.657+0000 7fd74ea00640 1 -- 172.21.15.19:0/2639029735 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748039070 con 0x7fd7500d1610 2024-03-28T13:18:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.657+0000 7fd74ea00640 1 -- 172.21.15.19:0/2639029735 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd74802cbd0 con 0x7fd7500d1610 2024-03-28T13:18:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.657+0000 7fd74ea00640 1 -- 172.21.15.19:0/2639029735 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd748034440 con 0x7fd7500d1610 2024-03-28T13:18:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.657+0000 7fd756600640 1 -- 172.21.15.19:0/2639029735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7500d1610 msgr2=0x7fd7500c7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.657+0000 7fd756600640 1 --2- 172.21.15.19:0/2639029735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7500d1610 0x7fd7500c7640 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd748009d90 tx=0x7fd74802be20 comp rx=0 tx=0).stop 2024-03-28T13:18:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.657+0000 7fd756600640 1 -- 172.21.15.19:0/2639029735 shutdown_connections 2024-03-28T13:18:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.657+0000 7fd756600640 1 --2- 172.21.15.19:0/2639029735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7500d1610 0x7fd7500c7640 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.657+0000 7fd756600640 1 --2- 172.21.15.19:0/2639029735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7500b4530 0x7fd7500c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.657+0000 7fd756600640 1 -- 172.21.15.19:0/2639029735 >> 172.21.15.19:0/2639029735 conn(0x7fd7500d2240 msgr2=0x7fd7500d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.657+0000 7fd756600640 1 -- 172.21.15.19:0/2639029735 shutdown_connections 2024-03-28T13:18:34.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.657+0000 7fd756600640 1 -- 172.21.15.19:0/2639029735 wait complete. 2024-03-28T13:18:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.659+0000 7fd756600640 1 Processor -- start 2024-03-28T13:18:34.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.659+0000 7fd756600640 1 -- start start 2024-03-28T13:18:34.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.660+0000 7fd756600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7500b4530 0x7fd750148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.660+0000 7fd756600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7500d1610 0x7fd750161540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.660+0000 7fd756600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd75015e700 0x7fd7500b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.660+0000 7fd756600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd75006d1b0 con 0x7fd7500b4530 2024-03-28T13:18:34.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.660+0000 7fd756600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd7500d0a30 con 0x7fd7500d1610 2024-03-28T13:18:34.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.660+0000 7fd756600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd75006d330 con 0x7fd75015e700 2024-03-28T13:18:34.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.661+0000 7fd74f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7500b4530 0x7fd750148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.661+0000 7fd74f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7500b4530 0x7fd750148dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49416/0 (socket says 172.21.15.19:49416) 2024-03-28T13:18:34.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.661+0000 7fd74f400640 1 -- 172.21.15.19:0/1472610103 learned_addr learned my addr 172.21.15.19:0/1472610103 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.661+0000 7fd747e00640 1 --2- 172.21.15.19:0/1472610103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7500d1610 0x7fd750161540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.661+0000 7fd74fe00640 1 --2- 172.21.15.19:0/1472610103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd75015e700 0x7fd7500b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.661+0000 7fd74f400640 1 -- 172.21.15.19:0/1472610103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd75015e700 msgr2=0x7fd7500b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.661+0000 7fd74f400640 1 --2- 172.21.15.19:0/1472610103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd75015e700 0x7fd7500b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.661+0000 7fd74f400640 1 -- 172.21.15.19:0/1472610103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7500d1610 msgr2=0x7fd750161540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.661+0000 7fd74f400640 1 --2- 172.21.15.19:0/1472610103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7500d1610 0x7fd750161540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.661+0000 7fd74f400640 1 -- 172.21.15.19:0/1472610103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7501604a0 con 0x7fd7500b4530 2024-03-28T13:18:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.662+0000 7fd74f400640 1 --2- 172.21.15.19:0/1472610103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7500b4530 0x7fd750148dd0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fd74000c3c0 tx=0x7fd74000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.662+0000 7fd747400640 1 -- 172.21.15.19:0/1472610103 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd740017070 con 0x7fd7500b4530 2024-03-28T13:18:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.662+0000 7fd747400640 1 -- 172.21.15.19:0/1472610103 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd740005ce0 con 0x7fd7500b4530 2024-03-28T13:18:34.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.662+0000 7fd747400640 1 -- 172.21.15.19:0/1472610103 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7400072f0 con 0x7fd7500b4530 2024-03-28T13:18:34.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.662+0000 7fd756600640 1 -- 172.21.15.19:0/1472610103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd750160780 con 0x7fd7500b4530 2024-03-28T13:18:34.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.662+0000 7fd756600640 1 -- 172.21.15.19:0/1472610103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd750160d40 con 0x7fd7500b4530 2024-03-28T13:18:34.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.663+0000 7fd756600640 1 -- 172.21.15.19:0/1472610103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7500685c0 con 0x7fd7500b4530 2024-03-28T13:18:34.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.664+0000 7fd747400640 1 -- 172.21.15.19:0/1472610103 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd74001a070 con 0x7fd7500b4530 2024-03-28T13:18:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.666+0000 7fd747400640 1 --2- 172.21.15.19:0/1472610103 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd72c06c530 0x7fd72c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:34.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.666+0000 7fd747400640 1 -- 172.21.15.19:0/1472610103 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7fd74008ab10 con 0x7fd7500b4530 2024-03-28T13:18:34.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.666+0000 7fd747e00640 1 --2- 172.21.15.19:0/1472610103 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd72c06c530 0x7fd72c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:34.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.667+0000 7fd747e00640 1 --2- 172.21.15.19:0/1472610103 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd72c06c530 0x7fd72c06e9f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd738009560 tx=0x7fd738008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:34.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.671+0000 7fd747400640 1 -- 172.21.15.19:0/1472610103 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd740059070 con 0x7fd7500b4530 2024-03-28T13:18:34.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.918+0000 7fd756600640 1 -- 172.21.15.19:0/1472610103 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd75015aec0 con 0x7fd7500b4530 2024-03-28T13:18:34.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.919+0000 7fd747400640 1 -- 172.21.15.19:0/1472610103 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5188 (secure 0 0 0) 0x7fd740058970 con 0x7fd7500b4530 2024-03-28T13:18:34.918 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:34.918 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1144703580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1144703580},{"type":"v1","addr":"172.21.15.188:6836","nonce":1144703580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:17:59.710623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6040":{"gid":6040,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":28,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6040,"qdb_cluster":[6040]},"id":6}]} 2024-03-28T13:18:34.918 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 74 2024-03-28T13:18:34.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.922+0000 7fd756600640 1 -- 172.21.15.19:0/1472610103 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd72c06c530 msgr2=0x7fd72c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.922+0000 7fd756600640 1 --2- 172.21.15.19:0/1472610103 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd72c06c530 0x7fd72c06e9f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd738009560 tx=0x7fd738008040 comp rx=0 tx=0).stop 2024-03-28T13:18:34.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.922+0000 7fd756600640 1 -- 172.21.15.19:0/1472610103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7500b4530 msgr2=0x7fd750148dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:34.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.922+0000 7fd756600640 1 --2- 172.21.15.19:0/1472610103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7500b4530 0x7fd750148dd0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fd74000c3c0 tx=0x7fd74000c880 comp rx=0 tx=0).stop 2024-03-28T13:18:34.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.922+0000 7fd756600640 1 -- 172.21.15.19:0/1472610103 shutdown_connections 2024-03-28T13:18:34.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.922+0000 7fd756600640 1 --2- 172.21.15.19:0/1472610103 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd72c06c530 0x7fd72c06e9f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.922+0000 7fd756600640 1 --2- 172.21.15.19:0/1472610103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd75015e700 0x7fd7500b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.922+0000 7fd756600640 1 --2- 172.21.15.19:0/1472610103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7500d1610 0x7fd750161540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.922+0000 7fd756600640 1 --2- 172.21.15.19:0/1472610103 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7500b4530 0x7fd750148dd0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:34.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.922+0000 7fd756600640 1 -- 172.21.15.19:0/1472610103 >> 172.21.15.19:0/1472610103 conn(0x7fd7500d2240 msgr2=0x7fd75015a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:34.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.922+0000 7fd756600640 1 -- 172.21.15.19:0/1472610103 shutdown_connections 2024-03-28T13:18:34.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:34.922+0000 7fd756600640 1 -- 172.21.15.19:0/1472610103 wait complete. 2024-03-28T13:18:34.940 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-03-28T13:18:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.160+0000 7f2701400640 1 Processor -- start 2024-03-28T13:18:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.160+0000 7f2701400640 1 -- start start 2024-03-28T13:18:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.162+0000 7f2701400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26fc0d37b0 0x7f26fc0b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.162+0000 7f2701400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26fc0d2710 con 0x7f26fc15e4c0 2024-03-28T13:18:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.162+0000 7f2701400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26fc0d29f0 con 0x7f26fc0b67f0 2024-03-28T13:18:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.162+0000 7f2701400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f26fc15e8a0 con 0x7f26fc0d37b0 2024-03-28T13:18:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.162+0000 7f26fbe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f26fc15e4c0 0x7f26fc0b9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:36982/0 (socket says 172.21.15.19:36982) 2024-03-28T13:18:35.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.162+0000 7f26fbe00640 1 -- 172.21.15.19:0/3583608844 learned_addr learned my addr 172.21.15.19:0/3583608844 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:35.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.163+0000 7f26fb400640 1 --2- 172.21.15.19:0/3583608844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26fc0d37b0 0x7f26fc0b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.163+0000 7f26faa00640 1 -- 172.21.15.19:0/3583608844 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4225739646 0 0) 0x7f26fc0d2710 con 0x7f26fc15e4c0 2024-03-28T13:18:35.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.163+0000 7f26faa00640 1 -- 172.21.15.19:0/3583608844 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26e0003520 con 0x7f26fc15e4c0 2024-03-28T13:18:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.163+0000 7f26faa00640 1 -- 172.21.15.19:0/3583608844 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1545118626 0 0) 0x7f26e0003520 con 0x7f26fc15e4c0 2024-03-28T13:18:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.163+0000 7f26faa00640 1 -- 172.21.15.19:0/3583608844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26fc0d37b0 msgr2=0x7f26fc0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.163+0000 7f26faa00640 1 --2- 172.21.15.19:0/3583608844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26fc0d37b0 0x7f26fc0b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.163+0000 7f26faa00640 1 -- 172.21.15.19:0/3583608844 >> v1:172.21.15.188:6789/0 conn(0x7f26fc0b67f0 legacy=0x7f26fc0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.163+0000 7f26faa00640 1 -- 172.21.15.19:0/3583608844 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc0d2cd0 con 0x7f26fc15e4c0 2024-03-28T13:18:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.163+0000 7f26faa00640 1 -- 172.21.15.19:0/3583608844 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f26ec001670 con 0x7f26fc15e4c0 2024-03-28T13:18:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.163+0000 7f26faa00640 1 -- 172.21.15.19:0/3583608844 >> v1:172.21.15.19:6789/0 conn(0x7f26fc15e4c0 legacy=0x7f26fc0b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.164+0000 7f26faa00640 1 --2- 172.21.15.19:0/3583608844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e00039e0 0x7f26e0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.164+0000 7f26faa00640 1 --2- 172.21.15.19:0/3583608844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e0004390 0x7f26e0004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.164+0000 7f26faa00640 1 --2- 172.21.15.19:0/3583608844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26fc0d37b0 0x7f26e00050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.164+0000 7f26faa00640 1 -- 172.21.15.19:0/3583608844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26e000b9c0 con 0x7f26e00039e0 2024-03-28T13:18:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.164+0000 7f26faa00640 1 -- 172.21.15.19:0/3583608844 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26e000bb40 con 0x7f26e0004390 2024-03-28T13:18:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.164+0000 7f26faa00640 1 -- 172.21.15.19:0/3583608844 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f26e000bd40 con 0x7f26fc0d37b0 2024-03-28T13:18:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.164+0000 7f26fbe00640 1 --2- 172.21.15.19:0/3583608844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e00039e0 0x7f26e0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.164+0000 7f26fb400640 1 --2- 172.21.15.19:0/3583608844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e0004390 0x7f26e0004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.164+0000 7f2700a00640 1 --2- 172.21.15.19:0/3583608844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26fc0d37b0 0x7f26e00050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.164+0000 7f26fbe00640 1 -- 172.21.15.19:0/3583608844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26fc0d37b0 msgr2=0x7f26e00050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.164+0000 7f26fbe00640 1 --2- 172.21.15.19:0/3583608844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26fc0d37b0 0x7f26e00050c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.164+0000 7f26fbe00640 1 -- 172.21.15.19:0/3583608844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e0004390 msgr2=0x7f26e0004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.164+0000 7f26fbe00640 1 --2- 172.21.15.19:0/3583608844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e0004390 0x7f26e0004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.164+0000 7f26fbe00640 1 -- 172.21.15.19:0/3583608844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e0003520 con 0x7f26e00039e0 2024-03-28T13:18:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.165+0000 7f26fb400640 1 --2- 172.21.15.19:0/3583608844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e0004390 0x7f26e0004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.166+0000 7f26fbe00640 1 --2- 172.21.15.19:0/3583608844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e00039e0 0x7f26e0003dc0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00c240 tx=0x7f26e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f32ac6d32e990d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.167+0000 7f26faa00640 1 -- 172.21.15.19:0/3583608844 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec007d30 con 0x7f26e00039e0 2024-03-28T13:18:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.167+0000 7f26faa00640 1 -- 172.21.15.19:0/3583608844 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26ec030070 con 0x7f26e00039e0 2024-03-28T13:18:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.167+0000 7f26faa00640 1 -- 172.21.15.19:0/3583608844 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ec02cd60 con 0x7f26e00039e0 2024-03-28T13:18:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.167+0000 7f2701400640 1 -- 172.21.15.19:0/3583608844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e00039e0 msgr2=0x7f26e0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.167+0000 7f2701400640 1 --2- 172.21.15.19:0/3583608844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26e00039e0 0x7f26e0003dc0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00c240 tx=0x7f26e0005830 comp rx=0 tx=0).stop 2024-03-28T13:18:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.167+0000 7f2700a00640 1 -- 172.21.15.19:0/3583608844 reap_dead start 2024-03-28T13:18:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.167+0000 7f2701400640 1 -- 172.21.15.19:0/3583608844 shutdown_connections 2024-03-28T13:18:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.167+0000 7f2701400640 1 -- 172.21.15.19:0/3583608844 >> 172.21.15.19:0/3583608844 conn(0x7f26fc0676a0 msgr2=0x7f26fc067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.167+0000 7f2701400640 1 -- 172.21.15.19:0/3583608844 shutdown_connections 2024-03-28T13:18:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.168+0000 7f2701400640 1 -- 172.21.15.19:0/3583608844 wait complete. 2024-03-28T13:18:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.169+0000 7f2701400640 1 Processor -- start 2024-03-28T13:18:35.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.169+0000 7f2701400640 1 -- start start 2024-03-28T13:18:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.170+0000 7f2701400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e00039e0 0x7f26fc0b9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.170+0000 7f2701400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26fc160300 0x7f26fc1535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.170+0000 7f2701400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc1f6b40 0x7f26fc1f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.170+0000 7f2701400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc0d3320 con 0x7f26fc1f6b40 2024-03-28T13:18:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.170+0000 7f2701400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc0d4040 con 0x7f26e00039e0 2024-03-28T13:18:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.170+0000 7f2701400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f26fc0d2350 con 0x7f26fc160300 2024-03-28T13:18:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.171+0000 7f2700a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc1f6b40 0x7f26fc1f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.171+0000 7f2700a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc1f6b40 0x7f26fc1f6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49438/0 (socket says 172.21.15.19:49438) 2024-03-28T13:18:35.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.171+0000 7f2700a00640 1 -- 172.21.15.19:0/1985938339 learned_addr learned my addr 172.21.15.19:0/1985938339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.171+0000 7f26fbe00640 1 --2- 172.21.15.19:0/1985938339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e00039e0 0x7f26fc0b9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.171+0000 7f26fb400640 1 --2- 172.21.15.19:0/1985938339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26fc160300 0x7f26fc1535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.171+0000 7f2700a00640 1 -- 172.21.15.19:0/1985938339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26fc160300 msgr2=0x7f26fc1535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.171+0000 7f2700a00640 1 --2- 172.21.15.19:0/1985938339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26fc160300 0x7f26fc1535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.171+0000 7f2700a00640 1 -- 172.21.15.19:0/1985938339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e00039e0 msgr2=0x7f26fc0b9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.171+0000 7f2700a00640 1 --2- 172.21.15.19:0/1985938339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e00039e0 0x7f26fc0b9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.171+0000 7f2700a00640 1 -- 172.21.15.19:0/1985938339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc160790 con 0x7f26fc1f6b40 2024-03-28T13:18:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.172+0000 7f2700a00640 1 --2- 172.21.15.19:0/1985938339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc1f6b40 0x7f26fc1f6fd0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f26fc161e90 tx=0x7f26f400db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.172+0000 7f26f3e00640 1 -- 172.21.15.19:0/1985938339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4018070 con 0x7f26fc1f6b40 2024-03-28T13:18:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.172+0000 7f26f3e00640 1 -- 172.21.15.19:0/1985938339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f4003b70 con 0x7f26fc1f6b40 2024-03-28T13:18:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.172+0000 7f26f3e00640 1 -- 172.21.15.19:0/1985938339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4005070 con 0x7f26fc1f6b40 2024-03-28T13:18:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.172+0000 7f2701400640 1 -- 172.21.15.19:0/1985938339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26fc160a70 con 0x7f26fc1f6b40 2024-03-28T13:18:35.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.172+0000 7f2701400640 1 -- 172.21.15.19:0/1985938339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26fc161030 con 0x7f26fc1f6b40 2024-03-28T13:18:35.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.175+0000 7f26f3e00640 1 -- 172.21.15.19:0/1985938339 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f26f401b070 con 0x7f26fc1f6b40 2024-03-28T13:18:35.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.176+0000 7f26f3e00640 1 --2- 172.21.15.19:0/1985938339 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f26dc06c560 0x7f26dc06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.176+0000 7f26f3e00640 1 -- 172.21.15.19:0/1985938339 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f26f408afe0 con 0x7f26fc1f6b40 2024-03-28T13:18:35.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.176+0000 7f2705764740 1 Processor -- start 2024-03-28T13:18:35.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.176+0000 7f26fbe00640 1 --2- 172.21.15.19:0/1985938339 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f26dc06c560 0x7f26dc06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.177+0000 7f2705764740 1 -- start start 2024-03-28T13:18:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.178+0000 7f2705764740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764de28a0 0x557764de2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.178+0000 7f2705764740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764db28c0 0x557764ddfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.178+0000 7f2705764740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764db6420 0x557764de0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.178+0000 7f26fbe00640 1 --2- 172.21.15.19:0/1985938339 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f26dc06c560 0x7f26dc06ea20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00c240 tx=0x7f26ec009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.178+0000 7f2705764740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557764d1c5a0 con 0x557764db28c0 2024-03-28T13:18:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.178+0000 7f2705764740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557764cc4390 con 0x557764de28a0 2024-03-28T13:18:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.178+0000 7f2705764740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x557764d18550 con 0x557764db6420 2024-03-28T13:18:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.178+0000 7f26fb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764db28c0 0x557764ddfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.178+0000 7f26fb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764db28c0 0x557764ddfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49454/0 (socket says 172.21.15.19:49454) 2024-03-28T13:18:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.178+0000 7f26fb400640 1 -- 172.21.15.19:0/3591581117 learned_addr learned my addr 172.21.15.19:0/3591581117 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.178+0000 7f26fbe00640 1 --2- 172.21.15.19:0/3591581117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764de28a0 0x557764de2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.178+0000 7f2700a00640 1 --2- 172.21.15.19:0/3591581117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764db6420 0x557764de0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.178+0000 7f26fb400640 1 -- 172.21.15.19:0/3591581117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764db6420 msgr2=0x557764de0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.178+0000 7f26fb400640 1 --2- 172.21.15.19:0/3591581117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764db6420 0x557764de0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.179+0000 7f26fb400640 1 -- 172.21.15.19:0/3591581117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764de28a0 msgr2=0x557764de2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.179+0000 7f26fb400640 1 --2- 172.21.15.19:0/3591581117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764de28a0 0x557764de2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.179+0000 7f26fb400640 1 -- 172.21.15.19:0/3591581117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557764afc7c0 con 0x557764db28c0 2024-03-28T13:18:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.179+0000 7f2700a00640 1 --2- 172.21.15.19:0/3591581117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764db6420 0x557764de0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.179+0000 7f26fb400640 1 --2- 172.21.15.19:0/3591581117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764db28c0 0x557764ddfb10 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009a10 tx=0x7f26e8009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e36035f5f82d10 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.179+0000 7f26e7e00640 1 -- 172.21.15.19:0/3591581117 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8019070 con 0x557764db28c0 2024-03-28T13:18:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.179+0000 7f26e7e00640 1 -- 172.21.15.19:0/3591581117 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e8004230 con 0x557764db28c0 2024-03-28T13:18:35.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.179+0000 7f26e7e00640 1 -- 172.21.15.19:0/3591581117 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e800f040 con 0x557764db28c0 2024-03-28T13:18:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.180+0000 7f2705764740 1 -- 172.21.15.19:0/3591581117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764db28c0 msgr2=0x557764ddfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.180+0000 7f2705764740 1 --2- 172.21.15.19:0/3591581117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764db28c0 0x557764ddfb10 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f26e8009a10 tx=0x7f26e8009190 comp rx=0 tx=0).stop 2024-03-28T13:18:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.180+0000 7f2705764740 1 -- 172.21.15.19:0/3591581117 shutdown_connections 2024-03-28T13:18:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.180+0000 7f2705764740 1 --2- 172.21.15.19:0/3591581117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764db6420 0x557764de0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.180+0000 7f2705764740 1 --2- 172.21.15.19:0/3591581117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764db28c0 0x557764ddfb10 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.180+0000 7f2705764740 1 --2- 172.21.15.19:0/3591581117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764de28a0 0x557764de2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.180+0000 7f2705764740 1 -- 172.21.15.19:0/3591581117 >> 172.21.15.19:0/3591581117 conn(0x557764db4b30 msgr2=0x557764dfd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:35.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.180+0000 7f2705764740 1 -- 172.21.15.19:0/3591581117 shutdown_connections 2024-03-28T13:18:35.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.180+0000 7f2705764740 1 -- 172.21.15.19:0/3591581117 wait complete. 2024-03-28T13:18:35.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.181+0000 7f2705764740 1 -- start start 2024-03-28T13:18:35.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.181+0000 7f2705764740 1 Processor -- start 2024-03-28T13:18:35.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.182+0000 7f2705764740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 0x557764de5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.182+0000 7f2705764740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de28a0 0x557764ddd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.182+0000 7f2705764740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764dd81d0 0x557764dcd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.182+0000 7f2705764740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557764db75d0 con 0x557764dd81d0 2024-03-28T13:18:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.182+0000 7f2705764740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557764cd7780 con 0x557764db6420 2024-03-28T13:18:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.182+0000 7f2705764740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x557764db69c0 con 0x557764de28a0 2024-03-28T13:18:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.182+0000 7f2705764740 10 client.0 _ll_register_callbacks cb 0x557764d13fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:18:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.182+0000 7f2700a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764dd81d0 0x557764dcd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.183+0000 7f2700a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764dd81d0 0x557764dcd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49460/0 (socket says 172.21.15.19:49460) 2024-03-28T13:18:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.183+0000 7f2700a00640 1 -- 172.21.15.19:0/2218605013 learned_addr learned my addr 172.21.15.19:0/2218605013 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.183+0000 7f26fb400640 1 --2- 172.21.15.19:0/2218605013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de28a0 0x557764ddd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.183+0000 7f26fbe00640 1 --2- 172.21.15.19:0/2218605013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 0x557764de5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.183+0000 7f2700a00640 1 -- 172.21.15.19:0/2218605013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de28a0 msgr2=0x557764ddd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.183+0000 7f2700a00640 1 --2- 172.21.15.19:0/2218605013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de28a0 0x557764ddd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.183+0000 7f2700a00640 1 -- 172.21.15.19:0/2218605013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 msgr2=0x557764de5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.183+0000 7f2700a00640 1 --2- 172.21.15.19:0/2218605013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 0x557764de5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.183+0000 7f2700a00640 1 -- 172.21.15.19:0/2218605013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557764b5cf50 con 0x557764dd81d0 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.183+0000 7f26fbe00640 1 --2- 172.21.15.19:0/2218605013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 0x557764de5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.183+0000 7f2700a00640 1 --2- 172.21.15.19:0/2218605013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764dd81d0 0x557764dcd710 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f26f4089380 tx=0x7f26f4058f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bbf3e4c224c038 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.183+0000 7f26e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.183+0000 7f26e4c00640 1 -- 172.21.15.19:0/2218605013 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4087060 con 0x557764dd81d0 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.183+0000 7f26e4c00640 1 -- 172.21.15.19:0/2218605013 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f26f4006b50 con 0x557764dd81d0 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.183+0000 7f2705764740 1 -- 172.21.15.19:0/2218605013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26e8004230 con 0x557764dd81d0 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.184+0000 7f26e4c00640 1 -- 172.21.15.19:0/2218605013 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f26f405c070 con 0x557764dd81d0 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.184+0000 7f2705764740 10 client.7827 fetch_fsmap learned FSMap version 74 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.184+0000 7f2705764740 1 -- 172.21.15.19:0/2218605013 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5577649f00b0 con 0x557764dd81d0 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.184+0000 7f26e4c00640 1 -- 172.21.15.19:0/2218605013 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e8004230 con 0x557764dd81d0 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.184+0000 7f26e4c00640 1 -- 172.21.15.19:0/2218605013 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f26f4058220 con 0x557764dd81d0 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.184+0000 7f2705764740 10 client.7827 fetch_fsmap finished waiting for FSMap version 74 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.184+0000 7f2705764740 10 client.7827 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.184+0000 7f2705764740 20 client.7827 populate_metadata read hostname 'smithi019' 2024-03-28T13:18:35.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.184+0000 7f2705764740 1 --2- 172.21.15.19:0/2218605013 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x557764dd8830 0x557764dd8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.184+0000 7f2705764740 4 client.7827 mds_command: new command op to 6040 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:18:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.184+0000 7f2705764740 1 -- 172.21.15.19:0/2218605013 --> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557764cf0870 con 0x557764dd8830 2024-03-28T13:18:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.185+0000 7f26fbe00640 1 --2- 172.21.15.19:0/2218605013 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x557764dd8830 0x557764dd8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.185+0000 7f26fbe00640 1 --2- 172.21.15.19:0/2218605013 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x557764dd8830 0x557764dd8c10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9a3c35d3652c143 server_cookie=8b4cc1568b2180ef in_seq=0 out_seq=0 2024-03-28T13:18:35.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.185+0000 7f26e4c00640 10 client.7827 ms_handle_connect on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.186+0000 7f26e4c00640 1 -- 172.21.15.19:0/2218605013 <== mds.0 v2:172.21.15.19:6832/2209799371 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557764cf0870 con 0x557764dd8830 2024-03-28T13:18:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.186+0000 7f26e4c00640 10 client.7827 handle_command_reply: tid=0 2024-03-28T13:18:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.186+0000 7f2705764740 1 client.7827 shutdown 2024-03-28T13:18:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.186+0000 7f2705764740 1 -- 172.21.15.19:0/2218605013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764dd81d0 msgr2=0x557764dcd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.186+0000 7f2705764740 1 --2- 172.21.15.19:0/2218605013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764dd81d0 0x557764dcd710 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f26f4089380 tx=0x7f26f4058f10 comp rx=0 tx=0).stop 2024-03-28T13:18:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.186+0000 7f2705764740 1 -- 172.21.15.19:0/2218605013 shutdown_connections 2024-03-28T13:18:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.186+0000 7f2705764740 1 --2- 172.21.15.19:0/2218605013 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x557764dd8830 0x557764dd8c10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.186+0000 7f2705764740 1 --2- 172.21.15.19:0/2218605013 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764dd81d0 0x557764dcd710 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.186+0000 7f2705764740 1 --2- 172.21.15.19:0/2218605013 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de28a0 0x557764ddd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.186+0000 7f2705764740 1 --2- 172.21.15.19:0/2218605013 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 0x557764de5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.186+0000 7f2705764740 1 -- 172.21.15.19:0/2218605013 >> 172.21.15.19:0/2218605013 conn(0x557764db28c0 msgr2=0x557764e00bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:35.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.186+0000 7f26e4c00640 0 client.7827 ms_handle_reset on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:35.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.186+0000 7f2705764740 1 -- 172.21.15.19:0/2218605013 shutdown_connections 2024-03-28T13:18:35.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.187+0000 7f2705764740 1 -- 172.21.15.19:0/2218605013 wait complete. 2024-03-28T13:18:35.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.187+0000 7f2705764740 20 client.7827 trim_cache size 0 max 16384 2024-03-28T13:18:35.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.217+0000 7f2705764740 1 Processor -- start 2024-03-28T13:18:35.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.217+0000 7f2705764740 1 -- start start 2024-03-28T13:18:35.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f2705764740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 0x557764e00070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f2705764740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de28a0 0x557764e21c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f2705764740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764de4ad0 0x557764e10ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f2705764740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1fff80 con 0x557764de4ad0 2024-03-28T13:18:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f2705764740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc0c9b20 con 0x557764db6420 2024-03-28T13:18:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f2705764740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f26fc1ff930 con 0x557764de28a0 2024-03-28T13:18:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f2700a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764de4ad0 0x557764e10ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f2700a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764de4ad0 0x557764e10ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49476/0 (socket says 172.21.15.19:49476) 2024-03-28T13:18:35.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f2700a00640 1 -- 172.21.15.19:0/2968130845 learned_addr learned my addr 172.21.15.19:0/2968130845 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:35.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f26fb400640 1 --2- 172.21.15.19:0/2968130845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de28a0 0x557764e21c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f26fbe00640 1 --2- 172.21.15.19:0/2968130845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 0x557764e00070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f2700a00640 1 -- 172.21.15.19:0/2968130845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de28a0 msgr2=0x557764e21c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f2700a00640 1 --2- 172.21.15.19:0/2968130845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de28a0 0x557764e21c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f2700a00640 1 -- 172.21.15.19:0/2968130845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 msgr2=0x557764e00070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f2700a00640 1 --2- 172.21.15.19:0/2968130845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 0x557764e00070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.219+0000 7f2700a00640 1 -- 172.21.15.19:0/2968130845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557764e00630 con 0x557764de4ad0 2024-03-28T13:18:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.220+0000 7f2700a00640 1 --2- 172.21.15.19:0/2968130845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764de4ad0 0x557764e10ad0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f26f4005040 tx=0x7f26f4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd248910fece4c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.220+0000 7f26e7400640 1 -- 172.21.15.19:0/2968130845 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4071100 con 0x557764de4ad0 2024-03-28T13:18:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.220+0000 7f26e7400640 1 -- 172.21.15.19:0/2968130845 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f405f380 con 0x557764de4ad0 2024-03-28T13:18:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.220+0000 7f26e7400640 1 -- 172.21.15.19:0/2968130845 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4054b20 con 0x557764de4ad0 2024-03-28T13:18:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.221+0000 7f2705764740 1 -- 172.21.15.19:0/2968130845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764de4ad0 msgr2=0x557764e10ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.221+0000 7f2705764740 1 --2- 172.21.15.19:0/2968130845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764de4ad0 0x557764e10ad0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f26f4005040 tx=0x7f26f4014000 comp rx=0 tx=0).stop 2024-03-28T13:18:35.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.221+0000 7f2705764740 1 -- 172.21.15.19:0/2968130845 shutdown_connections 2024-03-28T13:18:35.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.221+0000 7f2705764740 1 --2- 172.21.15.19:0/2968130845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764de4ad0 0x557764e10ad0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.221+0000 7f2705764740 1 --2- 172.21.15.19:0/2968130845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de28a0 0x557764e21c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.221+0000 7f2705764740 1 --2- 172.21.15.19:0/2968130845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 0x557764e00070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.221+0000 7f2705764740 1 -- 172.21.15.19:0/2968130845 >> 172.21.15.19:0/2968130845 conn(0x557764db28c0 msgr2=0x557764ddb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:35.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.221+0000 7f2705764740 1 -- 172.21.15.19:0/2968130845 shutdown_connections 2024-03-28T13:18:35.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.221+0000 7f2705764740 1 -- 172.21.15.19:0/2968130845 wait complete. 2024-03-28T13:18:35.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.222+0000 7f2705764740 1 -- start start 2024-03-28T13:18:35.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.222+0000 7f2705764740 1 Processor -- start 2024-03-28T13:18:35.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f2705764740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 0x557764dd73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f2705764740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764de28a0 0x557764dd78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f2705764740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de4ad0 0x557764ddb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f2705764740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557764e0e150 con 0x557764de28a0 2024-03-28T13:18:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f2705764740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557764de35c0 con 0x557764db6420 2024-03-28T13:18:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f2705764740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x557764de3fb0 con 0x557764de4ad0 2024-03-28T13:18:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f2705764740 10 client.0 _ll_register_callbacks cb 0x7f26f4055070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:18:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f26fb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764de28a0 0x557764dd78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f26fb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764de28a0 0x557764dd78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49486/0 (socket says 172.21.15.19:49486) 2024-03-28T13:18:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f26fb400640 1 -- 172.21.15.19:0/1926696576 learned_addr learned my addr 172.21.15.19:0/1926696576 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f2700a00640 1 --2- 172.21.15.19:0/1926696576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de4ad0 0x557764ddb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f26fb400640 1 -- 172.21.15.19:0/1926696576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de4ad0 msgr2=0x557764ddb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f26fb400640 1 --2- 172.21.15.19:0/1926696576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de4ad0 0x557764ddb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f26fbe00640 1 --2- 172.21.15.19:0/1926696576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 0x557764dd73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f26fb400640 1 -- 172.21.15.19:0/1926696576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 msgr2=0x557764dd73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.223+0000 7f26fb400640 1 --2- 172.21.15.19:0/1926696576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 0x557764dd73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.224+0000 7f26fb400640 1 -- 172.21.15.19:0/1926696576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557764e7aca0 con 0x557764de28a0 2024-03-28T13:18:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.224+0000 7f2700a00640 1 --2- 172.21.15.19:0/1926696576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de4ad0 0x557764ddb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.224+0000 7f26fb400640 1 --2- 172.21.15.19:0/1926696576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764de28a0 0x557764dd78f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f26e8000c00 tx=0x7f26e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b62ad38692a5993 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:35.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.224+0000 7f26e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:18:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.224+0000 7f26e4c00640 1 -- 172.21.15.19:0/1926696576 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e802c060 con 0x557764de28a0 2024-03-28T13:18:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.224+0000 7f26e4c00640 1 -- 172.21.15.19:0/1926696576 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f26e8004470 con 0x557764de28a0 2024-03-28T13:18:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.224+0000 7f2705764740 1 -- 172.21.15.19:0/1926696576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26f405f380 con 0x557764de28a0 2024-03-28T13:18:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.225+0000 7f26e4c00640 1 -- 172.21.15.19:0/1926696576 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f26e8032050 con 0x557764de28a0 2024-03-28T13:18:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.225+0000 7f2705764740 10 client.7833 fetch_fsmap learned FSMap version 74 2024-03-28T13:18:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.225+0000 7f2705764740 1 -- 172.21.15.19:0/1926696576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x557764dd6320 con 0x557764de28a0 2024-03-28T13:18:35.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.225+0000 7f26e4c00640 1 -- 172.21.15.19:0/1926696576 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f405f380 con 0x557764de28a0 2024-03-28T13:18:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.225+0000 7f26e4c00640 1 -- 172.21.15.19:0/1926696576 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f26e8035020 con 0x557764de28a0 2024-03-28T13:18:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.225+0000 7f2705764740 10 client.7833 fetch_fsmap finished waiting for FSMap version 74 2024-03-28T13:18:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.225+0000 7f2705764740 10 client.7833 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:18:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.225+0000 7f2705764740 20 client.7833 populate_metadata read hostname 'smithi019' 2024-03-28T13:18:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.225+0000 7f2705764740 1 --2- 172.21.15.19:0/1926696576 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x557764ddbde0 0x557764dcc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.225+0000 7f2705764740 4 client.7833 mds_command: new command op to 6040 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-28T13:18:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.225+0000 7f2705764740 1 -- 172.21.15.19:0/1926696576 --> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x557764db30c0 con 0x557764ddbde0 2024-03-28T13:18:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.225+0000 7f26fbe00640 1 --2- 172.21.15.19:0/1926696576 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x557764ddbde0 0x557764dcc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.226+0000 7f26fbe00640 1 --2- 172.21.15.19:0/1926696576 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x557764ddbde0 0x557764dcc190 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42f36d0deb2cb9ac server_cookie=868fd2217e1c37c1 in_seq=0 out_seq=0 2024-03-28T13:18:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.226+0000 7f26e4c00640 10 client.7833 ms_handle_connect on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.226+0000 7f26e4c00640 1 -- 172.21.15.19:0/1926696576 <== mds.0 v2:172.21.15.19:6832/2209799371 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557764db30c0 con 0x557764ddbde0 2024-03-28T13:18:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.226+0000 7f26e4c00640 10 client.7833 handle_command_reply: tid=0 2024-03-28T13:18:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.226+0000 7f2705764740 1 client.7833 shutdown 2024-03-28T13:18:35.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.226+0000 7f2705764740 1 -- 172.21.15.19:0/1926696576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764de28a0 msgr2=0x557764dd78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.226+0000 7f2705764740 1 --2- 172.21.15.19:0/1926696576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764de28a0 0x557764dd78f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f26e8000c00 tx=0x7f26e80046c0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.226+0000 7f2705764740 1 -- 172.21.15.19:0/1926696576 shutdown_connections 2024-03-28T13:18:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.226+0000 7f2705764740 1 --2- 172.21.15.19:0/1926696576 >> [v2:172.21.15.19:6832/2209799371,v1:172.21.15.19:6834/2209799371] conn(0x557764ddbde0 0x557764dcc190 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.227+0000 7f2705764740 1 --2- 172.21.15.19:0/1926696576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557764de4ad0 0x557764ddb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.227+0000 7f2705764740 1 --2- 172.21.15.19:0/1926696576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557764de28a0 0x557764dd78f0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.227+0000 7f26e4c00640 0 client.7833 ms_handle_reset on v2:172.21.15.19:6832/2209799371 2024-03-28T13:18:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.227+0000 7f2705764740 1 --2- 172.21.15.19:0/1926696576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557764db6420 0x557764dd73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.227+0000 7f2705764740 1 -- 172.21.15.19:0/1926696576 >> 172.21.15.19:0/1926696576 conn(0x557764db28c0 msgr2=0x557764de5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.227+0000 7f2705764740 1 -- 172.21.15.19:0/1926696576 shutdown_connections 2024-03-28T13:18:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.227+0000 7f2705764740 1 -- 172.21.15.19:0/1926696576 wait complete. 2024-03-28T13:18:35.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.227+0000 7f2705764740 20 client.7833 trim_cache size 0 max 16384 2024-03-28T13:18:35.228 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:35.229 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-03-28T13:18:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.227+0000 7f2701400640 1 -- 172.21.15.19:0/1985938339 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f26dc06c560 msgr2=0x7f26dc06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.227+0000 7f2701400640 1 --2- 172.21.15.19:0/1985938339 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f26dc06c560 0x7f26dc06ea20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f26ec00c240 tx=0x7f26ec009030 comp rx=0 tx=0).stop 2024-03-28T13:18:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.228+0000 7f2701400640 1 -- 172.21.15.19:0/1985938339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc1f6b40 msgr2=0x7f26fc1f6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.228+0000 7f2701400640 1 --2- 172.21.15.19:0/1985938339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc1f6b40 0x7f26fc1f6fd0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f26fc161e90 tx=0x7f26f400db50 comp rx=0 tx=0).stop 2024-03-28T13:18:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.228+0000 7f2701400640 1 -- 172.21.15.19:0/1985938339 shutdown_connections 2024-03-28T13:18:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.228+0000 7f2701400640 1 --2- 172.21.15.19:0/1985938339 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f26dc06c560 0x7f26dc06ea20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.228+0000 7f2701400640 1 --2- 172.21.15.19:0/1985938339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26fc1f6b40 0x7f26fc1f6fd0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.228+0000 7f2701400640 1 --2- 172.21.15.19:0/1985938339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26fc160300 0x7f26fc1535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.228+0000 7f2701400640 1 --2- 172.21.15.19:0/1985938339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26e00039e0 0x7f26fc0b9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.228+0000 7f2701400640 1 -- 172.21.15.19:0/1985938339 >> 172.21.15.19:0/1985938339 conn(0x7f26fc0676a0 msgr2=0x7f26fc15bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.228+0000 7f2701400640 1 -- 172.21.15.19:0/1985938339 shutdown_connections 2024-03-28T13:18:35.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.228+0000 7f2701400640 1 -- 172.21.15.19:0/1985938339 wait complete. 2024-03-28T13:18:35.243 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-28T13:18:35.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.454+0000 7fe88cc00640 1 Processor -- start 2024-03-28T13:18:35.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.455+0000 7fe88cc00640 1 -- start start 2024-03-28T13:18:35.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.456+0000 7fe88cc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe8880b4af0 0x7fe88815ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.456+0000 7fe88cc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe888141680 0x7fe88815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.456+0000 7fe88cc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe88815d7c0 con 0x7fe8881412a0 2024-03-28T13:18:35.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.456+0000 7fe88cc00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe888141be0 con 0x7fe8880b4af0 2024-03-28T13:18:35.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.456+0000 7fe88cc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe888141ec0 con 0x7fe888141680 2024-03-28T13:18:35.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.457+0000 7fe885e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe8881412a0 0x7fe8880b43e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52018/0 (socket says 172.21.15.19:52018) 2024-03-28T13:18:35.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.457+0000 7fe885e00640 1 -- 172.21.15.19:0/2007715283 learned_addr learned my addr 172.21.15.19:0/2007715283 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:35.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.457+0000 7fe885400640 1 --2- 172.21.15.19:0/2007715283 >> v2:172.21.15.19:3300/0 conn(0x7fe8880b4af0 0x7fe88815ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.457+0000 7fe886800640 1 --2- 172.21.15.19:0/2007715283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe888141680 0x7fe88815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.457+0000 7fe884a00640 1 -- 172.21.15.19:0/2007715283 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4149182809 0 0) 0x7fe88815d7c0 con 0x7fe8881412a0 2024-03-28T13:18:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.457+0000 7fe884a00640 1 -- 172.21.15.19:0/2007715283 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe86c003520 con 0x7fe8881412a0 2024-03-28T13:18:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.458+0000 7fe885400640 1 -- 172.21.15.19:0/2007715283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe888141680 msgr2=0x7fe88815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.458+0000 7fe885400640 1 --2- 172.21.15.19:0/2007715283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe888141680 0x7fe88815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.458+0000 7fe885400640 1 -- 172.21.15.19:0/2007715283 >> v1:172.21.15.19:6789/0 conn(0x7fe8881412a0 legacy=0x7fe8880b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.458+0000 7fe885400640 1 -- 172.21.15.19:0/2007715283 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8881421a0 con 0x7fe8880b4af0 2024-03-28T13:18:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.459+0000 7fe885400640 1 --2- 172.21.15.19:0/2007715283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8880b4af0 0x7fe88815ca50 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fe8740097a0 tx=0x7fe87402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace527633d684d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:35.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.459+0000 7fe884a00640 1 -- 172.21.15.19:0/2007715283 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874035020 con 0x7fe8880b4af0 2024-03-28T13:18:35.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.459+0000 7fe884a00640 1 -- 172.21.15.19:0/2007715283 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe874031830 con 0x7fe8880b4af0 2024-03-28T13:18:35.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.459+0000 7fe884a00640 1 -- 172.21.15.19:0/2007715283 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe874031c00 con 0x7fe8880b4af0 2024-03-28T13:18:35.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.460+0000 7fe88cc00640 1 -- 172.21.15.19:0/2007715283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8880b4af0 msgr2=0x7fe88815ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.460+0000 7fe88cc00640 1 --2- 172.21.15.19:0/2007715283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8880b4af0 0x7fe88815ca50 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fe8740097a0 tx=0x7fe87402b8d0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.460+0000 7fe88cc00640 1 -- 172.21.15.19:0/2007715283 shutdown_connections 2024-03-28T13:18:35.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.460+0000 7fe88cc00640 1 --2- 172.21.15.19:0/2007715283 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe888141680 0x7fe88815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.460+0000 7fe88cc00640 1 --2- 172.21.15.19:0/2007715283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8880b4af0 0x7fe88815ca50 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.460+0000 7fe88cc00640 1 -- 172.21.15.19:0/2007715283 >> 172.21.15.19:0/2007715283 conn(0x7fe888067670 msgr2=0x7fe888067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:35.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.460+0000 7fe88cc00640 1 -- 172.21.15.19:0/2007715283 shutdown_connections 2024-03-28T13:18:35.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.460+0000 7fe88cc00640 1 -- 172.21.15.19:0/2007715283 wait complete. 2024-03-28T13:18:35.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.462+0000 7fe88cc00640 1 Processor -- start 2024-03-28T13:18:35.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.462+0000 7fe88cc00640 1 -- start start 2024-03-28T13:18:35.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe88cc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8880b4af0 0x7fe88813ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe88cc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8881412a0 0x7fe8880cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe88cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888141680 0x7fe88815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe88cc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8881429a0 con 0x7fe888141680 2024-03-28T13:18:35.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe88cc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe88815d4d0 con 0x7fe8880b4af0 2024-03-28T13:18:35.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe88cc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe8880cec40 con 0x7fe8881412a0 2024-03-28T13:18:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe886800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888141680 0x7fe88815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe886800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888141680 0x7fe88815ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54058/0 (socket says 172.21.15.19:54058) 2024-03-28T13:18:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe886800640 1 -- 172.21.15.19:0/2142909653 learned_addr learned my addr 172.21.15.19:0/2142909653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe885400640 1 --2- 172.21.15.19:0/2142909653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8881412a0 0x7fe8880cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe885e00640 1 --2- 172.21.15.19:0/2142909653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8880b4af0 0x7fe88813ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe886800640 1 -- 172.21.15.19:0/2142909653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8881412a0 msgr2=0x7fe8880cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe886800640 1 --2- 172.21.15.19:0/2142909653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8881412a0 0x7fe8880cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe886800640 1 -- 172.21.15.19:0/2142909653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8880b4af0 msgr2=0x7fe88813ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe886800640 1 --2- 172.21.15.19:0/2142909653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8880b4af0 0x7fe88813ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe886800640 1 -- 172.21.15.19:0/2142909653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8880d19d0 con 0x7fe888141680 2024-03-28T13:18:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.463+0000 7fe885400640 1 --2- 172.21.15.19:0/2142909653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8881412a0 0x7fe8880cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:35.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.464+0000 7fe886800640 1 --2- 172.21.15.19:0/2142909653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888141680 0x7fe88815ebb0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe88000efc0 tx=0x7fe88000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.464+0000 7fe87e000640 1 -- 172.21.15.19:0/2142909653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880018020 con 0x7fe888141680 2024-03-28T13:18:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.464+0000 7fe87e000640 1 -- 172.21.15.19:0/2142909653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe880014480 con 0x7fe888141680 2024-03-28T13:18:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.464+0000 7fe88cc00640 1 -- 172.21.15.19:0/2142909653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8880d1cb0 con 0x7fe888141680 2024-03-28T13:18:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.464+0000 7fe87e000640 1 -- 172.21.15.19:0/2142909653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880014870 con 0x7fe888141680 2024-03-28T13:18:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.464+0000 7fe88cc00640 1 -- 172.21.15.19:0/2142909653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8880d2270 con 0x7fe888141680 2024-03-28T13:18:35.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.465+0000 7fe88cc00640 1 -- 172.21.15.19:0/2142909653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe888074f30 con 0x7fe888141680 2024-03-28T13:18:35.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.466+0000 7fe87e000640 1 -- 172.21.15.19:0/2142909653 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe880005d10 con 0x7fe888141680 2024-03-28T13:18:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.467+0000 7fe87e000640 1 --2- 172.21.15.19:0/2142909653 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe86806c530 0x7fe86806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.467+0000 7fe87e000640 1 -- 172.21.15.19:0/2142909653 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7fe88008c790 con 0x7fe888141680 2024-03-28T13:18:35.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.468+0000 7fe885e00640 1 --2- 172.21.15.19:0/2142909653 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe86806c530 0x7fe86806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:35.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.469+0000 7fe885e00640 1 --2- 172.21.15.19:0/2142909653 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe86806c530 0x7fe86806e9f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe878000f60 tx=0x7fe878009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:35.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.474+0000 7fe87e000640 1 -- 172.21.15.19:0/2142909653 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe880022030 con 0x7fe888141680 2024-03-28T13:18:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.818+0000 7fe88cc00640 1 -- 172.21.15.19:0/2142909653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fe88813f240 con 0x7fe888141680 2024-03-28T13:18:35.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.819+0000 7fe87e000640 1 -- 172.21.15.19:0/2142909653 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fe880010040 con 0x7fe888141680 2024-03-28T13:18:35.818 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:35.818 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-28T13:18:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.821+0000 7fe88cc00640 1 -- 172.21.15.19:0/2142909653 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe86806c530 msgr2=0x7fe86806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.821+0000 7fe88cc00640 1 --2- 172.21.15.19:0/2142909653 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe86806c530 0x7fe86806e9f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe878000f60 tx=0x7fe878009040 comp rx=0 tx=0).stop 2024-03-28T13:18:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.821+0000 7fe88cc00640 1 -- 172.21.15.19:0/2142909653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888141680 msgr2=0x7fe88815ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.821+0000 7fe88cc00640 1 --2- 172.21.15.19:0/2142909653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888141680 0x7fe88815ebb0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe88000efc0 tx=0x7fe88000c490 comp rx=0 tx=0).stop 2024-03-28T13:18:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.821+0000 7fe88cc00640 1 -- 172.21.15.19:0/2142909653 shutdown_connections 2024-03-28T13:18:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.821+0000 7fe88cc00640 1 --2- 172.21.15.19:0/2142909653 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe86806c530 0x7fe86806e9f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.821+0000 7fe88cc00640 1 --2- 172.21.15.19:0/2142909653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe888141680 0x7fe88815ebb0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.821+0000 7fe88cc00640 1 --2- 172.21.15.19:0/2142909653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8881412a0 0x7fe8880cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.821+0000 7fe88cc00640 1 --2- 172.21.15.19:0/2142909653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe8880b4af0 0x7fe88813ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:35.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.822+0000 7fe88cc00640 1 -- 172.21.15.19:0/2142909653 >> 172.21.15.19:0/2142909653 conn(0x7fe888067670 msgr2=0x7fe8880b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:35.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.822+0000 7fe88cc00640 1 -- 172.21.15.19:0/2142909653 shutdown_connections 2024-03-28T13:18:35.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:35.822+0000 7fe88cc00640 1 -- 172.21.15.19:0/2142909653 wait complete. 2024-03-28T13:18:35.838 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-28T13:18:35.839 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-28T13:18:35.839 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:18:35.944 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:18:36.070 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-28T13:18:36.070 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:18:36.070 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:18:36.070 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:18:36.127 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:18:36.127 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:18:36.177 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:18:36.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.442+0000 7f2eac200640 1 Processor -- start 2024-03-28T13:18:36.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.442+0000 7f2eac200640 1 -- start start 2024-03-28T13:18:36.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.443+0000 7f2eac200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ea40af830 0x7f2ea4073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:36.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.443+0000 7f2eac200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ea415fe80 con 0x7f2ea415c390 2024-03-28T13:18:36.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.443+0000 7f2eac200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ea40ad120 con 0x7f2ea40d0650 2024-03-28T13:18:36.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.443+0000 7f2eac200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2ea415c770 con 0x7f2ea40af830 2024-03-28T13:18:36.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.444+0000 7f2ea8e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ea40af830 0x7f2ea4073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:36.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.444+0000 7f2ea9800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2ea415c390 0x7f2ea4160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52020/0 (socket says 172.21.15.19:52020) 2024-03-28T13:18:36.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.444+0000 7f2ea8e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ea40af830 0x7f2ea4073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54062/0 (socket says 172.21.15.19:54062) 2024-03-28T13:18:36.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.444+0000 7f2ea9800640 1 -- 172.21.15.19:0/3214273111 learned_addr learned my addr 172.21.15.19:0/3214273111 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:36.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.444+0000 7f2ea8e00640 1 -- 172.21.15.19:0/3214273111 >> v1:172.21.15.188:6789/0 conn(0x7f2ea40d0650 legacy=0x7f2ea4071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:36.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.444+0000 7f2ea8e00640 1 -- 172.21.15.19:0/3214273111 >> v1:172.21.15.19:6789/0 conn(0x7f2ea415c390 legacy=0x7f2ea4160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:36.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.444+0000 7f2ea8e00640 1 -- 172.21.15.19:0/3214273111 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea40ad400 con 0x7f2ea40af830 2024-03-28T13:18:36.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.445+0000 7f2ea8e00640 1 --2- 172.21.15.19:0/3214273111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea40af830 0x7f2ea4073790 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f2e940097a0 tx=0x7f2e9402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5b0e1ebe8d0749 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:36.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.446+0000 7f2ea3e00640 1 -- 172.21.15.19:0/3214273111 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e94034070 con 0x7f2ea40af830 2024-03-28T13:18:36.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.446+0000 7f2ea3e00640 1 -- 172.21.15.19:0/3214273111 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e94030610 con 0x7f2ea40af830 2024-03-28T13:18:36.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.446+0000 7f2ea3e00640 1 -- 172.21.15.19:0/3214273111 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e940309e0 con 0x7f2ea40af830 2024-03-28T13:18:36.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.446+0000 7f2eac200640 1 -- 172.21.15.19:0/3214273111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea40af830 msgr2=0x7f2ea4073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:36.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.446+0000 7f2eac200640 1 --2- 172.21.15.19:0/3214273111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea40af830 0x7f2ea4073790 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f2e940097a0 tx=0x7f2e9402b960 comp rx=0 tx=0).stop 2024-03-28T13:18:36.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.446+0000 7f2eac200640 1 -- 172.21.15.19:0/3214273111 shutdown_connections 2024-03-28T13:18:36.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.446+0000 7f2eac200640 1 --2- 172.21.15.19:0/3214273111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea40af830 0x7f2ea4073790 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:36.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.446+0000 7f2eac200640 1 -- 172.21.15.19:0/3214273111 >> 172.21.15.19:0/3214273111 conn(0x7f2ea406b990 msgr2=0x7f2ea406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:36.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.447+0000 7f2eac200640 1 -- 172.21.15.19:0/3214273111 shutdown_connections 2024-03-28T13:18:36.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.447+0000 7f2eac200640 1 -- 172.21.15.19:0/3214273111 wait complete. 2024-03-28T13:18:36.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.448+0000 7f2eac200640 1 Processor -- start 2024-03-28T13:18:36.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.448+0000 7f2eac200640 1 -- start start 2024-03-28T13:18:36.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.449+0000 7f2eac200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea40af830 0x7f2ea4152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:36.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.449+0000 7f2eac200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea40d0650 0x7f2ea4152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:36.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.449+0000 7f2eac200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea415c390 0x7f2ea4142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:36.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.449+0000 7f2eac200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea4160d50 con 0x7f2ea415c390 2024-03-28T13:18:36.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.449+0000 7f2eac200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea4160120 con 0x7f2ea40af830 2024-03-28T13:18:36.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.449+0000 7f2eac200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea40cf980 con 0x7f2ea40d0650 2024-03-28T13:18:36.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.450+0000 7f2eaa200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea415c390 0x7f2ea4142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:36.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.450+0000 7f2eaa200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea415c390 0x7f2ea4142d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54078/0 (socket says 172.21.15.19:54078) 2024-03-28T13:18:36.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.450+0000 7f2eaa200640 1 -- 172.21.15.19:0/3406392371 learned_addr learned my addr 172.21.15.19:0/3406392371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:36.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.450+0000 7f2ea8e00640 1 --2- 172.21.15.19:0/3406392371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea40d0650 0x7f2ea4152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:36.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.450+0000 7f2ea9800640 1 --2- 172.21.15.19:0/3406392371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea40af830 0x7f2ea4152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:36.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.450+0000 7f2eaa200640 1 -- 172.21.15.19:0/3406392371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea40d0650 msgr2=0x7f2ea4152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:36.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.450+0000 7f2eaa200640 1 --2- 172.21.15.19:0/3406392371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea40d0650 0x7f2ea4152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:36.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.450+0000 7f2eaa200640 1 -- 172.21.15.19:0/3406392371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea40af830 msgr2=0x7f2ea4152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:36.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.450+0000 7f2eaa200640 1 --2- 172.21.15.19:0/3406392371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea40af830 0x7f2ea4152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:36.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.450+0000 7f2eaa200640 1 -- 172.21.15.19:0/3406392371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea41f5320 con 0x7f2ea415c390 2024-03-28T13:18:36.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.450+0000 7f2ea9800640 1 --2- 172.21.15.19:0/3406392371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea40af830 0x7f2ea4152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:36.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.451+0000 7f2eaa200640 1 --2- 172.21.15.19:0/3406392371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea415c390 0x7f2ea4142d80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00c7a0 tx=0x7f2e9c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:36.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.451+0000 7f2ea1600640 1 -- 172.21.15.19:0/3406392371 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c003b70 con 0x7f2ea415c390 2024-03-28T13:18:36.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.451+0000 7f2ea1600640 1 -- 172.21.15.19:0/3406392371 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e9c003d30 con 0x7f2ea415c390 2024-03-28T13:18:36.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.451+0000 7f2eac200640 1 -- 172.21.15.19:0/3406392371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea41f5600 con 0x7f2ea415c390 2024-03-28T13:18:36.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.451+0000 7f2ea1600640 1 -- 172.21.15.19:0/3406392371 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c00aa90 con 0x7f2ea415c390 2024-03-28T13:18:36.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.451+0000 7f2eac200640 1 -- 172.21.15.19:0/3406392371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea41f5b40 con 0x7f2ea415c390 2024-03-28T13:18:36.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.452+0000 7f2eac200640 1 -- 172.21.15.19:0/3406392371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea40685c0 con 0x7f2ea415c390 2024-03-28T13:18:36.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.454+0000 7f2ea1600640 1 -- 172.21.15.19:0/3406392371 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2e9c00ac30 con 0x7f2ea415c390 2024-03-28T13:18:36.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.456+0000 7f2ea1600640 1 --2- 172.21.15.19:0/3406392371 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e8806c580 0x7f2e8806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:36.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.456+0000 7f2ea1600640 1 -- 172.21.15.19:0/3406392371 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f2e9c08b610 con 0x7f2ea415c390 2024-03-28T13:18:36.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.456+0000 7f2ea9800640 1 --2- 172.21.15.19:0/3406392371 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e8806c580 0x7f2e8806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:36.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.457+0000 7f2ea9800640 1 --2- 172.21.15.19:0/3406392371 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e8806c580 0x7f2e8806ea40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2e98000f60 tx=0x7f2e98009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:36.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.460+0000 7f2ea1600640 1 -- 172.21.15.19:0/3406392371 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2e9c059bf0 con 0x7f2ea415c390 2024-03-28T13:18:36.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.664+0000 7f2eac200640 1 -- 172.21.15.19:0/3406392371 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ea406f1b0 con 0x7f2ea415c390 2024-03-28T13:18:36.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.666+0000 7f2ea1600640 1 -- 172.21.15.19:0/3406392371 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16935 (secure 0 0 0) 0x7f2e9c0594f0 con 0x7f2ea415c390 2024-03-28T13:18:36.665 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:36.666 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":94,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:17:55.195667+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:16:23.901120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-28T13:16:24.990904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6836/1749969053":"2024-03-29T13:17:55.195632+0000","172.21.15.188:6834/1749969053":"2024-03-29T13:17:55.195632+0000","172.21.15.188:6836/2260087679":"2024-03-29T13:16:39.492982+0000","172.21.15.188:6834/2260087679":"2024-03-29T13:16:39.492982+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:18:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.668+0000 7f2eac200640 1 -- 172.21.15.19:0/3406392371 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e8806c580 msgr2=0x7f2e8806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.668+0000 7f2eac200640 1 --2- 172.21.15.19:0/3406392371 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e8806c580 0x7f2e8806ea40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2e98000f60 tx=0x7f2e98009040 comp rx=0 tx=0).stop 2024-03-28T13:18:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.669+0000 7f2eac200640 1 -- 172.21.15.19:0/3406392371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea415c390 msgr2=0x7f2ea4142d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.669+0000 7f2eac200640 1 --2- 172.21.15.19:0/3406392371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea415c390 0x7f2ea4142d80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00c7a0 tx=0x7f2e9c00cc60 comp rx=0 tx=0).stop 2024-03-28T13:18:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.669+0000 7f2eac200640 1 -- 172.21.15.19:0/3406392371 shutdown_connections 2024-03-28T13:18:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.669+0000 7f2eac200640 1 --2- 172.21.15.19:0/3406392371 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e8806c580 0x7f2e8806ea40 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.669+0000 7f2eac200640 1 --2- 172.21.15.19:0/3406392371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ea415c390 0x7f2ea4142d80 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.669+0000 7f2eac200640 1 --2- 172.21.15.19:0/3406392371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ea40d0650 0x7f2ea4152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.669+0000 7f2eac200640 1 --2- 172.21.15.19:0/3406392371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ea40af830 0x7f2ea4152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.669+0000 7f2eac200640 1 -- 172.21.15.19:0/3406392371 >> 172.21.15.19:0/3406392371 conn(0x7f2ea406b990 msgr2=0x7f2ea415b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.669+0000 7f2eac200640 1 -- 172.21.15.19:0/3406392371 shutdown_connections 2024-03-28T13:18:36.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.669+0000 7f2eac200640 1 -- 172.21.15.19:0/3406392371 wait complete. 2024-03-28T13:18:36.690 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:16:23.901120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:16:24.990904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:18:36.691 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:18:36.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.928+0000 7fda00800640 1 Processor -- start 2024-03-28T13:18:36.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.928+0000 7fda00800640 1 -- start start 2024-03-28T13:18:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.930+0000 7fda00800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f815a870 0x7fd9f815e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.930+0000 7fda00800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd9f815ac50 0x7fd9f815ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.930+0000 7fda00800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9f80b4c00 con 0x7fd9f80d0280 2024-03-28T13:18:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.930+0000 7fda00800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd9f806d5b0 con 0x7fd9f815ac50 2024-03-28T13:18:36.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.930+0000 7fda00800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f806db70 con 0x7fd9f815a870 2024-03-28T13:18:36.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.931+0000 7fd9fd400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd9f815ac50 0x7fd9f815ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:36.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.931+0000 7fd9fd400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd9f815ac50 0x7fd9f815ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54080/0 (socket says 172.21.15.19:54080) 2024-03-28T13:18:36.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.931+0000 7fd9fd400640 1 -- 172.21.15.19:0/2231050846 learned_addr learned my addr 172.21.15.19:0/2231050846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:36.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.931+0000 7fd9fd400640 1 -- 172.21.15.19:0/2231050846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f815a870 msgr2=0x7fd9f815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:36.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.931+0000 7fd9fde00640 1 --2- 172.21.15.19:0/2231050846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f815a870 0x7fd9f815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:36.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.931+0000 7fd9fd400640 1 --2- 172.21.15.19:0/2231050846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f815a870 0x7fd9f815e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:36.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.931+0000 7fd9fd400640 1 -- 172.21.15.19:0/2231050846 >> v1:172.21.15.188:6789/0 conn(0x7fd9f80d0280 legacy=0x7fd9f80b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:36.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.931+0000 7fd9fd400640 1 -- 172.21.15.19:0/2231050846 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f80b4ee0 con 0x7fd9f815ac50 2024-03-28T13:18:36.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.932+0000 7fd9fd400640 1 --2- 172.21.15.19:0/2231050846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f815ac50 0x7fd9f815ead0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8009870 tx=0x7fd9e802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b1fc77a7a7ea12 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:36.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.933+0000 7fd9fca00640 1 -- 172.21.15.19:0/2231050846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8035020 con 0x7fd9f815ac50 2024-03-28T13:18:36.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.933+0000 7fd9fca00640 1 -- 172.21.15.19:0/2231050846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9e8031640 con 0x7fd9f815ac50 2024-03-28T13:18:36.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.933+0000 7fd9fca00640 1 -- 172.21.15.19:0/2231050846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e802d040 con 0x7fd9f815ac50 2024-03-28T13:18:36.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.933+0000 7fda00800640 1 -- 172.21.15.19:0/2231050846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f815ac50 msgr2=0x7fd9f815ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:36.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.933+0000 7fda00800640 1 --2- 172.21.15.19:0/2231050846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f815ac50 0x7fd9f815ead0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fd9e8009870 tx=0x7fd9e802b8f0 comp rx=0 tx=0).stop 2024-03-28T13:18:36.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.934+0000 7fda00800640 1 -- 172.21.15.19:0/2231050846 shutdown_connections 2024-03-28T13:18:36.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.934+0000 7fda00800640 1 --2- 172.21.15.19:0/2231050846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f815ac50 0x7fd9f815ead0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:36.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.934+0000 7fda00800640 1 --2- 172.21.15.19:0/2231050846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f815a870 0x7fd9f815e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:36.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.934+0000 7fda00800640 1 -- 172.21.15.19:0/2231050846 >> 172.21.15.19:0/2231050846 conn(0x7fd9f806b750 msgr2=0x7fd9f806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:36.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.934+0000 7fda00800640 1 -- 172.21.15.19:0/2231050846 shutdown_connections 2024-03-28T13:18:36.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.934+0000 7fda00800640 1 -- 172.21.15.19:0/2231050846 wait complete. 2024-03-28T13:18:36.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.935+0000 7fda00800640 1 Processor -- start 2024-03-28T13:18:36.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.936+0000 7fda00800640 1 -- start start 2024-03-28T13:18:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.936+0000 7fda00800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f80d0280 0x7fd9f8142b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.937+0000 7fda00800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f815a870 0x7fd9f8141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.937+0000 7fda00800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f815ac50 0x7fd9f8095df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.937+0000 7fda00800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f80d08b0 con 0x7fd9f815ac50 2024-03-28T13:18:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.937+0000 7fda00800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f80d0730 con 0x7fd9f80d0280 2024-03-28T13:18:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.937+0000 7fda00800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd9f80b4620 con 0x7fd9f815a870 2024-03-28T13:18:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.937+0000 7fd9fe800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f815ac50 0x7fd9f8095df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.937+0000 7fd9fe800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f815ac50 0x7fd9f8095df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54088/0 (socket says 172.21.15.19:54088) 2024-03-28T13:18:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.937+0000 7fd9fe800640 1 -- 172.21.15.19:0/3751346731 learned_addr learned my addr 172.21.15.19:0/3751346731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.937+0000 7fd9fde00640 1 --2- 172.21.15.19:0/3751346731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f80d0280 0x7fd9f8142b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.937+0000 7fd9fd400640 1 --2- 172.21.15.19:0/3751346731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f815a870 0x7fd9f8141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.937+0000 7fd9fe800640 1 -- 172.21.15.19:0/3751346731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f815a870 msgr2=0x7fd9f8141060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.937+0000 7fd9fe800640 1 --2- 172.21.15.19:0/3751346731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f815a870 0x7fd9f8141060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.937+0000 7fd9fe800640 1 -- 172.21.15.19:0/3751346731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f80d0280 msgr2=0x7fd9f8142b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.937+0000 7fd9fe800640 1 --2- 172.21.15.19:0/3751346731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f80d0280 0x7fd9f8142b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.937+0000 7fd9fe800640 1 -- 172.21.15.19:0/3751346731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f8165640 con 0x7fd9f815ac50 2024-03-28T13:18:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.938+0000 7fd9fe800640 1 --2- 172.21.15.19:0/3751346731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f815ac50 0x7fd9f8095df0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c440 tx=0x7fd9f400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.938+0000 7fd9ee000640 1 -- 172.21.15.19:0/3751346731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4011790 con 0x7fd9f815ac50 2024-03-28T13:18:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.938+0000 7fd9ee000640 1 -- 172.21.15.19:0/3751346731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f4017070 con 0x7fd9f815ac50 2024-03-28T13:18:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.938+0000 7fd9ee000640 1 -- 172.21.15.19:0/3751346731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4011b70 con 0x7fd9f815ac50 2024-03-28T13:18:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.938+0000 7fda00800640 1 -- 172.21.15.19:0/3751346731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f8165920 con 0x7fd9f815ac50 2024-03-28T13:18:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.938+0000 7fda00800640 1 -- 172.21.15.19:0/3751346731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f80ad490 con 0x7fd9f815ac50 2024-03-28T13:18:36.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.940+0000 7fda00800640 1 -- 172.21.15.19:0/3751346731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9f80685c0 con 0x7fd9f815ac50 2024-03-28T13:18:36.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.940+0000 7fd9ee000640 1 -- 172.21.15.19:0/3751346731 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd9f4011d10 con 0x7fd9f815ac50 2024-03-28T13:18:36.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.942+0000 7fd9ee000640 1 --2- 172.21.15.19:0/3751346731 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd9dc06c530 0x7fd9dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:36.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.942+0000 7fd9ee000640 1 -- 172.21.15.19:0/3751346731 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7fd9f408b5e0 con 0x7fd9f815ac50 2024-03-28T13:18:36.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.942+0000 7fd9fde00640 1 --2- 172.21.15.19:0/3751346731 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd9dc06c530 0x7fd9dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:36.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.944+0000 7fd9fde00640 1 --2- 172.21.15.19:0/3751346731 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd9dc06c530 0x7fd9dc06e9f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd9f815e330 tx=0x7fd9f00079a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:36.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:36.947+0000 7fd9ee000640 1 -- 172.21.15.19:0/3751346731 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd9f4059b40 con 0x7fd9f815ac50 2024-03-28T13:18:37.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.160+0000 7fda00800640 1 -- 172.21.15.19:0/3751346731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd9f80afda0 con 0x7fd9f815ac50 2024-03-28T13:18:37.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.160+0000 7fd9ee000640 1 -- 172.21.15.19:0/3751346731 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fd9f4059440 con 0x7fd9f815ac50 2024-03-28T13:18:37.159 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:18:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.164+0000 7fda00800640 1 -- 172.21.15.19:0/3751346731 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd9dc06c530 msgr2=0x7fd9dc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.164+0000 7fda00800640 1 --2- 172.21.15.19:0/3751346731 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd9dc06c530 0x7fd9dc06e9f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd9f815e330 tx=0x7fd9f00079a0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.164+0000 7fda00800640 1 -- 172.21.15.19:0/3751346731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f815ac50 msgr2=0x7fd9f8095df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.164+0000 7fda00800640 1 --2- 172.21.15.19:0/3751346731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f815ac50 0x7fd9f8095df0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c440 tx=0x7fd9f400c900 comp rx=0 tx=0).stop 2024-03-28T13:18:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.164+0000 7fda00800640 1 -- 172.21.15.19:0/3751346731 shutdown_connections 2024-03-28T13:18:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.164+0000 7fda00800640 1 --2- 172.21.15.19:0/3751346731 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd9dc06c530 0x7fd9dc06e9f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.164+0000 7fda00800640 1 --2- 172.21.15.19:0/3751346731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9f815ac50 0x7fd9f8095df0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.164+0000 7fda00800640 1 --2- 172.21.15.19:0/3751346731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9f815a870 0x7fd9f8141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.164+0000 7fda00800640 1 --2- 172.21.15.19:0/3751346731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9f80d0280 0x7fd9f8142b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.164+0000 7fda00800640 1 -- 172.21.15.19:0/3751346731 >> 172.21.15.19:0/3751346731 conn(0x7fd9f806b750 msgr2=0x7fd9f8071610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:37.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.164+0000 7fda00800640 1 -- 172.21.15.19:0/3751346731 shutdown_connections 2024-03-28T13:18:37.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.164+0000 7fda00800640 1 -- 172.21.15.19:0/3751346731 wait complete. 2024-03-28T13:18:37.181 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:18:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.403+0000 7f0f96c00640 1 Processor -- start 2024-03-28T13:18:37.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.404+0000 7f0f96c00640 1 -- start start 2024-03-28T13:18:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.406+0000 7f0f96c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0f90160a30 0x7f0f9006b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.406+0000 7f0f96c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f900b5ed0 con 0x7f0f90145d60 2024-03-28T13:18:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.406+0000 7f0f96c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f900b61b0 con 0x7f0f901457e0 2024-03-28T13:18:37.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.406+0000 7f0f96c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f0f90145bc0 con 0x7f0f90160a30 2024-03-28T13:18:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.407+0000 7f0f94c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0f90145d60 0x7f0f900b0130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52036/0 (socket says 172.21.15.19:52036) 2024-03-28T13:18:37.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.407+0000 7f0f94c00640 1 -- 172.21.15.19:0/2293468477 learned_addr learned my addr 172.21.15.19:0/2293468477 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.407+0000 7f0f87e00640 1 --2- 172.21.15.19:0/2293468477 >> v2:172.21.15.188:3300/0 conn(0x7f0f90160a30 0x7f0f9006b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.407+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1239891530 0 0) 0x7f0f900b5ed0 con 0x7f0f90145d60 2024-03-28T13:18:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.407+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f74003520 con 0x7f0f90145d60 2024-03-28T13:18:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.407+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 519623847 0 0) 0x7f0f900b61b0 con 0x7f0f901457e0 2024-03-28T13:18:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.407+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f900b5ed0 con 0x7f0f901457e0 2024-03-28T13:18:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.407+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3866214659 0 0) 0x7f0f74003520 con 0x7f0f90145d60 2024-03-28T13:18:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.407+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 >> v2:172.21.15.188:3300/0 conn(0x7f0f90160a30 msgr2=0x7f0f9006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.407+0000 7f0f8f400640 1 --2- 172.21.15.19:0/2293468477 >> v2:172.21.15.188:3300/0 conn(0x7f0f90160a30 0x7f0f9006b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.407+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 >> v1:172.21.15.188:6789/0 conn(0x7f0f901457e0 legacy=0x7f0f900b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f900b4750 con 0x7f0f90145d60 2024-03-28T13:18:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f0f880019f0 con 0x7f0f90145d60 2024-03-28T13:18:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 >> v1:172.21.15.19:6789/0 conn(0x7f0f90145d60 legacy=0x7f0f900b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f8f400640 1 --2- 172.21.15.19:0/2293468477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f74003b90 0x7f0f74003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f8f400640 1 --2- 172.21.15.19:0/2293468477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f74004800 0x7f0f74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:37.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f8f400640 1 --2- 172.21.15.19:0/2293468477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f74005210 0x7f0f7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:37.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f90145bc0 con 0x7f0f74003b90 2024-03-28T13:18:37.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f7400c140 con 0x7f0f74004800 2024-03-28T13:18:37.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0f7400c340 con 0x7f0f74005210 2024-03-28T13:18:37.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f8fe00640 1 --2- 172.21.15.19:0/2293468477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f74003b90 0x7f0f74003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:37.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f8fe00640 1 -- 172.21.15.19:0/2293468477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f74005210 msgr2=0x7f0f7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:37.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f8fe00640 1 --2- 172.21.15.19:0/2293468477 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f74005210 0x7f0f7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f8fe00640 1 -- 172.21.15.19:0/2293468477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f74004800 msgr2=0x7f0f74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f8fe00640 1 --2- 172.21.15.19:0/2293468477 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f74004800 0x7f0f74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f8fe00640 1 -- 172.21.15.19:0/2293468477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f74003520 con 0x7f0f74003b90 2024-03-28T13:18:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.408+0000 7f0f94c00640 1 -- 172.21.15.19:0/2293468477 reap_dead start 2024-03-28T13:18:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.409+0000 7f0f8fe00640 1 --2- 172.21.15.19:0/2293468477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f74003b90 0x7f0f74003f90 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f0f800018d0 tx=0x7f0f74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6decf0cc09a7c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.410+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80037070 con 0x7f0f74003b90 2024-03-28T13:18:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.410+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f800050d0 con 0x7f0f74003b90 2024-03-28T13:18:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.410+0000 7f0f8f400640 1 -- 172.21.15.19:0/2293468477 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f800014f0 con 0x7f0f74003b90 2024-03-28T13:18:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.410+0000 7f0f96c00640 1 -- 172.21.15.19:0/2293468477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f74003b90 msgr2=0x7f0f74003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.410+0000 7f0f96c00640 1 --2- 172.21.15.19:0/2293468477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f74003b90 0x7f0f74003f90 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f0f800018d0 tx=0x7f0f74005830 comp rx=0 tx=0).stop 2024-03-28T13:18:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.410+0000 7f0f96c00640 1 -- 172.21.15.19:0/2293468477 shutdown_connections 2024-03-28T13:18:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.410+0000 7f0f96c00640 1 --2- 172.21.15.19:0/2293468477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f74003b90 0x7f0f74003f90 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.410+0000 7f0f96c00640 1 -- 172.21.15.19:0/2293468477 >> 172.21.15.19:0/2293468477 conn(0x7f0f900b65a0 msgr2=0x7f0f900b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.410+0000 7f0f96c00640 1 -- 172.21.15.19:0/2293468477 shutdown_connections 2024-03-28T13:18:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.411+0000 7f0f96c00640 1 -- 172.21.15.19:0/2293468477 wait complete. 2024-03-28T13:18:37.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.412+0000 7f0f96c00640 1 Processor -- start 2024-03-28T13:18:37.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.412+0000 7f0f96c00640 1 -- start start 2024-03-28T13:18:37.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.413+0000 7f0f96c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f74003b90 0x7f0f901e86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:37.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.413+0000 7f0f96c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f901e5cb0 0x7f0f901e7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:37.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.413+0000 7f0f96c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f9015bf00 0x7f0f9015e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:37.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.413+0000 7f0f96c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f900ae1e0 con 0x7f0f74003b90 2024-03-28T13:18:37.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.413+0000 7f0f96c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f900afca0 con 0x7f0f9015bf00 2024-03-28T13:18:37.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.413+0000 7f0f96c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0f900ae360 con 0x7f0f901e5cb0 2024-03-28T13:18:37.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.414+0000 7f0f8fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f74003b90 0x7f0f901e86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:37.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.414+0000 7f0f8fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f74003b90 0x7f0f901e86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54116/0 (socket says 172.21.15.19:54116) 2024-03-28T13:18:37.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.414+0000 7f0f8fe00640 1 -- 172.21.15.19:0/372926256 learned_addr learned my addr 172.21.15.19:0/372926256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:37.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.414+0000 7f0f87e00640 1 --2- 172.21.15.19:0/372926256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f901e5cb0 0x7f0f901e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:37.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.414+0000 7f0f8fe00640 1 -- 172.21.15.19:0/372926256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f901e5cb0 msgr2=0x7f0f901e7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:37.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.414+0000 7f0f8fe00640 1 --2- 172.21.15.19:0/372926256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f901e5cb0 0x7f0f901e7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.414+0000 7f0f8fe00640 1 -- 172.21.15.19:0/372926256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f9015bf00 msgr2=0x7f0f9015e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.414+0000 7f0f8fe00640 1 --2- 172.21.15.19:0/372926256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f9015bf00 0x7f0f9015e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.414+0000 7f0f8fe00640 1 -- 172.21.15.19:0/372926256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f901e6190 con 0x7f0f74003b90 2024-03-28T13:18:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.415+0000 7f0f8fe00640 1 --2- 172.21.15.19:0/372926256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f74003b90 0x7f0f901e86f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f0f80035040 tx=0x7f0f800012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.415+0000 7f0f8cc00640 1 -- 172.21.15.19:0/372926256 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80042020 con 0x7f0f74003b90 2024-03-28T13:18:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.415+0000 7f0f8cc00640 1 -- 172.21.15.19:0/372926256 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f80038040 con 0x7f0f74003b90 2024-03-28T13:18:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.415+0000 7f0f8cc00640 1 -- 172.21.15.19:0/372926256 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80037070 con 0x7f0f74003b90 2024-03-28T13:18:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.415+0000 7f0f96c00640 1 -- 172.21.15.19:0/372926256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f901e6470 con 0x7f0f74003b90 2024-03-28T13:18:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.415+0000 7f0f96c00640 1 -- 172.21.15.19:0/372926256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f901e6a30 con 0x7f0f74003b90 2024-03-28T13:18:37.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.416+0000 7f0f96c00640 1 -- 172.21.15.19:0/372926256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f90068cd0 con 0x7f0f74003b90 2024-03-28T13:18:37.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.417+0000 7f0f8cc00640 1 -- 172.21.15.19:0/372926256 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0f80045020 con 0x7f0f74003b90 2024-03-28T13:18:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.418+0000 7f0f8cc00640 1 --2- 172.21.15.19:0/372926256 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f6c06c560 0x7f0f6c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.418+0000 7f0f8cc00640 1 -- 172.21.15.19:0/372926256 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f0f8002f020 con 0x7f0f74003b90 2024-03-28T13:18:37.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.419+0000 7f0f87e00640 1 --2- 172.21.15.19:0/372926256 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f6c06c560 0x7f0f6c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:37.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.420+0000 7f0f87e00640 1 --2- 172.21.15.19:0/372926256 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f6c06c560 0x7f0f6c06ea20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0f900d0a30 tx=0x7f0f78008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:37.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.427+0000 7f0f8cc00640 1 -- 172.21.15.19:0/372926256 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0f8007b900 con 0x7f0f74003b90 2024-03-28T13:18:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.630+0000 7f0f96c00640 1 -- 172.21.15.19:0/372926256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0f900d0f40 con 0x7f0f74003b90 2024-03-28T13:18:37.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.631+0000 7f0f8cc00640 1 -- 172.21.15.19:0/372926256 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f0f8007b200 con 0x7f0f74003b90 2024-03-28T13:18:37.630 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:18:37.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.633+0000 7f0f96c00640 1 -- 172.21.15.19:0/372926256 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f6c06c560 msgr2=0x7f0f6c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:37.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.633+0000 7f0f96c00640 1 --2- 172.21.15.19:0/372926256 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f6c06c560 0x7f0f6c06ea20 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0f900d0a30 tx=0x7f0f78008040 comp rx=0 tx=0).stop 2024-03-28T13:18:37.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.633+0000 7f0f96c00640 1 -- 172.21.15.19:0/372926256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f74003b90 msgr2=0x7f0f901e86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:37.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.633+0000 7f0f96c00640 1 --2- 172.21.15.19:0/372926256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f74003b90 0x7f0f901e86f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f0f80035040 tx=0x7f0f800012d0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.633+0000 7f0f96c00640 1 -- 172.21.15.19:0/372926256 shutdown_connections 2024-03-28T13:18:37.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.633+0000 7f0f96c00640 1 --2- 172.21.15.19:0/372926256 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f6c06c560 0x7f0f6c06ea20 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.633+0000 7f0f96c00640 1 --2- 172.21.15.19:0/372926256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f9015bf00 0x7f0f9015e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.633+0000 7f0f96c00640 1 --2- 172.21.15.19:0/372926256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f901e5cb0 0x7f0f901e7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.633+0000 7f0f96c00640 1 --2- 172.21.15.19:0/372926256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f74003b90 0x7f0f901e86f0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.634+0000 7f0f96c00640 1 -- 172.21.15.19:0/372926256 >> 172.21.15.19:0/372926256 conn(0x7f0f900b65a0 msgr2=0x7f0f9015ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:37.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.634+0000 7f0f96c00640 1 -- 172.21.15.19:0/372926256 shutdown_connections 2024-03-28T13:18:37.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.634+0000 7f0f96c00640 1 -- 172.21.15.19:0/372926256 wait complete. 2024-03-28T13:18:37.651 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:18:37.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.870+0000 7fb90b200640 1 Processor -- start 2024-03-28T13:18:37.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.870+0000 7fb90b200640 1 -- start start 2024-03-28T13:18:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.873+0000 7fb90b200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb90c0cd890 0x7fb90c15d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.873+0000 7fb90b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90c1529d0 0x7fb90c152e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.873+0000 7fb90b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb90c0b3ee0 con 0x7fb90c0afaa0 2024-03-28T13:18:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.873+0000 7fb90b200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb90c0b41c0 con 0x7fb90c0cd890 2024-03-28T13:18:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.873+0000 7fb90b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb90c0b44a0 con 0x7fb90c1529d0 2024-03-28T13:18:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.873+0000 7fb909e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb90c0afaa0 0x7fb90c0cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52048/0 (socket says 172.21.15.19:52048) 2024-03-28T13:18:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.873+0000 7fb909e00640 1 -- 172.21.15.19:0/2251111331 learned_addr learned my addr 172.21.15.19:0/2251111331 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.873+0000 7fb909400640 1 --2- 172.21.15.19:0/2251111331 >> v2:172.21.15.188:3300/0 conn(0x7fb90c0cd890 0x7fb90c15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:37.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.873+0000 7fb90a800640 1 --2- 172.21.15.19:0/2251111331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90c1529d0 0x7fb90c152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.874+0000 7fb909400640 1 -- 172.21.15.19:0/2251111331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90c1529d0 msgr2=0x7fb90c152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.874+0000 7fb909400640 1 --2- 172.21.15.19:0/2251111331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90c1529d0 0x7fb90c152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.874+0000 7fb909400640 1 -- 172.21.15.19:0/2251111331 >> v1:172.21.15.19:6789/0 conn(0x7fb90c0afaa0 legacy=0x7fb90c0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:37.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.874+0000 7fb909400640 1 -- 172.21.15.19:0/2251111331 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90c0b4780 con 0x7fb90c0cd890 2024-03-28T13:18:37.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.876+0000 7fb909400640 1 --2- 172.21.15.19:0/2251111331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90c0cd890 0x7fb90c15d230 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb8f80097a0 tx=0x7fb8f802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5459043def1fb52 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:37.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.879+0000 7fb908a00640 1 -- 172.21.15.19:0/2251111331 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f80315a0 con 0x7fb90c0cd890 2024-03-28T13:18:37.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.879+0000 7fb908a00640 1 -- 172.21.15.19:0/2251111331 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f8037070 con 0x7fb90c0cd890 2024-03-28T13:18:37.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.879+0000 7fb908a00640 1 -- 172.21.15.19:0/2251111331 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8031960 con 0x7fb90c0cd890 2024-03-28T13:18:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.880+0000 7fb90b200640 1 -- 172.21.15.19:0/2251111331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90c0cd890 msgr2=0x7fb90c15d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.880+0000 7fb90b200640 1 --2- 172.21.15.19:0/2251111331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90c0cd890 0x7fb90c15d230 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb8f80097a0 tx=0x7fb8f802b840 comp rx=0 tx=0).stop 2024-03-28T13:18:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.880+0000 7fb90b200640 1 -- 172.21.15.19:0/2251111331 shutdown_connections 2024-03-28T13:18:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.880+0000 7fb90b200640 1 --2- 172.21.15.19:0/2251111331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90c1529d0 0x7fb90c152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.880+0000 7fb90b200640 1 --2- 172.21.15.19:0/2251111331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90c0cd890 0x7fb90c15d230 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.880+0000 7fb90b200640 1 -- 172.21.15.19:0/2251111331 >> 172.21.15.19:0/2251111331 conn(0x7fb90c0d0770 msgr2=0x7fb90c0d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:37.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.880+0000 7fb90b200640 1 -- 172.21.15.19:0/2251111331 shutdown_connections 2024-03-28T13:18:37.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.880+0000 7fb90b200640 1 -- 172.21.15.19:0/2251111331 wait complete. 2024-03-28T13:18:37.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.882+0000 7fb90b200640 1 Processor -- start 2024-03-28T13:18:37.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.882+0000 7fb90b200640 1 -- start start 2024-03-28T13:18:37.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.883+0000 7fb90b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90c0afaa0 0x7fb90c155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:37.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.883+0000 7fb90b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90c0cd890 0x7fb90c147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:37.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.883+0000 7fb90b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c1529d0 0x7fb90c154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:37.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.883+0000 7fb90b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c0afe80 con 0x7fb90c1529d0 2024-03-28T13:18:37.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.883+0000 7fb90b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c0b4a60 con 0x7fb90c0cd890 2024-03-28T13:18:37.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.883+0000 7fb90b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb90c0b0000 con 0x7fb90c0afaa0 2024-03-28T13:18:37.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.884+0000 7fb90a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c1529d0 0x7fb90c154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:37.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.884+0000 7fb90a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c1529d0 0x7fb90c154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54130/0 (socket says 172.21.15.19:54130) 2024-03-28T13:18:37.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.884+0000 7fb90a800640 1 -- 172.21.15.19:0/2149828429 learned_addr learned my addr 172.21.15.19:0/2149828429 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:37.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.884+0000 7fb909e00640 1 --2- 172.21.15.19:0/2149828429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90c0afaa0 0x7fb90c155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:37.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.884+0000 7fb909400640 1 --2- 172.21.15.19:0/2149828429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90c0cd890 0x7fb90c147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:37.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.884+0000 7fb90a800640 1 -- 172.21.15.19:0/2149828429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90c0afaa0 msgr2=0x7fb90c155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:37.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.884+0000 7fb90a800640 1 --2- 172.21.15.19:0/2149828429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90c0afaa0 0x7fb90c155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.884+0000 7fb90a800640 1 -- 172.21.15.19:0/2149828429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90c0cd890 msgr2=0x7fb90c147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:37.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.884+0000 7fb90a800640 1 --2- 172.21.15.19:0/2149828429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90c0cd890 0x7fb90c147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:37.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.884+0000 7fb90a800640 1 -- 172.21.15.19:0/2149828429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90c156140 con 0x7fb90c1529d0 2024-03-28T13:18:37.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.885+0000 7fb90a800640 1 --2- 172.21.15.19:0/2149828429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c1529d0 0x7fb90c154210 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb90400efc0 tx=0x7fb90400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:37.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.885+0000 7fb902000640 1 -- 172.21.15.19:0/2149828429 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904018020 con 0x7fb90c1529d0 2024-03-28T13:18:37.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.885+0000 7fb902000640 1 -- 172.21.15.19:0/2149828429 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb904014480 con 0x7fb90c1529d0 2024-03-28T13:18:37.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.885+0000 7fb90b200640 1 -- 172.21.15.19:0/2149828429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90c154b60 con 0x7fb90c1529d0 2024-03-28T13:18:37.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.885+0000 7fb902000640 1 -- 172.21.15.19:0/2149828429 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904014870 con 0x7fb90c1529d0 2024-03-28T13:18:37.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.885+0000 7fb90b200640 1 -- 172.21.15.19:0/2149828429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb90c155120 con 0x7fb90c1529d0 2024-03-28T13:18:37.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.886+0000 7fb90b200640 1 -- 172.21.15.19:0/2149828429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb90c1558e0 con 0x7fb90c1529d0 2024-03-28T13:18:37.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.889+0000 7fb902000640 1 -- 172.21.15.19:0/2149828429 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb904005d10 con 0x7fb90c1529d0 2024-03-28T13:18:37.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.889+0000 7fb902000640 1 --2- 172.21.15.19:0/2149828429 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb8ec06c530 0x7fb8ec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:37.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.890+0000 7fb902000640 1 -- 172.21.15.19:0/2149828429 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7fb90408c290 con 0x7fb90c1529d0 2024-03-28T13:18:37.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.890+0000 7fb909e00640 1 --2- 172.21.15.19:0/2149828429 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb8ec06c530 0x7fb8ec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:37.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.891+0000 7fb909e00640 1 --2- 172.21.15.19:0/2149828429 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb8ec06c530 0x7fb8ec06e9f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb90c14ea80 tx=0x7fb8fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:37.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:37.894+0000 7fb902000640 1 -- 172.21.15.19:0/2149828429 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb90c1558e0 con 0x7fb90c1529d0 2024-03-28T13:18:38.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.102+0000 7fb90b200640 1 -- 172.21.15.19:0/2149828429 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb90c1524f0 con 0x7fb90c1529d0 2024-03-28T13:18:38.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.103+0000 7fb902000640 1 -- 172.21.15.19:0/2149828429 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fb904022070 con 0x7fb90c1529d0 2024-03-28T13:18:38.106 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:18:38.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.105+0000 7fb90b200640 1 -- 172.21.15.19:0/2149828429 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb8ec06c530 msgr2=0x7fb8ec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:38.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.105+0000 7fb90b200640 1 --2- 172.21.15.19:0/2149828429 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb8ec06c530 0x7fb8ec06e9f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb90c14ea80 tx=0x7fb8fc009040 comp rx=0 tx=0).stop 2024-03-28T13:18:38.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.105+0000 7fb90b200640 1 -- 172.21.15.19:0/2149828429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c1529d0 msgr2=0x7fb90c154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:38.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.105+0000 7fb90b200640 1 --2- 172.21.15.19:0/2149828429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c1529d0 0x7fb90c154210 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb90400efc0 tx=0x7fb90400c490 comp rx=0 tx=0).stop 2024-03-28T13:18:38.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.105+0000 7fb90b200640 1 -- 172.21.15.19:0/2149828429 shutdown_connections 2024-03-28T13:18:38.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.105+0000 7fb90b200640 1 --2- 172.21.15.19:0/2149828429 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb8ec06c530 0x7fb8ec06e9f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:38.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.105+0000 7fb90b200640 1 --2- 172.21.15.19:0/2149828429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb90c1529d0 0x7fb90c154210 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:38.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.105+0000 7fb90b200640 1 --2- 172.21.15.19:0/2149828429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb90c0cd890 0x7fb90c147a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:38.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.105+0000 7fb90b200640 1 --2- 172.21.15.19:0/2149828429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb90c0afaa0 0x7fb90c155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:38.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.105+0000 7fb90b200640 1 -- 172.21.15.19:0/2149828429 >> 172.21.15.19:0/2149828429 conn(0x7fb90c0d0770 msgr2=0x7fb90c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.105+0000 7fb90b200640 1 -- 172.21.15.19:0/2149828429 shutdown_connections 2024-03-28T13:18:38.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.105+0000 7fb90b200640 1 -- 172.21.15.19:0/2149828429 wait complete. 2024-03-28T13:18:38.125 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:18:38.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.337+0000 7fa76b200640 1 Processor -- start 2024-03-28T13:18:38.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.337+0000 7fa76b200640 1 -- start start 2024-03-28T13:18:38.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.338+0000 7fa76b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7640af4b0 0x7fa764167460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:38.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.338+0000 7fa76b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7640afab0 con 0x7fa764153580 2024-03-28T13:18:38.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.338+0000 7fa76b200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa76415cc80 con 0x7fa7640d0990 2024-03-28T13:18:38.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.338+0000 7fa76b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa76415da00 con 0x7fa7640af4b0 2024-03-28T13:18:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.339+0000 7fa763e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa764153580 0x7fa7640d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52062/0 (socket says 172.21.15.19:52062) 2024-03-28T13:18:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.339+0000 7fa763e00640 1 -- 172.21.15.19:0/1036271998 learned_addr learned my addr 172.21.15.19:0/1036271998 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.339+0000 7fa769200640 1 --2- 172.21.15.19:0/1036271998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7640af4b0 0x7fa764167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.339+0000 7fa762a00640 1 -- 172.21.15.19:0/1036271998 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2053921566 0 0) 0x7fa7640afab0 con 0x7fa764153580 2024-03-28T13:18:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.339+0000 7fa762a00640 1 -- 172.21.15.19:0/1036271998 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa744003520 con 0x7fa764153580 2024-03-28T13:18:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.340+0000 7fa762a00640 1 -- 172.21.15.19:0/1036271998 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3463246696 0 0) 0x7fa744003520 con 0x7fa764153580 2024-03-28T13:18:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.340+0000 7fa762a00640 1 -- 172.21.15.19:0/1036271998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7640af4b0 msgr2=0x7fa764167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.340+0000 7fa762a00640 1 --2- 172.21.15.19:0/1036271998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7640af4b0 0x7fa764167460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.340+0000 7fa762a00640 1 -- 172.21.15.19:0/1036271998 >> v1:172.21.15.188:6789/0 conn(0x7fa7640d0990 legacy=0x7fa7640aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.340+0000 7fa762a00640 1 -- 172.21.15.19:0/1036271998 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76415cf20 con 0x7fa764153580 2024-03-28T13:18:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.340+0000 7fa762a00640 1 -- 172.21.15.19:0/1036271998 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fa754001670 con 0x7fa764153580 2024-03-28T13:18:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.340+0000 7fa762a00640 1 -- 172.21.15.19:0/1036271998 >> v1:172.21.15.19:6789/0 conn(0x7fa764153580 legacy=0x7fa7640d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:38.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.340+0000 7fa762a00640 1 --2- 172.21.15.19:0/1036271998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7440039e0 0x7fa744003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.340+0000 7fa762a00640 1 --2- 172.21.15.19:0/1036271998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa744004800 0x7fa744004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.340+0000 7fa762a00640 1 --2- 172.21.15.19:0/1036271998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7640af4b0 0x7fa7440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.340+0000 7fa762a00640 1 -- 172.21.15.19:0/1036271998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa76415da00 con 0x7fa744004800 2024-03-28T13:18:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.340+0000 7fa762a00640 1 -- 172.21.15.19:0/1036271998 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa74400bac0 con 0x7fa7440039e0 2024-03-28T13:18:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.340+0000 7fa762a00640 1 -- 172.21.15.19:0/1036271998 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa74400bcc0 con 0x7fa7640af4b0 2024-03-28T13:18:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.340+0000 7fa763400640 1 --2- 172.21.15.19:0/1036271998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa744004800 0x7fa744004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.340+0000 7fa763e00640 1 --2- 172.21.15.19:0/1036271998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7440039e0 0x7fa744003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.341+0000 7fa769200640 1 --2- 172.21.15.19:0/1036271998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7640af4b0 0x7fa7440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.341+0000 7fa763400640 1 -- 172.21.15.19:0/1036271998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7640af4b0 msgr2=0x7fa7440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.341+0000 7fa763400640 1 --2- 172.21.15.19:0/1036271998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7640af4b0 0x7fa7440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.341+0000 7fa763400640 1 -- 172.21.15.19:0/1036271998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7440039e0 msgr2=0x7fa744003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:38.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.341+0000 7fa763400640 1 --2- 172.21.15.19:0/1036271998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7440039e0 0x7fa744003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:38.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.341+0000 7fa763400640 1 -- 172.21.15.19:0/1036271998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa744003520 con 0x7fa744004800 2024-03-28T13:18:38.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.341+0000 7fa769200640 1 --2- 172.21.15.19:0/1036271998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa7640af4b0 0x7fa7440051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:38.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.341+0000 7fa763e00640 1 --2- 172.21.15.19:0/1036271998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa7440039e0 0x7fa744003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:18:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.342+0000 7fa763400640 1 --2- 172.21.15.19:0/1036271998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa744004800 0x7fa744004c80 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fa76406f820 tx=0x7fa744005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fddabeec50fb47c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.342+0000 7fa762a00640 1 -- 172.21.15.19:0/1036271998 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c001b30 con 0x7fa744004800 2024-03-28T13:18:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.342+0000 7fa762a00640 1 -- 172.21.15.19:0/1036271998 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa74c003b70 con 0x7fa744004800 2024-03-28T13:18:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.342+0000 7fa762a00640 1 -- 172.21.15.19:0/1036271998 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0050d0 con 0x7fa744004800 2024-03-28T13:18:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.343+0000 7fa76b200640 1 -- 172.21.15.19:0/1036271998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa744004800 msgr2=0x7fa744004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.343+0000 7fa76b200640 1 --2- 172.21.15.19:0/1036271998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa744004800 0x7fa744004c80 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fa76406f820 tx=0x7fa744005830 comp rx=0 tx=0).stop 2024-03-28T13:18:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.343+0000 7fa769200640 1 -- 172.21.15.19:0/1036271998 reap_dead start 2024-03-28T13:18:38.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.343+0000 7fa76b200640 1 -- 172.21.15.19:0/1036271998 shutdown_connections 2024-03-28T13:18:38.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.343+0000 7fa76b200640 1 -- 172.21.15.19:0/1036271998 >> 172.21.15.19:0/1036271998 conn(0x7fa76406b750 msgr2=0x7fa76406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:38.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.343+0000 7fa76b200640 1 -- 172.21.15.19:0/1036271998 shutdown_connections 2024-03-28T13:18:38.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.343+0000 7fa76b200640 1 -- 172.21.15.19:0/1036271998 wait complete. 2024-03-28T13:18:38.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.345+0000 7fa76b200640 1 Processor -- start 2024-03-28T13:18:38.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.345+0000 7fa76b200640 1 -- start start 2024-03-28T13:18:38.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.346+0000 7fa76b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa76414aa60 0x7fa764072940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:38.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.346+0000 7fa76b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa76414ae40 0x7fa76414c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:38.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.346+0000 7fa76b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76414b270 0x7fa76415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:38.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.346+0000 7fa76b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa764153960 con 0x7fa76414b270 2024-03-28T13:18:38.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.346+0000 7fa76b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa76415d4e0 con 0x7fa76414aa60 2024-03-28T13:18:38.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.346+0000 7fa76b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa764165450 con 0x7fa76414ae40 2024-03-28T13:18:38.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.346+0000 7fa769200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76414b270 0x7fa76415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:38.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.346+0000 7fa769200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76414b270 0x7fa76415b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54134/0 (socket says 172.21.15.19:54134) 2024-03-28T13:18:38.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.346+0000 7fa769200640 1 -- 172.21.15.19:0/1831112685 learned_addr learned my addr 172.21.15.19:0/1831112685 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:38.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.346+0000 7fa763e00640 1 --2- 172.21.15.19:0/1831112685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa76414aa60 0x7fa764072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:38.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.346+0000 7fa763400640 1 --2- 172.21.15.19:0/1831112685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa76414ae40 0x7fa76414c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:38.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.347+0000 7fa769200640 1 -- 172.21.15.19:0/1831112685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa76414ae40 msgr2=0x7fa76414c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:38.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.347+0000 7fa769200640 1 --2- 172.21.15.19:0/1831112685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa76414ae40 0x7fa76414c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:38.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.347+0000 7fa769200640 1 -- 172.21.15.19:0/1831112685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa76414aa60 msgr2=0x7fa764072940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:38.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.347+0000 7fa769200640 1 --2- 172.21.15.19:0/1831112685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa76414aa60 0x7fa764072940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.347+0000 7fa769200640 1 -- 172.21.15.19:0/1831112685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7640718e0 con 0x7fa76414b270 2024-03-28T13:18:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.347+0000 7fa769200640 1 --2- 172.21.15.19:0/1831112685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76414b270 0x7fa76415b5b0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fa7640b5220 tx=0x7fa75c010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.347+0000 7fa75be00640 1 -- 172.21.15.19:0/1831112685 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00f040 con 0x7fa76414b270 2024-03-28T13:18:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.347+0000 7fa75be00640 1 -- 172.21.15.19:0/1831112685 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa75c01c070 con 0x7fa76414b270 2024-03-28T13:18:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.348+0000 7fa76b200640 1 -- 172.21.15.19:0/1831112685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa764071b80 con 0x7fa76414b270 2024-03-28T13:18:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.348+0000 7fa75be00640 1 -- 172.21.15.19:0/1831112685 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00eaa0 con 0x7fa76414b270 2024-03-28T13:18:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.348+0000 7fa76b200640 1 -- 172.21.15.19:0/1831112685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa764072140 con 0x7fa76414b270 2024-03-28T13:18:38.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.349+0000 7fa76b200640 1 -- 172.21.15.19:0/1831112685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7640685c0 con 0x7fa76414b270 2024-03-28T13:18:38.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.350+0000 7fa75be00640 1 -- 172.21.15.19:0/1831112685 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa75c00ec40 con 0x7fa76414b270 2024-03-28T13:18:38.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.351+0000 7fa75be00640 1 --2- 172.21.15.19:0/1831112685 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa74006c560 0x7fa74006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:38.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.351+0000 7fa75be00640 1 -- 172.21.15.19:0/1831112685 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7fa75c08afb0 con 0x7fa76414b270 2024-03-28T13:18:38.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.351+0000 7fa763e00640 1 --2- 172.21.15.19:0/1831112685 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa74006c560 0x7fa74006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:38.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.353+0000 7fa763e00640 1 --2- 172.21.15.19:0/1831112685 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa74006c560 0x7fa74006ea20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa754000ff0 tx=0x7fa754009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:38.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.355+0000 7fa75be00640 1 -- 172.21.15.19:0/1831112685 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa75c025020 con 0x7fa76414b270 2024-03-28T13:18:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.572+0000 7fa76b200640 1 -- 172.21.15.19:0/1831112685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa7640ae380 con 0x7fa76414b270 2024-03-28T13:18:38.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.573+0000 7fa75be00640 1 -- 172.21.15.19:0/1831112685 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v94) ==== 80+0+248 (secure 0 0 0) 0x7fa75c059480 con 0x7fa76414b270 2024-03-28T13:18:38.573 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.188:6836/1749969053 2024-03-29T13:17:55.195632+0000 2024-03-28T13:18:38.573 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.188:6834/1749969053 2024-03-29T13:17:55.195632+0000 2024-03-28T13:18:38.573 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.188:6836/2260087679 2024-03-29T13:16:39.492982+0000 2024-03-28T13:18:38.573 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.188:6834/2260087679 2024-03-29T13:16:39.492982+0000 2024-03-28T13:18:38.573 INFO:teuthology.orchestra.run.smithi019.stderr:listed 4 entries 2024-03-28T13:18:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.575+0000 7fa76b200640 1 -- 172.21.15.19:0/1831112685 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa74006c560 msgr2=0x7fa74006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.575+0000 7fa76b200640 1 --2- 172.21.15.19:0/1831112685 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa74006c560 0x7fa74006ea20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa754000ff0 tx=0x7fa754009040 comp rx=0 tx=0).stop 2024-03-28T13:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.575+0000 7fa76b200640 1 -- 172.21.15.19:0/1831112685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76414b270 msgr2=0x7fa76415b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.575+0000 7fa76b200640 1 --2- 172.21.15.19:0/1831112685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76414b270 0x7fa76415b5b0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fa7640b5220 tx=0x7fa75c010c20 comp rx=0 tx=0).stop 2024-03-28T13:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.576+0000 7fa76b200640 1 -- 172.21.15.19:0/1831112685 shutdown_connections 2024-03-28T13:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.576+0000 7fa76b200640 1 --2- 172.21.15.19:0/1831112685 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa74006c560 0x7fa74006ea20 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.576+0000 7fa76b200640 1 --2- 172.21.15.19:0/1831112685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa76414b270 0x7fa76415b5b0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.576+0000 7fa76b200640 1 --2- 172.21.15.19:0/1831112685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa76414ae40 0x7fa76414c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.576+0000 7fa76b200640 1 --2- 172.21.15.19:0/1831112685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa76414aa60 0x7fa764072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.576+0000 7fa76b200640 1 -- 172.21.15.19:0/1831112685 >> 172.21.15.19:0/1831112685 conn(0x7fa76406b750 msgr2=0x7fa7640d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.576+0000 7fa76b200640 1 -- 172.21.15.19:0/1831112685 shutdown_connections 2024-03-28T13:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:38.576+0000 7fa76b200640 1 -- 172.21.15.19:0/1831112685 wait complete. 2024-03-28T13:18:38.594 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:18:38.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:18:38.594 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T13:18:38.697 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:18:38.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:18:38.698 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:18:38.746 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:18:38.746 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:18:38.746 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:18:38.804 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:18:38.804 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:18:38.804 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:18:38.804 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:18:38.857 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:18:38.858 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:18:38.858 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:18:38.858 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:18:38.912 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:18:38.912 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:18:39.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.131+0000 7f0f66a00640 1 Processor -- start 2024-03-28T13:18:39.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.132+0000 7f0f66a00640 1 -- start start 2024-03-28T13:18:39.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.133+0000 7f0f66a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f600d0390 0x7f0f600cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:39.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.133+0000 7f0f66a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f60161f30 con 0x7f0f60147a40 2024-03-28T13:18:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.134+0000 7f0f66a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f601621d0 con 0x7f0f600ce1b0 2024-03-28T13:18:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.134+0000 7f0f66a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0f60162b00 con 0x7f0f600d0390 2024-03-28T13:18:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.134+0000 7f0f65600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0f60147a40 0x7f0f600cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52064/0 (socket says 172.21.15.19:52064) 2024-03-28T13:18:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.134+0000 7f0f65600640 1 -- 172.21.15.19:0/3531035517 learned_addr learned my addr 172.21.15.19:0/3531035517 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.134+0000 7f0f64c00640 1 --2- 172.21.15.19:0/3531035517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f600d0390 0x7f0f600cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.135+0000 7f0f5fe00640 1 -- 172.21.15.19:0/3531035517 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2530509407 0 0) 0x7f0f60161f30 con 0x7f0f60147a40 2024-03-28T13:18:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.135+0000 7f0f5fe00640 1 -- 172.21.15.19:0/3531035517 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f48003520 con 0x7f0f60147a40 2024-03-28T13:18:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.135+0000 7f0f5fe00640 1 -- 172.21.15.19:0/3531035517 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4228091418 0 0) 0x7f0f601621d0 con 0x7f0f600ce1b0 2024-03-28T13:18:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.135+0000 7f0f5fe00640 1 -- 172.21.15.19:0/3531035517 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f60161f30 con 0x7f0f600ce1b0 2024-03-28T13:18:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.135+0000 7f0f5fe00640 1 -- 172.21.15.19:0/3531035517 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1220673739 0 0) 0x7f0f48003520 con 0x7f0f60147a40 2024-03-28T13:18:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.135+0000 7f0f64c00640 1 -- 172.21.15.19:0/3531035517 >> v1:172.21.15.188:6789/0 conn(0x7f0f600ce1b0 legacy=0x7f0f6006da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.135+0000 7f0f64c00640 1 -- 172.21.15.19:0/3531035517 >> v1:172.21.15.19:6789/0 conn(0x7f0f60147a40 legacy=0x7f0f600cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:39.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.135+0000 7f0f64c00640 1 -- 172.21.15.19:0/3531035517 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f601624b0 con 0x7f0f600d0390 2024-03-28T13:18:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.137+0000 7f0f64c00640 1 --2- 172.21.15.19:0/3531035517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f600d0390 0x7f0f600cdc70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0097d0 tx=0x7f0f4c02b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9abbd854da388db server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.137+0000 7f0f5fe00640 1 -- 172.21.15.19:0/3531035517 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c037070 con 0x7f0f600d0390 2024-03-28T13:18:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.137+0000 7f0f5fe00640 1 -- 172.21.15.19:0/3531035517 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f4c030510 con 0x7f0f600d0390 2024-03-28T13:18:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.137+0000 7f0f5fe00640 1 -- 172.21.15.19:0/3531035517 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54001690 con 0x7f0f600d0390 2024-03-28T13:18:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.138+0000 7f0f66a00640 1 -- 172.21.15.19:0/3531035517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f600d0390 msgr2=0x7f0f600cdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.138+0000 7f0f66a00640 1 --2- 172.21.15.19:0/3531035517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f600d0390 0x7f0f600cdc70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0097d0 tx=0x7f0f4c02b800 comp rx=0 tx=0).stop 2024-03-28T13:18:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.138+0000 7f0f66a00640 1 -- 172.21.15.19:0/3531035517 shutdown_connections 2024-03-28T13:18:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.138+0000 7f0f66a00640 1 --2- 172.21.15.19:0/3531035517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f600d0390 0x7f0f600cdc70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.138+0000 7f0f66a00640 1 -- 172.21.15.19:0/3531035517 >> 172.21.15.19:0/3531035517 conn(0x7f0f600ccad0 msgr2=0x7f0f600cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.138+0000 7f0f66a00640 1 -- 172.21.15.19:0/3531035517 shutdown_connections 2024-03-28T13:18:39.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.138+0000 7f0f66a00640 1 -- 172.21.15.19:0/3531035517 wait complete. 2024-03-28T13:18:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.140+0000 7f0f66a00640 1 Processor -- start 2024-03-28T13:18:39.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.140+0000 7f0f66a00640 1 -- start start 2024-03-28T13:18:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.141+0000 7f0f66a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f600ce1b0 0x7f0f600b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.141+0000 7f0f66a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f600d0390 0x7f0f6015d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.141+0000 7f0f66a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f60147a40 0x7f0f600725f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.141+0000 7f0f66a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f60147fa0 con 0x7f0f600ce1b0 2024-03-28T13:18:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.141+0000 7f0f66a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f60147e20 con 0x7f0f60147a40 2024-03-28T13:18:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.141+0000 7f0f66a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0f600ce640 con 0x7f0f600d0390 2024-03-28T13:18:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.141+0000 7f0f65600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f600ce1b0 0x7f0f600b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.141+0000 7f0f65600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f600ce1b0 0x7f0f600b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54150/0 (socket says 172.21.15.19:54150) 2024-03-28T13:18:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.141+0000 7f0f65600640 1 -- 172.21.15.19:0/2562446600 learned_addr learned my addr 172.21.15.19:0/2562446600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.141+0000 7f0f66000640 1 --2- 172.21.15.19:0/2562446600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f60147a40 0x7f0f600725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.141+0000 7f0f64c00640 1 --2- 172.21.15.19:0/2562446600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f600d0390 0x7f0f6015d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.142+0000 7f0f65600640 1 -- 172.21.15.19:0/2562446600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f600d0390 msgr2=0x7f0f6015d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.142+0000 7f0f65600640 1 --2- 172.21.15.19:0/2562446600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f600d0390 0x7f0f6015d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.142+0000 7f0f65600640 1 -- 172.21.15.19:0/2562446600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f60147a40 msgr2=0x7f0f600725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.142+0000 7f0f65600640 1 --2- 172.21.15.19:0/2562446600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f60147a40 0x7f0f600725f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.142+0000 7f0f65600640 1 -- 172.21.15.19:0/2562446600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f60145420 con 0x7f0f600ce1b0 2024-03-28T13:18:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.142+0000 7f0f65600640 1 --2- 172.21.15.19:0/2562446600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f600ce1b0 0x7f0f600b09c0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400c700 tx=0x7f0f5400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.142+0000 7f0f5d600640 1 -- 172.21.15.19:0/2562446600 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54018070 con 0x7f0f600ce1b0 2024-03-28T13:18:39.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.142+0000 7f0f5d600640 1 -- 172.21.15.19:0/2562446600 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f540072d0 con 0x7f0f600ce1b0 2024-03-28T13:18:39.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.142+0000 7f0f5d600640 1 -- 172.21.15.19:0/2562446600 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f540047e0 con 0x7f0f600ce1b0 2024-03-28T13:18:39.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.142+0000 7f0f66a00640 1 -- 172.21.15.19:0/2562446600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f60145700 con 0x7f0f600ce1b0 2024-03-28T13:18:39.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.143+0000 7f0f66a00640 1 -- 172.21.15.19:0/2562446600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f60145cc0 con 0x7f0f600ce1b0 2024-03-28T13:18:39.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.144+0000 7f0f66a00640 1 -- 172.21.15.19:0/2562446600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f60075940 con 0x7f0f600ce1b0 2024-03-28T13:18:39.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.145+0000 7f0f5d600640 1 -- 172.21.15.19:0/2562446600 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0f54004980 con 0x7f0f600ce1b0 2024-03-28T13:18:39.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.146+0000 7f0f5d600640 1 --2- 172.21.15.19:0/2562446600 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f4006c560 0x7f0f4006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:39.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.146+0000 7f0f5d600640 1 -- 172.21.15.19:0/2562446600 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f0f5408ac40 con 0x7f0f600ce1b0 2024-03-28T13:18:39.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.147+0000 7f0f64c00640 1 --2- 172.21.15.19:0/2562446600 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f4006c560 0x7f0f4006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:39.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.148+0000 7f0f64c00640 1 --2- 172.21.15.19:0/2562446600 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f4006c560 0x7f0f4006ea20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c034000 tx=0x7f0f4c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:39.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.151+0000 7f0f5d600640 1 -- 172.21.15.19:0/2562446600 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0f54059110 con 0x7f0f600ce1b0 2024-03-28T13:18:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.421+0000 7f0f66a00640 1 -- 172.21.15.19:0/2562446600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f600af6f0 con 0x7f0f600ce1b0 2024-03-28T13:18:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.422+0000 7f0f5d600640 1 -- 172.21.15.19:0/2562446600 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5188 (secure 0 0 0) 0x7f0f54058a10 con 0x7f0f600ce1b0 2024-03-28T13:18:39.422 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:39.422 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1144703580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1144703580},{"type":"v1","addr":"172.21.15.188:6836","nonce":1144703580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:17:59.710623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6040":{"gid":6040,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":28,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6040,"qdb_cluster":[6040]},"id":6}]} 2024-03-28T13:18:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 74 2024-03-28T13:18:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.425+0000 7f0f66a00640 1 -- 172.21.15.19:0/2562446600 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f4006c560 msgr2=0x7f0f4006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.426+0000 7f0f66a00640 1 --2- 172.21.15.19:0/2562446600 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f4006c560 0x7f0f4006ea20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c034000 tx=0x7f0f4c008030 comp rx=0 tx=0).stop 2024-03-28T13:18:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.426+0000 7f0f66a00640 1 -- 172.21.15.19:0/2562446600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f600ce1b0 msgr2=0x7f0f600b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.426+0000 7f0f66a00640 1 --2- 172.21.15.19:0/2562446600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f600ce1b0 0x7f0f600b09c0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400c700 tx=0x7f0f5400cbc0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.426+0000 7f0f66a00640 1 -- 172.21.15.19:0/2562446600 shutdown_connections 2024-03-28T13:18:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.426+0000 7f0f66a00640 1 --2- 172.21.15.19:0/2562446600 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f4006c560 0x7f0f4006ea20 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.426+0000 7f0f66a00640 1 --2- 172.21.15.19:0/2562446600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f60147a40 0x7f0f600725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.426+0000 7f0f66a00640 1 --2- 172.21.15.19:0/2562446600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f600d0390 0x7f0f6015d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.426+0000 7f0f66a00640 1 --2- 172.21.15.19:0/2562446600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f600ce1b0 0x7f0f600b09c0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.426+0000 7f0f66a00640 1 -- 172.21.15.19:0/2562446600 >> 172.21.15.19:0/2562446600 conn(0x7f0f600ccad0 msgr2=0x7f0f6006da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.426+0000 7f0f66a00640 1 -- 172.21.15.19:0/2562446600 shutdown_connections 2024-03-28T13:18:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.426+0000 7f0f66a00640 1 -- 172.21.15.19:0/2562446600 wait complete. 2024-03-28T13:18:39.444 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:18:39.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.675+0000 7f9bee800640 1 Processor -- start 2024-03-28T13:18:39.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.675+0000 7f9bee800640 1 -- start start 2024-03-28T13:18:39.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.676+0000 7f9bee800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9be815e4a0 0x7f9be80b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:39.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.676+0000 7f9bee800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9be80d2310 0x7f9be80ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:39.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.676+0000 7f9bee800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9be815e880 con 0x7f9be80cf530 2024-03-28T13:18:39.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.676+0000 7f9bee800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9be80d34e0 con 0x7f9be80d2310 2024-03-28T13:18:39.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.676+0000 7f9bee800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9be80d28d0 con 0x7f9be815e4a0 2024-03-28T13:18:39.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.677+0000 7f9beca00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f9be80d2310 0x7f9be80ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:39.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.677+0000 7f9bede00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9be80cf530 0x7f9be80d3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52068/0 (socket says 172.21.15.19:52068) 2024-03-28T13:18:39.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.677+0000 7f9bede00640 1 -- 172.21.15.19:0/397177135 learned_addr learned my addr 172.21.15.19:0/397177135 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:39.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.677+0000 7f9bed400640 1 --2- 172.21.15.19:0/397177135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9be815e4a0 0x7f9be80b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:39.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.677+0000 7f9beca00640 1 -- 172.21.15.19:0/397177135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9be815e4a0 msgr2=0x7f9be80b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:39.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.677+0000 7f9beca00640 1 --2- 172.21.15.19:0/397177135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9be815e4a0 0x7f9be80b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.677+0000 7f9beca00640 1 -- 172.21.15.19:0/397177135 >> v1:172.21.15.19:6789/0 conn(0x7f9be80cf530 legacy=0x7f9be80d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:39.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.677+0000 7f9be7e00640 1 -- 172.21.15.19:0/397177135 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2847637351 0 0) 0x7f9be815e880 con 0x7f9be80cf530 2024-03-28T13:18:39.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.677+0000 7f9beca00640 1 -- 172.21.15.19:0/397177135 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be80d2bb0 con 0x7f9be80d2310 2024-03-28T13:18:39.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.679+0000 7f9beca00640 1 --2- 172.21.15.19:0/397177135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be80d2310 0x7f9be80ceff0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40097a0 tx=0x7f9bd402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef1b58761c5a0b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:39.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.679+0000 7f9be7e00640 1 -- 172.21.15.19:0/397177135 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4035020 con 0x7f9be80d2310 2024-03-28T13:18:39.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.679+0000 7f9be7e00640 1 -- 172.21.15.19:0/397177135 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bd4031830 con 0x7f9be80d2310 2024-03-28T13:18:39.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.679+0000 7f9be7e00640 1 -- 172.21.15.19:0/397177135 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4031c00 con 0x7f9be80d2310 2024-03-28T13:18:39.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.679+0000 7f9bee800640 1 -- 172.21.15.19:0/397177135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be80d2310 msgr2=0x7f9be80ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:39.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.679+0000 7f9bee800640 1 --2- 172.21.15.19:0/397177135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be80d2310 0x7f9be80ceff0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40097a0 tx=0x7f9bd402b8d0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.680+0000 7f9bee800640 1 -- 172.21.15.19:0/397177135 shutdown_connections 2024-03-28T13:18:39.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.680+0000 7f9bee800640 1 --2- 172.21.15.19:0/397177135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be80d2310 0x7f9be80ceff0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.680+0000 7f9bee800640 1 --2- 172.21.15.19:0/397177135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9be815e4a0 0x7f9be80b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.680+0000 7f9bee800640 1 -- 172.21.15.19:0/397177135 >> 172.21.15.19:0/397177135 conn(0x7f9be80676d0 msgr2=0x7f9be8067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:39.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.680+0000 7f9bee800640 1 -- 172.21.15.19:0/397177135 shutdown_connections 2024-03-28T13:18:39.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.680+0000 7f9bee800640 1 -- 172.21.15.19:0/397177135 wait complete. 2024-03-28T13:18:39.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.681+0000 7f9bee800640 1 Processor -- start 2024-03-28T13:18:39.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.682+0000 7f9bee800640 1 -- start start 2024-03-28T13:18:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.682+0000 7f9bee800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9be80cf530 0x7f9be81f7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9bee800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9be80d2310 0x7f9be80c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9bee800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be815e4a0 0x7f9be8143310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9bee800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9be80d3970 con 0x7f9be815e4a0 2024-03-28T13:18:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9bee800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9be80d3170 con 0x7f9be80d2310 2024-03-28T13:18:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9bee800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9be80ba1a0 con 0x7f9be80cf530 2024-03-28T13:18:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9bede00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be815e4a0 0x7f9be8143310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9bede00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be815e4a0 0x7f9be8143310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54162/0 (socket says 172.21.15.19:54162) 2024-03-28T13:18:39.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9bede00640 1 -- 172.21.15.19:0/2753393149 learned_addr learned my addr 172.21.15.19:0/2753393149 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:39.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9beca00640 1 --2- 172.21.15.19:0/2753393149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9be80d2310 0x7f9be80c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:39.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9bed400640 1 --2- 172.21.15.19:0/2753393149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9be80cf530 0x7f9be81f7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:39.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9bede00640 1 -- 172.21.15.19:0/2753393149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9be80cf530 msgr2=0x7f9be81f7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:39.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9bede00640 1 --2- 172.21.15.19:0/2753393149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9be80cf530 0x7f9be81f7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9bede00640 1 -- 172.21.15.19:0/2753393149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9be80d2310 msgr2=0x7f9be80c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:39.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9bede00640 1 --2- 172.21.15.19:0/2753393149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9be80d2310 0x7f9be80c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9bede00640 1 -- 172.21.15.19:0/2753393149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be81f77b0 con 0x7f9be815e4a0 2024-03-28T13:18:39.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.683+0000 7f9beca00640 1 --2- 172.21.15.19:0/2753393149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9be80d2310 0x7f9be80c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:39.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.684+0000 7f9bede00640 1 --2- 172.21.15.19:0/2753393149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be815e4a0 0x7f9be8143310 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c730 tx=0x7f9be000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:39.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.684+0000 7f9be5600640 1 -- 172.21.15.19:0/2753393149 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0018070 con 0x7f9be815e4a0 2024-03-28T13:18:39.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.684+0000 7f9be5600640 1 -- 172.21.15.19:0/2753393149 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be000ce40 con 0x7f9be815e4a0 2024-03-28T13:18:39.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.684+0000 7f9bee800640 1 -- 172.21.15.19:0/2753393149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be80c8cc0 con 0x7f9be815e4a0 2024-03-28T13:18:39.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.684+0000 7f9be5600640 1 -- 172.21.15.19:0/2753393149 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be000a860 con 0x7f9be815e4a0 2024-03-28T13:18:39.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.684+0000 7f9bee800640 1 -- 172.21.15.19:0/2753393149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be80c9860 con 0x7f9be815e4a0 2024-03-28T13:18:39.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.686+0000 7f9bee800640 1 -- 172.21.15.19:0/2753393149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9be8075940 con 0x7f9be815e4a0 2024-03-28T13:18:39.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.688+0000 7f9be5600640 1 -- 172.21.15.19:0/2753393149 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9be001b070 con 0x7f9be815e4a0 2024-03-28T13:18:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.689+0000 7f9be5600640 1 --2- 172.21.15.19:0/2753393149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9bc806c530 0x7f9bc806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:39.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.689+0000 7f9be5600640 1 -- 172.21.15.19:0/2753393149 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f9be008ad90 con 0x7f9be815e4a0 2024-03-28T13:18:39.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.690+0000 7f9bed400640 1 --2- 172.21.15.19:0/2753393149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9bc806c530 0x7f9bc806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:39.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.691+0000 7f9bed400640 1 --2- 172.21.15.19:0/2753393149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9bc806c530 0x7f9bc806e9f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc006fd0 tx=0x7f9bdc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:39.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.694+0000 7f9be5600640 1 -- 172.21.15.19:0/2753393149 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9be0059260 con 0x7f9be815e4a0 2024-03-28T13:18:39.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.901+0000 7f9bee800640 1 -- 172.21.15.19:0/2753393149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9be80b3d30 con 0x7f9be815e4a0 2024-03-28T13:18:39.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.903+0000 7f9be5600640 1 -- 172.21.15.19:0/2753393149 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16935 (secure 0 0 0) 0x7f9be0058b60 con 0x7f9be815e4a0 2024-03-28T13:18:39.903 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:39.903 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":94,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:17:55.195667+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:16:23.901120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-28T13:16:24.990904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6836/1749969053":"2024-03-29T13:17:55.195632+0000","172.21.15.188:6834/1749969053":"2024-03-29T13:17:55.195632+0000","172.21.15.188:6836/2260087679":"2024-03-29T13:16:39.492982+0000","172.21.15.188:6834/2260087679":"2024-03-29T13:16:39.492982+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:18:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.906+0000 7f9bee800640 1 -- 172.21.15.19:0/2753393149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9bc806c530 msgr2=0x7f9bc806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.906+0000 7f9bee800640 1 --2- 172.21.15.19:0/2753393149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9bc806c530 0x7f9bc806e9f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc006fd0 tx=0x7f9bdc008040 comp rx=0 tx=0).stop 2024-03-28T13:18:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.906+0000 7f9bee800640 1 -- 172.21.15.19:0/2753393149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be815e4a0 msgr2=0x7f9be8143310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.906+0000 7f9bee800640 1 --2- 172.21.15.19:0/2753393149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be815e4a0 0x7f9be8143310 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c730 tx=0x7f9be000cbf0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.906+0000 7f9bee800640 1 -- 172.21.15.19:0/2753393149 shutdown_connections 2024-03-28T13:18:39.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.906+0000 7f9bee800640 1 --2- 172.21.15.19:0/2753393149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9bc806c530 0x7f9bc806e9f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.906+0000 7f9bee800640 1 --2- 172.21.15.19:0/2753393149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9be815e4a0 0x7f9be8143310 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.906+0000 7f9bee800640 1 --2- 172.21.15.19:0/2753393149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9be80d2310 0x7f9be80c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.906+0000 7f9bee800640 1 --2- 172.21.15.19:0/2753393149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9be80cf530 0x7f9be81f7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.906+0000 7f9bee800640 1 -- 172.21.15.19:0/2753393149 >> 172.21.15.19:0/2753393149 conn(0x7f9be80676d0 msgr2=0x7f9be80b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.906+0000 7f9bee800640 1 -- 172.21.15.19:0/2753393149 shutdown_connections 2024-03-28T13:18:39.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:39.906+0000 7f9bee800640 1 -- 172.21.15.19:0/2753393149 wait complete. 2024-03-28T13:18:39.926 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:16:23.901120+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:16:24.990904+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:18:39.926 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:18:40.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.135+0000 7f0b18200640 1 Processor -- start 2024-03-28T13:18:40.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.136+0000 7f0b18200640 1 -- start start 2024-03-28T13:18:40.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.137+0000 7f0b18200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0b10153580 0x7f0b100d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.137+0000 7f0b18200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b100af4b0 0x7f0b101600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.137+0000 7f0b18200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b100afab0 con 0x7f0b100d07c0 2024-03-28T13:18:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.137+0000 7f0b18200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f0b100d0c30 con 0x7f0b10153580 2024-03-28T13:18:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.137+0000 7f0b18200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1015cc80 con 0x7f0b100af4b0 2024-03-28T13:18:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.137+0000 7f0b14e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0b100d07c0 0x7f0b100aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52070/0 (socket says 172.21.15.19:52070) 2024-03-28T13:18:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.137+0000 7f0b14e00640 1 -- 172.21.15.19:0/1530850753 learned_addr learned my addr 172.21.15.19:0/1530850753 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.138+0000 7f0b16200640 1 --2- 172.21.15.19:0/1530850753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b100af4b0 0x7f0b101600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.138+0000 7f0b15800640 1 --2- 172.21.15.19:0/1530850753 >> v2:172.21.15.188:3300/0 conn(0x7f0b10153580 0x7f0b100d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.138+0000 7f0b0fe00640 1 -- 172.21.15.19:0/1530850753 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3801585888 0 0) 0x7f0b100afab0 con 0x7f0b100d07c0 2024-03-28T13:18:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.138+0000 7f0b0fe00640 1 -- 172.21.15.19:0/1530850753 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0af8003520 con 0x7f0b100d07c0 2024-03-28T13:18:40.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.138+0000 7f0b0fe00640 1 -- 172.21.15.19:0/1530850753 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1786800606 0 0) 0x7f0af8003520 con 0x7f0b100d07c0 2024-03-28T13:18:40.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.138+0000 7f0b0fe00640 1 -- 172.21.15.19:0/1530850753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b100af4b0 msgr2=0x7f0b101600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.138+0000 7f0b0fe00640 1 --2- 172.21.15.19:0/1530850753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b100af4b0 0x7f0b101600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.138+0000 7f0b0fe00640 1 -- 172.21.15.19:0/1530850753 >> v2:172.21.15.188:3300/0 conn(0x7f0b10153580 msgr2=0x7f0b100d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.138+0000 7f0b0fe00640 1 --2- 172.21.15.19:0/1530850753 >> v2:172.21.15.188:3300/0 conn(0x7f0b10153580 0x7f0b100d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.138+0000 7f0b0fe00640 1 -- 172.21.15.19:0/1530850753 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1015da00 con 0x7f0b100d07c0 2024-03-28T13:18:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.138+0000 7f0b0fe00640 1 -- 172.21.15.19:0/1530850753 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f0afc001670 con 0x7f0b100d07c0 2024-03-28T13:18:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.139+0000 7f0b0fe00640 1 -- 172.21.15.19:0/1530850753 >> v1:172.21.15.19:6789/0 conn(0x7f0b100d07c0 legacy=0x7f0b100aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.139+0000 7f0b0fe00640 1 --2- 172.21.15.19:0/1530850753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b100af4b0 0x7f0af80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.139+0000 7f0b0fe00640 1 --2- 172.21.15.19:0/1530850753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8003ef0 0x7f0af8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.139+0000 7f0b0fe00640 1 --2- 172.21.15.19:0/1530850753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af8004d40 0x7f0af80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.139+0000 7f0b0fe00640 1 -- 172.21.15.19:0/1530850753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b100d0c30 con 0x7f0af8003ef0 2024-03-28T13:18:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.139+0000 7f0b0fe00640 1 -- 172.21.15.19:0/1530850753 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1015cc80 con 0x7f0af8004d40 2024-03-28T13:18:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.139+0000 7f0b0fe00640 1 -- 172.21.15.19:0/1530850753 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0af800b9c0 con 0x7f0b100af4b0 2024-03-28T13:18:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.139+0000 7f0b14e00640 1 --2- 172.21.15.19:0/1530850753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8003ef0 0x7f0af8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.140+0000 7f0b15800640 1 --2- 172.21.15.19:0/1530850753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b100af4b0 0x7f0af80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:40.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.140+0000 7f0b16200640 1 --2- 172.21.15.19:0/1530850753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af8004d40 0x7f0af80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.140+0000 7f0b14e00640 1 -- 172.21.15.19:0/1530850753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b100af4b0 msgr2=0x7f0af80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.140+0000 7f0b14e00640 1 --2- 172.21.15.19:0/1530850753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b100af4b0 0x7f0af80039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.140+0000 7f0b14e00640 1 -- 172.21.15.19:0/1530850753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af8004d40 msgr2=0x7f0af80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.140+0000 7f0b14e00640 1 --2- 172.21.15.19:0/1530850753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0af8004d40 0x7f0af80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.140+0000 7f0b14e00640 1 -- 172.21.15.19:0/1530850753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af8003520 con 0x7f0af8003ef0 2024-03-28T13:18:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.141+0000 7f0b14e00640 1 --2- 172.21.15.19:0/1530850753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8003ef0 0x7f0af8004800 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0afc005160 tx=0x7f0af8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df304e263c0a787 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.141+0000 7f0b0fe00640 1 -- 172.21.15.19:0/1530850753 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc004d30 con 0x7f0af8003ef0 2024-03-28T13:18:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.141+0000 7f0b0fe00640 1 -- 172.21.15.19:0/1530850753 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0afc005ad0 con 0x7f0af8003ef0 2024-03-28T13:18:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.141+0000 7f0b0fe00640 1 -- 172.21.15.19:0/1530850753 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0080d0 con 0x7f0af8003ef0 2024-03-28T13:18:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.142+0000 7f0b18200640 1 -- 172.21.15.19:0/1530850753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8003ef0 msgr2=0x7f0af8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:40.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.142+0000 7f0b18200640 1 --2- 172.21.15.19:0/1530850753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8003ef0 0x7f0af8004800 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0afc005160 tx=0x7f0af8005830 comp rx=0 tx=0).stop 2024-03-28T13:18:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.142+0000 7f0b16200640 1 -- 172.21.15.19:0/1530850753 reap_dead start 2024-03-28T13:18:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.142+0000 7f0b18200640 1 -- 172.21.15.19:0/1530850753 shutdown_connections 2024-03-28T13:18:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.142+0000 7f0b18200640 1 -- 172.21.15.19:0/1530850753 >> 172.21.15.19:0/1530850753 conn(0x7f0b1006b750 msgr2=0x7f0b1006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.142+0000 7f0b18200640 1 -- 172.21.15.19:0/1530850753 shutdown_connections 2024-03-28T13:18:40.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.142+0000 7f0b18200640 1 -- 172.21.15.19:0/1530850753 wait complete. 2024-03-28T13:18:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.144+0000 7f0b18200640 1 Processor -- start 2024-03-28T13:18:40.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.145+0000 7f0b18200640 1 -- start start 2024-03-28T13:18:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.146+0000 7f0b18200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8003ef0 0x7f0b1014c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.146+0000 7f0b18200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b100b3e00 0x7f0b100b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.146+0000 7f0b18200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b100ce590 0x7f0b100cea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.146+0000 7f0b18200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1015d4e0 con 0x7f0af8003ef0 2024-03-28T13:18:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.146+0000 7f0b18200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1015d200 con 0x7f0b100ce590 2024-03-28T13:18:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.146+0000 7f0b18200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0b10153960 con 0x7f0b100b3e00 2024-03-28T13:18:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.146+0000 7f0b15800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8003ef0 0x7f0b1014c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.146+0000 7f0b15800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8003ef0 0x7f0b1014c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54178/0 (socket says 172.21.15.19:54178) 2024-03-28T13:18:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.146+0000 7f0b15800640 1 -- 172.21.15.19:0/303687885 learned_addr learned my addr 172.21.15.19:0/303687885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.146+0000 7f0b14e00640 1 --2- 172.21.15.19:0/303687885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b100b3e00 0x7f0b100b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.147+0000 7f0b16200640 1 --2- 172.21.15.19:0/303687885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b100ce590 0x7f0b100cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.147+0000 7f0b15800640 1 -- 172.21.15.19:0/303687885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b100b3e00 msgr2=0x7f0b100b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:40.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.147+0000 7f0b15800640 1 --2- 172.21.15.19:0/303687885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b100b3e00 0x7f0b100b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.147+0000 7f0b15800640 1 -- 172.21.15.19:0/303687885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b100ce590 msgr2=0x7f0b100cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.147+0000 7f0b15800640 1 --2- 172.21.15.19:0/303687885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b100ce590 0x7f0b100cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.147+0000 7f0b15800640 1 -- 172.21.15.19:0/303687885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b100cef50 con 0x7f0af8003ef0 2024-03-28T13:18:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.147+0000 7f0b16200640 1 --2- 172.21.15.19:0/303687885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b100ce590 0x7f0b100cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:40.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.147+0000 7f0b15800640 1 --2- 172.21.15.19:0/303687885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8003ef0 0x7f0b1014c5a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f0b04010940 tx=0x7f0b04010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.148+0000 7f0b0d600640 1 -- 172.21.15.19:0/303687885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0400f040 con 0x7f0af8003ef0 2024-03-28T13:18:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.148+0000 7f0b0d600640 1 -- 172.21.15.19:0/303687885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b0400ec20 con 0x7f0af8003ef0 2024-03-28T13:18:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.148+0000 7f0b0d600640 1 -- 172.21.15.19:0/303687885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0401c400 con 0x7f0af8003ef0 2024-03-28T13:18:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.148+0000 7f0b18200640 1 -- 172.21.15.19:0/303687885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b10150370 con 0x7f0af8003ef0 2024-03-28T13:18:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.148+0000 7f0b18200640 1 -- 172.21.15.19:0/303687885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b101508f0 con 0x7f0af8003ef0 2024-03-28T13:18:40.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.149+0000 7f0b18200640 1 -- 172.21.15.19:0/303687885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b100685c0 con 0x7f0af8003ef0 2024-03-28T13:18:40.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.150+0000 7f0b0d600640 1 -- 172.21.15.19:0/303687885 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0b0401c5a0 con 0x7f0af8003ef0 2024-03-28T13:18:40.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.151+0000 7f0b0d600640 1 --2- 172.21.15.19:0/303687885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0af006c530 0x7f0af006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.151+0000 7f0b0d600640 1 -- 172.21.15.19:0/303687885 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f0b0408be90 con 0x7f0af8003ef0 2024-03-28T13:18:40.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.152+0000 7f0b14e00640 1 --2- 172.21.15.19:0/303687885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0af006c530 0x7f0af006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:40.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.153+0000 7f0b14e00640 1 --2- 172.21.15.19:0/303687885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0af006c530 0x7f0af006e9f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0afc005130 tx=0x7f0afc037040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:40.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.156+0000 7f0b0d600640 1 -- 172.21.15.19:0/303687885 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0b04018030 con 0x7f0af8003ef0 2024-03-28T13:18:40.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.355+0000 7f0b18200640 1 -- 172.21.15.19:0/303687885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0b1006eca0 con 0x7f0af8003ef0 2024-03-28T13:18:40.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.356+0000 7f0b0d600640 1 -- 172.21.15.19:0/303687885 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f0b0405a400 con 0x7f0af8003ef0 2024-03-28T13:18:40.355 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:18:40.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.358+0000 7f0b18200640 1 -- 172.21.15.19:0/303687885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0af006c530 msgr2=0x7f0af006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:40.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.358+0000 7f0b18200640 1 --2- 172.21.15.19:0/303687885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0af006c530 0x7f0af006e9f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0afc005130 tx=0x7f0afc037040 comp rx=0 tx=0).stop 2024-03-28T13:18:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.358+0000 7f0b18200640 1 -- 172.21.15.19:0/303687885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8003ef0 msgr2=0x7f0b1014c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.358+0000 7f0b18200640 1 --2- 172.21.15.19:0/303687885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8003ef0 0x7f0b1014c5a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f0b04010940 tx=0x7f0b04010e00 comp rx=0 tx=0).stop 2024-03-28T13:18:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.358+0000 7f0b18200640 1 -- 172.21.15.19:0/303687885 shutdown_connections 2024-03-28T13:18:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.358+0000 7f0b18200640 1 --2- 172.21.15.19:0/303687885 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0af006c530 0x7f0af006e9f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.358+0000 7f0b18200640 1 --2- 172.21.15.19:0/303687885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b100ce590 0x7f0b100cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.358+0000 7f0b18200640 1 --2- 172.21.15.19:0/303687885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b100b3e00 0x7f0b100b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.358+0000 7f0b18200640 1 --2- 172.21.15.19:0/303687885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0af8003ef0 0x7f0b1014c5a0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.358+0000 7f0b18200640 1 -- 172.21.15.19:0/303687885 >> 172.21.15.19:0/303687885 conn(0x7f0b1006b750 msgr2=0x7f0b1015f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.358+0000 7f0b18200640 1 -- 172.21.15.19:0/303687885 shutdown_connections 2024-03-28T13:18:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.358+0000 7f0b18200640 1 -- 172.21.15.19:0/303687885 wait complete. 2024-03-28T13:18:40.375 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:18:40.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.576+0000 7f2eb9800640 1 Processor -- start 2024-03-28T13:18:40.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.576+0000 7f2eb9800640 1 -- start start 2024-03-28T13:18:40.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.578+0000 7f2eb9800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40709b0 0x7f2eb4169300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.578+0000 7f2eb9800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eb40cfe80 con 0x7f2eb406bc80 2024-03-28T13:18:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.578+0000 7f2eb9800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2eb40d0160 con 0x7f2eb4147c80 2024-03-28T13:18:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.578+0000 7f2eb9800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb40d0f20 con 0x7f2eb40709b0 2024-03-28T13:18:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.578+0000 7f2eabe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f2eb406bc80 0x7f2eb40702a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52078/0 (socket says 172.21.15.19:52078) 2024-03-28T13:18:40.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.578+0000 7f2eabe00640 1 -- 172.21.15.19:0/903235158 learned_addr learned my addr 172.21.15.19:0/903235158 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.578+0000 7f2eb8e00640 1 --2- 172.21.15.19:0/903235158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40709b0 0x7f2eb4169300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.579+0000 7f2eb3400640 1 -- 172.21.15.19:0/903235158 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1437929949 0 0) 0x7f2eb40cfe80 con 0x7f2eb406bc80 2024-03-28T13:18:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.579+0000 7f2eb3400640 1 -- 172.21.15.19:0/903235158 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e98003520 con 0x7f2eb406bc80 2024-03-28T13:18:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.579+0000 7f2eb3400640 1 -- 172.21.15.19:0/903235158 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2935517374 0 0) 0x7f2e98003520 con 0x7f2eb406bc80 2024-03-28T13:18:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.579+0000 7f2eb3400640 1 -- 172.21.15.19:0/903235158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40709b0 msgr2=0x7f2eb4169300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.579+0000 7f2eb3400640 1 --2- 172.21.15.19:0/903235158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40709b0 0x7f2eb4169300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.579+0000 7f2eb3400640 1 -- 172.21.15.19:0/903235158 >> v1:172.21.15.188:6789/0 conn(0x7f2eb4147c80 legacy=0x7f2eb406b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.579+0000 7f2eb3400640 1 -- 172.21.15.19:0/903235158 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb40d0440 con 0x7f2eb406bc80 2024-03-28T13:18:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.579+0000 7f2eb3400640 1 -- 172.21.15.19:0/903235158 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f2e9c001670 con 0x7f2eb406bc80 2024-03-28T13:18:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.579+0000 7f2eb3400640 1 -- 172.21.15.19:0/903235158 >> v1:172.21.15.19:6789/0 conn(0x7f2eb406bc80 legacy=0x7f2eb40702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.579+0000 7f2eb3400640 1 --2- 172.21.15.19:0/903235158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40709b0 0x7f2e980039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.579+0000 7f2eb3400640 1 --2- 172.21.15.19:0/903235158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e98003fb0 0x7f2e98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.579+0000 7f2eb3400640 1 --2- 172.21.15.19:0/903235158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e98004d40 0x7f2e980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.579+0000 7f2eb3400640 1 -- 172.21.15.19:0/903235158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb40d0f20 con 0x7f2e98003fb0 2024-03-28T13:18:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.579+0000 7f2eb3400640 1 -- 172.21.15.19:0/903235158 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9800bb00 con 0x7f2e98004d40 2024-03-28T13:18:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.579+0000 7f2eb3400640 1 -- 172.21.15.19:0/903235158 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2e9800bd00 con 0x7f2eb40709b0 2024-03-28T13:18:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.580+0000 7f2eabe00640 1 --2- 172.21.15.19:0/903235158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e98003fb0 0x7f2e98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.580+0000 7f2eabe00640 1 -- 172.21.15.19:0/903235158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40709b0 msgr2=0x7f2e980039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.580+0000 7f2eabe00640 1 --2- 172.21.15.19:0/903235158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40709b0 0x7f2e980039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.580+0000 7f2eabe00640 1 -- 172.21.15.19:0/903235158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e98004d40 msgr2=0x7f2e980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.580+0000 7f2eabe00640 1 --2- 172.21.15.19:0/903235158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2e98004d40 0x7f2e980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.580+0000 7f2eabe00640 1 -- 172.21.15.19:0/903235158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e98003520 con 0x7f2e98003fb0 2024-03-28T13:18:40.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.581+0000 7f2eabe00640 1 --2- 172.21.15.19:0/903235158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e98003fb0 0x7f2e98004800 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00c160 tx=0x7f2e98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c862cd48fb9f1023 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.581+0000 7f2eb3400640 1 -- 172.21.15.19:0/903235158 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c007c60 con 0x7f2e98003fb0 2024-03-28T13:18:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.581+0000 7f2eb3400640 1 -- 172.21.15.19:0/903235158 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e9c030070 con 0x7f2e98003fb0 2024-03-28T13:18:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.581+0000 7f2eb3400640 1 -- 172.21.15.19:0/903235158 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c005db0 con 0x7f2e98003fb0 2024-03-28T13:18:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.582+0000 7f2eb9800640 1 -- 172.21.15.19:0/903235158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e98003fb0 msgr2=0x7f2e98004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.582+0000 7f2eb9800640 1 --2- 172.21.15.19:0/903235158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e98003fb0 0x7f2e98004800 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00c160 tx=0x7f2e98005830 comp rx=0 tx=0).stop 2024-03-28T13:18:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.582+0000 7f2eb8e00640 1 -- 172.21.15.19:0/903235158 reap_dead start 2024-03-28T13:18:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.582+0000 7f2eb9800640 1 -- 172.21.15.19:0/903235158 shutdown_connections 2024-03-28T13:18:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.582+0000 7f2eb9800640 1 -- 172.21.15.19:0/903235158 >> 172.21.15.19:0/903235158 conn(0x7f2eb406e730 msgr2=0x7f2eb406eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.582+0000 7f2eb9800640 1 -- 172.21.15.19:0/903235158 shutdown_connections 2024-03-28T13:18:40.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.582+0000 7f2eb9800640 1 -- 172.21.15.19:0/903235158 wait complete. 2024-03-28T13:18:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.583+0000 7f2eb9800640 1 Processor -- start 2024-03-28T13:18:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.584+0000 7f2eb9800640 1 -- start start 2024-03-28T13:18:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.584+0000 7f2eb9800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e98003fb0 0x7f2eb40b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.585+0000 7f2eb9800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40b16d0 0x7f2eb40b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.585+0000 7f2eb9800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eb40b31f0 0x7f2eb40b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.585+0000 7f2eb9800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb40cfb50 con 0x7f2e98003fb0 2024-03-28T13:18:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.585+0000 7f2eb9800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2eb40d0a90 con 0x7f2eb40b31f0 2024-03-28T13:18:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.585+0000 7f2eb9800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2eb4073500 con 0x7f2eb40b16d0 2024-03-28T13:18:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.585+0000 7f2eb3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e98003fb0 0x7f2eb40b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.585+0000 7f2eb3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e98003fb0 0x7f2eb40b5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54200/0 (socket says 172.21.15.19:54200) 2024-03-28T13:18:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.585+0000 7f2eb3e00640 1 -- 172.21.15.19:0/2720460645 learned_addr learned my addr 172.21.15.19:0/2720460645 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:40.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.585+0000 7f2eabe00640 1 --2- 172.21.15.19:0/2720460645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40b16d0 0x7f2eb40b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.585+0000 7f2eb8e00640 1 --2- 172.21.15.19:0/2720460645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eb40b31f0 0x7f2eb40b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.585+0000 7f2eb3e00640 1 -- 172.21.15.19:0/2720460645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40b16d0 msgr2=0x7f2eb40b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.585+0000 7f2eb3e00640 1 --2- 172.21.15.19:0/2720460645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40b16d0 0x7f2eb40b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.585+0000 7f2eb3e00640 1 -- 172.21.15.19:0/2720460645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eb40b31f0 msgr2=0x7f2eb40b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.585+0000 7f2eb3e00640 1 --2- 172.21.15.19:0/2720460645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eb40b31f0 0x7f2eb40b3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.585+0000 7f2eb3e00640 1 -- 172.21.15.19:0/2720460645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eb40b2150 con 0x7f2e98003fb0 2024-03-28T13:18:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.586+0000 7f2eb3e00640 1 --2- 172.21.15.19:0/2720460645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e98003fb0 0x7f2eb40b5690 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400c500 tx=0x7f2ea400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.586+0000 7f2eb0c00640 1 -- 172.21.15.19:0/2720460645 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40052d0 con 0x7f2e98003fb0 2024-03-28T13:18:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.586+0000 7f2eb0c00640 1 -- 172.21.15.19:0/2720460645 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ea4005470 con 0x7f2e98003fb0 2024-03-28T13:18:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.586+0000 7f2eb0c00640 1 -- 172.21.15.19:0/2720460645 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4012870 con 0x7f2e98003fb0 2024-03-28T13:18:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.586+0000 7f2eb9800640 1 -- 172.21.15.19:0/2720460645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eb40b2430 con 0x7f2e98003fb0 2024-03-28T13:18:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.586+0000 7f2eb9800640 1 -- 172.21.15.19:0/2720460645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eb40b29f0 con 0x7f2e98003fb0 2024-03-28T13:18:40.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.587+0000 7f2eb9800640 1 -- 172.21.15.19:0/2720460645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eb4075940 con 0x7f2e98003fb0 2024-03-28T13:18:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.589+0000 7f2eb0c00640 1 -- 172.21.15.19:0/2720460645 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2ea4012a10 con 0x7f2e98003fb0 2024-03-28T13:18:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.590+0000 7f2eb0c00640 1 --2- 172.21.15.19:0/2720460645 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e9006c560 0x7f2e9006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.591+0000 7f2eb0c00640 1 -- 172.21.15.19:0/2720460645 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f2ea408b420 con 0x7f2e98003fb0 2024-03-28T13:18:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.591+0000 7f2eabe00640 1 --2- 172.21.15.19:0/2720460645 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e9006c560 0x7f2e9006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.593+0000 7f2eabe00640 1 --2- 172.21.15.19:0/2720460645 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e9006c560 0x7f2e9006ea20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c001050 tx=0x7f2e9c007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.595+0000 7f2eb0c00640 1 -- 172.21.15.19:0/2720460645 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2ea4059980 con 0x7f2e98003fb0 2024-03-28T13:18:40.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.794+0000 7f2eb9800640 1 -- 172.21.15.19:0/2720460645 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2eb415f900 con 0x7f2e98003fb0 2024-03-28T13:18:40.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.794+0000 7f2eb0c00640 1 -- 172.21.15.19:0/2720460645 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f2ea4059280 con 0x7f2e98003fb0 2024-03-28T13:18:40.794 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:18:40.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.797+0000 7f2eb9800640 1 -- 172.21.15.19:0/2720460645 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e9006c560 msgr2=0x7f2e9006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:40.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.797+0000 7f2eb9800640 1 --2- 172.21.15.19:0/2720460645 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e9006c560 0x7f2e9006ea20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c001050 tx=0x7f2e9c007000 comp rx=0 tx=0).stop 2024-03-28T13:18:40.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.797+0000 7f2eb9800640 1 -- 172.21.15.19:0/2720460645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e98003fb0 msgr2=0x7f2eb40b5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:40.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.797+0000 7f2eb9800640 1 --2- 172.21.15.19:0/2720460645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e98003fb0 0x7f2eb40b5690 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400c500 tx=0x7f2ea400c9c0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.797+0000 7f2eb9800640 1 -- 172.21.15.19:0/2720460645 shutdown_connections 2024-03-28T13:18:40.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.797+0000 7f2eb9800640 1 --2- 172.21.15.19:0/2720460645 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2e9006c560 0x7f2e9006ea20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.797+0000 7f2eb9800640 1 --2- 172.21.15.19:0/2720460645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2eb40b31f0 0x7f2eb40b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.797+0000 7f2eb9800640 1 --2- 172.21.15.19:0/2720460645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2eb40b16d0 0x7f2eb40b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.797+0000 7f2eb9800640 1 --2- 172.21.15.19:0/2720460645 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2e98003fb0 0x7f2eb40b5690 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:40.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.797+0000 7f2eb9800640 1 -- 172.21.15.19:0/2720460645 >> 172.21.15.19:0/2720460645 conn(0x7f2eb406e730 msgr2=0x7f2eb406eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:40.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.797+0000 7f2eb9800640 1 -- 172.21.15.19:0/2720460645 shutdown_connections 2024-03-28T13:18:40.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:40.797+0000 7f2eb9800640 1 -- 172.21.15.19:0/2720460645 wait complete. 2024-03-28T13:18:40.816 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:18:41.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.044+0000 7fc32dc00640 1 Processor -- start 2024-03-28T13:18:41.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.045+0000 7fc32dc00640 1 -- start start 2024-03-28T13:18:41.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.046+0000 7fc32dc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc32813f190 0x7fc3280cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:41.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.046+0000 7fc32dc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc3280cc880 0x7fc3280cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:41.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.046+0000 7fc32dc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc32806d620 0x7fc32806da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:41.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.046+0000 7fc32dc00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc3280cd230 con 0x7fc32813f190 2024-03-28T13:18:41.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.047+0000 7fc32dc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc3280ce890 con 0x7fc3280cc880 2024-03-28T13:18:41.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.047+0000 7fc32dc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc3280ceb50 con 0x7fc32806d620 2024-03-28T13:18:41.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.047+0000 7fc327e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc32813f190 0x7fc3280cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:41.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.047+0000 7fc327e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc32813f190 0x7fc3280cf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54202/0 (socket says 172.21.15.19:54202) 2024-03-28T13:18:41.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.047+0000 7fc327e00640 1 -- 172.21.15.19:0/1526062346 learned_addr learned my addr 172.21.15.19:0/1526062346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.047+0000 7fc32d200640 1 --2- 172.21.15.19:0/1526062346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc32806d620 0x7fc32806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.047+0000 7fc31fe00640 1 --2- 172.21.15.19:0/1526062346 >> v2:172.21.15.188:3300/0 conn(0x7fc3280cc880 0x7fc3280cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.047+0000 7fc327e00640 1 -- 172.21.15.19:0/1526062346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc32806d620 msgr2=0x7fc32806da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.048+0000 7fc327e00640 1 --2- 172.21.15.19:0/1526062346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc32806d620 0x7fc32806da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.048+0000 7fc327e00640 1 -- 172.21.15.19:0/1526062346 >> v2:172.21.15.188:3300/0 conn(0x7fc3280cc880 msgr2=0x7fc3280cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.048+0000 7fc327e00640 1 --2- 172.21.15.19:0/1526062346 >> v2:172.21.15.188:3300/0 conn(0x7fc3280cc880 0x7fc3280cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.048+0000 7fc327e00640 1 -- 172.21.15.19:0/1526062346 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3280cee30 con 0x7fc32813f190 2024-03-28T13:18:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.049+0000 7fc327e00640 1 --2- 172.21.15.19:0/1526062346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc32813f190 0x7fc3280cf630 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fc3280cc7b0 tx=0x7fc31802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c360df988c6f047b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.049+0000 7fc327400640 1 -- 172.21.15.19:0/1526062346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc318035020 con 0x7fc32813f190 2024-03-28T13:18:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.049+0000 7fc327400640 1 -- 172.21.15.19:0/1526062346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31802d040 con 0x7fc32813f190 2024-03-28T13:18:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.049+0000 7fc327400640 1 -- 172.21.15.19:0/1526062346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc318031a80 con 0x7fc32813f190 2024-03-28T13:18:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.050+0000 7fc32dc00640 1 -- 172.21.15.19:0/1526062346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc32813f190 msgr2=0x7fc3280cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.050+0000 7fc32dc00640 1 --2- 172.21.15.19:0/1526062346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc32813f190 0x7fc3280cf630 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fc3280cc7b0 tx=0x7fc31802b900 comp rx=0 tx=0).stop 2024-03-28T13:18:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.050+0000 7fc32dc00640 1 -- 172.21.15.19:0/1526062346 shutdown_connections 2024-03-28T13:18:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.050+0000 7fc32dc00640 1 --2- 172.21.15.19:0/1526062346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc32806d620 0x7fc32806da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.050+0000 7fc32dc00640 1 --2- 172.21.15.19:0/1526062346 >> v2:172.21.15.188:3300/0 conn(0x7fc3280cc880 0x7fc3280cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.050+0000 7fc32dc00640 1 --2- 172.21.15.19:0/1526062346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc32813f190 0x7fc3280cf630 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.050+0000 7fc32dc00640 1 -- 172.21.15.19:0/1526062346 >> 172.21.15.19:0/1526062346 conn(0x7fc3280b1ba0 msgr2=0x7fc3280b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.050+0000 7fc32dc00640 1 -- 172.21.15.19:0/1526062346 shutdown_connections 2024-03-28T13:18:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.050+0000 7fc32dc00640 1 -- 172.21.15.19:0/1526062346 wait complete. 2024-03-28T13:18:41.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.052+0000 7fc32dc00640 1 Processor -- start 2024-03-28T13:18:41.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.052+0000 7fc32dc00640 1 -- start start 2024-03-28T13:18:41.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.053+0000 7fc32dc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc32806d620 0x7fc3281f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:41.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.053+0000 7fc32dc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3280cc880 0x7fc32815d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:41.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.053+0000 7fc32dc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc32813f190 0x7fc32815de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.053+0000 7fc32dc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc32813f570 con 0x7fc3280cc880 2024-03-28T13:18:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.053+0000 7fc32dc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc328160c70 con 0x7fc32813f190 2024-03-28T13:18:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.053+0000 7fc32dc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc3280cfb70 con 0x7fc32806d620 2024-03-28T13:18:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.053+0000 7fc31fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3280cc880 0x7fc32815d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.053+0000 7fc31fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3280cc880 0x7fc32815d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54212/0 (socket says 172.21.15.19:54212) 2024-03-28T13:18:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.053+0000 7fc31fe00640 1 -- 172.21.15.19:0/130059709 learned_addr learned my addr 172.21.15.19:0/130059709 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.054+0000 7fc327e00640 1 --2- 172.21.15.19:0/130059709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc32806d620 0x7fc3281f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.054+0000 7fc32d200640 1 --2- 172.21.15.19:0/130059709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc32813f190 0x7fc32815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.054+0000 7fc31fe00640 1 -- 172.21.15.19:0/130059709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc32806d620 msgr2=0x7fc3281f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.054+0000 7fc31fe00640 1 --2- 172.21.15.19:0/130059709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc32806d620 0x7fc3281f7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.054+0000 7fc31fe00640 1 -- 172.21.15.19:0/130059709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc32813f190 msgr2=0x7fc32815de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.054+0000 7fc31fe00640 1 --2- 172.21.15.19:0/130059709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc32813f190 0x7fc32815de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.054+0000 7fc31fe00640 1 -- 172.21.15.19:0/130059709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3280aa180 con 0x7fc3280cc880 2024-03-28T13:18:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.054+0000 7fc32d200640 1 --2- 172.21.15.19:0/130059709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc32813f190 0x7fc32815de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.054+0000 7fc31fe00640 1 --2- 172.21.15.19:0/130059709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3280cc880 0x7fc32815d930 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fc31000d830 tx=0x7fc31000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.054+0000 7fc324c00640 1 -- 172.21.15.19:0/130059709 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31000b6c0 con 0x7fc3280cc880 2024-03-28T13:18:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.055+0000 7fc324c00640 1 -- 172.21.15.19:0/130059709 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc310019070 con 0x7fc3280cc880 2024-03-28T13:18:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.055+0000 7fc32dc00640 1 -- 172.21.15.19:0/130059709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3280aa460 con 0x7fc3280cc880 2024-03-28T13:18:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.055+0000 7fc324c00640 1 -- 172.21.15.19:0/130059709 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31000ba80 con 0x7fc3280cc880 2024-03-28T13:18:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.055+0000 7fc32dc00640 1 -- 172.21.15.19:0/130059709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3280aaa20 con 0x7fc3280cc880 2024-03-28T13:18:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.056+0000 7fc32dc00640 1 -- 172.21.15.19:0/130059709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc328075940 con 0x7fc3280cc880 2024-03-28T13:18:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.057+0000 7fc324c00640 1 -- 172.21.15.19:0/130059709 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc31001e460 con 0x7fc3280cc880 2024-03-28T13:18:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.058+0000 7fc324c00640 1 --2- 172.21.15.19:0/130059709 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc30406c530 0x7fc30406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.058+0000 7fc324c00640 1 -- 172.21.15.19:0/130059709 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7fc31008bda0 con 0x7fc3280cc880 2024-03-28T13:18:41.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.058+0000 7fc327e00640 1 --2- 172.21.15.19:0/130059709 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc30406c530 0x7fc30406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:41.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.060+0000 7fc327e00640 1 --2- 172.21.15.19:0/130059709 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc30406c530 0x7fc30406e9f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc318036000 tx=0x7fc318008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:41.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.063+0000 7fc324c00640 1 -- 172.21.15.19:0/130059709 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc31005a300 con 0x7fc3280cc880 2024-03-28T13:18:41.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.268+0000 7fc32dc00640 1 -- 172.21.15.19:0/130059709 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc3280b2870 con 0x7fc3280cc880 2024-03-28T13:18:41.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.269+0000 7fc324c00640 1 -- 172.21.15.19:0/130059709 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fc310059c00 con 0x7fc3280cc880 2024-03-28T13:18:41.268 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:18:41.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.271+0000 7fc32dc00640 1 -- 172.21.15.19:0/130059709 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc30406c530 msgr2=0x7fc30406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:41.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.271+0000 7fc32dc00640 1 --2- 172.21.15.19:0/130059709 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc30406c530 0x7fc30406e9f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fc318036000 tx=0x7fc318008030 comp rx=0 tx=0).stop 2024-03-28T13:18:41.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.271+0000 7fc32dc00640 1 -- 172.21.15.19:0/130059709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3280cc880 msgr2=0x7fc32815d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:41.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.271+0000 7fc32dc00640 1 --2- 172.21.15.19:0/130059709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3280cc880 0x7fc32815d930 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fc31000d830 tx=0x7fc31000dcf0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.272+0000 7fc32dc00640 1 -- 172.21.15.19:0/130059709 shutdown_connections 2024-03-28T13:18:41.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.272+0000 7fc32dc00640 1 --2- 172.21.15.19:0/130059709 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc30406c530 0x7fc30406e9f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.272+0000 7fc32dc00640 1 --2- 172.21.15.19:0/130059709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc32813f190 0x7fc32815de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.272+0000 7fc32dc00640 1 --2- 172.21.15.19:0/130059709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3280cc880 0x7fc32815d930 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.272+0000 7fc32dc00640 1 --2- 172.21.15.19:0/130059709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc32806d620 0x7fc3281f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.272+0000 7fc32dc00640 1 -- 172.21.15.19:0/130059709 >> 172.21.15.19:0/130059709 conn(0x7fc3280b1ba0 msgr2=0x7fc3280b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:41.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.272+0000 7fc32dc00640 1 -- 172.21.15.19:0/130059709 shutdown_connections 2024-03-28T13:18:41.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.272+0000 7fc32dc00640 1 -- 172.21.15.19:0/130059709 wait complete. 2024-03-28T13:18:41.300 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:18:41.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.549+0000 7f2a8d400640 1 Processor -- start 2024-03-28T13:18:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.549+0000 7f2a8d400640 1 -- start start 2024-03-28T13:18:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.551+0000 7f2a8d400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a8815aa90 0x7f2a8815b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.551+0000 7f2a8d400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2a880717f0 0x7f2a880b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.551+0000 7f2a8d400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a88072100 0x7f2a88072550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.551+0000 7f2a8d400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2a880b5190 con 0x7f2a880717f0 2024-03-28T13:18:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.551+0000 7f2a8d400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f2a88072a90 con 0x7f2a8815aa90 2024-03-28T13:18:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.551+0000 7f2a8d400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2a880b4240 con 0x7f2a88072100 2024-03-28T13:18:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.551+0000 7f2a85e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2a880717f0 0x7f2a880b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.551+0000 7f2a85e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2a880717f0 0x7f2a880b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54228/0 (socket says 172.21.15.19:54228) 2024-03-28T13:18:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.551+0000 7f2a85e00640 1 -- 172.21.15.19:0/3566526967 learned_addr learned my addr 172.21.15.19:0/3566526967 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.551+0000 7f2a87200640 1 --2- 172.21.15.19:0/3566526967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a88072100 0x7f2a88072550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.551+0000 7f2a86800640 1 --2- 172.21.15.19:0/3566526967 >> v2:172.21.15.188:3300/0 conn(0x7f2a8815aa90 0x7f2a8815b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.552+0000 7f2a85e00640 1 -- 172.21.15.19:0/3566526967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a88072100 msgr2=0x7f2a88072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.552+0000 7f2a85e00640 1 --2- 172.21.15.19:0/3566526967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a88072100 0x7f2a88072550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.552+0000 7f2a85e00640 1 -- 172.21.15.19:0/3566526967 >> v2:172.21.15.188:3300/0 conn(0x7f2a8815aa90 msgr2=0x7f2a8815b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:41.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.552+0000 7f2a85e00640 1 --2- 172.21.15.19:0/3566526967 >> v2:172.21.15.188:3300/0 conn(0x7f2a8815aa90 0x7f2a8815b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.552+0000 7f2a85e00640 1 -- 172.21.15.19:0/3566526967 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8815bee0 con 0x7f2a880717f0 2024-03-28T13:18:41.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.553+0000 7f2a85e00640 1 --2- 172.21.15.19:0/3566526967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a880717f0 0x7f2a880b46d0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f2a88071200 tx=0x7f2a7402e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5c797e9aea3b27 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:41.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.553+0000 7f2a85400640 1 -- 172.21.15.19:0/3566526967 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7403b070 con 0x7f2a880717f0 2024-03-28T13:18:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.554+0000 7f2a85400640 1 -- 172.21.15.19:0/3566526967 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a74036440 con 0x7f2a880717f0 2024-03-28T13:18:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.554+0000 7f2a85400640 1 -- 172.21.15.19:0/3566526967 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74036810 con 0x7f2a880717f0 2024-03-28T13:18:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.554+0000 7f2a8d400640 1 -- 172.21.15.19:0/3566526967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a880717f0 msgr2=0x7f2a880b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.554+0000 7f2a8d400640 1 --2- 172.21.15.19:0/3566526967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a880717f0 0x7f2a880b46d0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f2a88071200 tx=0x7f2a7402e0d0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.555+0000 7f2a8d400640 1 -- 172.21.15.19:0/3566526967 shutdown_connections 2024-03-28T13:18:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.555+0000 7f2a8d400640 1 --2- 172.21.15.19:0/3566526967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a88072100 0x7f2a88072550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.555+0000 7f2a8d400640 1 --2- 172.21.15.19:0/3566526967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a880717f0 0x7f2a880b46d0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.555+0000 7f2a8d400640 1 --2- 172.21.15.19:0/3566526967 >> v2:172.21.15.188:3300/0 conn(0x7f2a8815aa90 0x7f2a8815b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.555+0000 7f2a8d400640 1 -- 172.21.15.19:0/3566526967 >> 172.21.15.19:0/3566526967 conn(0x7f2a8806b990 msgr2=0x7f2a8806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.555+0000 7f2a8d400640 1 -- 172.21.15.19:0/3566526967 shutdown_connections 2024-03-28T13:18:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.555+0000 7f2a8d400640 1 -- 172.21.15.19:0/3566526967 wait complete. 2024-03-28T13:18:41.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.558+0000 7f2a8d400640 1 Processor -- start 2024-03-28T13:18:41.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.558+0000 7f2a8d400640 1 -- start start 2024-03-28T13:18:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.559+0000 7f2a8d400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a880717f0 0x7f2a881f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.559+0000 7f2a8d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a88072100 0x7f2a880b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.559+0000 7f2a8d400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a8815aa90 0x7f2a8814b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.559+0000 7f2a8d400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a880b3ed0 con 0x7f2a88072100 2024-03-28T13:18:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.559+0000 7f2a8d400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a880b3bf0 con 0x7f2a880717f0 2024-03-28T13:18:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.559+0000 7f2a8d400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2a880b4c10 con 0x7f2a8815aa90 2024-03-28T13:18:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.559+0000 7f2a85e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a88072100 0x7f2a880b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.559+0000 7f2a85e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a88072100 0x7f2a880b0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54232/0 (socket says 172.21.15.19:54232) 2024-03-28T13:18:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.559+0000 7f2a85e00640 1 -- 172.21.15.19:0/642222918 learned_addr learned my addr 172.21.15.19:0/642222918 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.560+0000 7f2a85e00640 1 -- 172.21.15.19:0/642222918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a8815aa90 msgr2=0x7f2a8814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.560+0000 7f2a85e00640 1 --2- 172.21.15.19:0/642222918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a8815aa90 0x7f2a8814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.560+0000 7f2a86800640 1 --2- 172.21.15.19:0/642222918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a880717f0 0x7f2a881f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.560+0000 7f2a85e00640 1 -- 172.21.15.19:0/642222918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a880717f0 msgr2=0x7f2a881f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.560+0000 7f2a85e00640 1 --2- 172.21.15.19:0/642222918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a880717f0 0x7f2a881f3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.560+0000 7f2a85e00640 1 -- 172.21.15.19:0/642222918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a88152e80 con 0x7f2a88072100 2024-03-28T13:18:41.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.560+0000 7f2a85e00640 1 --2- 172.21.15.19:0/642222918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a88072100 0x7f2a880b0f70 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f2a7402e110 tx=0x7f2a740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.561+0000 7f2a7ea00640 1 -- 172.21.15.19:0/642222918 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74045020 con 0x7f2a88072100 2024-03-28T13:18:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.561+0000 7f2a7ea00640 1 -- 172.21.15.19:0/642222918 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a740346c0 con 0x7f2a88072100 2024-03-28T13:18:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.561+0000 7f2a7ea00640 1 -- 172.21.15.19:0/642222918 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7403b040 con 0x7f2a88072100 2024-03-28T13:18:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.561+0000 7f2a8d400640 1 -- 172.21.15.19:0/642222918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a88153160 con 0x7f2a88072100 2024-03-28T13:18:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.561+0000 7f2a8d400640 1 -- 172.21.15.19:0/642222918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a88153720 con 0x7f2a88072100 2024-03-28T13:18:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.562+0000 7f2a8d400640 1 -- 172.21.15.19:0/642222918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a880685c0 con 0x7f2a88072100 2024-03-28T13:18:41.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.563+0000 7f2a7ea00640 1 -- 172.21.15.19:0/642222918 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2a74048020 con 0x7f2a88072100 2024-03-28T13:18:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.564+0000 7f2a7ea00640 1 --2- 172.21.15.19:0/642222918 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a6806c530 0x7f2a6806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.564+0000 7f2a7ea00640 1 -- 172.21.15.19:0/642222918 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f2a740b02b0 con 0x7f2a88072100 2024-03-28T13:18:41.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.565+0000 7f2a86800640 1 --2- 172.21.15.19:0/642222918 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a6806c530 0x7f2a6806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:41.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.566+0000 7f2a86800640 1 --2- 172.21.15.19:0/642222918 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a6806c530 0x7f2a6806e9f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2a78006fd0 tx=0x7f2a78008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:41.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.568+0000 7f2a7ea00640 1 -- 172.21.15.19:0/642222918 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2a7404f030 con 0x7f2a88072100 2024-03-28T13:18:41.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.815+0000 7f2a8d400640 1 -- 172.21.15.19:0/642222918 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a8806bf10 con 0x7f2a88072100 2024-03-28T13:18:41.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.816+0000 7f2a7ea00640 1 -- 172.21.15.19:0/642222918 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5188 (secure 0 0 0) 0x7f2a8815bee0 con 0x7f2a88072100 2024-03-28T13:18:41.816 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:41.816 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1144703580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1144703580},{"type":"v1","addr":"172.21.15.188:6836","nonce":1144703580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:17:59.710623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6040":{"gid":6040,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":28,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6040,"qdb_cluster":[6040]},"id":6}]} 2024-03-28T13:18:41.816 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 74 2024-03-28T13:18:41.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.819+0000 7f2a8d400640 1 -- 172.21.15.19:0/642222918 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a6806c530 msgr2=0x7f2a6806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:41.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.819+0000 7f2a8d400640 1 --2- 172.21.15.19:0/642222918 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a6806c530 0x7f2a6806e9f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2a78006fd0 tx=0x7f2a78008040 comp rx=0 tx=0).stop 2024-03-28T13:18:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.819+0000 7f2a8d400640 1 -- 172.21.15.19:0/642222918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a88072100 msgr2=0x7f2a880b0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.819+0000 7f2a8d400640 1 --2- 172.21.15.19:0/642222918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a88072100 0x7f2a880b0f70 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f2a7402e110 tx=0x7f2a740046c0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.819+0000 7f2a8d400640 1 -- 172.21.15.19:0/642222918 shutdown_connections 2024-03-28T13:18:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.819+0000 7f2a8d400640 1 --2- 172.21.15.19:0/642222918 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2a6806c530 0x7f2a6806e9f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.819+0000 7f2a8d400640 1 --2- 172.21.15.19:0/642222918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a8815aa90 0x7f2a8814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.819+0000 7f2a8d400640 1 --2- 172.21.15.19:0/642222918 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a88072100 0x7f2a880b0f70 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.819+0000 7f2a8d400640 1 --2- 172.21.15.19:0/642222918 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a880717f0 0x7f2a881f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:41.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.819+0000 7f2a8d400640 1 -- 172.21.15.19:0/642222918 >> 172.21.15.19:0/642222918 conn(0x7f2a8806b990 msgr2=0x7f2a8815ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:41.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.819+0000 7f2a8d400640 1 -- 172.21.15.19:0/642222918 shutdown_connections 2024-03-28T13:18:41.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:41.819+0000 7f2a8d400640 1 -- 172.21.15.19:0/642222918 wait complete. 2024-03-28T13:18:41.839 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:18:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.041+0000 7f865d400640 1 Processor -- start 2024-03-28T13:18:42.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.041+0000 7f865d400640 1 -- start start 2024-03-28T13:18:42.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.043+0000 7f865d400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f865815fc50 0x7f86580cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:42.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.043+0000 7f865d400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86580cc490 0x7f86580d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:42.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.043+0000 7f865d400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86580b3150 con 0x7f86580cc0b0 2024-03-28T13:18:42.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.043+0000 7f865d400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f865815c610 con 0x7f865815fc50 2024-03-28T13:18:42.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.043+0000 7f865d400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f86581600e0 con 0x7f86580cc490 2024-03-28T13:18:42.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.043+0000 7f8657e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f86580cc0b0 0x7f865806dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52088/0 (socket says 172.21.15.19:52088) 2024-03-28T13:18:42.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.043+0000 7f8657e00640 1 -- 172.21.15.19:0/3496980083 learned_addr learned my addr 172.21.15.19:0/3496980083 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:42.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f864f400640 1 --2- 172.21.15.19:0/3496980083 >> v2:172.21.15.188:3300/0 conn(0x7f865815fc50 0x7f86580cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:42.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 -- 172.21.15.19:0/3496980083 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3566007873 0 0) 0x7f86580b3150 con 0x7f86580cc0b0 2024-03-28T13:18:42.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f865ca00640 1 --2- 172.21.15.19:0/3496980083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86580cc490 0x7f86580d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 -- 172.21.15.19:0/3496980083 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f863c003520 con 0x7f86580cc0b0 2024-03-28T13:18:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 -- 172.21.15.19:0/3496980083 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 293986085 0 0) 0x7f863c003520 con 0x7f86580cc0b0 2024-03-28T13:18:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 -- 172.21.15.19:0/3496980083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86580cc490 msgr2=0x7f86580d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 --2- 172.21.15.19:0/3496980083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86580cc490 0x7f86580d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 -- 172.21.15.19:0/3496980083 >> v2:172.21.15.188:3300/0 conn(0x7f865815fc50 msgr2=0x7f86580cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 --2- 172.21.15.19:0/3496980083 >> v2:172.21.15.188:3300/0 conn(0x7f865815fc50 0x7f86580cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 -- 172.21.15.19:0/3496980083 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86581603c0 con 0x7f86580cc0b0 2024-03-28T13:18:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 -- 172.21.15.19:0/3496980083 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f86480019f0 con 0x7f86580cc0b0 2024-03-28T13:18:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 -- 172.21.15.19:0/3496980083 >> v1:172.21.15.19:6789/0 conn(0x7f86580cc0b0 legacy=0x7f865806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 --2- 172.21.15.19:0/3496980083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86580cc490 0x7f863c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 --2- 172.21.15.19:0/3496980083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f863c004030 0x7f863c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 --2- 172.21.15.19:0/3496980083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f863c004d40 0x7f863c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 -- 172.21.15.19:0/3496980083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f865815c610 con 0x7f863c004d40 2024-03-28T13:18:42.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 -- 172.21.15.19:0/3496980083 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f86581600e0 con 0x7f863c004030 2024-03-28T13:18:42.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.044+0000 7f8657400640 1 -- 172.21.15.19:0/3496980083 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f863c00bb20 con 0x7f86580cc490 2024-03-28T13:18:42.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.045+0000 7f865ca00640 1 --2- 172.21.15.19:0/3496980083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f863c004d40 0x7f863c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:42.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.045+0000 7f864f400640 1 --2- 172.21.15.19:0/3496980083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f863c004030 0x7f863c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:42.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.045+0000 7f8657e00640 1 --2- 172.21.15.19:0/3496980083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86580cc490 0x7f863c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:42.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.045+0000 7f865ca00640 1 -- 172.21.15.19:0/3496980083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86580cc490 msgr2=0x7f863c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:42.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.045+0000 7f865ca00640 1 --2- 172.21.15.19:0/3496980083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86580cc490 0x7f863c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.045+0000 7f865ca00640 1 -- 172.21.15.19:0/3496980083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f863c004030 msgr2=0x7f863c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:42.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.045+0000 7f865ca00640 1 --2- 172.21.15.19:0/3496980083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f863c004030 0x7f863c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.045+0000 7f865ca00640 1 -- 172.21.15.19:0/3496980083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c003520 con 0x7f863c004d40 2024-03-28T13:18:42.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.047+0000 7f865ca00640 1 --2- 172.21.15.19:0/3496980083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f863c004d40 0x7f863c0051c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f865000ba00 tx=0x7f863c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cd6133521ccff2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:42.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.047+0000 7f8657400640 1 -- 172.21.15.19:0/3496980083 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8650036020 con 0x7f863c004d40 2024-03-28T13:18:42.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.047+0000 7f8657400640 1 -- 172.21.15.19:0/3496980083 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8650032620 con 0x7f863c004d40 2024-03-28T13:18:42.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.047+0000 7f8657400640 1 -- 172.21.15.19:0/3496980083 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f865000b600 con 0x7f863c004d40 2024-03-28T13:18:42.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.048+0000 7f865d400640 1 -- 172.21.15.19:0/3496980083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f863c004d40 msgr2=0x7f863c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:42.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.048+0000 7f865d400640 1 --2- 172.21.15.19:0/3496980083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f863c004d40 0x7f863c0051c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f865000ba00 tx=0x7f863c005830 comp rx=0 tx=0).stop 2024-03-28T13:18:42.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.048+0000 7f865ca00640 1 -- 172.21.15.19:0/3496980083 reap_dead start 2024-03-28T13:18:42.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.048+0000 7f865d400640 1 -- 172.21.15.19:0/3496980083 shutdown_connections 2024-03-28T13:18:42.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.048+0000 7f865d400640 1 -- 172.21.15.19:0/3496980083 >> 172.21.15.19:0/3496980083 conn(0x7f86580ccad0 msgr2=0x7f86580cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:42.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.048+0000 7f865d400640 1 -- 172.21.15.19:0/3496980083 shutdown_connections 2024-03-28T13:18:42.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.048+0000 7f865d400640 1 -- 172.21.15.19:0/3496980083 wait complete. 2024-03-28T13:18:42.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.050+0000 7f865d400640 1 Processor -- start 2024-03-28T13:18:42.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.050+0000 7f865d400640 1 -- start start 2024-03-28T13:18:42.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.051+0000 7f865d400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86581f5520 0x7f8658156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:42.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.051+0000 7f865d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86580aee30 0x7f86581f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:42.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.051+0000 7f865d400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86580b0ef0 0x7f86580b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:42.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.051+0000 7f865d400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86580d0300 con 0x7f86580aee30 2024-03-28T13:18:42.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.051+0000 7f865d400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8658160980 con 0x7f86580b0ef0 2024-03-28T13:18:42.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.051+0000 7f865d400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f86580d0480 con 0x7f86581f5520 2024-03-28T13:18:42.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.051+0000 7f864f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86580aee30 0x7f86581f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:42.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.051+0000 7f864f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86580aee30 0x7f86581f2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54248/0 (socket says 172.21.15.19:54248) 2024-03-28T13:18:42.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.051+0000 7f864f400640 1 -- 172.21.15.19:0/2223916431 learned_addr learned my addr 172.21.15.19:0/2223916431 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:42.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.052+0000 7f8657e00640 1 --2- 172.21.15.19:0/2223916431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86581f5520 0x7f8658156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:42.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.052+0000 7f865ca00640 1 --2- 172.21.15.19:0/2223916431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86580b0ef0 0x7f86580b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:42.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.052+0000 7f864f400640 1 -- 172.21.15.19:0/2223916431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86581f5520 msgr2=0x7f8658156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:42.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.052+0000 7f864f400640 1 --2- 172.21.15.19:0/2223916431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86581f5520 0x7f8658156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.052+0000 7f864f400640 1 -- 172.21.15.19:0/2223916431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86580b0ef0 msgr2=0x7f86580b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:42.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.052+0000 7f864f400640 1 --2- 172.21.15.19:0/2223916431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86580b0ef0 0x7f86580b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.052+0000 7f864f400640 1 -- 172.21.15.19:0/2223916431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86580aebf0 con 0x7f86580aee30 2024-03-28T13:18:42.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.052+0000 7f864f400640 1 --2- 172.21.15.19:0/2223916431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86580aee30 0x7f86581f2650 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f864000f5d0 tx=0x7f864000fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.052+0000 7f8654c00640 1 -- 172.21.15.19:0/2223916431 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640019020 con 0x7f86580aee30 2024-03-28T13:18:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.052+0000 7f8654c00640 1 -- 172.21.15.19:0/2223916431 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8640014870 con 0x7f86580aee30 2024-03-28T13:18:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.052+0000 7f865d400640 1 -- 172.21.15.19:0/2223916431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8658156990 con 0x7f86580aee30 2024-03-28T13:18:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.052+0000 7f8654c00640 1 -- 172.21.15.19:0/2223916431 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640014c20 con 0x7f86580aee30 2024-03-28T13:18:42.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.052+0000 7f865d400640 1 -- 172.21.15.19:0/2223916431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8658153b00 con 0x7f86580aee30 2024-03-28T13:18:42.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.054+0000 7f865d400640 1 -- 172.21.15.19:0/2223916431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8658075940 con 0x7f86580aee30 2024-03-28T13:18:42.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.055+0000 7f8654c00640 1 -- 172.21.15.19:0/2223916431 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f864001c020 con 0x7f86580aee30 2024-03-28T13:18:42.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.056+0000 7f8654c00640 1 --2- 172.21.15.19:0/2223916431 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f863406c560 0x7f863406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:42.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.056+0000 7f8654c00640 1 -- 172.21.15.19:0/2223916431 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f864008bf90 con 0x7f86580aee30 2024-03-28T13:18:42.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.057+0000 7f8657e00640 1 --2- 172.21.15.19:0/2223916431 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f863406c560 0x7f863406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:42.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.058+0000 7f8657e00640 1 --2- 172.21.15.19:0/2223916431 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f863406c560 0x7f863406ea20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f86480013d0 tx=0x7f8648001f20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:42.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.060+0000 7f8654c00640 1 -- 172.21.15.19:0/2223916431 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8640023020 con 0x7f86580aee30 2024-03-28T13:18:42.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.259+0000 7f865d400640 1 -- 172.21.15.19:0/2223916431 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86580b5c70 con 0x7f86580aee30 2024-03-28T13:18:42.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.261+0000 7f8654c00640 1 -- 172.21.15.19:0/2223916431 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16935 (secure 0 0 0) 0x7f8640026350 con 0x7f86580aee30 2024-03-28T13:18:42.260 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:42.261 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":94,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:17:55.195667+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:16:23.901120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-28T13:16:24.990904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6836/1749969053":"2024-03-29T13:17:55.195632+0000","172.21.15.188:6834/1749969053":"2024-03-29T13:17:55.195632+0000","172.21.15.188:6836/2260087679":"2024-03-29T13:16:39.492982+0000","172.21.15.188:6834/2260087679":"2024-03-29T13:16:39.492982+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:18:42.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.264+0000 7f865d400640 1 -- 172.21.15.19:0/2223916431 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f863406c560 msgr2=0x7f863406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:42.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.264+0000 7f865d400640 1 --2- 172.21.15.19:0/2223916431 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f863406c560 0x7f863406ea20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f86480013d0 tx=0x7f8648001f20 comp rx=0 tx=0).stop 2024-03-28T13:18:42.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.264+0000 7f865d400640 1 -- 172.21.15.19:0/2223916431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86580aee30 msgr2=0x7f86581f2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:42.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.264+0000 7f865d400640 1 --2- 172.21.15.19:0/2223916431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86580aee30 0x7f86581f2650 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f864000f5d0 tx=0x7f864000fa90 comp rx=0 tx=0).stop 2024-03-28T13:18:42.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.264+0000 7f865d400640 1 -- 172.21.15.19:0/2223916431 shutdown_connections 2024-03-28T13:18:42.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.264+0000 7f865d400640 1 --2- 172.21.15.19:0/2223916431 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f863406c560 0x7f863406ea20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.264+0000 7f865d400640 1 --2- 172.21.15.19:0/2223916431 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f86580b0ef0 0x7f86580b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.264+0000 7f865d400640 1 --2- 172.21.15.19:0/2223916431 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86580aee30 0x7f86581f2650 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.264+0000 7f865d400640 1 --2- 172.21.15.19:0/2223916431 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f86581f5520 0x7f8658156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.264+0000 7f865d400640 1 -- 172.21.15.19:0/2223916431 >> 172.21.15.19:0/2223916431 conn(0x7f86580ccad0 msgr2=0x7f865815efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:42.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.264+0000 7f865d400640 1 -- 172.21.15.19:0/2223916431 shutdown_connections 2024-03-28T13:18:42.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.264+0000 7f865d400640 1 -- 172.21.15.19:0/2223916431 wait complete. 2024-03-28T13:18:42.289 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-28T13:18:42.289 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:18:42.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.495+0000 7f5dfbe00640 1 Processor -- start 2024-03-28T13:18:42.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.495+0000 7f5dfbe00640 1 -- start start 2024-03-28T13:18:42.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.496+0000 7f5dfbe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5dfc15e4a0 0x7f5dfc0b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:42.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.496+0000 7f5dfbe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc0d2310 0x7f5dfc0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:42.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.496+0000 7f5dfbe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dfc15e880 con 0x7f5dfc0cf530 2024-03-28T13:18:42.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.496+0000 7f5dfbe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5dfc0d34e0 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.496+0000 7f5dfbe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5dfc0d28d0 con 0x7f5dfc0d2310 2024-03-28T13:18:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.497+0000 7f5dfaa00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5dfc15e4a0 0x7f5dfc0b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.497+0000 7f5df2000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc0d2310 0x7f5dfc0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.497+0000 7f5df2000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc0d2310 0x7f5dfc0ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:37490/0 (socket says 172.21.15.19:37490) 2024-03-28T13:18:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.497+0000 7f5df2000640 1 -- 172.21.15.19:0/2137763886 learned_addr learned my addr 172.21.15.19:0/2137763886 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:42.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.497+0000 7f5dfaa00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5dfc15e4a0 0x7f5dfc0b9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:52426/0 (socket says 172.21.15.19:52426) 2024-03-28T13:18:42.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.498+0000 7f5dfaa00640 1 -- 172.21.15.19:0/2137763886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc0d2310 msgr2=0x7f5dfc0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:42.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.498+0000 7f5dfaa00640 1 --2- 172.21.15.19:0/2137763886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc0d2310 0x7f5dfc0ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.498+0000 7f5dfaa00640 1 -- 172.21.15.19:0/2137763886 >> v1:172.21.15.188:6789/0 conn(0x7f5dfc0cf530 legacy=0x7f5dfc0d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:42.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.498+0000 7f5dfaa00640 1 -- 172.21.15.19:0/2137763886 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dfc0d2bb0 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.498+0000 7f5df2000640 1 --2- 172.21.15.19:0/2137763886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc0d2310 0x7f5dfc0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:18:42.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.499+0000 7f5dfaa00640 1 --2- 172.21.15.19:0/2137763886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dfc15e4a0 0x7f5dfc0b9c60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0097a0 tx=0x7f5dec02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd9d3f6246e79041 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:42.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.500+0000 7f5dfa000640 1 -- 172.21.15.19:0/2137763886 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec035020 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.500+0000 7f5dfa000640 1 -- 172.21.15.19:0/2137763886 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dec031810 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.500+0000 7f5dfa000640 1 -- 172.21.15.19:0/2137763886 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dec031be0 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.500+0000 7f5dfbe00640 1 -- 172.21.15.19:0/2137763886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dfc15e4a0 msgr2=0x7f5dfc0b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:42.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.500+0000 7f5dfbe00640 1 --2- 172.21.15.19:0/2137763886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dfc15e4a0 0x7f5dfc0b9c60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5dec0097a0 tx=0x7f5dec02b8b0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.500+0000 7f5dfbe00640 1 -- 172.21.15.19:0/2137763886 shutdown_connections 2024-03-28T13:18:42.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.500+0000 7f5dfbe00640 1 --2- 172.21.15.19:0/2137763886 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc0d2310 0x7f5dfc0ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.500+0000 7f5dfbe00640 1 --2- 172.21.15.19:0/2137763886 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dfc15e4a0 0x7f5dfc0b9c60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.500+0000 7f5dfbe00640 1 -- 172.21.15.19:0/2137763886 >> 172.21.15.19:0/2137763886 conn(0x7f5dfc0676d0 msgr2=0x7f5dfc067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:42.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.501+0000 7f5dfbe00640 1 -- 172.21.15.19:0/2137763886 shutdown_connections 2024-03-28T13:18:42.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.501+0000 7f5dfbe00640 1 -- 172.21.15.19:0/2137763886 wait complete. 2024-03-28T13:18:42.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.503+0000 7f5dfbe00640 1 Processor -- start 2024-03-28T13:18:42.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.503+0000 7f5dfbe00640 1 -- start start 2024-03-28T13:18:42.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.504+0000 7f5dfbe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dfc0cf530 0x7f5dfc1547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:42.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.504+0000 7f5dfbe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc0d2310 0x7f5dfc0cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:42.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.504+0000 7f5dfbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dfc15e4a0 0x7f5dfc0cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:42.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.504+0000 7f5dfbe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc0d3970 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.504+0000 7f5dfbe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc0d3170 con 0x7f5dfc0cf530 2024-03-28T13:18:42.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.504+0000 7f5dfbe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5dfc0ba1a0 con 0x7f5dfc0d2310 2024-03-28T13:18:42.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.504+0000 7f5dfb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dfc15e4a0 0x7f5dfc0cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:42.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.504+0000 7f5dfb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dfc15e4a0 0x7f5dfc0cc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54258/0 (socket says 172.21.15.19:54258) 2024-03-28T13:18:42.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.504+0000 7f5dfb400640 1 -- 172.21.15.19:0/1124973163 learned_addr learned my addr 172.21.15.19:0/1124973163 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.505+0000 7f5df2000640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc0d2310 0x7f5dfc0cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.505+0000 7f5dfaa00640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dfc0cf530 0x7f5dfc1547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.505+0000 7f5dfb400640 1 -- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc0d2310 msgr2=0x7f5dfc0cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.505+0000 7f5dfb400640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc0d2310 0x7f5dfc0cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.505+0000 7f5dfb400640 1 -- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dfc0cf530 msgr2=0x7f5dfc1547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.505+0000 7f5dfb400640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dfc0cf530 0x7f5dfc1547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.505+0000 7f5dfb400640 1 -- 172.21.15.19:0/1124973163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dfc0cc810 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.505+0000 7f5dfaa00640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dfc0cf530 0x7f5dfc1547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.505+0000 7f5df2000640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc0d2310 0x7f5dfc0cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.505+0000 7f5dfb400640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dfc15e4a0 0x7f5dfc0cc2d0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f5df400c6e0 tx=0x7f5df400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.506+0000 7f5df3400640 1 -- 172.21.15.19:0/1124973163 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4003b70 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.506+0000 7f5df3400640 1 -- 172.21.15.19:0/1124973163 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5df4003d30 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.506+0000 7f5df3400640 1 -- 172.21.15.19:0/1124973163 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4005310 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.506+0000 7f5dfbe00640 1 -- 172.21.15.19:0/1124973163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dfc0cca80 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.506+0000 7f5dfbe00640 1 -- 172.21.15.19:0/1124973163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dfc0cd040 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.507+0000 7f5dfbe00640 1 -- 172.21.15.19:0/1124973163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dfc075940 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.508+0000 7f5df3400640 1 -- 172.21.15.19:0/1124973163 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5df40054b0 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.509+0000 7f5df3400640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5dd806c530 0x7f5dd806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:42.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.509+0000 7f5df3400640 1 -- 172.21.15.19:0/1124973163 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f5df408b5c0 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.510+0000 7f5dfaa00640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5dd806c530 0x7f5dd806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:42.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.511+0000 7f5dfaa00640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5dd806c530 0x7f5dd806e9f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5dec036000 tx=0x7f5dec008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:42.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.514+0000 7f5df3400640 1 -- 172.21.15.19:0/1124973163 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5df4059b30 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.830+0000 7f5dfbe00640 1 -- 172.21.15.19:0/1124973163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dfc0d4980 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.830+0000 7f5df3400640 1 -- 172.21.15.19:0/1124973163 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5188 (secure 0 0 0) 0x7f5df4059430 con 0x7f5dfc15e4a0 2024-03-28T13:18:42.830 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:42.830 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1144703580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1144703580},{"type":"v1","addr":"172.21.15.188:6836","nonce":1144703580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:17:59.710623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6040":{"gid":6040,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":28,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6040,"qdb_cluster":[6040]},"id":6}]} 2024-03-28T13:18:42.831 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 74 2024-03-28T13:18:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.833+0000 7f5dfbe00640 1 -- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5dd806c530 msgr2=0x7f5dd806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:42.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.833+0000 7f5dfbe00640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5dd806c530 0x7f5dd806e9f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5dec036000 tx=0x7f5dec008030 comp rx=0 tx=0).stop 2024-03-28T13:18:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.834+0000 7f5dfbe00640 1 -- 172.21.15.19:0/1124973163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dfc15e4a0 msgr2=0x7f5dfc0cc2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.834+0000 7f5dfbe00640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dfc15e4a0 0x7f5dfc0cc2d0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f5df400c6e0 tx=0x7f5df400cba0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.834+0000 7f5dfbe00640 1 -- 172.21.15.19:0/1124973163 shutdown_connections 2024-03-28T13:18:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.834+0000 7f5dfbe00640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5dd806c530 0x7f5dd806e9f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.834+0000 7f5dfbe00640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5dfc15e4a0 0x7f5dfc0cc2d0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.834+0000 7f5dfbe00640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5dfc0d2310 0x7f5dfc0cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.834+0000 7f5dfbe00640 1 --2- 172.21.15.19:0/1124973163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5dfc0cf530 0x7f5dfc1547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.834+0000 7f5dfbe00640 1 -- 172.21.15.19:0/1124973163 >> 172.21.15.19:0/1124973163 conn(0x7f5dfc0676d0 msgr2=0x7f5dfc0b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.834+0000 7f5dfbe00640 1 -- 172.21.15.19:0/1124973163 shutdown_connections 2024-03-28T13:18:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:42.834+0000 7f5dfbe00640 1 -- 172.21.15.19:0/1124973163 wait complete. 2024-03-28T13:18:42.853 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:18:43.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.058+0000 7f825a200640 1 Processor -- start 2024-03-28T13:18:43.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.059+0000 7f825a200640 1 -- start start 2024-03-28T13:18:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.060+0000 7f825a200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f825406b570 0x7f825406b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.060+0000 7f825a200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82540d08c0 con 0x7f825415e6e0 2024-03-28T13:18:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.060+0000 7f825a200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82540d0ba0 con 0x7f82540d0120 2024-03-28T13:18:43.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.060+0000 7f825a200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f825406bf00 con 0x7f825406b570 2024-03-28T13:18:43.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.061+0000 7f8258e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f825415e6e0 0x7f82540b0580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:52094/0 (socket says 172.21.15.19:52094) 2024-03-28T13:18:43.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.061+0000 7f8258e00640 1 -- 172.21.15.19:0/3321379977 learned_addr learned my addr 172.21.15.19:0/3321379977 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:43.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.061+0000 7f8259800640 1 --2- 172.21.15.19:0/3321379977 >> v2:172.21.15.188:3300/0 conn(0x7f825406b570 0x7f825406b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:43.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.061+0000 7f8253400640 1 -- 172.21.15.19:0/3321379977 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1830799015 0 0) 0x7f82540d08c0 con 0x7f825415e6e0 2024-03-28T13:18:43.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.061+0000 7f8253400640 1 -- 172.21.15.19:0/3321379977 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f823c003520 con 0x7f825415e6e0 2024-03-28T13:18:43.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.061+0000 7f8253400640 1 -- 172.21.15.19:0/3321379977 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1993457704 0 0) 0x7f823c003520 con 0x7f825415e6e0 2024-03-28T13:18:43.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.061+0000 7f8253400640 1 -- 172.21.15.19:0/3321379977 >> v2:172.21.15.188:3300/0 conn(0x7f825406b570 msgr2=0x7f825406b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:43.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.061+0000 7f8253400640 1 --2- 172.21.15.19:0/3321379977 >> v2:172.21.15.188:3300/0 conn(0x7f825406b570 0x7f825406b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.061+0000 7f8253400640 1 -- 172.21.15.19:0/3321379977 >> v1:172.21.15.188:6789/0 conn(0x7f82540d0120 legacy=0x7f8254162880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:43.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.061+0000 7f8253400640 1 -- 172.21.15.19:0/3321379977 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82540d0e80 con 0x7f825415e6e0 2024-03-28T13:18:43.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.061+0000 7f8253400640 1 -- 172.21.15.19:0/3321379977 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f8248001670 con 0x7f825415e6e0 2024-03-28T13:18:43.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.062+0000 7f8253400640 1 -- 172.21.15.19:0/3321379977 >> v1:172.21.15.19:6789/0 conn(0x7f825415e6e0 legacy=0x7f82540b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:43.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.062+0000 7f8253400640 1 --2- 172.21.15.19:0/3321379977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0039b0 0x7f823c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:43.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.062+0000 7f8253400640 1 --2- 172.21.15.19:0/3321379977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f823c004800 0x7f823c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:43.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.062+0000 7f8253400640 1 --2- 172.21.15.19:0/3321379977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f823c0051c0 0x7f823c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:43.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.062+0000 7f8253400640 1 -- 172.21.15.19:0/3321379977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f825406bf00 con 0x7f823c0051c0 2024-03-28T13:18:43.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.062+0000 7f8253400640 1 -- 172.21.15.19:0/3321379977 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f823c00c060 con 0x7f823c004800 2024-03-28T13:18:43.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.062+0000 7f8253400640 1 -- 172.21.15.19:0/3321379977 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f823c00c260 con 0x7f823c0039b0 2024-03-28T13:18:43.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.062+0000 7f8259800640 1 --2- 172.21.15.19:0/3321379977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f823c0051c0 0x7f823c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:43.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.062+0000 7f8258e00640 1 --2- 172.21.15.19:0/3321379977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0039b0 0x7f823c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:43.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.062+0000 7f8259800640 1 -- 172.21.15.19:0/3321379977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0039b0 msgr2=0x7f823c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:43.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.062+0000 7f8259800640 1 --2- 172.21.15.19:0/3321379977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0039b0 0x7f823c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.062+0000 7f8259800640 1 -- 172.21.15.19:0/3321379977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f823c004800 msgr2=0x7f823c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:43.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.062+0000 7f8253e00640 1 --2- 172.21.15.19:0/3321379977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f823c004800 0x7f823c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:43.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.063+0000 7f8259800640 1 --2- 172.21.15.19:0/3321379977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f823c004800 0x7f823c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.063+0000 7f8259800640 1 -- 172.21.15.19:0/3321379977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823c003520 con 0x7f823c0051c0 2024-03-28T13:18:43.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.063+0000 7f8253e00640 1 --2- 172.21.15.19:0/3321379977 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f823c004800 0x7f823c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:43.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.064+0000 7f8259800640 1 -- 172.21.15.19:0/3321379977 reap_dead start 2024-03-28T13:18:43.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.064+0000 7f8259800640 1 --2- 172.21.15.19:0/3321379977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f823c0051c0 0x7f823c00b9c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f824c009fc0 tx=0x7f823c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec545ca97bdd15e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:43.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.064+0000 7f8253400640 1 -- 172.21.15.19:0/3321379977 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c009c10 con 0x7f823c0051c0 2024-03-28T13:18:43.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.064+0000 7f8253400640 1 -- 172.21.15.19:0/3321379977 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f824c02cca0 con 0x7f823c0051c0 2024-03-28T13:18:43.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.064+0000 7f8253400640 1 -- 172.21.15.19:0/3321379977 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c003bd0 con 0x7f823c0051c0 2024-03-28T13:18:43.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.065+0000 7f825a200640 1 -- 172.21.15.19:0/3321379977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f823c0051c0 msgr2=0x7f823c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:43.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.065+0000 7f825a200640 1 --2- 172.21.15.19:0/3321379977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f823c0051c0 0x7f823c00b9c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f824c009fc0 tx=0x7f823c005830 comp rx=0 tx=0).stop 2024-03-28T13:18:43.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.065+0000 7f825a200640 1 -- 172.21.15.19:0/3321379977 shutdown_connections 2024-03-28T13:18:43.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.065+0000 7f825a200640 1 --2- 172.21.15.19:0/3321379977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f823c0051c0 0x7f823c00b9c0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.065+0000 7f825a200640 1 -- 172.21.15.19:0/3321379977 >> 172.21.15.19:0/3321379977 conn(0x7f825409aa20 msgr2=0x7f82540b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:43.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.065+0000 7f825a200640 1 -- 172.21.15.19:0/3321379977 shutdown_connections 2024-03-28T13:18:43.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.065+0000 7f825a200640 1 -- 172.21.15.19:0/3321379977 wait complete. 2024-03-28T13:18:43.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.066+0000 7f825a200640 1 Processor -- start 2024-03-28T13:18:43.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.067+0000 7f825a200640 1 -- start start 2024-03-28T13:18:43.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.067+0000 7f825a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0051c0 0x7f825415c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:43.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.068+0000 7f825a200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82540b6b70 0x7f82540b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:43.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.068+0000 7f825a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82540b3f80 0x7f82540b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:43.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.068+0000 7f825a200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f825406b9c0 con 0x7f82540b3f80 2024-03-28T13:18:43.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.068+0000 7f825a200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f82540d0580 con 0x7f82540b6b70 2024-03-28T13:18:43.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.068+0000 7f825a200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f825406bb40 con 0x7f823c0051c0 2024-03-28T13:18:43.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.068+0000 7f8259800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82540b3f80 0x7f82540b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:43.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.068+0000 7f8259800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82540b3f80 0x7f82540b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54282/0 (socket says 172.21.15.19:54282) 2024-03-28T13:18:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.068+0000 7f8259800640 1 -- 172.21.15.19:0/1172055183 learned_addr learned my addr 172.21.15.19:0/1172055183 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.068+0000 7f8258e00640 1 --2- 172.21.15.19:0/1172055183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0051c0 0x7f825415c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.068+0000 7f8259800640 1 -- 172.21.15.19:0/1172055183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0051c0 msgr2=0x7f825415c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.068+0000 7f8259800640 1 --2- 172.21.15.19:0/1172055183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0051c0 0x7f825415c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.068+0000 7f8259800640 1 -- 172.21.15.19:0/1172055183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82540b6b70 msgr2=0x7f82540b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.068+0000 7f8259800640 1 --2- 172.21.15.19:0/1172055183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82540b6b70 0x7f82540b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.068+0000 7f8259800640 1 -- 172.21.15.19:0/1172055183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82540b1620 con 0x7f82540b3f80 2024-03-28T13:18:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.069+0000 7f8259800640 1 --2- 172.21.15.19:0/1172055183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82540b3f80 0x7f82540b4410 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f824c02c6d0 tx=0x7f824c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.069+0000 7f8250c00640 1 -- 172.21.15.19:0/1172055183 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c042020 con 0x7f82540b3f80 2024-03-28T13:18:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.069+0000 7f8250c00640 1 -- 172.21.15.19:0/1172055183 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f824c02ce50 con 0x7f82540b3f80 2024-03-28T13:18:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.069+0000 7f825a200640 1 -- 172.21.15.19:0/1172055183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82540b1900 con 0x7f82540b3f80 2024-03-28T13:18:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.069+0000 7f8250c00640 1 -- 172.21.15.19:0/1172055183 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c009c10 con 0x7f82540b3f80 2024-03-28T13:18:43.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.069+0000 7f825a200640 1 -- 172.21.15.19:0/1172055183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82540b1ec0 con 0x7f82540b3f80 2024-03-28T13:18:43.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.071+0000 7f825a200640 1 -- 172.21.15.19:0/1172055183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8254075940 con 0x7f82540b3f80 2024-03-28T13:18:43.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.071+0000 7f8250c00640 1 -- 172.21.15.19:0/1172055183 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f824c045020 con 0x7f82540b3f80 2024-03-28T13:18:43.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.072+0000 7f8250c00640 1 --2- 172.21.15.19:0/1172055183 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f823406c5b0 0x7f823406ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:43.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.072+0000 7f8250c00640 1 -- 172.21.15.19:0/1172055183 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f824c0ae200 con 0x7f82540b3f80 2024-03-28T13:18:43.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.073+0000 7f8258e00640 1 --2- 172.21.15.19:0/1172055183 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f823406c5b0 0x7f823406ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:43.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.074+0000 7f8258e00640 1 --2- 172.21.15.19:0/1172055183 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f823406c5b0 0x7f823406ea70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8248001200 tx=0x7f8248000f30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.077+0000 7f8250c00640 1 -- 172.21.15.19:0/1172055183 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f824c04c020 con 0x7f82540b3f80 2024-03-28T13:18:43.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.316+0000 7f825a200640 1 -- 172.21.15.19:0/1172055183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f825406de30 con 0x7f82540b3f80 2024-03-28T13:18:43.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.317+0000 7f8250c00640 1 -- 172.21.15.19:0/1172055183 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5188 (secure 0 0 0) 0x7f824c07c8e0 con 0x7f82540b3f80 2024-03-28T13:18:43.317 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:43.317 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1144703580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1144703580},{"type":"v1","addr":"172.21.15.188:6836","nonce":1144703580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:16:25.995386+0000","modified":"2024-03-28T13:17:59.710623+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6040":{"gid":6040,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":28,"addr":"172.21.15.19:6834/2209799371","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2209799371},{"type":"v1","addr":"172.21.15.19:6834","nonce":2209799371}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6040,"qdb_cluster":[6040]},"id":6}]} 2024-03-28T13:18:43.317 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 74 2024-03-28T13:18:43.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.320+0000 7f825a200640 1 -- 172.21.15.19:0/1172055183 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f823406c5b0 msgr2=0x7f823406ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:43.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.320+0000 7f825a200640 1 --2- 172.21.15.19:0/1172055183 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f823406c5b0 0x7f823406ea70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8248001200 tx=0x7f8248000f30 comp rx=0 tx=0).stop 2024-03-28T13:18:43.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.320+0000 7f825a200640 1 -- 172.21.15.19:0/1172055183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82540b3f80 msgr2=0x7f82540b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:43.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.320+0000 7f825a200640 1 --2- 172.21.15.19:0/1172055183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82540b3f80 0x7f82540b4410 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f824c02c6d0 tx=0x7f824c009750 comp rx=0 tx=0).stop 2024-03-28T13:18:43.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.320+0000 7f825a200640 1 -- 172.21.15.19:0/1172055183 shutdown_connections 2024-03-28T13:18:43.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.320+0000 7f825a200640 1 --2- 172.21.15.19:0/1172055183 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f823406c5b0 0x7f823406ea70 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.320+0000 7f825a200640 1 --2- 172.21.15.19:0/1172055183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f82540b3f80 0x7f82540b4410 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.320+0000 7f825a200640 1 --2- 172.21.15.19:0/1172055183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82540b6b70 0x7f82540b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.320+0000 7f825a200640 1 --2- 172.21.15.19:0/1172055183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f823c0051c0 0x7f825415c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.320+0000 7f825a200640 1 -- 172.21.15.19:0/1172055183 >> 172.21.15.19:0/1172055183 conn(0x7f825409aa20 msgr2=0x7f825406e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:43.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.321+0000 7f825a200640 1 -- 172.21.15.19:0/1172055183 shutdown_connections 2024-03-28T13:18:43.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.321+0000 7f825a200640 1 -- 172.21.15.19:0/1172055183 wait complete. 2024-03-28T13:18:43.337 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:18:43.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.551+0000 7fc3a2c00640 1 Processor -- start 2024-03-28T13:18:43.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.551+0000 7fc3a2c00640 1 -- start start 2024-03-28T13:18:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.553+0000 7fc3a2c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc39c0d1990 0x7fc39c0b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.553+0000 7fc3a2c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc39c0c7810 0x7fc39c0cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.553+0000 7fc3a2c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc39c162ed0 con 0x7fc39c162af0 2024-03-28T13:18:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.553+0000 7fc3a2c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc39c06d990 con 0x7fc39c0d1990 2024-03-28T13:18:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.553+0000 7fc3a2c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc39c06cd80 con 0x7fc39c0c7810 2024-03-28T13:18:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.553+0000 7fc39b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc39c0d1990 0x7fc39c0b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.553+0000 7fc39b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc39c0d1990 0x7fc39c0b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54294/0 (socket says 172.21.15.19:54294) 2024-03-28T13:18:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.553+0000 7fc39b400640 1 -- 172.21.15.19:0/3393010834 learned_addr learned my addr 172.21.15.19:0/3393010834 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:43.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.553+0000 7fc3a0c00640 1 --2- 172.21.15.19:0/3393010834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc39c0c7810 0x7fc39c0cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:43.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.554+0000 7fc39b400640 1 -- 172.21.15.19:0/3393010834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc39c0c7810 msgr2=0x7fc39c0cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:43.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.554+0000 7fc39b400640 1 --2- 172.21.15.19:0/3393010834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc39c0c7810 0x7fc39c0cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.554+0000 7fc39b400640 1 -- 172.21.15.19:0/3393010834 >> v1:172.21.15.188:6789/0 conn(0x7fc39c162af0 legacy=0x7fc39c0c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:43.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.554+0000 7fc39b400640 1 -- 172.21.15.19:0/3393010834 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39c06d060 con 0x7fc39c0d1990 2024-03-28T13:18:43.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.556+0000 7fc39b400640 1 --2- 172.21.15.19:0/3393010834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc39c0d1990 0x7fc39c0b4980 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc388009870 tx=0x7fc38802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14b8df6697ff666 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:43.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.556+0000 7fc39aa00640 1 -- 172.21.15.19:0/3393010834 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388034070 con 0x7fc39c0d1990 2024-03-28T13:18:43.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.556+0000 7fc39aa00640 1 -- 172.21.15.19:0/3393010834 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc388030840 con 0x7fc39c0d1990 2024-03-28T13:18:43.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.556+0000 7fc39aa00640 1 -- 172.21.15.19:0/3393010834 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc388030c10 con 0x7fc39c0d1990 2024-03-28T13:18:43.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.556+0000 7fc3a2c00640 1 -- 172.21.15.19:0/3393010834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc39c0d1990 msgr2=0x7fc39c0b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:43.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.556+0000 7fc3a2c00640 1 --2- 172.21.15.19:0/3393010834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc39c0d1990 0x7fc39c0b4980 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc388009870 tx=0x7fc38802b980 comp rx=0 tx=0).stop 2024-03-28T13:18:43.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.557+0000 7fc3a2c00640 1 -- 172.21.15.19:0/3393010834 shutdown_connections 2024-03-28T13:18:43.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.557+0000 7fc3a2c00640 1 --2- 172.21.15.19:0/3393010834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc39c0d1990 0x7fc39c0b4980 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.557+0000 7fc3a2c00640 1 --2- 172.21.15.19:0/3393010834 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc39c0c7810 0x7fc39c0cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.557+0000 7fc3a2c00640 1 -- 172.21.15.19:0/3393010834 >> 172.21.15.19:0/3393010834 conn(0x7fc39c0b3e90 msgr2=0x7fc39c0b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:43.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.557+0000 7fc3a2c00640 1 -- 172.21.15.19:0/3393010834 shutdown_connections 2024-03-28T13:18:43.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.557+0000 7fc3a2c00640 1 -- 172.21.15.19:0/3393010834 wait complete. 2024-03-28T13:18:43.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.560+0000 7fc3a2c00640 1 Processor -- start 2024-03-28T13:18:43.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.560+0000 7fc3a2c00640 1 -- start start 2024-03-28T13:18:43.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.563+0000 7fc3a2c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc39c0c7810 0x7fc39c1444c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:43.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.563+0000 7fc3a2c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc39c0d1990 0x7fc39c160a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:43.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.563+0000 7fc3a2c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc39c162af0 0x7fc39c160f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:43.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.563+0000 7fc3a2c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c06ca30 con 0x7fc39c0c7810 2024-03-28T13:18:43.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.563+0000 7fc3a2c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc39c06d620 con 0x7fc39c162af0 2024-03-28T13:18:43.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.563+0000 7fc3a2c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc39c0d0370 con 0x7fc39c0d1990 2024-03-28T13:18:43.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.563+0000 7fc39be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc39c0c7810 0x7fc39c1444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:43.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.563+0000 7fc39be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc39c0c7810 0x7fc39c1444c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54306/0 (socket says 172.21.15.19:54306) 2024-03-28T13:18:43.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.563+0000 7fc39be00640 1 -- 172.21.15.19:0/2414093440 learned_addr learned my addr 172.21.15.19:0/2414093440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:43.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.563+0000 7fc39b400640 1 --2- 172.21.15.19:0/2414093440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc39c0d1990 0x7fc39c160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.563+0000 7fc3a0c00640 1 --2- 172.21.15.19:0/2414093440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc39c162af0 0x7fc39c160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.564+0000 7fc39be00640 1 -- 172.21.15.19:0/2414093440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc39c0d1990 msgr2=0x7fc39c160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.564+0000 7fc39be00640 1 --2- 172.21.15.19:0/2414093440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc39c0d1990 0x7fc39c160a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.564+0000 7fc39be00640 1 -- 172.21.15.19:0/2414093440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc39c162af0 msgr2=0x7fc39c160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.564+0000 7fc39be00640 1 --2- 172.21.15.19:0/2414093440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc39c162af0 0x7fc39c160f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.564+0000 7fc39be00640 1 -- 172.21.15.19:0/2414093440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc39c163320 con 0x7fc39c0c7810 2024-03-28T13:18:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.564+0000 7fc39be00640 1 --2- 172.21.15.19:0/2414093440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc39c0c7810 0x7fc39c1444c0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc38c00c370 tx=0x7fc38c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.564+0000 7fc393e00640 1 -- 172.21.15.19:0/2414093440 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c017070 con 0x7fc39c0c7810 2024-03-28T13:18:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.564+0000 7fc393e00640 1 -- 172.21.15.19:0/2414093440 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc38c0052d0 con 0x7fc39c0c7810 2024-03-28T13:18:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.564+0000 7fc393e00640 1 -- 172.21.15.19:0/2414093440 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc38c003850 con 0x7fc39c0c7810 2024-03-28T13:18:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.564+0000 7fc3a2c00640 1 -- 172.21.15.19:0/2414093440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc39c163600 con 0x7fc39c0c7810 2024-03-28T13:18:43.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.564+0000 7fc3a2c00640 1 -- 172.21.15.19:0/2414093440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc39c1640e0 con 0x7fc39c0c7810 2024-03-28T13:18:43.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.565+0000 7fc3a2c00640 1 -- 172.21.15.19:0/2414093440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc39c068cd0 con 0x7fc39c0c7810 2024-03-28T13:18:43.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.567+0000 7fc393e00640 1 -- 172.21.15.19:0/2414093440 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc38c01a070 con 0x7fc39c0c7810 2024-03-28T13:18:43.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.567+0000 7fc393e00640 1 --2- 172.21.15.19:0/2414093440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc37c06c530 0x7fc37c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:43.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.568+0000 7fc393e00640 1 -- 172.21.15.19:0/2414093440 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7fc38c08a970 con 0x7fc39c0c7810 2024-03-28T13:18:43.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.568+0000 7fc39b400640 1 --2- 172.21.15.19:0/2414093440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc37c06c530 0x7fc37c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:43.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.569+0000 7fc39b400640 1 --2- 172.21.15.19:0/2414093440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc37c06c530 0x7fc37c06e9f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc388036000 tx=0x7fc38803b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:43.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.576+0000 7fc393e00640 1 -- 172.21.15.19:0/2414093440 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc38c058ed0 con 0x7fc39c0c7810 2024-03-28T13:18:43.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.945+0000 7fc3a2c00640 1 -- 172.21.15.19:0/2414093440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc39c0b1b50 con 0x7fc39c0c7810 2024-03-28T13:18:43.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.947+0000 7fc393e00640 1 -- 172.21.15.19:0/2414093440 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+16935 (secure 0 0 0) 0x7fc38c0587d0 con 0x7fc39c0c7810 2024-03-28T13:18:43.947 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:43.947 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":94,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:17:55.195667+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:16:23.901120+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-28T13:16:24.990904+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6836/1749969053":"2024-03-29T13:17:55.195632+0000","172.21.15.188:6834/1749969053":"2024-03-29T13:17:55.195632+0000","172.21.15.188:6836/2260087679":"2024-03-29T13:16:39.492982+0000","172.21.15.188:6834/2260087679":"2024-03-29T13:16:39.492982+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:18:43.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.950+0000 7fc3a2c00640 1 -- 172.21.15.19:0/2414093440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc37c06c530 msgr2=0x7fc37c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:43.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.950+0000 7fc3a2c00640 1 --2- 172.21.15.19:0/2414093440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc37c06c530 0x7fc37c06e9f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc388036000 tx=0x7fc38803b040 comp rx=0 tx=0).stop 2024-03-28T13:18:43.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.950+0000 7fc3a2c00640 1 -- 172.21.15.19:0/2414093440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc39c0c7810 msgr2=0x7fc39c1444c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:43.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.950+0000 7fc3a2c00640 1 --2- 172.21.15.19:0/2414093440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc39c0c7810 0x7fc39c1444c0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc38c00c370 tx=0x7fc38c00c830 comp rx=0 tx=0).stop 2024-03-28T13:18:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.950+0000 7fc3a2c00640 1 -- 172.21.15.19:0/2414093440 shutdown_connections 2024-03-28T13:18:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.950+0000 7fc3a2c00640 1 --2- 172.21.15.19:0/2414093440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc37c06c530 0x7fc37c06e9f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.950+0000 7fc3a2c00640 1 --2- 172.21.15.19:0/2414093440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc39c162af0 0x7fc39c160f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.950+0000 7fc3a2c00640 1 --2- 172.21.15.19:0/2414093440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc39c0d1990 0x7fc39c160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.950+0000 7fc3a2c00640 1 --2- 172.21.15.19:0/2414093440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc39c0c7810 0x7fc39c1444c0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.951+0000 7fc3a2c00640 1 -- 172.21.15.19:0/2414093440 >> 172.21.15.19:0/2414093440 conn(0x7fc39c0b3e90 msgr2=0x7fc39c0b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.951+0000 7fc3a2c00640 1 -- 172.21.15.19:0/2414093440 shutdown_connections 2024-03-28T13:18:43.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:43.951+0000 7fc3a2c00640 1 -- 172.21.15.19:0/2414093440 wait complete. 2024-03-28T13:18:43.978 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T13:18:44.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.195+0000 7fc9e7e00640 1 Processor -- start 2024-03-28T13:18:44.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.195+0000 7fc9e7e00640 1 -- start start 2024-03-28T13:18:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.197+0000 7fc9e7e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc9e815e440 0x7fc9e80b9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.197+0000 7fc9e7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9e80cef00 0x7fc9e80cf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.197+0000 7fc9e7e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e80d2d80 con 0x7fc9e80d3e20 2024-03-28T13:18:44.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.197+0000 7fc9e7e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc9e80cf8b0 con 0x7fc9e815e440 2024-03-28T13:18:44.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.197+0000 7fc9e7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e80b8300 con 0x7fc9e80cef00 2024-03-28T13:18:44.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.198+0000 7fc9e6a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc9e815e440 0x7fc9e80b9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:44.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.198+0000 7fc9e7400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9e80cef00 0x7fc9e80cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:44.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.198+0000 7fc9e6a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc9e815e440 0x7fc9e80b9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:52492/0 (socket says 172.21.15.19:52492) 2024-03-28T13:18:44.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.198+0000 7fc9e6a00640 1 -- 172.21.15.19:0/2467417903 learned_addr learned my addr 172.21.15.19:0/2467417903 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:44.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.198+0000 7fc9e6a00640 1 -- 172.21.15.19:0/2467417903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9e80cef00 msgr2=0x7fc9e80cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:44.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.199+0000 7fc9e6a00640 1 --2- 172.21.15.19:0/2467417903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9e80cef00 0x7fc9e80cf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:44.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.199+0000 7fc9e6a00640 1 -- 172.21.15.19:0/2467417903 >> v1:172.21.15.188:6789/0 conn(0x7fc9e80d3e20 legacy=0x7fc9e80d2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:44.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.199+0000 7fc9e5600640 1 -- 172.21.15.19:0/2467417903 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2369794446 0 0) 0x7fc9e80d2d80 con 0x7fc9e80d3e20 2024-03-28T13:18:44.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.199+0000 7fc9e6a00640 1 -- 172.21.15.19:0/2467417903 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e80d3060 con 0x7fc9e815e440 2024-03-28T13:18:44.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.199+0000 7fc9e7400640 1 --2- 172.21.15.19:0/2467417903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9e80cef00 0x7fc9e80cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:18:44.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.200+0000 7fc9e6a00640 1 --2- 172.21.15.19:0/2467417903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e815e440 0x7fc9e80b9e00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009870 tx=0x7fc9d802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b933cf338c040a8c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:44.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.201+0000 7fc9e5600640 1 -- 172.21.15.19:0/2467417903 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8034070 con 0x7fc9e815e440 2024-03-28T13:18:44.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.201+0000 7fc9e5600640 1 -- 172.21.15.19:0/2467417903 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d8030840 con 0x7fc9e815e440 2024-03-28T13:18:44.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.201+0000 7fc9e5600640 1 -- 172.21.15.19:0/2467417903 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8030c10 con 0x7fc9e815e440 2024-03-28T13:18:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.201+0000 7fc9e7e00640 1 -- 172.21.15.19:0/2467417903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e815e440 msgr2=0x7fc9e80b9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.201+0000 7fc9e7e00640 1 --2- 172.21.15.19:0/2467417903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e815e440 0x7fc9e80b9e00 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009870 tx=0x7fc9d802b980 comp rx=0 tx=0).stop 2024-03-28T13:18:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.201+0000 7fc9e7e00640 1 -- 172.21.15.19:0/2467417903 shutdown_connections 2024-03-28T13:18:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.201+0000 7fc9e7e00640 1 --2- 172.21.15.19:0/2467417903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9e80cef00 0x7fc9e80cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.201+0000 7fc9e7e00640 1 --2- 172.21.15.19:0/2467417903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e815e440 0x7fc9e80b9e00 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.201+0000 7fc9e7e00640 1 -- 172.21.15.19:0/2467417903 >> 172.21.15.19:0/2467417903 conn(0x7fc9e80678b0 msgr2=0x7fc9e80d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.202+0000 7fc9e7e00640 1 -- 172.21.15.19:0/2467417903 shutdown_connections 2024-03-28T13:18:44.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.202+0000 7fc9e7e00640 1 -- 172.21.15.19:0/2467417903 wait complete. 2024-03-28T13:18:44.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.203+0000 7fc9e7e00640 1 Processor -- start 2024-03-28T13:18:44.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.204+0000 7fc9e7e00640 1 -- start start 2024-03-28T13:18:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.204+0000 7fc9e7e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e80cef00 0x7fc9e81e8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9e80d3e20 0x7fc9e81e37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9e815e440 0x7fc9e814b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e80d3900 con 0x7fc9e815e440 2024-03-28T13:18:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e7e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e80d3620 con 0x7fc9e80cef00 2024-03-28T13:18:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e80b72f0 con 0x7fc9e80d3e20 2024-03-28T13:18:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9e815e440 0x7fc9e814b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:44.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9e815e440 0x7fc9e814b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54312/0 (socket says 172.21.15.19:54312) 2024-03-28T13:18:44.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e7400640 1 -- 172.21.15.19:0/314076590 learned_addr learned my addr 172.21.15.19:0/314076590 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:44.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e6a00640 1 --2- 172.21.15.19:0/314076590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e80cef00 0x7fc9e81e8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:44.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e6000640 1 --2- 172.21.15.19:0/314076590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9e80d3e20 0x7fc9e81e37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:44.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e7400640 1 -- 172.21.15.19:0/314076590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9e80d3e20 msgr2=0x7fc9e81e37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:44.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e7400640 1 --2- 172.21.15.19:0/314076590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9e80d3e20 0x7fc9e81e37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:44.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e7400640 1 -- 172.21.15.19:0/314076590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e80cef00 msgr2=0x7fc9e81e8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:44.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e7400640 1 --2- 172.21.15.19:0/314076590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e80cef00 0x7fc9e81e8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:44.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e7400640 1 -- 172.21.15.19:0/314076590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e80cd310 con 0x7fc9e815e440 2024-03-28T13:18:44.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.205+0000 7fc9e6a00640 1 --2- 172.21.15.19:0/314076590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e80cef00 0x7fc9e81e8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:44.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.206+0000 7fc9e7400640 1 --2- 172.21.15.19:0/314076590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9e815e440 0x7fc9e814b760 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000efc0 tx=0x7fc9e0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:44.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.206+0000 7fc9dea00640 1 -- 172.21.15.19:0/314076590 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0018030 con 0x7fc9e815e440 2024-03-28T13:18:44.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.206+0000 7fc9dea00640 1 -- 172.21.15.19:0/314076590 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e000cde0 con 0x7fc9e815e440 2024-03-28T13:18:44.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.206+0000 7fc9dea00640 1 -- 172.21.15.19:0/314076590 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0003bb0 con 0x7fc9e815e440 2024-03-28T13:18:44.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.206+0000 7fc9e7e00640 1 -- 172.21.15.19:0/314076590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e80cd5f0 con 0x7fc9e815e440 2024-03-28T13:18:44.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.206+0000 7fc9e7e00640 1 -- 172.21.15.19:0/314076590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e80cdbb0 con 0x7fc9e815e440 2024-03-28T13:18:44.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.207+0000 7fc9e7e00640 1 -- 172.21.15.19:0/314076590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e80d1d50 con 0x7fc9e815e440 2024-03-28T13:18:44.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.208+0000 7fc9dea00640 1 -- 172.21.15.19:0/314076590 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc9e001b070 con 0x7fc9e815e440 2024-03-28T13:18:44.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.209+0000 7fc9dea00640 1 --2- 172.21.15.19:0/314076590 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc9c806c530 0x7fc9c806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:44.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.210+0000 7fc9dea00640 1 -- 172.21.15.19:0/314076590 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7fc9e008c150 con 0x7fc9e815e440 2024-03-28T13:18:44.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.210+0000 7fc9e6a00640 1 --2- 172.21.15.19:0/314076590 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc9c806c530 0x7fc9c806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:44.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.211+0000 7fc9e6a00640 1 --2- 172.21.15.19:0/314076590 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc9c806c530 0x7fc9c806e9f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8036000 tx=0x7fc9d8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:44.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.218+0000 7fc9dea00640 1 -- 172.21.15.19:0/314076590 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc9e80d1d50 con 0x7fc9e815e440 2024-03-28T13:18:44.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.467+0000 7fc9e7e00640 1 -- 172.21.15.19:0/314076590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc9e8075940 con 0x7fc9e815e440 2024-03-28T13:18:44.468 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:18:44.469+0000 7f6b74400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T13:18:44.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.472+0000 7fc9dea00640 1 -- 172.21.15.19:0/314076590 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) ==== 161+0+0 (secure 0 0 0) 0x7fc9e0010040 con 0x7fc9e815e440 2024-03-28T13:18:44.472 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T13:18:44.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.475+0000 7fc9e7e00640 1 -- 172.21.15.19:0/314076590 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc9c806c530 msgr2=0x7fc9c806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:44.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.475+0000 7fc9e7e00640 1 --2- 172.21.15.19:0/314076590 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc9c806c530 0x7fc9c806e9f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8036000 tx=0x7fc9d8008030 comp rx=0 tx=0).stop 2024-03-28T13:18:44.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.475+0000 7fc9e7e00640 1 -- 172.21.15.19:0/314076590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9e815e440 msgr2=0x7fc9e814b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:44.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.475+0000 7fc9e7e00640 1 --2- 172.21.15.19:0/314076590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9e815e440 0x7fc9e814b760 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000efc0 tx=0x7fc9e0009f90 comp rx=0 tx=0).stop 2024-03-28T13:18:44.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.475+0000 7fc9e7e00640 1 -- 172.21.15.19:0/314076590 shutdown_connections 2024-03-28T13:18:44.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.475+0000 7fc9e7e00640 1 --2- 172.21.15.19:0/314076590 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc9c806c530 0x7fc9c806e9f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:44.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.475+0000 7fc9e7e00640 1 --2- 172.21.15.19:0/314076590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9e815e440 0x7fc9e814b760 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:44.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.475+0000 7fc9e7e00640 1 --2- 172.21.15.19:0/314076590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9e80d3e20 0x7fc9e81e37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:44.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.475+0000 7fc9e7e00640 1 --2- 172.21.15.19:0/314076590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9e80cef00 0x7fc9e81e8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:44.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.475+0000 7fc9e7e00640 1 -- 172.21.15.19:0/314076590 >> 172.21.15.19:0/314076590 conn(0x7fc9e80678b0 msgr2=0x7fc9e80d4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:44.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.476+0000 7fc9e7e00640 1 -- 172.21.15.19:0/314076590 shutdown_connections 2024-03-28T13:18:44.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.476+0000 7fc9e7e00640 1 -- 172.21.15.19:0/314076590 wait complete. 2024-03-28T13:18:44.497 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-28T13:18:44.502 INFO:tasks.ceph.mds.a.smithi019.stderr: -2614> 2024-03-28T13:18:00.488+0000 7f87b4a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:18:44.503 INFO:tasks.ceph.mds.a.smithi019.stderr: -2040> 2024-03-28T13:18:03.827+0000 7f87afa00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | dirtyparent=1 authpin=1 scrubqueue=0 0x5578cd576680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-28T13:18:44.573 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-03-28T13:18:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.699+0000 7fd1df000640 1 Processor -- start 2024-03-28T13:18:44.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.700+0000 7fd1df000640 1 -- start start 2024-03-28T13:18:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.701+0000 7fd1df000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd1d80d0280 0x7fd1d80d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.701+0000 7fd1df000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd1d8141c10 0x7fd1d8071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:44.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.701+0000 7fd1df000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1d815ec80 con 0x7fd1d8071eb0 2024-03-28T13:18:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.702+0000 7fd1df000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d80d0ba0 con 0x7fd1d80d0280 2024-03-28T13:18:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.702+0000 7fd1df000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fd1d815ef20 con 0x7fd1d8141c10 2024-03-28T13:18:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.702+0000 7fd1d7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd1d80d0280 0x7fd1d80d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.702+0000 7fd1d7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd1d80d0280 0x7fd1d80d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54340/0 (socket says 172.21.15.19:54340) 2024-03-28T13:18:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.702+0000 7fd1d7e00640 1 -- 172.21.15.19:0/778838263 learned_addr learned my addr 172.21.15.19:0/778838263 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:44.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.702+0000 7fd1d7e00640 1 -- 172.21.15.19:0/778838263 >> v2:172.21.15.188:3300/0 conn(0x7fd1d8141c10 msgr2=0x7fd1d8071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:44.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.702+0000 7fd1d7e00640 1 --2- 172.21.15.19:0/778838263 >> v2:172.21.15.188:3300/0 conn(0x7fd1d8141c10 0x7fd1d8071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:44.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.702+0000 7fd1d7e00640 1 -- 172.21.15.19:0/778838263 >> v1:172.21.15.188:6789/0 conn(0x7fd1d8071eb0 legacy=0x7fd1d806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:44.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.702+0000 7fd1d7e00640 1 -- 172.21.15.19:0/778838263 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d815f200 con 0x7fd1d80d0280 2024-03-28T13:18:44.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.704+0000 7fd1d7e00640 1 --2- 172.21.15.19:0/778838263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d80d0280 0x7fd1d80d0660 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80b5270 tx=0x7fd1c802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e194b1f84d6f1e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:44.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.704+0000 7fd1d7400640 1 -- 172.21.15.19:0/778838263 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8035020 con 0x7fd1d80d0280 2024-03-28T13:18:44.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.704+0000 7fd1d7400640 1 -- 172.21.15.19:0/778838263 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1c8031890 con 0x7fd1d80d0280 2024-03-28T13:18:44.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.704+0000 7fd1d7400640 1 -- 172.21.15.19:0/778838263 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8031c60 con 0x7fd1d80d0280 2024-03-28T13:18:44.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.704+0000 7fd1df000640 1 -- 172.21.15.19:0/778838263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d80d0280 msgr2=0x7fd1d80d0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:44.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.704+0000 7fd1df000640 1 --2- 172.21.15.19:0/778838263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d80d0280 0x7fd1d80d0660 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80b5270 tx=0x7fd1c802b930 comp rx=0 tx=0).stop 2024-03-28T13:18:44.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.705+0000 7fd1df000640 1 -- 172.21.15.19:0/778838263 shutdown_connections 2024-03-28T13:18:44.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.705+0000 7fd1df000640 1 --2- 172.21.15.19:0/778838263 >> v2:172.21.15.188:3300/0 conn(0x7fd1d8141c10 0x7fd1d8071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:44.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.705+0000 7fd1df000640 1 --2- 172.21.15.19:0/778838263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d80d0280 0x7fd1d80d0660 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:44.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.705+0000 7fd1df000640 1 -- 172.21.15.19:0/778838263 >> 172.21.15.19:0/778838263 conn(0x7fd1d806b750 msgr2=0x7fd1d806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:44.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.705+0000 7fd1df000640 1 -- 172.21.15.19:0/778838263 shutdown_connections 2024-03-28T13:18:44.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.705+0000 7fd1df000640 1 -- 172.21.15.19:0/778838263 wait complete. 2024-03-28T13:18:44.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.706+0000 7fd1df000640 1 Processor -- start 2024-03-28T13:18:44.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.707+0000 7fd1df000640 1 -- start start 2024-03-28T13:18:44.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1df000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d8071eb0 0x7fd1d8160cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:44.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1df000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1d80d0280 0x7fd1d80ae330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:44.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1df000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d8141c10 0x7fd1d80ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:44.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1df000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d8158e90 con 0x7fd1d8141c10 2024-03-28T13:18:44.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1df000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd1d815f7c0 con 0x7fd1d8071eb0 2024-03-28T13:18:44.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1df000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd1d8099770 con 0x7fd1d80d0280 2024-03-28T13:18:44.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1dd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d8141c10 0x7fd1d80ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:44.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1dd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d8141c10 0x7fd1d80ad010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54344/0 (socket says 172.21.15.19:54344) 2024-03-28T13:18:44.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1dd000640 1 -- 172.21.15.19:0/157755265 learned_addr learned my addr 172.21.15.19:0/157755265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:44.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1dd000640 1 -- 172.21.15.19:0/157755265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1d80d0280 msgr2=0x7fd1d80ae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:44.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1cfe00640 1 --2- 172.21.15.19:0/157755265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1d80d0280 0x7fd1d80ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:44.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1dd000640 1 --2- 172.21.15.19:0/157755265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1d80d0280 0x7fd1d80ae330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:44.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1dd000640 1 -- 172.21.15.19:0/157755265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d8071eb0 msgr2=0x7fd1d8160cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:44.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1d7e00640 1 --2- 172.21.15.19:0/157755265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d8071eb0 0x7fd1d8160cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:44.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1dd000640 1 --2- 172.21.15.19:0/157755265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d8071eb0 0x7fd1d8160cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:44.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1dd000640 1 -- 172.21.15.19:0/157755265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1d8098b60 con 0x7fd1d8141c10 2024-03-28T13:18:44.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.708+0000 7fd1d7e00640 1 --2- 172.21.15.19:0/157755265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d8071eb0 0x7fd1d8160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:44.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.709+0000 7fd1dd000640 1 --2- 172.21.15.19:0/157755265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d8141c10 0x7fd1d80ad010 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000c780 tx=0x7fd1d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:44.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.709+0000 7fd1d4c00640 1 -- 172.21.15.19:0/157755265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d0017070 con 0x7fd1d8141c10 2024-03-28T13:18:44.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.709+0000 7fd1d4c00640 1 -- 172.21.15.19:0/157755265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1d0003da0 con 0x7fd1d8141c10 2024-03-28T13:18:44.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.709+0000 7fd1d4c00640 1 -- 172.21.15.19:0/157755265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1d00052b0 con 0x7fd1d8141c10 2024-03-28T13:18:44.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.709+0000 7fd1df000640 1 -- 172.21.15.19:0/157755265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1d8098e40 con 0x7fd1d8141c10 2024-03-28T13:18:44.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.709+0000 7fd1df000640 1 -- 172.21.15.19:0/157755265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1d8099400 con 0x7fd1d8141c10 2024-03-28T13:18:44.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.711+0000 7fd1df000640 1 -- 172.21.15.19:0/157755265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1d80685c0 con 0x7fd1d8141c10 2024-03-28T13:18:44.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.712+0000 7fd1d4c00640 1 -- 172.21.15.19:0/157755265 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd1d0005450 con 0x7fd1d8141c10 2024-03-28T13:18:44.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.713+0000 7fd1d4c00640 1 --2- 172.21.15.19:0/157755265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd1b406c560 0x7fd1b406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:44.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.713+0000 7fd1d4c00640 1 -- 172.21.15.19:0/157755265 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5967+0+0 (secure 0 0 0) 0x7fd1d008adb0 con 0x7fd1d8141c10 2024-03-28T13:18:44.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.713+0000 7fd1d7e00640 1 --2- 172.21.15.19:0/157755265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd1b406c560 0x7fd1b406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:44.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.715+0000 7fd1d7e00640 1 --2- 172.21.15.19:0/157755265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd1b406c560 0x7fd1b406ea20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8036000 tx=0x7fd1c803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:44.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.718+0000 7fd1d4c00640 1 -- 172.21.15.19:0/157755265 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd1d0059240 con 0x7fd1d8141c10 2024-03-28T13:18:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:44.968+0000 7fd1df000640 1 -- 172.21.15.19:0/157755265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd1d80988a0 con 0x7fd1d8141c10 2024-03-28T13:18:45.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.476+0000 7fd1d4c00640 1 -- 172.21.15.19:0/157755265 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) ==== 104+0+0 (secure 0 0 0) 0x7fd1d0058b40 con 0x7fd1d8141c10 2024-03-28T13:18:45.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.480+0000 7fd1df000640 1 -- 172.21.15.19:0/157755265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd1b406c560 msgr2=0x7fd1b406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:45.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.480+0000 7fd1df000640 1 --2- 172.21.15.19:0/157755265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd1b406c560 0x7fd1b406ea20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8036000 tx=0x7fd1c803c040 comp rx=0 tx=0).stop 2024-03-28T13:18:45.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.480+0000 7fd1df000640 1 -- 172.21.15.19:0/157755265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d8141c10 msgr2=0x7fd1d80ad010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:45.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.480+0000 7fd1df000640 1 --2- 172.21.15.19:0/157755265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d8141c10 0x7fd1d80ad010 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fd1d000c780 tx=0x7fd1d000cc40 comp rx=0 tx=0).stop 2024-03-28T13:18:45.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.480+0000 7fd1df000640 1 -- 172.21.15.19:0/157755265 shutdown_connections 2024-03-28T13:18:45.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.480+0000 7fd1df000640 1 --2- 172.21.15.19:0/157755265 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd1b406c560 0x7fd1b406ea20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:45.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.480+0000 7fd1df000640 1 --2- 172.21.15.19:0/157755265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1d8141c10 0x7fd1d80ad010 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:45.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.480+0000 7fd1df000640 1 --2- 172.21.15.19:0/157755265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1d80d0280 0x7fd1d80ae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:45.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.480+0000 7fd1df000640 1 --2- 172.21.15.19:0/157755265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1d8071eb0 0x7fd1d8160cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:45.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.480+0000 7fd1df000640 1 -- 172.21.15.19:0/157755265 >> 172.21.15.19:0/157755265 conn(0x7fd1d806b750 msgr2=0x7fd1d80b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:45.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.480+0000 7fd1df000640 1 -- 172.21.15.19:0/157755265 shutdown_connections 2024-03-28T13:18:45.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.481+0000 7fd1df000640 1 -- 172.21.15.19:0/157755265 wait complete. 2024-03-28T13:18:45.503 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-28T13:18:45.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.776+0000 7f7e6e400640 1 Processor -- start 2024-03-28T13:18:45.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.776+0000 7f7e6e400640 1 -- start start 2024-03-28T13:18:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.777+0000 7f7e6e400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7e6809dcf0 0x7f7e681453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:45.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.777+0000 7f7e6e400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7e68071710 0x7f7e68071b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.778+0000 7f7e6e400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e6815cb90 con 0x7f7e680acf10 2024-03-28T13:18:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.778+0000 7f7e6e400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7e68145f60 con 0x7f7e6809dcf0 2024-03-28T13:18:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.778+0000 7f7e6e400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7e6815c950 con 0x7f7e68071710 2024-03-28T13:18:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.778+0000 7f7e67400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7e680acf10 0x7f7e680ad2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35592/0 (socket says 172.21.15.19:35592) 2024-03-28T13:18:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.778+0000 7f7e67400640 1 -- 172.21.15.19:0/3092531526 learned_addr learned my addr 172.21.15.19:0/3092531526 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:45.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.778+0000 7f7e66a00640 1 --2- 172.21.15.19:0/3092531526 >> v2:172.21.15.19:3300/0 conn(0x7f7e6809dcf0 0x7f7e681453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.778+0000 7f7e67e00640 1 --2- 172.21.15.19:0/3092531526 >> v2:172.21.15.188:3300/0 conn(0x7f7e68071710 0x7f7e68071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.778+0000 7f7e66000640 1 -- 172.21.15.19:0/3092531526 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 595063514 0 0) 0x7f7e6815cb90 con 0x7f7e680acf10 2024-03-28T13:18:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.779+0000 7f7e66000640 1 -- 172.21.15.19:0/3092531526 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e4c003520 con 0x7f7e680acf10 2024-03-28T13:18:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.779+0000 7f7e66a00640 1 -- 172.21.15.19:0/3092531526 >> v2:172.21.15.188:3300/0 conn(0x7f7e68071710 msgr2=0x7f7e68071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.779+0000 7f7e66a00640 1 --2- 172.21.15.19:0/3092531526 >> v2:172.21.15.188:3300/0 conn(0x7f7e68071710 0x7f7e68071b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.779+0000 7f7e66a00640 1 -- 172.21.15.19:0/3092531526 >> v1:172.21.15.19:6789/0 conn(0x7f7e680acf10 legacy=0x7f7e680ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:45.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.779+0000 7f7e66a00640 1 -- 172.21.15.19:0/3092531526 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6815ce70 con 0x7f7e6809dcf0 2024-03-28T13:18:45.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.780+0000 7f7e66a00640 1 --2- 172.21.15.19:0/3092531526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6809dcf0 0x7f7e681453f0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f7e6815feb0 tx=0x7f7e5402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c978aa4ffc0b8a46 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:45.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.780+0000 7f7e66000640 1 -- 172.21.15.19:0/3092531526 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e540315a0 con 0x7f7e6809dcf0 2024-03-28T13:18:45.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.780+0000 7f7e66000640 1 -- 172.21.15.19:0/3092531526 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e54037070 con 0x7f7e6809dcf0 2024-03-28T13:18:45.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.780+0000 7f7e66000640 1 -- 172.21.15.19:0/3092531526 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e54031960 con 0x7f7e6809dcf0 2024-03-28T13:18:45.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.781+0000 7f7e6e400640 1 -- 172.21.15.19:0/3092531526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6809dcf0 msgr2=0x7f7e681453f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:45.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.781+0000 7f7e6e400640 1 --2- 172.21.15.19:0/3092531526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6809dcf0 0x7f7e681453f0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f7e6815feb0 tx=0x7f7e5402b840 comp rx=0 tx=0).stop 2024-03-28T13:18:45.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.781+0000 7f7e6e400640 1 -- 172.21.15.19:0/3092531526 shutdown_connections 2024-03-28T13:18:45.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.781+0000 7f7e6e400640 1 --2- 172.21.15.19:0/3092531526 >> v2:172.21.15.188:3300/0 conn(0x7f7e68071710 0x7f7e68071b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:45.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.781+0000 7f7e6e400640 1 --2- 172.21.15.19:0/3092531526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6809dcf0 0x7f7e681453f0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:45.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.781+0000 7f7e6e400640 1 -- 172.21.15.19:0/3092531526 >> 172.21.15.19:0/3092531526 conn(0x7f7e6806bba0 msgr2=0x7f7e680b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:45.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.781+0000 7f7e6e400640 1 -- 172.21.15.19:0/3092531526 shutdown_connections 2024-03-28T13:18:45.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.781+0000 7f7e6e400640 1 -- 172.21.15.19:0/3092531526 wait complete. 2024-03-28T13:18:45.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.783+0000 7f7e6e400640 1 Processor -- start 2024-03-28T13:18:45.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.783+0000 7f7e6e400640 1 -- start start 2024-03-28T13:18:45.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.784+0000 7f7e6e400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e68071710 0x7f7e68160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:45.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.784+0000 7f7e6e400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6809dcf0 0x7f7e68141590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:45.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.784+0000 7f7e6e400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e680acf10 0x7f7e68141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:45.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.784+0000 7f7e6e400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6815d430 con 0x7f7e6809dcf0 2024-03-28T13:18:45.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.784+0000 7f7e6e400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6815d7a0 con 0x7f7e680acf10 2024-03-28T13:18:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.784+0000 7f7e6e400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7e68145930 con 0x7f7e68071710 2024-03-28T13:18:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.784+0000 7f7e66a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6809dcf0 0x7f7e68141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.784+0000 7f7e66a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6809dcf0 0x7f7e68141590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60960/0 (socket says 172.21.15.19:60960) 2024-03-28T13:18:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.784+0000 7f7e66a00640 1 -- 172.21.15.19:0/1792529970 learned_addr learned my addr 172.21.15.19:0/1792529970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.785+0000 7f7e67400640 1 --2- 172.21.15.19:0/1792529970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e68071710 0x7f7e68160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.785+0000 7f7e67e00640 1 --2- 172.21.15.19:0/1792529970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e680acf10 0x7f7e68141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.785+0000 7f7e66a00640 1 -- 172.21.15.19:0/1792529970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e68071710 msgr2=0x7f7e68160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.785+0000 7f7e66a00640 1 --2- 172.21.15.19:0/1792529970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e68071710 0x7f7e68160430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.785+0000 7f7e66a00640 1 -- 172.21.15.19:0/1792529970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e680acf10 msgr2=0x7f7e68141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.785+0000 7f7e66a00640 1 --2- 172.21.15.19:0/1792529970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e680acf10 0x7f7e68141ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.785+0000 7f7e66a00640 1 -- 172.21.15.19:0/1792529970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e681435e0 con 0x7f7e6809dcf0 2024-03-28T13:18:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.785+0000 7f7e67e00640 1 --2- 172.21.15.19:0/1792529970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e680acf10 0x7f7e68141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:45.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.785+0000 7f7e66a00640 1 --2- 172.21.15.19:0/1792529970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6809dcf0 0x7f7e68141590 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009890 tx=0x7f7e540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:45.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.785+0000 7f7e5f400640 1 -- 172.21.15.19:0/1792529970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e54032630 con 0x7f7e6809dcf0 2024-03-28T13:18:45.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.786+0000 7f7e5f400640 1 -- 172.21.15.19:0/1792529970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e540327f0 con 0x7f7e6809dcf0 2024-03-28T13:18:45.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.786+0000 7f7e5f400640 1 -- 172.21.15.19:0/1792529970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e54032be0 con 0x7f7e6809dcf0 2024-03-28T13:18:45.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.786+0000 7f7e6e400640 1 -- 172.21.15.19:0/1792529970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e681438c0 con 0x7f7e6809dcf0 2024-03-28T13:18:45.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.786+0000 7f7e6e400640 1 -- 172.21.15.19:0/1792529970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e68143e80 con 0x7f7e6809dcf0 2024-03-28T13:18:45.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.787+0000 7f7e6e400640 1 -- 172.21.15.19:0/1792529970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e680685c0 con 0x7f7e6809dcf0 2024-03-28T13:18:45.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.788+0000 7f7e5f400640 1 -- 172.21.15.19:0/1792529970 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7e5404c020 con 0x7f7e6809dcf0 2024-03-28T13:18:45.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.789+0000 7f7e5f400640 1 --2- 172.21.15.19:0/1792529970 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e4806c530 0x7f7e4806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:45.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.789+0000 7f7e5f400640 1 -- 172.21.15.19:0/1792529970 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5967+0+0 (secure 0 0 0) 0x7f7e5402f020 con 0x7f7e6809dcf0 2024-03-28T13:18:45.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.789+0000 7f7e67400640 1 --2- 172.21.15.19:0/1792529970 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e4806c530 0x7f7e4806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:45.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.791+0000 7f7e67400640 1 --2- 172.21.15.19:0/1792529970 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e4806c530 0x7f7e4806e9f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7e58007fd0 tx=0x7f7e58009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:45.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.796+0000 7f7e5f400640 1 -- 172.21.15.19:0/1792529970 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7e54040070 con 0x7f7e6809dcf0 2024-03-28T13:18:45.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:45.999+0000 7f7e6e400640 1 -- 172.21.15.19:0/1792529970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7e680b3030 con 0x7f7e6809dcf0 2024-03-28T13:18:46.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:46.937+0000 7f7e5f400640 1 -- 172.21.15.19:0/1792529970 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) ==== 181+0+0 (secure 0 0 0) 0x7f7e5407b9d0 con 0x7f7e6809dcf0 2024-03-28T13:18:46.937 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-03-28T13:18:46.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:46.942+0000 7f7e6e400640 1 -- 172.21.15.19:0/1792529970 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e4806c530 msgr2=0x7f7e4806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:46.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:46.942+0000 7f7e6e400640 1 --2- 172.21.15.19:0/1792529970 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e4806c530 0x7f7e4806e9f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7e58007fd0 tx=0x7f7e58009040 comp rx=0 tx=0).stop 2024-03-28T13:18:46.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:46.942+0000 7f7e6e400640 1 -- 172.21.15.19:0/1792529970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6809dcf0 msgr2=0x7f7e68141590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:46.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:46.942+0000 7f7e6e400640 1 --2- 172.21.15.19:0/1792529970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6809dcf0 0x7f7e68141590 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009890 tx=0x7f7e540046c0 comp rx=0 tx=0).stop 2024-03-28T13:18:46.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:46.943+0000 7f7e6e400640 1 -- 172.21.15.19:0/1792529970 shutdown_connections 2024-03-28T13:18:46.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:46.943+0000 7f7e6e400640 1 --2- 172.21.15.19:0/1792529970 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7e4806c530 0x7f7e4806e9f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:46.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:46.943+0000 7f7e6e400640 1 --2- 172.21.15.19:0/1792529970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e680acf10 0x7f7e68141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:46.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:46.943+0000 7f7e6e400640 1 --2- 172.21.15.19:0/1792529970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e6809dcf0 0x7f7e68141590 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:46.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:46.943+0000 7f7e6e400640 1 --2- 172.21.15.19:0/1792529970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e68071710 0x7f7e68160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:46.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:46.943+0000 7f7e6e400640 1 -- 172.21.15.19:0/1792529970 >> 172.21.15.19:0/1792529970 conn(0x7f7e6806bba0 msgr2=0x7f7e6815ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:46.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:46.944+0000 7f7e6e400640 1 -- 172.21.15.19:0/1792529970 shutdown_connections 2024-03-28T13:18:46.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:46.944+0000 7f7e6e400640 1 -- 172.21.15.19:0/1792529970 wait complete. 2024-03-28T13:18:46.973 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-28T13:18:47.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.189+0000 7fded7e00640 1 Processor -- start 2024-03-28T13:18:47.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.190+0000 7fded7e00640 1 -- start start 2024-03-28T13:18:47.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.191+0000 7fded7e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fded8141d20 0x7fded806e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:47.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.191+0000 7fded7e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fded80cfbf0 con 0x7fded813f190 2024-03-28T13:18:47.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.192+0000 7fded7e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fded80cf350 con 0x7fded8141890 2024-03-28T13:18:47.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.192+0000 7fded7e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fded813f570 con 0x7fded8141d20 2024-03-28T13:18:47.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.192+0000 7fded6a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fded813f190 0x7fded80ce800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35598/0 (socket says 172.21.15.19:35598) 2024-03-28T13:18:47.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.192+0000 7fded6a00640 1 -- 172.21.15.19:0/4055915631 learned_addr learned my addr 172.21.15.19:0/4055915631 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.192+0000 7fded6000640 1 -- 172.21.15.19:0/4055915631 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 405219542 0 0) 0x7fded80cfbf0 con 0x7fded813f190 2024-03-28T13:18:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.192+0000 7fded7400640 1 --2- 172.21.15.19:0/4055915631 >> v2:172.21.15.188:3300/0 conn(0x7fded8141d20 0x7fded806e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.192+0000 7fded6000640 1 -- 172.21.15.19:0/4055915631 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdebc003520 con 0x7fded813f190 2024-03-28T13:18:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.193+0000 7fded6000640 1 -- 172.21.15.19:0/4055915631 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3209558565 0 0) 0x7fdebc003520 con 0x7fded813f190 2024-03-28T13:18:47.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.193+0000 7fded6000640 1 -- 172.21.15.19:0/4055915631 >> v2:172.21.15.188:3300/0 conn(0x7fded8141d20 msgr2=0x7fded806e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.193+0000 7fded6000640 1 --2- 172.21.15.19:0/4055915631 >> v2:172.21.15.188:3300/0 conn(0x7fded8141d20 0x7fded806e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.193+0000 7fded6000640 1 -- 172.21.15.19:0/4055915631 >> v1:172.21.15.188:6789/0 conn(0x7fded8141890 legacy=0x7fded806db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.193+0000 7fded6000640 1 -- 172.21.15.19:0/4055915631 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded80cf630 con 0x7fded813f190 2024-03-28T13:18:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.193+0000 7fded6000640 1 -- 172.21.15.19:0/4055915631 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fdec8001670 con 0x7fded813f190 2024-03-28T13:18:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.193+0000 7fded7400640 1 --2- 172.21.15.19:0/4055915631 >> v2:172.21.15.188:3300/0 conn(0x7fded8141d20 0x7fded806e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:18:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.193+0000 7fded6000640 1 -- 172.21.15.19:0/4055915631 >> v1:172.21.15.19:6789/0 conn(0x7fded813f190 legacy=0x7fded80ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.193+0000 7fded6000640 1 --2- 172.21.15.19:0/4055915631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdebc003a00 0x7fdebc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.193+0000 7fded6000640 1 --2- 172.21.15.19:0/4055915631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdebc004800 0x7fdebc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.193+0000 7fded6000640 1 --2- 172.21.15.19:0/4055915631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdebc005210 0x7fdebc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.193+0000 7fded6000640 1 -- 172.21.15.19:0/4055915631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc00c130 con 0x7fdebc003a00 2024-03-28T13:18:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.193+0000 7fded6000640 1 -- 172.21.15.19:0/4055915631 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdebc00c2e0 con 0x7fdebc004800 2024-03-28T13:18:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.193+0000 7fded6000640 1 -- 172.21.15.19:0/4055915631 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdebc00c4e0 con 0x7fdebc005210 2024-03-28T13:18:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.193+0000 7fded6a00640 1 --2- 172.21.15.19:0/4055915631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdebc003a00 0x7fdebc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:47.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.194+0000 7fded7400640 1 --2- 172.21.15.19:0/4055915631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdebc005210 0x7fdebc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.194+0000 7fded6a00640 1 -- 172.21.15.19:0/4055915631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdebc005210 msgr2=0x7fdebc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.194+0000 7fded6a00640 1 --2- 172.21.15.19:0/4055915631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdebc005210 0x7fdebc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.194+0000 7fded6a00640 1 -- 172.21.15.19:0/4055915631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdebc004800 msgr2=0x7fdebc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.194+0000 7fded6a00640 1 --2- 172.21.15.19:0/4055915631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdebc004800 0x7fdebc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.194+0000 7fded6a00640 1 -- 172.21.15.19:0/4055915631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdebc003520 con 0x7fdebc003a00 2024-03-28T13:18:47.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.194+0000 7fded7400640 1 -- 172.21.15.19:0/4055915631 reap_dead start 2024-03-28T13:18:47.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.195+0000 7fded6a00640 1 --2- 172.21.15.19:0/4055915631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdebc003a00 0x7fdebc003e20 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fdec800c190 tx=0x7fdebc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413e490dbb6491a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.195+0000 7fded6000640 1 -- 172.21.15.19:0/4055915631 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec8007c60 con 0x7fdebc003a00 2024-03-28T13:18:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.195+0000 7fded6000640 1 -- 172.21.15.19:0/4055915631 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec8030070 con 0x7fdebc003a00 2024-03-28T13:18:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.195+0000 7fded6000640 1 -- 172.21.15.19:0/4055915631 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec8005d90 con 0x7fdebc003a00 2024-03-28T13:18:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.195+0000 7fded7e00640 1 -- 172.21.15.19:0/4055915631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdebc003a00 msgr2=0x7fdebc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.195+0000 7fded7e00640 1 --2- 172.21.15.19:0/4055915631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdebc003a00 0x7fdebc003e20 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fdec800c190 tx=0x7fdebc005830 comp rx=0 tx=0).stop 2024-03-28T13:18:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.196+0000 7fded7e00640 1 -- 172.21.15.19:0/4055915631 shutdown_connections 2024-03-28T13:18:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.196+0000 7fded7e00640 1 --2- 172.21.15.19:0/4055915631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdebc003a00 0x7fdebc003e20 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.196+0000 7fded7e00640 1 -- 172.21.15.19:0/4055915631 >> 172.21.15.19:0/4055915631 conn(0x7fded80b1d80 msgr2=0x7fded80b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.196+0000 7fded7e00640 1 -- 172.21.15.19:0/4055915631 shutdown_connections 2024-03-28T13:18:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.196+0000 7fded7e00640 1 -- 172.21.15.19:0/4055915631 wait complete. 2024-03-28T13:18:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.198+0000 7fded7e00640 1 Processor -- start 2024-03-28T13:18:47.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.198+0000 7fded7e00640 1 -- start start 2024-03-28T13:18:47.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.199+0000 7fded7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdebc003a00 0x7fded81f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.199+0000 7fded7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fded81f3ee0 0x7fded80ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.199+0000 7fded7e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fded80b4450 0x7fded80b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.199+0000 7fded7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fded8142130 con 0x7fded81f3ee0 2024-03-28T13:18:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.199+0000 7fded7e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fded806d670 con 0x7fded80b4450 2024-03-28T13:18:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.199+0000 7fded7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fded8142550 con 0x7fdebc003a00 2024-03-28T13:18:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.199+0000 7fdecfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fded81f3ee0 0x7fded80ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.199+0000 7fdecfe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fded81f3ee0 0x7fded80ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60984/0 (socket says 172.21.15.19:60984) 2024-03-28T13:18:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.199+0000 7fdecfe00640 1 -- 172.21.15.19:0/4139445869 learned_addr learned my addr 172.21.15.19:0/4139445869 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.199+0000 7fded6a00640 1 --2- 172.21.15.19:0/4139445869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdebc003a00 0x7fded81f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.199+0000 7fded7400640 1 --2- 172.21.15.19:0/4139445869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fded80b4450 0x7fded80b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.200+0000 7fdecfe00640 1 -- 172.21.15.19:0/4139445869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdebc003a00 msgr2=0x7fded81f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:47.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.200+0000 7fdecfe00640 1 --2- 172.21.15.19:0/4139445869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdebc003a00 0x7fded81f4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.200+0000 7fdecfe00640 1 -- 172.21.15.19:0/4139445869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fded80b4450 msgr2=0x7fded80b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.200+0000 7fdecfe00640 1 --2- 172.21.15.19:0/4139445869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fded80b4450 0x7fded80b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.200+0000 7fdecfe00640 1 -- 172.21.15.19:0/4139445869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded81f4d50 con 0x7fded81f3ee0 2024-03-28T13:18:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.200+0000 7fded6a00640 1 --2- 172.21.15.19:0/4139445869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdebc003a00 0x7fded81f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:18:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.200+0000 7fdecfe00640 1 --2- 172.21.15.19:0/4139445869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fded81f3ee0 0x7fded80ad890 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fdec000c460 tx=0x7fdec000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.200+0000 7fdecf400640 1 -- 172.21.15.19:0/4139445869 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec00052d0 con 0x7fded81f3ee0 2024-03-28T13:18:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.200+0000 7fdecf400640 1 -- 172.21.15.19:0/4139445869 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec0005490 con 0x7fded81f3ee0 2024-03-28T13:18:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.200+0000 7fdecf400640 1 -- 172.21.15.19:0/4139445869 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0012870 con 0x7fded81f3ee0 2024-03-28T13:18:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.200+0000 7fded7e00640 1 -- 172.21.15.19:0/4139445869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded80ae200 con 0x7fded81f3ee0 2024-03-28T13:18:47.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.200+0000 7fded7e00640 1 -- 172.21.15.19:0/4139445869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded80ae7c0 con 0x7fded81f3ee0 2024-03-28T13:18:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.202+0000 7fded7e00640 1 -- 172.21.15.19:0/4139445869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded8075940 con 0x7fded81f3ee0 2024-03-28T13:18:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.203+0000 7fdecf400640 1 -- 172.21.15.19:0/4139445869 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdec0012a10 con 0x7fded81f3ee0 2024-03-28T13:18:47.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.204+0000 7fdecf400640 1 --2- 172.21.15.19:0/4139445869 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdeb406c560 0x7fdeb406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.204+0000 7fdecf400640 1 -- 172.21.15.19:0/4139445869 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5529+0+0 (secure 0 0 0) 0x7fdec008b1e0 con 0x7fded81f3ee0 2024-03-28T13:18:47.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.205+0000 7fded6a00640 1 --2- 172.21.15.19:0/4139445869 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdeb406c560 0x7fdeb406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:47.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.206+0000 7fded6a00640 1 --2- 172.21.15.19:0/4139445869 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdeb406c560 0x7fdeb406ea20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdec800c160 tx=0x7fdec8039040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:47.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.209+0000 7fdecf400640 1 -- 172.21.15.19:0/4139445869 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fded81f4d50 con 0x7fded81f3ee0 2024-03-28T13:18:47.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.418+0000 7fded7e00640 1 -- 172.21.15.19:0/4139445869 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fded80b3710 con 0x7fded81f3ee0 2024-03-28T13:18:47.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.940+0000 7fdecf400640 1 -- 172.21.15.19:0/4139445869 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) ==== 169+0+0 (secure 0 0 0) 0x7fdec00598b0 con 0x7fded81f3ee0 2024-03-28T13:18:47.941 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-03-28T13:18:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.944+0000 7fded7e00640 1 -- 172.21.15.19:0/4139445869 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdeb406c560 msgr2=0x7fdeb406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.944+0000 7fded7e00640 1 --2- 172.21.15.19:0/4139445869 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdeb406c560 0x7fdeb406ea20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdec800c160 tx=0x7fdec8039040 comp rx=0 tx=0).stop 2024-03-28T13:18:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.944+0000 7fded7e00640 1 -- 172.21.15.19:0/4139445869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fded81f3ee0 msgr2=0x7fded80ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:47.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.944+0000 7fded7e00640 1 --2- 172.21.15.19:0/4139445869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fded81f3ee0 0x7fded80ad890 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fdec000c460 tx=0x7fdec000c920 comp rx=0 tx=0).stop 2024-03-28T13:18:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.944+0000 7fded7e00640 1 -- 172.21.15.19:0/4139445869 shutdown_connections 2024-03-28T13:18:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.944+0000 7fded7e00640 1 --2- 172.21.15.19:0/4139445869 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdeb406c560 0x7fdeb406ea20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.944+0000 7fded7e00640 1 --2- 172.21.15.19:0/4139445869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fded80b4450 0x7fded80b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.944+0000 7fded7e00640 1 --2- 172.21.15.19:0/4139445869 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fded81f3ee0 0x7fded80ad890 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.944+0000 7fded7e00640 1 --2- 172.21.15.19:0/4139445869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdebc003a00 0x7fded81f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.944+0000 7fded7e00640 1 -- 172.21.15.19:0/4139445869 >> 172.21.15.19:0/4139445869 conn(0x7fded80b1d80 msgr2=0x7fded815a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.944+0000 7fded7e00640 1 -- 172.21.15.19:0/4139445869 shutdown_connections 2024-03-28T13:18:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:47.945+0000 7fded7e00640 1 -- 172.21.15.19:0/4139445869 wait complete. 2024-03-28T13:18:47.973 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-28T13:18:48.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.185+0000 7fa0ca600640 1 Processor -- start 2024-03-28T13:18:48.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.185+0000 7fa0ca600640 1 -- start start 2024-03-28T13:18:48.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.186+0000 7fa0ca600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa0c40d0280 0x7fa0c40d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.186+0000 7fa0ca600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0c4141c10 0x7fa0c4071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.186+0000 7fa0ca600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0c415ec80 con 0x7fa0c4071eb0 2024-03-28T13:18:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.186+0000 7fa0ca600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa0c40d0ba0 con 0x7fa0c4141c10 2024-03-28T13:18:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.186+0000 7fa0ca600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa0c415ef20 con 0x7fa0c40d0280 2024-03-28T13:18:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.187+0000 7fa0c2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0c4141c10 0x7fa0c4071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.187+0000 7fa0c2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa0c4141c10 0x7fa0c4071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32768/0 (socket says 172.21.15.19:32768) 2024-03-28T13:18:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.187+0000 7fa0c2a00640 1 -- 172.21.15.19:0/3518825336 learned_addr learned my addr 172.21.15.19:0/3518825336 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.187+0000 7fa0c3e00640 1 --1- 172.21.15.19:0/3518825336 >> v1:172.21.15.19:6789/0 conn(0x7fa0c4071eb0 0x7fa0c406d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35602/0 (socket says 172.21.15.19:35602) 2024-03-28T13:18:48.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.187+0000 7fa0c3400640 1 --2- 172.21.15.19:0/3518825336 >> v2:172.21.15.188:3300/0 conn(0x7fa0c40d0280 0x7fa0c40d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:48.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.187+0000 7fa0c2a00640 1 -- 172.21.15.19:0/3518825336 >> v2:172.21.15.188:3300/0 conn(0x7fa0c40d0280 msgr2=0x7fa0c40d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:48.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.187+0000 7fa0c2a00640 1 --2- 172.21.15.19:0/3518825336 >> v2:172.21.15.188:3300/0 conn(0x7fa0c40d0280 0x7fa0c40d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:48.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.187+0000 7fa0c2a00640 1 -- 172.21.15.19:0/3518825336 >> v1:172.21.15.19:6789/0 conn(0x7fa0c4071eb0 legacy=0x7fa0c406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:48.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.187+0000 7fa0c2a00640 1 -- 172.21.15.19:0/3518825336 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c415f200 con 0x7fa0c4141c10 2024-03-28T13:18:48.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.189+0000 7fa0c2a00640 1 --2- 172.21.15.19:0/3518825336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c4141c10 0x7fa0c4071970 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40d1050 tx=0x7fa0b002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6768e1bc900e4fb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:48.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.189+0000 7fa0c2000640 1 -- 172.21.15.19:0/3518825336 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0034070 con 0x7fa0c4141c10 2024-03-28T13:18:48.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.189+0000 7fa0c2000640 1 -- 172.21.15.19:0/3518825336 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0b0030860 con 0x7fa0c4141c10 2024-03-28T13:18:48.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.189+0000 7fa0c2000640 1 -- 172.21.15.19:0/3518825336 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0030c30 con 0x7fa0c4141c10 2024-03-28T13:18:48.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.190+0000 7fa0ca600640 1 -- 172.21.15.19:0/3518825336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c4141c10 msgr2=0x7fa0c4071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:48.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.190+0000 7fa0ca600640 1 --2- 172.21.15.19:0/3518825336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c4141c10 0x7fa0c4071970 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40d1050 tx=0x7fa0b002b9a0 comp rx=0 tx=0).stop 2024-03-28T13:18:48.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.190+0000 7fa0ca600640 1 -- 172.21.15.19:0/3518825336 shutdown_connections 2024-03-28T13:18:48.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.190+0000 7fa0ca600640 1 --2- 172.21.15.19:0/3518825336 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c4141c10 0x7fa0c4071970 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:48.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.190+0000 7fa0ca600640 1 --2- 172.21.15.19:0/3518825336 >> v2:172.21.15.188:3300/0 conn(0x7fa0c40d0280 0x7fa0c40d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:48.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.190+0000 7fa0ca600640 1 -- 172.21.15.19:0/3518825336 >> 172.21.15.19:0/3518825336 conn(0x7fa0c406b750 msgr2=0x7fa0c406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:48.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.190+0000 7fa0ca600640 1 -- 172.21.15.19:0/3518825336 shutdown_connections 2024-03-28T13:18:48.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.190+0000 7fa0ca600640 1 -- 172.21.15.19:0/3518825336 wait complete. 2024-03-28T13:18:48.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.192+0000 7fa0ca600640 1 Processor -- start 2024-03-28T13:18:48.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.192+0000 7fa0ca600640 1 -- start start 2024-03-28T13:18:48.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.193+0000 7fa0ca600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c4071eb0 0x7fa0c40ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:48.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.193+0000 7fa0ca600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40d0280 0x7fa0c4099460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.193+0000 7fa0ca600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c4141c10 0x7fa0c40999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.193+0000 7fa0ca600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c4158e90 con 0x7fa0c40d0280 2024-03-28T13:18:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.193+0000 7fa0ca600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c415f7c0 con 0x7fa0c4141c10 2024-03-28T13:18:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.193+0000 7fa0ca600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa0c41e9730 con 0x7fa0c4071eb0 2024-03-28T13:18:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.193+0000 7fa0c2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40d0280 0x7fa0c4099460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.193+0000 7fa0c2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40d0280 0x7fa0c4099460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32772/0 (socket says 172.21.15.19:32772) 2024-03-28T13:18:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.193+0000 7fa0c2a00640 1 -- 172.21.15.19:0/3150549214 learned_addr learned my addr 172.21.15.19:0/3150549214 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.194+0000 7fa0c3400640 1 --2- 172.21.15.19:0/3150549214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c4071eb0 0x7fa0c40ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.194+0000 7fa0c3e00640 1 --2- 172.21.15.19:0/3150549214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c4141c10 0x7fa0c40999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.194+0000 7fa0c2a00640 1 -- 172.21.15.19:0/3150549214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c4071eb0 msgr2=0x7fa0c40ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.194+0000 7fa0c2a00640 1 --2- 172.21.15.19:0/3150549214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c4071eb0 0x7fa0c40ae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.194+0000 7fa0c2a00640 1 -- 172.21.15.19:0/3150549214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c4141c10 msgr2=0x7fa0c40999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.194+0000 7fa0c2a00640 1 --2- 172.21.15.19:0/3150549214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c4141c10 0x7fa0c40999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:48.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.194+0000 7fa0c2a00640 1 -- 172.21.15.19:0/3150549214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c41e9450 con 0x7fa0c40d0280 2024-03-28T13:18:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.194+0000 7fa0c3e00640 1 --2- 172.21.15.19:0/3150549214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c4141c10 0x7fa0c40999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.194+0000 7fa0c2a00640 1 --2- 172.21.15.19:0/3150549214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40d0280 0x7fa0c4099460 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fa0b002b9e0 tx=0x7fa0b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:48.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.194+0000 7fa0bb400640 1 -- 172.21.15.19:0/3150549214 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0034040 con 0x7fa0c40d0280 2024-03-28T13:18:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.194+0000 7fa0bb400640 1 -- 172.21.15.19:0/3150549214 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0b0039d90 con 0x7fa0c40d0280 2024-03-28T13:18:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.194+0000 7fa0bb400640 1 -- 172.21.15.19:0/3150549214 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0043610 con 0x7fa0c40d0280 2024-03-28T13:18:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.195+0000 7fa0ca600640 1 -- 172.21.15.19:0/3150549214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c40adb50 con 0x7fa0c40d0280 2024-03-28T13:18:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.195+0000 7fa0ca600640 1 -- 172.21.15.19:0/3150549214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c40ae110 con 0x7fa0c40d0280 2024-03-28T13:18:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.196+0000 7fa0ca600640 1 -- 172.21.15.19:0/3150549214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c40685c0 con 0x7fa0c40d0280 2024-03-28T13:18:48.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.197+0000 7fa0bb400640 1 -- 172.21.15.19:0/3150549214 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa0b004a060 con 0x7fa0c40d0280 2024-03-28T13:18:48.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.198+0000 7fa0bb400640 1 --2- 172.21.15.19:0/3150549214 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0a406c560 0x7fa0a406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:48.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.198+0000 7fa0bb400640 1 -- 172.21.15.19:0/3150549214 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5115+0+0 (secure 0 0 0) 0x7fa0b00ad530 con 0x7fa0c40d0280 2024-03-28T13:18:48.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.198+0000 7fa0c3400640 1 --2- 172.21.15.19:0/3150549214 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0a406c560 0x7fa0a406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:48.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.200+0000 7fa0c3400640 1 --2- 172.21.15.19:0/3150549214 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0a406c560 0x7fa0a406ea20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40b44b0 tx=0x7fa0b400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:48.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.203+0000 7fa0bb400640 1 -- 172.21.15.19:0/3150549214 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa0b003e070 con 0x7fa0c40d0280 2024-03-28T13:18:48.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.408+0000 7fa0ca600640 1 -- 172.21.15.19:0/3150549214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fa0c40cfdb0 con 0x7fa0c40d0280 2024-03-28T13:18:48.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.413+0000 7fa0bb400640 1 -- 172.21.15.19:0/3150549214 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fa0b0037360 con 0x7fa0c40d0280 2024-03-28T13:18:48.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.416+0000 7fa0ca600640 1 -- 172.21.15.19:0/3150549214 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0a406c560 msgr2=0x7fa0a406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:48.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.416+0000 7fa0ca600640 1 --2- 172.21.15.19:0/3150549214 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0a406c560 0x7fa0a406ea20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40b44b0 tx=0x7fa0b400b040 comp rx=0 tx=0).stop 2024-03-28T13:18:48.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.416+0000 7fa0ca600640 1 -- 172.21.15.19:0/3150549214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40d0280 msgr2=0x7fa0c4099460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:48.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.416+0000 7fa0ca600640 1 --2- 172.21.15.19:0/3150549214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40d0280 0x7fa0c4099460 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fa0b002b9e0 tx=0x7fa0b00046c0 comp rx=0 tx=0).stop 2024-03-28T13:18:48.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.417+0000 7fa0ca600640 1 -- 172.21.15.19:0/3150549214 shutdown_connections 2024-03-28T13:18:48.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.417+0000 7fa0ca600640 1 --2- 172.21.15.19:0/3150549214 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0a406c560 0x7fa0a406ea20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:48.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.417+0000 7fa0ca600640 1 --2- 172.21.15.19:0/3150549214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c4141c10 0x7fa0c40999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:48.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.417+0000 7fa0ca600640 1 --2- 172.21.15.19:0/3150549214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c40d0280 0x7fa0c4099460 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:48.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.417+0000 7fa0ca600640 1 --2- 172.21.15.19:0/3150549214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0c4071eb0 0x7fa0c40ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:48.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.417+0000 7fa0ca600640 1 -- 172.21.15.19:0/3150549214 >> 172.21.15.19:0/3150549214 conn(0x7fa0c406b750 msgr2=0x7fa0c4144440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:48.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.417+0000 7fa0ca600640 1 -- 172.21.15.19:0/3150549214 shutdown_connections 2024-03-28T13:18:48.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.417+0000 7fa0ca600640 1 -- 172.21.15.19:0/3150549214 wait complete. 2024-03-28T13:18:48.436 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-03-28T13:18:48.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.655+0000 7fe935e00640 1 Processor -- start 2024-03-28T13:18:48.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.655+0000 7fe935e00640 1 -- start start 2024-03-28T13:18:48.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.657+0000 7fe935e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe9300cfef0 0x7fe9300d0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:48.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.657+0000 7fe935e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe93006df70 con 0x7fe9300cd420 2024-03-28T13:18:48.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.657+0000 7fe935e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe93006e250 con 0x7fe93014c360 2024-03-28T13:18:48.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.657+0000 7fe935e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe9300d0890 con 0x7fe9300cfef0 2024-03-28T13:18:48.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.657+0000 7fe92fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe9300cfef0 0x7fe9300d0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:48.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.657+0000 7fe935400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe9300cd420 0x7fe9300cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35608/0 (socket says 172.21.15.19:35608) 2024-03-28T13:18:48.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.657+0000 7fe935400640 1 -- 172.21.15.19:0/2633837403 learned_addr learned my addr 172.21.15.19:0/2633837403 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:48.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.658+0000 7fe92fe00640 1 -- 172.21.15.19:0/2633837403 >> v1:172.21.15.188:6789/0 conn(0x7fe93014c360 legacy=0x7fe93006d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:48.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.658+0000 7fe92fe00640 1 -- 172.21.15.19:0/2633837403 >> v1:172.21.15.19:6789/0 conn(0x7fe9300cd420 legacy=0x7fe9300cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:48.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.658+0000 7fe92fe00640 1 -- 172.21.15.19:0/2633837403 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9300ccc20 con 0x7fe9300cfef0 2024-03-28T13:18:48.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.659+0000 7fe92fe00640 1 --2- 172.21.15.19:0/2633837403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9300cfef0 0x7fe9300d0350 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0097a0 tx=0x7fe91c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ed336ba1e18acd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.659+0000 7fe92f400640 1 -- 172.21.15.19:0/2633837403 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c035020 con 0x7fe9300cfef0 2024-03-28T13:18:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.659+0000 7fe92f400640 1 -- 172.21.15.19:0/2633837403 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe91c031830 con 0x7fe9300cfef0 2024-03-28T13:18:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.659+0000 7fe92f400640 1 -- 172.21.15.19:0/2633837403 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c031c00 con 0x7fe9300cfef0 2024-03-28T13:18:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.660+0000 7fe935e00640 1 -- 172.21.15.19:0/2633837403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9300cfef0 msgr2=0x7fe9300d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:48.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.660+0000 7fe935e00640 1 --2- 172.21.15.19:0/2633837403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9300cfef0 0x7fe9300d0350 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0097a0 tx=0x7fe91c02b8d0 comp rx=0 tx=0).stop 2024-03-28T13:18:48.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.660+0000 7fe935e00640 1 -- 172.21.15.19:0/2633837403 shutdown_connections 2024-03-28T13:18:48.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.660+0000 7fe935e00640 1 --2- 172.21.15.19:0/2633837403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe9300cfef0 0x7fe9300d0350 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:48.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.660+0000 7fe935e00640 1 -- 172.21.15.19:0/2633837403 >> 172.21.15.19:0/2633837403 conn(0x7fe93006b570 msgr2=0x7fe93006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:48.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.660+0000 7fe935e00640 1 -- 172.21.15.19:0/2633837403 shutdown_connections 2024-03-28T13:18:48.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.660+0000 7fe935e00640 1 -- 172.21.15.19:0/2633837403 wait complete. 2024-03-28T13:18:48.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.661+0000 7fe935e00640 1 Processor -- start 2024-03-28T13:18:48.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.662+0000 7fe935e00640 1 -- start start 2024-03-28T13:18:48.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe935e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9300cd420 0x7fe930149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:48.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe935e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9300cfef0 0x7fe9301e1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:48.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe935e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93014c360 0x7fe9301ec540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:48.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe935e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe9300cc9d0 con 0x7fe93014c360 2024-03-28T13:18:48.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe935e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe93014c740 con 0x7fe9300cd420 2024-03-28T13:18:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe935e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe93006dc00 con 0x7fe9300cfef0 2024-03-28T13:18:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe935400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93014c360 0x7fe9301ec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe935400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93014c360 0x7fe9301ec540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32796/0 (socket says 172.21.15.19:32796) 2024-03-28T13:18:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe935400640 1 -- 172.21.15.19:0/2606846123 learned_addr learned my addr 172.21.15.19:0/2606846123 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe92fe00640 1 --2- 172.21.15.19:0/2606846123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9300cfef0 0x7fe9301e1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe934a00640 1 --2- 172.21.15.19:0/2606846123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9300cd420 0x7fe930149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe935400640 1 -- 172.21.15.19:0/2606846123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9300cfef0 msgr2=0x7fe9301e1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe935400640 1 --2- 172.21.15.19:0/2606846123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9300cfef0 0x7fe9301e1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe935400640 1 -- 172.21.15.19:0/2606846123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9300cd420 msgr2=0x7fe930149d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe935400640 1 --2- 172.21.15.19:0/2606846123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9300cd420 0x7fe930149d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe935400640 1 -- 172.21.15.19:0/2606846123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9301e50c0 con 0x7fe93014c360 2024-03-28T13:18:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.663+0000 7fe934a00640 1 --2- 172.21.15.19:0/2606846123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9300cd420 0x7fe930149d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:48.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.664+0000 7fe935400640 1 --2- 172.21.15.19:0/2606846123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93014c360 0x7fe9301ec540 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fe92800ca80 tx=0x7fe92800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:48.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.664+0000 7fe92cc00640 1 -- 172.21.15.19:0/2606846123 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928017070 con 0x7fe93014c360 2024-03-28T13:18:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.664+0000 7fe92cc00640 1 -- 172.21.15.19:0/2606846123 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92800ac30 con 0x7fe93014c360 2024-03-28T13:18:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.664+0000 7fe92cc00640 1 -- 172.21.15.19:0/2606846123 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928012400 con 0x7fe93014c360 2024-03-28T13:18:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.664+0000 7fe935e00640 1 -- 172.21.15.19:0/2606846123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9301e53a0 con 0x7fe93014c360 2024-03-28T13:18:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.664+0000 7fe935e00640 1 -- 172.21.15.19:0/2606846123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9301e5960 con 0x7fe93014c360 2024-03-28T13:18:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.666+0000 7fe935e00640 1 -- 172.21.15.19:0/2606846123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe930075940 con 0x7fe93014c360 2024-03-28T13:18:48.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.666+0000 7fe92cc00640 1 -- 172.21.15.19:0/2606846123 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe92801a070 con 0x7fe93014c360 2024-03-28T13:18:48.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.667+0000 7fe92cc00640 1 --2- 172.21.15.19:0/2606846123 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe91006c530 0x7fe91006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:48.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.667+0000 7fe92cc00640 1 -- 172.21.15.19:0/2606846123 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5115+0+0 (secure 0 0 0) 0x7fe92808b140 con 0x7fe93014c360 2024-03-28T13:18:48.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.668+0000 7fe934a00640 1 --2- 172.21.15.19:0/2606846123 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe91006c530 0x7fe91006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:48.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.669+0000 7fe934a00640 1 --2- 172.21.15.19:0/2606846123 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe91006c530 0x7fe91006e9f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe920003eb0 tx=0x7fe920009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:48.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.673+0000 7fe92cc00640 1 -- 172.21.15.19:0/2606846123 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe9280592c0 con 0x7fe93014c360 2024-03-28T13:18:48.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:48.965+0000 7fe935e00640 1 -- 172.21.15.19:0/2606846123 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7fe9301ee5c0 con 0x7fe93014c360 2024-03-28T13:18:49.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:49.972+0000 7fe92cc00640 1 -- 172.21.15.19:0/2606846123 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v443) ==== 167+0+0 (secure 0 0 0) 0x7fe92805d5c0 con 0x7fe93014c360 2024-03-28T13:18:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:49.976+0000 7fe935e00640 1 -- 172.21.15.19:0/2606846123 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe91006c530 msgr2=0x7fe91006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:49.976+0000 7fe935e00640 1 --2- 172.21.15.19:0/2606846123 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe91006c530 0x7fe91006e9f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe920003eb0 tx=0x7fe920009040 comp rx=0 tx=0).stop 2024-03-28T13:18:49.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:49.976+0000 7fe935e00640 1 -- 172.21.15.19:0/2606846123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93014c360 msgr2=0x7fe9301ec540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:49.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:49.976+0000 7fe935e00640 1 --2- 172.21.15.19:0/2606846123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93014c360 0x7fe9301ec540 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fe92800ca80 tx=0x7fe92800cf40 comp rx=0 tx=0).stop 2024-03-28T13:18:49.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:49.976+0000 7fe935e00640 1 -- 172.21.15.19:0/2606846123 shutdown_connections 2024-03-28T13:18:49.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:49.976+0000 7fe935e00640 1 --2- 172.21.15.19:0/2606846123 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe91006c530 0x7fe91006e9f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:49.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:49.976+0000 7fe935e00640 1 --2- 172.21.15.19:0/2606846123 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe93014c360 0x7fe9301ec540 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:49.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:49.976+0000 7fe935e00640 1 --2- 172.21.15.19:0/2606846123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9300cfef0 0x7fe9301e1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:49.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:49.976+0000 7fe935e00640 1 --2- 172.21.15.19:0/2606846123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9300cd420 0x7fe930149d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:49.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:49.976+0000 7fe935e00640 1 -- 172.21.15.19:0/2606846123 >> 172.21.15.19:0/2606846123 conn(0x7fe93006b570 msgr2=0x7fe9301e0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:49.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:49.976+0000 7fe935e00640 1 -- 172.21.15.19:0/2606846123 shutdown_connections 2024-03-28T13:18:49.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:49.976+0000 7fe935e00640 1 -- 172.21.15.19:0/2606846123 wait complete. 2024-03-28T13:18:49.999 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-03-28T13:18:49.999 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-28T13:18:50.000 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:18:50.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.217+0000 7ff2b7e00640 1 Processor -- start 2024-03-28T13:18:50.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.217+0000 7ff2b7e00640 1 -- start start 2024-03-28T13:18:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.219+0000 7ff2b7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2b8147a40 0x7ff2b80cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.219+0000 7ff2b7e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff2b80ce380 0x7ff2b806da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.219+0000 7ff2b7e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2b8161f30 con 0x7ff2b80d01c0 2024-03-28T13:18:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.219+0000 7ff2b7e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b8162b00 con 0x7ff2b8147a40 2024-03-28T13:18:50.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.219+0000 7ff2b7e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff2b80ce7e0 con 0x7ff2b80ce380 2024-03-28T13:18:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.220+0000 7ff2b6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2b8147a40 0x7ff2b80cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.220+0000 7ff2b6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff2b8147a40 0x7ff2b80cfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32808/0 (socket says 172.21.15.19:32808) 2024-03-28T13:18:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.220+0000 7ff2b6a00640 1 -- 172.21.15.19:0/1142773108 learned_addr learned my addr 172.21.15.19:0/1142773108 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.220+0000 7ff2b7400640 1 --2- 172.21.15.19:0/1142773108 >> v2:172.21.15.188:3300/0 conn(0x7ff2b80ce380 0x7ff2b806da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.220+0000 7ff2b6a00640 1 -- 172.21.15.19:0/1142773108 >> v2:172.21.15.188:3300/0 conn(0x7ff2b80ce380 msgr2=0x7ff2b806da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.220+0000 7ff2b6a00640 1 --2- 172.21.15.19:0/1142773108 >> v2:172.21.15.188:3300/0 conn(0x7ff2b80ce380 0x7ff2b806da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.220+0000 7ff2b6a00640 1 -- 172.21.15.19:0/1142773108 >> v1:172.21.15.188:6789/0 conn(0x7ff2b80d01c0 legacy=0x7ff2b80cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:50.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.220+0000 7ff2b6a00640 1 -- 172.21.15.19:0/1142773108 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b81621d0 con 0x7ff2b8147a40 2024-03-28T13:18:50.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.221+0000 7ff2b6a00640 1 --2- 172.21.15.19:0/1142773108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2b8147a40 0x7ff2b80cfc80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80d0990 tx=0x7ff2a802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df10ba828a4a682b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:50.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.222+0000 7ff2b6000640 1 -- 172.21.15.19:0/1142773108 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8034070 con 0x7ff2b8147a40 2024-03-28T13:18:50.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.222+0000 7ff2b6000640 1 -- 172.21.15.19:0/1142773108 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2a8030860 con 0x7ff2b8147a40 2024-03-28T13:18:50.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.222+0000 7ff2b6000640 1 -- 172.21.15.19:0/1142773108 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8030c30 con 0x7ff2b8147a40 2024-03-28T13:18:50.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.222+0000 7ff2b7e00640 1 -- 172.21.15.19:0/1142773108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2b8147a40 msgr2=0x7ff2b80cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:50.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.222+0000 7ff2b7e00640 1 --2- 172.21.15.19:0/1142773108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2b8147a40 0x7ff2b80cfc80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80d0990 tx=0x7ff2a802b9a0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.222+0000 7ff2b7e00640 1 -- 172.21.15.19:0/1142773108 shutdown_connections 2024-03-28T13:18:50.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.222+0000 7ff2b7e00640 1 --2- 172.21.15.19:0/1142773108 >> v2:172.21.15.188:3300/0 conn(0x7ff2b80ce380 0x7ff2b806da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.222+0000 7ff2b7e00640 1 --2- 172.21.15.19:0/1142773108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2b8147a40 0x7ff2b80cfc80 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.223+0000 7ff2b7e00640 1 -- 172.21.15.19:0/1142773108 >> 172.21.15.19:0/1142773108 conn(0x7ff2b80ccad0 msgr2=0x7ff2b80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:50.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.223+0000 7ff2b7e00640 1 -- 172.21.15.19:0/1142773108 shutdown_connections 2024-03-28T13:18:50.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.223+0000 7ff2b7e00640 1 -- 172.21.15.19:0/1142773108 wait complete. 2024-03-28T13:18:50.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.224+0000 7ff2b7e00640 1 Processor -- start 2024-03-28T13:18:50.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.224+0000 7ff2b7e00640 1 -- start start 2024-03-28T13:18:50.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.225+0000 7ff2b7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2b80ce380 0x7ff2b8143ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.225+0000 7ff2b7e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2b80d01c0 0x7ff2b8146cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.225+0000 7ff2b7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2b8147a40 0x7ff2b81f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.225+0000 7ff2b7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b80d0620 con 0x7ff2b8147a40 2024-03-28T13:18:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.225+0000 7ff2b7e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b8162790 con 0x7ff2b80d01c0 2024-03-28T13:18:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.225+0000 7ff2b7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff2b8147e20 con 0x7ff2b80ce380 2024-03-28T13:18:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.226+0000 7ff2b7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2b8147a40 0x7ff2b81f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.226+0000 7ff2b7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2b8147a40 0x7ff2b81f76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32818/0 (socket says 172.21.15.19:32818) 2024-03-28T13:18:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.226+0000 7ff2b7400640 1 -- 172.21.15.19:0/3528939086 learned_addr learned my addr 172.21.15.19:0/3528939086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.226+0000 7ff2b6a00640 1 --2- 172.21.15.19:0/3528939086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2b80ce380 0x7ff2b8143ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:50.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.226+0000 7ff2ae000640 1 --2- 172.21.15.19:0/3528939086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2b80d01c0 0x7ff2b8146cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:50.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.226+0000 7ff2b7400640 1 -- 172.21.15.19:0/3528939086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2b80ce380 msgr2=0x7ff2b8143ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:50.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.226+0000 7ff2b7400640 1 --2- 172.21.15.19:0/3528939086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2b80ce380 0x7ff2b8143ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.227+0000 7ff2b7400640 1 -- 172.21.15.19:0/3528939086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2b80d01c0 msgr2=0x7ff2b8146cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:50.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.227+0000 7ff2b7400640 1 --2- 172.21.15.19:0/3528939086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2b80d01c0 0x7ff2b8146cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.227+0000 7ff2b7400640 1 -- 172.21.15.19:0/3528939086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b80b5690 con 0x7ff2b8147a40 2024-03-28T13:18:50.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.227+0000 7ff2ae000640 1 --2- 172.21.15.19:0/3528939086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2b80d01c0 0x7ff2b8146cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:18:50.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.227+0000 7ff2b7400640 1 --2- 172.21.15.19:0/3528939086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2b8147a40 0x7ff2b81f76c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0007e70 tx=0x7ff2b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:50.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.227+0000 7ff2afe00640 1 -- 172.21.15.19:0/3528939086 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0014620 con 0x7ff2b8147a40 2024-03-28T13:18:50.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.227+0000 7ff2afe00640 1 -- 172.21.15.19:0/3528939086 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b0019070 con 0x7ff2b8147a40 2024-03-28T13:18:50.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.227+0000 7ff2afe00640 1 -- 172.21.15.19:0/3528939086 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0014a20 con 0x7ff2b8147a40 2024-03-28T13:18:50.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.227+0000 7ff2b7e00640 1 -- 172.21.15.19:0/3528939086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b80b5910 con 0x7ff2b8147a40 2024-03-28T13:18:50.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.227+0000 7ff2b7e00640 1 -- 172.21.15.19:0/3528939086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b80b07a0 con 0x7ff2b8147a40 2024-03-28T13:18:50.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.228+0000 7ff2b7e00640 1 -- 172.21.15.19:0/3528939086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b8075940 con 0x7ff2b8147a40 2024-03-28T13:18:50.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.229+0000 7ff2afe00640 1 -- 172.21.15.19:0/3528939086 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff2b0014bc0 con 0x7ff2b8147a40 2024-03-28T13:18:50.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.230+0000 7ff2afe00640 1 --2- 172.21.15.19:0/3528939086 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff29406c530 0x7ff29406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:50.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.230+0000 7ff2afe00640 1 -- 172.21.15.19:0/3528939086 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5115+0+0 (secure 0 0 0) 0x7ff2b008bd90 con 0x7ff2b8147a40 2024-03-28T13:18:50.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.231+0000 7ff2b6a00640 1 --2- 172.21.15.19:0/3528939086 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff29406c530 0x7ff29406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:50.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.232+0000 7ff2b6a00640 1 --2- 172.21.15.19:0/3528939086 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff29406c530 0x7ff29406e9f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8036000 tx=0x7ff2a803b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:50.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.235+0000 7ff2afe00640 1 -- 172.21.15.19:0/3528939086 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff2b80b5690 con 0x7ff2b8147a40 2024-03-28T13:18:50.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.437+0000 7ff2b7e00640 1 -- 172.21.15.19:0/3528939086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2b80af6f0 con 0x7ff2b8147a40 2024-03-28T13:18:50.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.439+0000 7ff2afe00640 1 -- 172.21.15.19:0/3528939086 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) ==== 74+0+13285 (secure 0 0 0) 0x7ff2b005a560 con 0x7ff2b8147a40 2024-03-28T13:18:50.439 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:50.439 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":97,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:18:47.937923+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/2209799371":"2024-03-29T13:18:44.469848+0000","172.21.15.19:6832/2209799371":"2024-03-29T13:18:44.469848+0000","172.21.15.188:6836/1749969053":"2024-03-29T13:17:55.195632+0000","172.21.15.188:6834/1749969053":"2024-03-29T13:17:55.195632+0000","172.21.15.188:6836/2260087679":"2024-03-29T13:16:39.492982+0000","172.21.15.188:6834/2260087679":"2024-03-29T13:16:39.492982+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:18:50.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.442+0000 7ff2b7e00640 1 -- 172.21.15.19:0/3528939086 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff29406c530 msgr2=0x7ff29406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:50.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.442+0000 7ff2b7e00640 1 --2- 172.21.15.19:0/3528939086 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff29406c530 0x7ff29406e9f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8036000 tx=0x7ff2a803b040 comp rx=0 tx=0).stop 2024-03-28T13:18:50.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.442+0000 7ff2b7e00640 1 -- 172.21.15.19:0/3528939086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2b8147a40 msgr2=0x7ff2b81f76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:50.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.442+0000 7ff2b7e00640 1 --2- 172.21.15.19:0/3528939086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2b8147a40 0x7ff2b81f76c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0007e70 tx=0x7ff2b000e4b0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.443+0000 7ff2b7e00640 1 -- 172.21.15.19:0/3528939086 shutdown_connections 2024-03-28T13:18:50.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.443+0000 7ff2b7e00640 1 --2- 172.21.15.19:0/3528939086 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff29406c530 0x7ff29406e9f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.443+0000 7ff2b7e00640 1 --2- 172.21.15.19:0/3528939086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff2b8147a40 0x7ff2b81f76c0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.443+0000 7ff2b7e00640 1 --2- 172.21.15.19:0/3528939086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2b80d01c0 0x7ff2b8146cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.443+0000 7ff2b7e00640 1 --2- 172.21.15.19:0/3528939086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2b80ce380 0x7ff2b8143ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.443+0000 7ff2b7e00640 1 -- 172.21.15.19:0/3528939086 >> 172.21.15.19:0/3528939086 conn(0x7ff2b80ccad0 msgr2=0x7ff2b815f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:50.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.443+0000 7ff2b7e00640 1 -- 172.21.15.19:0/3528939086 shutdown_connections 2024-03-28T13:18:50.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.443+0000 7ff2b7e00640 1 -- 172.21.15.19:0/3528939086 wait complete. 2024-03-28T13:18:50.463 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:18:50.464 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:18:50.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.688+0000 7f2bfa200640 1 Processor -- start 2024-03-28T13:18:50.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.689+0000 7f2bfa200640 1 -- start start 2024-03-28T13:18:50.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.690+0000 7f2bfa200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2bf4153580 0x7f2bf40d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:50.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.690+0000 7f2bfa200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bf40d07c0 0x7f2bf40aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:50.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.690+0000 7f2bfa200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2bf40af2e0 0x7f2bf406d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:50.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.690+0000 7f2bfa200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf40d0c00 con 0x7f2bf4153580 2024-03-28T13:18:50.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.690+0000 7f2bfa200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf40afab0 con 0x7f2bf40af2e0 2024-03-28T13:18:50.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.690+0000 7f2bfa200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf415cc80 con 0x7f2bf40d07c0 2024-03-28T13:18:50.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.691+0000 7f2bf3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2bf4153580 0x7f2bf40d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:50.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.691+0000 7f2bf3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2bf4153580 0x7f2bf40d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32824/0 (socket says 172.21.15.19:32824) 2024-03-28T13:18:50.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.691+0000 7f2bf3400640 1 -- 172.21.15.19:0/3982541319 learned_addr learned my addr 172.21.15.19:0/3982541319 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:50.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.691+0000 7f2bf3e00640 1 --2- 172.21.15.19:0/3982541319 >> v2:172.21.15.188:3300/0 conn(0x7f2bf40af2e0 0x7f2bf406d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:50.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.691+0000 7f2bebe00640 1 --2- 172.21.15.19:0/3982541319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bf40d07c0 0x7f2bf40aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:50.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.691+0000 7f2bf3400640 1 -- 172.21.15.19:0/3982541319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bf40d07c0 msgr2=0x7f2bf40aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:50.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.691+0000 7f2bf3400640 1 --2- 172.21.15.19:0/3982541319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bf40d07c0 0x7f2bf40aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.691+0000 7f2bf3400640 1 -- 172.21.15.19:0/3982541319 >> v2:172.21.15.188:3300/0 conn(0x7f2bf40af2e0 msgr2=0x7f2bf406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:50.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.691+0000 7f2bf3400640 1 --2- 172.21.15.19:0/3982541319 >> v2:172.21.15.188:3300/0 conn(0x7f2bf40af2e0 0x7f2bf406d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.691+0000 7f2bf3400640 1 -- 172.21.15.19:0/3982541319 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf415da00 con 0x7f2bf4153580 2024-03-28T13:18:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.692+0000 7f2bf3400640 1 --2- 172.21.15.19:0/3982541319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bf4153580 0x7f2bf40d0280 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40cff70 tx=0x7f2be402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293677bf490607c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.693+0000 7f2bf2a00640 1 -- 172.21.15.19:0/3982541319 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be40317d0 con 0x7f2bf4153580 2024-03-28T13:18:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.693+0000 7f2bf2a00640 1 -- 172.21.15.19:0/3982541319 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be4037070 con 0x7f2bf4153580 2024-03-28T13:18:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.693+0000 7f2bf2a00640 1 -- 172.21.15.19:0/3982541319 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4031b90 con 0x7f2bf4153580 2024-03-28T13:18:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.693+0000 7f2bfa200640 1 -- 172.21.15.19:0/3982541319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bf4153580 msgr2=0x7f2bf40d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.693+0000 7f2bfa200640 1 --2- 172.21.15.19:0/3982541319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bf4153580 0x7f2bf40d0280 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40cff70 tx=0x7f2be402b870 comp rx=0 tx=0).stop 2024-03-28T13:18:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.693+0000 7f2bfa200640 1 -- 172.21.15.19:0/3982541319 shutdown_connections 2024-03-28T13:18:50.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.693+0000 7f2bfa200640 1 --2- 172.21.15.19:0/3982541319 >> v2:172.21.15.188:3300/0 conn(0x7f2bf40af2e0 0x7f2bf406d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.693+0000 7f2bfa200640 1 --2- 172.21.15.19:0/3982541319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bf40d07c0 0x7f2bf40aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.693+0000 7f2bfa200640 1 --2- 172.21.15.19:0/3982541319 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bf4153580 0x7f2bf40d0280 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.693+0000 7f2bfa200640 1 -- 172.21.15.19:0/3982541319 >> 172.21.15.19:0/3982541319 conn(0x7f2bf406b750 msgr2=0x7f2bf406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:50.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.694+0000 7f2bfa200640 1 -- 172.21.15.19:0/3982541319 shutdown_connections 2024-03-28T13:18:50.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.694+0000 7f2bfa200640 1 -- 172.21.15.19:0/3982541319 wait complete. 2024-03-28T13:18:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.695+0000 7f2bfa200640 1 Processor -- start 2024-03-28T13:18:50.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.695+0000 7f2bfa200640 1 -- start start 2024-03-28T13:18:50.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.696+0000 7f2bfa200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bf40af2e0 0x7f2bf4099840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:50.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.696+0000 7f2bfa200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bf40d07c0 0x7f2bf409c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:50.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.696+0000 7f2bfa200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bf4153580 0x7f2bf4151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:50.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.696+0000 7f2bfa200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf415d4e0 con 0x7f2bf4153580 2024-03-28T13:18:50.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.696+0000 7f2bfa200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf415d200 con 0x7f2bf40af2e0 2024-03-28T13:18:50.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.696+0000 7f2bfa200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf4153960 con 0x7f2bf40d07c0 2024-03-28T13:18:50.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.697+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bf4153580 0x7f2bf4151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:50.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.697+0000 7f2bf3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bf4153580 0x7f2bf4151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32840/0 (socket says 172.21.15.19:32840) 2024-03-28T13:18:50.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.697+0000 7f2bf3e00640 1 -- 172.21.15.19:0/400224522 learned_addr learned my addr 172.21.15.19:0/400224522 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:50.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.697+0000 7f2bf3400640 1 --2- 172.21.15.19:0/400224522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bf40af2e0 0x7f2bf4099840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:50.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.697+0000 7f2bebe00640 1 --2- 172.21.15.19:0/400224522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bf40d07c0 0x7f2bf409c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:50.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.697+0000 7f2bf3e00640 1 -- 172.21.15.19:0/400224522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bf40d07c0 msgr2=0x7f2bf409c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:50.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.697+0000 7f2bf3e00640 1 --2- 172.21.15.19:0/400224522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bf40d07c0 0x7f2bf409c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.697+0000 7f2bf3e00640 1 -- 172.21.15.19:0/400224522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bf40af2e0 msgr2=0x7f2bf4099840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:50.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.697+0000 7f2bf3e00640 1 --2- 172.21.15.19:0/400224522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bf40af2e0 0x7f2bf4099840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.697+0000 7f2bf3e00640 1 -- 172.21.15.19:0/400224522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf409caa0 con 0x7f2bf4153580 2024-03-28T13:18:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.698+0000 7f2bf3e00640 1 --2- 172.21.15.19:0/400224522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bf4153580 0x7f2bf4151bb0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f2bec007d20 tx=0x7f2bec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.698+0000 7f2beb400640 1 -- 172.21.15.19:0/400224522 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec014420 con 0x7f2bf4153580 2024-03-28T13:18:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.698+0000 7f2beb400640 1 -- 172.21.15.19:0/400224522 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bec019070 con 0x7f2bf4153580 2024-03-28T13:18:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.698+0000 7f2beb400640 1 -- 172.21.15.19:0/400224522 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bec014820 con 0x7f2bf4153580 2024-03-28T13:18:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.698+0000 7f2bfa200640 1 -- 172.21.15.19:0/400224522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf409cd80 con 0x7f2bf4153580 2024-03-28T13:18:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.698+0000 7f2bfa200640 1 -- 172.21.15.19:0/400224522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf409d340 con 0x7f2bf4153580 2024-03-28T13:18:50.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.699+0000 7f2bfa200640 1 -- 172.21.15.19:0/400224522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf40685c0 con 0x7f2bf4153580 2024-03-28T13:18:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.700+0000 7f2beb400640 1 -- 172.21.15.19:0/400224522 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2bec0149c0 con 0x7f2bf4153580 2024-03-28T13:18:50.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.701+0000 7f2beb400640 1 --2- 172.21.15.19:0/400224522 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2bd006c530 0x7f2bd006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:50.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.701+0000 7f2beb400640 1 -- 172.21.15.19:0/400224522 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5115+0+0 (secure 0 0 0) 0x7f2bec08bd40 con 0x7f2bf4153580 2024-03-28T13:18:50.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.702+0000 7f2bf3400640 1 --2- 172.21.15.19:0/400224522 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2bd006c530 0x7f2bd006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:50.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.703+0000 7f2bf3400640 1 --2- 172.21.15.19:0/400224522 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2bd006c530 0x7f2bd006e9f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2be4036000 tx=0x7f2be403d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:50.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.706+0000 7f2beb400640 1 -- 172.21.15.19:0/400224522 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f2bec05a510 con 0x7f2bf4153580 2024-03-28T13:18:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.907+0000 7f2bfa200640 1 -- 172.21.15.19:0/400224522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2bf415b450 con 0x7f2bf4153580 2024-03-28T13:18:50.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.908+0000 7f2beb400640 1 -- 172.21.15.19:0/400224522 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) ==== 93+0+10 (secure 0 0 0) 0x7f2bec059e10 con 0x7f2bf4153580 2024-03-28T13:18:50.907 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:18:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.910+0000 7f2bfa200640 1 -- 172.21.15.19:0/400224522 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2bd006c530 msgr2=0x7f2bd006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.910+0000 7f2bfa200640 1 --2- 172.21.15.19:0/400224522 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2bd006c530 0x7f2bd006e9f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f2be4036000 tx=0x7f2be403d040 comp rx=0 tx=0).stop 2024-03-28T13:18:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.910+0000 7f2bfa200640 1 -- 172.21.15.19:0/400224522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bf4153580 msgr2=0x7f2bf4151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.910+0000 7f2bfa200640 1 --2- 172.21.15.19:0/400224522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bf4153580 0x7f2bf4151bb0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f2bec007d20 tx=0x7f2bec00e4b0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.910+0000 7f2bfa200640 1 -- 172.21.15.19:0/400224522 shutdown_connections 2024-03-28T13:18:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.910+0000 7f2bfa200640 1 --2- 172.21.15.19:0/400224522 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2bd006c530 0x7f2bd006e9f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.910+0000 7f2bfa200640 1 --2- 172.21.15.19:0/400224522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2bf4153580 0x7f2bf4151bb0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.910+0000 7f2bfa200640 1 --2- 172.21.15.19:0/400224522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bf40d07c0 0x7f2bf409c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.910+0000 7f2bfa200640 1 --2- 172.21.15.19:0/400224522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bf40af2e0 0x7f2bf4099840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.910+0000 7f2bfa200640 1 -- 172.21.15.19:0/400224522 >> 172.21.15.19:0/400224522 conn(0x7f2bf406b750 msgr2=0x7f2bf415a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.910+0000 7f2bfa200640 1 -- 172.21.15.19:0/400224522 shutdown_connections 2024-03-28T13:18:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:50.910+0000 7f2bfa200640 1 -- 172.21.15.19:0/400224522 wait complete. 2024-03-28T13:18:50.930 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-03-28T13:18:51.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.137+0000 7f01d0600640 1 Processor -- start 2024-03-28T13:18:51.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.137+0000 7f01d0600640 1 -- start start 2024-03-28T13:18:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.139+0000 7f01d0600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f01c8153580 0x7f01c80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.139+0000 7f01d0600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f01c80d07c0 0x7f01c80aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.139+0000 7f01d0600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c80afab0 con 0x7f01c80af2e0 2024-03-28T13:18:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.139+0000 7f01d0600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f01c80d0c30 con 0x7f01c8153580 2024-03-28T13:18:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.139+0000 7f01d0600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f01c815cc80 con 0x7f01c80d07c0 2024-03-28T13:18:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.139+0000 7f01cdc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f01c8153580 0x7f01c80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.139+0000 7f01ce600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f01c80af2e0 0x7f01c806d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35616/0 (socket says 172.21.15.19:35616) 2024-03-28T13:18:51.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.139+0000 7f01ce600640 1 -- 172.21.15.19:0/818806937 learned_addr learned my addr 172.21.15.19:0/818806937 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.140+0000 7f01cd200640 1 --2- 172.21.15.19:0/818806937 >> v2:172.21.15.188:3300/0 conn(0x7f01c80d07c0 0x7f01c80aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.140+0000 7f01cdc00640 1 -- 172.21.15.19:0/818806937 >> v2:172.21.15.188:3300/0 conn(0x7f01c80d07c0 msgr2=0x7f01c80aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.140+0000 7f01cdc00640 1 --2- 172.21.15.19:0/818806937 >> v2:172.21.15.188:3300/0 conn(0x7f01c80d07c0 0x7f01c80aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.140+0000 7f01cdc00640 1 -- 172.21.15.19:0/818806937 >> v1:172.21.15.19:6789/0 conn(0x7f01c80af2e0 legacy=0x7f01c806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:51.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.140+0000 7f01cdc00640 1 -- 172.21.15.19:0/818806937 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c815da00 con 0x7f01c8153580 2024-03-28T13:18:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.141+0000 7f01cdc00640 1 --2- 172.21.15.19:0/818806937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c8153580 0x7f01c80d0280 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f01c80cff70 tx=0x7f01bc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6024a5570b34fb03 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.141+0000 7f01c7e00640 1 -- 172.21.15.19:0/818806937 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc0315a0 con 0x7f01c8153580 2024-03-28T13:18:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.142+0000 7f01c7e00640 1 -- 172.21.15.19:0/818806937 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01bc037070 con 0x7f01c8153580 2024-03-28T13:18:51.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.142+0000 7f01c7e00640 1 -- 172.21.15.19:0/818806937 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc031960 con 0x7f01c8153580 2024-03-28T13:18:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.142+0000 7f01d0600640 1 -- 172.21.15.19:0/818806937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c8153580 msgr2=0x7f01c80d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.142+0000 7f01d0600640 1 --2- 172.21.15.19:0/818806937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c8153580 0x7f01c80d0280 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f01c80cff70 tx=0x7f01bc02b840 comp rx=0 tx=0).stop 2024-03-28T13:18:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.142+0000 7f01d0600640 1 -- 172.21.15.19:0/818806937 shutdown_connections 2024-03-28T13:18:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.142+0000 7f01d0600640 1 --2- 172.21.15.19:0/818806937 >> v2:172.21.15.188:3300/0 conn(0x7f01c80d07c0 0x7f01c80aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.142+0000 7f01d0600640 1 --2- 172.21.15.19:0/818806937 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c8153580 0x7f01c80d0280 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:51.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.142+0000 7f01d0600640 1 -- 172.21.15.19:0/818806937 >> 172.21.15.19:0/818806937 conn(0x7f01c806b750 msgr2=0x7f01c806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.142+0000 7f01d0600640 1 -- 172.21.15.19:0/818806937 shutdown_connections 2024-03-28T13:18:51.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.142+0000 7f01d0600640 1 -- 172.21.15.19:0/818806937 wait complete. 2024-03-28T13:18:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.144+0000 7f01d0600640 1 Processor -- start 2024-03-28T13:18:51.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.144+0000 7f01d0600640 1 -- start start 2024-03-28T13:18:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.145+0000 7f01d0600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01c80af2e0 0x7f01c81507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.145+0000 7f01d0600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c80d07c0 0x7f01c8157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.145+0000 7f01d0600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01c8153580 0x7f01c8154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.145+0000 7f01d0600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f01c815d4e0 con 0x7f01c80d07c0 2024-03-28T13:18:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.145+0000 7f01d0600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f01c815d200 con 0x7f01c80af2e0 2024-03-28T13:18:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.145+0000 7f01d0600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f01c8153960 con 0x7f01c8153580 2024-03-28T13:18:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.146+0000 7f01cd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c80d07c0 0x7f01c8157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.146+0000 7f01cd200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c80d07c0 0x7f01c8157b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32862/0 (socket says 172.21.15.19:32862) 2024-03-28T13:18:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.146+0000 7f01cd200640 1 -- 172.21.15.19:0/1485646269 learned_addr learned my addr 172.21.15.19:0/1485646269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.146+0000 7f01cdc00640 1 --2- 172.21.15.19:0/1485646269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01c80af2e0 0x7f01c81507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:51.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.146+0000 7f01ce600640 1 --2- 172.21.15.19:0/1485646269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01c8153580 0x7f01c8154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:51.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.146+0000 7f01cd200640 1 -- 172.21.15.19:0/1485646269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01c8153580 msgr2=0x7f01c8154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:51.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.146+0000 7f01cd200640 1 --2- 172.21.15.19:0/1485646269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01c8153580 0x7f01c8154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:51.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.146+0000 7f01cd200640 1 -- 172.21.15.19:0/1485646269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01c80af2e0 msgr2=0x7f01c81507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:51.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.146+0000 7f01cd200640 1 --2- 172.21.15.19:0/1485646269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01c80af2e0 0x7f01c81507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:51.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.146+0000 7f01cd200640 1 -- 172.21.15.19:0/1485646269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c80ccbd0 con 0x7f01c80d07c0 2024-03-28T13:18:51.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.146+0000 7f01cd200640 1 --2- 172.21.15.19:0/1485646269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c80d07c0 0x7f01c8157b80 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f01b400d800 tx=0x7f01b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.147+0000 7f01c5600640 1 -- 172.21.15.19:0/1485646269 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b400b6c0 con 0x7f01c80d07c0 2024-03-28T13:18:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.147+0000 7f01c5600640 1 -- 172.21.15.19:0/1485646269 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b4019070 con 0x7f01c80d07c0 2024-03-28T13:18:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.147+0000 7f01c5600640 1 -- 172.21.15.19:0/1485646269 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b400ba80 con 0x7f01c80d07c0 2024-03-28T13:18:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.147+0000 7f01d0600640 1 -- 172.21.15.19:0/1485646269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c80cceb0 con 0x7f01c80d07c0 2024-03-28T13:18:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.147+0000 7f01d0600640 1 -- 172.21.15.19:0/1485646269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c80cd470 con 0x7f01c80d07c0 2024-03-28T13:18:51.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.148+0000 7f01d0600640 1 -- 172.21.15.19:0/1485646269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c80685c0 con 0x7f01c80d07c0 2024-03-28T13:18:51.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.149+0000 7f01c5600640 1 -- 172.21.15.19:0/1485646269 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f01b400bc20 con 0x7f01c80d07c0 2024-03-28T13:18:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.150+0000 7f01c5600640 1 --2- 172.21.15.19:0/1485646269 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f01a806c530 0x7f01a806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.150+0000 7f01c5600640 1 -- 172.21.15.19:0/1485646269 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5115+0+0 (secure 0 0 0) 0x7f01b408b970 con 0x7f01c80d07c0 2024-03-28T13:18:51.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.151+0000 7f01cdc00640 1 --2- 172.21.15.19:0/1485646269 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f01a806c530 0x7f01a806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:51.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.152+0000 7f01cdc00640 1 --2- 172.21.15.19:0/1485646269 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f01a806c530 0x7f01a806e9f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f01bc036000 tx=0x7f01bc03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:51.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.155+0000 7f01c5600640 1 -- 172.21.15.19:0/1485646269 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f01b401c360 con 0x7f01c80d07c0 2024-03-28T13:18:51.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.427+0000 7f01d0600640 1 -- 172.21.15.19:0/1485646269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f01c806eca0 con 0x7f01c80d07c0 2024-03-28T13:18:51.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.984+0000 7f01c5600640 1 -- 172.21.15.19:0/1485646269 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v445) ==== 149+0+0 (secure 0 0 0) 0x7f01b405a1d0 con 0x7f01c80d07c0 2024-03-28T13:18:51.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.988+0000 7f01d0600640 1 -- 172.21.15.19:0/1485646269 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f01a806c530 msgr2=0x7f01a806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:51.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.988+0000 7f01d0600640 1 --2- 172.21.15.19:0/1485646269 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f01a806c530 0x7f01a806e9f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f01bc036000 tx=0x7f01bc03d040 comp rx=0 tx=0).stop 2024-03-28T13:18:51.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.988+0000 7f01d0600640 1 -- 172.21.15.19:0/1485646269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c80d07c0 msgr2=0x7f01c8157b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:51.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.988+0000 7f01d0600640 1 --2- 172.21.15.19:0/1485646269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c80d07c0 0x7f01c8157b80 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f01b400d800 tx=0x7f01b400dcc0 comp rx=0 tx=0).stop 2024-03-28T13:18:51.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.988+0000 7f01d0600640 1 -- 172.21.15.19:0/1485646269 shutdown_connections 2024-03-28T13:18:51.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.988+0000 7f01d0600640 1 --2- 172.21.15.19:0/1485646269 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f01a806c530 0x7f01a806e9f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:51.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.988+0000 7f01d0600640 1 --2- 172.21.15.19:0/1485646269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01c8153580 0x7f01c8154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:51.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.988+0000 7f01d0600640 1 --2- 172.21.15.19:0/1485646269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01c80d07c0 0x7f01c8157b80 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:51.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.988+0000 7f01d0600640 1 --2- 172.21.15.19:0/1485646269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01c80af2e0 0x7f01c81507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:51.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.988+0000 7f01d0600640 1 -- 172.21.15.19:0/1485646269 >> 172.21.15.19:0/1485646269 conn(0x7f01c806b750 msgr2=0x7f01c815b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:51.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.988+0000 7f01d0600640 1 -- 172.21.15.19:0/1485646269 shutdown_connections 2024-03-28T13:18:51.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:51.989+0000 7f01d0600640 1 -- 172.21.15.19:0/1485646269 wait complete. 2024-03-28T13:18:52.009 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-28T13:18:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.228+0000 7f7457200640 1 Processor -- start 2024-03-28T13:18:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.228+0000 7f7457200640 1 -- start start 2024-03-28T13:18:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.229+0000 7f7457200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74500d0280 0x7f74500d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.229+0000 7f7457200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7450141c10 0x7f7450071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.229+0000 7f7457200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f745015ec80 con 0x7f7450071eb0 2024-03-28T13:18:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.229+0000 7f7457200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f74500d0ba0 con 0x7f7450141c10 2024-03-28T13:18:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.229+0000 7f7457200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f745015ef20 con 0x7f74500d0280 2024-03-28T13:18:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.230+0000 7f7455200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7450071eb0 0x7f745006d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:35626/0 (socket says 172.21.15.19:35626) 2024-03-28T13:18:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.230+0000 7f7455200640 1 -- 172.21.15.19:0/2248522034 learned_addr learned my addr 172.21.15.19:0/2248522034 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:52.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.230+0000 7f744fe00640 1 --2- 172.21.15.19:0/2248522034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74500d0280 0x7f74500d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:52.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.230+0000 7f7447400640 1 --2- 172.21.15.19:0/2248522034 >> v2:172.21.15.188:3300/0 conn(0x7f7450141c10 0x7f7450071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:52.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.230+0000 7f744f400640 1 -- 172.21.15.19:0/2248522034 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1299667541 0 0) 0x7f745015ec80 con 0x7f7450071eb0 2024-03-28T13:18:52.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.230+0000 7f744f400640 1 -- 172.21.15.19:0/2248522034 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7434003520 con 0x7f7450071eb0 2024-03-28T13:18:52.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.231+0000 7f744f400640 1 -- 172.21.15.19:0/2248522034 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2466206576 0 0) 0x7f7434003520 con 0x7f7450071eb0 2024-03-28T13:18:52.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.231+0000 7f744f400640 1 -- 172.21.15.19:0/2248522034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74500d0280 msgr2=0x7f74500d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:52.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.231+0000 7f744f400640 1 --2- 172.21.15.19:0/2248522034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74500d0280 0x7f74500d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:52.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.231+0000 7f744f400640 1 -- 172.21.15.19:0/2248522034 >> v2:172.21.15.188:3300/0 conn(0x7f7450141c10 msgr2=0x7f7450071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:52.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.231+0000 7f744f400640 1 --2- 172.21.15.19:0/2248522034 >> v2:172.21.15.188:3300/0 conn(0x7f7450141c10 0x7f7450071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:52.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.231+0000 7f744f400640 1 -- 172.21.15.19:0/2248522034 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745015f200 con 0x7f7450071eb0 2024-03-28T13:18:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.231+0000 7f744f400640 1 -- 172.21.15.19:0/2248522034 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f74480019f0 con 0x7f7450071eb0 2024-03-28T13:18:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.231+0000 7f744f400640 1 -- 172.21.15.19:0/2248522034 >> v1:172.21.15.19:6789/0 conn(0x7f7450071eb0 legacy=0x7f745006d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.231+0000 7f744f400640 1 --2- 172.21.15.19:0/2248522034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74500d0280 0x7f74340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.231+0000 7f744f400640 1 --2- 172.21.15.19:0/2248522034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7434003f80 0x7f7434004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.231+0000 7f744f400640 1 --2- 172.21.15.19:0/2248522034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7434004d40 0x7f74340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.231+0000 7f744f400640 1 -- 172.21.15.19:0/2248522034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f74500d0ba0 con 0x7f7434004d40 2024-03-28T13:18:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.231+0000 7f744f400640 1 -- 172.21.15.19:0/2248522034 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f745015ef20 con 0x7f7434003f80 2024-03-28T13:18:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.231+0000 7f744f400640 1 -- 172.21.15.19:0/2248522034 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f743400b9c0 con 0x7f74500d0280 2024-03-28T13:18:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.231+0000 7f7455200640 1 --2- 172.21.15.19:0/2248522034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7434004d40 0x7f74340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.232+0000 7f7447400640 1 --2- 172.21.15.19:0/2248522034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7434003f80 0x7f7434004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.232+0000 7f744fe00640 1 --2- 172.21.15.19:0/2248522034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74500d0280 0x7f74340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.232+0000 7f7455200640 1 -- 172.21.15.19:0/2248522034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74500d0280 msgr2=0x7f74340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.232+0000 7f7455200640 1 --2- 172.21.15.19:0/2248522034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74500d0280 0x7f74340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.232+0000 7f7455200640 1 -- 172.21.15.19:0/2248522034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7434003f80 msgr2=0x7f7434004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:52.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.232+0000 7f7455200640 1 --2- 172.21.15.19:0/2248522034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7434003f80 0x7f7434004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.232+0000 7f7455200640 1 -- 172.21.15.19:0/2248522034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7434003520 con 0x7f7434004d40 2024-03-28T13:18:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.232+0000 7f7447400640 1 --2- 172.21.15.19:0/2248522034 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7434003f80 0x7f7434004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:18:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.233+0000 7f7455200640 1 --2- 172.21.15.19:0/2248522034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7434004d40 0x7f74340051c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f744800a7d0 tx=0x7f7434005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12af215e04b313fd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.233+0000 7f744f400640 1 -- 172.21.15.19:0/2248522034 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744800a3f0 con 0x7f7434004d40 2024-03-28T13:18:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.233+0000 7f744f400640 1 -- 172.21.15.19:0/2248522034 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7448034650 con 0x7f7434004d40 2024-03-28T13:18:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.233+0000 7f744f400640 1 -- 172.21.15.19:0/2248522034 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448034a80 con 0x7f7434004d40 2024-03-28T13:18:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.234+0000 7f7457200640 1 -- 172.21.15.19:0/2248522034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7434004d40 msgr2=0x7f74340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.234+0000 7f7457200640 1 --2- 172.21.15.19:0/2248522034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7434004d40 0x7f74340051c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f744800a7d0 tx=0x7f7434005830 comp rx=0 tx=0).stop 2024-03-28T13:18:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.234+0000 7f7455200640 1 -- 172.21.15.19:0/2248522034 reap_dead start 2024-03-28T13:18:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.234+0000 7f7457200640 1 -- 172.21.15.19:0/2248522034 shutdown_connections 2024-03-28T13:18:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.234+0000 7f7457200640 1 -- 172.21.15.19:0/2248522034 >> 172.21.15.19:0/2248522034 conn(0x7f745006b750 msgr2=0x7f745006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.234+0000 7f7457200640 1 -- 172.21.15.19:0/2248522034 shutdown_connections 2024-03-28T13:18:52.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.234+0000 7f7457200640 1 -- 172.21.15.19:0/2248522034 wait complete. 2024-03-28T13:18:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.236+0000 7f7457200640 1 Processor -- start 2024-03-28T13:18:52.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.236+0000 7f7457200640 1 -- start start 2024-03-28T13:18:52.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.237+0000 7f7457200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7434004d40 0x7f7450146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.237+0000 7f7457200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74501471c0 0x7f745009d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.237+0000 7f7457200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74501424e0 0x7f7450142960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.237+0000 7f7457200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7450158e90 con 0x7f74501471c0 2024-03-28T13:18:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.237+0000 7f7457200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f745015f7c0 con 0x7f74501424e0 2024-03-28T13:18:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.237+0000 7f7457200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7450146b80 con 0x7f7434004d40 2024-03-28T13:18:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.237+0000 7f7447400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74501471c0 0x7f745009d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.237+0000 7f7447400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74501471c0 0x7f745009d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32870/0 (socket says 172.21.15.19:32870) 2024-03-28T13:18:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.237+0000 7f7447400640 1 -- 172.21.15.19:0/3594810758 learned_addr learned my addr 172.21.15.19:0/3594810758 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.238+0000 7f744fe00640 1 --2- 172.21.15.19:0/3594810758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7434004d40 0x7f7450146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.238+0000 7f7447400640 1 -- 172.21.15.19:0/3594810758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7434004d40 msgr2=0x7f7450146350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:52.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.238+0000 7f7455200640 1 --2- 172.21.15.19:0/3594810758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74501424e0 0x7f7450142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.238+0000 7f7447400640 1 --2- 172.21.15.19:0/3594810758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7434004d40 0x7f7450146350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.238+0000 7f7447400640 1 -- 172.21.15.19:0/3594810758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74501424e0 msgr2=0x7f7450142960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.238+0000 7f7447400640 1 --2- 172.21.15.19:0/3594810758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74501424e0 0x7f7450142960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.238+0000 7f7447400640 1 -- 172.21.15.19:0/3594810758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7450144930 con 0x7f74501471c0 2024-03-28T13:18:52.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.238+0000 7f7447400640 1 --2- 172.21.15.19:0/3594810758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74501471c0 0x7f745009d560 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f7438006c00 tx=0x7f74380119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:52.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.238+0000 7f744cc00640 1 -- 172.21.15.19:0/3594810758 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438015870 con 0x7f74501471c0 2024-03-28T13:18:52.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.239+0000 7f744cc00640 1 -- 172.21.15.19:0/3594810758 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7438015a10 con 0x7f74501471c0 2024-03-28T13:18:52.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.239+0000 7f744cc00640 1 -- 172.21.15.19:0/3594810758 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7438015de0 con 0x7f74501471c0 2024-03-28T13:18:52.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.239+0000 7f7457200640 1 -- 172.21.15.19:0/3594810758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7450144c10 con 0x7f74501471c0 2024-03-28T13:18:52.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.239+0000 7f7457200640 1 -- 172.21.15.19:0/3594810758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7450146890 con 0x7f74501471c0 2024-03-28T13:18:52.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.240+0000 7f7457200640 1 -- 172.21.15.19:0/3594810758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74500685c0 con 0x7f74501471c0 2024-03-28T13:18:52.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.241+0000 7f744cc00640 1 -- 172.21.15.19:0/3594810758 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f743801d400 con 0x7f74501471c0 2024-03-28T13:18:52.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.242+0000 7f744cc00640 1 --2- 172.21.15.19:0/3594810758 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f742c06c530 0x7f742c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:52.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.242+0000 7f744cc00640 1 -- 172.21.15.19:0/3594810758 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5115+0+0 (secure 0 0 0) 0x7f743808bd90 con 0x7f74501471c0 2024-03-28T13:18:52.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.242+0000 7f744fe00640 1 --2- 172.21.15.19:0/3594810758 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f742c06c530 0x7f742c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:52.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.244+0000 7f744fe00640 1 --2- 172.21.15.19:0/3594810758 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f742c06c530 0x7f742c06e9f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7440009990 tx=0x7f7440008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:52.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.247+0000 7f744cc00640 1 -- 172.21.15.19:0/3594810758 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7438019030 con 0x7f74501471c0 2024-03-28T13:18:52.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.451+0000 7f7457200640 1 -- 172.21.15.19:0/3594810758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f74500cfdb0 con 0x7f74501471c0 2024-03-28T13:18:52.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.457+0000 7f744cc00640 1 -- 172.21.15.19:0/3594810758 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f743805a560 con 0x7f74501471c0 2024-03-28T13:18:52.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.459+0000 7f7457200640 1 -- 172.21.15.19:0/3594810758 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f742c06c530 msgr2=0x7f742c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.459+0000 7f7457200640 1 --2- 172.21.15.19:0/3594810758 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f742c06c530 0x7f742c06e9f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7440009990 tx=0x7f7440008040 comp rx=0 tx=0).stop 2024-03-28T13:18:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.459+0000 7f7457200640 1 -- 172.21.15.19:0/3594810758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74501471c0 msgr2=0x7f745009d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.460+0000 7f7457200640 1 --2- 172.21.15.19:0/3594810758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74501471c0 0x7f745009d560 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f7438006c00 tx=0x7f74380119f0 comp rx=0 tx=0).stop 2024-03-28T13:18:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.460+0000 7f7457200640 1 -- 172.21.15.19:0/3594810758 shutdown_connections 2024-03-28T13:18:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.460+0000 7f7457200640 1 --2- 172.21.15.19:0/3594810758 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f742c06c530 0x7f742c06e9f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.460+0000 7f7457200640 1 --2- 172.21.15.19:0/3594810758 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74501424e0 0x7f7450142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.460+0000 7f7457200640 1 --2- 172.21.15.19:0/3594810758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74501471c0 0x7f745009d560 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.460+0000 7f7457200640 1 --2- 172.21.15.19:0/3594810758 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7434004d40 0x7f7450146350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.460+0000 7f7457200640 1 -- 172.21.15.19:0/3594810758 >> 172.21.15.19:0/3594810758 conn(0x7f745006b750 msgr2=0x7f745015fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.460+0000 7f7457200640 1 -- 172.21.15.19:0/3594810758 shutdown_connections 2024-03-28T13:18:52.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:52.460+0000 7f7457200640 1 -- 172.21.15.19:0/3594810758 wait complete. 2024-03-28T13:18:52.480 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-28T13:18:52.480 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:18:52.595 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:18:52.711 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-28T13:18:52.711 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:18:52.711 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:18:52.771 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:18:52.771 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:18:52.771 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:18:52.829 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:18:52.829 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:18:52.829 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:18:52.885 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:18:52.886 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:18:52.886 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:18:52.944 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:18:52.944 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-28T13:18:52.945 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:18:53.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.173+0000 7f98fbe00640 1 Processor -- start 2024-03-28T13:18:53.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.174+0000 7f98fbe00640 1 -- start start 2024-03-28T13:18:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.175+0000 7f98fbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98fc0d1910 0x7f98fc0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.175+0000 7f98fbe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98fc166450 con 0x7f98fc145db0 2024-03-28T13:18:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.175+0000 7f98fbe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98fc166730 con 0x7f98fc070180 2024-03-28T13:18:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.175+0000 7f98fbe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f98fc166a10 con 0x7f98fc0d1910 2024-03-28T13:18:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.175+0000 7f98faa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98fc0d1910 0x7f98fc0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:53.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.176+0000 7f98faa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98fc0d1910 0x7f98fc0b5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32884/0 (socket says 172.21.15.19:32884) 2024-03-28T13:18:53.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.176+0000 7f98faa00640 1 -- 172.21.15.19:0/1707286303 learned_addr learned my addr 172.21.15.19:0/1707286303 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:53.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.176+0000 7f98faa00640 1 -- 172.21.15.19:0/1707286303 >> v1:172.21.15.188:6789/0 conn(0x7f98fc070180 legacy=0x7f98fc162e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:53.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.176+0000 7f98faa00640 1 -- 172.21.15.19:0/1707286303 >> v1:172.21.15.19:6789/0 conn(0x7f98fc145db0 legacy=0x7f98fc06fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:53.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.176+0000 7f98faa00640 1 -- 172.21.15.19:0/1707286303 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98fc166cf0 con 0x7f98fc0d1910 2024-03-28T13:18:53.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.178+0000 7f98faa00640 1 --2- 172.21.15.19:0/1707286303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98fc0d1910 0x7f98fc0b5890 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009870 tx=0x7f98ec02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b85fdb0e58e3c3b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:53.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.178+0000 7f98f9600640 1 -- 172.21.15.19:0/1707286303 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec034070 con 0x7f98fc0d1910 2024-03-28T13:18:53.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.178+0000 7f98f9600640 1 -- 172.21.15.19:0/1707286303 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98ec030860 con 0x7f98fc0d1910 2024-03-28T13:18:53.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.178+0000 7f98f9600640 1 -- 172.21.15.19:0/1707286303 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec030c30 con 0x7f98fc0d1910 2024-03-28T13:18:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.179+0000 7f98fbe00640 1 -- 172.21.15.19:0/1707286303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98fc0d1910 msgr2=0x7f98fc0b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.179+0000 7f98fbe00640 1 --2- 172.21.15.19:0/1707286303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98fc0d1910 0x7f98fc0b5890 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009870 tx=0x7f98ec02b9a0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.179+0000 7f98fbe00640 1 -- 172.21.15.19:0/1707286303 shutdown_connections 2024-03-28T13:18:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.179+0000 7f98fbe00640 1 --2- 172.21.15.19:0/1707286303 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98fc0d1910 0x7f98fc0b5890 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.179+0000 7f98fbe00640 1 -- 172.21.15.19:0/1707286303 >> 172.21.15.19:0/1707286303 conn(0x7f98fc067460 msgr2=0x7f98fc067840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.179+0000 7f98fbe00640 1 -- 172.21.15.19:0/1707286303 shutdown_connections 2024-03-28T13:18:53.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.179+0000 7f98fbe00640 1 -- 172.21.15.19:0/1707286303 wait complete. 2024-03-28T13:18:53.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.181+0000 7f98fbe00640 1 Processor -- start 2024-03-28T13:18:53.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.181+0000 7f98fbe00640 1 -- start start 2024-03-28T13:18:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.182+0000 7f98fbe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98fc070180 0x7f98fc137af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.182+0000 7f98fbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98fc0d1910 0x7f98fc0d0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.182+0000 7f98fbe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98fc145db0 0x7f98fc15ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.182+0000 7f98fbe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc0b5f50 con 0x7f98fc0d1910 2024-03-28T13:18:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.182+0000 7f98fbe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc0b5dd0 con 0x7f98fc145db0 2024-03-28T13:18:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.183+0000 7f98fbe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f98fc0b53b0 con 0x7f98fc070180 2024-03-28T13:18:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.183+0000 7f98fa000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98fc0d1910 0x7f98fc0d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.183+0000 7f98fa000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98fc0d1910 0x7f98fc0d0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32892/0 (socket says 172.21.15.19:32892) 2024-03-28T13:18:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.183+0000 7f98fa000640 1 -- 172.21.15.19:0/1697942648 learned_addr learned my addr 172.21.15.19:0/1697942648 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:53.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.183+0000 7f98faa00640 1 --2- 172.21.15.19:0/1697942648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98fc070180 0x7f98fc137af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.183+0000 7f98fb400640 1 --2- 172.21.15.19:0/1697942648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98fc145db0 0x7f98fc15ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.183+0000 7f98fa000640 1 -- 172.21.15.19:0/1697942648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98fc070180 msgr2=0x7f98fc137af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.183+0000 7f98fa000640 1 --2- 172.21.15.19:0/1697942648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98fc070180 0x7f98fc137af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.183+0000 7f98fa000640 1 -- 172.21.15.19:0/1697942648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98fc145db0 msgr2=0x7f98fc15ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.183+0000 7f98fa000640 1 --2- 172.21.15.19:0/1697942648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98fc145db0 0x7f98fc15ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.183+0000 7f98fa000640 1 -- 172.21.15.19:0/1697942648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98fc13a980 con 0x7f98fc0d1910 2024-03-28T13:18:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.184+0000 7f98fa000640 1 --2- 172.21.15.19:0/1697942648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98fc0d1910 0x7f98fc0d0420 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f98e400c430 tx=0x7f98e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.184+0000 7f98f2a00640 1 -- 172.21.15.19:0/1697942648 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4017070 con 0x7f98fc0d1910 2024-03-28T13:18:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.184+0000 7f98f2a00640 1 -- 172.21.15.19:0/1697942648 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98e40052d0 con 0x7f98fc0d1910 2024-03-28T13:18:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.184+0000 7f98fbe00640 1 -- 172.21.15.19:0/1697942648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98fc138140 con 0x7f98fc0d1910 2024-03-28T13:18:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.184+0000 7f98f2a00640 1 -- 172.21.15.19:0/1697942648 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e4003830 con 0x7f98fc0d1910 2024-03-28T13:18:53.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.184+0000 7f98fbe00640 1 -- 172.21.15.19:0/1697942648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98fc138700 con 0x7f98fc0d1910 2024-03-28T13:18:53.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.185+0000 7f98fbe00640 1 -- 172.21.15.19:0/1697942648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98fc075940 con 0x7f98fc0d1910 2024-03-28T13:18:53.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.186+0000 7f98f2a00640 1 -- 172.21.15.19:0/1697942648 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f98e401a070 con 0x7f98fc0d1910 2024-03-28T13:18:53.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.187+0000 7f98f2a00640 1 --2- 172.21.15.19:0/1697942648 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f98d806c530 0x7f98d806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:53.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.187+0000 7f98f2a00640 1 -- 172.21.15.19:0/1697942648 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5115+0+0 (secure 0 0 0) 0x7f98e408a800 con 0x7f98fc0d1910 2024-03-28T13:18:53.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.188+0000 7f98faa00640 1 --2- 172.21.15.19:0/1697942648 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f98d806c530 0x7f98d806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:53.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.189+0000 7f98faa00640 1 --2- 172.21.15.19:0/1697942648 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f98d806c530 0x7f98d806e9f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f98ec036000 tx=0x7f98ec03b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:53.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.192+0000 7f98f2a00640 1 -- 172.21.15.19:0/1697942648 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f98fc13a980 con 0x7f98fc0d1910 2024-03-28T13:18:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.436+0000 7f98fbe00640 1 -- 172.21.15.19:0/1697942648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98fc071d40 con 0x7f98fc0d1910 2024-03-28T13:18:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.437+0000 7f98f2a00640 1 -- 172.21.15.19:0/1697942648 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+3872 (secure 0 0 0) 0x7f98e4059060 con 0x7f98fc0d1910 2024-03-28T13:18:53.437 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:53.437 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5636,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1787033820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1787033820},{"type":"v1","addr":"172.21.15.19:6835","nonce":1787033820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5642,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/332688308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":332688308},{"type":"v1","addr":"172.21.15.188:6837","nonce":332688308}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":5852,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/1144703580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":1144703580},{"type":"v1","addr":"172.21.15.188:6836","nonce":1144703580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":7941,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3261144754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3261144754},{"type":"v1","addr":"172.21.15.19:6834","nonce":3261144754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[]} 2024-03-28T13:18:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 76 2024-03-28T13:18:53.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.440+0000 7f98fbe00640 1 -- 172.21.15.19:0/1697942648 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f98d806c530 msgr2=0x7f98d806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:53.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.440+0000 7f98fbe00640 1 --2- 172.21.15.19:0/1697942648 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f98d806c530 0x7f98d806e9f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f98ec036000 tx=0x7f98ec03b040 comp rx=0 tx=0).stop 2024-03-28T13:18:53.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.440+0000 7f98fbe00640 1 -- 172.21.15.19:0/1697942648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98fc0d1910 msgr2=0x7f98fc0d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:53.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.440+0000 7f98fbe00640 1 --2- 172.21.15.19:0/1697942648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98fc0d1910 0x7f98fc0d0420 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f98e400c430 tx=0x7f98e400c8f0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.440+0000 7f98fbe00640 1 -- 172.21.15.19:0/1697942648 shutdown_connections 2024-03-28T13:18:53.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.440+0000 7f98fbe00640 1 --2- 172.21.15.19:0/1697942648 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f98d806c530 0x7f98d806e9f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.440+0000 7f98fbe00640 1 --2- 172.21.15.19:0/1697942648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98fc145db0 0x7f98fc15ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.440+0000 7f98fbe00640 1 --2- 172.21.15.19:0/1697942648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98fc0d1910 0x7f98fc0d0420 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.440+0000 7f98fbe00640 1 --2- 172.21.15.19:0/1697942648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f98fc070180 0x7f98fc137af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.440+0000 7f98fbe00640 1 -- 172.21.15.19:0/1697942648 >> 172.21.15.19:0/1697942648 conn(0x7f98fc067460 msgr2=0x7f98fc147d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:53.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.440+0000 7f98fbe00640 1 -- 172.21.15.19:0/1697942648 shutdown_connections 2024-03-28T13:18:53.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.440+0000 7f98fbe00640 1 -- 172.21.15.19:0/1697942648 wait complete. 2024-03-28T13:18:53.458 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-28T13:18:53.458 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-28T13:18:53.458 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-28T13:18:53.458 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:18:53.458 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-28T13:18:53.458 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-28T13:18:53.459 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-28T13:18:53.459 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:18:53.459 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-28T13:18:53.459 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-28T13:18:53.459 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-28T13:18:53.460 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:18:53.460 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-28T13:18:53.460 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-28T13:18:53.460 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-28T13:18:53.460 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:18:53.476 INFO:tasks.ceph.mds.a:Stopped 2024-03-28T13:18:53.476 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-28T13:18:53.479 INFO:tasks.ceph.mds.c:Stopped 2024-03-28T13:18:53.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-28T13:18:53.480 INFO:tasks.ceph.mds.b:Stopped 2024-03-28T13:18:53.480 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-28T13:18:53.482 INFO:tasks.ceph.mds.d:Stopped 2024-03-28T13:18:53.482 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-28T13:18:53.525 INFO:tasks.ceph.mds.a:Started 2024-03-28T13:18:53.526 DEBUG:teuthology.parallel:result is None 2024-03-28T13:18:53.528 INFO:tasks.ceph.mds.b:Started 2024-03-28T13:18:53.529 DEBUG:teuthology.parallel:result is None 2024-03-28T13:18:53.532 INFO:tasks.ceph.mds.c:Started 2024-03-28T13:18:53.532 DEBUG:teuthology.parallel:result is None 2024-03-28T13:18:53.535 INFO:tasks.ceph.mds.d:Started 2024-03-28T13:18:53.535 DEBUG:teuthology.parallel:result is None 2024-03-28T13:18:53.535 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-28T13:18:53.686 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-03-28T13:18:53.687 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-03-28T13:18:53.688 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-03-28T13:18:53.688 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-03-28T13:18:53.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.725+0000 7fc7cb000640 1 Processor -- start 2024-03-28T13:18:53.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.725+0000 7fc7cb000640 1 -- start start 2024-03-28T13:18:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.726+0000 7fc7cb000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c415e860 0x7fc7c406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.726+0000 7fc7cb000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc7c406da30 0x7fc7c40d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.726+0000 7fc7cb000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc7c40d4910 0x7fc7c40d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.726+0000 7fc7cb000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc7c40d4d60 con 0x7fc7c406da30 2024-03-28T13:18:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.726+0000 7fc7cb000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc7c40cf220 con 0x7fc7c40d4910 2024-03-28T13:18:53.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.726+0000 7fc7cb000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c406dfa0 con 0x7fc7c415e860 2024-03-28T13:18:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.726+0000 7fc7c3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc7c406da30 0x7fc7c40d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.727+0000 7fc7c3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc7c406da30 0x7fc7c40d43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32936/0 (socket says 172.21.15.19:32936) 2024-03-28T13:18:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.727+0000 7fc7c3400640 1 -- 172.21.15.19:0/1030021960 learned_addr learned my addr 172.21.15.19:0/1030021960 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.727+0000 7fc7c3e00640 1 --2- 172.21.15.19:0/1030021960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c415e860 0x7fc7c406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.727+0000 7fc7c9000640 1 --2- 172.21.15.19:0/1030021960 >> v2:172.21.15.188:3300/0 conn(0x7fc7c40d4910 0x7fc7c40d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:53.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.727+0000 7fc7c3400640 1 -- 172.21.15.19:0/1030021960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c415e860 msgr2=0x7fc7c406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.727+0000 7fc7c3400640 1 --2- 172.21.15.19:0/1030021960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c415e860 0x7fc7c406d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.727+0000 7fc7c3400640 1 -- 172.21.15.19:0/1030021960 >> v2:172.21.15.188:3300/0 conn(0x7fc7c40d4910 msgr2=0x7fc7c40d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.727+0000 7fc7c3400640 1 --2- 172.21.15.19:0/1030021960 >> v2:172.21.15.188:3300/0 conn(0x7fc7c40d4910 0x7fc7c40d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.727+0000 7fc7c3400640 1 -- 172.21.15.19:0/1030021960 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c406e240 con 0x7fc7c406da30 2024-03-28T13:18:53.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.728+0000 7fc7c3400640 1 --2- 172.21.15.19:0/1030021960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c406da30 0x7fc7c40d43d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40d5520 tx=0x7fc7b002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5275ea262d0f76c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.728+0000 7fc7c2a00640 1 -- 172.21.15.19:0/1030021960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0035020 con 0x7fc7c406da30 2024-03-28T13:18:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.729+0000 7fc7c2a00640 1 -- 172.21.15.19:0/1030021960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b0031870 con 0x7fc7c406da30 2024-03-28T13:18:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.729+0000 7fc7c2a00640 1 -- 172.21.15.19:0/1030021960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0031c40 con 0x7fc7c406da30 2024-03-28T13:18:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.729+0000 7fc7cb000640 1 -- 172.21.15.19:0/1030021960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c406da30 msgr2=0x7fc7c40d43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.729+0000 7fc7cb000640 1 --2- 172.21.15.19:0/1030021960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c406da30 0x7fc7c40d43d0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40d5520 tx=0x7fc7b002b910 comp rx=0 tx=0).stop 2024-03-28T13:18:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.729+0000 7fc7cb000640 1 -- 172.21.15.19:0/1030021960 shutdown_connections 2024-03-28T13:18:53.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.729+0000 7fc7cb000640 1 --2- 172.21.15.19:0/1030021960 >> v2:172.21.15.188:3300/0 conn(0x7fc7c40d4910 0x7fc7c40d0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.729+0000 7fc7cb000640 1 --2- 172.21.15.19:0/1030021960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c406da30 0x7fc7c40d43d0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.729+0000 7fc7cb000640 1 --2- 172.21.15.19:0/1030021960 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c415e860 0x7fc7c406d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.729+0000 7fc7cb000640 1 -- 172.21.15.19:0/1030021960 >> 172.21.15.19:0/1030021960 conn(0x7fc7c409d7d0 msgr2=0x7fc7c408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:53.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.729+0000 7fc7cb000640 1 -- 172.21.15.19:0/1030021960 shutdown_connections 2024-03-28T13:18:53.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.729+0000 7fc7cb000640 1 -- 172.21.15.19:0/1030021960 wait complete. 2024-03-28T13:18:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.731+0000 7fc7cb000640 1 Processor -- start 2024-03-28T13:18:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.731+0000 7fc7cb000640 1 -- start start 2024-03-28T13:18:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.732+0000 7fc7cb000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c406da30 0x7fc7c40b9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.732+0000 7fc7cb000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7c40d4910 0x7fc7c40ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.732+0000 7fc7cb000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c415e860 0x7fc7c40ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.732+0000 7fc7cb000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c40cedb0 con 0x7fc7c415e860 2024-03-28T13:18:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.732+0000 7fc7cb000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c40d2c50 con 0x7fc7c40d4910 2024-03-28T13:18:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.732+0000 7fc7cb000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c40cef30 con 0x7fc7c406da30 2024-03-28T13:18:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.733+0000 7fc7c9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c415e860 0x7fc7c40ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:53.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.733+0000 7fc7c9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c415e860 0x7fc7c40ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32946/0 (socket says 172.21.15.19:32946) 2024-03-28T13:18:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.733+0000 7fc7c9000640 1 -- 172.21.15.19:0/1475200291 learned_addr learned my addr 172.21.15.19:0/1475200291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.733+0000 7fc7c3400640 1 --2- 172.21.15.19:0/1475200291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7c40d4910 0x7fc7c40ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.733+0000 7fc7c3e00640 1 --2- 172.21.15.19:0/1475200291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c406da30 0x7fc7c40b9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.733+0000 7fc7c9000640 1 -- 172.21.15.19:0/1475200291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c406da30 msgr2=0x7fc7c40b9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.733+0000 7fc7c9000640 1 --2- 172.21.15.19:0/1475200291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c406da30 0x7fc7c40b9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.733+0000 7fc7c9000640 1 -- 172.21.15.19:0/1475200291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7c40d4910 msgr2=0x7fc7c40ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.733+0000 7fc7c9000640 1 --2- 172.21.15.19:0/1475200291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7c40d4910 0x7fc7c40ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.733+0000 7fc7c9000640 1 -- 172.21.15.19:0/1475200291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c40c9500 con 0x7fc7c415e860 2024-03-28T13:18:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.733+0000 7fc7c9000640 1 --2- 172.21.15.19:0/1475200291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c415e860 0x7fc7c40ca600 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00eed0 tx=0x7fc7bc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.734+0000 7fc7bbe00640 1 -- 172.21.15.19:0/1475200291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc018020 con 0x7fc7c415e860 2024-03-28T13:18:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.734+0000 7fc7bbe00640 1 -- 172.21.15.19:0/1475200291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7bc014400 con 0x7fc7c415e860 2024-03-28T13:18:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.734+0000 7fc7bbe00640 1 -- 172.21.15.19:0/1475200291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc0147f0 con 0x7fc7c415e860 2024-03-28T13:18:53.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.734+0000 7fc7cb000640 1 -- 172.21.15.19:0/1475200291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c40c8980 con 0x7fc7c415e860 2024-03-28T13:18:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.734+0000 7fc7cb000640 1 -- 172.21.15.19:0/1475200291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c40c8f40 con 0x7fc7c415e860 2024-03-28T13:18:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.735+0000 7fc7cb000640 1 -- 172.21.15.19:0/1475200291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c40685c0 con 0x7fc7c415e860 2024-03-28T13:18:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.736+0000 7fc7bbe00640 1 -- 172.21.15.19:0/1475200291 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc7bc005d10 con 0x7fc7c415e860 2024-03-28T13:18:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.737+0000 7fc7bbe00640 1 --2- 172.21.15.19:0/1475200291 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc7a406c560 0x7fc7a406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.737+0000 7fc7bbe00640 1 -- 172.21.15.19:0/1475200291 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 5115+0+0 (secure 0 0 0) 0x7fc7bc08ba10 con 0x7fc7c415e860 2024-03-28T13:18:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.737+0000 7fc7c3e00640 1 --2- 172.21.15.19:0/1475200291 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc7a406c560 0x7fc7a406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.738+0000 7fc7c3e00640 1 --2- 172.21.15.19:0/1475200291 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc7a406c560 0x7fc7a406ea20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009aa0 tx=0x7fc7b4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:53.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.741+0000 7fc7bbe00640 1 -- 172.21.15.19:0/1475200291 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc7bc022070 con 0x7fc7c415e860 2024-03-28T13:18:53.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:53.956+0000 7fc7cb000640 1 -- 172.21.15.19:0/1475200291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc7c40d00f0 con 0x7fc7c415e860 2024-03-28T13:18:54.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.013+0000 7fc7bbe00640 1 -- 172.21.15.19:0/1475200291 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) ==== 97+0+0 (secure 0 0 0) 0x7fc7bc059c20 con 0x7fc7c415e860 2024-03-28T13:18:54.014 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-03-28T13:18:54.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.018+0000 7fc7cb000640 1 -- 172.21.15.19:0/1475200291 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc7a406c560 msgr2=0x7fc7a406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:54.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.018+0000 7fc7cb000640 1 --2- 172.21.15.19:0/1475200291 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc7a406c560 0x7fc7a406ea20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009aa0 tx=0x7fc7b4009190 comp rx=0 tx=0).stop 2024-03-28T13:18:54.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.018+0000 7fc7cb000640 1 -- 172.21.15.19:0/1475200291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c415e860 msgr2=0x7fc7c40ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:54.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.018+0000 7fc7cb000640 1 --2- 172.21.15.19:0/1475200291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c415e860 0x7fc7c40ca600 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00eed0 tx=0x7fc7bc00b480 comp rx=0 tx=0).stop 2024-03-28T13:18:54.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.018+0000 7fc7cb000640 1 -- 172.21.15.19:0/1475200291 shutdown_connections 2024-03-28T13:18:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.018+0000 7fc7cb000640 1 --2- 172.21.15.19:0/1475200291 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc7a406c560 0x7fc7a406ea20 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.018+0000 7fc7cb000640 1 --2- 172.21.15.19:0/1475200291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7c415e860 0x7fc7c40ca600 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.018+0000 7fc7cb000640 1 --2- 172.21.15.19:0/1475200291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7c40d4910 0x7fc7c40ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.018+0000 7fc7cb000640 1 --2- 172.21.15.19:0/1475200291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7c406da30 0x7fc7c40b9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.018+0000 7fc7cb000640 1 -- 172.21.15.19:0/1475200291 >> 172.21.15.19:0/1475200291 conn(0x7fc7c409d7d0 msgr2=0x7fc7c40b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.018+0000 7fc7cb000640 1 -- 172.21.15.19:0/1475200291 shutdown_connections 2024-03-28T13:18:54.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.019+0000 7fc7cb000640 1 -- 172.21.15.19:0/1475200291 wait complete. 2024-03-28T13:18:54.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-28T13:18:54.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.260+0000 7faced000640 1 Processor -- start 2024-03-28T13:18:54.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.261+0000 7faced000640 1 -- start start 2024-03-28T13:18:54.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.262+0000 7faced000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face8155550 0x7face8155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:54.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.262+0000 7faced000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7face815f800 0x7face80d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:54.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.262+0000 7faced000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7face80b40b0 con 0x7face80d07c0 2024-03-28T13:18:54.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.262+0000 7faced000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7face80d0c30 con 0x7face815f800 2024-03-28T13:18:54.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.262+0000 7faced000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7face80b4390 con 0x7face8155550 2024-03-28T13:18:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.263+0000 7face5e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face8155550 0x7face8155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.263+0000 7face5e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face8155550 0x7face8155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:33104/0 (socket says 172.21.15.19:33104) 2024-03-28T13:18:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.263+0000 7face5e00640 1 -- 172.21.15.19:0/289344031 learned_addr learned my addr 172.21.15.19:0/289344031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:54.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.263+0000 7face5400640 1 --2- 172.21.15.19:0/289344031 >> v2:172.21.15.188:3300/0 conn(0x7face815f800 0x7face80d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.264+0000 7face5e00640 1 -- 172.21.15.19:0/289344031 >> v2:172.21.15.188:3300/0 conn(0x7face815f800 msgr2=0x7face80d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.264+0000 7face5e00640 1 --2- 172.21.15.19:0/289344031 >> v2:172.21.15.188:3300/0 conn(0x7face815f800 0x7face80d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.264+0000 7face5e00640 1 -- 172.21.15.19:0/289344031 >> v1:172.21.15.188:6789/0 conn(0x7face80d07c0 legacy=0x7face80aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:54.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.264+0000 7face5e00640 1 -- 172.21.15.19:0/289344031 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face80b4670 con 0x7face8155550 2024-03-28T13:18:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.265+0000 7face5e00640 1 --2- 172.21.15.19:0/289344031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face8155550 0x7face8155930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7facd8009870 tx=0x7facd802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b55ea29fda9dde4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.265+0000 7face4a00640 1 -- 172.21.15.19:0/289344031 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8034070 con 0x7face8155550 2024-03-28T13:18:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.266+0000 7face4a00640 1 -- 172.21.15.19:0/289344031 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd8030840 con 0x7face8155550 2024-03-28T13:18:54.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.266+0000 7face4a00640 1 -- 172.21.15.19:0/289344031 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8030c10 con 0x7face8155550 2024-03-28T13:18:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.266+0000 7faced000640 1 -- 172.21.15.19:0/289344031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face8155550 msgr2=0x7face8155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.266+0000 7faced000640 1 --2- 172.21.15.19:0/289344031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face8155550 0x7face8155930 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7facd8009870 tx=0x7facd802b980 comp rx=0 tx=0).stop 2024-03-28T13:18:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.266+0000 7faced000640 1 -- 172.21.15.19:0/289344031 shutdown_connections 2024-03-28T13:18:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.266+0000 7faced000640 1 --2- 172.21.15.19:0/289344031 >> v2:172.21.15.188:3300/0 conn(0x7face815f800 0x7face80d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.266+0000 7faced000640 1 --2- 172.21.15.19:0/289344031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face8155550 0x7face8155930 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.266+0000 7faced000640 1 -- 172.21.15.19:0/289344031 >> 172.21.15.19:0/289344031 conn(0x7face806b750 msgr2=0x7face806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.266+0000 7faced000640 1 -- 172.21.15.19:0/289344031 shutdown_connections 2024-03-28T13:18:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.266+0000 7faced000640 1 -- 172.21.15.19:0/289344031 wait complete. 2024-03-28T13:18:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.268+0000 7faced000640 1 Processor -- start 2024-03-28T13:18:54.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.268+0000 7faced000640 1 -- start start 2024-03-28T13:18:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.269+0000 7faced000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face80d07c0 0x7face8156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.269+0000 7faced000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face8155550 0x7face81f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.269+0000 7faced000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face815f800 0x7face81f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.269+0000 7faced000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7face815fc80 con 0x7face8155550 2024-03-28T13:18:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.269+0000 7faced000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7face80b4c30 con 0x7face815f800 2024-03-28T13:18:54.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.269+0000 7faced000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7face815ba20 con 0x7face80d07c0 2024-03-28T13:18:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.269+0000 7face5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face8155550 0x7face81f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.269+0000 7face5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face8155550 0x7face81f3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32954/0 (socket says 172.21.15.19:32954) 2024-03-28T13:18:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.269+0000 7face5400640 1 -- 172.21.15.19:0/4179208256 learned_addr learned my addr 172.21.15.19:0/4179208256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.270+0000 7face5e00640 1 --2- 172.21.15.19:0/4179208256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face80d07c0 0x7face8156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.270+0000 7face6800640 1 --2- 172.21.15.19:0/4179208256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face815f800 0x7face81f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.270+0000 7face5400640 1 -- 172.21.15.19:0/4179208256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face80d07c0 msgr2=0x7face8156270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.270+0000 7face5400640 1 --2- 172.21.15.19:0/4179208256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face80d07c0 0x7face8156270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.270+0000 7face5400640 1 -- 172.21.15.19:0/4179208256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face815f800 msgr2=0x7face81f43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.270+0000 7face5400640 1 --2- 172.21.15.19:0/4179208256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face815f800 0x7face81f43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.270+0000 7face5400640 1 -- 172.21.15.19:0/4179208256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face8156da0 con 0x7face8155550 2024-03-28T13:18:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.270+0000 7face5e00640 1 --2- 172.21.15.19:0/4179208256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face80d07c0 0x7face8156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.270+0000 7face5400640 1 --2- 172.21.15.19:0/4179208256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face8155550 0x7face81f3e70 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7facd0007ef0 tx=0x7facd000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.270+0000 7facde000640 1 -- 172.21.15.19:0/4179208256 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0014620 con 0x7face8155550 2024-03-28T13:18:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.270+0000 7facde000640 1 -- 172.21.15.19:0/4179208256 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd0019070 con 0x7face8155550 2024-03-28T13:18:54.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.270+0000 7facde000640 1 -- 172.21.15.19:0/4179208256 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0014a00 con 0x7face8155550 2024-03-28T13:18:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.270+0000 7faced000640 1 -- 172.21.15.19:0/4179208256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face8157080 con 0x7face8155550 2024-03-28T13:18:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.271+0000 7faced000640 1 -- 172.21.15.19:0/4179208256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face8157640 con 0x7face8155550 2024-03-28T13:18:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.272+0000 7faced000640 1 -- 172.21.15.19:0/4179208256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face80685c0 con 0x7face8155550 2024-03-28T13:18:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.273+0000 7facde000640 1 -- 172.21.15.19:0/4179208256 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7facd0014ba0 con 0x7face8155550 2024-03-28T13:18:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.274+0000 7facde000640 1 --2- 172.21.15.19:0/4179208256 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7facc406c530 0x7facc406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:54.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.274+0000 7facde000640 1 -- 172.21.15.19:0/4179208256 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4857+0+0 (secure 0 0 0) 0x7facd008bd80 con 0x7face8155550 2024-03-28T13:18:54.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.274+0000 7face5e00640 1 --2- 172.21.15.19:0/4179208256 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7facc406c530 0x7facc406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:54.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.276+0000 7face5e00640 1 --2- 172.21.15.19:0/4179208256 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7facc406c530 0x7facc406e9f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7facd8036000 tx=0x7facd803b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:54.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.278+0000 7facde000640 1 -- 172.21.15.19:0/4179208256 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7facd005a650 con 0x7face8155550 2024-03-28T13:18:54.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.552+0000 7faced000640 1 -- 172.21.15.19:0/4179208256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7face80b3ad0 con 0x7face8155550 2024-03-28T13:18:54.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.553+0000 7facde000640 1 -- 172.21.15.19:0/4179208256 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7facd0059f50 con 0x7face8155550 2024-03-28T13:18:54.553 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:54.553 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:18:54.553 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-03-28T13:18:54.554 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.554 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-03-28T13:18:54.554 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmNRIkABAAq/nJioDZjp1zFWAYOLPBYw==", 2024-03-28T13:18:54.554 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.554 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:18:54.554 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:18:54.554 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:18:54.554 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:18:54.554 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.554 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.554 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-03-28T13:18:54.554 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmc10CIxAAQWCSjH0hW3KUk8/qQm91QA==", 2024-03-28T13:18:54.554 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.554 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:18:54.555 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:18:54.555 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:18:54.555 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:18:54.555 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.555 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.555 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-03-28T13:18:54.555 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmmlUoEhAA4FUGXoZ/h/ve5fThJNdE2Q==", 2024-03-28T13:18:54.555 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.555 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:18:54.555 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:18:54.555 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:18:54.555 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:18:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-03-28T13:18:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmWDRnMxAA34LiLI79N5fVJnbcPacbQA==", 2024-03-28T13:18:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:18:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:18:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:18:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:18:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-03-28T13:18:54.556 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDYawVmxhwWLBAAm7C56UGConFx2D7Lbq2SSA==", 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDbawVmC2BiNxAALQrAcXU44m2dTCjks5yy/Q==", 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.557 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-03-28T13:18:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfawVmozrJFRAA7YV1uMucIW7H5SI4KY8aDw==", 2024-03-28T13:18:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:18:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:18:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:18:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-03-28T13:18:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiawVmccbTMBAATxk24Bf4LuQSqU92HdwgUg==", 2024-03-28T13:18:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:18:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:18:54.558 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDpawVmdwnILRAAjUEBwnBEeSltu2CLRGhmDQ==", 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDtawVmkwW5ARAAZJ/Nd/Wtq7LcbUjeqPNUag==", 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.559 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:18:54.560 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:18:54.560 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:18:54.560 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.560 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.560 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-03-28T13:18:54.560 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDwawVm254wKRAAmCYODqSd2Whn4YKQzywrng==", 2024-03-28T13:18:54.560 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.560 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:18:54.560 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:18:54.560 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:18:54.560 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.560 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.560 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-03-28T13:18:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDzawVmV8z+MhAA9JeF55K7ph1mQRWalT7XhA==", 2024-03-28T13:18:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:18:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:18:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:18:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-03-28T13:18:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q==", 2024-03-28T13:18:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:18:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-03-28T13:18:54.561 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-28T13:18:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-03-28T13:18:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmJYGjHBAAAx2TtR0Gb3fafnZU26FCMA==", 2024-03-28T13:18:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:18:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:18:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:18:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:18:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.562 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmE7V/ERAAYcdvolIMDd5ff+pCn7hN1w==", 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmk6q9JhAAeC3JHQDnK8IXGVoK6qGEFA==", 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:18:54.563 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDRawVm9IzFLRAA5i1Yl7LQ7eNNHp7v4H2kog==", 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDSawVm5H1FLxAAtyjOPcCEUQyvTpBdu3Fn3g==", 2024-03-28T13:18:54.564 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDSawVmebzfIRAAA0+qIIaprr4SfEZT6qRooA==", 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-03-28T13:18:54.565 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:18:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.556+0000 7faced000640 1 -- 172.21.15.19:0/4179208256 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7facc406c530 msgr2=0x7facc406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.556+0000 7faced000640 1 --2- 172.21.15.19:0/4179208256 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7facc406c530 0x7facc406e9f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7facd8036000 tx=0x7facd803b040 comp rx=0 tx=0).stop 2024-03-28T13:18:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.556+0000 7faced000640 1 -- 172.21.15.19:0/4179208256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face8155550 msgr2=0x7face81f3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.556+0000 7faced000640 1 --2- 172.21.15.19:0/4179208256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face8155550 0x7face81f3e70 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7facd0007ef0 tx=0x7facd000e4b0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.556+0000 7faced000640 1 -- 172.21.15.19:0/4179208256 shutdown_connections 2024-03-28T13:18:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.556+0000 7faced000640 1 --2- 172.21.15.19:0/4179208256 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7facc406c530 0x7facc406e9f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.556+0000 7faced000640 1 --2- 172.21.15.19:0/4179208256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face815f800 0x7face81f43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.556+0000 7faced000640 1 --2- 172.21.15.19:0/4179208256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7face8155550 0x7face81f3e70 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.556+0000 7faced000640 1 --2- 172.21.15.19:0/4179208256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face80d07c0 0x7face8156270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.556+0000 7faced000640 1 -- 172.21.15.19:0/4179208256 >> 172.21.15.19:0/4179208256 conn(0x7face806b750 msgr2=0x7face815e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.556+0000 7faced000640 1 -- 172.21.15.19:0/4179208256 shutdown_connections 2024-03-28T13:18:54.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.556+0000 7faced000640 1 -- 172.21.15.19:0/4179208256 wait complete. 2024-03-28T13:18:54.576 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:18:54.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.789+0000 7f30c2800640 1 Processor -- start 2024-03-28T13:18:54.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.789+0000 7f30c2800640 1 -- start start 2024-03-28T13:18:54.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.791+0000 7f30c2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f30bc15b310 0x7f30bc0727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:54.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.791+0000 7f30c2800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30bc0b4f90 0x7f30bc15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:54.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.791+0000 7f30c2800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f30bc0d0780 0x7f30bc0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:54.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.791+0000 7f30c2800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f30bc15b6f0 con 0x7f30bc15b310 2024-03-28T13:18:54.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.791+0000 7f30c2800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f30bc072e80 con 0x7f30bc0d0780 2024-03-28T13:18:54.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.791+0000 7f30c2800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f30bc073160 con 0x7f30bc0b4f90 2024-03-28T13:18:54.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.791+0000 7f30bb400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f30bc15b310 0x7f30bc0727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:54.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.791+0000 7f30bb400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f30bc15b310 0x7f30bc0727f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32962/0 (socket says 172.21.15.19:32962) 2024-03-28T13:18:54.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.791+0000 7f30bb400640 1 -- 172.21.15.19:0/2732566651 learned_addr learned my addr 172.21.15.19:0/2732566651 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:54.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.791+0000 7f30bbe00640 1 --2- 172.21.15.19:0/2732566651 >> v2:172.21.15.188:3300/0 conn(0x7f30bc0d0780 0x7f30bc0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:54.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.791+0000 7f30b2a00640 1 --2- 172.21.15.19:0/2732566651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30bc0b4f90 0x7f30bc15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:54.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.792+0000 7f30bb400640 1 -- 172.21.15.19:0/2732566651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30bc0b4f90 msgr2=0x7f30bc15f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:54.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.792+0000 7f30bb400640 1 --2- 172.21.15.19:0/2732566651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30bc0b4f90 0x7f30bc15f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.792+0000 7f30bb400640 1 -- 172.21.15.19:0/2732566651 >> v2:172.21.15.188:3300/0 conn(0x7f30bc0d0780 msgr2=0x7f30bc0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:54.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.792+0000 7f30bb400640 1 --2- 172.21.15.19:0/2732566651 >> v2:172.21.15.188:3300/0 conn(0x7f30bc0d0780 0x7f30bc0d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.792+0000 7f30bb400640 1 -- 172.21.15.19:0/2732566651 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc15efe0 con 0x7f30bc15b310 2024-03-28T13:18:54.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.793+0000 7f30bb400640 1 --2- 172.21.15.19:0/2732566651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc15b310 0x7f30bc0727f0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0ad710 tx=0x7f30ac02dfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59f6eb7f5c0e044 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:54.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.793+0000 7f30baa00640 1 -- 172.21.15.19:0/2732566651 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac02e970 con 0x7f30bc15b310 2024-03-28T13:18:54.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.793+0000 7f30baa00640 1 -- 172.21.15.19:0/2732566651 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30ac02eb10 con 0x7f30bc15b310 2024-03-28T13:18:54.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.793+0000 7f30baa00640 1 -- 172.21.15.19:0/2732566651 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac009600 con 0x7f30bc15b310 2024-03-28T13:18:54.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.794+0000 7f30c2800640 1 -- 172.21.15.19:0/2732566651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc15b310 msgr2=0x7f30bc0727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:54.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.794+0000 7f30c2800640 1 --2- 172.21.15.19:0/2732566651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc15b310 0x7f30bc0727f0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0ad710 tx=0x7f30ac02dfd0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.794+0000 7f30c2800640 1 -- 172.21.15.19:0/2732566651 shutdown_connections 2024-03-28T13:18:54.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.794+0000 7f30c2800640 1 --2- 172.21.15.19:0/2732566651 >> v2:172.21.15.188:3300/0 conn(0x7f30bc0d0780 0x7f30bc0d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.794+0000 7f30c2800640 1 --2- 172.21.15.19:0/2732566651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30bc0b4f90 0x7f30bc15f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.794+0000 7f30c2800640 1 --2- 172.21.15.19:0/2732566651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc15b310 0x7f30bc0727f0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.794+0000 7f30c2800640 1 -- 172.21.15.19:0/2732566651 >> 172.21.15.19:0/2732566651 conn(0x7f30bc06b990 msgr2=0x7f30bc06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:54.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.794+0000 7f30c2800640 1 -- 172.21.15.19:0/2732566651 shutdown_connections 2024-03-28T13:18:54.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.794+0000 7f30c2800640 1 -- 172.21.15.19:0/2732566651 wait complete. 2024-03-28T13:18:54.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.796+0000 7f30c2800640 1 Processor -- start 2024-03-28T13:18:54.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.796+0000 7f30c2800640 1 -- start start 2024-03-28T13:18:54.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.797+0000 7f30c2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0b4f90 0x7f30bc14d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:54.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.797+0000 7f30c2800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30bc0d0780 0x7f30bc099820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:54.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.797+0000 7f30c2800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30bc15b310 0x7f30bc0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:54.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.797+0000 7f30c2800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc0b5370 con 0x7f30bc0b4f90 2024-03-28T13:18:54.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.797+0000 7f30c2800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f30bc073440 con 0x7f30bc0d0780 2024-03-28T13:18:54.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.797+0000 7f30c2800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f30bc15fd20 con 0x7f30bc15b310 2024-03-28T13:18:54.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.797+0000 7f30bb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0b4f90 0x7f30bc14d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:54.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.797+0000 7f30bb400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0b4f90 0x7f30bc14d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32968/0 (socket says 172.21.15.19:32968) 2024-03-28T13:18:54.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.797+0000 7f30bb400640 1 -- 172.21.15.19:0/3085073657 learned_addr learned my addr 172.21.15.19:0/3085073657 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:54.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.798+0000 7f30b2a00640 1 --2- 172.21.15.19:0/3085073657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30bc0d0780 0x7f30bc099820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:54.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.798+0000 7f30bbe00640 1 --2- 172.21.15.19:0/3085073657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30bc15b310 0x7f30bc0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:54.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.798+0000 7f30bb400640 1 -- 172.21.15.19:0/3085073657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30bc15b310 msgr2=0x7f30bc0afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:54.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.798+0000 7f30bb400640 1 --2- 172.21.15.19:0/3085073657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30bc15b310 0x7f30bc0afdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.798+0000 7f30bb400640 1 -- 172.21.15.19:0/3085073657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30bc0d0780 msgr2=0x7f30bc099820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:54.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.798+0000 7f30bb400640 1 --2- 172.21.15.19:0/3085073657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30bc0d0780 0x7f30bc099820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:54.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.798+0000 7f30bb400640 1 -- 172.21.15.19:0/3085073657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30bc14de80 con 0x7f30bc0b4f90 2024-03-28T13:18:54.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.798+0000 7f30bb400640 1 --2- 172.21.15.19:0/3085073657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0b4f90 0x7f30bc14d610 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f30ac000f80 tx=0x7f30ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:54.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.798+0000 7f30b3e00640 1 -- 172.21.15.19:0/3085073657 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac02e970 con 0x7f30bc0b4f90 2024-03-28T13:18:54.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.798+0000 7f30b3e00640 1 -- 172.21.15.19:0/3085073657 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30ac045070 con 0x7f30bc0b4f90 2024-03-28T13:18:54.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.798+0000 7f30b3e00640 1 -- 172.21.15.19:0/3085073657 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac00b720 con 0x7f30bc0b4f90 2024-03-28T13:18:54.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.798+0000 7f30c2800640 1 -- 172.21.15.19:0/3085073657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30bc154e10 con 0x7f30bc0b4f90 2024-03-28T13:18:54.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.799+0000 7f30c2800640 1 -- 172.21.15.19:0/3085073657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30bc14db50 con 0x7f30bc0b4f90 2024-03-28T13:18:54.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.800+0000 7f30c2800640 1 -- 172.21.15.19:0/3085073657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30bc0685c0 con 0x7f30bc0b4f90 2024-03-28T13:18:54.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.800+0000 7f30b3e00640 1 -- 172.21.15.19:0/3085073657 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f30ac036a60 con 0x7f30bc0b4f90 2024-03-28T13:18:54.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.801+0000 7f30b3e00640 1 --2- 172.21.15.19:0/3085073657 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f309806c560 0x7f309806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:54.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.801+0000 7f30b3e00640 1 -- 172.21.15.19:0/3085073657 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4857+0+0 (secure 0 0 0) 0x7f30ac0adcc0 con 0x7f30bc0b4f90 2024-03-28T13:18:54.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.802+0000 7f30b2a00640 1 --2- 172.21.15.19:0/3085073657 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f309806c560 0x7f309806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:54.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.803+0000 7f30b2a00640 1 --2- 172.21.15.19:0/3085073657 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f309806c560 0x7f309806ea20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f30a4006fd0 tx=0x7f30a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:54.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:54.806+0000 7f30b3e00640 1 -- 172.21.15.19:0/3085073657 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f30ac0b4080 con 0x7f30bc0b4f90 2024-03-28T13:18:55.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.014+0000 7f30c2800640 1 -- 172.21.15.19:0/3085073657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30bc14ec40 con 0x7f30bc0b4f90 2024-03-28T13:18:55.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.015+0000 7f30b3e00640 1 -- 172.21.15.19:0/3085073657 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+12893 (secure 0 0 0) 0x7f30ac04e020 con 0x7f30bc0b4f90 2024-03-28T13:18:55.014 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:55.015 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":101,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:18:54.027590+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.018+0000 7f30c2800640 1 -- 172.21.15.19:0/3085073657 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f309806c560 msgr2=0x7f309806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.018+0000 7f30c2800640 1 --2- 172.21.15.19:0/3085073657 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f309806c560 0x7f309806ea20 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f30a4006fd0 tx=0x7f30a4008040 comp rx=0 tx=0).stop 2024-03-28T13:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.018+0000 7f30c2800640 1 -- 172.21.15.19:0/3085073657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0b4f90 msgr2=0x7f30bc14d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.018+0000 7f30c2800640 1 --2- 172.21.15.19:0/3085073657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0b4f90 0x7f30bc14d610 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f30ac000f80 tx=0x7f30ac004a40 comp rx=0 tx=0).stop 2024-03-28T13:18:55.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.018+0000 7f30c2800640 1 -- 172.21.15.19:0/3085073657 shutdown_connections 2024-03-28T13:18:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.018+0000 7f30c2800640 1 --2- 172.21.15.19:0/3085073657 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f309806c560 0x7f309806ea20 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.018+0000 7f30c2800640 1 --2- 172.21.15.19:0/3085073657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f30bc15b310 0x7f30bc0afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.018+0000 7f30c2800640 1 --2- 172.21.15.19:0/3085073657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f30bc0d0780 0x7f30bc099820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.018+0000 7f30c2800640 1 --2- 172.21.15.19:0/3085073657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f30bc0b4f90 0x7f30bc14d610 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.018+0000 7f30c2800640 1 -- 172.21.15.19:0/3085073657 >> 172.21.15.19:0/3085073657 conn(0x7f30bc06b990 msgr2=0x7f30bc06bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.019+0000 7f30c2800640 1 -- 172.21.15.19:0/3085073657 shutdown_connections 2024-03-28T13:18:55.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.019+0000 7f30c2800640 1 -- 172.21.15.19:0/3085073657 wait complete. 2024-03-28T13:18:55.040 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:18:55.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:18:55.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.262+0000 7fb39f000640 1 Processor -- start 2024-03-28T13:18:55.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.262+0000 7fb39f000640 1 -- start start 2024-03-28T13:18:55.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.264+0000 7fb39f000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb398153580 0x7fb3980d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:55.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.264+0000 7fb39f000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3980d07c0 0x7fb3980aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:55.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.264+0000 7fb39f000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3980afab0 con 0x7fb3980af2e0 2024-03-28T13:18:55.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.264+0000 7fb39f000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb3980d0c00 con 0x7fb398153580 2024-03-28T13:18:55.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.264+0000 7fb39f000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb39815cc80 con 0x7fb3980d07c0 2024-03-28T13:18:55.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.265+0000 7fb38fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3980d07c0 0x7fb3980aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:55.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.265+0000 7fb38fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3980d07c0 0x7fb3980aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:33130/0 (socket says 172.21.15.19:33130) 2024-03-28T13:18:55.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.265+0000 7fb38fe00640 1 -- 172.21.15.19:0/685340911 learned_addr learned my addr 172.21.15.19:0/685340911 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:55.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.265+0000 7fb397e00640 1 --2- 172.21.15.19:0/685340911 >> v2:172.21.15.188:3300/0 conn(0x7fb398153580 0x7fb3980d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:55.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.266+0000 7fb38fe00640 1 -- 172.21.15.19:0/685340911 >> v2:172.21.15.188:3300/0 conn(0x7fb398153580 msgr2=0x7fb3980d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:55.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.266+0000 7fb38fe00640 1 --2- 172.21.15.19:0/685340911 >> v2:172.21.15.188:3300/0 conn(0x7fb398153580 0x7fb3980d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.266+0000 7fb38fe00640 1 -- 172.21.15.19:0/685340911 >> v1:172.21.15.188:6789/0 conn(0x7fb3980af2e0 legacy=0x7fb39806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:55.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.266+0000 7fb38fe00640 1 -- 172.21.15.19:0/685340911 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39815da00 con 0x7fb3980d07c0 2024-03-28T13:18:55.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.267+0000 7fb38fe00640 1 --2- 172.21.15.19:0/685340911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3980d07c0 0x7fb3980aeda0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb380009870 tx=0x7fb38002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd85556e83450d08 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:55.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.268+0000 7fb397400640 1 -- 172.21.15.19:0/685340911 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380035020 con 0x7fb3980d07c0 2024-03-28T13:18:55.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.268+0000 7fb397400640 1 -- 172.21.15.19:0/685340911 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb380031640 con 0x7fb3980d07c0 2024-03-28T13:18:55.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.268+0000 7fb397400640 1 -- 172.21.15.19:0/685340911 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38002d040 con 0x7fb3980d07c0 2024-03-28T13:18:55.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.268+0000 7fb39f000640 1 -- 172.21.15.19:0/685340911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3980d07c0 msgr2=0x7fb3980aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:55.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.268+0000 7fb39f000640 1 --2- 172.21.15.19:0/685340911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3980d07c0 0x7fb3980aeda0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb380009870 tx=0x7fb38002b8f0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.269+0000 7fb39f000640 1 -- 172.21.15.19:0/685340911 shutdown_connections 2024-03-28T13:18:55.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.269+0000 7fb39f000640 1 --2- 172.21.15.19:0/685340911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3980d07c0 0x7fb3980aeda0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.269+0000 7fb39f000640 1 --2- 172.21.15.19:0/685340911 >> v2:172.21.15.188:3300/0 conn(0x7fb398153580 0x7fb3980d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.269+0000 7fb39f000640 1 -- 172.21.15.19:0/685340911 >> 172.21.15.19:0/685340911 conn(0x7fb39806b750 msgr2=0x7fb39806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:55.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.269+0000 7fb39f000640 1 -- 172.21.15.19:0/685340911 shutdown_connections 2024-03-28T13:18:55.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.269+0000 7fb39f000640 1 -- 172.21.15.19:0/685340911 wait complete. 2024-03-28T13:18:55.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.270+0000 7fb39f000640 1 Processor -- start 2024-03-28T13:18:55.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.271+0000 7fb39f000640 1 -- start start 2024-03-28T13:18:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.271+0000 7fb39f000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3980af2e0 0x7fb398151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.272+0000 7fb39f000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3980d07c0 0x7fb3980cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.272+0000 7fb39f000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398153580 0x7fb3980ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.272+0000 7fb39f000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb39815d4e0 con 0x7fb398153580 2024-03-28T13:18:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.272+0000 7fb39f000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb39815d200 con 0x7fb3980af2e0 2024-03-28T13:18:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.272+0000 7fb39f000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb398153960 con 0x7fb3980d07c0 2024-03-28T13:18:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.272+0000 7fb39d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398153580 0x7fb3980ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.272+0000 7fb39d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398153580 0x7fb3980ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32976/0 (socket says 172.21.15.19:32976) 2024-03-28T13:18:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.272+0000 7fb39d000640 1 -- 172.21.15.19:0/1098087769 learned_addr learned my addr 172.21.15.19:0/1098087769 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.272+0000 7fb397e00640 1 --2- 172.21.15.19:0/1098087769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3980af2e0 0x7fb398151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:55.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.272+0000 7fb38fe00640 1 --2- 172.21.15.19:0/1098087769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3980d07c0 0x7fb3980cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:55.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.273+0000 7fb39d000640 1 -- 172.21.15.19:0/1098087769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3980d07c0 msgr2=0x7fb3980cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:55.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.273+0000 7fb39d000640 1 --2- 172.21.15.19:0/1098087769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3980d07c0 0x7fb3980cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.273+0000 7fb39d000640 1 -- 172.21.15.19:0/1098087769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3980af2e0 msgr2=0x7fb398151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:55.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.273+0000 7fb39d000640 1 --2- 172.21.15.19:0/1098087769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3980af2e0 0x7fb398151450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.273+0000 7fb39d000640 1 -- 172.21.15.19:0/1098087769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3980b27c0 con 0x7fb398153580 2024-03-28T13:18:55.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.273+0000 7fb39d000640 1 --2- 172.21.15.19:0/1098087769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398153580 0x7fb3980ce470 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fb39000c6e0 tx=0x7fb39000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:55.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.273+0000 7fb394c00640 1 -- 172.21.15.19:0/1098087769 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390003b70 con 0x7fb398153580 2024-03-28T13:18:55.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.273+0000 7fb394c00640 1 -- 172.21.15.19:0/1098087769 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb390003d30 con 0x7fb398153580 2024-03-28T13:18:55.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.273+0000 7fb394c00640 1 -- 172.21.15.19:0/1098087769 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb390005310 con 0x7fb398153580 2024-03-28T13:18:55.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.273+0000 7fb39f000640 1 -- 172.21.15.19:0/1098087769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3980b2aa0 con 0x7fb398153580 2024-03-28T13:18:55.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.273+0000 7fb39f000640 1 -- 172.21.15.19:0/1098087769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3980b3060 con 0x7fb398153580 2024-03-28T13:18:55.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.275+0000 7fb39f000640 1 -- 172.21.15.19:0/1098087769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3980685c0 con 0x7fb398153580 2024-03-28T13:18:55.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.275+0000 7fb394c00640 1 -- 172.21.15.19:0/1098087769 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb3900054b0 con 0x7fb398153580 2024-03-28T13:18:55.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.277+0000 7fb394c00640 1 --2- 172.21.15.19:0/1098087769 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb37406c530 0x7fb37406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:55.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.277+0000 7fb394c00640 1 -- 172.21.15.19:0/1098087769 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4857+0+0 (secure 0 0 0) 0x7fb39008b160 con 0x7fb398153580 2024-03-28T13:18:55.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.277+0000 7fb397e00640 1 --2- 172.21.15.19:0/1098087769 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb37406c530 0x7fb37406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:55.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.279+0000 7fb397e00640 1 --2- 172.21.15.19:0/1098087769 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb37406c530 0x7fb37406e9f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb39815ae80 tx=0x7fb388009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:55.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.282+0000 7fb394c00640 1 -- 172.21.15.19:0/1098087769 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb390059ac0 con 0x7fb398153580 2024-03-28T13:18:55.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.484+0000 7fb39f000640 1 -- 172.21.15.19:0/1098087769 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb3980cdcf0 con 0x7fb398153580 2024-03-28T13:18:55.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.485+0000 7fb394c00640 1 -- 172.21.15.19:0/1098087769 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7fb3900593c0 con 0x7fb398153580 2024-03-28T13:18:55.485 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:18:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.488+0000 7fb39f000640 1 -- 172.21.15.19:0/1098087769 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb37406c530 msgr2=0x7fb37406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.488+0000 7fb39f000640 1 --2- 172.21.15.19:0/1098087769 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb37406c530 0x7fb37406e9f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb39815ae80 tx=0x7fb388009190 comp rx=0 tx=0).stop 2024-03-28T13:18:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.488+0000 7fb39f000640 1 -- 172.21.15.19:0/1098087769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398153580 msgr2=0x7fb3980ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.488+0000 7fb39f000640 1 --2- 172.21.15.19:0/1098087769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398153580 0x7fb3980ce470 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fb39000c6e0 tx=0x7fb39000cba0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.488+0000 7fb39f000640 1 -- 172.21.15.19:0/1098087769 shutdown_connections 2024-03-28T13:18:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.488+0000 7fb39f000640 1 --2- 172.21.15.19:0/1098087769 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb37406c530 0x7fb37406e9f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.488+0000 7fb39f000640 1 --2- 172.21.15.19:0/1098087769 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb398153580 0x7fb3980ce470 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.488+0000 7fb39f000640 1 --2- 172.21.15.19:0/1098087769 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3980d07c0 0x7fb3980cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.488+0000 7fb39f000640 1 --2- 172.21.15.19:0/1098087769 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3980af2e0 0x7fb398151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.488+0000 7fb39f000640 1 -- 172.21.15.19:0/1098087769 >> 172.21.15.19:0/1098087769 conn(0x7fb39806b750 msgr2=0x7fb398151c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:55.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.488+0000 7fb39f000640 1 -- 172.21.15.19:0/1098087769 shutdown_connections 2024-03-28T13:18:55.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.488+0000 7fb39f000640 1 -- 172.21.15.19:0/1098087769 wait complete. 2024-03-28T13:18:55.507 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-28T13:18:55.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.729+0000 7f3206200640 1 Processor -- start 2024-03-28T13:18:55.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.730+0000 7f3206200640 1 -- start start 2024-03-28T13:18:55.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.731+0000 7f3206200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f320015eca0 0x7f32000b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:55.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.731+0000 7f3206200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f320015ac50 0x7f32000b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:55.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.731+0000 7f3206200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32000b4990 con 0x7f320015a870 2024-03-28T13:18:55.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.731+0000 7f3206200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f320006d5b0 con 0x7f320015ac50 2024-03-28T13:18:55.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.731+0000 7f3206200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f320006db70 con 0x7f320015eca0 2024-03-28T13:18:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.732+0000 7f31ff400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f320015a870 0x7f320015e590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57336/0 (socket says 172.21.15.19:57336) 2024-03-28T13:18:55.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.732+0000 7f31ff400640 1 -- 172.21.15.19:0/2983449785 learned_addr learned my addr 172.21.15.19:0/2983449785 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:55.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.732+0000 7f31fea00640 1 --2- 172.21.15.19:0/2983449785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f320015eca0 0x7f32000b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:55.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.732+0000 7f31ffe00640 1 --2- 172.21.15.19:0/2983449785 >> v2:172.21.15.188:3300/0 conn(0x7f320015ac50 0x7f32000b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:55.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.732+0000 7f31fe000640 1 -- 172.21.15.19:0/2983449785 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3815132658 0 0) 0x7f32000b4990 con 0x7f320015a870 2024-03-28T13:18:55.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.732+0000 7f31fe000640 1 -- 172.21.15.19:0/2983449785 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31e4005730 con 0x7f320015a870 2024-03-28T13:18:55.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31fe000640 1 -- 172.21.15.19:0/2983449785 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 672348601 0 0) 0x7f31e4005730 con 0x7f320015a870 2024-03-28T13:18:55.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31fe000640 1 -- 172.21.15.19:0/2983449785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f320015eca0 msgr2=0x7f32000b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:55.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31fe000640 1 --2- 172.21.15.19:0/2983449785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f320015eca0 0x7f32000b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31fe000640 1 -- 172.21.15.19:0/2983449785 >> v2:172.21.15.188:3300/0 conn(0x7f320015ac50 msgr2=0x7f32000b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:55.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31fe000640 1 --2- 172.21.15.19:0/2983449785 >> v2:172.21.15.188:3300/0 conn(0x7f320015ac50 0x7f32000b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31fe000640 1 -- 172.21.15.19:0/2983449785 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32000b4c00 con 0x7f320015a870 2024-03-28T13:18:55.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31fe000640 1 -- 172.21.15.19:0/2983449785 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f31f0001670 con 0x7f320015a870 2024-03-28T13:18:55.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31fe000640 1 -- 172.21.15.19:0/2983449785 >> v1:172.21.15.19:6789/0 conn(0x7f320015a870 legacy=0x7f320015e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:55.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31fe000640 1 --2- 172.21.15.19:0/2983449785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e4005bc0 0x7f31e4005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:55.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31fe000640 1 --2- 172.21.15.19:0/2983449785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31e4006a10 0x7f31e4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31fe000640 1 --2- 172.21.15.19:0/2983449785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f320015eca0 0x7f31e40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31fe000640 1 -- 172.21.15.19:0/2983449785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f320006d5b0 con 0x7f31e4006a10 2024-03-28T13:18:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31fe000640 1 -- 172.21.15.19:0/2983449785 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f320006db70 con 0x7f31e4005bc0 2024-03-28T13:18:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31fe000640 1 -- 172.21.15.19:0/2983449785 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f31e400dbd0 con 0x7f320015eca0 2024-03-28T13:18:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31fea00640 1 --2- 172.21.15.19:0/2983449785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31e4006a10 0x7f31e4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.733+0000 7f31ff400640 1 --2- 172.21.15.19:0/2983449785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e4005bc0 0x7f31e4005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.734+0000 7f31fea00640 1 -- 172.21.15.19:0/2983449785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f320015eca0 msgr2=0x7f31e40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.734+0000 7f31ffe00640 1 --2- 172.21.15.19:0/2983449785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f320015eca0 0x7f31e40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.734+0000 7f31fea00640 1 --2- 172.21.15.19:0/2983449785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f320015eca0 0x7f31e40073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.734+0000 7f31fea00640 1 -- 172.21.15.19:0/2983449785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e4005bc0 msgr2=0x7f31e4005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.734+0000 7f31fea00640 1 --2- 172.21.15.19:0/2983449785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f31e4005bc0 0x7f31e4005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.734+0000 7f31fea00640 1 -- 172.21.15.19:0/2983449785 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31e4005730 con 0x7f31e4006a10 2024-03-28T13:18:55.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.735+0000 7f31fea00640 1 --2- 172.21.15.19:0/2983449785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31e4006a10 0x7f31e4006e90 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009ac0 tx=0x7f31e4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=affd30b4ef1f7508 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.735+0000 7f31fe000640 1 -- 172.21.15.19:0/2983449785 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec039070 con 0x7f31e4006a10 2024-03-28T13:18:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.735+0000 7f31fe000640 1 -- 172.21.15.19:0/2983449785 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ec02c6b0 con 0x7f31e4006a10 2024-03-28T13:18:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.735+0000 7f31fe000640 1 -- 172.21.15.19:0/2983449785 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec009690 con 0x7f31e4006a10 2024-03-28T13:18:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.736+0000 7f3206200640 1 -- 172.21.15.19:0/2983449785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31e4006a10 msgr2=0x7f31e4006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.736+0000 7f3206200640 1 --2- 172.21.15.19:0/2983449785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31e4006a10 0x7f31e4006e90 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009ac0 tx=0x7f31e4007a40 comp rx=0 tx=0).stop 2024-03-28T13:18:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.736+0000 7f31ffe00640 1 -- 172.21.15.19:0/2983449785 reap_dead start 2024-03-28T13:18:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.736+0000 7f3206200640 1 -- 172.21.15.19:0/2983449785 shutdown_connections 2024-03-28T13:18:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.736+0000 7f3206200640 1 -- 172.21.15.19:0/2983449785 >> 172.21.15.19:0/2983449785 conn(0x7f320006b750 msgr2=0x7f320006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.736+0000 7f3206200640 1 -- 172.21.15.19:0/2983449785 shutdown_connections 2024-03-28T13:18:55.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.736+0000 7f3206200640 1 -- 172.21.15.19:0/2983449785 wait complete. 2024-03-28T13:18:55.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.738+0000 7f3206200640 1 Processor -- start 2024-03-28T13:18:55.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.738+0000 7f3206200640 1 -- start start 2024-03-28T13:18:55.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f3206200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31e4006a10 0x7f32001f0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:55.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f3206200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f320015d1e0 0x7f320015dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:55.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f3206200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32000aeb80 0x7f32000cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:55.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f3206200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32000b3f10 con 0x7f31e4006a10 2024-03-28T13:18:55.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f3206200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f320006d800 con 0x7f320015d1e0 2024-03-28T13:18:55.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f3206200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f32000b4090 con 0x7f32000aeb80 2024-03-28T13:18:55.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f31ff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31e4006a10 0x7f32001f0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:55.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f31ff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31e4006a10 0x7f32001f0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59220/0 (socket says 172.21.15.19:59220) 2024-03-28T13:18:55.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f31ff400640 1 -- 172.21.15.19:0/1055647969 learned_addr learned my addr 172.21.15.19:0/1055647969 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:55.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f31fea00640 1 --2- 172.21.15.19:0/1055647969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f320015d1e0 0x7f320015dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:55.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f31ff400640 1 -- 172.21.15.19:0/1055647969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32000aeb80 msgr2=0x7f32000cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:55.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f31ff400640 1 --2- 172.21.15.19:0/1055647969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32000aeb80 0x7f32000cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f31ff400640 1 -- 172.21.15.19:0/1055647969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f320015d1e0 msgr2=0x7f320015dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:55.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f31ff400640 1 --2- 172.21.15.19:0/1055647969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f320015d1e0 0x7f320015dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f31ff400640 1 -- 172.21.15.19:0/1055647969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f320015c180 con 0x7f31e4006a10 2024-03-28T13:18:55.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.739+0000 7f31fea00640 1 --2- 172.21.15.19:0/1055647969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f320015d1e0 0x7f320015dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:55.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.740+0000 7f31ff400640 1 --2- 172.21.15.19:0/1055647969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31e4006a10 0x7f32001f0d20 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f31f000e730 tx=0x7f31f000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:55.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.740+0000 7f31f7400640 1 -- 172.21.15.19:0/1055647969 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f000ee40 con 0x7f31e4006a10 2024-03-28T13:18:55.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.740+0000 7f31f7400640 1 -- 172.21.15.19:0/1055647969 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f0018070 con 0x7f31e4006a10 2024-03-28T13:18:55.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.740+0000 7f31f7400640 1 -- 172.21.15.19:0/1055647969 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f000ca90 con 0x7f31e4006a10 2024-03-28T13:18:55.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.740+0000 7f3206200640 1 -- 172.21.15.19:0/1055647969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320015c420 con 0x7f31e4006a10 2024-03-28T13:18:55.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.740+0000 7f3206200640 1 -- 172.21.15.19:0/1055647969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f320015c9e0 con 0x7f31e4006a10 2024-03-28T13:18:55.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.741+0000 7f3206200640 1 -- 172.21.15.19:0/1055647969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32000685c0 con 0x7f31e4006a10 2024-03-28T13:18:55.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.742+0000 7f31f7400640 1 -- 172.21.15.19:0/1055647969 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f31f000cc30 con 0x7f31e4006a10 2024-03-28T13:18:55.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.743+0000 7f31f7400640 1 --2- 172.21.15.19:0/1055647969 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f31e006c560 0x7f31e006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:55.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.743+0000 7f31f7400640 1 -- 172.21.15.19:0/1055647969 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4857+0+0 (secure 0 0 0) 0x7f31f008b210 con 0x7f31e4006a10 2024-03-28T13:18:55.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.744+0000 7f31fea00640 1 --2- 172.21.15.19:0/1055647969 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f31e006c560 0x7f31e006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:55.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.745+0000 7f31fea00640 1 --2- 172.21.15.19:0/1055647969 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f31e006c560 0x7f31e006ea20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f31ec036000 tx=0x7f31ec030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:55.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.748+0000 7f31f7400640 1 -- 172.21.15.19:0/1055647969 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f31f0059ae0 con 0x7f31e4006a10 2024-03-28T13:18:55.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.954+0000 7f3206200640 1 -- 172.21.15.19:0/1055647969 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f32000b0570 con 0x7f31e4006a10 2024-03-28T13:18:55.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.955+0000 7f31f7400640 1 -- 172.21.15.19:0/1055647969 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v101) ==== 84+0+65 (secure 0 0 0) 0x7f31f00593e0 con 0x7f31e4006a10 2024-03-28T13:18:55.955 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:55.955 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-03-28T13:18:55.955 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:18:55.955 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-03-28T13:18:55.955 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-03-28T13:18:55.955 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:18:55.956 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-03-28T13:18:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.958+0000 7f3206200640 1 -- 172.21.15.19:0/1055647969 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f31e006c560 msgr2=0x7f31e006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.958+0000 7f3206200640 1 --2- 172.21.15.19:0/1055647969 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f31e006c560 0x7f31e006ea20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f31ec036000 tx=0x7f31ec030000 comp rx=0 tx=0).stop 2024-03-28T13:18:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.958+0000 7f3206200640 1 -- 172.21.15.19:0/1055647969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31e4006a10 msgr2=0x7f32001f0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.958+0000 7f3206200640 1 --2- 172.21.15.19:0/1055647969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31e4006a10 0x7f32001f0d20 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f31f000e730 tx=0x7f31f000ebf0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.958+0000 7f3206200640 1 -- 172.21.15.19:0/1055647969 shutdown_connections 2024-03-28T13:18:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.958+0000 7f3206200640 1 --2- 172.21.15.19:0/1055647969 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f31e006c560 0x7f31e006ea20 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.958+0000 7f3206200640 1 --2- 172.21.15.19:0/1055647969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f32000aeb80 0x7f32000cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.958+0000 7f3206200640 1 --2- 172.21.15.19:0/1055647969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f320015d1e0 0x7f320015dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.958+0000 7f3206200640 1 --2- 172.21.15.19:0/1055647969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f31e4006a10 0x7f32001f0d20 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.958+0000 7f3206200640 1 -- 172.21.15.19:0/1055647969 >> 172.21.15.19:0/1055647969 conn(0x7f320006b750 msgr2=0x7f3200155ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.958+0000 7f3206200640 1 -- 172.21.15.19:0/1055647969 shutdown_connections 2024-03-28T13:18:55.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:55.958+0000 7f3206200640 1 -- 172.21.15.19:0/1055647969 wait complete. 2024-03-28T13:18:55.977 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-28T13:18:55.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-28T13:18:56.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.194+0000 7f7a2fc00640 1 Processor -- start 2024-03-28T13:18:56.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.194+0000 7f7a2fc00640 1 -- start start 2024-03-28T13:18:56.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.196+0000 7f7a2fc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7a280acf10 0x7f7a2809dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:56.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.196+0000 7f7a2fc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7a28071710 0x7f7a28071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:56.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.196+0000 7f7a2fc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a2815cb90 con 0x7f7a280ad2f0 2024-03-28T13:18:56.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.196+0000 7f7a2fc00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7a28145f60 con 0x7f7a280acf10 2024-03-28T13:18:56.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.196+0000 7f7a2fc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7a2815c950 con 0x7f7a28071710 2024-03-28T13:18:56.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.196+0000 7f7a2d200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7a280acf10 0x7f7a2809dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:56.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.197+0000 7f7a2d200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7a280acf10 0x7f7a2809dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59232/0 (socket says 172.21.15.19:59232) 2024-03-28T13:18:56.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.197+0000 7f7a2d200640 1 -- 172.21.15.19:0/2281832318 learned_addr learned my addr 172.21.15.19:0/2281832318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:56.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.197+0000 7f7a2d200640 1 -- 172.21.15.19:0/2281832318 >> v2:172.21.15.188:3300/0 conn(0x7f7a28071710 msgr2=0x7f7a28071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:56.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.197+0000 7f7a2d200640 1 --2- 172.21.15.19:0/2281832318 >> v2:172.21.15.188:3300/0 conn(0x7f7a28071710 0x7f7a28071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:56.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.197+0000 7f7a2d200640 1 -- 172.21.15.19:0/2281832318 >> v1:172.21.15.188:6789/0 conn(0x7f7a280ad2f0 legacy=0x7f7a281453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:56.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.197+0000 7f7a2d200640 1 -- 172.21.15.19:0/2281832318 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2815ce70 con 0x7f7a280acf10 2024-03-28T13:18:56.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.198+0000 7f7a2d200640 1 --2- 172.21.15.19:0/2281832318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a280acf10 0x7f7a2809dcf0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f7a280aeb70 tx=0x7f7a1802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa853427745d4f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:56.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.198+0000 7f7a27400640 1 -- 172.21.15.19:0/2281832318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1802c9f0 con 0x7f7a280acf10 2024-03-28T13:18:56.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.198+0000 7f7a27400640 1 -- 172.21.15.19:0/2281832318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a1802cb90 con 0x7f7a280acf10 2024-03-28T13:18:56.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.198+0000 7f7a27400640 1 -- 172.21.15.19:0/2281832318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a180343f0 con 0x7f7a280acf10 2024-03-28T13:18:56.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.199+0000 7f7a2fc00640 1 -- 172.21.15.19:0/2281832318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a280acf10 msgr2=0x7f7a2809dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:56.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.199+0000 7f7a2fc00640 1 --2- 172.21.15.19:0/2281832318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a280acf10 0x7f7a2809dcf0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f7a280aeb70 tx=0x7f7a1802be50 comp rx=0 tx=0).stop 2024-03-28T13:18:56.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.199+0000 7f7a2fc00640 1 -- 172.21.15.19:0/2281832318 shutdown_connections 2024-03-28T13:18:56.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.199+0000 7f7a2fc00640 1 --2- 172.21.15.19:0/2281832318 >> v2:172.21.15.188:3300/0 conn(0x7f7a28071710 0x7f7a28071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:56.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.199+0000 7f7a2fc00640 1 --2- 172.21.15.19:0/2281832318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a280acf10 0x7f7a2809dcf0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:56.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.199+0000 7f7a2fc00640 1 -- 172.21.15.19:0/2281832318 >> 172.21.15.19:0/2281832318 conn(0x7f7a2806bba0 msgr2=0x7f7a280b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:56.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.199+0000 7f7a2fc00640 1 -- 172.21.15.19:0/2281832318 shutdown_connections 2024-03-28T13:18:56.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.199+0000 7f7a2fc00640 1 -- 172.21.15.19:0/2281832318 wait complete. 2024-03-28T13:18:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.201+0000 7f7a2fc00640 1 Processor -- start 2024-03-28T13:18:56.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.201+0000 7f7a2fc00640 1 -- start start 2024-03-28T13:18:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.202+0000 7f7a2fc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a28071710 0x7f7a2813f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.202+0000 7f7a2fc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a280acf10 0x7f7a2815b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.202+0000 7f7a2fc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a280ad2f0 0x7f7a2815e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.202+0000 7f7a2fc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2815d430 con 0x7f7a28071710 2024-03-28T13:18:56.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.202+0000 7f7a2fc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2815d7a0 con 0x7f7a280acf10 2024-03-28T13:18:56.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.202+0000 7f7a2fc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7a280ad7a0 con 0x7f7a280ad2f0 2024-03-28T13:18:56.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.203+0000 7f7a2d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a28071710 0x7f7a2813f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:56.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.203+0000 7f7a2d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a28071710 0x7f7a2813f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59244/0 (socket says 172.21.15.19:59244) 2024-03-28T13:18:56.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.203+0000 7f7a2d200640 1 -- 172.21.15.19:0/652400344 learned_addr learned my addr 172.21.15.19:0/652400344 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:56.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.203+0000 7f7a2dc00640 1 --2- 172.21.15.19:0/652400344 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a280ad2f0 0x7f7a2815e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:56.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.203+0000 7f7a2d200640 1 -- 172.21.15.19:0/652400344 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a280ad2f0 msgr2=0x7f7a2815e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:56.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.203+0000 7f7a2d200640 1 --2- 172.21.15.19:0/652400344 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a280ad2f0 0x7f7a2815e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:56.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.203+0000 7f7a2d200640 1 -- 172.21.15.19:0/652400344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a280acf10 msgr2=0x7f7a2815b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:56.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.203+0000 7f7a27e00640 1 --2- 172.21.15.19:0/652400344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a280acf10 0x7f7a2815b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:56.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.203+0000 7f7a2d200640 1 --2- 172.21.15.19:0/652400344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a280acf10 0x7f7a2815b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:56.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.203+0000 7f7a2d200640 1 -- 172.21.15.19:0/652400344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a281f48a0 con 0x7f7a28071710 2024-03-28T13:18:56.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.203+0000 7f7a2dc00640 1 --2- 172.21.15.19:0/652400344 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a280ad2f0 0x7f7a2815e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:56.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.204+0000 7f7a2d200640 1 --2- 172.21.15.19:0/652400344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a28071710 0x7f7a2813f800 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f7a180093e0 tx=0x7f7a18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:56.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.204+0000 7f7a24c00640 1 -- 172.21.15.19:0/652400344 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18043020 con 0x7f7a28071710 2024-03-28T13:18:56.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.204+0000 7f7a24c00640 1 -- 172.21.15.19:0/652400344 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a1803e450 con 0x7f7a28071710 2024-03-28T13:18:56.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.204+0000 7f7a24c00640 1 -- 172.21.15.19:0/652400344 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1803e840 con 0x7f7a28071710 2024-03-28T13:18:56.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.204+0000 7f7a2fc00640 1 -- 172.21.15.19:0/652400344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2813a360 con 0x7f7a28071710 2024-03-28T13:18:56.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.204+0000 7f7a2fc00640 1 -- 172.21.15.19:0/652400344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2815db40 con 0x7f7a28071710 2024-03-28T13:18:56.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.205+0000 7f7a2fc00640 1 -- 172.21.15.19:0/652400344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a280685c0 con 0x7f7a28071710 2024-03-28T13:18:56.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.206+0000 7f7a24c00640 1 -- 172.21.15.19:0/652400344 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7a18046020 con 0x7f7a28071710 2024-03-28T13:18:56.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.207+0000 7f7a24c00640 1 --2- 172.21.15.19:0/652400344 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a0806c530 0x7f7a0806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:56.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.207+0000 7f7a24c00640 1 -- 172.21.15.19:0/652400344 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4857+0+0 (secure 0 0 0) 0x7f7a180ade30 con 0x7f7a28071710 2024-03-28T13:18:56.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.208+0000 7f7a27e00640 1 --2- 172.21.15.19:0/652400344 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a0806c530 0x7f7a0806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:56.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.209+0000 7f7a27e00640 1 --2- 172.21.15.19:0/652400344 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a0806c530 0x7f7a0806e9f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7a280b18b0 tx=0x7f7a14008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:56.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.212+0000 7f7a24c00640 1 -- 172.21.15.19:0/652400344 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7a1804d030 con 0x7f7a28071710 2024-03-28T13:18:56.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:56.417+0000 7f7a2fc00640 1 -- 172.21.15.19:0/652400344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7a280b2900 con 0x7f7a28071710 2024-03-28T13:18:57.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.026+0000 7f7a24c00640 1 -- 172.21.15.19:0/652400344 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v102) ==== 138+0+0 (secure 0 0 0) 0x7f7a1807c700 con 0x7f7a28071710 2024-03-28T13:18:57.026 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-03-28T13:18:57.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.030+0000 7f7a2fc00640 1 -- 172.21.15.19:0/652400344 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a0806c530 msgr2=0x7f7a0806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:57.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.030+0000 7f7a2fc00640 1 --2- 172.21.15.19:0/652400344 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a0806c530 0x7f7a0806e9f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7a280b18b0 tx=0x7f7a14008040 comp rx=0 tx=0).stop 2024-03-28T13:18:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.030+0000 7f7a2fc00640 1 -- 172.21.15.19:0/652400344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a28071710 msgr2=0x7f7a2813f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.030+0000 7f7a2fc00640 1 --2- 172.21.15.19:0/652400344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a28071710 0x7f7a2813f800 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f7a180093e0 tx=0x7f7a18004a40 comp rx=0 tx=0).stop 2024-03-28T13:18:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.031+0000 7f7a2fc00640 1 -- 172.21.15.19:0/652400344 shutdown_connections 2024-03-28T13:18:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.031+0000 7f7a2fc00640 1 --2- 172.21.15.19:0/652400344 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a0806c530 0x7f7a0806e9f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.031+0000 7f7a2fc00640 1 --2- 172.21.15.19:0/652400344 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a280ad2f0 0x7f7a2815e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.031+0000 7f7a2fc00640 1 --2- 172.21.15.19:0/652400344 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a280acf10 0x7f7a2815b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.031+0000 7f7a2fc00640 1 --2- 172.21.15.19:0/652400344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a28071710 0x7f7a2813f800 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.031+0000 7f7a2fc00640 1 -- 172.21.15.19:0/652400344 >> 172.21.15.19:0/652400344 conn(0x7f7a2806bba0 msgr2=0x7f7a2809d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.031+0000 7f7a2fc00640 1 -- 172.21.15.19:0/652400344 shutdown_connections 2024-03-28T13:18:57.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.031+0000 7f7a2fc00640 1 -- 172.21.15.19:0/652400344 wait complete. 2024-03-28T13:18:57.062 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-28T13:18:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.292+0000 7f6870e00640 1 Processor -- start 2024-03-28T13:18:57.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.292+0000 7f6870e00640 1 -- start start 2024-03-28T13:18:57.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.294+0000 7f6870e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f686c15e420 0x7f686c0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:57.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.294+0000 7f6870e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f686c0b3460 0x7f686c0d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:57.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.294+0000 7f6870e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f686c0d2fd0 con 0x7f686c0d18c0 2024-03-28T13:18:57.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.294+0000 7f6870e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f686c0d1cf0 con 0x7f686c0b3460 2024-03-28T13:18:57.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.294+0000 7f6870e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f686c0cfa70 con 0x7f686c15e420 2024-03-28T13:18:57.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.295+0000 7f686b400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f686c15e420 0x7f686c0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:57.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.295+0000 7f686be00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f686c0d18c0 0x7f686c161ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:39722/0 (socket says 172.21.15.19:39722) 2024-03-28T13:18:57.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.295+0000 7f686be00640 1 -- 172.21.15.19:0/2618582035 learned_addr learned my addr 172.21.15.19:0/2618582035 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:57.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.295+0000 7f6862a00640 1 --2- 172.21.15.19:0/2618582035 >> v2:172.21.15.188:3300/0 conn(0x7f686c0b3460 0x7f686c0d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:57.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.296+0000 7f686b400640 1 -- 172.21.15.19:0/2618582035 >> v2:172.21.15.188:3300/0 conn(0x7f686c0b3460 msgr2=0x7f686c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:57.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.296+0000 7f686b400640 1 --2- 172.21.15.19:0/2618582035 >> v2:172.21.15.188:3300/0 conn(0x7f686c0b3460 0x7f686c0d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:57.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.296+0000 7f686b400640 1 -- 172.21.15.19:0/2618582035 >> v1:172.21.15.188:6789/0 conn(0x7f686c0d18c0 legacy=0x7f686c161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:57.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.296+0000 7f686b400640 1 -- 172.21.15.19:0/2618582035 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686c0cf790 con 0x7f686c15e420 2024-03-28T13:18:57.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.297+0000 7f686b400640 1 --2- 172.21.15.19:0/2618582035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f686c15e420 0x7f686c0b2f20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f685c009d90 tx=0x7f685c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ebae55b8565c85d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:57.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.298+0000 7f686aa00640 1 -- 172.21.15.19:0/2618582035 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c039070 con 0x7f686c15e420 2024-03-28T13:18:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.298+0000 7f686aa00640 1 -- 172.21.15.19:0/2618582035 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f685c02cc50 con 0x7f686c15e420 2024-03-28T13:18:57.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.298+0000 7f686aa00640 1 -- 172.21.15.19:0/2618582035 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c034440 con 0x7f686c15e420 2024-03-28T13:18:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.298+0000 7f6870e00640 1 -- 172.21.15.19:0/2618582035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f686c15e420 msgr2=0x7f686c0b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.298+0000 7f6870e00640 1 --2- 172.21.15.19:0/2618582035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f686c15e420 0x7f686c0b2f20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f685c009d90 tx=0x7f685c02bea0 comp rx=0 tx=0).stop 2024-03-28T13:18:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.299+0000 7f6870e00640 1 -- 172.21.15.19:0/2618582035 shutdown_connections 2024-03-28T13:18:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.299+0000 7f6870e00640 1 --2- 172.21.15.19:0/2618582035 >> v2:172.21.15.188:3300/0 conn(0x7f686c0b3460 0x7f686c0d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.299+0000 7f6870e00640 1 --2- 172.21.15.19:0/2618582035 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f686c15e420 0x7f686c0b2f20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.299+0000 7f6870e00640 1 -- 172.21.15.19:0/2618582035 >> 172.21.15.19:0/2618582035 conn(0x7f686c067670 msgr2=0x7f686c067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.299+0000 7f6870e00640 1 -- 172.21.15.19:0/2618582035 shutdown_connections 2024-03-28T13:18:57.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.299+0000 7f6870e00640 1 -- 172.21.15.19:0/2618582035 wait complete. 2024-03-28T13:18:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.301+0000 7f6870e00640 1 Processor -- start 2024-03-28T13:18:57.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.301+0000 7f6870e00640 1 -- start start 2024-03-28T13:18:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.302+0000 7f6870e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f686c0b3460 0x7f686c152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.302+0000 7f6870e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f686c0d18c0 0x7f686c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.302+0000 7f6870e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f686c15e420 0x7f686c160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.302+0000 7f6870e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f686c15e800 con 0x7f686c15e420 2024-03-28T13:18:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.302+0000 7f6870e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f686c0d3550 con 0x7f686c0d18c0 2024-03-28T13:18:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.302+0000 7f6870e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f686c0cf420 con 0x7f686c0b3460 2024-03-28T13:18:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.302+0000 7f686be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f686c15e420 0x7f686c160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.302+0000 7f686be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f686c15e420 0x7f686c160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59248/0 (socket says 172.21.15.19:59248) 2024-03-28T13:18:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.302+0000 7f686be00640 1 -- 172.21.15.19:0/4163202491 learned_addr learned my addr 172.21.15.19:0/4163202491 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:57.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.302+0000 7f6862a00640 1 --2- 172.21.15.19:0/4163202491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f686c0d18c0 0x7f686c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:57.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.302+0000 7f686b400640 1 --2- 172.21.15.19:0/4163202491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f686c0b3460 0x7f686c152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.303+0000 7f686be00640 1 -- 172.21.15.19:0/4163202491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f686c0b3460 msgr2=0x7f686c152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.303+0000 7f686be00640 1 --2- 172.21.15.19:0/4163202491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f686c0b3460 0x7f686c152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.303+0000 7f686be00640 1 -- 172.21.15.19:0/4163202491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f686c0d18c0 msgr2=0x7f686c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.303+0000 7f686be00640 1 --2- 172.21.15.19:0/4163202491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f686c0d18c0 0x7f686c0ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.303+0000 7f686be00640 1 -- 172.21.15.19:0/4163202491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686c0cd850 con 0x7f686c15e420 2024-03-28T13:18:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.303+0000 7f6862a00640 1 --2- 172.21.15.19:0/4163202491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f686c0d18c0 0x7f686c0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:18:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.303+0000 7f686be00640 1 --2- 172.21.15.19:0/4163202491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f686c15e420 0x7f686c160eb0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f686400c420 tx=0x7f686400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.304+0000 7f6863e00640 1 -- 172.21.15.19:0/4163202491 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864017070 con 0x7f686c15e420 2024-03-28T13:18:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.304+0000 7f6863e00640 1 -- 172.21.15.19:0/4163202491 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6864005ce0 con 0x7f686c15e420 2024-03-28T13:18:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.304+0000 7f6863e00640 1 -- 172.21.15.19:0/4163202491 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68640072d0 con 0x7f686c15e420 2024-03-28T13:18:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.304+0000 7f6870e00640 1 -- 172.21.15.19:0/4163202491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686c0cdb30 con 0x7f686c15e420 2024-03-28T13:18:57.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.304+0000 7f6870e00640 1 -- 172.21.15.19:0/4163202491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f686c1534f0 con 0x7f686c15e420 2024-03-28T13:18:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.305+0000 7f6870e00640 1 -- 172.21.15.19:0/4163202491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f686c075940 con 0x7f686c15e420 2024-03-28T13:18:57.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.306+0000 7f6863e00640 1 -- 172.21.15.19:0/4163202491 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f686401a070 con 0x7f686c15e420 2024-03-28T13:18:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.307+0000 7f6863e00640 1 --2- 172.21.15.19:0/4163202491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f684806c530 0x7f684806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.307+0000 7f6863e00640 1 -- 172.21.15.19:0/4163202491 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5227+0+0 (secure 0 0 0) 0x7f686408a8b0 con 0x7f686c15e420 2024-03-28T13:18:57.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.308+0000 7f686b400640 1 --2- 172.21.15.19:0/4163202491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f684806c530 0x7f684806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:57.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.309+0000 7f686b400640 1 --2- 172.21.15.19:0/4163202491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f684806c530 0x7f684806e9f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f685c036000 tx=0x7f685c030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:57.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.312+0000 7f6863e00640 1 -- 172.21.15.19:0/4163202491 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6864058940 con 0x7f686c15e420 2024-03-28T13:18:57.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:57.522+0000 7f6870e00640 1 -- 172.21.15.19:0/4163202491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f686c0d4590 con 0x7f686c15e420 2024-03-28T13:18:58.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.043+0000 7f6863e00640 1 -- 172.21.15.19:0/4163202491 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v103) ==== 170+0+0 (secure 0 0 0) 0x7f686405cc40 con 0x7f686c15e420 2024-03-28T13:18:58.044 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-03-28T13:18:58.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.047+0000 7f6870e00640 1 -- 172.21.15.19:0/4163202491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f684806c530 msgr2=0x7f684806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:58.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.047+0000 7f6870e00640 1 --2- 172.21.15.19:0/4163202491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f684806c530 0x7f684806e9f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f685c036000 tx=0x7f685c030000 comp rx=0 tx=0).stop 2024-03-28T13:18:58.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.047+0000 7f6870e00640 1 -- 172.21.15.19:0/4163202491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f686c15e420 msgr2=0x7f686c160eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:58.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.047+0000 7f6870e00640 1 --2- 172.21.15.19:0/4163202491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f686c15e420 0x7f686c160eb0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f686400c420 tx=0x7f686400c8e0 comp rx=0 tx=0).stop 2024-03-28T13:18:58.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.047+0000 7f6870e00640 1 -- 172.21.15.19:0/4163202491 shutdown_connections 2024-03-28T13:18:58.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.047+0000 7f6870e00640 1 --2- 172.21.15.19:0/4163202491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f684806c530 0x7f684806e9f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:58.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.047+0000 7f6870e00640 1 --2- 172.21.15.19:0/4163202491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f686c15e420 0x7f686c160eb0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:58.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.047+0000 7f6870e00640 1 --2- 172.21.15.19:0/4163202491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f686c0d18c0 0x7f686c0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:58.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.047+0000 7f6870e00640 1 --2- 172.21.15.19:0/4163202491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f686c0b3460 0x7f686c152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:58.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.047+0000 7f6870e00640 1 -- 172.21.15.19:0/4163202491 >> 172.21.15.19:0/4163202491 conn(0x7f686c067670 msgr2=0x7f686c0b3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:58.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.048+0000 7f6870e00640 1 -- 172.21.15.19:0/4163202491 shutdown_connections 2024-03-28T13:18:58.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.048+0000 7f6870e00640 1 -- 172.21.15.19:0/4163202491 wait complete. 2024-03-28T13:18:58.085 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-28T13:18:58.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.277+0000 7fb253000640 1 Processor -- start 2024-03-28T13:18:58.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.277+0000 7fb253000640 1 -- start start 2024-03-28T13:18:58.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.278+0000 7fb253000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb24c141c10 0x7fb24c071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:58.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.278+0000 7fb253000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb24c15ec80 con 0x7fb24c071eb0 2024-03-28T13:18:58.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.278+0000 7fb253000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb24c15ef20 con 0x7fb24c0d0280 2024-03-28T13:18:58.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.278+0000 7fb253000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb24c15f200 con 0x7fb24c141c10 2024-03-28T13:18:58.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.279+0000 7fb251000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb24c071eb0 0x7fb24c162160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57346/0 (socket says 172.21.15.19:57346) 2024-03-28T13:18:58.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.279+0000 7fb251000640 1 -- 172.21.15.19:0/134596107 learned_addr learned my addr 172.21.15.19:0/134596107 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:18:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.279+0000 7fb24b400640 1 --2- 172.21.15.19:0/134596107 >> v2:172.21.15.188:3300/0 conn(0x7fb24c141c10 0x7fb24c071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.279+0000 7fb24aa00640 1 -- 172.21.15.19:0/134596107 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1626744524 0 0) 0x7fb24c15ec80 con 0x7fb24c071eb0 2024-03-28T13:18:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.279+0000 7fb24aa00640 1 -- 172.21.15.19:0/134596107 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb230003520 con 0x7fb24c071eb0 2024-03-28T13:18:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.280+0000 7fb24aa00640 1 -- 172.21.15.19:0/134596107 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1039945619 0 0) 0x7fb230003520 con 0x7fb24c071eb0 2024-03-28T13:18:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.280+0000 7fb24aa00640 1 -- 172.21.15.19:0/134596107 >> v2:172.21.15.188:3300/0 conn(0x7fb24c141c10 msgr2=0x7fb24c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.280+0000 7fb24aa00640 1 --2- 172.21.15.19:0/134596107 >> v2:172.21.15.188:3300/0 conn(0x7fb24c141c10 0x7fb24c071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.280+0000 7fb24aa00640 1 -- 172.21.15.19:0/134596107 >> v1:172.21.15.188:6789/0 conn(0x7fb24c0d0280 legacy=0x7fb24c0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.280+0000 7fb24aa00640 1 -- 172.21.15.19:0/134596107 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c15f4e0 con 0x7fb24c071eb0 2024-03-28T13:18:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.280+0000 7fb24aa00640 1 -- 172.21.15.19:0/134596107 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fb2440019f0 con 0x7fb24c071eb0 2024-03-28T13:18:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.280+0000 7fb24aa00640 1 -- 172.21.15.19:0/134596107 >> v1:172.21.15.19:6789/0 conn(0x7fb24c071eb0 legacy=0x7fb24c162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.280+0000 7fb24aa00640 1 --2- 172.21.15.19:0/134596107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2300039b0 0x7fb230003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.280+0000 7fb24aa00640 1 --2- 172.21.15.19:0/134596107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb230004800 0x7fb230004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.280+0000 7fb24aa00640 1 --2- 172.21.15.19:0/134596107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2300051c0 0x7fb23000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:58.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.280+0000 7fb24aa00640 1 -- 172.21.15.19:0/134596107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c15f200 con 0x7fb2300051c0 2024-03-28T13:18:58.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.280+0000 7fb24aa00640 1 -- 172.21.15.19:0/134596107 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb23000c060 con 0x7fb2300039b0 2024-03-28T13:18:58.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.280+0000 7fb24aa00640 1 -- 172.21.15.19:0/134596107 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb23000c260 con 0x7fb230004800 2024-03-28T13:18:58.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.280+0000 7fb251000640 1 --2- 172.21.15.19:0/134596107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2300051c0 0x7fb23000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:58.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.280+0000 7fb24be00640 1 --2- 172.21.15.19:0/134596107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2300039b0 0x7fb230003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:58.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.281+0000 7fb24b400640 1 --2- 172.21.15.19:0/134596107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb230004800 0x7fb230004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:58.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.281+0000 7fb251000640 1 -- 172.21.15.19:0/134596107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb230004800 msgr2=0x7fb230004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:58.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.281+0000 7fb251000640 1 --2- 172.21.15.19:0/134596107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb230004800 0x7fb230004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:58.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.281+0000 7fb251000640 1 -- 172.21.15.19:0/134596107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2300039b0 msgr2=0x7fb230003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:58.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.281+0000 7fb251000640 1 --2- 172.21.15.19:0/134596107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2300039b0 0x7fb230003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:58.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.281+0000 7fb251000640 1 -- 172.21.15.19:0/134596107 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb230003520 con 0x7fb2300051c0 2024-03-28T13:18:58.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.282+0000 7fb251000640 1 -- 172.21.15.19:0/134596107 reap_dead start 2024-03-28T13:18:58.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.282+0000 7fb251000640 1 --2- 172.21.15.19:0/134596107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2300051c0 0x7fb23000b9c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fb24400a840 tx=0x7fb230005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d0a2ecfdb0c2ac server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:58.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.282+0000 7fb24aa00640 1 -- 172.21.15.19:0/134596107 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24400a460 con 0x7fb2300051c0 2024-03-28T13:18:58.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.282+0000 7fb24aa00640 1 -- 172.21.15.19:0/134596107 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb244034650 con 0x7fb2300051c0 2024-03-28T13:18:58.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.282+0000 7fb24aa00640 1 -- 172.21.15.19:0/134596107 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244034a40 con 0x7fb2300051c0 2024-03-28T13:18:58.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.283+0000 7fb253000640 1 -- 172.21.15.19:0/134596107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2300051c0 msgr2=0x7fb23000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:58.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.283+0000 7fb253000640 1 --2- 172.21.15.19:0/134596107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2300051c0 0x7fb23000b9c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fb24400a840 tx=0x7fb230005830 comp rx=0 tx=0).stop 2024-03-28T13:18:58.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.283+0000 7fb253000640 1 -- 172.21.15.19:0/134596107 shutdown_connections 2024-03-28T13:18:58.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.283+0000 7fb253000640 1 --2- 172.21.15.19:0/134596107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb2300051c0 0x7fb23000b9c0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:58.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.283+0000 7fb253000640 1 -- 172.21.15.19:0/134596107 >> 172.21.15.19:0/134596107 conn(0x7fb24c06b750 msgr2=0x7fb24c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:58.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.283+0000 7fb253000640 1 -- 172.21.15.19:0/134596107 shutdown_connections 2024-03-28T13:18:58.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.283+0000 7fb253000640 1 -- 172.21.15.19:0/134596107 wait complete. 2024-03-28T13:18:58.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.285+0000 7fb253000640 1 Processor -- start 2024-03-28T13:18:58.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.285+0000 7fb253000640 1 -- start start 2024-03-28T13:18:58.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.286+0000 7fb253000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2300051c0 0x7fb24c09dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:58.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb253000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb24c1432e0 0x7fb24c1f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:58.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb253000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c143760 0x7fb24c1f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:58.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb253000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c158e90 con 0x7fb24c143760 2024-03-28T13:18:58.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb253000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb24c06f340 con 0x7fb2300051c0 2024-03-28T13:18:58.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb253000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb24c14f550 con 0x7fb24c1432e0 2024-03-28T13:18:58.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb251000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c143760 0x7fb24c1f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:58.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb251000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c143760 0x7fb24c1f78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59262/0 (socket says 172.21.15.19:59262) 2024-03-28T13:18:58.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb251000640 1 -- 172.21.15.19:0/466986775 learned_addr learned my addr 172.21.15.19:0/466986775 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:58.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb24be00640 1 --2- 172.21.15.19:0/466986775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2300051c0 0x7fb24c09dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:58.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb24b400640 1 --2- 172.21.15.19:0/466986775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb24c1432e0 0x7fb24c1f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:58.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb251000640 1 -- 172.21.15.19:0/466986775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb24c1432e0 msgr2=0x7fb24c1f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:58.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb251000640 1 --2- 172.21.15.19:0/466986775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb24c1432e0 0x7fb24c1f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:58.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb251000640 1 -- 172.21.15.19:0/466986775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2300051c0 msgr2=0x7fb24c09dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:58.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb251000640 1 --2- 172.21.15.19:0/466986775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2300051c0 0x7fb24c09dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:58.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb251000640 1 -- 172.21.15.19:0/466986775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24c09cff0 con 0x7fb24c143760 2024-03-28T13:18:58.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.287+0000 7fb24be00640 1 --2- 172.21.15.19:0/466986775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2300051c0 0x7fb24c09dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:18:58.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.288+0000 7fb251000640 1 --2- 172.21.15.19:0/466986775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c143760 0x7fb24c1f78e0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb244001cf0 tx=0x7fb24c15f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:58.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.288+0000 7fb243e00640 1 -- 172.21.15.19:0/466986775 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244039040 con 0x7fb24c143760 2024-03-28T13:18:58.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.288+0000 7fb243e00640 1 -- 172.21.15.19:0/466986775 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb244043070 con 0x7fb24c143760 2024-03-28T13:18:58.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.288+0000 7fb243e00640 1 -- 172.21.15.19:0/466986775 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb244004d60 con 0x7fb24c143760 2024-03-28T13:18:58.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.288+0000 7fb253000640 1 -- 172.21.15.19:0/466986775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24c09d2d0 con 0x7fb24c143760 2024-03-28T13:18:58.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.288+0000 7fb253000640 1 -- 172.21.15.19:0/466986775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb24c09e010 con 0x7fb24c143760 2024-03-28T13:18:58.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.289+0000 7fb253000640 1 -- 172.21.15.19:0/466986775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb24c06e8e0 con 0x7fb24c143760 2024-03-28T13:18:58.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.291+0000 7fb243e00640 1 -- 172.21.15.19:0/466986775 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb244034650 con 0x7fb24c143760 2024-03-28T13:18:58.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.291+0000 7fb243e00640 1 --2- 172.21.15.19:0/466986775 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb22c06c560 0x7fb22c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:58.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.292+0000 7fb243e00640 1 -- 172.21.15.19:0/466986775 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5609+0+0 (secure 0 0 0) 0x7fb2440ad440 con 0x7fb24c143760 2024-03-28T13:18:58.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.292+0000 7fb24be00640 1 --2- 172.21.15.19:0/466986775 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb22c06c560 0x7fb22c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:58.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.293+0000 7fb24be00640 1 --2- 172.21.15.19:0/466986775 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb22c06c560 0x7fb22c06ea20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb24c147c30 tx=0x7fb23c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:58.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.297+0000 7fb243e00640 1 -- 172.21.15.19:0/466986775 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb24407bab0 con 0x7fb24c143760 2024-03-28T13:18:58.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:58.539+0000 7fb253000640 1 -- 172.21.15.19:0/466986775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb24c0685c0 con 0x7fb24c143760 2024-03-28T13:18:59.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.072+0000 7fb243e00640 1 -- 172.21.15.19:0/466986775 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v79) ==== 174+0+0 (secure 0 0 0) 0x7fb24407b3b0 con 0x7fb24c143760 2024-03-28T13:18:59.072 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 14 and data pool 15 2024-03-28T13:18:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.077+0000 7fb253000640 1 -- 172.21.15.19:0/466986775 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb22c06c560 msgr2=0x7fb22c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.077+0000 7fb253000640 1 --2- 172.21.15.19:0/466986775 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb22c06c560 0x7fb22c06ea20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb24c147c30 tx=0x7fb23c009040 comp rx=0 tx=0).stop 2024-03-28T13:18:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.077+0000 7fb253000640 1 -- 172.21.15.19:0/466986775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c143760 msgr2=0x7fb24c1f78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:59.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.077+0000 7fb253000640 1 --2- 172.21.15.19:0/466986775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c143760 0x7fb24c1f78e0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fb244001cf0 tx=0x7fb24c15f6b0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.080+0000 7fb253000640 1 -- 172.21.15.19:0/466986775 shutdown_connections 2024-03-28T13:18:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.080+0000 7fb253000640 1 --2- 172.21.15.19:0/466986775 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb22c06c560 0x7fb22c06ea20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.080+0000 7fb253000640 1 --2- 172.21.15.19:0/466986775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb24c143760 0x7fb24c1f78e0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.080+0000 7fb253000640 1 --2- 172.21.15.19:0/466986775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb24c1432e0 0x7fb24c1f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.080+0000 7fb253000640 1 --2- 172.21.15.19:0/466986775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb2300051c0 0x7fb24c09dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.080+0000 7fb253000640 1 -- 172.21.15.19:0/466986775 >> 172.21.15.19:0/466986775 conn(0x7fb24c06b750 msgr2=0x7fb24c0b2eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:59.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.081+0000 7fb253000640 1 -- 172.21.15.19:0/466986775 shutdown_connections 2024-03-28T13:18:59.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.081+0000 7fb253000640 1 -- 172.21.15.19:0/466986775 wait complete. 2024-03-28T13:18:59.118 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:18:59.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.301+0000 7f340d400640 1 Processor -- start 2024-03-28T13:18:59.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.301+0000 7f340d400640 1 -- start start 2024-03-28T13:18:59.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.302+0000 7f340d400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f34080af830 0x7f3408073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:59.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.302+0000 7f340d400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f34080d0650 0x7f34080d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:59.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.302+0000 7f340d400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f340815fe80 con 0x7f340815c390 2024-03-28T13:18:59.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.302+0000 7f340d400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f340815c770 con 0x7f34080af830 2024-03-28T13:18:59.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.302+0000 7f340d400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f34080d0fc0 con 0x7f34080d0650 2024-03-28T13:18:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.303+0000 7f3405e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f34080af830 0x7f3408073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.303+0000 7f3405e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f34080af830 0x7f3408073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59276/0 (socket says 172.21.15.19:59276) 2024-03-28T13:18:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.303+0000 7f3405e00640 1 -- 172.21.15.19:0/4040445192 learned_addr learned my addr 172.21.15.19:0/4040445192 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.303+0000 7f3407200640 1 --2- 172.21.15.19:0/4040445192 >> v2:172.21.15.188:3300/0 conn(0x7f34080d0650 0x7f34080d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.303+0000 7f3405e00640 1 -- 172.21.15.19:0/4040445192 >> v2:172.21.15.188:3300/0 conn(0x7f34080d0650 msgr2=0x7f34080d0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:59.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.303+0000 7f3405e00640 1 --2- 172.21.15.19:0/4040445192 >> v2:172.21.15.188:3300/0 conn(0x7f34080d0650 0x7f34080d0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.303+0000 7f3405e00640 1 -- 172.21.15.19:0/4040445192 >> v1:172.21.15.188:6789/0 conn(0x7f340815c390 legacy=0x7f3408160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:59.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.303+0000 7f3405e00640 1 -- 172.21.15.19:0/4040445192 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34080ad120 con 0x7f34080af830 2024-03-28T13:18:59.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.303+0000 7f3407200640 1 --2- 172.21.15.19:0/4040445192 >> v2:172.21.15.188:3300/0 conn(0x7f34080d0650 0x7f34080d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:59.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.304+0000 7f3405e00640 1 --2- 172.21.15.19:0/4040445192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34080af830 0x7f3408073790 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f340806eaa0 tx=0x7f33f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb091351b591a7e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:59.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.305+0000 7f3405400640 1 -- 172.21.15.19:0/4040445192 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4035020 con 0x7f34080af830 2024-03-28T13:18:59.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.305+0000 7f3405400640 1 -- 172.21.15.19:0/4040445192 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33f4031830 con 0x7f34080af830 2024-03-28T13:18:59.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.305+0000 7f3405400640 1 -- 172.21.15.19:0/4040445192 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4031c00 con 0x7f34080af830 2024-03-28T13:18:59.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.305+0000 7f340d400640 1 -- 172.21.15.19:0/4040445192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34080af830 msgr2=0x7f3408073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:59.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.305+0000 7f340d400640 1 --2- 172.21.15.19:0/4040445192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34080af830 0x7f3408073790 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f340806eaa0 tx=0x7f33f402b8d0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.306+0000 7f340d400640 1 -- 172.21.15.19:0/4040445192 shutdown_connections 2024-03-28T13:18:59.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.306+0000 7f340d400640 1 --2- 172.21.15.19:0/4040445192 >> v2:172.21.15.188:3300/0 conn(0x7f34080d0650 0x7f34080d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.306+0000 7f340d400640 1 --2- 172.21.15.19:0/4040445192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34080af830 0x7f3408073790 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.306+0000 7f340d400640 1 -- 172.21.15.19:0/4040445192 >> 172.21.15.19:0/4040445192 conn(0x7f340806b990 msgr2=0x7f340806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:59.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.306+0000 7f340d400640 1 -- 172.21.15.19:0/4040445192 shutdown_connections 2024-03-28T13:18:59.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.306+0000 7f340d400640 1 -- 172.21.15.19:0/4040445192 wait complete. 2024-03-28T13:18:59.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.307+0000 7f340d400640 1 Processor -- start 2024-03-28T13:18:59.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.308+0000 7f340d400640 1 -- start start 2024-03-28T13:18:59.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.308+0000 7f340d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34080af830 0x7f340815e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:59.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.308+0000 7f340d400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34080d0650 0x7f34081f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:59.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.308+0000 7f340d400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340815c390 0x7f340814e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:59.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.308+0000 7f340d400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3408160120 con 0x7f34080af830 2024-03-28T13:18:59.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.308+0000 7f340d400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f34080ad6e0 con 0x7f340815c390 2024-03-28T13:18:59.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.308+0000 7f340d400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3408160d50 con 0x7f34080d0650 2024-03-28T13:18:59.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.309+0000 7f3406800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34080af830 0x7f340815e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:59.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.309+0000 7f3406800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34080af830 0x7f340815e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59284/0 (socket says 172.21.15.19:59284) 2024-03-28T13:18:59.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.309+0000 7f3406800640 1 -- 172.21.15.19:0/1671715223 learned_addr learned my addr 172.21.15.19:0/1671715223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:59.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.309+0000 7f3407200640 1 --2- 172.21.15.19:0/1671715223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340815c390 0x7f340814e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:59.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.309+0000 7f3405e00640 1 --2- 172.21.15.19:0/1671715223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34080d0650 0x7f34081f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:59.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.309+0000 7f3406800640 1 -- 172.21.15.19:0/1671715223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34080d0650 msgr2=0x7f34081f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:59.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.309+0000 7f3406800640 1 --2- 172.21.15.19:0/1671715223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34080d0650 0x7f34081f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.309+0000 7f3406800640 1 -- 172.21.15.19:0/1671715223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340815c390 msgr2=0x7f340814e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:59.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.309+0000 7f3406800640 1 --2- 172.21.15.19:0/1671715223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340815c390 0x7f340814e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.309+0000 7f3406800640 1 -- 172.21.15.19:0/1671715223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340814ede0 con 0x7f34080af830 2024-03-28T13:18:59.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.310+0000 7f3406800640 1 --2- 172.21.15.19:0/1671715223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34080af830 0x7f340815e6b0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f33f800c460 tx=0x7f33f800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:59.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.310+0000 7f33fea00640 1 -- 172.21.15.19:0/1671715223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8016020 con 0x7f34080af830 2024-03-28T13:18:59.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.310+0000 7f33fea00640 1 -- 172.21.15.19:0/1671715223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33f8012870 con 0x7f34080af830 2024-03-28T13:18:59.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.310+0000 7f340d400640 1 -- 172.21.15.19:0/1671715223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340815f150 con 0x7f34080af830 2024-03-28T13:18:59.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.310+0000 7f33fea00640 1 -- 172.21.15.19:0/1671715223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8012c60 con 0x7f34080af830 2024-03-28T13:18:59.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.310+0000 7f340d400640 1 -- 172.21.15.19:0/1671715223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340815f6d0 con 0x7f34080af830 2024-03-28T13:18:59.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.311+0000 7f340d400640 1 -- 172.21.15.19:0/1671715223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34080685c0 con 0x7f34080af830 2024-03-28T13:18:59.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.312+0000 7f33fea00640 1 -- 172.21.15.19:0/1671715223 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f33f8019020 con 0x7f34080af830 2024-03-28T13:18:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.313+0000 7f33fea00640 1 --2- 172.21.15.19:0/1671715223 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33e806c560 0x7f33e806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.313+0000 7f33fea00640 1 -- 172.21.15.19:0/1671715223 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f33f808b190 con 0x7f34080af830 2024-03-28T13:18:59.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.314+0000 7f3405e00640 1 --2- 172.21.15.19:0/1671715223 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33e806c560 0x7f33e806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:59.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.315+0000 7f3405e00640 1 --2- 172.21.15.19:0/1671715223 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33e806c560 0x7f33e806ea20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f33f4036000 tx=0x7f33f403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:59.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.318+0000 7f33fea00640 1 -- 172.21.15.19:0/1671715223 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f33f8020070 con 0x7f34080af830 2024-03-28T13:18:59.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.514+0000 7f340d400640 1 -- 172.21.15.19:0/1671715223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f340806f1b0 con 0x7f34080af830 2024-03-28T13:18:59.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.518+0000 7f33fea00640 1 -- 172.21.15.19:0/1671715223 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16632 (secure 0 0 0) 0x7f33f80597a0 con 0x7f34080af830 2024-03-28T13:18:59.518 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:59.518 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":104,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:18:58.541028+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:18:56.432417+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-28T13:18:57.540515+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:18:59.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.520+0000 7f340d400640 1 -- 172.21.15.19:0/1671715223 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33e806c560 msgr2=0x7f33e806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:59.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.520+0000 7f340d400640 1 --2- 172.21.15.19:0/1671715223 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33e806c560 0x7f33e806ea20 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f33f4036000 tx=0x7f33f403c040 comp rx=0 tx=0).stop 2024-03-28T13:18:59.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.520+0000 7f340d400640 1 -- 172.21.15.19:0/1671715223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34080af830 msgr2=0x7f340815e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:59.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.520+0000 7f340d400640 1 --2- 172.21.15.19:0/1671715223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34080af830 0x7f340815e6b0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f33f800c460 tx=0x7f33f800c920 comp rx=0 tx=0).stop 2024-03-28T13:18:59.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.520+0000 7f340d400640 1 -- 172.21.15.19:0/1671715223 shutdown_connections 2024-03-28T13:18:59.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.520+0000 7f340d400640 1 --2- 172.21.15.19:0/1671715223 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f33e806c560 0x7f33e806ea20 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.520+0000 7f340d400640 1 --2- 172.21.15.19:0/1671715223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340815c390 0x7f340814e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.520+0000 7f340d400640 1 --2- 172.21.15.19:0/1671715223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34080d0650 0x7f34081f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.520+0000 7f340d400640 1 --2- 172.21.15.19:0/1671715223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34080af830 0x7f340815e6b0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.520+0000 7f340d400640 1 -- 172.21.15.19:0/1671715223 >> 172.21.15.19:0/1671715223 conn(0x7f340806b990 msgr2=0x7f340806d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:59.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.520+0000 7f340d400640 1 -- 172.21.15.19:0/1671715223 shutdown_connections 2024-03-28T13:18:59.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.521+0000 7f340d400640 1 -- 172.21.15.19:0/1671715223 wait complete. 2024-03-28T13:18:59.538 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:18:59.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.732+0000 7f35ddc00640 1 Processor -- start 2024-03-28T13:18:59.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.732+0000 7f35ddc00640 1 -- start start 2024-03-28T13:18:59.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.734+0000 7f35ddc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f35d81442c0 0x7f35d80ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:59.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.734+0000 7f35ddc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35d80d02a0 0x7f35d80b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:59.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.734+0000 7f35ddc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d8071460 con 0x7f35d80cefe0 2024-03-28T13:18:59.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.734+0000 7f35ddc00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f35d8071d20 con 0x7f35d80d02a0 2024-03-28T13:18:59.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.734+0000 7f35ddc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f35d80cf420 con 0x7f35d81442c0 2024-03-28T13:18:59.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.734+0000 7f35dd200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35d80d02a0 0x7f35d80b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:59.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.734+0000 7f35dd200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f35d80d02a0 0x7f35d80b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59286/0 (socket says 172.21.15.19:59286) 2024-03-28T13:18:59.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.734+0000 7f35dd200640 1 -- 172.21.15.19:0/3713035891 learned_addr learned my addr 172.21.15.19:0/3713035891 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:59.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.734+0000 7f35d7e00640 1 --2- 172.21.15.19:0/3713035891 >> v2:172.21.15.188:3300/0 conn(0x7f35d81442c0 0x7f35d80ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:59.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.735+0000 7f35dd200640 1 -- 172.21.15.19:0/3713035891 >> v2:172.21.15.188:3300/0 conn(0x7f35d81442c0 msgr2=0x7f35d80ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:59.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.735+0000 7f35dd200640 1 --2- 172.21.15.19:0/3713035891 >> v2:172.21.15.188:3300/0 conn(0x7f35d81442c0 0x7f35d80ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.735+0000 7f35dd200640 1 -- 172.21.15.19:0/3713035891 >> v1:172.21.15.188:6789/0 conn(0x7f35d80cefe0 legacy=0x7f35d80cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:18:59.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.735+0000 7f35dd200640 1 -- 172.21.15.19:0/3713035891 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d80721b0 con 0x7f35d80d02a0 2024-03-28T13:18:59.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.736+0000 7f35dd200640 1 --2- 172.21.15.19:0/3713035891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d80d02a0 0x7f35d80b3b20 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f35d80d0ae0 tx=0x7f35d002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b98fe8d04934ef server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:59.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.736+0000 7f35d6a00640 1 -- 172.21.15.19:0/3713035891 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d002cb20 con 0x7f35d80d02a0 2024-03-28T13:18:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.736+0000 7f35d6a00640 1 -- 172.21.15.19:0/3713035891 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35d002ccc0 con 0x7f35d80d02a0 2024-03-28T13:18:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.736+0000 7f35d6a00640 1 -- 172.21.15.19:0/3713035891 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0034610 con 0x7f35d80d02a0 2024-03-28T13:18:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.737+0000 7f35ddc00640 1 -- 172.21.15.19:0/3713035891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d80d02a0 msgr2=0x7f35d80b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.737+0000 7f35ddc00640 1 --2- 172.21.15.19:0/3713035891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d80d02a0 0x7f35d80b3b20 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f35d80d0ae0 tx=0x7f35d002bf80 comp rx=0 tx=0).stop 2024-03-28T13:18:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.737+0000 7f35ddc00640 1 -- 172.21.15.19:0/3713035891 shutdown_connections 2024-03-28T13:18:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.737+0000 7f35ddc00640 1 --2- 172.21.15.19:0/3713035891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d80d02a0 0x7f35d80b3b20 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.737+0000 7f35ddc00640 1 --2- 172.21.15.19:0/3713035891 >> v2:172.21.15.188:3300/0 conn(0x7f35d81442c0 0x7f35d80ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.737+0000 7f35ddc00640 1 -- 172.21.15.19:0/3713035891 >> 172.21.15.19:0/3713035891 conn(0x7f35d80ccad0 msgr2=0x7f35d80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.737+0000 7f35ddc00640 1 -- 172.21.15.19:0/3713035891 shutdown_connections 2024-03-28T13:18:59.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.737+0000 7f35ddc00640 1 -- 172.21.15.19:0/3713035891 wait complete. 2024-03-28T13:18:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.739+0000 7f35ddc00640 1 Processor -- start 2024-03-28T13:18:59.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.739+0000 7f35ddc00640 1 -- start start 2024-03-28T13:18:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.740+0000 7f35ddc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d80cefe0 0x7f35d81f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.740+0000 7f35ddc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d80d02a0 0x7f35d80b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.740+0000 7f35ddc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35d81442c0 0x7f35d80b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.740+0000 7f35ddc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35d80d0770 con 0x7f35d80d02a0 2024-03-28T13:18:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.740+0000 7f35ddc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f35d80719b0 con 0x7f35d80cefe0 2024-03-28T13:18:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.740+0000 7f35ddc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f35d81446a0 con 0x7f35d81442c0 2024-03-28T13:18:59.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.740+0000 7f35d7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d80d02a0 0x7f35d80b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:59.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.740+0000 7f35d7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d80d02a0 0x7f35d80b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59296/0 (socket says 172.21.15.19:59296) 2024-03-28T13:18:59.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.740+0000 7f35d7400640 1 -- 172.21.15.19:0/733523917 learned_addr learned my addr 172.21.15.19:0/733523917 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:18:59.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.740+0000 7f35d7e00640 1 --2- 172.21.15.19:0/733523917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d80cefe0 0x7f35d81f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:59.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.741+0000 7f35dd200640 1 --2- 172.21.15.19:0/733523917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35d81442c0 0x7f35d80b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:59.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.741+0000 7f35d7400640 1 -- 172.21.15.19:0/733523917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35d81442c0 msgr2=0x7f35d80b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:59.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.741+0000 7f35d7400640 1 --2- 172.21.15.19:0/733523917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35d81442c0 0x7f35d80b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.741+0000 7f35d7400640 1 -- 172.21.15.19:0/733523917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d80cefe0 msgr2=0x7f35d81f83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:59.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.741+0000 7f35d7400640 1 --2- 172.21.15.19:0/733523917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d80cefe0 0x7f35d81f83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.741+0000 7f35d7400640 1 -- 172.21.15.19:0/733523917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d8142640 con 0x7f35d80d02a0 2024-03-28T13:18:59.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.741+0000 7f35dd200640 1 --2- 172.21.15.19:0/733523917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35d81442c0 0x7f35d80b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:18:59.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.741+0000 7f35d7400640 1 --2- 172.21.15.19:0/733523917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d80d02a0 0x7f35d80b1e50 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f35c400b420 tx=0x7f35c400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:59.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.741+0000 7f35cfe00640 1 -- 172.21.15.19:0/733523917 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4011830 con 0x7f35d80d02a0 2024-03-28T13:18:59.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.741+0000 7f35cfe00640 1 -- 172.21.15.19:0/733523917 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c4016070 con 0x7f35d80d02a0 2024-03-28T13:18:59.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.741+0000 7f35ddc00640 1 -- 172.21.15.19:0/733523917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d815dc90 con 0x7f35d80d02a0 2024-03-28T13:18:59.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.741+0000 7f35cfe00640 1 -- 172.21.15.19:0/733523917 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4011c10 con 0x7f35d80d02a0 2024-03-28T13:18:59.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.741+0000 7f35ddc00640 1 -- 172.21.15.19:0/733523917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d815e250 con 0x7f35d80d02a0 2024-03-28T13:18:59.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.743+0000 7f35ddc00640 1 -- 172.21.15.19:0/733523917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d8075940 con 0x7f35d80d02a0 2024-03-28T13:18:59.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.744+0000 7f35cfe00640 1 -- 172.21.15.19:0/733523917 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f35c401b460 con 0x7f35d80d02a0 2024-03-28T13:18:59.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.745+0000 7f35cfe00640 1 --2- 172.21.15.19:0/733523917 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f35b806c530 0x7f35b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:18:59.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.745+0000 7f35cfe00640 1 -- 172.21.15.19:0/733523917 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f35c408b2a0 con 0x7f35d80d02a0 2024-03-28T13:18:59.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.745+0000 7f35d7e00640 1 --2- 172.21.15.19:0/733523917 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f35b806c530 0x7f35b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:18:59.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.747+0000 7f35d7e00640 1 --2- 172.21.15.19:0/733523917 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f35b806c530 0x7f35b806e9f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f35c8006fd0 tx=0x7f35c8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:18:59.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.750+0000 7f35cfe00640 1 -- 172.21.15.19:0/733523917 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f35c4020020 con 0x7f35d80d02a0 2024-03-28T13:18:59.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.959+0000 7f35ddc00640 1 -- 172.21.15.19:0/733523917 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35d80cdb40 con 0x7f35d80d02a0 2024-03-28T13:18:59.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.962+0000 7f35cfe00640 1 -- 172.21.15.19:0/733523917 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16632 (secure 0 0 0) 0x7f35d8142640 con 0x7f35d80d02a0 2024-03-28T13:18:59.962 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:18:59.962 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":104,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:18:58.541028+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:18:56.432417+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-28T13:18:57.540515+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:18:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.964+0000 7f35ddc00640 1 -- 172.21.15.19:0/733523917 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f35b806c530 msgr2=0x7f35b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.964+0000 7f35ddc00640 1 --2- 172.21.15.19:0/733523917 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f35b806c530 0x7f35b806e9f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f35c8006fd0 tx=0x7f35c8008040 comp rx=0 tx=0).stop 2024-03-28T13:18:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.964+0000 7f35ddc00640 1 -- 172.21.15.19:0/733523917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d80d02a0 msgr2=0x7f35d80b1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:18:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.964+0000 7f35ddc00640 1 --2- 172.21.15.19:0/733523917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d80d02a0 0x7f35d80b1e50 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f35c400b420 tx=0x7f35c400b8e0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.965+0000 7f35ddc00640 1 -- 172.21.15.19:0/733523917 shutdown_connections 2024-03-28T13:18:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.965+0000 7f35ddc00640 1 --2- 172.21.15.19:0/733523917 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f35b806c530 0x7f35b806e9f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.965+0000 7f35ddc00640 1 --2- 172.21.15.19:0/733523917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35d81442c0 0x7f35d80b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.965+0000 7f35ddc00640 1 --2- 172.21.15.19:0/733523917 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35d80d02a0 0x7f35d80b1e50 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.965+0000 7f35ddc00640 1 --2- 172.21.15.19:0/733523917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35d80cefe0 0x7f35d81f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:18:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.965+0000 7f35ddc00640 1 -- 172.21.15.19:0/733523917 >> 172.21.15.19:0/733523917 conn(0x7f35d80ccad0 msgr2=0x7f35d813d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:18:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.965+0000 7f35ddc00640 1 -- 172.21.15.19:0/733523917 shutdown_connections 2024-03-28T13:18:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:18:59.965+0000 7f35ddc00640 1 -- 172.21.15.19:0/733523917 wait complete. 2024-03-28T13:18:59.984 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-28T13:19:00.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.232+0000 7fd2f6000640 1 Processor -- start 2024-03-28T13:19:00.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.232+0000 7fd2f6000640 1 -- start start 2024-03-28T13:19:00.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.233+0000 7fd2f6000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f00b0670 0x7fd2f015f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:00.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.234+0000 7fd2f6000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd2f006bc80 0x7fd2f0168450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:00.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.234+0000 7fd2f6000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2f00b5b10 con 0x7fd2f0070120 2024-03-28T13:19:00.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.234+0000 7fd2f6000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fd2f00b63e0 con 0x7fd2f006bc80 2024-03-28T13:19:00.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.234+0000 7fd2f6000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd2f006c3f0 con 0x7fd2f00b0670 2024-03-28T13:19:00.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.234+0000 7fd2e7e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd2f0070120 0x7fd2f006b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57358/0 (socket says 172.21.15.19:57358) 2024-03-28T13:19:00.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.234+0000 7fd2e7e00640 1 -- 172.21.15.19:0/2762241775 learned_addr learned my addr 172.21.15.19:0/2762241775 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:00.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.234+0000 7fd2efe00640 1 --2- 172.21.15.19:0/2762241775 >> v2:172.21.15.188:3300/0 conn(0x7fd2f006bc80 0x7fd2f0168450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:00.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.234+0000 7fd2ef400640 1 --2- 172.21.15.19:0/2762241775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f00b0670 0x7fd2f015f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:00.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.234+0000 7fd2eea00640 1 -- 172.21.15.19:0/2762241775 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 171243851 0 0) 0x7fd2d8001110 con 0x7fd2f0070120 2024-03-28T13:19:00.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2eea00640 1 -- 172.21.15.19:0/2762241775 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2d4003520 con 0x7fd2f0070120 2024-03-28T13:19:00.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2eea00640 1 -- 172.21.15.19:0/2762241775 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2481136189 0 0) 0x7fd2d4003520 con 0x7fd2f0070120 2024-03-28T13:19:00.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2eea00640 1 -- 172.21.15.19:0/2762241775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f00b0670 msgr2=0x7fd2f015f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:00.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2eea00640 1 --2- 172.21.15.19:0/2762241775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f00b0670 0x7fd2f015f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:00.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2eea00640 1 -- 172.21.15.19:0/2762241775 >> v2:172.21.15.188:3300/0 conn(0x7fd2f006bc80 msgr2=0x7fd2f0168450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:00.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2eea00640 1 --2- 172.21.15.19:0/2762241775 >> v2:172.21.15.188:3300/0 conn(0x7fd2f006bc80 0x7fd2f0168450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:00.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2eea00640 1 -- 172.21.15.19:0/2762241775 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f015fdf0 con 0x7fd2f0070120 2024-03-28T13:19:00.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2eea00640 1 -- 172.21.15.19:0/2762241775 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fd2d8001860 con 0x7fd2f0070120 2024-03-28T13:19:00.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2eea00640 1 -- 172.21.15.19:0/2762241775 >> v1:172.21.15.19:6789/0 conn(0x7fd2f0070120 legacy=0x7fd2f006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:00.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2eea00640 1 --2- 172.21.15.19:0/2762241775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d40039e0 0x7fd2d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:00.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2eea00640 1 --2- 172.21.15.19:0/2762241775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d4004800 0x7fd2d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:00.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2eea00640 1 --2- 172.21.15.19:0/2762241775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f00b0670 0x7fd2d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:00.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2eea00640 1 -- 172.21.15.19:0/2762241775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f00b63e0 con 0x7fd2d4004800 2024-03-28T13:19:00.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2eea00640 1 -- 172.21.15.19:0/2762241775 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f006c3f0 con 0x7fd2d40039e0 2024-03-28T13:19:00.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2eea00640 1 -- 172.21.15.19:0/2762241775 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d400b9c0 con 0x7fd2f00b0670 2024-03-28T13:19:00.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.235+0000 7fd2e7e00640 1 --2- 172.21.15.19:0/2762241775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d4004800 0x7fd2d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.236+0000 7fd2ef400640 1 --2- 172.21.15.19:0/2762241775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d40039e0 0x7fd2d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.236+0000 7fd2efe00640 1 --2- 172.21.15.19:0/2762241775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f00b0670 0x7fd2d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.236+0000 7fd2e7e00640 1 -- 172.21.15.19:0/2762241775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f00b0670 msgr2=0x7fd2d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.236+0000 7fd2e7e00640 1 --2- 172.21.15.19:0/2762241775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f00b0670 0x7fd2d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.236+0000 7fd2e7e00640 1 -- 172.21.15.19:0/2762241775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d40039e0 msgr2=0x7fd2d4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.236+0000 7fd2e7e00640 1 --2- 172.21.15.19:0/2762241775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d40039e0 0x7fd2d4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.236+0000 7fd2e7e00640 1 -- 172.21.15.19:0/2762241775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d4003520 con 0x7fd2d4004800 2024-03-28T13:19:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.236+0000 7fd2efe00640 1 --2- 172.21.15.19:0/2762241775 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f00b0670 0x7fd2d40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.236+0000 7fd2ef400640 1 --2- 172.21.15.19:0/2762241775 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d40039e0 0x7fd2d4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:00.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.237+0000 7fd2e7e00640 1 --2- 172.21.15.19:0/2762241775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d4004800 0x7fd2d4004c80 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8007fd0 tx=0x7fd2d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7812060fee8bea4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:00.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.237+0000 7fd2eea00640 1 -- 172.21.15.19:0/2762241775 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8007e30 con 0x7fd2d4004800 2024-03-28T13:19:00.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.237+0000 7fd2eea00640 1 -- 172.21.15.19:0/2762241775 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2d8030070 con 0x7fd2d4004800 2024-03-28T13:19:00.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.237+0000 7fd2eea00640 1 -- 172.21.15.19:0/2762241775 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d802ce00 con 0x7fd2d4004800 2024-03-28T13:19:00.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.238+0000 7fd2f6000640 1 -- 172.21.15.19:0/2762241775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d4004800 msgr2=0x7fd2d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:00.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.238+0000 7fd2f6000640 1 --2- 172.21.15.19:0/2762241775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2d4004800 0x7fd2d4004c80 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8007fd0 tx=0x7fd2d4005830 comp rx=0 tx=0).stop 2024-03-28T13:19:00.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.238+0000 7fd2efe00640 1 -- 172.21.15.19:0/2762241775 reap_dead start 2024-03-28T13:19:00.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.238+0000 7fd2f6000640 1 -- 172.21.15.19:0/2762241775 shutdown_connections 2024-03-28T13:19:00.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.238+0000 7fd2f6000640 1 -- 172.21.15.19:0/2762241775 >> 172.21.15.19:0/2762241775 conn(0x7fd2f00b65a0 msgr2=0x7fd2f00b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:00.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.238+0000 7fd2f6000640 1 -- 172.21.15.19:0/2762241775 shutdown_connections 2024-03-28T13:19:00.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.238+0000 7fd2f6000640 1 -- 172.21.15.19:0/2762241775 wait complete. 2024-03-28T13:19:00.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.241+0000 7fd2f6000640 1 Processor -- start 2024-03-28T13:19:00.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.241+0000 7fd2f6000640 1 -- start start 2024-03-28T13:19:00.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2f6000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d4004800 0x7fd2f00d0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:00.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2f6000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2f013f550 0x7fd2f00cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:00.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2f6000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f013fbf0 0x7fd2f00b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:00.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2f6000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f00b6070 con 0x7fd2f013f550 2024-03-28T13:19:00.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2f6000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f00b5d90 con 0x7fd2d4004800 2024-03-28T13:19:00.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2f6000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd2f00b0a50 con 0x7fd2f013fbf0 2024-03-28T13:19:00.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2e7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2f013f550 0x7fd2f00cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:00.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2e7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2f013f550 0x7fd2f00cea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59322/0 (socket says 172.21.15.19:59322) 2024-03-28T13:19:00.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2e7e00640 1 -- 172.21.15.19:0/1768242889 learned_addr learned my addr 172.21.15.19:0/1768242889 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:00.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2ef400640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d4004800 0x7fd2f00d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:00.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2efe00640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f013fbf0 0x7fd2f00b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:00.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2e7e00640 1 -- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f013fbf0 msgr2=0x7fd2f00b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:00.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2e7e00640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f013fbf0 0x7fd2f00b3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:00.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2e7e00640 1 -- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d4004800 msgr2=0x7fd2f00d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:00.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2e7e00640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d4004800 0x7fd2f00d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:00.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2e7e00640 1 -- 172.21.15.19:0/1768242889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f0140440 con 0x7fd2f013f550 2024-03-28T13:19:00.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2ef400640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d4004800 0x7fd2f00d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:00.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.243+0000 7fd2efe00640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f013fbf0 0x7fd2f00b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:00.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.244+0000 7fd2e7e00640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2f013f550 0x7fd2f00cea40 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800c3b0 tx=0x7fd2f015ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:00.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.244+0000 7fd2e7400640 1 -- 172.21.15.19:0/1768242889 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8045070 con 0x7fd2f013f550 2024-03-28T13:19:00.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.244+0000 7fd2e7400640 1 -- 172.21.15.19:0/1768242889 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2d8004c10 con 0x7fd2f013f550 2024-03-28T13:19:00.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.244+0000 7fd2e7400640 1 -- 172.21.15.19:0/1768242889 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8007e30 con 0x7fd2f013f550 2024-03-28T13:19:00.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.244+0000 7fd2f6000640 1 -- 172.21.15.19:0/1768242889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f013f9b0 con 0x7fd2f013f550 2024-03-28T13:19:00.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.244+0000 7fd2f6000640 1 -- 172.21.15.19:0/1768242889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f00cdf60 con 0x7fd2f013f550 2024-03-28T13:19:00.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.245+0000 7fd2f6000640 1 -- 172.21.15.19:0/1768242889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2f00b5410 con 0x7fd2f013f550 2024-03-28T13:19:00.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.246+0000 7fd2e7400640 1 -- 172.21.15.19:0/1768242889 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd2d8007070 con 0x7fd2f013f550 2024-03-28T13:19:00.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.247+0000 7fd2e7400640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd2cc06c530 0x7fd2cc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:00.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.248+0000 7fd2e7400640 1 -- 172.21.15.19:0/1768242889 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7fd2d80ac770 con 0x7fd2f013f550 2024-03-28T13:19:00.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.248+0000 7fd2ef400640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd2cc06c530 0x7fd2cc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:00.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.249+0000 7fd2ef400640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd2cc06c530 0x7fd2cc06e9f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00099c0 tx=0x7fd2e0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:00.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.252+0000 7fd2e7400640 1 -- 172.21.15.19:0/1768242889 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd2f00b5410 con 0x7fd2f013f550 2024-03-28T13:19:00.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:00.495+0000 7fd2f6000640 1 -- 172.21.15.19:0/1768242889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd2f0068cd0 con 0x7fd2f013f550 2024-03-28T13:19:01.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.085+0000 7fd2e7400640 1 -- 172.21.15.19:0/1768242889 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v82) ==== 118+0+0 (secure 0 0 0) 0x7fd2f015fdf0 con 0x7fd2f013f550 2024-03-28T13:19:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.089+0000 7fd2f6000640 1 -- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd2cc06c530 msgr2=0x7fd2cc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.089+0000 7fd2f6000640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd2cc06c530 0x7fd2cc06e9f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00099c0 tx=0x7fd2e0008040 comp rx=0 tx=0).stop 2024-03-28T13:19:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.089+0000 7fd2f6000640 1 -- 172.21.15.19:0/1768242889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2f013f550 msgr2=0x7fd2f00cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.089+0000 7fd2f6000640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2f013f550 0x7fd2f00cea40 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800c3b0 tx=0x7fd2f015ee30 comp rx=0 tx=0).stop 2024-03-28T13:19:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.089+0000 7fd2f6000640 1 -- 172.21.15.19:0/1768242889 shutdown_connections 2024-03-28T13:19:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.089+0000 7fd2f6000640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd2cc06c530 0x7fd2cc06e9f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.089+0000 7fd2f6000640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2f013fbf0 0x7fd2f00b3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.089+0000 7fd2f6000640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd2f013f550 0x7fd2f00cea40 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.089+0000 7fd2f6000640 1 --2- 172.21.15.19:0/1768242889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d4004800 0x7fd2f00d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:01.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.089+0000 7fd2f6000640 1 -- 172.21.15.19:0/1768242889 >> 172.21.15.19:0/1768242889 conn(0x7fd2f00b65a0 msgr2=0x7fd2f00b0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:01.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.089+0000 7fd2f6000640 1 -- 172.21.15.19:0/1768242889 shutdown_connections 2024-03-28T13:19:01.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.090+0000 7fd2f6000640 1 -- 172.21.15.19:0/1768242889 wait complete. 2024-03-28T13:19:01.115 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-28T13:19:01.115 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-28T13:19:01.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.343+0000 7f7cf9c00640 1 Processor -- start 2024-03-28T13:19:01.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.343+0000 7f7cf9c00640 1 -- start start 2024-03-28T13:19:01.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.344+0000 7f7cf9c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf415e460 0x7f7cf40b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:01.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.344+0000 7f7cf9c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7cf40d3e40 0x7f7cf40d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:01.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.344+0000 7f7cf9c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cf40b8320 con 0x7f7cf40cef20 2024-03-28T13:19:01.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.344+0000 7f7cf9c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7cf40d2da0 con 0x7f7cf40d3e40 2024-03-28T13:19:01.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.344+0000 7f7cf9c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf40d3080 con 0x7f7cf415e460 2024-03-28T13:19:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.345+0000 7f7cebe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7cf40d3e40 0x7f7cf40d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.345+0000 7f7cebe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7cf40d3e40 0x7f7cf40d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59332/0 (socket says 172.21.15.19:59332) 2024-03-28T13:19:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.345+0000 7f7cebe00640 1 -- 172.21.15.19:0/997732783 learned_addr learned my addr 172.21.15.19:0/997732783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.345+0000 7f7cf3e00640 1 --2- 172.21.15.19:0/997732783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf415e460 0x7f7cf40b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:01.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.345+0000 7f7cebe00640 1 -- 172.21.15.19:0/997732783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf415e460 msgr2=0x7f7cf40b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:01.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.345+0000 7f7cebe00640 1 --2- 172.21.15.19:0/997732783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf415e460 0x7f7cf40b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:01.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.345+0000 7f7cebe00640 1 -- 172.21.15.19:0/997732783 >> v1:172.21.15.188:6789/0 conn(0x7f7cf40cef20 legacy=0x7f7cf406c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:01.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.346+0000 7f7cebe00640 1 -- 172.21.15.19:0/997732783 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf40d3360 con 0x7f7cf40d3e40 2024-03-28T13:19:01.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.346+0000 7f7cf3e00640 1 --2- 172.21.15.19:0/997732783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf415e460 0x7f7cf40b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:19:01.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.347+0000 7f7cebe00640 1 --2- 172.21.15.19:0/997732783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cf40d3e40 0x7f7cf40d2550 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc009850 tx=0x7f7cdc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dcdd498117f038 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:01.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.348+0000 7f7cf3400640 1 -- 172.21.15.19:0/997732783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc004230 con 0x7f7cf40d3e40 2024-03-28T13:19:01.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.348+0000 7f7cf3400640 1 -- 172.21.15.19:0/997732783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cdc0043d0 con 0x7f7cf40d3e40 2024-03-28T13:19:01.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.348+0000 7f7cf3400640 1 -- 172.21.15.19:0/997732783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc003810 con 0x7f7cf40d3e40 2024-03-28T13:19:01.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.348+0000 7f7cf9c00640 1 -- 172.21.15.19:0/997732783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cf40d3e40 msgr2=0x7f7cf40d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.348+0000 7f7cf9c00640 1 --2- 172.21.15.19:0/997732783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cf40d3e40 0x7f7cf40d2550 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc009850 tx=0x7f7cdc02b960 comp rx=0 tx=0).stop 2024-03-28T13:19:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.349+0000 7f7cf9c00640 1 -- 172.21.15.19:0/997732783 shutdown_connections 2024-03-28T13:19:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.349+0000 7f7cf9c00640 1 --2- 172.21.15.19:0/997732783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cf40d3e40 0x7f7cf40d2550 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.349+0000 7f7cf9c00640 1 --2- 172.21.15.19:0/997732783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf415e460 0x7f7cf40b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.349+0000 7f7cf9c00640 1 -- 172.21.15.19:0/997732783 >> 172.21.15.19:0/997732783 conn(0x7f7cf40678b0 msgr2=0x7f7cf40d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.349+0000 7f7cf9c00640 1 -- 172.21.15.19:0/997732783 shutdown_connections 2024-03-28T13:19:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.349+0000 7f7cf9c00640 1 -- 172.21.15.19:0/997732783 wait complete. 2024-03-28T13:19:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.350+0000 7f7cf9c00640 1 Processor -- start 2024-03-28T13:19:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.350+0000 7f7cf9c00640 1 -- start start 2024-03-28T13:19:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.351+0000 7f7cf9c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cf40cef20 0x7f7cf41e6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.351+0000 7f7cf9c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cf40d3e40 0x7f7cf414d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.351+0000 7f7cf9c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf415e460 0x7f7cf414da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.351+0000 7f7cf9c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf40d2a90 con 0x7f7cf40cef20 2024-03-28T13:19:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.351+0000 7f7cf9c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf40d3920 con 0x7f7cf40d3e40 2024-03-28T13:19:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.351+0000 7f7cf9c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf40cf3b0 con 0x7f7cf415e460 2024-03-28T13:19:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.352+0000 7f7cf3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cf40cef20 0x7f7cf41e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.352+0000 7f7cf3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cf40cef20 0x7f7cf41e6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59338/0 (socket says 172.21.15.19:59338) 2024-03-28T13:19:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.352+0000 7f7cf3e00640 1 -- 172.21.15.19:0/1447457498 learned_addr learned my addr 172.21.15.19:0/1447457498 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.352+0000 7f7cebe00640 1 --2- 172.21.15.19:0/1447457498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cf40d3e40 0x7f7cf414d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.352+0000 7f7cf3e00640 1 -- 172.21.15.19:0/1447457498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf415e460 msgr2=0x7f7cf414da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.352+0000 7f7cf3e00640 1 --2- 172.21.15.19:0/1447457498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf415e460 0x7f7cf414da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.352+0000 7f7cf3e00640 1 -- 172.21.15.19:0/1447457498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cf40d3e40 msgr2=0x7f7cf414d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.352+0000 7f7cf3e00640 1 --2- 172.21.15.19:0/1447457498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cf40d3e40 0x7f7cf414d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.352+0000 7f7cf3e00640 1 -- 172.21.15.19:0/1447457498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf41e3aa0 con 0x7f7cf40cef20 2024-03-28T13:19:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.352+0000 7f7cf3e00640 1 --2- 172.21.15.19:0/1447457498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cf40cef20 0x7f7cf41e6590 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400d7d0 tx=0x7f7ce400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.353+0000 7f7cf0c00640 1 -- 172.21.15.19:0/1447457498 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4004030 con 0x7f7cf40cef20 2024-03-28T13:19:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.353+0000 7f7cf0c00640 1 -- 172.21.15.19:0/1447457498 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce40041d0 con 0x7f7cf40cef20 2024-03-28T13:19:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.353+0000 7f7cf0c00640 1 -- 172.21.15.19:0/1447457498 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce400b6c0 con 0x7f7cf40cef20 2024-03-28T13:19:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.353+0000 7f7cf9c00640 1 -- 172.21.15.19:0/1447457498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf41e3d80 con 0x7f7cf40cef20 2024-03-28T13:19:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.353+0000 7f7cf9c00640 1 -- 172.21.15.19:0/1447457498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf41e6ad0 con 0x7f7cf40cef20 2024-03-28T13:19:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.354+0000 7f7cf9c00640 1 -- 172.21.15.19:0/1447457498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cf40d1fe0 con 0x7f7cf40cef20 2024-03-28T13:19:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.355+0000 7f7cf0c00640 1 -- 172.21.15.19:0/1447457498 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7ce400b860 con 0x7f7cf40cef20 2024-03-28T13:19:01.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.356+0000 7f7cf0c00640 1 --2- 172.21.15.19:0/1447457498 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7cd006c530 0x7f7cd006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.356+0000 7f7cf0c00640 1 -- 172.21.15.19:0/1447457498 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f7ce408b9a0 con 0x7f7cf40cef20 2024-03-28T13:19:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.357+0000 7f7cebe00640 1 --2- 172.21.15.19:0/1447457498 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7cd006c530 0x7f7cd006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.358+0000 7f7cebe00640 1 --2- 172.21.15.19:0/1447457498 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7cd006c530 0x7f7cd006e9f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc035000 tx=0x7f7cdc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:01.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.368+0000 7f7cf0c00640 1 -- 172.21.15.19:0/1447457498 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7ce4059fb0 con 0x7f7cf40cef20 2024-03-28T13:19:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:01.612+0000 7f7cf9c00640 1 -- 172.21.15.19:0/1447457498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f7cf4075940 con 0x7f7cf40cef20 2024-03-28T13:19:02.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.104+0000 7f7cf0c00640 1 -- 172.21.15.19:0/1447457498 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v83) ==== 122+0+0 (secure 0 0 0) 0x7f7ce40598b0 con 0x7f7cf40cef20 2024-03-28T13:19:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.107+0000 7f7cf9c00640 1 -- 172.21.15.19:0/1447457498 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7cd006c530 msgr2=0x7f7cd006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.107+0000 7f7cf9c00640 1 --2- 172.21.15.19:0/1447457498 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7cd006c530 0x7f7cd006e9f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc035000 tx=0x7f7cdc008030 comp rx=0 tx=0).stop 2024-03-28T13:19:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.107+0000 7f7cf9c00640 1 -- 172.21.15.19:0/1447457498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cf40cef20 msgr2=0x7f7cf41e6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.107+0000 7f7cf9c00640 1 --2- 172.21.15.19:0/1447457498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cf40cef20 0x7f7cf41e6590 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400d7d0 tx=0x7f7ce400dc90 comp rx=0 tx=0).stop 2024-03-28T13:19:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.107+0000 7f7cf9c00640 1 -- 172.21.15.19:0/1447457498 shutdown_connections 2024-03-28T13:19:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.107+0000 7f7cf9c00640 1 --2- 172.21.15.19:0/1447457498 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7cd006c530 0x7f7cd006e9f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.107+0000 7f7cf9c00640 1 --2- 172.21.15.19:0/1447457498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cf415e460 0x7f7cf414da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.107+0000 7f7cf9c00640 1 --2- 172.21.15.19:0/1447457498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cf40d3e40 0x7f7cf414d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.107+0000 7f7cf9c00640 1 --2- 172.21.15.19:0/1447457498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7cf40cef20 0x7f7cf41e6590 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.107+0000 7f7cf9c00640 1 -- 172.21.15.19:0/1447457498 >> 172.21.15.19:0/1447457498 conn(0x7f7cf40678b0 msgr2=0x7f7cf40b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:02.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.107+0000 7f7cf9c00640 1 -- 172.21.15.19:0/1447457498 shutdown_connections 2024-03-28T13:19:02.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.107+0000 7f7cf9c00640 1 -- 172.21.15.19:0/1447457498 wait complete. 2024-03-28T13:19:02.129 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-28T13:19:02.129 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:19:02.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.374+0000 7f892d800640 1 Processor -- start 2024-03-28T13:19:02.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.374+0000 7f892d800640 1 -- start start 2024-03-28T13:19:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.376+0000 7f892d800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8928070250 0x7f89280d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.376+0000 7f892d800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f89280d26e0 0x7f89280d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.376+0000 7f892d800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89280d2ed0 con 0x7f892813f100 2024-03-28T13:19:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.376+0000 7f892d800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8928142270 con 0x7f89280d26e0 2024-03-28T13:19:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.377+0000 7f892d800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f892813f4e0 con 0x7f8928070250 2024-03-28T13:19:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.377+0000 7f892ce00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f89280d26e0 0x7f89280d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.377+0000 7f8927e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f892813f100 0x7f892806fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57374/0 (socket says 172.21.15.19:57374) 2024-03-28T13:19:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.377+0000 7f8927e00640 1 -- 172.21.15.19:0/3998857613 learned_addr learned my addr 172.21.15.19:0/3998857613 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.377+0000 7f891f400640 1 --2- 172.21.15.19:0/3998857613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8928070250 0x7f89280d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.377+0000 7f892ce00640 1 -- 172.21.15.19:0/3998857613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8928070250 msgr2=0x7f89280d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.377+0000 7f892ce00640 1 --2- 172.21.15.19:0/3998857613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8928070250 0x7f89280d21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.377+0000 7f8927400640 1 -- 172.21.15.19:0/3998857613 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1144402657 0 0) 0x7f89280d2ed0 con 0x7f892813f100 2024-03-28T13:19:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.377+0000 7f892ce00640 1 -- 172.21.15.19:0/3998857613 >> v1:172.21.15.19:6789/0 conn(0x7f892813f100 legacy=0x7f892806fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.377+0000 7f892ce00640 1 -- 172.21.15.19:0/3998857613 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8928141940 con 0x7f89280d26e0 2024-03-28T13:19:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.377+0000 7f891f400640 1 --2- 172.21.15.19:0/3998857613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8928070250 0x7f89280d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.379+0000 7f892ce00640 1 --2- 172.21.15.19:0/3998857613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89280d26e0 0x7f89280d0a50 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f8920009d90 tx=0x7f892002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3faac582caa956c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.379+0000 7f8927400640 1 -- 172.21.15.19:0/3998857613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920039070 con 0x7f89280d26e0 2024-03-28T13:19:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.379+0000 7f8927400640 1 -- 172.21.15.19:0/3998857613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892002cc50 con 0x7f89280d26e0 2024-03-28T13:19:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.379+0000 7f8927400640 1 -- 172.21.15.19:0/3998857613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920034440 con 0x7f89280d26e0 2024-03-28T13:19:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.379+0000 7f892d800640 1 -- 172.21.15.19:0/3998857613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89280d26e0 msgr2=0x7f89280d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.379+0000 7f892d800640 1 --2- 172.21.15.19:0/3998857613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89280d26e0 0x7f89280d0a50 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f8920009d90 tx=0x7f892002bea0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.380+0000 7f892d800640 1 -- 172.21.15.19:0/3998857613 shutdown_connections 2024-03-28T13:19:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.380+0000 7f892d800640 1 --2- 172.21.15.19:0/3998857613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f89280d26e0 0x7f89280d0a50 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.380+0000 7f892d800640 1 --2- 172.21.15.19:0/3998857613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8928070250 0x7f89280d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.380+0000 7f892d800640 1 -- 172.21.15.19:0/3998857613 >> 172.21.15.19:0/3998857613 conn(0x7f8928067460 msgr2=0x7f8928067840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.380+0000 7f892d800640 1 -- 172.21.15.19:0/3998857613 shutdown_connections 2024-03-28T13:19:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.380+0000 7f892d800640 1 -- 172.21.15.19:0/3998857613 wait complete. 2024-03-28T13:19:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.381+0000 7f892d800640 1 Processor -- start 2024-03-28T13:19:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.382+0000 7f892d800640 1 -- start start 2024-03-28T13:19:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.382+0000 7f892d800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8928070250 0x7f89281f4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f892d800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89280d26e0 0x7f89281f1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f892d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f892813f100 0x7f89280b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f892d800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8928142700 con 0x7f892813f100 2024-03-28T13:19:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f892d800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8928141f00 con 0x7f8928070250 2024-03-28T13:19:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f892d800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f89280d2b70 con 0x7f89280d26e0 2024-03-28T13:19:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f892ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f892813f100 0x7f89280b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f892ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f892813f100 0x7f89280b6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59362/0 (socket says 172.21.15.19:59362) 2024-03-28T13:19:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f892ce00640 1 -- 172.21.15.19:0/3803768966 learned_addr learned my addr 172.21.15.19:0/3803768966 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f8927e00640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8928070250 0x7f89281f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f891f400640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89280d26e0 0x7f89281f1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f892ce00640 1 -- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89280d26e0 msgr2=0x7f89281f1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f892ce00640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89280d26e0 0x7f89281f1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f892ce00640 1 -- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8928070250 msgr2=0x7f89281f4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f892ce00640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8928070250 0x7f89281f4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f892ce00640 1 -- 172.21.15.19:0/3803768966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89281535b0 con 0x7f892813f100 2024-03-28T13:19:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f8927e00640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8928070250 0x7f89281f4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.383+0000 7f891f400640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89280d26e0 0x7f89281f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.384+0000 7f892ce00640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f892813f100 0x7f89280b6360 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8920009d90 tx=0x7f8920004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.384+0000 7f8924c00640 1 -- 172.21.15.19:0/3803768966 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920044020 con 0x7f892813f100 2024-03-28T13:19:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.384+0000 7f8924c00640 1 -- 172.21.15.19:0/3803768966 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892003a040 con 0x7f892813f100 2024-03-28T13:19:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.384+0000 7f892d800640 1 -- 172.21.15.19:0/3803768966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89280b22b0 con 0x7f892813f100 2024-03-28T13:19:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.384+0000 7f8924c00640 1 -- 172.21.15.19:0/3803768966 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920039050 con 0x7f892813f100 2024-03-28T13:19:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.384+0000 7f892d800640 1 -- 172.21.15.19:0/3803768966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89280b2b50 con 0x7f892813f100 2024-03-28T13:19:02.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.385+0000 7f892d800640 1 -- 172.21.15.19:0/3803768966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89280b26a0 con 0x7f892813f100 2024-03-28T13:19:02.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.387+0000 7f8924c00640 1 -- 172.21.15.19:0/3803768966 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8920003990 con 0x7f892813f100 2024-03-28T13:19:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.389+0000 7f8924c00640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f890406c530 0x7f890406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:02.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.390+0000 7f8924c00640 1 -- 172.21.15.19:0/3803768966 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f89200adc30 con 0x7f892813f100 2024-03-28T13:19:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.390+0000 7f8927e00640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f890406c530 0x7f890406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.391+0000 7f8927e00640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f890406c530 0x7f890406e9f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8918000f60 tx=0x7f8918009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.398+0000 7f8924c00640 1 -- 172.21.15.19:0/3803768966 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f892004e060 con 0x7f892813f100 2024-03-28T13:19:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.656+0000 7f892d800640 1 -- 172.21.15.19:0/3803768966 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8928071d40 con 0x7f892813f100 2024-03-28T13:19:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.658+0000 7f8924c00640 1 -- 172.21.15.19:0/3803768966 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5189 (secure 0 0 0) 0x7f89281535b0 con 0x7f892813f100 2024-03-28T13:19:02.658 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:02.658 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4247326257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4247326257},{"type":"v1","addr":"172.21.15.188:6836","nonce":4247326257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3452274156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3452274156},{"type":"v1","addr":"172.21.15.19:6835","nonce":3452274156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3383190607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3383190607},{"type":"v1","addr":"172.21.15.188:6837","nonce":3383190607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:18:58.541599+0000","modified":"2024-03-28T13:19:01.614319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/1020599987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1020599987},{"type":"v1","addr":"172.21.15.19:6834","nonce":1020599987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8040,"qdb_cluster":[8040]},"id":7}]} 2024-03-28T13:19:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 83 2024-03-28T13:19:02.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.660+0000 7f892d800640 1 -- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f890406c530 msgr2=0x7f890406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.660+0000 7f892d800640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f890406c530 0x7f890406e9f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8918000f60 tx=0x7f8918009040 comp rx=0 tx=0).stop 2024-03-28T13:19:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.660+0000 7f892d800640 1 -- 172.21.15.19:0/3803768966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f892813f100 msgr2=0x7f89280b6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.660+0000 7f892d800640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f892813f100 0x7f89280b6360 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f8920009d90 tx=0x7f8920004a40 comp rx=0 tx=0).stop 2024-03-28T13:19:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.661+0000 7f892d800640 1 -- 172.21.15.19:0/3803768966 shutdown_connections 2024-03-28T13:19:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.661+0000 7f892d800640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f890406c530 0x7f890406e9f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.661+0000 7f892d800640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f892813f100 0x7f89280b6360 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.661+0000 7f892d800640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89280d26e0 0x7f89281f1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.661+0000 7f892d800640 1 --2- 172.21.15.19:0/3803768966 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8928070250 0x7f89281f4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.661+0000 7f892d800640 1 -- 172.21.15.19:0/3803768966 >> 172.21.15.19:0/3803768966 conn(0x7f8928067460 msgr2=0x7f892815e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.661+0000 7f892d800640 1 -- 172.21.15.19:0/3803768966 shutdown_connections 2024-03-28T13:19:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.661+0000 7f892d800640 1 -- 172.21.15.19:0/3803768966 wait complete. 2024-03-28T13:19:02.679 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:19:02.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.903+0000 7ff5b3200640 1 Processor -- start 2024-03-28T13:19:02.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.903+0000 7ff5b3200640 1 -- start start 2024-03-28T13:19:02.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.905+0000 7ff5b3200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ac15e3b0 0x7ff5ac0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:02.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.905+0000 7ff5b3200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5ac0b63c0 0x7ff5ac0b7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:02.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.905+0000 7ff5b3200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5ac0d2cf0 con 0x7ff5ac0d3d90 2024-03-28T13:19:02.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.905+0000 7ff5b3200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff5ac0d2fd0 con 0x7ff5ac0b63c0 2024-03-28T13:19:02.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.905+0000 7ff5b3200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ac0d32b0 con 0x7ff5ac15e3b0 2024-03-28T13:19:02.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.905+0000 7ff5b1e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ac15e3b0 0x7ff5ac0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.906+0000 7ff5b1e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ac15e3b0 0x7ff5ac0d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:46724/0 (socket says 172.21.15.19:46724) 2024-03-28T13:19:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.906+0000 7ff5b1e00640 1 -- 172.21.15.19:0/1130287037 learned_addr learned my addr 172.21.15.19:0/1130287037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.906+0000 7ff5b2800640 1 --2- 172.21.15.19:0/1130287037 >> v2:172.21.15.188:3300/0 conn(0x7ff5ac0b63c0 0x7ff5ac0b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.906+0000 7ff5b1e00640 1 -- 172.21.15.19:0/1130287037 >> v2:172.21.15.188:3300/0 conn(0x7ff5ac0b63c0 msgr2=0x7ff5ac0b7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.906+0000 7ff5b1e00640 1 --2- 172.21.15.19:0/1130287037 >> v2:172.21.15.188:3300/0 conn(0x7ff5ac0b63c0 0x7ff5ac0b7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.906+0000 7ff5b1e00640 1 -- 172.21.15.19:0/1130287037 >> v1:172.21.15.188:6789/0 conn(0x7ff5ac0d3d90 legacy=0x7ff5ac0b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:02.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.906+0000 7ff5b1e00640 1 -- 172.21.15.19:0/1130287037 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ac0d3590 con 0x7ff5ac15e3b0 2024-03-28T13:19:02.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.907+0000 7ff5b2800640 1 --2- 172.21.15.19:0/1130287037 >> v2:172.21.15.188:3300/0 conn(0x7ff5ac0b63c0 0x7ff5ac0b7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:19:02.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.908+0000 7ff5b1e00640 1 --2- 172.21.15.19:0/1130287037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ac15e3b0 0x7ff5ac0d22e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009870 tx=0x7ff5a002ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18c6fa14fb7d6423 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:02.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.909+0000 7ff5b0a00640 1 -- 172.21.15.19:0/1130287037 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a002f830 con 0x7ff5ac15e3b0 2024-03-28T13:19:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.909+0000 7ff5b0a00640 1 -- 172.21.15.19:0/1130287037 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5a002f9d0 con 0x7ff5ac15e3b0 2024-03-28T13:19:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.909+0000 7ff5b0a00640 1 -- 172.21.15.19:0/1130287037 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a002fda0 con 0x7ff5ac15e3b0 2024-03-28T13:19:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.909+0000 7ff5b3200640 1 -- 172.21.15.19:0/1130287037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ac15e3b0 msgr2=0x7ff5ac0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.909+0000 7ff5b3200640 1 --2- 172.21.15.19:0/1130287037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ac15e3b0 0x7ff5ac0d22e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009870 tx=0x7ff5a002ba10 comp rx=0 tx=0).stop 2024-03-28T13:19:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.910+0000 7ff5b3200640 1 -- 172.21.15.19:0/1130287037 shutdown_connections 2024-03-28T13:19:02.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.910+0000 7ff5b3200640 1 --2- 172.21.15.19:0/1130287037 >> v2:172.21.15.188:3300/0 conn(0x7ff5ac0b63c0 0x7ff5ac0b7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.910+0000 7ff5b3200640 1 --2- 172.21.15.19:0/1130287037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ac15e3b0 0x7ff5ac0d22e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.910+0000 7ff5b3200640 1 -- 172.21.15.19:0/1130287037 >> 172.21.15.19:0/1130287037 conn(0x7ff5ac0678b0 msgr2=0x7ff5ac0d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.910+0000 7ff5b3200640 1 -- 172.21.15.19:0/1130287037 shutdown_connections 2024-03-28T13:19:02.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.910+0000 7ff5b3200640 1 -- 172.21.15.19:0/1130287037 wait complete. 2024-03-28T13:19:02.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.911+0000 7ff5b3200640 1 Processor -- start 2024-03-28T13:19:02.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.912+0000 7ff5b3200640 1 -- start start 2024-03-28T13:19:02.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.912+0000 7ff5b3200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5ac0b63c0 0x7ff5ac160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:02.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.912+0000 7ff5b3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5ac0d3d90 0x7ff5ac1f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:02.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.912+0000 7ff5b3200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ac0c8770 0x7ff5ac1f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:02.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.912+0000 7ff5b3200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac0ba030 con 0x7ff5ac0d3d90 2024-03-28T13:19:02.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.912+0000 7ff5b3200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac15e790 con 0x7ff5ac0b63c0 2024-03-28T13:19:02.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.912+0000 7ff5b3200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ac0ba1b0 con 0x7ff5ac0c8770 2024-03-28T13:19:02.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.913+0000 7ff5b1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5ac0d3d90 0x7ff5ac1f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:02.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.913+0000 7ff5b1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5ac0d3d90 0x7ff5ac1f7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59378/0 (socket says 172.21.15.19:59378) 2024-03-28T13:19:02.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.913+0000 7ff5b1400640 1 -- 172.21.15.19:0/2123158826 learned_addr learned my addr 172.21.15.19:0/2123158826 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:02.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.913+0000 7ff5b1e00640 1 --2- 172.21.15.19:0/2123158826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5ac0b63c0 0x7ff5ac160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:02.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.913+0000 7ff5b2800640 1 --2- 172.21.15.19:0/2123158826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ac0c8770 0x7ff5ac1f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:02.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.913+0000 7ff5b1400640 1 -- 172.21.15.19:0/2123158826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ac0c8770 msgr2=0x7ff5ac1f5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:02.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.913+0000 7ff5b1400640 1 --2- 172.21.15.19:0/2123158826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ac0c8770 0x7ff5ac1f5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.913+0000 7ff5b1400640 1 -- 172.21.15.19:0/2123158826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5ac0b63c0 msgr2=0x7ff5ac160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:02.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.913+0000 7ff5b1400640 1 --2- 172.21.15.19:0/2123158826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5ac0b63c0 0x7ff5ac160ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:02.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.913+0000 7ff5b1400640 1 -- 172.21.15.19:0/2123158826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ac1f5840 con 0x7ff5ac0d3d90 2024-03-28T13:19:02.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.914+0000 7ff5b1400640 1 --2- 172.21.15.19:0/2123158826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5ac0d3d90 0x7ff5ac1f7050 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00c3c0 tx=0x7ff59c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:02.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.914+0000 7ff5a6000640 1 -- 172.21.15.19:0/2123158826 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0052d0 con 0x7ff5ac0d3d90 2024-03-28T13:19:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.914+0000 7ff5a6000640 1 -- 172.21.15.19:0/2123158826 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff59c0054b0 con 0x7ff5ac0d3d90 2024-03-28T13:19:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.914+0000 7ff5a6000640 1 -- 172.21.15.19:0/2123158826 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c006a40 con 0x7ff5ac0d3d90 2024-03-28T13:19:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.914+0000 7ff5b3200640 1 -- 172.21.15.19:0/2123158826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ac15ead0 con 0x7ff5ac0d3d90 2024-03-28T13:19:02.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.914+0000 7ff5b3200640 1 -- 172.21.15.19:0/2123158826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ac15f090 con 0x7ff5ac0d3d90 2024-03-28T13:19:02.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.916+0000 7ff5b3200640 1 -- 172.21.15.19:0/2123158826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5ac075940 con 0x7ff5ac0d3d90 2024-03-28T13:19:02.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.917+0000 7ff5a6000640 1 -- 172.21.15.19:0/2123158826 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff59c006be0 con 0x7ff5ac0d3d90 2024-03-28T13:19:02.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.917+0000 7ff5a6000640 1 --2- 172.21.15.19:0/2123158826 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff59006c530 0x7ff59006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:02.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.917+0000 7ff5a6000640 1 -- 172.21.15.19:0/2123158826 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7ff59c08ab40 con 0x7ff5ac0d3d90 2024-03-28T13:19:02.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.918+0000 7ff5b1e00640 1 --2- 172.21.15.19:0/2123158826 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff59006c530 0x7ff59006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:02.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.919+0000 7ff5b1e00640 1 --2- 172.21.15.19:0/2123158826 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff59006c530 0x7ff59006e9f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0036000 tx=0x7ff5a0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:02.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:02.922+0000 7ff5a6000640 1 -- 172.21.15.19:0/2123158826 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff59c059170 con 0x7ff5ac0d3d90 2024-03-28T13:19:03.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.129+0000 7ff5b3200640 1 -- 172.21.15.19:0/2123158826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5ac0b3b90 con 0x7ff5ac0d3d90 2024-03-28T13:19:03.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.133+0000 7ff5a6000640 1 -- 172.21.15.19:0/2123158826 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16619 (secure 0 0 0) 0x7ff59c058a70 con 0x7ff5ac0d3d90 2024-03-28T13:19:03.133 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:03.133 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":105,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:19:00.072522+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:18:56.432417+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-28T13:18:57.540515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:19:03.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.136+0000 7ff5b3200640 1 -- 172.21.15.19:0/2123158826 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff59006c530 msgr2=0x7ff59006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:03.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.136+0000 7ff5b3200640 1 --2- 172.21.15.19:0/2123158826 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff59006c530 0x7ff59006e9f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0036000 tx=0x7ff5a0008030 comp rx=0 tx=0).stop 2024-03-28T13:19:03.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.136+0000 7ff5b3200640 1 -- 172.21.15.19:0/2123158826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5ac0d3d90 msgr2=0x7ff5ac1f7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:03.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.136+0000 7ff5b3200640 1 --2- 172.21.15.19:0/2123158826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5ac0d3d90 0x7ff5ac1f7050 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00c3c0 tx=0x7ff59c00c880 comp rx=0 tx=0).stop 2024-03-28T13:19:03.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.136+0000 7ff5b3200640 1 -- 172.21.15.19:0/2123158826 shutdown_connections 2024-03-28T13:19:03.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.136+0000 7ff5b3200640 1 --2- 172.21.15.19:0/2123158826 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff59006c530 0x7ff59006e9f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.136+0000 7ff5b3200640 1 --2- 172.21.15.19:0/2123158826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ac0c8770 0x7ff5ac1f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.136+0000 7ff5b3200640 1 --2- 172.21.15.19:0/2123158826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff5ac0d3d90 0x7ff5ac1f7050 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.136+0000 7ff5b3200640 1 --2- 172.21.15.19:0/2123158826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5ac0b63c0 0x7ff5ac160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.136+0000 7ff5b3200640 1 -- 172.21.15.19:0/2123158826 >> 172.21.15.19:0/2123158826 conn(0x7ff5ac0678b0 msgr2=0x7ff5ac0b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:03.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.137+0000 7ff5b3200640 1 -- 172.21.15.19:0/2123158826 shutdown_connections 2024-03-28T13:19:03.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.137+0000 7ff5b3200640 1 -- 172.21.15.19:0/2123158826 wait complete. 2024-03-28T13:19:03.164 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-28T13:19:03.164 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:19:03.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.385+0000 7fec81c00640 1 Processor -- start 2024-03-28T13:19:03.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.386+0000 7fec81c00640 1 -- start start 2024-03-28T13:19:03.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.387+0000 7fec81c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec7c0d02a0 0x7fec7c165c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:03.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.387+0000 7fec81c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec7c071460 con 0x7fec7c0cf1b0 2024-03-28T13:19:03.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.387+0000 7fec81c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec7c0721b0 con 0x7fec7c1442c0 2024-03-28T13:19:03.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.387+0000 7fec81c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fec7c071d20 con 0x7fec7c0d02a0 2024-03-28T13:19:03.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.388+0000 7fec73400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fec7c0cf1b0 0x7fec7c0cfb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57382/0 (socket says 172.21.15.19:57382) 2024-03-28T13:19:03.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.388+0000 7fec73400640 1 -- 172.21.15.19:0/3747265 learned_addr learned my addr 172.21.15.19:0/3747265 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:03.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.388+0000 7fec81200640 1 --2- 172.21.15.19:0/3747265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec7c0d02a0 0x7fec7c165c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:03.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.388+0000 7fec7b400640 1 -- 172.21.15.19:0/3747265 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3140103246 0 0) 0x7fec7c071460 con 0x7fec7c0cf1b0 2024-03-28T13:19:03.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.388+0000 7fec7b400640 1 -- 172.21.15.19:0/3747265 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec60005730 con 0x7fec7c0cf1b0 2024-03-28T13:19:03.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.389+0000 7fec7b400640 1 -- 172.21.15.19:0/3747265 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 676601863 0 0) 0x7fec60005730 con 0x7fec7c0cf1b0 2024-03-28T13:19:03.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.389+0000 7fec7b400640 1 -- 172.21.15.19:0/3747265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec7c0d02a0 msgr2=0x7fec7c165c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:03.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.389+0000 7fec7b400640 1 --2- 172.21.15.19:0/3747265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec7c0d02a0 0x7fec7c165c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.389+0000 7fec7b400640 1 -- 172.21.15.19:0/3747265 >> v1:172.21.15.188:6789/0 conn(0x7fec7c1442c0 legacy=0x7fec7c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:03.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.389+0000 7fec7b400640 1 -- 172.21.15.19:0/3747265 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c0716d0 con 0x7fec7c0cf1b0 2024-03-28T13:19:03.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.389+0000 7fec7b400640 1 -- 172.21.15.19:0/3747265 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fec64001670 con 0x7fec7c0cf1b0 2024-03-28T13:19:03.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.389+0000 7fec7b400640 1 -- 172.21.15.19:0/3747265 >> v1:172.21.15.19:6789/0 conn(0x7fec7c0cf1b0 legacy=0x7fec7c0cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:03.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.389+0000 7fec7b400640 1 --2- 172.21.15.19:0/3747265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec60005bf0 0x7fec60006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:03.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.389+0000 7fec7b400640 1 --2- 172.21.15.19:0/3747265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec60006a10 0x7fec60006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:03.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.389+0000 7fec7b400640 1 --2- 172.21.15.19:0/3747265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec7c0d02a0 0x7fec600073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:03.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.389+0000 7fec7b400640 1 -- 172.21.15.19:0/3747265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c071d20 con 0x7fec60005bf0 2024-03-28T13:19:03.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.389+0000 7fec7b400640 1 -- 172.21.15.19:0/3747265 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fec6000dd40 con 0x7fec60006a10 2024-03-28T13:19:03.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.389+0000 7fec7b400640 1 -- 172.21.15.19:0/3747265 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fec6000df40 con 0x7fec7c0d02a0 2024-03-28T13:19:03.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.389+0000 7fec7be00640 1 --2- 172.21.15.19:0/3747265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec60005bf0 0x7fec60006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:03.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.389+0000 7fec81200640 1 --2- 172.21.15.19:0/3747265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec7c0d02a0 0x7fec600073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:03.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.390+0000 7fec73400640 1 --2- 172.21.15.19:0/3747265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec60006a10 0x7fec60006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:03.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.390+0000 7fec7be00640 1 -- 172.21.15.19:0/3747265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec7c0d02a0 msgr2=0x7fec600073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:03.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.390+0000 7fec7be00640 1 --2- 172.21.15.19:0/3747265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec7c0d02a0 0x7fec600073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.390+0000 7fec7be00640 1 -- 172.21.15.19:0/3747265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec60006a10 msgr2=0x7fec60006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:03.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.390+0000 7fec7be00640 1 --2- 172.21.15.19:0/3747265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec60006a10 0x7fec60006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.390+0000 7fec7be00640 1 -- 172.21.15.19:0/3747265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec60005730 con 0x7fec60005bf0 2024-03-28T13:19:03.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.390+0000 7fec73400640 1 --2- 172.21.15.19:0/3747265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec60006a10 0x7fec60006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:03.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.391+0000 7fec7be00640 1 --2- 172.21.15.19:0/3747265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec60005bf0 0x7fec60006010 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fec6c001e20 tx=0x7fec60007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b351ccb3316789de server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:03.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.391+0000 7fec7b400640 1 -- 172.21.15.19:0/3747265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c001a70 con 0x7fec60005bf0 2024-03-28T13:19:03.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.391+0000 7fec7b400640 1 -- 172.21.15.19:0/3747265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec6c003b70 con 0x7fec60005bf0 2024-03-28T13:19:03.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.391+0000 7fec7b400640 1 -- 172.21.15.19:0/3747265 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c0050d0 con 0x7fec60005bf0 2024-03-28T13:19:03.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.391+0000 7fec81c00640 1 -- 172.21.15.19:0/3747265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec60005bf0 msgr2=0x7fec60006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:03.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.391+0000 7fec81c00640 1 --2- 172.21.15.19:0/3747265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec60005bf0 0x7fec60006010 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fec6c001e20 tx=0x7fec60007a40 comp rx=0 tx=0).stop 2024-03-28T13:19:03.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.392+0000 7fec81200640 1 -- 172.21.15.19:0/3747265 reap_dead start 2024-03-28T13:19:03.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.392+0000 7fec81c00640 1 -- 172.21.15.19:0/3747265 shutdown_connections 2024-03-28T13:19:03.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.392+0000 7fec81c00640 1 -- 172.21.15.19:0/3747265 >> 172.21.15.19:0/3747265 conn(0x7fec7c0ccad0 msgr2=0x7fec7c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:03.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.392+0000 7fec81c00640 1 -- 172.21.15.19:0/3747265 shutdown_connections 2024-03-28T13:19:03.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.392+0000 7fec81c00640 1 -- 172.21.15.19:0/3747265 wait complete. 2024-03-28T13:19:03.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.393+0000 7fec81c00640 1 Processor -- start 2024-03-28T13:19:03.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.394+0000 7fec81c00640 1 -- start start 2024-03-28T13:19:03.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec81c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec60005bf0 0x7fec7c1e4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:03.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec81c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec7c1e51e0 0x7fec7c15e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:03.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec81c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec7c0b5450 0x7fec7c15a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:03.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec81c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c1446a0 con 0x7fec60005bf0 2024-03-28T13:19:03.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec81c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c0d0770 con 0x7fec7c1e51e0 2024-03-28T13:19:03.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec81c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fec7c1e48f0 con 0x7fec7c0b5450 2024-03-28T13:19:03.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec7be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec60005bf0 0x7fec7c1e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:03.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec7be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec60005bf0 0x7fec7c1e4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59394/0 (socket says 172.21.15.19:59394) 2024-03-28T13:19:03.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec7be00640 1 -- 172.21.15.19:0/3338920425 learned_addr learned my addr 172.21.15.19:0/3338920425 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:03.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec81200640 1 --2- 172.21.15.19:0/3338920425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec7c0b5450 0x7fec7c15a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:03.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec73400640 1 --2- 172.21.15.19:0/3338920425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec7c1e51e0 0x7fec7c15e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:03.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec7be00640 1 -- 172.21.15.19:0/3338920425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec7c0b5450 msgr2=0x7fec7c15a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:03.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec7be00640 1 --2- 172.21.15.19:0/3338920425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec7c0b5450 0x7fec7c15a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec7be00640 1 -- 172.21.15.19:0/3338920425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec7c1e51e0 msgr2=0x7fec7c15e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:03.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec7be00640 1 --2- 172.21.15.19:0/3338920425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec7c1e51e0 0x7fec7c15e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec7be00640 1 -- 172.21.15.19:0/3338920425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c15d480 con 0x7fec60005bf0 2024-03-28T13:19:03.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.395+0000 7fec73400640 1 --2- 172.21.15.19:0/3338920425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec7c1e51e0 0x7fec7c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:03.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.396+0000 7fec7be00640 1 --2- 172.21.15.19:0/3338920425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec60005bf0 0x7fec7c1e4200 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fec6c001f40 tx=0x7fec6c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:03.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.396+0000 7fec78c00640 1 -- 172.21.15.19:0/3338920425 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c003970 con 0x7fec60005bf0 2024-03-28T13:19:03.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.396+0000 7fec78c00640 1 -- 172.21.15.19:0/3338920425 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec6c006b50 con 0x7fec60005bf0 2024-03-28T13:19:03.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.396+0000 7fec78c00640 1 -- 172.21.15.19:0/3338920425 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6c007030 con 0x7fec60005bf0 2024-03-28T13:19:03.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.396+0000 7fec81c00640 1 -- 172.21.15.19:0/3338920425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7c15d760 con 0x7fec60005bf0 2024-03-28T13:19:03.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.396+0000 7fec81c00640 1 -- 172.21.15.19:0/3338920425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7c15e780 con 0x7fec60005bf0 2024-03-28T13:19:03.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.397+0000 7fec81c00640 1 -- 172.21.15.19:0/3338920425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec7c075940 con 0x7fec60005bf0 2024-03-28T13:19:03.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.398+0000 7fec78c00640 1 -- 172.21.15.19:0/3338920425 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fec6c0071d0 con 0x7fec60005bf0 2024-03-28T13:19:03.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.399+0000 7fec78c00640 1 --2- 172.21.15.19:0/3338920425 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fec5806c530 0x7fec5806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:03.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.399+0000 7fec78c00640 1 -- 172.21.15.19:0/3338920425 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7fec6c0ad320 con 0x7fec60005bf0 2024-03-28T13:19:03.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.400+0000 7fec73400640 1 --2- 172.21.15.19:0/3338920425 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fec5806c530 0x7fec5806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:03.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.401+0000 7fec73400640 1 --2- 172.21.15.19:0/3338920425 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fec5806c530 0x7fec5806e9f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0b6370 tx=0x7fec64009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:03.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.404+0000 7fec78c00640 1 -- 172.21.15.19:0/3338920425 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fec6c033020 con 0x7fec60005bf0 2024-03-28T13:19:03.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.605+0000 7fec81c00640 1 -- 172.21.15.19:0/3338920425 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fec7c068ae0 con 0x7fec5806c530 2024-03-28T13:19:03.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.625+0000 7fec78c00640 1 -- 172.21.15.19:0/3338920425 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251926 (secure 0 0 0) 0x7fec6400e840 con 0x7fec5806c530 2024-03-28T13:19:03.628 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:03.630 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:19:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.630+0000 7fec81c00640 1 -- 172.21.15.19:0/3338920425 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fec5806c530 msgr2=0x7fec5806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.630+0000 7fec81c00640 1 --2- 172.21.15.19:0/3338920425 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fec5806c530 0x7fec5806e9f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fec7c0b6370 tx=0x7fec64009040 comp rx=0 tx=0).stop 2024-03-28T13:19:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.630+0000 7fec81c00640 1 -- 172.21.15.19:0/3338920425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec60005bf0 msgr2=0x7fec7c1e4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.630+0000 7fec81c00640 1 --2- 172.21.15.19:0/3338920425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec60005bf0 0x7fec7c1e4200 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fec6c001f40 tx=0x7fec6c0014a0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.630+0000 7fec81c00640 1 -- 172.21.15.19:0/3338920425 shutdown_connections 2024-03-28T13:19:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.630+0000 7fec81c00640 1 --2- 172.21.15.19:0/3338920425 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fec5806c530 0x7fec5806e9f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.630+0000 7fec81c00640 1 --2- 172.21.15.19:0/3338920425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec7c0b5450 0x7fec7c15a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.630+0000 7fec81c00640 1 --2- 172.21.15.19:0/3338920425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec7c1e51e0 0x7fec7c15e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.630+0000 7fec81c00640 1 --2- 172.21.15.19:0/3338920425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec60005bf0 0x7fec7c1e4200 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.630+0000 7fec81c00640 1 -- 172.21.15.19:0/3338920425 >> 172.21.15.19:0/3338920425 conn(0x7fec7c0ccad0 msgr2=0x7fec7c06df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:03.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.631+0000 7fec81c00640 1 -- 172.21.15.19:0/3338920425 shutdown_connections 2024-03-28T13:19:03.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.631+0000 7fec81c00640 1 -- 172.21.15.19:0/3338920425 wait complete. 2024-03-28T13:19:03.650 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-03-28T13:19:02.197436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260556,"kb_used_data":11100,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714164,"statfs":{"total":773094113280,"available":772827303936,"internally_reserved":0,"allocated":11366400,"data_stored":5557640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.167405"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.091975+0000","last_change":"2024-03-28T13:18:59.085702+0000","last_active":"2024-03-28T13:18:59.091975+0000","last_peered":"2024-03-28T13:18:59.091975+0000","last_clean":"2024-03-28T13:18:59.091975+0000","last_became_active":"2024-03-28T13:18:59.085413+0000","last_became_peered":"2024-03-28T13:18:59.085413+0000","last_unstale":"2024-03-28T13:18:59.091975+0000","last_undegraded":"2024-03-28T13:18:59.091975+0000","last_fullsized":"2024-03-28T13:18:59.091975+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:05:04.270844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094323+0000","last_change":"2024-03-28T13:18:59.090611+0000","last_active":"2024-03-28T13:18:59.094323+0000","last_peered":"2024-03-28T13:18:59.094323+0000","last_clean":"2024-03-28T13:18:59.094323+0000","last_became_active":"2024-03-28T13:18:59.089972+0000","last_became_peered":"2024-03-28T13:18:59.089972+0000","last_unstale":"2024-03-28T13:18:59.094323+0000","last_undegraded":"2024-03-28T13:18:59.094323+0000","last_fullsized":"2024-03-28T13:18:59.094323+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:35:49.070231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.100115+0000","last_change":"2024-03-28T13:18:59.098655+0000","last_active":"2024-03-28T13:18:59.100115+0000","last_peered":"2024-03-28T13:18:59.100115+0000","last_clean":"2024-03-28T13:18:59.100115+0000","last_became_active":"2024-03-28T13:18:59.098399+0000","last_became_peered":"2024-03-28T13:18:59.098399+0000","last_unstale":"2024-03-28T13:18:59.100115+0000","last_undegraded":"2024-03-28T13:18:59.100115+0000","last_fullsized":"2024-03-28T13:18:59.100115+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:20:44.200653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.083142+0000","last_change":"2024-03-28T13:18:59.074664+0000","last_active":"2024-03-28T13:18:59.083142+0000","last_peered":"2024-03-28T13:18:59.083142+0000","last_clean":"2024-03-28T13:18:59.083142+0000","last_became_active":"2024-03-28T13:18:59.073993+0000","last_became_peered":"2024-03-28T13:18:59.073993+0000","last_unstale":"2024-03-28T13:18:59.083142+0000","last_undegraded":"2024-03-28T13:18:59.083142+0000","last_fullsized":"2024-03-28T13:18:59.083142+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:29.899339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091305+0000","last_change":"2024-03-28T13:18:59.090589+0000","last_active":"2024-03-28T13:19:00.091305+0000","last_peered":"2024-03-28T13:19:00.091305+0000","last_clean":"2024-03-28T13:19:00.091305+0000","last_became_active":"2024-03-28T13:18:59.089952+0000","last_became_peered":"2024-03-28T13:18:59.089952+0000","last_unstale":"2024-03-28T13:19:00.091305+0000","last_undegraded":"2024-03-28T13:19:00.091305+0000","last_fullsized":"2024-03-28T13:19:00.091305+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:09.506244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091182+0000","last_change":"2024-03-28T13:18:59.085728+0000","last_active":"2024-03-28T13:19:00.091182+0000","last_peered":"2024-03-28T13:19:00.091182+0000","last_clean":"2024-03-28T13:19:00.091182+0000","last_became_active":"2024-03-28T13:18:59.085447+0000","last_became_peered":"2024-03-28T13:18:59.085447+0000","last_unstale":"2024-03-28T13:19:00.091182+0000","last_undegraded":"2024-03-28T13:19:00.091182+0000","last_fullsized":"2024-03-28T13:19:00.091182+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:16.716159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091117+0000","last_change":"2024-03-28T13:18:59.090899+0000","last_active":"2024-03-28T13:19:00.091117+0000","last_peered":"2024-03-28T13:19:00.091117+0000","last_clean":"2024-03-28T13:19:00.091117+0000","last_became_active":"2024-03-28T13:18:59.086628+0000","last_became_peered":"2024-03-28T13:18:59.086628+0000","last_unstale":"2024-03-28T13:19:00.091117+0000","last_undegraded":"2024-03-28T13:19:00.091117+0000","last_fullsized":"2024-03-28T13:19:00.091117+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:05:31.597035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.092180+0000","last_change":"2024-03-28T13:18:59.082961+0000","last_active":"2024-03-28T13:18:59.092180+0000","last_peered":"2024-03-28T13:18:59.092180+0000","last_clean":"2024-03-28T13:18:59.092180+0000","last_became_active":"2024-03-28T13:18:59.082608+0000","last_became_peered":"2024-03-28T13:18:59.082608+0000","last_unstale":"2024-03-28T13:18:59.092180+0000","last_undegraded":"2024-03-28T13:18:59.092180+0000","last_fullsized":"2024-03-28T13:18:59.092180+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:59.363172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092358+0000","last_change":"2024-03-28T13:18:59.084837+0000","last_active":"2024-03-28T13:19:00.092358+0000","last_peered":"2024-03-28T13:19:00.092358+0000","last_clean":"2024-03-28T13:19:00.092358+0000","last_became_active":"2024-03-28T13:18:59.083085+0000","last_became_peered":"2024-03-28T13:18:59.083085+0000","last_unstale":"2024-03-28T13:19:00.092358+0000","last_undegraded":"2024-03-28T13:19:00.092358+0000","last_fullsized":"2024-03-28T13:19:00.092358+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:16:46.004371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094293+0000","last_change":"2024-03-28T13:18:59.088878+0000","last_active":"2024-03-28T13:18:59.094293+0000","last_peered":"2024-03-28T13:18:59.094293+0000","last_clean":"2024-03-28T13:18:59.094293+0000","last_became_active":"2024-03-28T13:18:59.088493+0000","last_became_peered":"2024-03-28T13:18:59.088493+0000","last_unstale":"2024-03-28T13:18:59.094293+0000","last_undegraded":"2024-03-28T13:18:59.094293+0000","last_fullsized":"2024-03-28T13:18:59.094293+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:48:04.926545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091805+0000","last_change":"2024-03-28T13:18:59.100683+0000","last_active":"2024-03-28T13:19:00.091805+0000","last_peered":"2024-03-28T13:19:00.091805+0000","last_clean":"2024-03-28T13:19:00.091805+0000","last_became_active":"2024-03-28T13:18:59.099214+0000","last_became_peered":"2024-03-28T13:18:59.099214+0000","last_unstale":"2024-03-28T13:19:00.091805+0000","last_undegraded":"2024-03-28T13:19:00.091805+0000","last_fullsized":"2024-03-28T13:19:00.091805+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:41.892054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.095497+0000","last_change":"2024-03-28T13:18:59.090740+0000","last_active":"2024-03-28T13:18:59.095497+0000","last_peered":"2024-03-28T13:18:59.095497+0000","last_clean":"2024-03-28T13:18:59.095497+0000","last_became_active":"2024-03-28T13:18:59.090473+0000","last_became_peered":"2024-03-28T13:18:59.090473+0000","last_unstale":"2024-03-28T13:18:59.095497+0000","last_undegraded":"2024-03-28T13:18:59.095497+0000","last_fullsized":"2024-03-28T13:18:59.095497+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:31.353091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-03-28T13:18:58.069477+0000","last_change":"2024-03-28T13:18:58.069477+0000","last_active":"2024-03-28T13:18:58.028202+0000","last_peered":"2024-03-28T13:18:58.028202+0000","last_clean":"2024-03-28T13:18:58.028202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:18:58.069477+0000","last_undegraded":"2024-03-28T13:18:58.069477+0000","last_fullsized":"2024-03-28T13:18:58.069477+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087240+0000","last_change":"2024-03-28T13:18:59.080979+0000","last_active":"2024-03-28T13:18:59.087240+0000","last_peered":"2024-03-28T13:18:59.087240+0000","last_clean":"2024-03-28T13:18:59.087240+0000","last_became_active":"2024-03-28T13:18:59.080676+0000","last_became_peered":"2024-03-28T13:18:59.080676+0000","last_unstale":"2024-03-28T13:18:59.087240+0000","last_undegraded":"2024-03-28T13:18:59.087240+0000","last_fullsized":"2024-03-28T13:18:59.087240+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:00.261227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087400+0000","last_change":"2024-03-28T13:18:59.077084+0000","last_active":"2024-03-28T13:18:59.087400+0000","last_peered":"2024-03-28T13:18:59.087400+0000","last_clean":"2024-03-28T13:18:59.087400+0000","last_became_active":"2024-03-28T13:18:59.076136+0000","last_became_peered":"2024-03-28T13:18:59.076136+0000","last_unstale":"2024-03-28T13:18:59.087400+0000","last_undegraded":"2024-03-28T13:18:59.087400+0000","last_fullsized":"2024-03-28T13:18:59.087400+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:20.977905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090808+0000","last_change":"2024-03-28T13:18:59.074866+0000","last_active":"2024-03-28T13:19:00.090808+0000","last_peered":"2024-03-28T13:19:00.090808+0000","last_clean":"2024-03-28T13:19:00.090808+0000","last_became_active":"2024-03-28T13:18:59.074514+0000","last_became_peered":"2024-03-28T13:18:59.074514+0000","last_unstale":"2024-03-28T13:19:00.090808+0000","last_undegraded":"2024-03-28T13:19:00.090808+0000","last_fullsized":"2024-03-28T13:19:00.090808+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:10:44.680345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.090237+0000","last_change":"2024-03-28T13:18:59.084712+0000","last_active":"2024-03-28T13:18:59.090237+0000","last_peered":"2024-03-28T13:18:59.090237+0000","last_clean":"2024-03-28T13:18:59.090237+0000","last_became_active":"2024-03-28T13:18:59.084435+0000","last_became_peered":"2024-03-28T13:18:59.084435+0000","last_unstale":"2024-03-28T13:18:59.090237+0000","last_undegraded":"2024-03-28T13:18:59.090237+0000","last_fullsized":"2024-03-28T13:18:59.090237+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:08:49.381746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087507+0000","last_change":"2024-03-28T13:18:59.083710+0000","last_active":"2024-03-28T13:18:59.087507+0000","last_peered":"2024-03-28T13:18:59.087507+0000","last_clean":"2024-03-28T13:18:59.087507+0000","last_became_active":"2024-03-28T13:18:59.083366+0000","last_became_peered":"2024-03-28T13:18:59.083366+0000","last_unstale":"2024-03-28T13:18:59.087507+0000","last_undegraded":"2024-03-28T13:18:59.087507+0000","last_fullsized":"2024-03-28T13:18:59.087507+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:19.489710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.081195+0000","last_change":"2024-03-28T13:18:59.076959+0000","last_active":"2024-03-28T13:18:59.081195+0000","last_peered":"2024-03-28T13:18:59.081195+0000","last_clean":"2024-03-28T13:18:59.081195+0000","last_became_active":"2024-03-28T13:18:59.075601+0000","last_became_peered":"2024-03-28T13:18:59.075601+0000","last_unstale":"2024-03-28T13:18:59.081195+0000","last_undegraded":"2024-03-28T13:18:59.081195+0000","last_fullsized":"2024-03-28T13:18:59.081195+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:00:07.119299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.093215+0000","last_change":"2024-03-28T13:18:59.087556+0000","last_active":"2024-03-28T13:18:59.093215+0000","last_peered":"2024-03-28T13:18:59.093215+0000","last_clean":"2024-03-28T13:18:59.093215+0000","last_became_active":"2024-03-28T13:18:59.087183+0000","last_became_peered":"2024-03-28T13:18:59.087183+0000","last_unstale":"2024-03-28T13:18:59.093215+0000","last_undegraded":"2024-03-28T13:18:59.093215+0000","last_fullsized":"2024-03-28T13:18:59.093215+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:28:49.643546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.092574+0000","last_change":"2024-03-28T13:18:59.085370+0000","last_active":"2024-03-28T13:18:59.092574+0000","last_peered":"2024-03-28T13:18:59.092574+0000","last_clean":"2024-03-28T13:18:59.092574+0000","last_became_active":"2024-03-28T13:18:59.084826+0000","last_became_peered":"2024-03-28T13:18:59.084826+0000","last_unstale":"2024-03-28T13:18:59.092574+0000","last_undegraded":"2024-03-28T13:18:59.092574+0000","last_fullsized":"2024-03-28T13:18:59.092574+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:31.353091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094564+0000","last_change":"2024-03-28T13:18:59.091677+0000","last_active":"2024-03-28T13:18:59.094564+0000","last_peered":"2024-03-28T13:18:59.094564+0000","last_clean":"2024-03-28T13:18:59.094564+0000","last_became_active":"2024-03-28T13:18:59.091299+0000","last_became_peered":"2024-03-28T13:18:59.091299+0000","last_unstale":"2024-03-28T13:18:59.094564+0000","last_undegraded":"2024-03-28T13:18:59.094564+0000","last_fullsized":"2024-03-28T13:18:59.094564+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:40.290650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087426+0000","last_change":"2024-03-28T13:18:59.083262+0000","last_active":"2024-03-28T13:18:59.087426+0000","last_peered":"2024-03-28T13:18:59.087426+0000","last_clean":"2024-03-28T13:18:59.087426+0000","last_became_active":"2024-03-28T13:18:59.082908+0000","last_became_peered":"2024-03-28T13:18:59.082908+0000","last_unstale":"2024-03-28T13:18:59.087426+0000","last_undegraded":"2024-03-28T13:18:59.087426+0000","last_fullsized":"2024-03-28T13:18:59.087426+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:27:52.931365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.089502+0000","last_change":"2024-03-28T13:18:59.085398+0000","last_active":"2024-03-28T13:18:59.089502+0000","last_peered":"2024-03-28T13:18:59.089502+0000","last_clean":"2024-03-28T13:18:59.089502+0000","last_became_active":"2024-03-28T13:18:59.083434+0000","last_became_peered":"2024-03-28T13:18:59.083434+0000","last_unstale":"2024-03-28T13:18:59.089502+0000","last_undegraded":"2024-03-28T13:18:59.089502+0000","last_fullsized":"2024-03-28T13:18:59.089502+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:00.261227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.090300+0000","last_change":"2024-03-28T13:18:59.084798+0000","last_active":"2024-03-28T13:18:59.090300+0000","last_peered":"2024-03-28T13:18:59.090300+0000","last_clean":"2024-03-28T13:18:59.090300+0000","last_became_active":"2024-03-28T13:18:59.084216+0000","last_became_peered":"2024-03-28T13:18:59.084216+0000","last_unstale":"2024-03-28T13:18:59.090300+0000","last_undegraded":"2024-03-28T13:18:59.090300+0000","last_fullsized":"2024-03-28T13:18:59.090300+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:45.824120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-03-28T13:18:58.071873+0000","last_change":"2024-03-28T13:18:58.071873+0000","last_active":"2024-03-28T13:18:58.028202+0000","last_peered":"2024-03-28T13:18:58.028202+0000","last_clean":"2024-03-28T13:18:58.028202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:18:58.071873+0000","last_undegraded":"2024-03-28T13:18:58.071873+0000","last_fullsized":"2024-03-28T13:18:58.071873+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.086078+0000","last_change":"2024-03-28T13:18:59.080256+0000","last_active":"2024-03-28T13:18:59.086078+0000","last_peered":"2024-03-28T13:18:59.086078+0000","last_clean":"2024-03-28T13:18:59.086078+0000","last_became_active":"2024-03-28T13:18:59.079957+0000","last_became_peered":"2024-03-28T13:18:59.079957+0000","last_unstale":"2024-03-28T13:18:59.086078+0000","last_undegraded":"2024-03-28T13:18:59.086078+0000","last_fullsized":"2024-03-28T13:18:59.086078+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:20.977905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.084557+0000","last_change":"2024-03-28T13:18:59.078806+0000","last_active":"2024-03-28T13:18:59.084557+0000","last_peered":"2024-03-28T13:18:59.084557+0000","last_clean":"2024-03-28T13:18:59.084557+0000","last_became_active":"2024-03-28T13:18:59.078526+0000","last_became_peered":"2024-03-28T13:18:59.078526+0000","last_unstale":"2024-03-28T13:18:59.084557+0000","last_undegraded":"2024-03-28T13:18:59.084557+0000","last_fullsized":"2024-03-28T13:18:59.084557+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:26:09.194396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-03-28T13:18:58.068259+0000","last_change":"2024-03-28T13:18:58.068259+0000","last_active":"2024-03-28T13:18:58.028202+0000","last_peered":"2024-03-28T13:18:58.028202+0000","last_clean":"2024-03-28T13:18:58.028202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:18:58.068259+0000","last_undegraded":"2024-03-28T13:18:58.068259+0000","last_fullsized":"2024-03-28T13:18:58.068259+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-03-28T13:18:58.074341+0000","last_change":"2024-03-28T13:18:58.074341+0000","last_active":"2024-03-28T13:18:58.028202+0000","last_peered":"2024-03-28T13:18:58.028202+0000","last_clean":"2024-03-28T13:18:58.028202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:18:58.074341+0000","last_undegraded":"2024-03-28T13:18:58.074341+0000","last_fullsized":"2024-03-28T13:18:58.074341+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.092498+0000","last_change":"2024-03-28T13:18:59.086506+0000","last_active":"2024-03-28T13:18:59.092498+0000","last_peered":"2024-03-28T13:18:59.092498+0000","last_clean":"2024-03-28T13:18:59.092498+0000","last_became_active":"2024-03-28T13:18:59.086216+0000","last_became_peered":"2024-03-28T13:18:59.086216+0000","last_unstale":"2024-03-28T13:18:59.092498+0000","last_undegraded":"2024-03-28T13:18:59.092498+0000","last_fullsized":"2024-03-28T13:18:59.092498+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:15.681052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.082999+0000","last_change":"2024-03-28T13:18:59.075805+0000","last_active":"2024-03-28T13:18:59.082999+0000","last_peered":"2024-03-28T13:18:59.082999+0000","last_clean":"2024-03-28T13:18:59.082999+0000","last_became_active":"2024-03-28T13:18:59.074979+0000","last_became_peered":"2024-03-28T13:18:59.074979+0000","last_unstale":"2024-03-28T13:18:59.082999+0000","last_undegraded":"2024-03-28T13:18:59.082999+0000","last_fullsized":"2024-03-28T13:18:59.082999+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:29.522394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094055+0000","last_change":"2024-03-28T13:18:59.091367+0000","last_active":"2024-03-28T13:18:59.094055+0000","last_peered":"2024-03-28T13:18:59.094055+0000","last_clean":"2024-03-28T13:18:59.094055+0000","last_became_active":"2024-03-28T13:18:59.091083+0000","last_became_peered":"2024-03-28T13:18:59.091083+0000","last_unstale":"2024-03-28T13:18:59.094055+0000","last_undegraded":"2024-03-28T13:18:59.094055+0000","last_fullsized":"2024-03-28T13:18:59.094055+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:22:45.214275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-03-28T13:18:58.092690+0000","last_change":"2024-03-28T13:18:58.087418+0000","last_active":"2024-03-28T13:18:58.092690+0000","last_peered":"2024-03-28T13:18:58.092690+0000","last_clean":"2024-03-28T13:18:58.092690+0000","last_became_active":"2024-03-28T13:18:58.086932+0000","last_became_peered":"2024-03-28T13:18:58.086932+0000","last_unstale":"2024-03-28T13:18:58.092690+0000","last_undegraded":"2024-03-28T13:18:58.092690+0000","last_fullsized":"2024-03-28T13:18:58.092690+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:25:34.643120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-03-28T13:18:58.070236+0000","last_change":"2024-03-28T13:18:58.070236+0000","last_active":"2024-03-28T13:18:58.028202+0000","last_peered":"2024-03-28T13:18:58.028202+0000","last_clean":"2024-03-28T13:18:58.028202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:18:58.070236+0000","last_undegraded":"2024-03-28T13:18:58.070236+0000","last_fullsized":"2024-03-28T13:18:58.070236+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092249+0000","last_change":"2024-03-28T13:18:58.077769+0000","last_active":"2024-03-28T13:19:00.092249+0000","last_peered":"2024-03-28T13:19:00.092249+0000","last_clean":"2024-03-28T13:19:00.092249+0000","last_became_active":"2024-03-28T13:18:58.077234+0000","last_became_peered":"2024-03-28T13:18:58.077234+0000","last_unstale":"2024-03-28T13:19:00.092249+0000","last_undegraded":"2024-03-28T13:19:00.092249+0000","last_fullsized":"2024-03-28T13:19:00.092249+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:10:27.168742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.088777+0000","last_change":"2024-03-28T13:18:59.084112+0000","last_active":"2024-03-28T13:18:59.088777+0000","last_peered":"2024-03-28T13:18:59.088777+0000","last_clean":"2024-03-28T13:18:59.088777+0000","last_became_active":"2024-03-28T13:18:59.083805+0000","last_became_peered":"2024-03-28T13:18:59.083805+0000","last_unstale":"2024-03-28T13:18:59.088777+0000","last_undegraded":"2024-03-28T13:18:59.088777+0000","last_fullsized":"2024-03-28T13:18:59.088777+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:18:27.237579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.103124+0000","last_change":"2024-03-28T13:18:58.087832+0000","last_active":"2024-03-28T13:18:59.103124+0000","last_peered":"2024-03-28T13:18:59.103124+0000","last_clean":"2024-03-28T13:18:59.103124+0000","last_became_active":"2024-03-28T13:18:58.087250+0000","last_became_peered":"2024-03-28T13:18:58.087250+0000","last_unstale":"2024-03-28T13:18:59.103124+0000","last_undegraded":"2024-03-28T13:18:59.103124+0000","last_fullsized":"2024-03-28T13:18:59.103124+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:55:22.065081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.099692+0000","last_change":"2024-03-28T13:18:59.097206+0000","last_active":"2024-03-28T13:18:59.099692+0000","last_peered":"2024-03-28T13:18:59.099692+0000","last_clean":"2024-03-28T13:18:59.099692+0000","last_became_active":"2024-03-28T13:18:59.096903+0000","last_became_peered":"2024-03-28T13:18:59.096903+0000","last_unstale":"2024-03-28T13:18:59.099692+0000","last_undegraded":"2024-03-28T13:18:59.099692+0000","last_fullsized":"2024-03-28T13:18:59.099692+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:09.396769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.107170+0000","last_change":"2024-03-28T13:18:58.083531+0000","last_active":"2024-03-28T13:18:59.107170+0000","last_peered":"2024-03-28T13:18:59.107170+0000","last_clean":"2024-03-28T13:18:59.107170+0000","last_became_active":"2024-03-28T13:18:58.082970+0000","last_became_peered":"2024-03-28T13:18:58.082970+0000","last_unstale":"2024-03-28T13:18:59.107170+0000","last_undegraded":"2024-03-28T13:18:59.107170+0000","last_fullsized":"2024-03-28T13:18:59.107170+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:08.383486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.100576+0000","last_change":"2024-03-28T13:18:59.098346+0000","last_active":"2024-03-28T13:18:59.100576+0000","last_peered":"2024-03-28T13:18:59.100576+0000","last_clean":"2024-03-28T13:18:59.100576+0000","last_became_active":"2024-03-28T13:18:59.098052+0000","last_became_peered":"2024-03-28T13:18:59.098052+0000","last_unstale":"2024-03-28T13:18:59.100576+0000","last_undegraded":"2024-03-28T13:18:59.100576+0000","last_fullsized":"2024-03-28T13:18:59.100576+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:20:44.200653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.107233+0000","last_change":"2024-03-28T13:18:58.064634+0000","last_active":"2024-03-28T13:18:59.107233+0000","last_peered":"2024-03-28T13:18:59.107233+0000","last_clean":"2024-03-28T13:18:59.107233+0000","last_became_active":"2024-03-28T13:18:58.064194+0000","last_became_peered":"2024-03-28T13:18:58.064194+0000","last_unstale":"2024-03-28T13:18:59.107233+0000","last_undegraded":"2024-03-28T13:18:59.107233+0000","last_fullsized":"2024-03-28T13:18:59.107233+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:21:46.733401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091377+0000","last_change":"2024-03-28T13:18:59.094754+0000","last_active":"2024-03-28T13:19:00.091377+0000","last_peered":"2024-03-28T13:19:00.091377+0000","last_clean":"2024-03-28T13:19:00.091377+0000","last_became_active":"2024-03-28T13:18:59.090317+0000","last_became_peered":"2024-03-28T13:18:59.090317+0000","last_unstale":"2024-03-28T13:19:00.091377+0000","last_undegraded":"2024-03-28T13:19:00.091377+0000","last_fullsized":"2024-03-28T13:19:00.091377+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:09:17.961351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.101692+0000","last_change":"2024-03-28T13:18:58.082525+0000","last_active":"2024-03-28T13:18:59.101692+0000","last_peered":"2024-03-28T13:18:59.101692+0000","last_clean":"2024-03-28T13:18:59.101692+0000","last_became_active":"2024-03-28T13:18:58.082039+0000","last_became_peered":"2024-03-28T13:18:58.082039+0000","last_unstale":"2024-03-28T13:18:59.101692+0000","last_undegraded":"2024-03-28T13:18:59.101692+0000","last_fullsized":"2024-03-28T13:18:59.101692+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:42.609780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-03-28T13:18:58.072372+0000","last_change":"2024-03-28T13:18:58.072372+0000","last_active":"2024-03-28T13:18:58.028202+0000","last_peered":"2024-03-28T13:18:58.028202+0000","last_clean":"2024-03-28T13:18:58.028202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:18:58.072372+0000","last_undegraded":"2024-03-28T13:18:58.072372+0000","last_fullsized":"2024-03-28T13:18:58.072372+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090824+0000","last_change":"2024-03-28T13:18:58.078267+0000","last_active":"2024-03-28T13:19:00.090824+0000","last_peered":"2024-03-28T13:19:00.090824+0000","last_clean":"2024-03-28T13:19:00.090824+0000","last_became_active":"2024-03-28T13:18:58.077891+0000","last_became_peered":"2024-03-28T13:18:58.077891+0000","last_unstale":"2024-03-28T13:19:00.090824+0000","last_undegraded":"2024-03-28T13:19:00.090824+0000","last_fullsized":"2024-03-28T13:19:00.090824+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:35:04.854788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.102069+0000","last_change":"2024-03-28T13:18:59.099035+0000","last_active":"2024-03-28T13:18:59.102069+0000","last_peered":"2024-03-28T13:18:59.102069+0000","last_clean":"2024-03-28T13:18:59.102069+0000","last_became_active":"2024-03-28T13:18:59.098118+0000","last_became_peered":"2024-03-28T13:18:59.098118+0000","last_unstale":"2024-03-28T13:18:59.102069+0000","last_undegraded":"2024-03-28T13:18:59.102069+0000","last_fullsized":"2024-03-28T13:18:59.102069+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:21:47.746684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.059194+0000","last_change":"2024-03-28T13:18:58.076072+0000","last_active":"2024-03-28T13:18:59.059194+0000","last_peered":"2024-03-28T13:18:59.059194+0000","last_clean":"2024-03-28T13:18:59.059194+0000","last_became_active":"2024-03-28T13:18:58.075808+0000","last_became_peered":"2024-03-28T13:18:58.075808+0000","last_unstale":"2024-03-28T13:18:59.059194+0000","last_undegraded":"2024-03-28T13:18:59.059194+0000","last_fullsized":"2024-03-28T13:18:59.059194+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:11:22.068054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.083232+0000","last_change":"2024-03-28T13:18:59.075982+0000","last_active":"2024-03-28T13:18:59.083232+0000","last_peered":"2024-03-28T13:18:59.083232+0000","last_clean":"2024-03-28T13:18:59.083232+0000","last_became_active":"2024-03-28T13:18:59.075606+0000","last_became_peered":"2024-03-28T13:18:59.075606+0000","last_unstale":"2024-03-28T13:18:59.083232+0000","last_undegraded":"2024-03-28T13:18:59.083232+0000","last_fullsized":"2024-03-28T13:18:59.083232+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:26:09.194396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.078062+0000","last_change":"2024-03-28T13:18:58.077577+0000","last_active":"2024-03-28T13:18:59.078062+0000","last_peered":"2024-03-28T13:18:59.078062+0000","last_clean":"2024-03-28T13:18:59.078062+0000","last_became_active":"2024-03-28T13:18:58.077275+0000","last_became_peered":"2024-03-28T13:18:58.077275+0000","last_unstale":"2024-03-28T13:18:59.078062+0000","last_undegraded":"2024-03-28T13:18:59.078062+0000","last_fullsized":"2024-03-28T13:18:59.078062+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:28:48.630263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.089723+0000","last_change":"2024-03-28T13:18:59.086040+0000","last_active":"2024-03-28T13:18:59.089723+0000","last_peered":"2024-03-28T13:18:59.089723+0000","last_clean":"2024-03-28T13:18:59.089723+0000","last_became_active":"2024-03-28T13:18:59.085749+0000","last_became_peered":"2024-03-28T13:18:59.085749+0000","last_unstale":"2024-03-28T13:18:59.089723+0000","last_undegraded":"2024-03-28T13:18:59.089723+0000","last_fullsized":"2024-03-28T13:18:59.089723+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:27:52.931365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.061457+0000","last_change":"2024-03-28T13:18:58.083463+0000","last_active":"2024-03-28T13:18:59.061457+0000","last_peered":"2024-03-28T13:18:59.061457+0000","last_clean":"2024-03-28T13:18:59.061457+0000","last_became_active":"2024-03-28T13:18:58.083152+0000","last_became_peered":"2024-03-28T13:18:58.083152+0000","last_unstale":"2024-03-28T13:18:59.061457+0000","last_undegraded":"2024-03-28T13:18:59.061457+0000","last_fullsized":"2024-03-28T13:18:59.061457+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:48:03.913262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091380+0000","last_change":"2024-03-28T13:18:59.081727+0000","last_active":"2024-03-28T13:19:00.091380+0000","last_peered":"2024-03-28T13:19:00.091380+0000","last_clean":"2024-03-28T13:19:00.091380+0000","last_became_active":"2024-03-28T13:18:59.081436+0000","last_became_peered":"2024-03-28T13:18:59.081436+0000","last_unstale":"2024-03-28T13:19:00.091380+0000","last_undegraded":"2024-03-28T13:19:00.091380+0000","last_fullsized":"2024-03-28T13:19:00.091380+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:55:41.223520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"104'3","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092029+0000","last_change":"2024-03-28T13:18:58.068721+0000","last_active":"2024-03-28T13:19:00.092029+0000","last_peered":"2024-03-28T13:19:00.092029+0000","last_clean":"2024-03-28T13:19:00.092029+0000","last_became_active":"2024-03-28T13:18:58.068416+0000","last_became_peered":"2024-03-28T13:18:58.068416+0000","last_unstale":"2024-03-28T13:19:00.092029+0000","last_undegraded":"2024-03-28T13:19:00.092029+0000","last_fullsized":"2024-03-28T13:19:00.092029+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:16:13.945885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.086220+0000","last_change":"2024-03-28T13:18:59.080942+0000","last_active":"2024-03-28T13:18:59.086220+0000","last_peered":"2024-03-28T13:18:59.086220+0000","last_clean":"2024-03-28T13:18:59.086220+0000","last_became_active":"2024-03-28T13:18:59.080654+0000","last_became_peered":"2024-03-28T13:18:59.080654+0000","last_unstale":"2024-03-28T13:18:59.086220+0000","last_undegraded":"2024-03-28T13:18:59.086220+0000","last_fullsized":"2024-03-28T13:18:59.086220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:17:42.586454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"104'6","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.110193+0000","last_change":"2024-03-28T13:18:58.076129+0000","last_active":"2024-03-28T13:18:59.110193+0000","last_peered":"2024-03-28T13:18:59.110193+0000","last_clean":"2024-03-28T13:18:59.110193+0000","last_became_active":"2024-03-28T13:18:58.075809+0000","last_became_peered":"2024-03-28T13:18:58.075809+0000","last_unstale":"2024-03-28T13:18:59.110193+0000","last_undegraded":"2024-03-28T13:18:59.110193+0000","last_fullsized":"2024-03-28T13:18:59.110193+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:49:19.123418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087223+0000","last_change":"2024-03-28T13:18:59.077667+0000","last_active":"2024-03-28T13:18:59.087223+0000","last_peered":"2024-03-28T13:18:59.087223+0000","last_clean":"2024-03-28T13:18:59.087223+0000","last_became_active":"2024-03-28T13:18:59.077304+0000","last_became_peered":"2024-03-28T13:18:59.077304+0000","last_unstale":"2024-03-28T13:18:59.087223+0000","last_undegraded":"2024-03-28T13:18:59.087223+0000","last_fullsized":"2024-03-28T13:18:59.087223+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:43:24.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.105401+0000","last_change":"2024-03-28T13:18:58.077229+0000","last_active":"2024-03-28T13:18:59.105401+0000","last_peered":"2024-03-28T13:18:59.105401+0000","last_clean":"2024-03-28T13:18:59.105401+0000","last_became_active":"2024-03-28T13:18:58.076961+0000","last_became_peered":"2024-03-28T13:18:58.076961+0000","last_unstale":"2024-03-28T13:18:59.105401+0000","last_undegraded":"2024-03-28T13:18:59.105401+0000","last_fullsized":"2024-03-28T13:18:59.105401+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:20:48.852402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.092457+0000","last_change":"2024-03-28T13:18:59.084784+0000","last_active":"2024-03-28T13:18:59.092457+0000","last_peered":"2024-03-28T13:18:59.092457+0000","last_clean":"2024-03-28T13:18:59.092457+0000","last_became_active":"2024-03-28T13:18:59.084518+0000","last_became_peered":"2024-03-28T13:18:59.084518+0000","last_unstale":"2024-03-28T13:18:59.092457+0000","last_undegraded":"2024-03-28T13:18:59.092457+0000","last_fullsized":"2024-03-28T13:18:59.092457+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:49:20.136701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.059356+0000","last_change":"2024-03-28T13:18:58.083463+0000","last_active":"2024-03-28T13:18:59.059356+0000","last_peered":"2024-03-28T13:18:59.059356+0000","last_clean":"2024-03-28T13:18:59.059356+0000","last_became_active":"2024-03-28T13:18:58.083122+0000","last_became_peered":"2024-03-28T13:18:58.083122+0000","last_unstale":"2024-03-28T13:18:59.059356+0000","last_undegraded":"2024-03-28T13:18:59.059356+0000","last_fullsized":"2024-03-28T13:18:59.059356+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:20:43.187370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090923+0000","last_change":"2024-03-28T13:18:59.082251+0000","last_active":"2024-03-28T13:19:00.090923+0000","last_peered":"2024-03-28T13:19:00.090923+0000","last_clean":"2024-03-28T13:19:00.090923+0000","last_became_active":"2024-03-28T13:18:59.081951+0000","last_became_peered":"2024-03-28T13:18:59.081951+0000","last_unstale":"2024-03-28T13:19:00.090923+0000","last_undegraded":"2024-03-28T13:19:00.090923+0000","last_fullsized":"2024-03-28T13:19:00.090923+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:53.418554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087871+0000","last_change":"2024-03-28T13:18:58.066532+0000","last_active":"2024-03-28T13:18:59.087871+0000","last_peered":"2024-03-28T13:18:59.087871+0000","last_clean":"2024-03-28T13:18:59.087871+0000","last_became_active":"2024-03-28T13:18:58.066208+0000","last_became_peered":"2024-03-28T13:18:58.066208+0000","last_unstale":"2024-03-28T13:18:59.087871+0000","last_undegraded":"2024-03-28T13:18:59.087871+0000","last_fullsized":"2024-03-28T13:18:59.087871+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:58.349889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.093693+0000","last_change":"2024-03-28T13:18:59.090442+0000","last_active":"2024-03-28T13:18:59.093693+0000","last_peered":"2024-03-28T13:18:59.093693+0000","last_clean":"2024-03-28T13:18:59.093693+0000","last_became_active":"2024-03-28T13:18:59.090071+0000","last_became_peered":"2024-03-28T13:18:59.090071+0000","last_unstale":"2024-03-28T13:18:59.093693+0000","last_undegraded":"2024-03-28T13:18:59.093693+0000","last_fullsized":"2024-03-28T13:18:59.093693+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:17:18.839037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"104'8","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.108859+0000","last_change":"2024-03-28T13:18:58.071341+0000","last_active":"2024-03-28T13:18:59.108859+0000","last_peered":"2024-03-28T13:18:59.108859+0000","last_clean":"2024-03-28T13:18:59.108859+0000","last_became_active":"2024-03-28T13:18:58.069574+0000","last_became_peered":"2024-03-28T13:18:58.069574+0000","last_unstale":"2024-03-28T13:18:59.108859+0000","last_undegraded":"2024-03-28T13:18:59.108859+0000","last_fullsized":"2024-03-28T13:18:59.108859+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:13.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094648+0000","last_change":"2024-03-28T13:18:59.093731+0000","last_active":"2024-03-28T13:18:59.094648+0000","last_peered":"2024-03-28T13:18:59.094648+0000","last_clean":"2024-03-28T13:18:59.094648+0000","last_became_active":"2024-03-28T13:18:59.093471+0000","last_became_peered":"2024-03-28T13:18:59.093471+0000","last_unstale":"2024-03-28T13:18:59.094648+0000","last_undegraded":"2024-03-28T13:18:59.094648+0000","last_fullsized":"2024-03-28T13:18:59.094648+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:18:27.237579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.108148+0000","last_change":"2024-03-28T13:18:58.080576+0000","last_active":"2024-03-28T13:18:59.108148+0000","last_peered":"2024-03-28T13:18:59.108148+0000","last_clean":"2024-03-28T13:18:59.108148+0000","last_became_active":"2024-03-28T13:18:58.080272+0000","last_became_peered":"2024-03-28T13:18:58.080272+0000","last_unstale":"2024-03-28T13:18:59.108148+0000","last_undegraded":"2024-03-28T13:18:59.108148+0000","last_fullsized":"2024-03-28T13:18:59.108148+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:13.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094243+0000","last_change":"2024-03-28T13:18:59.092760+0000","last_active":"2024-03-28T13:18:59.094243+0000","last_peered":"2024-03-28T13:18:59.094243+0000","last_clean":"2024-03-28T13:18:59.094243+0000","last_became_active":"2024-03-28T13:18:59.092409+0000","last_became_peered":"2024-03-28T13:18:59.092409+0000","last_unstale":"2024-03-28T13:18:59.094243+0000","last_undegraded":"2024-03-28T13:18:59.094243+0000","last_fullsized":"2024-03-28T13:18:59.094243+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:19.489710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092132+0000","last_change":"2024-03-28T13:18:58.065630+0000","last_active":"2024-03-28T13:19:00.092132+0000","last_peered":"2024-03-28T13:19:00.092132+0000","last_clean":"2024-03-28T13:19:00.092132+0000","last_became_active":"2024-03-28T13:18:58.065333+0000","last_became_peered":"2024-03-28T13:18:58.065333+0000","last_unstale":"2024-03-28T13:19:00.092132+0000","last_undegraded":"2024-03-28T13:19:00.092132+0000","last_fullsized":"2024-03-28T13:19:00.092132+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:54:14.505307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.099342+0000","last_change":"2024-03-28T13:18:59.092928+0000","last_active":"2024-03-28T13:18:59.099342+0000","last_peered":"2024-03-28T13:18:59.099342+0000","last_clean":"2024-03-28T13:18:59.099342+0000","last_became_active":"2024-03-28T13:18:59.092579+0000","last_became_peered":"2024-03-28T13:18:59.092579+0000","last_unstale":"2024-03-28T13:18:59.099342+0000","last_undegraded":"2024-03-28T13:18:59.099342+0000","last_fullsized":"2024-03-28T13:18:59.099342+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:37:45.650491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.064300+0000","last_change":"2024-03-28T13:18:58.075199+0000","last_active":"2024-03-28T13:18:59.064300+0000","last_peered":"2024-03-28T13:18:59.064300+0000","last_clean":"2024-03-28T13:18:59.064300+0000","last_became_active":"2024-03-28T13:18:58.073665+0000","last_became_peered":"2024-03-28T13:18:58.073665+0000","last_unstale":"2024-03-28T13:18:59.064300+0000","last_undegraded":"2024-03-28T13:18:59.064300+0000","last_fullsized":"2024-03-28T13:18:59.064300+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:08:41.324830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.081523+0000","last_change":"2024-03-28T13:18:59.076634+0000","last_active":"2024-03-28T13:18:59.081523+0000","last_peered":"2024-03-28T13:18:59.081523+0000","last_clean":"2024-03-28T13:18:59.081523+0000","last_became_active":"2024-03-28T13:18:59.076295+0000","last_became_peered":"2024-03-28T13:18:59.076295+0000","last_unstale":"2024-03-28T13:18:59.081523+0000","last_undegraded":"2024-03-28T13:18:59.081523+0000","last_fullsized":"2024-03-28T13:18:59.081523+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:40.743749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.107170+0000","last_change":"2024-03-28T13:18:58.084188+0000","last_active":"2024-03-28T13:18:59.107170+0000","last_peered":"2024-03-28T13:18:59.107170+0000","last_clean":"2024-03-28T13:18:59.107170+0000","last_became_active":"2024-03-28T13:18:58.081720+0000","last_became_peered":"2024-03-28T13:18:58.081720+0000","last_unstale":"2024-03-28T13:18:59.107170+0000","last_undegraded":"2024-03-28T13:18:59.107170+0000","last_fullsized":"2024-03-28T13:18:59.107170+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:44.810836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.082259+0000","last_change":"2024-03-28T13:18:59.079466+0000","last_active":"2024-03-28T13:18:59.082259+0000","last_peered":"2024-03-28T13:18:59.082259+0000","last_clean":"2024-03-28T13:18:59.082259+0000","last_became_active":"2024-03-28T13:18:59.078752+0000","last_became_peered":"2024-03-28T13:18:59.078752+0000","last_unstale":"2024-03-28T13:18:59.082259+0000","last_undegraded":"2024-03-28T13:18:59.082259+0000","last_fullsized":"2024-03-28T13:18:59.082259+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:12:28.089180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.066401+0000","last_change":"2024-03-28T13:18:58.079436+0000","last_active":"2024-03-28T13:18:59.066401+0000","last_peered":"2024-03-28T13:18:59.066401+0000","last_clean":"2024-03-28T13:18:59.066401+0000","last_became_active":"2024-03-28T13:18:58.079066+0000","last_became_peered":"2024-03-28T13:18:58.079066+0000","last_unstale":"2024-03-28T13:18:59.066401+0000","last_undegraded":"2024-03-28T13:18:59.066401+0000","last_fullsized":"2024-03-28T13:18:59.066401+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:08.383486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087869+0000","last_change":"2024-03-28T13:18:59.080751+0000","last_active":"2024-03-28T13:18:59.087869+0000","last_peered":"2024-03-28T13:18:59.087869+0000","last_clean":"2024-03-28T13:18:59.087869+0000","last_became_active":"2024-03-28T13:18:59.080449+0000","last_became_peered":"2024-03-28T13:18:59.080449+0000","last_unstale":"2024-03-28T13:18:59.087869+0000","last_undegraded":"2024-03-28T13:18:59.087869+0000","last_fullsized":"2024-03-28T13:18:59.087869+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:17:42.586454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.075738+0000","last_change":"2024-03-28T13:18:58.079341+0000","last_active":"2024-03-28T13:18:59.075738+0000","last_peered":"2024-03-28T13:18:59.075738+0000","last_clean":"2024-03-28T13:18:59.075738+0000","last_became_active":"2024-03-28T13:18:58.079008+0000","last_became_peered":"2024-03-28T13:18:58.079008+0000","last_unstale":"2024-03-28T13:18:59.075738+0000","last_undegraded":"2024-03-28T13:18:59.075738+0000","last_fullsized":"2024-03-28T13:18:59.075738+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:21:46.733401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.086441+0000","last_change":"2024-03-28T13:18:59.080827+0000","last_active":"2024-03-28T13:18:59.086441+0000","last_peered":"2024-03-28T13:18:59.086441+0000","last_clean":"2024-03-28T13:18:59.086441+0000","last_became_active":"2024-03-28T13:18:59.080524+0000","last_became_peered":"2024-03-28T13:18:59.080524+0000","last_unstale":"2024-03-28T13:18:59.086441+0000","last_undegraded":"2024-03-28T13:18:59.086441+0000","last_fullsized":"2024-03-28T13:18:59.086441+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:43:24.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.072151+0000","last_change":"2024-03-28T13:18:58.090817+0000","last_active":"2024-03-28T13:18:59.072151+0000","last_peered":"2024-03-28T13:18:59.072151+0000","last_clean":"2024-03-28T13:18:59.072151+0000","last_became_active":"2024-03-28T13:18:58.090469+0000","last_became_peered":"2024-03-28T13:18:58.090469+0000","last_unstale":"2024-03-28T13:18:59.072151+0000","last_undegraded":"2024-03-28T13:18:59.072151+0000","last_fullsized":"2024-03-28T13:18:59.072151+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:52.858677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.080088+0000","last_change":"2024-03-28T13:18:59.071403+0000","last_active":"2024-03-28T13:18:59.080088+0000","last_peered":"2024-03-28T13:18:59.080088+0000","last_clean":"2024-03-28T13:18:59.080088+0000","last_became_active":"2024-03-28T13:18:59.071095+0000","last_became_peered":"2024-03-28T13:18:59.071095+0000","last_unstale":"2024-03-28T13:18:59.080088+0000","last_undegraded":"2024-03-28T13:18:59.080088+0000","last_fullsized":"2024-03-28T13:18:59.080088+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:18:33.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.105978+0000","last_change":"2024-03-28T13:18:58.071038+0000","last_active":"2024-03-28T13:18:59.105978+0000","last_peered":"2024-03-28T13:18:59.105978+0000","last_clean":"2024-03-28T13:18:59.105978+0000","last_became_active":"2024-03-28T13:18:58.069807+0000","last_became_peered":"2024-03-28T13:18:58.069807+0000","last_unstale":"2024-03-28T13:18:59.105978+0000","last_undegraded":"2024-03-28T13:18:59.105978+0000","last_fullsized":"2024-03-28T13:18:59.105978+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:17:17.825753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.080889+0000","last_change":"2024-03-28T13:18:59.076727+0000","last_active":"2024-03-28T13:18:59.080889+0000","last_peered":"2024-03-28T13:18:59.080889+0000","last_clean":"2024-03-28T13:18:59.080889+0000","last_became_active":"2024-03-28T13:18:59.076415+0000","last_became_peered":"2024-03-28T13:18:59.076415+0000","last_unstale":"2024-03-28T13:18:59.080889+0000","last_undegraded":"2024-03-28T13:18:59.080889+0000","last_fullsized":"2024-03-28T13:18:59.080889+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:14:07.278326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.066449+0000","last_change":"2024-03-28T13:18:58.086609+0000","last_active":"2024-03-28T13:18:59.066449+0000","last_peered":"2024-03-28T13:18:59.066449+0000","last_clean":"2024-03-28T13:18:59.066449+0000","last_became_active":"2024-03-28T13:18:58.086223+0000","last_became_peered":"2024-03-28T13:18:58.086223+0000","last_unstale":"2024-03-28T13:18:59.066449+0000","last_undegraded":"2024-03-28T13:18:59.066449+0000","last_fullsized":"2024-03-28T13:18:59.066449+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:17.621966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.099448+0000","last_change":"2024-03-28T13:18:59.098557+0000","last_active":"2024-03-28T13:18:59.099448+0000","last_peered":"2024-03-28T13:18:59.099448+0000","last_clean":"2024-03-28T13:18:59.099448+0000","last_became_active":"2024-03-28T13:18:59.098274+0000","last_became_peered":"2024-03-28T13:18:59.098274+0000","last_unstale":"2024-03-28T13:18:59.099448+0000","last_undegraded":"2024-03-28T13:18:59.099448+0000","last_fullsized":"2024-03-28T13:18:59.099448+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:14.876459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.065223+0000","last_change":"2024-03-28T13:18:58.072039+0000","last_active":"2024-03-28T13:18:59.065223+0000","last_peered":"2024-03-28T13:18:59.065223+0000","last_clean":"2024-03-28T13:18:59.065223+0000","last_became_active":"2024-03-28T13:18:58.071683+0000","last_became_peered":"2024-03-28T13:18:58.071683+0000","last_unstale":"2024-03-28T13:18:59.065223+0000","last_undegraded":"2024-03-28T13:18:59.065223+0000","last_fullsized":"2024-03-28T13:18:59.065223+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:47.118216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091918+0000","last_change":"2024-03-28T13:18:59.087471+0000","last_active":"2024-03-28T13:19:00.091918+0000","last_peered":"2024-03-28T13:19:00.091918+0000","last_clean":"2024-03-28T13:19:00.091918+0000","last_became_active":"2024-03-28T13:18:59.085466+0000","last_became_peered":"2024-03-28T13:18:59.085466+0000","last_unstale":"2024-03-28T13:19:00.091918+0000","last_undegraded":"2024-03-28T13:19:00.091918+0000","last_fullsized":"2024-03-28T13:19:00.091918+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:30:44.259677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.067207+0000","last_change":"2024-03-28T13:18:58.068025+0000","last_active":"2024-03-28T13:18:59.067207+0000","last_peered":"2024-03-28T13:18:59.067207+0000","last_clean":"2024-03-28T13:18:59.067207+0000","last_became_active":"2024-03-28T13:18:58.067671+0000","last_became_peered":"2024-03-28T13:18:58.067671+0000","last_unstale":"2024-03-28T13:18:59.067207+0000","last_undegraded":"2024-03-28T13:18:59.067207+0000","last_fullsized":"2024-03-28T13:18:59.067207+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:28.509110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.079227+0000","last_change":"2024-03-28T13:18:59.073922+0000","last_active":"2024-03-28T13:18:59.079227+0000","last_peered":"2024-03-28T13:18:59.079227+0000","last_clean":"2024-03-28T13:18:59.079227+0000","last_became_active":"2024-03-28T13:18:59.073566+0000","last_became_peered":"2024-03-28T13:18:59.073566+0000","last_unstale":"2024-03-28T13:18:59.079227+0000","last_undegraded":"2024-03-28T13:18:59.079227+0000","last_fullsized":"2024-03-28T13:18:59.079227+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:44:49.466448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.072738+0000","last_change":"2024-03-28T13:18:58.085176+0000","last_active":"2024-03-28T13:18:59.072738+0000","last_peered":"2024-03-28T13:18:59.072738+0000","last_clean":"2024-03-28T13:18:59.072738+0000","last_became_active":"2024-03-28T13:18:58.084898+0000","last_became_peered":"2024-03-28T13:18:58.084898+0000","last_unstale":"2024-03-28T13:18:59.072738+0000","last_undegraded":"2024-03-28T13:18:59.072738+0000","last_fullsized":"2024-03-28T13:18:59.072738+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:13:25.784483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091562+0000","last_change":"2024-03-28T13:18:59.076130+0000","last_active":"2024-03-28T13:19:00.091562+0000","last_peered":"2024-03-28T13:19:00.091562+0000","last_clean":"2024-03-28T13:19:00.091562+0000","last_became_active":"2024-03-28T13:18:59.075785+0000","last_became_peered":"2024-03-28T13:18:59.075785+0000","last_unstale":"2024-03-28T13:19:00.091562+0000","last_undegraded":"2024-03-28T13:19:00.091562+0000","last_fullsized":"2024-03-28T13:19:00.091562+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:31.836556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.065847+0000","last_change":"2024-03-28T13:18:58.073984+0000","last_active":"2024-03-28T13:18:59.065847+0000","last_peered":"2024-03-28T13:18:59.065847+0000","last_clean":"2024-03-28T13:18:59.065847+0000","last_became_active":"2024-03-28T13:18:58.073520+0000","last_became_peered":"2024-03-28T13:18:58.073520+0000","last_unstale":"2024-03-28T13:18:59.065847+0000","last_undegraded":"2024-03-28T13:18:59.065847+0000","last_fullsized":"2024-03-28T13:18:59.065847+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:28.886056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.082579+0000","last_change":"2024-03-28T13:18:59.078032+0000","last_active":"2024-03-28T13:18:59.082579+0000","last_peered":"2024-03-28T13:18:59.082579+0000","last_clean":"2024-03-28T13:18:59.082579+0000","last_became_active":"2024-03-28T13:18:59.077665+0000","last_became_peered":"2024-03-28T13:18:59.077665+0000","last_unstale":"2024-03-28T13:18:59.082579+0000","last_undegraded":"2024-03-28T13:18:59.082579+0000","last_fullsized":"2024-03-28T13:18:59.082579+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:04:28.366038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-03-28T13:18:58.085201+0000","last_change":"2024-03-28T13:18:58.083533+0000","last_active":"2024-03-28T13:18:58.085201+0000","last_peered":"2024-03-28T13:18:58.085201+0000","last_clean":"2024-03-28T13:18:58.085201+0000","last_became_active":"2024-03-28T13:18:58.083220+0000","last_became_peered":"2024-03-28T13:18:58.083220+0000","last_unstale":"2024-03-28T13:18:58.085201+0000","last_undegraded":"2024-03-28T13:18:58.085201+0000","last_fullsized":"2024-03-28T13:18:58.085201+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:28:30.860196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.098913+0000","last_change":"2024-03-28T13:18:59.091706+0000","last_active":"2024-03-28T13:18:59.098913+0000","last_peered":"2024-03-28T13:18:59.098913+0000","last_clean":"2024-03-28T13:18:59.098913+0000","last_became_active":"2024-03-28T13:18:59.091410+0000","last_became_peered":"2024-03-28T13:18:59.091410+0000","last_unstale":"2024-03-28T13:18:59.098913+0000","last_undegraded":"2024-03-28T13:18:59.098913+0000","last_fullsized":"2024-03-28T13:18:59.098913+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:15.681052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.072573+0000","last_change":"2024-03-28T13:18:58.086792+0000","last_active":"2024-03-28T13:18:59.072573+0000","last_peered":"2024-03-28T13:18:59.072573+0000","last_clean":"2024-03-28T13:18:59.072573+0000","last_became_active":"2024-03-28T13:18:58.086544+0000","last_became_peered":"2024-03-28T13:18:58.086544+0000","last_unstale":"2024-03-28T13:18:59.072573+0000","last_undegraded":"2024-03-28T13:18:59.072573+0000","last_fullsized":"2024-03-28T13:18:59.072573+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:51:50.282781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.092724+0000","last_change":"2024-03-28T13:18:59.088616+0000","last_active":"2024-03-28T13:18:59.092724+0000","last_peered":"2024-03-28T13:18:59.092724+0000","last_clean":"2024-03-28T13:18:59.092724+0000","last_became_active":"2024-03-28T13:18:59.088292+0000","last_became_peered":"2024-03-28T13:18:59.088292+0000","last_unstale":"2024-03-28T13:18:59.092724+0000","last_undegraded":"2024-03-28T13:18:59.092724+0000","last_fullsized":"2024-03-28T13:18:59.092724+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:37:45.650491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.107904+0000","last_change":"2024-03-28T13:18:58.075299+0000","last_active":"2024-03-28T13:18:59.107904+0000","last_peered":"2024-03-28T13:18:59.107904+0000","last_clean":"2024-03-28T13:18:59.107904+0000","last_became_active":"2024-03-28T13:18:58.075035+0000","last_became_peered":"2024-03-28T13:18:58.075035+0000","last_unstale":"2024-03-28T13:18:59.107904+0000","last_undegraded":"2024-03-28T13:18:59.107904+0000","last_fullsized":"2024-03-28T13:18:59.107904+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:39.277367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":320,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087250+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:18:59.087250+0000","last_peered":"2024-03-28T13:18:59.087250+0000","last_clean":"2024-03-28T13:18:59.087250+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:18:59.087250+0000","last_undegraded":"2024-03-28T13:18:59.087250+0000","last_fullsized":"2024-03-28T13:18:59.087250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476856,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":550446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":550322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1888256,"data_stored":1142344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1273856,"data_stored":550252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1261568,"data_stored":543477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1855488,"data_stored":1133845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1261568,"data_stored":543477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1261568,"data_stored":543477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:19:03.655 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:19:03.871 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:19:03.879 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:19:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.909+0000 7f04b6600640 1 Processor -- start 2024-03-28T13:19:03.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.909+0000 7f04b6600640 1 -- start start 2024-03-28T13:19:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.911+0000 7f04b6600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f04b015e750 0x7f04b00cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.911+0000 7f04b6600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f04b006bc80 0x7f04b00726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.911+0000 7f04b6600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04b00cd830 con 0x7f04b00cd270 2024-03-28T13:19:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.911+0000 7f04b6600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f04b00cdb10 con 0x7f04b006bc80 2024-03-28T13:19:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.911+0000 7f04b6600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f04b00cddf0 con 0x7f04b015e750 2024-03-28T13:19:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.911+0000 7f04afe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f04b006bc80 0x7f04b00726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.911+0000 7f04afe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f04b006bc80 0x7f04b00726b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59398/0 (socket says 172.21.15.19:59398) 2024-03-28T13:19:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.911+0000 7f04afe00640 1 -- 172.21.15.19:0/3914999162 learned_addr learned my addr 172.21.15.19:0/3914999162 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:03.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.911+0000 7f04af400640 1 --2- 172.21.15.19:0/3914999162 >> v2:172.21.15.188:3300/0 conn(0x7f04b015e750 0x7f04b00cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:03.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.912+0000 7f04afe00640 1 -- 172.21.15.19:0/3914999162 >> v2:172.21.15.188:3300/0 conn(0x7f04b015e750 msgr2=0x7f04b00cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:03.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.912+0000 7f04afe00640 1 --2- 172.21.15.19:0/3914999162 >> v2:172.21.15.188:3300/0 conn(0x7f04b015e750 0x7f04b00cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.912+0000 7f04afe00640 1 -- 172.21.15.19:0/3914999162 >> v1:172.21.15.188:6789/0 conn(0x7f04b00cd270 legacy=0x7f04b006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:03.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.912+0000 7f04afe00640 1 -- 172.21.15.19:0/3914999162 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b00ce0d0 con 0x7f04b006bc80 2024-03-28T13:19:03.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.912+0000 7f04ae000640 1 -- 172.21.15.19:0/3914999162 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1953178426 0 0) 0x7f04b00cd830 con 0x7f04b00cd270 2024-03-28T13:19:03.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.914+0000 7f04afe00640 1 --2- 172.21.15.19:0/3914999162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b006bc80 0x7f04b00726b0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f04b00d10a0 tx=0x7f04a802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8735029bb5aaa9ae server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.914+0000 7f04ae000640 1 -- 172.21.15.19:0/3914999162 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a802cb10 con 0x7f04b006bc80 2024-03-28T13:19:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.914+0000 7f04ae000640 1 -- 172.21.15.19:0/3914999162 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a802ccb0 con 0x7f04b006bc80 2024-03-28T13:19:03.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.914+0000 7f04ae000640 1 -- 172.21.15.19:0/3914999162 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a802ce60 con 0x7f04b006bc80 2024-03-28T13:19:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.915+0000 7f04b6600640 1 -- 172.21.15.19:0/3914999162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b006bc80 msgr2=0x7f04b00726b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.915+0000 7f04b6600640 1 --2- 172.21.15.19:0/3914999162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b006bc80 0x7f04b00726b0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f04b00d10a0 tx=0x7f04a802bf70 comp rx=0 tx=0).stop 2024-03-28T13:19:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.915+0000 7f04b6600640 1 -- 172.21.15.19:0/3914999162 shutdown_connections 2024-03-28T13:19:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.915+0000 7f04b6600640 1 --2- 172.21.15.19:0/3914999162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b006bc80 0x7f04b00726b0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.915+0000 7f04b6600640 1 --2- 172.21.15.19:0/3914999162 >> v2:172.21.15.188:3300/0 conn(0x7f04b015e750 0x7f04b00cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.915+0000 7f04b6600640 1 -- 172.21.15.19:0/3914999162 >> 172.21.15.19:0/3914999162 conn(0x7f04b00b2cb0 msgr2=0x7f04b00b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.915+0000 7f04b6600640 1 -- 172.21.15.19:0/3914999162 shutdown_connections 2024-03-28T13:19:03.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.915+0000 7f04b6600640 1 -- 172.21.15.19:0/3914999162 wait complete. 2024-03-28T13:19:03.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.917+0000 7f04b6600640 1 Processor -- start 2024-03-28T13:19:03.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.917+0000 7f04b6600640 1 -- start start 2024-03-28T13:19:03.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04b6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b006bc80 0x7f04b01f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:03.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04b6600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04b00cd270 0x7f04b0158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:03.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04b6600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04b015e750 0x7f04b015afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:03.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04b6600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f04b00ce8d0 con 0x7f04b006bc80 2024-03-28T13:19:03.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04b6600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f04b015eb30 con 0x7f04b015e750 2024-03-28T13:19:03.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04b6600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f04b00cea50 con 0x7f04b00cd270 2024-03-28T13:19:03.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04af400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b006bc80 0x7f04b01f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04af400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b006bc80 0x7f04b01f8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59404/0 (socket says 172.21.15.19:59404) 2024-03-28T13:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04af400640 1 -- 172.21.15.19:0/4143524857 learned_addr learned my addr 172.21.15.19:0/4143524857 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04afe00640 1 --2- 172.21.15.19:0/4143524857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04b015e750 0x7f04b015afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04aea00640 1 --2- 172.21.15.19:0/4143524857 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04b00cd270 0x7f04b0158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04af400640 1 -- 172.21.15.19:0/4143524857 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04b00cd270 msgr2=0x7f04b0158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04af400640 1 --2- 172.21.15.19:0/4143524857 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04b00cd270 0x7f04b0158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04af400640 1 -- 172.21.15.19:0/4143524857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04b015e750 msgr2=0x7f04b015afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04af400640 1 --2- 172.21.15.19:0/4143524857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04b015e750 0x7f04b015afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04af400640 1 -- 172.21.15.19:0/4143524857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b015b510 con 0x7f04b006bc80 2024-03-28T13:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.918+0000 7f04aea00640 1 --2- 172.21.15.19:0/4143524857 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04b00cd270 0x7f04b0158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.919+0000 7f04af400640 1 --2- 172.21.15.19:0/4143524857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b006bc80 0x7f04b01f8040 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f04a000e870 tx=0x7f04a000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.919+0000 7f04a7400640 1 -- 172.21.15.19:0/4143524857 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0017070 con 0x7f04b006bc80 2024-03-28T13:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.919+0000 7f04a7400640 1 -- 172.21.15.19:0/4143524857 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a000c940 con 0x7f04b006bc80 2024-03-28T13:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.919+0000 7f04a7400640 1 -- 172.21.15.19:0/4143524857 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a000cd30 con 0x7f04b006bc80 2024-03-28T13:19:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.919+0000 7f04b6600640 1 -- 172.21.15.19:0/4143524857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b015b7b0 con 0x7f04b006bc80 2024-03-28T13:19:03.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.920+0000 7f04b6600640 1 -- 172.21.15.19:0/4143524857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b01f7840 con 0x7f04b006bc80 2024-03-28T13:19:03.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.922+0000 7f04b6600640 1 -- 172.21.15.19:0/4143524857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b01f7c70 con 0x7f04b006bc80 2024-03-28T13:19:03.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.922+0000 7f04a7400640 1 -- 172.21.15.19:0/4143524857 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f04a001a070 con 0x7f04b006bc80 2024-03-28T13:19:03.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.923+0000 7f04a7400640 1 --2- 172.21.15.19:0/4143524857 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f049006c580 0x7f049006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:03.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.923+0000 7f04a7400640 1 -- 172.21.15.19:0/4143524857 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f04a008b500 con 0x7f04b006bc80 2024-03-28T13:19:03.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.924+0000 7f04aea00640 1 --2- 172.21.15.19:0/4143524857 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f049006c580 0x7f049006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:03.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.925+0000 7f04aea00640 1 --2- 172.21.15.19:0/4143524857 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f049006c580 0x7f049006ea40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f04b01f7420 tx=0x7f049c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:03.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:03.928+0000 7f04a7400640 1 -- 172.21.15.19:0/4143524857 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f04a00594b0 con 0x7f04b006bc80 2024-03-28T13:19:04.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.133+0000 7f04b6600640 1 -- 172.21.15.19:0/4143524857 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f04b00685c0 con 0x7f049006c580 2024-03-28T13:19:04.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.151+0000 7f04a7400640 1 -- 172.21.15.19:0/4143524857 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251926 (secure 0 0 0) 0x7f04b00685c0 con 0x7f049006c580 2024-03-28T13:19:04.154 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:04.155 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:19:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.156+0000 7f04b6600640 1 -- 172.21.15.19:0/4143524857 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f049006c580 msgr2=0x7f049006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.156+0000 7f04b6600640 1 --2- 172.21.15.19:0/4143524857 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f049006c580 0x7f049006ea40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f04b01f7420 tx=0x7f049c009040 comp rx=0 tx=0).stop 2024-03-28T13:19:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.156+0000 7f04b6600640 1 -- 172.21.15.19:0/4143524857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b006bc80 msgr2=0x7f04b01f8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.156+0000 7f04b6600640 1 --2- 172.21.15.19:0/4143524857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b006bc80 0x7f04b01f8040 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f04a000e870 tx=0x7f04a000ed30 comp rx=0 tx=0).stop 2024-03-28T13:19:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.156+0000 7f04b6600640 1 -- 172.21.15.19:0/4143524857 shutdown_connections 2024-03-28T13:19:04.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.156+0000 7f04b6600640 1 --2- 172.21.15.19:0/4143524857 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f049006c580 0x7f049006ea40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.156+0000 7f04b6600640 1 --2- 172.21.15.19:0/4143524857 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f04b015e750 0x7f04b015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.156+0000 7f04b6600640 1 --2- 172.21.15.19:0/4143524857 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f04b00cd270 0x7f04b0158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.156+0000 7f04b6600640 1 --2- 172.21.15.19:0/4143524857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f04b006bc80 0x7f04b01f8040 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.156+0000 7f04b6600640 1 -- 172.21.15.19:0/4143524857 >> 172.21.15.19:0/4143524857 conn(0x7f04b00b2cb0 msgr2=0x7f04b0157f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.157+0000 7f04b6600640 1 -- 172.21.15.19:0/4143524857 shutdown_connections 2024-03-28T13:19:04.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.157+0000 7f04b6600640 1 -- 172.21.15.19:0/4143524857 wait complete. 2024-03-28T13:19:04.176 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-03-28T13:19:02.197436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":199,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260556,"kb_used_data":11100,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714164,"statfs":{"total":773094113280,"available":772827303936,"internally_reserved":0,"allocated":11366400,"data_stored":5557640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.167405"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.091975+0000","last_change":"2024-03-28T13:18:59.085702+0000","last_active":"2024-03-28T13:18:59.091975+0000","last_peered":"2024-03-28T13:18:59.091975+0000","last_clean":"2024-03-28T13:18:59.091975+0000","last_became_active":"2024-03-28T13:18:59.085413+0000","last_became_peered":"2024-03-28T13:18:59.085413+0000","last_unstale":"2024-03-28T13:18:59.091975+0000","last_undegraded":"2024-03-28T13:18:59.091975+0000","last_fullsized":"2024-03-28T13:18:59.091975+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:05:04.270844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094323+0000","last_change":"2024-03-28T13:18:59.090611+0000","last_active":"2024-03-28T13:18:59.094323+0000","last_peered":"2024-03-28T13:18:59.094323+0000","last_clean":"2024-03-28T13:18:59.094323+0000","last_became_active":"2024-03-28T13:18:59.089972+0000","last_became_peered":"2024-03-28T13:18:59.089972+0000","last_unstale":"2024-03-28T13:18:59.094323+0000","last_undegraded":"2024-03-28T13:18:59.094323+0000","last_fullsized":"2024-03-28T13:18:59.094323+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:35:49.070231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.100115+0000","last_change":"2024-03-28T13:18:59.098655+0000","last_active":"2024-03-28T13:18:59.100115+0000","last_peered":"2024-03-28T13:18:59.100115+0000","last_clean":"2024-03-28T13:18:59.100115+0000","last_became_active":"2024-03-28T13:18:59.098399+0000","last_became_peered":"2024-03-28T13:18:59.098399+0000","last_unstale":"2024-03-28T13:18:59.100115+0000","last_undegraded":"2024-03-28T13:18:59.100115+0000","last_fullsized":"2024-03-28T13:18:59.100115+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:20:44.200653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.083142+0000","last_change":"2024-03-28T13:18:59.074664+0000","last_active":"2024-03-28T13:18:59.083142+0000","last_peered":"2024-03-28T13:18:59.083142+0000","last_clean":"2024-03-28T13:18:59.083142+0000","last_became_active":"2024-03-28T13:18:59.073993+0000","last_became_peered":"2024-03-28T13:18:59.073993+0000","last_unstale":"2024-03-28T13:18:59.083142+0000","last_undegraded":"2024-03-28T13:18:59.083142+0000","last_fullsized":"2024-03-28T13:18:59.083142+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:29.899339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091305+0000","last_change":"2024-03-28T13:18:59.090589+0000","last_active":"2024-03-28T13:19:00.091305+0000","last_peered":"2024-03-28T13:19:00.091305+0000","last_clean":"2024-03-28T13:19:00.091305+0000","last_became_active":"2024-03-28T13:18:59.089952+0000","last_became_peered":"2024-03-28T13:18:59.089952+0000","last_unstale":"2024-03-28T13:19:00.091305+0000","last_undegraded":"2024-03-28T13:19:00.091305+0000","last_fullsized":"2024-03-28T13:19:00.091305+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:09.506244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091182+0000","last_change":"2024-03-28T13:18:59.085728+0000","last_active":"2024-03-28T13:19:00.091182+0000","last_peered":"2024-03-28T13:19:00.091182+0000","last_clean":"2024-03-28T13:19:00.091182+0000","last_became_active":"2024-03-28T13:18:59.085447+0000","last_became_peered":"2024-03-28T13:18:59.085447+0000","last_unstale":"2024-03-28T13:19:00.091182+0000","last_undegraded":"2024-03-28T13:19:00.091182+0000","last_fullsized":"2024-03-28T13:19:00.091182+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:16.716159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091117+0000","last_change":"2024-03-28T13:18:59.090899+0000","last_active":"2024-03-28T13:19:00.091117+0000","last_peered":"2024-03-28T13:19:00.091117+0000","last_clean":"2024-03-28T13:19:00.091117+0000","last_became_active":"2024-03-28T13:18:59.086628+0000","last_became_peered":"2024-03-28T13:18:59.086628+0000","last_unstale":"2024-03-28T13:19:00.091117+0000","last_undegraded":"2024-03-28T13:19:00.091117+0000","last_fullsized":"2024-03-28T13:19:00.091117+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:05:31.597035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.092180+0000","last_change":"2024-03-28T13:18:59.082961+0000","last_active":"2024-03-28T13:18:59.092180+0000","last_peered":"2024-03-28T13:18:59.092180+0000","last_clean":"2024-03-28T13:18:59.092180+0000","last_became_active":"2024-03-28T13:18:59.082608+0000","last_became_peered":"2024-03-28T13:18:59.082608+0000","last_unstale":"2024-03-28T13:18:59.092180+0000","last_undegraded":"2024-03-28T13:18:59.092180+0000","last_fullsized":"2024-03-28T13:18:59.092180+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:59.363172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092358+0000","last_change":"2024-03-28T13:18:59.084837+0000","last_active":"2024-03-28T13:19:00.092358+0000","last_peered":"2024-03-28T13:19:00.092358+0000","last_clean":"2024-03-28T13:19:00.092358+0000","last_became_active":"2024-03-28T13:18:59.083085+0000","last_became_peered":"2024-03-28T13:18:59.083085+0000","last_unstale":"2024-03-28T13:19:00.092358+0000","last_undegraded":"2024-03-28T13:19:00.092358+0000","last_fullsized":"2024-03-28T13:19:00.092358+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:16:46.004371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094293+0000","last_change":"2024-03-28T13:18:59.088878+0000","last_active":"2024-03-28T13:18:59.094293+0000","last_peered":"2024-03-28T13:18:59.094293+0000","last_clean":"2024-03-28T13:18:59.094293+0000","last_became_active":"2024-03-28T13:18:59.088493+0000","last_became_peered":"2024-03-28T13:18:59.088493+0000","last_unstale":"2024-03-28T13:18:59.094293+0000","last_undegraded":"2024-03-28T13:18:59.094293+0000","last_fullsized":"2024-03-28T13:18:59.094293+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:48:04.926545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091805+0000","last_change":"2024-03-28T13:18:59.100683+0000","last_active":"2024-03-28T13:19:00.091805+0000","last_peered":"2024-03-28T13:19:00.091805+0000","last_clean":"2024-03-28T13:19:00.091805+0000","last_became_active":"2024-03-28T13:18:59.099214+0000","last_became_peered":"2024-03-28T13:18:59.099214+0000","last_unstale":"2024-03-28T13:19:00.091805+0000","last_undegraded":"2024-03-28T13:19:00.091805+0000","last_fullsized":"2024-03-28T13:19:00.091805+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:41.892054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.095497+0000","last_change":"2024-03-28T13:18:59.090740+0000","last_active":"2024-03-28T13:18:59.095497+0000","last_peered":"2024-03-28T13:18:59.095497+0000","last_clean":"2024-03-28T13:18:59.095497+0000","last_became_active":"2024-03-28T13:18:59.090473+0000","last_became_peered":"2024-03-28T13:18:59.090473+0000","last_unstale":"2024-03-28T13:18:59.095497+0000","last_undegraded":"2024-03-28T13:18:59.095497+0000","last_fullsized":"2024-03-28T13:18:59.095497+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:31.353091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-03-28T13:18:58.069477+0000","last_change":"2024-03-28T13:18:58.069477+0000","last_active":"2024-03-28T13:18:58.028202+0000","last_peered":"2024-03-28T13:18:58.028202+0000","last_clean":"2024-03-28T13:18:58.028202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:18:58.069477+0000","last_undegraded":"2024-03-28T13:18:58.069477+0000","last_fullsized":"2024-03-28T13:18:58.069477+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087240+0000","last_change":"2024-03-28T13:18:59.080979+0000","last_active":"2024-03-28T13:18:59.087240+0000","last_peered":"2024-03-28T13:18:59.087240+0000","last_clean":"2024-03-28T13:18:59.087240+0000","last_became_active":"2024-03-28T13:18:59.080676+0000","last_became_peered":"2024-03-28T13:18:59.080676+0000","last_unstale":"2024-03-28T13:18:59.087240+0000","last_undegraded":"2024-03-28T13:18:59.087240+0000","last_fullsized":"2024-03-28T13:18:59.087240+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:00.261227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087400+0000","last_change":"2024-03-28T13:18:59.077084+0000","last_active":"2024-03-28T13:18:59.087400+0000","last_peered":"2024-03-28T13:18:59.087400+0000","last_clean":"2024-03-28T13:18:59.087400+0000","last_became_active":"2024-03-28T13:18:59.076136+0000","last_became_peered":"2024-03-28T13:18:59.076136+0000","last_unstale":"2024-03-28T13:18:59.087400+0000","last_undegraded":"2024-03-28T13:18:59.087400+0000","last_fullsized":"2024-03-28T13:18:59.087400+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:20.977905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090808+0000","last_change":"2024-03-28T13:18:59.074866+0000","last_active":"2024-03-28T13:19:00.090808+0000","last_peered":"2024-03-28T13:19:00.090808+0000","last_clean":"2024-03-28T13:19:00.090808+0000","last_became_active":"2024-03-28T13:18:59.074514+0000","last_became_peered":"2024-03-28T13:18:59.074514+0000","last_unstale":"2024-03-28T13:19:00.090808+0000","last_undegraded":"2024-03-28T13:19:00.090808+0000","last_fullsized":"2024-03-28T13:19:00.090808+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:10:44.680345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.090237+0000","last_change":"2024-03-28T13:18:59.084712+0000","last_active":"2024-03-28T13:18:59.090237+0000","last_peered":"2024-03-28T13:18:59.090237+0000","last_clean":"2024-03-28T13:18:59.090237+0000","last_became_active":"2024-03-28T13:18:59.084435+0000","last_became_peered":"2024-03-28T13:18:59.084435+0000","last_unstale":"2024-03-28T13:18:59.090237+0000","last_undegraded":"2024-03-28T13:18:59.090237+0000","last_fullsized":"2024-03-28T13:18:59.090237+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:08:49.381746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087507+0000","last_change":"2024-03-28T13:18:59.083710+0000","last_active":"2024-03-28T13:18:59.087507+0000","last_peered":"2024-03-28T13:18:59.087507+0000","last_clean":"2024-03-28T13:18:59.087507+0000","last_became_active":"2024-03-28T13:18:59.083366+0000","last_became_peered":"2024-03-28T13:18:59.083366+0000","last_unstale":"2024-03-28T13:18:59.087507+0000","last_undegraded":"2024-03-28T13:18:59.087507+0000","last_fullsized":"2024-03-28T13:18:59.087507+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:19.489710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.081195+0000","last_change":"2024-03-28T13:18:59.076959+0000","last_active":"2024-03-28T13:18:59.081195+0000","last_peered":"2024-03-28T13:18:59.081195+0000","last_clean":"2024-03-28T13:18:59.081195+0000","last_became_active":"2024-03-28T13:18:59.075601+0000","last_became_peered":"2024-03-28T13:18:59.075601+0000","last_unstale":"2024-03-28T13:18:59.081195+0000","last_undegraded":"2024-03-28T13:18:59.081195+0000","last_fullsized":"2024-03-28T13:18:59.081195+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:00:07.119299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.093215+0000","last_change":"2024-03-28T13:18:59.087556+0000","last_active":"2024-03-28T13:18:59.093215+0000","last_peered":"2024-03-28T13:18:59.093215+0000","last_clean":"2024-03-28T13:18:59.093215+0000","last_became_active":"2024-03-28T13:18:59.087183+0000","last_became_peered":"2024-03-28T13:18:59.087183+0000","last_unstale":"2024-03-28T13:18:59.093215+0000","last_undegraded":"2024-03-28T13:18:59.093215+0000","last_fullsized":"2024-03-28T13:18:59.093215+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:28:49.643546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.092574+0000","last_change":"2024-03-28T13:18:59.085370+0000","last_active":"2024-03-28T13:18:59.092574+0000","last_peered":"2024-03-28T13:18:59.092574+0000","last_clean":"2024-03-28T13:18:59.092574+0000","last_became_active":"2024-03-28T13:18:59.084826+0000","last_became_peered":"2024-03-28T13:18:59.084826+0000","last_unstale":"2024-03-28T13:18:59.092574+0000","last_undegraded":"2024-03-28T13:18:59.092574+0000","last_fullsized":"2024-03-28T13:18:59.092574+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:31.353091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094564+0000","last_change":"2024-03-28T13:18:59.091677+0000","last_active":"2024-03-28T13:18:59.094564+0000","last_peered":"2024-03-28T13:18:59.094564+0000","last_clean":"2024-03-28T13:18:59.094564+0000","last_became_active":"2024-03-28T13:18:59.091299+0000","last_became_peered":"2024-03-28T13:18:59.091299+0000","last_unstale":"2024-03-28T13:18:59.094564+0000","last_undegraded":"2024-03-28T13:18:59.094564+0000","last_fullsized":"2024-03-28T13:18:59.094564+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:40.290650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087426+0000","last_change":"2024-03-28T13:18:59.083262+0000","last_active":"2024-03-28T13:18:59.087426+0000","last_peered":"2024-03-28T13:18:59.087426+0000","last_clean":"2024-03-28T13:18:59.087426+0000","last_became_active":"2024-03-28T13:18:59.082908+0000","last_became_peered":"2024-03-28T13:18:59.082908+0000","last_unstale":"2024-03-28T13:18:59.087426+0000","last_undegraded":"2024-03-28T13:18:59.087426+0000","last_fullsized":"2024-03-28T13:18:59.087426+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:27:52.931365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.089502+0000","last_change":"2024-03-28T13:18:59.085398+0000","last_active":"2024-03-28T13:18:59.089502+0000","last_peered":"2024-03-28T13:18:59.089502+0000","last_clean":"2024-03-28T13:18:59.089502+0000","last_became_active":"2024-03-28T13:18:59.083434+0000","last_became_peered":"2024-03-28T13:18:59.083434+0000","last_unstale":"2024-03-28T13:18:59.089502+0000","last_undegraded":"2024-03-28T13:18:59.089502+0000","last_fullsized":"2024-03-28T13:18:59.089502+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:00.261227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.090300+0000","last_change":"2024-03-28T13:18:59.084798+0000","last_active":"2024-03-28T13:18:59.090300+0000","last_peered":"2024-03-28T13:18:59.090300+0000","last_clean":"2024-03-28T13:18:59.090300+0000","last_became_active":"2024-03-28T13:18:59.084216+0000","last_became_peered":"2024-03-28T13:18:59.084216+0000","last_unstale":"2024-03-28T13:18:59.090300+0000","last_undegraded":"2024-03-28T13:18:59.090300+0000","last_fullsized":"2024-03-28T13:18:59.090300+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:45.824120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-03-28T13:18:58.071873+0000","last_change":"2024-03-28T13:18:58.071873+0000","last_active":"2024-03-28T13:18:58.028202+0000","last_peered":"2024-03-28T13:18:58.028202+0000","last_clean":"2024-03-28T13:18:58.028202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:18:58.071873+0000","last_undegraded":"2024-03-28T13:18:58.071873+0000","last_fullsized":"2024-03-28T13:18:58.071873+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.086078+0000","last_change":"2024-03-28T13:18:59.080256+0000","last_active":"2024-03-28T13:18:59.086078+0000","last_peered":"2024-03-28T13:18:59.086078+0000","last_clean":"2024-03-28T13:18:59.086078+0000","last_became_active":"2024-03-28T13:18:59.079957+0000","last_became_peered":"2024-03-28T13:18:59.079957+0000","last_unstale":"2024-03-28T13:18:59.086078+0000","last_undegraded":"2024-03-28T13:18:59.086078+0000","last_fullsized":"2024-03-28T13:18:59.086078+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:20.977905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.084557+0000","last_change":"2024-03-28T13:18:59.078806+0000","last_active":"2024-03-28T13:18:59.084557+0000","last_peered":"2024-03-28T13:18:59.084557+0000","last_clean":"2024-03-28T13:18:59.084557+0000","last_became_active":"2024-03-28T13:18:59.078526+0000","last_became_peered":"2024-03-28T13:18:59.078526+0000","last_unstale":"2024-03-28T13:18:59.084557+0000","last_undegraded":"2024-03-28T13:18:59.084557+0000","last_fullsized":"2024-03-28T13:18:59.084557+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:26:09.194396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-03-28T13:18:58.068259+0000","last_change":"2024-03-28T13:18:58.068259+0000","last_active":"2024-03-28T13:18:58.028202+0000","last_peered":"2024-03-28T13:18:58.028202+0000","last_clean":"2024-03-28T13:18:58.028202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:18:58.068259+0000","last_undegraded":"2024-03-28T13:18:58.068259+0000","last_fullsized":"2024-03-28T13:18:58.068259+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-03-28T13:18:58.074341+0000","last_change":"2024-03-28T13:18:58.074341+0000","last_active":"2024-03-28T13:18:58.028202+0000","last_peered":"2024-03-28T13:18:58.028202+0000","last_clean":"2024-03-28T13:18:58.028202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:18:58.074341+0000","last_undegraded":"2024-03-28T13:18:58.074341+0000","last_fullsized":"2024-03-28T13:18:58.074341+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.092498+0000","last_change":"2024-03-28T13:18:59.086506+0000","last_active":"2024-03-28T13:18:59.092498+0000","last_peered":"2024-03-28T13:18:59.092498+0000","last_clean":"2024-03-28T13:18:59.092498+0000","last_became_active":"2024-03-28T13:18:59.086216+0000","last_became_peered":"2024-03-28T13:18:59.086216+0000","last_unstale":"2024-03-28T13:18:59.092498+0000","last_undegraded":"2024-03-28T13:18:59.092498+0000","last_fullsized":"2024-03-28T13:18:59.092498+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:15.681052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.082999+0000","last_change":"2024-03-28T13:18:59.075805+0000","last_active":"2024-03-28T13:18:59.082999+0000","last_peered":"2024-03-28T13:18:59.082999+0000","last_clean":"2024-03-28T13:18:59.082999+0000","last_became_active":"2024-03-28T13:18:59.074979+0000","last_became_peered":"2024-03-28T13:18:59.074979+0000","last_unstale":"2024-03-28T13:18:59.082999+0000","last_undegraded":"2024-03-28T13:18:59.082999+0000","last_fullsized":"2024-03-28T13:18:59.082999+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:29.522394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094055+0000","last_change":"2024-03-28T13:18:59.091367+0000","last_active":"2024-03-28T13:18:59.094055+0000","last_peered":"2024-03-28T13:18:59.094055+0000","last_clean":"2024-03-28T13:18:59.094055+0000","last_became_active":"2024-03-28T13:18:59.091083+0000","last_became_peered":"2024-03-28T13:18:59.091083+0000","last_unstale":"2024-03-28T13:18:59.094055+0000","last_undegraded":"2024-03-28T13:18:59.094055+0000","last_fullsized":"2024-03-28T13:18:59.094055+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:22:45.214275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-03-28T13:18:58.092690+0000","last_change":"2024-03-28T13:18:58.087418+0000","last_active":"2024-03-28T13:18:58.092690+0000","last_peered":"2024-03-28T13:18:58.092690+0000","last_clean":"2024-03-28T13:18:58.092690+0000","last_became_active":"2024-03-28T13:18:58.086932+0000","last_became_peered":"2024-03-28T13:18:58.086932+0000","last_unstale":"2024-03-28T13:18:58.092690+0000","last_undegraded":"2024-03-28T13:18:58.092690+0000","last_fullsized":"2024-03-28T13:18:58.092690+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:25:34.643120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-03-28T13:18:58.070236+0000","last_change":"2024-03-28T13:18:58.070236+0000","last_active":"2024-03-28T13:18:58.028202+0000","last_peered":"2024-03-28T13:18:58.028202+0000","last_clean":"2024-03-28T13:18:58.028202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:18:58.070236+0000","last_undegraded":"2024-03-28T13:18:58.070236+0000","last_fullsized":"2024-03-28T13:18:58.070236+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092249+0000","last_change":"2024-03-28T13:18:58.077769+0000","last_active":"2024-03-28T13:19:00.092249+0000","last_peered":"2024-03-28T13:19:00.092249+0000","last_clean":"2024-03-28T13:19:00.092249+0000","last_became_active":"2024-03-28T13:18:58.077234+0000","last_became_peered":"2024-03-28T13:18:58.077234+0000","last_unstale":"2024-03-28T13:19:00.092249+0000","last_undegraded":"2024-03-28T13:19:00.092249+0000","last_fullsized":"2024-03-28T13:19:00.092249+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:10:27.168742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.088777+0000","last_change":"2024-03-28T13:18:59.084112+0000","last_active":"2024-03-28T13:18:59.088777+0000","last_peered":"2024-03-28T13:18:59.088777+0000","last_clean":"2024-03-28T13:18:59.088777+0000","last_became_active":"2024-03-28T13:18:59.083805+0000","last_became_peered":"2024-03-28T13:18:59.083805+0000","last_unstale":"2024-03-28T13:18:59.088777+0000","last_undegraded":"2024-03-28T13:18:59.088777+0000","last_fullsized":"2024-03-28T13:18:59.088777+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:18:27.237579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.103124+0000","last_change":"2024-03-28T13:18:58.087832+0000","last_active":"2024-03-28T13:18:59.103124+0000","last_peered":"2024-03-28T13:18:59.103124+0000","last_clean":"2024-03-28T13:18:59.103124+0000","last_became_active":"2024-03-28T13:18:58.087250+0000","last_became_peered":"2024-03-28T13:18:58.087250+0000","last_unstale":"2024-03-28T13:18:59.103124+0000","last_undegraded":"2024-03-28T13:18:59.103124+0000","last_fullsized":"2024-03-28T13:18:59.103124+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:55:22.065081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.099692+0000","last_change":"2024-03-28T13:18:59.097206+0000","last_active":"2024-03-28T13:18:59.099692+0000","last_peered":"2024-03-28T13:18:59.099692+0000","last_clean":"2024-03-28T13:18:59.099692+0000","last_became_active":"2024-03-28T13:18:59.096903+0000","last_became_peered":"2024-03-28T13:18:59.096903+0000","last_unstale":"2024-03-28T13:18:59.099692+0000","last_undegraded":"2024-03-28T13:18:59.099692+0000","last_fullsized":"2024-03-28T13:18:59.099692+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:09.396769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.107170+0000","last_change":"2024-03-28T13:18:58.083531+0000","last_active":"2024-03-28T13:18:59.107170+0000","last_peered":"2024-03-28T13:18:59.107170+0000","last_clean":"2024-03-28T13:18:59.107170+0000","last_became_active":"2024-03-28T13:18:58.082970+0000","last_became_peered":"2024-03-28T13:18:58.082970+0000","last_unstale":"2024-03-28T13:18:59.107170+0000","last_undegraded":"2024-03-28T13:18:59.107170+0000","last_fullsized":"2024-03-28T13:18:59.107170+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:08.383486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.100576+0000","last_change":"2024-03-28T13:18:59.098346+0000","last_active":"2024-03-28T13:18:59.100576+0000","last_peered":"2024-03-28T13:18:59.100576+0000","last_clean":"2024-03-28T13:18:59.100576+0000","last_became_active":"2024-03-28T13:18:59.098052+0000","last_became_peered":"2024-03-28T13:18:59.098052+0000","last_unstale":"2024-03-28T13:18:59.100576+0000","last_undegraded":"2024-03-28T13:18:59.100576+0000","last_fullsized":"2024-03-28T13:18:59.100576+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:20:44.200653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.107233+0000","last_change":"2024-03-28T13:18:58.064634+0000","last_active":"2024-03-28T13:18:59.107233+0000","last_peered":"2024-03-28T13:18:59.107233+0000","last_clean":"2024-03-28T13:18:59.107233+0000","last_became_active":"2024-03-28T13:18:58.064194+0000","last_became_peered":"2024-03-28T13:18:58.064194+0000","last_unstale":"2024-03-28T13:18:59.107233+0000","last_undegraded":"2024-03-28T13:18:59.107233+0000","last_fullsized":"2024-03-28T13:18:59.107233+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:21:46.733401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091377+0000","last_change":"2024-03-28T13:18:59.094754+0000","last_active":"2024-03-28T13:19:00.091377+0000","last_peered":"2024-03-28T13:19:00.091377+0000","last_clean":"2024-03-28T13:19:00.091377+0000","last_became_active":"2024-03-28T13:18:59.090317+0000","last_became_peered":"2024-03-28T13:18:59.090317+0000","last_unstale":"2024-03-28T13:19:00.091377+0000","last_undegraded":"2024-03-28T13:19:00.091377+0000","last_fullsized":"2024-03-28T13:19:00.091377+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:09:17.961351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.101692+0000","last_change":"2024-03-28T13:18:58.082525+0000","last_active":"2024-03-28T13:18:59.101692+0000","last_peered":"2024-03-28T13:18:59.101692+0000","last_clean":"2024-03-28T13:18:59.101692+0000","last_became_active":"2024-03-28T13:18:58.082039+0000","last_became_peered":"2024-03-28T13:18:58.082039+0000","last_unstale":"2024-03-28T13:18:59.101692+0000","last_undegraded":"2024-03-28T13:18:59.101692+0000","last_fullsized":"2024-03-28T13:18:59.101692+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:42.609780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":103,"state":"creating+peering","last_fresh":"2024-03-28T13:18:58.072372+0000","last_change":"2024-03-28T13:18:58.072372+0000","last_active":"2024-03-28T13:18:58.028202+0000","last_peered":"2024-03-28T13:18:58.028202+0000","last_clean":"2024-03-28T13:18:58.028202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:18:58.072372+0000","last_undegraded":"2024-03-28T13:18:58.072372+0000","last_fullsized":"2024-03-28T13:18:58.072372+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090824+0000","last_change":"2024-03-28T13:18:58.078267+0000","last_active":"2024-03-28T13:19:00.090824+0000","last_peered":"2024-03-28T13:19:00.090824+0000","last_clean":"2024-03-28T13:19:00.090824+0000","last_became_active":"2024-03-28T13:18:58.077891+0000","last_became_peered":"2024-03-28T13:18:58.077891+0000","last_unstale":"2024-03-28T13:19:00.090824+0000","last_undegraded":"2024-03-28T13:19:00.090824+0000","last_fullsized":"2024-03-28T13:19:00.090824+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:35:04.854788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.102069+0000","last_change":"2024-03-28T13:18:59.099035+0000","last_active":"2024-03-28T13:18:59.102069+0000","last_peered":"2024-03-28T13:18:59.102069+0000","last_clean":"2024-03-28T13:18:59.102069+0000","last_became_active":"2024-03-28T13:18:59.098118+0000","last_became_peered":"2024-03-28T13:18:59.098118+0000","last_unstale":"2024-03-28T13:18:59.102069+0000","last_undegraded":"2024-03-28T13:18:59.102069+0000","last_fullsized":"2024-03-28T13:18:59.102069+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:21:47.746684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.059194+0000","last_change":"2024-03-28T13:18:58.076072+0000","last_active":"2024-03-28T13:18:59.059194+0000","last_peered":"2024-03-28T13:18:59.059194+0000","last_clean":"2024-03-28T13:18:59.059194+0000","last_became_active":"2024-03-28T13:18:58.075808+0000","last_became_peered":"2024-03-28T13:18:58.075808+0000","last_unstale":"2024-03-28T13:18:59.059194+0000","last_undegraded":"2024-03-28T13:18:59.059194+0000","last_fullsized":"2024-03-28T13:18:59.059194+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:11:22.068054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.083232+0000","last_change":"2024-03-28T13:18:59.075982+0000","last_active":"2024-03-28T13:18:59.083232+0000","last_peered":"2024-03-28T13:18:59.083232+0000","last_clean":"2024-03-28T13:18:59.083232+0000","last_became_active":"2024-03-28T13:18:59.075606+0000","last_became_peered":"2024-03-28T13:18:59.075606+0000","last_unstale":"2024-03-28T13:18:59.083232+0000","last_undegraded":"2024-03-28T13:18:59.083232+0000","last_fullsized":"2024-03-28T13:18:59.083232+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:26:09.194396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.078062+0000","last_change":"2024-03-28T13:18:58.077577+0000","last_active":"2024-03-28T13:18:59.078062+0000","last_peered":"2024-03-28T13:18:59.078062+0000","last_clean":"2024-03-28T13:18:59.078062+0000","last_became_active":"2024-03-28T13:18:58.077275+0000","last_became_peered":"2024-03-28T13:18:58.077275+0000","last_unstale":"2024-03-28T13:18:59.078062+0000","last_undegraded":"2024-03-28T13:18:59.078062+0000","last_fullsized":"2024-03-28T13:18:59.078062+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:28:48.630263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.089723+0000","last_change":"2024-03-28T13:18:59.086040+0000","last_active":"2024-03-28T13:18:59.089723+0000","last_peered":"2024-03-28T13:18:59.089723+0000","last_clean":"2024-03-28T13:18:59.089723+0000","last_became_active":"2024-03-28T13:18:59.085749+0000","last_became_peered":"2024-03-28T13:18:59.085749+0000","last_unstale":"2024-03-28T13:18:59.089723+0000","last_undegraded":"2024-03-28T13:18:59.089723+0000","last_fullsized":"2024-03-28T13:18:59.089723+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:27:52.931365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.061457+0000","last_change":"2024-03-28T13:18:58.083463+0000","last_active":"2024-03-28T13:18:59.061457+0000","last_peered":"2024-03-28T13:18:59.061457+0000","last_clean":"2024-03-28T13:18:59.061457+0000","last_became_active":"2024-03-28T13:18:58.083152+0000","last_became_peered":"2024-03-28T13:18:58.083152+0000","last_unstale":"2024-03-28T13:18:59.061457+0000","last_undegraded":"2024-03-28T13:18:59.061457+0000","last_fullsized":"2024-03-28T13:18:59.061457+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:48:03.913262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091380+0000","last_change":"2024-03-28T13:18:59.081727+0000","last_active":"2024-03-28T13:19:00.091380+0000","last_peered":"2024-03-28T13:19:00.091380+0000","last_clean":"2024-03-28T13:19:00.091380+0000","last_became_active":"2024-03-28T13:18:59.081436+0000","last_became_peered":"2024-03-28T13:18:59.081436+0000","last_unstale":"2024-03-28T13:19:00.091380+0000","last_undegraded":"2024-03-28T13:19:00.091380+0000","last_fullsized":"2024-03-28T13:19:00.091380+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:55:41.223520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"104'3","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092029+0000","last_change":"2024-03-28T13:18:58.068721+0000","last_active":"2024-03-28T13:19:00.092029+0000","last_peered":"2024-03-28T13:19:00.092029+0000","last_clean":"2024-03-28T13:19:00.092029+0000","last_became_active":"2024-03-28T13:18:58.068416+0000","last_became_peered":"2024-03-28T13:18:58.068416+0000","last_unstale":"2024-03-28T13:19:00.092029+0000","last_undegraded":"2024-03-28T13:19:00.092029+0000","last_fullsized":"2024-03-28T13:19:00.092029+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:16:13.945885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.086220+0000","last_change":"2024-03-28T13:18:59.080942+0000","last_active":"2024-03-28T13:18:59.086220+0000","last_peered":"2024-03-28T13:18:59.086220+0000","last_clean":"2024-03-28T13:18:59.086220+0000","last_became_active":"2024-03-28T13:18:59.080654+0000","last_became_peered":"2024-03-28T13:18:59.080654+0000","last_unstale":"2024-03-28T13:18:59.086220+0000","last_undegraded":"2024-03-28T13:18:59.086220+0000","last_fullsized":"2024-03-28T13:18:59.086220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:17:42.586454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"104'6","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.110193+0000","last_change":"2024-03-28T13:18:58.076129+0000","last_active":"2024-03-28T13:18:59.110193+0000","last_peered":"2024-03-28T13:18:59.110193+0000","last_clean":"2024-03-28T13:18:59.110193+0000","last_became_active":"2024-03-28T13:18:58.075809+0000","last_became_peered":"2024-03-28T13:18:58.075809+0000","last_unstale":"2024-03-28T13:18:59.110193+0000","last_undegraded":"2024-03-28T13:18:59.110193+0000","last_fullsized":"2024-03-28T13:18:59.110193+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:49:19.123418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087223+0000","last_change":"2024-03-28T13:18:59.077667+0000","last_active":"2024-03-28T13:18:59.087223+0000","last_peered":"2024-03-28T13:18:59.087223+0000","last_clean":"2024-03-28T13:18:59.087223+0000","last_became_active":"2024-03-28T13:18:59.077304+0000","last_became_peered":"2024-03-28T13:18:59.077304+0000","last_unstale":"2024-03-28T13:18:59.087223+0000","last_undegraded":"2024-03-28T13:18:59.087223+0000","last_fullsized":"2024-03-28T13:18:59.087223+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:43:24.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.105401+0000","last_change":"2024-03-28T13:18:58.077229+0000","last_active":"2024-03-28T13:18:59.105401+0000","last_peered":"2024-03-28T13:18:59.105401+0000","last_clean":"2024-03-28T13:18:59.105401+0000","last_became_active":"2024-03-28T13:18:58.076961+0000","last_became_peered":"2024-03-28T13:18:58.076961+0000","last_unstale":"2024-03-28T13:18:59.105401+0000","last_undegraded":"2024-03-28T13:18:59.105401+0000","last_fullsized":"2024-03-28T13:18:59.105401+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:20:48.852402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.092457+0000","last_change":"2024-03-28T13:18:59.084784+0000","last_active":"2024-03-28T13:18:59.092457+0000","last_peered":"2024-03-28T13:18:59.092457+0000","last_clean":"2024-03-28T13:18:59.092457+0000","last_became_active":"2024-03-28T13:18:59.084518+0000","last_became_peered":"2024-03-28T13:18:59.084518+0000","last_unstale":"2024-03-28T13:18:59.092457+0000","last_undegraded":"2024-03-28T13:18:59.092457+0000","last_fullsized":"2024-03-28T13:18:59.092457+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:49:20.136701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.059356+0000","last_change":"2024-03-28T13:18:58.083463+0000","last_active":"2024-03-28T13:18:59.059356+0000","last_peered":"2024-03-28T13:18:59.059356+0000","last_clean":"2024-03-28T13:18:59.059356+0000","last_became_active":"2024-03-28T13:18:58.083122+0000","last_became_peered":"2024-03-28T13:18:58.083122+0000","last_unstale":"2024-03-28T13:18:59.059356+0000","last_undegraded":"2024-03-28T13:18:59.059356+0000","last_fullsized":"2024-03-28T13:18:59.059356+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:20:43.187370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090923+0000","last_change":"2024-03-28T13:18:59.082251+0000","last_active":"2024-03-28T13:19:00.090923+0000","last_peered":"2024-03-28T13:19:00.090923+0000","last_clean":"2024-03-28T13:19:00.090923+0000","last_became_active":"2024-03-28T13:18:59.081951+0000","last_became_peered":"2024-03-28T13:18:59.081951+0000","last_unstale":"2024-03-28T13:19:00.090923+0000","last_undegraded":"2024-03-28T13:19:00.090923+0000","last_fullsized":"2024-03-28T13:19:00.090923+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:53.418554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087871+0000","last_change":"2024-03-28T13:18:58.066532+0000","last_active":"2024-03-28T13:18:59.087871+0000","last_peered":"2024-03-28T13:18:59.087871+0000","last_clean":"2024-03-28T13:18:59.087871+0000","last_became_active":"2024-03-28T13:18:58.066208+0000","last_became_peered":"2024-03-28T13:18:58.066208+0000","last_unstale":"2024-03-28T13:18:59.087871+0000","last_undegraded":"2024-03-28T13:18:59.087871+0000","last_fullsized":"2024-03-28T13:18:59.087871+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:58.349889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.093693+0000","last_change":"2024-03-28T13:18:59.090442+0000","last_active":"2024-03-28T13:18:59.093693+0000","last_peered":"2024-03-28T13:18:59.093693+0000","last_clean":"2024-03-28T13:18:59.093693+0000","last_became_active":"2024-03-28T13:18:59.090071+0000","last_became_peered":"2024-03-28T13:18:59.090071+0000","last_unstale":"2024-03-28T13:18:59.093693+0000","last_undegraded":"2024-03-28T13:18:59.093693+0000","last_fullsized":"2024-03-28T13:18:59.093693+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:17:18.839037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"104'8","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.108859+0000","last_change":"2024-03-28T13:18:58.071341+0000","last_active":"2024-03-28T13:18:59.108859+0000","last_peered":"2024-03-28T13:18:59.108859+0000","last_clean":"2024-03-28T13:18:59.108859+0000","last_became_active":"2024-03-28T13:18:58.069574+0000","last_became_peered":"2024-03-28T13:18:58.069574+0000","last_unstale":"2024-03-28T13:18:59.108859+0000","last_undegraded":"2024-03-28T13:18:59.108859+0000","last_fullsized":"2024-03-28T13:18:59.108859+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:13.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094648+0000","last_change":"2024-03-28T13:18:59.093731+0000","last_active":"2024-03-28T13:18:59.094648+0000","last_peered":"2024-03-28T13:18:59.094648+0000","last_clean":"2024-03-28T13:18:59.094648+0000","last_became_active":"2024-03-28T13:18:59.093471+0000","last_became_peered":"2024-03-28T13:18:59.093471+0000","last_unstale":"2024-03-28T13:18:59.094648+0000","last_undegraded":"2024-03-28T13:18:59.094648+0000","last_fullsized":"2024-03-28T13:18:59.094648+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:18:27.237579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.108148+0000","last_change":"2024-03-28T13:18:58.080576+0000","last_active":"2024-03-28T13:18:59.108148+0000","last_peered":"2024-03-28T13:18:59.108148+0000","last_clean":"2024-03-28T13:18:59.108148+0000","last_became_active":"2024-03-28T13:18:58.080272+0000","last_became_peered":"2024-03-28T13:18:58.080272+0000","last_unstale":"2024-03-28T13:18:59.108148+0000","last_undegraded":"2024-03-28T13:18:59.108148+0000","last_fullsized":"2024-03-28T13:18:59.108148+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:13.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094243+0000","last_change":"2024-03-28T13:18:59.092760+0000","last_active":"2024-03-28T13:18:59.094243+0000","last_peered":"2024-03-28T13:18:59.094243+0000","last_clean":"2024-03-28T13:18:59.094243+0000","last_became_active":"2024-03-28T13:18:59.092409+0000","last_became_peered":"2024-03-28T13:18:59.092409+0000","last_unstale":"2024-03-28T13:18:59.094243+0000","last_undegraded":"2024-03-28T13:18:59.094243+0000","last_fullsized":"2024-03-28T13:18:59.094243+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:19.489710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092132+0000","last_change":"2024-03-28T13:18:58.065630+0000","last_active":"2024-03-28T13:19:00.092132+0000","last_peered":"2024-03-28T13:19:00.092132+0000","last_clean":"2024-03-28T13:19:00.092132+0000","last_became_active":"2024-03-28T13:18:58.065333+0000","last_became_peered":"2024-03-28T13:18:58.065333+0000","last_unstale":"2024-03-28T13:19:00.092132+0000","last_undegraded":"2024-03-28T13:19:00.092132+0000","last_fullsized":"2024-03-28T13:19:00.092132+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:54:14.505307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.099342+0000","last_change":"2024-03-28T13:18:59.092928+0000","last_active":"2024-03-28T13:18:59.099342+0000","last_peered":"2024-03-28T13:18:59.099342+0000","last_clean":"2024-03-28T13:18:59.099342+0000","last_became_active":"2024-03-28T13:18:59.092579+0000","last_became_peered":"2024-03-28T13:18:59.092579+0000","last_unstale":"2024-03-28T13:18:59.099342+0000","last_undegraded":"2024-03-28T13:18:59.099342+0000","last_fullsized":"2024-03-28T13:18:59.099342+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:37:45.650491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.064300+0000","last_change":"2024-03-28T13:18:58.075199+0000","last_active":"2024-03-28T13:18:59.064300+0000","last_peered":"2024-03-28T13:18:59.064300+0000","last_clean":"2024-03-28T13:18:59.064300+0000","last_became_active":"2024-03-28T13:18:58.073665+0000","last_became_peered":"2024-03-28T13:18:58.073665+0000","last_unstale":"2024-03-28T13:18:59.064300+0000","last_undegraded":"2024-03-28T13:18:59.064300+0000","last_fullsized":"2024-03-28T13:18:59.064300+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:08:41.324830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.081523+0000","last_change":"2024-03-28T13:18:59.076634+0000","last_active":"2024-03-28T13:18:59.081523+0000","last_peered":"2024-03-28T13:18:59.081523+0000","last_clean":"2024-03-28T13:18:59.081523+0000","last_became_active":"2024-03-28T13:18:59.076295+0000","last_became_peered":"2024-03-28T13:18:59.076295+0000","last_unstale":"2024-03-28T13:18:59.081523+0000","last_undegraded":"2024-03-28T13:18:59.081523+0000","last_fullsized":"2024-03-28T13:18:59.081523+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:40.743749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.107170+0000","last_change":"2024-03-28T13:18:58.084188+0000","last_active":"2024-03-28T13:18:59.107170+0000","last_peered":"2024-03-28T13:18:59.107170+0000","last_clean":"2024-03-28T13:18:59.107170+0000","last_became_active":"2024-03-28T13:18:58.081720+0000","last_became_peered":"2024-03-28T13:18:58.081720+0000","last_unstale":"2024-03-28T13:18:59.107170+0000","last_undegraded":"2024-03-28T13:18:59.107170+0000","last_fullsized":"2024-03-28T13:18:59.107170+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:44.810836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.082259+0000","last_change":"2024-03-28T13:18:59.079466+0000","last_active":"2024-03-28T13:18:59.082259+0000","last_peered":"2024-03-28T13:18:59.082259+0000","last_clean":"2024-03-28T13:18:59.082259+0000","last_became_active":"2024-03-28T13:18:59.078752+0000","last_became_peered":"2024-03-28T13:18:59.078752+0000","last_unstale":"2024-03-28T13:18:59.082259+0000","last_undegraded":"2024-03-28T13:18:59.082259+0000","last_fullsized":"2024-03-28T13:18:59.082259+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:12:28.089180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.066401+0000","last_change":"2024-03-28T13:18:58.079436+0000","last_active":"2024-03-28T13:18:59.066401+0000","last_peered":"2024-03-28T13:18:59.066401+0000","last_clean":"2024-03-28T13:18:59.066401+0000","last_became_active":"2024-03-28T13:18:58.079066+0000","last_became_peered":"2024-03-28T13:18:58.079066+0000","last_unstale":"2024-03-28T13:18:59.066401+0000","last_undegraded":"2024-03-28T13:18:59.066401+0000","last_fullsized":"2024-03-28T13:18:59.066401+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:08.383486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087869+0000","last_change":"2024-03-28T13:18:59.080751+0000","last_active":"2024-03-28T13:18:59.087869+0000","last_peered":"2024-03-28T13:18:59.087869+0000","last_clean":"2024-03-28T13:18:59.087869+0000","last_became_active":"2024-03-28T13:18:59.080449+0000","last_became_peered":"2024-03-28T13:18:59.080449+0000","last_unstale":"2024-03-28T13:18:59.087869+0000","last_undegraded":"2024-03-28T13:18:59.087869+0000","last_fullsized":"2024-03-28T13:18:59.087869+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:17:42.586454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.075738+0000","last_change":"2024-03-28T13:18:58.079341+0000","last_active":"2024-03-28T13:18:59.075738+0000","last_peered":"2024-03-28T13:18:59.075738+0000","last_clean":"2024-03-28T13:18:59.075738+0000","last_became_active":"2024-03-28T13:18:58.079008+0000","last_became_peered":"2024-03-28T13:18:58.079008+0000","last_unstale":"2024-03-28T13:18:59.075738+0000","last_undegraded":"2024-03-28T13:18:59.075738+0000","last_fullsized":"2024-03-28T13:18:59.075738+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:21:46.733401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.086441+0000","last_change":"2024-03-28T13:18:59.080827+0000","last_active":"2024-03-28T13:18:59.086441+0000","last_peered":"2024-03-28T13:18:59.086441+0000","last_clean":"2024-03-28T13:18:59.086441+0000","last_became_active":"2024-03-28T13:18:59.080524+0000","last_became_peered":"2024-03-28T13:18:59.080524+0000","last_unstale":"2024-03-28T13:18:59.086441+0000","last_undegraded":"2024-03-28T13:18:59.086441+0000","last_fullsized":"2024-03-28T13:18:59.086441+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:43:24.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.072151+0000","last_change":"2024-03-28T13:18:58.090817+0000","last_active":"2024-03-28T13:18:59.072151+0000","last_peered":"2024-03-28T13:18:59.072151+0000","last_clean":"2024-03-28T13:18:59.072151+0000","last_became_active":"2024-03-28T13:18:58.090469+0000","last_became_peered":"2024-03-28T13:18:58.090469+0000","last_unstale":"2024-03-28T13:18:59.072151+0000","last_undegraded":"2024-03-28T13:18:59.072151+0000","last_fullsized":"2024-03-28T13:18:59.072151+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:52.858677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.080088+0000","last_change":"2024-03-28T13:18:59.071403+0000","last_active":"2024-03-28T13:18:59.080088+0000","last_peered":"2024-03-28T13:18:59.080088+0000","last_clean":"2024-03-28T13:18:59.080088+0000","last_became_active":"2024-03-28T13:18:59.071095+0000","last_became_peered":"2024-03-28T13:18:59.071095+0000","last_unstale":"2024-03-28T13:18:59.080088+0000","last_undegraded":"2024-03-28T13:18:59.080088+0000","last_fullsized":"2024-03-28T13:18:59.080088+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:18:33.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.105978+0000","last_change":"2024-03-28T13:18:58.071038+0000","last_active":"2024-03-28T13:18:59.105978+0000","last_peered":"2024-03-28T13:18:59.105978+0000","last_clean":"2024-03-28T13:18:59.105978+0000","last_became_active":"2024-03-28T13:18:58.069807+0000","last_became_peered":"2024-03-28T13:18:58.069807+0000","last_unstale":"2024-03-28T13:18:59.105978+0000","last_undegraded":"2024-03-28T13:18:59.105978+0000","last_fullsized":"2024-03-28T13:18:59.105978+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:17:17.825753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.080889+0000","last_change":"2024-03-28T13:18:59.076727+0000","last_active":"2024-03-28T13:18:59.080889+0000","last_peered":"2024-03-28T13:18:59.080889+0000","last_clean":"2024-03-28T13:18:59.080889+0000","last_became_active":"2024-03-28T13:18:59.076415+0000","last_became_peered":"2024-03-28T13:18:59.076415+0000","last_unstale":"2024-03-28T13:18:59.080889+0000","last_undegraded":"2024-03-28T13:18:59.080889+0000","last_fullsized":"2024-03-28T13:18:59.080889+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:14:07.278326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.066449+0000","last_change":"2024-03-28T13:18:58.086609+0000","last_active":"2024-03-28T13:18:59.066449+0000","last_peered":"2024-03-28T13:18:59.066449+0000","last_clean":"2024-03-28T13:18:59.066449+0000","last_became_active":"2024-03-28T13:18:58.086223+0000","last_became_peered":"2024-03-28T13:18:58.086223+0000","last_unstale":"2024-03-28T13:18:59.066449+0000","last_undegraded":"2024-03-28T13:18:59.066449+0000","last_fullsized":"2024-03-28T13:18:59.066449+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:17.621966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.099448+0000","last_change":"2024-03-28T13:18:59.098557+0000","last_active":"2024-03-28T13:18:59.099448+0000","last_peered":"2024-03-28T13:18:59.099448+0000","last_clean":"2024-03-28T13:18:59.099448+0000","last_became_active":"2024-03-28T13:18:59.098274+0000","last_became_peered":"2024-03-28T13:18:59.098274+0000","last_unstale":"2024-03-28T13:18:59.099448+0000","last_undegraded":"2024-03-28T13:18:59.099448+0000","last_fullsized":"2024-03-28T13:18:59.099448+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:14.876459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.065223+0000","last_change":"2024-03-28T13:18:58.072039+0000","last_active":"2024-03-28T13:18:59.065223+0000","last_peered":"2024-03-28T13:18:59.065223+0000","last_clean":"2024-03-28T13:18:59.065223+0000","last_became_active":"2024-03-28T13:18:58.071683+0000","last_became_peered":"2024-03-28T13:18:58.071683+0000","last_unstale":"2024-03-28T13:18:59.065223+0000","last_undegraded":"2024-03-28T13:18:59.065223+0000","last_fullsized":"2024-03-28T13:18:59.065223+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:47.118216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091918+0000","last_change":"2024-03-28T13:18:59.087471+0000","last_active":"2024-03-28T13:19:00.091918+0000","last_peered":"2024-03-28T13:19:00.091918+0000","last_clean":"2024-03-28T13:19:00.091918+0000","last_became_active":"2024-03-28T13:18:59.085466+0000","last_became_peered":"2024-03-28T13:18:59.085466+0000","last_unstale":"2024-03-28T13:19:00.091918+0000","last_undegraded":"2024-03-28T13:19:00.091918+0000","last_fullsized":"2024-03-28T13:19:00.091918+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:30:44.259677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.067207+0000","last_change":"2024-03-28T13:18:58.068025+0000","last_active":"2024-03-28T13:18:59.067207+0000","last_peered":"2024-03-28T13:18:59.067207+0000","last_clean":"2024-03-28T13:18:59.067207+0000","last_became_active":"2024-03-28T13:18:58.067671+0000","last_became_peered":"2024-03-28T13:18:58.067671+0000","last_unstale":"2024-03-28T13:18:59.067207+0000","last_undegraded":"2024-03-28T13:18:59.067207+0000","last_fullsized":"2024-03-28T13:18:59.067207+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:28.509110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.079227+0000","last_change":"2024-03-28T13:18:59.073922+0000","last_active":"2024-03-28T13:18:59.079227+0000","last_peered":"2024-03-28T13:18:59.079227+0000","last_clean":"2024-03-28T13:18:59.079227+0000","last_became_active":"2024-03-28T13:18:59.073566+0000","last_became_peered":"2024-03-28T13:18:59.073566+0000","last_unstale":"2024-03-28T13:18:59.079227+0000","last_undegraded":"2024-03-28T13:18:59.079227+0000","last_fullsized":"2024-03-28T13:18:59.079227+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:44:49.466448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.072738+0000","last_change":"2024-03-28T13:18:58.085176+0000","last_active":"2024-03-28T13:18:59.072738+0000","last_peered":"2024-03-28T13:18:59.072738+0000","last_clean":"2024-03-28T13:18:59.072738+0000","last_became_active":"2024-03-28T13:18:58.084898+0000","last_became_peered":"2024-03-28T13:18:58.084898+0000","last_unstale":"2024-03-28T13:18:59.072738+0000","last_undegraded":"2024-03-28T13:18:59.072738+0000","last_fullsized":"2024-03-28T13:18:59.072738+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:13:25.784483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091562+0000","last_change":"2024-03-28T13:18:59.076130+0000","last_active":"2024-03-28T13:19:00.091562+0000","last_peered":"2024-03-28T13:19:00.091562+0000","last_clean":"2024-03-28T13:19:00.091562+0000","last_became_active":"2024-03-28T13:18:59.075785+0000","last_became_peered":"2024-03-28T13:18:59.075785+0000","last_unstale":"2024-03-28T13:19:00.091562+0000","last_undegraded":"2024-03-28T13:19:00.091562+0000","last_fullsized":"2024-03-28T13:19:00.091562+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:31.836556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.065847+0000","last_change":"2024-03-28T13:18:58.073984+0000","last_active":"2024-03-28T13:18:59.065847+0000","last_peered":"2024-03-28T13:18:59.065847+0000","last_clean":"2024-03-28T13:18:59.065847+0000","last_became_active":"2024-03-28T13:18:58.073520+0000","last_became_peered":"2024-03-28T13:18:58.073520+0000","last_unstale":"2024-03-28T13:18:59.065847+0000","last_undegraded":"2024-03-28T13:18:59.065847+0000","last_fullsized":"2024-03-28T13:18:59.065847+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:28.886056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.082579+0000","last_change":"2024-03-28T13:18:59.078032+0000","last_active":"2024-03-28T13:18:59.082579+0000","last_peered":"2024-03-28T13:18:59.082579+0000","last_clean":"2024-03-28T13:18:59.082579+0000","last_became_active":"2024-03-28T13:18:59.077665+0000","last_became_peered":"2024-03-28T13:18:59.077665+0000","last_unstale":"2024-03-28T13:18:59.082579+0000","last_undegraded":"2024-03-28T13:18:59.082579+0000","last_fullsized":"2024-03-28T13:18:59.082579+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:04:28.366038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-03-28T13:18:58.085201+0000","last_change":"2024-03-28T13:18:58.083533+0000","last_active":"2024-03-28T13:18:58.085201+0000","last_peered":"2024-03-28T13:18:58.085201+0000","last_clean":"2024-03-28T13:18:58.085201+0000","last_became_active":"2024-03-28T13:18:58.083220+0000","last_became_peered":"2024-03-28T13:18:58.083220+0000","last_unstale":"2024-03-28T13:18:58.085201+0000","last_undegraded":"2024-03-28T13:18:58.085201+0000","last_fullsized":"2024-03-28T13:18:58.085201+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:28:30.860196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.098913+0000","last_change":"2024-03-28T13:18:59.091706+0000","last_active":"2024-03-28T13:18:59.098913+0000","last_peered":"2024-03-28T13:18:59.098913+0000","last_clean":"2024-03-28T13:18:59.098913+0000","last_became_active":"2024-03-28T13:18:59.091410+0000","last_became_peered":"2024-03-28T13:18:59.091410+0000","last_unstale":"2024-03-28T13:18:59.098913+0000","last_undegraded":"2024-03-28T13:18:59.098913+0000","last_fullsized":"2024-03-28T13:18:59.098913+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:15.681052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.072573+0000","last_change":"2024-03-28T13:18:58.086792+0000","last_active":"2024-03-28T13:18:59.072573+0000","last_peered":"2024-03-28T13:18:59.072573+0000","last_clean":"2024-03-28T13:18:59.072573+0000","last_became_active":"2024-03-28T13:18:58.086544+0000","last_became_peered":"2024-03-28T13:18:58.086544+0000","last_unstale":"2024-03-28T13:18:59.072573+0000","last_undegraded":"2024-03-28T13:18:59.072573+0000","last_fullsized":"2024-03-28T13:18:59.072573+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:51:50.282781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.092724+0000","last_change":"2024-03-28T13:18:59.088616+0000","last_active":"2024-03-28T13:18:59.092724+0000","last_peered":"2024-03-28T13:18:59.092724+0000","last_clean":"2024-03-28T13:18:59.092724+0000","last_became_active":"2024-03-28T13:18:59.088292+0000","last_became_peered":"2024-03-28T13:18:59.088292+0000","last_unstale":"2024-03-28T13:18:59.092724+0000","last_undegraded":"2024-03-28T13:18:59.092724+0000","last_fullsized":"2024-03-28T13:18:59.092724+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:37:45.650491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.107904+0000","last_change":"2024-03-28T13:18:58.075299+0000","last_active":"2024-03-28T13:18:59.107904+0000","last_peered":"2024-03-28T13:18:59.107904+0000","last_clean":"2024-03-28T13:18:59.107904+0000","last_became_active":"2024-03-28T13:18:58.075035+0000","last_became_peered":"2024-03-28T13:18:58.075035+0000","last_unstale":"2024-03-28T13:18:59.107904+0000","last_undegraded":"2024-03-28T13:18:59.107904+0000","last_fullsized":"2024-03-28T13:18:59.107904+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:39.277367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":320,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087250+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:18:59.087250+0000","last_peered":"2024-03-28T13:18:59.087250+0000","last_clean":"2024-03-28T13:18:59.087250+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:18:59.087250+0000","last_undegraded":"2024-03-28T13:18:59.087250+0000","last_fullsized":"2024-03-28T13:18:59.087250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":34,"ondisk_log_size":34,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476856,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":550446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":550322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1888256,"data_stored":1142344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1273856,"data_stored":550252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1261568,"data_stored":543477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1855488,"data_stored":1133845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1261568,"data_stored":543477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1261568,"data_stored":543477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:19:04.180 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:19:04.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.433+0000 7f2138e00640 1 Processor -- start 2024-03-28T13:19:04.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.434+0000 7f2138e00640 1 -- start start 2024-03-28T13:19:04.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.435+0000 7f2138e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f213406b570 0x7f21340d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:04.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.435+0000 7f2138e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21340d0ca0 0x7f21340ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:04.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.435+0000 7f2138e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21340ce360 con 0x7f213415c650 2024-03-28T13:19:04.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.435+0000 7f2138e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f21340d02d0 con 0x7f21340d0ca0 2024-03-28T13:19:04.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.435+0000 7f2138e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f21340ce640 con 0x7f213406b570 2024-03-28T13:19:04.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.435+0000 7f2132800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21340d0ca0 0x7f21340ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:04.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.436+0000 7f2132800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21340d0ca0 0x7f21340ae020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59408/0 (socket says 172.21.15.19:59408) 2024-03-28T13:19:04.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.436+0000 7f2132800640 1 -- 172.21.15.19:0/201542032 learned_addr learned my addr 172.21.15.19:0/201542032 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:04.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.436+0000 7f2132800640 1 -- 172.21.15.19:0/201542032 >> v2:172.21.15.188:3300/0 conn(0x7f213406b570 msgr2=0x7f21340d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:04.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.436+0000 7f2131400640 1 --2- 172.21.15.19:0/201542032 >> v2:172.21.15.188:3300/0 conn(0x7f213406b570 0x7f21340d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:04.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.436+0000 7f2132800640 1 --2- 172.21.15.19:0/201542032 >> v2:172.21.15.188:3300/0 conn(0x7f213406b570 0x7f21340d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:04.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.436+0000 7f2132800640 1 -- 172.21.15.19:0/201542032 >> v1:172.21.15.188:6789/0 conn(0x7f213415c650 legacy=0x7f21340cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:04.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.436+0000 7f2132800640 1 -- 172.21.15.19:0/201542032 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21340ce920 con 0x7f21340d0ca0 2024-03-28T13:19:04.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.436+0000 7f2131400640 1 --2- 172.21.15.19:0/201542032 >> v2:172.21.15.188:3300/0 conn(0x7f213406b570 0x7f21340d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:04.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.437+0000 7f2132800640 1 --2- 172.21.15.19:0/201542032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21340d0ca0 0x7f21340ae020 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f21340c4c70 tx=0x7f212c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d5eaa59cea3404 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:04.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.438+0000 7f2130a00640 1 -- 172.21.15.19:0/201542032 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c034070 con 0x7f21340d0ca0 2024-03-28T13:19:04.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.438+0000 7f2130a00640 1 -- 172.21.15.19:0/201542032 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f212c030860 con 0x7f21340d0ca0 2024-03-28T13:19:04.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.438+0000 7f2130a00640 1 -- 172.21.15.19:0/201542032 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f212c030c30 con 0x7f21340d0ca0 2024-03-28T13:19:04.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.438+0000 7f2138e00640 1 -- 172.21.15.19:0/201542032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21340d0ca0 msgr2=0x7f21340ae020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:04.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.438+0000 7f2138e00640 1 --2- 172.21.15.19:0/201542032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21340d0ca0 0x7f21340ae020 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f21340c4c70 tx=0x7f212c02b9a0 comp rx=0 tx=0).stop 2024-03-28T13:19:04.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.438+0000 7f2138e00640 1 -- 172.21.15.19:0/201542032 shutdown_connections 2024-03-28T13:19:04.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.438+0000 7f2138e00640 1 --2- 172.21.15.19:0/201542032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21340d0ca0 0x7f21340ae020 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:04.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.438+0000 7f2138e00640 1 --2- 172.21.15.19:0/201542032 >> v2:172.21.15.188:3300/0 conn(0x7f213406b570 0x7f21340d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:04.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.438+0000 7f2138e00640 1 -- 172.21.15.19:0/201542032 >> 172.21.15.19:0/201542032 conn(0x7f21340b2d40 msgr2=0x7f21340b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:04.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.439+0000 7f2138e00640 1 -- 172.21.15.19:0/201542032 shutdown_connections 2024-03-28T13:19:04.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.439+0000 7f2138e00640 1 -- 172.21.15.19:0/201542032 wait complete. 2024-03-28T13:19:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.440+0000 7f2138e00640 1 Processor -- start 2024-03-28T13:19:04.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.441+0000 7f2138e00640 1 -- start start 2024-03-28T13:19:04.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.441+0000 7f2138e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f213406b570 0x7f21340c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:04.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.441+0000 7f2138e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21340d0ca0 0x7f21340cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:04.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.441+0000 7f2138e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f213415c650 0x7f2134154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:04.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.441+0000 7f2138e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21340ae6e0 con 0x7f213406b570 2024-03-28T13:19:04.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.441+0000 7f2138e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f21340ae560 con 0x7f213415c650 2024-03-28T13:19:04.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.441+0000 7f2138e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f21340ae860 con 0x7f21340d0ca0 2024-03-28T13:19:04.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.442+0000 7f2131e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f213406b570 0x7f21340c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:04.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.442+0000 7f2131e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f213406b570 0x7f21340c4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59418/0 (socket says 172.21.15.19:59418) 2024-03-28T13:19:04.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.442+0000 7f2131e00640 1 -- 172.21.15.19:0/482655105 learned_addr learned my addr 172.21.15.19:0/482655105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:04.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.442+0000 7f2131400640 1 --2- 172.21.15.19:0/482655105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21340d0ca0 0x7f21340cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:04.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.442+0000 7f2132800640 1 --2- 172.21.15.19:0/482655105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f213415c650 0x7f2134154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:04.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.442+0000 7f2131e00640 1 -- 172.21.15.19:0/482655105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21340d0ca0 msgr2=0x7f21340cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:04.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.442+0000 7f2131e00640 1 --2- 172.21.15.19:0/482655105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21340d0ca0 0x7f21340cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:04.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.442+0000 7f2131e00640 1 -- 172.21.15.19:0/482655105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f213415c650 msgr2=0x7f2134154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:04.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.442+0000 7f2131e00640 1 --2- 172.21.15.19:0/482655105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f213415c650 0x7f2134154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:04.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.442+0000 7f2131e00640 1 -- 172.21.15.19:0/482655105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21340c9b20 con 0x7f213406b570 2024-03-28T13:19:04.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.442+0000 7f2132800640 1 --2- 172.21.15.19:0/482655105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f213415c650 0x7f2134154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:04.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.443+0000 7f2131e00640 1 --2- 172.21.15.19:0/482655105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f213406b570 0x7f21340c4be0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f212400c460 tx=0x7f212400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:04.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.443+0000 7f212a000640 1 -- 172.21.15.19:0/482655105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124016020 con 0x7f213406b570 2024-03-28T13:19:04.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.443+0000 7f212a000640 1 -- 172.21.15.19:0/482655105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2124012870 con 0x7f213406b570 2024-03-28T13:19:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.443+0000 7f212a000640 1 -- 172.21.15.19:0/482655105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2124012c60 con 0x7f213406b570 2024-03-28T13:19:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.443+0000 7f2138e00640 1 -- 172.21.15.19:0/482655105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21340c9e00 con 0x7f213406b570 2024-03-28T13:19:04.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.443+0000 7f2138e00640 1 -- 172.21.15.19:0/482655105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21340ca3c0 con 0x7f213406b570 2024-03-28T13:19:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.444+0000 7f2138e00640 1 -- 172.21.15.19:0/482655105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21340cc870 con 0x7f213406b570 2024-03-28T13:19:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.445+0000 7f212a000640 1 -- 172.21.15.19:0/482655105 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2124019020 con 0x7f213406b570 2024-03-28T13:19:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.446+0000 7f212a000640 1 --2- 172.21.15.19:0/482655105 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f211006c560 0x7f211006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.446+0000 7f212a000640 1 -- 172.21.15.19:0/482655105 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f212408b460 con 0x7f213406b570 2024-03-28T13:19:04.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.447+0000 7f2131400640 1 --2- 172.21.15.19:0/482655105 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f211006c560 0x7f211006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:04.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.448+0000 7f2131400640 1 --2- 172.21.15.19:0/482655105 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f211006c560 0x7f211006ea20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f211c007510 tx=0x7f211c0074a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:04.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.451+0000 7f212a000640 1 -- 172.21.15.19:0/482655105 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f21340cc870 con 0x7f213406b570 2024-03-28T13:19:04.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.691+0000 7f2138e00640 1 -- 172.21.15.19:0/482655105 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21340cde30 con 0x7f211006c560 2024-03-28T13:19:04.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.709+0000 7f212a000640 1 -- 172.21.15.19:0/482655105 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252465 (secure 0 0 0) 0x7f21340cde30 con 0x7f211006c560 2024-03-28T13:19:04.712 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:04.713 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:19:04.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.715+0000 7f2138e00640 1 -- 172.21.15.19:0/482655105 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f211006c560 msgr2=0x7f211006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:04.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.715+0000 7f2138e00640 1 --2- 172.21.15.19:0/482655105 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f211006c560 0x7f211006ea20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f211c007510 tx=0x7f211c0074a0 comp rx=0 tx=0).stop 2024-03-28T13:19:04.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.715+0000 7f2138e00640 1 -- 172.21.15.19:0/482655105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f213406b570 msgr2=0x7f21340c4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:04.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.715+0000 7f2138e00640 1 --2- 172.21.15.19:0/482655105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f213406b570 0x7f21340c4be0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f212400c460 tx=0x7f212400c920 comp rx=0 tx=0).stop 2024-03-28T13:19:04.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.715+0000 7f2138e00640 1 -- 172.21.15.19:0/482655105 shutdown_connections 2024-03-28T13:19:04.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.715+0000 7f2138e00640 1 --2- 172.21.15.19:0/482655105 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f211006c560 0x7f211006ea20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.715+0000 7f2138e00640 1 --2- 172.21.15.19:0/482655105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f213415c650 0x7f2134154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.715+0000 7f2138e00640 1 --2- 172.21.15.19:0/482655105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21340d0ca0 0x7f21340cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.715+0000 7f2138e00640 1 --2- 172.21.15.19:0/482655105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f213406b570 0x7f21340c4be0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.715+0000 7f2138e00640 1 -- 172.21.15.19:0/482655105 >> 172.21.15.19:0/482655105 conn(0x7f21340b2d40 msgr2=0x7f21340b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.715+0000 7f2138e00640 1 -- 172.21.15.19:0/482655105 shutdown_connections 2024-03-28T13:19:04.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.715+0000 7f2138e00640 1 -- 172.21.15.19:0/482655105 wait complete. 2024-03-28T13:19:04.735 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-03-28T13:19:04.198067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260672,"kb_used_data":11224,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714048,"statfs":{"total":773094113280,"available":772827185152,"internally_reserved":0,"allocated":11493376,"data_stored":5612772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.149054"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091482+0000","last_change":"2024-03-28T13:18:59.085702+0000","last_active":"2024-03-28T13:19:00.091482+0000","last_peered":"2024-03-28T13:19:00.091482+0000","last_clean":"2024-03-28T13:19:00.091482+0000","last_became_active":"2024-03-28T13:18:59.085413+0000","last_became_peered":"2024-03-28T13:18:59.085413+0000","last_unstale":"2024-03-28T13:19:00.091482+0000","last_undegraded":"2024-03-28T13:19:00.091482+0000","last_fullsized":"2024-03-28T13:19:00.091482+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:04:28.366038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091242+0000","last_change":"2024-03-28T13:18:59.090611+0000","last_active":"2024-03-28T13:19:00.091242+0000","last_peered":"2024-03-28T13:19:00.091242+0000","last_clean":"2024-03-28T13:19:00.091242+0000","last_became_active":"2024-03-28T13:18:59.089972+0000","last_became_peered":"2024-03-28T13:18:59.089972+0000","last_unstale":"2024-03-28T13:19:00.091242+0000","last_undegraded":"2024-03-28T13:19:00.091242+0000","last_fullsized":"2024-03-28T13:19:00.091242+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:40.743749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101633+0000","last_change":"2024-03-28T13:18:59.098655+0000","last_active":"2024-03-28T13:19:00.101633+0000","last_peered":"2024-03-28T13:19:00.101633+0000","last_clean":"2024-03-28T13:19:00.101633+0000","last_became_active":"2024-03-28T13:18:59.098399+0000","last_became_peered":"2024-03-28T13:18:59.098399+0000","last_unstale":"2024-03-28T13:19:00.101633+0000","last_undegraded":"2024-03-28T13:19:00.101633+0000","last_fullsized":"2024-03-28T13:19:00.101633+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:17:42.586454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090818+0000","last_change":"2024-03-28T13:18:59.074664+0000","last_active":"2024-03-28T13:19:00.090818+0000","last_peered":"2024-03-28T13:19:00.090818+0000","last_clean":"2024-03-28T13:19:00.090818+0000","last_became_active":"2024-03-28T13:18:59.073993+0000","last_became_peered":"2024-03-28T13:18:59.073993+0000","last_unstale":"2024-03-28T13:19:00.090818+0000","last_undegraded":"2024-03-28T13:19:00.090818+0000","last_fullsized":"2024-03-28T13:19:00.090818+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:44:49.466448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091305+0000","last_change":"2024-03-28T13:18:59.090589+0000","last_active":"2024-03-28T13:19:00.091305+0000","last_peered":"2024-03-28T13:19:00.091305+0000","last_clean":"2024-03-28T13:19:00.091305+0000","last_became_active":"2024-03-28T13:18:59.089952+0000","last_became_peered":"2024-03-28T13:18:59.089952+0000","last_unstale":"2024-03-28T13:19:00.091305+0000","last_undegraded":"2024-03-28T13:19:00.091305+0000","last_fullsized":"2024-03-28T13:19:00.091305+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:09.506244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091182+0000","last_change":"2024-03-28T13:18:59.085728+0000","last_active":"2024-03-28T13:19:00.091182+0000","last_peered":"2024-03-28T13:19:00.091182+0000","last_clean":"2024-03-28T13:19:00.091182+0000","last_became_active":"2024-03-28T13:18:59.085447+0000","last_became_peered":"2024-03-28T13:18:59.085447+0000","last_unstale":"2024-03-28T13:19:00.091182+0000","last_undegraded":"2024-03-28T13:19:00.091182+0000","last_fullsized":"2024-03-28T13:19:00.091182+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:16.716159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091117+0000","last_change":"2024-03-28T13:18:59.090899+0000","last_active":"2024-03-28T13:19:00.091117+0000","last_peered":"2024-03-28T13:19:00.091117+0000","last_clean":"2024-03-28T13:19:00.091117+0000","last_became_active":"2024-03-28T13:18:59.086628+0000","last_became_peered":"2024-03-28T13:18:59.086628+0000","last_unstale":"2024-03-28T13:19:00.091117+0000","last_undegraded":"2024-03-28T13:19:00.091117+0000","last_fullsized":"2024-03-28T13:19:00.091117+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:05:31.597035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100303+0000","last_change":"2024-03-28T13:18:59.082961+0000","last_active":"2024-03-28T13:19:00.100303+0000","last_peered":"2024-03-28T13:19:00.100303+0000","last_clean":"2024-03-28T13:19:00.100303+0000","last_became_active":"2024-03-28T13:18:59.082608+0000","last_became_peered":"2024-03-28T13:18:59.082608+0000","last_unstale":"2024-03-28T13:19:00.100303+0000","last_undegraded":"2024-03-28T13:19:00.100303+0000","last_fullsized":"2024-03-28T13:19:00.100303+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:45.824120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092358+0000","last_change":"2024-03-28T13:18:59.084837+0000","last_active":"2024-03-28T13:19:00.092358+0000","last_peered":"2024-03-28T13:19:00.092358+0000","last_clean":"2024-03-28T13:19:00.092358+0000","last_became_active":"2024-03-28T13:18:59.083085+0000","last_became_peered":"2024-03-28T13:18:59.083085+0000","last_unstale":"2024-03-28T13:19:00.092358+0000","last_undegraded":"2024-03-28T13:19:00.092358+0000","last_fullsized":"2024-03-28T13:19:00.092358+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:16:46.004371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094293+0000","last_change":"2024-03-28T13:18:59.088878+0000","last_active":"2024-03-28T13:18:59.094293+0000","last_peered":"2024-03-28T13:18:59.094293+0000","last_clean":"2024-03-28T13:18:59.094293+0000","last_became_active":"2024-03-28T13:18:59.088493+0000","last_became_peered":"2024-03-28T13:18:59.088493+0000","last_unstale":"2024-03-28T13:18:59.094293+0000","last_undegraded":"2024-03-28T13:18:59.094293+0000","last_fullsized":"2024-03-28T13:18:59.094293+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:48:04.926545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091805+0000","last_change":"2024-03-28T13:18:59.100683+0000","last_active":"2024-03-28T13:19:00.091805+0000","last_peered":"2024-03-28T13:19:00.091805+0000","last_clean":"2024-03-28T13:19:00.091805+0000","last_became_active":"2024-03-28T13:18:59.099214+0000","last_became_peered":"2024-03-28T13:18:59.099214+0000","last_unstale":"2024-03-28T13:19:00.091805+0000","last_undegraded":"2024-03-28T13:19:00.091805+0000","last_fullsized":"2024-03-28T13:19:00.091805+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:41.892054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.095497+0000","last_change":"2024-03-28T13:18:59.090740+0000","last_active":"2024-03-28T13:18:59.095497+0000","last_peered":"2024-03-28T13:18:59.095497+0000","last_clean":"2024-03-28T13:18:59.095497+0000","last_became_active":"2024-03-28T13:18:59.090473+0000","last_became_peered":"2024-03-28T13:18:59.090473+0000","last_unstale":"2024-03-28T13:18:59.095497+0000","last_undegraded":"2024-03-28T13:18:59.095497+0000","last_fullsized":"2024-03-28T13:18:59.095497+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:31.353091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.226045+0000","last_change":"2024-03-28T13:18:59.088146+0000","last_active":"2024-03-28T13:19:01.226045+0000","last_peered":"2024-03-28T13:19:01.226045+0000","last_clean":"2024-03-28T13:19:01.226045+0000","last_became_active":"2024-03-28T13:18:59.087788+0000","last_became_peered":"2024-03-28T13:18:59.087788+0000","last_unstale":"2024-03-28T13:19:01.226045+0000","last_undegraded":"2024-03-28T13:19:01.226045+0000","last_fullsized":"2024-03-28T13:19:01.226045+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:31:19.713409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.089797+0000","last_change":"2024-03-28T13:18:59.080979+0000","last_active":"2024-03-28T13:19:00.089797+0000","last_peered":"2024-03-28T13:19:00.089797+0000","last_clean":"2024-03-28T13:19:00.089797+0000","last_became_active":"2024-03-28T13:18:59.080676+0000","last_became_peered":"2024-03-28T13:18:59.080676+0000","last_unstale":"2024-03-28T13:19:00.089797+0000","last_undegraded":"2024-03-28T13:19:00.089797+0000","last_fullsized":"2024-03-28T13:19:00.089797+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:20:49.865685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090048+0000","last_change":"2024-03-28T13:18:59.077084+0000","last_active":"2024-03-28T13:19:00.090048+0000","last_peered":"2024-03-28T13:19:00.090048+0000","last_clean":"2024-03-28T13:19:00.090048+0000","last_became_active":"2024-03-28T13:18:59.076136+0000","last_became_peered":"2024-03-28T13:18:59.076136+0000","last_unstale":"2024-03-28T13:19:00.090048+0000","last_undegraded":"2024-03-28T13:19:00.090048+0000","last_fullsized":"2024-03-28T13:19:00.090048+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:08:42.338113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090808+0000","last_change":"2024-03-28T13:18:59.074866+0000","last_active":"2024-03-28T13:19:00.090808+0000","last_peered":"2024-03-28T13:19:00.090808+0000","last_clean":"2024-03-28T13:19:00.090808+0000","last_became_active":"2024-03-28T13:18:59.074514+0000","last_became_peered":"2024-03-28T13:18:59.074514+0000","last_unstale":"2024-03-28T13:19:00.090808+0000","last_undegraded":"2024-03-28T13:19:00.090808+0000","last_fullsized":"2024-03-28T13:19:00.090808+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:10:44.680345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.094816+0000","last_change":"2024-03-28T13:18:59.084712+0000","last_active":"2024-03-28T13:19:00.094816+0000","last_peered":"2024-03-28T13:19:00.094816+0000","last_clean":"2024-03-28T13:19:00.094816+0000","last_became_active":"2024-03-28T13:18:59.084435+0000","last_became_peered":"2024-03-28T13:18:59.084435+0000","last_unstale":"2024-03-28T13:19:00.094816+0000","last_undegraded":"2024-03-28T13:19:00.094816+0000","last_fullsized":"2024-03-28T13:19:00.094816+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:44:01.634969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091450+0000","last_change":"2024-03-28T13:18:59.083710+0000","last_active":"2024-03-28T13:19:00.091450+0000","last_peered":"2024-03-28T13:19:00.091450+0000","last_clean":"2024-03-28T13:19:00.091450+0000","last_became_active":"2024-03-28T13:18:59.083366+0000","last_became_peered":"2024-03-28T13:18:59.083366+0000","last_unstale":"2024-03-28T13:19:00.091450+0000","last_undegraded":"2024-03-28T13:19:00.091450+0000","last_fullsized":"2024-03-28T13:19:00.091450+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:00:07.119299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093855+0000","last_change":"2024-03-28T13:18:59.076959+0000","last_active":"2024-03-28T13:19:00.093855+0000","last_peered":"2024-03-28T13:19:00.093855+0000","last_clean":"2024-03-28T13:19:00.093855+0000","last_became_active":"2024-03-28T13:18:59.075601+0000","last_became_peered":"2024-03-28T13:18:59.075601+0000","last_unstale":"2024-03-28T13:19:00.093855+0000","last_undegraded":"2024-03-28T13:19:00.093855+0000","last_fullsized":"2024-03-28T13:19:00.093855+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:43:21.159934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.093215+0000","last_change":"2024-03-28T13:18:59.087556+0000","last_active":"2024-03-28T13:18:59.093215+0000","last_peered":"2024-03-28T13:18:59.093215+0000","last_clean":"2024-03-28T13:18:59.093215+0000","last_became_active":"2024-03-28T13:18:59.087183+0000","last_became_peered":"2024-03-28T13:18:59.087183+0000","last_unstale":"2024-03-28T13:18:59.093215+0000","last_undegraded":"2024-03-28T13:18:59.093215+0000","last_fullsized":"2024-03-28T13:18:59.093215+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:28:49.643546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100753+0000","last_change":"2024-03-28T13:18:59.085370+0000","last_active":"2024-03-28T13:19:00.100753+0000","last_peered":"2024-03-28T13:19:00.100753+0000","last_clean":"2024-03-28T13:19:00.100753+0000","last_became_active":"2024-03-28T13:18:59.084826+0000","last_became_peered":"2024-03-28T13:18:59.084826+0000","last_unstale":"2024-03-28T13:19:00.100753+0000","last_undegraded":"2024-03-28T13:19:00.100753+0000","last_fullsized":"2024-03-28T13:19:00.100753+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:09.396769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101032+0000","last_change":"2024-03-28T13:18:59.091677+0000","last_active":"2024-03-28T13:19:00.101032+0000","last_peered":"2024-03-28T13:19:00.101032+0000","last_clean":"2024-03-28T13:19:00.101032+0000","last_became_active":"2024-03-28T13:18:59.091299+0000","last_became_peered":"2024-03-28T13:18:59.091299+0000","last_unstale":"2024-03-28T13:19:00.101032+0000","last_undegraded":"2024-03-28T13:19:00.101032+0000","last_fullsized":"2024-03-28T13:19:00.101032+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:26:09.194396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090234+0000","last_change":"2024-03-28T13:18:59.083262+0000","last_active":"2024-03-28T13:19:00.090234+0000","last_peered":"2024-03-28T13:19:00.090234+0000","last_clean":"2024-03-28T13:19:00.090234+0000","last_became_active":"2024-03-28T13:18:59.082908+0000","last_became_peered":"2024-03-28T13:18:59.082908+0000","last_unstale":"2024-03-28T13:19:00.090234+0000","last_undegraded":"2024-03-28T13:19:00.090234+0000","last_fullsized":"2024-03-28T13:19:00.090234+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:48.131499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.089502+0000","last_change":"2024-03-28T13:18:59.085398+0000","last_active":"2024-03-28T13:18:59.089502+0000","last_peered":"2024-03-28T13:18:59.089502+0000","last_clean":"2024-03-28T13:18:59.089502+0000","last_became_active":"2024-03-28T13:18:59.083434+0000","last_became_peered":"2024-03-28T13:18:59.083434+0000","last_unstale":"2024-03-28T13:18:59.089502+0000","last_undegraded":"2024-03-28T13:18:59.089502+0000","last_fullsized":"2024-03-28T13:18:59.089502+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:00.261227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.090300+0000","last_change":"2024-03-28T13:18:59.084798+0000","last_active":"2024-03-28T13:18:59.090300+0000","last_peered":"2024-03-28T13:18:59.090300+0000","last_clean":"2024-03-28T13:18:59.090300+0000","last_became_active":"2024-03-28T13:18:59.084216+0000","last_became_peered":"2024-03-28T13:18:59.084216+0000","last_unstale":"2024-03-28T13:18:59.090300+0000","last_undegraded":"2024-03-28T13:18:59.090300+0000","last_fullsized":"2024-03-28T13:18:59.090300+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:45.824120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225950+0000","last_change":"2024-03-28T13:18:59.086762+0000","last_active":"2024-03-28T13:19:01.225950+0000","last_peered":"2024-03-28T13:19:01.225950+0000","last_clean":"2024-03-28T13:19:01.225950+0000","last_became_active":"2024-03-28T13:18:59.086511+0000","last_became_peered":"2024-03-28T13:18:59.086511+0000","last_unstale":"2024-03-28T13:19:01.225950+0000","last_undegraded":"2024-03-28T13:19:01.225950+0000","last_fullsized":"2024-03-28T13:19:01.225950+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:29:57.882927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.086078+0000","last_change":"2024-03-28T13:18:59.080256+0000","last_active":"2024-03-28T13:18:59.086078+0000","last_peered":"2024-03-28T13:18:59.086078+0000","last_clean":"2024-03-28T13:18:59.086078+0000","last_became_active":"2024-03-28T13:18:59.079957+0000","last_became_peered":"2024-03-28T13:18:59.079957+0000","last_unstale":"2024-03-28T13:18:59.086078+0000","last_undegraded":"2024-03-28T13:18:59.086078+0000","last_fullsized":"2024-03-28T13:18:59.086078+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:20.977905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.084557+0000","last_change":"2024-03-28T13:18:59.078806+0000","last_active":"2024-03-28T13:18:59.084557+0000","last_peered":"2024-03-28T13:18:59.084557+0000","last_clean":"2024-03-28T13:18:59.084557+0000","last_became_active":"2024-03-28T13:18:59.078526+0000","last_became_peered":"2024-03-28T13:18:59.078526+0000","last_unstale":"2024-03-28T13:18:59.084557+0000","last_undegraded":"2024-03-28T13:18:59.084557+0000","last_fullsized":"2024-03-28T13:18:59.084557+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:26:09.194396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.226080+0000","last_change":"2024-03-28T13:18:59.093088+0000","last_active":"2024-03-28T13:19:01.226080+0000","last_peered":"2024-03-28T13:19:01.226080+0000","last_clean":"2024-03-28T13:19:01.226080+0000","last_became_active":"2024-03-28T13:18:59.092704+0000","last_became_peered":"2024-03-28T13:18:59.092704+0000","last_unstale":"2024-03-28T13:19:01.226080+0000","last_undegraded":"2024-03-28T13:19:01.226080+0000","last_fullsized":"2024-03-28T13:19:01.226080+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:55:14.850361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225935+0000","last_change":"2024-03-28T13:18:59.089095+0000","last_active":"2024-03-28T13:19:01.225935+0000","last_peered":"2024-03-28T13:19:01.225935+0000","last_clean":"2024-03-28T13:19:01.225935+0000","last_became_active":"2024-03-28T13:18:59.088783+0000","last_became_peered":"2024-03-28T13:18:59.088783+0000","last_unstale":"2024-03-28T13:19:01.225935+0000","last_undegraded":"2024-03-28T13:19:01.225935+0000","last_fullsized":"2024-03-28T13:19:01.225935+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:51:28.159619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100578+0000","last_change":"2024-03-28T13:18:59.086506+0000","last_active":"2024-03-28T13:19:00.100578+0000","last_peered":"2024-03-28T13:19:00.100578+0000","last_clean":"2024-03-28T13:19:00.100578+0000","last_became_active":"2024-03-28T13:18:59.086216+0000","last_became_peered":"2024-03-28T13:18:59.086216+0000","last_unstale":"2024-03-28T13:19:00.100578+0000","last_undegraded":"2024-03-28T13:19:00.100578+0000","last_fullsized":"2024-03-28T13:19:00.100578+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:48:04.926545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.089655+0000","last_change":"2024-03-28T13:18:59.075805+0000","last_active":"2024-03-28T13:19:00.089655+0000","last_peered":"2024-03-28T13:19:00.089655+0000","last_clean":"2024-03-28T13:19:00.089655+0000","last_became_active":"2024-03-28T13:18:59.074979+0000","last_became_peered":"2024-03-28T13:18:59.074979+0000","last_unstale":"2024-03-28T13:19:00.089655+0000","last_undegraded":"2024-03-28T13:19:00.089655+0000","last_fullsized":"2024-03-28T13:19:00.089655+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:11:23.081337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.094204+0000","last_change":"2024-03-28T13:18:59.091367+0000","last_active":"2024-03-28T13:19:00.094204+0000","last_peered":"2024-03-28T13:19:00.094204+0000","last_clean":"2024-03-28T13:19:00.094204+0000","last_became_active":"2024-03-28T13:18:59.091083+0000","last_became_peered":"2024-03-28T13:18:59.091083+0000","last_unstale":"2024-03-28T13:19:00.094204+0000","last_undegraded":"2024-03-28T13:19:00.094204+0000","last_fullsized":"2024-03-28T13:19:00.094204+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:49:04.881309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"104'4","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.226241+0000","last_change":"2024-03-28T13:18:58.087418+0000","last_active":"2024-03-28T13:19:01.226241+0000","last_peered":"2024-03-28T13:19:01.226241+0000","last_clean":"2024-03-28T13:19:01.226241+0000","last_became_active":"2024-03-28T13:18:58.086932+0000","last_became_peered":"2024-03-28T13:18:58.086932+0000","last_unstale":"2024-03-28T13:19:01.226241+0000","last_undegraded":"2024-03-28T13:19:01.226241+0000","last_fullsized":"2024-03-28T13:19:01.226241+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:38.575107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.226336+0000","last_change":"2024-03-28T13:18:59.088501+0000","last_active":"2024-03-28T13:19:01.226336+0000","last_peered":"2024-03-28T13:19:01.226336+0000","last_clean":"2024-03-28T13:19:01.226336+0000","last_became_active":"2024-03-28T13:18:59.088117+0000","last_became_peered":"2024-03-28T13:18:59.088117+0000","last_unstale":"2024-03-28T13:19:01.226336+0000","last_undegraded":"2024-03-28T13:19:01.226336+0000","last_fullsized":"2024-03-28T13:19:01.226336+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:24:41.712000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092249+0000","last_change":"2024-03-28T13:18:58.077769+0000","last_active":"2024-03-28T13:19:00.092249+0000","last_peered":"2024-03-28T13:19:00.092249+0000","last_clean":"2024-03-28T13:19:00.092249+0000","last_became_active":"2024-03-28T13:18:58.077234+0000","last_became_peered":"2024-03-28T13:18:58.077234+0000","last_unstale":"2024-03-28T13:19:00.092249+0000","last_undegraded":"2024-03-28T13:19:00.092249+0000","last_fullsized":"2024-03-28T13:19:00.092249+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:10:27.168742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091333+0000","last_change":"2024-03-28T13:18:59.084112+0000","last_active":"2024-03-28T13:19:00.091333+0000","last_peered":"2024-03-28T13:19:00.091333+0000","last_clean":"2024-03-28T13:19:00.091333+0000","last_became_active":"2024-03-28T13:18:59.083805+0000","last_became_peered":"2024-03-28T13:18:59.083805+0000","last_unstale":"2024-03-28T13:19:00.091333+0000","last_undegraded":"2024-03-28T13:19:00.091333+0000","last_fullsized":"2024-03-28T13:19:00.091333+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:14:07.278326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101343+0000","last_change":"2024-03-28T13:18:58.087832+0000","last_active":"2024-03-28T13:19:00.101343+0000","last_peered":"2024-03-28T13:19:00.101343+0000","last_clean":"2024-03-28T13:19:00.101343+0000","last_became_active":"2024-03-28T13:18:58.087250+0000","last_became_peered":"2024-03-28T13:18:58.087250+0000","last_unstale":"2024-03-28T13:19:00.101343+0000","last_undegraded":"2024-03-28T13:19:00.101343+0000","last_fullsized":"2024-03-28T13:19:00.101343+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:55:22.065081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.099692+0000","last_change":"2024-03-28T13:18:59.097206+0000","last_active":"2024-03-28T13:18:59.099692+0000","last_peered":"2024-03-28T13:18:59.099692+0000","last_clean":"2024-03-28T13:18:59.099692+0000","last_became_active":"2024-03-28T13:18:59.096903+0000","last_became_peered":"2024-03-28T13:18:59.096903+0000","last_unstale":"2024-03-28T13:18:59.099692+0000","last_undegraded":"2024-03-28T13:18:59.099692+0000","last_fullsized":"2024-03-28T13:18:59.099692+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:09.396769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091455+0000","last_change":"2024-03-28T13:18:58.083531+0000","last_active":"2024-03-28T13:19:00.091455+0000","last_peered":"2024-03-28T13:19:00.091455+0000","last_clean":"2024-03-28T13:19:00.091455+0000","last_became_active":"2024-03-28T13:18:58.082970+0000","last_became_peered":"2024-03-28T13:18:58.082970+0000","last_unstale":"2024-03-28T13:19:00.091455+0000","last_undegraded":"2024-03-28T13:19:00.091455+0000","last_fullsized":"2024-03-28T13:19:00.091455+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:08.383486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.100576+0000","last_change":"2024-03-28T13:18:59.098346+0000","last_active":"2024-03-28T13:18:59.100576+0000","last_peered":"2024-03-28T13:18:59.100576+0000","last_clean":"2024-03-28T13:18:59.100576+0000","last_became_active":"2024-03-28T13:18:59.098052+0000","last_became_peered":"2024-03-28T13:18:59.098052+0000","last_unstale":"2024-03-28T13:18:59.100576+0000","last_undegraded":"2024-03-28T13:18:59.100576+0000","last_fullsized":"2024-03-28T13:18:59.100576+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:20:44.200653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090424+0000","last_change":"2024-03-28T13:18:58.064634+0000","last_active":"2024-03-28T13:19:00.090424+0000","last_peered":"2024-03-28T13:19:00.090424+0000","last_clean":"2024-03-28T13:19:00.090424+0000","last_became_active":"2024-03-28T13:18:58.064194+0000","last_became_peered":"2024-03-28T13:18:58.064194+0000","last_unstale":"2024-03-28T13:19:00.090424+0000","last_undegraded":"2024-03-28T13:19:00.090424+0000","last_fullsized":"2024-03-28T13:19:00.090424+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:21:46.733401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091377+0000","last_change":"2024-03-28T13:18:59.094754+0000","last_active":"2024-03-28T13:19:00.091377+0000","last_peered":"2024-03-28T13:19:00.091377+0000","last_clean":"2024-03-28T13:19:00.091377+0000","last_became_active":"2024-03-28T13:18:59.090317+0000","last_became_peered":"2024-03-28T13:18:59.090317+0000","last_unstale":"2024-03-28T13:19:00.091377+0000","last_undegraded":"2024-03-28T13:19:00.091377+0000","last_fullsized":"2024-03-28T13:19:00.091377+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:09:17.961351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100702+0000","last_change":"2024-03-28T13:18:58.082525+0000","last_active":"2024-03-28T13:19:00.100702+0000","last_peered":"2024-03-28T13:19:00.100702+0000","last_clean":"2024-03-28T13:19:00.100702+0000","last_became_active":"2024-03-28T13:18:58.082039+0000","last_became_peered":"2024-03-28T13:18:58.082039+0000","last_unstale":"2024-03-28T13:19:00.100702+0000","last_undegraded":"2024-03-28T13:19:00.100702+0000","last_fullsized":"2024-03-28T13:19:00.100702+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:42.609780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225536+0000","last_change":"2024-03-28T13:18:59.093894+0000","last_active":"2024-03-28T13:19:01.225536+0000","last_peered":"2024-03-28T13:19:01.225536+0000","last_clean":"2024-03-28T13:19:01.225536+0000","last_became_active":"2024-03-28T13:18:59.093258+0000","last_became_peered":"2024-03-28T13:18:59.093258+0000","last_unstale":"2024-03-28T13:19:01.225536+0000","last_undegraded":"2024-03-28T13:19:01.225536+0000","last_fullsized":"2024-03-28T13:19:01.225536+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:34:15.217129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090824+0000","last_change":"2024-03-28T13:18:58.078267+0000","last_active":"2024-03-28T13:19:00.090824+0000","last_peered":"2024-03-28T13:19:00.090824+0000","last_clean":"2024-03-28T13:19:00.090824+0000","last_became_active":"2024-03-28T13:18:58.077891+0000","last_became_peered":"2024-03-28T13:18:58.077891+0000","last_unstale":"2024-03-28T13:19:00.090824+0000","last_undegraded":"2024-03-28T13:19:00.090824+0000","last_fullsized":"2024-03-28T13:19:00.090824+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:35:04.854788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.102069+0000","last_change":"2024-03-28T13:18:59.099035+0000","last_active":"2024-03-28T13:18:59.102069+0000","last_peered":"2024-03-28T13:18:59.102069+0000","last_clean":"2024-03-28T13:18:59.102069+0000","last_became_active":"2024-03-28T13:18:59.098118+0000","last_became_peered":"2024-03-28T13:18:59.098118+0000","last_unstale":"2024-03-28T13:18:59.102069+0000","last_undegraded":"2024-03-28T13:18:59.102069+0000","last_fullsized":"2024-03-28T13:18:59.102069+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:21:47.746684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092964+0000","last_change":"2024-03-28T13:18:58.076072+0000","last_active":"2024-03-28T13:19:00.092964+0000","last_peered":"2024-03-28T13:19:00.092964+0000","last_clean":"2024-03-28T13:19:00.092964+0000","last_became_active":"2024-03-28T13:18:58.075808+0000","last_became_peered":"2024-03-28T13:18:58.075808+0000","last_unstale":"2024-03-28T13:19:00.092964+0000","last_undegraded":"2024-03-28T13:19:00.092964+0000","last_fullsized":"2024-03-28T13:19:00.092964+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:11:22.068054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091695+0000","last_change":"2024-03-28T13:18:59.075982+0000","last_active":"2024-03-28T13:19:00.091695+0000","last_peered":"2024-03-28T13:19:00.091695+0000","last_clean":"2024-03-28T13:19:00.091695+0000","last_became_active":"2024-03-28T13:18:59.075606+0000","last_became_peered":"2024-03-28T13:18:59.075606+0000","last_unstale":"2024-03-28T13:19:00.091695+0000","last_undegraded":"2024-03-28T13:19:00.091695+0000","last_fullsized":"2024-03-28T13:19:00.091695+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:12:28.089180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101379+0000","last_change":"2024-03-28T13:18:58.077577+0000","last_active":"2024-03-28T13:19:00.101379+0000","last_peered":"2024-03-28T13:19:00.101379+0000","last_clean":"2024-03-28T13:19:00.101379+0000","last_became_active":"2024-03-28T13:18:58.077275+0000","last_became_peered":"2024-03-28T13:18:58.077275+0000","last_unstale":"2024-03-28T13:19:00.101379+0000","last_undegraded":"2024-03-28T13:19:00.101379+0000","last_fullsized":"2024-03-28T13:19:00.101379+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:28:48.630263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.089723+0000","last_change":"2024-03-28T13:18:59.086040+0000","last_active":"2024-03-28T13:18:59.089723+0000","last_peered":"2024-03-28T13:18:59.089723+0000","last_clean":"2024-03-28T13:18:59.089723+0000","last_became_active":"2024-03-28T13:18:59.085749+0000","last_became_peered":"2024-03-28T13:18:59.085749+0000","last_unstale":"2024-03-28T13:18:59.089723+0000","last_undegraded":"2024-03-28T13:18:59.089723+0000","last_fullsized":"2024-03-28T13:18:59.089723+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:27:52.931365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091810+0000","last_change":"2024-03-28T13:18:58.083463+0000","last_active":"2024-03-28T13:19:00.091810+0000","last_peered":"2024-03-28T13:19:00.091810+0000","last_clean":"2024-03-28T13:19:00.091810+0000","last_became_active":"2024-03-28T13:18:58.083152+0000","last_became_peered":"2024-03-28T13:18:58.083152+0000","last_unstale":"2024-03-28T13:19:00.091810+0000","last_undegraded":"2024-03-28T13:19:00.091810+0000","last_fullsized":"2024-03-28T13:19:00.091810+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:48:03.913262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091380+0000","last_change":"2024-03-28T13:18:59.081727+0000","last_active":"2024-03-28T13:19:00.091380+0000","last_peered":"2024-03-28T13:19:00.091380+0000","last_clean":"2024-03-28T13:19:00.091380+0000","last_became_active":"2024-03-28T13:18:59.081436+0000","last_became_peered":"2024-03-28T13:18:59.081436+0000","last_unstale":"2024-03-28T13:19:00.091380+0000","last_undegraded":"2024-03-28T13:19:00.091380+0000","last_fullsized":"2024-03-28T13:19:00.091380+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:55:41.223520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"104'3","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092029+0000","last_change":"2024-03-28T13:18:58.068721+0000","last_active":"2024-03-28T13:19:00.092029+0000","last_peered":"2024-03-28T13:19:00.092029+0000","last_clean":"2024-03-28T13:19:00.092029+0000","last_became_active":"2024-03-28T13:18:58.068416+0000","last_became_peered":"2024-03-28T13:18:58.068416+0000","last_unstale":"2024-03-28T13:19:00.092029+0000","last_undegraded":"2024-03-28T13:19:00.092029+0000","last_fullsized":"2024-03-28T13:19:00.092029+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:16:13.945885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.086220+0000","last_change":"2024-03-28T13:18:59.080942+0000","last_active":"2024-03-28T13:18:59.086220+0000","last_peered":"2024-03-28T13:18:59.086220+0000","last_clean":"2024-03-28T13:18:59.086220+0000","last_became_active":"2024-03-28T13:18:59.080654+0000","last_became_peered":"2024-03-28T13:18:59.080654+0000","last_unstale":"2024-03-28T13:18:59.086220+0000","last_undegraded":"2024-03-28T13:18:59.086220+0000","last_fullsized":"2024-03-28T13:18:59.086220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:17:42.586454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"104'6","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.110193+0000","last_change":"2024-03-28T13:18:58.076129+0000","last_active":"2024-03-28T13:18:59.110193+0000","last_peered":"2024-03-28T13:18:59.110193+0000","last_clean":"2024-03-28T13:18:59.110193+0000","last_became_active":"2024-03-28T13:18:58.075809+0000","last_became_peered":"2024-03-28T13:18:58.075809+0000","last_unstale":"2024-03-28T13:18:59.110193+0000","last_undegraded":"2024-03-28T13:18:59.110193+0000","last_fullsized":"2024-03-28T13:18:59.110193+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:49:19.123418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090517+0000","last_change":"2024-03-28T13:18:59.077667+0000","last_active":"2024-03-28T13:19:00.090517+0000","last_peered":"2024-03-28T13:19:00.090517+0000","last_clean":"2024-03-28T13:19:00.090517+0000","last_became_active":"2024-03-28T13:18:59.077304+0000","last_became_peered":"2024-03-28T13:18:59.077304+0000","last_unstale":"2024-03-28T13:19:00.090517+0000","last_undegraded":"2024-03-28T13:19:00.090517+0000","last_fullsized":"2024-03-28T13:19:00.090517+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:18.635249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.094221+0000","last_change":"2024-03-28T13:18:58.077229+0000","last_active":"2024-03-28T13:19:00.094221+0000","last_peered":"2024-03-28T13:19:00.094221+0000","last_clean":"2024-03-28T13:19:00.094221+0000","last_became_active":"2024-03-28T13:18:58.076961+0000","last_became_peered":"2024-03-28T13:18:58.076961+0000","last_unstale":"2024-03-28T13:19:00.094221+0000","last_undegraded":"2024-03-28T13:19:00.094221+0000","last_fullsized":"2024-03-28T13:19:00.094221+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:20:48.852402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100869+0000","last_change":"2024-03-28T13:18:59.084784+0000","last_active":"2024-03-28T13:19:00.100869+0000","last_peered":"2024-03-28T13:19:00.100869+0000","last_clean":"2024-03-28T13:19:00.100869+0000","last_became_active":"2024-03-28T13:18:59.084518+0000","last_became_peered":"2024-03-28T13:18:59.084518+0000","last_unstale":"2024-03-28T13:19:00.100869+0000","last_undegraded":"2024-03-28T13:19:00.100869+0000","last_fullsized":"2024-03-28T13:19:00.100869+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:29.522394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090380+0000","last_change":"2024-03-28T13:18:58.083463+0000","last_active":"2024-03-28T13:19:00.090380+0000","last_peered":"2024-03-28T13:19:00.090380+0000","last_clean":"2024-03-28T13:19:00.090380+0000","last_became_active":"2024-03-28T13:18:58.083122+0000","last_became_peered":"2024-03-28T13:18:58.083122+0000","last_unstale":"2024-03-28T13:19:00.090380+0000","last_undegraded":"2024-03-28T13:19:00.090380+0000","last_fullsized":"2024-03-28T13:19:00.090380+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:20:43.187370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090923+0000","last_change":"2024-03-28T13:18:59.082251+0000","last_active":"2024-03-28T13:19:00.090923+0000","last_peered":"2024-03-28T13:19:00.090923+0000","last_clean":"2024-03-28T13:19:00.090923+0000","last_became_active":"2024-03-28T13:18:59.081951+0000","last_became_peered":"2024-03-28T13:18:59.081951+0000","last_unstale":"2024-03-28T13:19:00.090923+0000","last_undegraded":"2024-03-28T13:19:00.090923+0000","last_fullsized":"2024-03-28T13:19:00.090923+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:53.418554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087871+0000","last_change":"2024-03-28T13:18:58.066532+0000","last_active":"2024-03-28T13:18:59.087871+0000","last_peered":"2024-03-28T13:18:59.087871+0000","last_clean":"2024-03-28T13:18:59.087871+0000","last_became_active":"2024-03-28T13:18:58.066208+0000","last_became_peered":"2024-03-28T13:18:58.066208+0000","last_unstale":"2024-03-28T13:18:59.087871+0000","last_undegraded":"2024-03-28T13:18:59.087871+0000","last_fullsized":"2024-03-28T13:18:59.087871+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:58.349889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100798+0000","last_change":"2024-03-28T13:18:59.090442+0000","last_active":"2024-03-28T13:19:00.100798+0000","last_peered":"2024-03-28T13:19:00.100798+0000","last_clean":"2024-03-28T13:19:00.100798+0000","last_became_active":"2024-03-28T13:18:59.090071+0000","last_became_peered":"2024-03-28T13:18:59.090071+0000","last_unstale":"2024-03-28T13:19:00.100798+0000","last_undegraded":"2024-03-28T13:19:00.100798+0000","last_fullsized":"2024-03-28T13:19:00.100798+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:29.899339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"104'8","reported_seq":22,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.089921+0000","last_change":"2024-03-28T13:18:58.071341+0000","last_active":"2024-03-28T13:19:00.089921+0000","last_peered":"2024-03-28T13:19:00.089921+0000","last_clean":"2024-03-28T13:19:00.089921+0000","last_became_active":"2024-03-28T13:18:58.069574+0000","last_became_peered":"2024-03-28T13:18:58.069574+0000","last_unstale":"2024-03-28T13:19:00.089921+0000","last_undegraded":"2024-03-28T13:19:00.089921+0000","last_fullsized":"2024-03-28T13:19:00.089921+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:13.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094648+0000","last_change":"2024-03-28T13:18:59.093731+0000","last_active":"2024-03-28T13:18:59.094648+0000","last_peered":"2024-03-28T13:18:59.094648+0000","last_clean":"2024-03-28T13:18:59.094648+0000","last_became_active":"2024-03-28T13:18:59.093471+0000","last_became_peered":"2024-03-28T13:18:59.093471+0000","last_unstale":"2024-03-28T13:18:59.094648+0000","last_undegraded":"2024-03-28T13:18:59.094648+0000","last_fullsized":"2024-03-28T13:18:59.094648+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:18:27.237579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.108148+0000","last_change":"2024-03-28T13:18:58.080576+0000","last_active":"2024-03-28T13:18:59.108148+0000","last_peered":"2024-03-28T13:18:59.108148+0000","last_clean":"2024-03-28T13:18:59.108148+0000","last_became_active":"2024-03-28T13:18:58.080272+0000","last_became_peered":"2024-03-28T13:18:58.080272+0000","last_unstale":"2024-03-28T13:18:59.108148+0000","last_undegraded":"2024-03-28T13:18:59.108148+0000","last_fullsized":"2024-03-28T13:18:59.108148+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:13.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094243+0000","last_change":"2024-03-28T13:18:59.092760+0000","last_active":"2024-03-28T13:18:59.094243+0000","last_peered":"2024-03-28T13:18:59.094243+0000","last_clean":"2024-03-28T13:18:59.094243+0000","last_became_active":"2024-03-28T13:18:59.092409+0000","last_became_peered":"2024-03-28T13:18:59.092409+0000","last_unstale":"2024-03-28T13:18:59.094243+0000","last_undegraded":"2024-03-28T13:18:59.094243+0000","last_fullsized":"2024-03-28T13:18:59.094243+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:19.489710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092132+0000","last_change":"2024-03-28T13:18:58.065630+0000","last_active":"2024-03-28T13:19:00.092132+0000","last_peered":"2024-03-28T13:19:00.092132+0000","last_clean":"2024-03-28T13:19:00.092132+0000","last_became_active":"2024-03-28T13:18:58.065333+0000","last_became_peered":"2024-03-28T13:18:58.065333+0000","last_unstale":"2024-03-28T13:19:00.092132+0000","last_undegraded":"2024-03-28T13:19:00.092132+0000","last_fullsized":"2024-03-28T13:19:00.092132+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:54:14.505307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.099342+0000","last_change":"2024-03-28T13:18:59.092928+0000","last_active":"2024-03-28T13:18:59.099342+0000","last_peered":"2024-03-28T13:18:59.099342+0000","last_clean":"2024-03-28T13:18:59.099342+0000","last_became_active":"2024-03-28T13:18:59.092579+0000","last_became_peered":"2024-03-28T13:18:59.092579+0000","last_unstale":"2024-03-28T13:18:59.099342+0000","last_undegraded":"2024-03-28T13:18:59.099342+0000","last_fullsized":"2024-03-28T13:18:59.099342+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:37:45.650491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093878+0000","last_change":"2024-03-28T13:18:58.075199+0000","last_active":"2024-03-28T13:19:00.093878+0000","last_peered":"2024-03-28T13:19:00.093878+0000","last_clean":"2024-03-28T13:19:00.093878+0000","last_became_active":"2024-03-28T13:18:58.073665+0000","last_became_peered":"2024-03-28T13:18:58.073665+0000","last_unstale":"2024-03-28T13:19:00.093878+0000","last_undegraded":"2024-03-28T13:19:00.093878+0000","last_fullsized":"2024-03-28T13:19:00.093878+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:08:41.324830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093360+0000","last_change":"2024-03-28T13:18:59.076634+0000","last_active":"2024-03-28T13:19:00.093360+0000","last_peered":"2024-03-28T13:19:00.093360+0000","last_clean":"2024-03-28T13:19:00.093360+0000","last_became_active":"2024-03-28T13:18:59.076295+0000","last_became_peered":"2024-03-28T13:18:59.076295+0000","last_unstale":"2024-03-28T13:19:00.093360+0000","last_undegraded":"2024-03-28T13:19:00.093360+0000","last_fullsized":"2024-03-28T13:19:00.093360+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:11:39.100208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090726+0000","last_change":"2024-03-28T13:18:58.084188+0000","last_active":"2024-03-28T13:19:00.090726+0000","last_peered":"2024-03-28T13:19:00.090726+0000","last_clean":"2024-03-28T13:19:00.090726+0000","last_became_active":"2024-03-28T13:18:58.081720+0000","last_became_peered":"2024-03-28T13:18:58.081720+0000","last_unstale":"2024-03-28T13:19:00.090726+0000","last_undegraded":"2024-03-28T13:19:00.090726+0000","last_fullsized":"2024-03-28T13:19:00.090726+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:44.810836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093931+0000","last_change":"2024-03-28T13:18:59.079466+0000","last_active":"2024-03-28T13:19:00.093931+0000","last_peered":"2024-03-28T13:19:00.093931+0000","last_clean":"2024-03-28T13:19:00.093931+0000","last_became_active":"2024-03-28T13:18:59.078752+0000","last_became_peered":"2024-03-28T13:18:59.078752+0000","last_unstale":"2024-03-28T13:19:00.093931+0000","last_undegraded":"2024-03-28T13:19:00.093931+0000","last_fullsized":"2024-03-28T13:19:00.093931+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:50.468775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.066401+0000","last_change":"2024-03-28T13:18:58.079436+0000","last_active":"2024-03-28T13:18:59.066401+0000","last_peered":"2024-03-28T13:18:59.066401+0000","last_clean":"2024-03-28T13:18:59.066401+0000","last_became_active":"2024-03-28T13:18:58.079066+0000","last_became_peered":"2024-03-28T13:18:58.079066+0000","last_unstale":"2024-03-28T13:18:59.066401+0000","last_undegraded":"2024-03-28T13:18:59.066401+0000","last_fullsized":"2024-03-28T13:18:59.066401+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:08.383486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090744+0000","last_change":"2024-03-28T13:18:59.080751+0000","last_active":"2024-03-28T13:19:00.090744+0000","last_peered":"2024-03-28T13:19:00.090744+0000","last_clean":"2024-03-28T13:19:00.090744+0000","last_became_active":"2024-03-28T13:18:59.080449+0000","last_became_peered":"2024-03-28T13:18:59.080449+0000","last_unstale":"2024-03-28T13:19:00.090744+0000","last_undegraded":"2024-03-28T13:19:00.090744+0000","last_fullsized":"2024-03-28T13:19:00.090744+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:18:33.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.075738+0000","last_change":"2024-03-28T13:18:58.079341+0000","last_active":"2024-03-28T13:18:59.075738+0000","last_peered":"2024-03-28T13:18:59.075738+0000","last_clean":"2024-03-28T13:18:59.075738+0000","last_became_active":"2024-03-28T13:18:58.079008+0000","last_became_peered":"2024-03-28T13:18:58.079008+0000","last_unstale":"2024-03-28T13:18:59.075738+0000","last_undegraded":"2024-03-28T13:18:59.075738+0000","last_fullsized":"2024-03-28T13:18:59.075738+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:21:46.733401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.086441+0000","last_change":"2024-03-28T13:18:59.080827+0000","last_active":"2024-03-28T13:18:59.086441+0000","last_peered":"2024-03-28T13:18:59.086441+0000","last_clean":"2024-03-28T13:18:59.086441+0000","last_became_active":"2024-03-28T13:18:59.080524+0000","last_became_peered":"2024-03-28T13:18:59.080524+0000","last_unstale":"2024-03-28T13:18:59.086441+0000","last_undegraded":"2024-03-28T13:18:59.086441+0000","last_fullsized":"2024-03-28T13:18:59.086441+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:43:24.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100342+0000","last_change":"2024-03-28T13:18:58.090817+0000","last_active":"2024-03-28T13:19:00.100342+0000","last_peered":"2024-03-28T13:19:00.100342+0000","last_clean":"2024-03-28T13:19:00.100342+0000","last_became_active":"2024-03-28T13:18:58.090469+0000","last_became_peered":"2024-03-28T13:18:58.090469+0000","last_unstale":"2024-03-28T13:19:00.100342+0000","last_undegraded":"2024-03-28T13:19:00.100342+0000","last_fullsized":"2024-03-28T13:19:00.100342+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:52.858677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.094597+0000","last_change":"2024-03-28T13:18:59.071403+0000","last_active":"2024-03-28T13:19:00.094597+0000","last_peered":"2024-03-28T13:19:00.094597+0000","last_clean":"2024-03-28T13:19:00.094597+0000","last_became_active":"2024-03-28T13:18:59.071095+0000","last_became_peered":"2024-03-28T13:18:59.071095+0000","last_unstale":"2024-03-28T13:19:00.094597+0000","last_undegraded":"2024-03-28T13:19:00.094597+0000","last_fullsized":"2024-03-28T13:19:00.094597+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:09:52.654125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.105978+0000","last_change":"2024-03-28T13:18:58.071038+0000","last_active":"2024-03-28T13:18:59.105978+0000","last_peered":"2024-03-28T13:18:59.105978+0000","last_clean":"2024-03-28T13:18:59.105978+0000","last_became_active":"2024-03-28T13:18:58.069807+0000","last_became_peered":"2024-03-28T13:18:58.069807+0000","last_unstale":"2024-03-28T13:18:59.105978+0000","last_undegraded":"2024-03-28T13:18:59.105978+0000","last_fullsized":"2024-03-28T13:18:59.105978+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:17:17.825753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.095041+0000","last_change":"2024-03-28T13:18:59.076727+0000","last_active":"2024-03-28T13:19:00.095041+0000","last_peered":"2024-03-28T13:19:00.095041+0000","last_clean":"2024-03-28T13:19:00.095041+0000","last_became_active":"2024-03-28T13:18:59.076415+0000","last_became_peered":"2024-03-28T13:18:59.076415+0000","last_unstale":"2024-03-28T13:19:00.095041+0000","last_undegraded":"2024-03-28T13:19:00.095041+0000","last_fullsized":"2024-03-28T13:19:00.095041+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:56:27.830992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"105'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.095140+0000","last_change":"2024-03-28T13:18:58.086609+0000","last_active":"2024-03-28T13:19:00.095140+0000","last_peered":"2024-03-28T13:19:00.095140+0000","last_clean":"2024-03-28T13:19:00.095140+0000","last_became_active":"2024-03-28T13:18:58.086223+0000","last_became_peered":"2024-03-28T13:18:58.086223+0000","last_unstale":"2024-03-28T13:19:00.095140+0000","last_undegraded":"2024-03-28T13:19:00.095140+0000","last_fullsized":"2024-03-28T13:19:00.095140+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:17.621966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101315+0000","last_change":"2024-03-28T13:18:59.098557+0000","last_active":"2024-03-28T13:19:00.101315+0000","last_peered":"2024-03-28T13:19:00.101315+0000","last_clean":"2024-03-28T13:19:00.101315+0000","last_became_active":"2024-03-28T13:18:59.098274+0000","last_became_peered":"2024-03-28T13:18:59.098274+0000","last_unstale":"2024-03-28T13:19:00.101315+0000","last_undegraded":"2024-03-28T13:19:00.101315+0000","last_fullsized":"2024-03-28T13:19:00.101315+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:20.977905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.094124+0000","last_change":"2024-03-28T13:18:58.072039+0000","last_active":"2024-03-28T13:19:00.094124+0000","last_peered":"2024-03-28T13:19:00.094124+0000","last_clean":"2024-03-28T13:19:00.094124+0000","last_became_active":"2024-03-28T13:18:58.071683+0000","last_became_peered":"2024-03-28T13:18:58.071683+0000","last_unstale":"2024-03-28T13:19:00.094124+0000","last_undegraded":"2024-03-28T13:19:00.094124+0000","last_fullsized":"2024-03-28T13:19:00.094124+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:47.118216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091918+0000","last_change":"2024-03-28T13:18:59.087471+0000","last_active":"2024-03-28T13:19:00.091918+0000","last_peered":"2024-03-28T13:19:00.091918+0000","last_clean":"2024-03-28T13:19:00.091918+0000","last_became_active":"2024-03-28T13:18:59.085466+0000","last_became_peered":"2024-03-28T13:18:59.085466+0000","last_unstale":"2024-03-28T13:19:00.091918+0000","last_undegraded":"2024-03-28T13:19:00.091918+0000","last_fullsized":"2024-03-28T13:19:00.091918+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:30:44.259677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.067207+0000","last_change":"2024-03-28T13:18:58.068025+0000","last_active":"2024-03-28T13:18:59.067207+0000","last_peered":"2024-03-28T13:18:59.067207+0000","last_clean":"2024-03-28T13:18:59.067207+0000","last_became_active":"2024-03-28T13:18:58.067671+0000","last_became_peered":"2024-03-28T13:18:58.067671+0000","last_unstale":"2024-03-28T13:18:59.067207+0000","last_undegraded":"2024-03-28T13:18:59.067207+0000","last_fullsized":"2024-03-28T13:18:59.067207+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:28.509110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093804+0000","last_change":"2024-03-28T13:18:59.073922+0000","last_active":"2024-03-28T13:19:00.093804+0000","last_peered":"2024-03-28T13:19:00.093804+0000","last_clean":"2024-03-28T13:19:00.093804+0000","last_became_active":"2024-03-28T13:18:59.073566+0000","last_became_peered":"2024-03-28T13:18:59.073566+0000","last_unstale":"2024-03-28T13:19:00.093804+0000","last_undegraded":"2024-03-28T13:19:00.093804+0000","last_fullsized":"2024-03-28T13:19:00.093804+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:38:33.010172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101273+0000","last_change":"2024-03-28T13:18:58.085176+0000","last_active":"2024-03-28T13:19:00.101273+0000","last_peered":"2024-03-28T13:19:00.101273+0000","last_clean":"2024-03-28T13:19:00.101273+0000","last_became_active":"2024-03-28T13:18:58.084898+0000","last_became_peered":"2024-03-28T13:18:58.084898+0000","last_unstale":"2024-03-28T13:19:00.101273+0000","last_undegraded":"2024-03-28T13:19:00.101273+0000","last_fullsized":"2024-03-28T13:19:00.101273+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:13:25.784483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091562+0000","last_change":"2024-03-28T13:18:59.076130+0000","last_active":"2024-03-28T13:19:00.091562+0000","last_peered":"2024-03-28T13:19:00.091562+0000","last_clean":"2024-03-28T13:19:00.091562+0000","last_became_active":"2024-03-28T13:18:59.075785+0000","last_became_peered":"2024-03-28T13:18:59.075785+0000","last_unstale":"2024-03-28T13:19:00.091562+0000","last_undegraded":"2024-03-28T13:19:00.091562+0000","last_fullsized":"2024-03-28T13:19:00.091562+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:31.836556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.065847+0000","last_change":"2024-03-28T13:18:58.073984+0000","last_active":"2024-03-28T13:18:59.065847+0000","last_peered":"2024-03-28T13:18:59.065847+0000","last_clean":"2024-03-28T13:18:59.065847+0000","last_became_active":"2024-03-28T13:18:58.073520+0000","last_became_peered":"2024-03-28T13:18:58.073520+0000","last_unstale":"2024-03-28T13:18:59.065847+0000","last_undegraded":"2024-03-28T13:18:59.065847+0000","last_fullsized":"2024-03-28T13:18:59.065847+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:28.886056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093125+0000","last_change":"2024-03-28T13:18:59.078032+0000","last_active":"2024-03-28T13:19:00.093125+0000","last_peered":"2024-03-28T13:19:00.093125+0000","last_clean":"2024-03-28T13:19:00.093125+0000","last_became_active":"2024-03-28T13:18:59.077665+0000","last_became_peered":"2024-03-28T13:18:59.077665+0000","last_unstale":"2024-03-28T13:19:00.093125+0000","last_undegraded":"2024-03-28T13:19:00.093125+0000","last_fullsized":"2024-03-28T13:19:00.093125+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:14:14.261611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.226316+0000","last_change":"2024-03-28T13:18:58.083533+0000","last_active":"2024-03-28T13:19:01.226316+0000","last_peered":"2024-03-28T13:19:01.226316+0000","last_clean":"2024-03-28T13:19:01.226316+0000","last_became_active":"2024-03-28T13:18:58.083220+0000","last_became_peered":"2024-03-28T13:18:58.083220+0000","last_unstale":"2024-03-28T13:19:01.226316+0000","last_undegraded":"2024-03-28T13:19:01.226316+0000","last_fullsized":"2024-03-28T13:19:01.226316+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:44:18.338917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.098913+0000","last_change":"2024-03-28T13:18:59.091706+0000","last_active":"2024-03-28T13:18:59.098913+0000","last_peered":"2024-03-28T13:18:59.098913+0000","last_clean":"2024-03-28T13:18:59.098913+0000","last_became_active":"2024-03-28T13:18:59.091410+0000","last_became_peered":"2024-03-28T13:18:59.091410+0000","last_unstale":"2024-03-28T13:18:59.098913+0000","last_undegraded":"2024-03-28T13:18:59.098913+0000","last_fullsized":"2024-03-28T13:18:59.098913+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:15.681052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100688+0000","last_change":"2024-03-28T13:18:58.086792+0000","last_active":"2024-03-28T13:19:00.100688+0000","last_peered":"2024-03-28T13:19:00.100688+0000","last_clean":"2024-03-28T13:19:00.100688+0000","last_became_active":"2024-03-28T13:18:58.086544+0000","last_became_peered":"2024-03-28T13:18:58.086544+0000","last_unstale":"2024-03-28T13:19:00.100688+0000","last_undegraded":"2024-03-28T13:19:00.100688+0000","last_fullsized":"2024-03-28T13:19:00.100688+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:51:50.282781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101376+0000","last_change":"2024-03-28T13:18:59.088616+0000","last_active":"2024-03-28T13:19:00.101376+0000","last_peered":"2024-03-28T13:19:00.101376+0000","last_clean":"2024-03-28T13:19:00.101376+0000","last_became_active":"2024-03-28T13:18:59.088292+0000","last_became_peered":"2024-03-28T13:18:59.088292+0000","last_unstale":"2024-03-28T13:19:00.101376+0000","last_undegraded":"2024-03-28T13:19:00.101376+0000","last_fullsized":"2024-03-28T13:19:00.101376+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:43:24.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.107904+0000","last_change":"2024-03-28T13:18:58.075299+0000","last_active":"2024-03-28T13:18:59.107904+0000","last_peered":"2024-03-28T13:18:59.107904+0000","last_clean":"2024-03-28T13:18:59.107904+0000","last_became_active":"2024-03-28T13:18:58.075035+0000","last_became_peered":"2024-03-28T13:18:58.075035+0000","last_unstale":"2024-03-28T13:18:59.107904+0000","last_undegraded":"2024-03-28T13:18:59.107904+0000","last_fullsized":"2024-03-28T13:18:59.107904+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:39.277367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":320,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087250+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:18:59.087250+0000","last_peered":"2024-03-28T13:18:59.087250+0000","last_clean":"2024-03-28T13:18:59.087250+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:18:59.087250+0000","last_undegraded":"2024-03-28T13:18:59.087250+0000","last_fullsized":"2024-03-28T13:18:59.087250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476856,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":550446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":550322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1888256,"data_stored":1142344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1273856,"data_stored":550252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509558,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":556552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509559,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1900544,"data_stored":1148588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509559,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":557134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542263,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":557134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:19:04.739 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-28T13:19:04.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.995+0000 7ff342c00640 1 Processor -- start 2024-03-28T13:19:05.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.996+0000 7ff342c00640 1 -- start start 2024-03-28T13:19:05.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.997+0000 7ff342c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33c0cf050 0x7ff33c15f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:05.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.997+0000 7ff342c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff33c0cf6c0 con 0x7ff33c0d2110 2024-03-28T13:19:05.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.997+0000 7ff342c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff33c0cf9a0 con 0x7ff33c0d0e70 2024-03-28T13:19:05.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.997+0000 7ff342c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff33c15fc20 con 0x7ff33c0cf050 2024-03-28T13:19:05.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.998+0000 7ff340c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff33c0d2110 0x7ff33c0d25a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:57386/0 (socket says 172.21.15.19:57386) 2024-03-28T13:19:05.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.998+0000 7ff340c00640 1 -- 172.21.15.19:0/1223314287 learned_addr learned my addr 172.21.15.19:0/1223314287 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:05.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.998+0000 7ff333e00640 1 --2- 172.21.15.19:0/1223314287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33c0cf050 0x7ff33c15f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:05.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.999+0000 7ff33b400640 1 -- 172.21.15.19:0/1223314287 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2211470113 0 0) 0x7ff33c0cf6c0 con 0x7ff33c0d2110 2024-03-28T13:19:05.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.999+0000 7ff33b400640 1 -- 172.21.15.19:0/1223314287 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff320003520 con 0x7ff33c0d2110 2024-03-28T13:19:05.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.999+0000 7ff33b400640 1 -- 172.21.15.19:0/1223314287 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 679765940 0 0) 0x7ff33c0cf9a0 con 0x7ff33c0d0e70 2024-03-28T13:19:05.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.999+0000 7ff33b400640 1 -- 172.21.15.19:0/1223314287 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff33c0cf6c0 con 0x7ff33c0d0e70 2024-03-28T13:19:05.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.999+0000 7ff333e00640 1 -- 172.21.15.19:0/1223314287 >> v1:172.21.15.188:6789/0 conn(0x7ff33c0d0e70 legacy=0x7ff33c0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:05.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.999+0000 7ff333e00640 1 -- 172.21.15.19:0/1223314287 >> v1:172.21.15.19:6789/0 conn(0x7ff33c0d2110 legacy=0x7ff33c0d25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:05.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:04.999+0000 7ff333e00640 1 -- 172.21.15.19:0/1223314287 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c0cfc80 con 0x7ff33c0cf050 2024-03-28T13:19:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.000+0000 7ff333e00640 1 --2- 172.21.15.19:0/1223314287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33c0cf050 0x7ff33c15f6e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff3240097d0 tx=0x7ff32402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e51f830cdb2f0445 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.001+0000 7ff33b400640 1 -- 172.21.15.19:0/1223314287 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324035020 con 0x7ff33c0cf050 2024-03-28T13:19:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.001+0000 7ff33b400640 1 -- 172.21.15.19:0/1223314287 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff324031890 con 0x7ff33c0cf050 2024-03-28T13:19:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.001+0000 7ff33b400640 1 -- 172.21.15.19:0/1223314287 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff324031c60 con 0x7ff33c0cf050 2024-03-28T13:19:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.001+0000 7ff342c00640 1 -- 172.21.15.19:0/1223314287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33c0cf050 msgr2=0x7ff33c15f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.001+0000 7ff342c00640 1 --2- 172.21.15.19:0/1223314287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33c0cf050 0x7ff33c15f6e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff3240097d0 tx=0x7ff32402b930 comp rx=0 tx=0).stop 2024-03-28T13:19:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.001+0000 7ff342c00640 1 -- 172.21.15.19:0/1223314287 shutdown_connections 2024-03-28T13:19:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.001+0000 7ff342c00640 1 --2- 172.21.15.19:0/1223314287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33c0cf050 0x7ff33c15f6e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.001+0000 7ff342c00640 1 -- 172.21.15.19:0/1223314287 >> 172.21.15.19:0/1223314287 conn(0x7ff33c08bfb0 msgr2=0x7ff33c0d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.002+0000 7ff342c00640 1 -- 172.21.15.19:0/1223314287 shutdown_connections 2024-03-28T13:19:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.002+0000 7ff342c00640 1 -- 172.21.15.19:0/1223314287 wait complete. 2024-03-28T13:19:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.003+0000 7ff342c00640 1 Processor -- start 2024-03-28T13:19:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.004+0000 7ff342c00640 1 -- start start 2024-03-28T13:19:05.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.005+0000 7ff342c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33c0cf050 0x7ff33c14ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:05.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.005+0000 7ff342c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff33c0d0e70 0x7ff33c0c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:05.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.005+0000 7ff342c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff33c0d2110 0x7ff33c0b9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:05.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.005+0000 7ff342c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c15e910 con 0x7ff33c0d2110 2024-03-28T13:19:05.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.005+0000 7ff342c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c0d0240 con 0x7ff33c0d0e70 2024-03-28T13:19:05.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.005+0000 7ff342c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff33c15ea90 con 0x7ff33c0cf050 2024-03-28T13:19:05.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.005+0000 7ff340c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff33c0d2110 0x7ff33c0b9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:05.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.005+0000 7ff340c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff33c0d2110 0x7ff33c0b9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59422/0 (socket says 172.21.15.19:59422) 2024-03-28T13:19:05.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.005+0000 7ff340c00640 1 -- 172.21.15.19:0/2521090930 learned_addr learned my addr 172.21.15.19:0/2521090930 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:05.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.005+0000 7ff33be00640 1 --2- 172.21.15.19:0/2521090930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33c0cf050 0x7ff33c14ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:05.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.005+0000 7ff333e00640 1 --2- 172.21.15.19:0/2521090930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff33c0d0e70 0x7ff33c0c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:05.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.006+0000 7ff340c00640 1 -- 172.21.15.19:0/2521090930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33c0cf050 msgr2=0x7ff33c14ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:05.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.006+0000 7ff340c00640 1 --2- 172.21.15.19:0/2521090930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33c0cf050 0x7ff33c14ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.006+0000 7ff340c00640 1 -- 172.21.15.19:0/2521090930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff33c0d0e70 msgr2=0x7ff33c0c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:05.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.006+0000 7ff340c00640 1 --2- 172.21.15.19:0/2521090930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff33c0d0e70 0x7ff33c0c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.006+0000 7ff340c00640 1 -- 172.21.15.19:0/2521090930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c0b8890 con 0x7ff33c0d2110 2024-03-28T13:19:05.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.006+0000 7ff340c00640 1 --2- 172.21.15.19:0/2521090930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff33c0d2110 0x7ff33c0b9930 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ff33400caf0 tx=0x7ff33400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:05.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.006+0000 7ff338c00640 1 -- 172.21.15.19:0/2521090930 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334017070 con 0x7ff33c0d2110 2024-03-28T13:19:05.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.006+0000 7ff338c00640 1 -- 172.21.15.19:0/2521090930 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff33400ac40 con 0x7ff33c0d2110 2024-03-28T13:19:05.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.006+0000 7ff338c00640 1 -- 172.21.15.19:0/2521090930 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334012400 con 0x7ff33c0d2110 2024-03-28T13:19:05.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.006+0000 7ff342c00640 1 -- 172.21.15.19:0/2521090930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33c0b8b70 con 0x7ff33c0d2110 2024-03-28T13:19:05.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.007+0000 7ff342c00640 1 -- 172.21.15.19:0/2521090930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33c0b9130 con 0x7ff33c0d2110 2024-03-28T13:19:05.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.008+0000 7ff342c00640 1 -- 172.21.15.19:0/2521090930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff33c068cd0 con 0x7ff33c0d2110 2024-03-28T13:19:05.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.009+0000 7ff338c00640 1 -- 172.21.15.19:0/2521090930 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff33401a070 con 0x7ff33c0d2110 2024-03-28T13:19:05.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.010+0000 7ff338c00640 1 --2- 172.21.15.19:0/2521090930 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff31806c530 0x7ff31806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:05.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.010+0000 7ff338c00640 1 -- 172.21.15.19:0/2521090930 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7ff33408b5f0 con 0x7ff33c0d2110 2024-03-28T13:19:05.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.011+0000 7ff33be00640 1 --2- 172.21.15.19:0/2521090930 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff31806c530 0x7ff31806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:05.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.012+0000 7ff33be00640 1 --2- 172.21.15.19:0/2521090930 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff31806c530 0x7ff31806e9f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff32c001650 tx=0x7ff32c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:05.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.014+0000 7ff338c00640 1 -- 172.21.15.19:0/2521090930 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff334059ae0 con 0x7ff33c0d2110 2024-03-28T13:19:05.103 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-03-28T13:19:05.020+0000 7f299cc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:19:05.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.382+0000 7ff342c00640 1 -- 172.21.15.19:0/2521090930 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ff33c0d5050 con 0x7ff33c0d2110 2024-03-28T13:19:05.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.383+0000 7ff338c00640 1 -- 172.21.15.19:0/2521090930 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1305 (secure 0 0 0) 0x7ff3340593e0 con 0x7ff33c0d2110 2024-03-28T13:19:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:05.382 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":588,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":105,"num_osds":8,"num_up_osds":8,"osd_up_since":1711631369,"num_in_osds":8,"osd_in_since":1711631359,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":266928128,"bytes_avail":772827185152,"bytes_total":773094113280,"write_bytes_sec":2331,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":83,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":8040}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-03-28T13:18:16.180006+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-28T13:19:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.385+0000 7ff342c00640 1 -- 172.21.15.19:0/2521090930 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff31806c530 msgr2=0x7ff31806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:05.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.385+0000 7ff342c00640 1 --2- 172.21.15.19:0/2521090930 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff31806c530 0x7ff31806e9f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7ff32c001650 tx=0x7ff32c009040 comp rx=0 tx=0).stop 2024-03-28T13:19:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.385+0000 7ff342c00640 1 -- 172.21.15.19:0/2521090930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff33c0d2110 msgr2=0x7ff33c0b9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.385+0000 7ff342c00640 1 --2- 172.21.15.19:0/2521090930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff33c0d2110 0x7ff33c0b9930 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ff33400caf0 tx=0x7ff33400a060 comp rx=0 tx=0).stop 2024-03-28T13:19:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.385+0000 7ff342c00640 1 -- 172.21.15.19:0/2521090930 shutdown_connections 2024-03-28T13:19:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.385+0000 7ff342c00640 1 --2- 172.21.15.19:0/2521090930 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff31806c530 0x7ff31806e9f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.385+0000 7ff342c00640 1 --2- 172.21.15.19:0/2521090930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff33c0d2110 0x7ff33c0b9930 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.385+0000 7ff342c00640 1 --2- 172.21.15.19:0/2521090930 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff33c0d0e70 0x7ff33c0c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.385+0000 7ff342c00640 1 --2- 172.21.15.19:0/2521090930 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff33c0cf050 0x7ff33c14ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.385+0000 7ff342c00640 1 -- 172.21.15.19:0/2521090930 >> 172.21.15.19:0/2521090930 conn(0x7ff33c08bfb0 msgr2=0x7ff33c15f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.385+0000 7ff342c00640 1 -- 172.21.15.19:0/2521090930 shutdown_connections 2024-03-28T13:19:05.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.386+0000 7ff342c00640 1 -- 172.21.15.19:0/2521090930 wait complete. 2024-03-28T13:19:05.403 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-28T13:19:05.404 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:19:05.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.609+0000 7f598d400640 1 Processor -- start 2024-03-28T13:19:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.609+0000 7f598d400640 1 -- start start 2024-03-28T13:19:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.611+0000 7f598d400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f598815c390 0x7f5988073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.611+0000 7f598d400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f59880d0650 0x7f59880d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.611+0000 7f598d400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f598815fe80 con 0x7f59880af830 2024-03-28T13:19:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.611+0000 7f598d400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f598815c770 con 0x7f59880d0650 2024-03-28T13:19:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.611+0000 7f598d400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f59880d0fe0 con 0x7f598815c390 2024-03-28T13:19:05.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.611+0000 7f5987200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f59880d0650 0x7f59880d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:05.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.612+0000 7f5987200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f59880d0650 0x7f59880d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58428/0 (socket says 172.21.15.19:58428) 2024-03-28T13:19:05.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.612+0000 7f5987200640 1 -- 172.21.15.19:0/2160713158 learned_addr learned my addr 172.21.15.19:0/2160713158 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:05.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.612+0000 7f5986800640 1 --2- 172.21.15.19:0/2160713158 >> v2:172.21.15.188:3300/0 conn(0x7f598815c390 0x7f5988073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:05.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.612+0000 7f5987200640 1 -- 172.21.15.19:0/2160713158 >> v2:172.21.15.188:3300/0 conn(0x7f598815c390 msgr2=0x7f5988073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:05.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.612+0000 7f5987200640 1 --2- 172.21.15.19:0/2160713158 >> v2:172.21.15.188:3300/0 conn(0x7f598815c390 0x7f5988073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.612+0000 7f5987200640 1 -- 172.21.15.19:0/2160713158 >> v1:172.21.15.188:6789/0 conn(0x7f59880af830 legacy=0x7f5988160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:05.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.612+0000 7f5987200640 1 -- 172.21.15.19:0/2160713158 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59880ad120 con 0x7f59880d0650 2024-03-28T13:19:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.613+0000 7f5987200640 1 --2- 172.21.15.19:0/2160713158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59880d0650 0x7f59880d0aa0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f59880b63e0 tx=0x7f598002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee60e2be7c26721 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.614+0000 7f5985e00640 1 -- 172.21.15.19:0/2160713158 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598002cb10 con 0x7f59880d0650 2024-03-28T13:19:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.614+0000 7f5985e00640 1 -- 172.21.15.19:0/2160713158 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f598002ccb0 con 0x7f59880d0650 2024-03-28T13:19:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.614+0000 7f5985e00640 1 -- 172.21.15.19:0/2160713158 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598002ce60 con 0x7f59880d0650 2024-03-28T13:19:05.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.614+0000 7f598d400640 1 -- 172.21.15.19:0/2160713158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59880d0650 msgr2=0x7f59880d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:05.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.614+0000 7f598d400640 1 --2- 172.21.15.19:0/2160713158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59880d0650 0x7f59880d0aa0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f59880b63e0 tx=0x7f598002bf70 comp rx=0 tx=0).stop 2024-03-28T13:19:05.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.614+0000 7f598d400640 1 -- 172.21.15.19:0/2160713158 shutdown_connections 2024-03-28T13:19:05.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.614+0000 7f598d400640 1 --2- 172.21.15.19:0/2160713158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f59880d0650 0x7f59880d0aa0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.614+0000 7f598d400640 1 --2- 172.21.15.19:0/2160713158 >> v2:172.21.15.188:3300/0 conn(0x7f598815c390 0x7f5988073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.614+0000 7f598d400640 1 -- 172.21.15.19:0/2160713158 >> 172.21.15.19:0/2160713158 conn(0x7f598806b990 msgr2=0x7f598806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:05.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.615+0000 7f598d400640 1 -- 172.21.15.19:0/2160713158 shutdown_connections 2024-03-28T13:19:05.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.615+0000 7f598d400640 1 -- 172.21.15.19:0/2160713158 wait complete. 2024-03-28T13:19:05.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.616+0000 7f598d400640 1 Processor -- start 2024-03-28T13:19:05.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.616+0000 7f598d400640 1 -- start start 2024-03-28T13:19:05.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.617+0000 7f598d400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59880af830 0x7f598815e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:05.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.617+0000 7f598d400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59880d0650 0x7f5988071f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:05.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.617+0000 7f598d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f598815c390 0x7f59880afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:05.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.617+0000 7f598d400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5988160120 con 0x7f598815c390 2024-03-28T13:19:05.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.617+0000 7f598d400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f59880ad6e0 con 0x7f59880af830 2024-03-28T13:19:05.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.617+0000 7f598d400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5988160d50 con 0x7f59880d0650 2024-03-28T13:19:05.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.618+0000 7f5987200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f598815c390 0x7f59880afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:05.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.618+0000 7f5987200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f598815c390 0x7f59880afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58436/0 (socket says 172.21.15.19:58436) 2024-03-28T13:19:05.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.618+0000 7f5987200640 1 -- 172.21.15.19:0/2980719778 learned_addr learned my addr 172.21.15.19:0/2980719778 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.618+0000 7f5986800640 1 --2- 172.21.15.19:0/2980719778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59880af830 0x7f598815e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.618+0000 7f597fe00640 1 --2- 172.21.15.19:0/2980719778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59880d0650 0x7f5988071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.618+0000 7f5987200640 1 -- 172.21.15.19:0/2980719778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59880d0650 msgr2=0x7f5988071f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.618+0000 7f5987200640 1 --2- 172.21.15.19:0/2980719778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59880d0650 0x7f5988071f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.618+0000 7f5987200640 1 -- 172.21.15.19:0/2980719778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59880af830 msgr2=0x7f598815e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.618+0000 7f5987200640 1 --2- 172.21.15.19:0/2980719778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59880af830 0x7f598815e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.618+0000 7f5987200640 1 -- 172.21.15.19:0/2980719778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5988147d60 con 0x7f598815c390 2024-03-28T13:19:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.618+0000 7f597fe00640 1 --2- 172.21.15.19:0/2980719778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59880d0650 0x7f5988071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.619+0000 7f5987200640 1 --2- 172.21.15.19:0/2980719778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f598815c390 0x7f59880afd80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f5980036040 tx=0x7f5980004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.619+0000 7f597ea00640 1 -- 172.21.15.19:0/2980719778 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980043020 con 0x7f598815c390 2024-03-28T13:19:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.619+0000 7f597ea00640 1 -- 172.21.15.19:0/2980719778 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f598003e450 con 0x7f598815c390 2024-03-28T13:19:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.619+0000 7f597ea00640 1 -- 172.21.15.19:0/2980719778 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598002ce60 con 0x7f598815c390 2024-03-28T13:19:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.619+0000 7f598d400640 1 -- 172.21.15.19:0/2980719778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5988148040 con 0x7f598815c390 2024-03-28T13:19:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.619+0000 7f598d400640 1 -- 172.21.15.19:0/2980719778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5988148600 con 0x7f598815c390 2024-03-28T13:19:05.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.620+0000 7f598d400640 1 -- 172.21.15.19:0/2980719778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59880685c0 con 0x7f598815c390 2024-03-28T13:19:05.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.621+0000 7f597ea00640 1 -- 172.21.15.19:0/2980719778 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5980046020 con 0x7f598815c390 2024-03-28T13:19:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.622+0000 7f597ea00640 1 --2- 172.21.15.19:0/2980719778 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f596406c530 0x7f596406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.622+0000 7f597ea00640 1 -- 172.21.15.19:0/2980719778 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f59800add00 con 0x7f598815c390 2024-03-28T13:19:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.623+0000 7f5986800640 1 --2- 172.21.15.19:0/2980719778 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f596406c530 0x7f596406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:05.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.624+0000 7f5986800640 1 --2- 172.21.15.19:0/2980719778 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f596406c530 0x7f596406e9f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5978006fd0 tx=0x7f5978008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.628+0000 7f597ea00640 1 -- 172.21.15.19:0/2980719778 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f598004d030 con 0x7f598815c390 2024-03-28T13:19:05.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.828+0000 7f598d400640 1 -- 172.21.15.19:0/2980719778 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f598806f1b0 con 0x7f596406c530 2024-03-28T13:19:05.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.847+0000 7f597ea00640 1 -- 172.21.15.19:0/2980719778 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252465 (secure 0 0 0) 0x7f598806f1b0 con 0x7f596406c530 2024-03-28T13:19:05.849 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:05.851 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:19:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.853+0000 7f598d400640 1 -- 172.21.15.19:0/2980719778 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f596406c530 msgr2=0x7f596406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.853+0000 7f598d400640 1 --2- 172.21.15.19:0/2980719778 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f596406c530 0x7f596406e9f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5978006fd0 tx=0x7f5978008040 comp rx=0 tx=0).stop 2024-03-28T13:19:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.853+0000 7f598d400640 1 -- 172.21.15.19:0/2980719778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f598815c390 msgr2=0x7f59880afd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.853+0000 7f598d400640 1 --2- 172.21.15.19:0/2980719778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f598815c390 0x7f59880afd80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f5980036040 tx=0x7f5980004a40 comp rx=0 tx=0).stop 2024-03-28T13:19:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.853+0000 7f598d400640 1 -- 172.21.15.19:0/2980719778 shutdown_connections 2024-03-28T13:19:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.853+0000 7f598d400640 1 --2- 172.21.15.19:0/2980719778 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f596406c530 0x7f596406e9f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.853+0000 7f598d400640 1 --2- 172.21.15.19:0/2980719778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f598815c390 0x7f59880afd80 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.853+0000 7f598d400640 1 --2- 172.21.15.19:0/2980719778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59880d0650 0x7f5988071f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.853+0000 7f598d400640 1 --2- 172.21.15.19:0/2980719778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59880af830 0x7f598815e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.853+0000 7f598d400640 1 -- 172.21.15.19:0/2980719778 >> 172.21.15.19:0/2980719778 conn(0x7f598806b990 msgr2=0x7f5988157270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:05.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.853+0000 7f598d400640 1 -- 172.21.15.19:0/2980719778 shutdown_connections 2024-03-28T13:19:05.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:05.853+0000 7f598d400640 1 -- 172.21.15.19:0/2980719778 wait complete. 2024-03-28T13:19:05.870 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-03-28T13:19:04.198067+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260672,"kb_used_data":11224,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714048,"statfs":{"total":773094113280,"available":772827185152,"internally_reserved":0,"allocated":11493376,"data_stored":5612772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.149054"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091482+0000","last_change":"2024-03-28T13:18:59.085702+0000","last_active":"2024-03-28T13:19:00.091482+0000","last_peered":"2024-03-28T13:19:00.091482+0000","last_clean":"2024-03-28T13:19:00.091482+0000","last_became_active":"2024-03-28T13:18:59.085413+0000","last_became_peered":"2024-03-28T13:18:59.085413+0000","last_unstale":"2024-03-28T13:19:00.091482+0000","last_undegraded":"2024-03-28T13:19:00.091482+0000","last_fullsized":"2024-03-28T13:19:00.091482+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:04:28.366038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091242+0000","last_change":"2024-03-28T13:18:59.090611+0000","last_active":"2024-03-28T13:19:00.091242+0000","last_peered":"2024-03-28T13:19:00.091242+0000","last_clean":"2024-03-28T13:19:00.091242+0000","last_became_active":"2024-03-28T13:18:59.089972+0000","last_became_peered":"2024-03-28T13:18:59.089972+0000","last_unstale":"2024-03-28T13:19:00.091242+0000","last_undegraded":"2024-03-28T13:19:00.091242+0000","last_fullsized":"2024-03-28T13:19:00.091242+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:40.743749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101633+0000","last_change":"2024-03-28T13:18:59.098655+0000","last_active":"2024-03-28T13:19:00.101633+0000","last_peered":"2024-03-28T13:19:00.101633+0000","last_clean":"2024-03-28T13:19:00.101633+0000","last_became_active":"2024-03-28T13:18:59.098399+0000","last_became_peered":"2024-03-28T13:18:59.098399+0000","last_unstale":"2024-03-28T13:19:00.101633+0000","last_undegraded":"2024-03-28T13:19:00.101633+0000","last_fullsized":"2024-03-28T13:19:00.101633+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:17:42.586454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090818+0000","last_change":"2024-03-28T13:18:59.074664+0000","last_active":"2024-03-28T13:19:00.090818+0000","last_peered":"2024-03-28T13:19:00.090818+0000","last_clean":"2024-03-28T13:19:00.090818+0000","last_became_active":"2024-03-28T13:18:59.073993+0000","last_became_peered":"2024-03-28T13:18:59.073993+0000","last_unstale":"2024-03-28T13:19:00.090818+0000","last_undegraded":"2024-03-28T13:19:00.090818+0000","last_fullsized":"2024-03-28T13:19:00.090818+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:44:49.466448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091305+0000","last_change":"2024-03-28T13:18:59.090589+0000","last_active":"2024-03-28T13:19:00.091305+0000","last_peered":"2024-03-28T13:19:00.091305+0000","last_clean":"2024-03-28T13:19:00.091305+0000","last_became_active":"2024-03-28T13:18:59.089952+0000","last_became_peered":"2024-03-28T13:18:59.089952+0000","last_unstale":"2024-03-28T13:19:00.091305+0000","last_undegraded":"2024-03-28T13:19:00.091305+0000","last_fullsized":"2024-03-28T13:19:00.091305+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:09.506244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091182+0000","last_change":"2024-03-28T13:18:59.085728+0000","last_active":"2024-03-28T13:19:00.091182+0000","last_peered":"2024-03-28T13:19:00.091182+0000","last_clean":"2024-03-28T13:19:00.091182+0000","last_became_active":"2024-03-28T13:18:59.085447+0000","last_became_peered":"2024-03-28T13:18:59.085447+0000","last_unstale":"2024-03-28T13:19:00.091182+0000","last_undegraded":"2024-03-28T13:19:00.091182+0000","last_fullsized":"2024-03-28T13:19:00.091182+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:16.716159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091117+0000","last_change":"2024-03-28T13:18:59.090899+0000","last_active":"2024-03-28T13:19:00.091117+0000","last_peered":"2024-03-28T13:19:00.091117+0000","last_clean":"2024-03-28T13:19:00.091117+0000","last_became_active":"2024-03-28T13:18:59.086628+0000","last_became_peered":"2024-03-28T13:18:59.086628+0000","last_unstale":"2024-03-28T13:19:00.091117+0000","last_undegraded":"2024-03-28T13:19:00.091117+0000","last_fullsized":"2024-03-28T13:19:00.091117+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:05:31.597035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100303+0000","last_change":"2024-03-28T13:18:59.082961+0000","last_active":"2024-03-28T13:19:00.100303+0000","last_peered":"2024-03-28T13:19:00.100303+0000","last_clean":"2024-03-28T13:19:00.100303+0000","last_became_active":"2024-03-28T13:18:59.082608+0000","last_became_peered":"2024-03-28T13:18:59.082608+0000","last_unstale":"2024-03-28T13:19:00.100303+0000","last_undegraded":"2024-03-28T13:19:00.100303+0000","last_fullsized":"2024-03-28T13:19:00.100303+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:45.824120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092358+0000","last_change":"2024-03-28T13:18:59.084837+0000","last_active":"2024-03-28T13:19:00.092358+0000","last_peered":"2024-03-28T13:19:00.092358+0000","last_clean":"2024-03-28T13:19:00.092358+0000","last_became_active":"2024-03-28T13:18:59.083085+0000","last_became_peered":"2024-03-28T13:18:59.083085+0000","last_unstale":"2024-03-28T13:19:00.092358+0000","last_undegraded":"2024-03-28T13:19:00.092358+0000","last_fullsized":"2024-03-28T13:19:00.092358+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:16:46.004371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094293+0000","last_change":"2024-03-28T13:18:59.088878+0000","last_active":"2024-03-28T13:18:59.094293+0000","last_peered":"2024-03-28T13:18:59.094293+0000","last_clean":"2024-03-28T13:18:59.094293+0000","last_became_active":"2024-03-28T13:18:59.088493+0000","last_became_peered":"2024-03-28T13:18:59.088493+0000","last_unstale":"2024-03-28T13:18:59.094293+0000","last_undegraded":"2024-03-28T13:18:59.094293+0000","last_fullsized":"2024-03-28T13:18:59.094293+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:48:04.926545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091805+0000","last_change":"2024-03-28T13:18:59.100683+0000","last_active":"2024-03-28T13:19:00.091805+0000","last_peered":"2024-03-28T13:19:00.091805+0000","last_clean":"2024-03-28T13:19:00.091805+0000","last_became_active":"2024-03-28T13:18:59.099214+0000","last_became_peered":"2024-03-28T13:18:59.099214+0000","last_unstale":"2024-03-28T13:19:00.091805+0000","last_undegraded":"2024-03-28T13:19:00.091805+0000","last_fullsized":"2024-03-28T13:19:00.091805+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:41.892054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.095497+0000","last_change":"2024-03-28T13:18:59.090740+0000","last_active":"2024-03-28T13:18:59.095497+0000","last_peered":"2024-03-28T13:18:59.095497+0000","last_clean":"2024-03-28T13:18:59.095497+0000","last_became_active":"2024-03-28T13:18:59.090473+0000","last_became_peered":"2024-03-28T13:18:59.090473+0000","last_unstale":"2024-03-28T13:18:59.095497+0000","last_undegraded":"2024-03-28T13:18:59.095497+0000","last_fullsized":"2024-03-28T13:18:59.095497+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:14:31.353091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.226045+0000","last_change":"2024-03-28T13:18:59.088146+0000","last_active":"2024-03-28T13:19:01.226045+0000","last_peered":"2024-03-28T13:19:01.226045+0000","last_clean":"2024-03-28T13:19:01.226045+0000","last_became_active":"2024-03-28T13:18:59.087788+0000","last_became_peered":"2024-03-28T13:18:59.087788+0000","last_unstale":"2024-03-28T13:19:01.226045+0000","last_undegraded":"2024-03-28T13:19:01.226045+0000","last_fullsized":"2024-03-28T13:19:01.226045+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:31:19.713409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.089797+0000","last_change":"2024-03-28T13:18:59.080979+0000","last_active":"2024-03-28T13:19:00.089797+0000","last_peered":"2024-03-28T13:19:00.089797+0000","last_clean":"2024-03-28T13:19:00.089797+0000","last_became_active":"2024-03-28T13:18:59.080676+0000","last_became_peered":"2024-03-28T13:18:59.080676+0000","last_unstale":"2024-03-28T13:19:00.089797+0000","last_undegraded":"2024-03-28T13:19:00.089797+0000","last_fullsized":"2024-03-28T13:19:00.089797+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:20:49.865685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090048+0000","last_change":"2024-03-28T13:18:59.077084+0000","last_active":"2024-03-28T13:19:00.090048+0000","last_peered":"2024-03-28T13:19:00.090048+0000","last_clean":"2024-03-28T13:19:00.090048+0000","last_became_active":"2024-03-28T13:18:59.076136+0000","last_became_peered":"2024-03-28T13:18:59.076136+0000","last_unstale":"2024-03-28T13:19:00.090048+0000","last_undegraded":"2024-03-28T13:19:00.090048+0000","last_fullsized":"2024-03-28T13:19:00.090048+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:08:42.338113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090808+0000","last_change":"2024-03-28T13:18:59.074866+0000","last_active":"2024-03-28T13:19:00.090808+0000","last_peered":"2024-03-28T13:19:00.090808+0000","last_clean":"2024-03-28T13:19:00.090808+0000","last_became_active":"2024-03-28T13:18:59.074514+0000","last_became_peered":"2024-03-28T13:18:59.074514+0000","last_unstale":"2024-03-28T13:19:00.090808+0000","last_undegraded":"2024-03-28T13:19:00.090808+0000","last_fullsized":"2024-03-28T13:19:00.090808+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:10:44.680345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.094816+0000","last_change":"2024-03-28T13:18:59.084712+0000","last_active":"2024-03-28T13:19:00.094816+0000","last_peered":"2024-03-28T13:19:00.094816+0000","last_clean":"2024-03-28T13:19:00.094816+0000","last_became_active":"2024-03-28T13:18:59.084435+0000","last_became_peered":"2024-03-28T13:18:59.084435+0000","last_unstale":"2024-03-28T13:19:00.094816+0000","last_undegraded":"2024-03-28T13:19:00.094816+0000","last_fullsized":"2024-03-28T13:19:00.094816+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:44:01.634969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091450+0000","last_change":"2024-03-28T13:18:59.083710+0000","last_active":"2024-03-28T13:19:00.091450+0000","last_peered":"2024-03-28T13:19:00.091450+0000","last_clean":"2024-03-28T13:19:00.091450+0000","last_became_active":"2024-03-28T13:18:59.083366+0000","last_became_peered":"2024-03-28T13:18:59.083366+0000","last_unstale":"2024-03-28T13:19:00.091450+0000","last_undegraded":"2024-03-28T13:19:00.091450+0000","last_fullsized":"2024-03-28T13:19:00.091450+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:00:07.119299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093855+0000","last_change":"2024-03-28T13:18:59.076959+0000","last_active":"2024-03-28T13:19:00.093855+0000","last_peered":"2024-03-28T13:19:00.093855+0000","last_clean":"2024-03-28T13:19:00.093855+0000","last_became_active":"2024-03-28T13:18:59.075601+0000","last_became_peered":"2024-03-28T13:18:59.075601+0000","last_unstale":"2024-03-28T13:19:00.093855+0000","last_undegraded":"2024-03-28T13:19:00.093855+0000","last_fullsized":"2024-03-28T13:19:00.093855+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:43:21.159934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.093215+0000","last_change":"2024-03-28T13:18:59.087556+0000","last_active":"2024-03-28T13:18:59.093215+0000","last_peered":"2024-03-28T13:18:59.093215+0000","last_clean":"2024-03-28T13:18:59.093215+0000","last_became_active":"2024-03-28T13:18:59.087183+0000","last_became_peered":"2024-03-28T13:18:59.087183+0000","last_unstale":"2024-03-28T13:18:59.093215+0000","last_undegraded":"2024-03-28T13:18:59.093215+0000","last_fullsized":"2024-03-28T13:18:59.093215+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:28:49.643546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100753+0000","last_change":"2024-03-28T13:18:59.085370+0000","last_active":"2024-03-28T13:19:00.100753+0000","last_peered":"2024-03-28T13:19:00.100753+0000","last_clean":"2024-03-28T13:19:00.100753+0000","last_became_active":"2024-03-28T13:18:59.084826+0000","last_became_peered":"2024-03-28T13:18:59.084826+0000","last_unstale":"2024-03-28T13:19:00.100753+0000","last_undegraded":"2024-03-28T13:19:00.100753+0000","last_fullsized":"2024-03-28T13:19:00.100753+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:09.396769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101032+0000","last_change":"2024-03-28T13:18:59.091677+0000","last_active":"2024-03-28T13:19:00.101032+0000","last_peered":"2024-03-28T13:19:00.101032+0000","last_clean":"2024-03-28T13:19:00.101032+0000","last_became_active":"2024-03-28T13:18:59.091299+0000","last_became_peered":"2024-03-28T13:18:59.091299+0000","last_unstale":"2024-03-28T13:19:00.101032+0000","last_undegraded":"2024-03-28T13:19:00.101032+0000","last_fullsized":"2024-03-28T13:19:00.101032+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:26:09.194396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090234+0000","last_change":"2024-03-28T13:18:59.083262+0000","last_active":"2024-03-28T13:19:00.090234+0000","last_peered":"2024-03-28T13:19:00.090234+0000","last_clean":"2024-03-28T13:19:00.090234+0000","last_became_active":"2024-03-28T13:18:59.082908+0000","last_became_peered":"2024-03-28T13:18:59.082908+0000","last_unstale":"2024-03-28T13:19:00.090234+0000","last_undegraded":"2024-03-28T13:19:00.090234+0000","last_fullsized":"2024-03-28T13:19:00.090234+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:48.131499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.089502+0000","last_change":"2024-03-28T13:18:59.085398+0000","last_active":"2024-03-28T13:18:59.089502+0000","last_peered":"2024-03-28T13:18:59.089502+0000","last_clean":"2024-03-28T13:18:59.089502+0000","last_became_active":"2024-03-28T13:18:59.083434+0000","last_became_peered":"2024-03-28T13:18:59.083434+0000","last_unstale":"2024-03-28T13:18:59.089502+0000","last_undegraded":"2024-03-28T13:18:59.089502+0000","last_fullsized":"2024-03-28T13:18:59.089502+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:00.261227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.090300+0000","last_change":"2024-03-28T13:18:59.084798+0000","last_active":"2024-03-28T13:18:59.090300+0000","last_peered":"2024-03-28T13:18:59.090300+0000","last_clean":"2024-03-28T13:18:59.090300+0000","last_became_active":"2024-03-28T13:18:59.084216+0000","last_became_peered":"2024-03-28T13:18:59.084216+0000","last_unstale":"2024-03-28T13:18:59.090300+0000","last_undegraded":"2024-03-28T13:18:59.090300+0000","last_fullsized":"2024-03-28T13:18:59.090300+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:45.824120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225950+0000","last_change":"2024-03-28T13:18:59.086762+0000","last_active":"2024-03-28T13:19:01.225950+0000","last_peered":"2024-03-28T13:19:01.225950+0000","last_clean":"2024-03-28T13:19:01.225950+0000","last_became_active":"2024-03-28T13:18:59.086511+0000","last_became_peered":"2024-03-28T13:18:59.086511+0000","last_unstale":"2024-03-28T13:19:01.225950+0000","last_undegraded":"2024-03-28T13:19:01.225950+0000","last_fullsized":"2024-03-28T13:19:01.225950+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:29:57.882927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.086078+0000","last_change":"2024-03-28T13:18:59.080256+0000","last_active":"2024-03-28T13:18:59.086078+0000","last_peered":"2024-03-28T13:18:59.086078+0000","last_clean":"2024-03-28T13:18:59.086078+0000","last_became_active":"2024-03-28T13:18:59.079957+0000","last_became_peered":"2024-03-28T13:18:59.079957+0000","last_unstale":"2024-03-28T13:18:59.086078+0000","last_undegraded":"2024-03-28T13:18:59.086078+0000","last_fullsized":"2024-03-28T13:18:59.086078+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:20.977905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.084557+0000","last_change":"2024-03-28T13:18:59.078806+0000","last_active":"2024-03-28T13:18:59.084557+0000","last_peered":"2024-03-28T13:18:59.084557+0000","last_clean":"2024-03-28T13:18:59.084557+0000","last_became_active":"2024-03-28T13:18:59.078526+0000","last_became_peered":"2024-03-28T13:18:59.078526+0000","last_unstale":"2024-03-28T13:18:59.084557+0000","last_undegraded":"2024-03-28T13:18:59.084557+0000","last_fullsized":"2024-03-28T13:18:59.084557+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:26:09.194396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.226080+0000","last_change":"2024-03-28T13:18:59.093088+0000","last_active":"2024-03-28T13:19:01.226080+0000","last_peered":"2024-03-28T13:19:01.226080+0000","last_clean":"2024-03-28T13:19:01.226080+0000","last_became_active":"2024-03-28T13:18:59.092704+0000","last_became_peered":"2024-03-28T13:18:59.092704+0000","last_unstale":"2024-03-28T13:19:01.226080+0000","last_undegraded":"2024-03-28T13:19:01.226080+0000","last_fullsized":"2024-03-28T13:19:01.226080+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:55:14.850361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225935+0000","last_change":"2024-03-28T13:18:59.089095+0000","last_active":"2024-03-28T13:19:01.225935+0000","last_peered":"2024-03-28T13:19:01.225935+0000","last_clean":"2024-03-28T13:19:01.225935+0000","last_became_active":"2024-03-28T13:18:59.088783+0000","last_became_peered":"2024-03-28T13:18:59.088783+0000","last_unstale":"2024-03-28T13:19:01.225935+0000","last_undegraded":"2024-03-28T13:19:01.225935+0000","last_fullsized":"2024-03-28T13:19:01.225935+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:51:28.159619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100578+0000","last_change":"2024-03-28T13:18:59.086506+0000","last_active":"2024-03-28T13:19:00.100578+0000","last_peered":"2024-03-28T13:19:00.100578+0000","last_clean":"2024-03-28T13:19:00.100578+0000","last_became_active":"2024-03-28T13:18:59.086216+0000","last_became_peered":"2024-03-28T13:18:59.086216+0000","last_unstale":"2024-03-28T13:19:00.100578+0000","last_undegraded":"2024-03-28T13:19:00.100578+0000","last_fullsized":"2024-03-28T13:19:00.100578+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:48:04.926545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.089655+0000","last_change":"2024-03-28T13:18:59.075805+0000","last_active":"2024-03-28T13:19:00.089655+0000","last_peered":"2024-03-28T13:19:00.089655+0000","last_clean":"2024-03-28T13:19:00.089655+0000","last_became_active":"2024-03-28T13:18:59.074979+0000","last_became_peered":"2024-03-28T13:18:59.074979+0000","last_unstale":"2024-03-28T13:19:00.089655+0000","last_undegraded":"2024-03-28T13:19:00.089655+0000","last_fullsized":"2024-03-28T13:19:00.089655+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:11:23.081337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.094204+0000","last_change":"2024-03-28T13:18:59.091367+0000","last_active":"2024-03-28T13:19:00.094204+0000","last_peered":"2024-03-28T13:19:00.094204+0000","last_clean":"2024-03-28T13:19:00.094204+0000","last_became_active":"2024-03-28T13:18:59.091083+0000","last_became_peered":"2024-03-28T13:18:59.091083+0000","last_unstale":"2024-03-28T13:19:00.094204+0000","last_undegraded":"2024-03-28T13:19:00.094204+0000","last_fullsized":"2024-03-28T13:19:00.094204+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:49:04.881309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"104'4","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.226241+0000","last_change":"2024-03-28T13:18:58.087418+0000","last_active":"2024-03-28T13:19:01.226241+0000","last_peered":"2024-03-28T13:19:01.226241+0000","last_clean":"2024-03-28T13:19:01.226241+0000","last_became_active":"2024-03-28T13:18:58.086932+0000","last_became_peered":"2024-03-28T13:18:58.086932+0000","last_unstale":"2024-03-28T13:19:01.226241+0000","last_undegraded":"2024-03-28T13:19:01.226241+0000","last_fullsized":"2024-03-28T13:19:01.226241+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:38.575107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.226336+0000","last_change":"2024-03-28T13:18:59.088501+0000","last_active":"2024-03-28T13:19:01.226336+0000","last_peered":"2024-03-28T13:19:01.226336+0000","last_clean":"2024-03-28T13:19:01.226336+0000","last_became_active":"2024-03-28T13:18:59.088117+0000","last_became_peered":"2024-03-28T13:18:59.088117+0000","last_unstale":"2024-03-28T13:19:01.226336+0000","last_undegraded":"2024-03-28T13:19:01.226336+0000","last_fullsized":"2024-03-28T13:19:01.226336+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:24:41.712000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092249+0000","last_change":"2024-03-28T13:18:58.077769+0000","last_active":"2024-03-28T13:19:00.092249+0000","last_peered":"2024-03-28T13:19:00.092249+0000","last_clean":"2024-03-28T13:19:00.092249+0000","last_became_active":"2024-03-28T13:18:58.077234+0000","last_became_peered":"2024-03-28T13:18:58.077234+0000","last_unstale":"2024-03-28T13:19:00.092249+0000","last_undegraded":"2024-03-28T13:19:00.092249+0000","last_fullsized":"2024-03-28T13:19:00.092249+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:10:27.168742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091333+0000","last_change":"2024-03-28T13:18:59.084112+0000","last_active":"2024-03-28T13:19:00.091333+0000","last_peered":"2024-03-28T13:19:00.091333+0000","last_clean":"2024-03-28T13:19:00.091333+0000","last_became_active":"2024-03-28T13:18:59.083805+0000","last_became_peered":"2024-03-28T13:18:59.083805+0000","last_unstale":"2024-03-28T13:19:00.091333+0000","last_undegraded":"2024-03-28T13:19:00.091333+0000","last_fullsized":"2024-03-28T13:19:00.091333+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:14:07.278326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101343+0000","last_change":"2024-03-28T13:18:58.087832+0000","last_active":"2024-03-28T13:19:00.101343+0000","last_peered":"2024-03-28T13:19:00.101343+0000","last_clean":"2024-03-28T13:19:00.101343+0000","last_became_active":"2024-03-28T13:18:58.087250+0000","last_became_peered":"2024-03-28T13:18:58.087250+0000","last_unstale":"2024-03-28T13:19:00.101343+0000","last_undegraded":"2024-03-28T13:19:00.101343+0000","last_fullsized":"2024-03-28T13:19:00.101343+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:55:22.065081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.099692+0000","last_change":"2024-03-28T13:18:59.097206+0000","last_active":"2024-03-28T13:18:59.099692+0000","last_peered":"2024-03-28T13:18:59.099692+0000","last_clean":"2024-03-28T13:18:59.099692+0000","last_became_active":"2024-03-28T13:18:59.096903+0000","last_became_peered":"2024-03-28T13:18:59.096903+0000","last_unstale":"2024-03-28T13:18:59.099692+0000","last_undegraded":"2024-03-28T13:18:59.099692+0000","last_fullsized":"2024-03-28T13:18:59.099692+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:09.396769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091455+0000","last_change":"2024-03-28T13:18:58.083531+0000","last_active":"2024-03-28T13:19:00.091455+0000","last_peered":"2024-03-28T13:19:00.091455+0000","last_clean":"2024-03-28T13:19:00.091455+0000","last_became_active":"2024-03-28T13:18:58.082970+0000","last_became_peered":"2024-03-28T13:18:58.082970+0000","last_unstale":"2024-03-28T13:19:00.091455+0000","last_undegraded":"2024-03-28T13:19:00.091455+0000","last_fullsized":"2024-03-28T13:19:00.091455+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:08.383486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.100576+0000","last_change":"2024-03-28T13:18:59.098346+0000","last_active":"2024-03-28T13:18:59.100576+0000","last_peered":"2024-03-28T13:18:59.100576+0000","last_clean":"2024-03-28T13:18:59.100576+0000","last_became_active":"2024-03-28T13:18:59.098052+0000","last_became_peered":"2024-03-28T13:18:59.098052+0000","last_unstale":"2024-03-28T13:18:59.100576+0000","last_undegraded":"2024-03-28T13:18:59.100576+0000","last_fullsized":"2024-03-28T13:18:59.100576+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:20:44.200653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090424+0000","last_change":"2024-03-28T13:18:58.064634+0000","last_active":"2024-03-28T13:19:00.090424+0000","last_peered":"2024-03-28T13:19:00.090424+0000","last_clean":"2024-03-28T13:19:00.090424+0000","last_became_active":"2024-03-28T13:18:58.064194+0000","last_became_peered":"2024-03-28T13:18:58.064194+0000","last_unstale":"2024-03-28T13:19:00.090424+0000","last_undegraded":"2024-03-28T13:19:00.090424+0000","last_fullsized":"2024-03-28T13:19:00.090424+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:21:46.733401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091377+0000","last_change":"2024-03-28T13:18:59.094754+0000","last_active":"2024-03-28T13:19:00.091377+0000","last_peered":"2024-03-28T13:19:00.091377+0000","last_clean":"2024-03-28T13:19:00.091377+0000","last_became_active":"2024-03-28T13:18:59.090317+0000","last_became_peered":"2024-03-28T13:18:59.090317+0000","last_unstale":"2024-03-28T13:19:00.091377+0000","last_undegraded":"2024-03-28T13:19:00.091377+0000","last_fullsized":"2024-03-28T13:19:00.091377+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:09:17.961351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100702+0000","last_change":"2024-03-28T13:18:58.082525+0000","last_active":"2024-03-28T13:19:00.100702+0000","last_peered":"2024-03-28T13:19:00.100702+0000","last_clean":"2024-03-28T13:19:00.100702+0000","last_became_active":"2024-03-28T13:18:58.082039+0000","last_became_peered":"2024-03-28T13:18:58.082039+0000","last_unstale":"2024-03-28T13:19:00.100702+0000","last_undegraded":"2024-03-28T13:19:00.100702+0000","last_fullsized":"2024-03-28T13:19:00.100702+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:42.609780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225536+0000","last_change":"2024-03-28T13:18:59.093894+0000","last_active":"2024-03-28T13:19:01.225536+0000","last_peered":"2024-03-28T13:19:01.225536+0000","last_clean":"2024-03-28T13:19:01.225536+0000","last_became_active":"2024-03-28T13:18:59.093258+0000","last_became_peered":"2024-03-28T13:18:59.093258+0000","last_unstale":"2024-03-28T13:19:01.225536+0000","last_undegraded":"2024-03-28T13:19:01.225536+0000","last_fullsized":"2024-03-28T13:19:01.225536+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:34:15.217129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090824+0000","last_change":"2024-03-28T13:18:58.078267+0000","last_active":"2024-03-28T13:19:00.090824+0000","last_peered":"2024-03-28T13:19:00.090824+0000","last_clean":"2024-03-28T13:19:00.090824+0000","last_became_active":"2024-03-28T13:18:58.077891+0000","last_became_peered":"2024-03-28T13:18:58.077891+0000","last_unstale":"2024-03-28T13:19:00.090824+0000","last_undegraded":"2024-03-28T13:19:00.090824+0000","last_fullsized":"2024-03-28T13:19:00.090824+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:35:04.854788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.102069+0000","last_change":"2024-03-28T13:18:59.099035+0000","last_active":"2024-03-28T13:18:59.102069+0000","last_peered":"2024-03-28T13:18:59.102069+0000","last_clean":"2024-03-28T13:18:59.102069+0000","last_became_active":"2024-03-28T13:18:59.098118+0000","last_became_peered":"2024-03-28T13:18:59.098118+0000","last_unstale":"2024-03-28T13:18:59.102069+0000","last_undegraded":"2024-03-28T13:18:59.102069+0000","last_fullsized":"2024-03-28T13:18:59.102069+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:21:47.746684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092964+0000","last_change":"2024-03-28T13:18:58.076072+0000","last_active":"2024-03-28T13:19:00.092964+0000","last_peered":"2024-03-28T13:19:00.092964+0000","last_clean":"2024-03-28T13:19:00.092964+0000","last_became_active":"2024-03-28T13:18:58.075808+0000","last_became_peered":"2024-03-28T13:18:58.075808+0000","last_unstale":"2024-03-28T13:19:00.092964+0000","last_undegraded":"2024-03-28T13:19:00.092964+0000","last_fullsized":"2024-03-28T13:19:00.092964+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:11:22.068054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091695+0000","last_change":"2024-03-28T13:18:59.075982+0000","last_active":"2024-03-28T13:19:00.091695+0000","last_peered":"2024-03-28T13:19:00.091695+0000","last_clean":"2024-03-28T13:19:00.091695+0000","last_became_active":"2024-03-28T13:18:59.075606+0000","last_became_peered":"2024-03-28T13:18:59.075606+0000","last_unstale":"2024-03-28T13:19:00.091695+0000","last_undegraded":"2024-03-28T13:19:00.091695+0000","last_fullsized":"2024-03-28T13:19:00.091695+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:12:28.089180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101379+0000","last_change":"2024-03-28T13:18:58.077577+0000","last_active":"2024-03-28T13:19:00.101379+0000","last_peered":"2024-03-28T13:19:00.101379+0000","last_clean":"2024-03-28T13:19:00.101379+0000","last_became_active":"2024-03-28T13:18:58.077275+0000","last_became_peered":"2024-03-28T13:18:58.077275+0000","last_unstale":"2024-03-28T13:19:00.101379+0000","last_undegraded":"2024-03-28T13:19:00.101379+0000","last_fullsized":"2024-03-28T13:19:00.101379+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:28:48.630263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.089723+0000","last_change":"2024-03-28T13:18:59.086040+0000","last_active":"2024-03-28T13:18:59.089723+0000","last_peered":"2024-03-28T13:18:59.089723+0000","last_clean":"2024-03-28T13:18:59.089723+0000","last_became_active":"2024-03-28T13:18:59.085749+0000","last_became_peered":"2024-03-28T13:18:59.085749+0000","last_unstale":"2024-03-28T13:18:59.089723+0000","last_undegraded":"2024-03-28T13:18:59.089723+0000","last_fullsized":"2024-03-28T13:18:59.089723+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:27:52.931365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091810+0000","last_change":"2024-03-28T13:18:58.083463+0000","last_active":"2024-03-28T13:19:00.091810+0000","last_peered":"2024-03-28T13:19:00.091810+0000","last_clean":"2024-03-28T13:19:00.091810+0000","last_became_active":"2024-03-28T13:18:58.083152+0000","last_became_peered":"2024-03-28T13:18:58.083152+0000","last_unstale":"2024-03-28T13:19:00.091810+0000","last_undegraded":"2024-03-28T13:19:00.091810+0000","last_fullsized":"2024-03-28T13:19:00.091810+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:48:03.913262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091380+0000","last_change":"2024-03-28T13:18:59.081727+0000","last_active":"2024-03-28T13:19:00.091380+0000","last_peered":"2024-03-28T13:19:00.091380+0000","last_clean":"2024-03-28T13:19:00.091380+0000","last_became_active":"2024-03-28T13:18:59.081436+0000","last_became_peered":"2024-03-28T13:18:59.081436+0000","last_unstale":"2024-03-28T13:19:00.091380+0000","last_undegraded":"2024-03-28T13:19:00.091380+0000","last_fullsized":"2024-03-28T13:19:00.091380+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:55:41.223520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"104'3","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092029+0000","last_change":"2024-03-28T13:18:58.068721+0000","last_active":"2024-03-28T13:19:00.092029+0000","last_peered":"2024-03-28T13:19:00.092029+0000","last_clean":"2024-03-28T13:19:00.092029+0000","last_became_active":"2024-03-28T13:18:58.068416+0000","last_became_peered":"2024-03-28T13:18:58.068416+0000","last_unstale":"2024-03-28T13:19:00.092029+0000","last_undegraded":"2024-03-28T13:19:00.092029+0000","last_fullsized":"2024-03-28T13:19:00.092029+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:16:13.945885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.086220+0000","last_change":"2024-03-28T13:18:59.080942+0000","last_active":"2024-03-28T13:18:59.086220+0000","last_peered":"2024-03-28T13:18:59.086220+0000","last_clean":"2024-03-28T13:18:59.086220+0000","last_became_active":"2024-03-28T13:18:59.080654+0000","last_became_peered":"2024-03-28T13:18:59.080654+0000","last_unstale":"2024-03-28T13:18:59.086220+0000","last_undegraded":"2024-03-28T13:18:59.086220+0000","last_fullsized":"2024-03-28T13:18:59.086220+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:17:42.586454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"104'6","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.110193+0000","last_change":"2024-03-28T13:18:58.076129+0000","last_active":"2024-03-28T13:18:59.110193+0000","last_peered":"2024-03-28T13:18:59.110193+0000","last_clean":"2024-03-28T13:18:59.110193+0000","last_became_active":"2024-03-28T13:18:58.075809+0000","last_became_peered":"2024-03-28T13:18:58.075809+0000","last_unstale":"2024-03-28T13:18:59.110193+0000","last_undegraded":"2024-03-28T13:18:59.110193+0000","last_fullsized":"2024-03-28T13:18:59.110193+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:49:19.123418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090517+0000","last_change":"2024-03-28T13:18:59.077667+0000","last_active":"2024-03-28T13:19:00.090517+0000","last_peered":"2024-03-28T13:19:00.090517+0000","last_clean":"2024-03-28T13:19:00.090517+0000","last_became_active":"2024-03-28T13:18:59.077304+0000","last_became_peered":"2024-03-28T13:18:59.077304+0000","last_unstale":"2024-03-28T13:19:00.090517+0000","last_undegraded":"2024-03-28T13:19:00.090517+0000","last_fullsized":"2024-03-28T13:19:00.090517+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:18.635249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.094221+0000","last_change":"2024-03-28T13:18:58.077229+0000","last_active":"2024-03-28T13:19:00.094221+0000","last_peered":"2024-03-28T13:19:00.094221+0000","last_clean":"2024-03-28T13:19:00.094221+0000","last_became_active":"2024-03-28T13:18:58.076961+0000","last_became_peered":"2024-03-28T13:18:58.076961+0000","last_unstale":"2024-03-28T13:19:00.094221+0000","last_undegraded":"2024-03-28T13:19:00.094221+0000","last_fullsized":"2024-03-28T13:19:00.094221+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:20:48.852402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100869+0000","last_change":"2024-03-28T13:18:59.084784+0000","last_active":"2024-03-28T13:19:00.100869+0000","last_peered":"2024-03-28T13:19:00.100869+0000","last_clean":"2024-03-28T13:19:00.100869+0000","last_became_active":"2024-03-28T13:18:59.084518+0000","last_became_peered":"2024-03-28T13:18:59.084518+0000","last_unstale":"2024-03-28T13:19:00.100869+0000","last_undegraded":"2024-03-28T13:19:00.100869+0000","last_fullsized":"2024-03-28T13:19:00.100869+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:29.522394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090380+0000","last_change":"2024-03-28T13:18:58.083463+0000","last_active":"2024-03-28T13:19:00.090380+0000","last_peered":"2024-03-28T13:19:00.090380+0000","last_clean":"2024-03-28T13:19:00.090380+0000","last_became_active":"2024-03-28T13:18:58.083122+0000","last_became_peered":"2024-03-28T13:18:58.083122+0000","last_unstale":"2024-03-28T13:19:00.090380+0000","last_undegraded":"2024-03-28T13:19:00.090380+0000","last_fullsized":"2024-03-28T13:19:00.090380+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:20:43.187370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090923+0000","last_change":"2024-03-28T13:18:59.082251+0000","last_active":"2024-03-28T13:19:00.090923+0000","last_peered":"2024-03-28T13:19:00.090923+0000","last_clean":"2024-03-28T13:19:00.090923+0000","last_became_active":"2024-03-28T13:18:59.081951+0000","last_became_peered":"2024-03-28T13:18:59.081951+0000","last_unstale":"2024-03-28T13:19:00.090923+0000","last_undegraded":"2024-03-28T13:19:00.090923+0000","last_fullsized":"2024-03-28T13:19:00.090923+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:53.418554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087871+0000","last_change":"2024-03-28T13:18:58.066532+0000","last_active":"2024-03-28T13:18:59.087871+0000","last_peered":"2024-03-28T13:18:59.087871+0000","last_clean":"2024-03-28T13:18:59.087871+0000","last_became_active":"2024-03-28T13:18:58.066208+0000","last_became_peered":"2024-03-28T13:18:58.066208+0000","last_unstale":"2024-03-28T13:18:59.087871+0000","last_undegraded":"2024-03-28T13:18:59.087871+0000","last_fullsized":"2024-03-28T13:18:59.087871+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:58.349889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100798+0000","last_change":"2024-03-28T13:18:59.090442+0000","last_active":"2024-03-28T13:19:00.100798+0000","last_peered":"2024-03-28T13:19:00.100798+0000","last_clean":"2024-03-28T13:19:00.100798+0000","last_became_active":"2024-03-28T13:18:59.090071+0000","last_became_peered":"2024-03-28T13:18:59.090071+0000","last_unstale":"2024-03-28T13:19:00.100798+0000","last_undegraded":"2024-03-28T13:19:00.100798+0000","last_fullsized":"2024-03-28T13:19:00.100798+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:29.899339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"104'8","reported_seq":22,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.089921+0000","last_change":"2024-03-28T13:18:58.071341+0000","last_active":"2024-03-28T13:19:00.089921+0000","last_peered":"2024-03-28T13:19:00.089921+0000","last_clean":"2024-03-28T13:19:00.089921+0000","last_became_active":"2024-03-28T13:18:58.069574+0000","last_became_peered":"2024-03-28T13:18:58.069574+0000","last_unstale":"2024-03-28T13:19:00.089921+0000","last_undegraded":"2024-03-28T13:19:00.089921+0000","last_fullsized":"2024-03-28T13:19:00.089921+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:13.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094648+0000","last_change":"2024-03-28T13:18:59.093731+0000","last_active":"2024-03-28T13:18:59.094648+0000","last_peered":"2024-03-28T13:18:59.094648+0000","last_clean":"2024-03-28T13:18:59.094648+0000","last_became_active":"2024-03-28T13:18:59.093471+0000","last_became_peered":"2024-03-28T13:18:59.093471+0000","last_unstale":"2024-03-28T13:18:59.094648+0000","last_undegraded":"2024-03-28T13:18:59.094648+0000","last_fullsized":"2024-03-28T13:18:59.094648+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:18:27.237579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.108148+0000","last_change":"2024-03-28T13:18:58.080576+0000","last_active":"2024-03-28T13:18:59.108148+0000","last_peered":"2024-03-28T13:18:59.108148+0000","last_clean":"2024-03-28T13:18:59.108148+0000","last_became_active":"2024-03-28T13:18:58.080272+0000","last_became_peered":"2024-03-28T13:18:58.080272+0000","last_unstale":"2024-03-28T13:18:59.108148+0000","last_undegraded":"2024-03-28T13:18:59.108148+0000","last_fullsized":"2024-03-28T13:18:59.108148+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:13.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.094243+0000","last_change":"2024-03-28T13:18:59.092760+0000","last_active":"2024-03-28T13:18:59.094243+0000","last_peered":"2024-03-28T13:18:59.094243+0000","last_clean":"2024-03-28T13:18:59.094243+0000","last_became_active":"2024-03-28T13:18:59.092409+0000","last_became_peered":"2024-03-28T13:18:59.092409+0000","last_unstale":"2024-03-28T13:18:59.094243+0000","last_undegraded":"2024-03-28T13:18:59.094243+0000","last_fullsized":"2024-03-28T13:18:59.094243+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:19.489710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092132+0000","last_change":"2024-03-28T13:18:58.065630+0000","last_active":"2024-03-28T13:19:00.092132+0000","last_peered":"2024-03-28T13:19:00.092132+0000","last_clean":"2024-03-28T13:19:00.092132+0000","last_became_active":"2024-03-28T13:18:58.065333+0000","last_became_peered":"2024-03-28T13:18:58.065333+0000","last_unstale":"2024-03-28T13:19:00.092132+0000","last_undegraded":"2024-03-28T13:19:00.092132+0000","last_fullsized":"2024-03-28T13:19:00.092132+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:54:14.505307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.099342+0000","last_change":"2024-03-28T13:18:59.092928+0000","last_active":"2024-03-28T13:18:59.099342+0000","last_peered":"2024-03-28T13:18:59.099342+0000","last_clean":"2024-03-28T13:18:59.099342+0000","last_became_active":"2024-03-28T13:18:59.092579+0000","last_became_peered":"2024-03-28T13:18:59.092579+0000","last_unstale":"2024-03-28T13:18:59.099342+0000","last_undegraded":"2024-03-28T13:18:59.099342+0000","last_fullsized":"2024-03-28T13:18:59.099342+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:37:45.650491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093878+0000","last_change":"2024-03-28T13:18:58.075199+0000","last_active":"2024-03-28T13:19:00.093878+0000","last_peered":"2024-03-28T13:19:00.093878+0000","last_clean":"2024-03-28T13:19:00.093878+0000","last_became_active":"2024-03-28T13:18:58.073665+0000","last_became_peered":"2024-03-28T13:18:58.073665+0000","last_unstale":"2024-03-28T13:19:00.093878+0000","last_undegraded":"2024-03-28T13:19:00.093878+0000","last_fullsized":"2024-03-28T13:19:00.093878+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:08:41.324830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093360+0000","last_change":"2024-03-28T13:18:59.076634+0000","last_active":"2024-03-28T13:19:00.093360+0000","last_peered":"2024-03-28T13:19:00.093360+0000","last_clean":"2024-03-28T13:19:00.093360+0000","last_became_active":"2024-03-28T13:18:59.076295+0000","last_became_peered":"2024-03-28T13:18:59.076295+0000","last_unstale":"2024-03-28T13:19:00.093360+0000","last_undegraded":"2024-03-28T13:19:00.093360+0000","last_fullsized":"2024-03-28T13:19:00.093360+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:11:39.100208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090726+0000","last_change":"2024-03-28T13:18:58.084188+0000","last_active":"2024-03-28T13:19:00.090726+0000","last_peered":"2024-03-28T13:19:00.090726+0000","last_clean":"2024-03-28T13:19:00.090726+0000","last_became_active":"2024-03-28T13:18:58.081720+0000","last_became_peered":"2024-03-28T13:18:58.081720+0000","last_unstale":"2024-03-28T13:19:00.090726+0000","last_undegraded":"2024-03-28T13:19:00.090726+0000","last_fullsized":"2024-03-28T13:19:00.090726+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:44.810836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093931+0000","last_change":"2024-03-28T13:18:59.079466+0000","last_active":"2024-03-28T13:19:00.093931+0000","last_peered":"2024-03-28T13:19:00.093931+0000","last_clean":"2024-03-28T13:19:00.093931+0000","last_became_active":"2024-03-28T13:18:59.078752+0000","last_became_peered":"2024-03-28T13:18:59.078752+0000","last_unstale":"2024-03-28T13:19:00.093931+0000","last_undegraded":"2024-03-28T13:19:00.093931+0000","last_fullsized":"2024-03-28T13:19:00.093931+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:50.468775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.066401+0000","last_change":"2024-03-28T13:18:58.079436+0000","last_active":"2024-03-28T13:18:59.066401+0000","last_peered":"2024-03-28T13:18:59.066401+0000","last_clean":"2024-03-28T13:18:59.066401+0000","last_became_active":"2024-03-28T13:18:58.079066+0000","last_became_peered":"2024-03-28T13:18:58.079066+0000","last_unstale":"2024-03-28T13:18:59.066401+0000","last_undegraded":"2024-03-28T13:18:59.066401+0000","last_fullsized":"2024-03-28T13:18:59.066401+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:08.383486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090744+0000","last_change":"2024-03-28T13:18:59.080751+0000","last_active":"2024-03-28T13:19:00.090744+0000","last_peered":"2024-03-28T13:19:00.090744+0000","last_clean":"2024-03-28T13:19:00.090744+0000","last_became_active":"2024-03-28T13:18:59.080449+0000","last_became_peered":"2024-03-28T13:18:59.080449+0000","last_unstale":"2024-03-28T13:19:00.090744+0000","last_undegraded":"2024-03-28T13:19:00.090744+0000","last_fullsized":"2024-03-28T13:19:00.090744+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:18:33.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.075738+0000","last_change":"2024-03-28T13:18:58.079341+0000","last_active":"2024-03-28T13:18:59.075738+0000","last_peered":"2024-03-28T13:18:59.075738+0000","last_clean":"2024-03-28T13:18:59.075738+0000","last_became_active":"2024-03-28T13:18:58.079008+0000","last_became_peered":"2024-03-28T13:18:58.079008+0000","last_unstale":"2024-03-28T13:18:59.075738+0000","last_undegraded":"2024-03-28T13:18:59.075738+0000","last_fullsized":"2024-03-28T13:18:59.075738+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:21:46.733401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.086441+0000","last_change":"2024-03-28T13:18:59.080827+0000","last_active":"2024-03-28T13:18:59.086441+0000","last_peered":"2024-03-28T13:18:59.086441+0000","last_clean":"2024-03-28T13:18:59.086441+0000","last_became_active":"2024-03-28T13:18:59.080524+0000","last_became_peered":"2024-03-28T13:18:59.080524+0000","last_unstale":"2024-03-28T13:18:59.086441+0000","last_undegraded":"2024-03-28T13:18:59.086441+0000","last_fullsized":"2024-03-28T13:18:59.086441+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:43:24.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100342+0000","last_change":"2024-03-28T13:18:58.090817+0000","last_active":"2024-03-28T13:19:00.100342+0000","last_peered":"2024-03-28T13:19:00.100342+0000","last_clean":"2024-03-28T13:19:00.100342+0000","last_became_active":"2024-03-28T13:18:58.090469+0000","last_became_peered":"2024-03-28T13:18:58.090469+0000","last_unstale":"2024-03-28T13:19:00.100342+0000","last_undegraded":"2024-03-28T13:19:00.100342+0000","last_fullsized":"2024-03-28T13:19:00.100342+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:52.858677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.094597+0000","last_change":"2024-03-28T13:18:59.071403+0000","last_active":"2024-03-28T13:19:00.094597+0000","last_peered":"2024-03-28T13:19:00.094597+0000","last_clean":"2024-03-28T13:19:00.094597+0000","last_became_active":"2024-03-28T13:18:59.071095+0000","last_became_peered":"2024-03-28T13:18:59.071095+0000","last_unstale":"2024-03-28T13:19:00.094597+0000","last_undegraded":"2024-03-28T13:19:00.094597+0000","last_fullsized":"2024-03-28T13:19:00.094597+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:09:52.654125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"104'2","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.105978+0000","last_change":"2024-03-28T13:18:58.071038+0000","last_active":"2024-03-28T13:18:59.105978+0000","last_peered":"2024-03-28T13:18:59.105978+0000","last_clean":"2024-03-28T13:18:59.105978+0000","last_became_active":"2024-03-28T13:18:58.069807+0000","last_became_peered":"2024-03-28T13:18:58.069807+0000","last_unstale":"2024-03-28T13:18:59.105978+0000","last_undegraded":"2024-03-28T13:18:59.105978+0000","last_fullsized":"2024-03-28T13:18:59.105978+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:17:17.825753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.095041+0000","last_change":"2024-03-28T13:18:59.076727+0000","last_active":"2024-03-28T13:19:00.095041+0000","last_peered":"2024-03-28T13:19:00.095041+0000","last_clean":"2024-03-28T13:19:00.095041+0000","last_became_active":"2024-03-28T13:18:59.076415+0000","last_became_peered":"2024-03-28T13:18:59.076415+0000","last_unstale":"2024-03-28T13:19:00.095041+0000","last_undegraded":"2024-03-28T13:19:00.095041+0000","last_fullsized":"2024-03-28T13:19:00.095041+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:56:27.830992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"105'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.095140+0000","last_change":"2024-03-28T13:18:58.086609+0000","last_active":"2024-03-28T13:19:00.095140+0000","last_peered":"2024-03-28T13:19:00.095140+0000","last_clean":"2024-03-28T13:19:00.095140+0000","last_became_active":"2024-03-28T13:18:58.086223+0000","last_became_peered":"2024-03-28T13:18:58.086223+0000","last_unstale":"2024-03-28T13:19:00.095140+0000","last_undegraded":"2024-03-28T13:19:00.095140+0000","last_fullsized":"2024-03-28T13:19:00.095140+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:17.621966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101315+0000","last_change":"2024-03-28T13:18:59.098557+0000","last_active":"2024-03-28T13:19:00.101315+0000","last_peered":"2024-03-28T13:19:00.101315+0000","last_clean":"2024-03-28T13:19:00.101315+0000","last_became_active":"2024-03-28T13:18:59.098274+0000","last_became_peered":"2024-03-28T13:18:59.098274+0000","last_unstale":"2024-03-28T13:19:00.101315+0000","last_undegraded":"2024-03-28T13:19:00.101315+0000","last_fullsized":"2024-03-28T13:19:00.101315+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:20.977905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.094124+0000","last_change":"2024-03-28T13:18:58.072039+0000","last_active":"2024-03-28T13:19:00.094124+0000","last_peered":"2024-03-28T13:19:00.094124+0000","last_clean":"2024-03-28T13:19:00.094124+0000","last_became_active":"2024-03-28T13:18:58.071683+0000","last_became_peered":"2024-03-28T13:18:58.071683+0000","last_unstale":"2024-03-28T13:19:00.094124+0000","last_undegraded":"2024-03-28T13:19:00.094124+0000","last_fullsized":"2024-03-28T13:19:00.094124+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:47.118216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091918+0000","last_change":"2024-03-28T13:18:59.087471+0000","last_active":"2024-03-28T13:19:00.091918+0000","last_peered":"2024-03-28T13:19:00.091918+0000","last_clean":"2024-03-28T13:19:00.091918+0000","last_became_active":"2024-03-28T13:18:59.085466+0000","last_became_peered":"2024-03-28T13:18:59.085466+0000","last_unstale":"2024-03-28T13:19:00.091918+0000","last_undegraded":"2024-03-28T13:19:00.091918+0000","last_fullsized":"2024-03-28T13:19:00.091918+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:30:44.259677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.067207+0000","last_change":"2024-03-28T13:18:58.068025+0000","last_active":"2024-03-28T13:18:59.067207+0000","last_peered":"2024-03-28T13:18:59.067207+0000","last_clean":"2024-03-28T13:18:59.067207+0000","last_became_active":"2024-03-28T13:18:58.067671+0000","last_became_peered":"2024-03-28T13:18:58.067671+0000","last_unstale":"2024-03-28T13:18:59.067207+0000","last_undegraded":"2024-03-28T13:18:59.067207+0000","last_fullsized":"2024-03-28T13:18:59.067207+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:28.509110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093804+0000","last_change":"2024-03-28T13:18:59.073922+0000","last_active":"2024-03-28T13:19:00.093804+0000","last_peered":"2024-03-28T13:19:00.093804+0000","last_clean":"2024-03-28T13:19:00.093804+0000","last_became_active":"2024-03-28T13:18:59.073566+0000","last_became_peered":"2024-03-28T13:18:59.073566+0000","last_unstale":"2024-03-28T13:19:00.093804+0000","last_undegraded":"2024-03-28T13:19:00.093804+0000","last_fullsized":"2024-03-28T13:19:00.093804+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:38:33.010172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101273+0000","last_change":"2024-03-28T13:18:58.085176+0000","last_active":"2024-03-28T13:19:00.101273+0000","last_peered":"2024-03-28T13:19:00.101273+0000","last_clean":"2024-03-28T13:19:00.101273+0000","last_became_active":"2024-03-28T13:18:58.084898+0000","last_became_peered":"2024-03-28T13:18:58.084898+0000","last_unstale":"2024-03-28T13:19:00.101273+0000","last_undegraded":"2024-03-28T13:19:00.101273+0000","last_fullsized":"2024-03-28T13:19:00.101273+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:13:25.784483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091562+0000","last_change":"2024-03-28T13:18:59.076130+0000","last_active":"2024-03-28T13:19:00.091562+0000","last_peered":"2024-03-28T13:19:00.091562+0000","last_clean":"2024-03-28T13:19:00.091562+0000","last_became_active":"2024-03-28T13:18:59.075785+0000","last_became_peered":"2024-03-28T13:18:59.075785+0000","last_unstale":"2024-03-28T13:19:00.091562+0000","last_undegraded":"2024-03-28T13:19:00.091562+0000","last_fullsized":"2024-03-28T13:19:00.091562+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:31.836556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.065847+0000","last_change":"2024-03-28T13:18:58.073984+0000","last_active":"2024-03-28T13:18:59.065847+0000","last_peered":"2024-03-28T13:18:59.065847+0000","last_clean":"2024-03-28T13:18:59.065847+0000","last_became_active":"2024-03-28T13:18:58.073520+0000","last_became_peered":"2024-03-28T13:18:58.073520+0000","last_unstale":"2024-03-28T13:18:59.065847+0000","last_undegraded":"2024-03-28T13:18:59.065847+0000","last_fullsized":"2024-03-28T13:18:59.065847+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:28.886056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093125+0000","last_change":"2024-03-28T13:18:59.078032+0000","last_active":"2024-03-28T13:19:00.093125+0000","last_peered":"2024-03-28T13:19:00.093125+0000","last_clean":"2024-03-28T13:19:00.093125+0000","last_became_active":"2024-03-28T13:18:59.077665+0000","last_became_peered":"2024-03-28T13:18:59.077665+0000","last_unstale":"2024-03-28T13:19:00.093125+0000","last_undegraded":"2024-03-28T13:19:00.093125+0000","last_fullsized":"2024-03-28T13:19:00.093125+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:14:14.261611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.226316+0000","last_change":"2024-03-28T13:18:58.083533+0000","last_active":"2024-03-28T13:19:01.226316+0000","last_peered":"2024-03-28T13:19:01.226316+0000","last_clean":"2024-03-28T13:19:01.226316+0000","last_became_active":"2024-03-28T13:18:58.083220+0000","last_became_peered":"2024-03-28T13:18:58.083220+0000","last_unstale":"2024-03-28T13:19:01.226316+0000","last_undegraded":"2024-03-28T13:19:01.226316+0000","last_fullsized":"2024-03-28T13:19:01.226316+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:44:18.338917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.098913+0000","last_change":"2024-03-28T13:18:59.091706+0000","last_active":"2024-03-28T13:18:59.098913+0000","last_peered":"2024-03-28T13:18:59.098913+0000","last_clean":"2024-03-28T13:18:59.098913+0000","last_became_active":"2024-03-28T13:18:59.091410+0000","last_became_peered":"2024-03-28T13:18:59.091410+0000","last_unstale":"2024-03-28T13:18:59.098913+0000","last_undegraded":"2024-03-28T13:18:59.098913+0000","last_fullsized":"2024-03-28T13:18:59.098913+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:15.681052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100688+0000","last_change":"2024-03-28T13:18:58.086792+0000","last_active":"2024-03-28T13:19:00.100688+0000","last_peered":"2024-03-28T13:19:00.100688+0000","last_clean":"2024-03-28T13:19:00.100688+0000","last_became_active":"2024-03-28T13:18:58.086544+0000","last_became_peered":"2024-03-28T13:18:58.086544+0000","last_unstale":"2024-03-28T13:19:00.100688+0000","last_undegraded":"2024-03-28T13:19:00.100688+0000","last_fullsized":"2024-03-28T13:19:00.100688+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:51:50.282781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101376+0000","last_change":"2024-03-28T13:18:59.088616+0000","last_active":"2024-03-28T13:19:00.101376+0000","last_peered":"2024-03-28T13:19:00.101376+0000","last_clean":"2024-03-28T13:19:00.101376+0000","last_became_active":"2024-03-28T13:18:59.088292+0000","last_became_peered":"2024-03-28T13:18:59.088292+0000","last_unstale":"2024-03-28T13:19:00.101376+0000","last_undegraded":"2024-03-28T13:19:00.101376+0000","last_fullsized":"2024-03-28T13:19:00.101376+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:43:24.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"104'1","reported_seq":13,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.107904+0000","last_change":"2024-03-28T13:18:58.075299+0000","last_active":"2024-03-28T13:18:59.107904+0000","last_peered":"2024-03-28T13:18:59.107904+0000","last_clean":"2024-03-28T13:18:59.107904+0000","last_became_active":"2024-03-28T13:18:58.075035+0000","last_became_peered":"2024-03-28T13:18:58.075035+0000","last_unstale":"2024-03-28T13:18:59.107904+0000","last_undegraded":"2024-03-28T13:18:59.107904+0000","last_fullsized":"2024-03-28T13:18:59.107904+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:39.277367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":320,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-03-28T13:18:59.087250+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:18:59.087250+0000","last_peered":"2024-03-28T13:18:59.087250+0000","last_clean":"2024-03-28T13:18:59.087250+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:18:59.087250+0000","last_undegraded":"2024-03-28T13:18:59.087250+0000","last_fullsized":"2024-03-28T13:18:59.087250+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476856,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":550446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1277952,"data_stored":550322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33028,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338812,"statfs":{"total":96636764160,"available":96602943488,"internally_reserved":0,"allocated":1888256,"data_stored":1142344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1273856,"data_stored":550252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509558,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":556552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509559,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1900544,"data_stored":1148588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509559,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":557134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542263,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":557134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:19:08.875 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:19:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.104+0000 7f6fabc00640 1 Processor -- start 2024-03-28T13:19:09.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.104+0000 7f6fabc00640 1 -- start start 2024-03-28T13:19:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.105+0000 7f6fabc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6fa415b310 0x7f6fa40727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.106+0000 7f6fabc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6fa40d0780 0x7f6fa40d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.106+0000 7f6fabc00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fa415b6f0 con 0x7f6fa40b4f90 2024-03-28T13:19:09.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.106+0000 7f6fabc00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa4072e80 con 0x7f6fa415b310 2024-03-28T13:19:09.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.106+0000 7f6fabc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f6fa4073160 con 0x7f6fa40d0780 2024-03-28T13:19:09.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.106+0000 7f6fa9200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6fa415b310 0x7f6fa40727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.106+0000 7f6fa3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6fa40b4f90 0x7f6fa415f7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45068/0 (socket says 172.21.15.19:45068) 2024-03-28T13:19:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.106+0000 7f6fa3e00640 1 -- 172.21.15.19:0/1345267703 learned_addr learned my addr 172.21.15.19:0/1345267703 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.106+0000 7f6fa9c00640 1 --2- 172.21.15.19:0/1345267703 >> v2:172.21.15.188:3300/0 conn(0x7f6fa40d0780 0x7f6fa40d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.107+0000 7f6fa9200640 1 -- 172.21.15.19:0/1345267703 >> v2:172.21.15.188:3300/0 conn(0x7f6fa40d0780 msgr2=0x7f6fa40d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.107+0000 7f6fa9200640 1 --2- 172.21.15.19:0/1345267703 >> v2:172.21.15.188:3300/0 conn(0x7f6fa40d0780 0x7f6fa40d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.107+0000 7f6fa9200640 1 -- 172.21.15.19:0/1345267703 >> v1:172.21.15.19:6789/0 conn(0x7f6fa40b4f90 legacy=0x7f6fa415f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.107+0000 7f6fa9200640 1 -- 172.21.15.19:0/1345267703 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa415efe0 con 0x7f6fa415b310 2024-03-28T13:19:09.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.107+0000 7f6fa9c00640 1 --2- 172.21.15.19:0/1345267703 >> v2:172.21.15.188:3300/0 conn(0x7f6fa40d0780 0x7f6fa40d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:19:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.108+0000 7f6fa9200640 1 --2- 172.21.15.19:0/1345267703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa415b310 0x7f6fa40727f0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40ad7c0 tx=0x7f6f9402bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8426f352254136 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.108+0000 7f6fa3400640 1 -- 172.21.15.19:0/1345267703 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94039070 con 0x7f6fa415b310 2024-03-28T13:19:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.108+0000 7f6fa3400640 1 -- 172.21.15.19:0/1345267703 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f9402cb50 con 0x7f6fa415b310 2024-03-28T13:19:09.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.108+0000 7f6fa3400640 1 -- 172.21.15.19:0/1345267703 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94034440 con 0x7f6fa415b310 2024-03-28T13:19:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.109+0000 7f6fabc00640 1 -- 172.21.15.19:0/1345267703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa415b310 msgr2=0x7f6fa40727f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.109+0000 7f6fabc00640 1 --2- 172.21.15.19:0/1345267703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa415b310 0x7f6fa40727f0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40ad7c0 tx=0x7f6f9402bda0 comp rx=0 tx=0).stop 2024-03-28T13:19:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.109+0000 7f6fabc00640 1 -- 172.21.15.19:0/1345267703 shutdown_connections 2024-03-28T13:19:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.109+0000 7f6fabc00640 1 --2- 172.21.15.19:0/1345267703 >> v2:172.21.15.188:3300/0 conn(0x7f6fa40d0780 0x7f6fa40d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.109+0000 7f6fabc00640 1 --2- 172.21.15.19:0/1345267703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa415b310 0x7f6fa40727f0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.109+0000 7f6fabc00640 1 -- 172.21.15.19:0/1345267703 >> 172.21.15.19:0/1345267703 conn(0x7f6fa406b990 msgr2=0x7f6fa406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.109+0000 7f6fabc00640 1 -- 172.21.15.19:0/1345267703 shutdown_connections 2024-03-28T13:19:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.109+0000 7f6fabc00640 1 -- 172.21.15.19:0/1345267703 wait complete. 2024-03-28T13:19:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.111+0000 7f6fabc00640 1 Processor -- start 2024-03-28T13:19:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.111+0000 7f6fabc00640 1 -- start start 2024-03-28T13:19:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.112+0000 7f6fabc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa40b4f90 0x7f6fa414ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.112+0000 7f6fabc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa40d0780 0x7f6fa41512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.112+0000 7f6fabc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa415b310 0x7f6fa4159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.112+0000 7f6fabc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa4073440 con 0x7f6fa40d0780 2024-03-28T13:19:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.112+0000 7f6fabc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa40b53f0 con 0x7f6fa415b310 2024-03-28T13:19:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.112+0000 7f6fabc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa40adc80 con 0x7f6fa40b4f90 2024-03-28T13:19:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.112+0000 7f6fa3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa40d0780 0x7f6fa41512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.112+0000 7f6fa3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa40d0780 0x7f6fa41512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58458/0 (socket says 172.21.15.19:58458) 2024-03-28T13:19:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.112+0000 7f6fa3e00640 1 -- 172.21.15.19:0/2927963749 learned_addr learned my addr 172.21.15.19:0/2927963749 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.113+0000 7f6fa9c00640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa415b310 0x7f6fa4159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.113+0000 7f6fa9200640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa40b4f90 0x7f6fa414ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.113+0000 7f6fa3e00640 1 -- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa40b4f90 msgr2=0x7f6fa414ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.113+0000 7f6fa3e00640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa40b4f90 0x7f6fa414ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.113+0000 7f6fa3e00640 1 -- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa415b310 msgr2=0x7f6fa4159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.113+0000 7f6fa3e00640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa415b310 0x7f6fa4159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.113+0000 7f6fa3e00640 1 -- 172.21.15.19:0/2927963749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa41f5950 con 0x7f6fa40d0780 2024-03-28T13:19:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.113+0000 7f6fa9c00640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa415b310 0x7f6fa4159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.113+0000 7f6fa9200640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa40b4f90 0x7f6fa414ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.113+0000 7f6fa3e00640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa40d0780 0x7f6fa41512c0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000c400 tx=0x7f6f9000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.114+0000 7f6fa0c00640 1 -- 172.21.15.19:0/2927963749 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90017070 con 0x7f6fa40d0780 2024-03-28T13:19:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.114+0000 7f6fa0c00640 1 -- 172.21.15.19:0/2927963749 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f90005ce0 con 0x7f6fa40d0780 2024-03-28T13:19:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.114+0000 7f6fa0c00640 1 -- 172.21.15.19:0/2927963749 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f900072f0 con 0x7f6fa40d0780 2024-03-28T13:19:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.114+0000 7f6fabc00640 1 -- 172.21.15.19:0/2927963749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa41f5c30 con 0x7f6fa40d0780 2024-03-28T13:19:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.114+0000 7f6fabc00640 1 -- 172.21.15.19:0/2927963749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa415bba0 con 0x7f6fa40d0780 2024-03-28T13:19:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.115+0000 7f6fabc00640 1 -- 172.21.15.19:0/2927963749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa4070190 con 0x7f6fa40d0780 2024-03-28T13:19:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.116+0000 7f6fa0c00640 1 -- 172.21.15.19:0/2927963749 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6f9001a070 con 0x7f6fa40d0780 2024-03-28T13:19:09.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.118+0000 7f6fa0c00640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6f8406c530 0x7f6f8406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:09.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.118+0000 7f6fa0c00640 1 -- 172.21.15.19:0/2927963749 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f6f9008aac0 con 0x7f6fa40d0780 2024-03-28T13:19:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.118+0000 7f6fa9200640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6f8406c530 0x7f6f8406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.120+0000 7f6fa9200640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6f8406c530 0x7f6f8406e9f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6f94036000 tx=0x7f6f94030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:09.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.122+0000 7f6fa0c00640 1 -- 172.21.15.19:0/2927963749 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6fa4070190 con 0x7f6fa40d0780 2024-03-28T13:19:09.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.324+0000 7f6fabc00640 1 -- 172.21.15.19:0/2927963749 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6fa40685c0 con 0x7f6f8406c530 2024-03-28T13:19:09.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.343+0000 7f6fa0c00640 1 -- 172.21.15.19:0/2927963749 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252460 (secure 0 0 0) 0x7f6fa40685c0 con 0x7f6f8406c530 2024-03-28T13:19:09.346 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:09.348 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:19:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.348+0000 7f6fabc00640 1 -- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6f8406c530 msgr2=0x7f6f8406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.348+0000 7f6fabc00640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6f8406c530 0x7f6f8406e9f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6f94036000 tx=0x7f6f94030000 comp rx=0 tx=0).stop 2024-03-28T13:19:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.348+0000 7f6fabc00640 1 -- 172.21.15.19:0/2927963749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa40d0780 msgr2=0x7f6fa41512c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.348+0000 7f6fabc00640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa40d0780 0x7f6fa41512c0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000c400 tx=0x7f6f9000c8c0 comp rx=0 tx=0).stop 2024-03-28T13:19:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.348+0000 7f6fabc00640 1 -- 172.21.15.19:0/2927963749 shutdown_connections 2024-03-28T13:19:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.348+0000 7f6fabc00640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6f8406c530 0x7f6f8406e9f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.348+0000 7f6fabc00640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fa415b310 0x7f6fa4159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.348+0000 7f6fabc00640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa40d0780 0x7f6fa41512c0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.348+0000 7f6fabc00640 1 --2- 172.21.15.19:0/2927963749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fa40b4f90 0x7f6fa414ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.348+0000 7f6fabc00640 1 -- 172.21.15.19:0/2927963749 >> 172.21.15.19:0/2927963749 conn(0x7f6fa406b990 msgr2=0x7f6fa40afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.348+0000 7f6fabc00640 1 -- 172.21.15.19:0/2927963749 shutdown_connections 2024-03-28T13:19:09.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.348+0000 7f6fabc00640 1 -- 172.21.15.19:0/2927963749 wait complete. 2024-03-28T13:19:09.367 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-03-28T13:19:08.199722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260728,"kb_used_data":11272,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713992,"statfs":{"total":773094113280,"available":772827127808,"internally_reserved":0,"allocated":11542528,"data_stored":5637972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.146875"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091482+0000","last_change":"2024-03-28T13:18:59.085702+0000","last_active":"2024-03-28T13:19:00.091482+0000","last_peered":"2024-03-28T13:19:00.091482+0000","last_clean":"2024-03-28T13:19:00.091482+0000","last_became_active":"2024-03-28T13:18:59.085413+0000","last_became_peered":"2024-03-28T13:18:59.085413+0000","last_unstale":"2024-03-28T13:19:00.091482+0000","last_undegraded":"2024-03-28T13:19:00.091482+0000","last_fullsized":"2024-03-28T13:19:00.091482+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:04:28.366038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091242+0000","last_change":"2024-03-28T13:18:59.090611+0000","last_active":"2024-03-28T13:19:00.091242+0000","last_peered":"2024-03-28T13:19:00.091242+0000","last_clean":"2024-03-28T13:19:00.091242+0000","last_became_active":"2024-03-28T13:18:59.089972+0000","last_became_peered":"2024-03-28T13:18:59.089972+0000","last_unstale":"2024-03-28T13:19:00.091242+0000","last_undegraded":"2024-03-28T13:19:00.091242+0000","last_fullsized":"2024-03-28T13:19:00.091242+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:45:40.743749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101633+0000","last_change":"2024-03-28T13:18:59.098655+0000","last_active":"2024-03-28T13:19:00.101633+0000","last_peered":"2024-03-28T13:19:00.101633+0000","last_clean":"2024-03-28T13:19:00.101633+0000","last_became_active":"2024-03-28T13:18:59.098399+0000","last_became_peered":"2024-03-28T13:18:59.098399+0000","last_unstale":"2024-03-28T13:19:00.101633+0000","last_undegraded":"2024-03-28T13:19:00.101633+0000","last_fullsized":"2024-03-28T13:19:00.101633+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:17:42.586454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090818+0000","last_change":"2024-03-28T13:18:59.074664+0000","last_active":"2024-03-28T13:19:00.090818+0000","last_peered":"2024-03-28T13:19:00.090818+0000","last_clean":"2024-03-28T13:19:00.090818+0000","last_became_active":"2024-03-28T13:18:59.073993+0000","last_became_peered":"2024-03-28T13:18:59.073993+0000","last_unstale":"2024-03-28T13:19:00.090818+0000","last_undegraded":"2024-03-28T13:19:00.090818+0000","last_fullsized":"2024-03-28T13:19:00.090818+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:44:49.466448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091305+0000","last_change":"2024-03-28T13:18:59.090589+0000","last_active":"2024-03-28T13:19:00.091305+0000","last_peered":"2024-03-28T13:19:00.091305+0000","last_clean":"2024-03-28T13:19:00.091305+0000","last_became_active":"2024-03-28T13:18:59.089952+0000","last_became_peered":"2024-03-28T13:18:59.089952+0000","last_unstale":"2024-03-28T13:19:00.091305+0000","last_undegraded":"2024-03-28T13:19:00.091305+0000","last_fullsized":"2024-03-28T13:19:00.091305+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:54:09.506244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091182+0000","last_change":"2024-03-28T13:18:59.085728+0000","last_active":"2024-03-28T13:19:00.091182+0000","last_peered":"2024-03-28T13:19:00.091182+0000","last_clean":"2024-03-28T13:19:00.091182+0000","last_became_active":"2024-03-28T13:18:59.085447+0000","last_became_peered":"2024-03-28T13:18:59.085447+0000","last_unstale":"2024-03-28T13:19:00.091182+0000","last_undegraded":"2024-03-28T13:19:00.091182+0000","last_fullsized":"2024-03-28T13:19:00.091182+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:40:16.716159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091117+0000","last_change":"2024-03-28T13:18:59.090899+0000","last_active":"2024-03-28T13:19:00.091117+0000","last_peered":"2024-03-28T13:19:00.091117+0000","last_clean":"2024-03-28T13:19:00.091117+0000","last_became_active":"2024-03-28T13:18:59.086628+0000","last_became_peered":"2024-03-28T13:18:59.086628+0000","last_unstale":"2024-03-28T13:19:00.091117+0000","last_undegraded":"2024-03-28T13:19:00.091117+0000","last_fullsized":"2024-03-28T13:19:00.091117+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:05:31.597035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100303+0000","last_change":"2024-03-28T13:18:59.082961+0000","last_active":"2024-03-28T13:19:00.100303+0000","last_peered":"2024-03-28T13:19:00.100303+0000","last_clean":"2024-03-28T13:19:00.100303+0000","last_became_active":"2024-03-28T13:18:59.082608+0000","last_became_peered":"2024-03-28T13:18:59.082608+0000","last_unstale":"2024-03-28T13:19:00.100303+0000","last_undegraded":"2024-03-28T13:19:00.100303+0000","last_fullsized":"2024-03-28T13:19:00.100303+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:45.824120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092358+0000","last_change":"2024-03-28T13:18:59.084837+0000","last_active":"2024-03-28T13:19:00.092358+0000","last_peered":"2024-03-28T13:19:00.092358+0000","last_clean":"2024-03-28T13:19:00.092358+0000","last_became_active":"2024-03-28T13:18:59.083085+0000","last_became_peered":"2024-03-28T13:18:59.083085+0000","last_unstale":"2024-03-28T13:19:00.092358+0000","last_undegraded":"2024-03-28T13:19:00.092358+0000","last_fullsized":"2024-03-28T13:19:00.092358+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:16:46.004371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.938923+0000","last_change":"2024-03-28T13:18:59.088878+0000","last_active":"2024-03-28T13:19:00.938923+0000","last_peered":"2024-03-28T13:19:00.938923+0000","last_clean":"2024-03-28T13:19:00.938923+0000","last_became_active":"2024-03-28T13:18:59.088493+0000","last_became_peered":"2024-03-28T13:18:59.088493+0000","last_unstale":"2024-03-28T13:19:00.938923+0000","last_undegraded":"2024-03-28T13:19:00.938923+0000","last_fullsized":"2024-03-28T13:19:00.938923+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:20.977905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091805+0000","last_change":"2024-03-28T13:18:59.100683+0000","last_active":"2024-03-28T13:19:00.091805+0000","last_peered":"2024-03-28T13:19:00.091805+0000","last_clean":"2024-03-28T13:19:00.091805+0000","last_became_active":"2024-03-28T13:18:59.099214+0000","last_became_peered":"2024-03-28T13:18:59.099214+0000","last_unstale":"2024-03-28T13:19:00.091805+0000","last_undegraded":"2024-03-28T13:19:00.091805+0000","last_fullsized":"2024-03-28T13:19:00.091805+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:02:41.892054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.224601+0000","last_change":"2024-03-28T13:18:59.090740+0000","last_active":"2024-03-28T13:19:01.224601+0000","last_peered":"2024-03-28T13:19:01.224601+0000","last_clean":"2024-03-28T13:19:01.224601+0000","last_became_active":"2024-03-28T13:18:59.090473+0000","last_became_peered":"2024-03-28T13:18:59.090473+0000","last_unstale":"2024-03-28T13:19:01.224601+0000","last_undegraded":"2024-03-28T13:19:01.224601+0000","last_fullsized":"2024-03-28T13:19:01.224601+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:09.396769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.226045+0000","last_change":"2024-03-28T13:18:59.088146+0000","last_active":"2024-03-28T13:19:01.226045+0000","last_peered":"2024-03-28T13:19:01.226045+0000","last_clean":"2024-03-28T13:19:01.226045+0000","last_became_active":"2024-03-28T13:18:59.087788+0000","last_became_peered":"2024-03-28T13:18:59.087788+0000","last_unstale":"2024-03-28T13:19:01.226045+0000","last_undegraded":"2024-03-28T13:19:01.226045+0000","last_fullsized":"2024-03-28T13:19:01.226045+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:31:19.713409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.089797+0000","last_change":"2024-03-28T13:18:59.080979+0000","last_active":"2024-03-28T13:19:00.089797+0000","last_peered":"2024-03-28T13:19:00.089797+0000","last_clean":"2024-03-28T13:19:00.089797+0000","last_became_active":"2024-03-28T13:18:59.080676+0000","last_became_peered":"2024-03-28T13:18:59.080676+0000","last_unstale":"2024-03-28T13:19:00.089797+0000","last_undegraded":"2024-03-28T13:19:00.089797+0000","last_fullsized":"2024-03-28T13:19:00.089797+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:20:49.865685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090048+0000","last_change":"2024-03-28T13:18:59.077084+0000","last_active":"2024-03-28T13:19:00.090048+0000","last_peered":"2024-03-28T13:19:00.090048+0000","last_clean":"2024-03-28T13:19:00.090048+0000","last_became_active":"2024-03-28T13:18:59.076136+0000","last_became_peered":"2024-03-28T13:18:59.076136+0000","last_unstale":"2024-03-28T13:19:00.090048+0000","last_undegraded":"2024-03-28T13:19:00.090048+0000","last_fullsized":"2024-03-28T13:19:00.090048+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:08:42.338113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090808+0000","last_change":"2024-03-28T13:18:59.074866+0000","last_active":"2024-03-28T13:19:00.090808+0000","last_peered":"2024-03-28T13:19:00.090808+0000","last_clean":"2024-03-28T13:19:00.090808+0000","last_became_active":"2024-03-28T13:18:59.074514+0000","last_became_peered":"2024-03-28T13:18:59.074514+0000","last_unstale":"2024-03-28T13:19:00.090808+0000","last_undegraded":"2024-03-28T13:19:00.090808+0000","last_fullsized":"2024-03-28T13:19:00.090808+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:10:44.680345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.094816+0000","last_change":"2024-03-28T13:18:59.084712+0000","last_active":"2024-03-28T13:19:00.094816+0000","last_peered":"2024-03-28T13:19:00.094816+0000","last_clean":"2024-03-28T13:19:00.094816+0000","last_became_active":"2024-03-28T13:18:59.084435+0000","last_became_peered":"2024-03-28T13:18:59.084435+0000","last_unstale":"2024-03-28T13:19:00.094816+0000","last_undegraded":"2024-03-28T13:19:00.094816+0000","last_fullsized":"2024-03-28T13:19:00.094816+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:44:01.634969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091450+0000","last_change":"2024-03-28T13:18:59.083710+0000","last_active":"2024-03-28T13:19:00.091450+0000","last_peered":"2024-03-28T13:19:00.091450+0000","last_clean":"2024-03-28T13:19:00.091450+0000","last_became_active":"2024-03-28T13:18:59.083366+0000","last_became_peered":"2024-03-28T13:18:59.083366+0000","last_unstale":"2024-03-28T13:19:00.091450+0000","last_undegraded":"2024-03-28T13:19:00.091450+0000","last_fullsized":"2024-03-28T13:19:00.091450+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:00:07.119299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093855+0000","last_change":"2024-03-28T13:18:59.076959+0000","last_active":"2024-03-28T13:19:00.093855+0000","last_peered":"2024-03-28T13:19:00.093855+0000","last_clean":"2024-03-28T13:19:00.093855+0000","last_became_active":"2024-03-28T13:18:59.075601+0000","last_became_peered":"2024-03-28T13:18:59.075601+0000","last_unstale":"2024-03-28T13:19:00.093855+0000","last_undegraded":"2024-03-28T13:19:00.093855+0000","last_fullsized":"2024-03-28T13:19:00.093855+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:43:21.159934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.224666+0000","last_change":"2024-03-28T13:18:59.087556+0000","last_active":"2024-03-28T13:19:01.224666+0000","last_peered":"2024-03-28T13:19:01.224666+0000","last_clean":"2024-03-28T13:19:01.224666+0000","last_became_active":"2024-03-28T13:18:59.087183+0000","last_became_peered":"2024-03-28T13:18:59.087183+0000","last_unstale":"2024-03-28T13:19:01.224666+0000","last_undegraded":"2024-03-28T13:19:01.224666+0000","last_fullsized":"2024-03-28T13:19:01.224666+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:29.899339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100753+0000","last_change":"2024-03-28T13:18:59.085370+0000","last_active":"2024-03-28T13:19:00.100753+0000","last_peered":"2024-03-28T13:19:00.100753+0000","last_clean":"2024-03-28T13:19:00.100753+0000","last_became_active":"2024-03-28T13:18:59.084826+0000","last_became_peered":"2024-03-28T13:18:59.084826+0000","last_unstale":"2024-03-28T13:19:00.100753+0000","last_undegraded":"2024-03-28T13:19:00.100753+0000","last_fullsized":"2024-03-28T13:19:00.100753+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:09.396769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101032+0000","last_change":"2024-03-28T13:18:59.091677+0000","last_active":"2024-03-28T13:19:00.101032+0000","last_peered":"2024-03-28T13:19:00.101032+0000","last_clean":"2024-03-28T13:19:00.101032+0000","last_became_active":"2024-03-28T13:18:59.091299+0000","last_became_peered":"2024-03-28T13:18:59.091299+0000","last_unstale":"2024-03-28T13:19:00.101032+0000","last_undegraded":"2024-03-28T13:19:00.101032+0000","last_fullsized":"2024-03-28T13:19:00.101032+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:26:09.194396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090234+0000","last_change":"2024-03-28T13:18:59.083262+0000","last_active":"2024-03-28T13:19:00.090234+0000","last_peered":"2024-03-28T13:19:00.090234+0000","last_clean":"2024-03-28T13:19:00.090234+0000","last_became_active":"2024-03-28T13:18:59.082908+0000","last_became_peered":"2024-03-28T13:18:59.082908+0000","last_unstale":"2024-03-28T13:19:00.090234+0000","last_undegraded":"2024-03-28T13:19:00.090234+0000","last_fullsized":"2024-03-28T13:19:00.090234+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:48.131499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.224507+0000","last_change":"2024-03-28T13:18:59.085398+0000","last_active":"2024-03-28T13:19:01.224507+0000","last_peered":"2024-03-28T13:19:01.224507+0000","last_clean":"2024-03-28T13:19:01.224507+0000","last_became_active":"2024-03-28T13:18:59.083434+0000","last_became_peered":"2024-03-28T13:18:59.083434+0000","last_unstale":"2024-03-28T13:19:01.224507+0000","last_undegraded":"2024-03-28T13:19:01.224507+0000","last_fullsized":"2024-03-28T13:19:01.224507+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:05:04.270844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.939237+0000","last_change":"2024-03-28T13:18:59.084798+0000","last_active":"2024-03-28T13:19:00.939237+0000","last_peered":"2024-03-28T13:19:00.939237+0000","last_clean":"2024-03-28T13:19:00.939237+0000","last_became_active":"2024-03-28T13:18:59.084216+0000","last_became_peered":"2024-03-28T13:18:59.084216+0000","last_unstale":"2024-03-28T13:19:00.939237+0000","last_undegraded":"2024-03-28T13:19:00.939237+0000","last_fullsized":"2024-03-28T13:19:00.939237+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:43:24.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225950+0000","last_change":"2024-03-28T13:18:59.086762+0000","last_active":"2024-03-28T13:19:01.225950+0000","last_peered":"2024-03-28T13:19:01.225950+0000","last_clean":"2024-03-28T13:19:01.225950+0000","last_became_active":"2024-03-28T13:18:59.086511+0000","last_became_peered":"2024-03-28T13:18:59.086511+0000","last_unstale":"2024-03-28T13:19:01.225950+0000","last_undegraded":"2024-03-28T13:19:01.225950+0000","last_fullsized":"2024-03-28T13:19:01.225950+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:29:57.882927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225971+0000","last_change":"2024-03-28T13:18:59.080256+0000","last_active":"2024-03-28T13:19:01.225971+0000","last_peered":"2024-03-28T13:19:01.225971+0000","last_clean":"2024-03-28T13:19:01.225971+0000","last_became_active":"2024-03-28T13:18:59.079957+0000","last_became_peered":"2024-03-28T13:18:59.079957+0000","last_unstale":"2024-03-28T13:19:01.225971+0000","last_undegraded":"2024-03-28T13:19:01.225971+0000","last_fullsized":"2024-03-28T13:19:01.225971+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:18:33.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225656+0000","last_change":"2024-03-28T13:18:59.078806+0000","last_active":"2024-03-28T13:19:01.225656+0000","last_peered":"2024-03-28T13:19:01.225656+0000","last_clean":"2024-03-28T13:19:01.225656+0000","last_became_active":"2024-03-28T13:18:59.078526+0000","last_became_peered":"2024-03-28T13:18:59.078526+0000","last_unstale":"2024-03-28T13:19:01.225656+0000","last_undegraded":"2024-03-28T13:19:01.225656+0000","last_fullsized":"2024-03-28T13:19:01.225656+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:20:49.865685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.226080+0000","last_change":"2024-03-28T13:18:59.093088+0000","last_active":"2024-03-28T13:19:01.226080+0000","last_peered":"2024-03-28T13:19:01.226080+0000","last_clean":"2024-03-28T13:19:01.226080+0000","last_became_active":"2024-03-28T13:18:59.092704+0000","last_became_peered":"2024-03-28T13:18:59.092704+0000","last_unstale":"2024-03-28T13:19:01.226080+0000","last_undegraded":"2024-03-28T13:19:01.226080+0000","last_fullsized":"2024-03-28T13:19:01.226080+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:55:14.850361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225935+0000","last_change":"2024-03-28T13:18:59.089095+0000","last_active":"2024-03-28T13:19:01.225935+0000","last_peered":"2024-03-28T13:19:01.225935+0000","last_clean":"2024-03-28T13:19:01.225935+0000","last_became_active":"2024-03-28T13:18:59.088783+0000","last_became_peered":"2024-03-28T13:18:59.088783+0000","last_unstale":"2024-03-28T13:19:01.225935+0000","last_undegraded":"2024-03-28T13:19:01.225935+0000","last_fullsized":"2024-03-28T13:19:01.225935+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:51:28.159619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100578+0000","last_change":"2024-03-28T13:18:59.086506+0000","last_active":"2024-03-28T13:19:00.100578+0000","last_peered":"2024-03-28T13:19:00.100578+0000","last_clean":"2024-03-28T13:19:00.100578+0000","last_became_active":"2024-03-28T13:18:59.086216+0000","last_became_peered":"2024-03-28T13:18:59.086216+0000","last_unstale":"2024-03-28T13:19:00.100578+0000","last_undegraded":"2024-03-28T13:19:00.100578+0000","last_fullsized":"2024-03-28T13:19:00.100578+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:48:04.926545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.089655+0000","last_change":"2024-03-28T13:18:59.075805+0000","last_active":"2024-03-28T13:19:00.089655+0000","last_peered":"2024-03-28T13:19:00.089655+0000","last_clean":"2024-03-28T13:19:00.089655+0000","last_became_active":"2024-03-28T13:18:59.074979+0000","last_became_peered":"2024-03-28T13:18:59.074979+0000","last_unstale":"2024-03-28T13:19:00.089655+0000","last_undegraded":"2024-03-28T13:19:00.089655+0000","last_fullsized":"2024-03-28T13:19:00.089655+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:11:23.081337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.094204+0000","last_change":"2024-03-28T13:18:59.091367+0000","last_active":"2024-03-28T13:19:00.094204+0000","last_peered":"2024-03-28T13:19:00.094204+0000","last_clean":"2024-03-28T13:19:00.094204+0000","last_became_active":"2024-03-28T13:18:59.091083+0000","last_became_peered":"2024-03-28T13:18:59.091083+0000","last_unstale":"2024-03-28T13:19:00.094204+0000","last_undegraded":"2024-03-28T13:19:00.094204+0000","last_fullsized":"2024-03-28T13:19:00.094204+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:49:04.881309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"104'4","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.226241+0000","last_change":"2024-03-28T13:18:58.087418+0000","last_active":"2024-03-28T13:19:01.226241+0000","last_peered":"2024-03-28T13:19:01.226241+0000","last_clean":"2024-03-28T13:19:01.226241+0000","last_became_active":"2024-03-28T13:18:58.086932+0000","last_became_peered":"2024-03-28T13:18:58.086932+0000","last_unstale":"2024-03-28T13:19:01.226241+0000","last_undegraded":"2024-03-28T13:19:01.226241+0000","last_fullsized":"2024-03-28T13:19:01.226241+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:10:38.575107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.226336+0000","last_change":"2024-03-28T13:18:59.088501+0000","last_active":"2024-03-28T13:19:01.226336+0000","last_peered":"2024-03-28T13:19:01.226336+0000","last_clean":"2024-03-28T13:19:01.226336+0000","last_became_active":"2024-03-28T13:18:59.088117+0000","last_became_peered":"2024-03-28T13:18:59.088117+0000","last_unstale":"2024-03-28T13:19:01.226336+0000","last_undegraded":"2024-03-28T13:19:01.226336+0000","last_fullsized":"2024-03-28T13:19:01.226336+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:24:41.712000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092249+0000","last_change":"2024-03-28T13:18:58.077769+0000","last_active":"2024-03-28T13:19:00.092249+0000","last_peered":"2024-03-28T13:19:00.092249+0000","last_clean":"2024-03-28T13:19:00.092249+0000","last_became_active":"2024-03-28T13:18:58.077234+0000","last_became_peered":"2024-03-28T13:18:58.077234+0000","last_unstale":"2024-03-28T13:19:00.092249+0000","last_undegraded":"2024-03-28T13:19:00.092249+0000","last_fullsized":"2024-03-28T13:19:00.092249+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:10:27.168742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091333+0000","last_change":"2024-03-28T13:18:59.084112+0000","last_active":"2024-03-28T13:19:00.091333+0000","last_peered":"2024-03-28T13:19:00.091333+0000","last_clean":"2024-03-28T13:19:00.091333+0000","last_became_active":"2024-03-28T13:18:59.083805+0000","last_became_peered":"2024-03-28T13:18:59.083805+0000","last_unstale":"2024-03-28T13:19:00.091333+0000","last_undegraded":"2024-03-28T13:19:00.091333+0000","last_fullsized":"2024-03-28T13:19:00.091333+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:14:07.278326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101343+0000","last_change":"2024-03-28T13:18:58.087832+0000","last_active":"2024-03-28T13:19:00.101343+0000","last_peered":"2024-03-28T13:19:00.101343+0000","last_clean":"2024-03-28T13:19:00.101343+0000","last_became_active":"2024-03-28T13:18:58.087250+0000","last_became_peered":"2024-03-28T13:18:58.087250+0000","last_unstale":"2024-03-28T13:19:00.101343+0000","last_undegraded":"2024-03-28T13:19:00.101343+0000","last_fullsized":"2024-03-28T13:19:00.101343+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:55:22.065081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.938951+0000","last_change":"2024-03-28T13:18:59.097206+0000","last_active":"2024-03-28T13:19:00.938951+0000","last_peered":"2024-03-28T13:19:00.938951+0000","last_clean":"2024-03-28T13:19:00.938951+0000","last_became_active":"2024-03-28T13:18:59.096903+0000","last_became_peered":"2024-03-28T13:18:59.096903+0000","last_unstale":"2024-03-28T13:19:00.938951+0000","last_undegraded":"2024-03-28T13:19:00.938951+0000","last_fullsized":"2024-03-28T13:19:00.938951+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:26:09.194396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091455+0000","last_change":"2024-03-28T13:18:58.083531+0000","last_active":"2024-03-28T13:19:00.091455+0000","last_peered":"2024-03-28T13:19:00.091455+0000","last_clean":"2024-03-28T13:19:00.091455+0000","last_became_active":"2024-03-28T13:18:58.082970+0000","last_became_peered":"2024-03-28T13:18:58.082970+0000","last_unstale":"2024-03-28T13:19:00.091455+0000","last_undegraded":"2024-03-28T13:19:00.091455+0000","last_fullsized":"2024-03-28T13:19:00.091455+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:08.383486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.224237+0000","last_change":"2024-03-28T13:18:59.098346+0000","last_active":"2024-03-28T13:19:01.224237+0000","last_peered":"2024-03-28T13:19:01.224237+0000","last_clean":"2024-03-28T13:19:01.224237+0000","last_became_active":"2024-03-28T13:18:59.098052+0000","last_became_peered":"2024-03-28T13:18:59.098052+0000","last_unstale":"2024-03-28T13:19:01.224237+0000","last_undegraded":"2024-03-28T13:19:01.224237+0000","last_fullsized":"2024-03-28T13:19:01.224237+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:48:04.926545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090424+0000","last_change":"2024-03-28T13:18:58.064634+0000","last_active":"2024-03-28T13:19:00.090424+0000","last_peered":"2024-03-28T13:19:00.090424+0000","last_clean":"2024-03-28T13:19:00.090424+0000","last_became_active":"2024-03-28T13:18:58.064194+0000","last_became_peered":"2024-03-28T13:18:58.064194+0000","last_unstale":"2024-03-28T13:19:00.090424+0000","last_undegraded":"2024-03-28T13:19:00.090424+0000","last_fullsized":"2024-03-28T13:19:00.090424+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:21:46.733401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091377+0000","last_change":"2024-03-28T13:18:59.094754+0000","last_active":"2024-03-28T13:19:00.091377+0000","last_peered":"2024-03-28T13:19:00.091377+0000","last_clean":"2024-03-28T13:19:00.091377+0000","last_became_active":"2024-03-28T13:18:59.090317+0000","last_became_peered":"2024-03-28T13:18:59.090317+0000","last_unstale":"2024-03-28T13:19:00.091377+0000","last_undegraded":"2024-03-28T13:19:00.091377+0000","last_fullsized":"2024-03-28T13:19:00.091377+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:09:17.961351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100702+0000","last_change":"2024-03-28T13:18:58.082525+0000","last_active":"2024-03-28T13:19:00.100702+0000","last_peered":"2024-03-28T13:19:00.100702+0000","last_clean":"2024-03-28T13:19:00.100702+0000","last_became_active":"2024-03-28T13:18:58.082039+0000","last_became_peered":"2024-03-28T13:18:58.082039+0000","last_unstale":"2024-03-28T13:19:00.100702+0000","last_undegraded":"2024-03-28T13:19:00.100702+0000","last_fullsized":"2024-03-28T13:19:00.100702+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:42.609780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225536+0000","last_change":"2024-03-28T13:18:59.093894+0000","last_active":"2024-03-28T13:19:01.225536+0000","last_peered":"2024-03-28T13:19:01.225536+0000","last_clean":"2024-03-28T13:19:01.225536+0000","last_became_active":"2024-03-28T13:18:59.093258+0000","last_became_peered":"2024-03-28T13:18:59.093258+0000","last_unstale":"2024-03-28T13:19:01.225536+0000","last_undegraded":"2024-03-28T13:19:01.225536+0000","last_fullsized":"2024-03-28T13:19:01.225536+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:34:15.217129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090824+0000","last_change":"2024-03-28T13:18:58.078267+0000","last_active":"2024-03-28T13:19:00.090824+0000","last_peered":"2024-03-28T13:19:00.090824+0000","last_clean":"2024-03-28T13:19:00.090824+0000","last_became_active":"2024-03-28T13:18:58.077891+0000","last_became_peered":"2024-03-28T13:18:58.077891+0000","last_unstale":"2024-03-28T13:19:00.090824+0000","last_undegraded":"2024-03-28T13:19:00.090824+0000","last_fullsized":"2024-03-28T13:19:00.090824+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:35:04.854788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.224715+0000","last_change":"2024-03-28T13:18:59.099035+0000","last_active":"2024-03-28T13:19:01.224715+0000","last_peered":"2024-03-28T13:19:01.224715+0000","last_clean":"2024-03-28T13:19:01.224715+0000","last_became_active":"2024-03-28T13:18:59.098118+0000","last_became_peered":"2024-03-28T13:18:59.098118+0000","last_unstale":"2024-03-28T13:19:01.224715+0000","last_undegraded":"2024-03-28T13:19:01.224715+0000","last_fullsized":"2024-03-28T13:19:01.224715+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:29.522394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092964+0000","last_change":"2024-03-28T13:18:58.076072+0000","last_active":"2024-03-28T13:19:00.092964+0000","last_peered":"2024-03-28T13:19:00.092964+0000","last_clean":"2024-03-28T13:19:00.092964+0000","last_became_active":"2024-03-28T13:18:58.075808+0000","last_became_peered":"2024-03-28T13:18:58.075808+0000","last_unstale":"2024-03-28T13:19:00.092964+0000","last_undegraded":"2024-03-28T13:19:00.092964+0000","last_fullsized":"2024-03-28T13:19:00.092964+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:11:22.068054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091695+0000","last_change":"2024-03-28T13:18:59.075982+0000","last_active":"2024-03-28T13:19:00.091695+0000","last_peered":"2024-03-28T13:19:00.091695+0000","last_clean":"2024-03-28T13:19:00.091695+0000","last_became_active":"2024-03-28T13:18:59.075606+0000","last_became_peered":"2024-03-28T13:18:59.075606+0000","last_unstale":"2024-03-28T13:19:00.091695+0000","last_undegraded":"2024-03-28T13:19:00.091695+0000","last_fullsized":"2024-03-28T13:19:00.091695+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:12:28.089180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101379+0000","last_change":"2024-03-28T13:18:58.077577+0000","last_active":"2024-03-28T13:19:00.101379+0000","last_peered":"2024-03-28T13:19:00.101379+0000","last_clean":"2024-03-28T13:19:00.101379+0000","last_became_active":"2024-03-28T13:18:58.077275+0000","last_became_peered":"2024-03-28T13:18:58.077275+0000","last_unstale":"2024-03-28T13:19:00.101379+0000","last_undegraded":"2024-03-28T13:19:00.101379+0000","last_fullsized":"2024-03-28T13:19:00.101379+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:28:48.630263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225712+0000","last_change":"2024-03-28T13:18:59.086040+0000","last_active":"2024-03-28T13:19:01.225712+0000","last_peered":"2024-03-28T13:19:01.225712+0000","last_clean":"2024-03-28T13:19:01.225712+0000","last_became_active":"2024-03-28T13:18:59.085749+0000","last_became_peered":"2024-03-28T13:18:59.085749+0000","last_unstale":"2024-03-28T13:19:01.225712+0000","last_undegraded":"2024-03-28T13:19:01.225712+0000","last_fullsized":"2024-03-28T13:19:01.225712+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:08:42.338113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091810+0000","last_change":"2024-03-28T13:18:58.083463+0000","last_active":"2024-03-28T13:19:00.091810+0000","last_peered":"2024-03-28T13:19:00.091810+0000","last_clean":"2024-03-28T13:19:00.091810+0000","last_became_active":"2024-03-28T13:18:58.083152+0000","last_became_peered":"2024-03-28T13:18:58.083152+0000","last_unstale":"2024-03-28T13:19:00.091810+0000","last_undegraded":"2024-03-28T13:19:00.091810+0000","last_fullsized":"2024-03-28T13:19:00.091810+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:48:03.913262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091380+0000","last_change":"2024-03-28T13:18:59.081727+0000","last_active":"2024-03-28T13:19:00.091380+0000","last_peered":"2024-03-28T13:19:00.091380+0000","last_clean":"2024-03-28T13:19:00.091380+0000","last_became_active":"2024-03-28T13:18:59.081436+0000","last_became_peered":"2024-03-28T13:18:59.081436+0000","last_unstale":"2024-03-28T13:19:00.091380+0000","last_undegraded":"2024-03-28T13:19:00.091380+0000","last_fullsized":"2024-03-28T13:19:00.091380+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:55:41.223520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"104'3","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092029+0000","last_change":"2024-03-28T13:18:58.068721+0000","last_active":"2024-03-28T13:19:00.092029+0000","last_peered":"2024-03-28T13:19:00.092029+0000","last_clean":"2024-03-28T13:19:00.092029+0000","last_became_active":"2024-03-28T13:18:58.068416+0000","last_became_peered":"2024-03-28T13:18:58.068416+0000","last_unstale":"2024-03-28T13:19:00.092029+0000","last_undegraded":"2024-03-28T13:19:00.092029+0000","last_fullsized":"2024-03-28T13:19:00.092029+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:16:13.945885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225743+0000","last_change":"2024-03-28T13:18:59.080942+0000","last_active":"2024-03-28T13:19:01.225743+0000","last_peered":"2024-03-28T13:19:01.225743+0000","last_clean":"2024-03-28T13:19:01.225743+0000","last_became_active":"2024-03-28T13:18:59.080654+0000","last_became_peered":"2024-03-28T13:18:59.080654+0000","last_unstale":"2024-03-28T13:19:01.225743+0000","last_undegraded":"2024-03-28T13:19:01.225743+0000","last_fullsized":"2024-03-28T13:19:01.225743+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:48.131499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"104'6","reported_seq":20,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.224754+0000","last_change":"2024-03-28T13:18:58.076129+0000","last_active":"2024-03-28T13:19:01.224754+0000","last_peered":"2024-03-28T13:19:01.224754+0000","last_clean":"2024-03-28T13:19:01.224754+0000","last_became_active":"2024-03-28T13:18:58.075809+0000","last_became_peered":"2024-03-28T13:18:58.075809+0000","last_unstale":"2024-03-28T13:19:01.224754+0000","last_undegraded":"2024-03-28T13:19:01.224754+0000","last_fullsized":"2024-03-28T13:19:01.224754+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:49:19.123418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090517+0000","last_change":"2024-03-28T13:18:59.077667+0000","last_active":"2024-03-28T13:19:00.090517+0000","last_peered":"2024-03-28T13:19:00.090517+0000","last_clean":"2024-03-28T13:19:00.090517+0000","last_became_active":"2024-03-28T13:18:59.077304+0000","last_became_peered":"2024-03-28T13:18:59.077304+0000","last_unstale":"2024-03-28T13:19:00.090517+0000","last_undegraded":"2024-03-28T13:19:00.090517+0000","last_fullsized":"2024-03-28T13:19:00.090517+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:18.635249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.094221+0000","last_change":"2024-03-28T13:18:58.077229+0000","last_active":"2024-03-28T13:19:00.094221+0000","last_peered":"2024-03-28T13:19:00.094221+0000","last_clean":"2024-03-28T13:19:00.094221+0000","last_became_active":"2024-03-28T13:18:58.076961+0000","last_became_peered":"2024-03-28T13:18:58.076961+0000","last_unstale":"2024-03-28T13:19:00.094221+0000","last_undegraded":"2024-03-28T13:19:00.094221+0000","last_fullsized":"2024-03-28T13:19:00.094221+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:20:48.852402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100869+0000","last_change":"2024-03-28T13:18:59.084784+0000","last_active":"2024-03-28T13:19:00.100869+0000","last_peered":"2024-03-28T13:19:00.100869+0000","last_clean":"2024-03-28T13:19:00.100869+0000","last_became_active":"2024-03-28T13:18:59.084518+0000","last_became_peered":"2024-03-28T13:18:59.084518+0000","last_unstale":"2024-03-28T13:19:00.100869+0000","last_undegraded":"2024-03-28T13:19:00.100869+0000","last_fullsized":"2024-03-28T13:19:00.100869+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:29.522394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090380+0000","last_change":"2024-03-28T13:18:58.083463+0000","last_active":"2024-03-28T13:19:00.090380+0000","last_peered":"2024-03-28T13:19:00.090380+0000","last_clean":"2024-03-28T13:19:00.090380+0000","last_became_active":"2024-03-28T13:18:58.083122+0000","last_became_peered":"2024-03-28T13:18:58.083122+0000","last_unstale":"2024-03-28T13:19:00.090380+0000","last_undegraded":"2024-03-28T13:19:00.090380+0000","last_fullsized":"2024-03-28T13:19:00.090380+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:20:43.187370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090923+0000","last_change":"2024-03-28T13:18:59.082251+0000","last_active":"2024-03-28T13:19:00.090923+0000","last_peered":"2024-03-28T13:19:00.090923+0000","last_clean":"2024-03-28T13:19:00.090923+0000","last_became_active":"2024-03-28T13:18:59.081951+0000","last_became_peered":"2024-03-28T13:18:59.081951+0000","last_unstale":"2024-03-28T13:19:00.090923+0000","last_undegraded":"2024-03-28T13:19:00.090923+0000","last_fullsized":"2024-03-28T13:19:00.090923+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:53.418554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.224062+0000","last_change":"2024-03-28T13:18:58.066532+0000","last_active":"2024-03-28T13:19:01.224062+0000","last_peered":"2024-03-28T13:19:01.224062+0000","last_clean":"2024-03-28T13:19:01.224062+0000","last_became_active":"2024-03-28T13:18:58.066208+0000","last_became_peered":"2024-03-28T13:18:58.066208+0000","last_unstale":"2024-03-28T13:19:01.224062+0000","last_undegraded":"2024-03-28T13:19:01.224062+0000","last_fullsized":"2024-03-28T13:19:01.224062+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:58.349889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100798+0000","last_change":"2024-03-28T13:18:59.090442+0000","last_active":"2024-03-28T13:19:00.100798+0000","last_peered":"2024-03-28T13:19:00.100798+0000","last_clean":"2024-03-28T13:19:00.100798+0000","last_became_active":"2024-03-28T13:18:59.090071+0000","last_became_peered":"2024-03-28T13:18:59.090071+0000","last_unstale":"2024-03-28T13:19:00.100798+0000","last_undegraded":"2024-03-28T13:19:00.100798+0000","last_fullsized":"2024-03-28T13:19:00.100798+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:29.899339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"104'8","reported_seq":22,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.089921+0000","last_change":"2024-03-28T13:18:58.071341+0000","last_active":"2024-03-28T13:19:00.089921+0000","last_peered":"2024-03-28T13:19:00.089921+0000","last_clean":"2024-03-28T13:19:00.089921+0000","last_became_active":"2024-03-28T13:18:58.069574+0000","last_became_peered":"2024-03-28T13:18:58.069574+0000","last_unstale":"2024-03-28T13:19:00.089921+0000","last_undegraded":"2024-03-28T13:19:00.089921+0000","last_fullsized":"2024-03-28T13:19:00.089921+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:13.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225655+0000","last_change":"2024-03-28T13:18:59.093731+0000","last_active":"2024-03-28T13:19:01.225655+0000","last_peered":"2024-03-28T13:19:01.225655+0000","last_clean":"2024-03-28T13:19:01.225655+0000","last_became_active":"2024-03-28T13:18:59.093471+0000","last_became_peered":"2024-03-28T13:18:59.093471+0000","last_unstale":"2024-03-28T13:19:01.225655+0000","last_undegraded":"2024-03-28T13:19:01.225655+0000","last_fullsized":"2024-03-28T13:19:01.225655+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:11:23.081337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.224144+0000","last_change":"2024-03-28T13:18:58.080576+0000","last_active":"2024-03-28T13:19:01.224144+0000","last_peered":"2024-03-28T13:19:01.224144+0000","last_clean":"2024-03-28T13:19:01.224144+0000","last_became_active":"2024-03-28T13:18:58.080272+0000","last_became_peered":"2024-03-28T13:18:58.080272+0000","last_unstale":"2024-03-28T13:19:01.224144+0000","last_undegraded":"2024-03-28T13:19:01.224144+0000","last_fullsized":"2024-03-28T13:19:01.224144+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:37:13.863176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225903+0000","last_change":"2024-03-28T13:18:59.092760+0000","last_active":"2024-03-28T13:19:01.225903+0000","last_peered":"2024-03-28T13:19:01.225903+0000","last_clean":"2024-03-28T13:19:01.225903+0000","last_became_active":"2024-03-28T13:18:59.092409+0000","last_became_peered":"2024-03-28T13:18:59.092409+0000","last_unstale":"2024-03-28T13:19:01.225903+0000","last_undegraded":"2024-03-28T13:19:01.225903+0000","last_fullsized":"2024-03-28T13:19:01.225903+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:18.635249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.092132+0000","last_change":"2024-03-28T13:18:58.065630+0000","last_active":"2024-03-28T13:19:00.092132+0000","last_peered":"2024-03-28T13:19:00.092132+0000","last_clean":"2024-03-28T13:19:00.092132+0000","last_became_active":"2024-03-28T13:18:58.065333+0000","last_became_peered":"2024-03-28T13:18:58.065333+0000","last_unstale":"2024-03-28T13:19:00.092132+0000","last_undegraded":"2024-03-28T13:19:00.092132+0000","last_fullsized":"2024-03-28T13:19:00.092132+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:54:14.505307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.224743+0000","last_change":"2024-03-28T13:18:59.092928+0000","last_active":"2024-03-28T13:19:01.224743+0000","last_peered":"2024-03-28T13:19:01.224743+0000","last_clean":"2024-03-28T13:19:01.224743+0000","last_became_active":"2024-03-28T13:18:59.092579+0000","last_became_peered":"2024-03-28T13:18:59.092579+0000","last_unstale":"2024-03-28T13:19:01.224743+0000","last_undegraded":"2024-03-28T13:19:01.224743+0000","last_fullsized":"2024-03-28T13:19:01.224743+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:26:09.194396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093878+0000","last_change":"2024-03-28T13:18:58.075199+0000","last_active":"2024-03-28T13:19:00.093878+0000","last_peered":"2024-03-28T13:19:00.093878+0000","last_clean":"2024-03-28T13:19:00.093878+0000","last_became_active":"2024-03-28T13:18:58.073665+0000","last_became_peered":"2024-03-28T13:18:58.073665+0000","last_unstale":"2024-03-28T13:19:00.093878+0000","last_undegraded":"2024-03-28T13:19:00.093878+0000","last_fullsized":"2024-03-28T13:19:00.093878+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:08:41.324830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093360+0000","last_change":"2024-03-28T13:18:59.076634+0000","last_active":"2024-03-28T13:19:00.093360+0000","last_peered":"2024-03-28T13:19:00.093360+0000","last_clean":"2024-03-28T13:19:00.093360+0000","last_became_active":"2024-03-28T13:18:59.076295+0000","last_became_peered":"2024-03-28T13:18:59.076295+0000","last_unstale":"2024-03-28T13:19:00.093360+0000","last_undegraded":"2024-03-28T13:19:00.093360+0000","last_fullsized":"2024-03-28T13:19:00.093360+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:11:39.100208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090726+0000","last_change":"2024-03-28T13:18:58.084188+0000","last_active":"2024-03-28T13:19:00.090726+0000","last_peered":"2024-03-28T13:19:00.090726+0000","last_clean":"2024-03-28T13:19:00.090726+0000","last_became_active":"2024-03-28T13:18:58.081720+0000","last_became_peered":"2024-03-28T13:18:58.081720+0000","last_unstale":"2024-03-28T13:19:00.090726+0000","last_undegraded":"2024-03-28T13:19:00.090726+0000","last_fullsized":"2024-03-28T13:19:00.090726+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:44.810836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093931+0000","last_change":"2024-03-28T13:18:59.079466+0000","last_active":"2024-03-28T13:19:00.093931+0000","last_peered":"2024-03-28T13:19:00.093931+0000","last_clean":"2024-03-28T13:19:00.093931+0000","last_became_active":"2024-03-28T13:18:59.078752+0000","last_became_peered":"2024-03-28T13:18:59.078752+0000","last_unstale":"2024-03-28T13:19:00.093931+0000","last_undegraded":"2024-03-28T13:19:00.093931+0000","last_fullsized":"2024-03-28T13:19:00.093931+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:36:50.468775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.224759+0000","last_change":"2024-03-28T13:18:58.079436+0000","last_active":"2024-03-28T13:19:01.224759+0000","last_peered":"2024-03-28T13:19:01.224759+0000","last_clean":"2024-03-28T13:19:01.224759+0000","last_became_active":"2024-03-28T13:18:58.079066+0000","last_became_peered":"2024-03-28T13:18:58.079066+0000","last_unstale":"2024-03-28T13:19:01.224759+0000","last_undegraded":"2024-03-28T13:19:01.224759+0000","last_fullsized":"2024-03-28T13:19:01.224759+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:44:08.383486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.090744+0000","last_change":"2024-03-28T13:18:59.080751+0000","last_active":"2024-03-28T13:19:00.090744+0000","last_peered":"2024-03-28T13:19:00.090744+0000","last_clean":"2024-03-28T13:19:00.090744+0000","last_became_active":"2024-03-28T13:18:59.080449+0000","last_became_peered":"2024-03-28T13:18:59.080449+0000","last_unstale":"2024-03-28T13:19:00.090744+0000","last_undegraded":"2024-03-28T13:19:00.090744+0000","last_fullsized":"2024-03-28T13:19:00.090744+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:18:33.953457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.938351+0000","last_change":"2024-03-28T13:18:58.079341+0000","last_active":"2024-03-28T13:19:00.938351+0000","last_peered":"2024-03-28T13:19:00.938351+0000","last_clean":"2024-03-28T13:19:00.938351+0000","last_became_active":"2024-03-28T13:18:58.079008+0000","last_became_peered":"2024-03-28T13:18:58.079008+0000","last_unstale":"2024-03-28T13:19:00.938351+0000","last_undegraded":"2024-03-28T13:19:00.938351+0000","last_fullsized":"2024-03-28T13:19:00.938351+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:21:46.733401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225399+0000","last_change":"2024-03-28T13:18:59.080827+0000","last_active":"2024-03-28T13:19:01.225399+0000","last_peered":"2024-03-28T13:19:01.225399+0000","last_clean":"2024-03-28T13:19:01.225399+0000","last_became_active":"2024-03-28T13:18:59.080524+0000","last_became_peered":"2024-03-28T13:18:59.080524+0000","last_unstale":"2024-03-28T13:19:01.225399+0000","last_undegraded":"2024-03-28T13:19:01.225399+0000","last_fullsized":"2024-03-28T13:19:01.225399+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:35:49.070231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100342+0000","last_change":"2024-03-28T13:18:58.090817+0000","last_active":"2024-03-28T13:19:00.100342+0000","last_peered":"2024-03-28T13:19:00.100342+0000","last_clean":"2024-03-28T13:19:00.100342+0000","last_became_active":"2024-03-28T13:18:58.090469+0000","last_became_peered":"2024-03-28T13:18:58.090469+0000","last_unstale":"2024-03-28T13:19:00.100342+0000","last_undegraded":"2024-03-28T13:19:00.100342+0000","last_fullsized":"2024-03-28T13:19:00.100342+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:11:52.858677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.094597+0000","last_change":"2024-03-28T13:18:59.071403+0000","last_active":"2024-03-28T13:19:00.094597+0000","last_peered":"2024-03-28T13:19:00.094597+0000","last_clean":"2024-03-28T13:19:00.094597+0000","last_became_active":"2024-03-28T13:18:59.071095+0000","last_became_peered":"2024-03-28T13:18:59.071095+0000","last_unstale":"2024-03-28T13:19:00.094597+0000","last_undegraded":"2024-03-28T13:19:00.094597+0000","last_fullsized":"2024-03-28T13:19:00.094597+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:09:52.654125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"104'2","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225066+0000","last_change":"2024-03-28T13:18:58.071038+0000","last_active":"2024-03-28T13:19:01.225066+0000","last_peered":"2024-03-28T13:19:01.225066+0000","last_clean":"2024-03-28T13:19:01.225066+0000","last_became_active":"2024-03-28T13:18:58.069807+0000","last_became_peered":"2024-03-28T13:18:58.069807+0000","last_unstale":"2024-03-28T13:19:01.225066+0000","last_undegraded":"2024-03-28T13:19:01.225066+0000","last_fullsized":"2024-03-28T13:19:01.225066+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:17:17.825753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.095041+0000","last_change":"2024-03-28T13:18:59.076727+0000","last_active":"2024-03-28T13:19:00.095041+0000","last_peered":"2024-03-28T13:19:00.095041+0000","last_clean":"2024-03-28T13:19:00.095041+0000","last_became_active":"2024-03-28T13:18:59.076415+0000","last_became_peered":"2024-03-28T13:18:59.076415+0000","last_unstale":"2024-03-28T13:19:00.095041+0000","last_undegraded":"2024-03-28T13:19:00.095041+0000","last_fullsized":"2024-03-28T13:19:00.095041+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:56:27.830992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"105'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.095140+0000","last_change":"2024-03-28T13:18:58.086609+0000","last_active":"2024-03-28T13:19:00.095140+0000","last_peered":"2024-03-28T13:19:00.095140+0000","last_clean":"2024-03-28T13:19:00.095140+0000","last_became_active":"2024-03-28T13:18:58.086223+0000","last_became_peered":"2024-03-28T13:18:58.086223+0000","last_unstale":"2024-03-28T13:19:00.095140+0000","last_undegraded":"2024-03-28T13:19:00.095140+0000","last_fullsized":"2024-03-28T13:19:00.095140+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:15:17.621966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101315+0000","last_change":"2024-03-28T13:18:59.098557+0000","last_active":"2024-03-28T13:19:00.101315+0000","last_peered":"2024-03-28T13:19:00.101315+0000","last_clean":"2024-03-28T13:19:00.101315+0000","last_became_active":"2024-03-28T13:18:59.098274+0000","last_became_peered":"2024-03-28T13:18:59.098274+0000","last_unstale":"2024-03-28T13:19:00.101315+0000","last_undegraded":"2024-03-28T13:19:00.101315+0000","last_fullsized":"2024-03-28T13:19:00.101315+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:20.977905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.094124+0000","last_change":"2024-03-28T13:18:58.072039+0000","last_active":"2024-03-28T13:19:00.094124+0000","last_peered":"2024-03-28T13:19:00.094124+0000","last_clean":"2024-03-28T13:19:00.094124+0000","last_became_active":"2024-03-28T13:18:58.071683+0000","last_became_peered":"2024-03-28T13:18:58.071683+0000","last_unstale":"2024-03-28T13:19:00.094124+0000","last_undegraded":"2024-03-28T13:19:00.094124+0000","last_fullsized":"2024-03-28T13:19:00.094124+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:47:47.118216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091918+0000","last_change":"2024-03-28T13:18:59.087471+0000","last_active":"2024-03-28T13:19:00.091918+0000","last_peered":"2024-03-28T13:19:00.091918+0000","last_clean":"2024-03-28T13:19:00.091918+0000","last_became_active":"2024-03-28T13:18:59.085466+0000","last_became_peered":"2024-03-28T13:18:59.085466+0000","last_unstale":"2024-03-28T13:19:00.091918+0000","last_undegraded":"2024-03-28T13:19:00.091918+0000","last_fullsized":"2024-03-28T13:19:00.091918+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:30:44.259677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.225989+0000","last_change":"2024-03-28T13:18:58.068025+0000","last_active":"2024-03-28T13:19:01.225989+0000","last_peered":"2024-03-28T13:19:01.225989+0000","last_clean":"2024-03-28T13:19:01.225989+0000","last_became_active":"2024-03-28T13:18:58.067671+0000","last_became_peered":"2024-03-28T13:18:58.067671+0000","last_unstale":"2024-03-28T13:19:01.225989+0000","last_undegraded":"2024-03-28T13:19:01.225989+0000","last_fullsized":"2024-03-28T13:19:01.225989+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:28.509110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093804+0000","last_change":"2024-03-28T13:18:59.073922+0000","last_active":"2024-03-28T13:19:00.093804+0000","last_peered":"2024-03-28T13:19:00.093804+0000","last_clean":"2024-03-28T13:19:00.093804+0000","last_became_active":"2024-03-28T13:18:59.073566+0000","last_became_peered":"2024-03-28T13:18:59.073566+0000","last_unstale":"2024-03-28T13:19:00.093804+0000","last_undegraded":"2024-03-28T13:19:00.093804+0000","last_fullsized":"2024-03-28T13:19:00.093804+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:38:33.010172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101273+0000","last_change":"2024-03-28T13:18:58.085176+0000","last_active":"2024-03-28T13:19:00.101273+0000","last_peered":"2024-03-28T13:19:00.101273+0000","last_clean":"2024-03-28T13:19:00.101273+0000","last_became_active":"2024-03-28T13:18:58.084898+0000","last_became_peered":"2024-03-28T13:18:58.084898+0000","last_unstale":"2024-03-28T13:19:00.101273+0000","last_undegraded":"2024-03-28T13:19:00.101273+0000","last_fullsized":"2024-03-28T13:19:00.101273+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:13:25.784483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.091562+0000","last_change":"2024-03-28T13:18:59.076130+0000","last_active":"2024-03-28T13:19:00.091562+0000","last_peered":"2024-03-28T13:19:00.091562+0000","last_clean":"2024-03-28T13:19:00.091562+0000","last_became_active":"2024-03-28T13:18:59.075785+0000","last_became_peered":"2024-03-28T13:18:59.075785+0000","last_unstale":"2024-03-28T13:19:00.091562+0000","last_undegraded":"2024-03-28T13:19:00.091562+0000","last_fullsized":"2024-03-28T13:19:00.091562+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:04:31.836556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.224938+0000","last_change":"2024-03-28T13:18:58.073984+0000","last_active":"2024-03-28T13:19:01.224938+0000","last_peered":"2024-03-28T13:19:01.224938+0000","last_clean":"2024-03-28T13:19:01.224938+0000","last_became_active":"2024-03-28T13:18:58.073520+0000","last_became_peered":"2024-03-28T13:18:58.073520+0000","last_unstale":"2024-03-28T13:19:01.224938+0000","last_undegraded":"2024-03-28T13:19:01.224938+0000","last_fullsized":"2024-03-28T13:19:01.224938+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:29:28.886056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.093125+0000","last_change":"2024-03-28T13:18:59.078032+0000","last_active":"2024-03-28T13:19:00.093125+0000","last_peered":"2024-03-28T13:19:00.093125+0000","last_clean":"2024-03-28T13:19:00.093125+0000","last_became_active":"2024-03-28T13:18:59.077665+0000","last_became_peered":"2024-03-28T13:18:59.077665+0000","last_unstale":"2024-03-28T13:19:00.093125+0000","last_undegraded":"2024-03-28T13:19:00.093125+0000","last_fullsized":"2024-03-28T13:19:00.093125+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:14:14.261611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.226316+0000","last_change":"2024-03-28T13:18:58.083533+0000","last_active":"2024-03-28T13:19:01.226316+0000","last_peered":"2024-03-28T13:19:01.226316+0000","last_clean":"2024-03-28T13:19:01.226316+0000","last_became_active":"2024-03-28T13:18:58.083220+0000","last_became_peered":"2024-03-28T13:18:58.083220+0000","last_unstale":"2024-03-28T13:19:01.226316+0000","last_undegraded":"2024-03-28T13:19:01.226316+0000","last_fullsized":"2024-03-28T13:19:01.226316+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:44:18.338917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.223889+0000","last_change":"2024-03-28T13:18:59.091706+0000","last_active":"2024-03-28T13:19:01.223889+0000","last_peered":"2024-03-28T13:19:01.223889+0000","last_clean":"2024-03-28T13:19:01.223889+0000","last_became_active":"2024-03-28T13:18:59.091410+0000","last_became_peered":"2024-03-28T13:18:59.091410+0000","last_unstale":"2024-03-28T13:19:01.223889+0000","last_undegraded":"2024-03-28T13:19:01.223889+0000","last_fullsized":"2024-03-28T13:19:01.223889+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:34:45.824120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.100688+0000","last_change":"2024-03-28T13:18:58.086792+0000","last_active":"2024-03-28T13:19:00.100688+0000","last_peered":"2024-03-28T13:19:00.100688+0000","last_clean":"2024-03-28T13:19:00.100688+0000","last_became_active":"2024-03-28T13:18:58.086544+0000","last_became_peered":"2024-03-28T13:18:58.086544+0000","last_unstale":"2024-03-28T13:19:00.100688+0000","last_undegraded":"2024-03-28T13:19:00.100688+0000","last_fullsized":"2024-03-28T13:19:00.100688+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:51:50.282781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:00.101376+0000","last_change":"2024-03-28T13:18:59.088616+0000","last_active":"2024-03-28T13:19:00.101376+0000","last_peered":"2024-03-28T13:19:00.101376+0000","last_clean":"2024-03-28T13:19:00.101376+0000","last_became_active":"2024-03-28T13:18:59.088292+0000","last_became_peered":"2024-03-28T13:18:59.088292+0000","last_unstale":"2024-03-28T13:19:00.101376+0000","last_undegraded":"2024-03-28T13:19:00.101376+0000","last_fullsized":"2024-03-28T13:19:00.101376+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:58.028202+0000","last_clean_scrub_stamp":"2024-03-28T13:18:58.028202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:43:24.586808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"104'1","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.224125+0000","last_change":"2024-03-28T13:18:58.075299+0000","last_active":"2024-03-28T13:19:01.224125+0000","last_peered":"2024-03-28T13:19:01.224125+0000","last_clean":"2024-03-28T13:19:01.224125+0000","last_became_active":"2024-03-28T13:18:58.075035+0000","last_became_peered":"2024-03-28T13:18:58.075035+0000","last_unstale":"2024-03-28T13:19:01.224125+0000","last_undegraded":"2024-03-28T13:19:01.224125+0000","last_fullsized":"2024-03-28T13:19:01.224125+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:18:57.014919+0000","last_clean_scrub_stamp":"2024-03-28T13:18:57.014919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:27:39.277367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":322,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-28T13:19:01.223805+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:19:01.223805+0000","last_peered":"2024-03-28T13:19:01.223805+0000","last_clean":"2024-03-28T13:19:01.223805+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:19:01.223805+0000","last_undegraded":"2024-03-28T13:19:01.223805+0000","last_fullsized":"2024-03-28T13:19:01.223805+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476857,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1298432,"data_stored":556746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1290240,"data_stored":556622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1900544,"data_stored":1148644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476855,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1286144,"data_stored":556552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509558,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":556552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509559,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33040,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338800,"statfs":{"total":96636764160,"available":96602931200,"internally_reserved":0,"allocated":1900544,"data_stored":1148588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509559,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":557134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542264,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1290240,"data_stored":557134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:19:09.371 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-28T13:19:09.372 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-28T13:19:09.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.638+0000 7f6b23000640 1 Processor -- start 2024-03-28T13:19:09.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.639+0000 7f6b23000640 1 -- start start 2024-03-28T13:19:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.642+0000 7f6b23000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6b1c06e030 0x7f6b1c15fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:09.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.642+0000 7f6b23000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b1c142880 con 0x7f6b1c1602d0 2024-03-28T13:19:09.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.642+0000 7f6b23000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b1c1422c0 con 0x7f6b1c13f300 2024-03-28T13:19:09.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.642+0000 7f6b23000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6b1c160720 con 0x7f6b1c06e030 2024-03-28T13:19:09.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.642+0000 7f6b21200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6b1c06e030 0x7f6b1c15fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:09.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.642+0000 7f6b22600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6b1c1602d0 0x7f6b1c0cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45076/0 (socket says 172.21.15.19:45076) 2024-03-28T13:19:09.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.642+0000 7f6b22600640 1 -- 172.21.15.19:0/156626244 learned_addr learned my addr 172.21.15.19:0/156626244 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:09.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.643+0000 7f6b21200640 1 -- 172.21.15.19:0/156626244 >> v1:172.21.15.188:6789/0 conn(0x7f6b1c13f300 legacy=0x7f6b1c1419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:09.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.643+0000 7f6b21200640 1 -- 172.21.15.19:0/156626244 >> v1:172.21.15.19:6789/0 conn(0x7f6b1c1602d0 legacy=0x7f6b1c0cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:09.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.643+0000 7f6b21200640 1 -- 172.21.15.19:0/156626244 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c1425a0 con 0x7f6b1c06e030 2024-03-28T13:19:09.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.643+0000 7f6b1be00640 1 -- 172.21.15.19:0/156626244 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 913413562 0 0) 0x7f6b1c142880 con 0x7f6b1c1602d0 2024-03-28T13:19:09.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.644+0000 7f6b21200640 1 --2- 172.21.15.19:0/156626244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c06e030 0x7f6b1c15fd90 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6b080097a0 tx=0x7f6b0802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21f520a440c7e8f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:09.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.644+0000 7f6b1be00640 1 -- 172.21.15.19:0/156626244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08034070 con 0x7f6b1c06e030 2024-03-28T13:19:09.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.645+0000 7f6b1be00640 1 -- 172.21.15.19:0/156626244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b08030610 con 0x7f6b1c06e030 2024-03-28T13:19:09.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.645+0000 7f6b1be00640 1 -- 172.21.15.19:0/156626244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b080309e0 con 0x7f6b1c06e030 2024-03-28T13:19:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.645+0000 7f6b23000640 1 -- 172.21.15.19:0/156626244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c06e030 msgr2=0x7f6b1c15fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.645+0000 7f6b23000640 1 --2- 172.21.15.19:0/156626244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c06e030 0x7f6b1c15fd90 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f6b080097a0 tx=0x7f6b0802b960 comp rx=0 tx=0).stop 2024-03-28T13:19:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.645+0000 7f6b23000640 1 -- 172.21.15.19:0/156626244 shutdown_connections 2024-03-28T13:19:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.645+0000 7f6b23000640 1 --2- 172.21.15.19:0/156626244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c06e030 0x7f6b1c15fd90 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.645+0000 7f6b23000640 1 -- 172.21.15.19:0/156626244 >> 172.21.15.19:0/156626244 conn(0x7f6b1c06b570 msgr2=0x7f6b1c06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.645+0000 7f6b23000640 1 -- 172.21.15.19:0/156626244 shutdown_connections 2024-03-28T13:19:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.645+0000 7f6b23000640 1 -- 172.21.15.19:0/156626244 wait complete. 2024-03-28T13:19:09.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.647+0000 7f6b23000640 1 Processor -- start 2024-03-28T13:19:09.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.647+0000 7f6b23000640 1 -- start start 2024-03-28T13:19:09.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.648+0000 7f6b23000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c06e030 0x7f6b1c13cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:09.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.648+0000 7f6b23000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b1c13f300 0x7f6b1c0aa850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:09.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.648+0000 7f6b23000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c1602d0 0x7f6b1c072e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:09.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.648+0000 7f6b23000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c0cfa30 con 0x7f6b1c1602d0 2024-03-28T13:19:09.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.648+0000 7f6b23000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c0cee90 con 0x7f6b1c06e030 2024-03-28T13:19:09.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.648+0000 7f6b23000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1c0cfbb0 con 0x7f6b1c13f300 2024-03-28T13:19:09.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.649+0000 7f6b22600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c1602d0 0x7f6b1c072e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:09.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.649+0000 7f6b22600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c1602d0 0x7f6b1c072e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58482/0 (socket says 172.21.15.19:58482) 2024-03-28T13:19:09.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.649+0000 7f6b22600640 1 -- 172.21.15.19:0/1635029609 learned_addr learned my addr 172.21.15.19:0/1635029609 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:09.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.649+0000 7f6b21200640 1 --2- 172.21.15.19:0/1635029609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b1c13f300 0x7f6b1c0aa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:09.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.649+0000 7f6b21c00640 1 --2- 172.21.15.19:0/1635029609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c06e030 0x7f6b1c13cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:09.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.649+0000 7f6b22600640 1 -- 172.21.15.19:0/1635029609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b1c13f300 msgr2=0x7f6b1c0aa850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:09.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.649+0000 7f6b22600640 1 --2- 172.21.15.19:0/1635029609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b1c13f300 0x7f6b1c0aa850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:09.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.649+0000 7f6b22600640 1 -- 172.21.15.19:0/1635029609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c06e030 msgr2=0x7f6b1c13cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:09.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.649+0000 7f6b22600640 1 --2- 172.21.15.19:0/1635029609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c06e030 0x7f6b1c13cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:09.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.649+0000 7f6b22600640 1 -- 172.21.15.19:0/1635029609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c0b1b30 con 0x7f6b1c1602d0 2024-03-28T13:19:09.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.649+0000 7f6b21c00640 1 --2- 172.21.15.19:0/1635029609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c06e030 0x7f6b1c13cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:19:09.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.650+0000 7f6b22600640 1 --2- 172.21.15.19:0/1635029609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c1602d0 0x7f6b1c072e20 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400c700 tx=0x7f6b1400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:09.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.650+0000 7f6b19600640 1 -- 172.21.15.19:0/1635029609 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14018070 con 0x7f6b1c1602d0 2024-03-28T13:19:09.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.650+0000 7f6b19600640 1 -- 172.21.15.19:0/1635029609 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1400ce10 con 0x7f6b1c1602d0 2024-03-28T13:19:09.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.650+0000 7f6b19600640 1 -- 172.21.15.19:0/1635029609 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400a860 con 0x7f6b1c1602d0 2024-03-28T13:19:09.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.650+0000 7f6b23000640 1 -- 172.21.15.19:0/1635029609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c0b1e10 con 0x7f6b1c1602d0 2024-03-28T13:19:09.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.650+0000 7f6b23000640 1 -- 172.21.15.19:0/1635029609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c0b23d0 con 0x7f6b1c1602d0 2024-03-28T13:19:09.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.651+0000 7f6b23000640 1 -- 172.21.15.19:0/1635029609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1c075940 con 0x7f6b1c1602d0 2024-03-28T13:19:09.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.652+0000 7f6b19600640 1 -- 172.21.15.19:0/1635029609 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6b1401b070 con 0x7f6b1c1602d0 2024-03-28T13:19:09.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.653+0000 7f6b19600640 1 --2- 172.21.15.19:0/1635029609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6afc06c530 0x7f6afc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:09.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.653+0000 7f6b19600640 1 -- 172.21.15.19:0/1635029609 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f6b1408ace0 con 0x7f6b1c1602d0 2024-03-28T13:19:09.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.654+0000 7f6b21c00640 1 --2- 172.21.15.19:0/1635029609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6afc06c530 0x7f6afc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:09.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.655+0000 7f6b21c00640 1 --2- 172.21.15.19:0/1635029609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6afc06c530 0x7f6afc06e9f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c1f7840 tx=0x7f6b10009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:09.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:09.657+0000 7f6b19600640 1 -- 172.21.15.19:0/1635029609 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6b140592f0 con 0x7f6b1c1602d0 2024-03-28T13:19:10.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.081+0000 7f6b23000640 1 -- 172.21.15.19:0/1635029609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6b1c074920 con 0x7f6b1c1602d0 2024-03-28T13:19:10.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.086+0000 7f6b19600640 1 -- 172.21.15.19:0/1635029609 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f6b14058bf0 con 0x7f6b1c1602d0 2024-03-28T13:19:10.086 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-03-28T13:19:10.086 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q== 2024-03-28T13:19:10.086 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-03-28T13:19:10.086 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-03-28T13:19:10.086 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-28T13:19:10.087 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-03-28T13:19:10.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.088+0000 7f6b23000640 1 -- 172.21.15.19:0/1635029609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6afc06c530 msgr2=0x7f6afc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:10.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.088+0000 7f6b23000640 1 --2- 172.21.15.19:0/1635029609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6afc06c530 0x7f6afc06e9f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c1f7840 tx=0x7f6b10009040 comp rx=0 tx=0).stop 2024-03-28T13:19:10.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.088+0000 7f6b23000640 1 -- 172.21.15.19:0/1635029609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c1602d0 msgr2=0x7f6b1c072e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:10.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.088+0000 7f6b23000640 1 --2- 172.21.15.19:0/1635029609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c1602d0 0x7f6b1c072e20 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400c700 tx=0x7f6b1400cbc0 comp rx=0 tx=0).stop 2024-03-28T13:19:10.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.089+0000 7f6b23000640 1 -- 172.21.15.19:0/1635029609 shutdown_connections 2024-03-28T13:19:10.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.089+0000 7f6b23000640 1 --2- 172.21.15.19:0/1635029609 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6afc06c530 0x7f6afc06e9f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:10.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.089+0000 7f6b23000640 1 --2- 172.21.15.19:0/1635029609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c1602d0 0x7f6b1c072e20 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:10.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.089+0000 7f6b23000640 1 --2- 172.21.15.19:0/1635029609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b1c13f300 0x7f6b1c0aa850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:10.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.089+0000 7f6b23000640 1 --2- 172.21.15.19:0/1635029609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b1c06e030 0x7f6b1c13cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:10.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.089+0000 7f6b23000640 1 -- 172.21.15.19:0/1635029609 >> 172.21.15.19:0/1635029609 conn(0x7f6b1c06b570 msgr2=0x7f6b1c0cffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:10.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.089+0000 7f6b23000640 1 -- 172.21.15.19:0/1635029609 shutdown_connections 2024-03-28T13:19:10.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.089+0000 7f6b23000640 1 -- 172.21.15.19:0/1635029609 wait complete. 2024-03-28T13:19:10.116 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:19:10.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.358+0000 7f4b38a00640 1 Processor -- start 2024-03-28T13:19:10.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.358+0000 7f4b38a00640 1 -- start start 2024-03-28T13:19:10.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.361+0000 7f4b38a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4b3415e3b0 0x7f4b340d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:10.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.361+0000 7f4b38a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4b340b63c0 0x7f4b340b7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:10.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.361+0000 7f4b38a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b340d2cf0 con 0x7f4b340d3d90 2024-03-28T13:19:10.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.361+0000 7f4b38a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4b340d2fd0 con 0x7f4b3415e3b0 2024-03-28T13:19:10.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.361+0000 7f4b38a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f4b340d32b0 con 0x7f4b340b63c0 2024-03-28T13:19:10.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.361+0000 7f4b33400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4b3415e3b0 0x7f4b340d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:10.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.361+0000 7f4b33400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4b3415e3b0 0x7f4b340d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58488/0 (socket says 172.21.15.19:58488) 2024-03-28T13:19:10.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.361+0000 7f4b33400640 1 -- 172.21.15.19:0/1057018040 learned_addr learned my addr 172.21.15.19:0/1057018040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:10.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.362+0000 7f4b33e00640 1 --2- 172.21.15.19:0/1057018040 >> v2:172.21.15.188:3300/0 conn(0x7f4b340b63c0 0x7f4b340b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:10.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.362+0000 7f4b33400640 1 -- 172.21.15.19:0/1057018040 >> v2:172.21.15.188:3300/0 conn(0x7f4b340b63c0 msgr2=0x7f4b340b7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:10.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.362+0000 7f4b33400640 1 --2- 172.21.15.19:0/1057018040 >> v2:172.21.15.188:3300/0 conn(0x7f4b340b63c0 0x7f4b340b7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:10.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.362+0000 7f4b33400640 1 -- 172.21.15.19:0/1057018040 >> v1:172.21.15.188:6789/0 conn(0x7f4b340d3d90 legacy=0x7f4b340b5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:10.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.362+0000 7f4b33400640 1 -- 172.21.15.19:0/1057018040 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b340d3590 con 0x7f4b3415e3b0 2024-03-28T13:19:10.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.363+0000 7f4b33400640 1 --2- 172.21.15.19:0/1057018040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b3415e3b0 0x7f4b340d22e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009870 tx=0x7f4b2402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf859b235b106f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:10.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.363+0000 7f4b32a00640 1 -- 172.21.15.19:0/1057018040 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24034070 con 0x7f4b3415e3b0 2024-03-28T13:19:10.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.363+0000 7f4b32a00640 1 -- 172.21.15.19:0/1057018040 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b24030860 con 0x7f4b3415e3b0 2024-03-28T13:19:10.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.363+0000 7f4b32a00640 1 -- 172.21.15.19:0/1057018040 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b24030c30 con 0x7f4b3415e3b0 2024-03-28T13:19:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.364+0000 7f4b38a00640 1 -- 172.21.15.19:0/1057018040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b3415e3b0 msgr2=0x7f4b340d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.364+0000 7f4b38a00640 1 --2- 172.21.15.19:0/1057018040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b3415e3b0 0x7f4b340d22e0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009870 tx=0x7f4b2402b9a0 comp rx=0 tx=0).stop 2024-03-28T13:19:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.364+0000 7f4b38a00640 1 -- 172.21.15.19:0/1057018040 shutdown_connections 2024-03-28T13:19:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.364+0000 7f4b38a00640 1 --2- 172.21.15.19:0/1057018040 >> v2:172.21.15.188:3300/0 conn(0x7f4b340b63c0 0x7f4b340b7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.364+0000 7f4b38a00640 1 --2- 172.21.15.19:0/1057018040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b3415e3b0 0x7f4b340d22e0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.364+0000 7f4b38a00640 1 -- 172.21.15.19:0/1057018040 >> 172.21.15.19:0/1057018040 conn(0x7f4b340678b0 msgr2=0x7f4b340d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.364+0000 7f4b38a00640 1 -- 172.21.15.19:0/1057018040 shutdown_connections 2024-03-28T13:19:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.364+0000 7f4b38a00640 1 -- 172.21.15.19:0/1057018040 wait complete. 2024-03-28T13:19:10.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.367+0000 7f4b38a00640 1 Processor -- start 2024-03-28T13:19:10.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.367+0000 7f4b38a00640 1 -- start start 2024-03-28T13:19:10.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.369+0000 7f4b38a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b340b63c0 0x7f4b3414b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:10.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.369+0000 7f4b38a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b340d3d90 0x7f4b340cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:10.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.369+0000 7f4b38a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b3415e3b0 0x7f4b340c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:10.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.369+0000 7f4b38a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b340ba030 con 0x7f4b3415e3b0 2024-03-28T13:19:10.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.369+0000 7f4b38a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4b3415e790 con 0x7f4b340b63c0 2024-03-28T13:19:10.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.369+0000 7f4b38a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4b340ba1b0 con 0x7f4b340d3d90 2024-03-28T13:19:10.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.370+0000 7f4b33400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b340b63c0 0x7f4b3414b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:10.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.370+0000 7f4b33400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b340b63c0 0x7f4b3414b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:60834/0 (socket says 172.21.15.19:60834) 2024-03-28T13:19:10.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.370+0000 7f4b33400640 1 -- 172.21.15.19:0/3988507067 learned_addr learned my addr 172.21.15.19:0/3988507067 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:10.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.370+0000 7f4b33e00640 1 --2- 172.21.15.19:0/3988507067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b3415e3b0 0x7f4b340c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:10.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.370+0000 7f4b2be00640 1 --2- 172.21.15.19:0/3988507067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b340d3d90 0x7f4b340cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:10.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.371+0000 7f4b33e00640 1 -- 172.21.15.19:0/3988507067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b340d3d90 msgr2=0x7f4b340cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:10.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.371+0000 7f4b33e00640 1 --2- 172.21.15.19:0/3988507067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b340d3d90 0x7f4b340cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:10.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.371+0000 7f4b33e00640 1 -- 172.21.15.19:0/3988507067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b340b63c0 msgr2=0x7f4b3414b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:10.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.371+0000 7f4b33e00640 1 --2- 172.21.15.19:0/3988507067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b340b63c0 0x7f4b3414b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:10.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.371+0000 7f4b33e00640 1 -- 172.21.15.19:0/3988507067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b340cc9c0 con 0x7f4b3415e3b0 2024-03-28T13:19:10.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.371+0000 7f4b33e00640 1 --2- 172.21.15.19:0/3988507067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b3415e3b0 0x7f4b340c9900 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00ccb0 tx=0x7f4b2c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:10.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.371+0000 7f4b2b400640 1 -- 172.21.15.19:0/3988507067 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c003b90 con 0x7f4b3415e3b0 2024-03-28T13:19:10.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.371+0000 7f4b2b400640 1 -- 172.21.15.19:0/3988507067 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b2c003d30 con 0x7f4b3415e3b0 2024-03-28T13:19:10.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.371+0000 7f4b2b400640 1 -- 172.21.15.19:0/3988507067 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c004240 con 0x7f4b3415e3b0 2024-03-28T13:19:10.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.371+0000 7f4b38a00640 1 -- 172.21.15.19:0/3988507067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b340ccca0 con 0x7f4b3415e3b0 2024-03-28T13:19:10.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.372+0000 7f4b38a00640 1 -- 172.21.15.19:0/3988507067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b340cd260 con 0x7f4b3415e3b0 2024-03-28T13:19:10.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.373+0000 7f4b38a00640 1 -- 172.21.15.19:0/3988507067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b34075940 con 0x7f4b3415e3b0 2024-03-28T13:19:10.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.375+0000 7f4b2b400640 1 -- 172.21.15.19:0/3988507067 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4b2c0043e0 con 0x7f4b3415e3b0 2024-03-28T13:19:10.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.376+0000 7f4b2b400640 1 --2- 172.21.15.19:0/3988507067 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4b1006c560 0x7f4b1006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:10.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.376+0000 7f4b2b400640 1 -- 172.21.15.19:0/3988507067 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f4b2c08c070 con 0x7f4b3415e3b0 2024-03-28T13:19:10.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.376+0000 7f4b33400640 1 --2- 172.21.15.19:0/3988507067 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4b1006c560 0x7f4b1006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:10.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.378+0000 7f4b33400640 1 --2- 172.21.15.19:0/3988507067 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4b1006c560 0x7f4b1006ea20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4b2402b9e0 tx=0x7f4b2403b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:10.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.380+0000 7f4b2b400640 1 -- 172.21.15.19:0/3988507067 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4b2c059fa0 con 0x7f4b3415e3b0 2024-03-28T13:19:10.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.626+0000 7f4b38a00640 1 -- 172.21.15.19:0/3988507067 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b340c7670 con 0x7f4b3415e3b0 2024-03-28T13:19:10.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.627+0000 7f4b2b400640 1 -- 172.21.15.19:0/3988507067 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5189 (secure 0 0 0) 0x7f4b2c05e2a0 con 0x7f4b3415e3b0 2024-03-28T13:19:10.627 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:10.627 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4247326257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4247326257},{"type":"v1","addr":"172.21.15.188:6836","nonce":4247326257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3452274156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3452274156},{"type":"v1","addr":"172.21.15.19:6835","nonce":3452274156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3383190607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3383190607},{"type":"v1","addr":"172.21.15.188:6837","nonce":3383190607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:18:58.541599+0000","modified":"2024-03-28T13:19:01.614319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/1020599987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1020599987},{"type":"v1","addr":"172.21.15.19:6834","nonce":1020599987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8040,"qdb_cluster":[8040]},"id":7}]} 2024-03-28T13:19:10.627 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 83 2024-03-28T13:19:10.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.630+0000 7f4b38a00640 1 -- 172.21.15.19:0/3988507067 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4b1006c560 msgr2=0x7f4b1006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:10.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.630+0000 7f4b38a00640 1 --2- 172.21.15.19:0/3988507067 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4b1006c560 0x7f4b1006ea20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4b2402b9e0 tx=0x7f4b2403b040 comp rx=0 tx=0).stop 2024-03-28T13:19:10.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.630+0000 7f4b38a00640 1 -- 172.21.15.19:0/3988507067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b3415e3b0 msgr2=0x7f4b340c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:10.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.630+0000 7f4b38a00640 1 --2- 172.21.15.19:0/3988507067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b3415e3b0 0x7f4b340c9900 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00ccb0 tx=0x7f4b2c00a660 comp rx=0 tx=0).stop 2024-03-28T13:19:10.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.630+0000 7f4b38a00640 1 -- 172.21.15.19:0/3988507067 shutdown_connections 2024-03-28T13:19:10.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.630+0000 7f4b38a00640 1 --2- 172.21.15.19:0/3988507067 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4b1006c560 0x7f4b1006ea20 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:10.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.630+0000 7f4b38a00640 1 --2- 172.21.15.19:0/3988507067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b3415e3b0 0x7f4b340c9900 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:10.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.630+0000 7f4b38a00640 1 --2- 172.21.15.19:0/3988507067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b340d3d90 0x7f4b340cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:10.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.630+0000 7f4b38a00640 1 --2- 172.21.15.19:0/3988507067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b340b63c0 0x7f4b3414b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:10.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.630+0000 7f4b38a00640 1 -- 172.21.15.19:0/3988507067 >> 172.21.15.19:0/3988507067 conn(0x7f4b340678b0 msgr2=0x7f4b340b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:10.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.630+0000 7f4b38a00640 1 -- 172.21.15.19:0/3988507067 shutdown_connections 2024-03-28T13:19:10.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:10.631+0000 7f4b38a00640 1 -- 172.21.15.19:0/3988507067 wait complete. 2024-03-28T13:19:11.052 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:18:58.541599+0000', 'modified': '2024-03-28T13:19:01.614319+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8040}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8040': {'gid': 8040, 'name': 'c', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.19:6834/1020599987', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 1020599987}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 1020599987}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8040, 'qdb_cluster': [8040]} 2024-03-28T13:19:11.052 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T13:19:11.052 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-28T13:19:11.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.299+0000 7f07a9800640 1 Processor -- start 2024-03-28T13:19:11.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.299+0000 7f07a9800640 1 -- start start 2024-03-28T13:19:11.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.300+0000 7f07a9800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f07a415e860 0x7f07a406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.301+0000 7f07a9800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07a406dfa0 con 0x7f07a40d4ae0 2024-03-28T13:19:11.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.301+0000 7f07a9800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07a406e240 con 0x7f07a406da30 2024-03-28T13:19:11.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.301+0000 7f07a9800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f07a40cf220 con 0x7f07a415e860 2024-03-28T13:19:11.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.301+0000 7f07a2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f07a415e860 0x7f07a406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.301+0000 7f07a3400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f07a40d4ae0 0x7f07a40cf6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45082/0 (socket says 172.21.15.19:45082) 2024-03-28T13:19:11.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.301+0000 7f07a3400640 1 -- 172.21.15.19:0/872809615 learned_addr learned my addr 172.21.15.19:0/872809615 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:11.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.301+0000 7f07a2a00640 1 -- 172.21.15.19:0/872809615 >> v1:172.21.15.188:6789/0 conn(0x7f07a406da30 legacy=0x7f07a40d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:11.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.301+0000 7f07a2a00640 1 -- 172.21.15.19:0/872809615 >> v1:172.21.15.19:6789/0 conn(0x7f07a40d4ae0 legacy=0x7f07a40cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.301+0000 7f07a2a00640 1 -- 172.21.15.19:0/872809615 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a406e520 con 0x7f07a415e860 2024-03-28T13:19:11.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.301+0000 7f07a2000640 1 -- 172.21.15.19:0/872809615 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1333027332 0 0) 0x7f07a406dfa0 con 0x7f07a40d4ae0 2024-03-28T13:19:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.303+0000 7f07a2a00640 1 --2- 172.21.15.19:0/872809615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a415e860 0x7f07a406d4f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f07940097a0 tx=0x7f079402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b18cb38a5295756 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.303+0000 7f07a2000640 1 -- 172.21.15.19:0/872809615 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794035020 con 0x7f07a415e860 2024-03-28T13:19:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.303+0000 7f07a2000640 1 -- 172.21.15.19:0/872809615 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0794031830 con 0x7f07a415e860 2024-03-28T13:19:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.303+0000 7f07a2000640 1 -- 172.21.15.19:0/872809615 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794031c00 con 0x7f07a415e860 2024-03-28T13:19:11.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.303+0000 7f07a9800640 1 -- 172.21.15.19:0/872809615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a415e860 msgr2=0x7f07a406d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.303+0000 7f07a9800640 1 --2- 172.21.15.19:0/872809615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a415e860 0x7f07a406d4f0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f07940097a0 tx=0x7f079402b8d0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.304+0000 7f07a9800640 1 -- 172.21.15.19:0/872809615 shutdown_connections 2024-03-28T13:19:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.304+0000 7f07a9800640 1 --2- 172.21.15.19:0/872809615 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a415e860 0x7f07a406d4f0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.304+0000 7f07a9800640 1 -- 172.21.15.19:0/872809615 >> 172.21.15.19:0/872809615 conn(0x7f07a409d7d0 msgr2=0x7f07a408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.304+0000 7f07a9800640 1 -- 172.21.15.19:0/872809615 shutdown_connections 2024-03-28T13:19:11.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.304+0000 7f07a9800640 1 -- 172.21.15.19:0/872809615 wait complete. 2024-03-28T13:19:11.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.305+0000 7f07a9800640 1 Processor -- start 2024-03-28T13:19:11.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.306+0000 7f07a9800640 1 -- start start 2024-03-28T13:19:11.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.307+0000 7f07a9800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a406da30 0x7f07a4154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.307+0000 7f07a9800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07a40d4ae0 0x7f07a40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.307+0000 7f07a9800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07a415e860 0x7f07a4151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.307+0000 7f07a9800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07a40cef30 con 0x7f07a406da30 2024-03-28T13:19:11.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.307+0000 7f07a9800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07a40cedb0 con 0x7f07a40d4ae0 2024-03-28T13:19:11.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.307+0000 7f07a9800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f07a40cfdc0 con 0x7f07a415e860 2024-03-28T13:19:11.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.307+0000 7f07a2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a406da30 0x7f07a4154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.307+0000 7f07a2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a406da30 0x7f07a4154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58534/0 (socket says 172.21.15.19:58534) 2024-03-28T13:19:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.307+0000 7f07a2a00640 1 -- 172.21.15.19:0/3664665443 learned_addr learned my addr 172.21.15.19:0/3664665443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.307+0000 7f07a3400640 1 --2- 172.21.15.19:0/3664665443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07a415e860 0x7f07a4151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.307+0000 7f079be00640 1 --2- 172.21.15.19:0/3664665443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07a40d4ae0 0x7f07a40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.307+0000 7f07a2a00640 1 -- 172.21.15.19:0/3664665443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07a415e860 msgr2=0x7f07a4151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.307+0000 7f07a2a00640 1 --2- 172.21.15.19:0/3664665443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07a415e860 0x7f07a4151670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.307+0000 7f07a2a00640 1 -- 172.21.15.19:0/3664665443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07a40d4ae0 msgr2=0x7f07a40cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.307+0000 7f07a2a00640 1 --2- 172.21.15.19:0/3664665443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07a40d4ae0 0x7f07a40cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.308+0000 7f07a2a00640 1 -- 172.21.15.19:0/3664665443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a41520d0 con 0x7f07a406da30 2024-03-28T13:19:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.308+0000 7f07a2a00640 1 --2- 172.21.15.19:0/3664665443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a406da30 0x7f07a4154b60 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0794034040 tx=0x7f07940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.308+0000 7f079aa00640 1 -- 172.21.15.19:0/3664665443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794004230 con 0x7f07a406da30 2024-03-28T13:19:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.308+0000 7f079aa00640 1 -- 172.21.15.19:0/3664665443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07940043f0 con 0x7f07a406da30 2024-03-28T13:19:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.308+0000 7f079aa00640 1 -- 172.21.15.19:0/3664665443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794003850 con 0x7f07a406da30 2024-03-28T13:19:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.308+0000 7f07a9800640 1 -- 172.21.15.19:0/3664665443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a41523b0 con 0x7f07a406da30 2024-03-28T13:19:11.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.308+0000 7f07a9800640 1 -- 172.21.15.19:0/3664665443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a4152970 con 0x7f07a406da30 2024-03-28T13:19:11.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.311+0000 7f079aa00640 1 -- 172.21.15.19:0/3664665443 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0794038020 con 0x7f07a406da30 2024-03-28T13:19:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.312+0000 7f079aa00640 1 --2- 172.21.15.19:0/3664665443 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f078006c530 0x7f078006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.312+0000 7f079aa00640 1 -- 172.21.15.19:0/3664665443 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f07940ad390 con 0x7f07a406da30 2024-03-28T13:19:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.313+0000 7f079be00640 1 --2- 172.21.15.19:0/3664665443 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f078006c530 0x7f078006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.313+0000 7f07ac472740 1 Processor -- start 2024-03-28T13:19:11.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.313+0000 7f07ac472740 1 -- start start 2024-03-28T13:19:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.314+0000 7f07ac472740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 0x5579dc932f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.314+0000 7f07ac472740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 0x5579dc962d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.314+0000 7f07ac472740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9334b0 0x5579dc9632a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.314+0000 7f07ac472740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5579dc8fa920 con 0x5579dc9334b0 2024-03-28T13:19:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.314+0000 7f07ac472740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5579dc84afa0 con 0x5579dc95ffd0 2024-03-28T13:19:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.314+0000 7f07ac472740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5579dc89cf90 con 0x5579dc9603b0 2024-03-28T13:19:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.314+0000 7f079be00640 1 --2- 172.21.15.19:0/3664665443 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f078006c530 0x7f078006e9f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f07a40d1380 tx=0x7f078c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.314+0000 7f07a3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9334b0 0x5579dc9632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.314+0000 7f07a3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9334b0 0x5579dc9632a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58540/0 (socket says 172.21.15.19:58540) 2024-03-28T13:19:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.314+0000 7f07a3400640 1 -- 172.21.15.19:0/1142300998 learned_addr learned my addr 172.21.15.19:0/1142300998 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.315+0000 7f07a2a00640 1 --2- 172.21.15.19:0/1142300998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 0x5579dc932f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.315+0000 7f079be00640 1 --2- 172.21.15.19:0/1142300998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 0x5579dc962d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.315+0000 7f07a3400640 1 -- 172.21.15.19:0/1142300998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 msgr2=0x5579dc962d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.315+0000 7f07a3400640 1 --2- 172.21.15.19:0/1142300998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 0x5579dc962d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.315+0000 7f07a3400640 1 -- 172.21.15.19:0/1142300998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 msgr2=0x5579dc932f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.315+0000 7f07a3400640 1 --2- 172.21.15.19:0/1142300998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 0x5579dc932f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.315+0000 7f07a3400640 1 -- 172.21.15.19:0/1142300998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579dc5b12d0 con 0x5579dc9334b0 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.315+0000 7f07a2a00640 1 --2- 172.21.15.19:0/1142300998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 0x5579dc932f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.315+0000 7f07a3400640 1 --2- 172.21.15.19:0/1142300998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9334b0 0x5579dc9632a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f079c0072e0 tx=0x7f079c0077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a5de310ca8054b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.315+0000 7f0793400640 1 -- 172.21.15.19:0/1142300998 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c019070 con 0x5579dc9334b0 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.315+0000 7f0793400640 1 -- 172.21.15.19:0/1142300998 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079c004d70 con 0x5579dc9334b0 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.315+0000 7f0793400640 1 -- 172.21.15.19:0/1142300998 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c0079f0 con 0x5579dc9334b0 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.316+0000 7f07ac472740 1 -- 172.21.15.19:0/1142300998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9334b0 msgr2=0x5579dc9632a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.316+0000 7f07ac472740 1 --2- 172.21.15.19:0/1142300998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9334b0 0x5579dc9632a0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f079c0072e0 tx=0x7f079c0077a0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.316+0000 7f07ac472740 1 -- 172.21.15.19:0/1142300998 shutdown_connections 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.316+0000 7f07ac472740 1 --2- 172.21.15.19:0/1142300998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9334b0 0x5579dc9632a0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.316+0000 7f07ac472740 1 --2- 172.21.15.19:0/1142300998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 0x5579dc962d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.316+0000 7f07ac472740 1 --2- 172.21.15.19:0/1142300998 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 0x5579dc932f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.316+0000 7f07ac472740 1 -- 172.21.15.19:0/1142300998 >> 172.21.15.19:0/1142300998 conn(0x5579dc9351e0 msgr2=0x5579dc97dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:11.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.316+0000 7f07ac472740 1 -- 172.21.15.19:0/1142300998 shutdown_connections 2024-03-28T13:19:11.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.316+0000 7f07ac472740 1 -- 172.21.15.19:0/1142300998 wait complete. 2024-03-28T13:19:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.317+0000 7f07ac472740 1 -- start start 2024-03-28T13:19:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.317+0000 7f07ac472740 1 Processor -- start 2024-03-28T13:19:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.318+0000 7f07ac472740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 0x5579dc949f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.318+0000 7f07ac472740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 0x5579dc961e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.318+0000 7f07ac472740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9623b0 0x5579dc974880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.318+0000 7f07ac472740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5579dc898f40 con 0x5579dc9623b0 2024-03-28T13:19:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.318+0000 7f07ac472740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5579dc845650 con 0x5579dc95ffd0 2024-03-28T13:19:11.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.318+0000 7f07ac472740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5579dc858a10 con 0x5579dc9603b0 2024-03-28T13:19:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.318+0000 7f07ac472740 10 client.0 _ll_register_callbacks cb 0x5579dc897ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:19:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.319+0000 7f07a3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9623b0 0x5579dc974880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.319+0000 7f07a3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9623b0 0x5579dc974880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58542/0 (socket says 172.21.15.19:58542) 2024-03-28T13:19:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.319+0000 7f07a3400640 1 -- 172.21.15.19:0/223050703 learned_addr learned my addr 172.21.15.19:0/223050703 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.319+0000 7f079be00640 1 --2- 172.21.15.19:0/223050703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 0x5579dc961e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.319+0000 7f07a2a00640 1 --2- 172.21.15.19:0/223050703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 0x5579dc949f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.319+0000 7f07a3400640 1 -- 172.21.15.19:0/223050703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 msgr2=0x5579dc961e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.319+0000 7f07a3400640 1 --2- 172.21.15.19:0/223050703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 0x5579dc961e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.319+0000 7f07a3400640 1 -- 172.21.15.19:0/223050703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 msgr2=0x5579dc949f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.319+0000 7f07a3400640 1 --2- 172.21.15.19:0/223050703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 0x5579dc949f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.319+0000 7f07a3400640 1 -- 172.21.15.19:0/223050703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579dc67b6b0 con 0x5579dc9623b0 2024-03-28T13:19:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.319+0000 7f07a2a00640 1 --2- 172.21.15.19:0/223050703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 0x5579dc949f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.319+0000 7f07a3400640 1 --2- 172.21.15.19:0/223050703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9623b0 0x5579dc974880 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f079c012000 tx=0x7f079c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d689f09a9a0208b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:11.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.319+0000 7f0787e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:19:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.320+0000 7f0787e00640 1 -- 172.21.15.19:0/223050703 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c019040 con 0x5579dc9623b0 2024-03-28T13:19:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.320+0000 7f07ac472740 1 -- 172.21.15.19:0/223050703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f079c004d70 con 0x5579dc9623b0 2024-03-28T13:19:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.320+0000 7f0787e00640 1 -- 172.21.15.19:0/223050703 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f079c014450 con 0x5579dc9623b0 2024-03-28T13:19:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.320+0000 7f0787e00640 1 -- 172.21.15.19:0/223050703 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f079c02b050 con 0x5579dc9623b0 2024-03-28T13:19:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.320+0000 7f07ac472740 10 client.8190 fetch_fsmap learned FSMap version 83 2024-03-28T13:19:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.320+0000 7f07ac472740 1 -- 172.21.15.19:0/223050703 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x5579dc94af60 con 0x5579dc9623b0 2024-03-28T13:19:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.321+0000 7f0787e00640 1 -- 172.21.15.19:0/223050703 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f079c004d70 con 0x5579dc9623b0 2024-03-28T13:19:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.321+0000 7f0787e00640 1 -- 172.21.15.19:0/223050703 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f079c02e020 con 0x5579dc9623b0 2024-03-28T13:19:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.321+0000 7f07ac472740 10 client.8190 fetch_fsmap finished waiting for FSMap version 83 2024-03-28T13:19:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.321+0000 7f07ac472740 10 client.8190 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T13:19:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.321+0000 7f07ac472740 20 client.8190 populate_metadata read hostname 'smithi019' 2024-03-28T13:19:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.321+0000 7f07ac472740 1 --2- 172.21.15.19:0/223050703 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5579dc94bc30 0x5579dc94ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.321+0000 7f07ac472740 4 client.8190 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:19:11.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.321+0000 7f07ac472740 1 -- 172.21.15.19:0/223050703 --> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579dc8aa4d0 con 0x5579dc94bc30 2024-03-28T13:19:11.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.321+0000 7f07a2a00640 1 --2- 172.21.15.19:0/223050703 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5579dc94bc30 0x5579dc94ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.322+0000 7f07a2a00640 1 --2- 172.21.15.19:0/223050703 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5579dc94bc30 0x5579dc94ce30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f36772ee4bdef71 server_cookie=54ec4d1af7c47883 in_seq=0 out_seq=0 2024-03-28T13:19:11.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.322+0000 7f0787e00640 10 client.8190 ms_handle_connect on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:11.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.323+0000 7f0787e00640 1 -- 172.21.15.19:0/223050703 <== mds.0 v2:172.21.15.19:6832/1020599987 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579dc8aa4d0 con 0x5579dc94bc30 2024-03-28T13:19:11.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.323+0000 7f0787e00640 10 client.8190 handle_command_reply: tid=0 2024-03-28T13:19:11.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.323+0000 7f07ac472740 1 client.8190 shutdown 2024-03-28T13:19:11.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.323+0000 7f07ac472740 1 -- 172.21.15.19:0/223050703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9623b0 msgr2=0x5579dc974880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.323+0000 7f07ac472740 1 --2- 172.21.15.19:0/223050703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9623b0 0x5579dc974880 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f079c012000 tx=0x7f079c007d90 comp rx=0 tx=0).stop 2024-03-28T13:19:11.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.323+0000 7f07ac472740 1 -- 172.21.15.19:0/223050703 shutdown_connections 2024-03-28T13:19:11.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.323+0000 7f07ac472740 1 --2- 172.21.15.19:0/223050703 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5579dc94bc30 0x5579dc94ce30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.323+0000 7f07ac472740 1 --2- 172.21.15.19:0/223050703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9623b0 0x5579dc974880 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.323+0000 7f07ac472740 1 --2- 172.21.15.19:0/223050703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 0x5579dc961e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.323+0000 7f07ac472740 1 --2- 172.21.15.19:0/223050703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 0x5579dc949f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.323+0000 7f07ac472740 1 -- 172.21.15.19:0/223050703 >> 172.21.15.19:0/223050703 conn(0x5579dc9334b0 msgr2=0x5579dc95f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.323+0000 7f0787e00640 0 client.8190 ms_handle_reset on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.323+0000 7f07ac472740 1 -- 172.21.15.19:0/223050703 shutdown_connections 2024-03-28T13:19:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.323+0000 7f07ac472740 1 -- 172.21.15.19:0/223050703 wait complete. 2024-03-28T13:19:11.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.323+0000 7f07ac472740 20 client.8190 trim_cache size 0 max 16384 2024-03-28T13:19:11.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.343+0000 7f07ac472740 1 Processor -- start 2024-03-28T13:19:11.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.343+0000 7f07ac472740 1 -- start start 2024-03-28T13:19:11.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.344+0000 7f07ac472740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc95ffd0 0x5579dc9a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.344+0000 7f07ac472740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc9603b0 0x5579dc936ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.344+0000 7f07ac472740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9623b0 0x5579dc937010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.344+0000 7f07ac472740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41ffbb0 con 0x5579dc95ffd0 2024-03-28T13:19:11.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.344+0000 7f07ac472740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07a4153170 con 0x5579dc9603b0 2024-03-28T13:19:11.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.344+0000 7f07ac472740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f07a41537c0 con 0x5579dc9623b0 2024-03-28T13:19:11.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.344+0000 7f07a2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc95ffd0 0x5579dc9a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.345+0000 7f07a2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc95ffd0 0x5579dc9a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58544/0 (socket says 172.21.15.19:58544) 2024-03-28T13:19:11.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.345+0000 7f07a2a00640 1 -- 172.21.15.19:0/1610806972 learned_addr learned my addr 172.21.15.19:0/1610806972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:11.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.345+0000 7f07a2a00640 1 -- 172.21.15.19:0/1610806972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9623b0 msgr2=0x5579dc937010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:11.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.345+0000 7f07a2a00640 1 --2- 172.21.15.19:0/1610806972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9623b0 0x5579dc937010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.345+0000 7f07a2a00640 1 -- 172.21.15.19:0/1610806972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc9603b0 msgr2=0x5579dc936ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:11.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.345+0000 7f07a2a00640 1 --2- 172.21.15.19:0/1610806972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc9603b0 0x5579dc936ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.345+0000 7f07a2a00640 1 -- 172.21.15.19:0/1610806972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579dc932580 con 0x5579dc95ffd0 2024-03-28T13:19:11.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.345+0000 7f07a2a00640 1 --2- 172.21.15.19:0/1610806972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc95ffd0 0x5579dc9a18a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f07940097a0 tx=0x7f07940ab6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78199132ccb2d52e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:11.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.345+0000 7f0792a00640 1 -- 172.21.15.19:0/1610806972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794079220 con 0x5579dc95ffd0 2024-03-28T13:19:11.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.345+0000 7f0792a00640 1 -- 172.21.15.19:0/1610806972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07940a6070 con 0x5579dc95ffd0 2024-03-28T13:19:11.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.345+0000 7f0792a00640 1 -- 172.21.15.19:0/1610806972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794079480 con 0x5579dc95ffd0 2024-03-28T13:19:11.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.346+0000 7f07ac472740 1 -- 172.21.15.19:0/1610806972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc95ffd0 msgr2=0x5579dc9a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.346+0000 7f07ac472740 1 --2- 172.21.15.19:0/1610806972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc95ffd0 0x5579dc9a18a0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f07940097a0 tx=0x7f07940ab6f0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.346+0000 7f07ac472740 1 -- 172.21.15.19:0/1610806972 shutdown_connections 2024-03-28T13:19:11.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.346+0000 7f07ac472740 1 --2- 172.21.15.19:0/1610806972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9623b0 0x5579dc937010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.346+0000 7f07ac472740 1 --2- 172.21.15.19:0/1610806972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc9603b0 0x5579dc936ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.346+0000 7f07ac472740 1 --2- 172.21.15.19:0/1610806972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc95ffd0 0x5579dc9a18a0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.346+0000 7f07ac472740 1 -- 172.21.15.19:0/1610806972 >> 172.21.15.19:0/1610806972 conn(0x5579dc9334b0 msgr2=0x5579dc95f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:11.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.346+0000 7f07ac472740 1 -- 172.21.15.19:0/1610806972 shutdown_connections 2024-03-28T13:19:11.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.346+0000 7f07ac472740 1 -- 172.21.15.19:0/1610806972 wait complete. 2024-03-28T13:19:11.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.347+0000 7f07ac472740 1 -- start start 2024-03-28T13:19:11.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.348+0000 7f07ac472740 1 Processor -- start 2024-03-28T13:19:11.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.349+0000 7f07ac472740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 0x5579dc937d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.349+0000 7f07ac472740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 0x5579dc981270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.349+0000 7f07ac472740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9623b0 0x5579dc963210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.350+0000 7f07ac472740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5579dc964410 con 0x5579dc9623b0 2024-03-28T13:19:11.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.350+0000 7f07ac472740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5579dc94b240 con 0x5579dc95ffd0 2024-03-28T13:19:11.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.350+0000 7f07ac472740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5579dc94c100 con 0x5579dc9603b0 2024-03-28T13:19:11.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.350+0000 7f07ac472740 10 client.0 _ll_register_callbacks cb 0x7f079c017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:19:11.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.350+0000 7f07a3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9623b0 0x5579dc963210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.350+0000 7f07a3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9623b0 0x5579dc963210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58554/0 (socket says 172.21.15.19:58554) 2024-03-28T13:19:11.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.350+0000 7f07a3400640 1 -- 172.21.15.19:0/2682629443 learned_addr learned my addr 172.21.15.19:0/2682629443 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:11.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.350+0000 7f07a2a00640 1 --2- 172.21.15.19:0/2682629443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 0x5579dc937d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.350+0000 7f079be00640 1 --2- 172.21.15.19:0/2682629443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 0x5579dc981270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.350+0000 7f07a3400640 1 -- 172.21.15.19:0/2682629443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 msgr2=0x5579dc981270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:11.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.350+0000 7f07a3400640 1 --2- 172.21.15.19:0/2682629443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 0x5579dc981270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.350+0000 7f07a3400640 1 -- 172.21.15.19:0/2682629443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 msgr2=0x5579dc937d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:11.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.350+0000 7f07a3400640 1 --2- 172.21.15.19:0/2682629443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 0x5579dc937d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.350+0000 7f07a3400640 1 -- 172.21.15.19:0/2682629443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579dc95b040 con 0x5579dc9623b0 2024-03-28T13:19:11.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.350+0000 7f079be00640 1 --2- 172.21.15.19:0/2682629443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 0x5579dc981270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:11.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.351+0000 7f07a3400640 1 --2- 172.21.15.19:0/2682629443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9623b0 0x5579dc963210 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f079c007b60 tx=0x7f079c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7627961a697a9c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:11.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.351+0000 7f0787e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:19:11.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.351+0000 7f0787e00640 1 -- 172.21.15.19:0/2682629443 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c019070 con 0x5579dc9623b0 2024-03-28T13:19:11.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.351+0000 7f0787e00640 1 -- 172.21.15.19:0/2682629443 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f079c027430 con 0x5579dc9623b0 2024-03-28T13:19:11.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.351+0000 7f07ac472740 1 -- 172.21.15.19:0/2682629443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07940a6070 con 0x5579dc9623b0 2024-03-28T13:19:11.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.351+0000 7f0787e00640 1 -- 172.21.15.19:0/2682629443 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f079c03e050 con 0x5579dc9623b0 2024-03-28T13:19:11.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.351+0000 7f07ac472740 10 client.8196 fetch_fsmap learned FSMap version 83 2024-03-28T13:19:11.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.351+0000 7f07ac472740 1 -- 172.21.15.19:0/2682629443 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x5579dc937a40 con 0x5579dc9623b0 2024-03-28T13:19:11.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.352+0000 7f0787e00640 1 -- 172.21.15.19:0/2682629443 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07940a6070 con 0x5579dc9623b0 2024-03-28T13:19:11.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.352+0000 7f0787e00640 1 -- 172.21.15.19:0/2682629443 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f079c041020 con 0x5579dc9623b0 2024-03-28T13:19:11.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.352+0000 7f07ac472740 10 client.8196 fetch_fsmap finished waiting for FSMap version 83 2024-03-28T13:19:11.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.352+0000 7f07ac472740 10 client.8196 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T13:19:11.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.352+0000 7f07ac472740 20 client.8196 populate_metadata read hostname 'smithi019' 2024-03-28T13:19:11.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.352+0000 7f07ac472740 1 --2- 172.21.15.19:0/2682629443 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5579dc963870 0x5579dc948820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.352+0000 7f07ac472740 4 client.8196 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T13:19:11.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.352+0000 7f07ac472740 1 -- 172.21.15.19:0/2682629443 --> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] -- command(tid 0: {"prefix": "status"}) -- 0x5579dc8a4820 con 0x5579dc963870 2024-03-28T13:19:11.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.352+0000 7f07a2a00640 1 --2- 172.21.15.19:0/2682629443 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5579dc963870 0x5579dc948820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f07a2a00640 1 --2- 172.21.15.19:0/2682629443 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5579dc963870 0x5579dc948820 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b33a0bf882ea645d server_cookie=4002efec81a0a81f in_seq=0 out_seq=0 2024-03-28T13:19:11.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f0787e00640 10 client.8196 ms_handle_connect on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:11.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f0787e00640 1 -- 172.21.15.19:0/2682629443 <== mds.0 v2:172.21.15.19:6832/1020599987 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5579dc8a4820 con 0x5579dc963870 2024-03-28T13:19:11.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f0787e00640 10 client.8196 handle_command_reply: tid=0 2024-03-28T13:19:11.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f07ac472740 1 client.8196 shutdown 2024-03-28T13:19:11.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f07ac472740 1 -- 172.21.15.19:0/2682629443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9623b0 msgr2=0x5579dc963210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f07ac472740 1 --2- 172.21.15.19:0/2682629443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9623b0 0x5579dc963210 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f079c007b60 tx=0x7f079c001250 comp rx=0 tx=0).stop 2024-03-28T13:19:11.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f07ac472740 1 -- 172.21.15.19:0/2682629443 shutdown_connections 2024-03-28T13:19:11.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f07ac472740 1 --2- 172.21.15.19:0/2682629443 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5579dc963870 0x5579dc948820 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f07ac472740 1 --2- 172.21.15.19:0/2682629443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5579dc9623b0 0x5579dc963210 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f07ac472740 1 --2- 172.21.15.19:0/2682629443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5579dc9603b0 0x5579dc981270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f0787e00640 0 client.8196 ms_handle_reset on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:11.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f07ac472740 1 --2- 172.21.15.19:0/2682629443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5579dc95ffd0 0x5579dc937d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f07ac472740 1 -- 172.21.15.19:0/2682629443 >> 172.21.15.19:0/2682629443 conn(0x5579dc9334b0 msgr2=0x5579dc92e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:11.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f07ac472740 1 -- 172.21.15.19:0/2682629443 shutdown_connections 2024-03-28T13:19:11.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.353+0000 7f07ac472740 1 -- 172.21.15.19:0/2682629443 wait complete. 2024-03-28T13:19:11.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.354+0000 7f07ac472740 20 client.8196 trim_cache size 0 max 16384 2024-03-28T13:19:11.358 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:19:11.358 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "e6b569d9-f2ce-49dc-bbe7-c0ab33080203", 2024-03-28T13:19:11.358 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-03-28T13:19:11.358 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 8040, 2024-03-28T13:19:11.358 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-03-28T13:19:11.358 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-03-28T13:19:11.358 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-03-28T13:19:11.358 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 12.279390040999999, 2024-03-28T13:19:11.358 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 83, 2024-03-28T13:19:11.358 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 105, 2024-03-28T13:19:11.358 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 105, 2024-03-28T13:19:11.358 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 17.66208658 2024-03-28T13:19:11.359 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:19:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.354+0000 7f07a9800640 1 -- 172.21.15.19:0/3664665443 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f078006c530 msgr2=0x7f078006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.354+0000 7f07a9800640 1 --2- 172.21.15.19:0/3664665443 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f078006c530 0x7f078006e9f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f07a40d1380 tx=0x7f078c009040 comp rx=0 tx=0).stop 2024-03-28T13:19:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.354+0000 7f07a9800640 1 -- 172.21.15.19:0/3664665443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a406da30 msgr2=0x7f07a4154b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.354+0000 7f07a9800640 1 --2- 172.21.15.19:0/3664665443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a406da30 0x7f07a4154b60 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0794034040 tx=0x7f07940046c0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.354+0000 7f07a9800640 1 -- 172.21.15.19:0/3664665443 shutdown_connections 2024-03-28T13:19:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.354+0000 7f07a9800640 1 --2- 172.21.15.19:0/3664665443 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f078006c530 0x7f078006e9f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.354+0000 7f07a9800640 1 --2- 172.21.15.19:0/3664665443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07a415e860 0x7f07a4151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.354+0000 7f07a9800640 1 --2- 172.21.15.19:0/3664665443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07a40d4ae0 0x7f07a40cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.354+0000 7f07a9800640 1 --2- 172.21.15.19:0/3664665443 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07a406da30 0x7f07a4154b60 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.354+0000 7f07a9800640 1 -- 172.21.15.19:0/3664665443 >> 172.21.15.19:0/3664665443 conn(0x7f07a409d7d0 msgr2=0x7f07a40b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:11.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.354+0000 7f07a9800640 1 -- 172.21.15.19:0/3664665443 shutdown_connections 2024-03-28T13:19:11.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.354+0000 7f07a9800640 1 -- 172.21.15.19:0/3664665443 wait complete. 2024-03-28T13:19:11.379 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:19:11.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.605+0000 7f21d6600640 1 Processor -- start 2024-03-28T13:19:11.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.606+0000 7f21d6600640 1 -- start start 2024-03-28T13:19:11.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.607+0000 7f21d6600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f21d01670b0 0x7f21d0163ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.607+0000 7f21d6600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21d0071d60 con 0x7f21d00d0990 2024-03-28T13:19:11.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.607+0000 7f21d6600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21d01665d0 con 0x7f21d0162920 2024-03-28T13:19:11.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.607+0000 7f21d6600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f21d01674d0 con 0x7f21d01670b0 2024-03-28T13:19:11.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.607+0000 7f21c6a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f21d00d0990 0x7f21d00ad010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45088/0 (socket says 172.21.15.19:45088) 2024-03-28T13:19:11.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.607+0000 7f21c6a00640 1 -- 172.21.15.19:0/1363832944 learned_addr learned my addr 172.21.15.19:0/1363832944 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:11.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.608+0000 7f21cfe00640 1 --2- 172.21.15.19:0/1363832944 >> v2:172.21.15.19:3300/0 conn(0x7f21d01670b0 0x7f21d0163ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.608+0000 7f21cfe00640 1 -- 172.21.15.19:0/1363832944 >> v1:172.21.15.188:6789/0 conn(0x7f21d0162920 legacy=0x7f21d00d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.608+0000 7f21cfe00640 1 -- 172.21.15.19:0/1363832944 >> v1:172.21.15.19:6789/0 conn(0x7f21d00d0990 legacy=0x7f21d00ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.608+0000 7f21cea00640 1 -- 172.21.15.19:0/1363832944 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2250359890 0 0) 0x7f21d0071d60 con 0x7f21d00d0990 2024-03-28T13:19:11.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.608+0000 7f21cfe00640 1 -- 172.21.15.19:0/1363832944 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d01668b0 con 0x7f21d01670b0 2024-03-28T13:19:11.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.610+0000 7f21cfe00640 1 --2- 172.21.15.19:0/1363832944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d01670b0 0x7f21d0163ac0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f21d0167650 tx=0x7f21c802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecade5d7671f704 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:11.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.610+0000 7f21cea00640 1 -- 172.21.15.19:0/1363832944 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c8034070 con 0x7f21d01670b0 2024-03-28T13:19:11.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.610+0000 7f21cea00640 1 -- 172.21.15.19:0/1363832944 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c8030610 con 0x7f21d01670b0 2024-03-28T13:19:11.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.610+0000 7f21cea00640 1 -- 172.21.15.19:0/1363832944 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c80309e0 con 0x7f21d01670b0 2024-03-28T13:19:11.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.611+0000 7f21d6600640 1 -- 172.21.15.19:0/1363832944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d01670b0 msgr2=0x7f21d0163ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.611+0000 7f21d6600640 1 --2- 172.21.15.19:0/1363832944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d01670b0 0x7f21d0163ac0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f21d0167650 tx=0x7f21c802b960 comp rx=0 tx=0).stop 2024-03-28T13:19:11.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.611+0000 7f21d6600640 1 -- 172.21.15.19:0/1363832944 shutdown_connections 2024-03-28T13:19:11.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.611+0000 7f21d6600640 1 --2- 172.21.15.19:0/1363832944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d01670b0 0x7f21d0163ac0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.611+0000 7f21d6600640 1 -- 172.21.15.19:0/1363832944 >> 172.21.15.19:0/1363832944 conn(0x7f21d006b750 msgr2=0x7f21d006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:11.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.611+0000 7f21d6600640 1 -- 172.21.15.19:0/1363832944 shutdown_connections 2024-03-28T13:19:11.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.611+0000 7f21d6600640 1 -- 172.21.15.19:0/1363832944 wait complete. 2024-03-28T13:19:11.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.614+0000 7f21d6600640 1 Processor -- start 2024-03-28T13:19:11.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.614+0000 7f21d6600640 1 -- start start 2024-03-28T13:19:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.616+0000 7f21d6600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d00d0990 0x7f21d0165a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.616+0000 7f21d6600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d0162920 0x7f21d014de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.616+0000 7f21d6600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d01670b0 0x7f21d00cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.616+0000 7f21d6600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f21d0072040 con 0x7f21d00d0990 2024-03-28T13:19:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.616+0000 7f21d6600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f21d0162d00 con 0x7f21d0162920 2024-03-28T13:19:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.616+0000 7f21d6600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f21d00721c0 con 0x7f21d01670b0 2024-03-28T13:19:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.616+0000 7f21cf400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d00d0990 0x7f21d0165a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.616+0000 7f21cf400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d00d0990 0x7f21d0165a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58584/0 (socket says 172.21.15.19:58584) 2024-03-28T13:19:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.616+0000 7f21cf400640 1 -- 172.21.15.19:0/1154677861 learned_addr learned my addr 172.21.15.19:0/1154677861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.617+0000 7f21cfe00640 1 --2- 172.21.15.19:0/1154677861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d01670b0 0x7f21d00cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.617+0000 7f21c6a00640 1 --2- 172.21.15.19:0/1154677861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d0162920 0x7f21d014de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.617+0000 7f21cf400640 1 -- 172.21.15.19:0/1154677861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d01670b0 msgr2=0x7f21d00cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.617+0000 7f21cf400640 1 --2- 172.21.15.19:0/1154677861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d01670b0 0x7f21d00cf520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.617+0000 7f21cf400640 1 -- 172.21.15.19:0/1154677861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d0162920 msgr2=0x7f21d014de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.617+0000 7f21cf400640 1 --2- 172.21.15.19:0/1154677861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d0162920 0x7f21d014de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.617+0000 7f21cf400640 1 -- 172.21.15.19:0/1154677861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d0166090 con 0x7f21d00d0990 2024-03-28T13:19:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.617+0000 7f21c6a00640 1 --2- 172.21.15.19:0/1154677861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d0162920 0x7f21d014de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:11.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.617+0000 7f21cf400640 1 --2- 172.21.15.19:0/1154677861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d00d0990 0x7f21d0165a80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f21c000c8a0 tx=0x7f21c000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.618+0000 7f21c7e00640 1 -- 172.21.15.19:0/1154677861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c0003b70 con 0x7f21d00d0990 2024-03-28T13:19:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.618+0000 7f21c7e00640 1 -- 172.21.15.19:0/1154677861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c0003d30 con 0x7f21d00d0990 2024-03-28T13:19:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.618+0000 7f21c7e00640 1 -- 172.21.15.19:0/1154677861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c000aa90 con 0x7f21d00d0990 2024-03-28T13:19:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.618+0000 7f21d6600640 1 -- 172.21.15.19:0/1154677861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d00cfa60 con 0x7f21d00d0990 2024-03-28T13:19:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.618+0000 7f21d6600640 1 -- 172.21.15.19:0/1154677861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d014e860 con 0x7f21d00d0990 2024-03-28T13:19:11.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.619+0000 7f21d6600640 1 -- 172.21.15.19:0/1154677861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21d00cc320 con 0x7f21d00d0990 2024-03-28T13:19:11.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.621+0000 7f21c7e00640 1 -- 172.21.15.19:0/1154677861 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f21c000ac30 con 0x7f21d00d0990 2024-03-28T13:19:11.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.621+0000 7f21c7e00640 1 --2- 172.21.15.19:0/1154677861 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21ac06c530 0x7f21ac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:11.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.621+0000 7f21c7e00640 1 -- 172.21.15.19:0/1154677861 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f21c008b790 con 0x7f21d00d0990 2024-03-28T13:19:11.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.622+0000 7f21c6a00640 1 --2- 172.21.15.19:0/1154677861 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21ac06c530 0x7f21ac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:11.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.623+0000 7f21c6a00640 1 --2- 172.21.15.19:0/1154677861 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21ac06c530 0x7f21ac06e9f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f21d0161b70 tx=0x7f21b8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:11.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.626+0000 7f21c7e00640 1 -- 172.21.15.19:0/1154677861 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f21d00cc320 con 0x7f21d00d0990 2024-03-28T13:19:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.879+0000 7f21d6600640 1 -- 172.21.15.19:0/1154677861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f21d00685c0 con 0x7f21d00d0990 2024-03-28T13:19:11.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.879+0000 7f21c7e00640 1 -- 172.21.15.19:0/1154677861 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) ==== 53+0+73 (secure 0 0 0) 0x7f21c0059c80 con 0x7f21d00d0990 2024-03-28T13:19:11.879 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:19:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.882+0000 7f21d6600640 1 -- 172.21.15.19:0/1154677861 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21ac06c530 msgr2=0x7f21ac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.882+0000 7f21d6600640 1 --2- 172.21.15.19:0/1154677861 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21ac06c530 0x7f21ac06e9f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f21d0161b70 tx=0x7f21b8009040 comp rx=0 tx=0).stop 2024-03-28T13:19:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.882+0000 7f21d6600640 1 -- 172.21.15.19:0/1154677861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d00d0990 msgr2=0x7f21d0165a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.882+0000 7f21d6600640 1 --2- 172.21.15.19:0/1154677861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d00d0990 0x7f21d0165a80 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f21c000c8a0 tx=0x7f21c000cd60 comp rx=0 tx=0).stop 2024-03-28T13:19:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.882+0000 7f21d6600640 1 -- 172.21.15.19:0/1154677861 shutdown_connections 2024-03-28T13:19:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.882+0000 7f21d6600640 1 --2- 172.21.15.19:0/1154677861 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f21ac06c530 0x7f21ac06e9f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.882+0000 7f21d6600640 1 --2- 172.21.15.19:0/1154677861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d01670b0 0x7f21d00cf520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.882+0000 7f21d6600640 1 --2- 172.21.15.19:0/1154677861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d0162920 0x7f21d014de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.882+0000 7f21d6600640 1 --2- 172.21.15.19:0/1154677861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f21d00d0990 0x7f21d0165a80 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.882+0000 7f21d6600640 1 -- 172.21.15.19:0/1154677861 >> 172.21.15.19:0/1154677861 conn(0x7f21d006b750 msgr2=0x7f21d006de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.882+0000 7f21d6600640 1 -- 172.21.15.19:0/1154677861 shutdown_connections 2024-03-28T13:19:11.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:11.883+0000 7f21d6600640 1 -- 172.21.15.19:0/1154677861 wait complete. 2024-03-28T13:19:11.902 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-28T13:19:11.902 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:19:11.902 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:19:11.902 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:19:11.902 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:19:11.903 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-28T13:19:11.903 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-28T13:19:11.903 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:19:11.903 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:19:11.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:11.903 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:19:11.957 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:19:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:19:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:19:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:19:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:19:11.957 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:19:11.957 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:19:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:19:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:19:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:19:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:19:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:19:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:19:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:19:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:19:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:19:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42003sec preferred_lft 42003sec 2024-03-28T13:19:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:19:11.958 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:19:11.958 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:19:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:19:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:19:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:19:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:19:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:19:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:19:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:19:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:19:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:19:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:19:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:19:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:19:11.959 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:19:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether e6:17:c5:8e:2b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:19:11.959 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::e417:c5ff:fe8e:2bcf/64 scope link 2024-03-28T13:19:11.960 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:19:11.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:11.961 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:19:12.054 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:19:12.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:12.059 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:19:12.111 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:19:12.111 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:19:12.112 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:19:12.112 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:19:12.112 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:19:12.113 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:19:12.113 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:19:12.113 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:19:12.113 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:19:12.113 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:19:12.113 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:19:12.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:19:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:19:12.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:12.357 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:19:12.406 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:19:12.406 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:19:12.406 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:19:12.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:12.406 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:19:12.503 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:19:12.503 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:19:12.551 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:19:12.551 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:19:12.551 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:19:12.596 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:19:12.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:12.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:19:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:19:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:19:12.777 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:19:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:19:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:19:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:19:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:19:12.798 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:19:12.825 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:19:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:19:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:19:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:19:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:19:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:19:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:19:12.826 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:19:12.830 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:19:12.830 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:19:12.830 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:19:12.830 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:19:12.830 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:19:12.834 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:19:12.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:12.834 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:19:12.909 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:19:13.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:19:13.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:19:13.038 INFO:teuthology.orchestra.run.smithi019.stdout:client8208 2024-03-28T13:19:13.038 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:19:13.038 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:19:13.092 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:19:13.092 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:19:13.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:13.140 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-03-28T13:19:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:6+0 records in 2024-03-28T13:19:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:6+0 records out 2024-03-28T13:19:13.427 INFO:teuthology.orchestra.run.smithi019.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.181607 s, 34.6 MB/s 2024-03-28T13:19:13.428 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:19:13.429 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:19:13.480 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:19:13.480 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:19:13.534 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:19:13.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.742+0000 7f1546200640 1 Processor -- start 2024-03-28T13:19:13.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.743+0000 7f1546200640 1 -- start start 2024-03-28T13:19:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.744+0000 7f1546200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f15400d2310 0x7f15400ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.744+0000 7f1546200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f154015e880 con 0x7f154015e4a0 2024-03-28T13:19:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.744+0000 7f1546200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15400d28d0 con 0x7f15400cf530 2024-03-28T13:19:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.744+0000 7f1546200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f15400cf9a0 con 0x7f15400d2310 2024-03-28T13:19:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.744+0000 7f1544e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f154015e4a0 0x7f15400b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45102/0 (socket says 172.21.15.19:45102) 2024-03-28T13:19:13.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.744+0000 7f1544e00640 1 -- 172.21.15.19:0/3001029755 learned_addr learned my addr 172.21.15.19:0/3001029755 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.745+0000 7f153fe00640 1 -- 172.21.15.19:0/3001029755 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1822322968 0 0) 0x7f154015e880 con 0x7f154015e4a0 2024-03-28T13:19:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.745+0000 7f153fe00640 1 -- 172.21.15.19:0/3001029755 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1524003520 con 0x7f154015e4a0 2024-03-28T13:19:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.745+0000 7f1537e00640 1 --2- 172.21.15.19:0/3001029755 >> v2:172.21.15.188:3300/0 conn(0x7f15400d2310 0x7f15400ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:13.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.745+0000 7f153fe00640 1 -- 172.21.15.19:0/3001029755 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1803024405 0 0) 0x7f1524003520 con 0x7f154015e4a0 2024-03-28T13:19:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.745+0000 7f153fe00640 1 -- 172.21.15.19:0/3001029755 >> v2:172.21.15.188:3300/0 conn(0x7f15400d2310 msgr2=0x7f15400ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.745+0000 7f153fe00640 1 --2- 172.21.15.19:0/3001029755 >> v2:172.21.15.188:3300/0 conn(0x7f15400d2310 0x7f15400ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.745+0000 7f153fe00640 1 -- 172.21.15.19:0/3001029755 >> v1:172.21.15.188:6789/0 conn(0x7f15400cf530 legacy=0x7f15400b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.746+0000 7f153fe00640 1 -- 172.21.15.19:0/3001029755 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15400d2bb0 con 0x7f154015e4a0 2024-03-28T13:19:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.746+0000 7f153fe00640 1 -- 172.21.15.19:0/3001029755 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f1530001670 con 0x7f154015e4a0 2024-03-28T13:19:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.746+0000 7f153fe00640 1 -- 172.21.15.19:0/3001029755 >> v1:172.21.15.19:6789/0 conn(0x7f154015e4a0 legacy=0x7f15400b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.746+0000 7f153fe00640 1 --2- 172.21.15.19:0/3001029755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1524003a00 0x7f1524003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.746+0000 7f153fe00640 1 --2- 172.21.15.19:0/3001029755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1524004800 0x7f1524004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.746+0000 7f153fe00640 1 --2- 172.21.15.19:0/3001029755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15240052e0 0x7f152400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.746+0000 7f153fe00640 1 -- 172.21.15.19:0/3001029755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15400cf9a0 con 0x7f15240052e0 2024-03-28T13:19:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.746+0000 7f153fe00640 1 -- 172.21.15.19:0/3001029755 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f152400c200 con 0x7f1524003a00 2024-03-28T13:19:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.746+0000 7f153fe00640 1 -- 172.21.15.19:0/3001029755 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f152400c400 con 0x7f1524004800 2024-03-28T13:19:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.746+0000 7f1545800640 1 --2- 172.21.15.19:0/3001029755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15240052e0 0x7f152400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:13.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.746+0000 7f1537e00640 1 --2- 172.21.15.19:0/3001029755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1524004800 0x7f1524004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.746+0000 7f1545800640 1 -- 172.21.15.19:0/3001029755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1524004800 msgr2=0x7f1524004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.746+0000 7f1545800640 1 --2- 172.21.15.19:0/3001029755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1524004800 0x7f1524004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.746+0000 7f1544e00640 1 --2- 172.21.15.19:0/3001029755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1524003a00 0x7f1524003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.746+0000 7f1545800640 1 -- 172.21.15.19:0/3001029755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1524003a00 msgr2=0x7f1524003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.747+0000 7f1545800640 1 --2- 172.21.15.19:0/3001029755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1524003a00 0x7f1524003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.747+0000 7f1545800640 1 -- 172.21.15.19:0/3001029755 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1524003520 con 0x7f15240052e0 2024-03-28T13:19:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.747+0000 7f1537e00640 1 --2- 172.21.15.19:0/3001029755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1524004800 0x7f1524004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:13.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.747+0000 7f1544e00640 1 --2- 172.21.15.19:0/3001029755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1524003a00 0x7f1524003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:13.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.747+0000 7f1545800640 1 -- 172.21.15.19:0/3001029755 reap_dead start 2024-03-28T13:19:13.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.748+0000 7f1545800640 1 --2- 172.21.15.19:0/3001029755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15240052e0 0x7f152400b9c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f1538001e50 tx=0x7f1524005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeea26c0e187bdfe server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:13.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.748+0000 7f153fe00640 1 -- 172.21.15.19:0/3001029755 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1538001a70 con 0x7f15240052e0 2024-03-28T13:19:13.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.748+0000 7f153fe00640 1 -- 172.21.15.19:0/3001029755 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1538003bb0 con 0x7f15240052e0 2024-03-28T13:19:13.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.748+0000 7f153fe00640 1 -- 172.21.15.19:0/3001029755 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15380050d0 con 0x7f15240052e0 2024-03-28T13:19:13.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.749+0000 7f1546200640 1 -- 172.21.15.19:0/3001029755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15240052e0 msgr2=0x7f152400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:13.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.749+0000 7f1546200640 1 --2- 172.21.15.19:0/3001029755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15240052e0 0x7f152400b9c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f1538001e50 tx=0x7f1524005830 comp rx=0 tx=0).stop 2024-03-28T13:19:13.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.749+0000 7f1546200640 1 -- 172.21.15.19:0/3001029755 shutdown_connections 2024-03-28T13:19:13.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.749+0000 7f1546200640 1 --2- 172.21.15.19:0/3001029755 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15240052e0 0x7f152400b9c0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:13.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.749+0000 7f1546200640 1 -- 172.21.15.19:0/3001029755 >> 172.21.15.19:0/3001029755 conn(0x7f15400676d0 msgr2=0x7f1540067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:13.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.749+0000 7f1546200640 1 -- 172.21.15.19:0/3001029755 shutdown_connections 2024-03-28T13:19:13.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.749+0000 7f1546200640 1 -- 172.21.15.19:0/3001029755 wait complete. 2024-03-28T13:19:13.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.752+0000 7f1546200640 1 Processor -- start 2024-03-28T13:19:13.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.752+0000 7f1546200640 1 -- start start 2024-03-28T13:19:13.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1546200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15240052e0 0x7f15401f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:13.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1546200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15401f5530 0x7f15400abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:13.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1546200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15400ac080 0x7f15401591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:13.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1546200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15400d3170 con 0x7f15240052e0 2024-03-28T13:19:13.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1546200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f15400d34e0 con 0x7f15400ac080 2024-03-28T13:19:13.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1546200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f15400d3970 con 0x7f15401f5530 2024-03-28T13:19:13.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1544e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15240052e0 0x7f15401f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:13.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1544e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15240052e0 0x7f15401f4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58596/0 (socket says 172.21.15.19:58596) 2024-03-28T13:19:13.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1544e00640 1 -- 172.21.15.19:0/2931706275 learned_addr learned my addr 172.21.15.19:0/2931706275 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:13.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1545800640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15400ac080 0x7f15401591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:13.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1537e00640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15401f5530 0x7f15400abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:13.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1544e00640 1 -- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15401f5530 msgr2=0x7f15400abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:13.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1544e00640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15401f5530 0x7f15400abb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:13.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1544e00640 1 -- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15400ac080 msgr2=0x7f15401591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:13.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1544e00640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15400ac080 0x7f15401591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:13.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1544e00640 1 -- 172.21.15.19:0/2931706275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154015f460 con 0x7f15240052e0 2024-03-28T13:19:13.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1537e00640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15401f5530 0x7f15400abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:13.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.754+0000 7f1545800640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15400ac080 0x7f15401591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:13.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.755+0000 7f1544e00640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15240052e0 0x7f15401f4ff0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f153000d7a0 tx=0x7f153000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.755+0000 7f153d600640 1 -- 172.21.15.19:0/2931706275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530019070 con 0x7f15240052e0 2024-03-28T13:19:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.755+0000 7f153d600640 1 -- 172.21.15.19:0/2931706275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1530007500 con 0x7f15240052e0 2024-03-28T13:19:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.755+0000 7f153d600640 1 -- 172.21.15.19:0/2931706275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530004a20 con 0x7f15240052e0 2024-03-28T13:19:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.755+0000 7f1546200640 1 -- 172.21.15.19:0/2931706275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f154015f740 con 0x7f15240052e0 2024-03-28T13:19:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.755+0000 7f1546200640 1 -- 172.21.15.19:0/2931706275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15401f16b0 con 0x7f15240052e0 2024-03-28T13:19:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.756+0000 7f1546200640 1 -- 172.21.15.19:0/2931706275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1540075940 con 0x7f15240052e0 2024-03-28T13:19:13.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.757+0000 7f153d600640 1 -- 172.21.15.19:0/2931706275 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f153001c070 con 0x7f15240052e0 2024-03-28T13:19:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.758+0000 7f153d600640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f151c06c530 0x7f151c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.758+0000 7f153d600640 1 -- 172.21.15.19:0/2931706275 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f153008b100 con 0x7f15240052e0 2024-03-28T13:19:13.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.759+0000 7f1537e00640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f151c06c530 0x7f151c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:13.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.760+0000 7f1537e00640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f151c06c530 0x7f151c06e9f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f15400ca860 tx=0x7f1528008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:13.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.763+0000 7f153d600640 1 -- 172.21.15.19:0/2931706275 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1530059710 con 0x7f15240052e0 2024-03-28T13:19:13.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.970+0000 7f1546200640 1 -- 172.21.15.19:0/2931706275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1540067cc0 con 0x7f15240052e0 2024-03-28T13:19:13.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.972+0000 7f153d600640 1 -- 172.21.15.19:0/2931706275 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16619 (secure 0 0 0) 0x7f1530059010 con 0x7f15240052e0 2024-03-28T13:19:13.972 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:13.972 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":105,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:19:00.072522+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:18:56.432417+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-28T13:18:57.540515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:19:13.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.974+0000 7f1546200640 1 -- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f151c06c530 msgr2=0x7f151c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:13.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.974+0000 7f1546200640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f151c06c530 0x7f151c06e9f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f15400ca860 tx=0x7f1528008040 comp rx=0 tx=0).stop 2024-03-28T13:19:13.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.974+0000 7f1546200640 1 -- 172.21.15.19:0/2931706275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15240052e0 msgr2=0x7f15401f4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:13.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.974+0000 7f1546200640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15240052e0 0x7f15401f4ff0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f153000d7a0 tx=0x7f153000dc60 comp rx=0 tx=0).stop 2024-03-28T13:19:13.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.974+0000 7f1546200640 1 -- 172.21.15.19:0/2931706275 shutdown_connections 2024-03-28T13:19:13.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.974+0000 7f1546200640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f151c06c530 0x7f151c06e9f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:13.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.974+0000 7f1546200640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15400ac080 0x7f15401591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:13.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.974+0000 7f1546200640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15401f5530 0x7f15400abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:13.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.974+0000 7f1546200640 1 --2- 172.21.15.19:0/2931706275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15240052e0 0x7f15401f4ff0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:13.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.974+0000 7f1546200640 1 -- 172.21.15.19:0/2931706275 >> 172.21.15.19:0/2931706275 conn(0x7f15400676d0 msgr2=0x7f154015d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:13.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.974+0000 7f1546200640 1 -- 172.21.15.19:0/2931706275 shutdown_connections 2024-03-28T13:19:13.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:13.975+0000 7f1546200640 1 -- 172.21.15.19:0/2931706275 wait complete. 2024-03-28T13:19:13.992 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:18:56.432417+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:18:57.540515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:19:13.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:19:14.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.218+0000 7f42ed000640 1 Processor -- start 2024-03-28T13:19:14.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.219+0000 7f42ed000640 1 -- start start 2024-03-28T13:19:14.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.220+0000 7f42ed000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f42e80d3e40 0x7f42e80b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:14.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.220+0000 7f42ed000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f42e80cef20 0x7f42e80cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:14.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.220+0000 7f42ed000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42e80d2da0 con 0x7f42e815e460 2024-03-28T13:19:14.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.220+0000 7f42ed000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f42e80cf8d0 con 0x7f42e80d3e40 2024-03-28T13:19:14.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.220+0000 7f42ed000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f42e80b8320 con 0x7f42e80cef20 2024-03-28T13:19:14.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.221+0000 7f42e6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f42e80d3e40 0x7f42e80b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:14.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.221+0000 7f42e6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f42e80d3e40 0x7f42e80b9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58612/0 (socket says 172.21.15.19:58612) 2024-03-28T13:19:14.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.221+0000 7f42e6a00640 1 -- 172.21.15.19:0/2933937727 learned_addr learned my addr 172.21.15.19:0/2933937727 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:14.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.221+0000 7f42e6a00640 1 -- 172.21.15.19:0/2933937727 >> v2:172.21.15.188:3300/0 conn(0x7f42e80cef20 msgr2=0x7f42e80cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:14.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.221+0000 7f42e6a00640 1 --2- 172.21.15.19:0/2933937727 >> v2:172.21.15.188:3300/0 conn(0x7f42e80cef20 0x7f42e80cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.221+0000 7f42e6a00640 1 -- 172.21.15.19:0/2933937727 >> v1:172.21.15.188:6789/0 conn(0x7f42e815e460 legacy=0x7f42e80d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:14.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.221+0000 7f42e6a00640 1 -- 172.21.15.19:0/2933937727 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e80d3080 con 0x7f42e80d3e40 2024-03-28T13:19:14.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.222+0000 7f42e6a00640 1 --2- 172.21.15.19:0/2933937727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e80d3e40 0x7f42e80b9e20 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f42e80d0660 tx=0x7f42d402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3329d5bd32793e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:14.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.222+0000 7f42e6000640 1 -- 172.21.15.19:0/2933937727 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d402d040 con 0x7f42e80d3e40 2024-03-28T13:19:14.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.223+0000 7f42e6000640 1 -- 172.21.15.19:0/2933937727 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d4004230 con 0x7f42e80d3e40 2024-03-28T13:19:14.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.223+0000 7f42e6000640 1 -- 172.21.15.19:0/2933937727 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4003610 con 0x7f42e80d3e40 2024-03-28T13:19:14.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.223+0000 7f42ed000640 1 -- 172.21.15.19:0/2933937727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e80d3e40 msgr2=0x7f42e80b9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:14.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.223+0000 7f42ed000640 1 --2- 172.21.15.19:0/2933937727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e80d3e40 0x7f42e80b9e20 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f42e80d0660 tx=0x7f42d402b8f0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.223+0000 7f42ed000640 1 -- 172.21.15.19:0/2933937727 shutdown_connections 2024-03-28T13:19:14.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.223+0000 7f42ed000640 1 --2- 172.21.15.19:0/2933937727 >> v2:172.21.15.188:3300/0 conn(0x7f42e80cef20 0x7f42e80cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.223+0000 7f42ed000640 1 --2- 172.21.15.19:0/2933937727 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e80d3e40 0x7f42e80b9e20 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.223+0000 7f42ed000640 1 -- 172.21.15.19:0/2933937727 >> 172.21.15.19:0/2933937727 conn(0x7f42e80678b0 msgr2=0x7f42e80d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:14.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.223+0000 7f42ed000640 1 -- 172.21.15.19:0/2933937727 shutdown_connections 2024-03-28T13:19:14.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.223+0000 7f42ed000640 1 -- 172.21.15.19:0/2933937727 wait complete. 2024-03-28T13:19:14.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.225+0000 7f42ed000640 1 Processor -- start 2024-03-28T13:19:14.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.225+0000 7f42ed000640 1 -- start start 2024-03-28T13:19:14.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.226+0000 7f42ed000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42e80cef20 0x7f42e80cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:14.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.226+0000 7f42ed000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e80d3e40 0x7f42e8161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.226+0000 7f42ed000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42e815e460 0x7f42e814f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.226+0000 7f42ed000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f42e80d3920 con 0x7f42e80d3e40 2024-03-28T13:19:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.226+0000 7f42ed000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f42e80d3640 con 0x7f42e815e460 2024-03-28T13:19:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.226+0000 7f42ed000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f42e80b7310 con 0x7f42e80cef20 2024-03-28T13:19:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.226+0000 7f42e6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e80d3e40 0x7f42e8161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.226+0000 7f42e6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e80d3e40 0x7f42e8161140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58624/0 (socket says 172.21.15.19:58624) 2024-03-28T13:19:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.226+0000 7f42e6a00640 1 -- 172.21.15.19:0/425931687 learned_addr learned my addr 172.21.15.19:0/425931687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.227+0000 7f42e7400640 1 --2- 172.21.15.19:0/425931687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42e80cef20 0x7f42e80cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.227+0000 7f42e7e00640 1 --2- 172.21.15.19:0/425931687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42e815e460 0x7f42e814f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.227+0000 7f42e6a00640 1 -- 172.21.15.19:0/425931687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42e80cef20 msgr2=0x7f42e80cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.227+0000 7f42e6a00640 1 --2- 172.21.15.19:0/425931687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42e80cef20 0x7f42e80cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.227+0000 7f42e6a00640 1 -- 172.21.15.19:0/425931687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42e815e460 msgr2=0x7f42e814f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.227+0000 7f42e6a00640 1 --2- 172.21.15.19:0/425931687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42e815e460 0x7f42e814f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.227+0000 7f42e6a00640 1 -- 172.21.15.19:0/425931687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e80cc3f0 con 0x7f42e80d3e40 2024-03-28T13:19:14.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.227+0000 7f42e6a00640 1 --2- 172.21.15.19:0/425931687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e80d3e40 0x7f42e8161140 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f42d402b930 tx=0x7f42d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:14.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.228+0000 7f42df400640 1 -- 172.21.15.19:0/425931687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4003e50 con 0x7f42e80d3e40 2024-03-28T13:19:14.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.228+0000 7f42df400640 1 -- 172.21.15.19:0/425931687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42d4006040 con 0x7f42e80d3e40 2024-03-28T13:19:14.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.228+0000 7f42df400640 1 -- 172.21.15.19:0/425931687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d4006430 con 0x7f42e80d3e40 2024-03-28T13:19:14.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.228+0000 7f42ed000640 1 -- 172.21.15.19:0/425931687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e80cc6d0 con 0x7f42e80d3e40 2024-03-28T13:19:14.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.228+0000 7f42ed000640 1 -- 172.21.15.19:0/425931687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e814fb70 con 0x7f42e80d3e40 2024-03-28T13:19:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.229+0000 7f42ed000640 1 -- 172.21.15.19:0/425931687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e8075940 con 0x7f42e80d3e40 2024-03-28T13:19:14.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.230+0000 7f42df400640 1 -- 172.21.15.19:0/425931687 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f42d40065d0 con 0x7f42e80d3e40 2024-03-28T13:19:14.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.231+0000 7f42df400640 1 --2- 172.21.15.19:0/425931687 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f42c806c6d0 0x7f42c806eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.231+0000 7f42df400640 1 -- 172.21.15.19:0/425931687 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f42d40ada30 con 0x7f42e80d3e40 2024-03-28T13:19:14.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.232+0000 7f42e7400640 1 --2- 172.21.15.19:0/425931687 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f42c806c6d0 0x7f42c806eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:14.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.233+0000 7f42e7400640 1 --2- 172.21.15.19:0/425931687 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f42c806c6d0 0x7f42c806eb90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f42d8006fd0 tx=0x7f42d8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:14.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.236+0000 7f42df400640 1 -- 172.21.15.19:0/425931687 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f42d407c040 con 0x7f42e80d3e40 2024-03-28T13:19:14.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.469+0000 7f42ed000640 1 -- 172.21.15.19:0/425931687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f42e80d1250 con 0x7f42e80d3e40 2024-03-28T13:19:14.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.469+0000 7f42df400640 1 -- 172.21.15.19:0/425931687 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f42d4004200 con 0x7f42e80d3e40 2024-03-28T13:19:14.469 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:19:14.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.472+0000 7f42ed000640 1 -- 172.21.15.19:0/425931687 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f42c806c6d0 msgr2=0x7f42c806eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:14.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.472+0000 7f42ed000640 1 --2- 172.21.15.19:0/425931687 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f42c806c6d0 0x7f42c806eb90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f42d8006fd0 tx=0x7f42d8008040 comp rx=0 tx=0).stop 2024-03-28T13:19:14.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.472+0000 7f42ed000640 1 -- 172.21.15.19:0/425931687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e80d3e40 msgr2=0x7f42e8161140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:14.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.472+0000 7f42ed000640 1 --2- 172.21.15.19:0/425931687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e80d3e40 0x7f42e8161140 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f42d402b930 tx=0x7f42d40046c0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.472+0000 7f42ed000640 1 -- 172.21.15.19:0/425931687 shutdown_connections 2024-03-28T13:19:14.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.472+0000 7f42ed000640 1 --2- 172.21.15.19:0/425931687 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f42c806c6d0 0x7f42c806eb90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.472+0000 7f42ed000640 1 --2- 172.21.15.19:0/425931687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42e815e460 0x7f42e814f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.472+0000 7f42ed000640 1 --2- 172.21.15.19:0/425931687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f42e80d3e40 0x7f42e8161140 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.472+0000 7f42ed000640 1 --2- 172.21.15.19:0/425931687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42e80cef20 0x7f42e80cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.472+0000 7f42ed000640 1 -- 172.21.15.19:0/425931687 >> 172.21.15.19:0/425931687 conn(0x7f42e80678b0 msgr2=0x7f42e80d4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:14.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.473+0000 7f42ed000640 1 -- 172.21.15.19:0/425931687 shutdown_connections 2024-03-28T13:19:14.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.473+0000 7f42ed000640 1 -- 172.21.15.19:0/425931687 wait complete. 2024-03-28T13:19:14.492 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:19:14.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.708+0000 7f4243000640 1 Processor -- start 2024-03-28T13:19:14.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.709+0000 7f4243000640 1 -- start start 2024-03-28T13:19:14.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.711+0000 7f4243000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f423c15e4c0 0x7f423c0b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:14.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.711+0000 7f4243000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f423c0b69c0 0x7f423c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:14.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.712+0000 7f4243000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f423c0d2710 con 0x7f423c0d37b0 2024-03-28T13:19:14.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.712+0000 7f4243000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f423c15e8a0 con 0x7f423c15e4c0 2024-03-28T13:19:14.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.712+0000 7f4243000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f423c0d4040 con 0x7f423c0b69c0 2024-03-28T13:19:14.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.712+0000 7f4241200640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f423c0d37b0 0x7f423c0b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:36468/0 (socket says 172.21.15.19:36468) 2024-03-28T13:19:14.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.712+0000 7f4241200640 1 -- 172.21.15.19:0/3830941297 learned_addr learned my addr 172.21.15.19:0/3830941297 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:14.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.712+0000 7f4242600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f423c0b69c0 0x7f423c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:14.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.713+0000 7f4241c00640 1 --2- 172.21.15.19:0/3830941297 >> v2:172.21.15.188:3300/0 conn(0x7f423c15e4c0 0x7f423c0b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:14.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.713+0000 7f423be00640 1 -- 172.21.15.19:0/3830941297 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4098102913 0 0) 0x7f423c0d2710 con 0x7f423c0d37b0 2024-03-28T13:19:14.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.713+0000 7f423be00640 1 -- 172.21.15.19:0/3830941297 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4224003520 con 0x7f423c0d37b0 2024-03-28T13:19:14.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.714+0000 7f4242600640 1 -- 172.21.15.19:0/3830941297 >> v2:172.21.15.188:3300/0 conn(0x7f423c15e4c0 msgr2=0x7f423c0b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:14.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.714+0000 7f4242600640 1 --2- 172.21.15.19:0/3830941297 >> v2:172.21.15.188:3300/0 conn(0x7f423c15e4c0 0x7f423c0b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.714+0000 7f4242600640 1 -- 172.21.15.19:0/3830941297 >> v1:172.21.15.188:6789/0 conn(0x7f423c0d37b0 legacy=0x7f423c0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:14.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.714+0000 7f4242600640 1 -- 172.21.15.19:0/3830941297 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c0d29f0 con 0x7f423c0b69c0 2024-03-28T13:19:14.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.714+0000 7f4241c00640 1 --2- 172.21.15.19:0/3830941297 >> v2:172.21.15.188:3300/0 conn(0x7f423c15e4c0 0x7f423c0b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:19:14.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.715+0000 7f4242600640 1 --2- 172.21.15.19:0/3830941297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f423c0b69c0 0x7f423c0b7400 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4234009d90 tx=0x7f423402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e60b3ab63e564f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:14.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.716+0000 7f423be00640 1 -- 172.21.15.19:0/3830941297 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234039070 con 0x7f423c0b69c0 2024-03-28T13:19:14.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.716+0000 7f423be00640 1 -- 172.21.15.19:0/3830941297 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f423402cc50 con 0x7f423c0b69c0 2024-03-28T13:19:14.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.716+0000 7f423be00640 1 -- 172.21.15.19:0/3830941297 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234034440 con 0x7f423c0b69c0 2024-03-28T13:19:14.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.717+0000 7f4243000640 1 -- 172.21.15.19:0/3830941297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f423c0b69c0 msgr2=0x7f423c0b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:14.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.717+0000 7f4243000640 1 --2- 172.21.15.19:0/3830941297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f423c0b69c0 0x7f423c0b7400 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4234009d90 tx=0x7f423402bea0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.717+0000 7f4243000640 1 -- 172.21.15.19:0/3830941297 shutdown_connections 2024-03-28T13:19:14.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.717+0000 7f4243000640 1 --2- 172.21.15.19:0/3830941297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f423c0b69c0 0x7f423c0b7400 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.717+0000 7f4243000640 1 --2- 172.21.15.19:0/3830941297 >> v2:172.21.15.188:3300/0 conn(0x7f423c15e4c0 0x7f423c0b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.717+0000 7f4243000640 1 -- 172.21.15.19:0/3830941297 >> 172.21.15.19:0/3830941297 conn(0x7f423c0676a0 msgr2=0x7f423c067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:14.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.717+0000 7f4243000640 1 -- 172.21.15.19:0/3830941297 shutdown_connections 2024-03-28T13:19:14.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.717+0000 7f4243000640 1 -- 172.21.15.19:0/3830941297 wait complete. 2024-03-28T13:19:14.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.720+0000 7f4243000640 1 Processor -- start 2024-03-28T13:19:14.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.721+0000 7f4243000640 1 -- start start 2024-03-28T13:19:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.723+0000 7f4243000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c0b69c0 0x7f423c0c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.723+0000 7f4243000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f423c0d37b0 0x7f423c0ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.723+0000 7f4243000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f423c15e4c0 0x7f423c157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.723+0000 7f4243000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f423c0d2fb0 con 0x7f423c0d37b0 2024-03-28T13:19:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.723+0000 7f4243000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f423c0d3320 con 0x7f423c0b69c0 2024-03-28T13:19:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.723+0000 7f4243000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f423c0ba130 con 0x7f423c15e4c0 2024-03-28T13:19:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.723+0000 7f4241200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f423c0d37b0 0x7f423c0ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.723+0000 7f4241200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f423c0d37b0 0x7f423c0ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58626/0 (socket says 172.21.15.19:58626) 2024-03-28T13:19:14.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.723+0000 7f4241200640 1 -- 172.21.15.19:0/2163637440 learned_addr learned my addr 172.21.15.19:0/2163637440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.723+0000 7f4241c00640 1 --2- 172.21.15.19:0/2163637440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c0b69c0 0x7f423c0c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.723+0000 7f4242600640 1 --2- 172.21.15.19:0/2163637440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f423c15e4c0 0x7f423c157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.723+0000 7f4241200640 1 -- 172.21.15.19:0/2163637440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f423c15e4c0 msgr2=0x7f423c157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.724+0000 7f4241200640 1 --2- 172.21.15.19:0/2163637440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f423c15e4c0 0x7f423c157780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.724+0000 7f4241200640 1 -- 172.21.15.19:0/2163637440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c0b69c0 msgr2=0x7f423c0c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.724+0000 7f4241200640 1 --2- 172.21.15.19:0/2163637440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c0b69c0 0x7f423c0c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.724+0000 7f4241200640 1 -- 172.21.15.19:0/2163637440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c0c8470 con 0x7f423c0d37b0 2024-03-28T13:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.724+0000 7f4241c00640 1 --2- 172.21.15.19:0/2163637440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c0b69c0 0x7f423c0c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.724+0000 7f4241200640 1 --2- 172.21.15.19:0/2163637440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f423c0d37b0 0x7f423c0ca760 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f422800c500 tx=0x7f422800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.724+0000 7f4239600640 1 -- 172.21.15.19:0/2163637440 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228017070 con 0x7f423c0d37b0 2024-03-28T13:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.724+0000 7f4239600640 1 -- 172.21.15.19:0/2163637440 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4228005ce0 con 0x7f423c0d37b0 2024-03-28T13:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.724+0000 7f4239600640 1 -- 172.21.15.19:0/2163637440 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228007330 con 0x7f423c0d37b0 2024-03-28T13:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.724+0000 7f4243000640 1 -- 172.21.15.19:0/2163637440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423c15ea30 con 0x7f423c0d37b0 2024-03-28T13:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.725+0000 7f4243000640 1 -- 172.21.15.19:0/2163637440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423c15eff0 con 0x7f423c0d37b0 2024-03-28T13:19:14.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.726+0000 7f4243000640 1 -- 172.21.15.19:0/2163637440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f423c075940 con 0x7f423c0d37b0 2024-03-28T13:19:14.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.727+0000 7f4239600640 1 -- 172.21.15.19:0/2163637440 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f422801a070 con 0x7f423c0d37b0 2024-03-28T13:19:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.728+0000 7f4239600640 1 --2- 172.21.15.19:0/2163637440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f421c06c530 0x7f421c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:14.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.728+0000 7f4239600640 1 -- 172.21.15.19:0/2163637440 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f422808ac50 con 0x7f423c0d37b0 2024-03-28T13:19:14.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.728+0000 7f4241c00640 1 --2- 172.21.15.19:0/2163637440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f421c06c530 0x7f421c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:14.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.730+0000 7f4241c00640 1 --2- 172.21.15.19:0/2163637440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f421c06c530 0x7f421c06e9f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4230009a20 tx=0x7f4230008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:14.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:14.733+0000 7f4239600640 1 -- 172.21.15.19:0/2163637440 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4228059290 con 0x7f423c0d37b0 2024-03-28T13:19:15.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.004+0000 7f4243000640 1 -- 172.21.15.19:0/2163637440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f423c15d910 con 0x7f423c0d37b0 2024-03-28T13:19:15.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.005+0000 7f4239600640 1 -- 172.21.15.19:0/2163637440 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7f4228058b90 con 0x7f423c0d37b0 2024-03-28T13:19:15.005 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:19:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.007+0000 7f4243000640 1 -- 172.21.15.19:0/2163637440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f421c06c530 msgr2=0x7f421c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:15.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.007+0000 7f4243000640 1 --2- 172.21.15.19:0/2163637440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f421c06c530 0x7f421c06e9f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4230009a20 tx=0x7f4230008040 comp rx=0 tx=0).stop 2024-03-28T13:19:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.007+0000 7f4243000640 1 -- 172.21.15.19:0/2163637440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f423c0d37b0 msgr2=0x7f423c0ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.007+0000 7f4243000640 1 --2- 172.21.15.19:0/2163637440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f423c0d37b0 0x7f423c0ca760 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f422800c500 tx=0x7f422800c9c0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.008+0000 7f4243000640 1 -- 172.21.15.19:0/2163637440 shutdown_connections 2024-03-28T13:19:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.008+0000 7f4243000640 1 --2- 172.21.15.19:0/2163637440 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f421c06c530 0x7f421c06e9f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.008+0000 7f4243000640 1 --2- 172.21.15.19:0/2163637440 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f423c15e4c0 0x7f423c157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.008+0000 7f4243000640 1 --2- 172.21.15.19:0/2163637440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f423c0d37b0 0x7f423c0ca760 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.008+0000 7f4243000640 1 --2- 172.21.15.19:0/2163637440 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f423c0b69c0 0x7f423c0c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.008+0000 7f4243000640 1 -- 172.21.15.19:0/2163637440 >> 172.21.15.19:0/2163637440 conn(0x7f423c0676a0 msgr2=0x7f423c06c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.008+0000 7f4243000640 1 -- 172.21.15.19:0/2163637440 shutdown_connections 2024-03-28T13:19:15.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.008+0000 7f4243000640 1 -- 172.21.15.19:0/2163637440 wait complete. 2024-03-28T13:19:15.024 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:19:15.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.223+0000 7f0387e00640 1 Processor -- start 2024-03-28T13:19:15.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.224+0000 7f0387e00640 1 -- start start 2024-03-28T13:19:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.225+0000 7f0387e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f038806ea20 0x7f038806bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.225+0000 7f0387e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f038815ed90 con 0x7f03880afd50 2024-03-28T13:19:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.225+0000 7f0387e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f038815f070 con 0x7f03880b3e70 2024-03-28T13:19:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.225+0000 7f0387e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f038815f350 con 0x7f038806ea20 2024-03-28T13:19:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.225+0000 7f0386a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f03880afd50 0x7f03880b3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:45110/0 (socket says 172.21.15.19:45110) 2024-03-28T13:19:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.226+0000 7f0386a00640 1 -- 172.21.15.19:0/3511924758 learned_addr learned my addr 172.21.15.19:0/3511924758 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.226+0000 7f0387400640 1 --2- 172.21.15.19:0/3511924758 >> v2:172.21.15.19:3300/0 conn(0x7f038806ea20 0x7f038806bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.226+0000 7f0385600640 1 -- 172.21.15.19:0/3511924758 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 878935562 0 0) 0x7f038815ed90 con 0x7f03880afd50 2024-03-28T13:19:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.226+0000 7f0385600640 1 -- 172.21.15.19:0/3511924758 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f036c003520 con 0x7f03880afd50 2024-03-28T13:19:15.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.226+0000 7f0387400640 1 -- 172.21.15.19:0/3511924758 >> v1:172.21.15.188:6789/0 conn(0x7f03880b3e70 legacy=0x7f038806b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:15.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.226+0000 7f0387400640 1 -- 172.21.15.19:0/3511924758 >> v1:172.21.15.19:6789/0 conn(0x7f03880afd50 legacy=0x7f03880b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:15.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.226+0000 7f0387400640 1 -- 172.21.15.19:0/3511924758 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038815f630 con 0x7f038806ea20 2024-03-28T13:19:15.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.227+0000 7f0387400640 1 --2- 172.21.15.19:0/3511924758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f038806ea20 0x7f038806bfb0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f0380009d90 tx=0x7f038002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8976e6b79816e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:15.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.228+0000 7f0385600640 1 -- 172.21.15.19:0/3511924758 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380039070 con 0x7f038806ea20 2024-03-28T13:19:15.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.228+0000 7f0385600640 1 -- 172.21.15.19:0/3511924758 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f038002cd00 con 0x7f038806ea20 2024-03-28T13:19:15.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.228+0000 7f0385600640 1 -- 172.21.15.19:0/3511924758 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0380034640 con 0x7f038806ea20 2024-03-28T13:19:15.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.228+0000 7f0387e00640 1 -- 172.21.15.19:0/3511924758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f038806ea20 msgr2=0x7f038806bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:15.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.228+0000 7f0387e00640 1 --2- 172.21.15.19:0/3511924758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f038806ea20 0x7f038806bfb0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f0380009d90 tx=0x7f038002bf50 comp rx=0 tx=0).stop 2024-03-28T13:19:15.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.228+0000 7f0387e00640 1 -- 172.21.15.19:0/3511924758 shutdown_connections 2024-03-28T13:19:15.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.228+0000 7f0387e00640 1 --2- 172.21.15.19:0/3511924758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f038806ea20 0x7f038806bfb0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.228+0000 7f0387e00640 1 -- 172.21.15.19:0/3511924758 >> 172.21.15.19:0/3511924758 conn(0x7f03880d0770 msgr2=0x7f03880d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.229+0000 7f0387e00640 1 -- 172.21.15.19:0/3511924758 shutdown_connections 2024-03-28T13:19:15.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.229+0000 7f0387e00640 1 -- 172.21.15.19:0/3511924758 wait complete. 2024-03-28T13:19:15.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.230+0000 7f0387e00640 1 Processor -- start 2024-03-28T13:19:15.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.230+0000 7f0387e00640 1 -- start start 2024-03-28T13:19:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.231+0000 7f0387e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f038806ea20 0x7f0388153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.231+0000 7f0387e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03880afd50 0x7f0388155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.231+0000 7f0387e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03880b3e70 0x7f0388155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.231+0000 7f0387e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f038806fa20 con 0x7f03880afd50 2024-03-28T13:19:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.231+0000 7f0387e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f03880b0130 con 0x7f038806ea20 2024-03-28T13:19:15.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.231+0000 7f0387e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f038806fba0 con 0x7f03880b3e70 2024-03-28T13:19:15.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.231+0000 7f0386000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03880afd50 0x7f0388155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:15.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.232+0000 7f0386000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03880afd50 0x7f0388155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58636/0 (socket says 172.21.15.19:58636) 2024-03-28T13:19:15.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.232+0000 7f0386000640 1 -- 172.21.15.19:0/437268346 learned_addr learned my addr 172.21.15.19:0/437268346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:15.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.232+0000 7f0386000640 1 -- 172.21.15.19:0/437268346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03880b3e70 msgr2=0x7f0388155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:15.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.232+0000 7f0387400640 1 --2- 172.21.15.19:0/437268346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03880b3e70 0x7f0388155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:15.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.232+0000 7f0386a00640 1 --2- 172.21.15.19:0/437268346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f038806ea20 0x7f0388153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:15.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.232+0000 7f0386000640 1 --2- 172.21.15.19:0/437268346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03880b3e70 0x7f0388155fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.232+0000 7f0386000640 1 -- 172.21.15.19:0/437268346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f038806ea20 msgr2=0x7f0388153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:15.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.232+0000 7f0386000640 1 --2- 172.21.15.19:0/437268346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f038806ea20 0x7f0388153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.232+0000 7f0386000640 1 -- 172.21.15.19:0/437268346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03881565f0 con 0x7f03880afd50 2024-03-28T13:19:15.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.232+0000 7f0386000640 1 --2- 172.21.15.19:0/437268346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03880afd50 0x7f0388155a80 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f037400c3c0 tx=0x7f037400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:15.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.233+0000 7f037ea00640 1 -- 172.21.15.19:0/437268346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03740052d0 con 0x7f03880afd50 2024-03-28T13:19:15.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.233+0000 7f037ea00640 1 -- 172.21.15.19:0/437268346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0374005490 con 0x7f03880afd50 2024-03-28T13:19:15.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.233+0000 7f037ea00640 1 -- 172.21.15.19:0/437268346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0374007260 con 0x7f03880afd50 2024-03-28T13:19:15.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.233+0000 7f0387e00640 1 -- 172.21.15.19:0/437268346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03881568d0 con 0x7f03880afd50 2024-03-28T13:19:15.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.233+0000 7f0387e00640 1 -- 172.21.15.19:0/437268346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0388156e90 con 0x7f03880afd50 2024-03-28T13:19:15.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.234+0000 7f0387e00640 1 -- 172.21.15.19:0/437268346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0388075940 con 0x7f03880afd50 2024-03-28T13:19:15.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.236+0000 7f037ea00640 1 -- 172.21.15.19:0/437268346 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0374007400 con 0x7f03880afd50 2024-03-28T13:19:15.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.237+0000 7f037ea00640 1 --2- 172.21.15.19:0/437268346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f036806c530 0x7f036806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:15.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.237+0000 7f037ea00640 1 -- 172.21.15.19:0/437268346 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f037408ac30 con 0x7f03880afd50 2024-03-28T13:19:15.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.238+0000 7f0386a00640 1 --2- 172.21.15.19:0/437268346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f036806c530 0x7f036806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:15.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.239+0000 7f0386a00640 1 --2- 172.21.15.19:0/437268346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f036806c530 0x7f036806e9f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f03780011e0 tx=0x7f0378009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:15.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.246+0000 7f037ea00640 1 -- 172.21.15.19:0/437268346 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0374059240 con 0x7f03880afd50 2024-03-28T13:19:15.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.447+0000 7f0387e00640 1 -- 172.21.15.19:0/437268346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f03881554c0 con 0x7f03880afd50 2024-03-28T13:19:15.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.448+0000 7f037ea00640 1 -- 172.21.15.19:0/437268346 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7f0374058b40 con 0x7f03880afd50 2024-03-28T13:19:15.448 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:19:15.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.451+0000 7f0387e00640 1 -- 172.21.15.19:0/437268346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f036806c530 msgr2=0x7f036806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:15.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.451+0000 7f0387e00640 1 --2- 172.21.15.19:0/437268346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f036806c530 0x7f036806e9f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f03780011e0 tx=0x7f0378009040 comp rx=0 tx=0).stop 2024-03-28T13:19:15.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.451+0000 7f0387e00640 1 -- 172.21.15.19:0/437268346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03880afd50 msgr2=0x7f0388155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:15.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.451+0000 7f0387e00640 1 --2- 172.21.15.19:0/437268346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03880afd50 0x7f0388155a80 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f037400c3c0 tx=0x7f037400c880 comp rx=0 tx=0).stop 2024-03-28T13:19:15.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.451+0000 7f0387e00640 1 -- 172.21.15.19:0/437268346 shutdown_connections 2024-03-28T13:19:15.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.451+0000 7f0387e00640 1 --2- 172.21.15.19:0/437268346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f036806c530 0x7f036806e9f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.451+0000 7f0387e00640 1 --2- 172.21.15.19:0/437268346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03880b3e70 0x7f0388155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.451+0000 7f0387e00640 1 --2- 172.21.15.19:0/437268346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03880afd50 0x7f0388155a80 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.451+0000 7f0387e00640 1 --2- 172.21.15.19:0/437268346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f038806ea20 0x7f0388153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.451+0000 7f0387e00640 1 -- 172.21.15.19:0/437268346 >> 172.21.15.19:0/437268346 conn(0x7f03880d0770 msgr2=0x7f038815de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:15.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.451+0000 7f0387e00640 1 -- 172.21.15.19:0/437268346 shutdown_connections 2024-03-28T13:19:15.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.451+0000 7f0387e00640 1 -- 172.21.15.19:0/437268346 wait complete. 2024-03-28T13:19:15.470 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:19:15.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.689+0000 7fc592200640 1 Processor -- start 2024-03-28T13:19:15.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.689+0000 7fc592200640 1 -- start start 2024-03-28T13:19:15.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.690+0000 7fc592200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc58c0d0990 0x7fc58c0aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:15.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.691+0000 7fc592200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc58c0b40b0 con 0x7fc58c155550 2024-03-28T13:19:15.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.691+0000 7fc592200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc58c0b4390 con 0x7fc58c155930 2024-03-28T13:19:15.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.691+0000 7fc592200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc58c0b4670 con 0x7fc58c0d0990 2024-03-28T13:19:15.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.691+0000 7fc58b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc58c155550 0x7fc58c15f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34134/0 (socket says 172.21.15.19:34134) 2024-03-28T13:19:15.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.691+0000 7fc58b400640 1 -- 172.21.15.19:0/2678442159 learned_addr learned my addr 172.21.15.19:0/2678442159 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:15.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.691+0000 7fc58a000640 1 -- 172.21.15.19:0/2678442159 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2675797577 0 0) 0x7fc58c0b40b0 con 0x7fc58c155550 2024-03-28T13:19:15.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.691+0000 7fc58a000640 1 -- 172.21.15.19:0/2678442159 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc570005730 con 0x7fc58c155550 2024-03-28T13:19:15.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.691+0000 7fc58be00640 1 --2- 172.21.15.19:0/2678442159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc58c0d0990 0x7fc58c0aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:15.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.692+0000 7fc58a000640 1 -- 172.21.15.19:0/2678442159 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1156309285 0 0) 0x7fc570005730 con 0x7fc58c155550 2024-03-28T13:19:15.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.692+0000 7fc58a000640 1 -- 172.21.15.19:0/2678442159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc58c0d0990 msgr2=0x7fc58c0aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:15.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.692+0000 7fc58a000640 1 --2- 172.21.15.19:0/2678442159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc58c0d0990 0x7fc58c0aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.692+0000 7fc58a000640 1 -- 172.21.15.19:0/2678442159 >> v1:172.21.15.188:6789/0 conn(0x7fc58c155930 legacy=0x7fc58c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:15.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.692+0000 7fc58a000640 1 -- 172.21.15.19:0/2678442159 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58c0b4950 con 0x7fc58c155550 2024-03-28T13:19:15.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.692+0000 7fc58a000640 1 -- 172.21.15.19:0/2678442159 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fc57c001670 con 0x7fc58c155550 2024-03-28T13:19:15.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.692+0000 7fc58a000640 1 -- 172.21.15.19:0/2678442159 >> v1:172.21.15.19:6789/0 conn(0x7fc58c155550 legacy=0x7fc58c15f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:15.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.692+0000 7fc58a000640 1 --2- 172.21.15.19:0/2678442159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc570005bf0 0x7fc570005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:15.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.692+0000 7fc58a000640 1 --2- 172.21.15.19:0/2678442159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc570006a10 0x7fc570006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:15.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.692+0000 7fc58a000640 1 --2- 172.21.15.19:0/2678442159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc58c0d0990 0x7fc5700073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:15.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.692+0000 7fc58a000640 1 -- 172.21.15.19:0/2678442159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c0b4670 con 0x7fc570006a10 2024-03-28T13:19:15.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.692+0000 7fc58a000640 1 -- 172.21.15.19:0/2678442159 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc57000dd10 con 0x7fc570005bf0 2024-03-28T13:19:15.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.692+0000 7fc58a000640 1 -- 172.21.15.19:0/2678442159 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc57000df10 con 0x7fc58c0d0990 2024-03-28T13:19:15.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.692+0000 7fc58aa00640 1 --2- 172.21.15.19:0/2678442159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc570006a10 0x7fc570006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:15.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.693+0000 7fc58be00640 1 --2- 172.21.15.19:0/2678442159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc58c0d0990 0x7fc5700073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:15.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.693+0000 7fc58aa00640 1 -- 172.21.15.19:0/2678442159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc58c0d0990 msgr2=0x7fc5700073f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:15.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.693+0000 7fc58aa00640 1 --2- 172.21.15.19:0/2678442159 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc58c0d0990 0x7fc5700073f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.693+0000 7fc58aa00640 1 -- 172.21.15.19:0/2678442159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc570005bf0 msgr2=0x7fc570005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:15.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.693+0000 7fc58aa00640 1 --2- 172.21.15.19:0/2678442159 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc570005bf0 0x7fc570005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.693+0000 7fc58aa00640 1 -- 172.21.15.19:0/2678442159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc570005730 con 0x7fc570006a10 2024-03-28T13:19:15.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.694+0000 7fc58aa00640 1 --2- 172.21.15.19:0/2678442159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc570006a10 0x7fc570006e90 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fc578003db0 tx=0x7fc570007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250a5eec83931f28 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:15.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.694+0000 7fc58a000640 1 -- 172.21.15.19:0/2678442159 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578037070 con 0x7fc570006a10 2024-03-28T13:19:15.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.694+0000 7fc58a000640 1 -- 172.21.15.19:0/2678442159 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc578032600 con 0x7fc570006a10 2024-03-28T13:19:15.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.694+0000 7fc58a000640 1 -- 172.21.15.19:0/2678442159 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578003980 con 0x7fc570006a10 2024-03-28T13:19:15.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.694+0000 7fc592200640 1 -- 172.21.15.19:0/2678442159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc570006a10 msgr2=0x7fc570006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:15.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.694+0000 7fc592200640 1 --2- 172.21.15.19:0/2678442159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc570006a10 0x7fc570006e90 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fc578003db0 tx=0x7fc570007a40 comp rx=0 tx=0).stop 2024-03-28T13:19:15.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.695+0000 7fc58be00640 1 -- 172.21.15.19:0/2678442159 reap_dead start 2024-03-28T13:19:15.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.695+0000 7fc592200640 1 -- 172.21.15.19:0/2678442159 shutdown_connections 2024-03-28T13:19:15.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.695+0000 7fc592200640 1 -- 172.21.15.19:0/2678442159 >> 172.21.15.19:0/2678442159 conn(0x7fc58c06b750 msgr2=0x7fc58c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:15.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.695+0000 7fc592200640 1 -- 172.21.15.19:0/2678442159 shutdown_connections 2024-03-28T13:19:15.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.695+0000 7fc592200640 1 -- 172.21.15.19:0/2678442159 wait complete. 2024-03-28T13:19:15.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.696+0000 7fc592200640 1 Processor -- start 2024-03-28T13:19:15.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.697+0000 7fc592200640 1 -- start start 2024-03-28T13:19:15.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.697+0000 7fc592200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc570006a10 0x7fc58c1e5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:15.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.697+0000 7fc592200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc58c099820 0x7fc58c0720c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:15.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.697+0000 7fc592200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc58c099ca0 0x7fc58c072600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:15.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.697+0000 7fc592200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c15bba0 con 0x7fc570006a10 2024-03-28T13:19:15.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.697+0000 7fc592200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc58c15ba20 con 0x7fc58c099ca0 2024-03-28T13:19:15.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.697+0000 7fc592200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc58c0b3cf0 con 0x7fc58c099820 2024-03-28T13:19:15.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.698+0000 7fc58b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc570006a10 0x7fc58c1e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:15.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.698+0000 7fc58b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc570006a10 0x7fc58c1e5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53822/0 (socket says 172.21.15.19:53822) 2024-03-28T13:19:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.698+0000 7fc58b400640 1 -- 172.21.15.19:0/2491026873 learned_addr learned my addr 172.21.15.19:0/2491026873 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.698+0000 7fc58aa00640 1 --2- 172.21.15.19:0/2491026873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc58c099820 0x7fc58c0720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.698+0000 7fc58b400640 1 -- 172.21.15.19:0/2491026873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc58c099820 msgr2=0x7fc58c0720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.698+0000 7fc58b400640 1 --2- 172.21.15.19:0/2491026873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc58c099820 0x7fc58c0720c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.698+0000 7fc58b400640 1 -- 172.21.15.19:0/2491026873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc58c099ca0 msgr2=0x7fc58c072600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.698+0000 7fc58b400640 1 --2- 172.21.15.19:0/2491026873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc58c099ca0 0x7fc58c072600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.698+0000 7fc58b400640 1 -- 172.21.15.19:0/2491026873 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58c1e47c0 con 0x7fc570006a10 2024-03-28T13:19:15.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.699+0000 7fc58b400640 1 --2- 172.21.15.19:0/2491026873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc570006a10 0x7fc58c1e5540 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00c7a0 tx=0x7fc57c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:15.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.699+0000 7fc583400640 1 -- 172.21.15.19:0/2491026873 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c018070 con 0x7fc570006a10 2024-03-28T13:19:15.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.699+0000 7fc583400640 1 -- 172.21.15.19:0/2491026873 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57c0074e0 con 0x7fc570006a10 2024-03-28T13:19:15.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.699+0000 7fc583400640 1 -- 172.21.15.19:0/2491026873 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c004a00 con 0x7fc570006a10 2024-03-28T13:19:15.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.699+0000 7fc592200640 1 -- 172.21.15.19:0/2491026873 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58c1e4a60 con 0x7fc570006a10 2024-03-28T13:19:15.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.699+0000 7fc592200640 1 -- 172.21.15.19:0/2491026873 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc58c0b0cb0 con 0x7fc570006a10 2024-03-28T13:19:15.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.701+0000 7fc592200640 1 -- 172.21.15.19:0/2491026873 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc58c0685c0 con 0x7fc570006a10 2024-03-28T13:19:15.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.701+0000 7fc583400640 1 -- 172.21.15.19:0/2491026873 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc57c004ba0 con 0x7fc570006a10 2024-03-28T13:19:15.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.702+0000 7fc583400640 1 --2- 172.21.15.19:0/2491026873 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc56c06c530 0x7fc56c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:15.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.702+0000 7fc583400640 1 -- 172.21.15.19:0/2491026873 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7fc57c08ac70 con 0x7fc570006a10 2024-03-28T13:19:15.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.703+0000 7fc58aa00640 1 --2- 172.21.15.19:0/2491026873 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc56c06c530 0x7fc56c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:15.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.704+0000 7fc58aa00640 1 --2- 172.21.15.19:0/2491026873 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc56c06c530 0x7fc56c06e9f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc578034000 tx=0x7fc578009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:15.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:15.706+0000 7fc583400640 1 -- 172.21.15.19:0/2491026873 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc57c058ba0 con 0x7fc570006a10 2024-03-28T13:19:16.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.134+0000 7fc592200640 1 -- 172.21.15.19:0/2491026873 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc58c0b3ad0 con 0x7fc570006a10 2024-03-28T13:19:16.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.134+0000 7fc583400640 1 -- 172.21.15.19:0/2491026873 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) ==== 80+0+0 (secure 0 0 0) 0x7fc57c05cea0 con 0x7fc570006a10 2024-03-28T13:19:16.134 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-03-28T13:19:16.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.137+0000 7fc592200640 1 -- 172.21.15.19:0/2491026873 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc56c06c530 msgr2=0x7fc56c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:16.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.137+0000 7fc592200640 1 --2- 172.21.15.19:0/2491026873 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc56c06c530 0x7fc56c06e9f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc578034000 tx=0x7fc578009030 comp rx=0 tx=0).stop 2024-03-28T13:19:16.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.137+0000 7fc592200640 1 -- 172.21.15.19:0/2491026873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc570006a10 msgr2=0x7fc58c1e5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:16.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.137+0000 7fc592200640 1 --2- 172.21.15.19:0/2491026873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc570006a10 0x7fc58c1e5540 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00c7a0 tx=0x7fc57c00cc60 comp rx=0 tx=0).stop 2024-03-28T13:19:16.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.137+0000 7fc592200640 1 -- 172.21.15.19:0/2491026873 shutdown_connections 2024-03-28T13:19:16.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.137+0000 7fc592200640 1 --2- 172.21.15.19:0/2491026873 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc56c06c530 0x7fc56c06e9f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:16.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.137+0000 7fc592200640 1 --2- 172.21.15.19:0/2491026873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc58c099ca0 0x7fc58c072600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:16.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.137+0000 7fc592200640 1 --2- 172.21.15.19:0/2491026873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc58c099820 0x7fc58c0720c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:16.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.137+0000 7fc592200640 1 --2- 172.21.15.19:0/2491026873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc570006a10 0x7fc58c1e5540 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:16.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.137+0000 7fc592200640 1 -- 172.21.15.19:0/2491026873 >> 172.21.15.19:0/2491026873 conn(0x7fc58c06b750 msgr2=0x7fc58c06e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:16.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.137+0000 7fc592200640 1 -- 172.21.15.19:0/2491026873 shutdown_connections 2024-03-28T13:19:16.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.137+0000 7fc592200640 1 -- 172.21.15.19:0/2491026873 wait complete. 2024-03-28T13:19:16.152 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:19:16.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:16.153 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T13:19:16.251 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:19:16.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:16.251 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:19:16.301 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:19:16.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.506+0000 7f24b3e00640 1 Processor -- start 2024-03-28T13:19:16.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.506+0000 7f24b3e00640 1 -- start start 2024-03-28T13:19:16.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.508+0000 7f24b3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b40d2310 0x7f24b40ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:16.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.509+0000 7f24b3e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f24b40cf530 0x7f24b40b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:16.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.509+0000 7f24b3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24b415e880 con 0x7f24b415e4a0 2024-03-28T13:19:16.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.509+0000 7f24b3e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f24b40d34e0 con 0x7f24b40cf530 2024-03-28T13:19:16.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.509+0000 7f24b3e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f24b40d28d0 con 0x7f24b40d2310 2024-03-28T13:19:16.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.509+0000 7f24b2a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f24b415e4a0 0x7f24b40b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34146/0 (socket says 172.21.15.19:34146) 2024-03-28T13:19:16.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.509+0000 7f24b2a00640 1 -- 172.21.15.19:0/1689458874 learned_addr learned my addr 172.21.15.19:0/1689458874 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:16.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.509+0000 7f24b3400640 1 --2- 172.21.15.19:0/1689458874 >> v2:172.21.15.188:3300/0 conn(0x7f24b40cf530 0x7f24b40b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:16.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.509+0000 7f24b2000640 1 --2- 172.21.15.19:0/1689458874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b40d2310 0x7f24b40ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:16.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 -- 172.21.15.19:0/1689458874 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2884335433 0 0) 0x7f24b415e880 con 0x7f24b415e4a0 2024-03-28T13:19:16.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 -- 172.21.15.19:0/1689458874 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2498003520 con 0x7f24b415e4a0 2024-03-28T13:19:16.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 -- 172.21.15.19:0/1689458874 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2280474233 0 0) 0x7f2498003520 con 0x7f24b415e4a0 2024-03-28T13:19:16.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 -- 172.21.15.19:0/1689458874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b40d2310 msgr2=0x7f24b40ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:16.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 --2- 172.21.15.19:0/1689458874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b40d2310 0x7f24b40ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:16.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 -- 172.21.15.19:0/1689458874 >> v2:172.21.15.188:3300/0 conn(0x7f24b40cf530 msgr2=0x7f24b40b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:16.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 --2- 172.21.15.19:0/1689458874 >> v2:172.21.15.188:3300/0 conn(0x7f24b40cf530 0x7f24b40b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:16.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 -- 172.21.15.19:0/1689458874 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b40d2bb0 con 0x7f24b415e4a0 2024-03-28T13:19:16.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 -- 172.21.15.19:0/1689458874 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f24a4001670 con 0x7f24b415e4a0 2024-03-28T13:19:16.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 -- 172.21.15.19:0/1689458874 >> v1:172.21.15.19:6789/0 conn(0x7f24b415e4a0 legacy=0x7f24b40b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:16.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 --2- 172.21.15.19:0/1689458874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24980039b0 0x7f2498003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:16.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 --2- 172.21.15.19:0/1689458874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2498004800 0x7f2498004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:16.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 --2- 172.21.15.19:0/1689458874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b40d2310 0x7f24980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:16.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 -- 172.21.15.19:0/1689458874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24b40d34e0 con 0x7f24980039b0 2024-03-28T13:19:16.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 -- 172.21.15.19:0/1689458874 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f24b40d28d0 con 0x7f2498004800 2024-03-28T13:19:16.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.510+0000 7f24b1600640 1 -- 172.21.15.19:0/1689458874 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f249800b9c0 con 0x7f24b40d2310 2024-03-28T13:19:16.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.511+0000 7f24b2a00640 1 --2- 172.21.15.19:0/1689458874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24980039b0 0x7f2498003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:16.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.511+0000 7f24b2000640 1 --2- 172.21.15.19:0/1689458874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2498004800 0x7f2498004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:16.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.511+0000 7f24b3400640 1 --2- 172.21.15.19:0/1689458874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b40d2310 0x7f24980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:16.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.511+0000 7f24b2a00640 1 -- 172.21.15.19:0/1689458874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b40d2310 msgr2=0x7f24980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:16.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.511+0000 7f24b2a00640 1 --2- 172.21.15.19:0/1689458874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b40d2310 0x7f24980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:16.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.511+0000 7f24b2a00640 1 -- 172.21.15.19:0/1689458874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2498004800 msgr2=0x7f2498004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:16.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.511+0000 7f24b2a00640 1 --2- 172.21.15.19:0/1689458874 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2498004800 0x7f2498004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:16.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.511+0000 7f24b2a00640 1 -- 172.21.15.19:0/1689458874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2498003520 con 0x7f24980039b0 2024-03-28T13:19:16.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.513+0000 7f24b2a00640 1 --2- 172.21.15.19:0/1689458874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24980039b0 0x7f2498003dd0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f24a400c240 tx=0x7f2498005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8e637aea1412d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:16.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.513+0000 7f24b1600640 1 -- 172.21.15.19:0/1689458874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4007d30 con 0x7f24980039b0 2024-03-28T13:19:16.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.513+0000 7f24b1600640 1 -- 172.21.15.19:0/1689458874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a4030070 con 0x7f24980039b0 2024-03-28T13:19:16.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.513+0000 7f24b1600640 1 -- 172.21.15.19:0/1689458874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a402cd60 con 0x7f24980039b0 2024-03-28T13:19:16.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.514+0000 7f24b3e00640 1 -- 172.21.15.19:0/1689458874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24980039b0 msgr2=0x7f2498003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:16.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.514+0000 7f24b3e00640 1 --2- 172.21.15.19:0/1689458874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24980039b0 0x7f2498003dd0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f24a400c240 tx=0x7f2498005830 comp rx=0 tx=0).stop 2024-03-28T13:19:16.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.514+0000 7f24b3400640 1 -- 172.21.15.19:0/1689458874 reap_dead start 2024-03-28T13:19:16.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.514+0000 7f24b3e00640 1 -- 172.21.15.19:0/1689458874 shutdown_connections 2024-03-28T13:19:16.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.514+0000 7f24b3e00640 1 -- 172.21.15.19:0/1689458874 >> 172.21.15.19:0/1689458874 conn(0x7f24b40676d0 msgr2=0x7f24b4067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:16.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.514+0000 7f24b3e00640 1 -- 172.21.15.19:0/1689458874 shutdown_connections 2024-03-28T13:19:16.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.514+0000 7f24b3e00640 1 -- 172.21.15.19:0/1689458874 wait complete. 2024-03-28T13:19:16.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.516+0000 7f24b3e00640 1 Processor -- start 2024-03-28T13:19:16.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.516+0000 7f24b3e00640 1 -- start start 2024-03-28T13:19:16.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.517+0000 7f24b3e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24980039b0 0x7f24b40c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:16.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.517+0000 7f24b3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b40b7db0 0x7f24b40b8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:16.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.517+0000 7f24b3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24b4155090 0x7f24b4155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:16.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.517+0000 7f24b3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24b40d3970 con 0x7f24b4155090 2024-03-28T13:19:16.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.517+0000 7f24b3e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f24b40d3170 con 0x7f24980039b0 2024-03-28T13:19:16.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.517+0000 7f24b3e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f24b40ceff0 con 0x7f24b40b7db0 2024-03-28T13:19:16.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.517+0000 7f24b3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24b4155090 0x7f24b4155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:16.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.517+0000 7f24b3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24b4155090 0x7f24b4155540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53842/0 (socket says 172.21.15.19:53842) 2024-03-28T13:19:16.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.517+0000 7f24b3400640 1 -- 172.21.15.19:0/3247788777 learned_addr learned my addr 172.21.15.19:0/3247788777 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:16.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.517+0000 7f24b2a00640 1 --2- 172.21.15.19:0/3247788777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24980039b0 0x7f24b40c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:16.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.518+0000 7f24b3400640 1 -- 172.21.15.19:0/3247788777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b40b7db0 msgr2=0x7f24b40b8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:16.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.518+0000 7f24b2000640 1 --2- 172.21.15.19:0/3247788777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b40b7db0 0x7f24b40b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:16.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.518+0000 7f24b3400640 1 --2- 172.21.15.19:0/3247788777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b40b7db0 0x7f24b40b8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:16.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.518+0000 7f24b3400640 1 -- 172.21.15.19:0/3247788777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24980039b0 msgr2=0x7f24b40c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:16.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.518+0000 7f24b3400640 1 --2- 172.21.15.19:0/3247788777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24980039b0 0x7f24b40c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:16.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.518+0000 7f24b3400640 1 -- 172.21.15.19:0/3247788777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b40b8920 con 0x7f24b4155090 2024-03-28T13:19:16.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.518+0000 7f24b2000640 1 --2- 172.21.15.19:0/3247788777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b40b7db0 0x7f24b40b8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:16.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.518+0000 7f24b3400640 1 --2- 172.21.15.19:0/3247788777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24b4155090 0x7f24b4155540 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f24b40d0d30 tx=0x7f24ac0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:16.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.518+0000 7f24aaa00640 1 -- 172.21.15.19:0/3247788777 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac003990 con 0x7f24b4155090 2024-03-28T13:19:16.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.518+0000 7f24aaa00640 1 -- 172.21.15.19:0/3247788777 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24ac01b070 con 0x7f24b4155090 2024-03-28T13:19:16.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.518+0000 7f24aaa00640 1 -- 172.21.15.19:0/3247788777 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac003b70 con 0x7f24b4155090 2024-03-28T13:19:16.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.518+0000 7f24b3e00640 1 -- 172.21.15.19:0/3247788777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b40b8c00 con 0x7f24b4155090 2024-03-28T13:19:16.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.519+0000 7f24b3e00640 1 -- 172.21.15.19:0/3247788777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b4152db0 con 0x7f24b4155090 2024-03-28T13:19:16.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.520+0000 7f24b3e00640 1 -- 172.21.15.19:0/3247788777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b4075940 con 0x7f24b4155090 2024-03-28T13:19:16.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.521+0000 7f24aaa00640 1 -- 172.21.15.19:0/3247788777 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f24ac003d10 con 0x7f24b4155090 2024-03-28T13:19:16.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.522+0000 7f24aaa00640 1 --2- 172.21.15.19:0/3247788777 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f249406c530 0x7f249406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:16.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.522+0000 7f24aaa00640 1 -- 172.21.15.19:0/3247788777 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f24ac08bf30 con 0x7f24b4155090 2024-03-28T13:19:16.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.523+0000 7f24b2a00640 1 --2- 172.21.15.19:0/3247788777 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f249406c530 0x7f249406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:16.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.524+0000 7f24b2a00640 1 --2- 172.21.15.19:0/3247788777 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f249406c530 0x7f249406e9f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f24a4034000 tx=0x7f24a4007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:16.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.526+0000 7f24aaa00640 1 -- 172.21.15.19:0/3247788777 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f24ac05a540 con 0x7f24b4155090 2024-03-28T13:19:16.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.769+0000 7f24b3e00640 1 -- 172.21.15.19:0/3247788777 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24b4154e50 con 0x7f24b4155090 2024-03-28T13:19:16.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.772+0000 7f24aaa00640 1 -- 172.21.15.19:0/3247788777 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5189 (secure 0 0 0) 0x7f24ac059e40 con 0x7f24b4155090 2024-03-28T13:19:16.772 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:16.772 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4247326257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4247326257},{"type":"v1","addr":"172.21.15.188:6836","nonce":4247326257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3452274156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3452274156},{"type":"v1","addr":"172.21.15.19:6835","nonce":3452274156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3383190607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3383190607},{"type":"v1","addr":"172.21.15.188:6837","nonce":3383190607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:18:58.541599+0000","modified":"2024-03-28T13:19:01.614319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/1020599987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1020599987},{"type":"v1","addr":"172.21.15.19:6834","nonce":1020599987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8040,"qdb_cluster":[8040]},"id":7}]} 2024-03-28T13:19:16.772 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 83 2024-03-28T13:19:16.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.775+0000 7f24b3e00640 1 -- 172.21.15.19:0/3247788777 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f249406c530 msgr2=0x7f249406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:16.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.775+0000 7f24b3e00640 1 --2- 172.21.15.19:0/3247788777 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f249406c530 0x7f249406e9f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f24a4034000 tx=0x7f24a4007000 comp rx=0 tx=0).stop 2024-03-28T13:19:16.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.775+0000 7f24b3e00640 1 -- 172.21.15.19:0/3247788777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24b4155090 msgr2=0x7f24b4155540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:16.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.775+0000 7f24b3e00640 1 --2- 172.21.15.19:0/3247788777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24b4155090 0x7f24b4155540 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f24b40d0d30 tx=0x7f24ac0115a0 comp rx=0 tx=0).stop 2024-03-28T13:19:16.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.775+0000 7f24b3e00640 1 -- 172.21.15.19:0/3247788777 shutdown_connections 2024-03-28T13:19:16.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.775+0000 7f24b3e00640 1 --2- 172.21.15.19:0/3247788777 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f249406c530 0x7f249406e9f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:16.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.775+0000 7f24b3e00640 1 --2- 172.21.15.19:0/3247788777 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24b4155090 0x7f24b4155540 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:16.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.775+0000 7f24b3e00640 1 --2- 172.21.15.19:0/3247788777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b40b7db0 0x7f24b40b8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:16.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.775+0000 7f24b3e00640 1 --2- 172.21.15.19:0/3247788777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24980039b0 0x7f24b40c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:16.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.775+0000 7f24b3e00640 1 -- 172.21.15.19:0/3247788777 >> 172.21.15.19:0/3247788777 conn(0x7f24b40676d0 msgr2=0x7f24b40ca410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:16.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.775+0000 7f24b3e00640 1 -- 172.21.15.19:0/3247788777 shutdown_connections 2024-03-28T13:19:16.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:16.775+0000 7f24b3e00640 1 -- 172.21.15.19:0/3247788777 wait complete. 2024-03-28T13:19:16.793 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-28T13:19:17.042 INFO:teuthology.orchestra.run.smithi019.stdout:{"message":"","return_code":0} 2024-03-28T13:19:17.042 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-28T13:19:17.043 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:19:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.258+0000 7f3affe00640 1 Processor -- start 2024-03-28T13:19:17.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.259+0000 7f3affe00640 1 -- start start 2024-03-28T13:19:17.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.260+0000 7f3affe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3b000d2310 0x7f3b000ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.260+0000 7f3affe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b0015e880 con 0x7f3b000cf530 2024-03-28T13:19:17.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.260+0000 7f3affe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b000d28d0 con 0x7f3b0015e4a0 2024-03-28T13:19:17.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.260+0000 7f3affe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f3b000cf9a0 con 0x7f3b000d2310 2024-03-28T13:19:17.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.261+0000 7f3af7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3b000d2310 0x7f3b000ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.261+0000 7f3aff400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3b000cf530 0x7f3b000b73e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34160/0 (socket says 172.21.15.19:34160) 2024-03-28T13:19:17.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.261+0000 7f3af7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f3b000d2310 0x7f3b000ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53852/0 (socket says 172.21.15.19:53852) 2024-03-28T13:19:17.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.261+0000 7f3aff400640 1 -- 172.21.15.19:0/240684848 learned_addr learned my addr 172.21.15.19:0/240684848 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:17.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.261+0000 7f3af7e00640 1 -- 172.21.15.19:0/240684848 >> v1:172.21.15.188:6789/0 conn(0x7f3b0015e4a0 legacy=0x7f3b000b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.261+0000 7f3af7e00640 1 -- 172.21.15.19:0/240684848 >> v1:172.21.15.19:6789/0 conn(0x7f3b000cf530 legacy=0x7f3b000b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.261+0000 7f3af7e00640 1 -- 172.21.15.19:0/240684848 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b000d2bb0 con 0x7f3b000d2310 2024-03-28T13:19:17.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.263+0000 7f3af7e00640 1 --2- 172.21.15.19:0/240684848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b000d2310 0x7f3b000ceff0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3ae80097a0 tx=0x7f3ae802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07a165efcc0dd44 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:17.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.263+0000 7f3afe000640 1 -- 172.21.15.19:0/240684848 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8035020 con 0x7f3b000d2310 2024-03-28T13:19:17.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.263+0000 7f3afe000640 1 -- 172.21.15.19:0/240684848 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae8031830 con 0x7f3b000d2310 2024-03-28T13:19:17.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.263+0000 7f3afe000640 1 -- 172.21.15.19:0/240684848 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8031c00 con 0x7f3b000d2310 2024-03-28T13:19:17.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.263+0000 7f3affe00640 1 -- 172.21.15.19:0/240684848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b000d2310 msgr2=0x7f3b000ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.264+0000 7f3affe00640 1 --2- 172.21.15.19:0/240684848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b000d2310 0x7f3b000ceff0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3ae80097a0 tx=0x7f3ae802b8d0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.264+0000 7f3affe00640 1 -- 172.21.15.19:0/240684848 shutdown_connections 2024-03-28T13:19:17.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.264+0000 7f3affe00640 1 --2- 172.21.15.19:0/240684848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b000d2310 0x7f3b000ceff0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.264+0000 7f3affe00640 1 -- 172.21.15.19:0/240684848 >> 172.21.15.19:0/240684848 conn(0x7f3b000676d0 msgr2=0x7f3b00067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:17.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.264+0000 7f3affe00640 1 -- 172.21.15.19:0/240684848 shutdown_connections 2024-03-28T13:19:17.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.264+0000 7f3affe00640 1 -- 172.21.15.19:0/240684848 wait complete. 2024-03-28T13:19:17.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.266+0000 7f3affe00640 1 Processor -- start 2024-03-28T13:19:17.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.266+0000 7f3affe00640 1 -- start start 2024-03-28T13:19:17.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.267+0000 7f3affe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b000cf530 0x7f3b000c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.267+0000 7f3affe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b000d2310 0x7f3b001570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.267+0000 7f3affe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b0015e4a0 0x7f3b00157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.267+0000 7f3affe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3b000d3170 con 0x7f3b0015e4a0 2024-03-28T13:19:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.267+0000 7f3affe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3b000d34e0 con 0x7f3b000d2310 2024-03-28T13:19:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.267+0000 7f3affe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3b000d3970 con 0x7f3b000cf530 2024-03-28T13:19:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.267+0000 7f3aff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b0015e4a0 0x7f3b00157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.267+0000 7f3aff400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b0015e4a0 0x7f3b00157630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53864/0 (socket says 172.21.15.19:53864) 2024-03-28T13:19:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.267+0000 7f3aff400640 1 -- 172.21.15.19:0/1163996470 learned_addr learned my addr 172.21.15.19:0/1163996470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.267+0000 7f3af7e00640 1 --2- 172.21.15.19:0/1163996470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b000d2310 0x7f3b001570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.267+0000 7f3afea00640 1 --2- 172.21.15.19:0/1163996470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b000cf530 0x7f3b000c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.267+0000 7f3aff400640 1 -- 172.21.15.19:0/1163996470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b000cf530 msgr2=0x7f3b000c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.267+0000 7f3aff400640 1 --2- 172.21.15.19:0/1163996470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b000cf530 0x7f3b000c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.267+0000 7f3aff400640 1 -- 172.21.15.19:0/1163996470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b000d2310 msgr2=0x7f3b001570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.268+0000 7f3aff400640 1 --2- 172.21.15.19:0/1163996470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b000d2310 0x7f3b001570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.268+0000 7f3aff400640 1 -- 172.21.15.19:0/1163996470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b00156d50 con 0x7f3b0015e4a0 2024-03-28T13:19:17.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.268+0000 7f3afea00640 1 --2- 172.21.15.19:0/1163996470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b000cf530 0x7f3b000c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:17.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.268+0000 7f3aff400640 1 --2- 172.21.15.19:0/1163996470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b0015e4a0 0x7f3b00157630 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3af800cbf0 tx=0x7f3af800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:17.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.268+0000 7f3af7400640 1 -- 172.21.15.19:0/1163996470 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8017070 con 0x7f3b0015e4a0 2024-03-28T13:19:17.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.268+0000 7f3af7400640 1 -- 172.21.15.19:0/1163996470 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af800ad40 con 0x7f3b0015e4a0 2024-03-28T13:19:17.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.268+0000 7f3af7400640 1 -- 172.21.15.19:0/1163996470 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8012600 con 0x7f3b0015e4a0 2024-03-28T13:19:17.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.268+0000 7f3affe00640 1 -- 172.21.15.19:0/1163996470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b00156a70 con 0x7f3b0015e4a0 2024-03-28T13:19:17.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.268+0000 7f3affe00640 1 -- 172.21.15.19:0/1163996470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b001f58d0 con 0x7f3b0015e4a0 2024-03-28T13:19:17.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.269+0000 7f3affe00640 1 -- 172.21.15.19:0/1163996470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b00075940 con 0x7f3b0015e4a0 2024-03-28T13:19:17.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.272+0000 7f3af7400640 1 -- 172.21.15.19:0/1163996470 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3af801a070 con 0x7f3b0015e4a0 2024-03-28T13:19:17.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.272+0000 7f3af7400640 1 --2- 172.21.15.19:0/1163996470 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3adc06c530 0x7f3adc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.273+0000 7f3af7400640 1 -- 172.21.15.19:0/1163996470 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f3af808b400 con 0x7f3b0015e4a0 2024-03-28T13:19:17.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.273+0000 7f3afea00640 1 --2- 172.21.15.19:0/1163996470 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3adc06c530 0x7f3adc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.274+0000 7f3afea00640 1 --2- 172.21.15.19:0/1163996470 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3adc06c530 0x7f3adc06e9f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3b000ce2d0 tx=0x7f3af0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:17.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.278+0000 7f3af7400640 1 -- 172.21.15.19:0/1163996470 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3af8059330 con 0x7f3b0015e4a0 2024-03-28T13:19:17.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.527+0000 7f3affe00640 1 -- 172.21.15.19:0/1163996470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b000b4c70 con 0x7f3b0015e4a0 2024-03-28T13:19:17.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.529+0000 7f3af7400640 1 -- 172.21.15.19:0/1163996470 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5189 (secure 0 0 0) 0x7f3af805d630 con 0x7f3b0015e4a0 2024-03-28T13:19:17.529 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:17.529 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4247326257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4247326257},{"type":"v1","addr":"172.21.15.188:6836","nonce":4247326257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3452274156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3452274156},{"type":"v1","addr":"172.21.15.19:6835","nonce":3452274156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3383190607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3383190607},{"type":"v1","addr":"172.21.15.188:6837","nonce":3383190607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:18:58.541599+0000","modified":"2024-03-28T13:19:01.614319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/1020599987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1020599987},{"type":"v1","addr":"172.21.15.19:6834","nonce":1020599987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8040,"qdb_cluster":[8040]},"id":7}]} 2024-03-28T13:19:17.529 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 83 2024-03-28T13:19:17.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.532+0000 7f3affe00640 1 -- 172.21.15.19:0/1163996470 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3adc06c530 msgr2=0x7f3adc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.532+0000 7f3affe00640 1 --2- 172.21.15.19:0/1163996470 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3adc06c530 0x7f3adc06e9f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3b000ce2d0 tx=0x7f3af0009040 comp rx=0 tx=0).stop 2024-03-28T13:19:17.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.532+0000 7f3affe00640 1 -- 172.21.15.19:0/1163996470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b0015e4a0 msgr2=0x7f3b00157630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.532+0000 7f3affe00640 1 --2- 172.21.15.19:0/1163996470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b0015e4a0 0x7f3b00157630 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3af800cbf0 tx=0x7f3af800a430 comp rx=0 tx=0).stop 2024-03-28T13:19:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.532+0000 7f3affe00640 1 -- 172.21.15.19:0/1163996470 shutdown_connections 2024-03-28T13:19:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.532+0000 7f3affe00640 1 --2- 172.21.15.19:0/1163996470 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3adc06c530 0x7f3adc06e9f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.532+0000 7f3affe00640 1 --2- 172.21.15.19:0/1163996470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3b0015e4a0 0x7f3b00157630 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.532+0000 7f3affe00640 1 --2- 172.21.15.19:0/1163996470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b000d2310 0x7f3b001570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.532+0000 7f3affe00640 1 --2- 172.21.15.19:0/1163996470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b000cf530 0x7f3b000c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.532+0000 7f3affe00640 1 -- 172.21.15.19:0/1163996470 >> 172.21.15.19:0/1163996470 conn(0x7f3b000676d0 msgr2=0x7f3b000b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:17.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.532+0000 7f3affe00640 1 -- 172.21.15.19:0/1163996470 shutdown_connections 2024-03-28T13:19:17.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.532+0000 7f3affe00640 1 -- 172.21.15.19:0/1163996470 wait complete. 2024-03-28T13:19:17.549 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-03-28T13:19:17.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.664+0000 7fa4c3786f40 1 Processor -- start 2024-03-28T13:19:17.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.664+0000 7fa4c3786f40 1 -- start start 2024-03-28T13:19:17.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.665+0000 7fa4c3786f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56354fbfa510 0x56354fc94d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.665+0000 7fa4c3786f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354fc8a590 0x56354fbf82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.665+0000 7fa4c3786f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56354fbf9760 0x56354fbf8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.665+0000 7fa4c3786f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56354fbb30e0 con 0x56354fc8a590 2024-03-28T13:19:17.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.665+0000 7fa4c3786f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56354fc8e560 con 0x56354fbfa510 2024-03-28T13:19:17.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.665+0000 7fa4c3786f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x56354fc8d820 con 0x56354fbf9760 2024-03-28T13:19:17.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.665+0000 7fa4bbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354fc8a590 0x56354fbf82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.665+0000 7fa4bbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354fc8a590 0x56354fbf82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53878/0 (socket says 172.21.15.19:53878) 2024-03-28T13:19:17.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.665+0000 7fa4bbe00640 1 -- 172.21.15.19:0/1640945600 learned_addr learned my addr 172.21.15.19:0/1640945600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:17.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.666+0000 7fa4c0a00640 1 --2- 172.21.15.19:0/1640945600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56354fbfa510 0x56354fc94d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.666+0000 7fa4bbe00640 1 -- 172.21.15.19:0/1640945600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56354fbf9760 msgr2=0x56354fbf8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:17.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.666+0000 7fa4bbe00640 1 --2- 172.21.15.19:0/1640945600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56354fbf9760 0x56354fbf8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.666+0000 7fa4bbe00640 1 -- 172.21.15.19:0/1640945600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56354fbfa510 msgr2=0x56354fc94d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:17.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.666+0000 7fa4bbe00640 1 --2- 172.21.15.19:0/1640945600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56354fbfa510 0x56354fc94d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.666+0000 7fa4bbe00640 1 -- 172.21.15.19:0/1640945600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56354faf74b0 con 0x56354fc8a590 2024-03-28T13:19:17.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.666+0000 7fa4c0a00640 1 --2- 172.21.15.19:0/1640945600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56354fbfa510 0x56354fc94d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:17.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.666+0000 7fa4bbe00640 1 --2- 172.21.15.19:0/1640945600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354fc8a590 0x56354fbf82f0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac000c00 tx=0x7fa4ac04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6bfd344fc922997 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:17.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.667+0000 7fa4baa00640 1 -- 172.21.15.19:0/1640945600 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac057040 con 0x56354fc8a590 2024-03-28T13:19:17.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.667+0000 7fa4baa00640 1 -- 172.21.15.19:0/1640945600 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ac062070 con 0x56354fc8a590 2024-03-28T13:19:17.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.667+0000 7fa4baa00640 1 -- 172.21.15.19:0/1640945600 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac05c8a0 con 0x56354fc8a590 2024-03-28T13:19:17.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.667+0000 7fa4c3786f40 1 -- 172.21.15.19:0/1640945600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354fc8a590 msgr2=0x56354fbf82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.667+0000 7fa4c3786f40 1 --2- 172.21.15.19:0/1640945600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354fc8a590 0x56354fbf82f0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac000c00 tx=0x7fa4ac04c000 comp rx=0 tx=0).stop 2024-03-28T13:19:17.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.667+0000 7fa4c3786f40 1 -- 172.21.15.19:0/1640945600 shutdown_connections 2024-03-28T13:19:17.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.667+0000 7fa4c3786f40 1 --2- 172.21.15.19:0/1640945600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56354fbf9760 0x56354fbf8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.667+0000 7fa4c3786f40 1 --2- 172.21.15.19:0/1640945600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354fc8a590 0x56354fbf82f0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.667+0000 7fa4c3786f40 1 --2- 172.21.15.19:0/1640945600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56354fbfa510 0x56354fc94d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.667+0000 7fa4c3786f40 1 -- 172.21.15.19:0/1640945600 >> 172.21.15.19:0/1640945600 conn(0x56354fbb38a0 msgr2=0x56354fbb4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:17.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.667+0000 7fa4c3786f40 1 -- 172.21.15.19:0/1640945600 shutdown_connections 2024-03-28T13:19:17.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.667+0000 7fa4c3786f40 1 -- 172.21.15.19:0/1640945600 wait complete. 2024-03-28T13:19:17.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.668+0000 7fa4c3786f40 1 Processor -- start 2024-03-28T13:19:17.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.668+0000 7fa4c3786f40 1 -- start start 2024-03-28T13:19:17.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.669+0000 7fa4c3786f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354fbf9760 0x56354fc7fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.669+0000 7fa4c3786f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56354fbfa510 0x56354fc83ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.669+0000 7fa4c3786f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56354fc8a590 0x56354fbb3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.669+0000 7fa4c3786f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56354fc8ce50 con 0x56354fbf9760 2024-03-28T13:19:17.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.669+0000 7fa4c3786f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56354fc8e000 con 0x56354fbfa510 2024-03-28T13:19:17.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.669+0000 7fa4c3786f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x56354fbe4e70 con 0x56354fc8a590 2024-03-28T13:19:17.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.669+0000 7fa4c0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354fbf9760 0x56354fc7fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.670+0000 7fa4c0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354fbf9760 0x56354fc7fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53882/0 (socket says 172.21.15.19:53882) 2024-03-28T13:19:17.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.670+0000 7fa4c0a00640 1 -- 172.21.15.19:0/3129641224 learned_addr learned my addr 172.21.15.19:0/3129641224 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:17.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.670+0000 7fa4bbe00640 1 --2- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56354fbfa510 0x56354fc83ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.670+0000 7fa4c1400640 1 --2- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56354fc8a590 0x56354fbb3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.670+0000 7fa4c0a00640 1 -- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56354fc8a590 msgr2=0x56354fbb3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.670+0000 7fa4c0a00640 1 --2- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56354fc8a590 0x56354fbb3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.670+0000 7fa4c0a00640 1 -- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56354fbfa510 msgr2=0x56354fc83ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.670+0000 7fa4c0a00640 1 --2- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56354fbfa510 0x56354fc83ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.670+0000 7fa4c0a00640 1 -- 172.21.15.19:0/3129641224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56354fbb6ab0 con 0x56354fbf9760 2024-03-28T13:19:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.670+0000 7fa4c0a00640 1 --2- 172.21.15.19:0/3129641224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354fbf9760 0x56354fc7fb90 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400bfa0 tx=0x7fa4b40074b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.671+0000 7fa4b9600640 1 -- 172.21.15.19:0/3129641224 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b400b0d0 con 0x56354fbf9760 2024-03-28T13:19:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.671+0000 7fa4b9600640 1 -- 172.21.15.19:0/3129641224 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b400b270 con 0x56354fbf9760 2024-03-28T13:19:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.671+0000 7fa4b9600640 1 -- 172.21.15.19:0/3129641224 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4015660 con 0x56354fbf9760 2024-03-28T13:19:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.671+0000 7fa4c3786f40 1 -- 172.21.15.19:0/3129641224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56354fbb6d50 con 0x56354fbf9760 2024-03-28T13:19:17.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.671+0000 7fa4c3786f40 1 -- 172.21.15.19:0/3129641224 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56354fbb7310 con 0x56354fbf9760 2024-03-28T13:19:17.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.673+0000 7fa4b9600640 1 -- 172.21.15.19:0/3129641224 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa4b401e440 con 0x56354fbf9760 2024-03-28T13:19:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.674+0000 7fa4b9600640 1 --2- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa4a006af10 0x7fa4a006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.674+0000 7fa4b9600640 1 -- 172.21.15.19:0/3129641224 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7fa4b4089b90 con 0x56354fbf9760 2024-03-28T13:19:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.675+0000 7fa4c3786f40 1 --2- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x56354fbb5da0 0x56354fbb6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.675+0000 7fa4c3786f40 1 -- 172.21.15.19:0/3129641224 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e105) -- 0x56354fd25160 con 0x56354fbb5da0 2024-03-28T13:19:17.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.675+0000 7fa4bbe00640 1 --2- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa4a006af10 0x7fa4a006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.675+0000 7fa4c1400640 1 --2- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x56354fbb5da0 0x56354fbb6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.676+0000 7fa4bbe00640 1 --2- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa4a006af10 0x7fa4a006d3d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac055040 tx=0x7fa4ac04f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.676+0000 7fa4c1400640 1 --2- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x56354fbb5da0 0x56354fbb6180 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:17.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.677+0000 7fa4c1400640 1 -- 172.21.15.19:0/3129641224 <== osd.5 v2:172.21.15.188:6800/3324247755 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7fa4bc00e5e0 con 0x56354fbb5da0 2024-03-28T13:19:17.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.677+0000 7fa4c3786f40 1 -- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x56354fbb5da0 msgr2=0x56354fbb6180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.677+0000 7fa4c3786f40 1 --2- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x56354fbb5da0 0x56354fbb6180 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.677+0000 7fa4c3786f40 1 -- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa4a006af10 msgr2=0x7fa4a006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.677+0000 7fa4c3786f40 1 --2- 172.21.15.19:0/3129641224 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa4a006af10 0x7fa4a006d3d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac055040 tx=0x7fa4ac04f000 comp rx=0 tx=0).stop 2024-03-28T13:19:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.677+0000 7fa4c3786f40 1 -- 172.21.15.19:0/3129641224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354fbf9760 msgr2=0x56354fc7fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.677+0000 7fa4c3786f40 1 --2- 172.21.15.19:0/3129641224 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56354fbf9760 0x56354fc7fb90 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fa4b400bfa0 tx=0x7fa4b40074b0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.677+0000 7fa4c1400640 1 -- 172.21.15.19:0/3129641224 reap_dead start 2024-03-28T13:19:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.677+0000 7fa4c3786f40 1 -- 172.21.15.19:0/3129641224 shutdown_connections 2024-03-28T13:19:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.677+0000 7fa4c3786f40 1 -- 172.21.15.19:0/3129641224 >> 172.21.15.19:0/3129641224 conn(0x56354fbb38a0 msgr2=0x56354fc88550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.678+0000 7fa4c3786f40 1 -- 172.21.15.19:0/3129641224 shutdown_connections 2024-03-28T13:19:17.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.678+0000 7fa4c3786f40 1 -- 172.21.15.19:0/3129641224 wait complete. 2024-03-28T13:19:17.681 INFO:teuthology.orchestra.run.smithi019.stdout:����� 2024-03-28T13:19:17.682 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:19:17.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.932+0000 7f0c1f200640 1 Processor -- start 2024-03-28T13:19:17.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.933+0000 7f0c1f200640 1 -- start start 2024-03-28T13:19:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.934+0000 7f0c1f200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0c2015e440 0x7f0c200cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.934+0000 7f0c1f200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0c200cf0f0 0x7f0c200b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.934+0000 7f0c1f200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c200b7580 con 0x7f0c2015e820 2024-03-28T13:19:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.934+0000 7f0c1f200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0c200cf670 con 0x7f0c2015e440 2024-03-28T13:19:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.934+0000 7f0c1f200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f0c200cf950 con 0x7f0c200cf0f0 2024-03-28T13:19:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.935+0000 7f0c1de00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0c2015e440 0x7f0c200cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.935+0000 7f0c1de00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0c2015e440 0x7f0c200cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53894/0 (socket says 172.21.15.19:53894) 2024-03-28T13:19:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.935+0000 7f0c1de00640 1 -- 172.21.15.19:0/3004399857 learned_addr learned my addr 172.21.15.19:0/3004399857 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.935+0000 7f0c1de00640 1 -- 172.21.15.19:0/3004399857 >> v2:172.21.15.188:3300/0 conn(0x7f0c200cf0f0 msgr2=0x7f0c200b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.935+0000 7f0c1de00640 1 --2- 172.21.15.19:0/3004399857 >> v2:172.21.15.188:3300/0 conn(0x7f0c200cf0f0 0x7f0c200b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.935+0000 7f0c1de00640 1 -- 172.21.15.19:0/3004399857 >> v1:172.21.15.188:6789/0 conn(0x7f0c2015e820 legacy=0x7f0c2006c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.935+0000 7f0c1de00640 1 -- 172.21.15.19:0/3004399857 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c200d2670 con 0x7f0c2015e440 2024-03-28T13:19:17.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.936+0000 7f0c1de00640 1 --2- 172.21.15.19:0/3004399857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2015e440 0x7f0c200cebb0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f0c200d19a0 tx=0x7f0c1002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47960e507ae9ba1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:17.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.937+0000 7f0c1d400640 1 -- 172.21.15.19:0/3004399857 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10035020 con 0x7f0c2015e440 2024-03-28T13:19:17.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.937+0000 7f0c1d400640 1 -- 172.21.15.19:0/3004399857 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c10031870 con 0x7f0c2015e440 2024-03-28T13:19:17.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.937+0000 7f0c1d400640 1 -- 172.21.15.19:0/3004399857 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c10031c40 con 0x7f0c2015e440 2024-03-28T13:19:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.937+0000 7f0c1f200640 1 -- 172.21.15.19:0/3004399857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2015e440 msgr2=0x7f0c200cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.937+0000 7f0c1f200640 1 --2- 172.21.15.19:0/3004399857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2015e440 0x7f0c200cebb0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f0c200d19a0 tx=0x7f0c1002b910 comp rx=0 tx=0).stop 2024-03-28T13:19:17.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.938+0000 7f0c1f200640 1 -- 172.21.15.19:0/3004399857 shutdown_connections 2024-03-28T13:19:17.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.938+0000 7f0c1f200640 1 --2- 172.21.15.19:0/3004399857 >> v2:172.21.15.188:3300/0 conn(0x7f0c200cf0f0 0x7f0c200b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.938+0000 7f0c1f200640 1 --2- 172.21.15.19:0/3004399857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2015e440 0x7f0c200cebb0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.938+0000 7f0c1f200640 1 -- 172.21.15.19:0/3004399857 >> 172.21.15.19:0/3004399857 conn(0x7f0c200676a0 msgr2=0x7f0c20067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:17.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.938+0000 7f0c1f200640 1 -- 172.21.15.19:0/3004399857 shutdown_connections 2024-03-28T13:19:17.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.938+0000 7f0c1f200640 1 -- 172.21.15.19:0/3004399857 wait complete. 2024-03-28T13:19:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.939+0000 7f0c1f200640 1 Processor -- start 2024-03-28T13:19:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.940+0000 7f0c1f200640 1 -- start start 2024-03-28T13:19:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c1f200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c200cf0f0 0x7f0c20158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c1f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2015e440 0x7f0c201608d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c1f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c2015e820 0x7f0c20160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c1f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0c20162560 con 0x7f0c2015e440 2024-03-28T13:19:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c1f200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0c201623e0 con 0x7f0c200cf0f0 2024-03-28T13:19:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c1f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0c200d3e80 con 0x7f0c2015e820 2024-03-28T13:19:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c17e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2015e440 0x7f0c201608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c17e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2015e440 0x7f0c201608d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53906/0 (socket says 172.21.15.19:53906) 2024-03-28T13:19:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c17e00640 1 -- 172.21.15.19:0/1850082084 learned_addr learned my addr 172.21.15.19:0/1850082084 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c1e800640 1 --2- 172.21.15.19:0/1850082084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c2015e820 0x7f0c20160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c1de00640 1 --2- 172.21.15.19:0/1850082084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c200cf0f0 0x7f0c20158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c17e00640 1 -- 172.21.15.19:0/1850082084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c2015e820 msgr2=0x7f0c20160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c17e00640 1 --2- 172.21.15.19:0/1850082084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c2015e820 0x7f0c20160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c17e00640 1 -- 172.21.15.19:0/1850082084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c200cf0f0 msgr2=0x7f0c20158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c17e00640 1 --2- 172.21.15.19:0/1850082084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c200cf0f0 0x7f0c20158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.941+0000 7f0c17e00640 1 -- 172.21.15.19:0/1850082084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c20161670 con 0x7f0c2015e440 2024-03-28T13:19:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.942+0000 7f0c17e00640 1 --2- 172.21.15.19:0/1850082084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2015e440 0x7f0c201608d0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800b4a0 tx=0x7f0c0800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.942+0000 7f0c16000640 1 -- 172.21.15.19:0/1850082084 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08014070 con 0x7f0c2015e440 2024-03-28T13:19:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.942+0000 7f0c16000640 1 -- 172.21.15.19:0/1850082084 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c08010610 con 0x7f0c2015e440 2024-03-28T13:19:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.942+0000 7f0c16000640 1 -- 172.21.15.19:0/1850082084 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08010a00 con 0x7f0c2015e440 2024-03-28T13:19:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.942+0000 7f0c1f200640 1 -- 172.21.15.19:0/1850082084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c20161950 con 0x7f0c2015e440 2024-03-28T13:19:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.942+0000 7f0c1f200640 1 -- 172.21.15.19:0/1850082084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c200c8010 con 0x7f0c2015e440 2024-03-28T13:19:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.944+0000 7f0c1f200640 1 -- 172.21.15.19:0/1850082084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c20075940 con 0x7f0c2015e440 2024-03-28T13:19:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.946+0000 7f0c16000640 1 -- 172.21.15.19:0/1850082084 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0c08017070 con 0x7f0c2015e440 2024-03-28T13:19:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.946+0000 7f0c16000640 1 --2- 172.21.15.19:0/1850082084 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0bfc06c530 0x7f0bfc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.947+0000 7f0c16000640 1 -- 172.21.15.19:0/1850082084 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f0c0808b1d0 con 0x7f0c2015e440 2024-03-28T13:19:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.947+0000 7f0c1de00640 1 --2- 172.21.15.19:0/1850082084 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0bfc06c530 0x7f0bfc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.948+0000 7f0c1de00640 1 --2- 172.21.15.19:0/1850082084 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0bfc06c530 0x7f0bfc06e9f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0c10036000 tx=0x7f0c1003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:17.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:17.952+0000 7f0c16000640 1 -- 172.21.15.19:0/1850082084 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0c08059750 con 0x7f0c2015e440 2024-03-28T13:19:18.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18.199+0000 7f0c1f200640 1 -- 172.21.15.19:0/1850082084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0c200ba290 con 0x7f0c2015e440 2024-03-28T13:19:18.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18.200+0000 7f0c16000640 1 -- 172.21.15.19:0/1850082084 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v83) ==== 53+0+73 (secure 0 0 0) 0x7f0c08059050 con 0x7f0c2015e440 2024-03-28T13:19:18.200 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:19:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18.202+0000 7f0c1f200640 1 -- 172.21.15.19:0/1850082084 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0bfc06c530 msgr2=0x7f0bfc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18.202+0000 7f0c1f200640 1 --2- 172.21.15.19:0/1850082084 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0bfc06c530 0x7f0bfc06e9f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0c10036000 tx=0x7f0c1003c040 comp rx=0 tx=0).stop 2024-03-28T13:19:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18.202+0000 7f0c1f200640 1 -- 172.21.15.19:0/1850082084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2015e440 msgr2=0x7f0c201608d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18.202+0000 7f0c1f200640 1 --2- 172.21.15.19:0/1850082084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2015e440 0x7f0c201608d0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800b4a0 tx=0x7f0c0800b960 comp rx=0 tx=0).stop 2024-03-28T13:19:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18.203+0000 7f0c1f200640 1 -- 172.21.15.19:0/1850082084 shutdown_connections 2024-03-28T13:19:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18.203+0000 7f0c1f200640 1 --2- 172.21.15.19:0/1850082084 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0bfc06c530 0x7f0bfc06e9f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18.203+0000 7f0c1f200640 1 --2- 172.21.15.19:0/1850082084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0c2015e820 0x7f0c20160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18.203+0000 7f0c1f200640 1 --2- 172.21.15.19:0/1850082084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0c2015e440 0x7f0c201608d0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:18.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18.203+0000 7f0c1f200640 1 --2- 172.21.15.19:0/1850082084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0c200cf0f0 0x7f0c20158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18.203+0000 7f0c1f200640 1 -- 172.21.15.19:0/1850082084 >> 172.21.15.19:0/1850082084 conn(0x7f0c200676a0 msgr2=0x7f0c2015c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18.203+0000 7f0c1f200640 1 -- 172.21.15.19:0/1850082084 shutdown_connections 2024-03-28T13:19:18.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18.203+0000 7f0c1f200640 1 -- 172.21.15.19:0/1850082084 wait complete. 2024-03-28T13:19:18.221 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-28T13:19:18.221 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:19:18.221 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:19:18.221 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:19:18.221 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:19:18.221 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-28T13:19:18.222 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-28T13:19:18.222 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:19:18.222 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:19:18.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:18.222 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:19:18.268 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:19:18.268 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:19:18.268 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:19:18.269 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:19:18.269 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:19:18.269 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:19:18.269 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:19:18.269 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:19:18.269 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:19:18.269 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:19:18.269 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:19:18.269 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:19:18.269 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:19:18.269 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:19:18.269 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:19:18.269 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:19:18.270 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41996sec preferred_lft 41996sec 2024-03-28T13:19:18.270 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:19:18.270 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:19:18.270 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:19:18.270 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:19:18.270 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:19:18.270 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:19:18.270 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:19:18.270 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:19:18.270 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:19:18.270 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:19:18.270 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:19:18.270 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:19:18.271 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:19:18.271 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:19:18.271 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:19:18.271 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:19:18.271 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether e6:17:c5:8e:2b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:19:18.271 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::e417:c5ff:fe8e:2bcf/64 scope link 2024-03-28T13:19:18.271 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:19:18.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:18.272 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:19:18.398 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:19:18.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:18.403 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:19:18.450 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:19:18.450 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:19:18.451 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:19:18.451 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:19:18.451 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:19:18.452 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:19:18.452 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:19:18.452 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:19:18.452 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:19:18.452 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:19:18.452 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:19:18.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:19:18.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:19:18.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:18.690 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:19:18.745 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:19:18.745 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:19:18.747 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:19:18.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:18.747 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:19:18.842 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:19:18.842 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:19:18.897 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:19:18.897 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:19:18.897 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:19:18.947 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:19:18.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:18.948 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:19:19.085 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:19:19.085 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:19:19.085 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:19:19.107 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:19:19.107 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:19:19.107 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:19:19.107 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:19:19.107 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:19:19.133 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:19:19.133 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:19:19.133 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:19:19.133 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:19:19.134 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:19:19.134 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:19:19.134 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:19:19.134 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:19:19.135 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:19:19.135 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:19:19.135 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:19:19.135 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:19:19.135 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:19:19.142 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:19:19.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:19.142 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:19:19.225 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:19:19.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:19:19.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:19:19.348 INFO:teuthology.orchestra.run.smithi019.stdout:client6730 2024-03-28T13:19:19.349 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:19:19.349 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:19:19.401 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:19:19.401 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:19:19.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:19.453 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-03-28T13:19:19.662 INFO:teuthology.orchestra.run.smithi019.stderr:6+0 records in 2024-03-28T13:19:19.663 INFO:teuthology.orchestra.run.smithi019.stderr:6+0 records out 2024-03-28T13:19:19.663 INFO:teuthology.orchestra.run.smithi019.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.153075 s, 41.1 MB/s 2024-03-28T13:19:19.664 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:19:19.664 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:19:19.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:19.718 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-03-28T13:19:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:6+0 records in 2024-03-28T13:19:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:6+0 records out 2024-03-28T13:19:19.908 INFO:teuthology.orchestra.run.smithi019.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.133532 s, 47.1 MB/s 2024-03-28T13:19:19.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:19.910 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-28T13:19:19.966 INFO:teuthology.orchestra.run.smithi019.stdout:./ 2024-03-28T13:19:19.966 INFO:teuthology.orchestra.run.smithi019.stdout:./file2_sixmegs 2024-03-28T13:19:19.966 INFO:teuthology.orchestra.run.smithi019.stdout:./file3_sixmegs 2024-03-28T13:19:19.966 INFO:teuthology.orchestra.run.smithi019.stdout:./file1_sixmegs 2024-03-28T13:19:19.967 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:19:19.967 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:19:19.967 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:19:19.967 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:19:19.968 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-28T13:19:19.968 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:19:20.097 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:19:20.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:19:20.111 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:19:20.112 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:19:20.112 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:19:20.112 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:19:20.112 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-03-28T13:19:20.112 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:19:20.293 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627777 2024-03-28T13:19:20.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:19:20.307 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:19:20.307 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:19:20.307 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:19:20.307 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:19:20.307 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-03-28T13:19:20.307 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:19:20.487 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627778 2024-03-28T13:19:20.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:19:20.502 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:19:20.502 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:19:20.502 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:19:20.502 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:19:20.502 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-03-28T13:19:20.502 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:19:20.683 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627776 2024-03-28T13:19:20.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:19:20.696 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:19:20.696 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:19:20.793 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:19:20.793 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:19:20.839 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:19:21.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.097+0000 7efc13000640 1 Processor -- start 2024-03-28T13:19:21.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.097+0000 7efc13000640 1 -- start start 2024-03-28T13:19:21.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.099+0000 7efc13000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efc0c0d1950 0x7efc0c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:21.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.099+0000 7efc13000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efc0c0d2d60 0x7efc0c0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:21.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.099+0000 7efc13000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc0c15ebc0 con 0x7efc0c15e7e0 2024-03-28T13:19:21.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.099+0000 7efc13000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7efc0c0d1db0 con 0x7efc0c0d1950 2024-03-28T13:19:21.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.099+0000 7efc13000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7efc0c06d1c0 con 0x7efc0c0d2d60 2024-03-28T13:19:21.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.099+0000 7efc0be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efc0c15e7e0 0x7efc0c06ca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34168/0 (socket says 172.21.15.19:34168) 2024-03-28T13:19:21.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.099+0000 7efc0be00640 1 -- 172.21.15.19:0/2696718982 learned_addr learned my addr 172.21.15.19:0/2696718982 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:21.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.099+0000 7efc0b400640 1 --2- 172.21.15.19:0/2696718982 >> v2:172.21.15.19:3300/0 conn(0x7efc0c0d1950 0x7efc0c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:21.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.099+0000 7efc11000640 1 --2- 172.21.15.19:0/2696718982 >> v2:172.21.15.188:3300/0 conn(0x7efc0c0d2d60 0x7efc0c0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:21.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.100+0000 7efc0aa00640 1 -- 172.21.15.19:0/2696718982 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 118580863 0 0) 0x7efc0c15ebc0 con 0x7efc0c15e7e0 2024-03-28T13:19:21.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.100+0000 7efc0b400640 1 -- 172.21.15.19:0/2696718982 >> v2:172.21.15.188:3300/0 conn(0x7efc0c0d2d60 msgr2=0x7efc0c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:21.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.100+0000 7efc0b400640 1 --2- 172.21.15.19:0/2696718982 >> v2:172.21.15.188:3300/0 conn(0x7efc0c0d2d60 0x7efc0c0cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.100+0000 7efc0b400640 1 -- 172.21.15.19:0/2696718982 >> v1:172.21.15.19:6789/0 conn(0x7efc0c15e7e0 legacy=0x7efc0c06ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:21.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.100+0000 7efc0b400640 1 -- 172.21.15.19:0/2696718982 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c06d4a0 con 0x7efc0c0d1950 2024-03-28T13:19:21.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.100+0000 7efc11000640 1 --2- 172.21.15.19:0/2696718982 >> v2:172.21.15.188:3300/0 conn(0x7efc0c0d2d60 0x7efc0c0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:19:21.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.101+0000 7efc0b400640 1 --2- 172.21.15.19:0/2696718982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c0d1950 0x7efc0c0d2820 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0d4fa0 tx=0x7efbf802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d90f7ba9aae682a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:21.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.101+0000 7efc0aa00640 1 -- 172.21.15.19:0/2696718982 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf80315a0 con 0x7efc0c0d1950 2024-03-28T13:19:21.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.101+0000 7efc0aa00640 1 -- 172.21.15.19:0/2696718982 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbf8037070 con 0x7efc0c0d1950 2024-03-28T13:19:21.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.101+0000 7efc0aa00640 1 -- 172.21.15.19:0/2696718982 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf8031960 con 0x7efc0c0d1950 2024-03-28T13:19:21.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.102+0000 7efc13000640 1 -- 172.21.15.19:0/2696718982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c0d1950 msgr2=0x7efc0c0d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:21.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.102+0000 7efc13000640 1 --2- 172.21.15.19:0/2696718982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c0d1950 0x7efc0c0d2820 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7efc0c0d4fa0 tx=0x7efbf802b840 comp rx=0 tx=0).stop 2024-03-28T13:19:21.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.102+0000 7efc13000640 1 -- 172.21.15.19:0/2696718982 shutdown_connections 2024-03-28T13:19:21.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.102+0000 7efc13000640 1 --2- 172.21.15.19:0/2696718982 >> v2:172.21.15.188:3300/0 conn(0x7efc0c0d2d60 0x7efc0c0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.102+0000 7efc13000640 1 --2- 172.21.15.19:0/2696718982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c0d1950 0x7efc0c0d2820 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.102+0000 7efc13000640 1 -- 172.21.15.19:0/2696718982 >> 172.21.15.19:0/2696718982 conn(0x7efc0c09d7c0 msgr2=0x7efc0c06bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:21.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.102+0000 7efc13000640 1 -- 172.21.15.19:0/2696718982 shutdown_connections 2024-03-28T13:19:21.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.102+0000 7efc13000640 1 -- 172.21.15.19:0/2696718982 wait complete. 2024-03-28T13:19:21.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.104+0000 7efc13000640 1 Processor -- start 2024-03-28T13:19:21.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.104+0000 7efc13000640 1 -- start start 2024-03-28T13:19:21.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.105+0000 7efc13000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc0c0d1950 0x7efc0c15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:21.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.105+0000 7efc13000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc0c0d2d60 0x7efc0c06bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:21.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.105+0000 7efc13000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c15e7e0 0x7efc0c15b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:21.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.105+0000 7efc13000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c0cefd0 con 0x7efc0c15e7e0 2024-03-28T13:19:21.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.105+0000 7efc13000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efc0c0cf600 con 0x7efc0c0d1950 2024-03-28T13:19:21.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.105+0000 7efc13000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efc0c0d0060 con 0x7efc0c0d2d60 2024-03-28T13:19:21.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.106+0000 7efc11000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c15e7e0 0x7efc0c15b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:21.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.106+0000 7efc11000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c15e7e0 0x7efc0c15b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53924/0 (socket says 172.21.15.19:53924) 2024-03-28T13:19:21.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.106+0000 7efc11000640 1 -- 172.21.15.19:0/1996069264 learned_addr learned my addr 172.21.15.19:0/1996069264 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:21.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.106+0000 7efc0be00640 1 --2- 172.21.15.19:0/1996069264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc0c0d1950 0x7efc0c15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:21.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.106+0000 7efc11000640 1 -- 172.21.15.19:0/1996069264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc0c0d2d60 msgr2=0x7efc0c06bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:21.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.106+0000 7efc11000640 1 --2- 172.21.15.19:0/1996069264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc0c0d2d60 0x7efc0c06bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.106+0000 7efc11000640 1 -- 172.21.15.19:0/1996069264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc0c0d1950 msgr2=0x7efc0c15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:21.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.106+0000 7efc11000640 1 --2- 172.21.15.19:0/1996069264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc0c0d1950 0x7efc0c15dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.106+0000 7efc11000640 1 -- 172.21.15.19:0/1996069264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0c15ebc0 con 0x7efc0c15e7e0 2024-03-28T13:19:21.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.107+0000 7efc11000640 1 --2- 172.21.15.19:0/1996069264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c15e7e0 0x7efc0c15b990 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7efc04009fa0 tx=0x7efc0400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:21.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.107+0000 7efc03e00640 1 -- 172.21.15.19:0/1996069264 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0401a070 con 0x7efc0c15e7e0 2024-03-28T13:19:21.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.107+0000 7efc03e00640 1 -- 172.21.15.19:0/1996069264 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc04003bb0 con 0x7efc0c15e7e0 2024-03-28T13:19:21.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.107+0000 7efc13000640 1 -- 172.21.15.19:0/1996069264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0c0a0c90 con 0x7efc0c15e7e0 2024-03-28T13:19:21.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.107+0000 7efc03e00640 1 -- 172.21.15.19:0/1996069264 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc04004030 con 0x7efc0c15e7e0 2024-03-28T13:19:21.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.107+0000 7efc13000640 1 -- 172.21.15.19:0/1996069264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc0c0a1250 con 0x7efc0c15e7e0 2024-03-28T13:19:21.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.108+0000 7efc13000640 1 -- 172.21.15.19:0/1996069264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc0c0685c0 con 0x7efc0c15e7e0 2024-03-28T13:19:21.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.110+0000 7efc03e00640 1 -- 172.21.15.19:0/1996069264 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efc0401d070 con 0x7efc0c15e7e0 2024-03-28T13:19:21.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.110+0000 7efc03e00640 1 --2- 172.21.15.19:0/1996069264 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efbec06c530 0x7efbec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:21.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.111+0000 7efc03e00640 1 -- 172.21.15.19:0/1996069264 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7efc0408b9e0 con 0x7efc0c15e7e0 2024-03-28T13:19:21.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.111+0000 7efc0be00640 1 --2- 172.21.15.19:0/1996069264 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efbec06c530 0x7efbec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:21.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.112+0000 7efc0be00640 1 --2- 172.21.15.19:0/1996069264 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efbec06c530 0x7efbec06e9f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efbfc000f60 tx=0x7efbfc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:21.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.115+0000 7efc03e00640 1 -- 172.21.15.19:0/1996069264 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efc04010070 con 0x7efc0c15e7e0 2024-03-28T13:19:21.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.322+0000 7efc13000640 1 -- 172.21.15.19:0/1996069264 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc0c0ba0b0 con 0x7efc0c15e7e0 2024-03-28T13:19:21.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.327+0000 7efc03e00640 1 -- 172.21.15.19:0/1996069264 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16619 (secure 0 0 0) 0x7efc04059ff0 con 0x7efc0c15e7e0 2024-03-28T13:19:21.328 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:21.328 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":105,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:19:00.072522+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:18:56.432417+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-28T13:18:57.540515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:19:21.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.330+0000 7efc13000640 1 -- 172.21.15.19:0/1996069264 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efbec06c530 msgr2=0x7efbec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:21.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.330+0000 7efc13000640 1 --2- 172.21.15.19:0/1996069264 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efbec06c530 0x7efbec06e9f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efbfc000f60 tx=0x7efbfc009040 comp rx=0 tx=0).stop 2024-03-28T13:19:21.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.330+0000 7efc13000640 1 -- 172.21.15.19:0/1996069264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c15e7e0 msgr2=0x7efc0c15b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:21.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.330+0000 7efc13000640 1 --2- 172.21.15.19:0/1996069264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c15e7e0 0x7efc0c15b990 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7efc04009fa0 tx=0x7efc0400d560 comp rx=0 tx=0).stop 2024-03-28T13:19:21.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.330+0000 7efc13000640 1 -- 172.21.15.19:0/1996069264 shutdown_connections 2024-03-28T13:19:21.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.330+0000 7efc13000640 1 --2- 172.21.15.19:0/1996069264 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efbec06c530 0x7efbec06e9f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.330+0000 7efc13000640 1 --2- 172.21.15.19:0/1996069264 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc0c15e7e0 0x7efc0c15b990 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.330+0000 7efc13000640 1 --2- 172.21.15.19:0/1996069264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc0c0d2d60 0x7efc0c06bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.330+0000 7efc13000640 1 --2- 172.21.15.19:0/1996069264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc0c0d1950 0x7efc0c15dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.330+0000 7efc13000640 1 -- 172.21.15.19:0/1996069264 >> 172.21.15.19:0/1996069264 conn(0x7efc0c09d7c0 msgr2=0x7efc0c0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:21.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.330+0000 7efc13000640 1 -- 172.21.15.19:0/1996069264 shutdown_connections 2024-03-28T13:19:21.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.330+0000 7efc13000640 1 -- 172.21.15.19:0/1996069264 wait complete. 2024-03-28T13:19:21.350 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:18:56.432417+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:18:57.540515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:19:21.351 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:19:21.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.594+0000 7efcc7e00640 1 Processor -- start 2024-03-28T13:19:21.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.594+0000 7efcc7e00640 1 -- start start 2024-03-28T13:19:21.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.595+0000 7efcc7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcc815e4c0 0x7efcc80b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:21.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.595+0000 7efcc7e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc80d2710 con 0x7efcc80b69c0 2024-03-28T13:19:21.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.595+0000 7efcc7e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcc80d29f0 con 0x7efcc80d37b0 2024-03-28T13:19:21.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.595+0000 7efcc7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efcc815e8a0 con 0x7efcc815e4c0 2024-03-28T13:19:21.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.596+0000 7efcc7400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efcc80b69c0 0x7efcc80b7400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34174/0 (socket says 172.21.15.19:34174) 2024-03-28T13:19:21.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.596+0000 7efcc7400640 1 -- 172.21.15.19:0/3720420156 learned_addr learned my addr 172.21.15.19:0/3720420156 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:21.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.596+0000 7efcc6a00640 1 --2- 172.21.15.19:0/3720420156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcc815e4c0 0x7efcc80b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:21.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.596+0000 7efcc6000640 1 -- 172.21.15.19:0/3720420156 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2973828613 0 0) 0x7efcc80d2710 con 0x7efcc80b69c0 2024-03-28T13:19:21.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.596+0000 7efcc6000640 1 -- 172.21.15.19:0/3720420156 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcac003520 con 0x7efcc80b69c0 2024-03-28T13:19:21.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.597+0000 7efcc6000640 1 -- 172.21.15.19:0/3720420156 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1714461394 0 0) 0x7efcac003520 con 0x7efcc80b69c0 2024-03-28T13:19:21.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.597+0000 7efcc6000640 1 -- 172.21.15.19:0/3720420156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcc815e4c0 msgr2=0x7efcc80b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:21.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.597+0000 7efcc6000640 1 --2- 172.21.15.19:0/3720420156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcc815e4c0 0x7efcc80b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.597+0000 7efcc6000640 1 -- 172.21.15.19:0/3720420156 >> v1:172.21.15.188:6789/0 conn(0x7efcc80d37b0 legacy=0x7efcc80b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:21.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.597+0000 7efcc6000640 1 -- 172.21.15.19:0/3720420156 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc80d2cd0 con 0x7efcc80b69c0 2024-03-28T13:19:21.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.597+0000 7efcc6000640 1 -- 172.21.15.19:0/3720420156 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7efcc00019f0 con 0x7efcc80b69c0 2024-03-28T13:19:21.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.597+0000 7efcc6000640 1 -- 172.21.15.19:0/3720420156 >> v1:172.21.15.19:6789/0 conn(0x7efcc80b69c0 legacy=0x7efcc80b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:21.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.597+0000 7efcc6000640 1 --2- 172.21.15.19:0/3720420156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcac0039e0 0x7efcac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:21.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.597+0000 7efcc6000640 1 --2- 172.21.15.19:0/3720420156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efcac004800 0x7efcac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:21.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.597+0000 7efcc6000640 1 --2- 172.21.15.19:0/3720420156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcc815e4c0 0x7efcac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:21.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.597+0000 7efcc6000640 1 -- 172.21.15.19:0/3720420156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcac00ba70 con 0x7efcac0039e0 2024-03-28T13:19:21.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.597+0000 7efcc6000640 1 -- 172.21.15.19:0/3720420156 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efcac00bc20 con 0x7efcac004800 2024-03-28T13:19:21.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.597+0000 7efcc6000640 1 -- 172.21.15.19:0/3720420156 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efcac00be20 con 0x7efcc815e4c0 2024-03-28T13:19:21.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.597+0000 7efcc6a00640 1 --2- 172.21.15.19:0/3720420156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcac0039e0 0x7efcac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:21.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.598+0000 7efcbfe00640 1 --2- 172.21.15.19:0/3720420156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efcac004800 0x7efcac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:21.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.598+0000 7efcc7400640 1 --2- 172.21.15.19:0/3720420156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcc815e4c0 0x7efcac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:21.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.598+0000 7efcc6a00640 1 -- 172.21.15.19:0/3720420156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcc815e4c0 msgr2=0x7efcac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:21.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.598+0000 7efcc6a00640 1 --2- 172.21.15.19:0/3720420156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcc815e4c0 0x7efcac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.598+0000 7efcc6a00640 1 -- 172.21.15.19:0/3720420156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efcac004800 msgr2=0x7efcac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:21.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.598+0000 7efcc6a00640 1 --2- 172.21.15.19:0/3720420156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efcac004800 0x7efcac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.598+0000 7efcc6a00640 1 -- 172.21.15.19:0/3720420156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcac003520 con 0x7efcac0039e0 2024-03-28T13:19:21.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.598+0000 7efcbfe00640 1 --2- 172.21.15.19:0/3720420156 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efcac004800 0x7efcac004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:21.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.598+0000 7efcc7400640 1 --2- 172.21.15.19:0/3720420156 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcc815e4c0 0x7efcac0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:21.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.599+0000 7efcc6a00640 1 --2- 172.21.15.19:0/3720420156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcac0039e0 0x7efcac003e00 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009a90 tx=0x7efcac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d5f16ba265334d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:21.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.599+0000 7efcc6000640 1 -- 172.21.15.19:0/3720420156 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8039070 con 0x7efcac0039e0 2024-03-28T13:19:21.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.599+0000 7efcc6000640 1 -- 172.21.15.19:0/3720420156 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcb802c660 con 0x7efcac0039e0 2024-03-28T13:19:21.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.599+0000 7efcc6000640 1 -- 172.21.15.19:0/3720420156 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8009690 con 0x7efcac0039e0 2024-03-28T13:19:21.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.600+0000 7efcc7e00640 1 -- 172.21.15.19:0/3720420156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcac0039e0 msgr2=0x7efcac003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:21.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.600+0000 7efcc7e00640 1 --2- 172.21.15.19:0/3720420156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcac0039e0 0x7efcac003e00 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009a90 tx=0x7efcac005830 comp rx=0 tx=0).stop 2024-03-28T13:19:21.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.600+0000 7efcc7400640 1 -- 172.21.15.19:0/3720420156 reap_dead start 2024-03-28T13:19:21.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.600+0000 7efcc7e00640 1 -- 172.21.15.19:0/3720420156 shutdown_connections 2024-03-28T13:19:21.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.600+0000 7efcc7e00640 1 -- 172.21.15.19:0/3720420156 >> 172.21.15.19:0/3720420156 conn(0x7efcc80676a0 msgr2=0x7efcc8067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:21.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.600+0000 7efcc7e00640 1 -- 172.21.15.19:0/3720420156 shutdown_connections 2024-03-28T13:19:21.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.600+0000 7efcc7e00640 1 -- 172.21.15.19:0/3720420156 wait complete. 2024-03-28T13:19:21.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.602+0000 7efcc7e00640 1 Processor -- start 2024-03-28T13:19:21.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.602+0000 7efcc7e00640 1 -- start start 2024-03-28T13:19:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.603+0000 7efcc7e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efcac0039e0 0x7efcc814d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.603+0000 7efcc7e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcc80aacb0 0x7efcc80cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.603+0000 7efcc7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcc80cdf00 0x7efcc814e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.603+0000 7efcc7e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcc80d3320 con 0x7efcc80cdf00 2024-03-28T13:19:21.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.603+0000 7efcc7e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efcc80d4040 con 0x7efcac0039e0 2024-03-28T13:19:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.603+0000 7efcc7e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efcc80d2350 con 0x7efcc80aacb0 2024-03-28T13:19:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.603+0000 7efcc7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcc80cdf00 0x7efcc814e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.603+0000 7efcc7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcc80cdf00 0x7efcc814e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53936/0 (socket says 172.21.15.19:53936) 2024-03-28T13:19:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.603+0000 7efcc7400640 1 -- 172.21.15.19:0/1257546309 learned_addr learned my addr 172.21.15.19:0/1257546309 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.604+0000 7efcc6a00640 1 --2- 172.21.15.19:0/1257546309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efcac0039e0 0x7efcc814d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.604+0000 7efcbfe00640 1 --2- 172.21.15.19:0/1257546309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcc80aacb0 0x7efcc80cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.604+0000 7efcc7400640 1 -- 172.21.15.19:0/1257546309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcc80aacb0 msgr2=0x7efcc80cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.604+0000 7efcc7400640 1 --2- 172.21.15.19:0/1257546309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcc80aacb0 0x7efcc80cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.604+0000 7efcc7400640 1 -- 172.21.15.19:0/1257546309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efcac0039e0 msgr2=0x7efcc814d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.604+0000 7efcc7400640 1 --2- 172.21.15.19:0/1257546309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efcac0039e0 0x7efcc814d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.604+0000 7efcc7400640 1 -- 172.21.15.19:0/1257546309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc80cc920 con 0x7efcc80cdf00 2024-03-28T13:19:21.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.604+0000 7efcc7400640 1 --2- 172.21.15.19:0/1257546309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcc80cdf00 0x7efcc814e270 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7efcc80d52e0 tx=0x7efcc000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.604+0000 7efcbea00640 1 -- 172.21.15.19:0/1257546309 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0004d60 con 0x7efcc80cdf00 2024-03-28T13:19:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.604+0000 7efcbea00640 1 -- 172.21.15.19:0/1257546309 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcc000bb20 con 0x7efcc80cdf00 2024-03-28T13:19:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.605+0000 7efcc7e00640 1 -- 172.21.15.19:0/1257546309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc80ccc00 con 0x7efcc80cdf00 2024-03-28T13:19:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.605+0000 7efcbea00640 1 -- 172.21.15.19:0/1257546309 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0007110 con 0x7efcc80cdf00 2024-03-28T13:19:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.605+0000 7efcc7e00640 1 -- 172.21.15.19:0/1257546309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc80cd1c0 con 0x7efcc80cdf00 2024-03-28T13:19:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.606+0000 7efcc7e00640 1 -- 172.21.15.19:0/1257546309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc8075940 con 0x7efcc80cdf00 2024-03-28T13:19:21.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.607+0000 7efcbea00640 1 -- 172.21.15.19:0/1257546309 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efcc001b020 con 0x7efcc80cdf00 2024-03-28T13:19:21.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.608+0000 7efcbea00640 1 --2- 172.21.15.19:0/1257546309 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efca406c560 0x7efca406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:21.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.608+0000 7efcbea00640 1 -- 172.21.15.19:0/1257546309 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7efcc008b7b0 con 0x7efcc80cdf00 2024-03-28T13:19:21.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.609+0000 7efcc6a00640 1 --2- 172.21.15.19:0/1257546309 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efca406c560 0x7efca406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:21.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.610+0000 7efcc6a00640 1 --2- 172.21.15.19:0/1257546309 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efca406c560 0x7efca406ea20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efcb8036000 tx=0x7efcb8030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:21.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.613+0000 7efcbea00640 1 -- 172.21.15.19:0/1257546309 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efcc0059dc0 con 0x7efcc80cdf00 2024-03-28T13:19:21.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.816+0000 7efcc7e00640 1 -- 172.21.15.19:0/1257546309 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efcc815ae60 con 0x7efcc80cdf00 2024-03-28T13:19:21.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.817+0000 7efcbea00640 1 -- 172.21.15.19:0/1257546309 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7efcc00596c0 con 0x7efcc80cdf00 2024-03-28T13:19:21.817 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:19:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.819+0000 7efcc7e00640 1 -- 172.21.15.19:0/1257546309 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efca406c560 msgr2=0x7efca406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.819+0000 7efcc7e00640 1 --2- 172.21.15.19:0/1257546309 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efca406c560 0x7efca406ea20 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efcb8036000 tx=0x7efcb8030000 comp rx=0 tx=0).stop 2024-03-28T13:19:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.819+0000 7efcc7e00640 1 -- 172.21.15.19:0/1257546309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcc80cdf00 msgr2=0x7efcc814e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.819+0000 7efcc7e00640 1 --2- 172.21.15.19:0/1257546309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcc80cdf00 0x7efcc814e270 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7efcc80d52e0 tx=0x7efcc000b4a0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.819+0000 7efcc7e00640 1 -- 172.21.15.19:0/1257546309 shutdown_connections 2024-03-28T13:19:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.820+0000 7efcc7e00640 1 --2- 172.21.15.19:0/1257546309 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efca406c560 0x7efca406ea20 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.820+0000 7efcc7e00640 1 --2- 172.21.15.19:0/1257546309 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcc80cdf00 0x7efcc814e270 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.820+0000 7efcc7e00640 1 --2- 172.21.15.19:0/1257546309 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcc80aacb0 0x7efcc80cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.820+0000 7efcc7e00640 1 --2- 172.21.15.19:0/1257546309 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efcac0039e0 0x7efcc814d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.820+0000 7efcc7e00640 1 -- 172.21.15.19:0/1257546309 >> 172.21.15.19:0/1257546309 conn(0x7efcc80676a0 msgr2=0x7efcc815d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.820+0000 7efcc7e00640 1 -- 172.21.15.19:0/1257546309 shutdown_connections 2024-03-28T13:19:21.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:21.820+0000 7efcc7e00640 1 -- 172.21.15.19:0/1257546309 wait complete. 2024-03-28T13:19:21.841 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:19:22.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.075+0000 7f34d9c00640 1 Processor -- start 2024-03-28T13:19:22.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.075+0000 7f34d9c00640 1 -- start start 2024-03-28T13:19:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.077+0000 7f34d9c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34d40d0e70 0x7f34d40cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.077+0000 7f34d9c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f34d40d2110 0x7f34d40d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.077+0000 7f34d9c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34d40cf6c0 con 0x7f34d415f6e0 2024-03-28T13:19:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.077+0000 7f34d9c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f34d40cf9a0 con 0x7f34d40d2110 2024-03-28T13:19:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.077+0000 7f34d9c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f34d40cfc80 con 0x7f34d40d0e70 2024-03-28T13:19:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.077+0000 7f34d3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f34d40d2110 0x7f34d40d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.077+0000 7f34d3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f34d40d2110 0x7f34d40d25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53940/0 (socket says 172.21.15.19:53940) 2024-03-28T13:19:22.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.077+0000 7f34d3400640 1 -- 172.21.15.19:0/4272409874 learned_addr learned my addr 172.21.15.19:0/4272409874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:22.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.077+0000 7f34d2a00640 1 --2- 172.21.15.19:0/4272409874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34d40d0e70 0x7f34d40cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:22.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.078+0000 7f34d3400640 1 -- 172.21.15.19:0/4272409874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34d40d0e70 msgr2=0x7f34d40cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:22.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.078+0000 7f34d3400640 1 --2- 172.21.15.19:0/4272409874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34d40d0e70 0x7f34d40cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.078+0000 7f34d3400640 1 -- 172.21.15.19:0/4272409874 >> v1:172.21.15.188:6789/0 conn(0x7f34d415f6e0 legacy=0x7f34d40d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:22.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.078+0000 7f34d3400640 1 -- 172.21.15.19:0/4272409874 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d40cff60 con 0x7f34d40d2110 2024-03-28T13:19:22.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.079+0000 7f34d3400640 1 --2- 172.21.15.19:0/4272409874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d40d2110 0x7f34d40d25a0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009e50 tx=0x7f34cc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e247128147071687 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:22.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.079+0000 7f34d1600640 1 -- 172.21.15.19:0/4272409874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc02ca70 con 0x7f34d40d2110 2024-03-28T13:19:22.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.079+0000 7f34d1600640 1 -- 172.21.15.19:0/4272409874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34cc02cc10 con 0x7f34d40d2110 2024-03-28T13:19:22.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.080+0000 7f34d1600640 1 -- 172.21.15.19:0/4272409874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc0343f0 con 0x7f34d40d2110 2024-03-28T13:19:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.080+0000 7f34d9c00640 1 -- 172.21.15.19:0/4272409874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d40d2110 msgr2=0x7f34d40d25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.080+0000 7f34d9c00640 1 --2- 172.21.15.19:0/4272409874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d40d2110 0x7f34d40d25a0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009e50 tx=0x7f34cc02bed0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.080+0000 7f34d9c00640 1 -- 172.21.15.19:0/4272409874 shutdown_connections 2024-03-28T13:19:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.080+0000 7f34d9c00640 1 --2- 172.21.15.19:0/4272409874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d40d2110 0x7f34d40d25a0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.080+0000 7f34d9c00640 1 --2- 172.21.15.19:0/4272409874 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34d40d0e70 0x7f34d40cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.080+0000 7f34d9c00640 1 -- 172.21.15.19:0/4272409874 >> 172.21.15.19:0/4272409874 conn(0x7f34d408bfb0 msgr2=0x7f34d40d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.080+0000 7f34d9c00640 1 -- 172.21.15.19:0/4272409874 shutdown_connections 2024-03-28T13:19:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.080+0000 7f34d9c00640 1 -- 172.21.15.19:0/4272409874 wait complete. 2024-03-28T13:19:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.082+0000 7f34d9c00640 1 Processor -- start 2024-03-28T13:19:22.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.082+0000 7f34d9c00640 1 -- start start 2024-03-28T13:19:22.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.083+0000 7f34d9c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34d40d0e70 0x7f34d40b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:22.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.083+0000 7f34d9c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34d40d2110 0x7f34d40c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.083+0000 7f34d9c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d415f6e0 0x7f34d414a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.083+0000 7f34d9c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f34d415e910 con 0x7f34d415f6e0 2024-03-28T13:19:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.083+0000 7f34d9c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f34d415fb40 con 0x7f34d40d0e70 2024-03-28T13:19:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.083+0000 7f34d9c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f34d415ea90 con 0x7f34d40d2110 2024-03-28T13:19:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.083+0000 7f34d3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d415f6e0 0x7f34d414a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.084+0000 7f34d3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d415f6e0 0x7f34d414a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53942/0 (socket says 172.21.15.19:53942) 2024-03-28T13:19:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.084+0000 7f34d3400640 1 -- 172.21.15.19:0/747686673 learned_addr learned my addr 172.21.15.19:0/747686673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.084+0000 7f34d2a00640 1 --2- 172.21.15.19:0/747686673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34d40d0e70 0x7f34d40b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.084+0000 7f34d2000640 1 --2- 172.21.15.19:0/747686673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34d40d2110 0x7f34d40c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.084+0000 7f34d3400640 1 -- 172.21.15.19:0/747686673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34d40d2110 msgr2=0x7f34d40c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.084+0000 7f34d3400640 1 --2- 172.21.15.19:0/747686673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34d40d2110 0x7f34d40c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.084+0000 7f34d3400640 1 -- 172.21.15.19:0/747686673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34d40d0e70 msgr2=0x7f34d40b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.084+0000 7f34d3400640 1 --2- 172.21.15.19:0/747686673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34d40d0e70 0x7f34d40b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.084+0000 7f34d3400640 1 -- 172.21.15.19:0/747686673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d415d880 con 0x7f34d415f6e0 2024-03-28T13:19:22.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.084+0000 7f34d3400640 1 --2- 172.21.15.19:0/747686673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d415f6e0 0x7f34d414a730 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009e50 tx=0x7f34cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:22.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.085+0000 7f34caa00640 1 -- 172.21.15.19:0/747686673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc043020 con 0x7f34d415f6e0 2024-03-28T13:19:22.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.085+0000 7f34caa00640 1 -- 172.21.15.19:0/747686673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34cc03e450 con 0x7f34d415f6e0 2024-03-28T13:19:22.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.085+0000 7f34caa00640 1 -- 172.21.15.19:0/747686673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc009510 con 0x7f34d415f6e0 2024-03-28T13:19:22.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.085+0000 7f34d9c00640 1 -- 172.21.15.19:0/747686673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d414ac70 con 0x7f34d415f6e0 2024-03-28T13:19:22.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.085+0000 7f34d9c00640 1 -- 172.21.15.19:0/747686673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d415d5a0 con 0x7f34d415f6e0 2024-03-28T13:19:22.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.086+0000 7f34d9c00640 1 -- 172.21.15.19:0/747686673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d4068cd0 con 0x7f34d415f6e0 2024-03-28T13:19:22.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.087+0000 7f34caa00640 1 -- 172.21.15.19:0/747686673 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f34cc046020 con 0x7f34d415f6e0 2024-03-28T13:19:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.089+0000 7f34caa00640 1 --2- 172.21.15.19:0/747686673 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f34b406c560 0x7f34b406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.089+0000 7f34caa00640 1 -- 172.21.15.19:0/747686673 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f34cc0adf50 con 0x7f34d415f6e0 2024-03-28T13:19:22.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.089+0000 7f34d2a00640 1 --2- 172.21.15.19:0/747686673 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f34b406c560 0x7f34b406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:22.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.090+0000 7f34d2a00640 1 --2- 172.21.15.19:0/747686673 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f34b406c560 0x7f34b406ea20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f34c4006fd0 tx=0x7f34c4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:22.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.094+0000 7f34caa00640 1 -- 172.21.15.19:0/747686673 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f34cc04d030 con 0x7f34d415f6e0 2024-03-28T13:19:22.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.298+0000 7f34d9c00640 1 -- 172.21.15.19:0/747686673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f34d40d1b80 con 0x7f34d415f6e0 2024-03-28T13:19:22.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.299+0000 7f34caa00640 1 -- 172.21.15.19:0/747686673 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v105) ==== 104+0+11 (secure 0 0 0) 0x7f34cc07c4d0 con 0x7f34d415f6e0 2024-03-28T13:19:22.298 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:19:22.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.301+0000 7f34d9c00640 1 -- 172.21.15.19:0/747686673 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f34b406c560 msgr2=0x7f34b406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:22.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.301+0000 7f34d9c00640 1 --2- 172.21.15.19:0/747686673 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f34b406c560 0x7f34b406ea20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f34c4006fd0 tx=0x7f34c4008040 comp rx=0 tx=0).stop 2024-03-28T13:19:22.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.301+0000 7f34d9c00640 1 -- 172.21.15.19:0/747686673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d415f6e0 msgr2=0x7f34d414a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:22.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.301+0000 7f34d9c00640 1 --2- 172.21.15.19:0/747686673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d415f6e0 0x7f34d414a730 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f34cc009e50 tx=0x7f34cc004a40 comp rx=0 tx=0).stop 2024-03-28T13:19:22.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.302+0000 7f34d9c00640 1 -- 172.21.15.19:0/747686673 shutdown_connections 2024-03-28T13:19:22.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.302+0000 7f34d9c00640 1 --2- 172.21.15.19:0/747686673 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f34b406c560 0x7f34b406ea20 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.302+0000 7f34d9c00640 1 --2- 172.21.15.19:0/747686673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f34d415f6e0 0x7f34d414a730 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.302+0000 7f34d9c00640 1 --2- 172.21.15.19:0/747686673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34d40d2110 0x7f34d40c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.302+0000 7f34d9c00640 1 --2- 172.21.15.19:0/747686673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34d40d0e70 0x7f34d40b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.302+0000 7f34d9c00640 1 -- 172.21.15.19:0/747686673 >> 172.21.15.19:0/747686673 conn(0x7f34d408bfb0 msgr2=0x7f34d4160820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:22.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.302+0000 7f34d9c00640 1 -- 172.21.15.19:0/747686673 shutdown_connections 2024-03-28T13:19:22.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.302+0000 7f34d9c00640 1 -- 172.21.15.19:0/747686673 wait complete. 2024-03-28T13:19:22.325 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:19:22.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.547+0000 7fc80d200640 1 Processor -- start 2024-03-28T13:19:22.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.547+0000 7fc80d200640 1 -- start start 2024-03-28T13:19:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.549+0000 7fc80d200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc8080d1560 0x7fc80806c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.549+0000 7fc80d200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80806cbc0 0x7fc8080cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.549+0000 7fc80d200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc80806cfd0 con 0x7fc80815e590 2024-03-28T13:19:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.549+0000 7fc80d200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc8080d19d0 con 0x7fc8080d1560 2024-03-28T13:19:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.549+0000 7fc80d200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc808071060 con 0x7fc80806cbc0 2024-03-28T13:19:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.549+0000 7fc805e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc80815e590 0x7fc8080b47d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34180/0 (socket says 172.21.15.19:34180) 2024-03-28T13:19:22.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.549+0000 7fc805e00640 1 -- 172.21.15.19:0/3424867170 learned_addr learned my addr 172.21.15.19:0/3424867170 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.550+0000 7fc7ffe00640 1 --2- 172.21.15.19:0/3424867170 >> v2:172.21.15.188:3300/0 conn(0x7fc8080d1560 0x7fc80806c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:22.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.550+0000 7fc806800640 1 --2- 172.21.15.19:0/3424867170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80806cbc0 0x7fc8080cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.550+0000 7fc805400640 1 -- 172.21.15.19:0/3424867170 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3878727663 0 0) 0x7fc7f8001110 con 0x7fc80815e590 2024-03-28T13:19:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.550+0000 7fc805400640 1 -- 172.21.15.19:0/3424867170 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7ec003520 con 0x7fc80815e590 2024-03-28T13:19:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.550+0000 7fc805400640 1 -- 172.21.15.19:0/3424867170 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2936619719 0 0) 0x7fc7ec003520 con 0x7fc80815e590 2024-03-28T13:19:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.550+0000 7fc805400640 1 -- 172.21.15.19:0/3424867170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80806cbc0 msgr2=0x7fc8080cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.550+0000 7fc805400640 1 --2- 172.21.15.19:0/3424867170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80806cbc0 0x7fc8080cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.550+0000 7fc805400640 1 -- 172.21.15.19:0/3424867170 >> v2:172.21.15.188:3300/0 conn(0x7fc8080d1560 msgr2=0x7fc80806c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.550+0000 7fc805400640 1 --2- 172.21.15.19:0/3424867170 >> v2:172.21.15.188:3300/0 conn(0x7fc8080d1560 0x7fc80806c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.550+0000 7fc805400640 1 -- 172.21.15.19:0/3424867170 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc808071340 con 0x7fc80815e590 2024-03-28T13:19:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.550+0000 7fc805400640 1 -- 172.21.15.19:0/3424867170 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fc7f8001840 con 0x7fc80815e590 2024-03-28T13:19:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.550+0000 7fc805400640 1 -- 172.21.15.19:0/3424867170 >> v1:172.21.15.19:6789/0 conn(0x7fc80815e590 legacy=0x7fc8080b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc805400640 1 --2- 172.21.15.19:0/3424867170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ec0039b0 0x7fc7ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc805400640 1 --2- 172.21.15.19:0/3424867170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7ec004800 0x7fc7ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc805400640 1 --2- 172.21.15.19:0/3424867170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80806cbc0 0x7fc7ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:22.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc805400640 1 -- 172.21.15.19:0/3424867170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc808071060 con 0x7fc7ec0039b0 2024-03-28T13:19:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc805400640 1 -- 172.21.15.19:0/3424867170 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec00b9c0 con 0x7fc7ec004800 2024-03-28T13:19:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc805400640 1 -- 172.21.15.19:0/3424867170 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ec00bb40 con 0x7fc80806cbc0 2024-03-28T13:19:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc805e00640 1 --2- 172.21.15.19:0/3424867170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ec0039b0 0x7fc7ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc7ffe00640 1 --2- 172.21.15.19:0/3424867170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7ec004800 0x7fc7ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc806800640 1 --2- 172.21.15.19:0/3424867170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80806cbc0 0x7fc7ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc805e00640 1 -- 172.21.15.19:0/3424867170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80806cbc0 msgr2=0x7fc7ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc805e00640 1 --2- 172.21.15.19:0/3424867170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80806cbc0 0x7fc7ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc805e00640 1 -- 172.21.15.19:0/3424867170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7ec004800 msgr2=0x7fc7ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc805e00640 1 --2- 172.21.15.19:0/3424867170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7ec004800 0x7fc7ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc805e00640 1 -- 172.21.15.19:0/3424867170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec003520 con 0x7fc7ec0039b0 2024-03-28T13:19:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc7ffe00640 1 --2- 172.21.15.19:0/3424867170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7ec004800 0x7fc7ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:22.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.551+0000 7fc806800640 1 --2- 172.21.15.19:0/3424867170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80806cbc0 0x7fc7ec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:22.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.552+0000 7fc805e00640 1 --2- 172.21.15.19:0/3424867170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ec0039b0 0x7fc7ec003dd0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800c3e0 tx=0x7fc7ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1487e2497942fd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:22.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.553+0000 7fc805400640 1 -- 172.21.15.19:0/3424867170 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f800c0e0 con 0x7fc7ec0039b0 2024-03-28T13:19:22.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.553+0000 7fc805400640 1 -- 172.21.15.19:0/3424867170 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7f8030070 con 0x7fc7ec0039b0 2024-03-28T13:19:22.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.553+0000 7fc805400640 1 -- 172.21.15.19:0/3424867170 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80058d0 con 0x7fc7ec0039b0 2024-03-28T13:19:22.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.553+0000 7fc80d200640 1 -- 172.21.15.19:0/3424867170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ec0039b0 msgr2=0x7fc7ec003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:22.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.553+0000 7fc80d200640 1 --2- 172.21.15.19:0/3424867170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc7ec0039b0 0x7fc7ec003dd0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800c3e0 tx=0x7fc7ec005830 comp rx=0 tx=0).stop 2024-03-28T13:19:22.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.553+0000 7fc806800640 1 -- 172.21.15.19:0/3424867170 reap_dead start 2024-03-28T13:19:22.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.553+0000 7fc80d200640 1 -- 172.21.15.19:0/3424867170 shutdown_connections 2024-03-28T13:19:22.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.553+0000 7fc80d200640 1 -- 172.21.15.19:0/3424867170 >> 172.21.15.19:0/3424867170 conn(0x7fc8080d2240 msgr2=0x7fc8080d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:22.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.554+0000 7fc80d200640 1 -- 172.21.15.19:0/3424867170 shutdown_connections 2024-03-28T13:19:22.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.554+0000 7fc80d200640 1 -- 172.21.15.19:0/3424867170 wait complete. 2024-03-28T13:19:22.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.555+0000 7fc80d200640 1 Processor -- start 2024-03-28T13:19:22.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.556+0000 7fc80d200640 1 -- start start 2024-03-28T13:19:22.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.557+0000 7fc80d200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7ec0039b0 0x7fc8080b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:22.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.557+0000 7fc80d200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80815a9e0 0x7fc8081598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:22.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.557+0000 7fc80d200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc80815bff0 0x7fc80814c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:22.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.557+0000 7fc80d200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc808071be0 con 0x7fc80815bff0 2024-03-28T13:19:22.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.557+0000 7fc80d200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc808071900 con 0x7fc7ec0039b0 2024-03-28T13:19:22.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.557+0000 7fc80d200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc80815e970 con 0x7fc80815a9e0 2024-03-28T13:19:22.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.557+0000 7fc806800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc80815bff0 0x7fc80814c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:22.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.557+0000 7fc806800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc80815bff0 0x7fc80814c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53968/0 (socket says 172.21.15.19:53968) 2024-03-28T13:19:22.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.557+0000 7fc806800640 1 -- 172.21.15.19:0/3093708654 learned_addr learned my addr 172.21.15.19:0/3093708654 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:22.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.557+0000 7fc805e00640 1 --2- 172.21.15.19:0/3093708654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7ec0039b0 0x7fc8080b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:22.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.557+0000 7fc7ffe00640 1 --2- 172.21.15.19:0/3093708654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80815a9e0 0x7fc8081598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:22.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.557+0000 7fc806800640 1 -- 172.21.15.19:0/3093708654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80815a9e0 msgr2=0x7fc8081598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:22.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.557+0000 7fc806800640 1 --2- 172.21.15.19:0/3093708654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80815a9e0 0x7fc8081598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.557+0000 7fc806800640 1 -- 172.21.15.19:0/3093708654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7ec0039b0 msgr2=0x7fc8080b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:22.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.558+0000 7fc806800640 1 --2- 172.21.15.19:0/3093708654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7ec0039b0 0x7fc8080b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.558+0000 7fc806800640 1 -- 172.21.15.19:0/3093708654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80815af50 con 0x7fc80815bff0 2024-03-28T13:19:22.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.558+0000 7fc805e00640 1 --2- 172.21.15.19:0/3093708654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7ec0039b0 0x7fc8080b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:19:22.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.558+0000 7fc806800640 1 --2- 172.21.15.19:0/3093708654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc80815bff0 0x7fc80814c420 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fc808070c90 tx=0x7fc80000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:22.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.558+0000 7fc7fe000640 1 -- 172.21.15.19:0/3093708654 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80001c070 con 0x7fc80815bff0 2024-03-28T13:19:22.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.558+0000 7fc7fe000640 1 -- 172.21.15.19:0/3093708654 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc800017400 con 0x7fc80815bff0 2024-03-28T13:19:22.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.558+0000 7fc7fe000640 1 -- 172.21.15.19:0/3093708654 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8000177d0 con 0x7fc80815bff0 2024-03-28T13:19:22.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.558+0000 7fc80d200640 1 -- 172.21.15.19:0/3093708654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80815b230 con 0x7fc80815bff0 2024-03-28T13:19:22.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.558+0000 7fc80d200640 1 -- 172.21.15.19:0/3093708654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80815b7f0 con 0x7fc80815bff0 2024-03-28T13:19:22.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.560+0000 7fc80d200640 1 -- 172.21.15.19:0/3093708654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8080685c0 con 0x7fc80815bff0 2024-03-28T13:19:22.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.560+0000 7fc7fe000640 1 -- 172.21.15.19:0/3093708654 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc80001f070 con 0x7fc80815bff0 2024-03-28T13:19:22.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.561+0000 7fc7fe000640 1 --2- 172.21.15.19:0/3093708654 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc7e406c530 0x7fc7e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:22.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.561+0000 7fc7fe000640 1 -- 172.21.15.19:0/3093708654 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7fc80008b930 con 0x7fc80815bff0 2024-03-28T13:19:22.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.562+0000 7fc805e00640 1 --2- 172.21.15.19:0/3093708654 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc7e406c530 0x7fc7e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:22.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.563+0000 7fc805e00640 1 --2- 172.21.15.19:0/3093708654 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc7e406c530 0x7fc7e406e9f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8034000 tx=0x7fc7f8007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:22.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.566+0000 7fc7fe000640 1 -- 172.21.15.19:0/3093708654 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc800012070 con 0x7fc80815bff0 2024-03-28T13:19:22.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.775+0000 7fc80d200640 1 -- 172.21.15.19:0/3093708654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc8080ce4f0 con 0x7fc80815bff0 2024-03-28T13:19:22.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.775+0000 7fc7fe000640 1 -- 172.21.15.19:0/3093708654 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v105) ==== 100+0+11 (secure 0 0 0) 0x7fc800059eb0 con 0x7fc80815bff0 2024-03-28T13:19:22.775 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:19:22.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.779+0000 7fc80d200640 1 -- 172.21.15.19:0/3093708654 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc7e406c530 msgr2=0x7fc7e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:22.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.779+0000 7fc80d200640 1 --2- 172.21.15.19:0/3093708654 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc7e406c530 0x7fc7e406e9f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8034000 tx=0x7fc7f8007000 comp rx=0 tx=0).stop 2024-03-28T13:19:22.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.779+0000 7fc80d200640 1 -- 172.21.15.19:0/3093708654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc80815bff0 msgr2=0x7fc80814c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:22.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.779+0000 7fc80d200640 1 --2- 172.21.15.19:0/3093708654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc80815bff0 0x7fc80814c420 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fc808070c90 tx=0x7fc80000f4a0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.779+0000 7fc80d200640 1 -- 172.21.15.19:0/3093708654 shutdown_connections 2024-03-28T13:19:22.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.779+0000 7fc80d200640 1 --2- 172.21.15.19:0/3093708654 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc7e406c530 0x7fc7e406e9f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.779+0000 7fc80d200640 1 --2- 172.21.15.19:0/3093708654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc80815bff0 0x7fc80814c420 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.779+0000 7fc80d200640 1 --2- 172.21.15.19:0/3093708654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc80815a9e0 0x7fc8081598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.779+0000 7fc80d200640 1 --2- 172.21.15.19:0/3093708654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7ec0039b0 0x7fc8080b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:22.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.779+0000 7fc80d200640 1 -- 172.21.15.19:0/3093708654 >> 172.21.15.19:0/3093708654 conn(0x7fc8080d2240 msgr2=0x7fc8080cd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:22.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.780+0000 7fc80d200640 1 -- 172.21.15.19:0/3093708654 shutdown_connections 2024-03-28T13:19:22.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:22.780+0000 7fc80d200640 1 -- 172.21.15.19:0/3093708654 wait complete. 2024-03-28T13:19:22.800 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:19:23.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.005+0000 7fa329e00640 1 Processor -- start 2024-03-28T13:19:23.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.005+0000 7fa329e00640 1 -- start start 2024-03-28T13:19:23.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.007+0000 7fa329e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa32415e860 0x7fa32406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:23.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.007+0000 7fa329e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa32406dfa0 con 0x7fa32406da30 2024-03-28T13:19:23.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.007+0000 7fa329e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa32406e240 con 0x7fa3240d4ae0 2024-03-28T13:19:23.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.007+0000 7fa329e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa3240cf220 con 0x7fa32415e860 2024-03-28T13:19:23.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.007+0000 7fa31be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa32406da30 0x7fa3240d43d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34194/0 (socket says 172.21.15.19:34194) 2024-03-28T13:19:23.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.007+0000 7fa31be00640 1 -- 172.21.15.19:0/862568449 learned_addr learned my addr 172.21.15.19:0/862568449 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:23.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.008+0000 7fa322a00640 1 --2- 172.21.15.19:0/862568449 >> v2:172.21.15.188:3300/0 conn(0x7fa32415e860 0x7fa32406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:23.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.008+0000 7fa322000640 1 -- 172.21.15.19:0/862568449 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3023860957 0 0) 0x7fa32406dfa0 con 0x7fa32406da30 2024-03-28T13:19:23.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.008+0000 7fa322000640 1 -- 172.21.15.19:0/862568449 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa308003520 con 0x7fa32406da30 2024-03-28T13:19:23.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.008+0000 7fa322000640 1 -- 172.21.15.19:0/862568449 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1473065202 0 0) 0x7fa308003520 con 0x7fa32406da30 2024-03-28T13:19:23.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.008+0000 7fa322000640 1 -- 172.21.15.19:0/862568449 >> v2:172.21.15.188:3300/0 conn(0x7fa32415e860 msgr2=0x7fa32406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:23.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.008+0000 7fa322000640 1 --2- 172.21.15.19:0/862568449 >> v2:172.21.15.188:3300/0 conn(0x7fa32415e860 0x7fa32406d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.008+0000 7fa322000640 1 -- 172.21.15.19:0/862568449 >> v1:172.21.15.188:6789/0 conn(0x7fa3240d4ae0 legacy=0x7fa3240cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:23.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.008+0000 7fa322000640 1 -- 172.21.15.19:0/862568449 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32406e520 con 0x7fa32406da30 2024-03-28T13:19:23.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.009+0000 7fa322000640 1 -- 172.21.15.19:0/862568449 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fa30c001670 con 0x7fa32406da30 2024-03-28T13:19:23.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.009+0000 7fa322000640 1 -- 172.21.15.19:0/862568449 >> v1:172.21.15.19:6789/0 conn(0x7fa32406da30 legacy=0x7fa3240d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:23.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.009+0000 7fa322000640 1 --2- 172.21.15.19:0/862568449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa308003a00 0x7fa308003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:23.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.009+0000 7fa322000640 1 --2- 172.21.15.19:0/862568449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa308004800 0x7fa308004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:23.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.009+0000 7fa322000640 1 --2- 172.21.15.19:0/862568449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3080051c0 0x7fa30800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:23.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.009+0000 7fa322000640 1 -- 172.21.15.19:0/862568449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3240cf220 con 0x7fa3080051c0 2024-03-28T13:19:23.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.009+0000 7fa322000640 1 -- 172.21.15.19:0/862568449 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa30800c0c0 con 0x7fa308003a00 2024-03-28T13:19:23.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.009+0000 7fa322000640 1 -- 172.21.15.19:0/862568449 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa30800c2c0 con 0x7fa308004800 2024-03-28T13:19:23.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.009+0000 7fa323400640 1 --2- 172.21.15.19:0/862568449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3080051c0 0x7fa30800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:23.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.009+0000 7fa322a00640 1 --2- 172.21.15.19:0/862568449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa308003a00 0x7fa308003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:23.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.009+0000 7fa31be00640 1 --2- 172.21.15.19:0/862568449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa308004800 0x7fa308004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:23.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.009+0000 7fa323400640 1 -- 172.21.15.19:0/862568449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa308004800 msgr2=0x7fa308004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:23.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.010+0000 7fa323400640 1 --2- 172.21.15.19:0/862568449 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa308004800 0x7fa308004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.010+0000 7fa323400640 1 -- 172.21.15.19:0/862568449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa308003a00 msgr2=0x7fa308003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:23.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.010+0000 7fa323400640 1 --2- 172.21.15.19:0/862568449 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa308003a00 0x7fa308003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.010+0000 7fa323400640 1 -- 172.21.15.19:0/862568449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa308003520 con 0x7fa3080051c0 2024-03-28T13:19:23.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.010+0000 7fa323400640 1 -- 172.21.15.19:0/862568449 reap_dead start 2024-03-28T13:19:23.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.011+0000 7fa323400640 1 --2- 172.21.15.19:0/862568449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3080051c0 0x7fa30800b9c0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fa31c001e50 tx=0x7fa308005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca6253b8d15fdcd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:23.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.011+0000 7fa322000640 1 -- 172.21.15.19:0/862568449 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c001a70 con 0x7fa3080051c0 2024-03-28T13:19:23.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.011+0000 7fa322000640 1 -- 172.21.15.19:0/862568449 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa31c003bb0 con 0x7fa3080051c0 2024-03-28T13:19:23.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.011+0000 7fa322000640 1 -- 172.21.15.19:0/862568449 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c0050d0 con 0x7fa3080051c0 2024-03-28T13:19:23.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.011+0000 7fa329e00640 1 -- 172.21.15.19:0/862568449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3080051c0 msgr2=0x7fa30800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:23.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.011+0000 7fa329e00640 1 --2- 172.21.15.19:0/862568449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3080051c0 0x7fa30800b9c0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fa31c001e50 tx=0x7fa308005830 comp rx=0 tx=0).stop 2024-03-28T13:19:23.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.012+0000 7fa329e00640 1 -- 172.21.15.19:0/862568449 shutdown_connections 2024-03-28T13:19:23.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.012+0000 7fa329e00640 1 --2- 172.21.15.19:0/862568449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3080051c0 0x7fa30800b9c0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.012+0000 7fa329e00640 1 -- 172.21.15.19:0/862568449 >> 172.21.15.19:0/862568449 conn(0x7fa32409d7d0 msgr2=0x7fa32408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:23.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.012+0000 7fa329e00640 1 -- 172.21.15.19:0/862568449 shutdown_connections 2024-03-28T13:19:23.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.012+0000 7fa329e00640 1 -- 172.21.15.19:0/862568449 wait complete. 2024-03-28T13:19:23.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.013+0000 7fa329e00640 1 Processor -- start 2024-03-28T13:19:23.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.014+0000 7fa329e00640 1 -- start start 2024-03-28T13:19:23.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.014+0000 7fa329e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3080051c0 0x7fa3241f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:23.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.014+0000 7fa329e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3240b8e30 0x7fa3240b92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:23.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.014+0000 7fa329e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3240b6fc0 0x7fa3240b7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:23.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.014+0000 7fa329e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa3240cef30 con 0x7fa3240b6fc0 2024-03-28T13:19:23.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.014+0000 7fa329e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa3240cedb0 con 0x7fa3240b8e30 2024-03-28T13:19:23.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.014+0000 7fa329e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa32406d4f0 con 0x7fa3080051c0 2024-03-28T13:19:23.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.015+0000 7fa323400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3240b6fc0 0x7fa3240b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:23.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.015+0000 7fa323400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3240b6fc0 0x7fa3240b7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53976/0 (socket says 172.21.15.19:53976) 2024-03-28T13:19:23.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.015+0000 7fa323400640 1 -- 172.21.15.19:0/3195101888 learned_addr learned my addr 172.21.15.19:0/3195101888 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:23.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.015+0000 7fa322a00640 1 --2- 172.21.15.19:0/3195101888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3080051c0 0x7fa3241f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:23.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.015+0000 7fa31be00640 1 --2- 172.21.15.19:0/3195101888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3240b8e30 0x7fa3240b92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:23.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.015+0000 7fa323400640 1 -- 172.21.15.19:0/3195101888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3080051c0 msgr2=0x7fa3241f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:23.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.015+0000 7fa323400640 1 --2- 172.21.15.19:0/3195101888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3080051c0 0x7fa3241f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.015+0000 7fa323400640 1 -- 172.21.15.19:0/3195101888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3240b8e30 msgr2=0x7fa3240b92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:23.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.015+0000 7fa323400640 1 --2- 172.21.15.19:0/3195101888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3240b8e30 0x7fa3240b92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.015+0000 7fa323400640 1 -- 172.21.15.19:0/3195101888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3240b99f0 con 0x7fa3240b6fc0 2024-03-28T13:19:23.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.015+0000 7fa31be00640 1 --2- 172.21.15.19:0/3195101888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3240b8e30 0x7fa3240b92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:23.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.016+0000 7fa323400640 1 --2- 172.21.15.19:0/3195101888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3240b6fc0 0x7fa3240b7440 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fa31c001f70 tx=0x7fa31c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:23.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.016+0000 7fa31b400640 1 -- 172.21.15.19:0/3195101888 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c039480 con 0x7fa3240b6fc0 2024-03-28T13:19:23.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.016+0000 7fa31b400640 1 -- 172.21.15.19:0/3195101888 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa31c039620 con 0x7fa3240b6fc0 2024-03-28T13:19:23.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.016+0000 7fa329e00640 1 -- 172.21.15.19:0/3195101888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3240b9cd0 con 0x7fa3240b6fc0 2024-03-28T13:19:23.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.016+0000 7fa31b400640 1 -- 172.21.15.19:0/3195101888 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c0399f0 con 0x7fa3240b6fc0 2024-03-28T13:19:23.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.016+0000 7fa329e00640 1 -- 172.21.15.19:0/3195101888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3240c9060 con 0x7fa3240b6fc0 2024-03-28T13:19:23.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.018+0000 7fa329e00640 1 -- 172.21.15.19:0/3195101888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3240c9260 con 0x7fa3240b6fc0 2024-03-28T13:19:23.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.020+0000 7fa31b400640 1 -- 172.21.15.19:0/3195101888 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa31c039b90 con 0x7fa3240b6fc0 2024-03-28T13:19:23.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.020+0000 7fa31b400640 1 --2- 172.21.15.19:0/3195101888 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa30006c560 0x7fa30006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:23.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.021+0000 7fa31b400640 1 -- 172.21.15.19:0/3195101888 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7fa31c0ad830 con 0x7fa3240b6fc0 2024-03-28T13:19:23.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.021+0000 7fa322a00640 1 --2- 172.21.15.19:0/3195101888 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa30006c560 0x7fa30006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:23.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.022+0000 7fa322a00640 1 --2- 172.21.15.19:0/3195101888 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa30006c560 0x7fa30006ea20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa314009990 tx=0x7fa314008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:23.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.025+0000 7fa31b400640 1 -- 172.21.15.19:0/3195101888 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa3240c9260 con 0x7fa3240b6fc0 2024-03-28T13:19:23.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.249+0000 7fa329e00640 1 -- 172.21.15.19:0/3195101888 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa3240685c0 con 0x7fa3240b6fc0 2024-03-28T13:19:23.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.249+0000 7fa31b400640 1 -- 172.21.15.19:0/3195101888 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v105) ==== 80+0+0 (secure 0 0 0) 0x7fa31c03e070 con 0x7fa3240b6fc0 2024-03-28T13:19:23.250 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-03-28T13:19:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.252+0000 7fa329e00640 1 -- 172.21.15.19:0/3195101888 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa30006c560 msgr2=0x7fa30006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.252+0000 7fa329e00640 1 --2- 172.21.15.19:0/3195101888 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa30006c560 0x7fa30006ea20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fa314009990 tx=0x7fa314008040 comp rx=0 tx=0).stop 2024-03-28T13:19:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.252+0000 7fa329e00640 1 -- 172.21.15.19:0/3195101888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3240b6fc0 msgr2=0x7fa3240b7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.252+0000 7fa329e00640 1 --2- 172.21.15.19:0/3195101888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3240b6fc0 0x7fa3240b7440 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fa31c001f70 tx=0x7fa31c0014a0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.252+0000 7fa329e00640 1 -- 172.21.15.19:0/3195101888 shutdown_connections 2024-03-28T13:19:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.252+0000 7fa329e00640 1 --2- 172.21.15.19:0/3195101888 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa30006c560 0x7fa30006ea20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.252+0000 7fa329e00640 1 --2- 172.21.15.19:0/3195101888 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa3240b6fc0 0x7fa3240b7440 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.252+0000 7fa329e00640 1 --2- 172.21.15.19:0/3195101888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3240b8e30 0x7fa3240b92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.252+0000 7fa329e00640 1 --2- 172.21.15.19:0/3195101888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3080051c0 0x7fa3241f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.252+0000 7fa329e00640 1 -- 172.21.15.19:0/3195101888 >> 172.21.15.19:0/3195101888 conn(0x7fa32409d7d0 msgr2=0x7fa3240b4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:23.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.252+0000 7fa329e00640 1 -- 172.21.15.19:0/3195101888 shutdown_connections 2024-03-28T13:19:23.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.252+0000 7fa329e00640 1 -- 172.21.15.19:0/3195101888 wait complete. 2024-03-28T13:19:23.272 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:19:23.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:23.272 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T13:19:23.419 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:19:23.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:23.419 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:19:23.475 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:19:23.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.655+0000 7efbfbe00640 1 Processor -- start 2024-03-28T13:19:23.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.655+0000 7efbfbe00640 1 -- start start 2024-03-28T13:19:23.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.656+0000 7efbfbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efbfc0cf1b0 0x7efbfc0cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:23.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.656+0000 7efbfbe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efbfc071460 con 0x7efbfc1442c0 2024-03-28T13:19:23.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.656+0000 7efbfbe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efbfc0721b0 con 0x7efbfc0d00d0 2024-03-28T13:19:23.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.656+0000 7efbfbe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7efbfc071d20 con 0x7efbfc0cf1b0 2024-03-28T13:19:23.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.657+0000 7efbf2000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efbfc0cf1b0 0x7efbfc0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:23.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.657+0000 7efbf2000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efbfc0cf1b0 0x7efbfc0cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53988/0 (socket says 172.21.15.19:53988) 2024-03-28T13:19:23.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.657+0000 7efbf2000640 1 -- 172.21.15.19:0/1110798941 learned_addr learned my addr 172.21.15.19:0/1110798941 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:23.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.657+0000 7efbf2000640 1 -- 172.21.15.19:0/1110798941 >> v1:172.21.15.188:6789/0 conn(0x7efbfc0d00d0 legacy=0x7efbfc0b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:23.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.657+0000 7efbf2000640 1 -- 172.21.15.19:0/1110798941 >> v1:172.21.15.19:6789/0 conn(0x7efbfc1442c0 legacy=0x7efbfc0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:23.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.657+0000 7efbf2000640 1 -- 172.21.15.19:0/1110798941 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbfc0716d0 con 0x7efbfc0cf1b0 2024-03-28T13:19:23.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.658+0000 7efbf2000640 1 --2- 172.21.15.19:0/1110798941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc0cf1b0 0x7efbfc0cfb90 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7efbe4009870 tx=0x7efbe402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e1748b07d080e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:23.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.659+0000 7efbfa000640 1 -- 172.21.15.19:0/1110798941 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe402f850 con 0x7efbfc0cf1b0 2024-03-28T13:19:23.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.659+0000 7efbfa000640 1 -- 172.21.15.19:0/1110798941 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbe402f9f0 con 0x7efbfc0cf1b0 2024-03-28T13:19:23.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.659+0000 7efbfa000640 1 -- 172.21.15.19:0/1110798941 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe402fdc0 con 0x7efbfc0cf1b0 2024-03-28T13:19:23.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.659+0000 7efbfbe00640 1 -- 172.21.15.19:0/1110798941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc0cf1b0 msgr2=0x7efbfc0cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:23.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.659+0000 7efbfbe00640 1 --2- 172.21.15.19:0/1110798941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc0cf1b0 0x7efbfc0cfb90 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7efbe4009870 tx=0x7efbe402ba30 comp rx=0 tx=0).stop 2024-03-28T13:19:23.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.659+0000 7efbfbe00640 1 -- 172.21.15.19:0/1110798941 shutdown_connections 2024-03-28T13:19:23.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.659+0000 7efbfbe00640 1 --2- 172.21.15.19:0/1110798941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc0cf1b0 0x7efbfc0cfb90 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.659+0000 7efbfbe00640 1 -- 172.21.15.19:0/1110798941 >> 172.21.15.19:0/1110798941 conn(0x7efbfc0ccad0 msgr2=0x7efbfc0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:23.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.660+0000 7efbfbe00640 1 -- 172.21.15.19:0/1110798941 shutdown_connections 2024-03-28T13:19:23.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.660+0000 7efbfbe00640 1 -- 172.21.15.19:0/1110798941 wait complete. 2024-03-28T13:19:23.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.661+0000 7efbfbe00640 1 Processor -- start 2024-03-28T13:19:23.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.661+0000 7efbfbe00640 1 -- start start 2024-03-28T13:19:23.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.662+0000 7efbfbe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efbfc0cf1b0 0x7efbfc15e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:23.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.662+0000 7efbfbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc0d00d0 0x7efbfc1401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:23.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.662+0000 7efbfbe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efbfc1442c0 0x7efbfc1e58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:23.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.663+0000 7efbfbe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1446a0 con 0x7efbfc0d00d0 2024-03-28T13:19:23.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.663+0000 7efbfbe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc0d0770 con 0x7efbfc0cf1b0 2024-03-28T13:19:23.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.663+0000 7efbfbe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efbfc15c1d0 con 0x7efbfc1442c0 2024-03-28T13:19:23.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.663+0000 7efbf2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc0d00d0 0x7efbfc1401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:23.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.663+0000 7efbf2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc0d00d0 0x7efbfc1401e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54000/0 (socket says 172.21.15.19:54000) 2024-03-28T13:19:23.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.663+0000 7efbf2000640 1 -- 172.21.15.19:0/423045822 learned_addr learned my addr 172.21.15.19:0/423045822 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:23.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.663+0000 7efbf2000640 1 -- 172.21.15.19:0/423045822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efbfc1442c0 msgr2=0x7efbfc1e58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:23.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.663+0000 7efbfb400640 1 --2- 172.21.15.19:0/423045822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efbfc1442c0 0x7efbfc1e58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:23.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.663+0000 7efbfaa00640 1 --2- 172.21.15.19:0/423045822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efbfc0cf1b0 0x7efbfc15e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:23.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.663+0000 7efbf2000640 1 --2- 172.21.15.19:0/423045822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efbfc1442c0 0x7efbfc1e58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.663+0000 7efbf2000640 1 -- 172.21.15.19:0/423045822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efbfc0cf1b0 msgr2=0x7efbfc15e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:23.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.663+0000 7efbf2000640 1 --2- 172.21.15.19:0/423045822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efbfc0cf1b0 0x7efbfc15e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.663+0000 7efbf2000640 1 -- 172.21.15.19:0/423045822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbfc15c4b0 con 0x7efbfc0d00d0 2024-03-28T13:19:23.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.664+0000 7efbf2000640 1 --2- 172.21.15.19:0/423045822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc0d00d0 0x7efbfc1401e0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7efbe4009870 tx=0x7efbe4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:23.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.664+0000 7efbf3400640 1 -- 172.21.15.19:0/423045822 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe402f820 con 0x7efbfc0d00d0 2024-03-28T13:19:23.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.664+0000 7efbf3400640 1 -- 172.21.15.19:0/423045822 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbe4046070 con 0x7efbfc0d00d0 2024-03-28T13:19:23.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.664+0000 7efbfbe00640 1 -- 172.21.15.19:0/423045822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbfc15c790 con 0x7efbfc0d00d0 2024-03-28T13:19:23.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.664+0000 7efbf3400640 1 -- 172.21.15.19:0/423045822 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe4041640 con 0x7efbfc0d00d0 2024-03-28T13:19:23.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.664+0000 7efbfbe00640 1 -- 172.21.15.19:0/423045822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbfc15cd50 con 0x7efbfc0d00d0 2024-03-28T13:19:23.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.665+0000 7efbfbe00640 1 -- 172.21.15.19:0/423045822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbfc0cda60 con 0x7efbfc0d00d0 2024-03-28T13:19:23.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.667+0000 7efbf3400640 1 -- 172.21.15.19:0/423045822 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efbe4040070 con 0x7efbfc0d00d0 2024-03-28T13:19:23.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.667+0000 7efbf3400640 1 --2- 172.21.15.19:0/423045822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efbd806c530 0x7efbd806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:23.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.668+0000 7efbf3400640 1 -- 172.21.15.19:0/423045822 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7efbe40adfe0 con 0x7efbfc0d00d0 2024-03-28T13:19:23.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.668+0000 7efbfaa00640 1 --2- 172.21.15.19:0/423045822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efbd806c530 0x7efbd806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:23.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.669+0000 7efbfaa00640 1 --2- 172.21.15.19:0/423045822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efbd806c530 0x7efbd806e9f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7efbfc15df40 tx=0x7efbec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:23.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.672+0000 7efbf3400640 1 -- 172.21.15.19:0/423045822 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efbe4033030 con 0x7efbfc0d00d0 2024-03-28T13:19:23.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.916+0000 7efbfbe00640 1 -- 172.21.15.19:0/423045822 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efbfc15e760 con 0x7efbfc0d00d0 2024-03-28T13:19:23.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.917+0000 7efbf3400640 1 -- 172.21.15.19:0/423045822 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+5189 (secure 0 0 0) 0x7efbe402f9c0 con 0x7efbfc0d00d0 2024-03-28T13:19:23.917 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:23.917 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4247326257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4247326257},{"type":"v1","addr":"172.21.15.188:6836","nonce":4247326257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3452274156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3452274156},{"type":"v1","addr":"172.21.15.19:6835","nonce":3452274156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3383190607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3383190607},{"type":"v1","addr":"172.21.15.188:6837","nonce":3383190607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:18:58.541599+0000","modified":"2024-03-28T13:19:01.614319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8040},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8040":{"gid":8040,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.19:6834/1020599987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":1020599987},{"type":"v1","addr":"172.21.15.19:6834","nonce":1020599987}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8040,"qdb_cluster":[8040]},"id":7}]} 2024-03-28T13:19:23.917 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 83 2024-03-28T13:19:23.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.920+0000 7efbfbe00640 1 -- 172.21.15.19:0/423045822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efbd806c530 msgr2=0x7efbd806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:23.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.920+0000 7efbfbe00640 1 --2- 172.21.15.19:0/423045822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efbd806c530 0x7efbd806e9f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7efbfc15df40 tx=0x7efbec009040 comp rx=0 tx=0).stop 2024-03-28T13:19:23.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.920+0000 7efbfbe00640 1 -- 172.21.15.19:0/423045822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc0d00d0 msgr2=0x7efbfc1401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:23.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.920+0000 7efbfbe00640 1 --2- 172.21.15.19:0/423045822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc0d00d0 0x7efbfc1401e0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7efbe4009870 tx=0x7efbe4004730 comp rx=0 tx=0).stop 2024-03-28T13:19:23.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.920+0000 7efbfbe00640 1 -- 172.21.15.19:0/423045822 shutdown_connections 2024-03-28T13:19:23.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.920+0000 7efbfbe00640 1 --2- 172.21.15.19:0/423045822 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efbd806c530 0x7efbd806e9f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.920+0000 7efbfbe00640 1 --2- 172.21.15.19:0/423045822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efbfc1442c0 0x7efbfc1e58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.920+0000 7efbfbe00640 1 --2- 172.21.15.19:0/423045822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efbfc0d00d0 0x7efbfc1401e0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.920+0000 7efbfbe00640 1 --2- 172.21.15.19:0/423045822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efbfc0cf1b0 0x7efbfc15e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:23.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.920+0000 7efbfbe00640 1 -- 172.21.15.19:0/423045822 >> 172.21.15.19:0/423045822 conn(0x7efbfc0ccad0 msgr2=0x7efbfc06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:23.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.920+0000 7efbfbe00640 1 -- 172.21.15.19:0/423045822 shutdown_connections 2024-03-28T13:19:23.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:23.920+0000 7efbfbe00640 1 -- 172.21.15.19:0/423045822 wait complete. 2024-03-28T13:19:23.942 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-28T13:19:24.192 INFO:teuthology.orchestra.run.smithi019.stdout:{"message":"","return_code":0} 2024-03-28T13:19:24.193 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-28T13:19:24.193 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:19:24.193 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:19:24.246 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:19:24.246 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:19:24.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:24.247 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:19:24.297 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-28T13:19:24.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T13:19:24.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo daemon-helper kill ceph --cluster ceph -w 2024-03-28T13:19:24.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:24.304 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-03-28T13:19:24.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.511+0000 7f7a74c00640 1 Processor -- start 2024-03-28T13:19:24.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.511+0000 7f7a74c00640 1 -- start start 2024-03-28T13:19:24.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.513+0000 7f7a74c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7a70161f10 0x7f7a700d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.513+0000 7f7a74c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7a700b3270 0x7f7a700d4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.513+0000 7f7a74c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a7006d370 con 0x7f7a7006da00 2024-03-28T13:19:24.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.513+0000 7f7a74c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7a7006d650 con 0x7f7a700b3270 2024-03-28T13:19:24.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.513+0000 7f7a74c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7a700d0f00 con 0x7f7a70161f10 2024-03-28T13:19:24.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.513+0000 7f7a6d400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7a700b3270 0x7f7a700d4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.513+0000 7f7a6d400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7a700b3270 0x7f7a700d4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54004/0 (socket says 172.21.15.19:54004) 2024-03-28T13:19:24.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.513+0000 7f7a6d400640 1 -- 172.21.15.19:0/2832108326 learned_addr learned my addr 172.21.15.19:0/2832108326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:24.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.514+0000 7f7a6d400640 1 -- 172.21.15.19:0/2832108326 >> v2:172.21.15.188:3300/0 conn(0x7f7a70161f10 msgr2=0x7f7a700d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.514+0000 7f7a6d400640 1 --2- 172.21.15.19:0/2832108326 >> v2:172.21.15.188:3300/0 conn(0x7f7a70161f10 0x7f7a700d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.514+0000 7f7a6d400640 1 -- 172.21.15.19:0/2832108326 >> v1:172.21.15.188:6789/0 conn(0x7f7a7006da00 legacy=0x7f7a700d2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.514+0000 7f7a6d400640 1 -- 172.21.15.19:0/2832108326 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a700d11e0 con 0x7f7a700b3270 2024-03-28T13:19:24.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.516+0000 7f7a6d400640 1 --2- 172.21.15.19:0/2832108326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a700b3270 0x7f7a700d4b10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f7a700b6fc0 tx=0x7f7a5c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ba12246114b092 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.516+0000 7f7a6ca00640 1 -- 172.21.15.19:0/2832108326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c035020 con 0x7f7a700b3270 2024-03-28T13:19:24.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.516+0000 7f7a6ca00640 1 -- 172.21.15.19:0/2832108326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a5c031870 con 0x7f7a700b3270 2024-03-28T13:19:24.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.516+0000 7f7a6ca00640 1 -- 172.21.15.19:0/2832108326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c031c40 con 0x7f7a700b3270 2024-03-28T13:19:24.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.517+0000 7f7a74c00640 1 -- 172.21.15.19:0/2832108326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a700b3270 msgr2=0x7f7a700d4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.517+0000 7f7a74c00640 1 --2- 172.21.15.19:0/2832108326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a700b3270 0x7f7a700d4b10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f7a700b6fc0 tx=0x7f7a5c02b910 comp rx=0 tx=0).stop 2024-03-28T13:19:24.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.517+0000 7f7a74c00640 1 -- 172.21.15.19:0/2832108326 shutdown_connections 2024-03-28T13:19:24.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.517+0000 7f7a74c00640 1 --2- 172.21.15.19:0/2832108326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a700b3270 0x7f7a700d4b10 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.517+0000 7f7a74c00640 1 --2- 172.21.15.19:0/2832108326 >> v2:172.21.15.188:3300/0 conn(0x7f7a70161f10 0x7f7a700d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.517+0000 7f7a74c00640 1 -- 172.21.15.19:0/2832108326 >> 172.21.15.19:0/2832108326 conn(0x7f7a7008bfb0 msgr2=0x7f7a700d4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:24.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.517+0000 7f7a74c00640 1 -- 172.21.15.19:0/2832108326 shutdown_connections 2024-03-28T13:19:24.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.517+0000 7f7a74c00640 1 -- 172.21.15.19:0/2832108326 wait complete. 2024-03-28T13:19:24.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.519+0000 7f7a74c00640 1 Processor -- start 2024-03-28T13:19:24.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.519+0000 7f7a74c00640 1 -- start start 2024-03-28T13:19:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.520+0000 7f7a74c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a7006da00 0x7f7a700c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.520+0000 7f7a74c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a700b3270 0x7f7a7015c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.520+0000 7f7a74c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a70161f10 0x7f7a7015cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.520+0000 7f7a74c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7006cff0 con 0x7f7a7006da00 2024-03-28T13:19:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.520+0000 7f7a74c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701622f0 con 0x7f7a70161f10 2024-03-28T13:19:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.520+0000 7f7a74c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7a7006de70 con 0x7f7a700b3270 2024-03-28T13:19:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.520+0000 7f7a6de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a7006da00 0x7f7a700c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.521+0000 7f7a6de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a7006da00 0x7f7a700c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54020/0 (socket says 172.21.15.19:54020) 2024-03-28T13:19:24.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.521+0000 7f7a6de00640 1 -- 172.21.15.19:0/3441155678 learned_addr learned my addr 172.21.15.19:0/3441155678 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:24.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.521+0000 7f7a6d400640 1 --2- 172.21.15.19:0/3441155678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a700b3270 0x7f7a7015c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.521+0000 7f7a6e800640 1 --2- 172.21.15.19:0/3441155678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a70161f10 0x7f7a7015cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.521+0000 7f7a6de00640 1 -- 172.21.15.19:0/3441155678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a700b3270 msgr2=0x7f7a7015c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.521+0000 7f7a6de00640 1 --2- 172.21.15.19:0/3441155678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a700b3270 0x7f7a7015c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.521+0000 7f7a6de00640 1 -- 172.21.15.19:0/3441155678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a70161f10 msgr2=0x7f7a7015cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.521+0000 7f7a6de00640 1 --2- 172.21.15.19:0/3441155678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a70161f10 0x7f7a7015cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.521+0000 7f7a6de00640 1 -- 172.21.15.19:0/3441155678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a700c8e80 con 0x7f7a7006da00 2024-03-28T13:19:24.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.522+0000 7f7a6de00640 1 --2- 172.21.15.19:0/3441155678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a7006da00 0x7f7a700c8940 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000b410 tx=0x7f7a6000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.522+0000 7f7a66000640 1 -- 172.21.15.19:0/3441155678 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60015020 con 0x7f7a7006da00 2024-03-28T13:19:24.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.522+0000 7f7a66000640 1 -- 172.21.15.19:0/3441155678 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a60011850 con 0x7f7a7006da00 2024-03-28T13:19:24.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.522+0000 7f7a74c00640 1 -- 172.21.15.19:0/3441155678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a700ce1d0 con 0x7f7a7006da00 2024-03-28T13:19:24.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.522+0000 7f7a66000640 1 -- 172.21.15.19:0/3441155678 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60011c40 con 0x7f7a7006da00 2024-03-28T13:19:24.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.522+0000 7f7a74c00640 1 -- 172.21.15.19:0/3441155678 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a700c8210 con 0x7f7a7006da00 2024-03-28T13:19:24.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.525+0000 7f7a66000640 1 -- 172.21.15.19:0/3441155678 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7a60018020 con 0x7f7a7006da00 2024-03-28T13:19:24.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.526+0000 7f7a66000640 1 --2- 172.21.15.19:0/3441155678 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a5006c530 0x7f7a5006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.526+0000 7f7a66000640 1 -- 172.21.15.19:0/3441155678 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5767+0+0 (secure 0 0 0) 0x7f7a6008b6d0 con 0x7f7a7006da00 2024-03-28T13:19:24.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.526+0000 7f7a6d400640 1 --2- 172.21.15.19:0/3441155678 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a5006c530 0x7f7a5006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.527+0000 7f7a77825740 1 Processor -- start 2024-03-28T13:19:24.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.527+0000 7f7a77825740 1 -- start start 2024-03-28T13:19:24.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.528+0000 7f7a6d400640 1 --2- 172.21.15.19:0/3441155678 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a5006c530 0x7f7a5006e9f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c036000 tx=0x7f7a5c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.529+0000 7f7a77825740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb8e3fc0 0x5639fb8e43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.529+0000 7f7a77825740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb8e9de0 0x5639fb8e8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.529+0000 7f7a77825740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb9316d0 0x5639fb931b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.529+0000 7f7a77825740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5639fb84e280 con 0x5639fb8e3fc0 2024-03-28T13:19:24.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.529+0000 7f7a77825740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5639fb7f66f0 con 0x5639fb9316d0 2024-03-28T13:19:24.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.529+0000 7f7a77825740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5639fb84a230 con 0x5639fb8e9de0 2024-03-28T13:19:24.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.529+0000 7f7a6de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb8e3fc0 0x5639fb8e43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.529+0000 7f7a6de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb8e3fc0 0x5639fb8e43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54024/0 (socket says 172.21.15.19:54024) 2024-03-28T13:19:24.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.529+0000 7f7a6de00640 1 -- 172.21.15.19:0/1975516141 learned_addr learned my addr 172.21.15.19:0/1975516141 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:24.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.530+0000 7f7a6d400640 1 --2- 172.21.15.19:0/1975516141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb8e9de0 0x5639fb8e8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.530+0000 7f7a6e800640 1 --2- 172.21.15.19:0/1975516141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb9316d0 0x5639fb931b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.530+0000 7f7a6de00640 1 -- 172.21.15.19:0/1975516141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb8e9de0 msgr2=0x5639fb8e8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.530+0000 7f7a6de00640 1 --2- 172.21.15.19:0/1975516141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb8e9de0 0x5639fb8e8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.530+0000 7f7a6de00640 1 -- 172.21.15.19:0/1975516141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb9316d0 msgr2=0x5639fb931b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.530+0000 7f7a6de00640 1 --2- 172.21.15.19:0/1975516141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb9316d0 0x5639fb931b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.530+0000 7f7a6de00640 1 -- 172.21.15.19:0/1975516141 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639fb62a7e0 con 0x5639fb8e3fc0 2024-03-28T13:19:24.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.530+0000 7f7a6e800640 1 --2- 172.21.15.19:0/1975516141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb9316d0 0x5639fb931b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:24.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.530+0000 7f7a6de00640 1 --2- 172.21.15.19:0/1975516141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb8e3fc0 0x5639fb8e43a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f7a60061bc0 tx=0x7f7a60058fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213fba9838e85d57 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.530+0000 7f7a5a600640 1 -- 172.21.15.19:0/1975516141 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60071020 con 0x5639fb8e3fc0 2024-03-28T13:19:24.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.531+0000 7f7a5a600640 1 -- 172.21.15.19:0/1975516141 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a600575b0 con 0x5639fb8e3fc0 2024-03-28T13:19:24.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.531+0000 7f7a5a600640 1 -- 172.21.15.19:0/1975516141 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60055b80 con 0x5639fb8e3fc0 2024-03-28T13:19:24.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.532+0000 7f7a77825740 1 -- 172.21.15.19:0/1975516141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb8e3fc0 msgr2=0x5639fb8e43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.532+0000 7f7a77825740 1 --2- 172.21.15.19:0/1975516141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb8e3fc0 0x5639fb8e43a0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f7a60061bc0 tx=0x7f7a60058fa0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.532+0000 7f7a77825740 1 -- 172.21.15.19:0/1975516141 shutdown_connections 2024-03-28T13:19:24.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.532+0000 7f7a77825740 1 --2- 172.21.15.19:0/1975516141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb9316d0 0x5639fb931b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.532+0000 7f7a77825740 1 --2- 172.21.15.19:0/1975516141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb8e9de0 0x5639fb8e8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.532+0000 7f7a77825740 1 --2- 172.21.15.19:0/1975516141 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb8e3fc0 0x5639fb8e43a0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.532+0000 7f7a77825740 1 -- 172.21.15.19:0/1975516141 >> 172.21.15.19:0/1975516141 conn(0x5639fb8e6040 msgr2=0x5639fb8e7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:24.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.532+0000 7f7a77825740 1 -- 172.21.15.19:0/1975516141 shutdown_connections 2024-03-28T13:19:24.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.532+0000 7f7a77825740 1 -- 172.21.15.19:0/1975516141 wait complete. 2024-03-28T13:19:24.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.533+0000 7f7a77825740 1 -- start start 2024-03-28T13:19:24.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.533+0000 7f7a77825740 1 Processor -- start 2024-03-28T13:19:24.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.534+0000 7f7a77825740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb8e9de0 0x5639fb903190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.534+0000 7f7a77825740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb9316d0 0x5639fb917ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.534+0000 7f7a77825740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb927470 0x5639fb93fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.534+0000 7f7a77825740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5639fb910730 con 0x5639fb9316d0 2024-03-28T13:19:24.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.534+0000 7f7a77825740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5639fb809aa0 con 0x5639fb927470 2024-03-28T13:19:24.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.534+0000 7f7a77825740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5639fb910980 con 0x5639fb8e9de0 2024-03-28T13:19:24.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.534+0000 7f7a77825740 10 client.0 _ll_register_callbacks cb 0x5639fb809490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:19:24.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.534+0000 7f7a6d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb9316d0 0x5639fb917ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.534+0000 7f7a6d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb9316d0 0x5639fb917ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54028/0 (socket says 172.21.15.19:54028) 2024-03-28T13:19:24.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.534+0000 7f7a6d400640 1 -- 172.21.15.19:0/4106441433 learned_addr learned my addr 172.21.15.19:0/4106441433 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:24.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.534+0000 7f7a6de00640 1 --2- 172.21.15.19:0/4106441433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb8e9de0 0x5639fb903190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.534+0000 7f7a6e800640 1 --2- 172.21.15.19:0/4106441433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb927470 0x5639fb93fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.535+0000 7f7a6d400640 1 -- 172.21.15.19:0/4106441433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb8e9de0 msgr2=0x5639fb903190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.535+0000 7f7a6d400640 1 --2- 172.21.15.19:0/4106441433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb8e9de0 0x5639fb903190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.535+0000 7f7a6d400640 1 -- 172.21.15.19:0/4106441433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb927470 msgr2=0x5639fb93fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.535+0000 7f7a6d400640 1 --2- 172.21.15.19:0/4106441433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb927470 0x5639fb93fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.535+0000 7f7a6d400640 1 -- 172.21.15.19:0/4106441433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639fb68af70 con 0x5639fb9316d0 2024-03-28T13:19:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.535+0000 7f7a6d400640 1 --2- 172.21.15.19:0/4106441433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb9316d0 0x5639fb917ac0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c02bf20 tx=0x7f7a5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d0c3b7ba81782e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.535+0000 7f7a4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:19:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.535+0000 7f7a4ea00640 1 -- 172.21.15.19:0/4106441433 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c054060 con 0x5639fb9316d0 2024-03-28T13:19:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.536+0000 7f7a4ea00640 1 -- 172.21.15.19:0/4106441433 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 5767+0+0 (secure 0 0 0) 0x7f7a5c044810 con 0x5639fb9316d0 2024-03-28T13:19:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.536+0000 7f7a77825740 1 -- 172.21.15.19:0/4106441433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a600575b0 con 0x5639fb9316d0 2024-03-28T13:19:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.536+0000 7f7a4ea00640 1 -- 172.21.15.19:0/4106441433 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f7a5c044a10 con 0x5639fb9316d0 2024-03-28T13:19:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.536+0000 7f7a77825740 10 client.8322 fetch_fsmap learned FSMap version 83 2024-03-28T13:19:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.536+0000 7f7a77825740 1 -- 172.21.15.19:0/4106441433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x5639fb522170 con 0x5639fb9316d0 2024-03-28T13:19:24.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.536+0000 7f7a4ea00640 1 -- 172.21.15.19:0/4106441433 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a600575b0 con 0x5639fb9316d0 2024-03-28T13:19:24.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.536+0000 7f7a4ea00640 1 -- 172.21.15.19:0/4106441433 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f7a5c04e070 con 0x5639fb9316d0 2024-03-28T13:19:24.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.537+0000 7f7a77825740 10 client.8322 fetch_fsmap finished waiting for FSMap version 83 2024-03-28T13:19:24.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.537+0000 7f7a77825740 10 client.8322 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-28T13:19:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.537+0000 7f7a77825740 20 client.8322 populate_metadata read hostname 'smithi019' 2024-03-28T13:19:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.537+0000 7f7a77825740 1 --2- 172.21.15.19:0/4106441433 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5639fb913ee0 0x5639fb914ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.537+0000 7f7a77825740 4 client.8322 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:19:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.537+0000 7f7a77825740 1 -- 172.21.15.19:0/4106441433 --> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639fb85b7c0 con 0x5639fb913ee0 2024-03-28T13:19:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.537+0000 7f7a6de00640 1 --2- 172.21.15.19:0/4106441433 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5639fb913ee0 0x5639fb914ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.538+0000 7f7a6de00640 1 --2- 172.21.15.19:0/4106441433 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5639fb913ee0 0x5639fb914ef0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ce279a3bdb77b48 server_cookie=df8d2dba5d391d35 in_seq=0 out_seq=0 2024-03-28T13:19:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.538+0000 7f7a4ea00640 10 client.8322 ms_handle_connect on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.539+0000 7f7a4ea00640 1 -- 172.21.15.19:0/4106441433 <== mds.0 v2:172.21.15.19:6832/1020599987 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639fb85b7c0 con 0x5639fb913ee0 2024-03-28T13:19:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.539+0000 7f7a4ea00640 10 client.8322 handle_command_reply: tid=0 2024-03-28T13:19:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.539+0000 7f7a77825740 1 client.8322 shutdown 2024-03-28T13:19:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.539+0000 7f7a77825740 1 -- 172.21.15.19:0/4106441433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb9316d0 msgr2=0x5639fb917ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.539+0000 7f7a77825740 1 --2- 172.21.15.19:0/4106441433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb9316d0 0x5639fb917ac0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c02bf20 tx=0x7f7a5c004730 comp rx=0 tx=0).stop 2024-03-28T13:19:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.539+0000 7f7a77825740 1 -- 172.21.15.19:0/4106441433 shutdown_connections 2024-03-28T13:19:24.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.539+0000 7f7a77825740 1 --2- 172.21.15.19:0/4106441433 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5639fb913ee0 0x5639fb914ef0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.539+0000 7f7a77825740 1 --2- 172.21.15.19:0/4106441433 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb927470 0x5639fb93fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.539+0000 7f7a77825740 1 --2- 172.21.15.19:0/4106441433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb9316d0 0x5639fb917ac0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.539+0000 7f7a77825740 1 --2- 172.21.15.19:0/4106441433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb8e9de0 0x5639fb903190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.539+0000 7f7a77825740 1 -- 172.21.15.19:0/4106441433 >> 172.21.15.19:0/4106441433 conn(0x5639fb8e3fc0 msgr2=0x5639fb930140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:24.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.539+0000 7f7a4ea00640 0 client.8322 ms_handle_reset on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:24.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.539+0000 7f7a77825740 1 -- 172.21.15.19:0/4106441433 shutdown_connections 2024-03-28T13:19:24.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.539+0000 7f7a77825740 1 -- 172.21.15.19:0/4106441433 wait complete. 2024-03-28T13:19:24.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.540+0000 7f7a77825740 20 client.8322 trim_cache size 0 max 16384 2024-03-28T13:19:24.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.553+0000 7f4de1a00640 1 Processor -- start 2024-03-28T13:19:24.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.553+0000 7f4de1a00640 1 -- start start 2024-03-28T13:19:24.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.555+0000 7f4de1a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ddc0aebf0 0x7f4ddc0af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.555+0000 7f4de1a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ddc06e4e0 con 0x7f4ddc145ce0 2024-03-28T13:19:24.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.555+0000 7f4de1a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ddc06e7c0 con 0x7f4ddc071850 2024-03-28T13:19:24.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.555+0000 7f4de1a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4ddc06eaa0 con 0x7f4ddc0aebf0 2024-03-28T13:19:24.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.556+0000 7f4ddaa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4ddc145ce0 0x7f4ddc0acff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34214/0 (socket says 172.21.15.19:34214) 2024-03-28T13:19:24.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.556+0000 7f4ddaa00640 1 -- 172.21.15.19:0/3420922009 learned_addr learned my addr 172.21.15.19:0/3420922009 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:24.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.556+0000 7f4dda000640 1 --2- 172.21.15.19:0/3420922009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ddc0aebf0 0x7f4ddc0af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.556+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2894686993 0 0) 0x7f4ddc06e4e0 con 0x7f4ddc145ce0 2024-03-28T13:19:24.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.556+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dc0003520 con 0x7f4ddc145ce0 2024-03-28T13:19:24.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3913068715 0 0) 0x7f4ddc06e7c0 con 0x7f4ddc071850 2024-03-28T13:19:24.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ddc06e4e0 con 0x7f4ddc071850 2024-03-28T13:19:24.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2708164647 0 0) 0x7f4dc0003520 con 0x7f4ddc145ce0 2024-03-28T13:19:24.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ddc0aebf0 msgr2=0x7f4ddc0af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dd9600640 1 --2- 172.21.15.19:0/3420922009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ddc0aebf0 0x7f4ddc0af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 >> v1:172.21.15.188:6789/0 conn(0x7f4ddc071850 legacy=0x7f4ddc071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc06ed80 con 0x7f4ddc145ce0 2024-03-28T13:19:24.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f4dcc001690 con 0x7f4ddc145ce0 2024-03-28T13:19:24.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 >> v1:172.21.15.19:6789/0 conn(0x7f4ddc145ce0 legacy=0x7f4ddc0acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dd9600640 1 --2- 172.21.15.19:0/3420922009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dc0003b70 0x7f4dc0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dd9600640 1 --2- 172.21.15.19:0/3420922009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dc0004800 0x7f4dc0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dd9600640 1 --2- 172.21.15.19:0/3420922009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ddc0aebf0 0x7f4dc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc06eaa0 con 0x7f4dc0004800 2024-03-28T13:19:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc000ba70 con 0x7f4dc0003b70 2024-03-28T13:19:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4dc000bc40 con 0x7f4ddc0aebf0 2024-03-28T13:19:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4dda000640 1 --2- 172.21.15.19:0/3420922009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dc0004800 0x7f4dc0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4ddaa00640 1 --2- 172.21.15.19:0/3420922009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dc0003b70 0x7f4dc0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.557+0000 7f4ddb400640 1 --2- 172.21.15.19:0/3420922009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ddc0aebf0 0x7f4dc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.558+0000 7f4dda000640 1 -- 172.21.15.19:0/3420922009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ddc0aebf0 msgr2=0x7f4dc00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.558+0000 7f4dda000640 1 --2- 172.21.15.19:0/3420922009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ddc0aebf0 0x7f4dc00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.558+0000 7f4dda000640 1 -- 172.21.15.19:0/3420922009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dc0003b70 msgr2=0x7f4dc0003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.558+0000 7f4dda000640 1 --2- 172.21.15.19:0/3420922009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4dc0003b70 0x7f4dc0003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.558+0000 7f4dda000640 1 -- 172.21.15.19:0/3420922009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc0003520 con 0x7f4dc0004800 2024-03-28T13:19:24.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.559+0000 7f4dda000640 1 --2- 172.21.15.19:0/3420922009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dc0004800 0x7f4dc0004c80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009ac0 tx=0x7f4dc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0d3fb3376e0756 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.559+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8039070 con 0x7f4dc0004800 2024-03-28T13:19:24.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.559+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc802c660 con 0x7f4dc0004800 2024-03-28T13:19:24.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.559+0000 7f4dd9600640 1 -- 172.21.15.19:0/3420922009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8009690 con 0x7f4dc0004800 2024-03-28T13:19:24.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.560+0000 7f4de1a00640 1 -- 172.21.15.19:0/3420922009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dc0004800 msgr2=0x7f4dc0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.560+0000 7f4de1a00640 1 --2- 172.21.15.19:0/3420922009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4dc0004800 0x7f4dc0004c80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009ac0 tx=0x7f4dc0005830 comp rx=0 tx=0).stop 2024-03-28T13:19:24.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.560+0000 7f4ddb400640 1 -- 172.21.15.19:0/3420922009 reap_dead start 2024-03-28T13:19:24.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.560+0000 7f4de1a00640 1 -- 172.21.15.19:0/3420922009 shutdown_connections 2024-03-28T13:19:24.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.560+0000 7f4de1a00640 1 -- 172.21.15.19:0/3420922009 >> 172.21.15.19:0/3420922009 conn(0x7f4ddc06b570 msgr2=0x7f4ddc06b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:24.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.560+0000 7f4de1a00640 1 -- 172.21.15.19:0/3420922009 shutdown_connections 2024-03-28T13:19:24.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.560+0000 7f4de1a00640 1 -- 172.21.15.19:0/3420922009 wait complete. 2024-03-28T13:19:24.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.560+0000 7f7a77825740 1 Processor -- start 2024-03-28T13:19:24.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.560+0000 7f7a77825740 1 -- start start 2024-03-28T13:19:24.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.561+0000 7f7a77825740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb8e9de0 0x5639fb926ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.561+0000 7f7a77825740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb927470 0x5639fb8e8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.561+0000 7f7a77825740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb9316d0 0x5639fb93f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.561+0000 7f7a77825740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70200c90 con 0x5639fb8e9de0 2024-03-28T13:19:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.561+0000 7f7a77825740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7a700cd500 con 0x5639fb927470 2024-03-28T13:19:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.561+0000 7f7a77825740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7a7015b780 con 0x5639fb9316d0 2024-03-28T13:19:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.562+0000 7f7a6de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb8e9de0 0x5639fb926ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.562+0000 7f7a6de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb8e9de0 0x5639fb926ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54044/0 (socket says 172.21.15.19:54044) 2024-03-28T13:19:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.562+0000 7f7a6de00640 1 -- 172.21.15.19:0/3319555122 learned_addr learned my addr 172.21.15.19:0/3319555122 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.562+0000 7f7a6e800640 1 --2- 172.21.15.19:0/3319555122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb9316d0 0x5639fb93f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.562+0000 7f7a6d400640 1 --2- 172.21.15.19:0/3319555122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb927470 0x5639fb8e8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.562+0000 7f4de1a00640 1 Processor -- start 2024-03-28T13:19:24.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.562+0000 7f4de1a00640 1 -- start start 2024-03-28T13:19:24.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.562+0000 7f7a6de00640 1 -- 172.21.15.19:0/3319555122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb9316d0 msgr2=0x5639fb93f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.562+0000 7f7a6de00640 1 --2- 172.21.15.19:0/3319555122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb9316d0 0x5639fb93f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.562+0000 7f7a6de00640 1 -- 172.21.15.19:0/3319555122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb927470 msgr2=0x5639fb8e8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.562+0000 7f7a6de00640 1 --2- 172.21.15.19:0/3319555122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb927470 0x5639fb8e8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.562+0000 7f7a6de00640 1 -- 172.21.15.19:0/3319555122 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639fb9309f0 con 0x5639fb8e9de0 2024-03-28T13:19:24.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.563+0000 7f7a6de00640 1 --2- 172.21.15.19:0/3319555122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb8e9de0 0x5639fb926ba0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f7a60055970 tx=0x7f7a6005b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c0e9a8835715df server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.563+0000 7f7a59c00640 1 -- 172.21.15.19:0/3319555122 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60071040 con 0x5639fb8e9de0 2024-03-28T13:19:24.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.563+0000 7f7a59c00640 1 -- 172.21.15.19:0/3319555122 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a60077070 con 0x5639fb8e9de0 2024-03-28T13:19:24.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.563+0000 7f7a59c00640 1 -- 172.21.15.19:0/3319555122 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60057810 con 0x5639fb8e9de0 2024-03-28T13:19:24.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.563+0000 7f4de1a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4dc0004800 0x7f4ddc1f9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.563+0000 7f4de1a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ddc15f490 0x7f4ddc09cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f4de1a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ddc154e10 0x7f4ddc155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f4de1a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc0ad700 con 0x7f4ddc15f490 2024-03-28T13:19:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f4de1a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ddc06f060 con 0x7f4ddc154e10 2024-03-28T13:19:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f4de1a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4ddc1460c0 con 0x7f4dc0004800 2024-03-28T13:19:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f4dda000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ddc15f490 0x7f4ddc09cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f7a77825740 1 -- 172.21.15.19:0/3319555122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb8e9de0 msgr2=0x5639fb926ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f7a77825740 1 --2- 172.21.15.19:0/3319555122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb8e9de0 0x5639fb926ba0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f7a60055970 tx=0x7f7a6005b000 comp rx=0 tx=0).stop 2024-03-28T13:19:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f7a77825740 1 -- 172.21.15.19:0/3319555122 shutdown_connections 2024-03-28T13:19:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f7a77825740 1 --2- 172.21.15.19:0/3319555122 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb9316d0 0x5639fb93f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f7a77825740 1 --2- 172.21.15.19:0/3319555122 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb927470 0x5639fb8e8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f7a77825740 1 --2- 172.21.15.19:0/3319555122 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb8e9de0 0x5639fb926ba0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f7a77825740 1 -- 172.21.15.19:0/3319555122 >> 172.21.15.19:0/3319555122 conn(0x5639fb8e3fc0 msgr2=0x5639fb9537f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:24.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f4dda000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ddc15f490 0x7f4ddc09cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54052/0 (socket says 172.21.15.19:54052) 2024-03-28T13:19:24.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f4dda000640 1 -- 172.21.15.19:0/555064690 learned_addr learned my addr 172.21.15.19:0/555064690 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:24.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f7a77825740 1 -- 172.21.15.19:0/3319555122 shutdown_connections 2024-03-28T13:19:24.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f4ddaa00640 1 --2- 172.21.15.19:0/555064690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4dc0004800 0x7f4ddc1f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f4ddb400640 1 --2- 172.21.15.19:0/555064690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ddc154e10 0x7f4ddc155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f7a77825740 1 -- 172.21.15.19:0/3319555122 wait complete. 2024-03-28T13:19:24.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f4dda000640 1 -- 172.21.15.19:0/555064690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4dc0004800 msgr2=0x7f4ddc1f9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f4dda000640 1 --2- 172.21.15.19:0/555064690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4dc0004800 0x7f4ddc1f9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f4dda000640 1 -- 172.21.15.19:0/555064690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ddc154e10 msgr2=0x7f4ddc155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.564+0000 7f4dda000640 1 --2- 172.21.15.19:0/555064690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ddc154e10 0x7f4ddc155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.565+0000 7f4dda000640 1 -- 172.21.15.19:0/555064690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ddc15f910 con 0x7f4ddc15f490 2024-03-28T13:19:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.565+0000 7f4ddb400640 1 --2- 172.21.15.19:0/555064690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ddc154e10 0x7f4ddc155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:19:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.565+0000 7f4dda000640 1 --2- 172.21.15.19:0/555064690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ddc15f490 0x7f4ddc09cdd0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8037040 tx=0x7f4dc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.565+0000 7f7a77825740 1 -- start start 2024-03-28T13:19:24.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.565+0000 7f7a77825740 1 Processor -- start 2024-03-28T13:19:24.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.565+0000 7f4dd2a00640 1 -- 172.21.15.19:0/555064690 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8042020 con 0x7f4ddc15f490 2024-03-28T13:19:24.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.565+0000 7f4dd2a00640 1 -- 172.21.15.19:0/555064690 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dc803a040 con 0x7f4ddc15f490 2024-03-28T13:19:24.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.565+0000 7f4dd2a00640 1 -- 172.21.15.19:0/555064690 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8039070 con 0x7f4ddc15f490 2024-03-28T13:19:24.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.565+0000 7f4de1a00640 1 -- 172.21.15.19:0/555064690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ddc1fc220 con 0x7f4ddc15f490 2024-03-28T13:19:24.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.566+0000 7f4de1a00640 1 -- 172.21.15.19:0/555064690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ddc1fc7e0 con 0x7f4ddc15f490 2024-03-28T13:19:24.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.566+0000 7f4de1a00640 1 -- 172.21.15.19:0/555064690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f4ddc0685c0 con 0x7f4ddc15f490 2024-03-28T13:19:24.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.566+0000 7f7a77825740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb8e9de0 0x5639fb949a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.566+0000 7f7a77825740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb927470 0x5639fb9a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.566+0000 7f7a77825740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb9316d0 0x5639fb9ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.566+0000 7f7a77825740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a700cceb0 con 0x5639fb927470 2024-03-28T13:19:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.566+0000 7f7a77825740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5639fb8e6430 con 0x5639fb9316d0 2024-03-28T13:19:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.566+0000 7f7a77825740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5639fb8e8660 con 0x5639fb8e9de0 2024-03-28T13:19:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.567+0000 7f7a77825740 10 client.0 _ll_register_callbacks cb 0x7f7a5c0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:19:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.567+0000 7f7a6d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb927470 0x5639fb9a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.567+0000 7f7a6d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb927470 0x5639fb9a97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54056/0 (socket says 172.21.15.19:54056) 2024-03-28T13:19:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.567+0000 7f7a6d400640 1 -- 172.21.15.19:0/3083183008 learned_addr learned my addr 172.21.15.19:0/3083183008 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.567+0000 7f7a6de00640 1 --2- 172.21.15.19:0/3083183008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb8e9de0 0x5639fb949a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.567+0000 7f7a6e800640 1 --2- 172.21.15.19:0/3083183008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb9316d0 0x5639fb9ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.567+0000 7f7a6d400640 1 -- 172.21.15.19:0/3083183008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb8e9de0 msgr2=0x5639fb949a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.567+0000 7f7a6d400640 1 --2- 172.21.15.19:0/3083183008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb8e9de0 0x5639fb949a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.567+0000 7f7a6d400640 1 -- 172.21.15.19:0/3083183008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb9316d0 msgr2=0x5639fb9ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.567+0000 7f7a6d400640 1 --2- 172.21.15.19:0/3083183008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb9316d0 0x5639fb9ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.567+0000 7f7a6d400640 1 -- 172.21.15.19:0/3083183008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639fb925820 con 0x5639fb927470 2024-03-28T13:19:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.568+0000 7f7a6d400640 1 --2- 172.21.15.19:0/3083183008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb927470 0x5639fb9a97f0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c02b950 tx=0x7f7a5c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7107dd6fb29ae0c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.568+0000 7f7a4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:19:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.568+0000 7f7a4ea00640 1 -- 172.21.15.19:0/3083183008 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c05e080 con 0x5639fb927470 2024-03-28T13:19:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.568+0000 7f7a4ea00640 1 -- 172.21.15.19:0/3083183008 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 5767+0+0 (secure 0 0 0) 0x7f7a5c059d50 con 0x5639fb927470 2024-03-28T13:19:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.568+0000 7f7a77825740 1 -- 172.21.15.19:0/3083183008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a60077070 con 0x5639fb927470 2024-03-28T13:19:24.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.568+0000 7f4dd2a00640 1 -- 172.21.15.19:0/555064690 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4dc8045020 con 0x7f4ddc15f490 2024-03-28T13:19:24.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.568+0000 7f7a4ea00640 1 -- 172.21.15.19:0/3083183008 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f7a5c068050 con 0x5639fb927470 2024-03-28T13:19:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.568+0000 7f7a77825740 10 client.8334 fetch_fsmap learned FSMap version 83 2024-03-28T13:19:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.569+0000 7f7a77825740 1 -- 172.21.15.19:0/3083183008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x5639fb93fd20 con 0x5639fb927470 2024-03-28T13:19:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.569+0000 7f7a4ea00640 1 -- 172.21.15.19:0/3083183008 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a60077070 con 0x5639fb927470 2024-03-28T13:19:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.569+0000 7f7a4ea00640 1 -- 172.21.15.19:0/3083183008 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f7a5c06b020 con 0x5639fb927470 2024-03-28T13:19:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.569+0000 7f7a77825740 10 client.8334 fetch_fsmap finished waiting for FSMap version 83 2024-03-28T13:19:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.569+0000 7f7a77825740 10 client.8334 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-28T13:19:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.569+0000 7f7a77825740 20 client.8334 populate_metadata read hostname 'smithi019' 2024-03-28T13:19:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.569+0000 7f7a77825740 1 --2- 172.21.15.19:0/3083183008 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5639fb9ad9a0 0x5639fb948980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.569+0000 7f7a77825740 4 client.8334 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-03-28T13:19:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.569+0000 7f7a77825740 1 -- 172.21.15.19:0/3083183008 --> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x5639fb855b10 con 0x5639fb9ad9a0 2024-03-28T13:19:24.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.569+0000 7f7a6de00640 1 --2- 172.21.15.19:0/3083183008 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5639fb9ad9a0 0x5639fb948980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.570+0000 7f4dd2a00640 1 --2- 172.21.15.19:0/555064690 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4dbc06c560 0x7f4dbc06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.570+0000 7f4dd2a00640 1 -- 172.21.15.19:0/555064690 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5767+0+0 (secure 0 0 0) 0x7f4dc80acda0 con 0x7f4ddc15f490 2024-03-28T13:19:24.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.570+0000 7f4dd2a00640 1 -- 172.21.15.19:0/555064690 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+514 (secure 0 0 0) 0x7f4dc804c020 con 0x7f4ddc15f490 2024-03-28T13:19:24.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.570+0000 7f4ddaa00640 1 --2- 172.21.15.19:0/555064690 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4dbc06c560 0x7f4dbc06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.570+0000 7f7a6de00640 1 --2- 172.21.15.19:0/3083183008 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5639fb9ad9a0 0x5639fb948980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb3e78f7f7f9a320 server_cookie=6379d5c4199d89f in_seq=0 out_seq=0 2024-03-28T13:19:24.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.570+0000 7f7a4ea00640 10 client.8334 ms_handle_connect on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:24.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.570+0000 7f4de1a00640 1 -- 172.21.15.19:0/555064690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f4ddc13c380 con 0x7f4ddc15f490 2024-03-28T13:19:24.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.571+0000 7f4dd2a00640 1 -- 172.21.15.19:0/555064690 <== mon.0 v2:172.21.15.19:3300/0 7 ==== log(4 entries from seq 163 at 2024-03-28T13:19:24.249082+0000) ==== 1122+0+0 (secure 0 0 0) 0x7f4dc8078da0 con 0x7f4ddc15f490 2024-03-28T13:19:24.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.571+0000 7f7a4ea00640 1 -- 172.21.15.19:0/3083183008 <== mds.0 v2:172.21.15.19:6832/1020599987 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5639fb855b10 con 0x5639fb9ad9a0 2024-03-28T13:19:24.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.571+0000 7f7a4ea00640 10 client.8334 handle_command_reply: tid=0 2024-03-28T13:19:24.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.571+0000 7f7a77825740 1 client.8334 shutdown 2024-03-28T13:19:24.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.572+0000 7f7a77825740 1 -- 172.21.15.19:0/3083183008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb927470 msgr2=0x5639fb9a97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.572+0000 7f7a77825740 1 --2- 172.21.15.19:0/3083183008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb927470 0x5639fb9a97f0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c02b950 tx=0x7f7a5c009240 comp rx=0 tx=0).stop 2024-03-28T13:19:24.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.572+0000 7f4ddaa00640 1 --2- 172.21.15.19:0/555064690 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4dbc06c560 0x7f4dbc06ea20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc0cf750 tx=0x7f4dcc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.572+0000 7f7a77825740 1 -- 172.21.15.19:0/3083183008 shutdown_connections 2024-03-28T13:19:24.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.572+0000 7f7a77825740 1 --2- 172.21.15.19:0/3083183008 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5639fb9ad9a0 0x5639fb948980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.572+0000 7f7a77825740 1 --2- 172.21.15.19:0/3083183008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5639fb9316d0 0x5639fb9ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.572+0000 7f7a77825740 1 --2- 172.21.15.19:0/3083183008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5639fb927470 0x5639fb9a97f0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.572+0000 7f7a77825740 1 --2- 172.21.15.19:0/3083183008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5639fb8e9de0 0x5639fb949a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.572+0000 7f7a77825740 1 -- 172.21.15.19:0/3083183008 >> 172.21.15.19:0/3083183008 conn(0x5639fb8e3fc0 msgr2=0x5639fb9262b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:24.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.572+0000 7f7a4ea00640 0 client.8334 ms_handle_reset on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:24.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.572+0000 7f7a77825740 1 -- 172.21.15.19:0/3083183008 shutdown_connections 2024-03-28T13:19:24.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.572+0000 7f7a77825740 1 -- 172.21.15.19:0/3083183008 wait complete. 2024-03-28T13:19:24.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.572+0000 7f7a77825740 20 client.8334 trim_cache size 0 max 16384 2024-03-28T13:19:24.583 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:19:24.584 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-03-28T13:19:24.584 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "07478f43-9ddb-433e-85a1-a9a331de9b22", 2024-03-28T13:19:24.584 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-03-28T13:19:24.584 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:19:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.573+0000 7f7a74c00640 1 -- 172.21.15.19:0/3441155678 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a5006c530 msgr2=0x7f7a5006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.573+0000 7f7a74c00640 1 --2- 172.21.15.19:0/3441155678 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a5006c530 0x7f7a5006e9f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c036000 tx=0x7f7a5c03c040 comp rx=0 tx=0).stop 2024-03-28T13:19:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.573+0000 7f7a74c00640 1 -- 172.21.15.19:0/3441155678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a7006da00 msgr2=0x7f7a700c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.573+0000 7f7a74c00640 1 --2- 172.21.15.19:0/3441155678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a7006da00 0x7f7a700c8940 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000b410 tx=0x7f7a6000b8d0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.573+0000 7f7a74c00640 1 -- 172.21.15.19:0/3441155678 shutdown_connections 2024-03-28T13:19:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.573+0000 7f7a74c00640 1 --2- 172.21.15.19:0/3441155678 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a5006c530 0x7f7a5006e9f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.573+0000 7f7a74c00640 1 --2- 172.21.15.19:0/3441155678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a70161f10 0x7f7a7015cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.573+0000 7f7a74c00640 1 --2- 172.21.15.19:0/3441155678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a700b3270 0x7f7a7015c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.573+0000 7f7a74c00640 1 --2- 172.21.15.19:0/3441155678 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a7006da00 0x7f7a700c8940 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.573+0000 7f7a74c00640 1 -- 172.21.15.19:0/3441155678 >> 172.21.15.19:0/3441155678 conn(0x7f7a7008bfb0 msgr2=0x7f7a70162d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:24.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.574+0000 7f7a74c00640 1 -- 172.21.15.19:0/3441155678 shutdown_connections 2024-03-28T13:19:24.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.574+0000 7f7a74c00640 1 -- 172.21.15.19:0/3441155678 wait complete. 2024-03-28T13:19:24.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:24.592 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-28T13:19:24.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.784+0000 7f31a2a00640 1 Processor -- start 2024-03-28T13:19:24.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.785+0000 7f31a2a00640 1 -- start start 2024-03-28T13:19:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.786+0000 7f31a2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f319c153580 0x7f319c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.786+0000 7f31a2a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f319c0afab0 con 0x7f319c0af4b0 2024-03-28T13:19:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.786+0000 7f31a2a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f319c15cc80 con 0x7f319c0d07c0 2024-03-28T13:19:24.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.786+0000 7f31a2a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f319c0d0c30 con 0x7f319c153580 2024-03-28T13:19:24.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.786+0000 7f31a0a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f319c0af4b0 0x7f319c1600d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34218/0 (socket says 172.21.15.19:34218) 2024-03-28T13:19:24.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.786+0000 7f31a0a00640 1 -- 172.21.15.19:0/3147303552 learned_addr learned my addr 172.21.15.19:0/3147303552 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:24.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.786+0000 7f319be00640 1 --2- 172.21.15.19:0/3147303552 >> v2:172.21.15.19:3300/0 conn(0x7f319c153580 0x7f319c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.787+0000 7f319be00640 1 -- 172.21.15.19:0/3147303552 >> v1:172.21.15.188:6789/0 conn(0x7f319c0d07c0 legacy=0x7f319c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.787+0000 7f319be00640 1 -- 172.21.15.19:0/3147303552 >> v1:172.21.15.19:6789/0 conn(0x7f319c0af4b0 legacy=0x7f319c1600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.787+0000 7f319aa00640 1 -- 172.21.15.19:0/3147303552 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2933910756 0 0) 0x7f319c0afab0 con 0x7f319c0af4b0 2024-03-28T13:19:24.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.787+0000 7f319be00640 1 -- 172.21.15.19:0/3147303552 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c15da00 con 0x7f319c153580 2024-03-28T13:19:24.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.788+0000 7f319be00640 1 --2- 172.21.15.19:0/3147303552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f319c153580 0x7f319c0d0280 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f318c0097a0 tx=0x7f318c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219c324a24e30d2a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.788+0000 7f319aa00640 1 -- 172.21.15.19:0/3147303552 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c035020 con 0x7f319c153580 2024-03-28T13:19:24.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.788+0000 7f319aa00640 1 -- 172.21.15.19:0/3147303552 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f318c031830 con 0x7f319c153580 2024-03-28T13:19:24.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.788+0000 7f319aa00640 1 -- 172.21.15.19:0/3147303552 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c031c00 con 0x7f319c153580 2024-03-28T13:19:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.789+0000 7f31a2a00640 1 -- 172.21.15.19:0/3147303552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f319c153580 msgr2=0x7f319c0d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.789+0000 7f31a2a00640 1 --2- 172.21.15.19:0/3147303552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f319c153580 0x7f319c0d0280 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f318c0097a0 tx=0x7f318c02b8d0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.789+0000 7f31a2a00640 1 -- 172.21.15.19:0/3147303552 shutdown_connections 2024-03-28T13:19:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.789+0000 7f31a2a00640 1 --2- 172.21.15.19:0/3147303552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f319c153580 0x7f319c0d0280 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.789+0000 7f31a2a00640 1 -- 172.21.15.19:0/3147303552 >> 172.21.15.19:0/3147303552 conn(0x7f319c06b750 msgr2=0x7f319c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.789+0000 7f31a2a00640 1 -- 172.21.15.19:0/3147303552 shutdown_connections 2024-03-28T13:19:24.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.789+0000 7f31a2a00640 1 -- 172.21.15.19:0/3147303552 wait complete. 2024-03-28T13:19:24.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.791+0000 7f31a2a00640 1 Processor -- start 2024-03-28T13:19:24.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.791+0000 7f31a2a00640 1 -- start start 2024-03-28T13:19:24.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.792+0000 7f31a2a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f319c0af4b0 0x7f319c071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.792+0000 7f31a2a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319c0d07c0 0x7f319c073280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.792+0000 7f31a2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f319c153580 0x7f319c14a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.792+0000 7f31a2a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f319c15d4e0 con 0x7f319c153580 2024-03-28T13:19:24.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.792+0000 7f31a2a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f319c15d200 con 0x7f319c0af4b0 2024-03-28T13:19:24.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.792+0000 7f31a2a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f319c153960 con 0x7f319c0d07c0 2024-03-28T13:19:24.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.792+0000 7f31a0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f319c153580 0x7f319c14a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.792+0000 7f31a0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f319c153580 0x7f319c14a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54082/0 (socket says 172.21.15.19:54082) 2024-03-28T13:19:24.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.792+0000 7f31a0a00640 1 -- 172.21.15.19:0/4028803892 learned_addr learned my addr 172.21.15.19:0/4028803892 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:24.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.793+0000 7f319be00640 1 --2- 172.21.15.19:0/4028803892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f319c0af4b0 0x7f319c071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.793+0000 7f319b400640 1 --2- 172.21.15.19:0/4028803892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319c0d07c0 0x7f319c073280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.793+0000 7f31a0a00640 1 -- 172.21.15.19:0/4028803892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319c0d07c0 msgr2=0x7f319c073280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.793+0000 7f31a0a00640 1 --2- 172.21.15.19:0/4028803892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319c0d07c0 0x7f319c073280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.793+0000 7f31a0a00640 1 -- 172.21.15.19:0/4028803892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f319c0af4b0 msgr2=0x7f319c071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.793+0000 7f31a0a00640 1 --2- 172.21.15.19:0/4028803892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f319c0af4b0 0x7f319c071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.793+0000 7f31a0a00640 1 -- 172.21.15.19:0/4028803892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c0721e0 con 0x7f319c153580 2024-03-28T13:19:24.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.793+0000 7f31a0a00640 1 --2- 172.21.15.19:0/4028803892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f319c153580 0x7f319c14a2c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f319400c700 tx=0x7f319400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.794+0000 7f3193e00640 1 -- 172.21.15.19:0/4028803892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194018070 con 0x7f319c153580 2024-03-28T13:19:24.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.794+0000 7f3193e00640 1 -- 172.21.15.19:0/4028803892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f319400ce10 con 0x7f319c153580 2024-03-28T13:19:24.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.794+0000 7f3193e00640 1 -- 172.21.15.19:0/4028803892 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319400a860 con 0x7f319c153580 2024-03-28T13:19:24.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.794+0000 7f31a2a00640 1 -- 172.21.15.19:0/4028803892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c0724c0 con 0x7f319c153580 2024-03-28T13:19:24.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.794+0000 7f31a2a00640 1 -- 172.21.15.19:0/4028803892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c072a80 con 0x7f319c153580 2024-03-28T13:19:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.797+0000 7f3193e00640 1 -- 172.21.15.19:0/4028803892 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f319400aa00 con 0x7f319c153580 2024-03-28T13:19:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.798+0000 7f31a5782740 1 Processor -- start 2024-03-28T13:19:24.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.798+0000 7f31a5782740 1 -- start start 2024-03-28T13:19:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.798+0000 7f3193e00640 1 --2- 172.21.15.19:0/4028803892 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f317c06c530 0x7f317c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.798+0000 7f3193e00640 1 -- 172.21.15.19:0/4028803892 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5767+0+0 (secure 0 0 0) 0x7f319408adc0 con 0x7f319c153580 2024-03-28T13:19:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.799+0000 7f31a5782740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f5c560 0x561650f2f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.799+0000 7f31a5782740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f5c940 0x561650f5f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.799+0000 7f31a5782740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f2fa40 0x561650f5f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.799+0000 7f31a5782740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561650e9a6b0 con 0x561650f2fa40 2024-03-28T13:19:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.799+0000 7f31a5782740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561650e42750 con 0x561650f5c940 2024-03-28T13:19:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.799+0000 7f31a5782740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x561650e96660 con 0x561650f5c560 2024-03-28T13:19:24.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.799+0000 7f31a0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f2fa40 0x561650f5f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.799+0000 7f319b400640 1 --2- 172.21.15.19:0/4028803892 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f317c06c530 0x7f317c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.799+0000 7f31a0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f2fa40 0x561650f5f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54092/0 (socket says 172.21.15.19:54092) 2024-03-28T13:19:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.799+0000 7f31a0a00640 1 -- 172.21.15.19:0/2257173384 learned_addr learned my addr 172.21.15.19:0/2257173384 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.799+0000 7f319be00640 1 --2- 172.21.15.19:0/2257173384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f5c560 0x561650f2f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.799+0000 7f319b400640 1 --2- 172.21.15.19:0/2257173384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f5c940 0x561650f5f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.800+0000 7f31a0a00640 1 -- 172.21.15.19:0/2257173384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f5c560 msgr2=0x561650f2f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.800+0000 7f31a0a00640 1 --2- 172.21.15.19:0/2257173384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f5c560 0x561650f2f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.800+0000 7f31a0a00640 1 -- 172.21.15.19:0/2257173384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f5c940 msgr2=0x561650f5f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.800+0000 7f31a0a00640 1 --2- 172.21.15.19:0/2257173384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f5c940 0x561650f5f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.800+0000 7f31a0a00640 1 -- 172.21.15.19:0/2257173384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561650c787d0 con 0x561650f2fa40 2024-03-28T13:19:24.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.800+0000 7f319b400640 1 --2- 172.21.15.19:0/2257173384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f5c940 0x561650f5f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:24.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.800+0000 7f31a0a00640 1 --2- 172.21.15.19:0/2257173384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f2fa40 0x561650f5f830 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f3194089180 tx=0x7f3194054fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6634924c7248018 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.800+0000 7f3187e00640 1 -- 172.21.15.19:0/2257173384 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194056f00 con 0x561650f2fa40 2024-03-28T13:19:24.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.800+0000 7f3187e00640 1 -- 172.21.15.19:0/2257173384 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f319400acf0 con 0x561650f2fa40 2024-03-28T13:19:24.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.800+0000 7f3187e00640 1 -- 172.21.15.19:0/2257173384 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194058050 con 0x561650f2fa40 2024-03-28T13:19:24.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.801+0000 7f31a5782740 1 -- 172.21.15.19:0/2257173384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f2fa40 msgr2=0x561650f5f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.801+0000 7f31a5782740 1 --2- 172.21.15.19:0/2257173384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f2fa40 0x561650f5f830 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f3194089180 tx=0x7f3194054fd0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.801+0000 7f319b400640 1 --2- 172.21.15.19:0/4028803892 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f317c06c530 0x7f317c06e9f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x561650f60010 tx=0x7f3188007cd0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.801+0000 7f31a5782740 1 -- 172.21.15.19:0/2257173384 shutdown_connections 2024-03-28T13:19:24.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.801+0000 7f31a5782740 1 --2- 172.21.15.19:0/2257173384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f2fa40 0x561650f5f830 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.801+0000 7f31a5782740 1 --2- 172.21.15.19:0/2257173384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f5c940 0x561650f5f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.801+0000 7f31a5782740 1 --2- 172.21.15.19:0/2257173384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f5c560 0x561650f2f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.801+0000 7f31a5782740 1 -- 172.21.15.19:0/2257173384 >> 172.21.15.19:0/2257173384 conn(0x561650f31770 msgr2=0x561650f7a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:24.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.801+0000 7f31a5782740 1 -- 172.21.15.19:0/2257173384 shutdown_connections 2024-03-28T13:19:24.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.801+0000 7f31a5782740 1 -- 172.21.15.19:0/2257173384 wait complete. 2024-03-28T13:19:24.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.802+0000 7f31a5782740 1 -- start start 2024-03-28T13:19:24.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.802+0000 7f31a5782740 1 Processor -- start 2024-03-28T13:19:24.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.803+0000 7f31a5782740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c560 0x561650f55300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.803+0000 7f31a5782740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f5c940 0x561650f55840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.803+0000 7f31a5782740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f573f0 0x561650f57830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.803+0000 7f31a5782740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561650f33e10 con 0x561650f5c560 2024-03-28T13:19:24.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.803+0000 7f31a5782740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561650e55a80 con 0x561650f5c940 2024-03-28T13:19:24.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.803+0000 7f31a5782740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x561650f36650 con 0x561650f573f0 2024-03-28T13:19:24.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.803+0000 7f31a5782740 10 client.0 _ll_register_callbacks cb 0x561650e55470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:19:24.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.803+0000 7f319be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c560 0x561650f55300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.803+0000 7f319be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c560 0x561650f55300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54096/0 (socket says 172.21.15.19:54096) 2024-03-28T13:19:24.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.803+0000 7f319be00640 1 -- 172.21.15.19:0/1193261509 learned_addr learned my addr 172.21.15.19:0/1193261509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:24.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.804+0000 7f31a0a00640 1 --2- 172.21.15.19:0/1193261509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f573f0 0x561650f57830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.804+0000 7f319be00640 1 -- 172.21.15.19:0/1193261509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f573f0 msgr2=0x561650f57830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.804+0000 7f319be00640 1 --2- 172.21.15.19:0/1193261509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f573f0 0x561650f57830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.804+0000 7f319be00640 1 -- 172.21.15.19:0/1193261509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f5c940 msgr2=0x561650f55840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.804+0000 7f319be00640 1 --2- 172.21.15.19:0/1193261509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f5c940 0x561650f55840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.804+0000 7f319be00640 1 -- 172.21.15.19:0/1193261509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561650bae3f0 con 0x561650f5c560 2024-03-28T13:19:24.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.804+0000 7f319be00640 1 --2- 172.21.15.19:0/1193261509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c560 0x561650f55300 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f318c02b910 tx=0x7f318c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df9205a45932110 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.804+0000 7f3184c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:19:24.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.804+0000 7f3184c00640 1 -- 172.21.15.19:0/1193261509 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c04a060 con 0x561650f5c560 2024-03-28T13:19:24.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.804+0000 7f3184c00640 1 -- 172.21.15.19:0/1193261509 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 5767+0+0 (secure 0 0 0) 0x7f318c031a40 con 0x561650f5c560 2024-03-28T13:19:24.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.804+0000 7f31a5782740 1 -- 172.21.15.19:0/1193261509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f319400acf0 con 0x561650f5c560 2024-03-28T13:19:24.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.805+0000 7f3184c00640 1 -- 172.21.15.19:0/1193261509 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f318c050050 con 0x561650f5c560 2024-03-28T13:19:24.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.805+0000 7f31a5782740 10 client.8349 fetch_fsmap learned FSMap version 83 2024-03-28T13:19:24.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.805+0000 7f31a5782740 1 -- 172.21.15.19:0/1193261509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x561650b6e0f0 con 0x561650f5c560 2024-03-28T13:19:24.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.805+0000 7f3184c00640 1 -- 172.21.15.19:0/1193261509 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f319400acf0 con 0x561650f5c560 2024-03-28T13:19:24.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.805+0000 7f3184c00640 1 -- 172.21.15.19:0/1193261509 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f318c053020 con 0x561650f5c560 2024-03-28T13:19:24.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.805+0000 7f31a5782740 10 client.8349 fetch_fsmap finished waiting for FSMap version 83 2024-03-28T13:19:24.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.805+0000 7f31a5782740 10 client.8349 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-28T13:19:24.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.805+0000 7f31a5782740 20 client.8349 populate_metadata read hostname 'smithi019' 2024-03-28T13:19:24.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.805+0000 7f31a5782740 1 --2- 172.21.15.19:0/1193261509 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x561650f5e450 0x561650f5e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.805+0000 7f31a5782740 4 client.8349 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:19:24.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.805+0000 7f31a5782740 1 -- 172.21.15.19:0/1193261509 --> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561650ea7bf0 con 0x561650f5e450 2024-03-28T13:19:24.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.805+0000 7f319b400640 1 --2- 172.21.15.19:0/1193261509 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x561650f5e450 0x561650f5e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.806+0000 7f319b400640 1 --2- 172.21.15.19:0/1193261509 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x561650f5e450 0x561650f5e890 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a754f608d87830c9 server_cookie=14b0ab5562be9e17 in_seq=0 out_seq=0 2024-03-28T13:19:24.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.806+0000 7f3184c00640 10 client.8349 ms_handle_connect on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:24.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.807+0000 7f3184c00640 1 -- 172.21.15.19:0/1193261509 <== mds.0 v2:172.21.15.19:6832/1020599987 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561650ea7bf0 con 0x561650f5e450 2024-03-28T13:19:24.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.807+0000 7f3184c00640 10 client.8349 handle_command_reply: tid=0 2024-03-28T13:19:24.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.807+0000 7f31a5782740 1 client.8349 shutdown 2024-03-28T13:19:24.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.807+0000 7f31a5782740 1 -- 172.21.15.19:0/1193261509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c560 msgr2=0x561650f55300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.807+0000 7f31a5782740 1 --2- 172.21.15.19:0/1193261509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c560 0x561650f55300 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f318c02b910 tx=0x7f318c004730 comp rx=0 tx=0).stop 2024-03-28T13:19:24.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.807+0000 7f31a5782740 1 -- 172.21.15.19:0/1193261509 shutdown_connections 2024-03-28T13:19:24.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.807+0000 7f31a5782740 1 --2- 172.21.15.19:0/1193261509 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x561650f5e450 0x561650f5e890 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.807+0000 7f31a5782740 1 --2- 172.21.15.19:0/1193261509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f573f0 0x561650f57830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.807+0000 7f31a5782740 1 --2- 172.21.15.19:0/1193261509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f5c940 0x561650f55840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.807+0000 7f31a5782740 1 --2- 172.21.15.19:0/1193261509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c560 0x561650f55300 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.807+0000 7f31a5782740 1 -- 172.21.15.19:0/1193261509 >> 172.21.15.19:0/1193261509 conn(0x561650f2fa40 msgr2=0x561650f334e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:24.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.807+0000 7f3184c00640 0 client.8349 ms_handle_reset on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:24.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.807+0000 7f31a5782740 1 -- 172.21.15.19:0/1193261509 shutdown_connections 2024-03-28T13:19:24.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.808+0000 7f31a5782740 1 -- 172.21.15.19:0/1193261509 wait complete. 2024-03-28T13:19:24.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.808+0000 7f31a5782740 20 client.8349 trim_cache size 0 max 16384 2024-03-28T13:19:24.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.827+0000 7f31a5782740 1 Processor -- start 2024-03-28T13:19:24.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.827+0000 7f31a5782740 1 -- start start 2024-03-28T13:19:24.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.828+0000 7f31a5782740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f573f0 0x561650f5e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.828+0000 7f31a5782740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f5c560 0x561650f2e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.828+0000 7f31a5782740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c940 0x561650f2e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.828+0000 7f31a5782740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f319c200c20 con 0x561650f5c940 2024-03-28T13:19:24.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.828+0000 7f31a5782740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1ffc00 con 0x561650f573f0 2024-03-28T13:19:24.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.828+0000 7f31a5782740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f319c1f4fd0 con 0x561650f5c560 2024-03-28T13:19:24.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.828+0000 7f31a0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c940 0x561650f2e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.828+0000 7f31a0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c940 0x561650f2e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54104/0 (socket says 172.21.15.19:54104) 2024-03-28T13:19:24.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.828+0000 7f31a0a00640 1 -- 172.21.15.19:0/462291436 learned_addr learned my addr 172.21.15.19:0/462291436 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:24.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.829+0000 7f319be00640 1 --2- 172.21.15.19:0/462291436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f573f0 0x561650f5e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.829+0000 7f319b400640 1 --2- 172.21.15.19:0/462291436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f5c560 0x561650f2e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.829+0000 7f31a0a00640 1 -- 172.21.15.19:0/462291436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f5c560 msgr2=0x561650f2e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.829+0000 7f31a0a00640 1 --2- 172.21.15.19:0/462291436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f5c560 0x561650f2e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.829+0000 7f31a0a00640 1 -- 172.21.15.19:0/462291436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f573f0 msgr2=0x561650f5e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.829+0000 7f31a0a00640 1 --2- 172.21.15.19:0/462291436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f573f0 0x561650f5e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.829+0000 7f31a0a00640 1 -- 172.21.15.19:0/462291436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561650f57ba0 con 0x561650f5c940 2024-03-28T13:19:24.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.829+0000 7f31a0a00640 1 --2- 172.21.15.19:0/462291436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c940 0x561650f2e8d0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3194058b90 tx=0x7f319405b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d069e328707df1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.829+0000 7f3187400640 1 -- 172.21.15.19:0/462291436 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31940617e0 con 0x561650f5c940 2024-03-28T13:19:24.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.829+0000 7f3187400640 1 -- 172.21.15.19:0/462291436 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3194086020 con 0x561650f5c940 2024-03-28T13:19:24.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.829+0000 7f3187400640 1 -- 172.21.15.19:0/462291436 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194056f00 con 0x561650f5c940 2024-03-28T13:19:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.830+0000 7f31a5782740 1 -- 172.21.15.19:0/462291436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c940 msgr2=0x561650f2e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.830+0000 7f31a5782740 1 --2- 172.21.15.19:0/462291436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c940 0x561650f2e8d0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3194058b90 tx=0x7f319405b000 comp rx=0 tx=0).stop 2024-03-28T13:19:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.830+0000 7f31a5782740 1 -- 172.21.15.19:0/462291436 shutdown_connections 2024-03-28T13:19:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.830+0000 7f31a5782740 1 --2- 172.21.15.19:0/462291436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c940 0x561650f2e8d0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.830+0000 7f31a5782740 1 --2- 172.21.15.19:0/462291436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f5c560 0x561650f2e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.830+0000 7f31a5782740 1 --2- 172.21.15.19:0/462291436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f573f0 0x561650f5e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.830+0000 7f31a5782740 1 -- 172.21.15.19:0/462291436 >> 172.21.15.19:0/462291436 conn(0x561650f2fa40 msgr2=0x561650f2ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.830+0000 7f31a5782740 1 -- 172.21.15.19:0/462291436 shutdown_connections 2024-03-28T13:19:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.830+0000 7f31a5782740 1 -- 172.21.15.19:0/462291436 wait complete. 2024-03-28T13:19:24.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.831+0000 7f31a5782740 1 -- start start 2024-03-28T13:19:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.831+0000 7f31a5782740 1 Processor -- start 2024-03-28T13:19:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.832+0000 7f31a5782740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f573f0 0x561650f63160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.832+0000 7f31a5782740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f5c560 0x561650f55530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.832+0000 7f31a5782740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c940 0x561650f2e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.832+0000 7f31a5782740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1558c0 con 0x561650f5c940 2024-03-28T13:19:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.832+0000 7f31a5782740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561650f36910 con 0x561650f573f0 2024-03-28T13:19:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.832+0000 7f31a5782740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x561650f8f960 con 0x561650f5c560 2024-03-28T13:19:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.832+0000 7f31a5782740 10 client.0 _ll_register_callbacks cb 0x7f318c0040f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:19:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.832+0000 7f31a0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c940 0x561650f2e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.832+0000 7f31a0a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c940 0x561650f2e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54106/0 (socket says 172.21.15.19:54106) 2024-03-28T13:19:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.832+0000 7f31a0a00640 1 -- 172.21.15.19:0/1864681687 learned_addr learned my addr 172.21.15.19:0/1864681687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.832+0000 7f319be00640 1 --2- 172.21.15.19:0/1864681687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f573f0 0x561650f63160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.833+0000 7f319b400640 1 --2- 172.21.15.19:0/1864681687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f5c560 0x561650f55530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.833+0000 7f31a0a00640 1 -- 172.21.15.19:0/1864681687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f5c560 msgr2=0x561650f55530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.833+0000 7f31a0a00640 1 --2- 172.21.15.19:0/1864681687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f5c560 0x561650f55530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.833+0000 7f31a0a00640 1 -- 172.21.15.19:0/1864681687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f573f0 msgr2=0x561650f63160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.833+0000 7f31a0a00640 1 --2- 172.21.15.19:0/1864681687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f573f0 0x561650f63160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.833+0000 7f31a0a00640 1 -- 172.21.15.19:0/1864681687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561650f437a0 con 0x561650f5c940 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.833+0000 7f319be00640 1 --2- 172.21.15.19:0/1864681687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f573f0 0x561650f63160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.833+0000 7f31a0a00640 1 --2- 172.21.15.19:0/1864681687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c940 0x561650f2e390 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f3194058b90 tx=0x7f3194001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92a06358756fdae server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.833+0000 7f3184c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.833+0000 7f3184c00640 1 -- 172.21.15.19:0/1864681687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194082b70 con 0x561650f5c940 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.833+0000 7f3184c00640 1 -- 172.21.15.19:0/1864681687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 5767+0+0 (secure 0 0 0) 0x7f3194082d10 con 0x561650f5c940 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.833+0000 7f31a5782740 1 -- 172.21.15.19:0/1864681687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3194086020 con 0x561650f5c940 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.834+0000 7f3184c00640 1 -- 172.21.15.19:0/1864681687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f3194058050 con 0x561650f5c940 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.834+0000 7f31a5782740 10 client.8355 fetch_fsmap learned FSMap version 83 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.834+0000 7f31a5782740 1 -- 172.21.15.19:0/1864681687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x561650f8f6a0 con 0x561650f5c940 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.834+0000 7f3184c00640 1 -- 172.21.15.19:0/1864681687 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3194086020 con 0x561650f5c940 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.834+0000 7f3184c00640 1 -- 172.21.15.19:0/1864681687 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7f31940581f0 con 0x561650f5c940 2024-03-28T13:19:24.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.834+0000 7f31a5782740 10 client.8355 fetch_fsmap finished waiting for FSMap version 83 2024-03-28T13:19:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.834+0000 7f31a5782740 10 client.8355 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-28T13:19:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.834+0000 7f31a5782740 20 client.8355 populate_metadata read hostname 'smithi019' 2024-03-28T13:19:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.834+0000 7f31a5782740 1 --2- 172.21.15.19:0/1864681687 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x561650f9cf30 0x561650f2e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.834+0000 7f31a5782740 4 client.8355 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:19:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.834+0000 7f31a5782740 1 -- 172.21.15.19:0/1864681687 --> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561650ea1f40 con 0x561650f9cf30 2024-03-28T13:19:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.834+0000 7f319be00640 1 --2- 172.21.15.19:0/1864681687 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x561650f9cf30 0x561650f2e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.835+0000 7f319be00640 1 --2- 172.21.15.19:0/1864681687 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x561650f9cf30 0x561650f2e8d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf22bb5605ea3bc1 server_cookie=86fb34e790b2176b in_seq=0 out_seq=0 2024-03-28T13:19:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.835+0000 7f3184c00640 10 client.8355 ms_handle_connect on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.835+0000 7f3184c00640 1 -- 172.21.15.19:0/1864681687 <== mds.0 v2:172.21.15.19:6832/1020599987 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x561650ea1f40 con 0x561650f9cf30 2024-03-28T13:19:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.835+0000 7f3184c00640 10 client.8355 handle_command_reply: tid=0 2024-03-28T13:19:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.835+0000 7f31a5782740 1 client.8355 shutdown 2024-03-28T13:19:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.835+0000 7f31a5782740 1 -- 172.21.15.19:0/1864681687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c940 msgr2=0x561650f2e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.835+0000 7f31a5782740 1 --2- 172.21.15.19:0/1864681687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c940 0x561650f2e390 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f3194058b90 tx=0x7f3194001250 comp rx=0 tx=0).stop 2024-03-28T13:19:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.836+0000 7f31a5782740 1 -- 172.21.15.19:0/1864681687 shutdown_connections 2024-03-28T13:19:24.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.836+0000 7f31a5782740 1 --2- 172.21.15.19:0/1864681687 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x561650f9cf30 0x561650f2e8d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.836+0000 7f31a5782740 1 --2- 172.21.15.19:0/1864681687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561650f5c940 0x561650f2e390 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.836+0000 7f31a5782740 1 --2- 172.21.15.19:0/1864681687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561650f5c560 0x561650f55530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.836+0000 7f31a5782740 1 --2- 172.21.15.19:0/1864681687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561650f573f0 0x561650f63160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.836+0000 7f3184c00640 0 client.8355 ms_handle_reset on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:24.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.836+0000 7f31a5782740 1 -- 172.21.15.19:0/1864681687 >> 172.21.15.19:0/1864681687 conn(0x561650f2fa40 msgr2=0x561650f8ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:24.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.836+0000 7f31a5782740 1 -- 172.21.15.19:0/1864681687 shutdown_connections 2024-03-28T13:19:24.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.836+0000 7f31a5782740 1 -- 172.21.15.19:0/1864681687 wait complete. 2024-03-28T13:19:24.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.836+0000 7f31a5782740 20 client.8355 trim_cache size 0 max 16384 2024-03-28T13:19:24.841 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:19:24.841 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T13:19:24.841 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-03-28T13:19:24.841 INFO:teuthology.orchestra.run.smithi019.stdout: "07478f43-9ddb-433e-85a1-a9a331de9b22": { 2024-03-28T13:19:24.842 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-03-28T13:19:24.842 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "07478f43-9ddb-433e-85a1-a9a331de9b22", 2024-03-28T13:19:24.842 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,repair" 2024-03-28T13:19:24.842 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:19:24.842 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:19:24.842 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:19:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.837+0000 7f31a2a00640 1 -- 172.21.15.19:0/4028803892 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f317c06c530 msgr2=0x7f317c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.837+0000 7f31a2a00640 1 --2- 172.21.15.19:0/4028803892 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f317c06c530 0x7f317c06e9f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x561650f60010 tx=0x7f3188007cd0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.837+0000 7f31a2a00640 1 -- 172.21.15.19:0/4028803892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f319c153580 msgr2=0x7f319c14a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.837+0000 7f31a2a00640 1 --2- 172.21.15.19:0/4028803892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f319c153580 0x7f319c14a2c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f319400c700 tx=0x7f319400cbc0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.837+0000 7f31a2a00640 1 -- 172.21.15.19:0/4028803892 shutdown_connections 2024-03-28T13:19:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.837+0000 7f31a2a00640 1 --2- 172.21.15.19:0/4028803892 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f317c06c530 0x7f317c06e9f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.837+0000 7f31a2a00640 1 --2- 172.21.15.19:0/4028803892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f319c153580 0x7f319c14a2c0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.837+0000 7f31a2a00640 1 --2- 172.21.15.19:0/4028803892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f319c0d07c0 0x7f319c073280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.837+0000 7f31a2a00640 1 --2- 172.21.15.19:0/4028803892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f319c0af4b0 0x7f319c071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.837+0000 7f31a2a00640 1 -- 172.21.15.19:0/4028803892 >> 172.21.15.19:0/4028803892 conn(0x7f319c06b750 msgr2=0x7f319c15a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.837+0000 7f31a2a00640 1 -- 172.21.15.19:0/4028803892 shutdown_connections 2024-03-28T13:19:24.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:24.837+0000 7f31a2a00640 1 -- 172.21.15.19:0/4028803892 wait complete. 2024-03-28T13:19:24.853 INFO:tasks.cephfs.filesystem:scrub status for tag:07478f43-9ddb-433e-85a1-a9a331de9b22 - {'path': '/', 'tag': '07478f43-9ddb-433e-85a1-a9a331de9b22', 'options': 'recursive,repair'} 2024-03-28T13:19:25.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:25.349+0000 7f4dd2a00640 1 -- 172.21.15.19:0/555064690 <== mon.0 v2:172.21.15.19:3300/0 8 ==== log(6 entries from seq 1017 at 2024-03-28T13:19:24.345455+0000) ==== 1216+0+0 (secure 0 0 0) 0x7f4dc8079000 con 0x7f4ddc15f490 2024-03-28T13:19:25.350 INFO:teuthology.orchestra.run.smithi019.stdout: cluster: 2024-03-28T13:19:25.350 INFO:teuthology.orchestra.run.smithi019.stdout: id: e6b569d9-f2ce-49dc-bbe7-c0ab33080203 2024-03-28T13:19:25.350 INFO:teuthology.orchestra.run.smithi019.stdout: health: HEALTH_OK 2024-03-28T13:19:25.350 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:25.350 INFO:teuthology.orchestra.run.smithi019.stdout: services: 2024-03-28T13:19:25.350 INFO:teuthology.orchestra.run.smithi019.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-03-28T13:19:25.350 INFO:teuthology.orchestra.run.smithi019.stdout: mgr: x(active, since 10m), standbys: y 2024-03-28T13:19:25.351 INFO:teuthology.orchestra.run.smithi019.stdout: mds: 1/1 daemons up, 3 standby 2024-03-28T13:19:25.351 INFO:teuthology.orchestra.run.smithi019.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 10m) 2024-03-28T13:19:25.351 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:25.351 INFO:teuthology.orchestra.run.smithi019.stdout: data: 2024-03-28T13:19:25.351 INFO:teuthology.orchestra.run.smithi019.stdout: volumes: 1/1 healthy 2024-03-28T13:19:25.351 INFO:teuthology.orchestra.run.smithi019.stdout: pools: 3 pools, 97 pgs 2024-03-28T13:19:25.351 INFO:teuthology.orchestra.run.smithi019.stdout: objects: 30 objects, 19 MiB 2024-03-28T13:19:25.352 INFO:teuthology.orchestra.run.smithi019.stdout: usage: 289 MiB used, 720 GiB / 720 GiB avail 2024-03-28T13:19:25.352 INFO:teuthology.orchestra.run.smithi019.stdout: pgs: 97 active+clean 2024-03-28T13:19:25.352 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:25.352 INFO:teuthology.orchestra.run.smithi019.stdout: io: 2024-03-28T13:19:25.352 INFO:teuthology.orchestra.run.smithi019.stdout: client: 85 B/s rd, 1.5 MiB/s wr, 0 op/s rd, 3 op/s wr 2024-03-28T13:19:25.352 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:25.352 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:19:24.572323+0000 mds.c [INF] scrub queued for path: / 2024-03-28T13:19:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:19:24.572327+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-03-28T13:19:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:19:24.572333+0000 mds.c [INF] scrub summary: active paths [/] 2024-03-28T13:19:25.352 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:19:24.582328+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-03-28T13:19:29.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:29.378+0000 7f4dd2a00640 1 -- 172.21.15.19:0/555064690 <== mon.0 v2:172.21.15.19:3300/0 9 ==== log(1 entries from seq 5 at 2024-03-28T13:19:28.696259+0000) ==== 189+0+0 (secure 0 0 0) 0x7f4dc80ab300 con 0x7f4ddc15f490 2024-03-28T13:19:29.378 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:19:28.696259+0000 mds.c [INF] scrub summary: idle 2024-03-28T13:19:34.009 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:19:34.017 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:19:54.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:19:54.854 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-28T13:19:55.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.078+0000 7ffba9000640 1 Processor -- start 2024-03-28T13:19:55.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.079+0000 7ffba9000640 1 -- start start 2024-03-28T13:19:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.080+0000 7ffba9000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40d43d0 0x7ffba40d47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.080+0000 7ffba9000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffba40cf6b0 0x7ffba40cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.080+0000 7ffba9000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba406dfa0 con 0x7ffba415e860 2024-03-28T13:19:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.080+0000 7ffba9000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ffba40cf220 con 0x7ffba40cf6b0 2024-03-28T13:19:55.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.080+0000 7ffba9000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffba40d4cf0 con 0x7ffba40d43d0 2024-03-28T13:19:55.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.081+0000 7ffb9be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40d43d0 0x7ffba40d47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.081+0000 7ffb9be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40d43d0 0x7ffba40d47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:60626/0 (socket says 172.21.15.19:60626) 2024-03-28T13:19:55.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.081+0000 7ffb9be00640 1 -- 172.21.15.19:0/521505334 learned_addr learned my addr 172.21.15.19:0/521505334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:55.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.081+0000 7ffba2800640 1 --2- 172.21.15.19:0/521505334 >> v2:172.21.15.188:3300/0 conn(0x7ffba40cf6b0 0x7ffba40cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.082+0000 7ffb9be00640 1 -- 172.21.15.19:0/521505334 >> v2:172.21.15.188:3300/0 conn(0x7ffba40cf6b0 msgr2=0x7ffba40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:55.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.082+0000 7ffb9be00640 1 --2- 172.21.15.19:0/521505334 >> v2:172.21.15.188:3300/0 conn(0x7ffba40cf6b0 0x7ffba40cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.082+0000 7ffb9be00640 1 -- 172.21.15.19:0/521505334 >> v1:172.21.15.188:6789/0 conn(0x7ffba415e860 legacy=0x7ffba406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.082+0000 7ffb9be00640 1 -- 172.21.15.19:0/521505334 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba406e240 con 0x7ffba40d43d0 2024-03-28T13:19:55.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.084+0000 7ffb9be00640 1 --2- 172.21.15.19:0/521505334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40d43d0 0x7ffba40d47b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffba406e170 tx=0x7ffb8c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa8fe4806f888aca server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.085+0000 7ffba1400640 1 -- 172.21.15.19:0/521505334 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c034070 con 0x7ffba40d43d0 2024-03-28T13:19:55.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.085+0000 7ffba1400640 1 -- 172.21.15.19:0/521505334 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8c030840 con 0x7ffba40d43d0 2024-03-28T13:19:55.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.085+0000 7ffba1400640 1 -- 172.21.15.19:0/521505334 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c030c10 con 0x7ffba40d43d0 2024-03-28T13:19:55.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.086+0000 7ffba9000640 1 -- 172.21.15.19:0/521505334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40d43d0 msgr2=0x7ffba40d47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.086+0000 7ffba9000640 1 --2- 172.21.15.19:0/521505334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40d43d0 0x7ffba40d47b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffba406e170 tx=0x7ffb8c02b980 comp rx=0 tx=0).stop 2024-03-28T13:19:55.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.086+0000 7ffba9000640 1 -- 172.21.15.19:0/521505334 shutdown_connections 2024-03-28T13:19:55.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.086+0000 7ffba9000640 1 --2- 172.21.15.19:0/521505334 >> v2:172.21.15.188:3300/0 conn(0x7ffba40cf6b0 0x7ffba40cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.086+0000 7ffba9000640 1 --2- 172.21.15.19:0/521505334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba40d43d0 0x7ffba40d47b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.086+0000 7ffba9000640 1 -- 172.21.15.19:0/521505334 >> 172.21.15.19:0/521505334 conn(0x7ffba409d7d0 msgr2=0x7ffba408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:55.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.086+0000 7ffba9000640 1 -- 172.21.15.19:0/521505334 shutdown_connections 2024-03-28T13:19:55.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.086+0000 7ffba9000640 1 -- 172.21.15.19:0/521505334 wait complete. 2024-03-28T13:19:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.089+0000 7ffba9000640 1 Processor -- start 2024-03-28T13:19:55.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.089+0000 7ffba9000640 1 -- start start 2024-03-28T13:19:55.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.090+0000 7ffba9000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40cf6b0 0x7ffba40b96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.090+0000 7ffba9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba40d43d0 0x7ffba40b9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.090+0000 7ffba9000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba415e860 0x7ffba40cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.090+0000 7ffba9000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffba406dc00 con 0x7ffba40d43d0 2024-03-28T13:19:55.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.090+0000 7ffba9000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffba40d2c50 con 0x7ffba40cf6b0 2024-03-28T13:19:55.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.090+0000 7ffba9000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffba406dd80 con 0x7ffba415e860 2024-03-28T13:19:55.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.090+0000 7ffb9be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba40d43d0 0x7ffba40b9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.090+0000 7ffb9be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba40d43d0 0x7ffba40b9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44684/0 (socket says 172.21.15.19:44684) 2024-03-28T13:19:55.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.090+0000 7ffb9be00640 1 -- 172.21.15.19:0/1198560808 learned_addr learned my addr 172.21.15.19:0/1198560808 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:55.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.091+0000 7ffba1e00640 1 --2- 172.21.15.19:0/1198560808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40cf6b0 0x7ffba40b96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.091+0000 7ffba2800640 1 --2- 172.21.15.19:0/1198560808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba415e860 0x7ffba40cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.091+0000 7ffb9be00640 1 -- 172.21.15.19:0/1198560808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba415e860 msgr2=0x7ffba40cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.091+0000 7ffb9be00640 1 --2- 172.21.15.19:0/1198560808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba415e860 0x7ffba40cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.091+0000 7ffb9be00640 1 -- 172.21.15.19:0/1198560808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40cf6b0 msgr2=0x7ffba40b96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.091+0000 7ffb9be00640 1 --2- 172.21.15.19:0/1198560808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40cf6b0 0x7ffba40b96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.091+0000 7ffb9be00640 1 -- 172.21.15.19:0/1198560808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba40cd1c0 con 0x7ffba40d43d0 2024-03-28T13:19:55.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.091+0000 7ffb9be00640 1 --2- 172.21.15.19:0/1198560808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba40d43d0 0x7ffba40b9c30 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c02b9c0 tx=0x7ffb8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.092+0000 7ffb9a000640 1 -- 172.21.15.19:0/1198560808 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c048070 con 0x7ffba40d43d0 2024-03-28T13:19:55.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.092+0000 7ffb9a000640 1 -- 172.21.15.19:0/1198560808 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8c030640 con 0x7ffba40d43d0 2024-03-28T13:19:55.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.092+0000 7ffb9a000640 1 -- 172.21.15.19:0/1198560808 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c034040 con 0x7ffba40d43d0 2024-03-28T13:19:55.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.092+0000 7ffba9000640 1 -- 172.21.15.19:0/1198560808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba40cc120 con 0x7ffba40d43d0 2024-03-28T13:19:55.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.092+0000 7ffba9000640 1 -- 172.21.15.19:0/1198560808 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba40cc6e0 con 0x7ffba40d43d0 2024-03-28T13:19:55.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.095+0000 7ffb9a000640 1 -- 172.21.15.19:0/1198560808 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ffb8c037070 con 0x7ffba40d43d0 2024-03-28T13:19:55.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.097+0000 7ffbabe11740 1 Processor -- start 2024-03-28T13:19:55.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.097+0000 7ffb9a000640 1 --2- 172.21.15.19:0/1198560808 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8006c560 0x7ffb8006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.097+0000 7ffb9a000640 1 -- 172.21.15.19:0/1198560808 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7ffb8c0ad470 con 0x7ffba40d43d0 2024-03-28T13:19:55.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.097+0000 7ffbabe11740 1 -- start start 2024-03-28T13:19:55.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.098+0000 7ffba1e00640 1 --2- 172.21.15.19:0/1198560808 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8006c560 0x7ffb8006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.098+0000 7ffbabe11740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917d8560 0x5645917ab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.098+0000 7ffbabe11740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917d8940 0x5645917db2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.098+0000 7ffbabe11740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917aba40 0x5645917db830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.098+0000 7ffbabe11740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5645917166b0 con 0x5645917d8940 2024-03-28T13:19:55.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.098+0000 7ffbabe11740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5645916be750 con 0x5645917d8560 2024-03-28T13:19:55.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.098+0000 7ffbabe11740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x564591712660 con 0x5645917aba40 2024-03-28T13:19:55.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.099+0000 7ffb9be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917d8940 0x5645917db2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.099+0000 7ffb9be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917d8940 0x5645917db2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44696/0 (socket says 172.21.15.19:44696) 2024-03-28T13:19:55.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.099+0000 7ffb9be00640 1 -- 172.21.15.19:0/1408888066 learned_addr learned my addr 172.21.15.19:0/1408888066 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:55.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.099+0000 7ffba1e00640 1 --2- 172.21.15.19:0/1198560808 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8006c560 0x7ffb8006ea20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ffb94007fd0 tx=0x7ffb94009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.099+0000 7ffba1e00640 1 --2- 172.21.15.19:0/1408888066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917d8560 0x5645917ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.099+0000 7ffba2800640 1 --2- 172.21.15.19:0/1408888066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917aba40 0x5645917db830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.099+0000 7ffb9be00640 1 -- 172.21.15.19:0/1408888066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917aba40 msgr2=0x5645917db830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.099+0000 7ffb9be00640 1 --2- 172.21.15.19:0/1408888066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917aba40 0x5645917db830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.099+0000 7ffb9be00640 1 -- 172.21.15.19:0/1408888066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917d8560 msgr2=0x5645917ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.099+0000 7ffb9be00640 1 --2- 172.21.15.19:0/1408888066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917d8560 0x5645917ab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.099+0000 7ffb9be00640 1 -- 172.21.15.19:0/1408888066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645914f47d0 con 0x5645917d8940 2024-03-28T13:19:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.099+0000 7ffba1e00640 1 --2- 172.21.15.19:0/1408888066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917d8560 0x5645917ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.100+0000 7ffb9be00640 1 --2- 172.21.15.19:0/1408888066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917d8940 0x5645917db2f0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0ab800 tx=0x7ffb8c0774c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3f5bd61452d567 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.100+0000 7ffb92a00640 1 -- 172.21.15.19:0/1408888066 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c083e90 con 0x5645917d8940 2024-03-28T13:19:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.100+0000 7ffb92a00640 1 -- 172.21.15.19:0/1408888066 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8c0792c0 con 0x5645917d8940 2024-03-28T13:19:55.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.100+0000 7ffb92a00640 1 -- 172.21.15.19:0/1408888066 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8c077890 con 0x5645917d8940 2024-03-28T13:19:55.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.101+0000 7ffbabe11740 1 -- 172.21.15.19:0/1408888066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917d8940 msgr2=0x5645917db2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.101+0000 7ffbabe11740 1 --2- 172.21.15.19:0/1408888066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917d8940 0x5645917db2f0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c0ab800 tx=0x7ffb8c0774c0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.101+0000 7ffbabe11740 1 -- 172.21.15.19:0/1408888066 shutdown_connections 2024-03-28T13:19:55.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.101+0000 7ffbabe11740 1 --2- 172.21.15.19:0/1408888066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917aba40 0x5645917db830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.101+0000 7ffbabe11740 1 --2- 172.21.15.19:0/1408888066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917d8940 0x5645917db2f0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.101+0000 7ffbabe11740 1 --2- 172.21.15.19:0/1408888066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917d8560 0x5645917ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.101+0000 7ffbabe11740 1 -- 172.21.15.19:0/1408888066 >> 172.21.15.19:0/1408888066 conn(0x5645917ad770 msgr2=0x5645917f62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:55.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.101+0000 7ffbabe11740 1 -- 172.21.15.19:0/1408888066 shutdown_connections 2024-03-28T13:19:55.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.101+0000 7ffbabe11740 1 -- 172.21.15.19:0/1408888066 wait complete. 2024-03-28T13:19:55.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.102+0000 7ffbabe11740 1 -- start start 2024-03-28T13:19:55.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.102+0000 7ffbabe11740 1 Processor -- start 2024-03-28T13:19:55.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.103+0000 7ffbabe11740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917d8560 0x5645917da910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.103+0000 7ffbabe11740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917f6130 0x5645917f6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.103+0000 7ffbabe11740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f4030 0x5645917f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.103+0000 7ffbabe11740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5645917166b0 con 0x5645917f4030 2024-03-28T13:19:55.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.103+0000 7ffbabe11740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5645916d1a80 con 0x5645917f6130 2024-03-28T13:19:55.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.103+0000 7ffbabe11740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5645917afe10 con 0x5645917d8560 2024-03-28T13:19:55.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.103+0000 7ffbabe11740 10 client.0 _ll_register_callbacks cb 0x5645916d1470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:19:55.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.104+0000 7ffba2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f4030 0x5645917f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.104+0000 7ffba2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f4030 0x5645917f44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44704/0 (socket says 172.21.15.19:44704) 2024-03-28T13:19:55.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.104+0000 7ffba2800640 1 -- 172.21.15.19:0/1674717864 learned_addr learned my addr 172.21.15.19:0/1674717864 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:55.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.104+0000 7ffba1e00640 1 --2- 172.21.15.19:0/1674717864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917d8560 0x5645917da910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.104+0000 7ffb9be00640 1 --2- 172.21.15.19:0/1674717864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917f6130 0x5645917f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.104+0000 7ffba2800640 1 -- 172.21.15.19:0/1674717864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917d8560 msgr2=0x5645917da910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:55.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.104+0000 7ffba2800640 1 --2- 172.21.15.19:0/1674717864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917d8560 0x5645917da910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.104+0000 7ffba2800640 1 -- 172.21.15.19:0/1674717864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917f6130 msgr2=0x5645917f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:55.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.104+0000 7ffba2800640 1 --2- 172.21.15.19:0/1674717864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917f6130 0x5645917f6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.104+0000 7ffba2800640 1 -- 172.21.15.19:0/1674717864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56459142a3f0 con 0x5645917f4030 2024-03-28T13:19:55.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.104+0000 7ffba2800640 1 --2- 172.21.15.19:0/1674717864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f4030 0x5645917f44b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c007dd0 tx=0x7ffb9c012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790d0c93ab372332 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.104+0000 7ffb87400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:19:55.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.105+0000 7ffb87400640 1 -- 172.21.15.19:0/1674717864 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c01c020 con 0x5645917f4030 2024-03-28T13:19:55.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.105+0000 7ffb87400640 1 -- 172.21.15.19:0/1674717864 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7ffb9c018670 con 0x5645917f4030 2024-03-28T13:19:55.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.105+0000 7ffbabe11740 1 -- 172.21.15.19:0/1674717864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb8c0792c0 con 0x5645917f4030 2024-03-28T13:19:55.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.105+0000 7ffb87400640 1 -- 172.21.15.19:0/1674717864 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7ffb9c026050 con 0x5645917f4030 2024-03-28T13:19:55.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.105+0000 7ffbabe11740 10 client.8364 fetch_fsmap learned FSMap version 83 2024-03-28T13:19:55.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.105+0000 7ffbabe11740 1 -- 172.21.15.19:0/1674717864 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x5645913ea0f0 con 0x5645917f4030 2024-03-28T13:19:55.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.105+0000 7ffb87400640 1 -- 172.21.15.19:0/1674717864 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb8c0792c0 con 0x5645917f4030 2024-03-28T13:19:55.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.105+0000 7ffb87400640 1 -- 172.21.15.19:0/1674717864 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7ffb9c029020 con 0x5645917f4030 2024-03-28T13:19:55.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.106+0000 7ffbabe11740 10 client.8364 fetch_fsmap finished waiting for FSMap version 83 2024-03-28T13:19:55.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.106+0000 7ffbabe11740 10 client.8364 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-28T13:19:55.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.106+0000 7ffbabe11740 20 client.8364 populate_metadata read hostname 'smithi019' 2024-03-28T13:19:55.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.106+0000 7ffbabe11740 1 --2- 172.21.15.19:0/1674717864 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5645917c12b0 0x5645917c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.106+0000 7ffbabe11740 4 client.8364 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:19:55.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.106+0000 7ffbabe11740 1 -- 172.21.15.19:0/1674717864 --> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564591723bf0 con 0x5645917c12b0 2024-03-28T13:19:55.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.106+0000 7ffba1e00640 1 --2- 172.21.15.19:0/1674717864 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5645917c12b0 0x5645917c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.106+0000 7ffba1e00640 1 --2- 172.21.15.19:0/1674717864 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5645917c12b0 0x5645917c16c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b30095980abacabd server_cookie=7349be1e8f9c4ed1 in_seq=0 out_seq=0 2024-03-28T13:19:55.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.106+0000 7ffb87400640 10 client.8364 ms_handle_connect on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:55.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.107+0000 7ffb87400640 1 -- 172.21.15.19:0/1674717864 <== mds.0 v2:172.21.15.19:6832/1020599987 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564591723bf0 con 0x5645917c12b0 2024-03-28T13:19:55.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.107+0000 7ffb87400640 10 client.8364 handle_command_reply: tid=0 2024-03-28T13:19:55.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.107+0000 7ffbabe11740 1 client.8364 shutdown 2024-03-28T13:19:55.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.107+0000 7ffbabe11740 1 -- 172.21.15.19:0/1674717864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f4030 msgr2=0x5645917f44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.107+0000 7ffbabe11740 1 --2- 172.21.15.19:0/1674717864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f4030 0x5645917f44b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c007dd0 tx=0x7ffb9c012610 comp rx=0 tx=0).stop 2024-03-28T13:19:55.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.108+0000 7ffbabe11740 1 -- 172.21.15.19:0/1674717864 shutdown_connections 2024-03-28T13:19:55.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.108+0000 7ffbabe11740 1 --2- 172.21.15.19:0/1674717864 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x5645917c12b0 0x5645917c16c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.108+0000 7ffbabe11740 1 --2- 172.21.15.19:0/1674717864 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f4030 0x5645917f44b0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.108+0000 7ffbabe11740 1 --2- 172.21.15.19:0/1674717864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917f6130 0x5645917f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.108+0000 7ffbabe11740 1 --2- 172.21.15.19:0/1674717864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917d8560 0x5645917da910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.108+0000 7ffbabe11740 1 -- 172.21.15.19:0/1674717864 >> 172.21.15.19:0/1674717864 conn(0x5645917aba40 msgr2=0x5645917f7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:55.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.108+0000 7ffb87400640 0 client.8364 ms_handle_reset on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:55.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.108+0000 7ffbabe11740 1 -- 172.21.15.19:0/1674717864 shutdown_connections 2024-03-28T13:19:55.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.108+0000 7ffbabe11740 1 -- 172.21.15.19:0/1674717864 wait complete. 2024-03-28T13:19:55.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.108+0000 7ffbabe11740 20 client.8364 trim_cache size 0 max 16384 2024-03-28T13:19:55.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.127+0000 7ffbabe11740 1 Processor -- start 2024-03-28T13:19:55.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.128+0000 7ffbabe11740 1 -- start start 2024-03-28T13:19:55.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.128+0000 7ffbabe11740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917d8560 0x5645917c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.128+0000 7ffbabe11740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917f4030 0x564591819f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.128+0000 7ffbabe11740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f6130 0x56459181a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.128+0000 7ffbabe11740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffba4200d00 con 0x5645917f6130 2024-03-28T13:19:55.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.128+0000 7ffbabe11740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffba41ffd40 con 0x5645917d8560 2024-03-28T13:19:55.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.128+0000 7ffbabe11740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffba40b7d30 con 0x5645917f4030 2024-03-28T13:19:55.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.129+0000 7ffba2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f6130 0x56459181a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.129+0000 7ffba2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f6130 0x56459181a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44716/0 (socket says 172.21.15.19:44716) 2024-03-28T13:19:55.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.129+0000 7ffba2800640 1 -- 172.21.15.19:0/1936685142 learned_addr learned my addr 172.21.15.19:0/1936685142 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:55.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.129+0000 7ffb9be00640 1 --2- 172.21.15.19:0/1936685142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917f4030 0x564591819f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.129+0000 7ffba1e00640 1 --2- 172.21.15.19:0/1936685142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917d8560 0x5645917c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.129+0000 7ffba2800640 1 -- 172.21.15.19:0/1936685142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917f4030 msgr2=0x564591819f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:55.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.129+0000 7ffba2800640 1 --2- 172.21.15.19:0/1936685142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917f4030 0x564591819f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.129+0000 7ffba2800640 1 -- 172.21.15.19:0/1936685142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917d8560 msgr2=0x5645917c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:55.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.129+0000 7ffba2800640 1 --2- 172.21.15.19:0/1936685142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917d8560 0x5645917c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.129+0000 7ffba2800640 1 -- 172.21.15.19:0/1936685142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645917f93e0 con 0x5645917f6130 2024-03-28T13:19:55.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.129+0000 7ffba1e00640 1 --2- 172.21.15.19:0/1936685142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917d8560 0x5645917c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:55.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.130+0000 7ffba2800640 1 --2- 172.21.15.19:0/1936685142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f6130 0x56459181a450 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00b010 tx=0x7ffb9c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf254d2ab779d02 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.130+0000 7ffb92000640 1 -- 172.21.15.19:0/1936685142 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c02f070 con 0x5645917f6130 2024-03-28T13:19:55.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.130+0000 7ffb92000640 1 -- 172.21.15.19:0/1936685142 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c016890 con 0x5645917f6130 2024-03-28T13:19:55.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.130+0000 7ffb92000640 1 -- 172.21.15.19:0/1936685142 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c01c050 con 0x5645917f6130 2024-03-28T13:19:55.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.131+0000 7ffbabe11740 1 -- 172.21.15.19:0/1936685142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f6130 msgr2=0x56459181a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.131+0000 7ffbabe11740 1 --2- 172.21.15.19:0/1936685142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f6130 0x56459181a450 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c00b010 tx=0x7ffb9c018000 comp rx=0 tx=0).stop 2024-03-28T13:19:55.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.131+0000 7ffbabe11740 1 -- 172.21.15.19:0/1936685142 shutdown_connections 2024-03-28T13:19:55.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.131+0000 7ffbabe11740 1 --2- 172.21.15.19:0/1936685142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f6130 0x56459181a450 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.131+0000 7ffbabe11740 1 --2- 172.21.15.19:0/1936685142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917f4030 0x564591819f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.131+0000 7ffbabe11740 1 --2- 172.21.15.19:0/1936685142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917d8560 0x5645917c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.131+0000 7ffbabe11740 1 -- 172.21.15.19:0/1936685142 >> 172.21.15.19:0/1936685142 conn(0x5645917aba40 msgr2=0x5645917c16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:55.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.131+0000 7ffbabe11740 1 -- 172.21.15.19:0/1936685142 shutdown_connections 2024-03-28T13:19:55.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.131+0000 7ffbabe11740 1 -- 172.21.15.19:0/1936685142 wait complete. 2024-03-28T13:19:55.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.132+0000 7ffbabe11740 1 -- start start 2024-03-28T13:19:55.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.132+0000 7ffbabe11740 1 Processor -- start 2024-03-28T13:19:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.133+0000 7ffbabe11740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917d8560 0x5645918745e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.133+0000 7ffbabe11740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917f4030 0x564591810180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.133+0000 7ffbabe11740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f6130 0x5645917f95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.133+0000 7ffbabe11740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffba41511a0 con 0x5645917f6130 2024-03-28T13:19:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.133+0000 7ffbabe11740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5645917b18b0 con 0x5645917f4030 2024-03-28T13:19:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.133+0000 7ffbabe11740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x564591874b20 con 0x5645917d8560 2024-03-28T13:19:55.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.133+0000 7ffbabe11740 10 client.0 _ll_register_callbacks cb 0x7ffb9c0079b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:19:55.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.133+0000 7ffba2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f6130 0x5645917f95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.133+0000 7ffba2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f6130 0x5645917f95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44730/0 (socket says 172.21.15.19:44730) 2024-03-28T13:19:55.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.133+0000 7ffba2800640 1 -- 172.21.15.19:0/3053735539 learned_addr learned my addr 172.21.15.19:0/3053735539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:55.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.133+0000 7ffba1e00640 1 --2- 172.21.15.19:0/3053735539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917d8560 0x5645918745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.134+0000 7ffba2800640 1 -- 172.21.15.19:0/3053735539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917d8560 msgr2=0x5645918745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:55.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.134+0000 7ffba2800640 1 --2- 172.21.15.19:0/3053735539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917d8560 0x5645918745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.134+0000 7ffb9be00640 1 --2- 172.21.15.19:0/3053735539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917f4030 0x564591810180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.134+0000 7ffba2800640 1 -- 172.21.15.19:0/3053735539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917f4030 msgr2=0x564591810180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:55.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.134+0000 7ffba2800640 1 --2- 172.21.15.19:0/3053735539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917f4030 0x564591810180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.134+0000 7ffba2800640 1 -- 172.21.15.19:0/3053735539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645917c5e20 con 0x5645917f6130 2024-03-28T13:19:55.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.134+0000 7ffba2800640 1 --2- 172.21.15.19:0/3053735539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f6130 0x5645917f95f0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c007ac0 tx=0x7ffb9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f737b0d0f872fb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.134+0000 7ffb87400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:19:55.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.135+0000 7ffb87400640 1 -- 172.21.15.19:0/3053735539 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c03a060 con 0x5645917f6130 2024-03-28T13:19:55.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.135+0000 7ffb87400640 1 -- 172.21.15.19:0/3053735539 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7ffb9c018a30 con 0x5645917f6130 2024-03-28T13:19:55.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.135+0000 7ffbabe11740 1 -- 172.21.15.19:0/3053735539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb9c016890 con 0x5645917f6130 2024-03-28T13:19:55.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.135+0000 7ffb87400640 1 -- 172.21.15.19:0/3053735539 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7ffb9c040050 con 0x5645917f6130 2024-03-28T13:19:55.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.135+0000 7ffbabe11740 10 client.8370 fetch_fsmap learned FSMap version 83 2024-03-28T13:19:55.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.135+0000 7ffbabe11740 1 -- 172.21.15.19:0/3053735539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) -- 0x5645918740c0 con 0x5645917f6130 2024-03-28T13:19:55.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.135+0000 7ffb87400640 1 -- 172.21.15.19:0/3053735539 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c016890 con 0x5645917f6130 2024-03-28T13:19:55.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.135+0000 7ffb87400640 1 -- 172.21.15.19:0/3053735539 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 83) ==== 3494+0+0 (secure 0 0 0) 0x7ffb9c043020 con 0x5645917f6130 2024-03-28T13:19:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.135+0000 7ffbabe11740 10 client.8370 fetch_fsmap finished waiting for FSMap version 83 2024-03-28T13:19:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.135+0000 7ffbabe11740 10 client.8370 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-28T13:19:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.135+0000 7ffbabe11740 20 client.8370 populate_metadata read hostname 'smithi019' 2024-03-28T13:19:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.135+0000 7ffbabe11740 1 --2- 172.21.15.19:0/3053735539 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x564591867290 0x5645917f9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.135+0000 7ffbabe11740 4 client.8370 mds_command: new command op to 8040 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:19:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.135+0000 7ffbabe11740 1 -- 172.21.15.19:0/3053735539 --> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56459171df40 con 0x564591867290 2024-03-28T13:19:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.136+0000 7ffba1e00640 1 --2- 172.21.15.19:0/3053735539 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x564591867290 0x5645917f9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.136+0000 7ffb87400640 10 client.8370 ms_handle_connect on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.136+0000 7ffba1e00640 1 --2- 172.21.15.19:0/3053735539 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x564591867290 0x5645917f9b30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c8edfd45dee679 server_cookie=25f413addea022e1 in_seq=0 out_seq=0 2024-03-28T13:19:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.136+0000 7ffb87400640 1 -- 172.21.15.19:0/3053735539 <== mds.0 v2:172.21.15.19:6832/1020599987 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56459171df40 con 0x564591867290 2024-03-28T13:19:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.136+0000 7ffb87400640 10 client.8370 handle_command_reply: tid=0 2024-03-28T13:19:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.136+0000 7ffbabe11740 1 client.8370 shutdown 2024-03-28T13:19:55.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.137+0000 7ffbabe11740 1 -- 172.21.15.19:0/3053735539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f6130 msgr2=0x5645917f95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.137+0000 7ffbabe11740 1 --2- 172.21.15.19:0/3053735539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f6130 0x5645917f95f0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c007ac0 tx=0x7ffb9c004a40 comp rx=0 tx=0).stop 2024-03-28T13:19:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.137+0000 7ffbabe11740 1 -- 172.21.15.19:0/3053735539 shutdown_connections 2024-03-28T13:19:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.137+0000 7ffbabe11740 1 --2- 172.21.15.19:0/3053735539 >> [v2:172.21.15.19:6832/1020599987,v1:172.21.15.19:6834/1020599987] conn(0x564591867290 0x5645917f9b30 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.137+0000 7ffbabe11740 1 --2- 172.21.15.19:0/3053735539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5645917f6130 0x5645917f95f0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.137+0000 7ffbabe11740 1 --2- 172.21.15.19:0/3053735539 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5645917f4030 0x564591810180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.137+0000 7ffbabe11740 1 --2- 172.21.15.19:0/3053735539 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5645917d8560 0x5645918745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.137+0000 7ffb87400640 0 client.8370 ms_handle_reset on v2:172.21.15.19:6832/1020599987 2024-03-28T13:19:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.137+0000 7ffbabe11740 1 -- 172.21.15.19:0/3053735539 >> 172.21.15.19:0/3053735539 conn(0x5645917aba40 msgr2=0x5645917f8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.137+0000 7ffbabe11740 1 -- 172.21.15.19:0/3053735539 shutdown_connections 2024-03-28T13:19:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.137+0000 7ffbabe11740 1 -- 172.21.15.19:0/3053735539 wait complete. 2024-03-28T13:19:55.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.137+0000 7ffbabe11740 20 client.8370 trim_cache size 0 max 16384 2024-03-28T13:19:55.142 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:19:55.142 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-03-28T13:19:55.142 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-03-28T13:19:55.142 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:19:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.138+0000 7ffba9000640 1 -- 172.21.15.19:0/1198560808 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8006c560 msgr2=0x7ffb8006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.138+0000 7ffba9000640 1 --2- 172.21.15.19:0/1198560808 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8006c560 0x7ffb8006ea20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ffb94007fd0 tx=0x7ffb94009040 comp rx=0 tx=0).stop 2024-03-28T13:19:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.138+0000 7ffba9000640 1 -- 172.21.15.19:0/1198560808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba40d43d0 msgr2=0x7ffba40b9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.138+0000 7ffba9000640 1 --2- 172.21.15.19:0/1198560808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba40d43d0 0x7ffba40b9c30 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c02b9c0 tx=0x7ffb8c004730 comp rx=0 tx=0).stop 2024-03-28T13:19:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.138+0000 7ffba9000640 1 -- 172.21.15.19:0/1198560808 shutdown_connections 2024-03-28T13:19:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.138+0000 7ffba9000640 1 --2- 172.21.15.19:0/1198560808 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ffb8006c560 0x7ffb8006ea20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.138+0000 7ffba9000640 1 --2- 172.21.15.19:0/1198560808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffba415e860 0x7ffba40cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.138+0000 7ffba9000640 1 --2- 172.21.15.19:0/1198560808 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffba40d43d0 0x7ffba40b9c30 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.138+0000 7ffba9000640 1 --2- 172.21.15.19:0/1198560808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffba40cf6b0 0x7ffba40b96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.138+0000 7ffba9000640 1 -- 172.21.15.19:0/1198560808 >> 172.21.15.19:0/1198560808 conn(0x7ffba409d7d0 msgr2=0x7ffba40cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:55.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.138+0000 7ffba9000640 1 -- 172.21.15.19:0/1198560808 shutdown_connections 2024-03-28T13:19:55.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.138+0000 7ffba9000640 1 -- 172.21.15.19:0/1198560808 wait complete. 2024-03-28T13:19:55.155 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T13:19:55.256 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T13:19:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.469+0000 7f9e60800640 1 Processor -- start 2024-03-28T13:19:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.469+0000 7f9e60800640 1 -- start start 2024-03-28T13:19:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.471+0000 7f9e60800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9e5815e590 0x7f9e580d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.471+0000 7f9e60800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e5806cd90 0x7f9e580cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.471+0000 7f9e60800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e58071060 con 0x7f9e580b47d0 2024-03-28T13:19:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.471+0000 7f9e60800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9e58071340 con 0x7f9e5815e590 2024-03-28T13:19:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.471+0000 7f9e60800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9e58071620 con 0x7f9e5806cd90 2024-03-28T13:19:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.472+0000 7f9e5e800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e5806cd90 0x7f9e580cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.472+0000 7f9e5d400640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f9e580b47d0 0x7f9e5806c680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:39720/0 (socket says 172.21.15.19:39720) 2024-03-28T13:19:55.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.472+0000 7f9e5d400640 1 -- 172.21.15.19:0/2063635033 learned_addr learned my addr 172.21.15.19:0/2063635033 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:55.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.472+0000 7f9e5de00640 1 --2- 172.21.15.19:0/2063635033 >> v2:172.21.15.188:3300/0 conn(0x7f9e5815e590 0x7f9e580d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.473+0000 7f9e5e800640 1 -- 172.21.15.19:0/2063635033 >> v2:172.21.15.188:3300/0 conn(0x7f9e5815e590 msgr2=0x7f9e580d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:55.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.473+0000 7f9e5e800640 1 --2- 172.21.15.19:0/2063635033 >> v2:172.21.15.188:3300/0 conn(0x7f9e5815e590 0x7f9e580d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.473+0000 7f9e5e800640 1 -- 172.21.15.19:0/2063635033 >> v1:172.21.15.188:6789/0 conn(0x7f9e580b47d0 legacy=0x7f9e5806c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.473+0000 7f9e5e800640 1 -- 172.21.15.19:0/2063635033 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e58071900 con 0x7f9e5806cd90 2024-03-28T13:19:55.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.473+0000 7f9e5de00640 1 --2- 172.21.15.19:0/2063635033 >> v2:172.21.15.188:3300/0 conn(0x7f9e5815e590 0x7f9e580d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:19:55.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.475+0000 7f9e5e800640 1 --2- 172.21.15.19:0/2063635033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e5806cd90 0x7f9e580cfd80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009d90 tx=0x7f9e5402beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f41cad343cefbb36 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.475+0000 7f9e5ca00640 1 -- 172.21.15.19:0/2063635033 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54039070 con 0x7f9e5806cd90 2024-03-28T13:19:55.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.475+0000 7f9e5ca00640 1 -- 172.21.15.19:0/2063635033 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e5402cc60 con 0x7f9e5806cd90 2024-03-28T13:19:55.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.475+0000 7f9e5ca00640 1 -- 172.21.15.19:0/2063635033 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e54034660 con 0x7f9e5806cd90 2024-03-28T13:19:55.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.476+0000 7f9e60800640 1 -- 172.21.15.19:0/2063635033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e5806cd90 msgr2=0x7f9e580cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.476+0000 7f9e60800640 1 --2- 172.21.15.19:0/2063635033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e5806cd90 0x7f9e580cfd80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e54009d90 tx=0x7f9e5402beb0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.476+0000 7f9e60800640 1 -- 172.21.15.19:0/2063635033 shutdown_connections 2024-03-28T13:19:55.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.476+0000 7f9e60800640 1 --2- 172.21.15.19:0/2063635033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e5806cd90 0x7f9e580cfd80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.476+0000 7f9e60800640 1 --2- 172.21.15.19:0/2063635033 >> v2:172.21.15.188:3300/0 conn(0x7f9e5815e590 0x7f9e580d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.476+0000 7f9e60800640 1 -- 172.21.15.19:0/2063635033 >> 172.21.15.19:0/2063635033 conn(0x7f9e580d2240 msgr2=0x7f9e580d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:55.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.476+0000 7f9e60800640 1 -- 172.21.15.19:0/2063635033 shutdown_connections 2024-03-28T13:19:55.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.476+0000 7f9e60800640 1 -- 172.21.15.19:0/2063635033 wait complete. 2024-03-28T13:19:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.478+0000 7f9e60800640 1 Processor -- start 2024-03-28T13:19:55.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.478+0000 7f9e60800640 1 -- start start 2024-03-28T13:19:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.479+0000 7f9e60800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5806cd90 0x7f9e5814daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.479+0000 7f9e60800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e580b47d0 0x7f9e58152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.479+0000 7f9e60800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e5815e590 0x7f9e580b7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.479+0000 7f9e60800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5815e970 con 0x7f9e5806cd90 2024-03-28T13:19:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.479+0000 7f9e60800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9e580b4c60 con 0x7f9e5815e590 2024-03-28T13:19:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.479+0000 7f9e60800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9e5815eaf0 con 0x7f9e580b47d0 2024-03-28T13:19:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.479+0000 7f9e5de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5806cd90 0x7f9e5814daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.479+0000 7f9e5de00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5806cd90 0x7f9e5814daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47670/0 (socket says 172.21.15.19:47670) 2024-03-28T13:19:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.479+0000 7f9e5de00640 1 -- 172.21.15.19:0/911496247 learned_addr learned my addr 172.21.15.19:0/911496247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.480+0000 7f9e5d400640 1 --2- 172.21.15.19:0/911496247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e580b47d0 0x7f9e58152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.480+0000 7f9e5de00640 1 -- 172.21.15.19:0/911496247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e580b47d0 msgr2=0x7f9e58152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.480+0000 7f9e5de00640 1 --2- 172.21.15.19:0/911496247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e580b47d0 0x7f9e58152e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.480+0000 7f9e5e800640 1 --2- 172.21.15.19:0/911496247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e5815e590 0x7f9e580b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.480+0000 7f9e5de00640 1 -- 172.21.15.19:0/911496247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e5815e590 msgr2=0x7f9e580b7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.480+0000 7f9e5de00640 1 --2- 172.21.15.19:0/911496247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e5815e590 0x7f9e580b7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.480+0000 7f9e5de00640 1 -- 172.21.15.19:0/911496247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e58153490 con 0x7f9e5806cd90 2024-03-28T13:19:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.480+0000 7f9e5de00640 1 --2- 172.21.15.19:0/911496247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5806cd90 0x7f9e5814daf0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00e990 tx=0x7f9e4c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.481+0000 7f9e52000640 1 -- 172.21.15.19:0/911496247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c017070 con 0x7f9e5806cd90 2024-03-28T13:19:55.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.481+0000 7f9e52000640 1 -- 172.21.15.19:0/911496247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4c00c9e0 con 0x7f9e5806cd90 2024-03-28T13:19:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.481+0000 7f9e52000640 1 -- 172.21.15.19:0/911496247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c00cdb0 con 0x7f9e5806cd90 2024-03-28T13:19:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.481+0000 7f9e60800640 1 -- 172.21.15.19:0/911496247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e58154be0 con 0x7f9e5806cd90 2024-03-28T13:19:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.481+0000 7f9e60800640 1 -- 172.21.15.19:0/911496247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e58154e80 con 0x7f9e5806cd90 2024-03-28T13:19:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.482+0000 7f9e60800640 1 -- 172.21.15.19:0/911496247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e580cd220 con 0x7f9e5806cd90 2024-03-28T13:19:55.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.483+0000 7f9e52000640 1 -- 172.21.15.19:0/911496247 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9e4c01a070 con 0x7f9e5806cd90 2024-03-28T13:19:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.484+0000 7f9e52000640 1 --2- 172.21.15.19:0/911496247 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9e3c06c530 0x7f9e3c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.484+0000 7f9e52000640 1 -- 172.21.15.19:0/911496247 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f9e4c08b6a0 con 0x7f9e5806cd90 2024-03-28T13:19:55.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.484+0000 7f9e5d400640 1 --2- 172.21.15.19:0/911496247 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9e3c06c530 0x7f9e3c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.486+0000 7f9e5d400640 1 --2- 172.21.15.19:0/911496247 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9e3c06c530 0x7f9e3c06e9f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9e58157f00 tx=0x7f9e48009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.489+0000 7f9e52000640 1 -- 172.21.15.19:0/911496247 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9e580cd220 con 0x7f9e5806cd90 2024-03-28T13:19:55.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.816+0000 7f9e60800640 1 -- 172.21.15.19:0/911496247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9e581555a0 con 0x7f9e5806cd90 2024-03-28T13:19:55.817 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:19:55.817+0000 7f6b74400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T13:19:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.821+0000 7f9e52000640 1 -- 172.21.15.19:0/911496247 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) ==== 161+0+0 (secure 0 0 0) 0x7f9e4c059c70 con 0x7f9e5806cd90 2024-03-28T13:19:55.821 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T13:19:55.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.823+0000 7f9e60800640 1 -- 172.21.15.19:0/911496247 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9e3c06c530 msgr2=0x7f9e3c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.823+0000 7f9e60800640 1 --2- 172.21.15.19:0/911496247 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9e3c06c530 0x7f9e3c06e9f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9e58157f00 tx=0x7f9e48009040 comp rx=0 tx=0).stop 2024-03-28T13:19:55.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.823+0000 7f9e60800640 1 -- 172.21.15.19:0/911496247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5806cd90 msgr2=0x7f9e5814daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.823+0000 7f9e60800640 1 --2- 172.21.15.19:0/911496247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5806cd90 0x7f9e5814daf0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00e990 tx=0x7f9e4c00ee50 comp rx=0 tx=0).stop 2024-03-28T13:19:55.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.824+0000 7f9e60800640 1 -- 172.21.15.19:0/911496247 shutdown_connections 2024-03-28T13:19:55.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.824+0000 7f9e60800640 1 --2- 172.21.15.19:0/911496247 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9e3c06c530 0x7f9e3c06e9f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.824+0000 7f9e60800640 1 --2- 172.21.15.19:0/911496247 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e5815e590 0x7f9e580b7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.824+0000 7f9e60800640 1 --2- 172.21.15.19:0/911496247 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e580b47d0 0x7f9e58152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.824+0000 7f9e60800640 1 --2- 172.21.15.19:0/911496247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e5806cd90 0x7f9e5814daf0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.824+0000 7f9e60800640 1 -- 172.21.15.19:0/911496247 >> 172.21.15.19:0/911496247 conn(0x7f9e580d2240 msgr2=0x7f9e580ccb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:55.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.824+0000 7f9e60800640 1 -- 172.21.15.19:0/911496247 shutdown_connections 2024-03-28T13:19:55.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.824+0000 7f9e60800640 1 -- 172.21.15.19:0/911496247 wait complete. 2024-03-28T13:19:55.841 INFO:tasks.ceph.mds.c.smithi019.stderr: -7443> 2024-03-28T13:19:05.020+0000 7f299cc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:19:55.846 DEBUG:teuthology.orchestra.run.smithi019:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-03-28T13:19:55.909 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-03-28T13:19:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.938+0000 7f62a5837280 1 -- start start 2024-03-28T13:19:55.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.938+0000 7f62a5837280 1 Processor -- start 2024-03-28T13:19:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.939+0000 7f62a5837280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560f1496ac00 0x560f15738000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.939+0000 7f62a5837280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1496b800 0x560f15738580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.939+0000 7f62a5837280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f1496bc00 0x560f15738b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.939+0000 7f62a5837280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560f15824180 con 0x560f1496b800 2024-03-28T13:19:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.939+0000 7f62a5837280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560f15824300 con 0x560f1496bc00 2024-03-28T13:19:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.939+0000 7f62a5837280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x560f15824480 con 0x560f1496ac00 2024-03-28T13:19:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.940+0000 7f62a2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1496b800 0x560f15738580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.940+0000 7f62a2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1496b800 0x560f15738580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47710/0 (socket says 172.21.15.19:47710) 2024-03-28T13:19:55.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.940+0000 7f62a2600640 1 -- 172.21.15.19:0/3528705183 learned_addr learned my addr 172.21.15.19:0/3528705183 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.940+0000 7f62a2600640 1 -- 172.21.15.19:0/3528705183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560f1496ac00 msgr2=0x560f15738000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.940+0000 7f62a2600640 1 --2- 172.21.15.19:0/3528705183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560f1496ac00 0x560f15738000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.940+0000 7f62a2600640 1 -- 172.21.15.19:0/3528705183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f1496bc00 msgr2=0x560f15738b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.940+0000 7f62a2600640 1 --2- 172.21.15.19:0/3528705183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f1496bc00 0x560f15738b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.940+0000 7f62a2600640 1 -- 172.21.15.19:0/3528705183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f1499e780 con 0x560f1496b800 2024-03-28T13:19:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.940+0000 7f62a2600640 1 --2- 172.21.15.19:0/3528705183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1496b800 0x560f15738580 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x560f1578ca80 tx=0x560f156ff3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e017150a9a26f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.940+0000 7f629fe00640 1 -- 172.21.15.19:0/3528705183 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560f149909c0 con 0x560f1496b800 2024-03-28T13:19:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.940+0000 7f629fe00640 1 -- 172.21.15.19:0/3528705183 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560f14996fc0 con 0x560f1496b800 2024-03-28T13:19:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.940+0000 7f629fe00640 1 -- 172.21.15.19:0/3528705183 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560f14990680 con 0x560f1496b800 2024-03-28T13:19:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.941+0000 7f62a5837280 1 -- 172.21.15.19:0/3528705183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f1499f4a0 con 0x560f1496b800 2024-03-28T13:19:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.941+0000 7f62a5837280 1 -- 172.21.15.19:0/3528705183 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=0}) -- 0x560f1499e000 con 0x560f1496b800 2024-03-28T13:19:55.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.941+0000 7f62a5837280 4 waiting for MDS map... 2024-03-28T13:19:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.941+0000 7f629fe00640 1 -- 172.21.15.19:0/3528705183 <== mon.0 v2:172.21.15.19:3300/0 4 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x560f1583e200 con 0x560f1496b800 2024-03-28T13:19:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.941+0000 7f629fe00640 1 -- 172.21.15.19:0/3528705183 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 84) ==== 2885+0+0 (secure 0 0 0) 0x560f149c5180 con 0x560f1496b800 2024-03-28T13:19:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.941+0000 7f62a5837280 4 Got MDS map 84 2024-03-28T13:19:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.941+0000 7f62a5837280 10 main: JournalTool::main 2024-03-28T13:19:55.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.941+0000 7f62a5837280 4 main: JournalTool: connecting to RADOS... 2024-03-28T13:19:55.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.942+0000 7f62a5837280 1 Processor -- start 2024-03-28T13:19:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.942+0000 7f62a5837280 1 -- start start 2024-03-28T13:19:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.943+0000 7f62a5837280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560f1589c000 0x560f15739080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.943+0000 7f62a5837280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f1589c400 0x560f15739600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.943+0000 7f62a5837280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1589c800 0x560f15739b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.943+0000 7f62a5837280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560f15824600 con 0x560f1589c800 2024-03-28T13:19:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.943+0000 7f62a5837280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560f15824780 con 0x560f1589c400 2024-03-28T13:19:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.943+0000 7f62a5837280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x560f15824900 con 0x560f1589c000 2024-03-28T13:19:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.943+0000 7f62a2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1589c800 0x560f15739b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.943+0000 7f62a2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1589c800 0x560f15739b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47712/0 (socket says 172.21.15.19:47712) 2024-03-28T13:19:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.943+0000 7f62a2600640 1 -- 172.21.15.19:0/2141909810 learned_addr learned my addr 172.21.15.19:0/2141909810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.943+0000 7f62a2600640 1 -- 172.21.15.19:0/2141909810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560f1589c000 msgr2=0x560f15739080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.943+0000 7f62a2600640 1 --2- 172.21.15.19:0/2141909810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560f1589c000 0x560f15739080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.943+0000 7f62a2600640 1 -- 172.21.15.19:0/2141909810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f1589c400 msgr2=0x560f15739600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.943+0000 7f62a2600640 1 --2- 172.21.15.19:0/2141909810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f1589c400 0x560f15739600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.943+0000 7f62a2600640 1 -- 172.21.15.19:0/2141909810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f1499e3c0 con 0x560f1589c800 2024-03-28T13:19:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.944+0000 7f62a2600640 1 --2- 172.21.15.19:0/2141909810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1589c800 0x560f15739b80 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x560f15896750 tx=0x560f156ff4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d42d6bd6936d402 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.944+0000 7f629d600640 1 -- 172.21.15.19:0/2141909810 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560f14990b60 con 0x560f1589c800 2024-03-28T13:19:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.944+0000 7f629d600640 1 -- 172.21.15.19:0/2141909810 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560f14996e00 con 0x560f1589c800 2024-03-28T13:19:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.944+0000 7f629d600640 1 -- 172.21.15.19:0/2141909810 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560f14990820 con 0x560f1589c800 2024-03-28T13:19:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.944+0000 7f62a5837280 1 -- 172.21.15.19:0/2141909810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1589c800 msgr2=0x560f15739b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.944+0000 7f62a5837280 1 --2- 172.21.15.19:0/2141909810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1589c800 0x560f15739b80 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x560f15896750 tx=0x560f156ff4a0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.944+0000 7f62a5837280 1 -- 172.21.15.19:0/2141909810 shutdown_connections 2024-03-28T13:19:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.944+0000 7f62a5837280 1 --2- 172.21.15.19:0/2141909810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1589c800 0x560f15739b80 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.944+0000 7f62a5837280 1 --2- 172.21.15.19:0/2141909810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f1589c400 0x560f15739600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.944+0000 7f62a5837280 1 --2- 172.21.15.19:0/2141909810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560f1589c000 0x560f15739080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.944+0000 7f62a5837280 1 -- 172.21.15.19:0/2141909810 >> 172.21.15.19:0/2141909810 conn(0x560f15751c00 msgr2=0x560f14a5f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.944+0000 7f62a5837280 1 -- 172.21.15.19:0/2141909810 shutdown_connections 2024-03-28T13:19:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.944+0000 7f62a5837280 1 -- 172.21.15.19:0/2141909810 wait complete. 2024-03-28T13:19:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.945+0000 7f62a5837280 1 Processor -- start 2024-03-28T13:19:55.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.945+0000 7f62a5837280 1 -- start start 2024-03-28T13:19:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.946+0000 7f62a5837280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560f1589c000 0x560f15739080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.946+0000 7f62a5837280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1589c400 0x560f15739600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.946+0000 7f62a5837280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f1589c800 0x560f15739b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.946+0000 7f62a5837280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560f15825500 con 0x560f1589c400 2024-03-28T13:19:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.946+0000 7f62a5837280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x560f15824a80 con 0x560f1589c800 2024-03-28T13:19:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.946+0000 7f62a5837280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x560f15825680 con 0x560f1589c000 2024-03-28T13:19:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.946+0000 7f62a3000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1589c400 0x560f15739600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.946+0000 7f62a3000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1589c400 0x560f15739600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47718/0 (socket says 172.21.15.19:47718) 2024-03-28T13:19:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.946+0000 7f62a3000640 1 -- 172.21.15.19:0/1688123708 learned_addr learned my addr 172.21.15.19:0/1688123708 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f62a2600640 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f1589c800 0x560f15739b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f62a3a00640 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560f1589c000 0x560f15739080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f62a3000640 1 -- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560f1589c000 msgr2=0x560f15739080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f62a3000640 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560f1589c000 0x560f15739080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f62a3000640 1 -- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f1589c800 msgr2=0x560f15739b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f62a3000640 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f1589c800 0x560f15739b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f62a3000640 1 -- 172.21.15.19:0/1688123708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f1499e960 con 0x560f1589c400 2024-03-28T13:19:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f62a2600640 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f1589c800 0x560f15739b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f62a3a00640 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560f1589c000 0x560f15739080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f62a3000640 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1589c400 0x560f15739600 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x560f158974a0 tx=0x560f1584ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f629b800640 1 -- 172.21.15.19:0/1688123708 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560f14990b60 con 0x560f1589c400 2024-03-28T13:19:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f629b800640 1 -- 172.21.15.19:0/1688123708 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x560f14996c40 con 0x560f1589c400 2024-03-28T13:19:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f629b800640 1 -- 172.21.15.19:0/1688123708 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560f14990820 con 0x560f1589c400 2024-03-28T13:19:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f62a5837280 1 -- 172.21.15.19:0/1688123708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560f1499f680 con 0x560f1589c400 2024-03-28T13:19:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f62a5837280 1 -- 172.21.15.19:0/1688123708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f1499f860 con 0x560f1589c400 2024-03-28T13:19:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.947+0000 7f62a5837280 4 main: JournalTool: resolving pool 14 2024-03-28T13:19:55.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.949+0000 7f629b800640 1 -- 172.21.15.19:0/1688123708 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x560f15762c00 con 0x560f1589c400 2024-03-28T13:19:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.949+0000 7f629b800640 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x560f15768c00 0x560f1573a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.950+0000 7f629b800640 1 -- 172.21.15.19:0/1688123708 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x560f158a2e00 con 0x560f1589c400 2024-03-28T13:19:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.950+0000 7f62a5837280 4 main: JournalTool: creating IoCtx.. 2024-03-28T13:19:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.950+0000 7f62a5837280 4 main: Executing for rank 0 2024-03-28T13:19:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.950+0000 7f62a5837280 4 Filtering by inode '1099511627777' 2024-03-28T13:19:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.950+0000 7f62a5837280 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x560f1589cc00 0x560f1573a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.950+0000 7f62a5837280 1 -- 172.21.15.19:0/1688123708 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x560f1589d000 con 0x560f1589cc00 2024-03-28T13:19:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.950+0000 7f62a2600640 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x560f1589cc00 0x560f1573a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.950+0000 7f62a3a00640 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x560f15768c00 0x560f1573a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.951+0000 7f62a2600640 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x560f1589cc00 0x560f1573a680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.951+0000 7f62a2600640 1 -- 172.21.15.19:0/1688123708 <== osd.1 v2:172.21.15.19:6801/1998544260 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x560f156ec000 con 0x560f1589cc00 2024-03-28T13:19:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.951+0000 7f62a5837280 4 Pointer 400.00000000 is readable 2024-03-28T13:19:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.951+0000 7f62a5837280 4 JournalScanner::scan: reading header object '200.00000000' 2024-03-28T13:19:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.951+0000 7f62a5837280 1 -- 172.21.15.19:0/1688123708 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x560f15769c00 con 0x560f1589cc00 2024-03-28T13:19:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.951+0000 7f62a3a00640 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x560f15768c00 0x560f1573a100 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x560f156e6810 tx=0x560f1584f0e0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.952+0000 7f62a2600640 1 -- 172.21.15.19:0/1688123708 <== osd.1 v2:172.21.15.19:6801/1998544260 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv32 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x560f156ec000 con 0x560f1589cc00 2024-03-28T13:19:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.952+0000 7f62a5837280 10 Header 0x400000 0x407d95 0x408111 2024-03-28T13:19:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.952+0000 7f62a5837280 10 Starting journal scan from offset 0x407d95 2024-03-28T13:19:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.952+0000 7f62a5837280 1 -- 172.21.15.19:0/1688123708 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 32149~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x560f14cda000 con 0x560f1589cc00 2024-03-28T13:19:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.952+0000 7f62a2600640 1 -- 172.21.15.19:0/1688123708 <== osd.1 v2:172.21.15.19:6801/1998544260 3 ==== osd_op_reply(3 200.00000001 [read 32149~892 out=892b] v0'0 uv29 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x560f156ec000 con 0x560f1589cc00 2024-03-28T13:19:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.952+0000 7f62a5837280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-03-28T13:19:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.952+0000 7f62a5837280 10 Parsing data, 0x37c bytes available 2024-03-28T13:19:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.952+0000 7f62a5837280 10 Attempting decode at 0x407d95 2024-03-28T13:19:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.952+0000 7f62a5837280 10 Consumed 0x37c bytes 2024-03-28T13:19:55.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.952+0000 7f62a5837280 10 Valid entry at 0x407d95 2024-03-28T13:19:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.952+0000 7f62a5837280 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x560f14cda400 0x560f1573ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.952+0000 7f62a5837280 1 -- 172.21.15.19:0/1688123708 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x560f14cda800 con 0x560f14cda400 2024-03-28T13:19:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.953+0000 7f62a3a00640 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x560f14cda400 0x560f1573ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:55.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.954+0000 7f62a3a00640 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x560f14cda400 0x560f1573ac00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a3a00640 1 -- 172.21.15.19:0/1688123708 <== osd.6 v2:172.21.15.188:6816/3597358155 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x560f156ec500 con 0x560f14cda400 2024-03-28T13:19:55.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 4 Reached end of journal objects 2024-03-28T13:19:55.960 INFO:teuthology.orchestra.run.smithi019.stdout:Events by type: 2024-03-28T13:19:55.960 INFO:teuthology.orchestra.run.smithi019.stdout:Errors: 0 2024-03-28T13:19:55.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 4 Scanned objects, 0 missing, 1 valid 2024-03-28T13:19:55.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 4 Events scanned, 0 gaps 2024-03-28T13:19:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 4 Found 1 valid events 2024-03-28T13:19:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 4 Selected 0 events events for processing 2024-03-28T13:19:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 4 0 events 2024-03-28T13:19:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 1 -- 172.21.15.19:0/1688123708 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x560f1589cc00 msgr2=0x560f1573a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x560f1589cc00 0x560f1573a680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 1 -- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x560f14cda400 msgr2=0x560f1573ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x560f14cda400 0x560f1573ac00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 1 -- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x560f15768c00 msgr2=0x560f1573a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x560f15768c00 0x560f1573a100 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x560f156e6810 tx=0x560f1584f0e0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 1 -- 172.21.15.19:0/1688123708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1589c400 msgr2=0x560f15739600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 1 --2- 172.21.15.19:0/1688123708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1589c400 0x560f15739600 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x560f158974a0 tx=0x560f1584ec60 comp rx=0 tx=0).stop 2024-03-28T13:19:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a3000640 1 -- 172.21.15.19:0/1688123708 reap_dead start 2024-03-28T13:19:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a3000640 1 -- 172.21.15.19:0/1688123708 reap_dead start 2024-03-28T13:19:55.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 1 -- 172.21.15.19:0/1688123708 shutdown_connections 2024-03-28T13:19:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.960+0000 7f62a5837280 1 -- 172.21.15.19:0/1688123708 >> 172.21.15.19:0/1688123708 conn(0x560f15751c00 msgr2=0x560f14a5f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.961+0000 7f62a5837280 1 -- 172.21.15.19:0/1688123708 shutdown_connections 2024-03-28T13:19:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.961+0000 7f62a5837280 1 -- 172.21.15.19:0/1688123708 wait complete. 2024-03-28T13:19:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.961+0000 7f62a5837280 1 -- 172.21.15.19:0/3528705183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1496b800 msgr2=0x560f15738580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.961+0000 7f62a5837280 1 --2- 172.21.15.19:0/3528705183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1496b800 0x560f15738580 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x560f1578ca80 tx=0x560f156ff3e0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.961+0000 7f62a5837280 1 -- 172.21.15.19:0/3528705183 shutdown_connections 2024-03-28T13:19:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.961+0000 7f62a5837280 1 --2- 172.21.15.19:0/3528705183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560f1496bc00 0x560f15738b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.961+0000 7f62a5837280 1 --2- 172.21.15.19:0/3528705183 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560f1496b800 0x560f15738580 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.961+0000 7f62a5837280 1 --2- 172.21.15.19:0/3528705183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560f1496ac00 0x560f15738000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.961+0000 7f62a5837280 1 -- 172.21.15.19:0/3528705183 >> 172.21.15.19:0/3528705183 conn(0x560f1496b000 msgr2=0x560f14a5e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.961+0000 7f62a5837280 1 -- 172.21.15.19:0/3528705183 shutdown_connections 2024-03-28T13:19:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:55.961+0000 7f62a5837280 1 -- 172.21.15.19:0/3528705183 wait complete. 2024-03-28T13:19:55.967 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.121192, result: Events by type: Errors: 0 2024-03-28T13:19:55.967 DEBUG:teuthology.orchestra.run.smithi019:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-03-28T13:19:56.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.049+0000 7f22a2783280 1 -- start start 2024-03-28T13:19:56.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.049+0000 7f22a2783280 1 Processor -- start 2024-03-28T13:19:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.050+0000 7f22a2783280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a83cc4c00 0x556a84a20000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.050+0000 7f22a2783280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a83cc5800 0x556a84a20580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.050+0000 7f22a2783280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a83cc5c00 0x556a84a20b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.050+0000 7f22a2783280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556a84a28180 con 0x556a83cc5c00 2024-03-28T13:19:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.050+0000 7f22a2783280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556a83d26900 con 0x556a83cc5800 2024-03-28T13:19:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.050+0000 7f22a2783280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x556a83d26480 con 0x556a83cc4c00 2024-03-28T13:19:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.050+0000 7f22a1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a83cc5c00 0x556a84a20b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.050+0000 7f22a1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a83cc5c00 0x556a84a20b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47736/0 (socket says 172.21.15.19:47736) 2024-03-28T13:19:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.050+0000 7f22a1200640 1 -- 172.21.15.19:0/1251745893 learned_addr learned my addr 172.21.15.19:0/1251745893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:56.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.051+0000 7f22a1200640 1 -- 172.21.15.19:0/1251745893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a83cc4c00 msgr2=0x556a84a20000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.051+0000 7f22a1200640 1 --2- 172.21.15.19:0/1251745893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a83cc4c00 0x556a84a20000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.051+0000 7f229fe00640 1 --2- 172.21.15.19:0/1251745893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a83cc5800 0x556a84a20580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.051+0000 7f22a1200640 1 -- 172.21.15.19:0/1251745893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a83cc5800 msgr2=0x556a84a20580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.051+0000 7f22a1200640 1 --2- 172.21.15.19:0/1251745893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a83cc5800 0x556a84a20580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.051+0000 7f22a1200640 1 -- 172.21.15.19:0/1251745893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a83cf8780 con 0x556a83cc5c00 2024-03-28T13:19:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.051+0000 7f22a1200640 1 --2- 172.21.15.19:0/1251745893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a83cc5c00 0x556a84a20b00 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x556a84ae7440 tx=0x556a84a59080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f777f0f9e9abfc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.051+0000 7f229d600640 1 -- 172.21.15.19:0/1251745893 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556a83cea680 con 0x556a83cc5c00 2024-03-28T13:19:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.051+0000 7f229d600640 1 -- 172.21.15.19:0/1251745893 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556a83cf0700 con 0x556a83cc5c00 2024-03-28T13:19:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.051+0000 7f229d600640 1 -- 172.21.15.19:0/1251745893 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556a83ceab60 con 0x556a83cc5c00 2024-03-28T13:19:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.052+0000 7f22a2783280 1 -- 172.21.15.19:0/1251745893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a83cf94a0 con 0x556a83cc5c00 2024-03-28T13:19:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.052+0000 7f22a2783280 1 -- 172.21.15.19:0/1251745893 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=0}) -- 0x556a83cf8000 con 0x556a83cc5c00 2024-03-28T13:19:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.052+0000 7f22a2783280 4 waiting for MDS map... 2024-03-28T13:19:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.052+0000 7f229d600640 1 -- 172.21.15.19:0/1251745893 <== mon.0 v2:172.21.15.19:3300/0 4 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x556a84b8e600 con 0x556a83cc5c00 2024-03-28T13:19:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.052+0000 7f229d600640 1 -- 172.21.15.19:0/1251745893 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 84) ==== 2885+0+0 (secure 0 0 0) 0x556a83d1f180 con 0x556a83cc5c00 2024-03-28T13:19:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.052+0000 7f22a2783280 4 Got MDS map 84 2024-03-28T13:19:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.052+0000 7f22a2783280 10 main: JournalTool::main 2024-03-28T13:19:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.052+0000 7f22a2783280 4 main: JournalTool: connecting to RADOS... 2024-03-28T13:19:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.053+0000 7f22a2783280 1 Processor -- start 2024-03-28T13:19:56.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.053+0000 7f22a2783280 1 -- start start 2024-03-28T13:19:56.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.054+0000 7f22a2783280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a84b91400 0x556a84a21080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.054+0000 7f22a2783280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a84b91800 0x556a84a21600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.054+0000 7f22a2783280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a84b91c00 0x556a84a21b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.054+0000 7f22a2783280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556a83d26180 con 0x556a84b91c00 2024-03-28T13:19:56.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.054+0000 7f22a2783280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556a84a28d80 con 0x556a84b91400 2024-03-28T13:19:56.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.054+0000 7f22a2783280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x556a84a28f00 con 0x556a84b91800 2024-03-28T13:19:56.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.054+0000 7f229fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a84b91c00 0x556a84a21b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.054+0000 7f229fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a84b91c00 0x556a84a21b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47750/0 (socket says 172.21.15.19:47750) 2024-03-28T13:19:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.054+0000 7f229fe00640 1 -- 172.21.15.19:0/2213989639 learned_addr learned my addr 172.21.15.19:0/2213989639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.055+0000 7f229fe00640 1 --2- 172.21.15.19:0/2213989639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a84b91400 0x556a84a21080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.055+0000 7f22a0800640 1 --2- 172.21.15.19:0/2213989639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a84b91800 0x556a84a21600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.055+0000 7f229fe00640 1 -- 172.21.15.19:0/2213989639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a84b91800 msgr2=0x556a84a21600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.055+0000 7f229fe00640 1 --2- 172.21.15.19:0/2213989639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a84b91800 0x556a84a21600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.055+0000 7f229fe00640 1 -- 172.21.15.19:0/2213989639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a84b91400 msgr2=0x556a84a21080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.055+0000 7f229fe00640 1 --2- 172.21.15.19:0/2213989639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a84b91400 0x556a84a21080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.055+0000 7f229fe00640 1 -- 172.21.15.19:0/2213989639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a83cf83c0 con 0x556a84b91c00 2024-03-28T13:19:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.055+0000 7f22a0800640 1 --2- 172.21.15.19:0/2213989639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a84b91800 0x556a84a21600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.055+0000 7f229fe00640 1 --2- 172.21.15.19:0/2213989639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a84b91c00 0x556a84a21b80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x556a84ba6300 tx=0x556a84a58e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478c202f62faec38 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.055+0000 7f229ae00640 1 -- 172.21.15.19:0/2213989639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556a83cea9c0 con 0x556a84b91c00 2024-03-28T13:19:56.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.055+0000 7f229ae00640 1 -- 172.21.15.19:0/2213989639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556a83cf0fc0 con 0x556a84b91c00 2024-03-28T13:19:56.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.055+0000 7f229ae00640 1 -- 172.21.15.19:0/2213989639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556a83cea680 con 0x556a84b91c00 2024-03-28T13:19:56.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.056+0000 7f22a2783280 1 -- 172.21.15.19:0/2213989639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a84b91c00 msgr2=0x556a84a21b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.056+0000 7f22a2783280 1 --2- 172.21.15.19:0/2213989639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a84b91c00 0x556a84a21b80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x556a84ba6300 tx=0x556a84a58e40 comp rx=0 tx=0).stop 2024-03-28T13:19:56.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.056+0000 7f22a2783280 1 -- 172.21.15.19:0/2213989639 shutdown_connections 2024-03-28T13:19:56.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.056+0000 7f22a2783280 1 --2- 172.21.15.19:0/2213989639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a84b91c00 0x556a84a21b80 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.056+0000 7f22a2783280 1 --2- 172.21.15.19:0/2213989639 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a84b91800 0x556a84a21600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.056+0000 7f22a2783280 1 --2- 172.21.15.19:0/2213989639 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a84b91400 0x556a84a21080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.056+0000 7f22a2783280 1 -- 172.21.15.19:0/2213989639 >> 172.21.15.19:0/2213989639 conn(0x556a84b91000 msgr2=0x556a83dd1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:56.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.056+0000 7f22a2783280 1 -- 172.21.15.19:0/2213989639 shutdown_connections 2024-03-28T13:19:56.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.056+0000 7f22a2783280 1 -- 172.21.15.19:0/2213989639 wait complete. 2024-03-28T13:19:56.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.056+0000 7f22a2783280 1 Processor -- start 2024-03-28T13:19:56.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.057+0000 7f22a2783280 1 -- start start 2024-03-28T13:19:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.057+0000 7f22a2783280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a84b91400 0x556a84a21080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.057+0000 7f22a2783280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a84b91800 0x556a84a21600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.057+0000 7f22a2783280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a84b91c00 0x556a84a21b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.057+0000 7f22a2783280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556a84a29200 con 0x556a84b91800 2024-03-28T13:19:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.057+0000 7f22a2783280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x556a84a29080 con 0x556a84b91400 2024-03-28T13:19:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.057+0000 7f22a2783280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x556a84a29380 con 0x556a84b91c00 2024-03-28T13:19:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.058+0000 7f22a0800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a84b91800 0x556a84a21600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.058+0000 7f22a0800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a84b91800 0x556a84a21600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47754/0 (socket says 172.21.15.19:47754) 2024-03-28T13:19:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.058+0000 7f22a0800640 1 -- 172.21.15.19:0/1210453547 learned_addr learned my addr 172.21.15.19:0/1210453547 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.058+0000 7f22a0800640 1 -- 172.21.15.19:0/1210453547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a84b91c00 msgr2=0x556a84a21b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.058+0000 7f22a0800640 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a84b91c00 0x556a84a21b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.058+0000 7f22a0800640 1 -- 172.21.15.19:0/1210453547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a84b91400 msgr2=0x556a84a21080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.058+0000 7f22a0800640 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a84b91400 0x556a84a21080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.058+0000 7f22a0800640 1 -- 172.21.15.19:0/1210453547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a83cf8780 con 0x556a84b91800 2024-03-28T13:19:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.058+0000 7f22a0800640 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a84b91800 0x556a84a21600 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x556a84ba7650 tx=0x556a84bacd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.059+0000 7f2299000640 1 -- 172.21.15.19:0/1210453547 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556a83cea9c0 con 0x556a84b91800 2024-03-28T13:19:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.059+0000 7f2299000640 1 -- 172.21.15.19:0/1210453547 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x556a83cf0a80 con 0x556a84b91800 2024-03-28T13:19:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.059+0000 7f22a2783280 1 -- 172.21.15.19:0/1210453547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556a83cf94a0 con 0x556a84b91800 2024-03-28T13:19:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.059+0000 7f2299000640 1 -- 172.21.15.19:0/1210453547 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556a83cea680 con 0x556a84b91800 2024-03-28T13:19:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.059+0000 7f22a2783280 1 -- 172.21.15.19:0/1210453547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a83cf8960 con 0x556a84b91800 2024-03-28T13:19:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.059+0000 7f22a2783280 4 main: JournalTool: resolving pool 14 2024-03-28T13:19:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.060+0000 7f2299000640 1 -- 172.21.15.19:0/1210453547 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x556a84bc0c00 con 0x556a84b91800 2024-03-28T13:19:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.061+0000 7f2299000640 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x556a84abcc00 0x556a84a22100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.061+0000 7f2299000640 1 -- 172.21.15.19:0/1210453547 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x556a84abea00 con 0x556a84b91800 2024-03-28T13:19:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.061+0000 7f22a2783280 4 main: JournalTool: creating IoCtx.. 2024-03-28T13:19:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.061+0000 7f22a2783280 4 main: Executing for rank 0 2024-03-28T13:19:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.061+0000 7f22a2783280 4 Filtering by inode '1099511627778' 2024-03-28T13:19:56.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.061+0000 7f22a2783280 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x556a84abd000 0x556a84a22680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.061+0000 7f22a2783280 1 -- 172.21.15.19:0/1210453547 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x556a84abd400 con 0x556a84abd000 2024-03-28T13:19:56.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.061+0000 7f229fe00640 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x556a84abd000 0x556a84a22680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.061+0000 7f229fe00640 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x556a84abcc00 0x556a84a22100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.062+0000 7f229fe00640 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x556a84abd000 0x556a84a22680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.062+0000 7f229fe00640 1 -- 172.21.15.19:0/1210453547 <== osd.1 v2:172.21.15.19:6801/1998544260 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x556a83dc0000 con 0x556a84abd000 2024-03-28T13:19:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.062+0000 7f22a2783280 4 Pointer 400.00000000 is readable 2024-03-28T13:19:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.062+0000 7f22a2783280 4 JournalScanner::scan: reading header object '200.00000000' 2024-03-28T13:19:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.062+0000 7f22a2783280 1 -- 172.21.15.19:0/1210453547 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x556a84abd800 con 0x556a84abd000 2024-03-28T13:19:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.063+0000 7f229fe00640 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x556a84abcc00 0x556a84a22100 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x556a84ba6570 tx=0x556a84a58ea0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.063+0000 7f229fe00640 1 -- 172.21.15.19:0/1210453547 <== osd.1 v2:172.21.15.19:6801/1998544260 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv32 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x556a83dc0000 con 0x556a84abd000 2024-03-28T13:19:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.063+0000 7f22a2783280 10 Header 0x400000 0x407d95 0x408111 2024-03-28T13:19:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.063+0000 7f22a2783280 10 Starting journal scan from offset 0x407d95 2024-03-28T13:19:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.063+0000 7f22a2783280 1 -- 172.21.15.19:0/1210453547 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 32149~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x556a84abdc00 con 0x556a84abd000 2024-03-28T13:19:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.063+0000 7f229fe00640 1 -- 172.21.15.19:0/1210453547 <== osd.1 v2:172.21.15.19:6801/1998544260 3 ==== osd_op_reply(3 200.00000001 [read 32149~892 out=892b] v0'0 uv29 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x556a83dc0000 con 0x556a84abd000 2024-03-28T13:19:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.063+0000 7f22a2783280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-03-28T13:19:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.063+0000 7f22a2783280 10 Parsing data, 0x37c bytes available 2024-03-28T13:19:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.063+0000 7f22a2783280 10 Attempting decode at 0x407d95 2024-03-28T13:19:56.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.063+0000 7f22a2783280 10 Consumed 0x37c bytes 2024-03-28T13:19:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.063+0000 7f22a2783280 10 Valid entry at 0x407d95 2024-03-28T13:19:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.063+0000 7f22a2783280 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x556a83fba000 0x556a84a22c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.063+0000 7f22a2783280 1 -- 172.21.15.19:0/1210453547 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e108) -- 0x556a83fba400 con 0x556a83fba000 2024-03-28T13:19:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.064+0000 7f22a1200640 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x556a83fba000 0x556a84a22c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.065+0000 7f22a1200640 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x556a83fba000 0x556a84a22c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a1200640 1 -- 172.21.15.19:0/1210453547 <== osd.6 v2:172.21.15.188:6816/3597358155 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x556a83dc0500 con 0x556a83fba000 2024-03-28T13:19:56.067 INFO:teuthology.orchestra.run.smithi019.stdout:Events by type: 2024-03-28T13:19:56.067 INFO:teuthology.orchestra.run.smithi019.stdout:Errors: 0 2024-03-28T13:19:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 4 Reached end of journal objects 2024-03-28T13:19:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 4 Scanned objects, 0 missing, 1 valid 2024-03-28T13:19:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 4 Events scanned, 0 gaps 2024-03-28T13:19:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 4 Found 1 valid events 2024-03-28T13:19:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 4 Selected 0 events events for processing 2024-03-28T13:19:56.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 4 0 events 2024-03-28T13:19:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 1 -- 172.21.15.19:0/1210453547 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x556a84abd000 msgr2=0x556a84a22680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x556a84abd000 0x556a84a22680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 1 -- 172.21.15.19:0/1210453547 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x556a83fba000 msgr2=0x556a84a22c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x556a83fba000 0x556a84a22c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 1 -- 172.21.15.19:0/1210453547 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x556a84abcc00 msgr2=0x556a84a22100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x556a84abcc00 0x556a84a22100 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x556a84ba6570 tx=0x556a84a58ea0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 1 -- 172.21.15.19:0/1210453547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a84b91800 msgr2=0x556a84a21600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 1 --2- 172.21.15.19:0/1210453547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a84b91800 0x556a84a21600 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x556a84ba7650 tx=0x556a84bacd20 comp rx=0 tx=0).stop 2024-03-28T13:19:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a0800640 1 -- 172.21.15.19:0/1210453547 reap_dead start 2024-03-28T13:19:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a0800640 1 -- 172.21.15.19:0/1210453547 reap_dead start 2024-03-28T13:19:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 1 -- 172.21.15.19:0/1210453547 shutdown_connections 2024-03-28T13:19:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 1 -- 172.21.15.19:0/1210453547 >> 172.21.15.19:0/1210453547 conn(0x556a84b91000 msgr2=0x556a83dd1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 1 -- 172.21.15.19:0/1210453547 shutdown_connections 2024-03-28T13:19:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.066+0000 7f22a2783280 1 -- 172.21.15.19:0/1210453547 wait complete. 2024-03-28T13:19:56.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.067+0000 7f22a2783280 1 -- 172.21.15.19:0/1251745893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a83cc5c00 msgr2=0x556a84a20b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.067+0000 7f22a2783280 1 --2- 172.21.15.19:0/1251745893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a83cc5c00 0x556a84a20b00 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x556a84ae7440 tx=0x556a84a59080 comp rx=0 tx=0).stop 2024-03-28T13:19:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.067+0000 7f22a2783280 1 -- 172.21.15.19:0/1251745893 shutdown_connections 2024-03-28T13:19:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.067+0000 7f22a2783280 1 --2- 172.21.15.19:0/1251745893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a83cc5c00 0x556a84a20b00 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.067+0000 7f22a2783280 1 --2- 172.21.15.19:0/1251745893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556a83cc5800 0x556a84a20580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.067+0000 7f22a2783280 1 --2- 172.21.15.19:0/1251745893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556a83cc4c00 0x556a84a20000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.067+0000 7f22a2783280 1 -- 172.21.15.19:0/1251745893 >> 172.21.15.19:0/1251745893 conn(0x556a83cc5000 msgr2=0x556a83dd0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.067+0000 7f22a2783280 1 -- 172.21.15.19:0/1251745893 shutdown_connections 2024-03-28T13:19:56.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.067+0000 7f22a2783280 1 -- 172.21.15.19:0/1251745893 wait complete. 2024-03-28T13:19:56.071 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.104198, result: Events by type: Errors: 0 2024-03-28T13:19:56.071 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-03-28T13:19:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.159+0000 7fe314e40f40 1 Processor -- start 2024-03-28T13:19:56.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.159+0000 7fe314e40f40 1 -- start start 2024-03-28T13:19:56.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.160+0000 7fe314e40f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602eb3f1250 0x5602eb409c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.160+0000 7fe314e40f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602eb3f22b0 0x5602eb48f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.160+0000 7fe314e40f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5602eb491230 0x5602eb3ecdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.160+0000 7fe314e40f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5602eb3662c0 con 0x5602eb3f22b0 2024-03-28T13:19:56.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.160+0000 7fe314e40f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5602eb36a310 con 0x5602eb3f1250 2024-03-28T13:19:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.160+0000 7fe314e40f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5602eb40bb20 con 0x5602eb491230 2024-03-28T13:19:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.160+0000 7fe30be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602eb3f22b0 0x5602eb48f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.160+0000 7fe30be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602eb3f22b0 0x5602eb48f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47778/0 (socket says 172.21.15.19:47778) 2024-03-28T13:19:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.160+0000 7fe30be00640 1 -- 172.21.15.19:0/2349193450 learned_addr learned my addr 172.21.15.19:0/2349193450 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.161+0000 7fe30be00640 1 -- 172.21.15.19:0/2349193450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5602eb491230 msgr2=0x5602eb3ecdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.161+0000 7fe30be00640 1 --2- 172.21.15.19:0/2349193450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5602eb491230 0x5602eb3ecdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.161+0000 7fe311a00640 1 --2- 172.21.15.19:0/2349193450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602eb3f1250 0x5602eb409c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.161+0000 7fe30be00640 1 -- 172.21.15.19:0/2349193450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602eb3f1250 msgr2=0x5602eb409c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.161+0000 7fe30be00640 1 --2- 172.21.15.19:0/2349193450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602eb3f1250 0x5602eb409c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.161+0000 7fe30be00640 1 -- 172.21.15.19:0/2349193450 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602eb4a1fd0 con 0x5602eb3f22b0 2024-03-28T13:19:56.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.161+0000 7fe30be00640 1 --2- 172.21.15.19:0/2349193450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602eb3f22b0 0x5602eb48f050 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc002de0 tx=0x7fe2fc00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb735268d28b162 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.161+0000 7fe30b400640 1 -- 172.21.15.19:0/2349193450 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc00f400 con 0x5602eb3f22b0 2024-03-28T13:19:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.161+0000 7fe30b400640 1 -- 172.21.15.19:0/2349193450 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2fc014070 con 0x5602eb3f22b0 2024-03-28T13:19:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.161+0000 7fe30b400640 1 -- 172.21.15.19:0/2349193450 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc00f7e0 con 0x5602eb3f22b0 2024-03-28T13:19:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.162+0000 7fe314e40f40 1 -- 172.21.15.19:0/2349193450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602eb3f22b0 msgr2=0x5602eb48f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.162+0000 7fe314e40f40 1 --2- 172.21.15.19:0/2349193450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602eb3f22b0 0x5602eb48f050 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc002de0 tx=0x7fe2fc00b5b0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.162+0000 7fe314e40f40 1 -- 172.21.15.19:0/2349193450 shutdown_connections 2024-03-28T13:19:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.162+0000 7fe314e40f40 1 --2- 172.21.15.19:0/2349193450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5602eb491230 0x5602eb3ecdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.162+0000 7fe314e40f40 1 --2- 172.21.15.19:0/2349193450 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602eb3f22b0 0x5602eb48f050 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.162+0000 7fe314e40f40 1 --2- 172.21.15.19:0/2349193450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602eb3f1250 0x5602eb409c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.162+0000 7fe314e40f40 1 -- 172.21.15.19:0/2349193450 >> 172.21.15.19:0/2349193450 conn(0x5602eb40d360 msgr2=0x5602eb3ee6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.162+0000 7fe314e40f40 1 -- 172.21.15.19:0/2349193450 shutdown_connections 2024-03-28T13:19:56.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.162+0000 7fe314e40f40 1 -- 172.21.15.19:0/2349193450 wait complete. 2024-03-28T13:19:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.163+0000 7fe314e40f40 1 Processor -- start 2024-03-28T13:19:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.163+0000 7fe314e40f40 1 -- start start 2024-03-28T13:19:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.164+0000 7fe314e40f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602eb3f1250 0x5602eb4877e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.164+0000 7fe314e40f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5602eb3f22b0 0x5602eb3bec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.164+0000 7fe314e40f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602eb491230 0x5602eb3f1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.164+0000 7fe314e40f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5602eb4a2270 con 0x5602eb3f1250 2024-03-28T13:19:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.164+0000 7fe314e40f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5602eb3ee390 con 0x5602eb491230 2024-03-28T13:19:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.164+0000 7fe314e40f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5602eb4a1b20 con 0x5602eb3f22b0 2024-03-28T13:19:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.164+0000 7fe311a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602eb3f1250 0x5602eb4877e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.164+0000 7fe311a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602eb3f1250 0x5602eb4877e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47788/0 (socket says 172.21.15.19:47788) 2024-03-28T13:19:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.164+0000 7fe311a00640 1 -- 172.21.15.19:0/4165092402 learned_addr learned my addr 172.21.15.19:0/4165092402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.164+0000 7fe311a00640 1 -- 172.21.15.19:0/4165092402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5602eb3f22b0 msgr2=0x5602eb3bec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.164+0000 7fe311a00640 1 --2- 172.21.15.19:0/4165092402 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5602eb3f22b0 0x5602eb3bec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.164+0000 7fe311a00640 1 -- 172.21.15.19:0/4165092402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602eb491230 msgr2=0x5602eb3f1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.164+0000 7fe311a00640 1 --2- 172.21.15.19:0/4165092402 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5602eb491230 0x5602eb3f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.164+0000 7fe311a00640 1 -- 172.21.15.19:0/4165092402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602eb3bf620 con 0x5602eb3f1250 2024-03-28T13:19:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.165+0000 7fe311a00640 1 --2- 172.21.15.19:0/4165092402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602eb3f1250 0x5602eb4877e0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fe30400ce20 tx=0x7fe30400a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.165+0000 7fe30a000640 1 -- 172.21.15.19:0/4165092402 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe304008040 con 0x5602eb3f1250 2024-03-28T13:19:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.165+0000 7fe30a000640 1 -- 172.21.15.19:0/4165092402 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe304014020 con 0x5602eb3f1250 2024-03-28T13:19:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.165+0000 7fe314e40f40 1 -- 172.21.15.19:0/4165092402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5602eb52fdb0 con 0x5602eb3f1250 2024-03-28T13:19:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.165+0000 7fe30a000640 1 -- 172.21.15.19:0/4165092402 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30400f400 con 0x5602eb3f1250 2024-03-28T13:19:56.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.165+0000 7fe314e40f40 1 -- 172.21.15.19:0/4165092402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602eb487d20 con 0x5602eb3f1250 2024-03-28T13:19:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.167+0000 7fe30a000640 1 -- 172.21.15.19:0/4165092402 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe30400f5a0 con 0x5602eb3f1250 2024-03-28T13:19:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.168+0000 7fe30a000640 1 --2- 172.21.15.19:0/4165092402 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2f006af10 0x7fe2f006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.168+0000 7fe30a000640 1 -- 172.21.15.19:0/4165092402 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7fe30408a110 con 0x5602eb3f1250 2024-03-28T13:19:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.169+0000 7fe314e40f40 1 --2- 172.21.15.19:0/4165092402 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5602eb3bf180 0x5602eb533560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.169+0000 7fe314e40f40 1 -- 172.21.15.19:0/4165092402 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e108) -- 0x5602eb533aa0 con 0x5602eb3bf180 2024-03-28T13:19:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.169+0000 7fe30be00640 1 --2- 172.21.15.19:0/4165092402 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2f006af10 0x7fe2f006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.169+0000 7fe312400640 1 --2- 172.21.15.19:0/4165092402 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5602eb3bf180 0x5602eb533560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.170+0000 7fe30be00640 1 --2- 172.21.15.19:0/4165092402 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2f006af10 0x7fe2f006d3d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x5602eb48fef0 tx=0x7fe2fc00f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.171+0000 7fe312400640 1 --2- 172.21.15.19:0/4165092402 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5602eb3bf180 0x5602eb533560 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.174+0000 7fe312400640 1 -- 172.21.15.19:0/4165092402 <== osd.5 v2:172.21.15.188:6800/3324247755 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v108'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7fe30c052500 con 0x5602eb3bf180 2024-03-28T13:19:56.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.174+0000 7fe314e40f40 1 -- 172.21.15.19:0/4165092402 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5602eb3bf180 msgr2=0x5602eb533560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.174+0000 7fe314e40f40 1 --2- 172.21.15.19:0/4165092402 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5602eb3bf180 0x5602eb533560 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.174+0000 7fe314e40f40 1 -- 172.21.15.19:0/4165092402 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2f006af10 msgr2=0x7fe2f006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.174+0000 7fe314e40f40 1 --2- 172.21.15.19:0/4165092402 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2f006af10 0x7fe2f006d3d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x5602eb48fef0 tx=0x7fe2fc00f000 comp rx=0 tx=0).stop 2024-03-28T13:19:56.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.174+0000 7fe314e40f40 1 -- 172.21.15.19:0/4165092402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602eb3f1250 msgr2=0x5602eb4877e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.174+0000 7fe314e40f40 1 --2- 172.21.15.19:0/4165092402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5602eb3f1250 0x5602eb4877e0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fe30400ce20 tx=0x7fe30400a5d0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.174+0000 7fe312400640 1 -- 172.21.15.19:0/4165092402 reap_dead start 2024-03-28T13:19:56.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.174+0000 7fe314e40f40 1 -- 172.21.15.19:0/4165092402 shutdown_connections 2024-03-28T13:19:56.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.174+0000 7fe314e40f40 1 -- 172.21.15.19:0/4165092402 >> 172.21.15.19:0/4165092402 conn(0x5602eb40d360 msgr2=0x5602eb496a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:56.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.174+0000 7fe314e40f40 1 -- 172.21.15.19:0/4165092402 shutdown_connections 2024-03-28T13:19:56.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.174+0000 7fe314e40f40 1 -- 172.21.15.19:0/4165092402 wait complete. 2024-03-28T13:19:56.178 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-28T13:19:56.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.388+0000 7fa84a000640 1 Processor -- start 2024-03-28T13:19:56.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.388+0000 7fa84a000640 1 -- start start 2024-03-28T13:19:56.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.390+0000 7fa84a000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa84415bf50 0x7fa8440b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.390+0000 7fa84a000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8440b3cf0 0x7fa84406d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.390+0000 7fa84a000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa84415c330 con 0x7fa84406db60 2024-03-28T13:19:56.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.390+0000 7fa84a000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa84406dff0 con 0x7fa84415bf50 2024-03-28T13:19:56.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.390+0000 7fa84a000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa84406ee40 con 0x7fa8440b3cf0 2024-03-28T13:19:56.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.390+0000 7fa848c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa84415bf50 0x7fa8440b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.390+0000 7fa848c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa84415bf50 0x7fa8440b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47804/0 (socket says 172.21.15.19:47804) 2024-03-28T13:19:56.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.390+0000 7fa848c00640 1 -- 172.21.15.19:0/2755720735 learned_addr learned my addr 172.21.15.19:0/2755720735 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:56.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.391+0000 7fa843e00640 1 --2- 172.21.15.19:0/2755720735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8440b3cf0 0x7fa84406d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.391+0000 7fa848c00640 1 -- 172.21.15.19:0/2755720735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8440b3cf0 msgr2=0x7fa84406d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.391+0000 7fa848c00640 1 --2- 172.21.15.19:0/2755720735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8440b3cf0 0x7fa84406d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.391+0000 7fa848c00640 1 -- 172.21.15.19:0/2755720735 >> v1:172.21.15.19:6789/0 conn(0x7fa84406db60 legacy=0x7fa844161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.391+0000 7fa848c00640 1 -- 172.21.15.19:0/2755720735 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84406e230 con 0x7fa84415bf50 2024-03-28T13:19:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.392+0000 7fa848c00640 1 --2- 172.21.15.19:0/2755720735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa84415bf50 0x7fa8440b37b0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fa838009870 tx=0x7fa83802ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84b1b2db4a2361f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.392+0000 7fa843400640 1 -- 172.21.15.19:0/2755720735 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83802f830 con 0x7fa84415bf50 2024-03-28T13:19:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.392+0000 7fa843400640 1 -- 172.21.15.19:0/2755720735 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa83802f9d0 con 0x7fa84415bf50 2024-03-28T13:19:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.392+0000 7fa843400640 1 -- 172.21.15.19:0/2755720735 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83802fda0 con 0x7fa84415bf50 2024-03-28T13:19:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.393+0000 7fa84a000640 1 -- 172.21.15.19:0/2755720735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa84415bf50 msgr2=0x7fa8440b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.393+0000 7fa84a000640 1 --2- 172.21.15.19:0/2755720735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa84415bf50 0x7fa8440b37b0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fa838009870 tx=0x7fa83802ba10 comp rx=0 tx=0).stop 2024-03-28T13:19:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.393+0000 7fa84a000640 1 -- 172.21.15.19:0/2755720735 shutdown_connections 2024-03-28T13:19:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.393+0000 7fa84a000640 1 --2- 172.21.15.19:0/2755720735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8440b3cf0 0x7fa84406d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.393+0000 7fa84a000640 1 --2- 172.21.15.19:0/2755720735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa84415bf50 0x7fa8440b37b0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.393+0000 7fa84a000640 1 -- 172.21.15.19:0/2755720735 >> 172.21.15.19:0/2755720735 conn(0x7fa8440b1ba0 msgr2=0x7fa8440b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:56.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.393+0000 7fa84a000640 1 -- 172.21.15.19:0/2755720735 shutdown_connections 2024-03-28T13:19:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.393+0000 7fa84a000640 1 -- 172.21.15.19:0/2755720735 wait complete. 2024-03-28T13:19:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.395+0000 7fa84a000640 1 Processor -- start 2024-03-28T13:19:56.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.395+0000 7fa84a000640 1 -- start start 2024-03-28T13:19:56.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.396+0000 7fa84a000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa84406db60 0x7fa8440af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.396+0000 7fa84a000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8440b3cf0 0x7fa8440c5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.396+0000 7fa84a000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa84415bf50 0x7fa84415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.396+0000 7fa84a000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa84415ed30 con 0x7fa84406db60 2024-03-28T13:19:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.396+0000 7fa84a000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8440b4130 con 0x7fa84415bf50 2024-03-28T13:19:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.396+0000 7fa84a000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa84406e7f0 con 0x7fa8440b3cf0 2024-03-28T13:19:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.396+0000 7fa848c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa84406db60 0x7fa8440af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.396+0000 7fa848c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa84406db60 0x7fa8440af4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47818/0 (socket says 172.21.15.19:47818) 2024-03-28T13:19:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.396+0000 7fa848c00640 1 -- 172.21.15.19:0/3398162698 learned_addr learned my addr 172.21.15.19:0/3398162698 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.396+0000 7fa848c00640 1 -- 172.21.15.19:0/3398162698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8440b3cf0 msgr2=0x7fa8440c5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.396+0000 7fa848c00640 1 --2- 172.21.15.19:0/3398162698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8440b3cf0 0x7fa8440c5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.396+0000 7fa848c00640 1 -- 172.21.15.19:0/3398162698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa84415bf50 msgr2=0x7fa84415ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.396+0000 7fa848c00640 1 --2- 172.21.15.19:0/3398162698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa84415bf50 0x7fa84415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.396+0000 7fa848c00640 1 -- 172.21.15.19:0/3398162698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8440afaa0 con 0x7fa84406db60 2024-03-28T13:19:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.397+0000 7fa848c00640 1 --2- 172.21.15.19:0/3398162698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa84406db60 0x7fa8440af4c0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa83802ba50 tx=0x7fa838004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.397+0000 7fa840c00640 1 -- 172.21.15.19:0/3398162698 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838047070 con 0x7fa84406db60 2024-03-28T13:19:56.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.397+0000 7fa840c00640 1 -- 172.21.15.19:0/3398162698 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa838037d30 con 0x7fa84406db60 2024-03-28T13:19:56.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.397+0000 7fa840c00640 1 -- 172.21.15.19:0/3398162698 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8380426c0 con 0x7fa84406db60 2024-03-28T13:19:56.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.397+0000 7fa84a000640 1 -- 172.21.15.19:0/3398162698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8440afd80 con 0x7fa84406db60 2024-03-28T13:19:56.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.397+0000 7fa84a000640 1 -- 172.21.15.19:0/3398162698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8440b0620 con 0x7fa84406db60 2024-03-28T13:19:56.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.398+0000 7fa84a000640 1 -- 172.21.15.19:0/3398162698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8440aff50 con 0x7fa84406db60 2024-03-28T13:19:56.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.399+0000 7fa840c00640 1 -- 172.21.15.19:0/3398162698 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa83804a070 con 0x7fa84406db60 2024-03-28T13:19:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.400+0000 7fa840c00640 1 --2- 172.21.15.19:0/3398162698 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa82406c530 0x7fa82406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.400+0000 7fa840c00640 1 -- 172.21.15.19:0/3398162698 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7fa8380ad9d0 con 0x7fa84406db60 2024-03-28T13:19:56.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.401+0000 7fa843e00640 1 --2- 172.21.15.19:0/3398162698 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa82406c530 0x7fa82406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.402+0000 7fa843e00640 1 --2- 172.21.15.19:0/3398162698 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa82406c530 0x7fa82406e9f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa8440b3070 tx=0x7fa830008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.405+0000 7fa840c00640 1 -- 172.21.15.19:0/3398162698 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa838033030 con 0x7fa84406db60 2024-03-28T13:19:56.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.653+0000 7fa84a000640 1 -- 172.21.15.19:0/3398162698 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fa844075940 con 0x7fa84406db60 2024-03-28T13:19:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.732+0000 7fa840c00640 1 -- 172.21.15.19:0/3398162698 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v85) ==== 162+0+0 (secure 0 0 0) 0x7fa83807bec0 con 0x7fa84406db60 2024-03-28T13:19:56.733 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-28T13:19:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.736+0000 7fa84a000640 1 -- 172.21.15.19:0/3398162698 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa82406c530 msgr2=0x7fa82406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.736+0000 7fa84a000640 1 --2- 172.21.15.19:0/3398162698 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa82406c530 0x7fa82406e9f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa8440b3070 tx=0x7fa830008040 comp rx=0 tx=0).stop 2024-03-28T13:19:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.736+0000 7fa84a000640 1 -- 172.21.15.19:0/3398162698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa84406db60 msgr2=0x7fa8440af4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.736+0000 7fa84a000640 1 --2- 172.21.15.19:0/3398162698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa84406db60 0x7fa8440af4c0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa83802ba50 tx=0x7fa838004730 comp rx=0 tx=0).stop 2024-03-28T13:19:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.736+0000 7fa84a000640 1 -- 172.21.15.19:0/3398162698 shutdown_connections 2024-03-28T13:19:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.736+0000 7fa84a000640 1 --2- 172.21.15.19:0/3398162698 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa82406c530 0x7fa82406e9f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.736+0000 7fa84a000640 1 --2- 172.21.15.19:0/3398162698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa84415bf50 0x7fa84415ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.736+0000 7fa84a000640 1 --2- 172.21.15.19:0/3398162698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8440b3cf0 0x7fa8440c5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.736+0000 7fa84a000640 1 --2- 172.21.15.19:0/3398162698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa84406db60 0x7fa8440af4c0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.736+0000 7fa84a000640 1 -- 172.21.15.19:0/3398162698 >> 172.21.15.19:0/3398162698 conn(0x7fa8440b1ba0 msgr2=0x7fa8440b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:56.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.736+0000 7fa84a000640 1 -- 172.21.15.19:0/3398162698 shutdown_connections 2024-03-28T13:19:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.736+0000 7fa84a000640 1 -- 172.21.15.19:0/3398162698 wait complete. 2024-03-28T13:19:56.758 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:19:56.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.960+0000 7f8812e00640 1 Processor -- start 2024-03-28T13:19:56.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.961+0000 7f8812e00640 1 -- start start 2024-03-28T13:19:56.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.962+0000 7f8812e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f880c0c8040 0x7f880c15fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.962+0000 7f8812e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f880c0c84a0 0x7f880c160270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.962+0000 7f8812e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f880c0b4c00 con 0x7f880c06c680 2024-03-28T13:19:56.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.962+0000 7f8812e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f880c0c7310 con 0x7f880c0c8040 2024-03-28T13:19:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.962+0000 7f8812e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f880c0c7560 con 0x7f880c0c84a0 2024-03-28T13:19:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.963+0000 7f8803e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f880c0c8040 0x7f880c15fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.963+0000 7f8803e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f880c0c8040 0x7f880c15fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47834/0 (socket says 172.21.15.19:47834) 2024-03-28T13:19:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.963+0000 7f8803e00640 1 -- 172.21.15.19:0/3243011143 learned_addr learned my addr 172.21.15.19:0/3243011143 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.963+0000 7f8803e00640 1 -- 172.21.15.19:0/3243011143 >> v2:172.21.15.188:3300/0 conn(0x7f880c0c84a0 msgr2=0x7f880c160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.963+0000 7f8803e00640 1 --2- 172.21.15.19:0/3243011143 >> v2:172.21.15.188:3300/0 conn(0x7f880c0c84a0 0x7f880c160270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.963+0000 7f8803e00640 1 -- 172.21.15.19:0/3243011143 >> v1:172.21.15.188:6789/0 conn(0x7f880c06c680 legacy=0x7f880c06ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:56.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.963+0000 7f8803e00640 1 -- 172.21.15.19:0/3243011143 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c0c7840 con 0x7f880c0c8040 2024-03-28T13:19:56.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.965+0000 7f8803e00640 1 --2- 172.21.15.19:0/3243011143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f880c0c8040 0x7f880c15fd30 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f880c070d20 tx=0x7f87f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9901bf0b9f46c5f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.965+0000 7f880b400640 1 -- 172.21.15.19:0/3243011143 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4035020 con 0x7f880c0c8040 2024-03-28T13:19:56.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.965+0000 7f880b400640 1 -- 172.21.15.19:0/3243011143 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f4031870 con 0x7f880c0c8040 2024-03-28T13:19:56.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.965+0000 7f880b400640 1 -- 172.21.15.19:0/3243011143 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4031c40 con 0x7f880c0c8040 2024-03-28T13:19:56.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.966+0000 7f8812e00640 1 -- 172.21.15.19:0/3243011143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f880c0c8040 msgr2=0x7f880c15fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.966+0000 7f8812e00640 1 --2- 172.21.15.19:0/3243011143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f880c0c8040 0x7f880c15fd30 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f880c070d20 tx=0x7f87f402b910 comp rx=0 tx=0).stop 2024-03-28T13:19:56.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.966+0000 7f8812e00640 1 -- 172.21.15.19:0/3243011143 shutdown_connections 2024-03-28T13:19:56.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.966+0000 7f8812e00640 1 --2- 172.21.15.19:0/3243011143 >> v2:172.21.15.188:3300/0 conn(0x7f880c0c84a0 0x7f880c160270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.966+0000 7f8812e00640 1 --2- 172.21.15.19:0/3243011143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f880c0c8040 0x7f880c15fd30 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.966+0000 7f8812e00640 1 -- 172.21.15.19:0/3243011143 >> 172.21.15.19:0/3243011143 conn(0x7f880c0d2240 msgr2=0x7f880c0d2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:56.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.966+0000 7f8812e00640 1 -- 172.21.15.19:0/3243011143 shutdown_connections 2024-03-28T13:19:56.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.966+0000 7f8812e00640 1 -- 172.21.15.19:0/3243011143 wait complete. 2024-03-28T13:19:56.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.968+0000 7f8812e00640 1 Processor -- start 2024-03-28T13:19:56.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.968+0000 7f8812e00640 1 -- start start 2024-03-28T13:19:56.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.969+0000 7f8812e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f880c06c680 0x7f880c1f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.969+0000 7f8812e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f880c0c8040 0x7f880c0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.969+0000 7f8812e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f880c0c84a0 0x7f880c0c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.969+0000 7f8812e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f880c1481b0 con 0x7f880c0c84a0 2024-03-28T13:19:56.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.969+0000 7f8812e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f880c148030 con 0x7f880c0c8040 2024-03-28T13:19:56.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.969+0000 7f8812e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f880c0b3e90 con 0x7f880c06c680 2024-03-28T13:19:56.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.969+0000 7f8810e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f880c0c84a0 0x7f880c0c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.969+0000 7f8810e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f880c0c84a0 0x7f880c0c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47842/0 (socket says 172.21.15.19:47842) 2024-03-28T13:19:56.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.969+0000 7f8810e00640 1 -- 172.21.15.19:0/691021603 learned_addr learned my addr 172.21.15.19:0/691021603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:56.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.969+0000 7f880be00640 1 --2- 172.21.15.19:0/691021603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f880c06c680 0x7f880c1f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.969+0000 7f8803e00640 1 --2- 172.21.15.19:0/691021603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f880c0c8040 0x7f880c0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.970+0000 7f8810e00640 1 -- 172.21.15.19:0/691021603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f880c06c680 msgr2=0x7f880c1f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.970+0000 7f8810e00640 1 --2- 172.21.15.19:0/691021603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f880c06c680 0x7f880c1f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.970+0000 7f8810e00640 1 -- 172.21.15.19:0/691021603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f880c0c8040 msgr2=0x7f880c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.970+0000 7f8810e00640 1 --2- 172.21.15.19:0/691021603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f880c0c8040 0x7f880c0cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.970+0000 7f8810e00640 1 -- 172.21.15.19:0/691021603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c0ccf10 con 0x7f880c0c84a0 2024-03-28T13:19:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.970+0000 7f8803e00640 1 --2- 172.21.15.19:0/691021603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f880c0c8040 0x7f880c0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:19:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.970+0000 7f8810e00640 1 --2- 172.21.15.19:0/691021603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f880c0c84a0 0x7f880c0c9970 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f880400b970 tx=0x7f880400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.970+0000 7f8808c00640 1 -- 172.21.15.19:0/691021603 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804019070 con 0x7f880c0c84a0 2024-03-28T13:19:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.970+0000 7f8808c00640 1 -- 172.21.15.19:0/691021603 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880400cb60 con 0x7f880c0c84a0 2024-03-28T13:19:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.970+0000 7f8812e00640 1 -- 172.21.15.19:0/691021603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c0cd1f0 con 0x7f880c0c84a0 2024-03-28T13:19:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.970+0000 7f8808c00640 1 -- 172.21.15.19:0/691021603 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804014440 con 0x7f880c0c84a0 2024-03-28T13:19:56.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.970+0000 7f8812e00640 1 -- 172.21.15.19:0/691021603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c0cd7b0 con 0x7f880c0c84a0 2024-03-28T13:19:56.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.971+0000 7f8812e00640 1 -- 172.21.15.19:0/691021603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880c0cc5e0 con 0x7f880c0c84a0 2024-03-28T13:19:56.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.972+0000 7f8808c00640 1 -- 172.21.15.19:0/691021603 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f880401c070 con 0x7f880c0c84a0 2024-03-28T13:19:56.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.973+0000 7f8808c00640 1 --2- 172.21.15.19:0/691021603 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f87e806c530 0x7f87e806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:56.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.973+0000 7f8808c00640 1 -- 172.21.15.19:0/691021603 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f880408b540 con 0x7f880c0c84a0 2024-03-28T13:19:56.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.974+0000 7f880be00640 1 --2- 172.21.15.19:0/691021603 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f87e806c530 0x7f87e806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:56.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.975+0000 7f880be00640 1 --2- 172.21.15.19:0/691021603 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f87e806c530 0x7f87e806e9f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f87fc007fd0 tx=0x7f87fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:56.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:56.978+0000 7f8808c00640 1 -- 172.21.15.19:0/691021603 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8804059ac0 con 0x7f880c0c84a0 2024-03-28T13:19:57.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:57.223+0000 7f8812e00640 1 -- 172.21.15.19:0/691021603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f880c0685c0 con 0x7f880c0c84a0 2024-03-28T13:19:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:57.224+0000 7f8808c00640 1 -- 172.21.15.19:0/691021603 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5184 (secure 0 0 0) 0x7f88040593c0 con 0x7f880c0c84a0 2024-03-28T13:19:57.224 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:57.224 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4247326257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4247326257},{"type":"v1","addr":"172.21.15.188:6836","nonce":4247326257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3452274156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3452274156},{"type":"v1","addr":"172.21.15.19:6835","nonce":3452274156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3383190607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3383190607},{"type":"v1","addr":"172.21.15.188:6837","nonce":3383190607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:18:58.541599+0000","modified":"2024-03-28T13:19:56.729780+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8379":{"gid":8379,"name":"c","rank":0,"incarnation":85,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6834/2851646201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2851646201},{"type":"v1","addr":"172.21.15.19:6834","nonce":2851646201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-03-28T13:19:57.224 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 85 2024-03-28T13:19:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:57.227+0000 7f8812e00640 1 -- 172.21.15.19:0/691021603 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f87e806c530 msgr2=0x7f87e806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:57.227+0000 7f8812e00640 1 --2- 172.21.15.19:0/691021603 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f87e806c530 0x7f87e806e9f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f87fc007fd0 tx=0x7f87fc009040 comp rx=0 tx=0).stop 2024-03-28T13:19:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:57.227+0000 7f8812e00640 1 -- 172.21.15.19:0/691021603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f880c0c84a0 msgr2=0x7f880c0c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:57.227+0000 7f8812e00640 1 --2- 172.21.15.19:0/691021603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f880c0c84a0 0x7f880c0c9970 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f880400b970 tx=0x7f880400be30 comp rx=0 tx=0).stop 2024-03-28T13:19:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:57.227+0000 7f8812e00640 1 -- 172.21.15.19:0/691021603 shutdown_connections 2024-03-28T13:19:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:57.227+0000 7f8812e00640 1 --2- 172.21.15.19:0/691021603 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f87e806c530 0x7f87e806e9f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:57.227+0000 7f8812e00640 1 --2- 172.21.15.19:0/691021603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f880c0c84a0 0x7f880c0c9970 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:57.227+0000 7f8812e00640 1 --2- 172.21.15.19:0/691021603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f880c0c8040 0x7f880c0cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:57.227+0000 7f8812e00640 1 --2- 172.21.15.19:0/691021603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f880c06c680 0x7f880c1f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:57.227+0000 7f8812e00640 1 -- 172.21.15.19:0/691021603 >> 172.21.15.19:0/691021603 conn(0x7f880c0d2240 msgr2=0x7f880c0d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:57.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:57.227+0000 7f8812e00640 1 -- 172.21.15.19:0/691021603 shutdown_connections 2024-03-28T13:19:57.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:57.227+0000 7f8812e00640 1 -- 172.21.15.19:0/691021603 wait complete. 2024-03-28T13:19:57.649 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:18:58.541599+0000', 'modified': '2024-03-28T13:19:56.729780+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8379}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8379': {'gid': 8379, 'name': 'c', 'rank': 0, 'incarnation': 85, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6834/2851646201', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 2851646201}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 2851646201}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-28T13:19:57.649 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8379:up:replay 2024-03-28T13:19:58.650 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:19:58.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.883+0000 7f0840e00640 1 Processor -- start 2024-03-28T13:19:58.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.883+0000 7f0840e00640 1 -- start start 2024-03-28T13:19:58.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.885+0000 7f0840e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c0cc0b0 0x7f083c06dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:58.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.885+0000 7f0840e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f083c15fc50 0x7f083c0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:58.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.885+0000 7f0840e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f083c06e4f0 con 0x7f083c0cc490 2024-03-28T13:19:58.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.885+0000 7f0840e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f083c15c610 con 0x7f083c15fc50 2024-03-28T13:19:58.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.885+0000 7f0840e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f083c0b3150 con 0x7f083c0cc0b0 2024-03-28T13:19:58.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.885+0000 7f0833e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f083c15fc50 0x7f083c0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:58.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.885+0000 7f083be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f083c0cc490 0x7f083c15ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50716/0 (socket says 172.21.15.19:50716) 2024-03-28T13:19:58.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.885+0000 7f083be00640 1 -- 172.21.15.19:0/2177201233 learned_addr learned my addr 172.21.15.19:0/2177201233 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:19:58.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.885+0000 7f083b400640 1 --2- 172.21.15.19:0/2177201233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c0cc0b0 0x7f083c06dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:58.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.886+0000 7f0833e00640 1 -- 172.21.15.19:0/2177201233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c0cc0b0 msgr2=0x7f083c06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:19:58.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.886+0000 7f0833e00640 1 --2- 172.21.15.19:0/2177201233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c0cc0b0 0x7f083c06dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:58.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.886+0000 7f0833e00640 1 -- 172.21.15.19:0/2177201233 >> v1:172.21.15.19:6789/0 conn(0x7f083c0cc490 legacy=0x7f083c15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:58.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.886+0000 7f0833e00640 1 -- 172.21.15.19:0/2177201233 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c1600e0 con 0x7f083c15fc50 2024-03-28T13:19:58.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.887+0000 7f0833e00640 1 --2- 172.21.15.19:0/2177201233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f083c15fc50 0x7f083c0cfdc0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f08240097a0 tx=0x7f082402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b98e30db69a94b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:58.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.887+0000 7f083aa00640 1 -- 172.21.15.19:0/2177201233 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824035020 con 0x7f083c15fc50 2024-03-28T13:19:58.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.887+0000 7f083aa00640 1 -- 172.21.15.19:0/2177201233 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0824031810 con 0x7f083c15fc50 2024-03-28T13:19:58.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.887+0000 7f083aa00640 1 -- 172.21.15.19:0/2177201233 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0824031be0 con 0x7f083c15fc50 2024-03-28T13:19:58.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.888+0000 7f0840e00640 1 -- 172.21.15.19:0/2177201233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f083c15fc50 msgr2=0x7f083c0cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:58.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.888+0000 7f0840e00640 1 --2- 172.21.15.19:0/2177201233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f083c15fc50 0x7f083c0cfdc0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f08240097a0 tx=0x7f082402b8b0 comp rx=0 tx=0).stop 2024-03-28T13:19:58.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.888+0000 7f0840e00640 1 -- 172.21.15.19:0/2177201233 shutdown_connections 2024-03-28T13:19:58.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.888+0000 7f0840e00640 1 --2- 172.21.15.19:0/2177201233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f083c15fc50 0x7f083c0cfdc0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:58.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.888+0000 7f0840e00640 1 --2- 172.21.15.19:0/2177201233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c0cc0b0 0x7f083c06dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:58.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.888+0000 7f0840e00640 1 -- 172.21.15.19:0/2177201233 >> 172.21.15.19:0/2177201233 conn(0x7f083c0ccad0 msgr2=0x7f083c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:58.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.888+0000 7f0840e00640 1 -- 172.21.15.19:0/2177201233 shutdown_connections 2024-03-28T13:19:58.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.888+0000 7f0840e00640 1 -- 172.21.15.19:0/2177201233 wait complete. 2024-03-28T13:19:58.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.890+0000 7f0840e00640 1 Processor -- start 2024-03-28T13:19:58.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.890+0000 7f0840e00640 1 -- start start 2024-03-28T13:19:58.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.891+0000 7f0840e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f083c0cc0b0 0x7f083c13e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:58.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.891+0000 7f0840e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c0cc490 0x7f083c15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.891+0000 7f0840e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f083c15b670 0x7f083c0cee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.891+0000 7f0840e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f083c160980 con 0x7f083c15b670 2024-03-28T13:19:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.891+0000 7f0840e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1606a0 con 0x7f083c0cc0b0 2024-03-28T13:19:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.891+0000 7f0840e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f083c0d0300 con 0x7f083c0cc490 2024-03-28T13:19:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.891+0000 7f083be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f083c15b670 0x7f083c0cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.891+0000 7f083be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f083c15b670 0x7f083c0cee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47864/0 (socket says 172.21.15.19:47864) 2024-03-28T13:19:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.891+0000 7f083be00640 1 -- 172.21.15.19:0/3410924284 learned_addr learned my addr 172.21.15.19:0/3410924284 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:19:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.891+0000 7f0833e00640 1 --2- 172.21.15.19:0/3410924284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c0cc490 0x7f083c15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.891+0000 7f083b400640 1 --2- 172.21.15.19:0/3410924284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f083c0cc0b0 0x7f083c13e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.891+0000 7f083be00640 1 -- 172.21.15.19:0/3410924284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c0cc490 msgr2=0x7f083c15b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.891+0000 7f083be00640 1 --2- 172.21.15.19:0/3410924284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c0cc490 0x7f083c15b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.892+0000 7f083be00640 1 -- 172.21.15.19:0/3410924284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f083c0cc0b0 msgr2=0x7f083c13e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:58.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.892+0000 7f083be00640 1 --2- 172.21.15.19:0/3410924284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f083c0cc0b0 0x7f083c13e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.892+0000 7f083be00640 1 -- 172.21.15.19:0/3410924284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c1f4740 con 0x7f083c15b670 2024-03-28T13:19:58.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.892+0000 7f083be00640 1 --2- 172.21.15.19:0/3410924284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f083c15b670 0x7f083c0cee20 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f083400cbc0 tx=0x7f083400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.892+0000 7f0833400640 1 -- 172.21.15.19:0/3410924284 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834017070 con 0x7f083c15b670 2024-03-28T13:19:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.892+0000 7f0833400640 1 -- 172.21.15.19:0/3410924284 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083400ad40 con 0x7f083c15b670 2024-03-28T13:19:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.892+0000 7f0833400640 1 -- 172.21.15.19:0/3410924284 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834012600 con 0x7f083c15b670 2024-03-28T13:19:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.892+0000 7f0840e00640 1 -- 172.21.15.19:0/3410924284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083c1f49e0 con 0x7f083c15b670 2024-03-28T13:19:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.892+0000 7f0840e00640 1 -- 172.21.15.19:0/3410924284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083c1f4fa0 con 0x7f083c15b670 2024-03-28T13:19:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.894+0000 7f0840e00640 1 -- 172.21.15.19:0/3410924284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f083c0d0120 con 0x7f083c15b670 2024-03-28T13:19:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.894+0000 7f0833400640 1 -- 172.21.15.19:0/3410924284 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f083401a070 con 0x7f083c15b670 2024-03-28T13:19:58.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.895+0000 7f0833400640 1 --2- 172.21.15.19:0/3410924284 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f081806c530 0x7f081806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:19:58.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.895+0000 7f0833400640 1 -- 172.21.15.19:0/3410924284 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f083408b3a0 con 0x7f083c15b670 2024-03-28T13:19:58.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.896+0000 7f083b400640 1 --2- 172.21.15.19:0/3410924284 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f081806c530 0x7f081806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:19:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.897+0000 7f083b400640 1 --2- 172.21.15.19:0/3410924284 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f081806c530 0x7f081806e9f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f082c006fd0 tx=0x7f082c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:19:58.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:58.902+0000 7f0833400640 1 -- 172.21.15.19:0/3410924284 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f083c0d0120 con 0x7f083c15b670 2024-03-28T13:19:59.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:59.151+0000 7f0840e00640 1 -- 172.21.15.19:0/3410924284 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f083c075940 con 0x7f083c15b670 2024-03-28T13:19:59.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:59.152+0000 7f0833400640 1 -- 172.21.15.19:0/3410924284 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+5184 (secure 0 0 0) 0x7f08340592d0 con 0x7f083c15b670 2024-03-28T13:19:59.152 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:19:59.152 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4247326257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4247326257},{"type":"v1","addr":"172.21.15.188:6836","nonce":4247326257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3452274156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3452274156},{"type":"v1","addr":"172.21.15.19:6835","nonce":3452274156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3383190607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3383190607},{"type":"v1","addr":"172.21.15.188:6837","nonce":3383190607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:18:58.541599+0000","modified":"2024-03-28T13:19:57.740581+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8379":{"gid":8379,"name":"c","rank":0,"incarnation":85,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.19:6834/2851646201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2851646201},{"type":"v1","addr":"172.21.15.19:6834","nonce":2851646201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-03-28T13:19:59.153 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 87 2024-03-28T13:19:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:59.155+0000 7f0840e00640 1 -- 172.21.15.19:0/3410924284 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f081806c530 msgr2=0x7f081806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:59.155+0000 7f0840e00640 1 --2- 172.21.15.19:0/3410924284 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f081806c530 0x7f081806e9f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f082c006fd0 tx=0x7f082c008040 comp rx=0 tx=0).stop 2024-03-28T13:19:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:59.155+0000 7f0840e00640 1 -- 172.21.15.19:0/3410924284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f083c15b670 msgr2=0x7f083c0cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:19:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:59.155+0000 7f0840e00640 1 --2- 172.21.15.19:0/3410924284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f083c15b670 0x7f083c0cee20 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f083400cbc0 tx=0x7f083400a430 comp rx=0 tx=0).stop 2024-03-28T13:19:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:59.155+0000 7f0840e00640 1 -- 172.21.15.19:0/3410924284 shutdown_connections 2024-03-28T13:19:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:59.155+0000 7f0840e00640 1 --2- 172.21.15.19:0/3410924284 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f081806c530 0x7f081806e9f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:59.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:59.155+0000 7f0840e00640 1 --2- 172.21.15.19:0/3410924284 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f083c15b670 0x7f083c0cee20 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:59.155+0000 7f0840e00640 1 --2- 172.21.15.19:0/3410924284 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f083c0cc490 0x7f083c15b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:59.155+0000 7f0840e00640 1 --2- 172.21.15.19:0/3410924284 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f083c0cc0b0 0x7f083c13e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:19:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:59.155+0000 7f0840e00640 1 -- 172.21.15.19:0/3410924284 >> 172.21.15.19:0/3410924284 conn(0x7f083c0ccad0 msgr2=0x7f083c15dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:19:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:59.156+0000 7f0840e00640 1 -- 172.21.15.19:0/3410924284 shutdown_connections 2024-03-28T13:19:59.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:19:59.156+0000 7f0840e00640 1 -- 172.21.15.19:0/3410924284 wait complete. 2024-03-28T13:19:59.579 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:18:58.541599+0000', 'modified': '2024-03-28T13:19:57.740581+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8379}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8379': {'gid': 8379, 'name': 'c', 'rank': 0, 'incarnation': 85, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.19:6834/2851646201', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 2851646201}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 2851646201}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-28T13:19:59.579 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8379:up:rejoin 2024-03-28T13:20:00.580 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:20:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.810+0000 7f6c82c00640 1 Processor -- start 2024-03-28T13:20:00.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.810+0000 7f6c82c00640 1 -- start start 2024-03-28T13:20:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.811+0000 7f6c82c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0af830 0x7f6c7c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.811+0000 7f6c82c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c7c15fe80 con 0x7f6c7c15c390 2024-03-28T13:20:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.811+0000 7f6c82c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c7c0ad120 con 0x7f6c7c0d0650 2024-03-28T13:20:00.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.811+0000 7f6c82c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7c15c770 con 0x7f6c7c0af830 2024-03-28T13:20:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.812+0000 7f6c7be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f6c7c15c390 0x7f6c7c160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50718/0 (socket says 172.21.15.19:50718) 2024-03-28T13:20:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.812+0000 7f6c7be00640 1 -- 172.21.15.19:0/257319112 learned_addr learned my addr 172.21.15.19:0/257319112 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:00.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.812+0000 7f6c7b400640 1 --2- 172.21.15.19:0/257319112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0af830 0x7f6c7c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:00.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3928754666 0 0) 0x7f6c7c15fe80 con 0x7f6c7c15c390 2024-03-28T13:20:00.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c60003520 con 0x7f6c7c15c390 2024-03-28T13:20:00.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3113897227 0 0) 0x7f6c7c0ad120 con 0x7f6c7c0d0650 2024-03-28T13:20:00.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c7c15fe80 con 0x7f6c7c0d0650 2024-03-28T13:20:00.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 266854794 0 0) 0x7f6c60003520 con 0x7f6c7c15c390 2024-03-28T13:20:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0af830 msgr2=0x7f6c7c073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 --2- 172.21.15.19:0/257319112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0af830 0x7f6c7c073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 >> v1:172.21.15.188:6789/0 conn(0x7f6c7c0d0650 legacy=0x7f6c7c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c0ad400 con 0x7f6c7c15c390 2024-03-28T13:20:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f6c6c0019f0 con 0x7f6c7c15c390 2024-03-28T13:20:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 >> v1:172.21.15.19:6789/0 conn(0x7f6c7c15c390 legacy=0x7f6c7c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 --2- 172.21.15.19:0/257319112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c60003b70 0x7f6c60003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 --2- 172.21.15.19:0/257319112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c60004800 0x7f6c60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 --2- 172.21.15.19:0/257319112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c600051c0 0x7f6c6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6000bfb0 con 0x7f6c60003b70 2024-03-28T13:20:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6c6000c130 con 0x7f6c600051c0 2024-03-28T13:20:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6c6000c330 con 0x7f6c60004800 2024-03-28T13:20:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.813+0000 7f6c7b400640 1 --2- 172.21.15.19:0/257319112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0af830 0x7f6c7c073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:20:00.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.814+0000 7f6c7be00640 1 --2- 172.21.15.19:0/257319112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c60003b70 0x7f6c60003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.814+0000 7f6c80c00640 1 --2- 172.21.15.19:0/257319112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c600051c0 0x7f6c6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.814+0000 7f6c7be00640 1 -- 172.21.15.19:0/257319112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c60004800 msgr2=0x7f6c60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.814+0000 7f6c7b400640 1 --2- 172.21.15.19:0/257319112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c60004800 0x7f6c60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.814+0000 7f6c7be00640 1 --2- 172.21.15.19:0/257319112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c60004800 0x7f6c60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.814+0000 7f6c7be00640 1 -- 172.21.15.19:0/257319112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c600051c0 msgr2=0x7f6c6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.814+0000 7f6c7be00640 1 --2- 172.21.15.19:0/257319112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c600051c0 0x7f6c6000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.814+0000 7f6c7be00640 1 -- 172.21.15.19:0/257319112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c60003520 con 0x7f6c60003b70 2024-03-28T13:20:00.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.814+0000 7f6c7b400640 1 --2- 172.21.15.19:0/257319112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c60004800 0x7f6c60004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.816+0000 7f6c7be00640 1 --2- 172.21.15.19:0/257319112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c60003b70 0x7f6c60003f70 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00a7c0 tx=0x7f6c60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d1edd46365ecd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.816+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c00a410 con 0x7f6c60003b70 2024-03-28T13:20:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.816+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c6c034650 con 0x7f6c60003b70 2024-03-28T13:20:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.816+0000 7f6c7aa00640 1 -- 172.21.15.19:0/257319112 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c02ce60 con 0x7f6c60003b70 2024-03-28T13:20:00.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.817+0000 7f6c82c00640 1 -- 172.21.15.19:0/257319112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c60003b70 msgr2=0x7f6c60003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.817+0000 7f6c82c00640 1 --2- 172.21.15.19:0/257319112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c60003b70 0x7f6c60003f70 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00a7c0 tx=0x7f6c60005830 comp rx=0 tx=0).stop 2024-03-28T13:20:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.817+0000 7f6c80c00640 1 -- 172.21.15.19:0/257319112 reap_dead start 2024-03-28T13:20:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.817+0000 7f6c82c00640 1 -- 172.21.15.19:0/257319112 shutdown_connections 2024-03-28T13:20:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.817+0000 7f6c82c00640 1 -- 172.21.15.19:0/257319112 >> 172.21.15.19:0/257319112 conn(0x7f6c7c06b990 msgr2=0x7f6c7c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.817+0000 7f6c82c00640 1 -- 172.21.15.19:0/257319112 shutdown_connections 2024-03-28T13:20:00.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.818+0000 7f6c82c00640 1 -- 172.21.15.19:0/257319112 wait complete. 2024-03-28T13:20:00.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.819+0000 7f6c82c00640 1 Processor -- start 2024-03-28T13:20:00.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.819+0000 7f6c82c00640 1 -- start start 2024-03-28T13:20:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c82c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c60003b70 0x7f6c7c1476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c82c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0ce9d0 0x7f6c7c0b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c82c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c7c1512b0 0x7f6c7c0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c82c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c160d50 con 0x7f6c60003b70 2024-03-28T13:20:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c82c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c160120 con 0x7f6c7c1512b0 2024-03-28T13:20:00.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c82c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7c0cf980 con 0x7f6c7c0ce9d0 2024-03-28T13:20:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c7be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c60003b70 0x7f6c7c1476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c7be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c60003b70 0x7f6c7c1476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47878/0 (socket says 172.21.15.19:47878) 2024-03-28T13:20:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c7be00640 1 -- 172.21.15.19:0/3761689619 learned_addr learned my addr 172.21.15.19:0/3761689619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c7b400640 1 --2- 172.21.15.19:0/3761689619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0ce9d0 0x7f6c7c0b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c80c00640 1 --2- 172.21.15.19:0/3761689619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c7c1512b0 0x7f6c7c0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c7be00640 1 -- 172.21.15.19:0/3761689619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0ce9d0 msgr2=0x7f6c7c0b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c7be00640 1 --2- 172.21.15.19:0/3761689619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0ce9d0 0x7f6c7c0b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c7be00640 1 -- 172.21.15.19:0/3761689619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c7c1512b0 msgr2=0x7f6c7c0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c7be00640 1 --2- 172.21.15.19:0/3761689619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c7c1512b0 0x7f6c7c0b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.821+0000 7f6c7be00640 1 -- 172.21.15.19:0/3761689619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c150f20 con 0x7f6c60003b70 2024-03-28T13:20:00.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.822+0000 7f6c7be00640 1 --2- 172.21.15.19:0/3761689619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c60003b70 0x7f6c7c1476d0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00a8b0 tx=0x7f6c7c0ad5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:00.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.822+0000 7f6c73e00640 1 -- 172.21.15.19:0/3761689619 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c034800 con 0x7f6c60003b70 2024-03-28T13:20:00.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.822+0000 7f6c73e00640 1 -- 172.21.15.19:0/3761689619 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c6c044070 con 0x7f6c60003b70 2024-03-28T13:20:00.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.822+0000 7f6c73e00640 1 -- 172.21.15.19:0/3761689619 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c034be0 con 0x7f6c60003b70 2024-03-28T13:20:00.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.822+0000 7f6c82c00640 1 -- 172.21.15.19:0/3761689619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7c150c40 con 0x7f6c60003b70 2024-03-28T13:20:00.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.822+0000 7f6c82c00640 1 -- 172.21.15.19:0/3761689619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7c0b5120 con 0x7f6c60003b70 2024-03-28T13:20:00.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.823+0000 7f6c82c00640 1 -- 172.21.15.19:0/3761689619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c7c0685c0 con 0x7f6c60003b70 2024-03-28T13:20:00.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.825+0000 7f6c73e00640 1 -- 172.21.15.19:0/3761689619 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6c6c049460 con 0x7f6c60003b70 2024-03-28T13:20:00.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.826+0000 7f6c73e00640 1 --2- 172.21.15.19:0/3761689619 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6c5c06c530 0x7f6c5c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:00.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.826+0000 7f6c73e00640 1 -- 172.21.15.19:0/3761689619 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f6c6c0adf60 con 0x7f6c60003b70 2024-03-28T13:20:00.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.826+0000 7f6c7b400640 1 --2- 172.21.15.19:0/3761689619 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6c5c06c530 0x7f6c5c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:00.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.827+0000 7f6c7b400640 1 --2- 172.21.15.19:0/3761689619 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6c5c06c530 0x7f6c5c06e9f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c15f950 tx=0x7f6c68008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:00.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:00.830+0000 7f6c73e00640 1 -- 172.21.15.19:0/3761689619 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6c6c032380 con 0x7f6c60003b70 2024-03-28T13:20:01.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.071+0000 7f6c82c00640 1 -- 172.21.15.19:0/3761689619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c7c06f1b0 con 0x7f6c60003b70 2024-03-28T13:20:01.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.072+0000 7f6c73e00640 1 -- 172.21.15.19:0/3761689619 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5191 (secure 0 0 0) 0x7f6c6c07c580 con 0x7f6c60003b70 2024-03-28T13:20:01.072 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:01.072 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":88,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4247326257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4247326257},{"type":"v1","addr":"172.21.15.188:6836","nonce":4247326257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3452274156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3452274156},{"type":"v1","addr":"172.21.15.19:6835","nonce":3452274156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3383190607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3383190607},{"type":"v1","addr":"172.21.15.188:6837","nonce":3383190607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:18:58.541599+0000","modified":"2024-03-28T13:19:59.751848+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":108,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8379},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8379":{"gid":8379,"name":"c","rank":0,"incarnation":85,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6834/2851646201","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":2851646201},{"type":"v1","addr":"172.21.15.19:6834","nonce":2851646201}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8379,"qdb_cluster":[8379]},"id":7}]} 2024-03-28T13:20:01.073 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 88 2024-03-28T13:20:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.074+0000 7f6c82c00640 1 -- 172.21.15.19:0/3761689619 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6c5c06c530 msgr2=0x7f6c5c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.074+0000 7f6c82c00640 1 --2- 172.21.15.19:0/3761689619 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6c5c06c530 0x7f6c5c06e9f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c15f950 tx=0x7f6c68008040 comp rx=0 tx=0).stop 2024-03-28T13:20:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.074+0000 7f6c82c00640 1 -- 172.21.15.19:0/3761689619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c60003b70 msgr2=0x7f6c7c1476d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.074+0000 7f6c82c00640 1 --2- 172.21.15.19:0/3761689619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c60003b70 0x7f6c7c1476d0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00a8b0 tx=0x7f6c7c0ad5d0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.075+0000 7f6c82c00640 1 -- 172.21.15.19:0/3761689619 shutdown_connections 2024-03-28T13:20:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.075+0000 7f6c82c00640 1 --2- 172.21.15.19:0/3761689619 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6c5c06c530 0x7f6c5c06e9f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.075+0000 7f6c82c00640 1 --2- 172.21.15.19:0/3761689619 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c7c1512b0 0x7f6c7c0b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.075+0000 7f6c82c00640 1 --2- 172.21.15.19:0/3761689619 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c7c0ce9d0 0x7f6c7c0b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.075+0000 7f6c82c00640 1 --2- 172.21.15.19:0/3761689619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c60003b70 0x7f6c7c1476d0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.075+0000 7f6c82c00640 1 -- 172.21.15.19:0/3761689619 >> 172.21.15.19:0/3761689619 conn(0x7f6c7c06b990 msgr2=0x7f6c7c0afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.075+0000 7f6c82c00640 1 -- 172.21.15.19:0/3761689619 shutdown_connections 2024-03-28T13:20:01.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.075+0000 7f6c82c00640 1 -- 172.21.15.19:0/3761689619 wait complete. 2024-03-28T13:20:01.497 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:18:58.541599+0000', 'modified': '2024-03-28T13:19:59.751848+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 108, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8379}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8379': {'gid': 8379, 'name': 'c', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.19:6834/2851646201', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6832', 'nonce': 2851646201}, {'type': 'v1', 'addr': '172.21.15.19:6834', 'nonce': 2851646201}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8379, 'qdb_cluster': [8379]} 2024-03-28T13:20:01.497 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T13:20:01.498 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-28T13:20:01.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.726+0000 7f07b7200640 1 Processor -- start 2024-03-28T13:20:01.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.727+0000 7f07b7200640 1 -- start start 2024-03-28T13:20:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.728+0000 7f07b7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07b80b5970 0x7f07b8071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.728+0000 7f07b7200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f07b80720f0 0x7f07b806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.728+0000 7f07b7200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07b80ce650 con 0x7f07b806ec60 2024-03-28T13:20:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.728+0000 7f07b7200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f07b80ce930 con 0x7f07b80720f0 2024-03-28T13:20:01.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.728+0000 7f07b7200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f07b80cec10 con 0x7f07b80b5970 2024-03-28T13:20:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.729+0000 7f07afe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07b80b5970 0x7f07b8071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.729+0000 7f07b5e00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f07b806ec60 0x7f07b815b760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:39750/0 (socket says 172.21.15.19:39750) 2024-03-28T13:20:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.729+0000 7f07b5e00640 1 -- 172.21.15.19:0/3431471798 learned_addr learned my addr 172.21.15.19:0/3431471798 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:01.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.729+0000 7f07b6800640 1 --2- 172.21.15.19:0/3431471798 >> v2:172.21.15.188:3300/0 conn(0x7f07b80720f0 0x7f07b806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.730+0000 7f07afe00640 1 -- 172.21.15.19:0/3431471798 >> v2:172.21.15.188:3300/0 conn(0x7f07b80720f0 msgr2=0x7f07b806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.730+0000 7f07afe00640 1 --2- 172.21.15.19:0/3431471798 >> v2:172.21.15.188:3300/0 conn(0x7f07b80720f0 0x7f07b806d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.730+0000 7f07afe00640 1 -- 172.21.15.19:0/3431471798 >> v1:172.21.15.188:6789/0 conn(0x7f07b806ec60 legacy=0x7f07b815b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.730+0000 7f07afe00640 1 -- 172.21.15.19:0/3431471798 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b80ceef0 con 0x7f07b80b5970 2024-03-28T13:20:01.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.730+0000 7f07b6800640 1 --2- 172.21.15.19:0/3431471798 >> v2:172.21.15.188:3300/0 conn(0x7f07b80720f0 0x7f07b806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:20:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.732+0000 7f07afe00640 1 --2- 172.21.15.19:0/3431471798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07b80b5970 0x7f07b8071bb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f07a00097a0 tx=0x7f07a002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fed8dbeba33d2dc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:01.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.732+0000 7f07b5400640 1 -- 172.21.15.19:0/3431471798 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0031780 con 0x7f07b80b5970 2024-03-28T13:20:01.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.732+0000 7f07b5400640 1 -- 172.21.15.19:0/3431471798 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a0037070 con 0x7f07b80b5970 2024-03-28T13:20:01.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.732+0000 7f07b5400640 1 -- 172.21.15.19:0/3431471798 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0031b40 con 0x7f07b80b5970 2024-03-28T13:20:01.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.733+0000 7f07b7200640 1 -- 172.21.15.19:0/3431471798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07b80b5970 msgr2=0x7f07b8071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:01.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.733+0000 7f07b7200640 1 --2- 172.21.15.19:0/3431471798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07b80b5970 0x7f07b8071bb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f07a00097a0 tx=0x7f07a002b820 comp rx=0 tx=0).stop 2024-03-28T13:20:01.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.733+0000 7f07b7200640 1 -- 172.21.15.19:0/3431471798 shutdown_connections 2024-03-28T13:20:01.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.733+0000 7f07b7200640 1 --2- 172.21.15.19:0/3431471798 >> v2:172.21.15.188:3300/0 conn(0x7f07b80720f0 0x7f07b806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.733+0000 7f07b7200640 1 --2- 172.21.15.19:0/3431471798 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07b80b5970 0x7f07b8071bb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.733+0000 7f07b7200640 1 -- 172.21.15.19:0/3431471798 >> 172.21.15.19:0/3431471798 conn(0x7f07b80ccad0 msgr2=0x7f07b80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:01.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.733+0000 7f07b7200640 1 -- 172.21.15.19:0/3431471798 shutdown_connections 2024-03-28T13:20:01.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.733+0000 7f07b7200640 1 -- 172.21.15.19:0/3431471798 wait complete. 2024-03-28T13:20:01.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.735+0000 7f07b7200640 1 Processor -- start 2024-03-28T13:20:01.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.735+0000 7f07b7200640 1 -- start start 2024-03-28T13:20:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.736+0000 7f07b7200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07b806ec60 0x7f07b8154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.736+0000 7f07b7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07b80720f0 0x7f07b80b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.736+0000 7f07b7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07b80b5970 0x7f07b8152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.736+0000 7f07b7200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07b806db10 con 0x7f07b80b5970 2024-03-28T13:20:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.736+0000 7f07b7200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8072500 con 0x7f07b806ec60 2024-03-28T13:20:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.736+0000 7f07b7200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f07b806dc90 con 0x7f07b80720f0 2024-03-28T13:20:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.737+0000 7f07b6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07b80b5970 0x7f07b8152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.737+0000 7f07b6800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07b80b5970 0x7f07b8152840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47890/0 (socket says 172.21.15.19:47890) 2024-03-28T13:20:01.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.737+0000 7f07b6800640 1 -- 172.21.15.19:0/749167981 learned_addr learned my addr 172.21.15.19:0/749167981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.737+0000 7f07b5e00640 1 --2- 172.21.15.19:0/749167981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07b806ec60 0x7f07b8154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.737+0000 7f07afe00640 1 --2- 172.21.15.19:0/749167981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07b80720f0 0x7f07b80b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.737+0000 7f07b6800640 1 -- 172.21.15.19:0/749167981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07b80720f0 msgr2=0x7f07b80b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.737+0000 7f07b6800640 1 --2- 172.21.15.19:0/749167981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07b80720f0 0x7f07b80b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.737+0000 7f07b6800640 1 -- 172.21.15.19:0/749167981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07b806ec60 msgr2=0x7f07b8154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.737+0000 7f07b6800640 1 --2- 172.21.15.19:0/749167981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07b806ec60 0x7f07b8154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.737+0000 7f07b6800640 1 -- 172.21.15.19:0/749167981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b80b3640 con 0x7f07b80b5970 2024-03-28T13:20:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.737+0000 7f07b5e00640 1 --2- 172.21.15.19:0/749167981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07b806ec60 0x7f07b8154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:20:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.738+0000 7f07b6800640 1 --2- 172.21.15.19:0/749167981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07b80b5970 0x7f07b8152840 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009f00 tx=0x7f07b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:01.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.738+0000 7f07ae000640 1 -- 172.21.15.19:0/749167981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0018030 con 0x7f07b80b5970 2024-03-28T13:20:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.738+0000 7f07ae000640 1 -- 172.21.15.19:0/749167981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07b000ce50 con 0x7f07b80b5970 2024-03-28T13:20:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.738+0000 7f07ae000640 1 -- 172.21.15.19:0/749167981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0003bb0 con 0x7f07b80b5970 2024-03-28T13:20:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.738+0000 7f07b7200640 1 -- 172.21.15.19:0/749167981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b80b3920 con 0x7f07b80b5970 2024-03-28T13:20:01.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.738+0000 7f07b7200640 1 -- 172.21.15.19:0/749167981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b80b3ee0 con 0x7f07b80b5970 2024-03-28T13:20:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.740+0000 7f07ae000640 1 -- 172.21.15.19:0/749167981 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f07b001b070 con 0x7f07b80b5970 2024-03-28T13:20:01.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.741+0000 7f07ae000640 1 --2- 172.21.15.19:0/749167981 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f079406c530 0x7f079406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.741+0000 7f07ae000640 1 -- 172.21.15.19:0/749167981 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f07b008c210 con 0x7f07b80b5970 2024-03-28T13:20:01.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.742+0000 7f07b5e00640 1 --2- 172.21.15.19:0/749167981 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f079406c530 0x7f079406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.743+0000 7f07b5e00640 1 --2- 172.21.15.19:0/749167981 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f079406c530 0x7f079406e9f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f07a8000f60 tx=0x7f07a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:01.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.744+0000 7f07bfa02740 1 Processor -- start 2024-03-28T13:20:01.744 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-03-28T13:20:01.744+0000 7fde1e400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:20:01.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.744+0000 7f07bfa02740 1 -- start start 2024-03-28T13:20:01.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.746+0000 7f07bfa02740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d91d0 0x55d03a3ac170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.746+0000 7f07bfa02740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3d95b0 0x55d03a3dbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.746+0000 7f07bfa02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3ac6b0 0x55d03a3dc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.746+0000 7f07bfa02740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d03a373920 con 0x55d03a3d91d0 2024-03-28T13:20:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.746+0000 7f07bfa02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d03a2c3fa0 con 0x55d03a3ac6b0 2024-03-28T13:20:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.746+0000 7f07bfa02740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55d03a315f90 con 0x55d03a3d95b0 2024-03-28T13:20:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.746+0000 7f07b5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d91d0 0x55d03a3ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.746+0000 7f07b5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d91d0 0x55d03a3ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47900/0 (socket says 172.21.15.19:47900) 2024-03-28T13:20:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.746+0000 7f07b5e00640 1 -- 172.21.15.19:0/2539821119 learned_addr learned my addr 172.21.15.19:0/2539821119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.747+0000 7f07b6800640 1 --2- 172.21.15.19:0/2539821119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3ac6b0 0x55d03a3dc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.747+0000 7f07afe00640 1 --2- 172.21.15.19:0/2539821119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3d95b0 0x55d03a3dbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.747+0000 7f07b5e00640 1 -- 172.21.15.19:0/2539821119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3d95b0 msgr2=0x55d03a3dbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.747+0000 7f07b5e00640 1 --2- 172.21.15.19:0/2539821119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3d95b0 0x55d03a3dbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.747+0000 7f07b5e00640 1 -- 172.21.15.19:0/2539821119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3ac6b0 msgr2=0x55d03a3dc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.747+0000 7f07b5e00640 1 --2- 172.21.15.19:0/2539821119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3ac6b0 0x55d03a3dc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.747+0000 7f07b5e00640 1 -- 172.21.15.19:0/2539821119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03a02a2d0 con 0x55d03a3d91d0 2024-03-28T13:20:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.747+0000 7f07afe00640 1 --2- 172.21.15.19:0/2539821119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3d95b0 0x55d03a3dbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:01.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.747+0000 7f07b5e00640 1 --2- 172.21.15.19:0/2539821119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d91d0 0x55d03a3ac170 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f07a80058e0 tx=0x7f07a8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97701a8023f9d187 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:01.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.747+0000 7f07a6600640 1 -- 172.21.15.19:0/2539821119 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8005e60 con 0x55d03a3d91d0 2024-03-28T13:20:01.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.747+0000 7f07a6600640 1 -- 172.21.15.19:0/2539821119 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a800e7e0 con 0x55d03a3d91d0 2024-03-28T13:20:01.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.747+0000 7f07a6600640 1 -- 172.21.15.19:0/2539821119 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a800eb90 con 0x55d03a3d91d0 2024-03-28T13:20:01.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.748+0000 7f07bfa02740 1 -- 172.21.15.19:0/2539821119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d91d0 msgr2=0x55d03a3ac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:01.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.748+0000 7f07bfa02740 1 --2- 172.21.15.19:0/2539821119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d91d0 0x55d03a3ac170 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f07a80058e0 tx=0x7f07a8011000 comp rx=0 tx=0).stop 2024-03-28T13:20:01.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.748+0000 7f07bfa02740 1 -- 172.21.15.19:0/2539821119 shutdown_connections 2024-03-28T13:20:01.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.748+0000 7f07bfa02740 1 --2- 172.21.15.19:0/2539821119 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3ac6b0 0x55d03a3dc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.748+0000 7f07bfa02740 1 --2- 172.21.15.19:0/2539821119 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3d95b0 0x55d03a3dbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.748+0000 7f07bfa02740 1 --2- 172.21.15.19:0/2539821119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d91d0 0x55d03a3ac170 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.748+0000 7f07bfa02740 1 -- 172.21.15.19:0/2539821119 >> 172.21.15.19:0/2539821119 conn(0x55d03a3ae3e0 msgr2=0x55d03a3f6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:01.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.748+0000 7f07bfa02740 1 -- 172.21.15.19:0/2539821119 shutdown_connections 2024-03-28T13:20:01.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.748+0000 7f07bfa02740 1 -- 172.21.15.19:0/2539821119 wait complete. 2024-03-28T13:20:01.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.749+0000 7f07bfa02740 1 -- start start 2024-03-28T13:20:01.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.749+0000 7f07bfa02740 1 Processor -- start 2024-03-28T13:20:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.750+0000 7f07bfa02740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3d91d0 0x55d03a3d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.750+0000 7f07bfa02740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d95b0 0x55d03a3f7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.750+0000 7f07bfa02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3f5700 0x55d03a3f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.750+0000 7f07bfa02740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d03a311f40 con 0x55d03a3d95b0 2024-03-28T13:20:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.750+0000 7f07bfa02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d03a2be650 con 0x55d03a3f5700 2024-03-28T13:20:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.750+0000 7f07bfa02740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55d03a2d1a10 con 0x55d03a3d91d0 2024-03-28T13:20:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.750+0000 7f07bfa02740 10 client.0 _ll_register_callbacks cb 0x55d03a310ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:20:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.750+0000 7f07afe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d95b0 0x55d03a3f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.750+0000 7f07afe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d95b0 0x55d03a3f7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47914/0 (socket says 172.21.15.19:47914) 2024-03-28T13:20:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.750+0000 7f07afe00640 1 -- 172.21.15.19:0/1969937312 learned_addr learned my addr 172.21.15.19:0/1969937312 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.751+0000 7f07b6800640 1 --2- 172.21.15.19:0/1969937312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3f5700 0x55d03a3f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.751+0000 7f07b5e00640 1 --2- 172.21.15.19:0/1969937312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3d91d0 0x55d03a3d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.751+0000 7f07afe00640 1 -- 172.21.15.19:0/1969937312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3d91d0 msgr2=0x55d03a3d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.751+0000 7f07afe00640 1 --2- 172.21.15.19:0/1969937312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3d91d0 0x55d03a3d16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.751+0000 7f07afe00640 1 -- 172.21.15.19:0/1969937312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3f5700 msgr2=0x55d03a3f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.751+0000 7f07afe00640 1 --2- 172.21.15.19:0/1969937312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3f5700 0x55d03a3f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.751+0000 7f07afe00640 1 -- 172.21.15.19:0/1969937312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03a0f46b0 con 0x55d03a3d95b0 2024-03-28T13:20:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.751+0000 7f07b6800640 1 --2- 172.21.15.19:0/1969937312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3f5700 0x55d03a3f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.751+0000 7f07afe00640 1 --2- 172.21.15.19:0/1969937312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d95b0 0x55d03a3f7900 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f07a002b860 tx=0x7f07a002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47eeeec08f3ed587 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.751+0000 7f079aa00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:20:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.751+0000 7f079aa00640 1 -- 172.21.15.19:0/1969937312 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0006e10 con 0x55d03a3d95b0 2024-03-28T13:20:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.751+0000 7f079aa00640 1 -- 172.21.15.19:0/1969937312 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f07a0032610 con 0x55d03a3d95b0 2024-03-28T13:20:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.751+0000 7f07bfa02740 1 -- 172.21.15.19:0/1969937312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a800e7e0 con 0x55d03a3d95b0 2024-03-28T13:20:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.752+0000 7f079aa00640 1 -- 172.21.15.19:0/1969937312 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f07a0032810 con 0x55d03a3d95b0 2024-03-28T13:20:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.752+0000 7f07bfa02740 10 client.8457 fetch_fsmap learned FSMap version 88 2024-03-28T13:20:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.752+0000 7f07bfa02740 1 -- 172.21.15.19:0/1969937312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55d03a3df440 con 0x55d03a3d95b0 2024-03-28T13:20:01.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.752+0000 7f079aa00640 1 -- 172.21.15.19:0/1969937312 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a800e7e0 con 0x55d03a3d95b0 2024-03-28T13:20:01.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.752+0000 7f079aa00640 1 -- 172.21.15.19:0/1969937312 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f07a0055020 con 0x55d03a3d95b0 2024-03-28T13:20:01.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.752+0000 7f07bfa02740 10 client.8457 fetch_fsmap finished waiting for FSMap version 88 2024-03-28T13:20:01.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.752+0000 7f07bfa02740 10 client.8457 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T13:20:01.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.752+0000 7f07bfa02740 20 client.8457 populate_metadata read hostname 'smithi019' 2024-03-28T13:20:01.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.752+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1969937312 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55d03a3f3980 0x55d03a3f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.752+0000 7f07bfa02740 4 client.8457 mds_command: new command op to 8379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:20:01.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.752+0000 7f07bfa02740 1 -- 172.21.15.19:0/1969937312 --> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d03a3234d0 con 0x55d03a3f3980 2024-03-28T13:20:01.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.752+0000 7f07b5e00640 1 --2- 172.21.15.19:0/1969937312 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55d03a3f3980 0x55d03a3f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.753+0000 7f07b5e00640 1 --2- 172.21.15.19:0/1969937312 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55d03a3f3980 0x55d03a3f3d60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd433237582b57b server_cookie=fa73239fd086ff49 in_seq=0 out_seq=0 2024-03-28T13:20:01.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.753+0000 7f079aa00640 10 client.8457 ms_handle_connect on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:01.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.754+0000 7f079aa00640 1 -- 172.21.15.19:0/1969937312 <== mds.0 v2:172.21.15.19:6832/2851646201 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d03a3234d0 con 0x55d03a3f3980 2024-03-28T13:20:01.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.754+0000 7f079aa00640 10 client.8457 handle_command_reply: tid=0 2024-03-28T13:20:01.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.754+0000 7f07bfa02740 1 client.8457 shutdown 2024-03-28T13:20:01.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.754+0000 7f07bfa02740 1 -- 172.21.15.19:0/1969937312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d95b0 msgr2=0x55d03a3f7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:01.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.754+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1969937312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d95b0 0x55d03a3f7900 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f07a002b860 tx=0x7f07a002e000 comp rx=0 tx=0).stop 2024-03-28T13:20:01.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.754+0000 7f07bfa02740 1 -- 172.21.15.19:0/1969937312 shutdown_connections 2024-03-28T13:20:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.754+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1969937312 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55d03a3f3980 0x55d03a3f3d60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.754+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1969937312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3f5700 0x55d03a3f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.754+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1969937312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d95b0 0x55d03a3f7900 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.754+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1969937312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3d91d0 0x55d03a3d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.754+0000 7f079aa00640 0 client.8457 ms_handle_reset on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.754+0000 7f07bfa02740 1 -- 172.21.15.19:0/1969937312 >> 172.21.15.19:0/1969937312 conn(0x55d03a3ac6b0 msgr2=0x55d03a3ab890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.755+0000 7f07bfa02740 1 -- 172.21.15.19:0/1969937312 shutdown_connections 2024-03-28T13:20:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.755+0000 7f07bfa02740 1 -- 172.21.15.19:0/1969937312 wait complete. 2024-03-28T13:20:01.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.755+0000 7f07bfa02740 20 client.8457 trim_cache size 0 max 16384 2024-03-28T13:20:01.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.777+0000 7f07bfa02740 1 Processor -- start 2024-03-28T13:20:01.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.777+0000 7f07bfa02740 1 -- start start 2024-03-28T13:20:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.778+0000 7f07bfa02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3d91d0 0x55d03a3de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.778+0000 7f07bfa02740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d95b0 0x55d03a3fa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.778+0000 7f07bfa02740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3f5700 0x55d03a3fa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.778+0000 7f07bfa02740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8168ee0 con 0x55d03a3d95b0 2024-03-28T13:20:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.778+0000 7f07bfa02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8167fa0 con 0x55d03a3d91d0 2024-03-28T13:20:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.778+0000 7f07bfa02740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f07b81f6a20 con 0x55d03a3f5700 2024-03-28T13:20:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.779+0000 7f07afe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d95b0 0x55d03a3fa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.779+0000 7f07afe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d95b0 0x55d03a3fa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47930/0 (socket says 172.21.15.19:47930) 2024-03-28T13:20:01.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.779+0000 7f07afe00640 1 -- 172.21.15.19:0/1630591186 learned_addr learned my addr 172.21.15.19:0/1630591186 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.779+0000 7f07b5e00640 1 --2- 172.21.15.19:0/1630591186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3d91d0 0x55d03a3de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.779+0000 7f07afe00640 1 -- 172.21.15.19:0/1630591186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3f5700 msgr2=0x55d03a3fa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.779+0000 7f07b6800640 1 --2- 172.21.15.19:0/1630591186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3f5700 0x55d03a3fa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.779+0000 7f07afe00640 1 --2- 172.21.15.19:0/1630591186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3f5700 0x55d03a3fa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.779+0000 7f07afe00640 1 -- 172.21.15.19:0/1630591186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3d91d0 msgr2=0x55d03a3de580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.779+0000 7f07afe00640 1 --2- 172.21.15.19:0/1630591186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3d91d0 0x55d03a3de580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.779+0000 7f07afe00640 1 -- 172.21.15.19:0/1630591186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d03a3f2dc0 con 0x55d03a3d95b0 2024-03-28T13:20:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.779+0000 7f07b5e00640 1 --2- 172.21.15.19:0/1630591186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3d91d0 0x55d03a3de580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.779+0000 7f07afe00640 1 --2- 172.21.15.19:0/1630591186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d95b0 0x55d03a3fa470 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f07a002be20 tx=0x7f07a0031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584b47528b433683 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.780+0000 7f07a5c00640 1 -- 172.21.15.19:0/1630591186 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a0004030 con 0x55d03a3d95b0 2024-03-28T13:20:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.780+0000 7f07a5c00640 1 -- 172.21.15.19:0/1630591186 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a004a070 con 0x55d03a3d95b0 2024-03-28T13:20:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.780+0000 7f07a5c00640 1 -- 172.21.15.19:0/1630591186 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a00043f0 con 0x55d03a3d95b0 2024-03-28T13:20:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.780+0000 7f07bfa02740 1 -- 172.21.15.19:0/1630591186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d95b0 msgr2=0x55d03a3fa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.780+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1630591186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d95b0 0x55d03a3fa470 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f07a002be20 tx=0x7f07a0031000 comp rx=0 tx=0).stop 2024-03-28T13:20:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.780+0000 7f07bfa02740 1 -- 172.21.15.19:0/1630591186 shutdown_connections 2024-03-28T13:20:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.781+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1630591186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3f5700 0x55d03a3fa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.781+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1630591186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d95b0 0x55d03a3fa470 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.781+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1630591186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3d91d0 0x55d03a3de580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.781+0000 7f07bfa02740 1 -- 172.21.15.19:0/1630591186 >> 172.21.15.19:0/1630591186 conn(0x55d03a3ac6b0 msgr2=0x55d03a3a74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.781+0000 7f07bfa02740 1 -- 172.21.15.19:0/1630591186 shutdown_connections 2024-03-28T13:20:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.781+0000 7f07bfa02740 1 -- 172.21.15.19:0/1630591186 wait complete. 2024-03-28T13:20:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.781+0000 7f07bfa02740 1 -- start start 2024-03-28T13:20:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.782+0000 7f07bfa02740 1 Processor -- start 2024-03-28T13:20:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.783+0000 7f07bfa02740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d91d0 0x55d03a3abc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.783+0000 7f07bfa02740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3d95b0 0x55d03a3f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.783+0000 7f07bfa02740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3f5700 0x55d03a409650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.783+0000 7f07bfa02740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d03a3dfa00 con 0x55d03a3d91d0 2024-03-28T13:20:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.783+0000 7f07bfa02740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f07b8167950 con 0x55d03a3d95b0 2024-03-28T13:20:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.783+0000 7f07bfa02740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55d03a3deba0 con 0x55d03a3f5700 2024-03-28T13:20:01.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.783+0000 7f07bfa02740 10 client.0 _ll_register_callbacks cb 0x7f07b00095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:20:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.783+0000 7f07b5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d91d0 0x55d03a3abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.783+0000 7f07afe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3d95b0 0x55d03a3f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.783+0000 7f07b6800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3f5700 0x55d03a409650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.783+0000 7f07b5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d91d0 0x55d03a3abc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47940/0 (socket says 172.21.15.19:47940) 2024-03-28T13:20:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.783+0000 7f07b5e00640 1 -- 172.21.15.19:0/1905756113 learned_addr learned my addr 172.21.15.19:0/1905756113 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.784+0000 7f07b5e00640 1 -- 172.21.15.19:0/1905756113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3f5700 msgr2=0x55d03a409650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.784+0000 7f07b5e00640 1 --2- 172.21.15.19:0/1905756113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3f5700 0x55d03a409650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.784+0000 7f07b5e00640 1 -- 172.21.15.19:0/1905756113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3d95b0 msgr2=0x55d03a3f67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.784+0000 7f07b5e00640 1 --2- 172.21.15.19:0/1905756113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3d95b0 0x55d03a3f67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.784+0000 7f07b5e00640 1 -- 172.21.15.19:0/1905756113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d03a3f6cf0 con 0x55d03a3d91d0 2024-03-28T13:20:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.784+0000 7f07b5e00640 1 --2- 172.21.15.19:0/1905756113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d91d0 0x55d03a3abc80 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f07a8014000 tx=0x7f07a8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4c7577623888f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:01.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.784+0000 7f079aa00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:20:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.784+0000 7f079aa00640 1 -- 172.21.15.19:0/1905756113 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07a8034060 con 0x55d03a3d91d0 2024-03-28T13:20:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.785+0000 7f07bfa02740 1 -- 172.21.15.19:0/1905756113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07a004a070 con 0x55d03a3d91d0 2024-03-28T13:20:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.785+0000 7f079aa00640 1 -- 172.21.15.19:0/1905756113 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f07a802f030 con 0x55d03a3d91d0 2024-03-28T13:20:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.785+0000 7f079aa00640 1 -- 172.21.15.19:0/1905756113 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f07a803a050 con 0x55d03a3d91d0 2024-03-28T13:20:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.785+0000 7f07bfa02740 10 client.8463 fetch_fsmap learned FSMap version 88 2024-03-28T13:20:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.785+0000 7f07bfa02740 1 -- 172.21.15.19:0/1905756113 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55d03a3ab200 con 0x55d03a3d91d0 2024-03-28T13:20:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.785+0000 7f079aa00640 1 -- 172.21.15.19:0/1905756113 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07a004a070 con 0x55d03a3d91d0 2024-03-28T13:20:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.785+0000 7f079aa00640 1 -- 172.21.15.19:0/1905756113 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f07a803d020 con 0x55d03a3d91d0 2024-03-28T13:20:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.785+0000 7f07bfa02740 10 client.8463 fetch_fsmap finished waiting for FSMap version 88 2024-03-28T13:20:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.785+0000 7f07bfa02740 10 client.8463 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-28T13:20:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.785+0000 7f07bfa02740 20 client.8463 populate_metadata read hostname 'smithi019' 2024-03-28T13:20:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.785+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1905756113 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55d03a3b1190 0x55d03a3b1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.785+0000 7f07bfa02740 4 client.8463 mds_command: new command op to 8379 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.785+0000 7f07bfa02740 1 -- 172.21.15.19:0/1905756113 --> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] -- command(tid 0: {"prefix": "status"}) -- 0x55d03a31d820 con 0x55d03a3b1190 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.786+0000 7f07afe00640 1 --2- 172.21.15.19:0/1905756113 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55d03a3b1190 0x55d03a3b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.786+0000 7f07afe00640 1 --2- 172.21.15.19:0/1905756113 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55d03a3b1190 0x55d03a3b1570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2dde1097e4e5208 server_cookie=b018a6d204807086 in_seq=0 out_seq=0 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.786+0000 7f079aa00640 10 client.8463 ms_handle_connect on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.787+0000 7f079aa00640 1 -- 172.21.15.19:0/1905756113 <== mds.0 v2:172.21.15.19:6832/2851646201 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55d03a31d820 con 0x55d03a3b1190 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.787+0000 7f079aa00640 10 client.8463 handle_command_reply: tid=0 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.787+0000 7f07bfa02740 1 client.8463 shutdown 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.787+0000 7f07bfa02740 1 -- 172.21.15.19:0/1905756113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d91d0 msgr2=0x55d03a3abc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.787+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1905756113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d91d0 0x55d03a3abc80 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f07a8014000 tx=0x7f07a8000bc0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.787+0000 7f07bfa02740 1 -- 172.21.15.19:0/1905756113 shutdown_connections 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.787+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1905756113 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55d03a3b1190 0x55d03a3b1570 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.787+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1905756113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d03a3f5700 0x55d03a409650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.787+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1905756113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d03a3d95b0 0x55d03a3f67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.787+0000 7f07bfa02740 1 --2- 172.21.15.19:0/1905756113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d03a3d91d0 0x55d03a3abc80 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.787+0000 7f079aa00640 0 client.8463 ms_handle_reset on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.787+0000 7f07bfa02740 1 -- 172.21.15.19:0/1905756113 >> 172.21.15.19:0/1905756113 conn(0x55d03a3ac6b0 msgr2=0x55d03a3fa470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:01.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.787+0000 7f07bfa02740 1 -- 172.21.15.19:0/1905756113 shutdown_connections 2024-03-28T13:20:01.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.787+0000 7f07bfa02740 1 -- 172.21.15.19:0/1905756113 wait complete. 2024-03-28T13:20:01.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.787+0000 7f07bfa02740 20 client.8463 trim_cache size 0 max 16384 2024-03-28T13:20:01.789 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:20:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "e6b569d9-f2ce-49dc-bbe7-c0ab33080203", 2024-03-28T13:20:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-03-28T13:20:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 8379, 2024-03-28T13:20:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-03-28T13:20:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-03-28T13:20:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-03-28T13:20:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 5.0524132430000002, 2024-03-28T13:20:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 88, 2024-03-28T13:20:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 108, 2024-03-28T13:20:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 108, 2024-03-28T13:20:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 5.8739678749999999 2024-03-28T13:20:01.790 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:20:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.788+0000 7f07b7200640 1 -- 172.21.15.19:0/749167981 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f079406c530 msgr2=0x7f079406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.788+0000 7f07b7200640 1 --2- 172.21.15.19:0/749167981 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f079406c530 0x7f079406e9f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f07a8000f60 tx=0x7f07a8009040 comp rx=0 tx=0).stop 2024-03-28T13:20:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.788+0000 7f07b7200640 1 -- 172.21.15.19:0/749167981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07b80b5970 msgr2=0x7f07b8152840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.788+0000 7f07b7200640 1 --2- 172.21.15.19:0/749167981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07b80b5970 0x7f07b8152840 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009f00 tx=0x7f07b000c490 comp rx=0 tx=0).stop 2024-03-28T13:20:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.788+0000 7f07b7200640 1 -- 172.21.15.19:0/749167981 shutdown_connections 2024-03-28T13:20:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.788+0000 7f07b7200640 1 --2- 172.21.15.19:0/749167981 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f079406c530 0x7f079406e9f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.788+0000 7f07b7200640 1 --2- 172.21.15.19:0/749167981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f07b80b5970 0x7f07b8152840 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.788+0000 7f07b7200640 1 --2- 172.21.15.19:0/749167981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07b80720f0 0x7f07b80b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.788+0000 7f07b7200640 1 --2- 172.21.15.19:0/749167981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07b806ec60 0x7f07b8154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.788+0000 7f07b7200640 1 -- 172.21.15.19:0/749167981 >> 172.21.15.19:0/749167981 conn(0x7f07b80ccad0 msgr2=0x7f07b815c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.788+0000 7f07b7200640 1 -- 172.21.15.19:0/749167981 shutdown_connections 2024-03-28T13:20:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:01.788+0000 7f07b7200640 1 -- 172.21.15.19:0/749167981 wait complete. 2024-03-28T13:20:01.805 DEBUG:teuthology.orchestra.run.smithi019:> sudo daemon-helper kill ceph --cluster ceph -w 2024-03-28T13:20:01.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:20:01.812 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-03-28T13:20:02.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.001+0000 7f7f30a00640 1 Processor -- start 2024-03-28T13:20:02.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.001+0000 7f7f30a00640 1 -- start start 2024-03-28T13:20:02.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.003+0000 7f7f30a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f2c15e500 0x7f7f2c0d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.003+0000 7f7f30a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7f2c0d0bc0 0x7f7f2c0d1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.003+0000 7f7f30a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f2c0d0180 con 0x7f7f2c0d4ee0 2024-03-28T13:20:02.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.003+0000 7f7f30a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7f2c0d1dc0 con 0x7f7f2c0d0bc0 2024-03-28T13:20:02.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.003+0000 7f7f30a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2c0d20a0 con 0x7f7f2c15e500 2024-03-28T13:20:02.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.003+0000 7f7f2b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7f2c0d0bc0 0x7f7f2c0d1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.003+0000 7f7f2b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7f2c0d0bc0 0x7f7f2c0d1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47944/0 (socket says 172.21.15.19:47944) 2024-03-28T13:20:02.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.003+0000 7f7f2b200640 1 -- 172.21.15.19:0/3849474715 learned_addr learned my addr 172.21.15.19:0/3849474715 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:02.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.004+0000 7f7f2a800640 1 --2- 172.21.15.19:0/3849474715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f2c15e500 0x7f7f2c0d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.004+0000 7f7f2b200640 1 -- 172.21.15.19:0/3849474715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f2c15e500 msgr2=0x7f7f2c0d2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.004+0000 7f7f2b200640 1 --2- 172.21.15.19:0/3849474715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f2c15e500 0x7f7f2c0d2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.004+0000 7f7f2b200640 1 -- 172.21.15.19:0/3849474715 >> v1:172.21.15.19:6789/0 conn(0x7f7f2c0d4ee0 legacy=0x7f7f2c0cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.004+0000 7f7f2b200640 1 -- 172.21.15.19:0/3849474715 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c0d2380 con 0x7f7f2c0d0bc0 2024-03-28T13:20:02.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.004+0000 7f7f29400640 1 -- 172.21.15.19:0/3849474715 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 593791193 0 0) 0x7f7f2c0d0180 con 0x7f7f2c0d4ee0 2024-03-28T13:20:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.005+0000 7f7f2b200640 1 --2- 172.21.15.19:0/3849474715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f2c0d0bc0 0x7f7f2c0d1070 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009e80 tx=0x7f7f2402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ba3322a400184c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.005+0000 7f7f29400640 1 -- 172.21.15.19:0/3849474715 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f2402ca50 con 0x7f7f2c0d0bc0 2024-03-28T13:20:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.005+0000 7f7f29400640 1 -- 172.21.15.19:0/3849474715 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f2402cbf0 con 0x7f7f2c0d0bc0 2024-03-28T13:20:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.005+0000 7f7f29400640 1 -- 172.21.15.19:0/3849474715 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f240343f0 con 0x7f7f2c0d0bc0 2024-03-28T13:20:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.006+0000 7f7f30a00640 1 -- 172.21.15.19:0/3849474715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f2c0d0bc0 msgr2=0x7f7f2c0d1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.006+0000 7f7f30a00640 1 --2- 172.21.15.19:0/3849474715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f2c0d0bc0 0x7f7f2c0d1070 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f7f24009e80 tx=0x7f7f2402beb0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.006+0000 7f7f30a00640 1 -- 172.21.15.19:0/3849474715 shutdown_connections 2024-03-28T13:20:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.006+0000 7f7f30a00640 1 --2- 172.21.15.19:0/3849474715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f2c0d0bc0 0x7f7f2c0d1070 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.006+0000 7f7f30a00640 1 --2- 172.21.15.19:0/3849474715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f2c15e500 0x7f7f2c0d2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.006+0000 7f7f30a00640 1 -- 172.21.15.19:0/3849474715 >> 172.21.15.19:0/3849474715 conn(0x7f7f2c09d530 msgr2=0x7f7f2c0d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.006+0000 7f7f30a00640 1 -- 172.21.15.19:0/3849474715 shutdown_connections 2024-03-28T13:20:02.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.006+0000 7f7f30a00640 1 -- 172.21.15.19:0/3849474715 wait complete. 2024-03-28T13:20:02.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.008+0000 7f7f30a00640 1 Processor -- start 2024-03-28T13:20:02.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.008+0000 7f7f30a00640 1 -- start start 2024-03-28T13:20:02.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.009+0000 7f7f30a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f2c0d0bc0 0x7f7f2c154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.009+0000 7f7f30a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f2c0d4ee0 0x7f7f2c160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.009+0000 7f7f30a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f2c15e500 0x7f7f2c161350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.009+0000 7f7f30a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c0d30c0 con 0x7f7f2c15e500 2024-03-28T13:20:02.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.009+0000 7f7f30a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c15e8e0 con 0x7f7f2c0d0bc0 2024-03-28T13:20:02.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.009+0000 7f7f30a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2c0b3ae0 con 0x7f7f2c0d4ee0 2024-03-28T13:20:02.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.009+0000 7f7f2b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f2c15e500 0x7f7f2c161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.010+0000 7f7f2b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f2c15e500 0x7f7f2c161350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47952/0 (socket says 172.21.15.19:47952) 2024-03-28T13:20:02.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.010+0000 7f7f2b200640 1 -- 172.21.15.19:0/2142222663 learned_addr learned my addr 172.21.15.19:0/2142222663 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.010+0000 7f7f2a800640 1 --2- 172.21.15.19:0/2142222663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f2c0d0bc0 0x7f7f2c154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.010+0000 7f7f29e00640 1 --2- 172.21.15.19:0/2142222663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f2c0d4ee0 0x7f7f2c160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.010+0000 7f7f2b200640 1 -- 172.21.15.19:0/2142222663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f2c0d4ee0 msgr2=0x7f7f2c160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.010+0000 7f7f2b200640 1 --2- 172.21.15.19:0/2142222663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f2c0d4ee0 0x7f7f2c160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.010+0000 7f7f2b200640 1 -- 172.21.15.19:0/2142222663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f2c0d0bc0 msgr2=0x7f7f2c154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.010+0000 7f7f2b200640 1 --2- 172.21.15.19:0/2142222663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f2c0d0bc0 0x7f7f2c154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.010+0000 7f7f2b200640 1 -- 172.21.15.19:0/2142222663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2c154bb0 con 0x7f7f2c15e500 2024-03-28T13:20:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.010+0000 7f7f29e00640 1 --2- 172.21.15.19:0/2142222663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f2c0d4ee0 0x7f7f2c160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.011+0000 7f7f2b200640 1 --2- 172.21.15.19:0/2142222663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f2c15e500 0x7f7f2c161350 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f7f24036040 tx=0x7f7f24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.011+0000 7f7f22a00640 1 -- 172.21.15.19:0/2142222663 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f24043020 con 0x7f7f2c15e500 2024-03-28T13:20:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.011+0000 7f7f22a00640 1 -- 172.21.15.19:0/2142222663 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f2403e450 con 0x7f7f2c15e500 2024-03-28T13:20:02.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.011+0000 7f7f22a00640 1 -- 172.21.15.19:0/2142222663 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f24009510 con 0x7f7f2c15e500 2024-03-28T13:20:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.011+0000 7f7f30a00640 1 -- 172.21.15.19:0/2142222663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2c0ccbc0 con 0x7f7f2c15e500 2024-03-28T13:20:02.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.011+0000 7f7f30a00640 1 -- 172.21.15.19:0/2142222663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f2c15ecb0 con 0x7f7f2c15e500 2024-03-28T13:20:02.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.014+0000 7f7f22a00640 1 -- 172.21.15.19:0/2142222663 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7f24046020 con 0x7f7f2c15e500 2024-03-28T13:20:02.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.015+0000 7f7f22a00640 1 --2- 172.21.15.19:0/2142222663 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f0c06c530 0x7f7f0c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.015+0000 7f7f22a00640 1 -- 172.21.15.19:0/2142222663 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f7f240adfd0 con 0x7f7f2c15e500 2024-03-28T13:20:02.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.015+0000 7f7f2a800640 1 --2- 172.21.15.19:0/2142222663 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f0c06c530 0x7f7f0c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.016+0000 7f7f2a800640 1 --2- 172.21.15.19:0/2142222663 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f0c06c530 0x7f7f0c06e9f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006fd0 tx=0x7f7f1c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.017+0000 7f7f34230740 1 Processor -- start 2024-03-28T13:20:02.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.017+0000 7f7f34230740 1 -- start start 2024-03-28T13:20:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.018+0000 7f7f34230740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cffc9180 0x5642cffc9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.019+0000 7f7f34230740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 0x5642cffcd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.019+0000 7f7f34230740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642d0016890 0x5642d0016d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.019+0000 7f7f34230740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5642cff33280 con 0x5642cffc9180 2024-03-28T13:20:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.019+0000 7f7f34230740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5642cfedb6f0 con 0x5642d0016890 2024-03-28T13:20:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.019+0000 7f7f34230740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5642cff2f230 con 0x5642cffcefa0 2024-03-28T13:20:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.019+0000 7f7f2a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cffc9180 0x5642cffc9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.019+0000 7f7f2a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cffc9180 0x5642cffc9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47954/0 (socket says 172.21.15.19:47954) 2024-03-28T13:20:02.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.019+0000 7f7f2a800640 1 -- 172.21.15.19:0/1736911056 learned_addr learned my addr 172.21.15.19:0/1736911056 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:02.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.019+0000 7f7f2b200640 1 --2- 172.21.15.19:0/1736911056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642d0016890 0x5642d0016d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.019+0000 7f7f29e00640 1 --2- 172.21.15.19:0/1736911056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 0x5642cffcd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.019+0000 7f7f2a800640 1 -- 172.21.15.19:0/1736911056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 msgr2=0x5642cffcd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.019+0000 7f7f2a800640 1 --2- 172.21.15.19:0/1736911056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 0x5642cffcd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.019+0000 7f7f2a800640 1 -- 172.21.15.19:0/1736911056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642d0016890 msgr2=0x5642d0016d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.019+0000 7f7f2a800640 1 --2- 172.21.15.19:0/1736911056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642d0016890 0x5642d0016d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.019+0000 7f7f2a800640 1 -- 172.21.15.19:0/1736911056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cfd0f7e0 con 0x5642cffc9180 2024-03-28T13:20:02.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.020+0000 7f7f2a800640 1 --2- 172.21.15.19:0/1736911056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cffc9180 0x5642cffc9560 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00fa20 tx=0x7f7f1c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ce317428e8fb72 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.020+0000 7f7f17000640 1 -- 172.21.15.19:0/1736911056 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c01b050 con 0x5642cffc9180 2024-03-28T13:20:02.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.020+0000 7f7f17000640 1 -- 172.21.15.19:0/1736911056 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f1c025070 con 0x5642cffc9180 2024-03-28T13:20:02.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.020+0000 7f7f17000640 1 -- 172.21.15.19:0/1736911056 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c014de0 con 0x5642cffc9180 2024-03-28T13:20:02.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.021+0000 7f7f34230740 1 -- 172.21.15.19:0/1736911056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cffc9180 msgr2=0x5642cffc9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.021+0000 7f7f34230740 1 --2- 172.21.15.19:0/1736911056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cffc9180 0x5642cffc9560 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c00fa20 tx=0x7f7f1c012000 comp rx=0 tx=0).stop 2024-03-28T13:20:02.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.021+0000 7f7f34230740 1 -- 172.21.15.19:0/1736911056 shutdown_connections 2024-03-28T13:20:02.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.021+0000 7f7f34230740 1 --2- 172.21.15.19:0/1736911056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642d0016890 0x5642d0016d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.021+0000 7f7f34230740 1 --2- 172.21.15.19:0/1736911056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 0x5642cffcd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.021+0000 7f7f34230740 1 --2- 172.21.15.19:0/1736911056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cffc9180 0x5642cffc9560 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.021+0000 7f7f34230740 1 -- 172.21.15.19:0/1736911056 >> 172.21.15.19:0/1736911056 conn(0x5642cffcb200 msgr2=0x5642cffccac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:02.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.021+0000 7f7f34230740 1 -- 172.21.15.19:0/1736911056 shutdown_connections 2024-03-28T13:20:02.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.021+0000 7f7f34230740 1 -- 172.21.15.19:0/1736911056 wait complete. 2024-03-28T13:20:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.022+0000 7f7f34230740 1 -- start start 2024-03-28T13:20:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.022+0000 7f7f34230740 1 Processor -- start 2024-03-28T13:20:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.023+0000 7f7f34230740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 0x5642cfffb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.023+0000 7f7f34230740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642d0016890 0x5642cfff4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.023+0000 7f7f34230740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642cfff1870 0x5642cfff4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.023+0000 7f7f34230740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5642cfff58f0 con 0x5642d0016890 2024-03-28T13:20:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.023+0000 7f7f34230740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5642cfeeeaa0 con 0x5642cfff1870 2024-03-28T13:20:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.023+0000 7f7f34230740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5642cfff5b40 con 0x5642cffcefa0 2024-03-28T13:20:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.023+0000 7f7f34230740 10 client.0 _ll_register_callbacks cb 0x5642cfeee490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:20:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.024+0000 7f7f29e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642d0016890 0x5642cfff4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.024+0000 7f7f29e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642d0016890 0x5642cfff4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47966/0 (socket says 172.21.15.19:47966) 2024-03-28T13:20:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.024+0000 7f7f29e00640 1 -- 172.21.15.19:0/3921690487 learned_addr learned my addr 172.21.15.19:0/3921690487 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:02.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.024+0000 7f7f2b200640 1 --2- 172.21.15.19:0/3921690487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642cfff1870 0x5642cfff4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.024+0000 7f7f2a800640 1 --2- 172.21.15.19:0/3921690487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 0x5642cfffb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.024+0000 7f7f29e00640 1 -- 172.21.15.19:0/3921690487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 msgr2=0x5642cfffb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.024+0000 7f7f29e00640 1 --2- 172.21.15.19:0/3921690487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 0x5642cfffb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.024+0000 7f7f29e00640 1 -- 172.21.15.19:0/3921690487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642cfff1870 msgr2=0x5642cfff4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.024+0000 7f7f29e00640 1 --2- 172.21.15.19:0/3921690487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642cfff1870 0x5642cfff4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.024+0000 7f7f29e00640 1 -- 172.21.15.19:0/3921690487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cfd6ff70 con 0x5642d0016890 2024-03-28T13:20:02.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.025+0000 7f7f29e00640 1 --2- 172.21.15.19:0/3921690487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642d0016890 0x5642cfff4330 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f7f18001e60 tx=0x7f7f1800f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1779dae3a43c4ba4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.025+0000 7f7f0b400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:20:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.025+0000 7f7f0b400640 1 -- 172.21.15.19:0/3921690487 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1800fe90 con 0x5642d0016890 2024-03-28T13:20:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.025+0000 7f7f0b400640 1 -- 172.21.15.19:0/3921690487 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f7f180052e0 con 0x5642d0016890 2024-03-28T13:20:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.025+0000 7f7f34230740 1 -- 172.21.15.19:0/3921690487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f1c025070 con 0x5642d0016890 2024-03-28T13:20:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.025+0000 7f7f0b400640 1 -- 172.21.15.19:0/3921690487 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f7f18020050 con 0x5642d0016890 2024-03-28T13:20:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.025+0000 7f7f34230740 10 client.8478 fetch_fsmap learned FSMap version 88 2024-03-28T13:20:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.025+0000 7f7f34230740 1 -- 172.21.15.19:0/3921690487 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5642cfc07170 con 0x5642d0016890 2024-03-28T13:20:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.026+0000 7f7f0b400640 1 -- 172.21.15.19:0/3921690487 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f1c025070 con 0x5642d0016890 2024-03-28T13:20:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.026+0000 7f7f0b400640 1 -- 172.21.15.19:0/3921690487 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f7f18023020 con 0x5642d0016890 2024-03-28T13:20:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.026+0000 7f7f34230740 10 client.8478 fetch_fsmap finished waiting for FSMap version 88 2024-03-28T13:20:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.026+0000 7f7f34230740 10 client.8478 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-28T13:20:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.026+0000 7f7f34230740 20 client.8478 populate_metadata read hostname 'smithi019' 2024-03-28T13:20:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.026+0000 7f7f34230740 1 --2- 172.21.15.19:0/3921690487 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5642cfffa550 0x5642cfffa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.026+0000 7f7f34230740 4 client.8478 mds_command: new command op to 8379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:20:02.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.026+0000 7f7f34230740 1 -- 172.21.15.19:0/3921690487 --> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642cff407c0 con 0x5642cfffa550 2024-03-28T13:20:02.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.026+0000 7f7f2a800640 1 --2- 172.21.15.19:0/3921690487 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5642cfffa550 0x5642cfffa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.027+0000 7f7f2a800640 1 --2- 172.21.15.19:0/3921690487 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5642cfffa550 0x5642cfffa930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8bde3a89265d2e5 server_cookie=1d066190a7c5c657 in_seq=0 out_seq=0 2024-03-28T13:20:02.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.027+0000 7f7f0b400640 10 client.8478 ms_handle_connect on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:02.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.028+0000 7f7f0b400640 1 -- 172.21.15.19:0/3921690487 <== mds.0 v2:172.21.15.19:6832/2851646201 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642cff407c0 con 0x5642cfffa550 2024-03-28T13:20:02.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.028+0000 7f7f0b400640 10 client.8478 handle_command_reply: tid=0 2024-03-28T13:20:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.028+0000 7f7f34230740 1 client.8478 shutdown 2024-03-28T13:20:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.029+0000 7f7f34230740 1 -- 172.21.15.19:0/3921690487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642d0016890 msgr2=0x5642cfff4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.029+0000 7f7f34230740 1 --2- 172.21.15.19:0/3921690487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642d0016890 0x5642cfff4330 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f7f18001e60 tx=0x7f7f1800f660 comp rx=0 tx=0).stop 2024-03-28T13:20:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.029+0000 7f7f34230740 1 -- 172.21.15.19:0/3921690487 shutdown_connections 2024-03-28T13:20:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.029+0000 7f7f34230740 1 --2- 172.21.15.19:0/3921690487 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5642cfffa550 0x5642cfffa930 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.029+0000 7f7f34230740 1 --2- 172.21.15.19:0/3921690487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642cfff1870 0x5642cfff4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.029+0000 7f7f34230740 1 --2- 172.21.15.19:0/3921690487 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642d0016890 0x5642cfff4330 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.029+0000 7f7f34230740 1 --2- 172.21.15.19:0/3921690487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 0x5642cfffb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.029+0000 7f7f34230740 1 -- 172.21.15.19:0/3921690487 >> 172.21.15.19:0/3921690487 conn(0x5642cffc9180 msgr2=0x5642cffc8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.029+0000 7f7f0b400640 0 client.8478 ms_handle_reset on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.029+0000 7f7f34230740 1 -- 172.21.15.19:0/3921690487 shutdown_connections 2024-03-28T13:20:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.029+0000 7f7f34230740 1 -- 172.21.15.19:0/3921690487 wait complete. 2024-03-28T13:20:02.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.029+0000 7f7f34230740 20 client.8478 trim_cache size 0 max 16384 2024-03-28T13:20:02.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.037+0000 7fa645800640 1 Processor -- start 2024-03-28T13:20:02.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.037+0000 7fa645800640 1 -- start start 2024-03-28T13:20:02.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.038+0000 7fa645800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa640145ce0 0x7fa6400acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.038+0000 7fa645800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6400af300 0x7fa640071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.038+0000 7fa645800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa64006e4e0 con 0x7fa6400ad530 2024-03-28T13:20:02.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.038+0000 7fa645800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa64006e7c0 con 0x7fa640145ce0 2024-03-28T13:20:02.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.038+0000 7fa645800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa64006eaa0 con 0x7fa6400af300 2024-03-28T13:20:02.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.038+0000 7fa637e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa6400ad530 0x7fa6400aebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50736/0 (socket says 172.21.15.19:50736) 2024-03-28T13:20:02.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.038+0000 7fa637e00640 1 -- 172.21.15.19:0/1365308732 learned_addr learned my addr 172.21.15.19:0/1365308732 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:02.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.039+0000 7fa63ea00640 1 --2- 172.21.15.19:0/1365308732 >> v2:172.21.15.188:3300/0 conn(0x7fa640145ce0 0x7fa6400acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.039+0000 7fa63f400640 1 --2- 172.21.15.19:0/1365308732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6400af300 0x7fa640071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.039+0000 7fa63e000640 1 -- 172.21.15.19:0/1365308732 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1957691759 0 0) 0x7fa64006e4e0 con 0x7fa6400ad530 2024-03-28T13:20:02.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.039+0000 7fa63e000640 1 -- 172.21.15.19:0/1365308732 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa624003520 con 0x7fa6400ad530 2024-03-28T13:20:02.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.039+0000 7fa63e000640 1 -- 172.21.15.19:0/1365308732 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2585324934 0 0) 0x7fa624003520 con 0x7fa6400ad530 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.039+0000 7fa63e000640 1 -- 172.21.15.19:0/1365308732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6400af300 msgr2=0x7fa640071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.039+0000 7fa63e000640 1 --2- 172.21.15.19:0/1365308732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6400af300 0x7fa640071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.039+0000 7fa63e000640 1 -- 172.21.15.19:0/1365308732 >> v2:172.21.15.188:3300/0 conn(0x7fa640145ce0 msgr2=0x7fa6400acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.039+0000 7fa63e000640 1 --2- 172.21.15.19:0/1365308732 >> v2:172.21.15.188:3300/0 conn(0x7fa640145ce0 0x7fa6400acff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.039+0000 7fa63e000640 1 -- 172.21.15.19:0/1365308732 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64006ed80 con 0x7fa6400ad530 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.039+0000 7fa63e000640 1 -- 172.21.15.19:0/1365308732 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fa628001690 con 0x7fa6400ad530 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.039+0000 7fa63e000640 1 -- 172.21.15.19:0/1365308732 >> v1:172.21.15.19:6789/0 conn(0x7fa6400ad530 legacy=0x7fa6400aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.040+0000 7fa63e000640 1 --2- 172.21.15.19:0/1365308732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6240039b0 0x7fa624003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.040+0000 7fa63e000640 1 --2- 172.21.15.19:0/1365308732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa624004800 0x7fa624004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.040+0000 7fa63e000640 1 --2- 172.21.15.19:0/1365308732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6400af300 0x7fa6240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.040+0000 7fa63e000640 1 -- 172.21.15.19:0/1365308732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa64006e7c0 con 0x7fa6240039b0 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.040+0000 7fa63e000640 1 -- 172.21.15.19:0/1365308732 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa64006eaa0 con 0x7fa624004800 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.040+0000 7fa63e000640 1 -- 172.21.15.19:0/1365308732 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa62400fdd0 con 0x7fa6400af300 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.040+0000 7fa63ea00640 1 --2- 172.21.15.19:0/1365308732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6240039b0 0x7fa624003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.040+0000 7fa637e00640 1 --2- 172.21.15.19:0/1365308732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa624004800 0x7fa624004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.040+0000 7fa63ea00640 1 -- 172.21.15.19:0/1365308732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6400af300 msgr2=0x7fa6240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.040+0000 7fa63ea00640 1 --2- 172.21.15.19:0/1365308732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6400af300 0x7fa6240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.040+0000 7fa63ea00640 1 -- 172.21.15.19:0/1365308732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa624004800 msgr2=0x7fa624004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.040+0000 7fa63ea00640 1 --2- 172.21.15.19:0/1365308732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa624004800 0x7fa624004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.040+0000 7fa63ea00640 1 -- 172.21.15.19:0/1365308732 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa624003520 con 0x7fa6240039b0 2024-03-28T13:20:02.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.040+0000 7fa637e00640 1 --2- 172.21.15.19:0/1365308732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa624004800 0x7fa624004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:02.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.041+0000 7fa63ea00640 1 --2- 172.21.15.19:0/1365308732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6240039b0 0x7fa624003dd0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fa630009ac0 tx=0x7fa624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5405714512b7089 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.042+0000 7fa63e000640 1 -- 172.21.15.19:0/1365308732 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630039070 con 0x7fa6240039b0 2024-03-28T13:20:02.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.042+0000 7fa63e000640 1 -- 172.21.15.19:0/1365308732 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa63002c710 con 0x7fa6240039b0 2024-03-28T13:20:02.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.042+0000 7fa63e000640 1 -- 172.21.15.19:0/1365308732 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa630009690 con 0x7fa6240039b0 2024-03-28T13:20:02.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.042+0000 7fa645800640 1 -- 172.21.15.19:0/1365308732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6240039b0 msgr2=0x7fa624003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.042+0000 7fa645800640 1 --2- 172.21.15.19:0/1365308732 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa6240039b0 0x7fa624003dd0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fa630009ac0 tx=0x7fa624005830 comp rx=0 tx=0).stop 2024-03-28T13:20:02.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.042+0000 7fa63f400640 1 -- 172.21.15.19:0/1365308732 reap_dead start 2024-03-28T13:20:02.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.042+0000 7fa645800640 1 -- 172.21.15.19:0/1365308732 shutdown_connections 2024-03-28T13:20:02.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.042+0000 7fa645800640 1 -- 172.21.15.19:0/1365308732 >> 172.21.15.19:0/1365308732 conn(0x7fa64006b570 msgr2=0x7fa64006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:02.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.043+0000 7fa645800640 1 -- 172.21.15.19:0/1365308732 shutdown_connections 2024-03-28T13:20:02.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.043+0000 7fa645800640 1 -- 172.21.15.19:0/1365308732 wait complete. 2024-03-28T13:20:02.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.045+0000 7fa645800640 1 Processor -- start 2024-03-28T13:20:02.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.046+0000 7fa645800640 1 -- start start 2024-03-28T13:20:02.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.047+0000 7fa645800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6240039b0 0x7fa6401408f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.047+0000 7fa645800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6400728d0 0x7fa6401427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.047+0000 7fa645800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa64015ba70 0x7fa64015bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.047+0000 7fa645800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa6401460c0 con 0x7fa64015ba70 2024-03-28T13:20:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.047+0000 7fa645800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa64006f060 con 0x7fa6400728d0 2024-03-28T13:20:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.047+0000 7fa645800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa640146240 con 0x7fa6240039b0 2024-03-28T13:20:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.047+0000 7fa63f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa64015ba70 0x7fa64015bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.047+0000 7fa63f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa64015ba70 0x7fa64015bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47996/0 (socket says 172.21.15.19:47996) 2024-03-28T13:20:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.047+0000 7fa63f400640 1 -- 172.21.15.19:0/2221956982 learned_addr learned my addr 172.21.15.19:0/2221956982 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.047+0000 7fa63ea00640 1 --2- 172.21.15.19:0/2221956982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6240039b0 0x7fa6401408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.047+0000 7fa637e00640 1 --2- 172.21.15.19:0/2221956982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6400728d0 0x7fa6401427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.048+0000 7fa63f400640 1 -- 172.21.15.19:0/2221956982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6240039b0 msgr2=0x7fa6401408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.048+0000 7fa63f400640 1 --2- 172.21.15.19:0/2221956982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6240039b0 0x7fa6401408f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.048+0000 7fa63f400640 1 -- 172.21.15.19:0/2221956982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6400728d0 msgr2=0x7fa6401427a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.048+0000 7fa63f400640 1 --2- 172.21.15.19:0/2221956982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6400728d0 0x7fa6401427a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.048+0000 7fa63f400640 1 -- 172.21.15.19:0/2221956982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa640141da0 con 0x7fa64015ba70 2024-03-28T13:20:02.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.048+0000 7fa63f400640 1 --2- 172.21.15.19:0/2221956982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa64015ba70 0x7fa64015bef0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fa6400afb70 tx=0x7fa63800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.048+0000 7fa636a00640 1 -- 172.21.15.19:0/2221956982 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638018020 con 0x7fa64015ba70 2024-03-28T13:20:02.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.048+0000 7fa636a00640 1 -- 172.21.15.19:0/2221956982 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa63800bae0 con 0x7fa64015ba70 2024-03-28T13:20:02.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.048+0000 7fa636a00640 1 -- 172.21.15.19:0/2221956982 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638014400 con 0x7fa64015ba70 2024-03-28T13:20:02.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.048+0000 7fa645800640 1 -- 172.21.15.19:0/2221956982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa640141220 con 0x7fa64015ba70 2024-03-28T13:20:02.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.049+0000 7fa645800640 1 -- 172.21.15.19:0/2221956982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6401417e0 con 0x7fa64015ba70 2024-03-28T13:20:02.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.049+0000 7fa645800640 1 -- 172.21.15.19:0/2221956982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fa6400685c0 con 0x7fa64015ba70 2024-03-28T13:20:02.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.051+0000 7fa636a00640 1 -- 172.21.15.19:0/2221956982 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa638005d10 con 0x7fa64015ba70 2024-03-28T13:20:02.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.052+0000 7f7f34230740 1 Processor -- start 2024-03-28T13:20:02.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.052+0000 7f7f34230740 1 -- start start 2024-03-28T13:20:02.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.052+0000 7fa636a00640 1 --2- 172.21.15.19:0/2221956982 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa61c06c560 0x7fa61c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.052+0000 7fa636a00640 1 -- 172.21.15.19:0/2221956982 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7fa63808c580 con 0x7fa64015ba70 2024-03-28T13:20:02.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.053+0000 7fa636a00640 1 -- 172.21.15.19:0/2221956982 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+516 (secure 0 0 0) 0x7fa638022030 con 0x7fa64015ba70 2024-03-28T13:20:02.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.053+0000 7fa645800640 1 -- 172.21.15.19:0/2221956982 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7fa64006f700 con 0x7fa64015ba70 2024-03-28T13:20:02.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.053+0000 7fa63ea00640 1 --2- 172.21.15.19:0/2221956982 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa61c06c560 0x7fa61c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.053+0000 7f7f34230740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 0x5642cffcdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.053+0000 7f7f34230740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cfff1870 0x5642cffcc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.053+0000 7f7f34230740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642d0016890 0x5642cffcced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.053+0000 7f7f34230740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c200a10 con 0x5642cfff1870 2024-03-28T13:20:02.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.053+0000 7f7f34230740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1ff9f0 con 0x5642d0016890 2024-03-28T13:20:02.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.053+0000 7f7f34230740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7f2c1f8400 con 0x5642cffcefa0 2024-03-28T13:20:02.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.053+0000 7f7f29e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cfff1870 0x5642cffcc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.053+0000 7f7f29e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cfff1870 0x5642cffcc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48010/0 (socket says 172.21.15.19:48010) 2024-03-28T13:20:02.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.053+0000 7f7f29e00640 1 -- 172.21.15.19:0/4056911456 learned_addr learned my addr 172.21.15.19:0/4056911456 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:02.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.054+0000 7f7f2a800640 1 --2- 172.21.15.19:0/4056911456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 0x5642cffcdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.054+0000 7f7f29e00640 1 -- 172.21.15.19:0/4056911456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 msgr2=0x5642cffcdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.054+0000 7f7f29e00640 1 --2- 172.21.15.19:0/4056911456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 0x5642cffcdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.054+0000 7f7f29e00640 1 -- 172.21.15.19:0/4056911456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642d0016890 msgr2=0x5642cffcced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.054+0000 7f7f29e00640 1 --2- 172.21.15.19:0/4056911456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642d0016890 0x5642cffcced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.054+0000 7f7f29e00640 1 -- 172.21.15.19:0/4056911456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642cfffaca0 con 0x5642cfff1870 2024-03-28T13:20:02.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.054+0000 7f7f2a800640 1 --2- 172.21.15.19:0/4056911456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 0x5642cffcdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:02.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.054+0000 7fa63ea00640 1 --2- 172.21.15.19:0/2221956982 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa61c06c560 0x7fa61c06ea20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa630036000 tx=0x7fa630030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.054+0000 7f7f29e00640 1 --2- 172.21.15.19:0/4056911456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cfff1870 0x5642cffcc990 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f7f18007d50 tx=0x7f7f1800b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ef981de81a0c45 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.055+0000 7f7f16600640 1 -- 172.21.15.19:0/4056911456 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1800fb60 con 0x5642cfff1870 2024-03-28T13:20:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.055+0000 7f7f16600640 1 -- 172.21.15.19:0/4056911456 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f18028030 con 0x5642cfff1870 2024-03-28T13:20:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.055+0000 7f7f16600640 1 -- 172.21.15.19:0/4056911456 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f18003b40 con 0x5642cfff1870 2024-03-28T13:20:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.055+0000 7f7f34230740 1 -- 172.21.15.19:0/4056911456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cfff1870 msgr2=0x5642cffcc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.055+0000 7f7f34230740 1 --2- 172.21.15.19:0/4056911456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cfff1870 0x5642cffcc990 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f7f18007d50 tx=0x7f7f1800b000 comp rx=0 tx=0).stop 2024-03-28T13:20:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.056+0000 7f7f34230740 1 -- 172.21.15.19:0/4056911456 shutdown_connections 2024-03-28T13:20:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.056+0000 7f7f34230740 1 --2- 172.21.15.19:0/4056911456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642d0016890 0x5642cffcced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.056+0000 7f7f34230740 1 --2- 172.21.15.19:0/4056911456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cfff1870 0x5642cffcc990 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.056+0000 7f7f34230740 1 --2- 172.21.15.19:0/4056911456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cffcefa0 0x5642cffcdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.056+0000 7f7f34230740 1 -- 172.21.15.19:0/4056911456 >> 172.21.15.19:0/4056911456 conn(0x5642cffc9180 msgr2=0x5642cfff08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.056+0000 7f7f34230740 1 -- 172.21.15.19:0/4056911456 shutdown_connections 2024-03-28T13:20:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.056+0000 7f7f34230740 1 -- 172.21.15.19:0/4056911456 wait complete. 2024-03-28T13:20:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.056+0000 7f7f34230740 1 -- start start 2024-03-28T13:20:02.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.057+0000 7f7f34230740 1 Processor -- start 2024-03-28T13:20:02.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f34230740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cffcefa0 0x5642d00834f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f34230740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cfff1870 0x5642d0017950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f34230740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642d0016890 0x5642d002d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f34230740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5642cff407c0 con 0x5642cffcefa0 2024-03-28T13:20:02.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f34230740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7f2c1601f0 con 0x5642d0016890 2024-03-28T13:20:02.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f34230740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5642cffca200 con 0x5642cfff1870 2024-03-28T13:20:02.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f34230740 10 client.0 _ll_register_callbacks cb 0x7f7f18007c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:20:02.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f2a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cffcefa0 0x5642d00834f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f2a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cffcefa0 0x5642d00834f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48016/0 (socket says 172.21.15.19:48016) 2024-03-28T13:20:02.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f2a800640 1 -- 172.21.15.19:0/3351761136 learned_addr learned my addr 172.21.15.19:0/3351761136 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f2b200640 1 --2- 172.21.15.19:0/3351761136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642d0016890 0x5642d002d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f2a800640 1 -- 172.21.15.19:0/3351761136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cfff1870 msgr2=0x5642d0017950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f2a800640 1 --2- 172.21.15.19:0/3351761136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cfff1870 0x5642d0017950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f2a800640 1 -- 172.21.15.19:0/3351761136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642d0016890 msgr2=0x5642d002d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f2a800640 1 --2- 172.21.15.19:0/3351761136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642d0016890 0x5642d002d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f2a800640 1 -- 172.21.15.19:0/3351761136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642cfff77d0 con 0x5642cffcefa0 2024-03-28T13:20:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.058+0000 7f7f2b200640 1 --2- 172.21.15.19:0/3351761136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642d0016890 0x5642d002d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.059+0000 7f7f2a800640 1 --2- 172.21.15.19:0/3351761136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cffcefa0 0x5642d00834f0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c021000 tx=0x7f7f1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b93239e23da0ae server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.059+0000 7f7f0b400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:20:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.059+0000 7f7f0b400640 1 -- 172.21.15.19:0/3351761136 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c03c060 con 0x5642cffcefa0 2024-03-28T13:20:02.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.059+0000 7f7f34230740 1 -- 172.21.15.19:0/3351761136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f18028030 con 0x5642cffcefa0 2024-03-28T13:20:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.059+0000 7f7f0b400640 1 -- 172.21.15.19:0/3351761136 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f7f1c0360d0 con 0x5642cffcefa0 2024-03-28T13:20:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.059+0000 7f7f0b400640 1 -- 172.21.15.19:0/3351761136 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f7f1c042050 con 0x5642cffcefa0 2024-03-28T13:20:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.059+0000 7f7f34230740 10 client.8490 fetch_fsmap learned FSMap version 88 2024-03-28T13:20:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.059+0000 7f7f34230740 1 -- 172.21.15.19:0/3351761136 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5642d0017430 con 0x5642cffcefa0 2024-03-28T13:20:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.060+0000 7f7f0b400640 1 -- 172.21.15.19:0/3351761136 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f18028030 con 0x5642cffcefa0 2024-03-28T13:20:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.060+0000 7f7f0b400640 1 -- 172.21.15.19:0/3351761136 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f7f1c045020 con 0x5642cffcefa0 2024-03-28T13:20:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.060+0000 7f7f34230740 10 client.8490 fetch_fsmap finished waiting for FSMap version 88 2024-03-28T13:20:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.060+0000 7f7f34230740 10 client.8490 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-28T13:20:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.060+0000 7f7f34230740 20 client.8490 populate_metadata read hostname 'smithi019' 2024-03-28T13:20:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.060+0000 7f7f34230740 1 --2- 172.21.15.19:0/3351761136 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5642d00229c0 0x5642cfff8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.060+0000 7f7f34230740 4 client.8490 mds_command: new command op to 8379 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-03-28T13:20:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.060+0000 7f7f34230740 1 -- 172.21.15.19:0/3351761136 --> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x5642cffcb960 con 0x5642d00229c0 2024-03-28T13:20:02.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.060+0000 7f7f29e00640 1 --2- 172.21.15.19:0/3351761136 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5642d00229c0 0x5642cfff8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.061+0000 7f7f29e00640 1 --2- 172.21.15.19:0/3351761136 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5642d00229c0 0x5642cfff8b70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d8c41284fe3f5c3 server_cookie=c8e765cd7bfef478 in_seq=0 out_seq=0 2024-03-28T13:20:02.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.061+0000 7f7f0b400640 10 client.8490 ms_handle_connect on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:02.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.062+0000 7f7f0b400640 1 -- 172.21.15.19:0/3351761136 <== mds.0 v2:172.21.15.19:6832/2851646201 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5642cffcb960 con 0x5642d00229c0 2024-03-28T13:20:02.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.062+0000 7f7f0b400640 10 client.8490 handle_command_reply: tid=0 2024-03-28T13:20:02.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.062+0000 7f7f34230740 1 client.8490 shutdown 2024-03-28T13:20:02.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.062+0000 7f7f34230740 1 -- 172.21.15.19:0/3351761136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cffcefa0 msgr2=0x5642d00834f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.062+0000 7f7f34230740 1 --2- 172.21.15.19:0/3351761136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cffcefa0 0x5642d00834f0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c021000 tx=0x7f7f1c004730 comp rx=0 tx=0).stop 2024-03-28T13:20:02.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.063+0000 7f7f34230740 1 -- 172.21.15.19:0/3351761136 shutdown_connections 2024-03-28T13:20:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.063+0000 7f7f34230740 1 --2- 172.21.15.19:0/3351761136 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5642d00229c0 0x5642cfff8b70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.063+0000 7f7f34230740 1 --2- 172.21.15.19:0/3351761136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5642d0016890 0x5642d002d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.063+0000 7f7f34230740 1 --2- 172.21.15.19:0/3351761136 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5642cfff1870 0x5642d0017950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.063+0000 7f7f34230740 1 --2- 172.21.15.19:0/3351761136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5642cffcefa0 0x5642d00834f0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.063+0000 7f7f0b400640 0 client.8490 ms_handle_reset on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.063+0000 7f7f34230740 1 -- 172.21.15.19:0/3351761136 >> 172.21.15.19:0/3351761136 conn(0x5642cffc9180 msgr2=0x5642cfffb350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.063+0000 7f7f34230740 1 -- 172.21.15.19:0/3351761136 shutdown_connections 2024-03-28T13:20:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.063+0000 7f7f34230740 1 -- 172.21.15.19:0/3351761136 wait complete. 2024-03-28T13:20:02.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.063+0000 7f7f34230740 20 client.8490 trim_cache size 0 max 16384 2024-03-28T13:20:02.065 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-03-28T13:20:02.063+0000 7fde19400640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-03-28T13:20:02.065 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-03-28T13:20:02.063+0000 7fde19400640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-03-28T13:20:02.065 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-03-28T13:20:02.063+0000 7fde19400640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-03-28T13:20:02.066 INFO:tasks.ceph.mds.c.smithi019.stderr:2024-03-28T13:20:02.063+0000 7fde19400640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-03-28T13:20:02.066 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:20:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-03-28T13:20:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "25409d8f-2398-46f0-a454-ec67a07fda3f", 2024-03-28T13:20:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-03-28T13:20:02.066 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:20:02.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.064+0000 7f7f30a00640 1 -- 172.21.15.19:0/2142222663 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f0c06c530 msgr2=0x7f7f0c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.064+0000 7f7f30a00640 1 --2- 172.21.15.19:0/2142222663 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f0c06c530 0x7f7f0c06e9f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006fd0 tx=0x7f7f1c008040 comp rx=0 tx=0).stop 2024-03-28T13:20:02.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.064+0000 7f7f30a00640 1 -- 172.21.15.19:0/2142222663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f2c15e500 msgr2=0x7f7f2c161350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.064+0000 7f7f30a00640 1 --2- 172.21.15.19:0/2142222663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f2c15e500 0x7f7f2c161350 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f7f24036040 tx=0x7f7f24004a40 comp rx=0 tx=0).stop 2024-03-28T13:20:02.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.064+0000 7f7f30a00640 1 -- 172.21.15.19:0/2142222663 shutdown_connections 2024-03-28T13:20:02.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.064+0000 7f7f30a00640 1 --2- 172.21.15.19:0/2142222663 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f0c06c530 0x7f7f0c06e9f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.064+0000 7f7f30a00640 1 --2- 172.21.15.19:0/2142222663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f2c15e500 0x7f7f2c161350 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.064+0000 7f7f30a00640 1 --2- 172.21.15.19:0/2142222663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f2c0d4ee0 0x7f7f2c160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.064+0000 7f7f30a00640 1 --2- 172.21.15.19:0/2142222663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f2c0d0bc0 0x7f7f2c154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.064+0000 7f7f30a00640 1 -- 172.21.15.19:0/2142222663 >> 172.21.15.19:0/2142222663 conn(0x7f7f2c09d530 msgr2=0x7f7f2c0d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:02.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.064+0000 7f7f30a00640 1 -- 172.21.15.19:0/2142222663 shutdown_connections 2024-03-28T13:20:02.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.064+0000 7f7f30a00640 1 -- 172.21.15.19:0/2142222663 wait complete. 2024-03-28T13:20:02.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:20:02.082 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-28T13:20:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.270+0000 7f920d600640 1 Processor -- start 2024-03-28T13:20:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.271+0000 7f920d600640 1 -- start start 2024-03-28T13:20:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.274+0000 7f920d600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f92081602d0 0x7f92080b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.274+0000 7f920d600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92080d2400 con 0x7f920815fe90 2024-03-28T13:20:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.274+0000 7f920d600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92080d26e0 con 0x7f9208145cd0 2024-03-28T13:20:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.274+0000 7f920d600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f92081460b0 con 0x7f92081602d0 2024-03-28T13:20:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.274+0000 7f9207400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f920815fe90 0x7f92080b56d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:50738/0 (socket says 172.21.15.19:50738) 2024-03-28T13:20:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.274+0000 7f9207400640 1 -- 172.21.15.19:0/3668045211 learned_addr learned my addr 172.21.15.19:0/3668045211 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.274+0000 7f920cc00640 1 --2- 172.21.15.19:0/3668045211 >> v2:172.21.15.188:3300/0 conn(0x7f92081602d0 0x7f92080b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2099758298 0 0) 0x7f92080d2400 con 0x7f920815fe90 2024-03-28T13:20:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91ec003520 con 0x7f920815fe90 2024-03-28T13:20:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 388520567 0 0) 0x7f92080d26e0 con 0x7f9208145cd0 2024-03-28T13:20:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92080d2400 con 0x7f9208145cd0 2024-03-28T13:20:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1284342140 0 0) 0x7f91ec003520 con 0x7f920815fe90 2024-03-28T13:20:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 >> v2:172.21.15.188:3300/0 conn(0x7f92081602d0 msgr2=0x7f92080b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 --2- 172.21.15.19:0/3668045211 >> v2:172.21.15.188:3300/0 conn(0x7f92081602d0 0x7f92080b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 >> v1:172.21.15.188:6789/0 conn(0x7f9208145cd0 legacy=0x7f92080c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92080d29c0 con 0x7f920815fe90 2024-03-28T13:20:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f91f0001670 con 0x7f920815fe90 2024-03-28T13:20:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 >> v1:172.21.15.19:6789/0 conn(0x7f920815fe90 legacy=0x7f92080b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 --2- 172.21.15.19:0/3668045211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91ec003b40 0x7f91ec003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 --2- 172.21.15.19:0/3668045211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91ec004800 0x7f91ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 --2- 172.21.15.19:0/3668045211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91ec0051c0 0x7f91ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92081460b0 con 0x7f91ec003b40 2024-03-28T13:20:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec00c040 con 0x7f91ec004800 2024-03-28T13:20:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.275+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f91ec00c240 con 0x7f91ec0051c0 2024-03-28T13:20:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.276+0000 7f9207e00640 1 --2- 172.21.15.19:0/3668045211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91ec003b40 0x7f91ec003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.276+0000 7f9207400640 1 --2- 172.21.15.19:0/3668045211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91ec004800 0x7f91ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.276+0000 7f9207e00640 1 -- 172.21.15.19:0/3668045211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91ec0051c0 msgr2=0x7f91ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.276+0000 7f9207e00640 1 --2- 172.21.15.19:0/3668045211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91ec0051c0 0x7f91ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.276+0000 7f9207e00640 1 -- 172.21.15.19:0/3668045211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91ec004800 msgr2=0x7f91ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.276+0000 7f9207e00640 1 --2- 172.21.15.19:0/3668045211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91ec004800 0x7f91ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.276+0000 7f9207e00640 1 -- 172.21.15.19:0/3668045211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec003520 con 0x7f91ec003b40 2024-03-28T13:20:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.276+0000 7f920cc00640 1 -- 172.21.15.19:0/3668045211 reap_dead start 2024-03-28T13:20:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.278+0000 7f9207e00640 1 --2- 172.21.15.19:0/3668045211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91ec003b40 0x7f91ec003f40 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f91f8001fd0 tx=0x7f91ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e54494665dca2a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.278+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8001bf0 con 0x7f91ec003b40 2024-03-28T13:20:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.278+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f8003b70 con 0x7f91ec003b40 2024-03-28T13:20:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.278+0000 7f9206a00640 1 -- 172.21.15.19:0/3668045211 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f80050d0 con 0x7f91ec003b40 2024-03-28T13:20:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.279+0000 7f920d600640 1 -- 172.21.15.19:0/3668045211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91ec003b40 msgr2=0x7f91ec003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.279+0000 7f920d600640 1 --2- 172.21.15.19:0/3668045211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91ec003b40 0x7f91ec003f40 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f91f8001fd0 tx=0x7f91ec005830 comp rx=0 tx=0).stop 2024-03-28T13:20:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.279+0000 7f920d600640 1 -- 172.21.15.19:0/3668045211 shutdown_connections 2024-03-28T13:20:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.279+0000 7f920d600640 1 --2- 172.21.15.19:0/3668045211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91ec003b40 0x7f91ec003f40 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.279+0000 7f920d600640 1 -- 172.21.15.19:0/3668045211 >> 172.21.15.19:0/3668045211 conn(0x7f9208067280 msgr2=0x7f9208067660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.279+0000 7f920d600640 1 -- 172.21.15.19:0/3668045211 shutdown_connections 2024-03-28T13:20:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.279+0000 7f920d600640 1 -- 172.21.15.19:0/3668045211 wait complete. 2024-03-28T13:20:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.282+0000 7f920d600640 1 Processor -- start 2024-03-28T13:20:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.282+0000 7f920d600640 1 -- start start 2024-03-28T13:20:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.284+0000 7f920d600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91ec003b40 0x7f920815ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.284+0000 7f920d600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92080c91b0 0x7f92080b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.284+0000 7f920d600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92080ca7f0 0x7f92080cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.284+0000 7f920d600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92081606e0 con 0x7f91ec003b40 2024-03-28T13:20:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.284+0000 7f920d600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f92080b5240 con 0x7f92080c91b0 2024-03-28T13:20:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.284+0000 7f920d600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f92080c6f80 con 0x7f92080ca7f0 2024-03-28T13:20:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.284+0000 7f9207e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91ec003b40 0x7f920815ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.284+0000 7f9207e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91ec003b40 0x7f920815ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48022/0 (socket says 172.21.15.19:48022) 2024-03-28T13:20:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.284+0000 7f9207e00640 1 -- 172.21.15.19:0/2564882287 learned_addr learned my addr 172.21.15.19:0/2564882287 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.285+0000 7f9207400640 1 --2- 172.21.15.19:0/2564882287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92080c91b0 0x7f92080b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.285+0000 7f920cc00640 1 --2- 172.21.15.19:0/2564882287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92080ca7f0 0x7f92080cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.285+0000 7f9207e00640 1 -- 172.21.15.19:0/2564882287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92080ca7f0 msgr2=0x7f92080cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.285+0000 7f9207e00640 1 --2- 172.21.15.19:0/2564882287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92080ca7f0 0x7f92080cac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.285+0000 7f9207e00640 1 -- 172.21.15.19:0/2564882287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92080c91b0 msgr2=0x7f92080b7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.285+0000 7f9207e00640 1 --2- 172.21.15.19:0/2564882287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92080c91b0 0x7f92080b7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.285+0000 7f9207e00640 1 -- 172.21.15.19:0/2564882287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92080c9750 con 0x7f91ec003b40 2024-03-28T13:20:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.285+0000 7f920cc00640 1 --2- 172.21.15.19:0/2564882287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92080ca7f0 0x7f92080cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.285+0000 7f9207e00640 1 --2- 172.21.15.19:0/2564882287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91ec003b40 0x7f920815ba40 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f91f8001fd0 tx=0x7f91f8001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.286+0000 7f91ffe00640 1 -- 172.21.15.19:0/2564882287 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8039040 con 0x7f91ec003b40 2024-03-28T13:20:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.286+0000 7f91ffe00640 1 -- 172.21.15.19:0/2564882287 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f8042070 con 0x7f91ec003b40 2024-03-28T13:20:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.286+0000 7f91ffe00640 1 -- 172.21.15.19:0/2564882287 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8001bf0 con 0x7f91ec003b40 2024-03-28T13:20:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.286+0000 7f920d600640 1 -- 172.21.15.19:0/2564882287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92080c9a30 con 0x7f91ec003b40 2024-03-28T13:20:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.286+0000 7f920d600640 1 -- 172.21.15.19:0/2564882287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92080c9ff0 con 0x7f91ec003b40 2024-03-28T13:20:02.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.289+0000 7f91ffe00640 1 -- 172.21.15.19:0/2564882287 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f91f802c530 con 0x7f91ec003b40 2024-03-28T13:20:02.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.290+0000 7f9211a34740 1 Processor -- start 2024-03-28T13:20:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.290+0000 7f9211a34740 1 -- start start 2024-03-28T13:20:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.291+0000 7f9211a34740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad6e720 0x55b09ad416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.291+0000 7f9211a34740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6eb00 0x55b09ad714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.291+0000 7f9211a34740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad41c00 0x55b09ad719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.291+0000 7f9211a34740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b09acac6b0 con 0x55b09ad6e720 2024-03-28T13:20:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.291+0000 7f9211a34740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b09ac54750 con 0x55b09ad41c00 2024-03-28T13:20:02.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.291+0000 7f9211a34740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55b09aca8660 con 0x55b09ad6eb00 2024-03-28T13:20:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.291+0000 7f91ffe00640 1 --2- 172.21.15.19:0/2564882287 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f91e406c530 0x7f91e406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.291+0000 7f91ffe00640 1 -- 172.21.15.19:0/2564882287 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f91f80ad0d0 con 0x7f91ec003b40 2024-03-28T13:20:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.291+0000 7f920cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad6e720 0x55b09ad416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.291+0000 7f920cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad6e720 0x55b09ad416c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48026/0 (socket says 172.21.15.19:48026) 2024-03-28T13:20:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.291+0000 7f920cc00640 1 -- 172.21.15.19:0/2551871063 learned_addr learned my addr 172.21.15.19:0/2551871063 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.292+0000 7f920cc00640 1 -- 172.21.15.19:0/2551871063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6eb00 msgr2=0x55b09ad714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.292+0000 7f920cc00640 1 --2- 172.21.15.19:0/2551871063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6eb00 0x55b09ad714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.292+0000 7f920cc00640 1 -- 172.21.15.19:0/2551871063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad41c00 msgr2=0x55b09ad719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.292+0000 7f920cc00640 1 --2- 172.21.15.19:0/2551871063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad41c00 0x55b09ad719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.292+0000 7f920cc00640 1 -- 172.21.15.19:0/2551871063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b09aa8a7d0 con 0x55b09ad6e720 2024-03-28T13:20:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.292+0000 7f920cc00640 1 --2- 172.21.15.19:0/2564882287 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f91e406c530 0x7f91e406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.292+0000 7f920cc00640 1 --2- 172.21.15.19:0/2551871063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad6e720 0x55b09ad416c0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f91ec004780 tx=0x7f9200009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8165671dcb16374c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.292+0000 7f91f7e00640 1 -- 172.21.15.19:0/2551871063 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920000f040 con 0x55b09ad6e720 2024-03-28T13:20:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.292+0000 7f91f7e00640 1 -- 172.21.15.19:0/2551871063 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9200016690 con 0x55b09ad6e720 2024-03-28T13:20:02.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.292+0000 7f91f7e00640 1 -- 172.21.15.19:0/2551871063 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200016ac0 con 0x55b09ad6e720 2024-03-28T13:20:02.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.293+0000 7f9211a34740 1 -- 172.21.15.19:0/2551871063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad6e720 msgr2=0x55b09ad416c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.293+0000 7f920cc00640 1 --2- 172.21.15.19:0/2564882287 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f91e406c530 0x7f91e406e9f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f920001f000 tx=0x7f920001c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.293+0000 7f9211a34740 1 --2- 172.21.15.19:0/2551871063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad6e720 0x55b09ad416c0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f91ec004780 tx=0x7f9200009e80 comp rx=0 tx=0).stop 2024-03-28T13:20:02.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.293+0000 7f9211a34740 1 -- 172.21.15.19:0/2551871063 shutdown_connections 2024-03-28T13:20:02.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.293+0000 7f9211a34740 1 --2- 172.21.15.19:0/2551871063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad41c00 0x55b09ad719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.293+0000 7f9211a34740 1 --2- 172.21.15.19:0/2551871063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6eb00 0x55b09ad714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.293+0000 7f9211a34740 1 --2- 172.21.15.19:0/2551871063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad6e720 0x55b09ad416c0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.293+0000 7f9211a34740 1 -- 172.21.15.19:0/2551871063 >> 172.21.15.19:0/2551871063 conn(0x55b09ad43930 msgr2=0x55b09ad8c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:02.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.293+0000 7f9211a34740 1 -- 172.21.15.19:0/2551871063 shutdown_connections 2024-03-28T13:20:02.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.293+0000 7f9211a34740 1 -- 172.21.15.19:0/2551871063 wait complete. 2024-03-28T13:20:02.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.294+0000 7f9211a34740 1 -- start start 2024-03-28T13:20:02.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.294+0000 7f9211a34740 1 Processor -- start 2024-03-28T13:20:02.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.295+0000 7f9211a34740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6e720 0x55b09ad8c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.295+0000 7f9211a34740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad6eb00 0x55b09ad858c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.295+0000 7f9211a34740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad843b0 0x55b09ad74cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.295+0000 7f9211a34740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b09ad45fd0 con 0x55b09ad843b0 2024-03-28T13:20:02.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.295+0000 7f9211a34740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b09ac67a80 con 0x55b09ad6eb00 2024-03-28T13:20:02.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.295+0000 7f9211a34740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55b09ad48810 con 0x55b09ad6e720 2024-03-28T13:20:02.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.295+0000 7f9211a34740 10 client.0 _ll_register_callbacks cb 0x55b09ac67470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:20:02.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.296+0000 7f920cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad843b0 0x55b09ad74cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.296+0000 7f920cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad843b0 0x55b09ad74cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48032/0 (socket says 172.21.15.19:48032) 2024-03-28T13:20:02.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.296+0000 7f920cc00640 1 -- 172.21.15.19:0/1435832865 learned_addr learned my addr 172.21.15.19:0/1435832865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:02.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.296+0000 7f9207400640 1 --2- 172.21.15.19:0/1435832865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad6eb00 0x55b09ad858c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.296+0000 7f9207e00640 1 --2- 172.21.15.19:0/1435832865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6e720 0x55b09ad8c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.296+0000 7f920cc00640 1 -- 172.21.15.19:0/1435832865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6e720 msgr2=0x55b09ad8c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.296+0000 7f920cc00640 1 --2- 172.21.15.19:0/1435832865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6e720 0x55b09ad8c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.296+0000 7f920cc00640 1 -- 172.21.15.19:0/1435832865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad6eb00 msgr2=0x55b09ad858c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.296+0000 7f920cc00640 1 --2- 172.21.15.19:0/1435832865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad6eb00 0x55b09ad858c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.296+0000 7f920cc00640 1 -- 172.21.15.19:0/1435832865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b09a9c03f0 con 0x55b09ad843b0 2024-03-28T13:20:02.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.297+0000 7f920cc00640 1 --2- 172.21.15.19:0/1435832865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad843b0 0x55b09ad74cc0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f92000125e0 tx=0x7f9200004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62329d10c902853d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.297+0000 7f91f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:20:02.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.297+0000 7f91f4c00640 1 -- 172.21.15.19:0/1435832865 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200027040 con 0x55b09ad843b0 2024-03-28T13:20:02.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.297+0000 7f91f4c00640 1 -- 172.21.15.19:0/1435832865 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f9200015d40 con 0x55b09ad843b0 2024-03-28T13:20:02.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.297+0000 7f9211a34740 1 -- 172.21.15.19:0/1435832865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9200016690 con 0x55b09ad843b0 2024-03-28T13:20:02.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.297+0000 7f91f4c00640 1 -- 172.21.15.19:0/1435832865 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f9200016840 con 0x55b09ad843b0 2024-03-28T13:20:02.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.297+0000 7f9211a34740 10 client.8502 fetch_fsmap learned FSMap version 88 2024-03-28T13:20:02.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.297+0000 7f9211a34740 1 -- 172.21.15.19:0/1435832865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55b09a9800f0 con 0x55b09ad843b0 2024-03-28T13:20:02.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.298+0000 7f91f4c00640 1 -- 172.21.15.19:0/1435832865 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f920003b070 con 0x55b09ad843b0 2024-03-28T13:20:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.298+0000 7f91f4c00640 1 -- 172.21.15.19:0/1435832865 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f920002d040 con 0x55b09ad843b0 2024-03-28T13:20:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.298+0000 7f9211a34740 10 client.8502 fetch_fsmap finished waiting for FSMap version 88 2024-03-28T13:20:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.298+0000 7f9211a34740 10 client.8502 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-28T13:20:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.298+0000 7f9211a34740 20 client.8502 populate_metadata read hostname 'smithi019' 2024-03-28T13:20:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.298+0000 7f9211a34740 1 --2- 172.21.15.19:0/1435832865 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55b09ad86d80 0x55b09ad871a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.298+0000 7f9211a34740 4 client.8502 mds_command: new command op to 8379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:20:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.298+0000 7f9211a34740 1 -- 172.21.15.19:0/1435832865 --> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b09acb9bf0 con 0x55b09ad86d80 2024-03-28T13:20:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.298+0000 7f9207e00640 1 --2- 172.21.15.19:0/1435832865 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55b09ad86d80 0x55b09ad871a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.299+0000 7f9207e00640 1 --2- 172.21.15.19:0/1435832865 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55b09ad86d80 0x55b09ad871a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba8dc54478efdfd9 server_cookie=d5d613e5dc13de51 in_seq=0 out_seq=0 2024-03-28T13:20:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.299+0000 7f91f4c00640 10 client.8502 ms_handle_connect on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.299+0000 7f91f4c00640 1 -- 172.21.15.19:0/1435832865 <== mds.0 v2:172.21.15.19:6832/2851646201 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b09acb9bf0 con 0x55b09ad86d80 2024-03-28T13:20:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.299+0000 7f91f4c00640 10 client.8502 handle_command_reply: tid=0 2024-03-28T13:20:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.299+0000 7f9211a34740 1 client.8502 shutdown 2024-03-28T13:20:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.300+0000 7f9211a34740 1 -- 172.21.15.19:0/1435832865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad843b0 msgr2=0x55b09ad74cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.300+0000 7f9211a34740 1 --2- 172.21.15.19:0/1435832865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad843b0 0x55b09ad74cc0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f92000125e0 tx=0x7f9200004730 comp rx=0 tx=0).stop 2024-03-28T13:20:02.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.300+0000 7f9211a34740 1 -- 172.21.15.19:0/1435832865 shutdown_connections 2024-03-28T13:20:02.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.300+0000 7f9211a34740 1 --2- 172.21.15.19:0/1435832865 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55b09ad86d80 0x55b09ad871a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.300+0000 7f9211a34740 1 --2- 172.21.15.19:0/1435832865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad843b0 0x55b09ad74cc0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.300+0000 7f9211a34740 1 --2- 172.21.15.19:0/1435832865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad6eb00 0x55b09ad858c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.300+0000 7f9211a34740 1 --2- 172.21.15.19:0/1435832865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6e720 0x55b09ad8c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.300+0000 7f91f4c00640 0 client.8502 ms_handle_reset on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:02.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.300+0000 7f9211a34740 1 -- 172.21.15.19:0/1435832865 >> 172.21.15.19:0/1435832865 conn(0x55b09ad41c00 msgr2=0x55b09ad455d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:02.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.300+0000 7f9211a34740 1 -- 172.21.15.19:0/1435832865 shutdown_connections 2024-03-28T13:20:02.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.300+0000 7f9211a34740 1 -- 172.21.15.19:0/1435832865 wait complete. 2024-03-28T13:20:02.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.300+0000 7f9211a34740 20 client.8502 trim_cache size 0 max 16384 2024-03-28T13:20:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.319+0000 7f9211a34740 1 Processor -- start 2024-03-28T13:20:02.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.319+0000 7f9211a34740 1 -- start start 2024-03-28T13:20:02.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.320+0000 7f9211a34740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6e720 0x55b09ad405b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.320+0000 7f9211a34740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad6eb00 0x55b09ad712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.320+0000 7f9211a34740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad8e810 0x55b09ad717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.320+0000 7f9211a34740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92082011a0 con 0x55b09ad8e810 2024-03-28T13:20:02.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.320+0000 7f9211a34740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9208200230 con 0x55b09ad6eb00 2024-03-28T13:20:02.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.320+0000 7f9211a34740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f92080b5a30 con 0x55b09ad6e720 2024-03-28T13:20:02.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.320+0000 7f920cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad8e810 0x55b09ad717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.321+0000 7f920cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad8e810 0x55b09ad717e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48034/0 (socket says 172.21.15.19:48034) 2024-03-28T13:20:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.321+0000 7f920cc00640 1 -- 172.21.15.19:0/140299418 learned_addr learned my addr 172.21.15.19:0/140299418 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.321+0000 7f920cc00640 1 -- 172.21.15.19:0/140299418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6e720 msgr2=0x55b09ad405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.321+0000 7f920cc00640 1 --2- 172.21.15.19:0/140299418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6e720 0x55b09ad405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.321+0000 7f920cc00640 1 -- 172.21.15.19:0/140299418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad6eb00 msgr2=0x55b09ad712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.321+0000 7f920cc00640 1 --2- 172.21.15.19:0/140299418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad6eb00 0x55b09ad712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.321+0000 7f920cc00640 1 -- 172.21.15.19:0/140299418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b09ad85c30 con 0x55b09ad8e810 2024-03-28T13:20:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.321+0000 7f920cc00640 1 --2- 172.21.15.19:0/140299418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad8e810 0x55b09ad717e0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f9200009ec0 tx=0x7f920002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d40398ed230c84 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.322+0000 7f91f7400640 1 -- 172.21.15.19:0/140299418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200016d20 con 0x55b09ad8e810 2024-03-28T13:20:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.322+0000 7f91f7400640 1 -- 172.21.15.19:0/140299418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f920003b040 con 0x55b09ad8e810 2024-03-28T13:20:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.322+0000 7f91f7400640 1 -- 172.21.15.19:0/140299418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920002e720 con 0x55b09ad8e810 2024-03-28T13:20:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.322+0000 7f9211a34740 1 -- 172.21.15.19:0/140299418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad8e810 msgr2=0x55b09ad717e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.322+0000 7f9211a34740 1 --2- 172.21.15.19:0/140299418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad8e810 0x55b09ad717e0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f9200009ec0 tx=0x7f920002e000 comp rx=0 tx=0).stop 2024-03-28T13:20:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.322+0000 7f9211a34740 1 -- 172.21.15.19:0/140299418 shutdown_connections 2024-03-28T13:20:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.322+0000 7f9211a34740 1 --2- 172.21.15.19:0/140299418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad8e810 0x55b09ad717e0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.322+0000 7f9211a34740 1 --2- 172.21.15.19:0/140299418 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad6eb00 0x55b09ad712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.322+0000 7f9211a34740 1 --2- 172.21.15.19:0/140299418 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6e720 0x55b09ad405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.322+0000 7f9211a34740 1 -- 172.21.15.19:0/140299418 >> 172.21.15.19:0/140299418 conn(0x55b09ad41c00 msgr2=0x55b09ad3ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:02.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.322+0000 7f9211a34740 1 -- 172.21.15.19:0/140299418 shutdown_connections 2024-03-28T13:20:02.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.323+0000 7f9211a34740 1 -- 172.21.15.19:0/140299418 wait complete. 2024-03-28T13:20:02.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.324+0000 7f9211a34740 1 -- start start 2024-03-28T13:20:02.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.324+0000 7f9211a34740 1 Processor -- start 2024-03-28T13:20:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9211a34740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad6e720 0x55b09ae05ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9211a34740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6eb00 0x55b09ad8c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9211a34740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad8e810 0x55b09ad8c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9211a34740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b09ad85a90 con 0x55b09ad6e720 2024-03-28T13:20:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9211a34740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f92080b6080 con 0x55b09ad8e810 2024-03-28T13:20:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9211a34740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55b09ad474d0 con 0x55b09ad6eb00 2024-03-28T13:20:02.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9211a34740 10 client.0 _ll_register_callbacks cb 0x7f9200009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9207e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad6e720 0x55b09ae05ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9207e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad6e720 0x55b09ae05ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48042/0 (socket says 172.21.15.19:48042) 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9207e00640 1 -- 172.21.15.19:0/23027037 learned_addr learned my addr 172.21.15.19:0/23027037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9207400640 1 --2- 172.21.15.19:0/23027037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6eb00 0x55b09ad8c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9207e00640 1 -- 172.21.15.19:0/23027037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6eb00 msgr2=0x55b09ad8c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f920cc00640 1 --2- 172.21.15.19:0/23027037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad8e810 0x55b09ad8c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9207e00640 1 --2- 172.21.15.19:0/23027037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6eb00 0x55b09ad8c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9207e00640 1 -- 172.21.15.19:0/23027037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad8e810 msgr2=0x55b09ad8c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9207e00640 1 --2- 172.21.15.19:0/23027037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad8e810 0x55b09ad8c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9207e00640 1 -- 172.21.15.19:0/23027037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b09ad71560 con 0x55b09ad6e720 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.326+0000 7f9207400640 1 --2- 172.21.15.19:0/23027037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6eb00 0x55b09ad8c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.327+0000 7f9207e00640 1 --2- 172.21.15.19:0/23027037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad6e720 0x55b09ae05ce0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f91f80ad3c0 tx=0x7f91f807b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a818db2acf58d366 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.327+0000 7f91f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.327+0000 7f91f4c00640 1 -- 172.21.15.19:0/23027037 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8083aa0 con 0x55b09ad6e720 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.327+0000 7f91f4c00640 1 -- 172.21.15.19:0/23027037 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f91f8079130 con 0x55b09ad6e720 2024-03-28T13:20:02.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.327+0000 7f9211a34740 1 -- 172.21.15.19:0/23027037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f920003b040 con 0x55b09ad6e720 2024-03-28T13:20:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.327+0000 7f91f4c00640 1 -- 172.21.15.19:0/23027037 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f91f80ab5e0 con 0x55b09ad6e720 2024-03-28T13:20:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.327+0000 7f9211a34740 10 client.8508 fetch_fsmap learned FSMap version 88 2024-03-28T13:20:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.327+0000 7f9211a34740 1 -- 172.21.15.19:0/23027037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x55b09ad9c880 con 0x55b09ad6e720 2024-03-28T13:20:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.328+0000 7f91f4c00640 1 -- 172.21.15.19:0/23027037 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f920003b040 con 0x55b09ad6e720 2024-03-28T13:20:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.328+0000 7f91f4c00640 1 -- 172.21.15.19:0/23027037 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f91f803c070 con 0x55b09ad6e720 2024-03-28T13:20:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.328+0000 7f9211a34740 10 client.8508 fetch_fsmap finished waiting for FSMap version 88 2024-03-28T13:20:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.328+0000 7f9211a34740 10 client.8508 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-28T13:20:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.328+0000 7f9211a34740 20 client.8508 populate_metadata read hostname 'smithi019' 2024-03-28T13:20:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.328+0000 7f9211a34740 1 --2- 172.21.15.19:0/23027037 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55b09ad71070 0x55b09ad8f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.328+0000 7f9211a34740 4 client.8508 mds_command: new command op to 8379 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:20:02.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.328+0000 7f9211a34740 1 -- 172.21.15.19:0/23027037 --> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b09acb3f40 con 0x55b09ad71070 2024-03-28T13:20:02.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.328+0000 7f9207400640 1 --2- 172.21.15.19:0/23027037 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55b09ad71070 0x55b09ad8f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:02.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f9207400640 1 --2- 172.21.15.19:0/23027037 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55b09ad71070 0x55b09ad8f9c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ff73bdfe156e01 server_cookie=16692df64cecc565 in_seq=0 out_seq=0 2024-03-28T13:20:02.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f91f4c00640 10 client.8508 ms_handle_connect on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:02.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f91f4c00640 1 -- 172.21.15.19:0/23027037 <== mds.0 v2:172.21.15.19:6832/2851646201 1 ==== command_reply(tid 0: 0 ) ==== 8+0+260 (crc 0 0 0) 0x55b09acb3f40 con 0x55b09ad71070 2024-03-28T13:20:02.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f91f4c00640 10 client.8508 handle_command_reply: tid=0 2024-03-28T13:20:02.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f9211a34740 1 client.8508 shutdown 2024-03-28T13:20:02.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f9211a34740 1 -- 172.21.15.19:0/23027037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad6e720 msgr2=0x55b09ae05ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f9211a34740 1 --2- 172.21.15.19:0/23027037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad6e720 0x55b09ae05ce0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f91f80ad3c0 tx=0x7f91f807b5f0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f9211a34740 1 -- 172.21.15.19:0/23027037 shutdown_connections 2024-03-28T13:20:02.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f9211a34740 1 --2- 172.21.15.19:0/23027037 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x55b09ad71070 0x55b09ad8f9c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f9211a34740 1 --2- 172.21.15.19:0/23027037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b09ad8e810 0x55b09ad8c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f9211a34740 1 --2- 172.21.15.19:0/23027037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b09ad6eb00 0x55b09ad8c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f9211a34740 1 --2- 172.21.15.19:0/23027037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b09ad6e720 0x55b09ae05ce0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f91f4c00640 0 client.8508 ms_handle_reset on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:02.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f9211a34740 1 -- 172.21.15.19:0/23027037 >> 172.21.15.19:0/23027037 conn(0x55b09ad41c00 msgr2=0x55b09ad405b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:02.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f9211a34740 1 -- 172.21.15.19:0/23027037 shutdown_connections 2024-03-28T13:20:02.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.329+0000 7f9211a34740 1 -- 172.21.15.19:0/23027037 wait complete. 2024-03-28T13:20:02.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.330+0000 7f9211a34740 20 client.8508 trim_cache size 0 max 16384 2024-03-28T13:20:02.332 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:20:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-28T13:20:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-03-28T13:20:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "25409d8f-2398-46f0-a454-ec67a07fda3f": { 2024-03-28T13:20:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-03-28T13:20:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "25409d8f-2398-46f0-a454-ec67a07fda3f", 2024-03-28T13:20:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,repair" 2024-03-28T13:20:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:02.333 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:20:02.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.330+0000 7f920d600640 1 -- 172.21.15.19:0/2564882287 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f91e406c530 msgr2=0x7f91e406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.330+0000 7f920d600640 1 --2- 172.21.15.19:0/2564882287 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f91e406c530 0x7f91e406e9f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f920001f000 tx=0x7f920001c000 comp rx=0 tx=0).stop 2024-03-28T13:20:02.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.330+0000 7f920d600640 1 -- 172.21.15.19:0/2564882287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91ec003b40 msgr2=0x7f920815ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:02.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.330+0000 7f920d600640 1 --2- 172.21.15.19:0/2564882287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91ec003b40 0x7f920815ba40 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f91f8001fd0 tx=0x7f91f8001650 comp rx=0 tx=0).stop 2024-03-28T13:20:02.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.330+0000 7f920d600640 1 -- 172.21.15.19:0/2564882287 shutdown_connections 2024-03-28T13:20:02.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.330+0000 7f920d600640 1 --2- 172.21.15.19:0/2564882287 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f91e406c530 0x7f91e406e9f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.330+0000 7f920d600640 1 --2- 172.21.15.19:0/2564882287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92080ca7f0 0x7f92080cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.330+0000 7f920d600640 1 --2- 172.21.15.19:0/2564882287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92080c91b0 0x7f92080b7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.330+0000 7f920d600640 1 --2- 172.21.15.19:0/2564882287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91ec003b40 0x7f920815ba40 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:02.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.330+0000 7f920d600640 1 -- 172.21.15.19:0/2564882287 >> 172.21.15.19:0/2564882287 conn(0x7f9208067280 msgr2=0x7f9208141b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:02.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.330+0000 7f920d600640 1 -- 172.21.15.19:0/2564882287 shutdown_connections 2024-03-28T13:20:02.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:02.331+0000 7f920d600640 1 -- 172.21.15.19:0/2564882287 wait complete. 2024-03-28T13:20:02.347 INFO:tasks.cephfs.filesystem:scrub status for tag:25409d8f-2398-46f0-a454-ec67a07fda3f - {'path': '/', 'tag': '25409d8f-2398-46f0-a454-ec67a07fda3f', 'options': 'recursive,repair'} 2024-03-28T13:20:03.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:03.783+0000 7fa636a00640 1 -- 172.21.15.19:0/2221956982 <== mon.0 v2:172.21.15.19:3300/0 7 ==== log(8 entries from seq 1 at 2024-03-28T13:20:02.063123+0000) ==== 1407+0+0 (secure 0 0 0) 0x7fa638058520 con 0x7fa64015ba70 2024-03-28T13:20:03.784 INFO:teuthology.orchestra.run.smithi019.stdout: cluster: 2024-03-28T13:20:03.784 INFO:teuthology.orchestra.run.smithi019.stdout: id: e6b569d9-f2ce-49dc-bbe7-c0ab33080203 2024-03-28T13:20:03.784 INFO:teuthology.orchestra.run.smithi019.stdout: health: HEALTH_OK 2024-03-28T13:20:03.784 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:03.785 INFO:teuthology.orchestra.run.smithi019.stdout: services: 2024-03-28T13:20:03.785 INFO:teuthology.orchestra.run.smithi019.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-03-28T13:20:03.785 INFO:teuthology.orchestra.run.smithi019.stdout: mgr: x(active, since 10m), standbys: y 2024-03-28T13:20:03.785 INFO:teuthology.orchestra.run.smithi019.stdout: mds: 1/1 daemons up, 3 standby 2024-03-28T13:20:03.785 INFO:teuthology.orchestra.run.smithi019.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-03-28T13:20:03.785 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: data: 2024-03-28T13:20:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: volumes: 1/1 healthy 2024-03-28T13:20:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: pools: 3 pools, 97 pgs 2024-03-28T13:20:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: objects: 30 objects, 19 MiB 2024-03-28T13:20:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-03-28T13:20:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: pgs: 97 active+clean 2024-03-28T13:20:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: io: 2024-03-28T13:20:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: client: 4.2 KiB/s rd, 102 B/s wr, 4 op/s rd, 0 op/s wr 2024-03-28T13:20:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:03.786 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:03.786 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:20:02.063123+0000 mds.c [INF] scrub queued for path: / 2024-03-28T13:20:03.786 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:20:02.063129+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-03-28T13:20:03.787 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:20:02.063149+0000 mds.c [INF] scrub summary: active paths [/] 2024-03-28T13:20:03.787 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:20:02.064768+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-03-28T13:20:03.787 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:20:02.064774+0000 mds.c [ERR] inode table repaired for inode: 0x10000000001 2024-03-28T13:20:03.787 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:20:02.064867+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-03-28T13:20:03.787 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:20:02.064871+0000 mds.c [ERR] inode table repaired for inode: 0x10000000002 2024-03-28T13:20:03.787 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:20:02.065382+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-03-28T13:20:04.154 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:20:04.165 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:20:07.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:07.795+0000 7fa636a00640 1 -- 172.21.15.19:0/2221956982 <== mon.0 v2:172.21.15.19:3300/0 8 ==== log(1 entries from seq 9 at 2024-03-28T13:20:05.917036+0000) ==== 189+0+0 (secure 0 0 0) 0x7fa638058780 con 0x7fa64015ba70 2024-03-28T13:20:07.795 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:20:05.917036+0000 mds.c [INF] scrub summary: idle 2024-03-28T13:20:32.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:20:32.348 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-28T13:20:32.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.577+0000 7f1d2fe00640 1 Processor -- start 2024-03-28T13:20:32.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.577+0000 7f1d2fe00640 1 -- start start 2024-03-28T13:20:32.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.578+0000 7f1d2fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d3015e4c0 0x7f1d300b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.578+0000 7f1d2fe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d300d2710 con 0x7f1d300d37b0 2024-03-28T13:20:32.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.578+0000 7f1d2fe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d300d29f0 con 0x7f1d300b69c0 2024-03-28T13:20:32.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.578+0000 7f1d2fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3015e8a0 con 0x7f1d3015e4c0 2024-03-28T13:20:32.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.578+0000 7f1d2e000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1d300d37b0 0x7f1d300b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58354/0 (socket says 172.21.15.19:58354) 2024-03-28T13:20:32.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.578+0000 7f1d2e000640 1 -- 172.21.15.19:0/294052260 learned_addr learned my addr 172.21.15.19:0/294052260 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:32.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.579+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/294052260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d3015e4c0 0x7f1d300b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.579+0000 7f1d2d600640 1 -- 172.21.15.19:0/294052260 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3107373250 0 0) 0x7f1d300d2710 con 0x7f1d300d37b0 2024-03-28T13:20:32.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.579+0000 7f1d2d600640 1 -- 172.21.15.19:0/294052260 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d10003520 con 0x7f1d300d37b0 2024-03-28T13:20:32.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.579+0000 7f1d2d600640 1 -- 172.21.15.19:0/294052260 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2480960635 0 0) 0x7f1d10003520 con 0x7f1d300d37b0 2024-03-28T13:20:32.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.579+0000 7f1d2d600640 1 -- 172.21.15.19:0/294052260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d3015e4c0 msgr2=0x7f1d300b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:32.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.579+0000 7f1d2d600640 1 --2- 172.21.15.19:0/294052260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d3015e4c0 0x7f1d300b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.579+0000 7f1d2d600640 1 -- 172.21.15.19:0/294052260 >> v1:172.21.15.188:6789/0 conn(0x7f1d300b69c0 legacy=0x7f1d300b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.580+0000 7f1d2d600640 1 -- 172.21.15.19:0/294052260 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d300d2cd0 con 0x7f1d300d37b0 2024-03-28T13:20:32.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.580+0000 7f1d2d600640 1 -- 172.21.15.19:0/294052260 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f1d180019f0 con 0x7f1d300d37b0 2024-03-28T13:20:32.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.580+0000 7f1d2d600640 1 -- 172.21.15.19:0/294052260 >> v1:172.21.15.19:6789/0 conn(0x7f1d300d37b0 legacy=0x7f1d300b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.580+0000 7f1d2d600640 1 --2- 172.21.15.19:0/294052260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d100039e0 0x7f1d10003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.580+0000 7f1d2d600640 1 --2- 172.21.15.19:0/294052260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10004390 0x7f1d10004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.580+0000 7f1d2d600640 1 --2- 172.21.15.19:0/294052260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d3015e4c0 0x7f1d100050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.580+0000 7f1d2d600640 1 -- 172.21.15.19:0/294052260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1000b9c0 con 0x7f1d10004390 2024-03-28T13:20:32.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.580+0000 7f1d2d600640 1 -- 172.21.15.19:0/294052260 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1000bb40 con 0x7f1d100039e0 2024-03-28T13:20:32.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.580+0000 7f1d2d600640 1 -- 172.21.15.19:0/294052260 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1d1000bd40 con 0x7f1d3015e4c0 2024-03-28T13:20:32.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.580+0000 7f1d2e000640 1 --2- 172.21.15.19:0/294052260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10004390 0x7f1d10004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.580+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/294052260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d100039e0 0x7f1d10003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.581+0000 7f1d2e000640 1 -- 172.21.15.19:0/294052260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d3015e4c0 msgr2=0x7f1d100050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:32.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.581+0000 7f1d2f400640 1 --2- 172.21.15.19:0/294052260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d3015e4c0 0x7f1d100050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.581+0000 7f1d2e000640 1 --2- 172.21.15.19:0/294052260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d3015e4c0 0x7f1d100050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.581+0000 7f1d2e000640 1 -- 172.21.15.19:0/294052260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d100039e0 msgr2=0x7f1d10003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:32.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.581+0000 7f1d2e000640 1 --2- 172.21.15.19:0/294052260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d100039e0 0x7f1d10003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.581+0000 7f1d2e000640 1 -- 172.21.15.19:0/294052260 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d10003520 con 0x7f1d10004390 2024-03-28T13:20:32.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.581+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/294052260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d100039e0 0x7f1d10003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:20:32.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.581+0000 7f1d2f400640 1 --2- 172.21.15.19:0/294052260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d3015e4c0 0x7f1d100050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:32.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.582+0000 7f1d2e000640 1 --2- 172.21.15.19:0/294052260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10004390 0x7f1d10004860 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800a860 tx=0x7f1d10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea0800e7beeddf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:32.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.582+0000 7f1d2d600640 1 -- 172.21.15.19:0/294052260 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1800a480 con 0x7f1d10004390 2024-03-28T13:20:32.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.582+0000 7f1d2d600640 1 -- 172.21.15.19:0/294052260 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d18034630 con 0x7f1d10004390 2024-03-28T13:20:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.582+0000 7f1d2d600640 1 -- 172.21.15.19:0/294052260 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18034a40 con 0x7f1d10004390 2024-03-28T13:20:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.583+0000 7f1d2fe00640 1 -- 172.21.15.19:0/294052260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10004390 msgr2=0x7f1d10004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.583+0000 7f1d2fe00640 1 --2- 172.21.15.19:0/294052260 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10004390 0x7f1d10004860 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800a860 tx=0x7f1d10005830 comp rx=0 tx=0).stop 2024-03-28T13:20:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.583+0000 7f1d2f400640 1 -- 172.21.15.19:0/294052260 reap_dead start 2024-03-28T13:20:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.583+0000 7f1d2fe00640 1 -- 172.21.15.19:0/294052260 shutdown_connections 2024-03-28T13:20:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.583+0000 7f1d2fe00640 1 -- 172.21.15.19:0/294052260 >> 172.21.15.19:0/294052260 conn(0x7f1d300676a0 msgr2=0x7f1d30067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.583+0000 7f1d2fe00640 1 -- 172.21.15.19:0/294052260 shutdown_connections 2024-03-28T13:20:32.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.583+0000 7f1d2fe00640 1 -- 172.21.15.19:0/294052260 wait complete. 2024-03-28T13:20:32.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.585+0000 7f1d2fe00640 1 Processor -- start 2024-03-28T13:20:32.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.585+0000 7f1d2fe00640 1 -- start start 2024-03-28T13:20:32.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.586+0000 7f1d2fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10004390 0x7f1d301f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.586+0000 7f1d2fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d301f4440 0x7f1d301613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.586+0000 7f1d2fe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d301595f0 0x7f1d30159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.586+0000 7f1d2fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d300d3320 con 0x7f1d10004390 2024-03-28T13:20:32.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.586+0000 7f1d2fe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1d300d4040 con 0x7f1d301595f0 2024-03-28T13:20:32.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.586+0000 7f1d2fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1d300d2350 con 0x7f1d301f4440 2024-03-28T13:20:32.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.586+0000 7f1d2ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10004390 0x7f1d301f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.586+0000 7f1d2ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10004390 0x7f1d301f6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57174/0 (socket says 172.21.15.19:57174) 2024-03-28T13:20:32.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.586+0000 7f1d2ea00640 1 -- 172.21.15.19:0/3612577897 learned_addr learned my addr 172.21.15.19:0/3612577897 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:32.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.587+0000 7f1d2f400640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d301595f0 0x7f1d30159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.587+0000 7f1d2e000640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d301f4440 0x7f1d301613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.587+0000 7f1d2ea00640 1 -- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d301f4440 msgr2=0x7f1d301613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.587+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d301f4440 0x7f1d301613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.587+0000 7f1d2ea00640 1 -- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d301595f0 msgr2=0x7f1d30159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.587+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d301595f0 0x7f1d30159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.587+0000 7f1d2ea00640 1 -- 172.21.15.19:0/3612577897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d301f48c0 con 0x7f1d10004390 2024-03-28T13:20:32.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.587+0000 7f1d2f400640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d301595f0 0x7f1d30159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:32.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.587+0000 7f1d2e000640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d301f4440 0x7f1d301613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:32.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.587+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10004390 0x7f1d301f6fb0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f1d200116d0 tx=0x7f1d20011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:32.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.588+0000 7f1d26a00640 1 -- 172.21.15.19:0/3612577897 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2000f9c0 con 0x7f1d10004390 2024-03-28T13:20:32.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.588+0000 7f1d26a00640 1 -- 172.21.15.19:0/3612577897 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d2000fb60 con 0x7f1d10004390 2024-03-28T13:20:32.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.588+0000 7f1d2fe00640 1 -- 172.21.15.19:0/3612577897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d301f7590 con 0x7f1d10004390 2024-03-28T13:20:32.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.588+0000 7f1d26a00640 1 -- 172.21.15.19:0/3612577897 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20015460 con 0x7f1d10004390 2024-03-28T13:20:32.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.588+0000 7f1d2fe00640 1 -- 172.21.15.19:0/3612577897 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d301f7b50 con 0x7f1d10004390 2024-03-28T13:20:32.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.590+0000 7f1d26a00640 1 -- 172.21.15.19:0/3612577897 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1d20015600 con 0x7f1d10004390 2024-03-28T13:20:32.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.591+0000 7f1d26a00640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d0c06c560 0x7f1d0c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.591+0000 7f1d26a00640 1 -- 172.21.15.19:0/3612577897 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f1d2008c1f0 con 0x7f1d10004390 2024-03-28T13:20:32.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.592+0000 7f1d3886d740 1 Processor -- start 2024-03-28T13:20:32.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.592+0000 7f1d2e000640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d0c06c560 0x7f1d0c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.592+0000 7f1d3886d740 1 -- start start 2024-03-28T13:20:32.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.593+0000 7f1d3886d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6e720 0x5622bfc416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.593+0000 7f1d3886d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc6eb00 0x5622bfc714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.593+0000 7f1d3886d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc41c00 0x5622bfc719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.593+0000 7f1d3886d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5622bfbac6b0 con 0x5622bfc6e720 2024-03-28T13:20:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.593+0000 7f1d2e000640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d0c06c560 0x7f1d0c06ea20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800a860 tx=0x7f1d18009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.593+0000 7f1d3886d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5622bfb54750 con 0x5622bfc41c00 2024-03-28T13:20:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.593+0000 7f1d3886d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5622bfba8660 con 0x5622bfc6eb00 2024-03-28T13:20:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.594+0000 7f1d2ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6e720 0x5622bfc416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.594+0000 7f1d2ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6e720 0x5622bfc416c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57186/0 (socket says 172.21.15.19:57186) 2024-03-28T13:20:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.594+0000 7f1d2ea00640 1 -- 172.21.15.19:0/3366593153 learned_addr learned my addr 172.21.15.19:0/3366593153 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.594+0000 7f1d2f400640 1 --2- 172.21.15.19:0/3366593153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc41c00 0x5622bfc719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.594+0000 7f1d2e000640 1 --2- 172.21.15.19:0/3366593153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc6eb00 0x5622bfc714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.594+0000 7f1d2ea00640 1 -- 172.21.15.19:0/3366593153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc6eb00 msgr2=0x5622bfc714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.594+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/3366593153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc6eb00 0x5622bfc714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.594+0000 7f1d2ea00640 1 -- 172.21.15.19:0/3366593153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc41c00 msgr2=0x5622bfc719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.594+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/3366593153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc41c00 0x5622bfc719f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.594+0000 7f1d2ea00640 1 -- 172.21.15.19:0/3366593153 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622bf98a7d0 con 0x5622bfc6e720 2024-03-28T13:20:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.594+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/3366593153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6e720 0x5622bfc416c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f1d20062690 tx=0x7f1d200563b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190efcf9e9c9699f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.595+0000 7f1d1f400640 1 -- 172.21.15.19:0/3366593153 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20088070 con 0x5622bfc6e720 2024-03-28T13:20:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.595+0000 7f1d1f400640 1 -- 172.21.15.19:0/3366593153 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d2001bdc0 con 0x5622bfc6e720 2024-03-28T13:20:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.595+0000 7f1d1f400640 1 -- 172.21.15.19:0/3366593153 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20056650 con 0x5622bfc6e720 2024-03-28T13:20:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.595+0000 7f1d3886d740 1 -- 172.21.15.19:0/3366593153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6e720 msgr2=0x5622bfc416c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.595+0000 7f1d3886d740 1 --2- 172.21.15.19:0/3366593153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6e720 0x5622bfc416c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f1d20062690 tx=0x7f1d200563b0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.595+0000 7f1d3886d740 1 -- 172.21.15.19:0/3366593153 shutdown_connections 2024-03-28T13:20:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.595+0000 7f1d3886d740 1 --2- 172.21.15.19:0/3366593153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc41c00 0x5622bfc719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.595+0000 7f1d3886d740 1 --2- 172.21.15.19:0/3366593153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc6eb00 0x5622bfc714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.595+0000 7f1d3886d740 1 --2- 172.21.15.19:0/3366593153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6e720 0x5622bfc416c0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.595+0000 7f1d3886d740 1 -- 172.21.15.19:0/3366593153 >> 172.21.15.19:0/3366593153 conn(0x5622bfc43930 msgr2=0x5622bfc8c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.596+0000 7f1d3886d740 1 -- 172.21.15.19:0/3366593153 shutdown_connections 2024-03-28T13:20:32.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.596+0000 7f1d3886d740 1 -- 172.21.15.19:0/3366593153 wait complete. 2024-03-28T13:20:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.596+0000 7f1d3886d740 1 -- start start 2024-03-28T13:20:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.597+0000 7f1d3886d740 1 Processor -- start 2024-03-28T13:20:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.597+0000 7f1d3886d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6e720 0x5622bfc8cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d3886d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc6eb00 0x5622bfc68560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d3886d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc70b80 0x5622bfc69fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d3886d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5622bfc45fd0 con 0x5622bfc6e720 2024-03-28T13:20:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d3886d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5622bfb67a80 con 0x5622bfc70b80 2024-03-28T13:20:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d3886d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5622bfc48810 con 0x5622bfc6eb00 2024-03-28T13:20:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d3886d740 10 client.0 _ll_register_callbacks cb 0x5622bfb67470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:20:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d2ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6e720 0x5622bfc8cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d2ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6e720 0x5622bfc8cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57202/0 (socket says 172.21.15.19:57202) 2024-03-28T13:20:32.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d2ea00640 1 -- 172.21.15.19:0/1681309061 learned_addr learned my addr 172.21.15.19:0/1681309061 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d2e000640 1 --2- 172.21.15.19:0/1681309061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc6eb00 0x5622bfc68560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d2f400640 1 --2- 172.21.15.19:0/1681309061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc70b80 0x5622bfc69fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d2ea00640 1 -- 172.21.15.19:0/1681309061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc6eb00 msgr2=0x5622bfc68560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/1681309061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc6eb00 0x5622bfc68560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d2ea00640 1 -- 172.21.15.19:0/1681309061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc70b80 msgr2=0x5622bfc69fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/1681309061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc70b80 0x5622bfc69fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d2ea00640 1 -- 172.21.15.19:0/1681309061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622bf8c03f0 con 0x5622bfc6e720 2024-03-28T13:20:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.598+0000 7f1d2f400640 1 --2- 172.21.15.19:0/1681309061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc70b80 0x5622bfc69fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.599+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/1681309061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6e720 0x5622bfc8cce0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f1d20056440 tx=0x7f1d20009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27db37c7b2e9e259 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:32.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.599+0000 7f1d17e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:20:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.599+0000 7f1d17e00640 1 -- 172.21.15.19:0/1681309061 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20088070 con 0x5622bfc6e720 2024-03-28T13:20:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.599+0000 7f1d17e00640 1 -- 172.21.15.19:0/1681309061 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f1d20062730 con 0x5622bfc6e720 2024-03-28T13:20:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.599+0000 7f1d3886d740 1 -- 172.21.15.19:0/1681309061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d2001bdc0 con 0x5622bfc6e720 2024-03-28T13:20:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.599+0000 7f1d17e00640 1 -- 172.21.15.19:0/1681309061 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f1d20079420 con 0x5622bfc6e720 2024-03-28T13:20:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.599+0000 7f1d3886d740 10 client.8520 fetch_fsmap learned FSMap version 88 2024-03-28T13:20:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.599+0000 7f1d3886d740 1 -- 172.21.15.19:0/1681309061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5622bf8800f0 con 0x5622bfc6e720 2024-03-28T13:20:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.599+0000 7f1d17e00640 1 -- 172.21.15.19:0/1681309061 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d2001bdc0 con 0x5622bfc6e720 2024-03-28T13:20:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.600+0000 7f1d17e00640 1 -- 172.21.15.19:0/1681309061 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f1d20059430 con 0x5622bfc6e720 2024-03-28T13:20:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.600+0000 7f1d3886d740 10 client.8520 fetch_fsmap finished waiting for FSMap version 88 2024-03-28T13:20:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.600+0000 7f1d3886d740 10 client.8520 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-28T13:20:32.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.600+0000 7f1d3886d740 20 client.8520 populate_metadata read hostname 'smithi019' 2024-03-28T13:20:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.600+0000 7f1d3886d740 1 --2- 172.21.15.19:0/1681309061 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5622bfca0790 0x5622bfca0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.600+0000 7f1d3886d740 4 client.8520 mds_command: new command op to 8379 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:20:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.600+0000 7f1d3886d740 1 -- 172.21.15.19:0/1681309061 --> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622bfbb9bf0 con 0x5622bfca0790 2024-03-28T13:20:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.600+0000 7f1d2e000640 1 --2- 172.21.15.19:0/1681309061 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5622bfca0790 0x5622bfca0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.601+0000 7f1d17e00640 10 client.8520 ms_handle_connect on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.601+0000 7f1d2e000640 1 --2- 172.21.15.19:0/1681309061 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5622bfca0790 0x5622bfca0b70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=445c0805dab66ece server_cookie=7689f577dc7f1e86 in_seq=0 out_seq=0 2024-03-28T13:20:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.601+0000 7f1d17e00640 1 -- 172.21.15.19:0/1681309061 <== mds.0 v2:172.21.15.19:6832/2851646201 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5622bfbb9bf0 con 0x5622bfca0790 2024-03-28T13:20:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.601+0000 7f1d17e00640 10 client.8520 handle_command_reply: tid=0 2024-03-28T13:20:32.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.601+0000 7f1d3886d740 1 client.8520 shutdown 2024-03-28T13:20:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.602+0000 7f1d3886d740 1 -- 172.21.15.19:0/1681309061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6e720 msgr2=0x5622bfc8cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.602+0000 7f1d3886d740 1 --2- 172.21.15.19:0/1681309061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6e720 0x5622bfc8cce0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f1d20056440 tx=0x7f1d20009f70 comp rx=0 tx=0).stop 2024-03-28T13:20:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.602+0000 7f1d3886d740 1 -- 172.21.15.19:0/1681309061 shutdown_connections 2024-03-28T13:20:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.602+0000 7f1d3886d740 1 --2- 172.21.15.19:0/1681309061 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5622bfca0790 0x5622bfca0b70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.602+0000 7f1d3886d740 1 --2- 172.21.15.19:0/1681309061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc70b80 0x5622bfc69fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.602+0000 7f1d3886d740 1 --2- 172.21.15.19:0/1681309061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc6eb00 0x5622bfc68560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.602+0000 7f1d3886d740 1 --2- 172.21.15.19:0/1681309061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6e720 0x5622bfc8cce0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.602+0000 7f1d3886d740 1 -- 172.21.15.19:0/1681309061 >> 172.21.15.19:0/1681309061 conn(0x5622bfc41c00 msgr2=0x5622bfc465c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.602+0000 7f1d17e00640 0 client.8520 ms_handle_reset on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.602+0000 7f1d3886d740 1 -- 172.21.15.19:0/1681309061 shutdown_connections 2024-03-28T13:20:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.602+0000 7f1d3886d740 1 -- 172.21.15.19:0/1681309061 wait complete. 2024-03-28T13:20:32.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.602+0000 7f1d3886d740 20 client.8520 trim_cache size 0 max 16384 2024-03-28T13:20:32.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.621+0000 7f1d3886d740 1 Processor -- start 2024-03-28T13:20:32.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.621+0000 7f1d3886d740 1 -- start start 2024-03-28T13:20:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.622+0000 7f1d3886d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc6e720 0x5622bfc6faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.622+0000 7f1d3886d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6eb00 0x5622bfc6df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.622+0000 7f1d3886d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc70b80 0x5622bfc6f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.622+0000 7f1d3886d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1d302010e0 con 0x5622bfc6eb00 2024-03-28T13:20:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.622+0000 7f1d3886d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1d302000e0 con 0x5622bfc6e720 2024-03-28T13:20:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.622+0000 7f1d3886d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1d301ff470 con 0x5622bfc70b80 2024-03-28T13:20:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.622+0000 7f1d2e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6eb00 0x5622bfc6df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.622+0000 7f1d2e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6eb00 0x5622bfc6df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57206/0 (socket says 172.21.15.19:57206) 2024-03-28T13:20:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.623+0000 7f1d2e000640 1 -- 172.21.15.19:0/2031274453 learned_addr learned my addr 172.21.15.19:0/2031274453 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.623+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/2031274453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc6e720 0x5622bfc6faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.623+0000 7f1d2f400640 1 --2- 172.21.15.19:0/2031274453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc70b80 0x5622bfc6f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.623+0000 7f1d2e000640 1 -- 172.21.15.19:0/2031274453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc70b80 msgr2=0x5622bfc6f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.623+0000 7f1d2e000640 1 --2- 172.21.15.19:0/2031274453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc70b80 0x5622bfc6f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.623+0000 7f1d2e000640 1 -- 172.21.15.19:0/2031274453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc6e720 msgr2=0x5622bfc6faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.623+0000 7f1d2e000640 1 --2- 172.21.15.19:0/2031274453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc6e720 0x5622bfc6faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.623+0000 7f1d2e000640 1 -- 172.21.15.19:0/2031274453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622bfc445c0 con 0x5622bfc6eb00 2024-03-28T13:20:32.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.623+0000 7f1d2e000640 1 --2- 172.21.15.19:0/2031274453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6eb00 0x5622bfc6df20 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f1d1802cc50 tx=0x7f1d18007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e173bec569a024 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.624+0000 7f1d1ea00640 1 -- 172.21.15.19:0/2031274453 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d18043040 con 0x5622bfc6eb00 2024-03-28T13:20:32.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.624+0000 7f1d1ea00640 1 -- 172.21.15.19:0/2031274453 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d18004be0 con 0x5622bfc6eb00 2024-03-28T13:20:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.624+0000 7f1d1ea00640 1 -- 172.21.15.19:0/2031274453 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d180512b0 con 0x5622bfc6eb00 2024-03-28T13:20:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.624+0000 7f1d3886d740 1 -- 172.21.15.19:0/2031274453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6eb00 msgr2=0x5622bfc6df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.624+0000 7f1d3886d740 1 --2- 172.21.15.19:0/2031274453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6eb00 0x5622bfc6df20 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f1d1802cc50 tx=0x7f1d18007000 comp rx=0 tx=0).stop 2024-03-28T13:20:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.624+0000 7f1d3886d740 1 -- 172.21.15.19:0/2031274453 shutdown_connections 2024-03-28T13:20:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.624+0000 7f1d3886d740 1 --2- 172.21.15.19:0/2031274453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc70b80 0x5622bfc6f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.624+0000 7f1d3886d740 1 --2- 172.21.15.19:0/2031274453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6eb00 0x5622bfc6df20 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.624+0000 7f1d3886d740 1 --2- 172.21.15.19:0/2031274453 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc6e720 0x5622bfc6faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.625+0000 7f1d3886d740 1 -- 172.21.15.19:0/2031274453 >> 172.21.15.19:0/2031274453 conn(0x5622bfc41c00 msgr2=0x5622bfc72d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:32.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.625+0000 7f1d3886d740 1 -- 172.21.15.19:0/2031274453 shutdown_connections 2024-03-28T13:20:32.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.625+0000 7f1d3886d740 1 -- 172.21.15.19:0/2031274453 wait complete. 2024-03-28T13:20:32.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.626+0000 7f1d3886d740 1 -- start start 2024-03-28T13:20:32.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.626+0000 7f1d3886d740 1 Processor -- start 2024-03-28T13:20:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.628+0000 7f1d3886d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc6e720 0x5622bfd0fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.628+0000 7f1d3886d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6eb00 0x5622bfc9f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.628+0000 7f1d3886d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc70b80 0x5622bfd12560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.628+0000 7f1d3886d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5622bfc67b60 con 0x5622bfc6eb00 2024-03-28T13:20:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.628+0000 7f1d3886d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301ffac0 con 0x5622bfc70b80 2024-03-28T13:20:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.628+0000 7f1d3886d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5622bfc67f20 con 0x5622bfc6e720 2024-03-28T13:20:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.628+0000 7f1d3886d740 10 client.0 _ll_register_callbacks cb 0x7f1d20086070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:20:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.628+0000 7f1d2e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6eb00 0x5622bfc9f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.628+0000 7f1d2e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6eb00 0x5622bfc9f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57220/0 (socket says 172.21.15.19:57220) 2024-03-28T13:20:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.628+0000 7f1d2e000640 1 -- 172.21.15.19:0/4277729331 learned_addr learned my addr 172.21.15.19:0/4277729331 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:32.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.628+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/4277729331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc6e720 0x5622bfd0fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.628+0000 7f1d2f400640 1 --2- 172.21.15.19:0/4277729331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc70b80 0x5622bfd12560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.628+0000 7f1d2e000640 1 -- 172.21.15.19:0/4277729331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc6e720 msgr2=0x5622bfd0fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.628+0000 7f1d2e000640 1 --2- 172.21.15.19:0/4277729331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc6e720 0x5622bfd0fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.628+0000 7f1d2e000640 1 -- 172.21.15.19:0/4277729331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc70b80 msgr2=0x5622bfd12560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.629+0000 7f1d2e000640 1 --2- 172.21.15.19:0/4277729331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc70b80 0x5622bfd12560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.629+0000 7f1d2e000640 1 -- 172.21.15.19:0/4277729331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622bfc8c5b0 con 0x5622bfc6eb00 2024-03-28T13:20:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.629+0000 7f1d2e000640 1 --2- 172.21.15.19:0/4277729331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6eb00 0x5622bfc9f350 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f1d18037040 tx=0x7f1d300d3290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a484718555bdce server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.629+0000 7f1d17e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:20:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.629+0000 7f1d17e00640 1 -- 172.21.15.19:0/4277729331 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1805f020 con 0x5622bfc6eb00 2024-03-28T13:20:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.629+0000 7f1d17e00640 1 -- 172.21.15.19:0/4277729331 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f1d180512b0 con 0x5622bfc6eb00 2024-03-28T13:20:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.629+0000 7f1d3886d740 1 -- 172.21.15.19:0/4277729331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d18004be0 con 0x5622bfc6eb00 2024-03-28T13:20:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.630+0000 7f1d17e00640 1 -- 172.21.15.19:0/4277729331 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f1d18043070 con 0x5622bfc6eb00 2024-03-28T13:20:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.630+0000 7f1d3886d740 10 client.8526 fetch_fsmap learned FSMap version 88 2024-03-28T13:20:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.630+0000 7f1d3886d740 1 -- 172.21.15.19:0/4277729331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) -- 0x5622bfd0f570 con 0x5622bfc6eb00 2024-03-28T13:20:32.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.630+0000 7f1d17e00640 1 -- 172.21.15.19:0/4277729331 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d18004be0 con 0x5622bfc6eb00 2024-03-28T13:20:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.630+0000 7f1d17e00640 1 -- 172.21.15.19:0/4277729331 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 88) ==== 3494+0+0 (secure 0 0 0) 0x7f1d1806c020 con 0x5622bfc6eb00 2024-03-28T13:20:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.630+0000 7f1d3886d740 10 client.8526 fetch_fsmap finished waiting for FSMap version 88 2024-03-28T13:20:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.630+0000 7f1d3886d740 10 client.8526 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-28T13:20:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.630+0000 7f1d3886d740 20 client.8526 populate_metadata read hostname 'smithi019' 2024-03-28T13:20:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.630+0000 7f1d3886d740 1 --2- 172.21.15.19:0/4277729331 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5622bfcf8390 0x5622bfc70020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.630+0000 7f1d3886d740 4 client.8526 mds_command: new command op to 8379 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-28T13:20:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.630+0000 7f1d3886d740 1 -- 172.21.15.19:0/4277729331 --> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5622bfbb3f40 con 0x5622bfcf8390 2024-03-28T13:20:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.630+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/4277729331 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5622bfcf8390 0x5622bfc70020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.631+0000 7f1d2ea00640 1 --2- 172.21.15.19:0/4277729331 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5622bfcf8390 0x5622bfc70020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80be1a027e451bd3 server_cookie=3002744122339255 in_seq=0 out_seq=0 2024-03-28T13:20:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.631+0000 7f1d17e00640 10 client.8526 ms_handle_connect on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.631+0000 7f1d17e00640 1 -- 172.21.15.19:0/4277729331 <== mds.0 v2:172.21.15.19:6832/2851646201 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5622bfbb3f40 con 0x5622bfcf8390 2024-03-28T13:20:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.631+0000 7f1d17e00640 10 client.8526 handle_command_reply: tid=0 2024-03-28T13:20:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.631+0000 7f1d3886d740 1 client.8526 shutdown 2024-03-28T13:20:32.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.631+0000 7f1d3886d740 1 -- 172.21.15.19:0/4277729331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6eb00 msgr2=0x5622bfc9f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.632+0000 7f1d3886d740 1 --2- 172.21.15.19:0/4277729331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6eb00 0x5622bfc9f350 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f1d18037040 tx=0x7f1d300d3290 comp rx=0 tx=0).stop 2024-03-28T13:20:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.632+0000 7f1d3886d740 1 -- 172.21.15.19:0/4277729331 shutdown_connections 2024-03-28T13:20:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.632+0000 7f1d3886d740 1 --2- 172.21.15.19:0/4277729331 >> [v2:172.21.15.19:6832/2851646201,v1:172.21.15.19:6834/2851646201] conn(0x5622bfcf8390 0x5622bfc70020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.632+0000 7f1d3886d740 1 --2- 172.21.15.19:0/4277729331 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5622bfc70b80 0x5622bfd12560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.632+0000 7f1d3886d740 1 --2- 172.21.15.19:0/4277729331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5622bfc6eb00 0x5622bfc9f350 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.632+0000 7f1d3886d740 1 --2- 172.21.15.19:0/4277729331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5622bfc6e720 0x5622bfd0fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.632+0000 7f1d17e00640 0 client.8526 ms_handle_reset on v2:172.21.15.19:6832/2851646201 2024-03-28T13:20:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.632+0000 7f1d3886d740 1 -- 172.21.15.19:0/4277729331 >> 172.21.15.19:0/4277729331 conn(0x5622bfc41c00 msgr2=0x5622bfc72d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.632+0000 7f1d3886d740 1 -- 172.21.15.19:0/4277729331 shutdown_connections 2024-03-28T13:20:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.632+0000 7f1d3886d740 1 -- 172.21.15.19:0/4277729331 wait complete. 2024-03-28T13:20:32.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.632+0000 7f1d3886d740 20 client.8526 trim_cache size 0 max 16384 2024-03-28T13:20:32.636 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:20:32.636 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-03-28T13:20:32.636 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-03-28T13:20:32.636 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:20:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.633+0000 7f1d2fe00640 1 -- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d0c06c560 msgr2=0x7f1d0c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.633+0000 7f1d2fe00640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d0c06c560 0x7f1d0c06ea20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800a860 tx=0x7f1d18009030 comp rx=0 tx=0).stop 2024-03-28T13:20:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.633+0000 7f1d2fe00640 1 -- 172.21.15.19:0/3612577897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10004390 msgr2=0x7f1d301f6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.633+0000 7f1d2fe00640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10004390 0x7f1d301f6fb0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f1d200116d0 tx=0x7f1d20011b90 comp rx=0 tx=0).stop 2024-03-28T13:20:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.633+0000 7f1d2fe00640 1 -- 172.21.15.19:0/3612577897 shutdown_connections 2024-03-28T13:20:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.633+0000 7f1d2fe00640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1d0c06c560 0x7f1d0c06ea20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.633+0000 7f1d2fe00640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d301595f0 0x7f1d30159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.633+0000 7f1d2fe00640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d301f4440 0x7f1d301613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.633+0000 7f1d2fe00640 1 --2- 172.21.15.19:0/3612577897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1d10004390 0x7f1d301f6fb0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.633+0000 7f1d2fe00640 1 -- 172.21.15.19:0/3612577897 >> 172.21.15.19:0/3612577897 conn(0x7f1d300676a0 msgr2=0x7f1d300b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.633+0000 7f1d2fe00640 1 -- 172.21.15.19:0/3612577897 shutdown_connections 2024-03-28T13:20:32.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.633+0000 7f1d2fe00640 1 -- 172.21.15.19:0/3612577897 wait complete. 2024-03-28T13:20:32.650 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-28T13:20:32.751 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T13:20:32.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.967+0000 7f7f92e00640 1 Processor -- start 2024-03-28T13:20:32.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.967+0000 7f7f92e00640 1 -- start start 2024-03-28T13:20:32.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.968+0000 7f7f92e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c15eca0 0x7f7f8c0b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.968+0000 7f7f92e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f8c0b4c00 con 0x7f7f8c15a870 2024-03-28T13:20:32.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.968+0000 7f7f92e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f8c0b4ee0 con 0x7f7f8c15ac50 2024-03-28T13:20:32.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.968+0000 7f7f92e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7f8c06d5b0 con 0x7f7f8c15eca0 2024-03-28T13:20:32.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.968+0000 7f7f8be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7f8c15a870 0x7f7f8c15e590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58358/0 (socket says 172.21.15.19:58358) 2024-03-28T13:20:32.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.969+0000 7f7f8be00640 1 -- 172.21.15.19:0/2499919993 learned_addr learned my addr 172.21.15.19:0/2499919993 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.969+0000 7f7f8b400640 1 --2- 172.21.15.19:0/2499919993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c15eca0 0x7f7f8c0b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.969+0000 7f7f8aa00640 1 -- 172.21.15.19:0/2499919993 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1439575296 0 0) 0x7f7f8c0b4c00 con 0x7f7f8c15a870 2024-03-28T13:20:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.969+0000 7f7f8aa00640 1 -- 172.21.15.19:0/2499919993 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f70004630 con 0x7f7f8c15a870 2024-03-28T13:20:32.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.969+0000 7f7f8aa00640 1 -- 172.21.15.19:0/2499919993 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3038735781 0 0) 0x7f7f70004630 con 0x7f7f8c15a870 2024-03-28T13:20:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.969+0000 7f7f8aa00640 1 -- 172.21.15.19:0/2499919993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c15eca0 msgr2=0x7f7f8c0b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.969+0000 7f7f8aa00640 1 --2- 172.21.15.19:0/2499919993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c15eca0 0x7f7f8c0b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.969+0000 7f7f8aa00640 1 -- 172.21.15.19:0/2499919993 >> v1:172.21.15.188:6789/0 conn(0x7f7f8c15ac50 legacy=0x7f7f8c0b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.969+0000 7f7f8aa00640 1 -- 172.21.15.19:0/2499919993 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8c06d800 con 0x7f7f8c15a870 2024-03-28T13:20:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.969+0000 7f7f8aa00640 1 -- 172.21.15.19:0/2499919993 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f7f7c001670 con 0x7f7f8c15a870 2024-03-28T13:20:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.969+0000 7f7f8aa00640 1 -- 172.21.15.19:0/2499919993 >> v1:172.21.15.19:6789/0 conn(0x7f7f8c15a870 legacy=0x7f7f8c15e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.970+0000 7f7f8aa00640 1 --2- 172.21.15.19:0/2499919993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f70004af0 0x7f7f70004f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.970+0000 7f7f8aa00640 1 --2- 172.21.15.19:0/2499919993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f70005910 0x7f7f70005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.970+0000 7f7f8aa00640 1 --2- 172.21.15.19:0/2499919993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c15eca0 0x7f7f700062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.970+0000 7f7f8aa00640 1 -- 172.21.15.19:0/2499919993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c06d5b0 con 0x7f7f70005910 2024-03-28T13:20:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.970+0000 7f7f8aa00640 1 -- 172.21.15.19:0/2499919993 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7f70003510 con 0x7f7f70004af0 2024-03-28T13:20:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.970+0000 7f7f8aa00640 1 -- 172.21.15.19:0/2499919993 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7f70003710 con 0x7f7f8c15eca0 2024-03-28T13:20:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.970+0000 7f7f8b400640 1 --2- 172.21.15.19:0/2499919993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f70005910 0x7f7f70005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.970+0000 7f7f8be00640 1 --2- 172.21.15.19:0/2499919993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f70004af0 0x7f7f70004f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.970+0000 7f7f90e00640 1 --2- 172.21.15.19:0/2499919993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c15eca0 0x7f7f700062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.970+0000 7f7f8b400640 1 -- 172.21.15.19:0/2499919993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c15eca0 msgr2=0x7f7f700062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.970+0000 7f7f8b400640 1 --2- 172.21.15.19:0/2499919993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f8c15eca0 0x7f7f700062d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.970+0000 7f7f8b400640 1 -- 172.21.15.19:0/2499919993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f70004af0 msgr2=0x7f7f70004f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.970+0000 7f7f8b400640 1 --2- 172.21.15.19:0/2499919993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f70004af0 0x7f7f70004f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.970+0000 7f7f8b400640 1 -- 172.21.15.19:0/2499919993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f70004630 con 0x7f7f70005910 2024-03-28T13:20:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.972+0000 7f7f8b400640 1 --2- 172.21.15.19:0/2499919993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f70005910 0x7f7f70005d90 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009ac0 tx=0x7f7f70006940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9baed36df6bcd308 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.972+0000 7f7f8aa00640 1 -- 172.21.15.19:0/2499919993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78039070 con 0x7f7f70005910 2024-03-28T13:20:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.972+0000 7f7f8aa00640 1 -- 172.21.15.19:0/2499919993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f7802c6b0 con 0x7f7f70005910 2024-03-28T13:20:32.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.972+0000 7f7f8aa00640 1 -- 172.21.15.19:0/2499919993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78009690 con 0x7f7f70005910 2024-03-28T13:20:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.972+0000 7f7f92e00640 1 -- 172.21.15.19:0/2499919993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f70005910 msgr2=0x7f7f70005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.972+0000 7f7f92e00640 1 --2- 172.21.15.19:0/2499919993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f70005910 0x7f7f70005d90 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009ac0 tx=0x7f7f70006940 comp rx=0 tx=0).stop 2024-03-28T13:20:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.972+0000 7f7f90e00640 1 -- 172.21.15.19:0/2499919993 reap_dead start 2024-03-28T13:20:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.973+0000 7f7f92e00640 1 -- 172.21.15.19:0/2499919993 shutdown_connections 2024-03-28T13:20:32.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.973+0000 7f7f92e00640 1 -- 172.21.15.19:0/2499919993 >> 172.21.15.19:0/2499919993 conn(0x7f7f8c06b750 msgr2=0x7f7f8c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:32.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.973+0000 7f7f92e00640 1 -- 172.21.15.19:0/2499919993 shutdown_connections 2024-03-28T13:20:32.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.973+0000 7f7f92e00640 1 -- 172.21.15.19:0/2499919993 wait complete. 2024-03-28T13:20:32.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.974+0000 7f7f92e00640 1 Processor -- start 2024-03-28T13:20:32.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.975+0000 7f7f92e00640 1 -- start start 2024-03-28T13:20:32.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.975+0000 7f7f92e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f70005910 0x7f7f8c1534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.975+0000 7f7f92e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f8c099830 0x7f7f8c153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.975+0000 7f7f92e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f8c0b2830 0x7f7f8c0ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.975+0000 7f7f92e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c06db70 con 0x7f7f8c0b2830 2024-03-28T13:20:32.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.975+0000 7f7f92e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8c06f220 con 0x7f7f8c099830 2024-03-28T13:20:32.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.975+0000 7f7f92e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7f8c0b3f10 con 0x7f7f70005910 2024-03-28T13:20:32.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.976+0000 7f7f90e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f8c0b2830 0x7f7f8c0ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.976+0000 7f7f90e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f8c0b2830 0x7f7f8c0ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57228/0 (socket says 172.21.15.19:57228) 2024-03-28T13:20:32.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.976+0000 7f7f90e00640 1 -- 172.21.15.19:0/2564300990 learned_addr learned my addr 172.21.15.19:0/2564300990 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:32.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.976+0000 7f7f8b400640 1 --2- 172.21.15.19:0/2564300990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f8c099830 0x7f7f8c153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.976+0000 7f7f8be00640 1 --2- 172.21.15.19:0/2564300990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f70005910 0x7f7f8c1534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.976+0000 7f7f90e00640 1 -- 172.21.15.19:0/2564300990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f70005910 msgr2=0x7f7f8c1534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.976+0000 7f7f90e00640 1 --2- 172.21.15.19:0/2564300990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f70005910 0x7f7f8c1534e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.976+0000 7f7f90e00640 1 -- 172.21.15.19:0/2564300990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f8c099830 msgr2=0x7f7f8c153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:32.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.976+0000 7f7f90e00640 1 --2- 172.21.15.19:0/2564300990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f8c099830 0x7f7f8c153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:32.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.976+0000 7f7f90e00640 1 -- 172.21.15.19:0/2564300990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8c1f08e0 con 0x7f7f8c0b2830 2024-03-28T13:20:32.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.976+0000 7f7f8b400640 1 --2- 172.21.15.19:0/2564300990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f8c099830 0x7f7f8c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:32.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.977+0000 7f7f90e00640 1 --2- 172.21.15.19:0/2564300990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f8c0b2830 0x7f7f8c0ce490 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c0ad260 tx=0x7f7f8400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:32.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.977+0000 7f7f83e00640 1 -- 172.21.15.19:0/2564300990 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84018070 con 0x7f7f8c0b2830 2024-03-28T13:20:32.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.977+0000 7f7f83e00640 1 -- 172.21.15.19:0/2564300990 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f84003b70 con 0x7f7f8c0b2830 2024-03-28T13:20:32.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.977+0000 7f7f83e00640 1 -- 172.21.15.19:0/2564300990 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f84005070 con 0x7f7f8c0b2830 2024-03-28T13:20:32.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.977+0000 7f7f92e00640 1 -- 172.21.15.19:0/2564300990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f8c0cf740 con 0x7f7f8c0b2830 2024-03-28T13:20:32.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.977+0000 7f7f92e00640 1 -- 172.21.15.19:0/2564300990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f8c0ccce0 con 0x7f7f8c0b2830 2024-03-28T13:20:32.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.979+0000 7f7f92e00640 1 -- 172.21.15.19:0/2564300990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f8c0cc7f0 con 0x7f7f8c0b2830 2024-03-28T13:20:32.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.980+0000 7f7f83e00640 1 -- 172.21.15.19:0/2564300990 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7f8401b070 con 0x7f7f8c0b2830 2024-03-28T13:20:32.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.980+0000 7f7f83e00640 1 --2- 172.21.15.19:0/2564300990 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f6c06c530 0x7f7f6c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:32.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.981+0000 7f7f83e00640 1 -- 172.21.15.19:0/2564300990 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5853+0+0 (secure 0 0 0) 0x7f7f8408af10 con 0x7f7f8c0b2830 2024-03-28T13:20:32.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.981+0000 7f7f8be00640 1 --2- 172.21.15.19:0/2564300990 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f6c06c530 0x7f7f6c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:32.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.982+0000 7f7f8be00640 1 --2- 172.21.15.19:0/2564300990 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f6c06c530 0x7f7f6c06e9f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0078d0 tx=0x7f7f7c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:32.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:32.989+0000 7f7f83e00640 1 -- 172.21.15.19:0/2564300990 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7f840594c0 con 0x7f7f8c0b2830 2024-03-28T13:20:33.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.232+0000 7f7f92e00640 1 -- 172.21.15.19:0/2564300990 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7f8c0685c0 con 0x7f7f8c0b2830 2024-03-28T13:20:33.233 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:20:33.233+0000 7f6b74400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T13:20:33.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.236+0000 7f7f83e00640 1 -- 172.21.15.19:0/2564300990 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v89) ==== 161+0+0 (secure 0 0 0) 0x7f7f84058dc0 con 0x7f7f8c0b2830 2024-03-28T13:20:33.236 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T13:20:33.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.239+0000 7f7f92e00640 1 -- 172.21.15.19:0/2564300990 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f6c06c530 msgr2=0x7f7f6c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:33.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.239+0000 7f7f92e00640 1 --2- 172.21.15.19:0/2564300990 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f6c06c530 0x7f7f6c06e9f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c0078d0 tx=0x7f7f7c009040 comp rx=0 tx=0).stop 2024-03-28T13:20:33.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.239+0000 7f7f92e00640 1 -- 172.21.15.19:0/2564300990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f8c0b2830 msgr2=0x7f7f8c0ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:33.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.239+0000 7f7f92e00640 1 --2- 172.21.15.19:0/2564300990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f8c0b2830 0x7f7f8c0ce490 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c0ad260 tx=0x7f7f8400dc00 comp rx=0 tx=0).stop 2024-03-28T13:20:33.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.239+0000 7f7f92e00640 1 -- 172.21.15.19:0/2564300990 shutdown_connections 2024-03-28T13:20:33.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.239+0000 7f7f92e00640 1 --2- 172.21.15.19:0/2564300990 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7f6c06c530 0x7f7f6c06e9f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.239+0000 7f7f92e00640 1 --2- 172.21.15.19:0/2564300990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f8c0b2830 0x7f7f8c0ce490 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.239+0000 7f7f92e00640 1 --2- 172.21.15.19:0/2564300990 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f8c099830 0x7f7f8c153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.239+0000 7f7f92e00640 1 --2- 172.21.15.19:0/2564300990 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f70005910 0x7f7f8c1534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.239+0000 7f7f92e00640 1 -- 172.21.15.19:0/2564300990 >> 172.21.15.19:0/2564300990 conn(0x7f7f8c06b750 msgr2=0x7f7f8c0d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:33.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.240+0000 7f7f92e00640 1 -- 172.21.15.19:0/2564300990 shutdown_connections 2024-03-28T13:20:33.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.240+0000 7f7f92e00640 1 -- 172.21.15.19:0/2564300990 wait complete. 2024-03-28T13:20:33.250 INFO:tasks.ceph.mds.c.smithi019.stderr: -1318> 2024-03-28T13:20:01.744+0000 7fde1e400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:20:33.251 INFO:tasks.ceph.mds.c.smithi019.stderr: -1121> 2024-03-28T13:20:02.063+0000 7fde19400640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-03-28T13:20:33.251 INFO:tasks.ceph.mds.c.smithi019.stderr: -1118> 2024-03-28T13:20:02.063+0000 7fde19400640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-03-28T13:20:33.251 INFO:tasks.ceph.mds.c.smithi019.stderr: -1101> 2024-03-28T13:20:02.063+0000 7fde19400640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-03-28T13:20:33.251 INFO:tasks.ceph.mds.c.smithi019.stderr: -1098> 2024-03-28T13:20:02.063+0000 7fde19400640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-03-28T13:20:33.258 DEBUG:teuthology.orchestra.run.smithi019:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-03-28T13:20:33.307 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-03-28T13:20:33.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.338+0000 7f1064a93280 1 -- start start 2024-03-28T13:20:33.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.338+0000 7f1064a93280 1 Processor -- start 2024-03-28T13:20:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.339+0000 7f1064a93280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d795c9ec00 0x55d796a00000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.339+0000 7f1064a93280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d795c9f800 0x55d796a00580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.339+0000 7f1064a93280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d795c9fc00 0x55d796a00b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.339+0000 7f1064a93280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d796a76180 con 0x55d795c9f800 2024-03-28T13:20:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.339+0000 7f1064a93280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d796a76300 con 0x55d795c9ec00 2024-03-28T13:20:33.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.339+0000 7f1064a93280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55d796a76480 con 0x55d795c9fc00 2024-03-28T13:20:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.339+0000 7f1062200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d795c9f800 0x55d796a00580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.340+0000 7f1062200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d795c9f800 0x55d796a00580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57254/0 (socket says 172.21.15.19:57254) 2024-03-28T13:20:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.340+0000 7f1062200640 1 -- 172.21.15.19:0/1030633786 learned_addr learned my addr 172.21.15.19:0/1030633786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.340+0000 7f1062200640 1 -- 172.21.15.19:0/1030633786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d795c9fc00 msgr2=0x55d796a00b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.340+0000 7f1063600640 1 --2- 172.21.15.19:0/1030633786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d795c9fc00 0x55d796a00b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.340+0000 7f1062c00640 1 --2- 172.21.15.19:0/1030633786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d795c9ec00 0x55d796a00000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:33.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.340+0000 7f1062200640 1 --2- 172.21.15.19:0/1030633786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d795c9fc00 0x55d796a00b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.340+0000 7f1062200640 1 -- 172.21.15.19:0/1030633786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d795c9ec00 msgr2=0x55d796a00000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.340+0000 7f1062200640 1 --2- 172.21.15.19:0/1030633786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d795c9ec00 0x55d796a00000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.340+0000 7f1062200640 1 -- 172.21.15.19:0/1030633786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d795cd2780 con 0x55d795c9f800 2024-03-28T13:20:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.340+0000 7f1062c00640 1 --2- 172.21.15.19:0/1030633786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d795c9ec00 0x55d796a00000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.340+0000 7f1062200640 1 --2- 172.21.15.19:0/1030633786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d795c9f800 0x55d796a00580 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x55d796ac4d50 tx=0x55d796a30ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14b15bf2bd7c942 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.340+0000 7f105fa00640 1 -- 172.21.15.19:0/1030633786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d795cc49c0 con 0x55d795c9f800 2024-03-28T13:20:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.341+0000 7f105fa00640 1 -- 172.21.15.19:0/1030633786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d795cca700 con 0x55d795c9f800 2024-03-28T13:20:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.341+0000 7f105fa00640 1 -- 172.21.15.19:0/1030633786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d795cc4680 con 0x55d795c9f800 2024-03-28T13:20:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.341+0000 7f1064a93280 1 -- 172.21.15.19:0/1030633786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d795cd34a0 con 0x55d795c9f800 2024-03-28T13:20:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.341+0000 7f1064a93280 1 -- 172.21.15.19:0/1030633786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55d795cd2000 con 0x55d795c9f800 2024-03-28T13:20:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.341+0000 7f1064a93280 4 waiting for MDS map... 2024-03-28T13:20:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.341+0000 7f105fa00640 1 -- 172.21.15.19:0/1030633786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== osd_map(109..109 src has 1..109) ==== 5939+0+0 (secure 0 0 0) 0x55d796b5e200 con 0x55d795c9f800 2024-03-28T13:20:33.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.341+0000 7f105fa00640 1 -- 172.21.15.19:0/1030633786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 89) ==== 2885+0+0 (secure 0 0 0) 0x55d795cf9180 con 0x55d795c9f800 2024-03-28T13:20:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.341+0000 7f1064a93280 4 Got MDS map 89 2024-03-28T13:20:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.341+0000 7f1064a93280 10 main: main 2024-03-28T13:20:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.341+0000 7f1064a93280 4 main: connecting to RADOS... 2024-03-28T13:20:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.342+0000 7f1064a93280 1 Processor -- start 2024-03-28T13:20:33.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.342+0000 7f1064a93280 1 -- start start 2024-03-28T13:20:33.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.343+0000 7f1064a93280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d796bb6000 0x55d796a01080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:33.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.343+0000 7f1064a93280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d796bb6400 0x55d796a01600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:33.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.343+0000 7f1064a93280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d796bb6800 0x55d796a01b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:33.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.343+0000 7f1064a93280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d796a76600 con 0x55d796bb6400 2024-03-28T13:20:33.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.343+0000 7f1064a93280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d796a76780 con 0x55d796bb6800 2024-03-28T13:20:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.343+0000 7f1064a93280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55d796a76900 con 0x55d796bb6000 2024-03-28T13:20:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.343+0000 7f1062c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d796bb6400 0x55d796a01600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.343+0000 7f1062c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d796bb6400 0x55d796a01600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57270/0 (socket says 172.21.15.19:57270) 2024-03-28T13:20:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.343+0000 7f1062c00640 1 -- 172.21.15.19:0/1268605592 learned_addr learned my addr 172.21.15.19:0/1268605592 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.343+0000 7f1063600640 1 --2- 172.21.15.19:0/1268605592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d796bb6000 0x55d796a01080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.344+0000 7f1062200640 1 --2- 172.21.15.19:0/1268605592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d796bb6800 0x55d796a01b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.344+0000 7f1062c00640 1 -- 172.21.15.19:0/1268605592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d796bb6000 msgr2=0x55d796a01080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.344+0000 7f1062c00640 1 --2- 172.21.15.19:0/1268605592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d796bb6000 0x55d796a01080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.344+0000 7f1062c00640 1 -- 172.21.15.19:0/1268605592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d796bb6800 msgr2=0x55d796a01b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.344+0000 7f1062c00640 1 --2- 172.21.15.19:0/1268605592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d796bb6800 0x55d796a01b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.344+0000 7f1062c00640 1 -- 172.21.15.19:0/1268605592 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d795cd23c0 con 0x55d796bb6400 2024-03-28T13:20:33.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.344+0000 7f1062c00640 1 --2- 172.21.15.19:0/1268605592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d796bb6400 0x55d796a01600 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x55d796bc40c0 tx=0x55d796a94d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d2daf38904ca server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.344+0000 7f105d200640 1 -- 172.21.15.19:0/1268605592 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d795cc49c0 con 0x55d796bb6400 2024-03-28T13:20:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.344+0000 7f105d200640 1 -- 172.21.15.19:0/1268605592 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d795ccafc0 con 0x55d796bb6400 2024-03-28T13:20:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.344+0000 7f105d200640 1 -- 172.21.15.19:0/1268605592 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d795cc4680 con 0x55d796bb6400 2024-03-28T13:20:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.345+0000 7f1064a93280 1 -- 172.21.15.19:0/1268605592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d796bb6400 msgr2=0x55d796a01600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.345+0000 7f1064a93280 1 --2- 172.21.15.19:0/1268605592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d796bb6400 0x55d796a01600 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x55d796bc40c0 tx=0x55d796a94d80 comp rx=0 tx=0).stop 2024-03-28T13:20:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.345+0000 7f1064a93280 1 -- 172.21.15.19:0/1268605592 shutdown_connections 2024-03-28T13:20:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.345+0000 7f1064a93280 1 --2- 172.21.15.19:0/1268605592 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d796bb6800 0x55d796a01b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.345+0000 7f1064a93280 1 --2- 172.21.15.19:0/1268605592 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d796bb6400 0x55d796a01600 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.345+0000 7f1064a93280 1 --2- 172.21.15.19:0/1268605592 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d796bb6000 0x55d796a01080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.345+0000 7f1064a93280 1 -- 172.21.15.19:0/1268605592 >> 172.21.15.19:0/1268605592 conn(0x55d796a7bc00 msgr2=0x55d795d87000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.345+0000 7f1064a93280 1 -- 172.21.15.19:0/1268605592 shutdown_connections 2024-03-28T13:20:33.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.345+0000 7f1064a93280 1 -- 172.21.15.19:0/1268605592 wait complete. 2024-03-28T13:20:33.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.345+0000 7f1064a93280 1 Processor -- start 2024-03-28T13:20:33.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.346+0000 7f1064a93280 1 -- start start 2024-03-28T13:20:33.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.346+0000 7f1064a93280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d796bb6000 0x55d796a01080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:33.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.346+0000 7f1064a93280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d796bb6400 0x55d796a01600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:33.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.346+0000 7f1064a93280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d796bb6800 0x55d796a01b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:33.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.346+0000 7f1064a93280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d796a77c80 con 0x55d796bb6800 2024-03-28T13:20:33.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.346+0000 7f1064a93280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55d796a77b00 con 0x55d796bb6400 2024-03-28T13:20:33.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.346+0000 7f1064a93280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55d796a77980 con 0x55d796bb6000 2024-03-28T13:20:33.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.347+0000 7f1062200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d796bb6800 0x55d796a01b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:33.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.347+0000 7f1062200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d796bb6800 0x55d796a01b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57282/0 (socket says 172.21.15.19:57282) 2024-03-28T13:20:33.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.347+0000 7f1062200640 1 -- 172.21.15.19:0/1176711385 learned_addr learned my addr 172.21.15.19:0/1176711385 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:33.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.347+0000 7f1062200640 1 -- 172.21.15.19:0/1176711385 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d796bb6000 msgr2=0x55d796a01080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.347+0000 7f1062200640 1 --2- 172.21.15.19:0/1176711385 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d796bb6000 0x55d796a01080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.347+0000 7f1062200640 1 -- 172.21.15.19:0/1176711385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d796bb6400 msgr2=0x55d796a01600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.347+0000 7f1062200640 1 --2- 172.21.15.19:0/1176711385 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d796bb6400 0x55d796a01600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.347+0000 7f1062200640 1 -- 172.21.15.19:0/1176711385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d795cd2960 con 0x55d796bb6800 2024-03-28T13:20:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.347+0000 7f1062200640 1 --2- 172.21.15.19:0/1176711385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d796bb6800 0x55d796a01b80 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x55d796ac5170 tx=0x55d796a308a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.347+0000 7f105b400640 1 -- 172.21.15.19:0/1176711385 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d795cc4b60 con 0x55d796bb6800 2024-03-28T13:20:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.347+0000 7f105b400640 1 -- 172.21.15.19:0/1176711385 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55d795ccae00 con 0x55d796bb6800 2024-03-28T13:20:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.347+0000 7f105b400640 1 -- 172.21.15.19:0/1176711385 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d795cc4820 con 0x55d796bb6800 2024-03-28T13:20:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.347+0000 7f1064a93280 1 -- 172.21.15.19:0/1176711385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d795cd3680 con 0x55d796bb6800 2024-03-28T13:20:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.348+0000 7f1064a93280 1 -- 172.21.15.19:0/1176711385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d795cd3860 con 0x55d796bb6800 2024-03-28T13:20:33.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.348+0000 7f1064a93280 4 main: resolving pool 14 2024-03-28T13:20:33.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.349+0000 7f105b400640 1 -- 172.21.15.19:0/1176711385 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x55d796bcac00 con 0x55d796bb6800 2024-03-28T13:20:33.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.350+0000 7f105b400640 1 --2- 172.21.15.19:0/1176711385 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x55d796bd3000 0x55d796a02100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:33.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.350+0000 7f105b400640 1 -- 172.21.15.19:0/1176711385 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5939+0+0 (secure 0 0 0) 0x55d796bbd200 con 0x55d796bb6800 2024-03-28T13:20:33.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.350+0000 7f1064a93280 4 main: creating IoCtx.. 2024-03-28T13:20:33.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.350+0000 7f1064a93280 1 --2- 172.21.15.19:0/1176711385 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x55d796bb6c00 0x55d796a02680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:33.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.350+0000 7f1064a93280 1 -- 172.21.15.19:0/1176711385 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) -- 0x55d796bb7000 con 0x55d796bb6c00 2024-03-28T13:20:33.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.351+0000 7f1063600640 1 --2- 172.21.15.19:0/1176711385 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x55d796bd3000 0x55d796a02100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:33.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.351+0000 7f1062c00640 1 --2- 172.21.15.19:0/1176711385 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x55d796bb6c00 0x55d796a02680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:33.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.352+0000 7f1063600640 1 --2- 172.21.15.19:0/1176711385 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x55d796bd3000 0x55d796a02100 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x55d796ac4c60 tx=0x55d796a95020 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:33.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.353+0000 7f1062c00640 1 --2- 172.21.15.19:0/1176711385 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x55d796bb6c00 0x55d796a02680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:33.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.354+0000 7f1062c00640 1 -- 172.21.15.19:0/1176711385 <== osd.5 v2:172.21.15.188:6800/3324247755 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x55d796a20000 con 0x55d796bb6c00 2024-03-28T13:20:33.355 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:20:33.355 INFO:teuthology.orchestra.run.smithi019.stdout: "0": { 2024-03-28T13:20:33.355 INFO:teuthology.orchestra.run.smithi019.stdout: "data": { 2024-03-28T13:20:33.355 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 5, 2024-03-28T13:20:33.355 INFO:teuthology.orchestra.run.smithi019.stdout: "inotable": { 2024-03-28T13:20:33.356 INFO:teuthology.orchestra.run.smithi019.stdout: "projected_free": [ 2024-03-28T13:20:33.356 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:33.356 INFO:teuthology.orchestra.run.smithi019.stdout: "start": 1099511627779, 2024-03-28T13:20:33.356 INFO:teuthology.orchestra.run.smithi019.stdout: "len": 1099511627773 2024-03-28T13:20:33.356 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:33.356 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-03-28T13:20:33.356 INFO:teuthology.orchestra.run.smithi019.stdout: "free": [ 2024-03-28T13:20:33.356 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:33.356 INFO:teuthology.orchestra.run.smithi019.stdout: "start": 1099511627779, 2024-03-28T13:20:33.356 INFO:teuthology.orchestra.run.smithi019.stdout: "len": 1099511627773 2024-03-28T13:20:33.356 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:33.356 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-03-28T13:20:33.356 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:33.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:33.357 INFO:teuthology.orchestra.run.smithi019.stdout: "result": 0 2024-03-28T13:20:33.357 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:33.357 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:20:33.357 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:33.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.354+0000 7f1064a93280 1 -- 172.21.15.19:0/1176711385 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x55d796bb6c00 msgr2=0x55d796a02680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:33.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.354+0000 7f1064a93280 1 --2- 172.21.15.19:0/1176711385 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x55d796bb6c00 0x55d796a02680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.354+0000 7f1064a93280 1 -- 172.21.15.19:0/1176711385 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x55d796bd3000 msgr2=0x55d796a02100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.354+0000 7f1064a93280 1 --2- 172.21.15.19:0/1176711385 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x55d796bd3000 0x55d796a02100 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x55d796ac4c60 tx=0x55d796a95020 comp rx=0 tx=0).stop 2024-03-28T13:20:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.354+0000 7f1064a93280 1 -- 172.21.15.19:0/1176711385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d796bb6800 msgr2=0x55d796a01b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.354+0000 7f1064a93280 1 --2- 172.21.15.19:0/1176711385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d796bb6800 0x55d796a01b80 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x55d796ac5170 tx=0x55d796a308a0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.354+0000 7f1062c00640 1 -- 172.21.15.19:0/1176711385 reap_dead start 2024-03-28T13:20:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.354+0000 7f1064a93280 1 -- 172.21.15.19:0/1176711385 shutdown_connections 2024-03-28T13:20:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.354+0000 7f1064a93280 1 -- 172.21.15.19:0/1176711385 >> 172.21.15.19:0/1176711385 conn(0x55d796a7bc00 msgr2=0x55d795d87000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.355+0000 7f1064a93280 1 -- 172.21.15.19:0/1176711385 shutdown_connections 2024-03-28T13:20:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.355+0000 7f1064a93280 1 -- 172.21.15.19:0/1176711385 wait complete. 2024-03-28T13:20:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.355+0000 7f1064a93280 1 -- 172.21.15.19:0/1030633786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d795c9f800 msgr2=0x55d796a00580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.355+0000 7f1064a93280 1 --2- 172.21.15.19:0/1030633786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d795c9f800 0x55d796a00580 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x55d796ac4d50 tx=0x55d796a30ae0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.355+0000 7f1064a93280 1 -- 172.21.15.19:0/1030633786 shutdown_connections 2024-03-28T13:20:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.355+0000 7f1064a93280 1 --2- 172.21.15.19:0/1030633786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d795c9fc00 0x55d796a00b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.355+0000 7f1064a93280 1 --2- 172.21.15.19:0/1030633786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d795c9f800 0x55d796a00580 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.355+0000 7f1064a93280 1 --2- 172.21.15.19:0/1030633786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d795c9ec00 0x55d796a00000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.355+0000 7f1064a93280 1 -- 172.21.15.19:0/1030633786 >> 172.21.15.19:0/1030633786 conn(0x55d795c9f000 msgr2=0x55d795d86800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.355+0000 7f1064a93280 1 -- 172.21.15.19:0/1030633786 shutdown_connections 2024-03-28T13:20:33.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:33.355+0000 7f1064a93280 1 -- 172.21.15.19:0/1030633786 wait complete. 2024-03-28T13:20:33.361 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.102753, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-03-28T13:20:33.361 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-28T13:20:33.362 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:20:33.463 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:20:33.584 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-28T13:20:33.584 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:20:33.584 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:20:33.584 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:20:33.641 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:20:33.641 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:20:33.642 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:20:33.642 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:20:33.699 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:20:33.699 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:20:33.699 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:20:33.699 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:20:33.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:20:33.760 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:20:33.760 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:20:33.760 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:20:33.816 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:20:33.816 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:20:34.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.046+0000 7f9ccd400640 1 Processor -- start 2024-03-28T13:20:34.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.046+0000 7f9ccd400640 1 -- start start 2024-03-28T13:20:34.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.047+0000 7f9ccd400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9cc80d0d60 0x7f9cc8073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.047+0000 7f9ccd400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cc815fe80 con 0x7f9cc815c390 2024-03-28T13:20:34.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.047+0000 7f9ccd400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cc80ad120 con 0x7f9cc80af830 2024-03-28T13:20:34.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.047+0000 7f9ccd400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9cc815c770 con 0x7f9cc80d0d60 2024-03-28T13:20:34.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.048+0000 7f9cc6800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9cc815c390 0x7f9cc8160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58370/0 (socket says 172.21.15.19:58370) 2024-03-28T13:20:34.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.048+0000 7f9cc6800640 1 -- 172.21.15.19:0/194354116 learned_addr learned my addr 172.21.15.19:0/194354116 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:34.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.048+0000 7f9cc5400640 1 -- 172.21.15.19:0/194354116 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1175007122 0 0) 0x7f9cc815fe80 con 0x7f9cc815c390 2024-03-28T13:20:34.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.048+0000 7f9cc5400640 1 -- 172.21.15.19:0/194354116 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9cac003520 con 0x7f9cc815c390 2024-03-28T13:20:34.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.048+0000 7f9cc7200640 1 --2- 172.21.15.19:0/194354116 >> v2:172.21.15.188:3300/0 conn(0x7f9cc80d0d60 0x7f9cc8073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:34.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.048+0000 7f9cc5400640 1 -- 172.21.15.19:0/194354116 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3272638108 0 0) 0x7f9cac003520 con 0x7f9cc815c390 2024-03-28T13:20:34.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.048+0000 7f9cc5400640 1 -- 172.21.15.19:0/194354116 >> v2:172.21.15.188:3300/0 conn(0x7f9cc80d0d60 msgr2=0x7f9cc8073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:34.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.048+0000 7f9cc5400640 1 --2- 172.21.15.19:0/194354116 >> v2:172.21.15.188:3300/0 conn(0x7f9cc80d0d60 0x7f9cc8073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.048+0000 7f9cc5400640 1 -- 172.21.15.19:0/194354116 >> v1:172.21.15.188:6789/0 conn(0x7f9cc80af830 legacy=0x7f9cc80d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:34.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc5400640 1 -- 172.21.15.19:0/194354116 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc80ad400 con 0x7f9cc815c390 2024-03-28T13:20:34.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc5400640 1 -- 172.21.15.19:0/194354116 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f9cb8001690 con 0x7f9cc815c390 2024-03-28T13:20:34.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc5400640 1 -- 172.21.15.19:0/194354116 >> v1:172.21.15.19:6789/0 conn(0x7f9cc815c390 legacy=0x7f9cc8160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:34.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc5400640 1 --2- 172.21.15.19:0/194354116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cac003a00 0x7f9cac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc5400640 1 --2- 172.21.15.19:0/194354116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cac004800 0x7f9cac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc5400640 1 --2- 172.21.15.19:0/194354116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cac005210 0x7f9cac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc5400640 1 -- 172.21.15.19:0/194354116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac00c130 con 0x7f9cac003a00 2024-03-28T13:20:34.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc5400640 1 -- 172.21.15.19:0/194354116 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac00c2e0 con 0x7f9cac005210 2024-03-28T13:20:34.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc5400640 1 -- 172.21.15.19:0/194354116 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9cac00c4e0 con 0x7f9cac004800 2024-03-28T13:20:34.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc6800640 1 --2- 172.21.15.19:0/194354116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cac003a00 0x7f9cac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:34.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc5e00640 1 --2- 172.21.15.19:0/194354116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cac004800 0x7f9cac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:34.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc7200640 1 --2- 172.21.15.19:0/194354116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cac005210 0x7f9cac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:34.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc6800640 1 -- 172.21.15.19:0/194354116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cac004800 msgr2=0x7f9cac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:34.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc6800640 1 --2- 172.21.15.19:0/194354116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cac004800 0x7f9cac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc6800640 1 -- 172.21.15.19:0/194354116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cac005210 msgr2=0x7f9cac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:34.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc6800640 1 --2- 172.21.15.19:0/194354116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cac005210 0x7f9cac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc6800640 1 -- 172.21.15.19:0/194354116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac003520 con 0x7f9cac003a00 2024-03-28T13:20:34.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.049+0000 7f9cc7200640 1 -- 172.21.15.19:0/194354116 reap_dead start 2024-03-28T13:20:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.051+0000 7f9cc6800640 1 --2- 172.21.15.19:0/194354116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cac003a00 0x7f9cac003e20 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800c0e0 tx=0x7f9cac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69a060959a007dd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.051+0000 7f9cc5400640 1 -- 172.21.15.19:0/194354116 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8007c40 con 0x7f9cac003a00 2024-03-28T13:20:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.051+0000 7f9cc5400640 1 -- 172.21.15.19:0/194354116 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb8030070 con 0x7f9cac003a00 2024-03-28T13:20:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.051+0000 7f9cc5400640 1 -- 172.21.15.19:0/194354116 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb8005d50 con 0x7f9cac003a00 2024-03-28T13:20:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.051+0000 7f9ccd400640 1 -- 172.21.15.19:0/194354116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cac003a00 msgr2=0x7f9cac003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.051+0000 7f9ccd400640 1 --2- 172.21.15.19:0/194354116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cac003a00 0x7f9cac003e20 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f9cb800c0e0 tx=0x7f9cac005830 comp rx=0 tx=0).stop 2024-03-28T13:20:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.052+0000 7f9ccd400640 1 -- 172.21.15.19:0/194354116 shutdown_connections 2024-03-28T13:20:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.052+0000 7f9ccd400640 1 --2- 172.21.15.19:0/194354116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cac003a00 0x7f9cac003e20 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.052+0000 7f9ccd400640 1 -- 172.21.15.19:0/194354116 >> 172.21.15.19:0/194354116 conn(0x7f9cc806b990 msgr2=0x7f9cc806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:34.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.052+0000 7f9ccd400640 1 -- 172.21.15.19:0/194354116 shutdown_connections 2024-03-28T13:20:34.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.052+0000 7f9ccd400640 1 -- 172.21.15.19:0/194354116 wait complete. 2024-03-28T13:20:34.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.053+0000 7f9ccd400640 1 Processor -- start 2024-03-28T13:20:34.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.054+0000 7f9ccd400640 1 -- start start 2024-03-28T13:20:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.055+0000 7f9ccd400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cac003a00 0x7f9cc81587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.055+0000 7f9ccd400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cc815e880 0x7f9cc8152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.055+0000 7f9ccd400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cc80b3c00 0x7f9cc80b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.055+0000 7f9ccd400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc80cf980 con 0x7f9cc815e880 2024-03-28T13:20:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.055+0000 7f9ccd400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc8160120 con 0x7f9cc80b3c00 2024-03-28T13:20:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.055+0000 7f9ccd400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc8073790 con 0x7f9cac003a00 2024-03-28T13:20:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.055+0000 7f9cc5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cc815e880 0x7f9cc8152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.055+0000 7f9cc5e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cc815e880 0x7f9cc8152070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57306/0 (socket says 172.21.15.19:57306) 2024-03-28T13:20:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.055+0000 7f9cc5e00640 1 -- 172.21.15.19:0/1898499333 learned_addr learned my addr 172.21.15.19:0/1898499333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.055+0000 7f9cc7200640 1 --2- 172.21.15.19:0/1898499333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cc80b3c00 0x7f9cc80b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.055+0000 7f9cc6800640 1 --2- 172.21.15.19:0/1898499333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cac003a00 0x7f9cc81587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.055+0000 7f9cc5e00640 1 -- 172.21.15.19:0/1898499333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cac003a00 msgr2=0x7f9cc81587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.055+0000 7f9cc5e00640 1 --2- 172.21.15.19:0/1898499333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cac003a00 0x7f9cc81587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.056+0000 7f9cc5e00640 1 -- 172.21.15.19:0/1898499333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cc80b3c00 msgr2=0x7f9cc80b40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.056+0000 7f9cc5e00640 1 --2- 172.21.15.19:0/1898499333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cc80b3c00 0x7f9cc80b40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.056+0000 7f9cc5e00640 1 -- 172.21.15.19:0/1898499333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc8158d20 con 0x7f9cc815e880 2024-03-28T13:20:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.056+0000 7f9cc6800640 1 --2- 172.21.15.19:0/1898499333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cac003a00 0x7f9cc81587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.056+0000 7f9cc5e00640 1 --2- 172.21.15.19:0/1898499333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cc815e880 0x7f9cc8152070 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000e4d0 tx=0x7f9cb000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:34.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.056+0000 7f9cbea00640 1 -- 172.21.15.19:0/1898499333 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb00053a0 con 0x7f9cc815e880 2024-03-28T13:20:34.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.056+0000 7f9cbea00640 1 -- 172.21.15.19:0/1898499333 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cb0018070 con 0x7f9cc815e880 2024-03-28T13:20:34.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.056+0000 7f9cbea00640 1 -- 172.21.15.19:0/1898499333 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0003810 con 0x7f9cc815e880 2024-03-28T13:20:34.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.056+0000 7f9ccd400640 1 -- 172.21.15.19:0/1898499333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc81525b0 con 0x7f9cc815e880 2024-03-28T13:20:34.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.056+0000 7f9ccd400640 1 -- 172.21.15.19:0/1898499333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc81592a0 con 0x7f9cc815e880 2024-03-28T13:20:34.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.057+0000 7f9ccd400640 1 -- 172.21.15.19:0/1898499333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc8073100 con 0x7f9cc815e880 2024-03-28T13:20:34.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.058+0000 7f9cbea00640 1 -- 172.21.15.19:0/1898499333 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9cb00039b0 con 0x7f9cc815e880 2024-03-28T13:20:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.059+0000 7f9cbea00640 1 --2- 172.21.15.19:0/1898499333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ca406c560 0x7f9ca406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.059+0000 7f9cbea00640 1 -- 172.21.15.19:0/1898499333 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5939+0+0 (secure 0 0 0) 0x7f9cb008b470 con 0x7f9cc815e880 2024-03-28T13:20:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.060+0000 7f9cc6800640 1 --2- 172.21.15.19:0/1898499333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ca406c560 0x7f9ca406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:34.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.061+0000 7f9cc6800640 1 --2- 172.21.15.19:0/1898499333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ca406c560 0x7f9ca406ea20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8034000 tx=0x7f9cb8039040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:34.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.064+0000 7f9cbea00640 1 -- 172.21.15.19:0/1898499333 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9cb0059900 con 0x7f9cc815e880 2024-03-28T13:20:34.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.313+0000 7f9ccd400640 1 -- 172.21.15.19:0/1898499333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9cc80685c0 con 0x7f9cc815e880 2024-03-28T13:20:34.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.314+0000 7f9cbea00640 1 -- 172.21.15.19:0/1898499333 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5176 (secure 0 0 0) 0x7f9cb0059200 con 0x7f9cc815e880 2024-03-28T13:20:34.314 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:34.314 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4247326257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4247326257},{"type":"v1","addr":"172.21.15.188:6836","nonce":4247326257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3452274156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3452274156},{"type":"v1","addr":"172.21.15.19:6835","nonce":3452274156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3383190607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3383190607},{"type":"v1","addr":"172.21.15.188:6837","nonce":3383190607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3193953635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3193953635},{"type":"v1","addr":"172.21.15.19:6834","nonce":3193953635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:18:58.541599+0000","modified":"2024-03-28T13:20:33.234961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-03-28T13:20:34.314 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 90 2024-03-28T13:20:34.314 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:20:34.320 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:20:34.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.317+0000 7f9ccd400640 1 -- 172.21.15.19:0/1898499333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ca406c560 msgr2=0x7f9ca406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:34.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.317+0000 7f9ccd400640 1 --2- 172.21.15.19:0/1898499333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ca406c560 0x7f9ca406ea20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8034000 tx=0x7f9cb8039040 comp rx=0 tx=0).stop 2024-03-28T13:20:34.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.317+0000 7f9ccd400640 1 -- 172.21.15.19:0/1898499333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cc815e880 msgr2=0x7f9cc8152070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:34.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.317+0000 7f9ccd400640 1 --2- 172.21.15.19:0/1898499333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cc815e880 0x7f9cc8152070 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000e4d0 tx=0x7f9cb000e990 comp rx=0 tx=0).stop 2024-03-28T13:20:34.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.317+0000 7f9ccd400640 1 -- 172.21.15.19:0/1898499333 shutdown_connections 2024-03-28T13:20:34.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.317+0000 7f9ccd400640 1 --2- 172.21.15.19:0/1898499333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9ca406c560 0x7f9ca406ea20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.317+0000 7f9ccd400640 1 --2- 172.21.15.19:0/1898499333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9cc80b3c00 0x7f9cc80b40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.317+0000 7f9ccd400640 1 --2- 172.21.15.19:0/1898499333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9cc815e880 0x7f9cc8152070 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.317+0000 7f9ccd400640 1 --2- 172.21.15.19:0/1898499333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9cac003a00 0x7f9cc81587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.317+0000 7f9ccd400640 1 -- 172.21.15.19:0/1898499333 >> 172.21.15.19:0/1898499333 conn(0x7f9cc806b990 msgr2=0x7f9cc80b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:34.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.317+0000 7f9ccd400640 1 -- 172.21.15.19:0/1898499333 shutdown_connections 2024-03-28T13:20:34.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.318+0000 7f9ccd400640 1 -- 172.21.15.19:0/1898499333 wait complete. 2024-03-28T13:20:34.337 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:20:34.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.588+0000 7f4b8ea00640 1 Processor -- start 2024-03-28T13:20:34.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.588+0000 7f4b8ea00640 1 -- start start 2024-03-28T13:20:34.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.590+0000 7f4b8ea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b8815fc50 0x7f4b880cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.590+0000 7f4b8ea00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b880b3150 con 0x7f4b880cc0b0 2024-03-28T13:20:34.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.590+0000 7f4b8ea00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b881600e0 con 0x7f4b880cc490 2024-03-28T13:20:34.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.590+0000 7f4b8ea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8815c610 con 0x7f4b8815fc50 2024-03-28T13:20:34.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.590+0000 7f4b8d600640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4b880cc0b0 0x7f4b8806dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58372/0 (socket says 172.21.15.19:58372) 2024-03-28T13:20:34.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.590+0000 7f4b8d600640 1 -- 172.21.15.19:0/1327588832 learned_addr learned my addr 172.21.15.19:0/1327588832 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:34.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.591+0000 7f4b8cc00640 1 --2- 172.21.15.19:0/1327588832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b8815fc50 0x7f4b880cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:34.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.591+0000 7f4b87e00640 1 -- 172.21.15.19:0/1327588832 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1725659995 0 0) 0x7f4b880b3150 con 0x7f4b880cc0b0 2024-03-28T13:20:34.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.591+0000 7f4b87e00640 1 -- 172.21.15.19:0/1327588832 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b70003520 con 0x7f4b880cc0b0 2024-03-28T13:20:34.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.591+0000 7f4b87e00640 1 -- 172.21.15.19:0/1327588832 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2218014961 0 0) 0x7f4b70003520 con 0x7f4b880cc0b0 2024-03-28T13:20:34.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.591+0000 7f4b87e00640 1 -- 172.21.15.19:0/1327588832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b8815fc50 msgr2=0x7f4b880cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:34.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.591+0000 7f4b87e00640 1 --2- 172.21.15.19:0/1327588832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b8815fc50 0x7f4b880cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.591+0000 7f4b87e00640 1 -- 172.21.15.19:0/1327588832 >> v1:172.21.15.188:6789/0 conn(0x7f4b880cc490 legacy=0x7f4b8815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:34.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.591+0000 7f4b87e00640 1 -- 172.21.15.19:0/1327588832 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b881603c0 con 0x7f4b880cc0b0 2024-03-28T13:20:34.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.591+0000 7f4b87e00640 1 -- 172.21.15.19:0/1327588832 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f4b7c001670 con 0x7f4b880cc0b0 2024-03-28T13:20:34.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.591+0000 7f4b87e00640 1 -- 172.21.15.19:0/1327588832 >> v1:172.21.15.19:6789/0 conn(0x7f4b880cc0b0 legacy=0x7f4b8806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:34.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.591+0000 7f4b87e00640 1 --2- 172.21.15.19:0/1327588832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b700039e0 0x7f4b70003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.591+0000 7f4b87e00640 1 --2- 172.21.15.19:0/1327588832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b70004800 0x7f4b70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.591+0000 7f4b87e00640 1 --2- 172.21.15.19:0/1327588832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b70005180 0x7f4b7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.592+0000 7f4b87e00640 1 -- 172.21.15.19:0/1327588832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8815c610 con 0x7f4b70005180 2024-03-28T13:20:34.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.592+0000 7f4b87e00640 1 -- 172.21.15.19:0/1327588832 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7000bf00 con 0x7f4b700039e0 2024-03-28T13:20:34.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.592+0000 7f4b87e00640 1 -- 172.21.15.19:0/1327588832 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4b7000c0d0 con 0x7f4b70004800 2024-03-28T13:20:34.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.592+0000 7f4b8e000640 1 --2- 172.21.15.19:0/1327588832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b70005180 0x7f4b7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:34.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.592+0000 7f4b8d600640 1 --2- 172.21.15.19:0/1327588832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b700039e0 0x7f4b70003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:34.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.592+0000 7f4b8cc00640 1 --2- 172.21.15.19:0/1327588832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b70004800 0x7f4b70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:34.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.592+0000 7f4b8e000640 1 -- 172.21.15.19:0/1327588832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b70004800 msgr2=0x7f4b70004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:34.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.592+0000 7f4b8e000640 1 --2- 172.21.15.19:0/1327588832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b70004800 0x7f4b70004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.592+0000 7f4b8e000640 1 -- 172.21.15.19:0/1327588832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b700039e0 msgr2=0x7f4b70003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:34.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.592+0000 7f4b8e000640 1 --2- 172.21.15.19:0/1327588832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b700039e0 0x7f4b70003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.592+0000 7f4b8e000640 1 -- 172.21.15.19:0/1327588832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b70003520 con 0x7f4b70005180 2024-03-28T13:20:34.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.593+0000 7f4b8e000640 1 --2- 172.21.15.19:0/1327588832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b70005180 0x7f4b7000b9c0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f4b80001e50 tx=0x7f4b70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9a2aacc8695999 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:34.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.594+0000 7f4b87e00640 1 -- 172.21.15.19:0/1327588832 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b80001a70 con 0x7f4b70005180 2024-03-28T13:20:34.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.594+0000 7f4b87e00640 1 -- 172.21.15.19:0/1327588832 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b80003b70 con 0x7f4b70005180 2024-03-28T13:20:34.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.594+0000 7f4b87e00640 1 -- 172.21.15.19:0/1327588832 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b800050d0 con 0x7f4b70005180 2024-03-28T13:20:34.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.594+0000 7f4b8ea00640 1 -- 172.21.15.19:0/1327588832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b70005180 msgr2=0x7f4b7000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:34.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.594+0000 7f4b8ea00640 1 --2- 172.21.15.19:0/1327588832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b70005180 0x7f4b7000b9c0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f4b80001e50 tx=0x7f4b70005830 comp rx=0 tx=0).stop 2024-03-28T13:20:34.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.594+0000 7f4b8e000640 1 -- 172.21.15.19:0/1327588832 reap_dead start 2024-03-28T13:20:34.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.594+0000 7f4b8ea00640 1 -- 172.21.15.19:0/1327588832 shutdown_connections 2024-03-28T13:20:34.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.594+0000 7f4b8ea00640 1 -- 172.21.15.19:0/1327588832 >> 172.21.15.19:0/1327588832 conn(0x7f4b880ccad0 msgr2=0x7f4b880cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:34.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.594+0000 7f4b8ea00640 1 -- 172.21.15.19:0/1327588832 shutdown_connections 2024-03-28T13:20:34.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.594+0000 7f4b8ea00640 1 -- 172.21.15.19:0/1327588832 wait complete. 2024-03-28T13:20:34.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.596+0000 7f4b8ea00640 1 Processor -- start 2024-03-28T13:20:34.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.596+0000 7f4b8ea00640 1 -- start start 2024-03-28T13:20:34.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.597+0000 7f4b8ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b70005180 0x7f4b880b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.597+0000 7f4b8ea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b881560b0 0x7f4b880cf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.597+0000 7f4b8ea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b8813e210 0x7f4b8813e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.597+0000 7f4b8ea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4b880b5b40 con 0x7f4b70005180 2024-03-28T13:20:34.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.597+0000 7f4b8ea00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4b88160980 con 0x7f4b8813e210 2024-03-28T13:20:34.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.597+0000 7f4b8ea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4b880b5cc0 con 0x7f4b881560b0 2024-03-28T13:20:34.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.597+0000 7f4b8d600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b70005180 0x7f4b880b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:34.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.597+0000 7f4b8d600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b70005180 0x7f4b880b2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57324/0 (socket says 172.21.15.19:57324) 2024-03-28T13:20:34.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.597+0000 7f4b8d600640 1 -- 172.21.15.19:0/531040532 learned_addr learned my addr 172.21.15.19:0/531040532 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:34.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.598+0000 7f4b8d600640 1 -- 172.21.15.19:0/531040532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b881560b0 msgr2=0x7f4b880cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:34.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.598+0000 7f4b8d600640 1 --2- 172.21.15.19:0/531040532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b881560b0 0x7f4b880cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.598+0000 7f4b8e000640 1 --2- 172.21.15.19:0/531040532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b8813e210 0x7f4b8813e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:34.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.598+0000 7f4b8d600640 1 -- 172.21.15.19:0/531040532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b8813e210 msgr2=0x7f4b8813e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:34.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.598+0000 7f4b8d600640 1 --2- 172.21.15.19:0/531040532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b8813e210 0x7f4b8813e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.598+0000 7f4b8d600640 1 -- 172.21.15.19:0/531040532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b88156840 con 0x7f4b70005180 2024-03-28T13:20:34.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.598+0000 7f4b8d600640 1 --2- 172.21.15.19:0/531040532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b70005180 0x7f4b880b2500 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00e860 tx=0x7f4b7c00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.599+0000 7f4b85600640 1 -- 172.21.15.19:0/531040532 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c00c8d0 con 0x7f4b70005180 2024-03-28T13:20:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.599+0000 7f4b85600640 1 -- 172.21.15.19:0/531040532 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b7c018070 con 0x7f4b70005180 2024-03-28T13:20:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.599+0000 7f4b85600640 1 -- 172.21.15.19:0/531040532 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c00ccd0 con 0x7f4b70005180 2024-03-28T13:20:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.599+0000 7f4b8ea00640 1 -- 172.21.15.19:0/531040532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b88156b20 con 0x7f4b70005180 2024-03-28T13:20:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.599+0000 7f4b8ea00640 1 -- 172.21.15.19:0/531040532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b881570e0 con 0x7f4b70005180 2024-03-28T13:20:34.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.600+0000 7f4b8ea00640 1 -- 172.21.15.19:0/531040532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b88075940 con 0x7f4b70005180 2024-03-28T13:20:34.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.601+0000 7f4b85600640 1 -- 172.21.15.19:0/531040532 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4b7c01d460 con 0x7f4b70005180 2024-03-28T13:20:34.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.602+0000 7f4b85600640 1 --2- 172.21.15.19:0/531040532 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4b6806c530 0x7f4b6806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:34.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.602+0000 7f4b85600640 1 -- 172.21.15.19:0/531040532 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5939+0+0 (secure 0 0 0) 0x7f4b7c08b770 con 0x7f4b70005180 2024-03-28T13:20:34.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.602+0000 7f4b8cc00640 1 --2- 172.21.15.19:0/531040532 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4b6806c530 0x7f4b6806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:34.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.604+0000 7f4b8cc00640 1 --2- 172.21.15.19:0/531040532 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4b6806c530 0x7f4b6806e9f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009910 tx=0x7f4b74008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:34.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.606+0000 7f4b85600640 1 -- 172.21.15.19:0/531040532 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4b7c059c00 con 0x7f4b70005180 2024-03-28T13:20:34.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.809+0000 7f4b8ea00640 1 -- 172.21.15.19:0/531040532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4b88073830 con 0x7f4b70005180 2024-03-28T13:20:34.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.813+0000 7f4b85600640 1 -- 172.21.15.19:0/531040532 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16975 (secure 0 0 0) 0x7f4b7c059500 con 0x7f4b70005180 2024-03-28T13:20:34.813 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:34.813 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":109,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:20:33.234662+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:18:56.432417+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-28T13:18:57.540515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.c","mappings":[{"from":5,"to":0}]},{"pgid":"14.18","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/2851646201":"2024-03-29T13:20:33.234624+0000","172.21.15.19:6832/2851646201":"2024-03-29T13:20:33.234624+0000","172.21.15.19:6834/1020599987":"2024-03-29T13:19:55.818562+0000","172.21.15.19:6832/1020599987":"2024-03-29T13:19:55.818562+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:20:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.815+0000 7f4b8ea00640 1 -- 172.21.15.19:0/531040532 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4b6806c530 msgr2=0x7f4b6806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.815+0000 7f4b8ea00640 1 --2- 172.21.15.19:0/531040532 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4b6806c530 0x7f4b6806e9f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009910 tx=0x7f4b74008040 comp rx=0 tx=0).stop 2024-03-28T13:20:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.815+0000 7f4b8ea00640 1 -- 172.21.15.19:0/531040532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b70005180 msgr2=0x7f4b880b2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.815+0000 7f4b8ea00640 1 --2- 172.21.15.19:0/531040532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b70005180 0x7f4b880b2500 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00e860 tx=0x7f4b7c00ed20 comp rx=0 tx=0).stop 2024-03-28T13:20:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.815+0000 7f4b8ea00640 1 -- 172.21.15.19:0/531040532 shutdown_connections 2024-03-28T13:20:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.815+0000 7f4b8ea00640 1 --2- 172.21.15.19:0/531040532 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4b6806c530 0x7f4b6806e9f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.816+0000 7f4b8ea00640 1 --2- 172.21.15.19:0/531040532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b8813e210 0x7f4b8813e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.816+0000 7f4b8ea00640 1 --2- 172.21.15.19:0/531040532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b881560b0 0x7f4b880cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.816+0000 7f4b8ea00640 1 --2- 172.21.15.19:0/531040532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4b70005180 0x7f4b880b2500 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.816+0000 7f4b8ea00640 1 -- 172.21.15.19:0/531040532 >> 172.21.15.19:0/531040532 conn(0x7f4b880ccad0 msgr2=0x7f4b880cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.816+0000 7f4b8ea00640 1 -- 172.21.15.19:0/531040532 shutdown_connections 2024-03-28T13:20:34.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:34.816+0000 7f4b8ea00640 1 -- 172.21.15.19:0/531040532 wait complete. 2024-03-28T13:20:34.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:18:56.432417+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:18:57.540515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:20:34.835 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:20:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.054+0000 7efd83e00640 1 Processor -- start 2024-03-28T13:20:35.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.054+0000 7efd83e00640 1 -- start start 2024-03-28T13:20:35.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.056+0000 7efd83e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7c0d0990 0x7efd7c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:35.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.056+0000 7efd83e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd7c0afab0 con 0x7efd7c0af2e0 2024-03-28T13:20:35.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.056+0000 7efd83e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd7c15cc80 con 0x7efd7c153580 2024-03-28T13:20:35.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.056+0000 7efd83e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efd7c15da00 con 0x7efd7c0d0990 2024-03-28T13:20:35.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.056+0000 7efd81e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efd7c0af2e0 0x7efd7c1600d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:58378/0 (socket says 172.21.15.19:58378) 2024-03-28T13:20:35.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.056+0000 7efd81e00640 1 -- 172.21.15.19:0/373775894 learned_addr learned my addr 172.21.15.19:0/373775894 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:35.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.057+0000 7efd80a00640 1 --2- 172.21.15.19:0/373775894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7c0d0990 0x7efd7c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:35.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.057+0000 7efd7be00640 1 -- 172.21.15.19:0/373775894 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2861761774 0 0) 0x7efd7c0afab0 con 0x7efd7c0af2e0 2024-03-28T13:20:35.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.057+0000 7efd7be00640 1 -- 172.21.15.19:0/373775894 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd64003520 con 0x7efd7c0af2e0 2024-03-28T13:20:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.057+0000 7efd7be00640 1 -- 172.21.15.19:0/373775894 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2752756655 0 0) 0x7efd64003520 con 0x7efd7c0af2e0 2024-03-28T13:20:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.057+0000 7efd7be00640 1 -- 172.21.15.19:0/373775894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7c0d0990 msgr2=0x7efd7c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.057+0000 7efd7be00640 1 --2- 172.21.15.19:0/373775894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7c0d0990 0x7efd7c0aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.057+0000 7efd7be00640 1 -- 172.21.15.19:0/373775894 >> v1:172.21.15.188:6789/0 conn(0x7efd7c153580 legacy=0x7efd7c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.057+0000 7efd7be00640 1 -- 172.21.15.19:0/373775894 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c15cf20 con 0x7efd7c0af2e0 2024-03-28T13:20:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.057+0000 7efd7be00640 1 -- 172.21.15.19:0/373775894 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7efd740019f0 con 0x7efd7c0af2e0 2024-03-28T13:20:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.058+0000 7efd7be00640 1 -- 172.21.15.19:0/373775894 >> v1:172.21.15.19:6789/0 conn(0x7efd7c0af2e0 legacy=0x7efd7c1600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.058+0000 7efd7be00640 1 --2- 172.21.15.19:0/373775894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd640039e0 0x7efd64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.058+0000 7efd7be00640 1 --2- 172.21.15.19:0/373775894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7c0d0990 0x7efd64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.058+0000 7efd7be00640 1 --2- 172.21.15.19:0/373775894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd64004d40 0x7efd640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.058+0000 7efd7be00640 1 -- 172.21.15.19:0/373775894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c15da00 con 0x7efd64004d40 2024-03-28T13:20:35.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.058+0000 7efd7be00640 1 -- 172.21.15.19:0/373775894 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd6400baf0 con 0x7efd640039e0 2024-03-28T13:20:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.058+0000 7efd7be00640 1 -- 172.21.15.19:0/373775894 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efd6400bcf0 con 0x7efd7c0d0990 2024-03-28T13:20:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.058+0000 7efd81e00640 1 --2- 172.21.15.19:0/373775894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd64004d40 0x7efd640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.058+0000 7efd80a00640 1 --2- 172.21.15.19:0/373775894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7c0d0990 0x7efd64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.058+0000 7efd81400640 1 --2- 172.21.15.19:0/373775894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd640039e0 0x7efd64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.058+0000 7efd81e00640 1 -- 172.21.15.19:0/373775894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7c0d0990 msgr2=0x7efd64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.058+0000 7efd81e00640 1 --2- 172.21.15.19:0/373775894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7c0d0990 0x7efd64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.058+0000 7efd81e00640 1 -- 172.21.15.19:0/373775894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd640039e0 msgr2=0x7efd64003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.058+0000 7efd81e00640 1 --2- 172.21.15.19:0/373775894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd640039e0 0x7efd64003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.058+0000 7efd81e00640 1 -- 172.21.15.19:0/373775894 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd64003520 con 0x7efd64004d40 2024-03-28T13:20:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.060+0000 7efd81e00640 1 --2- 172.21.15.19:0/373775894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd64004d40 0x7efd640051c0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7efd7400a850 tx=0x7efd64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7708027501945729 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.060+0000 7efd7be00640 1 -- 172.21.15.19:0/373775894 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7400a4a0 con 0x7efd64004d40 2024-03-28T13:20:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.060+0000 7efd7be00640 1 -- 172.21.15.19:0/373775894 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd74034650 con 0x7efd64004d40 2024-03-28T13:20:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.060+0000 7efd7be00640 1 -- 172.21.15.19:0/373775894 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74034a40 con 0x7efd64004d40 2024-03-28T13:20:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.061+0000 7efd83e00640 1 -- 172.21.15.19:0/373775894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd64004d40 msgr2=0x7efd640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.061+0000 7efd83e00640 1 --2- 172.21.15.19:0/373775894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd64004d40 0x7efd640051c0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7efd7400a850 tx=0x7efd64005830 comp rx=0 tx=0).stop 2024-03-28T13:20:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.061+0000 7efd81e00640 1 -- 172.21.15.19:0/373775894 reap_dead start 2024-03-28T13:20:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.061+0000 7efd83e00640 1 -- 172.21.15.19:0/373775894 shutdown_connections 2024-03-28T13:20:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.061+0000 7efd83e00640 1 -- 172.21.15.19:0/373775894 >> 172.21.15.19:0/373775894 conn(0x7efd7c06b750 msgr2=0x7efd7c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.061+0000 7efd83e00640 1 -- 172.21.15.19:0/373775894 shutdown_connections 2024-03-28T13:20:35.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.061+0000 7efd83e00640 1 -- 172.21.15.19:0/373775894 wait complete. 2024-03-28T13:20:35.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.063+0000 7efd83e00640 1 Processor -- start 2024-03-28T13:20:35.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.063+0000 7efd83e00640 1 -- start start 2024-03-28T13:20:35.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd83e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd64004d40 0x7efd7c14a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:35.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd83e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7c0cf0e0 0x7efd7c0b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:35.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd83e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd7c0b0c50 0x7efd7c1f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:35.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd83e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c153960 con 0x7efd64004d40 2024-03-28T13:20:35.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd83e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c15d4e0 con 0x7efd7c0b0c50 2024-03-28T13:20:35.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd83e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efd7c0af710 con 0x7efd7c0cf0e0 2024-03-28T13:20:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd81400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd64004d40 0x7efd7c14a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd81400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd64004d40 0x7efd7c14a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57340/0 (socket says 172.21.15.19:57340) 2024-03-28T13:20:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd81400640 1 -- 172.21.15.19:0/279111520 learned_addr learned my addr 172.21.15.19:0/279111520 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd80a00640 1 --2- 172.21.15.19:0/279111520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7c0cf0e0 0x7efd7c0b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd81400640 1 -- 172.21.15.19:0/279111520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7c0cf0e0 msgr2=0x7efd7c0b0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd81400640 1 --2- 172.21.15.19:0/279111520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7c0cf0e0 0x7efd7c0b0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd81e00640 1 --2- 172.21.15.19:0/279111520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd7c0b0c50 0x7efd7c1f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd81400640 1 -- 172.21.15.19:0/279111520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd7c0b0c50 msgr2=0x7efd7c1f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd81400640 1 --2- 172.21.15.19:0/279111520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd7c0b0c50 0x7efd7c1f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd81400640 1 -- 172.21.15.19:0/279111520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c0cf560 con 0x7efd64004d40 2024-03-28T13:20:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.065+0000 7efd81e00640 1 --2- 172.21.15.19:0/279111520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd7c0b0c50 0x7efd7c1f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.066+0000 7efd81400640 1 --2- 172.21.15.19:0/279111520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd64004d40 0x7efd7c14a830 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7efd7000d5a0 tx=0x7efd7000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.066+0000 7efd79600640 1 -- 172.21.15.19:0/279111520 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70005480 con 0x7efd64004d40 2024-03-28T13:20:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.066+0000 7efd79600640 1 -- 172.21.15.19:0/279111520 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd70017070 con 0x7efd64004d40 2024-03-28T13:20:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.066+0000 7efd79600640 1 -- 172.21.15.19:0/279111520 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70003810 con 0x7efd64004d40 2024-03-28T13:20:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.066+0000 7efd83e00640 1 -- 172.21.15.19:0/279111520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7c072910 con 0x7efd64004d40 2024-03-28T13:20:35.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.066+0000 7efd83e00640 1 -- 172.21.15.19:0/279111520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7c072ed0 con 0x7efd64004d40 2024-03-28T13:20:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.068+0000 7efd83e00640 1 -- 172.21.15.19:0/279111520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7c0685c0 con 0x7efd64004d40 2024-03-28T13:20:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.068+0000 7efd79600640 1 -- 172.21.15.19:0/279111520 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efd700039b0 con 0x7efd64004d40 2024-03-28T13:20:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.069+0000 7efd79600640 1 --2- 172.21.15.19:0/279111520 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd5c06c530 0x7efd5c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:35.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.069+0000 7efd79600640 1 -- 172.21.15.19:0/279111520 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5939+0+0 (secure 0 0 0) 0x7efd7008b710 con 0x7efd64004d40 2024-03-28T13:20:35.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.070+0000 7efd80a00640 1 --2- 172.21.15.19:0/279111520 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd5c06c530 0x7efd5c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:35.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.071+0000 7efd80a00640 1 --2- 172.21.15.19:0/279111520 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd5c06c530 0x7efd5c06e9f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0b0370 tx=0x7efd68008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:35.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.074+0000 7efd79600640 1 -- 172.21.15.19:0/279111520 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efd70059ba0 con 0x7efd64004d40 2024-03-28T13:20:35.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.283+0000 7efd83e00640 1 -- 172.21.15.19:0/279111520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd7c0b1030 con 0x7efd64004d40 2024-03-28T13:20:35.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.284+0000 7efd79600640 1 -- 172.21.15.19:0/279111520 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7efd7001ae40 con 0x7efd64004d40 2024-03-28T13:20:35.284 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:20:35.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.287+0000 7efd83e00640 1 -- 172.21.15.19:0/279111520 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd5c06c530 msgr2=0x7efd5c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:35.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.287+0000 7efd83e00640 1 --2- 172.21.15.19:0/279111520 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd5c06c530 0x7efd5c06e9f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0b0370 tx=0x7efd68008040 comp rx=0 tx=0).stop 2024-03-28T13:20:35.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.287+0000 7efd83e00640 1 -- 172.21.15.19:0/279111520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd64004d40 msgr2=0x7efd7c14a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:35.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.287+0000 7efd83e00640 1 --2- 172.21.15.19:0/279111520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd64004d40 0x7efd7c14a830 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7efd7000d5a0 tx=0x7efd7000da60 comp rx=0 tx=0).stop 2024-03-28T13:20:35.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.287+0000 7efd83e00640 1 -- 172.21.15.19:0/279111520 shutdown_connections 2024-03-28T13:20:35.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.287+0000 7efd83e00640 1 --2- 172.21.15.19:0/279111520 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd5c06c530 0x7efd5c06e9f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.287+0000 7efd83e00640 1 --2- 172.21.15.19:0/279111520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd7c0b0c50 0x7efd7c1f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.287+0000 7efd83e00640 1 --2- 172.21.15.19:0/279111520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7c0cf0e0 0x7efd7c0b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.287+0000 7efd83e00640 1 --2- 172.21.15.19:0/279111520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd64004d40 0x7efd7c14a830 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.287+0000 7efd83e00640 1 -- 172.21.15.19:0/279111520 >> 172.21.15.19:0/279111520 conn(0x7efd7c06b750 msgr2=0x7efd7c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:35.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.287+0000 7efd83e00640 1 -- 172.21.15.19:0/279111520 shutdown_connections 2024-03-28T13:20:35.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.288+0000 7efd83e00640 1 -- 172.21.15.19:0/279111520 wait complete. 2024-03-28T13:20:35.306 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:20:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.532+0000 7f6100e00640 1 Processor -- start 2024-03-28T13:20:35.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.532+0000 7f6100e00640 1 -- start start 2024-03-28T13:20:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.534+0000 7f6100e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f60fc15bf50 0x7f60fc0b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.534+0000 7f6100e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60fc0b3cf0 0x7f60fc06d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.534+0000 7f6100e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f60fc06db60 0x7f60fc161f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.534+0000 7f6100e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f60fc06dff0 con 0x7f60fc06db60 2024-03-28T13:20:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.534+0000 7f6100e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f60fc06ee40 con 0x7f60fc15bf50 2024-03-28T13:20:35.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.534+0000 7f6100e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc0b4130 con 0x7f60fc0b3cf0 2024-03-28T13:20:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.535+0000 7f60fbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f60fc06db60 0x7f60fc161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.535+0000 7f60fbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f60fc06db60 0x7f60fc161f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52544/0 (socket says 172.21.15.19:52544) 2024-03-28T13:20:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.535+0000 7f60fbe00640 1 -- 172.21.15.19:0/583487476 learned_addr learned my addr 172.21.15.19:0/583487476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.535+0000 7f60fb400640 1 --2- 172.21.15.19:0/583487476 >> v2:172.21.15.188:3300/0 conn(0x7f60fc15bf50 0x7f60fc0b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.535+0000 7f60faa00640 1 --2- 172.21.15.19:0/583487476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60fc0b3cf0 0x7f60fc06d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:35.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.535+0000 7f60fbe00640 1 -- 172.21.15.19:0/583487476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60fc0b3cf0 msgr2=0x7f60fc06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.535+0000 7f60fbe00640 1 --2- 172.21.15.19:0/583487476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60fc0b3cf0 0x7f60fc06d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.535+0000 7f60fbe00640 1 -- 172.21.15.19:0/583487476 >> v2:172.21.15.188:3300/0 conn(0x7f60fc15bf50 msgr2=0x7f60fc0b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.535+0000 7f60fbe00640 1 --2- 172.21.15.19:0/583487476 >> v2:172.21.15.188:3300/0 conn(0x7f60fc15bf50 0x7f60fc0b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.535+0000 7f60fbe00640 1 -- 172.21.15.19:0/583487476 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc15c330 con 0x7f60fc06db60 2024-03-28T13:20:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.536+0000 7f60fbe00640 1 --2- 172.21.15.19:0/583487476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60fc06db60 0x7f60fc161f60 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0b27b0 tx=0x7f60f402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668c332962b67648 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.536+0000 7f60fa000640 1 -- 172.21.15.19:0/583487476 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f402cb00 con 0x7f60fc06db60 2024-03-28T13:20:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.536+0000 7f60fa000640 1 -- 172.21.15.19:0/583487476 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60f402cca0 con 0x7f60fc06db60 2024-03-28T13:20:35.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.536+0000 7f60fa000640 1 -- 172.21.15.19:0/583487476 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f4034610 con 0x7f60fc06db60 2024-03-28T13:20:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.537+0000 7f6100e00640 1 -- 172.21.15.19:0/583487476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60fc06db60 msgr2=0x7f60fc161f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.537+0000 7f6100e00640 1 --2- 172.21.15.19:0/583487476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60fc06db60 0x7f60fc161f60 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0b27b0 tx=0x7f60f402bf60 comp rx=0 tx=0).stop 2024-03-28T13:20:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.537+0000 7f6100e00640 1 -- 172.21.15.19:0/583487476 shutdown_connections 2024-03-28T13:20:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.537+0000 7f6100e00640 1 --2- 172.21.15.19:0/583487476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60fc06db60 0x7f60fc161f60 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.537+0000 7f6100e00640 1 --2- 172.21.15.19:0/583487476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60fc0b3cf0 0x7f60fc06d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.537+0000 7f6100e00640 1 --2- 172.21.15.19:0/583487476 >> v2:172.21.15.188:3300/0 conn(0x7f60fc15bf50 0x7f60fc0b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.537+0000 7f6100e00640 1 -- 172.21.15.19:0/583487476 >> 172.21.15.19:0/583487476 conn(0x7f60fc0b1ba0 msgr2=0x7f60fc0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.537+0000 7f6100e00640 1 -- 172.21.15.19:0/583487476 shutdown_connections 2024-03-28T13:20:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.537+0000 7f6100e00640 1 -- 172.21.15.19:0/583487476 wait complete. 2024-03-28T13:20:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.539+0000 7f6100e00640 1 Processor -- start 2024-03-28T13:20:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.539+0000 7f6100e00640 1 -- start start 2024-03-28T13:20:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.540+0000 7f6100e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60fc06db60 0x7f60fc1f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.540+0000 7f6100e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60fc0b3cf0 0x7f60fc15d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.540+0000 7f6100e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60fc15bf50 0x7f60fc15dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.540+0000 7f6100e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc06e510 con 0x7f60fc0b3cf0 2024-03-28T13:20:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.540+0000 7f6100e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f60fc15ed30 con 0x7f60fc15bf50 2024-03-28T13:20:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.540+0000 7f6100e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f60fc06e7f0 con 0x7f60fc06db60 2024-03-28T13:20:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.540+0000 7f60faa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60fc0b3cf0 0x7f60fc15d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.540+0000 7f60faa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60fc0b3cf0 0x7f60fc15d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52546/0 (socket says 172.21.15.19:52546) 2024-03-28T13:20:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.540+0000 7f60faa00640 1 -- 172.21.15.19:0/3127844228 learned_addr learned my addr 172.21.15.19:0/3127844228 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:35.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.541+0000 7f60fb400640 1 --2- 172.21.15.19:0/3127844228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60fc06db60 0x7f60fc1f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:35.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.541+0000 7f60fbe00640 1 --2- 172.21.15.19:0/3127844228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60fc15bf50 0x7f60fc15dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.541+0000 7f60faa00640 1 -- 172.21.15.19:0/3127844228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60fc06db60 msgr2=0x7f60fc1f68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.541+0000 7f60faa00640 1 --2- 172.21.15.19:0/3127844228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60fc06db60 0x7f60fc1f68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.541+0000 7f60faa00640 1 -- 172.21.15.19:0/3127844228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60fc15bf50 msgr2=0x7f60fc15dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.541+0000 7f60faa00640 1 --2- 172.21.15.19:0/3127844228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60fc15bf50 0x7f60fc15dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.541+0000 7f60faa00640 1 -- 172.21.15.19:0/3127844228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60fc0c6750 con 0x7f60fc0b3cf0 2024-03-28T13:20:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.541+0000 7f60faa00640 1 --2- 172.21.15.19:0/3127844228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60fc0b3cf0 0x7f60fc15d840 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f60e800d830 tx=0x7f60e800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.541+0000 7f60f3400640 1 -- 172.21.15.19:0/3127844228 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e800b6c0 con 0x7f60fc0b3cf0 2024-03-28T13:20:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.541+0000 7f60f3400640 1 -- 172.21.15.19:0/3127844228 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e8019070 con 0x7f60fc0b3cf0 2024-03-28T13:20:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.541+0000 7f60f3400640 1 -- 172.21.15.19:0/3127844228 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e800ba80 con 0x7f60fc0b3cf0 2024-03-28T13:20:35.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.542+0000 7f6100e00640 1 -- 172.21.15.19:0/3127844228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60fc0c6a30 con 0x7f60fc0b3cf0 2024-03-28T13:20:35.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.542+0000 7f6100e00640 1 -- 172.21.15.19:0/3127844228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60fc0c6ff0 con 0x7f60fc0b3cf0 2024-03-28T13:20:35.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.543+0000 7f6100e00640 1 -- 172.21.15.19:0/3127844228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60fc075940 con 0x7f60fc0b3cf0 2024-03-28T13:20:35.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.544+0000 7f60f3400640 1 -- 172.21.15.19:0/3127844228 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f60e801e460 con 0x7f60fc0b3cf0 2024-03-28T13:20:35.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.545+0000 7f60f3400640 1 --2- 172.21.15.19:0/3127844228 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f60dc06c580 0x7f60dc06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:35.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.545+0000 7f60f3400640 1 -- 172.21.15.19:0/3127844228 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5939+0+0 (secure 0 0 0) 0x7f60e808beb0 con 0x7f60fc0b3cf0 2024-03-28T13:20:35.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.546+0000 7f60fb400640 1 --2- 172.21.15.19:0/3127844228 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f60dc06c580 0x7f60dc06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:35.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.547+0000 7f60fb400640 1 --2- 172.21.15.19:0/3127844228 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f60dc06c580 0x7f60dc06ea40 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f60ec006fd0 tx=0x7f60ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:35.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.550+0000 7f60f3400640 1 -- 172.21.15.19:0/3127844228 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f60e805a450 con 0x7f60fc0b3cf0 2024-03-28T13:20:35.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.754+0000 7f6100e00640 1 -- 172.21.15.19:0/3127844228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f60fc0c73d0 con 0x7f60fc0b3cf0 2024-03-28T13:20:35.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.755+0000 7f60f3400640 1 -- 172.21.15.19:0/3127844228 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) ==== 104+0+11 (secure 0 0 0) 0x7f60e8059d50 con 0x7f60fc0b3cf0 2024-03-28T13:20:35.754 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:20:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.757+0000 7f6100e00640 1 -- 172.21.15.19:0/3127844228 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f60dc06c580 msgr2=0x7f60dc06ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.757+0000 7f6100e00640 1 --2- 172.21.15.19:0/3127844228 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f60dc06c580 0x7f60dc06ea40 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f60ec006fd0 tx=0x7f60ec008040 comp rx=0 tx=0).stop 2024-03-28T13:20:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.757+0000 7f6100e00640 1 -- 172.21.15.19:0/3127844228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60fc0b3cf0 msgr2=0x7f60fc15d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.757+0000 7f6100e00640 1 --2- 172.21.15.19:0/3127844228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60fc0b3cf0 0x7f60fc15d840 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f60e800d830 tx=0x7f60e800dcf0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.758+0000 7f6100e00640 1 -- 172.21.15.19:0/3127844228 shutdown_connections 2024-03-28T13:20:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.758+0000 7f6100e00640 1 --2- 172.21.15.19:0/3127844228 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f60dc06c580 0x7f60dc06ea40 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.758+0000 7f6100e00640 1 --2- 172.21.15.19:0/3127844228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60fc15bf50 0x7f60fc15dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.758+0000 7f6100e00640 1 --2- 172.21.15.19:0/3127844228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f60fc0b3cf0 0x7f60fc15d840 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.758+0000 7f6100e00640 1 --2- 172.21.15.19:0/3127844228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60fc06db60 0x7f60fc1f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.758+0000 7f6100e00640 1 -- 172.21.15.19:0/3127844228 >> 172.21.15.19:0/3127844228 conn(0x7f60fc0b1ba0 msgr2=0x7f60fc1600d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.758+0000 7f6100e00640 1 -- 172.21.15.19:0/3127844228 shutdown_connections 2024-03-28T13:20:35.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:35.758+0000 7f6100e00640 1 -- 172.21.15.19:0/3127844228 wait complete. 2024-03-28T13:20:35.778 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:20:36.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.010+0000 7f0b8aa00640 1 Processor -- start 2024-03-28T13:20:36.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.010+0000 7f0b8aa00640 1 -- start start 2024-03-28T13:20:36.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.012+0000 7f0b8aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b840cc880 0x7f0b840cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.012+0000 7f0b8aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b8406d620 0x7f0b8406da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.012+0000 7f0b8aa00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b840ce890 con 0x7f0b8413f190 2024-03-28T13:20:36.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.012+0000 7f0b8aa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0b840cd200 con 0x7f0b8406d620 2024-03-28T13:20:36.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.012+0000 7f0b8aa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0b840ceb50 con 0x7f0b840cc880 2024-03-28T13:20:36.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.012+0000 7f0b8a000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b8406d620 0x7f0b8406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.012+0000 7f0b8a000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b8406d620 0x7f0b8406da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52560/0 (socket says 172.21.15.19:52560) 2024-03-28T13:20:36.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.012+0000 7f0b8a000640 1 -- 172.21.15.19:0/716745965 learned_addr learned my addr 172.21.15.19:0/716745965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:36.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.012+0000 7f0b88c00640 1 --2- 172.21.15.19:0/716745965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b840cc880 0x7f0b840cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.013+0000 7f0b8a000640 1 -- 172.21.15.19:0/716745965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b840cc880 msgr2=0x7f0b840cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:36.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.013+0000 7f0b8a000640 1 --2- 172.21.15.19:0/716745965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b840cc880 0x7f0b840cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.013+0000 7f0b8a000640 1 -- 172.21.15.19:0/716745965 >> v1:172.21.15.188:6789/0 conn(0x7f0b8413f190 legacy=0x7f0b840cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:36.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.013+0000 7f0b8a000640 1 -- 172.21.15.19:0/716745965 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b840cee30 con 0x7f0b8406d620 2024-03-28T13:20:36.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.014+0000 7f0b8a000640 1 --2- 172.21.15.19:0/716745965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b8406d620 0x7f0b8406da70 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009e50 tx=0x7f0b7c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5508983dfa9b774 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:36.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.014+0000 7f0b83e00640 1 -- 172.21.15.19:0/716745965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c02ca70 con 0x7f0b8406d620 2024-03-28T13:20:36.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.014+0000 7f0b83e00640 1 -- 172.21.15.19:0/716745965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b7c02cc10 con 0x7f0b8406d620 2024-03-28T13:20:36.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.014+0000 7f0b83e00640 1 -- 172.21.15.19:0/716745965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c0343f0 con 0x7f0b8406d620 2024-03-28T13:20:36.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.015+0000 7f0b8aa00640 1 -- 172.21.15.19:0/716745965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b8406d620 msgr2=0x7f0b8406da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:36.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.015+0000 7f0b8aa00640 1 --2- 172.21.15.19:0/716745965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b8406d620 0x7f0b8406da70 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c009e50 tx=0x7f0b7c02bed0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.015+0000 7f0b8aa00640 1 -- 172.21.15.19:0/716745965 shutdown_connections 2024-03-28T13:20:36.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.015+0000 7f0b8aa00640 1 --2- 172.21.15.19:0/716745965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b8406d620 0x7f0b8406da70 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.015+0000 7f0b8aa00640 1 --2- 172.21.15.19:0/716745965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b840cc880 0x7f0b840cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.015+0000 7f0b8aa00640 1 -- 172.21.15.19:0/716745965 >> 172.21.15.19:0/716745965 conn(0x7f0b840b1ba0 msgr2=0x7f0b840b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:36.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.015+0000 7f0b8aa00640 1 -- 172.21.15.19:0/716745965 shutdown_connections 2024-03-28T13:20:36.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.015+0000 7f0b8aa00640 1 -- 172.21.15.19:0/716745965 wait complete. 2024-03-28T13:20:36.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.017+0000 7f0b8aa00640 1 Processor -- start 2024-03-28T13:20:36.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.017+0000 7f0b8aa00640 1 -- start start 2024-03-28T13:20:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b8aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b8406d620 0x7f0b841e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b8aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b840cc880 0x7f0b841e9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b8aa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b8413f190 0x7f0b84072430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b8aa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8413f570 con 0x7f0b840cc880 2024-03-28T13:20:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b8aa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0b84160c70 con 0x7f0b8413f190 2024-03-28T13:20:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b8aa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8406dfb0 con 0x7f0b8406d620 2024-03-28T13:20:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b88c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b840cc880 0x7f0b841e9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b88c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b840cc880 0x7f0b841e9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52564/0 (socket says 172.21.15.19:52564) 2024-03-28T13:20:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b88c00640 1 -- 172.21.15.19:0/509684848 learned_addr learned my addr 172.21.15.19:0/509684848 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b89600640 1 --2- 172.21.15.19:0/509684848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b8406d620 0x7f0b841e6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b8a000640 1 --2- 172.21.15.19:0/509684848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b8413f190 0x7f0b84072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b88c00640 1 -- 172.21.15.19:0/509684848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b8406d620 msgr2=0x7f0b841e6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b88c00640 1 --2- 172.21.15.19:0/509684848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b8406d620 0x7f0b841e6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b88c00640 1 -- 172.21.15.19:0/509684848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b8413f190 msgr2=0x7f0b84072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b88c00640 1 --2- 172.21.15.19:0/509684848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b8413f190 0x7f0b84072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b88c00640 1 -- 172.21.15.19:0/509684848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b841f57a0 con 0x7f0b840cc880 2024-03-28T13:20:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.018+0000 7f0b8a000640 1 --2- 172.21.15.19:0/509684848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b8413f190 0x7f0b84072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.019+0000 7f0b88c00640 1 --2- 172.21.15.19:0/509684848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b840cc880 0x7f0b841e9d90 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000d800 tx=0x7f0b7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.019+0000 7f0b81600640 1 -- 172.21.15.19:0/509684848 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000b6c0 con 0x7f0b840cc880 2024-03-28T13:20:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.019+0000 7f0b81600640 1 -- 172.21.15.19:0/509684848 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b70019070 con 0x7f0b840cc880 2024-03-28T13:20:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.019+0000 7f0b81600640 1 -- 172.21.15.19:0/509684848 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7000ba80 con 0x7f0b840cc880 2024-03-28T13:20:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.019+0000 7f0b8aa00640 1 -- 172.21.15.19:0/509684848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b841f5a80 con 0x7f0b840cc880 2024-03-28T13:20:36.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.019+0000 7f0b8aa00640 1 -- 172.21.15.19:0/509684848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b841f6040 con 0x7f0b840cc880 2024-03-28T13:20:36.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.021+0000 7f0b8aa00640 1 -- 172.21.15.19:0/509684848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b84075940 con 0x7f0b840cc880 2024-03-28T13:20:36.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.021+0000 7f0b81600640 1 -- 172.21.15.19:0/509684848 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0b7000bc20 con 0x7f0b840cc880 2024-03-28T13:20:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.022+0000 7f0b81600640 1 --2- 172.21.15.19:0/509684848 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b6406c530 0x7f0b6406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.022+0000 7f0b81600640 1 -- 172.21.15.19:0/509684848 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5939+0+0 (secure 0 0 0) 0x7f0b7008bd30 con 0x7f0b840cc880 2024-03-28T13:20:36.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.023+0000 7f0b89600640 1 --2- 172.21.15.19:0/509684848 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b6406c530 0x7f0b6406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.024+0000 7f0b89600640 1 --2- 172.21.15.19:0/509684848 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b6406c530 0x7f0b6406e9f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0b78004ea0 tx=0x7f0b78009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:36.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.027+0000 7f0b81600640 1 -- 172.21.15.19:0/509684848 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0b7005a1c0 con 0x7f0b840cc880 2024-03-28T13:20:36.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.230+0000 7f0b8aa00640 1 -- 172.21.15.19:0/509684848 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0b8406f050 con 0x7f0b840cc880 2024-03-28T13:20:36.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.231+0000 7f0b81600640 1 -- 172.21.15.19:0/509684848 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) ==== 100+0+11 (secure 0 0 0) 0x7f0b70059ac0 con 0x7f0b840cc880 2024-03-28T13:20:36.230 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:20:36.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.233+0000 7f0b8aa00640 1 -- 172.21.15.19:0/509684848 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b6406c530 msgr2=0x7f0b6406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:36.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.233+0000 7f0b8aa00640 1 --2- 172.21.15.19:0/509684848 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b6406c530 0x7f0b6406e9f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0b78004ea0 tx=0x7f0b78009040 comp rx=0 tx=0).stop 2024-03-28T13:20:36.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.233+0000 7f0b8aa00640 1 -- 172.21.15.19:0/509684848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b840cc880 msgr2=0x7f0b841e9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:36.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.233+0000 7f0b8aa00640 1 --2- 172.21.15.19:0/509684848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b840cc880 0x7f0b841e9d90 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000d800 tx=0x7f0b7000dcc0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.233+0000 7f0b8aa00640 1 -- 172.21.15.19:0/509684848 shutdown_connections 2024-03-28T13:20:36.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.233+0000 7f0b8aa00640 1 --2- 172.21.15.19:0/509684848 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b6406c530 0x7f0b6406e9f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.233+0000 7f0b8aa00640 1 --2- 172.21.15.19:0/509684848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b8413f190 0x7f0b84072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.233+0000 7f0b8aa00640 1 --2- 172.21.15.19:0/509684848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b840cc880 0x7f0b841e9d90 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.233+0000 7f0b8aa00640 1 --2- 172.21.15.19:0/509684848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b8406d620 0x7f0b841e6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.234+0000 7f0b8aa00640 1 -- 172.21.15.19:0/509684848 >> 172.21.15.19:0/509684848 conn(0x7f0b840b1ba0 msgr2=0x7f0b840afe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:36.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.234+0000 7f0b8aa00640 1 -- 172.21.15.19:0/509684848 shutdown_connections 2024-03-28T13:20:36.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.234+0000 7f0b8aa00640 1 -- 172.21.15.19:0/509684848 wait complete. 2024-03-28T13:20:36.254 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:20:36.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.479+0000 7f7b6be00640 1 Processor -- start 2024-03-28T13:20:36.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.479+0000 7f7b6be00640 1 -- start start 2024-03-28T13:20:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.481+0000 7f7b6be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b6c06ea90 0x7f7b6c0ce380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.481+0000 7f7b6be00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b6c161f30 con 0x7f7b6c0d0390 2024-03-28T13:20:36.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.481+0000 7f7b6be00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b6c1621d0 con 0x7f7b6c147a40 2024-03-28T13:20:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.481+0000 7f7b6be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6c162b00 con 0x7f7b6c06ea90 2024-03-28T13:20:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.481+0000 7f7b63e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7b6c0d0390 0x7f7b6c0cdc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47636/0 (socket says 172.21.15.19:47636) 2024-03-28T13:20:36.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.481+0000 7f7b63e00640 1 -- 172.21.15.19:0/1182019080 learned_addr learned my addr 172.21.15.19:0/1182019080 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.481+0000 7f7b6b400640 1 --2- 172.21.15.19:0/1182019080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b6c06ea90 0x7f7b6c0ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.482+0000 7f7b6a000640 1 -- 172.21.15.19:0/1182019080 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2513512171 0 0) 0x7f7b6c161f30 con 0x7f7b6c0d0390 2024-03-28T13:20:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.482+0000 7f7b6a000640 1 -- 172.21.15.19:0/1182019080 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b50003520 con 0x7f7b6c0d0390 2024-03-28T13:20:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.482+0000 7f7b6a000640 1 -- 172.21.15.19:0/1182019080 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1052913637 0 0) 0x7f7b50003520 con 0x7f7b6c0d0390 2024-03-28T13:20:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.482+0000 7f7b6a000640 1 -- 172.21.15.19:0/1182019080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b6c06ea90 msgr2=0x7f7b6c0ce380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.482+0000 7f7b6a000640 1 --2- 172.21.15.19:0/1182019080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b6c06ea90 0x7f7b6c0ce380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.482+0000 7f7b6a000640 1 -- 172.21.15.19:0/1182019080 >> v1:172.21.15.188:6789/0 conn(0x7f7b6c147a40 legacy=0x7f7b6c0cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.482+0000 7f7b6a000640 1 -- 172.21.15.19:0/1182019080 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c1624b0 con 0x7f7b6c0d0390 2024-03-28T13:20:36.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.482+0000 7f7b6a000640 1 -- 172.21.15.19:0/1182019080 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f7b54001690 con 0x7f7b6c0d0390 2024-03-28T13:20:36.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.482+0000 7f7b6a000640 1 -- 172.21.15.19:0/1182019080 >> v1:172.21.15.19:6789/0 conn(0x7f7b6c0d0390 legacy=0x7f7b6c0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:36.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.482+0000 7f7b6a000640 1 --2- 172.21.15.19:0/1182019080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b500039e0 0x7f7b50003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.482+0000 7f7b6a000640 1 --2- 172.21.15.19:0/1182019080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b6c06ea90 0x7f7b50004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.482+0000 7f7b6a000640 1 --2- 172.21.15.19:0/1182019080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b50004d40 0x7f7b500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.482+0000 7f7b6a000640 1 -- 172.21.15.19:0/1182019080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c162b00 con 0x7f7b50004d40 2024-03-28T13:20:36.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.482+0000 7f7b6a000640 1 -- 172.21.15.19:0/1182019080 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b5000baa0 con 0x7f7b500039e0 2024-03-28T13:20:36.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.482+0000 7f7b6a000640 1 -- 172.21.15.19:0/1182019080 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7b5000bca0 con 0x7f7b6c06ea90 2024-03-28T13:20:36.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.483+0000 7f7b6b400640 1 --2- 172.21.15.19:0/1182019080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b50004d40 0x7f7b500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.483+0000 7f7b63e00640 1 --2- 172.21.15.19:0/1182019080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b6c06ea90 0x7f7b50004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.483+0000 7f7b6aa00640 1 --2- 172.21.15.19:0/1182019080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b500039e0 0x7f7b50003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.483+0000 7f7b6b400640 1 -- 172.21.15.19:0/1182019080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b6c06ea90 msgr2=0x7f7b50004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.483+0000 7f7b6b400640 1 --2- 172.21.15.19:0/1182019080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b6c06ea90 0x7f7b50004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.483+0000 7f7b6b400640 1 -- 172.21.15.19:0/1182019080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b500039e0 msgr2=0x7f7b50003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.483+0000 7f7b6b400640 1 --2- 172.21.15.19:0/1182019080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b500039e0 0x7f7b50003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.483+0000 7f7b6b400640 1 -- 172.21.15.19:0/1182019080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b50003520 con 0x7f7b50004d40 2024-03-28T13:20:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.484+0000 7f7b6b400640 1 --2- 172.21.15.19:0/1182019080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b50004d40 0x7f7b500051c0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009f90 tx=0x7f7b50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea3e192f7ce310b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.484+0000 7f7b6a000640 1 -- 172.21.15.19:0/1182019080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64039070 con 0x7f7b50004d40 2024-03-28T13:20:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.484+0000 7f7b6a000640 1 -- 172.21.15.19:0/1182019080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b6402cbd0 con 0x7f7b50004d40 2024-03-28T13:20:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.484+0000 7f7b6a000640 1 -- 172.21.15.19:0/1182019080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64009c10 con 0x7f7b50004d40 2024-03-28T13:20:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.485+0000 7f7b6be00640 1 -- 172.21.15.19:0/1182019080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b50004d40 msgr2=0x7f7b500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:36.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.485+0000 7f7b6be00640 1 --2- 172.21.15.19:0/1182019080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b50004d40 0x7f7b500051c0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009f90 tx=0x7f7b50005830 comp rx=0 tx=0).stop 2024-03-28T13:20:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.485+0000 7f7b6b400640 1 -- 172.21.15.19:0/1182019080 reap_dead start 2024-03-28T13:20:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.485+0000 7f7b6be00640 1 -- 172.21.15.19:0/1182019080 shutdown_connections 2024-03-28T13:20:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.485+0000 7f7b6be00640 1 -- 172.21.15.19:0/1182019080 >> 172.21.15.19:0/1182019080 conn(0x7f7b6c0ccad0 msgr2=0x7f7b6c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.485+0000 7f7b6be00640 1 -- 172.21.15.19:0/1182019080 shutdown_connections 2024-03-28T13:20:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.485+0000 7f7b6be00640 1 -- 172.21.15.19:0/1182019080 wait complete. 2024-03-28T13:20:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.487+0000 7f7b6be00640 1 Processor -- start 2024-03-28T13:20:36.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.487+0000 7f7b6be00640 1 -- start start 2024-03-28T13:20:36.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.488+0000 7f7b6be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b6c13f7e0 0x7f7b6c1445f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.488+0000 7f7b6be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b6c13fbc0 0x7f7b6c0b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.488+0000 7f7b6be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b6c1f5990 0x7f7b6c0b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.488+0000 7f7b6be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c147fa0 con 0x7f7b6c13f7e0 2024-03-28T13:20:36.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.488+0000 7f7b6be00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c147e20 con 0x7f7b6c1f5990 2024-03-28T13:20:36.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.488+0000 7f7b6be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6c0ce380 con 0x7f7b6c13fbc0 2024-03-28T13:20:36.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.488+0000 7f7b6aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b6c13f7e0 0x7f7b6c1445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.488+0000 7f7b6aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b6c13f7e0 0x7f7b6c1445f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52584/0 (socket says 172.21.15.19:52584) 2024-03-28T13:20:36.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.488+0000 7f7b6aa00640 1 -- 172.21.15.19:0/585975526 learned_addr learned my addr 172.21.15.19:0/585975526 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:36.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.489+0000 7f7b63e00640 1 --2- 172.21.15.19:0/585975526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b6c13fbc0 0x7f7b6c0b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.489+0000 7f7b6aa00640 1 -- 172.21.15.19:0/585975526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b6c13fbc0 msgr2=0x7f7b6c0b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.489+0000 7f7b6aa00640 1 --2- 172.21.15.19:0/585975526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b6c13fbc0 0x7f7b6c0b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.489+0000 7f7b6aa00640 1 -- 172.21.15.19:0/585975526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b6c1f5990 msgr2=0x7f7b6c0b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.489+0000 7f7b6aa00640 1 --2- 172.21.15.19:0/585975526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b6c1f5990 0x7f7b6c0b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.489+0000 7f7b6aa00640 1 -- 172.21.15.19:0/585975526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c144ef0 con 0x7f7b6c13f7e0 2024-03-28T13:20:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.489+0000 7f7b6aa00640 1 --2- 172.21.15.19:0/585975526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b6c13f7e0 0x7f7b6c1445f0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00e550 tx=0x7f7b5c00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.489+0000 7f7b62a00640 1 -- 172.21.15.19:0/585975526 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0052e0 con 0x7f7b6c13f7e0 2024-03-28T13:20:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.490+0000 7f7b62a00640 1 -- 172.21.15.19:0/585975526 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b5c018070 con 0x7f7b6c13f7e0 2024-03-28T13:20:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.490+0000 7f7b62a00640 1 -- 172.21.15.19:0/585975526 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c003810 con 0x7f7b6c13f7e0 2024-03-28T13:20:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.490+0000 7f7b6be00640 1 -- 172.21.15.19:0/585975526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c1451d0 con 0x7f7b6c13f7e0 2024-03-28T13:20:36.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.490+0000 7f7b6be00640 1 -- 172.21.15.19:0/585975526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c0b2c60 con 0x7f7b6c13f7e0 2024-03-28T13:20:36.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.491+0000 7f7b6be00640 1 -- 172.21.15.19:0/585975526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6c075940 con 0x7f7b6c13f7e0 2024-03-28T13:20:36.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.492+0000 7f7b62a00640 1 -- 172.21.15.19:0/585975526 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7b5c0039b0 con 0x7f7b6c13f7e0 2024-03-28T13:20:36.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.493+0000 7f7b62a00640 1 --2- 172.21.15.19:0/585975526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b4806c530 0x7f7b4806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.493+0000 7f7b62a00640 1 -- 172.21.15.19:0/585975526 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5939+0+0 (secure 0 0 0) 0x7f7b5c08b4e0 con 0x7f7b6c13f7e0 2024-03-28T13:20:36.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.493+0000 7f7b63e00640 1 --2- 172.21.15.19:0/585975526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b4806c530 0x7f7b4806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.494+0000 7f7b63e00640 1 --2- 172.21.15.19:0/585975526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b4806c530 0x7f7b4806e9f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7b54004840 tx=0x7f7b54000fe0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:36.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.497+0000 7f7b62a00640 1 -- 172.21.15.19:0/585975526 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7b5c059970 con 0x7f7b6c13f7e0 2024-03-28T13:20:36.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.738+0000 7f7b6be00640 1 -- 172.21.15.19:0/585975526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b6c142010 con 0x7f7b6c13f7e0 2024-03-28T13:20:36.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.740+0000 7f7b62a00640 1 -- 172.21.15.19:0/585975526 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5176 (secure 0 0 0) 0x7f7b5c059270 con 0x7f7b6c13f7e0 2024-03-28T13:20:36.739 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:36.740 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4247326257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4247326257},{"type":"v1","addr":"172.21.15.188:6836","nonce":4247326257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3452274156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3452274156},{"type":"v1","addr":"172.21.15.19:6835","nonce":3452274156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3383190607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3383190607},{"type":"v1","addr":"172.21.15.188:6837","nonce":3383190607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3193953635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3193953635},{"type":"v1","addr":"172.21.15.19:6834","nonce":3193953635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:18:58.541599+0000","modified":"2024-03-28T13:20:33.234961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-03-28T13:20:36.740 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 90 2024-03-28T13:20:36.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.742+0000 7f7b6be00640 1 -- 172.21.15.19:0/585975526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b4806c530 msgr2=0x7f7b4806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:36.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.742+0000 7f7b6be00640 1 --2- 172.21.15.19:0/585975526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b4806c530 0x7f7b4806e9f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7b54004840 tx=0x7f7b54000fe0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.742+0000 7f7b6be00640 1 -- 172.21.15.19:0/585975526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b6c13f7e0 msgr2=0x7f7b6c1445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.742+0000 7f7b6be00640 1 --2- 172.21.15.19:0/585975526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b6c13f7e0 0x7f7b6c1445f0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00e550 tx=0x7f7b5c00ea10 comp rx=0 tx=0).stop 2024-03-28T13:20:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.743+0000 7f7b6be00640 1 -- 172.21.15.19:0/585975526 shutdown_connections 2024-03-28T13:20:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.743+0000 7f7b6be00640 1 --2- 172.21.15.19:0/585975526 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b4806c530 0x7f7b4806e9f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.743+0000 7f7b6be00640 1 --2- 172.21.15.19:0/585975526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b6c1f5990 0x7f7b6c0b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.743+0000 7f7b6be00640 1 --2- 172.21.15.19:0/585975526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b6c13fbc0 0x7f7b6c0b4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.743+0000 7f7b6be00640 1 --2- 172.21.15.19:0/585975526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b6c13f7e0 0x7f7b6c1445f0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.743+0000 7f7b6be00640 1 -- 172.21.15.19:0/585975526 >> 172.21.15.19:0/585975526 conn(0x7f7b6c0ccad0 msgr2=0x7f7b6c06d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.743+0000 7f7b6be00640 1 -- 172.21.15.19:0/585975526 shutdown_connections 2024-03-28T13:20:36.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.743+0000 7f7b6be00640 1 -- 172.21.15.19:0/585975526 wait complete. 2024-03-28T13:20:36.760 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:20:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.978+0000 7f49f9000640 1 Processor -- start 2024-03-28T13:20:36.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.979+0000 7f49f9000640 1 -- start start 2024-03-28T13:20:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.980+0000 7f49f9000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49f40d2310 0x7f49f40ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.980+0000 7f49f9000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f49f40cf530 0x7f49f40b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.980+0000 7f49f9000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49f415e880 con 0x7f49f415e4a0 2024-03-28T13:20:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.980+0000 7f49f9000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f49f40cf9a0 con 0x7f49f40d2310 2024-03-28T13:20:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.980+0000 7f49f9000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f49f40d34e0 con 0x7f49f40cf530 2024-03-28T13:20:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.981+0000 7f49f2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49f40d2310 0x7f49f40ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.981+0000 7f49f2a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49f40d2310 0x7f49f40ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52600/0 (socket says 172.21.15.19:52600) 2024-03-28T13:20:36.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.981+0000 7f49f2a00640 1 -- 172.21.15.19:0/807561781 learned_addr learned my addr 172.21.15.19:0/807561781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:36.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.981+0000 7f49f3e00640 1 --2- 172.21.15.19:0/807561781 >> v2:172.21.15.188:3300/0 conn(0x7f49f40cf530 0x7f49f40b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.981+0000 7f49f2a00640 1 -- 172.21.15.19:0/807561781 >> v2:172.21.15.188:3300/0 conn(0x7f49f40cf530 msgr2=0x7f49f40b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:36.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.981+0000 7f49f2a00640 1 --2- 172.21.15.19:0/807561781 >> v2:172.21.15.188:3300/0 conn(0x7f49f40cf530 0x7f49f40b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.981+0000 7f49f2a00640 1 -- 172.21.15.19:0/807561781 >> v1:172.21.15.188:6789/0 conn(0x7f49f415e4a0 legacy=0x7f49f40b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:36.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.981+0000 7f49f2a00640 1 -- 172.21.15.19:0/807561781 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f40d28d0 con 0x7f49f40d2310 2024-03-28T13:20:36.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.981+0000 7f49f3e00640 1 --2- 172.21.15.19:0/807561781 >> v2:172.21.15.188:3300/0 conn(0x7f49f40cf530 0x7f49f40b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:36.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.982+0000 7f49f2a00640 1 --2- 172.21.15.19:0/807561781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f40d2310 0x7f49f40ceff0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f49f40d3660 tx=0x7f49dc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca72c1ab3e6aafa server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:36.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.983+0000 7f49f2000640 1 -- 172.21.15.19:0/807561781 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc02c9e0 con 0x7f49f40d2310 2024-03-28T13:20:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.983+0000 7f49f2000640 1 -- 172.21.15.19:0/807561781 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49dc02cb80 con 0x7f49f40d2310 2024-03-28T13:20:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.983+0000 7f49f2000640 1 -- 172.21.15.19:0/807561781 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc0343f0 con 0x7f49f40d2310 2024-03-28T13:20:36.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.983+0000 7f49f9000640 1 -- 172.21.15.19:0/807561781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f40d2310 msgr2=0x7f49f40ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:36.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.983+0000 7f49f9000640 1 --2- 172.21.15.19:0/807561781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f40d2310 0x7f49f40ceff0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f49f40d3660 tx=0x7f49dc02be40 comp rx=0 tx=0).stop 2024-03-28T13:20:36.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.983+0000 7f49f9000640 1 -- 172.21.15.19:0/807561781 shutdown_connections 2024-03-28T13:20:36.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.984+0000 7f49f9000640 1 --2- 172.21.15.19:0/807561781 >> v2:172.21.15.188:3300/0 conn(0x7f49f40cf530 0x7f49f40b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.984+0000 7f49f9000640 1 --2- 172.21.15.19:0/807561781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f40d2310 0x7f49f40ceff0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.984+0000 7f49f9000640 1 -- 172.21.15.19:0/807561781 >> 172.21.15.19:0/807561781 conn(0x7f49f40676d0 msgr2=0x7f49f4067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:36.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.984+0000 7f49f9000640 1 -- 172.21.15.19:0/807561781 shutdown_connections 2024-03-28T13:20:36.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.984+0000 7f49f9000640 1 -- 172.21.15.19:0/807561781 wait complete. 2024-03-28T13:20:36.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.985+0000 7f49f9000640 1 Processor -- start 2024-03-28T13:20:36.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.985+0000 7f49f9000640 1 -- start start 2024-03-28T13:20:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.986+0000 7f49f9000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49f40cf530 0x7f49f41f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.986+0000 7f49f9000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f40d2310 0x7f49f41f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.986+0000 7f49f9000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f415e4a0 0x7f49f4157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.986+0000 7f49f9000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49f40d3170 con 0x7f49f40d2310 2024-03-28T13:20:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.986+0000 7f49f9000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49f40d2e90 con 0x7f49f415e4a0 2024-03-28T13:20:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.986+0000 7f49f9000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f49f40d3970 con 0x7f49f40cf530 2024-03-28T13:20:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.987+0000 7f49f2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f40d2310 0x7f49f41f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.987+0000 7f49f2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f40d2310 0x7f49f41f6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52604/0 (socket says 172.21.15.19:52604) 2024-03-28T13:20:36.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.987+0000 7f49f2a00640 1 -- 172.21.15.19:0/4180878485 learned_addr learned my addr 172.21.15.19:0/4180878485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:36.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.987+0000 7f49f3e00640 1 --2- 172.21.15.19:0/4180878485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f415e4a0 0x7f49f4157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.987+0000 7f49f2a00640 1 -- 172.21.15.19:0/4180878485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49f40cf530 msgr2=0x7f49f41f5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:36.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.987+0000 7f49f2a00640 1 --2- 172.21.15.19:0/4180878485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49f40cf530 0x7f49f41f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.987+0000 7f49f2a00640 1 -- 172.21.15.19:0/4180878485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f415e4a0 msgr2=0x7f49f4157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:36.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.987+0000 7f49f2a00640 1 --2- 172.21.15.19:0/4180878485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f415e4a0 0x7f49f4157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:36.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.987+0000 7f49f2a00640 1 -- 172.21.15.19:0/4180878485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f415f900 con 0x7f49f40d2310 2024-03-28T13:20:36.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.987+0000 7f49f3e00640 1 --2- 172.21.15.19:0/4180878485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f415e4a0 0x7f49f4157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:36.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.987+0000 7f49f2a00640 1 --2- 172.21.15.19:0/4180878485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f40d2310 0x7f49f41f6f00 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009f10 tx=0x7f49dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:36.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.988+0000 7f49eb400640 1 -- 172.21.15.19:0/4180878485 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc043020 con 0x7f49f40d2310 2024-03-28T13:20:36.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.988+0000 7f49eb400640 1 -- 172.21.15.19:0/4180878485 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49dc03e450 con 0x7f49f40d2310 2024-03-28T13:20:36.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.988+0000 7f49f9000640 1 -- 172.21.15.19:0/4180878485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f4155440 con 0x7f49f40d2310 2024-03-28T13:20:36.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.988+0000 7f49eb400640 1 -- 172.21.15.19:0/4180878485 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc009510 con 0x7f49f40d2310 2024-03-28T13:20:36.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.988+0000 7f49f9000640 1 -- 172.21.15.19:0/4180878485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f41f7440 con 0x7f49f40d2310 2024-03-28T13:20:36.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.989+0000 7f49f9000640 1 -- 172.21.15.19:0/4180878485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f4075940 con 0x7f49f40d2310 2024-03-28T13:20:36.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.990+0000 7f49eb400640 1 -- 172.21.15.19:0/4180878485 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f49dc046020 con 0x7f49f40d2310 2024-03-28T13:20:36.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.991+0000 7f49eb400640 1 --2- 172.21.15.19:0/4180878485 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49d006c560 0x7f49d006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:36.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.991+0000 7f49eb400640 1 -- 172.21.15.19:0/4180878485 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5939+0+0 (secure 0 0 0) 0x7f49dc0adfa0 con 0x7f49f40d2310 2024-03-28T13:20:36.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.992+0000 7f49f3400640 1 --2- 172.21.15.19:0/4180878485 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49d006c560 0x7f49d006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:36.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.993+0000 7f49f3400640 1 --2- 172.21.15.19:0/4180878485 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49d006c560 0x7f49d006ea20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f49e4006fd0 tx=0x7f49e4006bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:36.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:36.996+0000 7f49eb400640 1 -- 172.21.15.19:0/4180878485 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f49dc04d070 con 0x7f49f40d2310 2024-03-28T13:20:37.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.189+0000 7f49f9000640 1 -- 172.21.15.19:0/4180878485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49f40d4950 con 0x7f49f40d2310 2024-03-28T13:20:37.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.192+0000 7f49eb400640 1 -- 172.21.15.19:0/4180878485 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16975 (secure 0 0 0) 0x7f49dc07c430 con 0x7f49f40d2310 2024-03-28T13:20:37.192 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:37.192 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":109,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:20:33.234662+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:18:56.432417+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-28T13:18:57.540515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.c","mappings":[{"from":5,"to":0}]},{"pgid":"14.18","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/2851646201":"2024-03-29T13:20:33.234624+0000","172.21.15.19:6832/2851646201":"2024-03-29T13:20:33.234624+0000","172.21.15.19:6834/1020599987":"2024-03-29T13:19:55.818562+0000","172.21.15.19:6832/1020599987":"2024-03-29T13:19:55.818562+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:20:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.195+0000 7f49f9000640 1 -- 172.21.15.19:0/4180878485 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49d006c560 msgr2=0x7f49d006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.195+0000 7f49f9000640 1 --2- 172.21.15.19:0/4180878485 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49d006c560 0x7f49d006ea20 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f49e4006fd0 tx=0x7f49e4006bc0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.195+0000 7f49f9000640 1 -- 172.21.15.19:0/4180878485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f40d2310 msgr2=0x7f49f41f6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.195+0000 7f49f9000640 1 --2- 172.21.15.19:0/4180878485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f40d2310 0x7f49f41f6f00 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009f10 tx=0x7f49dc004a40 comp rx=0 tx=0).stop 2024-03-28T13:20:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.195+0000 7f49f9000640 1 -- 172.21.15.19:0/4180878485 shutdown_connections 2024-03-28T13:20:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.195+0000 7f49f9000640 1 --2- 172.21.15.19:0/4180878485 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f49d006c560 0x7f49d006ea20 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.195+0000 7f49f9000640 1 --2- 172.21.15.19:0/4180878485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49f415e4a0 0x7f49f4157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.195+0000 7f49f9000640 1 --2- 172.21.15.19:0/4180878485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49f40d2310 0x7f49f41f6f00 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.195+0000 7f49f9000640 1 --2- 172.21.15.19:0/4180878485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49f40cf530 0x7f49f41f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.195+0000 7f49f9000640 1 -- 172.21.15.19:0/4180878485 >> 172.21.15.19:0/4180878485 conn(0x7f49f40676d0 msgr2=0x7f49f40b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:37.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.195+0000 7f49f9000640 1 -- 172.21.15.19:0/4180878485 shutdown_connections 2024-03-28T13:20:37.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.195+0000 7f49f9000640 1 -- 172.21.15.19:0/4180878485 wait complete. 2024-03-28T13:20:37.213 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-28T13:20:37.214 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:20:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.427+0000 7f5063e00640 1 Processor -- start 2024-03-28T13:20:37.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.428+0000 7f5063e00640 1 -- start start 2024-03-28T13:20:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.429+0000 7f5063e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f505c0af830 0x7f505c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.429+0000 7f5063e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f505c0d0650 0x7f505c0d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.429+0000 7f5063e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f505c15fe80 con 0x7f505c15c390 2024-03-28T13:20:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.429+0000 7f5063e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f505c15c770 con 0x7f505c0d0650 2024-03-28T13:20:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.429+0000 7f5063e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f505c0d0fc0 con 0x7f505c0af830 2024-03-28T13:20:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.430+0000 7f5061e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f505c0d0650 0x7f505c0d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:37.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.430+0000 7f5061e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f505c0d0650 0x7f505c0d0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52612/0 (socket says 172.21.15.19:52612) 2024-03-28T13:20:37.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.430+0000 7f5061e00640 1 -- 172.21.15.19:0/665597676 learned_addr learned my addr 172.21.15.19:0/665597676 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:37.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.430+0000 7f5061400640 1 --1- 172.21.15.19:0/665597676 >> v1:172.21.15.19:6789/0 conn(0x7f505c15c390 0x7f505c160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47646/0 (socket says 172.21.15.19:47646) 2024-03-28T13:20:37.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.430+0000 7f5060a00640 1 --2- 172.21.15.19:0/665597676 >> v2:172.21.15.188:3300/0 conn(0x7f505c0af830 0x7f505c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:37.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.430+0000 7f5061e00640 1 -- 172.21.15.19:0/665597676 >> v2:172.21.15.188:3300/0 conn(0x7f505c0af830 msgr2=0x7f505c073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:37.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.430+0000 7f5061e00640 1 --2- 172.21.15.19:0/665597676 >> v2:172.21.15.188:3300/0 conn(0x7f505c0af830 0x7f505c073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.430+0000 7f5061e00640 1 -- 172.21.15.19:0/665597676 >> v1:172.21.15.19:6789/0 conn(0x7f505c15c390 legacy=0x7f505c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.430+0000 7f5061e00640 1 -- 172.21.15.19:0/665597676 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505c0ad120 con 0x7f505c0d0650 2024-03-28T13:20:37.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.430+0000 7f5060a00640 1 --2- 172.21.15.19:0/665597676 >> v2:172.21.15.188:3300/0 conn(0x7f505c0af830 0x7f505c073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.431+0000 7f5061e00640 1 --2- 172.21.15.19:0/665597676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f505c0d0650 0x7f505c0d0a80 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f505c06eaa0 tx=0x7f505402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92372e3a809dadef server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.432+0000 7f505be00640 1 -- 172.21.15.19:0/665597676 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505402cb10 con 0x7f505c0d0650 2024-03-28T13:20:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.432+0000 7f505be00640 1 -- 172.21.15.19:0/665597676 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f505402ccb0 con 0x7f505c0d0650 2024-03-28T13:20:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.432+0000 7f505be00640 1 -- 172.21.15.19:0/665597676 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505402ce60 con 0x7f505c0d0650 2024-03-28T13:20:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.432+0000 7f5063e00640 1 -- 172.21.15.19:0/665597676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f505c0d0650 msgr2=0x7f505c0d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.432+0000 7f5063e00640 1 --2- 172.21.15.19:0/665597676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f505c0d0650 0x7f505c0d0a80 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f505c06eaa0 tx=0x7f505402bf70 comp rx=0 tx=0).stop 2024-03-28T13:20:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.432+0000 7f5063e00640 1 -- 172.21.15.19:0/665597676 shutdown_connections 2024-03-28T13:20:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.432+0000 7f5063e00640 1 --2- 172.21.15.19:0/665597676 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f505c0d0650 0x7f505c0d0a80 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.432+0000 7f5063e00640 1 --2- 172.21.15.19:0/665597676 >> v2:172.21.15.188:3300/0 conn(0x7f505c0af830 0x7f505c073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.432+0000 7f5063e00640 1 -- 172.21.15.19:0/665597676 >> 172.21.15.19:0/665597676 conn(0x7f505c06b990 msgr2=0x7f505c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.432+0000 7f5063e00640 1 -- 172.21.15.19:0/665597676 shutdown_connections 2024-03-28T13:20:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.432+0000 7f5063e00640 1 -- 172.21.15.19:0/665597676 wait complete. 2024-03-28T13:20:37.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.434+0000 7f5063e00640 1 Processor -- start 2024-03-28T13:20:37.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.434+0000 7f5063e00640 1 -- start start 2024-03-28T13:20:37.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.435+0000 7f5063e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f505c0af830 0x7f505c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:37.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.435+0000 7f5063e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f505c0d0650 0x7f505c1f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:37.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.435+0000 7f5063e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f505c15c390 0x7f505c0b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:37.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.435+0000 7f5063e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f505c160120 con 0x7f505c15c390 2024-03-28T13:20:37.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.435+0000 7f5063e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f505c0ad6e0 con 0x7f505c0af830 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.435+0000 7f5063e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f505c160d50 con 0x7f505c0d0650 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.436+0000 7f5061e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f505c15c390 0x7f505c0b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.436+0000 7f5061e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f505c15c390 0x7f505c0b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52614/0 (socket says 172.21.15.19:52614) 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.436+0000 7f5061e00640 1 -- 172.21.15.19:0/4064356333 learned_addr learned my addr 172.21.15.19:0/4064356333 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.436+0000 7f5061400640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f505c0af830 0x7f505c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.436+0000 7f5060a00640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f505c0d0650 0x7f505c1f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.436+0000 7f5061e00640 1 -- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f505c0d0650 msgr2=0x7f505c1f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.436+0000 7f5061e00640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f505c0d0650 0x7f505c1f79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.436+0000 7f5061e00640 1 -- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f505c0af830 msgr2=0x7f505c15d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.436+0000 7f5061e00640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f505c0af830 0x7f505c15d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.436+0000 7f5061e00640 1 -- 172.21.15.19:0/4064356333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505c1f8340 con 0x7f505c15c390 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.436+0000 7f5061400640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f505c0af830 0x7f505c15d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.436+0000 7f5060a00640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f505c0d0650 0x7f505c1f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.436+0000 7f5061e00640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f505c15c390 0x7f505c0b1530 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f5054036040 tx=0x7f5054004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.437+0000 7f5059600640 1 -- 172.21.15.19:0/4064356333 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5054043020 con 0x7f505c15c390 2024-03-28T13:20:37.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.437+0000 7f5059600640 1 -- 172.21.15.19:0/4064356333 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f505403e450 con 0x7f505c15c390 2024-03-28T13:20:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.437+0000 7f5059600640 1 -- 172.21.15.19:0/4064356333 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f505402ce60 con 0x7f505c15c390 2024-03-28T13:20:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.437+0000 7f5063e00640 1 -- 172.21.15.19:0/4064356333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505c1f8620 con 0x7f505c15c390 2024-03-28T13:20:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.437+0000 7f5063e00640 1 -- 172.21.15.19:0/4064356333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f505c15cd80 con 0x7f505c15c390 2024-03-28T13:20:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.438+0000 7f5063e00640 1 -- 172.21.15.19:0/4064356333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f505c0685c0 con 0x7f505c15c390 2024-03-28T13:20:37.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.439+0000 7f5059600640 1 -- 172.21.15.19:0/4064356333 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5054046020 con 0x7f505c15c390 2024-03-28T13:20:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.440+0000 7f5059600640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f503c06c530 0x7f503c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:37.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.440+0000 7f5059600640 1 -- 172.21.15.19:0/4064356333 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5939+0+0 (secure 0 0 0) 0x7f50540ae090 con 0x7f505c15c390 2024-03-28T13:20:37.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.441+0000 7f5061400640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f503c06c530 0x7f503c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:37.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.442+0000 7f5061400640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f503c06c530 0x7f503c06e9f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f504c000f60 tx=0x7f504c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:37.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.445+0000 7f5059600640 1 -- 172.21.15.19:0/4064356333 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f505404d030 con 0x7f505c15c390 2024-03-28T13:20:37.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.686+0000 7f5063e00640 1 -- 172.21.15.19:0/4064356333 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f505c06f1b0 con 0x7f505c15c390 2024-03-28T13:20:37.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.687+0000 7f5059600640 1 -- 172.21.15.19:0/4064356333 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5176 (secure 0 0 0) 0x7f505407c520 con 0x7f505c15c390 2024-03-28T13:20:37.686 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:37.687 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4247326257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4247326257},{"type":"v1","addr":"172.21.15.188:6836","nonce":4247326257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3452274156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3452274156},{"type":"v1","addr":"172.21.15.19:6835","nonce":3452274156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3383190607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3383190607},{"type":"v1","addr":"172.21.15.188:6837","nonce":3383190607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3193953635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3193953635},{"type":"v1","addr":"172.21.15.19:6834","nonce":3193953635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:18:58.541599+0000","modified":"2024-03-28T13:20:33.234961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-03-28T13:20:37.687 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 90 2024-03-28T13:20:37.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.689+0000 7f5063e00640 1 -- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f503c06c530 msgr2=0x7f503c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:37.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.689+0000 7f5063e00640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f503c06c530 0x7f503c06e9f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f504c000f60 tx=0x7f504c009040 comp rx=0 tx=0).stop 2024-03-28T13:20:37.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.689+0000 7f5063e00640 1 -- 172.21.15.19:0/4064356333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f505c15c390 msgr2=0x7f505c0b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:37.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.689+0000 7f5063e00640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f505c15c390 0x7f505c0b1530 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f5054036040 tx=0x7f5054004a40 comp rx=0 tx=0).stop 2024-03-28T13:20:37.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.689+0000 7f5063e00640 1 -- 172.21.15.19:0/4064356333 shutdown_connections 2024-03-28T13:20:37.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.689+0000 7f5063e00640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f503c06c530 0x7f503c06e9f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.689+0000 7f5063e00640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f505c15c390 0x7f505c0b1530 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.689+0000 7f5063e00640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f505c0d0650 0x7f505c1f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.689+0000 7f5063e00640 1 --2- 172.21.15.19:0/4064356333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f505c0af830 0x7f505c15d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.689+0000 7f5063e00640 1 -- 172.21.15.19:0/4064356333 >> 172.21.15.19:0/4064356333 conn(0x7f505c06b990 msgr2=0x7f505c0aefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:37.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.690+0000 7f5063e00640 1 -- 172.21.15.19:0/4064356333 shutdown_connections 2024-03-28T13:20:37.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.690+0000 7f5063e00640 1 -- 172.21.15.19:0/4064356333 wait complete. 2024-03-28T13:20:37.708 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:20:37.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.927+0000 7fbff3e00640 1 Processor -- start 2024-03-28T13:20:37.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.927+0000 7fbff3e00640 1 -- start start 2024-03-28T13:20:37.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.929+0000 7fbff3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbff40d0390 0x7fbff40cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:37.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.929+0000 7fbff3e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbff40ce1b0 0x7fbff406da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:37.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.929+0000 7fbff3e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbff4161f30 con 0x7fbff4147a40 2024-03-28T13:20:37.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.929+0000 7fbff3e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fbff4162b00 con 0x7fbff40ce1b0 2024-03-28T13:20:37.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.929+0000 7fbff3e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbff41621d0 con 0x7fbff40d0390 2024-03-28T13:20:37.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.929+0000 7fbff2a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fbff4147a40 0x7fbff40cfc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47654/0 (socket says 172.21.15.19:47654) 2024-03-28T13:20:37.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.929+0000 7fbff2a00640 1 -- 172.21.15.19:0/3539060446 learned_addr learned my addr 172.21.15.19:0/3539060446 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff3400640 1 --2- 172.21.15.19:0/3539060446 >> v2:172.21.15.188:3300/0 conn(0x7fbff40ce1b0 0x7fbff406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 -- 172.21.15.19:0/3539060446 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1069660187 0 0) 0x7fbff4161f30 con 0x7fbff4147a40 2024-03-28T13:20:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 -- 172.21.15.19:0/3539060446 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfd8003520 con 0x7fbff4147a40 2024-03-28T13:20:37.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbfea000640 1 --2- 172.21.15.19:0/3539060446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbff40d0390 0x7fbff40cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 -- 172.21.15.19:0/3539060446 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 371439665 0 0) 0x7fbfd8003520 con 0x7fbff4147a40 2024-03-28T13:20:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 -- 172.21.15.19:0/3539060446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbff40d0390 msgr2=0x7fbff40cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 --2- 172.21.15.19:0/3539060446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbff40d0390 0x7fbff40cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 -- 172.21.15.19:0/3539060446 >> v2:172.21.15.188:3300/0 conn(0x7fbff40ce1b0 msgr2=0x7fbff406da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 --2- 172.21.15.19:0/3539060446 >> v2:172.21.15.188:3300/0 conn(0x7fbff40ce1b0 0x7fbff406da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 -- 172.21.15.19:0/3539060446 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff41624b0 con 0x7fbff4147a40 2024-03-28T13:20:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 -- 172.21.15.19:0/3539060446 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fbfe4001690 con 0x7fbff4147a40 2024-03-28T13:20:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 -- 172.21.15.19:0/3539060446 >> v1:172.21.15.19:6789/0 conn(0x7fbff4147a40 legacy=0x7fbff40cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 --2- 172.21.15.19:0/3539060446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbff40d0390 0x7fbfd80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 --2- 172.21.15.19:0/3539060446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd8003f80 0x7fbfd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 --2- 172.21.15.19:0/3539060446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfd8004d40 0x7fbfd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 -- 172.21.15.19:0/3539060446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbff4162b00 con 0x7fbfd8003f80 2024-03-28T13:20:37.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 -- 172.21.15.19:0/3539060446 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbff41621d0 con 0x7fbfd8004d40 2024-03-28T13:20:37.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.930+0000 7fbff2000640 1 -- 172.21.15.19:0/3539060446 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd800ba70 con 0x7fbff40d0390 2024-03-28T13:20:37.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.931+0000 7fbfea000640 1 --2- 172.21.15.19:0/3539060446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd8003f80 0x7fbfd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:37.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.931+0000 7fbff2a00640 1 --2- 172.21.15.19:0/3539060446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbff40d0390 0x7fbfd80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:37.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.931+0000 7fbff3400640 1 --2- 172.21.15.19:0/3539060446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfd8004d40 0x7fbfd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:37.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.931+0000 7fbfea000640 1 -- 172.21.15.19:0/3539060446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbff40d0390 msgr2=0x7fbfd80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:37.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.931+0000 7fbfea000640 1 --2- 172.21.15.19:0/3539060446 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbff40d0390 0x7fbfd80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.931+0000 7fbfea000640 1 -- 172.21.15.19:0/3539060446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfd8004d40 msgr2=0x7fbfd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:37.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.931+0000 7fbfea000640 1 --2- 172.21.15.19:0/3539060446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfd8004d40 0x7fbfd80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.931+0000 7fbfea000640 1 -- 172.21.15.19:0/3539060446 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd8003520 con 0x7fbfd8003f80 2024-03-28T13:20:37.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.931+0000 7fbff3400640 1 --2- 172.21.15.19:0/3539060446 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbfd8004d40 0x7fbfd80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:37.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.932+0000 7fbfea000640 1 --2- 172.21.15.19:0/3539060446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd8003f80 0x7fbfd8004800 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00ba30 tx=0x7fbfd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa66dbc304444a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:37.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.933+0000 7fbff2000640 1 -- 172.21.15.19:0/3539060446 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc036020 con 0x7fbfd8003f80 2024-03-28T13:20:37.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.933+0000 7fbff2000640 1 -- 172.21.15.19:0/3539060446 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfdc032640 con 0x7fbfd8003f80 2024-03-28T13:20:37.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.933+0000 7fbff2000640 1 -- 172.21.15.19:0/3539060446 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfdc00b600 con 0x7fbfd8003f80 2024-03-28T13:20:37.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.933+0000 7fbff3e00640 1 -- 172.21.15.19:0/3539060446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd8003f80 msgr2=0x7fbfd8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.933+0000 7fbff3e00640 1 --2- 172.21.15.19:0/3539060446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbfd8003f80 0x7fbfd8004800 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc00ba30 tx=0x7fbfd8005830 comp rx=0 tx=0).stop 2024-03-28T13:20:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.933+0000 7fbff3400640 1 -- 172.21.15.19:0/3539060446 reap_dead start 2024-03-28T13:20:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.933+0000 7fbff3e00640 1 -- 172.21.15.19:0/3539060446 shutdown_connections 2024-03-28T13:20:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.933+0000 7fbff3e00640 1 -- 172.21.15.19:0/3539060446 >> 172.21.15.19:0/3539060446 conn(0x7fbff40ccad0 msgr2=0x7fbff40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.933+0000 7fbff3e00640 1 -- 172.21.15.19:0/3539060446 shutdown_connections 2024-03-28T13:20:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.933+0000 7fbff3e00640 1 -- 172.21.15.19:0/3539060446 wait complete. 2024-03-28T13:20:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.935+0000 7fbff3e00640 1 Processor -- start 2024-03-28T13:20:37.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.935+0000 7fbff3e00640 1 -- start start 2024-03-28T13:20:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.936+0000 7fbff3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd8003f80 0x7fbff415dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.936+0000 7fbff3e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbff4141b80 0x7fbff40b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.936+0000 7fbff3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff40b5070 0x7fbff40b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.936+0000 7fbff3e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbff4147fa0 con 0x7fbff40b5070 2024-03-28T13:20:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.936+0000 7fbff3e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbff4147e20 con 0x7fbff4141b80 2024-03-28T13:20:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.936+0000 7fbff3e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbff40ce640 con 0x7fbfd8003f80 2024-03-28T13:20:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.936+0000 7fbff3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff40b5070 0x7fbff40b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:37.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.937+0000 7fbff3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff40b5070 0x7fbff40b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52624/0 (socket says 172.21.15.19:52624) 2024-03-28T13:20:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.937+0000 7fbff3400640 1 -- 172.21.15.19:0/1026159692 learned_addr learned my addr 172.21.15.19:0/1026159692 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.937+0000 7fbfea000640 1 --2- 172.21.15.19:0/1026159692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbff4141b80 0x7fbff40b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.937+0000 7fbff2a00640 1 --2- 172.21.15.19:0/1026159692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd8003f80 0x7fbff415dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.937+0000 7fbff3400640 1 -- 172.21.15.19:0/1026159692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd8003f80 msgr2=0x7fbff415dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.937+0000 7fbff3400640 1 --2- 172.21.15.19:0/1026159692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd8003f80 0x7fbff415dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.937+0000 7fbff3400640 1 -- 172.21.15.19:0/1026159692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbff4141b80 msgr2=0x7fbff40b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.937+0000 7fbff3400640 1 --2- 172.21.15.19:0/1026159692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbff4141b80 0x7fbff40b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.937+0000 7fbff3400640 1 -- 172.21.15.19:0/1026159692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff415e130 con 0x7fbff40b5070 2024-03-28T13:20:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.937+0000 7fbfea000640 1 --2- 172.21.15.19:0/1026159692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbff4141b80 0x7fbff40b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.937+0000 7fbff3400640 1 --2- 172.21.15.19:0/1026159692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff40b5070 0x7fbff40b5520 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fbff40d09c0 tx=0x7fbfec00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.937+0000 7fbfeb400640 1 -- 172.21.15.19:0/1026159692 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec01c070 con 0x7fbff40b5070 2024-03-28T13:20:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.938+0000 7fbfeb400640 1 -- 172.21.15.19:0/1026159692 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfec017400 con 0x7fbff40b5070 2024-03-28T13:20:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.938+0000 7fbfeb400640 1 -- 172.21.15.19:0/1026159692 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0177d0 con 0x7fbff40b5070 2024-03-28T13:20:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.938+0000 7fbff3e00640 1 -- 172.21.15.19:0/1026159692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff40b3560 con 0x7fbff40b5070 2024-03-28T13:20:37.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.938+0000 7fbff3e00640 1 -- 172.21.15.19:0/1026159692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff40b3b20 con 0x7fbff40b5070 2024-03-28T13:20:37.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.939+0000 7fbff3e00640 1 -- 172.21.15.19:0/1026159692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff40cd9b0 con 0x7fbff40b5070 2024-03-28T13:20:37.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.940+0000 7fbfeb400640 1 -- 172.21.15.19:0/1026159692 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbfec01f070 con 0x7fbff40b5070 2024-03-28T13:20:37.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.940+0000 7fbfeb400640 1 --2- 172.21.15.19:0/1026159692 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbfd006c560 0x7fbfd006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:37.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.941+0000 7fbfeb400640 1 -- 172.21.15.19:0/1026159692 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5939+0+0 (secure 0 0 0) 0x7fbfec08b8e0 con 0x7fbff40b5070 2024-03-28T13:20:37.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.941+0000 7fbff2a00640 1 --2- 172.21.15.19:0/1026159692 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbfd006c560 0x7fbfd006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:37.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.942+0000 7fbff2a00640 1 --2- 172.21.15.19:0/1026159692 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbfd006c560 0x7fbfd006ea20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4001eb0 tx=0x7fbfe4001150 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:37.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:37.945+0000 7fbfeb400640 1 -- 172.21.15.19:0/1026159692 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbff40cd9b0 con 0x7fbff40b5070 2024-03-28T13:20:38.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.193+0000 7fbff3e00640 1 -- 172.21.15.19:0/1026159692 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbff4075940 con 0x7fbff40b5070 2024-03-28T13:20:38.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.195+0000 7fbfeb400640 1 -- 172.21.15.19:0/1026159692 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5176 (secure 0 0 0) 0x7fbff415e130 con 0x7fbff40b5070 2024-03-28T13:20:38.195 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:38.195 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":90,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4247326257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4247326257},{"type":"v1","addr":"172.21.15.188:6836","nonce":4247326257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3452274156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3452274156},{"type":"v1","addr":"172.21.15.19:6835","nonce":3452274156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3383190607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3383190607},{"type":"v1","addr":"172.21.15.188:6837","nonce":3383190607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3193953635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3193953635},{"type":"v1","addr":"172.21.15.19:6834","nonce":3193953635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":89,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:18:58.541599+0000","modified":"2024-03-28T13:20:33.234961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-03-28T13:20:38.195 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 90 2024-03-28T13:20:38.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.197+0000 7fbff3e00640 1 -- 172.21.15.19:0/1026159692 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbfd006c560 msgr2=0x7fbfd006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:38.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.197+0000 7fbff3e00640 1 --2- 172.21.15.19:0/1026159692 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbfd006c560 0x7fbfd006ea20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4001eb0 tx=0x7fbfe4001150 comp rx=0 tx=0).stop 2024-03-28T13:20:38.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.197+0000 7fbff3e00640 1 -- 172.21.15.19:0/1026159692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff40b5070 msgr2=0x7fbff40b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:38.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.197+0000 7fbff3e00640 1 --2- 172.21.15.19:0/1026159692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff40b5070 0x7fbff40b5520 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fbff40d09c0 tx=0x7fbfec00f4a0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.197+0000 7fbff3e00640 1 -- 172.21.15.19:0/1026159692 shutdown_connections 2024-03-28T13:20:38.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.197+0000 7fbff3e00640 1 --2- 172.21.15.19:0/1026159692 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbfd006c560 0x7fbfd006ea20 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.197+0000 7fbff3e00640 1 --2- 172.21.15.19:0/1026159692 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff40b5070 0x7fbff40b5520 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.197+0000 7fbff3e00640 1 --2- 172.21.15.19:0/1026159692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbff4141b80 0x7fbff40b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.197+0000 7fbff3e00640 1 --2- 172.21.15.19:0/1026159692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbfd8003f80 0x7fbff415dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.197+0000 7fbff3e00640 1 -- 172.21.15.19:0/1026159692 >> 172.21.15.19:0/1026159692 conn(0x7fbff40ccad0 msgr2=0x7fbff41422c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:38.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.198+0000 7fbff3e00640 1 -- 172.21.15.19:0/1026159692 shutdown_connections 2024-03-28T13:20:38.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.198+0000 7fbff3e00640 1 -- 172.21.15.19:0/1026159692 wait complete. 2024-03-28T13:20:38.217 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:20:38.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.442+0000 7f8fa9c00640 1 Processor -- start 2024-03-28T13:20:38.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.442+0000 7f8fa9c00640 1 -- start start 2024-03-28T13:20:38.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.444+0000 7f8fa9c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8fa40d0390 0x7f8fa40cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:38.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.444+0000 7f8fa9c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fa4161f30 con 0x7f8fa40ce1b0 2024-03-28T13:20:38.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.444+0000 7f8fa9c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fa41621d0 con 0x7f8fa4147a40 2024-03-28T13:20:38.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.444+0000 7f8fa9c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f8fa4162b00 con 0x7f8fa40d0390 2024-03-28T13:20:38.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.444+0000 7f8fa9200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8fa40ce1b0 0x7f8fa406da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47656/0 (socket says 172.21.15.19:47656) 2024-03-28T13:20:38.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.444+0000 7f8fa9200640 1 -- 172.21.15.19:0/148740354 learned_addr learned my addr 172.21.15.19:0/148740354 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa3400640 1 --2- 172.21.15.19:0/148740354 >> v2:172.21.15.188:3300/0 conn(0x7f8fa40d0390 0x7f8fa40cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa2a00640 1 -- 172.21.15.19:0/148740354 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3627057220 0 0) 0x7f8fa4161f30 con 0x7f8fa40ce1b0 2024-03-28T13:20:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa2a00640 1 -- 172.21.15.19:0/148740354 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f88003520 con 0x7f8fa40ce1b0 2024-03-28T13:20:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa2a00640 1 -- 172.21.15.19:0/148740354 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3861392707 0 0) 0x7f8f88003520 con 0x7f8fa40ce1b0 2024-03-28T13:20:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa2a00640 1 -- 172.21.15.19:0/148740354 >> v2:172.21.15.188:3300/0 conn(0x7f8fa40d0390 msgr2=0x7f8fa40cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa2a00640 1 --2- 172.21.15.19:0/148740354 >> v2:172.21.15.188:3300/0 conn(0x7f8fa40d0390 0x7f8fa40cdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa2a00640 1 -- 172.21.15.19:0/148740354 >> v1:172.21.15.188:6789/0 conn(0x7f8fa4147a40 legacy=0x7f8fa40cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa2a00640 1 -- 172.21.15.19:0/148740354 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa41624b0 con 0x7f8fa40ce1b0 2024-03-28T13:20:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa2a00640 1 -- 172.21.15.19:0/148740354 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f8f9c001a10 con 0x7f8fa40ce1b0 2024-03-28T13:20:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa2a00640 1 -- 172.21.15.19:0/148740354 >> v1:172.21.15.19:6789/0 conn(0x7f8fa40ce1b0 legacy=0x7f8fa406da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa2a00640 1 --2- 172.21.15.19:0/148740354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f88003a00 0x7f8f88003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa2a00640 1 --2- 172.21.15.19:0/148740354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f88004800 0x7f8f88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa2a00640 1 --2- 172.21.15.19:0/148740354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f88005210 0x7f8f8800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa2a00640 1 -- 172.21.15.19:0/148740354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa4162b00 con 0x7f8f88004800 2024-03-28T13:20:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa2a00640 1 -- 172.21.15.19:0/148740354 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8800c160 con 0x7f8f88005210 2024-03-28T13:20:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.445+0000 7f8fa2a00640 1 -- 172.21.15.19:0/148740354 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8f8800c360 con 0x7f8f88003a00 2024-03-28T13:20:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.446+0000 7f8fa3400640 1 --2- 172.21.15.19:0/148740354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f88004800 0x7f8f88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.446+0000 7f8fa3e00640 1 --2- 172.21.15.19:0/148740354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f88003a00 0x7f8f88003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.446+0000 7f8fa3400640 1 -- 172.21.15.19:0/148740354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f88003a00 msgr2=0x7f8f88003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.446+0000 7f8fa9200640 1 --2- 172.21.15.19:0/148740354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f88005210 0x7f8f8800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:38.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.446+0000 7f8fa3400640 1 --2- 172.21.15.19:0/148740354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f88003a00 0x7f8f88003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.446+0000 7f8fa3400640 1 -- 172.21.15.19:0/148740354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f88005210 msgr2=0x7f8f8800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.446+0000 7f8fa3400640 1 --2- 172.21.15.19:0/148740354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f88005210 0x7f8f8800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.446+0000 7f8fa3400640 1 -- 172.21.15.19:0/148740354 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f88003520 con 0x7f8f88004800 2024-03-28T13:20:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.446+0000 7f8fa9200640 1 --2- 172.21.15.19:0/148740354 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f88005210 0x7f8f8800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.446+0000 7f8fa3e00640 1 --2- 172.21.15.19:0/148740354 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f88003a00 0x7f8f88003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.446+0000 7f8fa9200640 1 -- 172.21.15.19:0/148740354 reap_dead start 2024-03-28T13:20:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.447+0000 7f8fa3400640 1 --2- 172.21.15.19:0/148740354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f88004800 0x7f8f88004c80 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000ba30 tx=0x7f8f88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9156035b86972b92 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.447+0000 7f8fa2a00640 1 -- 172.21.15.19:0/148740354 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90036020 con 0x7f8f88004800 2024-03-28T13:20:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.447+0000 7f8fa2a00640 1 -- 172.21.15.19:0/148740354 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f90032640 con 0x7f8f88004800 2024-03-28T13:20:38.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.447+0000 7f8fa2a00640 1 -- 172.21.15.19:0/148740354 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9000b600 con 0x7f8f88004800 2024-03-28T13:20:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.448+0000 7f8fa9c00640 1 -- 172.21.15.19:0/148740354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f88004800 msgr2=0x7f8f88004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.448+0000 7f8fa9c00640 1 --2- 172.21.15.19:0/148740354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f88004800 0x7f8f88004c80 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000ba30 tx=0x7f8f88005830 comp rx=0 tx=0).stop 2024-03-28T13:20:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.448+0000 7f8fa9c00640 1 -- 172.21.15.19:0/148740354 shutdown_connections 2024-03-28T13:20:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.448+0000 7f8fa9c00640 1 --2- 172.21.15.19:0/148740354 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f88004800 0x7f8f88004c80 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.448+0000 7f8fa9c00640 1 -- 172.21.15.19:0/148740354 >> 172.21.15.19:0/148740354 conn(0x7f8fa40ccad0 msgr2=0x7f8fa40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:38.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.448+0000 7f8fa9c00640 1 -- 172.21.15.19:0/148740354 shutdown_connections 2024-03-28T13:20:38.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.448+0000 7f8fa9c00640 1 -- 172.21.15.19:0/148740354 wait complete. 2024-03-28T13:20:38.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.450+0000 7f8fa9c00640 1 Processor -- start 2024-03-28T13:20:38.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.450+0000 7f8fa9c00640 1 -- start start 2024-03-28T13:20:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.451+0000 7f8fa9c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f88004800 0x7f8fa415db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.451+0000 7f8fa9c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa40b0460 0x7f8fa40b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.451+0000 7f8fa9c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa4138420 0x7f8fa41388a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.451+0000 7f8fa9c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa4147fa0 con 0x7f8f88004800 2024-03-28T13:20:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.451+0000 7f8fa9c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa4147e20 con 0x7f8fa40b0460 2024-03-28T13:20:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.451+0000 7f8fa9c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8fa40ce610 con 0x7f8fa4138420 2024-03-28T13:20:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.451+0000 7f8fa3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f88004800 0x7f8fa415db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.451+0000 7f8fa3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f88004800 0x7f8fa415db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52648/0 (socket says 172.21.15.19:52648) 2024-03-28T13:20:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.451+0000 7f8fa3e00640 1 -- 172.21.15.19:0/3063930080 learned_addr learned my addr 172.21.15.19:0/3063930080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.452+0000 7f8fa3e00640 1 -- 172.21.15.19:0/3063930080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa4138420 msgr2=0x7f8fa41388a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.452+0000 7f8fa3e00640 1 --2- 172.21.15.19:0/3063930080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa4138420 0x7f8fa41388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.452+0000 7f8fa3e00640 1 -- 172.21.15.19:0/3063930080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa40b0460 msgr2=0x7f8fa40b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:38.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.452+0000 7f8fa3400640 1 --2- 172.21.15.19:0/3063930080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa40b0460 0x7f8fa40b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.452+0000 7f8fa3e00640 1 --2- 172.21.15.19:0/3063930080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa40b0460 0x7f8fa40b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.452+0000 7f8fa3e00640 1 -- 172.21.15.19:0/3063930080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa4139150 con 0x7f8f88004800 2024-03-28T13:20:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.452+0000 7f8fa3e00640 1 --2- 172.21.15.19:0/3063930080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f88004800 0x7f8fa415db60 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400d400 tx=0x7f8f9400d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.452+0000 7f8f9be00640 1 -- 172.21.15.19:0/3063930080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94018070 con 0x7f8f88004800 2024-03-28T13:20:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.452+0000 7f8f9be00640 1 -- 172.21.15.19:0/3063930080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f94003610 con 0x7f8f88004800 2024-03-28T13:20:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.452+0000 7f8fa9c00640 1 -- 172.21.15.19:0/3063930080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa4139430 con 0x7f8f88004800 2024-03-28T13:20:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.452+0000 7f8f9be00640 1 -- 172.21.15.19:0/3063930080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94003a00 con 0x7f8f88004800 2024-03-28T13:20:38.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.453+0000 7f8fa9c00640 1 -- 172.21.15.19:0/3063930080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa41399f0 con 0x7f8f88004800 2024-03-28T13:20:38.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.454+0000 7f8fa9c00640 1 -- 172.21.15.19:0/3063930080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa4075940 con 0x7f8f88004800 2024-03-28T13:20:38.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.455+0000 7f8f9be00640 1 -- 172.21.15.19:0/3063930080 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8f9401b070 con 0x7f8f88004800 2024-03-28T13:20:38.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.456+0000 7f8f9be00640 1 --2- 172.21.15.19:0/3063930080 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8406c560 0x7f8f8406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:38.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.456+0000 7f8f9be00640 1 -- 172.21.15.19:0/3063930080 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5939+0+0 (secure 0 0 0) 0x7f8f9408aee0 con 0x7f8f88004800 2024-03-28T13:20:38.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.456+0000 7f8fa3400640 1 --2- 172.21.15.19:0/3063930080 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8406c560 0x7f8f8406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:38.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.457+0000 7f8fa3400640 1 --2- 172.21.15.19:0/3063930080 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8406c560 0x7f8f8406ea20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000ba00 tx=0x7f8f90030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:38.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.460+0000 7f8f9be00640 1 -- 172.21.15.19:0/3063930080 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8f94058db0 con 0x7f8f88004800 2024-03-28T13:20:38.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.666+0000 7f8fa9c00640 1 -- 172.21.15.19:0/3063930080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fa40cc6d0 con 0x7f8f88004800 2024-03-28T13:20:38.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.669+0000 7f8f9be00640 1 -- 172.21.15.19:0/3063930080 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+16975 (secure 0 0 0) 0x7f8f9405d0b0 con 0x7f8f88004800 2024-03-28T13:20:38.669 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:38.669 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":109,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:20:33.234662+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:18:56.432417+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-28T13:18:57.540515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.c","mappings":[{"from":5,"to":0}]},{"pgid":"14.18","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/2851646201":"2024-03-29T13:20:33.234624+0000","172.21.15.19:6832/2851646201":"2024-03-29T13:20:33.234624+0000","172.21.15.19:6834/1020599987":"2024-03-29T13:19:55.818562+0000","172.21.15.19:6832/1020599987":"2024-03-29T13:19:55.818562+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:20:38.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.672+0000 7f8fa9c00640 1 -- 172.21.15.19:0/3063930080 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8406c560 msgr2=0x7f8f8406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:38.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.672+0000 7f8fa9c00640 1 --2- 172.21.15.19:0/3063930080 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8406c560 0x7f8f8406ea20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8f9000ba00 tx=0x7f8f90030000 comp rx=0 tx=0).stop 2024-03-28T13:20:38.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.672+0000 7f8fa9c00640 1 -- 172.21.15.19:0/3063930080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f88004800 msgr2=0x7f8fa415db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:38.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.672+0000 7f8fa9c00640 1 --2- 172.21.15.19:0/3063930080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f88004800 0x7f8fa415db60 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400d400 tx=0x7f8f9400d8c0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.672+0000 7f8fa9c00640 1 -- 172.21.15.19:0/3063930080 shutdown_connections 2024-03-28T13:20:38.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.672+0000 7f8fa9c00640 1 --2- 172.21.15.19:0/3063930080 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8406c560 0x7f8f8406ea20 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.672+0000 7f8fa9c00640 1 --2- 172.21.15.19:0/3063930080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa4138420 0x7f8fa41388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.672+0000 7f8fa9c00640 1 --2- 172.21.15.19:0/3063930080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa40b0460 0x7f8fa40b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.672+0000 7f8fa9c00640 1 --2- 172.21.15.19:0/3063930080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f88004800 0x7f8fa415db60 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.672+0000 7f8fa9c00640 1 -- 172.21.15.19:0/3063930080 >> 172.21.15.19:0/3063930080 conn(0x7f8fa40ccad0 msgr2=0x7f8fa40712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:38.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.672+0000 7f8fa9c00640 1 -- 172.21.15.19:0/3063930080 shutdown_connections 2024-03-28T13:20:38.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.672+0000 7f8fa9c00640 1 -- 172.21.15.19:0/3063930080 wait complete. 2024-03-28T13:20:38.692 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T13:20:38.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.914+0000 7f50c5a00640 1 Processor -- start 2024-03-28T13:20:38.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.914+0000 7f50c5a00640 1 -- start start 2024-03-28T13:20:38.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.916+0000 7f50c5a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50c0147a40 0x7f50c00cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:38.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.916+0000 7f50c5a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c00ce380 0x7f50c006da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:38.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.916+0000 7f50c5a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50c0161f30 con 0x7f50c00d01c0 2024-03-28T13:20:38.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.916+0000 7f50c5a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f50c0162b00 con 0x7f50c0147a40 2024-03-28T13:20:38.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.916+0000 7f50c5a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f50c00ce7e0 con 0x7f50c00ce380 2024-03-28T13:20:38.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.916+0000 7f50bfe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f50c0147a40 0x7f50c00cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:38.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.916+0000 7f50bf400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f50c00d01c0 0x7f50c00cdc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47662/0 (socket says 172.21.15.19:47662) 2024-03-28T13:20:38.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.916+0000 7f50bf400640 1 -- 172.21.15.19:0/795302538 learned_addr learned my addr 172.21.15.19:0/795302538 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:38.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.917+0000 7f50c5000640 1 --2- 172.21.15.19:0/795302538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c00ce380 0x7f50c006da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:38.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.917+0000 7f50bfe00640 1 -- 172.21.15.19:0/795302538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c00ce380 msgr2=0x7f50c006da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:38.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.917+0000 7f50bfe00640 1 --2- 172.21.15.19:0/795302538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c00ce380 0x7f50c006da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.917+0000 7f50bfe00640 1 -- 172.21.15.19:0/795302538 >> v1:172.21.15.19:6789/0 conn(0x7f50c00d01c0 legacy=0x7f50c00cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:38.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.917+0000 7f50bfe00640 1 -- 172.21.15.19:0/795302538 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c01621d0 con 0x7f50c0147a40 2024-03-28T13:20:38.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.918+0000 7f50bfe00640 1 --2- 172.21.15.19:0/795302538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c0147a40 0x7f50c00cfc80 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f50c0162c80 tx=0x7f50b002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45efa686d97e44b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:38.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.918+0000 7f50bea00640 1 -- 172.21.15.19:0/795302538 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0035020 con 0x7f50c0147a40 2024-03-28T13:20:38.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.919+0000 7f50bea00640 1 -- 172.21.15.19:0/795302538 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b0031810 con 0x7f50c0147a40 2024-03-28T13:20:38.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.919+0000 7f50bea00640 1 -- 172.21.15.19:0/795302538 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b0031be0 con 0x7f50c0147a40 2024-03-28T13:20:38.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.919+0000 7f50c5a00640 1 -- 172.21.15.19:0/795302538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c0147a40 msgr2=0x7f50c00cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:38.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.919+0000 7f50c5a00640 1 --2- 172.21.15.19:0/795302538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c0147a40 0x7f50c00cfc80 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f50c0162c80 tx=0x7f50b002b8b0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.919+0000 7f50c5a00640 1 -- 172.21.15.19:0/795302538 shutdown_connections 2024-03-28T13:20:38.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.919+0000 7f50c5a00640 1 --2- 172.21.15.19:0/795302538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c00ce380 0x7f50c006da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.919+0000 7f50c5a00640 1 --2- 172.21.15.19:0/795302538 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c0147a40 0x7f50c00cfc80 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.919+0000 7f50c5a00640 1 -- 172.21.15.19:0/795302538 >> 172.21.15.19:0/795302538 conn(0x7f50c00ccad0 msgr2=0x7f50c00cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:38.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.920+0000 7f50c5a00640 1 -- 172.21.15.19:0/795302538 shutdown_connections 2024-03-28T13:20:38.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.920+0000 7f50c5a00640 1 -- 172.21.15.19:0/795302538 wait complete. 2024-03-28T13:20:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.921+0000 7f50c5a00640 1 Processor -- start 2024-03-28T13:20:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.922+0000 7f50c5a00640 1 -- start start 2024-03-28T13:20:38.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.922+0000 7f50c5a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c00ce380 0x7f50c015cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:38.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.923+0000 7f50c5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c00d01c0 0x7f50c006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:38.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.923+0000 7f50c5a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c0147a40 0x7f50c006dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:38.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.923+0000 7f50c5a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f50c00d0620 con 0x7f50c00d01c0 2024-03-28T13:20:38.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.923+0000 7f50c5a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f50c0162790 con 0x7f50c00ce380 2024-03-28T13:20:38.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.923+0000 7f50c5a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f50c0147e20 con 0x7f50c0147a40 2024-03-28T13:20:38.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.923+0000 7f50bf400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c00d01c0 0x7f50c006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:38.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.923+0000 7f50bf400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c00d01c0 0x7f50c006da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52660/0 (socket says 172.21.15.19:52660) 2024-03-28T13:20:38.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.923+0000 7f50bf400640 1 -- 172.21.15.19:0/1481744501 learned_addr learned my addr 172.21.15.19:0/1481744501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.923+0000 7f50c5000640 1 --2- 172.21.15.19:0/1481744501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c0147a40 0x7f50c006dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.923+0000 7f50bfe00640 1 --2- 172.21.15.19:0/1481744501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c00ce380 0x7f50c015cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.923+0000 7f50bf400640 1 -- 172.21.15.19:0/1481744501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c0147a40 msgr2=0x7f50c006dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.923+0000 7f50bf400640 1 --2- 172.21.15.19:0/1481744501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c0147a40 0x7f50c006dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.924+0000 7f50bf400640 1 -- 172.21.15.19:0/1481744501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c00ce380 msgr2=0x7f50c015cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.924+0000 7f50bf400640 1 --2- 172.21.15.19:0/1481744501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c00ce380 0x7f50c015cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:38.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.924+0000 7f50bf400640 1 -- 172.21.15.19:0/1481744501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c00b1510 con 0x7f50c00d01c0 2024-03-28T13:20:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.924+0000 7f50bf400640 1 --2- 172.21.15.19:0/1481744501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c00d01c0 0x7f50c006da60 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00c430 tx=0x7f50ac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.924+0000 7f50b7e00640 1 -- 172.21.15.19:0/1481744501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac017070 con 0x7f50c00d01c0 2024-03-28T13:20:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.924+0000 7f50b7e00640 1 -- 172.21.15.19:0/1481744501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ac005ce0 con 0x7f50c00d01c0 2024-03-28T13:20:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.924+0000 7f50b7e00640 1 -- 172.21.15.19:0/1481744501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac007310 con 0x7f50c00d01c0 2024-03-28T13:20:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.924+0000 7f50c5a00640 1 -- 172.21.15.19:0/1481744501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c00b17f0 con 0x7f50c00d01c0 2024-03-28T13:20:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.925+0000 7f50c5a00640 1 -- 172.21.15.19:0/1481744501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c013a520 con 0x7f50c00d01c0 2024-03-28T13:20:38.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.926+0000 7f50c5a00640 1 -- 172.21.15.19:0/1481744501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50c00b19c0 con 0x7f50c00d01c0 2024-03-28T13:20:38.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.927+0000 7f50b7e00640 1 -- 172.21.15.19:0/1481744501 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f50ac01a070 con 0x7f50c00d01c0 2024-03-28T13:20:38.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.928+0000 7f50b7e00640 1 --2- 172.21.15.19:0/1481744501 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50a006c530 0x7f50a006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:38.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.928+0000 7f50b7e00640 1 -- 172.21.15.19:0/1481744501 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5939+0+0 (secure 0 0 0) 0x7f50ac08ac60 con 0x7f50c00d01c0 2024-03-28T13:20:38.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.928+0000 7f50bfe00640 1 --2- 172.21.15.19:0/1481744501 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50a006c530 0x7f50a006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:38.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.930+0000 7f50bfe00640 1 --2- 172.21.15.19:0/1481744501 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50a006c530 0x7f50a006e9f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f50b0036000 tx=0x7f50b003c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:38.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:38.932+0000 7f50b7e00640 1 -- 172.21.15.19:0/1481744501 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f50c00b19c0 con 0x7f50c00d01c0 2024-03-28T13:20:39.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.174+0000 7f50c5a00640 1 -- 172.21.15.19:0/1481744501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f50c00af6f0 con 0x7f50c00d01c0 2024-03-28T13:20:39.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.275+0000 7f50b7e00640 1 -- 172.21.15.19:0/1481744501 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) ==== 161+0+0 (secure 0 0 0) 0x7f50ac0590f0 con 0x7f50c00d01c0 2024-03-28T13:20:39.275 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T13:20:39.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.278+0000 7f50c5a00640 1 -- 172.21.15.19:0/1481744501 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50a006c530 msgr2=0x7f50a006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:39.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.278+0000 7f50c5a00640 1 --2- 172.21.15.19:0/1481744501 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50a006c530 0x7f50a006e9f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f50b0036000 tx=0x7f50b003c040 comp rx=0 tx=0).stop 2024-03-28T13:20:39.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.278+0000 7f50c5a00640 1 -- 172.21.15.19:0/1481744501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c00d01c0 msgr2=0x7f50c006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:39.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.278+0000 7f50c5a00640 1 --2- 172.21.15.19:0/1481744501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c00d01c0 0x7f50c006da60 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00c430 tx=0x7f50ac00c8f0 comp rx=0 tx=0).stop 2024-03-28T13:20:39.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.278+0000 7f50c5a00640 1 -- 172.21.15.19:0/1481744501 shutdown_connections 2024-03-28T13:20:39.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.278+0000 7f50c5a00640 1 --2- 172.21.15.19:0/1481744501 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f50a006c530 0x7f50a006e9f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:39.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.278+0000 7f50c5a00640 1 --2- 172.21.15.19:0/1481744501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50c0147a40 0x7f50c006dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:39.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.278+0000 7f50c5a00640 1 --2- 172.21.15.19:0/1481744501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f50c00d01c0 0x7f50c006da60 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:39.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.278+0000 7f50c5a00640 1 --2- 172.21.15.19:0/1481744501 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50c00ce380 0x7f50c015cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:39.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.278+0000 7f50c5a00640 1 -- 172.21.15.19:0/1481744501 >> 172.21.15.19:0/1481744501 conn(0x7f50c00ccad0 msgr2=0x7f50c015f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:39.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.278+0000 7f50c5a00640 1 -- 172.21.15.19:0/1481744501 shutdown_connections 2024-03-28T13:20:39.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.279+0000 7f50c5a00640 1 -- 172.21.15.19:0/1481744501 wait complete. 2024-03-28T13:20:39.298 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-28T13:20:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.516+0000 7fdfb4a00640 1 Processor -- start 2024-03-28T13:20:39.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.516+0000 7fdfb4a00640 1 -- start start 2024-03-28T13:20:39.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.518+0000 7fdfb4a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfb013f190 0x7fdfb00cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:39.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.518+0000 7fdfb4a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdfb00cc880 0x7fdfb00cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:39.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.518+0000 7fdfb4a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb00ce890 con 0x7fdfb006d620 2024-03-28T13:20:39.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.518+0000 7fdfb4a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fdfb00cd200 con 0x7fdfb00cc880 2024-03-28T13:20:39.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.518+0000 7fdfb4a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb00ceb50 con 0x7fdfb013f190 2024-03-28T13:20:39.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.518+0000 7fdfafe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdfb006d620 0x7fdfb006da70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47678/0 (socket says 172.21.15.19:47678) 2024-03-28T13:20:39.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.518+0000 7fdfafe00640 1 -- 172.21.15.19:0/3849151560 learned_addr learned my addr 172.21.15.19:0/3849151560 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfa7e00640 1 --2- 172.21.15.19:0/3849151560 >> v2:172.21.15.188:3300/0 conn(0x7fdfb00cc880 0x7fdfb00cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaf400640 1 --2- 172.21.15.19:0/3849151560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfb013f190 0x7fdfb00cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 -- 172.21.15.19:0/3849151560 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3021110549 0 0) 0x7fdfb00ce890 con 0x7fdfb006d620 2024-03-28T13:20:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 -- 172.21.15.19:0/3849151560 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf94003520 con 0x7fdfb006d620 2024-03-28T13:20:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 -- 172.21.15.19:0/3849151560 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 311705848 0 0) 0x7fdf94003520 con 0x7fdfb006d620 2024-03-28T13:20:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 -- 172.21.15.19:0/3849151560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfb013f190 msgr2=0x7fdfb00cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 --2- 172.21.15.19:0/3849151560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfb013f190 0x7fdfb00cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 -- 172.21.15.19:0/3849151560 >> v2:172.21.15.188:3300/0 conn(0x7fdfb00cc880 msgr2=0x7fdfb00cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 --2- 172.21.15.19:0/3849151560 >> v2:172.21.15.188:3300/0 conn(0x7fdfb00cc880 0x7fdfb00cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 -- 172.21.15.19:0/3849151560 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb00cee30 con 0x7fdfb006d620 2024-03-28T13:20:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 -- 172.21.15.19:0/3849151560 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fdfa80019f0 con 0x7fdfb006d620 2024-03-28T13:20:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaf400640 1 --2- 172.21.15.19:0/3849151560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfb013f190 0x7fdfb00cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:20:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 -- 172.21.15.19:0/3849151560 >> v1:172.21.15.19:6789/0 conn(0x7fdfb006d620 legacy=0x7fdfb006da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 --2- 172.21.15.19:0/3849151560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf940039b0 0x7fdf94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 --2- 172.21.15.19:0/3849151560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfb013f190 0x7fdf94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 --2- 172.21.15.19:0/3849151560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf94004d40 0x7fdf940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 -- 172.21.15.19:0/3849151560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb00cd200 con 0x7fdf940039b0 2024-03-28T13:20:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 -- 172.21.15.19:0/3849151560 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb00ceb50 con 0x7fdf94004d40 2024-03-28T13:20:39.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.519+0000 7fdfaea00640 1 -- 172.21.15.19:0/3849151560 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9400b9c0 con 0x7fdfb013f190 2024-03-28T13:20:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.520+0000 7fdfaf400640 1 --2- 172.21.15.19:0/3849151560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf940039b0 0x7fdf94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.520+0000 7fdfafe00640 1 --2- 172.21.15.19:0/3849151560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf94004d40 0x7fdf940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.520+0000 7fdfa7e00640 1 --2- 172.21.15.19:0/3849151560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfb013f190 0x7fdf94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.520+0000 7fdfaf400640 1 -- 172.21.15.19:0/3849151560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfb013f190 msgr2=0x7fdf94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.520+0000 7fdfaf400640 1 --2- 172.21.15.19:0/3849151560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfb013f190 0x7fdf94004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.520+0000 7fdfaf400640 1 -- 172.21.15.19:0/3849151560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf94004d40 msgr2=0x7fdf940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.520+0000 7fdfaf400640 1 --2- 172.21.15.19:0/3849151560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf94004d40 0x7fdf940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.520+0000 7fdfaf400640 1 -- 172.21.15.19:0/3849151560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf94003520 con 0x7fdf940039b0 2024-03-28T13:20:39.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.520+0000 7fdfafe00640 1 --2- 172.21.15.19:0/3849151560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf94004d40 0x7fdf940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:39.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.521+0000 7fdfaf400640 1 --2- 172.21.15.19:0/3849151560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf940039b0 0x7fdf94003dd0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00098a0 tx=0x7fdf94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07f6d83b9e2b4be server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:39.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.521+0000 7fdfaea00640 1 -- 172.21.15.19:0/3849151560 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa000f040 con 0x7fdf940039b0 2024-03-28T13:20:39.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.521+0000 7fdfaea00640 1 -- 172.21.15.19:0/3849151560 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa0004230 con 0x7fdf940039b0 2024-03-28T13:20:39.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.521+0000 7fdfaea00640 1 -- 172.21.15.19:0/3849151560 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0003610 con 0x7fdf940039b0 2024-03-28T13:20:39.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.522+0000 7fdfb4a00640 1 -- 172.21.15.19:0/3849151560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf940039b0 msgr2=0x7fdf94003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:39.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.522+0000 7fdfb4a00640 1 --2- 172.21.15.19:0/3849151560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf940039b0 0x7fdf94003dd0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00098a0 tx=0x7fdf94005830 comp rx=0 tx=0).stop 2024-03-28T13:20:39.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.522+0000 7fdfafe00640 1 -- 172.21.15.19:0/3849151560 reap_dead start 2024-03-28T13:20:39.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.522+0000 7fdfb4a00640 1 -- 172.21.15.19:0/3849151560 shutdown_connections 2024-03-28T13:20:39.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.522+0000 7fdfb4a00640 1 -- 172.21.15.19:0/3849151560 >> 172.21.15.19:0/3849151560 conn(0x7fdfb00b1ba0 msgr2=0x7fdfb00b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:39.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.522+0000 7fdfb4a00640 1 -- 172.21.15.19:0/3849151560 shutdown_connections 2024-03-28T13:20:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.522+0000 7fdfb4a00640 1 -- 172.21.15.19:0/3849151560 wait complete. 2024-03-28T13:20:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.524+0000 7fdfb4a00640 1 Processor -- start 2024-03-28T13:20:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.524+0000 7fdfb4a00640 1 -- start start 2024-03-28T13:20:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.525+0000 7fdfb4a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf940039b0 0x7fdfb0157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.525+0000 7fdfb4a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb01576a0 0x7fdfb01f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.525+0000 7fdfb4a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfb01f5060 0x7fdfb01f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.525+0000 7fdfb4a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb013f570 con 0x7fdfb01576a0 2024-03-28T13:20:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.525+0000 7fdfb4a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb0160c70 con 0x7fdf940039b0 2024-03-28T13:20:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.525+0000 7fdfb4a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb00cf630 con 0x7fdfb01f5060 2024-03-28T13:20:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.525+0000 7fdfa7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb01576a0 0x7fdfb01f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.525+0000 7fdfa7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb01576a0 0x7fdfb01f4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52668/0 (socket says 172.21.15.19:52668) 2024-03-28T13:20:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.525+0000 7fdfa7e00640 1 -- 172.21.15.19:0/3016138221 learned_addr learned my addr 172.21.15.19:0/3016138221 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.526+0000 7fdfaf400640 1 --2- 172.21.15.19:0/3016138221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf940039b0 0x7fdfb0157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.526+0000 7fdfa7e00640 1 -- 172.21.15.19:0/3016138221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfb01f5060 msgr2=0x7fdfb01f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.526+0000 7fdfafe00640 1 --2- 172.21.15.19:0/3016138221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfb01f5060 0x7fdfb01f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.526+0000 7fdfa7e00640 1 --2- 172.21.15.19:0/3016138221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfb01f5060 0x7fdfb01f59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.526+0000 7fdfa7e00640 1 -- 172.21.15.19:0/3016138221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf940039b0 msgr2=0x7fdfb0157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.526+0000 7fdfa7e00640 1 --2- 172.21.15.19:0/3016138221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf940039b0 0x7fdfb0157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.526+0000 7fdfa7e00640 1 -- 172.21.15.19:0/3016138221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb01f38f0 con 0x7fdfb01576a0 2024-03-28T13:20:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.526+0000 7fdfa7e00640 1 --2- 172.21.15.19:0/3016138221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb01576a0 0x7fdfb01f4b20 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fdf980108e0 tx=0x7fdf98010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.526+0000 7fdfa7400640 1 -- 172.21.15.19:0/3016138221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9800f040 con 0x7fdfb01576a0 2024-03-28T13:20:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.526+0000 7fdfa7400640 1 -- 172.21.15.19:0/3016138221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf9800eb60 con 0x7fdfb01576a0 2024-03-28T13:20:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.526+0000 7fdfa7400640 1 -- 172.21.15.19:0/3016138221 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf9801c400 con 0x7fdfb01576a0 2024-03-28T13:20:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.526+0000 7fdfb4a00640 1 -- 172.21.15.19:0/3016138221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb0072ed0 con 0x7fdfb01576a0 2024-03-28T13:20:39.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.527+0000 7fdfb4a00640 1 -- 172.21.15.19:0/3016138221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb01e6f30 con 0x7fdfb01576a0 2024-03-28T13:20:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.528+0000 7fdfb4a00640 1 -- 172.21.15.19:0/3016138221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb0075940 con 0x7fdfb01576a0 2024-03-28T13:20:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.529+0000 7fdfa7400640 1 -- 172.21.15.19:0/3016138221 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdf9801c5a0 con 0x7fdfb01576a0 2024-03-28T13:20:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.530+0000 7fdfa7400640 1 --2- 172.21.15.19:0/3016138221 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf8c06c560 0x7fdf8c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:39.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.530+0000 7fdfa7400640 1 -- 172.21.15.19:0/3016138221 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5939+0+0 (secure 0 0 0) 0x7fdf9808be60 con 0x7fdfb01576a0 2024-03-28T13:20:39.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.530+0000 7fdfaf400640 1 --2- 172.21.15.19:0/3016138221 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf8c06c560 0x7fdf8c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:39.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.531+0000 7fdfaf400640 1 --2- 172.21.15.19:0/3016138221 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf8c06c560 0x7fdf8c06ea20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0035000 tx=0x7fdfa003a040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:39.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.535+0000 7fdfa7400640 1 -- 172.21.15.19:0/3016138221 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdf98018030 con 0x7fdfb01576a0 2024-03-28T13:20:39.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:39.778+0000 7fdfb4a00640 1 -- 172.21.15.19:0/3016138221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdfb015c260 con 0x7fdfb01576a0 2024-03-28T13:20:40.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.296+0000 7fdfa7400640 1 -- 172.21.15.19:0/3016138221 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) ==== 104+0+0 (secure 0 0 0) 0x7fdf9805a380 con 0x7fdfb01576a0 2024-03-28T13:20:40.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.300+0000 7fdfb4a00640 1 -- 172.21.15.19:0/3016138221 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf8c06c560 msgr2=0x7fdf8c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:40.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.300+0000 7fdfb4a00640 1 --2- 172.21.15.19:0/3016138221 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf8c06c560 0x7fdf8c06ea20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0035000 tx=0x7fdfa003a040 comp rx=0 tx=0).stop 2024-03-28T13:20:40.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.300+0000 7fdfb4a00640 1 -- 172.21.15.19:0/3016138221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb01576a0 msgr2=0x7fdfb01f4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:40.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.300+0000 7fdfb4a00640 1 --2- 172.21.15.19:0/3016138221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb01576a0 0x7fdfb01f4b20 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fdf980108e0 tx=0x7fdf98010da0 comp rx=0 tx=0).stop 2024-03-28T13:20:40.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.301+0000 7fdfb4a00640 1 -- 172.21.15.19:0/3016138221 shutdown_connections 2024-03-28T13:20:40.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.301+0000 7fdfb4a00640 1 --2- 172.21.15.19:0/3016138221 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf8c06c560 0x7fdf8c06ea20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:40.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.301+0000 7fdfb4a00640 1 --2- 172.21.15.19:0/3016138221 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfb01f5060 0x7fdfb01f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:40.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.301+0000 7fdfb4a00640 1 --2- 172.21.15.19:0/3016138221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfb01576a0 0x7fdfb01f4b20 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:40.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.301+0000 7fdfb4a00640 1 --2- 172.21.15.19:0/3016138221 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf940039b0 0x7fdfb0157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:40.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.301+0000 7fdfb4a00640 1 -- 172.21.15.19:0/3016138221 >> 172.21.15.19:0/3016138221 conn(0x7fdfb00b1ba0 msgr2=0x7fdfb015f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:40.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.301+0000 7fdfb4a00640 1 -- 172.21.15.19:0/3016138221 shutdown_connections 2024-03-28T13:20:40.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.301+0000 7fdfb4a00640 1 -- 172.21.15.19:0/3016138221 wait complete. 2024-03-28T13:20:40.326 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-28T13:20:40.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.554+0000 7fb049200640 1 Processor -- start 2024-03-28T13:20:40.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.554+0000 7fb049200640 1 -- start start 2024-03-28T13:20:40.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.556+0000 7fb049200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb044144980 0x7fb04415ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:40.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.556+0000 7fb049200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0440b35b0 0x7fb044071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:40.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.556+0000 7fb049200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb044071540 con 0x7fb04415f150 2024-03-28T13:20:40.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.556+0000 7fb049200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb044144d60 con 0x7fb044144980 2024-03-28T13:20:40.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.556+0000 7fb049200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb0440b3b60 con 0x7fb0440b35b0 2024-03-28T13:20:40.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.556+0000 7fb043400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb044144980 0x7fb04415ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:40.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.556+0000 7fb043400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb044144980 0x7fb04415ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52670/0 (socket says 172.21.15.19:52670) 2024-03-28T13:20:40.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.556+0000 7fb043400640 1 -- 172.21.15.19:0/3391529698 learned_addr learned my addr 172.21.15.19:0/3391529698 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:40.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.557+0000 7fb043e00640 1 --2- 172.21.15.19:0/3391529698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0440b35b0 0x7fb044071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:40.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.557+0000 7fb043400640 1 -- 172.21.15.19:0/3391529698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0440b35b0 msgr2=0x7fb044071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:40.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.557+0000 7fb043400640 1 --2- 172.21.15.19:0/3391529698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0440b35b0 0x7fb044071a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:40.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.557+0000 7fb043400640 1 -- 172.21.15.19:0/3391529698 >> v1:172.21.15.188:6789/0 conn(0x7fb04415f150 legacy=0x7fb0440b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:40.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.557+0000 7fb043400640 1 -- 172.21.15.19:0/3391529698 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0440b3e40 con 0x7fb044144980 2024-03-28T13:20:40.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.558+0000 7fb043400640 1 --2- 172.21.15.19:0/3391529698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb044144980 0x7fb04415ec10 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fb03400bb10 tx=0x7fb03402e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39611331430c6bb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:40.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.558+0000 7fb042a00640 1 -- 172.21.15.19:0/3391529698 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03403b070 con 0x7fb044144980 2024-03-28T13:20:40.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.558+0000 7fb042a00640 1 -- 172.21.15.19:0/3391529698 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb03402ede0 con 0x7fb044144980 2024-03-28T13:20:40.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.558+0000 7fb042a00640 1 -- 172.21.15.19:0/3391529698 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb034036660 con 0x7fb044144980 2024-03-28T13:20:40.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.559+0000 7fb049200640 1 -- 172.21.15.19:0/3391529698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb044144980 msgr2=0x7fb04415ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:40.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.559+0000 7fb049200640 1 --2- 172.21.15.19:0/3391529698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb044144980 0x7fb04415ec10 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fb03400bb10 tx=0x7fb03402e030 comp rx=0 tx=0).stop 2024-03-28T13:20:40.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.559+0000 7fb049200640 1 -- 172.21.15.19:0/3391529698 shutdown_connections 2024-03-28T13:20:40.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.559+0000 7fb049200640 1 --2- 172.21.15.19:0/3391529698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0440b35b0 0x7fb044071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:40.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.559+0000 7fb049200640 1 --2- 172.21.15.19:0/3391529698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb044144980 0x7fb04415ec10 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:40.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.559+0000 7fb049200640 1 -- 172.21.15.19:0/3391529698 >> 172.21.15.19:0/3391529698 conn(0x7fb0440b1d80 msgr2=0x7fb0440b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:40.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.559+0000 7fb049200640 1 -- 172.21.15.19:0/3391529698 shutdown_connections 2024-03-28T13:20:40.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.559+0000 7fb049200640 1 -- 172.21.15.19:0/3391529698 wait complete. 2024-03-28T13:20:40.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.561+0000 7fb049200640 1 Processor -- start 2024-03-28T13:20:40.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.561+0000 7fb049200640 1 -- start start 2024-03-28T13:20:40.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.562+0000 7fb049200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0440b35b0 0x7fb044143160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.562+0000 7fb049200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb044144980 0x7fb0441f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.562+0000 7fb049200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb04415f150 0x7fb0441f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.562+0000 7fb049200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb044072100 con 0x7fb04415f150 2024-03-28T13:20:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.562+0000 7fb049200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb044071f80 con 0x7fb044144980 2024-03-28T13:20:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.562+0000 7fb049200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb044072280 con 0x7fb0440b35b0 2024-03-28T13:20:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.562+0000 7fb043e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb04415f150 0x7fb0441f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.563+0000 7fb043e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb04415f150 0x7fb0441f8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52678/0 (socket says 172.21.15.19:52678) 2024-03-28T13:20:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.563+0000 7fb043e00640 1 -- 172.21.15.19:0/1937194509 learned_addr learned my addr 172.21.15.19:0/1937194509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.563+0000 7fb043400640 1 --2- 172.21.15.19:0/1937194509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0440b35b0 0x7fb044143160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.563+0000 7fb03be00640 1 --2- 172.21.15.19:0/1937194509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb044144980 0x7fb0441f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.563+0000 7fb043e00640 1 -- 172.21.15.19:0/1937194509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0440b35b0 msgr2=0x7fb044143160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.563+0000 7fb043e00640 1 --2- 172.21.15.19:0/1937194509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0440b35b0 0x7fb044143160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.563+0000 7fb043e00640 1 -- 172.21.15.19:0/1937194509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb044144980 msgr2=0x7fb0441f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.563+0000 7fb043e00640 1 --2- 172.21.15.19:0/1937194509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb044144980 0x7fb0441f7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:40.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.563+0000 7fb043e00640 1 -- 172.21.15.19:0/1937194509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0441e6cd0 con 0x7fb04415f150 2024-03-28T13:20:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.563+0000 7fb043e00640 1 --2- 172.21.15.19:0/1937194509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb04415f150 0x7fb0441f8040 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00cd70 tx=0x7fb03c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.564+0000 7fb03b400640 1 -- 172.21.15.19:0/1937194509 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c003b90 con 0x7fb04415f150 2024-03-28T13:20:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.564+0000 7fb03b400640 1 -- 172.21.15.19:0/1937194509 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb03c003d30 con 0x7fb04415f150 2024-03-28T13:20:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.564+0000 7fb03b400640 1 -- 172.21.15.19:0/1937194509 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03c004240 con 0x7fb04415f150 2024-03-28T13:20:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.564+0000 7fb049200640 1 -- 172.21.15.19:0/1937194509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0441e5ae0 con 0x7fb04415f150 2024-03-28T13:20:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.564+0000 7fb049200640 1 -- 172.21.15.19:0/1937194509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0441e60a0 con 0x7fb04415f150 2024-03-28T13:20:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.565+0000 7fb049200640 1 -- 172.21.15.19:0/1937194509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb044075940 con 0x7fb04415f150 2024-03-28T13:20:40.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.566+0000 7fb03b400640 1 -- 172.21.15.19:0/1937194509 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb03c0043e0 con 0x7fb04415f150 2024-03-28T13:20:40.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.567+0000 7fb03b400640 1 --2- 172.21.15.19:0/1937194509 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb02006c580 0x7fb02006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:40.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.567+0000 7fb03b400640 1 -- 172.21.15.19:0/1937194509 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5939+0+0 (secure 0 0 0) 0x7fb03c08c2c0 con 0x7fb04415f150 2024-03-28T13:20:40.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.568+0000 7fb043400640 1 --2- 172.21.15.19:0/1937194509 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb02006c580 0x7fb02006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:40.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.569+0000 7fb043400640 1 --2- 172.21.15.19:0/1937194509 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb02006c580 0x7fb02006ea40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb034038000 tx=0x7fb034008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:40.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.571+0000 7fb03b400640 1 -- 172.21.15.19:0/1937194509 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb03c05a7d0 con 0x7fb04415f150 2024-03-28T13:20:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:40.773+0000 7fb049200640 1 -- 172.21.15.19:0/1937194509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb04406e720 con 0x7fb04415f150 2024-03-28T13:20:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.300+0000 7fb03b400640 1 -- 172.21.15.19:0/1937194509 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) ==== 181+0+0 (secure 0 0 0) 0x7fb03c05a0d0 con 0x7fb04415f150 2024-03-28T13:20:41.300 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-03-28T13:20:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.304+0000 7fb049200640 1 -- 172.21.15.19:0/1937194509 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb02006c580 msgr2=0x7fb02006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.304+0000 7fb049200640 1 --2- 172.21.15.19:0/1937194509 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb02006c580 0x7fb02006ea40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb034038000 tx=0x7fb034008040 comp rx=0 tx=0).stop 2024-03-28T13:20:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.304+0000 7fb049200640 1 -- 172.21.15.19:0/1937194509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb04415f150 msgr2=0x7fb0441f8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:41.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.304+0000 7fb049200640 1 --2- 172.21.15.19:0/1937194509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb04415f150 0x7fb0441f8040 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00cd70 tx=0x7fb03c00a660 comp rx=0 tx=0).stop 2024-03-28T13:20:41.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.305+0000 7fb049200640 1 -- 172.21.15.19:0/1937194509 shutdown_connections 2024-03-28T13:20:41.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.305+0000 7fb049200640 1 --2- 172.21.15.19:0/1937194509 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb02006c580 0x7fb02006ea40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:41.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.305+0000 7fb049200640 1 --2- 172.21.15.19:0/1937194509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb04415f150 0x7fb0441f8040 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:41.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.305+0000 7fb049200640 1 --2- 172.21.15.19:0/1937194509 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb044144980 0x7fb0441f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:41.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.305+0000 7fb049200640 1 --2- 172.21.15.19:0/1937194509 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0440b35b0 0x7fb044143160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:41.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.305+0000 7fb049200640 1 -- 172.21.15.19:0/1937194509 >> 172.21.15.19:0/1937194509 conn(0x7fb0440b1d80 msgr2=0x7fb0440cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:41.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.305+0000 7fb049200640 1 -- 172.21.15.19:0/1937194509 shutdown_connections 2024-03-28T13:20:41.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.306+0000 7fb049200640 1 -- 172.21.15.19:0/1937194509 wait complete. 2024-03-28T13:20:41.328 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-28T13:20:41.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.544+0000 7f9fc1000640 1 Processor -- start 2024-03-28T13:20:41.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.545+0000 7f9fc1000640 1 -- start start 2024-03-28T13:20:41.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.546+0000 7f9fc1000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9fbc0d1770 0x7f9fbc167340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:41.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.546+0000 7f9fc1000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fbc0b7410 con 0x7f9fbc0b7030 2024-03-28T13:20:41.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.546+0000 7f9fc1000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fbc160150 con 0x7f9fbc070010 2024-03-28T13:20:41.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.546+0000 7f9fc1000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9fbc160430 con 0x7f9fbc0d1770 2024-03-28T13:20:41.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.547+0000 7f9fb9e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9fbc0b7030 0x7f9fbc06f900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47688/0 (socket says 172.21.15.19:47688) 2024-03-28T13:20:41.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.547+0000 7f9fb9e00640 1 -- 172.21.15.19:0/658678456 learned_addr learned my addr 172.21.15.19:0/658678456 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:41.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.547+0000 7f9fb8a00640 1 -- 172.21.15.19:0/658678456 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2084002186 0 0) 0x7f9fbc0b7410 con 0x7f9fbc0b7030 2024-03-28T13:20:41.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.547+0000 7f9fba800640 1 --2- 172.21.15.19:0/658678456 >> v2:172.21.15.188:3300/0 conn(0x7f9fbc0d1770 0x7f9fbc167340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:41.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.547+0000 7f9fb8a00640 1 -- 172.21.15.19:0/658678456 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fa0003520 con 0x7f9fbc0b7030 2024-03-28T13:20:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fb8a00640 1 -- 172.21.15.19:0/658678456 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 896906498 0 0) 0x7f9fa0003520 con 0x7f9fbc0b7030 2024-03-28T13:20:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fb8a00640 1 -- 172.21.15.19:0/658678456 >> v2:172.21.15.188:3300/0 conn(0x7f9fbc0d1770 msgr2=0x7f9fbc167340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fb8a00640 1 --2- 172.21.15.19:0/658678456 >> v2:172.21.15.188:3300/0 conn(0x7f9fbc0d1770 0x7f9fbc167340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fb8a00640 1 -- 172.21.15.19:0/658678456 >> v1:172.21.15.188:6789/0 conn(0x7f9fbc070010 legacy=0x7f9fbc0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fb8a00640 1 -- 172.21.15.19:0/658678456 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fbc160710 con 0x7f9fbc0b7030 2024-03-28T13:20:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fb8a00640 1 -- 172.21.15.19:0/658678456 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f9fac001670 con 0x7f9fbc0b7030 2024-03-28T13:20:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fba800640 1 --2- 172.21.15.19:0/658678456 >> v2:172.21.15.188:3300/0 conn(0x7f9fbc0d1770 0x7f9fbc167340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:20:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fb8a00640 1 -- 172.21.15.19:0/658678456 >> v1:172.21.15.19:6789/0 conn(0x7f9fbc0b7030 legacy=0x7f9fbc06f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fb8a00640 1 --2- 172.21.15.19:0/658678456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fa00039b0 0x7f9fa0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fb8a00640 1 --2- 172.21.15.19:0/658678456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fa0004800 0x7f9fa0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fb8a00640 1 --2- 172.21.15.19:0/658678456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa00052e0 0x7f9fa000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fb8a00640 1 -- 172.21.15.19:0/658678456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc160430 con 0x7f9fa00052e0 2024-03-28T13:20:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fb8a00640 1 -- 172.21.15.19:0/658678456 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa000c1b0 con 0x7f9fa0004800 2024-03-28T13:20:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fb8a00640 1 -- 172.21.15.19:0/658678456 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa000c3b0 con 0x7f9fa00039b0 2024-03-28T13:20:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fba800640 1 --2- 172.21.15.19:0/658678456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa00052e0 0x7f9fa000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fb9400640 1 --2- 172.21.15.19:0/658678456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fa0004800 0x7f9fa0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.548+0000 7f9fb9e00640 1 --2- 172.21.15.19:0/658678456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fa00039b0 0x7f9fa0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.549+0000 7f9fba800640 1 -- 172.21.15.19:0/658678456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fa00039b0 msgr2=0x7f9fa0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.549+0000 7f9fba800640 1 --2- 172.21.15.19:0/658678456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fa00039b0 0x7f9fa0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.549+0000 7f9fba800640 1 -- 172.21.15.19:0/658678456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fa0004800 msgr2=0x7f9fa0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.549+0000 7f9fba800640 1 --2- 172.21.15.19:0/658678456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fa0004800 0x7f9fa0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.549+0000 7f9fba800640 1 -- 172.21.15.19:0/658678456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa0003520 con 0x7f9fa00052e0 2024-03-28T13:20:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.549+0000 7f9fb9e00640 1 --2- 172.21.15.19:0/658678456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fa00039b0 0x7f9fa0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:20:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.550+0000 7f9fba800640 1 -- 172.21.15.19:0/658678456 reap_dead start 2024-03-28T13:20:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.551+0000 7f9fba800640 1 --2- 172.21.15.19:0/658678456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa00052e0 0x7f9fa000b9c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40073b0 tx=0x7f9fa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cd4c365d58f4ba server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.551+0000 7f9fb8a00640 1 -- 172.21.15.19:0/658678456 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4039070 con 0x7f9fa00052e0 2024-03-28T13:20:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.551+0000 7f9fb8a00640 1 -- 172.21.15.19:0/658678456 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb402cd90 con 0x7f9fa00052e0 2024-03-28T13:20:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.551+0000 7f9fb8a00640 1 -- 172.21.15.19:0/658678456 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4009c80 con 0x7f9fa00052e0 2024-03-28T13:20:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.551+0000 7f9fc1000640 1 -- 172.21.15.19:0/658678456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa00052e0 msgr2=0x7f9fa000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.551+0000 7f9fc1000640 1 --2- 172.21.15.19:0/658678456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa00052e0 0x7f9fa000b9c0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40073b0 tx=0x7f9fa0005830 comp rx=0 tx=0).stop 2024-03-28T13:20:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.552+0000 7f9fc1000640 1 -- 172.21.15.19:0/658678456 shutdown_connections 2024-03-28T13:20:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.552+0000 7f9fc1000640 1 --2- 172.21.15.19:0/658678456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fa00052e0 0x7f9fa000b9c0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.552+0000 7f9fc1000640 1 -- 172.21.15.19:0/658678456 >> 172.21.15.19:0/658678456 conn(0x7f9fbc06bbd0 msgr2=0x7f9fbc0b7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.552+0000 7f9fc1000640 1 -- 172.21.15.19:0/658678456 shutdown_connections 2024-03-28T13:20:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.552+0000 7f9fc1000640 1 -- 172.21.15.19:0/658678456 wait complete. 2024-03-28T13:20:41.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.553+0000 7f9fc1000640 1 Processor -- start 2024-03-28T13:20:41.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.554+0000 7f9fc1000640 1 -- start start 2024-03-28T13:20:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.554+0000 7f9fc1000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fa00052e0 0x7f9fbc14fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.554+0000 7f9fc1000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fbc06d4e0 0x7f9fbc15c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.555+0000 7f9fc1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fbc15cd90 0x7f9fbc1f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.555+0000 7f9fc1000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc0b6400 con 0x7f9fbc15cd90 2024-03-28T13:20:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.555+0000 7f9fc1000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc0b6280 con 0x7f9fbc06d4e0 2024-03-28T13:20:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.555+0000 7f9fc1000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9fbc0d1060 con 0x7f9fa00052e0 2024-03-28T13:20:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.555+0000 7f9fba800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fbc15cd90 0x7f9fbc1f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:41.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.555+0000 7f9fba800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fbc15cd90 0x7f9fbc1f4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52698/0 (socket says 172.21.15.19:52698) 2024-03-28T13:20:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.555+0000 7f9fba800640 1 -- 172.21.15.19:0/838232688 learned_addr learned my addr 172.21.15.19:0/838232688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.555+0000 7f9fb9e00640 1 --2- 172.21.15.19:0/838232688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fa00052e0 0x7f9fbc14fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.555+0000 7f9fb9400640 1 --2- 172.21.15.19:0/838232688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fbc06d4e0 0x7f9fbc15c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.555+0000 7f9fba800640 1 -- 172.21.15.19:0/838232688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fa00052e0 msgr2=0x7f9fbc14fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.555+0000 7f9fba800640 1 --2- 172.21.15.19:0/838232688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fa00052e0 0x7f9fbc14fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.555+0000 7f9fba800640 1 -- 172.21.15.19:0/838232688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fbc06d4e0 msgr2=0x7f9fbc15c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.555+0000 7f9fba800640 1 --2- 172.21.15.19:0/838232688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fbc06d4e0 0x7f9fbc15c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.555+0000 7f9fba800640 1 -- 172.21.15.19:0/838232688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fbc1f4fc0 con 0x7f9fbc15cd90 2024-03-28T13:20:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.555+0000 7f9fb9400640 1 --2- 172.21.15.19:0/838232688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fbc06d4e0 0x7f9fbc15c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:41.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.556+0000 7f9fba800640 1 --2- 172.21.15.19:0/838232688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fbc15cd90 0x7f9fbc1f4250 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40074d0 tx=0x7f9fb4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:41.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.556+0000 7f9fb2000640 1 -- 172.21.15.19:0/838232688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4039040 con 0x7f9fbc15cd90 2024-03-28T13:20:41.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.556+0000 7f9fb2000640 1 -- 172.21.15.19:0/838232688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb402cb90 con 0x7f9fbc15cd90 2024-03-28T13:20:41.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.556+0000 7f9fb2000640 1 -- 172.21.15.19:0/838232688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4009c80 con 0x7f9fbc15cd90 2024-03-28T13:20:41.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.556+0000 7f9fc1000640 1 -- 172.21.15.19:0/838232688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fbc06c720 con 0x7f9fbc15cd90 2024-03-28T13:20:41.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.556+0000 7f9fc1000640 1 -- 172.21.15.19:0/838232688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fbc06cce0 con 0x7f9fbc15cd90 2024-03-28T13:20:41.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.557+0000 7f9fc1000640 1 -- 172.21.15.19:0/838232688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fbc068cd0 con 0x7f9fbc15cd90 2024-03-28T13:20:41.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.558+0000 7f9fb2000640 1 -- 172.21.15.19:0/838232688 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f9fb4046020 con 0x7f9fbc15cd90 2024-03-28T13:20:41.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.559+0000 7f9fb2000640 1 --2- 172.21.15.19:0/838232688 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9f9c06c530 0x7f9f9c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:41.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.559+0000 7f9fb2000640 1 -- 172.21.15.19:0/838232688 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5443+0+0 (secure 0 0 0) 0x7f9fb40acf60 con 0x7f9fbc15cd90 2024-03-28T13:20:41.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.560+0000 7f9fb9e00640 1 --2- 172.21.15.19:0/838232688 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9f9c06c530 0x7f9f9c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:41.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.561+0000 7f9fb9e00640 1 --2- 172.21.15.19:0/838232688 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9f9c06c530 0x7f9f9c06e9f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9fac000f60 tx=0x7f9fac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:41.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.565+0000 7f9fb2000640 1 -- 172.21.15.19:0/838232688 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9fb404d020 con 0x7f9fbc15cd90 2024-03-28T13:20:41.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:41.767+0000 7f9fc1000640 1 -- 172.21.15.19:0/838232688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9fbc0d2f90 con 0x7f9fbc15cd90 2024-03-28T13:20:42.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.313+0000 7f9fb2000640 1 -- 172.21.15.19:0/838232688 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) ==== 169+0+0 (secure 0 0 0) 0x7f9fb402cd60 con 0x7f9fbc15cd90 2024-03-28T13:20:42.314 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-03-28T13:20:42.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.317+0000 7f9fc1000640 1 -- 172.21.15.19:0/838232688 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9f9c06c530 msgr2=0x7f9f9c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:42.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.317+0000 7f9fc1000640 1 --2- 172.21.15.19:0/838232688 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9f9c06c530 0x7f9f9c06e9f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9fac000f60 tx=0x7f9fac009040 comp rx=0 tx=0).stop 2024-03-28T13:20:42.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.317+0000 7f9fc1000640 1 -- 172.21.15.19:0/838232688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fbc15cd90 msgr2=0x7f9fbc1f4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:42.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.317+0000 7f9fc1000640 1 --2- 172.21.15.19:0/838232688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fbc15cd90 0x7f9fbc1f4250 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40074d0 tx=0x7f9fb4004ab0 comp rx=0 tx=0).stop 2024-03-28T13:20:42.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.317+0000 7f9fc1000640 1 -- 172.21.15.19:0/838232688 shutdown_connections 2024-03-28T13:20:42.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.317+0000 7f9fc1000640 1 --2- 172.21.15.19:0/838232688 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f9f9c06c530 0x7f9f9c06e9f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:42.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.317+0000 7f9fc1000640 1 --2- 172.21.15.19:0/838232688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fbc15cd90 0x7f9fbc1f4250 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:42.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.317+0000 7f9fc1000640 1 --2- 172.21.15.19:0/838232688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9fbc06d4e0 0x7f9fbc15c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:42.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.317+0000 7f9fc1000640 1 --2- 172.21.15.19:0/838232688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9fa00052e0 0x7f9fbc14fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:42.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.317+0000 7f9fc1000640 1 -- 172.21.15.19:0/838232688 >> 172.21.15.19:0/838232688 conn(0x7f9fbc06bbd0 msgr2=0x7f9fbc067280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:42.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.318+0000 7f9fc1000640 1 -- 172.21.15.19:0/838232688 shutdown_connections 2024-03-28T13:20:42.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.318+0000 7f9fc1000640 1 -- 172.21.15.19:0/838232688 wait complete. 2024-03-28T13:20:42.342 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-28T13:20:42.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.573+0000 7fd94c400640 1 Processor -- start 2024-03-28T13:20:42.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.573+0000 7fd94c400640 1 -- start start 2024-03-28T13:20:42.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.574+0000 7fd94c400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9440d0990 0x7fd9440aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:42.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.574+0000 7fd94c400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd9440af2e0 0x7fd9441600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:42.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.574+0000 7fd94c400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9440afab0 con 0x7fd944153580 2024-03-28T13:20:42.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.574+0000 7fd94c400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fd94415cc80 con 0x7fd9440af2e0 2024-03-28T13:20:42.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.574+0000 7fd94c400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd94415da00 con 0x7fd9440d0990 2024-03-28T13:20:42.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.575+0000 7fd949000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9440d0990 0x7fd9440aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:42.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.575+0000 7fd94a400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd9440af2e0 0x7fd9441600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:42.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.575+0000 7fd949a00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd944153580 0x7fd9440d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:59752/0 (socket says 172.21.15.19:59752) 2024-03-28T13:20:42.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.575+0000 7fd949a00640 1 -- 172.21.15.19:0/576795447 learned_addr learned my addr 172.21.15.19:0/576795447 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:42.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.576+0000 7fd949000640 1 -- 172.21.15.19:0/576795447 >> v2:172.21.15.188:3300/0 conn(0x7fd9440af2e0 msgr2=0x7fd9441600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:42.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.576+0000 7fd949000640 1 --2- 172.21.15.19:0/576795447 >> v2:172.21.15.188:3300/0 conn(0x7fd9440af2e0 0x7fd9441600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:42.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.576+0000 7fd949000640 1 -- 172.21.15.19:0/576795447 >> v1:172.21.15.188:6789/0 conn(0x7fd944153580 legacy=0x7fd9440d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:42.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.576+0000 7fd949000640 1 -- 172.21.15.19:0/576795447 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd94415cf20 con 0x7fd9440d0990 2024-03-28T13:20:42.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.576+0000 7fd94a400640 1 --2- 172.21.15.19:0/576795447 >> v2:172.21.15.188:3300/0 conn(0x7fd9440af2e0 0x7fd9441600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:20:42.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.578+0000 7fd949000640 1 --2- 172.21.15.19:0/576795447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9440d0990 0x7fd9440aeda0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd9300097a0 tx=0x7fd93002b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eac44a884ef79ea server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:42.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.578+0000 7fd943e00640 1 -- 172.21.15.19:0/576795447 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930031780 con 0x7fd9440d0990 2024-03-28T13:20:42.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.578+0000 7fd943e00640 1 -- 172.21.15.19:0/576795447 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd930037070 con 0x7fd9440d0990 2024-03-28T13:20:42.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.578+0000 7fd943e00640 1 -- 172.21.15.19:0/576795447 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930031b40 con 0x7fd9440d0990 2024-03-28T13:20:42.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.579+0000 7fd94c400640 1 -- 172.21.15.19:0/576795447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9440d0990 msgr2=0x7fd9440aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:42.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.579+0000 7fd94c400640 1 --2- 172.21.15.19:0/576795447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9440d0990 0x7fd9440aeda0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd9300097a0 tx=0x7fd93002b820 comp rx=0 tx=0).stop 2024-03-28T13:20:42.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.579+0000 7fd94c400640 1 -- 172.21.15.19:0/576795447 shutdown_connections 2024-03-28T13:20:42.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.579+0000 7fd94c400640 1 --2- 172.21.15.19:0/576795447 >> v2:172.21.15.188:3300/0 conn(0x7fd9440af2e0 0x7fd9441600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:42.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.579+0000 7fd94c400640 1 --2- 172.21.15.19:0/576795447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9440d0990 0x7fd9440aeda0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:42.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.579+0000 7fd94c400640 1 -- 172.21.15.19:0/576795447 >> 172.21.15.19:0/576795447 conn(0x7fd94406b750 msgr2=0x7fd94406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:42.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.579+0000 7fd94c400640 1 -- 172.21.15.19:0/576795447 shutdown_connections 2024-03-28T13:20:42.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.579+0000 7fd94c400640 1 -- 172.21.15.19:0/576795447 wait complete. 2024-03-28T13:20:42.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.581+0000 7fd94c400640 1 Processor -- start 2024-03-28T13:20:42.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.581+0000 7fd94c400640 1 -- start start 2024-03-28T13:20:42.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.582+0000 7fd94c400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9440af2e0 0x7fd9440b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:42.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.582+0000 7fd94c400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9440d0990 0x7fd9441599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:42.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.582+0000 7fd94c400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd944153580 0x7fd944159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:42.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.582+0000 7fd94c400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd944153960 con 0x7fd944153580 2024-03-28T13:20:42.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.582+0000 7fd94c400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd94415d4e0 con 0x7fd9440af2e0 2024-03-28T13:20:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.582+0000 7fd94c400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd9440af710 con 0x7fd9440d0990 2024-03-28T13:20:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.582+0000 7fd94a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd944153580 0x7fd944159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.583+0000 7fd94a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd944153580 0x7fd944159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52706/0 (socket says 172.21.15.19:52706) 2024-03-28T13:20:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.583+0000 7fd94a400640 1 -- 172.21.15.19:0/4279051654 learned_addr learned my addr 172.21.15.19:0/4279051654 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.583+0000 7fd949a00640 1 --2- 172.21.15.19:0/4279051654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9440af2e0 0x7fd9440b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.583+0000 7fd94a400640 1 -- 172.21.15.19:0/4279051654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9440d0990 msgr2=0x7fd9441599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.583+0000 7fd94a400640 1 --2- 172.21.15.19:0/4279051654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9440d0990 0x7fd9441599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.583+0000 7fd94a400640 1 -- 172.21.15.19:0/4279051654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9440af2e0 msgr2=0x7fd9440b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.583+0000 7fd94a400640 1 --2- 172.21.15.19:0/4279051654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9440af2e0 0x7fd9440b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.583+0000 7fd94a400640 1 -- 172.21.15.19:0/4279051654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9440ce700 con 0x7fd944153580 2024-03-28T13:20:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.583+0000 7fd94a400640 1 --2- 172.21.15.19:0/4279051654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd944153580 0x7fd944159f10 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00efc0 tx=0x7fd93c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.584+0000 7fd941600640 1 -- 172.21.15.19:0/4279051654 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c018030 con 0x7fd944153580 2024-03-28T13:20:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.584+0000 7fd941600640 1 -- 172.21.15.19:0/4279051654 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd93c00cdf0 con 0x7fd944153580 2024-03-28T13:20:42.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.584+0000 7fd941600640 1 -- 172.21.15.19:0/4279051654 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd93c003bb0 con 0x7fd944153580 2024-03-28T13:20:42.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.584+0000 7fd94c400640 1 -- 172.21.15.19:0/4279051654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9440ce9e0 con 0x7fd944153580 2024-03-28T13:20:42.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.584+0000 7fd94c400640 1 -- 172.21.15.19:0/4279051654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9440cefa0 con 0x7fd944153580 2024-03-28T13:20:42.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.585+0000 7fd94c400640 1 -- 172.21.15.19:0/4279051654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9440685c0 con 0x7fd944153580 2024-03-28T13:20:42.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.586+0000 7fd941600640 1 -- 172.21.15.19:0/4279051654 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd93c01b070 con 0x7fd944153580 2024-03-28T13:20:42.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.587+0000 7fd941600640 1 --2- 172.21.15.19:0/4279051654 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd92406c530 0x7fd92406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:42.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.587+0000 7fd941600640 1 -- 172.21.15.19:0/4279051654 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5029+0+0 (secure 0 0 0) 0x7fd93c08be50 con 0x7fd944153580 2024-03-28T13:20:42.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.588+0000 7fd949a00640 1 --2- 172.21.15.19:0/4279051654 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd92406c530 0x7fd92406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:42.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.589+0000 7fd949a00640 1 --2- 172.21.15.19:0/4279051654 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd92406c530 0x7fd92406e9f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd94415c2d0 tx=0x7fd938009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:42.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.592+0000 7fd941600640 1 -- 172.21.15.19:0/4279051654 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd93c010040 con 0x7fd944153580 2024-03-28T13:20:42.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.799+0000 7fd94c400640 1 -- 172.21.15.19:0/4279051654 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd94406eca0 con 0x7fd944153580 2024-03-28T13:20:42.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.804+0000 7fd941600640 1 -- 172.21.15.19:0/4279051654 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7fd93c05a700 con 0x7fd944153580 2024-03-28T13:20:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.808+0000 7fd94c400640 1 -- 172.21.15.19:0/4279051654 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd92406c530 msgr2=0x7fd92406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.808+0000 7fd94c400640 1 --2- 172.21.15.19:0/4279051654 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd92406c530 0x7fd92406e9f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd94415c2d0 tx=0x7fd938009040 comp rx=0 tx=0).stop 2024-03-28T13:20:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.808+0000 7fd94c400640 1 -- 172.21.15.19:0/4279051654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd944153580 msgr2=0x7fd944159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.808+0000 7fd94c400640 1 --2- 172.21.15.19:0/4279051654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd944153580 0x7fd944159f10 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00efc0 tx=0x7fd93c00c490 comp rx=0 tx=0).stop 2024-03-28T13:20:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.808+0000 7fd94c400640 1 -- 172.21.15.19:0/4279051654 shutdown_connections 2024-03-28T13:20:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.808+0000 7fd94c400640 1 --2- 172.21.15.19:0/4279051654 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd92406c530 0x7fd92406e9f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.808+0000 7fd94c400640 1 --2- 172.21.15.19:0/4279051654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd944153580 0x7fd944159f10 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.808+0000 7fd94c400640 1 --2- 172.21.15.19:0/4279051654 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9440d0990 0x7fd9441599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:42.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.808+0000 7fd94c400640 1 --2- 172.21.15.19:0/4279051654 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9440af2e0 0x7fd9440b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.808+0000 7fd94c400640 1 -- 172.21.15.19:0/4279051654 >> 172.21.15.19:0/4279051654 conn(0x7fd94406b750 msgr2=0x7fd94406dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.808+0000 7fd94c400640 1 -- 172.21.15.19:0/4279051654 shutdown_connections 2024-03-28T13:20:42.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:42.808+0000 7fd94c400640 1 -- 172.21.15.19:0/4279051654 wait complete. 2024-03-28T13:20:42.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-03-28T13:20:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.037+0000 7fa836000640 1 Processor -- start 2024-03-28T13:20:43.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.038+0000 7fa836000640 1 -- start start 2024-03-28T13:20:43.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.039+0000 7fa836000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa83013f300 0x7fa83006e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:43.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.039+0000 7fa836000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa830141e40 0x7fa8300cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:43.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.039+0000 7fa836000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa830142880 con 0x7fa8301419e0 2024-03-28T13:20:43.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.039+0000 7fa836000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa8300cee90 con 0x7fa830141e40 2024-03-28T13:20:43.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.039+0000 7fa836000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa8301422c0 con 0x7fa83013f300 2024-03-28T13:20:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.040+0000 7fa834c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa83013f300 0x7fa83006e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.040+0000 7fa835600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa830141e40 0x7fa8300cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.040+0000 7fa834c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa83013f300 0x7fa83006e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:47406/0 (socket says 172.21.15.19:47406) 2024-03-28T13:20:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.040+0000 7fa82fe00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fa8301419e0 0x7fa83015fd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:59764/0 (socket says 172.21.15.19:59764) 2024-03-28T13:20:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.040+0000 7fa834c00640 1 -- 172.21.15.19:0/2457965404 learned_addr learned my addr 172.21.15.19:0/2457965404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.041+0000 7fa835600640 1 -- 172.21.15.19:0/2457965404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa83013f300 msgr2=0x7fa83006e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.041+0000 7fa835600640 1 --2- 172.21.15.19:0/2457965404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa83013f300 0x7fa83006e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.041+0000 7fa835600640 1 -- 172.21.15.19:0/2457965404 >> v1:172.21.15.188:6789/0 conn(0x7fa8301419e0 legacy=0x7fa83015fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.041+0000 7fa835600640 1 -- 172.21.15.19:0/2457965404 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8301425a0 con 0x7fa830141e40 2024-03-28T13:20:43.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.041+0000 7fa834c00640 1 --2- 172.21.15.19:0/2457965404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa83013f300 0x7fa83006e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:20:43.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.043+0000 7fa835600640 1 --2- 172.21.15.19:0/2457965404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa830141e40 0x7fa8300cf320 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa828009d90 tx=0x7fa82802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df6d24626f2e61a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:43.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.043+0000 7fa82f400640 1 -- 172.21.15.19:0/2457965404 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828039070 con 0x7fa830141e40 2024-03-28T13:20:43.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.043+0000 7fa82f400640 1 -- 172.21.15.19:0/2457965404 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa82802cc50 con 0x7fa830141e40 2024-03-28T13:20:43.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.043+0000 7fa82f400640 1 -- 172.21.15.19:0/2457965404 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828034440 con 0x7fa830141e40 2024-03-28T13:20:43.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.044+0000 7fa836000640 1 -- 172.21.15.19:0/2457965404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa830141e40 msgr2=0x7fa8300cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.044+0000 7fa836000640 1 --2- 172.21.15.19:0/2457965404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa830141e40 0x7fa8300cf320 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa828009d90 tx=0x7fa82802bea0 comp rx=0 tx=0).stop 2024-03-28T13:20:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.044+0000 7fa836000640 1 -- 172.21.15.19:0/2457965404 shutdown_connections 2024-03-28T13:20:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.044+0000 7fa836000640 1 --2- 172.21.15.19:0/2457965404 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa830141e40 0x7fa8300cf320 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.044+0000 7fa836000640 1 --2- 172.21.15.19:0/2457965404 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa83013f300 0x7fa83006e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.044+0000 7fa836000640 1 -- 172.21.15.19:0/2457965404 >> 172.21.15.19:0/2457965404 conn(0x7fa83006b570 msgr2=0x7fa83006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.044+0000 7fa836000640 1 -- 172.21.15.19:0/2457965404 shutdown_connections 2024-03-28T13:20:43.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.044+0000 7fa836000640 1 -- 172.21.15.19:0/2457965404 wait complete. 2024-03-28T13:20:43.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.046+0000 7fa836000640 1 Processor -- start 2024-03-28T13:20:43.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.046+0000 7fa836000640 1 -- start start 2024-03-28T13:20:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.047+0000 7fa836000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa83013f300 0x7fa8300b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.047+0000 7fa836000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8301419e0 0x7fa830155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.047+0000 7fa836000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa830141e40 0x7fa8301560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.047+0000 7fa836000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa830160620 con 0x7fa8301419e0 2024-03-28T13:20:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.047+0000 7fa836000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8301604a0 con 0x7fa830141e40 2024-03-28T13:20:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.047+0000 7fa836000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa8300cf860 con 0x7fa83013f300 2024-03-28T13:20:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.047+0000 7fa82fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8301419e0 0x7fa830155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.047+0000 7fa82fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8301419e0 0x7fa830155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52714/0 (socket says 172.21.15.19:52714) 2024-03-28T13:20:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.047+0000 7fa82fe00640 1 -- 172.21.15.19:0/1502347744 learned_addr learned my addr 172.21.15.19:0/1502347744 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.048+0000 7fa834c00640 1 --2- 172.21.15.19:0/1502347744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa83013f300 0x7fa8300b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.048+0000 7fa835600640 1 --2- 172.21.15.19:0/1502347744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa830141e40 0x7fa8301560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.048+0000 7fa82fe00640 1 -- 172.21.15.19:0/1502347744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa83013f300 msgr2=0x7fa8300b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.048+0000 7fa82fe00640 1 --2- 172.21.15.19:0/1502347744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa83013f300 0x7fa8300b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.048+0000 7fa82fe00640 1 -- 172.21.15.19:0/1502347744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa830141e40 msgr2=0x7fa8301560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.048+0000 7fa82fe00640 1 --2- 172.21.15.19:0/1502347744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa830141e40 0x7fa8301560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.048+0000 7fa82fe00640 1 -- 172.21.15.19:0/1502347744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa830141390 con 0x7fa8301419e0 2024-03-28T13:20:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.048+0000 7fa835600640 1 --2- 172.21.15.19:0/1502347744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa830141e40 0x7fa8301560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:20:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.048+0000 7fa82fe00640 1 --2- 172.21.15.19:0/1502347744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8301419e0 0x7fa830155bb0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fa82000c400 tx=0x7fa82000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.049+0000 7fa82cc00640 1 -- 172.21.15.19:0/1502347744 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8200074e0 con 0x7fa8301419e0 2024-03-28T13:20:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.049+0000 7fa82cc00640 1 -- 172.21.15.19:0/1502347744 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa820007680 con 0x7fa8301419e0 2024-03-28T13:20:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.049+0000 7fa82cc00640 1 -- 172.21.15.19:0/1502347744 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820004a00 con 0x7fa8301419e0 2024-03-28T13:20:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.049+0000 7fa836000640 1 -- 172.21.15.19:0/1502347744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa830141670 con 0x7fa8301419e0 2024-03-28T13:20:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.049+0000 7fa836000640 1 -- 172.21.15.19:0/1502347744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8300b2620 con 0x7fa8301419e0 2024-03-28T13:20:43.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.050+0000 7fa836000640 1 -- 172.21.15.19:0/1502347744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8300b2820 con 0x7fa8301419e0 2024-03-28T13:20:43.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.051+0000 7fa82cc00640 1 -- 172.21.15.19:0/1502347744 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa82001a020 con 0x7fa8301419e0 2024-03-28T13:20:43.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.052+0000 7fa82cc00640 1 --2- 172.21.15.19:0/1502347744 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa81406c530 0x7fa81406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:43.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.052+0000 7fa82cc00640 1 -- 172.21.15.19:0/1502347744 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5029+0+0 (secure 0 0 0) 0x7fa82008ab40 con 0x7fa8301419e0 2024-03-28T13:20:43.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.053+0000 7fa834c00640 1 --2- 172.21.15.19:0/1502347744 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa81406c530 0x7fa81406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:43.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.054+0000 7fa834c00640 1 --2- 172.21.15.19:0/1502347744 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa81406c530 0x7fa81406e9f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa824006ca0 tx=0x7fa824009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:43.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.057+0000 7fa82cc00640 1 -- 172.21.15.19:0/1502347744 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa8300b2820 con 0x7fa8301419e0 2024-03-28T13:20:43.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:43.347+0000 7fa836000640 1 -- 172.21.15.19:0/1502347744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7fa830075940 con 0x7fa8301419e0 2024-03-28T13:20:44.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.327+0000 7fa82cc00640 1 -- 172.21.15.19:0/1502347744 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v525) ==== 146+0+0 (secure 0 0 0) 0x7fa820059490 con 0x7fa8301419e0 2024-03-28T13:20:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.330+0000 7fa836000640 1 -- 172.21.15.19:0/1502347744 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa81406c530 msgr2=0x7fa81406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.330+0000 7fa836000640 1 --2- 172.21.15.19:0/1502347744 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa81406c530 0x7fa81406e9f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fa824006ca0 tx=0x7fa824009240 comp rx=0 tx=0).stop 2024-03-28T13:20:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.330+0000 7fa836000640 1 -- 172.21.15.19:0/1502347744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8301419e0 msgr2=0x7fa830155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.330+0000 7fa836000640 1 --2- 172.21.15.19:0/1502347744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8301419e0 0x7fa830155bb0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fa82000c400 tx=0x7fa82000c8c0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.330+0000 7fa836000640 1 -- 172.21.15.19:0/1502347744 shutdown_connections 2024-03-28T13:20:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.330+0000 7fa836000640 1 --2- 172.21.15.19:0/1502347744 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa81406c530 0x7fa81406e9f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.330+0000 7fa836000640 1 --2- 172.21.15.19:0/1502347744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa830141e40 0x7fa8301560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.330+0000 7fa836000640 1 --2- 172.21.15.19:0/1502347744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8301419e0 0x7fa830155bb0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.330+0000 7fa836000640 1 --2- 172.21.15.19:0/1502347744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa83013f300 0x7fa8300b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.331+0000 7fa836000640 1 -- 172.21.15.19:0/1502347744 >> 172.21.15.19:0/1502347744 conn(0x7fa83006b570 msgr2=0x7fa8300cfcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.331+0000 7fa836000640 1 -- 172.21.15.19:0/1502347744 shutdown_connections 2024-03-28T13:20:44.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.331+0000 7fa836000640 1 -- 172.21.15.19:0/1502347744 wait complete. 2024-03-28T13:20:44.352 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-03-28T13:20:44.352 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-28T13:20:44.353 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:20:44.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.569+0000 7f4a77200640 1 Processor -- start 2024-03-28T13:20:44.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.569+0000 7f4a77200640 1 -- start start 2024-03-28T13:20:44.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.571+0000 7f4a77200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a780cc0b0 0x7f4a7806dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:44.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.571+0000 7f4a77200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4a7815fc50 0x7f4a780cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:44.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.571+0000 7f4a77200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a780cc490 0x7f4a780d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:44.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.571+0000 7f4a77200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4a7815c610 con 0x7f4a7815fc50 2024-03-28T13:20:44.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.571+0000 7f4a77200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f4a7806e4f0 con 0x7f4a780cc490 2024-03-28T13:20:44.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.571+0000 7f4a77200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4a780b3150 con 0x7f4a780cc0b0 2024-03-28T13:20:44.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.572+0000 7f4a75400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4a7815fc50 0x7f4a780cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:44.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.572+0000 7f4a75400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4a7815fc50 0x7f4a780cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52720/0 (socket says 172.21.15.19:52720) 2024-03-28T13:20:44.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.572+0000 7f4a75400640 1 -- 172.21.15.19:0/1387207638 learned_addr learned my addr 172.21.15.19:0/1387207638 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:44.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.572+0000 7f4a76800640 1 --2- 172.21.15.19:0/1387207638 >> v2:172.21.15.188:3300/0 conn(0x7f4a780cc490 0x7f4a780d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:44.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.572+0000 7f4a75e00640 1 --2- 172.21.15.19:0/1387207638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a780cc0b0 0x7f4a7806dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:44.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.572+0000 7f4a75400640 1 -- 172.21.15.19:0/1387207638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a780cc0b0 msgr2=0x7f4a7806dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:44.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.572+0000 7f4a75400640 1 --2- 172.21.15.19:0/1387207638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a780cc0b0 0x7f4a7806dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.572+0000 7f4a75400640 1 -- 172.21.15.19:0/1387207638 >> v2:172.21.15.188:3300/0 conn(0x7f4a780cc490 msgr2=0x7f4a780d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:44.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.572+0000 7f4a75400640 1 --2- 172.21.15.19:0/1387207638 >> v2:172.21.15.188:3300/0 conn(0x7f4a780cc490 0x7f4a780d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.572+0000 7f4a75400640 1 -- 172.21.15.19:0/1387207638 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a781600e0 con 0x7f4a7815fc50 2024-03-28T13:20:44.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.572+0000 7f4a76800640 1 --2- 172.21.15.19:0/1387207638 >> v2:172.21.15.188:3300/0 conn(0x7f4a780cc490 0x7f4a780d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:20:44.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.573+0000 7f4a75400640 1 --2- 172.21.15.19:0/1387207638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a7815fc50 0x7f4a780cfdc0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f4a780afab0 tx=0x7f4a6002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3dd6a6fabb5418 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:44.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.574+0000 7f4a74a00640 1 -- 172.21.15.19:0/1387207638 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60035020 con 0x7f4a7815fc50 2024-03-28T13:20:44.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.574+0000 7f4a74a00640 1 -- 172.21.15.19:0/1387207638 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a60031870 con 0x7f4a7815fc50 2024-03-28T13:20:44.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.574+0000 7f4a74a00640 1 -- 172.21.15.19:0/1387207638 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a60031c40 con 0x7f4a7815fc50 2024-03-28T13:20:44.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.574+0000 7f4a77200640 1 -- 172.21.15.19:0/1387207638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a7815fc50 msgr2=0x7f4a780cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:44.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.574+0000 7f4a77200640 1 --2- 172.21.15.19:0/1387207638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a7815fc50 0x7f4a780cfdc0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f4a780afab0 tx=0x7f4a6002b910 comp rx=0 tx=0).stop 2024-03-28T13:20:44.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.575+0000 7f4a77200640 1 -- 172.21.15.19:0/1387207638 shutdown_connections 2024-03-28T13:20:44.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.575+0000 7f4a77200640 1 --2- 172.21.15.19:0/1387207638 >> v2:172.21.15.188:3300/0 conn(0x7f4a780cc490 0x7f4a780d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.575+0000 7f4a77200640 1 --2- 172.21.15.19:0/1387207638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a7815fc50 0x7f4a780cfdc0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.575+0000 7f4a77200640 1 --2- 172.21.15.19:0/1387207638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a780cc0b0 0x7f4a7806dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.575+0000 7f4a77200640 1 -- 172.21.15.19:0/1387207638 >> 172.21.15.19:0/1387207638 conn(0x7f4a780ccad0 msgr2=0x7f4a780cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:44.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.575+0000 7f4a77200640 1 -- 172.21.15.19:0/1387207638 shutdown_connections 2024-03-28T13:20:44.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.575+0000 7f4a77200640 1 -- 172.21.15.19:0/1387207638 wait complete. 2024-03-28T13:20:44.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.576+0000 7f4a77200640 1 Processor -- start 2024-03-28T13:20:44.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.577+0000 7f4a77200640 1 -- start start 2024-03-28T13:20:44.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.577+0000 7f4a77200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a780cc0b0 0x7f4a78138ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:44.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.577+0000 7f4a77200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a780cc490 0x7f4a781e6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:44.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.578+0000 7f4a77200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a78139220 0x7f4a780b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:44.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.578+0000 7f4a77200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a78160980 con 0x7f4a78139220 2024-03-28T13:20:44.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.578+0000 7f4a77200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a781606a0 con 0x7f4a780cc0b0 2024-03-28T13:20:44.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.578+0000 7f4a77200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4a7815ca70 con 0x7f4a780cc490 2024-03-28T13:20:44.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.578+0000 7f4a76800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a78139220 0x7f4a780b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:44.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.578+0000 7f4a76800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a78139220 0x7f4a780b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52734/0 (socket says 172.21.15.19:52734) 2024-03-28T13:20:44.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.578+0000 7f4a76800640 1 -- 172.21.15.19:0/1474565618 learned_addr learned my addr 172.21.15.19:0/1474565618 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:44.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.578+0000 7f4a76800640 1 -- 172.21.15.19:0/1474565618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a780cc490 msgr2=0x7f4a781e6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:44.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.578+0000 7f4a76800640 1 --2- 172.21.15.19:0/1474565618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a780cc490 0x7f4a781e6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.578+0000 7f4a76800640 1 -- 172.21.15.19:0/1474565618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a780cc0b0 msgr2=0x7f4a78138ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:44.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.578+0000 7f4a76800640 1 --2- 172.21.15.19:0/1474565618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a780cc0b0 0x7f4a78138ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.578+0000 7f4a76800640 1 -- 172.21.15.19:0/1474565618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a781397e0 con 0x7f4a78139220 2024-03-28T13:20:44.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.579+0000 7f4a76800640 1 --2- 172.21.15.19:0/1474565618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a78139220 0x7f4a780b1ad0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009fd0 tx=0x7f4a7000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:44.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.579+0000 7f4a6e000640 1 -- 172.21.15.19:0/1474565618 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7001a070 con 0x7f4a78139220 2024-03-28T13:20:44.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.579+0000 7f4a6e000640 1 -- 172.21.15.19:0/1474565618 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a70003bb0 con 0x7f4a78139220 2024-03-28T13:20:44.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.579+0000 7f4a6e000640 1 -- 172.21.15.19:0/1474565618 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a70004030 con 0x7f4a78139220 2024-03-28T13:20:44.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.579+0000 7f4a77200640 1 -- 172.21.15.19:0/1474565618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a78139ac0 con 0x7f4a78139220 2024-03-28T13:20:44.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.579+0000 7f4a77200640 1 -- 172.21.15.19:0/1474565618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a7813a080 con 0x7f4a78139220 2024-03-28T13:20:44.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.580+0000 7f4a77200640 1 -- 172.21.15.19:0/1474565618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a78075940 con 0x7f4a78139220 2024-03-28T13:20:44.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.581+0000 7f4a6e000640 1 -- 172.21.15.19:0/1474565618 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4a7001d070 con 0x7f4a78139220 2024-03-28T13:20:44.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.582+0000 7f4a6e000640 1 --2- 172.21.15.19:0/1474565618 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4a5406c560 0x7f4a5406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:44.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.582+0000 7f4a6e000640 1 -- 172.21.15.19:0/1474565618 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5029+0+0 (secure 0 0 0) 0x7f4a7008b700 con 0x7f4a78139220 2024-03-28T13:20:44.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.583+0000 7f4a75e00640 1 --2- 172.21.15.19:0/1474565618 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4a5406c560 0x7f4a5406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:44.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.584+0000 7f4a75e00640 1 --2- 172.21.15.19:0/1474565618 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4a5406c560 0x7f4a5406ea20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4a680098f0 tx=0x7f4a68008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:44.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.587+0000 7f4a6e000640 1 -- 172.21.15.19:0/1474565618 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4a70010070 con 0x7f4a78139220 2024-03-28T13:20:44.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.792+0000 7f4a77200640 1 -- 172.21.15.19:0/1474565618 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a78073830 con 0x7f4a78139220 2024-03-28T13:20:44.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.793+0000 7f4a6e000640 1 -- 172.21.15.19:0/1474565618 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+13160 (secure 0 0 0) 0x7f4a70059fb0 con 0x7f4a78139220 2024-03-28T13:20:44.793 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:44.793 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":111,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:20:42.302001+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6834/2851646201":"2024-03-29T13:20:33.234624+0000","172.21.15.19:6832/2851646201":"2024-03-29T13:20:33.234624+0000","172.21.15.19:6834/1020599987":"2024-03-29T13:19:55.818562+0000","172.21.15.19:6832/1020599987":"2024-03-29T13:19:55.818562+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:20:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.796+0000 7f4a77200640 1 -- 172.21.15.19:0/1474565618 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4a5406c560 msgr2=0x7f4a5406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.796+0000 7f4a77200640 1 --2- 172.21.15.19:0/1474565618 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4a5406c560 0x7f4a5406ea20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4a680098f0 tx=0x7f4a68008040 comp rx=0 tx=0).stop 2024-03-28T13:20:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.796+0000 7f4a77200640 1 -- 172.21.15.19:0/1474565618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a78139220 msgr2=0x7f4a780b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.796+0000 7f4a77200640 1 --2- 172.21.15.19:0/1474565618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a78139220 0x7f4a780b1ad0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009fd0 tx=0x7f4a7000d560 comp rx=0 tx=0).stop 2024-03-28T13:20:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.796+0000 7f4a77200640 1 -- 172.21.15.19:0/1474565618 shutdown_connections 2024-03-28T13:20:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.796+0000 7f4a77200640 1 --2- 172.21.15.19:0/1474565618 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4a5406c560 0x7f4a5406ea20 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.796+0000 7f4a77200640 1 --2- 172.21.15.19:0/1474565618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a78139220 0x7f4a780b1ad0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.796+0000 7f4a77200640 1 --2- 172.21.15.19:0/1474565618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a780cc490 0x7f4a781e6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.796+0000 7f4a77200640 1 --2- 172.21.15.19:0/1474565618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a780cc0b0 0x7f4a78138ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.796+0000 7f4a77200640 1 -- 172.21.15.19:0/1474565618 >> 172.21.15.19:0/1474565618 conn(0x7f4a780ccad0 msgr2=0x7f4a7815f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.796+0000 7f4a77200640 1 -- 172.21.15.19:0/1474565618 shutdown_connections 2024-03-28T13:20:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:44.796+0000 7f4a77200640 1 -- 172.21.15.19:0/1474565618 wait complete. 2024-03-28T13:20:44.814 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:20:44.814 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:20:45.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.039+0000 7fecd2600640 1 Processor -- start 2024-03-28T13:20:45.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.040+0000 7fecd2600640 1 -- start start 2024-03-28T13:20:45.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.041+0000 7fecd2600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feccc0b69c0 0x7feccc168f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:45.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.041+0000 7fecd2600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feccc0d2710 con 0x7feccc15e4c0 2024-03-28T13:20:45.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.041+0000 7fecd2600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feccc0d29f0 con 0x7feccc0d37b0 2024-03-28T13:20:45.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.041+0000 7fecd2600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7feccc15e8a0 con 0x7feccc0b69c0 2024-03-28T13:20:45.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.042+0000 7fecd1c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feccc0b69c0 0x7feccc168f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:45.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.042+0000 7fecd1c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feccc0b69c0 0x7feccc168f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52742/0 (socket says 172.21.15.19:52742) 2024-03-28T13:20:45.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.042+0000 7fecd1c00640 1 -- 172.21.15.19:0/1189941171 learned_addr learned my addr 172.21.15.19:0/1189941171 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:45.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.042+0000 7fecd1200640 1 --1- 172.21.15.19:0/1189941171 >> v1:172.21.15.19:6789/0 conn(0x7feccc15e4c0 0x7feccc0b9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:47698/0 (socket says 172.21.15.19:47698) 2024-03-28T13:20:45.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.042+0000 7fecd1c00640 1 -- 172.21.15.19:0/1189941171 >> v1:172.21.15.188:6789/0 conn(0x7feccc0d37b0 legacy=0x7feccc0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:45.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.042+0000 7fecd1c00640 1 -- 172.21.15.19:0/1189941171 >> v1:172.21.15.19:6789/0 conn(0x7feccc15e4c0 legacy=0x7feccc0b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:45.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.042+0000 7feccb400640 1 -- 172.21.15.19:0/1189941171 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 379992932 0 0) 0x7feccc0d2710 con 0x7feccc15e4c0 2024-03-28T13:20:45.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.042+0000 7fecd1c00640 1 -- 172.21.15.19:0/1189941171 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc0d2cd0 con 0x7feccc0b69c0 2024-03-28T13:20:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.043+0000 7fecd1c00640 1 --2- 172.21.15.19:0/1189941171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0b69c0 0x7feccc168f70 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009870 tx=0x7fecc402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e72e03dd5f655df server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.044+0000 7feccb400640 1 -- 172.21.15.19:0/1189941171 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4034070 con 0x7feccc0b69c0 2024-03-28T13:20:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.044+0000 7feccb400640 1 -- 172.21.15.19:0/1189941171 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc4030860 con 0x7feccc0b69c0 2024-03-28T13:20:45.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.044+0000 7feccb400640 1 -- 172.21.15.19:0/1189941171 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc4030c30 con 0x7feccc0b69c0 2024-03-28T13:20:45.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.044+0000 7fecd2600640 1 -- 172.21.15.19:0/1189941171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0b69c0 msgr2=0x7feccc168f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:45.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.044+0000 7fecd2600640 1 --2- 172.21.15.19:0/1189941171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0b69c0 0x7feccc168f70 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009870 tx=0x7fecc402b9a0 comp rx=0 tx=0).stop 2024-03-28T13:20:45.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.044+0000 7fecd2600640 1 -- 172.21.15.19:0/1189941171 shutdown_connections 2024-03-28T13:20:45.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.044+0000 7fecd2600640 1 --2- 172.21.15.19:0/1189941171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0b69c0 0x7feccc168f70 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:45.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.044+0000 7fecd2600640 1 -- 172.21.15.19:0/1189941171 >> 172.21.15.19:0/1189941171 conn(0x7feccc0676a0 msgr2=0x7feccc067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:45.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.045+0000 7fecd2600640 1 -- 172.21.15.19:0/1189941171 shutdown_connections 2024-03-28T13:20:45.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.045+0000 7fecd2600640 1 -- 172.21.15.19:0/1189941171 wait complete. 2024-03-28T13:20:45.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.046+0000 7fecd2600640 1 Processor -- start 2024-03-28T13:20:45.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.046+0000 7fecd2600640 1 -- start start 2024-03-28T13:20:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.047+0000 7fecd2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0b69c0 0x7feccc154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.047+0000 7fecd2600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc0d37b0 0x7feccc0ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.047+0000 7fecd2600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc15e4c0 0x7feccc148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.047+0000 7fecd2600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feccc0d3320 con 0x7feccc0b69c0 2024-03-28T13:20:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.047+0000 7fecd2600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feccc0d4040 con 0x7feccc15e4c0 2024-03-28T13:20:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.047+0000 7fecd2600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feccc0d2350 con 0x7feccc0d37b0 2024-03-28T13:20:45.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.047+0000 7fecd1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0b69c0 0x7feccc154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.048+0000 7fecd1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0b69c0 0x7feccc154a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52750/0 (socket says 172.21.15.19:52750) 2024-03-28T13:20:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.048+0000 7fecd1200640 1 -- 172.21.15.19:0/2948405275 learned_addr learned my addr 172.21.15.19:0/2948405275 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.048+0000 7fecd1200640 1 -- 172.21.15.19:0/2948405275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc0d37b0 msgr2=0x7feccc0ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.048+0000 7fecd1200640 1 --2- 172.21.15.19:0/2948405275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc0d37b0 0x7feccc0ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.048+0000 7fecd1200640 1 -- 172.21.15.19:0/2948405275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc15e4c0 msgr2=0x7feccc148ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.048+0000 7fecd1c00640 1 --2- 172.21.15.19:0/2948405275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc15e4c0 0x7feccc148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.048+0000 7fecd1200640 1 --2- 172.21.15.19:0/2948405275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc15e4c0 0x7feccc148ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.048+0000 7fecd1200640 1 -- 172.21.15.19:0/2948405275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feccc0cb3a0 con 0x7feccc0b69c0 2024-03-28T13:20:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.048+0000 7fecd1c00640 1 --2- 172.21.15.19:0/2948405275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc15e4c0 0x7feccc148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.048+0000 7fecd1200640 1 --2- 172.21.15.19:0/2948405275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0b69c0 0x7feccc154a10 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c8c0 tx=0x7fecbc00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.049+0000 7fecc8c00640 1 -- 172.21.15.19:0/2948405275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc018070 con 0x7feccc0b69c0 2024-03-28T13:20:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.049+0000 7fecc8c00640 1 -- 172.21.15.19:0/2948405275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecbc00a640 con 0x7feccc0b69c0 2024-03-28T13:20:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.049+0000 7fecc8c00640 1 -- 172.21.15.19:0/2948405275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc00aa30 con 0x7feccc0b69c0 2024-03-28T13:20:45.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.049+0000 7fecd2600640 1 -- 172.21.15.19:0/2948405275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feccc0cb680 con 0x7feccc0b69c0 2024-03-28T13:20:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.049+0000 7fecd2600640 1 -- 172.21.15.19:0/2948405275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feccc14bbc0 con 0x7feccc0b69c0 2024-03-28T13:20:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.050+0000 7fecd2600640 1 -- 172.21.15.19:0/2948405275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feccc075940 con 0x7feccc0b69c0 2024-03-28T13:20:45.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.051+0000 7fecc8c00640 1 -- 172.21.15.19:0/2948405275 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fecbc00abd0 con 0x7feccc0b69c0 2024-03-28T13:20:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.052+0000 7fecc8c00640 1 --2- 172.21.15.19:0/2948405275 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feca806c530 0x7feca806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:45.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.052+0000 7fecc8c00640 1 -- 172.21.15.19:0/2948405275 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5029+0+0 (secure 0 0 0) 0x7fecbc08ab80 con 0x7feccc0b69c0 2024-03-28T13:20:45.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.053+0000 7feccbe00640 1 --2- 172.21.15.19:0/2948405275 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feca806c530 0x7feca806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:45.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.054+0000 7feccbe00640 1 --2- 172.21.15.19:0/2948405275 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feca806c530 0x7feca806e9f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7feccc0b4c60 tx=0x7fecb40079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:45.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.056+0000 7fecc8c00640 1 -- 172.21.15.19:0/2948405275 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fecbc059430 con 0x7feccc0b69c0 2024-03-28T13:20:45.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.264+0000 7fecd2600640 1 -- 172.21.15.19:0/2948405275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feccc0cb850 con 0x7feccc0b69c0 2024-03-28T13:20:45.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.264+0000 7fecc8c00640 1 -- 172.21.15.19:0/2948405275 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) ==== 93+0+10 (secure 0 0 0) 0x7fecbc058d30 con 0x7feccc0b69c0 2024-03-28T13:20:45.264 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:20:45.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.267+0000 7fecd2600640 1 -- 172.21.15.19:0/2948405275 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feca806c530 msgr2=0x7feca806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:45.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.267+0000 7fecd2600640 1 --2- 172.21.15.19:0/2948405275 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feca806c530 0x7feca806e9f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7feccc0b4c60 tx=0x7fecb40079b0 comp rx=0 tx=0).stop 2024-03-28T13:20:45.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.267+0000 7fecd2600640 1 -- 172.21.15.19:0/2948405275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0b69c0 msgr2=0x7feccc154a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:45.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.267+0000 7fecd2600640 1 --2- 172.21.15.19:0/2948405275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0b69c0 0x7feccc154a10 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c8c0 tx=0x7fecbc00cd80 comp rx=0 tx=0).stop 2024-03-28T13:20:45.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.267+0000 7fecd2600640 1 -- 172.21.15.19:0/2948405275 shutdown_connections 2024-03-28T13:20:45.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.267+0000 7fecd2600640 1 --2- 172.21.15.19:0/2948405275 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feca806c530 0x7feca806e9f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:45.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.267+0000 7fecd2600640 1 --2- 172.21.15.19:0/2948405275 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feccc15e4c0 0x7feccc148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:45.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.267+0000 7fecd2600640 1 --2- 172.21.15.19:0/2948405275 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feccc0d37b0 0x7feccc0ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:45.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.267+0000 7fecd2600640 1 --2- 172.21.15.19:0/2948405275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feccc0b69c0 0x7feccc154a10 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:45.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.267+0000 7fecd2600640 1 -- 172.21.15.19:0/2948405275 >> 172.21.15.19:0/2948405275 conn(0x7feccc0676a0 msgr2=0x7feccc15cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:45.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.267+0000 7fecd2600640 1 -- 172.21.15.19:0/2948405275 shutdown_connections 2024-03-28T13:20:45.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.268+0000 7fecd2600640 1 -- 172.21.15.19:0/2948405275 wait complete. 2024-03-28T13:20:45.285 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-03-28T13:20:45.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.505+0000 7f7ab6a00640 1 Processor -- start 2024-03-28T13:20:45.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.505+0000 7f7ab6a00640 1 -- start start 2024-03-28T13:20:45.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.506+0000 7f7ab6a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab0141c10 0x7f7ab0071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:45.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.506+0000 7f7ab6a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ab0071eb0 0x7f7ab0162160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:45.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.506+0000 7f7ab6a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ab015ec80 con 0x7f7ab00d0280 2024-03-28T13:20:45.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.506+0000 7f7ab6a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7ab015ef20 con 0x7f7ab0071eb0 2024-03-28T13:20:45.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.506+0000 7f7ab6a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab015f200 con 0x7f7ab0141c10 2024-03-28T13:20:45.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.507+0000 7f7ab4a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ab0071eb0 0x7f7ab0162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:45.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.507+0000 7f7ab4a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ab0071eb0 0x7f7ab0162160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57886/0 (socket says 172.21.15.19:57886) 2024-03-28T13:20:45.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.507+0000 7f7ab4a00640 1 -- 172.21.15.19:0/942476832 learned_addr learned my addr 172.21.15.19:0/942476832 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:45.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.507+0000 7f7aaf400640 1 --2- 172.21.15.19:0/942476832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab0141c10 0x7f7ab0071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:45.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.508+0000 7f7ab4a00640 1 -- 172.21.15.19:0/942476832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab0141c10 msgr2=0x7f7ab0071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:45.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.508+0000 7f7ab4a00640 1 --2- 172.21.15.19:0/942476832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab0141c10 0x7f7ab0071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:45.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.508+0000 7f7ab4a00640 1 -- 172.21.15.19:0/942476832 >> v1:172.21.15.19:6789/0 conn(0x7f7ab00d0280 legacy=0x7f7ab00d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:45.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.508+0000 7f7ab4a00640 1 -- 172.21.15.19:0/942476832 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab015f4e0 con 0x7f7ab0071eb0 2024-03-28T13:20:45.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.509+0000 7f7ab4a00640 1 --2- 172.21.15.19:0/942476832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab0071eb0 0x7f7ab0162160 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009e50 tx=0x7f7aa802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ca87de46128935 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:45.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.509+0000 7f7aaea00640 1 -- 172.21.15.19:0/942476832 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa802cb00 con 0x7f7ab0071eb0 2024-03-28T13:20:45.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.509+0000 7f7aaea00640 1 -- 172.21.15.19:0/942476832 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa802cca0 con 0x7f7ab0071eb0 2024-03-28T13:20:45.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.509+0000 7f7aaea00640 1 -- 172.21.15.19:0/942476832 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8034610 con 0x7f7ab0071eb0 2024-03-28T13:20:45.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.510+0000 7f7ab6a00640 1 -- 172.21.15.19:0/942476832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab0071eb0 msgr2=0x7f7ab0162160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:45.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.510+0000 7f7ab6a00640 1 --2- 172.21.15.19:0/942476832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab0071eb0 0x7f7ab0162160 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009e50 tx=0x7f7aa802bf60 comp rx=0 tx=0).stop 2024-03-28T13:20:45.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.510+0000 7f7ab6a00640 1 -- 172.21.15.19:0/942476832 shutdown_connections 2024-03-28T13:20:45.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.510+0000 7f7ab6a00640 1 --2- 172.21.15.19:0/942476832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab0071eb0 0x7f7ab0162160 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:45.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.510+0000 7f7ab6a00640 1 --2- 172.21.15.19:0/942476832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab0141c10 0x7f7ab0071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:45.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.510+0000 7f7ab6a00640 1 -- 172.21.15.19:0/942476832 >> 172.21.15.19:0/942476832 conn(0x7f7ab006b750 msgr2=0x7f7ab006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:45.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.510+0000 7f7ab6a00640 1 -- 172.21.15.19:0/942476832 shutdown_connections 2024-03-28T13:20:45.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.510+0000 7f7ab6a00640 1 -- 172.21.15.19:0/942476832 wait complete. 2024-03-28T13:20:45.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.512+0000 7f7ab6a00640 1 Processor -- start 2024-03-28T13:20:45.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.512+0000 7f7ab6a00640 1 -- start start 2024-03-28T13:20:45.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.513+0000 7f7ab6a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab0071eb0 0x7f7ab01f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:45.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.513+0000 7f7ab6a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab00d0280 0x7f7ab00ae600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.513+0000 7f7ab6a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab0141c10 0x7f7ab00aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.513+0000 7f7ab6a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab0158e90 con 0x7f7ab0071eb0 2024-03-28T13:20:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.513+0000 7f7ab6a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab006f340 con 0x7f7ab0141c10 2024-03-28T13:20:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.513+0000 7f7ab6a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab009a880 con 0x7f7ab00d0280 2024-03-28T13:20:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.513+0000 7f7aafe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab0071eb0 0x7f7ab01f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.513+0000 7f7aafe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab0071eb0 0x7f7ab01f4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57894/0 (socket says 172.21.15.19:57894) 2024-03-28T13:20:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.513+0000 7f7aafe00640 1 -- 172.21.15.19:0/1559587870 learned_addr learned my addr 172.21.15.19:0/1559587870 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.514+0000 7f7aaf400640 1 --2- 172.21.15.19:0/1559587870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab00d0280 0x7f7ab00ae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.514+0000 7f7ab4a00640 1 --2- 172.21.15.19:0/1559587870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab0141c10 0x7f7ab00aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.514+0000 7f7aafe00640 1 -- 172.21.15.19:0/1559587870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab00d0280 msgr2=0x7f7ab00ae600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.514+0000 7f7aafe00640 1 --2- 172.21.15.19:0/1559587870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab00d0280 0x7f7ab00ae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.514+0000 7f7aafe00640 1 -- 172.21.15.19:0/1559587870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab0141c10 msgr2=0x7f7ab00aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.514+0000 7f7aafe00640 1 --2- 172.21.15.19:0/1559587870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab0141c10 0x7f7ab00aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.514+0000 7f7aafe00640 1 -- 172.21.15.19:0/1559587870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab0147790 con 0x7f7ab0071eb0 2024-03-28T13:20:45.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.514+0000 7f7aafe00640 1 --2- 172.21.15.19:0/1559587870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab0071eb0 0x7f7ab01f4830 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000c430 tx=0x7f7aa000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:45.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.514+0000 7f7aa7e00640 1 -- 172.21.15.19:0/1559587870 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00052d0 con 0x7f7ab0071eb0 2024-03-28T13:20:45.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.514+0000 7f7aa7e00640 1 -- 172.21.15.19:0/1559587870 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa0005490 con 0x7f7ab0071eb0 2024-03-28T13:20:45.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.514+0000 7f7aa7e00640 1 -- 172.21.15.19:0/1559587870 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0007260 con 0x7f7ab0071eb0 2024-03-28T13:20:45.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.514+0000 7f7ab6a00640 1 -- 172.21.15.19:0/1559587870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab0147a70 con 0x7f7ab0071eb0 2024-03-28T13:20:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.515+0000 7f7ab6a00640 1 -- 172.21.15.19:0/1559587870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab009a510 con 0x7f7ab0071eb0 2024-03-28T13:20:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.516+0000 7f7ab6a00640 1 -- 172.21.15.19:0/1559587870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab00685c0 con 0x7f7ab0071eb0 2024-03-28T13:20:45.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.517+0000 7f7aa7e00640 1 -- 172.21.15.19:0/1559587870 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7aa0007400 con 0x7f7ab0071eb0 2024-03-28T13:20:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.518+0000 7f7aa7e00640 1 --2- 172.21.15.19:0/1559587870 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a9006c530 0x7f7a9006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:45.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.518+0000 7f7aa7e00640 1 -- 172.21.15.19:0/1559587870 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5029+0+0 (secure 0 0 0) 0x7f7aa008aa10 con 0x7f7ab0071eb0 2024-03-28T13:20:45.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.518+0000 7f7aaf400640 1 --2- 172.21.15.19:0/1559587870 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a9006c530 0x7f7a9006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:45.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.519+0000 7f7aaf400640 1 --2- 172.21.15.19:0/1559587870 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a9006c530 0x7f7a9006e9f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c006fd0 tx=0x7f7a9c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:45.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.522+0000 7f7aa7e00640 1 -- 172.21.15.19:0/1559587870 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7aa0059230 con 0x7f7ab0071eb0 2024-03-28T13:20:45.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:45.808+0000 7f7ab6a00640 1 -- 172.21.15.19:0/1559587870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f7ab00b1aa0 con 0x7f7ab0071eb0 2024-03-28T13:20:46.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.339+0000 7f7aa7e00640 1 -- 172.21.15.19:0/1559587870 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v527) ==== 147+0+0 (secure 0 0 0) 0x7f7aa0058b30 con 0x7f7ab0071eb0 2024-03-28T13:20:46.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.343+0000 7f7ab6a00640 1 -- 172.21.15.19:0/1559587870 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a9006c530 msgr2=0x7f7a9006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:46.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.343+0000 7f7ab6a00640 1 --2- 172.21.15.19:0/1559587870 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a9006c530 0x7f7a9006e9f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c006fd0 tx=0x7f7a9c008040 comp rx=0 tx=0).stop 2024-03-28T13:20:46.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.343+0000 7f7ab6a00640 1 -- 172.21.15.19:0/1559587870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab0071eb0 msgr2=0x7f7ab01f4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:46.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.343+0000 7f7ab6a00640 1 --2- 172.21.15.19:0/1559587870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab0071eb0 0x7f7ab01f4830 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000c430 tx=0x7f7aa000c8f0 comp rx=0 tx=0).stop 2024-03-28T13:20:46.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.343+0000 7f7ab6a00640 1 -- 172.21.15.19:0/1559587870 shutdown_connections 2024-03-28T13:20:46.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.343+0000 7f7ab6a00640 1 --2- 172.21.15.19:0/1559587870 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a9006c530 0x7f7a9006e9f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:46.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.343+0000 7f7ab6a00640 1 --2- 172.21.15.19:0/1559587870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ab0141c10 0x7f7ab00aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:46.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.343+0000 7f7ab6a00640 1 --2- 172.21.15.19:0/1559587870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ab00d0280 0x7f7ab00ae600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:46.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.343+0000 7f7ab6a00640 1 --2- 172.21.15.19:0/1559587870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ab0071eb0 0x7f7ab01f4830 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:46.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.343+0000 7f7ab6a00640 1 -- 172.21.15.19:0/1559587870 >> 172.21.15.19:0/1559587870 conn(0x7f7ab006b750 msgr2=0x7f7ab0160740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:46.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.343+0000 7f7ab6a00640 1 -- 172.21.15.19:0/1559587870 shutdown_connections 2024-03-28T13:20:46.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.343+0000 7f7ab6a00640 1 -- 172.21.15.19:0/1559587870 wait complete. 2024-03-28T13:20:46.367 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-28T13:20:46.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.596+0000 7fdb50c00640 1 Processor -- start 2024-03-28T13:20:46.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.597+0000 7fdb50c00640 1 -- start start 2024-03-28T13:20:46.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.598+0000 7fdb50c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb4c13f190 0x7fdb4c0cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:46.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.598+0000 7fdb50c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c06dd30 0x7fdb4c0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:46.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.598+0000 7fdb50c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb4c0ce890 con 0x7fdb4c0cc880 2024-03-28T13:20:46.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.598+0000 7fdb50c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdb4c0cd230 con 0x7fdb4c13f190 2024-03-28T13:20:46.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.598+0000 7fdb50c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdb4c0ceb50 con 0x7fdb4c06dd30 2024-03-28T13:20:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.598+0000 7fdb4b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdb4c13f190 0x7fdb4c0cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.599+0000 7fdb4aa00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdb4c0cc880 0x7fdb4c06d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41214/0 (socket says 172.21.15.19:41214) 2024-03-28T13:20:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.599+0000 7fdb4aa00640 1 -- 172.21.15.19:0/1089934976 learned_addr learned my addr 172.21.15.19:0/1089934976 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.599+0000 7fdb4be00640 1 --2- 172.21.15.19:0/1089934976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c06dd30 0x7fdb4c0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.599+0000 7fdb4b400640 1 -- 172.21.15.19:0/1089934976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c06dd30 msgr2=0x7fdb4c0cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.599+0000 7fdb4b400640 1 --2- 172.21.15.19:0/1089934976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c06dd30 0x7fdb4c0cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.599+0000 7fdb4b400640 1 -- 172.21.15.19:0/1089934976 >> v1:172.21.15.19:6789/0 conn(0x7fdb4c0cc880 legacy=0x7fdb4c06d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.599+0000 7fdb4b400640 1 -- 172.21.15.19:0/1089934976 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4c0cee30 con 0x7fdb4c13f190 2024-03-28T13:20:46.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.599+0000 7fdb4a000640 1 -- 172.21.15.19:0/1089934976 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2495010218 0 0) 0x7fdb4c0ce890 con 0x7fdb4c0cc880 2024-03-28T13:20:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.600+0000 7fdb4b400640 1 --2- 172.21.15.19:0/1089934976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c13f190 0x7fdb4c0cf630 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009d90 tx=0x7fdb3c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9863402b568936ee server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.601+0000 7fdb4a000640 1 -- 172.21.15.19:0/1089934976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c039070 con 0x7fdb4c13f190 2024-03-28T13:20:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.601+0000 7fdb4a000640 1 -- 172.21.15.19:0/1089934976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb3c02cc50 con 0x7fdb4c13f190 2024-03-28T13:20:46.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.601+0000 7fdb4a000640 1 -- 172.21.15.19:0/1089934976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c034440 con 0x7fdb4c13f190 2024-03-28T13:20:46.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.601+0000 7fdb50c00640 1 -- 172.21.15.19:0/1089934976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c13f190 msgr2=0x7fdb4c0cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:46.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.601+0000 7fdb50c00640 1 --2- 172.21.15.19:0/1089934976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c13f190 0x7fdb4c0cf630 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009d90 tx=0x7fdb3c02bea0 comp rx=0 tx=0).stop 2024-03-28T13:20:46.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.601+0000 7fdb50c00640 1 -- 172.21.15.19:0/1089934976 shutdown_connections 2024-03-28T13:20:46.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.601+0000 7fdb50c00640 1 --2- 172.21.15.19:0/1089934976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c06dd30 0x7fdb4c0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:46.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.601+0000 7fdb50c00640 1 --2- 172.21.15.19:0/1089934976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c13f190 0x7fdb4c0cf630 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:46.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.601+0000 7fdb50c00640 1 -- 172.21.15.19:0/1089934976 >> 172.21.15.19:0/1089934976 conn(0x7fdb4c0b1ba0 msgr2=0x7fdb4c0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:46.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.601+0000 7fdb50c00640 1 -- 172.21.15.19:0/1089934976 shutdown_connections 2024-03-28T13:20:46.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.601+0000 7fdb50c00640 1 -- 172.21.15.19:0/1089934976 wait complete. 2024-03-28T13:20:46.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.603+0000 7fdb50c00640 1 Processor -- start 2024-03-28T13:20:46.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.603+0000 7fdb50c00640 1 -- start start 2024-03-28T13:20:46.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.604+0000 7fdb50c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c06dd30 0x7fdb4c1558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:46.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.604+0000 7fdb50c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c0cc880 0x7fdb4c1f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:46.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.604+0000 7fdb50c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb4c13f190 0x7fdb4c1f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:46.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.604+0000 7fdb50c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c13f570 con 0x7fdb4c06dd30 2024-03-28T13:20:46.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.604+0000 7fdb50c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4c160c70 con 0x7fdb4c13f190 2024-03-28T13:20:46.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.604+0000 7fdb50c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdb4c0cfb70 con 0x7fdb4c0cc880 2024-03-28T13:20:46.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.605+0000 7fdb4b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c06dd30 0x7fdb4c1558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:46.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.605+0000 7fdb4b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c06dd30 0x7fdb4c1558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57908/0 (socket says 172.21.15.19:57908) 2024-03-28T13:20:46.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.605+0000 7fdb4b400640 1 -- 172.21.15.19:0/2434476149 learned_addr learned my addr 172.21.15.19:0/2434476149 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:46.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.605+0000 7fdb4aa00640 1 --2- 172.21.15.19:0/2434476149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c0cc880 0x7fdb4c1f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:46.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.605+0000 7fdb4b400640 1 -- 172.21.15.19:0/2434476149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c0cc880 msgr2=0x7fdb4c1f7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.605+0000 7fdb4b400640 1 --2- 172.21.15.19:0/2434476149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c0cc880 0x7fdb4c1f7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.605+0000 7fdb4b400640 1 -- 172.21.15.19:0/2434476149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb4c13f190 msgr2=0x7fdb4c1f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.605+0000 7fdb4b400640 1 --2- 172.21.15.19:0/2434476149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb4c13f190 0x7fdb4c1f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.605+0000 7fdb4b400640 1 -- 172.21.15.19:0/2434476149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4c1f8780 con 0x7fdb4c06dd30 2024-03-28T13:20:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.605+0000 7fdb4aa00640 1 --2- 172.21.15.19:0/2434476149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c0cc880 0x7fdb4c1f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.605+0000 7fdb4b400640 1 --2- 172.21.15.19:0/2434476149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c06dd30 0x7fdb4c1558d0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c037040 tx=0x7fdb3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.606+0000 7fdb43400640 1 -- 172.21.15.19:0/2434476149 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c039050 con 0x7fdb4c06dd30 2024-03-28T13:20:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.606+0000 7fdb43400640 1 -- 172.21.15.19:0/2434476149 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb3c03a040 con 0x7fdb4c06dd30 2024-03-28T13:20:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.606+0000 7fdb43400640 1 -- 172.21.15.19:0/2434476149 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c02ca50 con 0x7fdb4c06dd30 2024-03-28T13:20:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.606+0000 7fdb50c00640 1 -- 172.21.15.19:0/2434476149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4c1f84a0 con 0x7fdb4c06dd30 2024-03-28T13:20:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.606+0000 7fdb50c00640 1 -- 172.21.15.19:0/2434476149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb4c0b3320 con 0x7fdb4c06dd30 2024-03-28T13:20:46.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.607+0000 7fdb50c00640 1 -- 172.21.15.19:0/2434476149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb4c075940 con 0x7fdb4c06dd30 2024-03-28T13:20:46.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.608+0000 7fdb43400640 1 -- 172.21.15.19:0/2434476149 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdb3c047020 con 0x7fdb4c06dd30 2024-03-28T13:20:46.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.609+0000 7fdb43400640 1 --2- 172.21.15.19:0/2434476149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdb2806c530 0x7fdb2806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:46.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.609+0000 7fdb43400640 1 -- 172.21.15.19:0/2434476149 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5029+0+0 (secure 0 0 0) 0x7fdb3c0ad680 con 0x7fdb4c06dd30 2024-03-28T13:20:46.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.610+0000 7fdb4aa00640 1 --2- 172.21.15.19:0/2434476149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdb2806c530 0x7fdb2806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:46.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.611+0000 7fdb4aa00640 1 --2- 172.21.15.19:0/2434476149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdb2806c530 0x7fdb2806e9f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0a9b60 tx=0x7fdb34009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:46.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.613+0000 7fdb43400640 1 -- 172.21.15.19:0/2434476149 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdb3c04e020 con 0x7fdb4c06dd30 2024-03-28T13:20:46.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.817+0000 7fdb50c00640 1 -- 172.21.15.19:0/2434476149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fdb4c15d080 con 0x7fdb4c06dd30 2024-03-28T13:20:46.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.823+0000 7fdb43400640 1 -- 172.21.15.19:0/2434476149 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) ==== 122+0+0 (secure 0 0 0) 0x7fdb3c07bea0 con 0x7fdb4c06dd30 2024-03-28T13:20:46.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.825+0000 7fdb50c00640 1 -- 172.21.15.19:0/2434476149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdb2806c530 msgr2=0x7fdb2806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.825+0000 7fdb50c00640 1 --2- 172.21.15.19:0/2434476149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdb2806c530 0x7fdb2806e9f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0a9b60 tx=0x7fdb34009040 comp rx=0 tx=0).stop 2024-03-28T13:20:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.825+0000 7fdb50c00640 1 -- 172.21.15.19:0/2434476149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c06dd30 msgr2=0x7fdb4c1558d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.825+0000 7fdb50c00640 1 --2- 172.21.15.19:0/2434476149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c06dd30 0x7fdb4c1558d0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c037040 tx=0x7fdb3c004a40 comp rx=0 tx=0).stop 2024-03-28T13:20:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.826+0000 7fdb50c00640 1 -- 172.21.15.19:0/2434476149 shutdown_connections 2024-03-28T13:20:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.826+0000 7fdb50c00640 1 --2- 172.21.15.19:0/2434476149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdb2806c530 0x7fdb2806e9f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.826+0000 7fdb50c00640 1 --2- 172.21.15.19:0/2434476149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdb4c13f190 0x7fdb4c1f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.826+0000 7fdb50c00640 1 --2- 172.21.15.19:0/2434476149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdb4c0cc880 0x7fdb4c1f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.826+0000 7fdb50c00640 1 --2- 172.21.15.19:0/2434476149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb4c06dd30 0x7fdb4c1558d0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.826+0000 7fdb50c00640 1 -- 172.21.15.19:0/2434476149 >> 172.21.15.19:0/2434476149 conn(0x7fdb4c0b1ba0 msgr2=0x7fdb4c15f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.826+0000 7fdb50c00640 1 -- 172.21.15.19:0/2434476149 shutdown_connections 2024-03-28T13:20:46.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:46.826+0000 7fdb50c00640 1 -- 172.21.15.19:0/2434476149 wait complete. 2024-03-28T13:20:46.847 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-28T13:20:46.847 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:20:46.966 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:20:47.083 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-28T13:20:47.083 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:20:47.083 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:20:47.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:20:47.143 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:20:47.143 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:20:47.199 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:20:47.199 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:20:47.199 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:20:47.258 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:20:47.258 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:20:47.258 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:20:47.314 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:20:47.314 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-28T13:20:47.315 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:20:47.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.536+0000 7f5ae7800640 1 Processor -- start 2024-03-28T13:20:47.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.536+0000 7f5ae7800640 1 -- start start 2024-03-28T13:20:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.538+0000 7f5ae7800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5ae00af830 0x7f5ae0073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.538+0000 7f5ae7800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ae00d0650 0x7f5ae00d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.538+0000 7f5ae7800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ae015fe80 con 0x7f5ae015c390 2024-03-28T13:20:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.538+0000 7f5ae7800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5ae015c770 con 0x7f5ae00d0650 2024-03-28T13:20:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.538+0000 7f5ae7800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5ae00d0fc0 con 0x7f5ae00af830 2024-03-28T13:20:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.538+0000 7f5ae5800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ae00d0650 0x7f5ae00d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.538+0000 7f5ae5800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5ae00d0650 0x7f5ae00d0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57912/0 (socket says 172.21.15.19:57912) 2024-03-28T13:20:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.538+0000 7f5ae5800640 1 -- 172.21.15.19:0/245023019 learned_addr learned my addr 172.21.15.19:0/245023019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.539+0000 7f5ae5800640 1 -- 172.21.15.19:0/245023019 >> v2:172.21.15.188:3300/0 conn(0x7f5ae00af830 msgr2=0x7f5ae0073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.539+0000 7f5ae5800640 1 --2- 172.21.15.19:0/245023019 >> v2:172.21.15.188:3300/0 conn(0x7f5ae00af830 0x7f5ae0073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.539+0000 7f5ae5800640 1 -- 172.21.15.19:0/245023019 >> v1:172.21.15.188:6789/0 conn(0x7f5ae015c390 legacy=0x7f5ae0160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.539+0000 7f5ae5800640 1 -- 172.21.15.19:0/245023019 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae00ad120 con 0x7f5ae00d0650 2024-03-28T13:20:47.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.540+0000 7f5ae5800640 1 --2- 172.21.15.19:0/245023019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae00d0650 0x7f5ae00d0a80 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00b5450 tx=0x7f5ad802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bef112488f866d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:47.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.540+0000 7f5adfe00640 1 -- 172.21.15.19:0/245023019 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad802ca50 con 0x7f5ae00d0650 2024-03-28T13:20:47.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.540+0000 7f5adfe00640 1 -- 172.21.15.19:0/245023019 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad802cbf0 con 0x7f5ae00d0650 2024-03-28T13:20:47.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.540+0000 7f5adfe00640 1 -- 172.21.15.19:0/245023019 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad80343f0 con 0x7f5ae00d0650 2024-03-28T13:20:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.541+0000 7f5ae7800640 1 -- 172.21.15.19:0/245023019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae00d0650 msgr2=0x7f5ae00d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.541+0000 7f5ae7800640 1 --2- 172.21.15.19:0/245023019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae00d0650 0x7f5ae00d0a80 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f5ae00b5450 tx=0x7f5ad802beb0 comp rx=0 tx=0).stop 2024-03-28T13:20:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.541+0000 7f5ae7800640 1 -- 172.21.15.19:0/245023019 shutdown_connections 2024-03-28T13:20:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.541+0000 7f5ae7800640 1 --2- 172.21.15.19:0/245023019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae00d0650 0x7f5ae00d0a80 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.541+0000 7f5ae7800640 1 --2- 172.21.15.19:0/245023019 >> v2:172.21.15.188:3300/0 conn(0x7f5ae00af830 0x7f5ae0073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.541+0000 7f5ae7800640 1 -- 172.21.15.19:0/245023019 >> 172.21.15.19:0/245023019 conn(0x7f5ae006b990 msgr2=0x7f5ae006bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.541+0000 7f5ae7800640 1 -- 172.21.15.19:0/245023019 shutdown_connections 2024-03-28T13:20:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.541+0000 7f5ae7800640 1 -- 172.21.15.19:0/245023019 wait complete. 2024-03-28T13:20:47.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.543+0000 7f5ae7800640 1 Processor -- start 2024-03-28T13:20:47.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.543+0000 7f5ae7800640 1 -- start start 2024-03-28T13:20:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.544+0000 7f5ae7800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ae00af830 0x7f5ae00b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.544+0000 7f5ae7800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ae00d0650 0x7f5ae015f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.544+0000 7f5ae7800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae015c390 0x7f5ae0156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.544+0000 7f5ae7800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae0160120 con 0x7f5ae015c390 2024-03-28T13:20:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.544+0000 7f5ae7800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae00ad6e0 con 0x7f5ae00d0650 2024-03-28T13:20:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.544+0000 7f5ae7800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae0160d50 con 0x7f5ae00af830 2024-03-28T13:20:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.545+0000 7f5ae5800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae015c390 0x7f5ae0156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.545+0000 7f5ae5800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae015c390 0x7f5ae0156a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57924/0 (socket says 172.21.15.19:57924) 2024-03-28T13:20:47.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.545+0000 7f5ae5800640 1 -- 172.21.15.19:0/1370567537 learned_addr learned my addr 172.21.15.19:0/1370567537 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:47.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.545+0000 7f5ad7e00640 1 --2- 172.21.15.19:0/1370567537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ae00d0650 0x7f5ae015f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:47.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.545+0000 7f5ae5800640 1 -- 172.21.15.19:0/1370567537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ae00af830 msgr2=0x7f5ae00b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:47.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.545+0000 7f5ae4e00640 1 --2- 172.21.15.19:0/1370567537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ae00af830 0x7f5ae00b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:47.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.545+0000 7f5ae5800640 1 --2- 172.21.15.19:0/1370567537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ae00af830 0x7f5ae00b3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:47.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.545+0000 7f5ae5800640 1 -- 172.21.15.19:0/1370567537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ae00d0650 msgr2=0x7f5ae015f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:47.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.545+0000 7f5ae5800640 1 --2- 172.21.15.19:0/1370567537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ae00d0650 0x7f5ae015f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:47.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.545+0000 7f5ae5800640 1 -- 172.21.15.19:0/1370567537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae00b18b0 con 0x7f5ae015c390 2024-03-28T13:20:47.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.546+0000 7f5ae5800640 1 --2- 172.21.15.19:0/1370567537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae015c390 0x7f5ae0156a00 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8036040 tx=0x7f5ad8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:47.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.546+0000 7f5add600640 1 -- 172.21.15.19:0/1370567537 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8043020 con 0x7f5ae015c390 2024-03-28T13:20:47.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.546+0000 7f5add600640 1 -- 172.21.15.19:0/1370567537 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ad803e450 con 0x7f5ae015c390 2024-03-28T13:20:47.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.546+0000 7f5add600640 1 -- 172.21.15.19:0/1370567537 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad803e820 con 0x7f5ae015c390 2024-03-28T13:20:47.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.546+0000 7f5ae7800640 1 -- 172.21.15.19:0/1370567537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae00b1b90 con 0x7f5ae015c390 2024-03-28T13:20:47.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.546+0000 7f5ae7800640 1 -- 172.21.15.19:0/1370567537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae00b2150 con 0x7f5ae015c390 2024-03-28T13:20:47.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.547+0000 7f5ae7800640 1 -- 172.21.15.19:0/1370567537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae00685c0 con 0x7f5ae015c390 2024-03-28T13:20:47.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.548+0000 7f5add600640 1 -- 172.21.15.19:0/1370567537 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5ad8046020 con 0x7f5ae015c390 2024-03-28T13:20:47.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.549+0000 7f5add600640 1 --2- 172.21.15.19:0/1370567537 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5abc06c530 0x7f5abc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:47.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.549+0000 7f5add600640 1 -- 172.21.15.19:0/1370567537 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5029+0+0 (secure 0 0 0) 0x7f5ad80addf0 con 0x7f5ae015c390 2024-03-28T13:20:47.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.550+0000 7f5ae4e00640 1 --2- 172.21.15.19:0/1370567537 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5abc06c530 0x7f5abc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:47.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.551+0000 7f5ae4e00640 1 --2- 172.21.15.19:0/1370567537 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5abc06c530 0x7f5abc06e9f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0003d90 tx=0x7f5ad0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:47.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.557+0000 7f5add600640 1 -- 172.21.15.19:0/1370567537 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5ad804d030 con 0x7f5ae015c390 2024-03-28T13:20:47.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.805+0000 7f5ae7800640 1 -- 172.21.15.19:0/1370567537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ae006f1b0 con 0x7f5ae015c390 2024-03-28T13:20:47.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.807+0000 7f5add600640 1 -- 172.21.15.19:0/1370567537 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+3875 (secure 0 0 0) 0x7f5ad807c610 con 0x7f5ae015c390 2024-03-28T13:20:47.806 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:47.807 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6029,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/4247326257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4247326257},{"type":"v1","addr":"172.21.15.188:6836","nonce":4247326257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":6038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3452274156","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3452274156},{"type":"v1","addr":"172.21.15.19:6835","nonce":3452274156}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":6574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3383190607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3383190607},{"type":"v1","addr":"172.21.15.188:6837","nonce":3383190607}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":8538,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/3193953635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":3193953635},{"type":"v1","addr":"172.21.15.19:6834","nonce":3193953635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":90}],"filesystems":[]} 2024-03-28T13:20:47.807 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 92 2024-03-28T13:20:47.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.809+0000 7f5ae7800640 1 -- 172.21.15.19:0/1370567537 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5abc06c530 msgr2=0x7f5abc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:47.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.809+0000 7f5ae7800640 1 --2- 172.21.15.19:0/1370567537 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5abc06c530 0x7f5abc06e9f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0003d90 tx=0x7f5ad0009040 comp rx=0 tx=0).stop 2024-03-28T13:20:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.809+0000 7f5ae7800640 1 -- 172.21.15.19:0/1370567537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae015c390 msgr2=0x7f5ae0156a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.809+0000 7f5ae7800640 1 --2- 172.21.15.19:0/1370567537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae015c390 0x7f5ae0156a00 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8036040 tx=0x7f5ad8004a40 comp rx=0 tx=0).stop 2024-03-28T13:20:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.810+0000 7f5ae7800640 1 -- 172.21.15.19:0/1370567537 shutdown_connections 2024-03-28T13:20:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.810+0000 7f5ae7800640 1 --2- 172.21.15.19:0/1370567537 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5abc06c530 0x7f5abc06e9f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.810+0000 7f5ae7800640 1 --2- 172.21.15.19:0/1370567537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ae015c390 0x7f5ae0156a00 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.810+0000 7f5ae7800640 1 --2- 172.21.15.19:0/1370567537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ae00d0650 0x7f5ae015f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.810+0000 7f5ae7800640 1 --2- 172.21.15.19:0/1370567537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ae00af830 0x7f5ae00b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.810+0000 7f5ae7800640 1 -- 172.21.15.19:0/1370567537 >> 172.21.15.19:0/1370567537 conn(0x7f5ae006b990 msgr2=0x7f5ae014e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.810+0000 7f5ae7800640 1 -- 172.21.15.19:0/1370567537 shutdown_connections 2024-03-28T13:20:47.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:47.810+0000 7f5ae7800640 1 -- 172.21.15.19:0/1370567537 wait complete. 2024-03-28T13:20:47.829 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-28T13:20:47.829 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-28T13:20:47.829 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-28T13:20:47.829 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:20:47.829 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-28T13:20:47.829 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-28T13:20:47.830 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-28T13:20:47.830 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:20:47.830 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-28T13:20:47.830 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-28T13:20:47.830 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-28T13:20:47.830 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:20:47.830 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-28T13:20:47.831 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-28T13:20:47.831 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-28T13:20:47.831 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:20:47.845 INFO:tasks.ceph.mds.c:Stopped 2024-03-28T13:20:47.845 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-28T13:20:47.847 INFO:tasks.ceph.mds.a:Stopped 2024-03-28T13:20:47.847 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-28T13:20:47.859 INFO:tasks.ceph.mds.b:Stopped 2024-03-28T13:20:47.859 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-28T13:20:47.860 INFO:tasks.ceph.mds.d:Stopped 2024-03-28T13:20:47.860 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-28T13:20:47.894 INFO:tasks.ceph.mds.c:Started 2024-03-28T13:20:47.895 DEBUG:teuthology.parallel:result is None 2024-03-28T13:20:47.901 INFO:tasks.ceph.mds.a:Started 2024-03-28T13:20:47.901 DEBUG:teuthology.parallel:result is None 2024-03-28T13:20:47.905 INFO:tasks.ceph.mds.b:Started 2024-03-28T13:20:47.905 DEBUG:teuthology.parallel:result is None 2024-03-28T13:20:47.910 INFO:tasks.ceph.mds.d:Started 2024-03-28T13:20:47.911 DEBUG:teuthology.parallel:result is None 2024-03-28T13:20:47.911 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-28T13:20:48.060 INFO:tasks.ceph.mds.c.smithi019.stdout:starting mds.c at 2024-03-28T13:20:48.061 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-03-28T13:20:48.072 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-03-28T13:20:48.073 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-03-28T13:20:48.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.101+0000 7f7ae6400640 1 Processor -- start 2024-03-28T13:20:48.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.101+0000 7f7ae6400640 1 -- start start 2024-03-28T13:20:48.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.102+0000 7f7ae6400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ae0153580 0x7f7ae00d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:48.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.102+0000 7f7ae6400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae00af4b0 0x7f7ae01600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:48.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.102+0000 7f7ae6400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ae00afab0 con 0x7f7ae00d07c0 2024-03-28T13:20:48.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.102+0000 7f7ae6400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae00d0c30 con 0x7f7ae0153580 2024-03-28T13:20:48.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.102+0000 7f7ae6400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae015cc80 con 0x7f7ae00af4b0 2024-03-28T13:20:48.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.102+0000 7f7adf400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ae0153580 0x7f7ae00d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:48.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.102+0000 7f7adf400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7ae0153580 0x7f7ae00d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57950/0 (socket says 172.21.15.19:57950) 2024-03-28T13:20:48.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.102+0000 7f7adf400640 1 -- 172.21.15.19:0/4130393238 learned_addr learned my addr 172.21.15.19:0/4130393238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:48.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.102+0000 7f7adfe00640 1 --2- 172.21.15.19:0/4130393238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae00af4b0 0x7f7ae01600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:48.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.103+0000 7f7adf400640 1 -- 172.21.15.19:0/4130393238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae00af4b0 msgr2=0x7f7ae01600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:48.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.103+0000 7f7adf400640 1 --2- 172.21.15.19:0/4130393238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae00af4b0 0x7f7ae01600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.103+0000 7f7adf400640 1 -- 172.21.15.19:0/4130393238 >> v1:172.21.15.188:6789/0 conn(0x7f7ae00d07c0 legacy=0x7f7ae00aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:48.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.103+0000 7f7adf400640 1 -- 172.21.15.19:0/4130393238 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae015da00 con 0x7f7ae0153580 2024-03-28T13:20:48.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.104+0000 7f7adf400640 1 --2- 172.21.15.19:0/4130393238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae0153580 0x7f7ae00d0280 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009870 tx=0x7f7ad002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b494440f11b21fc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:48.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.104+0000 7f7ade000640 1 -- 172.21.15.19:0/4130393238 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0035020 con 0x7f7ae0153580 2024-03-28T13:20:48.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.104+0000 7f7ade000640 1 -- 172.21.15.19:0/4130393238 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad0031640 con 0x7f7ae0153580 2024-03-28T13:20:48.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.104+0000 7f7ade000640 1 -- 172.21.15.19:0/4130393238 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad002d040 con 0x7f7ae0153580 2024-03-28T13:20:48.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.105+0000 7f7ae6400640 1 -- 172.21.15.19:0/4130393238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae0153580 msgr2=0x7f7ae00d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:48.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.105+0000 7f7ae6400640 1 --2- 172.21.15.19:0/4130393238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae0153580 0x7f7ae00d0280 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0009870 tx=0x7f7ad002b8f0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.105+0000 7f7ae6400640 1 -- 172.21.15.19:0/4130393238 shutdown_connections 2024-03-28T13:20:48.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.105+0000 7f7ae6400640 1 --2- 172.21.15.19:0/4130393238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae00af4b0 0x7f7ae01600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.105+0000 7f7ae6400640 1 --2- 172.21.15.19:0/4130393238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae0153580 0x7f7ae00d0280 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.105+0000 7f7ae6400640 1 -- 172.21.15.19:0/4130393238 >> 172.21.15.19:0/4130393238 conn(0x7f7ae006b750 msgr2=0x7f7ae006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:48.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.105+0000 7f7ae6400640 1 -- 172.21.15.19:0/4130393238 shutdown_connections 2024-03-28T13:20:48.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.105+0000 7f7ae6400640 1 -- 172.21.15.19:0/4130393238 wait complete. 2024-03-28T13:20:48.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.107+0000 7f7ae6400640 1 Processor -- start 2024-03-28T13:20:48.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.107+0000 7f7ae6400640 1 -- start start 2024-03-28T13:20:48.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.108+0000 7f7ae6400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae00af4b0 0x7f7ae014f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:48.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.108+0000 7f7ae6400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae00d07c0 0x7f7ae00b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:48.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.108+0000 7f7ae6400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae0153580 0x7f7ae0151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.108+0000 7f7ae6400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae015d4e0 con 0x7f7ae00af4b0 2024-03-28T13:20:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.108+0000 7f7ae6400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae015d200 con 0x7f7ae00d07c0 2024-03-28T13:20:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.108+0000 7f7ae6400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae0153960 con 0x7f7ae0153580 2024-03-28T13:20:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.108+0000 7f7adf400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae00af4b0 0x7f7ae014f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.108+0000 7f7adf400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae00af4b0 0x7f7ae014f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57958/0 (socket says 172.21.15.19:57958) 2024-03-28T13:20:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.108+0000 7f7adf400640 1 -- 172.21.15.19:0/227190259 learned_addr learned my addr 172.21.15.19:0/227190259 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.108+0000 7f7adea00640 1 --2- 172.21.15.19:0/227190259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae00d07c0 0x7f7ae00b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.109+0000 7f7adfe00640 1 --2- 172.21.15.19:0/227190259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae0153580 0x7f7ae0151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.109+0000 7f7adf400640 1 -- 172.21.15.19:0/227190259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae0153580 msgr2=0x7f7ae0151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.109+0000 7f7adf400640 1 --2- 172.21.15.19:0/227190259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae0153580 0x7f7ae0151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.109+0000 7f7adf400640 1 -- 172.21.15.19:0/227190259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae00d07c0 msgr2=0x7f7ae00b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.109+0000 7f7adf400640 1 --2- 172.21.15.19:0/227190259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae00d07c0 0x7f7ae00b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.109+0000 7f7adf400640 1 -- 172.21.15.19:0/227190259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae0071aa0 con 0x7f7ae00af4b0 2024-03-28T13:20:48.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.109+0000 7f7adfe00640 1 --2- 172.21.15.19:0/227190259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae0153580 0x7f7ae0151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:48.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.109+0000 7f7adf400640 1 --2- 172.21.15.19:0/227190259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae00af4b0 0x7f7ae014f8f0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0000c00 tx=0x7f7ad00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:48.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.109+0000 7f7ad7400640 1 -- 172.21.15.19:0/227190259 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0040070 con 0x7f7ae00af4b0 2024-03-28T13:20:48.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.109+0000 7f7ad7400640 1 -- 172.21.15.19:0/227190259 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad00399f0 con 0x7f7ae00af4b0 2024-03-28T13:20:48.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.109+0000 7f7ad7400640 1 -- 172.21.15.19:0/227190259 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad0035040 con 0x7f7ae00af4b0 2024-03-28T13:20:48.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.109+0000 7f7ae6400640 1 -- 172.21.15.19:0/227190259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae0071d80 con 0x7f7ae00af4b0 2024-03-28T13:20:48.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.110+0000 7f7ae6400640 1 -- 172.21.15.19:0/227190259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae0072340 con 0x7f7ae00af4b0 2024-03-28T13:20:48.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.111+0000 7f7ae6400640 1 -- 172.21.15.19:0/227190259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae00685c0 con 0x7f7ae00af4b0 2024-03-28T13:20:48.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.112+0000 7f7ad7400640 1 -- 172.21.15.19:0/227190259 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7ad004b060 con 0x7f7ae00af4b0 2024-03-28T13:20:48.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.112+0000 7f7ad7400640 1 --2- 172.21.15.19:0/227190259 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ac006c530 0x7f7ac006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:48.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.113+0000 7f7ad7400640 1 -- 172.21.15.19:0/227190259 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5029+0+0 (secure 0 0 0) 0x7f7ad00acdc0 con 0x7f7ae00af4b0 2024-03-28T13:20:48.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.113+0000 7f7adea00640 1 --2- 172.21.15.19:0/227190259 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ac006c530 0x7f7ac006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:48.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.114+0000 7f7adea00640 1 --2- 172.21.15.19:0/227190259 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ac006c530 0x7f7ac006e9f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7acc007fd0 tx=0x7f7acc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:48.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.117+0000 7f7ad7400640 1 -- 172.21.15.19:0/227190259 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7ad007b5e0 con 0x7f7ae00af4b0 2024-03-28T13:20:48.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.331+0000 7f7ae6400640 1 -- 172.21.15.19:0/227190259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7ae015f5f0 con 0x7f7ae00af4b0 2024-03-28T13:20:48.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.368+0000 7f7ad7400640 1 -- 172.21.15.19:0/227190259 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v113) ==== 97+0+0 (secure 0 0 0) 0x7f7ad007aee0 con 0x7f7ae00af4b0 2024-03-28T13:20:48.368 INFO:teuthology.orchestra.run.smithi019.stderr: removed all blocklist entries 2024-03-28T13:20:48.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.370+0000 7f7ae6400640 1 -- 172.21.15.19:0/227190259 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ac006c530 msgr2=0x7f7ac006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:48.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.370+0000 7f7ae6400640 1 --2- 172.21.15.19:0/227190259 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ac006c530 0x7f7ac006e9f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7acc007fd0 tx=0x7f7acc009040 comp rx=0 tx=0).stop 2024-03-28T13:20:48.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.370+0000 7f7ae6400640 1 -- 172.21.15.19:0/227190259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae00af4b0 msgr2=0x7f7ae014f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:48.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.370+0000 7f7ae6400640 1 --2- 172.21.15.19:0/227190259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae00af4b0 0x7f7ae014f8f0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f7ad0000c00 tx=0x7f7ad00046c0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.371+0000 7f7ae6400640 1 -- 172.21.15.19:0/227190259 shutdown_connections 2024-03-28T13:20:48.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.371+0000 7f7ae6400640 1 --2- 172.21.15.19:0/227190259 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7ac006c530 0x7f7ac006e9f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.371+0000 7f7ae6400640 1 --2- 172.21.15.19:0/227190259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae0153580 0x7f7ae0151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.371+0000 7f7ae6400640 1 --2- 172.21.15.19:0/227190259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae00d07c0 0x7f7ae00b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.371+0000 7f7ae6400640 1 --2- 172.21.15.19:0/227190259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ae00af4b0 0x7f7ae014f8f0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.371+0000 7f7ae6400640 1 -- 172.21.15.19:0/227190259 >> 172.21.15.19:0/227190259 conn(0x7f7ae006b750 msgr2=0x7f7ae015ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:48.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.371+0000 7f7ae6400640 1 -- 172.21.15.19:0/227190259 shutdown_connections 2024-03-28T13:20:48.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.371+0000 7f7ae6400640 1 -- 172.21.15.19:0/227190259 wait complete. 2024-03-28T13:20:48.390 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-28T13:20:48.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.590+0000 7f1b5d800640 1 Processor -- start 2024-03-28T13:20:48.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.590+0000 7f1b5d800640 1 -- start start 2024-03-28T13:20:48.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.591+0000 7f1b5d800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1b58153580 0x7f1b580d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:48.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.592+0000 7f1b5d800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1b580af4b0 0x7f1b581600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:48.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.592+0000 7f1b5d800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b580afab0 con 0x7f1b580d07c0 2024-03-28T13:20:48.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.592+0000 7f1b5d800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1b580d0c30 con 0x7f1b58153580 2024-03-28T13:20:48.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.592+0000 7f1b5d800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1b5815cc80 con 0x7f1b580af4b0 2024-03-28T13:20:48.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.592+0000 7f1b56a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1b58153580 0x7f1b580d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:48.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.592+0000 7f1b4e000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1b580d07c0 0x7f1b580aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41216/0 (socket says 172.21.15.19:41216) 2024-03-28T13:20:48.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.592+0000 7f1b4e000640 1 -- 172.21.15.19:0/1542189041 learned_addr learned my addr 172.21.15.19:0/1542189041 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:48.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.592+0000 7f1b56a00640 1 -- 172.21.15.19:0/1542189041 >> v2:172.21.15.188:3300/0 conn(0x7f1b580af4b0 msgr2=0x7f1b581600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:48.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.592+0000 7f1b56a00640 1 --2- 172.21.15.19:0/1542189041 >> v2:172.21.15.188:3300/0 conn(0x7f1b580af4b0 0x7f1b581600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.592+0000 7f1b56a00640 1 -- 172.21.15.19:0/1542189041 >> v1:172.21.15.19:6789/0 conn(0x7f1b580d07c0 legacy=0x7f1b580aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:48.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.592+0000 7f1b56a00640 1 -- 172.21.15.19:0/1542189041 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5815da00 con 0x7f1b58153580 2024-03-28T13:20:48.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.592+0000 7f1b56000640 1 -- 172.21.15.19:0/1542189041 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 210939528 0 0) 0x7f1b580afab0 con 0x7f1b580d07c0 2024-03-28T13:20:48.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.594+0000 7f1b56a00640 1 --2- 172.21.15.19:0/1542189041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58153580 0x7f1b580d0280 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f1b5815a6b0 tx=0x7f1b4802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecc6c7102969380 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:48.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.594+0000 7f1b56000640 1 -- 172.21.15.19:0/1542189041 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48039070 con 0x7f1b58153580 2024-03-28T13:20:48.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.594+0000 7f1b56000640 1 -- 172.21.15.19:0/1542189041 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4802cbe0 con 0x7f1b58153580 2024-03-28T13:20:48.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.594+0000 7f1b56000640 1 -- 172.21.15.19:0/1542189041 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b48034440 con 0x7f1b58153580 2024-03-28T13:20:48.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.594+0000 7f1b5d800640 1 -- 172.21.15.19:0/1542189041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58153580 msgr2=0x7f1b580d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:48.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.594+0000 7f1b5d800640 1 --2- 172.21.15.19:0/1542189041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58153580 0x7f1b580d0280 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f1b5815a6b0 tx=0x7f1b4802be30 comp rx=0 tx=0).stop 2024-03-28T13:20:48.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.595+0000 7f1b5d800640 1 -- 172.21.15.19:0/1542189041 shutdown_connections 2024-03-28T13:20:48.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.595+0000 7f1b5d800640 1 --2- 172.21.15.19:0/1542189041 >> v2:172.21.15.188:3300/0 conn(0x7f1b580af4b0 0x7f1b581600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.595+0000 7f1b5d800640 1 --2- 172.21.15.19:0/1542189041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58153580 0x7f1b580d0280 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.595+0000 7f1b5d800640 1 -- 172.21.15.19:0/1542189041 >> 172.21.15.19:0/1542189041 conn(0x7f1b5806b750 msgr2=0x7f1b5806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:48.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.595+0000 7f1b5d800640 1 -- 172.21.15.19:0/1542189041 shutdown_connections 2024-03-28T13:20:48.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.595+0000 7f1b5d800640 1 -- 172.21.15.19:0/1542189041 wait complete. 2024-03-28T13:20:48.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.596+0000 7f1b5d800640 1 Processor -- start 2024-03-28T13:20:48.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.596+0000 7f1b5d800640 1 -- start start 2024-03-28T13:20:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.597+0000 7f1b5d800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b580af4b0 0x7f1b581484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.597+0000 7f1b5d800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b580d07c0 0x7f1b580cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.597+0000 7f1b5d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58153580 0x7f1b580b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.597+0000 7f1b5d800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5815d4e0 con 0x7f1b58153580 2024-03-28T13:20:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.597+0000 7f1b5d800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5815d200 con 0x7f1b580d07c0 2024-03-28T13:20:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.597+0000 7f1b5d800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1b58153960 con 0x7f1b580af4b0 2024-03-28T13:20:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.598+0000 7f1b57400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58153580 0x7f1b580b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:48.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.598+0000 7f1b57400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58153580 0x7f1b580b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57974/0 (socket says 172.21.15.19:57974) 2024-03-28T13:20:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.598+0000 7f1b57400640 1 -- 172.21.15.19:0/735708279 learned_addr learned my addr 172.21.15.19:0/735708279 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.598+0000 7f1b4e000640 1 --2- 172.21.15.19:0/735708279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b580d07c0 0x7f1b580cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.598+0000 7f1b56a00640 1 --2- 172.21.15.19:0/735708279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b580af4b0 0x7f1b581484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.598+0000 7f1b57400640 1 -- 172.21.15.19:0/735708279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b580af4b0 msgr2=0x7f1b581484d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.598+0000 7f1b57400640 1 --2- 172.21.15.19:0/735708279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b580af4b0 0x7f1b581484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.598+0000 7f1b57400640 1 -- 172.21.15.19:0/735708279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b580d07c0 msgr2=0x7f1b580cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.598+0000 7f1b57400640 1 --2- 172.21.15.19:0/735708279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b580d07c0 0x7f1b580cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.598+0000 7f1b57400640 1 -- 172.21.15.19:0/735708279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b580c9600 con 0x7f1b58153580 2024-03-28T13:20:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.598+0000 7f1b4e000640 1 --2- 172.21.15.19:0/735708279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b580d07c0 0x7f1b580cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:48.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.598+0000 7f1b57400640 1 --2- 172.21.15.19:0/735708279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58153580 0x7f1b580b4410 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000b410 tx=0x7f1b5000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:48.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.599+0000 7f1b4f400640 1 -- 172.21.15.19:0/735708279 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50015020 con 0x7f1b58153580 2024-03-28T13:20:48.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.599+0000 7f1b4f400640 1 -- 172.21.15.19:0/735708279 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b50011850 con 0x7f1b58153580 2024-03-28T13:20:48.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.599+0000 7f1b5d800640 1 -- 172.21.15.19:0/735708279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b580c98e0 con 0x7f1b58153580 2024-03-28T13:20:48.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.599+0000 7f1b4f400640 1 -- 172.21.15.19:0/735708279 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b50011c40 con 0x7f1b58153580 2024-03-28T13:20:48.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.599+0000 7f1b5d800640 1 -- 172.21.15.19:0/735708279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b580c9ea0 con 0x7f1b58153580 2024-03-28T13:20:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.600+0000 7f1b5d800640 1 -- 172.21.15.19:0/735708279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b580685c0 con 0x7f1b58153580 2024-03-28T13:20:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.601+0000 7f1b4f400640 1 -- 172.21.15.19:0/735708279 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1b50018020 con 0x7f1b58153580 2024-03-28T13:20:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.602+0000 7f1b4f400640 1 --2- 172.21.15.19:0/735708279 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1b3406c530 0x7f1b3406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.602+0000 7f1b4f400640 1 -- 172.21.15.19:0/735708279 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 4857+0+0 (secure 0 0 0) 0x7f1b5008b420 con 0x7f1b58153580 2024-03-28T13:20:48.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.602+0000 7f1b56a00640 1 --2- 172.21.15.19:0/735708279 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1b3406c530 0x7f1b3406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:48.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.604+0000 7f1b56a00640 1 --2- 172.21.15.19:0/735708279 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1b3406c530 0x7f1b3406e9f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1b48036000 tx=0x7f1b48030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:48.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.606+0000 7f1b4f400640 1 -- 172.21.15.19:0/735708279 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1b5001f020 con 0x7f1b58153580 2024-03-28T13:20:48.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.870+0000 7f1b5d800640 1 -- 172.21.15.19:0/735708279 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1b5806eca0 con 0x7f1b58153580 2024-03-28T13:20:48.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.871+0000 7f1b4f400640 1 -- 172.21.15.19:0/735708279 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f1b500172c0 con 0x7f1b58153580 2024-03-28T13:20:48.871 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:48.872 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:20:48.872 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_dump": [ 2024-03-28T13:20:48.872 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.872 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.a", 2024-03-28T13:20:48.872 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmNRIkABAAq/nJioDZjp1zFWAYOLPBYw==", 2024-03-28T13:20:48.872 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.872 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:20:48.873 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:20:48.873 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:20:48.873 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:20:48.873 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.873 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.873 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.b", 2024-03-28T13:20:48.873 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmc10CIxAAQWCSjH0hW3KUk8/qQm91QA==", 2024-03-28T13:20:48.874 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.874 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:20:48.874 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:20:48.874 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:20:48.874 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:20:48.874 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.875 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.875 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.c", 2024-03-28T13:20:48.875 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmmlUoEhAA4FUGXoZ/h/ve5fThJNdE2Q==", 2024-03-28T13:20:48.875 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.875 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:20:48.875 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:20:48.875 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:20:48.876 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:20:48.876 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.876 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.876 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mds.d", 2024-03-28T13:20:48.876 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDTawVmWDRnMxAA34LiLI79N5fVJnbcPacbQA==", 2024-03-28T13:20:48.876 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.876 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:20:48.877 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:20:48.877 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:20:48.877 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:20:48.877 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.877 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.877 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.877 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.0", 2024-03-28T13:20:48.878 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDYawVmxhwWLBAAm7C56UGConFx2D7Lbq2SSA==", 2024-03-28T13:20:48.878 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.878 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:20:48.878 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:20:48.878 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:20:48.878 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.879 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.879 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.1", 2024-03-28T13:20:48.879 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDbawVmC2BiNxAALQrAcXU44m2dTCjks5yy/Q==", 2024-03-28T13:20:48.879 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.879 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:20:48.879 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:20:48.879 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:20:48.880 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.880 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.880 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.2", 2024-03-28T13:20:48.880 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDfawVmozrJFRAA7YV1uMucIW7H5SI4KY8aDw==", 2024-03-28T13:20:48.880 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.880 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:20:48.880 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:20:48.881 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:20:48.881 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.881 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.881 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.881 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.3", 2024-03-28T13:20:48.881 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDiawVmccbTMBAATxk24Bf4LuQSqU92HdwgUg==", 2024-03-28T13:20:48.881 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.881 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:20:48.882 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:20:48.882 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:20:48.882 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.882 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.882 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.4", 2024-03-28T13:20:48.882 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDpawVmdwnILRAAjUEBwnBEeSltu2CLRGhmDQ==", 2024-03-28T13:20:48.882 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.882 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:20:48.883 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:20:48.883 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:20:48.883 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.883 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.883 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.5", 2024-03-28T13:20:48.883 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDtawVmkwW5ARAAZJ/Nd/Wtq7LcbUjeqPNUag==", 2024-03-28T13:20:48.883 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.883 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:20:48.884 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:20:48.884 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:20:48.884 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.884 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.884 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.6", 2024-03-28T13:20:48.885 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDwawVm254wKRAAmCYODqSd2Whn4YKQzywrng==", 2024-03-28T13:20:48.885 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.885 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:20:48.885 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:20:48.885 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:20:48.885 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.885 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.885 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "osd.7", 2024-03-28T13:20:48.886 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDzawVmV8z+MhAA9JeF55K7ph1mQRWalT7XhA==", 2024-03-28T13:20:48.886 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.886 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow profile osd", 2024-03-28T13:20:48.886 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile osd", 2024-03-28T13:20:48.886 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:20:48.886 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.886 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.887 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.0", 2024-03-28T13:20:48.887 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q==", 2024-03-28T13:20:48.887 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.887 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:20:48.887 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow r", 2024-03-28T13:20:48.887 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-28T13:20:48.888 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.888 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.888 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.1", 2024-03-28T13:20:48.888 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmJYGjHBAAAx2TtR0Gb3fafnZU26FCMA==", 2024-03-28T13:20:48.888 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.888 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:20:48.888 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:20:48.889 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:20:48.889 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:20:48.889 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.889 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.889 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.889 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.2", 2024-03-28T13:20:48.889 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmE7V/ERAAYcdvolIMDd5ff+pCn7hN1w==", 2024-03-28T13:20:48.889 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.890 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:20:48.890 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:20:48.890 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:20:48.890 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:20:48.890 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.890 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.890 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.3", 2024-03-28T13:20:48.890 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDUawVmk6q9JhAAeC3JHQDnK8IXGVoK6qGEFA==", 2024-03-28T13:20:48.891 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.891 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow", 2024-03-28T13:20:48.891 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow r", 2024-03-28T13:20:48.891 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow rw", 2024-03-28T13:20:48.891 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow rwx" 2024-03-28T13:20:48.891 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.891 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.892 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "client.admin", 2024-03-28T13:20:48.892 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDRawVm9IzFLRAA5i1Yl7LQ7eNNHp7v4H2kog==", 2024-03-28T13:20:48.892 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.892 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:20:48.892 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": "allow *", 2024-03-28T13:20:48.892 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow *", 2024-03-28T13:20:48.892 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:20:48.892 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.893 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.893 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.x", 2024-03-28T13:20:48.893 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDSawVm5H1FLxAAtyjOPcCEUQyvTpBdu3Fn3g==", 2024-03-28T13:20:48.893 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.893 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:20:48.893 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-03-28T13:20:48.893 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:20:48.893 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-03-28T13:20:48.894 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:48.894 INFO:teuthology.orchestra.run.smithi019.stdout: "entity": "mgr.y", 2024-03-28T13:20:48.894 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "AQDSawVmebzfIRAAA0+qIIaprr4SfEZT6qRooA==", 2024-03-28T13:20:48.894 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": { 2024-03-28T13:20:48.894 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": "allow *", 2024-03-28T13:20:48.894 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": "allow profile mgr", 2024-03-28T13:20:48.894 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": "allow *" 2024-03-28T13:20:48.894 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.894 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:48.895 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-03-28T13:20:48.895 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:20:48.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.873+0000 7f1b5d800640 1 -- 172.21.15.19:0/735708279 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1b3406c530 msgr2=0x7f1b3406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:48.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.873+0000 7f1b5d800640 1 --2- 172.21.15.19:0/735708279 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1b3406c530 0x7f1b3406e9f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1b48036000 tx=0x7f1b48030000 comp rx=0 tx=0).stop 2024-03-28T13:20:48.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.873+0000 7f1b5d800640 1 -- 172.21.15.19:0/735708279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58153580 msgr2=0x7f1b580b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:48.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.873+0000 7f1b5d800640 1 --2- 172.21.15.19:0/735708279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58153580 0x7f1b580b4410 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000b410 tx=0x7f1b5000b8d0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.873+0000 7f1b5d800640 1 -- 172.21.15.19:0/735708279 shutdown_connections 2024-03-28T13:20:48.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.873+0000 7f1b5d800640 1 --2- 172.21.15.19:0/735708279 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1b3406c530 0x7f1b3406e9f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.873+0000 7f1b5d800640 1 --2- 172.21.15.19:0/735708279 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1b58153580 0x7f1b580b4410 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.873+0000 7f1b5d800640 1 --2- 172.21.15.19:0/735708279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1b580d07c0 0x7f1b580cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.873+0000 7f1b5d800640 1 --2- 172.21.15.19:0/735708279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1b580af4b0 0x7f1b581484d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:48.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.873+0000 7f1b5d800640 1 -- 172.21.15.19:0/735708279 >> 172.21.15.19:0/735708279 conn(0x7f1b5806b750 msgr2=0x7f1b580b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:48.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.874+0000 7f1b5d800640 1 -- 172.21.15.19:0/735708279 shutdown_connections 2024-03-28T13:20:48.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:48.874+0000 7f1b5d800640 1 -- 172.21.15.19:0/735708279 wait complete. 2024-03-28T13:20:48.898 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:20:49.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.113+0000 7f85afe00640 1 Processor -- start 2024-03-28T13:20:49.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.113+0000 7f85afe00640 1 -- start start 2024-03-28T13:20:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.114+0000 7f85afe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f85b00cc0b0 0x7f85b006dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.115+0000 7f85afe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b006e4f0 con 0x7f85b00d04d0 2024-03-28T13:20:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.115+0000 7f85afe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b00b3150 con 0x7f85b015fc50 2024-03-28T13:20:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.115+0000 7f85afe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f85b015c610 con 0x7f85b00cc0b0 2024-03-28T13:20:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.115+0000 7f85aea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f85b00cc0b0 0x7f85b006dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.115+0000 7f85af400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f85b00d04d0 0x7f85b015ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41230/0 (socket says 172.21.15.19:41230) 2024-03-28T13:20:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.115+0000 7f85af400640 1 -- 172.21.15.19:0/1586428434 learned_addr learned my addr 172.21.15.19:0/1586428434 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.115+0000 7f85aea00640 1 -- 172.21.15.19:0/1586428434 >> v1:172.21.15.188:6789/0 conn(0x7f85b015fc50 legacy=0x7f85b00cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.115+0000 7f85aea00640 1 -- 172.21.15.19:0/1586428434 >> v1:172.21.15.19:6789/0 conn(0x7f85b00d04d0 legacy=0x7f85b015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.115+0000 7f85aea00640 1 -- 172.21.15.19:0/1586428434 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b01600e0 con 0x7f85b00cc0b0 2024-03-28T13:20:49.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.117+0000 7f85aea00640 1 --2- 172.21.15.19:0/1586428434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b00cc0b0 0x7f85b006dfb0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f85a00097a0 tx=0x7f85a002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c60df11063b731 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:49.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.117+0000 7f85ae000640 1 -- 172.21.15.19:0/1586428434 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a00315a0 con 0x7f85b00cc0b0 2024-03-28T13:20:49.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.117+0000 7f85ae000640 1 -- 172.21.15.19:0/1586428434 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a0037070 con 0x7f85b00cc0b0 2024-03-28T13:20:49.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.117+0000 7f85ae000640 1 -- 172.21.15.19:0/1586428434 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0031960 con 0x7f85b00cc0b0 2024-03-28T13:20:49.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.117+0000 7f85afe00640 1 -- 172.21.15.19:0/1586428434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b00cc0b0 msgr2=0x7f85b006dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:49.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.117+0000 7f85afe00640 1 --2- 172.21.15.19:0/1586428434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b00cc0b0 0x7f85b006dfb0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f85a00097a0 tx=0x7f85a002b840 comp rx=0 tx=0).stop 2024-03-28T13:20:49.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.118+0000 7f85afe00640 1 -- 172.21.15.19:0/1586428434 shutdown_connections 2024-03-28T13:20:49.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.118+0000 7f85afe00640 1 --2- 172.21.15.19:0/1586428434 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b00cc0b0 0x7f85b006dfb0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.118+0000 7f85afe00640 1 -- 172.21.15.19:0/1586428434 >> 172.21.15.19:0/1586428434 conn(0x7f85b00ccad0 msgr2=0x7f85b00cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:49.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.118+0000 7f85afe00640 1 -- 172.21.15.19:0/1586428434 shutdown_connections 2024-03-28T13:20:49.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.118+0000 7f85afe00640 1 -- 172.21.15.19:0/1586428434 wait complete. 2024-03-28T13:20:49.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.119+0000 7f85afe00640 1 Processor -- start 2024-03-28T13:20:49.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.119+0000 7f85afe00640 1 -- start start 2024-03-28T13:20:49.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.120+0000 7f85afe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b00cc0b0 0x7f85b01554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:49.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.120+0000 7f85afe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b00d04d0 0x7f85b015dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.120+0000 7f85afe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85b015e0f0 0x7f85b00adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.120+0000 7f85afe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85b0160980 con 0x7f85b00cc0b0 2024-03-28T13:20:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.120+0000 7f85afe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85b01606a0 con 0x7f85b00d04d0 2024-03-28T13:20:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.120+0000 7f85afe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f85b00cc490 con 0x7f85b015e0f0 2024-03-28T13:20:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.121+0000 7f85aea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b00cc0b0 0x7f85b01554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.121+0000 7f85aea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b00cc0b0 0x7f85b01554e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57984/0 (socket says 172.21.15.19:57984) 2024-03-28T13:20:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.121+0000 7f85aea00640 1 -- 172.21.15.19:0/978992245 learned_addr learned my addr 172.21.15.19:0/978992245 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.121+0000 7f85a7e00640 1 --2- 172.21.15.19:0/978992245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b00d04d0 0x7f85b015dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.121+0000 7f85aea00640 1 -- 172.21.15.19:0/978992245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85b015e0f0 msgr2=0x7f85b00adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.121+0000 7f85aea00640 1 --2- 172.21.15.19:0/978992245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85b015e0f0 0x7f85b00adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.121+0000 7f85aea00640 1 -- 172.21.15.19:0/978992245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b00d04d0 msgr2=0x7f85b015dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.121+0000 7f85aea00640 1 --2- 172.21.15.19:0/978992245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b00d04d0 0x7f85b015dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.121+0000 7f85aea00640 1 -- 172.21.15.19:0/978992245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b0155a20 con 0x7f85b00cc0b0 2024-03-28T13:20:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.121+0000 7f85a7e00640 1 --2- 172.21.15.19:0/978992245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b00d04d0 0x7f85b015dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:49.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.122+0000 7f85aea00640 1 --2- 172.21.15.19:0/978992245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b00cc0b0 0x7f85b01554e0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f85a0000c00 tx=0x7f85a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:49.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.122+0000 7f85a6a00640 1 -- 172.21.15.19:0/978992245 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0032840 con 0x7f85b00cc0b0 2024-03-28T13:20:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.122+0000 7f85a6a00640 1 -- 172.21.15.19:0/978992245 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a00329e0 con 0x7f85b00cc0b0 2024-03-28T13:20:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.122+0000 7f85a6a00640 1 -- 172.21.15.19:0/978992245 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0032dd0 con 0x7f85b00cc0b0 2024-03-28T13:20:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.122+0000 7f85afe00640 1 -- 172.21.15.19:0/978992245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b00ae9d0 con 0x7f85b00cc0b0 2024-03-28T13:20:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.122+0000 7f85afe00640 1 -- 172.21.15.19:0/978992245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b00aef90 con 0x7f85b00cc0b0 2024-03-28T13:20:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.123+0000 7f85afe00640 1 -- 172.21.15.19:0/978992245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b0075940 con 0x7f85b00cc0b0 2024-03-28T13:20:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.124+0000 7f85a6a00640 1 -- 172.21.15.19:0/978992245 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f85a004e400 con 0x7f85b00cc0b0 2024-03-28T13:20:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.125+0000 7f85a6a00640 1 --2- 172.21.15.19:0/978992245 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f858c06c580 0x7f858c06ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.125+0000 7f85a6a00640 1 -- 172.21.15.19:0/978992245 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 4857+0+0 (secure 0 0 0) 0x7f85a002f070 con 0x7f85b00cc0b0 2024-03-28T13:20:49.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.126+0000 7f85a7e00640 1 --2- 172.21.15.19:0/978992245 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f858c06c580 0x7f858c06ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:49.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.127+0000 7f85a7e00640 1 --2- 172.21.15.19:0/978992245 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f858c06c580 0x7f858c06ea40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f85b01f76a0 tx=0x7f8598008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:49.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.129+0000 7f85a6a00640 1 -- 172.21.15.19:0/978992245 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f85a0040070 con 0x7f85b00cc0b0 2024-03-28T13:20:49.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.333+0000 7f85afe00640 1 -- 172.21.15.19:0/978992245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85b006f2a0 con 0x7f85b00cc0b0 2024-03-28T13:20:49.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.335+0000 7f85a6a00640 1 -- 172.21.15.19:0/978992245 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+12901 (secure 0 0 0) 0x7f85a007bc20 con 0x7f85b00cc0b0 2024-03-28T13:20:49.335 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:49.335 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":115,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:20:48.386319+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:20:49.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.337+0000 7f85afe00640 1 -- 172.21.15.19:0/978992245 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f858c06c580 msgr2=0x7f858c06ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.337+0000 7f85afe00640 1 --2- 172.21.15.19:0/978992245 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f858c06c580 0x7f858c06ea40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f85b01f76a0 tx=0x7f8598008040 comp rx=0 tx=0).stop 2024-03-28T13:20:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.338+0000 7f85afe00640 1 -- 172.21.15.19:0/978992245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b00cc0b0 msgr2=0x7f85b01554e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.338+0000 7f85afe00640 1 --2- 172.21.15.19:0/978992245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b00cc0b0 0x7f85b01554e0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f85a0000c00 tx=0x7f85a00046c0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.338+0000 7f85afe00640 1 -- 172.21.15.19:0/978992245 shutdown_connections 2024-03-28T13:20:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.338+0000 7f85afe00640 1 --2- 172.21.15.19:0/978992245 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f858c06c580 0x7f858c06ea40 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.338+0000 7f85afe00640 1 --2- 172.21.15.19:0/978992245 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85b015e0f0 0x7f85b00adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.338+0000 7f85afe00640 1 --2- 172.21.15.19:0/978992245 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85b00d04d0 0x7f85b015dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.338+0000 7f85afe00640 1 --2- 172.21.15.19:0/978992245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b00cc0b0 0x7f85b01554e0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.338+0000 7f85afe00640 1 -- 172.21.15.19:0/978992245 >> 172.21.15.19:0/978992245 conn(0x7f85b00ccad0 msgr2=0x7f85b013e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.338+0000 7f85afe00640 1 -- 172.21.15.19:0/978992245 shutdown_connections 2024-03-28T13:20:49.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.338+0000 7f85afe00640 1 -- 172.21.15.19:0/978992245 wait complete. 2024-03-28T13:20:49.358 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:20:49.358 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:20:49.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.574+0000 7f8faaa00640 1 Processor -- start 2024-03-28T13:20:49.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.574+0000 7f8faaa00640 1 -- start start 2024-03-28T13:20:49.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.576+0000 7f8faaa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8fa40c7810 0x7f8fa40b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:49.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.576+0000 7f8faaa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa40d18b0 0x7f8fa406ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:49.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.576+0000 7f8faaa00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fa40d2030 con 0x7f8fa4162a30 2024-03-28T13:20:49.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.576+0000 7f8faaa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8fa4166670 con 0x7f8fa40c7810 2024-03-28T13:20:49.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.576+0000 7f8faaa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8fa4166950 con 0x7f8fa40d18b0 2024-03-28T13:20:49.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.576+0000 7f8f9be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8fa40c7810 0x7f8fa40b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:49.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.576+0000 7f8fa3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f8fa4162a30 0x7f8fa40c7100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41242/0 (socket says 172.21.15.19:41242) 2024-03-28T13:20:49.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.576+0000 7f8fa3e00640 1 -- 172.21.15.19:0/3597980219 learned_addr learned my addr 172.21.15.19:0/3597980219 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:49.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.577+0000 7f8fa8a00640 1 --2- 172.21.15.19:0/3597980219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa40d18b0 0x7f8fa406ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:49.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.577+0000 7f8f9be00640 1 -- 172.21.15.19:0/3597980219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa40d18b0 msgr2=0x7f8fa406ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:49.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.577+0000 7f8f9be00640 1 --2- 172.21.15.19:0/3597980219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa40d18b0 0x7f8fa406ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.577+0000 7f8f9be00640 1 -- 172.21.15.19:0/3597980219 >> v1:172.21.15.19:6789/0 conn(0x7f8fa4162a30 legacy=0x7f8fa40c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:49.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.577+0000 7f8f9be00640 1 -- 172.21.15.19:0/3597980219 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa4166c30 con 0x7f8fa40c7810 2024-03-28T13:20:49.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.577+0000 7f8fa8a00640 1 --2- 172.21.15.19:0/3597980219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa40d18b0 0x7f8fa406ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.578+0000 7f8f9be00640 1 --2- 172.21.15.19:0/3597980219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa40c7810 0x7f8fa40b4980 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0097a0 tx=0x7f8f8c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f761bd3087fa87 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.578+0000 7f8fa3400640 1 -- 172.21.15.19:0/3597980219 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c031780 con 0x7f8fa40c7810 2024-03-28T13:20:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.579+0000 7f8fa3400640 1 -- 172.21.15.19:0/3597980219 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c037070 con 0x7f8fa40c7810 2024-03-28T13:20:49.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.579+0000 7f8fa3400640 1 -- 172.21.15.19:0/3597980219 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c031b40 con 0x7f8fa40c7810 2024-03-28T13:20:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.579+0000 7f8faaa00640 1 -- 172.21.15.19:0/3597980219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa40c7810 msgr2=0x7f8fa40b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.579+0000 7f8faaa00640 1 --2- 172.21.15.19:0/3597980219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa40c7810 0x7f8fa40b4980 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0097a0 tx=0x7f8f8c02b820 comp rx=0 tx=0).stop 2024-03-28T13:20:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.579+0000 7f8faaa00640 1 -- 172.21.15.19:0/3597980219 shutdown_connections 2024-03-28T13:20:49.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.579+0000 7f8faaa00640 1 --2- 172.21.15.19:0/3597980219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa40d18b0 0x7f8fa406ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.579+0000 7f8faaa00640 1 --2- 172.21.15.19:0/3597980219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa40c7810 0x7f8fa40b4980 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.579+0000 7f8faaa00640 1 -- 172.21.15.19:0/3597980219 >> 172.21.15.19:0/3597980219 conn(0x7f8fa40b3e90 msgr2=0x7f8fa40b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:49.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.580+0000 7f8faaa00640 1 -- 172.21.15.19:0/3597980219 shutdown_connections 2024-03-28T13:20:49.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.580+0000 7f8faaa00640 1 -- 172.21.15.19:0/3597980219 wait complete. 2024-03-28T13:20:49.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.582+0000 7f8faaa00640 1 Processor -- start 2024-03-28T13:20:49.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.582+0000 7f8faaa00640 1 -- start start 2024-03-28T13:20:49.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.583+0000 7f8faaa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa40c7810 0x7f8fa40b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:49.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.583+0000 7f8faaa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa40d18b0 0x7f8fa414ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:49.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.583+0000 7f8faaa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa4162a30 0x7f8fa41498a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:49.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.583+0000 7f8faaa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa406cf70 con 0x7f8fa40c7810 2024-03-28T13:20:49.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.583+0000 7f8faaa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa4162e10 con 0x7f8fa4162a30 2024-03-28T13:20:49.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.583+0000 7f8faaa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8fa406d0f0 con 0x7f8fa40d18b0 2024-03-28T13:20:49.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.584+0000 7f8fa3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa40c7810 0x7f8fa40b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:49.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.584+0000 7f8fa3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa40c7810 0x7f8fa40b4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57990/0 (socket says 172.21.15.19:57990) 2024-03-28T13:20:49.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.584+0000 7f8fa3e00640 1 -- 172.21.15.19:0/1254149004 learned_addr learned my addr 172.21.15.19:0/1254149004 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:49.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.584+0000 7f8fa3e00640 1 -- 172.21.15.19:0/1254149004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa40d18b0 msgr2=0x7f8fa414ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:49.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.584+0000 7f8f9be00640 1 --2- 172.21.15.19:0/1254149004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa40d18b0 0x7f8fa414ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:49.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.584+0000 7f8fa8a00640 1 --2- 172.21.15.19:0/1254149004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa4162a30 0x7f8fa41498a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:49.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.584+0000 7f8fa3e00640 1 --2- 172.21.15.19:0/1254149004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa40d18b0 0x7f8fa414ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.584+0000 7f8fa3e00640 1 -- 172.21.15.19:0/1254149004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa4162a30 msgr2=0x7f8fa41498a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:49.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.584+0000 7f8fa3e00640 1 --2- 172.21.15.19:0/1254149004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa4162a30 0x7f8fa41498a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.584+0000 7f8fa3e00640 1 -- 172.21.15.19:0/1254149004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa414d200 con 0x7f8fa40c7810 2024-03-28T13:20:49.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.584+0000 7f8fa8a00640 1 --2- 172.21.15.19:0/1254149004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa4162a30 0x7f8fa41498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:49.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.585+0000 7f8fa3e00640 1 --2- 172.21.15.19:0/1254149004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa40c7810 0x7f8fa40b4f50 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400c3c0 tx=0x7f8f9400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:49.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.585+0000 7f8fa0c00640 1 -- 172.21.15.19:0/1254149004 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94017070 con 0x7f8fa40c7810 2024-03-28T13:20:49.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.585+0000 7f8fa0c00640 1 -- 172.21.15.19:0/1254149004 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f94005ce0 con 0x7f8fa40c7810 2024-03-28T13:20:49.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.585+0000 7f8faaa00640 1 -- 172.21.15.19:0/1254149004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa40b55f0 con 0x7f8fa40c7810 2024-03-28T13:20:49.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.585+0000 7f8fa0c00640 1 -- 172.21.15.19:0/1254149004 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f940072f0 con 0x7f8fa40c7810 2024-03-28T13:20:49.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.585+0000 7f8faaa00640 1 -- 172.21.15.19:0/1254149004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa40b7000 con 0x7f8fa40c7810 2024-03-28T13:20:49.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.586+0000 7f8faaa00640 1 -- 172.21.15.19:0/1254149004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fa4068cd0 con 0x7f8fa40c7810 2024-03-28T13:20:49.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.587+0000 7f8fa0c00640 1 -- 172.21.15.19:0/1254149004 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8f9401a070 con 0x7f8fa40c7810 2024-03-28T13:20:49.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.588+0000 7f8fa0c00640 1 --2- 172.21.15.19:0/1254149004 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8006c530 0x7f8f8006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:49.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.588+0000 7f8fa0c00640 1 -- 172.21.15.19:0/1254149004 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 4857+0+0 (secure 0 0 0) 0x7f8f9408a830 con 0x7f8fa40c7810 2024-03-28T13:20:49.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.589+0000 7f8f9be00640 1 --2- 172.21.15.19:0/1254149004 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8006c530 0x7f8f8006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:49.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.590+0000 7f8f9be00640 1 --2- 172.21.15.19:0/1254149004 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8006c530 0x7f8f8006e9f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c036000 tx=0x7f8f8c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:49.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.594+0000 7f8fa0c00640 1 -- 172.21.15.19:0/1254149004 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8f94059080 con 0x7f8fa40c7810 2024-03-28T13:20:49.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.796+0000 7f8faaa00640 1 -- 172.21.15.19:0/1254149004 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8fa40c7100 con 0x7f8fa40c7810 2024-03-28T13:20:49.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.797+0000 7f8fa0c00640 1 -- 172.21.15.19:0/1254149004 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) ==== 93+0+10 (secure 0 0 0) 0x7f8f94058980 con 0x7f8fa40c7810 2024-03-28T13:20:49.797 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:20:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.801+0000 7f8faaa00640 1 -- 172.21.15.19:0/1254149004 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8006c530 msgr2=0x7f8f8006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.801+0000 7f8faaa00640 1 --2- 172.21.15.19:0/1254149004 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8006c530 0x7f8f8006e9f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c036000 tx=0x7f8f8c008030 comp rx=0 tx=0).stop 2024-03-28T13:20:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.801+0000 7f8faaa00640 1 -- 172.21.15.19:0/1254149004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa40c7810 msgr2=0x7f8fa40b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.801+0000 7f8faaa00640 1 --2- 172.21.15.19:0/1254149004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa40c7810 0x7f8fa40b4f50 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400c3c0 tx=0x7f8f9400c880 comp rx=0 tx=0).stop 2024-03-28T13:20:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.801+0000 7f8faaa00640 1 -- 172.21.15.19:0/1254149004 shutdown_connections 2024-03-28T13:20:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.801+0000 7f8faaa00640 1 --2- 172.21.15.19:0/1254149004 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8006c530 0x7f8f8006e9f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.801+0000 7f8faaa00640 1 --2- 172.21.15.19:0/1254149004 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa4162a30 0x7f8fa41498a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.801+0000 7f8faaa00640 1 --2- 172.21.15.19:0/1254149004 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa40d18b0 0x7f8fa414ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.801+0000 7f8faaa00640 1 --2- 172.21.15.19:0/1254149004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa40c7810 0x7f8fa40b4f50 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.801+0000 7f8faaa00640 1 -- 172.21.15.19:0/1254149004 >> 172.21.15.19:0/1254149004 conn(0x7f8fa40b3e90 msgr2=0x7f8fa4071000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.801+0000 7f8faaa00640 1 -- 172.21.15.19:0/1254149004 shutdown_connections 2024-03-28T13:20:49.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:49.801+0000 7f8faaa00640 1 -- 172.21.15.19:0/1254149004 wait complete. 2024-03-28T13:20:49.820 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-28T13:20:50.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.043+0000 7fbe08a00640 1 Processor -- start 2024-03-28T13:20:50.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.043+0000 7fbe08a00640 1 -- start start 2024-03-28T13:20:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.045+0000 7fbe08a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe0413f300 0x7fbe0406e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.045+0000 7fbe08a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbe041419e0 0x7fbe0415fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.045+0000 7fbe08a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbe04141e40 0x7fbe041602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.045+0000 7fbe08a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbe040cee90 con 0x7fbe04141e40 2024-03-28T13:20:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.045+0000 7fbe08a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fbe04142880 con 0x7fbe041419e0 2024-03-28T13:20:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.045+0000 7fbe08a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbe041422c0 con 0x7fbe0413f300 2024-03-28T13:20:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.045+0000 7fbe03e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbe04141e40 0x7fbe041602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.045+0000 7fbe03e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbe04141e40 0x7fbe041602d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58006/0 (socket says 172.21.15.19:58006) 2024-03-28T13:20:50.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.045+0000 7fbe03e00640 1 -- 172.21.15.19:0/1149963248 learned_addr learned my addr 172.21.15.19:0/1149963248 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:50.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.046+0000 7fbe03400640 1 --2- 172.21.15.19:0/1149963248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe0413f300 0x7fbe0406e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:50.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.046+0000 7fbe02a00640 1 --2- 172.21.15.19:0/1149963248 >> v2:172.21.15.188:3300/0 conn(0x7fbe041419e0 0x7fbe0415fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.046+0000 7fbe03e00640 1 -- 172.21.15.19:0/1149963248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe0413f300 msgr2=0x7fbe0406e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.046+0000 7fbe03e00640 1 --2- 172.21.15.19:0/1149963248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe0413f300 0x7fbe0406e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.046+0000 7fbe03e00640 1 -- 172.21.15.19:0/1149963248 >> v2:172.21.15.188:3300/0 conn(0x7fbe041419e0 msgr2=0x7fbe0415fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.046+0000 7fbe03e00640 1 --2- 172.21.15.19:0/1149963248 >> v2:172.21.15.188:3300/0 conn(0x7fbe041419e0 0x7fbe0415fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.046+0000 7fbe03e00640 1 -- 172.21.15.19:0/1149963248 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe041425a0 con 0x7fbe04141e40 2024-03-28T13:20:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.046+0000 7fbe03400640 1 --2- 172.21.15.19:0/1149963248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe0413f300 0x7fbe0406e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.046+0000 7fbe02a00640 1 --2- 172.21.15.19:0/1149963248 >> v2:172.21.15.188:3300/0 conn(0x7fbe041419e0 0x7fbe0415fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:50.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.047+0000 7fbe03e00640 1 --2- 172.21.15.19:0/1149963248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe04141e40 0x7fbe041602d0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fbe04068260 tx=0x7fbdfc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8f378a94d2f1a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.047+0000 7fbe02000640 1 -- 172.21.15.19:0/1149963248 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc02ca80 con 0x7fbe04141e40 2024-03-28T13:20:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.047+0000 7fbe02000640 1 -- 172.21.15.19:0/1149963248 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdfc02cc20 con 0x7fbe04141e40 2024-03-28T13:20:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.047+0000 7fbe02000640 1 -- 172.21.15.19:0/1149963248 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc0343f0 con 0x7fbe04141e40 2024-03-28T13:20:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.048+0000 7fbe08a00640 1 -- 172.21.15.19:0/1149963248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe04141e40 msgr2=0x7fbe041602d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:50.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.048+0000 7fbe08a00640 1 --2- 172.21.15.19:0/1149963248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe04141e40 0x7fbe041602d0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fbe04068260 tx=0x7fbdfc02bee0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.048+0000 7fbe08a00640 1 -- 172.21.15.19:0/1149963248 shutdown_connections 2024-03-28T13:20:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.048+0000 7fbe08a00640 1 --2- 172.21.15.19:0/1149963248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe04141e40 0x7fbe041602d0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.048+0000 7fbe08a00640 1 --2- 172.21.15.19:0/1149963248 >> v2:172.21.15.188:3300/0 conn(0x7fbe041419e0 0x7fbe0415fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.048+0000 7fbe08a00640 1 --2- 172.21.15.19:0/1149963248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe0413f300 0x7fbe0406e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.048+0000 7fbe08a00640 1 -- 172.21.15.19:0/1149963248 >> 172.21.15.19:0/1149963248 conn(0x7fbe0406b570 msgr2=0x7fbe0406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.048+0000 7fbe08a00640 1 -- 172.21.15.19:0/1149963248 shutdown_connections 2024-03-28T13:20:50.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.048+0000 7fbe08a00640 1 -- 172.21.15.19:0/1149963248 wait complete. 2024-03-28T13:20:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.050+0000 7fbe08a00640 1 Processor -- start 2024-03-28T13:20:50.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.051+0000 7fbe08a00640 1 -- start start 2024-03-28T13:20:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.052+0000 7fbe08a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe0413f300 0x7fbe041f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.052+0000 7fbe08a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe041419e0 0x7fbe041361b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.052+0000 7fbe08a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe04141e40 0x7fbe041366f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.052+0000 7fbe08a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0415c8f0 con 0x7fbe041419e0 2024-03-28T13:20:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.052+0000 7fbe08a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbe0415c770 con 0x7fbe04141e40 2024-03-28T13:20:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.052+0000 7fbe08a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbe0415ca70 con 0x7fbe0413f300 2024-03-28T13:20:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.052+0000 7fbe02a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe041419e0 0x7fbe041361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.052+0000 7fbe02a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe041419e0 0x7fbe041361b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58022/0 (socket says 172.21.15.19:58022) 2024-03-28T13:20:50.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.052+0000 7fbe02a00640 1 -- 172.21.15.19:0/1539829532 learned_addr learned my addr 172.21.15.19:0/1539829532 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.052+0000 7fbe02a00640 1 -- 172.21.15.19:0/1539829532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe0413f300 msgr2=0x7fbe041f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.052+0000 7fbe02a00640 1 --2- 172.21.15.19:0/1539829532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe0413f300 0x7fbe041f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.052+0000 7fbe02a00640 1 -- 172.21.15.19:0/1539829532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe04141e40 msgr2=0x7fbe041366f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.052+0000 7fbe03e00640 1 --2- 172.21.15.19:0/1539829532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe04141e40 0x7fbe041366f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.052+0000 7fbe02a00640 1 --2- 172.21.15.19:0/1539829532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe04141e40 0x7fbe041366f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.052+0000 7fbe02a00640 1 -- 172.21.15.19:0/1539829532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe04140f30 con 0x7fbe041419e0 2024-03-28T13:20:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.053+0000 7fbe02a00640 1 --2- 172.21.15.19:0/1539829532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe041419e0 0x7fbe041361b0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000c7f0 tx=0x7fbdf000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.053+0000 7fbdfb400640 1 -- 172.21.15.19:0/1539829532 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0015070 con 0x7fbe041419e0 2024-03-28T13:20:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.053+0000 7fbdfb400640 1 -- 172.21.15.19:0/1539829532 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf0007a30 con 0x7fbe041419e0 2024-03-28T13:20:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.053+0000 7fbdfb400640 1 -- 172.21.15.19:0/1539829532 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0007e00 con 0x7fbe041419e0 2024-03-28T13:20:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.053+0000 7fbe08a00640 1 -- 172.21.15.19:0/1539829532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe04141210 con 0x7fbe041419e0 2024-03-28T13:20:50.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.053+0000 7fbe08a00640 1 -- 172.21.15.19:0/1539829532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe040b0d50 con 0x7fbe041419e0 2024-03-28T13:20:50.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.054+0000 7fbe08a00640 1 -- 172.21.15.19:0/1539829532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe040b0860 con 0x7fbe041419e0 2024-03-28T13:20:50.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.055+0000 7fbdfb400640 1 -- 172.21.15.19:0/1539829532 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbdf0018070 con 0x7fbe041419e0 2024-03-28T13:20:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.057+0000 7fbdfb400640 1 --2- 172.21.15.19:0/1539829532 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbde406c530 0x7fbde406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:50.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.057+0000 7fbdfb400640 1 -- 172.21.15.19:0/1539829532 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 4857+0+0 (secure 0 0 0) 0x7fbdf008b3d0 con 0x7fbe041419e0 2024-03-28T13:20:50.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.057+0000 7fbe03400640 1 --2- 172.21.15.19:0/1539829532 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbde406c530 0x7fbde406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:50.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.058+0000 7fbe03400640 1 --2- 172.21.15.19:0/1539829532 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbde406c530 0x7fbde406e9f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4006fd0 tx=0x7fbdf4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:50.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.061+0000 7fbdfb400640 1 -- 172.21.15.19:0/1539829532 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbdf0059ca0 con 0x7fbe041419e0 2024-03-28T13:20:50.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.267+0000 7fbe08a00640 1 -- 172.21.15.19:0/1539829532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbe04075940 con 0x7fbe041419e0 2024-03-28T13:20:50.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.268+0000 7fbdfb400640 1 -- 172.21.15.19:0/1539829532 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v115) ==== 84+0+65 (secure 0 0 0) 0x7fbdf00595a0 con 0x7fbe041419e0 2024-03-28T13:20:50.268 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:50.268 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-03-28T13:20:50.268 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-03-28T13:20:50.268 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-03-28T13:20:50.268 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-03-28T13:20:50.268 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-03-28T13:20:50.268 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-03-28T13:20:50.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.271+0000 7fbe08a00640 1 -- 172.21.15.19:0/1539829532 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbde406c530 msgr2=0x7fbde406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:50.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.271+0000 7fbe08a00640 1 --2- 172.21.15.19:0/1539829532 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbde406c530 0x7fbde406e9f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4006fd0 tx=0x7fbdf4008040 comp rx=0 tx=0).stop 2024-03-28T13:20:50.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.271+0000 7fbe08a00640 1 -- 172.21.15.19:0/1539829532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe041419e0 msgr2=0x7fbe041361b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:50.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.271+0000 7fbe08a00640 1 --2- 172.21.15.19:0/1539829532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe041419e0 0x7fbe041361b0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000c7f0 tx=0x7fbdf000cbf0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.271+0000 7fbe08a00640 1 -- 172.21.15.19:0/1539829532 shutdown_connections 2024-03-28T13:20:50.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.271+0000 7fbe08a00640 1 --2- 172.21.15.19:0/1539829532 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbde406c530 0x7fbde406e9f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.271+0000 7fbe08a00640 1 --2- 172.21.15.19:0/1539829532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe04141e40 0x7fbe041366f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.271+0000 7fbe08a00640 1 --2- 172.21.15.19:0/1539829532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe041419e0 0x7fbe041361b0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.271+0000 7fbe08a00640 1 --2- 172.21.15.19:0/1539829532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe0413f300 0x7fbe041f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.271+0000 7fbe08a00640 1 -- 172.21.15.19:0/1539829532 >> 172.21.15.19:0/1539829532 conn(0x7fbe0406b570 msgr2=0x7fbe0415f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:50.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.271+0000 7fbe08a00640 1 -- 172.21.15.19:0/1539829532 shutdown_connections 2024-03-28T13:20:50.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.271+0000 7fbe08a00640 1 -- 172.21.15.19:0/1539829532 wait complete. 2024-03-28T13:20:50.291 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-28T13:20:50.291 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-28T13:20:50.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.509+0000 7f421c400640 1 Processor -- start 2024-03-28T13:20:50.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.509+0000 7f421c400640 1 -- start start 2024-03-28T13:20:50.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.510+0000 7f421c400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f421409dcf0 0x7f42141453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:50.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.510+0000 7f421c400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f421415cb90 con 0x7f42140acf10 2024-03-28T13:20:50.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.511+0000 7f421c400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f421415ce70 con 0x7f4214071710 2024-03-28T13:20:50.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.511+0000 7f421c400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f4214145f60 con 0x7f421409dcf0 2024-03-28T13:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.511+0000 7f4219a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f42140acf10 0x7f42140ad2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41246/0 (socket says 172.21.15.19:41246) 2024-03-28T13:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.511+0000 7f4219a00640 1 -- 172.21.15.19:0/637998800 learned_addr learned my addr 172.21.15.19:0/637998800 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.511+0000 7f4219000640 1 --2- 172.21.15.19:0/637998800 >> v2:172.21.15.188:3300/0 conn(0x7f421409dcf0 0x7f42141453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.511+0000 7f4213e00640 1 -- 172.21.15.19:0/637998800 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4250858908 0 0) 0x7f421415cb90 con 0x7f42140acf10 2024-03-28T13:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.511+0000 7f4213e00640 1 -- 172.21.15.19:0/637998800 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41fc003520 con 0x7f42140acf10 2024-03-28T13:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.512+0000 7f4213e00640 1 -- 172.21.15.19:0/637998800 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2009954442 0 0) 0x7f41fc003520 con 0x7f42140acf10 2024-03-28T13:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.512+0000 7f4213e00640 1 -- 172.21.15.19:0/637998800 >> v2:172.21.15.188:3300/0 conn(0x7f421409dcf0 msgr2=0x7f42141453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.512+0000 7f4213e00640 1 --2- 172.21.15.19:0/637998800 >> v2:172.21.15.188:3300/0 conn(0x7f421409dcf0 0x7f42141453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.512+0000 7f4213e00640 1 -- 172.21.15.19:0/637998800 >> v1:172.21.15.188:6789/0 conn(0x7f4214071710 legacy=0x7f4214071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.512+0000 7f4213e00640 1 -- 172.21.15.19:0/637998800 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421415d150 con 0x7f42140acf10 2024-03-28T13:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.512+0000 7f4213e00640 1 -- 172.21.15.19:0/637998800 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f4208001670 con 0x7f42140acf10 2024-03-28T13:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.512+0000 7f4213e00640 1 -- 172.21.15.19:0/637998800 >> v1:172.21.15.19:6789/0 conn(0x7f42140acf10 legacy=0x7f42140ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.512+0000 7f4213e00640 1 --2- 172.21.15.19:0/637998800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41fc003a00 0x7f41fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.512+0000 7f4213e00640 1 --2- 172.21.15.19:0/637998800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41fc004800 0x7f41fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.512+0000 7f4213e00640 1 --2- 172.21.15.19:0/637998800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc005210 0x7f41fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.512+0000 7f4213e00640 1 -- 172.21.15.19:0/637998800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4214145f60 con 0x7f41fc004800 2024-03-28T13:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.512+0000 7f4213e00640 1 -- 172.21.15.19:0/637998800 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc00c160 con 0x7f41fc005210 2024-03-28T13:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.512+0000 7f4213e00640 1 -- 172.21.15.19:0/637998800 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f41fc00c360 con 0x7f41fc003a00 2024-03-28T13:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.512+0000 7f4219000640 1 --2- 172.21.15.19:0/637998800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41fc004800 0x7f41fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.513+0000 7f421a400640 1 --2- 172.21.15.19:0/637998800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc005210 0x7f41fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.513+0000 7f4219000640 1 -- 172.21.15.19:0/637998800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41fc003a00 msgr2=0x7f41fc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.513+0000 7f4219a00640 1 --2- 172.21.15.19:0/637998800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41fc003a00 0x7f41fc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.513+0000 7f4219000640 1 --2- 172.21.15.19:0/637998800 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41fc003a00 0x7f41fc003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.513+0000 7f4219000640 1 -- 172.21.15.19:0/637998800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc005210 msgr2=0x7f41fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.513+0000 7f4219000640 1 --2- 172.21.15.19:0/637998800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc005210 0x7f41fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.513+0000 7f4219000640 1 -- 172.21.15.19:0/637998800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc003520 con 0x7f41fc004800 2024-03-28T13:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.513+0000 7f421a400640 1 --2- 172.21.15.19:0/637998800 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc005210 0x7f41fc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.513+0000 7f421a400640 1 -- 172.21.15.19:0/637998800 reap_dead start 2024-03-28T13:20:50.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.514+0000 7f4219000640 1 --2- 172.21.15.19:0/637998800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41fc004800 0x7f41fc004c80 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f4200009a90 tx=0x7f41fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8354797930965a3b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.514+0000 7f4213e00640 1 -- 172.21.15.19:0/637998800 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200039070 con 0x7f41fc004800 2024-03-28T13:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.514+0000 7f4213e00640 1 -- 172.21.15.19:0/637998800 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f420002c680 con 0x7f41fc004800 2024-03-28T13:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.514+0000 7f4213e00640 1 -- 172.21.15.19:0/637998800 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200009690 con 0x7f41fc004800 2024-03-28T13:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.515+0000 7f421c400640 1 -- 172.21.15.19:0/637998800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41fc004800 msgr2=0x7f41fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.515+0000 7f421c400640 1 --2- 172.21.15.19:0/637998800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41fc004800 0x7f41fc004c80 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f4200009a90 tx=0x7f41fc005830 comp rx=0 tx=0).stop 2024-03-28T13:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.515+0000 7f421c400640 1 -- 172.21.15.19:0/637998800 shutdown_connections 2024-03-28T13:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.515+0000 7f421c400640 1 --2- 172.21.15.19:0/637998800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41fc004800 0x7f41fc004c80 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.515+0000 7f421c400640 1 -- 172.21.15.19:0/637998800 >> 172.21.15.19:0/637998800 conn(0x7f421406bba0 msgr2=0x7f42140b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.515+0000 7f421c400640 1 -- 172.21.15.19:0/637998800 shutdown_connections 2024-03-28T13:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.515+0000 7f421c400640 1 -- 172.21.15.19:0/637998800 wait complete. 2024-03-28T13:20:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.517+0000 7f421c400640 1 Processor -- start 2024-03-28T13:20:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.517+0000 7f421c400640 1 -- start start 2024-03-28T13:20:50.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.518+0000 7f421c400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc004800 0x7f421413e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:50.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.518+0000 7f421c400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421406f030 0x7f42141f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:50.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.518+0000 7f421c400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42141f1e50 0x7f42141f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:50.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.518+0000 7f421c400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f421415d7a0 con 0x7f421406f030 2024-03-28T13:20:50.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.518+0000 7f421c400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f421415c950 con 0x7f41fc004800 2024-03-28T13:20:50.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.518+0000 7f421c400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f42140ad2f0 con 0x7f42141f1e50 2024-03-28T13:20:50.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.518+0000 7f4219000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421406f030 0x7f42141f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:50.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.518+0000 7f4219000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421406f030 0x7f42141f8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58038/0 (socket says 172.21.15.19:58038) 2024-03-28T13:20:50.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.518+0000 7f4219000640 1 -- 172.21.15.19:0/758864749 learned_addr learned my addr 172.21.15.19:0/758864749 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:50.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.519+0000 7f4219a00640 1 --2- 172.21.15.19:0/758864749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc004800 0x7f421413e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:50.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.519+0000 7f4219000640 1 -- 172.21.15.19:0/758864749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42141f1e50 msgr2=0x7f42141f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:50.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.519+0000 7f4219000640 1 --2- 172.21.15.19:0/758864749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42141f1e50 0x7f42141f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.519+0000 7f4219000640 1 -- 172.21.15.19:0/758864749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc004800 msgr2=0x7f421413e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:50.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.519+0000 7f4219000640 1 --2- 172.21.15.19:0/758864749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc004800 0x7f421413e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:50.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.519+0000 7f4219000640 1 -- 172.21.15.19:0/758864749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42141f2810 con 0x7f421406f030 2024-03-28T13:20:50.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.519+0000 7f4219a00640 1 --2- 172.21.15.19:0/758864749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc004800 0x7f421413e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:50.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.519+0000 7f4219000640 1 --2- 172.21.15.19:0/758864749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421406f030 0x7f42141f8680 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f4200037040 tx=0x7f4200004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:50.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.520+0000 7f4211600640 1 -- 172.21.15.19:0/758864749 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200039050 con 0x7f421406f030 2024-03-28T13:20:50.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.520+0000 7f4211600640 1 -- 172.21.15.19:0/758864749 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4200045070 con 0x7f421406f030 2024-03-28T13:20:50.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.520+0000 7f421c400640 1 -- 172.21.15.19:0/758864749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421413ef50 con 0x7f421406f030 2024-03-28T13:20:50.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.520+0000 7f4211600640 1 -- 172.21.15.19:0/758864749 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42000044e0 con 0x7f421406f030 2024-03-28T13:20:50.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.520+0000 7f421c400640 1 -- 172.21.15.19:0/758864749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f421413f510 con 0x7f421406f030 2024-03-28T13:20:50.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.521+0000 7f421c400640 1 -- 172.21.15.19:0/758864749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42140685c0 con 0x7f421406f030 2024-03-28T13:20:50.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.522+0000 7f4211600640 1 -- 172.21.15.19:0/758864749 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4200003a50 con 0x7f421406f030 2024-03-28T13:20:50.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.523+0000 7f4211600640 1 --2- 172.21.15.19:0/758864749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41f406c560 0x7f41f406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:50.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.523+0000 7f4211600640 1 -- 172.21.15.19:0/758864749 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 4857+0+0 (secure 0 0 0) 0x7f42000ace90 con 0x7f421406f030 2024-03-28T13:20:50.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.524+0000 7f4219a00640 1 --2- 172.21.15.19:0/758864749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41f406c560 0x7f41f406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:50.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.525+0000 7f4219a00640 1 --2- 172.21.15.19:0/758864749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41f406c560 0x7f41f406ea20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4214160b40 tx=0x7f4208009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:50.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.527+0000 7f4211600640 1 -- 172.21.15.19:0/758864749 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f420004e020 con 0x7f421406f030 2024-03-28T13:20:50.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:50.730+0000 7f421c400640 1 -- 172.21.15.19:0/758864749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f42140b2900 con 0x7f421406f030 2024-03-28T13:20:51.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.396+0000 7f4211600640 1 -- 172.21.15.19:0/758864749 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v116) ==== 138+0+0 (secure 0 0 0) 0x7f420007b7f0 con 0x7f421406f030 2024-03-28T13:20:51.396 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-03-28T13:20:51.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.399+0000 7f421c400640 1 -- 172.21.15.19:0/758864749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41f406c560 msgr2=0x7f41f406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:51.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.399+0000 7f421c400640 1 --2- 172.21.15.19:0/758864749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41f406c560 0x7f41f406ea20 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4214160b40 tx=0x7f4208009040 comp rx=0 tx=0).stop 2024-03-28T13:20:51.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.399+0000 7f421c400640 1 -- 172.21.15.19:0/758864749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421406f030 msgr2=0x7f42141f8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:51.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.399+0000 7f421c400640 1 --2- 172.21.15.19:0/758864749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421406f030 0x7f42141f8680 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f4200037040 tx=0x7f4200004730 comp rx=0 tx=0).stop 2024-03-28T13:20:51.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.399+0000 7f421c400640 1 -- 172.21.15.19:0/758864749 shutdown_connections 2024-03-28T13:20:51.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.399+0000 7f421c400640 1 --2- 172.21.15.19:0/758864749 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41f406c560 0x7f41f406ea20 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:51.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.399+0000 7f421c400640 1 --2- 172.21.15.19:0/758864749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42141f1e50 0x7f42141f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:51.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.399+0000 7f421c400640 1 --2- 172.21.15.19:0/758864749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f421406f030 0x7f42141f8680 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:51.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.399+0000 7f421c400640 1 --2- 172.21.15.19:0/758864749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc004800 0x7f421413e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:51.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.399+0000 7f421c400640 1 -- 172.21.15.19:0/758864749 >> 172.21.15.19:0/758864749 conn(0x7f421406bba0 msgr2=0x7f42140745e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:51.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.400+0000 7f421c400640 1 -- 172.21.15.19:0/758864749 shutdown_connections 2024-03-28T13:20:51.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.400+0000 7f421c400640 1 -- 172.21.15.19:0/758864749 wait complete. 2024-03-28T13:20:51.429 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-28T13:20:51.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.646+0000 7f9a0fe00640 1 Processor -- start 2024-03-28T13:20:51.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.646+0000 7f9a0fe00640 1 -- start start 2024-03-28T13:20:51.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.648+0000 7f9a0fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a080d0280 0x7f9a080d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:51.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.648+0000 7f9a0fe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9a08141c10 0x7f9a08071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:51.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.648+0000 7f9a0fe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a0815ec80 con 0x7f9a08071eb0 2024-03-28T13:20:51.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.648+0000 7f9a0fe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9a080d0ba0 con 0x7f9a08141c10 2024-03-28T13:20:51.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.648+0000 7f9a0fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0815ef20 con 0x7f9a080d0280 2024-03-28T13:20:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.649+0000 7f9a0de00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f9a08071eb0 0x7f9a0806d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41250/0 (socket says 172.21.15.19:41250) 2024-03-28T13:20:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.649+0000 7f9a0de00640 1 -- 172.21.15.19:0/2289165979 learned_addr learned my addr 172.21.15.19:0/2289165979 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.649+0000 7f9a0ca00640 1 --2- 172.21.15.19:0/2289165979 >> v2:172.21.15.188:3300/0 conn(0x7f9a08141c10 0x7f9a08071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.649+0000 7f9a0d400640 1 --2- 172.21.15.19:0/2289165979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a080d0280 0x7f9a080d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.649+0000 7f9a07e00640 1 -- 172.21.15.19:0/2289165979 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 345190980 0 0) 0x7f9a0815ec80 con 0x7f9a08071eb0 2024-03-28T13:20:51.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.649+0000 7f9a07e00640 1 -- 172.21.15.19:0/2289165979 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99f0003520 con 0x7f9a08071eb0 2024-03-28T13:20:51.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.650+0000 7f9a07e00640 1 -- 172.21.15.19:0/2289165979 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2430870134 0 0) 0x7f99f0003520 con 0x7f9a08071eb0 2024-03-28T13:20:51.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.650+0000 7f9a07e00640 1 -- 172.21.15.19:0/2289165979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a080d0280 msgr2=0x7f9a080d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:51.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.650+0000 7f9a07e00640 1 --2- 172.21.15.19:0/2289165979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a080d0280 0x7f9a080d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:51.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.650+0000 7f9a07e00640 1 -- 172.21.15.19:0/2289165979 >> v2:172.21.15.188:3300/0 conn(0x7f9a08141c10 msgr2=0x7f9a08071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:51.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.650+0000 7f9a07e00640 1 --2- 172.21.15.19:0/2289165979 >> v2:172.21.15.188:3300/0 conn(0x7f9a08141c10 0x7f9a08071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:51.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.650+0000 7f9a07e00640 1 -- 172.21.15.19:0/2289165979 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0815f200 con 0x7f9a08071eb0 2024-03-28T13:20:51.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.650+0000 7f9a07e00640 1 -- 172.21.15.19:0/2289165979 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f9a000019f0 con 0x7f9a08071eb0 2024-03-28T13:20:51.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.650+0000 7f9a07e00640 1 -- 172.21.15.19:0/2289165979 >> v1:172.21.15.19:6789/0 conn(0x7f9a08071eb0 legacy=0x7f9a0806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:51.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.650+0000 7f9a07e00640 1 --2- 172.21.15.19:0/2289165979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a080d0280 0x7f99f00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:51.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.650+0000 7f9a07e00640 1 --2- 172.21.15.19:0/2289165979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99f0003f30 0x7f99f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:51.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.650+0000 7f9a07e00640 1 --2- 172.21.15.19:0/2289165979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f0004d40 0x7f99f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:51.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.650+0000 7f9a07e00640 1 -- 172.21.15.19:0/2289165979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a080d0ba0 con 0x7f99f0004d40 2024-03-28T13:20:51.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.650+0000 7f9a07e00640 1 -- 172.21.15.19:0/2289165979 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0815ef20 con 0x7f99f0003f30 2024-03-28T13:20:51.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.650+0000 7f9a07e00640 1 -- 172.21.15.19:0/2289165979 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f99f000b9c0 con 0x7f9a080d0280 2024-03-28T13:20:51.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.650+0000 7f9a0de00640 1 --2- 172.21.15.19:0/2289165979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f0004d40 0x7f99f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:51.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.651+0000 7f9a0d400640 1 --2- 172.21.15.19:0/2289165979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a080d0280 0x7f99f00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:51.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.651+0000 7f9a0ca00640 1 --2- 172.21.15.19:0/2289165979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99f0003f30 0x7f99f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:51.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.651+0000 7f9a0de00640 1 -- 172.21.15.19:0/2289165979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a080d0280 msgr2=0x7f99f00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:51.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.651+0000 7f9a0de00640 1 --2- 172.21.15.19:0/2289165979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a080d0280 0x7f99f00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:51.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.651+0000 7f9a0de00640 1 -- 172.21.15.19:0/2289165979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99f0003f30 msgr2=0x7f99f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:51.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.651+0000 7f9a0de00640 1 --2- 172.21.15.19:0/2289165979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99f0003f30 0x7f99f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:51.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.651+0000 7f9a0de00640 1 -- 172.21.15.19:0/2289165979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f0003520 con 0x7f99f0004d40 2024-03-28T13:20:51.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.652+0000 7f9a0de00640 1 --2- 172.21.15.19:0/2289165979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f0004d40 0x7f99f00051c0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000a920 tx=0x7f99f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75caadd6c3b85cd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:51.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.652+0000 7f9a07e00640 1 -- 172.21.15.19:0/2289165979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0000a4c0 con 0x7f99f0004d40 2024-03-28T13:20:51.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.652+0000 7f9a07e00640 1 -- 172.21.15.19:0/2289165979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a00034650 con 0x7f99f0004d40 2024-03-28T13:20:51.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.652+0000 7f9a07e00640 1 -- 172.21.15.19:0/2289165979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00034a80 con 0x7f99f0004d40 2024-03-28T13:20:51.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.653+0000 7f9a0fe00640 1 -- 172.21.15.19:0/2289165979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f0004d40 msgr2=0x7f99f00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:51.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.653+0000 7f9a0fe00640 1 --2- 172.21.15.19:0/2289165979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99f0004d40 0x7f99f00051c0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000a920 tx=0x7f99f0005830 comp rx=0 tx=0).stop 2024-03-28T13:20:51.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.653+0000 7f9a0de00640 1 -- 172.21.15.19:0/2289165979 reap_dead start 2024-03-28T13:20:51.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.653+0000 7f9a0fe00640 1 -- 172.21.15.19:0/2289165979 shutdown_connections 2024-03-28T13:20:51.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.653+0000 7f9a0fe00640 1 -- 172.21.15.19:0/2289165979 >> 172.21.15.19:0/2289165979 conn(0x7f9a0806b750 msgr2=0x7f9a0806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:51.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.653+0000 7f9a0fe00640 1 -- 172.21.15.19:0/2289165979 shutdown_connections 2024-03-28T13:20:51.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.653+0000 7f9a0fe00640 1 -- 172.21.15.19:0/2289165979 wait complete. 2024-03-28T13:20:51.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.655+0000 7f9a0fe00640 1 Processor -- start 2024-03-28T13:20:51.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.655+0000 7f9a0fe00640 1 -- start start 2024-03-28T13:20:51.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.656+0000 7f9a0fe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99f0004d40 0x7f9a08141150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.656+0000 7f9a0fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a0809c060 0x7f9a0809a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.656+0000 7f9a0fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a0809c4e0 0x7f9a0809a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.656+0000 7f9a0fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a08158e90 con 0x7f9a0809c060 2024-03-28T13:20:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.656+0000 7f9a0fe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0815f7c0 con 0x7f99f0004d40 2024-03-28T13:20:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.656+0000 7f9a0fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0809c9b0 con 0x7f9a0809c4e0 2024-03-28T13:20:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.656+0000 7f9a0ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a0809c060 0x7f9a0809a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.657+0000 7f9a0ca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a0809c060 0x7f9a0809a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58058/0 (socket says 172.21.15.19:58058) 2024-03-28T13:20:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.657+0000 7f9a0ca00640 1 -- 172.21.15.19:0/3455286214 learned_addr learned my addr 172.21.15.19:0/3455286214 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.657+0000 7f9a0ca00640 1 -- 172.21.15.19:0/3455286214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a0809c4e0 msgr2=0x7f9a0809a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.657+0000 7f9a0d400640 1 --2- 172.21.15.19:0/3455286214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99f0004d40 0x7f9a08141150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.657+0000 7f9a0de00640 1 --2- 172.21.15.19:0/3455286214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a0809c4e0 0x7f9a0809a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:51.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.657+0000 7f9a0ca00640 1 --2- 172.21.15.19:0/3455286214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a0809c4e0 0x7f9a0809a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:51.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.657+0000 7f9a0ca00640 1 -- 172.21.15.19:0/3455286214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99f0004d40 msgr2=0x7f9a08141150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:51.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.657+0000 7f9a0ca00640 1 --2- 172.21.15.19:0/3455286214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99f0004d40 0x7f9a08141150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:51.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.657+0000 7f9a0ca00640 1 -- 172.21.15.19:0/3455286214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0809cc90 con 0x7f9a0809c060 2024-03-28T13:20:51.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.657+0000 7f9a0ca00640 1 --2- 172.21.15.19:0/3455286214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a0809c060 0x7f9a0809a190 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f99f40115b0 tx=0x7f99f4011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:51.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.658+0000 7f9a05600640 1 -- 172.21.15.19:0/3455286214 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4015870 con 0x7f9a0809c060 2024-03-28T13:20:51.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.658+0000 7f9a05600640 1 -- 172.21.15.19:0/3455286214 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f4015a10 con 0x7f9a0809c060 2024-03-28T13:20:51.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.658+0000 7f9a05600640 1 -- 172.21.15.19:0/3455286214 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4015dc0 con 0x7f9a0809c060 2024-03-28T13:20:51.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.658+0000 7f9a0fe00640 1 -- 172.21.15.19:0/3455286214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0809cf70 con 0x7f9a0809c060 2024-03-28T13:20:51.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.658+0000 7f9a0fe00640 1 -- 172.21.15.19:0/3455286214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a0809d530 con 0x7f9a0809c060 2024-03-28T13:20:51.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.659+0000 7f9a0fe00640 1 -- 172.21.15.19:0/3455286214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a080685c0 con 0x7f9a0809c060 2024-03-28T13:20:51.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.660+0000 7f9a05600640 1 -- 172.21.15.19:0/3455286214 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f99f400f480 con 0x7f9a0809c060 2024-03-28T13:20:51.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.661+0000 7f9a05600640 1 --2- 172.21.15.19:0/3455286214 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f99e806c530 0x7f99e806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:51.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.661+0000 7f9a05600640 1 -- 172.21.15.19:0/3455286214 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5227+0+0 (secure 0 0 0) 0x7f99f408bf40 con 0x7f9a0809c060 2024-03-28T13:20:51.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.662+0000 7f9a0d400640 1 --2- 172.21.15.19:0/3455286214 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f99e806c530 0x7f99e806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:51.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.663+0000 7f9a0d400640 1 --2- 172.21.15.19:0/3455286214 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f99e806c530 0x7f99e806e9f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9a080afc50 tx=0x7f99fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:51.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.666+0000 7f9a05600640 1 -- 172.21.15.19:0/3455286214 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f99f4019030 con 0x7f9a0809c060 2024-03-28T13:20:51.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:51.872+0000 7f9a0fe00640 1 -- 172.21.15.19:0/3455286214 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9a080cfdb0 con 0x7f9a0809c060 2024-03-28T13:20:52.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.401+0000 7f9a05600640 1 -- 172.21.15.19:0/3455286214 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v117) ==== 170+0+0 (secure 0 0 0) 0x7f99f405a720 con 0x7f9a0809c060 2024-03-28T13:20:52.402 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-03-28T13:20:52.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.406+0000 7f9a0fe00640 1 -- 172.21.15.19:0/3455286214 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f99e806c530 msgr2=0x7f99e806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:52.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.406+0000 7f9a0fe00640 1 --2- 172.21.15.19:0/3455286214 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f99e806c530 0x7f99e806e9f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9a080afc50 tx=0x7f99fc008040 comp rx=0 tx=0).stop 2024-03-28T13:20:52.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.406+0000 7f9a0fe00640 1 -- 172.21.15.19:0/3455286214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a0809c060 msgr2=0x7f9a0809a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:52.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.406+0000 7f9a0fe00640 1 --2- 172.21.15.19:0/3455286214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a0809c060 0x7f9a0809a190 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f99f40115b0 tx=0x7f99f4011a70 comp rx=0 tx=0).stop 2024-03-28T13:20:52.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.407+0000 7f9a0fe00640 1 -- 172.21.15.19:0/3455286214 shutdown_connections 2024-03-28T13:20:52.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.407+0000 7f9a0fe00640 1 --2- 172.21.15.19:0/3455286214 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f99e806c530 0x7f99e806e9f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.407+0000 7f9a0fe00640 1 --2- 172.21.15.19:0/3455286214 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a0809c4e0 0x7f9a0809a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.407+0000 7f9a0fe00640 1 --2- 172.21.15.19:0/3455286214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a0809c060 0x7f9a0809a190 secure :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f99f40115b0 tx=0x7f99f4011a70 comp rx=0 tx=0).stop 2024-03-28T13:20:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.407+0000 7f9a0fe00640 1 --2- 172.21.15.19:0/3455286214 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99f0004d40 0x7f9a08141150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.407+0000 7f9a0fe00640 1 -- 172.21.15.19:0/3455286214 >> 172.21.15.19:0/3455286214 conn(0x7f9a0806b750 msgr2=0x7f9a081604d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:52.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.410+0000 7f9a0fe00640 1 -- 172.21.15.19:0/3455286214 shutdown_connections 2024-03-28T13:20:52.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.412+0000 7f9a0fe00640 1 -- 172.21.15.19:0/3455286214 wait complete. 2024-03-28T13:20:52.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-28T13:20:52.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.636+0000 7f822b200640 1 Processor -- start 2024-03-28T13:20:52.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.636+0000 7f822b200640 1 -- start start 2024-03-28T13:20:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.637+0000 7f822b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f822c1442c0 0x7f822c0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.637+0000 7f822b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f822c0cefe0 0x7f822c0cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.637+0000 7f822b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f822c071460 con 0x7f822c0d00d0 2024-03-28T13:20:52.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.637+0000 7f822b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f822c071d20 con 0x7f822c0cefe0 2024-03-28T13:20:52.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.637+0000 7f822b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f822c0cf420 con 0x7f822c1442c0 2024-03-28T13:20:52.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.638+0000 7f8221400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f822c0cefe0 0x7f822c0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:52.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.638+0000 7f822a800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f822c0d00d0 0x7f822c06e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41260/0 (socket says 172.21.15.19:41260) 2024-03-28T13:20:52.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.638+0000 7f822a800640 1 -- 172.21.15.19:0/2701493890 learned_addr learned my addr 172.21.15.19:0/2701493890 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:52.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.638+0000 7f8221400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f822c0cefe0 0x7f822c0cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58068/0 (socket says 172.21.15.19:58068) 2024-03-28T13:20:52.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.638+0000 7f8229e00640 1 --2- 172.21.15.19:0/2701493890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f822c1442c0 0x7f822c0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:52.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.638+0000 7f8221400640 1 -- 172.21.15.19:0/2701493890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f822c1442c0 msgr2=0x7f822c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:52.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.638+0000 7f8221400640 1 --2- 172.21.15.19:0/2701493890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f822c1442c0 0x7f822c0ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:52.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.638+0000 7f8221400640 1 -- 172.21.15.19:0/2701493890 >> v1:172.21.15.19:6789/0 conn(0x7f822c0d00d0 legacy=0x7f822c06e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:52.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.638+0000 7f8221400640 1 -- 172.21.15.19:0/2701493890 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822c0721b0 con 0x7f822c0cefe0 2024-03-28T13:20:52.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.638+0000 7f8229400640 1 -- 172.21.15.19:0/2701493890 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3184538234 0 0) 0x7f822c071460 con 0x7f822c0d00d0 2024-03-28T13:20:52.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.640+0000 7f8221400640 1 --2- 172.21.15.19:0/2701493890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f822c0cefe0 0x7f822c0cfb90 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f8214009d90 tx=0x7f821402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6d91850929c6b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:52.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.640+0000 7f8229400640 1 -- 172.21.15.19:0/2701493890 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214039070 con 0x7f822c0cefe0 2024-03-28T13:20:52.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.640+0000 7f8229400640 1 -- 172.21.15.19:0/2701493890 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821402cce0 con 0x7f822c0cefe0 2024-03-28T13:20:52.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.640+0000 7f8229400640 1 -- 172.21.15.19:0/2701493890 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214034660 con 0x7f822c0cefe0 2024-03-28T13:20:52.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.640+0000 7f822b200640 1 -- 172.21.15.19:0/2701493890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f822c0cefe0 msgr2=0x7f822c0cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:52.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.640+0000 7f822b200640 1 --2- 172.21.15.19:0/2701493890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f822c0cefe0 0x7f822c0cfb90 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f8214009d90 tx=0x7f821402bf30 comp rx=0 tx=0).stop 2024-03-28T13:20:52.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.641+0000 7f822b200640 1 -- 172.21.15.19:0/2701493890 shutdown_connections 2024-03-28T13:20:52.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.641+0000 7f822b200640 1 --2- 172.21.15.19:0/2701493890 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f822c0cefe0 0x7f822c0cfb90 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:52.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.641+0000 7f822b200640 1 --2- 172.21.15.19:0/2701493890 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f822c1442c0 0x7f822c0ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:52.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.641+0000 7f822b200640 1 -- 172.21.15.19:0/2701493890 >> 172.21.15.19:0/2701493890 conn(0x7f822c0ccad0 msgr2=0x7f822c0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:52.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.641+0000 7f822b200640 1 -- 172.21.15.19:0/2701493890 shutdown_connections 2024-03-28T13:20:52.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.641+0000 7f822b200640 1 -- 172.21.15.19:0/2701493890 wait complete. 2024-03-28T13:20:52.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.642+0000 7f822b200640 1 Processor -- start 2024-03-28T13:20:52.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.643+0000 7f822b200640 1 -- start start 2024-03-28T13:20:52.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.643+0000 7f822b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f822c0cefe0 0x7f822c15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:52.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.643+0000 7f822b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f822c0d00d0 0x7f822c138a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:52.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.643+0000 7f822b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822c1442c0 0x7f822c138f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:52.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.643+0000 7f822b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f822c0719b0 con 0x7f822c0cefe0 2024-03-28T13:20:52.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.643+0000 7f822b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f822c0d0520 con 0x7f822c1442c0 2024-03-28T13:20:52.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.643+0000 7f822b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f822c0d0770 con 0x7f822c0d00d0 2024-03-28T13:20:52.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.644+0000 7f8229e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f822c0cefe0 0x7f822c15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:52.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.644+0000 7f8229e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f822c0cefe0 0x7f822c15a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58070/0 (socket says 172.21.15.19:58070) 2024-03-28T13:20:52.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.644+0000 7f8229e00640 1 -- 172.21.15.19:0/3543676602 learned_addr learned my addr 172.21.15.19:0/3543676602 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:52.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.644+0000 7f8221400640 1 --2- 172.21.15.19:0/3543676602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f822c0d00d0 0x7f822c138a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:52.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.644+0000 7f822a800640 1 --2- 172.21.15.19:0/3543676602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822c1442c0 0x7f822c138f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:52.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.644+0000 7f8229e00640 1 -- 172.21.15.19:0/3543676602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f822c0d00d0 msgr2=0x7f822c138a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:52.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.644+0000 7f8229e00640 1 --2- 172.21.15.19:0/3543676602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f822c0d00d0 0x7f822c138a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:52.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.644+0000 7f8229e00640 1 -- 172.21.15.19:0/3543676602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822c1442c0 msgr2=0x7f822c138f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:52.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.644+0000 7f8229e00640 1 --2- 172.21.15.19:0/3543676602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822c1442c0 0x7f822c138f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:52.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.644+0000 7f8229e00640 1 -- 172.21.15.19:0/3543676602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f822c15ac20 con 0x7f822c0cefe0 2024-03-28T13:20:52.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.645+0000 7f8229e00640 1 --2- 172.21.15.19:0/3543676602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f822c0cefe0 0x7f822c15a6e0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f821c00d5f0 tx=0x7f821c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:52.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.645+0000 7f8222a00640 1 -- 172.21.15.19:0/3543676602 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c0125f0 con 0x7f822c0cefe0 2024-03-28T13:20:52.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.645+0000 7f8222a00640 1 -- 172.21.15.19:0/3543676602 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821c019070 con 0x7f822c0cefe0 2024-03-28T13:20:52.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.645+0000 7f8222a00640 1 -- 172.21.15.19:0/3543676602 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c0129d0 con 0x7f822c0cefe0 2024-03-28T13:20:52.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.645+0000 7f822b200640 1 -- 172.21.15.19:0/3543676602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822c137df0 con 0x7f822c0cefe0 2024-03-28T13:20:52.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.645+0000 7f822b200640 1 -- 172.21.15.19:0/3543676602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f822c0b3f10 con 0x7f822c0cefe0 2024-03-28T13:20:52.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.646+0000 7f822b200640 1 -- 172.21.15.19:0/3543676602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f822c075940 con 0x7f822c0cefe0 2024-03-28T13:20:52.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.647+0000 7f8222a00640 1 -- 172.21.15.19:0/3543676602 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f821c012b70 con 0x7f822c0cefe0 2024-03-28T13:20:52.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.649+0000 7f8222a00640 1 --2- 172.21.15.19:0/3543676602 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f820806c530 0x7f820806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:52.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.649+0000 7f8222a00640 1 -- 172.21.15.19:0/3543676602 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5609+0+0 (secure 0 0 0) 0x7f821c08bb20 con 0x7f822c0cefe0 2024-03-28T13:20:52.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.650+0000 7f8221400640 1 --2- 172.21.15.19:0/3543676602 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f820806c530 0x7f820806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:52.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.651+0000 7f8221400640 1 --2- 172.21.15.19:0/3543676602 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f820806c530 0x7f820806e9f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8214036000 tx=0x7f8214030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:52.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.655+0000 7f8222a00640 1 -- 172.21.15.19:0/3543676602 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f821c05a100 con 0x7f822c0cefe0 2024-03-28T13:20:52.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:52.898+0000 7f822b200640 1 -- 172.21.15.19:0/3543676602 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f822c143a70 con 0x7f822c0cefe0 2024-03-28T13:20:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.456+0000 7f8222a00640 1 -- 172.21.15.19:0/3543676602 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v95) ==== 174+0+0 (secure 0 0 0) 0x7f821c059a00 con 0x7f822c0cefe0 2024-03-28T13:20:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 16 and data pool 17 2024-03-28T13:20:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.465+0000 7f822b200640 1 -- 172.21.15.19:0/3543676602 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f820806c530 msgr2=0x7f820806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.465+0000 7f822b200640 1 --2- 172.21.15.19:0/3543676602 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f820806c530 0x7f820806e9f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8214036000 tx=0x7f8214030000 comp rx=0 tx=0).stop 2024-03-28T13:20:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.465+0000 7f822b200640 1 -- 172.21.15.19:0/3543676602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f822c0cefe0 msgr2=0x7f822c15a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.465+0000 7f822b200640 1 --2- 172.21.15.19:0/3543676602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f822c0cefe0 0x7f822c15a6e0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f821c00d5f0 tx=0x7f821c00dab0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.465+0000 7f822b200640 1 -- 172.21.15.19:0/3543676602 shutdown_connections 2024-03-28T13:20:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.465+0000 7f822b200640 1 --2- 172.21.15.19:0/3543676602 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f820806c530 0x7f820806e9f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.465+0000 7f822b200640 1 --2- 172.21.15.19:0/3543676602 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f822c1442c0 0x7f822c138f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.465+0000 7f822b200640 1 --2- 172.21.15.19:0/3543676602 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f822c0d00d0 0x7f822c138a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.465+0000 7f822b200640 1 --2- 172.21.15.19:0/3543676602 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f822c0cefe0 0x7f822c15a6e0 secure :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f821c00d5f0 tx=0x7f821c00dab0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.465+0000 7f822b200640 1 -- 172.21.15.19:0/3543676602 >> 172.21.15.19:0/3543676602 conn(0x7f822c0ccad0 msgr2=0x7f822c0cc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:53.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.467+0000 7f822b200640 1 -- 172.21.15.19:0/3543676602 shutdown_connections 2024-03-28T13:20:53.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.467+0000 7f822b200640 1 -- 172.21.15.19:0/3543676602 wait complete. 2024-03-28T13:20:53.495 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:20:53.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.677+0000 7fbbefa00640 1 Processor -- start 2024-03-28T13:20:53.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.677+0000 7fbbefa00640 1 -- start start 2024-03-28T13:20:53.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.678+0000 7fbbefa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe815c390 0x7fbbe8073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:53.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.678+0000 7fbbefa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbe80af830 0x7fbbe8160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:53.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.679+0000 7fbbefa00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbe815fe80 con 0x7fbbe8160b80 2024-03-28T13:20:53.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.679+0000 7fbbefa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fbbe815c770 con 0x7fbbe80af830 2024-03-28T13:20:53.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.679+0000 7fbbefa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe80ad120 con 0x7fbbe815c390 2024-03-28T13:20:53.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.679+0000 7fbbe7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbe80af830 0x7fbbe8160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:53.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.679+0000 7fbbe7e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fbbe80af830 0x7fbbe8160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58072/0 (socket says 172.21.15.19:58072) 2024-03-28T13:20:53.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.679+0000 7fbbe7e00640 1 -- 172.21.15.19:0/742385916 learned_addr learned my addr 172.21.15.19:0/742385916 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:53.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.679+0000 7fbbed000640 1 --2- 172.21.15.19:0/742385916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe815c390 0x7fbbe8073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:53.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.680+0000 7fbbe7e00640 1 -- 172.21.15.19:0/742385916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe815c390 msgr2=0x7fbbe8073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:53.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.680+0000 7fbbe7e00640 1 --2- 172.21.15.19:0/742385916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe815c390 0x7fbbe8073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.680+0000 7fbbe7e00640 1 -- 172.21.15.19:0/742385916 >> v1:172.21.15.188:6789/0 conn(0x7fbbe8160b80 legacy=0x7fbbe80d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:53.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.680+0000 7fbbe7e00640 1 -- 172.21.15.19:0/742385916 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe80ad400 con 0x7fbbe80af830 2024-03-28T13:20:53.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.681+0000 7fbbe7e00640 1 --2- 172.21.15.19:0/742385916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbe80af830 0x7fbbe8160640 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009870 tx=0x7fbbd402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d55e3b7ce5d57b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:53.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.681+0000 7fbbe7400640 1 -- 172.21.15.19:0/742385916 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4035020 con 0x7fbbe80af830 2024-03-28T13:20:53.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.681+0000 7fbbe7400640 1 -- 172.21.15.19:0/742385916 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbd4031640 con 0x7fbbe80af830 2024-03-28T13:20:53.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.681+0000 7fbbe7400640 1 -- 172.21.15.19:0/742385916 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd402d040 con 0x7fbbe80af830 2024-03-28T13:20:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.682+0000 7fbbefa00640 1 -- 172.21.15.19:0/742385916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbe80af830 msgr2=0x7fbbe8160640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.682+0000 7fbbefa00640 1 --2- 172.21.15.19:0/742385916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbe80af830 0x7fbbe8160640 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009870 tx=0x7fbbd402b8f0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.682+0000 7fbbefa00640 1 -- 172.21.15.19:0/742385916 shutdown_connections 2024-03-28T13:20:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.682+0000 7fbbefa00640 1 --2- 172.21.15.19:0/742385916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbe80af830 0x7fbbe8160640 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.682+0000 7fbbefa00640 1 --2- 172.21.15.19:0/742385916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe815c390 0x7fbbe8073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.682+0000 7fbbefa00640 1 -- 172.21.15.19:0/742385916 >> 172.21.15.19:0/742385916 conn(0x7fbbe806b990 msgr2=0x7fbbe806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.682+0000 7fbbefa00640 1 -- 172.21.15.19:0/742385916 shutdown_connections 2024-03-28T13:20:53.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.682+0000 7fbbefa00640 1 -- 172.21.15.19:0/742385916 wait complete. 2024-03-28T13:20:53.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.684+0000 7fbbefa00640 1 Processor -- start 2024-03-28T13:20:53.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.684+0000 7fbbefa00640 1 -- start start 2024-03-28T13:20:53.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.685+0000 7fbbefa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe80af830 0x7fbbe80b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:53.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.685+0000 7fbbefa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbe815c390 0x7fbbe80b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:53.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.685+0000 7fbbefa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbe8160b80 0x7fbbe814d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:53.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.685+0000 7fbbefa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe80cf980 con 0x7fbbe8160b80 2024-03-28T13:20:53.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.685+0000 7fbbefa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8160120 con 0x7fbbe815c390 2024-03-28T13:20:53.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.685+0000 7fbbefa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe80d0d60 con 0x7fbbe80af830 2024-03-28T13:20:53.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.686+0000 7fbbeda00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbe8160b80 0x7fbbe814d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:53.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.686+0000 7fbbeda00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbe8160b80 0x7fbbe814d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58088/0 (socket says 172.21.15.19:58088) 2024-03-28T13:20:53.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.686+0000 7fbbeda00640 1 -- 172.21.15.19:0/1018354491 learned_addr learned my addr 172.21.15.19:0/1018354491 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:53.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.686+0000 7fbbed000640 1 --2- 172.21.15.19:0/1018354491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe80af830 0x7fbbe80b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:53.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.686+0000 7fbbe7e00640 1 --2- 172.21.15.19:0/1018354491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbe815c390 0x7fbbe80b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:53.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.686+0000 7fbbeda00640 1 -- 172.21.15.19:0/1018354491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe80af830 msgr2=0x7fbbe80b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:53.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.686+0000 7fbbeda00640 1 --2- 172.21.15.19:0/1018354491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe80af830 0x7fbbe80b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.686+0000 7fbbeda00640 1 -- 172.21.15.19:0/1018354491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbe815c390 msgr2=0x7fbbe80b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:53.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.686+0000 7fbbeda00640 1 --2- 172.21.15.19:0/1018354491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbe815c390 0x7fbbe80b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.686+0000 7fbbeda00640 1 -- 172.21.15.19:0/1018354491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe80b11d0 con 0x7fbbe8160b80 2024-03-28T13:20:53.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.686+0000 7fbbe7e00640 1 --2- 172.21.15.19:0/1018354491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbe815c390 0x7fbbe80b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:53.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.687+0000 7fbbeda00640 1 --2- 172.21.15.19:0/1018354491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbe8160b80 0x7fbbe814d120 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000b9e0 tx=0x7fbbe000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:53.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.687+0000 7fbbe4c00640 1 -- 172.21.15.19:0/1018354491 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0003b70 con 0x7fbbe8160b80 2024-03-28T13:20:53.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.687+0000 7fbbe4c00640 1 -- 172.21.15.19:0/1018354491 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbe0003d10 con 0x7fbbe8160b80 2024-03-28T13:20:53.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.687+0000 7fbbe4c00640 1 -- 172.21.15.19:0/1018354491 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe000cb60 con 0x7fbbe8160b80 2024-03-28T13:20:53.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.687+0000 7fbbefa00640 1 -- 172.21.15.19:0/1018354491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe814dbc0 con 0x7fbbe8160b80 2024-03-28T13:20:53.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.687+0000 7fbbefa00640 1 -- 172.21.15.19:0/1018354491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe814e180 con 0x7fbbe8160b80 2024-03-28T13:20:53.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.688+0000 7fbbefa00640 1 -- 172.21.15.19:0/1018354491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe815a480 con 0x7fbbe8160b80 2024-03-28T13:20:53.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.690+0000 7fbbe4c00640 1 -- 172.21.15.19:0/1018354491 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fbbe000cd00 con 0x7fbbe8160b80 2024-03-28T13:20:53.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.691+0000 7fbbe4c00640 1 --2- 172.21.15.19:0/1018354491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbbc806c560 0x7fbbc806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:53.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.691+0000 7fbbe4c00640 1 -- 172.21.15.19:0/1018354491 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7fbbe008b510 con 0x7fbbe8160b80 2024-03-28T13:20:53.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.691+0000 7fbbed000640 1 --2- 172.21.15.19:0/1018354491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbbc806c560 0x7fbbc806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:53.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.693+0000 7fbbed000640 1 --2- 172.21.15.19:0/1018354491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbbc806c560 0x7fbbc806ea20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc006fd0 tx=0x7fbbdc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:53.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.696+0000 7fbbe4c00640 1 -- 172.21.15.19:0/1018354491 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fbbe815a480 con 0x7fbbe8160b80 2024-03-28T13:20:53.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.900+0000 7fbbefa00640 1 -- 172.21.15.19:0/1018354491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbbe80685c0 con 0x7fbbe8160b80 2024-03-28T13:20:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.903+0000 7fbbe4c00640 1 -- 172.21.15.19:0/1018354491 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16696 (secure 0 0 0) 0x7fbbe0059b20 con 0x7fbbe8160b80 2024-03-28T13:20:53.903 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:53.903 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":118,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:20:52.900349+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:20:50.748212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-28T13:20:51.888444+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:20:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.906+0000 7fbbefa00640 1 -- 172.21.15.19:0/1018354491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbbc806c560 msgr2=0x7fbbc806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.906+0000 7fbbefa00640 1 --2- 172.21.15.19:0/1018354491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbbc806c560 0x7fbbc806ea20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc006fd0 tx=0x7fbbdc008040 comp rx=0 tx=0).stop 2024-03-28T13:20:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.906+0000 7fbbefa00640 1 -- 172.21.15.19:0/1018354491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbe8160b80 msgr2=0x7fbbe814d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.906+0000 7fbbefa00640 1 --2- 172.21.15.19:0/1018354491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbe8160b80 0x7fbbe814d120 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000b9e0 tx=0x7fbbe000bea0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.906+0000 7fbbefa00640 1 -- 172.21.15.19:0/1018354491 shutdown_connections 2024-03-28T13:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.906+0000 7fbbefa00640 1 --2- 172.21.15.19:0/1018354491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fbbc806c560 0x7fbbc806ea20 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.906+0000 7fbbefa00640 1 --2- 172.21.15.19:0/1018354491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbbe8160b80 0x7fbbe814d120 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.906+0000 7fbbefa00640 1 --2- 172.21.15.19:0/1018354491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbe815c390 0x7fbbe80b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.906+0000 7fbbefa00640 1 --2- 172.21.15.19:0/1018354491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbe80af830 0x7fbbe80b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.906+0000 7fbbefa00640 1 -- 172.21.15.19:0/1018354491 >> 172.21.15.19:0/1018354491 conn(0x7fbbe806b990 msgr2=0x7fbbe806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.906+0000 7fbbefa00640 1 -- 172.21.15.19:0/1018354491 shutdown_connections 2024-03-28T13:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:53.906+0000 7fbbefa00640 1 -- 172.21.15.19:0/1018354491 wait complete. 2024-03-28T13:20:53.925 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:20:54.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.143+0000 7f603da00640 1 Processor -- start 2024-03-28T13:20:54.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.143+0000 7f603da00640 1 -- start start 2024-03-28T13:20:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.144+0000 7f603da00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60380af830 0x7f6038073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.144+0000 7f603da00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f60380d0650 0x7f60380d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.144+0000 7f603da00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f603815fe80 con 0x7f603815c390 2024-03-28T13:20:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.144+0000 7f603da00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f603815c770 con 0x7f60380d0650 2024-03-28T13:20:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.144+0000 7f603da00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f60380ad120 con 0x7f60380af830 2024-03-28T13:20:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.145+0000 7f6036a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f603815c390 0x7f6038160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41270/0 (socket says 172.21.15.19:41270) 2024-03-28T13:20:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.145+0000 7f6036a00640 1 -- 172.21.15.19:0/1817469053 learned_addr learned my addr 172.21.15.19:0/1817469053 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:54.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.145+0000 7f6036000640 1 --2- 172.21.15.19:0/1817469053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60380af830 0x7f6038073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.145+0000 7f6037400640 1 --2- 172.21.15.19:0/1817469053 >> v2:172.21.15.188:3300/0 conn(0x7f60380d0650 0x7f60380d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.145+0000 7f6035600640 1 -- 172.21.15.19:0/1817469053 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3144446967 0 0) 0x7f603815fe80 con 0x7f603815c390 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.145+0000 7f6035600640 1 -- 172.21.15.19:0/1817469053 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6018003520 con 0x7f603815c390 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.145+0000 7f6035600640 1 -- 172.21.15.19:0/1817469053 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2417533618 0 0) 0x7f6018003520 con 0x7f603815c390 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.145+0000 7f6035600640 1 -- 172.21.15.19:0/1817469053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60380af830 msgr2=0x7f6038073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.145+0000 7f6035600640 1 --2- 172.21.15.19:0/1817469053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60380af830 0x7f6038073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.145+0000 7f6035600640 1 -- 172.21.15.19:0/1817469053 >> v2:172.21.15.188:3300/0 conn(0x7f60380d0650 msgr2=0x7f60380d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.145+0000 7f6035600640 1 --2- 172.21.15.19:0/1817469053 >> v2:172.21.15.188:3300/0 conn(0x7f60380d0650 0x7f60380d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.145+0000 7f6035600640 1 -- 172.21.15.19:0/1817469053 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60380ad400 con 0x7f603815c390 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6035600640 1 -- 172.21.15.19:0/1817469053 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f6028001670 con 0x7f603815c390 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6036000640 1 --2- 172.21.15.19:0/1817469053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60380af830 0x7f6038073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6035600640 1 -- 172.21.15.19:0/1817469053 >> v1:172.21.15.19:6789/0 conn(0x7f603815c390 legacy=0x7f6038160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6035600640 1 --2- 172.21.15.19:0/1817469053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60380af830 0x7f60180039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6035600640 1 --2- 172.21.15.19:0/1817469053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6018003f40 0x7f6018004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6035600640 1 --2- 172.21.15.19:0/1817469053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6018004d40 0x7f60180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6035600640 1 -- 172.21.15.19:0/1817469053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60380ad120 con 0x7f6018003f40 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6035600640 1 -- 172.21.15.19:0/1817469053 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f601800b9c0 con 0x7f6018004d40 2024-03-28T13:20:54.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6035600640 1 -- 172.21.15.19:0/1817469053 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f601800bbc0 con 0x7f60380af830 2024-03-28T13:20:54.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6036000640 1 --2- 172.21.15.19:0/1817469053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6018003f40 0x7f6018004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6036a00640 1 --2- 172.21.15.19:0/1817469053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60380af830 0x7f60180039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6037400640 1 --2- 172.21.15.19:0/1817469053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6018004d40 0x7f60180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6036000640 1 -- 172.21.15.19:0/1817469053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60380af830 msgr2=0x7f60180039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:54.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6036000640 1 --2- 172.21.15.19:0/1817469053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60380af830 0x7f60180039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6036000640 1 -- 172.21.15.19:0/1817469053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6018004d40 msgr2=0x7f60180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:54.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6036000640 1 --2- 172.21.15.19:0/1817469053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6018004d40 0x7f60180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6036000640 1 -- 172.21.15.19:0/1817469053 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6018003520 con 0x7f6018003f40 2024-03-28T13:20:54.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.146+0000 7f6037400640 1 --2- 172.21.15.19:0/1817469053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6018004d40 0x7f60180051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:54.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.147+0000 7f6036000640 1 --2- 172.21.15.19:0/1817469053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6018003f40 0x7f6018004800 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f60200073e0 tx=0x7f6018005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc210143053318a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:54.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.148+0000 7f6035600640 1 -- 172.21.15.19:0/1817469053 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020039070 con 0x7f6018003f40 2024-03-28T13:20:54.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.148+0000 7f6035600640 1 -- 172.21.15.19:0/1817469053 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f602002cd30 con 0x7f6018003f40 2024-03-28T13:20:54.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.148+0000 7f6035600640 1 -- 172.21.15.19:0/1817469053 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020009c80 con 0x7f6018003f40 2024-03-28T13:20:54.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.148+0000 7f603da00640 1 -- 172.21.15.19:0/1817469053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6018003f40 msgr2=0x7f6018004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:54.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.148+0000 7f603da00640 1 --2- 172.21.15.19:0/1817469053 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6018003f40 0x7f6018004800 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f60200073e0 tx=0x7f6018005830 comp rx=0 tx=0).stop 2024-03-28T13:20:54.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.148+0000 7f6037400640 1 -- 172.21.15.19:0/1817469053 reap_dead start 2024-03-28T13:20:54.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.148+0000 7f603da00640 1 -- 172.21.15.19:0/1817469053 shutdown_connections 2024-03-28T13:20:54.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.148+0000 7f603da00640 1 -- 172.21.15.19:0/1817469053 >> 172.21.15.19:0/1817469053 conn(0x7f603806b990 msgr2=0x7f603806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:54.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.149+0000 7f603da00640 1 -- 172.21.15.19:0/1817469053 shutdown_connections 2024-03-28T13:20:54.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.149+0000 7f603da00640 1 -- 172.21.15.19:0/1817469053 wait complete. 2024-03-28T13:20:54.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.150+0000 7f603da00640 1 Processor -- start 2024-03-28T13:20:54.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.150+0000 7f603da00640 1 -- start start 2024-03-28T13:20:54.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.151+0000 7f603da00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6018003f40 0x7f60381f4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.151+0000 7f603da00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603815a7e0 0x7f603815ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.151+0000 7f603da00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60381512d0 0x7f6038151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.151+0000 7f603da00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6038160d50 con 0x7f603815a7e0 2024-03-28T13:20:54.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.151+0000 7f603da00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6038160120 con 0x7f60381512d0 2024-03-28T13:20:54.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.151+0000 7f603da00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f60380cf980 con 0x7f6018003f40 2024-03-28T13:20:54.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.152+0000 7f6036000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603815a7e0 0x7f603815ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.152+0000 7f6036000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603815a7e0 0x7f603815ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58114/0 (socket says 172.21.15.19:58114) 2024-03-28T13:20:54.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.152+0000 7f6036000640 1 -- 172.21.15.19:0/1799451686 learned_addr learned my addr 172.21.15.19:0/1799451686 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:54.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.152+0000 7f6036a00640 1 --2- 172.21.15.19:0/1799451686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6018003f40 0x7f60381f4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.152+0000 7f6036000640 1 -- 172.21.15.19:0/1799451686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6018003f40 msgr2=0x7f60381f4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:54.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.152+0000 7f6036000640 1 --2- 172.21.15.19:0/1799451686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6018003f40 0x7f60381f4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.152+0000 7f6037400640 1 --2- 172.21.15.19:0/1799451686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60381512d0 0x7f6038151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.152+0000 7f6036000640 1 -- 172.21.15.19:0/1799451686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60381512d0 msgr2=0x7f6038151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.152+0000 7f6036000640 1 --2- 172.21.15.19:0/1799451686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60381512d0 0x7f6038151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.152+0000 7f6036000640 1 -- 172.21.15.19:0/1799451686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6038151ee0 con 0x7f603815a7e0 2024-03-28T13:20:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.152+0000 7f6036000640 1 --2- 172.21.15.19:0/1799451686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603815a7e0 0x7f603815ac60 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f60200073e0 tx=0x7f6020004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.153+0000 7f602ea00640 1 -- 172.21.15.19:0/1799451686 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020042020 con 0x7f603815a7e0 2024-03-28T13:20:54.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.153+0000 7f602ea00640 1 -- 172.21.15.19:0/1799451686 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f602003a040 con 0x7f603815a7e0 2024-03-28T13:20:54.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.153+0000 7f602ea00640 1 -- 172.21.15.19:0/1799451686 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6020039070 con 0x7f603815a7e0 2024-03-28T13:20:54.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.153+0000 7f603da00640 1 -- 172.21.15.19:0/1799451686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60381521c0 con 0x7f603815a7e0 2024-03-28T13:20:54.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.153+0000 7f603da00640 1 -- 172.21.15.19:0/1799451686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6038152780 con 0x7f603815a7e0 2024-03-28T13:20:54.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.154+0000 7f603da00640 1 -- 172.21.15.19:0/1799451686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60380685c0 con 0x7f603815a7e0 2024-03-28T13:20:54.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.155+0000 7f602ea00640 1 -- 172.21.15.19:0/1799451686 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6020045020 con 0x7f603815a7e0 2024-03-28T13:20:54.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.156+0000 7f602ea00640 1 --2- 172.21.15.19:0/1799451686 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f601406c560 0x7f601406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.156+0000 7f602ea00640 1 -- 172.21.15.19:0/1799451686 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5709+0+0 (secure 0 0 0) 0x7f60200ad6a0 con 0x7f603815a7e0 2024-03-28T13:20:54.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.157+0000 7f6036a00640 1 --2- 172.21.15.19:0/1799451686 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f601406c560 0x7f601406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.157+0000 7f6036a00640 1 --2- 172.21.15.19:0/1799451686 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f601406c560 0x7f601406ea20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6028001eb0 tx=0x7f6028000ed0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:54.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.160+0000 7f602ea00640 1 -- 172.21.15.19:0/1799451686 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f602004c030 con 0x7f603815a7e0 2024-03-28T13:20:54.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.357+0000 7f603da00640 1 -- 172.21.15.19:0/1799451686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f603806f1b0 con 0x7f603815a7e0 2024-03-28T13:20:54.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.359+0000 7f602ea00640 1 -- 172.21.15.19:0/1799451686 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+16696 (secure 0 0 0) 0x7f602007bb90 con 0x7f603815a7e0 2024-03-28T13:20:54.359 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:54.359 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":118,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:20:52.900349+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:20:50.748212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-28T13:20:51.888444+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:20:54.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.362+0000 7f603da00640 1 -- 172.21.15.19:0/1799451686 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f601406c560 msgr2=0x7f601406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:54.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.362+0000 7f603da00640 1 --2- 172.21.15.19:0/1799451686 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f601406c560 0x7f601406ea20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6028001eb0 tx=0x7f6028000ed0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.362+0000 7f603da00640 1 -- 172.21.15.19:0/1799451686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603815a7e0 msgr2=0x7f603815ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:54.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.362+0000 7f603da00640 1 --2- 172.21.15.19:0/1799451686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603815a7e0 0x7f603815ac60 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f60200073e0 tx=0x7f6020004ab0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.362+0000 7f603da00640 1 -- 172.21.15.19:0/1799451686 shutdown_connections 2024-03-28T13:20:54.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.362+0000 7f603da00640 1 --2- 172.21.15.19:0/1799451686 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f601406c560 0x7f601406ea20 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.362+0000 7f603da00640 1 --2- 172.21.15.19:0/1799451686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60381512d0 0x7f6038151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.362+0000 7f603da00640 1 --2- 172.21.15.19:0/1799451686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f603815a7e0 0x7f603815ac60 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.362+0000 7f603da00640 1 --2- 172.21.15.19:0/1799451686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6018003f40 0x7f60381f4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.362+0000 7f603da00640 1 -- 172.21.15.19:0/1799451686 >> 172.21.15.19:0/1799451686 conn(0x7f603806b990 msgr2=0x7f6038071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:54.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.362+0000 7f603da00640 1 -- 172.21.15.19:0/1799451686 shutdown_connections 2024-03-28T13:20:54.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.362+0000 7f603da00640 1 -- 172.21.15.19:0/1799451686 wait complete. 2024-03-28T13:20:54.381 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-28T13:20:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.605+0000 7fed4ac00640 1 Processor -- start 2024-03-28T13:20:54.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.605+0000 7fed4ac00640 1 -- start start 2024-03-28T13:20:54.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.606+0000 7fed4ac00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fed44155550 0x7fed44155930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.607+0000 7fed4ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4415f800 0x7fed440d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.607+0000 7fed4ac00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed440b40b0 con 0x7fed440d07c0 2024-03-28T13:20:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.607+0000 7fed4ac00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fed440d0c30 con 0x7fed44155550 2024-03-28T13:20:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.607+0000 7fed4ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fed440b4390 con 0x7fed4415f800 2024-03-28T13:20:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.607+0000 7fed48c00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fed440d07c0 0x7fed440aede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:41278/0 (socket says 172.21.15.19:41278) 2024-03-28T13:20:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.607+0000 7fed48c00640 1 -- 172.21.15.19:0/3944543098 learned_addr learned my addr 172.21.15.19:0/3944543098 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:54.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.607+0000 7fed43e00640 1 --2- 172.21.15.19:0/3944543098 >> v2:172.21.15.188:3300/0 conn(0x7fed44155550 0x7fed44155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.607+0000 7fed3be00640 1 --2- 172.21.15.19:0/3944543098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4415f800 0x7fed440d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.607+0000 7fed43400640 1 -- 172.21.15.19:0/3944543098 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3416880182 0 0) 0x7fed440b40b0 con 0x7fed440d07c0 2024-03-28T13:20:54.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed43400640 1 -- 172.21.15.19:0/3944543098 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed28003520 con 0x7fed440d07c0 2024-03-28T13:20:54.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed43400640 1 -- 172.21.15.19:0/3944543098 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3480938972 0 0) 0x7fed28003520 con 0x7fed440d07c0 2024-03-28T13:20:54.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed43400640 1 -- 172.21.15.19:0/3944543098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4415f800 msgr2=0x7fed440d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:54.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed43400640 1 --2- 172.21.15.19:0/3944543098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4415f800 0x7fed440d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed43400640 1 -- 172.21.15.19:0/3944543098 >> v2:172.21.15.188:3300/0 conn(0x7fed44155550 msgr2=0x7fed44155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:54.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed43400640 1 --2- 172.21.15.19:0/3944543098 >> v2:172.21.15.188:3300/0 conn(0x7fed44155550 0x7fed44155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed43400640 1 -- 172.21.15.19:0/3944543098 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed440b4670 con 0x7fed440d07c0 2024-03-28T13:20:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed43400640 1 -- 172.21.15.19:0/3944543098 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fed3c0019f0 con 0x7fed440d07c0 2024-03-28T13:20:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed43400640 1 -- 172.21.15.19:0/3944543098 >> v1:172.21.15.19:6789/0 conn(0x7fed440d07c0 legacy=0x7fed440aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed43400640 1 --2- 172.21.15.19:0/3944543098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed280039b0 0x7fed28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed43400640 1 --2- 172.21.15.19:0/3944543098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4415f800 0x7fed28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed43400640 1 --2- 172.21.15.19:0/3944543098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed28004d40 0x7fed280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed43400640 1 -- 172.21.15.19:0/3944543098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed440d0c30 con 0x7fed28004d40 2024-03-28T13:20:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed43400640 1 -- 172.21.15.19:0/3944543098 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed440b4390 con 0x7fed280039b0 2024-03-28T13:20:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed43400640 1 -- 172.21.15.19:0/3944543098 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fed2800b9c0 con 0x7fed4415f800 2024-03-28T13:20:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.608+0000 7fed48c00640 1 --2- 172.21.15.19:0/3944543098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed28004d40 0x7fed280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.609+0000 7fed43e00640 1 --2- 172.21.15.19:0/3944543098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed280039b0 0x7fed28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.609+0000 7fed3be00640 1 --2- 172.21.15.19:0/3944543098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4415f800 0x7fed28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.609+0000 7fed48c00640 1 -- 172.21.15.19:0/3944543098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4415f800 msgr2=0x7fed28004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:54.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.609+0000 7fed48c00640 1 --2- 172.21.15.19:0/3944543098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4415f800 0x7fed28004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.609+0000 7fed48c00640 1 -- 172.21.15.19:0/3944543098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed280039b0 msgr2=0x7fed28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.609+0000 7fed48c00640 1 --2- 172.21.15.19:0/3944543098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed280039b0 0x7fed28003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.609+0000 7fed48c00640 1 -- 172.21.15.19:0/3944543098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed28003520 con 0x7fed28004d40 2024-03-28T13:20:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.610+0000 7fed48c00640 1 --2- 172.21.15.19:0/3944543098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed28004d40 0x7fed280051c0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00a920 tx=0x7fed28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d4b41073a2d10a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.610+0000 7fed43400640 1 -- 172.21.15.19:0/3944543098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c00a4c0 con 0x7fed28004d40 2024-03-28T13:20:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.610+0000 7fed43400640 1 -- 172.21.15.19:0/3944543098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed3c034650 con 0x7fed28004d40 2024-03-28T13:20:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.610+0000 7fed43400640 1 -- 172.21.15.19:0/3944543098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c034a40 con 0x7fed28004d40 2024-03-28T13:20:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.611+0000 7fed4ac00640 1 -- 172.21.15.19:0/3944543098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed28004d40 msgr2=0x7fed280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.611+0000 7fed4ac00640 1 --2- 172.21.15.19:0/3944543098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed28004d40 0x7fed280051c0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00a920 tx=0x7fed28005830 comp rx=0 tx=0).stop 2024-03-28T13:20:54.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.611+0000 7fed48c00640 1 -- 172.21.15.19:0/3944543098 reap_dead start 2024-03-28T13:20:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.611+0000 7fed4ac00640 1 -- 172.21.15.19:0/3944543098 shutdown_connections 2024-03-28T13:20:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.611+0000 7fed4ac00640 1 -- 172.21.15.19:0/3944543098 >> 172.21.15.19:0/3944543098 conn(0x7fed4406b750 msgr2=0x7fed4406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.611+0000 7fed4ac00640 1 -- 172.21.15.19:0/3944543098 shutdown_connections 2024-03-28T13:20:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.611+0000 7fed4ac00640 1 -- 172.21.15.19:0/3944543098 wait complete. 2024-03-28T13:20:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.613+0000 7fed4ac00640 1 Processor -- start 2024-03-28T13:20:54.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.613+0000 7fed4ac00640 1 -- start start 2024-03-28T13:20:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed4ac00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed28004d40 0x7fed441446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed4ac00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed44072790 0x7fed44099d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed4ac00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4409a280 0x7fed4409d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed4ac00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed4415fc80 con 0x7fed44072790 2024-03-28T13:20:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed4ac00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fed440b4c30 con 0x7fed28004d40 2024-03-28T13:20:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed4ac00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fed4415ba20 con 0x7fed4409a280 2024-03-28T13:20:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed3be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed44072790 0x7fed44099d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed3be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed44072790 0x7fed44099d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58128/0 (socket says 172.21.15.19:58128) 2024-03-28T13:20:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed3be00640 1 -- 172.21.15.19:0/830645326 learned_addr learned my addr 172.21.15.19:0/830645326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed48c00640 1 --2- 172.21.15.19:0/830645326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4409a280 0x7fed4409d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed43e00640 1 --2- 172.21.15.19:0/830645326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed28004d40 0x7fed441446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed3be00640 1 -- 172.21.15.19:0/830645326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4409a280 msgr2=0x7fed4409d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed3be00640 1 --2- 172.21.15.19:0/830645326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4409a280 0x7fed4409d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed3be00640 1 -- 172.21.15.19:0/830645326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed28004d40 msgr2=0x7fed441446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed3be00640 1 --2- 172.21.15.19:0/830645326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed28004d40 0x7fed441446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.614+0000 7fed3be00640 1 -- 172.21.15.19:0/830645326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4415caa0 con 0x7fed44072790 2024-03-28T13:20:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.615+0000 7fed3be00640 1 --2- 172.21.15.19:0/830645326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed44072790 0x7fed44099d40 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fed2c011630 tx=0x7fed2c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.615+0000 7fed40c00640 1 -- 172.21.15.19:0/830645326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c00f920 con 0x7fed44072790 2024-03-28T13:20:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.615+0000 7fed40c00640 1 -- 172.21.15.19:0/830645326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed2c00fac0 con 0x7fed44072790 2024-03-28T13:20:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.615+0000 7fed40c00640 1 -- 172.21.15.19:0/830645326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c00fe70 con 0x7fed44072790 2024-03-28T13:20:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.615+0000 7fed4ac00640 1 -- 172.21.15.19:0/830645326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed44153c00 con 0x7fed44072790 2024-03-28T13:20:54.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.615+0000 7fed4ac00640 1 -- 172.21.15.19:0/830645326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed44099a20 con 0x7fed44072790 2024-03-28T13:20:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.616+0000 7fed4ac00640 1 -- 172.21.15.19:0/830645326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed440685c0 con 0x7fed44072790 2024-03-28T13:20:54.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.617+0000 7fed40c00640 1 -- 172.21.15.19:0/830645326 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fed2c015460 con 0x7fed44072790 2024-03-28T13:20:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.618+0000 7fed40c00640 1 --2- 172.21.15.19:0/830645326 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fed2006c530 0x7fed2006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.618+0000 7fed40c00640 1 -- 172.21.15.19:0/830645326 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7fed2c08c100 con 0x7fed44072790 2024-03-28T13:20:54.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.619+0000 7fed43e00640 1 --2- 172.21.15.19:0/830645326 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fed2006c530 0x7fed2006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:54.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.620+0000 7fed43e00640 1 --2- 172.21.15.19:0/830645326 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fed2006c530 0x7fed2006e9f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fed4406d640 tx=0x7fed34008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.623+0000 7fed40c00640 1 -- 172.21.15.19:0/830645326 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fed2c019030 con 0x7fed44072790 2024-03-28T13:20:54.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:54.872+0000 7fed4ac00640 1 -- 172.21.15.19:0/830645326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fed440b3ad0 con 0x7fed44072790 2024-03-28T13:20:55.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.465+0000 7fed40c00640 1 -- 172.21.15.19:0/830645326 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) ==== 118+0+0 (secure 0 0 0) 0x7fed2c05a710 con 0x7fed44072790 2024-03-28T13:20:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.468+0000 7fed4ac00640 1 -- 172.21.15.19:0/830645326 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fed2006c530 msgr2=0x7fed2006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.468+0000 7fed4ac00640 1 --2- 172.21.15.19:0/830645326 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fed2006c530 0x7fed2006e9f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fed4406d640 tx=0x7fed34008040 comp rx=0 tx=0).stop 2024-03-28T13:20:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.469+0000 7fed4ac00640 1 -- 172.21.15.19:0/830645326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed44072790 msgr2=0x7fed44099d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.469+0000 7fed4ac00640 1 --2- 172.21.15.19:0/830645326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed44072790 0x7fed44099d40 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fed2c011630 tx=0x7fed2c011af0 comp rx=0 tx=0).stop 2024-03-28T13:20:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.469+0000 7fed4ac00640 1 -- 172.21.15.19:0/830645326 shutdown_connections 2024-03-28T13:20:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.469+0000 7fed4ac00640 1 --2- 172.21.15.19:0/830645326 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fed2006c530 0x7fed2006e9f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.469+0000 7fed4ac00640 1 --2- 172.21.15.19:0/830645326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed4409a280 0x7fed4409d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.469+0000 7fed4ac00640 1 --2- 172.21.15.19:0/830645326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed44072790 0x7fed44099d40 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.469+0000 7fed4ac00640 1 --2- 172.21.15.19:0/830645326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed28004d40 0x7fed441446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.469+0000 7fed4ac00640 1 -- 172.21.15.19:0/830645326 >> 172.21.15.19:0/830645326 conn(0x7fed4406b750 msgr2=0x7fed4406eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.469+0000 7fed4ac00640 1 -- 172.21.15.19:0/830645326 shutdown_connections 2024-03-28T13:20:55.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.469+0000 7fed4ac00640 1 -- 172.21.15.19:0/830645326 wait complete. 2024-03-28T13:20:55.490 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-28T13:20:55.491 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-28T13:20:55.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.707+0000 7f0f9f200640 1 Processor -- start 2024-03-28T13:20:55.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.708+0000 7f0f9f200640 1 -- start start 2024-03-28T13:20:55.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.709+0000 7f0f9f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f98072c80 0x7f0f980b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:55.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.709+0000 7f0f9f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f980b4c70 con 0x7f0f980b4890 2024-03-28T13:20:55.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.709+0000 7f0f9f200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f9815fb30 con 0x7f0f9815f6a0 2024-03-28T13:20:55.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.709+0000 7f0f9f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0f9815b7b0 con 0x7f0f98072c80 2024-03-28T13:20:55.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.709+0000 7f0f9de00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f0f980b4890 0x7f0f98072570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53694/0 (socket says 172.21.15.19:53694) 2024-03-28T13:20:55.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.709+0000 7f0f9de00640 1 -- 172.21.15.19:0/381915928 learned_addr learned my addr 172.21.15.19:0/381915928 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9ca00640 1 -- 172.21.15.19:0/381915928 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 94060281 0 0) 0x7f0f980b4c70 con 0x7f0f980b4890 2024-03-28T13:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9ca00640 1 -- 172.21.15.19:0/381915928 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f80003520 con 0x7f0f980b4890 2024-03-28T13:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9e800640 1 --2- 172.21.15.19:0/381915928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f98072c80 0x7f0f980b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9ca00640 1 -- 172.21.15.19:0/381915928 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1078792082 0 0) 0x7f0f80003520 con 0x7f0f980b4890 2024-03-28T13:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9ca00640 1 -- 172.21.15.19:0/381915928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f98072c80 msgr2=0x7f0f980b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9ca00640 1 --2- 172.21.15.19:0/381915928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f98072c80 0x7f0f980b35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9ca00640 1 -- 172.21.15.19:0/381915928 >> v1:172.21.15.188:6789/0 conn(0x7f0f9815f6a0 legacy=0x7f0f980b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9ca00640 1 -- 172.21.15.19:0/381915928 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9815ba90 con 0x7f0f980b4890 2024-03-28T13:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9ca00640 1 -- 172.21.15.19:0/381915928 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f0f8c001670 con 0x7f0f980b4890 2024-03-28T13:20:55.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9ca00640 1 -- 172.21.15.19:0/381915928 >> v1:172.21.15.19:6789/0 conn(0x7f0f980b4890 legacy=0x7f0f98072570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9ca00640 1 --2- 172.21.15.19:0/381915928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f800039e0 0x7f0f80003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9ca00640 1 --2- 172.21.15.19:0/381915928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f98072c80 0x7f0f80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9ca00640 1 --2- 172.21.15.19:0/381915928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f80004d40 0x7f0f800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9ca00640 1 -- 172.21.15.19:0/381915928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9815b7b0 con 0x7f0f800039e0 2024-03-28T13:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9ca00640 1 -- 172.21.15.19:0/381915928 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8000bb50 con 0x7f0f80004d40 2024-03-28T13:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.710+0000 7f0f9ca00640 1 -- 172.21.15.19:0/381915928 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8000bd50 con 0x7f0f98072c80 2024-03-28T13:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.711+0000 7f0f9de00640 1 --2- 172.21.15.19:0/381915928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f800039e0 0x7f0f80003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.711+0000 7f0f9de00640 1 -- 172.21.15.19:0/381915928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f98072c80 msgr2=0x7f0f80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.711+0000 7f0f9de00640 1 --2- 172.21.15.19:0/381915928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f98072c80 0x7f0f80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.711+0000 7f0f9de00640 1 -- 172.21.15.19:0/381915928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f80004d40 msgr2=0x7f0f800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.711+0000 7f0f9de00640 1 --2- 172.21.15.19:0/381915928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f80004d40 0x7f0f800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:55.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.711+0000 7f0f9de00640 1 -- 172.21.15.19:0/381915928 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f80003520 con 0x7f0f800039e0 2024-03-28T13:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.712+0000 7f0f9de00640 1 --2- 172.21.15.19:0/381915928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f800039e0 0x7f0f80003e00 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00c240 tx=0x7f0f80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79ed3715a200371 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.712+0000 7f0f9ca00640 1 -- 172.21.15.19:0/381915928 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c007d10 con 0x7f0f800039e0 2024-03-28T13:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.712+0000 7f0f9ca00640 1 -- 172.21.15.19:0/381915928 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f8c030070 con 0x7f0f800039e0 2024-03-28T13:20:55.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.712+0000 7f0f9ca00640 1 -- 172.21.15.19:0/381915928 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c02cd60 con 0x7f0f800039e0 2024-03-28T13:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.713+0000 7f0f9f200640 1 -- 172.21.15.19:0/381915928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f800039e0 msgr2=0x7f0f80003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.713+0000 7f0f9f200640 1 --2- 172.21.15.19:0/381915928 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f800039e0 0x7f0f80003e00 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00c240 tx=0x7f0f80005830 comp rx=0 tx=0).stop 2024-03-28T13:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.713+0000 7f0f9e800640 1 -- 172.21.15.19:0/381915928 reap_dead start 2024-03-28T13:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.713+0000 7f0f9f200640 1 -- 172.21.15.19:0/381915928 shutdown_connections 2024-03-28T13:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.713+0000 7f0f9f200640 1 -- 172.21.15.19:0/381915928 >> 172.21.15.19:0/381915928 conn(0x7f0f980b1ba0 msgr2=0x7f0f980b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.713+0000 7f0f9f200640 1 -- 172.21.15.19:0/381915928 shutdown_connections 2024-03-28T13:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.713+0000 7f0f9f200640 1 -- 172.21.15.19:0/381915928 wait complete. 2024-03-28T13:20:55.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.715+0000 7f0f9f200640 1 Processor -- start 2024-03-28T13:20:55.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.715+0000 7f0f9f200640 1 -- start start 2024-03-28T13:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.716+0000 7f0f9f200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f800039e0 0x7f0f9814d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.716+0000 7f0f9f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f981506e0 0x7f0f98150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.716+0000 7f0f9f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9814e6c0 0x7f0f9814eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.716+0000 7f0f9f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9815c570 con 0x7f0f9814e6c0 2024-03-28T13:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.716+0000 7f0f9f200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0f9815c050 con 0x7f0f800039e0 2024-03-28T13:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.716+0000 7f0f9f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0f980b2900 con 0x7f0f981506e0 2024-03-28T13:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.716+0000 7f0f9e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9814e6c0 0x7f0f9814eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.716+0000 7f0f9e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9814e6c0 0x7f0f9814eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41846/0 (socket says 172.21.15.19:41846) 2024-03-28T13:20:55.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.716+0000 7f0f9e800640 1 -- 172.21.15.19:0/2779323128 learned_addr learned my addr 172.21.15.19:0/2779323128 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:55.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.717+0000 7f0f9de00640 1 --2- 172.21.15.19:0/2779323128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f800039e0 0x7f0f9814d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:55.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.717+0000 7f0f9d400640 1 --2- 172.21.15.19:0/2779323128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f981506e0 0x7f0f98150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:55.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.717+0000 7f0f9e800640 1 -- 172.21.15.19:0/2779323128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f981506e0 msgr2=0x7f0f98150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.717+0000 7f0f9e800640 1 --2- 172.21.15.19:0/2779323128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f981506e0 0x7f0f98150b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.717+0000 7f0f9e800640 1 -- 172.21.15.19:0/2779323128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f800039e0 msgr2=0x7f0f9814d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.717+0000 7f0f9e800640 1 --2- 172.21.15.19:0/2779323128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f800039e0 0x7f0f9814d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.717+0000 7f0f9e800640 1 -- 172.21.15.19:0/2779323128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9814da20 con 0x7f0f9814e6c0 2024-03-28T13:20:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.717+0000 7f0f9de00640 1 --2- 172.21.15.19:0/2779323128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f800039e0 0x7f0f9814d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.717+0000 7f0f9e800640 1 --2- 172.21.15.19:0/2779323128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9814e6c0 0x7f0f9814eb50 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f0f980b27b0 tx=0x7f0f9400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.717+0000 7f0f92000640 1 -- 172.21.15.19:0/2779323128 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9400edd0 con 0x7f0f9814e6c0 2024-03-28T13:20:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.717+0000 7f0f92000640 1 -- 172.21.15.19:0/2779323128 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f94019070 con 0x7f0f9814e6c0 2024-03-28T13:20:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.717+0000 7f0f92000640 1 -- 172.21.15.19:0/2779323128 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94014660 con 0x7f0f9814e6c0 2024-03-28T13:20:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.717+0000 7f0f9f200640 1 -- 172.21.15.19:0/2779323128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f98151240 con 0x7f0f9814e6c0 2024-03-28T13:20:55.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.717+0000 7f0f9f200640 1 -- 172.21.15.19:0/2779323128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f98151800 con 0x7f0f9814e6c0 2024-03-28T13:20:55.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.719+0000 7f0f9f200640 1 -- 172.21.15.19:0/2779323128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f98075940 con 0x7f0f9814e6c0 2024-03-28T13:20:55.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.719+0000 7f0f92000640 1 -- 172.21.15.19:0/2779323128 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0f94014800 con 0x7f0f9814e6c0 2024-03-28T13:20:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.720+0000 7f0f92000640 1 --2- 172.21.15.19:0/2779323128 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f7806c530 0x7f0f7806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.720+0000 7f0f92000640 1 -- 172.21.15.19:0/2779323128 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f0f9408bb80 con 0x7f0f9814e6c0 2024-03-28T13:20:55.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.721+0000 7f0f9de00640 1 --2- 172.21.15.19:0/2779323128 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f7806c530 0x7f0f7806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:55.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.722+0000 7f0f9de00640 1 --2- 172.21.15.19:0/2779323128 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f7806c530 0x7f0f7806e9f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00c240 tx=0x7f0f8c009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:55.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.725+0000 7f0f92000640 1 -- 172.21.15.19:0/2779323128 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0f9405a100 con 0x7f0f9814e6c0 2024-03-28T13:20:55.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:55.971+0000 7f0f9f200640 1 -- 172.21.15.19:0/2779323128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0f980b4e60 con 0x7f0f9814e6c0 2024-03-28T13:20:56.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.475+0000 7f0f92000640 1 -- 172.21.15.19:0/2779323128 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) ==== 122+0+0 (secure 0 0 0) 0x7f0f94059a00 con 0x7f0f9814e6c0 2024-03-28T13:20:56.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.479+0000 7f0f9f200640 1 -- 172.21.15.19:0/2779323128 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f7806c530 msgr2=0x7f0f7806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:56.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.479+0000 7f0f9f200640 1 --2- 172.21.15.19:0/2779323128 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f7806c530 0x7f0f7806e9f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00c240 tx=0x7f0f8c009030 comp rx=0 tx=0).stop 2024-03-28T13:20:56.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.479+0000 7f0f9f200640 1 -- 172.21.15.19:0/2779323128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9814e6c0 msgr2=0x7f0f9814eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:56.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.479+0000 7f0f9f200640 1 --2- 172.21.15.19:0/2779323128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9814e6c0 0x7f0f9814eb50 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f0f980b27b0 tx=0x7f0f9400e4b0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.479+0000 7f0f9f200640 1 -- 172.21.15.19:0/2779323128 shutdown_connections 2024-03-28T13:20:56.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.479+0000 7f0f9f200640 1 --2- 172.21.15.19:0/2779323128 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0f7806c530 0x7f0f7806e9f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.479+0000 7f0f9f200640 1 --2- 172.21.15.19:0/2779323128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f9814e6c0 0x7f0f9814eb50 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.479+0000 7f0f9f200640 1 --2- 172.21.15.19:0/2779323128 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f981506e0 0x7f0f98150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.479+0000 7f0f9f200640 1 --2- 172.21.15.19:0/2779323128 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f800039e0 0x7f0f9814d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.479+0000 7f0f9f200640 1 -- 172.21.15.19:0/2779323128 >> 172.21.15.19:0/2779323128 conn(0x7f0f980b1ba0 msgr2=0x7f0f98072570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:56.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.479+0000 7f0f9f200640 1 -- 172.21.15.19:0/2779323128 shutdown_connections 2024-03-28T13:20:56.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.479+0000 7f0f9f200640 1 -- 172.21.15.19:0/2779323128 wait complete. 2024-03-28T13:20:56.504 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-28T13:20:56.504 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:20:56.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.727+0000 7f5b9d800640 1 Processor -- start 2024-03-28T13:20:56.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.728+0000 7f5b9d800640 1 -- start start 2024-03-28T13:20:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.729+0000 7f5b9d800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5b9815b5a0 0x7f5b980b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.729+0000 7f5b9d800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5b9815f1b0 0x7f5b980ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.729+0000 7f5b9d800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9806c580 0x7f5b9806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.729+0000 7f5b9d800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f5b9806f7c0 con 0x7f5b9815f1b0 2024-03-28T13:20:56.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.729+0000 7f5b9d800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5b980cd1b0 con 0x7f5b9815b5a0 2024-03-28T13:20:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.729+0000 7f5b9d800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5b980b5b40 con 0x7f5b9806c580 2024-03-28T13:20:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.730+0000 7f5b96000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5b9815f1b0 0x7f5b980ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.730+0000 7f5b96000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f5b9815f1b0 0x7f5b980ce700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41856/0 (socket says 172.21.15.19:41856) 2024-03-28T13:20:56.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.730+0000 7f5b96000640 1 -- 172.21.15.19:0/2515999891 learned_addr learned my addr 172.21.15.19:0/2515999891 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:56.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.730+0000 7f5b96a00640 1 --2- 172.21.15.19:0/2515999891 >> v2:172.21.15.188:3300/0 conn(0x7f5b9815b5a0 0x7f5b980b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:56.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.730+0000 7f5b97400640 1 --2- 172.21.15.19:0/2515999891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9806c580 0x7f5b9806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:56.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.730+0000 7f5b96000640 1 -- 172.21.15.19:0/2515999891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9806c580 msgr2=0x7f5b9806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:56.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.730+0000 7f5b96000640 1 --2- 172.21.15.19:0/2515999891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9806c580 0x7f5b9806ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.730+0000 7f5b96000640 1 -- 172.21.15.19:0/2515999891 >> v2:172.21.15.188:3300/0 conn(0x7f5b9815b5a0 msgr2=0x7f5b980b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.730+0000 7f5b96000640 1 --2- 172.21.15.19:0/2515999891 >> v2:172.21.15.188:3300/0 conn(0x7f5b9815b5a0 0x7f5b980b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.730+0000 7f5b96000640 1 -- 172.21.15.19:0/2515999891 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b980b52a0 con 0x7f5b9815f1b0 2024-03-28T13:20:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.730+0000 7f5b97400640 1 --2- 172.21.15.19:0/2515999891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9806c580 0x7f5b9806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.731+0000 7f5b96000640 1 --2- 172.21.15.19:0/2515999891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9815f1b0 0x7f5b980ce700 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f5b98073020 tx=0x7f5b8002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395fb5aa0fa13f4a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.732+0000 7f5b95600640 1 -- 172.21.15.19:0/2515999891 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8002c9e0 con 0x7f5b9815f1b0 2024-03-28T13:20:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.732+0000 7f5b95600640 1 -- 172.21.15.19:0/2515999891 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b8002cb80 con 0x7f5b9815f1b0 2024-03-28T13:20:56.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.732+0000 7f5b95600640 1 -- 172.21.15.19:0/2515999891 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b800343f0 con 0x7f5b9815f1b0 2024-03-28T13:20:56.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.732+0000 7f5b9d800640 1 -- 172.21.15.19:0/2515999891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9815f1b0 msgr2=0x7f5b980ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:56.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.732+0000 7f5b9d800640 1 --2- 172.21.15.19:0/2515999891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9815f1b0 0x7f5b980ce700 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f5b98073020 tx=0x7f5b8002be40 comp rx=0 tx=0).stop 2024-03-28T13:20:56.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.733+0000 7f5b9d800640 1 -- 172.21.15.19:0/2515999891 shutdown_connections 2024-03-28T13:20:56.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.733+0000 7f5b9d800640 1 --2- 172.21.15.19:0/2515999891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9806c580 0x7f5b9806ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.733+0000 7f5b9d800640 1 --2- 172.21.15.19:0/2515999891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9815f1b0 0x7f5b980ce700 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.733+0000 7f5b9d800640 1 --2- 172.21.15.19:0/2515999891 >> v2:172.21.15.188:3300/0 conn(0x7f5b9815b5a0 0x7f5b980b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.733+0000 7f5b9d800640 1 -- 172.21.15.19:0/2515999891 >> 172.21.15.19:0/2515999891 conn(0x7f5b98071df0 msgr2=0x7f5b980721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.733+0000 7f5b9d800640 1 -- 172.21.15.19:0/2515999891 shutdown_connections 2024-03-28T13:20:56.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.733+0000 7f5b9d800640 1 -- 172.21.15.19:0/2515999891 wait complete. 2024-03-28T13:20:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.734+0000 7f5b9d800640 1 Processor -- start 2024-03-28T13:20:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.735+0000 7f5b9d800640 1 -- start start 2024-03-28T13:20:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.735+0000 7f5b9d800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9806c580 0x7f5b98159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.735+0000 7f5b9d800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9815b5a0 0x7f5b98157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.735+0000 7f5b9d800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b9815f1b0 0x7f5b9815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.735+0000 7f5b9d800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9815b980 con 0x7f5b9815b5a0 2024-03-28T13:20:56.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.735+0000 7f5b9d800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5b980b5860 con 0x7f5b9815f1b0 2024-03-28T13:20:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.736+0000 7f5b9d800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5b980cec40 con 0x7f5b9806c580 2024-03-28T13:20:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.736+0000 7f5b96000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9815b5a0 0x7f5b98157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.736+0000 7f5b96000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9815b5a0 0x7f5b98157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41858/0 (socket says 172.21.15.19:41858) 2024-03-28T13:20:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.736+0000 7f5b96000640 1 -- 172.21.15.19:0/2083198594 learned_addr learned my addr 172.21.15.19:0/2083198594 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.736+0000 7f5b97400640 1 --2- 172.21.15.19:0/2083198594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b9815f1b0 0x7f5b9815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.736+0000 7f5b96a00640 1 --2- 172.21.15.19:0/2083198594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9806c580 0x7f5b98159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.736+0000 7f5b96000640 1 -- 172.21.15.19:0/2083198594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9806c580 msgr2=0x7f5b98159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:56.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.736+0000 7f5b96000640 1 --2- 172.21.15.19:0/2083198594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9806c580 0x7f5b98159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.736+0000 7f5b96000640 1 -- 172.21.15.19:0/2083198594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b9815f1b0 msgr2=0x7f5b9815cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.736+0000 7f5b96000640 1 --2- 172.21.15.19:0/2083198594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b9815f1b0 0x7f5b9815cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.736+0000 7f5b96000640 1 -- 172.21.15.19:0/2083198594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b98158290 con 0x7f5b9815b5a0 2024-03-28T13:20:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.737+0000 7f5b96000640 1 --2- 172.21.15.19:0/2083198594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9815b5a0 0x7f5b98157890 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f5b80000f80 tx=0x7f5b80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.737+0000 7f5b8ea00640 1 -- 172.21.15.19:0/2083198594 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b80044020 con 0x7f5b9815b5a0 2024-03-28T13:20:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.737+0000 7f5b8ea00640 1 -- 172.21.15.19:0/2083198594 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b800406b0 con 0x7f5b9815b5a0 2024-03-28T13:20:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.737+0000 7f5b8ea00640 1 -- 172.21.15.19:0/2083198594 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b80009510 con 0x7f5b9815b5a0 2024-03-28T13:20:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.737+0000 7f5b9d800640 1 -- 172.21.15.19:0/2083198594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b98158570 con 0x7f5b9815b5a0 2024-03-28T13:20:56.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.737+0000 7f5b9d800640 1 -- 172.21.15.19:0/2083198594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b98158b30 con 0x7f5b9815b5a0 2024-03-28T13:20:56.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.738+0000 7f5b9d800640 1 -- 172.21.15.19:0/2083198594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b98067a90 con 0x7f5b9815b5a0 2024-03-28T13:20:56.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.739+0000 7f5b8ea00640 1 -- 172.21.15.19:0/2083198594 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5b80047020 con 0x7f5b9815b5a0 2024-03-28T13:20:56.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.740+0000 7f5b8ea00640 1 --2- 172.21.15.19:0/2083198594 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5b7406c530 0x7f5b7406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:56.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.740+0000 7f5b8ea00640 1 -- 172.21.15.19:0/2083198594 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f5b800ae230 con 0x7f5b9815b5a0 2024-03-28T13:20:56.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.741+0000 7f5b96a00640 1 --2- 172.21.15.19:0/2083198594 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5b7406c530 0x7f5b7406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:56.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.742+0000 7f5b96a00640 1 --2- 172.21.15.19:0/2083198594 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5b7406c530 0x7f5b7406e9f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5b88006fd0 tx=0x7f5b88008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:56.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.745+0000 7f5b8ea00640 1 -- 172.21.15.19:0/2083198594 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5b8004e030 con 0x7f5b9815b5a0 2024-03-28T13:20:56.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.990+0000 7f5b9d800640 1 -- 172.21.15.19:0/2083198594 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b980728e0 con 0x7f5b9815b5a0 2024-03-28T13:20:56.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.991+0000 7f5b8ea00640 1 -- 172.21.15.19:0/2083198594 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5186 (secure 0 0 0) 0x7f5b8007c840 con 0x7f5b9815b5a0 2024-03-28T13:20:56.991 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:56.991 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3823671990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3823671990},{"type":"v1","addr":"172.21.15.188:6837","nonce":3823671990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/230519675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":230519675},{"type":"v1","addr":"172.21.15.19:6834","nonce":230519675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3930967142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3930967142},{"type":"v1","addr":"172.21.15.19:6835","nonce":3930967142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:20:52.901269+0000","modified":"2024-03-28T13:20:55.973314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8703":{"gid":8703,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2088538359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2088538359},{"type":"v1","addr":"172.21.15.188:6836","nonce":2088538359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8703,"qdb_cluster":[8703]},"id":8}]} 2024-03-28T13:20:56.992 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 99 2024-03-28T13:20:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.994+0000 7f5b9d800640 1 -- 172.21.15.19:0/2083198594 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5b7406c530 msgr2=0x7f5b7406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.994+0000 7f5b9d800640 1 --2- 172.21.15.19:0/2083198594 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5b7406c530 0x7f5b7406e9f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5b88006fd0 tx=0x7f5b88008040 comp rx=0 tx=0).stop 2024-03-28T13:20:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.994+0000 7f5b9d800640 1 -- 172.21.15.19:0/2083198594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9815b5a0 msgr2=0x7f5b98157890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.994+0000 7f5b9d800640 1 --2- 172.21.15.19:0/2083198594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9815b5a0 0x7f5b98157890 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f5b80000f80 tx=0x7f5b80004a40 comp rx=0 tx=0).stop 2024-03-28T13:20:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.994+0000 7f5b9d800640 1 -- 172.21.15.19:0/2083198594 shutdown_connections 2024-03-28T13:20:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.994+0000 7f5b9d800640 1 --2- 172.21.15.19:0/2083198594 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5b7406c530 0x7f5b7406e9f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.994+0000 7f5b9d800640 1 --2- 172.21.15.19:0/2083198594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5b9815f1b0 0x7f5b9815cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.994+0000 7f5b9d800640 1 --2- 172.21.15.19:0/2083198594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b9815b5a0 0x7f5b98157890 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.994+0000 7f5b9d800640 1 --2- 172.21.15.19:0/2083198594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5b9806c580 0x7f5b98159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:56.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.994+0000 7f5b9d800640 1 -- 172.21.15.19:0/2083198594 >> 172.21.15.19:0/2083198594 conn(0x7f5b98071df0 msgr2=0x7f5b980b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.994+0000 7f5b9d800640 1 -- 172.21.15.19:0/2083198594 shutdown_connections 2024-03-28T13:20:56.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:56.994+0000 7f5b9d800640 1 -- 172.21.15.19:0/2083198594 wait complete. 2024-03-28T13:20:57.013 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:20:57.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.226+0000 7fb088a00640 1 Processor -- start 2024-03-28T13:20:57.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.226+0000 7fb088a00640 1 -- start start 2024-03-28T13:20:57.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.228+0000 7fb088a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb0840d04d0 0x7fb0840b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:57.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.228+0000 7fb088a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0840b3150 con 0x7fb0840cc0b0 2024-03-28T13:20:57.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.228+0000 7fb088a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0841600e0 con 0x7fb08415fc50 2024-03-28T13:20:57.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.228+0000 7fb088a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb08415c610 con 0x7fb0840d04d0 2024-03-28T13:20:57.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.228+0000 7fb082800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fb0840cc0b0 0x7fb08406dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53702/0 (socket says 172.21.15.19:53702) 2024-03-28T13:20:57.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.228+0000 7fb082800640 1 -- 172.21.15.19:0/194277957 learned_addr learned my addr 172.21.15.19:0/194277957 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:20:57.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.228+0000 7fb083200640 1 --2- 172.21.15.19:0/194277957 >> v2:172.21.15.188:3300/0 conn(0x7fb0840d04d0 0x7fb0840b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:57.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.229+0000 7fb081400640 1 -- 172.21.15.19:0/194277957 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3532781126 0 0) 0x7fb0840b3150 con 0x7fb0840cc0b0 2024-03-28T13:20:57.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.229+0000 7fb081400640 1 -- 172.21.15.19:0/194277957 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb068003520 con 0x7fb0840cc0b0 2024-03-28T13:20:57.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.229+0000 7fb081400640 1 -- 172.21.15.19:0/194277957 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1287419452 0 0) 0x7fb068003520 con 0x7fb0840cc0b0 2024-03-28T13:20:57.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.229+0000 7fb081400640 1 -- 172.21.15.19:0/194277957 >> v2:172.21.15.188:3300/0 conn(0x7fb0840d04d0 msgr2=0x7fb0840b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:57.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.229+0000 7fb081400640 1 --2- 172.21.15.19:0/194277957 >> v2:172.21.15.188:3300/0 conn(0x7fb0840d04d0 0x7fb0840b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.229+0000 7fb081400640 1 -- 172.21.15.19:0/194277957 >> v1:172.21.15.188:6789/0 conn(0x7fb08415fc50 legacy=0x7fb0840cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:57.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.229+0000 7fb081400640 1 -- 172.21.15.19:0/194277957 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0841603c0 con 0x7fb0840cc0b0 2024-03-28T13:20:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.229+0000 7fb081400640 1 -- 172.21.15.19:0/194277957 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fb074001670 con 0x7fb0840cc0b0 2024-03-28T13:20:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.229+0000 7fb081400640 1 -- 172.21.15.19:0/194277957 >> v1:172.21.15.19:6789/0 conn(0x7fb0840cc0b0 legacy=0x7fb08406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.229+0000 7fb081400640 1 --2- 172.21.15.19:0/194277957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb068003a00 0x7fb068003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.229+0000 7fb081400640 1 --2- 172.21.15.19:0/194277957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb068004800 0x7fb068004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.229+0000 7fb081400640 1 --2- 172.21.15.19:0/194277957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb068005210 0x7fb06800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.229+0000 7fb081400640 1 -- 172.21.15.19:0/194277957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb08415c610 con 0x7fb068004800 2024-03-28T13:20:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.229+0000 7fb081400640 1 -- 172.21.15.19:0/194277957 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb06800c160 con 0x7fb068003a00 2024-03-28T13:20:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.229+0000 7fb081400640 1 -- 172.21.15.19:0/194277957 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb06800c360 con 0x7fb068005210 2024-03-28T13:20:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.230+0000 7fb081e00640 1 --2- 172.21.15.19:0/194277957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb068004800 0x7fb068004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.230+0000 7fb082800640 1 --2- 172.21.15.19:0/194277957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb068003a00 0x7fb068003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.230+0000 7fb083200640 1 --2- 172.21.15.19:0/194277957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb068005210 0x7fb06800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.230+0000 7fb081e00640 1 -- 172.21.15.19:0/194277957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb068005210 msgr2=0x7fb06800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:57.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.230+0000 7fb081e00640 1 --2- 172.21.15.19:0/194277957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb068005210 0x7fb06800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.230+0000 7fb081e00640 1 -- 172.21.15.19:0/194277957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb068003a00 msgr2=0x7fb068003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:57.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.230+0000 7fb081e00640 1 --2- 172.21.15.19:0/194277957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb068003a00 0x7fb068003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.230+0000 7fb081e00640 1 -- 172.21.15.19:0/194277957 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb068003520 con 0x7fb068004800 2024-03-28T13:20:57.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.230+0000 7fb082800640 1 --2- 172.21.15.19:0/194277957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb068003a00 0x7fb068003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:57.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.230+0000 7fb083200640 1 -- 172.21.15.19:0/194277957 reap_dead start 2024-03-28T13:20:57.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.231+0000 7fb081e00640 1 --2- 172.21.15.19:0/194277957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb068004800 0x7fb068004c80 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fb070001720 tx=0x7fb068005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c489a2673eabf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:57.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.231+0000 7fb081400640 1 -- 172.21.15.19:0/194277957 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070001370 con 0x7fb068004800 2024-03-28T13:20:57.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.231+0000 7fb081400640 1 -- 172.21.15.19:0/194277957 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0700052d0 con 0x7fb068004800 2024-03-28T13:20:57.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.231+0000 7fb081400640 1 -- 172.21.15.19:0/194277957 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb070003830 con 0x7fb068004800 2024-03-28T13:20:57.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.232+0000 7fb088a00640 1 -- 172.21.15.19:0/194277957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb068004800 msgr2=0x7fb068004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:57.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.232+0000 7fb088a00640 1 --2- 172.21.15.19:0/194277957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb068004800 0x7fb068004c80 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fb070001720 tx=0x7fb068005830 comp rx=0 tx=0).stop 2024-03-28T13:20:57.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.232+0000 7fb088a00640 1 -- 172.21.15.19:0/194277957 shutdown_connections 2024-03-28T13:20:57.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.232+0000 7fb088a00640 1 --2- 172.21.15.19:0/194277957 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb068004800 0x7fb068004c80 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.232+0000 7fb088a00640 1 -- 172.21.15.19:0/194277957 >> 172.21.15.19:0/194277957 conn(0x7fb0840ccad0 msgr2=0x7fb0840cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:57.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.232+0000 7fb088a00640 1 -- 172.21.15.19:0/194277957 shutdown_connections 2024-03-28T13:20:57.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.232+0000 7fb088a00640 1 -- 172.21.15.19:0/194277957 wait complete. 2024-03-28T13:20:57.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.234+0000 7fb088a00640 1 Processor -- start 2024-03-28T13:20:57.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.234+0000 7fb088a00640 1 -- start start 2024-03-28T13:20:57.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.235+0000 7fb088a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb068004800 0x7fb0841356f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:57.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.235+0000 7fb088a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb08413e030 0x7fb084139fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:57.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.235+0000 7fb088a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb084152340 0x7fb0841527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:57.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.235+0000 7fb088a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0840cc490 con 0x7fb084152340 2024-03-28T13:20:57.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.235+0000 7fb088a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb084160980 con 0x7fb08413e030 2024-03-28T13:20:57.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.235+0000 7fb088a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb0840cc610 con 0x7fb068004800 2024-03-28T13:20:57.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.235+0000 7fb083200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb084152340 0x7fb0841527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:57.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.235+0000 7fb083200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb084152340 0x7fb0841527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41864/0 (socket says 172.21.15.19:41864) 2024-03-28T13:20:57.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.235+0000 7fb083200640 1 -- 172.21.15.19:0/3829322346 learned_addr learned my addr 172.21.15.19:0/3829322346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:57.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.236+0000 7fb082800640 1 --2- 172.21.15.19:0/3829322346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb068004800 0x7fb0841356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:57.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.236+0000 7fb081e00640 1 --2- 172.21.15.19:0/3829322346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb08413e030 0x7fb084139fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.236+0000 7fb083200640 1 -- 172.21.15.19:0/3829322346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb068004800 msgr2=0x7fb0841356f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.236+0000 7fb083200640 1 --2- 172.21.15.19:0/3829322346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb068004800 0x7fb0841356f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.236+0000 7fb083200640 1 -- 172.21.15.19:0/3829322346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb08413e030 msgr2=0x7fb084139fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.236+0000 7fb083200640 1 --2- 172.21.15.19:0/3829322346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb08413e030 0x7fb084139fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.236+0000 7fb083200640 1 -- 172.21.15.19:0/3829322346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb084152d00 con 0x7fb084152340 2024-03-28T13:20:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.236+0000 7fb083200640 1 --2- 172.21.15.19:0/3829322346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb084152340 0x7fb0841527c0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fb0840b2c60 tx=0x7fb07c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.236+0000 7fb07aa00640 1 -- 172.21.15.19:0/3829322346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c016620 con 0x7fb084152340 2024-03-28T13:20:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.237+0000 7fb07aa00640 1 -- 172.21.15.19:0/3829322346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07c01b070 con 0x7fb084152340 2024-03-28T13:20:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.237+0000 7fb07aa00640 1 -- 172.21.15.19:0/3829322346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c016a20 con 0x7fb084152340 2024-03-28T13:20:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.237+0000 7fb088a00640 1 -- 172.21.15.19:0/3829322346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08413e580 con 0x7fb084152340 2024-03-28T13:20:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.237+0000 7fb088a00640 1 -- 172.21.15.19:0/3829322346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08413eb40 con 0x7fb084152340 2024-03-28T13:20:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.238+0000 7fb088a00640 1 -- 172.21.15.19:0/3829322346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb084075940 con 0x7fb084152340 2024-03-28T13:20:57.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.239+0000 7fb07aa00640 1 -- 172.21.15.19:0/3829322346 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb07c016bc0 con 0x7fb084152340 2024-03-28T13:20:57.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.240+0000 7fb07aa00640 1 --2- 172.21.15.19:0/3829322346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb06406c530 0x7fb06406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:57.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.240+0000 7fb07aa00640 1 -- 172.21.15.19:0/3829322346 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7fb07c08c370 con 0x7fb084152340 2024-03-28T13:20:57.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.240+0000 7fb082800640 1 --2- 172.21.15.19:0/3829322346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb06406c530 0x7fb06406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:57.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.242+0000 7fb082800640 1 --2- 172.21.15.19:0/3829322346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb06406c530 0x7fb06406e9f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb074001eb0 tx=0x7fb074009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:57.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.245+0000 7fb07aa00640 1 -- 172.21.15.19:0/3829322346 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb07c024020 con 0x7fb084152340 2024-03-28T13:20:57.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.456+0000 7fb088a00640 1 -- 172.21.15.19:0/3829322346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb084073830 con 0x7fb084152340 2024-03-28T13:20:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.459+0000 7fb07aa00640 1 -- 172.21.15.19:0/3829322346 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16683 (secure 0 0 0) 0x7fb07c05a990 con 0x7fb084152340 2024-03-28T13:20:57.459 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:57.459 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":119,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:20:54.450982+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:20:50.748212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-28T13:20:51.888444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:20:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.461+0000 7fb088a00640 1 -- 172.21.15.19:0/3829322346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb06406c530 msgr2=0x7fb06406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.461+0000 7fb088a00640 1 --2- 172.21.15.19:0/3829322346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb06406c530 0x7fb06406e9f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb074001eb0 tx=0x7fb074009040 comp rx=0 tx=0).stop 2024-03-28T13:20:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.461+0000 7fb088a00640 1 -- 172.21.15.19:0/3829322346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb084152340 msgr2=0x7fb0841527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.461+0000 7fb088a00640 1 --2- 172.21.15.19:0/3829322346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb084152340 0x7fb0841527c0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fb0840b2c60 tx=0x7fb07c00f520 comp rx=0 tx=0).stop 2024-03-28T13:20:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.462+0000 7fb088a00640 1 -- 172.21.15.19:0/3829322346 shutdown_connections 2024-03-28T13:20:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.462+0000 7fb088a00640 1 --2- 172.21.15.19:0/3829322346 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb06406c530 0x7fb06406e9f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.462+0000 7fb088a00640 1 --2- 172.21.15.19:0/3829322346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb084152340 0x7fb0841527c0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.462+0000 7fb088a00640 1 --2- 172.21.15.19:0/3829322346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb08413e030 0x7fb084139fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.462+0000 7fb088a00640 1 --2- 172.21.15.19:0/3829322346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb068004800 0x7fb0841356f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.462+0000 7fb088a00640 1 -- 172.21.15.19:0/3829322346 >> 172.21.15.19:0/3829322346 conn(0x7fb0840ccad0 msgr2=0x7fb084072670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.462+0000 7fb088a00640 1 -- 172.21.15.19:0/3829322346 shutdown_connections 2024-03-28T13:20:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.462+0000 7fb088a00640 1 -- 172.21.15.19:0/3829322346 wait complete. 2024-03-28T13:20:57.482 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-28T13:20:57.482 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:20:57.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.700+0000 7f8b3cc00640 1 Processor -- start 2024-03-28T13:20:57.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.701+0000 7f8b3cc00640 1 -- start start 2024-03-28T13:20:57.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.702+0000 7f8b3cc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8b3815e4a0 0x7f8b380b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:57.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.702+0000 7f8b3cc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b380cf700 0x7f8b380b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:57.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.702+0000 7f8b3cc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b3815e880 con 0x7f8b380d2310 2024-03-28T13:20:57.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.702+0000 7f8b3cc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f8b380d34e0 con 0x7f8b3815e4a0 2024-03-28T13:20:57.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.702+0000 7f8b3cc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8b380d28d0 con 0x7f8b380cf700 2024-03-28T13:20:57.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.703+0000 7f8b37e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b380cf700 0x7f8b380b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:57.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.703+0000 7f8b37400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8b3815e4a0 0x7f8b380b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:57.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.703+0000 7f8b37400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8b3815e4a0 0x7f8b380b9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:54184/0 (socket says 172.21.15.19:54184) 2024-03-28T13:20:57.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.703+0000 7f8b37e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b380cf700 0x7f8b380b73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:56812/0 (socket says 172.21.15.19:56812) 2024-03-28T13:20:57.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.703+0000 7f8b37400640 1 -- 172.21.15.19:0/1696357170 learned_addr learned my addr 172.21.15.19:0/1696357170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:57.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.704+0000 7f8b37400640 1 -- 172.21.15.19:0/1696357170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b380cf700 msgr2=0x7f8b380b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:57.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.704+0000 7f8b37400640 1 --2- 172.21.15.19:0/1696357170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b380cf700 0x7f8b380b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.704+0000 7f8b37400640 1 -- 172.21.15.19:0/1696357170 >> v1:172.21.15.188:6789/0 conn(0x7f8b380d2310 legacy=0x7f8b380ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:57.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.704+0000 7f8b37400640 1 -- 172.21.15.19:0/1696357170 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b380d2bb0 con 0x7f8b3815e4a0 2024-03-28T13:20:57.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.705+0000 7f8b37400640 1 --2- 172.21.15.19:0/1696357170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b3815e4a0 0x7f8b380b9c60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8b380d3660 tx=0x7f8b2802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1727ec071f52286d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:57.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.706+0000 7f8b36000640 1 -- 172.21.15.19:0/1696357170 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28035020 con 0x7f8b3815e4a0 2024-03-28T13:20:57.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.706+0000 7f8b36000640 1 -- 172.21.15.19:0/1696357170 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b28031640 con 0x7f8b3815e4a0 2024-03-28T13:20:57.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.706+0000 7f8b36000640 1 -- 172.21.15.19:0/1696357170 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2802d040 con 0x7f8b3815e4a0 2024-03-28T13:20:57.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.706+0000 7f8b3cc00640 1 -- 172.21.15.19:0/1696357170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b3815e4a0 msgr2=0x7f8b380b9c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:57.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.706+0000 7f8b3cc00640 1 --2- 172.21.15.19:0/1696357170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b3815e4a0 0x7f8b380b9c60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8b380d3660 tx=0x7f8b2802b8f0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.706+0000 7f8b3cc00640 1 -- 172.21.15.19:0/1696357170 shutdown_connections 2024-03-28T13:20:57.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.706+0000 7f8b3cc00640 1 --2- 172.21.15.19:0/1696357170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b380cf700 0x7f8b380b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.706+0000 7f8b3cc00640 1 --2- 172.21.15.19:0/1696357170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b3815e4a0 0x7f8b380b9c60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.706+0000 7f8b3cc00640 1 -- 172.21.15.19:0/1696357170 >> 172.21.15.19:0/1696357170 conn(0x7f8b380676d0 msgr2=0x7f8b38067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:57.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.707+0000 7f8b3cc00640 1 -- 172.21.15.19:0/1696357170 shutdown_connections 2024-03-28T13:20:57.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.707+0000 7f8b3cc00640 1 -- 172.21.15.19:0/1696357170 wait complete. 2024-03-28T13:20:57.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.708+0000 7f8b3cc00640 1 Processor -- start 2024-03-28T13:20:57.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.709+0000 7f8b3cc00640 1 -- start start 2024-03-28T13:20:57.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.709+0000 7f8b3cc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b380cf700 0x7f8b381f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:57.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.709+0000 7f8b3cc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b380d2310 0x7f8b380ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:57.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.709+0000 7f8b3cc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3815e4a0 0x7f8b380cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:57.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.709+0000 7f8b3cc00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8b380d3970 con 0x7f8b3815e4a0 2024-03-28T13:20:57.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.709+0000 7f8b3cc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8b380d3170 con 0x7f8b380d2310 2024-03-28T13:20:57.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.709+0000 7f8b3cc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8b380ba1a0 con 0x7f8b380cf700 2024-03-28T13:20:57.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.710+0000 7f8b37e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3815e4a0 0x7f8b380cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:57.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.710+0000 7f8b37e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3815e4a0 0x7f8b380cd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41878/0 (socket says 172.21.15.19:41878) 2024-03-28T13:20:57.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.710+0000 7f8b37e00640 1 -- 172.21.15.19:0/3712760401 learned_addr learned my addr 172.21.15.19:0/3712760401 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:57.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.710+0000 7f8b37400640 1 --2- 172.21.15.19:0/3712760401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b380cf700 0x7f8b381f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:57.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.710+0000 7f8b36a00640 1 --2- 172.21.15.19:0/3712760401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b380d2310 0x7f8b380ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:57.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.710+0000 7f8b37e00640 1 -- 172.21.15.19:0/3712760401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b380cf700 msgr2=0x7f8b381f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:57.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.710+0000 7f8b37e00640 1 --2- 172.21.15.19:0/3712760401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b380cf700 0x7f8b381f3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.710+0000 7f8b37e00640 1 -- 172.21.15.19:0/3712760401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b380d2310 msgr2=0x7f8b380ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:57.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.710+0000 7f8b37e00640 1 --2- 172.21.15.19:0/3712760401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b380d2310 0x7f8b380ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.710+0000 7f8b37e00640 1 -- 172.21.15.19:0/3712760401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b380cc090 con 0x7f8b3815e4a0 2024-03-28T13:20:57.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.710+0000 7f8b36a00640 1 --2- 172.21.15.19:0/3712760401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b380d2310 0x7f8b380ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:57.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.711+0000 7f8b37e00640 1 --2- 172.21.15.19:0/3712760401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3815e4a0 0x7f8b380cd1e0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000e7f0 tx=0x7f8b3000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:57.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.711+0000 7f8b2f400640 1 -- 172.21.15.19:0/3712760401 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b30017070 con 0x7f8b3815e4a0 2024-03-28T13:20:57.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.711+0000 7f8b2f400640 1 -- 172.21.15.19:0/3712760401 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b3000c730 con 0x7f8b3815e4a0 2024-03-28T13:20:57.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.711+0000 7f8b2f400640 1 -- 172.21.15.19:0/3712760401 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3000cb20 con 0x7f8b3815e4a0 2024-03-28T13:20:57.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.711+0000 7f8b3cc00640 1 -- 172.21.15.19:0/3712760401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b380cc370 con 0x7f8b3815e4a0 2024-03-28T13:20:57.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.711+0000 7f8b3cc00640 1 -- 172.21.15.19:0/3712760401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b381f4330 con 0x7f8b3815e4a0 2024-03-28T13:20:57.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.712+0000 7f8b3cc00640 1 -- 172.21.15.19:0/3712760401 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b38075940 con 0x7f8b3815e4a0 2024-03-28T13:20:57.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.713+0000 7f8b2f400640 1 -- 172.21.15.19:0/3712760401 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8b3001a070 con 0x7f8b3815e4a0 2024-03-28T13:20:57.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.714+0000 7f8b2f400640 1 --2- 172.21.15.19:0/3712760401 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8b1406c530 0x7f8b1406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:57.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.714+0000 7f8b2f400640 1 -- 172.21.15.19:0/3712760401 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f8b3008b900 con 0x7f8b3815e4a0 2024-03-28T13:20:57.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.715+0000 7f8b37400640 1 --2- 172.21.15.19:0/3712760401 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8b1406c530 0x7f8b1406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:57.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.716+0000 7f8b37400640 1 --2- 172.21.15.19:0/3712760401 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8b1406c530 0x7f8b1406e9f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8b28036000 tx=0x7f8b2803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:57.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.719+0000 7f8b2f400640 1 -- 172.21.15.19:0/3712760401 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8b30059f10 con 0x7f8b3815e4a0 2024-03-28T13:20:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.918+0000 7f8b3cc00640 1 -- 172.21.15.19:0/3712760401 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b3815ce40 con 0x7f8b1406c530 2024-03-28T13:20:57.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.938+0000 7f8b2f400640 1 -- 172.21.15.19:0/3712760401 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251769 (secure 0 0 0) 0x7f8b3815ce40 con 0x7f8b1406c530 2024-03-28T13:20:57.943 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:20:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.945+0000 7f8b3cc00640 1 -- 172.21.15.19:0/3712760401 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8b1406c530 msgr2=0x7f8b1406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.945+0000 7f8b3cc00640 1 --2- 172.21.15.19:0/3712760401 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8b1406c530 0x7f8b1406e9f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8b28036000 tx=0x7f8b2803c040 comp rx=0 tx=0).stop 2024-03-28T13:20:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.945+0000 7f8b3cc00640 1 -- 172.21.15.19:0/3712760401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3815e4a0 msgr2=0x7f8b380cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.945+0000 7f8b3cc00640 1 --2- 172.21.15.19:0/3712760401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3815e4a0 0x7f8b380cd1e0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000e7f0 tx=0x7f8b3000ecb0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.945+0000 7f8b3cc00640 1 -- 172.21.15.19:0/3712760401 shutdown_connections 2024-03-28T13:20:57.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.945+0000 7f8b3cc00640 1 --2- 172.21.15.19:0/3712760401 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8b1406c530 0x7f8b1406e9f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.945+0000 7f8b3cc00640 1 --2- 172.21.15.19:0/3712760401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8b3815e4a0 0x7f8b380cd1e0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.945+0000 7f8b3cc00640 1 --2- 172.21.15.19:0/3712760401 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8b380d2310 0x7f8b380ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.945+0000 7f8b3cc00640 1 --2- 172.21.15.19:0/3712760401 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8b380cf700 0x7f8b381f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.945+0000 7f8b3cc00640 1 -- 172.21.15.19:0/3712760401 >> 172.21.15.19:0/3712760401 conn(0x7f8b380676d0 msgr2=0x7f8b3815b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.945+0000 7f8b3cc00640 1 -- 172.21.15.19:0/3712760401 shutdown_connections 2024-03-28T13:20:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:57.945+0000 7f8b3cc00640 1 -- 172.21.15.19:0/3712760401 wait complete. 2024-03-28T13:20:57.964 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-03-28T13:20:56.242327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592654,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":193,"num_write_kb":2422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261996,"kb_used_data":12536,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712724,"statfs":{"total":773094113280,"available":772825829376,"internally_reserved":0,"allocated":12836864,"data_stored":6232119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2286,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.840539"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.475027+0000","last_change":"2024-03-28T13:20:53.465038+0000","last_active":"2024-03-28T13:20:53.475027+0000","last_peered":"2024-03-28T13:20:53.475027+0000","last_clean":"2024-03-28T13:20:53.475027+0000","last_became_active":"2024-03-28T13:20:53.464734+0000","last_became_peered":"2024-03-28T13:20:53.464734+0000","last_unstale":"2024-03-28T13:20:53.475027+0000","last_undegraded":"2024-03-28T13:20:53.475027+0000","last_fullsized":"2024-03-28T13:20:53.475027+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:40.372683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.810183+0000","last_change":"2024-03-28T13:20:53.469422+0000","last_active":"2024-03-28T13:20:54.810183+0000","last_peered":"2024-03-28T13:20:54.810183+0000","last_clean":"2024-03-28T13:20:54.810183+0000","last_became_active":"2024-03-28T13:20:53.469142+0000","last_became_peered":"2024-03-28T13:20:53.469142+0000","last_unstale":"2024-03-28T13:20:54.810183+0000","last_undegraded":"2024-03-28T13:20:54.810183+0000","last_fullsized":"2024-03-28T13:20:54.810183+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:12:39.048657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807894+0000","last_change":"2024-03-28T13:20:53.463975+0000","last_active":"2024-03-28T13:20:54.807894+0000","last_peered":"2024-03-28T13:20:54.807894+0000","last_clean":"2024-03-28T13:20:54.807894+0000","last_became_active":"2024-03-28T13:20:53.463465+0000","last_became_peered":"2024-03-28T13:20:53.463465+0000","last_unstale":"2024-03-28T13:20:54.807894+0000","last_undegraded":"2024-03-28T13:20:54.807894+0000","last_fullsized":"2024-03-28T13:20:54.807894+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:08:00.681004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.476032+0000","last_change":"2024-03-28T13:20:53.471211+0000","last_active":"2024-03-28T13:20:53.476032+0000","last_peered":"2024-03-28T13:20:53.476032+0000","last_clean":"2024-03-28T13:20:53.476032+0000","last_became_active":"2024-03-28T13:20:53.468553+0000","last_became_peered":"2024-03-28T13:20:53.468553+0000","last_unstale":"2024-03-28T13:20:53.476032+0000","last_undegraded":"2024-03-28T13:20:53.476032+0000","last_fullsized":"2024-03-28T13:20:53.476032+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:47.786867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-03-28T13:20:52.433608+0000","last_change":"2024-03-28T13:20:52.433608+0000","last_active":"2024-03-28T13:20:52.396515+0000","last_peered":"2024-03-28T13:20:52.396515+0000","last_clean":"2024-03-28T13:20:52.396515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:20:52.433608+0000","last_undegraded":"2024-03-28T13:20:52.433608+0000","last_fullsized":"2024-03-28T13:20:52.433608+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.478879+0000","last_change":"2024-03-28T13:20:53.476132+0000","last_active":"2024-03-28T13:20:53.478879+0000","last_peered":"2024-03-28T13:20:53.478879+0000","last_clean":"2024-03-28T13:20:53.478879+0000","last_became_active":"2024-03-28T13:20:53.475536+0000","last_became_peered":"2024-03-28T13:20:53.475536+0000","last_unstale":"2024-03-28T13:20:53.478879+0000","last_undegraded":"2024-03-28T13:20:53.478879+0000","last_fullsized":"2024-03-28T13:20:53.478879+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:12.511095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.467759+0000","last_change":"2024-03-28T13:20:53.470890+0000","last_active":"2024-03-28T13:20:54.467759+0000","last_peered":"2024-03-28T13:20:54.467759+0000","last_clean":"2024-03-28T13:20:54.467759+0000","last_became_active":"2024-03-28T13:20:53.468470+0000","last_became_peered":"2024-03-28T13:20:53.468470+0000","last_unstale":"2024-03-28T13:20:54.467759+0000","last_undegraded":"2024-03-28T13:20:54.467759+0000","last_fullsized":"2024-03-28T13:20:54.467759+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:30:41.860382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468041+0000","last_change":"2024-03-28T13:20:53.459597+0000","last_active":"2024-03-28T13:20:54.468041+0000","last_peered":"2024-03-28T13:20:54.468041+0000","last_clean":"2024-03-28T13:20:54.468041+0000","last_became_active":"2024-03-28T13:20:53.459080+0000","last_became_peered":"2024-03-28T13:20:53.459080+0000","last_unstale":"2024-03-28T13:20:54.468041+0000","last_undegraded":"2024-03-28T13:20:54.468041+0000","last_fullsized":"2024-03-28T13:20:54.468041+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:48:59.909295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.475605+0000","last_change":"2024-03-28T13:20:53.470994+0000","last_active":"2024-03-28T13:20:53.475605+0000","last_peered":"2024-03-28T13:20:53.475605+0000","last_clean":"2024-03-28T13:20:53.475605+0000","last_became_active":"2024-03-28T13:20:53.470474+0000","last_became_peered":"2024-03-28T13:20:53.470474+0000","last_unstale":"2024-03-28T13:20:53.475605+0000","last_undegraded":"2024-03-28T13:20:53.475605+0000","last_fullsized":"2024-03-28T13:20:53.475605+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:22:55.029632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-03-28T13:20:52.434819+0000","last_change":"2024-03-28T13:20:52.434819+0000","last_active":"2024-03-28T13:20:52.396515+0000","last_peered":"2024-03-28T13:20:52.396515+0000","last_clean":"2024-03-28T13:20:52.396515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:20:52.434819+0000","last_undegraded":"2024-03-28T13:20:52.434819+0000","last_fullsized":"2024-03-28T13:20:52.434819+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468597+0000","last_change":"2024-03-28T13:20:53.467986+0000","last_active":"2024-03-28T13:20:54.468597+0000","last_peered":"2024-03-28T13:20:54.468597+0000","last_clean":"2024-03-28T13:20:54.468597+0000","last_became_active":"2024-03-28T13:20:53.467599+0000","last_became_peered":"2024-03-28T13:20:53.467599+0000","last_unstale":"2024-03-28T13:20:54.468597+0000","last_undegraded":"2024-03-28T13:20:54.468597+0000","last_fullsized":"2024-03-28T13:20:54.468597+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:48:15.038578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-03-28T13:20:52.433919+0000","last_change":"2024-03-28T13:20:52.433919+0000","last_active":"2024-03-28T13:20:52.396515+0000","last_peered":"2024-03-28T13:20:52.396515+0000","last_clean":"2024-03-28T13:20:52.396515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:20:52.433919+0000","last_undegraded":"2024-03-28T13:20:52.433919+0000","last_fullsized":"2024-03-28T13:20:52.433919+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.809406+0000","last_change":"2024-03-28T13:20:53.469025+0000","last_active":"2024-03-28T13:20:54.809406+0000","last_peered":"2024-03-28T13:20:54.809406+0000","last_clean":"2024-03-28T13:20:54.809406+0000","last_became_active":"2024-03-28T13:20:53.465798+0000","last_became_peered":"2024-03-28T13:20:53.465798+0000","last_unstale":"2024-03-28T13:20:54.809406+0000","last_undegraded":"2024-03-28T13:20:54.809406+0000","last_fullsized":"2024-03-28T13:20:54.809406+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:35:09.245847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.469711+0000","last_change":"2024-03-28T13:20:53.468381+0000","last_active":"2024-03-28T13:20:54.469711+0000","last_peered":"2024-03-28T13:20:54.469711+0000","last_clean":"2024-03-28T13:20:54.469711+0000","last_became_active":"2024-03-28T13:20:53.465568+0000","last_became_peered":"2024-03-28T13:20:53.465568+0000","last_unstale":"2024-03-28T13:20:54.469711+0000","last_undegraded":"2024-03-28T13:20:54.469711+0000","last_fullsized":"2024-03-28T13:20:54.469711+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:12:39.048657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.471822+0000","last_change":"2024-03-28T13:20:53.462502+0000","last_active":"2024-03-28T13:20:53.471822+0000","last_peered":"2024-03-28T13:20:53.471822+0000","last_clean":"2024-03-28T13:20:53.471822+0000","last_became_active":"2024-03-28T13:20:53.462152+0000","last_became_peered":"2024-03-28T13:20:53.462152+0000","last_unstale":"2024-03-28T13:20:53.471822+0000","last_undegraded":"2024-03-28T13:20:53.471822+0000","last_fullsized":"2024-03-28T13:20:53.471822+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:06:26.204869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-03-28T13:20:52.433502+0000","last_change":"2024-03-28T13:20:52.433502+0000","last_active":"2024-03-28T13:20:52.396515+0000","last_peered":"2024-03-28T13:20:52.396515+0000","last_clean":"2024-03-28T13:20:52.396515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:20:52.433502+0000","last_undegraded":"2024-03-28T13:20:52.433502+0000","last_fullsized":"2024-03-28T13:20:52.433502+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.808490+0000","last_change":"2024-03-28T13:20:53.470232+0000","last_active":"2024-03-28T13:20:54.808490+0000","last_peered":"2024-03-28T13:20:54.808490+0000","last_clean":"2024-03-28T13:20:54.808490+0000","last_became_active":"2024-03-28T13:20:53.469915+0000","last_became_peered":"2024-03-28T13:20:53.469915+0000","last_unstale":"2024-03-28T13:20:54.808490+0000","last_undegraded":"2024-03-28T13:20:54.808490+0000","last_fullsized":"2024-03-28T13:20:54.808490+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:05:02.392740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468212+0000","last_change":"2024-03-28T13:20:53.457481+0000","last_active":"2024-03-28T13:20:54.468212+0000","last_peered":"2024-03-28T13:20:54.468212+0000","last_clean":"2024-03-28T13:20:54.468212+0000","last_became_active":"2024-03-28T13:20:53.456906+0000","last_became_peered":"2024-03-28T13:20:53.456906+0000","last_unstale":"2024-03-28T13:20:54.468212+0000","last_undegraded":"2024-03-28T13:20:54.468212+0000","last_fullsized":"2024-03-28T13:20:54.468212+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:21:03.576502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.808218+0000","last_change":"2024-03-28T13:20:53.461472+0000","last_active":"2024-03-28T13:20:54.808218+0000","last_peered":"2024-03-28T13:20:54.808218+0000","last_clean":"2024-03-28T13:20:54.808218+0000","last_became_active":"2024-03-28T13:20:53.460857+0000","last_became_peered":"2024-03-28T13:20:53.460857+0000","last_unstale":"2024-03-28T13:20:54.808218+0000","last_undegraded":"2024-03-28T13:20:54.808218+0000","last_fullsized":"2024-03-28T13:20:54.808218+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:01:08.844704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807539+0000","last_change":"2024-03-28T13:20:53.472177+0000","last_active":"2024-03-28T13:20:54.807539+0000","last_peered":"2024-03-28T13:20:54.807539+0000","last_clean":"2024-03-28T13:20:54.807539+0000","last_became_active":"2024-03-28T13:20:53.471845+0000","last_became_peered":"2024-03-28T13:20:53.471845+0000","last_unstale":"2024-03-28T13:20:54.807539+0000","last_undegraded":"2024-03-28T13:20:54.807539+0000","last_fullsized":"2024-03-28T13:20:54.807539+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:41:40.224718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.469217+0000","last_change":"2024-03-28T13:20:53.473564+0000","last_active":"2024-03-28T13:20:54.469217+0000","last_peered":"2024-03-28T13:20:54.469217+0000","last_clean":"2024-03-28T13:20:54.469217+0000","last_became_active":"2024-03-28T13:20:53.472829+0000","last_became_peered":"2024-03-28T13:20:53.472829+0000","last_unstale":"2024-03-28T13:20:54.469217+0000","last_undegraded":"2024-03-28T13:20:54.469217+0000","last_fullsized":"2024-03-28T13:20:54.469217+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:43:57.592462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.464268+0000","last_change":"2024-03-28T13:20:53.462618+0000","last_active":"2024-03-28T13:20:53.464268+0000","last_peered":"2024-03-28T13:20:53.464268+0000","last_clean":"2024-03-28T13:20:53.464268+0000","last_became_active":"2024-03-28T13:20:53.462180+0000","last_became_peered":"2024-03-28T13:20:53.462180+0000","last_unstale":"2024-03-28T13:20:53.464268+0000","last_undegraded":"2024-03-28T13:20:53.464268+0000","last_fullsized":"2024-03-28T13:20:53.464268+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:49:12.296314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.469935+0000","last_change":"2024-03-28T13:20:53.469955+0000","last_active":"2024-03-28T13:20:54.469935+0000","last_peered":"2024-03-28T13:20:54.469935+0000","last_clean":"2024-03-28T13:20:54.469935+0000","last_became_active":"2024-03-28T13:20:53.469435+0000","last_became_peered":"2024-03-28T13:20:53.469435+0000","last_unstale":"2024-03-28T13:20:54.469935+0000","last_undegraded":"2024-03-28T13:20:54.469935+0000","last_fullsized":"2024-03-28T13:20:54.469935+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:47.786867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.474760+0000","last_change":"2024-03-28T13:20:53.469068+0000","last_active":"2024-03-28T13:20:53.474760+0000","last_peered":"2024-03-28T13:20:53.474760+0000","last_clean":"2024-03-28T13:20:53.474760+0000","last_became_active":"2024-03-28T13:20:53.467105+0000","last_became_peered":"2024-03-28T13:20:53.467105+0000","last_unstale":"2024-03-28T13:20:53.474760+0000","last_undegraded":"2024-03-28T13:20:53.474760+0000","last_fullsized":"2024-03-28T13:20:53.474760+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:12:39.048657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468484+0000","last_change":"2024-03-28T13:20:53.473290+0000","last_active":"2024-03-28T13:20:54.468484+0000","last_peered":"2024-03-28T13:20:54.468484+0000","last_clean":"2024-03-28T13:20:54.468484+0000","last_became_active":"2024-03-28T13:20:53.472228+0000","last_became_peered":"2024-03-28T13:20:53.472228+0000","last_unstale":"2024-03-28T13:20:54.468484+0000","last_undegraded":"2024-03-28T13:20:54.468484+0000","last_fullsized":"2024-03-28T13:20:54.468484+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:18:41.661696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.464008+0000","last_change":"2024-03-28T13:20:53.459988+0000","last_active":"2024-03-28T13:20:53.464008+0000","last_peered":"2024-03-28T13:20:53.464008+0000","last_clean":"2024-03-28T13:20:53.464008+0000","last_became_active":"2024-03-28T13:20:53.459664+0000","last_became_peered":"2024-03-28T13:20:53.459664+0000","last_unstale":"2024-03-28T13:20:53.464008+0000","last_undegraded":"2024-03-28T13:20:53.464008+0000","last_fullsized":"2024-03-28T13:20:53.464008+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:32.637893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.482349+0000","last_change":"2024-03-28T13:20:53.477331+0000","last_active":"2024-03-28T13:20:53.482349+0000","last_peered":"2024-03-28T13:20:53.482349+0000","last_clean":"2024-03-28T13:20:53.482349+0000","last_became_active":"2024-03-28T13:20:53.476374+0000","last_became_peered":"2024-03-28T13:20:53.476374+0000","last_unstale":"2024-03-28T13:20:53.482349+0000","last_undegraded":"2024-03-28T13:20:53.482349+0000","last_fullsized":"2024-03-28T13:20:53.482349+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:07:25.965347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.480519+0000","last_change":"2024-03-28T13:20:53.477904+0000","last_active":"2024-03-28T13:20:53.480519+0000","last_peered":"2024-03-28T13:20:53.480519+0000","last_clean":"2024-03-28T13:20:53.480519+0000","last_became_active":"2024-03-28T13:20:53.475255+0000","last_became_peered":"2024-03-28T13:20:53.475255+0000","last_unstale":"2024-03-28T13:20:53.480519+0000","last_undegraded":"2024-03-28T13:20:53.480519+0000","last_fullsized":"2024-03-28T13:20:53.480519+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:23:28.084957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.471227+0000","last_change":"2024-03-28T13:20:53.466345+0000","last_active":"2024-03-28T13:20:54.471227+0000","last_peered":"2024-03-28T13:20:54.471227+0000","last_clean":"2024-03-28T13:20:54.471227+0000","last_became_active":"2024-03-28T13:20:53.466004+0000","last_became_peered":"2024-03-28T13:20:53.466004+0000","last_unstale":"2024-03-28T13:20:54.471227+0000","last_undegraded":"2024-03-28T13:20:54.471227+0000","last_fullsized":"2024-03-28T13:20:54.471227+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:12.329663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468392+0000","last_change":"2024-03-28T13:20:53.459392+0000","last_active":"2024-03-28T13:20:54.468392+0000","last_peered":"2024-03-28T13:20:54.468392+0000","last_clean":"2024-03-28T13:20:54.468392+0000","last_became_active":"2024-03-28T13:20:53.459073+0000","last_became_peered":"2024-03-28T13:20:53.459073+0000","last_unstale":"2024-03-28T13:20:54.468392+0000","last_undegraded":"2024-03-28T13:20:54.468392+0000","last_fullsized":"2024-03-28T13:20:54.468392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:41.108125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.474697+0000","last_change":"2024-03-28T13:20:53.465014+0000","last_active":"2024-03-28T13:20:53.474697+0000","last_peered":"2024-03-28T13:20:53.474697+0000","last_clean":"2024-03-28T13:20:53.474697+0000","last_became_active":"2024-03-28T13:20:53.464721+0000","last_became_peered":"2024-03-28T13:20:53.464721+0000","last_unstale":"2024-03-28T13:20:53.474697+0000","last_undegraded":"2024-03-28T13:20:53.474697+0000","last_fullsized":"2024-03-28T13:20:53.474697+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:32:38.627989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-03-28T13:20:52.432291+0000","last_change":"2024-03-28T13:20:52.432291+0000","last_active":"2024-03-28T13:20:52.396515+0000","last_peered":"2024-03-28T13:20:52.396515+0000","last_clean":"2024-03-28T13:20:52.396515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:20:52.432291+0000","last_undegraded":"2024-03-28T13:20:52.432291+0000","last_fullsized":"2024-03-28T13:20:52.432291+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.475539+0000","last_change":"2024-03-28T13:20:53.473122+0000","last_active":"2024-03-28T13:20:53.475539+0000","last_peered":"2024-03-28T13:20:53.475539+0000","last_clean":"2024-03-28T13:20:53.475539+0000","last_became_active":"2024-03-28T13:20:53.470214+0000","last_became_peered":"2024-03-28T13:20:53.470214+0000","last_unstale":"2024-03-28T13:20:53.475539+0000","last_undegraded":"2024-03-28T13:20:53.475539+0000","last_fullsized":"2024-03-28T13:20:53.475539+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:27:15.147525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"118'4","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.471014+0000","last_change":"2024-03-28T13:20:52.426502+0000","last_active":"2024-03-28T13:20:54.471014+0000","last_peered":"2024-03-28T13:20:54.471014+0000","last_clean":"2024-03-28T13:20:54.471014+0000","last_became_active":"2024-03-28T13:20:52.426181+0000","last_became_peered":"2024-03-28T13:20:52.426181+0000","last_unstale":"2024-03-28T13:20:54.471014+0000","last_undegraded":"2024-03-28T13:20:54.471014+0000","last_fullsized":"2024-03-28T13:20:54.471014+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:18:08.312130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.470889+0000","last_change":"2024-03-28T13:20:53.468637+0000","last_active":"2024-03-28T13:20:54.470889+0000","last_peered":"2024-03-28T13:20:54.470889+0000","last_clean":"2024-03-28T13:20:54.470889+0000","last_became_active":"2024-03-28T13:20:53.468093+0000","last_became_peered":"2024-03-28T13:20:53.468093+0000","last_unstale":"2024-03-28T13:20:54.470889+0000","last_undegraded":"2024-03-28T13:20:54.470889+0000","last_fullsized":"2024-03-28T13:20:54.470889+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:35.591833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.809918+0000","last_change":"2024-03-28T13:20:52.426075+0000","last_active":"2024-03-28T13:20:54.809918+0000","last_peered":"2024-03-28T13:20:54.809918+0000","last_clean":"2024-03-28T13:20:54.809918+0000","last_became_active":"2024-03-28T13:20:52.425617+0000","last_became_peered":"2024-03-28T13:20:52.425617+0000","last_unstale":"2024-03-28T13:20:54.809918+0000","last_undegraded":"2024-03-28T13:20:54.809918+0000","last_fullsized":"2024-03-28T13:20:54.809918+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:26.395440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.462200+0000","last_change":"2024-03-28T13:20:53.454767+0000","last_active":"2024-03-28T13:20:53.462200+0000","last_peered":"2024-03-28T13:20:53.462200+0000","last_clean":"2024-03-28T13:20:53.462200+0000","last_became_active":"2024-03-28T13:20:53.454359+0000","last_became_peered":"2024-03-28T13:20:53.454359+0000","last_unstale":"2024-03-28T13:20:53.462200+0000","last_undegraded":"2024-03-28T13:20:53.462200+0000","last_fullsized":"2024-03-28T13:20:53.462200+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:39:02.869522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.488169+0000","last_change":"2024-03-28T13:20:52.442602+0000","last_active":"2024-03-28T13:20:53.488169+0000","last_peered":"2024-03-28T13:20:53.488169+0000","last_clean":"2024-03-28T13:20:53.488169+0000","last_became_active":"2024-03-28T13:20:52.442305+0000","last_became_peered":"2024-03-28T13:20:52.442305+0000","last_unstale":"2024-03-28T13:20:53.488169+0000","last_undegraded":"2024-03-28T13:20:53.488169+0000","last_fullsized":"2024-03-28T13:20:53.488169+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:07:59.665654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.808347+0000","last_change":"2024-03-28T13:20:53.463740+0000","last_active":"2024-03-28T13:20:54.808347+0000","last_peered":"2024-03-28T13:20:54.808347+0000","last_clean":"2024-03-28T13:20:54.808347+0000","last_became_active":"2024-03-28T13:20:53.463001+0000","last_became_peered":"2024-03-28T13:20:53.463001+0000","last_unstale":"2024-03-28T13:20:54.808347+0000","last_undegraded":"2024-03-28T13:20:54.808347+0000","last_fullsized":"2024-03-28T13:20:54.808347+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:00:41.753031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.488371+0000","last_change":"2024-03-28T13:20:52.444849+0000","last_active":"2024-03-28T13:20:53.488371+0000","last_peered":"2024-03-28T13:20:53.488371+0000","last_clean":"2024-03-28T13:20:53.488371+0000","last_became_active":"2024-03-28T13:20:52.444552+0000","last_became_peered":"2024-03-28T13:20:52.444552+0000","last_unstale":"2024-03-28T13:20:53.488371+0000","last_undegraded":"2024-03-28T13:20:53.488371+0000","last_fullsized":"2024-03-28T13:20:53.488371+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:05:01.377390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.473246+0000","last_change":"2024-03-28T13:20:53.466028+0000","last_active":"2024-03-28T13:20:53.473246+0000","last_peered":"2024-03-28T13:20:53.473246+0000","last_clean":"2024-03-28T13:20:53.473246+0000","last_became_active":"2024-03-28T13:20:53.465466+0000","last_became_peered":"2024-03-28T13:20:53.465466+0000","last_unstale":"2024-03-28T13:20:53.473246+0000","last_undegraded":"2024-03-28T13:20:53.473246+0000","last_fullsized":"2024-03-28T13:20:53.473246+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:25:31.976436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807563+0000","last_change":"2024-03-28T13:20:52.429598+0000","last_active":"2024-03-28T13:20:54.807563+0000","last_peered":"2024-03-28T13:20:54.807563+0000","last_clean":"2024-03-28T13:20:54.807563+0000","last_became_active":"2024-03-28T13:20:52.429132+0000","last_became_peered":"2024-03-28T13:20:52.429132+0000","last_unstale":"2024-03-28T13:20:54.807563+0000","last_undegraded":"2024-03-28T13:20:54.807563+0000","last_fullsized":"2024-03-28T13:20:54.807563+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:33:05.548959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468919+0000","last_change":"2024-03-28T13:20:53.461800+0000","last_active":"2024-03-28T13:20:54.468919+0000","last_peered":"2024-03-28T13:20:54.468919+0000","last_clean":"2024-03-28T13:20:54.468919+0000","last_became_active":"2024-03-28T13:20:53.458431+0000","last_became_peered":"2024-03-28T13:20:53.458431+0000","last_unstale":"2024-03-28T13:20:54.468919+0000","last_undegraded":"2024-03-28T13:20:54.468919+0000","last_fullsized":"2024-03-28T13:20:54.468919+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:30:03.655476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.487821+0000","last_change":"2024-03-28T13:20:52.438748+0000","last_active":"2024-03-28T13:20:53.487821+0000","last_peered":"2024-03-28T13:20:53.487821+0000","last_clean":"2024-03-28T13:20:53.487821+0000","last_became_active":"2024-03-28T13:20:52.436838+0000","last_became_peered":"2024-03-28T13:20:52.436838+0000","last_unstale":"2024-03-28T13:20:53.487821+0000","last_undegraded":"2024-03-28T13:20:53.487821+0000","last_fullsized":"2024-03-28T13:20:53.487821+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:07:24.949997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-03-28T13:20:52.425772+0000","last_change":"2024-03-28T13:20:52.425772+0000","last_active":"2024-03-28T13:20:52.396515+0000","last_peered":"2024-03-28T13:20:52.396515+0000","last_clean":"2024-03-28T13:20:52.396515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:20:52.425772+0000","last_undegraded":"2024-03-28T13:20:52.425772+0000","last_fullsized":"2024-03-28T13:20:52.425772+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468098+0000","last_change":"2024-03-28T13:20:52.442360+0000","last_active":"2024-03-28T13:20:54.468098+0000","last_peered":"2024-03-28T13:20:54.468098+0000","last_clean":"2024-03-28T13:20:54.468098+0000","last_became_active":"2024-03-28T13:20:52.441911+0000","last_became_peered":"2024-03-28T13:20:52.441911+0000","last_unstale":"2024-03-28T13:20:54.468098+0000","last_undegraded":"2024-03-28T13:20:54.468098+0000","last_fullsized":"2024-03-28T13:20:54.468098+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:12:57.608427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.467957+0000","last_change":"2024-03-28T13:20:53.461088+0000","last_active":"2024-03-28T13:20:54.467957+0000","last_peered":"2024-03-28T13:20:54.467957+0000","last_clean":"2024-03-28T13:20:54.467957+0000","last_became_active":"2024-03-28T13:20:53.460345+0000","last_became_peered":"2024-03-28T13:20:53.460345+0000","last_unstale":"2024-03-28T13:20:54.467957+0000","last_undegraded":"2024-03-28T13:20:54.467957+0000","last_fullsized":"2024-03-28T13:20:54.467957+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:43:57.040909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-03-28T13:20:52.450172+0000","last_change":"2024-03-28T13:20:52.448170+0000","last_active":"2024-03-28T13:20:52.450172+0000","last_peered":"2024-03-28T13:20:52.450172+0000","last_clean":"2024-03-28T13:20:52.450172+0000","last_became_active":"2024-03-28T13:20:52.447274+0000","last_became_peered":"2024-03-28T13:20:52.447274+0000","last_unstale":"2024-03-28T13:20:52.450172+0000","last_undegraded":"2024-03-28T13:20:52.450172+0000","last_fullsized":"2024-03-28T13:20:52.450172+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:09:34.403613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.809650+0000","last_change":"2024-03-28T13:20:53.476237+0000","last_active":"2024-03-28T13:20:54.809650+0000","last_peered":"2024-03-28T13:20:54.809650+0000","last_clean":"2024-03-28T13:20:54.809650+0000","last_became_active":"2024-03-28T13:20:53.475843+0000","last_became_peered":"2024-03-28T13:20:53.475843+0000","last_unstale":"2024-03-28T13:20:54.809650+0000","last_undegraded":"2024-03-28T13:20:54.809650+0000","last_fullsized":"2024-03-28T13:20:54.809650+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:16:26.602338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.464498+0000","last_change":"2024-03-28T13:20:52.441359+0000","last_active":"2024-03-28T13:20:53.464498+0000","last_peered":"2024-03-28T13:20:53.464498+0000","last_clean":"2024-03-28T13:20:53.464498+0000","last_became_active":"2024-03-28T13:20:52.440551+0000","last_became_peered":"2024-03-28T13:20:52.440551+0000","last_unstale":"2024-03-28T13:20:53.464498+0000","last_undegraded":"2024-03-28T13:20:53.464498+0000","last_fullsized":"2024-03-28T13:20:53.464498+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:01:07.829354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.810064+0000","last_change":"2024-03-28T13:20:53.468240+0000","last_active":"2024-03-28T13:20:54.810064+0000","last_peered":"2024-03-28T13:20:54.810064+0000","last_clean":"2024-03-28T13:20:54.810064+0000","last_became_active":"2024-03-28T13:20:53.466214+0000","last_became_peered":"2024-03-28T13:20:53.466214+0000","last_unstale":"2024-03-28T13:20:54.810064+0000","last_undegraded":"2024-03-28T13:20:54.810064+0000","last_fullsized":"2024-03-28T13:20:54.810064+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:43:37.800350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-03-28T13:20:52.450207+0000","last_change":"2024-03-28T13:20:52.448230+0000","last_active":"2024-03-28T13:20:52.450207+0000","last_peered":"2024-03-28T13:20:52.450207+0000","last_clean":"2024-03-28T13:20:52.450207+0000","last_became_active":"2024-03-28T13:20:52.447633+0000","last_became_peered":"2024-03-28T13:20:52.447633+0000","last_unstale":"2024-03-28T13:20:52.450207+0000","last_undegraded":"2024-03-28T13:20:52.450207+0000","last_fullsized":"2024-03-28T13:20:52.450207+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:11:16.170107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.478017+0000","last_change":"2024-03-28T13:20:53.472592+0000","last_active":"2024-03-28T13:20:53.478017+0000","last_peered":"2024-03-28T13:20:53.478017+0000","last_clean":"2024-03-28T13:20:53.478017+0000","last_became_active":"2024-03-28T13:20:53.469863+0000","last_became_peered":"2024-03-28T13:20:53.469863+0000","last_unstale":"2024-03-28T13:20:53.478017+0000","last_undegraded":"2024-03-28T13:20:53.478017+0000","last_fullsized":"2024-03-28T13:20:53.478017+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:36:19.800467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-03-28T13:20:52.450257+0000","last_change":"2024-03-28T13:20:52.448990+0000","last_active":"2024-03-28T13:20:52.450257+0000","last_peered":"2024-03-28T13:20:52.450257+0000","last_clean":"2024-03-28T13:20:52.450257+0000","last_became_active":"2024-03-28T13:20:52.448329+0000","last_became_peered":"2024-03-28T13:20:52.448329+0000","last_unstale":"2024-03-28T13:20:52.450257+0000","last_undegraded":"2024-03-28T13:20:52.450257+0000","last_fullsized":"2024-03-28T13:20:52.450257+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:13:23.957315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468775+0000","last_change":"2024-03-28T13:20:53.462231+0000","last_active":"2024-03-28T13:20:54.468775+0000","last_peered":"2024-03-28T13:20:54.468775+0000","last_clean":"2024-03-28T13:20:54.468775+0000","last_became_active":"2024-03-28T13:20:53.461889+0000","last_became_peered":"2024-03-28T13:20:53.461889+0000","last_unstale":"2024-03-28T13:20:54.468775+0000","last_undegraded":"2024-03-28T13:20:54.468775+0000","last_fullsized":"2024-03-28T13:20:54.468775+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:35.648328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"118'6","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.492037+0000","last_change":"2024-03-28T13:20:52.444751+0000","last_active":"2024-03-28T13:20:53.492037+0000","last_peered":"2024-03-28T13:20:53.492037+0000","last_clean":"2024-03-28T13:20:53.492037+0000","last_became_active":"2024-03-28T13:20:52.444439+0000","last_became_peered":"2024-03-28T13:20:52.444439+0000","last_unstale":"2024-03-28T13:20:53.492037+0000","last_undegraded":"2024-03-28T13:20:53.492037+0000","last_fullsized":"2024-03-28T13:20:53.492037+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:35:08.230497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-03-28T13:20:52.432491+0000","last_change":"2024-03-28T13:20:52.432491+0000","last_active":"2024-03-28T13:20:52.396515+0000","last_peered":"2024-03-28T13:20:52.396515+0000","last_clean":"2024-03-28T13:20:52.396515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:20:52.432491+0000","last_undegraded":"2024-03-28T13:20:52.432491+0000","last_fullsized":"2024-03-28T13:20:52.432491+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-03-28T13:20:52.450212+0000","last_change":"2024-03-28T13:20:52.448319+0000","last_active":"2024-03-28T13:20:52.450212+0000","last_peered":"2024-03-28T13:20:52.450212+0000","last_clean":"2024-03-28T13:20:52.450212+0000","last_became_active":"2024-03-28T13:20:52.447526+0000","last_became_peered":"2024-03-28T13:20:52.447526+0000","last_unstale":"2024-03-28T13:20:52.450212+0000","last_undegraded":"2024-03-28T13:20:52.450212+0000","last_fullsized":"2024-03-28T13:20:52.450212+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:34:54.206123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.483290+0000","last_change":"2024-03-28T13:20:53.479412+0000","last_active":"2024-03-28T13:20:53.483290+0000","last_peered":"2024-03-28T13:20:53.483290+0000","last_clean":"2024-03-28T13:20:53.483290+0000","last_became_active":"2024-03-28T13:20:53.479127+0000","last_became_peered":"2024-03-28T13:20:53.479127+0000","last_unstale":"2024-03-28T13:20:53.483290+0000","last_undegraded":"2024-03-28T13:20:53.483290+0000","last_fullsized":"2024-03-28T13:20:53.483290+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:11.084472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807986+0000","last_change":"2024-03-28T13:20:52.438117+0000","last_active":"2024-03-28T13:20:54.807986+0000","last_peered":"2024-03-28T13:20:54.807986+0000","last_clean":"2024-03-28T13:20:54.807986+0000","last_became_active":"2024-03-28T13:20:52.437864+0000","last_became_peered":"2024-03-28T13:20:52.437864+0000","last_unstale":"2024-03-28T13:20:54.807986+0000","last_undegraded":"2024-03-28T13:20:54.807986+0000","last_fullsized":"2024-03-28T13:20:54.807986+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:42:17.679510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.476303+0000","last_change":"2024-03-28T13:20:53.468630+0000","last_active":"2024-03-28T13:20:53.476303+0000","last_peered":"2024-03-28T13:20:53.476303+0000","last_clean":"2024-03-28T13:20:53.476303+0000","last_became_active":"2024-03-28T13:20:53.468281+0000","last_became_peered":"2024-03-28T13:20:53.468281+0000","last_unstale":"2024-03-28T13:20:53.476303+0000","last_undegraded":"2024-03-28T13:20:53.476303+0000","last_fullsized":"2024-03-28T13:20:53.476303+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:43:37.800350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"118'8","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.493619+0000","last_change":"2024-03-28T13:20:52.433015+0000","last_active":"2024-03-28T13:20:53.493619+0000","last_peered":"2024-03-28T13:20:53.493619+0000","last_clean":"2024-03-28T13:20:53.493619+0000","last_became_active":"2024-03-28T13:20:52.432759+0000","last_became_peered":"2024-03-28T13:20:52.432759+0000","last_unstale":"2024-03-28T13:20:53.493619+0000","last_undegraded":"2024-03-28T13:20:53.493619+0000","last_fullsized":"2024-03-28T13:20:53.493619+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:46.771516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.460780+0000","last_change":"2024-03-28T13:20:53.449284+0000","last_active":"2024-03-28T13:20:53.460780+0000","last_peered":"2024-03-28T13:20:53.460780+0000","last_clean":"2024-03-28T13:20:53.460780+0000","last_became_active":"2024-03-28T13:20:53.448572+0000","last_became_peered":"2024-03-28T13:20:53.448572+0000","last_unstale":"2024-03-28T13:20:53.460780+0000","last_undegraded":"2024-03-28T13:20:53.460780+0000","last_fullsized":"2024-03-28T13:20:53.460780+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:35.591833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.484172+0000","last_change":"2024-03-28T13:20:52.430184+0000","last_active":"2024-03-28T13:20:53.484172+0000","last_peered":"2024-03-28T13:20:53.484172+0000","last_clean":"2024-03-28T13:20:53.484172+0000","last_became_active":"2024-03-28T13:20:52.429698+0000","last_became_peered":"2024-03-28T13:20:52.429698+0000","last_unstale":"2024-03-28T13:20:53.484172+0000","last_undegraded":"2024-03-28T13:20:53.484172+0000","last_fullsized":"2024-03-28T13:20:53.484172+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:41.127141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.475585+0000","last_change":"2024-03-28T13:20:53.470762+0000","last_active":"2024-03-28T13:20:53.475585+0000","last_peered":"2024-03-28T13:20:53.475585+0000","last_clean":"2024-03-28T13:20:53.475585+0000","last_became_active":"2024-03-28T13:20:53.470040+0000","last_became_peered":"2024-03-28T13:20:53.470040+0000","last_unstale":"2024-03-28T13:20:53.475585+0000","last_undegraded":"2024-03-28T13:20:53.475585+0000","last_fullsized":"2024-03-28T13:20:53.475585+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:01:21.238792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-03-28T13:20:52.450242+0000","last_change":"2024-03-28T13:20:52.448629+0000","last_active":"2024-03-28T13:20:52.450242+0000","last_peered":"2024-03-28T13:20:52.450242+0000","last_clean":"2024-03-28T13:20:52.450242+0000","last_became_active":"2024-03-28T13:20:52.447976+0000","last_became_peered":"2024-03-28T13:20:52.447976+0000","last_unstale":"2024-03-28T13:20:52.450242+0000","last_undegraded":"2024-03-28T13:20:52.450242+0000","last_fullsized":"2024-03-28T13:20:52.450242+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:09:29.705457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468959+0000","last_change":"2024-03-28T13:20:53.457848+0000","last_active":"2024-03-28T13:20:54.468959+0000","last_peered":"2024-03-28T13:20:54.468959+0000","last_clean":"2024-03-28T13:20:54.468959+0000","last_became_active":"2024-03-28T13:20:53.457538+0000","last_became_peered":"2024-03-28T13:20:53.457538+0000","last_unstale":"2024-03-28T13:20:54.468959+0000","last_undegraded":"2024-03-28T13:20:54.468959+0000","last_fullsized":"2024-03-28T13:20:54.468959+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:43:46.400454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.471042+0000","last_change":"2024-03-28T13:20:52.435350+0000","last_active":"2024-03-28T13:20:54.471042+0000","last_peered":"2024-03-28T13:20:54.471042+0000","last_clean":"2024-03-28T13:20:54.471042+0000","last_became_active":"2024-03-28T13:20:52.435002+0000","last_became_peered":"2024-03-28T13:20:52.435002+0000","last_unstale":"2024-03-28T13:20:54.471042+0000","last_undegraded":"2024-03-28T13:20:54.471042+0000","last_fullsized":"2024-03-28T13:20:54.471042+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:56:08.871552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.470848+0000","last_change":"2024-03-28T13:20:53.465022+0000","last_active":"2024-03-28T13:20:54.470848+0000","last_peered":"2024-03-28T13:20:54.470848+0000","last_clean":"2024-03-28T13:20:54.470848+0000","last_became_active":"2024-03-28T13:20:53.464673+0000","last_became_peered":"2024-03-28T13:20:53.464673+0000","last_unstale":"2024-03-28T13:20:54.470848+0000","last_undegraded":"2024-03-28T13:20:54.470848+0000","last_fullsized":"2024-03-28T13:20:54.470848+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:56:03.874557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.470414+0000","last_change":"2024-03-28T13:20:52.439533+0000","last_active":"2024-03-28T13:20:54.470414+0000","last_peered":"2024-03-28T13:20:54.470414+0000","last_clean":"2024-03-28T13:20:54.470414+0000","last_became_active":"2024-03-28T13:20:52.439196+0000","last_became_peered":"2024-03-28T13:20:52.439196+0000","last_unstale":"2024-03-28T13:20:54.470414+0000","last_undegraded":"2024-03-28T13:20:54.470414+0000","last_fullsized":"2024-03-28T13:20:54.470414+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:36:59.221033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807641+0000","last_change":"2024-03-28T13:20:53.462615+0000","last_active":"2024-03-28T13:20:54.807641+0000","last_peered":"2024-03-28T13:20:54.807641+0000","last_clean":"2024-03-28T13:20:54.807641+0000","last_became_active":"2024-03-28T13:20:53.461933+0000","last_became_peered":"2024-03-28T13:20:53.461933+0000","last_unstale":"2024-03-28T13:20:54.807641+0000","last_undegraded":"2024-03-28T13:20:54.807641+0000","last_fullsized":"2024-03-28T13:20:54.807641+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:05:52.611316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-03-28T13:20:52.449620+0000","last_change":"2024-03-28T13:20:52.447056+0000","last_active":"2024-03-28T13:20:52.449620+0000","last_peered":"2024-03-28T13:20:52.449620+0000","last_clean":"2024-03-28T13:20:52.449620+0000","last_became_active":"2024-03-28T13:20:52.446652+0000","last_became_peered":"2024-03-28T13:20:52.446652+0000","last_unstale":"2024-03-28T13:20:52.449620+0000","last_undegraded":"2024-03-28T13:20:52.449620+0000","last_fullsized":"2024-03-28T13:20:52.449620+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:22:15.035005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.471532+0000","last_change":"2024-03-28T13:20:53.458081+0000","last_active":"2024-03-28T13:20:53.471532+0000","last_peered":"2024-03-28T13:20:53.471532+0000","last_clean":"2024-03-28T13:20:53.471532+0000","last_became_active":"2024-03-28T13:20:53.457743+0000","last_became_peered":"2024-03-28T13:20:53.457743+0000","last_unstale":"2024-03-28T13:20:53.471532+0000","last_undegraded":"2024-03-28T13:20:53.471532+0000","last_fullsized":"2024-03-28T13:20:53.471532+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:12.329663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.437791+0000","last_change":"2024-03-28T13:20:52.442543+0000","last_active":"2024-03-28T13:20:53.437791+0000","last_peered":"2024-03-28T13:20:53.437791+0000","last_clean":"2024-03-28T13:20:53.437791+0000","last_became_active":"2024-03-28T13:20:52.440638+0000","last_became_peered":"2024-03-28T13:20:52.440638+0000","last_unstale":"2024-03-28T13:20:53.437791+0000","last_undegraded":"2024-03-28T13:20:53.437791+0000","last_fullsized":"2024-03-28T13:20:53.437791+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:56:02.859206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.469578+0000","last_change":"2024-03-28T13:20:53.468819+0000","last_active":"2024-03-28T13:20:54.469578+0000","last_peered":"2024-03-28T13:20:54.469578+0000","last_clean":"2024-03-28T13:20:54.469578+0000","last_became_active":"2024-03-28T13:20:53.468111+0000","last_became_peered":"2024-03-28T13:20:53.468111+0000","last_unstale":"2024-03-28T13:20:54.469578+0000","last_undegraded":"2024-03-28T13:20:54.469578+0000","last_fullsized":"2024-03-28T13:20:54.469578+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:16:26.602338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.463015+0000","last_change":"2024-03-28T13:20:52.444756+0000","last_active":"2024-03-28T13:20:53.463015+0000","last_peered":"2024-03-28T13:20:53.463015+0000","last_clean":"2024-03-28T13:20:53.463015+0000","last_became_active":"2024-03-28T13:20:52.444280+0000","last_became_peered":"2024-03-28T13:20:52.444280+0000","last_unstale":"2024-03-28T13:20:53.463015+0000","last_undegraded":"2024-03-28T13:20:53.463015+0000","last_fullsized":"2024-03-28T13:20:53.463015+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:41:39.209368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":1,"reported_epoch":117,"state":"creating+peering","last_fresh":"2024-03-28T13:20:52.432554+0000","last_change":"2024-03-28T13:20:52.432554+0000","last_active":"2024-03-28T13:20:52.396515+0000","last_peered":"2024-03-28T13:20:52.396515+0000","last_clean":"2024-03-28T13:20:52.396515+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-28T13:20:52.432554+0000","last_undegraded":"2024-03-28T13:20:52.432554+0000","last_fullsized":"2024-03-28T13:20:52.432554+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.463412+0000","last_change":"2024-03-28T13:20:52.438937+0000","last_active":"2024-03-28T13:20:53.463412+0000","last_peered":"2024-03-28T13:20:53.463412+0000","last_clean":"2024-03-28T13:20:53.463412+0000","last_became_active":"2024-03-28T13:20:52.437894+0000","last_became_peered":"2024-03-28T13:20:52.437894+0000","last_unstale":"2024-03-28T13:20:53.463412+0000","last_undegraded":"2024-03-28T13:20:53.463412+0000","last_fullsized":"2024-03-28T13:20:53.463412+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:05:51.595965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.470684+0000","last_change":"2024-03-28T13:20:53.467384+0000","last_active":"2024-03-28T13:20:54.470684+0000","last_peered":"2024-03-28T13:20:54.470684+0000","last_clean":"2024-03-28T13:20:54.470684+0000","last_became_active":"2024-03-28T13:20:53.467098+0000","last_became_peered":"2024-03-28T13:20:53.467098+0000","last_unstale":"2024-03-28T13:20:54.470684+0000","last_undegraded":"2024-03-28T13:20:54.470684+0000","last_fullsized":"2024-03-28T13:20:54.470684+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:07:25.965347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.488125+0000","last_change":"2024-03-28T13:20:52.440581+0000","last_active":"2024-03-28T13:20:53.488125+0000","last_peered":"2024-03-28T13:20:53.488125+0000","last_clean":"2024-03-28T13:20:53.488125+0000","last_became_active":"2024-03-28T13:20:52.440279+0000","last_became_peered":"2024-03-28T13:20:52.440279+0000","last_unstale":"2024-03-28T13:20:53.488125+0000","last_undegraded":"2024-03-28T13:20:53.488125+0000","last_fullsized":"2024-03-28T13:20:53.488125+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:00:40.737680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.476022+0000","last_change":"2024-03-28T13:20:53.474638+0000","last_active":"2024-03-28T13:20:53.476022+0000","last_peered":"2024-03-28T13:20:53.476022+0000","last_clean":"2024-03-28T13:20:53.476022+0000","last_became_active":"2024-03-28T13:20:53.474304+0000","last_became_peered":"2024-03-28T13:20:53.474304+0000","last_unstale":"2024-03-28T13:20:53.476022+0000","last_undegraded":"2024-03-28T13:20:53.476022+0000","last_fullsized":"2024-03-28T13:20:53.476022+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:23:20.466097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"119'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.484389+0000","last_change":"2024-03-28T13:20:52.430955+0000","last_active":"2024-03-28T13:20:54.484389+0000","last_peered":"2024-03-28T13:20:54.484389+0000","last_clean":"2024-03-28T13:20:54.484389+0000","last_became_active":"2024-03-28T13:20:52.430582+0000","last_became_peered":"2024-03-28T13:20:52.430582+0000","last_unstale":"2024-03-28T13:20:54.484389+0000","last_undegraded":"2024-03-28T13:20:54.484389+0000","last_fullsized":"2024-03-28T13:20:54.484389+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:21:35.563041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.473489+0000","last_change":"2024-03-28T13:20:53.463792+0000","last_active":"2024-03-28T13:20:53.473489+0000","last_peered":"2024-03-28T13:20:53.473489+0000","last_clean":"2024-03-28T13:20:53.473489+0000","last_became_active":"2024-03-28T13:20:53.463451+0000","last_became_peered":"2024-03-28T13:20:53.463451+0000","last_unstale":"2024-03-28T13:20:53.473489+0000","last_undegraded":"2024-03-28T13:20:53.473489+0000","last_fullsized":"2024-03-28T13:20:53.473489+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:04:36.260367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.463943+0000","last_change":"2024-03-28T13:20:52.438608+0000","last_active":"2024-03-28T13:20:53.463943+0000","last_peered":"2024-03-28T13:20:53.463943+0000","last_clean":"2024-03-28T13:20:53.463943+0000","last_became_active":"2024-03-28T13:20:52.437785+0000","last_became_peered":"2024-03-28T13:20:52.437785+0000","last_unstale":"2024-03-28T13:20:53.463943+0000","last_undegraded":"2024-03-28T13:20:53.463943+0000","last_fullsized":"2024-03-28T13:20:53.463943+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:26.395440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.470704+0000","last_change":"2024-03-28T13:20:53.475497+0000","last_active":"2024-03-28T13:20:54.470704+0000","last_peered":"2024-03-28T13:20:54.470704+0000","last_clean":"2024-03-28T13:20:54.470704+0000","last_became_active":"2024-03-28T13:20:53.471389+0000","last_became_peered":"2024-03-28T13:20:53.471389+0000","last_unstale":"2024-03-28T13:20:54.470704+0000","last_undegraded":"2024-03-28T13:20:54.470704+0000","last_fullsized":"2024-03-28T13:20:54.470704+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:11.084472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.467282+0000","last_change":"2024-03-28T13:20:52.438113+0000","last_active":"2024-03-28T13:20:53.467282+0000","last_peered":"2024-03-28T13:20:53.467282+0000","last_clean":"2024-03-28T13:20:53.467282+0000","last_became_active":"2024-03-28T13:20:52.437814+0000","last_became_peered":"2024-03-28T13:20:52.437814+0000","last_unstale":"2024-03-28T13:20:53.467282+0000","last_undegraded":"2024-03-28T13:20:53.467282+0000","last_fullsized":"2024-03-28T13:20:53.467282+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:18:33.178252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.808096+0000","last_change":"2024-03-28T13:20:53.474595+0000","last_active":"2024-03-28T13:20:54.808096+0000","last_peered":"2024-03-28T13:20:54.808096+0000","last_clean":"2024-03-28T13:20:54.808096+0000","last_became_active":"2024-03-28T13:20:53.474117+0000","last_became_peered":"2024-03-28T13:20:53.474117+0000","last_unstale":"2024-03-28T13:20:54.808096+0000","last_undegraded":"2024-03-28T13:20:54.808096+0000","last_fullsized":"2024-03-28T13:20:54.808096+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:27.410791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.469334+0000","last_change":"2024-03-28T13:20:52.422254+0000","last_active":"2024-03-28T13:20:54.469334+0000","last_peered":"2024-03-28T13:20:54.469334+0000","last_clean":"2024-03-28T13:20:54.469334+0000","last_became_active":"2024-03-28T13:20:52.421865+0000","last_became_peered":"2024-03-28T13:20:52.421865+0000","last_unstale":"2024-03-28T13:20:54.469334+0000","last_undegraded":"2024-03-28T13:20:54.469334+0000","last_fullsized":"2024-03-28T13:20:54.469334+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:14.058698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.471509+0000","last_change":"2024-03-28T13:20:53.462490+0000","last_active":"2024-03-28T13:20:54.471509+0000","last_peered":"2024-03-28T13:20:54.471509+0000","last_clean":"2024-03-28T13:20:54.471509+0000","last_became_active":"2024-03-28T13:20:53.462053+0000","last_became_peered":"2024-03-28T13:20:53.462053+0000","last_unstale":"2024-03-28T13:20:54.471509+0000","last_undegraded":"2024-03-28T13:20:54.471509+0000","last_fullsized":"2024-03-28T13:20:54.471509+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:06:26.204869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.470094+0000","last_change":"2024-03-28T13:20:52.433980+0000","last_active":"2024-03-28T13:20:54.470094+0000","last_peered":"2024-03-28T13:20:54.470094+0000","last_clean":"2024-03-28T13:20:54.470094+0000","last_became_active":"2024-03-28T13:20:52.433677+0000","last_became_peered":"2024-03-28T13:20:52.433677+0000","last_unstale":"2024-03-28T13:20:54.470094+0000","last_undegraded":"2024-03-28T13:20:54.470094+0000","last_fullsized":"2024-03-28T13:20:54.470094+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:12:21.534987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.469571+0000","last_change":"2024-03-28T13:20:53.469829+0000","last_active":"2024-03-28T13:20:54.469571+0000","last_peered":"2024-03-28T13:20:54.469571+0000","last_clean":"2024-03-28T13:20:54.469571+0000","last_became_active":"2024-03-28T13:20:53.469522+0000","last_became_peered":"2024-03-28T13:20:53.469522+0000","last_unstale":"2024-03-28T13:20:54.469571+0000","last_undegraded":"2024-03-28T13:20:54.469571+0000","last_fullsized":"2024-03-28T13:20:54.469571+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:43:37.800350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.437363+0000","last_change":"2024-03-28T13:20:52.436068+0000","last_active":"2024-03-28T13:20:53.437363+0000","last_peered":"2024-03-28T13:20:53.437363+0000","last_clean":"2024-03-28T13:20:53.437363+0000","last_became_active":"2024-03-28T13:20:52.435482+0000","last_became_peered":"2024-03-28T13:20:52.435482+0000","last_unstale":"2024-03-28T13:20:53.437363+0000","last_undegraded":"2024-03-28T13:20:53.437363+0000","last_fullsized":"2024-03-28T13:20:53.437363+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:10.069121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.450743+0000","last_change":"2024-03-28T13:20:53.448750+0000","last_active":"2024-03-28T13:20:53.450743+0000","last_peered":"2024-03-28T13:20:53.450743+0000","last_clean":"2024-03-28T13:20:53.450743+0000","last_became_active":"2024-03-28T13:20:53.448091+0000","last_became_peered":"2024-03-28T13:20:53.448091+0000","last_unstale":"2024-03-28T13:20:53.450743+0000","last_undegraded":"2024-03-28T13:20:53.450743+0000","last_fullsized":"2024-03-28T13:20:53.450743+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:27:15.147525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.445206+0000","last_change":"2024-03-28T13:20:52.420196+0000","last_active":"2024-03-28T13:20:53.445206+0000","last_peered":"2024-03-28T13:20:53.445206+0000","last_clean":"2024-03-28T13:20:53.445206+0000","last_became_active":"2024-03-28T13:20:52.419420+0000","last_became_peered":"2024-03-28T13:20:52.419420+0000","last_unstale":"2024-03-28T13:20:53.445206+0000","last_undegraded":"2024-03-28T13:20:53.445206+0000","last_fullsized":"2024-03-28T13:20:53.445206+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:22:54.014282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.474107+0000","last_change":"2024-03-28T13:20:53.468311+0000","last_active":"2024-03-28T13:20:53.474107+0000","last_peered":"2024-03-28T13:20:53.474107+0000","last_clean":"2024-03-28T13:20:53.474107+0000","last_became_active":"2024-03-28T13:20:53.468000+0000","last_became_peered":"2024-03-28T13:20:53.468000+0000","last_unstale":"2024-03-28T13:20:53.474107+0000","last_undegraded":"2024-03-28T13:20:53.474107+0000","last_fullsized":"2024-03-28T13:20:53.474107+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:16:26.602338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.809073+0000","last_change":"2024-03-28T13:20:52.440290+0000","last_active":"2024-03-28T13:20:54.809073+0000","last_peered":"2024-03-28T13:20:54.809073+0000","last_clean":"2024-03-28T13:20:54.809073+0000","last_became_active":"2024-03-28T13:20:52.439943+0000","last_became_peered":"2024-03-28T13:20:52.439943+0000","last_unstale":"2024-03-28T13:20:54.809073+0000","last_undegraded":"2024-03-28T13:20:54.809073+0000","last_fullsized":"2024-03-28T13:20:54.809073+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:07:59.665654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":349,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807157+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:20:54.807157+0000","last_peered":"2024-03-28T13:20:54.807157+0000","last_clean":"2024-03-28T13:20:54.807157+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:20:54.807157+0000","last_undegraded":"2024-03-28T13:20:54.807157+0000","last_fullsized":"2024-03-28T13:20:54.807157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2286,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476879,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1454080,"data_stored":632097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476877,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1466368,"data_stored":632735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476877,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2048000,"data_stored":1222465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":632749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509580,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1454080,"data_stored":632097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509581,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2056192,"data_stored":1222557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509581,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1454080,"data_stored":632097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1441792,"data_stored":625322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:20:57.972 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:20:58.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.240+0000 7f492fe00640 1 Processor -- start 2024-03-28T13:20:58.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.240+0000 7f492fe00640 1 -- start start 2024-03-28T13:20:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.241+0000 7f492fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49301442c0 0x7f49300ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.241+0000 7f492fe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f49300d02a0 0x7f49300b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.241+0000 7f492fe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4930071460 con 0x7f49300cefe0 2024-03-28T13:20:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.241+0000 7f492fe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4930071d20 con 0x7f49301442c0 2024-03-28T13:20:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.241+0000 7f492fe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f49300cf420 con 0x7f49300d02a0 2024-03-28T13:20:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.242+0000 7f492ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49301442c0 0x7f49300ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.242+0000 7f492ea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f49301442c0 0x7f49300ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41880/0 (socket says 172.21.15.19:41880) 2024-03-28T13:20:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.242+0000 7f492ea00640 1 -- 172.21.15.19:0/2791439227 learned_addr learned my addr 172.21.15.19:0/2791439227 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.242+0000 7f492e000640 1 --1- 172.21.15.19:0/2791439227 >> v1:172.21.15.19:6789/0 conn(0x7f49300cefe0 0x7f49300cfb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53704/0 (socket says 172.21.15.19:53704) 2024-03-28T13:20:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.242+0000 7f492f400640 1 --2- 172.21.15.19:0/2791439227 >> v2:172.21.15.188:3300/0 conn(0x7f49300d02a0 0x7f49300b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.242+0000 7f492ea00640 1 -- 172.21.15.19:0/2791439227 >> v2:172.21.15.188:3300/0 conn(0x7f49300d02a0 msgr2=0x7f49300b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:58.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.242+0000 7f492ea00640 1 --2- 172.21.15.19:0/2791439227 >> v2:172.21.15.188:3300/0 conn(0x7f49300d02a0 0x7f49300b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.242+0000 7f492ea00640 1 -- 172.21.15.19:0/2791439227 >> v1:172.21.15.19:6789/0 conn(0x7f49300cefe0 legacy=0x7f49300cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.242+0000 7f492ea00640 1 -- 172.21.15.19:0/2791439227 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49300721b0 con 0x7f49301442c0 2024-03-28T13:20:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.243+0000 7f492ea00640 1 --2- 172.21.15.19:0/2791439227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49301442c0 0x7f49300ceaa0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f49300b3a70 tx=0x7f492002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bfd20387ecd339 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.244+0000 7f492d600640 1 -- 172.21.15.19:0/2791439227 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492002f850 con 0x7f49301442c0 2024-03-28T13:20:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.244+0000 7f492d600640 1 -- 172.21.15.19:0/2791439227 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f492002f9f0 con 0x7f49301442c0 2024-03-28T13:20:58.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.244+0000 7f492d600640 1 -- 172.21.15.19:0/2791439227 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492002fdc0 con 0x7f49301442c0 2024-03-28T13:20:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.244+0000 7f492fe00640 1 -- 172.21.15.19:0/2791439227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49301442c0 msgr2=0x7f49300ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.244+0000 7f492fe00640 1 --2- 172.21.15.19:0/2791439227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49301442c0 0x7f49300ceaa0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f49300b3a70 tx=0x7f492002ba30 comp rx=0 tx=0).stop 2024-03-28T13:20:58.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.245+0000 7f492fe00640 1 -- 172.21.15.19:0/2791439227 shutdown_connections 2024-03-28T13:20:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.245+0000 7f492fe00640 1 --2- 172.21.15.19:0/2791439227 >> v2:172.21.15.188:3300/0 conn(0x7f49300d02a0 0x7f49300b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.245+0000 7f492fe00640 1 --2- 172.21.15.19:0/2791439227 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49301442c0 0x7f49300ceaa0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.245+0000 7f492fe00640 1 -- 172.21.15.19:0/2791439227 >> 172.21.15.19:0/2791439227 conn(0x7f49300ccad0 msgr2=0x7f49300cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.245+0000 7f492fe00640 1 -- 172.21.15.19:0/2791439227 shutdown_connections 2024-03-28T13:20:58.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.245+0000 7f492fe00640 1 -- 172.21.15.19:0/2791439227 wait complete. 2024-03-28T13:20:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.246+0000 7f492fe00640 1 Processor -- start 2024-03-28T13:20:58.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.247+0000 7f492fe00640 1 -- start start 2024-03-28T13:20:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.247+0000 7f492fe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49300cefe0 0x7f4930154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:58.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.247+0000 7f492fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49300d02a0 0x7f4930156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.247+0000 7f492fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49301442c0 0x7f4930135310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.248+0000 7f492fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49300d0770 con 0x7f49300d02a0 2024-03-28T13:20:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.248+0000 7f492fe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f49300719b0 con 0x7f49300cefe0 2024-03-28T13:20:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.248+0000 7f492fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f49301446a0 con 0x7f49301442c0 2024-03-28T13:20:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.248+0000 7f492e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49300d02a0 0x7f4930156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.248+0000 7f492e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49300d02a0 0x7f4930156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41896/0 (socket says 172.21.15.19:41896) 2024-03-28T13:20:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.248+0000 7f492e000640 1 -- 172.21.15.19:0/2134553342 learned_addr learned my addr 172.21.15.19:0/2134553342 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.248+0000 7f492ea00640 1 --2- 172.21.15.19:0/2134553342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49300cefe0 0x7f4930154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.248+0000 7f492f400640 1 --2- 172.21.15.19:0/2134553342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49301442c0 0x7f4930135310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.248+0000 7f492e000640 1 -- 172.21.15.19:0/2134553342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49301442c0 msgr2=0x7f4930135310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.248+0000 7f492e000640 1 --2- 172.21.15.19:0/2134553342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49301442c0 0x7f4930135310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.248+0000 7f492e000640 1 -- 172.21.15.19:0/2134553342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49300cefe0 msgr2=0x7f4930154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:58.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.248+0000 7f492e000640 1 --2- 172.21.15.19:0/2134553342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49300cefe0 0x7f4930154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.248+0000 7f492e000640 1 -- 172.21.15.19:0/2134553342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4930134760 con 0x7f49300d02a0 2024-03-28T13:20:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.248+0000 7f492ea00640 1 --2- 172.21.15.19:0/2134553342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49300cefe0 0x7f4930154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:20:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.249+0000 7f492e000640 1 --2- 172.21.15.19:0/2134553342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49300d02a0 0x7f4930156870 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f491800c3c0 tx=0x7f491800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.249+0000 7f4927400640 1 -- 172.21.15.19:0/2134553342 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49180074e0 con 0x7f49300d02a0 2024-03-28T13:20:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.249+0000 7f4927400640 1 -- 172.21.15.19:0/2134553342 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4918007680 con 0x7f49300d02a0 2024-03-28T13:20:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.249+0000 7f4927400640 1 -- 172.21.15.19:0/2134553342 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918004a00 con 0x7f49300d02a0 2024-03-28T13:20:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.249+0000 7f492fe00640 1 -- 172.21.15.19:0/2134553342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4930134a40 con 0x7f49300d02a0 2024-03-28T13:20:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.249+0000 7f492fe00640 1 -- 172.21.15.19:0/2134553342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4930135000 con 0x7f49300d02a0 2024-03-28T13:20:58.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.250+0000 7f492fe00640 1 -- 172.21.15.19:0/2134553342 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4930075940 con 0x7f49300d02a0 2024-03-28T13:20:58.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.251+0000 7f4927400640 1 -- 172.21.15.19:0/2134553342 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f491801a020 con 0x7f49300d02a0 2024-03-28T13:20:58.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.252+0000 7f4927400640 1 --2- 172.21.15.19:0/2134553342 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f490c06c530 0x7f490c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:58.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.252+0000 7f4927400640 1 -- 172.21.15.19:0/2134553342 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f491808ae80 con 0x7f49300d02a0 2024-03-28T13:20:58.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.253+0000 7f492ea00640 1 --2- 172.21.15.19:0/2134553342 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f490c06c530 0x7f490c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:58.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.254+0000 7f492ea00640 1 --2- 172.21.15.19:0/2134553342 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f490c06c530 0x7f490c06e9f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4920036000 tx=0x7f4920039040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:58.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.257+0000 7f4927400640 1 -- 172.21.15.19:0/2134553342 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4918059490 con 0x7f49300d02a0 2024-03-28T13:20:58.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.457+0000 7f492fe00640 1 -- 172.21.15.19:0/2134553342 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4930155e40 con 0x7f490c06c530 2024-03-28T13:20:58.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.475+0000 7f4927400640 1 -- 172.21.15.19:0/2134553342 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252452 (secure 0 0 0) 0x7f4930155e40 con 0x7f490c06c530 2024-03-28T13:20:58.478 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:58.479 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:20:58.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.481+0000 7f492fe00640 1 -- 172.21.15.19:0/2134553342 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f490c06c530 msgr2=0x7f490c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:58.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.481+0000 7f492fe00640 1 --2- 172.21.15.19:0/2134553342 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f490c06c530 0x7f490c06e9f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4920036000 tx=0x7f4920039040 comp rx=0 tx=0).stop 2024-03-28T13:20:58.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.481+0000 7f492fe00640 1 -- 172.21.15.19:0/2134553342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49300d02a0 msgr2=0x7f4930156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:58.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.481+0000 7f492fe00640 1 --2- 172.21.15.19:0/2134553342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49300d02a0 0x7f4930156870 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f491800c3c0 tx=0x7f491800c880 comp rx=0 tx=0).stop 2024-03-28T13:20:58.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.481+0000 7f492fe00640 1 -- 172.21.15.19:0/2134553342 shutdown_connections 2024-03-28T13:20:58.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.481+0000 7f492fe00640 1 --2- 172.21.15.19:0/2134553342 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f490c06c530 0x7f490c06e9f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:58.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.481+0000 7f492fe00640 1 --2- 172.21.15.19:0/2134553342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49301442c0 0x7f4930135310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:58.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.481+0000 7f492fe00640 1 --2- 172.21.15.19:0/2134553342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f49300d02a0 0x7f4930156870 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:58.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.481+0000 7f492fe00640 1 --2- 172.21.15.19:0/2134553342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49300cefe0 0x7f4930154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:58.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.481+0000 7f492fe00640 1 -- 172.21.15.19:0/2134553342 >> 172.21.15.19:0/2134553342 conn(0x7f49300ccad0 msgr2=0x7f493006ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:58.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.481+0000 7f492fe00640 1 -- 172.21.15.19:0/2134553342 shutdown_connections 2024-03-28T13:20:58.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.481+0000 7f492fe00640 1 -- 172.21.15.19:0/2134553342 wait complete. 2024-03-28T13:20:58.502 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-03-28T13:20:58.243180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262048,"kb_used_data":12580,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712672,"statfs":{"total":773094113280,"available":772825776128,"internally_reserved":0,"allocated":12881920,"data_stored":6246930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002188"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.475027+0000","last_change":"2024-03-28T13:20:53.465038+0000","last_active":"2024-03-28T13:20:53.475027+0000","last_peered":"2024-03-28T13:20:53.475027+0000","last_clean":"2024-03-28T13:20:53.475027+0000","last_became_active":"2024-03-28T13:20:53.464734+0000","last_became_peered":"2024-03-28T13:20:53.464734+0000","last_unstale":"2024-03-28T13:20:53.475027+0000","last_undegraded":"2024-03-28T13:20:53.475027+0000","last_fullsized":"2024-03-28T13:20:53.475027+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:18:40.372683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.810183+0000","last_change":"2024-03-28T13:20:53.469422+0000","last_active":"2024-03-28T13:20:54.810183+0000","last_peered":"2024-03-28T13:20:54.810183+0000","last_clean":"2024-03-28T13:20:54.810183+0000","last_became_active":"2024-03-28T13:20:53.469142+0000","last_became_peered":"2024-03-28T13:20:53.469142+0000","last_unstale":"2024-03-28T13:20:54.810183+0000","last_undegraded":"2024-03-28T13:20:54.810183+0000","last_fullsized":"2024-03-28T13:20:54.810183+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:12:39.048657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807894+0000","last_change":"2024-03-28T13:20:53.463975+0000","last_active":"2024-03-28T13:20:54.807894+0000","last_peered":"2024-03-28T13:20:54.807894+0000","last_clean":"2024-03-28T13:20:54.807894+0000","last_became_active":"2024-03-28T13:20:53.463465+0000","last_became_peered":"2024-03-28T13:20:53.463465+0000","last_unstale":"2024-03-28T13:20:54.807894+0000","last_undegraded":"2024-03-28T13:20:54.807894+0000","last_fullsized":"2024-03-28T13:20:54.807894+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:08:00.681004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.476032+0000","last_change":"2024-03-28T13:20:53.471211+0000","last_active":"2024-03-28T13:20:53.476032+0000","last_peered":"2024-03-28T13:20:53.476032+0000","last_clean":"2024-03-28T13:20:53.476032+0000","last_became_active":"2024-03-28T13:20:53.468553+0000","last_became_peered":"2024-03-28T13:20:53.468553+0000","last_unstale":"2024-03-28T13:20:53.476032+0000","last_undegraded":"2024-03-28T13:20:53.476032+0000","last_fullsized":"2024-03-28T13:20:53.476032+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:47.786867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.806910+0000","last_change":"2024-03-28T13:20:53.465593+0000","last_active":"2024-03-28T13:20:54.806910+0000","last_peered":"2024-03-28T13:20:54.806910+0000","last_clean":"2024-03-28T13:20:54.806910+0000","last_became_active":"2024-03-28T13:20:53.465035+0000","last_became_peered":"2024-03-28T13:20:53.465035+0000","last_unstale":"2024-03-28T13:20:54.806910+0000","last_undegraded":"2024-03-28T13:20:54.806910+0000","last_fullsized":"2024-03-28T13:20:54.806910+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:40:10.341276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.478879+0000","last_change":"2024-03-28T13:20:53.476132+0000","last_active":"2024-03-28T13:20:53.478879+0000","last_peered":"2024-03-28T13:20:53.478879+0000","last_clean":"2024-03-28T13:20:53.478879+0000","last_became_active":"2024-03-28T13:20:53.475536+0000","last_became_peered":"2024-03-28T13:20:53.475536+0000","last_unstale":"2024-03-28T13:20:53.478879+0000","last_undegraded":"2024-03-28T13:20:53.478879+0000","last_fullsized":"2024-03-28T13:20:53.478879+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:34:12.511095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.467759+0000","last_change":"2024-03-28T13:20:53.470890+0000","last_active":"2024-03-28T13:20:54.467759+0000","last_peered":"2024-03-28T13:20:54.467759+0000","last_clean":"2024-03-28T13:20:54.467759+0000","last_became_active":"2024-03-28T13:20:53.468470+0000","last_became_peered":"2024-03-28T13:20:53.468470+0000","last_unstale":"2024-03-28T13:20:54.467759+0000","last_undegraded":"2024-03-28T13:20:54.467759+0000","last_fullsized":"2024-03-28T13:20:54.467759+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:30:41.860382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468041+0000","last_change":"2024-03-28T13:20:53.459597+0000","last_active":"2024-03-28T13:20:54.468041+0000","last_peered":"2024-03-28T13:20:54.468041+0000","last_clean":"2024-03-28T13:20:54.468041+0000","last_became_active":"2024-03-28T13:20:53.459080+0000","last_became_peered":"2024-03-28T13:20:53.459080+0000","last_unstale":"2024-03-28T13:20:54.468041+0000","last_undegraded":"2024-03-28T13:20:54.468041+0000","last_fullsized":"2024-03-28T13:20:54.468041+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:48:59.909295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.475605+0000","last_change":"2024-03-28T13:20:53.470994+0000","last_active":"2024-03-28T13:20:53.475605+0000","last_peered":"2024-03-28T13:20:53.475605+0000","last_clean":"2024-03-28T13:20:53.475605+0000","last_became_active":"2024-03-28T13:20:53.470474+0000","last_became_peered":"2024-03-28T13:20:53.470474+0000","last_unstale":"2024-03-28T13:20:53.475605+0000","last_undegraded":"2024-03-28T13:20:53.475605+0000","last_fullsized":"2024-03-28T13:20:53.475605+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:22:55.029632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807457+0000","last_change":"2024-03-28T13:20:53.467332+0000","last_active":"2024-03-28T13:20:54.807457+0000","last_peered":"2024-03-28T13:20:54.807457+0000","last_clean":"2024-03-28T13:20:54.807457+0000","last_became_active":"2024-03-28T13:20:53.467072+0000","last_became_peered":"2024-03-28T13:20:53.467072+0000","last_unstale":"2024-03-28T13:20:54.807457+0000","last_undegraded":"2024-03-28T13:20:54.807457+0000","last_fullsized":"2024-03-28T13:20:54.807457+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:25:04.843503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468597+0000","last_change":"2024-03-28T13:20:53.467986+0000","last_active":"2024-03-28T13:20:54.468597+0000","last_peered":"2024-03-28T13:20:54.468597+0000","last_clean":"2024-03-28T13:20:54.468597+0000","last_became_active":"2024-03-28T13:20:53.467599+0000","last_became_peered":"2024-03-28T13:20:53.467599+0000","last_unstale":"2024-03-28T13:20:54.468597+0000","last_undegraded":"2024-03-28T13:20:54.468597+0000","last_fullsized":"2024-03-28T13:20:54.468597+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:48:15.038578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807293+0000","last_change":"2024-03-28T13:20:53.465370+0000","last_active":"2024-03-28T13:20:54.807293+0000","last_peered":"2024-03-28T13:20:54.807293+0000","last_clean":"2024-03-28T13:20:54.807293+0000","last_became_active":"2024-03-28T13:20:53.465136+0000","last_became_peered":"2024-03-28T13:20:53.465136+0000","last_unstale":"2024-03-28T13:20:54.807293+0000","last_undegraded":"2024-03-28T13:20:54.807293+0000","last_fullsized":"2024-03-28T13:20:54.807293+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:46:38.513136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.809406+0000","last_change":"2024-03-28T13:20:53.469025+0000","last_active":"2024-03-28T13:20:54.809406+0000","last_peered":"2024-03-28T13:20:54.809406+0000","last_clean":"2024-03-28T13:20:54.809406+0000","last_became_active":"2024-03-28T13:20:53.465798+0000","last_became_peered":"2024-03-28T13:20:53.465798+0000","last_unstale":"2024-03-28T13:20:54.809406+0000","last_undegraded":"2024-03-28T13:20:54.809406+0000","last_fullsized":"2024-03-28T13:20:54.809406+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:35:09.245847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.469711+0000","last_change":"2024-03-28T13:20:53.468381+0000","last_active":"2024-03-28T13:20:54.469711+0000","last_peered":"2024-03-28T13:20:54.469711+0000","last_clean":"2024-03-28T13:20:54.469711+0000","last_became_active":"2024-03-28T13:20:53.465568+0000","last_became_peered":"2024-03-28T13:20:53.465568+0000","last_unstale":"2024-03-28T13:20:54.469711+0000","last_undegraded":"2024-03-28T13:20:54.469711+0000","last_fullsized":"2024-03-28T13:20:54.469711+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:12:39.048657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.471822+0000","last_change":"2024-03-28T13:20:53.462502+0000","last_active":"2024-03-28T13:20:53.471822+0000","last_peered":"2024-03-28T13:20:53.471822+0000","last_clean":"2024-03-28T13:20:53.471822+0000","last_became_active":"2024-03-28T13:20:53.462152+0000","last_became_peered":"2024-03-28T13:20:53.462152+0000","last_unstale":"2024-03-28T13:20:53.471822+0000","last_undegraded":"2024-03-28T13:20:53.471822+0000","last_fullsized":"2024-03-28T13:20:53.471822+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:06:26.204869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.806479+0000","last_change":"2024-03-28T13:20:53.470895+0000","last_active":"2024-03-28T13:20:54.806479+0000","last_peered":"2024-03-28T13:20:54.806479+0000","last_clean":"2024-03-28T13:20:54.806479+0000","last_became_active":"2024-03-28T13:20:53.467786+0000","last_became_peered":"2024-03-28T13:20:53.467786+0000","last_unstale":"2024-03-28T13:20:54.806479+0000","last_undegraded":"2024-03-28T13:20:54.806479+0000","last_fullsized":"2024-03-28T13:20:54.806479+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:29:48.281661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.808490+0000","last_change":"2024-03-28T13:20:53.470232+0000","last_active":"2024-03-28T13:20:54.808490+0000","last_peered":"2024-03-28T13:20:54.808490+0000","last_clean":"2024-03-28T13:20:54.808490+0000","last_became_active":"2024-03-28T13:20:53.469915+0000","last_became_peered":"2024-03-28T13:20:53.469915+0000","last_unstale":"2024-03-28T13:20:54.808490+0000","last_undegraded":"2024-03-28T13:20:54.808490+0000","last_fullsized":"2024-03-28T13:20:54.808490+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:05:02.392740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468212+0000","last_change":"2024-03-28T13:20:53.457481+0000","last_active":"2024-03-28T13:20:54.468212+0000","last_peered":"2024-03-28T13:20:54.468212+0000","last_clean":"2024-03-28T13:20:54.468212+0000","last_became_active":"2024-03-28T13:20:53.456906+0000","last_became_peered":"2024-03-28T13:20:53.456906+0000","last_unstale":"2024-03-28T13:20:54.468212+0000","last_undegraded":"2024-03-28T13:20:54.468212+0000","last_fullsized":"2024-03-28T13:20:54.468212+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:21:03.576502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.808218+0000","last_change":"2024-03-28T13:20:53.461472+0000","last_active":"2024-03-28T13:20:54.808218+0000","last_peered":"2024-03-28T13:20:54.808218+0000","last_clean":"2024-03-28T13:20:54.808218+0000","last_became_active":"2024-03-28T13:20:53.460857+0000","last_became_peered":"2024-03-28T13:20:53.460857+0000","last_unstale":"2024-03-28T13:20:54.808218+0000","last_undegraded":"2024-03-28T13:20:54.808218+0000","last_fullsized":"2024-03-28T13:20:54.808218+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:01:08.844704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807539+0000","last_change":"2024-03-28T13:20:53.472177+0000","last_active":"2024-03-28T13:20:54.807539+0000","last_peered":"2024-03-28T13:20:54.807539+0000","last_clean":"2024-03-28T13:20:54.807539+0000","last_became_active":"2024-03-28T13:20:53.471845+0000","last_became_peered":"2024-03-28T13:20:53.471845+0000","last_unstale":"2024-03-28T13:20:54.807539+0000","last_undegraded":"2024-03-28T13:20:54.807539+0000","last_fullsized":"2024-03-28T13:20:54.807539+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:41:40.224718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.469217+0000","last_change":"2024-03-28T13:20:53.473564+0000","last_active":"2024-03-28T13:20:54.469217+0000","last_peered":"2024-03-28T13:20:54.469217+0000","last_clean":"2024-03-28T13:20:54.469217+0000","last_became_active":"2024-03-28T13:20:53.472829+0000","last_became_peered":"2024-03-28T13:20:53.472829+0000","last_unstale":"2024-03-28T13:20:54.469217+0000","last_undegraded":"2024-03-28T13:20:54.469217+0000","last_fullsized":"2024-03-28T13:20:54.469217+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:43:57.592462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.464268+0000","last_change":"2024-03-28T13:20:53.462618+0000","last_active":"2024-03-28T13:20:53.464268+0000","last_peered":"2024-03-28T13:20:53.464268+0000","last_clean":"2024-03-28T13:20:53.464268+0000","last_became_active":"2024-03-28T13:20:53.462180+0000","last_became_peered":"2024-03-28T13:20:53.462180+0000","last_unstale":"2024-03-28T13:20:53.464268+0000","last_undegraded":"2024-03-28T13:20:53.464268+0000","last_fullsized":"2024-03-28T13:20:53.464268+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:49:12.296314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.469935+0000","last_change":"2024-03-28T13:20:53.469955+0000","last_active":"2024-03-28T13:20:54.469935+0000","last_peered":"2024-03-28T13:20:54.469935+0000","last_clean":"2024-03-28T13:20:54.469935+0000","last_became_active":"2024-03-28T13:20:53.469435+0000","last_became_peered":"2024-03-28T13:20:53.469435+0000","last_unstale":"2024-03-28T13:20:54.469935+0000","last_undegraded":"2024-03-28T13:20:54.469935+0000","last_fullsized":"2024-03-28T13:20:54.469935+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:47.786867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.474760+0000","last_change":"2024-03-28T13:20:53.469068+0000","last_active":"2024-03-28T13:20:53.474760+0000","last_peered":"2024-03-28T13:20:53.474760+0000","last_clean":"2024-03-28T13:20:53.474760+0000","last_became_active":"2024-03-28T13:20:53.467105+0000","last_became_peered":"2024-03-28T13:20:53.467105+0000","last_unstale":"2024-03-28T13:20:53.474760+0000","last_undegraded":"2024-03-28T13:20:53.474760+0000","last_fullsized":"2024-03-28T13:20:53.474760+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:12:39.048657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468484+0000","last_change":"2024-03-28T13:20:53.473290+0000","last_active":"2024-03-28T13:20:54.468484+0000","last_peered":"2024-03-28T13:20:54.468484+0000","last_clean":"2024-03-28T13:20:54.468484+0000","last_became_active":"2024-03-28T13:20:53.472228+0000","last_became_peered":"2024-03-28T13:20:53.472228+0000","last_unstale":"2024-03-28T13:20:54.468484+0000","last_undegraded":"2024-03-28T13:20:54.468484+0000","last_fullsized":"2024-03-28T13:20:54.468484+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:18:41.661696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.464008+0000","last_change":"2024-03-28T13:20:53.459988+0000","last_active":"2024-03-28T13:20:53.464008+0000","last_peered":"2024-03-28T13:20:53.464008+0000","last_clean":"2024-03-28T13:20:53.464008+0000","last_became_active":"2024-03-28T13:20:53.459664+0000","last_became_peered":"2024-03-28T13:20:53.459664+0000","last_unstale":"2024-03-28T13:20:53.464008+0000","last_undegraded":"2024-03-28T13:20:53.464008+0000","last_fullsized":"2024-03-28T13:20:53.464008+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:02:32.637893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.482349+0000","last_change":"2024-03-28T13:20:53.477331+0000","last_active":"2024-03-28T13:20:53.482349+0000","last_peered":"2024-03-28T13:20:53.482349+0000","last_clean":"2024-03-28T13:20:53.482349+0000","last_became_active":"2024-03-28T13:20:53.476374+0000","last_became_peered":"2024-03-28T13:20:53.476374+0000","last_unstale":"2024-03-28T13:20:53.482349+0000","last_undegraded":"2024-03-28T13:20:53.482349+0000","last_fullsized":"2024-03-28T13:20:53.482349+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:07:25.965347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.480519+0000","last_change":"2024-03-28T13:20:53.477904+0000","last_active":"2024-03-28T13:20:53.480519+0000","last_peered":"2024-03-28T13:20:53.480519+0000","last_clean":"2024-03-28T13:20:53.480519+0000","last_became_active":"2024-03-28T13:20:53.475255+0000","last_became_peered":"2024-03-28T13:20:53.475255+0000","last_unstale":"2024-03-28T13:20:53.480519+0000","last_undegraded":"2024-03-28T13:20:53.480519+0000","last_fullsized":"2024-03-28T13:20:53.480519+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:23:28.084957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.471227+0000","last_change":"2024-03-28T13:20:53.466345+0000","last_active":"2024-03-28T13:20:54.471227+0000","last_peered":"2024-03-28T13:20:54.471227+0000","last_clean":"2024-03-28T13:20:54.471227+0000","last_became_active":"2024-03-28T13:20:53.466004+0000","last_became_peered":"2024-03-28T13:20:53.466004+0000","last_unstale":"2024-03-28T13:20:54.471227+0000","last_undegraded":"2024-03-28T13:20:54.471227+0000","last_fullsized":"2024-03-28T13:20:54.471227+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:12.329663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468392+0000","last_change":"2024-03-28T13:20:53.459392+0000","last_active":"2024-03-28T13:20:54.468392+0000","last_peered":"2024-03-28T13:20:54.468392+0000","last_clean":"2024-03-28T13:20:54.468392+0000","last_became_active":"2024-03-28T13:20:53.459073+0000","last_became_peered":"2024-03-28T13:20:53.459073+0000","last_unstale":"2024-03-28T13:20:54.468392+0000","last_undegraded":"2024-03-28T13:20:54.468392+0000","last_fullsized":"2024-03-28T13:20:54.468392+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:26:41.108125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.474697+0000","last_change":"2024-03-28T13:20:53.465014+0000","last_active":"2024-03-28T13:20:53.474697+0000","last_peered":"2024-03-28T13:20:53.474697+0000","last_clean":"2024-03-28T13:20:53.474697+0000","last_became_active":"2024-03-28T13:20:53.464721+0000","last_became_peered":"2024-03-28T13:20:53.464721+0000","last_unstale":"2024-03-28T13:20:53.474697+0000","last_undegraded":"2024-03-28T13:20:53.474697+0000","last_fullsized":"2024-03-28T13:20:53.474697+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:32:38.627989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.806752+0000","last_change":"2024-03-28T13:20:53.470952+0000","last_active":"2024-03-28T13:20:54.806752+0000","last_peered":"2024-03-28T13:20:54.806752+0000","last_clean":"2024-03-28T13:20:54.806752+0000","last_became_active":"2024-03-28T13:20:53.470652+0000","last_became_peered":"2024-03-28T13:20:53.470652+0000","last_unstale":"2024-03-28T13:20:54.806752+0000","last_undegraded":"2024-03-28T13:20:54.806752+0000","last_fullsized":"2024-03-28T13:20:54.806752+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:56:38.094183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.475539+0000","last_change":"2024-03-28T13:20:53.473122+0000","last_active":"2024-03-28T13:20:53.475539+0000","last_peered":"2024-03-28T13:20:53.475539+0000","last_clean":"2024-03-28T13:20:53.475539+0000","last_became_active":"2024-03-28T13:20:53.470214+0000","last_became_peered":"2024-03-28T13:20:53.470214+0000","last_unstale":"2024-03-28T13:20:53.475539+0000","last_undegraded":"2024-03-28T13:20:53.475539+0000","last_fullsized":"2024-03-28T13:20:53.475539+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:27:15.147525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"118'4","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.471014+0000","last_change":"2024-03-28T13:20:52.426502+0000","last_active":"2024-03-28T13:20:54.471014+0000","last_peered":"2024-03-28T13:20:54.471014+0000","last_clean":"2024-03-28T13:20:54.471014+0000","last_became_active":"2024-03-28T13:20:52.426181+0000","last_became_peered":"2024-03-28T13:20:52.426181+0000","last_unstale":"2024-03-28T13:20:54.471014+0000","last_undegraded":"2024-03-28T13:20:54.471014+0000","last_fullsized":"2024-03-28T13:20:54.471014+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:18:08.312130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.470889+0000","last_change":"2024-03-28T13:20:53.468637+0000","last_active":"2024-03-28T13:20:54.470889+0000","last_peered":"2024-03-28T13:20:54.470889+0000","last_clean":"2024-03-28T13:20:54.470889+0000","last_became_active":"2024-03-28T13:20:53.468093+0000","last_became_peered":"2024-03-28T13:20:53.468093+0000","last_unstale":"2024-03-28T13:20:54.470889+0000","last_undegraded":"2024-03-28T13:20:54.470889+0000","last_fullsized":"2024-03-28T13:20:54.470889+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:35.591833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.809918+0000","last_change":"2024-03-28T13:20:52.426075+0000","last_active":"2024-03-28T13:20:54.809918+0000","last_peered":"2024-03-28T13:20:54.809918+0000","last_clean":"2024-03-28T13:20:54.809918+0000","last_became_active":"2024-03-28T13:20:52.425617+0000","last_became_peered":"2024-03-28T13:20:52.425617+0000","last_unstale":"2024-03-28T13:20:54.809918+0000","last_undegraded":"2024-03-28T13:20:54.809918+0000","last_fullsized":"2024-03-28T13:20:54.809918+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:26.395440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.462200+0000","last_change":"2024-03-28T13:20:53.454767+0000","last_active":"2024-03-28T13:20:53.462200+0000","last_peered":"2024-03-28T13:20:53.462200+0000","last_clean":"2024-03-28T13:20:53.462200+0000","last_became_active":"2024-03-28T13:20:53.454359+0000","last_became_peered":"2024-03-28T13:20:53.454359+0000","last_unstale":"2024-03-28T13:20:53.462200+0000","last_undegraded":"2024-03-28T13:20:53.462200+0000","last_fullsized":"2024-03-28T13:20:53.462200+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:39:02.869522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.488169+0000","last_change":"2024-03-28T13:20:52.442602+0000","last_active":"2024-03-28T13:20:53.488169+0000","last_peered":"2024-03-28T13:20:53.488169+0000","last_clean":"2024-03-28T13:20:53.488169+0000","last_became_active":"2024-03-28T13:20:52.442305+0000","last_became_peered":"2024-03-28T13:20:52.442305+0000","last_unstale":"2024-03-28T13:20:53.488169+0000","last_undegraded":"2024-03-28T13:20:53.488169+0000","last_fullsized":"2024-03-28T13:20:53.488169+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:07:59.665654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.808347+0000","last_change":"2024-03-28T13:20:53.463740+0000","last_active":"2024-03-28T13:20:54.808347+0000","last_peered":"2024-03-28T13:20:54.808347+0000","last_clean":"2024-03-28T13:20:54.808347+0000","last_became_active":"2024-03-28T13:20:53.463001+0000","last_became_peered":"2024-03-28T13:20:53.463001+0000","last_unstale":"2024-03-28T13:20:54.808347+0000","last_undegraded":"2024-03-28T13:20:54.808347+0000","last_fullsized":"2024-03-28T13:20:54.808347+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:00:41.753031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.488371+0000","last_change":"2024-03-28T13:20:52.444849+0000","last_active":"2024-03-28T13:20:53.488371+0000","last_peered":"2024-03-28T13:20:53.488371+0000","last_clean":"2024-03-28T13:20:53.488371+0000","last_became_active":"2024-03-28T13:20:52.444552+0000","last_became_peered":"2024-03-28T13:20:52.444552+0000","last_unstale":"2024-03-28T13:20:53.488371+0000","last_undegraded":"2024-03-28T13:20:53.488371+0000","last_fullsized":"2024-03-28T13:20:53.488371+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:05:01.377390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.473246+0000","last_change":"2024-03-28T13:20:53.466028+0000","last_active":"2024-03-28T13:20:53.473246+0000","last_peered":"2024-03-28T13:20:53.473246+0000","last_clean":"2024-03-28T13:20:53.473246+0000","last_became_active":"2024-03-28T13:20:53.465466+0000","last_became_peered":"2024-03-28T13:20:53.465466+0000","last_unstale":"2024-03-28T13:20:53.473246+0000","last_undegraded":"2024-03-28T13:20:53.473246+0000","last_fullsized":"2024-03-28T13:20:53.473246+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:25:31.976436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807563+0000","last_change":"2024-03-28T13:20:52.429598+0000","last_active":"2024-03-28T13:20:54.807563+0000","last_peered":"2024-03-28T13:20:54.807563+0000","last_clean":"2024-03-28T13:20:54.807563+0000","last_became_active":"2024-03-28T13:20:52.429132+0000","last_became_peered":"2024-03-28T13:20:52.429132+0000","last_unstale":"2024-03-28T13:20:54.807563+0000","last_undegraded":"2024-03-28T13:20:54.807563+0000","last_fullsized":"2024-03-28T13:20:54.807563+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:33:05.548959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468919+0000","last_change":"2024-03-28T13:20:53.461800+0000","last_active":"2024-03-28T13:20:54.468919+0000","last_peered":"2024-03-28T13:20:54.468919+0000","last_clean":"2024-03-28T13:20:54.468919+0000","last_became_active":"2024-03-28T13:20:53.458431+0000","last_became_peered":"2024-03-28T13:20:53.458431+0000","last_unstale":"2024-03-28T13:20:54.468919+0000","last_undegraded":"2024-03-28T13:20:54.468919+0000","last_fullsized":"2024-03-28T13:20:54.468919+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:30:03.655476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.487821+0000","last_change":"2024-03-28T13:20:52.438748+0000","last_active":"2024-03-28T13:20:53.487821+0000","last_peered":"2024-03-28T13:20:53.487821+0000","last_clean":"2024-03-28T13:20:53.487821+0000","last_became_active":"2024-03-28T13:20:52.436838+0000","last_became_peered":"2024-03-28T13:20:52.436838+0000","last_unstale":"2024-03-28T13:20:53.487821+0000","last_undegraded":"2024-03-28T13:20:53.487821+0000","last_fullsized":"2024-03-28T13:20:53.487821+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:07:24.949997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.806932+0000","last_change":"2024-03-28T13:20:53.470648+0000","last_active":"2024-03-28T13:20:54.806932+0000","last_peered":"2024-03-28T13:20:54.806932+0000","last_clean":"2024-03-28T13:20:54.806932+0000","last_became_active":"2024-03-28T13:20:53.469342+0000","last_became_peered":"2024-03-28T13:20:53.469342+0000","last_unstale":"2024-03-28T13:20:54.806932+0000","last_undegraded":"2024-03-28T13:20:54.806932+0000","last_fullsized":"2024-03-28T13:20:54.806932+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:52:58.120287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468098+0000","last_change":"2024-03-28T13:20:52.442360+0000","last_active":"2024-03-28T13:20:54.468098+0000","last_peered":"2024-03-28T13:20:54.468098+0000","last_clean":"2024-03-28T13:20:54.468098+0000","last_became_active":"2024-03-28T13:20:52.441911+0000","last_became_peered":"2024-03-28T13:20:52.441911+0000","last_unstale":"2024-03-28T13:20:54.468098+0000","last_undegraded":"2024-03-28T13:20:54.468098+0000","last_fullsized":"2024-03-28T13:20:54.468098+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:12:57.608427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.467957+0000","last_change":"2024-03-28T13:20:53.461088+0000","last_active":"2024-03-28T13:20:54.467957+0000","last_peered":"2024-03-28T13:20:54.467957+0000","last_clean":"2024-03-28T13:20:54.467957+0000","last_became_active":"2024-03-28T13:20:53.460345+0000","last_became_peered":"2024-03-28T13:20:53.460345+0000","last_unstale":"2024-03-28T13:20:54.467957+0000","last_undegraded":"2024-03-28T13:20:54.467957+0000","last_fullsized":"2024-03-28T13:20:54.467957+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:43:57.040909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.806938+0000","last_change":"2024-03-28T13:20:52.448170+0000","last_active":"2024-03-28T13:20:54.806938+0000","last_peered":"2024-03-28T13:20:54.806938+0000","last_clean":"2024-03-28T13:20:54.806938+0000","last_became_active":"2024-03-28T13:20:52.447274+0000","last_became_peered":"2024-03-28T13:20:52.447274+0000","last_unstale":"2024-03-28T13:20:54.806938+0000","last_undegraded":"2024-03-28T13:20:54.806938+0000","last_fullsized":"2024-03-28T13:20:54.806938+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:25:16.379571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.809650+0000","last_change":"2024-03-28T13:20:53.476237+0000","last_active":"2024-03-28T13:20:54.809650+0000","last_peered":"2024-03-28T13:20:54.809650+0000","last_clean":"2024-03-28T13:20:54.809650+0000","last_became_active":"2024-03-28T13:20:53.475843+0000","last_became_peered":"2024-03-28T13:20:53.475843+0000","last_unstale":"2024-03-28T13:20:54.809650+0000","last_undegraded":"2024-03-28T13:20:54.809650+0000","last_fullsized":"2024-03-28T13:20:54.809650+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:16:26.602338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.464498+0000","last_change":"2024-03-28T13:20:52.441359+0000","last_active":"2024-03-28T13:20:53.464498+0000","last_peered":"2024-03-28T13:20:53.464498+0000","last_clean":"2024-03-28T13:20:53.464498+0000","last_became_active":"2024-03-28T13:20:52.440551+0000","last_became_peered":"2024-03-28T13:20:52.440551+0000","last_unstale":"2024-03-28T13:20:53.464498+0000","last_undegraded":"2024-03-28T13:20:53.464498+0000","last_fullsized":"2024-03-28T13:20:53.464498+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:01:07.829354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.810064+0000","last_change":"2024-03-28T13:20:53.468240+0000","last_active":"2024-03-28T13:20:54.810064+0000","last_peered":"2024-03-28T13:20:54.810064+0000","last_clean":"2024-03-28T13:20:54.810064+0000","last_became_active":"2024-03-28T13:20:53.466214+0000","last_became_peered":"2024-03-28T13:20:53.466214+0000","last_unstale":"2024-03-28T13:20:54.810064+0000","last_undegraded":"2024-03-28T13:20:54.810064+0000","last_fullsized":"2024-03-28T13:20:54.810064+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:43:37.800350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807307+0000","last_change":"2024-03-28T13:20:52.448230+0000","last_active":"2024-03-28T13:20:54.807307+0000","last_peered":"2024-03-28T13:20:54.807307+0000","last_clean":"2024-03-28T13:20:54.807307+0000","last_became_active":"2024-03-28T13:20:52.447633+0000","last_became_peered":"2024-03-28T13:20:52.447633+0000","last_unstale":"2024-03-28T13:20:54.807307+0000","last_undegraded":"2024-03-28T13:20:54.807307+0000","last_fullsized":"2024-03-28T13:20:54.807307+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:54:02.411950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.478017+0000","last_change":"2024-03-28T13:20:53.472592+0000","last_active":"2024-03-28T13:20:53.478017+0000","last_peered":"2024-03-28T13:20:53.478017+0000","last_clean":"2024-03-28T13:20:53.478017+0000","last_became_active":"2024-03-28T13:20:53.469863+0000","last_became_peered":"2024-03-28T13:20:53.469863+0000","last_unstale":"2024-03-28T13:20:53.478017+0000","last_undegraded":"2024-03-28T13:20:53.478017+0000","last_fullsized":"2024-03-28T13:20:53.478017+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:36:19.800467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"118'3","reported_seq":17,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807132+0000","last_change":"2024-03-28T13:20:52.448990+0000","last_active":"2024-03-28T13:20:54.807132+0000","last_peered":"2024-03-28T13:20:54.807132+0000","last_clean":"2024-03-28T13:20:54.807132+0000","last_became_active":"2024-03-28T13:20:52.448329+0000","last_became_peered":"2024-03-28T13:20:52.448329+0000","last_unstale":"2024-03-28T13:20:54.807132+0000","last_undegraded":"2024-03-28T13:20:54.807132+0000","last_fullsized":"2024-03-28T13:20:54.807132+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:09:26.039996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468775+0000","last_change":"2024-03-28T13:20:53.462231+0000","last_active":"2024-03-28T13:20:54.468775+0000","last_peered":"2024-03-28T13:20:54.468775+0000","last_clean":"2024-03-28T13:20:54.468775+0000","last_became_active":"2024-03-28T13:20:53.461889+0000","last_became_peered":"2024-03-28T13:20:53.461889+0000","last_unstale":"2024-03-28T13:20:54.468775+0000","last_undegraded":"2024-03-28T13:20:54.468775+0000","last_fullsized":"2024-03-28T13:20:54.468775+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:27:35.648328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"118'6","reported_seq":18,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.492037+0000","last_change":"2024-03-28T13:20:52.444751+0000","last_active":"2024-03-28T13:20:53.492037+0000","last_peered":"2024-03-28T13:20:53.492037+0000","last_clean":"2024-03-28T13:20:53.492037+0000","last_became_active":"2024-03-28T13:20:52.444439+0000","last_became_peered":"2024-03-28T13:20:52.444439+0000","last_unstale":"2024-03-28T13:20:53.492037+0000","last_undegraded":"2024-03-28T13:20:53.492037+0000","last_fullsized":"2024-03-28T13:20:53.492037+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:35:08.230497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.806378+0000","last_change":"2024-03-28T13:20:53.467940+0000","last_active":"2024-03-28T13:20:54.806378+0000","last_peered":"2024-03-28T13:20:54.806378+0000","last_clean":"2024-03-28T13:20:54.806378+0000","last_became_active":"2024-03-28T13:20:53.467604+0000","last_became_peered":"2024-03-28T13:20:53.467604+0000","last_unstale":"2024-03-28T13:20:54.806378+0000","last_undegraded":"2024-03-28T13:20:54.806378+0000","last_fullsized":"2024-03-28T13:20:54.806378+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:21:54.864137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.806888+0000","last_change":"2024-03-28T13:20:52.448319+0000","last_active":"2024-03-28T13:20:54.806888+0000","last_peered":"2024-03-28T13:20:54.806888+0000","last_clean":"2024-03-28T13:20:54.806888+0000","last_became_active":"2024-03-28T13:20:52.447526+0000","last_became_peered":"2024-03-28T13:20:52.447526+0000","last_unstale":"2024-03-28T13:20:54.806888+0000","last_undegraded":"2024-03-28T13:20:54.806888+0000","last_fullsized":"2024-03-28T13:20:54.806888+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:15:27.835050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.483290+0000","last_change":"2024-03-28T13:20:53.479412+0000","last_active":"2024-03-28T13:20:53.483290+0000","last_peered":"2024-03-28T13:20:53.483290+0000","last_clean":"2024-03-28T13:20:53.483290+0000","last_became_active":"2024-03-28T13:20:53.479127+0000","last_became_peered":"2024-03-28T13:20:53.479127+0000","last_unstale":"2024-03-28T13:20:53.483290+0000","last_undegraded":"2024-03-28T13:20:53.483290+0000","last_fullsized":"2024-03-28T13:20:53.483290+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:11.084472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807986+0000","last_change":"2024-03-28T13:20:52.438117+0000","last_active":"2024-03-28T13:20:54.807986+0000","last_peered":"2024-03-28T13:20:54.807986+0000","last_clean":"2024-03-28T13:20:54.807986+0000","last_became_active":"2024-03-28T13:20:52.437864+0000","last_became_peered":"2024-03-28T13:20:52.437864+0000","last_unstale":"2024-03-28T13:20:54.807986+0000","last_undegraded":"2024-03-28T13:20:54.807986+0000","last_fullsized":"2024-03-28T13:20:54.807986+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T00:42:17.679510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.476303+0000","last_change":"2024-03-28T13:20:53.468630+0000","last_active":"2024-03-28T13:20:53.476303+0000","last_peered":"2024-03-28T13:20:53.476303+0000","last_clean":"2024-03-28T13:20:53.476303+0000","last_became_active":"2024-03-28T13:20:53.468281+0000","last_became_peered":"2024-03-28T13:20:53.468281+0000","last_unstale":"2024-03-28T13:20:53.476303+0000","last_undegraded":"2024-03-28T13:20:53.476303+0000","last_fullsized":"2024-03-28T13:20:53.476303+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:43:37.800350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"118'8","reported_seq":20,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.493619+0000","last_change":"2024-03-28T13:20:52.433015+0000","last_active":"2024-03-28T13:20:53.493619+0000","last_peered":"2024-03-28T13:20:53.493619+0000","last_clean":"2024-03-28T13:20:53.493619+0000","last_became_active":"2024-03-28T13:20:52.432759+0000","last_became_peered":"2024-03-28T13:20:52.432759+0000","last_unstale":"2024-03-28T13:20:53.493619+0000","last_undegraded":"2024-03-28T13:20:53.493619+0000","last_fullsized":"2024-03-28T13:20:53.493619+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:31:46.771516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.460780+0000","last_change":"2024-03-28T13:20:53.449284+0000","last_active":"2024-03-28T13:20:53.460780+0000","last_peered":"2024-03-28T13:20:53.460780+0000","last_clean":"2024-03-28T13:20:53.460780+0000","last_became_active":"2024-03-28T13:20:53.448572+0000","last_became_peered":"2024-03-28T13:20:53.448572+0000","last_unstale":"2024-03-28T13:20:53.460780+0000","last_undegraded":"2024-03-28T13:20:53.460780+0000","last_fullsized":"2024-03-28T13:20:53.460780+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:35.591833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"118'1","reported_seq":13,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.484172+0000","last_change":"2024-03-28T13:20:52.430184+0000","last_active":"2024-03-28T13:20:53.484172+0000","last_peered":"2024-03-28T13:20:53.484172+0000","last_clean":"2024-03-28T13:20:53.484172+0000","last_became_active":"2024-03-28T13:20:52.429698+0000","last_became_peered":"2024-03-28T13:20:52.429698+0000","last_unstale":"2024-03-28T13:20:53.484172+0000","last_undegraded":"2024-03-28T13:20:53.484172+0000","last_fullsized":"2024-03-28T13:20:53.484172+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:57:41.127141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.475585+0000","last_change":"2024-03-28T13:20:53.470762+0000","last_active":"2024-03-28T13:20:53.475585+0000","last_peered":"2024-03-28T13:20:53.475585+0000","last_clean":"2024-03-28T13:20:53.475585+0000","last_became_active":"2024-03-28T13:20:53.470040+0000","last_became_peered":"2024-03-28T13:20:53.470040+0000","last_unstale":"2024-03-28T13:20:53.475585+0000","last_undegraded":"2024-03-28T13:20:53.475585+0000","last_fullsized":"2024-03-28T13:20:53.475585+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:01:21.238792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807566+0000","last_change":"2024-03-28T13:20:52.448629+0000","last_active":"2024-03-28T13:20:54.807566+0000","last_peered":"2024-03-28T13:20:54.807566+0000","last_clean":"2024-03-28T13:20:54.807566+0000","last_became_active":"2024-03-28T13:20:52.447976+0000","last_became_peered":"2024-03-28T13:20:52.447976+0000","last_unstale":"2024-03-28T13:20:54.807566+0000","last_undegraded":"2024-03-28T13:20:54.807566+0000","last_fullsized":"2024-03-28T13:20:54.807566+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:20:25.185712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.468959+0000","last_change":"2024-03-28T13:20:53.457848+0000","last_active":"2024-03-28T13:20:54.468959+0000","last_peered":"2024-03-28T13:20:54.468959+0000","last_clean":"2024-03-28T13:20:54.468959+0000","last_became_active":"2024-03-28T13:20:53.457538+0000","last_became_peered":"2024-03-28T13:20:53.457538+0000","last_unstale":"2024-03-28T13:20:54.468959+0000","last_undegraded":"2024-03-28T13:20:54.468959+0000","last_fullsized":"2024-03-28T13:20:54.468959+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:43:46.400454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"118'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.471042+0000","last_change":"2024-03-28T13:20:52.435350+0000","last_active":"2024-03-28T13:20:54.471042+0000","last_peered":"2024-03-28T13:20:54.471042+0000","last_clean":"2024-03-28T13:20:54.471042+0000","last_became_active":"2024-03-28T13:20:52.435002+0000","last_became_peered":"2024-03-28T13:20:52.435002+0000","last_unstale":"2024-03-28T13:20:54.471042+0000","last_undegraded":"2024-03-28T13:20:54.471042+0000","last_fullsized":"2024-03-28T13:20:54.471042+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:56:08.871552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.470848+0000","last_change":"2024-03-28T13:20:53.465022+0000","last_active":"2024-03-28T13:20:54.470848+0000","last_peered":"2024-03-28T13:20:54.470848+0000","last_clean":"2024-03-28T13:20:54.470848+0000","last_became_active":"2024-03-28T13:20:53.464673+0000","last_became_peered":"2024-03-28T13:20:53.464673+0000","last_unstale":"2024-03-28T13:20:54.470848+0000","last_undegraded":"2024-03-28T13:20:54.470848+0000","last_fullsized":"2024-03-28T13:20:54.470848+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:56:03.874557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.470414+0000","last_change":"2024-03-28T13:20:52.439533+0000","last_active":"2024-03-28T13:20:54.470414+0000","last_peered":"2024-03-28T13:20:54.470414+0000","last_clean":"2024-03-28T13:20:54.470414+0000","last_became_active":"2024-03-28T13:20:52.439196+0000","last_became_peered":"2024-03-28T13:20:52.439196+0000","last_unstale":"2024-03-28T13:20:54.470414+0000","last_undegraded":"2024-03-28T13:20:54.470414+0000","last_fullsized":"2024-03-28T13:20:54.470414+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:36:59.221033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807641+0000","last_change":"2024-03-28T13:20:53.462615+0000","last_active":"2024-03-28T13:20:54.807641+0000","last_peered":"2024-03-28T13:20:54.807641+0000","last_clean":"2024-03-28T13:20:54.807641+0000","last_became_active":"2024-03-28T13:20:53.461933+0000","last_became_peered":"2024-03-28T13:20:53.461933+0000","last_unstale":"2024-03-28T13:20:54.807641+0000","last_undegraded":"2024-03-28T13:20:54.807641+0000","last_fullsized":"2024-03-28T13:20:54.807641+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:05:52.611316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"118'2","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807193+0000","last_change":"2024-03-28T13:20:52.447056+0000","last_active":"2024-03-28T13:20:54.807193+0000","last_peered":"2024-03-28T13:20:54.807193+0000","last_clean":"2024-03-28T13:20:54.807193+0000","last_became_active":"2024-03-28T13:20:52.446652+0000","last_became_peered":"2024-03-28T13:20:52.446652+0000","last_unstale":"2024-03-28T13:20:54.807193+0000","last_undegraded":"2024-03-28T13:20:54.807193+0000","last_fullsized":"2024-03-28T13:20:54.807193+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T17:03:53.875411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.471532+0000","last_change":"2024-03-28T13:20:53.458081+0000","last_active":"2024-03-28T13:20:53.471532+0000","last_peered":"2024-03-28T13:20:53.471532+0000","last_clean":"2024-03-28T13:20:53.471532+0000","last_became_active":"2024-03-28T13:20:53.457743+0000","last_became_peered":"2024-03-28T13:20:53.457743+0000","last_unstale":"2024-03-28T13:20:53.471532+0000","last_undegraded":"2024-03-28T13:20:53.471532+0000","last_fullsized":"2024-03-28T13:20:53.471532+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:11:12.329663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.437791+0000","last_change":"2024-03-28T13:20:52.442543+0000","last_active":"2024-03-28T13:20:53.437791+0000","last_peered":"2024-03-28T13:20:53.437791+0000","last_clean":"2024-03-28T13:20:53.437791+0000","last_became_active":"2024-03-28T13:20:52.440638+0000","last_became_peered":"2024-03-28T13:20:52.440638+0000","last_unstale":"2024-03-28T13:20:53.437791+0000","last_undegraded":"2024-03-28T13:20:53.437791+0000","last_fullsized":"2024-03-28T13:20:53.437791+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:56:02.859206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.469578+0000","last_change":"2024-03-28T13:20:53.468819+0000","last_active":"2024-03-28T13:20:54.469578+0000","last_peered":"2024-03-28T13:20:54.469578+0000","last_clean":"2024-03-28T13:20:54.469578+0000","last_became_active":"2024-03-28T13:20:53.468111+0000","last_became_peered":"2024-03-28T13:20:53.468111+0000","last_unstale":"2024-03-28T13:20:54.469578+0000","last_undegraded":"2024-03-28T13:20:54.469578+0000","last_fullsized":"2024-03-28T13:20:54.469578+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:16:26.602338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.463015+0000","last_change":"2024-03-28T13:20:52.444756+0000","last_active":"2024-03-28T13:20:53.463015+0000","last_peered":"2024-03-28T13:20:53.463015+0000","last_clean":"2024-03-28T13:20:53.463015+0000","last_became_active":"2024-03-28T13:20:52.444280+0000","last_became_peered":"2024-03-28T13:20:52.444280+0000","last_unstale":"2024-03-28T13:20:53.463015+0000","last_undegraded":"2024-03-28T13:20:53.463015+0000","last_fullsized":"2024-03-28T13:20:53.463015+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T14:41:39.209368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.806466+0000","last_change":"2024-03-28T13:20:53.472165+0000","last_active":"2024-03-28T13:20:54.806466+0000","last_peered":"2024-03-28T13:20:54.806466+0000","last_clean":"2024-03-28T13:20:54.806466+0000","last_became_active":"2024-03-28T13:20:53.471892+0000","last_became_peered":"2024-03-28T13:20:53.471892+0000","last_unstale":"2024-03-28T13:20:54.806466+0000","last_undegraded":"2024-03-28T13:20:54.806466+0000","last_fullsized":"2024-03-28T13:20:54.806466+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:06:15.693793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.463412+0000","last_change":"2024-03-28T13:20:52.438937+0000","last_active":"2024-03-28T13:20:53.463412+0000","last_peered":"2024-03-28T13:20:53.463412+0000","last_clean":"2024-03-28T13:20:53.463412+0000","last_became_active":"2024-03-28T13:20:52.437894+0000","last_became_peered":"2024-03-28T13:20:52.437894+0000","last_unstale":"2024-03-28T13:20:53.463412+0000","last_undegraded":"2024-03-28T13:20:53.463412+0000","last_fullsized":"2024-03-28T13:20:53.463412+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T23:05:51.595965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.470684+0000","last_change":"2024-03-28T13:20:53.467384+0000","last_active":"2024-03-28T13:20:54.470684+0000","last_peered":"2024-03-28T13:20:54.470684+0000","last_clean":"2024-03-28T13:20:54.470684+0000","last_became_active":"2024-03-28T13:20:53.467098+0000","last_became_peered":"2024-03-28T13:20:53.467098+0000","last_unstale":"2024-03-28T13:20:54.470684+0000","last_undegraded":"2024-03-28T13:20:54.470684+0000","last_fullsized":"2024-03-28T13:20:54.470684+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T16:07:25.965347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"118'2","reported_seq":14,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.488125+0000","last_change":"2024-03-28T13:20:52.440581+0000","last_active":"2024-03-28T13:20:53.488125+0000","last_peered":"2024-03-28T13:20:53.488125+0000","last_clean":"2024-03-28T13:20:53.488125+0000","last_became_active":"2024-03-28T13:20:52.440279+0000","last_became_peered":"2024-03-28T13:20:52.440279+0000","last_unstale":"2024-03-28T13:20:53.488125+0000","last_undegraded":"2024-03-28T13:20:53.488125+0000","last_fullsized":"2024-03-28T13:20:53.488125+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:00:40.737680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.476022+0000","last_change":"2024-03-28T13:20:53.474638+0000","last_active":"2024-03-28T13:20:53.476022+0000","last_peered":"2024-03-28T13:20:53.476022+0000","last_clean":"2024-03-28T13:20:53.476022+0000","last_became_active":"2024-03-28T13:20:53.474304+0000","last_became_peered":"2024-03-28T13:20:53.474304+0000","last_unstale":"2024-03-28T13:20:53.476022+0000","last_undegraded":"2024-03-28T13:20:53.476022+0000","last_fullsized":"2024-03-28T13:20:53.476022+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:23:20.466097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"119'1","reported_seq":15,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.484389+0000","last_change":"2024-03-28T13:20:52.430955+0000","last_active":"2024-03-28T13:20:54.484389+0000","last_peered":"2024-03-28T13:20:54.484389+0000","last_clean":"2024-03-28T13:20:54.484389+0000","last_became_active":"2024-03-28T13:20:52.430582+0000","last_became_peered":"2024-03-28T13:20:52.430582+0000","last_unstale":"2024-03-28T13:20:54.484389+0000","last_undegraded":"2024-03-28T13:20:54.484389+0000","last_fullsized":"2024-03-28T13:20:54.484389+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T13:21:35.563041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.473489+0000","last_change":"2024-03-28T13:20:53.463792+0000","last_active":"2024-03-28T13:20:53.473489+0000","last_peered":"2024-03-28T13:20:53.473489+0000","last_clean":"2024-03-28T13:20:53.473489+0000","last_became_active":"2024-03-28T13:20:53.463451+0000","last_became_peered":"2024-03-28T13:20:53.463451+0000","last_unstale":"2024-03-28T13:20:53.473489+0000","last_undegraded":"2024-03-28T13:20:53.473489+0000","last_fullsized":"2024-03-28T13:20:53.473489+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:04:36.260367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.463943+0000","last_change":"2024-03-28T13:20:52.438608+0000","last_active":"2024-03-28T13:20:53.463943+0000","last_peered":"2024-03-28T13:20:53.463943+0000","last_clean":"2024-03-28T13:20:53.463943+0000","last_became_active":"2024-03-28T13:20:52.437785+0000","last_became_peered":"2024-03-28T13:20:52.437785+0000","last_unstale":"2024-03-28T13:20:53.463943+0000","last_undegraded":"2024-03-28T13:20:53.463943+0000","last_fullsized":"2024-03-28T13:20:53.463943+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:26.395440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.470704+0000","last_change":"2024-03-28T13:20:53.475497+0000","last_active":"2024-03-28T13:20:54.470704+0000","last_peered":"2024-03-28T13:20:54.470704+0000","last_clean":"2024-03-28T13:20:54.470704+0000","last_became_active":"2024-03-28T13:20:53.471389+0000","last_became_peered":"2024-03-28T13:20:53.471389+0000","last_unstale":"2024-03-28T13:20:54.470704+0000","last_undegraded":"2024-03-28T13:20:54.470704+0000","last_fullsized":"2024-03-28T13:20:54.470704+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:11.084472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.467282+0000","last_change":"2024-03-28T13:20:52.438113+0000","last_active":"2024-03-28T13:20:53.467282+0000","last_peered":"2024-03-28T13:20:53.467282+0000","last_clean":"2024-03-28T13:20:53.467282+0000","last_became_active":"2024-03-28T13:20:52.437814+0000","last_became_peered":"2024-03-28T13:20:52.437814+0000","last_unstale":"2024-03-28T13:20:53.467282+0000","last_undegraded":"2024-03-28T13:20:53.467282+0000","last_fullsized":"2024-03-28T13:20:53.467282+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T01:18:33.178252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.808096+0000","last_change":"2024-03-28T13:20:53.474595+0000","last_active":"2024-03-28T13:20:54.808096+0000","last_peered":"2024-03-28T13:20:54.808096+0000","last_clean":"2024-03-28T13:20:54.808096+0000","last_became_active":"2024-03-28T13:20:53.474117+0000","last_became_peered":"2024-03-28T13:20:53.474117+0000","last_unstale":"2024-03-28T13:20:54.808096+0000","last_undegraded":"2024-03-28T13:20:54.808096+0000","last_fullsized":"2024-03-28T13:20:54.808096+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:45:27.410791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.469334+0000","last_change":"2024-03-28T13:20:52.422254+0000","last_active":"2024-03-28T13:20:54.469334+0000","last_peered":"2024-03-28T13:20:54.469334+0000","last_clean":"2024-03-28T13:20:54.469334+0000","last_became_active":"2024-03-28T13:20:52.421865+0000","last_became_peered":"2024-03-28T13:20:52.421865+0000","last_unstale":"2024-03-28T13:20:54.469334+0000","last_undegraded":"2024-03-28T13:20:54.469334+0000","last_fullsized":"2024-03-28T13:20:54.469334+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:22:14.058698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.471509+0000","last_change":"2024-03-28T13:20:53.462490+0000","last_active":"2024-03-28T13:20:54.471509+0000","last_peered":"2024-03-28T13:20:54.471509+0000","last_clean":"2024-03-28T13:20:54.471509+0000","last_became_active":"2024-03-28T13:20:53.462053+0000","last_became_peered":"2024-03-28T13:20:53.462053+0000","last_unstale":"2024-03-28T13:20:54.471509+0000","last_undegraded":"2024-03-28T13:20:54.471509+0000","last_fullsized":"2024-03-28T13:20:54.471509+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:06:26.204869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.470094+0000","last_change":"2024-03-28T13:20:52.433980+0000","last_active":"2024-03-28T13:20:54.470094+0000","last_peered":"2024-03-28T13:20:54.470094+0000","last_clean":"2024-03-28T13:20:54.470094+0000","last_became_active":"2024-03-28T13:20:52.433677+0000","last_became_peered":"2024-03-28T13:20:52.433677+0000","last_unstale":"2024-03-28T13:20:54.470094+0000","last_undegraded":"2024-03-28T13:20:54.470094+0000","last_fullsized":"2024-03-28T13:20:54.470094+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:12:21.534987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.469571+0000","last_change":"2024-03-28T13:20:53.469829+0000","last_active":"2024-03-28T13:20:54.469571+0000","last_peered":"2024-03-28T13:20:54.469571+0000","last_clean":"2024-03-28T13:20:54.469571+0000","last_became_active":"2024-03-28T13:20:53.469522+0000","last_became_peered":"2024-03-28T13:20:53.469522+0000","last_unstale":"2024-03-28T13:20:54.469571+0000","last_undegraded":"2024-03-28T13:20:54.469571+0000","last_fullsized":"2024-03-28T13:20:54.469571+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T22:43:37.800350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.437363+0000","last_change":"2024-03-28T13:20:52.436068+0000","last_active":"2024-03-28T13:20:53.437363+0000","last_peered":"2024-03-28T13:20:53.437363+0000","last_clean":"2024-03-28T13:20:53.437363+0000","last_became_active":"2024-03-28T13:20:52.435482+0000","last_became_peered":"2024-03-28T13:20:52.435482+0000","last_unstale":"2024-03-28T13:20:53.437363+0000","last_undegraded":"2024-03-28T13:20:53.437363+0000","last_fullsized":"2024-03-28T13:20:53.437363+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T18:42:10.069121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.450743+0000","last_change":"2024-03-28T13:20:53.448750+0000","last_active":"2024-03-28T13:20:53.450743+0000","last_peered":"2024-03-28T13:20:53.450743+0000","last_clean":"2024-03-28T13:20:53.450743+0000","last_became_active":"2024-03-28T13:20:53.448091+0000","last_became_peered":"2024-03-28T13:20:53.448091+0000","last_unstale":"2024-03-28T13:20:53.450743+0000","last_undegraded":"2024-03-28T13:20:53.450743+0000","last_fullsized":"2024-03-28T13:20:53.450743+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:27:15.147525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.445206+0000","last_change":"2024-03-28T13:20:52.420196+0000","last_active":"2024-03-28T13:20:53.445206+0000","last_peered":"2024-03-28T13:20:53.445206+0000","last_clean":"2024-03-28T13:20:53.445206+0000","last_became_active":"2024-03-28T13:20:52.419420+0000","last_became_peered":"2024-03-28T13:20:52.419420+0000","last_unstale":"2024-03-28T13:20:53.445206+0000","last_undegraded":"2024-03-28T13:20:53.445206+0000","last_fullsized":"2024-03-28T13:20:53.445206+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T21:22:54.014282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-03-28T13:20:53.474107+0000","last_change":"2024-03-28T13:20:53.468311+0000","last_active":"2024-03-28T13:20:53.474107+0000","last_peered":"2024-03-28T13:20:53.474107+0000","last_clean":"2024-03-28T13:20:53.474107+0000","last_became_active":"2024-03-28T13:20:53.468000+0000","last_became_peered":"2024-03-28T13:20:53.468000+0000","last_unstale":"2024-03-28T13:20:53.474107+0000","last_undegraded":"2024-03-28T13:20:53.474107+0000","last_fullsized":"2024-03-28T13:20:53.474107+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:52.396515+0000","last_clean_scrub_stamp":"2024-03-28T13:20:52.396515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:16:26.602338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.809073+0000","last_change":"2024-03-28T13:20:52.440290+0000","last_active":"2024-03-28T13:20:54.809073+0000","last_peered":"2024-03-28T13:20:54.809073+0000","last_clean":"2024-03-28T13:20:54.809073+0000","last_became_active":"2024-03-28T13:20:52.439943+0000","last_became_peered":"2024-03-28T13:20:52.439943+0000","last_unstale":"2024-03-28T13:20:54.809073+0000","last_undegraded":"2024-03-28T13:20:54.809073+0000","last_fullsized":"2024-03-28T13:20:54.809073+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":116,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:20:51.381164+0000","last_clean_scrub_stamp":"2024-03-28T13:20:51.381164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T20:07:59.665654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":349,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-28T13:20:54.807157+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:20:54.807157+0000","last_peered":"2024-03-28T13:20:54.807157+0000","last_clean":"2024-03-28T13:20:54.807157+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:20:54.807157+0000","last_undegraded":"2024-03-28T13:20:54.807157+0000","last_fullsized":"2024-03-28T13:20:54.807157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476879,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1454080,"data_stored":632097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476877,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32612,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339228,"statfs":{"total":96636764160,"available":96603369472,"internally_reserved":0,"allocated":1466368,"data_stored":632735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476877,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2048000,"data_stored":1222465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1462272,"data_stored":632749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509580,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1454080,"data_stored":632097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509581,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33188,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338652,"statfs":{"total":96636764160,"available":96602779648,"internally_reserved":0,"allocated":2056192,"data_stored":1222557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509581,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1454080,"data_stored":632097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542286,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1486848,"data_stored":640133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:20:58.507 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-28T13:20:58.507 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-28T13:20:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.771+0000 7fe7f9c00640 1 Processor -- start 2024-03-28T13:20:58.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.771+0000 7fe7f9c00640 1 -- start start 2024-03-28T13:20:58.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.773+0000 7fe7f9c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7f40d1380 0x7fe7f40d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.773+0000 7fe7f9c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe7f40d3a70 0x7fe7f40d3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.773+0000 7fe7f9c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f40d2fd0 con 0x7fe7f415e420 2024-03-28T13:20:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.773+0000 7fe7f9c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f40cfa70 con 0x7fe7f40d1380 2024-03-28T13:20:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.773+0000 7fe7f9c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f40cf790 con 0x7fe7f40d3a70 2024-03-28T13:20:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.773+0000 7fe7f3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7f40d1380 0x7fe7f40d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.773+0000 7fe7f3400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe7f40d1380 0x7fe7f40d1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41898/0 (socket says 172.21.15.19:41898) 2024-03-28T13:20:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.773+0000 7fe7f3400640 1 -- 172.21.15.19:0/3843005458 learned_addr learned my addr 172.21.15.19:0/3843005458 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.773+0000 7fe7f9200640 1 --2- 172.21.15.19:0/3843005458 >> v2:172.21.15.188:3300/0 conn(0x7fe7f40d3a70 0x7fe7f40d3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:58.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.773+0000 7fe7f3400640 1 -- 172.21.15.19:0/3843005458 >> v2:172.21.15.188:3300/0 conn(0x7fe7f40d3a70 msgr2=0x7fe7f40d3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:58.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.773+0000 7fe7f3400640 1 --2- 172.21.15.19:0/3843005458 >> v2:172.21.15.188:3300/0 conn(0x7fe7f40d3a70 0x7fe7f40d3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:58.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.773+0000 7fe7f3400640 1 -- 172.21.15.19:0/3843005458 >> v1:172.21.15.188:6789/0 conn(0x7fe7f415e420 legacy=0x7fe7f40b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:58.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.774+0000 7fe7f3400640 1 -- 172.21.15.19:0/3843005458 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f40d3270 con 0x7fe7f40d1380 2024-03-28T13:20:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.775+0000 7fe7f3400640 1 --2- 172.21.15.19:0/3843005458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f40d1380 0x7fe7f40d1810 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40d31a0 tx=0x7fe7e002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347af5971737fbc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.775+0000 7fe7f2a00640 1 -- 172.21.15.19:0/3843005458 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0034070 con 0x7fe7f40d1380 2024-03-28T13:20:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.775+0000 7fe7f2a00640 1 -- 172.21.15.19:0/3843005458 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e0030860 con 0x7fe7f40d1380 2024-03-28T13:20:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.775+0000 7fe7f2a00640 1 -- 172.21.15.19:0/3843005458 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e0030c30 con 0x7fe7f40d1380 2024-03-28T13:20:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.776+0000 7fe7f9c00640 1 -- 172.21.15.19:0/3843005458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f40d1380 msgr2=0x7fe7f40d1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.776+0000 7fe7f9c00640 1 --2- 172.21.15.19:0/3843005458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f40d1380 0x7fe7f40d1810 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40d31a0 tx=0x7fe7e002b9a0 comp rx=0 tx=0).stop 2024-03-28T13:20:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.776+0000 7fe7f9c00640 1 -- 172.21.15.19:0/3843005458 shutdown_connections 2024-03-28T13:20:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.776+0000 7fe7f9c00640 1 --2- 172.21.15.19:0/3843005458 >> v2:172.21.15.188:3300/0 conn(0x7fe7f40d3a70 0x7fe7f40d3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.776+0000 7fe7f9c00640 1 --2- 172.21.15.19:0/3843005458 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f40d1380 0x7fe7f40d1810 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.776+0000 7fe7f9c00640 1 -- 172.21.15.19:0/3843005458 >> 172.21.15.19:0/3843005458 conn(0x7fe7f4067670 msgr2=0x7fe7f4067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.776+0000 7fe7f9c00640 1 -- 172.21.15.19:0/3843005458 shutdown_connections 2024-03-28T13:20:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.776+0000 7fe7f9c00640 1 -- 172.21.15.19:0/3843005458 wait complete. 2024-03-28T13:20:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.778+0000 7fe7f9c00640 1 Processor -- start 2024-03-28T13:20:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.778+0000 7fe7f9c00640 1 -- start start 2024-03-28T13:20:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.779+0000 7fe7f9c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7f40d1380 0x7fe7f40cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.779+0000 7fe7f9c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7f40d3a70 0x7fe7f414d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.779+0000 7fe7f9c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f415e420 0x7fe7f414d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.779+0000 7fe7f9c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f40cf420 con 0x7fe7f415e420 2024-03-28T13:20:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.779+0000 7fe7f9c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f415e800 con 0x7fe7f40d3a70 2024-03-28T13:20:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.779+0000 7fe7f9c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f40b3630 con 0x7fe7f40d1380 2024-03-28T13:20:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.779+0000 7fe7f9200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f415e420 0x7fe7f414d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.779+0000 7fe7f9200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f415e420 0x7fe7f414d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41910/0 (socket says 172.21.15.19:41910) 2024-03-28T13:20:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.779+0000 7fe7f9200640 1 -- 172.21.15.19:0/48415596 learned_addr learned my addr 172.21.15.19:0/48415596 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.779+0000 7fe7f3e00640 1 --2- 172.21.15.19:0/48415596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7f40d1380 0x7fe7f40cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.779+0000 7fe7f3400640 1 --2- 172.21.15.19:0/48415596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7f40d3a70 0x7fe7f414d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.779+0000 7fe7f9200640 1 -- 172.21.15.19:0/48415596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7f40d1380 msgr2=0x7fe7f40cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.779+0000 7fe7f9200640 1 --2- 172.21.15.19:0/48415596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7f40d1380 0x7fe7f40cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.779+0000 7fe7f9200640 1 -- 172.21.15.19:0/48415596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7f40d3a70 msgr2=0x7fe7f414d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.779+0000 7fe7f9200640 1 --2- 172.21.15.19:0/48415596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7f40d3a70 0x7fe7f414d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.780+0000 7fe7f9200640 1 -- 172.21.15.19:0/48415596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f415fac0 con 0x7fe7f415e420 2024-03-28T13:20:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.780+0000 7fe7f9200640 1 --2- 172.21.15.19:0/48415596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f415e420 0x7fe7f414d920 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec007e40 tx=0x7fe7ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.780+0000 7fe7ebe00640 1 -- 172.21.15.19:0/48415596 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec014620 con 0x7fe7f415e420 2024-03-28T13:20:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.780+0000 7fe7ebe00640 1 -- 172.21.15.19:0/48415596 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ec019070 con 0x7fe7f415e420 2024-03-28T13:20:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.780+0000 7fe7ebe00640 1 -- 172.21.15.19:0/48415596 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec014a20 con 0x7fe7f415e420 2024-03-28T13:20:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.780+0000 7fe7f9c00640 1 -- 172.21.15.19:0/48415596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f415fda0 con 0x7fe7f415e420 2024-03-28T13:20:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.780+0000 7fe7f9c00640 1 -- 172.21.15.19:0/48415596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f4160360 con 0x7fe7f415e420 2024-03-28T13:20:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.781+0000 7fe7f9c00640 1 -- 172.21.15.19:0/48415596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f4075940 con 0x7fe7f415e420 2024-03-28T13:20:58.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.783+0000 7fe7ebe00640 1 -- 172.21.15.19:0/48415596 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe7ec014bc0 con 0x7fe7f415e420 2024-03-28T13:20:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.784+0000 7fe7ebe00640 1 --2- 172.21.15.19:0/48415596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe7d406c530 0x7fe7d406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.784+0000 7fe7ebe00640 1 -- 172.21.15.19:0/48415596 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7fe7ec08c110 con 0x7fe7f415e420 2024-03-28T13:20:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.785+0000 7fe7f3e00640 1 --2- 172.21.15.19:0/48415596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe7d406c530 0x7fe7d406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.786+0000 7fe7f3e00640 1 --2- 172.21.15.19:0/48415596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe7d406c530 0x7fe7d406e9f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4006fd0 tx=0x7fe7e4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:58.788+0000 7fe7ebe00640 1 -- 172.21.15.19:0/48415596 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe7ec05a600 con 0x7fe7f415e420 2024-03-28T13:20:59.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.066+0000 7fe7f9c00640 1 -- 172.21.15.19:0/48415596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe7f4067c60 con 0x7fe7f415e420 2024-03-28T13:20:59.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.071+0000 7fe7ebe00640 1 -- 172.21.15.19:0/48415596 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7fe7ec059f00 con 0x7fe7f415e420 2024-03-28T13:20:59.071 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-03-28T13:20:59.071 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQDUawVmROa0BRAAO7qrATU2j2hVyeNoyJY//Q== 2024-03-28T13:20:59.071 INFO:teuthology.orchestra.run.smithi019.stdout: caps mds = "allow" 2024-03-28T13:20:59.071 INFO:teuthology.orchestra.run.smithi019.stdout: caps mon = "allow r" 2024-03-28T13:20:59.071 INFO:teuthology.orchestra.run.smithi019.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-28T13:20:59.072 INFO:teuthology.orchestra.run.smithi019.stderr:updated caps for client.0 2024-03-28T13:20:59.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.074+0000 7fe7f9c00640 1 -- 172.21.15.19:0/48415596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe7d406c530 msgr2=0x7fe7d406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:59.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.074+0000 7fe7f9c00640 1 --2- 172.21.15.19:0/48415596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe7d406c530 0x7fe7d406e9f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4006fd0 tx=0x7fe7e4008040 comp rx=0 tx=0).stop 2024-03-28T13:20:59.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.074+0000 7fe7f9c00640 1 -- 172.21.15.19:0/48415596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f415e420 msgr2=0x7fe7f414d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.074+0000 7fe7f9c00640 1 --2- 172.21.15.19:0/48415596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f415e420 0x7fe7f414d920 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec007e40 tx=0x7fe7ec00e4b0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.074+0000 7fe7f9c00640 1 -- 172.21.15.19:0/48415596 shutdown_connections 2024-03-28T13:20:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.074+0000 7fe7f9c00640 1 --2- 172.21.15.19:0/48415596 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe7d406c530 0x7fe7d406e9f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.074+0000 7fe7f9c00640 1 --2- 172.21.15.19:0/48415596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe7f415e420 0x7fe7f414d920 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.075+0000 7fe7f9c00640 1 --2- 172.21.15.19:0/48415596 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7f40d3a70 0x7fe7f414d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.075+0000 7fe7f9c00640 1 --2- 172.21.15.19:0/48415596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7f40d1380 0x7fe7f40cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.075+0000 7fe7f9c00640 1 -- 172.21.15.19:0/48415596 >> 172.21.15.19:0/48415596 conn(0x7fe7f4067670 msgr2=0x7fe7f415a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.075+0000 7fe7f9c00640 1 -- 172.21.15.19:0/48415596 shutdown_connections 2024-03-28T13:20:59.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.075+0000 7fe7f9c00640 1 -- 172.21.15.19:0/48415596 wait complete. 2024-03-28T13:20:59.094 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:20:59.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.321+0000 7fb3a8a00640 1 Processor -- start 2024-03-28T13:20:59.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.321+0000 7fb3a8a00640 1 -- start start 2024-03-28T13:20:59.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.322+0000 7fb3a8a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb3a41442c0 0x7fb3a40ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:59.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.322+0000 7fb3a8a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb3a40cefe0 0x7fb3a40cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:59.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.323+0000 7fb3a8a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3a40d00d0 0x7fb3a406e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:59.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.323+0000 7fb3a8a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb3a4071d20 con 0x7fb3a41442c0 2024-03-28T13:20:59.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.323+0000 7fb3a8a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb3a40cf420 con 0x7fb3a40cefe0 2024-03-28T13:20:59.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.323+0000 7fb3a8a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a4071460 con 0x7fb3a40d00d0 2024-03-28T13:20:59.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.323+0000 7fb3a2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb3a41442c0 0x7fb3a40ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:59.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.323+0000 7fb3a2800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb3a41442c0 0x7fb3a40ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41920/0 (socket says 172.21.15.19:41920) 2024-03-28T13:20:59.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.323+0000 7fb3a2800640 1 -- 172.21.15.19:0/1702005768 learned_addr learned my addr 172.21.15.19:0/1702005768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:59.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.323+0000 7fb3a3200640 1 --2- 172.21.15.19:0/1702005768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3a40d00d0 0x7fb3a406e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:59.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.323+0000 7fb39be00640 1 --2- 172.21.15.19:0/1702005768 >> v2:172.21.15.188:3300/0 conn(0x7fb3a40cefe0 0x7fb3a40cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:59.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.324+0000 7fb3a2800640 1 -- 172.21.15.19:0/1702005768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3a40d00d0 msgr2=0x7fb3a406e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:59.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.324+0000 7fb3a2800640 1 --2- 172.21.15.19:0/1702005768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3a40d00d0 0x7fb3a406e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.324+0000 7fb3a2800640 1 -- 172.21.15.19:0/1702005768 >> v2:172.21.15.188:3300/0 conn(0x7fb3a40cefe0 msgr2=0x7fb3a40cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:20:59.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.324+0000 7fb3a2800640 1 --2- 172.21.15.19:0/1702005768 >> v2:172.21.15.188:3300/0 conn(0x7fb3a40cefe0 0x7fb3a40cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.324+0000 7fb3a2800640 1 -- 172.21.15.19:0/1702005768 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a40721b0 con 0x7fb3a41442c0 2024-03-28T13:20:59.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.325+0000 7fb3a2800640 1 --2- 172.21.15.19:0/1702005768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a41442c0 0x7fb3a40ceaa0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40b2c60 tx=0x7fb39402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2177ba99f119227f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:59.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.325+0000 7fb3a1e00640 1 -- 172.21.15.19:0/1702005768 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394034070 con 0x7fb3a41442c0 2024-03-28T13:20:59.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.325+0000 7fb3a1e00640 1 -- 172.21.15.19:0/1702005768 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb394030860 con 0x7fb3a41442c0 2024-03-28T13:20:59.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.325+0000 7fb3a1e00640 1 -- 172.21.15.19:0/1702005768 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb394030c30 con 0x7fb3a41442c0 2024-03-28T13:20:59.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.326+0000 7fb3a8a00640 1 -- 172.21.15.19:0/1702005768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a41442c0 msgr2=0x7fb3a40ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:59.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.326+0000 7fb3a8a00640 1 --2- 172.21.15.19:0/1702005768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a41442c0 0x7fb3a40ceaa0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40b2c60 tx=0x7fb39402b9a0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.326+0000 7fb3a8a00640 1 -- 172.21.15.19:0/1702005768 shutdown_connections 2024-03-28T13:20:59.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.326+0000 7fb3a8a00640 1 --2- 172.21.15.19:0/1702005768 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3a40d00d0 0x7fb3a406e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.326+0000 7fb3a8a00640 1 --2- 172.21.15.19:0/1702005768 >> v2:172.21.15.188:3300/0 conn(0x7fb3a40cefe0 0x7fb3a40cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.326+0000 7fb3a8a00640 1 --2- 172.21.15.19:0/1702005768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a41442c0 0x7fb3a40ceaa0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.326+0000 7fb3a8a00640 1 -- 172.21.15.19:0/1702005768 >> 172.21.15.19:0/1702005768 conn(0x7fb3a40ccad0 msgr2=0x7fb3a40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:59.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.326+0000 7fb3a8a00640 1 -- 172.21.15.19:0/1702005768 shutdown_connections 2024-03-28T13:20:59.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.326+0000 7fb3a8a00640 1 -- 172.21.15.19:0/1702005768 wait complete. 2024-03-28T13:20:59.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.328+0000 7fb3a8a00640 1 Processor -- start 2024-03-28T13:20:59.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.328+0000 7fb3a8a00640 1 -- start start 2024-03-28T13:20:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.330+0000 7fb3a8a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3a40cefe0 0x7fb3a4157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.330+0000 7fb3a8a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3a40d00d0 0x7fb3a413a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.330+0000 7fb3a8a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a41442c0 0x7fb3a415b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.330+0000 7fb3a8a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a40d0770 con 0x7fb3a41442c0 2024-03-28T13:20:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.330+0000 7fb3a8a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a40719b0 con 0x7fb3a40d00d0 2024-03-28T13:20:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.330+0000 7fb3a8a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a41446a0 con 0x7fb3a40cefe0 2024-03-28T13:20:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.330+0000 7fb3a3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a41442c0 0x7fb3a415b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.330+0000 7fb3a3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a41442c0 0x7fb3a415b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41930/0 (socket says 172.21.15.19:41930) 2024-03-28T13:20:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.330+0000 7fb3a3200640 1 -- 172.21.15.19:0/2520744312 learned_addr learned my addr 172.21.15.19:0/2520744312 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:20:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.330+0000 7fb39be00640 1 --2- 172.21.15.19:0/2520744312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3a40d00d0 0x7fb3a413a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:59.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.330+0000 7fb3a2800640 1 --2- 172.21.15.19:0/2520744312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3a40cefe0 0x7fb3a4157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:59.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.331+0000 7fb3a3200640 1 -- 172.21.15.19:0/2520744312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3a40cefe0 msgr2=0x7fb3a4157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.331+0000 7fb3a3200640 1 --2- 172.21.15.19:0/2520744312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3a40cefe0 0x7fb3a4157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.331+0000 7fb3a3200640 1 -- 172.21.15.19:0/2520744312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3a40d00d0 msgr2=0x7fb3a413a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.331+0000 7fb3a3200640 1 --2- 172.21.15.19:0/2520744312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3a40d00d0 0x7fb3a413a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.331+0000 7fb3a3200640 1 -- 172.21.15.19:0/2520744312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a4158750 con 0x7fb3a41442c0 2024-03-28T13:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.331+0000 7fb3a3200640 1 --2- 172.21.15.19:0/2520744312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a41442c0 0x7fb3a415b720 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fb39c007e40 tx=0x7fb39c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.331+0000 7fb39aa00640 1 -- 172.21.15.19:0/2520744312 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c014620 con 0x7fb3a41442c0 2024-03-28T13:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.331+0000 7fb39aa00640 1 -- 172.21.15.19:0/2520744312 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb39c019070 con 0x7fb3a41442c0 2024-03-28T13:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.331+0000 7fb3a8a00640 1 -- 172.21.15.19:0/2520744312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a4158a30 con 0x7fb3a41442c0 2024-03-28T13:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.331+0000 7fb39aa00640 1 -- 172.21.15.19:0/2520744312 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c014a20 con 0x7fb3a41442c0 2024-03-28T13:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.332+0000 7fb3a8a00640 1 -- 172.21.15.19:0/2520744312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a4158ff0 con 0x7fb3a41442c0 2024-03-28T13:20:59.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.333+0000 7fb3a8a00640 1 -- 172.21.15.19:0/2520744312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a4075940 con 0x7fb3a41442c0 2024-03-28T13:20:59.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.334+0000 7fb39aa00640 1 -- 172.21.15.19:0/2520744312 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb39c014bc0 con 0x7fb3a41442c0 2024-03-28T13:20:59.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.335+0000 7fb39aa00640 1 --2- 172.21.15.19:0/2520744312 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb38006c530 0x7fb38006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:20:59.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.335+0000 7fb39aa00640 1 -- 172.21.15.19:0/2520744312 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7fb39c08bfc0 con 0x7fb3a41442c0 2024-03-28T13:20:59.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.335+0000 7fb3a2800640 1 --2- 172.21.15.19:0/2520744312 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb38006c530 0x7fb38006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:20:59.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.337+0000 7fb3a2800640 1 --2- 172.21.15.19:0/2520744312 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb38006c530 0x7fb38006e9f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb394036000 tx=0x7fb39403b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:20:59.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.340+0000 7fb39aa00640 1 -- 172.21.15.19:0/2520744312 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb39c05a540 con 0x7fb3a41442c0 2024-03-28T13:20:59.379 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-03-28T13:20:59.379+0000 7f18b8000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:20:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.584+0000 7fb3a8a00640 1 -- 172.21.15.19:0/2520744312 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3a40b2b30 con 0x7fb3a41442c0 2024-03-28T13:20:59.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.585+0000 7fb39aa00640 1 -- 172.21.15.19:0/2520744312 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5186 (secure 0 0 0) 0x7fb39c059e40 con 0x7fb3a41442c0 2024-03-28T13:20:59.586 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:20:59.586 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3823671990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3823671990},{"type":"v1","addr":"172.21.15.188:6837","nonce":3823671990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/230519675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":230519675},{"type":"v1","addr":"172.21.15.19:6834","nonce":230519675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/3930967142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3930967142},{"type":"v1","addr":"172.21.15.19:6835","nonce":3930967142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:20:52.901269+0000","modified":"2024-03-28T13:20:55.973314+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8703},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8703":{"gid":8703,"name":"d","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6836/2088538359","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":2088538359},{"type":"v1","addr":"172.21.15.188:6836","nonce":2088538359}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8703,"qdb_cluster":[8703]},"id":8}]} 2024-03-28T13:20:59.587 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 99 2024-03-28T13:20:59.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.587+0000 7fb3a8a00640 1 -- 172.21.15.19:0/2520744312 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb38006c530 msgr2=0x7fb38006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:59.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.588+0000 7fb3a8a00640 1 --2- 172.21.15.19:0/2520744312 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb38006c530 0x7fb38006e9f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb394036000 tx=0x7fb39403b040 comp rx=0 tx=0).stop 2024-03-28T13:20:59.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.588+0000 7fb3a8a00640 1 -- 172.21.15.19:0/2520744312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a41442c0 msgr2=0x7fb3a415b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:20:59.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.588+0000 7fb3a8a00640 1 --2- 172.21.15.19:0/2520744312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a41442c0 0x7fb3a415b720 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fb39c007e40 tx=0x7fb39c00e4b0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.588+0000 7fb3a8a00640 1 -- 172.21.15.19:0/2520744312 shutdown_connections 2024-03-28T13:20:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.588+0000 7fb3a8a00640 1 --2- 172.21.15.19:0/2520744312 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb38006c530 0x7fb38006e9f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.588+0000 7fb3a8a00640 1 --2- 172.21.15.19:0/2520744312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3a41442c0 0x7fb3a415b720 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.588+0000 7fb3a8a00640 1 --2- 172.21.15.19:0/2520744312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3a40d00d0 0x7fb3a413a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.588+0000 7fb3a8a00640 1 --2- 172.21.15.19:0/2520744312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3a40cefe0 0x7fb3a4157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:20:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.588+0000 7fb3a8a00640 1 -- 172.21.15.19:0/2520744312 >> 172.21.15.19:0/2520744312 conn(0x7fb3a40ccad0 msgr2=0x7fb3a406ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:20:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.588+0000 7fb3a8a00640 1 -- 172.21.15.19:0/2520744312 shutdown_connections 2024-03-28T13:20:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:20:59.588+0000 7fb3a8a00640 1 -- 172.21.15.19:0/2520744312 wait complete. 2024-03-28T13:21:00.010 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:20:52.901269+0000', 'modified': '2024-03-28T13:20:55.973314+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8703}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8703': {'gid': 8703, 'name': 'd', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6836/2088538359', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 2088538359}, {'type': 'v1', 'addr': '172.21.15.188:6836', 'nonce': 2088538359}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8703, 'qdb_cluster': [8703]} 2024-03-28T13:21:00.010 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T13:21:00.011 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-03-28T13:21:00.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.244+0000 7f5395800640 1 Processor -- start 2024-03-28T13:21:00.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.244+0000 7f5395800640 1 -- start start 2024-03-28T13:21:00.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.246+0000 7f5395800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5390153580 0x7f53900d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.246+0000 7f5395800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f53900af4b0 0x7f53901600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.246+0000 7f5395800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53900afab0 con 0x7f53900d07c0 2024-03-28T13:21:00.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.246+0000 7f5395800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f53900d0c30 con 0x7f53900af4b0 2024-03-28T13:21:00.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.246+0000 7f5395800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f539015cc80 con 0x7f5390153580 2024-03-28T13:21:00.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.246+0000 7f538e000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f53900d07c0 0x7f53900aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53720/0 (socket says 172.21.15.19:53720) 2024-03-28T13:21:00.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.246+0000 7f538e000640 1 -- 172.21.15.19:0/2862418952 learned_addr learned my addr 172.21.15.19:0/2862418952 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:00.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.246+0000 7f538f400640 1 --2- 172.21.15.19:0/2862418952 >> v2:172.21.15.19:3300/0 conn(0x7f53900af4b0 0x7f53901600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.247+0000 7f538ea00640 1 --2- 172.21.15.19:0/2862418952 >> v2:172.21.15.188:3300/0 conn(0x7f5390153580 0x7f53900d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.247+0000 7f538d600640 1 -- 172.21.15.19:0/2862418952 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4179331648 0 0) 0x7f53900afab0 con 0x7f53900d07c0 2024-03-28T13:21:00.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.247+0000 7f538f400640 1 -- 172.21.15.19:0/2862418952 >> v2:172.21.15.188:3300/0 conn(0x7f5390153580 msgr2=0x7f53900d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:00.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.247+0000 7f538f400640 1 --2- 172.21.15.19:0/2862418952 >> v2:172.21.15.188:3300/0 conn(0x7f5390153580 0x7f53900d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.247+0000 7f538f400640 1 -- 172.21.15.19:0/2862418952 >> v1:172.21.15.19:6789/0 conn(0x7f53900d07c0 legacy=0x7f53900aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.247+0000 7f538f400640 1 -- 172.21.15.19:0/2862418952 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539015da00 con 0x7f53900af4b0 2024-03-28T13:21:00.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.248+0000 7f538f400640 1 --2- 172.21.15.19:0/2862418952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900af4b0 0x7f53901600d0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f53900b5270 tx=0x7f538802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc02b5e05c105af server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:00.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.248+0000 7f538d600640 1 -- 172.21.15.19:0/2862418952 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388039070 con 0x7f53900af4b0 2024-03-28T13:21:00.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.248+0000 7f538d600640 1 -- 172.21.15.19:0/2862418952 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538802cbf0 con 0x7f53900af4b0 2024-03-28T13:21:00.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.248+0000 7f538d600640 1 -- 172.21.15.19:0/2862418952 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388034440 con 0x7f53900af4b0 2024-03-28T13:21:00.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.249+0000 7f5395800640 1 -- 172.21.15.19:0/2862418952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900af4b0 msgr2=0x7f53901600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.249+0000 7f5395800640 1 --2- 172.21.15.19:0/2862418952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900af4b0 0x7f53901600d0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f53900b5270 tx=0x7f538802be40 comp rx=0 tx=0).stop 2024-03-28T13:21:00.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.249+0000 7f5395800640 1 -- 172.21.15.19:0/2862418952 shutdown_connections 2024-03-28T13:21:00.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.249+0000 7f5395800640 1 --2- 172.21.15.19:0/2862418952 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900af4b0 0x7f53901600d0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.249+0000 7f5395800640 1 --2- 172.21.15.19:0/2862418952 >> v2:172.21.15.188:3300/0 conn(0x7f5390153580 0x7f53900d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.249+0000 7f5395800640 1 -- 172.21.15.19:0/2862418952 >> 172.21.15.19:0/2862418952 conn(0x7f539006b750 msgr2=0x7f539006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:00.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.249+0000 7f5395800640 1 -- 172.21.15.19:0/2862418952 shutdown_connections 2024-03-28T13:21:00.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.249+0000 7f5395800640 1 -- 172.21.15.19:0/2862418952 wait complete. 2024-03-28T13:21:00.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.251+0000 7f5395800640 1 Processor -- start 2024-03-28T13:21:00.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.251+0000 7f5395800640 1 -- start start 2024-03-28T13:21:00.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.252+0000 7f5395800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900af4b0 0x7f53901f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.252+0000 7f5395800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53900d07c0 0x7f539015e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.252+0000 7f5395800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5390153580 0x7f539015ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.252+0000 7f5395800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f539015d4e0 con 0x7f53900af4b0 2024-03-28T13:21:00.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.252+0000 7f5395800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f539015d200 con 0x7f5390153580 2024-03-28T13:21:00.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.252+0000 7f5395800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5390153960 con 0x7f53900d07c0 2024-03-28T13:21:00.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.253+0000 7f538ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900af4b0 0x7f53901f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.253+0000 7f538ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900af4b0 0x7f53901f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41948/0 (socket says 172.21.15.19:41948) 2024-03-28T13:21:00.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.253+0000 7f538ea00640 1 -- 172.21.15.19:0/416380115 learned_addr learned my addr 172.21.15.19:0/416380115 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:00.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.253+0000 7f538e000640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53900d07c0 0x7f539015e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.253+0000 7f538f400640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5390153580 0x7f539015ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.253+0000 7f538ea00640 1 -- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53900d07c0 msgr2=0x7f539015e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.253+0000 7f538ea00640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53900d07c0 0x7f539015e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.253+0000 7f538ea00640 1 -- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5390153580 msgr2=0x7f539015ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.253+0000 7f538ea00640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5390153580 0x7f539015ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.253+0000 7f538ea00640 1 -- 172.21.15.19:0/416380115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539009cc40 con 0x7f53900af4b0 2024-03-28T13:21:00.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.253+0000 7f538e000640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53900d07c0 0x7f539015e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:00.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.253+0000 7f538f400640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5390153580 0x7f539015ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:00.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.254+0000 7f538ea00640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900af4b0 0x7f53901f5050 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f538000d800 tx=0x7f538000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:00.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.254+0000 7f5386a00640 1 -- 172.21.15.19:0/416380115 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538000b6c0 con 0x7f53900af4b0 2024-03-28T13:21:00.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.254+0000 7f5386a00640 1 -- 172.21.15.19:0/416380115 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5380019070 con 0x7f53900af4b0 2024-03-28T13:21:00.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.254+0000 7f5386a00640 1 -- 172.21.15.19:0/416380115 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538000ba80 con 0x7f53900af4b0 2024-03-28T13:21:00.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.254+0000 7f5395800640 1 -- 172.21.15.19:0/416380115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53901f5590 con 0x7f53900af4b0 2024-03-28T13:21:00.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.254+0000 7f5395800640 1 -- 172.21.15.19:0/416380115 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539009d1a0 con 0x7f53900af4b0 2024-03-28T13:21:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.256+0000 7f5386a00640 1 -- 172.21.15.19:0/416380115 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f538000bc20 con 0x7f53900af4b0 2024-03-28T13:21:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.258+0000 7f5386a00640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f536c06c530 0x7f536c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.258+0000 7f5386a00640 1 -- 172.21.15.19:0/416380115 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f538008bbb0 con 0x7f53900af4b0 2024-03-28T13:21:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.259+0000 7f538e000640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f536c06c530 0x7f536c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.259+0000 7f539856a740 1 Processor -- start 2024-03-28T13:21:00.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.259+0000 7f539856a740 1 -- start start 2024-03-28T13:21:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.260+0000 7f539856a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd90b50 0x55ca7dd8e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.260+0000 7f539856a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd48080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.260+0000 7f539856a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd485c0 0x55ca7dd72d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.260+0000 7f539856a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ca7dd0a920 con 0x55ca7dd70390 2024-03-28T13:21:00.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.260+0000 7f539856a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ca7dc5afa0 con 0x55ca7dd90b50 2024-03-28T13:21:00.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.260+0000 7f539856a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55ca7dcacf90 con 0x55ca7dd485c0 2024-03-28T13:21:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.260+0000 7f538e000640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f536c06c530 0x7f536c06e9f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5390150b80 tx=0x7f5378009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.260+0000 7f538e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd48080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.260+0000 7f538e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd48080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41956/0 (socket says 172.21.15.19:41956) 2024-03-28T13:21:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.260+0000 7f538e000640 1 -- 172.21.15.19:0/3229595100 learned_addr learned my addr 172.21.15.19:0/3229595100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.260+0000 7f538f400640 1 --2- 172.21.15.19:0/3229595100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd485c0 0x55ca7dd72d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.260+0000 7f538ea00640 1 --2- 172.21.15.19:0/3229595100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd90b50 0x55ca7dd8e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.261+0000 7f538e000640 1 -- 172.21.15.19:0/3229595100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd485c0 msgr2=0x55ca7dd72d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:00.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.261+0000 7f538e000640 1 --2- 172.21.15.19:0/3229595100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd485c0 0x55ca7dd72d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.261+0000 7f538e000640 1 -- 172.21.15.19:0/3229595100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd90b50 msgr2=0x55ca7dd8e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:00.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.261+0000 7f538e000640 1 --2- 172.21.15.19:0/3229595100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd90b50 0x55ca7dd8e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.261+0000 7f538e000640 1 -- 172.21.15.19:0/3229595100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca7d9c12d0 con 0x55ca7dd70390 2024-03-28T13:21:00.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.261+0000 7f538ea00640 1 --2- 172.21.15.19:0/3229595100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd90b50 0x55ca7dd8e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:21:00.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.261+0000 7f538e000640 1 --2- 172.21.15.19:0/3229595100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd48080 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f5378007390 tx=0x7f5378010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff308ca09c02e170 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:00.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.261+0000 7f537f400640 1 -- 172.21.15.19:0/3229595100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f537800d9a0 con 0x55ca7dd70390 2024-03-28T13:21:00.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.261+0000 7f537f400640 1 -- 172.21.15.19:0/3229595100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f537800db40 con 0x55ca7dd70390 2024-03-28T13:21:00.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.261+0000 7f537f400640 1 -- 172.21.15.19:0/3229595100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378020450 con 0x55ca7dd70390 2024-03-28T13:21:00.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.262+0000 7f539856a740 1 -- 172.21.15.19:0/3229595100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 msgr2=0x55ca7dd48080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.262+0000 7f539856a740 1 --2- 172.21.15.19:0/3229595100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd48080 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f5378007390 tx=0x7f5378010000 comp rx=0 tx=0).stop 2024-03-28T13:21:00.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.262+0000 7f539856a740 1 -- 172.21.15.19:0/3229595100 shutdown_connections 2024-03-28T13:21:00.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.262+0000 7f539856a740 1 --2- 172.21.15.19:0/3229595100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd485c0 0x55ca7dd72d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.262+0000 7f539856a740 1 --2- 172.21.15.19:0/3229595100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd48080 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.262+0000 7f539856a740 1 --2- 172.21.15.19:0/3229595100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd90b50 0x55ca7dd8e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.262+0000 7f539856a740 1 -- 172.21.15.19:0/3229595100 >> 172.21.15.19:0/3229595100 conn(0x55ca7dd451e0 msgr2=0x55ca7dd8dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.262+0000 7f539856a740 1 -- 172.21.15.19:0/3229595100 shutdown_connections 2024-03-28T13:21:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.262+0000 7f539856a740 1 -- 172.21.15.19:0/3229595100 wait complete. 2024-03-28T13:21:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.263+0000 7f539856a740 1 -- start start 2024-03-28T13:21:00.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.263+0000 7f539856a740 1 Processor -- start 2024-03-28T13:21:00.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.264+0000 7f539856a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd8bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.264+0000 7f539856a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd90b50 0x55ca7dd6b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.264+0000 7f539856a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd8ee30 0x55ca7dd8f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.264+0000 7f539856a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ca7dca8f40 con 0x55ca7dd70390 2024-03-28T13:21:00.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.264+0000 7f539856a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ca7dc55650 con 0x55ca7dd8ee30 2024-03-28T13:21:00.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.264+0000 7f539856a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55ca7dc68a10 con 0x55ca7dd90b50 2024-03-28T13:21:00.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.264+0000 7f539856a740 10 client.0 _ll_register_callbacks cb 0x55ca7dca7ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:21:00.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.265+0000 7f538ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd8bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.265+0000 7f538ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd8bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41962/0 (socket says 172.21.15.19:41962) 2024-03-28T13:21:00.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.265+0000 7f538ea00640 1 -- 172.21.15.19:0/127143975 learned_addr learned my addr 172.21.15.19:0/127143975 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.265+0000 7f538e000640 1 --2- 172.21.15.19:0/127143975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd90b50 0x55ca7dd6b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.265+0000 7f538f400640 1 --2- 172.21.15.19:0/127143975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd8ee30 0x55ca7dd8f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.265+0000 7f538ea00640 1 -- 172.21.15.19:0/127143975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd90b50 msgr2=0x55ca7dd6b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.265+0000 7f538ea00640 1 --2- 172.21.15.19:0/127143975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd90b50 0x55ca7dd6b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.265+0000 7f538ea00640 1 -- 172.21.15.19:0/127143975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd8ee30 msgr2=0x55ca7dd8f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.265+0000 7f538ea00640 1 --2- 172.21.15.19:0/127143975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd8ee30 0x55ca7dd8f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.265+0000 7f538ea00640 1 -- 172.21.15.19:0/127143975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca7da8b6b0 con 0x55ca7dd70390 2024-03-28T13:21:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.265+0000 7f538ea00640 1 --2- 172.21.15.19:0/127143975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd8bb80 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f5380059a50 tx=0x7f538005a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be098d2ed204acb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.265+0000 7f5373e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:21:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.266+0000 7f5373e00640 1 -- 172.21.15.19:0/127143975 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380056130 con 0x55ca7dd70390 2024-03-28T13:21:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.266+0000 7f5373e00640 1 -- 172.21.15.19:0/127143975 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f53800600b0 con 0x55ca7dd70390 2024-03-28T13:21:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.266+0000 7f539856a740 1 -- 172.21.15.19:0/127143975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f537800db40 con 0x55ca7dd70390 2024-03-28T13:21:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.266+0000 7f5373e00640 1 -- 172.21.15.19:0/127143975 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f53800562d0 con 0x55ca7dd70390 2024-03-28T13:21:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.266+0000 7f539856a740 10 client.8841 fetch_fsmap learned FSMap version 99 2024-03-28T13:21:00.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.266+0000 7f539856a740 1 -- 172.21.15.19:0/127143975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55ca7dd6caf0 con 0x55ca7dd70390 2024-03-28T13:21:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.266+0000 7f5373e00640 1 -- 172.21.15.19:0/127143975 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f537800db40 con 0x55ca7dd70390 2024-03-28T13:21:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.266+0000 7f5373e00640 1 -- 172.21.15.19:0/127143975 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7f5380013070 con 0x55ca7dd70390 2024-03-28T13:21:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.266+0000 7f539856a740 10 client.8841 fetch_fsmap finished waiting for FSMap version 99 2024-03-28T13:21:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.266+0000 7f539856a740 10 client.8841 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-28T13:21:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.266+0000 7f539856a740 20 client.8841 populate_metadata read hostname 'smithi019' 2024-03-28T13:21:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.266+0000 7f539856a740 1 --2- 172.21.15.19:0/127143975 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55ca7dd89880 0x55ca7ddb0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.266+0000 7f539856a740 4 client.8841 mds_command: new command op to 8703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:21:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.266+0000 7f539856a740 1 -- 172.21.15.19:0/127143975 --> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca7dcba4d0 con 0x55ca7dd89880 2024-03-28T13:21:00.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.267+0000 7f538e000640 1 --2- 172.21.15.19:0/127143975 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55ca7dd89880 0x55ca7ddb0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.268+0000 7f538e000640 1 --2- 172.21.15.19:0/127143975 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55ca7dd89880 0x55ca7ddb0a70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a7f30a05ea7e561 server_cookie=45f7ac20f6adf2e8 in_seq=0 out_seq=0 2024-03-28T13:21:00.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.268+0000 7f5373e00640 10 client.8841 ms_handle_connect on v2:172.21.15.188:6834/2088538359 2024-03-28T13:21:00.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.270+0000 7f5373e00640 1 -- 172.21.15.19:0/127143975 <== mds.0 v2:172.21.15.188:6834/2088538359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca7dcba4d0 con 0x55ca7dd89880 2024-03-28T13:21:00.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.270+0000 7f5373e00640 10 client.8841 handle_command_reply: tid=0 2024-03-28T13:21:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.270+0000 7f539856a740 1 client.8841 shutdown 2024-03-28T13:21:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.270+0000 7f539856a740 1 -- 172.21.15.19:0/127143975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 msgr2=0x55ca7dd8bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.270+0000 7f539856a740 1 --2- 172.21.15.19:0/127143975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd8bb80 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f5380059a50 tx=0x7f538005a300 comp rx=0 tx=0).stop 2024-03-28T13:21:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.270+0000 7f539856a740 1 -- 172.21.15.19:0/127143975 shutdown_connections 2024-03-28T13:21:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.270+0000 7f539856a740 1 --2- 172.21.15.19:0/127143975 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55ca7dd89880 0x55ca7ddb0a70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.270+0000 7f539856a740 1 --2- 172.21.15.19:0/127143975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd8ee30 0x55ca7dd8f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.270+0000 7f539856a740 1 --2- 172.21.15.19:0/127143975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd90b50 0x55ca7dd6b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.270+0000 7f539856a740 1 --2- 172.21.15.19:0/127143975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd8bb80 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.270+0000 7f5373e00640 0 client.8841 ms_handle_reset on v2:172.21.15.188:6834/2088538359 2024-03-28T13:21:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.270+0000 7f539856a740 1 -- 172.21.15.19:0/127143975 >> 172.21.15.19:0/127143975 conn(0x55ca7dd485c0 msgr2=0x55ca7dd91c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.270+0000 7f539856a740 1 -- 172.21.15.19:0/127143975 shutdown_connections 2024-03-28T13:21:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.270+0000 7f539856a740 1 -- 172.21.15.19:0/127143975 wait complete. 2024-03-28T13:21:00.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.271+0000 7f539856a740 20 client.8841 trim_cache size 0 max 16384 2024-03-28T13:21:00.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.290+0000 7f539856a740 1 Processor -- start 2024-03-28T13:21:00.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.291+0000 7f539856a740 1 -- start start 2024-03-28T13:21:00.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.291+0000 7f539856a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd5c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.291+0000 7f539856a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd8ee30 0x55ca7dd8dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.291+0000 7f539856a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd90b50 0x55ca7dd8e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.291+0000 7f539856a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5390155d20 con 0x55ca7dd70390 2024-03-28T13:21:00.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.291+0000 7f539856a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5390073510 con 0x55ca7dd90b50 2024-03-28T13:21:00.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.291+0000 7f539856a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5390072870 con 0x55ca7dd8ee30 2024-03-28T13:21:00.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.292+0000 7f538ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd5c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.292+0000 7f538ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd5c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41966/0 (socket says 172.21.15.19:41966) 2024-03-28T13:21:00.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.292+0000 7f538ea00640 1 -- 172.21.15.19:0/799214098 learned_addr learned my addr 172.21.15.19:0/799214098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:00.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.292+0000 7f538ea00640 1 -- 172.21.15.19:0/799214098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd8ee30 msgr2=0x55ca7dd8dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:00.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.292+0000 7f538e000640 1 --2- 172.21.15.19:0/799214098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd8ee30 0x55ca7dd8dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.292+0000 7f538ea00640 1 --2- 172.21.15.19:0/799214098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd8ee30 0x55ca7dd8dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.292+0000 7f538ea00640 1 -- 172.21.15.19:0/799214098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd90b50 msgr2=0x55ca7dd8e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:00.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.292+0000 7f538ea00640 1 --2- 172.21.15.19:0/799214098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd90b50 0x55ca7dd8e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.292+0000 7f538ea00640 1 -- 172.21.15.19:0/799214098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca7dd701c0 con 0x55ca7dd70390 2024-03-28T13:21:00.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.292+0000 7f538ea00640 1 --2- 172.21.15.19:0/799214098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd5c8c0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f5380000c00 tx=0x7f538005b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268201fe289a3838 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:00.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.293+0000 7f537ea00640 1 -- 172.21.15.19:0/799214098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380082070 con 0x55ca7dd70390 2024-03-28T13:21:00.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.293+0000 7f537ea00640 1 -- 172.21.15.19:0/799214098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5380058f10 con 0x55ca7dd70390 2024-03-28T13:21:00.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.293+0000 7f537ea00640 1 -- 172.21.15.19:0/799214098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5380056130 con 0x55ca7dd70390 2024-03-28T13:21:00.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.293+0000 7f539856a740 1 -- 172.21.15.19:0/799214098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 msgr2=0x55ca7dd5c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.293+0000 7f539856a740 1 --2- 172.21.15.19:0/799214098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd5c8c0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f5380000c00 tx=0x7f538005b000 comp rx=0 tx=0).stop 2024-03-28T13:21:00.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.293+0000 7f539856a740 1 -- 172.21.15.19:0/799214098 shutdown_connections 2024-03-28T13:21:00.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.293+0000 7f539856a740 1 --2- 172.21.15.19:0/799214098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd90b50 0x55ca7dd8e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.293+0000 7f539856a740 1 --2- 172.21.15.19:0/799214098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd8ee30 0x55ca7dd8dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.293+0000 7f539856a740 1 --2- 172.21.15.19:0/799214098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd70390 0x55ca7dd5c8c0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.293+0000 7f539856a740 1 -- 172.21.15.19:0/799214098 >> 172.21.15.19:0/799214098 conn(0x55ca7dd485c0 msgr2=0x55ca7dd91c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:00.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.293+0000 7f539856a740 1 -- 172.21.15.19:0/799214098 shutdown_connections 2024-03-28T13:21:00.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.293+0000 7f539856a740 1 -- 172.21.15.19:0/799214098 wait complete. 2024-03-28T13:21:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.294+0000 7f539856a740 1 -- start start 2024-03-28T13:21:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.294+0000 7f539856a740 1 Processor -- start 2024-03-28T13:21:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f539856a740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd70390 0x55ca7ddb4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f539856a740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd8ee30 0x55ca7de0fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f539856a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd90b50 0x55ca7de032b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f539856a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ca7dd6c530 con 0x55ca7dd90b50 2024-03-28T13:21:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f539856a740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5390072ec0 con 0x55ca7dd8ee30 2024-03-28T13:21:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f539856a740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55ca7dd88a00 con 0x55ca7dd70390 2024-03-28T13:21:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f539856a740 10 client.0 _ll_register_callbacks cb 0x7f538000bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:21:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f538f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd90b50 0x55ca7de032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f538f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd90b50 0x55ca7de032b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41974/0 (socket says 172.21.15.19:41974) 2024-03-28T13:21:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f538f400640 1 -- 172.21.15.19:0/1000833988 learned_addr learned my addr 172.21.15.19:0/1000833988 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f538f400640 1 -- 172.21.15.19:0/1000833988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd70390 msgr2=0x55ca7ddb4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:00.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f538f400640 1 --2- 172.21.15.19:0/1000833988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd70390 0x55ca7ddb4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f538f400640 1 -- 172.21.15.19:0/1000833988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd8ee30 msgr2=0x55ca7de0fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f538e000640 1 --2- 172.21.15.19:0/1000833988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd8ee30 0x55ca7de0fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f538f400640 1 --2- 172.21.15.19:0/1000833988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd8ee30 0x55ca7de0fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.296+0000 7f538f400640 1 -- 172.21.15.19:0/1000833988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca7dd8cfa0 con 0x55ca7dd90b50 2024-03-28T13:21:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.297+0000 7f538f400640 1 --2- 172.21.15.19:0/1000833988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd90b50 0x55ca7de032b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f5388009e80 tx=0x7f5388004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a51a57be348a67e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.297+0000 7f5373e00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:21:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.297+0000 7f5373e00640 1 -- 172.21.15.19:0/1000833988 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5388045030 con 0x55ca7dd90b50 2024-03-28T13:21:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.297+0000 7f5373e00640 1 -- 172.21.15.19:0/1000833988 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f5388003d90 con 0x55ca7dd90b50 2024-03-28T13:21:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.297+0000 7f539856a740 1 -- 172.21.15.19:0/1000833988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5380058f10 con 0x55ca7dd90b50 2024-03-28T13:21:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.297+0000 7f5373e00640 1 -- 172.21.15.19:0/1000833988 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f5388051050 con 0x55ca7dd90b50 2024-03-28T13:21:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.297+0000 7f539856a740 10 client.8847 fetch_fsmap learned FSMap version 99 2024-03-28T13:21:00.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.297+0000 7f539856a740 1 -- 172.21.15.19:0/1000833988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55ca7dd8e300 con 0x55ca7dd90b50 2024-03-28T13:21:00.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.298+0000 7f5373e00640 1 -- 172.21.15.19:0/1000833988 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5380058f10 con 0x55ca7dd90b50 2024-03-28T13:21:00.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.298+0000 7f5373e00640 1 -- 172.21.15.19:0/1000833988 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7f5388054020 con 0x55ca7dd90b50 2024-03-28T13:21:00.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.298+0000 7f539856a740 10 client.8847 fetch_fsmap finished waiting for FSMap version 99 2024-03-28T13:21:00.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.298+0000 7f539856a740 10 client.8847 resolve_mds: resolved name 'd' to daemon mds.d 2024-03-28T13:21:00.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.298+0000 7f539856a740 20 client.8847 populate_metadata read hostname 'smithi019' 2024-03-28T13:21:00.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.298+0000 7f539856a740 1 --2- 172.21.15.19:0/1000833988 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55ca7de0d2e0 0x55ca7dd69d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.298+0000 7f539856a740 4 client.8847 mds_command: new command op to 8703 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T13:21:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.298+0000 7f539856a740 1 -- 172.21.15.19:0/1000833988 --> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] -- command(tid 0: {"prefix": "status"}) -- 0x55ca7dd8c4b0 con 0x55ca7de0d2e0 2024-03-28T13:21:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.298+0000 7f538ea00640 1 --2- 172.21.15.19:0/1000833988 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55ca7de0d2e0 0x55ca7dd69d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.299+0000 7f538ea00640 1 --2- 172.21.15.19:0/1000833988 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55ca7de0d2e0 0x55ca7dd69d20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67ecc882350e2e6c server_cookie=987b95545125cfe1 in_seq=0 out_seq=0 2024-03-28T13:21:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.299+0000 7f5373e00640 10 client.8847 ms_handle_connect on v2:172.21.15.188:6834/2088538359 2024-03-28T13:21:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.300+0000 7f5373e00640 1 -- 172.21.15.19:0/1000833988 <== mds.0 v2:172.21.15.188:6834/2088538359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ca7dd8c4b0 con 0x55ca7de0d2e0 2024-03-28T13:21:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.300+0000 7f5373e00640 10 client.8847 handle_command_reply: tid=0 2024-03-28T13:21:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.300+0000 7f539856a740 1 client.8847 shutdown 2024-03-28T13:21:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.300+0000 7f539856a740 1 -- 172.21.15.19:0/1000833988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd90b50 msgr2=0x55ca7de032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.300+0000 7f539856a740 1 --2- 172.21.15.19:0/1000833988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd90b50 0x55ca7de032b0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f5388009e80 tx=0x7f5388004a40 comp rx=0 tx=0).stop 2024-03-28T13:21:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.300+0000 7f539856a740 1 -- 172.21.15.19:0/1000833988 shutdown_connections 2024-03-28T13:21:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.300+0000 7f539856a740 1 --2- 172.21.15.19:0/1000833988 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55ca7de0d2e0 0x55ca7dd69d20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.300+0000 7f539856a740 1 --2- 172.21.15.19:0/1000833988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ca7dd90b50 0x55ca7de032b0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.300+0000 7f539856a740 1 --2- 172.21.15.19:0/1000833988 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca7dd8ee30 0x55ca7de0fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.300+0000 7f539856a740 1 --2- 172.21.15.19:0/1000833988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca7dd70390 0x55ca7ddb4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.300+0000 7f5373e00640 0 client.8847 ms_handle_reset on v2:172.21.15.188:6834/2088538359 2024-03-28T13:21:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.300+0000 7f539856a740 1 -- 172.21.15.19:0/1000833988 >> 172.21.15.19:0/1000833988 conn(0x55ca7dd485c0 msgr2=0x55ca7dd6b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.300+0000 7f539856a740 1 -- 172.21.15.19:0/1000833988 shutdown_connections 2024-03-28T13:21:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.300+0000 7f539856a740 1 -- 172.21.15.19:0/1000833988 wait complete. 2024-03-28T13:21:00.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.300+0000 7f539856a740 20 client.8847 trim_cache size 0 max 16384 2024-03-28T13:21:00.302 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:21:00.302 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "e6b569d9-f2ce-49dc-bbe7-c0ab33080203", 2024-03-28T13:21:00.302 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-03-28T13:21:00.302 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 8703, 2024-03-28T13:21:00.302 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-03-28T13:21:00.302 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-03-28T13:21:00.303 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-03-28T13:21:00.303 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 6.8419414070000002, 2024-03-28T13:21:00.303 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 99, 2024-03-28T13:21:00.303 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 119, 2024-03-28T13:21:00.303 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 119, 2024-03-28T13:21:00.303 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 12.223296581 2024-03-28T13:21:00.303 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:21:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.301+0000 7f5395800640 1 -- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f536c06c530 msgr2=0x7f536c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.301+0000 7f5395800640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f536c06c530 0x7f536c06e9f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5390150b80 tx=0x7f5378009040 comp rx=0 tx=0).stop 2024-03-28T13:21:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.301+0000 7f5395800640 1 -- 172.21.15.19:0/416380115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900af4b0 msgr2=0x7f53901f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.301+0000 7f5395800640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900af4b0 0x7f53901f5050 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f538000d800 tx=0x7f538000dcc0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.301+0000 7f5395800640 1 -- 172.21.15.19:0/416380115 shutdown_connections 2024-03-28T13:21:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.301+0000 7f5395800640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f536c06c530 0x7f536c06e9f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.301+0000 7f5395800640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5390153580 0x7f539015ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.301+0000 7f5395800640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53900d07c0 0x7f539015e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.301+0000 7f5395800640 1 --2- 172.21.15.19:0/416380115 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53900af4b0 0x7f53901f5050 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.301+0000 7f5395800640 1 -- 172.21.15.19:0/416380115 >> 172.21.15.19:0/416380115 conn(0x7f539006b750 msgr2=0x7f5390152740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.301+0000 7f5395800640 1 -- 172.21.15.19:0/416380115 shutdown_connections 2024-03-28T13:21:00.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.301+0000 7f5395800640 1 -- 172.21.15.19:0/416380115 wait complete. 2024-03-28T13:21:00.319 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:21:00.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.538+0000 7fce7ee00640 1 Processor -- start 2024-03-28T13:21:00.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.538+0000 7fce7ee00640 1 -- start start 2024-03-28T13:21:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.539+0000 7fce7ee00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fce780d0990 0x7fce781670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.539+0000 7fce7ee00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce78163ee0 con 0x7fce78162920 2024-03-28T13:21:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.539+0000 7fce7ee00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce78071d60 con 0x7fce78163ac0 2024-03-28T13:21:00.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.540+0000 7fce7ee00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fce781665d0 con 0x7fce780d0990 2024-03-28T13:21:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.540+0000 7fce77e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fce78162920 0x7fce780d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53730/0 (socket says 172.21.15.19:53730) 2024-03-28T13:21:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.540+0000 7fce77e00640 1 -- 172.21.15.19:0/1881456212 learned_addr learned my addr 172.21.15.19:0/1881456212 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:00.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.540+0000 7fce77400640 1 --2- 172.21.15.19:0/1881456212 >> v2:172.21.15.188:3300/0 conn(0x7fce780d0990 0x7fce781670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.540+0000 7fce76a00640 1 -- 172.21.15.19:0/1881456212 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3071994907 0 0) 0x7fce78163ee0 con 0x7fce78162920 2024-03-28T13:21:00.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.540+0000 7fce76a00640 1 -- 172.21.15.19:0/1881456212 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce58003520 con 0x7fce78162920 2024-03-28T13:21:00.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.541+0000 7fce76a00640 1 -- 172.21.15.19:0/1881456212 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3707702667 0 0) 0x7fce58003520 con 0x7fce78162920 2024-03-28T13:21:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.541+0000 7fce76a00640 1 -- 172.21.15.19:0/1881456212 >> v2:172.21.15.188:3300/0 conn(0x7fce780d0990 msgr2=0x7fce781670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.541+0000 7fce76a00640 1 --2- 172.21.15.19:0/1881456212 >> v2:172.21.15.188:3300/0 conn(0x7fce780d0990 0x7fce781670b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.541+0000 7fce76a00640 1 -- 172.21.15.19:0/1881456212 >> v1:172.21.15.188:6789/0 conn(0x7fce78163ac0 legacy=0x7fce780ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.541+0000 7fce76a00640 1 -- 172.21.15.19:0/1881456212 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce781668b0 con 0x7fce78162920 2024-03-28T13:21:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.541+0000 7fce76a00640 1 -- 172.21.15.19:0/1881456212 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fce68001690 con 0x7fce78162920 2024-03-28T13:21:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.541+0000 7fce76a00640 1 -- 172.21.15.19:0/1881456212 >> v1:172.21.15.19:6789/0 conn(0x7fce78162920 legacy=0x7fce780d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.541+0000 7fce76a00640 1 --2- 172.21.15.19:0/1881456212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce580039b0 0x7fce58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.541+0000 7fce76a00640 1 --2- 172.21.15.19:0/1881456212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce58004800 0x7fce58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.541+0000 7fce76a00640 1 --2- 172.21.15.19:0/1881456212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce580051c0 0x7fce5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.541+0000 7fce76a00640 1 -- 172.21.15.19:0/1881456212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce781665d0 con 0x7fce580039b0 2024-03-28T13:21:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.541+0000 7fce76a00640 1 -- 172.21.15.19:0/1881456212 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fce5800c160 con 0x7fce580051c0 2024-03-28T13:21:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.541+0000 7fce76a00640 1 -- 172.21.15.19:0/1881456212 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fce5800c360 con 0x7fce58004800 2024-03-28T13:21:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.541+0000 7fce77e00640 1 --2- 172.21.15.19:0/1881456212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce580039b0 0x7fce58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.542+0000 7fce77400640 1 --2- 172.21.15.19:0/1881456212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce58004800 0x7fce58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.542+0000 7fce77e00640 1 -- 172.21.15.19:0/1881456212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce58004800 msgr2=0x7fce58004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.542+0000 7fce77e00640 1 --2- 172.21.15.19:0/1881456212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce58004800 0x7fce58004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.542+0000 7fce7ce00640 1 --2- 172.21.15.19:0/1881456212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce580051c0 0x7fce5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.542+0000 7fce77e00640 1 -- 172.21.15.19:0/1881456212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce580051c0 msgr2=0x7fce5800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.542+0000 7fce77e00640 1 --2- 172.21.15.19:0/1881456212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce580051c0 0x7fce5800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.542+0000 7fce7ce00640 1 -- 172.21.15.19:0/1881456212 reap_dead start 2024-03-28T13:21:00.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.542+0000 7fce77e00640 1 -- 172.21.15.19:0/1881456212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce58003520 con 0x7fce580039b0 2024-03-28T13:21:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.543+0000 7fce77e00640 1 --2- 172.21.15.19:0/1881456212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce580039b0 0x7fce58003dd0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fce6800c210 tx=0x7fce58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8b406a6c38d6e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.543+0000 7fce76a00640 1 -- 172.21.15.19:0/1881456212 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68007d30 con 0x7fce580039b0 2024-03-28T13:21:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.543+0000 7fce76a00640 1 -- 172.21.15.19:0/1881456212 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce68030070 con 0x7fce580039b0 2024-03-28T13:21:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.543+0000 7fce76a00640 1 -- 172.21.15.19:0/1881456212 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68005d30 con 0x7fce580039b0 2024-03-28T13:21:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.544+0000 7fce7ee00640 1 -- 172.21.15.19:0/1881456212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce580039b0 msgr2=0x7fce58003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.544+0000 7fce7ee00640 1 --2- 172.21.15.19:0/1881456212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce580039b0 0x7fce58003dd0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fce6800c210 tx=0x7fce58005830 comp rx=0 tx=0).stop 2024-03-28T13:21:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.544+0000 7fce7ee00640 1 -- 172.21.15.19:0/1881456212 shutdown_connections 2024-03-28T13:21:00.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.544+0000 7fce7ee00640 1 --2- 172.21.15.19:0/1881456212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce580039b0 0x7fce58003dd0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.544+0000 7fce7ee00640 1 -- 172.21.15.19:0/1881456212 >> 172.21.15.19:0/1881456212 conn(0x7fce7806b750 msgr2=0x7fce7806bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.544+0000 7fce7ee00640 1 -- 172.21.15.19:0/1881456212 shutdown_connections 2024-03-28T13:21:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.544+0000 7fce7ee00640 1 -- 172.21.15.19:0/1881456212 wait complete. 2024-03-28T13:21:00.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.546+0000 7fce7ee00640 1 Processor -- start 2024-03-28T13:21:00.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.546+0000 7fce7ee00640 1 -- start start 2024-03-28T13:21:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.547+0000 7fce7ee00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce580039b0 0x7fce781ede80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.547+0000 7fce7ee00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7815bae0 0x7fce781ec390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.547+0000 7fce7ee00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce7815bf40 0x7fce780b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.547+0000 7fce7ee00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fce78072040 con 0x7fce580039b0 2024-03-28T13:21:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.547+0000 7fce7ee00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fce78162d00 con 0x7fce7815bf40 2024-03-28T13:21:00.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.547+0000 7fce7ee00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fce780721c0 con 0x7fce7815bae0 2024-03-28T13:21:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.547+0000 7fce77e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce580039b0 0x7fce781ede80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.547+0000 7fce77e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce580039b0 0x7fce781ede80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41986/0 (socket says 172.21.15.19:41986) 2024-03-28T13:21:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.547+0000 7fce77e00640 1 -- 172.21.15.19:0/3619867656 learned_addr learned my addr 172.21.15.19:0/3619867656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.548+0000 7fce77e00640 1 -- 172.21.15.19:0/3619867656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7815bae0 msgr2=0x7fce781ec390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.548+0000 7fce77400640 1 --2- 172.21.15.19:0/3619867656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7815bae0 0x7fce781ec390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.548+0000 7fce77e00640 1 --2- 172.21.15.19:0/3619867656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7815bae0 0x7fce781ec390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.548+0000 7fce77e00640 1 -- 172.21.15.19:0/3619867656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce7815bf40 msgr2=0x7fce780b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.548+0000 7fce77e00640 1 --2- 172.21.15.19:0/3619867656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce7815bf40 0x7fce780b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.548+0000 7fce77e00640 1 -- 172.21.15.19:0/3619867656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce781ecde0 con 0x7fce580039b0 2024-03-28T13:21:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.548+0000 7fce77e00640 1 --2- 172.21.15.19:0/3619867656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce580039b0 0x7fce781ede80 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fce68001990 tx=0x7fce6802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.548+0000 7fce6fe00640 1 -- 172.21.15.19:0/3619867656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68046070 con 0x7fce580039b0 2024-03-28T13:21:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.548+0000 7fce6fe00640 1 -- 172.21.15.19:0/3619867656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce68004bf0 con 0x7fce580039b0 2024-03-28T13:21:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.549+0000 7fce6fe00640 1 -- 172.21.15.19:0/3619867656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce68007d30 con 0x7fce580039b0 2024-03-28T13:21:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.549+0000 7fce7ee00640 1 -- 172.21.15.19:0/3619867656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce781ed0c0 con 0x7fce580039b0 2024-03-28T13:21:00.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.549+0000 7fce7ee00640 1 -- 172.21.15.19:0/3619867656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce781ed680 con 0x7fce580039b0 2024-03-28T13:21:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.550+0000 7fce7ee00640 1 -- 172.21.15.19:0/3619867656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce78165db0 con 0x7fce580039b0 2024-03-28T13:21:00.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.551+0000 7fce6fe00640 1 -- 172.21.15.19:0/3619867656 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fce68007070 con 0x7fce580039b0 2024-03-28T13:21:00.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.552+0000 7fce6fe00640 1 --2- 172.21.15.19:0/3619867656 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fce5406c530 0x7fce5406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:00.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.552+0000 7fce6fe00640 1 -- 172.21.15.19:0/3619867656 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7fce680acc40 con 0x7fce580039b0 2024-03-28T13:21:00.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.552+0000 7fce77400640 1 --2- 172.21.15.19:0/3619867656 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fce5406c530 0x7fce5406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:00.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.553+0000 7fce77400640 1 --2- 172.21.15.19:0/3619867656 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fce5406c530 0x7fce5406e9f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fce780ce020 tx=0x7fce6000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:00.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.556+0000 7fce6fe00640 1 -- 172.21.15.19:0/3619867656 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fce78165db0 con 0x7fce580039b0 2024-03-28T13:21:00.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.800+0000 7fce7ee00640 1 -- 172.21.15.19:0/3619867656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fce780685c0 con 0x7fce580039b0 2024-03-28T13:21:00.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.801+0000 7fce6fe00640 1 -- 172.21.15.19:0/3619867656 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) ==== 53+0+73 (secure 0 0 0) 0x7fce6807b130 con 0x7fce580039b0 2024-03-28T13:21:00.800 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:21:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.804+0000 7fce7ee00640 1 -- 172.21.15.19:0/3619867656 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fce5406c530 msgr2=0x7fce5406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.804+0000 7fce7ee00640 1 --2- 172.21.15.19:0/3619867656 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fce5406c530 0x7fce5406e9f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fce780ce020 tx=0x7fce6000b040 comp rx=0 tx=0).stop 2024-03-28T13:21:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.804+0000 7fce7ee00640 1 -- 172.21.15.19:0/3619867656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce580039b0 msgr2=0x7fce781ede80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.804+0000 7fce7ee00640 1 --2- 172.21.15.19:0/3619867656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce580039b0 0x7fce781ede80 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fce68001990 tx=0x7fce6802e070 comp rx=0 tx=0).stop 2024-03-28T13:21:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.804+0000 7fce7ee00640 1 -- 172.21.15.19:0/3619867656 shutdown_connections 2024-03-28T13:21:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.804+0000 7fce7ee00640 1 --2- 172.21.15.19:0/3619867656 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fce5406c530 0x7fce5406e9f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.804+0000 7fce7ee00640 1 --2- 172.21.15.19:0/3619867656 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce7815bf40 0x7fce780b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.804+0000 7fce7ee00640 1 --2- 172.21.15.19:0/3619867656 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7815bae0 0x7fce781ec390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.804+0000 7fce7ee00640 1 --2- 172.21.15.19:0/3619867656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fce580039b0 0x7fce781ede80 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.804+0000 7fce7ee00640 1 -- 172.21.15.19:0/3619867656 >> 172.21.15.19:0/3619867656 conn(0x7fce7806b750 msgr2=0x7fce7806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.804+0000 7fce7ee00640 1 -- 172.21.15.19:0/3619867656 shutdown_connections 2024-03-28T13:21:00.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:00.804+0000 7fce7ee00640 1 -- 172.21.15.19:0/3619867656 wait complete. 2024-03-28T13:21:00.823 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-28T13:21:00.823 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:21:00.823 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:21:00.823 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:21:00.823 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:00.823 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-28T13:21:00.823 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-28T13:21:00.823 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:21:00.823 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:21:00.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:00.824 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:21:00.877 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:21:00.877 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:21:00.877 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:21:00.877 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:00.877 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:21:00.877 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:00.877 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:21:00.877 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:00.877 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:21:00.877 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:21:00.877 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:00.878 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:21:00.878 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:21:00.878 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:00.878 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:21:00.878 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:21:00.878 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41894sec preferred_lft 41894sec 2024-03-28T13:21:00.878 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:21:00.878 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:00.878 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:21:00.878 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:00.878 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:21:00.878 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:21:00.878 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:00.878 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:21:00.879 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:00.879 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:21:00.879 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:00.879 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:21:00.879 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:21:00.879 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:21:00.879 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:00.879 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:21:00.879 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether e6:17:c5:8e:2b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:21:00.879 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::e417:c5ff:fe8e:2bcf/64 scope link 2024-03-28T13:21:00.879 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:00.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:00.880 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:21:00.977 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:21:00.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:00.982 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:21:01.039 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:21:01.039 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:21:01.039 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:21:01.040 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:21:01.040 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:21:01.040 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:21:01.040 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:21:01.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:21:01.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:21:01.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:21:01.040 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:21:01.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:21:01.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:01.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:01.299 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:21:01.346 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:21:01.346 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:21:01.347 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:21:01.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:01.347 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:21:01.498 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:21:01.498 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:01.552 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:21:01.552 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:21:01.552 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:01.603 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:21:01.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:01.603 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:01.746 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:01.747 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:21:01.747 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:21:01.769 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:01.770 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:21:01.770 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:21:01.770 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:21:01.770 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:21:01.797 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:01.797 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:21:01.797 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:21:01.797 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:21:01.797 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:21:01.797 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:21:01.797 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:21:01.798 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:21:01.799 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:21:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:21:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:21:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:21:01.800 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:21:01.803 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:21:01.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:01.804 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:01.888 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:21:02.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:21:02.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:02.035 INFO:teuthology.orchestra.run.smithi019.stdout:client7051 2024-03-28T13:21:02.036 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:02.036 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:02.091 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:02.092 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:02.092 DEBUG:teuthology.orchestra.run.smithi019:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-03-28T13:21:02.092 DEBUG:teuthology.orchestra.run.smithi019:> parent = os.path.join(mount_point, "parent") 2024-03-28T13:21:02.092 DEBUG:teuthology.orchestra.run.smithi019:> os.mkdir(parent) 2024-03-28T13:21:02.092 DEBUG:teuthology.orchestra.run.smithi019:> flushed = os.path.join(parent, "flushed") 2024-03-28T13:21:02.092 DEBUG:teuthology.orchestra.run.smithi019:> os.mkdir(flushed) 2024-03-28T13:21:02.092 DEBUG:teuthology.orchestra.run.smithi019:> for f in ["alpha", "bravo", "charlie"]: 2024-03-28T13:21:02.092 DEBUG:teuthology.orchestra.run.smithi019:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-03-28T13:21:02.092 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:02.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:02.280 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-28T13:21:02.392 INFO:teuthology.orchestra.run.smithi019.stdout:./ 2024-03-28T13:21:02.392 INFO:teuthology.orchestra.run.smithi019.stdout:./parent 2024-03-28T13:21:02.392 INFO:teuthology.orchestra.run.smithi019.stdout:./parent/flushed 2024-03-28T13:21:02.392 INFO:teuthology.orchestra.run.smithi019.stdout:./parent/flushed/charlie 2024-03-28T13:21:02.392 INFO:teuthology.orchestra.run.smithi019.stdout:./parent/flushed/alpha 2024-03-28T13:21:02.392 INFO:teuthology.orchestra.run.smithi019.stdout:./parent/flushed/bravo 2024-03-28T13:21:02.393 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:02.393 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:02.394 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:02.394 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:02.394 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-28T13:21:02.394 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:02.557 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:21:02.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:02.572 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:02.572 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:02.572 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:02.572 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:02.572 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-03-28T13:21:02.572 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:02.770 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627776 2024-03-28T13:21:02.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:02.807 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:02.807 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:02.808 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:02.808 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:02.808 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-03-28T13:21:02.808 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:03.001 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627777 2024-03-28T13:21:03.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:03.017 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:03.017 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:03.017 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:03.017 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:03.017 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-03-28T13:21:03.017 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:03.210 INFO:teuthology.orchestra.run.smithi019.stdout:1099511628280 2024-03-28T13:21:03.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:03.225 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:03.225 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:03.226 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:03.226 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:03.226 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-03-28T13:21:03.226 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:03.418 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627778 2024-03-28T13:21:03.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:03.433 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:03.433 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:03.433 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:03.433 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:03.434 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-03-28T13:21:03.434 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:03.630 INFO:teuthology.orchestra.run.smithi019.stdout:1099511628279 2024-03-28T13:21:03.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:03.644 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:03.644 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:03.746 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:03.747 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:03.804 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:21:04.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.027+0000 7fb401400640 1 Processor -- start 2024-03-28T13:21:04.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.027+0000 7fb401400640 1 -- start start 2024-03-28T13:21:04.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.029+0000 7fb401400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb3fc1442c0 0x7fb3fc0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:04.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.029+0000 7fb401400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc0cefe0 0x7fb3fc0cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:04.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.029+0000 7fb401400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3fc071460 con 0x7fb3fc0d00d0 2024-03-28T13:21:04.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.029+0000 7fb401400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb3fc071d20 con 0x7fb3fc1442c0 2024-03-28T13:21:04.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.029+0000 7fb401400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb3fc0cf420 con 0x7fb3fc0cefe0 2024-03-28T13:21:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.030+0000 7fb400a00640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fb3fc0d00d0 0x7fb3fc06e6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:60874/0 (socket says 172.21.15.19:60874) 2024-03-28T13:21:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.030+0000 7fb400a00640 1 -- 172.21.15.19:0/1480481478 learned_addr learned my addr 172.21.15.19:0/1480481478 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.030+0000 7fb3fb400640 1 --2- 172.21.15.19:0/1480481478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc0cefe0 0x7fb3fc0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:04.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.030+0000 7fb3fbe00640 1 --2- 172.21.15.19:0/1480481478 >> v2:172.21.15.188:3300/0 conn(0x7fb3fc1442c0 0x7fb3fc0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.031+0000 7fb3faa00640 1 -- 172.21.15.19:0/1480481478 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3812839352 0 0) 0x7fb3fc071460 con 0x7fb3fc0d00d0 2024-03-28T13:21:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.031+0000 7fb3faa00640 1 -- 172.21.15.19:0/1480481478 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3e0003520 con 0x7fb3fc0d00d0 2024-03-28T13:21:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.031+0000 7fb3fb400640 1 -- 172.21.15.19:0/1480481478 >> v2:172.21.15.188:3300/0 conn(0x7fb3fc1442c0 msgr2=0x7fb3fc0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.031+0000 7fb3fb400640 1 --2- 172.21.15.19:0/1480481478 >> v2:172.21.15.188:3300/0 conn(0x7fb3fc1442c0 0x7fb3fc0ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.031+0000 7fb3fb400640 1 -- 172.21.15.19:0/1480481478 >> v1:172.21.15.188:6789/0 conn(0x7fb3fc0d00d0 legacy=0x7fb3fc06e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:04.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.031+0000 7fb3fb400640 1 -- 172.21.15.19:0/1480481478 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3fc0721b0 con 0x7fb3fc0cefe0 2024-03-28T13:21:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.033+0000 7fb3fb400640 1 --2- 172.21.15.19:0/1480481478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc0cefe0 0x7fb3fc0cfb90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80097a0 tx=0x7fb3e802b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=193c014f80046b04 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.033+0000 7fb3faa00640 1 -- 172.21.15.19:0/1480481478 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8034070 con 0x7fb3fc0cefe0 2024-03-28T13:21:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.033+0000 7fb3faa00640 1 -- 172.21.15.19:0/1480481478 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e8030610 con 0x7fb3fc0cefe0 2024-03-28T13:21:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.033+0000 7fb3faa00640 1 -- 172.21.15.19:0/1480481478 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80309e0 con 0x7fb3fc0cefe0 2024-03-28T13:21:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.034+0000 7fb401400640 1 -- 172.21.15.19:0/1480481478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc0cefe0 msgr2=0x7fb3fc0cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.034+0000 7fb401400640 1 --2- 172.21.15.19:0/1480481478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc0cefe0 0x7fb3fc0cfb90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80097a0 tx=0x7fb3e802b960 comp rx=0 tx=0).stop 2024-03-28T13:21:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.034+0000 7fb401400640 1 -- 172.21.15.19:0/1480481478 shutdown_connections 2024-03-28T13:21:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.034+0000 7fb401400640 1 --2- 172.21.15.19:0/1480481478 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc0cefe0 0x7fb3fc0cfb90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.034+0000 7fb401400640 1 --2- 172.21.15.19:0/1480481478 >> v2:172.21.15.188:3300/0 conn(0x7fb3fc1442c0 0x7fb3fc0ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.034+0000 7fb401400640 1 -- 172.21.15.19:0/1480481478 >> 172.21.15.19:0/1480481478 conn(0x7fb3fc0ccad0 msgr2=0x7fb3fc0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.034+0000 7fb401400640 1 -- 172.21.15.19:0/1480481478 shutdown_connections 2024-03-28T13:21:04.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.034+0000 7fb401400640 1 -- 172.21.15.19:0/1480481478 wait complete. 2024-03-28T13:21:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.036+0000 7fb401400640 1 Processor -- start 2024-03-28T13:21:04.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.036+0000 7fb401400640 1 -- start start 2024-03-28T13:21:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.037+0000 7fb401400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3fc0cefe0 0x7fb3fc15a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.037+0000 7fb401400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc0d00d0 0x7fb3fc1e38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.037+0000 7fb401400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3fc1442c0 0x7fb3fc1e3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.037+0000 7fb401400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc0d0770 con 0x7fb3fc1442c0 2024-03-28T13:21:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.037+0000 7fb401400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc0719b0 con 0x7fb3fc0cefe0 2024-03-28T13:21:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.037+0000 7fb401400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb3fc1446a0 con 0x7fb3fc0d00d0 2024-03-28T13:21:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.038+0000 7fb400a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3fc1442c0 0x7fb3fc1e3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.038+0000 7fb400a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3fc1442c0 0x7fb3fc1e3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41992/0 (socket says 172.21.15.19:41992) 2024-03-28T13:21:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.038+0000 7fb400a00640 1 -- 172.21.15.19:0/1308879442 learned_addr learned my addr 172.21.15.19:0/1308879442 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:04.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.038+0000 7fb3fbe00640 1 --2- 172.21.15.19:0/1308879442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3fc0cefe0 0x7fb3fc15a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:04.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.038+0000 7fb3fb400640 1 --2- 172.21.15.19:0/1308879442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc0d00d0 0x7fb3fc1e38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:04.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.038+0000 7fb400a00640 1 -- 172.21.15.19:0/1308879442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc0d00d0 msgr2=0x7fb3fc1e38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:04.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.038+0000 7fb400a00640 1 --2- 172.21.15.19:0/1308879442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc0d00d0 0x7fb3fc1e38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.038+0000 7fb400a00640 1 -- 172.21.15.19:0/1308879442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3fc0cefe0 msgr2=0x7fb3fc15a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:04.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.038+0000 7fb400a00640 1 --2- 172.21.15.19:0/1308879442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3fc0cefe0 0x7fb3fc15a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.038+0000 7fb400a00640 1 -- 172.21.15.19:0/1308879442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3fc142e80 con 0x7fb3fc1442c0 2024-03-28T13:21:04.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.038+0000 7fb400a00640 1 --2- 172.21.15.19:0/1308879442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3fc1442c0 0x7fb3fc1e3e00 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400cb40 tx=0x7fb3f400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.039+0000 7fb3f3e00640 1 -- 172.21.15.19:0/1308879442 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4017070 con 0x7fb3fc1442c0 2024-03-28T13:21:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.039+0000 7fb3f3e00640 1 -- 172.21.15.19:0/1308879442 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f400aca0 con 0x7fb3fc1442c0 2024-03-28T13:21:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.039+0000 7fb3f3e00640 1 -- 172.21.15.19:0/1308879442 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4012620 con 0x7fb3fc1442c0 2024-03-28T13:21:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.039+0000 7fb401400640 1 -- 172.21.15.19:0/1308879442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3fc15c900 con 0x7fb3fc1442c0 2024-03-28T13:21:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.039+0000 7fb401400640 1 -- 172.21.15.19:0/1308879442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3fc15cec0 con 0x7fb3fc1442c0 2024-03-28T13:21:04.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.040+0000 7fb401400640 1 -- 172.21.15.19:0/1308879442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3fc0af6f0 con 0x7fb3fc1442c0 2024-03-28T13:21:04.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.042+0000 7fb3f3e00640 1 -- 172.21.15.19:0/1308879442 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb3f401a070 con 0x7fb3fc1442c0 2024-03-28T13:21:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.042+0000 7fb3f3e00640 1 --2- 172.21.15.19:0/1308879442 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb3dc06c530 0x7fb3dc06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.043+0000 7fb3f3e00640 1 -- 172.21.15.19:0/1308879442 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7fb3f408b660 con 0x7fb3fc1442c0 2024-03-28T13:21:04.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.043+0000 7fb3fbe00640 1 --2- 172.21.15.19:0/1308879442 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb3dc06c530 0x7fb3dc06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:04.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.044+0000 7fb3fbe00640 1 --2- 172.21.15.19:0/1308879442 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb3dc06c530 0x7fb3dc06e9f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009a30 tx=0x7fb3ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:04.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.048+0000 7fb3f3e00640 1 -- 172.21.15.19:0/1308879442 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb3fc0af6f0 con 0x7fb3fc1442c0 2024-03-28T13:21:04.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.252+0000 7fb401400640 1 -- 172.21.15.19:0/1308879442 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3fc0ae8a0 con 0x7fb3fc1442c0 2024-03-28T13:21:04.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.255+0000 7fb3f3e00640 1 -- 172.21.15.19:0/1308879442 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16683 (secure 0 0 0) 0x7fb3f4059be0 con 0x7fb3fc1442c0 2024-03-28T13:21:04.255 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:04.255 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":119,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:20:54.450982+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:20:50.748212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-28T13:20:51.888444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:21:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.257+0000 7fb401400640 1 -- 172.21.15.19:0/1308879442 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb3dc06c530 msgr2=0x7fb3dc06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.257+0000 7fb401400640 1 --2- 172.21.15.19:0/1308879442 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb3dc06c530 0x7fb3dc06e9f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009a30 tx=0x7fb3ec008040 comp rx=0 tx=0).stop 2024-03-28T13:21:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.257+0000 7fb401400640 1 -- 172.21.15.19:0/1308879442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3fc1442c0 msgr2=0x7fb3fc1e3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.257+0000 7fb401400640 1 --2- 172.21.15.19:0/1308879442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3fc1442c0 0x7fb3fc1e3e00 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400cb40 tx=0x7fb3f400a060 comp rx=0 tx=0).stop 2024-03-28T13:21:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.257+0000 7fb401400640 1 -- 172.21.15.19:0/1308879442 shutdown_connections 2024-03-28T13:21:04.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.257+0000 7fb401400640 1 --2- 172.21.15.19:0/1308879442 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb3dc06c530 0x7fb3dc06e9f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.257+0000 7fb401400640 1 --2- 172.21.15.19:0/1308879442 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3fc1442c0 0x7fb3fc1e3e00 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.257+0000 7fb401400640 1 --2- 172.21.15.19:0/1308879442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3fc0d00d0 0x7fb3fc1e38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.257+0000 7fb401400640 1 --2- 172.21.15.19:0/1308879442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3fc0cefe0 0x7fb3fc15a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.258+0000 7fb401400640 1 -- 172.21.15.19:0/1308879442 >> 172.21.15.19:0/1308879442 conn(0x7fb3fc0ccad0 msgr2=0x7fb3fc06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:04.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.258+0000 7fb401400640 1 -- 172.21.15.19:0/1308879442 shutdown_connections 2024-03-28T13:21:04.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.258+0000 7fb401400640 1 -- 172.21.15.19:0/1308879442 wait complete. 2024-03-28T13:21:04.278 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:20:50.748212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:20:51.888444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:21:04.279 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:21:04.483 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:21:04.490 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:21:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.508+0000 7f016ba00640 1 Processor -- start 2024-03-28T13:21:04.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.508+0000 7f016ba00640 1 -- start start 2024-03-28T13:21:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.509+0000 7f016ba00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f016415e6d0 0x7f01640d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.509+0000 7f016ba00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f01640d4760 0x7f01640cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.509+0000 7f016ba00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01640ceef0 con 0x7f01640d0cb0 2024-03-28T13:21:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.509+0000 7f016ba00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f016415eab0 con 0x7f01640d4760 2024-03-28T13:21:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.509+0000 7f016ba00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f01640d1220 con 0x7f016415e6d0 2024-03-28T13:21:04.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.510+0000 7f0163e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f01640d0cb0 0x7f01640d4050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:53738/0 (socket says 172.21.15.19:53738) 2024-03-28T13:21:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.510+0000 7f0163e00640 1 -- 172.21.15.19:0/3851717882 learned_addr learned my addr 172.21.15.19:0/3851717882 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.510+0000 7f0169a00640 1 --2- 172.21.15.19:0/3851717882 >> v2:172.21.15.19:3300/0 conn(0x7f01640d4760 0x7f01640cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.510+0000 7f0169000640 1 --2- 172.21.15.19:0/3851717882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f016415e6d0 0x7f01640d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.510+0000 7f0163400640 1 -- 172.21.15.19:0/3851717882 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1032771994 0 0) 0x7f01640ceef0 con 0x7f01640d0cb0 2024-03-28T13:21:04.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.510+0000 7f0163400640 1 -- 172.21.15.19:0/3851717882 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f014c003520 con 0x7f01640d0cb0 2024-03-28T13:21:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.510+0000 7f0169a00640 1 -- 172.21.15.19:0/3851717882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f016415e6d0 msgr2=0x7f01640d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.510+0000 7f0169a00640 1 --2- 172.21.15.19:0/3851717882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f016415e6d0 0x7f01640d21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.510+0000 7f0169a00640 1 -- 172.21.15.19:0/3851717882 >> v1:172.21.15.19:6789/0 conn(0x7f01640d0cb0 legacy=0x7f01640d4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.510+0000 7f0169a00640 1 -- 172.21.15.19:0/3851717882 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01640d1500 con 0x7f01640d4760 2024-03-28T13:21:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.512+0000 7f0169a00640 1 --2- 172.21.15.19:0/3851717882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01640d4760 0x7f01640cf3d0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f015c009d90 tx=0x7f015c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2c2133d9c1f2d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:04.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.512+0000 7f0163400640 1 -- 172.21.15.19:0/3851717882 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c039070 con 0x7f01640d4760 2024-03-28T13:21:04.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.513+0000 7f0163400640 1 -- 172.21.15.19:0/3851717882 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f015c02cc50 con 0x7f01640d4760 2024-03-28T13:21:04.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.513+0000 7f0163400640 1 -- 172.21.15.19:0/3851717882 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c034440 con 0x7f01640d4760 2024-03-28T13:21:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.513+0000 7f016ba00640 1 -- 172.21.15.19:0/3851717882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01640d4760 msgr2=0x7f01640cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.513+0000 7f016ba00640 1 --2- 172.21.15.19:0/3851717882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01640d4760 0x7f01640cf3d0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f015c009d90 tx=0x7f015c02bea0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.513+0000 7f016ba00640 1 -- 172.21.15.19:0/3851717882 shutdown_connections 2024-03-28T13:21:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.513+0000 7f016ba00640 1 --2- 172.21.15.19:0/3851717882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01640d4760 0x7f01640cf3d0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.513+0000 7f016ba00640 1 --2- 172.21.15.19:0/3851717882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f016415e6d0 0x7f01640d21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.513+0000 7f016ba00640 1 -- 172.21.15.19:0/3851717882 >> 172.21.15.19:0/3851717882 conn(0x7f016409d7b0 msgr2=0x7f016408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.513+0000 7f016ba00640 1 -- 172.21.15.19:0/3851717882 shutdown_connections 2024-03-28T13:21:04.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.514+0000 7f016ba00640 1 -- 172.21.15.19:0/3851717882 wait complete. 2024-03-28T13:21:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.515+0000 7f016ba00640 1 Processor -- start 2024-03-28T13:21:04.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.516+0000 7f016ba00640 1 -- start start 2024-03-28T13:21:04.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.517+0000 7f016ba00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01640d0cb0 0x7f01640c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.517+0000 7f016ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01640d4760 0x7f016415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.517+0000 7f016ba00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f016415e6d0 0x7f016406cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.517+0000 7f016ba00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f01640d2720 con 0x7f01640d4760 2024-03-28T13:21:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.517+0000 7f016ba00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f01640d1ac0 con 0x7f01640d0cb0 2024-03-28T13:21:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.517+0000 7f016ba00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f016406c890 con 0x7f016415e6d0 2024-03-28T13:21:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.517+0000 7f0163e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01640d4760 0x7f016415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.517+0000 7f0163e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01640d4760 0x7f016415a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42008/0 (socket says 172.21.15.19:42008) 2024-03-28T13:21:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.517+0000 7f0163e00640 1 -- 172.21.15.19:0/1501840635 learned_addr learned my addr 172.21.15.19:0/1501840635 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.517+0000 7f0169a00640 1 --2- 172.21.15.19:0/1501840635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f016415e6d0 0x7f016406cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:04.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.517+0000 7f0169000640 1 --2- 172.21.15.19:0/1501840635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01640d0cb0 0x7f01640c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:04.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.518+0000 7f0163e00640 1 -- 172.21.15.19:0/1501840635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f016415e6d0 msgr2=0x7f016406cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.518+0000 7f0163e00640 1 --2- 172.21.15.19:0/1501840635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f016415e6d0 0x7f016406cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.518+0000 7f0163e00640 1 -- 172.21.15.19:0/1501840635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01640d0cb0 msgr2=0x7f01640c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.518+0000 7f0163e00640 1 --2- 172.21.15.19:0/1501840635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01640d0cb0 0x7f01640c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.518+0000 7f0163e00640 1 -- 172.21.15.19:0/1501840635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01640c9800 con 0x7f01640d4760 2024-03-28T13:21:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.518+0000 7f0169000640 1 --2- 172.21.15.19:0/1501840635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01640d0cb0 0x7f01640c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:21:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.518+0000 7f0163e00640 1 --2- 172.21.15.19:0/1501840635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01640d4760 0x7f016415a480 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f015000c700 tx=0x7f015000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.518+0000 7f0160c00640 1 -- 172.21.15.19:0/1501840635 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150017070 con 0x7f01640d4760 2024-03-28T13:21:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.518+0000 7f0160c00640 1 -- 172.21.15.19:0/1501840635 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0150007290 con 0x7f01640d4760 2024-03-28T13:21:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.518+0000 7f016ba00640 1 -- 172.21.15.19:0/1501840635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01640c9ae0 con 0x7f01640d4760 2024-03-28T13:21:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.518+0000 7f0160c00640 1 -- 172.21.15.19:0/1501840635 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150001d40 con 0x7f01640d4760 2024-03-28T13:21:04.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.518+0000 7f016ba00640 1 -- 172.21.15.19:0/1501840635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01640c9510 con 0x7f01640d4760 2024-03-28T13:21:04.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.520+0000 7f016ba00640 1 -- 172.21.15.19:0/1501840635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01640c9ed0 con 0x7f01640d4760 2024-03-28T13:21:04.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.521+0000 7f0160c00640 1 -- 172.21.15.19:0/1501840635 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f015001a070 con 0x7f01640d4760 2024-03-28T13:21:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.522+0000 7f0160c00640 1 --2- 172.21.15.19:0/1501840635 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f014406c530 0x7f014406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.522+0000 7f0160c00640 1 -- 172.21.15.19:0/1501840635 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f015008aa30 con 0x7f01640d4760 2024-03-28T13:21:04.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.523+0000 7f0169000640 1 --2- 172.21.15.19:0/1501840635 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f014406c530 0x7f014406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:04.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.524+0000 7f0169000640 1 --2- 172.21.15.19:0/1501840635 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f014406c530 0x7f014406e9f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0158009af0 tx=0x7f0158008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.528+0000 7f0160c00640 1 -- 172.21.15.19:0/1501840635 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f01500589f0 con 0x7f01640d4760 2024-03-28T13:21:04.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.731+0000 7f016ba00640 1 -- 172.21.15.19:0/1501840635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f01640685c0 con 0x7f01640d4760 2024-03-28T13:21:04.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.733+0000 7f0160c00640 1 -- 172.21.15.19:0/1501840635 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7f015005ccf0 con 0x7f01640d4760 2024-03-28T13:21:04.732 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:21:04.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.735+0000 7f016ba00640 1 -- 172.21.15.19:0/1501840635 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f014406c530 msgr2=0x7f014406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:04.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.735+0000 7f016ba00640 1 --2- 172.21.15.19:0/1501840635 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f014406c530 0x7f014406e9f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0158009af0 tx=0x7f0158008040 comp rx=0 tx=0).stop 2024-03-28T13:21:04.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.735+0000 7f016ba00640 1 -- 172.21.15.19:0/1501840635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01640d4760 msgr2=0x7f016415a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:04.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.735+0000 7f016ba00640 1 --2- 172.21.15.19:0/1501840635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01640d4760 0x7f016415a480 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f015000c700 tx=0x7f015000cbc0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.735+0000 7f016ba00640 1 -- 172.21.15.19:0/1501840635 shutdown_connections 2024-03-28T13:21:04.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.735+0000 7f016ba00640 1 --2- 172.21.15.19:0/1501840635 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f014406c530 0x7f014406e9f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.735+0000 7f016ba00640 1 --2- 172.21.15.19:0/1501840635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f016415e6d0 0x7f016406cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.735+0000 7f016ba00640 1 --2- 172.21.15.19:0/1501840635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f01640d4760 0x7f016415a480 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.735+0000 7f016ba00640 1 --2- 172.21.15.19:0/1501840635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01640d0cb0 0x7f01640c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.735+0000 7f016ba00640 1 -- 172.21.15.19:0/1501840635 >> 172.21.15.19:0/1501840635 conn(0x7f016409d7b0 msgr2=0x7f01640cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.735+0000 7f016ba00640 1 -- 172.21.15.19:0/1501840635 shutdown_connections 2024-03-28T13:21:04.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.735+0000 7f016ba00640 1 -- 172.21.15.19:0/1501840635 wait complete. 2024-03-28T13:21:04.757 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:21:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.998+0000 7f4f07200640 1 Processor -- start 2024-03-28T13:21:04.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:04.999+0000 7f4f07200640 1 -- start start 2024-03-28T13:21:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.000+0000 7f4f07200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4f080b4890 0x7f4f08072570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.000+0000 7f4f07200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f0815f6a0 0x7f4f08072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.000+0000 7f4f07200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f080b4c70 con 0x7f4f0806d620 2024-03-28T13:21:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.000+0000 7f4f07200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4f080b3eb0 con 0x7f4f080b4890 2024-03-28T13:21:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.000+0000 7f4f07200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4f0815fb30 con 0x7f4f0815f6a0 2024-03-28T13:21:05.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.001+0000 7f4f05e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4f080b4890 0x7f4f08072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.001+0000 7f4f05e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4f080b4890 0x7f4f08072570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42018/0 (socket says 172.21.15.19:42018) 2024-03-28T13:21:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.001+0000 7f4f05e00640 1 -- 172.21.15.19:0/3941731109 learned_addr learned my addr 172.21.15.19:0/3941731109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:05.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.001+0000 7f4f05400640 1 --2- 172.21.15.19:0/3941731109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f0815f6a0 0x7f4f08072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.001+0000 7f4f05e00640 1 -- 172.21.15.19:0/3941731109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f0815f6a0 msgr2=0x7f4f08072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.001+0000 7f4f05e00640 1 --2- 172.21.15.19:0/3941731109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f0815f6a0 0x7f4f08072ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.001+0000 7f4f05e00640 1 -- 172.21.15.19:0/3941731109 >> v1:172.21.15.188:6789/0 conn(0x7f4f0806d620 legacy=0x7f4f080713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.001+0000 7f4f05e00640 1 -- 172.21.15.19:0/3941731109 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0815b7b0 con 0x7f4f080b4890 2024-03-28T13:21:05.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.002+0000 7f4f05e00640 1 --2- 172.21.15.19:0/3941731109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f080b4890 0x7f4f08072570 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009870 tx=0x7f4ef802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87bd3db7934b1452 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:05.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.003+0000 7f4f04a00640 1 -- 172.21.15.19:0/3941731109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8035020 con 0x7f4f080b4890 2024-03-28T13:21:05.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.003+0000 7f4f04a00640 1 -- 172.21.15.19:0/3941731109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef8031640 con 0x7f4f080b4890 2024-03-28T13:21:05.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.003+0000 7f4f04a00640 1 -- 172.21.15.19:0/3941731109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef802d040 con 0x7f4f080b4890 2024-03-28T13:21:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.004+0000 7f4f07200640 1 -- 172.21.15.19:0/3941731109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f080b4890 msgr2=0x7f4f08072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.004+0000 7f4f07200640 1 --2- 172.21.15.19:0/3941731109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f080b4890 0x7f4f08072570 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009870 tx=0x7f4ef802b8f0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.004+0000 7f4f07200640 1 -- 172.21.15.19:0/3941731109 shutdown_connections 2024-03-28T13:21:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.004+0000 7f4f07200640 1 --2- 172.21.15.19:0/3941731109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f0815f6a0 0x7f4f08072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.004+0000 7f4f07200640 1 --2- 172.21.15.19:0/3941731109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f080b4890 0x7f4f08072570 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.004+0000 7f4f07200640 1 -- 172.21.15.19:0/3941731109 >> 172.21.15.19:0/3941731109 conn(0x7f4f080b1ba0 msgr2=0x7f4f080b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.004+0000 7f4f07200640 1 -- 172.21.15.19:0/3941731109 shutdown_connections 2024-03-28T13:21:05.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.004+0000 7f4f07200640 1 -- 172.21.15.19:0/3941731109 wait complete. 2024-03-28T13:21:05.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.006+0000 7f4f07200640 1 Processor -- start 2024-03-28T13:21:05.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.006+0000 7f4f07200640 1 -- start start 2024-03-28T13:21:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f07200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f0806d620 0x7f4f080a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f07200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f080b4890 0x7f4f081f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f07200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f080ce830 0x7f4f080adc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f07200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0815c050 con 0x7f4f0806d620 2024-03-28T13:21:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f07200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0815bd70 con 0x7f4f080ce830 2024-03-28T13:21:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f07200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4f0815c570 con 0x7f4f080b4890 2024-03-28T13:21:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f05e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f0806d620 0x7f4f080a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f05e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f0806d620 0x7f4f080a9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42034/0 (socket says 172.21.15.19:42034) 2024-03-28T13:21:05.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f05e00640 1 -- 172.21.15.19:0/3938840232 learned_addr learned my addr 172.21.15.19:0/3938840232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f05400640 1 --2- 172.21.15.19:0/3938840232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f080b4890 0x7f4f081f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f05e00640 1 -- 172.21.15.19:0/3938840232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f080b4890 msgr2=0x7f4f081f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f05e00640 1 --2- 172.21.15.19:0/3938840232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f080b4890 0x7f4f081f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f05e00640 1 -- 172.21.15.19:0/3938840232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f080ce830 msgr2=0x7f4f080adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f05e00640 1 --2- 172.21.15.19:0/3938840232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f080ce830 0x7f4f080adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f05e00640 1 -- 172.21.15.19:0/3938840232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f080a9b90 con 0x7f4f0806d620 2024-03-28T13:21:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.007+0000 7f4f05400640 1 --2- 172.21.15.19:0/3938840232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f080b4890 0x7f4f081f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.008+0000 7f4f05e00640 1 --2- 172.21.15.19:0/3938840232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f0806d620 0x7f4f080a9480 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8000c00 tx=0x7f4ef80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.008+0000 7f4efe000640 1 -- 172.21.15.19:0/3938840232 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8035040 con 0x7f4f0806d620 2024-03-28T13:21:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.008+0000 7f4efe000640 1 -- 172.21.15.19:0/3938840232 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef8039c10 con 0x7f4f0806d620 2024-03-28T13:21:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.008+0000 7f4efe000640 1 -- 172.21.15.19:0/3938840232 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8045480 con 0x7f4f0806d620 2024-03-28T13:21:05.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.008+0000 7f4f07200640 1 -- 172.21.15.19:0/3938840232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f080a9e70 con 0x7f4f0806d620 2024-03-28T13:21:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.008+0000 7f4f07200640 1 -- 172.21.15.19:0/3938840232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f080aa430 con 0x7f4f0806d620 2024-03-28T13:21:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.010+0000 7f4f07200640 1 -- 172.21.15.19:0/3938840232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f08075940 con 0x7f4f0806d620 2024-03-28T13:21:05.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.010+0000 7f4efe000640 1 -- 172.21.15.19:0/3938840232 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4ef8038020 con 0x7f4f0806d620 2024-03-28T13:21:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.011+0000 7f4efe000640 1 --2- 172.21.15.19:0/3938840232 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4ee806c560 0x7f4ee806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.011+0000 7f4efe000640 1 -- 172.21.15.19:0/3938840232 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f4ef80ad430 con 0x7f4f0806d620 2024-03-28T13:21:05.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.012+0000 7f4f05400640 1 --2- 172.21.15.19:0/3938840232 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4ee806c560 0x7f4ee806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.013+0000 7f4f05400640 1 --2- 172.21.15.19:0/3938840232 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4ee806c560 0x7f4ee806ea20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4f080d1180 tx=0x7f4ef4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:05.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.016+0000 7f4efe000640 1 -- 172.21.15.19:0/3938840232 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4ef807ba40 con 0x7f4f0806d620 2024-03-28T13:21:05.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.217+0000 7f4f07200640 1 -- 172.21.15.19:0/3938840232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4f080cc7e0 con 0x7f4f0806d620 2024-03-28T13:21:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.218+0000 7f4efe000640 1 -- 172.21.15.19:0/3938840232 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) ==== 104+0+11 (secure 0 0 0) 0x7f4ef807b340 con 0x7f4f0806d620 2024-03-28T13:21:05.218 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:21:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.221+0000 7f4f07200640 1 -- 172.21.15.19:0/3938840232 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4ee806c560 msgr2=0x7f4ee806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.221+0000 7f4f07200640 1 --2- 172.21.15.19:0/3938840232 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4ee806c560 0x7f4ee806ea20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4f080d1180 tx=0x7f4ef4008040 comp rx=0 tx=0).stop 2024-03-28T13:21:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.221+0000 7f4f07200640 1 -- 172.21.15.19:0/3938840232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f0806d620 msgr2=0x7f4f080a9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.221+0000 7f4f07200640 1 --2- 172.21.15.19:0/3938840232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f0806d620 0x7f4f080a9480 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8000c00 tx=0x7f4ef80046c0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.221+0000 7f4f07200640 1 -- 172.21.15.19:0/3938840232 shutdown_connections 2024-03-28T13:21:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.221+0000 7f4f07200640 1 --2- 172.21.15.19:0/3938840232 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4ee806c560 0x7f4ee806ea20 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.221+0000 7f4f07200640 1 --2- 172.21.15.19:0/3938840232 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f080ce830 0x7f4f080adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.221+0000 7f4f07200640 1 --2- 172.21.15.19:0/3938840232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f080b4890 0x7f4f081f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.221+0000 7f4f07200640 1 --2- 172.21.15.19:0/3938840232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4f0806d620 0x7f4f080a9480 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.221+0000 7f4f07200640 1 -- 172.21.15.19:0/3938840232 >> 172.21.15.19:0/3938840232 conn(0x7f4f080b1ba0 msgr2=0x7f4f080b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:05.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.221+0000 7f4f07200640 1 -- 172.21.15.19:0/3938840232 shutdown_connections 2024-03-28T13:21:05.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.221+0000 7f4f07200640 1 -- 172.21.15.19:0/3938840232 wait complete. 2024-03-28T13:21:05.242 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:21:05.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.463+0000 7fea1f200640 1 Processor -- start 2024-03-28T13:21:05.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.464+0000 7fea1f200640 1 -- start start 2024-03-28T13:21:05.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.465+0000 7fea1f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea2006dd30 0x7fea200cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.465+0000 7fea1f200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea200ce890 con 0x7fea2013f190 2024-03-28T13:21:05.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.465+0000 7fea1f200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea200ceb50 con 0x7fea200cc880 2024-03-28T13:21:05.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.465+0000 7fea1f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fea200cd230 con 0x7fea2006dd30 2024-03-28T13:21:05.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.466+0000 7fea1de00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fea2013f190 0x7fea200cf630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38570/0 (socket says 172.21.15.19:38570) 2024-03-28T13:21:05.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.466+0000 7fea1de00640 1 -- 172.21.15.19:0/3380997932 learned_addr learned my addr 172.21.15.19:0/3380997932 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:05.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.466+0000 7fea1e800640 1 --2- 172.21.15.19:0/3380997932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea2006dd30 0x7fea200cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.466+0000 7fea1ca00640 1 -- 172.21.15.19:0/3380997932 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2350614083 0 0) 0x7fea200ce890 con 0x7fea2013f190 2024-03-28T13:21:05.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.466+0000 7fea1ca00640 1 -- 172.21.15.19:0/3380997932 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea04003520 con 0x7fea2013f190 2024-03-28T13:21:05.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.466+0000 7fea1ca00640 1 -- 172.21.15.19:0/3380997932 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 37746937 0 0) 0x7fea04003520 con 0x7fea2013f190 2024-03-28T13:21:05.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1ca00640 1 -- 172.21.15.19:0/3380997932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea2006dd30 msgr2=0x7fea200cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:05.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1ca00640 1 --2- 172.21.15.19:0/3380997932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea2006dd30 0x7fea200cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1ca00640 1 -- 172.21.15.19:0/3380997932 >> v1:172.21.15.188:6789/0 conn(0x7fea200cc880 legacy=0x7fea2006d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1ca00640 1 -- 172.21.15.19:0/3380997932 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea200cee30 con 0x7fea2013f190 2024-03-28T13:21:05.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1ca00640 1 -- 172.21.15.19:0/3380997932 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fea10001690 con 0x7fea2013f190 2024-03-28T13:21:05.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1ca00640 1 -- 172.21.15.19:0/3380997932 >> v1:172.21.15.19:6789/0 conn(0x7fea2013f190 legacy=0x7fea200cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1ca00640 1 --2- 172.21.15.19:0/3380997932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea040039e0 0x7fea04003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1ca00640 1 --2- 172.21.15.19:0/3380997932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea2006dd30 0x7fea04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1ca00640 1 --2- 172.21.15.19:0/3380997932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea04004d40 0x7fea040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1ca00640 1 -- 172.21.15.19:0/3380997932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea200cd230 con 0x7fea04004d40 2024-03-28T13:21:05.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1ca00640 1 -- 172.21.15.19:0/3380997932 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea0400bad0 con 0x7fea040039e0 2024-03-28T13:21:05.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1ca00640 1 -- 172.21.15.19:0/3380997932 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fea0400bcd0 con 0x7fea2006dd30 2024-03-28T13:21:05.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1e800640 1 --2- 172.21.15.19:0/3380997932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea04004d40 0x7fea040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1d400640 1 --2- 172.21.15.19:0/3380997932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea2006dd30 0x7fea04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1de00640 1 --2- 172.21.15.19:0/3380997932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea040039e0 0x7fea04003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1e800640 1 -- 172.21.15.19:0/3380997932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea2006dd30 msgr2=0x7fea04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:05.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1e800640 1 --2- 172.21.15.19:0/3380997932 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea2006dd30 0x7fea04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1e800640 1 -- 172.21.15.19:0/3380997932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea040039e0 msgr2=0x7fea04003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:05.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.467+0000 7fea1e800640 1 --2- 172.21.15.19:0/3380997932 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea040039e0 0x7fea04003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.468+0000 7fea1e800640 1 -- 172.21.15.19:0/3380997932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea04003520 con 0x7fea04004d40 2024-03-28T13:21:05.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.469+0000 7fea1e800640 1 --2- 172.21.15.19:0/3380997932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea04004d40 0x7fea040051c0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fea18009f90 tx=0x7fea04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d44f3185e5586e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:05.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.469+0000 7fea1ca00640 1 -- 172.21.15.19:0/3380997932 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18039070 con 0x7fea04004d40 2024-03-28T13:21:05.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.469+0000 7fea1ca00640 1 -- 172.21.15.19:0/3380997932 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1802cbd0 con 0x7fea04004d40 2024-03-28T13:21:05.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.469+0000 7fea1ca00640 1 -- 172.21.15.19:0/3380997932 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18009c10 con 0x7fea04004d40 2024-03-28T13:21:05.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.470+0000 7fea1f200640 1 -- 172.21.15.19:0/3380997932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea04004d40 msgr2=0x7fea040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.470+0000 7fea1f200640 1 --2- 172.21.15.19:0/3380997932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea04004d40 0x7fea040051c0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fea18009f90 tx=0x7fea04005830 comp rx=0 tx=0).stop 2024-03-28T13:21:05.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.470+0000 7fea1e800640 1 -- 172.21.15.19:0/3380997932 reap_dead start 2024-03-28T13:21:05.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.470+0000 7fea1f200640 1 -- 172.21.15.19:0/3380997932 shutdown_connections 2024-03-28T13:21:05.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.470+0000 7fea1f200640 1 -- 172.21.15.19:0/3380997932 >> 172.21.15.19:0/3380997932 conn(0x7fea200b1ba0 msgr2=0x7fea200b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:05.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.470+0000 7fea1f200640 1 -- 172.21.15.19:0/3380997932 shutdown_connections 2024-03-28T13:21:05.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.470+0000 7fea1f200640 1 -- 172.21.15.19:0/3380997932 wait complete. 2024-03-28T13:21:05.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.472+0000 7fea1f200640 1 Processor -- start 2024-03-28T13:21:05.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.472+0000 7fea1f200640 1 -- start start 2024-03-28T13:21:05.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.473+0000 7fea1f200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea04004d40 0x7fea2013d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.473+0000 7fea1f200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea201f4290 0x7fea200b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.473+0000 7fea1f200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea201f4710 0x7fea200b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.473+0000 7fea1f200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea2013f570 con 0x7fea201f4710 2024-03-28T13:21:05.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.473+0000 7fea1f200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea20160c70 con 0x7fea201f4290 2024-03-28T13:21:05.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.473+0000 7fea1f200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fea200cccf0 con 0x7fea04004d40 2024-03-28T13:21:05.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.473+0000 7fea1e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea201f4710 0x7fea200b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.473+0000 7fea1e800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea201f4710 0x7fea200b0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33692/0 (socket says 172.21.15.19:33692) 2024-03-28T13:21:05.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.473+0000 7fea1e800640 1 -- 172.21.15.19:0/3564642237 learned_addr learned my addr 172.21.15.19:0/3564642237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:05.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.474+0000 7fea1de00640 1 --2- 172.21.15.19:0/3564642237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea04004d40 0x7fea2013d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.474+0000 7fea1d400640 1 --2- 172.21.15.19:0/3564642237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea201f4290 0x7fea200b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.474+0000 7fea1e800640 1 -- 172.21.15.19:0/3564642237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea04004d40 msgr2=0x7fea2013d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.474+0000 7fea1e800640 1 --2- 172.21.15.19:0/3564642237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea04004d40 0x7fea2013d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.474+0000 7fea1e800640 1 -- 172.21.15.19:0/3564642237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea201f4290 msgr2=0x7fea200b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.474+0000 7fea1e800640 1 --2- 172.21.15.19:0/3564642237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea201f4290 0x7fea200b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.474+0000 7fea1e800640 1 -- 172.21.15.19:0/3564642237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea200b1600 con 0x7fea201f4710 2024-03-28T13:21:05.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.474+0000 7fea1e800640 1 --2- 172.21.15.19:0/3564642237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea201f4710 0x7fea200b0dc0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fea1800e230 tx=0x7fea18009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:05.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.474+0000 7fea16000640 1 -- 172.21.15.19:0/3564642237 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18039070 con 0x7fea201f4710 2024-03-28T13:21:05.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.475+0000 7fea16000640 1 -- 172.21.15.19:0/3564642237 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea1803a040 con 0x7fea201f4710 2024-03-28T13:21:05.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.475+0000 7fea16000640 1 -- 172.21.15.19:0/3564642237 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18003990 con 0x7fea201f4710 2024-03-28T13:21:05.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.475+0000 7fea1f200640 1 -- 172.21.15.19:0/3564642237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea201e5ba0 con 0x7fea201f4710 2024-03-28T13:21:05.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.475+0000 7fea1f200640 1 -- 172.21.15.19:0/3564642237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea201e6160 con 0x7fea201f4710 2024-03-28T13:21:05.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.476+0000 7fea1f200640 1 -- 172.21.15.19:0/3564642237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea20075940 con 0x7fea201f4710 2024-03-28T13:21:05.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.477+0000 7fea16000640 1 -- 172.21.15.19:0/3564642237 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fea18046070 con 0x7fea201f4710 2024-03-28T13:21:05.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.478+0000 7fea16000640 1 --2- 172.21.15.19:0/3564642237 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe9fc06c5b0 0x7fe9fc06ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.479+0000 7fea16000640 1 -- 172.21.15.19:0/3564642237 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7fea180ad250 con 0x7fea201f4710 2024-03-28T13:21:05.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.479+0000 7fea1de00640 1 --2- 172.21.15.19:0/3564642237 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe9fc06c5b0 0x7fe9fc06ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.480+0000 7fea1de00640 1 --2- 172.21.15.19:0/3564642237 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe9fc06c5b0 0x7fe9fc06ea70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fea10001eb0 tx=0x7fea10009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:05.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.483+0000 7fea16000640 1 -- 172.21.15.19:0/3564642237 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fea1807b200 con 0x7fea201f4710 2024-03-28T13:21:05.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.688+0000 7fea1f200640 1 -- 172.21.15.19:0/3564642237 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fea2015dad0 con 0x7fea201f4710 2024-03-28T13:21:05.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.689+0000 7fea16000640 1 -- 172.21.15.19:0/3564642237 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) ==== 100+0+11 (secure 0 0 0) 0x7fea1807f500 con 0x7fea201f4710 2024-03-28T13:21:05.689 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:21:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.691+0000 7fea1f200640 1 -- 172.21.15.19:0/3564642237 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe9fc06c5b0 msgr2=0x7fe9fc06ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.691+0000 7fea1f200640 1 --2- 172.21.15.19:0/3564642237 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe9fc06c5b0 0x7fe9fc06ea70 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fea10001eb0 tx=0x7fea10009040 comp rx=0 tx=0).stop 2024-03-28T13:21:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.691+0000 7fea1f200640 1 -- 172.21.15.19:0/3564642237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea201f4710 msgr2=0x7fea200b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.691+0000 7fea1f200640 1 --2- 172.21.15.19:0/3564642237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea201f4710 0x7fea200b0dc0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fea1800e230 tx=0x7fea18009750 comp rx=0 tx=0).stop 2024-03-28T13:21:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.692+0000 7fea1f200640 1 -- 172.21.15.19:0/3564642237 shutdown_connections 2024-03-28T13:21:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.692+0000 7fea1f200640 1 --2- 172.21.15.19:0/3564642237 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe9fc06c5b0 0x7fe9fc06ea70 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.692+0000 7fea1f200640 1 --2- 172.21.15.19:0/3564642237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea201f4710 0x7fea200b0dc0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.692+0000 7fea1f200640 1 --2- 172.21.15.19:0/3564642237 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea201f4290 0x7fea200b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.692+0000 7fea1f200640 1 --2- 172.21.15.19:0/3564642237 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea04004d40 0x7fea2013d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.692+0000 7fea1f200640 1 -- 172.21.15.19:0/3564642237 >> 172.21.15.19:0/3564642237 conn(0x7fea200b1ba0 msgr2=0x7fea2015e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:05.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.692+0000 7fea1f200640 1 -- 172.21.15.19:0/3564642237 shutdown_connections 2024-03-28T13:21:05.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.692+0000 7fea1f200640 1 -- 172.21.15.19:0/3564642237 wait complete. 2024-03-28T13:21:05.712 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:21:05.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.936+0000 7f41c9e00640 1 Processor -- start 2024-03-28T13:21:05.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.937+0000 7f41c9e00640 1 -- start start 2024-03-28T13:21:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.938+0000 7f41c9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f41c40d0990 0x7f41c40aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.938+0000 7f41c9e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41c40afab0 con 0x7f41c4153580 2024-03-28T13:21:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.938+0000 7f41c9e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41c415cc80 con 0x7f41c40af2e0 2024-03-28T13:21:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.938+0000 7f41c9e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f41c415da00 con 0x7f41c40d0990 2024-03-28T13:21:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.938+0000 7f41bbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f41c40d0990 0x7f41c40aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.939+0000 7f41bbe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f41c40d0990 0x7f41c40aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33706/0 (socket says 172.21.15.19:33706) 2024-03-28T13:21:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.939+0000 7f41c2a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f41c4153580 0x7f41c40d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38574/0 (socket says 172.21.15.19:38574) 2024-03-28T13:21:05.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.939+0000 7f41bbe00640 1 -- 172.21.15.19:0/1689617879 learned_addr learned my addr 172.21.15.19:0/1689617879 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:05.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.939+0000 7f41bbe00640 1 -- 172.21.15.19:0/1689617879 >> v1:172.21.15.188:6789/0 conn(0x7f41c40af2e0 legacy=0x7f41c41600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:05.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.939+0000 7f41bbe00640 1 -- 172.21.15.19:0/1689617879 >> v1:172.21.15.19:6789/0 conn(0x7f41c4153580 legacy=0x7f41c40d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.939+0000 7f41bbe00640 1 -- 172.21.15.19:0/1689617879 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c415cf20 con 0x7f41c40d0990 2024-03-28T13:21:05.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.940+0000 7f41bbe00640 1 --2- 172.21.15.19:0/1689617879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c40d0990 0x7f41c40aeda0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009870 tx=0x7f41ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9075b33ffcf6823 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:05.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.941+0000 7f41c2000640 1 -- 172.21.15.19:0/1689617879 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac035020 con 0x7f41c40d0990 2024-03-28T13:21:05.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.941+0000 7f41c2000640 1 -- 172.21.15.19:0/1689617879 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ac031870 con 0x7f41c40d0990 2024-03-28T13:21:05.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.941+0000 7f41c2000640 1 -- 172.21.15.19:0/1689617879 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac031c40 con 0x7f41c40d0990 2024-03-28T13:21:05.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.941+0000 7f41c9e00640 1 -- 172.21.15.19:0/1689617879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c40d0990 msgr2=0x7f41c40aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.941+0000 7f41c9e00640 1 --2- 172.21.15.19:0/1689617879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c40d0990 0x7f41c40aeda0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009870 tx=0x7f41ac02b910 comp rx=0 tx=0).stop 2024-03-28T13:21:05.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.941+0000 7f41c9e00640 1 -- 172.21.15.19:0/1689617879 shutdown_connections 2024-03-28T13:21:05.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.941+0000 7f41c9e00640 1 --2- 172.21.15.19:0/1689617879 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c40d0990 0x7f41c40aeda0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.941+0000 7f41c9e00640 1 -- 172.21.15.19:0/1689617879 >> 172.21.15.19:0/1689617879 conn(0x7f41c406b750 msgr2=0x7f41c406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:05.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.941+0000 7f41c9e00640 1 -- 172.21.15.19:0/1689617879 shutdown_connections 2024-03-28T13:21:05.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.942+0000 7f41c9e00640 1 -- 172.21.15.19:0/1689617879 wait complete. 2024-03-28T13:21:05.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.943+0000 7f41c9e00640 1 Processor -- start 2024-03-28T13:21:05.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.943+0000 7f41c9e00640 1 -- start start 2024-03-28T13:21:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.944+0000 7f41c9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c40af2e0 0x7f41c40c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.944+0000 7f41c9e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41c40d0990 0x7f41c40cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.944+0000 7f41c9e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41c4153580 0x7f41c4157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.944+0000 7f41c9e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f41c4153960 con 0x7f41c40af2e0 2024-03-28T13:21:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.944+0000 7f41c9e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41c415d4e0 con 0x7f41c40d0990 2024-03-28T13:21:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.944+0000 7f41c9e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f41c40af6c0 con 0x7f41c4153580 2024-03-28T13:21:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.945+0000 7f41c2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c40af2e0 0x7f41c40c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.945+0000 7f41c2a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c40af2e0 0x7f41c40c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33714/0 (socket says 172.21.15.19:33714) 2024-03-28T13:21:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.945+0000 7f41c2a00640 1 -- 172.21.15.19:0/150875882 learned_addr learned my addr 172.21.15.19:0/150875882 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.945+0000 7f41bbe00640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41c40d0990 0x7f41c40cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.945+0000 7f41c3400640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41c4153580 0x7f41c4157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.945+0000 7f41c2a00640 1 -- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41c4153580 msgr2=0x7f41c4157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.945+0000 7f41c2a00640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41c4153580 0x7f41c4157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.945+0000 7f41c2a00640 1 -- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41c40d0990 msgr2=0x7f41c40cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:05.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.945+0000 7f41c2a00640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41c40d0990 0x7f41c40cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:05.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.945+0000 7f41c2a00640 1 -- 172.21.15.19:0/150875882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c40ca6d0 con 0x7f41c40af2e0 2024-03-28T13:21:05.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.945+0000 7f41bbe00640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41c40d0990 0x7f41c40cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:05.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.945+0000 7f41c3400640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41c4153580 0x7f41c4157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:05.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.946+0000 7f41c2a00640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c40af2e0 0x7f41c40c9fd0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c3c0 tx=0x7f41b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:05.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.946+0000 7f41bb400640 1 -- 172.21.15.19:0/150875882 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b40074e0 con 0x7f41c40af2e0 2024-03-28T13:21:05.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.946+0000 7f41bb400640 1 -- 172.21.15.19:0/150875882 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41b4007680 con 0x7f41c40af2e0 2024-03-28T13:21:05.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.946+0000 7f41bb400640 1 -- 172.21.15.19:0/150875882 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b4004a00 con 0x7f41c40af2e0 2024-03-28T13:21:05.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.946+0000 7f41c9e00640 1 -- 172.21.15.19:0/150875882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c40ca9b0 con 0x7f41c40af2e0 2024-03-28T13:21:05.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.946+0000 7f41c9e00640 1 -- 172.21.15.19:0/150875882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c41575b0 con 0x7f41c40af2e0 2024-03-28T13:21:05.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.947+0000 7f41c9e00640 1 -- 172.21.15.19:0/150875882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c40685c0 con 0x7f41c40af2e0 2024-03-28T13:21:05.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.948+0000 7f41bb400640 1 -- 172.21.15.19:0/150875882 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f41b401a020 con 0x7f41c40af2e0 2024-03-28T13:21:05.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.949+0000 7f41bb400640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41a006c530 0x7f41a006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:05.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.949+0000 7f41bb400640 1 -- 172.21.15.19:0/150875882 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f41b408ae70 con 0x7f41c40af2e0 2024-03-28T13:21:05.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.950+0000 7f41bbe00640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41a006c530 0x7f41a006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:05.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.951+0000 7f41bbe00640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41a006c530 0x7f41a006e9f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f41ac036000 tx=0x7f41ac008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:05.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:05.954+0000 7f41bb400640 1 -- 172.21.15.19:0/150875882 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f41b4059490 con 0x7f41c40af2e0 2024-03-28T13:21:06.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.167+0000 7f41c9e00640 1 -- 172.21.15.19:0/150875882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f41c406eca0 con 0x7f41c40af2e0 2024-03-28T13:21:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.168+0000 7f41bb400640 1 -- 172.21.15.19:0/150875882 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v119) ==== 80+0+0 (secure 0 0 0) 0x7f41b4058d90 con 0x7f41c40af2e0 2024-03-28T13:21:06.168 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-03-28T13:21:06.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.171+0000 7f41c9e00640 1 -- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41a006c530 msgr2=0x7f41a006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.171+0000 7f41c9e00640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41a006c530 0x7f41a006e9f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f41ac036000 tx=0x7f41ac008030 comp rx=0 tx=0).stop 2024-03-28T13:21:06.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.171+0000 7f41c9e00640 1 -- 172.21.15.19:0/150875882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c40af2e0 msgr2=0x7f41c40c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.171+0000 7f41c9e00640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c40af2e0 0x7f41c40c9fd0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f41b400c3c0 tx=0x7f41b400c880 comp rx=0 tx=0).stop 2024-03-28T13:21:06.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.171+0000 7f41c9e00640 1 -- 172.21.15.19:0/150875882 shutdown_connections 2024-03-28T13:21:06.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.171+0000 7f41c9e00640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f41a006c530 0x7f41a006e9f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.171+0000 7f41c9e00640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41c4153580 0x7f41c4157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.171+0000 7f41c9e00640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41c40d0990 0x7f41c40cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.171+0000 7f41c9e00640 1 --2- 172.21.15.19:0/150875882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f41c40af2e0 0x7f41c40c9fd0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.171+0000 7f41c9e00640 1 -- 172.21.15.19:0/150875882 >> 172.21.15.19:0/150875882 conn(0x7f41c406b750 msgr2=0x7f41c40d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:06.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.171+0000 7f41c9e00640 1 -- 172.21.15.19:0/150875882 shutdown_connections 2024-03-28T13:21:06.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.171+0000 7f41c9e00640 1 -- 172.21.15.19:0/150875882 wait complete. 2024-03-28T13:21:06.188 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:21:06.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:06.189 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:06.336 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:21:06.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:06.336 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:21:06.381 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-03-28T13:21:06.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.611+0000 7f8fa5200640 1 Processor -- start 2024-03-28T13:21:06.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.611+0000 7f8fa5200640 1 -- start start 2024-03-28T13:21:06.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.613+0000 7f8fa5200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa013f190 0x7f8fa00cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.613+0000 7f8fa5200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8fa006dd30 0x7f8fa00cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.613+0000 7f8fa5200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fa00ce890 con 0x7f8fa00cc880 2024-03-28T13:21:06.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.613+0000 7f8fa5200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f8fa00cd200 con 0x7f8fa006dd30 2024-03-28T13:21:06.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.613+0000 7f8fa5200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8fa00ceb50 con 0x7f8fa013f190 2024-03-28T13:21:06.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.613+0000 7f8f9fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8fa006dd30 0x7f8fa00cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.613+0000 7f8f9fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f8fa006dd30 0x7f8fa00cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33726/0 (socket says 172.21.15.19:33726) 2024-03-28T13:21:06.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.613+0000 7f8f9fe00640 1 -- 172.21.15.19:0/295913294 learned_addr learned my addr 172.21.15.19:0/295913294 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:06.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.613+0000 7f8f9f400640 1 --2- 172.21.15.19:0/295913294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa013f190 0x7f8fa00cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.614+0000 7f8f9fe00640 1 -- 172.21.15.19:0/295913294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa013f190 msgr2=0x7f8fa00cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:06.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.614+0000 7f8f9fe00640 1 --2- 172.21.15.19:0/295913294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa013f190 0x7f8fa00cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.614+0000 7f8f9fe00640 1 -- 172.21.15.19:0/295913294 >> v1:172.21.15.188:6789/0 conn(0x7f8fa00cc880 legacy=0x7f8fa006d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.614+0000 7f8f9fe00640 1 -- 172.21.15.19:0/295913294 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa00cee30 con 0x7f8fa006dd30 2024-03-28T13:21:06.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.615+0000 7f8f9fe00640 1 --2- 172.21.15.19:0/295913294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa006dd30 0x7f8fa00cccc0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009e50 tx=0x7f8f9802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f410f43d3a0a48e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:06.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.615+0000 7f8f9e000640 1 -- 172.21.15.19:0/295913294 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9802caf0 con 0x7f8fa006dd30 2024-03-28T13:21:06.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.615+0000 7f8f9e000640 1 -- 172.21.15.19:0/295913294 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f9802cc90 con 0x7f8fa006dd30 2024-03-28T13:21:06.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.615+0000 7f8f9e000640 1 -- 172.21.15.19:0/295913294 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f98034610 con 0x7f8fa006dd30 2024-03-28T13:21:06.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.616+0000 7f8fa5200640 1 -- 172.21.15.19:0/295913294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa006dd30 msgr2=0x7f8fa00cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.616+0000 7f8fa5200640 1 --2- 172.21.15.19:0/295913294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa006dd30 0x7f8fa00cccc0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009e50 tx=0x7f8f9802bf50 comp rx=0 tx=0).stop 2024-03-28T13:21:06.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.616+0000 7f8fa5200640 1 -- 172.21.15.19:0/295913294 shutdown_connections 2024-03-28T13:21:06.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.616+0000 7f8fa5200640 1 --2- 172.21.15.19:0/295913294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa006dd30 0x7f8fa00cccc0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.616+0000 7f8fa5200640 1 --2- 172.21.15.19:0/295913294 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa013f190 0x7f8fa00cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.616+0000 7f8fa5200640 1 -- 172.21.15.19:0/295913294 >> 172.21.15.19:0/295913294 conn(0x7f8fa00b1ba0 msgr2=0x7f8fa00b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:06.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.616+0000 7f8fa5200640 1 -- 172.21.15.19:0/295913294 shutdown_connections 2024-03-28T13:21:06.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.616+0000 7f8fa5200640 1 -- 172.21.15.19:0/295913294 wait complete. 2024-03-28T13:21:06.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.618+0000 7f8fa5200640 1 Processor -- start 2024-03-28T13:21:06.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.618+0000 7f8fa5200640 1 -- start start 2024-03-28T13:21:06.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.619+0000 7f8fa5200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa006dd30 0x7f8fa0138f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.619+0000 7f8fa5200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa00cc880 0x7f8fa015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.619+0000 7f8fa5200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa013f190 0x7f8fa01f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.619+0000 7f8fa5200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa013f570 con 0x7f8fa00cc880 2024-03-28T13:21:06.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.619+0000 7f8fa5200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa0160c70 con 0x7f8fa006dd30 2024-03-28T13:21:06.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.619+0000 7f8fa5200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8fa00cfb70 con 0x7f8fa013f190 2024-03-28T13:21:06.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.620+0000 7f8f9ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa00cc880 0x7f8fa015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.620+0000 7f8f9ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa00cc880 0x7f8fa015b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33732/0 (socket says 172.21.15.19:33732) 2024-03-28T13:21:06.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.620+0000 7f8f9ea00640 1 -- 172.21.15.19:0/2740595199 learned_addr learned my addr 172.21.15.19:0/2740595199 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:06.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.620+0000 7f8f9fe00640 1 --2- 172.21.15.19:0/2740595199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa013f190 0x7f8fa01f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.620+0000 7f8f9f400640 1 --2- 172.21.15.19:0/2740595199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa006dd30 0x7f8fa0138f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.620+0000 7f8f9ea00640 1 -- 172.21.15.19:0/2740595199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa013f190 msgr2=0x7f8fa01f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.620+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/2740595199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa013f190 0x7f8fa01f4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.620+0000 7f8f9ea00640 1 -- 172.21.15.19:0/2740595199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa006dd30 msgr2=0x7f8fa0138f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.620+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/2740595199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa006dd30 0x7f8fa0138f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.620+0000 7f8f9ea00640 1 -- 172.21.15.19:0/2740595199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fa0139930 con 0x7f8fa00cc880 2024-03-28T13:21:06.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.620+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/2740595199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa00cc880 0x7f8fa015b4d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00c450 tx=0x7f8f8c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:06.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.621+0000 7f8f97400640 1 -- 172.21.15.19:0/2740595199 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c0052d0 con 0x7f8fa00cc880 2024-03-28T13:21:06.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.621+0000 7f8f97400640 1 -- 172.21.15.19:0/2740595199 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c005470 con 0x7f8fa00cc880 2024-03-28T13:21:06.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.621+0000 7f8f97400640 1 -- 172.21.15.19:0/2740595199 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c012870 con 0x7f8fa00cc880 2024-03-28T13:21:06.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.621+0000 7f8fa5200640 1 -- 172.21.15.19:0/2740595199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fa0139c10 con 0x7f8fa00cc880 2024-03-28T13:21:06.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.621+0000 7f8fa5200640 1 -- 172.21.15.19:0/2740595199 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fa013a1d0 con 0x7f8fa00cc880 2024-03-28T13:21:06.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.623+0000 7f8f97400640 1 -- 172.21.15.19:0/2740595199 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f8f8c012a10 con 0x7f8fa00cc880 2024-03-28T13:21:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.625+0000 7f8f97400640 1 --2- 172.21.15.19:0/2740595199 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8006c530 0x7f8f8006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.625+0000 7f8f97400640 1 -- 172.21.15.19:0/2740595199 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f8f8c08b3d0 con 0x7f8fa00cc880 2024-03-28T13:21:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.626+0000 7f8fa9461740 1 Processor -- start 2024-03-28T13:21:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.626+0000 7f8f9f400640 1 --2- 172.21.15.19:0/2740595199 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8006c530 0x7f8f8006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.626+0000 7f8fa9461740 1 -- start start 2024-03-28T13:21:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.626+0000 7f8fa9461740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec6720 0x55decfe996c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.626+0000 7f8fa9461740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6b00 0x55decfec94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8fa9461740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfe99c00 0x55decfec99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8fa9461740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55decfe046b0 con 0x55decfec6b00 2024-03-28T13:21:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8fa9461740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55decfdac750 con 0x55decfec6720 2024-03-28T13:21:06.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8fa9461740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55decfe00660 con 0x55decfe99c00 2024-03-28T13:21:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8f9f400640 1 --2- 172.21.15.19:0/2740595199 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8006c530 0x7f8f8006e9f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009ad0 tx=0x7f8f90008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8f9ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6b00 0x55decfec94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8f9ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6b00 0x55decfec94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33734/0 (socket says 172.21.15.19:33734) 2024-03-28T13:21:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8f9ea00640 1 -- 172.21.15.19:0/1822899195 learned_addr learned my addr 172.21.15.19:0/1822899195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8f9f400640 1 --2- 172.21.15.19:0/1822899195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec6720 0x55decfe996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8f9fe00640 1 --2- 172.21.15.19:0/1822899195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfe99c00 0x55decfec99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8f9ea00640 1 -- 172.21.15.19:0/1822899195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfe99c00 msgr2=0x55decfec99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/1822899195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfe99c00 0x55decfec99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8f9ea00640 1 -- 172.21.15.19:0/1822899195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec6720 msgr2=0x55decfe996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/1822899195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec6720 0x55decfe996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8f9ea00640 1 -- 172.21.15.19:0/1822899195 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55decfbe27d0 con 0x55decfec6b00 2024-03-28T13:21:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.627+0000 7f8f9fe00640 1 --2- 172.21.15.19:0/1822899195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfe99c00 0x55decfec99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.628+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/1822899195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6b00 0x55decfec94b0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c000f30 tx=0x7f8f8c059a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bad00968765179 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.628+0000 7f8f8be00640 1 -- 172.21.15.19:0/1822899195 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c061e30 con 0x55decfec6b00 2024-03-28T13:21:06.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.628+0000 7f8f8be00640 1 -- 172.21.15.19:0/1822899195 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c057550 con 0x55decfec6b00 2024-03-28T13:21:06.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.628+0000 7f8f8be00640 1 -- 172.21.15.19:0/1822899195 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c012d00 con 0x55decfec6b00 2024-03-28T13:21:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.628+0000 7f8fa9461740 1 -- 172.21.15.19:0/1822899195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6b00 msgr2=0x55decfec94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.628+0000 7f8fa9461740 1 --2- 172.21.15.19:0/1822899195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6b00 0x55decfec94b0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c000f30 tx=0x7f8f8c059a90 comp rx=0 tx=0).stop 2024-03-28T13:21:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.629+0000 7f8fa9461740 1 -- 172.21.15.19:0/1822899195 shutdown_connections 2024-03-28T13:21:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.629+0000 7f8fa9461740 1 --2- 172.21.15.19:0/1822899195 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfe99c00 0x55decfec99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.629+0000 7f8fa9461740 1 --2- 172.21.15.19:0/1822899195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6b00 0x55decfec94b0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.629+0000 7f8fa9461740 1 --2- 172.21.15.19:0/1822899195 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec6720 0x55decfe996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.629+0000 7f8fa9461740 1 -- 172.21.15.19:0/1822899195 >> 172.21.15.19:0/1822899195 conn(0x55decfe9b930 msgr2=0x55decfee44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.629+0000 7f8fa9461740 1 -- 172.21.15.19:0/1822899195 shutdown_connections 2024-03-28T13:21:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.629+0000 7f8fa9461740 1 -- 172.21.15.19:0/1822899195 wait complete. 2024-03-28T13:21:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.629+0000 7f8fa9461740 1 -- start start 2024-03-28T13:21:06.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.630+0000 7f8fa9461740 1 Processor -- start 2024-03-28T13:21:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8fa9461740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6720 0x55decfebbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8fa9461740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec6b00 0x55decfec4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8fa9461740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec5290 0x55decfec3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8fa9461740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55decfe9dfd0 con 0x55decfec6720 2024-03-28T13:21:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8fa9461740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55decfdbfa80 con 0x55decfec5290 2024-03-28T13:21:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8fa9461740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55decfea0810 con 0x55decfec6b00 2024-03-28T13:21:06.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8fa9461740 10 client.0 _ll_register_callbacks cb 0x55decfdbf470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:21:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8f9f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6720 0x55decfebbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8f9f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6720 0x55decfebbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33736/0 (socket says 172.21.15.19:33736) 2024-03-28T13:21:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8f9f400640 1 -- 172.21.15.19:0/23597589 learned_addr learned my addr 172.21.15.19:0/23597589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/23597589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec6b00 0x55decfec4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8f9fe00640 1 --2- 172.21.15.19:0/23597589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec5290 0x55decfec3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8f9f400640 1 -- 172.21.15.19:0/23597589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec6b00 msgr2=0x55decfec4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8f9f400640 1 --2- 172.21.15.19:0/23597589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec6b00 0x55decfec4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8f9f400640 1 -- 172.21.15.19:0/23597589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec5290 msgr2=0x55decfec3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8f9f400640 1 --2- 172.21.15.19:0/23597589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec5290 0x55decfec3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.631+0000 7f8f9f400640 1 -- 172.21.15.19:0/23597589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55decfb183f0 con 0x55decfec6720 2024-03-28T13:21:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.632+0000 7f8f9f400640 1 --2- 172.21.15.19:0/23597589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6720 0x55decfebbbb0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f8f900157c0 tx=0x7f8f900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2920326add1c016a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.632+0000 7f8f88c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:21:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.632+0000 7f8f88c00640 1 -- 172.21.15.19:0/23597589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90025070 con 0x55decfec6720 2024-03-28T13:21:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.632+0000 7f8f88c00640 1 -- 172.21.15.19:0/23597589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f8f90015d40 con 0x55decfec6720 2024-03-28T13:21:06.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.632+0000 7f8fa9461740 1 -- 172.21.15.19:0/23597589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f8c057550 con 0x55decfec6720 2024-03-28T13:21:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.632+0000 7f8f88c00640 1 -- 172.21.15.19:0/23597589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f8f90030050 con 0x55decfec6720 2024-03-28T13:21:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.632+0000 7f8fa9461740 10 client.8898 fetch_fsmap learned FSMap version 99 2024-03-28T13:21:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.633+0000 7f8fa9461740 1 -- 172.21.15.19:0/23597589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55decfad80f0 con 0x55decfec6720 2024-03-28T13:21:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.633+0000 7f8f88c00640 1 -- 172.21.15.19:0/23597589 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c057550 con 0x55decfec6720 2024-03-28T13:21:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.633+0000 7f8f88c00640 1 -- 172.21.15.19:0/23597589 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7f8f90033020 con 0x55decfec6720 2024-03-28T13:21:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.633+0000 7f8fa9461740 10 client.8898 fetch_fsmap finished waiting for FSMap version 99 2024-03-28T13:21:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.633+0000 7f8fa9461740 10 client.8898 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-03-28T13:21:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.633+0000 7f8fa9461740 20 client.8898 populate_metadata read hostname 'smithi019' 2024-03-28T13:21:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.633+0000 7f8fa9461740 1 --2- 172.21.15.19:0/23597589 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55decfee2530 0x55decfee2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.633+0000 7f8fa9461740 4 client.8898 mds_command: new command op to 8703 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:21:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.633+0000 7f8fa9461740 1 -- 172.21.15.19:0/23597589 --> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55decfe11bf0 con 0x55decfee2530 2024-03-28T13:21:06.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.633+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/23597589 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55decfee2530 0x55decfee2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.634+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/23597589 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55decfee2530 0x55decfee2960 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3107c67d27044fe server_cookie=8d989fed87f71ff0 in_seq=0 out_seq=0 2024-03-28T13:21:06.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.634+0000 7f8f88c00640 10 client.8898 ms_handle_connect on v2:172.21.15.188:6834/2088538359 2024-03-28T13:21:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.636+0000 7f8f88c00640 1 -- 172.21.15.19:0/23597589 <== mds.0 v2:172.21.15.188:6834/2088538359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55decfe11bf0 con 0x55decfee2530 2024-03-28T13:21:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.636+0000 7f8f88c00640 10 client.8898 handle_command_reply: tid=0 2024-03-28T13:21:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.636+0000 7f8fa9461740 1 client.8898 shutdown 2024-03-28T13:21:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.636+0000 7f8fa9461740 1 -- 172.21.15.19:0/23597589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6720 msgr2=0x55decfebbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.636+0000 7f8fa9461740 1 --2- 172.21.15.19:0/23597589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6720 0x55decfebbbb0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f8f900157c0 tx=0x7f8f900046c0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.636+0000 7f8fa9461740 1 -- 172.21.15.19:0/23597589 shutdown_connections 2024-03-28T13:21:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.636+0000 7f8fa9461740 1 --2- 172.21.15.19:0/23597589 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55decfee2530 0x55decfee2960 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.636+0000 7f8fa9461740 1 --2- 172.21.15.19:0/23597589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec5290 0x55decfec3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.636+0000 7f8fa9461740 1 --2- 172.21.15.19:0/23597589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec6b00 0x55decfec4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.636+0000 7f8fa9461740 1 --2- 172.21.15.19:0/23597589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6720 0x55decfebbbb0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.636+0000 7f8f88c00640 0 client.8898 ms_handle_reset on v2:172.21.15.188:6834/2088538359 2024-03-28T13:21:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.636+0000 7f8fa9461740 1 -- 172.21.15.19:0/23597589 >> 172.21.15.19:0/23597589 conn(0x55decfe99c00 msgr2=0x55decfe9d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.636+0000 7f8fa9461740 1 -- 172.21.15.19:0/23597589 shutdown_connections 2024-03-28T13:21:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.637+0000 7f8fa9461740 1 -- 172.21.15.19:0/23597589 wait complete. 2024-03-28T13:21:06.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.637+0000 7f8fa9461740 20 client.8898 trim_cache size 0 max 16384 2024-03-28T13:21:06.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.653+0000 7f8fa9461740 1 Processor -- start 2024-03-28T13:21:06.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.653+0000 7f8fa9461740 1 -- start start 2024-03-28T13:21:06.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.654+0000 7f8fa9461740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec5290 0x55decfee42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.654+0000 7f8fa9461740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6720 0x55decff55740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.654+0000 7f8fa9461740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec6b00 0x55decfee3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.654+0000 7f8fa9461740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa0200940 con 0x55decfec6720 2024-03-28T13:21:06.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.654+0000 7f8fa9461740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa01ff9f0 con 0x55decfec6b00 2024-03-28T13:21:06.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.654+0000 7f8fa9461740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8fa00af980 con 0x55decfec5290 2024-03-28T13:21:06.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.654+0000 7f8f9ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6720 0x55decff55740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.654+0000 7f8f9ea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6720 0x55decff55740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33744/0 (socket says 172.21.15.19:33744) 2024-03-28T13:21:06.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.654+0000 7f8f9ea00640 1 -- 172.21.15.19:0/656569600 learned_addr learned my addr 172.21.15.19:0/656569600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:06.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.654+0000 7f8f9f400640 1 --2- 172.21.15.19:0/656569600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec5290 0x55decfee42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.654+0000 7f8f9ea00640 1 -- 172.21.15.19:0/656569600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec5290 msgr2=0x55decfee42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:06.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.654+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/656569600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec5290 0x55decfee42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.654+0000 7f8f9ea00640 1 -- 172.21.15.19:0/656569600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec6b00 msgr2=0x55decfee3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:06.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.654+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/656569600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec6b00 0x55decfee3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.654+0000 7f8f9ea00640 1 -- 172.21.15.19:0/656569600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55decfe95640 con 0x55decfec6720 2024-03-28T13:21:06.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.655+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/656569600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6720 0x55decff55740 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0591e0 tx=0x7f8f8c05c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a216a725f21734ca server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:06.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.655+0000 7f8f8b400640 1 -- 172.21.15.19:0/656569600 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c057550 con 0x55decfec6720 2024-03-28T13:21:06.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.655+0000 7f8f8b400640 1 -- 172.21.15.19:0/656569600 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c012d00 con 0x55decfec6720 2024-03-28T13:21:06.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.655+0000 7f8f8b400640 1 -- 172.21.15.19:0/656569600 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c08b040 con 0x55decfec6720 2024-03-28T13:21:06.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.655+0000 7f8fa9461740 1 -- 172.21.15.19:0/656569600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6720 msgr2=0x55decff55740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.655+0000 7f8fa9461740 1 --2- 172.21.15.19:0/656569600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6720 0x55decff55740 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c0591e0 tx=0x7f8f8c05c000 comp rx=0 tx=0).stop 2024-03-28T13:21:06.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.656+0000 7f8fa9461740 1 -- 172.21.15.19:0/656569600 shutdown_connections 2024-03-28T13:21:06.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.656+0000 7f8fa9461740 1 --2- 172.21.15.19:0/656569600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec6b00 0x55decfee3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.656+0000 7f8fa9461740 1 --2- 172.21.15.19:0/656569600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec6720 0x55decff55740 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.656+0000 7f8fa9461740 1 --2- 172.21.15.19:0/656569600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec5290 0x55decfee42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.656+0000 7f8fa9461740 1 -- 172.21.15.19:0/656569600 >> 172.21.15.19:0/656569600 conn(0x55decfe99c00 msgr2=0x55decfebb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:06.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.656+0000 7f8fa9461740 1 -- 172.21.15.19:0/656569600 shutdown_connections 2024-03-28T13:21:06.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.656+0000 7f8fa9461740 1 -- 172.21.15.19:0/656569600 wait complete. 2024-03-28T13:21:06.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.656+0000 7f8fa9461740 1 -- start start 2024-03-28T13:21:06.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.657+0000 7f8fa9461740 1 Processor -- start 2024-03-28T13:21:06.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.657+0000 7f8fa9461740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec5290 0x55decfefe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.657+0000 7f8fa9461740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec6720 0x55decfefee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.657+0000 7f8fa9461740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec6b00 0x55decfe985b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.658+0000 7f8fa9461740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8fa00affd0 con 0x55decfec5290 2024-03-28T13:21:06.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.658+0000 7f8fa9461740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55decfec3d40 con 0x55decfec6720 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.658+0000 7f8fa9461740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55decff5fc20 con 0x55decfec6b00 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.658+0000 7f8fa9461740 10 client.0 _ll_register_callbacks cb 0x7f8f90015440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.658+0000 7f8f9f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec5290 0x55decfefe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.658+0000 7f8f9f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec5290 0x55decfefe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33758/0 (socket says 172.21.15.19:33758) 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.658+0000 7f8f9f400640 1 -- 172.21.15.19:0/1230485748 learned_addr learned my addr 172.21.15.19:0/1230485748 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.658+0000 7f8f9fe00640 1 --2- 172.21.15.19:0/1230485748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec6b00 0x55decfe985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.658+0000 7f8f9f400640 1 -- 172.21.15.19:0/1230485748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec6b00 msgr2=0x55decfe985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.658+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/1230485748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec6720 0x55decfefee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.658+0000 7f8f9f400640 1 --2- 172.21.15.19:0/1230485748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec6b00 0x55decfe985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.658+0000 7f8f9f400640 1 -- 172.21.15.19:0/1230485748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec6720 msgr2=0x55decfefee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.658+0000 7f8f9f400640 1 --2- 172.21.15.19:0/1230485748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec6720 0x55decfefee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.658+0000 7f8f9f400640 1 -- 172.21.15.19:0/1230485748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55decfecb760 con 0x55decfec5290 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.658+0000 7f8f9fe00640 1 --2- 172.21.15.19:0/1230485748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec6b00 0x55decfe985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.659+0000 7f8f9f400640 1 --2- 172.21.15.19:0/1230485748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec5290 0x55decfefe8f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f8f9002abf0 tx=0x7f8f90016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a09b325bd9107a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.659+0000 7f8f88c00640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:21:06.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.659+0000 7f8f88c00640 1 -- 172.21.15.19:0/1230485748 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f90038070 con 0x55decfec5290 2024-03-28T13:21:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.659+0000 7f8f88c00640 1 -- 172.21.15.19:0/1230485748 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f8f90015d40 con 0x55decfec5290 2024-03-28T13:21:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.659+0000 7f8fa9461740 1 -- 172.21.15.19:0/1230485748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f8c012d00 con 0x55decfec5290 2024-03-28T13:21:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.659+0000 7f8f88c00640 1 -- 172.21.15.19:0/1230485748 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f8f90025040 con 0x55decfec5290 2024-03-28T13:21:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.659+0000 7f8fa9461740 10 client.8904 fetch_fsmap learned FSMap version 99 2024-03-28T13:21:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.659+0000 7f8fa9461740 1 -- 172.21.15.19:0/1230485748 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x55decff5f9c0 con 0x55decfec5290 2024-03-28T13:21:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.659+0000 7f8f88c00640 1 -- 172.21.15.19:0/1230485748 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c012d00 con 0x55decfec5290 2024-03-28T13:21:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.659+0000 7f8f88c00640 1 -- 172.21.15.19:0/1230485748 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 99) ==== 3494+0+0 (secure 0 0 0) 0x7f8f90048020 con 0x55decfec5290 2024-03-28T13:21:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.659+0000 7f8fa9461740 10 client.8904 fetch_fsmap finished waiting for FSMap version 99 2024-03-28T13:21:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.659+0000 7f8fa9461740 10 client.8904 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-03-28T13:21:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.659+0000 7f8fa9461740 20 client.8904 populate_metadata read hostname 'smithi019' 2024-03-28T13:21:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.659+0000 7f8fa9461740 1 --2- 172.21.15.19:0/1230485748 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55decff55740 0x55decfe98af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.660+0000 7f8fa9461740 4 client.8904 mds_command: new command op to 8703 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-28T13:21:06.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.660+0000 7f8fa9461740 1 -- 172.21.15.19:0/1230485748 --> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55decfe0bf40 con 0x55decff55740 2024-03-28T13:21:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.660+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/1230485748 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55decff55740 0x55decfe98af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.661+0000 7f8f9ea00640 1 --2- 172.21.15.19:0/1230485748 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55decff55740 0x55decfe98af0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6a7a8cc77da28c0 server_cookie=6e6793c46a8ffdcb in_seq=0 out_seq=0 2024-03-28T13:21:06.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.661+0000 7f8f88c00640 10 client.8904 ms_handle_connect on v2:172.21.15.188:6834/2088538359 2024-03-28T13:21:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.679+0000 7f8f88c00640 1 -- 172.21.15.19:0/1230485748 <== mds.0 v2:172.21.15.188:6834/2088538359 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55decfe0bf40 con 0x55decff55740 2024-03-28T13:21:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.679+0000 7f8f88c00640 10 client.8904 handle_command_reply: tid=0 2024-03-28T13:21:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.679+0000 7f8fa9461740 1 client.8904 shutdown 2024-03-28T13:21:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.679+0000 7f8fa9461740 1 -- 172.21.15.19:0/1230485748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec5290 msgr2=0x55decfefe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.679+0000 7f8fa9461740 1 --2- 172.21.15.19:0/1230485748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec5290 0x55decfefe8f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f8f9002abf0 tx=0x7f8f90016000 comp rx=0 tx=0).stop 2024-03-28T13:21:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.679+0000 7f8fa9461740 1 -- 172.21.15.19:0/1230485748 shutdown_connections 2024-03-28T13:21:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.679+0000 7f8fa9461740 1 --2- 172.21.15.19:0/1230485748 >> [v2:172.21.15.188:6834/2088538359,v1:172.21.15.188:6836/2088538359] conn(0x55decff55740 0x55decfe98af0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.679+0000 7f8fa9461740 1 --2- 172.21.15.19:0/1230485748 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55decfec6b00 0x55decfe985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.679+0000 7f8fa9461740 1 --2- 172.21.15.19:0/1230485748 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55decfec6720 0x55decfefee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.679+0000 7f8fa9461740 1 --2- 172.21.15.19:0/1230485748 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55decfec5290 0x55decfefe8f0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.679+0000 7f8fa9461740 1 -- 172.21.15.19:0/1230485748 >> 172.21.15.19:0/1230485748 conn(0x55decfe99c00 msgr2=0x55decfee5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.679+0000 7f8f88c00640 0 client.8904 ms_handle_reset on v2:172.21.15.188:6834/2088538359 2024-03-28T13:21:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.679+0000 7f8fa9461740 1 -- 172.21.15.19:0/1230485748 shutdown_connections 2024-03-28T13:21:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.679+0000 7f8fa9461740 1 -- 172.21.15.19:0/1230485748 wait complete. 2024-03-28T13:21:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.680+0000 7f8fa9461740 20 client.8904 trim_cache size 0 max 16384 2024-03-28T13:21:06.680 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:21:06.680 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "", 2024-03-28T13:21:06.681 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-03-28T13:21:06.681 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:21:06.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.680+0000 7f8fa5200640 1 -- 172.21.15.19:0/2740595199 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8006c530 msgr2=0x7f8f8006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.680+0000 7f8fa5200640 1 --2- 172.21.15.19:0/2740595199 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8006c530 0x7f8f8006e9f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009ad0 tx=0x7f8f90008040 comp rx=0 tx=0).stop 2024-03-28T13:21:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.680+0000 7f8fa5200640 1 -- 172.21.15.19:0/2740595199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa00cc880 msgr2=0x7f8fa015b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.680+0000 7f8fa5200640 1 --2- 172.21.15.19:0/2740595199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa00cc880 0x7f8fa015b4d0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00c450 tx=0x7f8f8c00c910 comp rx=0 tx=0).stop 2024-03-28T13:21:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.680+0000 7f8fa5200640 1 -- 172.21.15.19:0/2740595199 shutdown_connections 2024-03-28T13:21:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.680+0000 7f8fa5200640 1 --2- 172.21.15.19:0/2740595199 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f8f8006c530 0x7f8f8006e9f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.680+0000 7f8fa5200640 1 --2- 172.21.15.19:0/2740595199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa013f190 0x7f8fa01f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.680+0000 7f8fa5200640 1 --2- 172.21.15.19:0/2740595199 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fa00cc880 0x7f8fa015b4d0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.680+0000 7f8fa5200640 1 --2- 172.21.15.19:0/2740595199 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa006dd30 0x7f8fa0138f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.680+0000 7f8fa5200640 1 -- 172.21.15.19:0/2740595199 >> 172.21.15.19:0/2740595199 conn(0x7f8fa00b1ba0 msgr2=0x7f8fa006d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.681+0000 7f8fa5200640 1 -- 172.21.15.19:0/2740595199 shutdown_connections 2024-03-28T13:21:06.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.681+0000 7f8fa5200640 1 -- 172.21.15.19:0/2740595199 wait complete. 2024-03-28T13:21:06.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:21:06.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.911+0000 7faf1a200640 1 Processor -- start 2024-03-28T13:21:06.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.912+0000 7faf1a200640 1 -- start start 2024-03-28T13:21:06.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.913+0000 7faf1a200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faf1415e420 0x7faf140b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.913+0000 7faf1a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf140b3460 0x7faf140d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.913+0000 7faf1a200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf140d2fd0 con 0x7faf140d18c0 2024-03-28T13:21:06.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.913+0000 7faf1a200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7faf140d1cf0 con 0x7faf1415e420 2024-03-28T13:21:06.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.913+0000 7faf1a200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7faf140cfa70 con 0x7faf140b3460 2024-03-28T13:21:06.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.914+0000 7faf19800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7faf140d18c0 0x7faf14161ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38580/0 (socket says 172.21.15.19:38580) 2024-03-28T13:21:06.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.914+0000 7faf19800640 1 -- 172.21.15.19:0/1909981518 learned_addr learned my addr 172.21.15.19:0/1909981518 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:06.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.914+0000 7faf18e00640 1 --2- 172.21.15.19:0/1909981518 >> v2:172.21.15.188:3300/0 conn(0x7faf1415e420 0x7faf140b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.914+0000 7faf13e00640 1 --2- 172.21.15.19:0/1909981518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf140b3460 0x7faf140d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.914+0000 7faf13400640 1 -- 172.21.15.19:0/1909981518 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1912887277 0 0) 0x7faf140d2fd0 con 0x7faf140d18c0 2024-03-28T13:21:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.914+0000 7faf13400640 1 -- 172.21.15.19:0/1909981518 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faef8003520 con 0x7faf140d18c0 2024-03-28T13:21:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf13400640 1 -- 172.21.15.19:0/1909981518 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2111816327 0 0) 0x7faef8003520 con 0x7faf140d18c0 2024-03-28T13:21:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf13400640 1 -- 172.21.15.19:0/1909981518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf140b3460 msgr2=0x7faf140d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf13400640 1 --2- 172.21.15.19:0/1909981518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf140b3460 0x7faf140d1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf13400640 1 -- 172.21.15.19:0/1909981518 >> v2:172.21.15.188:3300/0 conn(0x7faf1415e420 msgr2=0x7faf140b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf13400640 1 --2- 172.21.15.19:0/1909981518 >> v2:172.21.15.188:3300/0 conn(0x7faf1415e420 0x7faf140b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf13400640 1 -- 172.21.15.19:0/1909981518 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf140cf790 con 0x7faf140d18c0 2024-03-28T13:21:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf13400640 1 -- 172.21.15.19:0/1909981518 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7faf0c0019f0 con 0x7faf140d18c0 2024-03-28T13:21:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf18e00640 1 --2- 172.21.15.19:0/1909981518 >> v2:172.21.15.188:3300/0 conn(0x7faf1415e420 0x7faf140b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:21:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf13400640 1 -- 172.21.15.19:0/1909981518 >> v1:172.21.15.19:6789/0 conn(0x7faf140d18c0 legacy=0x7faf14161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf13400640 1 --2- 172.21.15.19:0/1909981518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf140b3460 0x7faef80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf13400640 1 --2- 172.21.15.19:0/1909981518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faef8003f80 0x7faef8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf13400640 1 --2- 172.21.15.19:0/1909981518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faef8004d40 0x7faef80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf13400640 1 -- 172.21.15.19:0/1909981518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf140d1cf0 con 0x7faef8004d40 2024-03-28T13:21:06.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf13400640 1 -- 172.21.15.19:0/1909981518 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faf140cfa70 con 0x7faef8003f80 2024-03-28T13:21:06.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf13400640 1 -- 172.21.15.19:0/1909981518 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7faef800b9c0 con 0x7faf140b3460 2024-03-28T13:21:06.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf19800640 1 --2- 172.21.15.19:0/1909981518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faef8004d40 0x7faef80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.915+0000 7faf18e00640 1 --2- 172.21.15.19:0/1909981518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf140b3460 0x7faef80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.916+0000 7faf13e00640 1 --2- 172.21.15.19:0/1909981518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faef8003f80 0x7faef8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.916+0000 7faf19800640 1 -- 172.21.15.19:0/1909981518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf140b3460 msgr2=0x7faef80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:06.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.916+0000 7faf19800640 1 --2- 172.21.15.19:0/1909981518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf140b3460 0x7faef80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.916+0000 7faf19800640 1 -- 172.21.15.19:0/1909981518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faef8003f80 msgr2=0x7faef8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:06.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.916+0000 7faf19800640 1 --2- 172.21.15.19:0/1909981518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faef8003f80 0x7faef8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.916+0000 7faf19800640 1 -- 172.21.15.19:0/1909981518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef8003520 con 0x7faef8004d40 2024-03-28T13:21:06.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.916+0000 7faf13e00640 1 --2- 172.21.15.19:0/1909981518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faef8003f80 0x7faef8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:21:06.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.917+0000 7faf19800640 1 --2- 172.21.15.19:0/1909981518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faef8004d40 0x7faef80051c0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00a7c0 tx=0x7faef8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa48ccec7a1bf8a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.918+0000 7faf13400640 1 -- 172.21.15.19:0/1909981518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00a410 con 0x7faef8004d40 2024-03-28T13:21:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.918+0000 7faf13400640 1 -- 172.21.15.19:0/1909981518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c034630 con 0x7faef8004d40 2024-03-28T13:21:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.918+0000 7faf13400640 1 -- 172.21.15.19:0/1909981518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c034a20 con 0x7faef8004d40 2024-03-28T13:21:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.918+0000 7faf1a200640 1 -- 172.21.15.19:0/1909981518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faef8004d40 msgr2=0x7faef80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.918+0000 7faf1a200640 1 --2- 172.21.15.19:0/1909981518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faef8004d40 0x7faef80051c0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00a7c0 tx=0x7faef8005830 comp rx=0 tx=0).stop 2024-03-28T13:21:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.918+0000 7faf19800640 1 -- 172.21.15.19:0/1909981518 reap_dead start 2024-03-28T13:21:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.918+0000 7faf1a200640 1 -- 172.21.15.19:0/1909981518 shutdown_connections 2024-03-28T13:21:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.918+0000 7faf1a200640 1 -- 172.21.15.19:0/1909981518 >> 172.21.15.19:0/1909981518 conn(0x7faf14067670 msgr2=0x7faf14067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.918+0000 7faf1a200640 1 -- 172.21.15.19:0/1909981518 shutdown_connections 2024-03-28T13:21:06.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.918+0000 7faf1a200640 1 -- 172.21.15.19:0/1909981518 wait complete. 2024-03-28T13:21:06.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.920+0000 7faf1a200640 1 Processor -- start 2024-03-28T13:21:06.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.920+0000 7faf1a200640 1 -- start start 2024-03-28T13:21:06.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.921+0000 7faf1a200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf1414b980 0x7faf14157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.921+0000 7faf1a200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf1415e980 0x7faf140cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.921+0000 7faf1a200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf140cdbc0 0x7faf14156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.921+0000 7faf1a200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf1415e800 con 0x7faf140cdbc0 2024-03-28T13:21:06.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.921+0000 7faf1a200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faf140d3550 con 0x7faf1414b980 2024-03-28T13:21:06.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.921+0000 7faf1a200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7faf140cf420 con 0x7faf1415e980 2024-03-28T13:21:06.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.922+0000 7faf19800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf140cdbc0 0x7faf14156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.922+0000 7faf19800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf140cdbc0 0x7faf14156500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33764/0 (socket says 172.21.15.19:33764) 2024-03-28T13:21:06.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.922+0000 7faf19800640 1 -- 172.21.15.19:0/3686100167 learned_addr learned my addr 172.21.15.19:0/3686100167 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:06.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.922+0000 7faf13e00640 1 --2- 172.21.15.19:0/3686100167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf1415e980 0x7faf140cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.922+0000 7faf18e00640 1 --2- 172.21.15.19:0/3686100167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf1414b980 0x7faf14157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.922+0000 7faf19800640 1 -- 172.21.15.19:0/3686100167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf1415e980 msgr2=0x7faf140cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.922+0000 7faf19800640 1 --2- 172.21.15.19:0/3686100167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf1415e980 0x7faf140cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.922+0000 7faf19800640 1 -- 172.21.15.19:0/3686100167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf1414b980 msgr2=0x7faf14157ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:06.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.922+0000 7faf19800640 1 --2- 172.21.15.19:0/3686100167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf1414b980 0x7faf14157ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:06.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.922+0000 7faf19800640 1 -- 172.21.15.19:0/3686100167 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf14158020 con 0x7faf140cdbc0 2024-03-28T13:21:06.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.923+0000 7faf19800640 1 --2- 172.21.15.19:0/3686100167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf140cdbc0 0x7faf14156500 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7faf0c002230 tx=0x7faf0c001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:06.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.923+0000 7faf10c00640 1 -- 172.21.15.19:0/3686100167 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c039040 con 0x7faf140cdbc0 2024-03-28T13:21:06.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.923+0000 7faf10c00640 1 -- 172.21.15.19:0/3686100167 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf0c043070 con 0x7faf140cdbc0 2024-03-28T13:21:06.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.923+0000 7faf10c00640 1 -- 172.21.15.19:0/3686100167 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c001f70 con 0x7faf140cdbc0 2024-03-28T13:21:06.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.923+0000 7faf1a200640 1 -- 172.21.15.19:0/3686100167 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf141582c0 con 0x7faf140cdbc0 2024-03-28T13:21:06.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.923+0000 7faf1a200640 1 -- 172.21.15.19:0/3686100167 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf14158880 con 0x7faf140cdbc0 2024-03-28T13:21:06.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.924+0000 7faf1a200640 1 -- 172.21.15.19:0/3686100167 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf140ce040 con 0x7faf140cdbc0 2024-03-28T13:21:06.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.925+0000 7faf10c00640 1 -- 172.21.15.19:0/3686100167 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7faf0c034a20 con 0x7faf140cdbc0 2024-03-28T13:21:06.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.926+0000 7faf10c00640 1 --2- 172.21.15.19:0/3686100167 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faef406c560 0x7faef406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:06.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.926+0000 7faf10c00640 1 -- 172.21.15.19:0/3686100167 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7faf0c0ad4e0 con 0x7faf140cdbc0 2024-03-28T13:21:06.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.927+0000 7faf18e00640 1 --2- 172.21.15.19:0/3686100167 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faef406c560 0x7faef406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:06.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.928+0000 7faf18e00640 1 --2- 172.21.15.19:0/3686100167 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faef406c560 0x7faef406ea20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faf140b48d0 tx=0x7faf04008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:06.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:06.931+0000 7faf10c00640 1 -- 172.21.15.19:0/3686100167 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7faf0c07bb90 con 0x7faf140cdbc0 2024-03-28T13:21:07.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07.165+0000 7faf1a200640 1 -- 172.21.15.19:0/3686100167 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faf140b7000 con 0x7faf140cdbc0 2024-03-28T13:21:07.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07.165+0000 7faf10c00640 1 -- 172.21.15.19:0/3686100167 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) ==== 53+0+73 (secure 0 0 0) 0x7faf0c07b490 con 0x7faf140cdbc0 2024-03-28T13:21:07.165 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:21:07.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07.168+0000 7faf1a200640 1 -- 172.21.15.19:0/3686100167 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faef406c560 msgr2=0x7faef406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:07.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07.168+0000 7faf1a200640 1 --2- 172.21.15.19:0/3686100167 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faef406c560 0x7faef406ea20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7faf140b48d0 tx=0x7faf04008040 comp rx=0 tx=0).stop 2024-03-28T13:21:07.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07.168+0000 7faf1a200640 1 -- 172.21.15.19:0/3686100167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf140cdbc0 msgr2=0x7faf14156500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:07.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07.168+0000 7faf1a200640 1 --2- 172.21.15.19:0/3686100167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf140cdbc0 0x7faf14156500 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7faf0c002230 tx=0x7faf0c001360 comp rx=0 tx=0).stop 2024-03-28T13:21:07.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07.168+0000 7faf1a200640 1 -- 172.21.15.19:0/3686100167 shutdown_connections 2024-03-28T13:21:07.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07.168+0000 7faf1a200640 1 --2- 172.21.15.19:0/3686100167 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7faef406c560 0x7faef406ea20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:07.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07.168+0000 7faf1a200640 1 --2- 172.21.15.19:0/3686100167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf140cdbc0 0x7faf14156500 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:07.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07.168+0000 7faf1a200640 1 --2- 172.21.15.19:0/3686100167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf1415e980 0x7faf140cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:07.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07.168+0000 7faf1a200640 1 --2- 172.21.15.19:0/3686100167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf1414b980 0x7faf14157ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:07.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07.168+0000 7faf1a200640 1 -- 172.21.15.19:0/3686100167 >> 172.21.15.19:0/3686100167 conn(0x7faf14067670 msgr2=0x7faf140d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:07.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07.168+0000 7faf1a200640 1 -- 172.21.15.19:0/3686100167 shutdown_connections 2024-03-28T13:21:07.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07.168+0000 7faf1a200640 1 -- 172.21.15.19:0/3686100167 wait complete. 2024-03-28T13:21:07.186 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-28T13:21:07.186 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:21:07.186 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:21:07.186 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:21:07.186 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:07.187 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-28T13:21:07.187 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-28T13:21:07.187 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:21:07.187 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:21:07.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:07.187 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:21:07.243 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:21:07.243 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:21:07.244 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:21:07.244 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:07.244 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:21:07.244 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:07.244 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:21:07.244 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:07.244 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:21:07.244 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:21:07.244 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:07.245 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:21:07.245 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:21:07.245 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:07.245 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:21:07.245 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:21:07.246 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41887sec preferred_lft 41887sec 2024-03-28T13:21:07.246 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:21:07.246 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:07.246 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:21:07.246 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:07.246 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:21:07.246 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:21:07.247 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:07.247 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:21:07.247 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:07.247 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:21:07.247 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:07.247 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:21:07.247 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:21:07.247 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:21:07.247 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:07.248 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:21:07.248 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether e6:17:c5:8e:2b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:21:07.248 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::e417:c5ff:fe8e:2bcf/64 scope link 2024-03-28T13:21:07.248 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:07.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:07.249 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:21:07.341 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:21:07.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:07.345 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:21:07.398 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:21:07.398 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:21:07.398 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:21:07.398 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:21:07.398 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:21:07.399 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:21:07.399 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:21:07.399 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:21:07.399 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:21:07.399 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:21:07.399 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:21:07.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:21:07.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:07.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:07.663 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:21:07.716 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:21:07.716 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:21:07.717 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:21:07.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:07.717 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:21:07.819 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:21:07.819 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:07.874 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:21:07.874 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:21:07.874 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:07.926 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:21:07.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:07.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:08.077 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:08.078 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:21:08.078 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:21:08.101 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:08.101 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:21:08.101 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:21:08.102 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:21:08.102 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:21:08.128 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:08.129 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:21:08.129 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:21:08.129 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:21:08.129 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:21:08.129 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:21:08.129 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:21:08.129 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:21:08.131 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:21:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:21:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:21:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:21:08.131 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:21:08.135 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:21:08.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:08.135 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:08.227 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:21:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:21:08.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:08.354 INFO:teuthology.orchestra.run.smithi019.stdout:client6452 2024-03-28T13:21:08.355 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:08.355 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:08.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:08.412 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-03-28T13:21:08.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:08.479 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-03-28T13:21:08.602 INFO:teuthology.orchestra.run.smithi019.stderr:8+0 records in 2024-03-28T13:21:08.602 INFO:teuthology.orchestra.run.smithi019.stderr:8+0 records out 2024-03-28T13:21:08.602 INFO:teuthology.orchestra.run.smithi019.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0592704 s, 142 MB/s 2024-03-28T13:21:08.604 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:08.604 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:08.604 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:08.604 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:08.604 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-03-28T13:21:08.604 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:08.756 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627779 2024-03-28T13:21:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:08.770 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:08.770 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:08.770 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:08.771 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:08.771 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-03-28T13:21:08.771 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:08.964 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627780 2024-03-28T13:21:08.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:08.979 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:08.979 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:09.077 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:09.077 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:09.133 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:21:09.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.360+0000 7f44e7200640 1 Processor -- start 2024-03-28T13:21:09.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.360+0000 7f44e7200640 1 -- start start 2024-03-28T13:21:09.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.362+0000 7f44e7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44e8147a40 0x7f44e80cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:09.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.362+0000 7f44e7200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44e80ce380 0x7f44e806da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.362+0000 7f44e7200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44e80ce810 con 0x7f44e80d01c0 2024-03-28T13:21:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.362+0000 7f44e7200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f44e8162b00 con 0x7f44e80ce380 2024-03-28T13:21:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.362+0000 7f44e7200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f44e8161f30 con 0x7f44e8147a40 2024-03-28T13:21:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.363+0000 7f44e5e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44e8147a40 0x7f44e80cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.363+0000 7f44e6800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44e80ce380 0x7f44e806da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.363+0000 7f44e6800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44e80ce380 0x7f44e806da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:47186/0 (socket says 172.21.15.19:47186) 2024-03-28T13:21:09.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.363+0000 7f44e6800640 1 -- 172.21.15.19:0/388403631 learned_addr learned my addr 172.21.15.19:0/388403631 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.363+0000 7f44e6800640 1 -- 172.21.15.19:0/388403631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44e8147a40 msgr2=0x7f44e80cfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.363+0000 7f44e6800640 1 --2- 172.21.15.19:0/388403631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44e8147a40 0x7f44e80cfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.363+0000 7f44e6800640 1 -- 172.21.15.19:0/388403631 >> v1:172.21.15.188:6789/0 conn(0x7f44e80d01c0 legacy=0x7f44e80cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:09.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.363+0000 7f44e6800640 1 -- 172.21.15.19:0/388403631 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e81621d0 con 0x7f44e80ce380 2024-03-28T13:21:09.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.365+0000 7f44e6800640 1 --2- 172.21.15.19:0/388403631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44e80ce380 0x7f44e806da80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009e50 tx=0x7f44e002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cdb03fa6a59bbc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.365+0000 7f44e4a00640 1 -- 172.21.15.19:0/388403631 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e002cb00 con 0x7f44e80ce380 2024-03-28T13:21:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.365+0000 7f44e4a00640 1 -- 172.21.15.19:0/388403631 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e002cca0 con 0x7f44e80ce380 2024-03-28T13:21:09.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.365+0000 7f44e4a00640 1 -- 172.21.15.19:0/388403631 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0034610 con 0x7f44e80ce380 2024-03-28T13:21:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.366+0000 7f44e7200640 1 -- 172.21.15.19:0/388403631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44e80ce380 msgr2=0x7f44e806da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.366+0000 7f44e7200640 1 --2- 172.21.15.19:0/388403631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44e80ce380 0x7f44e806da80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009e50 tx=0x7f44e002bf60 comp rx=0 tx=0).stop 2024-03-28T13:21:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.366+0000 7f44e7200640 1 -- 172.21.15.19:0/388403631 shutdown_connections 2024-03-28T13:21:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.366+0000 7f44e7200640 1 --2- 172.21.15.19:0/388403631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44e80ce380 0x7f44e806da80 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.366+0000 7f44e7200640 1 --2- 172.21.15.19:0/388403631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44e8147a40 0x7f44e80cfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.366+0000 7f44e7200640 1 -- 172.21.15.19:0/388403631 >> 172.21.15.19:0/388403631 conn(0x7f44e80ccad0 msgr2=0x7f44e80cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.366+0000 7f44e7200640 1 -- 172.21.15.19:0/388403631 shutdown_connections 2024-03-28T13:21:09.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.366+0000 7f44e7200640 1 -- 172.21.15.19:0/388403631 wait complete. 2024-03-28T13:21:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.368+0000 7f44e7200640 1 Processor -- start 2024-03-28T13:21:09.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.369+0000 7f44e7200640 1 -- start start 2024-03-28T13:21:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.370+0000 7f44e7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44e80ce380 0x7f44e815da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.370+0000 7f44e7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e80d01c0 0x7f44e80b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.370+0000 7f44e7200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44e8147a40 0x7f44e80b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.370+0000 7f44e7200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f44e80d0620 con 0x7f44e80d01c0 2024-03-28T13:21:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.370+0000 7f44e7200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8162790 con 0x7f44e8147a40 2024-03-28T13:21:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.370+0000 7f44e7200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f44e8147e20 con 0x7f44e80ce380 2024-03-28T13:21:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.370+0000 7f44e5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e80d01c0 0x7f44e80b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.370+0000 7f44e5400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e80d01c0 0x7f44e80b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33778/0 (socket says 172.21.15.19:33778) 2024-03-28T13:21:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.370+0000 7f44e5400640 1 -- 172.21.15.19:0/2598119924 learned_addr learned my addr 172.21.15.19:0/2598119924 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.370+0000 7f44e5e00640 1 --2- 172.21.15.19:0/2598119924 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44e80ce380 0x7f44e815da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:09.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.370+0000 7f44e6800640 1 --2- 172.21.15.19:0/2598119924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44e8147a40 0x7f44e80b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:09.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.371+0000 7f44e5400640 1 -- 172.21.15.19:0/2598119924 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44e80ce380 msgr2=0x7f44e815da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.371+0000 7f44e5400640 1 --2- 172.21.15.19:0/2598119924 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44e80ce380 0x7f44e815da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.371+0000 7f44e5400640 1 -- 172.21.15.19:0/2598119924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44e8147a40 msgr2=0x7f44e80b4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.371+0000 7f44e5400640 1 --2- 172.21.15.19:0/2598119924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44e8147a40 0x7f44e80b4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.371+0000 7f44e5400640 1 -- 172.21.15.19:0/2598119924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e80b3610 con 0x7f44e80d01c0 2024-03-28T13:21:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.371+0000 7f44e5400640 1 --2- 172.21.15.19:0/2598119924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e80d01c0 0x7f44e80b2bc0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f44d400c400 tx=0x7f44d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.371+0000 7f44de000640 1 -- 172.21.15.19:0/2598119924 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d40052d0 con 0x7f44e80d01c0 2024-03-28T13:21:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.371+0000 7f44de000640 1 -- 172.21.15.19:0/2598119924 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44d4005490 con 0x7f44e80d01c0 2024-03-28T13:21:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.371+0000 7f44de000640 1 -- 172.21.15.19:0/2598119924 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d4007260 con 0x7f44e80d01c0 2024-03-28T13:21:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.371+0000 7f44e7200640 1 -- 172.21.15.19:0/2598119924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e80b38f0 con 0x7f44e80d01c0 2024-03-28T13:21:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.372+0000 7f44e7200640 1 -- 172.21.15.19:0/2598119924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e80b3eb0 con 0x7f44e80d01c0 2024-03-28T13:21:09.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.373+0000 7f44e7200640 1 -- 172.21.15.19:0/2598119924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44e8075940 con 0x7f44e80d01c0 2024-03-28T13:21:09.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.374+0000 7f44de000640 1 -- 172.21.15.19:0/2598119924 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f44d4007400 con 0x7f44e80d01c0 2024-03-28T13:21:09.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.374+0000 7f44de000640 1 --2- 172.21.15.19:0/2598119924 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f44c806c530 0x7f44c806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:09.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.375+0000 7f44de000640 1 -- 172.21.15.19:0/2598119924 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f44d408ac70 con 0x7f44e80d01c0 2024-03-28T13:21:09.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.375+0000 7f44e5e00640 1 --2- 172.21.15.19:0/2598119924 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f44c806c530 0x7f44c806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:09.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.376+0000 7f44e5e00640 1 --2- 172.21.15.19:0/2598119924 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f44c806c530 0x7f44c806e9f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009a30 tx=0x7f44d8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:09.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.379+0000 7f44de000640 1 -- 172.21.15.19:0/2598119924 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f44d4059270 con 0x7f44e80d01c0 2024-03-28T13:21:09.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.593+0000 7f44e7200640 1 -- 172.21.15.19:0/2598119924 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44e806e840 con 0x7f44e80d01c0 2024-03-28T13:21:09.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.595+0000 7f44de000640 1 -- 172.21.15.19:0/2598119924 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16683 (secure 0 0 0) 0x7f44d4058b70 con 0x7f44e80d01c0 2024-03-28T13:21:09.595 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:09.595 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":119,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:20:54.450982+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:20:50.748212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-28T13:20:51.888444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:21:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.598+0000 7f44e7200640 1 -- 172.21.15.19:0/2598119924 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f44c806c530 msgr2=0x7f44c806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.598+0000 7f44e7200640 1 --2- 172.21.15.19:0/2598119924 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f44c806c530 0x7f44c806e9f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009a30 tx=0x7f44d8008040 comp rx=0 tx=0).stop 2024-03-28T13:21:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.598+0000 7f44e7200640 1 -- 172.21.15.19:0/2598119924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e80d01c0 msgr2=0x7f44e80b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.598+0000 7f44e7200640 1 --2- 172.21.15.19:0/2598119924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e80d01c0 0x7f44e80b2bc0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f44d400c400 tx=0x7f44d400c8c0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.598+0000 7f44e7200640 1 -- 172.21.15.19:0/2598119924 shutdown_connections 2024-03-28T13:21:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.598+0000 7f44e7200640 1 --2- 172.21.15.19:0/2598119924 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f44c806c530 0x7f44c806e9f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.598+0000 7f44e7200640 1 --2- 172.21.15.19:0/2598119924 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44e8147a40 0x7f44e80b4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.598+0000 7f44e7200640 1 --2- 172.21.15.19:0/2598119924 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f44e80d01c0 0x7f44e80b2bc0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.598+0000 7f44e7200640 1 --2- 172.21.15.19:0/2598119924 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44e80ce380 0x7f44e815da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.598+0000 7f44e7200640 1 -- 172.21.15.19:0/2598119924 >> 172.21.15.19:0/2598119924 conn(0x7f44e80ccad0 msgr2=0x7f44e8160940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.598+0000 7f44e7200640 1 -- 172.21.15.19:0/2598119924 shutdown_connections 2024-03-28T13:21:09.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.598+0000 7f44e7200640 1 -- 172.21.15.19:0/2598119924 wait complete. 2024-03-28T13:21:09.619 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:20:50.748212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:20:51.888444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:21:09.619 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:21:09.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.845+0000 7f46bf200640 1 Processor -- start 2024-03-28T13:21:09.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.845+0000 7f46bf200640 1 -- start start 2024-03-28T13:21:09.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.846+0000 7f46bf200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f46c00b69c0 0x7f46c0168f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:09.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.847+0000 7f46bf200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46c00d2710 con 0x7f46c00d37b0 2024-03-28T13:21:09.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.847+0000 7f46bf200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46c00d29f0 con 0x7f46c015e4c0 2024-03-28T13:21:09.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.847+0000 7f46bf200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f46c015e8a0 con 0x7f46c00b69c0 2024-03-28T13:21:09.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.847+0000 7f46bd400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f46c00d37b0 0x7f46c00b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38588/0 (socket says 172.21.15.19:38588) 2024-03-28T13:21:09.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.847+0000 7f46bd400640 1 -- 172.21.15.19:0/3282495853 learned_addr learned my addr 172.21.15.19:0/3282495853 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:09.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.847+0000 7f46be800640 1 --2- 172.21.15.19:0/3282495853 >> v2:172.21.15.188:3300/0 conn(0x7f46c00b69c0 0x7f46c0168f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:09.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.847+0000 7f46bca00640 1 -- 172.21.15.19:0/3282495853 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 873636937 0 0) 0x7f46c00d2710 con 0x7f46c00d37b0 2024-03-28T13:21:09.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.848+0000 7f46bca00640 1 -- 172.21.15.19:0/3282495853 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46a4003520 con 0x7f46c00d37b0 2024-03-28T13:21:09.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.848+0000 7f46bca00640 1 -- 172.21.15.19:0/3282495853 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2519276467 0 0) 0x7f46a4003520 con 0x7f46c00d37b0 2024-03-28T13:21:09.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.848+0000 7f46bca00640 1 -- 172.21.15.19:0/3282495853 >> v2:172.21.15.188:3300/0 conn(0x7f46c00b69c0 msgr2=0x7f46c0168f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:09.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.848+0000 7f46bca00640 1 --2- 172.21.15.19:0/3282495853 >> v2:172.21.15.188:3300/0 conn(0x7f46c00b69c0 0x7f46c0168f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.848+0000 7f46bca00640 1 -- 172.21.15.19:0/3282495853 >> v1:172.21.15.188:6789/0 conn(0x7f46c015e4c0 legacy=0x7f46c00b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:09.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.848+0000 7f46bca00640 1 -- 172.21.15.19:0/3282495853 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c00d2cd0 con 0x7f46c00d37b0 2024-03-28T13:21:09.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.848+0000 7f46bca00640 1 -- 172.21.15.19:0/3282495853 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f46a8001670 con 0x7f46c00d37b0 2024-03-28T13:21:09.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.848+0000 7f46bca00640 1 -- 172.21.15.19:0/3282495853 >> v1:172.21.15.19:6789/0 conn(0x7f46c00d37b0 legacy=0x7f46c00b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:09.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.848+0000 7f46bca00640 1 --2- 172.21.15.19:0/3282495853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46a4003a00 0x7f46a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:09.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.848+0000 7f46bca00640 1 --2- 172.21.15.19:0/3282495853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46a4004800 0x7f46a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:09.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.848+0000 7f46bca00640 1 --2- 172.21.15.19:0/3282495853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a40051c0 0x7f46a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:09.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.848+0000 7f46bca00640 1 -- 172.21.15.19:0/3282495853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46a400c090 con 0x7f46a4004800 2024-03-28T13:21:09.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.848+0000 7f46bca00640 1 -- 172.21.15.19:0/3282495853 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f46a400c240 con 0x7f46a4003a00 2024-03-28T13:21:09.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.848+0000 7f46bca00640 1 -- 172.21.15.19:0/3282495853 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f46a400c440 con 0x7f46a40051c0 2024-03-28T13:21:09.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.848+0000 7f46bd400640 1 --2- 172.21.15.19:0/3282495853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46a4004800 0x7f46a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:09.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.849+0000 7f46bde00640 1 --2- 172.21.15.19:0/3282495853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46a4003a00 0x7f46a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:09.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.849+0000 7f46be800640 1 --2- 172.21.15.19:0/3282495853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a40051c0 0x7f46a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:09.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.849+0000 7f46bd400640 1 -- 172.21.15.19:0/3282495853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a40051c0 msgr2=0x7f46a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:09.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.849+0000 7f46bd400640 1 --2- 172.21.15.19:0/3282495853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a40051c0 0x7f46a400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.849+0000 7f46bd400640 1 -- 172.21.15.19:0/3282495853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46a4003a00 msgr2=0x7f46a4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:09.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.849+0000 7f46bd400640 1 --2- 172.21.15.19:0/3282495853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46a4003a00 0x7f46a4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.849+0000 7f46bd400640 1 -- 172.21.15.19:0/3282495853 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a4003520 con 0x7f46a4004800 2024-03-28T13:21:09.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.849+0000 7f46be800640 1 --2- 172.21.15.19:0/3282495853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a40051c0 0x7f46a400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:09.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.849+0000 7f46be800640 1 -- 172.21.15.19:0/3282495853 reap_dead start 2024-03-28T13:21:09.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.850+0000 7f46bd400640 1 --2- 172.21.15.19:0/3282495853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46a4004800 0x7f46a4004c80 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f46a800c0e0 tx=0x7f46a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1dd3ed1811c1048 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:09.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.850+0000 7f46bca00640 1 -- 172.21.15.19:0/3282495853 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8007c20 con 0x7f46a4004800 2024-03-28T13:21:09.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.850+0000 7f46bca00640 1 -- 172.21.15.19:0/3282495853 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46a8030070 con 0x7f46a4004800 2024-03-28T13:21:09.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.850+0000 7f46bca00640 1 -- 172.21.15.19:0/3282495853 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8005d70 con 0x7f46a4004800 2024-03-28T13:21:09.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.851+0000 7f46bf200640 1 -- 172.21.15.19:0/3282495853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46a4004800 msgr2=0x7f46a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:09.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.851+0000 7f46bf200640 1 --2- 172.21.15.19:0/3282495853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46a4004800 0x7f46a4004c80 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f46a800c0e0 tx=0x7f46a4005830 comp rx=0 tx=0).stop 2024-03-28T13:21:09.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.851+0000 7f46bf200640 1 -- 172.21.15.19:0/3282495853 shutdown_connections 2024-03-28T13:21:09.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.851+0000 7f46bf200640 1 --2- 172.21.15.19:0/3282495853 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46a4004800 0x7f46a4004c80 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.851+0000 7f46bf200640 1 -- 172.21.15.19:0/3282495853 >> 172.21.15.19:0/3282495853 conn(0x7f46c00676a0 msgr2=0x7f46c0067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:09.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.851+0000 7f46bf200640 1 -- 172.21.15.19:0/3282495853 shutdown_connections 2024-03-28T13:21:09.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.851+0000 7f46bf200640 1 -- 172.21.15.19:0/3282495853 wait complete. 2024-03-28T13:21:09.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.853+0000 7f46bf200640 1 Processor -- start 2024-03-28T13:21:09.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.853+0000 7f46bf200640 1 -- start start 2024-03-28T13:21:09.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.854+0000 7f46bf200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a4004800 0x7f46c01524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:09.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.854+0000 7f46bf200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46c00cb3a0 0x7f46c00c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:09.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.854+0000 7f46bf200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c0160a70 0x7f46c0160f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:09.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.854+0000 7f46bf200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46c00d3320 con 0x7f46c0160a70 2024-03-28T13:21:09.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.854+0000 7f46bf200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f46c00d4040 con 0x7f46c00cb3a0 2024-03-28T13:21:09.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.854+0000 7f46bf200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f46c00d2350 con 0x7f46a4004800 2024-03-28T13:21:09.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.854+0000 7f46be800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c0160a70 0x7f46c0160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:09.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.855+0000 7f46be800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c0160a70 0x7f46c0160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33792/0 (socket says 172.21.15.19:33792) 2024-03-28T13:21:09.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.855+0000 7f46be800640 1 -- 172.21.15.19:0/2788006908 learned_addr learned my addr 172.21.15.19:0/2788006908 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:09.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.855+0000 7f46bde00640 1 --2- 172.21.15.19:0/2788006908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a4004800 0x7f46c01524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:09.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.855+0000 7f46be800640 1 -- 172.21.15.19:0/2788006908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a4004800 msgr2=0x7f46c01524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:09.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.855+0000 7f46be800640 1 --2- 172.21.15.19:0/2788006908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a4004800 0x7f46c01524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.855+0000 7f46be800640 1 -- 172.21.15.19:0/2788006908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46c00cb3a0 msgr2=0x7f46c00c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:09.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.855+0000 7f46be800640 1 --2- 172.21.15.19:0/2788006908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46c00cb3a0 0x7f46c00c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:09.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.855+0000 7f46be800640 1 -- 172.21.15.19:0/2788006908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c00cb8a0 con 0x7f46c0160a70 2024-03-28T13:21:09.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.855+0000 7f46be800640 1 --2- 172.21.15.19:0/2788006908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c0160a70 0x7f46c0160f20 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f46c00d1730 tx=0x7f46b800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:09.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.855+0000 7f46b6000640 1 -- 172.21.15.19:0/2788006908 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8016420 con 0x7f46c0160a70 2024-03-28T13:21:09.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.856+0000 7f46b6000640 1 -- 172.21.15.19:0/2788006908 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46b801b070 con 0x7f46c0160a70 2024-03-28T13:21:09.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.856+0000 7f46bf200640 1 -- 172.21.15.19:0/2788006908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c00cbb80 con 0x7f46c0160a70 2024-03-28T13:21:09.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.856+0000 7f46b6000640 1 -- 172.21.15.19:0/2788006908 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8016820 con 0x7f46c0160a70 2024-03-28T13:21:09.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.856+0000 7f46bf200640 1 -- 172.21.15.19:0/2788006908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c00cc140 con 0x7f46c0160a70 2024-03-28T13:21:09.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.857+0000 7f46bf200640 1 -- 172.21.15.19:0/2788006908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46c0075940 con 0x7f46c0160a70 2024-03-28T13:21:09.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.858+0000 7f46b6000640 1 -- 172.21.15.19:0/2788006908 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f46b80169c0 con 0x7f46c0160a70 2024-03-28T13:21:09.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.859+0000 7f46b6000640 1 --2- 172.21.15.19:0/2788006908 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f469c06c560 0x7f469c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:09.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.859+0000 7f46b6000640 1 -- 172.21.15.19:0/2788006908 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f46b808c690 con 0x7f46c0160a70 2024-03-28T13:21:09.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.860+0000 7f46bde00640 1 --2- 172.21.15.19:0/2788006908 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f469c06c560 0x7f469c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:09.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.861+0000 7f46bde00640 1 --2- 172.21.15.19:0/2788006908 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f469c06c560 0x7f469c06ea20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f46b0001510 tx=0x7f46b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:09.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:09.864+0000 7f46b6000640 1 -- 172.21.15.19:0/2788006908 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f46b8025020 con 0x7f46c0160a70 2024-03-28T13:21:10.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.068+0000 7f46bf200640 1 -- 172.21.15.19:0/2788006908 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f46c00cf5b0 con 0x7f46c0160a70 2024-03-28T13:21:10.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.069+0000 7f46b6000640 1 -- 172.21.15.19:0/2788006908 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) ==== 93+0+10 (secure 0 0 0) 0x7f46b805aca0 con 0x7f46c0160a70 2024-03-28T13:21:10.069 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:21:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.071+0000 7f46bf200640 1 -- 172.21.15.19:0/2788006908 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f469c06c560 msgr2=0x7f469c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.071+0000 7f46bf200640 1 --2- 172.21.15.19:0/2788006908 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f469c06c560 0x7f469c06ea20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f46b0001510 tx=0x7f46b0009040 comp rx=0 tx=0).stop 2024-03-28T13:21:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.071+0000 7f46bf200640 1 -- 172.21.15.19:0/2788006908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c0160a70 msgr2=0x7f46c0160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.071+0000 7f46bf200640 1 --2- 172.21.15.19:0/2788006908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c0160a70 0x7f46c0160f20 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f46c00d1730 tx=0x7f46b800f520 comp rx=0 tx=0).stop 2024-03-28T13:21:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.072+0000 7f46bf200640 1 -- 172.21.15.19:0/2788006908 shutdown_connections 2024-03-28T13:21:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.072+0000 7f46bf200640 1 --2- 172.21.15.19:0/2788006908 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f469c06c560 0x7f469c06ea20 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.072+0000 7f46bf200640 1 --2- 172.21.15.19:0/2788006908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46c0160a70 0x7f46c0160f20 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.072+0000 7f46bf200640 1 --2- 172.21.15.19:0/2788006908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f46c00cb3a0 0x7f46c00c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.072+0000 7f46bf200640 1 --2- 172.21.15.19:0/2788006908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f46a4004800 0x7f46c01524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.072+0000 7f46bf200640 1 -- 172.21.15.19:0/2788006908 >> 172.21.15.19:0/2788006908 conn(0x7f46c00676a0 msgr2=0x7f46c0159fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.072+0000 7f46bf200640 1 -- 172.21.15.19:0/2788006908 shutdown_connections 2024-03-28T13:21:10.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.072+0000 7f46bf200640 1 -- 172.21.15.19:0/2788006908 wait complete. 2024-03-28T13:21:10.090 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:21:10.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.318+0000 7fa1abe00640 1 Processor -- start 2024-03-28T13:21:10.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.318+0000 7fa1abe00640 1 -- start start 2024-03-28T13:21:10.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.319+0000 7fa1abe00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa1a40d0e70 0x7fa1a40cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:10.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.319+0000 7fa1abe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa1a40d2110 0x7fa1a40d25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:10.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.319+0000 7fa1abe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a40cf6c0 con 0x7fa1a415f6e0 2024-03-28T13:21:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.319+0000 7fa1abe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a40cf9a0 con 0x7fa1a40d2110 2024-03-28T13:21:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.319+0000 7fa1abe00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a40cfc80 con 0x7fa1a40d0e70 2024-03-28T13:21:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.320+0000 7fa1a9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa1a40d2110 0x7fa1a40d25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.320+0000 7fa1a9e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa1a40d2110 0x7fa1a40d25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33800/0 (socket says 172.21.15.19:33800) 2024-03-28T13:21:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.320+0000 7fa1a9e00640 1 -- 172.21.15.19:0/1909064753 learned_addr learned my addr 172.21.15.19:0/1909064753 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.320+0000 7fa1a9400640 1 --2- 172.21.15.19:0/1909064753 >> v2:172.21.15.188:3300/0 conn(0x7fa1a40d0e70 0x7fa1a40cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.320+0000 7fa1a9e00640 1 -- 172.21.15.19:0/1909064753 >> v2:172.21.15.188:3300/0 conn(0x7fa1a40d0e70 msgr2=0x7fa1a40cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:10.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.320+0000 7fa1a9e00640 1 --2- 172.21.15.19:0/1909064753 >> v2:172.21.15.188:3300/0 conn(0x7fa1a40d0e70 0x7fa1a40cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.320+0000 7fa1a9e00640 1 -- 172.21.15.19:0/1909064753 >> v1:172.21.15.188:6789/0 conn(0x7fa1a415f6e0 legacy=0x7fa1a40d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:10.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.320+0000 7fa1a9e00640 1 -- 172.21.15.19:0/1909064753 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a40cff60 con 0x7fa1a40d2110 2024-03-28T13:21:10.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.322+0000 7fa1a9e00640 1 --2- 172.21.15.19:0/1909064753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a40d2110 0x7fa1a40d25a0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fa1a408cf80 tx=0x7fa19c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b13911eb6f6e8d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.322+0000 7fa1a3e00640 1 -- 172.21.15.19:0/1909064753 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c02c9f0 con 0x7fa1a40d2110 2024-03-28T13:21:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.322+0000 7fa1a3e00640 1 -- 172.21.15.19:0/1909064753 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa19c02cb90 con 0x7fa1a40d2110 2024-03-28T13:21:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.322+0000 7fa1a3e00640 1 -- 172.21.15.19:0/1909064753 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0343f0 con 0x7fa1a40d2110 2024-03-28T13:21:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.322+0000 7fa1abe00640 1 -- 172.21.15.19:0/1909064753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a40d2110 msgr2=0x7fa1a40d25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.322+0000 7fa1abe00640 1 --2- 172.21.15.19:0/1909064753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a40d2110 0x7fa1a40d25a0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fa1a408cf80 tx=0x7fa19c02be50 comp rx=0 tx=0).stop 2024-03-28T13:21:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.323+0000 7fa1abe00640 1 -- 172.21.15.19:0/1909064753 shutdown_connections 2024-03-28T13:21:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.323+0000 7fa1abe00640 1 --2- 172.21.15.19:0/1909064753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a40d2110 0x7fa1a40d25a0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.323+0000 7fa1abe00640 1 --2- 172.21.15.19:0/1909064753 >> v2:172.21.15.188:3300/0 conn(0x7fa1a40d0e70 0x7fa1a40cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.323+0000 7fa1abe00640 1 -- 172.21.15.19:0/1909064753 >> 172.21.15.19:0/1909064753 conn(0x7fa1a408bfb0 msgr2=0x7fa1a40d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.323+0000 7fa1abe00640 1 -- 172.21.15.19:0/1909064753 shutdown_connections 2024-03-28T13:21:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.323+0000 7fa1abe00640 1 -- 172.21.15.19:0/1909064753 wait complete. 2024-03-28T13:21:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.324+0000 7fa1abe00640 1 Processor -- start 2024-03-28T13:21:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.325+0000 7fa1abe00640 1 -- start start 2024-03-28T13:21:10.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.325+0000 7fa1abe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a40d0e70 0x7fa1a40ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:10.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.325+0000 7fa1abe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a40d2110 0x7fa1a414d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:10.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.325+0000 7fa1abe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a415f6e0 0x7fa1a40ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:10.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.326+0000 7fa1abe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a415e910 con 0x7fa1a40d0e70 2024-03-28T13:21:10.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.326+0000 7fa1abe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a415fb40 con 0x7fa1a40d2110 2024-03-28T13:21:10.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.326+0000 7fa1abe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a415ea90 con 0x7fa1a415f6e0 2024-03-28T13:21:10.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.326+0000 7fa1a9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a40d0e70 0x7fa1a40ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:10.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.326+0000 7fa1a9400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a40d0e70 0x7fa1a40ce2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33814/0 (socket says 172.21.15.19:33814) 2024-03-28T13:21:10.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.326+0000 7fa1a9400640 1 -- 172.21.15.19:0/3205139263 learned_addr learned my addr 172.21.15.19:0/3205139263 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:10.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.326+0000 7fa1a8a00640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a40d2110 0x7fa1a414d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:10.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.326+0000 7fa1a9e00640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a415f6e0 0x7fa1a40ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:10.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.326+0000 7fa1a9400640 1 -- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a415f6e0 msgr2=0x7fa1a40ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:10.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.326+0000 7fa1a9400640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a415f6e0 0x7fa1a40ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.326+0000 7fa1a9400640 1 -- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a40d2110 msgr2=0x7fa1a414d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:10.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.326+0000 7fa1a9400640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a40d2110 0x7fa1a414d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.326+0000 7fa1a9400640 1 -- 172.21.15.19:0/3205139263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a40cd250 con 0x7fa1a40d0e70 2024-03-28T13:21:10.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.326+0000 7fa1a8a00640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a40d2110 0x7fa1a414d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:10.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.326+0000 7fa1a9e00640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a415f6e0 0x7fa1a40ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:10.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.327+0000 7fa1a9400640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a40d0e70 0x7fa1a40ce2b0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fa19400d5f0 tx=0x7fa19400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:10.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.327+0000 7fa1a1600640 1 -- 172.21.15.19:0/3205139263 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1940125f0 con 0x7fa1a40d0e70 2024-03-28T13:21:10.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.327+0000 7fa1a1600640 1 -- 172.21.15.19:0/3205139263 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa194019070 con 0x7fa1a40d0e70 2024-03-28T13:21:10.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.327+0000 7fa1a1600640 1 -- 172.21.15.19:0/3205139263 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1940129d0 con 0x7fa1a40d0e70 2024-03-28T13:21:10.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.327+0000 7fa1abe00640 1 -- 172.21.15.19:0/3205139263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a40cd4f0 con 0x7fa1a40d0e70 2024-03-28T13:21:10.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.327+0000 7fa1abe00640 1 -- 172.21.15.19:0/3205139263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a40cdd90 con 0x7fa1a40d0e70 2024-03-28T13:21:10.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.328+0000 7fa1abe00640 1 -- 172.21.15.19:0/3205139263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a4068cd0 con 0x7fa1a40d0e70 2024-03-28T13:21:10.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.329+0000 7fa1a1600640 1 -- 172.21.15.19:0/3205139263 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa194012b70 con 0x7fa1a40d0e70 2024-03-28T13:21:10.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.330+0000 7fa1a1600640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa18406c560 0x7fa18406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:10.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.330+0000 7fa1a1600640 1 -- 172.21.15.19:0/3205139263 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7fa19408bb20 con 0x7fa1a40d0e70 2024-03-28T13:21:10.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.331+0000 7fa1a8a00640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa18406c560 0x7fa18406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:10.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.332+0000 7fa1a8a00640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa18406c560 0x7fa18406ea20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa1a41f8010 tx=0x7fa190009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:10.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.334+0000 7fa1a1600640 1 -- 172.21.15.19:0/3205139263 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa19405a0a0 con 0x7fa1a40d0e70 2024-03-28T13:21:10.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.539+0000 7fa1abe00640 1 -- 172.21.15.19:0/3205139263 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa1a40d1b80 con 0x7fa1a40d0e70 2024-03-28T13:21:10.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.540+0000 7fa1a1600640 1 -- 172.21.15.19:0/3205139263 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) ==== 104+0+11 (secure 0 0 0) 0x7fa1940599a0 con 0x7fa1a40d0e70 2024-03-28T13:21:10.540 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:21:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.543+0000 7fa1abe00640 1 -- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa18406c560 msgr2=0x7fa18406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.543+0000 7fa1abe00640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa18406c560 0x7fa18406ea20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa1a41f8010 tx=0x7fa190009040 comp rx=0 tx=0).stop 2024-03-28T13:21:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.543+0000 7fa1abe00640 1 -- 172.21.15.19:0/3205139263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a40d0e70 msgr2=0x7fa1a40ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.543+0000 7fa1abe00640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a40d0e70 0x7fa1a40ce2b0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fa19400d5f0 tx=0x7fa19400dab0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.543+0000 7fa1abe00640 1 -- 172.21.15.19:0/3205139263 shutdown_connections 2024-03-28T13:21:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.543+0000 7fa1abe00640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa18406c560 0x7fa18406ea20 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.543+0000 7fa1abe00640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1a415f6e0 0x7fa1a40ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.543+0000 7fa1abe00640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1a40d2110 0x7fa1a414d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.543+0000 7fa1abe00640 1 --2- 172.21.15.19:0/3205139263 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a40d0e70 0x7fa1a40ce2b0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.543+0000 7fa1abe00640 1 -- 172.21.15.19:0/3205139263 >> 172.21.15.19:0/3205139263 conn(0x7fa1a408bfb0 msgr2=0x7fa1a415ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.543+0000 7fa1abe00640 1 -- 172.21.15.19:0/3205139263 shutdown_connections 2024-03-28T13:21:10.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.544+0000 7fa1abe00640 1 -- 172.21.15.19:0/3205139263 wait complete. 2024-03-28T13:21:10.564 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:21:10.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.791+0000 7feb91000640 1 Processor -- start 2024-03-28T13:21:10.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.791+0000 7feb91000640 1 -- start start 2024-03-28T13:21:10.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.793+0000 7feb91000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feb8c0b4650 0x7feb8c06c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:10.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.793+0000 7feb91000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb8c158580 0x7feb8c1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:10.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.793+0000 7feb91000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb8c15ede0 con 0x7feb8c0b4a30 2024-03-28T13:21:10.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.793+0000 7feb91000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7feb8c0b6320 con 0x7feb8c158580 2024-03-28T13:21:10.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.793+0000 7feb91000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7feb8c141b30 con 0x7feb8c0b4650 2024-03-28T13:21:10.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.793+0000 7feb89400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7feb8c0b4a30 0x7feb8c0b5c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38602/0 (socket says 172.21.15.19:38602) 2024-03-28T13:21:10.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.793+0000 7feb8a800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7feb8c158580 0x7feb8c1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:10.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.793+0000 7feb89400640 1 -- 172.21.15.19:0/166217591 learned_addr learned my addr 172.21.15.19:0/166217591 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:10.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.794+0000 7feb89e00640 1 --2- 172.21.15.19:0/166217591 >> v2:172.21.15.188:3300/0 conn(0x7feb8c0b4650 0x7feb8c06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:10.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.794+0000 7feb88a00640 1 -- 172.21.15.19:0/166217591 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2549422464 0 0) 0x7feb8c15ede0 con 0x7feb8c0b4a30 2024-03-28T13:21:10.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.794+0000 7feb88a00640 1 -- 172.21.15.19:0/166217591 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb70003520 con 0x7feb8c0b4a30 2024-03-28T13:21:10.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.794+0000 7feb8a800640 1 -- 172.21.15.19:0/166217591 >> v2:172.21.15.188:3300/0 conn(0x7feb8c0b4650 msgr2=0x7feb8c06c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:10.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.794+0000 7feb8a800640 1 --2- 172.21.15.19:0/166217591 >> v2:172.21.15.188:3300/0 conn(0x7feb8c0b4650 0x7feb8c06c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.794+0000 7feb8a800640 1 -- 172.21.15.19:0/166217591 >> v1:172.21.15.19:6789/0 conn(0x7feb8c0b4a30 legacy=0x7feb8c0b5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:10.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.794+0000 7feb8a800640 1 -- 172.21.15.19:0/166217591 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8c141d80 con 0x7feb8c158580 2024-03-28T13:21:10.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.795+0000 7feb8a800640 1 --2- 172.21.15.19:0/166217591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c158580 0x7feb8c1589a0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0d32b0 tx=0x7feb8402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d604eddc632c5f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:10.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.796+0000 7feb88a00640 1 -- 172.21.15.19:0/166217591 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84039070 con 0x7feb8c158580 2024-03-28T13:21:10.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.796+0000 7feb88a00640 1 -- 172.21.15.19:0/166217591 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb8402cc80 con 0x7feb8c158580 2024-03-28T13:21:10.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.796+0000 7feb88a00640 1 -- 172.21.15.19:0/166217591 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84034640 con 0x7feb8c158580 2024-03-28T13:21:10.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.796+0000 7feb91000640 1 -- 172.21.15.19:0/166217591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c158580 msgr2=0x7feb8c1589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:10.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.796+0000 7feb91000640 1 --2- 172.21.15.19:0/166217591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c158580 0x7feb8c1589a0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0d32b0 tx=0x7feb8402bed0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.797+0000 7feb91000640 1 -- 172.21.15.19:0/166217591 shutdown_connections 2024-03-28T13:21:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.797+0000 7feb91000640 1 --2- 172.21.15.19:0/166217591 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c158580 0x7feb8c1589a0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.797+0000 7feb91000640 1 --2- 172.21.15.19:0/166217591 >> v2:172.21.15.188:3300/0 conn(0x7feb8c0b4650 0x7feb8c06c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.797+0000 7feb91000640 1 -- 172.21.15.19:0/166217591 >> 172.21.15.19:0/166217591 conn(0x7feb8c067670 msgr2=0x7feb8c067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.797+0000 7feb91000640 1 -- 172.21.15.19:0/166217591 shutdown_connections 2024-03-28T13:21:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.797+0000 7feb91000640 1 -- 172.21.15.19:0/166217591 wait complete. 2024-03-28T13:21:10.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.798+0000 7feb91000640 1 Processor -- start 2024-03-28T13:21:10.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.799+0000 7feb91000640 1 -- start start 2024-03-28T13:21:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.800+0000 7feb91000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb8c0b4650 0x7feb8c159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.800+0000 7feb91000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c0b4a30 0x7feb8c140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.800+0000 7feb91000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb8c158580 0x7feb8c15a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.800+0000 7feb91000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c142620 con 0x7feb8c0b4a30 2024-03-28T13:21:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.800+0000 7feb91000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c142340 con 0x7feb8c0b4650 2024-03-28T13:21:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.800+0000 7feb91000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feb8c142900 con 0x7feb8c158580 2024-03-28T13:21:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.800+0000 7feb89400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c0b4a30 0x7feb8c140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.800+0000 7feb89400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c0b4a30 0x7feb8c140cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33838/0 (socket says 172.21.15.19:33838) 2024-03-28T13:21:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.800+0000 7feb89400640 1 -- 172.21.15.19:0/3916071189 learned_addr learned my addr 172.21.15.19:0/3916071189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.800+0000 7feb89e00640 1 --2- 172.21.15.19:0/3916071189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb8c0b4650 0x7feb8c159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.800+0000 7feb89400640 1 -- 172.21.15.19:0/3916071189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb8c158580 msgr2=0x7feb8c15a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.800+0000 7feb89400640 1 --2- 172.21.15.19:0/3916071189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb8c158580 0x7feb8c15a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.800+0000 7feb89400640 1 -- 172.21.15.19:0/3916071189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb8c0b4650 msgr2=0x7feb8c159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:10.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.800+0000 7feb89400640 1 --2- 172.21.15.19:0/3916071189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb8c0b4650 0x7feb8c159b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:10.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.800+0000 7feb89400640 1 -- 172.21.15.19:0/3916071189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8c15ab60 con 0x7feb8c0b4a30 2024-03-28T13:21:10.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.801+0000 7feb89400640 1 --2- 172.21.15.19:0/3916071189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c0b4a30 0x7feb8c140cb0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7feb7800c410 tx=0x7feb7800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:10.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.801+0000 7feb82000640 1 -- 172.21.15.19:0/3916071189 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb78017070 con 0x7feb8c0b4a30 2024-03-28T13:21:10.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.801+0000 7feb82000640 1 -- 172.21.15.19:0/3916071189 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb78005ce0 con 0x7feb8c0b4a30 2024-03-28T13:21:10.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.801+0000 7feb91000640 1 -- 172.21.15.19:0/3916071189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb8c15ae40 con 0x7feb8c0b4a30 2024-03-28T13:21:10.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.801+0000 7feb82000640 1 -- 172.21.15.19:0/3916071189 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb780072f0 con 0x7feb8c0b4a30 2024-03-28T13:21:10.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.801+0000 7feb91000640 1 -- 172.21.15.19:0/3916071189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb8c15b400 con 0x7feb8c0b4a30 2024-03-28T13:21:10.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.802+0000 7feb91000640 1 -- 172.21.15.19:0/3916071189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb8c0685c0 con 0x7feb8c0b4a30 2024-03-28T13:21:10.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.803+0000 7feb82000640 1 -- 172.21.15.19:0/3916071189 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7feb7801a070 con 0x7feb8c0b4a30 2024-03-28T13:21:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.805+0000 7feb82000640 1 --2- 172.21.15.19:0/3916071189 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb6c06c530 0x7feb6c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.805+0000 7feb82000640 1 -- 172.21.15.19:0/3916071189 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7feb7808abe0 con 0x7feb8c0b4a30 2024-03-28T13:21:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.805+0000 7feb89e00640 1 --2- 172.21.15.19:0/3916071189 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb6c06c530 0x7feb6c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:10.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.807+0000 7feb89e00640 1 --2- 172.21.15.19:0/3916071189 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb6c06c530 0x7feb6c06e9f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009a10 tx=0x7feb7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:10.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:10.809+0000 7feb82000640 1 -- 172.21.15.19:0/3916071189 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7feb78059160 con 0x7feb8c0b4a30 2024-03-28T13:21:11.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.012+0000 7feb91000640 1 -- 172.21.15.19:0/3916071189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feb8c0b5cb0 con 0x7feb8c0b4a30 2024-03-28T13:21:11.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.013+0000 7feb82000640 1 -- 172.21.15.19:0/3916071189 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) ==== 100+0+11 (secure 0 0 0) 0x7feb78058a60 con 0x7feb8c0b4a30 2024-03-28T13:21:11.013 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:21:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.015+0000 7feb91000640 1 -- 172.21.15.19:0/3916071189 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb6c06c530 msgr2=0x7feb6c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.015+0000 7feb91000640 1 --2- 172.21.15.19:0/3916071189 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb6c06c530 0x7feb6c06e9f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009a10 tx=0x7feb7c008040 comp rx=0 tx=0).stop 2024-03-28T13:21:11.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.015+0000 7feb91000640 1 -- 172.21.15.19:0/3916071189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c0b4a30 msgr2=0x7feb8c140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:11.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.015+0000 7feb91000640 1 --2- 172.21.15.19:0/3916071189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c0b4a30 0x7feb8c140cb0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7feb7800c410 tx=0x7feb7800c8d0 comp rx=0 tx=0).stop 2024-03-28T13:21:11.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.015+0000 7feb91000640 1 -- 172.21.15.19:0/3916071189 shutdown_connections 2024-03-28T13:21:11.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.015+0000 7feb91000640 1 --2- 172.21.15.19:0/3916071189 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7feb6c06c530 0x7feb6c06e9f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:11.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.015+0000 7feb91000640 1 --2- 172.21.15.19:0/3916071189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb8c158580 0x7feb8c15a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:11.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.015+0000 7feb91000640 1 --2- 172.21.15.19:0/3916071189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feb8c0b4a30 0x7feb8c140cb0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:11.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.016+0000 7feb91000640 1 --2- 172.21.15.19:0/3916071189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb8c0b4650 0x7feb8c159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:11.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.016+0000 7feb91000640 1 -- 172.21.15.19:0/3916071189 >> 172.21.15.19:0/3916071189 conn(0x7feb8c067670 msgr2=0x7feb8c06cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:11.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.016+0000 7feb91000640 1 -- 172.21.15.19:0/3916071189 shutdown_connections 2024-03-28T13:21:11.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.016+0000 7feb91000640 1 -- 172.21.15.19:0/3916071189 wait complete. 2024-03-28T13:21:11.034 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:21:11.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.255+0000 7ff645c00640 1 Processor -- start 2024-03-28T13:21:11.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.255+0000 7ff645c00640 1 -- start start 2024-03-28T13:21:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.256+0000 7ff645c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400af4b0 0x7ff640167460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.257+0000 7ff645c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6400afab0 con 0x7ff6400d0990 2024-03-28T13:21:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.257+0000 7ff645c00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff64015cc80 con 0x7ff640153580 2024-03-28T13:21:11.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.257+0000 7ff645c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff64015da00 con 0x7ff6400af4b0 2024-03-28T13:21:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.257+0000 7ff63e000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff6400d0990 0x7ff6400aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38610/0 (socket says 172.21.15.19:38610) 2024-03-28T13:21:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.257+0000 7ff63e000640 1 -- 172.21.15.19:0/2873889823 learned_addr learned my addr 172.21.15.19:0/2873889823 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:11.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.257+0000 7ff63f400640 1 --2- 172.21.15.19:0/2873889823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400af4b0 0x7ff640167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.258+0000 7ff63d600640 1 -- 172.21.15.19:0/2873889823 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2885423353 0 0) 0x7ff6400afab0 con 0x7ff6400d0990 2024-03-28T13:21:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.258+0000 7ff63d600640 1 -- 172.21.15.19:0/2873889823 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff624003520 con 0x7ff6400d0990 2024-03-28T13:21:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.258+0000 7ff63d600640 1 -- 172.21.15.19:0/2873889823 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3114391394 0 0) 0x7ff624003520 con 0x7ff6400d0990 2024-03-28T13:21:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.258+0000 7ff63d600640 1 -- 172.21.15.19:0/2873889823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400af4b0 msgr2=0x7ff640167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.258+0000 7ff63d600640 1 --2- 172.21.15.19:0/2873889823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400af4b0 0x7ff640167460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.258+0000 7ff63d600640 1 -- 172.21.15.19:0/2873889823 >> v1:172.21.15.188:6789/0 conn(0x7ff640153580 legacy=0x7ff6400d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.258+0000 7ff63d600640 1 -- 172.21.15.19:0/2873889823 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64015cf20 con 0x7ff6400d0990 2024-03-28T13:21:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.258+0000 7ff63d600640 1 -- 172.21.15.19:0/2873889823 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7ff62c001670 con 0x7ff6400d0990 2024-03-28T13:21:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.258+0000 7ff63d600640 1 -- 172.21.15.19:0/2873889823 >> v1:172.21.15.19:6789/0 conn(0x7ff6400d0990 legacy=0x7ff6400aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:11.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.258+0000 7ff63d600640 1 --2- 172.21.15.19:0/2873889823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6240039e0 0x7ff624003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.259+0000 7ff63d600640 1 --2- 172.21.15.19:0/2873889823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff624004800 0x7ff624004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.259+0000 7ff63d600640 1 --2- 172.21.15.19:0/2873889823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400af4b0 0x7ff624005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.259+0000 7ff63d600640 1 -- 172.21.15.19:0/2873889823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff64015da00 con 0x7ff6240039e0 2024-03-28T13:21:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.259+0000 7ff63d600640 1 -- 172.21.15.19:0/2873889823 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff62400baf0 con 0x7ff624004800 2024-03-28T13:21:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.259+0000 7ff63d600640 1 -- 172.21.15.19:0/2873889823 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff62400bcf0 con 0x7ff6400af4b0 2024-03-28T13:21:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.259+0000 7ff63ea00640 1 --2- 172.21.15.19:0/2873889823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6240039e0 0x7ff624003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.259+0000 7ff63f400640 1 --2- 172.21.15.19:0/2873889823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400af4b0 0x7ff624005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.259+0000 7ff63e000640 1 --2- 172.21.15.19:0/2873889823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff624004800 0x7ff624004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.259+0000 7ff63ea00640 1 -- 172.21.15.19:0/2873889823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400af4b0 msgr2=0x7ff624005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.259+0000 7ff63ea00640 1 --2- 172.21.15.19:0/2873889823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400af4b0 0x7ff624005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.259+0000 7ff63ea00640 1 -- 172.21.15.19:0/2873889823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff624004800 msgr2=0x7ff624004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.259+0000 7ff63ea00640 1 --2- 172.21.15.19:0/2873889823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff624004800 0x7ff624004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:11.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.259+0000 7ff63ea00640 1 -- 172.21.15.19:0/2873889823 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff624003520 con 0x7ff6240039e0 2024-03-28T13:21:11.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.260+0000 7ff63ea00640 1 --2- 172.21.15.19:0/2873889823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6240039e0 0x7ff624003e00 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7ff630001720 tx=0x7ff624005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ddb77b3cfcfc52 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.261+0000 7ff63d600640 1 -- 172.21.15.19:0/2873889823 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630001370 con 0x7ff6240039e0 2024-03-28T13:21:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.261+0000 7ff63d600640 1 -- 172.21.15.19:0/2873889823 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6300052d0 con 0x7ff6240039e0 2024-03-28T13:21:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.261+0000 7ff63d600640 1 -- 172.21.15.19:0/2873889823 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630003830 con 0x7ff6240039e0 2024-03-28T13:21:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.261+0000 7ff645c00640 1 -- 172.21.15.19:0/2873889823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6240039e0 msgr2=0x7ff624003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.261+0000 7ff645c00640 1 --2- 172.21.15.19:0/2873889823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6240039e0 0x7ff624003e00 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7ff630001720 tx=0x7ff624005830 comp rx=0 tx=0).stop 2024-03-28T13:21:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.261+0000 7ff63f400640 1 -- 172.21.15.19:0/2873889823 reap_dead start 2024-03-28T13:21:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.262+0000 7ff645c00640 1 -- 172.21.15.19:0/2873889823 shutdown_connections 2024-03-28T13:21:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.262+0000 7ff645c00640 1 -- 172.21.15.19:0/2873889823 >> 172.21.15.19:0/2873889823 conn(0x7ff64006b750 msgr2=0x7ff64006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.262+0000 7ff645c00640 1 -- 172.21.15.19:0/2873889823 shutdown_connections 2024-03-28T13:21:11.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.262+0000 7ff645c00640 1 -- 172.21.15.19:0/2873889823 wait complete. 2024-03-28T13:21:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.264+0000 7ff645c00640 1 Processor -- start 2024-03-28T13:21:11.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.264+0000 7ff645c00640 1 -- start start 2024-03-28T13:21:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.265+0000 7ff645c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6240039e0 0x7ff64009da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.265+0000 7ff645c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff64009d560 0x7ff64015ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.265+0000 7ff645c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff64015f210 0x7ff64014c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.265+0000 7ff645c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff640153960 con 0x7ff64009d560 2024-03-28T13:21:11.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.265+0000 7ff645c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff64015d4e0 con 0x7ff64015f210 2024-03-28T13:21:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.265+0000 7ff645c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff640165450 con 0x7ff6240039e0 2024-03-28T13:21:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.265+0000 7ff63e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff64009d560 0x7ff64015ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.265+0000 7ff63e000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff64009d560 0x7ff64015ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33858/0 (socket says 172.21.15.19:33858) 2024-03-28T13:21:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.265+0000 7ff63e000640 1 -- 172.21.15.19:0/2469029318 learned_addr learned my addr 172.21.15.19:0/2469029318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.265+0000 7ff63ea00640 1 --2- 172.21.15.19:0/2469029318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6240039e0 0x7ff64009da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.265+0000 7ff63f400640 1 --2- 172.21.15.19:0/2469029318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff64015f210 0x7ff64014c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.266+0000 7ff63e000640 1 -- 172.21.15.19:0/2469029318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6240039e0 msgr2=0x7ff64009da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.266+0000 7ff63e000640 1 --2- 172.21.15.19:0/2469029318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6240039e0 0x7ff64009da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.266+0000 7ff63e000640 1 -- 172.21.15.19:0/2469029318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff64015f210 msgr2=0x7ff64014c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.266+0000 7ff63e000640 1 --2- 172.21.15.19:0/2469029318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff64015f210 0x7ff64014c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.266+0000 7ff63e000640 1 -- 172.21.15.19:0/2469029318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6400cdf70 con 0x7ff64009d560 2024-03-28T13:21:11.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.266+0000 7ff63e000640 1 --2- 172.21.15.19:0/2469029318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff64009d560 0x7ff64015ecd0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00d770 tx=0x7ff62c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.266+0000 7ff636a00640 1 -- 172.21.15.19:0/2469029318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c019070 con 0x7ff64009d560 2024-03-28T13:21:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.266+0000 7ff636a00640 1 -- 172.21.15.19:0/2469029318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62c0074e0 con 0x7ff64009d560 2024-03-28T13:21:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.266+0000 7ff636a00640 1 -- 172.21.15.19:0/2469029318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c004a00 con 0x7ff64009d560 2024-03-28T13:21:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.266+0000 7ff645c00640 1 -- 172.21.15.19:0/2469029318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6400ce250 con 0x7ff64009d560 2024-03-28T13:21:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.266+0000 7ff645c00640 1 -- 172.21.15.19:0/2469029318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6400ce810 con 0x7ff64009d560 2024-03-28T13:21:11.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.268+0000 7ff645c00640 1 -- 172.21.15.19:0/2469029318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6400685c0 con 0x7ff64009d560 2024-03-28T13:21:11.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.268+0000 7ff636a00640 1 -- 172.21.15.19:0/2469029318 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff62c01c070 con 0x7ff64009d560 2024-03-28T13:21:11.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.269+0000 7ff636a00640 1 --2- 172.21.15.19:0/2469029318 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff62006c560 0x7ff62006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:11.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.270+0000 7ff636a00640 1 -- 172.21.15.19:0/2469029318 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7ff62c08b110 con 0x7ff64009d560 2024-03-28T13:21:11.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.270+0000 7ff63ea00640 1 --2- 172.21.15.19:0/2469029318 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff62006c560 0x7ff62006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:11.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.271+0000 7ff63ea00640 1 --2- 172.21.15.19:0/2469029318 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff62006c560 0x7ff62006ea20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff630033000 tx=0x7ff630009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:11.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.274+0000 7ff636a00640 1 -- 172.21.15.19:0/2469029318 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7ff62c00de20 con 0x7ff64009d560 2024-03-28T13:21:11.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.491+0000 7ff645c00640 1 -- 172.21.15.19:0/2469029318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff64015f5f0 con 0x7ff64009d560 2024-03-28T13:21:11.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.491+0000 7ff636a00640 1 -- 172.21.15.19:0/2469029318 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v119) ==== 80+0+0 (secure 0 0 0) 0x7ff62c059690 con 0x7ff64009d560 2024-03-28T13:21:11.491 INFO:teuthology.orchestra.run.smithi019.stderr:listed 0 entries 2024-03-28T13:21:11.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.494+0000 7ff645c00640 1 -- 172.21.15.19:0/2469029318 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff62006c560 msgr2=0x7ff62006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:11.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.494+0000 7ff645c00640 1 --2- 172.21.15.19:0/2469029318 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff62006c560 0x7ff62006ea20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff630033000 tx=0x7ff630009030 comp rx=0 tx=0).stop 2024-03-28T13:21:11.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.494+0000 7ff645c00640 1 -- 172.21.15.19:0/2469029318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff64009d560 msgr2=0x7ff64015ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:11.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.494+0000 7ff645c00640 1 --2- 172.21.15.19:0/2469029318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff64009d560 0x7ff64015ecd0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00d770 tx=0x7ff62c00dc30 comp rx=0 tx=0).stop 2024-03-28T13:21:11.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.494+0000 7ff645c00640 1 -- 172.21.15.19:0/2469029318 shutdown_connections 2024-03-28T13:21:11.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.494+0000 7ff645c00640 1 --2- 172.21.15.19:0/2469029318 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff62006c560 0x7ff62006ea20 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:11.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.494+0000 7ff645c00640 1 --2- 172.21.15.19:0/2469029318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff64015f210 0x7ff64014c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:11.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.494+0000 7ff645c00640 1 --2- 172.21.15.19:0/2469029318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff64009d560 0x7ff64015ecd0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:11.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.494+0000 7ff645c00640 1 --2- 172.21.15.19:0/2469029318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6240039e0 0x7ff64009da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:11.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.494+0000 7ff645c00640 1 -- 172.21.15.19:0/2469029318 >> 172.21.15.19:0/2469029318 conn(0x7ff64006b750 msgr2=0x7ff6400d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:11.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.495+0000 7ff645c00640 1 -- 172.21.15.19:0/2469029318 shutdown_connections 2024-03-28T13:21:11.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:11.495+0000 7ff645c00640 1 -- 172.21.15.19:0/2469029318 wait complete. 2024-03-28T13:21:11.514 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:21:11.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:11.514 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:11.764 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:21:11.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:11.764 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:21:11.817 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:11.817 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:11.899 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:21:11.900 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:21:11.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:11.900 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:21:11.992 INFO:teuthology.orchestra.run.smithi019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-28T13:21:11.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T13:21:11.993 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T13:21:12.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.206+0000 7fddba000640 1 Processor -- start 2024-03-28T13:21:12.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.206+0000 7fddba000640 1 -- start start 2024-03-28T13:21:12.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.207+0000 7fddba000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fddb414ba20 0x7fddb40b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:12.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.207+0000 7fddba000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fddb414c1b0 0x7fddb4166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:12.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.207+0000 7fddba000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fddb40cfda0 0x7fddb40d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:12.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.207+0000 7fddba000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fddb40d0710 con 0x7fddb40cfda0 2024-03-28T13:21:12.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.207+0000 7fddba000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fddb40b33e0 con 0x7fddb414c1b0 2024-03-28T13:21:12.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.207+0000 7fddba000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fddb40cde20 con 0x7fddb414ba20 2024-03-28T13:21:12.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.208+0000 7fddb9600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fddb40cfda0 0x7fddb40d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:12.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.208+0000 7fddb9600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fddb40cfda0 0x7fddb40d01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33864/0 (socket says 172.21.15.19:33864) 2024-03-28T13:21:12.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.208+0000 7fddb9600640 1 -- 172.21.15.19:0/3701298925 learned_addr learned my addr 172.21.15.19:0/3701298925 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:12.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.208+0000 7fddb8c00640 1 --2- 172.21.15.19:0/3701298925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fddb414ba20 0x7fddb40b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:12.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.208+0000 7fddb3e00640 1 --2- 172.21.15.19:0/3701298925 >> v2:172.21.15.188:3300/0 conn(0x7fddb414c1b0 0x7fddb4166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:12.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.209+0000 7fddb9600640 1 -- 172.21.15.19:0/3701298925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fddb414ba20 msgr2=0x7fddb40b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:12.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.209+0000 7fddb9600640 1 --2- 172.21.15.19:0/3701298925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fddb414ba20 0x7fddb40b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:12.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.209+0000 7fddb9600640 1 -- 172.21.15.19:0/3701298925 >> v2:172.21.15.188:3300/0 conn(0x7fddb414c1b0 msgr2=0x7fddb4166290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:12.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.209+0000 7fddb9600640 1 --2- 172.21.15.19:0/3701298925 >> v2:172.21.15.188:3300/0 conn(0x7fddb414c1b0 0x7fddb4166290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:12.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.209+0000 7fddb9600640 1 -- 172.21.15.19:0/3701298925 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb4166ae0 con 0x7fddb40cfda0 2024-03-28T13:21:12.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.210+0000 7fddb9600640 1 --2- 172.21.15.19:0/3701298925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb40cfda0 0x7fddb40d01d0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fddb40d0ad0 tx=0x7fddac02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8149502363356346 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:12.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.210+0000 7fddb3400640 1 -- 172.21.15.19:0/3701298925 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac02ca70 con 0x7fddb40cfda0 2024-03-28T13:21:12.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.210+0000 7fddb3400640 1 -- 172.21.15.19:0/3701298925 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddac02cc10 con 0x7fddb40cfda0 2024-03-28T13:21:12.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.210+0000 7fddb3400640 1 -- 172.21.15.19:0/3701298925 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac0343f0 con 0x7fddb40cfda0 2024-03-28T13:21:12.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.211+0000 7fddba000640 1 -- 172.21.15.19:0/3701298925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb40cfda0 msgr2=0x7fddb40d01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.211+0000 7fddba000640 1 --2- 172.21.15.19:0/3701298925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb40cfda0 0x7fddb40d01d0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fddb40d0ad0 tx=0x7fddac02bed0 comp rx=0 tx=0).stop 2024-03-28T13:21:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.211+0000 7fddba000640 1 -- 172.21.15.19:0/3701298925 shutdown_connections 2024-03-28T13:21:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.211+0000 7fddba000640 1 --2- 172.21.15.19:0/3701298925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb40cfda0 0x7fddb40d01d0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.211+0000 7fddba000640 1 --2- 172.21.15.19:0/3701298925 >> v2:172.21.15.188:3300/0 conn(0x7fddb414c1b0 0x7fddb4166290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.211+0000 7fddba000640 1 --2- 172.21.15.19:0/3701298925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fddb414ba20 0x7fddb40b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.211+0000 7fddba000640 1 -- 172.21.15.19:0/3701298925 >> 172.21.15.19:0/3701298925 conn(0x7fddb40ccad0 msgr2=0x7fddb40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.211+0000 7fddba000640 1 -- 172.21.15.19:0/3701298925 shutdown_connections 2024-03-28T13:21:12.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.211+0000 7fddba000640 1 -- 172.21.15.19:0/3701298925 wait complete. 2024-03-28T13:21:12.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.213+0000 7fddba000640 1 Processor -- start 2024-03-28T13:21:12.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.213+0000 7fddba000640 1 -- start start 2024-03-28T13:21:12.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.214+0000 7fddba000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb40cfda0 0x7fddb41f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:12.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.214+0000 7fddba000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fddb414ba20 0x7fddb4145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:12.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.214+0000 7fddba000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddb414c1b0 0x7fddb41ecb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:12.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.214+0000 7fddba000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fddb40ce0c0 con 0x7fddb40cfda0 2024-03-28T13:21:12.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.214+0000 7fddba000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fddb41670a0 con 0x7fddb414c1b0 2024-03-28T13:21:12.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.214+0000 7fddba000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fddb41667d0 con 0x7fddb414ba20 2024-03-28T13:21:12.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.214+0000 7fddb9600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddb414c1b0 0x7fddb41ecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:12.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.214+0000 7fddb8c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb40cfda0 0x7fddb41f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:12.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.214+0000 7fddb9600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddb414c1b0 0x7fddb41ecb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:47326/0 (socket says 172.21.15.19:47326) 2024-03-28T13:21:12.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.214+0000 7fddb3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fddb414ba20 0x7fddb4145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:12.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.214+0000 7fddb9600640 1 -- 172.21.15.19:0/3597185373 learned_addr learned my addr 172.21.15.19:0/3597185373 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:12.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.215+0000 7fddb8c00640 1 -- 172.21.15.19:0/3597185373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fddb414ba20 msgr2=0x7fddb4145f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:12.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.215+0000 7fddb8c00640 1 --2- 172.21.15.19:0/3597185373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fddb414ba20 0x7fddb4145f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:12.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.215+0000 7fddb8c00640 1 -- 172.21.15.19:0/3597185373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddb414c1b0 msgr2=0x7fddb41ecb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:12.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.215+0000 7fddb8c00640 1 --2- 172.21.15.19:0/3597185373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddb414c1b0 0x7fddb41ecb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:12.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.215+0000 7fddb8c00640 1 -- 172.21.15.19:0/3597185373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb4146630 con 0x7fddb40cfda0 2024-03-28T13:21:12.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.215+0000 7fddb8c00640 1 --2- 172.21.15.19:0/3597185373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb40cfda0 0x7fddb41f8ac0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fdda800e870 tx=0x7fdda800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:12.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.216+0000 7fddb0c00640 1 -- 172.21.15.19:0/3597185373 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8017070 con 0x7fddb40cfda0 2024-03-28T13:21:12.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.216+0000 7fddb0c00640 1 -- 172.21.15.19:0/3597185373 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda800ee60 con 0x7fddb40cfda0 2024-03-28T13:21:12.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.216+0000 7fddb0c00640 1 -- 172.21.15.19:0/3597185373 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda800cb60 con 0x7fddb40cfda0 2024-03-28T13:21:12.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.216+0000 7fddba000640 1 -- 172.21.15.19:0/3597185373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb4146910 con 0x7fddb40cfda0 2024-03-28T13:21:12.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.216+0000 7fddba000640 1 -- 172.21.15.19:0/3597185373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb4146ed0 con 0x7fddb40cfda0 2024-03-28T13:21:12.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.217+0000 7fddba000640 1 -- 172.21.15.19:0/3597185373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb4075940 con 0x7fddb40cfda0 2024-03-28T13:21:12.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.218+0000 7fddb0c00640 1 -- 172.21.15.19:0/3597185373 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdda801a070 con 0x7fddb40cfda0 2024-03-28T13:21:12.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.219+0000 7fddb0c00640 1 --2- 172.21.15.19:0/3597185373 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdd9406c560 0x7fdd9406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:12.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.219+0000 7fddb0c00640 1 -- 172.21.15.19:0/3597185373 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7fdda808b7e0 con 0x7fddb40cfda0 2024-03-28T13:21:12.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.220+0000 7fddb3e00640 1 --2- 172.21.15.19:0/3597185373 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdd9406c560 0x7fdd9406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:12.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.221+0000 7fddb3e00640 1 --2- 172.21.15.19:0/3597185373 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdd9406c560 0x7fdd9406ea20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fddb40af6b0 tx=0x7fdda0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:12.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.224+0000 7fddb0c00640 1 -- 172.21.15.19:0/3597185373 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdda80597a0 con 0x7fddb40cfda0 2024-03-28T13:21:12.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.468+0000 7fddba000640 1 -- 172.21.15.19:0/3597185373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fddb4162a80 con 0x7fddb40cfda0 2024-03-28T13:21:12.468 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:21:12.469+0000 7f6b74400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T13:21:12.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.472+0000 7fddb0c00640 1 -- 172.21.15.19:0/3597185373 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v100) ==== 161+0+0 (secure 0 0 0) 0x7fdda805daa0 con 0x7fddb40cfda0 2024-03-28T13:21:12.472 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T13:21:12.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.475+0000 7fddba000640 1 -- 172.21.15.19:0/3597185373 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdd9406c560 msgr2=0x7fdd9406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:12.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.475+0000 7fddba000640 1 --2- 172.21.15.19:0/3597185373 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdd9406c560 0x7fdd9406ea20 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fddb40af6b0 tx=0x7fdda0009190 comp rx=0 tx=0).stop 2024-03-28T13:21:12.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.475+0000 7fddba000640 1 -- 172.21.15.19:0/3597185373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb40cfda0 msgr2=0x7fddb41f8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:12.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.475+0000 7fddba000640 1 --2- 172.21.15.19:0/3597185373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb40cfda0 0x7fddb41f8ac0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fdda800e870 tx=0x7fdda800ed30 comp rx=0 tx=0).stop 2024-03-28T13:21:12.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.475+0000 7fddba000640 1 -- 172.21.15.19:0/3597185373 shutdown_connections 2024-03-28T13:21:12.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.475+0000 7fddba000640 1 --2- 172.21.15.19:0/3597185373 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdd9406c560 0x7fdd9406ea20 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:12.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.475+0000 7fddba000640 1 --2- 172.21.15.19:0/3597185373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fddb414c1b0 0x7fddb41ecb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:12.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.475+0000 7fddba000640 1 --2- 172.21.15.19:0/3597185373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fddb414ba20 0x7fddb4145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:12.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.475+0000 7fddba000640 1 --2- 172.21.15.19:0/3597185373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fddb40cfda0 0x7fddb41f8ac0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:12.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.475+0000 7fddba000640 1 -- 172.21.15.19:0/3597185373 >> 172.21.15.19:0/3597185373 conn(0x7fddb40ccad0 msgr2=0x7fddb40afaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:12.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.476+0000 7fddba000640 1 -- 172.21.15.19:0/3597185373 shutdown_connections 2024-03-28T13:21:12.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:12.476+0000 7fddba000640 1 -- 172.21.15.19:0/3597185373 wait complete. 2024-03-28T13:21:12.495 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T13:21:12.500 INFO:tasks.ceph.mds.d.smithi188.stderr: -8591> 2024-03-28T13:20:59.379+0000 7f18b8000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:21:12.501 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T13:21:12.573 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-03-28T13:21:12.658 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-03-28T13:21:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-03-28T13:21:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T13:21:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-03-28T13:21:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-03-28T13:21:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-03-28T13:21:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-03-28T13:21:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-03-28T13:21:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.659 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-03-28T13:21:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-03-28T13:21:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T13:21:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-03-28T13:21:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T13:21:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T13:21:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-03-28T13:21:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-03-28T13:21:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-03-28T13:21:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-03-28T13:21:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-03-28T13:21:12.660 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-03-28T13:21:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-03-28T13:21:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-03-28T13:21:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T13:21:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T13:21:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-03-28T13:21:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T13:21:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-03-28T13:21:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-03-28T13:21:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-03-28T13:21:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-03-28T13:21:12.661 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-03-28T13:21:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.19,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-03-28T13:21:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2024-03-28T13:21:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = e6b569d9-f2ce-49dc-bbe7-c0ab33080203 2024-03-28T13:21:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.662 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-03-28T13:21:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-03-28T13:21:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-03-28T13:21:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-03-28T13:21:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-03-28T13:21:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-03-28T13:21:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-03-28T13:21:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.662 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-03-28T13:21:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-03-28T13:21:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-03-28T13:21:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-03-28T13:21:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-03-28T13:21:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-03-28T13:21:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-03-28T13:21:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-03-28T13:21:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-03-28T13:21:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.663 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: bdev async discard = True 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: bdev enable discard = True 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore allocator = bitmap 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 1/20 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 1/20 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 4/10 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T13:21:12.664 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.665 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-03-28T13:21:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-03-28T13:21:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-03-28T13:21:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-03-28T13:21:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-03-28T13:21:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-03-28T13:21:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-03-28T13:21:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-03-28T13:21:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T13:21:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T13:21:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T13:21:12.666 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T13:21:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T13:21:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T13:21:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-03-28T13:21:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-03-28T13:21:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-03-28T13:21:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: mon down mkfs grace = 300 2024-03-28T13:21:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2024-03-28T13:21:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.667 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-03-28T13:21:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-03-28T13:21:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-03-28T13:21:12.667 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-03-28T13:21:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T13:21:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T13:21:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2024-03-28T13:21:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-03-28T13:21:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2024-03-28T13:21:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-03-28T13:21:12.668 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-03-28T13:21:12.668 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-03-28T13:21:12.668 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-03-28T13:21:12.668 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-03-28T13:21:12.668 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-03-28T13:21:12.668 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-03-28T13:21:12.668 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-03-28T13:21:12.668 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-03-28T13:21:12.669 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2024-03-28T13:21:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2024-03-28T13:21:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2024-03-28T13:21:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2024-03-28T13:21:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = True 2024-03-28T13:21:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2024-03-28T13:21:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = False 2024-03-28T13:21:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-03-28T13:21:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-03-28T13:21:12.669 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-03-28T13:21:12.673 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-03-28T13:21:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2024-03-28T13:21:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T13:21:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2024-03-28T13:21:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2024-03-28T13:21:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2024-03-28T13:21:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2024-03-28T13:21:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2024-03-28T13:21:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.673 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2024-03-28T13:21:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2024-03-28T13:21:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T13:21:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2024-03-28T13:21:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T13:21:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T13:21:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2024-03-28T13:21:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2024-03-28T13:21:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2024-03-28T13:21:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2024-03-28T13:21:12.674 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2024-03-28T13:21:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2024-03-28T13:21:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2024-03-28T13:21:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2024-03-28T13:21:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T13:21:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T13:21:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: mon_allow_pool_size_one = true 2024-03-28T13:21:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T13:21:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2024-03-28T13:21:12.675 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2024-03-28T13:21:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2024-03-28T13:21:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2024-03-28T13:21:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: mon health detail to clog = false 2024-03-28T13:21:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = "172.21.15.19,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-03-28T13:21:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on skipped message = False 2024-03-28T13:21:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = e6b569d9-f2ce-49dc-bbe7-c0ab33080203 2024-03-28T13:21:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.676 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2024-03-28T13:21:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2024-03-28T13:21:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2024-03-28T13:21:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2024-03-28T13:21:12.676 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock profile = high_recovery_ops 2024-03-28T13:21:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2024-03-28T13:21:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2024-03-28T13:21:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2024-03-28T13:21:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2024-03-28T13:21:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2024-03-28T13:21:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug trim objects = true 2024-03-28T13:21:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2024-03-28T13:21:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2024-03-28T13:21:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2024-03-28T13:21:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.677 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: bdev async discard = True 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: bdev enable discard = True 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore allocator = bitmap 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore block size = 96636764160 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore fsck on mount = True 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluefs = 1/20 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluestore = 1/20 2024-03-28T13:21:12.678 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 20 2024-03-28T13:21:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: debug rocksdb = 4/10 2024-03-28T13:21:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T13:21:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd full ratio = 0.9 2024-03-28T13:21:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd nearfull ratio = 0.8 2024-03-28T13:21:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: osd failsafe full ratio = 0.95 2024-03-28T13:21:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: osd objectstore = bluestore 2024-03-28T13:21:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-03-28T13:21:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.679 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2024-03-28T13:21:12.679 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2024-03-28T13:21:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-03-28T13:21:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-03-28T13:21:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: mgr/telemetry/nag = false 2024-03-28T13:21:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-03-28T13:21:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.680 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2024-03-28T13:21:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-03-28T13:21:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2024-03-28T13:21:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-03-28T13:21:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2024-03-28T13:21:12.680 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: # 1m isn't quite enough 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: mon_down_mkfs_grace = 2m 2024-03-28T13:21:12.681 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_filestore_osds = false 2024-03-28T13:21:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: mon down mkfs grace = 300 2024-03-28T13:21:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: mon op complaint time = 120 2024-03-28T13:21:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.682 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2024-03-28T13:21:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2024-03-28T13:21:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2024-03-28T13:21:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2024-03-28T13:21:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T13:21:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T13:21:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: client mount timeout = 600 2024-03-28T13:21:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-03-28T13:21:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = "" 2024-03-28T13:21:12.682 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-03-28T13:21:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-03-28T13:21:12.683 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2024-03-28T13:21:12.683 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.b] 2024-03-28T13:21:12.683 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.c] 2024-03-28T13:21:12.683 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a] 2024-03-28T13:21:12.683 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c] 2024-03-28T13:21:12.683 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b] 2024-03-28T13:21:12.683 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d] 2024-03-28T13:21:12.683 INFO:teuthology.orchestra.run.smithi019.stdout:[mds] 2024-03-28T13:21:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds = 20 2024-03-28T13:21:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds balancer = 20 2024-03-28T13:21:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug frag = True 2024-03-28T13:21:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug scatterstat = True 2024-03-28T13:21:12.683 INFO:teuthology.orchestra.run.smithi019.stdout: mds op complaint time = 180 2024-03-28T13:21:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: mds verify scatter = False 2024-03-28T13:21:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-03-28T13:21:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-03-28T13:21:12.684 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-03-28T13:21:12.722 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T13:21:12.727 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T13:21:12.904 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-03-28T13:21:12.904 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2024-03-28T13:21:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T13:21:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2024-03-28T13:21:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2024-03-28T13:21:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2024-03-28T13:21:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2024-03-28T13:21:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2024-03-28T13:21:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2024-03-28T13:21:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2024-03-28T13:21:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.905 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T13:21:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2024-03-28T13:21:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T13:21:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T13:21:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2024-03-28T13:21:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2024-03-28T13:21:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2024-03-28T13:21:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2024-03-28T13:21:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2024-03-28T13:21:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2024-03-28T13:21:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2024-03-28T13:21:12.906 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: mon_allow_pool_size_one = true 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: mon health detail to clog = false 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = "172.21.15.19,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on skipped message = False 2024-03-28T13:21:12.907 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = e6b569d9-f2ce-49dc-bbe7-c0ab33080203 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock profile = high_recovery_ops 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug trim objects = true 2024-03-28T13:21:12.908 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: bdev async discard = True 2024-03-28T13:21:12.909 INFO:teuthology.orchestra.run.smithi019.stdout: bdev enable discard = True 2024-03-28T13:21:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore allocator = bitmap 2024-03-28T13:21:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore block size = 96636764160 2024-03-28T13:21:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore fsck on mount = True 2024-03-28T13:21:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluefs = 1/20 2024-03-28T13:21:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluestore = 1/20 2024-03-28T13:21:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 20 2024-03-28T13:21:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: debug rocksdb = 4/10 2024-03-28T13:21:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T13:21:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd full ratio = 0.9 2024-03-28T13:21:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd nearfull ratio = 0.8 2024-03-28T13:21:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: osd failsafe full ratio = 0.95 2024-03-28T13:21:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: osd objectstore = bluestore 2024-03-28T13:21:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-03-28T13:21:12.910 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: mgr/telemetry/nag = false 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-03-28T13:21:12.911 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2024-03-28T13:21:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2024-03-28T13:21:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2024-03-28T13:21:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2024-03-28T13:21:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T13:21:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T13:21:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T13:21:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T13:21:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T13:21:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T13:21:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.912 INFO:teuthology.orchestra.run.smithi019.stdout: # 1m isn't quite enough 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: mon_down_mkfs_grace = 2m 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_filestore_osds = false 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: mon down mkfs grace = 300 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: mon op complaint time = 120 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: client mount timeout = 600 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:12.913 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = "" 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.b] 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.c] 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a] 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c] 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b] 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d] 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout:[mds] 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds = 20 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds balancer = 20 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug frag = True 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug scatterstat = False 2024-03-28T13:21:12.914 INFO:teuthology.orchestra.run.smithi019.stdout: mds op complaint time = 180 2024-03-28T13:21:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: mds verify scatter = False 2024-03-28T13:21:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-03-28T13:21:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-03-28T13:21:12.915 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-03-28T13:21:12.915 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-03-28T13:21:12.915 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-03-28T13:21:12.915 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T13:21:12.915 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T13:21:12.916 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T13:21:12.917 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.917 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-03-28T13:21:12.917 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.917 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-03-28T13:21:12.917 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-03-28T13:21:12.917 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-03-28T13:21:12.917 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-03-28T13:21:12.917 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-03-28T13:21:12.917 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-03-28T13:21:12.917 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-03-28T13:21:12.917 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T13:21:12.917 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T13:21:12.917 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-03-28T13:21:12.917 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.917 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.19,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = e6b569d9-f2ce-49dc-bbe7-c0ab33080203 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.918 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-03-28T13:21:12.919 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: bdev async discard = True 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: bdev enable discard = True 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore allocator = bitmap 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 1/20 2024-03-28T13:21:12.920 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 1/20 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 4/10 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-03-28T13:21:12.921 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:12.922 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:12.922 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-03-28T13:21:12.922 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-03-28T13:21:12.922 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.922 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-03-28T13:21:12.922 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:12.922 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-03-28T13:21:12.922 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-03-28T13:21:12.922 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-03-28T13:21:12.922 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-03-28T13:21:12.922 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-03-28T13:21:12.922 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:12.922 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-03-28T13:21:12.922 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-03-28T13:21:12.922 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: mon down mkfs grace = 300 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2024-03-28T13:21:12.923 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-03-28T13:21:12.924 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-03-28T13:21:12.925 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-03-28T13:21:12.925 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-03-28T13:21:12.925 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-03-28T13:21:12.925 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2024-03-28T13:21:12.925 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2024-03-28T13:21:12.925 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2024-03-28T13:21:12.925 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:12.925 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2024-03-28T13:21:12.925 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = False 2024-03-28T13:21:12.925 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2024-03-28T13:21:12.925 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = False 2024-03-28T13:21:12.925 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-03-28T13:21:12.925 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-03-28T13:21:12.925 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-03-28T13:21:12.954 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-03-28T13:21:13.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.069+0000 7ff24cb2ef40 1 Processor -- start 2024-03-28T13:21:13.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.069+0000 7ff24cb2ef40 1 -- start start 2024-03-28T13:21:13.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.070+0000 7ff24cb2ef40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556004991a40 0x556004a394f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.070+0000 7ff24cb2ef40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560049b1d20 0x556004a42e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.070+0000 7ff24cb2ef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556004a40d60 0x556004a3d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.070+0000 7ff24cb2ef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55600490b2c0 con 0x556004a40d60 2024-03-28T13:21:13.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.070+0000 7ff24cb2ef40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5560049b1800 con 0x5560049b1d20 2024-03-28T13:21:13.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.070+0000 7ff24cb2ef40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x556004a3ce60 con 0x556004991a40 2024-03-28T13:21:13.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.070+0000 7ff24a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556004a40d60 0x556004a3d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.070+0000 7ff24a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556004a40d60 0x556004a3d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33876/0 (socket says 172.21.15.19:33876) 2024-03-28T13:21:13.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.070+0000 7ff24a800640 1 -- 172.21.15.19:0/3367423099 learned_addr learned my addr 172.21.15.19:0/3367423099 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:13.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.070+0000 7ff249e00640 1 --2- 172.21.15.19:0/3367423099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556004991a40 0x556004a394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.070+0000 7ff249400640 1 --2- 172.21.15.19:0/3367423099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560049b1d20 0x556004a42e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.071+0000 7ff24a800640 1 -- 172.21.15.19:0/3367423099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556004991a40 msgr2=0x556004a394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:13.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.071+0000 7ff24a800640 1 --2- 172.21.15.19:0/3367423099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556004991a40 0x556004a394f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.071+0000 7ff24a800640 1 -- 172.21.15.19:0/3367423099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560049b1d20 msgr2=0x556004a42e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:13.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.071+0000 7ff24a800640 1 --2- 172.21.15.19:0/3367423099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560049b1d20 0x556004a42e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.071+0000 7ff24a800640 1 -- 172.21.15.19:0/3367423099 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556004a3cb80 con 0x556004a40d60 2024-03-28T13:21:13.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.071+0000 7ff24a800640 1 --2- 172.21.15.19:0/3367423099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556004a40d60 0x556004a3d5b0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7ff244002850 tx=0x7ff244002d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb587b618eb9efc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:13.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.071+0000 7ff243e00640 1 -- 172.21.15.19:0/3367423099 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244016020 con 0x556004a40d60 2024-03-28T13:21:13.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.071+0000 7ff243e00640 1 -- 172.21.15.19:0/3367423099 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff24400d9c0 con 0x556004a40d60 2024-03-28T13:21:13.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.071+0000 7ff243e00640 1 -- 172.21.15.19:0/3367423099 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24400ddb0 con 0x556004a40d60 2024-03-28T13:21:13.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.072+0000 7ff24cb2ef40 1 -- 172.21.15.19:0/3367423099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556004a40d60 msgr2=0x556004a3d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:13.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.072+0000 7ff24cb2ef40 1 --2- 172.21.15.19:0/3367423099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556004a40d60 0x556004a3d5b0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7ff244002850 tx=0x7ff244002d10 comp rx=0 tx=0).stop 2024-03-28T13:21:13.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.072+0000 7ff24cb2ef40 1 -- 172.21.15.19:0/3367423099 shutdown_connections 2024-03-28T13:21:13.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.072+0000 7ff24cb2ef40 1 --2- 172.21.15.19:0/3367423099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556004a40d60 0x556004a3d5b0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.072+0000 7ff24cb2ef40 1 --2- 172.21.15.19:0/3367423099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5560049b1d20 0x556004a42e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.072+0000 7ff24cb2ef40 1 --2- 172.21.15.19:0/3367423099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x556004991a40 0x556004a394f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.072+0000 7ff24cb2ef40 1 -- 172.21.15.19:0/3367423099 >> 172.21.15.19:0/3367423099 conn(0x5560049b2630 msgr2=0x556004993750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:13.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.072+0000 7ff24cb2ef40 1 -- 172.21.15.19:0/3367423099 shutdown_connections 2024-03-28T13:21:13.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.072+0000 7ff24cb2ef40 1 -- 172.21.15.19:0/3367423099 wait complete. 2024-03-28T13:21:13.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.073+0000 7ff24cb2ef40 1 Processor -- start 2024-03-28T13:21:13.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.073+0000 7ff24cb2ef40 1 -- start start 2024-03-28T13:21:13.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.074+0000 7ff24cb2ef40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556004991a40 0x556004a33770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.074+0000 7ff24cb2ef40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560049b1d20 0x556004967da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.074+0000 7ff24cb2ef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556004a40d60 0x5560049af600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.074+0000 7ff24cb2ef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556004a3e280 con 0x556004a40d60 2024-03-28T13:21:13.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.074+0000 7ff24cb2ef40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5560049927a0 con 0x556004991a40 2024-03-28T13:21:13.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.074+0000 7ff24cb2ef40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x556004a3daf0 con 0x5560049b1d20 2024-03-28T13:21:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.074+0000 7ff24a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556004a40d60 0x5560049af600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.074+0000 7ff24a800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556004a40d60 0x5560049af600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33882/0 (socket says 172.21.15.19:33882) 2024-03-28T13:21:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.074+0000 7ff24a800640 1 -- 172.21.15.19:0/1378791222 learned_addr learned my addr 172.21.15.19:0/1378791222 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.075+0000 7ff249400640 1 --2- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560049b1d20 0x556004967da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.075+0000 7ff249e00640 1 --2- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556004991a40 0x556004a33770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.075+0000 7ff24a800640 1 -- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560049b1d20 msgr2=0x556004967da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.075+0000 7ff24a800640 1 --2- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5560049b1d20 0x556004967da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.075+0000 7ff24a800640 1 -- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556004991a40 msgr2=0x556004a33770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.075+0000 7ff24a800640 1 --2- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x556004991a40 0x556004a33770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.075+0000 7ff24a800640 1 -- 172.21.15.19:0/1378791222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556004969350 con 0x556004a40d60 2024-03-28T13:21:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.075+0000 7ff24a800640 1 --2- 172.21.15.19:0/1378791222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556004a40d60 0x5560049af600 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7ff244015040 tx=0x7ff2440033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.075+0000 7ff242a00640 1 -- 172.21.15.19:0/1378791222 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244012cf0 con 0x556004a40d60 2024-03-28T13:21:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.075+0000 7ff242a00640 1 -- 172.21.15.19:0/1378791222 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff24401f070 con 0x556004a40d60 2024-03-28T13:21:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.075+0000 7ff24cb2ef40 1 -- 172.21.15.19:0/1378791222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5560049672c0 con 0x556004a40d60 2024-03-28T13:21:13.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.075+0000 7ff242a00640 1 -- 172.21.15.19:0/1378791222 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2440246b0 con 0x556004a40d60 2024-03-28T13:21:13.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.076+0000 7ff24cb2ef40 1 -- 172.21.15.19:0/1378791222 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556004a333f0 con 0x556004a40d60 2024-03-28T13:21:13.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.078+0000 7ff242a00640 1 -- 172.21.15.19:0/1378791222 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff244019020 con 0x556004a40d60 2024-03-28T13:21:13.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.079+0000 7ff242a00640 1 --2- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff22c06af10 0x7ff22c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.079+0000 7ff242a00640 1 -- 172.21.15.19:0/1378791222 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7ff24408bea0 con 0x556004a40d60 2024-03-28T13:21:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.079+0000 7ff24cb2ef40 1 --2- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5560049afb40 0x556004a25340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.079+0000 7ff24cb2ef40 1 -- 172.21.15.19:0/1378791222 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120) -- 0x556004ad92f0 con 0x5560049afb40 2024-03-28T13:21:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.079+0000 7ff249e00640 1 --2- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff22c06af10 0x7ff22c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.080+0000 7ff249400640 1 --2- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5560049afb40 0x556004a25340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.081+0000 7ff249e00640 1 --2- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff22c06af10 0x7ff22c06d3d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff23c054000 tx=0x7ff23c04e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:13.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.081+0000 7ff249400640 1 --2- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5560049afb40 0x556004a25340 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:13.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.085+0000 7ff249400640 1 -- 172.21.15.19:0/1378791222 <== osd.4 v2:172.21.15.188:6808/1845695132 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v120'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff238010350 con 0x5560049afb40 2024-03-28T13:21:13.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.085+0000 7ff24cb2ef40 1 -- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5560049afb40 msgr2=0x556004a25340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:13.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.085+0000 7ff24cb2ef40 1 --2- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5560049afb40 0x556004a25340 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.085+0000 7ff24cb2ef40 1 -- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff22c06af10 msgr2=0x7ff22c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:13.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.085+0000 7ff24cb2ef40 1 --2- 172.21.15.19:0/1378791222 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff22c06af10 0x7ff22c06d3d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff23c054000 tx=0x7ff23c04e000 comp rx=0 tx=0).stop 2024-03-28T13:21:13.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.085+0000 7ff24cb2ef40 1 -- 172.21.15.19:0/1378791222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556004a40d60 msgr2=0x5560049af600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:13.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.085+0000 7ff24cb2ef40 1 --2- 172.21.15.19:0/1378791222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556004a40d60 0x5560049af600 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7ff244015040 tx=0x7ff2440033a0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.085+0000 7ff24a800640 1 -- 172.21.15.19:0/1378791222 reap_dead start 2024-03-28T13:21:13.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.085+0000 7ff24cb2ef40 1 -- 172.21.15.19:0/1378791222 shutdown_connections 2024-03-28T13:21:13.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.085+0000 7ff24cb2ef40 1 -- 172.21.15.19:0/1378791222 >> 172.21.15.19:0/1378791222 conn(0x5560049b2630 msgr2=0x556004a3e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:13.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.085+0000 7ff24cb2ef40 1 -- 172.21.15.19:0/1378791222 shutdown_connections 2024-03-28T13:21:13.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.085+0000 7ff24cb2ef40 1 -- 172.21.15.19:0/1378791222 wait complete. 2024-03-28T13:21:13.090 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-28T13:21:13.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.302+0000 7fd1b4000640 1 Processor -- start 2024-03-28T13:21:13.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.302+0000 7fd1b4000640 1 -- start start 2024-03-28T13:21:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.303+0000 7fd1b4000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd1ac162920 0x7fd1ac1670b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.303+0000 7fd1b4000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd1ac163ac0 0x7fd1ac071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.303+0000 7fd1b4000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1ac071d60 con 0x7fd1ac0d0280 2024-03-28T13:21:13.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.303+0000 7fd1b4000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fd1ac163f20 con 0x7fd1ac163ac0 2024-03-28T13:21:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.303+0000 7fd1b4000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fd1ac1665d0 con 0x7fd1ac162920 2024-03-28T13:21:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.304+0000 7fd1b0c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fd1ac163ac0 0x7fd1ac071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.304+0000 7fd1b2000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd1ac0d0280 0x7fd1ac0ad010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:38626/0 (socket says 172.21.15.19:38626) 2024-03-28T13:21:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.304+0000 7fd1b2000640 1 -- 172.21.15.19:0/1850183505 learned_addr learned my addr 172.21.15.19:0/1850183505 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:13.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.304+0000 7fd1b1600640 1 --2- 172.21.15.19:0/1850183505 >> v2:172.21.15.188:3300/0 conn(0x7fd1ac162920 0x7fd1ac1670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.304+0000 7fd1b0c00640 1 -- 172.21.15.19:0/1850183505 >> v2:172.21.15.188:3300/0 conn(0x7fd1ac162920 msgr2=0x7fd1ac1670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.304+0000 7fd1b0c00640 1 --2- 172.21.15.19:0/1850183505 >> v2:172.21.15.188:3300/0 conn(0x7fd1ac162920 0x7fd1ac1670b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.304+0000 7fd1b0c00640 1 -- 172.21.15.19:0/1850183505 >> v1:172.21.15.19:6789/0 conn(0x7fd1ac0d0280 legacy=0x7fd1ac0ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.305+0000 7fd1b0c00640 1 -- 172.21.15.19:0/1850183505 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ac1668b0 con 0x7fd1ac163ac0 2024-03-28T13:21:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.304+0000 7fd1abe00640 1 -- 172.21.15.19:0/1850183505 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3439860863 0 0) 0x7fd1ac071d60 con 0x7fd1ac0d0280 2024-03-28T13:21:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.306+0000 7fd1b0c00640 1 --2- 172.21.15.19:0/1850183505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1ac163ac0 0x7fd1ac071610 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0d0dd0 tx=0x7fd19802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50d59f90aea2a10 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:13.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.306+0000 7fd1abe00640 1 -- 172.21.15.19:0/1850183505 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1980315a0 con 0x7fd1ac163ac0 2024-03-28T13:21:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.306+0000 7fd1abe00640 1 -- 172.21.15.19:0/1850183505 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd198037070 con 0x7fd1ac163ac0 2024-03-28T13:21:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.306+0000 7fd1abe00640 1 -- 172.21.15.19:0/1850183505 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198031960 con 0x7fd1ac163ac0 2024-03-28T13:21:13.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.307+0000 7fd1b4000640 1 -- 172.21.15.19:0/1850183505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1ac163ac0 msgr2=0x7fd1ac071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.307+0000 7fd1b4000640 1 --2- 172.21.15.19:0/1850183505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1ac163ac0 0x7fd1ac071610 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac0d0dd0 tx=0x7fd19802b840 comp rx=0 tx=0).stop 2024-03-28T13:21:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.307+0000 7fd1b4000640 1 -- 172.21.15.19:0/1850183505 shutdown_connections 2024-03-28T13:21:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.307+0000 7fd1b4000640 1 --2- 172.21.15.19:0/1850183505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1ac163ac0 0x7fd1ac071610 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.307+0000 7fd1b4000640 1 --2- 172.21.15.19:0/1850183505 >> v2:172.21.15.188:3300/0 conn(0x7fd1ac162920 0x7fd1ac1670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.307+0000 7fd1b4000640 1 -- 172.21.15.19:0/1850183505 >> 172.21.15.19:0/1850183505 conn(0x7fd1ac06b750 msgr2=0x7fd1ac06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.307+0000 7fd1b4000640 1 -- 172.21.15.19:0/1850183505 shutdown_connections 2024-03-28T13:21:13.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.307+0000 7fd1b4000640 1 -- 172.21.15.19:0/1850183505 wait complete. 2024-03-28T13:21:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.308+0000 7fd1b4000640 1 Processor -- start 2024-03-28T13:21:13.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.309+0000 7fd1b4000640 1 -- start start 2024-03-28T13:21:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b4000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ac0d0280 0x7fd1ac09a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b4000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac162920 0x7fd1ac0cf400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b4000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1ac163ac0 0x7fd1ac0ce630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b4000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac072040 con 0x7fd1ac163ac0 2024-03-28T13:21:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b4000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac162d00 con 0x7fd1ac0d0280 2024-03-28T13:21:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b4000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd1ac0721c0 con 0x7fd1ac162920 2024-03-28T13:21:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1ac163ac0 0x7fd1ac0ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b2000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1ac163ac0 0x7fd1ac0ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33892/0 (socket says 172.21.15.19:33892) 2024-03-28T13:21:13.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b2000640 1 -- 172.21.15.19:0/2466696149 learned_addr learned my addr 172.21.15.19:0/2466696149 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b1600640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ac0d0280 0x7fd1ac09a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b0c00640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac162920 0x7fd1ac0cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b2000640 1 -- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac162920 msgr2=0x7fd1ac0cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b2000640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac162920 0x7fd1ac0cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b2000640 1 -- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ac0d0280 msgr2=0x7fd1ac09a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b2000640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ac0d0280 0x7fd1ac09a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b2000640 1 -- 172.21.15.19:0/2466696149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ac0cd870 con 0x7fd1ac163ac0 2024-03-28T13:21:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b1600640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ac0d0280 0x7fd1ac09a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.310+0000 7fd1b0c00640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac162920 0x7fd1ac0cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.311+0000 7fd1b2000640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1ac163ac0 0x7fd1ac0ce630 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400c440 tx=0x7fd1a400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.311+0000 7fd1a9600640 1 -- 172.21.15.19:0/2466696149 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a40074e0 con 0x7fd1ac163ac0 2024-03-28T13:21:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.311+0000 7fd1a9600640 1 -- 172.21.15.19:0/2466696149 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1a40047e0 con 0x7fd1ac163ac0 2024-03-28T13:21:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.311+0000 7fd1a9600640 1 -- 172.21.15.19:0/2466696149 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4004bd0 con 0x7fd1ac163ac0 2024-03-28T13:21:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.311+0000 7fd1b4000640 1 -- 172.21.15.19:0/2466696149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ac0cdb50 con 0x7fd1ac163ac0 2024-03-28T13:21:13.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.311+0000 7fd1b4000640 1 -- 172.21.15.19:0/2466696149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ac0ceb70 con 0x7fd1ac163ac0 2024-03-28T13:21:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.312+0000 7fd1b4000640 1 -- 172.21.15.19:0/2466696149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1ac0685c0 con 0x7fd1ac163ac0 2024-03-28T13:21:13.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.313+0000 7fd1a9600640 1 -- 172.21.15.19:0/2466696149 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd1a4011070 con 0x7fd1ac163ac0 2024-03-28T13:21:13.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.314+0000 7fd1a9600640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd18c06c530 0x7fd18c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.314+0000 7fd1a9600640 1 -- 172.21.15.19:0/2466696149 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7fd1a408b400 con 0x7fd1ac163ac0 2024-03-28T13:21:13.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.315+0000 7fd1b1600640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd18c06c530 0x7fd18c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.316+0000 7fd1b1600640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd18c06c530 0x7fd18c06e9f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0007330 tx=0x7fd1a000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:13.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.319+0000 7fd1a9600640 1 -- 172.21.15.19:0/2466696149 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd1a4059890 con 0x7fd1ac163ac0 2024-03-28T13:21:13.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.565+0000 7fd1b4000640 1 -- 172.21.15.19:0/2466696149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd1ac15f6f0 con 0x7fd1ac163ac0 2024-03-28T13:21:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.580+0000 7fd1a9600640 1 -- 172.21.15.19:0/2466696149 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v101) ==== 162+0+0 (secure 0 0 0) 0x7fd1a4059190 con 0x7fd1ac163ac0 2024-03-28T13:21:13.580 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-28T13:21:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.583+0000 7fd1b4000640 1 -- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd18c06c530 msgr2=0x7fd18c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.583+0000 7fd1b4000640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd18c06c530 0x7fd18c06e9f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0007330 tx=0x7fd1a000b040 comp rx=0 tx=0).stop 2024-03-28T13:21:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.583+0000 7fd1b4000640 1 -- 172.21.15.19:0/2466696149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1ac163ac0 msgr2=0x7fd1ac0ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.583+0000 7fd1b4000640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1ac163ac0 0x7fd1ac0ce630 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400c440 tx=0x7fd1a400c900 comp rx=0 tx=0).stop 2024-03-28T13:21:13.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.583+0000 7fd1b4000640 1 -- 172.21.15.19:0/2466696149 shutdown_connections 2024-03-28T13:21:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.583+0000 7fd1b4000640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd18c06c530 0x7fd18c06e9f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.583+0000 7fd1b4000640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd1ac163ac0 0x7fd1ac0ce630 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.583+0000 7fd1b4000640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ac162920 0x7fd1ac0cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.583+0000 7fd1b4000640 1 --2- 172.21.15.19:0/2466696149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ac0d0280 0x7fd1ac09a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.583+0000 7fd1b4000640 1 -- 172.21.15.19:0/2466696149 >> 172.21.15.19:0/2466696149 conn(0x7fd1ac06b750 msgr2=0x7fd1ac0b0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.583+0000 7fd1b4000640 1 -- 172.21.15.19:0/2466696149 shutdown_connections 2024-03-28T13:21:13.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.583+0000 7fd1b4000640 1 -- 172.21.15.19:0/2466696149 wait complete. 2024-03-28T13:21:13.605 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:21:13.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.815+0000 7fde92e00640 1 Processor -- start 2024-03-28T13:21:13.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.815+0000 7fde92e00640 1 -- start start 2024-03-28T13:21:13.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.817+0000 7fde92e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde8c161b00 0x7fde8c162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.817+0000 7fde92e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde8c0d0d60 0x7fde8c0b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.817+0000 7fde92e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde8c071b60 con 0x7fde8c0ad120 2024-03-28T13:21:13.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.817+0000 7fde92e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fde8c161ee0 con 0x7fde8c161b00 2024-03-28T13:21:13.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.817+0000 7fde92e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fde8c0b3450 con 0x7fde8c0d0d60 2024-03-28T13:21:13.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.817+0000 7fde8be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde8c161b00 0x7fde8c162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.818+0000 7fde8be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fde8c161b00 0x7fde8c162ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33896/0 (socket says 172.21.15.19:33896) 2024-03-28T13:21:13.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.818+0000 7fde8be00640 1 -- 172.21.15.19:0/2769265274 learned_addr learned my addr 172.21.15.19:0/2769265274 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:13.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.818+0000 7fde90e00640 1 --2- 172.21.15.19:0/2769265274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde8c0d0d60 0x7fde8c0b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.818+0000 7fde8be00640 1 -- 172.21.15.19:0/2769265274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde8c0d0d60 msgr2=0x7fde8c0b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:13.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.818+0000 7fde8be00640 1 --2- 172.21.15.19:0/2769265274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde8c0d0d60 0x7fde8c0b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.818+0000 7fde8be00640 1 -- 172.21.15.19:0/2769265274 >> v1:172.21.15.188:6789/0 conn(0x7fde8c0ad120 legacy=0x7fde8c0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:13.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.818+0000 7fde8be00640 1 -- 172.21.15.19:0/2769265274 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8c071e40 con 0x7fde8c161b00 2024-03-28T13:21:13.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.819+0000 7fde8be00640 1 --2- 172.21.15.19:0/2769265274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde8c161b00 0x7fde8c162ab0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009870 tx=0x7fde7c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16ae7235d306be1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:13.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.820+0000 7fde8b400640 1 -- 172.21.15.19:0/2769265274 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c035020 con 0x7fde8c161b00 2024-03-28T13:21:13.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.820+0000 7fde8b400640 1 -- 172.21.15.19:0/2769265274 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde7c031640 con 0x7fde8c161b00 2024-03-28T13:21:13.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.820+0000 7fde8b400640 1 -- 172.21.15.19:0/2769265274 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c02d040 con 0x7fde8c161b00 2024-03-28T13:21:13.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.820+0000 7fde92e00640 1 -- 172.21.15.19:0/2769265274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde8c161b00 msgr2=0x7fde8c162ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:13.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.820+0000 7fde92e00640 1 --2- 172.21.15.19:0/2769265274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde8c161b00 0x7fde8c162ab0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009870 tx=0x7fde7c02b8f0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.821+0000 7fde92e00640 1 -- 172.21.15.19:0/2769265274 shutdown_connections 2024-03-28T13:21:13.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.821+0000 7fde92e00640 1 --2- 172.21.15.19:0/2769265274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde8c0d0d60 0x7fde8c0b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.821+0000 7fde92e00640 1 --2- 172.21.15.19:0/2769265274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde8c161b00 0x7fde8c162ab0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.821+0000 7fde92e00640 1 -- 172.21.15.19:0/2769265274 >> 172.21.15.19:0/2769265274 conn(0x7fde8c06b990 msgr2=0x7fde8c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:13.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.821+0000 7fde92e00640 1 -- 172.21.15.19:0/2769265274 shutdown_connections 2024-03-28T13:21:13.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.821+0000 7fde92e00640 1 -- 172.21.15.19:0/2769265274 wait complete. 2024-03-28T13:21:13.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.823+0000 7fde92e00640 1 Processor -- start 2024-03-28T13:21:13.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.823+0000 7fde92e00640 1 -- start start 2024-03-28T13:21:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.824+0000 7fde92e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde8c0ad120 0x7fde8c14f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.824+0000 7fde92e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde8c0d0d60 0x7fde8c0b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.824+0000 7fde92e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde8c161b00 0x7fde8c0b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.824+0000 7fde92e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c162ff0 con 0x7fde8c161b00 2024-03-28T13:21:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.824+0000 7fde92e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c0b3730 con 0x7fde8c0d0d60 2024-03-28T13:21:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.824+0000 7fde92e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fde8c0717f0 con 0x7fde8c0ad120 2024-03-28T13:21:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.824+0000 7fde90e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde8c161b00 0x7fde8c0b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.824+0000 7fde90e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde8c161b00 0x7fde8c0b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33900/0 (socket says 172.21.15.19:33900) 2024-03-28T13:21:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.824+0000 7fde90e00640 1 -- 172.21.15.19:0/3252582643 learned_addr learned my addr 172.21.15.19:0/3252582643 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.824+0000 7fde8be00640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde8c0ad120 0x7fde8c14f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.824+0000 7fde83400640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde8c0d0d60 0x7fde8c0b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.825+0000 7fde90e00640 1 -- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde8c0ad120 msgr2=0x7fde8c14f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.825+0000 7fde90e00640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde8c0ad120 0x7fde8c14f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.825+0000 7fde90e00640 1 -- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde8c0d0d60 msgr2=0x7fde8c0b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.825+0000 7fde90e00640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde8c0d0d60 0x7fde8c0b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.825+0000 7fde90e00640 1 -- 172.21.15.19:0/3252582643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8c1f8990 con 0x7fde8c161b00 2024-03-28T13:21:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.825+0000 7fde83400640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde8c0d0d60 0x7fde8c0b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.825+0000 7fde8be00640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde8c0ad120 0x7fde8c14f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:21:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.825+0000 7fde90e00640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde8c161b00 0x7fde8c0b0d00 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fde84007de0 tx=0x7fde8400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.825+0000 7fde88c00640 1 -- 172.21.15.19:0/3252582643 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84014620 con 0x7fde8c161b00 2024-03-28T13:21:13.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.825+0000 7fde88c00640 1 -- 172.21.15.19:0/3252582643 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde84019070 con 0x7fde8c161b00 2024-03-28T13:21:13.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.825+0000 7fde88c00640 1 -- 172.21.15.19:0/3252582643 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84014a20 con 0x7fde8c161b00 2024-03-28T13:21:13.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.825+0000 7fde92e00640 1 -- 172.21.15.19:0/3252582643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde8c1f7e10 con 0x7fde8c161b00 2024-03-28T13:21:13.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.825+0000 7fde92e00640 1 -- 172.21.15.19:0/3252582643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde8c1f83d0 con 0x7fde8c161b00 2024-03-28T13:21:13.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.827+0000 7fde92e00640 1 -- 172.21.15.19:0/3252582643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde8c0685c0 con 0x7fde8c161b00 2024-03-28T13:21:13.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.828+0000 7fde88c00640 1 -- 172.21.15.19:0/3252582643 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fde84014bc0 con 0x7fde8c161b00 2024-03-28T13:21:13.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.829+0000 7fde88c00640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fde6806c530 0x7fde6806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:13.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.830+0000 7fde88c00640 1 -- 172.21.15.19:0/3252582643 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7fde8408c080 con 0x7fde8c161b00 2024-03-28T13:21:13.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.830+0000 7fde8be00640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fde6806c530 0x7fde6806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:13.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.831+0000 7fde8be00640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fde6806c530 0x7fde6806e9f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fde7c036000 tx=0x7fde7c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:13.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:13.834+0000 7fde88c00640 1 -- 172.21.15.19:0/3252582643 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fde8405a630 con 0x7fde8c161b00 2024-03-28T13:21:14.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:14.076+0000 7fde92e00640 1 -- 172.21.15.19:0/3252582643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde8c06f3a0 con 0x7fde8c161b00 2024-03-28T13:21:14.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:14.077+0000 7fde88c00640 1 -- 172.21.15.19:0/3252582643 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5179 (secure 0 0 0) 0x7fde84059f30 con 0x7fde8c161b00 2024-03-28T13:21:14.077 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:14.077 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":102,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3823671990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3823671990},{"type":"v1","addr":"172.21.15.188:6837","nonce":3823671990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/22158043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":22158043},{"type":"v1","addr":"172.21.15.188:6836","nonce":22158043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":8688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/230519675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":230519675},{"type":"v1","addr":"172.21.15.19:6834","nonce":230519675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:20:52.901269+0000","modified":"2024-03-28T13:21:13.585262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8691},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8691":{"gid":8691,"name":"a","rank":0,"incarnation":102,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/3930967142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3930967142},{"type":"v1","addr":"172.21.15.19:6835","nonce":3930967142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-03-28T13:21:14.077 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 102 2024-03-28T13:21:14.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:14.080+0000 7fde92e00640 1 -- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fde6806c530 msgr2=0x7fde6806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:14.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:14.080+0000 7fde92e00640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fde6806c530 0x7fde6806e9f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fde7c036000 tx=0x7fde7c03c040 comp rx=0 tx=0).stop 2024-03-28T13:21:14.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:14.080+0000 7fde92e00640 1 -- 172.21.15.19:0/3252582643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde8c161b00 msgr2=0x7fde8c0b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:14.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:14.080+0000 7fde92e00640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde8c161b00 0x7fde8c0b0d00 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fde84007de0 tx=0x7fde8400e4b0 comp rx=0 tx=0).stop 2024-03-28T13:21:14.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:14.080+0000 7fde92e00640 1 -- 172.21.15.19:0/3252582643 shutdown_connections 2024-03-28T13:21:14.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:14.080+0000 7fde92e00640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fde6806c530 0x7fde6806e9f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:14.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:14.080+0000 7fde92e00640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fde8c161b00 0x7fde8c0b0d00 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:14.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:14.080+0000 7fde92e00640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde8c0d0d60 0x7fde8c0b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:14.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:14.080+0000 7fde92e00640 1 --2- 172.21.15.19:0/3252582643 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde8c0ad120 0x7fde8c14f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:14.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:14.080+0000 7fde92e00640 1 -- 172.21.15.19:0/3252582643 >> 172.21.15.19:0/3252582643 conn(0x7fde8c06b990 msgr2=0x7fde8c0af250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:14.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:14.080+0000 7fde92e00640 1 -- 172.21.15.19:0/3252582643 shutdown_connections 2024-03-28T13:21:14.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:14.080+0000 7fde92e00640 1 -- 172.21.15.19:0/3252582643 wait complete. 2024-03-28T13:21:14.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:20:52.901269+0000', 'modified': '2024-03-28T13:21:13.585262+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8691}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8691': {'gid': 8691, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/3930967142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 3930967142}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 3930967142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-28T13:21:14.503 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8691:up:replay 2024-03-28T13:21:15.504 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:21:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.739+0000 7f2764000640 1 Processor -- start 2024-03-28T13:21:15.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.740+0000 7f2764000640 1 -- start start 2024-03-28T13:21:15.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.741+0000 7f2764000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f275c15d120 0x7f275c162e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:15.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.741+0000 7f2764000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f275c0af630 0x7f275c166b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:15.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.741+0000 7f2764000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f275c1633d0 con 0x7f275c167080 2024-03-28T13:21:15.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.741+0000 7f2764000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f275c0723c0 con 0x7f275c15d120 2024-03-28T13:21:15.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.741+0000 7f2764000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f275c163650 con 0x7f275c0af630 2024-03-28T13:21:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.742+0000 7f2761600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f275c15d120 0x7f275c162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.742+0000 7f2761600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f275c15d120 0x7f275c162e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52638/0 (socket says 172.21.15.19:52638) 2024-03-28T13:21:15.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.742+0000 7f2761600640 1 -- 172.21.15.19:0/501964 learned_addr learned my addr 172.21.15.19:0/501964 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:15.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.742+0000 7f2760c00640 1 --2- 172.21.15.19:0/501964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f275c0af630 0x7f275c166b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:15.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.743+0000 7f2761600640 1 -- 172.21.15.19:0/501964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f275c0af630 msgr2=0x7f275c166b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:15.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.743+0000 7f2761600640 1 --2- 172.21.15.19:0/501964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f275c0af630 0x7f275c166b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:15.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.743+0000 7f2761600640 1 -- 172.21.15.19:0/501964 >> v1:172.21.15.188:6789/0 conn(0x7f275c167080 legacy=0x7f275c0b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:15.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.743+0000 7f2761600640 1 -- 172.21.15.19:0/501964 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275c163930 con 0x7f275c15d120 2024-03-28T13:21:15.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.744+0000 7f2761600640 1 --2- 172.21.15.19:0/501964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c15d120 0x7f275c162e90 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f2750009870 tx=0x7f275002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816213c196780ab1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:15.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.744+0000 7f275be00640 1 -- 172.21.15.19:0/501964 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750035020 con 0x7f275c15d120 2024-03-28T13:21:15.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.744+0000 7f275be00640 1 -- 172.21.15.19:0/501964 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2750031640 con 0x7f275c15d120 2024-03-28T13:21:15.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.744+0000 7f275be00640 1 -- 172.21.15.19:0/501964 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275002d040 con 0x7f275c15d120 2024-03-28T13:21:15.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.745+0000 7f2764000640 1 -- 172.21.15.19:0/501964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c15d120 msgr2=0x7f275c162e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:15.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.745+0000 7f2764000640 1 --2- 172.21.15.19:0/501964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c15d120 0x7f275c162e90 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f2750009870 tx=0x7f275002b8f0 comp rx=0 tx=0).stop 2024-03-28T13:21:15.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.745+0000 7f2764000640 1 -- 172.21.15.19:0/501964 shutdown_connections 2024-03-28T13:21:15.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.745+0000 7f2764000640 1 --2- 172.21.15.19:0/501964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f275c0af630 0x7f275c166b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:15.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.745+0000 7f2764000640 1 --2- 172.21.15.19:0/501964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c15d120 0x7f275c162e90 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:15.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.745+0000 7f2764000640 1 -- 172.21.15.19:0/501964 >> 172.21.15.19:0/501964 conn(0x7f275c06b990 msgr2=0x7f275c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:15.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.745+0000 7f2764000640 1 -- 172.21.15.19:0/501964 shutdown_connections 2024-03-28T13:21:15.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.745+0000 7f2764000640 1 -- 172.21.15.19:0/501964 wait complete. 2024-03-28T13:21:15.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.747+0000 7f2764000640 1 Processor -- start 2024-03-28T13:21:15.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.747+0000 7f2764000640 1 -- start start 2024-03-28T13:21:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.748+0000 7f2764000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f275c0af630 0x7f275c156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.748+0000 7f2764000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c15d120 0x7f275c0b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:15.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.748+0000 7f2764000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f275c167080 0x7f275c1ec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.748+0000 7f2764000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f275c0b3c10 con 0x7f275c15d120 2024-03-28T13:21:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.748+0000 7f2764000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f275c166620 con 0x7f275c0af630 2024-03-28T13:21:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.748+0000 7f2764000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f275c0b3d90 con 0x7f275c167080 2024-03-28T13:21:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.748+0000 7f2760c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c15d120 0x7f275c0b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.748+0000 7f2760c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c15d120 0x7f275c0b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52654/0 (socket says 172.21.15.19:52654) 2024-03-28T13:21:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.748+0000 7f2760c00640 1 -- 172.21.15.19:0/2488374485 learned_addr learned my addr 172.21.15.19:0/2488374485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.749+0000 7f2761600640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f275c0af630 0x7f275c156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.749+0000 7f2760c00640 1 -- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f275c167080 msgr2=0x7f275c1ec9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.749+0000 7f2762000640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f275c167080 0x7f275c1ec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.749+0000 7f2760c00640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f275c167080 0x7f275c1ec9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.749+0000 7f2760c00640 1 -- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f275c0af630 msgr2=0x7f275c156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.749+0000 7f2760c00640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f275c0af630 0x7f275c156810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.749+0000 7f2760c00640 1 -- 172.21.15.19:0/2488374485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275c156d50 con 0x7f275c15d120 2024-03-28T13:21:15.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.749+0000 7f2761600640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f275c0af630 0x7f275c156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:15.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.749+0000 7f2762000640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f275c167080 0x7f275c1ec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:15.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.749+0000 7f2760c00640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c15d120 0x7f275c0b1810 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f274800d800 tx=0x7f274800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:15.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.749+0000 7f2759600640 1 -- 172.21.15.19:0/2488374485 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274800b6c0 con 0x7f275c15d120 2024-03-28T13:21:15.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.750+0000 7f2759600640 1 -- 172.21.15.19:0/2488374485 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2748019070 con 0x7f275c15d120 2024-03-28T13:21:15.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.750+0000 7f2759600640 1 -- 172.21.15.19:0/2488374485 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274800ba80 con 0x7f275c15d120 2024-03-28T13:21:15.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.750+0000 7f2764000640 1 -- 172.21.15.19:0/2488374485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275c0b27b0 con 0x7f275c15d120 2024-03-28T13:21:15.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.750+0000 7f2764000640 1 -- 172.21.15.19:0/2488374485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f275c0b24d0 con 0x7f275c15d120 2024-03-28T13:21:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.751+0000 7f2764000640 1 -- 172.21.15.19:0/2488374485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f275c0685c0 con 0x7f275c15d120 2024-03-28T13:21:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.752+0000 7f2759600640 1 -- 172.21.15.19:0/2488374485 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f274800bc20 con 0x7f275c15d120 2024-03-28T13:21:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.753+0000 7f2759600640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f273c06c530 0x7f273c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:15.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.753+0000 7f2759600640 1 -- 172.21.15.19:0/2488374485 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7f274808bc80 con 0x7f275c15d120 2024-03-28T13:21:15.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.753+0000 7f2761600640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f273c06c530 0x7f273c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:15.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.755+0000 7f2761600640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f273c06c530 0x7f273c06e9f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2750036000 tx=0x7f2750008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:15.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.757+0000 7f2759600640 1 -- 172.21.15.19:0/2488374485 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f274805a1a0 con 0x7f275c15d120 2024-03-28T13:21:15.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:15.999+0000 7f2764000640 1 -- 172.21.15.19:0/2488374485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f275c06f3a0 con 0x7f275c15d120 2024-03-28T13:21:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:16.000+0000 7f2759600640 1 -- 172.21.15.19:0/2488374485 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5179 (secure 0 0 0) 0x7f2748059aa0 con 0x7f275c15d120 2024-03-28T13:21:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:16.000 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3823671990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3823671990},{"type":"v1","addr":"172.21.15.188:6837","nonce":3823671990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/22158043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":22158043},{"type":"v1","addr":"172.21.15.188:6836","nonce":22158043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":8688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/230519675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":230519675},{"type":"v1","addr":"172.21.15.19:6834","nonce":230519675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:20:52.901269+0000","modified":"2024-03-28T13:21:14.605881+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8691},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8691":{"gid":8691,"name":"a","rank":0,"incarnation":102,"state":"up:rejoin","state_seq":9,"addr":"172.21.15.19:6835/3930967142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3930967142},{"type":"v1","addr":"172.21.15.19:6835","nonce":3930967142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-03-28T13:21:16.000 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 104 2024-03-28T13:21:16.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:16.003+0000 7f2764000640 1 -- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f273c06c530 msgr2=0x7f273c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:16.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:16.003+0000 7f2764000640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f273c06c530 0x7f273c06e9f0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2750036000 tx=0x7f2750008030 comp rx=0 tx=0).stop 2024-03-28T13:21:16.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:16.003+0000 7f2764000640 1 -- 172.21.15.19:0/2488374485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c15d120 msgr2=0x7f275c0b1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:16.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:16.003+0000 7f2764000640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c15d120 0x7f275c0b1810 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f274800d800 tx=0x7f274800dcc0 comp rx=0 tx=0).stop 2024-03-28T13:21:16.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:16.003+0000 7f2764000640 1 -- 172.21.15.19:0/2488374485 shutdown_connections 2024-03-28T13:21:16.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:16.003+0000 7f2764000640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f273c06c530 0x7f273c06e9f0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:16.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:16.003+0000 7f2764000640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f275c167080 0x7f275c1ec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:16.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:16.003+0000 7f2764000640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f275c15d120 0x7f275c0b1810 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:16.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:16.003+0000 7f2764000640 1 --2- 172.21.15.19:0/2488374485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f275c0af630 0x7f275c156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:16.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:16.003+0000 7f2764000640 1 -- 172.21.15.19:0/2488374485 >> 172.21.15.19:0/2488374485 conn(0x7f275c06b990 msgr2=0x7f275c06bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:16.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:16.003+0000 7f2764000640 1 -- 172.21.15.19:0/2488374485 shutdown_connections 2024-03-28T13:21:16.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:16.004+0000 7f2764000640 1 -- 172.21.15.19:0/2488374485 wait complete. 2024-03-28T13:21:16.425 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:20:52.901269+0000', 'modified': '2024-03-28T13:21:14.605881+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8691}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8691': {'gid': 8691, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:rejoin', 'state_seq': 9, 'addr': '172.21.15.19:6835/3930967142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 3930967142}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 3930967142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-28T13:21:16.425 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8691:up:rejoin 2024-03-28T13:21:17.426 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:21:17.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.646+0000 7f2899400640 1 Processor -- start 2024-03-28T13:21:17.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.646+0000 7f2899400640 1 -- start start 2024-03-28T13:21:17.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.647+0000 7f2899400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f28941442c0 0x7f28940ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:17.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.648+0000 7f2899400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2894071460 con 0x7f28940cefe0 2024-03-28T13:21:17.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.648+0000 7f2899400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28940721b0 con 0x7f28940d02a0 2024-03-28T13:21:17.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.648+0000 7f2899400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f2894071d20 con 0x7f28941442c0 2024-03-28T13:21:17.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.648+0000 7f288b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f28940cefe0 0x7f28940cfb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51716/0 (socket says 172.21.15.19:51716) 2024-03-28T13:21:17.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.648+0000 7f288b400640 1 -- 172.21.15.19:0/577444001 learned_addr learned my addr 172.21.15.19:0/577444001 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.648+0000 7f2893e00640 1 --2- 172.21.15.19:0/577444001 >> v2:172.21.15.188:3300/0 conn(0x7f28941442c0 0x7f28940ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.648+0000 7f2893400640 1 -- 172.21.15.19:0/577444001 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2340007402 0 0) 0x7f2894071460 con 0x7f28940cefe0 2024-03-28T13:21:17.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.649+0000 7f2893400640 1 -- 172.21.15.19:0/577444001 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2878003520 con 0x7f28940cefe0 2024-03-28T13:21:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.649+0000 7f2893400640 1 -- 172.21.15.19:0/577444001 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4179019105 0 0) 0x7f2878003520 con 0x7f28940cefe0 2024-03-28T13:21:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.649+0000 7f2893400640 1 -- 172.21.15.19:0/577444001 >> v2:172.21.15.188:3300/0 conn(0x7f28941442c0 msgr2=0x7f28940ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.649+0000 7f2893400640 1 --2- 172.21.15.19:0/577444001 >> v2:172.21.15.188:3300/0 conn(0x7f28941442c0 0x7f28940ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.649+0000 7f2893400640 1 -- 172.21.15.19:0/577444001 >> v1:172.21.15.188:6789/0 conn(0x7f28940d02a0 legacy=0x7f28940b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.649+0000 7f2893400640 1 -- 172.21.15.19:0/577444001 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28940716d0 con 0x7f28940cefe0 2024-03-28T13:21:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.649+0000 7f2893400640 1 -- 172.21.15.19:0/577444001 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f287c001670 con 0x7f28940cefe0 2024-03-28T13:21:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.649+0000 7f2893400640 1 -- 172.21.15.19:0/577444001 >> v1:172.21.15.19:6789/0 conn(0x7f28940cefe0 legacy=0x7f28940cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.649+0000 7f2893400640 1 --2- 172.21.15.19:0/577444001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28780039b0 0x7f2878003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.649+0000 7f2893400640 1 --2- 172.21.15.19:0/577444001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2878004800 0x7f2878004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.649+0000 7f2893400640 1 --2- 172.21.15.19:0/577444001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28780051c0 0x7f287800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.649+0000 7f2893400640 1 -- 172.21.15.19:0/577444001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2894071d20 con 0x7f28780039b0 2024-03-28T13:21:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.649+0000 7f2893400640 1 -- 172.21.15.19:0/577444001 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f287800c070 con 0x7f2878004800 2024-03-28T13:21:17.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.649+0000 7f2893400640 1 -- 172.21.15.19:0/577444001 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f287800c270 con 0x7f28780051c0 2024-03-28T13:21:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.649+0000 7f2893e00640 1 --2- 172.21.15.19:0/577444001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28780039b0 0x7f2878003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.650+0000 7f288b400640 1 --2- 172.21.15.19:0/577444001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2878004800 0x7f2878004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.650+0000 7f2893e00640 1 -- 172.21.15.19:0/577444001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28780051c0 msgr2=0x7f287800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.650+0000 7f2893e00640 1 --2- 172.21.15.19:0/577444001 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28780051c0 0x7f287800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.650+0000 7f2893e00640 1 -- 172.21.15.19:0/577444001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2878004800 msgr2=0x7f2878004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.650+0000 7f2893e00640 1 --2- 172.21.15.19:0/577444001 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2878004800 0x7f2878004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.650+0000 7f2893e00640 1 -- 172.21.15.19:0/577444001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2878003520 con 0x7f28780039b0 2024-03-28T13:21:17.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.650+0000 7f2898a00640 1 -- 172.21.15.19:0/577444001 reap_dead start 2024-03-28T13:21:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.651+0000 7f2893e00640 1 --2- 172.21.15.19:0/577444001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28780039b0 0x7f2878003df0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f2884009ac0 tx=0x7f2878005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861bf6fe137bb487 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.651+0000 7f2893400640 1 -- 172.21.15.19:0/577444001 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884039070 con 0x7f28780039b0 2024-03-28T13:21:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.651+0000 7f2893400640 1 -- 172.21.15.19:0/577444001 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f288402c6f0 con 0x7f28780039b0 2024-03-28T13:21:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.651+0000 7f2893400640 1 -- 172.21.15.19:0/577444001 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884009690 con 0x7f28780039b0 2024-03-28T13:21:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.652+0000 7f2899400640 1 -- 172.21.15.19:0/577444001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28780039b0 msgr2=0x7f2878003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.652+0000 7f2899400640 1 --2- 172.21.15.19:0/577444001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28780039b0 0x7f2878003df0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f2884009ac0 tx=0x7f2878005830 comp rx=0 tx=0).stop 2024-03-28T13:21:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.652+0000 7f2899400640 1 -- 172.21.15.19:0/577444001 shutdown_connections 2024-03-28T13:21:17.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.652+0000 7f2899400640 1 --2- 172.21.15.19:0/577444001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f28780039b0 0x7f2878003df0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.652+0000 7f2899400640 1 -- 172.21.15.19:0/577444001 >> 172.21.15.19:0/577444001 conn(0x7f28940ccad0 msgr2=0x7f28940cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.652+0000 7f2899400640 1 -- 172.21.15.19:0/577444001 shutdown_connections 2024-03-28T13:21:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.652+0000 7f2899400640 1 -- 172.21.15.19:0/577444001 wait complete. 2024-03-28T13:21:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.654+0000 7f2899400640 1 Processor -- start 2024-03-28T13:21:17.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.654+0000 7f2899400640 1 -- start start 2024-03-28T13:21:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.655+0000 7f2899400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28780039b0 0x7f289415a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.655+0000 7f2899400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f289415a650 0x7f289413e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.655+0000 7f2899400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f289413e920 0x7f289415d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.655+0000 7f2899400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f28940d0770 con 0x7f289415a650 2024-03-28T13:21:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.655+0000 7f2899400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f28940cf420 con 0x7f28780039b0 2024-03-28T13:21:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.655+0000 7f2899400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f28941446a0 con 0x7f289413e920 2024-03-28T13:21:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.655+0000 7f288b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f289415a650 0x7f289413e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:17.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.655+0000 7f288b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f289415a650 0x7f289413e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52674/0 (socket says 172.21.15.19:52674) 2024-03-28T13:21:17.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.655+0000 7f288b400640 1 -- 172.21.15.19:0/1389781578 learned_addr learned my addr 172.21.15.19:0/1389781578 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:17.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.655+0000 7f2893e00640 1 --2- 172.21.15.19:0/1389781578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28780039b0 0x7f289415a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:17.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.656+0000 7f2898a00640 1 --2- 172.21.15.19:0/1389781578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f289413e920 0x7f289415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:17.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.656+0000 7f288b400640 1 -- 172.21.15.19:0/1389781578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f289413e920 msgr2=0x7f289415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:17.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.656+0000 7f288b400640 1 --2- 172.21.15.19:0/1389781578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f289413e920 0x7f289415d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:17.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.656+0000 7f288b400640 1 -- 172.21.15.19:0/1389781578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28780039b0 msgr2=0x7f289415a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:17.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.656+0000 7f288b400640 1 --2- 172.21.15.19:0/1389781578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28780039b0 0x7f289415a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:17.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.656+0000 7f288b400640 1 -- 172.21.15.19:0/1389781578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289415b720 con 0x7f289415a650 2024-03-28T13:21:17.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.656+0000 7f288b400640 1 --2- 172.21.15.19:0/1389781578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f289415a650 0x7f289413e3e0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f287c00d8b0 tx=0x7f287c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:17.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.656+0000 7f2890c00640 1 -- 172.21.15.19:0/1389781578 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c019070 con 0x7f289415a650 2024-03-28T13:21:17.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.656+0000 7f2890c00640 1 -- 172.21.15.19:0/1389781578 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f287c0074e0 con 0x7f289415a650 2024-03-28T13:21:17.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.656+0000 7f2890c00640 1 -- 172.21.15.19:0/1389781578 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c0049e0 con 0x7f289415a650 2024-03-28T13:21:17.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.656+0000 7f2899400640 1 -- 172.21.15.19:0/1389781578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289415ba00 con 0x7f289415a650 2024-03-28T13:21:17.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.657+0000 7f2899400640 1 -- 172.21.15.19:0/1389781578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289415bfc0 con 0x7f289415a650 2024-03-28T13:21:17.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.658+0000 7f2899400640 1 -- 172.21.15.19:0/1389781578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2894075940 con 0x7f289415a650 2024-03-28T13:21:17.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.659+0000 7f2890c00640 1 -- 172.21.15.19:0/1389781578 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f287c004b80 con 0x7f289415a650 2024-03-28T13:21:17.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.660+0000 7f2890c00640 1 --2- 172.21.15.19:0/1389781578 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f287006c530 0x7f287006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:17.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.660+0000 7f2890c00640 1 -- 172.21.15.19:0/1389781578 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7f287c08b1f0 con 0x7f289415a650 2024-03-28T13:21:17.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.660+0000 7f2893e00640 1 --2- 172.21.15.19:0/1389781578 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f287006c530 0x7f287006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:17.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.661+0000 7f2893e00640 1 --2- 172.21.15.19:0/1389781578 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f287006c530 0x7f287006e9f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2884036000 tx=0x7f2884030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:17.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.664+0000 7f2890c00640 1 -- 172.21.15.19:0/1389781578 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f287c0597a0 con 0x7f289415a650 2024-03-28T13:21:17.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.975+0000 7f2899400640 1 -- 172.21.15.19:0/1389781578 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f289406c580 con 0x7f289415a650 2024-03-28T13:21:17.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.976+0000 7f2890c00640 1 -- 172.21.15.19:0/1389781578 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5187 (secure 0 0 0) 0x7f287c0590a0 con 0x7f289415a650 2024-03-28T13:21:17.976 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:17.976 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3823671990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3823671990},{"type":"v1","addr":"172.21.15.188:6837","nonce":3823671990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/22158043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":22158043},{"type":"v1","addr":"172.21.15.188:6836","nonce":22158043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":8688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/230519675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":230519675},{"type":"v1","addr":"172.21.15.19:6834","nonce":230519675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:20:52.901269+0000","modified":"2024-03-28T13:21:16.616377+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8691},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8691":{"gid":8691,"name":"a","rank":0,"incarnation":102,"state":"up:active","state_seq":10,"addr":"172.21.15.19:6835/3930967142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3930967142},{"type":"v1","addr":"172.21.15.19:6835","nonce":3930967142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8691,"qdb_cluster":[8691]},"id":8}]} 2024-03-28T13:21:17.976 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 105 2024-03-28T13:21:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.979+0000 7f2899400640 1 -- 172.21.15.19:0/1389781578 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f287006c530 msgr2=0x7f287006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.979+0000 7f2899400640 1 --2- 172.21.15.19:0/1389781578 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f287006c530 0x7f287006e9f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2884036000 tx=0x7f2884030000 comp rx=0 tx=0).stop 2024-03-28T13:21:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.979+0000 7f2899400640 1 -- 172.21.15.19:0/1389781578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f289415a650 msgr2=0x7f289413e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.979+0000 7f2899400640 1 --2- 172.21.15.19:0/1389781578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f289415a650 0x7f289413e3e0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f287c00d8b0 tx=0x7f287c00dd70 comp rx=0 tx=0).stop 2024-03-28T13:21:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.979+0000 7f2899400640 1 -- 172.21.15.19:0/1389781578 shutdown_connections 2024-03-28T13:21:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.979+0000 7f2899400640 1 --2- 172.21.15.19:0/1389781578 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f287006c530 0x7f287006e9f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.979+0000 7f2899400640 1 --2- 172.21.15.19:0/1389781578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f289413e920 0x7f289415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.979+0000 7f2899400640 1 --2- 172.21.15.19:0/1389781578 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f289415a650 0x7f289413e3e0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.979+0000 7f2899400640 1 --2- 172.21.15.19:0/1389781578 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28780039b0 0x7f289415a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.979+0000 7f2899400640 1 -- 172.21.15.19:0/1389781578 >> 172.21.15.19:0/1389781578 conn(0x7f28940ccad0 msgr2=0x7f289406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.979+0000 7f2899400640 1 -- 172.21.15.19:0/1389781578 shutdown_connections 2024-03-28T13:21:17.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:17.979+0000 7f2899400640 1 -- 172.21.15.19:0/1389781578 wait complete. 2024-03-28T13:21:18.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:20:52.901269+0000', 'modified': '2024-03-28T13:21:16.616377+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 120, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8691}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8691': {'gid': 8691, 'name': 'a', 'rank': 0, 'incarnation': 102, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.19:6835/3930967142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 3930967142}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 3930967142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8691, 'qdb_cluster': [8691]} 2024-03-28T13:21:18.408 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T13:21:18.408 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-28T13:21:18.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.636+0000 7f7429200640 1 Processor -- start 2024-03-28T13:21:18.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.637+0000 7f7429200640 1 -- start start 2024-03-28T13:21:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.638+0000 7f7429200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f74240d0990 0x7f74240aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.638+0000 7f7429200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74240afab0 con 0x7f74240af2e0 2024-03-28T13:21:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.638+0000 7f7429200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f742415cc80 con 0x7f7424153580 2024-03-28T13:21:18.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.638+0000 7f7429200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f742415da00 con 0x7f74240d0990 2024-03-28T13:21:18.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.639+0000 7f7422800640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f74240af2e0 0x7f74241600d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51730/0 (socket says 172.21.15.19:51730) 2024-03-28T13:21:18.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.639+0000 7f7422800640 1 -- 172.21.15.19:0/2725076460 learned_addr learned my addr 172.21.15.19:0/2725076460 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:18.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.639+0000 7f7419400640 1 --2- 172.21.15.19:0/2725076460 >> v2:172.21.15.188:3300/0 conn(0x7f74240d0990 0x7f74240aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.640+0000 7f7421400640 1 -- 172.21.15.19:0/2725076460 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2181599455 0 0) 0x7f74240afab0 con 0x7f74240af2e0 2024-03-28T13:21:18.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.640+0000 7f7419400640 1 -- 172.21.15.19:0/2725076460 >> v1:172.21.15.188:6789/0 conn(0x7f7424153580 legacy=0x7f74240d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:18.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.640+0000 7f7419400640 1 -- 172.21.15.19:0/2725076460 >> v1:172.21.15.19:6789/0 conn(0x7f74240af2e0 legacy=0x7f74241600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:18.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.640+0000 7f7419400640 1 -- 172.21.15.19:0/2725076460 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742415cf20 con 0x7f74240d0990 2024-03-28T13:21:18.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.641+0000 7f7419400640 1 --2- 172.21.15.19:0/2725076460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74240d0990 0x7f74240aeda0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f740c0097a0 tx=0x7f740c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c975a5fbeff54dfb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:18.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.642+0000 7f7421400640 1 -- 172.21.15.19:0/2725076460 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c035020 con 0x7f74240d0990 2024-03-28T13:21:18.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.642+0000 7f7421400640 1 -- 172.21.15.19:0/2725076460 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f740c031830 con 0x7f74240d0990 2024-03-28T13:21:18.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.642+0000 7f7421400640 1 -- 172.21.15.19:0/2725076460 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c031c00 con 0x7f74240d0990 2024-03-28T13:21:18.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.642+0000 7f7429200640 1 -- 172.21.15.19:0/2725076460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74240d0990 msgr2=0x7f74240aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:18.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.642+0000 7f7429200640 1 --2- 172.21.15.19:0/2725076460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74240d0990 0x7f74240aeda0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f740c0097a0 tx=0x7f740c02b8d0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.642+0000 7f7429200640 1 -- 172.21.15.19:0/2725076460 shutdown_connections 2024-03-28T13:21:18.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.643+0000 7f7429200640 1 --2- 172.21.15.19:0/2725076460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74240d0990 0x7f74240aeda0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.643+0000 7f7429200640 1 -- 172.21.15.19:0/2725076460 >> 172.21.15.19:0/2725076460 conn(0x7f742406b750 msgr2=0x7f742406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:18.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.643+0000 7f7429200640 1 -- 172.21.15.19:0/2725076460 shutdown_connections 2024-03-28T13:21:18.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.643+0000 7f7429200640 1 -- 172.21.15.19:0/2725076460 wait complete. 2024-03-28T13:21:18.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.645+0000 7f7429200640 1 Processor -- start 2024-03-28T13:21:18.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.645+0000 7f7429200640 1 -- start start 2024-03-28T13:21:18.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.646+0000 7f7429200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74240af2e0 0x7f74240cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.646+0000 7f7429200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74240d0990 0x7f742415ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.646+0000 7f7429200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7424153580 0x7f742415f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.646+0000 7f7429200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7424153960 con 0x7f74240d0990 2024-03-28T13:21:18.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.646+0000 7f7429200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f742415d4e0 con 0x7f74240af2e0 2024-03-28T13:21:18.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.646+0000 7f7429200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f74240af6c0 con 0x7f7424153580 2024-03-28T13:21:18.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.646+0000 7f7419400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74240d0990 0x7f742415ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.646+0000 7f7419400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74240d0990 0x7f742415ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52682/0 (socket says 172.21.15.19:52682) 2024-03-28T13:21:18.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.646+0000 7f7419400640 1 -- 172.21.15.19:0/2642093687 learned_addr learned my addr 172.21.15.19:0/2642093687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:18.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.646+0000 7f7421e00640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74240af2e0 0x7f74240cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.646+0000 7f7422800640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7424153580 0x7f742415f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.647+0000 7f7419400640 1 -- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7424153580 msgr2=0x7f742415f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:18.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.647+0000 7f7419400640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7424153580 0x7f742415f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.647+0000 7f7419400640 1 -- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74240af2e0 msgr2=0x7f74240cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.647+0000 7f7419400640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74240af2e0 0x7f74240cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.647+0000 7f7419400640 1 -- 172.21.15.19:0/2642093687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74241f55b0 con 0x7f74240d0990 2024-03-28T13:21:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.647+0000 7f7422800640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7424153580 0x7f742415f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.647+0000 7f7421e00640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74240af2e0 0x7f74240cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.647+0000 7f7419400640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74240d0990 0x7f742415ec60 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f740c0097a0 tx=0x7f740c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.647+0000 7f741aa00640 1 -- 172.21.15.19:0/2642093687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c035050 con 0x7f74240d0990 2024-03-28T13:21:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.647+0000 7f741aa00640 1 -- 172.21.15.19:0/2642093687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f740c004230 con 0x7f74240d0990 2024-03-28T13:21:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.647+0000 7f741aa00640 1 -- 172.21.15.19:0/2642093687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c03b740 con 0x7f74240d0990 2024-03-28T13:21:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.647+0000 7f7429200640 1 -- 172.21.15.19:0/2642093687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74241f5890 con 0x7f74240d0990 2024-03-28T13:21:18.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.647+0000 7f7429200640 1 -- 172.21.15.19:0/2642093687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74241f5e50 con 0x7f74240d0990 2024-03-28T13:21:18.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.650+0000 7f741aa00640 1 -- 172.21.15.19:0/2642093687 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f740c038020 con 0x7f74240d0990 2024-03-28T13:21:18.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.651+0000 7f741aa00640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f740006c530 0x7f740006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.651+0000 7f741aa00640 1 -- 172.21.15.19:0/2642093687 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7f740c0ad600 con 0x7f74240d0990 2024-03-28T13:21:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.652+0000 7f742bf63740 1 Processor -- start 2024-03-28T13:21:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.652+0000 7f7421e00640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f740006c530 0x7f740006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.652+0000 7f742bf63740 1 -- start start 2024-03-28T13:21:18.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.653+0000 7f742bf63740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 0x55b043bb4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.653+0000 7f742bf63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043b94370 0x55b043ba82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.653+0000 7f742bf63740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043ba87f0 0x55b043b96d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.653+0000 7f742bf63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b043b2e920 con 0x55b043b94370 2024-03-28T13:21:18.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.653+0000 7f742bf63740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b043a7efa0 con 0x55b043bb44e0 2024-03-28T13:21:18.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.653+0000 7f742bf63740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55b043ad0f90 con 0x55b043ba87f0 2024-03-28T13:21:18.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.653+0000 7f7419400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043b94370 0x55b043ba82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.653+0000 7f7419400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043b94370 0x55b043ba82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52692/0 (socket says 172.21.15.19:52692) 2024-03-28T13:21:18.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.653+0000 7f7419400640 1 -- 172.21.15.19:0/1879879234 learned_addr learned my addr 172.21.15.19:0/1879879234 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:18.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.653+0000 7f7421e00640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f740006c530 0x7f740006e9f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f74140012e0 tx=0x7f7414009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:18.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.654+0000 7f7422800640 1 --2- 172.21.15.19:0/1879879234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043ba87f0 0x55b043b96d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.654+0000 7f7419400640 1 -- 172.21.15.19:0/1879879234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043ba87f0 msgr2=0x55b043b96d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:18.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.654+0000 7f7419400640 1 --2- 172.21.15.19:0/1879879234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043ba87f0 0x55b043b96d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.654+0000 7f7419400640 1 -- 172.21.15.19:0/1879879234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 msgr2=0x55b043bb4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:18.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.654+0000 7f7419400640 1 --2- 172.21.15.19:0/1879879234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 0x55b043bb4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.654+0000 7f7419400640 1 -- 172.21.15.19:0/1879879234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0437e52d0 con 0x55b043b94370 2024-03-28T13:21:18.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.654+0000 7f7422800640 1 --2- 172.21.15.19:0/1879879234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043ba87f0 0x55b043b96d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:18.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.654+0000 7f7419400640 1 --2- 172.21.15.19:0/1879879234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043b94370 0x55b043ba82b0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f740c077880 tx=0x7f740c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c7c4b0dd65c20e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:18.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.654+0000 7f7412a00640 1 -- 172.21.15.19:0/1879879234 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c0ab070 con 0x55b043b94370 2024-03-28T13:21:18.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.654+0000 7f7412a00640 1 -- 172.21.15.19:0/1879879234 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f740c07f7d0 con 0x55b043b94370 2024-03-28T13:21:18.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.654+0000 7f7412a00640 1 -- 172.21.15.19:0/1879879234 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c084000 con 0x55b043b94370 2024-03-28T13:21:18.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.655+0000 7f742bf63740 1 -- 172.21.15.19:0/1879879234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043b94370 msgr2=0x55b043ba82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:18.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.655+0000 7f742bf63740 1 --2- 172.21.15.19:0/1879879234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043b94370 0x55b043ba82b0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f740c077880 tx=0x7f740c009190 comp rx=0 tx=0).stop 2024-03-28T13:21:18.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.655+0000 7f742bf63740 1 -- 172.21.15.19:0/1879879234 shutdown_connections 2024-03-28T13:21:18.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.655+0000 7f742bf63740 1 --2- 172.21.15.19:0/1879879234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043ba87f0 0x55b043b96d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.655+0000 7f742bf63740 1 --2- 172.21.15.19:0/1879879234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043b94370 0x55b043ba82b0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.655+0000 7f742bf63740 1 --2- 172.21.15.19:0/1879879234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 0x55b043bb4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.655+0000 7f742bf63740 1 -- 172.21.15.19:0/1879879234 >> 172.21.15.19:0/1879879234 conn(0x55b043b691e0 msgr2=0x55b043bb1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:18.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.655+0000 7f742bf63740 1 -- 172.21.15.19:0/1879879234 shutdown_connections 2024-03-28T13:21:18.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.655+0000 7f742bf63740 1 -- 172.21.15.19:0/1879879234 wait complete. 2024-03-28T13:21:18.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.656+0000 7f742bf63740 1 -- start start 2024-03-28T13:21:18.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.656+0000 7f742bf63740 1 Processor -- start 2024-03-28T13:21:18.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.657+0000 7f742bf63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043ba87f0 0x55b043b902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.657+0000 7f742bf63740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 0x55b043b99510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.657+0000 7f742bf63740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043b9a750 0x55b043b99a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.657+0000 7f742bf63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b043accf40 con 0x55b043ba87f0 2024-03-28T13:21:18.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.657+0000 7f742bf63740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b043a79650 con 0x55b043bb44e0 2024-03-28T13:21:18.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.657+0000 7f742bf63740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55b043a8ca10 con 0x55b043b9a750 2024-03-28T13:21:18.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.657+0000 7f742bf63740 10 client.0 _ll_register_callbacks cb 0x55b043acbff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:21:18.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.657+0000 7f7421e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043ba87f0 0x55b043b902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.657+0000 7f7421e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043ba87f0 0x55b043b902c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52708/0 (socket says 172.21.15.19:52708) 2024-03-28T13:21:18.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.657+0000 7f7421e00640 1 -- 172.21.15.19:0/3684754997 learned_addr learned my addr 172.21.15.19:0/3684754997 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:18.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.658+0000 7f7421e00640 1 -- 172.21.15.19:0/3684754997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043b9a750 msgr2=0x55b043b99a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:18.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.658+0000 7f7421e00640 1 --2- 172.21.15.19:0/3684754997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043b9a750 0x55b043b99a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.658+0000 7f7421e00640 1 -- 172.21.15.19:0/3684754997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 msgr2=0x55b043b99510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:18.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.658+0000 7f7421e00640 1 --2- 172.21.15.19:0/3684754997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 0x55b043b99510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.658+0000 7f7421e00640 1 -- 172.21.15.19:0/3684754997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0438af6b0 con 0x55b043ba87f0 2024-03-28T13:21:18.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.658+0000 7f7421e00640 1 --2- 172.21.15.19:0/3684754997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043ba87f0 0x55b043b902c0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f7414014450 tx=0x7f7414014850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10491ed21feb834 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:18.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.658+0000 7f7407400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:21:18.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.658+0000 7f7407400640 1 -- 172.21.15.19:0/3684754997 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414024060 con 0x55b043ba87f0 2024-03-28T13:21:18.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.658+0000 7f7407400640 1 -- 172.21.15.19:0/3684754997 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7f74140127e0 con 0x55b043ba87f0 2024-03-28T13:21:18.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.658+0000 7f742bf63740 1 -- 172.21.15.19:0/3684754997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f740c07f7d0 con 0x55b043ba87f0 2024-03-28T13:21:18.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.659+0000 7f7407400640 1 -- 172.21.15.19:0/3684754997 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f74140129e0 con 0x55b043ba87f0 2024-03-28T13:21:18.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.659+0000 7f742bf63740 10 client.9000 fetch_fsmap learned FSMap version 105 2024-03-28T13:21:18.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.659+0000 7f742bf63740 1 -- 172.21.15.19:0/3684754997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x55b043bce160 con 0x55b043ba87f0 2024-03-28T13:21:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.659+0000 7f7407400640 1 -- 172.21.15.19:0/3684754997 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f741402a070 con 0x55b043ba87f0 2024-03-28T13:21:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.659+0000 7f7407400640 1 -- 172.21.15.19:0/3684754997 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 105) ==== 3494+0+0 (secure 0 0 0) 0x7f7414012b80 con 0x55b043ba87f0 2024-03-28T13:21:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.659+0000 7f742bf63740 10 client.9000 fetch_fsmap finished waiting for FSMap version 105 2024-03-28T13:21:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.659+0000 7f742bf63740 10 client.9000 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:21:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.659+0000 7f742bf63740 20 client.9000 populate_metadata read hostname 'smithi019' 2024-03-28T13:21:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.659+0000 7f742bf63740 1 --2- 172.21.15.19:0/3684754997 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x55b043bcace0 0x55b043bd8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.659+0000 7f742bf63740 4 client.9000 mds_command: new command op to 8691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:21:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.659+0000 7f742bf63740 1 -- 172.21.15.19:0/3684754997 --> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b043ade4d0 con 0x55b043bcace0 2024-03-28T13:21:18.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.659+0000 7f7419400640 1 --2- 172.21.15.19:0/3684754997 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x55b043bcace0 0x55b043bd8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.660+0000 7f7419400640 1 --2- 172.21.15.19:0/3684754997 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x55b043bcace0 0x55b043bd8da0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ced39826103678 server_cookie=f60fbeed2f4487b0 in_seq=0 out_seq=0 2024-03-28T13:21:18.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.660+0000 7f7407400640 10 client.9000 ms_handle_connect on v2:172.21.15.19:6833/3930967142 2024-03-28T13:21:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.661+0000 7f7407400640 1 -- 172.21.15.19:0/3684754997 <== mds.0 v2:172.21.15.19:6833/3930967142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b043ade4d0 con 0x55b043bcace0 2024-03-28T13:21:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.661+0000 7f7407400640 10 client.9000 handle_command_reply: tid=0 2024-03-28T13:21:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.661+0000 7f742bf63740 1 client.9000 shutdown 2024-03-28T13:21:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.661+0000 7f742bf63740 1 -- 172.21.15.19:0/3684754997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043ba87f0 msgr2=0x55b043b902c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.661+0000 7f742bf63740 1 --2- 172.21.15.19:0/3684754997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043ba87f0 0x55b043b902c0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f7414014450 tx=0x7f7414014850 comp rx=0 tx=0).stop 2024-03-28T13:21:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.661+0000 7f742bf63740 1 -- 172.21.15.19:0/3684754997 shutdown_connections 2024-03-28T13:21:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.661+0000 7f742bf63740 1 --2- 172.21.15.19:0/3684754997 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x55b043bcace0 0x55b043bd8da0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.661+0000 7f742bf63740 1 --2- 172.21.15.19:0/3684754997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043b9a750 0x55b043b99a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.661+0000 7f742bf63740 1 --2- 172.21.15.19:0/3684754997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 0x55b043b99510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.661+0000 7f742bf63740 1 --2- 172.21.15.19:0/3684754997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043ba87f0 0x55b043b902c0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.661+0000 7f7407400640 0 client.9000 ms_handle_reset on v2:172.21.15.19:6833/3930967142 2024-03-28T13:21:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.661+0000 7f742bf63740 1 -- 172.21.15.19:0/3684754997 >> 172.21.15.19:0/3684754997 conn(0x55b043b94370 msgr2=0x55b043b6c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.661+0000 7f742bf63740 1 -- 172.21.15.19:0/3684754997 shutdown_connections 2024-03-28T13:21:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.661+0000 7f742bf63740 1 -- 172.21.15.19:0/3684754997 wait complete. 2024-03-28T13:21:18.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.662+0000 7f742bf63740 20 client.9000 trim_cache size 0 max 16384 2024-03-28T13:21:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.682+0000 7f742bf63740 1 Processor -- start 2024-03-28T13:21:18.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.682+0000 7f742bf63740 1 -- start start 2024-03-28T13:21:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.683+0000 7f742bf63740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043b9a750 0x55b043bb2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.683+0000 7f742bf63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043ba87f0 0x55b043b9e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.683+0000 7f742bf63740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 0x55b043b9edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.683+0000 7f742bf63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7424072b30 con 0x55b043ba87f0 2024-03-28T13:21:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.683+0000 7f742bf63740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f742414bd50 con 0x55b043bb44e0 2024-03-28T13:21:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.683+0000 7f742bf63740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f74241f2ca0 con 0x55b043b9a750 2024-03-28T13:21:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.683+0000 7f7419400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043ba87f0 0x55b043b9e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.683+0000 7f7419400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043ba87f0 0x55b043b9e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52712/0 (socket says 172.21.15.19:52712) 2024-03-28T13:21:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.683+0000 7f7419400640 1 -- 172.21.15.19:0/872490714 learned_addr learned my addr 172.21.15.19:0/872490714 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.683+0000 7f7419400640 1 -- 172.21.15.19:0/872490714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043b9a750 msgr2=0x55b043bb2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.683+0000 7f7419400640 1 --2- 172.21.15.19:0/872490714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043b9a750 0x55b043bb2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.683+0000 7f7419400640 1 -- 172.21.15.19:0/872490714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 msgr2=0x55b043b9edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:18.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.683+0000 7f7419400640 1 --2- 172.21.15.19:0/872490714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 0x55b043b9edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.683+0000 7f7419400640 1 -- 172.21.15.19:0/872490714 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b043b9b690 con 0x55b043ba87f0 2024-03-28T13:21:18.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.684+0000 7f7419400640 1 --2- 172.21.15.19:0/872490714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043ba87f0 0x55b043b9e8b0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f740c077880 tx=0x7f740c07d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f590ea42249def server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:18.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.684+0000 7f7412000640 1 -- 172.21.15.19:0/872490714 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c098070 con 0x55b043ba87f0 2024-03-28T13:21:18.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.684+0000 7f7412000640 1 -- 172.21.15.19:0/872490714 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f740c079720 con 0x55b043ba87f0 2024-03-28T13:21:18.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.684+0000 7f7412000640 1 -- 172.21.15.19:0/872490714 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c0ab040 con 0x55b043ba87f0 2024-03-28T13:21:18.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.684+0000 7f742bf63740 1 -- 172.21.15.19:0/872490714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043ba87f0 msgr2=0x55b043b9e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:18.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.684+0000 7f742bf63740 1 --2- 172.21.15.19:0/872490714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043ba87f0 0x55b043b9e8b0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f740c077880 tx=0x7f740c07d000 comp rx=0 tx=0).stop 2024-03-28T13:21:18.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.685+0000 7f742bf63740 1 -- 172.21.15.19:0/872490714 shutdown_connections 2024-03-28T13:21:18.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.685+0000 7f742bf63740 1 --2- 172.21.15.19:0/872490714 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 0x55b043b9edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.685+0000 7f742bf63740 1 --2- 172.21.15.19:0/872490714 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043ba87f0 0x55b043b9e8b0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.685+0000 7f742bf63740 1 --2- 172.21.15.19:0/872490714 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043b9a750 0x55b043bb2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.685+0000 7f742bf63740 1 -- 172.21.15.19:0/872490714 >> 172.21.15.19:0/872490714 conn(0x55b043b94370 msgr2=0x55b043b9a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:18.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.685+0000 7f742bf63740 1 -- 172.21.15.19:0/872490714 shutdown_connections 2024-03-28T13:21:18.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.685+0000 7f742bf63740 1 -- 172.21.15.19:0/872490714 wait complete. 2024-03-28T13:21:18.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.685+0000 7f742bf63740 1 -- start start 2024-03-28T13:21:18.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.686+0000 7f742bf63740 1 Processor -- start 2024-03-28T13:21:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f742bf63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043b9a750 0x55b043b9e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f742bf63740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043ba87f0 0x55b043b9edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f742bf63740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 0x55b043b8e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f742bf63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b043bce7a0 con 0x55b043b9a750 2024-03-28T13:21:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f742bf63740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55b043bce4a0 con 0x55b043bb44e0 2024-03-28T13:21:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f742bf63740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55b043b91a40 con 0x55b043ba87f0 2024-03-28T13:21:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f742bf63740 10 client.0 _ll_register_callbacks cb 0x7f7414001a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:21:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f7421e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043b9a750 0x55b043b9e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f7421e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043b9a750 0x55b043b9e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52716/0 (socket says 172.21.15.19:52716) 2024-03-28T13:21:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f7421e00640 1 -- 172.21.15.19:0/3334900252 learned_addr learned my addr 172.21.15.19:0/3334900252 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f7419400640 1 --2- 172.21.15.19:0/3334900252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043ba87f0 0x55b043b9edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f7422800640 1 --2- 172.21.15.19:0/3334900252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 0x55b043b8e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f7421e00640 1 -- 172.21.15.19:0/3334900252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043ba87f0 msgr2=0x55b043b9edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f7421e00640 1 --2- 172.21.15.19:0/3334900252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043ba87f0 0x55b043b9edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f7421e00640 1 -- 172.21.15.19:0/3334900252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 msgr2=0x55b043b8e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f7421e00640 1 --2- 172.21.15.19:0/3334900252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 0x55b043b8e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f7421e00640 1 -- 172.21.15.19:0/3334900252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b043bcfb30 con 0x55b043b9a750 2024-03-28T13:21:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.687+0000 7f7422800640 1 --2- 172.21.15.19:0/3334900252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 0x55b043b8e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.688+0000 7f7421e00640 1 --2- 172.21.15.19:0/3334900252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043b9a750 0x55b043b9e8b0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f741400d520 tx=0x7f7414012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a04e694878056d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.688+0000 7f7407400640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:21:18.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.688+0000 7f7407400640 1 -- 172.21.15.19:0/3334900252 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7414033060 con 0x55b043b9a750 2024-03-28T13:21:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.688+0000 7f7407400640 1 -- 172.21.15.19:0/3334900252 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7f74140127e0 con 0x55b043b9a750 2024-03-28T13:21:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.688+0000 7f742bf63740 1 -- 172.21.15.19:0/3334900252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f740c079720 con 0x55b043b9a750 2024-03-28T13:21:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.688+0000 7f7407400640 1 -- 172.21.15.19:0/3334900252 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7f7414024070 con 0x55b043b9a750 2024-03-28T13:21:18.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.689+0000 7f742bf63740 10 client.9006 fetch_fsmap learned FSMap version 105 2024-03-28T13:21:18.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.689+0000 7f742bf63740 1 -- 172.21.15.19:0/3334900252 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x55b043c35080 con 0x55b043b9a750 2024-03-28T13:21:18.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.689+0000 7f7407400640 1 -- 172.21.15.19:0/3334900252 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f740c079720 con 0x55b043b9a750 2024-03-28T13:21:18.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.689+0000 7f7407400640 1 -- 172.21.15.19:0/3334900252 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 105) ==== 3494+0+0 (secure 0 0 0) 0x7f7414006b50 con 0x55b043b9a750 2024-03-28T13:21:18.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.689+0000 7f742bf63740 10 client.9006 fetch_fsmap finished waiting for FSMap version 105 2024-03-28T13:21:18.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.689+0000 7f742bf63740 10 client.9006 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:21:18.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.689+0000 7f742bf63740 20 client.9006 populate_metadata read hostname 'smithi019' 2024-03-28T13:21:18.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.689+0000 7f742bf63740 1 --2- 172.21.15.19:0/3334900252 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x55b043bceab0 0x55b043bcee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.689+0000 7f742bf63740 4 client.9006 mds_command: new command op to 8691 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T13:21:18.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.689+0000 7f742bf63740 1 -- 172.21.15.19:0/3334900252 --> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] -- command(tid 0: {"prefix": "status"}) -- 0x55b043b67d20 con 0x55b043bceab0 2024-03-28T13:21:18.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.689+0000 7f7419400640 1 --2- 172.21.15.19:0/3334900252 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x55b043bceab0 0x55b043bcee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.690+0000 7f7419400640 1 --2- 172.21.15.19:0/3334900252 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x55b043bceab0 0x55b043bcee90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ba42c40601b2055 server_cookie=814605f7b34db030 in_seq=0 out_seq=0 2024-03-28T13:21:18.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.690+0000 7f7407400640 10 client.9006 ms_handle_connect on v2:172.21.15.19:6833/3930967142 2024-03-28T13:21:18.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.690+0000 7f7407400640 1 -- 172.21.15.19:0/3334900252 <== mds.0 v2:172.21.15.19:6833/3930967142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55b043b67d20 con 0x55b043bceab0 2024-03-28T13:21:18.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.690+0000 7f7407400640 10 client.9006 handle_command_reply: tid=0 2024-03-28T13:21:18.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.690+0000 7f742bf63740 1 client.9006 shutdown 2024-03-28T13:21:18.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.690+0000 7f742bf63740 1 -- 172.21.15.19:0/3334900252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043b9a750 msgr2=0x55b043b9e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:18.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.690+0000 7f742bf63740 1 --2- 172.21.15.19:0/3334900252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043b9a750 0x55b043b9e8b0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f741400d520 tx=0x7f7414012000 comp rx=0 tx=0).stop 2024-03-28T13:21:18.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.690+0000 7f742bf63740 1 -- 172.21.15.19:0/3334900252 shutdown_connections 2024-03-28T13:21:18.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.690+0000 7f742bf63740 1 --2- 172.21.15.19:0/3334900252 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x55b043bceab0 0x55b043bcee90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.690+0000 7f742bf63740 1 --2- 172.21.15.19:0/3334900252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b043bb44e0 0x55b043b8e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.690+0000 7f742bf63740 1 --2- 172.21.15.19:0/3334900252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b043ba87f0 0x55b043b9edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.690+0000 7f742bf63740 1 --2- 172.21.15.19:0/3334900252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b043b9a750 0x55b043b9e8b0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.690+0000 7f742bf63740 1 -- 172.21.15.19:0/3334900252 >> 172.21.15.19:0/3334900252 conn(0x55b043b94370 msgr2=0x55b043b9abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:18.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.690+0000 7f7407400640 0 client.9006 ms_handle_reset on v2:172.21.15.19:6833/3930967142 2024-03-28T13:21:18.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.691+0000 7f742bf63740 1 -- 172.21.15.19:0/3334900252 shutdown_connections 2024-03-28T13:21:18.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.691+0000 7f742bf63740 1 -- 172.21.15.19:0/3334900252 wait complete. 2024-03-28T13:21:18.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.691+0000 7f742bf63740 20 client.9006 trim_cache size 0 max 16384 2024-03-28T13:21:18.696 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:21:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "e6b569d9-f2ce-49dc-bbe7-c0ab33080203", 2024-03-28T13:21:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-03-28T13:21:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 8691, 2024-03-28T13:21:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-03-28T13:21:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-03-28T13:21:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-03-28T13:21:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 5.1016206579999999, 2024-03-28T13:21:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 105, 2024-03-28T13:21:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 120, 2024-03-28T13:21:18.696 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 120, 2024-03-28T13:21:18.697 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 30.624779948 2024-03-28T13:21:18.697 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:21:18.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.692+0000 7f7429200640 1 -- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f740006c530 msgr2=0x7f740006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:18.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.692+0000 7f7429200640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f740006c530 0x7f740006e9f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f74140012e0 tx=0x7f7414009040 comp rx=0 tx=0).stop 2024-03-28T13:21:18.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.692+0000 7f7429200640 1 -- 172.21.15.19:0/2642093687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74240d0990 msgr2=0x7f742415ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:18.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.692+0000 7f7429200640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74240d0990 0x7f742415ec60 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f740c0097a0 tx=0x7f740c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.692+0000 7f7429200640 1 -- 172.21.15.19:0/2642093687 shutdown_connections 2024-03-28T13:21:18.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.692+0000 7f7429200640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f740006c530 0x7f740006e9f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.692+0000 7f7429200640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7424153580 0x7f742415f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.692+0000 7f7429200640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f74240d0990 0x7f742415ec60 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.692+0000 7f7429200640 1 --2- 172.21.15.19:0/2642093687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74240af2e0 0x7f74240cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.692+0000 7f7429200640 1 -- 172.21.15.19:0/2642093687 >> 172.21.15.19:0/2642093687 conn(0x7f742406b750 msgr2=0x7f742406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:18.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.692+0000 7f7429200640 1 -- 172.21.15.19:0/2642093687 shutdown_connections 2024-03-28T13:21:18.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.692+0000 7f7429200640 1 -- 172.21.15.19:0/2642093687 wait complete. 2024-03-28T13:21:18.716 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:21:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.932+0000 7f401ce00640 1 Processor -- start 2024-03-28T13:21:18.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.932+0000 7f401ce00640 1 -- start start 2024-03-28T13:21:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.934+0000 7f401ce00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f401806e030 0x7f401815fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.934+0000 7f401ce00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f40181602d0 0x7f40180cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.934+0000 7f401ce00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4018142880 con 0x7f401813f300 2024-03-28T13:21:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.934+0000 7f401ce00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f40180cee90 con 0x7f40181602d0 2024-03-28T13:21:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.934+0000 7f401ce00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f40181422c0 con 0x7f401806e030 2024-03-28T13:21:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.934+0000 7f4017400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f401813f300 0x7f40181419e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51734/0 (socket says 172.21.15.19:51734) 2024-03-28T13:21:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.934+0000 7f4017400640 1 -- 172.21.15.19:0/164014464 learned_addr learned my addr 172.21.15.19:0/164014464 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:18.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.935+0000 7f4017e00640 1 --2- 172.21.15.19:0/164014464 >> v2:172.21.15.19:3300/0 conn(0x7f40181602d0 0x7f40180cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.935+0000 7f400fe00640 1 --2- 172.21.15.19:0/164014464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f401806e030 0x7f401815fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.935+0000 7f4017e00640 1 -- 172.21.15.19:0/164014464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f401806e030 msgr2=0x7f401815fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.935+0000 7f4017e00640 1 --2- 172.21.15.19:0/164014464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f401806e030 0x7f401815fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.935+0000 7f4017e00640 1 -- 172.21.15.19:0/164014464 >> v1:172.21.15.19:6789/0 conn(0x7f401813f300 legacy=0x7f40181419e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.935+0000 7f4016a00640 1 -- 172.21.15.19:0/164014464 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3213644068 0 0) 0x7f4018142880 con 0x7f401813f300 2024-03-28T13:21:18.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.935+0000 7f4017e00640 1 -- 172.21.15.19:0/164014464 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40181425a0 con 0x7f40181602d0 2024-03-28T13:21:18.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.936+0000 7f4017e00640 1 --2- 172.21.15.19:0/164014464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40181602d0 0x7f40180cf320 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f4010009d90 tx=0x7f401002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92a943430282dbe server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:18.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.937+0000 7f4016a00640 1 -- 172.21.15.19:0/164014464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010039070 con 0x7f40181602d0 2024-03-28T13:21:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.937+0000 7f4016a00640 1 -- 172.21.15.19:0/164014464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f401002cc50 con 0x7f40181602d0 2024-03-28T13:21:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.937+0000 7f4016a00640 1 -- 172.21.15.19:0/164014464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010034440 con 0x7f40181602d0 2024-03-28T13:21:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.937+0000 7f401ce00640 1 -- 172.21.15.19:0/164014464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40181602d0 msgr2=0x7f40180cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:18.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.937+0000 7f401ce00640 1 --2- 172.21.15.19:0/164014464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40181602d0 0x7f40180cf320 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f4010009d90 tx=0x7f401002bea0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.938+0000 7f401ce00640 1 -- 172.21.15.19:0/164014464 shutdown_connections 2024-03-28T13:21:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.938+0000 7f401ce00640 1 --2- 172.21.15.19:0/164014464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40181602d0 0x7f40180cf320 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.938+0000 7f401ce00640 1 --2- 172.21.15.19:0/164014464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f401806e030 0x7f401815fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.938+0000 7f401ce00640 1 -- 172.21.15.19:0/164014464 >> 172.21.15.19:0/164014464 conn(0x7f401806b570 msgr2=0x7f401806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.938+0000 7f401ce00640 1 -- 172.21.15.19:0/164014464 shutdown_connections 2024-03-28T13:21:18.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.938+0000 7f401ce00640 1 -- 172.21.15.19:0/164014464 wait complete. 2024-03-28T13:21:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.941+0000 7f401ce00640 1 Processor -- start 2024-03-28T13:21:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.941+0000 7f401ce00640 1 -- start start 2024-03-28T13:21:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.942+0000 7f401ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401806e030 0x7f4018140e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.942+0000 7f401ce00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f401813f300 0x7f40180b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.942+0000 7f401ce00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40181602d0 0x7f40181320c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.942+0000 7f401ce00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40180cf9e0 con 0x7f401806e030 2024-03-28T13:21:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.942+0000 7f401ce00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f40180cf860 con 0x7f401813f300 2024-03-28T13:21:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.942+0000 7f401ce00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f40180cfb60 con 0x7f40181602d0 2024-03-28T13:21:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.943+0000 7f4017400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401806e030 0x7f4018140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.943+0000 7f4017400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401806e030 0x7f4018140e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52730/0 (socket says 172.21.15.19:52730) 2024-03-28T13:21:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.943+0000 7f4017400640 1 -- 172.21.15.19:0/2895131036 learned_addr learned my addr 172.21.15.19:0/2895131036 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.943+0000 7f400fe00640 1 --2- 172.21.15.19:0/2895131036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f401813f300 0x7f40180b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.943+0000 7f4017e00640 1 --2- 172.21.15.19:0/2895131036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40181602d0 0x7f40181320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.943+0000 7f4017400640 1 -- 172.21.15.19:0/2895131036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40181602d0 msgr2=0x7f40181320c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.943+0000 7f4017400640 1 --2- 172.21.15.19:0/2895131036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40181602d0 0x7f40181320c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.943+0000 7f4017400640 1 -- 172.21.15.19:0/2895131036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f401813f300 msgr2=0x7f40180b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.943+0000 7f4017400640 1 --2- 172.21.15.19:0/2895131036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f401813f300 0x7f40180b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.943+0000 7f4017400640 1 -- 172.21.15.19:0/2895131036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40180b0d70 con 0x7f401806e030 2024-03-28T13:21:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.943+0000 7f4017e00640 1 --2- 172.21.15.19:0/2895131036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40181602d0 0x7f40181320c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.944+0000 7f4017400640 1 --2- 172.21.15.19:0/2895131036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401806e030 0x7f4018140e00 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f400800c500 tx=0x7f400800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.944+0000 7f400f400640 1 -- 172.21.15.19:0/2895131036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008018070 con 0x7f401806e030 2024-03-28T13:21:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.944+0000 7f400f400640 1 -- 172.21.15.19:0/2895131036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4008007580 con 0x7f401806e030 2024-03-28T13:21:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.944+0000 7f401ce00640 1 -- 172.21.15.19:0/2895131036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40180b1050 con 0x7f401806e030 2024-03-28T13:21:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.944+0000 7f400f400640 1 -- 172.21.15.19:0/2895131036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008004a20 con 0x7f401806e030 2024-03-28T13:21:18.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.944+0000 7f401ce00640 1 -- 172.21.15.19:0/2895131036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40180b1610 con 0x7f401806e030 2024-03-28T13:21:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.946+0000 7f401ce00640 1 -- 172.21.15.19:0/2895131036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4018075940 con 0x7f401806e030 2024-03-28T13:21:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.946+0000 7f400f400640 1 -- 172.21.15.19:0/2895131036 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f400801b070 con 0x7f401806e030 2024-03-28T13:21:18.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.947+0000 7f400f400640 1 --2- 172.21.15.19:0/2895131036 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ff406c530 0x7f3ff406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.947+0000 7f400f400640 1 -- 172.21.15.19:0/2895131036 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7f400808aaa0 con 0x7f401806e030 2024-03-28T13:21:18.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.948+0000 7f400fe00640 1 --2- 172.21.15.19:0/2895131036 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ff406c530 0x7f3ff406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:18.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.949+0000 7f400fe00640 1 --2- 172.21.15.19:0/2895131036 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ff406c530 0x7f3ff406e9f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4018141cd0 tx=0x7f4000008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:18.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:18.952+0000 7f400f400640 1 -- 172.21.15.19:0/2895131036 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4008059050 con 0x7f401806e030 2024-03-28T13:21:19.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19.254+0000 7f401ce00640 1 -- 172.21.15.19:0/2895131036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f40180d0380 con 0x7f401806e030 2024-03-28T13:21:19.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19.255+0000 7f400f400640 1 -- 172.21.15.19:0/2895131036 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v105) ==== 53+0+73 (secure 0 0 0) 0x7f4008058950 con 0x7f401806e030 2024-03-28T13:21:19.254 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:21:19.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19.258+0000 7f401ce00640 1 -- 172.21.15.19:0/2895131036 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ff406c530 msgr2=0x7f3ff406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:19.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19.258+0000 7f401ce00640 1 --2- 172.21.15.19:0/2895131036 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ff406c530 0x7f3ff406e9f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4018141cd0 tx=0x7f4000008040 comp rx=0 tx=0).stop 2024-03-28T13:21:19.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19.258+0000 7f401ce00640 1 -- 172.21.15.19:0/2895131036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401806e030 msgr2=0x7f4018140e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:19.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19.258+0000 7f401ce00640 1 --2- 172.21.15.19:0/2895131036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401806e030 0x7f4018140e00 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f400800c500 tx=0x7f400800c9c0 comp rx=0 tx=0).stop 2024-03-28T13:21:19.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19.258+0000 7f401ce00640 1 -- 172.21.15.19:0/2895131036 shutdown_connections 2024-03-28T13:21:19.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19.258+0000 7f401ce00640 1 --2- 172.21.15.19:0/2895131036 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f3ff406c530 0x7f3ff406e9f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:19.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19.258+0000 7f401ce00640 1 --2- 172.21.15.19:0/2895131036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f40181602d0 0x7f40181320c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:19.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19.258+0000 7f401ce00640 1 --2- 172.21.15.19:0/2895131036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f401813f300 0x7f40180b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:19.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19.258+0000 7f401ce00640 1 --2- 172.21.15.19:0/2895131036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401806e030 0x7f4018140e00 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:19.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19.258+0000 7f401ce00640 1 -- 172.21.15.19:0/2895131036 >> 172.21.15.19:0/2895131036 conn(0x7f401806b570 msgr2=0x7f401815ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:19.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19.259+0000 7f401ce00640 1 -- 172.21.15.19:0/2895131036 shutdown_connections 2024-03-28T13:21:19.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19.259+0000 7f401ce00640 1 -- 172.21.15.19:0/2895131036 wait complete. 2024-03-28T13:21:19.277 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-28T13:21:19.278 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:21:19.278 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:21:19.278 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:21:19.278 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:19.278 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-28T13:21:19.278 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-28T13:21:19.278 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:21:19.278 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:21:19.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:19.279 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:21:19.330 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:21:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:21:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:21:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41875sec preferred_lft 41875sec 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:21:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:21:19.332 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:21:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:21:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:21:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:19.332 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:21:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:21:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:21:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:19.332 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:21:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether e6:17:c5:8e:2b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:21:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::e417:c5ff:fe8e:2bcf/64 scope link 2024-03-28T13:21:19.333 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:19.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:19.333 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:21:19.378 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-03-28T13:21:19.378+0000 7f782aa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:21:19.422 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:21:19.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:19.426 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:21:19.522 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:21:19.522 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:21:19.523 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:21:19.523 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:21:19.523 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:21:19.523 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:21:19.523 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:21:19.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:21:19.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:21:19.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:21:19.523 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:21:19.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:21:19.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:19.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:19.782 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:21:19.831 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:21:19.831 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:21:19.831 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:21:19.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:19.831 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:21:19.911 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:21:19.911 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:19.962 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:21:19.962 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:21:19.962 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:20.010 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:21:20.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:20.010 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:20.144 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:20.145 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:21:20.145 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:21:20.166 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:20.166 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:21:20.166 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:21:20.166 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:21:20.166 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:21:20.192 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:20.193 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:21:20.193 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:21:20.193 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:21:20.193 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:21:20.193 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:21:20.193 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:21:20.193 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:21:20.194 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:21:20.194 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:21:20.194 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:21:20.195 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:21:20.195 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:21:20.198 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:21:20.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:20.198 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:20.271 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:21:20.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:21:20.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:20.388 INFO:teuthology.orchestra.run.smithi019.stdout:client9018 2024-03-28T13:21:20.388 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:20.388 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:20.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:20.434 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-28T13:21:20.542 INFO:teuthology.orchestra.run.smithi019.stdout:./ 2024-03-28T13:21:20.543 INFO:teuthology.orchestra.run.smithi019.stdout:./parent 2024-03-28T13:21:20.543 INFO:teuthology.orchestra.run.smithi019.stdout:./parent/flushed 2024-03-28T13:21:20.543 INFO:teuthology.orchestra.run.smithi019.stdout:./parent/flushed/charlie 2024-03-28T13:21:20.543 INFO:teuthology.orchestra.run.smithi019.stdout:./parent/flushed/alpha 2024-03-28T13:21:20.543 INFO:teuthology.orchestra.run.smithi019.stdout:./parent/unflushed 2024-03-28T13:21:20.543 INFO:teuthology.orchestra.run.smithi019.stdout:./parent/unflushed/jfile 2024-03-28T13:21:20.544 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:20.544 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:20.544 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:20.544 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:20.544 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-28T13:21:20.544 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:20.682 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:21:20.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:20.695 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:20.695 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:20.696 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:20.696 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:20.696 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-03-28T13:21:20.696 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:20.904 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627776 2024-03-28T13:21:20.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:20.918 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:20.918 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:20.918 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:20.918 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:20.918 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-03-28T13:21:20.918 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:21.103 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627777 2024-03-28T13:21:21.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:21.117 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:21.117 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:21.117 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:21.117 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:21.117 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-03-28T13:21:21.117 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:21.295 INFO:teuthology.orchestra.run.smithi019.stdout:1099511628280 2024-03-28T13:21:21.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:21.310 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:21.311 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:21.311 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:21.311 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:21.311 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-03-28T13:21:21.311 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:21.497 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627778 2024-03-28T13:21:21.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:21.512 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:21.512 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:21.512 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:21.512 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:21.512 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-03-28T13:21:21.512 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:21.689 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627779 2024-03-28T13:21:21.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:21.708 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:21.708 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:21.708 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:21.708 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:21.708 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-03-28T13:21:21.708 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:21.885 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627780 2024-03-28T13:21:21.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:21.898 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:21.898 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:21.997 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:21.997 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:22.044 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:21:22.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.345+0000 7fc92c200640 1 Processor -- start 2024-03-28T13:21:22.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.346+0000 7fc92c200640 1 -- start start 2024-03-28T13:21:22.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.347+0000 7fc92c200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc92415c390 0x7fc924073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:22.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.347+0000 7fc92c200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc9240d0650 0x7fc9240d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:22.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.347+0000 7fc92c200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc92415fe80 con 0x7fc9240af830 2024-03-28T13:21:22.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.347+0000 7fc92c200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc92415c770 con 0x7fc9240d0650 2024-03-28T13:21:22.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.347+0000 7fc92c200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc9240d0fe0 con 0x7fc92415c390 2024-03-28T13:21:22.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.348+0000 7fc92a200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc9240d0650 0x7fc9240d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:22.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.348+0000 7fc928e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc9240af830 0x7fc924160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51748/0 (socket says 172.21.15.19:51748) 2024-03-28T13:21:22.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.348+0000 7fc928e00640 1 -- 172.21.15.19:0/2809699376 learned_addr learned my addr 172.21.15.19:0/2809699376 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:22.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.348+0000 7fc929800640 1 --2- 172.21.15.19:0/2809699376 >> v2:172.21.15.188:3300/0 conn(0x7fc92415c390 0x7fc924073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:22.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.348+0000 7fc92a200640 1 -- 172.21.15.19:0/2809699376 >> v2:172.21.15.188:3300/0 conn(0x7fc92415c390 msgr2=0x7fc924073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:22.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.348+0000 7fc92a200640 1 --2- 172.21.15.19:0/2809699376 >> v2:172.21.15.188:3300/0 conn(0x7fc92415c390 0x7fc924073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.348+0000 7fc92a200640 1 -- 172.21.15.19:0/2809699376 >> v1:172.21.15.19:6789/0 conn(0x7fc9240af830 legacy=0x7fc924160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:22.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.349+0000 7fc92a200640 1 -- 172.21.15.19:0/2809699376 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9240ad120 con 0x7fc9240d0650 2024-03-28T13:21:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.350+0000 7fc92a200640 1 --2- 172.21.15.19:0/2809699376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9240d0650 0x7fc9240d0aa0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fc9240b63e0 tx=0x7fc91c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fddcfeef7c2b1f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.351+0000 7fc923e00640 1 -- 172.21.15.19:0/2809699376 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c035020 con 0x7fc9240d0650 2024-03-28T13:21:22.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.351+0000 7fc923e00640 1 -- 172.21.15.19:0/2809699376 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc91c031830 con 0x7fc9240d0650 2024-03-28T13:21:22.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.351+0000 7fc923e00640 1 -- 172.21.15.19:0/2809699376 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc91c031c00 con 0x7fc9240d0650 2024-03-28T13:21:22.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.351+0000 7fc92c200640 1 -- 172.21.15.19:0/2809699376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9240d0650 msgr2=0x7fc9240d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:22.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.351+0000 7fc92c200640 1 --2- 172.21.15.19:0/2809699376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9240d0650 0x7fc9240d0aa0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fc9240b63e0 tx=0x7fc91c02b8d0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.351+0000 7fc92c200640 1 -- 172.21.15.19:0/2809699376 shutdown_connections 2024-03-28T13:21:22.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.351+0000 7fc92c200640 1 --2- 172.21.15.19:0/2809699376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9240d0650 0x7fc9240d0aa0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.351+0000 7fc92c200640 1 --2- 172.21.15.19:0/2809699376 >> v2:172.21.15.188:3300/0 conn(0x7fc92415c390 0x7fc924073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.351+0000 7fc92c200640 1 -- 172.21.15.19:0/2809699376 >> 172.21.15.19:0/2809699376 conn(0x7fc92406b990 msgr2=0x7fc92406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:22.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.351+0000 7fc92c200640 1 -- 172.21.15.19:0/2809699376 shutdown_connections 2024-03-28T13:21:22.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.352+0000 7fc92c200640 1 -- 172.21.15.19:0/2809699376 wait complete. 2024-03-28T13:21:22.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.353+0000 7fc92c200640 1 Processor -- start 2024-03-28T13:21:22.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.353+0000 7fc92c200640 1 -- start start 2024-03-28T13:21:22.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.354+0000 7fc92c200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9240af830 0x7fc92414b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:22.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.354+0000 7fc92c200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9240d0650 0x7fc9240b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:22.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.354+0000 7fc92c200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc92415c390 0x7fc92415dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:22.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.354+0000 7fc92c200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc924160120 con 0x7fc9240af830 2024-03-28T13:21:22.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.354+0000 7fc92c200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc9240ad6e0 con 0x7fc9240d0650 2024-03-28T13:21:22.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.354+0000 7fc92c200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc924160d50 con 0x7fc92415c390 2024-03-28T13:21:22.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.354+0000 7fc929800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9240af830 0x7fc92414b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:22.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.355+0000 7fc929800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9240af830 0x7fc92414b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52762/0 (socket says 172.21.15.19:52762) 2024-03-28T13:21:22.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.355+0000 7fc929800640 1 -- 172.21.15.19:0/2162189282 learned_addr learned my addr 172.21.15.19:0/2162189282 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:22.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.355+0000 7fc928e00640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9240d0650 0x7fc9240b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:22.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.355+0000 7fc92a200640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc92415c390 0x7fc92415dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:22.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.355+0000 7fc929800640 1 -- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc92415c390 msgr2=0x7fc92415dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:22.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.355+0000 7fc929800640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc92415c390 0x7fc92415dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.355+0000 7fc929800640 1 -- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9240d0650 msgr2=0x7fc9240b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:22.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.355+0000 7fc929800640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9240d0650 0x7fc9240b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.355+0000 7fc929800640 1 -- 172.21.15.19:0/2162189282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9240b3ed0 con 0x7fc9240af830 2024-03-28T13:21:22.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.355+0000 7fc928e00640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9240d0650 0x7fc9240b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:22.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.355+0000 7fc92a200640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc92415c390 0x7fc92415dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:22.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.355+0000 7fc929800640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9240af830 0x7fc92414b230 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fc914007ef0 tx=0x7fc91400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:22.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.356+0000 7fc921600640 1 -- 172.21.15.19:0/2162189282 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914014620 con 0x7fc9240af830 2024-03-28T13:21:22.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.356+0000 7fc921600640 1 -- 172.21.15.19:0/2162189282 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc914019070 con 0x7fc9240af830 2024-03-28T13:21:22.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.356+0000 7fc921600640 1 -- 172.21.15.19:0/2162189282 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc914014a00 con 0x7fc9240af830 2024-03-28T13:21:22.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.356+0000 7fc92c200640 1 -- 172.21.15.19:0/2162189282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9240b41b0 con 0x7fc9240af830 2024-03-28T13:21:22.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.356+0000 7fc92c200640 1 -- 172.21.15.19:0/2162189282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9240b4c90 con 0x7fc9240af830 2024-03-28T13:21:22.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.357+0000 7fc92c200640 1 -- 172.21.15.19:0/2162189282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9240685c0 con 0x7fc9240af830 2024-03-28T13:21:22.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.358+0000 7fc921600640 1 -- 172.21.15.19:0/2162189282 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc914014ba0 con 0x7fc9240af830 2024-03-28T13:21:22.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.360+0000 7fc921600640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc90406c560 0x7fc90406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:22.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.360+0000 7fc921600640 1 -- 172.21.15.19:0/2162189282 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7fc91408bfb0 con 0x7fc9240af830 2024-03-28T13:21:22.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.361+0000 7fc928e00640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc90406c560 0x7fc90406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:22.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.362+0000 7fc928e00640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc90406c560 0x7fc90406ea20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc92415f400 tx=0x7fc910009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:22.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.364+0000 7fc921600640 1 -- 172.21.15.19:0/2162189282 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc91405a600 con 0x7fc9240af830 2024-03-28T13:21:22.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.652+0000 7fc92c200640 1 -- 172.21.15.19:0/2162189282 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc92415dbc0 con 0x7fc9240af830 2024-03-28T13:21:22.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.654+0000 7fc921600640 1 -- 172.21.15.19:0/2162189282 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+16814 (secure 0 0 0) 0x7fc914059f00 con 0x7fc9240af830 2024-03-28T13:21:22.654 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:22.655 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":120,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:21:12.469756+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:20:50.748212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-28T13:20:51.888444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6836/2088538359":"2024-03-29T13:21:12.469733+0000","172.21.15.188:6834/2088538359":"2024-03-29T13:21:12.469733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:21:22.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.657+0000 7fc92c200640 1 -- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc90406c560 msgr2=0x7fc90406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:22.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.657+0000 7fc92c200640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc90406c560 0x7fc90406ea20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc92415f400 tx=0x7fc910009040 comp rx=0 tx=0).stop 2024-03-28T13:21:22.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.657+0000 7fc92c200640 1 -- 172.21.15.19:0/2162189282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9240af830 msgr2=0x7fc92414b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:22.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.657+0000 7fc92c200640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9240af830 0x7fc92414b230 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fc914007ef0 tx=0x7fc91400e4b0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.657+0000 7fc92c200640 1 -- 172.21.15.19:0/2162189282 shutdown_connections 2024-03-28T13:21:22.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.657+0000 7fc92c200640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc90406c560 0x7fc90406ea20 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.657+0000 7fc92c200640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc92415c390 0x7fc92415dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.657+0000 7fc92c200640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9240d0650 0x7fc9240b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.657+0000 7fc92c200640 1 --2- 172.21.15.19:0/2162189282 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc9240af830 0x7fc92414b230 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.657+0000 7fc92c200640 1 -- 172.21.15.19:0/2162189282 >> 172.21.15.19:0/2162189282 conn(0x7fc92406b990 msgr2=0x7fc924157df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:22.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.657+0000 7fc92c200640 1 -- 172.21.15.19:0/2162189282 shutdown_connections 2024-03-28T13:21:22.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.657+0000 7fc92c200640 1 -- 172.21.15.19:0/2162189282 wait complete. 2024-03-28T13:21:22.676 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:20:50.748212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:20:51.888444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:21:22.676 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:21:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.893+0000 7f6707e00640 1 Processor -- start 2024-03-28T13:21:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.894+0000 7f6707e00640 1 -- start start 2024-03-28T13:21:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.895+0000 7f6707e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f67080cd890 0x7f670815d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.895+0000 7f6707e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67081529d0 0x7f6708152e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.895+0000 7f6707e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67080b3ee0 con 0x7f67080afaa0 2024-03-28T13:21:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.895+0000 7f6707e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f67080b41c0 con 0x7f67080cd890 2024-03-28T13:21:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.895+0000 7f6707e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f67080b44a0 con 0x7f67081529d0 2024-03-28T13:21:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.895+0000 7f6706000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f67080cd890 0x7f670815d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.895+0000 7f6706a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f67080afaa0 0x7f67080cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51764/0 (socket says 172.21.15.19:51764) 2024-03-28T13:21:22.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.895+0000 7f6706a00640 1 -- 172.21.15.19:0/101970151 learned_addr learned my addr 172.21.15.19:0/101970151 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.896+0000 7f6707400640 1 --2- 172.21.15.19:0/101970151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67081529d0 0x7f6708152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.896+0000 7f6706000640 1 -- 172.21.15.19:0/101970151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67081529d0 msgr2=0x7f6708152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.896+0000 7f6706000640 1 --2- 172.21.15.19:0/101970151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67081529d0 0x7f6708152e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.896+0000 7f6706000640 1 -- 172.21.15.19:0/101970151 >> v1:172.21.15.19:6789/0 conn(0x7f67080afaa0 legacy=0x7f67080cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.896+0000 7f6706000640 1 -- 172.21.15.19:0/101970151 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67080b4780 con 0x7f67080cd890 2024-03-28T13:21:22.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.896+0000 7f6705600640 1 -- 172.21.15.19:0/101970151 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1178060878 0 0) 0x7f67080b3ee0 con 0x7f67080afaa0 2024-03-28T13:21:22.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.897+0000 7f6706000640 1 --2- 172.21.15.19:0/101970151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67080cd890 0x7f670815d230 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f66f80097a0 tx=0x7f66f802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831f35375f42d8f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:22.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.897+0000 7f6705600640 1 -- 172.21.15.19:0/101970151 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8031780 con 0x7f67080cd890 2024-03-28T13:21:22.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.897+0000 7f6705600640 1 -- 172.21.15.19:0/101970151 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f8037070 con 0x7f67080cd890 2024-03-28T13:21:22.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.897+0000 7f6705600640 1 -- 172.21.15.19:0/101970151 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8031b40 con 0x7f67080cd890 2024-03-28T13:21:22.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.898+0000 7f6707e00640 1 -- 172.21.15.19:0/101970151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67080cd890 msgr2=0x7f670815d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:22.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.898+0000 7f6707e00640 1 --2- 172.21.15.19:0/101970151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67080cd890 0x7f670815d230 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f66f80097a0 tx=0x7f66f802b820 comp rx=0 tx=0).stop 2024-03-28T13:21:22.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.898+0000 7f6707e00640 1 -- 172.21.15.19:0/101970151 shutdown_connections 2024-03-28T13:21:22.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.898+0000 7f6707e00640 1 --2- 172.21.15.19:0/101970151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67081529d0 0x7f6708152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.898+0000 7f6707e00640 1 --2- 172.21.15.19:0/101970151 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67080cd890 0x7f670815d230 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.898+0000 7f6707e00640 1 -- 172.21.15.19:0/101970151 >> 172.21.15.19:0/101970151 conn(0x7f67080d0770 msgr2=0x7f67080d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:22.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.898+0000 7f6707e00640 1 -- 172.21.15.19:0/101970151 shutdown_connections 2024-03-28T13:21:22.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.898+0000 7f6707e00640 1 -- 172.21.15.19:0/101970151 wait complete. 2024-03-28T13:21:22.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.900+0000 7f6707e00640 1 Processor -- start 2024-03-28T13:21:22.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.900+0000 7f6707e00640 1 -- start start 2024-03-28T13:21:22.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.901+0000 7f6707e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67080afaa0 0x7f67081f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:22.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.901+0000 7f6707e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67080cd890 0x7f670814fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:22.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.901+0000 7f6707e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67081529d0 0x7f670815b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:22.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.901+0000 7f6707e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f67080afe80 con 0x7f67080cd890 2024-03-28T13:21:22.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.901+0000 7f6707e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f67080b4a60 con 0x7f67080afaa0 2024-03-28T13:21:22.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.901+0000 7f6707e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f67080b0000 con 0x7f67081529d0 2024-03-28T13:21:22.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.901+0000 7f6706000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67080cd890 0x7f670814fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:22.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.901+0000 7f6706000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67080cd890 0x7f670814fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52790/0 (socket says 172.21.15.19:52790) 2024-03-28T13:21:22.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.901+0000 7f6706000640 1 -- 172.21.15.19:0/96692323 learned_addr learned my addr 172.21.15.19:0/96692323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:22.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.901+0000 7f6706a00640 1 --2- 172.21.15.19:0/96692323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67080afaa0 0x7f67081f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:22.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.901+0000 7f6707400640 1 --2- 172.21.15.19:0/96692323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67081529d0 0x7f670815b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:22.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.902+0000 7f6706000640 1 -- 172.21.15.19:0/96692323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67081529d0 msgr2=0x7f670815b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:22.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.902+0000 7f6706000640 1 --2- 172.21.15.19:0/96692323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67081529d0 0x7f670815b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.902+0000 7f6706000640 1 -- 172.21.15.19:0/96692323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67080afaa0 msgr2=0x7f67081f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:22.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.902+0000 7f6706000640 1 --2- 172.21.15.19:0/96692323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67080afaa0 0x7f67081f1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:22.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.902+0000 7f6706000640 1 -- 172.21.15.19:0/96692323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6708156e80 con 0x7f67080cd890 2024-03-28T13:21:22.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.902+0000 7f6706000640 1 --2- 172.21.15.19:0/96692323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67080cd890 0x7f670814fe20 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f66f8034040 tx=0x7f66f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:22.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.902+0000 7f66fea00640 1 -- 172.21.15.19:0/96692323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8032820 con 0x7f67080cd890 2024-03-28T13:21:22.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.902+0000 7f66fea00640 1 -- 172.21.15.19:0/96692323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f80329e0 con 0x7f67080cd890 2024-03-28T13:21:22.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.902+0000 7f66fea00640 1 -- 172.21.15.19:0/96692323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f8032dd0 con 0x7f67080cd890 2024-03-28T13:21:22.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.902+0000 7f6707e00640 1 -- 172.21.15.19:0/96692323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670815be20 con 0x7f67080cd890 2024-03-28T13:21:22.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.902+0000 7f6707e00640 1 -- 172.21.15.19:0/96692323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6708157400 con 0x7f67080cd890 2024-03-28T13:21:22.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.904+0000 7f6707e00640 1 -- 172.21.15.19:0/96692323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670815c160 con 0x7f67080cd890 2024-03-28T13:21:22.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.904+0000 7f66fea00640 1 -- 172.21.15.19:0/96692323 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f66f804e440 con 0x7f67080cd890 2024-03-28T13:21:22.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.905+0000 7f66fea00640 1 --2- 172.21.15.19:0/96692323 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f66e406c560 0x7f66e406ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:22.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.905+0000 7f66fea00640 1 -- 172.21.15.19:0/96692323 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7f66f80ad640 con 0x7f67080cd890 2024-03-28T13:21:22.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.906+0000 7f6706a00640 1 --2- 172.21.15.19:0/96692323 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f66e406c560 0x7f66e406ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:22.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.907+0000 7f6706a00640 1 --2- 172.21.15.19:0/96692323 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f66e406c560 0x7f66e406ea20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f66f0000f60 tx=0x7f66f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:22.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:22.910+0000 7f66fea00640 1 -- 172.21.15.19:0/96692323 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f66f8040070 con 0x7f67080cd890 2024-03-28T13:21:23.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.111+0000 7f6707e00640 1 -- 172.21.15.19:0/96692323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6708075940 con 0x7f67080cd890 2024-03-28T13:21:23.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.111+0000 7f66fea00640 1 -- 172.21.15.19:0/96692323 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) ==== 93+0+10 (secure 0 0 0) 0x7f66f802f050 con 0x7f67080cd890 2024-03-28T13:21:23.111 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:21:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.114+0000 7f6707e00640 1 -- 172.21.15.19:0/96692323 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f66e406c560 msgr2=0x7f66e406ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.114+0000 7f6707e00640 1 --2- 172.21.15.19:0/96692323 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f66e406c560 0x7f66e406ea20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f66f0000f60 tx=0x7f66f0009040 comp rx=0 tx=0).stop 2024-03-28T13:21:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.114+0000 7f6707e00640 1 -- 172.21.15.19:0/96692323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67080cd890 msgr2=0x7f670814fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.114+0000 7f6707e00640 1 --2- 172.21.15.19:0/96692323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67080cd890 0x7f670814fe20 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f66f8034040 tx=0x7f66f80046c0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.114+0000 7f6707e00640 1 -- 172.21.15.19:0/96692323 shutdown_connections 2024-03-28T13:21:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.114+0000 7f6707e00640 1 --2- 172.21.15.19:0/96692323 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f66e406c560 0x7f66e406ea20 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.114+0000 7f6707e00640 1 --2- 172.21.15.19:0/96692323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67081529d0 0x7f670815b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.114+0000 7f6707e00640 1 --2- 172.21.15.19:0/96692323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67080cd890 0x7f670814fe20 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.114+0000 7f6707e00640 1 --2- 172.21.15.19:0/96692323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67080afaa0 0x7f67081f1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.114+0000 7f6707e00640 1 -- 172.21.15.19:0/96692323 >> 172.21.15.19:0/96692323 conn(0x7f67080d0770 msgr2=0x7f67080b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.114+0000 7f6707e00640 1 -- 172.21.15.19:0/96692323 shutdown_connections 2024-03-28T13:21:23.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.114+0000 7f6707e00640 1 -- 172.21.15.19:0/96692323 wait complete. 2024-03-28T13:21:23.134 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:21:23.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.348+0000 7fc0b3a00640 1 Processor -- start 2024-03-28T13:21:23.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.348+0000 7fc0b3a00640 1 -- start start 2024-03-28T13:21:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.349+0000 7fc0b3a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc0ac15e860 0x7fc0ac06d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.349+0000 7fc0b3a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0ac06da30 0x7fc0ac0d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.349+0000 7fc0b3a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0ac06dfa0 con 0x7fc0ac0d4910 2024-03-28T13:21:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.350+0000 7fc0b3a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fc0ac0cf220 con 0x7fc0ac06da30 2024-03-28T13:21:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.350+0000 7fc0b3a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc0ac0d4cf0 con 0x7fc0ac15e860 2024-03-28T13:21:23.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.350+0000 7fc0abe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0ac06da30 0x7fc0ac0d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.350+0000 7fc0abe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fc0ac06da30 0x7fc0ac0d43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52792/0 (socket says 172.21.15.19:52792) 2024-03-28T13:21:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.350+0000 7fc0abe00640 1 -- 172.21.15.19:0/3954961783 learned_addr learned my addr 172.21.15.19:0/3954961783 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.350+0000 7fc0b1a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fc0ac0d4910 0x7fc0ac0d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:51774/0 (socket says 172.21.15.19:51774) 2024-03-28T13:21:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.350+0000 7fc0b1000640 1 --2- 172.21.15.19:0/3954961783 >> v2:172.21.15.188:3300/0 conn(0x7fc0ac15e860 0x7fc0ac06d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.351+0000 7fc0abe00640 1 -- 172.21.15.19:0/3954961783 >> v2:172.21.15.188:3300/0 conn(0x7fc0ac15e860 msgr2=0x7fc0ac06d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.351+0000 7fc0abe00640 1 --2- 172.21.15.19:0/3954961783 >> v2:172.21.15.188:3300/0 conn(0x7fc0ac15e860 0x7fc0ac06d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.351+0000 7fc0abe00640 1 -- 172.21.15.19:0/3954961783 >> v1:172.21.15.19:6789/0 conn(0x7fc0ac0d4910 legacy=0x7fc0ac0d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.351+0000 7fc0abe00640 1 -- 172.21.15.19:0/3954961783 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac06e240 con 0x7fc0ac06da30 2024-03-28T13:21:23.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.351+0000 7fc0ab400640 1 -- 172.21.15.19:0/3954961783 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1950973191 0 0) 0x7fc0ac06dfa0 con 0x7fc0ac0d4910 2024-03-28T13:21:23.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.352+0000 7fc0abe00640 1 --2- 172.21.15.19:0/3954961783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac06da30 0x7fc0ac0d43d0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0d1fa0 tx=0x7fc09802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216151acf2ed86e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:23.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.352+0000 7fc0ab400640 1 -- 172.21.15.19:0/3954961783 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098034070 con 0x7fc0ac06da30 2024-03-28T13:21:23.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.352+0000 7fc0ab400640 1 -- 172.21.15.19:0/3954961783 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc098030860 con 0x7fc0ac06da30 2024-03-28T13:21:23.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.352+0000 7fc0ab400640 1 -- 172.21.15.19:0/3954961783 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098030c30 con 0x7fc0ac06da30 2024-03-28T13:21:23.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.353+0000 7fc0b3a00640 1 -- 172.21.15.19:0/3954961783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac06da30 msgr2=0x7fc0ac0d43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:23.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.353+0000 7fc0b3a00640 1 --2- 172.21.15.19:0/3954961783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac06da30 0x7fc0ac0d43d0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0d1fa0 tx=0x7fc09802b9a0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.353+0000 7fc0b3a00640 1 -- 172.21.15.19:0/3954961783 shutdown_connections 2024-03-28T13:21:23.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.353+0000 7fc0b3a00640 1 --2- 172.21.15.19:0/3954961783 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac06da30 0x7fc0ac0d43d0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.353+0000 7fc0b3a00640 1 --2- 172.21.15.19:0/3954961783 >> v2:172.21.15.188:3300/0 conn(0x7fc0ac15e860 0x7fc0ac06d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.353+0000 7fc0b3a00640 1 -- 172.21.15.19:0/3954961783 >> 172.21.15.19:0/3954961783 conn(0x7fc0ac09d7d0 msgr2=0x7fc0ac08bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:23.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.353+0000 7fc0b3a00640 1 -- 172.21.15.19:0/3954961783 shutdown_connections 2024-03-28T13:21:23.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.353+0000 7fc0b3a00640 1 -- 172.21.15.19:0/3954961783 wait complete. 2024-03-28T13:21:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.356+0000 7fc0b3a00640 1 Processor -- start 2024-03-28T13:21:23.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.356+0000 7fc0b3a00640 1 -- start start 2024-03-28T13:21:23.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.358+0000 7fc0b3a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac06da30 0x7fc0ac158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:23.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.358+0000 7fc0b3a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac0d4910 0x7fc0ac1f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:23.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.358+0000 7fc0b3a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0ac15e860 0x7fc0ac156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:23.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.358+0000 7fc0b3a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac0cedb0 con 0x7fc0ac06da30 2024-03-28T13:21:23.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.358+0000 7fc0b3a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac0d2c50 con 0x7fc0ac15e860 2024-03-28T13:21:23.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.358+0000 7fc0b3a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ac0cef30 con 0x7fc0ac0d4910 2024-03-28T13:21:23.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.358+0000 7fc0b1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac06da30 0x7fc0ac158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:23.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.358+0000 7fc0b1000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac06da30 0x7fc0ac158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52804/0 (socket says 172.21.15.19:52804) 2024-03-28T13:21:23.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.358+0000 7fc0b1000640 1 -- 172.21.15.19:0/2246143611 learned_addr learned my addr 172.21.15.19:0/2246143611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:23.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.358+0000 7fc0b1a00640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0ac15e860 0x7fc0ac156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:23.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.359+0000 7fc0abe00640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac0d4910 0x7fc0ac1f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:23.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.359+0000 7fc0b1000640 1 -- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac0d4910 msgr2=0x7fc0ac1f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:23.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.359+0000 7fc0b1000640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac0d4910 0x7fc0ac1f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.359+0000 7fc0b1000640 1 -- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0ac15e860 msgr2=0x7fc0ac156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:23.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.359+0000 7fc0b1000640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0ac15e860 0x7fc0ac156b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.359+0000 7fc0b1000640 1 -- 172.21.15.19:0/2246143611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac0b7020 con 0x7fc0ac06da30 2024-03-28T13:21:23.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.359+0000 7fc0b1a00640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0ac15e860 0x7fc0ac156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:23.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.359+0000 7fc0abe00640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac0d4910 0x7fc0ac1f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:23.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.359+0000 7fc0b1000640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac06da30 0x7fc0ac158e80 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00d620 tx=0x7fc09c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:23.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.359+0000 7fc0a8c00640 1 -- 172.21.15.19:0/2246143611 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c00b620 con 0x7fc0ac06da30 2024-03-28T13:21:23.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.359+0000 7fc0a8c00640 1 -- 172.21.15.19:0/2246143611 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc09c019070 con 0x7fc0ac06da30 2024-03-28T13:21:23.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.359+0000 7fc0b3a00640 1 -- 172.21.15.19:0/2246143611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ac1f8130 con 0x7fc0ac06da30 2024-03-28T13:21:23.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.359+0000 7fc0a8c00640 1 -- 172.21.15.19:0/2246143611 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c00ba00 con 0x7fc0ac06da30 2024-03-28T13:21:23.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.359+0000 7fc0b3a00640 1 -- 172.21.15.19:0/2246143611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ac1f86f0 con 0x7fc0ac06da30 2024-03-28T13:21:23.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.361+0000 7fc0b3a00640 1 -- 172.21.15.19:0/2246143611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ac0685c0 con 0x7fc0ac06da30 2024-03-28T13:21:23.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.362+0000 7fc0a8c00640 1 -- 172.21.15.19:0/2246143611 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc09c00bba0 con 0x7fc0ac06da30 2024-03-28T13:21:23.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.363+0000 7fc0a8c00640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc08c06c530 0x7fc08c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:23.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.363+0000 7fc0a8c00640 1 -- 172.21.15.19:0/2246143611 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7fc09c08baa0 con 0x7fc0ac06da30 2024-03-28T13:21:23.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.363+0000 7fc0abe00640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc08c06c530 0x7fc08c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:23.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.364+0000 7fc0abe00640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc08c06c530 0x7fc08c06e9f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc098036000 tx=0x7fc09803b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:23.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.367+0000 7fc0a8c00640 1 -- 172.21.15.19:0/2246143611 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc09c05a0f0 con 0x7fc0ac06da30 2024-03-28T13:21:23.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.578+0000 7fc0b3a00640 1 -- 172.21.15.19:0/2246143611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc0ac0d08d0 con 0x7fc0ac06da30 2024-03-28T13:21:23.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.579+0000 7fc0a8c00640 1 -- 172.21.15.19:0/2246143611 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v120) ==== 104+0+11 (secure 0 0 0) 0x7fc09c0599f0 con 0x7fc0ac06da30 2024-03-28T13:21:23.579 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:21:23.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.581+0000 7fc0b3a00640 1 -- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc08c06c530 msgr2=0x7fc08c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:23.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.581+0000 7fc0b3a00640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc08c06c530 0x7fc08c06e9f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc098036000 tx=0x7fc09803b040 comp rx=0 tx=0).stop 2024-03-28T13:21:23.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.581+0000 7fc0b3a00640 1 -- 172.21.15.19:0/2246143611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac06da30 msgr2=0x7fc0ac158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:23.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.581+0000 7fc0b3a00640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac06da30 0x7fc0ac158e80 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00d620 tx=0x7fc09c00dae0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.582+0000 7fc0b3a00640 1 -- 172.21.15.19:0/2246143611 shutdown_connections 2024-03-28T13:21:23.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.582+0000 7fc0b3a00640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc08c06c530 0x7fc08c06e9f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.582+0000 7fc0b3a00640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0ac15e860 0x7fc0ac156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.582+0000 7fc0b3a00640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0ac0d4910 0x7fc0ac1f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.582+0000 7fc0b3a00640 1 --2- 172.21.15.19:0/2246143611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0ac06da30 0x7fc0ac158e80 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.582+0000 7fc0b3a00640 1 -- 172.21.15.19:0/2246143611 >> 172.21.15.19:0/2246143611 conn(0x7fc0ac09d7d0 msgr2=0x7fc0ac15ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:23.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.582+0000 7fc0b3a00640 1 -- 172.21.15.19:0/2246143611 shutdown_connections 2024-03-28T13:21:23.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.582+0000 7fc0b3a00640 1 -- 172.21.15.19:0/2246143611 wait complete. 2024-03-28T13:21:23.600 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:21:23.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.812+0000 7f14e1400640 1 Processor -- start 2024-03-28T13:21:23.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.812+0000 7f14e1400640 1 -- start start 2024-03-28T13:21:23.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.815+0000 7f14e1400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14dc0d37b0 0x7f14dc0b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:23.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.815+0000 7f14e1400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f14dc0b67f0 0x7f14dc0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:23.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.815+0000 7f14e1400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14dc0d2710 con 0x7f14dc15e4c0 2024-03-28T13:21:23.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.815+0000 7f14e1400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f14dc15e8a0 con 0x7f14dc0b67f0 2024-03-28T13:21:23.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.815+0000 7f14e1400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f14dc0d4040 con 0x7f14dc0d37b0 2024-03-28T13:21:23.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.816+0000 7f14e0a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f14dc0b67f0 0x7f14dc0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:23.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.816+0000 7f14d3e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14dc0d37b0 0x7f14dc0b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:23.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.816+0000 7f14e0a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f14dc0b67f0 0x7f14dc0b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:60418/0 (socket says 172.21.15.19:60418) 2024-03-28T13:21:23.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.816+0000 7f14e0a00640 1 -- 172.21.15.19:0/2663489363 learned_addr learned my addr 172.21.15.19:0/2663489363 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:23.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.817+0000 7f14d3e00640 1 -- 172.21.15.19:0/2663489363 >> v2:172.21.15.188:3300/0 conn(0x7f14dc0b67f0 msgr2=0x7f14dc0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:23.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.817+0000 7f14d3e00640 1 --2- 172.21.15.19:0/2663489363 >> v2:172.21.15.188:3300/0 conn(0x7f14dc0b67f0 0x7f14dc0b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.817+0000 7f14d3e00640 1 -- 172.21.15.19:0/2663489363 >> v1:172.21.15.188:6789/0 conn(0x7f14dc15e4c0 legacy=0x7f14dc0b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:23.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.817+0000 7f14d3e00640 1 -- 172.21.15.19:0/2663489363 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc0d29f0 con 0x7f14dc0d37b0 2024-03-28T13:21:23.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.817+0000 7f14e0a00640 1 --2- 172.21.15.19:0/2663489363 >> v2:172.21.15.188:3300/0 conn(0x7f14dc0b67f0 0x7f14dc0b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:21:23.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.818+0000 7f14d3e00640 1 --2- 172.21.15.19:0/2663489363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14dc0d37b0 0x7f14dc0b62b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f14c40097a0 tx=0x7f14c402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b6f588be881a773 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:23.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.819+0000 7f14db400640 1 -- 172.21.15.19:0/2663489363 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4035020 con 0x7f14dc0d37b0 2024-03-28T13:21:23.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.819+0000 7f14db400640 1 -- 172.21.15.19:0/2663489363 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14c4031810 con 0x7f14dc0d37b0 2024-03-28T13:21:23.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.819+0000 7f14db400640 1 -- 172.21.15.19:0/2663489363 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4031be0 con 0x7f14dc0d37b0 2024-03-28T13:21:23.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.819+0000 7f14e1400640 1 -- 172.21.15.19:0/2663489363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14dc0d37b0 msgr2=0x7f14dc0b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:23.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.819+0000 7f14e1400640 1 --2- 172.21.15.19:0/2663489363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14dc0d37b0 0x7f14dc0b62b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f14c40097a0 tx=0x7f14c402b8b0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.819+0000 7f14e1400640 1 -- 172.21.15.19:0/2663489363 shutdown_connections 2024-03-28T13:21:23.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.819+0000 7f14e1400640 1 --2- 172.21.15.19:0/2663489363 >> v2:172.21.15.188:3300/0 conn(0x7f14dc0b67f0 0x7f14dc0b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.819+0000 7f14e1400640 1 --2- 172.21.15.19:0/2663489363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14dc0d37b0 0x7f14dc0b62b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.819+0000 7f14e1400640 1 -- 172.21.15.19:0/2663489363 >> 172.21.15.19:0/2663489363 conn(0x7f14dc0676a0 msgr2=0x7f14dc067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:23.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.820+0000 7f14e1400640 1 -- 172.21.15.19:0/2663489363 shutdown_connections 2024-03-28T13:21:23.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.820+0000 7f14e1400640 1 -- 172.21.15.19:0/2663489363 wait complete. 2024-03-28T13:21:23.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.821+0000 7f14e1400640 1 Processor -- start 2024-03-28T13:21:23.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.821+0000 7f14e1400640 1 -- start start 2024-03-28T13:21:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.822+0000 7f14e1400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14dc0b67f0 0x7f14dc15a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.822+0000 7f14e1400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14dc0d37b0 0x7f14dc15c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.822+0000 7f14e1400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14dc15e4c0 0x7f14dc15ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.822+0000 7f14e1400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc0d2fb0 con 0x7f14dc0b67f0 2024-03-28T13:21:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.822+0000 7f14e1400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc0d3320 con 0x7f14dc0d37b0 2024-03-28T13:21:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.822+0000 7f14e1400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f14dc0d2350 con 0x7f14dc15e4c0 2024-03-28T13:21:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.823+0000 7f14dbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14dc0b67f0 0x7f14dc15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.823+0000 7f14dbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14dc0b67f0 0x7f14dc15a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52810/0 (socket says 172.21.15.19:52810) 2024-03-28T13:21:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.823+0000 7f14dbe00640 1 -- 172.21.15.19:0/4206016345 learned_addr learned my addr 172.21.15.19:0/4206016345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.823+0000 7f14d3e00640 1 --2- 172.21.15.19:0/4206016345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14dc0d37b0 0x7f14dc15c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.823+0000 7f14e0a00640 1 --2- 172.21.15.19:0/4206016345 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14dc15e4c0 0x7f14dc15ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.823+0000 7f14dbe00640 1 -- 172.21.15.19:0/4206016345 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14dc15e4c0 msgr2=0x7f14dc15ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.823+0000 7f14dbe00640 1 --2- 172.21.15.19:0/4206016345 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14dc15e4c0 0x7f14dc15ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.823+0000 7f14dbe00640 1 -- 172.21.15.19:0/4206016345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14dc0d37b0 msgr2=0x7f14dc15c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.823+0000 7f14dbe00640 1 --2- 172.21.15.19:0/4206016345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14dc0d37b0 0x7f14dc15c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.823+0000 7f14dbe00640 1 -- 172.21.15.19:0/4206016345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc1e7a90 con 0x7f14dc0b67f0 2024-03-28T13:21:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.824+0000 7f14dbe00640 1 --2- 172.21.15.19:0/4206016345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14dc0b67f0 0x7f14dc15a260 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00c400 tx=0x7f14cc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.824+0000 7f14d8c00640 1 -- 172.21.15.19:0/4206016345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc017070 con 0x7f14dc0b67f0 2024-03-28T13:21:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.824+0000 7f14d8c00640 1 -- 172.21.15.19:0/4206016345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14cc0052d0 con 0x7f14dc0b67f0 2024-03-28T13:21:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.824+0000 7f14d8c00640 1 -- 172.21.15.19:0/4206016345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc003830 con 0x7f14dc0b67f0 2024-03-28T13:21:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.824+0000 7f14e1400640 1 -- 172.21.15.19:0/4206016345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14dc1e7d70 con 0x7f14dc0b67f0 2024-03-28T13:21:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.824+0000 7f14e1400640 1 -- 172.21.15.19:0/4206016345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14dc1e8330 con 0x7f14dc0b67f0 2024-03-28T13:21:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.825+0000 7f14e1400640 1 -- 172.21.15.19:0/4206016345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14dc15dcf0 con 0x7f14dc0b67f0 2024-03-28T13:21:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.827+0000 7f14d8c00640 1 -- 172.21.15.19:0/4206016345 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f14cc01a070 con 0x7f14dc0b67f0 2024-03-28T13:21:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.828+0000 7f14d8c00640 1 --2- 172.21.15.19:0/4206016345 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f14b806c530 0x7f14b806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.828+0000 7f14d8c00640 1 -- 172.21.15.19:0/4206016345 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7f14cc08ab70 con 0x7f14dc0b67f0 2024-03-28T13:21:23.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.829+0000 7f14d3e00640 1 --2- 172.21.15.19:0/4206016345 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f14b806c530 0x7f14b806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:23.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.830+0000 7f14d3e00640 1 --2- 172.21.15.19:0/4206016345 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f14b806c530 0x7f14b806e9f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f14c4036000 tx=0x7f14c4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:23.833+0000 7f14d8c00640 1 -- 172.21.15.19:0/4206016345 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f14dc15dcf0 con 0x7f14dc0b67f0 2024-03-28T13:21:24.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.035+0000 7f14e1400640 1 -- 172.21.15.19:0/4206016345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f14dc0d0830 con 0x7f14dc0b67f0 2024-03-28T13:21:24.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.036+0000 7f14d8c00640 1 -- 172.21.15.19:0/4206016345 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v120) ==== 100+0+11 (secure 0 0 0) 0x7f14dc0d0830 con 0x7f14dc0b67f0 2024-03-28T13:21:24.036 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:21:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.038+0000 7f14e1400640 1 -- 172.21.15.19:0/4206016345 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f14b806c530 msgr2=0x7f14b806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.038+0000 7f14e1400640 1 --2- 172.21.15.19:0/4206016345 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f14b806c530 0x7f14b806e9f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f14c4036000 tx=0x7f14c4008030 comp rx=0 tx=0).stop 2024-03-28T13:21:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.038+0000 7f14e1400640 1 -- 172.21.15.19:0/4206016345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14dc0b67f0 msgr2=0x7f14dc15a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.038+0000 7f14e1400640 1 --2- 172.21.15.19:0/4206016345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14dc0b67f0 0x7f14dc15a260 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00c400 tx=0x7f14cc00c8c0 comp rx=0 tx=0).stop 2024-03-28T13:21:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.039+0000 7f14e1400640 1 -- 172.21.15.19:0/4206016345 shutdown_connections 2024-03-28T13:21:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.039+0000 7f14e1400640 1 --2- 172.21.15.19:0/4206016345 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f14b806c530 0x7f14b806e9f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.039+0000 7f14e1400640 1 --2- 172.21.15.19:0/4206016345 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14dc15e4c0 0x7f14dc15ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.039+0000 7f14e1400640 1 --2- 172.21.15.19:0/4206016345 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14dc0d37b0 0x7f14dc15c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.039+0000 7f14e1400640 1 --2- 172.21.15.19:0/4206016345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14dc0b67f0 0x7f14dc15a260 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.039+0000 7f14e1400640 1 -- 172.21.15.19:0/4206016345 >> 172.21.15.19:0/4206016345 conn(0x7f14dc0676a0 msgr2=0x7f14dc15b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.039+0000 7f14e1400640 1 -- 172.21.15.19:0/4206016345 shutdown_connections 2024-03-28T13:21:24.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.039+0000 7f14e1400640 1 -- 172.21.15.19:0/4206016345 wait complete. 2024-03-28T13:21:24.059 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:21:24.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.281+0000 7fe07c800640 1 Processor -- start 2024-03-28T13:21:24.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.281+0000 7fe07c800640 1 -- start start 2024-03-28T13:21:24.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.282+0000 7fe07c800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe07415e860 0x7fe07406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:24.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.282+0000 7fe07c800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe07406da30 0x7fe0740d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:24.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.282+0000 7fe07c800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe07406dfa0 con 0x7fe0740d4910 2024-03-28T13:21:24.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.282+0000 7fe07c800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fe0740d4d60 con 0x7fe07415e860 2024-03-28T13:21:24.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.282+0000 7fe07c800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe0740cf220 con 0x7fe07406da30 2024-03-28T13:21:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.283+0000 7fe079e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe07415e860 0x7fe07406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.283+0000 7fe079e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fe07415e860 0x7fe07406d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52820/0 (socket says 172.21.15.19:52820) 2024-03-28T13:21:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.283+0000 7fe079e00640 1 -- 172.21.15.19:0/1975434841 learned_addr learned my addr 172.21.15.19:0/1975434841 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.283+0000 7fe079e00640 1 -- 172.21.15.19:0/1975434841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe07406da30 msgr2=0x7fe0740d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.283+0000 7fe079400640 1 --2- 172.21.15.19:0/1975434841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe07406da30 0x7fe0740d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:24.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.283+0000 7fe079e00640 1 --2- 172.21.15.19:0/1975434841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe07406da30 0x7fe0740d43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.283+0000 7fe079e00640 1 -- 172.21.15.19:0/1975434841 >> v1:172.21.15.188:6789/0 conn(0x7fe0740d4910 legacy=0x7fe0740d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.283+0000 7fe079e00640 1 -- 172.21.15.19:0/1975434841 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07406e240 con 0x7fe07415e860 2024-03-28T13:21:24.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.285+0000 7fe079e00640 1 --2- 172.21.15.19:0/1975434841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe07415e860 0x7fe07406d4f0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009e50 tx=0x7fe06c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583cd2d6217a169d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.285+0000 7fe078a00640 1 -- 172.21.15.19:0/1975434841 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c02ca70 con 0x7fe07415e860 2024-03-28T13:21:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.285+0000 7fe078a00640 1 -- 172.21.15.19:0/1975434841 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe06c02cc10 con 0x7fe07415e860 2024-03-28T13:21:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.285+0000 7fe078a00640 1 -- 172.21.15.19:0/1975434841 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c0343f0 con 0x7fe07415e860 2024-03-28T13:21:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.286+0000 7fe07c800640 1 -- 172.21.15.19:0/1975434841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe07415e860 msgr2=0x7fe07406d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.286+0000 7fe07c800640 1 --2- 172.21.15.19:0/1975434841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe07415e860 0x7fe07406d4f0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009e50 tx=0x7fe06c02bed0 comp rx=0 tx=0).stop 2024-03-28T13:21:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.286+0000 7fe07c800640 1 -- 172.21.15.19:0/1975434841 shutdown_connections 2024-03-28T13:21:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.286+0000 7fe07c800640 1 --2- 172.21.15.19:0/1975434841 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe07406da30 0x7fe0740d43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.286+0000 7fe07c800640 1 --2- 172.21.15.19:0/1975434841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe07415e860 0x7fe07406d4f0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.286+0000 7fe07c800640 1 -- 172.21.15.19:0/1975434841 >> 172.21.15.19:0/1975434841 conn(0x7fe07409d7d0 msgr2=0x7fe07408bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:24.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.286+0000 7fe07c800640 1 -- 172.21.15.19:0/1975434841 shutdown_connections 2024-03-28T13:21:24.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.286+0000 7fe07c800640 1 -- 172.21.15.19:0/1975434841 wait complete. 2024-03-28T13:21:24.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.288+0000 7fe07c800640 1 Processor -- start 2024-03-28T13:21:24.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.288+0000 7fe07c800640 1 -- start start 2024-03-28T13:21:24.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.289+0000 7fe07c800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe07406da30 0x7fe07415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:24.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.289+0000 7fe07c800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0740d4910 0x7fe07415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:24.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.289+0000 7fe07c800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe07415e860 0x7fe07414d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:24.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.289+0000 7fe07c800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe0740cedb0 con 0x7fe07406da30 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.289+0000 7fe07c800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe0740d2c50 con 0x7fe07415e860 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.289+0000 7fe07c800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe0740cef30 con 0x7fe0740d4910 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.289+0000 7fe079e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe07406da30 0x7fe07415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.289+0000 7fe079e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe07406da30 0x7fe07415fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52828/0 (socket says 172.21.15.19:52828) 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.289+0000 7fe079e00640 1 -- 172.21.15.19:0/2094800080 learned_addr learned my addr 172.21.15.19:0/2094800080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.289+0000 7fe079400640 1 --2- 172.21.15.19:0/2094800080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0740d4910 0x7fe07415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.289+0000 7fe07a800640 1 --2- 172.21.15.19:0/2094800080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe07415e860 0x7fe07414d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.290+0000 7fe079e00640 1 -- 172.21.15.19:0/2094800080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0740d4910 msgr2=0x7fe07415a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.290+0000 7fe079e00640 1 --2- 172.21.15.19:0/2094800080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0740d4910 0x7fe07415a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.290+0000 7fe079e00640 1 -- 172.21.15.19:0/2094800080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe07415e860 msgr2=0x7fe07414d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.290+0000 7fe079e00640 1 --2- 172.21.15.19:0/2094800080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe07415e860 0x7fe07414d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.290+0000 7fe079e00640 1 -- 172.21.15.19:0/2094800080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe074160450 con 0x7fe07406da30 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.290+0000 7fe079e00640 1 --2- 172.21.15.19:0/2094800080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe07406da30 0x7fe07415fd70 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fe06c036040 tx=0x7fe06c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.290+0000 7fe06a000640 1 -- 172.21.15.19:0/2094800080 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c043020 con 0x7fe07406da30 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.290+0000 7fe06a000640 1 -- 172.21.15.19:0/2094800080 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe06c03e450 con 0x7fe07406da30 2024-03-28T13:21:24.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.290+0000 7fe06a000640 1 -- 172.21.15.19:0/2094800080 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c03e840 con 0x7fe07406da30 2024-03-28T13:21:24.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.291+0000 7fe07c800640 1 -- 172.21.15.19:0/2094800080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe074160730 con 0x7fe07406da30 2024-03-28T13:21:24.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.291+0000 7fe07c800640 1 -- 172.21.15.19:0/2094800080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe074160cf0 con 0x7fe07406da30 2024-03-28T13:21:24.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.292+0000 7fe07c800640 1 -- 172.21.15.19:0/2094800080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0740685c0 con 0x7fe07406da30 2024-03-28T13:21:24.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.293+0000 7fe06a000640 1 -- 172.21.15.19:0/2094800080 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe06c046020 con 0x7fe07406da30 2024-03-28T13:21:24.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.294+0000 7fe06a000640 1 --2- 172.21.15.19:0/2094800080 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe05806c530 0x7fe05806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:24.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.294+0000 7fe06a000640 1 -- 172.21.15.19:0/2094800080 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5795+0+0 (secure 0 0 0) 0x7fe06c0adff0 con 0x7fe07406da30 2024-03-28T13:21:24.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.294+0000 7fe079400640 1 --2- 172.21.15.19:0/2094800080 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe05806c530 0x7fe05806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:24.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.296+0000 7fe079400640 1 --2- 172.21.15.19:0/2094800080 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe05806c530 0x7fe05806e9f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe0740b5410 tx=0x7fe064008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:24.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.298+0000 7fe06a000640 1 -- 172.21.15.19:0/2094800080 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe06c04d030 con 0x7fe07406da30 2024-03-28T13:21:24.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.552+0000 7fe07c800640 1 -- 172.21.15.19:0/2094800080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe07415ac50 con 0x7fe07406da30 2024-03-28T13:21:24.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.553+0000 7fe06a000640 1 -- 172.21.15.19:0/2094800080 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v120) ==== 80+0+124 (secure 0 0 0) 0x7fe06c07c5e0 con 0x7fe07406da30 2024-03-28T13:21:24.553 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.188:6836/2088538359 2024-03-29T13:21:12.469733+0000 2024-03-28T13:21:24.553 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.188:6834/2088538359 2024-03-29T13:21:12.469733+0000 2024-03-28T13:21:24.554 INFO:teuthology.orchestra.run.smithi019.stderr:listed 2 entries 2024-03-28T13:21:24.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.555+0000 7fe07c800640 1 -- 172.21.15.19:0/2094800080 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe05806c530 msgr2=0x7fe05806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:24.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.555+0000 7fe07c800640 1 --2- 172.21.15.19:0/2094800080 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe05806c530 0x7fe05806e9f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fe0740b5410 tx=0x7fe064008040 comp rx=0 tx=0).stop 2024-03-28T13:21:24.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.555+0000 7fe07c800640 1 -- 172.21.15.19:0/2094800080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe07406da30 msgr2=0x7fe07415fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:24.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.555+0000 7fe07c800640 1 --2- 172.21.15.19:0/2094800080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe07406da30 0x7fe07415fd70 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fe06c036040 tx=0x7fe06c004a40 comp rx=0 tx=0).stop 2024-03-28T13:21:24.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.556+0000 7fe07c800640 1 -- 172.21.15.19:0/2094800080 shutdown_connections 2024-03-28T13:21:24.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.556+0000 7fe07c800640 1 --2- 172.21.15.19:0/2094800080 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe05806c530 0x7fe05806e9f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:24.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.556+0000 7fe07c800640 1 --2- 172.21.15.19:0/2094800080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe07415e860 0x7fe07414d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:24.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.556+0000 7fe07c800640 1 --2- 172.21.15.19:0/2094800080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0740d4910 0x7fe07415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:24.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.556+0000 7fe07c800640 1 --2- 172.21.15.19:0/2094800080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe07406da30 0x7fe07415fd70 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:24.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.556+0000 7fe07c800640 1 -- 172.21.15.19:0/2094800080 >> 172.21.15.19:0/2094800080 conn(0x7fe07409d7d0 msgr2=0x7fe0740b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:24.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.556+0000 7fe07c800640 1 -- 172.21.15.19:0/2094800080 shutdown_connections 2024-03-28T13:21:24.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:24.556+0000 7fe07c800640 1 -- 172.21.15.19:0/2094800080 wait complete. 2024-03-28T13:21:24.575 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:21:24.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:24.576 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:24.690 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:21:24.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:24.690 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:21:24.738 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:21:25.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.007+0000 7fcb46800640 1 Processor -- start 2024-03-28T13:21:25.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.007+0000 7fcb46800640 1 -- start start 2024-03-28T13:21:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.009+0000 7fcb46800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcb400b3ac0 0x7fcb400aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.009+0000 7fcb46800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb4006cc90 0x7fcb400ad890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.009+0000 7fcb46800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb400b3f50 con 0x7fcb4015e560 2024-03-28T13:21:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.009+0000 7fcb46800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fcb4015f950 con 0x7fcb400b3ac0 2024-03-28T13:21:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.009+0000 7fcb46800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcb40162930 con 0x7fcb4006cc90 2024-03-28T13:21:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.009+0000 7fcb44a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcb400b3ac0 0x7fcb400aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.009+0000 7fcb44a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fcb400b3ac0 0x7fcb400aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52830/0 (socket says 172.21.15.19:52830) 2024-03-28T13:21:25.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.009+0000 7fcb44a00640 1 -- 172.21.15.19:0/2113897955 learned_addr learned my addr 172.21.15.19:0/2113897955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:25.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.010+0000 7fcb44a00640 1 -- 172.21.15.19:0/2113897955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb4006cc90 msgr2=0x7fcb400ad890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:25.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.010+0000 7fcb44a00640 1 --2- 172.21.15.19:0/2113897955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb4006cc90 0x7fcb400ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.010+0000 7fcb44a00640 1 -- 172.21.15.19:0/2113897955 >> v1:172.21.15.188:6789/0 conn(0x7fcb4015e560 legacy=0x7fcb4006c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:25.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.010+0000 7fcb44a00640 1 -- 172.21.15.19:0/2113897955 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb40162000 con 0x7fcb400b3ac0 2024-03-28T13:21:25.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.011+0000 7fcb44a00640 1 --2- 172.21.15.19:0/2113897955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb400b3ac0 0x7fcb400aff80 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fcb400ce7e0 tx=0x7fcb3002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80aa8af87694aca server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:25.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.011+0000 7fcb3fe00640 1 -- 172.21.15.19:0/2113897955 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb30035020 con 0x7fcb400b3ac0 2024-03-28T13:21:25.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.011+0000 7fcb3fe00640 1 -- 172.21.15.19:0/2113897955 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb30031870 con 0x7fcb400b3ac0 2024-03-28T13:21:25.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.011+0000 7fcb3fe00640 1 -- 172.21.15.19:0/2113897955 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb30031c40 con 0x7fcb400b3ac0 2024-03-28T13:21:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.012+0000 7fcb46800640 1 -- 172.21.15.19:0/2113897955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb400b3ac0 msgr2=0x7fcb400aff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.012+0000 7fcb46800640 1 --2- 172.21.15.19:0/2113897955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb400b3ac0 0x7fcb400aff80 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fcb400ce7e0 tx=0x7fcb3002b910 comp rx=0 tx=0).stop 2024-03-28T13:21:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.012+0000 7fcb46800640 1 -- 172.21.15.19:0/2113897955 shutdown_connections 2024-03-28T13:21:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.012+0000 7fcb46800640 1 --2- 172.21.15.19:0/2113897955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb400b3ac0 0x7fcb400aff80 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.012+0000 7fcb46800640 1 --2- 172.21.15.19:0/2113897955 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb4006cc90 0x7fcb400ad890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.012+0000 7fcb46800640 1 -- 172.21.15.19:0/2113897955 >> 172.21.15.19:0/2113897955 conn(0x7fcb40071e70 msgr2=0x7fcb40072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.012+0000 7fcb46800640 1 -- 172.21.15.19:0/2113897955 shutdown_connections 2024-03-28T13:21:25.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.012+0000 7fcb46800640 1 -- 172.21.15.19:0/2113897955 wait complete. 2024-03-28T13:21:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.013+0000 7fcb46800640 1 Processor -- start 2024-03-28T13:21:25.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.014+0000 7fcb46800640 1 -- start start 2024-03-28T13:21:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.014+0000 7fcb46800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb4006cc90 0x7fcb400cfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.015+0000 7fcb46800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb4015e560 0x7fcb40153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.015+0000 7fcb46800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb4015fad0 0x7fcb40147dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.015+0000 7fcb46800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcb40162dc0 con 0x7fcb4015fad0 2024-03-28T13:21:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.015+0000 7fcb46800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcb401625c0 con 0x7fcb4006cc90 2024-03-28T13:21:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.015+0000 7fcb46800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcb40160be0 con 0x7fcb4015e560 2024-03-28T13:21:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.015+0000 7fcb45e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb4015fad0 0x7fcb40147dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.015+0000 7fcb45e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb4015fad0 0x7fcb40147dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52846/0 (socket says 172.21.15.19:52846) 2024-03-28T13:21:25.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.015+0000 7fcb45e00640 1 -- 172.21.15.19:0/184596826 learned_addr learned my addr 172.21.15.19:0/184596826 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:25.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.015+0000 7fcb45e00640 1 -- 172.21.15.19:0/184596826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb4015e560 msgr2=0x7fcb40153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.015+0000 7fcb45400640 1 --2- 172.21.15.19:0/184596826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb4006cc90 0x7fcb400cfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.015+0000 7fcb44a00640 1 --2- 172.21.15.19:0/184596826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb4015e560 0x7fcb40153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.015+0000 7fcb45e00640 1 --2- 172.21.15.19:0/184596826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb4015e560 0x7fcb40153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.015+0000 7fcb45e00640 1 -- 172.21.15.19:0/184596826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb4006cc90 msgr2=0x7fcb400cfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.015+0000 7fcb45e00640 1 --2- 172.21.15.19:0/184596826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb4006cc90 0x7fcb400cfbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.015+0000 7fcb45e00640 1 -- 172.21.15.19:0/184596826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4015ffd0 con 0x7fcb4015fad0 2024-03-28T13:21:25.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.016+0000 7fcb45e00640 1 --2- 172.21.15.19:0/184596826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb4015fad0 0x7fcb40147dd0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800b9e0 tx=0x7fcb3800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:25.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.016+0000 7fcb3d600640 1 -- 172.21.15.19:0/184596826 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38019070 con 0x7fcb4015fad0 2024-03-28T13:21:25.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.016+0000 7fcb3d600640 1 -- 172.21.15.19:0/184596826 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3800cb40 con 0x7fcb4015fad0 2024-03-28T13:21:25.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.016+0000 7fcb46800640 1 -- 172.21.15.19:0/184596826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb401602b0 con 0x7fcb4015fad0 2024-03-28T13:21:25.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.016+0000 7fcb3d600640 1 -- 172.21.15.19:0/184596826 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38014440 con 0x7fcb4015fad0 2024-03-28T13:21:25.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.016+0000 7fcb46800640 1 -- 172.21.15.19:0/184596826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb40160870 con 0x7fcb4015fad0 2024-03-28T13:21:25.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.017+0000 7fcb46800640 1 -- 172.21.15.19:0/184596826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb40075940 con 0x7fcb4015fad0 2024-03-28T13:21:25.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.018+0000 7fcb3d600640 1 -- 172.21.15.19:0/184596826 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcb3801c070 con 0x7fcb4015fad0 2024-03-28T13:21:25.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.020+0000 7fcb3d600640 1 --2- 172.21.15.19:0/184596826 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcb2406c530 0x7fcb2406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.020+0000 7fcb3d600640 1 -- 172.21.15.19:0/184596826 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5853+0+0 (secure 0 0 0) 0x7fcb3808b400 con 0x7fcb4015fad0 2024-03-28T13:21:25.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.020+0000 7fcb45400640 1 --2- 172.21.15.19:0/184596826 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcb2406c530 0x7fcb2406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.021+0000 7fcb45400640 1 --2- 172.21.15.19:0/184596826 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcb2406c530 0x7fcb2406e9f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400a730 tx=0x7fcb34008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:25.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.029+0000 7fcb3d600640 1 -- 172.21.15.19:0/184596826 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fcb38059980 con 0x7fcb4015fad0 2024-03-28T13:21:25.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.274+0000 7fcb46800640 1 -- 172.21.15.19:0/184596826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb40067860 con 0x7fcb4015fad0 2024-03-28T13:21:25.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.275+0000 7fcb3d600640 1 -- 172.21.15.19:0/184596826 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5187 (secure 0 0 0) 0x7fcb38059280 con 0x7fcb4015fad0 2024-03-28T13:21:25.275 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:25.275 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":105,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3823671990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3823671990},{"type":"v1","addr":"172.21.15.188:6837","nonce":3823671990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/22158043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":22158043},{"type":"v1","addr":"172.21.15.188:6836","nonce":22158043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":8688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/230519675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":230519675},{"type":"v1","addr":"172.21.15.19:6834","nonce":230519675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:20:52.901269+0000","modified":"2024-03-28T13:21:16.616377+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":120,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8691},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8691":{"gid":8691,"name":"a","rank":0,"incarnation":102,"state":"up:active","state_seq":10,"addr":"172.21.15.19:6835/3930967142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":3930967142},{"type":"v1","addr":"172.21.15.19:6835","nonce":3930967142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8691,"qdb_cluster":[8691]},"id":8}]} 2024-03-28T13:21:25.275 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 105 2024-03-28T13:21:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.277+0000 7fcb46800640 1 -- 172.21.15.19:0/184596826 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcb2406c530 msgr2=0x7fcb2406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.277+0000 7fcb46800640 1 --2- 172.21.15.19:0/184596826 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcb2406c530 0x7fcb2406e9f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400a730 tx=0x7fcb34008040 comp rx=0 tx=0).stop 2024-03-28T13:21:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.278+0000 7fcb46800640 1 -- 172.21.15.19:0/184596826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb4015fad0 msgr2=0x7fcb40147dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.278+0000 7fcb46800640 1 --2- 172.21.15.19:0/184596826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb4015fad0 0x7fcb40147dd0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800b9e0 tx=0x7fcb3800bea0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.278+0000 7fcb46800640 1 -- 172.21.15.19:0/184596826 shutdown_connections 2024-03-28T13:21:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.278+0000 7fcb46800640 1 --2- 172.21.15.19:0/184596826 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fcb2406c530 0x7fcb2406e9f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.278+0000 7fcb46800640 1 --2- 172.21.15.19:0/184596826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcb4015fad0 0x7fcb40147dd0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.278+0000 7fcb46800640 1 --2- 172.21.15.19:0/184596826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb4015e560 0x7fcb40153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.278+0000 7fcb46800640 1 --2- 172.21.15.19:0/184596826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb4006cc90 0x7fcb400cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.278+0000 7fcb46800640 1 -- 172.21.15.19:0/184596826 >> 172.21.15.19:0/184596826 conn(0x7fcb40071e70 msgr2=0x7fcb4006b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.278+0000 7fcb46800640 1 -- 172.21.15.19:0/184596826 shutdown_connections 2024-03-28T13:21:25.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.278+0000 7fcb46800640 1 -- 172.21.15.19:0/184596826 wait complete. 2024-03-28T13:21:25.296 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parent mytag123 2024-03-28T13:21:25.546 INFO:teuthology.orchestra.run.smithi019.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-03-28T13:21:25.546 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-03-28T13:21:25.546 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-03-28T13:21:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.641+0000 7f6ea7236f40 1 Processor -- start 2024-03-28T13:21:25.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.641+0000 7f6ea7236f40 1 -- start start 2024-03-28T13:21:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.642+0000 7f6ea7236f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e4d8fcd0 0x5575e4dd7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.642+0000 7f6ea7236f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5575e4d944b0 0x5575e4dda8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.642+0000 7f6ea7236f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575e4e68ba0 0x5575e4dbb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.642+0000 7f6ea7236f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5575e4dbfd10 con 0x5575e4d8fcd0 2024-03-28T13:21:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.642+0000 7f6ea7236f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5575e4d900e0 con 0x5575e4e68ba0 2024-03-28T13:21:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.642+0000 7f6ea7236f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5575e4dbf1a0 con 0x5575e4d944b0 2024-03-28T13:21:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.643+0000 7f6ea3c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e4d8fcd0 0x5575e4dd7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.643+0000 7f6ea3c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e4d8fcd0 0x5575e4dd7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42136/0 (socket says 172.21.15.19:42136) 2024-03-28T13:21:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.643+0000 7f6ea3c00640 1 -- 172.21.15.19:0/1973540380 learned_addr learned my addr 172.21.15.19:0/1973540380 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.643+0000 7f6ea3c00640 1 -- 172.21.15.19:0/1973540380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5575e4d944b0 msgr2=0x5575e4dda8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.643+0000 7f6ea3c00640 1 --2- 172.21.15.19:0/1973540380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5575e4d944b0 0x5575e4dda8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.643+0000 7f6ea3c00640 1 -- 172.21.15.19:0/1973540380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575e4e68ba0 msgr2=0x5575e4dbb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.643+0000 7f6ea3c00640 1 --2- 172.21.15.19:0/1973540380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575e4e68ba0 0x5575e4dbb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.643+0000 7f6ea3c00640 1 -- 172.21.15.19:0/1973540380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e4cd44b0 con 0x5575e4d8fcd0 2024-03-28T13:21:25.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.643+0000 7f6ea3c00640 1 --2- 172.21.15.19:0/1973540380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e4d8fcd0 0x5575e4dd7560 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400a7a0 tx=0x7f6e9400a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f51eacc78fca5f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.644+0000 7f6ea2800640 1 -- 172.21.15.19:0/1973540380 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e94019070 con 0x5575e4d8fcd0 2024-03-28T13:21:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.644+0000 7f6ea2800640 1 -- 172.21.15.19:0/1973540380 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e94007600 con 0x5575e4d8fcd0 2024-03-28T13:21:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.644+0000 7f6ea2800640 1 -- 172.21.15.19:0/1973540380 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e940079b0 con 0x5575e4d8fcd0 2024-03-28T13:21:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.644+0000 7f6ea7236f40 1 -- 172.21.15.19:0/1973540380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e4d8fcd0 msgr2=0x5575e4dd7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.644+0000 7f6ea7236f40 1 --2- 172.21.15.19:0/1973540380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e4d8fcd0 0x5575e4dd7560 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400a7a0 tx=0x7f6e9400a890 comp rx=0 tx=0).stop 2024-03-28T13:21:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.644+0000 7f6ea7236f40 1 -- 172.21.15.19:0/1973540380 shutdown_connections 2024-03-28T13:21:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.644+0000 7f6ea7236f40 1 --2- 172.21.15.19:0/1973540380 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575e4e68ba0 0x5575e4dbb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.644+0000 7f6ea7236f40 1 --2- 172.21.15.19:0/1973540380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5575e4d944b0 0x5575e4dda8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.644+0000 7f6ea7236f40 1 --2- 172.21.15.19:0/1973540380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e4d8fcd0 0x5575e4dd7560 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.644+0000 7f6ea7236f40 1 -- 172.21.15.19:0/1973540380 >> 172.21.15.19:0/1973540380 conn(0x5575e4d908a0 msgr2=0x5575e4e65790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:25.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.645+0000 7f6ea7236f40 1 -- 172.21.15.19:0/1973540380 shutdown_connections 2024-03-28T13:21:25.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.645+0000 7f6ea7236f40 1 -- 172.21.15.19:0/1973540380 wait complete. 2024-03-28T13:21:25.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.645+0000 7f6ea7236f40 1 Processor -- start 2024-03-28T13:21:25.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.645+0000 7f6ea7236f40 1 -- start start 2024-03-28T13:21:25.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.646+0000 7f6ea7236f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5575e4d8fcd0 0x5575e4e609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.646+0000 7f6ea7236f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e4d944b0 0x5575e4e6a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.646+0000 7f6ea7236f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575e4e68ba0 0x5575e4e56da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.646+0000 7f6ea7236f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5575e4dd9860 con 0x5575e4d944b0 2024-03-28T13:21:25.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.646+0000 7f6ea7236f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5575e4dbd270 con 0x5575e4e68ba0 2024-03-28T13:21:25.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.646+0000 7f6ea7236f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5575e4ddae10 con 0x5575e4d8fcd0 2024-03-28T13:21:25.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.647+0000 7f6e9be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e4d944b0 0x5575e4e6a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.647+0000 7f6e9be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e4d944b0 0x5575e4e6a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42144/0 (socket says 172.21.15.19:42144) 2024-03-28T13:21:25.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.647+0000 7f6e9be00640 1 -- 172.21.15.19:0/3066590211 learned_addr learned my addr 172.21.15.19:0/3066590211 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:25.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.647+0000 7f6ea4600640 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575e4e68ba0 0x5575e4e56da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.647+0000 7f6ea3c00640 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5575e4d8fcd0 0x5575e4e609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.647+0000 7f6e9be00640 1 -- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5575e4d8fcd0 msgr2=0x5575e4e609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.647+0000 7f6e9be00640 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5575e4d8fcd0 0x5575e4e609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.647+0000 7f6e9be00640 1 -- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575e4e68ba0 msgr2=0x5575e4e56da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.647+0000 7f6e9be00640 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575e4e68ba0 0x5575e4e56da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.647+0000 7f6e9be00640 1 -- 172.21.15.19:0/3066590211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e4e6ac30 con 0x5575e4d944b0 2024-03-28T13:21:25.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.647+0000 7f6ea3c00640 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5575e4d8fcd0 0x5575e4e609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:25.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.647+0000 7f6ea4600640 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5575e4e68ba0 0x5575e4e56da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:25.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.647+0000 7f6e9be00640 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e4d944b0 0x5575e4e6a6f0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c055040 tx=0x7f6e8c052650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:25.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.648+0000 7f6ea1400640 1 -- 172.21.15.19:0/3066590211 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c060020 con 0x5575e4d944b0 2024-03-28T13:21:25.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.648+0000 7f6ea1400640 1 -- 172.21.15.19:0/3066590211 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e8c058040 con 0x5575e4d944b0 2024-03-28T13:21:25.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.648+0000 7f6ea1400640 1 -- 172.21.15.19:0/3066590211 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c057070 con 0x5575e4d944b0 2024-03-28T13:21:25.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.648+0000 7f6ea7236f40 1 -- 172.21.15.19:0/3066590211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5575e4e69230 con 0x5575e4d944b0 2024-03-28T13:21:25.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.648+0000 7f6ea7236f40 1 -- 172.21.15.19:0/3066590211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e4dd7ce0 con 0x5575e4d944b0 2024-03-28T13:21:25.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.650+0000 7f6ea1400640 1 -- 172.21.15.19:0/3066590211 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6e8c063020 con 0x5575e4d944b0 2024-03-28T13:21:25.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.651+0000 7f6ea1400640 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6e8006af10 0x7f6e8006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.651+0000 7f6ea1400640 1 -- 172.21.15.19:0/3066590211 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5853+0+0 (secure 0 0 0) 0x7f6e8c0cb870 con 0x5575e4d944b0 2024-03-28T13:21:25.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.651+0000 7f6ea7236f40 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5575e4e572e0 0x5575e4dbbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.651+0000 7f6ea7236f40 1 -- 172.21.15.19:0/3066590211 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:1 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e121) -- 0x5575e4e5bdb0 con 0x5575e4e572e0 2024-03-28T13:21:25.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.652+0000 7f6ea3c00640 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6e8006af10 0x7f6e8006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.652+0000 7f6ea4600640 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5575e4e572e0 0x5575e4dbbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.653+0000 7f6ea4600640 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5575e4e572e0 0x5575e4dbbce0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:25.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.653+0000 7f6ea3c00640 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6e8006af10 0x7f6e8006d3d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400a8d0 tx=0x7f6e9400c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:25.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.654+0000 7f6ea4600640 1 -- 172.21.15.19:0/3066590211 <== osd.7 v2:172.21.15.188:6824/861796513 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f6e9c00d530 con 0x5575e4e572e0 2024-03-28T13:21:25.655 INFO:teuthology.orchestra.run.smithi019.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-03-28T13:21:25.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.654+0000 7f6ea7236f40 1 -- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5575e4e572e0 msgr2=0x5575e4dbbce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.654+0000 7f6ea7236f40 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5575e4e572e0 0x5575e4dbbce0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.654+0000 7f6ea7236f40 1 -- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6e8006af10 msgr2=0x7f6e8006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.654+0000 7f6ea7236f40 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6e8006af10 0x7f6e8006d3d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400a8d0 tx=0x7f6e9400c000 comp rx=0 tx=0).stop 2024-03-28T13:21:25.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.654+0000 7f6ea7236f40 1 -- 172.21.15.19:0/3066590211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e4d944b0 msgr2=0x5575e4e6a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.654+0000 7f6ea7236f40 1 --2- 172.21.15.19:0/3066590211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5575e4d944b0 0x5575e4e6a6f0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c055040 tx=0x7f6e8c052650 comp rx=0 tx=0).stop 2024-03-28T13:21:25.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.654+0000 7f6ea4600640 1 -- 172.21.15.19:0/3066590211 reap_dead start 2024-03-28T13:21:25.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.655+0000 7f6ea7236f40 1 -- 172.21.15.19:0/3066590211 shutdown_connections 2024-03-28T13:21:25.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.655+0000 7f6ea7236f40 1 -- 172.21.15.19:0/3066590211 >> 172.21.15.19:0/3066590211 conn(0x5575e4d908a0 msgr2=0x5575e4e614c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:25.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.655+0000 7f6ea7236f40 1 -- 172.21.15.19:0/3066590211 shutdown_connections 2024-03-28T13:21:25.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.655+0000 7f6ea7236f40 1 -- 172.21.15.19:0/3066590211 wait complete. 2024-03-28T13:21:25.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T13:21:25.659 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-03-28T13:21:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.755+0000 7fcce07fef40 1 Processor -- start 2024-03-28T13:21:25.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.756+0000 7fcce07fef40 1 -- start start 2024-03-28T13:21:25.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.756+0000 7fcce07fef40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56096e8024c0 0x56096e889420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.756+0000 7fcce07fef40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56096e803b60 0x56096e892d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.756+0000 7fcce07fef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56096e88fad0 0x56096e7e6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.756+0000 7fcce07fef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56096e802cb0 con 0x56096e88fad0 2024-03-28T13:21:25.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.757+0000 7fcce07fef40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56096e7612c0 con 0x56096e8024c0 2024-03-28T13:21:25.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.757+0000 7fcce07fef40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x56096e802f50 con 0x56096e803b60 2024-03-28T13:21:25.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.757+0000 7fccde400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56096e88fad0 0x56096e7e6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.757+0000 7fccde400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56096e88fad0 0x56096e7e6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42162/0 (socket says 172.21.15.19:42162) 2024-03-28T13:21:25.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.757+0000 7fccde400640 1 -- 172.21.15.19:0/3455826451 learned_addr learned my addr 172.21.15.19:0/3455826451 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:25.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.757+0000 7fccdda00640 1 --2- 172.21.15.19:0/3455826451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56096e8024c0 0x56096e889420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.757+0000 7fccdd000640 1 --2- 172.21.15.19:0/3455826451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56096e803b60 0x56096e892d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.757+0000 7fccde400640 1 -- 172.21.15.19:0/3455826451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56096e803b60 msgr2=0x56096e892d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.757+0000 7fccde400640 1 --2- 172.21.15.19:0/3455826451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56096e803b60 0x56096e892d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.757+0000 7fccde400640 1 -- 172.21.15.19:0/3455826451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56096e8024c0 msgr2=0x56096e889420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.757+0000 7fccde400640 1 --2- 172.21.15.19:0/3455826451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56096e8024c0 0x56096e889420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.757+0000 7fccde400640 1 -- 172.21.15.19:0/3455826451 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56096e88ffc0 con 0x56096e88fad0 2024-03-28T13:21:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.757+0000 7fccdda00640 1 --2- 172.21.15.19:0/3455826451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56096e8024c0 0x56096e889420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.757+0000 7fccdd000640 1 --2- 172.21.15.19:0/3455826451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56096e803b60 0x56096e892d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.758+0000 7fccde400640 1 --2- 172.21.15.19:0/3455826451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56096e88fad0 0x56096e7e6ea0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fccd800a560 tx=0x7fccd804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57059fe108744a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.758+0000 7fccd7400640 1 -- 172.21.15.19:0/3455826451 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8057040 con 0x56096e88fad0 2024-03-28T13:21:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.758+0000 7fccd7400640 1 -- 172.21.15.19:0/3455826451 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccd8062070 con 0x56096e88fad0 2024-03-28T13:21:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.758+0000 7fccd7400640 1 -- 172.21.15.19:0/3455826451 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8052b60 con 0x56096e88fad0 2024-03-28T13:21:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.758+0000 7fcce07fef40 1 -- 172.21.15.19:0/3455826451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56096e88fad0 msgr2=0x56096e7e6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.758+0000 7fcce07fef40 1 --2- 172.21.15.19:0/3455826451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56096e88fad0 0x56096e7e6ea0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fccd800a560 tx=0x7fccd804c000 comp rx=0 tx=0).stop 2024-03-28T13:21:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.758+0000 7fcce07fef40 1 -- 172.21.15.19:0/3455826451 shutdown_connections 2024-03-28T13:21:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.758+0000 7fcce07fef40 1 --2- 172.21.15.19:0/3455826451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56096e88fad0 0x56096e7e6ea0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.758+0000 7fcce07fef40 1 --2- 172.21.15.19:0/3455826451 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56096e803b60 0x56096e892d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.759+0000 7fcce07fef40 1 --2- 172.21.15.19:0/3455826451 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56096e8024c0 0x56096e889420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.759+0000 7fcce07fef40 1 -- 172.21.15.19:0/3455826451 >> 172.21.15.19:0/3455826451 conn(0x56096e7e5cf0 msgr2=0x56096e890960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.759+0000 7fcce07fef40 1 -- 172.21.15.19:0/3455826451 shutdown_connections 2024-03-28T13:21:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.759+0000 7fcce07fef40 1 -- 172.21.15.19:0/3455826451 wait complete. 2024-03-28T13:21:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.760+0000 7fcce07fef40 1 Processor -- start 2024-03-28T13:21:25.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.760+0000 7fcce07fef40 1 -- start start 2024-03-28T13:21:25.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.762+0000 7fcce07fef40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56096e8024c0 0x56096e88c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.762+0000 7fcce07fef40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56096e803b60 0x56096e892ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.762+0000 7fcce07fef40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56096e88fad0 0x56096e928510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.762+0000 7fcce07fef40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56096e899dc0 con 0x56096e803b60 2024-03-28T13:21:25.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.762+0000 7fcce07fef40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56096e8902a0 con 0x56096e8024c0 2024-03-28T13:21:25.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.762+0000 7fcce07fef40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x56096e89a0a0 con 0x56096e88fad0 2024-03-28T13:21:25.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.762+0000 7fccdd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56096e803b60 0x56096e892ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.762+0000 7fccdd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56096e803b60 0x56096e892ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42164/0 (socket says 172.21.15.19:42164) 2024-03-28T13:21:25.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.762+0000 7fccdd000640 1 -- 172.21.15.19:0/4156566105 learned_addr learned my addr 172.21.15.19:0/4156566105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:25.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.762+0000 7fccdda00640 1 --2- 172.21.15.19:0/4156566105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56096e8024c0 0x56096e88c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.762+0000 7fccdd000640 1 -- 172.21.15.19:0/4156566105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56096e88fad0 msgr2=0x56096e928510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.762+0000 7fccdd000640 1 --2- 172.21.15.19:0/4156566105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56096e88fad0 0x56096e928510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.762+0000 7fccdd000640 1 -- 172.21.15.19:0/4156566105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56096e8024c0 msgr2=0x56096e88c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.762+0000 7fccdd000640 1 --2- 172.21.15.19:0/4156566105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56096e8024c0 0x56096e88c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.762+0000 7fccdd000640 1 -- 172.21.15.19:0/4156566105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56096e891710 con 0x56096e803b60 2024-03-28T13:21:25.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.763+0000 7fccdd000640 1 --2- 172.21.15.19:0/4156566105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56096e803b60 0x56096e892ee0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fcccc007e70 tx=0x7fcccc006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:25.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.763+0000 7fccd6000640 1 -- 172.21.15.19:0/4156566105 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc010710 con 0x56096e803b60 2024-03-28T13:21:25.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.763+0000 7fccd6000640 1 -- 172.21.15.19:0/4156566105 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcccc0108b0 con 0x56096e803b60 2024-03-28T13:21:25.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.763+0000 7fcce07fef40 1 -- 172.21.15.19:0/4156566105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56096e88be90 con 0x56096e803b60 2024-03-28T13:21:25.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.763+0000 7fccd6000640 1 -- 172.21.15.19:0/4156566105 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcccc010c80 con 0x56096e803b60 2024-03-28T13:21:25.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.763+0000 7fcce07fef40 1 -- 172.21.15.19:0/4156566105 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56096e891050 con 0x56096e803b60 2024-03-28T13:21:25.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.765+0000 7fccd6000640 1 -- 172.21.15.19:0/4156566105 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fcccc012400 con 0x56096e803b60 2024-03-28T13:21:25.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.766+0000 7fccd6000640 1 --2- 172.21.15.19:0/4156566105 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fccc006af10 0x7fccc006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.766+0000 7fccd6000640 1 -- 172.21.15.19:0/4156566105 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5853+0+0 (secure 0 0 0) 0x7fcccc089ba0 con 0x56096e803b60 2024-03-28T13:21:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.766+0000 7fcce07fef40 1 --2- 172.21.15.19:0/4156566105 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x56096e893420 0x56096e886570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.767+0000 7fcce07fef40 1 -- 172.21.15.19:0/4156566105 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:1 17.12 17:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) -- 0x56096e8992d0 con 0x56096e893420 2024-03-28T13:21:25.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.767+0000 7fccdda00640 1 --2- 172.21.15.19:0/4156566105 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fccc006af10 0x7fccc006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.767+0000 7fccde400640 1 --2- 172.21.15.19:0/4156566105 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x56096e893420 0x56096e886570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.768+0000 7fccdda00640 1 --2- 172.21.15.19:0/4156566105 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fccc006af10 0x7fccc006d3d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x56096e7e5c70 tx=0x7fccd00014a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:25.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.768+0000 7fccde400640 1 --2- 172.21.15.19:0/4156566105 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x56096e893420 0x56096e886570 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:25.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.774+0000 7fccde400640 1 -- 172.21.15.19:0/4156566105 <== osd.7 v2:172.21.15.188:6824/861796513 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fccd805a040 con 0x56096e893420 2024-03-28T13:21:25.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.774+0000 7fcce07fef40 1 -- 172.21.15.19:0/4156566105 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x56096e893420 msgr2=0x56096e886570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.774+0000 7fcce07fef40 1 --2- 172.21.15.19:0/4156566105 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x56096e893420 0x56096e886570 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.774+0000 7fcce07fef40 1 -- 172.21.15.19:0/4156566105 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fccc006af10 msgr2=0x7fccc006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.774+0000 7fcce07fef40 1 --2- 172.21.15.19:0/4156566105 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fccc006af10 0x7fccc006d3d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x56096e7e5c70 tx=0x7fccd00014a0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.774+0000 7fcce07fef40 1 -- 172.21.15.19:0/4156566105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56096e803b60 msgr2=0x56096e892ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.774+0000 7fcce07fef40 1 --2- 172.21.15.19:0/4156566105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56096e803b60 0x56096e892ee0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fcccc007e70 tx=0x7fcccc006750 comp rx=0 tx=0).stop 2024-03-28T13:21:25.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.774+0000 7fccde400640 1 -- 172.21.15.19:0/4156566105 reap_dead start 2024-03-28T13:21:25.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.774+0000 7fcce07fef40 1 -- 172.21.15.19:0/4156566105 shutdown_connections 2024-03-28T13:21:25.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.774+0000 7fcce07fef40 1 -- 172.21.15.19:0/4156566105 >> 172.21.15.19:0/4156566105 conn(0x56096e7e5cf0 msgr2=0x56096e7bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:25.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.774+0000 7fcce07fef40 1 -- 172.21.15.19:0/4156566105 shutdown_connections 2024-03-28T13:21:25.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.774+0000 7fcce07fef40 1 -- 172.21.15.19:0/4156566105 wait complete. 2024-03-28T13:21:25.779 INFO:teuthology.orchestra.run.smithi019.stdout:mytag123 2024-03-28T13:21:25.779 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-03-28T13:21:25.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.877+0000 7fc9ff246f40 1 Processor -- start 2024-03-28T13:21:25.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.877+0000 7fc9ff246f40 1 -- start start 2024-03-28T13:21:25.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.878+0000 7fc9ff246f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1289424c0 0x55e128927120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.878+0000 7fc9ff246f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e128943b60 0x55e1288fa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.878+0000 7fc9ff246f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1289cfad0 0x55e1288fa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.878+0000 7fc9ff246f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e1288a12c0 con 0x55e128943b60 2024-03-28T13:21:25.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.878+0000 7fc9ff246f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e1288a5310 con 0x55e1289cfad0 2024-03-28T13:21:25.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.878+0000 7fc9ff246f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55e128942f50 con 0x55e1289424c0 2024-03-28T13:21:25.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.879+0000 7fc9fb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e128943b60 0x55e1288fa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.879+0000 7fc9fb200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e128943b60 0x55e1288fa220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42170/0 (socket says 172.21.15.19:42170) 2024-03-28T13:21:25.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.879+0000 7fc9fb200640 1 -- 172.21.15.19:0/2173581029 learned_addr learned my addr 172.21.15.19:0/2173581029 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:25.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.879+0000 7fc9fc600640 1 --2- 172.21.15.19:0/2173581029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1289cfad0 0x55e1288fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.879+0000 7fc9fb200640 1 -- 172.21.15.19:0/2173581029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1289424c0 msgr2=0x55e128927120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:25.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.879+0000 7fc9fbc00640 1 --2- 172.21.15.19:0/2173581029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1289424c0 0x55e128927120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.879+0000 7fc9fb200640 1 --2- 172.21.15.19:0/2173581029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1289424c0 0x55e128927120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.879+0000 7fc9fb200640 1 -- 172.21.15.19:0/2173581029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1289cfad0 msgr2=0x55e1288fa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:25.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.879+0000 7fc9fb200640 1 --2- 172.21.15.19:0/2173581029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1289cfad0 0x55e1288fa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.879+0000 7fc9fb200640 1 -- 172.21.15.19:0/2173581029 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1288fbe50 con 0x55e128943b60 2024-03-28T13:21:25.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.879+0000 7fc9fb200640 1 --2- 172.21.15.19:0/2173581029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e128943b60 0x55e1288fa220 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fc9e801b880 tx=0x7fc9e804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a456e5c9f8bda7c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:25.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.880+0000 7fc9f9e00640 1 -- 172.21.15.19:0/2173581029 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8057070 con 0x55e128943b60 2024-03-28T13:21:25.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.880+0000 7fc9f9e00640 1 -- 172.21.15.19:0/2173581029 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e8058040 con 0x55e128943b60 2024-03-28T13:21:25.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.880+0000 7fc9f9e00640 1 -- 172.21.15.19:0/2173581029 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e804aa10 con 0x55e128943b60 2024-03-28T13:21:25.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.880+0000 7fc9ff246f40 1 -- 172.21.15.19:0/2173581029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e128943b60 msgr2=0x55e1288fa220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.880+0000 7fc9ff246f40 1 --2- 172.21.15.19:0/2173581029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e128943b60 0x55e1288fa220 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fc9e801b880 tx=0x7fc9e804c000 comp rx=0 tx=0).stop 2024-03-28T13:21:25.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.880+0000 7fc9ff246f40 1 -- 172.21.15.19:0/2173581029 shutdown_connections 2024-03-28T13:21:25.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.880+0000 7fc9ff246f40 1 --2- 172.21.15.19:0/2173581029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e1289cfad0 0x55e1288fa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.880+0000 7fc9ff246f40 1 --2- 172.21.15.19:0/2173581029 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e128943b60 0x55e1288fa220 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.880+0000 7fc9ff246f40 1 --2- 172.21.15.19:0/2173581029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1289424c0 0x55e128927120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.880+0000 7fc9ff246f40 1 -- 172.21.15.19:0/2173581029 >> 172.21.15.19:0/2173581029 conn(0x55e128925cf0 msgr2=0x55e1289406b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:25.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.880+0000 7fc9ff246f40 1 -- 172.21.15.19:0/2173581029 shutdown_connections 2024-03-28T13:21:25.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.881+0000 7fc9ff246f40 1 -- 172.21.15.19:0/2173581029 wait complete. 2024-03-28T13:21:25.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.881+0000 7fc9ff246f40 1 Processor -- start 2024-03-28T13:21:25.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.881+0000 7fc9ff246f40 1 -- start start 2024-03-28T13:21:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.882+0000 7fc9ff246f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1289424c0 0x55e1289c2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.882+0000 7fc9ff246f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e128943b60 0x55e1289cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.882+0000 7fc9ff246f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1289cfad0 0x55e1289dab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.882+0000 7fc9ff246f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e128940dc0 con 0x55e1289cfad0 2024-03-28T13:21:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.882+0000 7fc9ff246f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55e1289456a0 con 0x55e128943b60 2024-03-28T13:21:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.882+0000 7fc9ff246f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55e1289c4eb0 con 0x55e1289424c0 2024-03-28T13:21:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.883+0000 7fc9fc600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1289cfad0 0x55e1289dab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.883+0000 7fc9fc600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1289cfad0 0x55e1289dab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42182/0 (socket says 172.21.15.19:42182) 2024-03-28T13:21:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.883+0000 7fc9fc600640 1 -- 172.21.15.19:0/908167233 learned_addr learned my addr 172.21.15.19:0/908167233 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.883+0000 7fc9fbc00640 1 --2- 172.21.15.19:0/908167233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1289424c0 0x55e1289c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.883+0000 7fc9fc600640 1 -- 172.21.15.19:0/908167233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1289424c0 msgr2=0x55e1289c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.883+0000 7fc9fc600640 1 --2- 172.21.15.19:0/908167233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e1289424c0 0x55e1289c2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.883+0000 7fc9fb200640 1 --2- 172.21.15.19:0/908167233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e128943b60 0x55e1289cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.883+0000 7fc9fc600640 1 -- 172.21.15.19:0/908167233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e128943b60 msgr2=0x55e1289cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.883+0000 7fc9fc600640 1 --2- 172.21.15.19:0/908167233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e128943b60 0x55e1289cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.883+0000 7fc9fc600640 1 -- 172.21.15.19:0/908167233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1289c6ef0 con 0x55e1289cfad0 2024-03-28T13:21:25.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.883+0000 7fc9fc600640 1 --2- 172.21.15.19:0/908167233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1289cfad0 0x55e1289dab50 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4004240 tx=0x7fc9f400c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:25.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.884+0000 7fc9f8a00640 1 -- 172.21.15.19:0/908167233 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4017070 con 0x55e1289cfad0 2024-03-28T13:21:25.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.884+0000 7fc9f8a00640 1 -- 172.21.15.19:0/908167233 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9f40032f0 con 0x55e1289cfad0 2024-03-28T13:21:25.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.884+0000 7fc9f8a00640 1 -- 172.21.15.19:0/908167233 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4001870 con 0x55e1289cfad0 2024-03-28T13:21:25.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.884+0000 7fc9ff246f40 1 -- 172.21.15.19:0/908167233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e1289c7e50 con 0x55e1289cfad0 2024-03-28T13:21:25.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.884+0000 7fc9ff246f40 1 -- 172.21.15.19:0/908167233 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1289c7b20 con 0x55e1289cfad0 2024-03-28T13:21:25.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.887+0000 7fc9f8a00640 1 -- 172.21.15.19:0/908167233 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc9f401a070 con 0x55e1289cfad0 2024-03-28T13:21:25.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.888+0000 7fc9f8a00640 1 --2- 172.21.15.19:0/908167233 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc9dc06af10 0x7fc9dc06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.888+0000 7fc9f8a00640 1 -- 172.21.15.19:0/908167233 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5853+0+0 (secure 0 0 0) 0x7fc9f4089ef0 con 0x55e1289cfad0 2024-03-28T13:21:25.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.889+0000 7fc9ff246f40 1 --2- 172.21.15.19:0/908167233 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x55e1289409e0 0x55e1289ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:25.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.889+0000 7fc9ff246f40 1 -- 172.21.15.19:0/908167233 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:1 17.19 17:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) -- 0x55e1289c7210 con 0x55e1289409e0 2024-03-28T13:21:25.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.889+0000 7fc9fbc00640 1 --2- 172.21.15.19:0/908167233 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc9dc06af10 0x7fc9dc06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.889+0000 7fc9fb200640 1 --2- 172.21.15.19:0/908167233 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x55e1289409e0 0x55e1289ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:25.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.890+0000 7fc9fb200640 1 --2- 172.21.15.19:0/908167233 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x55e1289409e0 0x55e1289ceb40 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:25.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.890+0000 7fc9fbc00640 1 --2- 172.21.15.19:0/908167233 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc9dc06af10 0x7fc9dc06d3d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0011e0 tx=0x7fc9ec0015c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:25.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.890+0000 7fc9fb200640 1 -- 172.21.15.19:0/908167233 <== osd.3 v2:172.21.15.19:6819/3265515865 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fc9e805a070 con 0x55e1289409e0 2024-03-28T13:21:25.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.890+0000 7fc9ff246f40 1 -- 172.21.15.19:0/908167233 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x55e1289409e0 msgr2=0x55e1289ceb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.890+0000 7fc9ff246f40 1 --2- 172.21.15.19:0/908167233 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x55e1289409e0 0x55e1289ceb40 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.890+0000 7fc9ff246f40 1 -- 172.21.15.19:0/908167233 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc9dc06af10 msgr2=0x7fc9dc06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.890+0000 7fc9ff246f40 1 --2- 172.21.15.19:0/908167233 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc9dc06af10 0x7fc9dc06d3d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0011e0 tx=0x7fc9ec0015c0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.890+0000 7fc9ff246f40 1 -- 172.21.15.19:0/908167233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1289cfad0 msgr2=0x55e1289dab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:25.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.890+0000 7fc9ff246f40 1 --2- 172.21.15.19:0/908167233 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e1289cfad0 0x55e1289dab50 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4004240 tx=0x7fc9f400c5e0 comp rx=0 tx=0).stop 2024-03-28T13:21:25.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.890+0000 7fc9fc600640 1 -- 172.21.15.19:0/908167233 reap_dead start 2024-03-28T13:21:25.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.890+0000 7fc9ff246f40 1 -- 172.21.15.19:0/908167233 shutdown_connections 2024-03-28T13:21:25.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.890+0000 7fc9ff246f40 1 -- 172.21.15.19:0/908167233 >> 172.21.15.19:0/908167233 conn(0x55e128925cf0 msgr2=0x55e1288fb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:25.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.891+0000 7fc9ff246f40 1 -- 172.21.15.19:0/908167233 shutdown_connections 2024-03-28T13:21:25.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:25.891+0000 7fc9ff246f40 1 -- 172.21.15.19:0/908167233 wait complete. 2024-03-28T13:21:25.895 INFO:teuthology.orchestra.run.smithi019.stdout:mytag123 2024-03-28T13:21:25.895 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-03-28T13:21:25.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.000+0000 7f0498863f40 1 Processor -- start 2024-03-28T13:21:26.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.000+0000 7f0498863f40 1 -- start start 2024-03-28T13:21:26.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.001+0000 7f0498863f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56281c35a240 0x56281c350e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.001+0000 7f0498863f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56281c360270 0x56281c2b9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.001+0000 7f0498863f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56281c360d80 0x56281c2bade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.001+0000 7f0498863f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56281c2d5980 con 0x56281c360270 2024-03-28T13:21:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.001+0000 7f0498863f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56281c22f2c0 con 0x56281c360d80 2024-03-28T13:21:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.001+0000 7f0498863f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x56281c2d3160 con 0x56281c35a240 2024-03-28T13:21:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.001+0000 7f0495000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56281c360270 0x56281c2b9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.001+0000 7f0495000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56281c360270 0x56281c2b9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42194/0 (socket says 172.21.15.19:42194) 2024-03-28T13:21:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.001+0000 7f0495000640 1 -- 172.21.15.19:0/3718522567 learned_addr learned my addr 172.21.15.19:0/3718522567 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.001+0000 7f0495a00640 1 --2- 172.21.15.19:0/3718522567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56281c35a240 0x56281c350e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.001+0000 7f0496400640 1 --2- 172.21.15.19:0/3718522567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56281c360d80 0x56281c2bade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.001+0000 7f0495000640 1 -- 172.21.15.19:0/3718522567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56281c35a240 msgr2=0x56281c350e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:26.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.001+0000 7f0495000640 1 --2- 172.21.15.19:0/3718522567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56281c35a240 0x56281c350e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.001+0000 7f0495000640 1 -- 172.21.15.19:0/3718522567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56281c360d80 msgr2=0x56281c2bade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.001+0000 7f0495000640 1 --2- 172.21.15.19:0/3718522567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56281c360d80 0x56281c2bade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.002+0000 7f0495000640 1 -- 172.21.15.19:0/3718522567 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56281c34e800 con 0x56281c360270 2024-03-28T13:21:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.002+0000 7f0495a00640 1 --2- 172.21.15.19:0/3718522567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56281c35a240 0x56281c350e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.002+0000 7f0496400640 1 --2- 172.21.15.19:0/3718522567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56281c360d80 0x56281c2bade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.002+0000 7f0495000640 1 --2- 172.21.15.19:0/3718522567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56281c360270 0x56281c2b9a20 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f048400b310 tx=0x7f048400b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744e9a3857a53d61 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.002+0000 7f048f400640 1 -- 172.21.15.19:0/3718522567 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484008070 con 0x56281c360270 2024-03-28T13:21:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.002+0000 7f048f400640 1 -- 172.21.15.19:0/3718522567 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f048400d500 con 0x56281c360270 2024-03-28T13:21:26.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.002+0000 7f048f400640 1 -- 172.21.15.19:0/3718522567 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048400d8b0 con 0x56281c360270 2024-03-28T13:21:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.003+0000 7f0498863f40 1 -- 172.21.15.19:0/3718522567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56281c360270 msgr2=0x56281c2b9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.003+0000 7f0498863f40 1 --2- 172.21.15.19:0/3718522567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56281c360270 0x56281c2b9a20 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f048400b310 tx=0x7f048400b7d0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.003+0000 7f0498863f40 1 -- 172.21.15.19:0/3718522567 shutdown_connections 2024-03-28T13:21:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.003+0000 7f0498863f40 1 --2- 172.21.15.19:0/3718522567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56281c360d80 0x56281c2bade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.003+0000 7f0498863f40 1 --2- 172.21.15.19:0/3718522567 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56281c360270 0x56281c2b9a20 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.003+0000 7f0498863f40 1 --2- 172.21.15.19:0/3718522567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56281c35a240 0x56281c350e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.003+0000 7f0498863f40 1 -- 172.21.15.19:0/3718522567 >> 172.21.15.19:0/3718522567 conn(0x56281c2d6630 msgr2=0x56281c2b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.003+0000 7f0498863f40 1 -- 172.21.15.19:0/3718522567 shutdown_connections 2024-03-28T13:21:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.003+0000 7f0498863f40 1 -- 172.21.15.19:0/3718522567 wait complete. 2024-03-28T13:21:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.004+0000 7f0498863f40 1 Processor -- start 2024-03-28T13:21:26.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.004+0000 7f0498863f40 1 -- start start 2024-03-28T13:21:26.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.005+0000 7f0498863f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56281c35a240 0x56281c361fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.005+0000 7f0498863f40 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56281c360270 0x56281c357a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.005+0000 7f0498863f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56281c360d80 0x56281c3489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.005+0000 7f0498863f40 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56281c3513b0 con 0x56281c360270 2024-03-28T13:21:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.005+0000 7f0498863f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x56281c2b73b0 con 0x56281c360d80 2024-03-28T13:21:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.005+0000 7f0498863f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x56281c34a5d0 con 0x56281c35a240 2024-03-28T13:21:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.005+0000 7f0495000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56281c360270 0x56281c357a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.005+0000 7f0495000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56281c360270 0x56281c357a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42204/0 (socket says 172.21.15.19:42204) 2024-03-28T13:21:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.005+0000 7f0495000640 1 -- 172.21.15.19:0/130397506 learned_addr learned my addr 172.21.15.19:0/130397506 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.006+0000 7f0495a00640 1 --2- 172.21.15.19:0/130397506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56281c35a240 0x56281c361fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.006+0000 7f0496400640 1 --2- 172.21.15.19:0/130397506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56281c360d80 0x56281c3489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.006+0000 7f0495000640 1 -- 172.21.15.19:0/130397506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56281c35a240 msgr2=0x56281c361fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.006+0000 7f0495000640 1 --2- 172.21.15.19:0/130397506 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x56281c35a240 0x56281c361fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.006+0000 7f0495000640 1 -- 172.21.15.19:0/130397506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56281c360d80 msgr2=0x56281c3489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.006+0000 7f0495000640 1 --2- 172.21.15.19:0/130397506 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x56281c360d80 0x56281c3489d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.006+0000 7f0495000640 1 -- 172.21.15.19:0/130397506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56281c349ca0 con 0x56281c360270 2024-03-28T13:21:26.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.006+0000 7f0495000640 1 --2- 172.21.15.19:0/130397506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56281c360270 0x56281c357a10 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f0484000c00 tx=0x7f0484000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:26.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.006+0000 7f048e000640 1 -- 172.21.15.19:0/130397506 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048401c650 con 0x56281c360270 2024-03-28T13:21:26.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.007+0000 7f048e000640 1 -- 172.21.15.19:0/130397506 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0484021020 con 0x56281c360270 2024-03-28T13:21:26.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.007+0000 7f048e000640 1 -- 172.21.15.19:0/130397506 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048401c830 con 0x56281c360270 2024-03-28T13:21:26.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.007+0000 7f0498863f40 1 -- 172.21.15.19:0/130397506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56281c349f80 con 0x56281c360270 2024-03-28T13:21:26.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.007+0000 7f0498863f40 1 -- 172.21.15.19:0/130397506 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56281c2b7020 con 0x56281c360270 2024-03-28T13:21:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.009+0000 7f048e000640 1 -- 172.21.15.19:0/130397506 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0484017070 con 0x56281c360270 2024-03-28T13:21:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.010+0000 7f048e000640 1 --2- 172.21.15.19:0/130397506 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f047806af40 0x7f047806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.010+0000 7f048e000640 1 -- 172.21.15.19:0/130397506 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5853+0+0 (secure 0 0 0) 0x7f048408bfc0 con 0x56281c360270 2024-03-28T13:21:26.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.011+0000 7f0498863f40 1 --2- 172.21.15.19:0/130397506 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x56281c362510 0x56281c3f92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.011+0000 7f0498863f40 1 -- 172.21.15.19:0/130397506 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:1 17.5 17:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) -- 0x56281c357f50 con 0x56281c362510 2024-03-28T13:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.011+0000 7f0495a00640 1 --2- 172.21.15.19:0/130397506 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f047806af40 0x7f047806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.011+0000 7f0496400640 1 --2- 172.21.15.19:0/130397506 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x56281c362510 0x56281c3f92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.012+0000 7f0496400640 1 --2- 172.21.15.19:0/130397506 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x56281c362510 0x56281c3f92a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.012+0000 7f0495a00640 1 --2- 172.21.15.19:0/130397506 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f047806af40 0x7f047806d400 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0488009660 tx=0x7f04880058c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.012+0000 7f0496400640 1 -- 172.21.15.19:0/130397506 <== osd.0 v2:172.21.15.19:6800/4106483280 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f0490052500 con 0x56281c362510 2024-03-28T13:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.012+0000 7f0498863f40 1 -- 172.21.15.19:0/130397506 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x56281c362510 msgr2=0x56281c3f92a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.012+0000 7f0498863f40 1 --2- 172.21.15.19:0/130397506 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x56281c362510 0x56281c3f92a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.012+0000 7f0498863f40 1 -- 172.21.15.19:0/130397506 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f047806af40 msgr2=0x7f047806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.012+0000 7f0498863f40 1 --2- 172.21.15.19:0/130397506 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f047806af40 0x7f047806d400 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0488009660 tx=0x7f04880058c0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.012+0000 7f0498863f40 1 -- 172.21.15.19:0/130397506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56281c360270 msgr2=0x56281c357a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.012+0000 7f0498863f40 1 --2- 172.21.15.19:0/130397506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56281c360270 0x56281c357a10 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f0484000c00 tx=0x7f0484000ed0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.013+0000 7f0496400640 1 -- 172.21.15.19:0/130397506 reap_dead start 2024-03-28T13:21:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.013+0000 7f0498863f40 1 -- 172.21.15.19:0/130397506 shutdown_connections 2024-03-28T13:21:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.013+0000 7f0498863f40 1 -- 172.21.15.19:0/130397506 >> 172.21.15.19:0/130397506 conn(0x56281c2d6630 msgr2=0x56281c2bade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.013+0000 7f0498863f40 1 -- 172.21.15.19:0/130397506 shutdown_connections 2024-03-28T13:21:26.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.013+0000 7f0498863f40 1 -- 172.21.15.19:0/130397506 wait complete. 2024-03-28T13:21:26.017 INFO:teuthology.orchestra.run.smithi019.stdout:mytag123 2024-03-28T13:21:26.018 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-03-28T13:21:26.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.218+0000 7fb07b200640 1 Processor -- start 2024-03-28T13:21:26.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.218+0000 7fb07b200640 1 -- start start 2024-03-28T13:21:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.220+0000 7fb07b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07415b4e0 0x7fb0740b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.220+0000 7fb07b200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb07415f700 0x7fb07406e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.220+0000 7fb07b200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb07415b8c0 con 0x7fb0740afee0 2024-03-28T13:21:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.220+0000 7fb07b200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb0740b5810 con 0x7fb07415f700 2024-03-28T13:21:26.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.220+0000 7fb07b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb0740afab0 con 0x7fb07415b4e0 2024-03-28T13:21:26.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.221+0000 7fb079200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb07415f700 0x7fb07406e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.221+0000 7fb073e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07415b4e0 0x7fb0740b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.221+0000 7fb073400640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fb0740afee0 0x7fb07415eff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:44478/0 (socket says 172.21.15.19:44478) 2024-03-28T13:21:26.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.221+0000 7fb073400640 1 -- 172.21.15.19:0/2886322897 learned_addr learned my addr 172.21.15.19:0/2886322897 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:26.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.222+0000 7fb079200640 1 -- 172.21.15.19:0/2886322897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07415b4e0 msgr2=0x7fb0740b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:26.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.222+0000 7fb079200640 1 --2- 172.21.15.19:0/2886322897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07415b4e0 0x7fb0740b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.222+0000 7fb079200640 1 -- 172.21.15.19:0/2886322897 >> v1:172.21.15.188:6789/0 conn(0x7fb0740afee0 legacy=0x7fb07415eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.222+0000 7fb079200640 1 -- 172.21.15.19:0/2886322897 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07415fd70 con 0x7fb07415f700 2024-03-28T13:21:26.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.222+0000 7fb073e00640 1 --2- 172.21.15.19:0/2886322897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07415b4e0 0x7fb0740b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:21:26.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.223+0000 7fb079200640 1 --2- 172.21.15.19:0/2886322897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb07415f700 0x7fb07406e750 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb06c00bfa0 tx=0x7fb06c02e0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fdacd5d0c5852fc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:26.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.224+0000 7fb072a00640 1 -- 172.21.15.19:0/2886322897 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c02ea40 con 0x7fb07415f700 2024-03-28T13:21:26.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.224+0000 7fb072a00640 1 -- 172.21.15.19:0/2886322897 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb06c02ebe0 con 0x7fb07415f700 2024-03-28T13:21:26.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.224+0000 7fb072a00640 1 -- 172.21.15.19:0/2886322897 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c009540 con 0x7fb07415f700 2024-03-28T13:21:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.225+0000 7fb07b200640 1 -- 172.21.15.19:0/2886322897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb07415f700 msgr2=0x7fb07406e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.225+0000 7fb07b200640 1 --2- 172.21.15.19:0/2886322897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb07415f700 0x7fb07406e750 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb06c00bfa0 tx=0x7fb06c02e0a0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.225+0000 7fb07b200640 1 -- 172.21.15.19:0/2886322897 shutdown_connections 2024-03-28T13:21:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.225+0000 7fb07b200640 1 --2- 172.21.15.19:0/2886322897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb07415f700 0x7fb07406e750 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.225+0000 7fb07b200640 1 --2- 172.21.15.19:0/2886322897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07415b4e0 0x7fb0740b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.225+0000 7fb07b200640 1 -- 172.21.15.19:0/2886322897 >> 172.21.15.19:0/2886322897 conn(0x7fb0740b6560 msgr2=0x7fb0740b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.225+0000 7fb07b200640 1 -- 172.21.15.19:0/2886322897 shutdown_connections 2024-03-28T13:21:26.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.225+0000 7fb07b200640 1 -- 172.21.15.19:0/2886322897 wait complete. 2024-03-28T13:21:26.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.227+0000 7fb07b200640 1 Processor -- start 2024-03-28T13:21:26.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.227+0000 7fb07b200640 1 -- start start 2024-03-28T13:21:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.228+0000 7fb07b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0740afee0 0x7fb074151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.228+0000 7fb07b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07415b4e0 0x7fb074145c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.228+0000 7fb07b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07415f700 0x7fb074153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.228+0000 7fb07b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb0740b5010 con 0x7fb07415f700 2024-03-28T13:21:26.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.228+0000 7fb07b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb0740b4d30 con 0x7fb0740afee0 2024-03-28T13:21:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.228+0000 7fb07b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb0740b52f0 con 0x7fb07415b4e0 2024-03-28T13:21:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.229+0000 7fb079200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07415f700 0x7fb074153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.229+0000 7fb079200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07415f700 0x7fb074153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42212/0 (socket says 172.21.15.19:42212) 2024-03-28T13:21:26.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.229+0000 7fb079200640 1 -- 172.21.15.19:0/2079005109 learned_addr learned my addr 172.21.15.19:0/2079005109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.229+0000 7fb073400640 1 --2- 172.21.15.19:0/2079005109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07415b4e0 0x7fb074145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.229+0000 7fb073e00640 1 --2- 172.21.15.19:0/2079005109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0740afee0 0x7fb074151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.229+0000 7fb079200640 1 -- 172.21.15.19:0/2079005109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07415b4e0 msgr2=0x7fb074145c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.229+0000 7fb079200640 1 --2- 172.21.15.19:0/2079005109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07415b4e0 0x7fb074145c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.229+0000 7fb079200640 1 -- 172.21.15.19:0/2079005109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0740afee0 msgr2=0x7fb074151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.229+0000 7fb079200640 1 --2- 172.21.15.19:0/2079005109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0740afee0 0x7fb074151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.229+0000 7fb079200640 1 -- 172.21.15.19:0/2079005109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb07414e8b0 con 0x7fb07415f700 2024-03-28T13:21:26.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.230+0000 7fb079200640 1 --2- 172.21.15.19:0/2079005109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07415f700 0x7fb074153280 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fb06c038040 tx=0x7fb06c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:26.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.230+0000 7fb06be00640 1 -- 172.21.15.19:0/2079005109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c044020 con 0x7fb07415f700 2024-03-28T13:21:26.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.230+0000 7fb06be00640 1 -- 172.21.15.19:0/2079005109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb06c03f660 con 0x7fb07415f700 2024-03-28T13:21:26.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.230+0000 7fb07b200640 1 -- 172.21.15.19:0/2079005109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb07414eb50 con 0x7fb07415f700 2024-03-28T13:21:26.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.230+0000 7fb06be00640 1 -- 172.21.15.19:0/2079005109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06c00b660 con 0x7fb07415f700 2024-03-28T13:21:26.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.230+0000 7fb07b200640 1 -- 172.21.15.19:0/2079005109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb07414f110 con 0x7fb07415f700 2024-03-28T13:21:26.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.234+0000 7fb06be00640 1 -- 172.21.15.19:0/2079005109 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb06c047020 con 0x7fb07415f700 2024-03-28T13:21:26.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.234+0000 7fb06be00640 1 --2- 172.21.15.19:0/2079005109 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb05406c530 0x7fb05406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.235+0000 7fb06be00640 1 -- 172.21.15.19:0/2079005109 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5853+0+0 (secure 0 0 0) 0x7fb06c0ae500 con 0x7fb07415f700 2024-03-28T13:21:26.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.235+0000 7fb07dfa6740 1 Processor -- start 2024-03-28T13:21:26.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.235+0000 7fb07dfa6740 1 -- start start 2024-03-28T13:21:26.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.235+0000 7fb073e00640 1 --2- 172.21.15.19:0/2079005109 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb05406c530 0x7fb05406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.236+0000 7fb07dfa6740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641560 0x557477614500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.236+0000 7fb07dfa6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641940 0x5574776442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.236+0000 7fb07dfa6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477614a40 0x557477644830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.236+0000 7fb07dfa6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55747757f6b0 con 0x557477641940 2024-03-28T13:21:26.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.236+0000 7fb07dfa6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x557477527750 con 0x557477614a40 2024-03-28T13:21:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.236+0000 7fb07dfa6740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55747757b660 con 0x557477641560 2024-03-28T13:21:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.236+0000 7fb073400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641940 0x5574776442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.236+0000 7fb073400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641940 0x5574776442f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42214/0 (socket says 172.21.15.19:42214) 2024-03-28T13:21:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.236+0000 7fb073400640 1 -- 172.21.15.19:0/3449088326 learned_addr learned my addr 172.21.15.19:0/3449088326 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.236+0000 7fb073e00640 1 --2- 172.21.15.19:0/2079005109 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb05406c530 0x7fb05406e9f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb064009b10 tx=0x7fb064008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.236+0000 7fb073e00640 1 --2- 172.21.15.19:0/3449088326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641560 0x557477614500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.236+0000 7fb073400640 1 -- 172.21.15.19:0/3449088326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641560 msgr2=0x557477614500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.236+0000 7fb079200640 1 --2- 172.21.15.19:0/3449088326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477614a40 0x557477644830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.237+0000 7fb073400640 1 --2- 172.21.15.19:0/3449088326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641560 0x557477614500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.237+0000 7fb073400640 1 -- 172.21.15.19:0/3449088326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477614a40 msgr2=0x557477644830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.237+0000 7fb073400640 1 --2- 172.21.15.19:0/3449088326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477614a40 0x557477644830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.237+0000 7fb073400640 1 -- 172.21.15.19:0/3449088326 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55747735d7d0 con 0x557477641940 2024-03-28T13:21:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.237+0000 7fb079200640 1 --2- 172.21.15.19:0/3449088326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477614a40 0x557477644830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:26.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.237+0000 7fb073400640 1 --2- 172.21.15.19:0/3449088326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641940 0x5574776442f0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fb0740b3d50 tx=0x7fb0600018f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9d3ebef2924b60 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.237+0000 7fb068c00640 1 -- 172.21.15.19:0/3449088326 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060019070 con 0x557477641940 2024-03-28T13:21:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.237+0000 7fb068c00640 1 -- 172.21.15.19:0/3449088326 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb060005ac0 con 0x557477641940 2024-03-28T13:21:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.237+0000 7fb068c00640 1 -- 172.21.15.19:0/3449088326 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0600070d0 con 0x557477641940 2024-03-28T13:21:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.238+0000 7fb07dfa6740 1 -- 172.21.15.19:0/3449088326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641940 msgr2=0x5574776442f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.238+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/3449088326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641940 0x5574776442f0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fb0740b3d50 tx=0x7fb0600018f0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.238+0000 7fb07dfa6740 1 -- 172.21.15.19:0/3449088326 shutdown_connections 2024-03-28T13:21:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.238+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/3449088326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477614a40 0x557477644830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.238+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/3449088326 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641940 0x5574776442f0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.238+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/3449088326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641560 0x557477614500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.238+0000 7fb07dfa6740 1 -- 172.21.15.19:0/3449088326 >> 172.21.15.19:0/3449088326 conn(0x557477616770 msgr2=0x55747765f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.238+0000 7fb07dfa6740 1 -- 172.21.15.19:0/3449088326 shutdown_connections 2024-03-28T13:21:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.238+0000 7fb07dfa6740 1 -- 172.21.15.19:0/3449088326 wait complete. 2024-03-28T13:21:26.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.239+0000 7fb07dfa6740 1 -- start start 2024-03-28T13:21:26.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.239+0000 7fb07dfa6740 1 Processor -- start 2024-03-28T13:21:26.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.240+0000 7fb07dfa6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641560 0x55747763d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.240+0000 7fb07dfa6740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641940 0x557477673730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.240+0000 7fb07dfa6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477641d40 0x557477619400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.240+0000 7fb07dfa6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557477618e10 con 0x557477641560 2024-03-28T13:21:26.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.240+0000 7fb07dfa6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55747753aa80 con 0x557477641d40 2024-03-28T13:21:26.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.240+0000 7fb07dfa6740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55747761b650 con 0x557477641940 2024-03-28T13:21:26.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.240+0000 7fb07dfa6740 10 client.0 _ll_register_callbacks cb 0x55747753a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:21:26.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.240+0000 7fb073e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641560 0x55747763d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.240+0000 7fb073e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641560 0x55747763d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42216/0 (socket says 172.21.15.19:42216) 2024-03-28T13:21:26.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.240+0000 7fb073e00640 1 -- 172.21.15.19:0/1196407270 learned_addr learned my addr 172.21.15.19:0/1196407270 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:26.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.241+0000 7fb073400640 1 --2- 172.21.15.19:0/1196407270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641940 0x557477673730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.241+0000 7fb079200640 1 --2- 172.21.15.19:0/1196407270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477641d40 0x557477619400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.241+0000 7fb073e00640 1 -- 172.21.15.19:0/1196407270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641940 msgr2=0x557477673730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:26.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.241+0000 7fb073e00640 1 --2- 172.21.15.19:0/1196407270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641940 0x557477673730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.241+0000 7fb073e00640 1 -- 172.21.15.19:0/1196407270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477641d40 msgr2=0x557477619400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:26.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.241+0000 7fb073e00640 1 --2- 172.21.15.19:0/1196407270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477641d40 0x557477619400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.241+0000 7fb073e00640 1 -- 172.21.15.19:0/1196407270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574772933f0 con 0x557477641560 2024-03-28T13:21:26.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.241+0000 7fb073e00640 1 --2- 172.21.15.19:0/1196407270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641560 0x55747763d410 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fb064015700 tx=0x7fb0640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2800a90566af586c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:26.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.241+0000 7fb05d600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:21:26.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.241+0000 7fb05d600640 1 -- 172.21.15.19:0/1196407270 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064025070 con 0x557477641560 2024-03-28T13:21:26.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.242+0000 7fb05d600640 1 -- 172.21.15.19:0/1196407270 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 5853+0+0 (secure 0 0 0) 0x7fb064004020 con 0x557477641560 2024-03-28T13:21:26.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.242+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1196407270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb060005ac0 con 0x557477641560 2024-03-28T13:21:26.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.242+0000 7fb05d600640 1 -- 172.21.15.19:0/1196407270 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7fb064004220 con 0x557477641560 2024-03-28T13:21:26.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.242+0000 7fb07dfa6740 10 client.9108 fetch_fsmap learned FSMap version 105 2024-03-28T13:21:26.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.242+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1196407270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x5574772530f0 con 0x557477641560 2024-03-28T13:21:26.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.242+0000 7fb05d600640 1 -- 172.21.15.19:0/1196407270 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb060005ac0 con 0x557477641560 2024-03-28T13:21:26.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.242+0000 7fb05d600640 1 -- 172.21.15.19:0/1196407270 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 105) ==== 3494+0+0 (secure 0 0 0) 0x7fb064014870 con 0x557477641560 2024-03-28T13:21:26.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.242+0000 7fb07dfa6740 10 client.9108 fetch_fsmap finished waiting for FSMap version 105 2024-03-28T13:21:26.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.242+0000 7fb07dfa6740 10 client.9108 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-03-28T13:21:26.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.242+0000 7fb07dfa6740 20 client.9108 populate_metadata read hostname 'smithi019' 2024-03-28T13:21:26.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.242+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/1196407270 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x557477645f20 0x557477646370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.243+0000 7fb07dfa6740 4 client.9108 mds_command: new command op to 8691 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:21:26.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.243+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1196407270 --> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55747758cbf0 con 0x557477645f20 2024-03-28T13:21:26.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.243+0000 7fb073400640 1 --2- 172.21.15.19:0/1196407270 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x557477645f20 0x557477646370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.243+0000 7fb073400640 1 --2- 172.21.15.19:0/1196407270 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x557477645f20 0x557477646370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51139ce178f41772 server_cookie=687abaea8121bab3 in_seq=0 out_seq=0 2024-03-28T13:21:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.243+0000 7fb05d600640 10 client.9108 ms_handle_connect on v2:172.21.15.19:6833/3930967142 2024-03-28T13:21:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.244+0000 7fb05d600640 1 -- 172.21.15.19:0/1196407270 <== mds.0 v2:172.21.15.19:6833/3930967142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55747758cbf0 con 0x557477645f20 2024-03-28T13:21:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.244+0000 7fb05d600640 10 client.9108 handle_command_reply: tid=0 2024-03-28T13:21:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.244+0000 7fb07dfa6740 1 client.9108 shutdown 2024-03-28T13:21:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.245+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1196407270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641560 msgr2=0x55747763d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.245+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/1196407270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641560 0x55747763d410 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fb064015700 tx=0x7fb0640046c0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.245+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1196407270 shutdown_connections 2024-03-28T13:21:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.245+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/1196407270 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x557477645f20 0x557477646370 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.245+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/1196407270 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477641d40 0x557477619400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.245+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/1196407270 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641940 0x557477673730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.245+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/1196407270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641560 0x55747763d410 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.245+0000 7fb05d600640 0 client.9108 ms_handle_reset on v2:172.21.15.19:6833/3930967142 2024-03-28T13:21:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.245+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1196407270 >> 172.21.15.19:0/1196407270 conn(0x557477614a40 msgr2=0x5574776184e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.245+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1196407270 shutdown_connections 2024-03-28T13:21:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.245+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1196407270 wait complete. 2024-03-28T13:21:26.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.245+0000 7fb07dfa6740 20 client.9108 trim_cache size 0 max 16384 2024-03-28T13:21:26.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.261+0000 7fb07dfa6740 1 Processor -- start 2024-03-28T13:21:26.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.262+0000 7fb07dfa6740 1 -- start start 2024-03-28T13:21:26.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.262+0000 7fb07dfa6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641560 0x557477662ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.262+0000 7fb07dfa6740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641940 0x557477643320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.262+0000 7fb07dfa6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477641d40 0x5574776d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.262+0000 7fb07dfa6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb074200de0 con 0x557477641560 2024-03-28T13:21:26.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.262+0000 7fb07dfa6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb0741ffe50 con 0x557477641d40 2024-03-28T13:21:26.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.262+0000 7fb07dfa6740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb074150500 con 0x557477641940 2024-03-28T13:21:26.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.263+0000 7fb073e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641560 0x557477662ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.263+0000 7fb073e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641560 0x557477662ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42232/0 (socket says 172.21.15.19:42232) 2024-03-28T13:21:26.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.263+0000 7fb073e00640 1 -- 172.21.15.19:0/4084429981 learned_addr learned my addr 172.21.15.19:0/4084429981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:26.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.263+0000 7fb073400640 1 --2- 172.21.15.19:0/4084429981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641940 0x557477643320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.263+0000 7fb079200640 1 --2- 172.21.15.19:0/4084429981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477641d40 0x5574776d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.263+0000 7fb073e00640 1 -- 172.21.15.19:0/4084429981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641940 msgr2=0x557477643320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:26.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.263+0000 7fb073e00640 1 --2- 172.21.15.19:0/4084429981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641940 0x557477643320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.263+0000 7fb073e00640 1 -- 172.21.15.19:0/4084429981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477641d40 msgr2=0x5574776d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:26.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.263+0000 7fb073e00640 1 --2- 172.21.15.19:0/4084429981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477641d40 0x5574776d0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.263+0000 7fb073e00640 1 -- 172.21.15.19:0/4084429981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557477617e30 con 0x557477641560 2024-03-28T13:21:26.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.263+0000 7fb073400640 1 --2- 172.21.15.19:0/4084429981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641940 0x557477643320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:26.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.264+0000 7fb073e00640 1 --2- 172.21.15.19:0/4084429981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641560 0x557477662ed0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fb0640098a0 tx=0x7fb064021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55cdf6e93317c60 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:26.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.264+0000 7fb05fe00640 1 -- 172.21.15.19:0/4084429981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064025070 con 0x557477641560 2024-03-28T13:21:26.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.264+0000 7fb05fe00640 1 -- 172.21.15.19:0/4084429981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb064038070 con 0x557477641560 2024-03-28T13:21:26.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.264+0000 7fb05fe00640 1 -- 172.21.15.19:0/4084429981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0640043c0 con 0x557477641560 2024-03-28T13:21:26.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.265+0000 7fb07dfa6740 1 -- 172.21.15.19:0/4084429981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641560 msgr2=0x557477662ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.265+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/4084429981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641560 0x557477662ed0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fb0640098a0 tx=0x7fb064021000 comp rx=0 tx=0).stop 2024-03-28T13:21:26.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.265+0000 7fb07dfa6740 1 -- 172.21.15.19:0/4084429981 shutdown_connections 2024-03-28T13:21:26.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.265+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/4084429981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477641d40 0x5574776d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.265+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/4084429981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641940 0x557477643320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.265+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/4084429981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641560 0x557477662ed0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.265+0000 7fb07dfa6740 1 -- 172.21.15.19:0/4084429981 >> 172.21.15.19:0/4084429981 conn(0x557477614a40 msgr2=0x557477618290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:26.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.265+0000 7fb07dfa6740 1 -- 172.21.15.19:0/4084429981 shutdown_connections 2024-03-28T13:21:26.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.265+0000 7fb07dfa6740 1 -- 172.21.15.19:0/4084429981 wait complete. 2024-03-28T13:21:26.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.266+0000 7fb07dfa6740 1 -- start start 2024-03-28T13:21:26.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.266+0000 7fb07dfa6740 1 Processor -- start 2024-03-28T13:21:26.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.268+0000 7fb07dfa6740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641560 0x557477682080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.268+0000 7fb07dfa6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641940 0x5574776825c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.268+0000 7fb07dfa6740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477641d40 0x5574776d9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.268+0000 7fb07dfa6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5574772a0230 con 0x557477641940 2024-03-28T13:21:26.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.268+0000 7fb07dfa6740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb074150b50 con 0x557477641d40 2024-03-28T13:21:26.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.268+0000 7fb07dfa6740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5574776189c0 con 0x557477641560 2024-03-28T13:21:26.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.268+0000 7fb07dfa6740 10 client.0 _ll_register_callbacks cb 0x7fb064009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:21:26.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.269+0000 7fb073400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641940 0x5574776825c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.269+0000 7fb073400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641940 0x5574776825c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42238/0 (socket says 172.21.15.19:42238) 2024-03-28T13:21:26.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.269+0000 7fb073400640 1 -- 172.21.15.19:0/1094663638 learned_addr learned my addr 172.21.15.19:0/1094663638 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:26.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.269+0000 7fb073e00640 1 --2- 172.21.15.19:0/1094663638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641560 0x557477682080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.269+0000 7fb079200640 1 --2- 172.21.15.19:0/1094663638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477641d40 0x5574776d9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.269+0000 7fb073400640 1 -- 172.21.15.19:0/1094663638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641560 msgr2=0x557477682080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:26.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.269+0000 7fb073400640 1 --2- 172.21.15.19:0/1094663638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641560 0x557477682080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.269+0000 7fb073400640 1 -- 172.21.15.19:0/1094663638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477641d40 msgr2=0x5574776d9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:26.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.269+0000 7fb073400640 1 --2- 172.21.15.19:0/1094663638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477641d40 0x5574776d9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.269+0000 7fb073400640 1 -- 172.21.15.19:0/1094663638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747765ee00 con 0x557477641940 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.269+0000 7fb073400640 1 --2- 172.21.15.19:0/1094663638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641940 0x5574776825c0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fb060013000 tx=0x7fb060000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c01ee8a35a148ec server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.269+0000 7fb05d600640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.270+0000 7fb05d600640 1 -- 172.21.15.19:0/1094663638 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060019040 con 0x557477641940 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.270+0000 7fb05d600640 1 -- 172.21.15.19:0/1094663638 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 5853+0+0 (secure 0 0 0) 0x7fb060026780 con 0x557477641940 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.270+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1094663638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb064038070 con 0x557477641940 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.270+0000 7fb05d600640 1 -- 172.21.15.19:0/1094663638 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=105) ==== 24+0+0 (secure 0 0 0) 0x7fb06002d050 con 0x557477641940 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.270+0000 7fb07dfa6740 10 client.9114 fetch_fsmap learned FSMap version 105 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.270+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1094663638 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=105,monmap=2+}) -- 0x5574776dabb0 con 0x557477641940 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.270+0000 7fb05d600640 1 -- 172.21.15.19:0/1094663638 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb064038070 con 0x557477641940 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.270+0000 7fb05d600640 1 -- 172.21.15.19:0/1094663638 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 105) ==== 3494+0+0 (secure 0 0 0) 0x7fb060030020 con 0x557477641940 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.271+0000 7fb07dfa6740 10 client.9114 fetch_fsmap finished waiting for FSMap version 105 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.271+0000 7fb07dfa6740 10 client.9114 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.271+0000 7fb07dfa6740 20 client.9114 populate_metadata read hostname 'smithi019' 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.271+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/1094663638 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x557477679230 0x5574776da1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.271+0000 7fb07dfa6740 4 client.9114 mds_command: new command op to 8691 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-28T13:21:26.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.271+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1094663638 --> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557477586f40 con 0x557477679230 2024-03-28T13:21:26.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.271+0000 7fb073e00640 1 --2- 172.21.15.19:0/1094663638 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x557477679230 0x5574776da1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.271+0000 7fb073e00640 1 --2- 172.21.15.19:0/1094663638 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x557477679230 0x5574776da1f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65d906caff378865 server_cookie=13fca7a410d1b78f in_seq=0 out_seq=0 2024-03-28T13:21:26.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.271+0000 7fb05d600640 10 client.9114 ms_handle_connect on v2:172.21.15.19:6833/3930967142 2024-03-28T13:21:26.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.296+0000 7fb05d600640 1 -- 172.21.15.19:0/1094663638 <== mds.0 v2:172.21.15.19:6833/3930967142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557477586f40 con 0x557477679230 2024-03-28T13:21:26.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.296+0000 7fb05d600640 10 client.9114 handle_command_reply: tid=0 2024-03-28T13:21:26.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.296+0000 7fb07dfa6740 1 client.9114 shutdown 2024-03-28T13:21:26.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.296+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1094663638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641940 msgr2=0x5574776825c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.296+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/1094663638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641940 0x5574776825c0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fb060013000 tx=0x7fb060000bc0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.296+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1094663638 shutdown_connections 2024-03-28T13:21:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.296+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/1094663638 >> [v2:172.21.15.19:6833/3930967142,v1:172.21.15.19:6835/3930967142] conn(0x557477679230 0x5574776da1f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.296+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/1094663638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x557477641d40 0x5574776d9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.296+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/1094663638 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557477641940 0x5574776825c0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.296+0000 7fb07dfa6740 1 --2- 172.21.15.19:0/1094663638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x557477641560 0x557477682080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.296+0000 7fb05d600640 0 client.9114 ms_handle_reset on v2:172.21.15.19:6833/3930967142 2024-03-28T13:21:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.296+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1094663638 >> 172.21.15.19:0/1094663638 conn(0x557477614a40 msgr2=0x55747760fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.296+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1094663638 shutdown_connections 2024-03-28T13:21:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.297+0000 7fb07dfa6740 1 -- 172.21.15.19:0/1094663638 wait complete. 2024-03-28T13:21:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.297+0000 7fb07dfa6740 20 client.9114 trim_cache size 0 max 16384 2024-03-28T13:21:26.297 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:21:26.298 INFO:teuthology.orchestra.run.smithi019.stdout: "message": "", 2024-03-28T13:21:26.298 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0 2024-03-28T13:21:26.298 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:21:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.297+0000 7fb07b200640 1 -- 172.21.15.19:0/2079005109 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb05406c530 msgr2=0x7fb05406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.297+0000 7fb07b200640 1 --2- 172.21.15.19:0/2079005109 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb05406c530 0x7fb05406e9f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb064009b10 tx=0x7fb064008040 comp rx=0 tx=0).stop 2024-03-28T13:21:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.297+0000 7fb07b200640 1 -- 172.21.15.19:0/2079005109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07415f700 msgr2=0x7fb074153280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.297+0000 7fb07b200640 1 --2- 172.21.15.19:0/2079005109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07415f700 0x7fb074153280 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fb06c038040 tx=0x7fb06c004a40 comp rx=0 tx=0).stop 2024-03-28T13:21:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.298+0000 7fb07b200640 1 -- 172.21.15.19:0/2079005109 shutdown_connections 2024-03-28T13:21:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.298+0000 7fb07b200640 1 --2- 172.21.15.19:0/2079005109 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb05406c530 0x7fb05406e9f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.298+0000 7fb07b200640 1 --2- 172.21.15.19:0/2079005109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb07415f700 0x7fb074153280 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.298+0000 7fb07b200640 1 --2- 172.21.15.19:0/2079005109 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb07415b4e0 0x7fb074145c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.298+0000 7fb07b200640 1 --2- 172.21.15.19:0/2079005109 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb0740afee0 0x7fb074151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.298+0000 7fb07b200640 1 -- 172.21.15.19:0/2079005109 >> 172.21.15.19:0/2079005109 conn(0x7fb0740b6560 msgr2=0x7fb0740ae6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.298+0000 7fb07b200640 1 -- 172.21.15.19:0/2079005109 shutdown_connections 2024-03-28T13:21:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.298+0000 7fb07b200640 1 -- 172.21.15.19:0/2079005109 wait complete. 2024-03-28T13:21:26.317 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T13:21:26.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.525+0000 7f6846a00640 1 Processor -- start 2024-03-28T13:21:26.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.526+0000 7f6846a00640 1 -- start start 2024-03-28T13:21:26.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.527+0000 7f6846a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f684015e860 0x7f684006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.527+0000 7f6846a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68400d4ae0 0x7f68400cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.527+0000 7f6846a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f684006dfa0 con 0x7f684006da30 2024-03-28T13:21:26.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.527+0000 7f6846a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f68400cf220 con 0x7f684015e860 2024-03-28T13:21:26.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.527+0000 7f6846a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f684006e240 con 0x7f68400d4ae0 2024-03-28T13:21:26.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.527+0000 7f6837e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f684006da30 0x7f68400d43d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42606/0 (socket says 172.21.15.19:42606) 2024-03-28T13:21:26.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.527+0000 7f6837e00640 1 -- 172.21.15.19:0/111349553 learned_addr learned my addr 172.21.15.19:0/111349553 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:26.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.528+0000 7f683fe00640 1 --2- 172.21.15.19:0/111349553 >> v2:172.21.15.188:3300/0 conn(0x7f684015e860 0x7f684006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.528+0000 7f683f400640 1 -- 172.21.15.19:0/111349553 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1192646861 0 0) 0x7f684006dfa0 con 0x7f684006da30 2024-03-28T13:21:26.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.528+0000 7f683f400640 1 -- 172.21.15.19:0/111349553 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6824003520 con 0x7f684006da30 2024-03-28T13:21:26.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.528+0000 7f6844a00640 1 --2- 172.21.15.19:0/111349553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68400d4ae0 0x7f68400cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.528+0000 7f683f400640 1 -- 172.21.15.19:0/111349553 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 274770098 0 0) 0x7f6824003520 con 0x7f684006da30 2024-03-28T13:21:26.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.528+0000 7f683f400640 1 -- 172.21.15.19:0/111349553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68400d4ae0 msgr2=0x7f68400cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:26.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.528+0000 7f683f400640 1 --2- 172.21.15.19:0/111349553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68400d4ae0 0x7f68400cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.528+0000 7f683f400640 1 -- 172.21.15.19:0/111349553 >> v2:172.21.15.188:3300/0 conn(0x7f684015e860 msgr2=0x7f684006d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:26.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.528+0000 7f683f400640 1 --2- 172.21.15.19:0/111349553 >> v2:172.21.15.188:3300/0 conn(0x7f684015e860 0x7f684006d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.528+0000 7f683f400640 1 -- 172.21.15.19:0/111349553 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684006e520 con 0x7f684006da30 2024-03-28T13:21:26.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.528+0000 7f683f400640 1 -- 172.21.15.19:0/111349553 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f6828001690 con 0x7f684006da30 2024-03-28T13:21:26.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.528+0000 7f683f400640 1 -- 172.21.15.19:0/111349553 >> v1:172.21.15.19:6789/0 conn(0x7f684006da30 legacy=0x7f68400d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.528+0000 7f683f400640 1 --2- 172.21.15.19:0/111349553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68400d4ae0 0x7f68240039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.529+0000 7f683f400640 1 --2- 172.21.15.19:0/111349553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6824003f30 0x7f6824004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.529+0000 7f683f400640 1 --2- 172.21.15.19:0/111349553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6824004d40 0x7f68240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.529+0000 7f683f400640 1 -- 172.21.15.19:0/111349553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68400cf220 con 0x7f6824004d40 2024-03-28T13:21:26.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.529+0000 7f683f400640 1 -- 172.21.15.19:0/111349553 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f684006e240 con 0x7f6824003f30 2024-03-28T13:21:26.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.529+0000 7f683f400640 1 -- 172.21.15.19:0/111349553 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f682400b9c0 con 0x7f68400d4ae0 2024-03-28T13:21:26.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.529+0000 7f6844a00640 1 --2- 172.21.15.19:0/111349553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6824004d40 0x7f68240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.529+0000 7f6837e00640 1 --2- 172.21.15.19:0/111349553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6824003f30 0x7f6824004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.529+0000 7f683fe00640 1 --2- 172.21.15.19:0/111349553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68400d4ae0 0x7f68240039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.529+0000 7f6844a00640 1 -- 172.21.15.19:0/111349553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68400d4ae0 msgr2=0x7f68240039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:26.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.529+0000 7f6844a00640 1 --2- 172.21.15.19:0/111349553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68400d4ae0 0x7f68240039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.529+0000 7f6844a00640 1 -- 172.21.15.19:0/111349553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6824003f30 msgr2=0x7f6824004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:26.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.529+0000 7f6844a00640 1 --2- 172.21.15.19:0/111349553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6824003f30 0x7f6824004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.529+0000 7f6844a00640 1 -- 172.21.15.19:0/111349553 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6824003520 con 0x7f6824004d40 2024-03-28T13:21:26.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.531+0000 7f6844a00640 1 --2- 172.21.15.19:0/111349553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6824004d40 0x7f68240051c0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f6838009f90 tx=0x7f6824005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52292784cf1fd652 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:26.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.531+0000 7f683f400640 1 -- 172.21.15.19:0/111349553 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6838039070 con 0x7f6824004d40 2024-03-28T13:21:26.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.531+0000 7f683f400640 1 -- 172.21.15.19:0/111349553 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f683802cbc0 con 0x7f6824004d40 2024-03-28T13:21:26.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.531+0000 7f683f400640 1 -- 172.21.15.19:0/111349553 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6838009c10 con 0x7f6824004d40 2024-03-28T13:21:26.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.531+0000 7f6846a00640 1 -- 172.21.15.19:0/111349553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6824004d40 msgr2=0x7f68240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.531+0000 7f6846a00640 1 --2- 172.21.15.19:0/111349553 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6824004d40 0x7f68240051c0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f6838009f90 tx=0x7f6824005830 comp rx=0 tx=0).stop 2024-03-28T13:21:26.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.531+0000 7f6844a00640 1 -- 172.21.15.19:0/111349553 reap_dead start 2024-03-28T13:21:26.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.531+0000 7f6846a00640 1 -- 172.21.15.19:0/111349553 shutdown_connections 2024-03-28T13:21:26.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.531+0000 7f6846a00640 1 -- 172.21.15.19:0/111349553 >> 172.21.15.19:0/111349553 conn(0x7f684009d7d0 msgr2=0x7f684008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:26.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.532+0000 7f6846a00640 1 -- 172.21.15.19:0/111349553 shutdown_connections 2024-03-28T13:21:26.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.532+0000 7f6846a00640 1 -- 172.21.15.19:0/111349553 wait complete. 2024-03-28T13:21:26.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.533+0000 7f6846a00640 1 Processor -- start 2024-03-28T13:21:26.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.534+0000 7f6846a00640 1 -- start start 2024-03-28T13:21:26.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.534+0000 7f6846a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6824004d40 0x7f684015dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.534+0000 7f6846a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68400b7ea0 0x7f68400b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.535+0000 7f6846a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68400ca740 0x7f68400b9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.535+0000 7f6846a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f68400cef30 con 0x7f68400b7ea0 2024-03-28T13:21:26.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.535+0000 7f6846a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f68400cedb0 con 0x7f68400ca740 2024-03-28T13:21:26.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.535+0000 7f6846a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f68400cf6b0 con 0x7f6824004d40 2024-03-28T13:21:26.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.535+0000 7f6837e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68400b7ea0 0x7f68400b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.535+0000 7f6837e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68400b7ea0 0x7f68400b8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42246/0 (socket says 172.21.15.19:42246) 2024-03-28T13:21:26.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.535+0000 7f6837e00640 1 -- 172.21.15.19:0/103012094 learned_addr learned my addr 172.21.15.19:0/103012094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:26.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.535+0000 7f683fe00640 1 --2- 172.21.15.19:0/103012094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6824004d40 0x7f684015dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.535+0000 7f6837e00640 1 -- 172.21.15.19:0/103012094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6824004d40 msgr2=0x7f684015dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.535+0000 7f6837e00640 1 --2- 172.21.15.19:0/103012094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6824004d40 0x7f684015dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.535+0000 7f6837e00640 1 -- 172.21.15.19:0/103012094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68400ca740 msgr2=0x7f68400b9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.535+0000 7f6837e00640 1 --2- 172.21.15.19:0/103012094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68400ca740 0x7f68400b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.535+0000 7f6837e00640 1 -- 172.21.15.19:0/103012094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68400b8410 con 0x7f68400b7ea0 2024-03-28T13:21:26.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.536+0000 7f6837e00640 1 --2- 172.21.15.19:0/103012094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68400b7ea0 0x7f68400b8fe0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f682800e870 tx=0x7f682800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:26.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.536+0000 7f683cc00640 1 -- 172.21.15.19:0/103012094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682800c7f0 con 0x7f68400b7ea0 2024-03-28T13:21:26.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.536+0000 7f683cc00640 1 -- 172.21.15.19:0/103012094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6828018070 con 0x7f68400b7ea0 2024-03-28T13:21:26.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.536+0000 7f6846a00640 1 -- 172.21.15.19:0/103012094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68400b86f0 con 0x7f68400b7ea0 2024-03-28T13:21:26.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.536+0000 7f683cc00640 1 -- 172.21.15.19:0/103012094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682800cbd0 con 0x7f68400b7ea0 2024-03-28T13:21:26.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.536+0000 7f6846a00640 1 -- 172.21.15.19:0/103012094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68400a1ee0 con 0x7f68400b7ea0 2024-03-28T13:21:26.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.537+0000 7f6846a00640 1 -- 172.21.15.19:0/103012094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68400685c0 con 0x7f68400b7ea0 2024-03-28T13:21:26.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.538+0000 7f683cc00640 1 -- 172.21.15.19:0/103012094 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f682801d460 con 0x7f68400b7ea0 2024-03-28T13:21:26.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.539+0000 7f683cc00640 1 --2- 172.21.15.19:0/103012094 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f681c06c560 0x7f681c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:26.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.539+0000 7f683cc00640 1 -- 172.21.15.19:0/103012094 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5853+0+0 (secure 0 0 0) 0x7f682808b590 con 0x7f68400b7ea0 2024-03-28T13:21:26.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.540+0000 7f683fe00640 1 --2- 172.21.15.19:0/103012094 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f681c06c560 0x7f681c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:26.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.541+0000 7f683fe00640 1 --2- 172.21.15.19:0/103012094 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f681c06c560 0x7f681c06ea20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f68300099d0 tx=0x7f6830008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:26.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.544+0000 7f683cc00640 1 -- 172.21.15.19:0/103012094 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f6828059bb0 con 0x7f68400b7ea0 2024-03-28T13:21:26.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.919+0000 7f6846a00640 1 -- 172.21.15.19:0/103012094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f68400b8c10 con 0x7f68400b7ea0 2024-03-28T13:21:26.919 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:21:26.920+0000 7f6b74400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T13:21:26.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.924+0000 7f683cc00640 1 -- 172.21.15.19:0/103012094 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v106) ==== 161+0+0 (secure 0 0 0) 0x7f68280594b0 con 0x7f68400b7ea0 2024-03-28T13:21:26.924 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T13:21:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.927+0000 7f6846a00640 1 -- 172.21.15.19:0/103012094 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f681c06c560 msgr2=0x7f681c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.927+0000 7f6846a00640 1 --2- 172.21.15.19:0/103012094 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f681c06c560 0x7f681c06ea20 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f68300099d0 tx=0x7f6830008040 comp rx=0 tx=0).stop 2024-03-28T13:21:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.927+0000 7f6846a00640 1 -- 172.21.15.19:0/103012094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68400b7ea0 msgr2=0x7f68400b8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.927+0000 7f6846a00640 1 --2- 172.21.15.19:0/103012094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68400b7ea0 0x7f68400b8fe0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f682800e870 tx=0x7f682800ed30 comp rx=0 tx=0).stop 2024-03-28T13:21:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.927+0000 7f6846a00640 1 -- 172.21.15.19:0/103012094 shutdown_connections 2024-03-28T13:21:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.927+0000 7f6846a00640 1 --2- 172.21.15.19:0/103012094 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f681c06c560 0x7f681c06ea20 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.927+0000 7f6846a00640 1 --2- 172.21.15.19:0/103012094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68400ca740 0x7f68400b9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.927+0000 7f6846a00640 1 --2- 172.21.15.19:0/103012094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f68400b7ea0 0x7f68400b8fe0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.927+0000 7f6846a00640 1 --2- 172.21.15.19:0/103012094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6824004d40 0x7f684015dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:26.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.927+0000 7f6846a00640 1 -- 172.21.15.19:0/103012094 >> 172.21.15.19:0/103012094 conn(0x7f684009d7d0 msgr2=0x7f684008bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:26.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.928+0000 7f6846a00640 1 -- 172.21.15.19:0/103012094 shutdown_connections 2024-03-28T13:21:26.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:26.928+0000 7f6846a00640 1 -- 172.21.15.19:0/103012094 wait complete. 2024-03-28T13:21:26.935 INFO:tasks.ceph.mds.a.smithi019.stderr: -1946> 2024-03-28T13:21:19.378+0000 7f782aa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:21:26.949 DEBUG:teuthology.orchestra.run.smithi019:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-03-28T13:21:27.003 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-03-28T13:21:27.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.036+0000 7f96fb7b9280 1 -- start start 2024-03-28T13:21:27.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.036+0000 7f96fb7b9280 1 Processor -- start 2024-03-28T13:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.037+0000 7f96fb7b9280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559ac0cec00 0x5559ace96000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.037+0000 7f96fb7b9280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559ac0cf800 0x5559ace96580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.037+0000 7f96fb7b9280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559ac0cfc00 0x5559ace96b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.037+0000 7f96fb7b9280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5559acea8180 con 0x5559ac0cfc00 2024-03-28T13:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.037+0000 7f96fb7b9280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5559acea8300 con 0x5559ac0cf800 2024-03-28T13:21:27.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.037+0000 7f96fb7b9280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5559acea8480 con 0x5559ac0cec00 2024-03-28T13:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.038+0000 7f96f8800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559ac0cfc00 0x5559ace96b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.038+0000 7f96f8800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559ac0cfc00 0x5559ace96b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42284/0 (socket says 172.21.15.19:42284) 2024-03-28T13:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.038+0000 7f96f8800640 1 -- 172.21.15.19:0/4078434338 learned_addr learned my addr 172.21.15.19:0/4078434338 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.038+0000 7f96f7400640 1 --2- 172.21.15.19:0/4078434338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559ac0cf800 0x5559ace96580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.038+0000 7f96f7e00640 1 --2- 172.21.15.19:0/4078434338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559ac0cec00 0x5559ace96000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.038+0000 7f96f8800640 1 -- 172.21.15.19:0/4078434338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559ac0cec00 msgr2=0x5559ace96000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.038+0000 7f96f8800640 1 --2- 172.21.15.19:0/4078434338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559ac0cec00 0x5559ace96000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.038+0000 7f96f8800640 1 -- 172.21.15.19:0/4078434338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559ac0cf800 msgr2=0x5559ace96580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.038+0000 7f96f8800640 1 --2- 172.21.15.19:0/4078434338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559ac0cf800 0x5559ace96580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.038+0000 7f96f8800640 1 -- 172.21.15.19:0/4078434338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ac102780 con 0x5559ac0cfc00 2024-03-28T13:21:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.038+0000 7f96f7400640 1 --2- 172.21.15.19:0/4078434338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559ac0cf800 0x5559ace96580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:21:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.039+0000 7f96f8800640 1 --2- 172.21.15.19:0/4078434338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559ac0cfc00 0x5559ace96b00 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x5559aceeea80 tx=0x5559ace633e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b7666ef86ebce9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.039+0000 7f96f4c00640 1 -- 172.21.15.19:0/4078434338 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5559ac0f49c0 con 0x5559ac0cfc00 2024-03-28T13:21:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.039+0000 7f96f4c00640 1 -- 172.21.15.19:0/4078434338 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5559ac0fa700 con 0x5559ac0cfc00 2024-03-28T13:21:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.039+0000 7f96f4c00640 1 -- 172.21.15.19:0/4078434338 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5559ac0f4680 con 0x5559ac0cfc00 2024-03-28T13:21:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.039+0000 7f96fb7b9280 1 -- 172.21.15.19:0/4078434338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559ac1034a0 con 0x5559ac0cfc00 2024-03-28T13:21:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.039+0000 7f96fb7b9280 1 -- 172.21.15.19:0/4078434338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5559ac102000 con 0x5559ac0cfc00 2024-03-28T13:21:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.039+0000 7f96fb7b9280 4 waiting for MDS map... 2024-03-28T13:21:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.039+0000 7f96f4c00640 1 -- 172.21.15.19:0/4078434338 <== mon.0 v2:172.21.15.19:3300/0 4 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x5559acfa6200 con 0x5559ac0cfc00 2024-03-28T13:21:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.039+0000 7f96f4c00640 1 -- 172.21.15.19:0/4078434338 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 106) ==== 2885+0+0 (secure 0 0 0) 0x5559ac129180 con 0x5559ac0cfc00 2024-03-28T13:21:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.039+0000 7f96fb7b9280 4 Got MDS map 106 2024-03-28T13:21:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.039+0000 7f96fb7b9280 10 main: JournalTool::main 2024-03-28T13:21:27.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.039+0000 7f96fb7b9280 4 main: JournalTool: connecting to RADOS... 2024-03-28T13:21:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.040+0000 7f96fb7b9280 1 Processor -- start 2024-03-28T13:21:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.040+0000 7f96fb7b9280 1 -- start start 2024-03-28T13:21:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.041+0000 7f96fb7b9280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559acff8000 0x5559ace97080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.041+0000 7f96fb7b9280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559acff8400 0x5559ace97600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.041+0000 7f96fb7b9280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559acff8800 0x5559ace97b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.041+0000 7f96fb7b9280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5559acea8600 con 0x5559acff8000 2024-03-28T13:21:27.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.041+0000 7f96fb7b9280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5559acea8780 con 0x5559acff8800 2024-03-28T13:21:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.041+0000 7f96fb7b9280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5559acea8900 con 0x5559acff8400 2024-03-28T13:21:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.041+0000 7f96f7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559acff8000 0x5559ace97080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.041+0000 7f96f7400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559acff8000 0x5559ace97080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42288/0 (socket says 172.21.15.19:42288) 2024-03-28T13:21:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.041+0000 7f96f7400640 1 -- 172.21.15.19:0/4280698909 learned_addr learned my addr 172.21.15.19:0/4280698909 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.042+0000 7f96f7400640 1 -- 172.21.15.19:0/4280698909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559acff8400 msgr2=0x5559ace97600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.042+0000 7f96f7400640 1 --2- 172.21.15.19:0/4280698909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559acff8400 0x5559ace97600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.042+0000 7f96f7400640 1 -- 172.21.15.19:0/4280698909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559acff8800 msgr2=0x5559ace97b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.042+0000 7f96f7400640 1 --2- 172.21.15.19:0/4280698909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559acff8800 0x5559ace97b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.042+0000 7f96f7400640 1 -- 172.21.15.19:0/4280698909 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ac1023c0 con 0x5559acff8000 2024-03-28T13:21:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.042+0000 7f96f7400640 1 --2- 172.21.15.19:0/4280698909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559acff8000 0x5559ace97080 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x5559acff26f0 tx=0x5559acfa8c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3c9e73b89df8d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.042+0000 7f96f2400640 1 -- 172.21.15.19:0/4280698909 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5559ac0f49c0 con 0x5559acff8000 2024-03-28T13:21:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.042+0000 7f96f2400640 1 -- 172.21.15.19:0/4280698909 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5559ac0fafc0 con 0x5559acff8000 2024-03-28T13:21:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.042+0000 7f96f2400640 1 -- 172.21.15.19:0/4280698909 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5559ac0f4680 con 0x5559acff8000 2024-03-28T13:21:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.043+0000 7f96fb7b9280 1 -- 172.21.15.19:0/4280698909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559acff8000 msgr2=0x5559ace97080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.043+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/4280698909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559acff8000 0x5559ace97080 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x5559acff26f0 tx=0x5559acfa8c00 comp rx=0 tx=0).stop 2024-03-28T13:21:27.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.043+0000 7f96fb7b9280 1 -- 172.21.15.19:0/4280698909 shutdown_connections 2024-03-28T13:21:27.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.043+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/4280698909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559acff8800 0x5559ace97b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.043+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/4280698909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559acff8400 0x5559ace97600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.043+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/4280698909 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559acff8000 0x5559ace97080 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.043+0000 7f96fb7b9280 1 -- 172.21.15.19:0/4280698909 >> 172.21.15.19:0/4280698909 conn(0x5559aceb3c00 msgr2=0x5559ac1db000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:27.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.043+0000 7f96fb7b9280 1 -- 172.21.15.19:0/4280698909 shutdown_connections 2024-03-28T13:21:27.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.043+0000 7f96fb7b9280 1 -- 172.21.15.19:0/4280698909 wait complete. 2024-03-28T13:21:27.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.044+0000 7f96fb7b9280 1 Processor -- start 2024-03-28T13:21:27.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.044+0000 7f96fb7b9280 1 -- start start 2024-03-28T13:21:27.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.044+0000 7f96fb7b9280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559acff8000 0x5559ace97080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96fb7b9280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559acff8400 0x5559ace97600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96fb7b9280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559acff8800 0x5559ace97b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96fb7b9280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5559acea9c80 con 0x5559acff8400 2024-03-28T13:21:27.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96fb7b9280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5559acea9b00 con 0x5559acff8800 2024-03-28T13:21:27.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96fb7b9280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5559acea9980 con 0x5559acff8000 2024-03-28T13:21:27.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96f7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559acff8400 0x5559ace97600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96f7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559acff8400 0x5559ace97600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42290/0 (socket says 172.21.15.19:42290) 2024-03-28T13:21:27.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96f7e00640 1 -- 172.21.15.19:0/2716968537 learned_addr learned my addr 172.21.15.19:0/2716968537 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:27.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96f7400640 1 --2- 172.21.15.19:0/2716968537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559acff8000 0x5559ace97080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96f7400640 1 --2- 172.21.15.19:0/2716968537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559acff8800 0x5559ace97b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96f7e00640 1 -- 172.21.15.19:0/2716968537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559acff8000 msgr2=0x5559ace97080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96f7e00640 1 --2- 172.21.15.19:0/2716968537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559acff8000 0x5559ace97080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96f7e00640 1 -- 172.21.15.19:0/2716968537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559acff8800 msgr2=0x5559ace97b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96f7e00640 1 --2- 172.21.15.19:0/2716968537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559acff8800 0x5559ace97b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96f7e00640 1 -- 172.21.15.19:0/2716968537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ac102960 con 0x5559acff8400 2024-03-28T13:21:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.045+0000 7f96f7400640 1 --2- 172.21.15.19:0/2716968537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559acff8000 0x5559ace97080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.046+0000 7f96f7e00640 1 --2- 172.21.15.19:0/2716968537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559acff8400 0x5559ace97600 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x5559acff33b0 tx=0x5559acfa8d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.046+0000 7f96f0600640 1 -- 172.21.15.19:0/2716968537 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5559ac0f49c0 con 0x5559acff8400 2024-03-28T13:21:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.046+0000 7f96f0600640 1 -- 172.21.15.19:0/2716968537 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5559ac0fae00 con 0x5559acff8400 2024-03-28T13:21:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.046+0000 7f96f0600640 1 -- 172.21.15.19:0/2716968537 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5559ac0f4680 con 0x5559acff8400 2024-03-28T13:21:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.046+0000 7f96fb7b9280 1 -- 172.21.15.19:0/2716968537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5559ac103680 con 0x5559acff8400 2024-03-28T13:21:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.046+0000 7f96fb7b9280 1 -- 172.21.15.19:0/2716968537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559ac103860 con 0x5559acff8400 2024-03-28T13:21:27.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.046+0000 7f96fb7b9280 4 main: JournalTool: resolving pool 16 2024-03-28T13:21:27.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.048+0000 7f96f0600640 1 -- 172.21.15.19:0/2716968537 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x5559ad008c00 con 0x5559acff8400 2024-03-28T13:21:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.049+0000 7f96f0600640 1 --2- 172.21.15.19:0/2716968537 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5559ad014c00 0x5559ace98100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.049+0000 7f96f0600640 1 -- 172.21.15.19:0/2716968537 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x5559acffee00 con 0x5559acff8400 2024-03-28T13:21:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.049+0000 7f96fb7b9280 4 main: JournalTool: creating IoCtx.. 2024-03-28T13:21:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.049+0000 7f96fb7b9280 4 main: Executing for rank 0 2024-03-28T13:21:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.049+0000 7f96f7400640 1 --2- 172.21.15.19:0/2716968537 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5559ad014c00 0x5559ace98100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.050+0000 7f96fb7b9280 1 -- start start 2024-03-28T13:21:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.050+0000 7f96fb7b9280 1 Processor -- start 2024-03-28T13:21:27.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.050+0000 7f96f7400640 1 --2- 172.21.15.19:0/2716968537 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5559ad014c00 0x5559ace98100 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x5559ace50630 tx=0x5559ace60ae0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.051+0000 7f96fb7b9280 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559acff9000 0x5559ace98680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.051+0000 7f96fb7b9280 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559ad015000 0x5559ace98c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.051+0000 7f96fb7b9280 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559ad015400 0x5559ace99180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.051+0000 7f96fb7b9280 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5559acea9680 con 0x5559ad015000 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.051+0000 7f96fb7b9280 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x5559acea8600 con 0x5559acff9000 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.051+0000 7f96fb7b9280 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x5559acea8780 con 0x5559ad015400 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.051+0000 7f96f7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559ad015000 0x5559ace98c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.051+0000 7f96f7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559ad015000 0x5559ace98c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42300/0 (socket says 172.21.15.19:42300) 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.051+0000 7f96f7e00640 1 -- 172.21.15.19:0/1131374044 learned_addr learned my addr 172.21.15.19:0/1131374044 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.051+0000 7f96f8800640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559acff9000 0x5559ace98680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.051+0000 7f96f7400640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559ad015400 0x5559ace99180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.052+0000 7f96f7e00640 1 -- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559ad015400 msgr2=0x5559ace99180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.052+0000 7f96f7e00640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559ad015400 0x5559ace99180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.052+0000 7f96f7e00640 1 -- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559acff9000 msgr2=0x5559ace98680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.052+0000 7f96f7e00640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559acff9000 0x5559ace98680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.052+0000 7f96f7e00640 1 -- 172.21.15.19:0/1131374044 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ac103a40 con 0x5559ad015000 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.052+0000 7f96f7e00640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559ad015000 0x5559ace98c00 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x5559ad04d3b0 tx=0x5559acfa9680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce361d5b4cadb2f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.052+0000 7f96eca00640 1 -- 172.21.15.19:0/1131374044 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5559ac3cd520 con 0x5559ad015000 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.052+0000 7f96eca00640 1 -- 172.21.15.19:0/1131374044 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5559ac0fac40 con 0x5559ad015000 2024-03-28T13:21:27.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.052+0000 7f96eca00640 1 -- 172.21.15.19:0/1131374044 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5559ac3cd6c0 con 0x5559ad015000 2024-03-28T13:21:27.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.052+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559ac102960 con 0x5559ad015000 2024-03-28T13:21:27.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.052+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5559ac103680 con 0x5559ad015000 2024-03-28T13:21:27.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.052+0000 7f96fb7b9280 4 waiting for MDS map... 2024-03-28T13:21:27.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.052+0000 7f96eca00640 1 -- 172.21.15.19:0/1131374044 <== mon.0 v2:172.21.15.19:3300/0 4 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x5559ac4a2200 con 0x5559ad015000 2024-03-28T13:21:27.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.053+0000 7f96eca00640 1 -- 172.21.15.19:0/1131374044 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 106) ==== 2885+0+0 (secure 0 0 0) 0x5559ac129500 con 0x5559ad015000 2024-03-28T13:21:27.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.053+0000 7f96fb7b9280 4 Got MDS map 106 2024-03-28T13:21:27.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.053+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5559ad015800 0x5559ace99700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.053+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:1 16.f 16:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5559ad015c00 con 0x5559ad015800 2024-03-28T13:21:27.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.053+0000 7f96f8800640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5559ad015800 0x5559ace99700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.054+0000 7f96f8800640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5559ad015800 0x5559ace99700 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=54603775a032859c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.057+0000 7f96f8800640 1 -- 172.21.15.19:0/1131374044 <== osd.5 v2:172.21.15.188:6800/3324247755 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v123'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5559ac1ca000 con 0x5559ad015800 2024-03-28T13:21:27.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.058+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5559ac4b4400 0x5559ac4be000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.058+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:2 16.14 16:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5559ac4b4800 con 0x5559ac4b4400 2024-03-28T13:21:27.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.059+0000 7f96f7400640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5559ac4b4400 0x5559ac4be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.060+0000 7f96f7400640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5559ac4b4400 0x5559ac4be000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=f99ff6c08385a755 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.070+0000 7f96f7400640 1 -- 172.21.15.19:0/1131374044 <== osd.4 v2:172.21.15.188:6808/1845695132 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v123'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5559ac1ca500 con 0x5559ac4b4400 2024-03-28T13:21:27.070 INFO:teuthology.orchestra.run.smithi019.stdout:writing EResetJournal entry 2024-03-28T13:21:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.070+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:3 16.14 16:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5559ac4b4c00 con 0x5559ac4b4400 2024-03-28T13:21:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.070+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x5559ac4b5000 0x5559ac4be580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.070+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:4 16.1b 16:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5559ac4b5400 con 0x5559ac4b5000 2024-03-28T13:21:27.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.070+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5559ac4b5800 0x5559ac4beb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.070+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:5 16.b 16:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5559ac4b5c00 con 0x5559ac4b5800 2024-03-28T13:21:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.070+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x5559ac4cc000 0x5559ac4bf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.070+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:6 16.1f 16:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5559ac4cc400 con 0x5559ac4cc000 2024-03-28T13:21:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.070+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5559ac4cc800 0x5559ac4bf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.070+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:7 16.1a 16:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5559ac4ccc00 con 0x5559ac4cc800 2024-03-28T13:21:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.070+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x5559ac4cd000 0x5559ac4bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.070+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:8 16.1e 16:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5559ac4cd400 con 0x5559ac4cd000 2024-03-28T13:21:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.070+0000 7f96f8800640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x5559ac4b5000 0x5559ac4be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.070+0000 7f96f7400640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x5559ac4cc000 0x5559ac4bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.071+0000 7f96f7e00640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x5559ac4cd000 0x5559ac4bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.071+0000 7f96f7e00640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5559ac4b5800 0x5559ac4beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.071+0000 7f96f8800640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x5559ac4b5000 0x5559ac4be580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=a1010b6013221c17 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.071+0000 7f96f8800640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5559ac4cc800 0x5559ac4bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.071+0000 7f96f7400640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x5559ac4cc000 0x5559ac4bf080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=368d62b39e611da4 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.071+0000 7f96f7e00640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x5559ac4cd000 0x5559ac4bfb80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=7367baa6f08127e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.072+0000 7f96f8800640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5559ac4cc800 0x5559ac4bf600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=cbada9323dcadfdd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.073+0000 7f96f7e00640 1 -- 172.21.15.19:0/1131374044 <== osd.2 v2:172.21.15.19:6816/1693606238 1 ==== osd_op_reply(8 200.00000006 [delete] v123'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5559ac1ca780 con 0x5559ac4cd000 2024-03-28T13:21:27.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.073+0000 7f96f7e00640 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5559ac4b5800 0x5559ac4beb00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=5ab501ce584094d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.075+0000 7f96f7400640 1 -- 172.21.15.19:0/1131374044 <== osd.4 v2:172.21.15.188:6808/1845695132 2 ==== osd_op_reply(3 200.00000001 [delete] v123'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5559ac1ca500 con 0x5559ac4b4400 2024-03-28T13:21:27.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.076+0000 7f96f8800640 1 -- 172.21.15.19:0/1131374044 <== osd.6 v2:172.21.15.188:6816/3597358155 1 ==== osd_op_reply(7 200.00000005 [delete] v123'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5559ac1ca000 con 0x5559ac4cc800 2024-03-28T13:21:27.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.076+0000 7f96f8800640 1 -- 172.21.15.19:0/1131374044 <== osd.1 v2:172.21.15.19:6801/1998544260 1 ==== osd_op_reply(4 200.00000002 [delete] v123'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5559ac1ca000 con 0x5559ac4b5000 2024-03-28T13:21:27.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.076+0000 7f96f7400640 1 -- 172.21.15.19:0/1131374044 <== osd.0 v2:172.21.15.19:6800/4106483280 1 ==== osd_op_reply(6 200.00000004 [delete] v123'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5559ac1ca500 con 0x5559ac4cc000 2024-03-28T13:21:27.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.076+0000 7f96f7e00640 1 -- 172.21.15.19:0/1131374044 <== osd.7 v2:172.21.15.188:6824/861796513 1 ==== osd_op_reply(5 200.00000003 [delete] v123'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5559ac1ca780 con 0x5559ac4b5800 2024-03-28T13:21:27.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.076+0000 7f96eac00640 1 -- 172.21.15.19:0/1131374044 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:9 16.14 16:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x5559acff9c00 con 0x5559ac4b4400 2024-03-28T13:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.080+0000 7f96f7400640 1 -- 172.21.15.19:0/1131374044 <== osd.4 v2:172.21.15.188:6808/1845695132 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v123'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5559ac1ca500 con 0x5559ac4b4400 2024-03-28T13:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.080+0000 7f96fb7b9280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-03-28T13:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x5559ac4cc000 msgr2=0x5559ac4bf080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x5559ac4cc000 0x5559ac4bf080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x5559ac4b5000 msgr2=0x5559ac4be580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x5559ac4b5000 0x5559ac4be580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x5559ac4cd000 msgr2=0x5559ac4bfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x5559ac4cd000 0x5559ac4bfb80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5559ac4b4400 msgr2=0x5559ac4be000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x5559ac4b4400 0x5559ac4be000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5559ad015800 msgr2=0x5559ace99700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x5559ad015800 0x5559ace99700 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5559ac4cc800 msgr2=0x5559ac4bf600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x5559ac4cc800 0x5559ac4bf600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5559ac4b5800 msgr2=0x5559ac4beb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x5559ac4b5800 0x5559ac4beb00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559ad015000 msgr2=0x5559ace98c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/1131374044 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559ad015000 0x5559ace98c00 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x5559ad04d3b0 tx=0x5559acfa9680 comp rx=0 tx=0).stop 2024-03-28T13:21:27.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96f7400640 1 -- 172.21.15.19:0/1131374044 reap_dead start 2024-03-28T13:21:27.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96f7400640 1 -- 172.21.15.19:0/1131374044 reap_dead start 2024-03-28T13:21:27.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96f7400640 1 -- 172.21.15.19:0/1131374044 reap_dead start 2024-03-28T13:21:27.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96f7400640 1 -- 172.21.15.19:0/1131374044 reap_dead start 2024-03-28T13:21:27.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 shutdown_connections 2024-03-28T13:21:27.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 >> 172.21.15.19:0/1131374044 conn(0x5559acff8c00 msgr2=0x5559ac1db800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:27.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 shutdown_connections 2024-03-28T13:21:27.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.081+0000 7f96fb7b9280 1 -- 172.21.15.19:0/1131374044 wait complete. 2024-03-28T13:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.082+0000 7f96fb7b9280 1 -- 172.21.15.19:0/2716968537 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5559ad014c00 msgr2=0x5559ace98100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.082+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/2716968537 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5559ad014c00 0x5559ace98100 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x5559ace50630 tx=0x5559ace60ae0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.082+0000 7f96fb7b9280 1 -- 172.21.15.19:0/2716968537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559acff8400 msgr2=0x5559ace97600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.082+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/2716968537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559acff8400 0x5559ace97600 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x5559acff33b0 tx=0x5559acfa8d20 comp rx=0 tx=0).stop 2024-03-28T13:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.082+0000 7f96fb7b9280 1 -- 172.21.15.19:0/2716968537 shutdown_connections 2024-03-28T13:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.082+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/2716968537 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x5559ad014c00 0x5559ace98100 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.082+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/2716968537 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559acff8800 0x5559ace97b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.082+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/2716968537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559acff8400 0x5559ace97600 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.082+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/2716968537 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559acff8000 0x5559ace97080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.082+0000 7f96fb7b9280 1 -- 172.21.15.19:0/2716968537 >> 172.21.15.19:0/2716968537 conn(0x5559aceb3c00 msgr2=0x5559ac1db000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:27.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.082+0000 7f96fb7b9280 1 -- 172.21.15.19:0/2716968537 shutdown_connections 2024-03-28T13:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.082+0000 7f96fb7b9280 1 -- 172.21.15.19:0/2716968537 wait complete. 2024-03-28T13:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.082+0000 7f96fb7b9280 1 -- 172.21.15.19:0/4078434338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559ac0cfc00 msgr2=0x5559ace96b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.082+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/4078434338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559ac0cfc00 0x5559ace96b00 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x5559aceeea80 tx=0x5559ace633e0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.083+0000 7f96fb7b9280 1 -- 172.21.15.19:0/4078434338 shutdown_connections 2024-03-28T13:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.083+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/4078434338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5559ac0cfc00 0x5559ace96b00 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.083+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/4078434338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5559ac0cf800 0x5559ace96580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.083+0000 7f96fb7b9280 1 --2- 172.21.15.19:0/4078434338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5559ac0cec00 0x5559ace96000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.083+0000 7f96fb7b9280 1 -- 172.21.15.19:0/4078434338 >> 172.21.15.19:0/4078434338 conn(0x5559ac0cf000 msgr2=0x5559ac1da800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.083+0000 7f96fb7b9280 1 -- 172.21.15.19:0/4078434338 shutdown_connections 2024-03-28T13:21:27.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.083+0000 7f96fb7b9280 1 -- 172.21.15.19:0/4078434338 wait complete. 2024-03-28T13:21:27.087 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.138641, result: writing EResetJournal entry 2024-03-28T13:21:27.087 DEBUG:teuthology.orchestra.run.smithi019:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-03-28T13:21:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.168+0000 7f32de439740 1 -- start start 2024-03-28T13:21:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.168+0000 7f32de439740 1 Processor -- start 2024-03-28T13:21:27.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.169+0000 7f32de439740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561310730c00 0x5613108a4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.169+0000 7f32de439740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561310731800 0x5613108a4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.169+0000 7f32de439740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561310731c00 0x5613108a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.169+0000 7f32de439740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561311514180 con 0x561310730c00 2024-03-28T13:21:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.169+0000 7f32de439740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561311514300 con 0x561310731800 2024-03-28T13:21:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.169+0000 7f32de439740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x561311514480 con 0x561310731c00 2024-03-28T13:21:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.169+0000 7f32dbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561310730c00 0x5613108a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.169+0000 7f32dbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561310730c00 0x5613108a4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42312/0 (socket says 172.21.15.19:42312) 2024-03-28T13:21:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.169+0000 7f32dbe00640 1 -- 172.21.15.19:0/820399735 learned_addr learned my addr 172.21.15.19:0/820399735 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:27.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.169+0000 7f32db400640 1 --2- 172.21.15.19:0/820399735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561310731800 0x5613108a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.169+0000 7f32dc800640 1 --2- 172.21.15.19:0/820399735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561310731c00 0x5613108a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.170+0000 7f32dbe00640 1 -- 172.21.15.19:0/820399735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561310731c00 msgr2=0x5613108a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.170+0000 7f32dbe00640 1 --2- 172.21.15.19:0/820399735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561310731c00 0x5613108a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.170+0000 7f32dbe00640 1 -- 172.21.15.19:0/820399735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561310731800 msgr2=0x5613108a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.170+0000 7f32dbe00640 1 --2- 172.21.15.19:0/820399735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561310731800 0x5613108a4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.170+0000 7f32dbe00640 1 -- 172.21.15.19:0/820399735 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56131075d680 con 0x561310730c00 2024-03-28T13:21:27.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.170+0000 7f32db400640 1 --2- 172.21.15.19:0/820399735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561310731800 0x5613108a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:27.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.170+0000 7f32dc800640 1 --2- 172.21.15.19:0/820399735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561310731c00 0x5613108a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:27.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.170+0000 7f32dbe00640 1 --2- 172.21.15.19:0/820399735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561310730c00 0x5613108a4000 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x5613115510b0 tx=0x5613114c8a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738189e32d02c357 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.170+0000 7f32d8c00640 1 -- 172.21.15.19:0/820399735 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5613107569c0 con 0x561310730c00 2024-03-28T13:21:27.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.170+0000 7f32d8c00640 1 -- 172.21.15.19:0/820399735 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56131077a1c0 con 0x561310730c00 2024-03-28T13:21:27.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.170+0000 7f32de439740 1 -- 172.21.15.19:0/820399735 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56131075d4a0 con 0x561310730c00 2024-03-28T13:21:27.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.170+0000 7f32de439740 1 -- 172.21.15.19:0/820399735 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56131075da40 con 0x561310730c00 2024-03-28T13:21:27.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.170+0000 7f32de439740 4 waiting for MDS map... 2024-03-28T13:21:27.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.170+0000 7f32d8c00640 1 -- 172.21.15.19:0/820399735 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561310756680 con 0x561310730c00 2024-03-28T13:21:27.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.170+0000 7f32d8c00640 1 -- 172.21.15.19:0/820399735 <== mon.0 v2:172.21.15.19:3300/0 4 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x5613115f8200 con 0x561310730c00 2024-03-28T13:21:27.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.171+0000 7f32d8c00640 1 -- 172.21.15.19:0/820399735 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 106) ==== 2885+0+0 (secure 0 0 0) 0x561310789180 con 0x561310730c00 2024-03-28T13:21:27.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.171+0000 7f32de439740 4 Got MDS map 106 2024-03-28T13:21:27.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.171+0000 7f32de439740 4 datascan.main: Using metadata pool output 2024-03-28T13:21:27.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.171+0000 7f32de439740 4 datascan.main: connecting to RADOS... 2024-03-28T13:21:27.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.172+0000 7f32de439740 1 Processor -- start 2024-03-28T13:21:27.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.172+0000 7f32de439740 1 -- start start 2024-03-28T13:21:27.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.172+0000 7f32de439740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561311658000 0x5613108a5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.172+0000 7f32de439740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561311658400 0x5613108a5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.172+0000 7f32de439740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561311658800 0x5613108a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.172+0000 7f32de439740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561311514600 con 0x561311658800 2024-03-28T13:21:27.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.172+0000 7f32de439740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561311514780 con 0x561311658400 2024-03-28T13:21:27.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.172+0000 7f32de439740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x561311514900 con 0x561311658000 2024-03-28T13:21:27.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.173+0000 7f32db400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561311658800 0x5613108a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.173+0000 7f32db400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561311658800 0x5613108a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42326/0 (socket says 172.21.15.19:42326) 2024-03-28T13:21:27.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.173+0000 7f32db400640 1 -- 172.21.15.19:0/2824731845 learned_addr learned my addr 172.21.15.19:0/2824731845 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:27.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.173+0000 7f32dc800640 1 --2- 172.21.15.19:0/2824731845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561311658000 0x5613108a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.173+0000 7f32dbe00640 1 --2- 172.21.15.19:0/2824731845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561311658400 0x5613108a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.173+0000 7f32db400640 1 -- 172.21.15.19:0/2824731845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561311658000 msgr2=0x5613108a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.173+0000 7f32db400640 1 --2- 172.21.15.19:0/2824731845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561311658000 0x5613108a5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.173+0000 7f32db400640 1 -- 172.21.15.19:0/2824731845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561311658400 msgr2=0x5613108a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.173+0000 7f32db400640 1 --2- 172.21.15.19:0/2824731845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561311658400 0x5613108a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.173+0000 7f32db400640 1 -- 172.21.15.19:0/2824731845 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56131075d860 con 0x561311658800 2024-03-28T13:21:27.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.173+0000 7f32db400640 1 --2- 172.21.15.19:0/2824731845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561311658800 0x5613108a5b80 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x561311606cf0 tx=0x56131160ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1aeebf4bf520bce server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.174+0000 7f32d6400640 1 -- 172.21.15.19:0/2824731845 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5613107569c0 con 0x561311658800 2024-03-28T13:21:27.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.174+0000 7f32d6400640 1 -- 172.21.15.19:0/2824731845 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56131077ac40 con 0x561311658800 2024-03-28T13:21:27.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.174+0000 7f32d6400640 1 -- 172.21.15.19:0/2824731845 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561310756680 con 0x561311658800 2024-03-28T13:21:27.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.174+0000 7f32de439740 1 -- 172.21.15.19:0/2824731845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561311658800 msgr2=0x5613108a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.174+0000 7f32de439740 1 --2- 172.21.15.19:0/2824731845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561311658800 0x5613108a5b80 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x561311606cf0 tx=0x56131160ccc0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.174+0000 7f32de439740 1 -- 172.21.15.19:0/2824731845 shutdown_connections 2024-03-28T13:21:27.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.174+0000 7f32de439740 1 --2- 172.21.15.19:0/2824731845 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561311658800 0x5613108a5b80 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.174+0000 7f32de439740 1 --2- 172.21.15.19:0/2824731845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561311658400 0x5613108a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.174+0000 7f32de439740 1 --2- 172.21.15.19:0/2824731845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561311658000 0x5613108a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.174+0000 7f32de439740 1 -- 172.21.15.19:0/2824731845 >> 172.21.15.19:0/2824731845 conn(0x561311499c00 msgr2=0x5613114cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:27.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.174+0000 7f32de439740 1 -- 172.21.15.19:0/2824731845 shutdown_connections 2024-03-28T13:21:27.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.174+0000 7f32de439740 1 -- 172.21.15.19:0/2824731845 wait complete. 2024-03-28T13:21:27.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.175+0000 7f32de439740 1 Processor -- start 2024-03-28T13:21:27.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.175+0000 7f32de439740 1 -- start start 2024-03-28T13:21:27.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32de439740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561311658000 0x5613108a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32de439740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561311658400 0x5613108a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32de439740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561311658800 0x5613108a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32de439740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561311515c80 con 0x561311658400 2024-03-28T13:21:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32de439740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x561311515b00 con 0x561311658800 2024-03-28T13:21:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32de439740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x561311515980 con 0x561311658000 2024-03-28T13:21:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32dbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561311658400 0x5613108a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32dbe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561311658400 0x5613108a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42334/0 (socket says 172.21.15.19:42334) 2024-03-28T13:21:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 learned_addr learned my addr 172.21.15.19:0/913314735 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32dc800640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561311658000 0x5613108a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32db400640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561311658800 0x5613108a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561311658000 msgr2=0x5613108a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32dbe00640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561311658000 0x5613108a5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561311658800 msgr2=0x5613108a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32dbe00640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561311658800 0x5613108a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.176+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56131075d2c0 con 0x561311658400 2024-03-28T13:21:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.177+0000 7f32db400640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561311658800 0x5613108a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.177+0000 7f32dbe00640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561311658400 0x5613108a5600 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x561311551c20 tx=0x5613114c8ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.177+0000 7f32d4600640 1 -- 172.21.15.19:0/913314735 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561310756b60 con 0x561311658400 2024-03-28T13:21:27.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.177+0000 7f32d4600640 1 -- 172.21.15.19:0/913314735 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x56131077ae00 con 0x561311658400 2024-03-28T13:21:27.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.177+0000 7f32d4600640 1 -- 172.21.15.19:0/913314735 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x561310756820 con 0x561311658400 2024-03-28T13:21:27.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.177+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56131075d0e0 con 0x561311658400 2024-03-28T13:21:27.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.177+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56131075c3c0 con 0x561311658400 2024-03-28T13:21:27.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.177+0000 7f32de439740 4 datascan.init: resolving metadata pool 16 2024-03-28T13:21:27.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.179+0000 7f32d4600640 1 -- 172.21.15.19:0/913314735 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x561311668c00 con 0x561311658400 2024-03-28T13:21:27.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.179+0000 7f32d4600640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x56131166b000 0x5613108a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.179+0000 7f32d4600640 1 -- 172.21.15.19:0/913314735 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x56131165f200 con 0x561311658400 2024-03-28T13:21:27.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.180+0000 7f32de439740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-28T13:21:27.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.180+0000 7f32de439740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-03-28T13:21:27.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.180+0000 7f32de439740 4 datascan.main: opening data pool 'cephfs_data' 2024-03-28T13:21:27.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.180+0000 7f32de439740 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x561311658c00 0x5613108a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.180+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561311659000 con 0x561311658c00 2024-03-28T13:21:27.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.180+0000 7f32db400640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x561311658c00 0x5613108a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.180+0000 7f32dc800640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x56131166b000 0x5613108a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.180+0000 7f32db400640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x561311658c00 0x5613108a6680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.181+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.0 v2:172.21.15.19:6800/4106483280 1 ==== osd_op_reply(1 [pgnls start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311658c00 2024-03-28T13:21:27.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.181+0000 7f32d5a00640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x561311659000 0x561310aba000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.181+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:2 17.20 17:04000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x56131166b400 con 0x561311659000 2024-03-28T13:21:27.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.181+0000 7f32dc800640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x56131166b000 0x5613108a6100 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x5613114b0cf0 tx=0x56131160cfc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.182+0000 7f32dc800640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x561311659000 0x561310aba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.183+0000 7f32dc800640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x561311659000 0x561310aba000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.183+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.7 v2:172.21.15.188:6824/861796513 1 ==== osd_op_reply(2 [pgnls start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561311659000 2024-03-28T13:21:27.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.184+0000 7f32d6400640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x56131166bc00 0x561310aba580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.184+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:3 17.10 17:08000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310aae000 con 0x56131166bc00 2024-03-28T13:21:27.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.184+0000 7f32dbe00640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x56131166bc00 0x561310aba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.185+0000 7f32dbe00640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x56131166bc00 0x561310aba580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.186+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 <== osd.4 v2:172.21.15.188:6808/1845695132 1 ==== osd_op_reply(3 [pgnls start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2780 con 0x56131166bc00 2024-03-28T13:21:27.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.186+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:4 17.30 17:0c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x56131166b800 con 0x561311659000 2024-03-28T13:21:27.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.187+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.7 v2:172.21.15.188:6824/861796513 2 ==== osd_op_reply(4 [pgnls start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561311659000 2024-03-28T13:21:27.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.187+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:5 17.8 17:10000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310aae400 con 0x561311658c00 2024-03-28T13:21:27.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.187+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.0 v2:172.21.15.19:6800/4106483280 2 ==== osd_op_reply(5 [pgnls start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311658c00 2024-03-28T13:21:27.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.187+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:6 17.28 17:14000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310aae800 con 0x56131166bc00 2024-03-28T13:21:27.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.188+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 <== osd.4 v2:172.21.15.188:6808/1845695132 2 ==== osd_op_reply(6 [pgnls start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2780 con 0x56131166bc00 2024-03-28T13:21:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.188+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:7 17.18 17:18000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310aaf400 con 0x561311659000 2024-03-28T13:21:27.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.189+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.7 v2:172.21.15.188:6824/861796513 3 ==== osd_op_reply(7 [pgnls start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561311659000 2024-03-28T13:21:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.189+0000 7f32d5a00640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x561310aaec00 0x561310abab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.189+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:8 17.38 17:1c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310aaf000 con 0x561310aaec00 2024-03-28T13:21:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.189+0000 7f32db400640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x561310aaec00 0x561310abab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.190+0000 7f32db400640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x561310aaec00 0x561310abab00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.190+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.3 v2:172.21.15.19:6819/3265515865 1 ==== osd_op_reply(8 [pgnls start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561310aaec00 2024-03-28T13:21:27.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.190+0000 7f32d6400640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x561310aaf800 0x561310abb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.190+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:9 17.4 17:20000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310aafc00 con 0x561310aaf800 2024-03-28T13:21:27.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.191+0000 7f32dc800640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x561310aaf800 0x561310abb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.192+0000 7f32dc800640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x561310aaf800 0x561310abb600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.193+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.6 v2:172.21.15.188:6816/3597358155 1 ==== osd_op_reply(9 [pgnls start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561310aaf800 2024-03-28T13:21:27.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.193+0000 7f32d5a00640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x561310aaf400 0x561310abb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.193+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:10 17.24 17:24000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x56131166b400 con 0x561310aaf400 2024-03-28T13:21:27.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.193+0000 7f32dbe00640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x561310aaf400 0x561310abb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.194+0000 7f32dbe00640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x561310aaf400 0x561310abb080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.195+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 <== osd.5 v2:172.21.15.188:6800/3324247755 1 ==== osd_op_reply(10 [pgnls start_epoch 123 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2780 con 0x561310aaf400 2024-03-28T13:21:27.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.195+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:11 17.14 17:28000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad6000 con 0x561310aaf400 2024-03-28T13:21:27.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.196+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 <== osd.5 v2:172.21.15.188:6800/3324247755 2 ==== osd_op_reply(11 [pgnls start_epoch 123 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2780 con 0x561310aaf400 2024-03-28T13:21:27.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.196+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:12 17.34 17:2c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x56131166b800 con 0x561311659000 2024-03-28T13:21:27.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.197+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.7 v2:172.21.15.188:6824/861796513 4 ==== osd_op_reply(12 [pgnls start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561311659000 2024-03-28T13:21:27.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.197+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:13 17.c 17:30000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad6400 con 0x561310aaec00 2024-03-28T13:21:27.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.198+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.3 v2:172.21.15.19:6819/3265515865 2 ==== osd_op_reply(13 [pgnls start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561310aaec00 2024-03-28T13:21:27.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.198+0000 7f32d5a00640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x561311659800 0x561310abc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.198+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:14 17.2c 17:34000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad7000 con 0x561311659800 2024-03-28T13:21:27.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.198+0000 7f32db400640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x561311659800 0x561310abc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.199+0000 7f32db400640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x561311659800 0x561310abc100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.199+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.1 v2:172.21.15.19:6801/1998544260 1 ==== osd_op_reply(14 [pgnls start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311659800 2024-03-28T13:21:27.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.199+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:15 17.1c 17:38000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad6800 con 0x561310aaf400 2024-03-28T13:21:27.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.200+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 <== osd.5 v2:172.21.15.188:6800/3324247755 3 ==== osd_op_reply(15 [pgnls start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2780 con 0x561310aaf400 2024-03-28T13:21:27.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.200+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:16 17.3c 17:3c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad7400 con 0x56131166bc00 2024-03-28T13:21:27.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.201+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 <== osd.4 v2:172.21.15.188:6808/1845695132 3 ==== osd_op_reply(16 [pgnls start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2780 con 0x56131166bc00 2024-03-28T13:21:27.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.201+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:17 17.2 17:40000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad6c00 con 0x561311659800 2024-03-28T13:21:27.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.202+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.1 v2:172.21.15.19:6801/1998544260 2 ==== osd_op_reply(17 [pgnls start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311659800 2024-03-28T13:21:27.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.202+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:18 17.22 17:44000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad7800 con 0x561310aaec00 2024-03-28T13:21:27.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.202+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.3 v2:172.21.15.19:6819/3265515865 3 ==== osd_op_reply(18 [pgnls start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561310aaec00 2024-03-28T13:21:27.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.202+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:19 17.12 17:48000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x56131166ac00 con 0x561311659000 2024-03-28T13:21:27.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.203+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.7 v2:172.21.15.188:6824/861796513 5 ==== osd_op_reply(19 [pgnls start_epoch 123 out=81b] v121'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5613114b2500 con 0x561311659000 2024-03-28T13:21:27.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.203+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:20 17.12 17:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x561310ad6400 con 0x561311659000 2024-03-28T13:21:27.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.204+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.7 v2:172.21.15.188:6824/861796513 6 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5613114b2500 con 0x561311659000 2024-03-28T13:21:27.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.205+0000 7f32de439740 10 datascan.operator(): handling object 1099511627778.0 2024-03-28T13:21:27.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.205+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:21 17.12 17:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x561310ad7000 con 0x561311659000 2024-03-28T13:21:27.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.208+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.7 v2:172.21.15.188:6824/861796513 7 ==== osd_op_reply(21 10000000002.00000000 [call] v123'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5613114b2500 con 0x561311659000 2024-03-28T13:21:27.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.208+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:22 17.32 17:4c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad6c00 con 0x561311658c00 2024-03-28T13:21:27.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.209+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.0 v2:172.21.15.19:6800/4106483280 3 ==== osd_op_reply(22 [pgnls start_epoch 123 out=49b] v119'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311658c00 2024-03-28T13:21:27.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.209+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:23 17.a 17:50000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310aae400 con 0x561311659800 2024-03-28T13:21:27.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.209+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.1 v2:172.21.15.19:6801/1998544260 3 ==== osd_op_reply(23 [pgnls start_epoch 123 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311659800 2024-03-28T13:21:27.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.209+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:24 17.2a 17:54000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad7c00 con 0x561310aaf400 2024-03-28T13:21:27.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.210+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 <== osd.5 v2:172.21.15.188:6800/3324247755 4 ==== osd_op_reply(24 [pgnls start_epoch 123 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2780 con 0x561310aaf400 2024-03-28T13:21:27.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.210+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:25 17.1a 17:58000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310aaf000 con 0x56131166bc00 2024-03-28T13:21:27.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.211+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 <== osd.4 v2:172.21.15.188:6808/1845695132 4 ==== osd_op_reply(25 [pgnls start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2780 con 0x56131166bc00 2024-03-28T13:21:27.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.211+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:26 17.3a 17:5c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad6400 con 0x561310aaf400 2024-03-28T13:21:27.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.212+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 <== osd.5 v2:172.21.15.188:6800/3324247755 5 ==== osd_op_reply(26 [pgnls start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2780 con 0x561310aaf400 2024-03-28T13:21:27.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.212+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:27 17.6 17:60000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310aae000 con 0x561311658c00 2024-03-28T13:21:27.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.213+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.0 v2:172.21.15.19:6800/4106483280 4 ==== osd_op_reply(27 [pgnls start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311658c00 2024-03-28T13:21:27.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.213+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:28 17.26 17:64000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad7000 con 0x561310aaf400 2024-03-28T13:21:27.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.214+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 <== osd.5 v2:172.21.15.188:6800/3324247755 6 ==== osd_op_reply(28 [pgnls start_epoch 123 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2780 con 0x561310aaf400 2024-03-28T13:21:27.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.214+0000 7f32d6400640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x561310aae800 0x561310abbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.214+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:29 17.16 17:68000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae2000 con 0x561310aae800 2024-03-28T13:21:27.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.214+0000 7f32dc800640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x561310aae800 0x561310abbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.215+0000 7f32dc800640 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x561310aae800 0x561310abbb80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.216+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.2 v2:172.21.15.19:6816/1693606238 1 ==== osd_op_reply(29 [pgnls start_epoch 123 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561310aae800 2024-03-28T13:21:27.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.216+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:30 17.36 17:6c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x56131166b800 con 0x561311659000 2024-03-28T13:21:27.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.217+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.7 v2:172.21.15.188:6824/861796513 8 ==== osd_op_reply(30 [pgnls start_epoch 123 out=81b] v119'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5613114b2500 con 0x561311659000 2024-03-28T13:21:27.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.217+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:31 17.36 17:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x561310ad6000 con 0x561311659000 2024-03-28T13:21:27.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.218+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.7 v2:172.21.15.188:6824/861796513 9 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5613114b2500 con 0x561311659000 2024-03-28T13:21:27.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.218+0000 7f32de439740 10 datascan.operator(): handling object 1099511628279.0 2024-03-28T13:21:27.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.218+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:32 17.36 17:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x561310ad6800 con 0x561311659000 2024-03-28T13:21:27.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.222+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.7 v2:172.21.15.188:6824/861796513 10 ==== osd_op_reply(32 100000001f7.00000000 [call] v123'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5613114b2500 con 0x561311659000 2024-03-28T13:21:27.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.223+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:33 17.e 17:70000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x56131166b400 con 0x561310aaf800 2024-03-28T13:21:27.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.224+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.6 v2:172.21.15.188:6816/3597358155 2 ==== osd_op_reply(33 [pgnls start_epoch 123 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561310aaf800 2024-03-28T13:21:27.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.224+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:34 17.2e 17:74000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x56131166ac00 con 0x561310aaec00 2024-03-28T13:21:27.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.224+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.3 v2:172.21.15.19:6819/3265515865 4 ==== osd_op_reply(34 [pgnls start_epoch 123 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561310aaec00 2024-03-28T13:21:27.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.224+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:35 17.1e 17:78000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae2400 con 0x561310aaf800 2024-03-28T13:21:27.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.226+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.6 v2:172.21.15.188:6816/3597358155 3 ==== osd_op_reply(35 [pgnls start_epoch 123 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561310aaf800 2024-03-28T13:21:27.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.226+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:36 17.3e 17:7c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310aafc00 con 0x561310aae800 2024-03-28T13:21:27.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.227+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.2 v2:172.21.15.19:6816/1693606238 2 ==== osd_op_reply(36 [pgnls start_epoch 123 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561310aae800 2024-03-28T13:21:27.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.227+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:37 17.1 17:80000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae2800 con 0x561310aaf400 2024-03-28T13:21:27.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.228+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 <== osd.5 v2:172.21.15.188:6800/3324247755 7 ==== osd_op_reply(37 [pgnls start_epoch 123 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2780 con 0x561310aaf400 2024-03-28T13:21:27.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.228+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:38 17.21 17:84000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae2000 con 0x561310aaf800 2024-03-28T13:21:27.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.229+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.6 v2:172.21.15.188:6816/3597358155 4 ==== osd_op_reply(38 [pgnls start_epoch 123 out=49b] v119'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561310aaf800 2024-03-28T13:21:27.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.229+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:39 17.11 17:88000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae2c00 con 0x56131166bc00 2024-03-28T13:21:27.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.230+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 <== osd.4 v2:172.21.15.188:6808/1845695132 5 ==== osd_op_reply(39 [pgnls start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2780 con 0x56131166bc00 2024-03-28T13:21:27.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.230+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:40 17.31 17:8c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae2400 con 0x561310aaf800 2024-03-28T13:21:27.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.231+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.6 v2:172.21.15.188:6816/3597358155 5 ==== osd_op_reply(40 [pgnls start_epoch 123 out=81b] v119'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5613114b2500 con 0x561310aaf800 2024-03-28T13:21:27.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.231+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:41 17.31 17:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x561310ae2800 con 0x561310aaf800 2024-03-28T13:21:27.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.232+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.6 v2:172.21.15.188:6816/3597358155 6 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5613114b2500 con 0x561310aaf800 2024-03-28T13:21:27.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.232+0000 7f32de439740 10 datascan.operator(): handling object 1099511627780.1 2024-03-28T13:21:27.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.232+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:42 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x561310ae2c00 con 0x561311658c00 2024-03-28T13:21:27.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.236+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.0 v2:172.21.15.19:6800/4106483280 5 ==== osd_op_reply(42 10000000004.00000000 [call] v123'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5613114b2000 con 0x561311658c00 2024-03-28T13:21:27.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.236+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:43 17.9 17:90000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad7400 con 0x561311659000 2024-03-28T13:21:27.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.237+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.7 v2:172.21.15.188:6824/861796513 11 ==== osd_op_reply(43 [pgnls start_epoch 123 out=49b] v119'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561311659000 2024-03-28T13:21:27.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.237+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:44 17.29 17:94000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae3800 con 0x561311658c00 2024-03-28T13:21:27.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.238+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.0 v2:172.21.15.19:6800/4106483280 6 ==== osd_op_reply(44 [pgnls start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311658c00 2024-03-28T13:21:27.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.238+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:45 17.19 17:98000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae3000 con 0x561310aaec00 2024-03-28T13:21:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.239+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.3 v2:172.21.15.19:6819/3265515865 5 ==== osd_op_reply(45 [pgnls start_epoch 123 out=81b] v121'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5613114b2000 con 0x561310aaec00 2024-03-28T13:21:27.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.239+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:46 17.19 17:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x561310ad7c00 con 0x561310aaec00 2024-03-28T13:21:27.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.240+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.3 v2:172.21.15.19:6819/3265515865 6 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5613114b2000 con 0x561310aaec00 2024-03-28T13:21:27.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.240+0000 7f32de439740 10 datascan.operator(): handling object 1099511628280.0 2024-03-28T13:21:27.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.240+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:47 17.19 17:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x561310ad6400 con 0x561310aaec00 2024-03-28T13:21:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.244+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.3 v2:172.21.15.19:6819/3265515865 7 ==== osd_op_reply(47 100000001f8.00000000 [call] v123'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5613114b2000 con 0x561310aaec00 2024-03-28T13:21:27.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.244+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:48 17.39 17:9c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad7000 con 0x561310aaec00 2024-03-28T13:21:27.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.245+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.3 v2:172.21.15.19:6819/3265515865 8 ==== osd_op_reply(48 [pgnls start_epoch 123 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561310aaec00 2024-03-28T13:21:27.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.245+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:49 17.5 17:a0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae3400 con 0x561311658c00 2024-03-28T13:21:27.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.246+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.0 v2:172.21.15.19:6800/4106483280 7 ==== osd_op_reply(49 [pgnls start_epoch 123 out=81b] v123'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5613114b2000 con 0x561311658c00 2024-03-28T13:21:27.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.246+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:50 17.5 17:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x561310aaf000 con 0x561311658c00 2024-03-28T13:21:27.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.247+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.0 v2:172.21.15.19:6800/4106483280 8 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5613114b2000 con 0x561311658c00 2024-03-28T13:21:27.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.247+0000 7f32de439740 10 datascan.operator(): handling object 1099511627780.0 2024-03-28T13:21:27.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.247+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:51 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x561310aaf000 con 0x561311658c00 2024-03-28T13:21:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.251+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.0 v2:172.21.15.19:6800/4106483280 9 ==== osd_op_reply(51 10000000004.00000000 [call] v123'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5613114b2000 con 0x561311658c00 2024-03-28T13:21:27.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.251+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:52 17.25 17:a4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310aae400 con 0x56131166bc00 2024-03-28T13:21:27.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.252+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 <== osd.4 v2:172.21.15.188:6808/1845695132 6 ==== osd_op_reply(52 [pgnls start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2780 con 0x56131166bc00 2024-03-28T13:21:27.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.253+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:53 17.15 17:a8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae3800 con 0x561310aaec00 2024-03-28T13:21:27.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.253+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.3 v2:172.21.15.19:6819/3265515865 9 ==== osd_op_reply(53 [pgnls start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561310aaec00 2024-03-28T13:21:27.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.254+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:54 17.35 17:ac000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x56131166b800 con 0x561310aaec00 2024-03-28T13:21:27.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.254+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.3 v2:172.21.15.19:6819/3265515865 10 ==== osd_op_reply(54 [pgnls start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561310aaec00 2024-03-28T13:21:27.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.255+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:55 17.d 17:b0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae3000 con 0x561311658c00 2024-03-28T13:21:27.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.256+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.0 v2:172.21.15.19:6800/4106483280 10 ==== osd_op_reply(55 [pgnls start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311658c00 2024-03-28T13:21:27.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.256+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:56 17.2d 17:b4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x56131166b400 con 0x561311659800 2024-03-28T13:21:27.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.257+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.1 v2:172.21.15.19:6801/1998544260 4 ==== osd_op_reply(56 [pgnls start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311659800 2024-03-28T13:21:27.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.257+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:57 17.1d 17:b8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae3400 con 0x561311659800 2024-03-28T13:21:27.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.258+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.1 v2:172.21.15.19:6801/1998544260 5 ==== osd_op_reply(57 [pgnls start_epoch 123 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311659800 2024-03-28T13:21:27.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.258+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:58 17.3d 17:bc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad6000 con 0x561311659800 2024-03-28T13:21:27.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.259+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.1 v2:172.21.15.19:6801/1998544260 6 ==== osd_op_reply(58 [pgnls start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311659800 2024-03-28T13:21:27.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.259+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:59 17.3 17:c0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae2c00 con 0x561311658c00 2024-03-28T13:21:27.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.260+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.0 v2:172.21.15.19:6800/4106483280 11 ==== osd_op_reply(59 [pgnls start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311658c00 2024-03-28T13:21:27.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.260+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:60 17.23 17:c4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad6800 con 0x561311658c00 2024-03-28T13:21:27.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.261+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.0 v2:172.21.15.19:6800/4106483280 12 ==== osd_op_reply(60 [pgnls start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311658c00 2024-03-28T13:21:27.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.261+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:61 17.13 17:c8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae3c00 con 0x56131166bc00 2024-03-28T13:21:27.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.262+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 <== osd.4 v2:172.21.15.188:6808/1845695132 7 ==== osd_op_reply(61 [pgnls start_epoch 123 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2780 con 0x56131166bc00 2024-03-28T13:21:27.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.263+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:62 17.33 17:cc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae3800 con 0x561310aae800 2024-03-28T13:21:27.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.264+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.2 v2:172.21.15.19:6816/1693606238 3 ==== osd_op_reply(62 [pgnls start_epoch 123 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561310aae800 2024-03-28T13:21:27.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.264+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:63 17.b 17:d0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x56131166ac00 con 0x561311659800 2024-03-28T13:21:27.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.265+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.1 v2:172.21.15.19:6801/1998544260 7 ==== osd_op_reply(63 [pgnls start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311659800 2024-03-28T13:21:27.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.265+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:64 17.2b 17:d4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae3000 con 0x561310aaec00 2024-03-28T13:21:27.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.266+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.3 v2:172.21.15.19:6819/3265515865 11 ==== osd_op_reply(64 [pgnls start_epoch 123 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561310aaec00 2024-03-28T13:21:27.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.266+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:65 17.1b 17:d8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad8000 con 0x561310aaec00 2024-03-28T13:21:27.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.266+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.3 v2:172.21.15.19:6819/3265515865 12 ==== osd_op_reply(65 [pgnls start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561310aaec00 2024-03-28T13:21:27.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.267+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:66 17.3b 17:dc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae3400 con 0x561311659000 2024-03-28T13:21:27.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.268+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.7 v2:172.21.15.188:6824/861796513 12 ==== osd_op_reply(66 [pgnls start_epoch 123 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561311659000 2024-03-28T13:21:27.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.268+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:67 17.7 17:e0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad8400 con 0x561310aaf400 2024-03-28T13:21:27.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.269+0000 7f32dbe00640 1 -- 172.21.15.19:0/913314735 <== osd.5 v2:172.21.15.188:6800/3324247755 8 ==== osd_op_reply(67 [pgnls start_epoch 123 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2780 con 0x561310aaf400 2024-03-28T13:21:27.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.269+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:68 17.27 17:e4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ae2c00 con 0x561310aaec00 2024-03-28T13:21:27.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.270+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.3 v2:172.21.15.19:6819/3265515865 13 ==== osd_op_reply(68 [pgnls start_epoch 123 out=49b] v119'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561310aaec00 2024-03-28T13:21:27.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.270+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:69 17.17 17:e8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad8800 con 0x561310aae800 2024-03-28T13:21:27.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.271+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.2 v2:172.21.15.19:6816/1693606238 4 ==== osd_op_reply(69 [pgnls start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561310aae800 2024-03-28T13:21:27.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.271+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:70 17.37 17:ec000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad6000 con 0x561310aaf800 2024-03-28T13:21:27.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.272+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.6 v2:172.21.15.188:6816/3597358155 7 ==== osd_op_reply(70 [pgnls start_epoch 123 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561310aaf800 2024-03-28T13:21:27.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.272+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:71 17.f 17:f0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad8c00 con 0x561310aaf800 2024-03-28T13:21:27.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.273+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.6 v2:172.21.15.188:6816/3597358155 8 ==== osd_op_reply(71 [pgnls start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561310aaf800 2024-03-28T13:21:27.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.274+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:72 17.2f 17:f4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x56131166b800 con 0x561311659800 2024-03-28T13:21:27.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.275+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.1 v2:172.21.15.19:6801/1998544260 8 ==== osd_op_reply(72 [pgnls start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311659800 2024-03-28T13:21:27.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.275+0000 7f32d6400640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:73 17.1f 17:f8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x561310ad9000 con 0x561311658c00 2024-03-28T13:21:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.276+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 <== osd.0 v2:172.21.15.19:6800/4106483280 13 ==== osd_op_reply(73 [pgnls start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2000 con 0x561311658c00 2024-03-28T13:21:27.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.276+0000 7f32d5a00640 1 -- 172.21.15.19:0/913314735 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:74 17.3f 17:fc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x56131166b400 con 0x561310aaf800 2024-03-28T13:21:27.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32dc800640 1 -- 172.21.15.19:0/913314735 <== osd.6 v2:172.21.15.188:6816/3597358155 9 ==== osd_op_reply(74 [pgnls start_epoch 123 out=49b] v119'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5613114b2500 con 0x561310aaf800 2024-03-28T13:21:27.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x561311658c00 msgr2=0x5613108a6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x561311658c00 0x5613108a6680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x561311659800 msgr2=0x561310abc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x561311659800 0x561310abc100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x561310aae800 msgr2=0x561310abbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x561310aae800 0x561310abbb80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x561310aaec00 msgr2=0x561310abab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x561310aaec00 0x561310abab00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x56131166bc00 msgr2=0x561310aba580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x56131166bc00 0x561310aba580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x561310aaf400 msgr2=0x561310abb080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x561310aaf400 0x561310abb080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x561310aaf800 msgr2=0x561310abb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x561310aaf800 0x561310abb600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x561311659000 msgr2=0x561310aba000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.277+0000 7f32de439740 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x561311659000 0x561310aba000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.278+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 reap_dead start 2024-03-28T13:21:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.278+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 reap_dead start 2024-03-28T13:21:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.278+0000 7f32db400640 1 -- 172.21.15.19:0/913314735 reap_dead start 2024-03-28T13:21:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.278+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x56131166b000 msgr2=0x5613108a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.278+0000 7f32de439740 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x56131166b000 0x5613108a6100 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x5613114b0cf0 tx=0x56131160cfc0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.278+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561311658400 msgr2=0x5613108a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.278+0000 7f32de439740 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561311658400 0x5613108a5600 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x561311551c20 tx=0x5613114c8ae0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.278+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 shutdown_connections 2024-03-28T13:21:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.278+0000 7f32de439740 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x56131166b000 0x5613108a6100 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.278+0000 7f32de439740 1 --2- 172.21.15.19:0/913314735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561311658400 0x5613108a5600 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.278+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 >> 172.21.15.19:0/913314735 conn(0x561311499c00 msgr2=0x5613114cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:27.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.279+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 shutdown_connections 2024-03-28T13:21:27.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.279+0000 7f32de439740 1 -- 172.21.15.19:0/913314735 wait complete. 2024-03-28T13:21:27.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.280+0000 7f32de439740 1 -- 172.21.15.19:0/820399735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561310730c00 msgr2=0x5613108a4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.280+0000 7f32de439740 1 --2- 172.21.15.19:0/820399735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561310730c00 0x5613108a4000 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x5613115510b0 tx=0x5613114c8a20 comp rx=0 tx=0).stop 2024-03-28T13:21:27.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.280+0000 7f32de439740 1 -- 172.21.15.19:0/820399735 shutdown_connections 2024-03-28T13:21:27.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.280+0000 7f32de439740 1 --2- 172.21.15.19:0/820399735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561310731c00 0x5613108a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.280+0000 7f32de439740 1 --2- 172.21.15.19:0/820399735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561310731800 0x5613108a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.280+0000 7f32de439740 1 --2- 172.21.15.19:0/820399735 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561310730c00 0x5613108a4000 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.280+0000 7f32de439740 1 -- 172.21.15.19:0/820399735 >> 172.21.15.19:0/820399735 conn(0x561310731000 msgr2=0x5613114ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:27.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.280+0000 7f32de439740 1 -- 172.21.15.19:0/820399735 shutdown_connections 2024-03-28T13:21:27.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.280+0000 7f32de439740 1 -- 172.21.15.19:0/820399735 wait complete. 2024-03-28T13:21:27.292 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.204559, result: 2024-03-28T13:21:27.292 DEBUG:teuthology.orchestra.run.smithi019:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-03-28T13:21:27.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.395+0000 7fe97a81c740 1 -- start start 2024-03-28T13:21:27.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.395+0000 7fe97a81c740 1 Processor -- start 2024-03-28T13:21:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.396+0000 7fe97a81c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc70cec00 0x558cc7242000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.396+0000 7fe97a81c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc70cf800 0x558cc7242580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.396+0000 7fe97a81c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558cc70cfc00 0x558cc7242b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.396+0000 7fe97a81c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558cc7eae180 con 0x558cc70cec00 2024-03-28T13:21:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.396+0000 7fe97a81c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558cc7eae300 con 0x558cc70cf800 2024-03-28T13:21:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.396+0000 7fe97a81c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x558cc7eae480 con 0x558cc70cfc00 2024-03-28T13:21:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.396+0000 7fe977e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc70cec00 0x558cc7242000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.397+0000 7fe977e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc70cec00 0x558cc7242000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42348/0 (socket says 172.21.15.19:42348) 2024-03-28T13:21:27.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.397+0000 7fe977e00640 1 -- 172.21.15.19:0/264598805 learned_addr learned my addr 172.21.15.19:0/264598805 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:27.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.397+0000 7fe977400640 1 --2- 172.21.15.19:0/264598805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc70cf800 0x558cc7242580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.397+0000 7fe977e00640 1 -- 172.21.15.19:0/264598805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558cc70cfc00 msgr2=0x558cc7242b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.397+0000 7fe977e00640 1 --2- 172.21.15.19:0/264598805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558cc70cfc00 0x558cc7242b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.397+0000 7fe977e00640 1 -- 172.21.15.19:0/264598805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc70cf800 msgr2=0x558cc7242580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.397+0000 7fe977e00640 1 --2- 172.21.15.19:0/264598805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc70cf800 0x558cc7242580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.397+0000 7fe977e00640 1 -- 172.21.15.19:0/264598805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc70fb680 con 0x558cc70cec00 2024-03-28T13:21:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.397+0000 7fe977400640 1 --2- 172.21.15.19:0/264598805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc70cf800 0x558cc7242580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.397+0000 7fe977e00640 1 --2- 172.21.15.19:0/264598805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc70cec00 0x558cc7242000 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x558cc7ee9a10 tx=0x558cc7e664e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66a3d4d78181f63 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.398+0000 7fe974c00640 1 -- 172.21.15.19:0/264598805 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558cc70f4680 con 0x558cc70cec00 2024-03-28T13:21:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.398+0000 7fe974c00640 1 -- 172.21.15.19:0/264598805 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558cc71181c0 con 0x558cc70cec00 2024-03-28T13:21:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.398+0000 7fe974c00640 1 -- 172.21.15.19:0/264598805 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558cc70f4b60 con 0x558cc70cec00 2024-03-28T13:21:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.398+0000 7fe97a81c740 1 -- 172.21.15.19:0/264598805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc70fb860 con 0x558cc70cec00 2024-03-28T13:21:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.398+0000 7fe97a81c740 1 -- 172.21.15.19:0/264598805 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=0}) -- 0x558cc70fb2c0 con 0x558cc70cec00 2024-03-28T13:21:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.398+0000 7fe97a81c740 4 waiting for MDS map... 2024-03-28T13:21:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.398+0000 7fe974c00640 1 -- 172.21.15.19:0/264598805 <== mon.0 v2:172.21.15.19:3300/0 4 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x558cc7f98600 con 0x558cc70cec00 2024-03-28T13:21:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.398+0000 7fe974c00640 1 -- 172.21.15.19:0/264598805 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 106) ==== 2885+0+0 (secure 0 0 0) 0x558cc7127180 con 0x558cc70cec00 2024-03-28T13:21:27.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.398+0000 7fe97a81c740 4 Got MDS map 106 2024-03-28T13:21:27.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.398+0000 7fe97a81c740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-03-28T13:21:27.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.398+0000 7fe97a81c740 4 datascan.main: Using metadata pool output 2024-03-28T13:21:27.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.398+0000 7fe97a81c740 4 datascan.main: connecting to RADOS... 2024-03-28T13:21:27.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.399+0000 7fe97a81c740 1 Processor -- start 2024-03-28T13:21:27.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.399+0000 7fe97a81c740 1 -- start start 2024-03-28T13:21:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.400+0000 7fe97a81c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558cc7f9f400 0x558cc7243080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.400+0000 7fe97a81c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc7f9f800 0x558cc7243600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.400+0000 7fe97a81c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc7f9fc00 0x558cc7243b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.400+0000 7fe97a81c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558cc7eae600 con 0x558cc7f9fc00 2024-03-28T13:21:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.400+0000 7fe97a81c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558cc7eae780 con 0x558cc7f9f800 2024-03-28T13:21:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.400+0000 7fe97a81c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x558cc7eae900 con 0x558cc7f9f400 2024-03-28T13:21:27.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.400+0000 7fe977400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc7f9fc00 0x558cc7243b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.400+0000 7fe977400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc7f9fc00 0x558cc7243b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42364/0 (socket says 172.21.15.19:42364) 2024-03-28T13:21:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.400+0000 7fe977400640 1 -- 172.21.15.19:0/3850943116 learned_addr learned my addr 172.21.15.19:0/3850943116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.401+0000 7fe978800640 1 --2- 172.21.15.19:0/3850943116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558cc7f9f400 0x558cc7243080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.401+0000 7fe977e00640 1 --2- 172.21.15.19:0/3850943116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc7f9f800 0x558cc7243600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.401+0000 7fe977400640 1 -- 172.21.15.19:0/3850943116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558cc7f9f400 msgr2=0x558cc7243080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.401+0000 7fe977400640 1 --2- 172.21.15.19:0/3850943116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558cc7f9f400 0x558cc7243080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.401+0000 7fe977400640 1 -- 172.21.15.19:0/3850943116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc7f9f800 msgr2=0x558cc7243600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.401+0000 7fe977400640 1 --2- 172.21.15.19:0/3850943116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc7f9f800 0x558cc7243600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.401+0000 7fe977400640 1 -- 172.21.15.19:0/3850943116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc70fb4a0 con 0x558cc7f9fc00 2024-03-28T13:21:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.401+0000 7fe977400640 1 --2- 172.21.15.19:0/3850943116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc7f9fc00 0x558cc7243b80 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x558cc7f96de0 tx=0x558cc7fb8fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c003465f2e01854 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.401+0000 7fe972400640 1 -- 172.21.15.19:0/3850943116 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558cc70f4680 con 0x558cc7f9fc00 2024-03-28T13:21:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.401+0000 7fe972400640 1 -- 172.21.15.19:0/3850943116 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558cc7118e00 con 0x558cc7f9fc00 2024-03-28T13:21:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.401+0000 7fe972400640 1 -- 172.21.15.19:0/3850943116 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558cc70f4b60 con 0x558cc7f9fc00 2024-03-28T13:21:27.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.402+0000 7fe97a81c740 1 -- 172.21.15.19:0/3850943116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc7f9fc00 msgr2=0x558cc7243b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.402+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3850943116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc7f9fc00 0x558cc7243b80 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x558cc7f96de0 tx=0x558cc7fb8fc0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.402+0000 7fe97a81c740 1 -- 172.21.15.19:0/3850943116 shutdown_connections 2024-03-28T13:21:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.402+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3850943116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc7f9fc00 0x558cc7243b80 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.402+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3850943116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc7f9f800 0x558cc7243600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.402+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3850943116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558cc7f9f400 0x558cc7243080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.402+0000 7fe97a81c740 1 -- 172.21.15.19:0/3850943116 >> 172.21.15.19:0/3850943116 conn(0x558cc7f9f000 msgr2=0x558cc71bb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.402+0000 7fe97a81c740 1 -- 172.21.15.19:0/3850943116 shutdown_connections 2024-03-28T13:21:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.402+0000 7fe97a81c740 1 -- 172.21.15.19:0/3850943116 wait complete. 2024-03-28T13:21:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.403+0000 7fe97a81c740 1 Processor -- start 2024-03-28T13:21:27.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.403+0000 7fe97a81c740 1 -- start start 2024-03-28T13:21:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.403+0000 7fe97a81c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc7f9f400 0x558cc7243080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.403+0000 7fe97a81c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc7f9f800 0x558cc7243600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.404+0000 7fe97a81c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558cc7f9fc00 0x558cc7243b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.404+0000 7fe97a81c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558cc7eafc80 con 0x558cc7f9f400 2024-03-28T13:21:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.404+0000 7fe97a81c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x558cc7eafb00 con 0x558cc7f9f800 2024-03-28T13:21:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.404+0000 7fe97a81c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x558cc7eaf980 con 0x558cc7f9fc00 2024-03-28T13:21:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.404+0000 7fe978800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc7f9f400 0x558cc7243080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.404+0000 7fe978800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc7f9f400 0x558cc7243080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42380/0 (socket says 172.21.15.19:42380) 2024-03-28T13:21:27.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.404+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 learned_addr learned my addr 172.21.15.19:0/3183195907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:27.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.404+0000 7fe977e00640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc7f9f800 0x558cc7243600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.404+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558cc7f9fc00 msgr2=0x558cc7243b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.404+0000 7fe978800640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558cc7f9fc00 0x558cc7243b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.404+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc7f9f800 msgr2=0x558cc7243600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.404+0000 7fe978800640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc7f9f800 0x558cc7243600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.404+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cc70fb680 con 0x558cc7f9f400 2024-03-28T13:21:27.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.404+0000 7fe978800640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc7f9f400 0x558cc7243080 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x558cc7ee9020 tx=0x558cc7fb8de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.405+0000 7fe970600640 1 -- 172.21.15.19:0/3183195907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558cc70f49c0 con 0x558cc7f9f400 2024-03-28T13:21:27.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.405+0000 7fe970600640 1 -- 172.21.15.19:0/3183195907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x558cc7118c40 con 0x558cc7f9f400 2024-03-28T13:21:27.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.405+0000 7fe970600640 1 -- 172.21.15.19:0/3183195907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x558cc70f4680 con 0x558cc7f9f400 2024-03-28T13:21:27.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.405+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558cc70fb860 con 0x558cc7f9f400 2024-03-28T13:21:27.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.405+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cc70fb0e0 con 0x558cc7f9f400 2024-03-28T13:21:27.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.405+0000 7fe97a81c740 4 datascan.init: resolving metadata pool 16 2024-03-28T13:21:27.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.407+0000 7fe970600640 1 -- 172.21.15.19:0/3183195907 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x558cc8014c00 con 0x558cc7f9f400 2024-03-28T13:21:27.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.407+0000 7fe970600640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x558cc800cc00 0x558cc7244100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.407+0000 7fe970600640 1 -- 172.21.15.19:0/3183195907 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x558cc800ec00 con 0x558cc7f9f400 2024-03-28T13:21:27.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.407+0000 7fe97a81c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-28T13:21:27.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.407+0000 7fe97a81c740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-03-28T13:21:27.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.407+0000 7fe97a81c740 4 datascan.main: opening data pool 'cephfs_data' 2024-03-28T13:21:27.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.408+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x558cc800d000 0x558cc7244680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.408+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:1 16.14 16:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x558cc800d400 con 0x558cc800d000 2024-03-28T13:21:27.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.408+0000 7fe977e00640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x558cc800cc00 0x558cc7244100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.408+0000 7fe977400640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x558cc800d000 0x558cc7244680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.409+0000 7fe977e00640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x558cc800cc00 0x558cc7244100 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x558cc7ee9d10 tx=0x558cc7e66900 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.409+0000 7fe977400640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x558cc800d000 0x558cc7244680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.410+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.4 v2:172.21.15.188:6808/1845695132 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv34 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x558cc7e50000 con 0x558cc800d000 2024-03-28T13:21:27.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.410+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x558cc800d800 0x558cc7244c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.410+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:2 16.1f 16:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x558cc800dc00 con 0x558cc800d800 2024-03-28T13:21:27.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.411+0000 7fe978800640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x558cc800d800 0x558cc7244c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.411+0000 7fe978800640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x558cc800d800 0x558cc7244c00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.411+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.0 v2:172.21.15.19:6800/4106483280 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x558cc7e50500 con 0x558cc800d800 2024-03-28T13:21:27.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.412+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:3 17.0 17:00000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7443000 con 0x558cc800d800 2024-03-28T13:21:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.412+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.0 v2:172.21.15.19:6800/4106483280 2 ==== osd_op_reply(3 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc800d800 2024-03-28T13:21:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.412+0000 7fe971a00640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x558cc7443800 0x558cc7460000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.412+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:4 17.20 17:04000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7443c00 con 0x558cc7443800 2024-03-28T13:21:27.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.413+0000 7fe977e00640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x558cc7443800 0x558cc7460000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.414+0000 7fe977e00640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x558cc7443800 0x558cc7460000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.415+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.7 v2:172.21.15.188:6824/861796513 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7443800 2024-03-28T13:21:27.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.415+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:5 17.10 17:08000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7468000 con 0x558cc800d000 2024-03-28T13:21:27.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.416+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.4 v2:172.21.15.188:6808/1845695132 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc800d000 2024-03-28T13:21:27.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.416+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:6 17.30 17:0c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7468400 con 0x558cc7443800 2024-03-28T13:21:27.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.417+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.7 v2:172.21.15.188:6824/861796513 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7443800 2024-03-28T13:21:27.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.417+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:7 17.8 17:10000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7469000 con 0x558cc800d800 2024-03-28T13:21:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.417+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.0 v2:172.21.15.19:6800/4106483280 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc800d800 2024-03-28T13:21:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.417+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:8 17.28 17:14000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7468800 con 0x558cc800d000 2024-03-28T13:21:27.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.418+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.4 v2:172.21.15.188:6808/1845695132 3 ==== osd_op_reply(8 [pgnls-filter start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc800d000 2024-03-28T13:21:27.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.418+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:9 17.18 17:18000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7469400 con 0x558cc7443800 2024-03-28T13:21:27.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.419+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.7 v2:172.21.15.188:6824/861796513 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7443800 2024-03-28T13:21:27.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.419+0000 7fe971a00640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x558cc7468c00 0x558cc7460580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.419+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7442c00 con 0x558cc7468c00 2024-03-28T13:21:27.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.419+0000 7fe977400640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x558cc7468c00 0x558cc7460580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.420+0000 7fe977400640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x558cc7468c00 0x558cc7460580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.420+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.3 v2:172.21.15.19:6819/3265515865 1 ==== osd_op_reply(10 [pgnls-filter start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7468c00 2024-03-28T13:21:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.420+0000 7fe972400640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x558cc7468400 0x558cc7461080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.420+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7469400 con 0x558cc7468400 2024-03-28T13:21:27.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.421+0000 7fe978800640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x558cc7468400 0x558cc7461080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.422+0000 7fe978800640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x558cc7468400 0x558cc7461080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.423+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.6 v2:172.21.15.188:6816/3597358155 1 ==== osd_op_reply(11 [pgnls-filter start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc7468400 2024-03-28T13:21:27.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.423+0000 7fe971a00640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x558cc7442800 0x558cc7460b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.423+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7442400 con 0x558cc7442800 2024-03-28T13:21:27.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.424+0000 7fe977e00640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x558cc7442800 0x558cc7460b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.424+0000 7fe977e00640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x558cc7442800 0x558cc7460b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.425+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.5 v2:172.21.15.188:6800/3324247755 1 ==== osd_op_reply(12 [pgnls-filter start_epoch 123 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7442800 2024-03-28T13:21:27.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.425+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7469800 con 0x558cc7442800 2024-03-28T13:21:27.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.426+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.5 v2:172.21.15.188:6800/3324247755 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 123 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7442800 2024-03-28T13:21:27.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.426+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7443c00 con 0x558cc7443800 2024-03-28T13:21:27.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.428+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.7 v2:172.21.15.188:6824/861796513 4 ==== osd_op_reply(14 [pgnls-filter start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7443800 2024-03-28T13:21:27.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.428+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7469000 con 0x558cc7468c00 2024-03-28T13:21:27.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.428+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.3 v2:172.21.15.19:6819/3265515865 2 ==== osd_op_reply(15 [pgnls-filter start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7468c00 2024-03-28T13:21:27.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.428+0000 7fe971a00640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x558cc7469400 0x558cc7461600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.428+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7469c00 con 0x558cc7469400 2024-03-28T13:21:27.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.429+0000 7fe977400640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x558cc7469400 0x558cc7461600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.430+0000 7fe977400640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x558cc7469400 0x558cc7461600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.430+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.1 v2:172.21.15.19:6801/1998544260 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7469400 2024-03-28T13:21:27.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.430+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7443000 con 0x558cc7442800 2024-03-28T13:21:27.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.431+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.5 v2:172.21.15.188:6800/3324247755 3 ==== osd_op_reply(17 [pgnls-filter start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7442800 2024-03-28T13:21:27.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.431+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7468000 con 0x558cc800d000 2024-03-28T13:21:27.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.432+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.4 v2:172.21.15.188:6808/1845695132 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc800d000 2024-03-28T13:21:27.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.432+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc800dc00 con 0x558cc7469400 2024-03-28T13:21:27.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.433+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.1 v2:172.21.15.19:6801/1998544260 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7469400 2024-03-28T13:21:27.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.433+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7468800 con 0x558cc7468c00 2024-03-28T13:21:27.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.434+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.3 v2:172.21.15.19:6819/3265515865 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7468c00 2024-03-28T13:21:27.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.434+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7e41400 con 0x558cc7443800 2024-03-28T13:21:27.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.435+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.7 v2:172.21.15.188:6824/861796513 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 123 out=81b] v123'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7443800 2024-03-28T13:21:27.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.435+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:22 17.0 17:00000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7443400 con 0x558cc800d800 2024-03-28T13:21:27.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.436+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.0 v2:172.21.15.19:6800/4106483280 4 ==== osd_op_reply(22 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc800d800 2024-03-28T13:21:27.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.436+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:23 17.20 17:04000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747c000 con 0x558cc7443800 2024-03-28T13:21:27.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.437+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.7 v2:172.21.15.188:6824/861796513 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7443800 2024-03-28T13:21:27.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.437+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:24 17.10 17:08000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7469000 con 0x558cc800d000 2024-03-28T13:21:27.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.438+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.4 v2:172.21.15.188:6808/1845695132 5 ==== osd_op_reply(24 [pgnls-filter start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc800d000 2024-03-28T13:21:27.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.438+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:25 17.30 17:0c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747c400 con 0x558cc7443800 2024-03-28T13:21:27.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.439+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.7 v2:172.21.15.188:6824/861796513 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7443800 2024-03-28T13:21:27.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.439+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:26 17.8 17:10000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7469c00 con 0x558cc800d800 2024-03-28T13:21:27.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.440+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.0 v2:172.21.15.19:6800/4106483280 5 ==== osd_op_reply(26 [pgnls-filter start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc800d800 2024-03-28T13:21:27.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.440+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:27 17.28 17:14000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747c800 con 0x558cc800d000 2024-03-28T13:21:27.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.441+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.4 v2:172.21.15.188:6808/1845695132 6 ==== osd_op_reply(27 [pgnls-filter start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc800d000 2024-03-28T13:21:27.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.441+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:28 17.18 17:18000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7442c00 con 0x558cc7443800 2024-03-28T13:21:27.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.442+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.7 v2:172.21.15.188:6824/861796513 8 ==== osd_op_reply(28 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7443800 2024-03-28T13:21:27.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.442+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:29 17.38 17:1c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747cc00 con 0x558cc7468c00 2024-03-28T13:21:27.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.443+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.3 v2:172.21.15.19:6819/3265515865 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7468c00 2024-03-28T13:21:27.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.443+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:30 17.4 17:20000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7442400 con 0x558cc7468400 2024-03-28T13:21:27.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.444+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.6 v2:172.21.15.188:6816/3597358155 2 ==== osd_op_reply(30 [pgnls-filter start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc7468400 2024-03-28T13:21:27.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.444+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:31 17.24 17:24000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747d000 con 0x558cc7442800 2024-03-28T13:21:27.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.445+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.5 v2:172.21.15.188:6800/3324247755 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 123 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7442800 2024-03-28T13:21:27.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.446+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:32 17.14 17:28000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7443c00 con 0x558cc7442800 2024-03-28T13:21:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.446+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.5 v2:172.21.15.188:6800/3324247755 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 123 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7442800 2024-03-28T13:21:27.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.447+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:33 17.34 17:2c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747c800 con 0x558cc7443800 2024-03-28T13:21:27.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.448+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.7 v2:172.21.15.188:6824/861796513 9 ==== osd_op_reply(33 [pgnls-filter start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7443800 2024-03-28T13:21:27.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.448+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:34 17.c 17:30000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7468000 con 0x558cc7468c00 2024-03-28T13:21:27.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.449+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.3 v2:172.21.15.19:6819/3265515865 5 ==== osd_op_reply(34 [pgnls-filter start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7468c00 2024-03-28T13:21:27.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.449+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:35 17.2c 17:34000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747cc00 con 0x558cc7469400 2024-03-28T13:21:27.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.449+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.1 v2:172.21.15.19:6801/1998544260 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7469400 2024-03-28T13:21:27.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.450+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:36 17.1c 17:38000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7468800 con 0x558cc7442800 2024-03-28T13:21:27.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.451+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.5 v2:172.21.15.188:6800/3324247755 6 ==== osd_op_reply(36 [pgnls-filter start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7442800 2024-03-28T13:21:27.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.451+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:37 17.3c 17:3c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747c000 con 0x558cc800d000 2024-03-28T13:21:27.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.452+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.4 v2:172.21.15.188:6808/1845695132 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc800d000 2024-03-28T13:21:27.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.452+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:38 17.2 17:40000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7469000 con 0x558cc7469400 2024-03-28T13:21:27.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.453+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.1 v2:172.21.15.19:6801/1998544260 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7469400 2024-03-28T13:21:27.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.453+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:39 17.22 17:44000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747c400 con 0x558cc7468c00 2024-03-28T13:21:27.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.454+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.3 v2:172.21.15.19:6819/3265515865 6 ==== osd_op_reply(39 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7468c00 2024-03-28T13:21:27.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.454+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:40 17.12 17:48000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7469800 con 0x558cc7443800 2024-03-28T13:21:27.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.455+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.7 v2:172.21.15.188:6824/861796513 10 ==== osd_op_reply(40 [pgnls-filter start_epoch 123 out=49b] v123'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7443800 2024-03-28T13:21:27.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.455+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:41 17.32 17:4c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7e41400 con 0x558cc800d800 2024-03-28T13:21:27.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.456+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.0 v2:172.21.15.19:6800/4106483280 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 123 out=49b] v119'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc800d800 2024-03-28T13:21:27.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.456+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:42 17.a 17:50000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc800dc00 con 0x558cc7469400 2024-03-28T13:21:27.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.457+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.1 v2:172.21.15.19:6801/1998544260 5 ==== osd_op_reply(42 [pgnls-filter start_epoch 123 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7469400 2024-03-28T13:21:27.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.457+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:43 17.2a 17:54000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7443000 con 0x558cc7442800 2024-03-28T13:21:27.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.458+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.5 v2:172.21.15.188:6800/3324247755 7 ==== osd_op_reply(43 [pgnls-filter start_epoch 123 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7442800 2024-03-28T13:21:27.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.458+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:44 17.1a 17:58000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747d400 con 0x558cc800d000 2024-03-28T13:21:27.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.459+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.4 v2:172.21.15.188:6808/1845695132 8 ==== osd_op_reply(44 [pgnls-filter start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc800d000 2024-03-28T13:21:27.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.459+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:45 17.3a 17:5c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747cc00 con 0x558cc7442800 2024-03-28T13:21:27.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.460+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.5 v2:172.21.15.188:6800/3324247755 8 ==== osd_op_reply(45 [pgnls-filter start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7442800 2024-03-28T13:21:27.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.460+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:46 17.6 17:60000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747d800 con 0x558cc800d800 2024-03-28T13:21:27.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.461+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.0 v2:172.21.15.19:6800/4106483280 7 ==== osd_op_reply(46 [pgnls-filter start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc800d800 2024-03-28T13:21:27.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.461+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:47 17.26 17:64000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747c000 con 0x558cc7442800 2024-03-28T13:21:27.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.462+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.5 v2:172.21.15.188:6800/3324247755 9 ==== osd_op_reply(47 [pgnls-filter start_epoch 123 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7442800 2024-03-28T13:21:27.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.462+0000 7fe971a00640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x558cc747dc00 0x558cc7461b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.462+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:48 17.16 17:68000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747d000 con 0x558cc747dc00 2024-03-28T13:21:27.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.463+0000 7fe978800640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x558cc747dc00 0x558cc7461b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.464+0000 7fe978800640 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x558cc747dc00 0x558cc7461b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.465+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.2 v2:172.21.15.19:6816/1693606238 1 ==== osd_op_reply(48 [pgnls-filter start_epoch 123 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc747dc00 2024-03-28T13:21:27.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.465+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:49 17.36 17:6c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747c400 con 0x558cc7443800 2024-03-28T13:21:27.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.466+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.7 v2:172.21.15.188:6824/861796513 11 ==== osd_op_reply(49 [pgnls-filter start_epoch 123 out=81b] v123'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7443800 2024-03-28T13:21:27.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.466+0000 7fe97a81c740 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-03-28T13:21:27.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.467+0000 7fe97a81c740 10 datascan.operator(): handling object 100000001f7.0 2024-03-28T13:21:27.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.467+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:50 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x558cc800c000 con 0x558cc7443800 2024-03-28T13:21:27.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.468+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.7 v2:172.21.15.188:6824/861796513 12 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7443800 2024-03-28T13:21:27.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.468+0000 7fe97a81c740 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-03-28T13:21:27.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.468+0000 7fe97a81c740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-03-28T13:21:27.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.468+0000 7fe97a81c740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-03-28T13:21:27.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.468+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:51 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x558cc800c400 con 0x558cc800d000 2024-03-28T13:21:27.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.469+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.4 v2:172.21.15.188:6808/1845695132 9 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x558cc7e50000 con 0x558cc800d000 2024-03-28T13:21:27.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.469+0000 7fe97a81c740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-03-28T13:21:27.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.469+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:52 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x558cc800d400 con 0x558cc7468400 2024-03-28T13:21:27.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.471+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.6 v2:172.21.15.188:6816/3597358155 3 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=585b] v0'0 uv12 ondisk = 0) ==== 164+0+585 (crc 0 0 0) 0x558cc7e50500 con 0x558cc7468400 2024-03-28T13:21:27.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.471+0000 7fe97a81c740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-03-28T13:21:27.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.471+0000 7fe97a81c740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-03-28T13:21:27.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.471+0000 7fe97a81c740 20 datascan.get_frag_of: frag is * 2024-03-28T13:21:27.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.471+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:53 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x558cc747d800 con 0x558cc800d000 2024-03-28T13:21:27.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.472+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.4 v2:172.21.15.188:6808/1845695132 10 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x558cc7e50000 con 0x558cc800d000 2024-03-28T13:21:27.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.472+0000 7fe97a81c740 10 datascan.find_or_create_dirfrag: read_version = 8 2024-03-28T13:21:27.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.472+0000 7fe97a81c740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-03-28T13:21:27.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.472+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:54 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x558cc747d000 con 0x558cc800d000 2024-03-28T13:21:27.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.473+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.4 v2:172.21.15.188:6808/1845695132 11 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x558cc7e50000 con 0x558cc800d000 2024-03-28T13:21:27.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.473+0000 7fe97a81c740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-03-28T13:21:27.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.473+0000 7fe97a81c740 20 datascan.read_dentry: bravo_head not found in result 2024-03-28T13:21:27.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.473+0000 7fe97a81c740 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-03-28T13:21:27.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.473+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:55 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x558cc7e41400 con 0x558cc800d000 2024-03-28T13:21:27.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.478+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.4 v2:172.21.15.188:6808/1845695132 12 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v123'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x558cc7e50000 con 0x558cc800d000 2024-03-28T13:21:27.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.478+0000 7fe97a81c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-03-28T13:21:27.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.478+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:56 17.e 17:70000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7443400 con 0x558cc7468400 2024-03-28T13:21:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.478+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.6 v2:172.21.15.188:6816/3597358155 4 ==== osd_op_reply(56 [pgnls-filter start_epoch 123 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc7468400 2024-03-28T13:21:27.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.478+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:57 17.2e 17:74000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747d400 con 0x558cc7468c00 2024-03-28T13:21:27.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.479+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.3 v2:172.21.15.19:6819/3265515865 7 ==== osd_op_reply(57 [pgnls-filter start_epoch 123 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7468c00 2024-03-28T13:21:27.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.479+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:58 17.1e 17:78000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747c800 con 0x558cc7468400 2024-03-28T13:21:27.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.481+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.6 v2:172.21.15.188:6816/3597358155 5 ==== osd_op_reply(58 [pgnls-filter start_epoch 123 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc7468400 2024-03-28T13:21:27.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.481+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:59 17.3e 17:7c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc800dc00 con 0x558cc747dc00 2024-03-28T13:21:27.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.482+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.2 v2:172.21.15.19:6816/1693606238 2 ==== osd_op_reply(59 [pgnls-filter start_epoch 123 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc747dc00 2024-03-28T13:21:27.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.482+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:60 17.1 17:80000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747cc00 con 0x558cc7442800 2024-03-28T13:21:27.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.483+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.5 v2:172.21.15.188:6800/3324247755 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 123 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7442800 2024-03-28T13:21:27.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.483+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:61 17.21 17:84000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7468000 con 0x558cc7468400 2024-03-28T13:21:27.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.485+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.6 v2:172.21.15.188:6816/3597358155 6 ==== osd_op_reply(61 [pgnls-filter start_epoch 123 out=49b] v119'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc7468400 2024-03-28T13:21:27.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.485+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:62 17.11 17:88000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7442c00 con 0x558cc800d000 2024-03-28T13:21:27.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.486+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.4 v2:172.21.15.188:6808/1845695132 13 ==== osd_op_reply(62 [pgnls-filter start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc800d000 2024-03-28T13:21:27.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.486+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:63 17.31 17:8c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7469000 con 0x558cc7468400 2024-03-28T13:21:27.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.487+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.6 v2:172.21.15.188:6816/3597358155 7 ==== osd_op_reply(63 [pgnls-filter start_epoch 123 out=49b] v119'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc7468400 2024-03-28T13:21:27.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.488+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:64 17.9 17:90000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7443c00 con 0x558cc7443800 2024-03-28T13:21:27.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.489+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.7 v2:172.21.15.188:6824/861796513 13 ==== osd_op_reply(64 [pgnls-filter start_epoch 123 out=49b] v119'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7443800 2024-03-28T13:21:27.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.489+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:65 17.29 17:94000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc748c000 con 0x558cc800d800 2024-03-28T13:21:27.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.490+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.0 v2:172.21.15.19:6800/4106483280 8 ==== osd_op_reply(65 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc800d800 2024-03-28T13:21:27.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.490+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:66 17.19 17:98000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7443000 con 0x558cc7468c00 2024-03-28T13:21:27.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.491+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.3 v2:172.21.15.19:6819/3265515865 8 ==== osd_op_reply(66 [pgnls-filter start_epoch 123 out=49b] v123'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7468c00 2024-03-28T13:21:27.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.491+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:67 17.39 17:9c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc748c400 con 0x558cc7468c00 2024-03-28T13:21:27.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.492+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.3 v2:172.21.15.19:6819/3265515865 9 ==== osd_op_reply(67 [pgnls-filter start_epoch 123 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7468c00 2024-03-28T13:21:27.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.492+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7468800 con 0x558cc800d800 2024-03-28T13:21:27.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.493+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.0 v2:172.21.15.19:6800/4106483280 9 ==== osd_op_reply(68 [pgnls-filter start_epoch 123 out=49b] v123'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc800d800 2024-03-28T13:21:27.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.493+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:69 17.25 17:a4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc748c800 con 0x558cc800d000 2024-03-28T13:21:27.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.495+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.4 v2:172.21.15.188:6808/1845695132 14 ==== osd_op_reply(69 [pgnls-filter start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc800d000 2024-03-28T13:21:27.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.495+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:70 17.15 17:a8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7469800 con 0x558cc7468c00 2024-03-28T13:21:27.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.496+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.3 v2:172.21.15.19:6819/3265515865 10 ==== osd_op_reply(70 [pgnls-filter start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7468c00 2024-03-28T13:21:27.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.496+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:71 17.35 17:ac000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc748cc00 con 0x558cc7468c00 2024-03-28T13:21:27.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.497+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.3 v2:172.21.15.19:6819/3265515865 11 ==== osd_op_reply(71 [pgnls-filter start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7468c00 2024-03-28T13:21:27.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.497+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:72 17.d 17:b0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc800c400 con 0x558cc800d800 2024-03-28T13:21:27.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.498+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.0 v2:172.21.15.19:6800/4106483280 10 ==== osd_op_reply(72 [pgnls-filter start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc800d800 2024-03-28T13:21:27.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.498+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:73 17.2d 17:b4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc748d000 con 0x558cc7469400 2024-03-28T13:21:27.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.499+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.1 v2:172.21.15.19:6801/1998544260 6 ==== osd_op_reply(73 [pgnls-filter start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7469400 2024-03-28T13:21:27.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.499+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:74 17.1d 17:b8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc800d400 con 0x558cc7469400 2024-03-28T13:21:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.500+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.1 v2:172.21.15.19:6801/1998544260 7 ==== osd_op_reply(74 [pgnls-filter start_epoch 123 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7469400 2024-03-28T13:21:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.500+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:75 17.3d 17:bc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc748d400 con 0x558cc7469400 2024-03-28T13:21:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.501+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.1 v2:172.21.15.19:6801/1998544260 8 ==== osd_op_reply(75 [pgnls-filter start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7469400 2024-03-28T13:21:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.501+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:76 17.3 17:c0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747d800 con 0x558cc800d800 2024-03-28T13:21:27.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.502+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.0 v2:172.21.15.19:6800/4106483280 11 ==== osd_op_reply(76 [pgnls-filter start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc800d800 2024-03-28T13:21:27.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.503+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:77 17.23 17:c4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc748d800 con 0x558cc800d800 2024-03-28T13:21:27.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.504+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.0 v2:172.21.15.19:6800/4106483280 12 ==== osd_op_reply(77 [pgnls-filter start_epoch 123 out=49b] v119'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc800d800 2024-03-28T13:21:27.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.504+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:78 17.13 17:c8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747d000 con 0x558cc800d000 2024-03-28T13:21:27.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.505+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.4 v2:172.21.15.188:6808/1845695132 15 ==== osd_op_reply(78 [pgnls-filter start_epoch 123 out=49b] v119'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc800d000 2024-03-28T13:21:27.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.505+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:79 17.33 17:cc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc748dc00 con 0x558cc747dc00 2024-03-28T13:21:27.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.506+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.2 v2:172.21.15.19:6816/1693606238 3 ==== osd_op_reply(79 [pgnls-filter start_epoch 123 out=49b] v119'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc747dc00 2024-03-28T13:21:27.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.506+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:80 17.b 17:d0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747d400 con 0x558cc7469400 2024-03-28T13:21:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.507+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.1 v2:172.21.15.19:6801/1998544260 9 ==== osd_op_reply(80 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7469400 2024-03-28T13:21:27.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.507+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:81 17.2b 17:d4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc748c400 con 0x558cc7468c00 2024-03-28T13:21:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.508+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.3 v2:172.21.15.19:6819/3265515865 12 ==== osd_op_reply(81 [pgnls-filter start_epoch 123 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7468c00 2024-03-28T13:21:27.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.508+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:82 17.1b 17:d8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747c800 con 0x558cc7468c00 2024-03-28T13:21:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.509+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.3 v2:172.21.15.19:6819/3265515865 13 ==== osd_op_reply(82 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7468c00 2024-03-28T13:21:27.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.509+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:83 17.3b 17:dc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc748c000 con 0x558cc7443800 2024-03-28T13:21:27.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.510+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.7 v2:172.21.15.188:6824/861796513 14 ==== osd_op_reply(83 [pgnls-filter start_epoch 123 out=49b] v119'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7443800 2024-03-28T13:21:27.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.511+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:84 17.7 17:e0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7442c00 con 0x558cc7442800 2024-03-28T13:21:27.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.512+0000 7fe977e00640 1 -- 172.21.15.19:0/3183195907 <== osd.5 v2:172.21.15.188:6800/3324247755 11 ==== osd_op_reply(84 [pgnls-filter start_epoch 123 out=49b] v119'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50780 con 0x558cc7442800 2024-03-28T13:21:27.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.512+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:85 17.27 17:e4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7468000 con 0x558cc7468c00 2024-03-28T13:21:27.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.513+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.3 v2:172.21.15.19:6819/3265515865 14 ==== osd_op_reply(85 [pgnls-filter start_epoch 123 out=49b] v119'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7468c00 2024-03-28T13:21:27.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.513+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:86 17.17 17:e8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7443000 con 0x558cc747dc00 2024-03-28T13:21:27.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.514+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.2 v2:172.21.15.19:6816/1693606238 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc747dc00 2024-03-28T13:21:27.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.514+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:87 17.37 17:ec000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7469000 con 0x558cc7468400 2024-03-28T13:21:27.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.515+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.6 v2:172.21.15.188:6816/3597358155 8 ==== osd_op_reply(87 [pgnls-filter start_epoch 123 out=49b] v119'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc7468400 2024-03-28T13:21:27.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.515+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:88 17.f 17:f0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7443400 con 0x558cc7468400 2024-03-28T13:21:27.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.516+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.6 v2:172.21.15.188:6816/3597358155 9 ==== osd_op_reply(88 [pgnls-filter start_epoch 123 out=49b] v119'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc7468400 2024-03-28T13:21:27.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.517+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:89 17.2f 17:f4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc800dc00 con 0x558cc7469400 2024-03-28T13:21:27.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.518+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 <== osd.1 v2:172.21.15.19:6801/1998544260 10 ==== osd_op_reply(89 [pgnls-filter start_epoch 123 out=49b] v119'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50000 con 0x558cc7469400 2024-03-28T13:21:27.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.518+0000 7fe971a00640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:90 17.1f 17:f8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc7e41400 con 0x558cc800d800 2024-03-28T13:21:27.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.519+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.0 v2:172.21.15.19:6800/4106483280 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 123 out=49b] v119'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc800d800 2024-03-28T13:21:27.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.519+0000 7fe972400640 1 -- 172.21.15.19:0/3183195907 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:91 17.3f 17:fc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x558cc747d800 con 0x558cc7468400 2024-03-28T13:21:27.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe978800640 1 -- 172.21.15.19:0/3183195907 <== osd.6 v2:172.21.15.188:6816/3597358155 10 ==== osd_op_reply(91 [pgnls-filter start_epoch 123 out=49b] v119'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x558cc7e50500 con 0x558cc7468400 2024-03-28T13:21:27.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x558cc800d800 msgr2=0x558cc7244c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x558cc800d800 0x558cc7244c00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x558cc7469400 msgr2=0x558cc7461600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x558cc7469400 0x558cc7461600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x558cc747dc00 msgr2=0x558cc7461b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x558cc747dc00 0x558cc7461b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x558cc7468c00 msgr2=0x558cc7460580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x558cc7468c00 0x558cc7460580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x558cc800d000 msgr2=0x558cc7244680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x558cc800d000 0x558cc7244680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x558cc7442800 msgr2=0x558cc7460b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x558cc7442800 0x558cc7460b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x558cc7468400 msgr2=0x558cc7461080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x558cc7468400 0x558cc7461080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x558cc7443800 msgr2=0x558cc7460000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x558cc7443800 0x558cc7460000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 reap_dead start 2024-03-28T13:21:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.520+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 reap_dead start 2024-03-28T13:21:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.521+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 reap_dead start 2024-03-28T13:21:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.521+0000 7fe977400640 1 -- 172.21.15.19:0/3183195907 reap_dead start 2024-03-28T13:21:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.521+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x558cc800cc00 msgr2=0x558cc7244100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.521+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x558cc800cc00 0x558cc7244100 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x558cc7ee9d10 tx=0x558cc7e66900 comp rx=0 tx=0).stop 2024-03-28T13:21:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.521+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc7f9f400 msgr2=0x558cc7243080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.521+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc7f9f400 0x558cc7243080 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x558cc7ee9020 tx=0x558cc7fb8de0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.521+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 shutdown_connections 2024-03-28T13:21:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.521+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x558cc800cc00 0x558cc7244100 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.521+0000 7fe97a81c740 1 --2- 172.21.15.19:0/3183195907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc7f9f400 0x558cc7243080 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.521+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 >> 172.21.15.19:0/3183195907 conn(0x558cc7f9f000 msgr2=0x558cc71bb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:27.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.522+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 shutdown_connections 2024-03-28T13:21:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.522+0000 7fe97a81c740 1 -- 172.21.15.19:0/3183195907 wait complete. 2024-03-28T13:21:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.523+0000 7fe97a81c740 1 -- 172.21.15.19:0/264598805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc70cec00 msgr2=0x558cc7242000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.523+0000 7fe97a81c740 1 --2- 172.21.15.19:0/264598805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc70cec00 0x558cc7242000 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x558cc7ee9a10 tx=0x558cc7e664e0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.523+0000 7fe97a81c740 1 -- 172.21.15.19:0/264598805 shutdown_connections 2024-03-28T13:21:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.523+0000 7fe97a81c740 1 --2- 172.21.15.19:0/264598805 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x558cc70cfc00 0x558cc7242b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.523+0000 7fe97a81c740 1 --2- 172.21.15.19:0/264598805 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x558cc70cf800 0x558cc7242580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.523+0000 7fe97a81c740 1 --2- 172.21.15.19:0/264598805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558cc70cec00 0x558cc7242000 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.523+0000 7fe97a81c740 1 -- 172.21.15.19:0/264598805 >> 172.21.15.19:0/264598805 conn(0x558cc70cf000 msgr2=0x558cc71ba800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.524+0000 7fe97a81c740 1 -- 172.21.15.19:0/264598805 shutdown_connections 2024-03-28T13:21:27.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.524+0000 7fe97a81c740 1 -- 172.21.15.19:0/264598805 wait complete. 2024-03-28T13:21:27.536 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.243281, result: 2024-03-28T13:21:27.536 DEBUG:teuthology.orchestra.run.smithi019:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-03-28T13:21:27.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.645+0000 7fb29f752740 1 -- start start 2024-03-28T13:21:27.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.645+0000 7fb29f752740 1 Processor -- start 2024-03-28T13:21:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.645+0000 7fb29f752740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a2cbc4c00 0x563a2d9a0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29f752740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cbc5800 0x563a2d9a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29f752740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563a2cbc5c00 0x563a2d9a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29f752740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563a2d95a180 con 0x563a2cbc5800 2024-03-28T13:21:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29f752740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563a2d95a300 con 0x563a2cbc4c00 2024-03-28T13:21:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29f752740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x563a2d95a480 con 0x563a2cbc5c00 2024-03-28T13:21:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cbc5800 0x563a2d9a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cbc5800 0x563a2d9a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42400/0 (socket says 172.21.15.19:42400) 2024-03-28T13:21:27.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29ce00640 1 -- 172.21.15.19:0/653732715 learned_addr learned my addr 172.21.15.19:0/653732715 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29d800640 1 --2- 172.21.15.19:0/653732715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a2cbc4c00 0x563a2d9a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29e200640 1 --2- 172.21.15.19:0/653732715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563a2cbc5c00 0x563a2d9a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29ce00640 1 -- 172.21.15.19:0/653732715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563a2cbc5c00 msgr2=0x563a2d9a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29ce00640 1 --2- 172.21.15.19:0/653732715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563a2cbc5c00 0x563a2d9a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29ce00640 1 -- 172.21.15.19:0/653732715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a2cbc4c00 msgr2=0x563a2d9a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29ce00640 1 --2- 172.21.15.19:0/653732715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a2cbc4c00 0x563a2d9a0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29ce00640 1 -- 172.21.15.19:0/653732715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a2cbf1680 con 0x563a2cbc5800 2024-03-28T13:21:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29e200640 1 --2- 172.21.15.19:0/653732715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563a2cbc5c00 0x563a2d9a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.646+0000 7fb29d800640 1 --2- 172.21.15.19:0/653732715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a2cbc4c00 0x563a2d9a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.647+0000 7fb29ce00640 1 --2- 172.21.15.19:0/653732715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cbc5800 0x563a2d9a0580 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x563a2d9ebb60 tx=0x563a2d95f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378d949501f5ee5a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.647+0000 7fb29a600640 1 -- 172.21.15.19:0/653732715 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563a2cbea680 con 0x563a2cbc5800 2024-03-28T13:21:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.647+0000 7fb29a600640 1 -- 172.21.15.19:0/653732715 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563a2cc0ee00 con 0x563a2cbc5800 2024-03-28T13:21:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.647+0000 7fb29a600640 1 -- 172.21.15.19:0/653732715 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563a2cbeab60 con 0x563a2cbc5800 2024-03-28T13:21:27.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.647+0000 7fb29f752740 1 -- 172.21.15.19:0/653732715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a2cbf1860 con 0x563a2cbc5800 2024-03-28T13:21:27.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.647+0000 7fb29f752740 1 -- 172.21.15.19:0/653732715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=0}) -- 0x563a2cbf12c0 con 0x563a2cbc5800 2024-03-28T13:21:27.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.647+0000 7fb29f752740 4 waiting for MDS map... 2024-03-28T13:21:27.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.647+0000 7fb29a600640 1 -- 172.21.15.19:0/653732715 <== mon.0 v2:172.21.15.19:3300/0 4 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x563a2da9e600 con 0x563a2cbc5800 2024-03-28T13:21:27.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.648+0000 7fb29a600640 1 -- 172.21.15.19:0/653732715 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 106) ==== 2885+0+0 (secure 0 0 0) 0x563a2cc1d180 con 0x563a2cbc5800 2024-03-28T13:21:27.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.648+0000 7fb29f752740 4 Got MDS map 106 2024-03-28T13:21:27.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.648+0000 7fb29f752740 4 datascan.main: Using metadata pool output 2024-03-28T13:21:27.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.648+0000 7fb29f752740 4 datascan.main: connecting to RADOS... 2024-03-28T13:21:27.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.649+0000 7fb29f752740 1 Processor -- start 2024-03-28T13:21:27.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.649+0000 7fb29f752740 1 -- start start 2024-03-28T13:21:27.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.649+0000 7fb29f752740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2daa1400 0x563a2d9a1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29f752740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a2daa1800 0x563a2d9a1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29f752740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563a2daa1c00 0x563a2d9a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29f752740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563a2d95a600 con 0x563a2daa1400 2024-03-28T13:21:27.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29f752740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563a2d95a780 con 0x563a2daa1800 2024-03-28T13:21:27.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29f752740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x563a2d95a900 con 0x563a2daa1c00 2024-03-28T13:21:27.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2daa1400 0x563a2d9a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2daa1400 0x563a2d9a1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42412/0 (socket says 172.21.15.19:42412) 2024-03-28T13:21:27.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29e200640 1 -- 172.21.15.19:0/2080873483 learned_addr learned my addr 172.21.15.19:0/2080873483 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:27.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29d800640 1 --2- 172.21.15.19:0/2080873483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a2daa1800 0x563a2d9a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29ce00640 1 --2- 172.21.15.19:0/2080873483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563a2daa1c00 0x563a2d9a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29e200640 1 -- 172.21.15.19:0/2080873483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563a2daa1c00 msgr2=0x563a2d9a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29e200640 1 --2- 172.21.15.19:0/2080873483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563a2daa1c00 0x563a2d9a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29e200640 1 -- 172.21.15.19:0/2080873483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a2daa1800 msgr2=0x563a2d9a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:27.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29e200640 1 --2- 172.21.15.19:0/2080873483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a2daa1800 0x563a2d9a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29e200640 1 -- 172.21.15.19:0/2080873483 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a2cbf14a0 con 0x563a2daa1400 2024-03-28T13:21:27.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29d800640 1 --2- 172.21.15.19:0/2080873483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a2daa1800 0x563a2d9a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:27.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.650+0000 7fb29ce00640 1 --2- 172.21.15.19:0/2080873483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563a2daa1c00 0x563a2d9a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:27.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.651+0000 7fb29e200640 1 --2- 172.21.15.19:0/2080873483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2daa1400 0x563a2d9a1080 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x563a2dab0960 tx=0x563a2dabacc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6297001a44b502e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.651+0000 7fb297e00640 1 -- 172.21.15.19:0/2080873483 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563a2cbea680 con 0x563a2daa1400 2024-03-28T13:21:27.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.651+0000 7fb297e00640 1 -- 172.21.15.19:0/2080873483 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563a2cc0e540 con 0x563a2daa1400 2024-03-28T13:21:27.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.651+0000 7fb297e00640 1 -- 172.21.15.19:0/2080873483 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563a2cbeab60 con 0x563a2daa1400 2024-03-28T13:21:27.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.651+0000 7fb29f752740 1 -- 172.21.15.19:0/2080873483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2daa1400 msgr2=0x563a2d9a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.651+0000 7fb29f752740 1 --2- 172.21.15.19:0/2080873483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2daa1400 0x563a2d9a1080 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x563a2dab0960 tx=0x563a2dabacc0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.652+0000 7fb29f752740 1 -- 172.21.15.19:0/2080873483 shutdown_connections 2024-03-28T13:21:27.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.652+0000 7fb29f752740 1 --2- 172.21.15.19:0/2080873483 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563a2daa1c00 0x563a2d9a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.652+0000 7fb29f752740 1 --2- 172.21.15.19:0/2080873483 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a2daa1800 0x563a2d9a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.652+0000 7fb29f752740 1 --2- 172.21.15.19:0/2080873483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2daa1400 0x563a2d9a1080 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.652+0000 7fb29f752740 1 -- 172.21.15.19:0/2080873483 >> 172.21.15.19:0/2080873483 conn(0x563a2daa1000 msgr2=0x563a2ccd1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:27.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.652+0000 7fb29f752740 1 -- 172.21.15.19:0/2080873483 shutdown_connections 2024-03-28T13:21:27.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.652+0000 7fb29f752740 1 -- 172.21.15.19:0/2080873483 wait complete. 2024-03-28T13:21:27.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.652+0000 7fb29f752740 1 Processor -- start 2024-03-28T13:21:27.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.653+0000 7fb29f752740 1 -- start start 2024-03-28T13:21:27.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.653+0000 7fb29f752740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563a2daa1400 0x563a2d9a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.653+0000 7fb29f752740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a2daa1800 0x563a2d9a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.653+0000 7fb29f752740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2daa1c00 0x563a2d9a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.653+0000 7fb29f752740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563a2d95bc80 con 0x563a2daa1c00 2024-03-28T13:21:27.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.653+0000 7fb29f752740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x563a2d95bb00 con 0x563a2daa1800 2024-03-28T13:21:27.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.653+0000 7fb29f752740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x563a2d95b980 con 0x563a2daa1400 2024-03-28T13:21:27.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.654+0000 7fb29ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2daa1c00 0x563a2d9a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.654+0000 7fb29ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2daa1c00 0x563a2d9a1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42420/0 (socket says 172.21.15.19:42420) 2024-03-28T13:21:27.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.654+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 learned_addr learned my addr 172.21.15.19:0/2700798815 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:27.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.654+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563a2daa1400 msgr2=0x563a2d9a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.654+0000 7fb29ce00640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563a2daa1400 0x563a2d9a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.654+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a2daa1800 msgr2=0x563a2d9a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.654+0000 7fb29ce00640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a2daa1800 0x563a2d9a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.654+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a2cbf1680 con 0x563a2daa1c00 2024-03-28T13:21:27.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.654+0000 7fb29ce00640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2daa1c00 0x563a2d9a1b80 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x563a2dab1c20 tx=0x563a2d95ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.655+0000 7fb296000640 1 -- 172.21.15.19:0/2700798815 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563a2cbea820 con 0x563a2daa1c00 2024-03-28T13:21:27.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.655+0000 7fb296000640 1 -- 172.21.15.19:0/2700798815 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x563a2cc0ec40 con 0x563a2daa1c00 2024-03-28T13:21:27.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.655+0000 7fb296000640 1 -- 172.21.15.19:0/2700798815 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563a2cbea680 con 0x563a2daa1c00 2024-03-28T13:21:27.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.655+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563a2cbf1860 con 0x563a2daa1c00 2024-03-28T13:21:27.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.655+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a2cbf10e0 con 0x563a2daa1c00 2024-03-28T13:21:27.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.655+0000 7fb29f752740 4 datascan.init: resolving metadata pool 16 2024-03-28T13:21:27.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.657+0000 7fb296000640 1 -- 172.21.15.19:0/2700798815 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x563a2dad2c00 con 0x563a2daa1c00 2024-03-28T13:21:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.657+0000 7fb296000640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x563a2db17c00 0x563a2d9a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.657+0000 7fb296000640 1 -- 172.21.15.19:0/2700798815 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x563a2db1cc00 con 0x563a2daa1c00 2024-03-28T13:21:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.657+0000 7fb29f752740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-28T13:21:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.657+0000 7fb29f752740 4 datascan.main: resolving metadata pool 16 2024-03-28T13:21:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.658+0000 7fb29f752740 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x563a2cf50000 0x563a2d9a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.658+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf50400 con 0x563a2cf50000 2024-03-28T13:21:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.658+0000 7fb29e200640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x563a2db17c00 0x563a2d9a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.658+0000 7fb29d800640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x563a2cf50000 0x563a2d9a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.659+0000 7fb29d800640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x563a2cf50000 0x563a2d9a2680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.659+0000 7fb29e200640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x563a2db17c00 0x563a2d9a2100 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x563a2dab0960 tx=0x563a2dabac00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.660+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.5 v2:172.21.15.188:6800/3324247755 1 ==== osd_op_reply(1 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944000 con 0x563a2cf50000 2024-03-28T13:21:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.660+0000 7fb29d800640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x563a2cf50400 0x563a2cf5e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.660+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:2 16.10 16:08000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2d9ad400 con 0x563a2cf50400 2024-03-28T13:21:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.661+0000 7fb29e200640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x563a2cf50400 0x563a2cf5e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.662+0000 7fb29e200640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x563a2cf50400 0x563a2cf5e000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.663+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 1 ==== osd_op_reply(2 [pgnls start_epoch 123 out=177b] v122'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.663+0000 7fb29e200640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x563a2d9ad400 0x563a2cf5e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.663+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:3 16.8 16:10000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db16800 con 0x563a2d9ad400 2024-03-28T13:21:27.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.663+0000 7fb29d800640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x563a2d9ad400 0x563a2cf5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.664+0000 7fb29d800640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x563a2d9ad400 0x563a2cf5e580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.665+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 1 ==== osd_op_reply(3 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.665+0000 7fb29d800640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x563a2db16800 0x563a2cf5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.665+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:4 16.18 16:18000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2d9ad000 con 0x563a2db16800 2024-03-28T13:21:27.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.666+0000 7fb29ce00640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x563a2db16800 0x563a2cf5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.667+0000 7fb29ce00640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x563a2db16800 0x563a2cf5eb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.668+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 1 ==== osd_op_reply(4 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.668+0000 7fb29ce00640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x563a2d9ad000 0x563a2cf5f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.668+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:5 16.4 16:20000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17400 con 0x563a2d9ad000 2024-03-28T13:21:27.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.669+0000 7fb29e200640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x563a2d9ad000 0x563a2cf5f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.669+0000 7fb29e200640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x563a2d9ad000 0x563a2cf5f600 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.670+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.3 v2:172.21.15.19:6819/3265515865 1 ==== osd_op_reply(5 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2d9ad000 2024-03-28T13:21:27.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.670+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:6 16.14 16:28000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17400 con 0x563a2d9ad400 2024-03-28T13:21:27.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.671+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 2 ==== osd_op_reply(6 [pgnls start_epoch 123 out=149b] v123'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.671+0000 7fb29d800640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x563a2db17400 0x563a2cf5f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.671+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:7 16.c 16:30000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2d9ac800 con 0x563a2db17400 2024-03-28T13:21:27.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.671+0000 7fb29d800640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x563a2db17400 0x563a2cf5f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.672+0000 7fb29d800640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x563a2db17400 0x563a2cf5f080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.672+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.0 v2:172.21.15.19:6800/4106483280 1 ==== osd_op_reply(7 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944000 con 0x563a2db17400 2024-03-28T13:21:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.672+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:8 16.1c 16:38000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2d9ac800 con 0x563a2d9ad400 2024-03-28T13:21:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.673+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 3 ==== osd_op_reply(8 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.673+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:9 16.2 16:40000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2d9ac800 con 0x563a2db17400 2024-03-28T13:21:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.673+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.0 v2:172.21.15.19:6800/4106483280 2 ==== osd_op_reply(9 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944000 con 0x563a2db17400 2024-03-28T13:21:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.674+0000 7fb29d800640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x563a2d9ac800 0x563a2cf5fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.674+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:10 16.12 16:48000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2d9acc00 con 0x563a2d9ac800 2024-03-28T13:21:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.674+0000 7fb29ce00640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x563a2d9ac800 0x563a2cf5fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.675+0000 7fb29ce00640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x563a2d9ac800 0x563a2cf5fb80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.675+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.1 v2:172.21.15.19:6801/1998544260 1 ==== osd_op_reply(10 [pgnls start_epoch 123 out=81b] v122'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x563a2d944780 con 0x563a2d9ac800 2024-03-28T13:21:27.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.675+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:11 16.a 16:50000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2d9acc00 con 0x563a2cf50400 2024-03-28T13:21:27.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.676+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 2 ==== osd_op_reply(11 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.676+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:12 16.1a 16:58000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2d9acc00 con 0x563a2cf50400 2024-03-28T13:21:27.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.677+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 3 ==== osd_op_reply(12 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.677+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:13 16.6 16:60000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2d9acc00 con 0x563a2d9ad000 2024-03-28T13:21:27.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.677+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.3 v2:172.21.15.19:6819/3265515865 2 ==== osd_op_reply(13 [pgnls start_epoch 123 out=77b] v122'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x563a2d944500 con 0x563a2d9ad000 2024-03-28T13:21:27.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.677+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:14 16.16 16:68000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2d9acc00 con 0x563a2db16800 2024-03-28T13:21:27.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.678+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 2 ==== osd_op_reply(14 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.678+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:15 16.e 16:70000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2d9acc00 con 0x563a2db16800 2024-03-28T13:21:27.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.679+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 3 ==== osd_op_reply(15 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.679+0000 7fb29ce00640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x563a2d9acc00 0x563a2cf60c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:27.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.679+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:16 16.1e 16:78000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2d9acc00 2024-03-28T13:21:27.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.680+0000 7fb29e200640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x563a2d9acc00 0x563a2cf60c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:27.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.681+0000 7fb29e200640 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x563a2d9acc00 0x563a2cf60c00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:27.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.681+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.2 v2:172.21.15.19:6816/1693606238 1 ==== osd_op_reply(16 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2d9acc00 2024-03-28T13:21:27.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.681+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:17 16.1 16:80000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2cf50400 2024-03-28T13:21:27.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.682+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 4 ==== osd_op_reply(17 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.682+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:18 16.11 16:88000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2d9acc00 2024-03-28T13:21:27.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.683+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.2 v2:172.21.15.19:6816/1693606238 2 ==== osd_op_reply(18 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2d9acc00 2024-03-28T13:21:27.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.683+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:19 16.9 16:90000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2d9ad400 2024-03-28T13:21:27.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.684+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 4 ==== osd_op_reply(19 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.684+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:20 16.19 16:98000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2d9ad000 2024-03-28T13:21:27.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.685+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.3 v2:172.21.15.19:6819/3265515865 3 ==== osd_op_reply(20 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2d9ad000 2024-03-28T13:21:27.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.685+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:21 16.5 16:a0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2d9ad400 2024-03-28T13:21:27.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.686+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 5 ==== osd_op_reply(21 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.686+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:22 16.15 16:a8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2db16800 2024-03-28T13:21:27.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.687+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 4 ==== osd_op_reply(22 [pgnls start_epoch 123 out=97b] v122'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.687+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:23 16.d 16:b0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2db17400 2024-03-28T13:21:27.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.688+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.0 v2:172.21.15.19:6800/4106483280 3 ==== osd_op_reply(23 [pgnls start_epoch 123 out=74b] v118'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x563a2d944000 con 0x563a2db17400 2024-03-28T13:21:27.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.688+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:24 16.1d 16:b8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2d9ad400 2024-03-28T13:21:27.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.689+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 6 ==== osd_op_reply(24 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.689+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:25 16.3 16:c0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2d9ad000 2024-03-28T13:21:27.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.689+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.3 v2:172.21.15.19:6819/3265515865 4 ==== osd_op_reply(25 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2d9ad000 2024-03-28T13:21:27.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.690+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:26 16.13 16:c8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2db16800 2024-03-28T13:21:27.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.691+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 5 ==== osd_op_reply(26 [pgnls start_epoch 123 out=98b] v122'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.691+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:27 16.b 16:d0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2db16800 2024-03-28T13:21:27.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.692+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 6 ==== osd_op_reply(27 [pgnls start_epoch 123 out=76b] v123'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.692+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:28 16.1b 16:d8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2d9ac800 2024-03-28T13:21:27.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.692+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.1 v2:172.21.15.19:6801/1998544260 2 ==== osd_op_reply(28 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944780 con 0x563a2d9ac800 2024-03-28T13:21:27.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.693+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:29 16.7 16:e0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2d9ad400 2024-03-28T13:21:27.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.694+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 7 ==== osd_op_reply(29 [pgnls start_epoch 123 out=105b] v123'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.694+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:30 16.17 16:e8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2d9ad400 2024-03-28T13:21:27.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.694+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 8 ==== osd_op_reply(30 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.695+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:31 16.f 16:f0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2cf50000 2024-03-28T13:21:27.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.696+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.5 v2:172.21.15.188:6800/3324247755 2 ==== osd_op_reply(31 [pgnls start_epoch 123 out=73b] v123'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563a2d944000 con 0x563a2cf50000 2024-03-28T13:21:27.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.696+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:32 16.1f 16:f8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2db17400 2024-03-28T13:21:27.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.696+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.0 v2:172.21.15.19:6800/4106483280 4 ==== osd_op_reply(32 [pgnls start_epoch 123 out=101b] v123'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x563a2d944000 con 0x563a2db17400 2024-03-28T13:21:27.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.696+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-03-28T13:21:27.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.696+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:33 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf50800 con 0x563a2cf50400 2024-03-28T13:21:27.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.697+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.697+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-03-28T13:21:27.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.697+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:34 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf50c00 con 0x563a2cf50400 2024-03-28T13:21:27.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.699+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.699+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-03-28T13:21:27.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.699+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf51000 con 0x563a2cf50400 2024-03-28T13:21:27.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.700+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.700+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-03-28T13:21:27.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.700+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:36 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf51400 con 0x563a2cf50400 2024-03-28T13:21:27.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.702+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.702+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-03-28T13:21:27.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.702+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:37 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf51800 con 0x563a2cf50400 2024-03-28T13:21:27.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.703+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.703+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-03-28T13:21:27.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.703+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-28T13:21:27.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.703+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-03-28T13:21:27.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.703+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-28T13:21:27.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.703+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-03-28T13:21:27.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.703+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-28T13:21:27.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.703+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-03-28T13:21:27.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.703+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:38 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf51c00 con 0x563a2d9ad400 2024-03-28T13:21:27.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.705+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 9 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.705+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-03-28T13:21:27.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.705+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:39 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf50800 con 0x563a2d9ad400 2024-03-28T13:21:27.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.705+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 10 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.705+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-03-28T13:21:27.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.706+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:40 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf50c00 con 0x563a2d9ac800 2024-03-28T13:21:27.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.706+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.1 v2:172.21.15.19:6801/1998544260 3 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=592b] v0'0 uv3 ondisk = 0) ==== 164+0+592 (crc 0 0 0) 0x563a2d944780 con 0x563a2d9ac800 2024-03-28T13:21:27.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.707+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-03-28T13:21:27.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.707+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-28T13:21:27.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.707+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-03-28T13:21:27.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.707+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:41 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf51000 con 0x563a2db16800 2024-03-28T13:21:27.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.708+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.708+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-03-28T13:21:27.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.708+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:42 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf51400 con 0x563a2db16800 2024-03-28T13:21:27.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.709+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 8 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.709+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-03-28T13:21:27.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.709+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-28T13:21:27.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.709+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-03-28T13:21:27.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.709+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-28T13:21:27.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.709+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-03-28T13:21:27.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.709+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:43 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf80000 con 0x563a2d9ad400 2024-03-28T13:21:27.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.710+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 11 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.711+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-03-28T13:21:27.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.711+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-28T13:21:27.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.711+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-03-28T13:21:27.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.711+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:44 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf80400 con 0x563a2db16800 2024-03-28T13:21:27.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.713+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.713+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-03-28T13:21:27.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.713+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-28T13:21:27.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.713+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-03-28T13:21:27.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.713+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:45 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf80800 con 0x563a2d9ad400 2024-03-28T13:21:27.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.713+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 12 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.714+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-03-28T13:21:27.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.714+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf80c00 con 0x563a2d9ad400 2024-03-28T13:21:27.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.715+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 13 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1683b] v0'0 uv9 ondisk = 0) ==== 164+0+1683 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.715+0000 7fb29f752740 20 datascan.scan_links: injected ino detected 2024-03-28T13:21:27.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.715+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-03-28T13:21:27.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.715+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-28T13:21:27.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.715+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-03-28T13:21:27.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.715+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-28T13:21:27.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.715+0000 7fb29f752740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-03-28T13:21:27.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.715+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:47 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81000 con 0x563a2db17400 2024-03-28T13:21:27.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.716+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.0 v2:172.21.15.19:6800/4106483280 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x563a2d944000 con 0x563a2db17400 2024-03-28T13:21:27.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.716+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:48 16.0 16:00000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2cf50000 2024-03-28T13:21:27.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.717+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.5 v2:172.21.15.188:6800/3324247755 3 ==== osd_op_reply(48 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944000 con 0x563a2cf50000 2024-03-28T13:21:27.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.717+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:49 16.10 16:08000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2cf50400 2024-03-28T13:21:27.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.718+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 10 ==== osd_op_reply(49 [pgnls start_epoch 123 out=177b] v122'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.718+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:50 16.8 16:10000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9ad400 2024-03-28T13:21:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.720+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 14 ==== osd_op_reply(50 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.720+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:51 16.18 16:18000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2db16800 2024-03-28T13:21:27.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.721+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 10 ==== osd_op_reply(51 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.721+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:52 16.4 16:20000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9ad000 2024-03-28T13:21:27.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.721+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.3 v2:172.21.15.19:6819/3265515865 5 ==== osd_op_reply(52 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2d9ad000 2024-03-28T13:21:27.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.722+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:53 16.14 16:28000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9ad400 2024-03-28T13:21:27.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.723+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 15 ==== osd_op_reply(53 [pgnls start_epoch 123 out=149b] v123'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.723+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:54 16.c 16:30000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2db17400 2024-03-28T13:21:27.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.724+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.0 v2:172.21.15.19:6800/4106483280 6 ==== osd_op_reply(54 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944000 con 0x563a2db17400 2024-03-28T13:21:27.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.724+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:55 16.1c 16:38000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9ad400 2024-03-28T13:21:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.725+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 16 ==== osd_op_reply(55 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.725+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:56 16.2 16:40000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2db17400 2024-03-28T13:21:27.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.726+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.0 v2:172.21.15.19:6800/4106483280 7 ==== osd_op_reply(56 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944000 con 0x563a2db17400 2024-03-28T13:21:27.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.726+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:57 16.12 16:48000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9ac800 2024-03-28T13:21:27.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.726+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.1 v2:172.21.15.19:6801/1998544260 4 ==== osd_op_reply(57 [pgnls start_epoch 123 out=81b] v122'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x563a2d944780 con 0x563a2d9ac800 2024-03-28T13:21:27.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.726+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:58 16.a 16:50000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2cf50400 2024-03-28T13:21:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.728+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 11 ==== osd_op_reply(58 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.728+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:59 16.1a 16:58000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2cf50400 2024-03-28T13:21:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.729+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 12 ==== osd_op_reply(59 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.729+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:60 16.6 16:60000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9ad000 2024-03-28T13:21:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.730+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.3 v2:172.21.15.19:6819/3265515865 6 ==== osd_op_reply(60 [pgnls start_epoch 123 out=77b] v122'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x563a2d944500 con 0x563a2d9ad000 2024-03-28T13:21:27.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.730+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:61 16.16 16:68000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2db16800 2024-03-28T13:21:27.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.731+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 11 ==== osd_op_reply(61 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.731+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:62 16.e 16:70000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2db16800 2024-03-28T13:21:27.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.732+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 12 ==== osd_op_reply(62 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.732+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:63 16.1e 16:78000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9acc00 2024-03-28T13:21:27.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.732+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.2 v2:172.21.15.19:6816/1693606238 3 ==== osd_op_reply(63 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2d9acc00 2024-03-28T13:21:27.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.732+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:64 16.1 16:80000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2cf50400 2024-03-28T13:21:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.734+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 13 ==== osd_op_reply(64 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.734+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- osd_op(unknown.0.0:65 16.11 16:88000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9acc00 2024-03-28T13:21:27.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.735+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.2 v2:172.21.15.19:6816/1693606238 4 ==== osd_op_reply(65 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2d9acc00 2024-03-28T13:21:27.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.735+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:66 16.9 16:90000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9ad400 2024-03-28T13:21:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.736+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 17 ==== osd_op_reply(66 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.736+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:67 16.19 16:98000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9ad000 2024-03-28T13:21:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.737+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.3 v2:172.21.15.19:6819/3265515865 7 ==== osd_op_reply(67 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2d9ad000 2024-03-28T13:21:27.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.737+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:68 16.5 16:a0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9ad400 2024-03-28T13:21:27.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.738+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 18 ==== osd_op_reply(68 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.738+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:69 16.15 16:a8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2db16800 2024-03-28T13:21:27.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.739+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 13 ==== osd_op_reply(69 [pgnls start_epoch 123 out=97b] v122'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.739+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:70 16.d 16:b0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2db17400 2024-03-28T13:21:27.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.740+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.0 v2:172.21.15.19:6800/4106483280 8 ==== osd_op_reply(70 [pgnls start_epoch 123 out=74b] v118'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x563a2d944000 con 0x563a2db17400 2024-03-28T13:21:27.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.740+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:71 16.1d 16:b8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9ad400 2024-03-28T13:21:27.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.741+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 19 ==== osd_op_reply(71 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.741+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- osd_op(unknown.0.0:72 16.3 16:c0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9ad000 2024-03-28T13:21:27.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.742+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.3 v2:172.21.15.19:6819/3265515865 8 ==== osd_op_reply(72 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944500 con 0x563a2d9ad000 2024-03-28T13:21:27.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.742+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:73 16.13 16:c8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2db16800 2024-03-28T13:21:27.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.743+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 14 ==== osd_op_reply(73 [pgnls start_epoch 123 out=98b] v122'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.743+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:74 16.b 16:d0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2db16800 2024-03-28T13:21:27.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.745+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 15 ==== osd_op_reply(74 [pgnls start_epoch 123 out=76b] v123'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.745+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:75 16.1b 16:d8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9ac800 2024-03-28T13:21:27.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.745+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.1 v2:172.21.15.19:6801/1998544260 5 ==== osd_op_reply(75 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944780 con 0x563a2d9ac800 2024-03-28T13:21:27.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.745+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:76 16.7 16:e0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9ad400 2024-03-28T13:21:27.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.747+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 20 ==== osd_op_reply(76 [pgnls start_epoch 123 out=105b] v123'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.747+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:77 16.17 16:e8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2d9ad400 2024-03-28T13:21:27.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.747+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 21 ==== osd_op_reply(77 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.748+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- osd_op(unknown.0.0:78 16.f 16:f0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2cf50000 2024-03-28T13:21:27.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.748+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.5 v2:172.21.15.188:6800/3324247755 4 ==== osd_op_reply(78 [pgnls start_epoch 123 out=73b] v123'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x563a2d944000 con 0x563a2cf50000 2024-03-28T13:21:27.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.749+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:79 16.1f 16:f8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2db17400 2024-03-28T13:21:27.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.749+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.0 v2:172.21.15.19:6800/4106483280 9 ==== osd_op_reply(79 [pgnls start_epoch 123 out=101b] v123'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x563a2d944000 con 0x563a2db17400 2024-03-28T13:21:27.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.749+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-03-28T13:21:27.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.750+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:80 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81800 con 0x563a2cf50400 2024-03-28T13:21:27.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.751+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 14 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.751+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-03-28T13:21:27.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.751+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:81 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81c00 con 0x563a2cf50400 2024-03-28T13:21:27.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.753+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 15 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.753+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-03-28T13:21:27.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.753+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:82 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf80000 con 0x563a2cf50400 2024-03-28T13:21:27.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.754+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 16 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.754+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-03-28T13:21:27.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.754+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:83 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf80800 con 0x563a2cf50400 2024-03-28T13:21:27.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.755+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 17 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.756+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-03-28T13:21:27.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.756+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- osd_op(unknown.0.0:84 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf80c00 con 0x563a2cf50400 2024-03-28T13:21:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.757+0000 7fb29e200640 1 -- 172.21.15.19:0/2700798815 <== osd.6 v2:172.21.15.188:6816/3597358155 18 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944500 con 0x563a2cf50400 2024-03-28T13:21:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.757+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-03-28T13:21:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.757+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-28T13:21:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.757+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-03-28T13:21:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.757+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-28T13:21:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.757+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-03-28T13:21:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.757+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-28T13:21:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.757+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-03-28T13:21:27.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.757+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:85 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81000 con 0x563a2d9ad400 2024-03-28T13:21:27.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.758+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 22 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.758+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-03-28T13:21:27.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.758+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:86 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf51c00 con 0x563a2d9ad400 2024-03-28T13:21:27.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.760+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 23 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.760+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-03-28T13:21:27.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.760+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- osd_op(unknown.0.0:87 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf50800 con 0x563a2d9ac800 2024-03-28T13:21:27.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.761+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.1 v2:172.21.15.19:6801/1998544260 6 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=592b] v0'0 uv3 ondisk = 0) ==== 164+0+592 (crc 0 0 0) 0x563a2d944780 con 0x563a2d9ac800 2024-03-28T13:21:27.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.761+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-03-28T13:21:27.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.761+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-28T13:21:27.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.761+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-03-28T13:21:27.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.761+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:88 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2db16800 2024-03-28T13:21:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.762+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 16 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.762+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-03-28T13:21:27.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.763+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:89 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf80400 con 0x563a2db16800 2024-03-28T13:21:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.764+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 17 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.764+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-03-28T13:21:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.764+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-28T13:21:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.764+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-03-28T13:21:27.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.764+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-28T13:21:27.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.764+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-03-28T13:21:27.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.764+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:90 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81800 con 0x563a2d9ad400 2024-03-28T13:21:27.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.765+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 24 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.765+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-03-28T13:21:27.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.765+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-28T13:21:27.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.765+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-03-28T13:21:27.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.765+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:91 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81c00 con 0x563a2db16800 2024-03-28T13:21:27.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.767+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 18 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.767+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-03-28T13:21:27.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.767+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-28T13:21:27.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.767+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-03-28T13:21:27.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.767+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:92 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf80000 con 0x563a2d9ad400 2024-03-28T13:21:27.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.768+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 25 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.768+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-03-28T13:21:27.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.768+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:93 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf50c00 con 0x563a2d9ad400 2024-03-28T13:21:27.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.769+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 26 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1683b] v0'0 uv9 ondisk = 0) ==== 164+0+1683 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.769+0000 7fb29f752740 20 datascan.scan_links: injected ino detected 2024-03-28T13:21:27.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.769+0000 7fb29f752740 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-03-28T13:21:27.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.769+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-03-28T13:21:27.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.769+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-28T13:21:27.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.769+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-03-28T13:21:27.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.769+0000 7fb29f752740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-28T13:21:27.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.769+0000 7fb29f752740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-03-28T13:21:27.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.769+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:94 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf51000 con 0x563a2db17400 2024-03-28T13:21:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.770+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.0 v2:172.21.15.19:6800/4106483280 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x563a2d944000 con 0x563a2db17400 2024-03-28T13:21:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.770+0000 7fb29f752740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-03-28T13:21:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.770+0000 7fb29f752740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-03-28T13:21:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.770+0000 7fb29f752740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-03-28T13:21:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.770+0000 7fb29f752740 10 datascan.scan_links: processing 1 injected_inos 2024-03-28T13:21:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.770+0000 7fb29f752740 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-03-28T13:21:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.770+0000 7fb29f752740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-03-28T13:21:27.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.770+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:95 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf51400 con 0x563a2d9ad400 2024-03-28T13:21:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.772+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 27 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.772+0000 7fb29f752740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-03-28T13:21:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.772+0000 7fb29f752740 20 datascan.scan_links: first is now 2 2024-03-28T13:21:27.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.772+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- osd_op(unknown.0.0:96 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf50800 con 0x563a2d9ad400 2024-03-28T13:21:27.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.776+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.4 v2:172.21.15.188:6808/1845695132 28 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v123'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x563a2d944000 con 0x563a2d9ad400 2024-03-28T13:21:27.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.776+0000 7fb29f752740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-03-28T13:21:27.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.776+0000 7fb29f752740 10 datascan.scan_links: updating inotable 2024-03-28T13:21:27.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.776+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:97 16.13 16:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf51800 con 0x563a2db16800 2024-03-28T13:21:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.777+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 19 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.777+0000 7fb29f752740 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-03-28T13:21:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.777+0000 7fb29f752740 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-03-28T13:21:27.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.777+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- osd_op(unknown.0.0:98 16.13 16:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x563a2db17800 con 0x563a2db16800 2024-03-28T13:21:27.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.782+0000 7fb29ce00640 1 -- 172.21.15.19:0/2700798815 <== osd.7 v2:172.21.15.188:6824/861796513 20 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v123'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x563a2d944780 con 0x563a2db16800 2024-03-28T13:21:27.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.782+0000 7fb29f752740 10 datascan.scan_links: updating snaptable 2024-03-28T13:21:27.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.782+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- osd_op(unknown.0.0:99 16.d 16:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x563a2cf81400 con 0x563a2db17400 2024-03-28T13:21:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 <== osd.0 v2:172.21.15.19:6800/4106483280 11 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x563a2d944000 con 0x563a2db17400 2024-03-28T13:21:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x563a2db17400 msgr2=0x563a2cf5f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x563a2db17400 0x563a2cf5f080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x563a2d9ac800 msgr2=0x563a2cf5fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x563a2d9ac800 0x563a2cf5fb80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x563a2d9acc00 msgr2=0x563a2cf60c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x563a2d9acc00 0x563a2cf60c00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x563a2d9ad000 msgr2=0x563a2cf5f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x563a2d9ad000 0x563a2cf5f600 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x563a2d9ad400 msgr2=0x563a2cf5e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x563a2d9ad400 0x563a2cf5e580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x563a2cf50000 msgr2=0x563a2d9a2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x563a2cf50000 0x563a2d9a2680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x563a2cf50400 msgr2=0x563a2cf5e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x563a2cf50400 0x563a2cf5e000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 reap_dead start 2024-03-28T13:21:27.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x563a2db16800 msgr2=0x563a2cf5eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29f752740 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x563a2db16800 0x563a2cf5eb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 reap_dead start 2024-03-28T13:21:27.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.783+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 reap_dead start 2024-03-28T13:21:27.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.784+0000 7fb29d800640 1 -- 172.21.15.19:0/2700798815 reap_dead start 2024-03-28T13:21:27.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.784+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x563a2db17c00 msgr2=0x563a2d9a2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.784+0000 7fb29f752740 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x563a2db17c00 0x563a2d9a2100 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x563a2dab0960 tx=0x563a2dabac00 comp rx=0 tx=0).stop 2024-03-28T13:21:27.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.784+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2daa1c00 msgr2=0x563a2d9a1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.784+0000 7fb29f752740 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2daa1c00 0x563a2d9a1b80 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x563a2dab1c20 tx=0x563a2d95ede0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.784+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 shutdown_connections 2024-03-28T13:21:27.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.784+0000 7fb29f752740 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x563a2db17c00 0x563a2d9a2100 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.784+0000 7fb29f752740 1 --2- 172.21.15.19:0/2700798815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2daa1c00 0x563a2d9a1b80 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.784+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 >> 172.21.15.19:0/2700798815 conn(0x563a2daa1000 msgr2=0x563a2ccd1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:27.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.784+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 shutdown_connections 2024-03-28T13:21:27.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.785+0000 7fb29f752740 1 -- 172.21.15.19:0/2700798815 wait complete. 2024-03-28T13:21:27.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.786+0000 7fb29f752740 1 -- 172.21.15.19:0/653732715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cbc5800 msgr2=0x563a2d9a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:27.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.786+0000 7fb29f752740 1 --2- 172.21.15.19:0/653732715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cbc5800 0x563a2d9a0580 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x563a2d9ebb60 tx=0x563a2d95f1a0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.786+0000 7fb29f752740 1 -- 172.21.15.19:0/653732715 shutdown_connections 2024-03-28T13:21:27.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.786+0000 7fb29f752740 1 --2- 172.21.15.19:0/653732715 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563a2cbc5c00 0x563a2d9a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.786+0000 7fb29f752740 1 --2- 172.21.15.19:0/653732715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563a2cbc5800 0x563a2d9a0580 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.786+0000 7fb29f752740 1 --2- 172.21.15.19:0/653732715 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563a2cbc4c00 0x563a2d9a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:27.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.786+0000 7fb29f752740 1 -- 172.21.15.19:0/653732715 >> 172.21.15.19:0/653732715 conn(0x563a2cbc5000 msgr2=0x563a2ccd1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:27.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.786+0000 7fb29f752740 1 -- 172.21.15.19:0/653732715 shutdown_connections 2024-03-28T13:21:27.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:27.786+0000 7fb29f752740 1 -- 172.21.15.19:0/653732715 wait complete. 2024-03-28T13:21:27.798 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.262709, result: 2024-03-28T13:21:27.799 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T13:21:27.806 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T13:21:27.977 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-03-28T13:21:27.978 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-03-28T13:21:27.978 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T13:21:27.978 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-03-28T13:21:27.978 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.978 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-03-28T13:21:27.978 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.978 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-03-28T13:21:27.978 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.978 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-03-28T13:21:27.978 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-03-28T13:21:27.978 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.978 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-03-28T13:21:27.978 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-03-28T13:21:27.978 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-03-28T13:21:27.979 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T13:21:27.980 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T13:21:27.980 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-03-28T13:21:27.980 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.980 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T13:21:27.980 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.980 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-03-28T13:21:27.980 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.980 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-03-28T13:21:27.980 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.980 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-03-28T13:21:27.980 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-03-28T13:21:27.980 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-03-28T13:21:27.980 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.19,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-03-28T13:21:27.980 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = e6b569d9-f2ce-49dc-bbe7-c0ab33080203 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-03-28T13:21:27.981 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-03-28T13:21:27.982 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.982 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-03-28T13:21:27.982 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-03-28T13:21:27.982 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.982 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-03-28T13:21:27.982 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.982 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-03-28T13:21:27.982 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.982 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-03-28T13:21:27.982 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.982 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-03-28T13:21:27.982 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-03-28T13:21:27.982 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.982 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-03-28T13:21:27.982 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-03-28T13:21:27.983 INFO:teuthology.orchestra.run.smithi188.stdout: bdev async discard = True 2024-03-28T13:21:27.983 INFO:teuthology.orchestra.run.smithi188.stdout: bdev enable discard = True 2024-03-28T13:21:27.983 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore allocator = bitmap 2024-03-28T13:21:27.983 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-03-28T13:21:27.983 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-03-28T13:21:27.983 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 1/20 2024-03-28T13:21:27.983 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 1/20 2024-03-28T13:21:27.983 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:27.983 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-03-28T13:21:27.983 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 4/10 2024-03-28T13:21:27.983 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T13:21:27.983 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-03-28T13:21:27.983 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-03-28T13:21:27.983 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2024-03-28T13:21:27.983 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-03-28T13:21:27.984 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T13:21:27.985 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.986 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-03-28T13:21:27.986 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-03-28T13:21:27.986 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.986 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-03-28T13:21:27.986 INFO:teuthology.orchestra.run.smithi188.stdout: mon down mkfs grace = 300 2024-03-28T13:21:27.986 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2024-03-28T13:21:27.986 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:27.986 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-03-28T13:21:27.986 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-03-28T13:21:27.986 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-03-28T13:21:27.986 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-03-28T13:21:27.986 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T13:21:27.986 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T13:21:27.986 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2024-03-28T13:21:27.986 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:27.987 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2024-03-28T13:21:27.988 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = False 2024-03-28T13:21:27.988 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2024-03-28T13:21:27.988 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = True 2024-03-28T13:21:27.988 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-03-28T13:21:27.988 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-03-28T13:21:27.988 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-03-28T13:21:27.988 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-03-28T13:21:27.988 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2024-03-28T13:21:27.989 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T13:21:27.989 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2024-03-28T13:21:27.989 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.989 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2024-03-28T13:21:27.989 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.989 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2024-03-28T13:21:27.989 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.989 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2024-03-28T13:21:27.989 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2024-03-28T13:21:27.989 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.989 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2024-03-28T13:21:27.989 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2024-03-28T13:21:27.989 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.989 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T13:21:27.989 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2024-03-28T13:21:27.990 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.990 INFO:teuthology.orchestra.run.smithi019.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T13:21:27.990 INFO:teuthology.orchestra.run.smithi019.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T13:21:27.990 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.990 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2024-03-28T13:21:27.990 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.990 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2024-03-28T13:21:27.990 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2024-03-28T13:21:27.990 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2024-03-28T13:21:27.990 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2024-03-28T13:21:27.990 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2024-03-28T13:21:27.990 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2024-03-28T13:21:27.990 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2024-03-28T13:21:27.990 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T13:21:27.990 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: mon_allow_pool_size_one = true 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: mon health detail to clog = false 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = "172.21.15.19,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on skipped message = False 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = e6b569d9-f2ce-49dc-bbe7-c0ab33080203 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.991 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock profile = high_recovery_ops 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug trim objects = true 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2024-03-28T13:21:27.992 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2024-03-28T13:21:27.993 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.993 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2024-03-28T13:21:27.993 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.993 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2024-03-28T13:21:27.993 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.993 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2024-03-28T13:21:27.993 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.993 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2024-03-28T13:21:27.993 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2024-03-28T13:21:27.993 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.993 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2024-03-28T13:21:27.993 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2024-03-28T13:21:27.993 INFO:teuthology.orchestra.run.smithi019.stdout: bdev async discard = True 2024-03-28T13:21:27.993 INFO:teuthology.orchestra.run.smithi019.stdout: bdev enable discard = True 2024-03-28T13:21:27.993 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore allocator = bitmap 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore block size = 96636764160 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore fsck on mount = True 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluefs = 1/20 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluestore = 1/20 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 20 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout: debug rocksdb = 4/10 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd full ratio = 0.9 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd nearfull ratio = 0.8 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout: osd failsafe full ratio = 0.95 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout: osd objectstore = bluestore 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2024-03-28T13:21:27.994 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout: mgr/telemetry/nag = false 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2024-03-28T13:21:27.995 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2024-03-28T13:21:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2024-03-28T13:21:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T13:21:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T13:21:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T13:21:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T13:21:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T13:21:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T13:21:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: # 1m isn't quite enough 2024-03-28T13:21:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: mon_down_mkfs_grace = 2m 2024-03-28T13:21:27.996 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_filestore_osds = false 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout: mon down mkfs grace = 300 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout: mon op complaint time = 120 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout: client mount timeout = 600 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = "" 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-03-28T13:21:27.997 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.b] 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.c] 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a] 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c] 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b] 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d] 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout:[mds] 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds = 20 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds balancer = 20 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug frag = True 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug scatterstat = False 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout: mds op complaint time = 180 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout: mds verify scatter = True 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-03-28T13:21:27.998 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-03-28T13:21:27.999 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-03-28T13:21:28.034 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T13:21:28.039 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-28T13:21:28.212 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-03-28T13:21:28.212 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-03-28T13:21:28.212 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T13:21:28.212 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-03-28T13:21:28.212 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.212 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-03-28T13:21:28.212 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.212 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-03-28T13:21:28.212 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.212 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-03-28T13:21:28.212 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-03-28T13:21:28.213 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-03-28T13:21:28.214 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-03-28T13:21:28.214 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-03-28T13:21:28.214 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-03-28T13:21:28.214 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T13:21:28.214 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T13:21:28.214 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-03-28T13:21:28.214 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.214 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T13:21:28.214 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.214 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-03-28T13:21:28.214 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.214 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-03-28T13:21:28.214 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.214 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-03-28T13:21:28.214 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-03-28T13:21:28.215 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-03-28T13:21:28.215 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.19,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-03-28T13:21:28.215 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2024-03-28T13:21:28.215 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = e6b569d9-f2ce-49dc-bbe7-c0ab33080203 2024-03-28T13:21:28.215 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.215 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-03-28T13:21:28.215 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-03-28T13:21:28.215 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.215 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-03-28T13:21:28.215 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-03-28T13:21:28.215 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-03-28T13:21:28.215 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.215 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-03-28T13:21:28.215 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-03-28T13:21:28.215 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-03-28T13:21:28.216 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-03-28T13:21:28.217 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.217 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-03-28T13:21:28.217 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-03-28T13:21:28.217 INFO:teuthology.orchestra.run.smithi188.stdout: bdev async discard = True 2024-03-28T13:21:28.217 INFO:teuthology.orchestra.run.smithi188.stdout: bdev enable discard = True 2024-03-28T13:21:28.217 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore allocator = bitmap 2024-03-28T13:21:28.217 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-03-28T13:21:28.217 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-03-28T13:21:28.217 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 1/20 2024-03-28T13:21:28.217 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 1/20 2024-03-28T13:21:28.217 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:28.217 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-03-28T13:21:28.217 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 4/10 2024-03-28T13:21:28.217 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T13:21:28.217 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-03-28T13:21:28.218 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-03-28T13:21:28.218 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2024-03-28T13:21:28.218 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2024-03-28T13:21:28.218 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-03-28T13:21:28.218 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.218 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-03-28T13:21:28.218 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:28.218 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-03-28T13:21:28.218 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-03-28T13:21:28.218 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-03-28T13:21:28.218 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:28.218 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:28.218 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-03-28T13:21:28.218 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-03-28T13:21:28.218 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T13:21:28.219 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.220 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T13:21:28.220 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T13:21:28.220 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T13:21:28.220 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.220 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-03-28T13:21:28.220 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-03-28T13:21:28.220 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.220 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-03-28T13:21:28.220 INFO:teuthology.orchestra.run.smithi188.stdout: mon down mkfs grace = 300 2024-03-28T13:21:28.220 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2024-03-28T13:21:28.220 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-03-28T13:21:28.220 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-03-28T13:21:28.220 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-03-28T13:21:28.220 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-03-28T13:21:28.220 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-03-28T13:21:28.221 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2024-03-28T13:21:28.222 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2024-03-28T13:21:28.222 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2024-03-28T13:21:28.222 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-03-28T13:21:28.222 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2024-03-28T13:21:28.222 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = True 2024-03-28T13:21:28.222 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2024-03-28T13:21:28.222 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = True 2024-03-28T13:21:28.222 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-03-28T13:21:28.222 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-03-28T13:21:28.222 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-03-28T13:21:28.223 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-03-28T13:21:28.223 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2024-03-28T13:21:28.223 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-28T13:21:28.223 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2024-03-28T13:21:28.223 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.223 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2024-03-28T13:21:28.223 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.224 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2024-03-28T13:21:28.224 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.224 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2024-03-28T13:21:28.224 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2024-03-28T13:21:28.224 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.224 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2024-03-28T13:21:28.224 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2024-03-28T13:21:28.224 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.224 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-28T13:21:28.224 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2024-03-28T13:21:28.224 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.224 INFO:teuthology.orchestra.run.smithi019.stdout: # disable pg_autoscaler by default for new pools 2024-03-28T13:21:28.224 INFO:teuthology.orchestra.run.smithi019.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-28T13:21:28.224 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.224 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: mon_allow_pool_size_one = true 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2024-03-28T13:21:28.225 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.226 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2024-03-28T13:21:28.226 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.226 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2024-03-28T13:21:28.226 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2024-03-28T13:21:28.226 INFO:teuthology.orchestra.run.smithi019.stdout: mon health detail to clog = false 2024-03-28T13:21:28.226 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = "172.21.15.19,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-03-28T13:21:28.226 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on skipped message = False 2024-03-28T13:21:28.226 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = e6b569d9-f2ce-49dc-bbe7-c0ab33080203 2024-03-28T13:21:28.226 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.226 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2024-03-28T13:21:28.226 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2024-03-28T13:21:28.226 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.226 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2024-03-28T13:21:28.226 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2024-03-28T13:21:28.226 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock profile = high_recovery_ops 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug trim objects = true 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2024-03-28T13:21:28.227 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: bdev async discard = True 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: bdev enable discard = True 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore allocator = bitmap 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore block size = 96636764160 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore fsck on mount = True 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluefs = 1/20 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluestore = 1/20 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:28.228 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 20 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout: debug rocksdb = 4/10 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd backfillfull_ratio = 0.85 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd full ratio = 0.9 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd nearfull ratio = 0.8 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout: osd failsafe full ratio = 0.95 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout: osd objectstore = bluestore 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:28.229 INFO:teuthology.orchestra.run.smithi019.stdout: mgr/telemetry/nag = false 2024-03-28T13:21:28.230 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-03-28T13:21:28.230 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.230 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2024-03-28T13:21:28.230 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:28.230 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-03-28T13:21:28.230 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2024-03-28T13:21:28.230 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-03-28T13:21:28.230 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2024-03-28T13:21:28.230 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2024-03-28T13:21:28.230 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-03-28T13:21:28.230 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2024-03-28T13:21:28.230 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2024-03-28T13:21:28.230 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-03-28T13:21:28.230 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.230 INFO:teuthology.orchestra.run.smithi019.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: auth mon ticket ttl = 660 # 11m 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: auth service ticket ttl = 240 # 4m 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: # don't complain about insecure global_id in the test suite 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: # 1m isn't quite enough 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: mon_down_mkfs_grace = 2m 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_filestore_osds = false 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: mon down mkfs grace = 300 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: mon op complaint time = 120 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2024-03-28T13:21:28.231 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout: client mount timeout = 600 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout: debug client = 20 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = "" 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.b] 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.c] 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a] 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c] 2024-03-28T13:21:28.232 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b] 2024-03-28T13:21:28.233 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d] 2024-03-28T13:21:28.233 INFO:teuthology.orchestra.run.smithi019.stdout:[mds] 2024-03-28T13:21:28.233 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds = 20 2024-03-28T13:21:28.233 INFO:teuthology.orchestra.run.smithi019.stdout: debug mds balancer = 20 2024-03-28T13:21:28.233 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-03-28T13:21:28.233 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug frag = True 2024-03-28T13:21:28.233 INFO:teuthology.orchestra.run.smithi019.stdout: mds debug scatterstat = True 2024-03-28T13:21:28.233 INFO:teuthology.orchestra.run.smithi019.stdout: mds op complaint time = 180 2024-03-28T13:21:28.233 INFO:teuthology.orchestra.run.smithi019.stdout: mds verify scatter = True 2024-03-28T13:21:28.233 INFO:teuthology.orchestra.run.smithi019.stdout: osd op complaint time = 180 2024-03-28T13:21:28.233 INFO:teuthology.orchestra.run.smithi019.stdout: rados mon op timeout = 900 2024-03-28T13:21:28.233 INFO:teuthology.orchestra.run.smithi019.stdout: rados osd op timeout = 900 2024-03-28T13:21:28.265 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-28T13:21:28.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.492+0000 7f794b200640 1 Processor -- start 2024-03-28T13:21:28.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.493+0000 7f794b200640 1 -- start start 2024-03-28T13:21:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.494+0000 7f794b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f794415d630 0x7f79440c7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.494+0000 7f794b200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f794415ca70 0x7f79440cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.494+0000 7f794b200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79440c7dd0 con 0x7f79440d0370 2024-03-28T13:21:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.494+0000 7f794b200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f79440d07c0 con 0x7f794415ca70 2024-03-28T13:21:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.494+0000 7f794b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f79440c80b0 con 0x7f794415d630 2024-03-28T13:21:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.495+0000 7f7949200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f79440d0370 0x7f79440ce8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42624/0 (socket says 172.21.15.19:42624) 2024-03-28T13:21:28.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.495+0000 7f7949200640 1 -- 172.21.15.19:0/3440515063 learned_addr learned my addr 172.21.15.19:0/3440515063 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.495+0000 7f7943400640 1 --2- 172.21.15.19:0/3440515063 >> v2:172.21.15.188:3300/0 conn(0x7f794415ca70 0x7f79440cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.495+0000 7f7942a00640 1 -- 172.21.15.19:0/3440515063 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3813492416 0 0) 0x7f79440c7dd0 con 0x7f79440d0370 2024-03-28T13:21:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.495+0000 7f7942a00640 1 -- 172.21.15.19:0/3440515063 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7928003520 con 0x7f79440d0370 2024-03-28T13:21:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.495+0000 7f7943e00640 1 --2- 172.21.15.19:0/3440515063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f794415d630 0x7f79440c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.496+0000 7f7942a00640 1 -- 172.21.15.19:0/3440515063 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 453215585 0 0) 0x7f7928003520 con 0x7f79440d0370 2024-03-28T13:21:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.496+0000 7f7942a00640 1 -- 172.21.15.19:0/3440515063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f794415d630 msgr2=0x7f79440c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.496+0000 7f7942a00640 1 --2- 172.21.15.19:0/3440515063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f794415d630 0x7f79440c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.496+0000 7f7942a00640 1 -- 172.21.15.19:0/3440515063 >> v2:172.21.15.188:3300/0 conn(0x7f794415ca70 msgr2=0x7f79440cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.496+0000 7f7942a00640 1 --2- 172.21.15.19:0/3440515063 >> v2:172.21.15.188:3300/0 conn(0x7f794415ca70 0x7f79440cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.496+0000 7f7942a00640 1 -- 172.21.15.19:0/3440515063 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79440c8390 con 0x7f79440d0370 2024-03-28T13:21:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.496+0000 7f7942a00640 1 -- 172.21.15.19:0/3440515063 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f793c0019f0 con 0x7f79440d0370 2024-03-28T13:21:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.496+0000 7f7942a00640 1 -- 172.21.15.19:0/3440515063 >> v1:172.21.15.19:6789/0 conn(0x7f79440d0370 legacy=0x7f79440ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:28.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.496+0000 7f7942a00640 1 --2- 172.21.15.19:0/3440515063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f794415d630 0x7f79280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:28.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.496+0000 7f7942a00640 1 --2- 172.21.15.19:0/3440515063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7928003f80 0x7f7928004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:28.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.496+0000 7f7942a00640 1 --2- 172.21.15.19:0/3440515063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7928004d40 0x7f79280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:28.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.496+0000 7f7942a00640 1 -- 172.21.15.19:0/3440515063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79440d07c0 con 0x7f7928003f80 2024-03-28T13:21:28.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.496+0000 7f7942a00640 1 -- 172.21.15.19:0/3440515063 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f79440c80b0 con 0x7f7928004d40 2024-03-28T13:21:28.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.496+0000 7f7942a00640 1 -- 172.21.15.19:0/3440515063 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f792800b9c0 con 0x7f794415d630 2024-03-28T13:21:28.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.496+0000 7f7943400640 1 --2- 172.21.15.19:0/3440515063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7928003f80 0x7f7928004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:28.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.497+0000 7f7943e00640 1 --2- 172.21.15.19:0/3440515063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f794415d630 0x7f79280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:28.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.497+0000 7f7949200640 1 --2- 172.21.15.19:0/3440515063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7928004d40 0x7f79280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:28.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.497+0000 7f7943400640 1 -- 172.21.15.19:0/3440515063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f794415d630 msgr2=0x7f79280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:28.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.497+0000 7f7943400640 1 --2- 172.21.15.19:0/3440515063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f794415d630 0x7f79280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:28.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.497+0000 7f7943400640 1 -- 172.21.15.19:0/3440515063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7928004d40 msgr2=0x7f79280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:28.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.497+0000 7f7943400640 1 --2- 172.21.15.19:0/3440515063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7928004d40 0x7f79280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:28.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.497+0000 7f7943400640 1 -- 172.21.15.19:0/3440515063 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7928003520 con 0x7f7928003f80 2024-03-28T13:21:28.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.497+0000 7f7943e00640 1 --2- 172.21.15.19:0/3440515063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f794415d630 0x7f79280039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:28.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.498+0000 7f7943400640 1 --2- 172.21.15.19:0/3440515063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7928003f80 0x7f7928004800 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f7930009ac0 tx=0x7f7928005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87308795e96b4993 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:28.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.498+0000 7f7942a00640 1 -- 172.21.15.19:0/3440515063 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930039070 con 0x7f7928003f80 2024-03-28T13:21:28.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.498+0000 7f7942a00640 1 -- 172.21.15.19:0/3440515063 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f793002c6b0 con 0x7f7928003f80 2024-03-28T13:21:28.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.498+0000 7f7942a00640 1 -- 172.21.15.19:0/3440515063 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930009690 con 0x7f7928003f80 2024-03-28T13:21:28.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.499+0000 7f794b200640 1 -- 172.21.15.19:0/3440515063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7928003f80 msgr2=0x7f7928004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:28.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.499+0000 7f794b200640 1 --2- 172.21.15.19:0/3440515063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7928003f80 0x7f7928004800 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f7930009ac0 tx=0x7f7928005830 comp rx=0 tx=0).stop 2024-03-28T13:21:28.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.499+0000 7f7949200640 1 -- 172.21.15.19:0/3440515063 reap_dead start 2024-03-28T13:21:28.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.499+0000 7f794b200640 1 -- 172.21.15.19:0/3440515063 shutdown_connections 2024-03-28T13:21:28.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.499+0000 7f794b200640 1 -- 172.21.15.19:0/3440515063 >> 172.21.15.19:0/3440515063 conn(0x7f79440b3e90 msgr2=0x7f79440b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:28.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.499+0000 7f794b200640 1 -- 172.21.15.19:0/3440515063 shutdown_connections 2024-03-28T13:21:28.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.499+0000 7f794b200640 1 -- 172.21.15.19:0/3440515063 wait complete. 2024-03-28T13:21:28.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.502+0000 7f794b200640 1 Processor -- start 2024-03-28T13:21:28.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.502+0000 7f794b200640 1 -- start start 2024-03-28T13:21:28.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.503+0000 7f794b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7928003f80 0x7f79440b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:28.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.503+0000 7f794b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79440c9330 0x7f79440c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:28.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.503+0000 7f794b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79440cb790 0x7f79440cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:28.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.503+0000 7f794b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f79440cf160 con 0x7f7928003f80 2024-03-28T13:21:28.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.503+0000 7f794b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f79440cefe0 con 0x7f79440c9330 2024-03-28T13:21:28.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.503+0000 7f794b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f79440cf2e0 con 0x7f79440cb790 2024-03-28T13:21:28.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.503+0000 7f7943e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7928003f80 0x7f79440b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:28.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.504+0000 7f7943e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7928003f80 0x7f79440b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42436/0 (socket says 172.21.15.19:42436) 2024-03-28T13:21:28.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.504+0000 7f7943e00640 1 -- 172.21.15.19:0/2139448839 learned_addr learned my addr 172.21.15.19:0/2139448839 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:28.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.504+0000 7f7949200640 1 --2- 172.21.15.19:0/2139448839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79440cb790 0x7f79440cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:28.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.504+0000 7f7943400640 1 --2- 172.21.15.19:0/2139448839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79440c9330 0x7f79440c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:28.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.504+0000 7f7943e00640 1 -- 172.21.15.19:0/2139448839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79440cb790 msgr2=0x7f79440cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:28.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.504+0000 7f7943e00640 1 --2- 172.21.15.19:0/2139448839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79440cb790 0x7f79440cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:28.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.504+0000 7f7943e00640 1 -- 172.21.15.19:0/2139448839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79440c9330 msgr2=0x7f79440c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:28.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.504+0000 7f7943e00640 1 --2- 172.21.15.19:0/2139448839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79440c9330 0x7f79440c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:28.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.504+0000 7f7943e00640 1 -- 172.21.15.19:0/2139448839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79440c9ea0 con 0x7f7928003f80 2024-03-28T13:21:28.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.505+0000 7f7943e00640 1 --2- 172.21.15.19:0/2139448839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7928003f80 0x7f79440b5d80 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f7934010910 tx=0x7f7934010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:28.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.505+0000 7f793be00640 1 -- 172.21.15.19:0/2139448839 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793400f040 con 0x7f7928003f80 2024-03-28T13:21:28.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.505+0000 7f793be00640 1 -- 172.21.15.19:0/2139448839 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f793400ebc0 con 0x7f7928003f80 2024-03-28T13:21:28.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.505+0000 7f793be00640 1 -- 172.21.15.19:0/2139448839 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793401c400 con 0x7f7928003f80 2024-03-28T13:21:28.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.505+0000 7f794b200640 1 -- 172.21.15.19:0/2139448839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79440ca180 con 0x7f7928003f80 2024-03-28T13:21:28.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.505+0000 7f794b200640 1 -- 172.21.15.19:0/2139448839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79440ca740 con 0x7f7928003f80 2024-03-28T13:21:28.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.506+0000 7f794b200640 1 -- 172.21.15.19:0/2139448839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7944068cd0 con 0x7f7928003f80 2024-03-28T13:21:28.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.507+0000 7f793be00640 1 -- 172.21.15.19:0/2139448839 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f793401c5a0 con 0x7f7928003f80 2024-03-28T13:21:28.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.508+0000 7f793be00640 1 --2- 172.21.15.19:0/2139448839 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f792406c530 0x7f792406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:28.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.508+0000 7f793be00640 1 -- 172.21.15.19:0/2139448839 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f793408bd40 con 0x7f7928003f80 2024-03-28T13:21:28.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.509+0000 7f7943400640 1 --2- 172.21.15.19:0/2139448839 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f792406c530 0x7f792406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:28.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.510+0000 7f7943400640 1 --2- 172.21.15.19:0/2139448839 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f792406c530 0x7f792406e9f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7930036000 tx=0x7f7930030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:28.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.516+0000 7f793be00640 1 -- 172.21.15.19:0/2139448839 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7934018030 con 0x7f7928003f80 2024-03-28T13:21:28.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.761+0000 7f794b200640 1 -- 172.21.15.19:0/2139448839 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f79440b4d60 con 0x7f7928003f80 2024-03-28T13:21:28.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.939+0000 7f793be00640 1 -- 172.21.15.19:0/2139448839 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v108) ==== 162+0+0 (secure 0 0 0) 0x7f793405a270 con 0x7f7928003f80 2024-03-28T13:21:28.939 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-28T13:21:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.942+0000 7f794b200640 1 -- 172.21.15.19:0/2139448839 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f792406c530 msgr2=0x7f792406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.942+0000 7f794b200640 1 --2- 172.21.15.19:0/2139448839 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f792406c530 0x7f792406e9f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7930036000 tx=0x7f7930030000 comp rx=0 tx=0).stop 2024-03-28T13:21:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.942+0000 7f794b200640 1 -- 172.21.15.19:0/2139448839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7928003f80 msgr2=0x7f79440b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.942+0000 7f794b200640 1 --2- 172.21.15.19:0/2139448839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7928003f80 0x7f79440b5d80 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f7934010910 tx=0x7f7934010dd0 comp rx=0 tx=0).stop 2024-03-28T13:21:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.943+0000 7f794b200640 1 -- 172.21.15.19:0/2139448839 shutdown_connections 2024-03-28T13:21:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.943+0000 7f794b200640 1 --2- 172.21.15.19:0/2139448839 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f792406c530 0x7f792406e9f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.943+0000 7f794b200640 1 --2- 172.21.15.19:0/2139448839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79440cb790 0x7f79440cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.943+0000 7f794b200640 1 --2- 172.21.15.19:0/2139448839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79440c9330 0x7f79440c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.943+0000 7f794b200640 1 --2- 172.21.15.19:0/2139448839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7928003f80 0x7f79440b5d80 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:28.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.943+0000 7f794b200640 1 -- 172.21.15.19:0/2139448839 >> 172.21.15.19:0/2139448839 conn(0x7f79440b3e90 msgr2=0x7f794414d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:28.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.943+0000 7f794b200640 1 -- 172.21.15.19:0/2139448839 shutdown_connections 2024-03-28T13:21:28.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:28.943+0000 7f794b200640 1 -- 172.21.15.19:0/2139448839 wait complete. 2024-03-28T13:21:28.970 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:21:29.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.180+0000 7fdfe9200640 1 Processor -- start 2024-03-28T13:21:29.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.181+0000 7fdfe9200640 1 -- start start 2024-03-28T13:21:29.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.182+0000 7fdfe9200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdfe415f900 0x7fdfe415c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:29.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.182+0000 7fdfe9200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe40d0540 0x7fdfe40ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:29.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.182+0000 7fdfe9200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfe415cd90 con 0x7fdfe415da60 2024-03-28T13:21:29.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.182+0000 7fdfe9200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fdfe415d060 con 0x7fdfe415f900 2024-03-28T13:21:29.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.182+0000 7fdfe9200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe415d340 con 0x7fdfe40d0540 2024-03-28T13:21:29.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.183+0000 7fdfe1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdfe415f900 0x7fdfe415c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:29.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.183+0000 7fdfe1e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fdfe415f900 0x7fdfe415c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42442/0 (socket says 172.21.15.19:42442) 2024-03-28T13:21:29.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.183+0000 7fdfe1e00640 1 -- 172.21.15.19:0/88251707 learned_addr learned my addr 172.21.15.19:0/88251707 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:29.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.183+0000 7fdfe1e00640 1 -- 172.21.15.19:0/88251707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe40d0540 msgr2=0x7fdfe40ce8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:29.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.183+0000 7fdfe1e00640 1 --2- 172.21.15.19:0/88251707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe40d0540 0x7fdfe40ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:29.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.183+0000 7fdfe1e00640 1 -- 172.21.15.19:0/88251707 >> v1:172.21.15.188:6789/0 conn(0x7fdfe415da60 legacy=0x7fdfe40cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:29.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.183+0000 7fdfe1e00640 1 -- 172.21.15.19:0/88251707 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe415d620 con 0x7fdfe415f900 2024-03-28T13:21:29.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.184+0000 7fdfe1e00640 1 --2- 172.21.15.19:0/88251707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe415f900 0x7fdfe415c850 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009870 tx=0x7fdfd402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a9c3a9f833360c server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:29.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.185+0000 7fdfe0a00640 1 -- 172.21.15.19:0/88251707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4035020 con 0x7fdfe415f900 2024-03-28T13:21:29.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.185+0000 7fdfe0a00640 1 -- 172.21.15.19:0/88251707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfd4031870 con 0x7fdfe415f900 2024-03-28T13:21:29.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.185+0000 7fdfe0a00640 1 -- 172.21.15.19:0/88251707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4031c40 con 0x7fdfe415f900 2024-03-28T13:21:29.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.185+0000 7fdfe9200640 1 -- 172.21.15.19:0/88251707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe415f900 msgr2=0x7fdfe415c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:29.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.185+0000 7fdfe9200640 1 --2- 172.21.15.19:0/88251707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe415f900 0x7fdfe415c850 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009870 tx=0x7fdfd402b910 comp rx=0 tx=0).stop 2024-03-28T13:21:29.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.186+0000 7fdfe9200640 1 -- 172.21.15.19:0/88251707 shutdown_connections 2024-03-28T13:21:29.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.186+0000 7fdfe9200640 1 --2- 172.21.15.19:0/88251707 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe40d0540 0x7fdfe40ce8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:29.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.186+0000 7fdfe9200640 1 --2- 172.21.15.19:0/88251707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe415f900 0x7fdfe415c850 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:29.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.186+0000 7fdfe9200640 1 -- 172.21.15.19:0/88251707 >> 172.21.15.19:0/88251707 conn(0x7fdfe40b3e90 msgr2=0x7fdfe40b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:29.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.186+0000 7fdfe9200640 1 -- 172.21.15.19:0/88251707 shutdown_connections 2024-03-28T13:21:29.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.186+0000 7fdfe9200640 1 -- 172.21.15.19:0/88251707 wait complete. 2024-03-28T13:21:29.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.188+0000 7fdfe9200640 1 Processor -- start 2024-03-28T13:21:29.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.188+0000 7fdfe9200640 1 -- start start 2024-03-28T13:21:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.189+0000 7fdfe9200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe40d0540 0x7fdfe41e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.189+0000 7fdfe9200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe415da60 0x7fdfe41e5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.189+0000 7fdfe9200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe415f900 0x7fdfe415a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.189+0000 7fdfe9200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe4160010 con 0x7fdfe40d0540 2024-03-28T13:21:29.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.189+0000 7fdfe9200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe415fe90 con 0x7fdfe415f900 2024-03-28T13:21:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.189+0000 7fdfe9200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdfe4160190 con 0x7fdfe415da60 2024-03-28T13:21:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.190+0000 7fdfe1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe40d0540 0x7fdfe41e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.190+0000 7fdfe1e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe40d0540 0x7fdfe41e76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42454/0 (socket says 172.21.15.19:42454) 2024-03-28T13:21:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.190+0000 7fdfe1e00640 1 -- 172.21.15.19:0/2031398086 learned_addr learned my addr 172.21.15.19:0/2031398086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.190+0000 7fdfe2800640 1 --2- 172.21.15.19:0/2031398086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe415f900 0x7fdfe415a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.190+0000 7fdfe1400640 1 --2- 172.21.15.19:0/2031398086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe415da60 0x7fdfe41e5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.190+0000 7fdfe1e00640 1 -- 172.21.15.19:0/2031398086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe415da60 msgr2=0x7fdfe41e5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.190+0000 7fdfe1e00640 1 --2- 172.21.15.19:0/2031398086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe415da60 0x7fdfe41e5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.190+0000 7fdfe1e00640 1 -- 172.21.15.19:0/2031398086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe415f900 msgr2=0x7fdfe415a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.190+0000 7fdfe1e00640 1 --2- 172.21.15.19:0/2031398086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe415f900 0x7fdfe415a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.190+0000 7fdfe1e00640 1 -- 172.21.15.19:0/2031398086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe41e7c30 con 0x7fdfe40d0540 2024-03-28T13:21:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.190+0000 7fdfe1400640 1 --2- 172.21.15.19:0/2031398086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe415da60 0x7fdfe41e5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:29.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.191+0000 7fdfe1e00640 1 --2- 172.21.15.19:0/2031398086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe40d0540 0x7fdfe41e76f0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4000c00 tx=0x7fdfd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.191+0000 7fdfda000640 1 -- 172.21.15.19:0/2031398086 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd403ad60 con 0x7fdfe40d0540 2024-03-28T13:21:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.191+0000 7fdfda000640 1 -- 172.21.15.19:0/2031398086 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfd40443f0 con 0x7fdfe40d0540 2024-03-28T13:21:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.191+0000 7fdfda000640 1 -- 172.21.15.19:0/2031398086 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd40447c0 con 0x7fdfe40d0540 2024-03-28T13:21:29.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.191+0000 7fdfe9200640 1 -- 172.21.15.19:0/2031398086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe41e66b0 con 0x7fdfe40d0540 2024-03-28T13:21:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.191+0000 7fdfe9200640 1 -- 172.21.15.19:0/2031398086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe41e6c70 con 0x7fdfe40d0540 2024-03-28T13:21:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.192+0000 7fdfe9200640 1 -- 172.21.15.19:0/2031398086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe4068cd0 con 0x7fdfe40d0540 2024-03-28T13:21:29.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.193+0000 7fdfda000640 1 -- 172.21.15.19:0/2031398086 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdfd404b060 con 0x7fdfe40d0540 2024-03-28T13:21:29.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.194+0000 7fdfda000640 1 --2- 172.21.15.19:0/2031398086 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdfc006c530 0x7fdfc006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:29.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.195+0000 7fdfda000640 1 -- 172.21.15.19:0/2031398086 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7fdfd40ad9c0 con 0x7fdfe40d0540 2024-03-28T13:21:29.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.195+0000 7fdfe1400640 1 --2- 172.21.15.19:0/2031398086 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdfc006c530 0x7fdfc006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:29.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.196+0000 7fdfe1400640 1 --2- 172.21.15.19:0/2031398086 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdfc006c530 0x7fdfc006e9f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4071c20 tx=0x7fdfcc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:29.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.202+0000 7fdfda000640 1 -- 172.21.15.19:0/2031398086 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdfe41e7c30 con 0x7fdfe40d0540 2024-03-28T13:21:29.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.450+0000 7fdfe9200640 1 -- 172.21.15.19:0/2031398086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfe4074900 con 0x7fdfe40d0540 2024-03-28T13:21:29.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.451+0000 7fdfda000640 1 -- 172.21.15.19:0/2031398086 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5179 (secure 0 0 0) 0x7fdfd403f070 con 0x7fdfe40d0540 2024-03-28T13:21:29.451 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:29.451 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":109,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3823671990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3823671990},{"type":"v1","addr":"172.21.15.188:6837","nonce":3823671990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/22158043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":22158043},{"type":"v1","addr":"172.21.15.188:6836","nonce":22158043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":8688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/230519675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":230519675},{"type":"v1","addr":"172.21.15.19:6834","nonce":230519675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:20:52.901269+0000","modified":"2024-03-28T13:21:28.945409+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"a","rank":0,"incarnation":109,"state":"up:replay","state_seq":1,"addr":"172.21.15.19:6835/1527816903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1527816903},{"type":"v1","addr":"172.21.15.19:6835","nonce":1527816903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-03-28T13:21:29.452 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 109 2024-03-28T13:21:29.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.454+0000 7fdfe9200640 1 -- 172.21.15.19:0/2031398086 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdfc006c530 msgr2=0x7fdfc006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:29.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.454+0000 7fdfe9200640 1 --2- 172.21.15.19:0/2031398086 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdfc006c530 0x7fdfc006e9f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4071c20 tx=0x7fdfcc009040 comp rx=0 tx=0).stop 2024-03-28T13:21:29.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.454+0000 7fdfe9200640 1 -- 172.21.15.19:0/2031398086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe40d0540 msgr2=0x7fdfe41e76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:29.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.454+0000 7fdfe9200640 1 --2- 172.21.15.19:0/2031398086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe40d0540 0x7fdfe41e76f0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4000c00 tx=0x7fdfd40046c0 comp rx=0 tx=0).stop 2024-03-28T13:21:29.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.454+0000 7fdfe9200640 1 -- 172.21.15.19:0/2031398086 shutdown_connections 2024-03-28T13:21:29.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.454+0000 7fdfe9200640 1 --2- 172.21.15.19:0/2031398086 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdfc006c530 0x7fdfc006e9f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:29.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.454+0000 7fdfe9200640 1 --2- 172.21.15.19:0/2031398086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe415f900 0x7fdfe415a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:29.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.454+0000 7fdfe9200640 1 --2- 172.21.15.19:0/2031398086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfe415da60 0x7fdfe41e5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:29.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.454+0000 7fdfe9200640 1 --2- 172.21.15.19:0/2031398086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdfe40d0540 0x7fdfe41e76f0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:29.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.454+0000 7fdfe9200640 1 -- 172.21.15.19:0/2031398086 >> 172.21.15.19:0/2031398086 conn(0x7fdfe40b3e90 msgr2=0x7fdfe40b1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:29.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.454+0000 7fdfe9200640 1 -- 172.21.15.19:0/2031398086 shutdown_connections 2024-03-28T13:21:29.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:29.454+0000 7fdfe9200640 1 -- 172.21.15.19:0/2031398086 wait complete. 2024-03-28T13:21:29.875 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:20:52.901269+0000', 'modified': '2024-03-28T13:21:28.945409+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9123}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9123': {'gid': 9123, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.19:6835/1527816903', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 1527816903}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 1527816903}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-28T13:21:29.875 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9123:up:replay 2024-03-28T13:21:30.876 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:21:31.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.108+0000 7f4ea9e00640 1 Processor -- start 2024-03-28T13:21:31.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.108+0000 7f4ea9e00640 1 -- start start 2024-03-28T13:21:31.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.110+0000 7f4ea9e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea40cc0b0 0x7f4ea406dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:31.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.110+0000 7f4ea9e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4ea415fc50 0x7f4ea40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:31.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.110+0000 7f4ea9e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ea406e4f0 con 0x7f4ea40cc490 2024-03-28T13:21:31.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.110+0000 7f4ea9e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f4ea415c610 con 0x7f4ea415fc50 2024-03-28T13:21:31.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.110+0000 7f4ea9e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea40b3150 con 0x7f4ea40cc0b0 2024-03-28T13:21:31.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.111+0000 7f4ea9400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f4ea40cc490 0x7f4ea415ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:42628/0 (socket says 172.21.15.19:42628) 2024-03-28T13:21:31.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.111+0000 7f4ea9400640 1 -- 172.21.15.19:0/2770445831 learned_addr learned my addr 172.21.15.19:0/2770445831 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.111+0000 7f4e9be00640 1 --2- 172.21.15.19:0/2770445831 >> v2:172.21.15.188:3300/0 conn(0x7f4ea415fc50 0x7f4ea40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.111+0000 7f4ea8a00640 1 --2- 172.21.15.19:0/2770445831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea40cc0b0 0x7f4ea406dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.111+0000 7f4ea3e00640 1 -- 172.21.15.19:0/2770445831 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4103087866 0 0) 0x7f4ea406e4f0 con 0x7f4ea40cc490 2024-03-28T13:21:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.111+0000 7f4ea3e00640 1 -- 172.21.15.19:0/2770445831 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e88003520 con 0x7f4ea40cc490 2024-03-28T13:21:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.111+0000 7f4ea3e00640 1 -- 172.21.15.19:0/2770445831 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2732092220 0 0) 0x7f4e88003520 con 0x7f4ea40cc490 2024-03-28T13:21:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4ea3e00640 1 -- 172.21.15.19:0/2770445831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea40cc0b0 msgr2=0x7f4ea406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4ea3e00640 1 --2- 172.21.15.19:0/2770445831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea40cc0b0 0x7f4ea406dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4ea3e00640 1 -- 172.21.15.19:0/2770445831 >> v2:172.21.15.188:3300/0 conn(0x7f4ea415fc50 msgr2=0x7f4ea40cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4ea3e00640 1 --2- 172.21.15.19:0/2770445831 >> v2:172.21.15.188:3300/0 conn(0x7f4ea415fc50 0x7f4ea40cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4ea3e00640 1 -- 172.21.15.19:0/2770445831 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea41600e0 con 0x7f4ea40cc490 2024-03-28T13:21:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4ea3e00640 1 -- 172.21.15.19:0/2770445831 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f4e9c0019f0 con 0x7f4ea40cc490 2024-03-28T13:21:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4e9be00640 1 --2- 172.21.15.19:0/2770445831 >> v2:172.21.15.188:3300/0 conn(0x7f4ea415fc50 0x7f4ea40cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:21:31.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4ea3e00640 1 -- 172.21.15.19:0/2770445831 >> v1:172.21.15.19:6789/0 conn(0x7f4ea40cc490 legacy=0x7f4ea415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4ea3e00640 1 --2- 172.21.15.19:0/2770445831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e880039b0 0x7f4e88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4ea3e00640 1 --2- 172.21.15.19:0/2770445831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e88004800 0x7f4e88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4ea3e00640 1 --2- 172.21.15.19:0/2770445831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea40cc0b0 0x7f4e880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4ea3e00640 1 -- 172.21.15.19:0/2770445831 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea415c610 con 0x7f4e880039b0 2024-03-28T13:21:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4ea3e00640 1 -- 172.21.15.19:0/2770445831 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea40b3150 con 0x7f4e88004800 2024-03-28T13:21:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4ea3e00640 1 -- 172.21.15.19:0/2770445831 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4e8800b9c0 con 0x7f4ea40cc0b0 2024-03-28T13:21:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4ea8a00640 1 --2- 172.21.15.19:0/2770445831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e880039b0 0x7f4e88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4e9be00640 1 --2- 172.21.15.19:0/2770445831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e88004800 0x7f4e88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.112+0000 7f4ea9400640 1 --2- 172.21.15.19:0/2770445831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea40cc0b0 0x7f4e880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.113+0000 7f4ea8a00640 1 -- 172.21.15.19:0/2770445831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea40cc0b0 msgr2=0x7f4e880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.113+0000 7f4ea8a00640 1 --2- 172.21.15.19:0/2770445831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea40cc0b0 0x7f4e880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.113+0000 7f4ea8a00640 1 -- 172.21.15.19:0/2770445831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e88004800 msgr2=0x7f4e88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.113+0000 7f4ea8a00640 1 --2- 172.21.15.19:0/2770445831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e88004800 0x7f4e88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.113+0000 7f4ea8a00640 1 -- 172.21.15.19:0/2770445831 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e88003520 con 0x7f4e880039b0 2024-03-28T13:21:31.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.113+0000 7f4ea9400640 1 --2- 172.21.15.19:0/2770445831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4ea40cc0b0 0x7f4e880051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.114+0000 7f4ea8a00640 1 --2- 172.21.15.19:0/2770445831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e880039b0 0x7f4e88003dd0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009ac0 tx=0x7f4e88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fa995a94ea50fe server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.114+0000 7f4ea3e00640 1 -- 172.21.15.19:0/2770445831 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e94039070 con 0x7f4e880039b0 2024-03-28T13:21:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.114+0000 7f4ea3e00640 1 -- 172.21.15.19:0/2770445831 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e9402c6b0 con 0x7f4e880039b0 2024-03-28T13:21:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.114+0000 7f4ea3e00640 1 -- 172.21.15.19:0/2770445831 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e94009690 con 0x7f4e880039b0 2024-03-28T13:21:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.115+0000 7f4ea9e00640 1 -- 172.21.15.19:0/2770445831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e880039b0 msgr2=0x7f4e88003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.115+0000 7f4ea9e00640 1 --2- 172.21.15.19:0/2770445831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e880039b0 0x7f4e88003dd0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009ac0 tx=0x7f4e88005830 comp rx=0 tx=0).stop 2024-03-28T13:21:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.115+0000 7f4ea9400640 1 -- 172.21.15.19:0/2770445831 reap_dead start 2024-03-28T13:21:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.115+0000 7f4ea9e00640 1 -- 172.21.15.19:0/2770445831 shutdown_connections 2024-03-28T13:21:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.115+0000 7f4ea9e00640 1 -- 172.21.15.19:0/2770445831 >> 172.21.15.19:0/2770445831 conn(0x7f4ea40ccad0 msgr2=0x7f4ea40cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.115+0000 7f4ea9e00640 1 -- 172.21.15.19:0/2770445831 shutdown_connections 2024-03-28T13:21:31.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.115+0000 7f4ea9e00640 1 -- 172.21.15.19:0/2770445831 wait complete. 2024-03-28T13:21:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.117+0000 7f4ea9e00640 1 Processor -- start 2024-03-28T13:21:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.117+0000 7f4ea9e00640 1 -- start start 2024-03-28T13:21:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.118+0000 7f4ea9e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e880039b0 0x7f4ea4133440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:31.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.118+0000 7f4ea9e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea41f6220 0x7f4ea4071b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:31.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.118+0000 7f4ea9e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ea4072070 0x7f4ea40b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:31.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.118+0000 7f4ea9e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea4160980 con 0x7f4ea41f6220 2024-03-28T13:21:31.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.118+0000 7f4ea9e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea41606a0 con 0x7f4ea4072070 2024-03-28T13:21:31.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.118+0000 7f4ea9e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea406dfb0 con 0x7f4e880039b0 2024-03-28T13:21:31.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.118+0000 7f4e9be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea41f6220 0x7f4ea4071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:31.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.118+0000 7f4e9be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea41f6220 0x7f4ea4071b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42472/0 (socket says 172.21.15.19:42472) 2024-03-28T13:21:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.119+0000 7f4e9be00640 1 -- 172.21.15.19:0/1697420226 learned_addr learned my addr 172.21.15.19:0/1697420226 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.119+0000 7f4ea9400640 1 --2- 172.21.15.19:0/1697420226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ea4072070 0x7f4ea40b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.119+0000 7f4ea8a00640 1 --2- 172.21.15.19:0/1697420226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e880039b0 0x7f4ea4133440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.119+0000 7f4e9be00640 1 -- 172.21.15.19:0/1697420226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e880039b0 msgr2=0x7f4ea4133440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.119+0000 7f4e9be00640 1 --2- 172.21.15.19:0/1697420226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e880039b0 0x7f4ea4133440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.119+0000 7f4e9be00640 1 -- 172.21.15.19:0/1697420226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ea4072070 msgr2=0x7f4ea40b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.119+0000 7f4e9be00640 1 --2- 172.21.15.19:0/1697420226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ea4072070 0x7f4ea40b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.119+0000 7f4e9be00640 1 -- 172.21.15.19:0/1697420226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea4157ef0 con 0x7f4ea41f6220 2024-03-28T13:21:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.119+0000 7f4e9be00640 1 --2- 172.21.15.19:0/1697420226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea41f6220 0x7f4ea4071b30 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0107e0 tx=0x7f4e8c010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.120+0000 7f4ea1600640 1 -- 172.21.15.19:0/1697420226 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c01a020 con 0x7f4ea41f6220 2024-03-28T13:21:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.120+0000 7f4ea1600640 1 -- 172.21.15.19:0/1697420226 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e8c00eb60 con 0x7f4ea41f6220 2024-03-28T13:21:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.120+0000 7f4ea9e00640 1 -- 172.21.15.19:0/1697420226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea41581d0 con 0x7f4ea41f6220 2024-03-28T13:21:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.120+0000 7f4ea1600640 1 -- 172.21.15.19:0/1697420226 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00f040 con 0x7f4ea41f6220 2024-03-28T13:21:31.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.120+0000 7f4ea9e00640 1 -- 172.21.15.19:0/1697420226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea41f66a0 con 0x7f4ea41f6220 2024-03-28T13:21:31.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.121+0000 7f4ea9e00640 1 -- 172.21.15.19:0/1697420226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea4075940 con 0x7f4ea41f6220 2024-03-28T13:21:31.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.122+0000 7f4ea1600640 1 -- 172.21.15.19:0/1697420226 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4e8c01d020 con 0x7f4ea41f6220 2024-03-28T13:21:31.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.123+0000 7f4ea1600640 1 --2- 172.21.15.19:0/1697420226 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4e8006c560 0x7f4e8006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:31.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.123+0000 7f4ea1600640 1 -- 172.21.15.19:0/1697420226 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f4e8c08c150 con 0x7f4ea41f6220 2024-03-28T13:21:31.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.124+0000 7f4ea8a00640 1 --2- 172.21.15.19:0/1697420226 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4e8006c560 0x7f4e8006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:31.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.126+0000 7f4ea8a00640 1 --2- 172.21.15.19:0/1697420226 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4e8006c560 0x7f4e8006ea20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4e94036000 tx=0x7f4e94030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:31.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.128+0000 7f4ea1600640 1 -- 172.21.15.19:0/1697420226 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4e8c024020 con 0x7f4ea41f6220 2024-03-28T13:21:31.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.378+0000 7f4ea9e00640 1 -- 172.21.15.19:0/1697420226 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ea4072450 con 0x7f4ea41f6220 2024-03-28T13:21:31.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.379+0000 7f4ea1600640 1 -- 172.21.15.19:0/1697420226 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5179 (secure 0 0 0) 0x7f4e8c05a5e0 con 0x7f4ea41f6220 2024-03-28T13:21:31.379 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:31.379 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":111,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3823671990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3823671990},{"type":"v1","addr":"172.21.15.188:6837","nonce":3823671990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/22158043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":22158043},{"type":"v1","addr":"172.21.15.188:6836","nonce":22158043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":8688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/230519675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":230519675},{"type":"v1","addr":"172.21.15.19:6834","nonce":230519675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:20:52.901269+0000","modified":"2024-03-28T13:21:29.955451+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"a","rank":0,"incarnation":109,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.19:6835/1527816903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1527816903},{"type":"v1","addr":"172.21.15.19:6835","nonce":1527816903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-03-28T13:21:31.379 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 111 2024-03-28T13:21:31.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.382+0000 7f4ea9e00640 1 -- 172.21.15.19:0/1697420226 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4e8006c560 msgr2=0x7f4e8006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:31.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.382+0000 7f4ea9e00640 1 --2- 172.21.15.19:0/1697420226 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4e8006c560 0x7f4e8006ea20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4e94036000 tx=0x7f4e94030000 comp rx=0 tx=0).stop 2024-03-28T13:21:31.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.382+0000 7f4ea9e00640 1 -- 172.21.15.19:0/1697420226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea41f6220 msgr2=0x7f4ea4071b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.382+0000 7f4ea9e00640 1 --2- 172.21.15.19:0/1697420226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea41f6220 0x7f4ea4071b30 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0107e0 tx=0x7f4e8c010ca0 comp rx=0 tx=0).stop 2024-03-28T13:21:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.382+0000 7f4ea9e00640 1 -- 172.21.15.19:0/1697420226 shutdown_connections 2024-03-28T13:21:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.382+0000 7f4ea9e00640 1 --2- 172.21.15.19:0/1697420226 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4e8006c560 0x7f4e8006ea20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.382+0000 7f4ea9e00640 1 --2- 172.21.15.19:0/1697420226 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4ea4072070 0x7f4ea40b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.382+0000 7f4ea9e00640 1 --2- 172.21.15.19:0/1697420226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4ea41f6220 0x7f4ea4071b30 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.382+0000 7f4ea9e00640 1 --2- 172.21.15.19:0/1697420226 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e880039b0 0x7f4ea4133440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.382+0000 7f4ea9e00640 1 -- 172.21.15.19:0/1697420226 >> 172.21.15.19:0/1697420226 conn(0x7f4ea40ccad0 msgr2=0x7f4ea40712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.382+0000 7f4ea9e00640 1 -- 172.21.15.19:0/1697420226 shutdown_connections 2024-03-28T13:21:31.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:31.382+0000 7f4ea9e00640 1 -- 172.21.15.19:0/1697420226 wait complete. 2024-03-28T13:21:31.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:20:52.901269+0000', 'modified': '2024-03-28T13:21:29.955451+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9123}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9123': {'gid': 9123, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.19:6835/1527816903', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 1527816903}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 1527816903}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-03-28T13:21:31.805 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9123:up:rejoin 2024-03-28T13:21:32.806 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:21:32.940 INFO:tasks.ceph.mds.a.smithi019.stderr:2024-03-28T13:21:32.941+0000 7f4ca0000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:21:33.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.038+0000 7f0b5a400640 1 Processor -- start 2024-03-28T13:21:33.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.038+0000 7f0b5a400640 1 -- start start 2024-03-28T13:21:33.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.039+0000 7f0b5a400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0b540cc0b0 0x7f0b5406dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.039+0000 7f0b5a400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b5415fc50 0x7f0b540cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.040+0000 7f0b5a400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b540cc490 0x7f0b540d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.040+0000 7f0b5a400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0b5415c610 con 0x7f0b5415fc50 2024-03-28T13:21:33.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.040+0000 7f0b5a400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f0b5406e4f0 con 0x7f0b540cc0b0 2024-03-28T13:21:33.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.040+0000 7f0b5a400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0b540b3150 con 0x7f0b540cc490 2024-03-28T13:21:33.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.040+0000 7f0b53e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b5415fc50 0x7f0b540cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.040+0000 7f0b53e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0b5415fc50 0x7f0b540cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42478/0 (socket says 172.21.15.19:42478) 2024-03-28T13:21:33.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.040+0000 7f0b53e00640 1 -- 172.21.15.19:0/2626964809 learned_addr learned my addr 172.21.15.19:0/2626964809 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:33.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.040+0000 7f0b59000640 1 --2- 172.21.15.19:0/2626964809 >> v2:172.21.15.188:3300/0 conn(0x7f0b540cc0b0 0x7f0b5406dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.040+0000 7f0b53e00640 1 -- 172.21.15.19:0/2626964809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b540cc490 msgr2=0x7f0b540d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:33.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.040+0000 7f0b59a00640 1 --2- 172.21.15.19:0/2626964809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b540cc490 0x7f0b540d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.041+0000 7f0b53e00640 1 --2- 172.21.15.19:0/2626964809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b540cc490 0x7f0b540d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.041+0000 7f0b53e00640 1 -- 172.21.15.19:0/2626964809 >> v2:172.21.15.188:3300/0 conn(0x7f0b540cc0b0 msgr2=0x7f0b5406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:33.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.041+0000 7f0b53e00640 1 --2- 172.21.15.19:0/2626964809 >> v2:172.21.15.188:3300/0 conn(0x7f0b540cc0b0 0x7f0b5406dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.041+0000 7f0b53e00640 1 -- 172.21.15.19:0/2626964809 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b541600e0 con 0x7f0b5415fc50 2024-03-28T13:21:33.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.041+0000 7f0b59a00640 1 --2- 172.21.15.19:0/2626964809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b540cc490 0x7f0b540d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:33.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.042+0000 7f0b53e00640 1 --2- 172.21.15.19:0/2626964809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b5415fc50 0x7f0b540cfdc0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f0b540cfd10 tx=0x7f0b4002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6405bb2fd8731912 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:33.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.042+0000 7f0b53400640 1 -- 172.21.15.19:0/2626964809 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40035020 con 0x7f0b5415fc50 2024-03-28T13:21:33.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.042+0000 7f0b53400640 1 -- 172.21.15.19:0/2626964809 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b40031870 con 0x7f0b5415fc50 2024-03-28T13:21:33.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.042+0000 7f0b53400640 1 -- 172.21.15.19:0/2626964809 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40031c40 con 0x7f0b5415fc50 2024-03-28T13:21:33.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.043+0000 7f0b5a400640 1 -- 172.21.15.19:0/2626964809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b5415fc50 msgr2=0x7f0b540cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:33.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.043+0000 7f0b5a400640 1 --2- 172.21.15.19:0/2626964809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b5415fc50 0x7f0b540cfdc0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f0b540cfd10 tx=0x7f0b4002b910 comp rx=0 tx=0).stop 2024-03-28T13:21:33.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.043+0000 7f0b5a400640 1 -- 172.21.15.19:0/2626964809 shutdown_connections 2024-03-28T13:21:33.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.043+0000 7f0b5a400640 1 --2- 172.21.15.19:0/2626964809 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b540cc490 0x7f0b540d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.043+0000 7f0b5a400640 1 --2- 172.21.15.19:0/2626964809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b5415fc50 0x7f0b540cfdc0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.043+0000 7f0b5a400640 1 --2- 172.21.15.19:0/2626964809 >> v2:172.21.15.188:3300/0 conn(0x7f0b540cc0b0 0x7f0b5406dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.043+0000 7f0b5a400640 1 -- 172.21.15.19:0/2626964809 >> 172.21.15.19:0/2626964809 conn(0x7f0b540ccad0 msgr2=0x7f0b540cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:33.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.043+0000 7f0b5a400640 1 -- 172.21.15.19:0/2626964809 shutdown_connections 2024-03-28T13:21:33.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.043+0000 7f0b5a400640 1 -- 172.21.15.19:0/2626964809 wait complete. 2024-03-28T13:21:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.045+0000 7f0b5a400640 1 Processor -- start 2024-03-28T13:21:33.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.045+0000 7f0b5a400640 1 -- start start 2024-03-28T13:21:33.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.046+0000 7f0b5a400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b540cc0b0 0x7f0b54155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.046+0000 7f0b5a400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b540cc490 0x7f0b54139bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.046+0000 7f0b5a400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b541389f0 0x7f0b5415b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.046+0000 7f0b5a400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b54160980 con 0x7f0b541389f0 2024-03-28T13:21:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.046+0000 7f0b5a400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0b541606a0 con 0x7f0b540cc490 2024-03-28T13:21:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.046+0000 7f0b5a400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0b5415ca70 con 0x7f0b540cc0b0 2024-03-28T13:21:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.046+0000 7f0b59a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b541389f0 0x7f0b5415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.046+0000 7f0b59a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b541389f0 0x7f0b5415b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42490/0 (socket says 172.21.15.19:42490) 2024-03-28T13:21:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.046+0000 7f0b59a00640 1 -- 172.21.15.19:0/737458135 learned_addr learned my addr 172.21.15.19:0/737458135 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.047+0000 7f0b59000640 1 --2- 172.21.15.19:0/737458135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b540cc0b0 0x7f0b54155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.047+0000 7f0b53e00640 1 --2- 172.21.15.19:0/737458135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b540cc490 0x7f0b54139bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.047+0000 7f0b59a00640 1 -- 172.21.15.19:0/737458135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b540cc0b0 msgr2=0x7f0b54155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.047+0000 7f0b59a00640 1 --2- 172.21.15.19:0/737458135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b540cc0b0 0x7f0b54155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.047+0000 7f0b59a00640 1 -- 172.21.15.19:0/737458135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b540cc490 msgr2=0x7f0b54139bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.047+0000 7f0b59a00640 1 --2- 172.21.15.19:0/737458135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b540cc490 0x7f0b54139bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.047+0000 7f0b59a00640 1 -- 172.21.15.19:0/737458135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b540cfa10 con 0x7f0b541389f0 2024-03-28T13:21:33.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.047+0000 7f0b59a00640 1 --2- 172.21.15.19:0/737458135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b541389f0 0x7f0b5415b130 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00ce80 tx=0x7f0b4c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:33.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.048+0000 7f0b50c00640 1 -- 172.21.15.19:0/737458135 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c01a070 con 0x7f0b541389f0 2024-03-28T13:21:33.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.048+0000 7f0b50c00640 1 -- 172.21.15.19:0/737458135 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b4c00d880 con 0x7f0b541389f0 2024-03-28T13:21:33.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.048+0000 7f0b50c00640 1 -- 172.21.15.19:0/737458135 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c00dc70 con 0x7f0b541389f0 2024-03-28T13:21:33.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.048+0000 7f0b5a400640 1 -- 172.21.15.19:0/737458135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5413a560 con 0x7f0b541389f0 2024-03-28T13:21:33.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.048+0000 7f0b5a400640 1 -- 172.21.15.19:0/737458135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b540b1fc0 con 0x7f0b541389f0 2024-03-28T13:21:33.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.049+0000 7f0b5a400640 1 -- 172.21.15.19:0/737458135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b54075940 con 0x7f0b541389f0 2024-03-28T13:21:33.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.050+0000 7f0b50c00640 1 -- 172.21.15.19:0/737458135 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0b4c01d070 con 0x7f0b541389f0 2024-03-28T13:21:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.051+0000 7f0b50c00640 1 --2- 172.21.15.19:0/737458135 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b3406c530 0x7f0b3406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.051+0000 7f0b50c00640 1 -- 172.21.15.19:0/737458135 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f0b4c08bf40 con 0x7f0b541389f0 2024-03-28T13:21:33.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.052+0000 7f0b59000640 1 --2- 172.21.15.19:0/737458135 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b3406c530 0x7f0b3406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.053+0000 7f0b59000640 1 --2- 172.21.15.19:0/737458135 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b3406c530 0x7f0b3406e9f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0b48006fd0 tx=0x7f0b48008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:33.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.056+0000 7f0b50c00640 1 -- 172.21.15.19:0/737458135 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0b4c059cf0 con 0x7f0b541389f0 2024-03-28T13:21:33.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.300+0000 7f0b5a400640 1 -- 172.21.15.19:0/737458135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b54073830 con 0x7f0b541389f0 2024-03-28T13:21:33.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.301+0000 7f0b50c00640 1 -- 172.21.15.19:0/737458135 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5186 (secure 0 0 0) 0x7f0b4c05dff0 con 0x7f0b541389f0 2024-03-28T13:21:33.301 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:33.301 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3823671990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3823671990},{"type":"v1","addr":"172.21.15.188:6837","nonce":3823671990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/22158043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":22158043},{"type":"v1","addr":"172.21.15.188:6836","nonce":22158043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":8688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/230519675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":230519675},{"type":"v1","addr":"172.21.15.19:6834","nonce":230519675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:20:52.901269+0000","modified":"2024-03-28T13:21:31.965724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/1527816903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1527816903},{"type":"v1","addr":"172.21.15.19:6835","nonce":1527816903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9123,"qdb_cluster":[9123]},"id":8}]} 2024-03-28T13:21:33.301 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 112 2024-03-28T13:21:33.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.303+0000 7f0b5a400640 1 -- 172.21.15.19:0/737458135 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b3406c530 msgr2=0x7f0b3406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:33.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.303+0000 7f0b5a400640 1 --2- 172.21.15.19:0/737458135 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b3406c530 0x7f0b3406e9f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0b48006fd0 tx=0x7f0b48008040 comp rx=0 tx=0).stop 2024-03-28T13:21:33.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.303+0000 7f0b5a400640 1 -- 172.21.15.19:0/737458135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b541389f0 msgr2=0x7f0b5415b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:33.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.303+0000 7f0b5a400640 1 --2- 172.21.15.19:0/737458135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b541389f0 0x7f0b5415b130 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00ce80 tx=0x7f0b4c00a710 comp rx=0 tx=0).stop 2024-03-28T13:21:33.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.304+0000 7f0b5a400640 1 -- 172.21.15.19:0/737458135 shutdown_connections 2024-03-28T13:21:33.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.304+0000 7f0b5a400640 1 --2- 172.21.15.19:0/737458135 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0b3406c530 0x7f0b3406e9f0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.304+0000 7f0b5a400640 1 --2- 172.21.15.19:0/737458135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b541389f0 0x7f0b5415b130 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.304+0000 7f0b5a400640 1 --2- 172.21.15.19:0/737458135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b540cc490 0x7f0b54139bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.304+0000 7f0b5a400640 1 --2- 172.21.15.19:0/737458135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b540cc0b0 0x7f0b54155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.304+0000 7f0b5a400640 1 -- 172.21.15.19:0/737458135 >> 172.21.15.19:0/737458135 conn(0x7f0b540ccad0 msgr2=0x7f0b5415df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:33.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.304+0000 7f0b5a400640 1 -- 172.21.15.19:0/737458135 shutdown_connections 2024-03-28T13:21:33.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.304+0000 7f0b5a400640 1 -- 172.21.15.19:0/737458135 wait complete. 2024-03-28T13:21:33.725 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-28T13:20:52.901269+0000', 'modified': '2024-03-28T13:21:31.965724+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9123}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9123': {'gid': 9123, 'name': 'a', 'rank': 0, 'incarnation': 109, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.19:6835/1527816903', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6833', 'nonce': 1527816903}, {'type': 'v1', 'addr': '172.21.15.19:6835', 'nonce': 1527816903}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9123, 'qdb_cluster': [9123]} 2024-03-28T13:21:33.726 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-28T13:21:33.726 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-28T13:21:33.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.948+0000 7f2acea00640 1 Processor -- start 2024-03-28T13:21:33.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.948+0000 7f2acea00640 1 -- start start 2024-03-28T13:21:33.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.950+0000 7f2acea00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ac80d2310 0x7f2ac80ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.950+0000 7f2acea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ac80cf530 0x7f2ac80b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.950+0000 7f2acea00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ac815e880 con 0x7f2ac815e4a0 2024-03-28T13:21:33.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.950+0000 7f2acea00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f2ac80d34e0 con 0x7f2ac80d2310 2024-03-28T13:21:33.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.950+0000 7f2acea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac80d28d0 con 0x7f2ac80cf530 2024-03-28T13:21:33.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.950+0000 7f2accc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ac80d2310 0x7f2ac80ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.951+0000 7f2accc00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f2ac80d2310 0x7f2ac80ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42498/0 (socket says 172.21.15.19:42498) 2024-03-28T13:21:33.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.951+0000 7f2accc00640 1 -- 172.21.15.19:0/474844148 learned_addr learned my addr 172.21.15.19:0/474844148 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:33.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.951+0000 7f2ace000640 1 --2- 172.21.15.19:0/474844148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ac80cf530 0x7f2ac80b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.951+0000 7f2accc00640 1 -- 172.21.15.19:0/474844148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ac80cf530 msgr2=0x7f2ac80b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:33.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.951+0000 7f2accc00640 1 --2- 172.21.15.19:0/474844148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ac80cf530 0x7f2ac80b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.951+0000 7f2accc00640 1 -- 172.21.15.19:0/474844148 >> v1:172.21.15.188:6789/0 conn(0x7f2ac815e4a0 legacy=0x7f2ac80b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:33.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.951+0000 7f2accc00640 1 -- 172.21.15.19:0/474844148 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac80d2bb0 con 0x7f2ac80d2310 2024-03-28T13:21:33.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.952+0000 7f2accc00640 1 --2- 172.21.15.19:0/474844148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ac80d2310 0x7f2ac80ceff0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009870 tx=0x7f2ab402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f1f949f68646df server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:33.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.953+0000 7f2ac7e00640 1 -- 172.21.15.19:0/474844148 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4035020 con 0x7f2ac80d2310 2024-03-28T13:21:33.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.953+0000 7f2ac7e00640 1 -- 172.21.15.19:0/474844148 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab4031640 con 0x7f2ac80d2310 2024-03-28T13:21:33.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.953+0000 7f2ac7e00640 1 -- 172.21.15.19:0/474844148 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab402d040 con 0x7f2ac80d2310 2024-03-28T13:21:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.953+0000 7f2acea00640 1 -- 172.21.15.19:0/474844148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ac80d2310 msgr2=0x7f2ac80ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.953+0000 7f2acea00640 1 --2- 172.21.15.19:0/474844148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ac80d2310 0x7f2ac80ceff0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009870 tx=0x7f2ab402b8f0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.954+0000 7f2acea00640 1 -- 172.21.15.19:0/474844148 shutdown_connections 2024-03-28T13:21:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.954+0000 7f2acea00640 1 --2- 172.21.15.19:0/474844148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ac80d2310 0x7f2ac80ceff0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.954+0000 7f2acea00640 1 --2- 172.21.15.19:0/474844148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ac80cf530 0x7f2ac80b73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.954+0000 7f2acea00640 1 -- 172.21.15.19:0/474844148 >> 172.21.15.19:0/474844148 conn(0x7f2ac80676d0 msgr2=0x7f2ac8067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.954+0000 7f2acea00640 1 -- 172.21.15.19:0/474844148 shutdown_connections 2024-03-28T13:21:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.954+0000 7f2acea00640 1 -- 172.21.15.19:0/474844148 wait complete. 2024-03-28T13:21:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.955+0000 7f2acea00640 1 Processor -- start 2024-03-28T13:21:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.956+0000 7f2acea00640 1 -- start start 2024-03-28T13:21:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.956+0000 7f2acea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ac80cf530 0x7f2ac814e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.957+0000 7f2acea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ac80d2310 0x7f2ac80cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.957+0000 7f2acea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ac815e4a0 0x7f2ac80cda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.957+0000 7f2acea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac80d3970 con 0x7f2ac815e4a0 2024-03-28T13:21:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.957+0000 7f2acea00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac80d3170 con 0x7f2ac80cf530 2024-03-28T13:21:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.957+0000 7f2acea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac80b7920 con 0x7f2ac80d2310 2024-03-28T13:21:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.957+0000 7f2ace000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ac815e4a0 0x7f2ac80cda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.957+0000 7f2ace000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ac815e4a0 0x7f2ac80cda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42514/0 (socket says 172.21.15.19:42514) 2024-03-28T13:21:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.957+0000 7f2ace000640 1 -- 172.21.15.19:0/2440379127 learned_addr learned my addr 172.21.15.19:0/2440379127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.957+0000 7f2accc00640 1 --2- 172.21.15.19:0/2440379127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ac80d2310 0x7f2ac80cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.957+0000 7f2acd600640 1 --2- 172.21.15.19:0/2440379127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ac80cf530 0x7f2ac814e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.957+0000 7f2ace000640 1 -- 172.21.15.19:0/2440379127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ac80d2310 msgr2=0x7f2ac80cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.957+0000 7f2ace000640 1 --2- 172.21.15.19:0/2440379127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ac80d2310 0x7f2ac80cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.957+0000 7f2ace000640 1 -- 172.21.15.19:0/2440379127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ac80cf530 msgr2=0x7f2ac814e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.957+0000 7f2ace000640 1 --2- 172.21.15.19:0/2440379127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ac80cf530 0x7f2ac814e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.957+0000 7f2ace000640 1 -- 172.21.15.19:0/2440379127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac80aac20 con 0x7f2ac815e4a0 2024-03-28T13:21:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.958+0000 7f2ace000640 1 --2- 172.21.15.19:0/2440379127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ac815e4a0 0x7f2ac80cda90 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000e930 tx=0x7f2ac000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.958+0000 7f2ac5600640 1 -- 172.21.15.19:0/2440379127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0017070 con 0x7f2ac815e4a0 2024-03-28T13:21:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.958+0000 7f2ac5600640 1 -- 172.21.15.19:0/2440379127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac000c980 con 0x7f2ac815e4a0 2024-03-28T13:21:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.958+0000 7f2acea00640 1 -- 172.21.15.19:0/2440379127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac80aaf00 con 0x7f2ac815e4a0 2024-03-28T13:21:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.958+0000 7f2ac5600640 1 -- 172.21.15.19:0/2440379127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac000cd70 con 0x7f2ac815e4a0 2024-03-28T13:21:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.958+0000 7f2acea00640 1 -- 172.21.15.19:0/2440379127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac80ab4c0 con 0x7f2ac815e4a0 2024-03-28T13:21:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.961+0000 7f2ac5600640 1 -- 172.21.15.19:0/2440379127 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f2ac001a070 con 0x7f2ac815e4a0 2024-03-28T13:21:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.962+0000 7f2ad2e04740 1 Processor -- start 2024-03-28T13:21:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.962+0000 7f2ac5600640 1 --2- 172.21.15.19:0/2440379127 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2aa806c530 0x7f2aa806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.962+0000 7f2ad2e04740 1 -- start start 2024-03-28T13:21:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.963+0000 7f2ac5600640 1 -- 172.21.15.19:0/2440379127 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f2ac008b7f0 con 0x7f2ac815e4a0 2024-03-28T13:21:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.963+0000 7f2accc00640 1 --2- 172.21.15.19:0/2440379127 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2aa806c530 0x7f2aa806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.963+0000 7f2ad2e04740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a11d0 0x55f191074170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.963+0000 7f2ad2e04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 0x55f1910a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.963+0000 7f2ad2e04740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910746b0 0x55f1910a44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.963+0000 7f2ad2e04740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f19103b920 con 0x55f1910746b0 2024-03-28T13:21:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.963+0000 7f2ad2e04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f190f8bfa0 con 0x55f1910a15b0 2024-03-28T13:21:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.963+0000 7f2ad2e04740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55f190fddf90 con 0x55f1910a11d0 2024-03-28T13:21:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.964+0000 7f2ace000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910746b0 0x55f1910a44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.964+0000 7f2ace000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910746b0 0x55f1910a44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42530/0 (socket says 172.21.15.19:42530) 2024-03-28T13:21:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.964+0000 7f2ace000640 1 -- 172.21.15.19:0/2992477816 learned_addr learned my addr 172.21.15.19:0/2992477816 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.964+0000 7f2acd600640 1 --2- 172.21.15.19:0/2992477816 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a11d0 0x55f191074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.964+0000 7f2accc00640 1 --2- 172.21.15.19:0/2992477816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 0x55f1910a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.964+0000 7f2ace000640 1 -- 172.21.15.19:0/2992477816 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a11d0 msgr2=0x55f191074170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.964+0000 7f2ace000640 1 --2- 172.21.15.19:0/2992477816 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a11d0 0x55f191074170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.964+0000 7f2ace000640 1 -- 172.21.15.19:0/2992477816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 msgr2=0x55f1910a3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.964+0000 7f2ace000640 1 --2- 172.21.15.19:0/2992477816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 0x55f1910a3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.964+0000 7f2ace000640 1 -- 172.21.15.19:0/2992477816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f190cf22d0 con 0x55f1910746b0 2024-03-28T13:21:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.964+0000 7f2accc00640 1 --2- 172.21.15.19:0/2992477816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 0x55f1910a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.964+0000 7f2accc00640 1 --2- 172.21.15.19:0/2440379127 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2aa806c530 0x7f2aa806e9f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4036000 tx=0x7f2ab4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:33.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.965+0000 7f2ace000640 1 --2- 172.21.15.19:0/2992477816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910746b0 0x55f1910a44a0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0061d80 tx=0x7f2ac0059e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c25446854ed610 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.965+0000 7f2abdc00640 1 -- 172.21.15.19:0/2992477816 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0021070 con 0x55f1910746b0 2024-03-28T13:21:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.965+0000 7f2abdc00640 1 -- 172.21.15.19:0/2992477816 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0088070 con 0x55f1910746b0 2024-03-28T13:21:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.965+0000 7f2abdc00640 1 -- 172.21.15.19:0/2992477816 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00622b0 con 0x55f1910746b0 2024-03-28T13:21:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.965+0000 7f2ad2e04740 1 -- 172.21.15.19:0/2992477816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910746b0 msgr2=0x55f1910a44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.966+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/2992477816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910746b0 0x55f1910a44a0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0061d80 tx=0x7f2ac0059e90 comp rx=0 tx=0).stop 2024-03-28T13:21:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.966+0000 7f2ad2e04740 1 -- 172.21.15.19:0/2992477816 shutdown_connections 2024-03-28T13:21:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.966+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/2992477816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910746b0 0x55f1910a44a0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.966+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/2992477816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 0x55f1910a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.966+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/2992477816 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a11d0 0x55f191074170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.966+0000 7f2ad2e04740 1 -- 172.21.15.19:0/2992477816 >> 172.21.15.19:0/2992477816 conn(0x55f1910763e0 msgr2=0x55f1910bef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.966+0000 7f2ad2e04740 1 -- 172.21.15.19:0/2992477816 shutdown_connections 2024-03-28T13:21:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.966+0000 7f2ad2e04740 1 -- 172.21.15.19:0/2992477816 wait complete. 2024-03-28T13:21:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.967+0000 7f2ad2e04740 1 -- start start 2024-03-28T13:21:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.967+0000 7f2ad2e04740 1 Processor -- start 2024-03-28T13:21:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.969+0000 7f2ad2e04740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a11d0 0x55f1910d3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.969+0000 7f2ad2e04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 0x55f1910cf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.969+0000 7f2ad2e04740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a6fa0 0x55f1910c01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.969+0000 7f2ad2e04740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f190fd9f40 con 0x55f1910a11d0 2024-03-28T13:21:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.969+0000 7f2ad2e04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f190f86650 con 0x55f1910a15b0 2024-03-28T13:21:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.969+0000 7f2ad2e04740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55f190f99a10 con 0x55f1910a6fa0 2024-03-28T13:21:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.969+0000 7f2ad2e04740 10 client.0 _ll_register_callbacks cb 0x55f190fd8ff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:21:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.969+0000 7f2acd600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a11d0 0x55f1910d3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.969+0000 7f2acd600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a11d0 0x55f1910d3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42532/0 (socket says 172.21.15.19:42532) 2024-03-28T13:21:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.969+0000 7f2acd600640 1 -- 172.21.15.19:0/4265094788 learned_addr learned my addr 172.21.15.19:0/4265094788 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.970+0000 7f2accc00640 1 --2- 172.21.15.19:0/4265094788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 0x55f1910cf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.970+0000 7f2ace000640 1 --2- 172.21.15.19:0/4265094788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a6fa0 0x55f1910c01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.970+0000 7f2acd600640 1 -- 172.21.15.19:0/4265094788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a6fa0 msgr2=0x55f1910c01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.970+0000 7f2acd600640 1 --2- 172.21.15.19:0/4265094788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a6fa0 0x55f1910c01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.970+0000 7f2acd600640 1 -- 172.21.15.19:0/4265094788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 msgr2=0x55f1910cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.970+0000 7f2acd600640 1 --2- 172.21.15.19:0/4265094788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 0x55f1910cf240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.970+0000 7f2acd600640 1 -- 172.21.15.19:0/4265094788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f190dbc6b0 con 0x55f1910a11d0 2024-03-28T13:21:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.970+0000 7f2ace000640 1 --2- 172.21.15.19:0/4265094788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a6fa0 0x55f1910c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.970+0000 7f2acd600640 1 --2- 172.21.15.19:0/4265094788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a11d0 0x55f1910d3290 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80058d0 tx=0x7f2ab8001890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b69b61464e2eab server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.970+0000 7f2ab2000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:21:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.971+0000 7f2ab2000640 1 -- 172.21.15.19:0/4265094788 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8002060 con 0x55f1910a11d0 2024-03-28T13:21:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.971+0000 7f2ab2000640 1 -- 172.21.15.19:0/4265094788 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f2ab8010050 con 0x55f1910a11d0 2024-03-28T13:21:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.971+0000 7f2ad2e04740 1 -- 172.21.15.19:0/4265094788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ac0088070 con 0x55f1910a11d0 2024-03-28T13:21:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.971+0000 7f2ab2000640 1 -- 172.21.15.19:0/4265094788 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f2ab8006b50 con 0x55f1910a11d0 2024-03-28T13:21:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.971+0000 7f2ad2e04740 10 client.9213 fetch_fsmap learned FSMap version 112 2024-03-28T13:21:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.971+0000 7f2ad2e04740 1 -- 172.21.15.19:0/4265094788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55f1910d2a90 con 0x55f1910a11d0 2024-03-28T13:21:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.971+0000 7f2ab2000640 1 -- 172.21.15.19:0/4265094788 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0088070 con 0x55f1910a11d0 2024-03-28T13:21:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.971+0000 7f2ab2000640 1 -- 172.21.15.19:0/4265094788 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 112) ==== 3494+0+0 (secure 0 0 0) 0x7f2ab8013570 con 0x55f1910a11d0 2024-03-28T13:21:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.971+0000 7f2ad2e04740 10 client.9213 fetch_fsmap finished waiting for FSMap version 112 2024-03-28T13:21:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.972+0000 7f2ad2e04740 10 client.9213 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:21:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.972+0000 7f2ad2e04740 20 client.9213 populate_metadata read hostname 'smithi019' 2024-03-28T13:21:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.972+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/4265094788 >> [v2:172.21.15.19:6833/1527816903,v1:172.21.15.19:6835/1527816903] conn(0x55f1910a39c0 0x55f1910ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.972+0000 7f2ad2e04740 4 client.9213 mds_command: new command op to 9123 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-28T13:21:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.972+0000 7f2ad2e04740 1 -- 172.21.15.19:0/4265094788 --> [v2:172.21.15.19:6833/1527816903,v1:172.21.15.19:6835/1527816903] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f190feb4d0 con 0x55f1910a39c0 2024-03-28T13:21:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.972+0000 7f2accc00640 1 --2- 172.21.15.19:0/4265094788 >> [v2:172.21.15.19:6833/1527816903,v1:172.21.15.19:6835/1527816903] conn(0x55f1910a39c0 0x55f1910ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.972+0000 7f2accc00640 1 --2- 172.21.15.19:0/4265094788 >> [v2:172.21.15.19:6833/1527816903,v1:172.21.15.19:6835/1527816903] conn(0x55f1910a39c0 0x55f1910ddcd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=169bbb1c39c40495 server_cookie=f2b01d3e45e547e in_seq=0 out_seq=0 2024-03-28T13:21:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.972+0000 7f2ab2000640 10 client.9213 ms_handle_connect on v2:172.21.15.19:6833/1527816903 2024-03-28T13:21:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.973+0000 7f2ab2000640 1 -- 172.21.15.19:0/4265094788 <== mds.0 v2:172.21.15.19:6833/1527816903 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f190feb4d0 con 0x55f1910a39c0 2024-03-28T13:21:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.973+0000 7f2ab2000640 10 client.9213 handle_command_reply: tid=0 2024-03-28T13:21:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.973+0000 7f2ad2e04740 1 client.9213 shutdown 2024-03-28T13:21:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.974+0000 7f2ad2e04740 1 -- 172.21.15.19:0/4265094788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a11d0 msgr2=0x55f1910d3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.974+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/4265094788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a11d0 0x55f1910d3290 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80058d0 tx=0x7f2ab8001890 comp rx=0 tx=0).stop 2024-03-28T13:21:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.974+0000 7f2ad2e04740 1 -- 172.21.15.19:0/4265094788 shutdown_connections 2024-03-28T13:21:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.974+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/4265094788 >> [v2:172.21.15.19:6833/1527816903,v1:172.21.15.19:6835/1527816903] conn(0x55f1910a39c0 0x55f1910ddcd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.974+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/4265094788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a6fa0 0x55f1910c01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.974+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/4265094788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 0x55f1910cf240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.974+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/4265094788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a11d0 0x55f1910d3290 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.974+0000 7f2ab2000640 0 client.9213 ms_handle_reset on v2:172.21.15.19:6833/1527816903 2024-03-28T13:21:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.974+0000 7f2ad2e04740 1 -- 172.21.15.19:0/4265094788 >> 172.21.15.19:0/4265094788 conn(0x55f1910746b0 msgr2=0x55f1910c15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.974+0000 7f2ad2e04740 1 -- 172.21.15.19:0/4265094788 shutdown_connections 2024-03-28T13:21:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.974+0000 7f2ad2e04740 1 -- 172.21.15.19:0/4265094788 wait complete. 2024-03-28T13:21:33.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.974+0000 7f2ad2e04740 20 client.9213 trim_cache size 0 max 16384 2024-03-28T13:21:33.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.995+0000 7f2ad2e04740 1 Processor -- start 2024-03-28T13:21:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.995+0000 7f2ad2e04740 1 -- start start 2024-03-28T13:21:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.995+0000 7f2ad2e04740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a11d0 0x55f1910bf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2ad2e04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 0x55f1910c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2ad2e04740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a6fa0 0x55f1910c1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2ad2e04740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac81ffdb0 con 0x55f1910a6fa0 2024-03-28T13:21:33.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2ad2e04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac8160a60 con 0x55f1910a15b0 2024-03-28T13:21:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2ad2e04740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac81610b0 con 0x55f1910a11d0 2024-03-28T13:21:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2ace000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a6fa0 0x55f1910c1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2ace000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a6fa0 0x55f1910c1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42538/0 (socket says 172.21.15.19:42538) 2024-03-28T13:21:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2ace000640 1 -- 172.21.15.19:0/3031888441 learned_addr learned my addr 172.21.15.19:0/3031888441 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2accc00640 1 --2- 172.21.15.19:0/3031888441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 0x55f1910c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2acd600640 1 --2- 172.21.15.19:0/3031888441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a11d0 0x55f1910bf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2ace000640 1 -- 172.21.15.19:0/3031888441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a11d0 msgr2=0x55f1910bf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2ace000640 1 --2- 172.21.15.19:0/3031888441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a11d0 0x55f1910bf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2ace000640 1 -- 172.21.15.19:0/3031888441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 msgr2=0x55f1910c15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2ace000640 1 --2- 172.21.15.19:0/3031888441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 0x55f1910c15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2ace000640 1 -- 172.21.15.19:0/3031888441 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f191076af0 con 0x55f1910a6fa0 2024-03-28T13:21:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.996+0000 7f2accc00640 1 --2- 172.21.15.19:0/3031888441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 0x55f1910c15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.997+0000 7f2ace000640 1 --2- 172.21.15.19:0/3031888441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a6fa0 0x55f1910c1b00 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000e900 tx=0x7f2ac001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765773077525d84d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.997+0000 7f2abd200640 1 -- 172.21.15.19:0/3031888441 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00579d0 con 0x55f1910a6fa0 2024-03-28T13:21:33.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.997+0000 7f2abd200640 1 -- 172.21.15.19:0/3031888441 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0088040 con 0x55f1910a6fa0 2024-03-28T13:21:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.997+0000 7f2abd200640 1 -- 172.21.15.19:0/3031888441 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0058b20 con 0x55f1910a6fa0 2024-03-28T13:21:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.998+0000 7f2ad2e04740 1 -- 172.21.15.19:0/3031888441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a6fa0 msgr2=0x55f1910c1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.998+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/3031888441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a6fa0 0x55f1910c1b00 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000e900 tx=0x7f2ac001d000 comp rx=0 tx=0).stop 2024-03-28T13:21:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.998+0000 7f2ad2e04740 1 -- 172.21.15.19:0/3031888441 shutdown_connections 2024-03-28T13:21:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.998+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/3031888441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a6fa0 0x55f1910c1b00 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.998+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/3031888441 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a15b0 0x55f1910c15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.998+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/3031888441 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a11d0 0x55f1910bf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.998+0000 7f2ad2e04740 1 -- 172.21.15.19:0/3031888441 >> 172.21.15.19:0/3031888441 conn(0x55f1910746b0 msgr2=0x55f19107b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.998+0000 7f2ad2e04740 1 -- 172.21.15.19:0/3031888441 shutdown_connections 2024-03-28T13:21:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.998+0000 7f2ad2e04740 1 -- 172.21.15.19:0/3031888441 wait complete. 2024-03-28T13:21:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.999+0000 7f2ad2e04740 1 -- start start 2024-03-28T13:21:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:33.999+0000 7f2ad2e04740 1 Processor -- start 2024-03-28T13:21:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2ad2e04740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a11d0 0x55f1910be660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2ad2e04740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a15b0 0x55f1910e1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2ad2e04740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a6fa0 0x55f191078960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2ad2e04740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f1910b61d0 con 0x55f1910a15b0 2024-03-28T13:21:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2ad2e04740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55f1910d2dd0 con 0x55f1910a6fa0 2024-03-28T13:21:34.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2ad2e04740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55f1910a6050 con 0x55f1910a11d0 2024-03-28T13:21:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2ad2e04740 10 client.0 _ll_register_callbacks cb 0x7f2ab8003bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-28T13:21:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2accc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a15b0 0x55f1910e1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2accc00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a15b0 0x55f1910e1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42542/0 (socket says 172.21.15.19:42542) 2024-03-28T13:21:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2accc00640 1 -- 172.21.15.19:0/1643144472 learned_addr learned my addr 172.21.15.19:0/1643144472 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2accc00640 1 -- 172.21.15.19:0/1643144472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a11d0 msgr2=0x55f1910be660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2acd600640 1 --2- 172.21.15.19:0/1643144472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a11d0 0x55f1910be660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2ace000640 1 --2- 172.21.15.19:0/1643144472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a6fa0 0x55f191078960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2accc00640 1 --2- 172.21.15.19:0/1643144472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a11d0 0x55f1910be660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2accc00640 1 -- 172.21.15.19:0/1643144472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a6fa0 msgr2=0x55f191078960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2accc00640 1 --2- 172.21.15.19:0/1643144472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a6fa0 0x55f191078960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.000+0000 7f2accc00640 1 -- 172.21.15.19:0/1643144472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f19109cc60 con 0x55f1910a15b0 2024-03-28T13:21:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.001+0000 7f2accc00640 1 --2- 172.21.15.19:0/1643144472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a15b0 0x55f1910e1df0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4000c00 tx=0x7f2ab4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa22782ad19df07 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.001+0000 7f2ab2000640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-03-28T13:21:34.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.001+0000 7f2ab2000640 1 -- 172.21.15.19:0/1643144472 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4035040 con 0x55f1910a15b0 2024-03-28T13:21:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.001+0000 7f2ab2000640 1 -- 172.21.15.19:0/1643144472 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f2ab404f470 con 0x55f1910a15b0 2024-03-28T13:21:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.001+0000 7f2ad2e04740 1 -- 172.21.15.19:0/1643144472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ac0088040 con 0x55f1910a15b0 2024-03-28T13:21:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.001+0000 7f2ab2000640 1 -- 172.21.15.19:0/1643144472 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f2ab405e050 con 0x55f1910a15b0 2024-03-28T13:21:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.002+0000 7f2ad2e04740 10 client.9219 fetch_fsmap learned FSMap version 112 2024-03-28T13:21:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.002+0000 7f2ad2e04740 1 -- 172.21.15.19:0/1643144472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55f1910a43d0 con 0x55f1910a15b0 2024-03-28T13:21:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.002+0000 7f2ab2000640 1 -- 172.21.15.19:0/1643144472 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ac0088040 con 0x55f1910a15b0 2024-03-28T13:21:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.002+0000 7f2ab2000640 1 -- 172.21.15.19:0/1643144472 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 112) ==== 3494+0+0 (secure 0 0 0) 0x7f2ab4061020 con 0x55f1910a15b0 2024-03-28T13:21:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.002+0000 7f2ad2e04740 10 client.9219 fetch_fsmap finished waiting for FSMap version 112 2024-03-28T13:21:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.002+0000 7f2ad2e04740 10 client.9219 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-28T13:21:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.002+0000 7f2ad2e04740 20 client.9219 populate_metadata read hostname 'smithi019' 2024-03-28T13:21:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.002+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/1643144472 >> [v2:172.21.15.19:6833/1527816903,v1:172.21.15.19:6835/1527816903] conn(0x55f1910790b0 0x55f19109f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.002+0000 7f2ad2e04740 4 client.9219 mds_command: new command op to 9123 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-28T13:21:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.002+0000 7f2ad2e04740 1 -- 172.21.15.19:0/1643144472 --> [v2:172.21.15.19:6833/1527816903,v1:172.21.15.19:6835/1527816903] -- command(tid 0: {"prefix": "status"}) -- 0x55f190fe5820 con 0x55f1910790b0 2024-03-28T13:21:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.002+0000 7f2acd600640 1 --2- 172.21.15.19:0/1643144472 >> [v2:172.21.15.19:6833/1527816903,v1:172.21.15.19:6835/1527816903] conn(0x55f1910790b0 0x55f19109f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:34.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.003+0000 7f2acd600640 1 --2- 172.21.15.19:0/1643144472 >> [v2:172.21.15.19:6833/1527816903,v1:172.21.15.19:6835/1527816903] conn(0x55f1910790b0 0x55f19109f3f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdbd31a0a9188b5b server_cookie=4b0cd875c47c060b in_seq=0 out_seq=0 2024-03-28T13:21:34.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.003+0000 7f2ab2000640 10 client.9219 ms_handle_connect on v2:172.21.15.19:6833/1527816903 2024-03-28T13:21:34.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.003+0000 7f2ab2000640 1 -- 172.21.15.19:0/1643144472 <== mds.0 v2:172.21.15.19:6833/1527816903 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f190fe5820 con 0x55f1910790b0 2024-03-28T13:21:34.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.003+0000 7f2ab2000640 10 client.9219 handle_command_reply: tid=0 2024-03-28T13:21:34.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.003+0000 7f2ad2e04740 1 client.9219 shutdown 2024-03-28T13:21:34.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.004+0000 7f2ad2e04740 1 -- 172.21.15.19:0/1643144472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a15b0 msgr2=0x55f1910e1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:34.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.004+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/1643144472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a15b0 0x55f1910e1df0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4000c00 tx=0x7f2ab4004730 comp rx=0 tx=0).stop 2024-03-28T13:21:34.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.004+0000 7f2ad2e04740 1 -- 172.21.15.19:0/1643144472 shutdown_connections 2024-03-28T13:21:34.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.004+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/1643144472 >> [v2:172.21.15.19:6833/1527816903,v1:172.21.15.19:6835/1527816903] conn(0x55f1910790b0 0x55f19109f3f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.004+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/1643144472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55f1910a6fa0 0x55f191078960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.004+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/1643144472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f1910a15b0 0x55f1910e1df0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.004+0000 7f2ad2e04740 1 --2- 172.21.15.19:0/1643144472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55f1910a11d0 0x55f1910be660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.004+0000 7f2ab2000640 0 client.9219 ms_handle_reset on v2:172.21.15.19:6833/1527816903 2024-03-28T13:21:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.004+0000 7f2ad2e04740 1 -- 172.21.15.19:0/1643144472 >> 172.21.15.19:0/1643144472 conn(0x55f1910746b0 msgr2=0x55f1910c15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.004+0000 7f2ad2e04740 1 -- 172.21.15.19:0/1643144472 shutdown_connections 2024-03-28T13:21:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.004+0000 7f2ad2e04740 1 -- 172.21.15.19:0/1643144472 wait complete. 2024-03-28T13:21:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.004+0000 7f2ad2e04740 20 client.9219 trim_cache size 0 max 16384 2024-03-28T13:21:34.008 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:21:34.008 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "e6b569d9-f2ce-49dc-bbe7-c0ab33080203", 2024-03-28T13:21:34.008 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-03-28T13:21:34.008 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 9123, 2024-03-28T13:21:34.009 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-03-28T13:21:34.009 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-03-28T13:21:34.009 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-03-28T13:21:34.009 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 5.0554591379999998, 2024-03-28T13:21:34.009 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 112, 2024-03-28T13:21:34.009 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 123, 2024-03-28T13:21:34.009 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 123, 2024-03-28T13:21:34.009 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 6.994235615 2024-03-28T13:21:34.009 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:21:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.005+0000 7f2acea00640 1 -- 172.21.15.19:0/2440379127 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2aa806c530 msgr2=0x7f2aa806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.005+0000 7f2acea00640 1 --2- 172.21.15.19:0/2440379127 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2aa806c530 0x7f2aa806e9f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4036000 tx=0x7f2ab4008030 comp rx=0 tx=0).stop 2024-03-28T13:21:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.005+0000 7f2acea00640 1 -- 172.21.15.19:0/2440379127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ac815e4a0 msgr2=0x7f2ac80cda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.005+0000 7f2acea00640 1 --2- 172.21.15.19:0/2440379127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ac815e4a0 0x7f2ac80cda90 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000e930 tx=0x7f2ac000edf0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.005+0000 7f2acea00640 1 -- 172.21.15.19:0/2440379127 shutdown_connections 2024-03-28T13:21:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.005+0000 7f2acea00640 1 --2- 172.21.15.19:0/2440379127 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f2aa806c530 0x7f2aa806e9f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.005+0000 7f2acea00640 1 --2- 172.21.15.19:0/2440379127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2ac815e4a0 0x7f2ac80cda90 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.005+0000 7f2acea00640 1 --2- 172.21.15.19:0/2440379127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ac80d2310 0x7f2ac80cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.005+0000 7f2acea00640 1 --2- 172.21.15.19:0/2440379127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ac80cf530 0x7f2ac814e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.005+0000 7f2acea00640 1 -- 172.21.15.19:0/2440379127 >> 172.21.15.19:0/2440379127 conn(0x7f2ac80676d0 msgr2=0x7f2ac8074580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.005+0000 7f2acea00640 1 -- 172.21.15.19:0/2440379127 shutdown_connections 2024-03-28T13:21:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.005+0000 7f2acea00640 1 -- 172.21.15.19:0/2440379127 wait complete. 2024-03-28T13:21:34.026 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-28T13:21:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.259+0000 7fa9aa800640 1 Processor -- start 2024-03-28T13:21:34.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.259+0000 7fa9aa800640 1 -- start start 2024-03-28T13:21:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.260+0000 7fa9aa800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa9a40d0990 0x7fa9a40aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.260+0000 7fa9aa800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa9a40af2e0 0x7fa9a41600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.260+0000 7fa9aa800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a40afab0 con 0x7fa9a4153580 2024-03-28T13:21:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.260+0000 7fa9aa800640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa9a415cc80 con 0x7fa9a40d0990 2024-03-28T13:21:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.260+0000 7fa9aa800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa9a415da00 con 0x7fa9a40af2e0 2024-03-28T13:21:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.261+0000 7fa99aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa9a40d0990 0x7fa9a40aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.261+0000 7fa99aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa9a40d0990 0x7fa9a40aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42548/0 (socket says 172.21.15.19:42548) 2024-03-28T13:21:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.261+0000 7fa99aa00640 1 -- 172.21.15.19:0/4108256624 learned_addr learned my addr 172.21.15.19:0/4108256624 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:34.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.261+0000 7fa9a3e00640 1 --2- 172.21.15.19:0/4108256624 >> v2:172.21.15.188:3300/0 conn(0x7fa9a40af2e0 0x7fa9a41600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.261+0000 7fa99aa00640 1 -- 172.21.15.19:0/4108256624 >> v2:172.21.15.188:3300/0 conn(0x7fa9a40af2e0 msgr2=0x7fa9a41600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.261+0000 7fa99aa00640 1 --2- 172.21.15.19:0/4108256624 >> v2:172.21.15.188:3300/0 conn(0x7fa9a40af2e0 0x7fa9a41600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.261+0000 7fa99aa00640 1 -- 172.21.15.19:0/4108256624 >> v1:172.21.15.188:6789/0 conn(0x7fa9a4153580 legacy=0x7fa9a40d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.261+0000 7fa99aa00640 1 -- 172.21.15.19:0/4108256624 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a415cf20 con 0x7fa9a40d0990 2024-03-28T13:21:34.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.262+0000 7fa99aa00640 1 --2- 172.21.15.19:0/4108256624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a40d0990 0x7fa9a40aeda0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fa9a406efa0 tx=0x7fa98c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eaf41ca0d550bc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:34.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.263+0000 7fa9a2a00640 1 -- 172.21.15.19:0/4108256624 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c02e070 con 0x7fa9a40d0990 2024-03-28T13:21:34.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.263+0000 7fa9a2a00640 1 -- 172.21.15.19:0/4108256624 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa98c037070 con 0x7fa9a40d0990 2024-03-28T13:21:34.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.263+0000 7fa9a2a00640 1 -- 172.21.15.19:0/4108256624 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c031a00 con 0x7fa9a40d0990 2024-03-28T13:21:34.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.264+0000 7fa9aa800640 1 -- 172.21.15.19:0/4108256624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a40d0990 msgr2=0x7fa9a40aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:34.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.264+0000 7fa9aa800640 1 --2- 172.21.15.19:0/4108256624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a40d0990 0x7fa9a40aeda0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fa9a406efa0 tx=0x7fa98c02b880 comp rx=0 tx=0).stop 2024-03-28T13:21:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.264+0000 7fa9aa800640 1 -- 172.21.15.19:0/4108256624 shutdown_connections 2024-03-28T13:21:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.264+0000 7fa9aa800640 1 --2- 172.21.15.19:0/4108256624 >> v2:172.21.15.188:3300/0 conn(0x7fa9a40af2e0 0x7fa9a41600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.264+0000 7fa9aa800640 1 --2- 172.21.15.19:0/4108256624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a40d0990 0x7fa9a40aeda0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.264+0000 7fa9aa800640 1 -- 172.21.15.19:0/4108256624 >> 172.21.15.19:0/4108256624 conn(0x7fa9a406b750 msgr2=0x7fa9a406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.264+0000 7fa9aa800640 1 -- 172.21.15.19:0/4108256624 shutdown_connections 2024-03-28T13:21:34.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.264+0000 7fa9aa800640 1 -- 172.21.15.19:0/4108256624 wait complete. 2024-03-28T13:21:34.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.266+0000 7fa9aa800640 1 Processor -- start 2024-03-28T13:21:34.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.266+0000 7fa9aa800640 1 -- start start 2024-03-28T13:21:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.267+0000 7fa9aa800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a40af2e0 0x7fa9a4152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.267+0000 7fa9aa800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a40d0990 0x7fa9a40cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.267+0000 7fa9aa800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a4153580 0x7fa9a40ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.267+0000 7fa9aa800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a4153960 con 0x7fa9a40d0990 2024-03-28T13:21:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.267+0000 7fa9aa800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a415d4e0 con 0x7fa9a4153580 2024-03-28T13:21:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.267+0000 7fa9aa800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a40af6c0 con 0x7fa9a40af2e0 2024-03-28T13:21:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.267+0000 7fa99aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a40d0990 0x7fa9a40cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.267+0000 7fa99aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a40d0990 0x7fa9a40cd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42554/0 (socket says 172.21.15.19:42554) 2024-03-28T13:21:34.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.267+0000 7fa99aa00640 1 -- 172.21.15.19:0/2035201015 learned_addr learned my addr 172.21.15.19:0/2035201015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:34.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.267+0000 7fa9a3e00640 1 --2- 172.21.15.19:0/2035201015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a4153580 0x7fa9a40ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.267+0000 7fa9a3400640 1 --2- 172.21.15.19:0/2035201015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a40af2e0 0x7fa9a4152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.268+0000 7fa99aa00640 1 -- 172.21.15.19:0/2035201015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a40af2e0 msgr2=0x7fa9a4152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.268+0000 7fa99aa00640 1 --2- 172.21.15.19:0/2035201015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a40af2e0 0x7fa9a4152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.268+0000 7fa99aa00640 1 -- 172.21.15.19:0/2035201015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a4153580 msgr2=0x7fa9a40ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.268+0000 7fa99aa00640 1 --2- 172.21.15.19:0/2035201015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a4153580 0x7fa9a40ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.268+0000 7fa99aa00640 1 -- 172.21.15.19:0/2035201015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a40cb440 con 0x7fa9a40d0990 2024-03-28T13:21:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.268+0000 7fa9a3e00640 1 --2- 172.21.15.19:0/2035201015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a4153580 0x7fa9a40ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.268+0000 7fa99aa00640 1 --2- 172.21.15.19:0/2035201015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a40d0990 0x7fa9a40cd8b0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fa98c000c00 tx=0x7fa98c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.268+0000 7fa99be00640 1 -- 172.21.15.19:0/2035201015 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c032880 con 0x7fa9a40d0990 2024-03-28T13:21:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.268+0000 7fa99be00640 1 -- 172.21.15.19:0/2035201015 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa98c032a20 con 0x7fa9a40d0990 2024-03-28T13:21:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.268+0000 7fa99be00640 1 -- 172.21.15.19:0/2035201015 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c032e10 con 0x7fa9a40d0990 2024-03-28T13:21:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.268+0000 7fa9aa800640 1 -- 172.21.15.19:0/2035201015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a40ccaf0 con 0x7fa9a40d0990 2024-03-28T13:21:34.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.268+0000 7fa9aa800640 1 -- 172.21.15.19:0/2035201015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a40cd0b0 con 0x7fa9a40d0990 2024-03-28T13:21:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.270+0000 7fa9aa800640 1 -- 172.21.15.19:0/2035201015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a40685c0 con 0x7fa9a40d0990 2024-03-28T13:21:34.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.270+0000 7fa99be00640 1 -- 172.21.15.19:0/2035201015 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa98c04c060 con 0x7fa9a40d0990 2024-03-28T13:21:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.272+0000 7fa99be00640 1 --2- 172.21.15.19:0/2035201015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa98006c530 0x7fa98006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:34.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.272+0000 7fa99be00640 1 -- 172.21.15.19:0/2035201015 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7fa98c0ad140 con 0x7fa9a40d0990 2024-03-28T13:21:34.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.272+0000 7fa9a3400640 1 --2- 172.21.15.19:0/2035201015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa98006c530 0x7fa98006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:34.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.274+0000 7fa9a3400640 1 --2- 172.21.15.19:0/2035201015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa98006c530 0x7fa98006e9f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa994001db0 tx=0x7fa994009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:34.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.277+0000 7fa99be00640 1 -- 172.21.15.19:0/2035201015 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa98c040070 con 0x7fa9a40d0990 2024-03-28T13:21:34.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.530+0000 7fa9aa800640 1 -- 172.21.15.19:0/2035201015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa9a406eca0 con 0x7fa9a40d0990 2024-03-28T13:21:34.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.531+0000 7fa99be00640 1 -- 172.21.15.19:0/2035201015 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) ==== 53+0+73 (secure 0 0 0) 0x7fa98c07b5d0 con 0x7fa9a40d0990 2024-03-28T13:21:34.530 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-28T13:21:34.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.534+0000 7fa9aa800640 1 -- 172.21.15.19:0/2035201015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa98006c530 msgr2=0x7fa98006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.534+0000 7fa9aa800640 1 --2- 172.21.15.19:0/2035201015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa98006c530 0x7fa98006e9f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa994001db0 tx=0x7fa994009040 comp rx=0 tx=0).stop 2024-03-28T13:21:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.534+0000 7fa9aa800640 1 -- 172.21.15.19:0/2035201015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a40d0990 msgr2=0x7fa9a40cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.534+0000 7fa9aa800640 1 --2- 172.21.15.19:0/2035201015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a40d0990 0x7fa9a40cd8b0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fa98c000c00 tx=0x7fa98c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.534+0000 7fa9aa800640 1 -- 172.21.15.19:0/2035201015 shutdown_connections 2024-03-28T13:21:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.534+0000 7fa9aa800640 1 --2- 172.21.15.19:0/2035201015 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa98006c530 0x7fa98006e9f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.534+0000 7fa9aa800640 1 --2- 172.21.15.19:0/2035201015 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a4153580 0x7fa9a40ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.534+0000 7fa9aa800640 1 --2- 172.21.15.19:0/2035201015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a40d0990 0x7fa9a40cd8b0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.534+0000 7fa9aa800640 1 --2- 172.21.15.19:0/2035201015 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a40af2e0 0x7fa9a4152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.534+0000 7fa9aa800640 1 -- 172.21.15.19:0/2035201015 >> 172.21.15.19:0/2035201015 conn(0x7fa9a406b750 msgr2=0x7fa9a415ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.534+0000 7fa9aa800640 1 -- 172.21.15.19:0/2035201015 shutdown_connections 2024-03-28T13:21:34.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34.534+0000 7fa9aa800640 1 -- 172.21.15.19:0/2035201015 wait complete. 2024-03-28T13:21:34.554 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-03-28T13:21:34.554 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-28T13:21:34.554 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-03-28T13:21:34.554 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-28T13:21:34.555 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:34.555 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-03-28T13:21:34.555 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-03-28T13:21:34.555 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-28T13:21:34.555 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:21:34.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:34.555 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-03-28T13:21:34.609 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-28T13:21:34.609 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-28T13:21:34.609 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-03-28T13:21:34.609 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:34.609 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-03-28T13:21:34.609 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:34.609 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:21:34.609 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:34.609 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-03-28T13:21:34.609 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:21:34.609 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:34.609 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-03-28T13:21:34.609 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-28T13:21:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-03-28T13:21:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-03-28T13:21:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 41860sec preferred_lft 41860sec 2024-03-28T13:21:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link noprefixroute 2024-03-28T13:21:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:34.610 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-28T13:21:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-03-28T13:21:34.610 INFO:teuthology.orchestra.run.smithi019.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-28T13:21:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff 2024-03-28T13:21:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-28T13:21:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::2c41:d9ff:fe2f:ba5e/64 scope link 2024-03-28T13:21:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:34.611 INFO:teuthology.orchestra.run.smithi019.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:21:34.611 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 7a:d0:fb:bd:28:e8 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:21:34.611 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::78d0:fbff:febd:28e8/64 scope link 2024-03-28T13:21:34.611 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:34.611 INFO:teuthology.orchestra.run.smithi019.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-28T13:21:34.611 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether e6:17:c5:8e:2b:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:21:34.611 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::e417:c5ff:fe8e:2bcf/64 scope link 2024-03-28T13:21:34.611 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-03-28T13:21:34.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:34.612 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-28T13:21:34.612 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:21:34.624 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:21:34.695 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:21:34.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:34.699 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-03-28T13:21:34.832 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-03-28T13:21:34.832 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-28T13:21:34.832 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-03-28T13:21:34.832 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-03-28T13:21:34.832 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-28T13:21:34.833 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-28T13:21:34.833 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-03-28T13:21:34.833 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-28T13:21:34.833 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-28T13:21:34.833 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-28T13:21:34.833 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-03-28T13:21:34.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-28T13:21:35.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:35.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:35.095 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:21:35.149 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:21:35.149 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:21:35.150 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:21:35.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:35.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-03-28T13:21:35.227 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:21:35.228 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:35.277 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-28T13:21:35.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-28T13:21:35.278 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:35.327 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-03-28T13:21:35.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:35.328 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:35.478 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:35.478 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:21:35.478 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:21:35.502 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:35.502 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:21:35.502 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:21:35.502 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:21:35.502 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:21:35.531 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-03-28T13:21:35.532 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-03-28T13:21:35.532 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-03-28T13:21:35.532 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-03-28T13:21:35.532 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: switching to using v1 address with old syntax 2024-03-28T13:21:35.532 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790" 2024-03-28T13:21:35.532 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:6789,172.21.15.188:6789,172.21.15.188:6790:/ 2024-03-28T13:21:35.532 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=e6b569d9-f2ce-49dc-bbe7-c0ab33080203" will pass to kernel 2024-03-28T13:21:35.536 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-28T13:21:35.536 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-03-28T13:21:35.536 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-28T13:21:35.536 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-03-28T13:21:35.536 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-28T13:21:35.539 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-28T13:21:35.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:35.539 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:35.617 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-28T13:21:35.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-28T13:21:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:35.727 INFO:teuthology.orchestra.run.smithi019.stdout:client6563 2024-03-28T13:21:35.727 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:35.727 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:35.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:35.774 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-28T13:21:35.877 INFO:teuthology.orchestra.run.smithi019.stdout:./ 2024-03-28T13:21:35.877 INFO:teuthology.orchestra.run.smithi019.stdout:./parent 2024-03-28T13:21:35.877 INFO:teuthology.orchestra.run.smithi019.stdout:./parent/flushed 2024-03-28T13:21:35.877 INFO:teuthology.orchestra.run.smithi019.stdout:./parent/flushed/charlie 2024-03-28T13:21:35.877 INFO:teuthology.orchestra.run.smithi019.stdout:./parent/flushed/alpha 2024-03-28T13:21:35.877 INFO:teuthology.orchestra.run.smithi019.stdout:./parent/flushed/bravo 2024-03-28T13:21:35.877 INFO:teuthology.orchestra.run.smithi019.stdout:./parent/unflushed 2024-03-28T13:21:35.877 INFO:teuthology.orchestra.run.smithi019.stdout:./parent/unflushed/jfile 2024-03-28T13:21:35.879 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:35.879 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:35.879 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:35.879 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:35.879 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-28T13:21:35.879 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:36.022 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-03-28T13:21:36.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:36.037 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:36.037 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:36.037 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:36.037 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:36.037 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-03-28T13:21:36.037 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:36.218 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627776 2024-03-28T13:21:36.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:36.232 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:36.232 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:36.232 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:36.233 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:36.233 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-03-28T13:21:36.233 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:36.420 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627777 2024-03-28T13:21:36.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:36.434 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:36.434 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:36.434 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:36.434 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:36.434 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-03-28T13:21:36.434 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:36.618 INFO:teuthology.orchestra.run.smithi019.stdout:1099511628280 2024-03-28T13:21:36.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:36.633 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:36.633 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:36.633 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:36.633 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:36.633 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-03-28T13:21:36.633 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:36.812 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627778 2024-03-28T13:21:36.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:36.825 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:36.825 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:36.825 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:36.825 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:36.825 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-03-28T13:21:36.825 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:37.031 INFO:teuthology.orchestra.run.smithi019.stdout:1099511628279 2024-03-28T13:21:37.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:37.047 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:37.047 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:37.047 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:37.047 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:37.047 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-03-28T13:21:37.047 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:37.228 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627779 2024-03-28T13:21:37.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:37.242 DEBUG:teuthology.orchestra.run.smithi019:> stdin-killer -- python3 -c ' 2024-03-28T13:21:37.242 DEBUG:teuthology.orchestra.run.smithi019:> import os 2024-03-28T13:21:37.242 DEBUG:teuthology.orchestra.run.smithi019:> import stat 2024-03-28T13:21:37.242 DEBUG:teuthology.orchestra.run.smithi019:> 2024-03-28T13:21:37.242 DEBUG:teuthology.orchestra.run.smithi019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-03-28T13:21:37.242 DEBUG:teuthology.orchestra.run.smithi019:> ' 2024-03-28T13:21:37.436 INFO:teuthology.orchestra.run.smithi019.stdout:1099511627780 2024-03-28T13:21:37.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-28T13:21:37.450 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-28T13:21:37.451 DEBUG:teuthology.orchestra.run.smithi019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:21:37.641 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-28T13:21:37.756 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-28T13:21:37.756 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:21:37.756 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:37.756 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:37.814 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:37.814 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:37.866 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:21:38.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.076+0000 7f981d400640 1 Processor -- start 2024-03-28T13:21:38.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.076+0000 7f981d400640 1 -- start start 2024-03-28T13:21:38.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.077+0000 7f981d400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f981815c390 0x7f9818073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:38.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.077+0000 7f981d400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98180d0650 0x7f98180d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:38.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.077+0000 7f981d400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f981815fe80 con 0x7f98180af830 2024-03-28T13:21:38.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.078+0000 7f981d400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f981815c770 con 0x7f98180d0650 2024-03-28T13:21:38.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.078+0000 7f981d400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f98180d0fe0 con 0x7f981815c390 2024-03-28T13:21:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.078+0000 7f9815e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f98180af830 0x7f9818160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34106/0 (socket says 172.21.15.19:34106) 2024-03-28T13:21:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.078+0000 7f9815e00640 1 -- 172.21.15.19:0/1245147695 learned_addr learned my addr 172.21.15.19:0/1245147695 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.078+0000 7f9817200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f98180d0650 0x7f98180d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.078+0000 7f9816800640 1 --2- 172.21.15.19:0/1245147695 >> v2:172.21.15.188:3300/0 conn(0x7f981815c390 0x7f9818073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.079+0000 7f9815400640 1 -- 172.21.15.19:0/1245147695 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1102892774 0 0) 0x7f981815fe80 con 0x7f98180af830 2024-03-28T13:21:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.079+0000 7f9815400640 1 -- 172.21.15.19:0/1245147695 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97fc003520 con 0x7f98180af830 2024-03-28T13:21:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.079+0000 7f9817200640 1 -- 172.21.15.19:0/1245147695 >> v2:172.21.15.188:3300/0 conn(0x7f981815c390 msgr2=0x7f9818073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.079+0000 7f9817200640 1 --2- 172.21.15.19:0/1245147695 >> v2:172.21.15.188:3300/0 conn(0x7f981815c390 0x7f9818073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.079+0000 7f9817200640 1 -- 172.21.15.19:0/1245147695 >> v1:172.21.15.19:6789/0 conn(0x7f98180af830 legacy=0x7f9818160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:38.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.079+0000 7f9817200640 1 -- 172.21.15.19:0/1245147695 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98180ad120 con 0x7f98180d0650 2024-03-28T13:21:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.080+0000 7f9817200640 1 --2- 172.21.15.19:0/1245147695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98180d0650 0x7f98180d0aa0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f98180b63e0 tx=0x7f981002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b548fe98c6c8e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.080+0000 7f9815400640 1 -- 172.21.15.19:0/1245147695 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9810039070 con 0x7f98180d0650 2024-03-28T13:21:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.080+0000 7f9815400640 1 -- 172.21.15.19:0/1245147695 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f981002cc80 con 0x7f98180d0650 2024-03-28T13:21:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.080+0000 7f9815400640 1 -- 172.21.15.19:0/1245147695 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9810034660 con 0x7f98180d0650 2024-03-28T13:21:38.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.081+0000 7f981d400640 1 -- 172.21.15.19:0/1245147695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98180d0650 msgr2=0x7f98180d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:38.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.081+0000 7f981d400640 1 --2- 172.21.15.19:0/1245147695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98180d0650 0x7f98180d0aa0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f98180b63e0 tx=0x7f981002bed0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.081+0000 7f981d400640 1 -- 172.21.15.19:0/1245147695 shutdown_connections 2024-03-28T13:21:38.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.081+0000 7f981d400640 1 --2- 172.21.15.19:0/1245147695 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98180d0650 0x7f98180d0aa0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.081+0000 7f981d400640 1 --2- 172.21.15.19:0/1245147695 >> v2:172.21.15.188:3300/0 conn(0x7f981815c390 0x7f9818073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.081+0000 7f981d400640 1 -- 172.21.15.19:0/1245147695 >> 172.21.15.19:0/1245147695 conn(0x7f981806b990 msgr2=0x7f981806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:38.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.081+0000 7f981d400640 1 -- 172.21.15.19:0/1245147695 shutdown_connections 2024-03-28T13:21:38.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.081+0000 7f981d400640 1 -- 172.21.15.19:0/1245147695 wait complete. 2024-03-28T13:21:38.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.083+0000 7f981d400640 1 Processor -- start 2024-03-28T13:21:38.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.083+0000 7f981d400640 1 -- start start 2024-03-28T13:21:38.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.084+0000 7f981d400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98180af830 0x7f98180b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:38.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.084+0000 7f981d400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98180d0650 0x7f98181f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:38.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.084+0000 7f981d400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981815c390 0x7f98181f7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:38.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.084+0000 7f981d400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9818160120 con 0x7f98180d0650 2024-03-28T13:21:38.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.084+0000 7f981d400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f98180ad6e0 con 0x7f98180af830 2024-03-28T13:21:38.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.084+0000 7f981d400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9818160d50 con 0x7f981815c390 2024-03-28T13:21:38.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.084+0000 7f9815e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98180d0650 0x7f98181f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:38.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.084+0000 7f9815e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98180d0650 0x7f98181f6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54168/0 (socket says 172.21.15.19:54168) 2024-03-28T13:21:38.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.084+0000 7f9815e00640 1 -- 172.21.15.19:0/988766037 learned_addr learned my addr 172.21.15.19:0/988766037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:38.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.084+0000 7f9817200640 1 --2- 172.21.15.19:0/988766037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981815c390 0x7f98181f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:38.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.085+0000 7f9816800640 1 --2- 172.21.15.19:0/988766037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98180af830 0x7f98180b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:38.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.085+0000 7f9815e00640 1 -- 172.21.15.19:0/988766037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981815c390 msgr2=0x7f98181f7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:38.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.085+0000 7f9815e00640 1 --2- 172.21.15.19:0/988766037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981815c390 0x7f98181f7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.085+0000 7f9815e00640 1 -- 172.21.15.19:0/988766037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98180af830 msgr2=0x7f98180b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:38.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.085+0000 7f9815e00640 1 --2- 172.21.15.19:0/988766037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98180af830 0x7f98180b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.085+0000 7f9815e00640 1 -- 172.21.15.19:0/988766037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98180b0030 con 0x7f98180d0650 2024-03-28T13:21:38.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.085+0000 7f9815e00640 1 --2- 172.21.15.19:0/988766037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98180d0650 0x7f98181f6df0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f980000c390 tx=0x7f980000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:38.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.085+0000 7f980ea00640 1 -- 172.21.15.19:0/988766037 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800017070 con 0x7f98180d0650 2024-03-28T13:21:38.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.085+0000 7f980ea00640 1 -- 172.21.15.19:0/988766037 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9800005ce0 con 0x7f98180d0650 2024-03-28T13:21:38.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.085+0000 7f980ea00640 1 -- 172.21.15.19:0/988766037 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98000072f0 con 0x7f98180d0650 2024-03-28T13:21:38.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.085+0000 7f981d400640 1 -- 172.21.15.19:0/988766037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98180b0310 con 0x7f98180d0650 2024-03-28T13:21:38.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.086+0000 7f981d400640 1 -- 172.21.15.19:0/988766037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98180b08d0 con 0x7f98180d0650 2024-03-28T13:21:38.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.087+0000 7f981d400640 1 -- 172.21.15.19:0/988766037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98180685c0 con 0x7f98180d0650 2024-03-28T13:21:38.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.088+0000 7f980ea00640 1 -- 172.21.15.19:0/988766037 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f980001a070 con 0x7f98180d0650 2024-03-28T13:21:38.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.089+0000 7f980ea00640 1 --2- 172.21.15.19:0/988766037 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f97f406c530 0x7f97f406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:38.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.089+0000 7f980ea00640 1 -- 172.21.15.19:0/988766037 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f980008abe0 con 0x7f98180d0650 2024-03-28T13:21:38.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.089+0000 7f9816800640 1 --2- 172.21.15.19:0/988766037 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f97f406c530 0x7f97f406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:38.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.090+0000 7f9816800640 1 --2- 172.21.15.19:0/988766037 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f97f406c530 0x7f97f406e9f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9808009ad0 tx=0x7f9808008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:38.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.093+0000 7f980ea00640 1 -- 172.21.15.19:0/988766037 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f9800059100 con 0x7f98180d0650 2024-03-28T13:21:38.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.298+0000 7f981d400640 1 -- 172.21.15.19:0/988766037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f981806c050 con 0x7f98180d0650 2024-03-28T13:21:38.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.300+0000 7f980ea00640 1 -- 172.21.15.19:0/988766037 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+17028 (secure 0 0 0) 0x7f9800058a00 con 0x7f98180d0650 2024-03-28T13:21:38.300 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:38.300 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":123,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:21:26.921359+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:20:50.748212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-28T13:20:51.888444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":3}]},{"pgid":"17.b","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3930967142":"2024-03-29T13:21:26.921326+0000","172.21.15.19:6833/3930967142":"2024-03-29T13:21:26.921326+0000","172.21.15.188:6836/2088538359":"2024-03-29T13:21:12.469733+0000","172.21.15.188:6834/2088538359":"2024-03-29T13:21:12.469733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:21:38.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.303+0000 7f981d400640 1 -- 172.21.15.19:0/988766037 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f97f406c530 msgr2=0x7f97f406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:38.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.303+0000 7f981d400640 1 --2- 172.21.15.19:0/988766037 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f97f406c530 0x7f97f406e9f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f9808009ad0 tx=0x7f9808008040 comp rx=0 tx=0).stop 2024-03-28T13:21:38.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.303+0000 7f981d400640 1 -- 172.21.15.19:0/988766037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98180d0650 msgr2=0x7f98181f6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:38.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.303+0000 7f981d400640 1 --2- 172.21.15.19:0/988766037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98180d0650 0x7f98181f6df0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f980000c390 tx=0x7f980000c850 comp rx=0 tx=0).stop 2024-03-28T13:21:38.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.303+0000 7f981d400640 1 -- 172.21.15.19:0/988766037 shutdown_connections 2024-03-28T13:21:38.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.303+0000 7f981d400640 1 --2- 172.21.15.19:0/988766037 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f97f406c530 0x7f97f406e9f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.303+0000 7f981d400640 1 --2- 172.21.15.19:0/988766037 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f981815c390 0x7f98181f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.303+0000 7f981d400640 1 --2- 172.21.15.19:0/988766037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98180d0650 0x7f98181f6df0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.303+0000 7f981d400640 1 --2- 172.21.15.19:0/988766037 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f98180af830 0x7f98180b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.303+0000 7f981d400640 1 -- 172.21.15.19:0/988766037 >> 172.21.15.19:0/988766037 conn(0x7f981806b990 msgr2=0x7f9818159980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:38.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.303+0000 7f981d400640 1 -- 172.21.15.19:0/988766037 shutdown_connections 2024-03-28T13:21:38.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.303+0000 7f981d400640 1 -- 172.21.15.19:0/988766037 wait complete. 2024-03-28T13:21:38.324 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:20:50.748212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:20:51.888444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:21:38.324 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:21:38.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.536+0000 7fa611a00640 1 Processor -- start 2024-03-28T13:21:38.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.536+0000 7fa611a00640 1 -- start start 2024-03-28T13:21:38.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.537+0000 7fa611a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa60c0d0990 0x7fa60c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:38.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.538+0000 7fa611a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa60c0afab0 con 0x7fa60c0af2e0 2024-03-28T13:21:38.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.538+0000 7fa611a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa60c15cc80 con 0x7fa60c153580 2024-03-28T13:21:38.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.538+0000 7fa611a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa60c15da00 con 0x7fa60c0d0990 2024-03-28T13:21:38.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.538+0000 7fa60b400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa60c0af2e0 0x7fa60c1600d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34110/0 (socket says 172.21.15.19:34110) 2024-03-28T13:21:38.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.538+0000 7fa60b400640 1 -- 172.21.15.19:0/1033336780 learned_addr learned my addr 172.21.15.19:0/1033336780 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:38.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.538+0000 7fa602000640 1 --2- 172.21.15.19:0/1033336780 >> v2:172.21.15.188:3300/0 conn(0x7fa60c0d0990 0x7fa60c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:38.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.539+0000 7fa60a000640 1 -- 172.21.15.19:0/1033336780 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 249828383 0 0) 0x7fa60c0afab0 con 0x7fa60c0af2e0 2024-03-28T13:21:38.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.539+0000 7fa60a000640 1 -- 172.21.15.19:0/1033336780 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5f0003520 con 0x7fa60c0af2e0 2024-03-28T13:21:38.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.539+0000 7fa60a000640 1 -- 172.21.15.19:0/1033336780 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 178585609 0 0) 0x7fa5f0003520 con 0x7fa60c0af2e0 2024-03-28T13:21:38.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.539+0000 7fa60a000640 1 -- 172.21.15.19:0/1033336780 >> v2:172.21.15.188:3300/0 conn(0x7fa60c0d0990 msgr2=0x7fa60c0aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:38.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.539+0000 7fa60a000640 1 --2- 172.21.15.19:0/1033336780 >> v2:172.21.15.188:3300/0 conn(0x7fa60c0d0990 0x7fa60c0aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.539+0000 7fa60a000640 1 -- 172.21.15.19:0/1033336780 >> v1:172.21.15.188:6789/0 conn(0x7fa60c153580 legacy=0x7fa60c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:38.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.539+0000 7fa60a000640 1 -- 172.21.15.19:0/1033336780 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60c15cf20 con 0x7fa60c0af2e0 2024-03-28T13:21:38.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.539+0000 7fa60a000640 1 -- 172.21.15.19:0/1033336780 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fa6040019f0 con 0x7fa60c0af2e0 2024-03-28T13:21:38.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.539+0000 7fa602000640 1 --2- 172.21.15.19:0/1033336780 >> v2:172.21.15.188:3300/0 conn(0x7fa60c0d0990 0x7fa60c0aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:21:38.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.539+0000 7fa60a000640 1 -- 172.21.15.19:0/1033336780 >> v1:172.21.15.19:6789/0 conn(0x7fa60c0af2e0 legacy=0x7fa60c1600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.539+0000 7fa60a000640 1 --2- 172.21.15.19:0/1033336780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5f00039b0 0x7fa5f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.539+0000 7fa60a000640 1 --2- 172.21.15.19:0/1033336780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f0004800 0x7fa5f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.539+0000 7fa60a000640 1 --2- 172.21.15.19:0/1033336780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5f00051c0 0x7fa5f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.540+0000 7fa60a000640 1 -- 172.21.15.19:0/1033336780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c15da00 con 0x7fa5f0004800 2024-03-28T13:21:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.540+0000 7fa60a000640 1 -- 172.21.15.19:0/1033336780 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f000c160 con 0x7fa5f00051c0 2024-03-28T13:21:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.540+0000 7fa60a000640 1 -- 172.21.15.19:0/1033336780 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa5f000c360 con 0x7fa5f00039b0 2024-03-28T13:21:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.540+0000 7fa602000640 1 --2- 172.21.15.19:0/1033336780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f0004800 0x7fa5f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:38.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.540+0000 7fa60aa00640 1 --2- 172.21.15.19:0/1033336780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5f00039b0 0x7fa5f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:38.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.540+0000 7fa60b400640 1 --2- 172.21.15.19:0/1033336780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5f00051c0 0x7fa5f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:38.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.540+0000 7fa602000640 1 -- 172.21.15.19:0/1033336780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5f00039b0 msgr2=0x7fa5f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:38.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.540+0000 7fa602000640 1 --2- 172.21.15.19:0/1033336780 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5f00039b0 0x7fa5f0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.540+0000 7fa602000640 1 -- 172.21.15.19:0/1033336780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5f00051c0 msgr2=0x7fa5f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:38.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.540+0000 7fa602000640 1 --2- 172.21.15.19:0/1033336780 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5f00051c0 0x7fa5f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.540+0000 7fa602000640 1 -- 172.21.15.19:0/1033336780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f0003520 con 0x7fa5f0004800 2024-03-28T13:21:38.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.540+0000 7fa60b400640 1 -- 172.21.15.19:0/1033336780 reap_dead start 2024-03-28T13:21:38.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.541+0000 7fa602000640 1 --2- 172.21.15.19:0/1033336780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f0004800 0x7fa5f0004c80 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fa60c06f820 tx=0x7fa5f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a681f2cf20d2d4d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:38.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.542+0000 7fa60a000640 1 -- 172.21.15.19:0/1033336780 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4009860 con 0x7fa5f0004800 2024-03-28T13:21:38.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.542+0000 7fa60a000640 1 -- 172.21.15.19:0/1033336780 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f4004020 con 0x7fa5f0004800 2024-03-28T13:21:38.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.542+0000 7fa60a000640 1 -- 172.21.15.19:0/1033336780 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4004430 con 0x7fa5f0004800 2024-03-28T13:21:38.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.542+0000 7fa611a00640 1 -- 172.21.15.19:0/1033336780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f0004800 msgr2=0x7fa5f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:38.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.542+0000 7fa611a00640 1 --2- 172.21.15.19:0/1033336780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f0004800 0x7fa5f0004c80 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fa60c06f820 tx=0x7fa5f0005830 comp rx=0 tx=0).stop 2024-03-28T13:21:38.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.542+0000 7fa611a00640 1 -- 172.21.15.19:0/1033336780 shutdown_connections 2024-03-28T13:21:38.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.542+0000 7fa611a00640 1 --2- 172.21.15.19:0/1033336780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f0004800 0x7fa5f0004c80 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.542+0000 7fa611a00640 1 -- 172.21.15.19:0/1033336780 >> 172.21.15.19:0/1033336780 conn(0x7fa60c06b750 msgr2=0x7fa60c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:38.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.543+0000 7fa611a00640 1 -- 172.21.15.19:0/1033336780 shutdown_connections 2024-03-28T13:21:38.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.543+0000 7fa611a00640 1 -- 172.21.15.19:0/1033336780 wait complete. 2024-03-28T13:21:38.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.544+0000 7fa611a00640 1 Processor -- start 2024-03-28T13:21:38.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.545+0000 7fa611a00640 1 -- start start 2024-03-28T13:21:38.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.545+0000 7fa611a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f0004800 0x7fa60c14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:38.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.545+0000 7fa611a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa60c151cb0 0x7fa60c147b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:38.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.545+0000 7fa611a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa60c152130 0x7fa60c1480b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:38.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.545+0000 7fa611a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c153960 con 0x7fa5f0004800 2024-03-28T13:21:38.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.545+0000 7fa611a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa60c15d4e0 con 0x7fa60c152130 2024-03-28T13:21:38.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.545+0000 7fa611a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa60c0af6c0 con 0x7fa60c151cb0 2024-03-28T13:21:38.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.546+0000 7fa60aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f0004800 0x7fa60c14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:38.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.546+0000 7fa60aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f0004800 0x7fa60c14ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54198/0 (socket says 172.21.15.19:54198) 2024-03-28T13:21:38.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.546+0000 7fa60aa00640 1 -- 172.21.15.19:0/545027782 learned_addr learned my addr 172.21.15.19:0/545027782 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:38.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.546+0000 7fa602000640 1 --2- 172.21.15.19:0/545027782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa60c151cb0 0x7fa60c147b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:38.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.546+0000 7fa60aa00640 1 -- 172.21.15.19:0/545027782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa60c151cb0 msgr2=0x7fa60c147b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:38.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.546+0000 7fa60aa00640 1 --2- 172.21.15.19:0/545027782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa60c151cb0 0x7fa60c147b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.546+0000 7fa60aa00640 1 -- 172.21.15.19:0/545027782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa60c152130 msgr2=0x7fa60c1480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:38.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.546+0000 7fa60aa00640 1 --2- 172.21.15.19:0/545027782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa60c152130 0x7fa60c1480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.546+0000 7fa60aa00640 1 -- 172.21.15.19:0/545027782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa60c15c410 con 0x7fa5f0004800 2024-03-28T13:21:38.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.546+0000 7fa60aa00640 1 --2- 172.21.15.19:0/545027782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f0004800 0x7fa60c14ffd0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00e440 tx=0x7fa5fc00e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:38.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.547+0000 7fa603400640 1 -- 172.21.15.19:0/545027782 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc019070 con 0x7fa5f0004800 2024-03-28T13:21:38.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.547+0000 7fa603400640 1 -- 172.21.15.19:0/545027782 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5fc0052e0 con 0x7fa5f0004800 2024-03-28T13:21:38.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.547+0000 7fa611a00640 1 -- 172.21.15.19:0/545027782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa60c0cca50 con 0x7fa5f0004800 2024-03-28T13:21:38.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.547+0000 7fa603400640 1 -- 172.21.15.19:0/545027782 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc003830 con 0x7fa5f0004800 2024-03-28T13:21:38.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.547+0000 7fa611a00640 1 -- 172.21.15.19:0/545027782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa60c0cd010 con 0x7fa5f0004800 2024-03-28T13:21:38.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.548+0000 7fa611a00640 1 -- 172.21.15.19:0/545027782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa60c072020 con 0x7fa5f0004800 2024-03-28T13:21:38.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.549+0000 7fa603400640 1 -- 172.21.15.19:0/545027782 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa5fc01c070 con 0x7fa5f0004800 2024-03-28T13:21:38.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.550+0000 7fa603400640 1 --2- 172.21.15.19:0/545027782 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa5e806c530 0x7fa5e806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:38.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.550+0000 7fa603400640 1 -- 172.21.15.19:0/545027782 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7fa5fc08acc0 con 0x7fa5f0004800 2024-03-28T13:21:38.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.551+0000 7fa602000640 1 --2- 172.21.15.19:0/545027782 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa5e806c530 0x7fa5e806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:38.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.552+0000 7fa602000640 1 --2- 172.21.15.19:0/545027782 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa5e806c530 0x7fa5e806e9f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4035000 tx=0x7fa5f4030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:38.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.555+0000 7fa603400640 1 -- 172.21.15.19:0/545027782 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa5fc091080 con 0x7fa5f0004800 2024-03-28T13:21:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.785+0000 7fa611a00640 1 -- 172.21.15.19:0/545027782 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa60c0685c0 con 0x7fa5f0004800 2024-03-28T13:21:38.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.786+0000 7fa603400640 1 -- 172.21.15.19:0/545027782 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) ==== 93+0+10 (secure 0 0 0) 0x7fa5fc058b90 con 0x7fa5f0004800 2024-03-28T13:21:38.786 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:21:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.788+0000 7fa611a00640 1 -- 172.21.15.19:0/545027782 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa5e806c530 msgr2=0x7fa5e806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:38.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.788+0000 7fa611a00640 1 --2- 172.21.15.19:0/545027782 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa5e806c530 0x7fa5e806e9f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4035000 tx=0x7fa5f4030000 comp rx=0 tx=0).stop 2024-03-28T13:21:38.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.788+0000 7fa611a00640 1 -- 172.21.15.19:0/545027782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f0004800 msgr2=0x7fa60c14ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:38.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.788+0000 7fa611a00640 1 --2- 172.21.15.19:0/545027782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f0004800 0x7fa60c14ffd0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00e440 tx=0x7fa5fc00e840 comp rx=0 tx=0).stop 2024-03-28T13:21:38.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.788+0000 7fa611a00640 1 -- 172.21.15.19:0/545027782 shutdown_connections 2024-03-28T13:21:38.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.788+0000 7fa611a00640 1 --2- 172.21.15.19:0/545027782 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa5e806c530 0x7fa5e806e9f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.788+0000 7fa611a00640 1 --2- 172.21.15.19:0/545027782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa60c152130 0x7fa60c1480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.788+0000 7fa611a00640 1 --2- 172.21.15.19:0/545027782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa60c151cb0 0x7fa60c147b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.788+0000 7fa611a00640 1 --2- 172.21.15.19:0/545027782 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5f0004800 0x7fa60c14ffd0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.788+0000 7fa611a00640 1 -- 172.21.15.19:0/545027782 >> 172.21.15.19:0/545027782 conn(0x7fa60c06b750 msgr2=0x7fa60c15de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.788+0000 7fa611a00640 1 -- 172.21.15.19:0/545027782 shutdown_connections 2024-03-28T13:21:38.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:38.788+0000 7fa611a00640 1 -- 172.21.15.19:0/545027782 wait complete. 2024-03-28T13:21:38.809 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:21:39.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.041+0000 7f6468c00640 1 Processor -- start 2024-03-28T13:21:39.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.041+0000 7f6468c00640 1 -- start start 2024-03-28T13:21:39.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.043+0000 7f6468c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f646415b260 0x7f64640b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.043+0000 7f6468c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64640b06b0 0x7f64640aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.043+0000 7f6468c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f646415ff50 con 0x7f646415b640 2024-03-28T13:21:39.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.043+0000 7f6468c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f646415f340 con 0x7f646415b260 2024-03-28T13:21:39.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.043+0000 7f6468c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f646415f620 con 0x7f64640b06b0 2024-03-28T13:21:39.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.043+0000 7f6461400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f646415b640 0x7f64640affa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34124/0 (socket says 172.21.15.19:34124) 2024-03-28T13:21:39.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.043+0000 7f6461400640 1 -- 172.21.15.19:0/4214924569 learned_addr learned my addr 172.21.15.19:0/4214924569 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:39.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.043+0000 7f6461e00640 1 --2- 172.21.15.19:0/4214924569 >> v2:172.21.15.19:3300/0 conn(0x7f646415b260 0x7f64640b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.044+0000 7f6462800640 1 --2- 172.21.15.19:0/4214924569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64640b06b0 0x7f64640aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.044+0000 7f6461e00640 1 -- 172.21.15.19:0/4214924569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64640b06b0 msgr2=0x7f64640aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:39.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.044+0000 7f6461e00640 1 --2- 172.21.15.19:0/4214924569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64640b06b0 0x7f64640aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.044+0000 7f6461e00640 1 -- 172.21.15.19:0/4214924569 >> v1:172.21.15.19:6789/0 conn(0x7f646415b640 legacy=0x7f64640affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.044+0000 7f6461e00640 1 -- 172.21.15.19:0/4214924569 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646415f900 con 0x7f646415b260 2024-03-28T13:21:39.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.044+0000 7f6460a00640 1 -- 172.21.15.19:0/4214924569 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 168546319 0 0) 0x7f646415ff50 con 0x7f646415b640 2024-03-28T13:21:39.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.044+0000 7f6462800640 1 --2- 172.21.15.19:0/4214924569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64640b06b0 0x7f64640aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:21:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.045+0000 7f6461e00640 1 --2- 172.21.15.19:0/4214924569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f646415b260 0x7f64640b40b0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f64540097a0 tx=0x7f645402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb74cfe23665374 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.046+0000 7f6460a00640 1 -- 172.21.15.19:0/4214924569 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454035020 con 0x7f646415b260 2024-03-28T13:21:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.046+0000 7f6460a00640 1 -- 172.21.15.19:0/4214924569 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6454031830 con 0x7f646415b260 2024-03-28T13:21:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.046+0000 7f6460a00640 1 -- 172.21.15.19:0/4214924569 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454031c00 con 0x7f646415b260 2024-03-28T13:21:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.046+0000 7f6468c00640 1 -- 172.21.15.19:0/4214924569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f646415b260 msgr2=0x7f64640b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.046+0000 7f6468c00640 1 --2- 172.21.15.19:0/4214924569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f646415b260 0x7f64640b40b0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f64540097a0 tx=0x7f645402b8d0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.046+0000 7f6468c00640 1 -- 172.21.15.19:0/4214924569 shutdown_connections 2024-03-28T13:21:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.046+0000 7f6468c00640 1 --2- 172.21.15.19:0/4214924569 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64640b06b0 0x7f64640aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.046+0000 7f6468c00640 1 --2- 172.21.15.19:0/4214924569 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f646415b260 0x7f64640b40b0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.046+0000 7f6468c00640 1 -- 172.21.15.19:0/4214924569 >> 172.21.15.19:0/4214924569 conn(0x7f64640b65a0 msgr2=0x7f64640b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.046+0000 7f6468c00640 1 -- 172.21.15.19:0/4214924569 shutdown_connections 2024-03-28T13:21:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.046+0000 7f6468c00640 1 -- 172.21.15.19:0/4214924569 wait complete. 2024-03-28T13:21:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.048+0000 7f6468c00640 1 Processor -- start 2024-03-28T13:21:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.048+0000 7f6468c00640 1 -- start start 2024-03-28T13:21:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.049+0000 7f6468c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64640b06b0 0x7f64640cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.049+0000 7f6468c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f646415b260 0x7f64640b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.049+0000 7f6468c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f646415b640 0x7f6464152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.049+0000 7f6468c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f646406e770 con 0x7f646415b260 2024-03-28T13:21:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.049+0000 7f6468c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f646415f010 con 0x7f646415b640 2024-03-28T13:21:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.049+0000 7f6468c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f64640b45f0 con 0x7f64640b06b0 2024-03-28T13:21:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.050+0000 7f6461400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f646415b260 0x7f64640b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.050+0000 7f6461400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f646415b260 0x7f64640b4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54212/0 (socket says 172.21.15.19:54212) 2024-03-28T13:21:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.050+0000 7f6461400640 1 -- 172.21.15.19:0/2071431097 learned_addr learned my addr 172.21.15.19:0/2071431097 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.050+0000 7f6461e00640 1 --2- 172.21.15.19:0/2071431097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64640b06b0 0x7f64640cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.050+0000 7f6462800640 1 --2- 172.21.15.19:0/2071431097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f646415b640 0x7f6464152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.050+0000 7f6461400640 1 -- 172.21.15.19:0/2071431097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64640b06b0 msgr2=0x7f64640cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.050+0000 7f6461400640 1 --2- 172.21.15.19:0/2071431097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64640b06b0 0x7f64640cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.050+0000 7f6461400640 1 -- 172.21.15.19:0/2071431097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f646415b640 msgr2=0x7f6464152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.050+0000 7f6461400640 1 --2- 172.21.15.19:0/2071431097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f646415b640 0x7f6464152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.050+0000 7f6461400640 1 -- 172.21.15.19:0/2071431097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646415caa0 con 0x7f646415b260 2024-03-28T13:21:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.050+0000 7f6461400640 1 --2- 172.21.15.19:0/2071431097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f646415b260 0x7f64640b4dc0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f645000c410 tx=0x7f645000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.051+0000 7f645a000640 1 -- 172.21.15.19:0/2071431097 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64500074e0 con 0x7f646415b260 2024-03-28T13:21:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.051+0000 7f645a000640 1 -- 172.21.15.19:0/2071431097 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64500047e0 con 0x7f646415b260 2024-03-28T13:21:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.051+0000 7f6468c00640 1 -- 172.21.15.19:0/2071431097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646415bf20 con 0x7f646415b260 2024-03-28T13:21:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.051+0000 7f645a000640 1 -- 172.21.15.19:0/2071431097 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6450004bd0 con 0x7f646415b260 2024-03-28T13:21:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.051+0000 7f6468c00640 1 -- 172.21.15.19:0/2071431097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f646415c4e0 con 0x7f646415b260 2024-03-28T13:21:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.052+0000 7f6468c00640 1 -- 172.21.15.19:0/2071431097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64640b1690 con 0x7f646415b260 2024-03-28T13:21:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.054+0000 7f645a000640 1 -- 172.21.15.19:0/2071431097 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6450011070 con 0x7f646415b260 2024-03-28T13:21:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.055+0000 7f645a000640 1 --2- 172.21.15.19:0/2071431097 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f644406c530 0x7f644406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.055+0000 7f645a000640 1 -- 172.21.15.19:0/2071431097 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f645008b530 con 0x7f646415b260 2024-03-28T13:21:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.056+0000 7f6461e00640 1 --2- 172.21.15.19:0/2071431097 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f644406c530 0x7f644406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.057+0000 7f6461e00640 1 --2- 172.21.15.19:0/2071431097 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f644406c530 0x7f644406e9f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6454036000 tx=0x7f6454008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.060+0000 7f645a000640 1 -- 172.21.15.19:0/2071431097 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f645000e280 con 0x7f646415b260 2024-03-28T13:21:39.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.263+0000 7f6468c00640 1 -- 172.21.15.19:0/2071431097 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6464068cd0 con 0x7f646415b260 2024-03-28T13:21:39.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.264+0000 7f645a000640 1 -- 172.21.15.19:0/2071431097 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) ==== 104+0+11 (secure 0 0 0) 0x7f6450059930 con 0x7f646415b260 2024-03-28T13:21:39.264 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:21:39.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.266+0000 7f6468c00640 1 -- 172.21.15.19:0/2071431097 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f644406c530 msgr2=0x7f644406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.266+0000 7f6468c00640 1 --2- 172.21.15.19:0/2071431097 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f644406c530 0x7f644406e9f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f6454036000 tx=0x7f6454008030 comp rx=0 tx=0).stop 2024-03-28T13:21:39.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.266+0000 7f6468c00640 1 -- 172.21.15.19:0/2071431097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f646415b260 msgr2=0x7f64640b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.266+0000 7f6468c00640 1 --2- 172.21.15.19:0/2071431097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f646415b260 0x7f64640b4dc0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f645000c410 tx=0x7f645000c8d0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.267+0000 7f6468c00640 1 -- 172.21.15.19:0/2071431097 shutdown_connections 2024-03-28T13:21:39.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.267+0000 7f6468c00640 1 --2- 172.21.15.19:0/2071431097 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f644406c530 0x7f644406e9f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.267+0000 7f6468c00640 1 --2- 172.21.15.19:0/2071431097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f646415b640 0x7f6464152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.267+0000 7f6468c00640 1 --2- 172.21.15.19:0/2071431097 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f646415b260 0x7f64640b4dc0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.267+0000 7f6468c00640 1 --2- 172.21.15.19:0/2071431097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64640b06b0 0x7f64640cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.267+0000 7f6468c00640 1 -- 172.21.15.19:0/2071431097 >> 172.21.15.19:0/2071431097 conn(0x7f64640b65a0 msgr2=0x7f6464153b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:39.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.267+0000 7f6468c00640 1 -- 172.21.15.19:0/2071431097 shutdown_connections 2024-03-28T13:21:39.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.267+0000 7f6468c00640 1 -- 172.21.15.19:0/2071431097 wait complete. 2024-03-28T13:21:39.287 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:21:39.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.503+0000 7fa5ad200640 1 Processor -- start 2024-03-28T13:21:39.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.503+0000 7fa5ad200640 1 -- start start 2024-03-28T13:21:39.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.504+0000 7fa5ad200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa5a815e4c0 0x7fa5a80b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.504+0000 7fa5ad200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa5a80b69c0 0x7fa5a80b7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.504+0000 7fa5ad200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a80d2710 con 0x7fa5a80d37b0 2024-03-28T13:21:39.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.504+0000 7fa5ad200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a815e8a0 con 0x7fa5a815e4c0 2024-03-28T13:21:39.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.504+0000 7fa5ad200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a80d4040 con 0x7fa5a80b69c0 2024-03-28T13:21:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.505+0000 7fa5a7400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa5a815e4c0 0x7fa5a80b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.505+0000 7fa5a6a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa5a80d37b0 0x7fa5a80b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34140/0 (socket says 172.21.15.19:34140) 2024-03-28T13:21:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.505+0000 7fa5a6a00640 1 -- 172.21.15.19:0/1589090533 learned_addr learned my addr 172.21.15.19:0/1589090533 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:39.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.505+0000 7fa5a7e00640 1 --2- 172.21.15.19:0/1589090533 >> v2:172.21.15.188:3300/0 conn(0x7fa5a80b69c0 0x7fa5a80b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.505+0000 7fa5a7400640 1 -- 172.21.15.19:0/1589090533 >> v2:172.21.15.188:3300/0 conn(0x7fa5a80b69c0 msgr2=0x7fa5a80b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.505+0000 7fa5a7400640 1 --2- 172.21.15.19:0/1589090533 >> v2:172.21.15.188:3300/0 conn(0x7fa5a80b69c0 0x7fa5a80b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.506+0000 7fa5a7400640 1 -- 172.21.15.19:0/1589090533 >> v1:172.21.15.19:6789/0 conn(0x7fa5a80d37b0 legacy=0x7fa5a80b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.506+0000 7fa5a7400640 1 -- 172.21.15.19:0/1589090533 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a80d29f0 con 0x7fa5a815e4c0 2024-03-28T13:21:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.506+0000 7fa5a6000640 1 -- 172.21.15.19:0/1589090533 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3276986492 0 0) 0x7fa5a80d2710 con 0x7fa5a80d37b0 2024-03-28T13:21:39.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.506+0000 7fa5a7e00640 1 --2- 172.21.15.19:0/1589090533 >> v2:172.21.15.188:3300/0 conn(0x7fa5a80b69c0 0x7fa5a80b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:21:39.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.507+0000 7fa5a7400640 1 --2- 172.21.15.19:0/1589090533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a815e4c0 0x7fa5a80b9bf0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80d13c0 tx=0x7fa59802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a2c205a05739af server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:39.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.507+0000 7fa5a6000640 1 -- 172.21.15.19:0/1589090533 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598035020 con 0x7fa5a815e4c0 2024-03-28T13:21:39.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.507+0000 7fa5a6000640 1 -- 172.21.15.19:0/1589090533 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa598031830 con 0x7fa5a815e4c0 2024-03-28T13:21:39.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.507+0000 7fa5a6000640 1 -- 172.21.15.19:0/1589090533 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598031c00 con 0x7fa5a815e4c0 2024-03-28T13:21:39.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.508+0000 7fa5ad200640 1 -- 172.21.15.19:0/1589090533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a815e4c0 msgr2=0x7fa5a80b9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.508+0000 7fa5ad200640 1 --2- 172.21.15.19:0/1589090533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a815e4c0 0x7fa5a80b9bf0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fa5a80d13c0 tx=0x7fa59802b8d0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.508+0000 7fa5ad200640 1 -- 172.21.15.19:0/1589090533 shutdown_connections 2024-03-28T13:21:39.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.508+0000 7fa5ad200640 1 --2- 172.21.15.19:0/1589090533 >> v2:172.21.15.188:3300/0 conn(0x7fa5a80b69c0 0x7fa5a80b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.508+0000 7fa5ad200640 1 --2- 172.21.15.19:0/1589090533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a815e4c0 0x7fa5a80b9bf0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.508+0000 7fa5ad200640 1 -- 172.21.15.19:0/1589090533 >> 172.21.15.19:0/1589090533 conn(0x7fa5a80676a0 msgr2=0x7fa5a8067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:39.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.508+0000 7fa5ad200640 1 -- 172.21.15.19:0/1589090533 shutdown_connections 2024-03-28T13:21:39.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.508+0000 7fa5ad200640 1 -- 172.21.15.19:0/1589090533 wait complete. 2024-03-28T13:21:39.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.510+0000 7fa5ad200640 1 Processor -- start 2024-03-28T13:21:39.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.510+0000 7fa5ad200640 1 -- start start 2024-03-28T13:21:39.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.511+0000 7fa5ad200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a80b69c0 0x7fa5a80cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.511+0000 7fa5ad200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5a80d37b0 0x7fa5a81f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.511+0000 7fa5ad200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a815e4c0 0x7fa5a81505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.511+0000 7fa5ad200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a80d2fb0 con 0x7fa5a815e4c0 2024-03-28T13:21:39.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.511+0000 7fa5ad200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a80d3320 con 0x7fa5a80b69c0 2024-03-28T13:21:39.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.511+0000 7fa5ad200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a80ba130 con 0x7fa5a80d37b0 2024-03-28T13:21:39.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.512+0000 7fa5a7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a815e4c0 0x7fa5a81505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.512+0000 7fa5a7e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a815e4c0 0x7fa5a81505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54230/0 (socket says 172.21.15.19:54230) 2024-03-28T13:21:39.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.512+0000 7fa5a7e00640 1 -- 172.21.15.19:0/1397818904 learned_addr learned my addr 172.21.15.19:0/1397818904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:39.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.512+0000 7fa5a7400640 1 --2- 172.21.15.19:0/1397818904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a80b69c0 0x7fa5a80cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.512+0000 7fa5a6a00640 1 --2- 172.21.15.19:0/1397818904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5a80d37b0 0x7fa5a81f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.512+0000 7fa5a7e00640 1 -- 172.21.15.19:0/1397818904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5a80d37b0 msgr2=0x7fa5a81f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.512+0000 7fa5a7e00640 1 --2- 172.21.15.19:0/1397818904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5a80d37b0 0x7fa5a81f4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.512+0000 7fa5a7e00640 1 -- 172.21.15.19:0/1397818904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a80b69c0 msgr2=0x7fa5a80cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.512+0000 7fa5a7e00640 1 --2- 172.21.15.19:0/1397818904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a80b69c0 0x7fa5a80cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.512+0000 7fa5a7e00640 1 -- 172.21.15.19:0/1397818904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a80cc1f0 con 0x7fa5a815e4c0 2024-03-28T13:21:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.513+0000 7fa5a7e00640 1 --2- 172.21.15.19:0/1397818904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a815e4c0 0x7fa5a81505e0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0007ef0 tx=0x7fa5a000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.513+0000 7fa59f400640 1 -- 172.21.15.19:0/1397818904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a001a070 con 0x7fa5a815e4c0 2024-03-28T13:21:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.513+0000 7fa59f400640 1 -- 172.21.15.19:0/1397818904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a0015600 con 0x7fa5a815e4c0 2024-03-28T13:21:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.513+0000 7fa5ad200640 1 -- 172.21.15.19:0/1397818904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a81f46c0 con 0x7fa5a815e4c0 2024-03-28T13:21:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.513+0000 7fa59f400640 1 -- 172.21.15.19:0/1397818904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a00159f0 con 0x7fa5a815e4c0 2024-03-28T13:21:39.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.513+0000 7fa5ad200640 1 -- 172.21.15.19:0/1397818904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a8160130 con 0x7fa5a815e4c0 2024-03-28T13:21:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.514+0000 7fa5ad200640 1 -- 172.21.15.19:0/1397818904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a8075940 con 0x7fa5a815e4c0 2024-03-28T13:21:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.515+0000 7fa59f400640 1 -- 172.21.15.19:0/1397818904 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa5a001d070 con 0x7fa5a815e4c0 2024-03-28T13:21:39.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.516+0000 7fa59f400640 1 --2- 172.21.15.19:0/1397818904 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa58806c530 0x7fa58806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.516+0000 7fa59f400640 1 -- 172.21.15.19:0/1397818904 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7fa5a008bb00 con 0x7fa5a815e4c0 2024-03-28T13:21:39.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.517+0000 7fa5a7400640 1 --2- 172.21.15.19:0/1397818904 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa58806c530 0x7fa58806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.518+0000 7fa5a7400640 1 --2- 172.21.15.19:0/1397818904 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa58806c530 0x7fa58806e9f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa598036000 tx=0x7fa59803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:39.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.521+0000 7fa59f400640 1 -- 172.21.15.19:0/1397818904 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa5a80cc1f0 con 0x7fa5a815e4c0 2024-03-28T13:21:39.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.722+0000 7fa5ad200640 1 -- 172.21.15.19:0/1397818904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa5a806caf0 con 0x7fa5a815e4c0 2024-03-28T13:21:39.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.723+0000 7fa59f400640 1 -- 172.21.15.19:0/1397818904 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) ==== 100+0+11 (secure 0 0 0) 0x7fa5a008be20 con 0x7fa5a815e4c0 2024-03-28T13:21:39.722 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:21:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.725+0000 7fa5ad200640 1 -- 172.21.15.19:0/1397818904 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa58806c530 msgr2=0x7fa58806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.725+0000 7fa5ad200640 1 --2- 172.21.15.19:0/1397818904 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa58806c530 0x7fa58806e9f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa598036000 tx=0x7fa59803c040 comp rx=0 tx=0).stop 2024-03-28T13:21:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.725+0000 7fa5ad200640 1 -- 172.21.15.19:0/1397818904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a815e4c0 msgr2=0x7fa5a81505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.725+0000 7fa5ad200640 1 --2- 172.21.15.19:0/1397818904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a815e4c0 0x7fa5a81505e0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0007ef0 tx=0x7fa5a000d4a0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.725+0000 7fa5ad200640 1 -- 172.21.15.19:0/1397818904 shutdown_connections 2024-03-28T13:21:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.725+0000 7fa5ad200640 1 --2- 172.21.15.19:0/1397818904 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa58806c530 0x7fa58806e9f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.725+0000 7fa5ad200640 1 --2- 172.21.15.19:0/1397818904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa5a815e4c0 0x7fa5a81505e0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.725+0000 7fa5ad200640 1 --2- 172.21.15.19:0/1397818904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5a80d37b0 0x7fa5a81f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.725+0000 7fa5ad200640 1 --2- 172.21.15.19:0/1397818904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5a80b69c0 0x7fa5a80cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.725+0000 7fa5ad200640 1 -- 172.21.15.19:0/1397818904 >> 172.21.15.19:0/1397818904 conn(0x7fa5a80676a0 msgr2=0x7fa5a80c7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:39.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.725+0000 7fa5ad200640 1 -- 172.21.15.19:0/1397818904 shutdown_connections 2024-03-28T13:21:39.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.726+0000 7fa5ad200640 1 -- 172.21.15.19:0/1397818904 wait complete. 2024-03-28T13:21:39.741 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-28T13:21:39.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.979+0000 7f87da200640 1 Processor -- start 2024-03-28T13:21:39.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.979+0000 7f87da200640 1 -- start start 2024-03-28T13:21:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.980+0000 7f87da200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f87d415d230 0x7f87d41530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.981+0000 7f87da200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87d4153620 con 0x7f87d40afaa0 2024-03-28T13:21:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.981+0000 7f87da200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87d40b3ee0 con 0x7f87d40cd890 2024-03-28T13:21:39.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.981+0000 7f87da200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f87d40b41c0 con 0x7f87d415d230 2024-03-28T13:21:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.981+0000 7f87d8e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f87d40afaa0 0x7f87d40cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34154/0 (socket says 172.21.15.19:34154) 2024-03-28T13:21:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.981+0000 7f87d8e00640 1 -- 172.21.15.19:0/124816709 learned_addr learned my addr 172.21.15.19:0/124816709 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:39.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.981+0000 7f87d9800640 1 --2- 172.21.15.19:0/124816709 >> v2:172.21.15.19:3300/0 conn(0x7f87d415d230 0x7f87d41530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.981+0000 7f87d3400640 1 -- 172.21.15.19:0/124816709 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3141891422 0 0) 0x7f87d4153620 con 0x7f87d40afaa0 2024-03-28T13:21:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.982+0000 7f87d3400640 1 -- 172.21.15.19:0/124816709 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87bc003520 con 0x7f87d40afaa0 2024-03-28T13:21:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.982+0000 7f87d9800640 1 -- 172.21.15.19:0/124816709 >> v1:172.21.15.188:6789/0 conn(0x7f87d40cd890 legacy=0x7f87d41529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.982+0000 7f87d9800640 1 -- 172.21.15.19:0/124816709 >> v1:172.21.15.19:6789/0 conn(0x7f87d40afaa0 legacy=0x7f87d40cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.982+0000 7f87d9800640 1 -- 172.21.15.19:0/124816709 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d40b44a0 con 0x7f87d415d230 2024-03-28T13:21:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.983+0000 7f87d9800640 1 --2- 172.21.15.19:0/124816709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d415d230 0x7f87d41530e0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009dc0 tx=0x7f87cc02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5a708a1dbb5da4a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.983+0000 7f87d3400640 1 -- 172.21.15.19:0/124816709 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc039070 con 0x7f87d415d230 2024-03-28T13:21:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.983+0000 7f87d3400640 1 -- 172.21.15.19:0/124816709 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87cc02cc40 con 0x7f87d415d230 2024-03-28T13:21:39.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.983+0000 7f87d3400640 1 -- 172.21.15.19:0/124816709 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc034440 con 0x7f87d415d230 2024-03-28T13:21:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.984+0000 7f87da200640 1 -- 172.21.15.19:0/124816709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d415d230 msgr2=0x7f87d41530e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.984+0000 7f87da200640 1 --2- 172.21.15.19:0/124816709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d415d230 0x7f87d41530e0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009dc0 tx=0x7f87cc02be90 comp rx=0 tx=0).stop 2024-03-28T13:21:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.984+0000 7f87da200640 1 -- 172.21.15.19:0/124816709 shutdown_connections 2024-03-28T13:21:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.984+0000 7f87da200640 1 --2- 172.21.15.19:0/124816709 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d415d230 0x7f87d41530e0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.984+0000 7f87da200640 1 -- 172.21.15.19:0/124816709 >> 172.21.15.19:0/124816709 conn(0x7f87d40d0770 msgr2=0x7f87d40d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.984+0000 7f87da200640 1 -- 172.21.15.19:0/124816709 shutdown_connections 2024-03-28T13:21:39.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.984+0000 7f87da200640 1 -- 172.21.15.19:0/124816709 wait complete. 2024-03-28T13:21:39.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.987+0000 7f87da200640 1 Processor -- start 2024-03-28T13:21:39.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.987+0000 7f87da200640 1 -- start start 2024-03-28T13:21:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.988+0000 7f87da200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d40afaa0 0x7f87d40cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.988+0000 7f87da200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d40cd890 0x7f87d40aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.988+0000 7f87da200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87d415d230 0x7f87d406c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.988+0000 7f87da200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87d40b3b90 con 0x7f87d40cd890 2024-03-28T13:21:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.988+0000 7f87da200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f87d415d680 con 0x7f87d40afaa0 2024-03-28T13:21:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.988+0000 7f87da200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f87d40b4a60 con 0x7f87d415d230 2024-03-28T13:21:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.989+0000 7f87d3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d40cd890 0x7f87d40aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.989+0000 7f87d3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d40cd890 0x7f87d40aaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54252/0 (socket says 172.21.15.19:54252) 2024-03-28T13:21:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.989+0000 7f87d3e00640 1 -- 172.21.15.19:0/1758404297 learned_addr learned my addr 172.21.15.19:0/1758404297 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.989+0000 7f87d8e00640 1 --2- 172.21.15.19:0/1758404297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d40afaa0 0x7f87d40cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.989+0000 7f87d9800640 1 --2- 172.21.15.19:0/1758404297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87d415d230 0x7f87d406c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.989+0000 7f87d3e00640 1 -- 172.21.15.19:0/1758404297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87d415d230 msgr2=0x7f87d406c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.989+0000 7f87d3e00640 1 --2- 172.21.15.19:0/1758404297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87d415d230 0x7f87d406c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.989+0000 7f87d3e00640 1 -- 172.21.15.19:0/1758404297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d40afaa0 msgr2=0x7f87d40cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:39.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.989+0000 7f87d3e00640 1 --2- 172.21.15.19:0/1758404297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d40afaa0 0x7f87d40cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:39.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.989+0000 7f87d3e00640 1 -- 172.21.15.19:0/1758404297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d40ac3c0 con 0x7f87d40cd890 2024-03-28T13:21:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.990+0000 7f87d3e00640 1 --2- 172.21.15.19:0/1758404297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d40cd890 0x7f87d40aaee0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f87c000c430 tx=0x7f87c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.990+0000 7f87d0c00640 1 -- 172.21.15.19:0/1758404297 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c00052d0 con 0x7f87d40cd890 2024-03-28T13:21:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.990+0000 7f87d0c00640 1 -- 172.21.15.19:0/1758404297 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c0005490 con 0x7f87d40cd890 2024-03-28T13:21:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.990+0000 7f87d0c00640 1 -- 172.21.15.19:0/1758404297 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0007260 con 0x7f87d40cd890 2024-03-28T13:21:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.990+0000 7f87da200640 1 -- 172.21.15.19:0/1758404297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d40ac6a0 con 0x7f87d40cd890 2024-03-28T13:21:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.990+0000 7f87da200640 1 -- 172.21.15.19:0/1758404297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d40acc60 con 0x7f87d40cd890 2024-03-28T13:21:39.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.991+0000 7f87da200640 1 -- 172.21.15.19:0/1758404297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d4075940 con 0x7f87d40cd890 2024-03-28T13:21:39.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.993+0000 7f87d0c00640 1 -- 172.21.15.19:0/1758404297 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f87c0007400 con 0x7f87d40cd890 2024-03-28T13:21:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.994+0000 7f87d0c00640 1 --2- 172.21.15.19:0/1758404297 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f87b406c530 0x7f87b406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.994+0000 7f87d0c00640 1 -- 172.21.15.19:0/1758404297 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f87c008ac70 con 0x7f87d40cd890 2024-03-28T13:21:39.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.995+0000 7f87d8e00640 1 --2- 172.21.15.19:0/1758404297 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f87b406c530 0x7f87b406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:39.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.996+0000 7f87d8e00640 1 --2- 172.21.15.19:0/1758404297 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f87b406c530 0x7f87b406e9f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f87c80011e0 tx=0x7f87c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:39.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:39.999+0000 7f87d0c00640 1 -- 172.21.15.19:0/1758404297 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f87c0059230 con 0x7f87d40cd890 2024-03-28T13:21:40.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.359+0000 7f87da200640 1 -- 172.21.15.19:0/1758404297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f87d40ace30 con 0x7f87d40cd890 2024-03-28T13:21:40.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.360+0000 7f87d0c00640 1 -- 172.21.15.19:0/1758404297 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v123) ==== 80+0+246 (secure 0 0 0) 0x7f87c0058b30 con 0x7f87d40cd890 2024-03-28T13:21:40.360 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6835/3930967142 2024-03-29T13:21:26.921326+0000 2024-03-28T13:21:40.360 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6833/3930967142 2024-03-29T13:21:26.921326+0000 2024-03-28T13:21:40.360 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.188:6836/2088538359 2024-03-29T13:21:12.469733+0000 2024-03-28T13:21:40.360 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.188:6834/2088538359 2024-03-29T13:21:12.469733+0000 2024-03-28T13:21:40.360 INFO:teuthology.orchestra.run.smithi019.stderr:listed 4 entries 2024-03-28T13:21:40.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.363+0000 7f87da200640 1 -- 172.21.15.19:0/1758404297 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f87b406c530 msgr2=0x7f87b406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.363+0000 7f87da200640 1 --2- 172.21.15.19:0/1758404297 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f87b406c530 0x7f87b406e9f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f87c80011e0 tx=0x7f87c8009040 comp rx=0 tx=0).stop 2024-03-28T13:21:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.363+0000 7f87da200640 1 -- 172.21.15.19:0/1758404297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d40cd890 msgr2=0x7f87d40aaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.363+0000 7f87da200640 1 --2- 172.21.15.19:0/1758404297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d40cd890 0x7f87d40aaee0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f87c000c430 tx=0x7f87c000c8f0 comp rx=0 tx=0).stop 2024-03-28T13:21:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.363+0000 7f87da200640 1 -- 172.21.15.19:0/1758404297 shutdown_connections 2024-03-28T13:21:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.363+0000 7f87da200640 1 --2- 172.21.15.19:0/1758404297 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f87b406c530 0x7f87b406e9f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.363+0000 7f87da200640 1 --2- 172.21.15.19:0/1758404297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87d415d230 0x7f87d406c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.363+0000 7f87da200640 1 --2- 172.21.15.19:0/1758404297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87d40cd890 0x7f87d40aaee0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.363+0000 7f87da200640 1 --2- 172.21.15.19:0/1758404297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87d40afaa0 0x7f87d40cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.363+0000 7f87da200640 1 -- 172.21.15.19:0/1758404297 >> 172.21.15.19:0/1758404297 conn(0x7f87d40d0770 msgr2=0x7f87d415e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.363+0000 7f87da200640 1 -- 172.21.15.19:0/1758404297 shutdown_connections 2024-03-28T13:21:40.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.363+0000 7f87da200640 1 -- 172.21.15.19:0/1758404297 wait complete. 2024-03-28T13:21:40.380 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-28T13:21:40.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:40.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-28T13:21:40.488 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:21:40.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:40.488 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-28T13:21:40.530 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:21:40.530 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:21:40.530 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:40.587 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:21:40.588 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:21:40.588 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:40.588 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:40.641 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:21:40.641 INFO:tasks.cephfs.mount:Terminating background process 2024-03-28T13:21:40.641 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:21:40.641 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:40.697 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:21:40.698 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:21:40.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.912+0000 7f8033600640 1 Processor -- start 2024-03-28T13:21:40.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.912+0000 7f8033600640 1 -- start start 2024-03-28T13:21:40.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.913+0000 7f8033600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f802c15c390 0x7f802c073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.913+0000 7f8033600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f802c0af830 0x7f802c160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.913+0000 7f8033600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f802c15fe80 con 0x7f802c160b80 2024-03-28T13:21:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.913+0000 7f8033600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f802c15c770 con 0x7f802c15c390 2024-03-28T13:21:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.913+0000 7f8033600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f802c0ad120 con 0x7f802c0af830 2024-03-28T13:21:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.914+0000 7f8030c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f802c15c390 0x7f802c073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.914+0000 7f8030c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f802c15c390 0x7f802c073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54254/0 (socket says 172.21.15.19:54254) 2024-03-28T13:21:40.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.914+0000 7f8030c00640 1 -- 172.21.15.19:0/1507962261 learned_addr learned my addr 172.21.15.19:0/1507962261 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.914+0000 7f8030c00640 1 -- 172.21.15.19:0/1507962261 >> v2:172.21.15.188:3300/0 conn(0x7f802c0af830 msgr2=0x7f802c160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.914+0000 7f8030c00640 1 --2- 172.21.15.19:0/1507962261 >> v2:172.21.15.188:3300/0 conn(0x7f802c0af830 0x7f802c160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.914+0000 7f8030c00640 1 -- 172.21.15.19:0/1507962261 >> v1:172.21.15.188:6789/0 conn(0x7f802c160b80 legacy=0x7f802c0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.914+0000 7f8030c00640 1 -- 172.21.15.19:0/1507962261 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802c0ad400 con 0x7f802c15c390 2024-03-28T13:21:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.916+0000 7f8030c00640 1 --2- 172.21.15.19:0/1507962261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c15c390 0x7f802c073790 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f802c06eaa0 tx=0x7f801c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef38ff86d13fb98b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.916+0000 7f802b400640 1 -- 172.21.15.19:0/1507962261 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c02ca80 con 0x7f802c15c390 2024-03-28T13:21:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.916+0000 7f802b400640 1 -- 172.21.15.19:0/1507962261 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f801c02cc20 con 0x7f802c15c390 2024-03-28T13:21:40.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.916+0000 7f802b400640 1 -- 172.21.15.19:0/1507962261 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c0343f0 con 0x7f802c15c390 2024-03-28T13:21:40.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.917+0000 7f8033600640 1 -- 172.21.15.19:0/1507962261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c15c390 msgr2=0x7f802c073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:40.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.917+0000 7f8033600640 1 --2- 172.21.15.19:0/1507962261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c15c390 0x7f802c073790 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f802c06eaa0 tx=0x7f801c02bee0 comp rx=0 tx=0).stop 2024-03-28T13:21:40.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.917+0000 7f8033600640 1 -- 172.21.15.19:0/1507962261 shutdown_connections 2024-03-28T13:21:40.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.917+0000 7f8033600640 1 --2- 172.21.15.19:0/1507962261 >> v2:172.21.15.188:3300/0 conn(0x7f802c0af830 0x7f802c160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:40.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.917+0000 7f8033600640 1 --2- 172.21.15.19:0/1507962261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c15c390 0x7f802c073790 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:40.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.917+0000 7f8033600640 1 -- 172.21.15.19:0/1507962261 >> 172.21.15.19:0/1507962261 conn(0x7f802c06b990 msgr2=0x7f802c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:40.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.917+0000 7f8033600640 1 -- 172.21.15.19:0/1507962261 shutdown_connections 2024-03-28T13:21:40.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.917+0000 7f8033600640 1 -- 172.21.15.19:0/1507962261 wait complete. 2024-03-28T13:21:40.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.919+0000 7f8033600640 1 Processor -- start 2024-03-28T13:21:40.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.919+0000 7f8033600640 1 -- start start 2024-03-28T13:21:40.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.920+0000 7f8033600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0af830 0x7f802c15c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:40.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.920+0000 7f8033600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f802c15c390 0x7f802c14cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:40.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.920+0000 7f8033600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f802c160b80 0x7f802c0b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:40.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.920+0000 7f8033600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f802c0cf980 con 0x7f802c0af830 2024-03-28T13:21:40.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.920+0000 7f8033600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f802c160120 con 0x7f802c160b80 2024-03-28T13:21:40.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.920+0000 7f8033600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f802c0d0d60 con 0x7f802c15c390 2024-03-28T13:21:40.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.920+0000 7f8030c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0af830 0x7f802c15c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:40.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.920+0000 7f8030c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0af830 0x7f802c15c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54256/0 (socket says 172.21.15.19:54256) 2024-03-28T13:21:40.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.920+0000 7f8030c00640 1 -- 172.21.15.19:0/3322637941 learned_addr learned my addr 172.21.15.19:0/3322637941 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:40.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.920+0000 7f802be00640 1 --2- 172.21.15.19:0/3322637941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f802c15c390 0x7f802c14cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:40.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.920+0000 7f8030c00640 1 -- 172.21.15.19:0/3322637941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f802c15c390 msgr2=0x7f802c14cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:40.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.920+0000 7f8030c00640 1 --2- 172.21.15.19:0/3322637941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f802c15c390 0x7f802c14cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.920+0000 7f8031600640 1 --2- 172.21.15.19:0/3322637941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f802c160b80 0x7f802c0b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.921+0000 7f8030c00640 1 -- 172.21.15.19:0/3322637941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f802c160b80 msgr2=0x7f802c0b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.921+0000 7f8030c00640 1 --2- 172.21.15.19:0/3322637941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f802c160b80 0x7f802c0b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.921+0000 7f8030c00640 1 -- 172.21.15.19:0/3322637941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802c15d400 con 0x7f802c0af830 2024-03-28T13:21:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.921+0000 7f8031600640 1 --2- 172.21.15.19:0/3322637941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f802c160b80 0x7f802c0b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:40.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.921+0000 7f8030c00640 1 --2- 172.21.15.19:0/3322637941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0af830 0x7f802c15c900 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f801c036040 tx=0x7f801c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.921+0000 7f8028c00640 1 -- 172.21.15.19:0/3322637941 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c043020 con 0x7f802c0af830 2024-03-28T13:21:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.921+0000 7f8028c00640 1 -- 172.21.15.19:0/3322637941 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f801c03e450 con 0x7f802c0af830 2024-03-28T13:21:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.921+0000 7f8028c00640 1 -- 172.21.15.19:0/3322637941 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801c03e840 con 0x7f802c0af830 2024-03-28T13:21:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.921+0000 7f8033600640 1 -- 172.21.15.19:0/3322637941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802c159780 con 0x7f802c0af830 2024-03-28T13:21:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.921+0000 7f8033600640 1 -- 172.21.15.19:0/3322637941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f802c159d40 con 0x7f802c0af830 2024-03-28T13:21:40.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.923+0000 7f8033600640 1 -- 172.21.15.19:0/3322637941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f802c159570 con 0x7f802c0af830 2024-03-28T13:21:40.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.923+0000 7f8028c00640 1 -- 172.21.15.19:0/3322637941 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f801c046020 con 0x7f802c0af830 2024-03-28T13:21:40.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.925+0000 7f8028c00640 1 --2- 172.21.15.19:0/3322637941 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f800c06c530 0x7f800c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:40.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.925+0000 7f8028c00640 1 -- 172.21.15.19:0/3322637941 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f801c0ae1f0 con 0x7f802c0af830 2024-03-28T13:21:40.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.925+0000 7f802be00640 1 --2- 172.21.15.19:0/3322637941 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f800c06c530 0x7f800c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:40.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.927+0000 7f802be00640 1 --2- 172.21.15.19:0/3322637941 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f800c06c530 0x7f800c06e9f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f802c1f5d20 tx=0x7f8018008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:40.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:40.930+0000 7f8028c00640 1 -- 172.21.15.19:0/3322637941 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f801c04d030 con 0x7f802c0af830 2024-03-28T13:21:41.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.365+0000 7f8033600640 1 -- 172.21.15.19:0/3322637941 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f802c072d60 con 0x7f802c0af830 2024-03-28T13:21:41.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.366+0000 7f8028c00640 1 -- 172.21.15.19:0/3322637941 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5186 (secure 0 0 0) 0x7f801c07c720 con 0x7f802c0af830 2024-03-28T13:21:41.366 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:41.366 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3823671990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3823671990},{"type":"v1","addr":"172.21.15.188:6837","nonce":3823671990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/22158043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":22158043},{"type":"v1","addr":"172.21.15.188:6836","nonce":22158043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":8688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/230519675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":230519675},{"type":"v1","addr":"172.21.15.19:6834","nonce":230519675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:20:52.901269+0000","modified":"2024-03-28T13:21:31.965724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/1527816903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1527816903},{"type":"v1","addr":"172.21.15.19:6835","nonce":1527816903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9123,"qdb_cluster":[9123]},"id":8}]} 2024-03-28T13:21:41.366 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 112 2024-03-28T13:21:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.369+0000 7f8033600640 1 -- 172.21.15.19:0/3322637941 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f800c06c530 msgr2=0x7f800c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.369+0000 7f8033600640 1 --2- 172.21.15.19:0/3322637941 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f800c06c530 0x7f800c06e9f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f802c1f5d20 tx=0x7f8018008040 comp rx=0 tx=0).stop 2024-03-28T13:21:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.369+0000 7f8033600640 1 -- 172.21.15.19:0/3322637941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0af830 msgr2=0x7f802c15c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.369+0000 7f8033600640 1 --2- 172.21.15.19:0/3322637941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0af830 0x7f802c15c900 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f801c036040 tx=0x7f801c004a40 comp rx=0 tx=0).stop 2024-03-28T13:21:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.369+0000 7f8033600640 1 -- 172.21.15.19:0/3322637941 shutdown_connections 2024-03-28T13:21:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.369+0000 7f8033600640 1 --2- 172.21.15.19:0/3322637941 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f800c06c530 0x7f800c06e9f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.369+0000 7f8033600640 1 --2- 172.21.15.19:0/3322637941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f802c160b80 0x7f802c0b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.369+0000 7f8033600640 1 --2- 172.21.15.19:0/3322637941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f802c15c390 0x7f802c14cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.369+0000 7f8033600640 1 --2- 172.21.15.19:0/3322637941 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f802c0af830 0x7f802c15c900 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.369+0000 7f8033600640 1 -- 172.21.15.19:0/3322637941 >> 172.21.15.19:0/3322637941 conn(0x7f802c06b990 msgr2=0x7f802c156e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:41.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.369+0000 7f8033600640 1 -- 172.21.15.19:0/3322637941 shutdown_connections 2024-03-28T13:21:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.369+0000 7f8033600640 1 -- 172.21.15.19:0/3322637941 wait complete. 2024-03-28T13:21:41.388 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:21:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.611+0000 7f191f000640 1 Processor -- start 2024-03-28T13:21:41.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.611+0000 7f191f000640 1 -- start start 2024-03-28T13:21:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.612+0000 7f191f000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f191815c390 0x7f1918073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.612+0000 7f191f000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f19180d0650 0x7f19180d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:41.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.612+0000 7f191f000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f191815fe80 con 0x7f19180af830 2024-03-28T13:21:41.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.613+0000 7f191f000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f19180d1000 con 0x7f19180d0650 2024-03-28T13:21:41.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.613+0000 7f191f000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f191815c770 con 0x7f191815c390 2024-03-28T13:21:41.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.613+0000 7f1917e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f191815c390 0x7f1918073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:41.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.613+0000 7f191d000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f19180d0650 0x7f19180d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:41.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.613+0000 7f1917e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f191815c390 0x7f1918073790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:55800/0 (socket says 172.21.15.19:55800) 2024-03-28T13:21:41.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.613+0000 7f1917e00640 1 -- 172.21.15.19:0/3073516538 learned_addr learned my addr 172.21.15.19:0/3073516538 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:41.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.614+0000 7f191d000640 1 -- 172.21.15.19:0/3073516538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f191815c390 msgr2=0x7f1918073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:41.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.614+0000 7f191d000640 1 --2- 172.21.15.19:0/3073516538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f191815c390 0x7f1918073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:41.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.614+0000 7f191d000640 1 -- 172.21.15.19:0/3073516538 >> v1:172.21.15.188:6789/0 conn(0x7f19180af830 legacy=0x7f1918160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:41.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.614+0000 7f191d000640 1 -- 172.21.15.19:0/3073516538 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19180ad120 con 0x7f19180d0650 2024-03-28T13:21:41.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.614+0000 7f1917e00640 1 --2- 172.21.15.19:0/3073516538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f191815c390 0x7f1918073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:21:41.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.616+0000 7f191d000640 1 --2- 172.21.15.19:0/3073516538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19180d0650 0x7f19180d0ac0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1910009d90 tx=0x7f191002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39aa8aad70439d1a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:41.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.616+0000 7f1917400640 1 -- 172.21.15.19:0/3073516538 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910039070 con 0x7f19180d0650 2024-03-28T13:21:41.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.616+0000 7f1917400640 1 -- 172.21.15.19:0/3073516538 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f191002cc60 con 0x7f19180d0650 2024-03-28T13:21:41.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.616+0000 7f1917400640 1 -- 172.21.15.19:0/3073516538 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910034660 con 0x7f19180d0650 2024-03-28T13:21:41.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.617+0000 7f191f000640 1 -- 172.21.15.19:0/3073516538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19180d0650 msgr2=0x7f19180d0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:41.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.617+0000 7f191f000640 1 --2- 172.21.15.19:0/3073516538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19180d0650 0x7f19180d0ac0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1910009d90 tx=0x7f191002beb0 comp rx=0 tx=0).stop 2024-03-28T13:21:41.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.617+0000 7f191f000640 1 -- 172.21.15.19:0/3073516538 shutdown_connections 2024-03-28T13:21:41.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.617+0000 7f191f000640 1 --2- 172.21.15.19:0/3073516538 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19180d0650 0x7f19180d0ac0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:41.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.617+0000 7f191f000640 1 --2- 172.21.15.19:0/3073516538 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f191815c390 0x7f1918073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:41.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.617+0000 7f191f000640 1 -- 172.21.15.19:0/3073516538 >> 172.21.15.19:0/3073516538 conn(0x7f191806b990 msgr2=0x7f191806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:41.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.617+0000 7f191f000640 1 -- 172.21.15.19:0/3073516538 shutdown_connections 2024-03-28T13:21:41.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.617+0000 7f191f000640 1 -- 172.21.15.19:0/3073516538 wait complete. 2024-03-28T13:21:41.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.619+0000 7f191f000640 1 Processor -- start 2024-03-28T13:21:41.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.619+0000 7f191f000640 1 -- start start 2024-03-28T13:21:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.620+0000 7f191f000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19180af830 0x7f1918156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.620+0000 7f191f000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19180d0650 0x7f19180b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.620+0000 7f191f000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191815c390 0x7f191814e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.620+0000 7f191f000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1918160120 con 0x7f191815c390 2024-03-28T13:21:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.620+0000 7f191f000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f19180ad6e0 con 0x7f19180d0650 2024-03-28T13:21:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.620+0000 7f191f000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1918160d50 con 0x7f19180af830 2024-03-28T13:21:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.621+0000 7f191d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191815c390 0x7f191814e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.621+0000 7f191d000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191815c390 0x7f191814e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54258/0 (socket says 172.21.15.19:54258) 2024-03-28T13:21:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.621+0000 7f191d000640 1 -- 172.21.15.19:0/652658358 learned_addr learned my addr 172.21.15.19:0/652658358 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.621+0000 7f1917e00640 1 --2- 172.21.15.19:0/652658358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19180af830 0x7f1918156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.621+0000 7f190fe00640 1 --2- 172.21.15.19:0/652658358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19180d0650 0x7f19180b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.621+0000 7f191d000640 1 -- 172.21.15.19:0/652658358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19180af830 msgr2=0x7f1918156e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.621+0000 7f191d000640 1 --2- 172.21.15.19:0/652658358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19180af830 0x7f1918156e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:41.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.621+0000 7f191d000640 1 -- 172.21.15.19:0/652658358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19180d0650 msgr2=0x7f19180b2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:41.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.621+0000 7f191d000640 1 --2- 172.21.15.19:0/652658358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19180d0650 0x7f19180b2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:41.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.621+0000 7f191d000640 1 -- 172.21.15.19:0/652658358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1918157350 con 0x7f191815c390 2024-03-28T13:21:41.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.622+0000 7f191d000640 1 --2- 172.21.15.19:0/652658358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191815c390 0x7f191814e420 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f1910009eb0 tx=0x7f1910004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:41.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.622+0000 7f1914c00640 1 -- 172.21.15.19:0/652658358 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910044020 con 0x7f191815c390 2024-03-28T13:21:41.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.622+0000 7f1914c00640 1 -- 172.21.15.19:0/652658358 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f191003f470 con 0x7f191815c390 2024-03-28T13:21:41.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.622+0000 7f191f000640 1 -- 172.21.15.19:0/652658358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19181575f0 con 0x7f191815c390 2024-03-28T13:21:41.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.622+0000 7f1914c00640 1 -- 172.21.15.19:0/652658358 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910039040 con 0x7f191815c390 2024-03-28T13:21:41.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.622+0000 7f191f000640 1 -- 172.21.15.19:0/652658358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1918157bb0 con 0x7f191815c390 2024-03-28T13:21:41.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.623+0000 7f191f000640 1 -- 172.21.15.19:0/652658358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19180685c0 con 0x7f191815c390 2024-03-28T13:21:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.624+0000 7f1914c00640 1 -- 172.21.15.19:0/652658358 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1910047020 con 0x7f191815c390 2024-03-28T13:21:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.626+0000 7f1914c00640 1 --2- 172.21.15.19:0/652658358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f18f406c580 0x7f18f406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.626+0000 7f1914c00640 1 -- 172.21.15.19:0/652658358 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f19100ae2d0 con 0x7f191815c390 2024-03-28T13:21:41.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.627+0000 7f1917e00640 1 --2- 172.21.15.19:0/652658358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f18f406c580 0x7f18f406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:41.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.628+0000 7f1917e00640 1 --2- 172.21.15.19:0/652658358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f18f406c580 0x7f18f406ea40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1908009b70 tx=0x7f1908008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:41.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.631+0000 7f1914c00640 1 -- 172.21.15.19:0/652658358 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f191004e030 con 0x7f191815c390 2024-03-28T13:21:41.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.836+0000 7f191f000640 1 -- 172.21.15.19:0/652658358 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f191806f1b0 con 0x7f191815c390 2024-03-28T13:21:41.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.837+0000 7f1914c00640 1 -- 172.21.15.19:0/652658358 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+17028 (secure 0 0 0) 0x7f191007c7e0 con 0x7f191815c390 2024-03-28T13:21:41.837 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:41.838 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":123,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:21:26.921359+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:20:50.748212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-28T13:20:51.888444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":3}]},{"pgid":"17.b","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3930967142":"2024-03-29T13:21:26.921326+0000","172.21.15.19:6833/3930967142":"2024-03-29T13:21:26.921326+0000","172.21.15.188:6836/2088538359":"2024-03-29T13:21:12.469733+0000","172.21.15.188:6834/2088538359":"2024-03-29T13:21:12.469733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:21:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.840+0000 7f191f000640 1 -- 172.21.15.19:0/652658358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f18f406c580 msgr2=0x7f18f406ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.840+0000 7f191f000640 1 --2- 172.21.15.19:0/652658358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f18f406c580 0x7f18f406ea40 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1908009b70 tx=0x7f1908008040 comp rx=0 tx=0).stop 2024-03-28T13:21:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.840+0000 7f191f000640 1 -- 172.21.15.19:0/652658358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191815c390 msgr2=0x7f191814e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.840+0000 7f191f000640 1 --2- 172.21.15.19:0/652658358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191815c390 0x7f191814e420 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f1910009eb0 tx=0x7f1910004a40 comp rx=0 tx=0).stop 2024-03-28T13:21:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.840+0000 7f191f000640 1 -- 172.21.15.19:0/652658358 shutdown_connections 2024-03-28T13:21:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.840+0000 7f191f000640 1 --2- 172.21.15.19:0/652658358 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f18f406c580 0x7f18f406ea40 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.840+0000 7f191f000640 1 --2- 172.21.15.19:0/652658358 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f191815c390 0x7f191814e420 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.840+0000 7f191f000640 1 --2- 172.21.15.19:0/652658358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19180d0650 0x7f19180b2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.840+0000 7f191f000640 1 --2- 172.21.15.19:0/652658358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19180af830 0x7f1918156e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.840+0000 7f191f000640 1 -- 172.21.15.19:0/652658358 >> 172.21.15.19:0/652658358 conn(0x7f191806b990 msgr2=0x7f19180720d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.840+0000 7f191f000640 1 -- 172.21.15.19:0/652658358 shutdown_connections 2024-03-28T13:21:41.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:41.840+0000 7f191f000640 1 -- 172.21.15.19:0/652658358 wait complete. 2024-03-28T13:21:41.859 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-28T13:09:30.014312+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-28T13:20:50.748212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-28T13:20:51.888444+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-28T13:21:41.859 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-28T13:21:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.084+0000 7fa861400640 1 Processor -- start 2024-03-28T13:21:42.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.084+0000 7fa861400640 1 -- start start 2024-03-28T13:21:42.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.086+0000 7fa861400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa85c0d3e40 0x7fa85c0b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:42.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.086+0000 7fa861400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa85c0cef20 0x7fa85c0cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:42.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.086+0000 7fa861400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa85c0d2da0 con 0x7fa85c15e460 2024-03-28T13:21:42.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.086+0000 7fa861400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa85c0cf8d0 con 0x7fa85c0d3e40 2024-03-28T13:21:42.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.086+0000 7fa861400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa85c0b8320 con 0x7fa85c0cef20 2024-03-28T13:21:42.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.086+0000 7fa853e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa85c0d3e40 0x7fa85c0b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.086+0000 7fa85be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa85c15e460 0x7fa85c0d2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34158/0 (socket says 172.21.15.19:34158) 2024-03-28T13:21:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.086+0000 7fa85be00640 1 -- 172.21.15.19:0/3279062606 learned_addr learned my addr 172.21.15.19:0/3279062606 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.087+0000 7fa860a00640 1 --2- 172.21.15.19:0/3279062606 >> v2:172.21.15.188:3300/0 conn(0x7fa85c0cef20 0x7fa85c0cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.087+0000 7fa853e00640 1 -- 172.21.15.19:0/3279062606 >> v2:172.21.15.188:3300/0 conn(0x7fa85c0cef20 msgr2=0x7fa85c0cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.087+0000 7fa853e00640 1 --2- 172.21.15.19:0/3279062606 >> v2:172.21.15.188:3300/0 conn(0x7fa85c0cef20 0x7fa85c0cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.087+0000 7fa853e00640 1 -- 172.21.15.19:0/3279062606 >> v1:172.21.15.19:6789/0 conn(0x7fa85c15e460 legacy=0x7fa85c0d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.087+0000 7fa85b400640 1 -- 172.21.15.19:0/3279062606 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2795280143 0 0) 0x7fa85c0d2da0 con 0x7fa85c15e460 2024-03-28T13:21:42.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.087+0000 7fa853e00640 1 -- 172.21.15.19:0/3279062606 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c0d3080 con 0x7fa85c0d3e40 2024-03-28T13:21:42.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.090+0000 7fa853e00640 1 --2- 172.21.15.19:0/3279062606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa85c0d3e40 0x7fa85c0b9e20 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0d0660 tx=0x7fa84402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1ee101fa144fa server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.090+0000 7fa85b400640 1 -- 172.21.15.19:0/3279062606 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844035020 con 0x7fa85c0d3e40 2024-03-28T13:21:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.090+0000 7fa85b400640 1 -- 172.21.15.19:0/3279062606 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa844031830 con 0x7fa85c0d3e40 2024-03-28T13:21:42.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.090+0000 7fa85b400640 1 -- 172.21.15.19:0/3279062606 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa844031c00 con 0x7fa85c0d3e40 2024-03-28T13:21:42.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.091+0000 7fa861400640 1 -- 172.21.15.19:0/3279062606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa85c0d3e40 msgr2=0x7fa85c0b9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:42.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.091+0000 7fa861400640 1 --2- 172.21.15.19:0/3279062606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa85c0d3e40 0x7fa85c0b9e20 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0d0660 tx=0x7fa84402b8d0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.091+0000 7fa861400640 1 -- 172.21.15.19:0/3279062606 shutdown_connections 2024-03-28T13:21:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.091+0000 7fa861400640 1 --2- 172.21.15.19:0/3279062606 >> v2:172.21.15.188:3300/0 conn(0x7fa85c0cef20 0x7fa85c0cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.091+0000 7fa861400640 1 --2- 172.21.15.19:0/3279062606 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa85c0d3e40 0x7fa85c0b9e20 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.091+0000 7fa861400640 1 -- 172.21.15.19:0/3279062606 >> 172.21.15.19:0/3279062606 conn(0x7fa85c0678b0 msgr2=0x7fa85c0d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.091+0000 7fa861400640 1 -- 172.21.15.19:0/3279062606 shutdown_connections 2024-03-28T13:21:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.091+0000 7fa861400640 1 -- 172.21.15.19:0/3279062606 wait complete. 2024-03-28T13:21:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.093+0000 7fa861400640 1 Processor -- start 2024-03-28T13:21:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.093+0000 7fa861400640 1 -- start start 2024-03-28T13:21:42.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.094+0000 7fa861400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa85c0cef20 0x7fa85c152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:42.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.094+0000 7fa861400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa85c0d3e40 0x7fa85c0cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:42.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.094+0000 7fa861400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85c15e460 0x7fa85c0cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:42.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.094+0000 7fa861400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c0d3920 con 0x7fa85c0cef20 2024-03-28T13:21:42.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.094+0000 7fa861400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c0d3640 con 0x7fa85c15e460 2024-03-28T13:21:42.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.094+0000 7fa861400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa85c0b7310 con 0x7fa85c0d3e40 2024-03-28T13:21:42.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.094+0000 7fa85be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa85c0cef20 0x7fa85c152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:42.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.094+0000 7fa85be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa85c0cef20 0x7fa85c152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54282/0 (socket says 172.21.15.19:54282) 2024-03-28T13:21:42.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.094+0000 7fa85be00640 1 -- 172.21.15.19:0/3372054991 learned_addr learned my addr 172.21.15.19:0/3372054991 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:42.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.094+0000 7fa85be00640 1 -- 172.21.15.19:0/3372054991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa85c0d3e40 msgr2=0x7fa85c0cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:42.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.094+0000 7fa860a00640 1 --2- 172.21.15.19:0/3372054991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85c15e460 0x7fa85c0cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:42.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.094+0000 7fa85be00640 1 --2- 172.21.15.19:0/3372054991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa85c0d3e40 0x7fa85c0cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.095+0000 7fa85be00640 1 -- 172.21.15.19:0/3372054991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85c15e460 msgr2=0x7fa85c0cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:42.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.095+0000 7fa85be00640 1 --2- 172.21.15.19:0/3372054991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85c15e460 0x7fa85c0cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.095+0000 7fa85be00640 1 -- 172.21.15.19:0/3372054991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c0ccc20 con 0x7fa85c0cef20 2024-03-28T13:21:42.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.095+0000 7fa85be00640 1 --2- 172.21.15.19:0/3372054991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa85c0cef20 0x7fa85c152740 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00c430 tx=0x7fa84c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:42.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.095+0000 7fa858c00640 1 -- 172.21.15.19:0/3372054991 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c0074e0 con 0x7fa85c0cef20 2024-03-28T13:21:42.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.095+0000 7fa858c00640 1 -- 172.21.15.19:0/3372054991 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84c0047e0 con 0x7fa85c0cef20 2024-03-28T13:21:42.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.095+0000 7fa858c00640 1 -- 172.21.15.19:0/3372054991 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c004bd0 con 0x7fa85c0cef20 2024-03-28T13:21:42.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.095+0000 7fa861400640 1 -- 172.21.15.19:0/3372054991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85c15ea50 con 0x7fa85c0cef20 2024-03-28T13:21:42.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.095+0000 7fa861400640 1 -- 172.21.15.19:0/3372054991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85c0cbec0 con 0x7fa85c0cef20 2024-03-28T13:21:42.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.097+0000 7fa861400640 1 -- 172.21.15.19:0/3372054991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85c15e840 con 0x7fa85c0cef20 2024-03-28T13:21:42.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.097+0000 7fa858c00640 1 -- 172.21.15.19:0/3372054991 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa84c011070 con 0x7fa85c0cef20 2024-03-28T13:21:42.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.099+0000 7fa858c00640 1 --2- 172.21.15.19:0/3372054991 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa83806c530 0x7fa83806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:42.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.099+0000 7fa858c00640 1 -- 172.21.15.19:0/3372054991 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7fa84c08b4e0 con 0x7fa85c0cef20 2024-03-28T13:21:42.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.099+0000 7fa853e00640 1 --2- 172.21.15.19:0/3372054991 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa83806c530 0x7fa83806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:42.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.101+0000 7fa853e00640 1 --2- 172.21.15.19:0/3372054991 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa83806c530 0x7fa83806e9f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa844036000 tx=0x7fa844030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.104+0000 7fa858c00640 1 -- 172.21.15.19:0/3372054991 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa84c059970 con 0x7fa85c0cef20 2024-03-28T13:21:42.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.305+0000 7fa861400640 1 -- 172.21.15.19:0/3372054991 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa85c075940 con 0x7fa85c0cef20 2024-03-28T13:21:42.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.306+0000 7fa858c00640 1 -- 172.21.15.19:0/3372054991 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) ==== 93+0+10 (secure 0 0 0) 0x7fa84c059270 con 0x7fa85c0cef20 2024-03-28T13:21:42.306 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-03-28T13:21:42.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.309+0000 7fa861400640 1 -- 172.21.15.19:0/3372054991 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa83806c530 msgr2=0x7fa83806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:42.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.309+0000 7fa861400640 1 --2- 172.21.15.19:0/3372054991 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa83806c530 0x7fa83806e9f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa844036000 tx=0x7fa844030000 comp rx=0 tx=0).stop 2024-03-28T13:21:42.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.309+0000 7fa861400640 1 -- 172.21.15.19:0/3372054991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa85c0cef20 msgr2=0x7fa85c152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:42.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.309+0000 7fa861400640 1 --2- 172.21.15.19:0/3372054991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa85c0cef20 0x7fa85c152740 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00c430 tx=0x7fa84c00c8f0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.309+0000 7fa861400640 1 -- 172.21.15.19:0/3372054991 shutdown_connections 2024-03-28T13:21:42.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.309+0000 7fa861400640 1 --2- 172.21.15.19:0/3372054991 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa83806c530 0x7fa83806e9f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.309+0000 7fa861400640 1 --2- 172.21.15.19:0/3372054991 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85c15e460 0x7fa85c0cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.310+0000 7fa861400640 1 --2- 172.21.15.19:0/3372054991 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa85c0d3e40 0x7fa85c0cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.310+0000 7fa861400640 1 --2- 172.21.15.19:0/3372054991 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa85c0cef20 0x7fa85c152740 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.310+0000 7fa861400640 1 -- 172.21.15.19:0/3372054991 >> 172.21.15.19:0/3372054991 conn(0x7fa85c0678b0 msgr2=0x7fa85c0b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:42.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.310+0000 7fa861400640 1 -- 172.21.15.19:0/3372054991 shutdown_connections 2024-03-28T13:21:42.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.310+0000 7fa861400640 1 -- 172.21.15.19:0/3372054991 wait complete. 2024-03-28T13:21:42.335 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-28T13:21:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.613+0000 7fb3b9600640 1 Processor -- start 2024-03-28T13:21:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.614+0000 7fb3b9600640 1 -- start start 2024-03-28T13:21:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.615+0000 7fb3b9600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb3b415e4c0 0x7fb3b40b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.616+0000 7fb3b9600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb3b40d37b0 0x7fb3b40b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.616+0000 7fb3b9600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3b40d2710 con 0x7fb3b40b67f0 2024-03-28T13:21:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.616+0000 7fb3b9600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b415e8a0 con 0x7fb3b415e4c0 2024-03-28T13:21:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.616+0000 7fb3b9600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb3b40d4040 con 0x7fb3b40d37b0 2024-03-28T13:21:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.616+0000 7fb3b3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb3b415e4c0 0x7fb3b40b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.616+0000 7fb3b3e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fb3b415e4c0 0x7fb3b40b9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54288/0 (socket says 172.21.15.19:54288) 2024-03-28T13:21:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.616+0000 7fb3b3e00640 1 -- 172.21.15.19:0/3921854109 learned_addr learned my addr 172.21.15.19:0/3921854109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.616+0000 7fb3b3400640 1 --2- 172.21.15.19:0/3921854109 >> v2:172.21.15.188:3300/0 conn(0x7fb3b40d37b0 0x7fb3b40b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.617+0000 7fb3b3e00640 1 -- 172.21.15.19:0/3921854109 >> v2:172.21.15.188:3300/0 conn(0x7fb3b40d37b0 msgr2=0x7fb3b40b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.617+0000 7fb3b3e00640 1 --2- 172.21.15.19:0/3921854109 >> v2:172.21.15.188:3300/0 conn(0x7fb3b40d37b0 0x7fb3b40b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.617+0000 7fb3b3e00640 1 -- 172.21.15.19:0/3921854109 >> v1:172.21.15.188:6789/0 conn(0x7fb3b40b67f0 legacy=0x7fb3b40d4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.617+0000 7fb3b3e00640 1 -- 172.21.15.19:0/3921854109 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b40d29f0 con 0x7fb3b415e4c0 2024-03-28T13:21:42.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.617+0000 7fb3b3400640 1 --2- 172.21.15.19:0/3921854109 >> v2:172.21.15.188:3300/0 conn(0x7fb3b40d37b0 0x7fb3b40b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:21:42.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.619+0000 7fb3b3e00640 1 --2- 172.21.15.19:0/3921854109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b415e4c0 0x7fb3b40b9bf0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40b99b0 tx=0x7fb3a402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140714b569a53329 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:42.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.619+0000 7fb3b2a00640 1 -- 172.21.15.19:0/3921854109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a402cb20 con 0x7fb3b415e4c0 2024-03-28T13:21:42.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.619+0000 7fb3b2a00640 1 -- 172.21.15.19:0/3921854109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a402ccc0 con 0x7fb3b415e4c0 2024-03-28T13:21:42.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.619+0000 7fb3b2a00640 1 -- 172.21.15.19:0/3921854109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4034610 con 0x7fb3b415e4c0 2024-03-28T13:21:42.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.620+0000 7fb3b9600640 1 -- 172.21.15.19:0/3921854109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b415e4c0 msgr2=0x7fb3b40b9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:42.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.620+0000 7fb3b9600640 1 --2- 172.21.15.19:0/3921854109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b415e4c0 0x7fb3b40b9bf0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fb3b40b99b0 tx=0x7fb3a402bf80 comp rx=0 tx=0).stop 2024-03-28T13:21:42.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.620+0000 7fb3b9600640 1 -- 172.21.15.19:0/3921854109 shutdown_connections 2024-03-28T13:21:42.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.620+0000 7fb3b9600640 1 --2- 172.21.15.19:0/3921854109 >> v2:172.21.15.188:3300/0 conn(0x7fb3b40d37b0 0x7fb3b40b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.620+0000 7fb3b9600640 1 --2- 172.21.15.19:0/3921854109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b415e4c0 0x7fb3b40b9bf0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.620+0000 7fb3b9600640 1 -- 172.21.15.19:0/3921854109 >> 172.21.15.19:0/3921854109 conn(0x7fb3b40676a0 msgr2=0x7fb3b4067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:42.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.620+0000 7fb3b9600640 1 -- 172.21.15.19:0/3921854109 shutdown_connections 2024-03-28T13:21:42.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.620+0000 7fb3b9600640 1 -- 172.21.15.19:0/3921854109 wait complete. 2024-03-28T13:21:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.622+0000 7fb3b9600640 1 Processor -- start 2024-03-28T13:21:42.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.622+0000 7fb3b9600640 1 -- start start 2024-03-28T13:21:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.623+0000 7fb3b9600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b40b67f0 0x7fb3b4154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.623+0000 7fb3b9600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b40d37b0 0x7fb3b41f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.623+0000 7fb3b9600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3b415e4c0 0x7fb3b40c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.623+0000 7fb3b9600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b40b6c20 con 0x7fb3b40d37b0 2024-03-28T13:21:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.623+0000 7fb3b9600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b40d3320 con 0x7fb3b40b67f0 2024-03-28T13:21:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.623+0000 7fb3b9600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb3b40d2fb0 con 0x7fb3b415e4c0 2024-03-28T13:21:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.623+0000 7fb3b3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b40d37b0 0x7fb3b41f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.623+0000 7fb3b3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b40d37b0 0x7fb3b41f73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54296/0 (socket says 172.21.15.19:54296) 2024-03-28T13:21:42.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.623+0000 7fb3b3400640 1 -- 172.21.15.19:0/3313942673 learned_addr learned my addr 172.21.15.19:0/3313942673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.624+0000 7fb3b3e00640 1 --2- 172.21.15.19:0/3313942673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b40b67f0 0x7fb3b4154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.624+0000 7fb3b8c00640 1 --2- 172.21.15.19:0/3313942673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3b415e4c0 0x7fb3b40c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.624+0000 7fb3b3400640 1 -- 172.21.15.19:0/3313942673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3b415e4c0 msgr2=0x7fb3b40c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.624+0000 7fb3b3400640 1 --2- 172.21.15.19:0/3313942673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3b415e4c0 0x7fb3b40c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.624+0000 7fb3b3400640 1 -- 172.21.15.19:0/3313942673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b40b67f0 msgr2=0x7fb3b4154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.624+0000 7fb3b3400640 1 --2- 172.21.15.19:0/3313942673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b40b67f0 0x7fb3b4154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.624+0000 7fb3b3400640 1 -- 172.21.15.19:0/3313942673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b41f7dd0 con 0x7fb3b40d37b0 2024-03-28T13:21:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.624+0000 7fb3b3e00640 1 --2- 172.21.15.19:0/3313942673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b40b67f0 0x7fb3b4154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:42.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.624+0000 7fb3b3400640 1 --2- 172.21.15.19:0/3313942673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b40d37b0 0x7fb3b41f73f0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fb39c00e730 tx=0x7fb39c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:42.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.625+0000 7fb3abe00640 1 -- 172.21.15.19:0/3313942673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c00f040 con 0x7fb3b40d37b0 2024-03-28T13:21:42.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.625+0000 7fb3abe00640 1 -- 172.21.15.19:0/3313942673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb39c004230 con 0x7fb3b40d37b0 2024-03-28T13:21:42.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.625+0000 7fb3abe00640 1 -- 172.21.15.19:0/3313942673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb39c00ee40 con 0x7fb3b40d37b0 2024-03-28T13:21:42.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.625+0000 7fb3b9600640 1 -- 172.21.15.19:0/3313942673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b41f80b0 con 0x7fb3b40d37b0 2024-03-28T13:21:42.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.625+0000 7fb3b9600640 1 -- 172.21.15.19:0/3313942673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b41570a0 con 0x7fb3b40d37b0 2024-03-28T13:21:42.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.626+0000 7fb3b9600640 1 -- 172.21.15.19:0/3313942673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b4075940 con 0x7fb3b40d37b0 2024-03-28T13:21:42.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.627+0000 7fb3abe00640 1 -- 172.21.15.19:0/3313942673 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fb39c00c6d0 con 0x7fb3b40d37b0 2024-03-28T13:21:42.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.628+0000 7fb3abe00640 1 --2- 172.21.15.19:0/3313942673 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb39006c580 0x7fb39006ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:42.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.628+0000 7fb3abe00640 1 -- 172.21.15.19:0/3313942673 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7fb39c08b570 con 0x7fb3b40d37b0 2024-03-28T13:21:42.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.629+0000 7fb3b3e00640 1 --2- 172.21.15.19:0/3313942673 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb39006c580 0x7fb39006ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:42.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.630+0000 7fb3b3e00640 1 --2- 172.21.15.19:0/3313942673 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb39006c580 0x7fb39006ea40 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4035000 tx=0x7fb3a4030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:42.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.632+0000 7fb3abe00640 1 -- 172.21.15.19:0/3313942673 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fb39c017020 con 0x7fb3b40d37b0 2024-03-28T13:21:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.834+0000 7fb3b9600640 1 -- 172.21.15.19:0/3313942673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb3b40cf9b0 con 0x7fb3b40d37b0 2024-03-28T13:21:42.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.835+0000 7fb3abe00640 1 -- 172.21.15.19:0/3313942673 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) ==== 104+0+11 (secure 0 0 0) 0x7fb39c059a80 con 0x7fb3b40d37b0 2024-03-28T13:21:42.834 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-03-28T13:21:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.837+0000 7fb3b9600640 1 -- 172.21.15.19:0/3313942673 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb39006c580 msgr2=0x7fb39006ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.837+0000 7fb3b9600640 1 --2- 172.21.15.19:0/3313942673 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb39006c580 0x7fb39006ea40 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4035000 tx=0x7fb3a4030000 comp rx=0 tx=0).stop 2024-03-28T13:21:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.837+0000 7fb3b9600640 1 -- 172.21.15.19:0/3313942673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b40d37b0 msgr2=0x7fb3b41f73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.837+0000 7fb3b9600640 1 --2- 172.21.15.19:0/3313942673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b40d37b0 0x7fb3b41f73f0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fb39c00e730 tx=0x7fb39c00ebf0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.837+0000 7fb3b9600640 1 -- 172.21.15.19:0/3313942673 shutdown_connections 2024-03-28T13:21:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.837+0000 7fb3b9600640 1 --2- 172.21.15.19:0/3313942673 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fb39006c580 0x7fb39006ea40 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.837+0000 7fb3b9600640 1 --2- 172.21.15.19:0/3313942673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3b415e4c0 0x7fb3b40c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.837+0000 7fb3b9600640 1 --2- 172.21.15.19:0/3313942673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb3b40d37b0 0x7fb3b41f73f0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.837+0000 7fb3b9600640 1 --2- 172.21.15.19:0/3313942673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3b40b67f0 0x7fb3b4154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.837+0000 7fb3b9600640 1 -- 172.21.15.19:0/3313942673 >> 172.21.15.19:0/3313942673 conn(0x7fb3b40676a0 msgr2=0x7fb3b40c7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:42.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.837+0000 7fb3b9600640 1 -- 172.21.15.19:0/3313942673 shutdown_connections 2024-03-28T13:21:42.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:42.837+0000 7fb3b9600640 1 -- 172.21.15.19:0/3313942673 wait complete. 2024-03-28T13:21:42.856 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-28T13:21:43.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.069+0000 7fc86e200640 1 Processor -- start 2024-03-28T13:21:43.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.070+0000 7fc86e200640 1 -- start start 2024-03-28T13:21:43.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.071+0000 7fc86e200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc86815e460 0x7fc8680b9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:43.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.071+0000 7fc86e200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680d3e40 0x7fc8680d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:43.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.071+0000 7fc86e200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8680b8320 con 0x7fc8680cef20 2024-03-28T13:21:43.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.071+0000 7fc86e200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc8680d2da0 con 0x7fc86815e460 2024-03-28T13:21:43.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.071+0000 7fc86e200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc8680d3080 con 0x7fc8680d3e40 2024-03-28T13:21:43.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.072+0000 7fc867e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680d3e40 0x7fc8680d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:43.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.072+0000 7fc86ce00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc86815e460 0x7fc8680b9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:43.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.072+0000 7fc86ce00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc86815e460 0x7fc8680b9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:51602/0 (socket says 172.21.15.19:51602) 2024-03-28T13:21:43.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.072+0000 7fc867e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680d3e40 0x7fc8680d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:55830/0 (socket says 172.21.15.19:55830) 2024-03-28T13:21:43.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.072+0000 7fc86ce00640 1 -- 172.21.15.19:0/2265913842 learned_addr learned my addr 172.21.15.19:0/2265913842 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:43.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.072+0000 7fc86d800640 1 --1- 172.21.15.19:0/2265913842 >> v1:172.21.15.188:6789/0 conn(0x7fc8680cef20 0x7fc86806c610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.19:59352/0 (socket says 172.21.15.19:59352) 2024-03-28T13:21:43.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.073+0000 7fc867e00640 1 -- 172.21.15.19:0/2265913842 >> v2:172.21.15.188:3300/0 conn(0x7fc86815e460 msgr2=0x7fc8680b9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:43.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.073+0000 7fc867e00640 1 --2- 172.21.15.19:0/2265913842 >> v2:172.21.15.188:3300/0 conn(0x7fc86815e460 0x7fc8680b9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.073+0000 7fc867e00640 1 -- 172.21.15.19:0/2265913842 >> v1:172.21.15.188:6789/0 conn(0x7fc8680cef20 legacy=0x7fc86806c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:43.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.073+0000 7fc867e00640 1 -- 172.21.15.19:0/2265913842 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8680d3360 con 0x7fc8680d3e40 2024-03-28T13:21:43.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.073+0000 7fc86ce00640 1 --2- 172.21.15.19:0/2265913842 >> v2:172.21.15.188:3300/0 conn(0x7fc86815e460 0x7fc8680b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:21:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.074+0000 7fc867e00640 1 --2- 172.21.15.19:0/2265913842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680d3e40 0x7fc8680d2550 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc854009790 tx=0x7fc85402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecae9672417797dd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.075+0000 7fc867400640 1 -- 172.21.15.19:0/2265913842 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854035020 con 0x7fc8680d3e40 2024-03-28T13:21:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.075+0000 7fc867400640 1 -- 172.21.15.19:0/2265913842 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc854031800 con 0x7fc8680d3e40 2024-03-28T13:21:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.075+0000 7fc867400640 1 -- 172.21.15.19:0/2265913842 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc854031bd0 con 0x7fc8680d3e40 2024-03-28T13:21:43.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.075+0000 7fc86e200640 1 -- 172.21.15.19:0/2265913842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680d3e40 msgr2=0x7fc8680d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.075+0000 7fc86e200640 1 --2- 172.21.15.19:0/2265913842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680d3e40 0x7fc8680d2550 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc854009790 tx=0x7fc85402b8a0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.076+0000 7fc86e200640 1 -- 172.21.15.19:0/2265913842 shutdown_connections 2024-03-28T13:21:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.076+0000 7fc86e200640 1 --2- 172.21.15.19:0/2265913842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680d3e40 0x7fc8680d2550 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.076+0000 7fc86e200640 1 --2- 172.21.15.19:0/2265913842 >> v2:172.21.15.188:3300/0 conn(0x7fc86815e460 0x7fc8680b9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.076+0000 7fc86e200640 1 -- 172.21.15.19:0/2265913842 >> 172.21.15.19:0/2265913842 conn(0x7fc8680678b0 msgr2=0x7fc8680d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.076+0000 7fc86e200640 1 -- 172.21.15.19:0/2265913842 shutdown_connections 2024-03-28T13:21:43.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.076+0000 7fc86e200640 1 -- 172.21.15.19:0/2265913842 wait complete. 2024-03-28T13:21:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.078+0000 7fc86e200640 1 Processor -- start 2024-03-28T13:21:43.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.078+0000 7fc86e200640 1 -- start start 2024-03-28T13:21:43.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.079+0000 7fc86e200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8680cef20 0x7fc8680d4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:43.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.079+0000 7fc86e200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680d3e40 0x7fc8680ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:43.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.079+0000 7fc86e200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86815e460 0x7fc8680ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:43.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.079+0000 7fc86e200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc8680d2a90 con 0x7fc8680cef20 2024-03-28T13:21:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.079+0000 7fc86e200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc8680d3920 con 0x7fc86815e460 2024-03-28T13:21:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.079+0000 7fc86e200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc8680cf370 con 0x7fc8680d3e40 2024-03-28T13:21:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.079+0000 7fc86ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8680cef20 0x7fc8680d4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.079+0000 7fc86ce00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8680cef20 0x7fc8680d4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54308/0 (socket says 172.21.15.19:54308) 2024-03-28T13:21:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.079+0000 7fc86ce00640 1 -- 172.21.15.19:0/251177599 learned_addr learned my addr 172.21.15.19:0/251177599 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.079+0000 7fc867e00640 1 --2- 172.21.15.19:0/251177599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680d3e40 0x7fc8680ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.079+0000 7fc86d800640 1 --2- 172.21.15.19:0/251177599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86815e460 0x7fc8680ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.080+0000 7fc86ce00640 1 -- 172.21.15.19:0/251177599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680d3e40 msgr2=0x7fc8680ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.080+0000 7fc86ce00640 1 --2- 172.21.15.19:0/251177599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680d3e40 0x7fc8680ca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.080+0000 7fc86ce00640 1 -- 172.21.15.19:0/251177599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86815e460 msgr2=0x7fc8680ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.080+0000 7fc86ce00640 1 --2- 172.21.15.19:0/251177599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86815e460 0x7fc8680ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.080+0000 7fc86ce00640 1 -- 172.21.15.19:0/251177599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8680d5110 con 0x7fc8680cef20 2024-03-28T13:21:43.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.080+0000 7fc86ce00640 1 --2- 172.21.15.19:0/251177599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8680cef20 0x7fc8680d4bd0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00e9b0 tx=0x7fc85c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:43.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.080+0000 7fc864c00640 1 -- 172.21.15.19:0/251177599 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c017070 con 0x7fc8680cef20 2024-03-28T13:21:43.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.080+0000 7fc864c00640 1 -- 172.21.15.19:0/251177599 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc85c00c9a0 con 0x7fc8680cef20 2024-03-28T13:21:43.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.080+0000 7fc864c00640 1 -- 172.21.15.19:0/251177599 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc85c00cd50 con 0x7fc8680cef20 2024-03-28T13:21:43.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.080+0000 7fc86e200640 1 -- 172.21.15.19:0/251177599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8681f8660 con 0x7fc8680cef20 2024-03-28T13:21:43.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.080+0000 7fc86e200640 1 -- 172.21.15.19:0/251177599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8681f48e0 con 0x7fc8680cef20 2024-03-28T13:21:43.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.082+0000 7fc86e200640 1 -- 172.21.15.19:0/251177599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8680d0110 con 0x7fc8680cef20 2024-03-28T13:21:43.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.084+0000 7fc864c00640 1 -- 172.21.15.19:0/251177599 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fc85c01a070 con 0x7fc8680cef20 2024-03-28T13:21:43.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.085+0000 7fc864c00640 1 --2- 172.21.15.19:0/251177599 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc84806c530 0x7fc84806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:43.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.085+0000 7fc864c00640 1 -- 172.21.15.19:0/251177599 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7fc85c08b690 con 0x7fc8680cef20 2024-03-28T13:21:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.085+0000 7fc867e00640 1 --2- 172.21.15.19:0/251177599 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc84806c530 0x7fc84806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:43.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.086+0000 7fc867e00640 1 --2- 172.21.15.19:0/251177599 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc84806c530 0x7fc84806e9f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc854036000 tx=0x7fc854008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:43.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.090+0000 7fc864c00640 1 -- 172.21.15.19:0/251177599 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fc8680d0110 con 0x7fc8680cef20 2024-03-28T13:21:43.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.290+0000 7fc86e200640 1 -- 172.21.15.19:0/251177599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc8680ce560 con 0x7fc8680cef20 2024-03-28T13:21:43.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.290+0000 7fc864c00640 1 -- 172.21.15.19:0/251177599 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) ==== 100+0+11 (secure 0 0 0) 0x7fc8680d5110 con 0x7fc8680cef20 2024-03-28T13:21:43.290 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-03-28T13:21:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.293+0000 7fc86e200640 1 -- 172.21.15.19:0/251177599 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc84806c530 msgr2=0x7fc84806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.293+0000 7fc86e200640 1 --2- 172.21.15.19:0/251177599 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc84806c530 0x7fc84806e9f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc854036000 tx=0x7fc854008030 comp rx=0 tx=0).stop 2024-03-28T13:21:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.293+0000 7fc86e200640 1 -- 172.21.15.19:0/251177599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8680cef20 msgr2=0x7fc8680d4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.293+0000 7fc86e200640 1 --2- 172.21.15.19:0/251177599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8680cef20 0x7fc8680d4bd0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00e9b0 tx=0x7fc85c00ee70 comp rx=0 tx=0).stop 2024-03-28T13:21:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.293+0000 7fc86e200640 1 -- 172.21.15.19:0/251177599 shutdown_connections 2024-03-28T13:21:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.293+0000 7fc86e200640 1 --2- 172.21.15.19:0/251177599 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fc84806c530 0x7fc84806e9f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.293+0000 7fc86e200640 1 --2- 172.21.15.19:0/251177599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86815e460 0x7fc8680ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.293+0000 7fc86e200640 1 --2- 172.21.15.19:0/251177599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc8680d3e40 0x7fc8680ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.293+0000 7fc86e200640 1 --2- 172.21.15.19:0/251177599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc8680cef20 0x7fc8680d4bd0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.293+0000 7fc86e200640 1 -- 172.21.15.19:0/251177599 >> 172.21.15.19:0/251177599 conn(0x7fc8680678b0 msgr2=0x7fc8680b58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.293+0000 7fc86e200640 1 -- 172.21.15.19:0/251177599 shutdown_connections 2024-03-28T13:21:43.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.293+0000 7fc86e200640 1 -- 172.21.15.19:0/251177599 wait complete. 2024-03-28T13:21:43.310 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:21:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.518+0000 7f1aca400640 1 Processor -- start 2024-03-28T13:21:43.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.518+0000 7f1aca400640 1 -- start start 2024-03-28T13:21:43.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.519+0000 7f1aca400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f1ac415c390 0x7f1ac4073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:43.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.519+0000 7f1aca400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac40af830 0x7f1ac4160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:43.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.519+0000 7f1aca400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ac415fe80 con 0x7f1ac4160b80 2024-03-28T13:21:43.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.519+0000 7f1aca400640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f1ac415c770 con 0x7f1ac415c390 2024-03-28T13:21:43.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.519+0000 7f1aca400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac40ad120 con 0x7f1ac40af830 2024-03-28T13:21:43.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.520+0000 7f1ac3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f1ac4160b80 0x7f1ac40d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34160/0 (socket says 172.21.15.19:34160) 2024-03-28T13:21:43.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.520+0000 7f1ac3e00640 1 -- 172.21.15.19:0/2675716465 learned_addr learned my addr 172.21.15.19:0/2675716465 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:43.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.520+0000 7f1ac3400640 1 --2- 172.21.15.19:0/2675716465 >> v2:172.21.15.19:3300/0 conn(0x7f1ac415c390 0x7f1ac4073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.520+0000 7f1ac2a00640 1 --2- 172.21.15.19:0/2675716465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac40af830 0x7f1ac4160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:43.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.520+0000 7f1ac3400640 1 -- 172.21.15.19:0/2675716465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac40af830 msgr2=0x7f1ac4160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:43.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.520+0000 7f1ac3400640 1 --2- 172.21.15.19:0/2675716465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac40af830 0x7f1ac4160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.520+0000 7f1ac3400640 1 -- 172.21.15.19:0/2675716465 >> v1:172.21.15.19:6789/0 conn(0x7f1ac4160b80 legacy=0x7f1ac40d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:43.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.520+0000 7f1ac3400640 1 -- 172.21.15.19:0/2675716465 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac40ad400 con 0x7f1ac415c390 2024-03-28T13:21:43.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.520+0000 7f1ac2000640 1 -- 172.21.15.19:0/2675716465 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2525733010 0 0) 0x7f1ac415fe80 con 0x7f1ac4160b80 2024-03-28T13:21:43.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.522+0000 7f1ac3400640 1 --2- 172.21.15.19:0/2675716465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac415c390 0x7f1ac4073790 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009d90 tx=0x7f1abc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6630c24c7daa65 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:43.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.522+0000 7f1ac2000640 1 -- 172.21.15.19:0/2675716465 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc039070 con 0x7f1ac415c390 2024-03-28T13:21:43.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.522+0000 7f1ac2000640 1 -- 172.21.15.19:0/2675716465 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1abc02cc50 con 0x7f1ac415c390 2024-03-28T13:21:43.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.522+0000 7f1ac2000640 1 -- 172.21.15.19:0/2675716465 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc034440 con 0x7f1ac415c390 2024-03-28T13:21:43.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.523+0000 7f1aca400640 1 -- 172.21.15.19:0/2675716465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac415c390 msgr2=0x7f1ac4073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:43.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.523+0000 7f1aca400640 1 --2- 172.21.15.19:0/2675716465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac415c390 0x7f1ac4073790 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009d90 tx=0x7f1abc02bea0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.523+0000 7f1aca400640 1 -- 172.21.15.19:0/2675716465 shutdown_connections 2024-03-28T13:21:43.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.523+0000 7f1aca400640 1 --2- 172.21.15.19:0/2675716465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac40af830 0x7f1ac4160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.523+0000 7f1aca400640 1 --2- 172.21.15.19:0/2675716465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac415c390 0x7f1ac4073790 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.523+0000 7f1aca400640 1 -- 172.21.15.19:0/2675716465 >> 172.21.15.19:0/2675716465 conn(0x7f1ac406b990 msgr2=0x7f1ac406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:43.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.523+0000 7f1aca400640 1 -- 172.21.15.19:0/2675716465 shutdown_connections 2024-03-28T13:21:43.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.523+0000 7f1aca400640 1 -- 172.21.15.19:0/2675716465 wait complete. 2024-03-28T13:21:43.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.525+0000 7f1aca400640 1 Processor -- start 2024-03-28T13:21:43.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.525+0000 7f1aca400640 1 -- start start 2024-03-28T13:21:43.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.526+0000 7f1aca400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac40af830 0x7f1ac414d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:43.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.526+0000 7f1aca400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac415c390 0x7f1ac4155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:43.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.526+0000 7f1aca400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac4160b80 0x7f1ac414eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:43.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.526+0000 7f1aca400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac40cf980 con 0x7f1ac4160b80 2024-03-28T13:21:43.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.526+0000 7f1aca400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4160120 con 0x7f1ac40af830 2024-03-28T13:21:43.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.526+0000 7f1aca400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac40d0d60 con 0x7f1ac415c390 2024-03-28T13:21:43.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.526+0000 7f1ac3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac4160b80 0x7f1ac414eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:43.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.527+0000 7f1ac3e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac4160b80 0x7f1ac414eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54322/0 (socket says 172.21.15.19:54322) 2024-03-28T13:21:43.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.527+0000 7f1ac3e00640 1 -- 172.21.15.19:0/177869911 learned_addr learned my addr 172.21.15.19:0/177869911 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:43.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.527+0000 7f1ac2a00640 1 --2- 172.21.15.19:0/177869911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac415c390 0x7f1ac4155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:43.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.527+0000 7f1ac3400640 1 --2- 172.21.15.19:0/177869911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac40af830 0x7f1ac414d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:43.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.527+0000 7f1ac3e00640 1 -- 172.21.15.19:0/177869911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac415c390 msgr2=0x7f1ac4155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:43.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.527+0000 7f1ac3e00640 1 --2- 172.21.15.19:0/177869911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac415c390 0x7f1ac4155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.527+0000 7f1ac3e00640 1 -- 172.21.15.19:0/177869911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac40af830 msgr2=0x7f1ac414d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:43.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.527+0000 7f1ac3e00640 1 --2- 172.21.15.19:0/177869911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac40af830 0x7f1ac414d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.527+0000 7f1ac3e00640 1 -- 172.21.15.19:0/177869911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac41f2510 con 0x7f1ac4160b80 2024-03-28T13:21:43.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.527+0000 7f1ac3e00640 1 --2- 172.21.15.19:0/177869911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac4160b80 0x7f1ac414eb90 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400c410 tx=0x7f1ab400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:43.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.528+0000 7f1abb400640 1 -- 172.21.15.19:0/177869911 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40074e0 con 0x7f1ac4160b80 2024-03-28T13:21:43.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.528+0000 7f1abb400640 1 -- 172.21.15.19:0/177869911 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab40047e0 con 0x7f1ac4160b80 2024-03-28T13:21:43.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.528+0000 7f1abb400640 1 -- 172.21.15.19:0/177869911 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4004bb0 con 0x7f1ac4160b80 2024-03-28T13:21:43.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.528+0000 7f1aca400640 1 -- 172.21.15.19:0/177869911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac414dc50 con 0x7f1ac4160b80 2024-03-28T13:21:43.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.528+0000 7f1aca400640 1 -- 172.21.15.19:0/177869911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac415efd0 con 0x7f1ac4160b80 2024-03-28T13:21:43.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.529+0000 7f1aca400640 1 -- 172.21.15.19:0/177869911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac40685c0 con 0x7f1ac4160b80 2024-03-28T13:21:43.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.530+0000 7f1abb400640 1 -- 172.21.15.19:0/177869911 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f1ab4011070 con 0x7f1ac4160b80 2024-03-28T13:21:43.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.531+0000 7f1abb400640 1 --2- 172.21.15.19:0/177869911 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1aa006c530 0x7f1aa006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:43.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.531+0000 7f1abb400640 1 -- 172.21.15.19:0/177869911 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f1ab408b390 con 0x7f1ac4160b80 2024-03-28T13:21:43.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.532+0000 7f1ac3400640 1 --2- 172.21.15.19:0/177869911 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1aa006c530 0x7f1aa006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:43.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.533+0000 7f1ac3400640 1 --2- 172.21.15.19:0/177869911 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1aa006c530 0x7f1aa006e9f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1abc036000 tx=0x7f1abc030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:43.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.536+0000 7f1abb400640 1 -- 172.21.15.19:0/177869911 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f1ac414dc50 con 0x7f1ac4160b80 2024-03-28T13:21:43.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.769+0000 7f1aca400640 1 -- 172.21.15.19:0/177869911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ac40b0830 con 0x7f1ac4160b80 2024-03-28T13:21:43.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.770+0000 7f1abb400640 1 -- 172.21.15.19:0/177869911 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5186 (secure 0 0 0) 0x7f1ac41f2510 con 0x7f1ac4160b80 2024-03-28T13:21:43.770 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:43.770 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3823671990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3823671990},{"type":"v1","addr":"172.21.15.188:6837","nonce":3823671990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/22158043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":22158043},{"type":"v1","addr":"172.21.15.188:6836","nonce":22158043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":8688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/230519675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":230519675},{"type":"v1","addr":"172.21.15.19:6834","nonce":230519675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:20:52.901269+0000","modified":"2024-03-28T13:21:31.965724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/1527816903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1527816903},{"type":"v1","addr":"172.21.15.19:6835","nonce":1527816903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9123,"qdb_cluster":[9123]},"id":8}]} 2024-03-28T13:21:43.770 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 112 2024-03-28T13:21:43.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.773+0000 7f1aca400640 1 -- 172.21.15.19:0/177869911 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1aa006c530 msgr2=0x7f1aa006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:43.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.773+0000 7f1aca400640 1 --2- 172.21.15.19:0/177869911 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1aa006c530 0x7f1aa006e9f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1abc036000 tx=0x7f1abc030000 comp rx=0 tx=0).stop 2024-03-28T13:21:43.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.773+0000 7f1aca400640 1 -- 172.21.15.19:0/177869911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac4160b80 msgr2=0x7f1ac414eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:43.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.773+0000 7f1aca400640 1 --2- 172.21.15.19:0/177869911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac4160b80 0x7f1ac414eb90 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400c410 tx=0x7f1ab400c8d0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.773+0000 7f1aca400640 1 -- 172.21.15.19:0/177869911 shutdown_connections 2024-03-28T13:21:43.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.773+0000 7f1aca400640 1 --2- 172.21.15.19:0/177869911 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f1aa006c530 0x7f1aa006e9f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.773+0000 7f1aca400640 1 --2- 172.21.15.19:0/177869911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1ac4160b80 0x7f1ac414eb90 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.773+0000 7f1aca400640 1 --2- 172.21.15.19:0/177869911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ac415c390 0x7f1ac4155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.773+0000 7f1aca400640 1 --2- 172.21.15.19:0/177869911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ac40af830 0x7f1ac414d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:43.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.773+0000 7f1aca400640 1 -- 172.21.15.19:0/177869911 >> 172.21.15.19:0/177869911 conn(0x7f1ac406b990 msgr2=0x7f1ac40afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:43.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.773+0000 7f1aca400640 1 -- 172.21.15.19:0/177869911 shutdown_connections 2024-03-28T13:21:43.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.773+0000 7f1aca400640 1 -- 172.21.15.19:0/177869911 wait complete. 2024-03-28T13:21:43.790 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:21:43.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.997+0000 7fe2d3200640 1 Processor -- start 2024-03-28T13:21:43.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.997+0000 7fe2d3200640 1 -- start start 2024-03-28T13:21:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.999+0000 7fe2d3200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2d415e4a0 0x7fe2d40b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.999+0000 7fe2d3200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe2d40cf700 0x7fe2d40b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.999+0000 7fe2d3200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2d415e880 con 0x7fe2d40d2310 2024-03-28T13:21:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.999+0000 7fe2d3200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fe2d40d34e0 con 0x7fe2d40cf700 2024-03-28T13:21:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.999+0000 7fe2d3200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe2d40d28d0 con 0x7fe2d415e4a0 2024-03-28T13:21:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.999+0000 7fe2cbe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fe2d40d2310 0x7fe2d40ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34170/0 (socket says 172.21.15.19:34170) 2024-03-28T13:21:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:43.999+0000 7fe2cbe00640 1 -- 172.21.15.19:0/998561475 learned_addr learned my addr 172.21.15.19:0/998561475 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.000+0000 7fe2d1e00640 1 --2- 172.21.15.19:0/998561475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2d415e4a0 0x7fe2d40b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.000+0000 7fe2d2800640 1 --2- 172.21.15.19:0/998561475 >> v2:172.21.15.188:3300/0 conn(0x7fe2d40cf700 0x7fe2d40b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.000+0000 7fe2d1400640 1 -- 172.21.15.19:0/998561475 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 7362213 0 0) 0x7fe2d415e880 con 0x7fe2d40d2310 2024-03-28T13:21:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.000+0000 7fe2d1400640 1 -- 172.21.15.19:0/998561475 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2b8003520 con 0x7fe2d40d2310 2024-03-28T13:21:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.000+0000 7fe2d1400640 1 -- 172.21.15.19:0/998561475 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2286474125 0 0) 0x7fe2b8003520 con 0x7fe2d40d2310 2024-03-28T13:21:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.000+0000 7fe2d1400640 1 -- 172.21.15.19:0/998561475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2d415e4a0 msgr2=0x7fe2d40b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.000+0000 7fe2d1400640 1 --2- 172.21.15.19:0/998561475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2d415e4a0 0x7fe2d40b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.000+0000 7fe2d1400640 1 -- 172.21.15.19:0/998561475 >> v2:172.21.15.188:3300/0 conn(0x7fe2d40cf700 msgr2=0x7fe2d40b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.000+0000 7fe2d1400640 1 --2- 172.21.15.19:0/998561475 >> v2:172.21.15.188:3300/0 conn(0x7fe2d40cf700 0x7fe2d40b73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.000+0000 7fe2d1400640 1 -- 172.21.15.19:0/998561475 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d40d2bb0 con 0x7fe2d40d2310 2024-03-28T13:21:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.000+0000 7fe2d1400640 1 -- 172.21.15.19:0/998561475 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fe2bc001670 con 0x7fe2d40d2310 2024-03-28T13:21:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.000+0000 7fe2d1400640 1 -- 172.21.15.19:0/998561475 >> v1:172.21.15.19:6789/0 conn(0x7fe2d40d2310 legacy=0x7fe2d40ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.000+0000 7fe2d1400640 1 --2- 172.21.15.19:0/998561475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2b80039b0 0x7fe2b8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.001+0000 7fe2d1400640 1 --2- 172.21.15.19:0/998561475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2d415e4a0 0x7fe2b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.001+0000 7fe2d1400640 1 --2- 172.21.15.19:0/998561475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2b8004d40 0x7fe2b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.001+0000 7fe2d1400640 1 -- 172.21.15.19:0/998561475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d40d34e0 con 0x7fe2b80039b0 2024-03-28T13:21:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.001+0000 7fe2d1400640 1 -- 172.21.15.19:0/998561475 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d40d28d0 con 0x7fe2b8004d40 2024-03-28T13:21:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.001+0000 7fe2d1400640 1 -- 172.21.15.19:0/998561475 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b800b9c0 con 0x7fe2d415e4a0 2024-03-28T13:21:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.001+0000 7fe2d1e00640 1 --2- 172.21.15.19:0/998561475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2b80039b0 0x7fe2b8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.001+0000 7fe2cbe00640 1 --2- 172.21.15.19:0/998561475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2d415e4a0 0x7fe2b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.001+0000 7fe2d2800640 1 --2- 172.21.15.19:0/998561475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2b8004d40 0x7fe2b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.001+0000 7fe2d1e00640 1 -- 172.21.15.19:0/998561475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2d415e4a0 msgr2=0x7fe2b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.001+0000 7fe2d1e00640 1 --2- 172.21.15.19:0/998561475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2d415e4a0 0x7fe2b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.001+0000 7fe2d1e00640 1 -- 172.21.15.19:0/998561475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2b8004d40 msgr2=0x7fe2b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.001+0000 7fe2d1e00640 1 --2- 172.21.15.19:0/998561475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2b8004d40 0x7fe2b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.001+0000 7fe2d1e00640 1 -- 172.21.15.19:0/998561475 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b8003520 con 0x7fe2b80039b0 2024-03-28T13:21:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.002+0000 7fe2d1e00640 1 --2- 172.21.15.19:0/998561475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2b80039b0 0x7fe2b8003df0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009ac0 tx=0x7fe2b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55af3e9b71eaafb server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.003+0000 7fe2d1400640 1 -- 172.21.15.19:0/998561475 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4039070 con 0x7fe2b80039b0 2024-03-28T13:21:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.003+0000 7fe2d1400640 1 -- 172.21.15.19:0/998561475 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c402c6b0 con 0x7fe2b80039b0 2024-03-28T13:21:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.003+0000 7fe2d1400640 1 -- 172.21.15.19:0/998561475 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4009690 con 0x7fe2b80039b0 2024-03-28T13:21:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.003+0000 7fe2d3200640 1 -- 172.21.15.19:0/998561475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2b80039b0 msgr2=0x7fe2b8003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.003+0000 7fe2d3200640 1 --2- 172.21.15.19:0/998561475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2b80039b0 0x7fe2b8003df0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009ac0 tx=0x7fe2b8005830 comp rx=0 tx=0).stop 2024-03-28T13:21:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.003+0000 7fe2d2800640 1 -- 172.21.15.19:0/998561475 reap_dead start 2024-03-28T13:21:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.003+0000 7fe2d3200640 1 -- 172.21.15.19:0/998561475 shutdown_connections 2024-03-28T13:21:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.003+0000 7fe2d3200640 1 -- 172.21.15.19:0/998561475 >> 172.21.15.19:0/998561475 conn(0x7fe2d40676d0 msgr2=0x7fe2d4067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:44.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.004+0000 7fe2d3200640 1 -- 172.21.15.19:0/998561475 shutdown_connections 2024-03-28T13:21:44.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.004+0000 7fe2d3200640 1 -- 172.21.15.19:0/998561475 wait complete. 2024-03-28T13:21:44.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.006+0000 7fe2d3200640 1 Processor -- start 2024-03-28T13:21:44.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.007+0000 7fe2d3200640 1 -- start start 2024-03-28T13:21:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.008+0000 7fe2d3200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2b80039b0 0x7fe2d4154850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.008+0000 7fe2d3200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2d4157610 0x7fe2d4157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.008+0000 7fe2d3200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2d40c9ba0 0x7fe2d40ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.008+0000 7fe2d3200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d40d3970 con 0x7fe2d40c9ba0 2024-03-28T13:21:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.008+0000 7fe2d3200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d40d3170 con 0x7fe2b80039b0 2024-03-28T13:21:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.008+0000 7fe2d3200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe2d40b9c60 con 0x7fe2d4157610 2024-03-28T13:21:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.008+0000 7fe2d2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2d40c9ba0 0x7fe2d40ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.009+0000 7fe2d2800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2d40c9ba0 0x7fe2d40ca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54340/0 (socket says 172.21.15.19:54340) 2024-03-28T13:21:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.009+0000 7fe2d2800640 1 -- 172.21.15.19:0/533595394 learned_addr learned my addr 172.21.15.19:0/533595394 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.009+0000 7fe2d1e00640 1 --2- 172.21.15.19:0/533595394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2b80039b0 0x7fe2d4154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.009+0000 7fe2d2800640 1 -- 172.21.15.19:0/533595394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2d4157610 msgr2=0x7fe2d4157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.009+0000 7fe2d2800640 1 --2- 172.21.15.19:0/533595394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2d4157610 0x7fe2d4157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.009+0000 7fe2d2800640 1 -- 172.21.15.19:0/533595394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2b80039b0 msgr2=0x7fe2d4154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.009+0000 7fe2d2800640 1 --2- 172.21.15.19:0/533595394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2b80039b0 0x7fe2d4154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.009+0000 7fe2d2800640 1 -- 172.21.15.19:0/533595394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d4154d90 con 0x7fe2d40c9ba0 2024-03-28T13:21:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.009+0000 7fe2d2800640 1 --2- 172.21.15.19:0/533595394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2d40c9ba0 0x7fe2d40ca020 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc007d10 tx=0x7fe2cc0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.009+0000 7fe2ca000640 1 -- 172.21.15.19:0/533595394 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc011de0 con 0x7fe2d40c9ba0 2024-03-28T13:21:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.010+0000 7fe2ca000640 1 -- 172.21.15.19:0/533595394 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2cc01b070 con 0x7fe2d40c9ba0 2024-03-28T13:21:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.010+0000 7fe2ca000640 1 -- 172.21.15.19:0/533595394 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc003990 con 0x7fe2d40c9ba0 2024-03-28T13:21:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.010+0000 7fe2d3200640 1 -- 172.21.15.19:0/533595394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d40c7e60 con 0x7fe2d40c9ba0 2024-03-28T13:21:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.010+0000 7fe2d3200640 1 -- 172.21.15.19:0/533595394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d40c8100 con 0x7fe2d40c9ba0 2024-03-28T13:21:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.011+0000 7fe2d3200640 1 -- 172.21.15.19:0/533595394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d415b8e0 con 0x7fe2d40c9ba0 2024-03-28T13:21:44.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.012+0000 7fe2ca000640 1 -- 172.21.15.19:0/533595394 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fe2cc005d10 con 0x7fe2d40c9ba0 2024-03-28T13:21:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.013+0000 7fe2ca000640 1 --2- 172.21.15.19:0/533595394 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2b006c530 0x7fe2b006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.013+0000 7fe2ca000640 1 -- 172.21.15.19:0/533595394 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7fe2cc08bd60 con 0x7fe2d40c9ba0 2024-03-28T13:21:44.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.013+0000 7fe2d1e00640 1 --2- 172.21.15.19:0/533595394 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2b006c530 0x7fe2b006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.015+0000 7fe2d1e00640 1 --2- 172.21.15.19:0/533595394 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2b006c530 0x7fe2b006e9f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4036000 tx=0x7fe2c4030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:44.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.018+0000 7fe2ca000640 1 -- 172.21.15.19:0/533595394 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fe2d415b8e0 con 0x7fe2d40c9ba0 2024-03-28T13:21:44.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.211+0000 7fe2d3200640 1 -- 172.21.15.19:0/533595394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2d4075940 con 0x7fe2d40c9ba0 2024-03-28T13:21:44.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.213+0000 7fe2ca000640 1 -- 172.21.15.19:0/533595394 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+17028 (secure 0 0 0) 0x7fe2cc025020 con 0x7fe2d40c9ba0 2024-03-28T13:21:44.213 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:44.213 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":123,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:21:26.921359+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:20:50.748212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-28T13:20:51.888444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":3}]},{"pgid":"17.b","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3930967142":"2024-03-29T13:21:26.921326+0000","172.21.15.19:6833/3930967142":"2024-03-29T13:21:26.921326+0000","172.21.15.188:6836/2088538359":"2024-03-29T13:21:12.469733+0000","172.21.15.188:6834/2088538359":"2024-03-29T13:21:12.469733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:21:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.216+0000 7fe2d3200640 1 -- 172.21.15.19:0/533595394 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2b006c530 msgr2=0x7fe2b006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.216+0000 7fe2d3200640 1 --2- 172.21.15.19:0/533595394 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2b006c530 0x7fe2b006e9f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4036000 tx=0x7fe2c4030000 comp rx=0 tx=0).stop 2024-03-28T13:21:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.216+0000 7fe2d3200640 1 -- 172.21.15.19:0/533595394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2d40c9ba0 msgr2=0x7fe2d40ca020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.216+0000 7fe2d3200640 1 --2- 172.21.15.19:0/533595394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2d40c9ba0 0x7fe2d40ca020 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fe2cc007d10 tx=0x7fe2cc0115a0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.216+0000 7fe2d3200640 1 -- 172.21.15.19:0/533595394 shutdown_connections 2024-03-28T13:21:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.216+0000 7fe2d3200640 1 --2- 172.21.15.19:0/533595394 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fe2b006c530 0x7fe2b006e9f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.216+0000 7fe2d3200640 1 --2- 172.21.15.19:0/533595394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe2d40c9ba0 0x7fe2d40ca020 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.216+0000 7fe2d3200640 1 --2- 172.21.15.19:0/533595394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2d4157610 0x7fe2d4157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.216+0000 7fe2d3200640 1 --2- 172.21.15.19:0/533595394 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2b80039b0 0x7fe2d4154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.216+0000 7fe2d3200640 1 -- 172.21.15.19:0/533595394 >> 172.21.15.19:0/533595394 conn(0x7fe2d40676d0 msgr2=0x7fe2d415b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:44.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.216+0000 7fe2d3200640 1 -- 172.21.15.19:0/533595394 shutdown_connections 2024-03-28T13:21:44.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.216+0000 7fe2d3200640 1 -- 172.21.15.19:0/533595394 wait complete. 2024-03-28T13:21:44.235 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-28T13:21:44.235 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:21:44.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.476+0000 7f6796600640 1 Processor -- start 2024-03-28T13:21:44.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.476+0000 7f6796600640 1 -- start start 2024-03-28T13:21:44.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.478+0000 7f6796600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790147a40 0x7f67900cfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.478+0000 7f6796600640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67900ce810 con 0x7f67900d01c0 2024-03-28T13:21:44.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.478+0000 7f6796600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6790161f30 con 0x7f67900ce380 2024-03-28T13:21:44.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.478+0000 7f6796600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6790162b00 con 0x7f6790147a40 2024-03-28T13:21:44.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.478+0000 7f678fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f67900d01c0 0x7f67900cdc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34184/0 (socket says 172.21.15.19:34184) 2024-03-28T13:21:44.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.478+0000 7f678fe00640 1 -- 172.21.15.19:0/3656996167 learned_addr learned my addr 172.21.15.19:0/3656996167 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:44.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.479+0000 7f6795200640 1 --2- 172.21.15.19:0/3656996167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790147a40 0x7f67900cfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.479+0000 7f678f400640 1 -- 172.21.15.19:0/3656996167 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 559050356 0 0) 0x7f67900ce810 con 0x7f67900d01c0 2024-03-28T13:21:44.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.479+0000 7f678f400640 1 -- 172.21.15.19:0/3656996167 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6778003520 con 0x7f67900d01c0 2024-03-28T13:21:44.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.479+0000 7f6795200640 1 -- 172.21.15.19:0/3656996167 >> v1:172.21.15.188:6789/0 conn(0x7f67900ce380 legacy=0x7f679006da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.479+0000 7f6795200640 1 -- 172.21.15.19:0/3656996167 >> v1:172.21.15.19:6789/0 conn(0x7f67900d01c0 legacy=0x7f67900cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.479+0000 7f6795200640 1 -- 172.21.15.19:0/3656996167 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67901621d0 con 0x7f6790147a40 2024-03-28T13:21:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.481+0000 7f6795200640 1 --2- 172.21.15.19:0/3656996167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790147a40 0x7f67900cfc80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6790162c80 tx=0x7f678402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ee8e646fd2a3d10 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.481+0000 7f678f400640 1 -- 172.21.15.19:0/3656996167 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784034070 con 0x7f6790147a40 2024-03-28T13:21:44.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.481+0000 7f678f400640 1 -- 172.21.15.19:0/3656996167 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6784030800 con 0x7f6790147a40 2024-03-28T13:21:44.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.481+0000 7f678f400640 1 -- 172.21.15.19:0/3656996167 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6784030bd0 con 0x7f6790147a40 2024-03-28T13:21:44.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.482+0000 7f6796600640 1 -- 172.21.15.19:0/3656996167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790147a40 msgr2=0x7f67900cfc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.482+0000 7f6796600640 1 --2- 172.21.15.19:0/3656996167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790147a40 0x7f67900cfc80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6790162c80 tx=0x7f678402b940 comp rx=0 tx=0).stop 2024-03-28T13:21:44.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.482+0000 7f6796600640 1 -- 172.21.15.19:0/3656996167 shutdown_connections 2024-03-28T13:21:44.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.482+0000 7f6796600640 1 --2- 172.21.15.19:0/3656996167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790147a40 0x7f67900cfc80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.482+0000 7f6796600640 1 -- 172.21.15.19:0/3656996167 >> 172.21.15.19:0/3656996167 conn(0x7f67900ccad0 msgr2=0x7f67900cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:44.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.482+0000 7f6796600640 1 -- 172.21.15.19:0/3656996167 shutdown_connections 2024-03-28T13:21:44.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.482+0000 7f6796600640 1 -- 172.21.15.19:0/3656996167 wait complete. 2024-03-28T13:21:44.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.485+0000 7f6796600640 1 Processor -- start 2024-03-28T13:21:44.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.485+0000 7f6796600640 1 -- start start 2024-03-28T13:21:44.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.487+0000 7f6796600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67900ce380 0x7f67901f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.487+0000 7f6796600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900d01c0 0x7f679013f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.487+0000 7f6796600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790147a40 0x7f679013f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.487+0000 7f6796600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f67900d0620 con 0x7f67900d01c0 2024-03-28T13:21:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.487+0000 7f6796600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6790162790 con 0x7f67900ce380 2024-03-28T13:21:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.487+0000 7f6796600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6790147e20 con 0x7f6790147a40 2024-03-28T13:21:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.487+0000 7f678fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900d01c0 0x7f679013f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.487+0000 7f678fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900d01c0 0x7f679013f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54348/0 (socket says 172.21.15.19:54348) 2024-03-28T13:21:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.487+0000 7f678fe00640 1 -- 172.21.15.19:0/4219317190 learned_addr learned my addr 172.21.15.19:0/4219317190 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.488+0000 7f6795200640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67900ce380 0x7f67901f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.488+0000 7f6795c00640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790147a40 0x7f679013f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.488+0000 7f678fe00640 1 -- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790147a40 msgr2=0x7f679013f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.488+0000 7f678fe00640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790147a40 0x7f679013f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.488+0000 7f678fe00640 1 -- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67900ce380 msgr2=0x7f67901f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.488+0000 7f678fe00640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67900ce380 0x7f67901f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.488+0000 7f678fe00640 1 -- 172.21.15.19:0/4219317190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679013dee0 con 0x7f67900d01c0 2024-03-28T13:21:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.488+0000 7f6795200640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67900ce380 0x7f67901f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.488+0000 7f6795c00640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790147a40 0x7f679013f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.488+0000 7f678fe00640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900d01c0 0x7f679013f220 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f677c00c500 tx=0x7f677c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.489+0000 7f678cc00640 1 -- 172.21.15.19:0/4219317190 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c017070 con 0x7f67900d01c0 2024-03-28T13:21:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.489+0000 7f678cc00640 1 -- 172.21.15.19:0/4219317190 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f677c005ce0 con 0x7f67900d01c0 2024-03-28T13:21:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.489+0000 7f6796600640 1 -- 172.21.15.19:0/4219317190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679013e180 con 0x7f67900d01c0 2024-03-28T13:21:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.489+0000 7f678cc00640 1 -- 172.21.15.19:0/4219317190 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c007330 con 0x7f67900d01c0 2024-03-28T13:21:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.489+0000 7f6796600640 1 -- 172.21.15.19:0/4219317190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f679013e740 con 0x7f67900d01c0 2024-03-28T13:21:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.490+0000 7f6796600640 1 -- 172.21.15.19:0/4219317190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6790075940 con 0x7f67900d01c0 2024-03-28T13:21:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.491+0000 7f678cc00640 1 -- 172.21.15.19:0/4219317190 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f677c01a070 con 0x7f67900d01c0 2024-03-28T13:21:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.492+0000 7f678cc00640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f677006c530 0x7f677006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.492+0000 7f678cc00640 1 -- 172.21.15.19:0/4219317190 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f677c08acb0 con 0x7f67900d01c0 2024-03-28T13:21:44.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.493+0000 7f6795200640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f677006c530 0x7f677006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.494+0000 7f6795200640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f677006c530 0x7f677006e9f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6784036000 tx=0x7f678403b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:44.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.497+0000 7f678cc00640 1 -- 172.21.15.19:0/4219317190 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f677c059290 con 0x7f67900d01c0 2024-03-28T13:21:44.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.736+0000 7f6796600640 1 -- 172.21.15.19:0/4219317190 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67900af6f0 con 0x7f67900d01c0 2024-03-28T13:21:44.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.737+0000 7f678cc00640 1 -- 172.21.15.19:0/4219317190 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5186 (secure 0 0 0) 0x7f677c058b90 con 0x7f67900d01c0 2024-03-28T13:21:44.736 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:44.736 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3823671990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3823671990},{"type":"v1","addr":"172.21.15.188:6837","nonce":3823671990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/22158043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":22158043},{"type":"v1","addr":"172.21.15.188:6836","nonce":22158043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":8688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/230519675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":230519675},{"type":"v1","addr":"172.21.15.19:6834","nonce":230519675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:20:52.901269+0000","modified":"2024-03-28T13:21:31.965724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/1527816903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1527816903},{"type":"v1","addr":"172.21.15.19:6835","nonce":1527816903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9123,"qdb_cluster":[9123]},"id":8}]} 2024-03-28T13:21:44.737 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 112 2024-03-28T13:21:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.739+0000 7f6796600640 1 -- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f677006c530 msgr2=0x7f677006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.739+0000 7f6796600640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f677006c530 0x7f677006e9f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6784036000 tx=0x7f678403b040 comp rx=0 tx=0).stop 2024-03-28T13:21:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.739+0000 7f6796600640 1 -- 172.21.15.19:0/4219317190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900d01c0 msgr2=0x7f679013f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.739+0000 7f6796600640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900d01c0 0x7f679013f220 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f677c00c500 tx=0x7f677c00c9c0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.739+0000 7f6796600640 1 -- 172.21.15.19:0/4219317190 shutdown_connections 2024-03-28T13:21:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.739+0000 7f6796600640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f677006c530 0x7f677006e9f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.739+0000 7f6796600640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790147a40 0x7f679013f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.739+0000 7f6796600640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67900d01c0 0x7f679013f220 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.739+0000 7f6796600640 1 --2- 172.21.15.19:0/4219317190 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67900ce380 0x7f67901f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.739+0000 7f6796600640 1 -- 172.21.15.19:0/4219317190 >> 172.21.15.19:0/4219317190 conn(0x7f67900ccad0 msgr2=0x7f67900cc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.739+0000 7f6796600640 1 -- 172.21.15.19:0/4219317190 shutdown_connections 2024-03-28T13:21:44.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.739+0000 7f6796600640 1 -- 172.21.15.19:0/4219317190 wait complete. 2024-03-28T13:21:44.760 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:21:44.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.959+0000 7f3a21000640 1 Processor -- start 2024-03-28T13:21:44.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.959+0000 7f3a21000640 1 -- start start 2024-03-28T13:21:44.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.961+0000 7f3a21000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3a1c15e560 0x7f3a1c0aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.961+0000 7f3a21000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a1c0b3ac0 0x7f3a1c06c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.961+0000 7f3a21000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a1c0b3f50 con 0x7f3a1c06cac0 2024-03-28T13:21:44.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.961+0000 7f3a21000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3a1c162930 con 0x7f3a1c15e560 2024-03-28T13:21:44.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.961+0000 7f3a21000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3a1c0ae8a0 con 0x7f3a1c0b3ac0 2024-03-28T13:21:44.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.962+0000 7f3a1be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f3a1c06cac0 0x7f3a1c15e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34186/0 (socket says 172.21.15.19:34186) 2024-03-28T13:21:44.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.962+0000 7f3a1be00640 1 -- 172.21.15.19:0/903776747 learned_addr learned my addr 172.21.15.19:0/903776747 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:44.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.962+0000 7f3a1b400640 1 --2- 172.21.15.19:0/903776747 >> v2:172.21.15.188:3300/0 conn(0x7f3a1c15e560 0x7f3a1c0aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.962+0000 7f3a13e00640 1 --2- 172.21.15.19:0/903776747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a1c0b3ac0 0x7f3a1c06c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.962+0000 7f3a1aa00640 1 -- 172.21.15.19:0/903776747 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2714572413 0 0) 0x7f3a1c0b3f50 con 0x7f3a1c06cac0 2024-03-28T13:21:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.962+0000 7f3a1aa00640 1 -- 172.21.15.19:0/903776747 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a00003520 con 0x7f3a1c06cac0 2024-03-28T13:21:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a1aa00640 1 -- 172.21.15.19:0/903776747 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2918238850 0 0) 0x7f3a00003520 con 0x7f3a1c06cac0 2024-03-28T13:21:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a1aa00640 1 -- 172.21.15.19:0/903776747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a1c0b3ac0 msgr2=0x7f3a1c06c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a1aa00640 1 --2- 172.21.15.19:0/903776747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a1c0b3ac0 0x7f3a1c06c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a1aa00640 1 -- 172.21.15.19:0/903776747 >> v2:172.21.15.188:3300/0 conn(0x7f3a1c15e560 msgr2=0x7f3a1c0aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a1aa00640 1 --2- 172.21.15.19:0/903776747 >> v2:172.21.15.188:3300/0 conn(0x7f3a1c15e560 0x7f3a1c0aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a1aa00640 1 -- 172.21.15.19:0/903776747 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1c162000 con 0x7f3a1c06cac0 2024-03-28T13:21:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a1aa00640 1 -- 172.21.15.19:0/903776747 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f3a140019f0 con 0x7f3a1c06cac0 2024-03-28T13:21:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a1aa00640 1 -- 172.21.15.19:0/903776747 >> v1:172.21.15.19:6789/0 conn(0x7f3a1c06cac0 legacy=0x7f3a1c15e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a1aa00640 1 --2- 172.21.15.19:0/903776747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a000039b0 0x7f3a00003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a1aa00640 1 --2- 172.21.15.19:0/903776747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a00004310 0x7f3a00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a1aa00640 1 --2- 172.21.15.19:0/903776747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a00004e50 0x7f3a000052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a1aa00640 1 -- 172.21.15.19:0/903776747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c162930 con 0x7f3a00004e50 2024-03-28T13:21:44.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a1aa00640 1 -- 172.21.15.19:0/903776747 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c0ae8a0 con 0x7f3a00004310 2024-03-28T13:21:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a1aa00640 1 -- 172.21.15.19:0/903776747 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3a0000bd40 con 0x7f3a000039b0 2024-03-28T13:21:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a1be00640 1 --2- 172.21.15.19:0/903776747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a00004e50 0x7f3a000052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.963+0000 7f3a13e00640 1 --2- 172.21.15.19:0/903776747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a00004310 0x7f3a00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.964+0000 7f3a1b400640 1 --2- 172.21.15.19:0/903776747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a000039b0 0x7f3a00003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.964+0000 7f3a1be00640 1 -- 172.21.15.19:0/903776747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a000039b0 msgr2=0x7f3a00003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.964+0000 7f3a1be00640 1 --2- 172.21.15.19:0/903776747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a000039b0 0x7f3a00003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.964+0000 7f3a1be00640 1 -- 172.21.15.19:0/903776747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a00004310 msgr2=0x7f3a00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.964+0000 7f3a1be00640 1 --2- 172.21.15.19:0/903776747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a00004310 0x7f3a00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.964+0000 7f3a1be00640 1 -- 172.21.15.19:0/903776747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a00003520 con 0x7f3a00004e50 2024-03-28T13:21:44.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.965+0000 7f3a1be00640 1 --2- 172.21.15.19:0/903776747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a00004e50 0x7f3a000052d0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f3a140013d0 tx=0x7f3a00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234deb7314028fc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:44.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.965+0000 7f3a1aa00640 1 -- 172.21.15.19:0/903776747 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1400a4e0 con 0x7f3a00004e50 2024-03-28T13:21:44.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.965+0000 7f3a1aa00640 1 -- 172.21.15.19:0/903776747 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a14034650 con 0x7f3a00004e50 2024-03-28T13:21:44.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.965+0000 7f3a1aa00640 1 -- 172.21.15.19:0/903776747 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a14034a40 con 0x7f3a00004e50 2024-03-28T13:21:44.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.965+0000 7f3a21000640 1 -- 172.21.15.19:0/903776747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a00004e50 msgr2=0x7f3a000052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.965+0000 7f3a21000640 1 --2- 172.21.15.19:0/903776747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a00004e50 0x7f3a000052d0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f3a140013d0 tx=0x7f3a00005830 comp rx=0 tx=0).stop 2024-03-28T13:21:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.965+0000 7f3a1be00640 1 -- 172.21.15.19:0/903776747 reap_dead start 2024-03-28T13:21:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.966+0000 7f3a21000640 1 -- 172.21.15.19:0/903776747 shutdown_connections 2024-03-28T13:21:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.966+0000 7f3a21000640 1 -- 172.21.15.19:0/903776747 >> 172.21.15.19:0/903776747 conn(0x7f3a1c071e70 msgr2=0x7f3a1c072280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.966+0000 7f3a21000640 1 -- 172.21.15.19:0/903776747 shutdown_connections 2024-03-28T13:21:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.966+0000 7f3a21000640 1 -- 172.21.15.19:0/903776747 wait complete. 2024-03-28T13:21:44.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.967+0000 7f3a21000640 1 Processor -- start 2024-03-28T13:21:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.967+0000 7f3a21000640 1 -- start start 2024-03-28T13:21:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.968+0000 7f3a21000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a00004e50 0x7f3a1c145490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.968+0000 7f3a21000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a1c15b250 0x7f3a1c0b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.968+0000 7f3a21000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a1c15b6b0 0x7f3a1c0b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.968+0000 7f3a21000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c162dc0 con 0x7f3a00004e50 2024-03-28T13:21:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.968+0000 7f3a21000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1c1625c0 con 0x7f3a1c15b250 2024-03-28T13:21:44.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.968+0000 7f3a21000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3a1c15e940 con 0x7f3a1c15b6b0 2024-03-28T13:21:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.969+0000 7f3a1b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a00004e50 0x7f3a1c145490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.969+0000 7f3a1b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a00004e50 0x7f3a1c145490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54376/0 (socket says 172.21.15.19:54376) 2024-03-28T13:21:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.969+0000 7f3a1b400640 1 -- 172.21.15.19:0/2858917964 learned_addr learned my addr 172.21.15.19:0/2858917964 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.969+0000 7f3a1be00640 1 --2- 172.21.15.19:0/2858917964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a1c15b6b0 0x7f3a1c0b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.969+0000 7f3a1b400640 1 -- 172.21.15.19:0/2858917964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a1c15b6b0 msgr2=0x7f3a1c0b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.969+0000 7f3a1b400640 1 --2- 172.21.15.19:0/2858917964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a1c15b6b0 0x7f3a1c0b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.969+0000 7f3a13e00640 1 --2- 172.21.15.19:0/2858917964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a1c15b250 0x7f3a1c0b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.969+0000 7f3a1b400640 1 -- 172.21.15.19:0/2858917964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a1c15b250 msgr2=0x7f3a1c0b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.969+0000 7f3a1b400640 1 --2- 172.21.15.19:0/2858917964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a1c15b250 0x7f3a1c0b15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.969+0000 7f3a1b400640 1 -- 172.21.15.19:0/2858917964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1c15eb90 con 0x7f3a00004e50 2024-03-28T13:21:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.969+0000 7f3a1b400640 1 --2- 172.21.15.19:0/2858917964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a00004e50 0x7f3a1c145490 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0116d0 tx=0x7f3a0c011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.970+0000 7f3a13400640 1 -- 172.21.15.19:0/2858917964 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c00f9c0 con 0x7f3a00004e50 2024-03-28T13:21:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.970+0000 7f3a13400640 1 -- 172.21.15.19:0/2858917964 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a0c00fb60 con 0x7f3a00004e50 2024-03-28T13:21:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.970+0000 7f3a13400640 1 -- 172.21.15.19:0/2858917964 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c015460 con 0x7f3a00004e50 2024-03-28T13:21:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.970+0000 7f3a21000640 1 -- 172.21.15.19:0/2858917964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1c15ee70 con 0x7f3a00004e50 2024-03-28T13:21:44.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.970+0000 7f3a21000640 1 -- 172.21.15.19:0/2858917964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a1c161540 con 0x7f3a00004e50 2024-03-28T13:21:44.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.971+0000 7f3a21000640 1 -- 172.21.15.19:0/2858917964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a1c161050 con 0x7f3a00004e50 2024-03-28T13:21:44.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.972+0000 7f3a13400640 1 -- 172.21.15.19:0/2858917964 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f3a0c015600 con 0x7f3a00004e50 2024-03-28T13:21:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.973+0000 7f3a13400640 1 --2- 172.21.15.19:0/2858917964 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f39f806c560 0x7f39f806ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:44.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.973+0000 7f3a13400640 1 -- 172.21.15.19:0/2858917964 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f3a0c08c1d0 con 0x7f3a00004e50 2024-03-28T13:21:44.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.973+0000 7f3a13e00640 1 --2- 172.21.15.19:0/2858917964 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f39f806c560 0x7f39f806ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:44.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.974+0000 7f3a13e00640 1 --2- 172.21.15.19:0/2858917964 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f39f806c560 0x7f39f806ea20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c1525e0 tx=0x7f3a04008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:44.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:44.977+0000 7f3a13400640 1 -- 172.21.15.19:0/2858917964 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f3a0c019030 con 0x7f3a00004e50 2024-03-28T13:21:45.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.209+0000 7f3a21000640 1 -- 172.21.15.19:0/2858917964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a1c075940 con 0x7f3a00004e50 2024-03-28T13:21:45.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.210+0000 7f3a13400640 1 -- 172.21.15.19:0/2858917964 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5186 (secure 0 0 0) 0x7f3a0c05a6f0 con 0x7f3a00004e50 2024-03-28T13:21:45.210 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:45.210 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":112,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3823671990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3823671990},{"type":"v1","addr":"172.21.15.188:6837","nonce":3823671990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/22158043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":22158043},{"type":"v1","addr":"172.21.15.188:6836","nonce":22158043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":8688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/230519675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":230519675},{"type":"v1","addr":"172.21.15.19:6834","nonce":230519675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-28T13:20:52.901269+0000","modified":"2024-03-28T13:21:31.965724+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9123":{"gid":9123,"name":"a","rank":0,"incarnation":109,"state":"up:active","state_seq":4,"addr":"172.21.15.19:6835/1527816903","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1527816903},{"type":"v1","addr":"172.21.15.19:6835","nonce":1527816903}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9123,"qdb_cluster":[9123]},"id":8}]} 2024-03-28T13:21:45.210 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 112 2024-03-28T13:21:45.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.213+0000 7f3a21000640 1 -- 172.21.15.19:0/2858917964 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f39f806c560 msgr2=0x7f39f806ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:45.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.213+0000 7f3a21000640 1 --2- 172.21.15.19:0/2858917964 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f39f806c560 0x7f39f806ea20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c1525e0 tx=0x7f3a04008040 comp rx=0 tx=0).stop 2024-03-28T13:21:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.213+0000 7f3a21000640 1 -- 172.21.15.19:0/2858917964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a00004e50 msgr2=0x7f3a1c145490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.213+0000 7f3a21000640 1 --2- 172.21.15.19:0/2858917964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a00004e50 0x7f3a1c145490 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0116d0 tx=0x7f3a0c011b90 comp rx=0 tx=0).stop 2024-03-28T13:21:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.213+0000 7f3a21000640 1 -- 172.21.15.19:0/2858917964 shutdown_connections 2024-03-28T13:21:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.213+0000 7f3a21000640 1 --2- 172.21.15.19:0/2858917964 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f39f806c560 0x7f39f806ea20 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.213+0000 7f3a21000640 1 --2- 172.21.15.19:0/2858917964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a1c15b6b0 0x7f3a1c0b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.213+0000 7f3a21000640 1 --2- 172.21.15.19:0/2858917964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a1c15b250 0x7f3a1c0b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.213+0000 7f3a21000640 1 --2- 172.21.15.19:0/2858917964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a00004e50 0x7f3a1c145490 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.213+0000 7f3a21000640 1 -- 172.21.15.19:0/2858917964 >> 172.21.15.19:0/2858917964 conn(0x7f3a1c071e70 msgr2=0x7f3a1c0b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.213+0000 7f3a21000640 1 -- 172.21.15.19:0/2858917964 shutdown_connections 2024-03-28T13:21:45.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.213+0000 7f3a21000640 1 -- 172.21.15.19:0/2858917964 wait complete. 2024-03-28T13:21:45.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:21:45.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.445+0000 7f76e7200640 1 Processor -- start 2024-03-28T13:21:45.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.446+0000 7f76e7200640 1 -- start start 2024-03-28T13:21:45.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.447+0000 7f76e7200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f76e80d2310 0x7f76e80ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.447+0000 7f76e7200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f76e80cf530 0x7f76e80b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.447+0000 7f76e7200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76e815e880 con 0x7f76e815e4a0 2024-03-28T13:21:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.447+0000 7f76e7200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f76e80cf9a0 con 0x7f76e80cf530 2024-03-28T13:21:45.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.447+0000 7f76e7200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f76e80d34e0 con 0x7f76e80d2310 2024-03-28T13:21:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.448+0000 7f76e6800640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f76e80cf530 0x7f76e80b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.448+0000 7f76e5e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f76e815e4a0 0x7f76e80b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34248/0 (socket says 172.21.15.19:34248) 2024-03-28T13:21:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.448+0000 7f76e5e00640 1 -- 172.21.15.19:0/912591566 learned_addr learned my addr 172.21.15.19:0/912591566 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.448+0000 7f76dd400640 1 --2- 172.21.15.19:0/912591566 >> v2:172.21.15.188:3300/0 conn(0x7f76e80d2310 0x7f76e80ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.448+0000 7f76e6800640 1 -- 172.21.15.19:0/912591566 >> v2:172.21.15.188:3300/0 conn(0x7f76e80d2310 msgr2=0x7f76e80ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.448+0000 7f76e6800640 1 --2- 172.21.15.19:0/912591566 >> v2:172.21.15.188:3300/0 conn(0x7f76e80d2310 0x7f76e80ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.448+0000 7f76e6800640 1 -- 172.21.15.19:0/912591566 >> v1:172.21.15.19:6789/0 conn(0x7f76e815e4a0 legacy=0x7f76e80b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.448+0000 7f76e6800640 1 -- 172.21.15.19:0/912591566 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e80d28d0 con 0x7f76e80cf530 2024-03-28T13:21:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.448+0000 7f76e5400640 1 -- 172.21.15.19:0/912591566 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3670806053 0 0) 0x7f76e815e880 con 0x7f76e815e4a0 2024-03-28T13:21:45.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.448+0000 7f76dd400640 1 --2- 172.21.15.19:0/912591566 >> v2:172.21.15.188:3300/0 conn(0x7f76e80d2310 0x7f76e80ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:21:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.449+0000 7f76e6800640 1 --2- 172.21.15.19:0/912591566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76e80cf530 0x7f76e80b73e0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f76e80d3660 tx=0x7f76e002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cea80617bb7308 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.450+0000 7f76e5400640 1 -- 172.21.15.19:0/912591566 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0039070 con 0x7f76e80cf530 2024-03-28T13:21:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.450+0000 7f76e5400640 1 -- 172.21.15.19:0/912591566 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e002cbd0 con 0x7f76e80cf530 2024-03-28T13:21:45.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.450+0000 7f76e5400640 1 -- 172.21.15.19:0/912591566 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e0034440 con 0x7f76e80cf530 2024-03-28T13:21:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.450+0000 7f76e7200640 1 -- 172.21.15.19:0/912591566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76e80cf530 msgr2=0x7f76e80b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.450+0000 7f76e7200640 1 --2- 172.21.15.19:0/912591566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76e80cf530 0x7f76e80b73e0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f76e80d3660 tx=0x7f76e002be20 comp rx=0 tx=0).stop 2024-03-28T13:21:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.451+0000 7f76e7200640 1 -- 172.21.15.19:0/912591566 shutdown_connections 2024-03-28T13:21:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.451+0000 7f76e7200640 1 --2- 172.21.15.19:0/912591566 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76e80cf530 0x7f76e80b73e0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.451+0000 7f76e7200640 1 --2- 172.21.15.19:0/912591566 >> v2:172.21.15.188:3300/0 conn(0x7f76e80d2310 0x7f76e80ceff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.451+0000 7f76e7200640 1 -- 172.21.15.19:0/912591566 >> 172.21.15.19:0/912591566 conn(0x7f76e80676d0 msgr2=0x7f76e8067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:45.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.451+0000 7f76e7200640 1 -- 172.21.15.19:0/912591566 shutdown_connections 2024-03-28T13:21:45.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.451+0000 7f76e7200640 1 -- 172.21.15.19:0/912591566 wait complete. 2024-03-28T13:21:45.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.452+0000 7f76e7200640 1 Processor -- start 2024-03-28T13:21:45.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.453+0000 7f76e7200640 1 -- start start 2024-03-28T13:21:45.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.453+0000 7f76e7200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76e80cf530 0x7f76e815f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:45.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.453+0000 7f76e7200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76e80d2310 0x7f76e815fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:45.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.453+0000 7f76e7200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e815e4a0 0x7f76e81509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:45.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.453+0000 7f76e7200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76e80d3170 con 0x7f76e80d2310 2024-03-28T13:21:45.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.453+0000 7f76e7200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f76e80d2e90 con 0x7f76e815e4a0 2024-03-28T13:21:45.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.453+0000 7f76e7200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f76e80d3970 con 0x7f76e80cf530 2024-03-28T13:21:45.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.454+0000 7f76dd400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76e80d2310 0x7f76e815fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:45.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.454+0000 7f76dd400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76e80d2310 0x7f76e815fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44252/0 (socket says 172.21.15.19:44252) 2024-03-28T13:21:45.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.454+0000 7f76dd400640 1 -- 172.21.15.19:0/3610833600 learned_addr learned my addr 172.21.15.19:0/3610833600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.454+0000 7f76dd400640 1 -- 172.21.15.19:0/3610833600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76e80cf530 msgr2=0x7f76e815f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.454+0000 7f76dd400640 1 --2- 172.21.15.19:0/3610833600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76e80cf530 0x7f76e815f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.454+0000 7f76dd400640 1 -- 172.21.15.19:0/3610833600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e815e4a0 msgr2=0x7f76e81509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.454+0000 7f76dd400640 1 --2- 172.21.15.19:0/3610833600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e815e4a0 0x7f76e81509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.454+0000 7f76dd400640 1 -- 172.21.15.19:0/3610833600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e81602e0 con 0x7f76e80d2310 2024-03-28T13:21:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.454+0000 7f76dd400640 1 --2- 172.21.15.19:0/3610833600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76e80d2310 0x7f76e815fda0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f76d000e760 tx=0x7f76d000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.455+0000 7f76dea00640 1 -- 172.21.15.19:0/3610833600 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d000f040 con 0x7f76e80d2310 2024-03-28T13:21:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.455+0000 7f76dea00640 1 -- 172.21.15.19:0/3610833600 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76d0004230 con 0x7f76e80d2310 2024-03-28T13:21:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.455+0000 7f76dea00640 1 -- 172.21.15.19:0/3610833600 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d000ee70 con 0x7f76e80d2310 2024-03-28T13:21:45.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.455+0000 7f76e7200640 1 -- 172.21.15.19:0/3610833600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e8160980 con 0x7f76e80d2310 2024-03-28T13:21:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.455+0000 7f76e7200640 1 -- 172.21.15.19:0/3610833600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e8160f40 con 0x7f76e80d2310 2024-03-28T13:21:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.456+0000 7f76e7200640 1 -- 172.21.15.19:0/3610833600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e8075940 con 0x7f76e80d2310 2024-03-28T13:21:45.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.457+0000 7f76dea00640 1 -- 172.21.15.19:0/3610833600 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f76d000c670 con 0x7f76e80d2310 2024-03-28T13:21:45.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.458+0000 7f76dea00640 1 --2- 172.21.15.19:0/3610833600 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f76c406c530 0x7f76c406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:45.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.458+0000 7f76dea00640 1 -- 172.21.15.19:0/3610833600 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f76d008b430 con 0x7f76e80d2310 2024-03-28T13:21:45.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.459+0000 7f76e5e00640 1 --2- 172.21.15.19:0/3610833600 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f76c406c530 0x7f76c406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:45.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.460+0000 7f76e5e00640 1 --2- 172.21.15.19:0/3610833600 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f76c406c530 0x7f76c406e9f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f76d8007fd0 tx=0x7f76d8007960 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:45.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.463+0000 7f76dea00640 1 -- 172.21.15.19:0/3610833600 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f76d0017020 con 0x7f76e80d2310 2024-03-28T13:21:45.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.720+0000 7f76e7200640 1 -- 172.21.15.19:0/3610833600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76e815db40 con 0x7f76e80d2310 2024-03-28T13:21:45.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.721+0000 7f76dea00640 1 -- 172.21.15.19:0/3610833600 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) ==== 74+0+17028 (secure 0 0 0) 0x7f76d0059960 con 0x7f76e80d2310 2024-03-28T13:21:45.721 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:45.721 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":123,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:21:26.921359+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-28T13:20:50.748212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-28T13:20:51.888444+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":3}]},{"pgid":"17.b","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/3930967142":"2024-03-29T13:21:26.921326+0000","172.21.15.19:6833/3930967142":"2024-03-29T13:21:26.921326+0000","172.21.15.188:6836/2088538359":"2024-03-29T13:21:12.469733+0000","172.21.15.188:6834/2088538359":"2024-03-29T13:21:12.469733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:21:45.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.724+0000 7f76e7200640 1 -- 172.21.15.19:0/3610833600 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f76c406c530 msgr2=0x7f76c406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.724+0000 7f76e7200640 1 --2- 172.21.15.19:0/3610833600 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f76c406c530 0x7f76c406e9f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f76d8007fd0 tx=0x7f76d8007960 comp rx=0 tx=0).stop 2024-03-28T13:21:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.724+0000 7f76e7200640 1 -- 172.21.15.19:0/3610833600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76e80d2310 msgr2=0x7f76e815fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.724+0000 7f76e7200640 1 --2- 172.21.15.19:0/3610833600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76e80d2310 0x7f76e815fda0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f76d000e760 tx=0x7f76d000ec20 comp rx=0 tx=0).stop 2024-03-28T13:21:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.724+0000 7f76e7200640 1 -- 172.21.15.19:0/3610833600 shutdown_connections 2024-03-28T13:21:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.724+0000 7f76e7200640 1 --2- 172.21.15.19:0/3610833600 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f76c406c530 0x7f76c406e9f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.724+0000 7f76e7200640 1 --2- 172.21.15.19:0/3610833600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76e815e4a0 0x7f76e81509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.724+0000 7f76e7200640 1 --2- 172.21.15.19:0/3610833600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76e80d2310 0x7f76e815fda0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.724+0000 7f76e7200640 1 --2- 172.21.15.19:0/3610833600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f76e80cf530 0x7f76e815f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.724+0000 7f76e7200640 1 -- 172.21.15.19:0/3610833600 >> 172.21.15.19:0/3610833600 conn(0x7f76e80676d0 msgr2=0x7f76e8067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.724+0000 7f76e7200640 1 -- 172.21.15.19:0/3610833600 shutdown_connections 2024-03-28T13:21:45.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.724+0000 7f76e7200640 1 -- 172.21.15.19:0/3610833600 wait complete. 2024-03-28T13:21:45.742 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-28T13:21:45.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.957+0000 7f7a57c00640 1 Processor -- start 2024-03-28T13:21:45.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.958+0000 7f7a57c00640 1 -- start start 2024-03-28T13:21:45.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.959+0000 7f7a57c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a5015e7b0 0x7f7a500b4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:45.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.959+0000 7f7a57c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7a500d1d70 0x7f7a500c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:45.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.959+0000 7f7a57c00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7a500c7640 0x7f7a50071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:45.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.959+0000 7f7a57c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7a500d1240 con 0x7f7a500d1d70 2024-03-28T13:21:45.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.959+0000 7f7a57c00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7a500d01a0 con 0x7f7a500c7640 2024-03-28T13:21:45.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.959+0000 7f7a57c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7a500d0480 con 0x7f7a5015e7b0 2024-03-28T13:21:45.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.959+0000 7f7a47e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7a500d1d70 0x7f7a500c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:45.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.959+0000 7f7a47e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7a500d1d70 0x7f7a500c7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44256/0 (socket says 172.21.15.19:44256) 2024-03-28T13:21:45.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.959+0000 7f7a47e00640 1 -- 172.21.15.19:0/2954421523 learned_addr learned my addr 172.21.15.19:0/2954421523 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:45.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.960+0000 7f7a55c00640 1 --2- 172.21.15.19:0/2954421523 >> v2:172.21.15.188:3300/0 conn(0x7f7a500c7640 0x7f7a50071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:45.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.960+0000 7f7a55200640 1 --2- 172.21.15.19:0/2954421523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a5015e7b0 0x7f7a500b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:45.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.960+0000 7f7a47e00640 1 -- 172.21.15.19:0/2954421523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a5015e7b0 msgr2=0x7f7a500b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:45.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.960+0000 7f7a47e00640 1 --2- 172.21.15.19:0/2954421523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a5015e7b0 0x7f7a500b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.960+0000 7f7a47e00640 1 -- 172.21.15.19:0/2954421523 >> v2:172.21.15.188:3300/0 conn(0x7f7a500c7640 msgr2=0x7f7a50071000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:45.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.960+0000 7f7a47e00640 1 --2- 172.21.15.19:0/2954421523 >> v2:172.21.15.188:3300/0 conn(0x7f7a500c7640 0x7f7a50071000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.960+0000 7f7a47e00640 1 -- 172.21.15.19:0/2954421523 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a500d0760 con 0x7f7a500d1d70 2024-03-28T13:21:45.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.960+0000 7f7a55c00640 1 --2- 172.21.15.19:0/2954421523 >> v2:172.21.15.188:3300/0 conn(0x7f7a500c7640 0x7f7a50071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:45.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.960+0000 7f7a55200640 1 --2- 172.21.15.19:0/2954421523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a5015e7b0 0x7f7a500b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:45.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.961+0000 7f7a47e00640 1 --2- 172.21.15.19:0/2954421523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a500d1d70 0x7f7a500c7100 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f7a5006e4d0 tx=0x7f7a3802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725f76f4e9797f62 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:45.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.961+0000 7f7a4fe00640 1 -- 172.21.15.19:0/2954421523 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38035020 con 0x7f7a500d1d70 2024-03-28T13:21:45.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.961+0000 7f7a4fe00640 1 -- 172.21.15.19:0/2954421523 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a38031880 con 0x7f7a500d1d70 2024-03-28T13:21:45.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.961+0000 7f7a4fe00640 1 -- 172.21.15.19:0/2954421523 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38031c50 con 0x7f7a500d1d70 2024-03-28T13:21:45.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.962+0000 7f7a57c00640 1 -- 172.21.15.19:0/2954421523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a500d1d70 msgr2=0x7f7a500c7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:45.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.962+0000 7f7a57c00640 1 --2- 172.21.15.19:0/2954421523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a500d1d70 0x7f7a500c7100 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f7a5006e4d0 tx=0x7f7a3802b920 comp rx=0 tx=0).stop 2024-03-28T13:21:45.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.962+0000 7f7a57c00640 1 -- 172.21.15.19:0/2954421523 shutdown_connections 2024-03-28T13:21:45.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.962+0000 7f7a57c00640 1 --2- 172.21.15.19:0/2954421523 >> v2:172.21.15.188:3300/0 conn(0x7f7a500c7640 0x7f7a50071000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.962+0000 7f7a57c00640 1 --2- 172.21.15.19:0/2954421523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a500d1d70 0x7f7a500c7100 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.962+0000 7f7a57c00640 1 --2- 172.21.15.19:0/2954421523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a5015e7b0 0x7f7a500b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.962+0000 7f7a57c00640 1 -- 172.21.15.19:0/2954421523 >> 172.21.15.19:0/2954421523 conn(0x7f7a500b3e90 msgr2=0x7f7a500b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:45.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.962+0000 7f7a57c00640 1 -- 172.21.15.19:0/2954421523 shutdown_connections 2024-03-28T13:21:45.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.962+0000 7f7a57c00640 1 -- 172.21.15.19:0/2954421523 wait complete. 2024-03-28T13:21:45.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.965+0000 7f7a57c00640 1 Processor -- start 2024-03-28T13:21:45.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.965+0000 7f7a57c00640 1 -- start start 2024-03-28T13:21:45.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.967+0000 7f7a57c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a500c7640 0x7f7a50146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:45.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.967+0000 7f7a57c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a500d1d70 0x7f7a500cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:45.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.967+0000 7f7a57c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a5015e7b0 0x7f7a500cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:45.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.967+0000 7f7a57c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7a500cfe30 con 0x7f7a5015e7b0 2024-03-28T13:21:45.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.967+0000 7f7a57c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7a500d0d20 con 0x7f7a500c7640 2024-03-28T13:21:45.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.967+0000 7f7a57c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7a50071540 con 0x7f7a500d1d70 2024-03-28T13:21:45.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.967+0000 7f7a55c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a5015e7b0 0x7f7a500cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:45.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.967+0000 7f7a55c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a5015e7b0 0x7f7a500cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44258/0 (socket says 172.21.15.19:44258) 2024-03-28T13:21:45.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.967+0000 7f7a55c00640 1 -- 172.21.15.19:0/3166888184 learned_addr learned my addr 172.21.15.19:0/3166888184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:45.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.968+0000 7f7a55200640 1 --2- 172.21.15.19:0/3166888184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a500c7640 0x7f7a50146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:45.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.968+0000 7f7a47e00640 1 --2- 172.21.15.19:0/3166888184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a500d1d70 0x7f7a500cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:45.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.968+0000 7f7a55c00640 1 -- 172.21.15.19:0/3166888184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a500d1d70 msgr2=0x7f7a500cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:45.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.968+0000 7f7a55c00640 1 --2- 172.21.15.19:0/3166888184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a500d1d70 0x7f7a500cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.968+0000 7f7a55c00640 1 -- 172.21.15.19:0/3166888184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a500c7640 msgr2=0x7f7a50146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:45.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.968+0000 7f7a55c00640 1 --2- 172.21.15.19:0/3166888184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a500c7640 0x7f7a50146d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:45.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.968+0000 7f7a55c00640 1 -- 172.21.15.19:0/3166888184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5015f8a0 con 0x7f7a5015e7b0 2024-03-28T13:21:45.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.968+0000 7f7a55c00640 1 --2- 172.21.15.19:0/3166888184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a5015e7b0 0x7f7a500cb820 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800c820 tx=0x7f7a4800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:45.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.968+0000 7f7a4d600640 1 -- 172.21.15.19:0/3166888184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48015070 con 0x7f7a5015e7b0 2024-03-28T13:21:45.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.969+0000 7f7a4d600640 1 -- 172.21.15.19:0/3166888184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a48007a30 con 0x7f7a5015e7b0 2024-03-28T13:21:45.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.969+0000 7f7a57c00640 1 -- 172.21.15.19:0/3166888184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5015fb80 con 0x7f7a5015e7b0 2024-03-28T13:21:45.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.969+0000 7f7a4d600640 1 -- 172.21.15.19:0/3166888184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48007e20 con 0x7f7a5015e7b0 2024-03-28T13:21:45.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.969+0000 7f7a57c00640 1 -- 172.21.15.19:0/3166888184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a50160140 con 0x7f7a5015e7b0 2024-03-28T13:21:45.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.970+0000 7f7a57c00640 1 -- 172.21.15.19:0/3166888184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a50068cd0 con 0x7f7a5015e7b0 2024-03-28T13:21:45.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.971+0000 7f7a4d600640 1 -- 172.21.15.19:0/3166888184 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7a48018070 con 0x7f7a5015e7b0 2024-03-28T13:21:45.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.972+0000 7f7a4d600640 1 --2- 172.21.15.19:0/3166888184 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a2c06c530 0x7f7a2c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:45.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.972+0000 7f7a4d600640 1 -- 172.21.15.19:0/3166888184 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5939+0+0 (secure 0 0 0) 0x7f7a4808b830 con 0x7f7a5015e7b0 2024-03-28T13:21:45.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.973+0000 7f7a55200640 1 --2- 172.21.15.19:0/3166888184 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a2c06c530 0x7f7a2c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:45.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.974+0000 7f7a55200640 1 --2- 172.21.15.19:0/3166888184 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a2c06c530 0x7f7a2c06e9f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7a40007130 tx=0x7f7a40007060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:45.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:45.977+0000 7f7a4d600640 1 -- 172.21.15.19:0/3166888184 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7a48059cc0 con 0x7f7a5015e7b0 2024-03-28T13:21:46.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.249+0000 7f7a57c00640 1 -- 172.21.15.19:0/3166888184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7a5015d510 con 0x7f7a5015e7b0 2024-03-28T13:21:46.250 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-03-28T13:21:46.250+0000 7f6b74400640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-28T13:21:46.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.254+0000 7f7a4d600640 1 -- 172.21.15.19:0/3166888184 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) ==== 161+0+0 (secure 0 0 0) 0x7f7a480595c0 con 0x7f7a5015e7b0 2024-03-28T13:21:46.254 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-28T13:21:46.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.256+0000 7f7a57c00640 1 -- 172.21.15.19:0/3166888184 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a2c06c530 msgr2=0x7f7a2c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.256+0000 7f7a57c00640 1 --2- 172.21.15.19:0/3166888184 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a2c06c530 0x7f7a2c06e9f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7a40007130 tx=0x7f7a40007060 comp rx=0 tx=0).stop 2024-03-28T13:21:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.256+0000 7f7a57c00640 1 -- 172.21.15.19:0/3166888184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a5015e7b0 msgr2=0x7f7a500cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.256+0000 7f7a57c00640 1 --2- 172.21.15.19:0/3166888184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a5015e7b0 0x7f7a500cb820 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800c820 tx=0x7f7a4800cce0 comp rx=0 tx=0).stop 2024-03-28T13:21:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.257+0000 7f7a57c00640 1 -- 172.21.15.19:0/3166888184 shutdown_connections 2024-03-28T13:21:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.257+0000 7f7a57c00640 1 --2- 172.21.15.19:0/3166888184 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7a2c06c530 0x7f7a2c06e9f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.257+0000 7f7a57c00640 1 --2- 172.21.15.19:0/3166888184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7a5015e7b0 0x7f7a500cb820 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:46.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.257+0000 7f7a57c00640 1 --2- 172.21.15.19:0/3166888184 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7a500d1d70 0x7f7a500cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:46.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.257+0000 7f7a57c00640 1 --2- 172.21.15.19:0/3166888184 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7a500c7640 0x7f7a50146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:46.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.257+0000 7f7a57c00640 1 -- 172.21.15.19:0/3166888184 >> 172.21.15.19:0/3166888184 conn(0x7f7a500b3e90 msgr2=0x7f7a5015da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:46.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.257+0000 7f7a57c00640 1 -- 172.21.15.19:0/3166888184 shutdown_connections 2024-03-28T13:21:46.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.257+0000 7f7a57c00640 1 -- 172.21.15.19:0/3166888184 wait complete. 2024-03-28T13:21:46.266 INFO:tasks.ceph.mds.a.smithi019.stderr: -1552> 2024-03-28T13:21:32.941+0000 7f4ca0000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-28T13:21:46.277 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-28T13:21:46.340 INFO:tasks.ceph.mds.a.smithi019.stdout:starting mds.a at 2024-03-28T13:21:46.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.465+0000 7f25a2600640 1 Processor -- start 2024-03-28T13:21:46.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.466+0000 7f25a2600640 1 -- start start 2024-03-28T13:21:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.467+0000 7f25a2600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f259c0b4890 0x7f259c072570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.467+0000 7f25a2600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f259c15f6a0 0x7f259c072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.467+0000 7f25a2600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f259c0b4c70 con 0x7f259c06d620 2024-03-28T13:21:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.467+0000 7f25a2600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f259c0b3eb0 con 0x7f259c0b4890 2024-03-28T13:21:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.467+0000 7f25a2600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f259c15fb30 con 0x7f259c15f6a0 2024-03-28T13:21:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.467+0000 7f25a1200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f259c0b4890 0x7f259c072570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.467+0000 7f25a1200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f259c0b4890 0x7f259c072570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44298/0 (socket says 172.21.15.19:44298) 2024-03-28T13:21:46.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.467+0000 7f25a1200640 1 -- 172.21.15.19:0/2832054230 learned_addr learned my addr 172.21.15.19:0/2832054230 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:46.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.468+0000 7f25a1200640 1 -- 172.21.15.19:0/2832054230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f259c15f6a0 msgr2=0x7f259c072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:46.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.468+0000 7f259be00640 1 --2- 172.21.15.19:0/2832054230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f259c15f6a0 0x7f259c072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:46.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.468+0000 7f25a1200640 1 --2- 172.21.15.19:0/2832054230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f259c15f6a0 0x7f259c072ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:46.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.468+0000 7f25a1200640 1 -- 172.21.15.19:0/2832054230 >> v1:172.21.15.188:6789/0 conn(0x7f259c06d620 legacy=0x7f259c0713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:46.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.468+0000 7f25a1200640 1 -- 172.21.15.19:0/2832054230 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259c15b7b0 con 0x7f259c0b4890 2024-03-28T13:21:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.469+0000 7f25a1200640 1 --2- 172.21.15.19:0/2832054230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0b4890 0x7f259c072570 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f258c009870 tx=0x7f258c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27e6727a755bbc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.469+0000 7f259b400640 1 -- 172.21.15.19:0/2832054230 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c035020 con 0x7f259c0b4890 2024-03-28T13:21:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.469+0000 7f259b400640 1 -- 172.21.15.19:0/2832054230 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f258c031640 con 0x7f259c0b4890 2024-03-28T13:21:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.469+0000 7f259b400640 1 -- 172.21.15.19:0/2832054230 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c02d040 con 0x7f259c0b4890 2024-03-28T13:21:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.470+0000 7f25a2600640 1 -- 172.21.15.19:0/2832054230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0b4890 msgr2=0x7f259c072570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.470+0000 7f25a2600640 1 --2- 172.21.15.19:0/2832054230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0b4890 0x7f259c072570 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f258c009870 tx=0x7f258c02b8f0 comp rx=0 tx=0).stop 2024-03-28T13:21:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.470+0000 7f25a2600640 1 -- 172.21.15.19:0/2832054230 shutdown_connections 2024-03-28T13:21:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.470+0000 7f25a2600640 1 --2- 172.21.15.19:0/2832054230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f259c15f6a0 0x7f259c072ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.470+0000 7f25a2600640 1 --2- 172.21.15.19:0/2832054230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0b4890 0x7f259c072570 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:46.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.470+0000 7f25a2600640 1 -- 172.21.15.19:0/2832054230 >> 172.21.15.19:0/2832054230 conn(0x7f259c0b1ba0 msgr2=0x7f259c0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:46.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.470+0000 7f25a2600640 1 -- 172.21.15.19:0/2832054230 shutdown_connections 2024-03-28T13:21:46.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.470+0000 7f25a2600640 1 -- 172.21.15.19:0/2832054230 wait complete. 2024-03-28T13:21:46.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.472+0000 7f25a2600640 1 Processor -- start 2024-03-28T13:21:46.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.472+0000 7f25a2600640 1 -- start start 2024-03-28T13:21:46.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.473+0000 7f25a2600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f259c06d620 0x7f259c1525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:46.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.473+0000 7f25a2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0b4890 0x7f259c140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:46.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.473+0000 7f25a2600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f259c0cf5e0 0x7f259c0b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:46.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.473+0000 7f25a2600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f259c15c050 con 0x7f259c0b4890 2024-03-28T13:21:46.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.473+0000 7f25a2600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f259c15bd70 con 0x7f259c0cf5e0 2024-03-28T13:21:46.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.473+0000 7f25a2600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f259c15c570 con 0x7f259c06d620 2024-03-28T13:21:46.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.473+0000 7f259be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0b4890 0x7f259c140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:46.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.473+0000 7f259be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0b4890 0x7f259c140a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44314/0 (socket says 172.21.15.19:44314) 2024-03-28T13:21:46.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.473+0000 7f259be00640 1 -- 172.21.15.19:0/1334227718 learned_addr learned my addr 172.21.15.19:0/1334227718 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:46.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.474+0000 7f25a1200640 1 --2- 172.21.15.19:0/1334227718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f259c06d620 0x7f259c1525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:46.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.474+0000 7f259be00640 1 -- 172.21.15.19:0/1334227718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f259c06d620 msgr2=0x7f259c1525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:46.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.474+0000 7f259be00640 1 --2- 172.21.15.19:0/1334227718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f259c06d620 0x7f259c1525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:46.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.474+0000 7f259be00640 1 -- 172.21.15.19:0/1334227718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f259c0cf5e0 msgr2=0x7f259c0b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:46.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.474+0000 7f259be00640 1 --2- 172.21.15.19:0/1334227718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f259c0cf5e0 0x7f259c0b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:46.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.474+0000 7f259be00640 1 -- 172.21.15.19:0/1334227718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259c152b10 con 0x7f259c0b4890 2024-03-28T13:21:46.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.474+0000 7f259be00640 1 --2- 172.21.15.19:0/1334227718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0b4890 0x7f259c140a00 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f258800c7f0 tx=0x7f258800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:46.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.474+0000 7f2598c00640 1 -- 172.21.15.19:0/1334227718 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588015070 con 0x7f259c0b4890 2024-03-28T13:21:46.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.474+0000 7f2598c00640 1 -- 172.21.15.19:0/1334227718 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2588004240 con 0x7f259c0b4890 2024-03-28T13:21:46.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.474+0000 7f2598c00640 1 -- 172.21.15.19:0/1334227718 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588007620 con 0x7f259c0b4890 2024-03-28T13:21:46.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.474+0000 7f25a2600640 1 -- 172.21.15.19:0/1334227718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259c0cfb30 con 0x7f259c0b4890 2024-03-28T13:21:46.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.474+0000 7f25a2600640 1 -- 172.21.15.19:0/1334227718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f259c153980 con 0x7f259c0b4890 2024-03-28T13:21:46.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.476+0000 7f25a2600640 1 -- 172.21.15.19:0/1334227718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f259c075940 con 0x7f259c0b4890 2024-03-28T13:21:46.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.477+0000 7f2598c00640 1 -- 172.21.15.19:0/1334227718 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f25880079d0 con 0x7f259c0b4890 2024-03-28T13:21:46.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.478+0000 7f2598c00640 1 --2- 172.21.15.19:0/1334227718 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f257c06c530 0x7f257c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:46.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.478+0000 7f2598c00640 1 -- 172.21.15.19:0/1334227718 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6025+0+0 (secure 0 0 0) 0x7f258808bce0 con 0x7f259c0b4890 2024-03-28T13:21:46.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.479+0000 7f25a1200640 1 --2- 172.21.15.19:0/1334227718 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f257c06c530 0x7f257c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:46.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.480+0000 7f25a1200640 1 --2- 172.21.15.19:0/1334227718 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f257c06c530 0x7f257c06e9f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f258c036000 tx=0x7f258c03c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:46.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.483+0000 7f2598c00640 1 -- 172.21.15.19:0/1334227718 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f258805a120 con 0x7f259c0b4890 2024-03-28T13:21:46.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:46.717+0000 7f25a2600640 1 -- 172.21.15.19:0/1334227718 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f259c073120 con 0x7f259c0b4890 2024-03-28T13:21:47.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.257+0000 7f2598c00640 1 -- 172.21.15.19:0/1334227718 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v114) ==== 104+0+0 (secure 0 0 0) 0x7f2588059a20 con 0x7f259c0b4890 2024-03-28T13:21:47.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.261+0000 7f25a2600640 1 -- 172.21.15.19:0/1334227718 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f257c06c530 msgr2=0x7f257c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:47.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.261+0000 7f25a2600640 1 --2- 172.21.15.19:0/1334227718 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f257c06c530 0x7f257c06e9f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f258c036000 tx=0x7f258c03c040 comp rx=0 tx=0).stop 2024-03-28T13:21:47.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.261+0000 7f25a2600640 1 -- 172.21.15.19:0/1334227718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0b4890 msgr2=0x7f259c140a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:47.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.261+0000 7f25a2600640 1 --2- 172.21.15.19:0/1334227718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0b4890 0x7f259c140a00 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f258800c7f0 tx=0x7f258800ccb0 comp rx=0 tx=0).stop 2024-03-28T13:21:47.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.261+0000 7f25a2600640 1 -- 172.21.15.19:0/1334227718 shutdown_connections 2024-03-28T13:21:47.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.261+0000 7f25a2600640 1 --2- 172.21.15.19:0/1334227718 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f257c06c530 0x7f257c06e9f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:47.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.261+0000 7f25a2600640 1 --2- 172.21.15.19:0/1334227718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f259c0cf5e0 0x7f259c0b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:47.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.261+0000 7f25a2600640 1 --2- 172.21.15.19:0/1334227718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c0b4890 0x7f259c140a00 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:47.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.261+0000 7f25a2600640 1 --2- 172.21.15.19:0/1334227718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f259c06d620 0x7f259c1525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:47.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.261+0000 7f25a2600640 1 -- 172.21.15.19:0/1334227718 >> 172.21.15.19:0/1334227718 conn(0x7f259c0b1ba0 msgr2=0x7f259c0b3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:47.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.261+0000 7f25a2600640 1 -- 172.21.15.19:0/1334227718 shutdown_connections 2024-03-28T13:21:47.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.261+0000 7f25a2600640 1 -- 172.21.15.19:0/1334227718 wait complete. 2024-03-28T13:21:47.284 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-28T13:21:47.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.508+0000 7f720be00640 1 Processor -- start 2024-03-28T13:21:47.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.508+0000 7f720be00640 1 -- start start 2024-03-28T13:21:47.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.509+0000 7f720be00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f720c13f190 0x7f720c0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:47.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.509+0000 7f720be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f720c141fa0 0x7f720c06db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:47.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.509+0000 7f720be00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f720c0cfbf0 con 0x7f720c0ced40 2024-03-28T13:21:47.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.509+0000 7f720be00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f720c13f570 con 0x7f720c13f190 2024-03-28T13:21:47.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.509+0000 7f720be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f720c0cf350 con 0x7f720c141fa0 2024-03-28T13:21:47.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.509+0000 7f720a000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f720c0ced40 0x7f720c141890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34264/0 (socket says 172.21.15.19:34264) 2024-03-28T13:21:47.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.509+0000 7f720a000640 1 -- 172.21.15.19:0/678008092 learned_addr learned my addr 172.21.15.19:0/678008092 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:47.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.510+0000 7f720aa00640 1 --2- 172.21.15.19:0/678008092 >> v2:172.21.15.188:3300/0 conn(0x7f720c13f190 0x7f720c0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:47.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.510+0000 7f720b400640 1 --2- 172.21.15.19:0/678008092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f720c141fa0 0x7f720c06db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:47.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.510+0000 7f7209600640 1 -- 172.21.15.19:0/678008092 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1102622048 0 0) 0x7f720c0cfbf0 con 0x7f720c0ced40 2024-03-28T13:21:47.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.510+0000 7f7209600640 1 -- 172.21.15.19:0/678008092 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71f0003520 con 0x7f720c0ced40 2024-03-28T13:21:47.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.510+0000 7f7209600640 1 -- 172.21.15.19:0/678008092 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1720910526 0 0) 0x7f71f0003520 con 0x7f720c0ced40 2024-03-28T13:21:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.510+0000 7f7209600640 1 -- 172.21.15.19:0/678008092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f720c141fa0 msgr2=0x7f720c06db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.510+0000 7f7209600640 1 --2- 172.21.15.19:0/678008092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f720c141fa0 0x7f720c06db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.510+0000 7f7209600640 1 -- 172.21.15.19:0/678008092 >> v2:172.21.15.188:3300/0 conn(0x7f720c13f190 msgr2=0x7f720c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.510+0000 7f7209600640 1 --2- 172.21.15.19:0/678008092 >> v2:172.21.15.188:3300/0 conn(0x7f720c13f190 0x7f720c0ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.510+0000 7f7209600640 1 -- 172.21.15.19:0/678008092 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c0cf630 con 0x7f720c0ced40 2024-03-28T13:21:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.510+0000 7f7209600640 1 -- 172.21.15.19:0/678008092 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f71f4001690 con 0x7f720c0ced40 2024-03-28T13:21:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.510+0000 7f7209600640 1 -- 172.21.15.19:0/678008092 >> v1:172.21.15.19:6789/0 conn(0x7f720c0ced40 legacy=0x7f720c141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.510+0000 7f7209600640 1 --2- 172.21.15.19:0/678008092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71f00039b0 0x7f71f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.511+0000 7f7209600640 1 --2- 172.21.15.19:0/678008092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71f0004800 0x7f71f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.511+0000 7f7209600640 1 --2- 172.21.15.19:0/678008092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f720c141fa0 0x7f71f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.511+0000 7f7209600640 1 -- 172.21.15.19:0/678008092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f720c0cf350 con 0x7f71f0004800 2024-03-28T13:21:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.511+0000 7f7209600640 1 -- 172.21.15.19:0/678008092 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71f000b9c0 con 0x7f71f00039b0 2024-03-28T13:21:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.511+0000 7f7209600640 1 -- 172.21.15.19:0/678008092 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f71f000bb40 con 0x7f720c141fa0 2024-03-28T13:21:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.511+0000 7f720a000640 1 --2- 172.21.15.19:0/678008092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71f0004800 0x7f71f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:47.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.511+0000 7f720aa00640 1 --2- 172.21.15.19:0/678008092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71f00039b0 0x7f71f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:47.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.511+0000 7f720b400640 1 --2- 172.21.15.19:0/678008092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f720c141fa0 0x7f71f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:47.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.511+0000 7f720a000640 1 -- 172.21.15.19:0/678008092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f720c141fa0 msgr2=0x7f71f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:47.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.511+0000 7f720a000640 1 --2- 172.21.15.19:0/678008092 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f720c141fa0 0x7f71f00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:47.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.511+0000 7f720a000640 1 -- 172.21.15.19:0/678008092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71f00039b0 msgr2=0x7f71f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:47.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.511+0000 7f720a000640 1 --2- 172.21.15.19:0/678008092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71f00039b0 0x7f71f0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:47.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.511+0000 7f720a000640 1 -- 172.21.15.19:0/678008092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f0003520 con 0x7f71f0004800 2024-03-28T13:21:47.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.511+0000 7f720aa00640 1 --2- 172.21.15.19:0/678008092 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71f00039b0 0x7f71f0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:47.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.512+0000 7f720a000640 1 --2- 172.21.15.19:0/678008092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71f0004800 0x7f71f0004c80 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f71f4007fd0 tx=0x7f71f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807f1c2db02bbd00 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:47.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.513+0000 7f7209600640 1 -- 172.21.15.19:0/678008092 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4007ca0 con 0x7f71f0004800 2024-03-28T13:21:47.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.513+0000 7f7209600640 1 -- 172.21.15.19:0/678008092 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71f4030070 con 0x7f71f0004800 2024-03-28T13:21:47.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.513+0000 7f7209600640 1 -- 172.21.15.19:0/678008092 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4005d10 con 0x7f71f0004800 2024-03-28T13:21:47.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.513+0000 7f720be00640 1 -- 172.21.15.19:0/678008092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71f0004800 msgr2=0x7f71f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:47.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.513+0000 7f720be00640 1 --2- 172.21.15.19:0/678008092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71f0004800 0x7f71f0004c80 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f71f4007fd0 tx=0x7f71f0005830 comp rx=0 tx=0).stop 2024-03-28T13:21:47.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.513+0000 7f720b400640 1 -- 172.21.15.19:0/678008092 reap_dead start 2024-03-28T13:21:47.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.513+0000 7f720be00640 1 -- 172.21.15.19:0/678008092 shutdown_connections 2024-03-28T13:21:47.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.513+0000 7f720be00640 1 -- 172.21.15.19:0/678008092 >> 172.21.15.19:0/678008092 conn(0x7f720c0b1d80 msgr2=0x7f720c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:47.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.513+0000 7f720be00640 1 -- 172.21.15.19:0/678008092 shutdown_connections 2024-03-28T13:21:47.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.513+0000 7f720be00640 1 -- 172.21.15.19:0/678008092 wait complete. 2024-03-28T13:21:47.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.515+0000 7f720be00640 1 Processor -- start 2024-03-28T13:21:47.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.515+0000 7f720be00640 1 -- start start 2024-03-28T13:21:47.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.516+0000 7f720be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71f0004800 0x7f720c151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:47.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.516+0000 7f720be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f720c13a370 0x7f720c1503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:47.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.516+0000 7f720be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c13a7f0 0x7f720c1405c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:47.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.516+0000 7f720be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f720c142550 con 0x7f720c13a7f0 2024-03-28T13:21:47.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.516+0000 7f720be00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f720c06d670 con 0x7f720c13a370 2024-03-28T13:21:47.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.516+0000 7f720be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f720c1426d0 con 0x7f71f0004800 2024-03-28T13:21:47.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.517+0000 7f720b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c13a7f0 0x7f720c1405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:47.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.517+0000 7f720b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c13a7f0 0x7f720c1405c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44332/0 (socket says 172.21.15.19:44332) 2024-03-28T13:21:47.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.517+0000 7f720b400640 1 -- 172.21.15.19:0/412234804 learned_addr learned my addr 172.21.15.19:0/412234804 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:47.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.517+0000 7f720aa00640 1 --2- 172.21.15.19:0/412234804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71f0004800 0x7f720c151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:47.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.517+0000 7f720a000640 1 --2- 172.21.15.19:0/412234804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f720c13a370 0x7f720c1503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:47.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.517+0000 7f720b400640 1 -- 172.21.15.19:0/412234804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71f0004800 msgr2=0x7f720c151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:47.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.517+0000 7f720b400640 1 --2- 172.21.15.19:0/412234804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71f0004800 0x7f720c151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:47.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.517+0000 7f720b400640 1 -- 172.21.15.19:0/412234804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f720c13a370 msgr2=0x7f720c1503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:47.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.517+0000 7f720b400640 1 --2- 172.21.15.19:0/412234804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f720c13a370 0x7f720c1503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:47.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.517+0000 7f720b400640 1 -- 172.21.15.19:0/412234804 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720c150d70 con 0x7f720c13a7f0 2024-03-28T13:21:47.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.517+0000 7f720b400640 1 --2- 172.21.15.19:0/412234804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c13a7f0 0x7f720c1405c0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f720c142930 tx=0x7f7204010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:47.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.518+0000 7f7202a00640 1 -- 172.21.15.19:0/412234804 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720400f040 con 0x7f720c13a7f0 2024-03-28T13:21:47.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.518+0000 7f7202a00640 1 -- 172.21.15.19:0/412234804 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f720401c070 con 0x7f720c13a7f0 2024-03-28T13:21:47.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.518+0000 7f7202a00640 1 -- 172.21.15.19:0/412234804 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720400eb00 con 0x7f720c13a7f0 2024-03-28T13:21:47.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.518+0000 7f720be00640 1 -- 172.21.15.19:0/412234804 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720c151050 con 0x7f720c13a7f0 2024-03-28T13:21:47.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.518+0000 7f720be00640 1 -- 172.21.15.19:0/412234804 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f720c151610 con 0x7f720c13a7f0 2024-03-28T13:21:47.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.519+0000 7f720be00640 1 -- 172.21.15.19:0/412234804 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f720c075940 con 0x7f720c13a7f0 2024-03-28T13:21:47.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.520+0000 7f7202a00640 1 -- 172.21.15.19:0/412234804 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f720400eca0 con 0x7f720c13a7f0 2024-03-28T13:21:47.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.522+0000 7f7202a00640 1 --2- 172.21.15.19:0/412234804 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f71e806c530 0x7f71e806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:47.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.522+0000 7f7202a00640 1 -- 172.21.15.19:0/412234804 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6025+0+0 (secure 0 0 0) 0x7f720408b3c0 con 0x7f720c13a7f0 2024-03-28T13:21:47.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.522+0000 7f720aa00640 1 --2- 172.21.15.19:0/412234804 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f71e806c530 0x7f71e806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:47.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.523+0000 7f720aa00640 1 --2- 172.21.15.19:0/412234804 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f71e806c530 0x7f71e806e9f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f71fc003be0 tx=0x7f71fc00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:47.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.528+0000 7f7202a00640 1 -- 172.21.15.19:0/412234804 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7204026020 con 0x7f720c13a7f0 2024-03-28T13:21:47.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:47.732+0000 7f720be00640 1 -- 172.21.15.19:0/412234804 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f720c0b1910 con 0x7f720c13a7f0 2024-03-28T13:21:48.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.387+0000 7f7202a00640 1 -- 172.21.15.19:0/412234804 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) ==== 181+0+0 (secure 0 0 0) 0x7f72040591b0 con 0x7f720c13a7f0 2024-03-28T13:21:48.386 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-03-28T13:21:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.391+0000 7f720be00640 1 -- 172.21.15.19:0/412234804 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f71e806c530 msgr2=0x7f71e806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.391+0000 7f720be00640 1 --2- 172.21.15.19:0/412234804 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f71e806c530 0x7f71e806e9f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f71fc003be0 tx=0x7f71fc00b040 comp rx=0 tx=0).stop 2024-03-28T13:21:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.391+0000 7f720be00640 1 -- 172.21.15.19:0/412234804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c13a7f0 msgr2=0x7f720c1405c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.391+0000 7f720be00640 1 --2- 172.21.15.19:0/412234804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c13a7f0 0x7f720c1405c0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f720c142930 tx=0x7f7204010c20 comp rx=0 tx=0).stop 2024-03-28T13:21:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.391+0000 7f720be00640 1 -- 172.21.15.19:0/412234804 shutdown_connections 2024-03-28T13:21:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.391+0000 7f720be00640 1 --2- 172.21.15.19:0/412234804 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f71e806c530 0x7f71e806e9f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.391+0000 7f720be00640 1 --2- 172.21.15.19:0/412234804 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f720c13a7f0 0x7f720c1405c0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.391+0000 7f720be00640 1 --2- 172.21.15.19:0/412234804 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f720c13a370 0x7f720c1503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:48.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.391+0000 7f720be00640 1 --2- 172.21.15.19:0/412234804 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71f0004800 0x7f720c151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.391+0000 7f720be00640 1 -- 172.21.15.19:0/412234804 >> 172.21.15.19:0/412234804 conn(0x7f720c0b1d80 msgr2=0x7f720c152fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.391+0000 7f720be00640 1 -- 172.21.15.19:0/412234804 shutdown_connections 2024-03-28T13:21:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.391+0000 7f720be00640 1 -- 172.21.15.19:0/412234804 wait complete. 2024-03-28T13:21:48.415 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-28T13:21:48.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.651+0000 7f0a77600640 1 Processor -- start 2024-03-28T13:21:48.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.651+0000 7f0a77600640 1 -- start start 2024-03-28T13:21:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.653+0000 7f0a77600640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0a700acf10 0x7f0a7009dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.653+0000 7f0a77600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0a70071710 0x7f0a70071b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.653+0000 7f0a77600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a7015cb90 con 0x7f0a700ad2f0 2024-03-28T13:21:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.653+0000 7f0a77600640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f0a70145f60 con 0x7f0a700acf10 2024-03-28T13:21:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.653+0000 7f0a77600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f0a7015c950 con 0x7f0a70071710 2024-03-28T13:21:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.653+0000 7f0a74c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0a700acf10 0x7f0a7009dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.653+0000 7f0a74c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f0a700acf10 0x7f0a7009dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44348/0 (socket says 172.21.15.19:44348) 2024-03-28T13:21:48.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.653+0000 7f0a74c00640 1 -- 172.21.15.19:0/1541364002 learned_addr learned my addr 172.21.15.19:0/1541364002 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.654+0000 7f0a75600640 1 --2- 172.21.15.19:0/1541364002 >> v2:172.21.15.188:3300/0 conn(0x7f0a70071710 0x7f0a70071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.654+0000 7f0a74c00640 1 -- 172.21.15.19:0/1541364002 >> v2:172.21.15.188:3300/0 conn(0x7f0a70071710 msgr2=0x7f0a70071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.654+0000 7f0a74c00640 1 --2- 172.21.15.19:0/1541364002 >> v2:172.21.15.188:3300/0 conn(0x7f0a70071710 0x7f0a70071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.654+0000 7f0a74c00640 1 -- 172.21.15.19:0/1541364002 >> v1:172.21.15.188:6789/0 conn(0x7f0a700ad2f0 legacy=0x7f0a701453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:48.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.654+0000 7f0a74c00640 1 -- 172.21.15.19:0/1541364002 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7015ce70 con 0x7f0a700acf10 2024-03-28T13:21:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.655+0000 7f0a74c00640 1 --2- 172.21.15.19:0/1541364002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a700acf10 0x7f0a7009dcf0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f0a700aeb70 tx=0x7f0a6002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252f60fffe55b371 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.655+0000 7f0a6f400640 1 -- 172.21.15.19:0/1541364002 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6002e070 con 0x7f0a700acf10 2024-03-28T13:21:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.655+0000 7f0a6f400640 1 -- 172.21.15.19:0/1541364002 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a60037070 con 0x7f0a700acf10 2024-03-28T13:21:48.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.655+0000 7f0a6f400640 1 -- 172.21.15.19:0/1541364002 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60031a00 con 0x7f0a700acf10 2024-03-28T13:21:48.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.656+0000 7f0a77600640 1 -- 172.21.15.19:0/1541364002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a700acf10 msgr2=0x7f0a7009dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:48.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.656+0000 7f0a77600640 1 --2- 172.21.15.19:0/1541364002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a700acf10 0x7f0a7009dcf0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f0a700aeb70 tx=0x7f0a6002b880 comp rx=0 tx=0).stop 2024-03-28T13:21:48.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.656+0000 7f0a77600640 1 -- 172.21.15.19:0/1541364002 shutdown_connections 2024-03-28T13:21:48.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.656+0000 7f0a77600640 1 --2- 172.21.15.19:0/1541364002 >> v2:172.21.15.188:3300/0 conn(0x7f0a70071710 0x7f0a70071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:48.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.656+0000 7f0a77600640 1 --2- 172.21.15.19:0/1541364002 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a700acf10 0x7f0a7009dcf0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:48.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.656+0000 7f0a77600640 1 -- 172.21.15.19:0/1541364002 >> 172.21.15.19:0/1541364002 conn(0x7f0a7006bba0 msgr2=0x7f0a700b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:48.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.656+0000 7f0a77600640 1 -- 172.21.15.19:0/1541364002 shutdown_connections 2024-03-28T13:21:48.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.656+0000 7f0a77600640 1 -- 172.21.15.19:0/1541364002 wait complete. 2024-03-28T13:21:48.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.658+0000 7f0a77600640 1 Processor -- start 2024-03-28T13:21:48.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.658+0000 7f0a77600640 1 -- start start 2024-03-28T13:21:48.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.659+0000 7f0a77600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a70071710 0x7f0a7006e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:48.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.659+0000 7f0a77600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a700acf10 0x7f0a7015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.659+0000 7f0a77600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a700ad2f0 0x7f0a701f8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.659+0000 7f0a77600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7015d430 con 0x7f0a700ad2f0 2024-03-28T13:21:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.659+0000 7f0a77600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7015d7a0 con 0x7f0a700acf10 2024-03-28T13:21:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.659+0000 7f0a77600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0a700ad7a0 con 0x7f0a70071710 2024-03-28T13:21:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.659+0000 7f0a75600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a700ad2f0 0x7f0a701f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.659+0000 7f0a75600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a700ad2f0 0x7f0a701f8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44358/0 (socket says 172.21.15.19:44358) 2024-03-28T13:21:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.659+0000 7f0a75600640 1 -- 172.21.15.19:0/1707212173 learned_addr learned my addr 172.21.15.19:0/1707212173 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.660+0000 7f0a74c00640 1 --2- 172.21.15.19:0/1707212173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a70071710 0x7f0a7006e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.660+0000 7f0a6fe00640 1 --2- 172.21.15.19:0/1707212173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a700acf10 0x7f0a7015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.660+0000 7f0a75600640 1 -- 172.21.15.19:0/1707212173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a70071710 msgr2=0x7f0a7006e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.660+0000 7f0a75600640 1 --2- 172.21.15.19:0/1707212173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a70071710 0x7f0a7006e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.660+0000 7f0a75600640 1 -- 172.21.15.19:0/1707212173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a700acf10 msgr2=0x7f0a7015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:48.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.660+0000 7f0a75600640 1 --2- 172.21.15.19:0/1707212173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a700acf10 0x7f0a7015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:48.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.660+0000 7f0a75600640 1 -- 172.21.15.19:0/1707212173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7006f420 con 0x7f0a700ad2f0 2024-03-28T13:21:48.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.660+0000 7f0a75600640 1 --2- 172.21.15.19:0/1707212173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a700ad2f0 0x7f0a701f8390 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800cd70 tx=0x7f0a6800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:48.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.660+0000 7f0a6cc00640 1 -- 172.21.15.19:0/1707212173 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68003b90 con 0x7f0a700ad2f0 2024-03-28T13:21:48.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.660+0000 7f0a6cc00640 1 -- 172.21.15.19:0/1707212173 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a68003d30 con 0x7f0a700ad2f0 2024-03-28T13:21:48.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.660+0000 7f0a6cc00640 1 -- 172.21.15.19:0/1707212173 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68004240 con 0x7f0a700ad2f0 2024-03-28T13:21:48.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.661+0000 7f0a77600640 1 -- 172.21.15.19:0/1707212173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a701e8650 con 0x7f0a700ad2f0 2024-03-28T13:21:48.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.661+0000 7f0a77600640 1 -- 172.21.15.19:0/1707212173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a701e8c10 con 0x7f0a700ad2f0 2024-03-28T13:21:48.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.662+0000 7f0a77600640 1 -- 172.21.15.19:0/1707212173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a700685c0 con 0x7f0a700ad2f0 2024-03-28T13:21:48.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.663+0000 7f0a6cc00640 1 -- 172.21.15.19:0/1707212173 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f0a680043e0 con 0x7f0a700ad2f0 2024-03-28T13:21:48.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.664+0000 7f0a6cc00640 1 --2- 172.21.15.19:0/1707212173 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0a5006c530 0x7f0a5006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:48.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.664+0000 7f0a6cc00640 1 -- 172.21.15.19:0/1707212173 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5558+0+0 (secure 0 0 0) 0x7f0a6808c180 con 0x7f0a700ad2f0 2024-03-28T13:21:48.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.664+0000 7f0a74c00640 1 --2- 172.21.15.19:0/1707212173 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0a5006c530 0x7f0a5006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:48.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.666+0000 7f0a74c00640 1 --2- 172.21.15.19:0/1707212173 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0a5006c530 0x7f0a5006e9f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0a60036000 tx=0x7f0a6003d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:48.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.668+0000 7f0a6cc00640 1 -- 172.21.15.19:0/1707212173 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f0a6805a790 con 0x7f0a700ad2f0 2024-03-28T13:21:48.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:48.880+0000 7f0a77600640 1 -- 172.21.15.19:0/1707212173 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0a700b3030 con 0x7f0a700ad2f0 2024-03-28T13:21:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.391+0000 7f0a6cc00640 1 -- 172.21.15.19:0/1707212173 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) ==== 169+0+0 (secure 0 0 0) 0x7f0a6805a090 con 0x7f0a700ad2f0 2024-03-28T13:21:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-03-28T13:21:49.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.394+0000 7f0a77600640 1 -- 172.21.15.19:0/1707212173 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0a5006c530 msgr2=0x7f0a5006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:49.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.394+0000 7f0a77600640 1 --2- 172.21.15.19:0/1707212173 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0a5006c530 0x7f0a5006e9f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0a60036000 tx=0x7f0a6003d040 comp rx=0 tx=0).stop 2024-03-28T13:21:49.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.394+0000 7f0a77600640 1 -- 172.21.15.19:0/1707212173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a700ad2f0 msgr2=0x7f0a701f8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:49.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.394+0000 7f0a77600640 1 --2- 172.21.15.19:0/1707212173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a700ad2f0 0x7f0a701f8390 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800cd70 tx=0x7f0a6800a660 comp rx=0 tx=0).stop 2024-03-28T13:21:49.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.394+0000 7f0a77600640 1 -- 172.21.15.19:0/1707212173 shutdown_connections 2024-03-28T13:21:49.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.394+0000 7f0a77600640 1 --2- 172.21.15.19:0/1707212173 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f0a5006c530 0x7f0a5006e9f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:49.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.394+0000 7f0a77600640 1 --2- 172.21.15.19:0/1707212173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0a700ad2f0 0x7f0a701f8390 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:49.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.394+0000 7f0a77600640 1 --2- 172.21.15.19:0/1707212173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0a700acf10 0x7f0a7015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:49.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.394+0000 7f0a77600640 1 --2- 172.21.15.19:0/1707212173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0a70071710 0x7f0a7006e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:49.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.394+0000 7f0a77600640 1 -- 172.21.15.19:0/1707212173 >> 172.21.15.19:0/1707212173 conn(0x7f0a7006bba0 msgr2=0x7f0a701453f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:49.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.394+0000 7f0a77600640 1 -- 172.21.15.19:0/1707212173 shutdown_connections 2024-03-28T13:21:49.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.395+0000 7f0a77600640 1 -- 172.21.15.19:0/1707212173 wait complete. 2024-03-28T13:21:49.424 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-28T13:21:49.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.634+0000 7f92d2200640 1 Processor -- start 2024-03-28T13:21:49.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.634+0000 7f92d2200640 1 -- start start 2024-03-28T13:21:49.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.636+0000 7f92d2200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f92cc0d2310 0x7f92cc0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:49.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.636+0000 7f92d2200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92cc15e880 con 0x7f92cc15e4a0 2024-03-28T13:21:49.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.636+0000 7f92d2200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92cc0d28d0 con 0x7f92cc0cf530 2024-03-28T13:21:49.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.636+0000 7f92d2200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f92cc0cf9a0 con 0x7f92cc0d2310 2024-03-28T13:21:49.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.636+0000 7f92d0e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f92cc15e4a0 0x7f92cc0b9c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34274/0 (socket says 172.21.15.19:34274) 2024-03-28T13:21:49.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.636+0000 7f92d0e00640 1 -- 172.21.15.19:0/843791631 learned_addr learned my addr 172.21.15.19:0/843791631 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:49.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.637+0000 7f92cbe00640 1 --2- 172.21.15.19:0/843791631 >> v2:172.21.15.188:3300/0 conn(0x7f92cc0d2310 0x7f92cc0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:49.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.637+0000 7f92cb400640 1 -- 172.21.15.19:0/843791631 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3289293383 0 0) 0x7f92cc15e880 con 0x7f92cc15e4a0 2024-03-28T13:21:49.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.637+0000 7f92cb400640 1 -- 172.21.15.19:0/843791631 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92b0003520 con 0x7f92cc15e4a0 2024-03-28T13:21:49.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.637+0000 7f92cb400640 1 -- 172.21.15.19:0/843791631 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 878521696 0 0) 0x7f92b0003520 con 0x7f92cc15e4a0 2024-03-28T13:21:49.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.637+0000 7f92cb400640 1 -- 172.21.15.19:0/843791631 >> v2:172.21.15.188:3300/0 conn(0x7f92cc0d2310 msgr2=0x7f92cc0ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:49.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.637+0000 7f92cb400640 1 --2- 172.21.15.19:0/843791631 >> v2:172.21.15.188:3300/0 conn(0x7f92cc0d2310 0x7f92cc0ceff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:49.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.637+0000 7f92cb400640 1 -- 172.21.15.19:0/843791631 >> v1:172.21.15.188:6789/0 conn(0x7f92cc0cf530 legacy=0x7f92cc0b73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:49.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.637+0000 7f92cb400640 1 -- 172.21.15.19:0/843791631 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92cc0d2bb0 con 0x7f92cc15e4a0 2024-03-28T13:21:49.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.637+0000 7f92cb400640 1 -- 172.21.15.19:0/843791631 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f92bc0019f0 con 0x7f92cc15e4a0 2024-03-28T13:21:49.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.637+0000 7f92cb400640 1 -- 172.21.15.19:0/843791631 >> v1:172.21.15.19:6789/0 conn(0x7f92cc15e4a0 legacy=0x7f92cc0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:49.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.637+0000 7f92cb400640 1 --2- 172.21.15.19:0/843791631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92b0003a00 0x7f92b0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:49.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.638+0000 7f92cb400640 1 --2- 172.21.15.19:0/843791631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92b0004800 0x7f92b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:49.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.638+0000 7f92cb400640 1 --2- 172.21.15.19:0/843791631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92b0005210 0x7f92b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:49.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.638+0000 7f92cb400640 1 -- 172.21.15.19:0/843791631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc0cf9a0 con 0x7f92b0004800 2024-03-28T13:21:49.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.638+0000 7f92cb400640 1 -- 172.21.15.19:0/843791631 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f92b000c140 con 0x7f92b0003a00 2024-03-28T13:21:49.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.638+0000 7f92cb400640 1 -- 172.21.15.19:0/843791631 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f92b000c340 con 0x7f92b0005210 2024-03-28T13:21:49.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.638+0000 7f92cbe00640 1 --2- 172.21.15.19:0/843791631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92b0004800 0x7f92b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:49.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.638+0000 7f92d1800640 1 --2- 172.21.15.19:0/843791631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92b0005210 0x7f92b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:49.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.638+0000 7f92d0e00640 1 --2- 172.21.15.19:0/843791631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92b0003a00 0x7f92b0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:49.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.638+0000 7f92cbe00640 1 -- 172.21.15.19:0/843791631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92b0005210 msgr2=0x7f92b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:49.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.638+0000 7f92cbe00640 1 --2- 172.21.15.19:0/843791631 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92b0005210 0x7f92b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:49.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.638+0000 7f92cbe00640 1 -- 172.21.15.19:0/843791631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92b0003a00 msgr2=0x7f92b0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:49.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.638+0000 7f92cbe00640 1 --2- 172.21.15.19:0/843791631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92b0003a00 0x7f92b0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:49.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.638+0000 7f92cbe00640 1 -- 172.21.15.19:0/843791631 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b0003520 con 0x7f92b0004800 2024-03-28T13:21:49.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.639+0000 7f92d1800640 1 -- 172.21.15.19:0/843791631 reap_dead start 2024-03-28T13:21:49.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.639+0000 7f92d0e00640 1 --2- 172.21.15.19:0/843791631 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92b0003a00 0x7f92b0003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:49.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.640+0000 7f92cbe00640 1 --2- 172.21.15.19:0/843791631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92b0004800 0x7f92b0004c80 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009ac0 tx=0x7f92b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d97d07fdd947711 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:49.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.640+0000 7f92cb400640 1 -- 172.21.15.19:0/843791631 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8039070 con 0x7f92b0004800 2024-03-28T13:21:49.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.640+0000 7f92cb400640 1 -- 172.21.15.19:0/843791631 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b802c6b0 con 0x7f92b0004800 2024-03-28T13:21:49.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.640+0000 7f92cb400640 1 -- 172.21.15.19:0/843791631 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b8009690 con 0x7f92b0004800 2024-03-28T13:21:49.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.640+0000 7f92d2200640 1 -- 172.21.15.19:0/843791631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92b0004800 msgr2=0x7f92b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:49.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.640+0000 7f92d2200640 1 --2- 172.21.15.19:0/843791631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92b0004800 0x7f92b0004c80 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f92b8009ac0 tx=0x7f92b0005830 comp rx=0 tx=0).stop 2024-03-28T13:21:49.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.641+0000 7f92d2200640 1 -- 172.21.15.19:0/843791631 shutdown_connections 2024-03-28T13:21:49.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.641+0000 7f92d2200640 1 --2- 172.21.15.19:0/843791631 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92b0004800 0x7f92b0004c80 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:49.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.641+0000 7f92d2200640 1 -- 172.21.15.19:0/843791631 >> 172.21.15.19:0/843791631 conn(0x7f92cc0676d0 msgr2=0x7f92cc067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:49.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.641+0000 7f92d2200640 1 -- 172.21.15.19:0/843791631 shutdown_connections 2024-03-28T13:21:49.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.641+0000 7f92d2200640 1 -- 172.21.15.19:0/843791631 wait complete. 2024-03-28T13:21:49.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.642+0000 7f92d2200640 1 Processor -- start 2024-03-28T13:21:49.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.643+0000 7f92d2200640 1 -- start start 2024-03-28T13:21:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.643+0000 7f92d2200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92b0004800 0x7f92cc15ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.643+0000 7f92d2200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92cc154350 0x7f92cc0c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92d2200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92cc153150 0x7f92cc1535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92d2200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc0d3170 con 0x7f92cc153150 2024-03-28T13:21:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92d2200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f92cc0d34e0 con 0x7f92cc154350 2024-03-28T13:21:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92d2200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f92cc0d3970 con 0x7f92b0004800 2024-03-28T13:21:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92d1800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92cc153150 0x7f92cc1535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92d1800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92cc153150 0x7f92cc1535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44370/0 (socket says 172.21.15.19:44370) 2024-03-28T13:21:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92d1800640 1 -- 172.21.15.19:0/3438286192 learned_addr learned my addr 172.21.15.19:0/3438286192 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92d0e00640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92b0004800 0x7f92cc15ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92cbe00640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92cc154350 0x7f92cc0c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92d1800640 1 -- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92b0004800 msgr2=0x7f92cc15ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92d1800640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92b0004800 0x7f92cc15ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:49.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92d1800640 1 -- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92cc154350 msgr2=0x7f92cc0c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:49.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92d1800640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92cc154350 0x7f92cc0c9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:49.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92d1800640 1 -- 172.21.15.19:0/3438286192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92cc15f2d0 con 0x7f92cc153150 2024-03-28T13:21:49.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92d0e00640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92b0004800 0x7f92cc15ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:49.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.644+0000 7f92cbe00640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92cc154350 0x7f92cc0c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:49.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.645+0000 7f92d1800640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92cc153150 0x7f92cc1535d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f92c400e1e0 tx=0x7f92c400e6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:49.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.645+0000 7f92c8c00640 1 -- 172.21.15.19:0/3438286192 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c40135f0 con 0x7f92cc153150 2024-03-28T13:21:49.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.645+0000 7f92c8c00640 1 -- 172.21.15.19:0/3438286192 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92c4019070 con 0x7f92cc153150 2024-03-28T13:21:49.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.645+0000 7f92c8c00640 1 -- 172.21.15.19:0/3438286192 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c40139f0 con 0x7f92cc153150 2024-03-28T13:21:49.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.645+0000 7f92d2200640 1 -- 172.21.15.19:0/3438286192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92cc15f5b0 con 0x7f92cc153150 2024-03-28T13:21:49.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.645+0000 7f92d2200640 1 -- 172.21.15.19:0/3438286192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92cc15fb70 con 0x7f92cc153150 2024-03-28T13:21:49.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.646+0000 7f92d2200640 1 -- 172.21.15.19:0/3438286192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92cc15d560 con 0x7f92cc153150 2024-03-28T13:21:49.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.647+0000 7f92c8c00640 1 -- 172.21.15.19:0/3438286192 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f92c4013b90 con 0x7f92cc153150 2024-03-28T13:21:49.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.648+0000 7f92c8c00640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f92ac06c530 0x7f92ac06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:49.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.648+0000 7f92c8c00640 1 -- 172.21.15.19:0/3438286192 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7f92c408b320 con 0x7f92cc153150 2024-03-28T13:21:49.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.649+0000 7f92d0e00640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f92ac06c530 0x7f92ac06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:49.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.650+0000 7f92d0e00640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f92ac06c530 0x7f92ac06e9f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f92cc06a480 tx=0x7f92bc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:49.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.653+0000 7f92c8c00640 1 -- 172.21.15.19:0/3438286192 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f92cc15d560 con 0x7f92cc153150 2024-03-28T13:21:49.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.857+0000 7f92d2200640 1 -- 172.21.15.19:0/3438286192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f92cc073830 con 0x7f92cc153150 2024-03-28T13:21:49.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.864+0000 7f92c8c00640 1 -- 172.21.15.19:0/3438286192 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7f92cc073830 con 0x7f92cc153150 2024-03-28T13:21:49.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.867+0000 7f92d2200640 1 -- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f92ac06c530 msgr2=0x7f92ac06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:49.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.867+0000 7f92d2200640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f92ac06c530 0x7f92ac06e9f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f92cc06a480 tx=0x7f92bc009040 comp rx=0 tx=0).stop 2024-03-28T13:21:49.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.867+0000 7f92d2200640 1 -- 172.21.15.19:0/3438286192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92cc153150 msgr2=0x7f92cc1535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:49.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.867+0000 7f92d2200640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92cc153150 0x7f92cc1535d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f92c400e1e0 tx=0x7f92c400e6a0 comp rx=0 tx=0).stop 2024-03-28T13:21:49.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.867+0000 7f92d2200640 1 -- 172.21.15.19:0/3438286192 shutdown_connections 2024-03-28T13:21:49.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.867+0000 7f92d2200640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f92ac06c530 0x7f92ac06e9f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:49.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.867+0000 7f92d2200640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f92cc153150 0x7f92cc1535d0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:49.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.867+0000 7f92d2200640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92cc154350 0x7f92cc0c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:49.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.867+0000 7f92d2200640 1 --2- 172.21.15.19:0/3438286192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92b0004800 0x7f92cc15ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:49.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.867+0000 7f92d2200640 1 -- 172.21.15.19:0/3438286192 >> 172.21.15.19:0/3438286192 conn(0x7f92cc0676d0 msgr2=0x7f92cc0b52c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:49.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.867+0000 7f92d2200640 1 -- 172.21.15.19:0/3438286192 shutdown_connections 2024-03-28T13:21:49.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:49.867+0000 7f92d2200640 1 -- 172.21.15.19:0/3438286192 wait complete. 2024-03-28T13:21:49.883 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-03-28T13:21:50.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.105+0000 7f55a4600640 1 Processor -- start 2024-03-28T13:21:50.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.105+0000 7f55a4600640 1 -- start start 2024-03-28T13:21:50.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.106+0000 7f55a4600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f559c153580 0x7f559c0d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:50.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.106+0000 7f55a4600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f559c0d07c0 0x7f559c0aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:50.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.107+0000 7f55a4600640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f559c0afab0 con 0x7f559c0af2e0 2024-03-28T13:21:50.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.107+0000 7f55a4600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f559c0d0c00 con 0x7f559c0d07c0 2024-03-28T13:21:50.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.107+0000 7f55a4600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f559c15cc80 con 0x7f559c153580 2024-03-28T13:21:50.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.107+0000 7f55a1200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f559c0d07c0 0x7f559c0aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:50.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.107+0000 7f55a1c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f559c153580 0x7f559c0d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:50.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.108+0000 7f55a1200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f559c0d07c0 0x7f559c0aeda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:53792/0 (socket says 172.21.15.19:53792) 2024-03-28T13:21:50.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.108+0000 7f55a1200640 1 -- 172.21.15.19:0/4216431412 learned_addr learned my addr 172.21.15.19:0/4216431412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:50.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.108+0000 7f55a1200640 1 -- 172.21.15.19:0/4216431412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f559c153580 msgr2=0x7f559c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:50.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.108+0000 7f55a1200640 1 --2- 172.21.15.19:0/4216431412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f559c153580 0x7f559c0d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:50.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.108+0000 7f55a1200640 1 -- 172.21.15.19:0/4216431412 >> v1:172.21.15.188:6789/0 conn(0x7f559c0af2e0 legacy=0x7f559c06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:50.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.108+0000 7f55a1200640 1 -- 172.21.15.19:0/4216431412 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c15da00 con 0x7f559c0d07c0 2024-03-28T13:21:50.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.108+0000 7f55a1c00640 1 --2- 172.21.15.19:0/4216431412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f559c153580 0x7f559c0d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:21:50.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.110+0000 7f55a1200640 1 --2- 172.21.15.19:0/4216431412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559c0d07c0 0x7f559c0aeda0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5588009870 tx=0x7f558802b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9d42e3e8b02c97e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:50.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.111+0000 7f559be00640 1 -- 172.21.15.19:0/4216431412 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588035020 con 0x7f559c0d07c0 2024-03-28T13:21:50.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.111+0000 7f559be00640 1 -- 172.21.15.19:0/4216431412 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5588031870 con 0x7f559c0d07c0 2024-03-28T13:21:50.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.111+0000 7f559be00640 1 -- 172.21.15.19:0/4216431412 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5588031c40 con 0x7f559c0d07c0 2024-03-28T13:21:50.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.111+0000 7f55a4600640 1 -- 172.21.15.19:0/4216431412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559c0d07c0 msgr2=0x7f559c0aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:50.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.111+0000 7f55a4600640 1 --2- 172.21.15.19:0/4216431412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559c0d07c0 0x7f559c0aeda0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5588009870 tx=0x7f558802b910 comp rx=0 tx=0).stop 2024-03-28T13:21:50.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.112+0000 7f55a4600640 1 -- 172.21.15.19:0/4216431412 shutdown_connections 2024-03-28T13:21:50.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.112+0000 7f55a4600640 1 --2- 172.21.15.19:0/4216431412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559c0d07c0 0x7f559c0aeda0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:50.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.112+0000 7f55a4600640 1 --2- 172.21.15.19:0/4216431412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f559c153580 0x7f559c0d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:50.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.112+0000 7f55a4600640 1 -- 172.21.15.19:0/4216431412 >> 172.21.15.19:0/4216431412 conn(0x7f559c06b750 msgr2=0x7f559c06bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:50.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.112+0000 7f55a4600640 1 -- 172.21.15.19:0/4216431412 shutdown_connections 2024-03-28T13:21:50.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.112+0000 7f55a4600640 1 -- 172.21.15.19:0/4216431412 wait complete. 2024-03-28T13:21:50.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.113+0000 7f55a4600640 1 Processor -- start 2024-03-28T13:21:50.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.114+0000 7f55a4600640 1 -- start start 2024-03-28T13:21:50.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.115+0000 7f55a4600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f559c0af2e0 0x7f559c149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:50.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.115+0000 7f55a4600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559c0d07c0 0x7f559c0cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:50.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.115+0000 7f55a4600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f559c153580 0x7f559c0b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:50.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.115+0000 7f55a4600640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f559c15d4e0 con 0x7f559c153580 2024-03-28T13:21:50.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.115+0000 7f55a4600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f559c15d200 con 0x7f559c0d07c0 2024-03-28T13:21:50.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.115+0000 7f55a4600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f559c153960 con 0x7f559c0af2e0 2024-03-28T13:21:50.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.115+0000 7f55a2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f559c153580 0x7f559c0b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:50.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.115+0000 7f55a2600640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f559c153580 0x7f559c0b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44382/0 (socket says 172.21.15.19:44382) 2024-03-28T13:21:50.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.115+0000 7f55a2600640 1 -- 172.21.15.19:0/3243247980 learned_addr learned my addr 172.21.15.19:0/3243247980 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:50.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.115+0000 7f55a1200640 1 --2- 172.21.15.19:0/3243247980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559c0d07c0 0x7f559c0cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:50.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.115+0000 7f55a1c00640 1 --2- 172.21.15.19:0/3243247980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f559c0af2e0 0x7f559c149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.116+0000 7f55a2600640 1 -- 172.21.15.19:0/3243247980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f559c0af2e0 msgr2=0x7f559c149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.116+0000 7f55a2600640 1 --2- 172.21.15.19:0/3243247980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f559c0af2e0 0x7f559c149e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.116+0000 7f55a2600640 1 -- 172.21.15.19:0/3243247980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559c0d07c0 msgr2=0x7f559c0cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.116+0000 7f55a2600640 1 --2- 172.21.15.19:0/3243247980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559c0d07c0 0x7f559c0cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.116+0000 7f55a2600640 1 -- 172.21.15.19:0/3243247980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c0cbdd0 con 0x7f559c153580 2024-03-28T13:21:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.116+0000 7f55a1c00640 1 --2- 172.21.15.19:0/3243247980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f559c0af2e0 0x7f559c149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:21:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.116+0000 7f55a2600640 1 --2- 172.21.15.19:0/3243247980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f559c153580 0x7f559c0b0420 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f559400c6b0 tx=0x7f559400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.116+0000 7f5599600640 1 -- 172.21.15.19:0/3243247980 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594003b70 con 0x7f559c153580 2024-03-28T13:21:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.116+0000 7f5599600640 1 -- 172.21.15.19:0/3243247980 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5594003d30 con 0x7f559c153580 2024-03-28T13:21:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.116+0000 7f5599600640 1 -- 172.21.15.19:0/3243247980 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594005310 con 0x7f559c153580 2024-03-28T13:21:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.116+0000 7f55a4600640 1 -- 172.21.15.19:0/3243247980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559c0719f0 con 0x7f559c153580 2024-03-28T13:21:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.117+0000 7f55a4600640 1 -- 172.21.15.19:0/3243247980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559c071fb0 con 0x7f559c153580 2024-03-28T13:21:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.118+0000 7f55a4600640 1 -- 172.21.15.19:0/3243247980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559c0685c0 con 0x7f559c153580 2024-03-28T13:21:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.119+0000 7f5599600640 1 -- 172.21.15.19:0/3243247980 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f55940054b0 con 0x7f559c153580 2024-03-28T13:21:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.120+0000 7f5599600640 1 --2- 172.21.15.19:0/3243247980 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f557c06c530 0x7f557c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.120+0000 7f5599600640 1 -- 172.21.15.19:0/3243247980 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7f559408b240 con 0x7f559c153580 2024-03-28T13:21:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.121+0000 7f55a1c00640 1 --2- 172.21.15.19:0/3243247980 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f557c06c530 0x7f557c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:50.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.122+0000 7f55a1c00640 1 --2- 172.21.15.19:0/3243247980 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f557c06c530 0x7f557c06e9f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f559c15bee0 tx=0x7f5590008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:50.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.125+0000 7f5599600640 1 -- 172.21.15.19:0/3243247980 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f5594059ab0 con 0x7f559c153580 2024-03-28T13:21:50.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.403+0000 7f55a4600640 1 -- 172.21.15.19:0/3243247980 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f559c15c520 con 0x7f559c153580 2024-03-28T13:21:50.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.407+0000 7f5599600640 1 -- 172.21.15.19:0/3243247980 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v585) ==== 144+0+0 (secure 0 0 0) 0x7f55940593b0 con 0x7f559c153580 2024-03-28T13:21:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.410+0000 7f55a4600640 1 -- 172.21.15.19:0/3243247980 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f557c06c530 msgr2=0x7f557c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.410+0000 7f55a4600640 1 --2- 172.21.15.19:0/3243247980 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f557c06c530 0x7f557c06e9f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f559c15bee0 tx=0x7f5590008040 comp rx=0 tx=0).stop 2024-03-28T13:21:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.410+0000 7f55a4600640 1 -- 172.21.15.19:0/3243247980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f559c153580 msgr2=0x7f559c0b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.410+0000 7f55a4600640 1 --2- 172.21.15.19:0/3243247980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f559c153580 0x7f559c0b0420 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f559400c6b0 tx=0x7f559400cb70 comp rx=0 tx=0).stop 2024-03-28T13:21:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.410+0000 7f55a4600640 1 -- 172.21.15.19:0/3243247980 shutdown_connections 2024-03-28T13:21:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.410+0000 7f55a4600640 1 --2- 172.21.15.19:0/3243247980 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f557c06c530 0x7f557c06e9f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.410+0000 7f55a4600640 1 --2- 172.21.15.19:0/3243247980 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f559c153580 0x7f559c0b0420 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.410+0000 7f55a4600640 1 --2- 172.21.15.19:0/3243247980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f559c0d07c0 0x7f559c0cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.410+0000 7f55a4600640 1 --2- 172.21.15.19:0/3243247980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f559c0af2e0 0x7f559c149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:50.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.410+0000 7f55a4600640 1 -- 172.21.15.19:0/3243247980 >> 172.21.15.19:0/3243247980 conn(0x7f559c06b750 msgr2=0x7f559c0b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.410+0000 7f55a4600640 1 -- 172.21.15.19:0/3243247980 shutdown_connections 2024-03-28T13:21:50.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:50.410+0000 7f55a4600640 1 -- 172.21.15.19:0/3243247980 wait complete. 2024-03-28T13:21:50.430 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-03-28T13:21:50.431 INFO:tasks.cephfs_test_runner: 2024-03-28T13:21:50.431 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-28T13:21:50.431 INFO:tasks.cephfs_test_runner:Ran 7 tests in 686.627s 2024-03-28T13:21:50.431 INFO:tasks.cephfs_test_runner: 2024-03-28T13:21:50.431 INFO:tasks.cephfs_test_runner:OK 2024-03-28T13:21:50.432 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-28T13:21:50.444 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-28T13:21:50.452 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-28T13:21:50.452 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:50.452 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:50.509 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:21:50.509 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-03-28T13:21:50.509 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:50.562 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054203,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249800k,nr_inodes=812450,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499604k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-28T13:21:50.563 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:21:50.563 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:50.620 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:21:50.620 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-03-28T13:21:50.620 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-28T13:21:50.672 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053418,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249172k,nr_inodes=812293,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498348k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-28T13:21:50.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:50.673 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:21:50.729 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-28T13:21:50.730 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-28T13:21:50.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:50.731 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-28T13:21:50.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:50.842 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-28T13:21:50.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:50.934 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-03-28T13:21:51.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:51.059 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-03-28T13:21:51.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:51.105 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-03-28T13:21:51.242 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-03-28T13:21:51.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T13:21:51.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:51.245 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-03-28T13:21:51.303 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-28T13:21:51.303 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-28T13:21:51.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:51.304 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-28T13:21:51.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:51.398 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-28T13:21:51.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:51.492 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-03-28T13:21:51.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:51.639 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-03-28T13:21:51.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-28T13:21:51.685 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-03-28T13:21:51.824 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2024-03-28T13:21:51.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T13:21:51.828 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-28T13:21:51.837 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-28T13:21:51.837 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:21:52.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.067+0000 7fa2d1a00640 1 Processor -- start 2024-03-28T13:21:52.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.067+0000 7fa2d1a00640 1 -- start start 2024-03-28T13:21:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.068+0000 7fa2d1a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa2cc15e7e0 0x7fa2cc0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.068+0000 7fa2d1a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa2cc06ca00 0x7fa2cc0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.068+0000 7fa2d1a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc0d2d60 0x7fa2cc0cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.068+0000 7fa2d1a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fa2cc15ebc0 con 0x7fa2cc15e7e0 2024-03-28T13:21:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.068+0000 7fa2d1a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa2cc06d1c0 con 0x7fa2cc06ca00 2024-03-28T13:21:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.068+0000 7fa2d1a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa2cc06d4a0 con 0x7fa2cc0d2d60 2024-03-28T13:21:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.068+0000 7fa2caa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa2cc15e7e0 0x7fa2cc0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.069+0000 7fa2caa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fa2cc15e7e0 0x7fa2cc0d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44394/0 (socket says 172.21.15.19:44394) 2024-03-28T13:21:52.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.069+0000 7fa2caa00640 1 -- 172.21.15.19:0/2081308555 learned_addr learned my addr 172.21.15.19:0/2081308555 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:52.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.069+0000 7fa2cb400640 1 --2- 172.21.15.19:0/2081308555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc0d2d60 0x7fa2cc0cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.069+0000 7fa2ca000640 1 --2- 172.21.15.19:0/2081308555 >> v2:172.21.15.188:3300/0 conn(0x7fa2cc06ca00 0x7fa2cc0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.069+0000 7fa2caa00640 1 -- 172.21.15.19:0/2081308555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc0d2d60 msgr2=0x7fa2cc0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:52.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.069+0000 7fa2caa00640 1 --2- 172.21.15.19:0/2081308555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc0d2d60 0x7fa2cc0cfb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.069+0000 7fa2caa00640 1 -- 172.21.15.19:0/2081308555 >> v2:172.21.15.188:3300/0 conn(0x7fa2cc06ca00 msgr2=0x7fa2cc0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:52.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.069+0000 7fa2caa00640 1 --2- 172.21.15.19:0/2081308555 >> v2:172.21.15.188:3300/0 conn(0x7fa2cc06ca00 0x7fa2cc0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.069+0000 7fa2caa00640 1 -- 172.21.15.19:0/2081308555 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2cc0cf320 con 0x7fa2cc15e7e0 2024-03-28T13:21:52.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.069+0000 7fa2cb400640 1 --2- 172.21.15.19:0/2081308555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc0d2d60 0x7fa2cc0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:52.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.070+0000 7fa2caa00640 1 --2- 172.21.15.19:0/2081308555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2cc15e7e0 0x7fa2cc0d1950 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0b6f50 tx=0x7fa2bc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4af7de4885e6f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:52.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.071+0000 7fa2c9600640 1 -- 172.21.15.19:0/2081308555 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc02e070 con 0x7fa2cc15e7e0 2024-03-28T13:21:52.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.071+0000 7fa2c9600640 1 -- 172.21.15.19:0/2081308555 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2bc037070 con 0x7fa2cc15e7e0 2024-03-28T13:21:52.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.071+0000 7fa2c9600640 1 -- 172.21.15.19:0/2081308555 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc031a00 con 0x7fa2cc15e7e0 2024-03-28T13:21:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.071+0000 7fa2d1a00640 1 -- 172.21.15.19:0/2081308555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2cc15e7e0 msgr2=0x7fa2cc0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.071+0000 7fa2d1a00640 1 --2- 172.21.15.19:0/2081308555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2cc15e7e0 0x7fa2cc0d1950 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc0b6f50 tx=0x7fa2bc02b880 comp rx=0 tx=0).stop 2024-03-28T13:21:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.071+0000 7fa2d1a00640 1 -- 172.21.15.19:0/2081308555 shutdown_connections 2024-03-28T13:21:52.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.071+0000 7fa2d1a00640 1 --2- 172.21.15.19:0/2081308555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc0d2d60 0x7fa2cc0cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.071+0000 7fa2d1a00640 1 --2- 172.21.15.19:0/2081308555 >> v2:172.21.15.188:3300/0 conn(0x7fa2cc06ca00 0x7fa2cc0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.071+0000 7fa2d1a00640 1 --2- 172.21.15.19:0/2081308555 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2cc15e7e0 0x7fa2cc0d1950 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.071+0000 7fa2d1a00640 1 -- 172.21.15.19:0/2081308555 >> 172.21.15.19:0/2081308555 conn(0x7fa2cc09d7c0 msgr2=0x7fa2cc06bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:52.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.071+0000 7fa2d1a00640 1 -- 172.21.15.19:0/2081308555 shutdown_connections 2024-03-28T13:21:52.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.071+0000 7fa2d1a00640 1 -- 172.21.15.19:0/2081308555 wait complete. 2024-03-28T13:21:52.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.073+0000 7fa2d1a00640 1 Processor -- start 2024-03-28T13:21:52.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.073+0000 7fa2d1a00640 1 -- start start 2024-03-28T13:21:52.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.074+0000 7fa2d1a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2cc06ca00 0x7fa2cc156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.074+0000 7fa2d1a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2cc0d2d60 0x7fa2cc0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.074+0000 7fa2d1a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc15e7e0 0x7fa2cc0ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.074+0000 7fa2d1a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc06ce60 con 0x7fa2cc0d2d60 2024-03-28T13:21:52.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.074+0000 7fa2d1a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc0cefd0 con 0x7fa2cc06ca00 2024-03-28T13:21:52.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.074+0000 7fa2d1a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa2cc0d0060 con 0x7fa2cc15e7e0 2024-03-28T13:21:52.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.075+0000 7fa2ca000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2cc0d2d60 0x7fa2cc0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.075+0000 7fa2ca000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2cc0d2d60 0x7fa2cc0c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44408/0 (socket says 172.21.15.19:44408) 2024-03-28T13:21:52.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.075+0000 7fa2ca000640 1 -- 172.21.15.19:0/3719443437 learned_addr learned my addr 172.21.15.19:0/3719443437 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:52.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.075+0000 7fa2caa00640 1 --2- 172.21.15.19:0/3719443437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2cc06ca00 0x7fa2cc156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.075+0000 7fa2cb400640 1 --2- 172.21.15.19:0/3719443437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc15e7e0 0x7fa2cc0ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.075+0000 7fa2ca000640 1 -- 172.21.15.19:0/3719443437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc15e7e0 msgr2=0x7fa2cc0ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:52.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.075+0000 7fa2ca000640 1 --2- 172.21.15.19:0/3719443437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc15e7e0 0x7fa2cc0ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.075+0000 7fa2ca000640 1 -- 172.21.15.19:0/3719443437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2cc06ca00 msgr2=0x7fa2cc156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:52.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.075+0000 7fa2ca000640 1 --2- 172.21.15.19:0/3719443437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2cc06ca00 0x7fa2cc156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.075+0000 7fa2ca000640 1 -- 172.21.15.19:0/3719443437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2cc0b9950 con 0x7fa2cc0d2d60 2024-03-28T13:21:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.075+0000 7fa2cb400640 1 --2- 172.21.15.19:0/3719443437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc15e7e0 0x7fa2cc0ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.075+0000 7fa2ca000640 1 --2- 172.21.15.19:0/3719443437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2cc0d2d60 0x7fa2cc0c9c90 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4007ef0 tx=0x7fa2b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.076+0000 7fa2c2a00640 1 -- 172.21.15.19:0/3719443437 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4014620 con 0x7fa2cc0d2d60 2024-03-28T13:21:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.076+0000 7fa2c2a00640 1 -- 172.21.15.19:0/3719443437 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b4019070 con 0x7fa2cc0d2d60 2024-03-28T13:21:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.076+0000 7fa2c2a00640 1 -- 172.21.15.19:0/3719443437 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4014a00 con 0x7fa2cc0d2d60 2024-03-28T13:21:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.076+0000 7fa2d1a00640 1 -- 172.21.15.19:0/3719443437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2cc0b8b90 con 0x7fa2cc0d2d60 2024-03-28T13:21:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.076+0000 7fa2d1a00640 1 -- 172.21.15.19:0/3719443437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2cc0b9150 con 0x7fa2cc0d2d60 2024-03-28T13:21:52.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.077+0000 7fa2d1a00640 1 -- 172.21.15.19:0/3719443437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2cc0685c0 con 0x7fa2cc0d2d60 2024-03-28T13:21:52.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.078+0000 7fa2c2a00640 1 -- 172.21.15.19:0/3719443437 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa2b4014ba0 con 0x7fa2cc0d2d60 2024-03-28T13:21:52.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.079+0000 7fa2c2a00640 1 --2- 172.21.15.19:0/3719443437 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa2a806c530 0x7fa2a806e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.079+0000 7fa2c2a00640 1 -- 172.21.15.19:0/3719443437 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7fa2b408bd90 con 0x7fa2cc0d2d60 2024-03-28T13:21:52.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.080+0000 7fa2caa00640 1 --2- 172.21.15.19:0/3719443437 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa2a806c530 0x7fa2a806e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.081+0000 7fa2caa00640 1 --2- 172.21.15.19:0/3719443437 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa2a806c530 0x7fa2a806e9f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc036000 tx=0x7fa2bc03d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:52.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.084+0000 7fa2c2a00640 1 -- 172.21.15.19:0/3719443437 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa2b405a5f0 con 0x7fa2cc0d2d60 2024-03-28T13:21:52.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.286+0000 7fa2d1a00640 1 -- 172.21.15.19:0/3719443437 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa2cc156ca0 con 0x7fa2a806c530 2024-03-28T13:21:52.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.288+0000 7fa2c2a00640 1 -- 172.21.15.19:0/3719443437 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13521 (secure 0 0 0) 0x7fa2bc037040 con 0x7fa2a806c530 2024-03-28T13:21:52.287 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:52.287 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:21:52.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.291+0000 7fa2d1a00640 1 -- 172.21.15.19:0/3719443437 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa2a806c530 msgr2=0x7fa2a806e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.291+0000 7fa2d1a00640 1 --2- 172.21.15.19:0/3719443437 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa2a806c530 0x7fa2a806e9f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc036000 tx=0x7fa2bc03d040 comp rx=0 tx=0).stop 2024-03-28T13:21:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.291+0000 7fa2d1a00640 1 -- 172.21.15.19:0/3719443437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2cc0d2d60 msgr2=0x7fa2cc0c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.291+0000 7fa2d1a00640 1 --2- 172.21.15.19:0/3719443437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2cc0d2d60 0x7fa2cc0c9c90 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4007ef0 tx=0x7fa2b400e4b0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.291+0000 7fa2d1a00640 1 -- 172.21.15.19:0/3719443437 shutdown_connections 2024-03-28T13:21:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.291+0000 7fa2d1a00640 1 --2- 172.21.15.19:0/3719443437 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa2a806c530 0x7fa2a806e9f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.291+0000 7fa2d1a00640 1 --2- 172.21.15.19:0/3719443437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa2cc15e7e0 0x7fa2cc0ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.291+0000 7fa2d1a00640 1 --2- 172.21.15.19:0/3719443437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa2cc0d2d60 0x7fa2cc0c9c90 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.292+0000 7fa2d1a00640 1 --2- 172.21.15.19:0/3719443437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa2cc06ca00 0x7fa2cc156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.292+0000 7fa2d1a00640 1 -- 172.21.15.19:0/3719443437 >> 172.21.15.19:0/3719443437 conn(0x7fa2cc09d7c0 msgr2=0x7fa2cc0d43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.292+0000 7fa2d1a00640 1 -- 172.21.15.19:0/3719443437 shutdown_connections 2024-03-28T13:21:52.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.292+0000 7fa2d1a00640 1 -- 172.21.15.19:0/3719443437 wait complete. 2024-03-28T13:21:52.310 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-03-28T13:21:52.265529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":279088,"kb_used_data":29644,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695632,"statfs":{"total":773094113280,"available":772808327168,"internally_reserved":0,"allocated":30355456,"data_stored":23359910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392250,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-178,"num_write_kb":-8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.007087"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":364,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-03-28T13:21:49.532212+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:21:49.532212+0000","last_peered":"2024-03-28T13:21:49.532212+0000","last_clean":"2024-03-28T13:21:49.532212+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:21:49.532212+0000","last_undegraded":"2024-03-28T13:21:49.532212+0000","last_fullsized":"2024-03-28T13:21:49.532212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":681169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":681169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2138112,"data_stored":1265783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476888,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36784,"kb_used_data":5604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":5738496,"data_stored":4869719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36784,"kb_used_data":5604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":5738496,"data_stored":4869719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509592,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33272,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338568,"statfs":{"total":96636764160,"available":96602693632,"internally_reserved":0,"allocated":2142208,"data_stored":1265788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509592,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36784,"kb_used_data":5604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":5738496,"data_stored":4866415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542296,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":5604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":5738496,"data_stored":4860148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:21:52.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:21:52.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.524+0000 7f43bea00640 1 Processor -- start 2024-03-28T13:21:52.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.525+0000 7f43bea00640 1 -- start start 2024-03-28T13:21:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.526+0000 7f43bea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b815c390 0x7f43b8073790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.526+0000 7f43bea00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f43b80d0650 0x7f43b80d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.526+0000 7f43bea00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43b815fe80 con 0x7f43b80af830 2024-03-28T13:21:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.526+0000 7f43bea00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f43b80d1000 con 0x7f43b80d0650 2024-03-28T13:21:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.526+0000 7f43bea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f43b815c770 con 0x7f43b815c390 2024-03-28T13:21:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.526+0000 7f43b7400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f43b80af830 0x7f43b8160640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34276/0 (socket says 172.21.15.19:34276) 2024-03-28T13:21:52.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.526+0000 7f43b7400640 1 -- 172.21.15.19:0/1416983061 learned_addr learned my addr 172.21.15.19:0/1416983061 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:52.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43b6a00640 1 -- 172.21.15.19:0/1416983061 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1186777538 0 0) 0x7f43b815fe80 con 0x7f43b80af830 2024-03-28T13:21:52.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43b7e00640 1 --2- 172.21.15.19:0/1416983061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b815c390 0x7f43b8073790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43b6a00640 1 -- 172.21.15.19:0/1416983061 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f439c003520 con 0x7f43b80af830 2024-03-28T13:21:52.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43bca00640 1 --2- 172.21.15.19:0/1416983061 >> v2:172.21.15.188:3300/0 conn(0x7f43b80d0650 0x7f43b80d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43b6a00640 1 -- 172.21.15.19:0/1416983061 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2517058628 0 0) 0x7f439c003520 con 0x7f43b80af830 2024-03-28T13:21:52.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43b6a00640 1 -- 172.21.15.19:0/1416983061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b815c390 msgr2=0x7f43b8073790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:52.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43b6a00640 1 --2- 172.21.15.19:0/1416983061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b815c390 0x7f43b8073790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43b6a00640 1 -- 172.21.15.19:0/1416983061 >> v2:172.21.15.188:3300/0 conn(0x7f43b80d0650 msgr2=0x7f43b80d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:52.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43b6a00640 1 --2- 172.21.15.19:0/1416983061 >> v2:172.21.15.188:3300/0 conn(0x7f43b80d0650 0x7f43b80d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43b6a00640 1 -- 172.21.15.19:0/1416983061 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b80ad120 con 0x7f43b80af830 2024-03-28T13:21:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43b6a00640 1 -- 172.21.15.19:0/1416983061 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f43a40019f0 con 0x7f43b80af830 2024-03-28T13:21:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43b6a00640 1 -- 172.21.15.19:0/1416983061 >> v1:172.21.15.19:6789/0 conn(0x7f43b80af830 legacy=0x7f43b8160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43b6a00640 1 --2- 172.21.15.19:0/1416983061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f439c0039b0 0x7f439c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43b6a00640 1 --2- 172.21.15.19:0/1416983061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f439c004800 0x7f439c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43b6a00640 1 --2- 172.21.15.19:0/1416983061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b815c390 0x7f439c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.527+0000 7f43b6a00640 1 -- 172.21.15.19:0/1416983061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43b80d1000 con 0x7f439c0039b0 2024-03-28T13:21:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.528+0000 7f43b6a00640 1 -- 172.21.15.19:0/1416983061 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f439c00b9f0 con 0x7f439c004800 2024-03-28T13:21:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.528+0000 7f43b6a00640 1 -- 172.21.15.19:0/1416983061 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f439c00bbf0 con 0x7f43b815c390 2024-03-28T13:21:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.528+0000 7f43b7e00640 1 --2- 172.21.15.19:0/1416983061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f439c0039b0 0x7f439c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.528+0000 7f43bca00640 1 --2- 172.21.15.19:0/1416983061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b815c390 0x7f439c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.528+0000 7f43b7400640 1 --2- 172.21.15.19:0/1416983061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f439c004800 0x7f439c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.528+0000 7f43b7e00640 1 -- 172.21.15.19:0/1416983061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b815c390 msgr2=0x7f439c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.528+0000 7f43b7e00640 1 --2- 172.21.15.19:0/1416983061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b815c390 0x7f439c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.528+0000 7f43b7e00640 1 -- 172.21.15.19:0/1416983061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f439c004800 msgr2=0x7f439c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:52.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.528+0000 7f43b7e00640 1 --2- 172.21.15.19:0/1416983061 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f439c004800 0x7f439c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.528+0000 7f43b7e00640 1 -- 172.21.15.19:0/1416983061 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c003520 con 0x7f439c0039b0 2024-03-28T13:21:52.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.528+0000 7f43bca00640 1 --2- 172.21.15.19:0/1416983061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b815c390 0x7f439c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:52.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.529+0000 7f43b7e00640 1 --2- 172.21.15.19:0/1416983061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f439c0039b0 0x7f439c003dd0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f43a800ba00 tx=0x7f439c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff00bef318181be server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:52.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.530+0000 7f43b6a00640 1 -- 172.21.15.19:0/1416983061 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8036020 con 0x7f439c0039b0 2024-03-28T13:21:52.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.530+0000 7f43b6a00640 1 -- 172.21.15.19:0/1416983061 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43a80325f0 con 0x7f439c0039b0 2024-03-28T13:21:52.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.530+0000 7f43b6a00640 1 -- 172.21.15.19:0/1416983061 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a800b600 con 0x7f439c0039b0 2024-03-28T13:21:52.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.530+0000 7f43bea00640 1 -- 172.21.15.19:0/1416983061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f439c0039b0 msgr2=0x7f439c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:52.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.530+0000 7f43bea00640 1 --2- 172.21.15.19:0/1416983061 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f439c0039b0 0x7f439c003dd0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f43a800ba00 tx=0x7f439c005830 comp rx=0 tx=0).stop 2024-03-28T13:21:52.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.530+0000 7f43bca00640 1 -- 172.21.15.19:0/1416983061 reap_dead start 2024-03-28T13:21:52.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.531+0000 7f43bea00640 1 -- 172.21.15.19:0/1416983061 shutdown_connections 2024-03-28T13:21:52.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.531+0000 7f43bea00640 1 -- 172.21.15.19:0/1416983061 >> 172.21.15.19:0/1416983061 conn(0x7f43b806b990 msgr2=0x7f43b806bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:52.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.531+0000 7f43bea00640 1 -- 172.21.15.19:0/1416983061 shutdown_connections 2024-03-28T13:21:52.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.531+0000 7f43bea00640 1 -- 172.21.15.19:0/1416983061 wait complete. 2024-03-28T13:21:52.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.532+0000 7f43bea00640 1 Processor -- start 2024-03-28T13:21:52.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.533+0000 7f43bea00640 1 -- start start 2024-03-28T13:21:52.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43bea00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f439c0039b0 0x7f43b81423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43bea00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b8151500 0x7f43b814faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43bea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43b8154da0 0x7f43b8155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43bea00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8160120 con 0x7f43b8154da0 2024-03-28T13:21:52.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43bea00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f43b80ad6e0 con 0x7f43b8151500 2024-03-28T13:21:52.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43bea00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f43b8160d50 con 0x7f439c0039b0 2024-03-28T13:21:52.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43bca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43b8154da0 0x7f43b8155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43bca00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43b8154da0 0x7f43b8155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44414/0 (socket says 172.21.15.19:44414) 2024-03-28T13:21:52.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43bca00640 1 -- 172.21.15.19:0/3808207728 learned_addr learned my addr 172.21.15.19:0/3808207728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:52.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43b7400640 1 --2- 172.21.15.19:0/3808207728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b8151500 0x7f43b814faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43b7e00640 1 --2- 172.21.15.19:0/3808207728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f439c0039b0 0x7f43b81423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43bca00640 1 -- 172.21.15.19:0/3808207728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f439c0039b0 msgr2=0x7f43b81423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:52.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43bca00640 1 --2- 172.21.15.19:0/3808207728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f439c0039b0 0x7f43b81423b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43bca00640 1 -- 172.21.15.19:0/3808207728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b8151500 msgr2=0x7f43b814faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:52.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43bca00640 1 --2- 172.21.15.19:0/3808207728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b8151500 0x7f43b814faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.534+0000 7f43bca00640 1 -- 172.21.15.19:0/3808207728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b8151d00 con 0x7f43b8154da0 2024-03-28T13:21:52.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.535+0000 7f43bca00640 1 --2- 172.21.15.19:0/3808207728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43b8154da0 0x7f43b8155220 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f43b80b63e0 tx=0x7f43b000e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:52.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.535+0000 7f43afe00640 1 -- 172.21.15.19:0/3808207728 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b000f040 con 0x7f43b8154da0 2024-03-28T13:21:52.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.535+0000 7f43afe00640 1 -- 172.21.15.19:0/3808207728 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b0019070 con 0x7f43b8154da0 2024-03-28T13:21:52.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.535+0000 7f43bea00640 1 -- 172.21.15.19:0/3808207728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b8151fe0 con 0x7f43b8154da0 2024-03-28T13:21:52.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.535+0000 7f43afe00640 1 -- 172.21.15.19:0/3808207728 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b0013850 con 0x7f43b8154da0 2024-03-28T13:21:52.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.535+0000 7f43bea00640 1 -- 172.21.15.19:0/3808207728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b815d410 con 0x7f43b8154da0 2024-03-28T13:21:52.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.536+0000 7f43bea00640 1 -- 172.21.15.19:0/3808207728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b815d610 con 0x7f43b8154da0 2024-03-28T13:21:52.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.537+0000 7f43afe00640 1 -- 172.21.15.19:0/3808207728 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f43b00139f0 con 0x7f43b8154da0 2024-03-28T13:21:52.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.538+0000 7f43afe00640 1 --2- 172.21.15.19:0/3808207728 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f439806c4d0 0x7f439806e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.538+0000 7f43afe00640 1 -- 172.21.15.19:0/3808207728 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7f43b008ad10 con 0x7f43b8154da0 2024-03-28T13:21:52.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.539+0000 7f43b7e00640 1 --2- 172.21.15.19:0/3808207728 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f439806c4d0 0x7f439806e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.540+0000 7f43b7e00640 1 --2- 172.21.15.19:0/3808207728 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f439806c4d0 0x7f439806e990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f439c0057e0 tx=0x7f43a8030000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:52.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.543+0000 7f43afe00640 1 -- 172.21.15.19:0/3808207728 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f43b815d610 con 0x7f43b8154da0 2024-03-28T13:21:52.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.742+0000 7f43bea00640 1 -- 172.21.15.19:0/3808207728 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f43b8152500 con 0x7f439806c4d0 2024-03-28T13:21:52.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.745+0000 7f43afe00640 1 -- 172.21.15.19:0/3808207728 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13521 (secure 0 0 0) 0x7f43b8152500 con 0x7f439806c4d0 2024-03-28T13:21:52.744 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:52.744 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:21:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.747+0000 7f43bea00640 1 -- 172.21.15.19:0/3808207728 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f439806c4d0 msgr2=0x7f439806e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.747+0000 7f43bea00640 1 --2- 172.21.15.19:0/3808207728 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f439806c4d0 0x7f439806e990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f439c0057e0 tx=0x7f43a8030000 comp rx=0 tx=0).stop 2024-03-28T13:21:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.747+0000 7f43bea00640 1 -- 172.21.15.19:0/3808207728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43b8154da0 msgr2=0x7f43b8155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.747+0000 7f43bea00640 1 --2- 172.21.15.19:0/3808207728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43b8154da0 0x7f43b8155220 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f43b80b63e0 tx=0x7f43b000e830 comp rx=0 tx=0).stop 2024-03-28T13:21:52.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.747+0000 7f43bea00640 1 -- 172.21.15.19:0/3808207728 shutdown_connections 2024-03-28T13:21:52.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.747+0000 7f43bea00640 1 --2- 172.21.15.19:0/3808207728 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f439806c4d0 0x7f439806e990 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.747+0000 7f43bea00640 1 --2- 172.21.15.19:0/3808207728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f43b8154da0 0x7f43b8155220 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.747+0000 7f43bea00640 1 --2- 172.21.15.19:0/3808207728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b8151500 0x7f43b814faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.747+0000 7f43bea00640 1 --2- 172.21.15.19:0/3808207728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f439c0039b0 0x7f43b81423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.748+0000 7f43bea00640 1 -- 172.21.15.19:0/3808207728 >> 172.21.15.19:0/3808207728 conn(0x7f43b806b990 msgr2=0x7f43b806d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:52.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.748+0000 7f43bea00640 1 -- 172.21.15.19:0/3808207728 shutdown_connections 2024-03-28T13:21:52.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.748+0000 7f43bea00640 1 -- 172.21.15.19:0/3808207728 wait complete. 2024-03-28T13:21:52.766 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-03-28T13:21:52.265529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":279088,"kb_used_data":29644,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695632,"statfs":{"total":773094113280,"available":772808327168,"internally_reserved":0,"allocated":30355456,"data_stored":23359910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392250,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-178,"num_write_kb":-8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.007087"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":364,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-03-28T13:21:49.532212+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:21:49.532212+0000","last_peered":"2024-03-28T13:21:49.532212+0000","last_clean":"2024-03-28T13:21:49.532212+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:21:49.532212+0000","last_undegraded":"2024-03-28T13:21:49.532212+0000","last_fullsized":"2024-03-28T13:21:49.532212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":681169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":681169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2138112,"data_stored":1265783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476888,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36784,"kb_used_data":5604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":5738496,"data_stored":4869719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36784,"kb_used_data":5604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":5738496,"data_stored":4869719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509592,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33272,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338568,"statfs":{"total":96636764160,"available":96602693632,"internally_reserved":0,"allocated":2142208,"data_stored":1265788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509592,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36784,"kb_used_data":5604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":5738496,"data_stored":4866415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542296,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":5604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":5738496,"data_stored":4860148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:21:52.766 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-28T13:21:52.766 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:21:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.991+0000 7f4af6200640 1 Processor -- start 2024-03-28T13:21:52.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.992+0000 7f4af6200640 1 -- start start 2024-03-28T13:21:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.993+0000 7f4af6200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af014ba20 0x7f4af00b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.993+0000 7f4af6200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4af014c1b0 0x7f4af0166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.993+0000 7f4af6200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4af00b33e0 con 0x7f4af00cfda0 2024-03-28T13:21:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.993+0000 7f4af6200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f4af00cde20 con 0x7f4af014c1b0 2024-03-28T13:21:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.993+0000 7f4af6200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4af0166ae0 con 0x7f4af014ba20 2024-03-28T13:21:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.994+0000 7f4aefe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4af014c1b0 0x7f4af0166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.994+0000 7f4aefe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f4af014c1b0 0x7f4af0166290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44426/0 (socket says 172.21.15.19:44426) 2024-03-28T13:21:52.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.994+0000 7f4aefe00640 1 -- 172.21.15.19:0/1435463972 learned_addr learned my addr 172.21.15.19:0/1435463972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:52.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.994+0000 7f4af4e00640 1 --2- 172.21.15.19:0/1435463972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af014ba20 0x7f4af00b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:52.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.994+0000 7f4aefe00640 1 -- 172.21.15.19:0/1435463972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af014ba20 msgr2=0x7f4af00b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:52.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.994+0000 7f4aefe00640 1 --2- 172.21.15.19:0/1435463972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af014ba20 0x7f4af00b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.994+0000 7f4aefe00640 1 -- 172.21.15.19:0/1435463972 >> v1:172.21.15.188:6789/0 conn(0x7f4af00cfda0 legacy=0x7f4af00ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:52.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.994+0000 7f4aefe00640 1 -- 172.21.15.19:0/1435463972 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af0166dc0 con 0x7f4af014c1b0 2024-03-28T13:21:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.995+0000 7f4aefe00640 1 --2- 172.21.15.19:0/1435463972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af014c1b0 0x7f4af0166290 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009870 tx=0x7f4adc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7ce9823d9a599b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.996+0000 7f4aef400640 1 -- 172.21.15.19:0/1435463972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc035020 con 0x7f4af014c1b0 2024-03-28T13:21:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.996+0000 7f4aef400640 1 -- 172.21.15.19:0/1435463972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4adc031640 con 0x7f4af014c1b0 2024-03-28T13:21:52.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.996+0000 7f4aef400640 1 -- 172.21.15.19:0/1435463972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc02d040 con 0x7f4af014c1b0 2024-03-28T13:21:52.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.996+0000 7f4af6200640 1 -- 172.21.15.19:0/1435463972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af014c1b0 msgr2=0x7f4af0166290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:52.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.996+0000 7f4af6200640 1 --2- 172.21.15.19:0/1435463972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af014c1b0 0x7f4af0166290 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f4adc009870 tx=0x7f4adc02b8f0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.996+0000 7f4af6200640 1 -- 172.21.15.19:0/1435463972 shutdown_connections 2024-03-28T13:21:52.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.996+0000 7f4af6200640 1 --2- 172.21.15.19:0/1435463972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af014c1b0 0x7f4af0166290 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.996+0000 7f4af6200640 1 --2- 172.21.15.19:0/1435463972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af014ba20 0x7f4af00b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.996+0000 7f4af6200640 1 -- 172.21.15.19:0/1435463972 >> 172.21.15.19:0/1435463972 conn(0x7f4af00ccad0 msgr2=0x7f4af00cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.997+0000 7f4af6200640 1 -- 172.21.15.19:0/1435463972 shutdown_connections 2024-03-28T13:21:52.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.997+0000 7f4af6200640 1 -- 172.21.15.19:0/1435463972 wait complete. 2024-03-28T13:21:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.998+0000 7f4af6200640 1 Processor -- start 2024-03-28T13:21:52.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:52.999+0000 7f4af6200640 1 -- start start 2024-03-28T13:21:52.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4af6200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af00cfda0 0x7f4af0161d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4af6200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af014ba20 0x7f4af0148940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4af6200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af01edba0 0x7f4af01ee050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:52.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4af6200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4af01667d0 con 0x7f4af014ba20 2024-03-28T13:21:53.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4af6200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4af00ce0c0 con 0x7f4af00cfda0 2024-03-28T13:21:53.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4af6200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4af014be00 con 0x7f4af01edba0 2024-03-28T13:21:53.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4aefe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af014ba20 0x7f4af0148940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:53.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4aefe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af014ba20 0x7f4af0148940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44440/0 (socket says 172.21.15.19:44440) 2024-03-28T13:21:53.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4aefe00640 1 -- 172.21.15.19:0/3807885590 learned_addr learned my addr 172.21.15.19:0/3807885590 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:53.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4af4e00640 1 --2- 172.21.15.19:0/3807885590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af00cfda0 0x7f4af0161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:53.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4aefe00640 1 -- 172.21.15.19:0/3807885590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af01edba0 msgr2=0x7f4af01ee050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:53.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4af5800640 1 --2- 172.21.15.19:0/3807885590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af01edba0 0x7f4af01ee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:53.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4aefe00640 1 --2- 172.21.15.19:0/3807885590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af01edba0 0x7f4af01ee050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4aefe00640 1 -- 172.21.15.19:0/3807885590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af00cfda0 msgr2=0x7f4af0161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:53.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4aefe00640 1 --2- 172.21.15.19:0/3807885590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af00cfda0 0x7f4af0161d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4aefe00640 1 -- 172.21.15.19:0/3807885590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af01fc300 con 0x7f4af014ba20 2024-03-28T13:21:53.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.000+0000 7f4af4e00640 1 --2- 172.21.15.19:0/3807885590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af00cfda0 0x7f4af0161d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:53.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.001+0000 7f4aefe00640 1 --2- 172.21.15.19:0/3807885590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af014ba20 0x7f4af0148940 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0090b0 tx=0x7f4adc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:53.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.001+0000 7f4aecc00640 1 -- 172.21.15.19:0/3807885590 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc040070 con 0x7f4af014ba20 2024-03-28T13:21:53.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.001+0000 7f4aecc00640 1 -- 172.21.15.19:0/3807885590 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4adc004030 con 0x7f4af014ba20 2024-03-28T13:21:53.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.001+0000 7f4aecc00640 1 -- 172.21.15.19:0/3807885590 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc035040 con 0x7f4af014ba20 2024-03-28T13:21:53.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.001+0000 7f4af6200640 1 -- 172.21.15.19:0/3807885590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af01fc5e0 con 0x7f4af014ba20 2024-03-28T13:21:53.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.001+0000 7f4af6200640 1 -- 172.21.15.19:0/3807885590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af01fcba0 con 0x7f4af014ba20 2024-03-28T13:21:53.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.003+0000 7f4af6200640 1 -- 172.21.15.19:0/3807885590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4af0075940 con 0x7f4af014ba20 2024-03-28T13:21:53.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.004+0000 7f4aecc00640 1 -- 172.21.15.19:0/3807885590 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f4adc038020 con 0x7f4af014ba20 2024-03-28T13:21:53.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.005+0000 7f4aecc00640 1 --2- 172.21.15.19:0/3807885590 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4ad006c530 0x7f4ad006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:53.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.005+0000 7f4aecc00640 1 -- 172.21.15.19:0/3807885590 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7f4adc0aced0 con 0x7f4af014ba20 2024-03-28T13:21:53.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.005+0000 7f4af4e00640 1 --2- 172.21.15.19:0/3807885590 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4ad006c530 0x7f4ad006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:53.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.006+0000 7f4af4e00640 1 --2- 172.21.15.19:0/3807885590 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4ad006c530 0x7f4ad006e9f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4006fd0 tx=0x7f4ae4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:53.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.010+0000 7f4aecc00640 1 -- 172.21.15.19:0/3807885590 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f4adc07b6a0 con 0x7f4af014ba20 2024-03-28T13:21:53.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.210+0000 7f4af6200640 1 -- 172.21.15.19:0/3807885590 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4af01622c0 con 0x7f4ad006c530 2024-03-28T13:21:53.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.213+0000 7f4aecc00640 1 -- 172.21.15.19:0/3807885590 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13521 (secure 0 0 0) 0x7f4ae4009e30 con 0x7f4ad006c530 2024-03-28T13:21:53.212 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:53.212 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:21:53.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.215+0000 7f4af6200640 1 -- 172.21.15.19:0/3807885590 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4ad006c530 msgr2=0x7f4ad006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:53.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.215+0000 7f4af6200640 1 --2- 172.21.15.19:0/3807885590 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4ad006c530 0x7f4ad006e9f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4006fd0 tx=0x7f4ae4008040 comp rx=0 tx=0).stop 2024-03-28T13:21:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.215+0000 7f4af6200640 1 -- 172.21.15.19:0/3807885590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af014ba20 msgr2=0x7f4af0148940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.215+0000 7f4af6200640 1 --2- 172.21.15.19:0/3807885590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af014ba20 0x7f4af0148940 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f4adc0090b0 tx=0x7f4adc0046c0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.216+0000 7f4af6200640 1 -- 172.21.15.19:0/3807885590 shutdown_connections 2024-03-28T13:21:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.216+0000 7f4af6200640 1 --2- 172.21.15.19:0/3807885590 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f4ad006c530 0x7f4ad006e9f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.216+0000 7f4af6200640 1 --2- 172.21.15.19:0/3807885590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4af01edba0 0x7f4af01ee050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.216+0000 7f4af6200640 1 --2- 172.21.15.19:0/3807885590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4af014ba20 0x7f4af0148940 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.216+0000 7f4af6200640 1 --2- 172.21.15.19:0/3807885590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4af00cfda0 0x7f4af0161d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.216+0000 7f4af6200640 1 -- 172.21.15.19:0/3807885590 >> 172.21.15.19:0/3807885590 conn(0x7f4af00ccad0 msgr2=0x7f4af00b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.216+0000 7f4af6200640 1 -- 172.21.15.19:0/3807885590 shutdown_connections 2024-03-28T13:21:53.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.216+0000 7f4af6200640 1 -- 172.21.15.19:0/3807885590 wait complete. 2024-03-28T13:21:53.233 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-03-28T13:21:52.265529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":279088,"kb_used_data":29644,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695632,"statfs":{"total":773094113280,"available":772808327168,"internally_reserved":0,"allocated":30355456,"data_stored":23359910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392250,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-178,"num_write_kb":-8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.007087"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":364,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-03-28T13:21:49.532212+0000","last_change":"2024-03-28T13:09:31.928293+0000","last_active":"2024-03-28T13:21:49.532212+0000","last_peered":"2024-03-28T13:21:49.532212+0000","last_clean":"2024-03-28T13:21:49.532212+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:21:49.532212+0000","last_undegraded":"2024-03-28T13:21:49.532212+0000","last_fullsized":"2024-03-28T13:21:49.532212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-28T13:09:30.439123+0000","last_clean_scrub_stamp":"2024-03-28T13:09:30.439123+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T15:56:04.726680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":681169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":681169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2138112,"data_stored":1265783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476888,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36784,"kb_used_data":5604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":5738496,"data_stored":4869719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36784,"kb_used_data":5604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":5738496,"data_stored":4869719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509592,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33272,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338568,"statfs":{"total":96636764160,"available":96602693632,"internally_reserved":0,"allocated":2142208,"data_stored":1265788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509592,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36784,"kb_used_data":5604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335056,"statfs":{"total":96636764160,"available":96599097344,"internally_reserved":0,"allocated":5738496,"data_stored":4866415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542296,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36780,"kb_used_data":5604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335060,"statfs":{"total":96636764160,"available":96599101440,"internally_reserved":0,"allocated":5738496,"data_stored":4860148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:21:53.234 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-28T13:21:53.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.457+0000 7fea45a00640 1 Processor -- start 2024-03-28T13:21:53.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.457+0000 7fea45a00640 1 -- start start 2024-03-28T13:21:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.458+0000 7fea45a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fea4015e530 0x7fea400cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.458+0000 7fea45a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7fea400cf330 0x7fea400d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.458+0000 7fea45a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea40162350 con 0x7fea4006e1b0 2024-03-28T13:21:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.458+0000 7fea45a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7fea40162ed0 con 0x7fea400cf330 2024-03-28T13:21:53.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.459+0000 7fea45a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fea40162630 con 0x7fea4015e530 2024-03-28T13:21:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.459+0000 7fea37e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fea4006e1b0 0x7fea400cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34292/0 (socket says 172.21.15.19:34292) 2024-03-28T13:21:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.459+0000 7fea37e00640 1 -- 172.21.15.19:0/4002452343 learned_addr learned my addr 172.21.15.19:0/4002452343 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:53.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.459+0000 7fea45000640 1 --2- 172.21.15.19:0/4002452343 >> v2:172.21.15.19:3300/0 conn(0x7fea400cf330 0x7fea400d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.459+0000 7fea3fe00640 1 --2- 172.21.15.19:0/4002452343 >> v2:172.21.15.188:3300/0 conn(0x7fea4015e530 0x7fea400cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.459+0000 7fea3f400640 1 -- 172.21.15.19:0/4002452343 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1499917752 0 0) 0x7fea40162350 con 0x7fea4006e1b0 2024-03-28T13:21:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.459+0000 7fea3f400640 1 -- 172.21.15.19:0/4002452343 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea24003520 con 0x7fea4006e1b0 2024-03-28T13:21:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.460+0000 7fea45000640 1 -- 172.21.15.19:0/4002452343 >> v2:172.21.15.188:3300/0 conn(0x7fea4015e530 msgr2=0x7fea400cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.460+0000 7fea45000640 1 --2- 172.21.15.19:0/4002452343 >> v2:172.21.15.188:3300/0 conn(0x7fea4015e530 0x7fea400cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.460+0000 7fea45000640 1 -- 172.21.15.19:0/4002452343 >> v1:172.21.15.19:6789/0 conn(0x7fea4006e1b0 legacy=0x7fea400cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.460+0000 7fea45000640 1 -- 172.21.15.19:0/4002452343 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea40162910 con 0x7fea400cf330 2024-03-28T13:21:53.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.460+0000 7fea3fe00640 1 --2- 172.21.15.19:0/4002452343 >> v2:172.21.15.188:3300/0 conn(0x7fea4015e530 0x7fea400cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:21:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.461+0000 7fea45000640 1 --2- 172.21.15.19:0/4002452343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea400cf330 0x7fea400d1a30 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fea400680e0 tx=0x7fea3802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90abd83e4156dd82 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.461+0000 7fea3f400640 1 -- 172.21.15.19:0/4002452343 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38039070 con 0x7fea400cf330 2024-03-28T13:21:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.461+0000 7fea3f400640 1 -- 172.21.15.19:0/4002452343 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea3802cbf0 con 0x7fea400cf330 2024-03-28T13:21:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.461+0000 7fea3f400640 1 -- 172.21.15.19:0/4002452343 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38034440 con 0x7fea400cf330 2024-03-28T13:21:53.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.462+0000 7fea45a00640 1 -- 172.21.15.19:0/4002452343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea400cf330 msgr2=0x7fea400d1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.462+0000 7fea45a00640 1 --2- 172.21.15.19:0/4002452343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea400cf330 0x7fea400d1a30 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fea400680e0 tx=0x7fea3802be40 comp rx=0 tx=0).stop 2024-03-28T13:21:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.462+0000 7fea45a00640 1 -- 172.21.15.19:0/4002452343 shutdown_connections 2024-03-28T13:21:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.462+0000 7fea45a00640 1 --2- 172.21.15.19:0/4002452343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea400cf330 0x7fea400d1a30 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.462+0000 7fea45a00640 1 --2- 172.21.15.19:0/4002452343 >> v2:172.21.15.188:3300/0 conn(0x7fea4015e530 0x7fea400cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.462+0000 7fea45a00640 1 -- 172.21.15.19:0/4002452343 >> 172.21.15.19:0/4002452343 conn(0x7fea40067280 msgr2=0x7fea40067660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.462+0000 7fea45a00640 1 -- 172.21.15.19:0/4002452343 shutdown_connections 2024-03-28T13:21:53.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.462+0000 7fea45a00640 1 -- 172.21.15.19:0/4002452343 wait complete. 2024-03-28T13:21:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.463+0000 7fea45a00640 1 Processor -- start 2024-03-28T13:21:53.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.464+0000 7fea45a00640 1 -- start start 2024-03-28T13:21:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea45a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea4006e1b0 0x7fea401610f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea45a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea400cf330 0x7fea401f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea45a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea4015e530 0x7fea4015c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea45a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea400cf780 con 0x7fea4006e1b0 2024-03-28T13:21:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea45a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fea40161fe0 con 0x7fea400cf330 2024-03-28T13:21:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea45a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fea400cf900 con 0x7fea4015e530 2024-03-28T13:21:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea3fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea4006e1b0 0x7fea401610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:53.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea3fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea4006e1b0 0x7fea401610f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44462/0 (socket says 172.21.15.19:44462) 2024-03-28T13:21:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea3fe00640 1 -- 172.21.15.19:0/4061153789 learned_addr learned my addr 172.21.15.19:0/4061153789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea37e00640 1 --2- 172.21.15.19:0/4061153789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea400cf330 0x7fea401f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea45000640 1 --2- 172.21.15.19:0/4061153789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea4015e530 0x7fea4015c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea3fe00640 1 -- 172.21.15.19:0/4061153789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea4015e530 msgr2=0x7fea4015c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea3fe00640 1 --2- 172.21.15.19:0/4061153789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea4015e530 0x7fea4015c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea3fe00640 1 -- 172.21.15.19:0/4061153789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea400cf330 msgr2=0x7fea401f35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea3fe00640 1 --2- 172.21.15.19:0/4061153789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea400cf330 0x7fea401f35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea3fe00640 1 -- 172.21.15.19:0/4061153789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4014bd20 con 0x7fea4006e1b0 2024-03-28T13:21:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.465+0000 7fea37e00640 1 --2- 172.21.15.19:0/4061153789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea400cf330 0x7fea401f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:53.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.466+0000 7fea3fe00640 1 --2- 172.21.15.19:0/4061153789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea4006e1b0 0x7fea401610f0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fea3000d7d0 tx=0x7fea3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:53.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.466+0000 7fea3cc00640 1 -- 172.21.15.19:0/4061153789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30004030 con 0x7fea4006e1b0 2024-03-28T13:21:53.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.466+0000 7fea3cc00640 1 -- 172.21.15.19:0/4061153789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea300041d0 con 0x7fea4006e1b0 2024-03-28T13:21:53.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.466+0000 7fea3cc00640 1 -- 172.21.15.19:0/4061153789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3000b6c0 con 0x7fea4006e1b0 2024-03-28T13:21:53.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.466+0000 7fea45a00640 1 -- 172.21.15.19:0/4061153789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea401f78d0 con 0x7fea4006e1b0 2024-03-28T13:21:53.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.466+0000 7fea45a00640 1 -- 172.21.15.19:0/4061153789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea401f7e90 con 0x7fea4006e1b0 2024-03-28T13:21:53.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.467+0000 7fea45a00640 1 -- 172.21.15.19:0/4061153789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea40075940 con 0x7fea4006e1b0 2024-03-28T13:21:53.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.468+0000 7fea3cc00640 1 -- 172.21.15.19:0/4061153789 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fea3000b860 con 0x7fea4006e1b0 2024-03-28T13:21:53.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.469+0000 7fea3cc00640 1 --2- 172.21.15.19:0/4061153789 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fea1c06c530 0x7fea1c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:53.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.469+0000 7fea3cc00640 1 -- 172.21.15.19:0/4061153789 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7fea3008b740 con 0x7fea4006e1b0 2024-03-28T13:21:53.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.470+0000 7fea37e00640 1 --2- 172.21.15.19:0/4061153789 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fea1c06c530 0x7fea1c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:53.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.471+0000 7fea37e00640 1 --2- 172.21.15.19:0/4061153789 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fea1c06c530 0x7fea1c06e9f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fea400b1490 tx=0x7fea28009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:53.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.474+0000 7fea3cc00640 1 -- 172.21.15.19:0/4061153789 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fea30059fa0 con 0x7fea4006e1b0 2024-03-28T13:21:53.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.677+0000 7fea45a00640 1 -- 172.21.15.19:0/4061153789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea400b1750 con 0x7fea4006e1b0 2024-03-28T13:21:53.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.678+0000 7fea3cc00640 1 -- 172.21.15.19:0/4061153789 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) ==== 74+0+13292 (secure 0 0 0) 0x7fea300598a0 con 0x7fea4006e1b0 2024-03-28T13:21:53.677 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:21:53.678 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":126,"fsid":"e6b569d9-f2ce-49dc-bbe7-c0ab33080203","created":"2024-03-28T13:09:16.587838+0000","modified":"2024-03-28T13:21:49.387072+0000","last_up_change":"2024-03-28T13:09:29.434240+0000","last_in_change":"2024-03-28T13:09:19.983398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-28T13:09:30.014312+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d13c2148-f3e7-4d77-9c12-0f722feeba16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6802","nonce":4106483280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6806","nonce":4106483280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6814","nonce":4106483280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":4106483280},{"type":"v1","addr":"172.21.15.19:6810","nonce":4106483280}]},"public_addr":"172.21.15.19:6802/4106483280","cluster_addr":"172.21.15.19:6806/4106483280","heartbeat_back_addr":"172.21.15.19:6814/4106483280","heartbeat_front_addr":"172.21.15.19:6810/4106483280","state":["exists","up"]},{"osd":1,"uuid":"9ece52d8-56dc-4c72-882a-46f1ea5685cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6801","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6803","nonce":1998544260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6805","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6807","nonce":1998544260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6813","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6815","nonce":1998544260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6809","nonce":1998544260},{"type":"v1","addr":"172.21.15.19:6811","nonce":1998544260}]},"public_addr":"172.21.15.19:6803/1998544260","cluster_addr":"172.21.15.19:6807/1998544260","heartbeat_back_addr":"172.21.15.19:6815/1998544260","heartbeat_front_addr":"172.21.15.19:6811/1998544260","state":["exists","up"]},{"osd":2,"uuid":"64162cf7-5b2e-483d-9b2b-1366fb36d134","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6817","nonce":1693606238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6820","nonce":1693606238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6829","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6831","nonce":1693606238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6823","nonce":1693606238},{"type":"v1","addr":"172.21.15.19:6826","nonce":1693606238}]},"public_addr":"172.21.15.19:6817/1693606238","cluster_addr":"172.21.15.19:6820/1693606238","heartbeat_back_addr":"172.21.15.19:6831/1693606238","heartbeat_front_addr":"172.21.15.19:6826/1693606238","state":["exists","up"]},{"osd":3,"uuid":"e8162dd0-8c0f-4a78-809e-03f92586a52d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6819","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6821","nonce":3265515865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6824","nonce":3265515865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6830","nonce":3265515865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6825","nonce":3265515865},{"type":"v1","addr":"172.21.15.19:6827","nonce":3265515865}]},"public_addr":"172.21.15.19:6821/3265515865","cluster_addr":"172.21.15.19:6824/3265515865","heartbeat_back_addr":"172.21.15.19:6830/3265515865","heartbeat_front_addr":"172.21.15.19:6827/3265515865","state":["exists","up"]},{"osd":4,"uuid":"72b8a390-bb07-4819-8f19-aa3890ec8ce4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6809","nonce":1845695132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6811","nonce":1845695132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6815","nonce":1845695132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":1845695132},{"type":"v1","addr":"172.21.15.188:6813","nonce":1845695132}]},"public_addr":"172.21.15.188:6809/1845695132","cluster_addr":"172.21.15.188:6811/1845695132","heartbeat_back_addr":"172.21.15.188:6815/1845695132","heartbeat_front_addr":"172.21.15.188:6813/1845695132","state":["exists","up"]},{"osd":5,"uuid":"bd4eddad-6788-4059-b203-521c62cdf1be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6801","nonce":3324247755}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6803","nonce":3324247755}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6807","nonce":3324247755}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":3324247755},{"type":"v1","addr":"172.21.15.188:6805","nonce":3324247755}]},"public_addr":"172.21.15.188:6801/3324247755","cluster_addr":"172.21.15.188:6803/3324247755","heartbeat_back_addr":"172.21.15.188:6807/3324247755","heartbeat_front_addr":"172.21.15.188:6805/3324247755","state":["exists","up"]},{"osd":6,"uuid":"9b3837e6-8631-4178-94d2-6d76f3e0c50a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6817","nonce":3597358155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6819","nonce":3597358155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6823","nonce":3597358155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":3597358155},{"type":"v1","addr":"172.21.15.188:6821","nonce":3597358155}]},"public_addr":"172.21.15.188:6817/3597358155","cluster_addr":"172.21.15.188:6819/3597358155","heartbeat_back_addr":"172.21.15.188:6823/3597358155","heartbeat_front_addr":"172.21.15.188:6821/3597358155","state":["exists","up"]},{"osd":7,"uuid":"2dbbb7c5-ce51-4d70-a98b-954428334789","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6825","nonce":861796513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6827","nonce":861796513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6831","nonce":861796513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":861796513},{"type":"v1","addr":"172.21.15.188:6829","nonce":861796513}]},"public_addr":"172.21.15.188:6825/861796513","cluster_addr":"172.21.15.188:6827/861796513","heartbeat_back_addr":"172.21.15.188:6831/861796513","heartbeat_front_addr":"172.21.15.188:6829/861796513","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.101677+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.249382+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.370674+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.286433+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.308477+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.290655+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:25.306303+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-28T13:09:24.862237+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:6835/1527816903":"2024-03-29T13:21:46.251638+0000","172.21.15.19:6833/1527816903":"2024-03-29T13:21:46.251638+0000","172.21.15.19:6835/3930967142":"2024-03-29T13:21:26.921326+0000","172.21.15.19:6833/3930967142":"2024-03-29T13:21:26.921326+0000","172.21.15.188:6836/2088538359":"2024-03-29T13:21:12.469733+0000","172.21.15.188:6834/2088538359":"2024-03-29T13:21:12.469733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-28T13:21:53.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.681+0000 7fea45a00640 1 -- 172.21.15.19:0/4061153789 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fea1c06c530 msgr2=0x7fea1c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:53.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.681+0000 7fea45a00640 1 --2- 172.21.15.19:0/4061153789 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fea1c06c530 0x7fea1c06e9f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fea400b1490 tx=0x7fea28009040 comp rx=0 tx=0).stop 2024-03-28T13:21:53.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.681+0000 7fea45a00640 1 -- 172.21.15.19:0/4061153789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea4006e1b0 msgr2=0x7fea401610f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:53.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.681+0000 7fea45a00640 1 --2- 172.21.15.19:0/4061153789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea4006e1b0 0x7fea401610f0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fea3000d7d0 tx=0x7fea3000dc90 comp rx=0 tx=0).stop 2024-03-28T13:21:53.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.681+0000 7fea45a00640 1 -- 172.21.15.19:0/4061153789 shutdown_connections 2024-03-28T13:21:53.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.681+0000 7fea45a00640 1 --2- 172.21.15.19:0/4061153789 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fea1c06c530 0x7fea1c06e9f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.681+0000 7fea45a00640 1 --2- 172.21.15.19:0/4061153789 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea4015e530 0x7fea4015c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.681+0000 7fea45a00640 1 --2- 172.21.15.19:0/4061153789 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea400cf330 0x7fea401f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.681+0000 7fea45a00640 1 --2- 172.21.15.19:0/4061153789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea4006e1b0 0x7fea401610f0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:53.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.681+0000 7fea45a00640 1 -- 172.21.15.19:0/4061153789 >> 172.21.15.19:0/4061153789 conn(0x7fea40067280 msgr2=0x7fea40067660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:53.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.681+0000 7fea45a00640 1 -- 172.21.15.19:0/4061153789 shutdown_connections 2024-03-28T13:21:53.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:53.681+0000 7fea45a00640 1 -- 172.21.15.19:0/4061153789 wait complete. 2024-03-28T13:21:54.702 INFO:tasks.ceph:Scrubbing osd.0 2024-03-28T13:21:54.703 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-28T13:21:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.942+0000 7f88b7a00640 1 Processor -- start 2024-03-28T13:21:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.942+0000 7f88b7a00640 1 -- start start 2024-03-28T13:21:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.944+0000 7f88b7a00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f88b01554d0 0x7f88b015f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:54.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.944+0000 7f88b7a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f88b006cc90 0x7f88b00ae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:54.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.944+0000 7f88b7a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88b00b3b50 con 0x7f88b00b5b00 2024-03-28T13:21:54.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.944+0000 7f88b7a00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f88b00b4740 con 0x7f88b01554d0 2024-03-28T13:21:54.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.944+0000 7f88b7a00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f88b00b3e10 con 0x7f88b006cc90 2024-03-28T13:21:54.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.944+0000 7f88b5000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f88b01554d0 0x7f88b015f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:54.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.944+0000 7f88b5000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f88b01554d0 0x7f88b015f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44478/0 (socket says 172.21.15.19:44478) 2024-03-28T13:21:54.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.944+0000 7f88b5000640 1 -- 172.21.15.19:0/3347874430 learned_addr learned my addr 172.21.15.19:0/3347874430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:54.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.945+0000 7f88b5a00640 1 --2- 172.21.15.19:0/3347874430 >> v2:172.21.15.188:3300/0 conn(0x7f88b006cc90 0x7f88b00ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.945+0000 7f88b5000640 1 -- 172.21.15.19:0/3347874430 >> v2:172.21.15.188:3300/0 conn(0x7f88b006cc90 msgr2=0x7f88b00ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.945+0000 7f88b5000640 1 --2- 172.21.15.19:0/3347874430 >> v2:172.21.15.188:3300/0 conn(0x7f88b006cc90 0x7f88b00ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.945+0000 7f88b5000640 1 -- 172.21.15.19:0/3347874430 >> v1:172.21.15.188:6789/0 conn(0x7f88b00b5b00 legacy=0x7f88b006c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.945+0000 7f88b5000640 1 -- 172.21.15.19:0/3347874430 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b00b40f0 con 0x7f88b01554d0 2024-03-28T13:21:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.946+0000 7f88b5000640 1 --2- 172.21.15.19:0/3347874430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b01554d0 0x7f88b015f520 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f88b015ea80 tx=0x7f88a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0d4a2dd4cf64dc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.946+0000 7f88af400640 1 -- 172.21.15.19:0/3347874430 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4035020 con 0x7f88b01554d0 2024-03-28T13:21:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.946+0000 7f88af400640 1 -- 172.21.15.19:0/3347874430 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88a4031870 con 0x7f88b01554d0 2024-03-28T13:21:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.946+0000 7f88af400640 1 -- 172.21.15.19:0/3347874430 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a4031c40 con 0x7f88b01554d0 2024-03-28T13:21:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.947+0000 7f88b7a00640 1 -- 172.21.15.19:0/3347874430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b01554d0 msgr2=0x7f88b015f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.947+0000 7f88b7a00640 1 --2- 172.21.15.19:0/3347874430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b01554d0 0x7f88b015f520 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f88b015ea80 tx=0x7f88a402b910 comp rx=0 tx=0).stop 2024-03-28T13:21:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.947+0000 7f88b7a00640 1 -- 172.21.15.19:0/3347874430 shutdown_connections 2024-03-28T13:21:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.947+0000 7f88b7a00640 1 --2- 172.21.15.19:0/3347874430 >> v2:172.21.15.188:3300/0 conn(0x7f88b006cc90 0x7f88b00ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.947+0000 7f88b7a00640 1 --2- 172.21.15.19:0/3347874430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b01554d0 0x7f88b015f520 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:54.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.947+0000 7f88b7a00640 1 -- 172.21.15.19:0/3347874430 >> 172.21.15.19:0/3347874430 conn(0x7f88b00720d0 msgr2=0x7f88b00724e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:54.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.947+0000 7f88b7a00640 1 -- 172.21.15.19:0/3347874430 shutdown_connections 2024-03-28T13:21:54.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.948+0000 7f88b7a00640 1 -- 172.21.15.19:0/3347874430 wait complete. 2024-03-28T13:21:54.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.950+0000 7f88b7a00640 1 Processor -- start 2024-03-28T13:21:54.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.950+0000 7f88b7a00640 1 -- start start 2024-03-28T13:21:54.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.951+0000 7f88b7a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b006cc90 0x7f88b01e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:54.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.951+0000 7f88b7a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88b00b5b00 0x7f88b006f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.952+0000 7f88b7a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b01554d0 0x7f88b006fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.952+0000 7f88b7a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88b01558b0 con 0x7f88b01554d0 2024-03-28T13:21:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.952+0000 7f88b7a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f88b00b5f70 con 0x7f88b006cc90 2024-03-28T13:21:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.952+0000 7f88b7a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f88b00aed70 con 0x7f88b00b5b00 2024-03-28T13:21:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.952+0000 7f88b5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b01554d0 0x7f88b006fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.952+0000 7f88b5a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b01554d0 0x7f88b006fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44492/0 (socket says 172.21.15.19:44492) 2024-03-28T13:21:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.952+0000 7f88b5a00640 1 -- 172.21.15.19:0/79466929 learned_addr learned my addr 172.21.15.19:0/79466929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.952+0000 7f88b5000640 1 --2- 172.21.15.19:0/79466929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b006cc90 0x7f88b01e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:54.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.952+0000 7f88afe00640 1 --2- 172.21.15.19:0/79466929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88b00b5b00 0x7f88b006f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:54.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.952+0000 7f88b5a00640 1 -- 172.21.15.19:0/79466929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88b00b5b00 msgr2=0x7f88b006f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:54.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.952+0000 7f88b5a00640 1 --2- 172.21.15.19:0/79466929 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88b00b5b00 0x7f88b006f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:54.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.952+0000 7f88b5a00640 1 -- 172.21.15.19:0/79466929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b006cc90 msgr2=0x7f88b01e7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:54.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.952+0000 7f88b5a00640 1 --2- 172.21.15.19:0/79466929 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88b006cc90 0x7f88b01e7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:54.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.952+0000 7f88b5a00640 1 -- 172.21.15.19:0/79466929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b01e8870 con 0x7f88b01554d0 2024-03-28T13:21:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.953+0000 7f88b5a00640 1 --2- 172.21.15.19:0/79466929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b01554d0 0x7f88b006fcd0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f88a800cd70 tx=0x7f88a800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.953+0000 7f88acc00640 1 -- 172.21.15.19:0/79466929 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8003b90 con 0x7f88b01554d0 2024-03-28T13:21:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.953+0000 7f88acc00640 1 -- 172.21.15.19:0/79466929 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88a8003d30 con 0x7f88b01554d0 2024-03-28T13:21:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.953+0000 7f88acc00640 1 -- 172.21.15.19:0/79466929 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a8004240 con 0x7f88b01554d0 2024-03-28T13:21:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.953+0000 7f88b7a00640 1 -- 172.21.15.19:0/79466929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b01e8b50 con 0x7f88b01554d0 2024-03-28T13:21:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.953+0000 7f88b7a00640 1 -- 172.21.15.19:0/79466929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b01e9110 con 0x7f88b01554d0 2024-03-28T13:21:54.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.954+0000 7f88b7a00640 1 -- 172.21.15.19:0/79466929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f88b00685c0 con 0x7f88b01554d0 2024-03-28T13:21:54.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.956+0000 7f88acc00640 1 -- 172.21.15.19:0/79466929 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f88a80043e0 con 0x7f88b01554d0 2024-03-28T13:21:54.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.957+0000 7f88acc00640 1 --2- 172.21.15.19:0/79466929 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f889006c530 0x7f889006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:54.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.957+0000 7f88acc00640 1 -- 172.21.15.19:0/79466929 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7f88a808bfe0 con 0x7f88b01554d0 2024-03-28T13:21:54.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.957+0000 7f88acc00640 1 --2- 172.21.15.19:0/79466929 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x7f8890072530 0x7f8890074950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:54.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.957+0000 7f88acc00640 1 -- 172.21.15.19:0/79466929 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f88a801ce50 con 0x7f8890072530 2024-03-28T13:21:54.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.957+0000 7f88acc00640 1 -- 172.21.15.19:0/79466929 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f88a808e5c0 con 0x7f88b01554d0 2024-03-28T13:21:54.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.957+0000 7f88afe00640 1 --2- 172.21.15.19:0/79466929 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x7f8890072530 0x7f8890074950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:54.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.957+0000 7f88b5000640 1 --2- 172.21.15.19:0/79466929 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f889006c530 0x7f889006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:54.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.958+0000 7f88afe00640 1 --2- 172.21.15.19:0/79466929 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x7f8890072530 0x7f8890074950 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:54.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.958+0000 7f88b5000640 1 --2- 172.21.15.19:0/79466929 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f889006c530 0x7f889006e9f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f88a4036000 tx=0x7f88a403c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:54.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.959+0000 7f88acc00640 1 -- 172.21.15.19:0/79466929 <== osd.0 v2:172.21.15.19:6800/4106483280 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f88a801ce50 con 0x7f8890072530 2024-03-28T13:21:54.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.974+0000 7f88b7a00640 1 -- 172.21.15.19:0/79466929 --> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f88b015e430 con 0x7f8890072530 2024-03-28T13:21:54.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.974+0000 7f88acc00640 1 -- 172.21.15.19:0/79466929 <== osd.0 v2:172.21.15.19:6800/4106483280 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f88b015e430 con 0x7f8890072530 2024-03-28T13:21:54.973 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:21:54.973 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T13:21:54.973 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:21:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.975+0000 7f88b7a00640 1 -- 172.21.15.19:0/79466929 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x7f8890072530 msgr2=0x7f8890074950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.975+0000 7f88b7a00640 1 --2- 172.21.15.19:0/79466929 >> [v2:172.21.15.19:6800/4106483280,v1:172.21.15.19:6802/4106483280] conn(0x7f8890072530 0x7f8890074950 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.975+0000 7f88b7a00640 1 -- 172.21.15.19:0/79466929 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f889006c530 msgr2=0x7f889006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.975+0000 7f88b7a00640 1 --2- 172.21.15.19:0/79466929 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f889006c530 0x7f889006e9f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f88a4036000 tx=0x7f88a403c040 comp rx=0 tx=0).stop 2024-03-28T13:21:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.975+0000 7f88b7a00640 1 -- 172.21.15.19:0/79466929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b01554d0 msgr2=0x7f88b006fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.975+0000 7f88b7a00640 1 --2- 172.21.15.19:0/79466929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f88b01554d0 0x7f88b006fcd0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f88a800cd70 tx=0x7f88a800a660 comp rx=0 tx=0).stop 2024-03-28T13:21:54.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.975+0000 7f88b5a00640 1 -- 172.21.15.19:0/79466929 reap_dead start 2024-03-28T13:21:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.975+0000 7f88b7a00640 1 -- 172.21.15.19:0/79466929 shutdown_connections 2024-03-28T13:21:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.975+0000 7f88b7a00640 1 -- 172.21.15.19:0/79466929 >> 172.21.15.19:0/79466929 conn(0x7f88b00720d0 msgr2=0x7f88b00b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.975+0000 7f88b7a00640 1 -- 172.21.15.19:0/79466929 shutdown_connections 2024-03-28T13:21:54.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:54.975+0000 7f88b7a00640 1 -- 172.21.15.19:0/79466929 wait complete. 2024-03-28T13:21:54.992 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-28T13:21:55.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.217+0000 7fd933e00640 1 Processor -- start 2024-03-28T13:21:55.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.217+0000 7fd933e00640 1 -- start start 2024-03-28T13:21:55.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.218+0000 7fd933e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd93415e4a0 0x7fd9340b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.219+0000 7fd933e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9340cf700 0x7fd9340b73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.219+0000 7fd933e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd93415e880 con 0x7fd9340d2310 2024-03-28T13:21:55.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.219+0000 7fd933e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fd9340d34e0 con 0x7fd93415e4a0 2024-03-28T13:21:55.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.219+0000 7fd933e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd9340d28d0 con 0x7fd9340cf700 2024-03-28T13:21:55.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.219+0000 7fd932000640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fd9340d2310 0x7fd9340ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:34296/0 (socket says 172.21.15.19:34296) 2024-03-28T13:21:55.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.219+0000 7fd932000640 1 -- 172.21.15.19:0/2053556788 learned_addr learned my addr 172.21.15.19:0/2053556788 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:55.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.219+0000 7fd933400640 1 --2- 172.21.15.19:0/2053556788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9340cf700 0x7fd9340b73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.219+0000 7fd932a00640 1 --2- 172.21.15.19:0/2053556788 >> v2:172.21.15.188:3300/0 conn(0x7fd93415e4a0 0x7fd9340b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.220+0000 7fd931600640 1 -- 172.21.15.19:0/2053556788 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2364880474 0 0) 0x7fd93415e880 con 0x7fd9340d2310 2024-03-28T13:21:55.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.220+0000 7fd931600640 1 -- 172.21.15.19:0/2053556788 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd918003520 con 0x7fd9340d2310 2024-03-28T13:21:55.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.220+0000 7fd933400640 1 -- 172.21.15.19:0/2053556788 >> v2:172.21.15.188:3300/0 conn(0x7fd93415e4a0 msgr2=0x7fd9340b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:55.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.220+0000 7fd933400640 1 --2- 172.21.15.19:0/2053556788 >> v2:172.21.15.188:3300/0 conn(0x7fd93415e4a0 0x7fd9340b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.220+0000 7fd931600640 1 -- 172.21.15.19:0/2053556788 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2709266078 0 0) 0x7fd918003520 con 0x7fd9340d2310 2024-03-28T13:21:55.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.220+0000 7fd933400640 1 -- 172.21.15.19:0/2053556788 >> v1:172.21.15.19:6789/0 conn(0x7fd9340d2310 legacy=0x7fd9340ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.220+0000 7fd933400640 1 -- 172.21.15.19:0/2053556788 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9340d2bb0 con 0x7fd9340cf700 2024-03-28T13:21:55.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.222+0000 7fd933400640 1 --2- 172.21.15.19:0/2053556788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9340cf700 0x7fd9340b73e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd9340d3660 tx=0x7fd92c02bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1c41ef29ec120d server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.222+0000 7fd931600640 1 -- 172.21.15.19:0/2053556788 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c039070 con 0x7fd9340cf700 2024-03-28T13:21:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.222+0000 7fd931600640 1 -- 172.21.15.19:0/2053556788 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd92c02c810 con 0x7fd9340cf700 2024-03-28T13:21:55.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.222+0000 7fd931600640 1 -- 172.21.15.19:0/2053556788 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd920001670 con 0x7fd9340cf700 2024-03-28T13:21:55.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.223+0000 7fd933e00640 1 -- 172.21.15.19:0/2053556788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9340cf700 msgr2=0x7fd9340b73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.223+0000 7fd933e00640 1 --2- 172.21.15.19:0/2053556788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9340cf700 0x7fd9340b73e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd9340d3660 tx=0x7fd92c02bce0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.223+0000 7fd933e00640 1 -- 172.21.15.19:0/2053556788 shutdown_connections 2024-03-28T13:21:55.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.223+0000 7fd933e00640 1 --2- 172.21.15.19:0/2053556788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9340cf700 0x7fd9340b73e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.223+0000 7fd933e00640 1 --2- 172.21.15.19:0/2053556788 >> v2:172.21.15.188:3300/0 conn(0x7fd93415e4a0 0x7fd9340b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.223+0000 7fd933e00640 1 -- 172.21.15.19:0/2053556788 >> 172.21.15.19:0/2053556788 conn(0x7fd9340676d0 msgr2=0x7fd934067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:55.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.223+0000 7fd933e00640 1 -- 172.21.15.19:0/2053556788 shutdown_connections 2024-03-28T13:21:55.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.223+0000 7fd933e00640 1 -- 172.21.15.19:0/2053556788 wait complete. 2024-03-28T13:21:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.225+0000 7fd933e00640 1 Processor -- start 2024-03-28T13:21:55.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.225+0000 7fd933e00640 1 -- start start 2024-03-28T13:21:55.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.226+0000 7fd933e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9340cf700 0x7fd9340b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.227+0000 7fd933e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9340d2310 0x7fd9341613c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.227+0000 7fd933e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd93415e4a0 0x7fd93415d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.227+0000 7fd933e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd9340d3970 con 0x7fd9340cf700 2024-03-28T13:21:55.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.227+0000 7fd933e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd9340d3170 con 0x7fd93415e4a0 2024-03-28T13:21:55.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.227+0000 7fd933e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd9340ba1a0 con 0x7fd9340d2310 2024-03-28T13:21:55.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.227+0000 7fd932a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9340cf700 0x7fd9340b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.227+0000 7fd932a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9340cf700 0x7fd9340b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44494/0 (socket says 172.21.15.19:44494) 2024-03-28T13:21:55.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.227+0000 7fd932a00640 1 -- 172.21.15.19:0/2746632491 learned_addr learned my addr 172.21.15.19:0/2746632491 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:55.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.227+0000 7fd933400640 1 --2- 172.21.15.19:0/2746632491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd93415e4a0 0x7fd93415d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.227+0000 7fd932000640 1 --2- 172.21.15.19:0/2746632491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9340d2310 0x7fd9341613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.227+0000 7fd932a00640 1 -- 172.21.15.19:0/2746632491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9340d2310 msgr2=0x7fd9341613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.227+0000 7fd932a00640 1 --2- 172.21.15.19:0/2746632491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9340d2310 0x7fd9341613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.227+0000 7fd932a00640 1 -- 172.21.15.19:0/2746632491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd93415e4a0 msgr2=0x7fd93415d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.227+0000 7fd932a00640 1 --2- 172.21.15.19:0/2746632491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd93415e4a0 0x7fd93415d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.228+0000 7fd932a00640 1 -- 172.21.15.19:0/2746632491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9340c82a0 con 0x7fd9340cf700 2024-03-28T13:21:55.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.228+0000 7fd932a00640 1 --2- 172.21.15.19:0/2746632491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9340cf700 0x7fd9340b3490 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fd92400e9b0 tx=0x7fd92400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:55.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.228+0000 7fd92aa00640 1 -- 172.21.15.19:0/2746632491 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924017070 con 0x7fd9340cf700 2024-03-28T13:21:55.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.228+0000 7fd92aa00640 1 -- 172.21.15.19:0/2746632491 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd92400c9a0 con 0x7fd9340cf700 2024-03-28T13:21:55.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.228+0000 7fd92aa00640 1 -- 172.21.15.19:0/2746632491 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92400cd90 con 0x7fd9340cf700 2024-03-28T13:21:55.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.228+0000 7fd933e00640 1 -- 172.21.15.19:0/2746632491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9340c8580 con 0x7fd9340cf700 2024-03-28T13:21:55.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.228+0000 7fd933e00640 1 -- 172.21.15.19:0/2746632491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9340c8b40 con 0x7fd9340cf700 2024-03-28T13:21:55.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.229+0000 7fd933e00640 1 -- 172.21.15.19:0/2746632491 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd934075940 con 0x7fd9340cf700 2024-03-28T13:21:55.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.230+0000 7fd92aa00640 1 -- 172.21.15.19:0/2746632491 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fd92401a070 con 0x7fd9340cf700 2024-03-28T13:21:55.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.231+0000 7fd92aa00640 1 --2- 172.21.15.19:0/2746632491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd91406c530 0x7fd91406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.231+0000 7fd92aa00640 1 -- 172.21.15.19:0/2746632491 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7fd92408b7a0 con 0x7fd9340cf700 2024-03-28T13:21:55.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.232+0000 7fd932000640 1 --2- 172.21.15.19:0/2746632491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd91406c530 0x7fd91406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.233+0000 7fd932000640 1 --2- 172.21.15.19:0/2746632491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd91406c530 0x7fd91406e9f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd934159f00 tx=0x7fd920009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:55.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.236+0000 7fd92aa00640 1 -- 172.21.15.19:0/2746632491 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fd92405a0a0 con 0x7fd9340cf700 2024-03-28T13:21:55.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.442+0000 7fd933e00640 1 -- 172.21.15.19:0/2746632491 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fd9340d3e00 con 0x7fd91406c530 2024-03-28T13:21:55.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.443+0000 7fd92aa00640 1 -- 172.21.15.19:0/2746632491 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd9340d3e00 con 0x7fd91406c530 2024-03-28T13:21:55.442 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 0 to deep-scrub 2024-03-28T13:21:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.446+0000 7fd933e00640 1 -- 172.21.15.19:0/2746632491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd91406c530 msgr2=0x7fd91406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.446+0000 7fd933e00640 1 --2- 172.21.15.19:0/2746632491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd91406c530 0x7fd91406e9f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd934159f00 tx=0x7fd920009040 comp rx=0 tx=0).stop 2024-03-28T13:21:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.446+0000 7fd933e00640 1 -- 172.21.15.19:0/2746632491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9340cf700 msgr2=0x7fd9340b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.446+0000 7fd933e00640 1 --2- 172.21.15.19:0/2746632491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9340cf700 0x7fd9340b3490 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fd92400e9b0 tx=0x7fd92400ee70 comp rx=0 tx=0).stop 2024-03-28T13:21:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.446+0000 7fd933e00640 1 -- 172.21.15.19:0/2746632491 shutdown_connections 2024-03-28T13:21:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.446+0000 7fd933e00640 1 --2- 172.21.15.19:0/2746632491 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fd91406c530 0x7fd91406e9f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.446+0000 7fd933e00640 1 --2- 172.21.15.19:0/2746632491 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd93415e4a0 0x7fd93415d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.446+0000 7fd933e00640 1 --2- 172.21.15.19:0/2746632491 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9340d2310 0x7fd9341613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.446+0000 7fd933e00640 1 --2- 172.21.15.19:0/2746632491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd9340cf700 0x7fd9340b3490 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.446+0000 7fd933e00640 1 -- 172.21.15.19:0/2746632491 >> 172.21.15.19:0/2746632491 conn(0x7fd9340676d0 msgr2=0x7fd9340ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.446+0000 7fd933e00640 1 -- 172.21.15.19:0/2746632491 shutdown_connections 2024-03-28T13:21:55.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.446+0000 7fd933e00640 1 -- 172.21.15.19:0/2746632491 wait complete. 2024-03-28T13:21:55.465 INFO:tasks.ceph:Scrubbing osd.1 2024-03-28T13:21:55.466 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-28T13:21:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.694+0000 7f5f17200640 1 Processor -- start 2024-03-28T13:21:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.694+0000 7f5f17200640 1 -- start start 2024-03-28T13:21:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.696+0000 7f5f17200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f18141890 0x7f5f18141cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.696+0000 7f5f17200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5f1806db90 0x7f5f1806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.696+0000 7f5f17200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f180cfbf0 con 0x7f5f1813f190 2024-03-28T13:21:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.696+0000 7f5f17200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5f1813f570 con 0x7f5f1806db90 2024-03-28T13:21:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.696+0000 7f5f17200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5f18142230 con 0x7f5f18141890 2024-03-28T13:21:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.697+0000 7f5f0d400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f18141890 0x7f5f18141cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.697+0000 7f5f16800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5f1806db90 0x7f5f1806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.697+0000 7f5f0d400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f18141890 0x7f5f18141cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:40590/0 (socket says 172.21.15.19:40590) 2024-03-28T13:21:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.697+0000 7f5f0d400640 1 -- 172.21.15.19:0/3458483282 learned_addr learned my addr 172.21.15.19:0/3458483282 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.697+0000 7f5f16800640 1 --2- 172.21.15.19:0/3458483282 >> v2:172.21.15.188:3300/0 conn(0x7f5f1806db90 0x7f5f1806e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:39036/0 (socket says 172.21.15.19:39036) 2024-03-28T13:21:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.698+0000 7f5f16800640 1 -- 172.21.15.19:0/3458483282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f18141890 msgr2=0x7f5f18141cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.698+0000 7f5f16800640 1 --2- 172.21.15.19:0/3458483282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f18141890 0x7f5f18141cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.698+0000 7f5f16800640 1 -- 172.21.15.19:0/3458483282 >> v1:172.21.15.188:6789/0 conn(0x7f5f1813f190 legacy=0x7f5f180ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.698+0000 7f5f16800640 1 -- 172.21.15.19:0/3458483282 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f180cf350 con 0x7f5f1806db90 2024-03-28T13:21:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.699+0000 7f5f16800640 1 --2- 172.21.15.19:0/3458483282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f1806db90 0x7f5f1806e000 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009d90 tx=0x7f5f1002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfe55d8e77305c6e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.700+0000 7f5f15400640 1 -- 172.21.15.19:0/3458483282 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10039070 con 0x7f5f1806db90 2024-03-28T13:21:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.700+0000 7f5f15400640 1 -- 172.21.15.19:0/3458483282 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1002cbc0 con 0x7f5f1806db90 2024-03-28T13:21:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.700+0000 7f5f15400640 1 -- 172.21.15.19:0/3458483282 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10034440 con 0x7f5f1806db90 2024-03-28T13:21:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.701+0000 7f5f17200640 1 -- 172.21.15.19:0/3458483282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f1806db90 msgr2=0x7f5f1806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.701+0000 7f5f17200640 1 --2- 172.21.15.19:0/3458483282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f1806db90 0x7f5f1806e000 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009d90 tx=0x7f5f1002be10 comp rx=0 tx=0).stop 2024-03-28T13:21:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.701+0000 7f5f17200640 1 -- 172.21.15.19:0/3458483282 shutdown_connections 2024-03-28T13:21:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.701+0000 7f5f17200640 1 --2- 172.21.15.19:0/3458483282 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f1806db90 0x7f5f1806e000 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.701+0000 7f5f17200640 1 --2- 172.21.15.19:0/3458483282 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f18141890 0x7f5f18141cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.701+0000 7f5f17200640 1 -- 172.21.15.19:0/3458483282 >> 172.21.15.19:0/3458483282 conn(0x7f5f180b1d80 msgr2=0x7f5f180b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.701+0000 7f5f17200640 1 -- 172.21.15.19:0/3458483282 shutdown_connections 2024-03-28T13:21:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.701+0000 7f5f17200640 1 -- 172.21.15.19:0/3458483282 wait complete. 2024-03-28T13:21:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.703+0000 7f5f17200640 1 Processor -- start 2024-03-28T13:21:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.704+0000 7f5f17200640 1 -- start start 2024-03-28T13:21:55.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.705+0000 7f5f17200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f1806db90 0x7f5f18137e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.705+0000 7f5f17200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f1813f190 0x7f5f18071920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.705+0000 7f5f17200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f18141890 0x7f5f18159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.705+0000 7f5f17200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1806d670 con 0x7f5f18141890 2024-03-28T13:21:55.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.705+0000 7f5f17200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5f180cf910 con 0x7f5f1806db90 2024-03-28T13:21:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.705+0000 7f5f17200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5f18142550 con 0x7f5f1813f190 2024-03-28T13:21:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.705+0000 7f5f16800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f18141890 0x7f5f18159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.705+0000 7f5f16800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f18141890 0x7f5f18159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45798/0 (socket says 172.21.15.19:45798) 2024-03-28T13:21:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.705+0000 7f5f16800640 1 -- 172.21.15.19:0/3247125861 learned_addr learned my addr 172.21.15.19:0/3247125861 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.705+0000 7f5f15e00640 1 --2- 172.21.15.19:0/3247125861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f1806db90 0x7f5f18137e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.706+0000 7f5f0d400640 1 --2- 172.21.15.19:0/3247125861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f1813f190 0x7f5f18071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.706+0000 7f5f16800640 1 -- 172.21.15.19:0/3247125861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f1813f190 msgr2=0x7f5f18071920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.706+0000 7f5f16800640 1 --2- 172.21.15.19:0/3247125861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f1813f190 0x7f5f18071920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.706+0000 7f5f16800640 1 -- 172.21.15.19:0/3247125861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f1806db90 msgr2=0x7f5f18137e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.706+0000 7f5f16800640 1 --2- 172.21.15.19:0/3247125861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f1806db90 0x7f5f18137e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.706+0000 7f5f16800640 1 -- 172.21.15.19:0/3247125861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f18071e60 con 0x7f5f18141890 2024-03-28T13:21:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.706+0000 7f5f16800640 1 --2- 172.21.15.19:0/3247125861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f18141890 0x7f5f18159390 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009d90 tx=0x7f5f10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:55.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.706+0000 7f5f0ea00640 1 -- 172.21.15.19:0/3247125861 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10044030 con 0x7f5f18141890 2024-03-28T13:21:55.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.706+0000 7f5f17200640 1 -- 172.21.15.19:0/3247125861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f181395e0 con 0x7f5f18141890 2024-03-28T13:21:55.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.706+0000 7f5f0ea00640 1 -- 172.21.15.19:0/3247125861 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f1002c9c0 con 0x7f5f18141890 2024-03-28T13:21:55.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.706+0000 7f5f0ea00640 1 -- 172.21.15.19:0/3247125861 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10039040 con 0x7f5f18141890 2024-03-28T13:21:55.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.707+0000 7f5f17200640 1 -- 172.21.15.19:0/3247125861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1813fc30 con 0x7f5f18141890 2024-03-28T13:21:55.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.707+0000 7f5f17200640 1 -- 172.21.15.19:0/3247125861 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5f18075940 con 0x7f5f18141890 2024-03-28T13:21:55.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.708+0000 7f5f0ea00640 1 -- 172.21.15.19:0/3247125861 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f5f10047070 con 0x7f5f18141890 2024-03-28T13:21:55.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.709+0000 7f5f0ea00640 1 --2- 172.21.15.19:0/3247125861 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ef406c530 0x7f5ef406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.709+0000 7f5f0ea00640 1 -- 172.21.15.19:0/3247125861 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7f5f100adc90 con 0x7f5f18141890 2024-03-28T13:21:55.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.710+0000 7f5f0ea00640 1 --2- 172.21.15.19:0/3247125861 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x7f5ef4072530 0x7f5ef4074950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.710+0000 7f5f0ea00640 1 -- 172.21.15.19:0/3247125861 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ef4075000 con 0x7f5ef4072530 2024-03-28T13:21:55.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.710+0000 7f5f0ea00640 1 -- 172.21.15.19:0/3247125861 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f5f100b32a0 con 0x7f5f18141890 2024-03-28T13:21:55.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.710+0000 7f5f0d400640 1 --2- 172.21.15.19:0/3247125861 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x7f5ef4072530 0x7f5ef4074950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.710+0000 7f5f15e00640 1 --2- 172.21.15.19:0/3247125861 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ef406c530 0x7f5ef406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.711+0000 7f5f0d400640 1 --2- 172.21.15.19:0/3247125861 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x7f5ef4072530 0x7f5ef4074950 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:55.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.711+0000 7f5f15e00640 1 --2- 172.21.15.19:0/3247125861 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ef406c530 0x7f5ef406e9f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006fd0 tx=0x7f5f08008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:55.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.712+0000 7f5f0ea00640 1 -- 172.21.15.19:0/3247125861 <== osd.1 v2:172.21.15.19:6801/1998544260 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ef4075000 con 0x7f5ef4072530 2024-03-28T13:21:55.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.727+0000 7f5f17200640 1 -- 172.21.15.19:0/3247125861 --> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5f1815a9e0 con 0x7f5ef4072530 2024-03-28T13:21:55.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.727+0000 7f5f0ea00640 1 -- 172.21.15.19:0/3247125861 <== osd.1 v2:172.21.15.19:6801/1998544260 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5f1815a9e0 con 0x7f5ef4072530 2024-03-28T13:21:55.726 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:21:55.726 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T13:21:55.726 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:21:55.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.727+0000 7f5f17200640 1 -- 172.21.15.19:0/3247125861 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x7f5ef4072530 msgr2=0x7f5ef4074950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.727+0000 7f5f17200640 1 --2- 172.21.15.19:0/3247125861 >> [v2:172.21.15.19:6801/1998544260,v1:172.21.15.19:6803/1998544260] conn(0x7f5ef4072530 0x7f5ef4074950 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.728+0000 7f5f17200640 1 -- 172.21.15.19:0/3247125861 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ef406c530 msgr2=0x7f5ef406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.728+0000 7f5f17200640 1 --2- 172.21.15.19:0/3247125861 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f5ef406c530 0x7f5ef406e9f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5f08006fd0 tx=0x7f5f08008040 comp rx=0 tx=0).stop 2024-03-28T13:21:55.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.728+0000 7f5f17200640 1 -- 172.21.15.19:0/3247125861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f18141890 msgr2=0x7f5f18159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.728+0000 7f5f17200640 1 --2- 172.21.15.19:0/3247125861 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5f18141890 0x7f5f18159390 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009d90 tx=0x7f5f10004a40 comp rx=0 tx=0).stop 2024-03-28T13:21:55.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.728+0000 7f5f16800640 1 -- 172.21.15.19:0/3247125861 reap_dead start 2024-03-28T13:21:55.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.728+0000 7f5f17200640 1 -- 172.21.15.19:0/3247125861 shutdown_connections 2024-03-28T13:21:55.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.728+0000 7f5f17200640 1 -- 172.21.15.19:0/3247125861 >> 172.21.15.19:0/3247125861 conn(0x7f5f180b1d80 msgr2=0x7f5f1815ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:55.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.728+0000 7f5f17200640 1 -- 172.21.15.19:0/3247125861 shutdown_connections 2024-03-28T13:21:55.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.728+0000 7f5f17200640 1 -- 172.21.15.19:0/3247125861 wait complete. 2024-03-28T13:21:55.743 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-28T13:21:55.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.966+0000 7f222aa00640 1 Processor -- start 2024-03-28T13:21:55.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.966+0000 7f222aa00640 1 -- start start 2024-03-28T13:21:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.967+0000 7f222aa00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2224153580 0x7f22240d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.967+0000 7f222aa00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f22240d07c0 0x7f22240aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.967+0000 7f222aa00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22240afab0 con 0x7f22240af2e0 2024-03-28T13:21:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.967+0000 7f222aa00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f22240d0c30 con 0x7f22240d07c0 2024-03-28T13:21:55.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.967+0000 7f222aa00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f222415cc80 con 0x7f2224153580 2024-03-28T13:21:55.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.968+0000 7f221be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f22240d07c0 0x7f22240aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.968+0000 7f2228a00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f22240af2e0 0x7f222406d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33790/0 (socket says 172.21.15.19:33790) 2024-03-28T13:21:55.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.968+0000 7f2228a00640 1 -- 172.21.15.19:0/3163961469 learned_addr learned my addr 172.21.15.19:0/3163961469 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:55.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.968+0000 7f2223e00640 1 --2- 172.21.15.19:0/3163961469 >> v2:172.21.15.188:3300/0 conn(0x7f2224153580 0x7f22240d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.968+0000 7f221be00640 1 -- 172.21.15.19:0/3163961469 >> v2:172.21.15.188:3300/0 conn(0x7f2224153580 msgr2=0x7f22240d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:55.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.968+0000 7f221be00640 1 --2- 172.21.15.19:0/3163961469 >> v2:172.21.15.188:3300/0 conn(0x7f2224153580 0x7f22240d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.968+0000 7f221be00640 1 -- 172.21.15.19:0/3163961469 >> v1:172.21.15.19:6789/0 conn(0x7f22240af2e0 legacy=0x7f222406d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.968+0000 7f221be00640 1 -- 172.21.15.19:0/3163961469 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f222415da00 con 0x7f22240d07c0 2024-03-28T13:21:55.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.969+0000 7f221be00640 1 --2- 172.21.15.19:0/3163961469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22240d07c0 0x7f22240aeda0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f22240b5270 tx=0x7f220c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8e6be4c263dbbd server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:55.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.970+0000 7f2223400640 1 -- 172.21.15.19:0/3163961469 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c0315a0 con 0x7f22240d07c0 2024-03-28T13:21:55.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.970+0000 7f2223400640 1 -- 172.21.15.19:0/3163961469 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f220c037070 con 0x7f22240d07c0 2024-03-28T13:21:55.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.970+0000 7f2223400640 1 -- 172.21.15.19:0/3163961469 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c031960 con 0x7f22240d07c0 2024-03-28T13:21:55.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.970+0000 7f222aa00640 1 -- 172.21.15.19:0/3163961469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22240d07c0 msgr2=0x7f22240aeda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.970+0000 7f222aa00640 1 --2- 172.21.15.19:0/3163961469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22240d07c0 0x7f22240aeda0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f22240b5270 tx=0x7f220c02b840 comp rx=0 tx=0).stop 2024-03-28T13:21:55.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.970+0000 7f222aa00640 1 -- 172.21.15.19:0/3163961469 shutdown_connections 2024-03-28T13:21:55.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.970+0000 7f222aa00640 1 --2- 172.21.15.19:0/3163961469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22240d07c0 0x7f22240aeda0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.970+0000 7f222aa00640 1 --2- 172.21.15.19:0/3163961469 >> v2:172.21.15.188:3300/0 conn(0x7f2224153580 0x7f22240d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.970+0000 7f222aa00640 1 -- 172.21.15.19:0/3163961469 >> 172.21.15.19:0/3163961469 conn(0x7f222406b750 msgr2=0x7f222406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:55.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.970+0000 7f222aa00640 1 -- 172.21.15.19:0/3163961469 shutdown_connections 2024-03-28T13:21:55.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.970+0000 7f222aa00640 1 -- 172.21.15.19:0/3163961469 wait complete. 2024-03-28T13:21:55.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.972+0000 7f222aa00640 1 Processor -- start 2024-03-28T13:21:55.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.972+0000 7f222aa00640 1 -- start start 2024-03-28T13:21:55.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.973+0000 7f222aa00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22240af2e0 0x7f22241561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.973+0000 7f222aa00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22240d07c0 0x7f22241421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.973+0000 7f222aa00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2224153580 0x7f2224159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.973+0000 7f222aa00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f222415d4e0 con 0x7f22240af2e0 2024-03-28T13:21:55.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.973+0000 7f222aa00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f222415d200 con 0x7f2224153580 2024-03-28T13:21:55.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.973+0000 7f222aa00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2224153960 con 0x7f22240d07c0 2024-03-28T13:21:55.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.973+0000 7f2223e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22240af2e0 0x7f22241561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.974+0000 7f2223e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22240af2e0 0x7f22241561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45828/0 (socket says 172.21.15.19:45828) 2024-03-28T13:21:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.974+0000 7f2223e00640 1 -- 172.21.15.19:0/277351314 learned_addr learned my addr 172.21.15.19:0/277351314 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.974+0000 7f2228a00640 1 --2- 172.21.15.19:0/277351314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2224153580 0x7f2224159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.974+0000 7f2223e00640 1 -- 172.21.15.19:0/277351314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22240d07c0 msgr2=0x7f22241421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.974+0000 7f221be00640 1 --2- 172.21.15.19:0/277351314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22240d07c0 0x7f22241421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.974+0000 7f2223e00640 1 --2- 172.21.15.19:0/277351314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22240d07c0 0x7f22241421d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.974+0000 7f2223e00640 1 -- 172.21.15.19:0/277351314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2224153580 msgr2=0x7f2224159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.974+0000 7f2223e00640 1 --2- 172.21.15.19:0/277351314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2224153580 0x7f2224159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.974+0000 7f2223e00640 1 -- 172.21.15.19:0/277351314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2224155110 con 0x7f22240af2e0 2024-03-28T13:21:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.974+0000 7f2228a00640 1 --2- 172.21.15.19:0/277351314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2224153580 0x7f2224159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.974+0000 7f2223e00640 1 --2- 172.21.15.19:0/277351314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22240af2e0 0x7f22241561b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f221400d800 tx=0x7f221400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.975+0000 7f2220c00640 1 -- 172.21.15.19:0/277351314 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221400b6c0 con 0x7f22240af2e0 2024-03-28T13:21:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.975+0000 7f2220c00640 1 -- 172.21.15.19:0/277351314 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2214019070 con 0x7f22240af2e0 2024-03-28T13:21:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.975+0000 7f2220c00640 1 -- 172.21.15.19:0/277351314 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221400ba80 con 0x7f22240af2e0 2024-03-28T13:21:55.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.975+0000 7f222aa00640 1 -- 172.21.15.19:0/277351314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22241553f0 con 0x7f22240af2e0 2024-03-28T13:21:55.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.975+0000 7f222aa00640 1 -- 172.21.15.19:0/277351314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22241559b0 con 0x7f22240af2e0 2024-03-28T13:21:55.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.976+0000 7f222aa00640 1 -- 172.21.15.19:0/277351314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f222415aec0 con 0x7f22240af2e0 2024-03-28T13:21:55.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.977+0000 7f2220c00640 1 -- 172.21.15.19:0/277351314 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f221400bc20 con 0x7f22240af2e0 2024-03-28T13:21:55.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.978+0000 7f2220c00640 1 --2- 172.21.15.19:0/277351314 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f220006c560 0x7f220006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:55.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.978+0000 7f2220c00640 1 -- 172.21.15.19:0/277351314 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7f221408b910 con 0x7f22240af2e0 2024-03-28T13:21:55.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.979+0000 7f221be00640 1 --2- 172.21.15.19:0/277351314 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f220006c560 0x7f220006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:55.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.980+0000 7f221be00640 1 --2- 172.21.15.19:0/277351314 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f220006c560 0x7f220006ea20 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f220c036000 tx=0x7f220c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:55.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:55.982+0000 7f2220c00640 1 -- 172.21.15.19:0/277351314 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f222415aec0 con 0x7f22240af2e0 2024-03-28T13:21:56.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.185+0000 7f222aa00640 1 -- 172.21.15.19:0/277351314 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f22240685c0 con 0x7f220006c560 2024-03-28T13:21:56.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.186+0000 7f2220c00640 1 -- 172.21.15.19:0/277351314 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f22240685c0 con 0x7f220006c560 2024-03-28T13:21:56.185 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 1 to deep-scrub 2024-03-28T13:21:56.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.189+0000 7f222aa00640 1 -- 172.21.15.19:0/277351314 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f220006c560 msgr2=0x7f220006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:56.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.189+0000 7f222aa00640 1 --2- 172.21.15.19:0/277351314 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f220006c560 0x7f220006ea20 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f220c036000 tx=0x7f220c008030 comp rx=0 tx=0).stop 2024-03-28T13:21:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.189+0000 7f222aa00640 1 -- 172.21.15.19:0/277351314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22240af2e0 msgr2=0x7f22241561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.189+0000 7f222aa00640 1 --2- 172.21.15.19:0/277351314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22240af2e0 0x7f22241561b0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f221400d800 tx=0x7f221400dcc0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.189+0000 7f222aa00640 1 -- 172.21.15.19:0/277351314 shutdown_connections 2024-03-28T13:21:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.189+0000 7f222aa00640 1 --2- 172.21.15.19:0/277351314 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f220006c560 0x7f220006ea20 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.189+0000 7f222aa00640 1 --2- 172.21.15.19:0/277351314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2224153580 0x7f2224159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.189+0000 7f222aa00640 1 --2- 172.21.15.19:0/277351314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22240d07c0 0x7f22241421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.189+0000 7f222aa00640 1 --2- 172.21.15.19:0/277351314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f22240af2e0 0x7f22241561b0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.189+0000 7f222aa00640 1 -- 172.21.15.19:0/277351314 >> 172.21.15.19:0/277351314 conn(0x7f222406b750 msgr2=0x7f222406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.189+0000 7f222aa00640 1 -- 172.21.15.19:0/277351314 shutdown_connections 2024-03-28T13:21:56.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.189+0000 7f222aa00640 1 -- 172.21.15.19:0/277351314 wait complete. 2024-03-28T13:21:56.206 INFO:tasks.ceph:Scrubbing osd.2 2024-03-28T13:21:56.206 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-28T13:21:56.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.437+0000 7ff7bf000640 1 Processor -- start 2024-03-28T13:21:56.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.438+0000 7ff7bf000640 1 -- start start 2024-03-28T13:21:56.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.439+0000 7ff7bf000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff7b8072720 0x7ff7b80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:56.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.439+0000 7ff7bf000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff7b8072b00 0x7ff7b806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:56.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.439+0000 7ff7bf000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7b806db10 0x7ff7b806df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:56.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.439+0000 7ff7bf000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff7b80b3e80 con 0x7ff7b8072b00 2024-03-28T13:21:56.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.439+0000 7ff7bf000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff7b806ea10 con 0x7ff7b8072720 2024-03-28T13:21:56.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.439+0000 7ff7bf000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b806ecf0 con 0x7ff7b806db10 2024-03-28T13:21:56.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.440+0000 7ff7b7400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff7b8072b00 0x7ff7b806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:56.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.440+0000 7ff7b7400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff7b8072b00 0x7ff7b806d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45836/0 (socket says 172.21.15.19:45836) 2024-03-28T13:21:56.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.440+0000 7ff7b7400640 1 -- 172.21.15.19:0/59289299 learned_addr learned my addr 172.21.15.19:0/59289299 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:56.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.440+0000 7ff7b7e00640 1 --2- 172.21.15.19:0/59289299 >> v2:172.21.15.188:3300/0 conn(0x7ff7b8072720 0x7ff7b80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:56.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.440+0000 7ff7bd000640 1 --2- 172.21.15.19:0/59289299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7b806db10 0x7ff7b806df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:56.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.440+0000 7ff7b7400640 1 -- 172.21.15.19:0/59289299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7b806db10 msgr2=0x7ff7b806df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:56.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.440+0000 7ff7b7400640 1 --2- 172.21.15.19:0/59289299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7b806db10 0x7ff7b806df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.440+0000 7ff7b7400640 1 -- 172.21.15.19:0/59289299 >> v2:172.21.15.188:3300/0 conn(0x7ff7b8072720 msgr2=0x7ff7b80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:56.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.440+0000 7ff7b7400640 1 --2- 172.21.15.19:0/59289299 >> v2:172.21.15.188:3300/0 conn(0x7ff7b8072720 0x7ff7b80b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.440+0000 7ff7b7400640 1 -- 172.21.15.19:0/59289299 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b806efd0 con 0x7ff7b8072b00 2024-03-28T13:21:56.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.440+0000 7ff7b7e00640 1 --2- 172.21.15.19:0/59289299 >> v2:172.21.15.188:3300/0 conn(0x7ff7b8072720 0x7ff7b80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:56.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.441+0000 7ff7b7400640 1 --2- 172.21.15.19:0/59289299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b8072b00 0x7ff7b806d5d0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8073390 tx=0x7ff7a002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53cdf4309dba00e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:56.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.442+0000 7ff7b6a00640 1 -- 172.21.15.19:0/59289299 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0035020 con 0x7ff7b8072b00 2024-03-28T13:21:56.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.442+0000 7ff7b6a00640 1 -- 172.21.15.19:0/59289299 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7a0031870 con 0x7ff7b8072b00 2024-03-28T13:21:56.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.442+0000 7ff7b6a00640 1 -- 172.21.15.19:0/59289299 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0031c40 con 0x7ff7b8072b00 2024-03-28T13:21:56.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.443+0000 7ff7bf000640 1 -- 172.21.15.19:0/59289299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b8072b00 msgr2=0x7ff7b806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.443+0000 7ff7bf000640 1 --2- 172.21.15.19:0/59289299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b8072b00 0x7ff7b806d5d0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8073390 tx=0x7ff7a002b910 comp rx=0 tx=0).stop 2024-03-28T13:21:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.443+0000 7ff7bf000640 1 -- 172.21.15.19:0/59289299 shutdown_connections 2024-03-28T13:21:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.443+0000 7ff7bf000640 1 --2- 172.21.15.19:0/59289299 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7b806db10 0x7ff7b806df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.443+0000 7ff7bf000640 1 --2- 172.21.15.19:0/59289299 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b8072b00 0x7ff7b806d5d0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.443+0000 7ff7bf000640 1 --2- 172.21.15.19:0/59289299 >> v2:172.21.15.188:3300/0 conn(0x7ff7b8072720 0x7ff7b80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.443+0000 7ff7bf000640 1 -- 172.21.15.19:0/59289299 >> 172.21.15.19:0/59289299 conn(0x7ff7b806bba0 msgr2=0x7ff7b80b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.443+0000 7ff7bf000640 1 -- 172.21.15.19:0/59289299 shutdown_connections 2024-03-28T13:21:56.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.443+0000 7ff7bf000640 1 -- 172.21.15.19:0/59289299 wait complete. 2024-03-28T13:21:56.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.445+0000 7ff7bf000640 1 Processor -- start 2024-03-28T13:21:56.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.445+0000 7ff7bf000640 1 -- start start 2024-03-28T13:21:56.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.446+0000 7ff7bf000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7b806db10 0x7ff7b8167870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:56.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.446+0000 7ff7bf000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7b8072720 0x7ff7b81403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:56.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.446+0000 7ff7bf000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b8072b00 0x7ff7b814c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:56.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.446+0000 7ff7bf000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b806e690 con 0x7ff7b8072b00 2024-03-28T13:21:56.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.446+0000 7ff7bf000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b806f590 con 0x7ff7b806db10 2024-03-28T13:21:56.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.446+0000 7ff7bf000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b8072f70 con 0x7ff7b8072720 2024-03-28T13:21:56.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.446+0000 7ff7bd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b8072b00 0x7ff7b814c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:56.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.446+0000 7ff7bd000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b8072b00 0x7ff7b814c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45850/0 (socket says 172.21.15.19:45850) 2024-03-28T13:21:56.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.446+0000 7ff7bd000640 1 -- 172.21.15.19:0/2665948384 learned_addr learned my addr 172.21.15.19:0/2665948384 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:56.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.446+0000 7ff7b7400640 1 --2- 172.21.15.19:0/2665948384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7b8072720 0x7ff7b81403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:56.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.446+0000 7ff7b7e00640 1 --2- 172.21.15.19:0/2665948384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7b806db10 0x7ff7b8167870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:56.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.447+0000 7ff7bd000640 1 -- 172.21.15.19:0/2665948384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7b8072720 msgr2=0x7ff7b81403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:56.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.447+0000 7ff7bd000640 1 --2- 172.21.15.19:0/2665948384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff7b8072720 0x7ff7b81403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.447+0000 7ff7bd000640 1 -- 172.21.15.19:0/2665948384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7b806db10 msgr2=0x7ff7b8167870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:56.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.447+0000 7ff7bd000640 1 --2- 172.21.15.19:0/2665948384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff7b806db10 0x7ff7b8167870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.447+0000 7ff7bd000640 1 -- 172.21.15.19:0/2665948384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b8162f60 con 0x7ff7b8072b00 2024-03-28T13:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.447+0000 7ff7bd000640 1 --2- 172.21.15.19:0/2665948384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b8072b00 0x7ff7b814c9c0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000cb80 tx=0x7ff7b000a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.447+0000 7ff7afe00640 1 -- 172.21.15.19:0/2665948384 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b0015070 con 0x7ff7b8072b00 2024-03-28T13:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.447+0000 7ff7afe00640 1 -- 172.21.15.19:0/2665948384 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b000abd0 con 0x7ff7b8072b00 2024-03-28T13:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.447+0000 7ff7afe00640 1 -- 172.21.15.19:0/2665948384 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000d040 con 0x7ff7b8072b00 2024-03-28T13:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.447+0000 7ff7bf000640 1 -- 172.21.15.19:0/2665948384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b8151310 con 0x7ff7b8072b00 2024-03-28T13:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.448+0000 7ff7bf000640 1 -- 172.21.15.19:0/2665948384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b81518d0 con 0x7ff7b8072b00 2024-03-28T13:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.449+0000 7ff7bf000640 1 -- 172.21.15.19:0/2665948384 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff7b81510c0 con 0x7ff7b8072b00 2024-03-28T13:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.450+0000 7ff7afe00640 1 -- 172.21.15.19:0/2665948384 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff7b0018070 con 0x7ff7b8072b00 2024-03-28T13:21:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.450+0000 7ff7afe00640 1 --2- 172.21.15.19:0/2665948384 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff79406c530 0x7ff79406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.451+0000 7ff7afe00640 1 -- 172.21.15.19:0/2665948384 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7ff7b008b0c0 con 0x7ff7b8072b00 2024-03-28T13:21:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.451+0000 7ff7afe00640 1 --2- 172.21.15.19:0/2665948384 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x7ff794072530 0x7ff794074950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.451+0000 7ff7afe00640 1 -- 172.21.15.19:0/2665948384 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff794075000 con 0x7ff794072530 2024-03-28T13:21:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.451+0000 7ff7afe00640 1 -- 172.21.15.19:0/2665948384 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7ff7b0092050 con 0x7ff7b8072b00 2024-03-28T13:21:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.451+0000 7ff7b7400640 1 --2- 172.21.15.19:0/2665948384 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x7ff794072530 0x7ff794074950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.451+0000 7ff7b7e00640 1 --2- 172.21.15.19:0/2665948384 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff79406c530 0x7ff79406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.452+0000 7ff7b7400640 1 --2- 172.21.15.19:0/2665948384 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x7ff794072530 0x7ff794074950 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.452+0000 7ff7b7e00640 1 --2- 172.21.15.19:0/2665948384 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff79406c530 0x7ff79406e9f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8148360 tx=0x7ff7a8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:56.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.453+0000 7ff7afe00640 1 -- 172.21.15.19:0/2665948384 <== osd.2 v2:172.21.15.19:6816/1693606238 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff794075000 con 0x7ff794072530 2024-03-28T13:21:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.467+0000 7ff7bf000640 1 -- 172.21.15.19:0/2665948384 --> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff7b80685c0 con 0x7ff794072530 2024-03-28T13:21:56.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.468+0000 7ff7afe00640 1 -- 172.21.15.19:0/2665948384 <== osd.2 v2:172.21.15.19:6816/1693606238 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff7b80685c0 con 0x7ff794072530 2024-03-28T13:21:56.467 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:21:56.467 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T13:21:56.467 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:21:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.468+0000 7ff7bf000640 1 -- 172.21.15.19:0/2665948384 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x7ff794072530 msgr2=0x7ff794074950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.468+0000 7ff7bf000640 1 --2- 172.21.15.19:0/2665948384 >> [v2:172.21.15.19:6816/1693606238,v1:172.21.15.19:6817/1693606238] conn(0x7ff794072530 0x7ff794074950 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.468+0000 7ff7bf000640 1 -- 172.21.15.19:0/2665948384 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff79406c530 msgr2=0x7ff79406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.468+0000 7ff7bf000640 1 --2- 172.21.15.19:0/2665948384 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff79406c530 0x7ff79406e9f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8148360 tx=0x7ff7a8008040 comp rx=0 tx=0).stop 2024-03-28T13:21:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.468+0000 7ff7bf000640 1 -- 172.21.15.19:0/2665948384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b8072b00 msgr2=0x7ff7b814c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.468+0000 7ff7bf000640 1 --2- 172.21.15.19:0/2665948384 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff7b8072b00 0x7ff7b814c9c0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000cb80 tx=0x7ff7b000a4c0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.469+0000 7ff7bd000640 1 -- 172.21.15.19:0/2665948384 reap_dead start 2024-03-28T13:21:56.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.469+0000 7ff7bf000640 1 -- 172.21.15.19:0/2665948384 shutdown_connections 2024-03-28T13:21:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.469+0000 7ff7bf000640 1 -- 172.21.15.19:0/2665948384 >> 172.21.15.19:0/2665948384 conn(0x7ff7b806bba0 msgr2=0x7ff7b80b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.469+0000 7ff7bf000640 1 -- 172.21.15.19:0/2665948384 shutdown_connections 2024-03-28T13:21:56.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.469+0000 7ff7bf000640 1 -- 172.21.15.19:0/2665948384 wait complete. 2024-03-28T13:21:56.489 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-28T13:21:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.711+0000 7f5216e00640 1 Processor -- start 2024-03-28T13:21:56.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.711+0000 7f5216e00640 1 -- start start 2024-03-28T13:21:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.713+0000 7f5216e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f521015f6e0 0x7f52100d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.713+0000 7f5216e00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52100cf6c0 con 0x7f52100cf050 2024-03-28T13:21:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.713+0000 7f5216e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52100cf9a0 con 0x7f52100d0e70 2024-03-28T13:21:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.713+0000 7f5216e00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f52100cfc80 con 0x7f521015f6e0 2024-03-28T13:21:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.713+0000 7f520f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f52100cf050 0x7f52100d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33800/0 (socket says 172.21.15.19:33800) 2024-03-28T13:21:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.713+0000 7f5214e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f521015f6e0 0x7f52100d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:56.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.713+0000 7f520f400640 1 -- 172.21.15.19:0/1260517198 learned_addr learned my addr 172.21.15.19:0/1260517198 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:56.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.714+0000 7f5214e00640 1 -- 172.21.15.19:0/1260517198 >> v1:172.21.15.188:6789/0 conn(0x7f52100d0e70 legacy=0x7f52100d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:56.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.714+0000 7f5214e00640 1 -- 172.21.15.19:0/1260517198 >> v1:172.21.15.19:6789/0 conn(0x7f52100cf050 legacy=0x7f52100d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:56.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.714+0000 7f5214e00640 1 -- 172.21.15.19:0/1260517198 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52100cff60 con 0x7f521015f6e0 2024-03-28T13:21:56.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.714+0000 7f520ea00640 1 -- 172.21.15.19:0/1260517198 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1851348001 0 0) 0x7f52100cf6c0 con 0x7f52100cf050 2024-03-28T13:21:56.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.715+0000 7f5214e00640 1 --2- 172.21.15.19:0/1260517198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f521015f6e0 0x7f52100d2820 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f52080097a0 tx=0x7f520802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a560daccddf89b70 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:56.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.715+0000 7f520ea00640 1 -- 172.21.15.19:0/1260517198 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208034070 con 0x7f521015f6e0 2024-03-28T13:21:56.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.715+0000 7f520ea00640 1 -- 172.21.15.19:0/1260517198 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5208030610 con 0x7f521015f6e0 2024-03-28T13:21:56.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.716+0000 7f520ea00640 1 -- 172.21.15.19:0/1260517198 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52080309e0 con 0x7f521015f6e0 2024-03-28T13:21:56.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.716+0000 7f5216e00640 1 -- 172.21.15.19:0/1260517198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f521015f6e0 msgr2=0x7f52100d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:56.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.716+0000 7f5216e00640 1 --2- 172.21.15.19:0/1260517198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f521015f6e0 0x7f52100d2820 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f52080097a0 tx=0x7f520802b960 comp rx=0 tx=0).stop 2024-03-28T13:21:56.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.716+0000 7f5216e00640 1 -- 172.21.15.19:0/1260517198 shutdown_connections 2024-03-28T13:21:56.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.716+0000 7f5216e00640 1 --2- 172.21.15.19:0/1260517198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f521015f6e0 0x7f52100d2820 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.716+0000 7f5216e00640 1 -- 172.21.15.19:0/1260517198 >> 172.21.15.19:0/1260517198 conn(0x7f521008bfb0 msgr2=0x7f52100d4460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:56.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.716+0000 7f5216e00640 1 -- 172.21.15.19:0/1260517198 shutdown_connections 2024-03-28T13:21:56.715 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.716+0000 7f5216e00640 1 -- 172.21.15.19:0/1260517198 wait complete. 2024-03-28T13:21:56.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.718+0000 7f5216e00640 1 Processor -- start 2024-03-28T13:21:56.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.718+0000 7f5216e00640 1 -- start start 2024-03-28T13:21:56.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.719+0000 7f5216e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100cf050 0x7f5210150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:56.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.719+0000 7f5216e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52100d0e70 0x7f52100c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:56.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.719+0000 7f5216e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f521015f6e0 0x7f521006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:56.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.719+0000 7f5216e00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f521015e910 con 0x7f52100d0e70 2024-03-28T13:21:56.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.719+0000 7f5216e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f521015fb70 con 0x7f521015f6e0 2024-03-28T13:21:56.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.719+0000 7f5216e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f521015ea90 con 0x7f52100cf050 2024-03-28T13:21:56.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.720+0000 7f520f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52100d0e70 0x7f52100c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:56.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.720+0000 7f520f400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52100d0e70 0x7f52100c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45880/0 (socket says 172.21.15.19:45880) 2024-03-28T13:21:56.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.720+0000 7f520f400640 1 -- 172.21.15.19:0/2880255012 learned_addr learned my addr 172.21.15.19:0/2880255012 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:56.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.720+0000 7f520fe00640 1 --2- 172.21.15.19:0/2880255012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100cf050 0x7f5210150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:56.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.720+0000 7f5214e00640 1 --2- 172.21.15.19:0/2880255012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f521015f6e0 0x7f521006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:56.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.720+0000 7f520f400640 1 -- 172.21.15.19:0/2880255012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100cf050 msgr2=0x7f5210150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:56.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.720+0000 7f520f400640 1 --2- 172.21.15.19:0/2880255012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100cf050 0x7f5210150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.720+0000 7f520f400640 1 -- 172.21.15.19:0/2880255012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f521015f6e0 msgr2=0x7f521006d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:56.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.720+0000 7f520f400640 1 --2- 172.21.15.19:0/2880255012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f521015f6e0 0x7f521006d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.720+0000 7f520f400640 1 -- 172.21.15.19:0/2880255012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52100c8040 con 0x7f52100d0e70 2024-03-28T13:21:56.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.720+0000 7f5214e00640 1 --2- 172.21.15.19:0/2880255012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f521015f6e0 0x7f521006d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:56.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.720+0000 7f520f400640 1 --2- 172.21.15.19:0/2880255012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52100d0e70 0x7f52100c9cc0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f51f800c4d0 tx=0x7f51f800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:56.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.721+0000 7f5207e00640 1 -- 172.21.15.19:0/2880255012 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8018070 con 0x7f52100d0e70 2024-03-28T13:21:56.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.721+0000 7f5207e00640 1 -- 172.21.15.19:0/2880255012 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f80074e0 con 0x7f52100d0e70 2024-03-28T13:21:56.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.721+0000 7f5207e00640 1 -- 172.21.15.19:0/2880255012 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8004a20 con 0x7f52100d0e70 2024-03-28T13:21:56.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.721+0000 7f5216e00640 1 -- 172.21.15.19:0/2880255012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52100c8320 con 0x7f52100d0e70 2024-03-28T13:21:56.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.721+0000 7f5216e00640 1 -- 172.21.15.19:0/2880255012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52100c88e0 con 0x7f52100d0e70 2024-03-28T13:21:56.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.722+0000 7f5216e00640 1 -- 172.21.15.19:0/2880255012 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5210068cd0 con 0x7f52100d0e70 2024-03-28T13:21:56.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.723+0000 7f5207e00640 1 -- 172.21.15.19:0/2880255012 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f51f8004bc0 con 0x7f52100d0e70 2024-03-28T13:21:56.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.724+0000 7f5207e00640 1 --2- 172.21.15.19:0/2880255012 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51ec06c530 0x7f51ec06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:56.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.724+0000 7f5207e00640 1 -- 172.21.15.19:0/2880255012 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7f51f808a770 con 0x7f52100d0e70 2024-03-28T13:21:56.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.725+0000 7f520fe00640 1 --2- 172.21.15.19:0/2880255012 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51ec06c530 0x7f51ec06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:56.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.726+0000 7f520fe00640 1 --2- 172.21.15.19:0/2880255012 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51ec06c530 0x7f51ec06e9f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f52100d5350 tx=0x7f5200009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:56.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.728+0000 7f5207e00640 1 -- 172.21.15.19:0/2880255012 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f51f8058f40 con 0x7f52100d0e70 2024-03-28T13:21:56.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.933+0000 7f5216e00640 1 -- 172.21.15.19:0/2880255012 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f52100d1b80 con 0x7f51ec06c530 2024-03-28T13:21:56.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.934+0000 7f5207e00640 1 -- 172.21.15.19:0/2880255012 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f52100d1b80 con 0x7f51ec06c530 2024-03-28T13:21:56.933 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 2 to deep-scrub 2024-03-28T13:21:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.937+0000 7f5216e00640 1 -- 172.21.15.19:0/2880255012 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51ec06c530 msgr2=0x7f51ec06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.937+0000 7f5216e00640 1 --2- 172.21.15.19:0/2880255012 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51ec06c530 0x7f51ec06e9f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f52100d5350 tx=0x7f5200009040 comp rx=0 tx=0).stop 2024-03-28T13:21:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.937+0000 7f5216e00640 1 -- 172.21.15.19:0/2880255012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52100d0e70 msgr2=0x7f52100c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.937+0000 7f5216e00640 1 --2- 172.21.15.19:0/2880255012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52100d0e70 0x7f52100c9cc0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f51f800c4d0 tx=0x7f51f800c990 comp rx=0 tx=0).stop 2024-03-28T13:21:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.937+0000 7f5216e00640 1 -- 172.21.15.19:0/2880255012 shutdown_connections 2024-03-28T13:21:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.937+0000 7f5216e00640 1 --2- 172.21.15.19:0/2880255012 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f51ec06c530 0x7f51ec06e9f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.937+0000 7f5216e00640 1 --2- 172.21.15.19:0/2880255012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f521015f6e0 0x7f521006d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.937+0000 7f5216e00640 1 --2- 172.21.15.19:0/2880255012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52100d0e70 0x7f52100c9cc0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.937+0000 7f5216e00640 1 --2- 172.21.15.19:0/2880255012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52100cf050 0x7f5210150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:56.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.937+0000 7f5216e00640 1 -- 172.21.15.19:0/2880255012 >> 172.21.15.19:0/2880255012 conn(0x7f521008bfb0 msgr2=0x7f5210160090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.937+0000 7f5216e00640 1 -- 172.21.15.19:0/2880255012 shutdown_connections 2024-03-28T13:21:56.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:56.937+0000 7f5216e00640 1 -- 172.21.15.19:0/2880255012 wait complete. 2024-03-28T13:21:56.955 INFO:tasks.ceph:Scrubbing osd.3 2024-03-28T13:21:56.956 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-28T13:21:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.178+0000 7f6b8be00640 1 Processor -- start 2024-03-28T13:21:57.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.179+0000 7f6b8be00640 1 -- start start 2024-03-28T13:21:57.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.180+0000 7f6b8be00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6b8c13f190 0x7f6b8c0ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.180+0000 7f6b8be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b8c0ced40 0x7f6b8c141890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.180+0000 7f6b8be00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6b8c141dd0 0x7f6b8c15fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.180+0000 7f6b8be00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f6b8c142240 con 0x7f6b8c141dd0 2024-03-28T13:21:57.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.180+0000 7f6b8be00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f6b8c13f570 con 0x7f6b8c13f190 2024-03-28T13:21:57.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.180+0000 7f6b8be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6b8c0cfbf0 con 0x7f6b8c0ced40 2024-03-28T13:21:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.181+0000 7f6b8b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6b8c141dd0 0x7f6b8c15fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.181+0000 7f6b8b400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f6b8c141dd0 0x7f6b8c15fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45892/0 (socket says 172.21.15.19:45892) 2024-03-28T13:21:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.181+0000 7f6b8b400640 1 -- 172.21.15.19:0/3157282391 learned_addr learned my addr 172.21.15.19:0/3157282391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.181+0000 7f6b8aa00640 1 --2- 172.21.15.19:0/3157282391 >> v2:172.21.15.188:3300/0 conn(0x7f6b8c13f190 0x7f6b8c0ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.181+0000 7f6b8a000640 1 --2- 172.21.15.19:0/3157282391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b8c0ced40 0x7f6b8c141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.181+0000 7f6b8b400640 1 -- 172.21.15.19:0/3157282391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b8c0ced40 msgr2=0x7f6b8c141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:57.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.181+0000 7f6b8b400640 1 --2- 172.21.15.19:0/3157282391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b8c0ced40 0x7f6b8c141890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.182+0000 7f6b8b400640 1 -- 172.21.15.19:0/3157282391 >> v2:172.21.15.188:3300/0 conn(0x7f6b8c13f190 msgr2=0x7f6b8c0ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:57.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.182+0000 7f6b8b400640 1 --2- 172.21.15.19:0/3157282391 >> v2:172.21.15.188:3300/0 conn(0x7f6b8c13f190 0x7f6b8c0ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.182+0000 7f6b8b400640 1 -- 172.21.15.19:0/3157282391 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c0cf350 con 0x7f6b8c141dd0 2024-03-28T13:21:57.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.182+0000 7f6b8a000640 1 --2- 172.21.15.19:0/3157282391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b8c0ced40 0x7f6b8c141890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:21:57.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.182+0000 7f6b8aa00640 1 --2- 172.21.15.19:0/3157282391 >> v2:172.21.15.188:3300/0 conn(0x7f6b8c13f190 0x7f6b8c0ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:21:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.183+0000 7f6b8b400640 1 --2- 172.21.15.19:0/3157282391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b8c141dd0 0x7f6b8c15fd00 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c073710 tx=0x7f6b8402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315a59c3000d714a server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.183+0000 7f6b89600640 1 -- 172.21.15.19:0/3157282391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8402ca70 con 0x7f6b8c141dd0 2024-03-28T13:21:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.183+0000 7f6b89600640 1 -- 172.21.15.19:0/3157282391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b8402cc10 con 0x7f6b8c141dd0 2024-03-28T13:21:57.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.183+0000 7f6b89600640 1 -- 172.21.15.19:0/3157282391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840343f0 con 0x7f6b8c141dd0 2024-03-28T13:21:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.184+0000 7f6b8be00640 1 -- 172.21.15.19:0/3157282391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b8c141dd0 msgr2=0x7f6b8c15fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.184+0000 7f6b8be00640 1 --2- 172.21.15.19:0/3157282391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b8c141dd0 0x7f6b8c15fd00 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c073710 tx=0x7f6b8402bed0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.184+0000 7f6b8be00640 1 -- 172.21.15.19:0/3157282391 shutdown_connections 2024-03-28T13:21:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.184+0000 7f6b8be00640 1 --2- 172.21.15.19:0/3157282391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b8c141dd0 0x7f6b8c15fd00 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.184+0000 7f6b8be00640 1 --2- 172.21.15.19:0/3157282391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b8c0ced40 0x7f6b8c141890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.184+0000 7f6b8be00640 1 --2- 172.21.15.19:0/3157282391 >> v2:172.21.15.188:3300/0 conn(0x7f6b8c13f190 0x7f6b8c0ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.184+0000 7f6b8be00640 1 -- 172.21.15.19:0/3157282391 >> 172.21.15.19:0/3157282391 conn(0x7f6b8c0b1d80 msgr2=0x7f6b8c0b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.184+0000 7f6b8be00640 1 -- 172.21.15.19:0/3157282391 shutdown_connections 2024-03-28T13:21:57.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.184+0000 7f6b8be00640 1 -- 172.21.15.19:0/3157282391 wait complete. 2024-03-28T13:21:57.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.187+0000 7f6b8be00640 1 Processor -- start 2024-03-28T13:21:57.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.187+0000 7f6b8be00640 1 -- start start 2024-03-28T13:21:57.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.188+0000 7f6b8be00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b8c0ced40 0x7f6b8c159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.188+0000 7f6b8be00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b8c13f190 0x7f6b8c15a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.188+0000 7f6b8be00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b8c141dd0 0x7f6b8c0b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.188+0000 7f6b8be00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c06d670 con 0x7f6b8c141dd0 2024-03-28T13:21:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.188+0000 7f6b8be00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c0cf910 con 0x7f6b8c0ced40 2024-03-28T13:21:57.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.188+0000 7f6b8be00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6b8c142550 con 0x7f6b8c13f190 2024-03-28T13:21:57.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.188+0000 7f6b8b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b8c141dd0 0x7f6b8c0b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.188+0000 7f6b8b400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b8c141dd0 0x7f6b8c0b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45908/0 (socket says 172.21.15.19:45908) 2024-03-28T13:21:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.188+0000 7f6b8b400640 1 -- 172.21.15.19:0/49440702 learned_addr learned my addr 172.21.15.19:0/49440702 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.189+0000 7f6b8aa00640 1 --2- 172.21.15.19:0/49440702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b8c0ced40 0x7f6b8c159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.189+0000 7f6b8a000640 1 --2- 172.21.15.19:0/49440702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b8c13f190 0x7f6b8c15a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.189+0000 7f6b8b400640 1 -- 172.21.15.19:0/49440702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b8c13f190 msgr2=0x7f6b8c15a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.189+0000 7f6b8b400640 1 --2- 172.21.15.19:0/49440702 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6b8c13f190 0x7f6b8c15a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.189+0000 7f6b8b400640 1 -- 172.21.15.19:0/49440702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b8c0ced40 msgr2=0x7f6b8c159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.189+0000 7f6b8b400640 1 --2- 172.21.15.19:0/49440702 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6b8c0ced40 0x7f6b8c159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.189+0000 7f6b8b400640 1 -- 172.21.15.19:0/49440702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c15ad60 con 0x7f6b8c141dd0 2024-03-28T13:21:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.189+0000 7f6b8b400640 1 --2- 172.21.15.19:0/49440702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b8c141dd0 0x7f6b8c0b4980 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f6b84036040 tx=0x7f6b84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.190+0000 7f6b82a00640 1 -- 172.21.15.19:0/49440702 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84043020 con 0x7f6b8c141dd0 2024-03-28T13:21:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.190+0000 7f6b82a00640 1 -- 172.21.15.19:0/49440702 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b8403e450 con 0x7f6b8c141dd0 2024-03-28T13:21:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.190+0000 7f6b82a00640 1 -- 172.21.15.19:0/49440702 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84009510 con 0x7f6b8c141dd0 2024-03-28T13:21:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.190+0000 7f6b8be00640 1 -- 172.21.15.19:0/49440702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8c15b040 con 0x7f6b8c141dd0 2024-03-28T13:21:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.190+0000 7f6b8be00640 1 -- 172.21.15.19:0/49440702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8c0717d0 con 0x7f6b8c141dd0 2024-03-28T13:21:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.191+0000 7f6b8be00640 1 -- 172.21.15.19:0/49440702 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b8c075940 con 0x7f6b8c141dd0 2024-03-28T13:21:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.192+0000 7f6b82a00640 1 -- 172.21.15.19:0/49440702 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f6b84046020 con 0x7f6b8c141dd0 2024-03-28T13:21:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.193+0000 7f6b82a00640 1 --2- 172.21.15.19:0/49440702 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6b6c06c530 0x7f6b6c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.193+0000 7f6b82a00640 1 -- 172.21.15.19:0/49440702 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7f6b840adfd0 con 0x7f6b8c141dd0 2024-03-28T13:21:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.193+0000 7f6b82a00640 1 --2- 172.21.15.19:0/49440702 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x7f6b6c072530 0x7f6b6c074950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.193+0000 7f6b82a00640 1 -- 172.21.15.19:0/49440702 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b84049e50 con 0x7f6b6c072530 2024-03-28T13:21:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.193+0000 7f6b82a00640 1 -- 172.21.15.19:0/49440702 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f6b840b05f0 con 0x7f6b8c141dd0 2024-03-28T13:21:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.194+0000 7f6b8aa00640 1 --2- 172.21.15.19:0/49440702 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6b6c06c530 0x7f6b6c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.194+0000 7f6b8a000640 1 --2- 172.21.15.19:0/49440702 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x7f6b6c072530 0x7f6b6c074950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.194+0000 7f6b8a000640 1 --2- 172.21.15.19:0/49440702 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x7f6b6c072530 0x7f6b6c074950 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:57.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.195+0000 7f6b8aa00640 1 --2- 172.21.15.19:0/49440702 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6b6c06c530 0x7f6b6c06e9f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0b5990 tx=0x7f6b7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:57.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.195+0000 7f6b82a00640 1 -- 172.21.15.19:0/49440702 <== osd.3 v2:172.21.15.19:6819/3265515865 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b84049e50 con 0x7f6b6c072530 2024-03-28T13:21:57.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.210+0000 7f6b8be00640 1 -- 172.21.15.19:0/49440702 --> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6b8c0b3710 con 0x7f6b6c072530 2024-03-28T13:21:57.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.211+0000 7f6b82a00640 1 -- 172.21.15.19:0/49440702 <== osd.3 v2:172.21.15.19:6819/3265515865 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6b8c0b3710 con 0x7f6b6c072530 2024-03-28T13:21:57.210 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:21:57.210 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T13:21:57.210 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:21:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.211+0000 7f6b8be00640 1 -- 172.21.15.19:0/49440702 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x7f6b6c072530 msgr2=0x7f6b6c074950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.211+0000 7f6b8be00640 1 --2- 172.21.15.19:0/49440702 >> [v2:172.21.15.19:6819/3265515865,v1:172.21.15.19:6821/3265515865] conn(0x7f6b6c072530 0x7f6b6c074950 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.211+0000 7f6b8be00640 1 -- 172.21.15.19:0/49440702 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6b6c06c530 msgr2=0x7f6b6c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.211+0000 7f6b8be00640 1 --2- 172.21.15.19:0/49440702 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f6b6c06c530 0x7f6b6c06e9f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0b5990 tx=0x7f6b7c008040 comp rx=0 tx=0).stop 2024-03-28T13:21:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.211+0000 7f6b8be00640 1 -- 172.21.15.19:0/49440702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b8c141dd0 msgr2=0x7f6b8c0b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.211+0000 7f6b8be00640 1 --2- 172.21.15.19:0/49440702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b8c141dd0 0x7f6b8c0b4980 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f6b84036040 tx=0x7f6b84004a40 comp rx=0 tx=0).stop 2024-03-28T13:21:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.211+0000 7f6b8b400640 1 -- 172.21.15.19:0/49440702 reap_dead start 2024-03-28T13:21:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.211+0000 7f6b8be00640 1 -- 172.21.15.19:0/49440702 shutdown_connections 2024-03-28T13:21:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.211+0000 7f6b8be00640 1 -- 172.21.15.19:0/49440702 >> 172.21.15.19:0/49440702 conn(0x7f6b8c0b1d80 msgr2=0x7f6b8c13cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.212+0000 7f6b8be00640 1 -- 172.21.15.19:0/49440702 shutdown_connections 2024-03-28T13:21:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.212+0000 7f6b8be00640 1 -- 172.21.15.19:0/49440702 wait complete. 2024-03-28T13:21:57.229 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-28T13:21:57.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.449+0000 7f026b200640 1 Processor -- start 2024-03-28T13:21:57.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.450+0000 7f026b200640 1 -- start start 2024-03-28T13:21:57.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.451+0000 7f026b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026c15e4a0 0x7f026c0b9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.451+0000 7f026b200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f026c0d2310 0x7f026c0ceff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.451+0000 7f026b200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f026c0cf530 0x7f026c0d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.451+0000 7f026b200640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f026c0d34e0 con 0x7f026c0d2310 2024-03-28T13:21:57.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.451+0000 7f026b200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f026c15e880 con 0x7f026c0cf530 2024-03-28T13:21:57.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.451+0000 7f026b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f026c0d28d0 con 0x7f026c15e4a0 2024-03-28T13:21:57.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.451+0000 7f0269400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f026c0d2310 0x7f026c0ceff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.451+0000 7f0269400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f026c0d2310 0x7f026c0ceff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45918/0 (socket says 172.21.15.19:45918) 2024-03-28T13:21:57.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.451+0000 7f0269400640 1 -- 172.21.15.19:0/1438916410 learned_addr learned my addr 172.21.15.19:0/1438916410 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:57.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.451+0000 7f0269e00640 1 --2- 172.21.15.19:0/1438916410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026c15e4a0 0x7f026c0b9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.451+0000 7f026a800640 1 --2- 172.21.15.19:0/1438916410 >> v2:172.21.15.188:3300/0 conn(0x7f026c0cf530 0x7f026c0d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.452+0000 7f0269400640 1 -- 172.21.15.19:0/1438916410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026c15e4a0 msgr2=0x7f026c0b9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:57.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.452+0000 7f0269400640 1 --2- 172.21.15.19:0/1438916410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026c15e4a0 0x7f026c0b9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.452+0000 7f0269400640 1 -- 172.21.15.19:0/1438916410 >> v2:172.21.15.188:3300/0 conn(0x7f026c0cf530 msgr2=0x7f026c0d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:57.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.452+0000 7f0269400640 1 --2- 172.21.15.19:0/1438916410 >> v2:172.21.15.188:3300/0 conn(0x7f026c0cf530 0x7f026c0d3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.452+0000 7f0269400640 1 -- 172.21.15.19:0/1438916410 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026c0d2bb0 con 0x7f026c0d2310 2024-03-28T13:21:57.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.452+0000 7f026a800640 1 --2- 172.21.15.19:0/1438916410 >> v2:172.21.15.188:3300/0 conn(0x7f026c0cf530 0x7f026c0d3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:57.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.453+0000 7f0269400640 1 --2- 172.21.15.19:0/1438916410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f026c0d2310 0x7f026c0ceff0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f026c0d50f0 tx=0x7f025802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1640d7a59d40600 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:57.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.453+0000 7f0268a00640 1 -- 172.21.15.19:0/1438916410 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258035020 con 0x7f026c0d2310 2024-03-28T13:21:57.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.453+0000 7f0268a00640 1 -- 172.21.15.19:0/1438916410 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025802d040 con 0x7f026c0d2310 2024-03-28T13:21:57.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.453+0000 7f0268a00640 1 -- 172.21.15.19:0/1438916410 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258031a80 con 0x7f026c0d2310 2024-03-28T13:21:57.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.454+0000 7f026b200640 1 -- 172.21.15.19:0/1438916410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f026c0d2310 msgr2=0x7f026c0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.454+0000 7f026b200640 1 --2- 172.21.15.19:0/1438916410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f026c0d2310 0x7f026c0ceff0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f026c0d50f0 tx=0x7f025802b900 comp rx=0 tx=0).stop 2024-03-28T13:21:57.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.454+0000 7f026b200640 1 -- 172.21.15.19:0/1438916410 shutdown_connections 2024-03-28T13:21:57.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.454+0000 7f026b200640 1 --2- 172.21.15.19:0/1438916410 >> v2:172.21.15.188:3300/0 conn(0x7f026c0cf530 0x7f026c0d3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.454+0000 7f026b200640 1 --2- 172.21.15.19:0/1438916410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f026c0d2310 0x7f026c0ceff0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.454+0000 7f026b200640 1 --2- 172.21.15.19:0/1438916410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026c15e4a0 0x7f026c0b9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.454+0000 7f026b200640 1 -- 172.21.15.19:0/1438916410 >> 172.21.15.19:0/1438916410 conn(0x7f026c0676d0 msgr2=0x7f026c067ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.454+0000 7f026b200640 1 -- 172.21.15.19:0/1438916410 shutdown_connections 2024-03-28T13:21:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.454+0000 7f026b200640 1 -- 172.21.15.19:0/1438916410 wait complete. 2024-03-28T13:21:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.456+0000 7f026b200640 1 Processor -- start 2024-03-28T13:21:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.456+0000 7f026b200640 1 -- start start 2024-03-28T13:21:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.457+0000 7f026b200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f026c0cf530 0x7f026c1f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.457+0000 7f026b200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026c0d2310 0x7f026c1f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.457+0000 7f026b200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f026c15e4a0 0x7f026c0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.457+0000 7f026b200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f026c0d3970 con 0x7f026c0cf530 2024-03-28T13:21:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.457+0000 7f026b200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f026c0d3170 con 0x7f026c15e4a0 2024-03-28T13:21:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.457+0000 7f026b200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f026c0ba1a0 con 0x7f026c0d2310 2024-03-28T13:21:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.457+0000 7f0269e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f026c0cf530 0x7f026c1f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.457+0000 7f0269e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f026c0cf530 0x7f026c1f4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45926/0 (socket says 172.21.15.19:45926) 2024-03-28T13:21:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.457+0000 7f0269e00640 1 -- 172.21.15.19:0/168706082 learned_addr learned my addr 172.21.15.19:0/168706082 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:57.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.457+0000 7f0269400640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026c0d2310 0x7f026c1f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.457+0000 7f026a800640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f026c15e4a0 0x7f026c0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.458+0000 7f0269e00640 1 -- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026c0d2310 msgr2=0x7f026c1f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.458+0000 7f0269e00640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026c0d2310 0x7f026c1f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.458+0000 7f0269e00640 1 -- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f026c15e4a0 msgr2=0x7f026c0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.458+0000 7f0269e00640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f026c15e4a0 0x7f026c0ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.458+0000 7f0269e00640 1 -- 172.21.15.19:0/168706082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026c1f7360 con 0x7f026c0cf530 2024-03-28T13:21:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.458+0000 7f0269400640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026c0d2310 0x7f026c1f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.458+0000 7f026a800640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f026c15e4a0 0x7f026c0ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.458+0000 7f0269e00640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f026c0cf530 0x7f026c1f4660 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f025c00d5f0 tx=0x7f025c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.458+0000 7f0262000640 1 -- 172.21.15.19:0/168706082 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c0125f0 con 0x7f026c0cf530 2024-03-28T13:21:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.458+0000 7f0262000640 1 -- 172.21.15.19:0/168706082 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f025c019070 con 0x7f026c0cf530 2024-03-28T13:21:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.458+0000 7f0262000640 1 -- 172.21.15.19:0/168706082 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c0129d0 con 0x7f026c0cf530 2024-03-28T13:21:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.458+0000 7f026b200640 1 -- 172.21.15.19:0/168706082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f026c0c8a40 con 0x7f026c0cf530 2024-03-28T13:21:57.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.458+0000 7f026b200640 1 -- 172.21.15.19:0/168706082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f026c0c9000 con 0x7f026c0cf530 2024-03-28T13:21:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.460+0000 7f026b200640 1 -- 172.21.15.19:0/168706082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f026c075940 con 0x7f026c0cf530 2024-03-28T13:21:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.461+0000 7f0262000640 1 -- 172.21.15.19:0/168706082 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f025c012b70 con 0x7f026c0cf530 2024-03-28T13:21:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.462+0000 7f0262000640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f024c06c530 0x7f024c06e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.462+0000 7f0262000640 1 -- 172.21.15.19:0/168706082 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7f025c08b800 con 0x7f026c0cf530 2024-03-28T13:21:57.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.462+0000 7f0269400640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f024c06c530 0x7f024c06e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.464+0000 7f0269400640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f024c06c530 0x7f024c06e9f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0258036000 tx=0x7f025803c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:57.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.467+0000 7f0262000640 1 -- 172.21.15.19:0/168706082 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f025c05a060 con 0x7f026c0cf530 2024-03-28T13:21:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.667+0000 7f026b200640 1 -- 172.21.15.19:0/168706082 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f026c0ce750 con 0x7f024c06c530 2024-03-28T13:21:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.669+0000 7f0262000640 1 -- 172.21.15.19:0/168706082 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f026c0ce750 con 0x7f024c06c530 2024-03-28T13:21:57.668 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 3 to deep-scrub 2024-03-28T13:21:57.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.671+0000 7f026b200640 1 -- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f024c06c530 msgr2=0x7f024c06e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.671+0000 7f026b200640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f024c06c530 0x7f024c06e9f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0258036000 tx=0x7f025803c040 comp rx=0 tx=0).stop 2024-03-28T13:21:57.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.671+0000 7f026b200640 1 -- 172.21.15.19:0/168706082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f026c0cf530 msgr2=0x7f026c1f4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.671+0000 7f026b200640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f026c0cf530 0x7f026c1f4660 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f025c00d5f0 tx=0x7f025c00dab0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.671+0000 7f026b200640 1 -- 172.21.15.19:0/168706082 shutdown_connections 2024-03-28T13:21:57.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.671+0000 7f026b200640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f024c06c530 0x7f024c06e9f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.671+0000 7f026b200640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f026c15e4a0 0x7f026c0ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.671+0000 7f026b200640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f026c0d2310 0x7f026c1f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.671+0000 7f026b200640 1 --2- 172.21.15.19:0/168706082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f026c0cf530 0x7f026c1f4660 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.671+0000 7f026b200640 1 -- 172.21.15.19:0/168706082 >> 172.21.15.19:0/168706082 conn(0x7f026c0676d0 msgr2=0x7f026c0b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:57.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.671+0000 7f026b200640 1 -- 172.21.15.19:0/168706082 shutdown_connections 2024-03-28T13:21:57.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.671+0000 7f026b200640 1 -- 172.21.15.19:0/168706082 wait complete. 2024-03-28T13:21:57.689 INFO:tasks.ceph:Scrubbing osd.4 2024-03-28T13:21:57.689 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-28T13:21:57.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.913+0000 7efd76200640 1 Processor -- start 2024-03-28T13:21:57.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.913+0000 7efd76200640 1 -- start start 2024-03-28T13:21:57.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.915+0000 7efd76200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efd7013f190 0x7efd700ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.915+0000 7efd76200640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd700cfbf0 con 0x7efd700ced40 2024-03-28T13:21:57.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.915+0000 7efd76200640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd700cf350 con 0x7efd70141fa0 2024-03-28T13:21:57.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.915+0000 7efd76200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7efd7013f570 con 0x7efd7013f190 2024-03-28T13:21:57.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.916+0000 7efd6fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efd700ced40 0x7efd70141890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33804/0 (socket says 172.21.15.19:33804) 2024-03-28T13:21:57.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.916+0000 7efd6fe00640 1 -- 172.21.15.19:0/4222440189 learned_addr learned my addr 172.21.15.19:0/4222440189 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.916+0000 7efd74e00640 1 --2- 172.21.15.19:0/4222440189 >> v2:172.21.15.188:3300/0 conn(0x7efd7013f190 0x7efd700ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.916+0000 7efd6f400640 1 -- 172.21.15.19:0/4222440189 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3901838414 0 0) 0x7efd700cfbf0 con 0x7efd700ced40 2024-03-28T13:21:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.916+0000 7efd6f400640 1 -- 172.21.15.19:0/4222440189 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd58003520 con 0x7efd700ced40 2024-03-28T13:21:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.916+0000 7efd6f400640 1 -- 172.21.15.19:0/4222440189 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2066172467 0 0) 0x7efd58003520 con 0x7efd700ced40 2024-03-28T13:21:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6f400640 1 -- 172.21.15.19:0/4222440189 >> v2:172.21.15.188:3300/0 conn(0x7efd7013f190 msgr2=0x7efd700ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:57.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6f400640 1 --2- 172.21.15.19:0/4222440189 >> v2:172.21.15.188:3300/0 conn(0x7efd7013f190 0x7efd700ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6f400640 1 -- 172.21.15.19:0/4222440189 >> v1:172.21.15.188:6789/0 conn(0x7efd70141fa0 legacy=0x7efd7006db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6f400640 1 -- 172.21.15.19:0/4222440189 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd700cf630 con 0x7efd700ced40 2024-03-28T13:21:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6f400640 1 -- 172.21.15.19:0/4222440189 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7efd5c001670 con 0x7efd700ced40 2024-03-28T13:21:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6f400640 1 -- 172.21.15.19:0/4222440189 >> v1:172.21.15.19:6789/0 conn(0x7efd700ced40 legacy=0x7efd70141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6f400640 1 --2- 172.21.15.19:0/4222440189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd58003a00 0x7efd58003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6f400640 1 --2- 172.21.15.19:0/4222440189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd58004800 0x7efd58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6f400640 1 --2- 172.21.15.19:0/4222440189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd580051c0 0x7efd5800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6f400640 1 -- 172.21.15.19:0/4222440189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd5800c090 con 0x7efd58004800 2024-03-28T13:21:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6f400640 1 -- 172.21.15.19:0/4222440189 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd5800c210 con 0x7efd580051c0 2024-03-28T13:21:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6f400640 1 -- 172.21.15.19:0/4222440189 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efd5800c410 con 0x7efd58003a00 2024-03-28T13:21:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6fe00640 1 --2- 172.21.15.19:0/4222440189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd58004800 0x7efd58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6fe00640 1 -- 172.21.15.19:0/4222440189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd58003a00 msgr2=0x7efd58003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6fe00640 1 --2- 172.21.15.19:0/4222440189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd58003a00 0x7efd58003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6fe00640 1 -- 172.21.15.19:0/4222440189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd580051c0 msgr2=0x7efd5800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:57.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd75800640 1 --2- 172.21.15.19:0/4222440189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd580051c0 0x7efd5800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.917+0000 7efd6fe00640 1 --2- 172.21.15.19:0/4222440189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd580051c0 0x7efd5800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.918+0000 7efd6fe00640 1 -- 172.21.15.19:0/4222440189 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd58003520 con 0x7efd58004800 2024-03-28T13:21:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.918+0000 7efd75800640 1 --2- 172.21.15.19:0/4222440189 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd580051c0 0x7efd5800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:57.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.918+0000 7efd75800640 1 -- 172.21.15.19:0/4222440189 reap_dead start 2024-03-28T13:21:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.919+0000 7efd6fe00640 1 --2- 172.21.15.19:0/4222440189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd58004800 0x7efd58004c80 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00c160 tx=0x7efd58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2db1aba524379b server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.919+0000 7efd6f400640 1 -- 172.21.15.19:0/4222440189 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c007c80 con 0x7efd58004800 2024-03-28T13:21:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.919+0000 7efd6f400640 1 -- 172.21.15.19:0/4222440189 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd5c030070 con 0x7efd58004800 2024-03-28T13:21:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.919+0000 7efd6f400640 1 -- 172.21.15.19:0/4222440189 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd5c005d70 con 0x7efd58004800 2024-03-28T13:21:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.919+0000 7efd76200640 1 -- 172.21.15.19:0/4222440189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd58004800 msgr2=0x7efd58004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.919+0000 7efd76200640 1 --2- 172.21.15.19:0/4222440189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd58004800 0x7efd58004c80 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00c160 tx=0x7efd58005830 comp rx=0 tx=0).stop 2024-03-28T13:21:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.919+0000 7efd76200640 1 -- 172.21.15.19:0/4222440189 shutdown_connections 2024-03-28T13:21:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.919+0000 7efd76200640 1 --2- 172.21.15.19:0/4222440189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd58004800 0x7efd58004c80 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.919+0000 7efd76200640 1 -- 172.21.15.19:0/4222440189 >> 172.21.15.19:0/4222440189 conn(0x7efd700b1d80 msgr2=0x7efd700b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.920+0000 7efd76200640 1 -- 172.21.15.19:0/4222440189 shutdown_connections 2024-03-28T13:21:57.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.920+0000 7efd76200640 1 -- 172.21.15.19:0/4222440189 wait complete. 2024-03-28T13:21:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.921+0000 7efd76200640 1 Processor -- start 2024-03-28T13:21:57.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.921+0000 7efd76200640 1 -- start start 2024-03-28T13:21:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.922+0000 7efd76200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd58004800 0x7efd70139ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.922+0000 7efd76200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7015caa0 0x7efd7013a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.922+0000 7efd76200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd70153ad0 0x7efd70153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.922+0000 7efd76200640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efd70142550 con 0x7efd70153ad0 2024-03-28T13:21:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.922+0000 7efd76200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd7006d670 con 0x7efd58004800 2024-03-28T13:21:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.922+0000 7efd76200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efd701426d0 con 0x7efd7015caa0 2024-03-28T13:21:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.923+0000 7efd75800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd70153ad0 0x7efd70153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.923+0000 7efd75800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd70153ad0 0x7efd70153f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45944/0 (socket says 172.21.15.19:45944) 2024-03-28T13:21:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.923+0000 7efd75800640 1 -- 172.21.15.19:0/363708014 learned_addr learned my addr 172.21.15.19:0/363708014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:57.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.923+0000 7efd6fe00640 1 --2- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7015caa0 0x7efd7013a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.923+0000 7efd74e00640 1 --2- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd58004800 0x7efd70139ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.923+0000 7efd75800640 1 -- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7015caa0 msgr2=0x7efd7013a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.923+0000 7efd75800640 1 --2- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd7015caa0 0x7efd7013a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.923+0000 7efd75800640 1 -- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd58004800 msgr2=0x7efd70139ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.923+0000 7efd75800640 1 --2- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd58004800 0x7efd70139ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.923+0000 7efd75800640 1 -- 172.21.15.19:0/363708014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd70154840 con 0x7efd70153ad0 2024-03-28T13:21:57.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.924+0000 7efd75800640 1 --2- 172.21.15.19:0/363708014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd70153ad0 0x7efd70153f30 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7efd700b2fc0 tx=0x7efd6800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.924+0000 7efd6cc00640 1 -- 172.21.15.19:0/363708014 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68006d50 con 0x7efd70153ad0 2024-03-28T13:21:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.924+0000 7efd6cc00640 1 -- 172.21.15.19:0/363708014 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd68004980 con 0x7efd70153ad0 2024-03-28T13:21:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.924+0000 7efd6cc00640 1 -- 172.21.15.19:0/363708014 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68004d70 con 0x7efd70153ad0 2024-03-28T13:21:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.924+0000 7efd76200640 1 -- 172.21.15.19:0/363708014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd700b40a0 con 0x7efd70153ad0 2024-03-28T13:21:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.924+0000 7efd76200640 1 -- 172.21.15.19:0/363708014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd70154470 con 0x7efd70153ad0 2024-03-28T13:21:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.925+0000 7efd76200640 1 -- 172.21.15.19:0/363708014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd700b3470 con 0x7efd70153ad0 2024-03-28T13:21:57.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.926+0000 7efd6cc00640 1 -- 172.21.15.19:0/363708014 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efd68012600 con 0x7efd70153ad0 2024-03-28T13:21:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.927+0000 7efd6cc00640 1 --2- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd5006c530 0x7efd5006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.927+0000 7efd6cc00640 1 -- 172.21.15.19:0/363708014 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7efd6808a880 con 0x7efd70153ad0 2024-03-28T13:21:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.927+0000 7efd6cc00640 1 --2- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x7efd50072530 0x7efd50074950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.927+0000 7efd6cc00640 1 -- 172.21.15.19:0/363708014 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd50075000 con 0x7efd50072530 2024-03-28T13:21:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.927+0000 7efd6cc00640 1 -- 172.21.15.19:0/363708014 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7efd6808f2a0 con 0x7efd70153ad0 2024-03-28T13:21:57.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.928+0000 7efd74e00640 1 --2- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd5006c530 0x7efd5006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.928+0000 7efd6fe00640 1 --2- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x7efd50072530 0x7efd50074950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.929+0000 7efd74e00640 1 --2- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd5006c530 0x7efd5006e9f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7efd640047b0 tx=0x7efd64008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:57.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.930+0000 7efd6fe00640 1 --2- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x7efd50072530 0x7efd50074950 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:57.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.932+0000 7efd6cc00640 1 -- 172.21.15.19:0/363708014 <== osd.4 v2:172.21.15.188:6808/1845695132 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd50075000 con 0x7efd50072530 2024-03-28T13:21:57.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.946+0000 7efd76200640 1 -- 172.21.15.19:0/363708014 --> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efd70075940 con 0x7efd50072530 2024-03-28T13:21:57.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.948+0000 7efd6cc00640 1 -- 172.21.15.19:0/363708014 <== osd.4 v2:172.21.15.188:6808/1845695132 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efd70075940 con 0x7efd50072530 2024-03-28T13:21:57.947 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:21:57.947 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T13:21:57.947 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:21:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.948+0000 7efd76200640 1 -- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x7efd50072530 msgr2=0x7efd50074950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.948+0000 7efd76200640 1 --2- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:6808/1845695132,v1:172.21.15.188:6809/1845695132] conn(0x7efd50072530 0x7efd50074950 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.949+0000 7efd76200640 1 -- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd5006c530 msgr2=0x7efd5006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.949+0000 7efd76200640 1 --2- 172.21.15.19:0/363708014 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efd5006c530 0x7efd5006e9f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7efd640047b0 tx=0x7efd64008040 comp rx=0 tx=0).stop 2024-03-28T13:21:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.949+0000 7efd76200640 1 -- 172.21.15.19:0/363708014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd70153ad0 msgr2=0x7efd70153f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:57.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.949+0000 7efd76200640 1 --2- 172.21.15.19:0/363708014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efd70153ad0 0x7efd70153f30 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7efd700b2fc0 tx=0x7efd6800d9d0 comp rx=0 tx=0).stop 2024-03-28T13:21:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.949+0000 7efd75800640 1 -- 172.21.15.19:0/363708014 reap_dead start 2024-03-28T13:21:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.949+0000 7efd76200640 1 -- 172.21.15.19:0/363708014 shutdown_connections 2024-03-28T13:21:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.949+0000 7efd76200640 1 -- 172.21.15.19:0/363708014 >> 172.21.15.19:0/363708014 conn(0x7efd700b1d80 msgr2=0x7efd7013bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.949+0000 7efd76200640 1 -- 172.21.15.19:0/363708014 shutdown_connections 2024-03-28T13:21:57.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:57.949+0000 7efd76200640 1 -- 172.21.15.19:0/363708014 wait complete. 2024-03-28T13:21:57.965 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-28T13:21:58.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.180+0000 7f8006800640 1 Processor -- start 2024-03-28T13:21:58.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.180+0000 7f8006800640 1 -- start start 2024-03-28T13:21:58.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.181+0000 7f8006800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f80000d04d0 0x7f80000b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.181+0000 7f8006800640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80000b3150 con 0x7f80000cc0b0 2024-03-28T13:21:58.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.181+0000 7f8006800640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80001600e0 con 0x7f800015fc50 2024-03-28T13:21:58.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.181+0000 7f8006800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f800015c610 con 0x7f80000d04d0 2024-03-28T13:21:58.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.181+0000 7f8005400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f80000cc0b0 0x7f800006dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33818/0 (socket says 172.21.15.19:33818) 2024-03-28T13:21:58.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.181+0000 7f8005400640 1 -- 172.21.15.19:0/1182650667 learned_addr learned my addr 172.21.15.19:0/1182650667 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:58.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.182+0000 7f8005e00640 1 --2- 172.21.15.19:0/1182650667 >> v2:172.21.15.188:3300/0 conn(0x7f80000d04d0 0x7f80000b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.182+0000 7f7fffe00640 1 -- 172.21.15.19:0/1182650667 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1894031193 0 0) 0x7f80000b3150 con 0x7f80000cc0b0 2024-03-28T13:21:58.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.182+0000 7f7fffe00640 1 -- 172.21.15.19:0/1182650667 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fe8003520 con 0x7f80000cc0b0 2024-03-28T13:21:58.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.182+0000 7f7fffe00640 1 -- 172.21.15.19:0/1182650667 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3315977234 0 0) 0x7f7fe8003520 con 0x7f80000cc0b0 2024-03-28T13:21:58.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.182+0000 7f7fffe00640 1 -- 172.21.15.19:0/1182650667 >> v2:172.21.15.188:3300/0 conn(0x7f80000d04d0 msgr2=0x7f80000b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:58.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.182+0000 7f7fffe00640 1 --2- 172.21.15.19:0/1182650667 >> v2:172.21.15.188:3300/0 conn(0x7f80000d04d0 0x7f80000b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.182+0000 7f7fffe00640 1 -- 172.21.15.19:0/1182650667 >> v1:172.21.15.188:6789/0 conn(0x7f800015fc50 legacy=0x7f80000cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.182+0000 7f7fffe00640 1 -- 172.21.15.19:0/1182650667 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80001603c0 con 0x7f80000cc0b0 2024-03-28T13:21:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.182+0000 7f7fffe00640 1 -- 172.21.15.19:0/1182650667 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7f7ff4001670 con 0x7f80000cc0b0 2024-03-28T13:21:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.182+0000 7f7fffe00640 1 -- 172.21.15.19:0/1182650667 >> v1:172.21.15.19:6789/0 conn(0x7f80000cc0b0 legacy=0x7f800006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.182+0000 7f7fffe00640 1 --2- 172.21.15.19:0/1182650667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fe80039b0 0x7f7fe8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.182+0000 7f7fffe00640 1 --2- 172.21.15.19:0/1182650667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fe8004800 0x7f7fe8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.182+0000 7f7fffe00640 1 --2- 172.21.15.19:0/1182650667 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe80052e0 0x7f7fe800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.183+0000 7f7fffe00640 1 -- 172.21.15.19:0/1182650667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f800015c610 con 0x7f7fe80039b0 2024-03-28T13:21:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.183+0000 7f7fffe00640 1 -- 172.21.15.19:0/1182650667 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe800c190 con 0x7f7fe8004800 2024-03-28T13:21:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.183+0000 7f7fffe00640 1 -- 172.21.15.19:0/1182650667 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7fe800c390 con 0x7f7fe80052e0 2024-03-28T13:21:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.183+0000 7f8005400640 1 --2- 172.21.15.19:0/1182650667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fe80039b0 0x7f7fe8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.183+0000 7f8005e00640 1 --2- 172.21.15.19:0/1182650667 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe80052e0 0x7f7fe800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.183+0000 7f8004a00640 1 --2- 172.21.15.19:0/1182650667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fe8004800 0x7f7fe8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.183+0000 7f8005400640 1 -- 172.21.15.19:0/1182650667 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe80052e0 msgr2=0x7f7fe800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.183+0000 7f8005400640 1 --2- 172.21.15.19:0/1182650667 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe80052e0 0x7f7fe800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.183+0000 7f8005400640 1 -- 172.21.15.19:0/1182650667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fe8004800 msgr2=0x7f7fe8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.183+0000 7f8005400640 1 --2- 172.21.15.19:0/1182650667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fe8004800 0x7f7fe8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.183+0000 7f8005400640 1 -- 172.21.15.19:0/1182650667 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe8003520 con 0x7f7fe80039b0 2024-03-28T13:21:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.183+0000 7f8004a00640 1 --2- 172.21.15.19:0/1182650667 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7fe8004800 0x7f7fe8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:58.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.183+0000 7f8005e00640 1 -- 172.21.15.19:0/1182650667 reap_dead start 2024-03-28T13:21:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.184+0000 7f8005400640 1 --2- 172.21.15.19:0/1182650667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fe80039b0 0x7f7fe8003db0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400c210 tx=0x7f7fe8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5338a7a60d6bce server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.185+0000 7f7fffe00640 1 -- 172.21.15.19:0/1182650667 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4007d30 con 0x7f7fe80039b0 2024-03-28T13:21:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.185+0000 7f7fffe00640 1 -- 172.21.15.19:0/1182650667 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff4030070 con 0x7f7fe80039b0 2024-03-28T13:21:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.185+0000 7f7fffe00640 1 -- 172.21.15.19:0/1182650667 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4005d30 con 0x7f7fe80039b0 2024-03-28T13:21:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.185+0000 7f8006800640 1 -- 172.21.15.19:0/1182650667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fe80039b0 msgr2=0x7f7fe8003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.185+0000 7f8006800640 1 --2- 172.21.15.19:0/1182650667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fe80039b0 0x7f7fe8003db0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400c210 tx=0x7f7fe8005830 comp rx=0 tx=0).stop 2024-03-28T13:21:58.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.185+0000 7f8006800640 1 -- 172.21.15.19:0/1182650667 shutdown_connections 2024-03-28T13:21:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.185+0000 7f8006800640 1 --2- 172.21.15.19:0/1182650667 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fe80039b0 0x7f7fe8003db0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.185+0000 7f8006800640 1 -- 172.21.15.19:0/1182650667 >> 172.21.15.19:0/1182650667 conn(0x7f80000ccad0 msgr2=0x7f80000cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.185+0000 7f8006800640 1 -- 172.21.15.19:0/1182650667 shutdown_connections 2024-03-28T13:21:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.185+0000 7f8006800640 1 -- 172.21.15.19:0/1182650667 wait complete. 2024-03-28T13:21:58.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.187+0000 7f8006800640 1 Processor -- start 2024-03-28T13:21:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.187+0000 7f8006800640 1 -- start start 2024-03-28T13:21:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.188+0000 7f8006800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe80039b0 0x7f80001f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.188+0000 7f8006800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f800015b130 0x7f800015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.188+0000 7f8006800640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80001f7b30 0x7f80001f7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.188+0000 7f8006800640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f80000cc490 con 0x7f80001f7b30 2024-03-28T13:21:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.188+0000 7f8006800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8000160980 con 0x7f800015b130 2024-03-28T13:21:58.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.188+0000 7f8006800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f80000cc610 con 0x7f7fe80039b0 2024-03-28T13:21:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.189+0000 7f8005e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80001f7b30 0x7f80001f7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.189+0000 7f8005e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80001f7b30 0x7f80001f7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45956/0 (socket says 172.21.15.19:45956) 2024-03-28T13:21:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.189+0000 7f8005e00640 1 -- 172.21.15.19:0/3469721140 learned_addr learned my addr 172.21.15.19:0/3469721140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.189+0000 7f8005400640 1 --2- 172.21.15.19:0/3469721140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe80039b0 0x7f80001f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.189+0000 7f8004a00640 1 --2- 172.21.15.19:0/3469721140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f800015b130 0x7f800015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.189+0000 7f8005e00640 1 -- 172.21.15.19:0/3469721140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe80039b0 msgr2=0x7f80001f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.189+0000 7f8005e00640 1 --2- 172.21.15.19:0/3469721140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe80039b0 0x7f80001f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.189+0000 7f8005e00640 1 -- 172.21.15.19:0/3469721140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f800015b130 msgr2=0x7f800015b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.189+0000 7f8005e00640 1 --2- 172.21.15.19:0/3469721140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f800015b130 0x7f800015b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.189+0000 7f8005e00640 1 -- 172.21.15.19:0/3469721140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8000153440 con 0x7f80001f7b30 2024-03-28T13:21:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.189+0000 7f8005e00640 1 --2- 172.21.15.19:0/3469721140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80001f7b30 0x7f80001f7fb0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009820 tx=0x7f7ff8009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.190+0000 7f7ffd600640 1 -- 172.21.15.19:0/3469721140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8019070 con 0x7f80001f7b30 2024-03-28T13:21:58.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.190+0000 7f7ffd600640 1 -- 172.21.15.19:0/3469721140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff8007d40 con 0x7f80001f7b30 2024-03-28T13:21:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.190+0000 7f7ffd600640 1 -- 172.21.15.19:0/3469721140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8014640 con 0x7f80001f7b30 2024-03-28T13:21:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.190+0000 7f8006800640 1 -- 172.21.15.19:0/3469721140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80001f8710 con 0x7f80001f7b30 2024-03-28T13:21:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.190+0000 7f8006800640 1 -- 172.21.15.19:0/3469721140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80000cdaa0 con 0x7f80001f7b30 2024-03-28T13:21:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.191+0000 7f8006800640 1 -- 172.21.15.19:0/3469721140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8000075940 con 0x7f80001f7b30 2024-03-28T13:21:58.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.192+0000 7f7ffd600640 1 -- 172.21.15.19:0/3469721140 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7ff801c070 con 0x7f80001f7b30 2024-03-28T13:21:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.193+0000 7f7ffd600640 1 --2- 172.21.15.19:0/3469721140 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fe006c530 0x7f7fe006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.193+0000 7f7ffd600640 1 -- 172.21.15.19:0/3469721140 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7f7ff808afe0 con 0x7f80001f7b30 2024-03-28T13:21:58.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.194+0000 7f8005400640 1 --2- 172.21.15.19:0/3469721140 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fe006c530 0x7f7fe006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.195+0000 7f8005400640 1 --2- 172.21.15.19:0/3469721140 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fe006c530 0x7f7fe006e9f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4034000 tx=0x7f7ff4009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:58.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.198+0000 7f7ffd600640 1 -- 172.21.15.19:0/3469721140 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8000153440 con 0x7f80001f7b30 2024-03-28T13:21:58.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.402+0000 7f8006800640 1 -- 172.21.15.19:0/3469721140 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f8000073830 con 0x7f7fe006c530 2024-03-28T13:21:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.403+0000 7f7ffd600640 1 -- 172.21.15.19:0/3469721140 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8000073830 con 0x7f7fe006c530 2024-03-28T13:21:58.402 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 4 to deep-scrub 2024-03-28T13:21:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.406+0000 7f8006800640 1 -- 172.21.15.19:0/3469721140 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fe006c530 msgr2=0x7f7fe006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.406+0000 7f8006800640 1 --2- 172.21.15.19:0/3469721140 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fe006c530 0x7f7fe006e9f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4034000 tx=0x7f7ff4009030 comp rx=0 tx=0).stop 2024-03-28T13:21:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.406+0000 7f8006800640 1 -- 172.21.15.19:0/3469721140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80001f7b30 msgr2=0x7f80001f7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.406+0000 7f8006800640 1 --2- 172.21.15.19:0/3469721140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80001f7b30 0x7f80001f7fb0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009820 tx=0x7f7ff8009ce0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.406+0000 7f8006800640 1 -- 172.21.15.19:0/3469721140 shutdown_connections 2024-03-28T13:21:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.406+0000 7f8006800640 1 --2- 172.21.15.19:0/3469721140 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7fe006c530 0x7f7fe006e9f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.406+0000 7f8006800640 1 --2- 172.21.15.19:0/3469721140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f80001f7b30 0x7f80001f7fb0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.406+0000 7f8006800640 1 --2- 172.21.15.19:0/3469721140 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f800015b130 0x7f800015b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.406+0000 7f8006800640 1 --2- 172.21.15.19:0/3469721140 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7fe80039b0 0x7f80001f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.406+0000 7f8006800640 1 -- 172.21.15.19:0/3469721140 >> 172.21.15.19:0/3469721140 conn(0x7f80000ccad0 msgr2=0x7f800015eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:58.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.406+0000 7f8006800640 1 -- 172.21.15.19:0/3469721140 shutdown_connections 2024-03-28T13:21:58.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.406+0000 7f8006800640 1 -- 172.21.15.19:0/3469721140 wait complete. 2024-03-28T13:21:58.425 INFO:tasks.ceph:Scrubbing osd.5 2024-03-28T13:21:58.426 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-28T13:21:58.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.646+0000 7ff359400640 1 Processor -- start 2024-03-28T13:21:58.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.646+0000 7ff359400640 1 -- start start 2024-03-28T13:21:58.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.647+0000 7ff359400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3540d3d50 0x7ff3540d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.647+0000 7ff359400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3540b6310 0x7ff3540b67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.647+0000 7ff359400640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3540d23b0 con 0x7ff35415e420 2024-03-28T13:21:58.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.647+0000 7ff359400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff3540d2fc0 con 0x7ff3540b6310 2024-03-28T13:21:58.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.647+0000 7ff359400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff3540d2690 con 0x7ff3540d3d50 2024-03-28T13:21:58.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.648+0000 7ff358a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3540b6310 0x7ff3540b67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.648+0000 7ff353400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3540d3d50 0x7ff3540d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.648+0000 7ff358a00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3540b6310 0x7ff3540b67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.19:39208/0 (socket says 172.21.15.19:39208) 2024-03-28T13:21:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.648+0000 7ff353400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3540d3d50 0x7ff3540d3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.19:40700/0 (socket says 172.21.15.19:40700) 2024-03-28T13:21:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.648+0000 7ff358a00640 1 -- 172.21.15.19:0/630214603 learned_addr learned my addr 172.21.15.19:0/630214603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.649+0000 7ff358a00640 1 -- 172.21.15.19:0/630214603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3540d3d50 msgr2=0x7ff3540d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.649+0000 7ff358a00640 1 --2- 172.21.15.19:0/630214603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3540d3d50 0x7ff3540d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.649+0000 7ff358a00640 1 -- 172.21.15.19:0/630214603 >> v1:172.21.15.188:6789/0 conn(0x7ff35415e420 legacy=0x7ff3540b9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.649+0000 7ff358a00640 1 -- 172.21.15.19:0/630214603 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3540d2970 con 0x7ff3540b6310 2024-03-28T13:21:58.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.649+0000 7ff353400640 1 --2- 172.21.15.19:0/630214603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3540d3d50 0x7ff3540d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-28T13:21:58.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.650+0000 7ff358a00640 1 --2- 172.21.15.19:0/630214603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3540b6310 0x7ff3540b67a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009e50 tx=0x7ff34c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44966fd4ceb822e server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.651+0000 7ff352a00640 1 -- 172.21.15.19:0/630214603 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34c02cb00 con 0x7ff3540b6310 2024-03-28T13:21:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.651+0000 7ff352a00640 1 -- 172.21.15.19:0/630214603 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff34c02cca0 con 0x7ff3540b6310 2024-03-28T13:21:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.651+0000 7ff352a00640 1 -- 172.21.15.19:0/630214603 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34c034610 con 0x7ff3540b6310 2024-03-28T13:21:58.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.652+0000 7ff359400640 1 -- 172.21.15.19:0/630214603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3540b6310 msgr2=0x7ff3540b67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.652+0000 7ff359400640 1 --2- 172.21.15.19:0/630214603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3540b6310 0x7ff3540b67a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff34c009e50 tx=0x7ff34c02bf60 comp rx=0 tx=0).stop 2024-03-28T13:21:58.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.652+0000 7ff359400640 1 -- 172.21.15.19:0/630214603 shutdown_connections 2024-03-28T13:21:58.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.652+0000 7ff359400640 1 --2- 172.21.15.19:0/630214603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3540b6310 0x7ff3540b67a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.652+0000 7ff359400640 1 --2- 172.21.15.19:0/630214603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3540d3d50 0x7ff3540d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.652+0000 7ff359400640 1 -- 172.21.15.19:0/630214603 >> 172.21.15.19:0/630214603 conn(0x7ff3540676a0 msgr2=0x7ff354067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:58.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.652+0000 7ff359400640 1 -- 172.21.15.19:0/630214603 shutdown_connections 2024-03-28T13:21:58.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.652+0000 7ff359400640 1 -- 172.21.15.19:0/630214603 wait complete. 2024-03-28T13:21:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.654+0000 7ff359400640 1 Processor -- start 2024-03-28T13:21:58.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.654+0000 7ff359400640 1 -- start start 2024-03-28T13:21:58.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.655+0000 7ff359400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3540b6310 0x7ff354153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.655+0000 7ff359400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3540d3d50 0x7ff354156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.655+0000 7ff359400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35415e420 0x7ff354156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.655+0000 7ff359400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3540d3990 con 0x7ff3540d3d50 2024-03-28T13:21:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.655+0000 7ff359400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff35415e800 con 0x7ff3540b6310 2024-03-28T13:21:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.655+0000 7ff359400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff3540b7460 con 0x7ff35415e420 2024-03-28T13:21:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.655+0000 7ff353400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3540d3d50 0x7ff354156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.655+0000 7ff353400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3540d3d50 0x7ff354156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45962/0 (socket says 172.21.15.19:45962) 2024-03-28T13:21:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.655+0000 7ff353400640 1 -- 172.21.15.19:0/2906148368 learned_addr learned my addr 172.21.15.19:0/2906148368 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.656+0000 7ff353e00640 1 --2- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3540b6310 0x7ff354153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.656+0000 7ff358a00640 1 --2- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35415e420 0x7ff354156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.656+0000 7ff353400640 1 -- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35415e420 msgr2=0x7ff354156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.656+0000 7ff353400640 1 --2- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35415e420 0x7ff354156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.656+0000 7ff353400640 1 -- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3540b6310 msgr2=0x7ff354153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.656+0000 7ff353400640 1 --2- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3540b6310 0x7ff354153830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.656+0000 7ff353400640 1 -- 172.21.15.19:0/2906148368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3540c88b0 con 0x7ff3540d3d50 2024-03-28T13:21:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.656+0000 7ff353400640 1 --2- 172.21.15.19:0/2906148368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3540d3d50 0x7ff354156800 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7ff34000ebe0 tx=0x7ff34000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.656+0000 7ff34be00640 1 -- 172.21.15.19:0/2906148368 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340017070 con 0x7ff3540d3d50 2024-03-28T13:21:58.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.656+0000 7ff34be00640 1 -- 172.21.15.19:0/2906148368 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff34000cc30 con 0x7ff3540d3d50 2024-03-28T13:21:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.656+0000 7ff34be00640 1 -- 172.21.15.19:0/2906148368 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340013470 con 0x7ff3540d3d50 2024-03-28T13:21:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.656+0000 7ff359400640 1 -- 172.21.15.19:0/2906148368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3540c8b90 con 0x7ff3540d3d50 2024-03-28T13:21:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.657+0000 7ff359400640 1 -- 172.21.15.19:0/2906148368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3540c7c60 con 0x7ff3540d3d50 2024-03-28T13:21:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.657+0000 7ff359400640 1 -- 172.21.15.19:0/2906148368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff354158280 con 0x7ff3540d3d50 2024-03-28T13:21:58.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.659+0000 7ff34be00640 1 -- 172.21.15.19:0/2906148368 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff34001a070 con 0x7ff3540d3d50 2024-03-28T13:21:58.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.659+0000 7ff34be00640 1 --2- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff33406c530 0x7ff33406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.660+0000 7ff34be00640 1 -- 172.21.15.19:0/2906148368 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7ff34008b7d0 con 0x7ff3540d3d50 2024-03-28T13:21:58.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.660+0000 7ff34be00640 1 --2- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x7ff334072530 0x7ff334074950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.660+0000 7ff34be00640 1 -- 172.21.15.19:0/2906148368 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff340006870 con 0x7ff334072530 2024-03-28T13:21:58.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.660+0000 7ff34be00640 1 -- 172.21.15.19:0/2906148368 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7ff340092050 con 0x7ff3540d3d50 2024-03-28T13:21:58.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.660+0000 7ff353e00640 1 --2- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff33406c530 0x7ff33406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.661+0000 7ff358a00640 1 --2- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x7ff334072530 0x7ff334074950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.661+0000 7ff353e00640 1 --2- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff33406c530 0x7ff33406e9f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff344007fd0 tx=0x7ff344009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:58.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.662+0000 7ff358a00640 1 --2- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x7ff334072530 0x7ff334074950 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:58.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.664+0000 7ff34be00640 1 -- 172.21.15.19:0/2906148368 <== osd.5 v2:172.21.15.188:6800/3324247755 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff340006870 con 0x7ff334072530 2024-03-28T13:21:58.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.679+0000 7ff359400640 1 -- 172.21.15.19:0/2906148368 --> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff354075940 con 0x7ff334072530 2024-03-28T13:21:58.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.680+0000 7ff34be00640 1 -- 172.21.15.19:0/2906148368 <== osd.5 v2:172.21.15.188:6800/3324247755 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff354075940 con 0x7ff334072530 2024-03-28T13:21:58.679 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:21:58.679 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T13:21:58.679 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:21:58.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.681+0000 7ff359400640 1 -- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x7ff334072530 msgr2=0x7ff334074950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.681+0000 7ff359400640 1 --2- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:6800/3324247755,v1:172.21.15.188:6801/3324247755] conn(0x7ff334072530 0x7ff334074950 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.681+0000 7ff359400640 1 -- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff33406c530 msgr2=0x7ff33406e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.681+0000 7ff359400640 1 --2- 172.21.15.19:0/2906148368 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff33406c530 0x7ff33406e9f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff344007fd0 tx=0x7ff344009040 comp rx=0 tx=0).stop 2024-03-28T13:21:58.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.681+0000 7ff359400640 1 -- 172.21.15.19:0/2906148368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3540d3d50 msgr2=0x7ff354156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.681+0000 7ff359400640 1 --2- 172.21.15.19:0/2906148368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3540d3d50 0x7ff354156800 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7ff34000ebe0 tx=0x7ff34000c520 comp rx=0 tx=0).stop 2024-03-28T13:21:58.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.681+0000 7ff358a00640 1 -- 172.21.15.19:0/2906148368 reap_dead start 2024-03-28T13:21:58.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.681+0000 7ff359400640 1 -- 172.21.15.19:0/2906148368 shutdown_connections 2024-03-28T13:21:58.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.681+0000 7ff359400640 1 -- 172.21.15.19:0/2906148368 >> 172.21.15.19:0/2906148368 conn(0x7ff3540676a0 msgr2=0x7ff35415ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:58.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.681+0000 7ff359400640 1 -- 172.21.15.19:0/2906148368 shutdown_connections 2024-03-28T13:21:58.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.681+0000 7ff359400640 1 -- 172.21.15.19:0/2906148368 wait complete. 2024-03-28T13:21:58.696 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-28T13:21:58.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.911+0000 7f7bb5000640 1 Processor -- start 2024-03-28T13:21:58.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.912+0000 7f7bb5000640 1 -- start start 2024-03-28T13:21:58.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.913+0000 7f7bb5000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7bb0146f60 0x7f7bb00ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.913+0000 7f7bb5000640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bb0147340 con 0x7f7bb0147570 2024-03-28T13:21:58.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.913+0000 7f7bb5000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bb00cf260 con 0x7f7bb006bc80 2024-03-28T13:21:58.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.913+0000 7f7bb5000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f7bb00cf030 con 0x7f7bb0146f60 2024-03-28T13:21:58.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.913+0000 7f7baf400640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f7bb0146f60 0x7f7bb00ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.913+0000 7f7baea00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f7bb0147570 0x7f7bb006b570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33822/0 (socket says 172.21.15.19:33822) 2024-03-28T13:21:58.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.913+0000 7f7baea00640 1 -- 172.21.15.19:0/2492453389 learned_addr learned my addr 172.21.15.19:0/2492453389 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:58.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.914+0000 7f7baf400640 1 -- 172.21.15.19:0/2492453389 >> v1:172.21.15.188:6789/0 conn(0x7f7bb006bc80 legacy=0x7f7bb0161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:58.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.914+0000 7f7baf400640 1 -- 172.21.15.19:0/2492453389 >> v1:172.21.15.19:6789/0 conn(0x7f7bb0147570 legacy=0x7f7bb006b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.914+0000 7f7baf400640 1 -- 172.21.15.19:0/2492453389 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb006e570 con 0x7f7bb0146f60 2024-03-28T13:21:58.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.915+0000 7f7baf400640 1 --2- 172.21.15.19:0/2492453389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb0146f60 0x7f7bb00ceaf0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00097a0 tx=0x7f7ba002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8497d3c4057a7bf server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:58.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.915+0000 7f7bae000640 1 -- 172.21.15.19:0/2492453389 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0035020 con 0x7f7bb0146f60 2024-03-28T13:21:58.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.915+0000 7f7bae000640 1 -- 172.21.15.19:0/2492453389 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba0031830 con 0x7f7bb0146f60 2024-03-28T13:21:58.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.915+0000 7f7bae000640 1 -- 172.21.15.19:0/2492453389 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0031c00 con 0x7f7bb0146f60 2024-03-28T13:21:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.916+0000 7f7bb5000640 1 -- 172.21.15.19:0/2492453389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb0146f60 msgr2=0x7f7bb00ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.916+0000 7f7bb5000640 1 --2- 172.21.15.19:0/2492453389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb0146f60 0x7f7bb00ceaf0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00097a0 tx=0x7f7ba002b8d0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.916+0000 7f7bb5000640 1 -- 172.21.15.19:0/2492453389 shutdown_connections 2024-03-28T13:21:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.916+0000 7f7bb5000640 1 --2- 172.21.15.19:0/2492453389 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb0146f60 0x7f7bb00ceaf0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.916+0000 7f7bb5000640 1 -- 172.21.15.19:0/2492453389 >> 172.21.15.19:0/2492453389 conn(0x7f7bb00b2c90 msgr2=0x7f7bb00b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.916+0000 7f7bb5000640 1 -- 172.21.15.19:0/2492453389 shutdown_connections 2024-03-28T13:21:58.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.916+0000 7f7bb5000640 1 -- 172.21.15.19:0/2492453389 wait complete. 2024-03-28T13:21:58.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.918+0000 7f7bb5000640 1 Processor -- start 2024-03-28T13:21:58.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.918+0000 7f7bb5000640 1 -- start start 2024-03-28T13:21:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.919+0000 7f7bb5000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bb006bc80 0x7f7bb00b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.919+0000 7f7bb5000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb0146f60 0x7f7bb01e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.919+0000 7f7bb5000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bb0147570 0x7f7bb01f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.919+0000 7f7bb5000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb006ebc0 con 0x7f7bb0146f60 2024-03-28T13:21:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.919+0000 7f7bb5000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb006e320 con 0x7f7bb006bc80 2024-03-28T13:21:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.919+0000 7f7bb5000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb006f050 con 0x7f7bb0147570 2024-03-28T13:21:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.919+0000 7f7baea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb0146f60 0x7f7bb01e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.919+0000 7f7baea00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb0146f60 0x7f7bb01e8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45984/0 (socket says 172.21.15.19:45984) 2024-03-28T13:21:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.919+0000 7f7baea00640 1 -- 172.21.15.19:0/3736168310 learned_addr learned my addr 172.21.15.19:0/3736168310 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.919+0000 7f7bafe00640 1 --2- 172.21.15.19:0/3736168310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bb0147570 0x7f7bb01f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.920+0000 7f7baf400640 1 --2- 172.21.15.19:0/3736168310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bb006bc80 0x7f7bb00b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.920+0000 7f7baea00640 1 -- 172.21.15.19:0/3736168310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bb0147570 msgr2=0x7f7bb01f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.920+0000 7f7baea00640 1 --2- 172.21.15.19:0/3736168310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bb0147570 0x7f7bb01f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.920+0000 7f7baea00640 1 -- 172.21.15.19:0/3736168310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bb006bc80 msgr2=0x7f7bb00b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.920+0000 7f7baea00640 1 --2- 172.21.15.19:0/3736168310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bb006bc80 0x7f7bb00b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.920+0000 7f7baea00640 1 -- 172.21.15.19:0/3736168310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb01447a0 con 0x7f7bb0146f60 2024-03-28T13:21:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.920+0000 7f7baea00640 1 --2- 172.21.15.19:0/3736168310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb0146f60 0x7f7bb01e8a00 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00caa0 tx=0x7f7b9c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.920+0000 7f7ba7400640 1 -- 172.21.15.19:0/3736168310 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c017070 con 0x7f7bb0146f60 2024-03-28T13:21:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.920+0000 7f7ba7400640 1 -- 172.21.15.19:0/3736168310 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b9c00ac00 con 0x7f7bb0146f60 2024-03-28T13:21:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.920+0000 7f7bb5000640 1 -- 172.21.15.19:0/3736168310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb0144a80 con 0x7f7bb0146f60 2024-03-28T13:21:58.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.920+0000 7f7ba7400640 1 -- 172.21.15.19:0/3736168310 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c012400 con 0x7f7bb0146f60 2024-03-28T13:21:58.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.920+0000 7f7bb5000640 1 -- 172.21.15.19:0/3736168310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb01f4bc0 con 0x7f7bb0146f60 2024-03-28T13:21:58.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.921+0000 7f7bb5000640 1 -- 172.21.15.19:0/3736168310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb0075940 con 0x7f7bb0146f60 2024-03-28T13:21:58.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.923+0000 7f7ba7400640 1 -- 172.21.15.19:0/3736168310 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f7b9c01a070 con 0x7f7bb0146f60 2024-03-28T13:21:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.924+0000 7f7ba7400640 1 --2- 172.21.15.19:0/3736168310 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b9006c530 0x7f7b9006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:58.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.924+0000 7f7ba7400640 1 -- 172.21.15.19:0/3736168310 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7f7b9c08b1c0 con 0x7f7bb0146f60 2024-03-28T13:21:58.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.924+0000 7f7baf400640 1 --2- 172.21.15.19:0/3736168310 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b9006c530 0x7f7b9006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:58.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.925+0000 7f7baf400640 1 --2- 172.21.15.19:0/3736168310 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b9006c530 0x7f7b9006e9f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0036000 tx=0x7f7ba0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:58.928+0000 7f7ba7400640 1 -- 172.21.15.19:0/3736168310 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f7b9c059a20 con 0x7f7bb0146f60 2024-03-28T13:21:59.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.132+0000 7f7bb5000640 1 -- 172.21.15.19:0/3736168310 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f7bb00b6bc0 con 0x7f7b9006c530 2024-03-28T13:21:59.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.133+0000 7f7ba7400640 1 -- 172.21.15.19:0/3736168310 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7bb00b6bc0 con 0x7f7b9006c530 2024-03-28T13:21:59.132 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 5 to deep-scrub 2024-03-28T13:21:59.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.136+0000 7f7bb5000640 1 -- 172.21.15.19:0/3736168310 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b9006c530 msgr2=0x7f7b9006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.136+0000 7f7bb5000640 1 --2- 172.21.15.19:0/3736168310 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b9006c530 0x7f7b9006e9f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0036000 tx=0x7f7ba0008030 comp rx=0 tx=0).stop 2024-03-28T13:21:59.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.136+0000 7f7bb5000640 1 -- 172.21.15.19:0/3736168310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb0146f60 msgr2=0x7f7bb01e8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.136+0000 7f7bb5000640 1 --2- 172.21.15.19:0/3736168310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb0146f60 0x7f7bb01e8a00 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00caa0 tx=0x7f7b9c00cf60 comp rx=0 tx=0).stop 2024-03-28T13:21:59.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.136+0000 7f7bb5000640 1 -- 172.21.15.19:0/3736168310 shutdown_connections 2024-03-28T13:21:59.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.136+0000 7f7bb5000640 1 --2- 172.21.15.19:0/3736168310 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f7b9006c530 0x7f7b9006e9f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.136+0000 7f7bb5000640 1 --2- 172.21.15.19:0/3736168310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bb0147570 0x7f7bb01f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.136+0000 7f7bb5000640 1 --2- 172.21.15.19:0/3736168310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb0146f60 0x7f7bb01e8a00 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.136+0000 7f7bb5000640 1 --2- 172.21.15.19:0/3736168310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bb006bc80 0x7f7bb00b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.136+0000 7f7bb5000640 1 -- 172.21.15.19:0/3736168310 >> 172.21.15.19:0/3736168310 conn(0x7f7bb00b2c90 msgr2=0x7f7bb00ae0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:59.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.136+0000 7f7bb5000640 1 -- 172.21.15.19:0/3736168310 shutdown_connections 2024-03-28T13:21:59.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.136+0000 7f7bb5000640 1 -- 172.21.15.19:0/3736168310 wait complete. 2024-03-28T13:21:59.155 INFO:tasks.ceph:Scrubbing osd.6 2024-03-28T13:21:59.155 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-28T13:21:59.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.389+0000 7ff63fe00640 1 Processor -- start 2024-03-28T13:21:59.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.390+0000 7ff63fe00640 1 -- start start 2024-03-28T13:21:59.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.391+0000 7ff63fe00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7ff6400cf230 0x7ff6400b9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:59.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.391+0000 7ff63fe00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6400cfa10 con 0x7ff6400d4890 2024-03-28T13:21:59.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.391+0000 7ff63fe00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff64015e810 con 0x7ff6400d44b0 2024-03-28T13:21:59.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.391+0000 7ff63fe00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7ff6400cf730 con 0x7ff6400cf230 2024-03-28T13:21:59.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.391+0000 7ff63f400640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7ff6400d4890 0x7ff6400b74c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33826/0 (socket says 172.21.15.19:33826) 2024-03-28T13:21:59.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.392+0000 7ff63f400640 1 -- 172.21.15.19:0/979076131 learned_addr learned my addr 172.21.15.19:0/979076131 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:59.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.392+0000 7ff637e00640 1 --2- 172.21.15.19:0/979076131 >> v2:172.21.15.19:3300/0 conn(0x7ff6400cf230 0x7ff6400b9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:59.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.392+0000 7ff63e000640 1 -- 172.21.15.19:0/979076131 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 76070727 0 0) 0x7ff638001490 con 0x7ff6400d4890 2024-03-28T13:21:59.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.392+0000 7ff63e000640 1 -- 172.21.15.19:0/979076131 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff624003520 con 0x7ff6400d4890 2024-03-28T13:21:59.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.392+0000 7ff637e00640 1 -- 172.21.15.19:0/979076131 >> v1:172.21.15.188:6789/0 conn(0x7ff6400d44b0 legacy=0x7ff6400d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.392+0000 7ff637e00640 1 -- 172.21.15.19:0/979076131 >> v1:172.21.15.19:6789/0 conn(0x7ff6400d4890 legacy=0x7ff6400b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.392+0000 7ff637e00640 1 -- 172.21.15.19:0/979076131 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6400d3a30 con 0x7ff6400cf230 2024-03-28T13:21:59.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.393+0000 7ff637e00640 1 --2- 172.21.15.19:0/979076131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6400cf230 0x7ff6400b9bd0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7ff6280097a0 tx=0x7ff62802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e959afec5a66e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:59.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.394+0000 7ff63e000640 1 -- 172.21.15.19:0/979076131 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62802f810 con 0x7ff6400cf230 2024-03-28T13:21:59.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.394+0000 7ff63e000640 1 -- 172.21.15.19:0/979076131 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff62802f9b0 con 0x7ff6400cf230 2024-03-28T13:21:59.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.394+0000 7ff63e000640 1 -- 172.21.15.19:0/979076131 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62802fd80 con 0x7ff6400cf230 2024-03-28T13:21:59.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.394+0000 7ff63fe00640 1 -- 172.21.15.19:0/979076131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6400cf230 msgr2=0x7ff6400b9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.394+0000 7ff63fe00640 1 --2- 172.21.15.19:0/979076131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6400cf230 0x7ff6400b9bd0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7ff6280097a0 tx=0x7ff62802b9f0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.394+0000 7ff63fe00640 1 -- 172.21.15.19:0/979076131 shutdown_connections 2024-03-28T13:21:59.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.394+0000 7ff63fe00640 1 --2- 172.21.15.19:0/979076131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6400cf230 0x7ff6400b9bd0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.394+0000 7ff63fe00640 1 -- 172.21.15.19:0/979076131 >> 172.21.15.19:0/979076131 conn(0x7ff6400676a0 msgr2=0x7ff640067ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:59.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.394+0000 7ff63fe00640 1 -- 172.21.15.19:0/979076131 shutdown_connections 2024-03-28T13:21:59.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.394+0000 7ff63fe00640 1 -- 172.21.15.19:0/979076131 wait complete. 2024-03-28T13:21:59.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.396+0000 7ff63fe00640 1 Processor -- start 2024-03-28T13:21:59.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.396+0000 7ff63fe00640 1 -- start start 2024-03-28T13:21:59.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.397+0000 7ff63fe00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6400cf230 0x7ff6400cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:59.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.397+0000 7ff63fe00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6400d44b0 0x7ff6401f7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:59.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.397+0000 7ff63fe00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400d4890 0x7ff6401f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:59.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.397+0000 7ff63fe00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff6400b7bd0 con 0x7ff6400d44b0 2024-03-28T13:21:59.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.397+0000 7ff63fe00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff6400ba110 con 0x7ff6400cf230 2024-03-28T13:21:59.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.397+0000 7ff63fe00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff6400b7d50 con 0x7ff6400d4890 2024-03-28T13:21:59.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.397+0000 7ff637e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6400d44b0 0x7ff6401f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:59.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.398+0000 7ff637e00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6400d44b0 0x7ff6401f7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45994/0 (socket says 172.21.15.19:45994) 2024-03-28T13:21:59.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.398+0000 7ff637e00640 1 -- 172.21.15.19:0/324067295 learned_addr learned my addr 172.21.15.19:0/324067295 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:59.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.398+0000 7ff63ea00640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6400cf230 0x7ff6400cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:59.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.398+0000 7ff63f400640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400d4890 0x7ff6401f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:59.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.398+0000 7ff637e00640 1 -- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400d4890 msgr2=0x7ff6401f7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.398+0000 7ff637e00640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400d4890 0x7ff6401f7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.398+0000 7ff637e00640 1 -- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6400cf230 msgr2=0x7ff6400cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.398+0000 7ff637e00640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6400cf230 0x7ff6400cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.398+0000 7ff637e00640 1 -- 172.21.15.19:0/324067295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6400c67f0 con 0x7ff6400d44b0 2024-03-28T13:21:59.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.398+0000 7ff63ea00640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6400cf230 0x7ff6400cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:59.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.398+0000 7ff63f400640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6400d4890 0x7ff6401f7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:59.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.398+0000 7ff637e00640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6400d44b0 0x7ff6401f7940 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7ff62802c000 tx=0x7ff628004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:59.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.399+0000 7ff636a00640 1 -- 172.21.15.19:0/324067295 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628047070 con 0x7ff6400d44b0 2024-03-28T13:21:59.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.399+0000 7ff636a00640 1 -- 172.21.15.19:0/324067295 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6280378f0 con 0x7ff6400d44b0 2024-03-28T13:21:59.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.399+0000 7ff636a00640 1 -- 172.21.15.19:0/324067295 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff628037ce0 con 0x7ff6400d44b0 2024-03-28T13:21:59.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.399+0000 7ff63fe00640 1 -- 172.21.15.19:0/324067295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6400cc820 con 0x7ff6400d44b0 2024-03-28T13:21:59.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.399+0000 7ff63fe00640 1 -- 172.21.15.19:0/324067295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6400cd0c0 con 0x7ff6400d44b0 2024-03-28T13:21:59.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.400+0000 7ff63fe00640 1 -- 172.21.15.19:0/324067295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff6400ccc10 con 0x7ff6400d44b0 2024-03-28T13:21:59.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.401+0000 7ff636a00640 1 -- 172.21.15.19:0/324067295 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7ff62804a070 con 0x7ff6400d44b0 2024-03-28T13:21:59.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.402+0000 7ff636a00640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff61c06c560 0x7ff61c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:59.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.402+0000 7ff636a00640 1 -- 172.21.15.19:0/324067295 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7ff6280acff0 con 0x7ff6400d44b0 2024-03-28T13:21:59.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.402+0000 7ff636a00640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x7ff61c072560 0x7ff61c074980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:59.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.402+0000 7ff636a00640 1 -- 172.21.15.19:0/324067295 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff61c075030 con 0x7ff61c072560 2024-03-28T13:21:59.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.402+0000 7ff636a00640 1 -- 172.21.15.19:0/324067295 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7ff6280b4050 con 0x7ff6400d44b0 2024-03-28T13:21:59.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.403+0000 7ff63ea00640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff61c06c560 0x7ff61c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:59.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.403+0000 7ff63f400640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x7ff61c072560 0x7ff61c074980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:59.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.404+0000 7ff63f400640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x7ff61c072560 0x7ff61c074980 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:59.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.404+0000 7ff63ea00640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff61c06c560 0x7ff61c06ea20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff630000f60 tx=0x7ff630009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:59.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.406+0000 7ff636a00640 1 -- 172.21.15.19:0/324067295 <== osd.6 v2:172.21.15.188:6816/3597358155 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff61c075030 con 0x7ff61c072560 2024-03-28T13:21:59.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.421+0000 7ff63fe00640 1 -- 172.21.15.19:0/324067295 --> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff640075940 con 0x7ff61c072560 2024-03-28T13:21:59.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.422+0000 7ff636a00640 1 -- 172.21.15.19:0/324067295 <== osd.6 v2:172.21.15.188:6816/3597358155 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff640075940 con 0x7ff61c072560 2024-03-28T13:21:59.420 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:21:59.421 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T13:21:59.421 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:21:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.422+0000 7ff63fe00640 1 -- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x7ff61c072560 msgr2=0x7ff61c074980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.422+0000 7ff63fe00640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:6816/3597358155,v1:172.21.15.188:6817/3597358155] conn(0x7ff61c072560 0x7ff61c074980 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.423+0000 7ff63fe00640 1 -- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff61c06c560 msgr2=0x7ff61c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.423+0000 7ff63fe00640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7ff61c06c560 0x7ff61c06ea20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff630000f60 tx=0x7ff630009040 comp rx=0 tx=0).stop 2024-03-28T13:21:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.423+0000 7ff63fe00640 1 -- 172.21.15.19:0/324067295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6400d44b0 msgr2=0x7ff6401f7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.423+0000 7ff63fe00640 1 --2- 172.21.15.19:0/324067295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff6400d44b0 0x7ff6401f7940 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7ff62802c000 tx=0x7ff628004730 comp rx=0 tx=0).stop 2024-03-28T13:21:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.423+0000 7ff63f400640 1 -- 172.21.15.19:0/324067295 reap_dead start 2024-03-28T13:21:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.423+0000 7ff63fe00640 1 -- 172.21.15.19:0/324067295 shutdown_connections 2024-03-28T13:21:59.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.423+0000 7ff63fe00640 1 -- 172.21.15.19:0/324067295 >> 172.21.15.19:0/324067295 conn(0x7ff6400676a0 msgr2=0x7ff6400d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:59.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.423+0000 7ff63fe00640 1 -- 172.21.15.19:0/324067295 shutdown_connections 2024-03-28T13:21:59.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.423+0000 7ff63fe00640 1 -- 172.21.15.19:0/324067295 wait complete. 2024-03-28T13:21:59.441 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-28T13:21:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.658+0000 7efc57c00640 1 Processor -- start 2024-03-28T13:21:59.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.658+0000 7efc57c00640 1 -- start start 2024-03-28T13:21:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.660+0000 7efc57c00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7efc50155930 0x7efc500d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.660+0000 7efc57c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc500d07c0 0x7efc500aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.660+0000 7efc57c00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc500b40b0 con 0x7efc50155550 2024-03-28T13:21:59.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.660+0000 7efc57c00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7efc500d0c30 con 0x7efc50155930 2024-03-28T13:21:59.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.660+0000 7efc57c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efc500b4390 con 0x7efc500d07c0 2024-03-28T13:21:59.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.660+0000 7efc55200640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7efc50155550 0x7efc5015f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33828/0 (socket says 172.21.15.19:33828) 2024-03-28T13:21:59.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.660+0000 7efc55200640 1 -- 172.21.15.19:0/2849290463 learned_addr learned my addr 172.21.15.19:0/2849290463 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:21:59.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.660+0000 7efc4fe00640 1 --2- 172.21.15.19:0/2849290463 >> v2:172.21.15.19:3300/0 conn(0x7efc50155930 0x7efc500d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:59.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.661+0000 7efc55c00640 1 --2- 172.21.15.19:0/2849290463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc500d07c0 0x7efc500aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:59.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.661+0000 7efc4fe00640 1 -- 172.21.15.19:0/2849290463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc500d07c0 msgr2=0x7efc500aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:21:59.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.661+0000 7efc4fe00640 1 --2- 172.21.15.19:0/2849290463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc500d07c0 0x7efc500aede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.661+0000 7efc4fe00640 1 -- 172.21.15.19:0/2849290463 >> v1:172.21.15.19:6789/0 conn(0x7efc50155550 legacy=0x7efc5015f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.661+0000 7efc4fe00640 1 -- 172.21.15.19:0/2849290463 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc500b4670 con 0x7efc50155930 2024-03-28T13:21:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.661+0000 7efc4f400640 1 -- 172.21.15.19:0/2849290463 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3144029858 0 0) 0x7efc500b40b0 con 0x7efc50155550 2024-03-28T13:21:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.662+0000 7efc4fe00640 1 --2- 172.21.15.19:0/2849290463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc50155930 0x7efc500d0280 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0097a0 tx=0x7efc3c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e7588b753152cc server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.662+0000 7efc4f400640 1 -- 172.21.15.19:0/2849290463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c035020 con 0x7efc50155930 2024-03-28T13:21:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.662+0000 7efc4f400640 1 -- 172.21.15.19:0/2849290463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc3c031810 con 0x7efc50155930 2024-03-28T13:21:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.662+0000 7efc4f400640 1 -- 172.21.15.19:0/2849290463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c031be0 con 0x7efc50155930 2024-03-28T13:21:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.663+0000 7efc57c00640 1 -- 172.21.15.19:0/2849290463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc50155930 msgr2=0x7efc500d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.663+0000 7efc57c00640 1 --2- 172.21.15.19:0/2849290463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc50155930 0x7efc500d0280 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0097a0 tx=0x7efc3c02b8b0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.663+0000 7efc57c00640 1 -- 172.21.15.19:0/2849290463 shutdown_connections 2024-03-28T13:21:59.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.663+0000 7efc57c00640 1 --2- 172.21.15.19:0/2849290463 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc500d07c0 0x7efc500aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.663+0000 7efc57c00640 1 --2- 172.21.15.19:0/2849290463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc50155930 0x7efc500d0280 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.663+0000 7efc57c00640 1 -- 172.21.15.19:0/2849290463 >> 172.21.15.19:0/2849290463 conn(0x7efc5006b750 msgr2=0x7efc5006bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:59.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.663+0000 7efc57c00640 1 -- 172.21.15.19:0/2849290463 shutdown_connections 2024-03-28T13:21:59.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.663+0000 7efc57c00640 1 -- 172.21.15.19:0/2849290463 wait complete. 2024-03-28T13:21:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.665+0000 7efc57c00640 1 Processor -- start 2024-03-28T13:21:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.665+0000 7efc57c00640 1 -- start start 2024-03-28T13:21:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.666+0000 7efc57c00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc500d07c0 0x7efc50154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.666+0000 7efc57c00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc50155550 0x7efc5009cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:59.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.666+0000 7efc57c00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc50155930 0x7efc5009d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.666+0000 7efc57c00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc5015ba20 con 0x7efc500d07c0 2024-03-28T13:21:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.666+0000 7efc57c00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efc500b4c30 con 0x7efc50155550 2024-03-28T13:21:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.666+0000 7efc57c00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efc5015bba0 con 0x7efc50155930 2024-03-28T13:21:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.666+0000 7efc55200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc500d07c0 0x7efc50154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.666+0000 7efc55200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc500d07c0 0x7efc50154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46008/0 (socket says 172.21.15.19:46008) 2024-03-28T13:21:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.666+0000 7efc55200640 1 -- 172.21.15.19:0/2170444235 learned_addr learned my addr 172.21.15.19:0/2170444235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:21:59.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.667+0000 7efc55c00640 1 --2- 172.21.15.19:0/2170444235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc50155930 0x7efc5009d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.667+0000 7efc4fe00640 1 --2- 172.21.15.19:0/2170444235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc50155550 0x7efc5009cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.667+0000 7efc55200640 1 -- 172.21.15.19:0/2170444235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc50155930 msgr2=0x7efc5009d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.667+0000 7efc55200640 1 --2- 172.21.15.19:0/2170444235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc50155930 0x7efc5009d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.667+0000 7efc55200640 1 -- 172.21.15.19:0/2170444235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc50155550 msgr2=0x7efc5009cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.667+0000 7efc55200640 1 --2- 172.21.15.19:0/2170444235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc50155550 0x7efc5009cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.667+0000 7efc55200640 1 -- 172.21.15.19:0/2170444235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5009bb80 con 0x7efc500d07c0 2024-03-28T13:21:59.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.667+0000 7efc55c00640 1 --2- 172.21.15.19:0/2170444235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc50155930 0x7efc5009d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:21:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.667+0000 7efc55200640 1 --2- 172.21.15.19:0/2170444235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc500d07c0 0x7efc50154940 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7efc4400c4d0 tx=0x7efc4400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.667+0000 7efc4cc00640 1 -- 172.21.15.19:0/2170444235 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44018070 con 0x7efc500d07c0 2024-03-28T13:21:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.667+0000 7efc4cc00640 1 -- 172.21.15.19:0/2170444235 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc44007500 con 0x7efc500d07c0 2024-03-28T13:21:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.667+0000 7efc4cc00640 1 -- 172.21.15.19:0/2170444235 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc44004a20 con 0x7efc500d07c0 2024-03-28T13:21:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.667+0000 7efc57c00640 1 -- 172.21.15.19:0/2170444235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5009be60 con 0x7efc500d07c0 2024-03-28T13:21:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.667+0000 7efc57c00640 1 -- 172.21.15.19:0/2170444235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5009c420 con 0x7efc500d07c0 2024-03-28T13:21:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.669+0000 7efc57c00640 1 -- 172.21.15.19:0/2170444235 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc500685c0 con 0x7efc500d07c0 2024-03-28T13:21:59.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.670+0000 7efc4cc00640 1 -- 172.21.15.19:0/2170444235 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7efc44004bc0 con 0x7efc500d07c0 2024-03-28T13:21:59.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.671+0000 7efc4cc00640 1 --2- 172.21.15.19:0/2170444235 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efc3006c560 0x7efc3006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:21:59.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.671+0000 7efc4cc00640 1 -- 172.21.15.19:0/2170444235 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7efc4408a830 con 0x7efc500d07c0 2024-03-28T13:21:59.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.672+0000 7efc4fe00640 1 --2- 172.21.15.19:0/2170444235 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efc3006c560 0x7efc3006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:21:59.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.673+0000 7efc4fe00640 1 --2- 172.21.15.19:0/2170444235 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efc3006c560 0x7efc3006ea20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7efc3c036000 tx=0x7efc3c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:21:59.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.676+0000 7efc4cc00640 1 -- 172.21.15.19:0/2170444235 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7efc44059000 con 0x7efc500d07c0 2024-03-28T13:21:59.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.880+0000 7efc57c00640 1 -- 172.21.15.19:0/2170444235 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7efc50072d30 con 0x7efc3006c560 2024-03-28T13:21:59.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.881+0000 7efc4cc00640 1 -- 172.21.15.19:0/2170444235 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efc50072d30 con 0x7efc3006c560 2024-03-28T13:21:59.881 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 6 to deep-scrub 2024-03-28T13:21:59.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.884+0000 7efc57c00640 1 -- 172.21.15.19:0/2170444235 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efc3006c560 msgr2=0x7efc3006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.884+0000 7efc57c00640 1 --2- 172.21.15.19:0/2170444235 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efc3006c560 0x7efc3006ea20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7efc3c036000 tx=0x7efc3c008030 comp rx=0 tx=0).stop 2024-03-28T13:21:59.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.884+0000 7efc57c00640 1 -- 172.21.15.19:0/2170444235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc500d07c0 msgr2=0x7efc50154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:21:59.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.884+0000 7efc57c00640 1 --2- 172.21.15.19:0/2170444235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc500d07c0 0x7efc50154940 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7efc4400c4d0 tx=0x7efc4400c990 comp rx=0 tx=0).stop 2024-03-28T13:21:59.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.884+0000 7efc57c00640 1 -- 172.21.15.19:0/2170444235 shutdown_connections 2024-03-28T13:21:59.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.884+0000 7efc57c00640 1 --2- 172.21.15.19:0/2170444235 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7efc3006c560 0x7efc3006ea20 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.884+0000 7efc57c00640 1 --2- 172.21.15.19:0/2170444235 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc50155930 0x7efc5009d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.884+0000 7efc57c00640 1 --2- 172.21.15.19:0/2170444235 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc50155550 0x7efc5009cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.884+0000 7efc57c00640 1 --2- 172.21.15.19:0/2170444235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc500d07c0 0x7efc50154940 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:21:59.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.884+0000 7efc57c00640 1 -- 172.21.15.19:0/2170444235 >> 172.21.15.19:0/2170444235 conn(0x7efc5006b750 msgr2=0x7efc5015c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:21:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.885+0000 7efc57c00640 1 -- 172.21.15.19:0/2170444235 shutdown_connections 2024-03-28T13:21:59.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:21:59.885+0000 7efc57c00640 1 -- 172.21.15.19:0/2170444235 wait complete. 2024-03-28T13:21:59.901 INFO:tasks.ceph:Scrubbing osd.7 2024-03-28T13:21:59.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-28T13:22:00.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.131+0000 7f911ae00640 1 Processor -- start 2024-03-28T13:22:00.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.132+0000 7f911ae00640 1 -- start start 2024-03-28T13:22:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.133+0000 7f911ae00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f91140acf10 0x7f911409dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.133+0000 7f911ae00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f91140ad2f0 0x7f91141453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.133+0000 7f911ae00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f911415cb90 con 0x7f911415ff70 2024-03-28T13:22:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.133+0000 7f911ae00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f9114145f60 con 0x7f91140acf10 2024-03-28T13:22:00.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.133+0000 7f911ae00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f911415c950 con 0x7f91140ad2f0 2024-03-28T13:22:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.134+0000 7f9113e00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f91140acf10 0x7f911409dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.134+0000 7f9118e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f911415ff70 0x7f9114160380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33844/0 (socket says 172.21.15.19:33844) 2024-03-28T13:22:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.134+0000 7f9118e00640 1 -- 172.21.15.19:0/420628202 learned_addr learned my addr 172.21.15.19:0/420628202 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:22:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.134+0000 7f9113400640 1 --2- 172.21.15.19:0/420628202 >> v2:172.21.15.188:3300/0 conn(0x7f91140ad2f0 0x7f91141453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.135+0000 7f9112a00640 1 -- 172.21.15.19:0/420628202 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 909067568 0 0) 0x7f911415cb90 con 0x7f911415ff70 2024-03-28T13:22:00.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.135+0000 7f9112a00640 1 -- 172.21.15.19:0/420628202 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90f4003520 con 0x7f911415ff70 2024-03-28T13:22:00.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.135+0000 7f9113e00640 1 -- 172.21.15.19:0/420628202 >> v2:172.21.15.188:3300/0 conn(0x7f91140ad2f0 msgr2=0x7f91141453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:22:00.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.135+0000 7f9113e00640 1 --2- 172.21.15.19:0/420628202 >> v2:172.21.15.188:3300/0 conn(0x7f91140ad2f0 0x7f91141453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.135+0000 7f9113e00640 1 -- 172.21.15.19:0/420628202 >> v1:172.21.15.19:6789/0 conn(0x7f911415ff70 legacy=0x7f9114160380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.135+0000 7f9113e00640 1 -- 172.21.15.19:0/420628202 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911415ce70 con 0x7f91140acf10 2024-03-28T13:22:00.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.135+0000 7f9113400640 1 --2- 172.21.15.19:0/420628202 >> v2:172.21.15.188:3300/0 conn(0x7f91140ad2f0 0x7f91141453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:22:00.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.136+0000 7f9113e00640 1 --2- 172.21.15.19:0/420628202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91140acf10 0x7f911409dcf0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f91140cf8c0 tx=0x7f910402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447cd32eac2723ca server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:00.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.136+0000 7f9112a00640 1 -- 172.21.15.19:0/420628202 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91040315a0 con 0x7f91140acf10 2024-03-28T13:22:00.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.136+0000 7f9112a00640 1 -- 172.21.15.19:0/420628202 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9104037070 con 0x7f91140acf10 2024-03-28T13:22:00.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.136+0000 7f9112a00640 1 -- 172.21.15.19:0/420628202 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104031940 con 0x7f91140acf10 2024-03-28T13:22:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.137+0000 7f911ae00640 1 -- 172.21.15.19:0/420628202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91140acf10 msgr2=0x7f911409dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.137+0000 7f911ae00640 1 --2- 172.21.15.19:0/420628202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91140acf10 0x7f911409dcf0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f91140cf8c0 tx=0x7f910402b840 comp rx=0 tx=0).stop 2024-03-28T13:22:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.137+0000 7f911ae00640 1 -- 172.21.15.19:0/420628202 shutdown_connections 2024-03-28T13:22:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.137+0000 7f911ae00640 1 --2- 172.21.15.19:0/420628202 >> v2:172.21.15.188:3300/0 conn(0x7f91140ad2f0 0x7f91141453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.137+0000 7f911ae00640 1 --2- 172.21.15.19:0/420628202 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91140acf10 0x7f911409dcf0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.137+0000 7f911ae00640 1 -- 172.21.15.19:0/420628202 >> 172.21.15.19:0/420628202 conn(0x7f911406bba0 msgr2=0x7f91140b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:22:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.137+0000 7f911ae00640 1 -- 172.21.15.19:0/420628202 shutdown_connections 2024-03-28T13:22:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.137+0000 7f911ae00640 1 -- 172.21.15.19:0/420628202 wait complete. 2024-03-28T13:22:00.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.139+0000 7f911ae00640 1 Processor -- start 2024-03-28T13:22:00.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.139+0000 7f911ae00640 1 -- start start 2024-03-28T13:22:00.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.140+0000 7f911ae00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91140acf10 0x7f91140af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.140+0000 7f911ae00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91140ad2f0 0x7f911409bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.140+0000 7f911ae00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f911415ff70 0x7f911409c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.140+0000 7f911ae00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f911415d430 con 0x7f91140ad2f0 2024-03-28T13:22:00.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.140+0000 7f911ae00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f911415d7a0 con 0x7f911415ff70 2024-03-28T13:22:00.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.140+0000 7f911ae00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f91140ad7a0 con 0x7f91140acf10 2024-03-28T13:22:00.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.141+0000 7f9113400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91140ad2f0 0x7f911409bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.141+0000 7f9113400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91140ad2f0 0x7f911409bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46024/0 (socket says 172.21.15.19:46024) 2024-03-28T13:22:00.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.141+0000 7f9113400640 1 -- 172.21.15.19:0/1059464689 learned_addr learned my addr 172.21.15.19:0/1059464689 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:22:00.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.141+0000 7f9113e00640 1 --2- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91140acf10 0x7f91140af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.141+0000 7f9118e00640 1 --2- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f911415ff70 0x7f911409c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.141+0000 7f9113400640 1 -- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91140acf10 msgr2=0x7f91140af090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.141+0000 7f9113400640 1 --2- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91140acf10 0x7f91140af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.141+0000 7f9113400640 1 -- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f911415ff70 msgr2=0x7f911409c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.141+0000 7f9113400640 1 --2- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f911415ff70 0x7f911409c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.141+0000 7f9113400640 1 -- 172.21.15.19:0/1059464689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911413a220 con 0x7f91140ad2f0 2024-03-28T13:22:00.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.141+0000 7f9113400640 1 --2- 172.21.15.19:0/1059464689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91140ad2f0 0x7f911409bc50 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009fc0 tx=0x7f90fc00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:00.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.142+0000 7f910be00640 1 -- 172.21.15.19:0/1059464689 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc01a070 con 0x7f91140ad2f0 2024-03-28T13:22:00.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.142+0000 7f910be00640 1 -- 172.21.15.19:0/1059464689 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90fc003bb0 con 0x7f91140ad2f0 2024-03-28T13:22:00.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.142+0000 7f911ae00640 1 -- 172.21.15.19:0/1059464689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91140af5d0 con 0x7f91140ad2f0 2024-03-28T13:22:00.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.142+0000 7f910be00640 1 -- 172.21.15.19:0/1059464689 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc004030 con 0x7f91140ad2f0 2024-03-28T13:22:00.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.142+0000 7f911ae00640 1 -- 172.21.15.19:0/1059464689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f911413aa50 con 0x7f91140ad2f0 2024-03-28T13:22:00.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.143+0000 7f911ae00640 1 -- 172.21.15.19:0/1059464689 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f911413a5a0 con 0x7f91140ad2f0 2024-03-28T13:22:00.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.144+0000 7f910be00640 1 -- 172.21.15.19:0/1059464689 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f90fc01d070 con 0x7f91140ad2f0 2024-03-28T13:22:00.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.145+0000 7f910be00640 1 --2- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f90f006c560 0x7f90f006ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.145+0000 7f910be00640 1 -- 172.21.15.19:0/1059464689 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7f90fc08b740 con 0x7f91140ad2f0 2024-03-28T13:22:00.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.145+0000 7f910be00640 1 --2- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x7f90f0072560 0x7f90f0074980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.145+0000 7f910be00640 1 -- 172.21.15.19:0/1059464689 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90fc011e50 con 0x7f90f0072560 2024-03-28T13:22:00.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.145+0000 7f910be00640 1 -- 172.21.15.19:0/1059464689 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) ==== 24+0+0 (secure 0 0 0) 0x7f90fc092050 con 0x7f91140ad2f0 2024-03-28T13:22:00.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.146+0000 7f9113e00640 1 --2- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f90f006c560 0x7f90f006ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.146+0000 7f9118e00640 1 --2- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x7f90f0072560 0x7f90f0074980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.147+0000 7f9113e00640 1 --2- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f90f006c560 0x7f90f006ea20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9104036000 tx=0x7f9104008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:00.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.147+0000 7f9118e00640 1 --2- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x7f90f0072560 0x7f90f0074980 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:00.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.149+0000 7f910be00640 1 -- 172.21.15.19:0/1059464689 <== osd.7 v2:172.21.15.188:6824/861796513 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90fc011e50 con 0x7f90f0072560 2024-03-28T13:22:00.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.164+0000 7f911ae00640 1 -- 172.21.15.19:0/1059464689 --> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f911413af30 con 0x7f90f0072560 2024-03-28T13:22:00.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.165+0000 7f910be00640 1 -- 172.21.15.19:0/1059464689 <== osd.7 v2:172.21.15.188:6824/861796513 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f911413af30 con 0x7f90f0072560 2024-03-28T13:22:00.164 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-03-28T13:22:00.164 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-28T13:22:00.164 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-03-28T13:22:00.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.165+0000 7f911ae00640 1 -- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x7f90f0072560 msgr2=0x7f90f0074980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.165+0000 7f911ae00640 1 --2- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:6824/861796513,v1:172.21.15.188:6825/861796513] conn(0x7f90f0072560 0x7f90f0074980 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.166+0000 7f911ae00640 1 -- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f90f006c560 msgr2=0x7f90f006ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.166+0000 7f911ae00640 1 --2- 172.21.15.19:0/1059464689 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f90f006c560 0x7f90f006ea20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9104036000 tx=0x7f9104008030 comp rx=0 tx=0).stop 2024-03-28T13:22:00.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.166+0000 7f911ae00640 1 -- 172.21.15.19:0/1059464689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91140ad2f0 msgr2=0x7f911409bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.166+0000 7f911ae00640 1 --2- 172.21.15.19:0/1059464689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91140ad2f0 0x7f911409bc50 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f90fc009fc0 tx=0x7f90fc00d560 comp rx=0 tx=0).stop 2024-03-28T13:22:00.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.166+0000 7f9118e00640 1 -- 172.21.15.19:0/1059464689 reap_dead start 2024-03-28T13:22:00.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.166+0000 7f911ae00640 1 -- 172.21.15.19:0/1059464689 shutdown_connections 2024-03-28T13:22:00.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.166+0000 7f911ae00640 1 -- 172.21.15.19:0/1059464689 >> 172.21.15.19:0/1059464689 conn(0x7f911406bba0 msgr2=0x7f911415db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:22:00.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.166+0000 7f911ae00640 1 -- 172.21.15.19:0/1059464689 shutdown_connections 2024-03-28T13:22:00.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.166+0000 7f911ae00640 1 -- 172.21.15.19:0/1059464689 wait complete. 2024-03-28T13:22:00.187 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-28T13:22:00.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.417+0000 7f879ba00640 1 Processor -- start 2024-03-28T13:22:00.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.417+0000 7f879ba00640 1 -- start start 2024-03-28T13:22:00.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.419+0000 7f879ba00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8794153580 0x7f87940d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.419+0000 7f879ba00640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f87940af4b0 0x7f87941600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.419+0000 7f879ba00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87940afab0 con 0x7f87940d07c0 2024-03-28T13:22:00.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.419+0000 7f879ba00640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f87940d0c30 con 0x7f87940af4b0 2024-03-28T13:22:00.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.419+0000 7f879ba00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f879415cc80 con 0x7f8794153580 2024-03-28T13:22:00.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.419+0000 7f878be00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7f87940d07c0 0x7f87940aeda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33856/0 (socket says 172.21.15.19:33856) 2024-03-28T13:22:00.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.419+0000 7f878be00640 1 -- 172.21.15.19:0/1557824092 learned_addr learned my addr 172.21.15.19:0/1557824092 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:22:00.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.419+0000 7f8799a00640 1 --2- 172.21.15.19:0/1557824092 >> v2:172.21.15.19:3300/0 conn(0x7f87940af4b0 0x7f87941600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.420+0000 7f8799000640 1 --2- 172.21.15.19:0/1557824092 >> v2:172.21.15.188:3300/0 conn(0x7f8794153580 0x7f87940d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.420+0000 7f8793e00640 1 -- 172.21.15.19:0/1557824092 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 63135046 0 0) 0x7f87940afab0 con 0x7f87940d07c0 2024-03-28T13:22:00.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.420+0000 7f8793e00640 1 -- 172.21.15.19:0/1557824092 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8778003520 con 0x7f87940d07c0 2024-03-28T13:22:00.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.420+0000 7f8799a00640 1 -- 172.21.15.19:0/1557824092 >> v2:172.21.15.188:3300/0 conn(0x7f8794153580 msgr2=0x7f87940d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:22:00.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.420+0000 7f8799a00640 1 --2- 172.21.15.19:0/1557824092 >> v2:172.21.15.188:3300/0 conn(0x7f8794153580 0x7f87940d0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.420+0000 7f8799a00640 1 -- 172.21.15.19:0/1557824092 >> v1:172.21.15.19:6789/0 conn(0x7f87940d07c0 legacy=0x7f87940aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.420+0000 7f8799a00640 1 -- 172.21.15.19:0/1557824092 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879415da00 con 0x7f87940af4b0 2024-03-28T13:22:00.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.421+0000 7f8799a00640 1 --2- 172.21.15.19:0/1557824092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87940af4b0 0x7f87941600d0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f87940b5270 tx=0x7f878c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42195ba6156cf6f server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:00.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.422+0000 7f8793e00640 1 -- 172.21.15.19:0/1557824092 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0315a0 con 0x7f87940af4b0 2024-03-28T13:22:00.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.422+0000 7f8793e00640 1 -- 172.21.15.19:0/1557824092 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f878c037070 con 0x7f87940af4b0 2024-03-28T13:22:00.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.422+0000 7f8793e00640 1 -- 172.21.15.19:0/1557824092 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c031940 con 0x7f87940af4b0 2024-03-28T13:22:00.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.422+0000 7f879ba00640 1 -- 172.21.15.19:0/1557824092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87940af4b0 msgr2=0x7f87941600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.422+0000 7f879ba00640 1 --2- 172.21.15.19:0/1557824092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87940af4b0 0x7f87941600d0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f87940b5270 tx=0x7f878c02b840 comp rx=0 tx=0).stop 2024-03-28T13:22:00.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.422+0000 7f879ba00640 1 -- 172.21.15.19:0/1557824092 shutdown_connections 2024-03-28T13:22:00.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.422+0000 7f879ba00640 1 --2- 172.21.15.19:0/1557824092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87940af4b0 0x7f87941600d0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.422+0000 7f879ba00640 1 --2- 172.21.15.19:0/1557824092 >> v2:172.21.15.188:3300/0 conn(0x7f8794153580 0x7f87940d0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.422+0000 7f879ba00640 1 -- 172.21.15.19:0/1557824092 >> 172.21.15.19:0/1557824092 conn(0x7f879406b750 msgr2=0x7f879406bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:22:00.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.422+0000 7f879ba00640 1 -- 172.21.15.19:0/1557824092 shutdown_connections 2024-03-28T13:22:00.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.422+0000 7f879ba00640 1 -- 172.21.15.19:0/1557824092 wait complete. 2024-03-28T13:22:00.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.424+0000 7f879ba00640 1 Processor -- start 2024-03-28T13:22:00.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.424+0000 7f879ba00640 1 -- start start 2024-03-28T13:22:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.425+0000 7f879ba00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87940af4b0 0x7f879414e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.425+0000 7f879ba00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87940d07c0 0x7f879413fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.425+0000 7f879ba00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8794153580 0x7f87940cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.425+0000 7f879ba00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f879415d4e0 con 0x7f8794153580 2024-03-28T13:22:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.425+0000 7f879ba00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f879415d200 con 0x7f87940d07c0 2024-03-28T13:22:00.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.425+0000 7f879ba00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8794153960 con 0x7f87940af4b0 2024-03-28T13:22:00.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.426+0000 7f8799a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8794153580 0x7f87940cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.426+0000 7f8799a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8794153580 0x7f87940cf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46050/0 (socket says 172.21.15.19:46050) 2024-03-28T13:22:00.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.426+0000 7f8799a00640 1 -- 172.21.15.19:0/3537306165 learned_addr learned my addr 172.21.15.19:0/3537306165 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:22:00.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.426+0000 7f878be00640 1 --2- 172.21.15.19:0/3537306165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87940d07c0 0x7f879413fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.426+0000 7f8799000640 1 --2- 172.21.15.19:0/3537306165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87940af4b0 0x7f879414e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.426+0000 7f8799a00640 1 -- 172.21.15.19:0/3537306165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87940af4b0 msgr2=0x7f879414e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.426+0000 7f8799a00640 1 --2- 172.21.15.19:0/3537306165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87940af4b0 0x7f879414e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.426+0000 7f8799a00640 1 -- 172.21.15.19:0/3537306165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87940d07c0 msgr2=0x7f879413fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.426+0000 7f8799a00640 1 --2- 172.21.15.19:0/3537306165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87940d07c0 0x7f879413fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.426+0000 7f8799a00640 1 -- 172.21.15.19:0/3537306165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8794140180 con 0x7f8794153580 2024-03-28T13:22:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.427+0000 7f8799a00640 1 --2- 172.21.15.19:0/3537306165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8794153580 0x7f87940cf5e0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f878c009890 tx=0x7f878c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.427+0000 7f8791600640 1 -- 172.21.15.19:0/3537306165 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c032630 con 0x7f8794153580 2024-03-28T13:22:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.427+0000 7f8791600640 1 -- 172.21.15.19:0/3537306165 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f878c0327d0 con 0x7f8794153580 2024-03-28T13:22:00.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.427+0000 7f8791600640 1 -- 172.21.15.19:0/3537306165 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c032b80 con 0x7f8794153580 2024-03-28T13:22:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.427+0000 7f879ba00640 1 -- 172.21.15.19:0/3537306165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87940cfb20 con 0x7f8794153580 2024-03-28T13:22:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.427+0000 7f879ba00640 1 -- 172.21.15.19:0/3537306165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f879414f2a0 con 0x7f8794153580 2024-03-28T13:22:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.428+0000 7f879ba00640 1 -- 172.21.15.19:0/3537306165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87940685c0 con 0x7f8794153580 2024-03-28T13:22:00.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.429+0000 7f8791600640 1 -- 172.21.15.19:0/3537306165 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f878c04c020 con 0x7f8794153580 2024-03-28T13:22:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.430+0000 7f8791600640 1 --2- 172.21.15.19:0/3537306165 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f877006c530 0x7f877006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.430+0000 7f8791600640 1 -- 172.21.15.19:0/3537306165 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7f878c02f020 con 0x7f8794153580 2024-03-28T13:22:00.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.431+0000 7f8799000640 1 --2- 172.21.15.19:0/3537306165 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f877006c530 0x7f877006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.433+0000 7f8799000640 1 --2- 172.21.15.19:0/3537306165 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f877006c530 0x7f877006e9f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8794156940 tx=0x7f878400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:00.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.435+0000 7f8791600640 1 -- 172.21.15.19:0/3537306165 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f8794140180 con 0x7f8794153580 2024-03-28T13:22:00.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.640+0000 7f879ba00640 1 -- 172.21.15.19:0/3537306165 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f87940cfd50 con 0x7f877006c530 2024-03-28T13:22:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.641+0000 7f8791600640 1 -- 172.21.15.19:0/3537306165 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f87940cfd50 con 0x7f877006c530 2024-03-28T13:22:00.640 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 7 to deep-scrub 2024-03-28T13:22:00.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.644+0000 7f879ba00640 1 -- 172.21.15.19:0/3537306165 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f877006c530 msgr2=0x7f877006e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.644+0000 7f879ba00640 1 --2- 172.21.15.19:0/3537306165 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f877006c530 0x7f877006e9f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8794156940 tx=0x7f878400b040 comp rx=0 tx=0).stop 2024-03-28T13:22:00.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.644+0000 7f879ba00640 1 -- 172.21.15.19:0/3537306165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8794153580 msgr2=0x7f87940cf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.644+0000 7f879ba00640 1 --2- 172.21.15.19:0/3537306165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8794153580 0x7f87940cf5e0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f878c009890 tx=0x7f878c0046c0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.644+0000 7f879ba00640 1 -- 172.21.15.19:0/3537306165 shutdown_connections 2024-03-28T13:22:00.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.644+0000 7f879ba00640 1 --2- 172.21.15.19:0/3537306165 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f877006c530 0x7f877006e9f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.644+0000 7f879ba00640 1 --2- 172.21.15.19:0/3537306165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8794153580 0x7f87940cf5e0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.644+0000 7f879ba00640 1 --2- 172.21.15.19:0/3537306165 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f87940d07c0 0x7f879413fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.644+0000 7f879ba00640 1 --2- 172.21.15.19:0/3537306165 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f87940af4b0 0x7f879414e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.644+0000 7f879ba00640 1 -- 172.21.15.19:0/3537306165 >> 172.21.15.19:0/3537306165 conn(0x7f879406b750 msgr2=0x7f879406bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:22:00.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.644+0000 7f879ba00640 1 -- 172.21.15.19:0/3537306165 shutdown_connections 2024-03-28T13:22:00.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.645+0000 7f879ba00640 1 -- 172.21.15.19:0/3537306165 wait complete. 2024-03-28T13:22:00.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-28T13:22:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.887+0000 7fdf95a00640 1 Processor -- start 2024-03-28T13:22:00.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.888+0000 7fdf95a00640 1 -- start start 2024-03-28T13:22:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.889+0000 7fdf95a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf900cf1b0 0x7fdf900cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.889+0000 7fdf95a00640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf90071460 con 0x7fdf901442c0 2024-03-28T13:22:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.889+0000 7fdf95a00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf900721b0 con 0x7fdf900d00d0 2024-03-28T13:22:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.889+0000 7fdf95a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdf90071d20 con 0x7fdf900cf1b0 2024-03-28T13:22:00.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.890+0000 7fdf8fe00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fdf901442c0 0x7fdf900ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33870/0 (socket says 172.21.15.19:33870) 2024-03-28T13:22:00.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.890+0000 7fdf8fe00640 1 -- 172.21.15.19:0/804882248 learned_addr learned my addr 172.21.15.19:0/804882248 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:22:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.890+0000 7fdf8f400640 1 -- 172.21.15.19:0/804882248 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 139948654 0 0) 0x7fdf90071460 con 0x7fdf901442c0 2024-03-28T13:22:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.890+0000 7fdf87e00640 1 --2- 172.21.15.19:0/804882248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf900cf1b0 0x7fdf900cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.890+0000 7fdf8f400640 1 -- 172.21.15.19:0/804882248 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf74003520 con 0x7fdf901442c0 2024-03-28T13:22:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.891+0000 7fdf8f400640 1 -- 172.21.15.19:0/804882248 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3692687774 0 0) 0x7fdf74003520 con 0x7fdf901442c0 2024-03-28T13:22:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.891+0000 7fdf8f400640 1 -- 172.21.15.19:0/804882248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf900cf1b0 msgr2=0x7fdf900cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:22:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.891+0000 7fdf8f400640 1 --2- 172.21.15.19:0/804882248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf900cf1b0 0x7fdf900cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.891+0000 7fdf8f400640 1 -- 172.21.15.19:0/804882248 >> v1:172.21.15.188:6789/0 conn(0x7fdf900d00d0 legacy=0x7fdf900b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.891+0000 7fdf8f400640 1 -- 172.21.15.19:0/804882248 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf900716d0 con 0x7fdf901442c0 2024-03-28T13:22:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.891+0000 7fdf8f400640 1 -- 172.21.15.19:0/804882248 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fdf80001670 con 0x7fdf901442c0 2024-03-28T13:22:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.891+0000 7fdf8f400640 1 -- 172.21.15.19:0/804882248 >> v1:172.21.15.19:6789/0 conn(0x7fdf901442c0 legacy=0x7fdf900ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.891+0000 7fdf8f400640 1 --2- 172.21.15.19:0/804882248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf740039e0 0x7fdf74003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.891+0000 7fdf8f400640 1 --2- 172.21.15.19:0/804882248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf74004800 0x7fdf74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.891+0000 7fdf8f400640 1 --2- 172.21.15.19:0/804882248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf740051c0 0x7fdf7400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.891+0000 7fdf8f400640 1 -- 172.21.15.19:0/804882248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf90071d20 con 0x7fdf74004800 2024-03-28T13:22:00.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.891+0000 7fdf8f400640 1 -- 172.21.15.19:0/804882248 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdf7400e280 con 0x7fdf740039e0 2024-03-28T13:22:00.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.891+0000 7fdf8f400640 1 -- 172.21.15.19:0/804882248 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdf7400e480 con 0x7fdf740051c0 2024-03-28T13:22:00.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.891+0000 7fdf87e00640 1 --2- 172.21.15.19:0/804882248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf900cf1b0 0x7fdf900cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-28T13:22:00.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.891+0000 7fdf87e00640 1 --2- 172.21.15.19:0/804882248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf74004800 0x7fdf74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.892+0000 7fdf8fe00640 1 --2- 172.21.15.19:0/804882248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf740039e0 0x7fdf74003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.892+0000 7fdf95000640 1 --2- 172.21.15.19:0/804882248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf740051c0 0x7fdf7400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.892+0000 7fdf87e00640 1 -- 172.21.15.19:0/804882248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf740051c0 msgr2=0x7fdf7400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:22:00.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.892+0000 7fdf87e00640 1 --2- 172.21.15.19:0/804882248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf740051c0 0x7fdf7400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.892+0000 7fdf87e00640 1 -- 172.21.15.19:0/804882248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf740039e0 msgr2=0x7fdf74003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:22:00.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.892+0000 7fdf87e00640 1 --2- 172.21.15.19:0/804882248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf740039e0 0x7fdf74003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.892+0000 7fdf87e00640 1 -- 172.21.15.19:0/804882248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf74003520 con 0x7fdf74004800 2024-03-28T13:22:00.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.893+0000 7fdf87e00640 1 --2- 172.21.15.19:0/804882248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf74004800 0x7fdf74004c80 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fdf78006fd0 tx=0x7fdf74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d267797dc521bf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:00.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.893+0000 7fdf8f400640 1 -- 172.21.15.19:0/804882248 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800f040 con 0x7fdf74004800 2024-03-28T13:22:00.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.893+0000 7fdf8f400640 1 -- 172.21.15.19:0/804882248 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf78033600 con 0x7fdf74004800 2024-03-28T13:22:00.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.893+0000 7fdf8f400640 1 -- 172.21.15.19:0/804882248 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78033a10 con 0x7fdf74004800 2024-03-28T13:22:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.894+0000 7fdf95a00640 1 -- 172.21.15.19:0/804882248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf74004800 msgr2=0x7fdf74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.894+0000 7fdf95a00640 1 --2- 172.21.15.19:0/804882248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf74004800 0x7fdf74004c80 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fdf78006fd0 tx=0x7fdf74005830 comp rx=0 tx=0).stop 2024-03-28T13:22:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.894+0000 7fdf95000640 1 -- 172.21.15.19:0/804882248 reap_dead start 2024-03-28T13:22:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.894+0000 7fdf95a00640 1 -- 172.21.15.19:0/804882248 shutdown_connections 2024-03-28T13:22:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.894+0000 7fdf95a00640 1 -- 172.21.15.19:0/804882248 >> 172.21.15.19:0/804882248 conn(0x7fdf900ccad0 msgr2=0x7fdf900cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:22:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.894+0000 7fdf95a00640 1 -- 172.21.15.19:0/804882248 shutdown_connections 2024-03-28T13:22:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.894+0000 7fdf95a00640 1 -- 172.21.15.19:0/804882248 wait complete. 2024-03-28T13:22:00.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.896+0000 7fdf95a00640 1 Processor -- start 2024-03-28T13:22:00.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.896+0000 7fdf95a00640 1 -- start start 2024-03-28T13:22:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.897+0000 7fdf95a00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf74004800 0x7fdf901e5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.897+0000 7fdf95a00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf9015b720 0x7fdf9015bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.897+0000 7fdf95a00640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf9015a720 0x7fdf9015aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.897+0000 7fdf95a00640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdf901446a0 con 0x7fdf9015a720 2024-03-28T13:22:00.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.897+0000 7fdf95a00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdf900d0770 con 0x7fdf74004800 2024-03-28T13:22:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.897+0000 7fdf95a00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdf9015b0e0 con 0x7fdf9015b720 2024-03-28T13:22:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.898+0000 7fdf95000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf9015a720 0x7fdf9015aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.898+0000 7fdf95000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf9015a720 0x7fdf9015aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46056/0 (socket says 172.21.15.19:46056) 2024-03-28T13:22:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.898+0000 7fdf95000640 1 -- 172.21.15.19:0/1660110987 learned_addr learned my addr 172.21.15.19:0/1660110987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:22:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.898+0000 7fdf8fe00640 1 --2- 172.21.15.19:0/1660110987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf74004800 0x7fdf901e5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.898+0000 7fdf87e00640 1 --2- 172.21.15.19:0/1660110987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf9015b720 0x7fdf9015bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.898+0000 7fdf95000640 1 -- 172.21.15.19:0/1660110987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf9015b720 msgr2=0x7fdf9015bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.898+0000 7fdf95000640 1 --2- 172.21.15.19:0/1660110987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf9015b720 0x7fdf9015bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.898+0000 7fdf95000640 1 -- 172.21.15.19:0/1660110987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf74004800 msgr2=0x7fdf901e5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.898+0000 7fdf95000640 1 --2- 172.21.15.19:0/1660110987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf74004800 0x7fdf901e5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.898+0000 7fdf95000640 1 -- 172.21.15.19:0/1660110987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9015c220 con 0x7fdf9015a720 2024-03-28T13:22:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.898+0000 7fdf95000640 1 --2- 172.21.15.19:0/1660110987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf9015a720 0x7fdf9015aba0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fdf900cdf20 tx=0x7fdf8800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.899+0000 7fdf8cc00640 1 -- 172.21.15.19:0/1660110987 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88003970 con 0x7fdf9015a720 2024-03-28T13:22:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.899+0000 7fdf8cc00640 1 -- 172.21.15.19:0/1660110987 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf88018070 con 0x7fdf9015a720 2024-03-28T13:22:00.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.899+0000 7fdf8cc00640 1 -- 172.21.15.19:0/1660110987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88003d50 con 0x7fdf9015a720 2024-03-28T13:22:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.899+0000 7fdf95a00640 1 -- 172.21.15.19:0/1660110987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9015c500 con 0x7fdf9015a720 2024-03-28T13:22:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.899+0000 7fdf95a00640 1 -- 172.21.15.19:0/1660110987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9015cac0 con 0x7fdf9015a720 2024-03-28T13:22:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.900+0000 7fdf95a00640 1 -- 172.21.15.19:0/1660110987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf90075940 con 0x7fdf9015a720 2024-03-28T13:22:00.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.901+0000 7fdf8cc00640 1 -- 172.21.15.19:0/1660110987 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fdf880050d0 con 0x7fdf9015a720 2024-03-28T13:22:00.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.902+0000 7fdf8cc00640 1 --2- 172.21.15.19:0/1660110987 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf6c06c560 0x7fdf6c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:00.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.902+0000 7fdf8cc00640 1 -- 172.21.15.19:0/1660110987 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7fdf8808b2d0 con 0x7fdf9015a720 2024-03-28T13:22:00.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.903+0000 7fdf8fe00640 1 --2- 172.21.15.19:0/1660110987 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf6c06c560 0x7fdf6c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:00.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.904+0000 7fdf8fe00640 1 --2- 172.21.15.19:0/1660110987 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf6c06c560 0x7fdf6c06ea20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdf80000f60 tx=0x7fdf80000ed0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:00.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:00.906+0000 7fdf8cc00640 1 -- 172.21.15.19:0/1660110987 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fdf88059a10 con 0x7fdf9015a720 2024-03-28T13:22:01.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.110+0000 7fdf95a00640 1 -- 172.21.15.19:0/1660110987 --> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdf900b0ac0 con 0x7fdf6c06c560 2024-03-28T13:22:01.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.112+0000 7fdf8cc00640 1 -- 172.21.15.19:0/1660110987 <== mgr.4108 v2:172.21.15.188:6832/3382915391 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13415 (secure 0 0 0) 0x7fdf900b0ac0 con 0x7fdf6c06c560 2024-03-28T13:22:01.111 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:22:01.112 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-03-28T13:22:01.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.115+0000 7fdf95a00640 1 -- 172.21.15.19:0/1660110987 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf6c06c560 msgr2=0x7fdf6c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.115+0000 7fdf95a00640 1 --2- 172.21.15.19:0/1660110987 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf6c06c560 0x7fdf6c06ea20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdf80000f60 tx=0x7fdf80000ed0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.115+0000 7fdf95a00640 1 -- 172.21.15.19:0/1660110987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf9015a720 msgr2=0x7fdf9015aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:01.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.115+0000 7fdf95a00640 1 --2- 172.21.15.19:0/1660110987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf9015a720 0x7fdf9015aba0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fdf900cdf20 tx=0x7fdf8800ec10 comp rx=0 tx=0).stop 2024-03-28T13:22:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.115+0000 7fdf95a00640 1 -- 172.21.15.19:0/1660110987 shutdown_connections 2024-03-28T13:22:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.115+0000 7fdf95a00640 1 --2- 172.21.15.19:0/1660110987 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fdf6c06c560 0x7fdf6c06ea20 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.115+0000 7fdf95a00640 1 --2- 172.21.15.19:0/1660110987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdf9015a720 0x7fdf9015aba0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.115+0000 7fdf95a00640 1 --2- 172.21.15.19:0/1660110987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf9015b720 0x7fdf9015bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.115+0000 7fdf95a00640 1 --2- 172.21.15.19:0/1660110987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf74004800 0x7fdf901e5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.115+0000 7fdf95a00640 1 -- 172.21.15.19:0/1660110987 >> 172.21.15.19:0/1660110987 conn(0x7fdf900ccad0 msgr2=0x7fdf9006ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:22:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.116+0000 7fdf95a00640 1 -- 172.21.15.19:0/1660110987 shutdown_connections 2024-03-28T13:22:01.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.116+0000 7fdf95a00640 1 -- 172.21.15.19:0/1660110987 wait complete. 2024-03-28T13:22:01.134 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2024-03-28T13:22:00.267776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262804,"kb_used_data":13352,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711916,"statfs":{"total":773094113280,"available":772825001984,"internally_reserved":0,"allocated":13672448,"data_stored":6630088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12748,"internal_metadata":255315508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.872675"},"pg_stats":[{"pgid":"1.0","version":"126'119","reported_seq":375,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-03-28T13:21:56.406202+0000","last_change":"2024-03-28T13:21:56.406259+0000","last_active":"2024-03-28T13:21:56.406202+0000","last_peered":"2024-03-28T13:21:56.406202+0000","last_clean":"2024-03-28T13:21:56.406202+0000","last_became_active":"2024-03-28T13:09:31.925975+0000","last_became_peered":"2024-03-28T13:09:31.925975+0000","last_unstale":"2024-03-28T13:21:56.406202+0000","last_undegraded":"2024-03-28T13:21:56.406202+0000","last_fullsized":"2024-03-28T13:21:56.406202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"126'119","last_scrub_stamp":"2024-03-28T13:21:56.406073+0000","last_deep_scrub":"126'119","last_deep_scrub_stamp":"2024-03-28T13:21:56.406073+0000","last_clean_scrub_stamp":"2024-03-28T13:21:56.406073+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-29T19:35:16.520653+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":681169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":681169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476890,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33280,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2154496,"data_stored":1271537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":681169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509593,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":681169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509594,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33280,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338560,"statfs":{"total":96636764160,"available":96602685440,"internally_reserved":0,"allocated":2154496,"data_stored":1271537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509594,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32708,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1560576,"data_stored":681169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542298,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32704,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1560576,"data_stored":681169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-28T13:22:01.138 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-28T13:22:01.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.348+0000 7f03f3000640 1 Processor -- start 2024-03-28T13:22:01.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.348+0000 7f03f3000640 1 -- start start 2024-03-28T13:22:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.350+0000 7f03f3000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f03ec0cc0b0 0x7f03ec06dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.350+0000 7f03f3000640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03ec15fc50 0x7f03ec0cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.350+0000 7f03f3000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03ec0cc490 0x7f03ec0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.350+0000 7f03f3000640 1 -- --> v2:172.21.15.19:3300/0 -- mon_getmap magic: 0 -- 0x7f03ec15c610 con 0x7f03ec15fc50 2024-03-28T13:22:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.350+0000 7f03f3000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f03ec06e4f0 con 0x7f03ec0cc0b0 2024-03-28T13:22:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.350+0000 7f03f3000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f03ec0b3150 con 0x7f03ec0cc490 2024-03-28T13:22:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.350+0000 7f03f1200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03ec15fc50 0x7f03ec0cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.350+0000 7f03f1200640 1 --2- >> v2:172.21.15.19:3300/0 conn(0x7f03ec15fc50 0x7f03ec0cfdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46066/0 (socket says 172.21.15.19:46066) 2024-03-28T13:22:01.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.350+0000 7f03f1200640 1 -- 172.21.15.19:0/4259507409 learned_addr learned my addr 172.21.15.19:0/4259507409 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:22:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.351+0000 7f03f1c00640 1 --2- 172.21.15.19:0/4259507409 >> v2:172.21.15.188:3300/0 conn(0x7f03ec0cc0b0 0x7f03ec06dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.351+0000 7f03f1200640 1 -- 172.21.15.19:0/4259507409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03ec0cc490 msgr2=0x7f03ec0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:22:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.351+0000 7f03f2600640 1 --2- 172.21.15.19:0/4259507409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03ec0cc490 0x7f03ec0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.351+0000 7f03f1200640 1 --2- 172.21.15.19:0/4259507409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03ec0cc490 0x7f03ec0d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.351+0000 7f03f1200640 1 -- 172.21.15.19:0/4259507409 >> v2:172.21.15.188:3300/0 conn(0x7f03ec0cc0b0 msgr2=0x7f03ec06dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:22:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.351+0000 7f03f1200640 1 --2- 172.21.15.19:0/4259507409 >> v2:172.21.15.188:3300/0 conn(0x7f03ec0cc0b0 0x7f03ec06dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.351+0000 7f03f1200640 1 -- 172.21.15.19:0/4259507409 --> v2:172.21.15.19:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ec1600e0 con 0x7f03ec15fc50 2024-03-28T13:22:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.351+0000 7f03f1c00640 1 --2- 172.21.15.19:0/4259507409 >> v2:172.21.15.188:3300/0 conn(0x7f03ec0cc0b0 0x7f03ec06dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:22:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.351+0000 7f03f2600640 1 --2- 172.21.15.19:0/4259507409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03ec0cc490 0x7f03ec0d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:22:01.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.352+0000 7f03f1200640 1 --2- 172.21.15.19:0/4259507409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ec15fc50 0x7f03ec0cfdc0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0cfd10 tx=0x7f03d802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd10f4e6d466a23 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.352+0000 7f03ebe00640 1 -- 172.21.15.19:0/4259507409 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8035020 con 0x7f03ec15fc50 2024-03-28T13:22:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.352+0000 7f03ebe00640 1 -- 172.21.15.19:0/4259507409 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d802d040 con 0x7f03ec15fc50 2024-03-28T13:22:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.352+0000 7f03ebe00640 1 -- 172.21.15.19:0/4259507409 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8031a80 con 0x7f03ec15fc50 2024-03-28T13:22:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.353+0000 7f03f3000640 1 -- 172.21.15.19:0/4259507409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ec15fc50 msgr2=0x7f03ec0cfdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.353+0000 7f03f3000640 1 --2- 172.21.15.19:0/4259507409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ec15fc50 0x7f03ec0cfdc0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0cfd10 tx=0x7f03d802b900 comp rx=0 tx=0).stop 2024-03-28T13:22:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.353+0000 7f03f3000640 1 -- 172.21.15.19:0/4259507409 shutdown_connections 2024-03-28T13:22:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.353+0000 7f03f3000640 1 --2- 172.21.15.19:0/4259507409 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03ec0cc490 0x7f03ec0d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.353+0000 7f03f3000640 1 --2- 172.21.15.19:0/4259507409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ec15fc50 0x7f03ec0cfdc0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.353+0000 7f03f3000640 1 --2- 172.21.15.19:0/4259507409 >> v2:172.21.15.188:3300/0 conn(0x7f03ec0cc0b0 0x7f03ec06dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.353+0000 7f03f3000640 1 -- 172.21.15.19:0/4259507409 >> 172.21.15.19:0/4259507409 conn(0x7f03ec0ccad0 msgr2=0x7f03ec0cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:22:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.353+0000 7f03f3000640 1 -- 172.21.15.19:0/4259507409 shutdown_connections 2024-03-28T13:22:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.353+0000 7f03f3000640 1 -- 172.21.15.19:0/4259507409 wait complete. 2024-03-28T13:22:01.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.355+0000 7f03f3000640 1 Processor -- start 2024-03-28T13:22:01.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.355+0000 7f03f3000640 1 -- start start 2024-03-28T13:22:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.356+0000 7f03f3000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03ec0cc0b0 0x7f03ec154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.356+0000 7f03f3000640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ec0cc490 0x7f03ec1f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.356+0000 7f03f3000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03ec145820 0x7f03ec13bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.356+0000 7f03f3000640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec160980 con 0x7f03ec0cc490 2024-03-28T13:22:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.356+0000 7f03f3000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f03ec1606a0 con 0x7f03ec0cc0b0 2024-03-28T13:22:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.356+0000 7f03f3000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f03ec15ca70 con 0x7f03ec145820 2024-03-28T13:22:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.357+0000 7f03f1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ec0cc490 0x7f03ec1f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.357+0000 7f03f1200640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ec0cc490 0x7f03ec1f6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46080/0 (socket says 172.21.15.19:46080) 2024-03-28T13:22:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.357+0000 7f03f1200640 1 -- 172.21.15.19:0/1648105403 learned_addr learned my addr 172.21.15.19:0/1648105403 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:22:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.357+0000 7f03f1c00640 1 --2- 172.21.15.19:0/1648105403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03ec0cc0b0 0x7f03ec154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.357+0000 7f03f2600640 1 --2- 172.21.15.19:0/1648105403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03ec145820 0x7f03ec13bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.357+0000 7f03f1200640 1 -- 172.21.15.19:0/1648105403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03ec145820 msgr2=0x7f03ec13bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.357+0000 7f03f1200640 1 --2- 172.21.15.19:0/1648105403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03ec145820 0x7f03ec13bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.357+0000 7f03f1200640 1 -- 172.21.15.19:0/1648105403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03ec0cc0b0 msgr2=0x7f03ec154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.357+0000 7f03f1200640 1 --2- 172.21.15.19:0/1648105403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03ec0cc0b0 0x7f03ec154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.357+0000 7f03f1200640 1 -- 172.21.15.19:0/1648105403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ec15f6f0 con 0x7f03ec0cc490 2024-03-28T13:22:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.357+0000 7f03f2600640 1 --2- 172.21.15.19:0/1648105403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03ec145820 0x7f03ec13bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-28T13:22:01.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.358+0000 7f03f1200640 1 --2- 172.21.15.19:0/1648105403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ec0cc490 0x7f03ec1f6de0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f03d8000c00 tx=0x7f03d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.358+0000 7f03e9600640 1 -- 172.21.15.19:0/1648105403 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8040070 con 0x7f03ec0cc490 2024-03-28T13:22:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.358+0000 7f03e9600640 1 -- 172.21.15.19:0/1648105403 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d8039bf0 con 0x7f03ec0cc490 2024-03-28T13:22:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.358+0000 7f03e9600640 1 -- 172.21.15.19:0/1648105403 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8035040 con 0x7f03ec0cc490 2024-03-28T13:22:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.358+0000 7f03f3000640 1 -- 172.21.15.19:0/1648105403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ec1553a0 con 0x7f03ec0cc490 2024-03-28T13:22:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.358+0000 7f03f3000640 1 -- 172.21.15.19:0/1648105403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ec154b00 con 0x7f03ec0cc490 2024-03-28T13:22:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.359+0000 7f03f3000640 1 -- 172.21.15.19:0/1648105403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03ec075940 con 0x7f03ec0cc490 2024-03-28T13:22:01.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.360+0000 7f03e9600640 1 -- 172.21.15.19:0/1648105403 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7f03d8038020 con 0x7f03ec0cc490 2024-03-28T13:22:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.361+0000 7f03e9600640 1 --2- 172.21.15.19:0/1648105403 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f03cc06c560 0x7f03cc06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.361+0000 7f03e9600640 1 -- 172.21.15.19:0/1648105403 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7f03d80ad230 con 0x7f03ec0cc490 2024-03-28T13:22:01.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.362+0000 7f03f1c00640 1 --2- 172.21.15.19:0/1648105403 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f03cc06c560 0x7f03cc06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.363+0000 7f03f1c00640 1 --2- 172.21.15.19:0/1648105403 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f03cc06c560 0x7f03cc06ea20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f03e000a990 tx=0x7f03e0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:01.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.366+0000 7f03e9600640 1 -- 172.21.15.19:0/1648105403 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7f03d807ba90 con 0x7f03ec0cc490 2024-03-28T13:22:01.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.580+0000 7f03f3000640 1 -- 172.21.15.19:0/1648105403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f03ec073830 con 0x7f03ec0cc490 2024-03-28T13:22:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.588+0000 7f03e9600640 1 -- 172.21.15.19:0/1648105403 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) ==== 123+0+0 (secure 0 0 0) 0x7f03d807b390 con 0x7f03ec0cc490 2024-03-28T13:22:01.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.590+0000 7f03e9600640 1 -- 172.21.15.19:0/1648105403 <== mon.0 v2:172.21.15.19:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f03d807f740 con 0x7f03ec0cc490 2024-03-28T13:22:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.591+0000 7f03f3000640 1 -- 172.21.15.19:0/1648105403 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f03cc06c560 msgr2=0x7f03cc06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.591+0000 7f03f3000640 1 --2- 172.21.15.19:0/1648105403 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f03cc06c560 0x7f03cc06ea20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f03e000a990 tx=0x7f03e0008040 comp rx=0 tx=0).stop 2024-03-28T13:22:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.592+0000 7f03f3000640 1 -- 172.21.15.19:0/1648105403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ec0cc490 msgr2=0x7f03ec1f6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.592+0000 7f03f3000640 1 --2- 172.21.15.19:0/1648105403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ec0cc490 0x7f03ec1f6de0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f03d8000c00 tx=0x7f03d80046c0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.592+0000 7f03f3000640 1 -- 172.21.15.19:0/1648105403 shutdown_connections 2024-03-28T13:22:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.592+0000 7f03f3000640 1 --2- 172.21.15.19:0/1648105403 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7f03cc06c560 0x7f03cc06ea20 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.592+0000 7f03f3000640 1 --2- 172.21.15.19:0/1648105403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03ec145820 0x7f03ec13bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.592+0000 7f03f3000640 1 --2- 172.21.15.19:0/1648105403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03ec0cc490 0x7f03ec1f6de0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.592+0000 7f03f3000640 1 --2- 172.21.15.19:0/1648105403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03ec0cc0b0 0x7f03ec154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.592+0000 7f03f3000640 1 -- 172.21.15.19:0/1648105403 >> 172.21.15.19:0/1648105403 conn(0x7f03ec0ccad0 msgr2=0x7f03ec15b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:22:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.592+0000 7f03f3000640 1 -- 172.21.15.19:0/1648105403 shutdown_connections 2024-03-28T13:22:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.592+0000 7f03f3000640 1 -- 172.21.15.19:0/1648105403 wait complete. 2024-03-28T13:22:01.611 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-28T13:22:01.611 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-28T13:22:01.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.832+0000 7fa0ea400640 1 Processor -- start 2024-03-28T13:22:01.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.833+0000 7fa0ea400640 1 -- start start 2024-03-28T13:22:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.834+0000 7fa0ea400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e4161b00 0x7fa0e4162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.834+0000 7fa0ea400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa0e40ad120 0x7fa0e40d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.834+0000 7fa0ea400640 1 -- --> v1:172.21.15.19:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0e4071b60 con 0x7fa0e40ad580 2024-03-28T13:22:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.834+0000 7fa0ea400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa0e4161ee0 con 0x7fa0e40ad120 2024-03-28T13:22:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.834+0000 7fa0ea400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e40b3450 con 0x7fa0e4161b00 2024-03-28T13:22:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.835+0000 7fa0e3e00640 1 --1- >> v1:172.21.15.19:6789/0 conn(0x7fa0e40ad580 0x7fa0e40b3bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.19:6789/0 says I am v1:172.21.15.19:33884/0 (socket says 172.21.15.19:33884) 2024-03-28T13:22:01.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.835+0000 7fa0e3e00640 1 -- 172.21.15.19:0/388043030 learned_addr learned my addr 172.21.15.19:0/388043030 (peer_addr_for_me v1:172.21.15.19:0/0) 2024-03-28T13:22:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.835+0000 7fa0e3400640 1 --2- 172.21.15.19:0/388043030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e4161b00 0x7fa0e4162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.835+0000 7fa0e2000640 1 -- 172.21.15.19:0/388043030 <== mon.0 v1:172.21.15.19:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 751251882 0 0) 0x7fa0e4071b60 con 0x7fa0e40ad580 2024-03-28T13:22:01.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.835+0000 7fa0e2000640 1 -- 172.21.15.19:0/388043030 --> v1:172.21.15.19:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0c8003520 con 0x7fa0e40ad580 2024-03-28T13:22:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.835+0000 7fa0e2a00640 1 --2- 172.21.15.19:0/388043030 >> v2:172.21.15.188:3300/0 conn(0x7fa0e40ad120 0x7fa0e40d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.835+0000 7fa0e2000640 1 -- 172.21.15.19:0/388043030 <== mon.0 v1:172.21.15.19:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3717976687 0 0) 0x7fa0c8003520 con 0x7fa0e40ad580 2024-03-28T13:22:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.835+0000 7fa0e2000640 1 -- 172.21.15.19:0/388043030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e4161b00 msgr2=0x7fa0e4162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:22:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.835+0000 7fa0e2000640 1 --2- 172.21.15.19:0/388043030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e4161b00 0x7fa0e4162ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e2000640 1 -- 172.21.15.19:0/388043030 >> v2:172.21.15.188:3300/0 conn(0x7fa0e40ad120 msgr2=0x7fa0e40d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:22:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e2000640 1 --2- 172.21.15.19:0/388043030 >> v2:172.21.15.188:3300/0 conn(0x7fa0e40ad120 0x7fa0e40d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e2000640 1 -- 172.21.15.19:0/388043030 --> v1:172.21.15.19:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e4071e40 con 0x7fa0e40ad580 2024-03-28T13:22:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e2000640 1 -- 172.21.15.19:0/388043030 <== mon.0 v1:172.21.15.19:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2603076269 0 0) 0x7fa0dc001670 con 0x7fa0e40ad580 2024-03-28T13:22:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e2000640 1 -- 172.21.15.19:0/388043030 >> v1:172.21.15.19:6789/0 conn(0x7fa0e40ad580 legacy=0x7fa0e40b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e2000640 1 --2- 172.21.15.19:0/388043030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c80039b0 0x7fa0c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e2000640 1 --2- 172.21.15.19:0/388043030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c8004800 0x7fa0c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e2000640 1 --2- 172.21.15.19:0/388043030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e4161b00 0x7fa0c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e2000640 1 -- 172.21.15.19:0/388043030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e4161ee0 con 0x7fa0c80039b0 2024-03-28T13:22:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e2000640 1 -- 172.21.15.19:0/388043030 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e40b3450 con 0x7fa0c8004800 2024-03-28T13:22:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e2000640 1 -- 172.21.15.19:0/388043030 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa0c800ba10 con 0x7fa0e4161b00 2024-03-28T13:22:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e3400640 1 --2- 172.21.15.19:0/388043030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c80039b0 0x7fa0c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e3e00640 1 --2- 172.21.15.19:0/388043030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e4161b00 0x7fa0c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e3400640 1 -- 172.21.15.19:0/388043030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e4161b00 msgr2=0x7fa0c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:22:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e3400640 1 --2- 172.21.15.19:0/388043030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e4161b00 0x7fa0c80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e3400640 1 -- 172.21.15.19:0/388043030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c8004800 msgr2=0x7fa0c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-28T13:22:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e2a00640 1 --2- 172.21.15.19:0/388043030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c8004800 0x7fa0c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e3400640 1 --2- 172.21.15.19:0/388043030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0c8004800 0x7fa0c8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.836+0000 7fa0e3400640 1 -- 172.21.15.19:0/388043030 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c8003520 con 0x7fa0c80039b0 2024-03-28T13:22:01.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.838+0000 7fa0e3400640 1 --2- 172.21.15.19:0/388043030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c80039b0 0x7fa0c8003dd0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009fc0 tx=0x7fa0c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965dede487e51755 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:01.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.838+0000 7fa0e2000640 1 -- 172.21.15.19:0/388043030 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4039070 con 0x7fa0c80039b0 2024-03-28T13:22:01.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.838+0000 7fa0e2000640 1 -- 172.21.15.19:0/388043030 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa0d402cc20 con 0x7fa0c80039b0 2024-03-28T13:22:01.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.838+0000 7fa0e2000640 1 -- 172.21.15.19:0/388043030 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4009c10 con 0x7fa0c80039b0 2024-03-28T13:22:01.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.838+0000 7fa0ea400640 1 -- 172.21.15.19:0/388043030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c80039b0 msgr2=0x7fa0c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:01.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.838+0000 7fa0ea400640 1 --2- 172.21.15.19:0/388043030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c80039b0 0x7fa0c8003dd0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009fc0 tx=0x7fa0c8005830 comp rx=0 tx=0).stop 2024-03-28T13:22:01.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.838+0000 7fa0e3e00640 1 -- 172.21.15.19:0/388043030 reap_dead start 2024-03-28T13:22:01.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.839+0000 7fa0ea400640 1 -- 172.21.15.19:0/388043030 shutdown_connections 2024-03-28T13:22:01.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.839+0000 7fa0ea400640 1 -- 172.21.15.19:0/388043030 >> 172.21.15.19:0/388043030 conn(0x7fa0e406b990 msgr2=0x7fa0e406bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:22:01.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.839+0000 7fa0ea400640 1 -- 172.21.15.19:0/388043030 shutdown_connections 2024-03-28T13:22:01.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.839+0000 7fa0ea400640 1 -- 172.21.15.19:0/388043030 wait complete. 2024-03-28T13:22:01.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.840+0000 7fa0ea400640 1 Processor -- start 2024-03-28T13:22:01.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.841+0000 7fa0ea400640 1 -- start start 2024-03-28T13:22:01.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0ea400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c80039b0 0x7fa0e4156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0ea400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e415f3f0 0x7fa0e415f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0ea400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e41f7d30 0x7fa0e41f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0ea400640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e40717f0 con 0x7fa0c80039b0 2024-03-28T13:22:01.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0ea400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e40b3730 con 0x7fa0e41f7d30 2024-03-28T13:22:01.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0ea400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e40b5020 con 0x7fa0e415f3f0 2024-03-28T13:22:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0e3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c80039b0 0x7fa0e4156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0e3400640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c80039b0 0x7fa0e4156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46098/0 (socket says 172.21.15.19:46098) 2024-03-28T13:22:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0e3400640 1 -- 172.21.15.19:0/4194210852 learned_addr learned my addr 172.21.15.19:0/4194210852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-03-28T13:22:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0e2a00640 1 --2- 172.21.15.19:0/4194210852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e415f3f0 0x7fa0e415f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0e3e00640 1 --2- 172.21.15.19:0/4194210852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e41f7d30 0x7fa0e41f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0e3400640 1 -- 172.21.15.19:0/4194210852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e415f3f0 msgr2=0x7fa0e415f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0e3400640 1 --2- 172.21.15.19:0/4194210852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e415f3f0 0x7fa0e415f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0e3400640 1 -- 172.21.15.19:0/4194210852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e41f7d30 msgr2=0x7fa0e41f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0e3400640 1 --2- 172.21.15.19:0/4194210852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e41f7d30 0x7fa0e41f81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.842+0000 7fa0e3400640 1 -- 172.21.15.19:0/4194210852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e415fed0 con 0x7fa0c80039b0 2024-03-28T13:22:01.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.843+0000 7fa0e3400640 1 --2- 172.21.15.19:0/4194210852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c80039b0 0x7fa0e4156300 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400e230 tx=0x7fa0d4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.843+0000 7fa0db400640 1 -- 172.21.15.19:0/4194210852 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4039050 con 0x7fa0c80039b0 2024-03-28T13:22:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.843+0000 7fa0db400640 1 -- 172.21.15.19:0/4194210852 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa0d403a040 con 0x7fa0c80039b0 2024-03-28T13:22:01.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.843+0000 7fa0db400640 1 -- 172.21.15.19:0/4194210852 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4009c10 con 0x7fa0c80039b0 2024-03-28T13:22:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.843+0000 7fa0ea400640 1 -- 172.21.15.19:0/4194210852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e41601b0 con 0x7fa0c80039b0 2024-03-28T13:22:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.843+0000 7fa0ea400640 1 -- 172.21.15.19:0/4194210852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e4160770 con 0x7fa0c80039b0 2024-03-28T13:22:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.845+0000 7fa0ea400640 1 -- 172.21.15.19:0/4194210852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0e40685c0 con 0x7fa0c80039b0 2024-03-28T13:22:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.846+0000 7fa0db400640 1 -- 172.21.15.19:0/4194210852 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 90856+0+0 (secure 0 0 0) 0x7fa0d4047030 con 0x7fa0c80039b0 2024-03-28T13:22:01.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.847+0000 7fa0db400640 1 --2- 172.21.15.19:0/4194210852 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c5b0 0x7fa0c006ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-28T13:22:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.847+0000 7fa0db400640 1 -- 172.21.15.19:0/4194210852 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5115+0+0 (secure 0 0 0) 0x7fa0d40ad800 con 0x7fa0c80039b0 2024-03-28T13:22:01.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.848+0000 7fa0e2a00640 1 --2- 172.21.15.19:0/4194210852 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c5b0 0x7fa0c006ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-28T13:22:01.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.849+0000 7fa0e2a00640 1 --2- 172.21.15.19:0/4194210852 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c5b0 0x7fa0c006ea70 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa0e415d070 tx=0x7fa0cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-28T13:22:01.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:01.852+0000 7fa0db400640 1 -- 172.21.15.19:0/4194210852 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186564 (secure 0 0 0) 0x7fa0d404e060 con 0x7fa0c80039b0 2024-03-28T13:22:02.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:02.097+0000 7fa0ea400640 1 -- 172.21.15.19:0/4194210852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0e406f3a0 con 0x7fa0c80039b0 2024-03-28T13:22:02.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:02.098+0000 7fa0db400640 1 -- 172.21.15.19:0/4194210852 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+3869 (secure 0 0 0) 0x7fa0d407c0c0 con 0x7fa0c80039b0 2024-03-28T13:22:02.097 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-03-28T13:22:02.097 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":114,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6837/3823671990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6835","nonce":3823671990},{"type":"v1","addr":"172.21.15.188:6837","nonce":3823671990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7123,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6836/22158043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":22158043},{"type":"v1","addr":"172.21.15.188:6836","nonce":22158043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":101},{"gid":8688,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6834/230519675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":230519675},{"type":"v1","addr":"172.21.15.19:6834","nonce":230519675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":9342,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1309170081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6833","nonce":1309170081},{"type":"v1","addr":"172.21.15.19:6835","nonce":1309170081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114}],"filesystems":[]} 2024-03-28T13:22:02.098 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 114 2024-03-28T13:22:02.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:02.101+0000 7fa0ea400640 1 -- 172.21.15.19:0/4194210852 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c5b0 msgr2=0x7fa0c006ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:02.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:02.101+0000 7fa0ea400640 1 --2- 172.21.15.19:0/4194210852 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c5b0 0x7fa0c006ea70 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa0e415d070 tx=0x7fa0cc008040 comp rx=0 tx=0).stop 2024-03-28T13:22:02.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:02.101+0000 7fa0ea400640 1 -- 172.21.15.19:0/4194210852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c80039b0 msgr2=0x7fa0e4156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-28T13:22:02.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:02.101+0000 7fa0ea400640 1 --2- 172.21.15.19:0/4194210852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c80039b0 0x7fa0e4156300 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400e230 tx=0x7fa0d4009750 comp rx=0 tx=0).stop 2024-03-28T13:22:02.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:02.101+0000 7fa0ea400640 1 -- 172.21.15.19:0/4194210852 shutdown_connections 2024-03-28T13:22:02.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:02.101+0000 7fa0ea400640 1 --2- 172.21.15.19:0/4194210852 >> [v2:172.21.15.188:6832/3382915391,v1:172.21.15.188:6833/3382915391] conn(0x7fa0c006c5b0 0x7fa0c006ea70 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:02.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:02.101+0000 7fa0ea400640 1 --2- 172.21.15.19:0/4194210852 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0e41f7d30 0x7fa0e41f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:02.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:02.101+0000 7fa0ea400640 1 --2- 172.21.15.19:0/4194210852 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0e415f3f0 0x7fa0e415f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:02.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:02.101+0000 7fa0ea400640 1 --2- 172.21.15.19:0/4194210852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0c80039b0 0x7fa0e4156300 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-28T13:22:02.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:02.101+0000 7fa0ea400640 1 -- 172.21.15.19:0/4194210852 >> 172.21.15.19:0/4194210852 conn(0x7fa0e406b990 msgr2=0x7fa0e41530c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-28T13:22:02.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:02.101+0000 7fa0ea400640 1 -- 172.21.15.19:0/4194210852 shutdown_connections 2024-03-28T13:22:02.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-03-28T13:22:02.101+0000 7fa0ea400640 1 -- 172.21.15.19:0/4194210852 wait complete. 2024-03-28T13:22:02.119 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-28T13:22:02.119 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-28T13:22:02.120 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-28T13:22:02.120 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:02.135 INFO:tasks.ceph.mds.a:Stopped 2024-03-28T13:22:02.136 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-28T13:22:02.136 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:02.210 INFO:tasks.ceph.mds.c:Stopped 2024-03-28T13:22:02.210 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-28T13:22:02.210 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:02.231 INFO:tasks.ceph.mds.b:Stopped 2024-03-28T13:22:02.232 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-28T13:22:02.232 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:02.304 INFO:tasks.ceph.mds.d:Stopped 2024-03-28T13:22:02.305 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-28T13:22:02.305 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-28T13:22:02.305 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:02.366 INFO:tasks.ceph.osd.0:Stopped 2024-03-28T13:22:02.366 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-28T13:22:02.366 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:02.443 INFO:tasks.ceph.osd.1:Stopped 2024-03-28T13:22:02.443 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-28T13:22:02.443 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:02.494 INFO:tasks.ceph.osd.2:Stopped 2024-03-28T13:22:02.494 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-28T13:22:02.494 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:02.569 INFO:tasks.ceph.osd.3:Stopped 2024-03-28T13:22:02.569 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-28T13:22:02.570 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:02.642 INFO:tasks.ceph.osd.4:Stopped 2024-03-28T13:22:02.642 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-28T13:22:02.642 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:02.697 INFO:tasks.ceph.osd.5:Stopped 2024-03-28T13:22:02.697 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-28T13:22:02.697 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:02.780 INFO:tasks.ceph.osd.6:Stopped 2024-03-28T13:22:02.781 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-28T13:22:02.781 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:02.848 INFO:tasks.ceph.osd.7:Stopped 2024-03-28T13:22:02.848 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-28T13:22:02.848 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-28T13:22:02.848 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:02.943 INFO:tasks.ceph.mgr.y:Stopped 2024-03-28T13:22:02.943 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-28T13:22:02.944 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:03.016 INFO:tasks.ceph.mgr.x:Stopped 2024-03-28T13:22:03.017 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-28T13:22:03.017 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-28T13:22:03.017 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:03.069 INFO:tasks.ceph.mon.a:Stopped 2024-03-28T13:22:03.070 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-28T13:22:03.070 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:03.111 INFO:tasks.ceph.mon.b:Stopped 2024-03-28T13:22:03.112 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-28T13:22:03.112 INFO:teuthology.orchestra.run:waiting for 300 2024-03-28T13:22:03.147 INFO:tasks.ceph.mon.c:Stopped 2024-03-28T13:22:03.147 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-28T13:22:03.147 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-03-28T13:22:03.275 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:14:52.820416+0000 mds.c (mds.0) 1 : cluster [WRN] Corrupt dentry 'file_to_be_damaged' in dir frag 0x10000000000: Invalid tag char 'd' pos 0: Malformed input [buffer:3](/subdir) 2024-03-28T13:22:03.276 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-03-28T13:22:03.276 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-03-28T13:22:03.373 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-03-28T13:22:03.468 INFO:teuthology.orchestra.run.smithi019.stdout:2024-03-28T13:16:47.382269+0000 mds.b (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-03-28T13:22:03.469 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:22:03.469 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-28T13:22:03.823 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:22:03.823 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-28T13:22:03.988 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:22:03.988 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-28T13:22:04.145 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:22:04.145 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-28T13:22:04.313 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi188.front.sepia.ceph.com 2024-03-28T13:22:04.313 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-28T13:22:04.627 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi188.front.sepia.ceph.com 2024-03-28T13:22:04.628 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-28T13:22:04.768 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:22:04.776 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:22:04.786 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi188.front.sepia.ceph.com 2024-03-28T13:22:04.786 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-28T13:22:05.024 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi188.front.sepia.ceph.com 2024-03-28T13:22:05.249 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-28T13:22:05.450 INFO:tasks.ceph:Archiving mon data... 2024-03-28T13:22:05.452 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849/data/mon.a.tgz 2024-03-28T13:22:05.452 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-03-28T13:22:05.506 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.192eX709nU 2024-03-28T13:22:05.507 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.192eX709nU 2024-03-28T13:22:05.712 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.192eX709nU 2024-03-28T13:22:05.861 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.192eX709nU is 450KB 2024-03-28T13:22:05.972 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.192eX709nU 2024-03-28T13:22:06.030 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849/data/mon.b.tgz 2024-03-28T13:22:06.030 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-03-28T13:22:06.088 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.FZV8UG4yOy 2024-03-28T13:22:06.088 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.FZV8UG4yOy 2024-03-28T13:22:06.302 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.FZV8UG4yOy 2024-03-28T13:22:06.452 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.FZV8UG4yOy is 467KB 2024-03-28T13:22:06.561 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.FZV8UG4yOy 2024-03-28T13:22:06.618 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849/data/mon.c.tgz 2024-03-28T13:22:06.618 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-03-28T13:22:06.671 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.cKaZhBddBA 2024-03-28T13:22:06.671 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.cKaZhBddBA 2024-03-28T13:22:06.869 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.cKaZhBddBA 2024-03-28T13:22:07.016 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.cKaZhBddBA is 467KB 2024-03-28T13:22:07.124 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.cKaZhBddBA 2024-03-28T13:22:07.180 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-28T13:22:07.180 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-28T13:22:07.188 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-28T13:22:07.337 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-28T13:22:07.393 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-28T13:22:07.393 INFO:tasks.ceph:Archiving crash dumps... 2024-03-28T13:22:07.394 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849/remote/smithi019/crash 2024-03-28T13:22:07.395 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-28T13:22:07.497 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849/remote/smithi188/crash 2024-03-28T13:22:07.498 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-28T13:22:07.594 INFO:tasks.ceph:Shutting down logrotate 2024-03-28T13:22:07.594 DEBUG:teuthology.orchestra.run.smithi019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:22:07.602 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-28T13:22:07.740 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-28T13:22:07.841 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-28T13:22:07.941 INFO:tasks.ceph:Compressing logs... 2024-03-28T13:22:07.941 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-28T13:22:07.949 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-28T13:22:08.037 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-28T13:22:08.038 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37196.log 2024-03-28T13:22:08.039 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-28T13:22:08.040 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60563.log 2024-03-28T13:22:08.040 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.37196.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-28T13:22:08.041 INFO:teuthology.orchestra.run.smithi188.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37196.log.gz 2024-03-28T13:22:08.042 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-28T13:22:08.042 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66041.log 2024-03-28T13:22:08.042 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63741.log 2024-03-28T13:22:08.043 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42516.log 2024-03-28T13:22:08.043 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.60563.log.gz 2024-03-28T13:22:08.044 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73635.log 2024-03-28T13:22:08.044 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66041.log.gz 2024-03-28T13:22:08.045 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76801.log 2024-03-28T13:22:08.045 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63741.log.gz 2024-03-28T13:22:08.045 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42535.log 2024-03-28T13:22:08.047 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.42516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42516.log.gz 2024-03-28T13:22:08.047 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37156.log 2024-03-28T13:22:08.048 INFO:teuthology.orchestra.run.smithi188.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-28T13:22:08.048 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5/var/log/ceph/ceph-client.1.42535.log: --verbose -- /var/log/ceph/ceph-client.admin.37076.log 2024-03-28T13:22:08.048 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42535.log.gz 2024-03-28T13:22:08.048 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73635.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73635.log.gz 2024-03-28T13:22:08.048 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46114.log 2024-03-28T13:22:08.048 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51857.log 2024-03-28T13:22:08.048 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.76801.log.gz 2024-03-28T13:22:08.048 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47347.log 2024-03-28T13:22:08.048 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46114.log.gz 2024-03-28T13:22:08.052 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.37156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37036.log 2024-03-28T13:22:08.052 INFO:teuthology.orchestra.run.smithi188.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37156.log.gz 2024-03-28T13:22:08.052 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.37076.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37076.log.gz 2024-03-28T13:22:08.052 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-28T13:22:08.052 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.37036.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-28T13:22:08.052 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.7.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37036.log.gz 2024-03-28T13:22:08.052 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37276.log 2024-03-28T13:22:08.054 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51857.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51857.log.gz 2024-03-28T13:22:08.054 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45988.log 2024-03-28T13:22:08.054 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78471.log 2024-03-28T13:22:08.054 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47347.log.gz 2024-03-28T13:22:08.054 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71585.log 2024-03-28T13:22:08.054 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45988.log.gz 2024-03-28T13:22:08.054 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49038.log 2024-03-28T13:22:08.054 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78471.log.gz 2024-03-28T13:22:08.054 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69413.log 2024-03-28T13:22:08.054 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.71585.log.gz 2024-03-28T13:22:08.054 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49038.log: gzip -5 --verbose -- 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49038.log.gz /var/log/ceph/ceph-client.admin.67884.log 2024-03-28T13:22:08.055 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-03-28T13:22:08.057 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42930.log 2024-03-28T13:22:08.057 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.37276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37316.log 2024-03-28T13:22:08.057 INFO:teuthology.orchestra.run.smithi188.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37276.log.gz 2024-03-28T13:22:08.057 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.3.42930.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42930.log.gz 2024-03-28T13:22:08.058 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50784.log 2024-03-28T13:22:08.059 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.69413.log.gz 2024-03-28T13:22:08.059 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73971.log 2024-03-28T13:22:08.059 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.67884.log.gz 2024-03-28T13:22:08.059 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50784.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50784.log.gz 2024-03-28T13:22:08.059 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44219.log 2024-03-28T13:22:08.059 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47677.log 2024-03-28T13:22:08.059 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73971.log.gz 2024-03-28T13:22:08.059 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64796.log 2024-03-28T13:22:08.059 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44219.log.gz 2024-03-28T13:22:08.062 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.37316.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37316.log.gz 2024-03-28T13:22:08.062 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-28T13:22:08.062 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37116.log 2024-03-28T13:22:08.062 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37236.log 2024-03-28T13:22:08.063 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70599.log 89.4% 2024-03-28T13:22:08.063 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47677.log.gz 2024-03-28T13:22:08.063 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64796.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-28T13:22:08.063 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64796.log.gz 2024-03-28T13:22:08.063 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.70599.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70599.log.gz 2024-03-28T13:22:08.063 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64946.log 2024-03-28T13:22:08.064 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62640.log 2024-03-28T13:22:08.065 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.37116.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37116.log.gz 2024-03-28T13:22:08.065 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.37236.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37236.log.gz 2024-03-28T13:22:08.065 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-28T13:22:08.066 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42962.log 2024-03-28T13:22:08.066 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.64946.log.gz 2024-03-28T13:22:08.066 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64712.log 2024-03-28T13:22:08.066 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62640.log.gz 2024-03-28T13:22:08.066 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63657.log 2024-03-28T13:22:08.067 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42962.log.gz 2024-03-28T13:22:08.067 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64712.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64712.log.gz 2024-03-28T13:22:08.068 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50826.log 2024-03-28T13:22:08.069 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42934.log 2024-03-28T13:22:08.069 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.3.42934.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-28T13:22:08.072 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77214.log 2024-03-28T13:22:08.072 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50826.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63657.log.gz 2024-03-28T13:22:08.072 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43254.log 2024-03-28T13:22:08.072 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50826.log.gz 2024-03-28T13:22:08.072 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70332.log 2024-03-28T13:22:08.072 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77214.log.gz 2024-03-28T13:22:08.072 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55958.log 2024-03-28T13:22:08.072 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43254.log.gz 2024-03-28T13:22:08.072 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68112.log 2024-03-28T13:22:08.074 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mon.b.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.42934.log.gz 2024-03-28T13:22:08.074 INFO:teuthology.orchestra.run.smithi188.stderr: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-28T13:22:08.074 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-03-28T13:22:08.075 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.70332.log.gz 2024-03-28T13:22:08.075 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51815.log 2024-03-28T13:22:08.075 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55958.log.gz 2024-03-28T13:22:08.075 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53999.log 2024-03-28T13:22:08.075 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.68112.log.gz 2024-03-28T13:22:08.075 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37420.log 2024-03-28T13:22:08.076 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-28T13:22:08.076 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51815.log.gz 2024-03-28T13:22:08.077 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56693.log 2024-03-28T13:22:08.077 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53999.log.gz 2024-03-28T13:22:08.077 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43925.log 2024-03-28T13:22:08.078 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37420.log.gz 2024-03-28T13:22:08.078 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58190.log 2024-03-28T13:22:08.079 INFO:teuthology.orchestra.run.smithi019.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.56693.log.gz 2024-03-28T13:22:08.079 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59888.log 2024-03-28T13:22:08.080 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43925.log.gz 2024-03-28T13:22:08.080 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46408.log 2024-03-28T13:22:08.081 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58190.log.gz 2024-03-28T13:22:08.081 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52720.log 2024-03-28T13:22:08.082 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59888.log.gz 2024-03-28T13:22:08.083 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46408.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46408.log.gz 2024-03-28T13:22:08.083 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49584.log 2024-03-28T13:22:08.084 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.52720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49392.log 2024-03-28T13:22:08.084 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52720.log.gz 2024-03-28T13:22:08.084 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75714.log 2024-03-28T13:22:08.084 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49584.log.gz 2024-03-28T13:22:08.085 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44093.log 2024-03-28T13:22:08.086 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49392.log.gz 2024-03-28T13:22:08.086 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75714.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75714.log.gz 2024-03-28T13:22:08.087 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65138.log 2024-03-28T13:22:08.088 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57116.log 2024-03-28T13:22:08.089 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.65138.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44093.log.gz 2024-03-28T13:22:08.089 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65138.log.gz 2024-03-28T13:22:08.089 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51771.log 2024-03-28T13:22:08.090 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53509.log 2024-03-28T13:22:08.090 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57116.log.gz 2024-03-28T13:22:08.091 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51348.log 2024-03-28T13:22:08.091 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51771.log.gz 2024-03-28T13:22:08.092 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48055.log 2024-03-28T13:22:08.092 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53509.log.gz 2024-03-28T13:22:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62116.log 2024-03-28T13:22:08.094 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51348.log.gz 2024-03-28T13:22:08.094 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62724.log 2024-03-28T13:22:08.095 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48055.log.gz 2024-03-28T13:22:08.095 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43598.log 2024-03-28T13:22:08.095 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.62116.log.gz 2024-03-28T13:22:08.096 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70270.log 2024-03-28T13:22:08.096 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62724.log.gz 2024-03-28T13:22:08.097 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43598.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43598.log.gz 2024-03-28T13:22:08.097 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53425.log 2024-03-28T13:22:08.098 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57494.log 2024-03-28T13:22:08.099 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70270.log.gz 2024-03-28T13:22:08.099 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72659.log 2024-03-28T13:22:08.099 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53425.log.gz 2024-03-28T13:22:08.100 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53915.log 2024-03-28T13:22:08.100 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57494.log.gz 2024-03-28T13:22:08.101 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58867.log 2024-03-28T13:22:08.101 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72659.log.gz 2024-03-28T13:22:08.102 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42749.log 2024-03-28T13:22:08.102 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53915.log.gz 2024-03-28T13:22:08.103 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56938.log 2024-03-28T13:22:08.103 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58867.log.gz 2024-03-28T13:22:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44303.log 2024-03-28T13:22:08.105 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42749.log.gz 2024-03-28T13:22:08.105 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47221.log 2024-03-28T13:22:08.105 INFO:teuthology.orchestra.run.smithi019.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56938.log.gz 2024-03-28T13:22:08.106 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68010.log 2024-03-28T13:22:08.106 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44303.log.gz 2024-03-28T13:22:08.107 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72304.log 2024-03-28T13:22:08.107 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47221.log.gz 2024-03-28T13:22:08.108 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61159.log 2024-03-28T13:22:08.108 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68010.log.gz 2024-03-28T13:22:08.109 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54354.log 2024-03-28T13:22:08.109 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72304.log.gz 2024-03-28T13:22:08.110 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72149.log 2024-03-28T13:22:08.111 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54354.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61159.log.gz 2024-03-28T13:22:08.111 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64543.log 2024-03-28T13:22:08.111 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54354.log.gz 2024-03-28T13:22:08.112 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48996.log 2024-03-28T13:22:08.112 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72149.log.gz 2024-03-28T13:22:08.113 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64543.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64543.log.gz 2024-03-28T13:22:08.113 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64585.log 2024-03-28T13:22:08.114 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54617.log 2024-03-28T13:22:08.115 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48996.log.gz 2024-03-28T13:22:08.115 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54905.log 2024-03-28T13:22:08.116 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64585.log.gz 2024-03-28T13:22:08.116 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62808.log 2024-03-28T13:22:08.117 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54617.log.gz 2024-03-28T13:22:08.117 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64101.log 2024-03-28T13:22:08.118 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54905.log.gz 2024-03-28T13:22:08.118 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72575.log 2024-03-28T13:22:08.119 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62808.log.gz 2024-03-28T13:22:08.119 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.64101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45568.log 2024-03-28T13:22:08.119 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64101.log.gz 2024-03-28T13:22:08.120 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63107.log 2024-03-28T13:22:08.120 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.72575.log.gz 2024-03-28T13:22:08.121 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48013.log 2024-03-28T13:22:08.122 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45568.log.gz 2024-03-28T13:22:08.122 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55764.log 2024-03-28T13:22:08.123 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63107.log.gz 2024-03-28T13:22:08.124 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67926.log 2024-03-28T13:22:08.124 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48013.log.gz 2024-03-28T13:22:08.124 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.55764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55764.log.gz 2024-03-28T13:22:08.125 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44387.log 2024-03-28T13:22:08.125 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47088.log 2024-03-28T13:22:08.126 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.67926.log.gz 2024-03-28T13:22:08.127 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60479.log 2024-03-28T13:22:08.127 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44387.log.gz 2024-03-28T13:22:08.131 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63275.log 2024-03-28T13:22:08.132 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47088.log.gz 2024-03-28T13:22:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68323.log 2024-03-28T13:22:08.132 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60479.log.gz 2024-03-28T13:22:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70108.log 2024-03-28T13:22:08.132 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63275.log.gz 2024-03-28T13:22:08.132 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37053.log 2024-03-28T13:22:08.132 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68323.log.gz 2024-03-28T13:22:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79394.log 2024-03-28T13:22:08.133 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70108.log.gz 2024-03-28T13:22:08.133 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50868.log 2024-03-28T13:22:08.134 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37053.log.gz 2024-03-28T13:22:08.134 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65817.log 2024-03-28T13:22:08.134 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79394.log.gz 2024-03-28T13:22:08.135 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65999.log 2024-03-28T13:22:08.135 INFO:teuthology.orchestra.run.smithi019.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50868.log.gz 2024-03-28T13:22:08.136 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48361.log 2024-03-28T13:22:08.136 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65817.log.gz 2024-03-28T13:22:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65999.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65999.log.gz 2024-03-28T13:22:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46282.log 2024-03-28T13:22:08.138 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58909.log 2024-03-28T13:22:08.139 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48361.log.gz 2024-03-28T13:22:08.139 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51264.log 2024-03-28T13:22:08.140 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46282.log.gz 2024-03-28T13:22:08.141 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55128.log 2024-03-28T13:22:08.141 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58909.log.gz 2024-03-28T13:22:08.142 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71460.log 2024-03-28T13:22:08.142 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51264.log.gz 2024-03-28T13:22:08.143 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79100.log 2024-03-28T13:22:08.143 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55128.log.gz 2024-03-28T13:22:08.143 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47263.log 2024-03-28T13:22:08.144 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71460.log.gz 2024-03-28T13:22:08.145 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58358.log 2024-03-28T13:22:08.145 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79100.log.gz 2024-03-28T13:22:08.146 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65558.log 2024-03-28T13:22:08.146 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47263.log.gz 2024-03-28T13:22:08.147 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60641.log 2024-03-28T13:22:08.147 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58358.log.gz 2024-03-28T13:22:08.148 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67842.log 2024-03-28T13:22:08.148 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65558.log.gz 2024-03-28T13:22:08.149 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66854.log 2024-03-28T13:22:08.150 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67842.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.60641.log.gz 2024-03-28T13:22:08.150 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45127.log 2024-03-28T13:22:08.150 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67842.log.gz 2024-03-28T13:22:08.150 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.66854.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66854.log.gz 2024-03-28T13:22:08.151 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73463.log 2024-03-28T13:22:08.152 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62410.log 2024-03-28T13:22:08.152 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45127.log.gz 2024-03-28T13:22:08.153 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64097.log 2024-03-28T13:22:08.153 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73463.log.gz 2024-03-28T13:22:08.154 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73719.log 2024-03-28T13:22:08.154 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62410.log.gz 2024-03-28T13:22:08.155 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.64097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.64097.log.gz 2024-03-28T13:22:08.155 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70603.log 2024-03-28T13:22:08.156 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47845.log 2024-03-28T13:22:08.156 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73719.log.gz 2024-03-28T13:22:08.157 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.70603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75123.log 2024-03-28T13:22:08.157 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70603.log.gz 2024-03-28T13:22:08.158 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.c.log: 92.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-03-28T13:22:08.159 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75426.log 2024-03-28T13:22:08.159 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47845.log.gz 2024-03-28T13:22:08.159 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.75123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75123.log.gz 2024-03-28T13:22:08.159 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59934.log 2024-03-28T13:22:08.160 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72785.log 2024-03-28T13:22:08.161 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59934.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75426.log.gz 2024-03-28T13:22:08.161 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52080.log 2024-03-28T13:22:08.161 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.59934.log.gz 2024-03-28T13:22:08.162 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43603.log 2024-03-28T13:22:08.163 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72785.log.gz 2024-03-28T13:22:08.163 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55086.log 2024-03-28T13:22:08.164 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52080.log.gz 2024-03-28T13:22:08.165 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64838.log 2024-03-28T13:22:08.165 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43603.log.gz 2024-03-28T13:22:08.166 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54312.log 2024-03-28T13:22:08.166 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55086.log.gz 2024-03-28T13:22:08.167 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67758.log 2024-03-28T13:22:08.167 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64838.log.gz 2024-03-28T13:22:08.168 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79939.log 2024-03-28T13:22:08.168 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54312.log.gz 2024-03-28T13:22:08.169 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79436.log 2024-03-28T13:22:08.169 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67758.log.gz 2024-03-28T13:22:08.170 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69227.log 2024-03-28T13:22:08.170 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79939.log.gz 2024-03-28T13:22:08.171 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50090.log 2024-03-28T13:22:08.171 INFO:teuthology.orchestra.run.smithi019.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79436.log.gz 2024-03-28T13:22:08.173 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59846.log 2024-03-28T13:22:08.173 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69227.log.gz 2024-03-28T13:22:08.174 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78219.log 2024-03-28T13:22:08.174 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50090.log.gz 2024-03-28T13:22:08.175 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59846.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-client.admin.50742.log -- replaced with /var/log/ceph/ceph-client.admin.59846.log.gz 2024-03-28T13:22:08.175 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-03-28T13:22:08.176 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78555.log 2024-03-28T13:22:08.176 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78219.log.gz 2024-03-28T13:22:08.177 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50742.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50742.log.gz 2024-03-28T13:22:08.178 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54228.log 2024-03-28T13:22:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43276.log 2024-03-28T13:22:08.179 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78555.log.gz 2024-03-28T13:22:08.179 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45610.log 2024-03-28T13:22:08.180 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43276.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54228.log.gz 2024-03-28T13:22:08.180 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43275.log 2024-03-28T13:22:08.181 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43276.log.gz 2024-03-28T13:22:08.182 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56980.log 2024-03-28T13:22:08.182 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45610.log.gz 2024-03-28T13:22:08.182 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61074.log 2024-03-28T13:22:08.183 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43275.log.gz 2024-03-28T13:22:08.184 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79898.log 2024-03-28T13:22:08.184 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56980.log.gz 2024-03-28T13:22:08.185 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69101.log 2024-03-28T13:22:08.186 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.61074.log.gz 2024-03-28T13:22:08.186 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57596.log 2024-03-28T13:22:08.187 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79898.log.gz 2024-03-28T13:22:08.187 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52462.log 2024-03-28T13:22:08.187 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.69101.log.gz 2024-03-28T13:22:08.188 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63531.log 2024-03-28T13:22:08.188 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57596.log.gz 2024-03-28T13:22:08.189 INFO:teuthology.orchestra.run.smithi188.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-28T13:22:08.189 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70863.log 2024-03-28T13:22:08.190 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63531.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52462.log.gz 2024-03-28T13:22:08.191 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72407.log 2024-03-28T13:22:08.191 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63531.log.gz 2024-03-28T13:22:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70863.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.70863.log.gz 2024-03-28T13:22:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63319.log 2024-03-28T13:22:08.192 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55768.log 2024-03-28T13:22:08.193 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72407.log.gz 2024-03-28T13:22:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78513.log 2024-03-28T13:22:08.194 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63319.log.gz 2024-03-28T13:22:08.194 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.55768.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55768.log.gz 2024-03-28T13:22:08.195 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69623.log 2024-03-28T13:22:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46534.log 2024-03-28T13:22:08.196 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78513.log.gz 2024-03-28T13:22:08.196 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42409.log 2024-03-28T13:22:08.197 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.69623.log.gz 2024-03-28T13:22:08.198 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61567.log 2024-03-28T13:22:08.198 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46534.log.gz 2024-03-28T13:22:08.198 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74745.log 2024-03-28T13:22:08.199 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42409.log.gz 2024-03-28T13:22:08.200 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.61567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63842.log 2024-03-28T13:22:08.200 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61567.log.gz 2024-03-28T13:22:08.201 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68746.log 2024-03-28T13:22:08.201 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74745.log.gz 2024-03-28T13:22:08.202 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49222.log 2024-03-28T13:22:08.203 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68746.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.63842.log.gz 2024-03-28T13:22:08.203 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48782.log 2024-03-28T13:22:08.203 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68746.log.gz 2024-03-28T13:22:08.204 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68962.log 2024-03-28T13:22:08.204 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49222.log.gz 2024-03-28T13:22:08.205 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51474.log 2024-03-28T13:22:08.205 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48782.log.gz 2024-03-28T13:22:08.206 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66850.log 2024-03-28T13:22:08.206 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68962.log.gz 2024-03-28T13:22:08.207 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51474.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51474.log.gz 2024-03-28T13:22:08.207 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58699.log 2024-03-28T13:22:08.208 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.66850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61563.log 2024-03-28T13:22:08.208 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66850.log.gz 2024-03-28T13:22:08.209 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74355.log 2024-03-28T13:22:08.209 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58699.log.gz 2024-03-28T13:22:08.210 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.61563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.61563.log.gz 2024-03-28T13:22:08.210 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47179.log 2024-03-28T13:22:08.211 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.74355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42970.log 2024-03-28T13:22:08.212 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74355.log.gz 2024-03-28T13:22:08.212 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75342.log 2024-03-28T13:22:08.212 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47179.log.gz 2024-03-28T13:22:08.213 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70024.log 2024-03-28T13:22:08.214 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42970.log.gz 2024-03-28T13:22:08.214 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74013.log 2024-03-28T13:22:08.215 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75342.log.gz 2024-03-28T13:22:08.216 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64381.log 2024-03-28T13:22:08.216 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.70024.log.gz 2024-03-28T13:22:08.217 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74013.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74013.log.gz 2024-03-28T13:22:08.217 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54396.log 2024-03-28T13:22:08.217 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59633.log 2024-03-28T13:22:08.218 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64381.log.gz 2024-03-28T13:22:08.219 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62850.log 2024-03-28T13:22:08.219 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54396.log.gz 2024-03-28T13:22:08.220 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77669.log 2024-03-28T13:22:08.220 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59633.log.gz 2024-03-28T13:22:08.221 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52294.log 2024-03-28T13:22:08.221 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62850.log.gz 2024-03-28T13:22:08.222 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.77669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57849.log 2024-03-28T13:22:08.222 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.77669.log.gz 2024-03-28T13:22:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52294.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52294.log.gz 2024-03-28T13:22:08.223 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64423.log 2024-03-28T13:22:08.224 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68281.log 2024-03-28T13:22:08.225 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.57849.log.gz 2024-03-28T13:22:08.225 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70150.log 2024-03-28T13:22:08.225 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64423.log.gz 2024-03-28T13:22:08.226 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68281.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68281.log.gz 2024-03-28T13:22:08.226 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72617.log 2024-03-28T13:22:08.227 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54438.log 2024-03-28T13:22:08.227 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70150.log.gz 2024-03-28T13:22:08.228 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46687.log 2024-03-28T13:22:08.228 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72617.log.gz 2024-03-28T13:22:08.229 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54438.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54438.log.gz 2024-03-28T13:22:08.229 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54125.log 2024-03-28T13:22:08.230 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72869.log 2024-03-28T13:22:08.231 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46687.log.gz 2024-03-28T13:22:08.231 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46156.log 2024-03-28T13:22:08.231 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54125.log.gz 2024-03-28T13:22:08.232 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59387.log 2024-03-28T13:22:08.233 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.72869.log.gz 2024-03-28T13:22:08.233 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54270.log 2024-03-28T13:22:08.234 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46156.log.gz 2024-03-28T13:22:08.234 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49476.log 2024-03-28T13:22:08.235 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59387.log.gz 2024-03-28T13:22:08.235 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54270.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54270.log.gz 2024-03-28T13:22:08.236 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59718.log 2024-03-28T13:22:08.237 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65957.log 2024-03-28T13:22:08.237 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59718.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49476.log.gz 2024-03-28T13:22:08.238 INFO:teuthology.orchestra.run.smithi019.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.59718.log.gz 2024-03-28T13:22:08.238 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68544.log 2024-03-28T13:22:08.239 INFO:teuthology.orchestra.run.smithi188.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-28T13:22:08.239 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65957.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65957.log.gz 2024-03-28T13:22:08.240 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52420.log 2024-03-28T13:22:08.240 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68918.log 2024-03-28T13:22:08.240 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68544.log.gz 2024-03-28T13:22:08.241 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52420.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.52420.log.gz 2024-03-28T13:22:08.241 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53107.log 2024-03-28T13:22:08.242 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71109.log 2024-03-28T13:22:08.242 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68918.log.gz 2024-03-28T13:22:08.243 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76569.log 2024-03-28T13:22:08.243 INFO:teuthology.orchestra.run.smithi019.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.53107.log.gz 2024-03-28T13:22:08.244 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43283.log 2024-03-28T13:22:08.245 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76569.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71109.log.gz 2024-03-28T13:22:08.246 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76569.log.gz 2024-03-28T13:22:08.246 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44345.log 2024-03-28T13:22:08.247 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43283.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43283.log.gz 2024-03-28T13:22:08.247 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48954.log 2024-03-28T13:22:08.248 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69311.log 2024-03-28T13:22:08.248 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44345.log.gz 2024-03-28T13:22:08.249 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48954.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48954.log.gz 2024-03-28T13:22:08.249 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58951.log 2024-03-28T13:22:08.250 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44994.log 2024-03-28T13:22:08.251 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58951.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69311.log.gz 2024-03-28T13:22:08.252 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71695.log 2024-03-28T13:22:08.252 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58951.log.gz 2024-03-28T13:22:08.252 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.44994.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44994.log.gz 2024-03-28T13:22:08.253 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57891.log 2024-03-28T13:22:08.254 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51432.log 2024-03-28T13:22:08.255 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57891.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.71695.log.gz 2024-03-28T13:22:08.255 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57891.log.gz 2024-03-28T13:22:08.255 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60900.log 2024-03-28T13:22:08.256 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51432.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51432.log.gz 2024-03-28T13:22:08.257 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56854.log 2024-03-28T13:22:08.257 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52849.log 2024-03-28T13:22:08.258 INFO:teuthology.orchestra.run.smithi019.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60900.log.gz 2024-03-28T13:22:08.259 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56854.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56854.log.gz 2024-03-28T13:22:08.259 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79562.log 2024-03-28T13:22:08.260 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48488.log 2024-03-28T13:22:08.260 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52849.log.gz 2024-03-28T13:22:08.260 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57242.log 2024-03-28T13:22:08.261 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.79562.log.gz 2024-03-28T13:22:08.262 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60437.log 2024-03-28T13:22:08.262 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48488.log.gz 2024-03-28T13:22:08.263 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43279.log 2024-03-28T13:22:08.263 INFO:teuthology.orchestra.run.smithi019.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57242.log.gz 2024-03-28T13:22:08.264 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50132.log 2024-03-28T13:22:08.264 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60437.log.gz 2024-03-28T13:22:08.265 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75468.log 2024-03-28T13:22:08.265 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43279.log.gz 2024-03-28T13:22:08.266 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67274.log 2024-03-28T13:22:08.266 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50132.log.gz 2024-03-28T13:22:08.270 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62242.log 2024-03-28T13:22:08.271 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75468.log.gz 2024-03-28T13:22:08.271 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67496.log 2024-03-28T13:22:08.271 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67274.log.gz 2024-03-28T13:22:08.271 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42973.log 2024-03-28T13:22:08.271 INFO:teuthology.orchestra.run.smithi019.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.62242.log.gz 2024-03-28T13:22:08.271 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56433.log 2024-03-28T13:22:08.271 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67496.log.gz 2024-03-28T13:22:08.272 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42973.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.42973.log.gz --verbose 2024-03-28T13:22:08.272 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.73803.log 2024-03-28T13:22:08.272 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74351.log 2024-03-28T13:22:08.273 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73803.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56433.log.gz 2024-03-28T13:22:08.273 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77999.log 2024-03-28T13:22:08.274 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73803.log.gz 2024-03-28T13:22:08.274 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.74351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74351.log.gz 2024-03-28T13:22:08.274 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59017.log 2024-03-28T13:22:08.276 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73677.log 2024-03-28T13:22:08.276 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59017.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77999.log.gz 2024-03-28T13:22:08.276 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44009.log 2024-03-28T13:22:08.277 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59017.log.gz 2024-03-28T13:22:08.278 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67968.log 2024-03-28T13:22:08.278 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73677.log.gz 2024-03-28T13:22:08.278 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59429.log 2024-03-28T13:22:08.279 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44009.log.gz 2024-03-28T13:22:08.280 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46030.log 2024-03-28T13:22:08.280 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67968.log.gz 2024-03-28T13:22:08.280 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59429.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.59429.log.gz 2024-03-28T13:22:08.281 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64465.log 2024-03-28T13:22:08.282 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57284.log 2024-03-28T13:22:08.282 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64465.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46030.log.gz 2024-03-28T13:22:08.283 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.64465.log.gz 2024-03-28T13:22:08.283 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54177.log 2024-03-28T13:22:08.284 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57284.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57284.log.gz 2024-03-28T13:22:08.284 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76485.log 2024-03-28T13:22:08.286 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47593.log 2024-03-28T13:22:08.286 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54177.log.gz 2024-03-28T13:22:08.287 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76485.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76485.log.gz 2024-03-28T13:22:08.287 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58402.log 2024-03-28T13:22:08.287 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53257.log 2024-03-28T13:22:08.288 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47593.log.gz 2024-03-28T13:22:08.288 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58402.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58402.log.gz 2024-03-28T13:22:08.288 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42497.log 2024-03-28T13:22:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78826.log 2024-03-28T13:22:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42497.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53257.log.gz 2024-03-28T13:22:08.291 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42497.log.gz 2024-03-28T13:22:08.291 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50401.log 2024-03-28T13:22:08.292 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50549.log 2024-03-28T13:22:08.292 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78826.log.gz 2024-03-28T13:22:08.294 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46891.log 2024-03-28T13:22:08.294 INFO:teuthology.orchestra.run.smithi019.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.50401.log.gz 2024-03-28T13:22:08.295 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50549.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50549.log.gz 2024-03-28T13:22:08.295 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58028.log 2024-03-28T13:22:08.296 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46771.log 2024-03-28T13:22:08.297 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58028.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46891.log.gz 2024-03-28T13:22:08.297 INFO:teuthology.orchestra.run.smithi019.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58028.log.gz 2024-03-28T13:22:08.297 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75855.log 2024-03-28T13:22:08.298 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46771.log: gzip -5 --verbose 88.4% -- -- replaced with /var/log/ceph/ceph-client.admin.46771.log.gz /var/log/ceph/ceph-client.admin.68239.log 2024-03-28T13:22:08.298 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-03-28T13:22:08.299 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73547.log 2024-03-28T13:22:08.299 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75855.log.gz 2024-03-28T13:22:08.300 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68239.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68239.log.gz 2024-03-28T13:22:08.300 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68365.log 2024-03-28T13:22:08.301 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58497.log 2024-03-28T13:22:08.302 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68365.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.73547.log.gz 2024-03-28T13:22:08.302 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68365.log.gz 2024-03-28T13:22:08.302 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66596.log 2024-03-28T13:22:08.304 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58497.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58497.log.gz 2024-03-28T13:22:08.304 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73505.log 2024-03-28T13:22:08.304 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67674.log 2024-03-28T13:22:08.305 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73505.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.66596.log.gz 2024-03-28T13:22:08.306 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73505.log.gz 2024-03-28T13:22:08.306 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54947.log 2024-03-28T13:22:08.307 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48656.log 2024-03-28T13:22:08.308 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.67674.log.gz 2024-03-28T13:22:08.308 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54947.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54947.log.gz 2024-03-28T13:22:08.308 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42539.log 2024-03-28T13:22:08.309 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48656.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48656.log.gz 2024-03-28T13:22:08.309 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78639.log 2024-03-28T13:22:08.310 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45431.log 2024-03-28T13:22:08.311 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42539.log.gz 2024-03-28T13:22:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78639.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78639.log.gz 2024-03-28T13:22:08.311 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56896.log 2024-03-28T13:22:08.312 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.2.45431.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45431.log.gz 2024-03-28T13:22:08.312 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68704.log 2024-03-28T13:22:08.313 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56896.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56896.log.gz 2024-03-28T13:22:08.313 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49266.log 2024-03-28T13:22:08.314 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71999.log 2024-03-28T13:22:08.314 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68704.log.gz 2024-03-28T13:22:08.315 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49266.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49266.log.gz 2024-03-28T13:22:08.315 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-28T13:22:08.316 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71222.log 2024-03-28T13:22:08.317 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph.audit.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.71999.log.gz 2024-03-28T13:22:08.318 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52892.log 2024-03-28T13:22:08.319 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46366.log 2024-03-28T13:22:08.319 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52892.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-28T13:22:08.320 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.71222.log.gz 2024-03-28T13:22:08.320 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65751.log 2024-03-28T13:22:08.320 INFO:teuthology.orchestra.run.smithi019.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52892.log.gz 2024-03-28T13:22:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46366.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46366.log.gz 2024-03-28T13:22:08.321 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57200.log 2024-03-28T13:22:08.322 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47719.log 2024-03-28T13:22:08.323 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57200.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.65751.log.gz 2024-03-28T13:22:08.324 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57200.log.gz 2024-03-28T13:22:08.324 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43335.log 2024-03-28T13:22:08.325 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56133.log 2024-03-28T13:22:08.326 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47719.log.gz 2024-03-28T13:22:08.326 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72533.log 2024-03-28T13:22:08.327 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43335.log.gz 2024-03-28T13:22:08.328 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78345.log 2024-03-28T13:22:08.328 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56133.log.gz 2024-03-28T13:22:08.328 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72365.log 2024-03-28T13:22:08.329 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72533.log.gz 2024-03-28T13:22:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78345.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78345.log.gz 2024-03-28T13:22:08.330 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46324.log 2024-03-28T13:22:08.332 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67454.log 2024-03-28T13:22:08.332 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72365.log.gz 2024-03-28T13:22:08.333 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64880.log 2024-03-28T13:22:08.333 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46324.log.gz 2024-03-28T13:22:08.333 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67454.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67454.log.gz 2024-03-28T13:22:08.334 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58741.log 2024-03-28T13:22:08.335 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53467.log 2024-03-28T13:22:08.337 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58741.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58741.log.gz 2024-03-28T13:22:08.337 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64880.log.gz 2024-03-28T13:22:08.337 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51306.log 2024-03-28T13:22:08.338 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62555.log 2024-03-28T13:22:08.338 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53467.log.gz 2024-03-28T13:22:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51306.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51306.log.gz 2024-03-28T13:22:08.339 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70411.log 2024-03-28T13:22:08.340 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77673.log 2024-03-28T13:22:08.341 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62555.log.gz 2024-03-28T13:22:08.342 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70411.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.70411.log.gz 2024-03-28T13:22:08.342 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61643.log 2024-03-28T13:22:08.342 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.77673.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.77673.log.gz 2024-03-28T13:22:08.343 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42017.log 2024-03-28T13:22:08.344 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52038.log 2024-03-28T13:22:08.344 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61643.log.gz 2024-03-28T13:22:08.345 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42017.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42017.log.gz 2024-03-28T13:22:08.345 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42367.log 2024-03-28T13:22:08.346 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56011.log 2024-03-28T13:22:08.347 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52038.log.gz 2024-03-28T13:22:08.348 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63025.log 2024-03-28T13:22:08.348 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42367.log.gz 2024-03-28T13:22:08.349 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56011.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.56011.log.gz 2024-03-28T13:22:08.349 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73591.log 2024-03-28T13:22:08.349 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63909.log 2024-03-28T13:22:08.350 INFO:teuthology.orchestra.run.smithi019.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63025.log.gz 2024-03-28T13:22:08.350 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73591.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73591.log.gz 2024-03-28T13:22:08.351 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79058.log 2024-03-28T13:22:08.353 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42623.log 2024-03-28T13:22:08.353 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63909.log.gz 2024-03-28T13:22:08.354 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66352.log 2024-03-28T13:22:08.354 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79058.log.gz 2024-03-28T13:22:08.355 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42623.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42623.log.gz 2024-03-28T13:22:08.355 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49122.log 2024-03-28T13:22:08.356 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46240.log 2024-03-28T13:22:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60792.log 2024-03-28T13:22:08.358 INFO:teuthology.orchestra.run.smithi019.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49122.log.gz 2024-03-28T13:22:08.358 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46240.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.66352.log.gz 2024-03-28T13:22:08.358 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46240.log.gz 2024-03-28T13:22:08.359 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57681.log 2024-03-28T13:22:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72172.log 2024-03-28T13:22:08.360 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57681.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.60792.log.gz 2024-03-28T13:22:08.361 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57681.log.gz 2024-03-28T13:22:08.361 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56391.log 2024-03-28T13:22:08.362 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69371.log 2024-03-28T13:22:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56391.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.72172.log.gz 2024-03-28T13:22:08.363 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56391.log.gz 2024-03-28T13:22:08.363 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31039.log 2024-03-28T13:22:08.364 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66310.log 2024-03-28T13:22:08.365 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69371.log.gz 2024-03-28T13:22:08.365 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph.tmp-client.admin.31039.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31039.log.gz 2024-03-28T13:22:08.365 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63615.log 2024-03-28T13:22:08.367 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54863.log 2024-03-28T13:22:08.368 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63615.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63615.log.gz 2024-03-28T13:22:08.368 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53831.log 2024-03-28T13:22:08.369 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54863.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.66310.log.gz 2024-03-28T13:22:08.370 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63573.log 2024-03-28T13:22:08.370 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54863.log.gz 2024-03-28T13:22:08.370 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53831.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53831.log.gz 2024-03-28T13:22:08.370 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74865.log 2024-03-28T13:22:08.372 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78868.log 2024-03-28T13:22:08.372 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63573.log.gz 2024-03-28T13:22:08.373 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75897.log 2024-03-28T13:22:08.373 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.74865.log.gz/var/log/ceph/ceph-client.admin.78868.log: 2024-03-28T13:22:08.374 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78868.log.gz 2024-03-28T13:22:08.374 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49350.log 2024-03-28T13:22:08.376 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51390.log 2024-03-28T13:22:08.376 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49350.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.75897.log.gz 2024-03-28T13:22:08.377 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41936.log 2024-03-28T13:22:08.377 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49350.log.gz 2024-03-28T13:22:08.378 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51390.log: gzip -5 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51390.log.gz --verbose 2024-03-28T13:22:08.378 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.68874.log 2024-03-28T13:22:08.380 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77397.log 2024-03-28T13:22:08.380 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41936.log.gz 2024-03-28T13:22:08.380 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61938.log 2024-03-28T13:22:08.381 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68874.log.gz 2024-03-28T13:22:08.382 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53019.log 2024-03-28T13:22:08.382 INFO:teuthology.orchestra.run.smithi019.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.77397.log.gz 2024-03-28T13:22:08.383 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78699.log 2024-03-28T13:22:08.383 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61938.log.gz 2024-03-28T13:22:08.384 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73761.log 2024-03-28T13:22:08.385 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78699.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53019.log.gz 2024-03-28T13:22:08.385 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78699.log.gz 2024-03-28T13:22:08.385 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50216.log 2024-03-28T13:22:08.387 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48912.log 2024-03-28T13:22:08.387 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73761.log.gz 2024-03-28T13:22:08.388 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65666.log 2024-03-28T13:22:08.388 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50216.log.gz 2024-03-28T13:22:08.389 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48912.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48912.log.gz 2024-03-28T13:22:08.389 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62682.log 2024-03-28T13:22:08.390 INFO:teuthology.orchestra.run.smithi188.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-28T13:22:08.390 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66245.log 2024-03-28T13:22:08.391 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62682.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.65666.log.gz 2024-03-28T13:22:08.392 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75658.log 2024-03-28T13:22:08.392 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62682.log.gz 2024-03-28T13:22:08.393 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66245.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66245.log.gz 2024-03-28T13:22:08.393 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75300.log 2024-03-28T13:22:08.394 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61812.log 2024-03-28T13:22:08.394 INFO:teuthology.orchestra.run.smithi019.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.75658.log.gz 2024-03-28T13:22:08.395 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75300.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.75300.log.gz 2024-03-28T13:22:08.395 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74163.log 2024-03-28T13:22:08.396 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51096.log 2024-03-28T13:22:08.397 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61812.log.gz 2024-03-28T13:22:08.398 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46729.log 2024-03-28T13:22:08.398 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74163.log.gz 2024-03-28T13:22:08.399 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51096.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51096.log.gz 2024-03-28T13:22:08.399 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76647.log 2024-03-28T13:22:08.399 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50658.log 2024-03-28T13:22:08.400 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46729.log.gz 2024-03-28T13:22:08.401 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76647.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76647.log.gz 2024-03-28T13:22:08.401 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78742.log 2024-03-28T13:22:08.402 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50658.log: gzip -5 --verbose -- 87.9% /var/log/ceph/ceph-client.admin.77309.log -- replaced with /var/log/ceph/ceph-client.admin.50658.log.gz 2024-03-28T13:22:08.403 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-03-28T13:22:08.403 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69013.log 2024-03-28T13:22:08.404 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78742.log.gz 2024-03-28T13:22:08.405 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77309.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77309.log.gz 2024-03-28T13:22:08.405 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79981.log 2024-03-28T13:22:08.406 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79226.log 2024-03-28T13:22:08.407 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69013.log.gz 2024-03-28T13:22:08.407 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68052.log 2024-03-28T13:22:08.407 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79981.log.gz 2024-03-28T13:22:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79226.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79226.log.gz 2024-03-28T13:22:08.408 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51987.log 2024-03-28T13:22:08.409 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72701.log 2024-03-28T13:22:08.410 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68052.log.gz 2024-03-28T13:22:08.411 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51987.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51987.log.gz 2024-03-28T13:22:08.411 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72491.log 2024-03-28T13:22:08.412 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42228.log 2024-03-28T13:22:08.412 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.72701.log.gz 2024-03-28T13:22:08.413 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72491.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.72491.log.gz 2024-03-28T13:22:08.413 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41840.log 2024-03-28T13:22:08.414 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42228.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.42228.log.gz --verbose 2024-03-28T13:22:08.414 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.55216.log 2024-03-28T13:22:08.415 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53191.log 2024-03-28T13:22:08.416 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41840.log.gz 2024-03-28T13:22:08.416 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55216.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55216.log.gz 2024-03-28T13:22:08.416 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53957.log 2024-03-28T13:22:08.418 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51180.log 2024-03-28T13:22:08.418 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53957.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.53191.log.gz 2024-03-28T13:22:08.419 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53957.log.gz 2024-03-28T13:22:08.419 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60521.log 2024-03-28T13:22:08.420 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47509.log 2024-03-28T13:22:08.420 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51180.log.gz 2024-03-28T13:22:08.421 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71310.log 2024-03-28T13:22:08.421 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.60521.log.gz 2024-03-28T13:22:08.422 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47509.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47509.log.gz 2024-03-28T13:22:08.422 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59513.log 2024-03-28T13:22:08.423 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44177.log 2024-03-28T13:22:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59513.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.71310.log.gz 2024-03-28T13:22:08.424 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72953.log 2024-03-28T13:22:08.425 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.59513.log.gz 2024-03-28T13:22:08.425 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44177.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44177.log.gz 2024-03-28T13:22:08.426 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76946.log 2024-03-28T13:22:08.427 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61375.log 2024-03-28T13:22:08.427 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.72953.log.gz 2024-03-28T13:22:08.428 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71641.log 2024-03-28T13:22:08.428 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61375.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.76946.log.gz 2024-03-28T13:22:08.429 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42102.log 2024-03-28T13:22:08.429 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61375.log.gz 2024-03-28T13:22:08.430 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50438.log 2024-03-28T13:22:08.431 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71641.log.gz 2024-03-28T13:22:08.431 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42961.log 2024-03-28T13:22:08.432 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42102.log.gz 2024-03-28T13:22:08.433 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61752.log 2024-03-28T13:22:08.433 INFO:teuthology.orchestra.run.smithi019.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50438.log.gz 2024-03-28T13:22:08.434 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51222.log 2024-03-28T13:22:08.434 INFO:teuthology.orchestra.run.smithi019.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42961.log.gz 2024-03-28T13:22:08.435 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64297.log 2024-03-28T13:22:08.435 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61752.log.gz 2024-03-28T13:22:08.436 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65384.log 2024-03-28T13:22:08.436 INFO:teuthology.orchestra.run.smithi019.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51222.log.gz 2024-03-28T13:22:08.437 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48826.log 2024-03-28T13:22:08.438 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64297.log.gz 2024-03-28T13:22:08.438 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77012.log 2024-03-28T13:22:08.439 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.65384.log.gz 2024-03-28T13:22:08.439 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79814.log 2024-03-28T13:22:08.440 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48826.log.gz 2024-03-28T13:22:08.441 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44610.log 2024-03-28T13:22:08.442 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79814.log: gzip 92.1% -- replaced with /var/log/ceph/ceph-client.admin.77012.log.gz -5 2024-03-28T13:22:08.442 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58108.log 2024-03-28T13:22:08.443 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79814.log.gz 2024-03-28T13:22:08.443 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76764.log 2024-03-28T13:22:08.444 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44610.log.gz 2024-03-28T13:22:08.445 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74931.log 2024-03-28T13:22:08.445 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58108.log.gz 2024-03-28T13:22:08.446 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62158.log 2024-03-28T13:22:08.446 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76764.log.gz 2024-03-28T13:22:08.447 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75579.log 2024-03-28T13:22:08.448 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74931.log.gz 2024-03-28T13:22:08.449 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62158.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-28T13:22:08.449 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62158.log.gz 2024-03-28T13:22:08.450 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71543.log 2024-03-28T13:22:08.450 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75579.log.gz 2024-03-28T13:22:08.451 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59555.log 2024-03-28T13:22:08.453 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79352.log 2024-03-28T13:22:08.453 INFO:teuthology.orchestra.run.smithi019.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.71543.log.gz 2024-03-28T13:22:08.454 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59792.log 2024-03-28T13:22:08.454 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59555.log.gz 2024-03-28T13:22:08.455 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72996.log 2024-03-28T13:22:08.456 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79352.log.gz 2024-03-28T13:22:08.457 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59792.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59792.log.gz 2024-03-28T13:22:08.457 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59205.log 2024-03-28T13:22:08.458 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72996.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.72996.log.gz 2024-03-28T13:22:08.459 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44471.log 2024-03-28T13:22:08.460 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.59205.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59205.log.gz 2024-03-28T13:22:08.460 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75119.log 2024-03-28T13:22:08.461 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79184.log 2024-03-28T13:22:08.461 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44471.log.gz 2024-03-28T13:22:08.462 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.75119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70947.log 2024-03-28T13:22:08.462 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75119.log.gz 2024-03-28T13:22:08.463 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79184.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79184.log.gz 2024-03-28T13:22:08.463 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52122.log 2024-03-28T13:22:08.464 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55510.log 2024-03-28T13:22:08.464 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.70947.log.gz 2024-03-28T13:22:08.465 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52122.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52122.log.gz 2024-03-28T13:22:08.465 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48572.log 2024-03-28T13:22:08.466 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66203.log 2024-03-28T13:22:08.467 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48572.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.55510.log.gz 2024-03-28T13:22:08.468 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48572.log.gz 2024-03-28T13:22:08.468 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78041.log 2024-03-28T13:22:08.469 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79688.log 2024-03-28T13:22:08.469 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66203.log.gz 2024-03-28T13:22:08.470 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69581.log 2024-03-28T13:22:08.471 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79688.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78041.log.gz 2024-03-28T13:22:08.471 INFO:teuthology.orchestra.run.smithi019.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79688.log.gz 2024-03-28T13:22:08.472 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73378.log 2024-03-28T13:22:08.473 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53593.log 2024-03-28T13:22:08.474 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73378.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69581.log.gz 2024-03-28T13:22:08.474 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73378.log.gz 2024-03-28T13:22:08.474 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52528.log 2024-03-28T13:22:08.476 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70989.log 2024-03-28T13:22:08.478 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52528.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53593.log.gz 2024-03-28T13:22:08.478 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52528.log.gz 2024-03-28T13:22:08.478 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57452.log 2024-03-28T13:22:08.480 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46072.log 2024-03-28T13:22:08.480 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57452.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.70989.log.gz 2024-03-28T13:22:08.480 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57452.log.gz 2024-03-28T13:22:08.481 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53789.log 2024-03-28T13:22:08.482 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46072.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46072.log.gz 2024-03-28T13:22:08.482 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55384.log 2024-03-28T13:22:08.483 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79772.log 2024-03-28T13:22:08.484 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53789.log.gz 2024-03-28T13:22:08.484 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55384.log: gzip -5 --verbose -- 89.0% /var/log/ceph/ceph-client.admin.64339.log -- replaced with /var/log/ceph/ceph-client.admin.55384.log.gz 2024-03-28T13:22:08.484 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-03-28T13:22:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79772.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79772.log.gz 2024-03-28T13:22:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59209.log 2024-03-28T13:22:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64339.log: gzip -5 --verbose 89.3% -- -- replaced with /var/log/ceph/ceph-client.admin.64339.log.gz 2024-03-28T13:22:08.487 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.42272.log 2024-03-28T13:22:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.59209.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59209.log.gz 2024-03-28T13:22:08.487 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51899.log 2024-03-28T13:22:08.490 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42272.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42272.log.gz 2024-03-28T13:22:08.490 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53551.log 2024-03-28T13:22:08.490 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51899.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51899.log.gz 2024-03-28T13:22:08.491 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49080.log 2024-03-28T13:22:08.491 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53551.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53551.log.gz 2024-03-28T13:22:08.491 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52716.log 2024-03-28T13:22:08.493 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49080.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49080.log.gz 2024-03-28T13:22:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62074.log 2024-03-28T13:22:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.52716.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52716.log.gz 2024-03-28T13:22:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50700.log 2024-03-28T13:22:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62074.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62074.log.gz 2024-03-28T13:22:08.494 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53300.log 2024-03-28T13:22:08.497 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50700.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50700.log.gz -5 2024-03-28T13:22:08.497 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.57723.log 2024-03-28T13:22:08.497 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69497.log 2024-03-28T13:22:08.497 INFO:teuthology.orchestra.run.smithi019.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.53300.log.gz 2024-03-28T13:22:08.498 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42967.log 2024-03-28T13:22:08.498 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57723.log.gz 2024-03-28T13:22:08.501 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69497.log: gzip -5 88.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.69497.log.gz 2024-03-28T13:22:08.501 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.65859.log 2024-03-28T13:22:08.501 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66932.log 89.0% 2024-03-28T13:22:08.502 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42967.log.gz 2024-03-28T13:22:08.502 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43596.log 2024-03-28T13:22:08.502 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.65859.log.gz 2024-03-28T13:22:08.502 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58274.log 2024-03-28T13:22:08.502 INFO:teuthology.orchestra.run.smithi019.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.66932.log.gz 2024-03-28T13:22:08.506 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43596.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43596.log.gz 2024-03-28T13:22:08.506 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.78303.log 2024-03-28T13:22:08.506 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45946.log 2024-03-28T13:22:08.507 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58274.log.gz 2024-03-28T13:22:08.507 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78303.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78303.log.gz 2024-03-28T13:22:08.507 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75767.log 2024-03-28T13:22:08.507 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71223.log 2024-03-28T13:22:08.507 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45946.log.gz 2024-03-28T13:22:08.507 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75767.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75767.log.gzgzip 2024-03-28T13:22:08.508 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.71827.log 2024-03-28T13:22:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54041.log 2024-03-28T13:22:08.510 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.71223.log.gz 2024-03-28T13:22:08.510 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76151.log 2024-03-28T13:22:08.510 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.71827.log.gz 2024-03-28T13:22:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54041.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54041.log.gz 2024-03-28T13:22:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51054.log 2024-03-28T13:22:08.511 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.76151.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76151.log.gz 2024-03-28T13:22:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47803.log 2024-03-28T13:22:08.512 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47467.log 2024-03-28T13:22:08.512 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51054.log.gz 2024-03-28T13:22:08.513 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47803.log: gzip -5 --verbose 88.6% -- -- replaced with /var/log/ceph/ceph-client.admin.47803.log.gz 2024-03-28T13:22:08.513 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.71784.log 2024-03-28T13:22:08.514 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74601.log 2024-03-28T13:22:08.514 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47467.log.gz 2024-03-28T13:22:08.515 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43754.log 2024-03-28T13:22:08.515 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.71784.log.gz 2024-03-28T13:22:08.516 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62512.log 2024-03-28T13:22:08.516 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74601.log.gz 2024-03-28T13:22:08.517 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43754.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43754.log.gz 2024-03-28T13:22:08.517 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66083.log 2024-03-28T13:22:08.518 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70066.log 2024-03-28T13:22:08.518 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62512.log.gz 2024-03-28T13:22:08.519 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66083.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66083.log.gz 2024-03-28T13:22:08.519 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47425.log 2024-03-28T13:22:08.520 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70066.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.70066.log.gz 2024-03-28T13:22:08.521 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54779.log 2024-03-28T13:22:08.522 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47425.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47425.log.gz 2024-03-28T13:22:08.522 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63489.log 2024-03-28T13:22:08.525 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60230.log 2024-03-28T13:22:08.525 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54779.log.gz 2024-03-28T13:22:08.525 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63489.log: gzip -5 89.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.63489.log.gz -- 2024-03-28T13:22:08.525 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.63233.log 2024-03-28T13:22:08.526 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.60230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60230.log.gz 2024-03-28T13:22:08.526 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78429.log 2024-03-28T13:22:08.528 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63233.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.63233.log.gz 2024-03-28T13:22:08.528 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67412.log 2024-03-28T13:22:08.529 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78429.log: gzip -5 88.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.78429.log.gz -- 2024-03-28T13:22:08.529 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.78597.log 2024-03-28T13:22:08.529 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67412.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67412.log.gz 2024-03-28T13:22:08.529 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57368.log 2024-03-28T13:22:08.532 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55258.log 2024-03-28T13:22:08.532 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78597.log.gz 2024-03-28T13:22:08.532 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57368.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57368.log.gz 2024-03-28T13:22:08.532 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75809.log 2024-03-28T13:22:08.532 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58783.log 2024-03-28T13:22:08.533 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55258.log.gz 2024-03-28T13:22:08.533 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75809.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.55468.log -- replaced with /var/log/ceph/ceph-client.admin.75809.log.gz 2024-03-28T13:22:08.533 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-03-28T13:22:08.534 INFO:teuthology.orchestra.run.smithi188.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-28T13:22:08.536 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57536.log 2024-03-28T13:22:08.536 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58783.log.gz 2024-03-28T13:22:08.536 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55468.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55468.log.gz 2024-03-28T13:22:08.537 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54991.log 2024-03-28T13:22:08.537 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55915.log 2024-03-28T13:22:08.537 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57536.log.gz 2024-03-28T13:22:08.537 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54991.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54991.log.gz 2024-03-28T13:22:08.537 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47929.log 2024-03-28T13:22:08.538 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51649.log 2024-03-28T13:22:08.539 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55915.log.gz 2024-03-28T13:22:08.539 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47929.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47929.log.gz 2024-03-28T13:22:08.539 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46198.log 2024-03-28T13:22:08.539 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51569.log 2024-03-28T13:22:08.540 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51649.log.gz 2024-03-28T13:22:08.540 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46198.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46198.log.gz 2024-03-28T13:22:08.541 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79604.log 2024-03-28T13:22:08.541 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55035.log 2024-03-28T13:22:08.542 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51569.log.gz 2024-03-28T13:22:08.543 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79604.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79604.log.gz 2024-03-28T13:22:08.543 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49776.log 2024-03-28T13:22:08.544 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55035.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-28T13:22:08.544 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55035.log.gz 2024-03-28T13:22:08.545 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.49776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49776.log.gz 2024-03-28T13:22:08.545 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52378.log 2024-03-28T13:22:08.546 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58825.log 2024-03-28T13:22:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52378.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.52378.log.gz -5 2024-03-28T13:22:08.548 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.69143.log 2024-03-28T13:22:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58825.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58825.log.gz 2024-03-28T13:22:08.549 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51729.log 2024-03-28T13:22:08.550 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61267.log 2024-03-28T13:22:08.551 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69143.log: /var/log/ceph/ceph-client.admin.51729.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69143.log.gz 2024-03-28T13:22:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.51729.log.gz 2024-03-28T13:22:08.552 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.66662.log 2024-03-28T13:22:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73122.log 2024-03-28T13:22:08.553 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.61267.log.gz 2024-03-28T13:22:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69185.log 2024-03-28T13:22:08.554 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66662.log.gz 2024-03-28T13:22:08.555 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72449.log 2024-03-28T13:22:08.556 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73122.log: /var/log/ceph/ceph-client.admin.69185.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73122.log.gz 2024-03-28T13:22:08.557 INFO:teuthology.orchestra.run.smithi019.stderr:gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.69185.log.gz --verbose 2024-03-28T13:22:08.557 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.0.69877.log 2024-03-28T13:22:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72449.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.72449.log.gz 2024-03-28T13:22:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48870.log 2024-03-28T13:22:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.69877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78784.log 2024-03-28T13:22:08.560 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69877.log.gz 2024-03-28T13:22:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53705.log 2024-03-28T13:22:08.561 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48870.log.gz 2024-03-28T13:22:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78784.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78784.log.gz 2024-03-28T13:22:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61225.log 2024-03-28T13:22:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44051.log 2024-03-28T13:22:08.564 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53705.log: /var/log/ceph/ceph-client.admin.61225.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53705.log.gz 2024-03-28T13:22:08.564 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61225.log.gz 2024-03-28T13:22:08.565 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50512.log 2024-03-28T13:22:08.566 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56217.log 2024-03-28T13:22:08.566 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44051.log.gz 2024-03-28T13:22:08.567 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50512.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.50512.log.gz 2024-03-28T13:22:08.567 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72911.log 2024-03-28T13:22:08.568 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45862.log 2024-03-28T13:22:08.570 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72911.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72911.log.gz 2024-03-28T13:22:08.570 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46492.log 2024-03-28T13:22:08.570 INFO:teuthology.orchestra.run.smithi019.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.56217.log.gz 2024-03-28T13:22:08.571 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42059.log 2024-03-28T13:22:08.571 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45862.log.gz 2024-03-28T13:22:08.572 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46492.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46492.log.gz 2024-03-28T13:22:08.572 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48446.log 2024-03-28T13:22:08.573 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67716.log 2024-03-28T13:22:08.574 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48446.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42059.log.gz 2024-03-28T13:22:08.575 INFO:teuthology.orchestra.run.smithi019.stderr:gzip 89.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.48446.log.gz --verbose 2024-03-28T13:22:08.575 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.0.65134.log 2024-03-28T13:22:08.576 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67716.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67716.log.gz 2024-03-28T13:22:08.576 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76690.log 2024-03-28T13:22:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.65134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71031.log 2024-03-28T13:22:08.578 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.65134.log.gz 2024-03-28T13:22:08.579 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58608.log 2024-03-28T13:22:08.579 INFO:teuthology.orchestra.run.smithi019.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.76690.log.gz 2024-03-28T13:22:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71031.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.71031.log.gz 2024-03-28T13:22:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43621.log 2024-03-28T13:22:08.581 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44135.log 2024-03-28T13:22:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43621.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58608.log.gz 2024-03-28T13:22:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66413.log 2024-03-28T13:22:08.583 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43621.log.gz 2024-03-28T13:22:08.584 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48614.log 2024-03-28T13:22:08.584 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44135.log.gz 2024-03-28T13:22:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61333.log 2024-03-28T13:22:08.585 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66413.log.gz 2024-03-28T13:22:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42455.log 2024-03-28T13:22:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61333.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48614.log.gz 2024-03-28T13:22:08.587 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61333.log.gz 2024-03-28T13:22:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71911.log 2024-03-28T13:22:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73929.log 2024-03-28T13:22:08.589 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42455.log.gz 2024-03-28T13:22:08.590 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56759.log 2024-03-28T13:22:08.590 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.71911.log.gz 2024-03-28T13:22:08.591 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43967.log 2024-03-28T13:22:08.592 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73929.log.gz 2024-03-28T13:22:08.592 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77355.log 2024-03-28T13:22:08.593 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56759.log.gz 2024-03-28T13:22:08.594 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63363.log 2024-03-28T13:22:08.594 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43967.log.gz 2024-03-28T13:22:08.595 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66268.log 2024-03-28T13:22:08.595 INFO:teuthology.orchestra.run.smithi019.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77355.log.gz 2024-03-28T13:22:08.596 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65233.log 2024-03-28T13:22:08.597 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66268.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63363.log.gz 2024-03-28T13:22:08.597 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45652.log 2024-03-28T13:22:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65233.log: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.66268.log.gz 2024-03-28T13:22:08.599 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63447.log 2024-03-28T13:22:08.599 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.65233.log.gz 2024-03-28T13:22:08.600 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78261.log 2024-03-28T13:22:08.600 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45652.log.gz 2024-03-28T13:22:08.601 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46645.log 2024-03-28T13:22:08.601 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63447.log.gz 2024-03-28T13:22:08.603 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78261.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78261.log.gz 2024-03-28T13:22:08.603 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76155.log 2024-03-28T13:22:08.604 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79856.log 2024-03-28T13:22:08.604 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.76155.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46645.log.gz 2024-03-28T13:22:08.604 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76155.log.gz 2024-03-28T13:22:08.605 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77096.log 2024-03-28T13:22:08.606 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79856.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79856.log.gz 2024-03-28T13:22:08.606 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53873.log 2024-03-28T13:22:08.608 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79730.log 2024-03-28T13:22:08.608 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53873.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.77096.log.gz 2024-03-28T13:22:08.609 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58316.log 2024-03-28T13:22:08.609 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53873.log.gz 2024-03-28T13:22:08.610 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79730.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79730.log.gz 2024-03-28T13:22:08.611 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64754.log 2024-03-28T13:22:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58316.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58316.log.gz 2024-03-28T13:22:08.612 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55300.log 2024-03-28T13:22:08.613 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64754.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64754.log.gz 2024-03-28T13:22:08.613 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45778.log 2024-03-28T13:22:08.614 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46952.log 2024-03-28T13:22:08.615 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45778.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55300.log.gz 2024-03-28T13:22:08.615 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 88.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.45778.log.gz /var/log/ceph/ceph-client.admin.74097.log 2024-03-28T13:22:08.616 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-03-28T13:22:08.617 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46952.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46952.log.gz 2024-03-28T13:22:08.617 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42144.log 2024-03-28T13:22:08.618 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50258.log 2024-03-28T13:22:08.619 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42144.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.74097.log.gz 2024-03-28T13:22:08.619 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62597.log 2024-03-28T13:22:08.619 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42144.log.gz 2024-03-28T13:22:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50258.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50258.log.gz 2024-03-28T13:22:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45694.log 2024-03-28T13:22:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79646.log 2024-03-28T13:22:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45694.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62597.log.gz 2024-03-28T13:22:08.623 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45694.log.gz 2024-03-28T13:22:08.623 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65299.log 2024-03-28T13:22:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79646.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79646.log.gz 2024-03-28T13:22:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43883.log 2024-03-28T13:22:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56499.log 2024-03-28T13:22:08.626 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43883.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.65299.log.gz 2024-03-28T13:22:08.627 INFO:teuthology.orchestra.run.smithi019.stderr:gzip 88.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.43883.log.gz -- 2024-03-28T13:22:08.627 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.47971.log 2024-03-28T13:22:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48530.log 2024-03-28T13:22:08.629 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.56499.log.gz 2024-03-28T13:22:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42791.log 2024-03-28T13:22:08.629 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47971.log.gz 2024-03-28T13:22:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57933.log 2024-03-28T13:22:08.631 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48530.log.gz 2024-03-28T13:22:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42791.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-28T13:22:08.632 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42791.log.gz 2024-03-28T13:22:08.633 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73845.log 2024-03-28T13:22:08.633 INFO:teuthology.orchestra.run.smithi019.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57933.log.gz 2024-03-28T13:22:08.634 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73887.log 2024-03-28T13:22:08.635 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43601.log 2024-03-28T13:22:08.636 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73845.log.gz 2024-03-28T13:22:08.637 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50475.log 2024-03-28T13:22:08.637 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73887.log.gz 2024-03-28T13:22:08.638 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66125.log 2024-03-28T13:22:08.639 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43601.log.gz 2024-03-28T13:22:08.639 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44261.log 2024-03-28T13:22:08.639 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50475.log.gz 2024-03-28T13:22:08.641 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67100.log 2024-03-28T13:22:08.641 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66125.log.gz 2024-03-28T13:22:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55174.log 2024-03-28T13:22:08.642 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44261.log.gz 2024-03-28T13:22:08.643 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49164.log 2024-03-28T13:22:08.644 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.67100.log.gz 2024-03-28T13:22:08.645 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79520.log 2024-03-28T13:22:08.645 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% 94.4% -- replaced with /var/log/ceph/ceph-client.admin.55174.log.gz 2024-03-28T13:22:08.645 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-28T13:22:08.645 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50336.log 2024-03-28T13:22:08.646 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49164.log.gz 2024-03-28T13:22:08.646 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79520.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79520.log.gz 2024-03-28T13:22:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57765.log 2024-03-28T13:22:08.648 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50910.log 2024-03-28T13:22:08.648 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57765.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50336.log.gz 2024-03-28T13:22:08.649 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69539.log 2024-03-28T13:22:08.649 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.57765.log.gz 2024-03-28T13:22:08.650 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45820.log 2024-03-28T13:22:08.651 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50910.log.gz 2024-03-28T13:22:08.652 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69539.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69539.log.gz 2024-03-28T13:22:08.652 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57410.log 2024-03-28T13:22:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53020.log 2024-03-28T13:22:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57410.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45820.log.gz 2024-03-28T13:22:08.654 INFO:teuthology.orchestra.run.smithi019.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.57410.log.gz 2024-03-28T13:22:08.654 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48698.log 2024-03-28T13:22:08.656 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62200.log 2024-03-28T13:22:08.656 INFO:teuthology.orchestra.run.smithi019.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53020.log.gz 2024-03-28T13:22:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48698.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48698.log.gz 2024-03-28T13:22:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51138.log 2024-03-28T13:22:08.658 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42971.log 2024-03-28T13:22:08.658 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62200.log.gz 2024-03-28T13:22:08.659 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50994.log 2024-03-28T13:22:08.659 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51138.log.gz 2024-03-28T13:22:08.660 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42971.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42971.log.gz 2024-03-28T13:22:08.660 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68790.log 2024-03-28T13:22:08.661 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42969.log 2024-03-28T13:22:08.662 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68790.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50994.log.gz 2024-03-28T13:22:08.662 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68790.log.gz 2024-03-28T13:22:08.663 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69455.log 2024-03-28T13:22:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62032.log 2024-03-28T13:22:08.665 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42969.log.gz 2024-03-28T13:22:08.665 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69455.log: gzip -5 88.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.69455.log.gz -- 2024-03-28T13:22:08.666 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.64608.log 2024-03-28T13:22:08.666 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47305.log 2024-03-28T13:22:08.667 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64608.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62032.log.gz 2024-03-28T13:22:08.668 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56651.log 2024-03-28T13:22:08.668 INFO:teuthology.orchestra.run.smithi019.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.64608.log.gz 2024-03-28T13:22:08.669 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47305.log: gzip -5 87.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.47305.log.gz -- 2024-03-28T13:22:08.669 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.43282.log 2024-03-28T13:22:08.670 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60858.log 2024-03-28T13:22:08.670 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.56651.log.gz 2024-03-28T13:22:08.671 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43282.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43282.log.gz 2024-03-28T13:22:08.671 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79268.log 2024-03-28T13:22:08.672 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60858.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60858.log.gz 2024-03-28T13:22:08.672 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76443.log 2024-03-28T13:22:08.673 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79268.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-28T13:22:08.674 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79268.log.gz 2024-03-28T13:22:08.675 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73080.log 2024-03-28T13:22:08.675 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76443.log.gz 2024-03-28T13:22:08.676 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70905.log 2024-03-28T13:22:08.677 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54821.log 2024-03-28T13:22:08.677 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73080.log.gz 2024-03-28T13:22:08.678 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47635.log 2024-03-28T13:22:08.678 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70905.log.gz 2024-03-28T13:22:08.679 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53747.log 2024-03-28T13:22:08.680 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54821.log.gz 2024-03-28T13:22:08.681 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57326.log 2024-03-28T13:22:08.681 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47635.log.gz 2024-03-28T13:22:08.682 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47551.log 2024-03-28T13:22:08.682 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53747.log.gz 2024-03-28T13:22:08.683 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69689.log 2024-03-28T13:22:08.683 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57326.log.gz 2024-03-28T13:22:08.685 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71737.log 2024-03-28T13:22:08.685 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47551.log.gz 2024-03-28T13:22:08.686 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57158.log 2024-03-28T13:22:08.686 INFO:teuthology.orchestra.run.smithi019.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.69689.log.gz 2024-03-28T13:22:08.687 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71483.log 2024-03-28T13:22:08.687 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.71737.log.gz 2024-03-28T13:22:08.689 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55342.log 2024-03-28T13:22:08.689 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57158.log.gz 2024-03-28T13:22:08.690 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58558.log 2024-03-28T13:22:08.690 INFO:teuthology.orchestra.run.smithi019.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.71483.log.gz 2024-03-28T13:22:08.691 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67166.log 2024-03-28T13:22:08.691 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55342.log.gz 2024-03-28T13:22:08.692 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63405.log 2024-03-28T13:22:08.693 INFO:teuthology.orchestra.run.smithi019.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58558.log.gz 2024-03-28T13:22:08.694 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67166.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.67166.log.gz 2024-03-28T13:22:08.694 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58657.log 2024-03-28T13:22:08.695 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63405.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63405.log.gz 2024-03-28T13:22:08.696 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78083.log 2024-03-28T13:22:08.697 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49772.log 2024-03-28T13:22:08.697 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58657.log.gz 2024-03-28T13:22:08.698 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58446.log 2024-03-28T13:22:08.698 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78083.log.gz 2024-03-28T13:22:08.699 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.49772.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.49772.log.gz 2024-03-28T13:22:08.699 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54480.log 2024-03-28T13:22:08.701 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42965.log 2024-03-28T13:22:08.701 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58446.log.gz 2024-03-28T13:22:08.702 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37309.log 2024-03-28T13:22:08.702 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54480.log.gz 2024-03-28T13:22:08.703 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60234.log 2024-03-28T13:22:08.703 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42965.log.gz 2024-03-28T13:22:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49308.log 2024-03-28T13:22:08.705 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37309.log.gz 2024-03-28T13:22:08.705 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.60234.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60234.log.gz 2024-03-28T13:22:08.706 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48200.log 2024-03-28T13:22:08.707 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66508.log 2024-03-28T13:22:08.707 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49308.log.gz 2024-03-28T13:22:08.708 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68155.log 2024-03-28T13:22:08.708 INFO:teuthology.orchestra.run.smithi019.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48200.log.gz 2024-03-28T13:22:08.710 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79310.log 2024-03-28T13:22:08.710 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66508.log.gz 2024-03-28T13:22:08.711 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77054.log 2024-03-28T13:22:08.711 INFO:teuthology.orchestra.run.smithi019.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68155.log.gz 2024-03-28T13:22:08.712 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47761.log 2024-03-28T13:22:08.712 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79310.log.gz 2024-03-28T13:22:08.713 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54083.log 2024-03-28T13:22:08.714 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55576.log 2024-03-28T13:22:08.715 INFO:teuthology.orchestra.run.smithi019.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.77054.log.gz 2024-03-28T13:22:08.715 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47761.log.gz 2024-03-28T13:22:08.716 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54083.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54083.log.gz 2024-03-28T13:22:08.716 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43628.log 2024-03-28T13:22:08.717 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55576.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.55576.log.gz --verbose 2024-03-28T13:22:08.717 INFO:teuthology.orchestra.run.smithi019.stderr: -- /var/log/ceph/ceph-client.admin.46450.log 2024-03-28T13:22:08.718 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69269.log 2024-03-28T13:22:08.718 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43628.log.gz 2024-03-28T13:22:08.719 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46450.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-28T13:22:08.720 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46450.log.gz 2024-03-28T13:22:08.721 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64645.log 2024-03-28T13:22:08.721 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69269.log.gz 2024-03-28T13:22:08.722 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63191.log 2024-03-28T13:22:08.724 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42186.log 2024-03-28T13:22:08.724 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64645.log.gz 2024-03-28T13:22:08.725 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63191.log: gzip -5 --verbose 87.8% -- /var/log/ceph/ceph-client.admin.52336.log -- replaced with /var/log/ceph/ceph-client.admin.63191.log.gz 2024-03-28T13:22:08.725 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-03-28T13:22:08.725 INFO:teuthology.orchestra.run.smithi019.stderr: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-28T13:22:08.726 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76401.log 2024-03-28T13:22:08.726 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42186.log.gz 2024-03-28T13:22:08.727 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59976.log 2024-03-28T13:22:08.728 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52336.log.gz 2024-03-28T13:22:08.728 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65600.log 2024-03-28T13:22:08.729 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76401.log.gz 2024-03-28T13:22:08.730 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48740.log 2024-03-28T13:22:08.731 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59976.log.gz 2024-03-28T13:22:08.731 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61854.log 2024-03-28T13:22:08.732 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.65600.log.gz 2024-03-28T13:22:08.733 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48740.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-28T13:22:08.733 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48740.log.gz 2024-03-28T13:22:08.734 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73038.log 2024-03-28T13:22:08.734 INFO:teuthology.orchestra.run.smithi019.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61854.log.gz 2024-03-28T13:22:08.735 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60966.log 2024-03-28T13:22:08.736 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56325.log 2024-03-28T13:22:08.737 INFO:teuthology.orchestra.run.smithi019.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73038.log.gz 2024-03-28T13:22:08.738 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76727.log 2024-03-28T13:22:08.738 INFO:teuthology.orchestra.run.smithi019.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60966.log.gz 2024-03-28T13:22:08.740 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56325.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56325.log.gz 2024-03-28T13:22:08.740 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62284.log 2024-03-28T13:22:08.740 INFO:teuthology.orchestra.run.smithi019.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.76727.log.gz 2024-03-28T13:22:08.741 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61008.log 2024-03-28T13:22:08.742 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61896.log 2024-03-28T13:22:08.743 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62284.log.gz 2024-03-28T13:22:08.744 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57639.log 2024-03-28T13:22:08.744 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61896.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.61008.log.gz 2024-03-28T13:22:08.745 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-03-28T13:22:08.745 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61896.log.gz 2024-03-28T13:22:08.747 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57639.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57639.log.gz 2024-03-28T13:22:08.747 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43841.log 2024-03-28T13:22:08.749 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69881.log 2024-03-28T13:22:08.750 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43841.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43841.log.gz 2024-03-28T13:22:08.751 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46997.log 2024-03-28T13:22:08.751 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.69881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.69881.log.gz 2024-03-28T13:22:08.752 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72743.log 2024-03-28T13:22:08.754 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46997.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46997.log.gz 2024-03-28T13:22:08.754 INFO:teuthology.orchestra.run.smithi188.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-28T13:22:08.755 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74055.log 2024-03-28T13:22:08.755 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72743.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.72743.log.gz 2024-03-28T13:22:08.757 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74055.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74055.log.gz 2024-03-28T13:22:08.758 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58232.log 2024-03-28T13:22:08.759 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52252.log 2024-03-28T13:22:08.760 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58232.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.58232.log.gz 2024-03-28T13:22:08.761 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72827.log 2024-03-28T13:22:08.762 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52252.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52252.log.gz 2024-03-28T13:22:08.763 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.72827.log: -- /var/log/ceph/ceph-client.admin.60042.log 2024-03-28T13:22:08.764 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72827.log.gz 2024-03-28T13:22:08.765 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48280.log 2024-03-28T13:22:08.766 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60042.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.60042.log.gz 2024-03-28T13:22:08.767 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.48280.log: /var/log/ceph/ceph-client.admin.67800.log 2024-03-28T13:22:08.768 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48280.log.gz 2024-03-28T13:22:08.769 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76527.log 2024-03-28T13:22:08.770 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67800.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67800.log.gz 2024-03-28T13:22:08.771 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54522.log 2024-03-28T13:22:08.771 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76527.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.76527.log.gz 2024-03-28T13:22:08.772 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59695.log 2024-03-28T13:22:08.774 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54522.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54522.log.gz 2024-03-28T13:22:08.774 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62766.log 2024-03-28T13:22:08.775 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59695.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59695.log.gz 2024-03-28T13:22:08.777 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62766.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62766.log.gz 2024-03-28T13:22:08.778 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60707.log 2024-03-28T13:22:08.778 INFO:teuthology.orchestra.run.smithi188.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-28T13:22:08.779 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42707.log 2024-03-28T13:22:08.781 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60707.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.60707.log.gz 2024-03-28T13:22:08.782 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67632.log 2024-03-28T13:22:08.782 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42707.log.gz 2024-03-28T13:22:08.783 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43574.log 2024-03-28T13:22:08.785 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67632.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67632.log.gz 2024-03-28T13:22:08.786 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43574.log: /var/log/ceph/ceph-client.admin.74643.log 2024-03-28T13:22:08.786 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43574.log.gz 2024-03-28T13:22:08.787 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47887.log 2024-03-28T13:22:08.789 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.74643.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74643.log.gz 2024-03-28T13:22:08.790 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5/var/log/ceph/ceph-client.admin.47887.log: --verbose -- /var/log/ceph/ceph-client.admin.70355.log 2024-03-28T13:22:08.790 INFO:teuthology.orchestra.run.smithi019.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47887.log.gz 2024-03-28T13:22:08.791 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51943.log 2024-03-28T13:22:08.793 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70355.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.70355.log.gz 2024-03-28T13:22:08.794 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59471.log 2024-03-28T13:22:08.794 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51943.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51943.log.gz 2024-03-28T13:22:08.795 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65450.log 2024-03-28T13:22:08.796 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59471.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.59471.log.gz 2024-03-28T13:22:08.797 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74787.log 2024-03-28T13:22:08.798 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65450.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65450.log.gz 2024-03-28T13:22:08.799 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.74787.log: /var/log/ceph/ceph-client.admin.48115.log 2024-03-28T13:22:08.800 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74787.log.gz 2024-03-28T13:22:08.801 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42316.log 2024-03-28T13:22:08.802 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48115.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48115.log.gz 2024-03-28T13:22:08.804 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42316.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42316.log.gz 2024-03-28T13:22:08.804 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75384.log 2024-03-28T13:22:08.805 INFO:teuthology.orchestra.run.smithi188.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-28T13:22:08.806 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78125.log 2024-03-28T13:22:08.807 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75384.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75384.log.gz 2024-03-28T13:22:08.808 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78125.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78125.log.gz 2024-03-28T13:22:08.809 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56067.log 2024-03-28T13:22:08.810 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62326.log 2024-03-28T13:22:08.813 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56067.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.56067.log.gz 2024-03-28T13:22:08.813 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72083.log 2024-03-28T13:22:08.813 INFO:teuthology.orchestra.run.smithi019.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62326.log.gz 2024-03-28T13:22:08.815 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44975.log 2024-03-28T13:22:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.72083.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.72083.log.gz 2024-03-28T13:22:08.817 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.0.44975.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54697.log 2024-03-28T13:22:08.817 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.0.44975.log.gz 2024-03-28T13:22:08.818 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77957.log 2024-03-28T13:22:08.820 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54697.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54697.log.gz 2024-03-28T13:22:08.821 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71394.log 2024-03-28T13:22:08.821 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77957.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77957.log.gz 2024-03-28T13:22:08.822 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62452.log 2024-03-28T13:22:08.824 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.71394.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.71394.log.gz 2024-03-28T13:22:08.825 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57022.log 2024-03-28T13:22:08.825 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.62452.log.gz 2024-03-28T13:22:08.826 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43306.log 2024-03-28T13:22:08.828 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57022.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57022.log.gz 2024-03-28T13:22:08.828 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48158.log 2024-03-28T13:22:08.829 INFO:teuthology.orchestra.run.smithi188.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-28T13:22:08.830 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43306.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43306.log.gz 2024-03-28T13:22:08.830 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63699.log 2024-03-28T13:22:08.831 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48158.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48158.log.gz 2024-03-28T13:22:08.832 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77267.log 2024-03-28T13:22:08.833 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63699.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63699.log.gz 2024-03-28T13:22:08.834 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60395.log 2024-03-28T13:22:08.835 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-03-28T13:22:08.835 INFO:teuthology.orchestra.run.smithi188.stderr:real 0m0.837s 2024-03-28T13:22:08.835 INFO:teuthology.orchestra.run.smithi188.stderr:user 0m3.834s 2024-03-28T13:22:08.835 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m0.400s 2024-03-28T13:22:08.836 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77267.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77267.log.gz 2024-03-28T13:22:08.836 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79142.log 2024-03-28T13:22:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60395.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60395.log.gz 2024-03-28T13:22:08.838 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73217.log 2024-03-28T13:22:08.839 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79142.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79142.log.gz 2024-03-28T13:22:08.840 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73217.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73217.log.gz 2024-03-28T13:22:08.841 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65492.log 2024-03-28T13:22:08.842 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70192.log 2024-03-28T13:22:08.844 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65492.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.65492.log.gz 2024-03-28T13:22:08.845 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.70192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48403.log 2024-03-28T13:22:08.845 INFO:teuthology.orchestra.run.smithi019.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70192.log.gz 2024-03-28T13:22:08.847 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48403.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48403.log.gz 2024-03-28T13:22:08.848 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44429.log 2024-03-28T13:22:08.849 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49518.log 2024-03-28T13:22:08.851 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44429.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44429.log.gz 2024-03-28T13:22:08.851 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66554.log 2024-03-28T13:22:08.853 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49518.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.49518.log.gz 2024-03-28T13:22:08.854 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.66554.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.76923.log 2024-03-28T13:22:08.854 INFO:teuthology.orchestra.run.smithi019.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66554.log.gz 2024-03-28T13:22:08.855 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52945.log 2024-03-28T13:22:08.856 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.76923.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.76923.log.gz 2024-03-28T13:22:08.857 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45904.log 2024-03-28T13:22:08.858 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52945.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52945.log.gz 2024-03-28T13:22:08.859 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67538.log 2024-03-28T13:22:08.860 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45904.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45904.log.gz 2024-03-28T13:22:08.861 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57807.log 2024-03-28T13:22:08.862 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67538.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67538.log.gz 2024-03-28T13:22:08.863 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77463.log 2024-03-28T13:22:08.864 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57807.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57807.log.gz 2024-03-28T13:22:08.865 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66466.log 2024-03-28T13:22:08.866 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.77463.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77463.log.gz 2024-03-28T13:22:08.867 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42665.log 2024-03-28T13:22:08.868 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66466.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66466.log.gz 2024-03-28T13:22:08.869 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46813.log 2024-03-28T13:22:08.870 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42665.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42665.log.gz 2024-03-28T13:22:08.871 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46813.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46813.log.gz 2024-03-28T13:22:08.872 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68197.log 2024-03-28T13:22:08.873 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62368.log 2024-03-28T13:22:08.875 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68197.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.68197.log.gz 2024-03-28T13:22:08.875 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78387.log 2024-03-28T13:22:08.876 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62368.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62368.log.gz 2024-03-28T13:22:08.877 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68624.log 2024-03-28T13:22:08.878 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.78387.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78387.log.gz 2024-03-28T13:22:08.879 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.68624.log: /var/log/ceph/ceph-client.admin.62945.log 2024-03-28T13:22:08.880 INFO:teuthology.orchestra.run.smithi019.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.68624.log.gz 2024-03-28T13:22:08.881 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71912.log 2024-03-28T13:22:08.883 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62945.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62945.log.gz 2024-03-28T13:22:08.883 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52168.log/var/log/ceph/ceph-client.admin.71912.log: 2024-03-28T13:22:08.884 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.71912.log.gz 2024-03-28T13:22:08.885 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76857.log 2024-03-28T13:22:08.886 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52168.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52168.log.gz 2024-03-28T13:22:08.887 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.76857.log: /var/log/ceph/ceph-client.admin.55426.log 2024-03-28T13:22:08.888 INFO:teuthology.orchestra.run.smithi019.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.76857.log.gz 2024-03-28T13:22:08.889 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.55426.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56585.log 2024-03-28T13:22:08.891 INFO:teuthology.orchestra.run.smithi019.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55426.log.gz 2024-03-28T13:22:08.891 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69055.log 2024-03-28T13:22:08.892 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56585.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.56585.log.gz 2024-03-28T13:22:08.894 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.69055.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.69055.log.gz 2024-03-28T13:22:08.894 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50952.log 2024-03-28T13:22:08.896 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45736.log 2024-03-28T13:22:08.897 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50952.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50952.log.gz 2024-03-28T13:22:08.898 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42581.log 2024-03-28T13:22:08.898 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45736.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45736.log.gz 2024-03-28T13:22:08.899 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67016.log 2024-03-28T13:22:08.900 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42581.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42581.log.gz 2024-03-28T13:22:08.901 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.67016.log: /var/log/ceph/ceph-client.admin.68449.log 2024-03-28T13:22:08.902 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.67016.log.gz 2024-03-28T13:22:08.903 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63149.log 2024-03-28T13:22:08.904 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68449.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68449.log.gz 2024-03-28T13:22:08.906 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63149.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63149.log.gz 2024-03-28T13:22:08.907 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50174.log 2024-03-28T13:22:08.908 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79478.log 2024-03-28T13:22:08.909 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50174.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50174.log.gz 2024-03-28T13:22:08.910 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74703.log 2024-03-28T13:22:08.911 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.79478.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79478.log.gz 2024-03-28T13:22:08.912 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.74703.log: /var/log/ceph/ceph-client.admin.68407.log 2024-03-28T13:22:08.913 INFO:teuthology.orchestra.run.smithi019.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74703.log.gz 2024-03-28T13:22:08.914 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56283.log 2024-03-28T13:22:08.915 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68407.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68407.log.gz 2024-03-28T13:22:08.916 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5/var/log/ceph/ceph-client.admin.56283.log: --verbose -- /var/log/ceph/ceph-client.admin.68832.log 2024-03-28T13:22:08.916 INFO:teuthology.orchestra.run.smithi019.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56283.log.gz 2024-03-28T13:22:08.917 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50378.log 2024-03-28T13:22:08.919 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.68832.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68832.log.gz 2024-03-28T13:22:08.920 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67370.log/var/log/ceph/ceph-client.admin.50378.log: 2024-03-28T13:22:08.920 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50378.log.gz 2024-03-28T13:22:08.921 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-28T13:22:08.922 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67370.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67370.log.gz 2024-03-28T13:22:08.923 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45435.log 2024-03-28T13:22:08.926 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.2.45435.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67208.log 2024-03-28T13:22:08.927 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.2.45435.log.gz 2024-03-28T13:22:08.928 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49434.log 2024-03-28T13:22:08.930 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63783.log 2024-03-28T13:22:08.931 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49434.log: /var/log/ceph/ceph-client.admin.67208.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49434.log.gz 2024-03-28T13:22:08.932 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.67208.log.gz 2024-03-28T13:22:08.934 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63783.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63783.log.gz 2024-03-28T13:22:08.934 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75963.log 2024-03-28T13:22:08.937 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.75963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47046.log 2024-03-28T13:22:08.938 INFO:teuthology.orchestra.run.smithi019.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75963.log.gz 2024-03-28T13:22:08.940 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52210.log 2024-03-28T13:22:08.943 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47046.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41741.log -- replaced with /var/log/ceph/ceph-client.admin.47046.log.gz 2024-03-28T13:22:08.943 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-03-28T13:22:08.943 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52210.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52210.log.gz 2024-03-28T13:22:08.946 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73420.log 2024-03-28T13:22:08.947 INFO:teuthology.orchestra.run.smithi019.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41741.log.gz 2024-03-28T13:22:08.949 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73297.log 2024-03-28T13:22:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.73420.log: /var/log/ceph/ceph-client.admin.73297.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73420.log.gz 2024-03-28T13:22:08.954 INFO:teuthology.orchestra.run.smithi019.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73297.log.gz 2024-03-28T13:22:08.962 INFO:teuthology.orchestra.run.smithi019.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-03-28T13:22:08.962 INFO:teuthology.orchestra.run.smithi019.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-28T13:22:09.067 INFO:teuthology.orchestra.run.smithi019.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-28T13:22:09.218 INFO:teuthology.orchestra.run.smithi019.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-28T13:22:09.312 INFO:teuthology.orchestra.run.smithi019.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-28T13:22:09.334 INFO:teuthology.orchestra.run.smithi019.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-28T13:22:09.435 INFO:teuthology.orchestra.run.smithi019.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-28T13:22:09.439 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-03-28T13:22:09.439 INFO:teuthology.orchestra.run.smithi019.stderr:real 0m1.441s 2024-03-28T13:22:09.439 INFO:teuthology.orchestra.run.smithi019.stderr:user 0m4.110s 2024-03-28T13:22:09.439 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m2.171s 2024-03-28T13:22:09.440 INFO:tasks.ceph:Archiving logs... 2024-03-28T13:22:09.441 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849/remote/smithi019/log 2024-03-28T13:22:09.441 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-28T13:22:10.669 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849/remote/smithi188/log 2024-03-28T13:22:10.670 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-28T13:22:11.302 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-28T13:22:11.312 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-28T13:22:11.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-28T13:22:11.320 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-28T13:22:11.475 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-28T13:22:11.484 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-28T13:22:11.484 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T13:22:11.492 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-28T13:22:11.541 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-03-28T13:22:11.545 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-03-28T13:22:11.552 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T13:22:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-03-28T13:22:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 61 +780us[ +780us] +/- 80ms 2024-03-28T13:22:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 62 -1176us[-1179us] +/- 80ms 2024-03-28T13:22:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv03.front.sepia.ceph.com 3 6 377 62 +289us[ +286us] +/- 78ms 2024-03-28T13:22:11.553 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T13:22:11.553 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-28T13:22:11.553 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-03-28T13:22:11.554 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 65 +799us[ +816us] +/- 80ms 2024-03-28T13:22:11.554 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 64 -1151us[-1131us] +/- 80ms 2024-03-28T13:22:11.554 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv03.front.sepia.ceph.com 3 6 377 0 +291us[ +313us] +/- 79ms 2024-03-28T13:22:11.554 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-03-28T13:22:11.555 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-28T13:22:11.565 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-28T13:22:11.566 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-28T13:22:11.575 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-28T13:22:11.837 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-28T13:22:12.053 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T13:22:12.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T13:22:12.164 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-28T13:22:12.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T13:22:12.264 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-28T13:22:12.275 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-28T13:22:12.305 INFO:teuthology.task.internal:Duration was 1344.960656 seconds 2024-03-28T13:22:12.305 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-28T13:22:12.316 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-28T13:22:12.316 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-28T13:22:12.324 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-28T13:22:12.490 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T13:22:12.492 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-28T13:22:12.800 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-28T13:22:12.800 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-03-28T13:22:12.800 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-28T13:22:12.867 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2024-03-28T13:22:12.867 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-28T13:22:12.944 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-28T13:22:12.944 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-28T13:22:12.951 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-28T13:22:13.046 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-28T13:22:13.046 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-28T13:22:13.051 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-28T13:22:13.268 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-28T13:22:13.278 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-28T13:22:13.279 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-28T13:22:13.377 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-28T13:22:13.476 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-28T13:22:13.486 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-28T13:22:13.494 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-28T13:22:13.602 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2024-03-28T13:22:13.603 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-03-28T13:22:13.659 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-28T13:22:13.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T13:22:13.709 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-28T13:22:13.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-28T13:22:13.764 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-28T13:22:13.774 INFO:teuthology.task.internal:Transferring archived files... 2024-03-28T13:22:13.775 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849/remote/smithi019 2024-03-28T13:22:13.775 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-28T13:22:13.890 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849/remote/smithi188 2024-03-28T13:22:13.891 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-28T13:22:13.998 INFO:teuthology.task.internal:Removing archive directory... 2024-03-28T13:22:13.998 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-28T13:22:14.006 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-28T13:22:14.061 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-28T13:22:14.073 INFO:teuthology.task.internal:Not uploading archives. 2024-03-28T13:22:14.073 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-28T13:22:14.083 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-28T13:22:14.084 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-28T13:22:14.091 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-28T13:22:14.143 INFO:teuthology.orchestra.run.smithi019.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 28 13:22 /home/ubuntu/cephtest 2024-03-28T13:22:14.145 INFO:teuthology.orchestra.run.smithi188.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 28 13:22 /home/ubuntu/cephtest 2024-03-28T13:22:14.148 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-28T13:22:14.158 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-28T13:22:14.178 INFO:teuthology.nuke:Checking targets against current locks 2024-03-28T13:22:14.207 DEBUG:teuthology.nuke:shortname: smithi019 2024-03-28T13:22:14.207 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-28T13:22:14.232 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 12:50:39.558001', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLMxKcBcq8C462aleVvkrsMm3/aiMC3doyAFgs+3Pf1icWW+2G/YktWZrB+nwUbFxs1mIEpQArVQzJHRg+TjPYI='} 2024-03-28T13:22:14.236 DEBUG:teuthology.nuke:shortname: smithi188 2024-03-28T13:22:14.236 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-28T13:22:14.255 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-28_07:14:07-fs-wip-batrick-testing-20240327.230800-distro-default-smithi/7627849', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-28 12:50:39.559414', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCzV9qvdqhto5/WsJrJRzziGvwAHvJbbvVaHtiQwq7cC3HReg10YTKULIyQYtmcXB/EpldjucBTx3GNsotvFC1w='} 2024-03-28T13:22:14.285 INFO:teuthology.orchestra.console.smithi019:Power off 2024-03-28T13:22:14.285 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-28T13:22:14.309 DEBUG:teuthology.orchestra.console.smithi019:power off output: Chassis Power Control: Down/Off 2024-03-28T13:22:14.309 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T13:22:14.324 INFO:teuthology.orchestra.console.smithi188:Power off 2024-03-28T13:22:14.324 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-28T13:22:14.333 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-28T13:22:14.344 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2024-03-28T13:22:14.345 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T13:22:14.367 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-03-28T13:22:18.335 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T13:22:18.368 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T13:22:18.445 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is on 2024-03-28T13:22:18.478 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2024-03-28T13:22:22.449 DEBUG:teuthology.orchestra.console.smithi019:pexpect command: ipmitool -H smithi019.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T13:22:22.479 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-28T13:22:22.560 DEBUG:teuthology.orchestra.console.smithi019:check power output: Chassis Power is off 2024-03-28T13:22:22.587 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2024-03-28T13:22:22.662 INFO:teuthology.orchestra.console.smithi019:Power off completed 2024-03-28T13:22:22.688 INFO:teuthology.orchestra.console.smithi188:Power off completed 2024-03-28T13:22:22.794 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 client.1-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1344.9606556892395 failure_reason: '"2024-03-28T13:16:47.382269+0000 mds.b (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-03-28T13:22:22.794 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-28T13:22:22.873 INFO:teuthology.run:FAIL